2024-09-10T08:25:03.560 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T08:25:03.587 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T08:25:03.670 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898925 branch: wip-rishabh-testing-20240909.165402-squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} email: null first_in_suite: false job_id: '7898925' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240909.165402-squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240909.165402-squid sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9893 sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b sleep_before_teardown: 0 subset: 197/1024 suite: fs suite_branch: wip-rishabh-testing-20240909.165402-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi172.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCNfKwRQVTKr4rd2ulqHJu8Eg7En/g0sDGzVOrUq9T1DHqzXZG7V7+iE2V+VwGp5xTl6qOqtJhTR3tivF/KK95M= smithi174.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ8CeOC51SC+/dLLTNciCCMlo+Oyyicwki9AukulFGkK/pAkOIY5pwTd22TY03Uq9JZXLKk2FqgvcJg6zRAFk5Q= smithi191.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCWmwfobYto+x64zOczpaFMbjmyE7+xa6hCTW2qjcpzkaQj8crIqLRHsq4cidpNuY19vYfmMJhJiNjMeMbpFE8I= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_06:58:10 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T08:25:03.670 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa; will attempt to use it 2024-09-10T08:25:03.671 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks 2024-09-10T08:25:03.671 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T08:25:03.672 INFO:teuthology.task.internal:Checking packages... 2024-09-10T08:25:03.694 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1cf819b7212a6048e92fc78c4eff963344f6d56b' 2024-09-10T08:25:03.694 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T08:25:03.694 INFO:teuthology.packaging:ref: None 2024-09-10T08:25:03.694 INFO:teuthology.packaging:tag: None 2024-09-10T08:25:03.695 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240909.165402-squid 2024-09-10T08:25:03.695 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:25:03.695 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240909.165402-squid 2024-09-10T08:25:03.833 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-305.g1cf819b7 2024-09-10T08:25:03.835 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T08:25:03.841 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T08:25:03.841 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T08:25:03.875 INFO:teuthology.task.internal:Saving configuration 2024-09-10T08:25:03.895 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T08:25:03.901 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T08:25:03.917 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898925', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 08:19:54.382517', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCNfKwRQVTKr4rd2ulqHJu8Eg7En/g0sDGzVOrUq9T1DHqzXZG7V7+iE2V+VwGp5xTl6qOqtJhTR3tivF/KK95M='} 2024-09-10T08:25:03.933 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898925', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 08:19:54.380598', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ8CeOC51SC+/dLLTNciCCMlo+Oyyicwki9AukulFGkK/pAkOIY5pwTd22TY03Uq9JZXLKk2FqgvcJg6zRAFk5Q='} 2024-09-10T08:25:03.945 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898925', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 08:19:54.384167', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCWmwfobYto+x64zOczpaFMbjmyE7+xa6hCTW2qjcpzkaQj8crIqLRHsq4cidpNuY19vYfmMJhJiNjMeMbpFE8I='} 2024-09-10T08:25:03.945 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T08:25:04.023 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-10T08:25:04.023 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-10T08:25:04.023 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-10T08:25:04.023 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T08:25:04.085 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f263ae009d0>, signals=[15]) 2024-09-10T08:25:04.085 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T08:25:04.091 INFO:teuthology.task.internal:Opening connections... 2024-09-10T08:25:04.091 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi172.front.sepia.ceph.com 2024-09-10T08:25:04.092 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:25:04.166 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi174.front.sepia.ceph.com 2024-09-10T08:25:04.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:25:04.243 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2024-09-10T08:25:04.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:25:04.319 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T08:25:04.330 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2024-09-10T08:25:04.348 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2024-09-10T08:25:04.348 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2024-09-10T08:25:04.407 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2024-09-10T08:25:04.407 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="9" 2024-09-10T08:25:04.407 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2024-09-10T08:25:04.407 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2024-09-10T08:25:04.407 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="9" 2024-09-10T08:25:04.407 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el9" 2024-09-10T08:25:04.407 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T08:25:04.407 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2024-09-10T08:25:04.407 INFO:teuthology.orchestra.run.smithi172.stdout:LOGO="fedora-logo-icon" 2024-09-10T08:25:04.407 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T08:25:04.407 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2024-09-10T08:25:04.408 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T08:25:04.408 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T08:25:04.408 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T08:25:04.408 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2024-09-10T08:25:04.422 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2024-09-10T08:25:04.442 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2024-09-10T08:25:04.442 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2024-09-10T08:25:04.499 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream" 2024-09-10T08:25:04.499 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9" 2024-09-10T08:25:04.499 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2024-09-10T08:25:04.500 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2024-09-10T08:25:04.500 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9" 2024-09-10T08:25:04.500 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9" 2024-09-10T08:25:04.500 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T08:25:04.500 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2024-09-10T08:25:04.500 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon" 2024-09-10T08:25:04.500 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T08:25:04.500 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2024-09-10T08:25:04.501 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T08:25:04.501 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T08:25:04.501 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T08:25:04.501 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2024-09-10T08:25:04.522 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2024-09-10T08:25:04.540 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2024-09-10T08:25:04.540 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-09-10T08:25:04.598 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream" 2024-09-10T08:25:04.599 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="9" 2024-09-10T08:25:04.599 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos" 2024-09-10T08:25:04.599 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora" 2024-09-10T08:25:04.599 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="9" 2024-09-10T08:25:04.599 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el9" 2024-09-10T08:25:04.599 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T08:25:04.599 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2024-09-10T08:25:04.599 INFO:teuthology.orchestra.run.smithi191.stdout:LOGO="fedora-logo-icon" 2024-09-10T08:25:04.599 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T08:25:04.599 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/" 2024-09-10T08:25:04.600 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T08:25:04.600 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T08:25:04.600 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T08:25:04.600 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-09-10T08:25:04.613 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T08:25:04.631 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T08:25:04.637 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T08:25:04.637 DEBUG:teuthology.orchestra.run.smithi172:> test '!' -e /home/ubuntu/cephtest 2024-09-10T08:25:04.640 DEBUG:teuthology.orchestra.run.smithi174:> test '!' -e /home/ubuntu/cephtest 2024-09-10T08:25:04.642 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2024-09-10T08:25:04.659 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T08:25:04.666 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T08:25:04.666 DEBUG:teuthology.orchestra.run.smithi172:> test -z $(ls -A /var/lib/ceph) 2024-09-10T08:25:04.697 DEBUG:teuthology.orchestra.run.smithi174:> test -z $(ls -A /var/lib/ceph) 2024-09-10T08:25:04.700 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2024-09-10T08:25:04.793 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T08:25:04.861 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T08:25:04.876 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T08:25:04.876 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-10T08:25:04.876 DEBUG:teuthology.orchestra.run.smithi172:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T08:25:04.877 DEBUG:teuthology.orchestra.run.smithi174:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T08:25:04.877 DEBUG:teuthology.orchestra.run.smithi191:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T08:25:04.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:25:04.894 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2024-09-10T08:25:04.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:25:04.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:25:04.896 DEBUG:teuthology.orchestra.run.smithi172:> uname -r 2024-09-10T08:25:04.896 DEBUG:teuthology.orchestra.run.smithi174:> uname -r 2024-09-10T08:25:04.952 INFO:teuthology.orchestra.run.smithi191.stdout:5.14.0-503.el9.x86_64 2024-09-10T08:25:04.953 INFO:teuthology.task.kernel:Running kernel on smithi191: 5.14.0-503.el9.x86_64 2024-09-10T08:25:04.953 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum install -y kernel 2024-09-10T08:25:04.954 INFO:teuthology.orchestra.run.smithi174.stdout:5.14.0-503.el9.x86_64 2024-09-10T08:25:04.954 INFO:teuthology.orchestra.run.smithi172.stdout:5.14.0-503.el9.x86_64 2024-09-10T08:25:04.955 INFO:teuthology.task.kernel:Running kernel on smithi174: 5.14.0-503.el9.x86_64 2024-09-10T08:25:04.955 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum install -y kernel 2024-09-10T08:25:04.955 INFO:teuthology.task.kernel:Running kernel on smithi172: 5.14.0-503.el9.x86_64 2024-09-10T08:25:04.955 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum install -y kernel 2024-09-10T08:25:07.269 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-10T08:25:07.516 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 9.6 MB/s | 8.2 MB 00:00 2024-09-10T08:25:07.573 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-10T08:25:10.387 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:01 2024-09-10T08:25:13.299 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - AppStream 4.4 MB/s | 20 MB 00:04 2024-09-10T08:25:15.616 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 2.9 MB/s | 20 MB 00:06 2024-09-10T08:25:15.687 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.5 MB 00:01 2024-09-10T08:25:17.196 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 19 kB 00:00 2024-09-10T08:25:17.700 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-10T08:25:17.959 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-10T08:25:19.926 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-10T08:25:20.706 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-10T08:25:21.874 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.5 MB 00:02 2024-09-10T08:25:23.433 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 19 kB 00:00 2024-09-10T08:25:23.439 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-09-10T08:25:24.221 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-10T08:25:25.529 INFO:teuthology.orchestra.run.smithi172.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T08:25:25.529 INFO:teuthology.orchestra.run.smithi172.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T08:25:25.623 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-10T08:25:25.628 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-10T08:25:25.628 INFO:teuthology.orchestra.run.smithi172.stdout: Package Architecture Version Repository Size 2024-09-10T08:25:25.628 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-10T08:25:25.628 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-10T08:25:25.629 INFO:teuthology.orchestra.run.smithi172.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T08:25:25.629 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-09-10T08:25:25.629 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T08:25:25.629 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T08:25:25.629 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T08:25:25.629 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:25:25.629 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-10T08:25:25.629 INFO:teuthology.orchestra.run.smithi172.stdout:================================================================================ 2024-09-10T08:25:25.629 INFO:teuthology.orchestra.run.smithi172.stdout:Install 4 Packages 2024-09-10T08:25:25.629 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:25:25.630 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 79 M 2024-09-10T08:25:25.630 INFO:teuthology.orchestra.run.smithi172.stdout:Installed size: 126 M 2024-09-10T08:25:25.630 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-10T08:25:26.162 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-10T08:25:26.165 INFO:teuthology.orchestra.run.smithi172.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 81 kB/s | 23 kB 00:00 2024-09-10T08:25:28.253 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T08:25:28.253 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T08:25:28.346 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-09-10T08:25:28.357 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-09-10T08:25:28.357 INFO:teuthology.orchestra.run.smithi191.stdout: Package Architecture Version Repository Size 2024-09-10T08:25:28.358 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-09-10T08:25:28.358 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-09-10T08:25:28.358 INFO:teuthology.orchestra.run.smithi191.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T08:25:28.358 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-09-10T08:25:28.358 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T08:25:28.358 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T08:25:28.358 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T08:25:28.358 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:25:28.358 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-09-10T08:25:28.358 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-09-10T08:25:28.358 INFO:teuthology.orchestra.run.smithi191.stdout:Install 4 Packages 2024-09-10T08:25:28.358 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:25:28.359 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 79 M 2024-09-10T08:25:28.359 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 126 M 2024-09-10T08:25:28.359 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-09-10T08:25:28.489 INFO:teuthology.orchestra.run.smithi191.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 269 kB/s | 23 kB 00:00 2024-09-10T08:25:28.873 INFO:teuthology.orchestra.run.smithi191.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-10T08:25:29.323 INFO:teuthology.orchestra.run.smithi191.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-09-10T08:25:29.685 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-09-10T08:25:29.707 INFO:teuthology.orchestra.run.smithi191.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 23 MB/s | 29 MB 00:01 2024-09-10T08:25:29.707 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-09-10T08:25:29.707 INFO:teuthology.orchestra.run.smithi191.stdout:Total 58 MB/s | 79 MB 00:01 2024-09-10T08:25:30.397 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-09-10T08:25:30.737 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-09-10T08:25:30.737 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-09-10T08:25:31.683 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-09-10T08:25:31.683 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-09-10T08:25:31.700 INFO:teuthology.orchestra.run.smithi174.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T08:25:31.700 INFO:teuthology.orchestra.run.smithi174.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T08:25:31.805 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-09-10T08:25:31.813 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-09-10T08:25:31.813 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2024-09-10T08:25:31.813 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-09-10T08:25:31.813 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-09-10T08:25:31.814 INFO:teuthology.orchestra.run.smithi174.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T08:25:31.814 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-09-10T08:25:31.814 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T08:25:31.814 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T08:25:31.814 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T08:25:31.814 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:25:31.814 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-09-10T08:25:31.814 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-09-10T08:25:31.814 INFO:teuthology.orchestra.run.smithi174.stdout:Install 4 Packages 2024-09-10T08:25:31.814 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:25:31.814 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 79 M 2024-09-10T08:25:31.814 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 126 M 2024-09-10T08:25:31.815 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-09-10T08:25:32.037 INFO:teuthology.orchestra.run.smithi174.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 183 kB/s | 23 kB 00:00 2024-09-10T08:25:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-09-10T08:25:34.896 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T08:25:34.989 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T08:25:35.910 INFO:teuthology.orchestra.run.smithi174.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 3.9 MB/s | 16 MB 00:03 2024-09-10T08:25:36.212 INFO:teuthology.orchestra.run.smithi174.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 8.0 MB/s | 35 MB 00:04 2024-09-10T08:25:37.062 INFO:teuthology.orchestra.run.smithi174.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 5.7 MB/s | 29 MB 00:05 2024-09-10T08:25:37.062 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-09-10T08:25:37.062 INFO:teuthology.orchestra.run.smithi174.stdout:Total 15 MB/s | 79 MB 00:05 2024-09-10T08:25:37.289 INFO:teuthology.orchestra.run.smithi172.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 2.6 MB/s | 29 MB 00:11 2024-09-10T08:25:37.743 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-09-10T08:25:37.776 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T08:25:37.919 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T08:25:38.095 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-09-10T08:25:38.095 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-09-10T08:25:39.053 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-09-10T08:25:39.054 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-09-10T08:25:41.164 INFO:teuthology.orchestra.run.smithi172.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 1.0 MB/s | 16 MB 00:15 2024-09-10T08:25:41.486 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-09-10T08:25:42.162 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T08:25:42.247 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T08:25:42.761 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T08:25:42.888 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:25:45.023 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T08:25:45.164 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T08:25:47.703 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:25:50.001 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T08:25:50.039 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:25:54.885 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:26:05.278 INFO:teuthology.orchestra.run.smithi172.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 898 kB/s | 35 MB 00:39 2024-09-10T08:26:05.278 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-10T08:26:05.278 INFO:teuthology.orchestra.run.smithi172.stdout:Total 2.0 MB/s | 79 MB 00:39 2024-09-10T08:26:05.960 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-10T08:26:06.317 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-10T08:26:06.317 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-10T08:26:07.275 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-10T08:26:07.276 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-10T08:26:09.721 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-10T08:26:10.381 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T08:26:10.495 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T08:26:13.299 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T08:26:13.424 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T08:26:18.274 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T08:26:18.391 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:26:23.243 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:26:25.496 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:26:25.536 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:26:25.904 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:26:25.904 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T08:26:25.904 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T08:26:25.905 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T08:26:26.762 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:26:26.762 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:26:26.762 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-09-10T08:26:26.762 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T08:26:26.763 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T08:26:26.763 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T08:26:26.763 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T08:26:26.763 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:26:26.763 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-09-10T08:26:27.093 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T08:26:27.195 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T08:26:27.196 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T08:26:27.196 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T08:26:27.196 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T08:26:27.196 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T08:26:27.197 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-10T08:26:27.197 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi191.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T08:26:27.197 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T08:26:27.256 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T08:26:27.256 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T08:26:27.256 DEBUG:teuthology.orchestra.run.smithi191:> sudo rpm -qi grub2-tools 2024-09-10T08:26:27.288 INFO:teuthology.orchestra.run.smithi191.stdout:Name : grub2-tools 2024-09-10T08:26:27.288 INFO:teuthology.orchestra.run.smithi191.stdout:Epoch : 1 2024-09-10T08:26:27.289 INFO:teuthology.orchestra.run.smithi191.stdout:Version : 2.06 2024-09-10T08:26:27.289 INFO:teuthology.orchestra.run.smithi191.stdout:Release : 82.el9 2024-09-10T08:26:27.289 INFO:teuthology.orchestra.run.smithi191.stdout:Architecture: x86_64 2024-09-10T08:26:27.289 INFO:teuthology.orchestra.run.smithi191.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T08:26:27.289 INFO:teuthology.orchestra.run.smithi191.stdout:Group : Unspecified 2024-09-10T08:26:27.289 INFO:teuthology.orchestra.run.smithi191.stdout:Size : 8274670 2024-09-10T08:26:27.290 INFO:teuthology.orchestra.run.smithi191.stdout:License : GPLv3+ 2024-09-10T08:26:27.290 INFO:teuthology.orchestra.run.smithi191.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T08:26:27.290 INFO:teuthology.orchestra.run.smithi191.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T08:26:27.290 INFO:teuthology.orchestra.run.smithi191.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T08:26:27.290 INFO:teuthology.orchestra.run.smithi191.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T08:26:27.290 INFO:teuthology.orchestra.run.smithi191.stdout:Packager : builder@centos.org 2024-09-10T08:26:27.290 INFO:teuthology.orchestra.run.smithi191.stdout:Vendor : CentOS 2024-09-10T08:26:27.290 INFO:teuthology.orchestra.run.smithi191.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T08:26:27.290 INFO:teuthology.orchestra.run.smithi191.stdout:Summary : Support tools for GRUB. 2024-09-10T08:26:27.290 INFO:teuthology.orchestra.run.smithi191.stdout:Description : 2024-09-10T08:26:27.290 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:26:27.291 INFO:teuthology.orchestra.run.smithi191.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T08:26:27.291 INFO:teuthology.orchestra.run.smithi191.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T08:26:27.291 INFO:teuthology.orchestra.run.smithi191.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T08:26:27.291 INFO:teuthology.orchestra.run.smithi191.stdout:hardware devices. 2024-09-10T08:26:27.291 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:26:27.291 INFO:teuthology.orchestra.run.smithi191.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T08:26:27.292 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T08:26:27.292 INFO:teuthology.task.kernel:Updating grub on smithi191 to boot 5.14.0-505.el9.x86_64 2024-09-10T08:26:27.293 DEBUG:teuthology.orchestra.run.smithi191:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T08:26:27.967 INFO:teuthology.orchestra.run.smithi191.stderr:Generating grub configuration file ... 2024-09-10T08:26:29.316 INFO:teuthology.orchestra.run.smithi191.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T08:26:29.344 INFO:teuthology.orchestra.run.smithi191.stderr:done 2024-09-10T08:26:29.346 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2024-09-10T08:26:29.361 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.gSiAtTauJK 2024-09-10T08:26:29.362 DEBUG:teuthology.orchestra.run.smithi191:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.gSiAtTauJK 2024-09-10T08:26:29.434 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.gSiAtTauJK 2024-09-10T08:26:29.582 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.gSiAtTauJK is 6KB 2024-09-10T08:26:29.631 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.gSiAtTauJK 2024-09-10T08:26:29.646 DEBUG:teuthology.orchestra.run.smithi191:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T08:26:29.711 INFO:teuthology.orchestra.run.smithi191.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T08:26:29.712 INFO:teuthology.orchestra.run.smithi191.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T08:26:29.712 INFO:teuthology.orchestra.run.smithi191.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T08:26:29.712 INFO:teuthology.orchestra.run.smithi191.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T08:26:29.713 DEBUG:teuthology.orchestra.run.smithi191:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T08:26:29.869 DEBUG:teuthology.orchestra.run.smithi191:> sudo shutdown -r now 2024-09-10T08:26:32.587 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:26:32.627 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:26:32.957 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:26:32.957 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T08:26:32.957 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T08:26:32.957 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T08:26:33.788 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:26:33.788 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:26:33.788 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-09-10T08:26:33.789 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T08:26:33.789 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T08:26:33.789 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T08:26:33.789 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T08:26:33.789 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:26:33.789 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-09-10T08:26:34.118 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T08:26:34.210 INFO:teuthology.orchestra.run.smithi174.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T08:26:34.210 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T08:26:34.210 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T08:26:34.210 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T08:26:34.210 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T08:26:34.210 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-10T08:26:34.211 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi174.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T08:26:34.211 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T08:26:34.270 INFO:teuthology.orchestra.run.smithi174.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T08:26:34.271 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T08:26:34.271 DEBUG:teuthology.orchestra.run.smithi174:> sudo rpm -qi grub2-tools 2024-09-10T08:26:34.302 INFO:teuthology.orchestra.run.smithi174.stdout:Name : grub2-tools 2024-09-10T08:26:34.303 INFO:teuthology.orchestra.run.smithi174.stdout:Epoch : 1 2024-09-10T08:26:34.303 INFO:teuthology.orchestra.run.smithi174.stdout:Version : 2.06 2024-09-10T08:26:34.303 INFO:teuthology.orchestra.run.smithi174.stdout:Release : 82.el9 2024-09-10T08:26:34.303 INFO:teuthology.orchestra.run.smithi174.stdout:Architecture: x86_64 2024-09-10T08:26:34.303 INFO:teuthology.orchestra.run.smithi174.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T08:26:34.303 INFO:teuthology.orchestra.run.smithi174.stdout:Group : Unspecified 2024-09-10T08:26:34.303 INFO:teuthology.orchestra.run.smithi174.stdout:Size : 8274670 2024-09-10T08:26:34.303 INFO:teuthology.orchestra.run.smithi174.stdout:License : GPLv3+ 2024-09-10T08:26:34.303 INFO:teuthology.orchestra.run.smithi174.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T08:26:34.303 INFO:teuthology.orchestra.run.smithi174.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T08:26:34.303 INFO:teuthology.orchestra.run.smithi174.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T08:26:34.304 INFO:teuthology.orchestra.run.smithi174.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T08:26:34.304 INFO:teuthology.orchestra.run.smithi174.stdout:Packager : builder@centos.org 2024-09-10T08:26:34.304 INFO:teuthology.orchestra.run.smithi174.stdout:Vendor : CentOS 2024-09-10T08:26:34.304 INFO:teuthology.orchestra.run.smithi174.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T08:26:34.304 INFO:teuthology.orchestra.run.smithi174.stdout:Summary : Support tools for GRUB. 2024-09-10T08:26:34.304 INFO:teuthology.orchestra.run.smithi174.stdout:Description : 2024-09-10T08:26:34.304 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:26:34.304 INFO:teuthology.orchestra.run.smithi174.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T08:26:34.304 INFO:teuthology.orchestra.run.smithi174.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T08:26:34.304 INFO:teuthology.orchestra.run.smithi174.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T08:26:34.304 INFO:teuthology.orchestra.run.smithi174.stdout:hardware devices. 2024-09-10T08:26:34.304 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:26:34.304 INFO:teuthology.orchestra.run.smithi174.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T08:26:34.305 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T08:26:34.305 INFO:teuthology.task.kernel:Updating grub on smithi174 to boot 5.14.0-505.el9.x86_64 2024-09-10T08:26:34.306 DEBUG:teuthology.orchestra.run.smithi174:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T08:26:34.923 INFO:teuthology.orchestra.run.smithi174.stderr:Generating grub configuration file ... 2024-09-10T08:26:36.151 INFO:teuthology.orchestra.run.smithi174.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T08:26:36.178 INFO:teuthology.orchestra.run.smithi174.stderr:done 2024-09-10T08:26:36.181 DEBUG:teuthology.orchestra.run.smithi174:> mktemp 2024-09-10T08:26:36.197 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.wKGyHeX09y 2024-09-10T08:26:36.197 DEBUG:teuthology.orchestra.run.smithi174:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wKGyHeX09y 2024-09-10T08:26:36.268 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.wKGyHeX09y 2024-09-10T08:26:36.410 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.wKGyHeX09y is 6KB 2024-09-10T08:26:36.457 DEBUG:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.wKGyHeX09y 2024-09-10T08:26:36.472 DEBUG:teuthology.orchestra.run.smithi174:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T08:26:36.535 INFO:teuthology.orchestra.run.smithi174.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T08:26:36.535 INFO:teuthology.orchestra.run.smithi174.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T08:26:36.535 INFO:teuthology.orchestra.run.smithi174.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T08:26:36.535 INFO:teuthology.orchestra.run.smithi174.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T08:26:36.536 DEBUG:teuthology.orchestra.run.smithi174:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T08:26:36.712 DEBUG:teuthology.orchestra.run.smithi174:> sudo shutdown -r now 2024-09-10T08:26:59.872 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-10T08:26:59.873 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-10T08:26:59.874 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:27:01.397 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:27:01.439 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:27:01.754 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:27:01.754 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T08:27:01.755 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T08:27:01.755 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T08:27:02.623 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T08:27:02.623 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:27:02.623 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-10T08:27:02.623 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T08:27:02.624 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T08:27:02.624 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T08:27:02.624 INFO:teuthology.orchestra.run.smithi172.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T08:27:02.624 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:27:02.624 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-10T08:27:02.969 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T08:27:03.048 INFO:teuthology.orchestra.run.smithi172.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T08:27:03.048 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T08:27:03.048 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T08:27:03.048 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T08:27:03.048 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T08:27:03.049 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-10T08:27:03.049 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi172.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T08:27:03.049 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T08:27:03.107 INFO:teuthology.orchestra.run.smithi172.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T08:27:03.108 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T08:27:03.108 DEBUG:teuthology.orchestra.run.smithi172:> sudo rpm -qi grub2-tools 2024-09-10T08:27:03.140 INFO:teuthology.orchestra.run.smithi172.stdout:Name : grub2-tools 2024-09-10T08:27:03.140 INFO:teuthology.orchestra.run.smithi172.stdout:Epoch : 1 2024-09-10T08:27:03.140 INFO:teuthology.orchestra.run.smithi172.stdout:Version : 2.06 2024-09-10T08:27:03.140 INFO:teuthology.orchestra.run.smithi172.stdout:Release : 82.el9 2024-09-10T08:27:03.140 INFO:teuthology.orchestra.run.smithi172.stdout:Architecture: x86_64 2024-09-10T08:27:03.140 INFO:teuthology.orchestra.run.smithi172.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T08:27:03.141 INFO:teuthology.orchestra.run.smithi172.stdout:Group : Unspecified 2024-09-10T08:27:03.141 INFO:teuthology.orchestra.run.smithi172.stdout:Size : 8274670 2024-09-10T08:27:03.141 INFO:teuthology.orchestra.run.smithi172.stdout:License : GPLv3+ 2024-09-10T08:27:03.141 INFO:teuthology.orchestra.run.smithi172.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T08:27:03.141 INFO:teuthology.orchestra.run.smithi172.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T08:27:03.141 INFO:teuthology.orchestra.run.smithi172.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T08:27:03.141 INFO:teuthology.orchestra.run.smithi172.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T08:27:03.141 INFO:teuthology.orchestra.run.smithi172.stdout:Packager : builder@centos.org 2024-09-10T08:27:03.141 INFO:teuthology.orchestra.run.smithi172.stdout:Vendor : CentOS 2024-09-10T08:27:03.141 INFO:teuthology.orchestra.run.smithi172.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T08:27:03.141 INFO:teuthology.orchestra.run.smithi172.stdout:Summary : Support tools for GRUB. 2024-09-10T08:27:03.141 INFO:teuthology.orchestra.run.smithi172.stdout:Description : 2024-09-10T08:27:03.141 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:27:03.142 INFO:teuthology.orchestra.run.smithi172.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T08:27:03.142 INFO:teuthology.orchestra.run.smithi172.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T08:27:03.142 INFO:teuthology.orchestra.run.smithi172.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T08:27:03.142 INFO:teuthology.orchestra.run.smithi172.stdout:hardware devices. 2024-09-10T08:27:03.142 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:27:03.142 INFO:teuthology.orchestra.run.smithi172.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T08:27:03.143 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T08:27:03.143 INFO:teuthology.task.kernel:Updating grub on smithi172 to boot 5.14.0-505.el9.x86_64 2024-09-10T08:27:03.143 DEBUG:teuthology.orchestra.run.smithi172:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T08:27:03.786 INFO:teuthology.orchestra.run.smithi172.stderr:Generating grub configuration file ... 2024-09-10T08:27:05.115 INFO:teuthology.orchestra.run.smithi172.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T08:27:05.151 INFO:teuthology.orchestra.run.smithi172.stderr:done 2024-09-10T08:27:05.153 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2024-09-10T08:27:05.168 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.QrhD3a598i 2024-09-10T08:27:05.169 DEBUG:teuthology.orchestra.run.smithi172:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.QrhD3a598i 2024-09-10T08:27:05.245 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0666 /tmp/tmp.QrhD3a598i 2024-09-10T08:27:05.383 DEBUG:teuthology.orchestra.remote:smithi172:/tmp/tmp.QrhD3a598i is 6KB 2024-09-10T08:27:05.431 DEBUG:teuthology.orchestra.run.smithi172:> rm -fr /tmp/tmp.QrhD3a598i 2024-09-10T08:27:05.446 DEBUG:teuthology.orchestra.run.smithi172:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T08:27:05.512 INFO:teuthology.orchestra.run.smithi172.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T08:27:05.512 INFO:teuthology.orchestra.run.smithi172.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T08:27:05.512 INFO:teuthology.orchestra.run.smithi172.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T08:27:05.512 INFO:teuthology.orchestra.run.smithi172.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T08:27:05.513 DEBUG:teuthology.orchestra.run.smithi172:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T08:27:05.665 DEBUG:teuthology.orchestra.run.smithi172:> sudo shutdown -r now 2024-09-10T08:27:06.718 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T08:27:06.718 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-10T08:27:06.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:27:35.697 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T08:27:35.698 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-10T08:27:35.698 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:27:54.045 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-09-10T08:27:59.881 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T08:28:03.049 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-10T08:28:03.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:28:06.109 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-09-10T08:28:06.722 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T08:28:08.884 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-10T08:28:08.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:28:09.322 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-09-10T08:28:09.753 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-10T08:28:09.753 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-505.el9.x86_64"... 2024-09-10T08:28:09.754 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2024-09-10T08:28:09.768 INFO:teuthology.orchestra.run.smithi191.stdout:5.14.0-505.el9.x86_64 2024-09-10T08:28:09.769 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T08:28:09.769 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T08:28:09.769 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T08:28:10.769 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-10T08:28:10.770 DEBUG:teuthology.orchestra.run.smithi191:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T08:28:10.872 INFO:teuthology.orchestra.run.smithi191.stdout:ttyS1 2024-09-10T08:28:10.892 DEBUG:teuthology.parallel:result is None 2024-09-10T08:28:15.724 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-10T08:28:15.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:28:16.029 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-09-10T08:28:16.420 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-10T08:28:16.421 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-10T08:28:16.421 DEBUG:teuthology.orchestra.run.smithi174:> uname -r 2024-09-10T08:28:16.435 INFO:teuthology.orchestra.run.smithi174.stdout:5.14.0-505.el9.x86_64 2024-09-10T08:28:16.436 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T08:28:16.436 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T08:28:16.436 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T08:28:17.437 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T08:28:17.437 DEBUG:teuthology.orchestra.run.smithi174:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T08:28:17.517 INFO:teuthology.orchestra.run.smithi174.stdout:ttyS1 2024-09-10T08:28:17.536 DEBUG:teuthology.parallel:result is None 2024-09-10T08:28:18.110 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-10T08:28:18.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:29:18.114 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T08:29:33.115 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-10T08:29:33.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:29:33.504 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-09-10T08:29:33.839 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-10T08:29:33.840 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-10T08:29:33.840 DEBUG:teuthology.orchestra.run.smithi172:> uname -r 2024-09-10T08:29:33.856 INFO:teuthology.orchestra.run.smithi172.stdout:5.14.0-505.el9.x86_64 2024-09-10T08:29:33.856 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T08:29:33.856 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T08:29:33.856 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T08:29:34.856 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T08:29:34.857 DEBUG:teuthology.orchestra.run.smithi172:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T08:29:35.227 INFO:teuthology.orchestra.run.smithi172.stdout:ttyS1 2024-09-10T08:29:35.250 DEBUG:teuthology.parallel:result is None 2024-09-10T08:29:35.250 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T08:29:35.257 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T08:29:35.257 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T08:29:35.260 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T08:29:35.263 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T08:29:35.283 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T08:29:35.291 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T08:29:35.298 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T08:29:35.298 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T08:29:35.318 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T08:29:35.322 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T08:29:35.358 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T08:29:35.365 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T08:29:35.365 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T08:29:35.380 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T08:29:35.391 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T08:29:35.421 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T08:29:35.423 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T08:29:35.427 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T08:29:35.432 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T08:29:35.435 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T08:29:35.439 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T08:29:35.440 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T08:29:35.446 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T08:29:35.447 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T08:29:35.476 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T08:29:35.480 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T08:29:35.508 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T08:29:35.517 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T08:29:35.517 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T08:29:35.543 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T08:29:35.549 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T08:29:35.565 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T08:29:35.639 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T08:29:35.685 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T08:29:35.761 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T08:29:35.807 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:29:35.808 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T08:29:35.872 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T08:29:35.912 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T08:29:35.973 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T08:29:36.050 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T08:29:36.098 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T08:29:36.099 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T08:29:36.166 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T08:29:36.213 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T08:29:36.267 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T08:29:36.354 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T08:29:36.395 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-10T08:29:36.395 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T08:29:36.463 DEBUG:teuthology.orchestra.run.smithi172:> sudo service rsyslog restart 2024-09-10T08:29:36.466 DEBUG:teuthology.orchestra.run.smithi174:> sudo service rsyslog restart 2024-09-10T08:29:36.468 DEBUG:teuthology.orchestra.run.smithi191:> sudo service rsyslog restart 2024-09-10T08:29:36.516 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T08:29:36.531 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T08:29:36.548 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T08:29:37.033 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T08:29:37.041 INFO:teuthology.task.internal:Starting timer... 2024-09-10T08:29:37.041 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T08:29:37.051 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T08:29:37.060 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-10T08:29:37.060 DEBUG:teuthology.orchestra.run.smithi172:> sudo service auditd rotate 2024-09-10T08:29:37.122 INFO:teuthology.orchestra.run.smithi172.stdout:Rotating logs: 2024-09-10T08:29:37.124 DEBUG:teuthology.orchestra.run.smithi174:> sudo service auditd rotate 2024-09-10T08:29:37.169 INFO:teuthology.orchestra.run.smithi174.stdout:Rotating logs: 2024-09-10T08:29:37.171 DEBUG:teuthology.orchestra.run.smithi191:> sudo service auditd rotate 2024-09-10T08:29:37.237 INFO:teuthology.orchestra.run.smithi191.stdout:Rotating logs: 2024-09-10T08:29:37.239 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T08:29:37.239 DEBUG:teuthology.orchestra.run.smithi172:> /usr/sbin/getenforce 2024-09-10T08:29:37.264 INFO:teuthology.orchestra.run.smithi172.stdout:Permissive 2024-09-10T08:29:37.264 DEBUG:teuthology.orchestra.run.smithi174:> /usr/sbin/getenforce 2024-09-10T08:29:37.288 INFO:teuthology.orchestra.run.smithi174.stdout:Permissive 2024-09-10T08:29:37.288 DEBUG:teuthology.orchestra.run.smithi191:> /usr/sbin/getenforce 2024-09-10T08:29:37.313 INFO:teuthology.orchestra.run.smithi191.stdout:Permissive 2024-09-10T08:29:37.313 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi172.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi174.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi191.front.sepia.ceph.com': 'permissive'} 2024-09-10T08:29:37.314 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T08:29:37.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:29:37.341 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T08:29:37.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:29:37.371 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T08:29:37.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T08:29:37.399 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T08:29:37.399 DEBUG:teuthology.orchestra.run.smithi172:> sudo /usr/sbin/setenforce permissive 2024-09-10T08:29:37.434 DEBUG:teuthology.orchestra.run.smithi174:> sudo /usr/sbin/setenforce permissive 2024-09-10T08:29:37.471 DEBUG:teuthology.orchestra.run.smithi191:> sudo /usr/sbin/setenforce permissive 2024-09-10T08:29:37.500 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T08:29:37.509 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T08:29:37.511 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T08:29:37.619 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T08:29:37.630 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T08:29:37.631 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi172.front.sepia.ceph.com,smithi174.front.sepia.ceph.com,smithi191.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T08:33:28.377 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi172.front.sepia.ceph.com'), Remote(name='ubuntu@smithi174.front.sepia.ceph.com'), Remote(name='ubuntu@smithi191.front.sepia.ceph.com')] 2024-09-10T08:33:28.378 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-10T08:33:28.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:33:28.456 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-09-10T08:33:28.532 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-10T08:33:28.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-10T08:33:28.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:33:28.609 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-09-10T08:33:28.688 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-10T08:33:28.688 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-10T08:33:28.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T08:33:28.766 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-09-10T08:33:28.852 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-10T08:33:28.853 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T08:33:28.863 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T08:33:28.863 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T08:33:28.864 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T08:33:28.866 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T08:33:28.867 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T08:33:28.869 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T08:33:28.869 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T08:33:28.894 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T08:33:28.896 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T08:33:28.908 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T08:33:28.911 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T08:33:28.923 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T08:33:28.933 INFO:teuthology.orchestra.run.smithi172.stderr:sudo: ntpd: command not found 2024-09-10T08:33:28.935 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: ntpd: command not found 2024-09-10T08:33:28.938 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T08:33:28.944 INFO:teuthology.orchestra.run.smithi172.stdout:506 Cannot talk to daemon 2024-09-10T08:33:28.946 INFO:teuthology.orchestra.run.smithi174.stdout:506 Cannot talk to daemon 2024-09-10T08:33:28.959 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T08:33:28.960 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T08:33:28.962 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: ntpd: command not found 2024-09-10T08:33:28.973 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T08:33:28.973 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T08:33:28.974 INFO:teuthology.orchestra.run.smithi191.stdout:506 Cannot talk to daemon 2024-09-10T08:33:28.988 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T08:33:29.002 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T08:33:29.030 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-09-10T08:33:29.033 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T08:33:29.034 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-09-10T08:33:29.034 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:33:29.034 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:33:29.034 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:33:29.034 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:33:29.034 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-09-10T08:33:29.037 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T08:33:29.037 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-09-10T08:33:29.038 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:33:29.038 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:33:29.038 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:33:29.038 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:33:29.057 INFO:teuthology.orchestra.run.smithi191.stderr:bash: line 1: ntpq: command not found 2024-09-10T08:33:29.061 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T08:33:29.061 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-09-10T08:33:29.061 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:33:29.061 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:33:29.062 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:33:29.062 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T08:33:29.062 INFO:teuthology.run_tasks:Running task install... 2024-09-10T08:33:29.070 DEBUG:teuthology.task.install:project ceph 2024-09-10T08:33:29.070 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}} 2024-09-10T08:33:29.070 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'} 2024-09-10T08:33:29.070 INFO:teuthology.task.install:Using flavor: default 2024-09-10T08:33:29.075 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T08:33:29.076 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T08:33:29.076 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T08:33:29.076 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:33:29.078 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T08:33:29.078 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:33:29.079 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T08:33:29.080 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:33:29.235 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T08:33:29.235 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T08:33:29.239 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T08:33:29.239 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T08:33:29.243 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T08:33:29.243 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T08:33:29.349 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T08:33:29.349 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-10T08:33:29.349 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T08:33:29.354 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T08:33:29.354 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T08:33:29.354 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T08:33:29.359 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T08:33:29.359 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:33:29.359 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T08:33:29.376 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T08:33:29.376 DEBUG:teuthology.orchestra.run.smithi191:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T08:33:29.381 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T08:33:29.381 DEBUG:teuthology.orchestra.run.smithi174:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T08:33:29.385 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T08:33:29.386 DEBUG:teuthology.orchestra.run.smithi172:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T08:33:29.442 DEBUG:teuthology.orchestra.run.smithi191:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T08:33:29.448 DEBUG:teuthology.orchestra.run.smithi174:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T08:33:29.452 DEBUG:teuthology.orchestra.run.smithi172:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T08:33:29.517 DEBUG:teuthology.orchestra.run.smithi191:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T08:33:29.522 DEBUG:teuthology.orchestra.run.smithi174:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T08:33:29.527 DEBUG:teuthology.orchestra.run.smithi172:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T08:33:29.582 INFO:teuthology.orchestra.run.smithi191.stdout:check_obsoletes = 1 2024-09-10T08:33:29.583 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum clean all 2024-09-10T08:33:29.586 INFO:teuthology.orchestra.run.smithi174.stdout:check_obsoletes = 1 2024-09-10T08:33:29.587 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum clean all 2024-09-10T08:33:29.591 INFO:teuthology.orchestra.run.smithi172.stdout:check_obsoletes = 1 2024-09-10T08:33:29.593 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum clean all 2024-09-10T08:33:29.807 INFO:teuthology.orchestra.run.smithi191.stdout:45 files removed 2024-09-10T08:33:29.829 INFO:teuthology.orchestra.run.smithi174.stdout:45 files removed 2024-09-10T08:33:29.834 INFO:teuthology.orchestra.run.smithi172.stdout:45 files removed 2024-09-10T08:33:29.835 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T08:33:29.856 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T08:33:29.861 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T08:33:30.399 INFO:teuthology.orchestra.run.smithi191.stdout:ceph packages for x86_64 302 kB/s | 84 kB 00:00 2024-09-10T08:33:30.402 INFO:teuthology.orchestra.run.smithi174.stdout:ceph packages for x86_64 323 kB/s | 84 kB 00:00 2024-09-10T08:33:30.408 INFO:teuthology.orchestra.run.smithi172.stdout:ceph packages for x86_64 320 kB/s | 84 kB 00:00 2024-09-10T08:33:30.677 INFO:teuthology.orchestra.run.smithi174.stdout:ceph noarch packages 51 kB/s | 12 kB 00:00 2024-09-10T08:33:30.683 INFO:teuthology.orchestra.run.smithi191.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T08:33:30.684 INFO:teuthology.orchestra.run.smithi172.stdout:ceph noarch packages 51 kB/s | 12 kB 00:00 2024-09-10T08:33:30.936 INFO:teuthology.orchestra.run.smithi174.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-10T08:33:30.959 INFO:teuthology.orchestra.run.smithi172.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-10T08:33:30.960 INFO:teuthology.orchestra.run.smithi191.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-10T08:33:31.643 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-10T08:33:32.625 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.2 MB 00:01 2024-09-10T08:33:35.877 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.2 MB 00:04 2024-09-10T08:33:35.985 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - AppStream 6.3 MB/s | 20 MB 00:03 2024-09-10T08:33:37.645 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-10T08:33:37.759 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - AppStream 5.1 MB/s | 20 MB 00:04 2024-09-10T08:33:41.503 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-10T08:33:42.946 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.5 MB 00:01 2024-09-10T08:33:42.978 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 19 kB 00:00 2024-09-10T08:33:43.260 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-10T08:33:43.916 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-10T08:33:44.836 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 19 kB 00:00 2024-09-10T08:33:45.272 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-10T08:33:46.065 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-10T08:33:46.234 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-10T08:33:49.372 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-09-10T08:33:51.471 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-10T08:33:51.590 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T08:33:51.751 INFO:teuthology.orchestra.run.smithi172.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T08:33:51.752 INFO:teuthology.orchestra.run.smithi172.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T08:33:51.766 INFO:teuthology.orchestra.run.smithi172.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T08:33:51.766 INFO:teuthology.orchestra.run.smithi172.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T08:33:51.767 INFO:teuthology.orchestra.run.smithi172.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T08:33:51.768 INFO:teuthology.orchestra.run.smithi172.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T08:33:51.770 INFO:teuthology.orchestra.run.smithi172.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T08:33:51.773 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T08:33:51.774 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T08:33:51.839 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-10T08:33:51.859 INFO:teuthology.orchestra.run.smithi172.stdout:====================================================================================== 2024-09-10T08:33:51.859 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-09-10T08:33:51.859 INFO:teuthology.orchestra.run.smithi172.stdout:====================================================================================== 2024-09-10T08:33:51.859 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-10T08:33:51.859 INFO:teuthology.orchestra.run.smithi172.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T08:33:51.859 INFO:teuthology.orchestra.run.smithi172.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T08:33:51.859 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T08:33:51.860 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T08:33:51.860 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T08:33:51.860 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T08:33:51.860 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T08:33:51.860 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T08:33:51.860 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T08:33:51.860 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T08:33:51.860 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T08:33:51.860 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T08:33:51.860 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T08:33:51.860 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T08:33:51.860 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T08:33:51.861 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T08:33:51.861 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T08:33:51.861 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T08:33:51.861 INFO:teuthology.orchestra.run.smithi172.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T08:33:51.861 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T08:33:51.861 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T08:33:51.861 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T08:33:51.861 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T08:33:51.861 INFO:teuthology.orchestra.run.smithi172.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T08:33:51.861 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T08:33:51.861 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T08:33:51.861 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T08:33:51.861 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T08:33:51.862 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T08:33:51.862 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T08:33:51.862 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T08:33:51.862 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T08:33:51.862 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T08:33:51.862 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T08:33:51.862 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T08:33:51.862 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T08:33:51.862 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T08:33:51.862 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrading: 2024-09-10T08:33:51.862 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T08:33:51.862 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T08:33:51.863 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T08:33:51.863 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T08:33:51.863 INFO:teuthology.orchestra.run.smithi172.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T08:33:51.863 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T08:33:51.863 INFO:teuthology.orchestra.run.smithi172.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T08:33:51.863 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T08:33:51.863 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-09-10T08:33:51.863 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T08:33:51.863 INFO:teuthology.orchestra.run.smithi172.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T08:33:51.863 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T08:33:51.863 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T08:33:51.863 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T08:33:51.863 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T08:33:51.864 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T08:33:51.864 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T08:33:51.864 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T08:33:51.864 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T08:33:51.864 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T08:33:51.864 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T08:33:51.864 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T08:33:51.864 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T08:33:51.864 INFO:teuthology.orchestra.run.smithi172.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T08:33:51.864 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T08:33:51.864 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T08:33:51.865 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T08:33:51.865 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T08:33:51.865 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T08:33:51.865 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T08:33:51.865 INFO:teuthology.orchestra.run.smithi172.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T08:33:51.865 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T08:33:51.865 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T08:33:51.865 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T08:33:51.865 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T08:33:51.865 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T08:33:51.865 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T08:33:51.865 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T08:33:51.865 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T08:33:51.866 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T08:33:51.866 INFO:teuthology.orchestra.run.smithi172.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T08:33:51.866 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T08:33:51.866 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T08:33:51.866 INFO:teuthology.orchestra.run.smithi172.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T08:33:51.866 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T08:33:51.866 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T08:33:51.866 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T08:33:51.866 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T08:33:51.866 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T08:33:51.866 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T08:33:51.866 INFO:teuthology.orchestra.run.smithi172.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T08:33:51.866 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T08:33:51.867 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T08:33:51.867 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T08:33:51.867 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T08:33:51.867 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T08:33:51.867 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T08:33:51.867 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T08:33:51.867 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T08:33:51.867 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T08:33:51.867 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T08:33:51.867 INFO:teuthology.orchestra.run.smithi172.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T08:33:51.867 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T08:33:51.867 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T08:33:51.867 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T08:33:51.868 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T08:33:51.868 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T08:33:51.868 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T08:33:51.868 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T08:33:51.868 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T08:33:51.868 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T08:33:51.868 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T08:33:51.868 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T08:33:51.868 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T08:33:51.868 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T08:33:51.868 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T08:33:51.868 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T08:33:51.868 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T08:33:51.869 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T08:33:51.869 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T08:33:51.869 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T08:33:51.869 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T08:33:51.869 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T08:33:51.869 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T08:33:51.869 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T08:33:51.869 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T08:33:51.869 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T08:33:51.869 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T08:33:51.869 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T08:33:51.869 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T08:33:51.870 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T08:33:51.870 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T08:33:51.870 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T08:33:51.870 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T08:33:51.870 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T08:33:51.870 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T08:33:51.870 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T08:33:51.870 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T08:33:51.870 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T08:33:51.870 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T08:33:51.870 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T08:33:51.870 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T08:33:51.870 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T08:33:51.870 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T08:33:51.871 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T08:33:51.871 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T08:33:51.871 INFO:teuthology.orchestra.run.smithi172.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T08:33:51.871 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T08:33:51.871 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T08:33:51.871 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T08:33:51.871 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T08:33:51.871 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T08:33:51.871 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T08:33:51.871 INFO:teuthology.orchestra.run.smithi172.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T08:33:51.871 INFO:teuthology.orchestra.run.smithi172.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T08:33:51.871 INFO:teuthology.orchestra.run.smithi172.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T08:33:51.872 INFO:teuthology.orchestra.run.smithi172.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T08:33:51.872 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T08:33:51.872 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T08:33:51.872 INFO:teuthology.orchestra.run.smithi172.stdout:Installing weak dependencies: 2024-09-10T08:33:51.872 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T08:33:51.872 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:33:51.872 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-10T08:33:51.872 INFO:teuthology.orchestra.run.smithi172.stdout:====================================================================================== 2024-09-10T08:33:51.872 INFO:teuthology.orchestra.run.smithi172.stdout:Install 148 Packages 2024-09-10T08:33:51.872 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrade 8 Packages 2024-09-10T08:33:51.872 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:33:51.873 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 226 M 2024-09-10T08:33:51.873 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-10T08:33:52.422 INFO:teuthology.orchestra.run.smithi172.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 54 kB/s | 6.5 kB 00:00 2024-09-10T08:33:52.614 INFO:teuthology.orchestra.run.smithi172.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 4.6 MB/s | 897 kB 00:00 2024-09-10T08:33:52.698 INFO:teuthology.orchestra.run.smithi172.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.7 MB/s | 145 kB 00:00 2024-09-10T08:33:52.782 INFO:teuthology.orchestra.run.smithi172.stdout:(4/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-10T08:33:52.890 INFO:teuthology.orchestra.run.smithi172.stdout:(5/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 14 MB/s | 1.5 MB 00:00 2024-09-10T08:33:52.940 INFO:teuthology.orchestra.run.smithi172.stdout:(6/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 9.9 MB/s | 2.4 MB 00:00 2024-09-10T08:33:53.174 INFO:teuthology.orchestra.run.smithi172.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 17 MB/s | 4.7 MB 00:00 2024-09-10T08:33:53.441 INFO:teuthology.orchestra.run.smithi172.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 19 MB/s | 21 MB 00:01 2024-09-10T08:33:53.483 INFO:teuthology.orchestra.run.smithi172.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 608 kB/s | 25 kB 00:00 2024-09-10T08:33:53.850 INFO:teuthology.orchestra.run.smithi191.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T08:33:53.851 INFO:teuthology.orchestra.run.smithi191.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T08:33:53.858 INFO:teuthology.orchestra.run.smithi172.stdout:(10/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 18 MB/s | 17 MB 00:00 2024-09-10T08:33:53.864 INFO:teuthology.orchestra.run.smithi191.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T08:33:53.865 INFO:teuthology.orchestra.run.smithi191.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T08:33:53.866 INFO:teuthology.orchestra.run.smithi191.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T08:33:53.867 INFO:teuthology.orchestra.run.smithi191.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T08:33:53.869 INFO:teuthology.orchestra.run.smithi191.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T08:33:53.872 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T08:33:53.873 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T08:33:53.908 INFO:teuthology.orchestra.run.smithi172.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 4.5 MB/s | 228 kB 00:00 2024-09-10T08:33:53.935 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-09-10T08:33:53.952 INFO:teuthology.orchestra.run.smithi191.stdout:====================================================================================== 2024-09-10T08:33:53.952 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-09-10T08:33:53.952 INFO:teuthology.orchestra.run.smithi191.stdout:====================================================================================== 2024-09-10T08:33:53.952 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-09-10T08:33:53.952 INFO:teuthology.orchestra.run.smithi191.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T08:33:53.953 INFO:teuthology.orchestra.run.smithi191.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T08:33:53.953 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T08:33:53.953 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T08:33:53.953 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T08:33:53.953 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T08:33:53.953 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T08:33:53.953 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T08:33:53.953 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T08:33:53.953 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T08:33:53.953 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T08:33:53.953 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T08:33:53.953 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T08:33:53.954 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T08:33:53.954 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T08:33:53.954 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T08:33:53.954 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T08:33:53.954 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T08:33:53.954 INFO:teuthology.orchestra.run.smithi191.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T08:33:53.954 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T08:33:53.954 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T08:33:53.954 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T08:33:53.954 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T08:33:53.954 INFO:teuthology.orchestra.run.smithi191.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T08:33:53.954 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T08:33:53.954 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T08:33:53.955 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T08:33:53.955 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T08:33:53.955 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T08:33:53.955 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T08:33:53.955 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T08:33:53.955 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T08:33:53.955 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T08:33:53.955 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T08:33:53.955 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T08:33:53.955 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T08:33:53.955 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T08:33:53.955 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrading: 2024-09-10T08:33:53.955 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T08:33:53.956 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T08:33:53.956 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T08:33:53.956 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T08:33:53.956 INFO:teuthology.orchestra.run.smithi191.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T08:33:53.956 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T08:33:53.956 INFO:teuthology.orchestra.run.smithi191.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T08:33:53.956 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T08:33:53.956 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-09-10T08:33:53.956 INFO:teuthology.orchestra.run.smithi191.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T08:33:53.956 INFO:teuthology.orchestra.run.smithi191.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T08:33:53.956 INFO:teuthology.orchestra.run.smithi191.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T08:33:53.956 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T08:33:53.957 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T08:33:53.957 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T08:33:53.957 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T08:33:53.957 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T08:33:53.957 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T08:33:53.957 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T08:33:53.957 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T08:33:53.957 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T08:33:53.957 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T08:33:53.957 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T08:33:53.957 INFO:teuthology.orchestra.run.smithi191.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T08:33:53.957 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T08:33:53.957 INFO:teuthology.orchestra.run.smithi191.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T08:33:53.958 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T08:33:53.958 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T08:33:53.958 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T08:33:53.958 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T08:33:53.958 INFO:teuthology.orchestra.run.smithi191.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T08:33:53.958 INFO:teuthology.orchestra.run.smithi191.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T08:33:53.958 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T08:33:53.958 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T08:33:53.958 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T08:33:53.958 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T08:33:53.958 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T08:33:53.958 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T08:33:53.958 INFO:teuthology.orchestra.run.smithi191.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T08:33:53.959 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T08:33:53.959 INFO:teuthology.orchestra.run.smithi191.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T08:33:53.959 INFO:teuthology.orchestra.run.smithi191.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T08:33:53.959 INFO:teuthology.orchestra.run.smithi191.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T08:33:53.959 INFO:teuthology.orchestra.run.smithi191.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T08:33:53.959 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T08:33:53.959 INFO:teuthology.orchestra.run.smithi191.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T08:33:53.959 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T08:33:53.959 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T08:33:53.959 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T08:33:53.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T08:33:53.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T08:33:53.959 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T08:33:53.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T08:33:53.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T08:33:53.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T08:33:53.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T08:33:53.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T08:33:53.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T08:33:53.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T08:33:53.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T08:33:53.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T08:33:53.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T08:33:53.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T08:33:53.960 INFO:teuthology.orchestra.run.smithi191.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T08:33:53.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T08:33:53.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T08:33:53.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T08:33:53.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T08:33:53.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T08:33:53.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T08:33:53.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T08:33:53.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T08:33:53.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T08:33:53.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T08:33:53.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T08:33:53.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T08:33:53.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T08:33:53.961 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T08:33:53.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T08:33:53.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T08:33:53.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T08:33:53.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T08:33:53.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T08:33:53.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T08:33:53.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T08:33:53.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T08:33:53.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T08:33:53.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T08:33:53.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T08:33:53.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T08:33:53.962 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T08:33:53.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T08:33:53.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T08:33:53.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T08:33:53.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T08:33:53.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T08:33:53.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T08:33:53.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T08:33:53.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T08:33:53.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T08:33:53.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T08:33:53.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T08:33:53.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T08:33:53.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T08:33:53.963 INFO:teuthology.orchestra.run.smithi191.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T08:33:53.964 INFO:teuthology.orchestra.run.smithi191.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T08:33:53.964 INFO:teuthology.orchestra.run.smithi191.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T08:33:53.964 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T08:33:53.964 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T08:33:53.964 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T08:33:53.964 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T08:33:53.964 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T08:33:53.964 INFO:teuthology.orchestra.run.smithi191.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T08:33:53.964 INFO:teuthology.orchestra.run.smithi191.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T08:33:53.964 INFO:teuthology.orchestra.run.smithi191.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T08:33:53.964 INFO:teuthology.orchestra.run.smithi191.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T08:33:53.964 INFO:teuthology.orchestra.run.smithi191.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T08:33:53.964 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T08:33:53.965 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T08:33:53.965 INFO:teuthology.orchestra.run.smithi191.stdout:Installing weak dependencies: 2024-09-10T08:33:53.965 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T08:33:53.965 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:33:53.965 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-09-10T08:33:53.965 INFO:teuthology.orchestra.run.smithi191.stdout:====================================================================================== 2024-09-10T08:33:53.965 INFO:teuthology.orchestra.run.smithi191.stdout:Install 148 Packages 2024-09-10T08:33:53.965 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrade 8 Packages 2024-09-10T08:33:53.965 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:33:53.966 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 226 M 2024-09-10T08:33:53.966 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-09-10T08:33:53.975 INFO:teuthology.orchestra.run.smithi172.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 406 kB/s | 27 kB 00:00 2024-09-10T08:33:54.008 INFO:teuthology.orchestra.run.smithi174.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T08:33:54.009 INFO:teuthology.orchestra.run.smithi174.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T08:33:54.020 INFO:teuthology.orchestra.run.smithi174.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T08:33:54.021 INFO:teuthology.orchestra.run.smithi174.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T08:33:54.022 INFO:teuthology.orchestra.run.smithi174.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T08:33:54.022 INFO:teuthology.orchestra.run.smithi174.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T08:33:54.024 INFO:teuthology.orchestra.run.smithi174.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T08:33:54.026 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T08:33:54.027 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T08:33:54.034 INFO:teuthology.orchestra.run.smithi172.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 544 kB/s | 32 kB 00:00 2024-09-10T08:33:54.089 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-09-10T08:33:54.113 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-09-10T08:33:54.113 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-09-10T08:33:54.113 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-09-10T08:33:54.113 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-09-10T08:33:54.113 INFO:teuthology.orchestra.run.smithi174.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T08:33:54.113 INFO:teuthology.orchestra.run.smithi174.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T08:33:54.113 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T08:33:54.113 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T08:33:54.114 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T08:33:54.114 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T08:33:54.114 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T08:33:54.114 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T08:33:54.114 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T08:33:54.114 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T08:33:54.114 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T08:33:54.114 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T08:33:54.114 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T08:33:54.114 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T08:33:54.114 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T08:33:54.114 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T08:33:54.114 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T08:33:54.115 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T08:33:54.115 INFO:teuthology.orchestra.run.smithi174.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T08:33:54.115 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T08:33:54.115 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T08:33:54.115 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T08:33:54.115 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T08:33:54.115 INFO:teuthology.orchestra.run.smithi174.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T08:33:54.115 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T08:33:54.115 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T08:33:54.115 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T08:33:54.115 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T08:33:54.115 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T08:33:54.115 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T08:33:54.115 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T08:33:54.116 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T08:33:54.116 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T08:33:54.116 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T08:33:54.116 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T08:33:54.116 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T08:33:54.116 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T08:33:54.116 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrading: 2024-09-10T08:33:54.116 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T08:33:54.116 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T08:33:54.116 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T08:33:54.116 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T08:33:54.116 INFO:teuthology.orchestra.run.smithi174.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T08:33:54.116 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T08:33:54.117 INFO:teuthology.orchestra.run.smithi174.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T08:33:54.117 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T08:33:54.117 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-09-10T08:33:54.117 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T08:33:54.117 INFO:teuthology.orchestra.run.smithi174.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T08:33:54.117 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T08:33:54.117 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T08:33:54.117 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T08:33:54.117 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T08:33:54.117 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T08:33:54.117 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T08:33:54.117 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T08:33:54.117 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T08:33:54.117 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T08:33:54.118 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T08:33:54.118 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T08:33:54.118 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T08:33:54.118 INFO:teuthology.orchestra.run.smithi174.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T08:33:54.118 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T08:33:54.118 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T08:33:54.118 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T08:33:54.118 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T08:33:54.118 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T08:33:54.118 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T08:33:54.118 INFO:teuthology.orchestra.run.smithi174.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T08:33:54.118 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T08:33:54.118 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T08:33:54.118 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T08:33:54.119 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T08:33:54.119 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T08:33:54.119 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T08:33:54.119 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T08:33:54.119 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T08:33:54.119 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T08:33:54.119 INFO:teuthology.orchestra.run.smithi174.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T08:33:54.119 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T08:33:54.119 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T08:33:54.120 INFO:teuthology.orchestra.run.smithi174.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T08:33:54.120 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T08:33:54.120 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T08:33:54.120 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T08:33:54.120 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T08:33:54.120 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T08:33:54.120 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T08:33:54.120 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T08:33:54.120 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T08:33:54.120 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T08:33:54.120 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T08:33:54.120 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T08:33:54.121 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T08:33:54.121 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T08:33:54.121 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T08:33:54.121 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T08:33:54.121 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T08:33:54.121 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T08:33:54.121 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T08:33:54.121 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T08:33:54.121 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T08:33:54.121 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T08:33:54.121 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T08:33:54.121 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T08:33:54.122 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T08:33:54.122 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T08:33:54.122 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T08:33:54.122 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T08:33:54.122 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T08:33:54.122 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T08:33:54.122 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T08:33:54.122 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T08:33:54.122 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T08:33:54.122 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T08:33:54.122 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T08:33:54.122 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T08:33:54.123 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T08:33:54.123 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T08:33:54.123 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T08:33:54.123 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T08:33:54.123 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T08:33:54.123 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T08:33:54.123 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T08:33:54.123 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T08:33:54.123 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T08:33:54.123 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T08:33:54.123 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T08:33:54.123 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T08:33:54.123 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T08:33:54.124 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T08:33:54.124 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T08:33:54.124 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T08:33:54.124 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T08:33:54.124 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T08:33:54.124 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T08:33:54.124 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T08:33:54.124 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T08:33:54.124 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T08:33:54.124 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T08:33:54.124 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T08:33:54.124 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T08:33:54.124 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T08:33:54.125 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T08:33:54.125 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T08:33:54.125 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T08:33:54.125 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T08:33:54.125 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T08:33:54.125 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T08:33:54.125 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T08:33:54.125 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T08:33:54.125 INFO:teuthology.orchestra.run.smithi174.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T08:33:54.125 INFO:teuthology.orchestra.run.smithi174.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T08:33:54.125 INFO:teuthology.orchestra.run.smithi174.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T08:33:54.125 INFO:teuthology.orchestra.run.smithi174.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T08:33:54.125 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T08:33:54.126 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T08:33:54.126 INFO:teuthology.orchestra.run.smithi174.stdout:Installing weak dependencies: 2024-09-10T08:33:54.126 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T08:33:54.126 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:33:54.126 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-09-10T08:33:54.126 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-09-10T08:33:54.126 INFO:teuthology.orchestra.run.smithi174.stdout:Install 148 Packages 2024-09-10T08:33:54.126 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrade 8 Packages 2024-09-10T08:33:54.126 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:33:54.127 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 226 M 2024-09-10T08:33:54.127 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-09-10T08:33:54.134 INFO:teuthology.orchestra.run.smithi172.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 7.4 MB/s | 751 kB 00:00 2024-09-10T08:33:54.184 INFO:teuthology.orchestra.run.smithi172.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 3.2 MB/s | 164 kB 00:00 2024-09-10T08:33:54.251 INFO:teuthology.orchestra.run.smithi172.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 1.9 MB/s | 127 kB 00:00 2024-09-10T08:33:54.264 INFO:teuthology.orchestra.run.smithi191.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 38 kB/s | 6.5 kB 00:00 2024-09-10T08:33:54.393 INFO:teuthology.orchestra.run.smithi172.stdout:(17/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 8.7 MB/s | 11 MB 00:01 2024-09-10T08:33:54.427 INFO:teuthology.orchestra.run.smithi172.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 2.8 MB/s | 506 kB 00:00 2024-09-10T08:33:54.510 INFO:teuthology.orchestra.run.smithi172.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 541 kB/s | 45 kB 00:00 2024-09-10T08:33:54.594 INFO:teuthology.orchestra.run.smithi172.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 1.6 MB/s | 136 kB 00:00 2024-09-10T08:33:54.655 INFO:teuthology.orchestra.run.smithi191.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 2.2 MB/s | 897 kB 00:00 2024-09-10T08:33:54.669 INFO:teuthology.orchestra.run.smithi172.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 2.1 MB/s | 161 kB 00:00 2024-09-10T08:33:54.739 INFO:teuthology.orchestra.run.smithi191.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.7 MB/s | 145 kB 00:00 2024-09-10T08:33:54.752 INFO:teuthology.orchestra.run.smithi172.stdout:(22/156): python3-rados-19.1.1-305.g1cf819b7.el 3.7 MB/s | 320 kB 00:00 2024-09-10T08:33:54.870 INFO:teuthology.orchestra.run.smithi172.stdout:(23/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 2.5 MB/s | 301 kB 00:00 2024-09-10T08:33:54.936 INFO:teuthology.orchestra.run.smithi172.stdout:(24/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 9.8 MB/s | 5.3 MB 00:00 2024-09-10T08:33:54.962 INFO:teuthology.orchestra.run.smithi172.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.1 MB/s | 99 kB 00:00 2024-09-10T08:33:54.999 INFO:teuthology.orchestra.run.smithi174.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 28 kB/s | 6.5 kB 00:00 2024-09-10T08:33:55.020 INFO:teuthology.orchestra.run.smithi172.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.0 MB/s | 85 kB 00:00 2024-09-10T08:33:55.254 INFO:teuthology.orchestra.run.smithi172.stdout:(27/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 11 MB/s | 3.1 MB 00:00 2024-09-10T08:33:55.373 INFO:teuthology.orchestra.run.smithi191.stdout:(4/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 3.8 MB/s | 2.4 MB 00:00 2024-09-10T08:33:55.500 INFO:teuthology.orchestra.run.smithi174.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 1.8 MB/s | 897 kB 00:00 2024-09-10T08:33:55.566 INFO:teuthology.orchestra.run.smithi174.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 2.1 MB/s | 145 kB 00:00 2024-09-10T08:33:55.673 INFO:teuthology.orchestra.run.smithi191.stdout:(5/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 13 MB/s | 21 MB 00:01 2024-09-10T08:33:55.696 INFO:teuthology.orchestra.run.smithi172.stdout:(28/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 22 MB/s | 49 MB 00:02 2024-09-10T08:33:55.713 INFO:teuthology.orchestra.run.smithi172.stdout:(29/156): ceph-grafana-dashboards-19.1.1-305.g1 69 kB/s | 32 kB 00:00 2024-09-10T08:33:55.738 INFO:teuthology.orchestra.run.smithi172.stdout:(30/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 237 kB/s | 170 kB 00:00 2024-09-10T08:33:55.740 INFO:teuthology.orchestra.run.smithi191.stdout:(6/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 3.3 MB/s | 5.4 MB 00:01 2024-09-10T08:33:55.764 INFO:teuthology.orchestra.run.smithi172.stdout:(31/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.1 MB/s | 146 kB 00:00 2024-09-10T08:33:55.790 INFO:teuthology.orchestra.run.smithi191.stdout:(7/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 3.6 MB/s | 1.5 MB 00:00 2024-09-10T08:33:55.847 INFO:teuthology.orchestra.run.smithi172.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 2.9 MB/s | 249 kB 00:00 2024-09-10T08:33:55.922 INFO:teuthology.orchestra.run.smithi172.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 658 kB/s | 49 kB 00:00 2024-09-10T08:33:55.964 INFO:teuthology.orchestra.run.smithi172.stdout:(34/156): ceph-prometheus-alerts-19.1.1-305.g1c 396 kB/s | 16 kB 00:00 2024-09-10T08:33:55.965 INFO:teuthology.orchestra.run.smithi191.stdout:(8/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 16 MB/s | 4.7 MB 00:00 2024-09-10T08:33:56.015 INFO:teuthology.orchestra.run.smithi172.stdout:(35/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 5.6 MB/s | 285 kB 00:00 2024-09-10T08:33:56.015 INFO:teuthology.orchestra.run.smithi191.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 505 kB/s | 25 kB 00:00 2024-09-10T08:33:56.107 INFO:teuthology.orchestra.run.smithi172.stdout:(36/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 9.0 MB/s | 3.6 MB 00:00 2024-09-10T08:33:56.132 INFO:teuthology.orchestra.run.smithi172.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 6.4 MB/s | 766 kB 00:00 2024-09-10T08:33:56.141 INFO:teuthology.orchestra.run.smithi174.stdout:(4/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 4.2 MB/s | 2.4 MB 00:00 2024-09-10T08:33:56.182 INFO:teuthology.orchestra.run.smithi172.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 242 kB/s | 18 kB 00:00 2024-09-10T08:33:56.557 INFO:teuthology.orchestra.run.smithi172.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 188 kB/s | 80 kB 00:00 2024-09-10T08:33:56.808 INFO:teuthology.orchestra.run.smithi174.stdout:(5/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 2.6 MB/s | 5.4 MB 00:02 2024-09-10T08:33:56.816 INFO:teuthology.orchestra.run.smithi172.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 740 kB/s | 191 kB 00:00 2024-09-10T08:33:56.950 INFO:teuthology.orchestra.run.smithi172.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 801 kB 00:00 2024-09-10T08:33:56.975 INFO:teuthology.orchestra.run.smithi174.stdout:(6/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 1.8 MB/s | 1.5 MB 00:00 2024-09-10T08:33:57.042 INFO:teuthology.orchestra.run.smithi172.stdout:(42/156): ceph-mgr-diskprediction-local-19.1.1- 5.7 MB/s | 7.4 MB 00:01 2024-09-10T08:33:57.075 INFO:teuthology.orchestra.run.smithi172.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 978 kB/s | 253 kB 00:00 2024-09-10T08:33:57.184 INFO:teuthology.orchestra.run.smithi172.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-10T08:33:57.293 INFO:teuthology.orchestra.run.smithi172.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-10T08:33:57.326 INFO:teuthology.orchestra.run.smithi172.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 3.3 MB/s | 1.2 MB 00:00 2024-09-10T08:33:57.401 INFO:teuthology.orchestra.run.smithi172.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-09-10T08:33:57.468 INFO:teuthology.orchestra.run.smithi172.stdout:(48/156): python3-ply-3.11-14.el9.noarch.rpm 250 kB/s | 106 kB 00:00 2024-09-10T08:33:57.577 INFO:teuthology.orchestra.run.smithi172.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 689 kB 00:00 2024-09-10T08:33:57.627 INFO:teuthology.orchestra.run.smithi172.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 675 kB/s | 107 kB 00:00 2024-09-10T08:33:57.719 INFO:teuthology.orchestra.run.smithi172.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-09-10T08:33:57.744 INFO:teuthology.orchestra.run.smithi172.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 130 kB/s | 22 kB 00:00 2024-09-10T08:33:57.769 INFO:teuthology.orchestra.run.smithi172.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-09-10T08:33:57.775 INFO:teuthology.orchestra.run.smithi174.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 4.9 MB/s | 4.7 MB 00:00 2024-09-10T08:33:57.791 INFO:teuthology.orchestra.run.smithi191.stdout:(10/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 27 MB/s | 49 MB 00:01 2024-09-10T08:33:57.795 INFO:teuthology.orchestra.run.smithi172.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 433 kB/s | 33 kB 00:00 2024-09-10T08:33:57.820 INFO:teuthology.orchestra.run.smithi172.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-10T08:33:57.845 INFO:teuthology.orchestra.run.smithi172.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-10T08:33:57.870 INFO:teuthology.orchestra.run.smithi172.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-10T08:33:57.896 INFO:teuthology.orchestra.run.smithi172.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-09-10T08:33:57.916 INFO:teuthology.orchestra.run.smithi191.stdout:(11/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 5.0 MB/s | 11 MB 00:02 2024-09-10T08:33:57.941 INFO:teuthology.orchestra.run.smithi191.stdout:(12/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 1.5 MB/s | 228 kB 00:00 2024-09-10T08:33:57.966 INFO:teuthology.orchestra.run.smithi191.stdout:(13/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 543 kB/s | 27 kB 00:00 2024-09-10T08:33:57.992 INFO:teuthology.orchestra.run.smithi191.stdout:(14/156): libcephfs-devel-19.1.1-305.g1cf819b7. 633 kB/s | 32 kB 00:00 2024-09-10T08:33:58.038 INFO:teuthology.orchestra.run.smithi172.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 4.0 MB/s | 581 kB 00:00 2024-09-10T08:33:58.042 INFO:teuthology.orchestra.run.smithi191.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 3.2 MB/s | 164 kB 00:00 2024-09-10T08:33:58.071 INFO:teuthology.orchestra.run.smithi172.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-10T08:33:58.075 INFO:teuthology.orchestra.run.smithi191.stdout:(16/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 6.8 MB/s | 751 kB 00:00 2024-09-10T08:33:58.089 INFO:teuthology.orchestra.run.smithi172.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 484 kB/s | 24 kB 00:00 2024-09-10T08:33:58.101 INFO:teuthology.orchestra.run.smithi191.stdout:(17/156): librados-devel-19.1.1-305.g1cf819b7.e 2.1 MB/s | 127 kB 00:00 2024-09-10T08:33:58.114 INFO:teuthology.orchestra.run.smithi172.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-10T08:33:58.134 INFO:teuthology.orchestra.run.smithi191.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 8.5 MB/s | 506 kB 00:00 2024-09-10T08:33:58.197 INFO:teuthology.orchestra.run.smithi172.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-10T08:33:58.201 INFO:teuthology.orchestra.run.smithi191.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 678 kB/s | 45 kB 00:00 2024-09-10T08:33:58.223 INFO:teuthology.orchestra.run.smithi172.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-10T08:33:58.248 INFO:teuthology.orchestra.run.smithi172.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-10T08:33:58.285 INFO:teuthology.orchestra.run.smithi191.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 1.6 MB/s | 136 kB 00:00 2024-09-10T08:33:58.377 INFO:teuthology.orchestra.run.smithi191.stdout:(21/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 19 MB/s | 5.3 MB 00:00 2024-09-10T08:33:58.390 INFO:teuthology.orchestra.run.smithi172.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.6 MB/s | 3.0 MB 00:00 2024-09-10T08:33:58.402 INFO:teuthology.orchestra.run.smithi191.stdout:(22/156): python3-cephfs-19.1.1-305.g1cf819b7.e 1.3 MB/s | 161 kB 00:00 2024-09-10T08:33:58.428 INFO:teuthology.orchestra.run.smithi191.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 6.3 MB/s | 320 kB 00:00 2024-09-10T08:33:58.453 INFO:teuthology.orchestra.run.smithi191.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 5.9 MB/s | 301 kB 00:00 2024-09-10T08:33:58.487 INFO:teuthology.orchestra.run.smithi191.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.7 MB/s | 99 kB 00:00 2024-09-10T08:33:58.512 INFO:teuthology.orchestra.run.smithi191.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-10T08:33:58.604 INFO:teuthology.orchestra.run.smithi191.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 1.8 MB/s | 170 kB 00:00 2024-09-10T08:33:58.654 INFO:teuthology.orchestra.run.smithi191.stdout:(28/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 19 MB/s | 3.1 MB 00:00 2024-09-10T08:33:58.680 INFO:teuthology.orchestra.run.smithi191.stdout:(29/156): ceph-grafana-dashboards-19.1.1-305.g1 423 kB/s | 32 kB 00:00 2024-09-10T08:33:58.705 INFO:teuthology.orchestra.run.smithi191.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.9 MB/s | 146 kB 00:00 2024-09-10T08:33:58.899 INFO:teuthology.orchestra.run.smithi172.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 3.1 MB/s | 1.6 MB 00:00 2024-09-10T08:33:58.997 INFO:teuthology.orchestra.run.smithi191.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 25 MB/s | 7.4 MB 00:00 2024-09-10T08:33:59.064 INFO:teuthology.orchestra.run.smithi191.stdout:(32/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 9.3 MB/s | 3.6 MB 00:00 2024-09-10T08:33:59.089 INFO:teuthology.orchestra.run.smithi191.stdout:(33/156): ceph-mgr-modules-core-19.1.1-305.g1cf 2.7 MB/s | 249 kB 00:00 2024-09-10T08:33:59.114 INFO:teuthology.orchestra.run.smithi191.stdout:(34/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 985 kB/s | 49 kB 00:00 2024-09-10T08:33:59.140 INFO:teuthology.orchestra.run.smithi191.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 328 kB/s | 16 kB 00:00 2024-09-10T08:33:59.165 INFO:teuthology.orchestra.run.smithi191.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 5.6 MB/s | 285 kB 00:00 2024-09-10T08:33:59.198 INFO:teuthology.orchestra.run.smithi191.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 13 MB/s | 766 kB 00:00 2024-09-10T08:33:59.224 INFO:teuthology.orchestra.run.smithi191.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 312 kB/s | 18 kB 00:00 2024-09-10T08:33:59.232 INFO:teuthology.orchestra.run.smithi172.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 2.7 MB/s | 910 kB 00:00 2024-09-10T08:33:59.309 INFO:teuthology.orchestra.run.smithi174.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 4.7 MB/s | 21 MB 00:04 2024-09-10T08:33:59.341 INFO:teuthology.orchestra.run.smithi191.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 563 kB/s | 80 kB 00:00 2024-09-10T08:33:59.357 INFO:teuthology.orchestra.run.smithi172.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.4 MB/s | 4.9 MB 00:01 2024-09-10T08:33:59.382 INFO:teuthology.orchestra.run.smithi172.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 915 kB/s | 137 kB 00:00 2024-09-10T08:33:59.408 INFO:teuthology.orchestra.run.smithi191.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.3 MB/s | 801 kB 00:00 2024-09-10T08:33:59.426 INFO:teuthology.orchestra.run.smithi174.stdout:(9/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9. 6.4 MB/s | 11 MB 00:01 2024-09-10T08:33:59.433 INFO:teuthology.orchestra.run.smithi191.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-10T08:33:59.451 INFO:teuthology.orchestra.run.smithi174.stdout:(10/156): ceph-selinux-19.1.1-305.g1cf819b7.el9 177 kB/s | 25 kB 00:00 2024-09-10T08:33:59.458 INFO:teuthology.orchestra.run.smithi191.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-09-10T08:33:59.500 INFO:teuthology.orchestra.run.smithi191.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-10T08:33:59.509 INFO:teuthology.orchestra.run.smithi174.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 3.8 MB/s | 228 kB 00:00 2024-09-10T08:33:59.524 INFO:teuthology.orchestra.run.smithi172.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-09-10T08:33:59.542 INFO:teuthology.orchestra.run.smithi191.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-09-10T08:33:59.551 INFO:teuthology.orchestra.run.smithi174.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 651 kB/s | 27 kB 00:00 2024-09-10T08:33:59.567 INFO:teuthology.orchestra.run.smithi191.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-10T08:33:59.593 INFO:teuthology.orchestra.run.smithi191.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-10T08:33:59.593 INFO:teuthology.orchestra.run.smithi174.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 763 kB/s | 32 kB 00:00 2024-09-10T08:33:59.618 INFO:teuthology.orchestra.run.smithi191.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-10T08:33:59.629 INFO:teuthology.orchestra.run.smithi172.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 841 kB/s | 87 kB 00:00 2024-09-10T08:33:59.725 INFO:teuthology.orchestra.run.smithi172.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.9 MB/s | 1.0 MB 00:00 2024-09-10T08:33:59.769 INFO:teuthology.orchestra.run.smithi174.stdout:(14/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 6.0 MB/s | 17 MB 00:02 2024-09-10T08:33:59.775 INFO:teuthology.orchestra.run.smithi172.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-10T08:33:59.802 INFO:teuthology.orchestra.run.smithi174.stdout:(15/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 3.5 MB/s | 751 kB 00:00 2024-09-10T08:33:59.827 INFO:teuthology.orchestra.run.smithi174.stdout:(16/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.7 MB/s | 164 kB 00:00 2024-09-10T08:33:59.853 INFO:teuthology.orchestra.run.smithi174.stdout:(17/156): librados-devel-19.1.1-305.g1cf819b7.e 2.5 MB/s | 127 kB 00:00 2024-09-10T08:33:59.884 INFO:teuthology.orchestra.run.smithi172.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-10T08:33:59.928 INFO:teuthology.orchestra.run.smithi174.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 4.9 MB/s | 506 kB 00:00 2024-09-10T08:33:59.944 INFO:teuthology.orchestra.run.smithi191.stdout:(48/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 4.0 MB/s | 17 MB 00:04 2024-09-10T08:33:59.978 INFO:teuthology.orchestra.run.smithi174.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 906 kB/s | 45 kB 00:00 2024-09-10T08:34:00.029 INFO:teuthology.orchestra.run.smithi174.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 2.7 MB/s | 136 kB 00:00 2024-09-10T08:34:00.079 INFO:teuthology.orchestra.run.smithi174.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 3.2 MB/s | 161 kB 00:00 2024-09-10T08:34:00.109 INFO:teuthology.orchestra.run.smithi172.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.1 MB/s | 248 kB 00:00 2024-09-10T08:34:00.146 INFO:teuthology.orchestra.run.smithi174.stdout:(22/156): python3-rados-19.1.1-305.g1cf819b7.el 4.7 MB/s | 320 kB 00:00 2024-09-10T08:34:00.159 INFO:teuthology.orchestra.run.smithi172.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 515 kB/s | 26 kB 00:00 2024-09-10T08:34:00.210 INFO:teuthology.orchestra.run.smithi172.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-10T08:34:00.213 INFO:teuthology.orchestra.run.smithi174.stdout:(23/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 4.4 MB/s | 301 kB 00:00 2024-09-10T08:34:00.263 INFO:teuthology.orchestra.run.smithi174.stdout:(24/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T08:34:00.293 INFO:teuthology.orchestra.run.smithi172.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-10T08:34:00.313 INFO:teuthology.orchestra.run.smithi174.stdout:(25/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-10T08:34:00.344 INFO:teuthology.orchestra.run.smithi172.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-10T08:34:00.377 INFO:teuthology.orchestra.run.smithi191.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 879 kB/s | 689 kB 00:00 2024-09-10T08:34:00.402 INFO:teuthology.orchestra.run.smithi191.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 233 kB/s | 107 kB 00:00 2024-09-10T08:34:00.494 INFO:teuthology.orchestra.run.smithi191.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 186 kB/s | 22 kB 00:00 2024-09-10T08:34:00.528 INFO:teuthology.orchestra.run.smithi191.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-09-10T08:34:00.530 INFO:teuthology.orchestra.run.smithi174.stdout:(26/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 7.9 MB/s | 5.3 MB 00:00 2024-09-10T08:34:00.577 INFO:teuthology.orchestra.run.smithi172.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 4.7 MB/s | 5.8 MB 00:01 2024-09-10T08:34:00.595 INFO:teuthology.orchestra.run.smithi191.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-09-10T08:34:00.597 INFO:teuthology.orchestra.run.smithi174.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.5 MB/s | 170 kB 00:00 2024-09-10T08:34:00.639 INFO:teuthology.orchestra.run.smithi174.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 768 kB/s | 32 kB 00:00 2024-09-10T08:34:00.661 INFO:teuthology.orchestra.run.smithi172.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-10T08:34:00.662 INFO:teuthology.orchestra.run.smithi191.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 1.2 MB/s | 313 kB 00:00 2024-09-10T08:34:00.689 INFO:teuthology.orchestra.run.smithi174.stdout:(29/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.9 MB/s | 146 kB 00:00 2024-09-10T08:34:00.695 INFO:teuthology.orchestra.run.smithi191.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-09-10T08:34:00.761 INFO:teuthology.orchestra.run.smithi172.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-10T08:34:00.762 INFO:teuthology.orchestra.run.smithi191.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-09-10T08:34:00.789 INFO:teuthology.orchestra.run.smithi174.stdout:(30/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 6.5 MB/s | 3.1 MB 00:00 2024-09-10T08:34:00.804 INFO:teuthology.orchestra.run.smithi191.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-10T08:34:00.819 INFO:teuthology.orchestra.run.smithi172.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-10T08:34:00.863 INFO:teuthology.orchestra.run.smithi191.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-09-10T08:34:00.870 INFO:teuthology.orchestra.run.smithi172.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-09-10T08:34:00.954 INFO:teuthology.orchestra.run.smithi172.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-09-10T08:34:01.012 INFO:teuthology.orchestra.run.smithi172.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-10T08:34:01.055 INFO:teuthology.orchestra.run.smithi191.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 3.0 MB/s | 581 kB 00:00 2024-09-10T08:34:01.089 INFO:teuthology.orchestra.run.smithi191.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-09-10T08:34:01.090 INFO:teuthology.orchestra.run.smithi174.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 8.9 MB/s | 3.6 MB 00:00 2024-09-10T08:34:01.095 INFO:teuthology.orchestra.run.smithi172.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-09-10T08:34:01.156 INFO:teuthology.orchestra.run.smithi191.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 244 kB/s | 24 kB 00:00 2024-09-10T08:34:01.157 INFO:teuthology.orchestra.run.smithi174.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 3.7 MB/s | 249 kB 00:00 2024-09-10T08:34:01.171 INFO:teuthology.orchestra.run.smithi172.stdout:(89/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.9 MB/s | 6.0 MB 00:01 2024-09-10T08:34:01.189 INFO:teuthology.orchestra.run.smithi191.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-09-10T08:34:01.198 INFO:teuthology.orchestra.run.smithi174.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 1.2 MB/s | 49 kB 00:00 2024-09-10T08:34:01.205 INFO:teuthology.orchestra.run.smithi172.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 468 kB/s | 51 kB 00:00 2024-09-10T08:34:01.230 INFO:teuthology.orchestra.run.smithi172.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 911 kB/s | 54 kB 00:00 2024-09-10T08:34:01.241 INFO:teuthology.orchestra.run.smithi174.stdout:(34/156): ceph-prometheus-alerts-19.1.1-305.g1c 396 kB/s | 16 kB 00:00 2024-09-10T08:34:01.264 INFO:teuthology.orchestra.run.smithi191.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-10T08:34:01.280 INFO:teuthology.orchestra.run.smithi172.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-09-10T08:34:01.298 INFO:teuthology.orchestra.run.smithi191.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-10T08:34:01.307 INFO:teuthology.orchestra.run.smithi174.stdout:(35/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 4.2 MB/s | 285 kB 00:00 2024-09-10T08:34:01.330 INFO:teuthology.orchestra.run.smithi172.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 889 kB/s | 44 kB 00:00 2024-09-10T08:34:01.348 INFO:teuthology.orchestra.run.smithi191.stdout:(65/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.6 MB/s | 3.0 MB 00:00 2024-09-10T08:34:01.365 INFO:teuthology.orchestra.run.smithi191.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 383 kB/s | 38 kB 00:00 2024-09-10T08:34:01.414 INFO:teuthology.orchestra.run.smithi172.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-10T08:34:01.425 INFO:teuthology.orchestra.run.smithi174.stdout:(36/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 6.4 MB/s | 766 kB 00:00 2024-09-10T08:34:01.465 INFO:teuthology.orchestra.run.smithi172.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-09-10T08:34:01.475 INFO:teuthology.orchestra.run.smithi174.stdout:(37/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 366 kB/s | 18 kB 00:00 2024-09-10T08:34:01.531 INFO:teuthology.orchestra.run.smithi172.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 713 kB/s | 47 kB 00:00 2024-09-10T08:34:01.623 INFO:teuthology.orchestra.run.smithi172.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.2 MB/s | 304 kB 00:00 2024-09-10T08:34:01.625 INFO:teuthology.orchestra.run.smithi174.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 8.8 MB/s | 7.4 MB 00:00 2024-09-10T08:34:01.648 INFO:teuthology.orchestra.run.smithi191.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-09-10T08:34:01.682 INFO:teuthology.orchestra.run.smithi172.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 994 kB/s | 58 kB 00:00 2024-09-10T08:34:01.741 INFO:teuthology.orchestra.run.smithi172.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.6 MB/s | 96 kB 00:00 2024-09-10T08:34:01.791 INFO:teuthology.orchestra.run.smithi172.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T08:34:01.799 INFO:teuthology.orchestra.run.smithi191.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 9.9 MB/s | 4.9 MB 00:00 2024-09-10T08:34:01.841 INFO:teuthology.orchestra.run.smithi191.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 4.6 MB/s | 910 kB 00:00 2024-09-10T08:34:01.841 INFO:teuthology.orchestra.run.smithi172.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-09-10T08:34:01.908 INFO:teuthology.orchestra.run.smithi191.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-10T08:34:01.941 INFO:teuthology.orchestra.run.smithi172.stdout:(102/156): python3-numpy-1.20.1-5.el9.x86_64.rp 3.2 MB/s | 5.1 MB 00:01 2024-09-10T08:34:01.999 INFO:teuthology.orchestra.run.smithi172.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-09-10T08:34:02.017 INFO:teuthology.orchestra.run.smithi172.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-10T08:34:02.039 INFO:teuthology.orchestra.run.smithi191.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 7.7 MB/s | 1.0 MB 00:00 2024-09-10T08:34:02.051 INFO:teuthology.orchestra.run.smithi172.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-10T08:34:02.068 INFO:teuthology.orchestra.run.smithi172.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-09-10T08:34:02.134 INFO:teuthology.orchestra.run.smithi172.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.4 MB/s | 863 kB 00:00 2024-09-10T08:34:02.142 INFO:teuthology.orchestra.run.smithi191.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 850 kB/s | 87 kB 00:00 2024-09-10T08:34:02.176 INFO:teuthology.orchestra.run.smithi172.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 627 kB/s | 26 kB 00:00 2024-09-10T08:34:02.184 INFO:teuthology.orchestra.run.smithi174.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 113 kB/s | 80 kB 00:00 2024-09-10T08:34:02.201 INFO:teuthology.orchestra.run.smithi172.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T08:34:02.235 INFO:teuthology.orchestra.run.smithi172.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-10T08:34:02.294 INFO:teuthology.orchestra.run.smithi172.stdout:(111/156): libarrow-9.0.0-11.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-10T08:34:02.319 INFO:teuthology.orchestra.run.smithi172.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-09-10T08:34:02.353 INFO:teuthology.orchestra.run.smithi172.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-10T08:34:02.378 INFO:teuthology.orchestra.run.smithi172.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T08:34:02.403 INFO:teuthology.orchestra.run.smithi172.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T08:34:02.428 INFO:teuthology.orchestra.run.smithi172.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 569 kB/s | 14 kB 00:00 2024-09-10T08:34:02.454 INFO:teuthology.orchestra.run.smithi172.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-09-10T08:34:02.479 INFO:teuthology.orchestra.run.smithi172.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T08:34:02.500 INFO:teuthology.orchestra.run.smithi174.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 916 kB/s | 801 kB 00:00 2024-09-10T08:34:02.521 INFO:teuthology.orchestra.run.smithi172.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-10T08:34:02.547 INFO:teuthology.orchestra.run.smithi172.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-10T08:34:02.572 INFO:teuthology.orchestra.run.smithi172.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T08:34:02.606 INFO:teuthology.orchestra.run.smithi172.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-10T08:34:02.631 INFO:teuthology.orchestra.run.smithi172.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-10T08:34:02.656 INFO:teuthology.orchestra.run.smithi172.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 706 kB/s | 18 kB 00:00 2024-09-10T08:34:02.681 INFO:teuthology.orchestra.run.smithi172.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 937 kB/s | 23 kB 00:00 2024-09-10T08:34:02.707 INFO:teuthology.orchestra.run.smithi172.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 782 kB/s | 19 kB 00:00 2024-09-10T08:34:02.732 INFO:teuthology.orchestra.run.smithi172.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 791 kB/s | 20 kB 00:00 2024-09-10T08:34:02.742 INFO:teuthology.orchestra.run.smithi191.stdout:(73/156): python3-babel-2.9.1-2.el9.noarch.rpm 9.9 MB/s | 6.0 MB 00:00 2024-09-10T08:34:02.791 INFO:teuthology.orchestra.run.smithi172.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-10T08:34:02.816 INFO:teuthology.orchestra.run.smithi172.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T08:34:02.841 INFO:teuthology.orchestra.run.smithi172.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T08:34:02.851 INFO:teuthology.orchestra.run.smithi191.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 567 kB/s | 61 kB 00:00 2024-09-10T08:34:02.867 INFO:teuthology.orchestra.run.smithi172.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T08:34:02.900 INFO:teuthology.orchestra.run.smithi172.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-10T08:34:02.934 INFO:teuthology.orchestra.run.smithi172.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-10T08:34:02.934 INFO:teuthology.orchestra.run.smithi174.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 255 kB/s | 191 kB 00:00 2024-09-10T08:34:02.959 INFO:teuthology.orchestra.run.smithi172.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-09-10T08:34:02.959 INFO:teuthology.orchestra.run.smithi191.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-10T08:34:02.984 INFO:teuthology.orchestra.run.smithi172.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-10T08:34:03.010 INFO:teuthology.orchestra.run.smithi172.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-10T08:34:03.035 INFO:teuthology.orchestra.run.smithi172.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-10T08:34:03.060 INFO:teuthology.orchestra.run.smithi172.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T08:34:03.076 INFO:teuthology.orchestra.run.smithi191.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-09-10T08:34:03.085 INFO:teuthology.orchestra.run.smithi172.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-10T08:34:03.110 INFO:teuthology.orchestra.run.smithi172.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-10T08:34:03.136 INFO:teuthology.orchestra.run.smithi172.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-10T08:34:03.161 INFO:teuthology.orchestra.run.smithi172.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T08:34:03.186 INFO:teuthology.orchestra.run.smithi172.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T08:34:03.201 INFO:teuthology.orchestra.run.smithi191.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 205 kB/s | 26 kB 00:00 2024-09-10T08:34:03.203 INFO:teuthology.orchestra.run.smithi172.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-10T08:34:03.229 INFO:teuthology.orchestra.run.smithi172.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-09-10T08:34:03.254 INFO:teuthology.orchestra.run.smithi172.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T08:34:03.304 INFO:teuthology.orchestra.run.smithi172.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-10T08:34:03.310 INFO:teuthology.orchestra.run.smithi191.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-09-10T08:34:03.317 INFO:teuthology.orchestra.run.smithi174.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 310 kB/s | 253 kB 00:00 2024-09-10T08:34:03.427 INFO:teuthology.orchestra.run.smithi191.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-10T08:34:03.472 INFO:teuthology.orchestra.run.smithi172.stdout:(148/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.2 MB/s | 16 MB 00:02 2024-09-10T08:34:03.527 INFO:teuthology.orchestra.run.smithi191.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 348 kB/s | 35 kB 00:00 2024-09-10T08:34:03.630 INFO:teuthology.orchestra.run.smithi172.stdout:(149/156): librados2-19.1.1-305.g1cf819b7.el9.x 9.1 MB/s | 3.4 MB 00:00 2024-09-10T08:34:03.739 INFO:teuthology.orchestra.run.smithi172.stdout:(150/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 7.3 MB/s | 3.2 MB 00:00 2024-09-10T08:34:04.014 INFO:teuthology.orchestra.run.smithi172.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 148 kB/s | 41 kB 00:00 2024-09-10T08:34:04.044 INFO:teuthology.orchestra.run.smithi191.stdout:(81/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.9 MB/s | 5.1 MB 00:00 2024-09-10T08:34:04.185 INFO:teuthology.orchestra.run.smithi174.stdout:(43/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 10 MB/s | 49 MB 00:04 2024-09-10T08:34:04.281 INFO:teuthology.orchestra.run.smithi172.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 933 kB/s | 249 kB 00:00 2024-09-10T08:34:04.293 INFO:teuthology.orchestra.run.smithi174.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 924 kB/s | 1.2 MB 00:01 2024-09-10T08:34:04.318 INFO:teuthology.orchestra.run.smithi174.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 106 kB/s | 106 kB 00:00 2024-09-10T08:34:04.394 INFO:teuthology.orchestra.run.smithi191.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 716 kB/s | 250 kB 00:00 2024-09-10T08:34:04.439 INFO:teuthology.orchestra.run.smithi172.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 2.4 MB/s | 2.3 MB 00:00 2024-09-10T08:34:04.565 INFO:teuthology.orchestra.run.smithi172.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 1.9 MB/s | 1.8 MB 00:00 2024-09-10T08:34:04.648 INFO:teuthology.orchestra.run.smithi172.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 3.7 MB/s | 1.4 MB 00:00 2024-09-10T08:34:04.740 INFO:teuthology.orchestra.run.smithi172.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 8.0 MB/s | 2.4 MB 00:00 2024-09-10T08:34:04.745 INFO:teuthology.orchestra.run.smithi191.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 635 kB/s | 222 kB 00:00 2024-09-10T08:34:04.745 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-10T08:34:04.745 INFO:teuthology.orchestra.run.smithi172.stdout:Total 18 MB/s | 226 MB 00:12 2024-09-10T08:34:04.937 INFO:teuthology.orchestra.run.smithi191.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 403 kB/s | 77 kB 00:00 2024-09-10T08:34:04.968 INFO:teuthology.orchestra.run.smithi174.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 173 kB/s | 135 kB 00:00 2024-09-10T08:34:05.002 INFO:teuthology.orchestra.run.smithi174.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 176 kB/s | 125 kB 00:00 2024-09-10T08:34:05.045 INFO:teuthology.orchestra.run.smithi191.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-09-10T08:34:05.077 INFO:teuthology.orchestra.run.smithi174.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 283 kB/s | 215 kB 00:00 2024-09-10T08:34:05.471 INFO:teuthology.orchestra.run.smithi191.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 634 kB/s | 269 kB 00:00 2024-09-10T08:34:05.502 INFO:teuthology.orchestra.run.smithi174.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 252 kB/s | 107 kB 00:00 2024-09-10T08:34:05.595 INFO:teuthology.orchestra.run.smithi174.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 237 kB/s | 22 kB 00:00 2024-09-10T08:34:05.738 INFO:teuthology.orchestra.run.smithi191.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 595 kB/s | 159 kB 00:00 2024-09-10T08:34:05.753 INFO:teuthology.orchestra.run.smithi174.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 879 kB/s | 689 kB 00:00 2024-09-10T08:34:05.862 INFO:teuthology.orchestra.run.smithi174.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 1.1 MB/s | 313 kB 00:00 2024-09-10T08:34:05.879 INFO:teuthology.orchestra.run.smithi174.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 261 kB/s | 33 kB 00:00 2024-09-10T08:34:05.912 INFO:teuthology.orchestra.run.smithi174.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-09-10T08:34:05.971 INFO:teuthology.orchestra.run.smithi174.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 198 kB/s | 18 kB 00:00 2024-09-10T08:34:05.982 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-10T08:34:06.012 INFO:teuthology.orchestra.run.smithi174.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-09-10T08:34:06.080 INFO:teuthology.orchestra.run.smithi174.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-10T08:34:06.121 INFO:teuthology.orchestra.run.smithi174.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-09-10T08:34:06.134 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-10T08:34:06.134 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-10T08:34:06.171 INFO:teuthology.orchestra.run.smithi191.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 643 kB/s | 279 kB 00:00 2024-09-10T08:34:06.280 INFO:teuthology.orchestra.run.smithi191.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 471 kB/s | 51 kB 00:00 2024-09-10T08:34:06.355 INFO:teuthology.orchestra.run.smithi174.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.1 MB/s | 3.0 MB 00:00 2024-09-10T08:34:06.380 INFO:teuthology.orchestra.run.smithi191.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 537 kB/s | 54 kB 00:00 2024-09-10T08:34:06.380 INFO:teuthology.orchestra.run.smithi174.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 2.2 MB/s | 581 kB 00:00 2024-09-10T08:34:06.422 INFO:teuthology.orchestra.run.smithi174.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 1.9 MB/s | 662 kB 00:00 2024-09-10T08:34:06.464 INFO:teuthology.orchestra.run.smithi174.stdout:(62/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 225 kB/s | 24 kB 00:00 2024-09-10T08:34:06.498 INFO:teuthology.orchestra.run.smithi174.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 405 kB/s | 47 kB 00:00 2024-09-10T08:34:06.531 INFO:teuthology.orchestra.run.smithi174.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-10T08:34:06.581 INFO:teuthology.orchestra.run.smithi174.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-10T08:34:06.607 INFO:teuthology.orchestra.run.smithi174.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 353 kB/s | 38 kB 00:00 2024-09-10T08:34:06.815 INFO:teuthology.orchestra.run.smithi174.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-09-10T08:34:06.924 INFO:teuthology.orchestra.run.smithi174.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-10T08:34:06.957 INFO:teuthology.orchestra.run.smithi174.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 6.3 MB/s | 910 kB 00:00 2024-09-10T08:34:07.191 INFO:teuthology.orchestra.run.smithi174.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 515 kB/s | 137 kB 00:00 2024-09-10T08:34:07.381 INFO:teuthology.orchestra.run.smithi191.stdout:(91/156): postgresql-server-13.16-1.el9.x86_64. 1.0 MB/s | 5.8 MB 00:05 2024-09-10T08:34:07.497 INFO:teuthology.orchestra.run.smithi191.stdout:(92/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 737 kB/s | 4.4 MB 00:06 2024-09-10T08:34:07.522 INFO:teuthology.orchestra.run.smithi191.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 294 kB/s | 42 kB 00:00 2024-09-10T08:34:07.598 INFO:teuthology.orchestra.run.smithi191.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-09-10T08:34:07.706 INFO:teuthology.orchestra.run.smithi191.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 331 kB/s | 36 kB 00:00 2024-09-10T08:34:07.815 INFO:teuthology.orchestra.run.smithi191.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 438 kB/s | 47 kB 00:00 2024-09-10T08:34:07.890 INFO:teuthology.orchestra.run.smithi191.stdout:(97/156): qatlib-24.02.0-1.el9.x86_64.rpm 601 kB/s | 221 kB 00:00 2024-09-10T08:34:07.991 INFO:teuthology.orchestra.run.smithi191.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 578 kB/s | 58 kB 00:00 2024-09-10T08:34:08.174 INFO:teuthology.orchestra.run.smithi191.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 523 kB/s | 96 kB 00:00 2024-09-10T08:34:08.266 INFO:teuthology.orchestra.run.smithi191.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 674 kB/s | 304 kB 00:00 2024-09-10T08:34:08.366 INFO:teuthology.orchestra.run.smithi191.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 332 kB/s | 64 kB 00:00 2024-09-10T08:34:08.391 INFO:teuthology.orchestra.run.smithi191.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 358 kB/s | 45 kB 00:00 2024-09-10T08:34:08.417 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-10T08:34:08.417 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-10T08:34:08.574 INFO:teuthology.orchestra.run.smithi174.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 747 kB/s | 1.0 MB 00:01 2024-09-10T08:34:08.758 INFO:teuthology.orchestra.run.smithi174.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 477 kB/s | 87 kB 00:00 2024-09-10T08:34:08.917 INFO:teuthology.orchestra.run.smithi191.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 40 kB/s | 22 kB 00:00 2024-09-10T08:34:09.009 INFO:teuthology.orchestra.run.smithi191.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.9 MB/s | 548 kB 00:00 2024-09-10T08:34:09.051 INFO:teuthology.orchestra.run.smithi191.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 7.2 MB/s | 309 kB 00:00 2024-09-10T08:34:09.076 INFO:teuthology.orchestra.run.smithi191.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 775 kB/s | 19 kB 00:00 2024-09-10T08:34:09.310 INFO:teuthology.orchestra.run.smithi191.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-09-10T08:34:09.343 INFO:teuthology.orchestra.run.smithi191.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 907 kB/s | 863 kB 00:00 2024-09-10T08:34:09.369 INFO:teuthology.orchestra.run.smithi191.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-09-10T08:34:09.423 INFO:teuthology.orchestra.run.smithi191.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2024-09-10T08:34:09.452 INFO:teuthology.orchestra.run.smithi191.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2024-09-10T08:34:09.477 INFO:teuthology.orchestra.run.smithi191.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.8 MB/s | 151 kB 00:00 2024-09-10T08:34:09.536 INFO:teuthology.orchestra.run.smithi191.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-10T08:34:09.561 INFO:teuthology.orchestra.run.smithi191.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T08:34:09.587 INFO:teuthology.orchestra.run.smithi191.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-09-10T08:34:09.612 INFO:teuthology.orchestra.run.smithi191.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-09-10T08:34:09.633 INFO:teuthology.orchestra.run.smithi174.stdout:(73/156): openblas-openmp-0.3.26-2.el9.x86_64.r 1.6 MB/s | 4.9 MB 00:03 2024-09-10T08:34:09.637 INFO:teuthology.orchestra.run.smithi191.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-10T08:34:09.663 INFO:teuthology.orchestra.run.smithi191.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T08:34:09.688 INFO:teuthology.orchestra.run.smithi191.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-10T08:34:09.713 INFO:teuthology.orchestra.run.smithi191.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-10T08:34:09.742 INFO:teuthology.orchestra.run.smithi174.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 566 kB/s | 61 kB 00:00 2024-09-10T08:34:09.747 INFO:teuthology.orchestra.run.smithi191.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-10T08:34:09.789 INFO:teuthology.orchestra.run.smithi191.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-09-10T08:34:09.814 INFO:teuthology.orchestra.run.smithi191.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-10T08:34:09.864 INFO:teuthology.orchestra.run.smithi191.stdout:(124/156): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-09-10T08:34:09.898 INFO:teuthology.orchestra.run.smithi191.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-09-10T08:34:09.923 INFO:teuthology.orchestra.run.smithi191.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 396 kB/s | 23 kB 00:00 2024-09-10T08:34:09.949 INFO:teuthology.orchestra.run.smithi191.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-10T08:34:09.974 INFO:teuthology.orchestra.run.smithi191.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-10T08:34:09.999 INFO:teuthology.orchestra.run.smithi191.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T08:34:10.024 INFO:teuthology.orchestra.run.smithi191.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T08:34:10.049 INFO:teuthology.orchestra.run.smithi191.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T08:34:10.091 INFO:teuthology.orchestra.run.smithi191.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-10T08:34:10.092 INFO:teuthology.orchestra.run.smithi174.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 699 kB/s | 245 kB 00:00 2024-09-10T08:34:10.117 INFO:teuthology.orchestra.run.smithi191.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-10T08:34:10.142 INFO:teuthology.orchestra.run.smithi191.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-10T08:34:10.168 INFO:teuthology.orchestra.run.smithi191.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-10T08:34:10.193 INFO:teuthology.orchestra.run.smithi191.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 559 kB/s | 28 kB 00:00 2024-09-10T08:34:10.218 INFO:teuthology.orchestra.run.smithi191.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-10T08:34:10.243 INFO:teuthology.orchestra.run.smithi191.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-09-10T08:34:10.269 INFO:teuthology.orchestra.run.smithi191.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-10T08:34:10.294 INFO:teuthology.orchestra.run.smithi191.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-10T08:34:10.319 INFO:teuthology.orchestra.run.smithi191.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T08:34:10.353 INFO:teuthology.orchestra.run.smithi191.stdout:(142/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2024-09-10T08:34:10.378 INFO:teuthology.orchestra.run.smithi191.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-10T08:34:10.395 INFO:teuthology.orchestra.run.smithi191.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-10T08:34:10.429 INFO:teuthology.orchestra.run.smithi191.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-10T08:34:10.442 INFO:teuthology.orchestra.run.smithi174.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 709 kB/s | 248 kB 00:00 2024-09-10T08:34:10.454 INFO:teuthology.orchestra.run.smithi191.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-10T08:34:10.529 INFO:teuthology.orchestra.run.smithi191.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-10T08:34:10.551 INFO:teuthology.orchestra.run.smithi174.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-09-10T08:34:10.651 INFO:teuthology.orchestra.run.smithi174.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 192 kB/s | 19 kB 00:00 2024-09-10T08:34:10.788 INFO:teuthology.orchestra.run.smithi191.stdout:(148/156): librados2-19.1.1-305.g1cf819b7.el9.x 10 MB/s | 3.4 MB 00:00 2024-09-10T08:34:10.918 INFO:teuthology.orchestra.run.smithi174.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 645 kB/s | 172 kB 00:00 2024-09-10T08:34:10.988 INFO:teuthology.orchestra.run.smithi191.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-09-10T08:34:11.026 INFO:teuthology.orchestra.run.smithi174.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-09-10T08:34:11.055 INFO:teuthology.orchestra.run.smithi191.stdout:(150/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 6.0 MB/s | 3.2 MB 00:00 2024-09-10T08:34:11.105 INFO:teuthology.orchestra.run.smithi191.stdout:(151/156): NetworkManager-libnm-1.49.90-1.el9.x 15 MB/s | 1.8 MB 00:00 2024-09-10T08:34:11.131 INFO:teuthology.orchestra.run.smithi191.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 542 kB/s | 41 kB 00:00 2024-09-10T08:34:11.164 INFO:teuthology.orchestra.run.smithi191.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 4.2 MB/s | 249 kB 00:00 2024-09-10T08:34:11.240 INFO:teuthology.orchestra.run.smithi191.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-09-10T08:34:11.298 INFO:teuthology.orchestra.run.smithi191.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 8.1 MB/s | 1.4 MB 00:00 2024-09-10T08:34:12.677 INFO:teuthology.orchestra.run.smithi174.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 1.0 MB/s | 5.8 MB 00:05 2024-09-10T08:34:13.019 INFO:teuthology.orchestra.run.smithi174.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 734 kB/s | 250 kB 00:00 2024-09-10T08:34:13.369 INFO:teuthology.orchestra.run.smithi174.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 635 kB/s | 222 kB 00:00 2024-09-10T08:34:13.561 INFO:teuthology.orchestra.run.smithi174.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 402 kB/s | 77 kB 00:00 2024-09-10T08:34:13.677 INFO:teuthology.orchestra.run.smithi174.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 363 kB/s | 42 kB 00:00 2024-09-10T08:34:14.111 INFO:teuthology.orchestra.run.smithi174.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 621 kB/s | 269 kB 00:00 2024-09-10T08:34:14.378 INFO:teuthology.orchestra.run.smithi174.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 596 kB/s | 159 kB 00:00 2024-09-10T08:34:14.820 INFO:teuthology.orchestra.run.smithi174.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 631 kB/s | 279 kB 00:00 2024-09-10T08:34:14.912 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-10T08:34:14.928 INFO:teuthology.orchestra.run.smithi174.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 471 kB/s | 51 kB 00:00 2024-09-10T08:34:15.029 INFO:teuthology.orchestra.run.smithi174.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 537 kB/s | 54 kB 00:00 2024-09-10T08:34:15.044 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T08:34:15.083 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T08:34:15.136 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T08:34:15.169 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T08:34:15.194 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T08:34:15.258 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T08:34:15.587 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T08:34:15.679 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:34:15.801 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:34:15.830 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:34:15.921 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:34:15.955 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T08:34:16.130 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T08:34:16.189 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T08:34:16.217 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:34:16.320 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:34:16.343 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T08:34:16.389 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T08:34:16.418 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T08:34:16.438 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T08:34:16.595 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T08:34:16.762 INFO:teuthology.orchestra.run.smithi174.stdout:(91/156): python3-babel-2.9.1-2.el9.noarch.rpm 763 kB/s | 6.0 MB 00:08 2024-09-10T08:34:16.793 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T08:34:16.862 INFO:teuthology.orchestra.run.smithi174.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 417 kB/s | 42 kB 00:00 2024-09-10T08:34:16.925 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T08:34:16.963 INFO:teuthology.orchestra.run.smithi174.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 442 kB/s | 44 kB 00:00 2024-09-10T08:34:17.046 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:34:17.110 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:34:17.306 INFO:teuthology.orchestra.run.smithi174.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 647 kB/s | 221 kB 00:00 2024-09-10T08:34:17.351 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:34:17.405 INFO:teuthology.orchestra.run.smithi174.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 359 kB/s | 36 kB 00:00 2024-09-10T08:34:17.456 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T08:34:17.497 INFO:teuthology.orchestra.run.smithi174.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 518 kB/s | 47 kB 00:00 2024-09-10T08:34:17.543 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T08:34:17.573 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T08:34:17.607 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T08:34:17.672 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T08:34:17.717 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T08:34:17.758 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T08:34:17.801 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T08:34:17.834 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T08:34:17.861 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T08:34:17.922 INFO:teuthology.orchestra.run.smithi174.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 716 kB/s | 304 kB 00:00 2024-09-10T08:34:17.932 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T08:34:17.989 INFO:teuthology.orchestra.run.smithi174.stdout:(98/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 750 kB/s | 5.1 MB 00:06 2024-09-10T08:34:17.992 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T08:34:18.022 INFO:teuthology.orchestra.run.smithi174.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 577 kB/s | 58 kB 00:00 2024-09-10T08:34:18.023 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T08:34:18.067 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T08:34:18.144 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T08:34:18.181 INFO:teuthology.orchestra.run.smithi174.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 499 kB/s | 96 kB 00:00 2024-09-10T08:34:18.185 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T08:34:18.206 INFO:teuthology.orchestra.run.smithi174.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 347 kB/s | 64 kB 00:00 2024-09-10T08:34:18.229 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T08:34:18.257 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T08:34:18.290 INFO:teuthology.orchestra.run.smithi174.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 412 kB/s | 45 kB 00:00 2024-09-10T08:34:18.315 INFO:teuthology.orchestra.run.smithi174.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-09-10T08:34:18.328 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T08:34:18.372 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T08:34:18.407 INFO:teuthology.orchestra.run.smithi174.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.9 MB/s | 548 kB 00:00 2024-09-10T08:34:18.440 INFO:teuthology.orchestra.run.smithi174.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-10T08:34:18.447 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T08:34:18.466 INFO:teuthology.orchestra.run.smithi174.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-09-10T08:34:18.499 INFO:teuthology.orchestra.run.smithi174.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-09-10T08:34:18.549 INFO:teuthology.orchestra.run.smithi174.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-09-10T08:34:18.575 INFO:teuthology.orchestra.run.smithi174.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T08:34:18.590 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T08:34:18.608 INFO:teuthology.orchestra.run.smithi174.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-10T08:34:18.641 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T08:34:18.650 INFO:teuthology.orchestra.run.smithi174.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-09-10T08:34:18.717 INFO:teuthology.orchestra.run.smithi174.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-10T08:34:18.776 INFO:teuthology.orchestra.run.smithi174.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-10T08:34:18.801 INFO:teuthology.orchestra.run.smithi174.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T08:34:18.827 INFO:teuthology.orchestra.run.smithi174.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T08:34:18.902 INFO:teuthology.orchestra.run.smithi174.stdout:(116/156): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-10T08:34:18.927 INFO:teuthology.orchestra.run.smithi174.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 141 kB/s | 14 kB 00:00 2024-09-10T08:34:18.953 INFO:teuthology.orchestra.run.smithi174.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T08:34:18.978 INFO:teuthology.orchestra.run.smithi174.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-10T08:34:19.003 INFO:teuthology.orchestra.run.smithi174.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-10T08:34:19.028 INFO:teuthology.orchestra.run.smithi174.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T08:34:19.065 INFO:teuthology.orchestra.run.smithi174.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-10T08:34:19.087 INFO:teuthology.orchestra.run.smithi174.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-10T08:34:19.112 INFO:teuthology.orchestra.run.smithi174.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 706 kB/s | 18 kB 00:00 2024-09-10T08:34:19.137 INFO:teuthology.orchestra.run.smithi174.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 930 kB/s | 23 kB 00:00 2024-09-10T08:34:19.177 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T08:34:19.188 INFO:teuthology.orchestra.run.smithi174.stdout:(126/156): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-09-10T08:34:19.213 INFO:teuthology.orchestra.run.smithi174.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-09-10T08:34:19.238 INFO:teuthology.orchestra.run.smithi174.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-10T08:34:19.260 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T08:34:19.263 INFO:teuthology.orchestra.run.smithi174.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T08:34:19.289 INFO:teuthology.orchestra.run.smithi174.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T08:34:19.290 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T08:34:19.314 INFO:teuthology.orchestra.run.smithi174.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T08:34:19.333 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T08:34:19.347 INFO:teuthology.orchestra.run.smithi174.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-09-10T08:34:19.381 INFO:teuthology.orchestra.run.smithi174.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 495 kB/s | 16 kB 00:00 2024-09-10T08:34:19.387 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T08:34:19.406 INFO:teuthology.orchestra.run.smithi174.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-10T08:34:19.430 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T08:34:19.440 INFO:teuthology.orchestra.run.smithi174.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-10T08:34:19.465 INFO:teuthology.orchestra.run.smithi174.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 484 kB/s | 28 kB 00:00 2024-09-10T08:34:19.491 INFO:teuthology.orchestra.run.smithi174.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-10T08:34:19.516 INFO:teuthology.orchestra.run.smithi174.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T08:34:19.541 INFO:teuthology.orchestra.run.smithi174.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-10T08:34:19.566 INFO:teuthology.orchestra.run.smithi174.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-10T08:34:19.591 INFO:teuthology.orchestra.run.smithi174.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-10T08:34:19.609 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T08:34:19.617 INFO:teuthology.orchestra.run.smithi174.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T08:34:19.642 INFO:teuthology.orchestra.run.smithi174.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T08:34:19.665 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T08:34:19.667 INFO:teuthology.orchestra.run.smithi174.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 796 kB/s | 20 kB 00:00 2024-09-10T08:34:19.693 INFO:teuthology.orchestra.run.smithi174.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-10T08:34:19.720 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T08:34:19.726 INFO:teuthology.orchestra.run.smithi174.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-10T08:34:19.785 INFO:teuthology.orchestra.run.smithi174.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-10T08:34:19.792 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T08:34:19.824 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T08:34:19.860 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T08:34:19.907 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T08:34:19.943 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T08:34:19.987 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T08:34:20.029 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T08:34:20.135 INFO:teuthology.orchestra.run.smithi174.stdout:(148/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 9.0 MB/s | 3.2 MB 00:00 2024-09-10T08:34:20.165 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T08:34:20.210 INFO:teuthology.orchestra.run.smithi174.stdout:(149/156): librados2-19.1.1-305.g1cf819b7.el9.x 7.1 MB/s | 3.4 MB 00:00 2024-09-10T08:34:20.221 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T08:34:20.263 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T08:34:20.365 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T08:34:20.397 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T08:34:20.437 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T08:34:20.462 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T08:34:20.486 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T08:34:20.588 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T08:34:20.921 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T08:34:20.969 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T08:34:21.005 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T08:34:21.042 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T08:34:21.079 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T08:34:21.194 INFO:teuthology.orchestra.run.smithi174.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 1.8 MB/s | 1.8 MB 00:00 2024-09-10T08:34:21.237 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T08:34:21.252 INFO:teuthology.orchestra.run.smithi174.stdout:(151/156): NetworkManager-1.49.90-1.el9.x86_64. 2.1 MB/s | 2.3 MB 00:01 2024-09-10T08:34:21.399 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T08:34:21.525 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T08:34:21.559 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T08:34:21.601 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T08:34:21.634 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T08:34:21.725 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T08:34:21.761 INFO:teuthology.orchestra.run.smithi174.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 72 kB/s | 41 kB 00:00 2024-09-10T08:34:21.773 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T08:34:21.804 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T08:34:21.908 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T08:34:21.963 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T08:34:22.086 INFO:teuthology.orchestra.run.smithi174.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 298 kB/s | 249 kB 00:00 2024-09-10T08:34:22.688 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T08:34:22.738 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T08:34:22.772 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T08:34:22.812 INFO:teuthology.orchestra.run.smithi174.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 1.3 MB/s | 1.4 MB 00:01 2024-09-10T08:34:22.872 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T08:34:22.918 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T08:34:22.963 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T08:34:23.059 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T08:34:23.104 INFO:teuthology.orchestra.run.smithi174.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.4 MB/s | 2.4 MB 00:01 2024-09-10T08:34:23.107 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T08:34:23.292 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T08:34:23.321 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T08:34:23.399 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T08:34:24.050 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T08:34:24.055 INFO:teuthology.orchestra.run.smithi191.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 942 kB/s | 16 MB 00:17 2024-09-10T08:34:24.060 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-09-10T08:34:24.061 INFO:teuthology.orchestra.run.smithi191.stdout:Total 7.5 MB/s | 226 MB 00:30 2024-09-10T08:34:24.162 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T08:34:25.234 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-09-10T08:34:25.385 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-09-10T08:34:25.385 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-09-10T08:34:25.429 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T08:34:25.525 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T08:34:25.552 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T08:34:25.585 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T08:34:25.894 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T08:34:25.940 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T08:34:26.039 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T08:34:26.070 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T08:34:26.106 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T08:34:26.590 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T08:34:26.618 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T08:34:26.689 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T08:34:26.761 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T08:34:27.598 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-09-10T08:34:27.599 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-09-10T08:34:29.055 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:34:29.092 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:34:29.638 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:34:29.680 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T08:34:29.773 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T08:34:29.773 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T08:34:29.773 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T08:34:29.774 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:34:29.813 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T08:34:33.917 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-09-10T08:34:34.022 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T08:34:34.089 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T08:34:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T08:34:34.176 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T08:34:34.201 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T08:34:34.256 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T08:34:34.586 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T08:34:34.694 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:34:34.807 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:34:34.837 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:34:34.928 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:34:34.965 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T08:34:35.154 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T08:34:35.204 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T08:34:35.232 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:34:35.328 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:34:35.350 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T08:34:35.418 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T08:34:35.451 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T08:34:35.471 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T08:34:35.672 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T08:34:35.840 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T08:34:35.983 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T08:34:36.093 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:34:36.160 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:34:36.194 INFO:teuthology.orchestra.run.smithi174.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 786 kB/s | 16 MB 00:21 2024-09-10T08:34:36.199 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-09-10T08:34:36.200 INFO:teuthology.orchestra.run.smithi174.stdout:Total 5.4 MB/s | 226 MB 00:42 2024-09-10T08:34:36.363 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:34:36.473 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T08:34:36.567 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T08:34:36.607 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T08:34:36.641 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T08:34:36.705 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T08:34:36.750 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T08:34:36.791 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T08:34:36.835 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T08:34:36.867 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T08:34:36.895 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T08:34:36.963 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T08:34:37.016 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T08:34:37.057 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T08:34:37.100 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T08:34:37.175 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T08:34:37.210 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T08:34:37.254 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T08:34:37.282 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T08:34:37.351 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T08:34:37.392 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-09-10T08:34:37.405 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T08:34:37.479 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T08:34:37.542 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-09-10T08:34:37.542 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-09-10T08:34:37.613 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T08:34:37.666 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T08:34:38.190 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T08:34:38.270 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T08:34:38.308 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T08:34:38.359 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T08:34:38.405 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T08:34:38.447 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T08:34:38.609 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T08:34:38.666 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T08:34:38.720 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T08:34:38.784 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T08:34:38.817 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T08:34:38.853 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T08:34:38.899 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T08:34:38.945 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T08:34:38.988 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T08:34:39.029 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T08:34:39.165 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T08:34:39.206 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T08:34:39.247 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T08:34:39.350 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T08:34:39.382 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T08:34:39.447 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T08:34:39.489 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T08:34:39.515 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T08:34:39.625 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T08:34:39.732 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-09-10T08:34:39.732 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-09-10T08:34:39.756 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T08:34:39.787 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T08:34:39.816 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T08:34:39.860 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T08:34:39.897 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T08:34:39.929 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T08:34:39.959 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T08:34:39.993 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T08:34:40.027 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T08:34:40.076 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T08:34:40.111 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T08:34:40.197 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T08:34:40.241 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T08:34:40.273 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T08:34:40.374 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T08:34:40.423 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T08:34:41.144 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T08:34:41.190 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T08:34:41.224 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T08:34:41.321 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T08:34:41.379 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T08:34:41.424 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T08:34:41.511 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T08:34:41.560 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T08:34:41.752 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T08:34:41.783 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T08:34:41.861 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T08:34:41.953 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T08:34:41.953 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /sys 2024-09-10T08:34:41.953 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /proc 2024-09-10T08:34:41.953 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /mnt 2024-09-10T08:34:41.953 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /var/tmp 2024-09-10T08:34:41.953 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /home 2024-09-10T08:34:41.953 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /root 2024-09-10T08:34:41.953 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /tmp 2024-09-10T08:34:41.953 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:34:42.141 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T08:34:42.157 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T08:34:42.157 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:34:42.157 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T08:34:42.157 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:34:42.502 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T08:34:42.623 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T08:34:43.660 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T08:34:43.687 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T08:34:43.687 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:34:43.687 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T08:34:43.687 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T08:34:43.687 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T08:34:43.688 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:34:43.849 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T08:34:43.919 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T08:34:43.939 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T08:34:43.942 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T08:34:43.942 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:34:43.943 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T08:34:43.943 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T08:34:43.943 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T08:34:43.943 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:34:43.966 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T08:34:44.008 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T08:34:44.332 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T08:34:44.380 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T08:34:44.386 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T08:34:44.410 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T08:34:44.410 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:34:44.410 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T08:34:44.410 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T08:34:44.410 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T08:34:44.410 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:34:44.469 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T08:34:44.501 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T08:34:44.516 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T08:34:44.546 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T08:34:44.564 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T08:34:44.620 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T08:34:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T08:34:44.853 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T08:34:44.922 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T08:34:45.011 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T08:34:45.166 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T08:34:45.212 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T08:34:45.221 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T08:34:45.249 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T08:34:45.260 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T08:34:45.288 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T08:34:45.320 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T08:34:45.351 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T08:34:45.750 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T08:34:45.778 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T08:34:45.810 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T08:34:45.872 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T08:34:45.901 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T08:34:46.002 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-09-10T08:34:46.098 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T08:34:46.144 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T08:34:46.198 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T08:34:46.240 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T08:34:46.265 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T08:34:46.336 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T08:34:46.665 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T08:34:46.774 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:34:46.896 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:34:46.934 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:34:46.940 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T08:34:46.969 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T08:34:47.025 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:34:47.079 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T08:34:47.261 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T08:34:47.309 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T08:34:47.346 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:34:47.458 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:34:47.487 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T08:34:47.559 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T08:34:47.596 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T08:34:47.622 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T08:34:47.657 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:34:47.697 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:34:47.816 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T08:34:47.932 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T08:34:47.953 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T08:34:47.984 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T08:34:48.056 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T08:34:48.116 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T08:34:48.225 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:34:48.234 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T08:34:48.247 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:34:48.263 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T08:34:48.288 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T08:34:48.288 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:34:48.288 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T08:34:48.288 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T08:34:48.288 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T08:34:48.289 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:34:48.292 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T08:34:48.320 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:34:48.362 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T08:34:48.374 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T08:34:48.397 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T08:34:48.397 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T08:34:48.398 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T08:34:48.398 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:34:48.435 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T08:34:48.473 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T08:34:48.530 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T08:34:48.542 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T08:34:48.649 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T08:34:48.745 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T08:34:48.775 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T08:34:48.809 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T08:34:48.882 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T08:34:48.935 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T08:34:48.968 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T08:34:49.011 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T08:34:49.044 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T08:34:49.072 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T08:34:49.139 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T08:34:49.193 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T08:34:49.234 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T08:34:49.285 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T08:34:49.360 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T08:34:49.395 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T08:34:49.439 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T08:34:49.459 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T08:34:49.467 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T08:34:49.500 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T08:34:49.524 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T08:34:49.524 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:34:49.524 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T08:34:49.524 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T08:34:49.524 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T08:34:49.525 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:34:49.546 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T08:34:49.567 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T08:34:49.591 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T08:34:49.591 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:34:49.591 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T08:34:49.591 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:34:49.591 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T08:34:49.634 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T08:34:49.658 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T08:34:49.658 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:34:49.658 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T08:34:49.658 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T08:34:49.658 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T08:34:49.658 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:34:49.665 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T08:34:49.806 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T08:34:49.867 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T08:34:49.968 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T08:34:49.993 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T08:34:49.993 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:34:49.993 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T08:34:49.993 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T08:34:49.993 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T08:34:49.993 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:34:50.388 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T08:34:50.471 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T08:34:50.509 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T08:34:50.560 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T08:34:50.598 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T08:34:50.641 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T08:34:50.809 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T08:34:50.868 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T08:34:50.923 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T08:34:50.996 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T08:34:51.027 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T08:34:51.071 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T08:34:51.126 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T08:34:51.162 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T08:34:51.205 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T08:34:51.247 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T08:34:51.381 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T08:34:51.441 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T08:34:51.482 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T08:34:51.583 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T08:34:51.625 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T08:34:51.689 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T08:34:51.714 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T08:34:51.739 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T08:34:51.843 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T08:34:51.972 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T08:34:52.013 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T08:34:52.041 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T08:34:52.088 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T08:34:52.131 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T08:34:52.163 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T08:34:52.192 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T08:34:52.227 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T08:34:52.260 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T08:34:52.302 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T08:34:52.336 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T08:34:52.422 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T08:34:52.475 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T08:34:52.514 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T08:34:52.616 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T08:34:52.673 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T08:34:53.392 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T08:34:53.432 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T08:34:53.466 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T08:34:53.566 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T08:34:53.628 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T08:34:53.682 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T08:34:53.769 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T08:34:53.801 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T08:34:53.984 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T08:34:54.015 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T08:34:54.096 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T08:34:54.564 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T08:34:54.605 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T08:34:54.646 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T08:34:54.678 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T08:34:54.705 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T08:34:54.742 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T08:34:54.855 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T08:34:54.912 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T08:34:54.960 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T08:34:54.990 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T08:34:55.037 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T08:34:55.083 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T08:34:55.719 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:34:55.756 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:34:55.776 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:34:56.105 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T08:34:56.193 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T08:34:56.220 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T08:34:56.256 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T08:34:56.566 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T08:34:56.608 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T08:34:56.639 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T08:34:56.699 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T08:34:56.727 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T08:34:56.730 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T08:34:56.766 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T08:34:56.773 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T08:34:56.859 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T08:34:56.888 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T08:34:56.913 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T08:34:56.940 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T08:34:56.940 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T08:34:57.031 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T08:34:57.031 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T08:34:57.124 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T08:34:57.149 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T08:34:57.154 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:34:57.155 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:34:57.245 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T08:34:57.278 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T08:34:57.343 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:34:57.343 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T08:34:57.349 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T08:34:57.371 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T08:34:57.427 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T08:34:57.491 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:34:59.617 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:34:59.651 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:35:00.195 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T08:35:00.237 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T08:35:00.329 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T08:35:00.330 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T08:35:00.330 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T08:35:00.330 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:35:00.372 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T08:35:00.472 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T08:35:00.472 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /sys 2024-09-10T08:35:00.472 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /proc 2024-09-10T08:35:00.472 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /mnt 2024-09-10T08:35:00.472 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /var/tmp 2024-09-10T08:35:00.472 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /home 2024-09-10T08:35:00.472 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /root 2024-09-10T08:35:00.472 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /tmp 2024-09-10T08:35:00.472 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:35:00.646 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T08:35:00.661 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T08:35:00.661 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:00.661 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T08:35:00.661 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:35:02.140 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T08:35:02.166 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T08:35:02.166 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:02.167 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T08:35:02.167 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T08:35:02.167 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T08:35:02.167 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:35:02.391 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T08:35:02.414 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T08:35:02.414 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:02.414 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T08:35:02.414 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T08:35:02.414 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T08:35:02.414 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:35:02.850 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T08:35:02.873 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T08:35:02.873 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:02.873 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T08:35:02.873 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T08:35:02.873 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T08:35:02.873 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:35:02.943 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:35:02.943 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T08:35:02.943 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T08:35:02.943 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T08:35:02.943 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T08:35:02.943 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T08:35:02.943 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T08:35:02.943 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T08:35:02.943 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:35:02.944 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:35:02.944 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T08:35:02.944 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T08:35:02.944 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T08:35:02.944 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:35:02.944 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T08:35:02.944 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T08:35:02.944 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T08:35:02.944 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T08:35:02.944 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T08:35:02.944 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T08:35:02.945 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T08:35:02.945 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T08:35:02.945 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T08:35:02.945 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T08:35:02.945 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T08:35:02.945 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T08:35:02.945 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T08:35:02.945 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T08:35:02.945 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T08:35:02.945 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T08:35:02.957 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T08:35:02.957 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T08:35:02.958 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T08:35:02.958 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T08:35:02.958 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T08:35:02.958 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T08:35:02.958 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T08:35:02.958 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T08:35:02.958 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T08:35:02.958 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T08:35:02.958 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T08:35:02.958 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T08:35:02.958 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T08:35:02.958 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T08:35:02.959 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T08:35:02.959 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T08:35:02.959 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T08:35:02.959 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T08:35:02.959 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T08:35:02.959 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T08:35:02.959 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T08:35:02.959 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T08:35:02.959 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T08:35:02.959 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T08:35:02.959 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T08:35:02.959 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T08:35:02.960 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T08:35:02.960 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T08:35:02.960 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T08:35:02.960 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T08:35:02.960 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T08:35:02.960 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T08:35:02.960 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T08:35:02.960 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T08:35:02.960 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T08:35:02.960 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T08:35:02.960 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T08:35:02.960 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T08:35:02.961 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T08:35:02.961 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T08:35:02.961 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T08:35:02.961 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T08:35:02.961 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T08:35:02.961 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T08:35:02.961 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T08:35:02.961 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T08:35:02.961 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T08:35:02.961 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T08:35:02.961 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T08:35:02.961 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T08:35:02.962 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T08:35:02.962 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T08:35:02.962 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T08:35:02.962 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T08:35:02.962 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T08:35:02.962 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T08:35:02.962 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T08:35:02.962 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T08:35:02.962 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T08:35:02.962 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T08:35:02.962 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T08:35:02.962 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T08:35:02.962 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T08:35:02.963 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T08:35:02.963 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T08:35:02.963 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T08:35:02.963 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T08:35:02.963 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T08:35:02.963 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T08:35:02.963 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T08:35:02.963 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T08:35:02.963 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T08:35:02.963 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T08:35:02.963 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T08:35:02.963 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T08:35:02.963 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T08:35:02.963 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T08:35:02.964 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T08:35:02.964 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T08:35:02.964 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T08:35:02.964 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T08:35:02.964 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T08:35:02.964 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T08:35:02.964 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T08:35:02.964 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T08:35:02.964 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T08:35:02.964 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T08:35:02.964 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T08:35:02.964 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T08:35:02.964 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T08:35:02.965 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T08:35:02.965 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T08:35:02.965 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T08:35:02.965 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T08:35:02.965 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T08:35:02.965 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T08:35:02.965 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T08:35:02.965 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T08:35:02.965 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T08:35:02.965 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T08:35:02.965 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T08:35:02.965 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T08:35:02.965 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T08:35:02.966 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T08:35:02.966 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T08:35:02.966 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T08:35:02.966 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T08:35:02.966 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T08:35:02.966 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T08:35:02.966 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T08:35:02.966 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T08:35:02.966 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T08:35:02.966 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T08:35:02.966 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T08:35:02.966 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T08:35:02.966 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T08:35:02.967 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T08:35:02.967 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T08:35:02.967 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T08:35:02.967 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T08:35:02.967 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T08:35:02.967 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T08:35:02.967 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T08:35:02.967 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T08:35:02.967 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T08:35:02.967 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T08:35:02.967 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T08:35:02.967 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T08:35:02.967 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T08:35:02.967 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T08:35:02.968 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T08:35:02.968 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T08:35:02.968 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T08:35:02.968 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T08:35:02.980 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T08:35:03.028 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T08:35:03.085 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T08:35:03.268 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T08:35:03.317 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T08:35:03.394 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T08:35:03.492 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T08:35:03.630 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T08:35:03.676 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T08:35:03.724 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T08:35:03.752 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T08:35:04.114 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T08:35:04.151 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T08:35:04.196 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T08:35:04.252 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T08:35:04.282 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T08:35:05.322 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T08:35:05.367 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T08:35:05.486 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:35:05.486 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:35:05.486 INFO:teuthology.orchestra.run.smithi172.stdout:Upgraded: 2024-09-10T08:35:05.487 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T08:35:05.487 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T08:35:05.487 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T08:35:05.487 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T08:35:05.487 INFO:teuthology.orchestra.run.smithi172.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.487 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.487 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T08:35:05.487 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T08:35:05.487 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-10T08:35:05.487 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T08:35:05.487 INFO:teuthology.orchestra.run.smithi172.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T08:35:05.487 INFO:teuthology.orchestra.run.smithi172.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T08:35:05.488 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T08:35:05.488 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.488 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.488 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.488 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.488 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:05.488 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.488 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.488 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.488 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:05.488 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:05.489 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:05.489 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:05.489 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:05.489 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.489 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.489 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:05.489 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.489 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.489 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.489 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:05.489 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:05.489 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.489 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.490 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:05.490 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T08:35:05.490 INFO:teuthology.orchestra.run.smithi172.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T08:35:05.490 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T08:35:05.490 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T08:35:05.490 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T08:35:05.490 INFO:teuthology.orchestra.run.smithi172.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T08:35:05.490 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T08:35:05.490 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T08:35:05.490 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T08:35:05.490 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T08:35:05.490 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T08:35:05.491 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.491 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.491 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.491 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T08:35:05.491 INFO:teuthology.orchestra.run.smithi172.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T08:35:05.491 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T08:35:05.491 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T08:35:05.491 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T08:35:05.491 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.491 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.491 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T08:35:05.492 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.492 INFO:teuthology.orchestra.run.smithi172.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T08:35:05.492 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T08:35:05.492 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T08:35:05.492 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T08:35:05.492 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T08:35:05.492 INFO:teuthology.orchestra.run.smithi172.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T08:35:05.492 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T08:35:05.492 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T08:35:05.492 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T08:35:05.492 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T08:35:05.492 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T08:35:05.493 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T08:35:05.493 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T08:35:05.493 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T08:35:05.493 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T08:35:05.493 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T08:35:05.493 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T08:35:05.493 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T08:35:05.493 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T08:35:05.493 INFO:teuthology.orchestra.run.smithi172.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T08:35:05.493 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T08:35:05.494 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T08:35:05.494 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T08:35:05.494 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.494 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.494 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.494 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T08:35:05.494 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T08:35:05.494 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T08:35:05.494 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T08:35:05.494 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T08:35:05.494 INFO:teuthology.orchestra.run.smithi172.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T08:35:05.494 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T08:35:05.494 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T08:35:05.495 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T08:35:05.495 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T08:35:05.495 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T08:35:05.495 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T08:35:05.495 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T08:35:05.495 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T08:35:05.495 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T08:35:05.495 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T08:35:05.495 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T08:35:05.495 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T08:35:05.495 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T08:35:05.495 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T08:35:05.495 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T08:35:05.496 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T08:35:05.496 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T08:35:05.496 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T08:35:05.496 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T08:35:05.496 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T08:35:05.496 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T08:35:05.496 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T08:35:05.496 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T08:35:05.496 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T08:35:05.496 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T08:35:05.496 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T08:35:05.496 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T08:35:05.497 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T08:35:05.497 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T08:35:05.497 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T08:35:05.497 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T08:35:05.497 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T08:35:05.497 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T08:35:05.497 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T08:35:05.497 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.497 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.497 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T08:35:05.497 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T08:35:05.497 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T08:35:05.497 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.498 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T08:35:05.498 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T08:35:05.498 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T08:35:05.498 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T08:35:05.498 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T08:35:05.498 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T08:35:05.498 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T08:35:05.498 INFO:teuthology.orchestra.run.smithi172.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T08:35:05.498 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T08:35:05.498 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T08:35:05.498 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T08:35:05.498 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T08:35:05.499 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T08:35:05.499 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T08:35:05.499 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T08:35:05.499 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.499 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.499 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:05.499 INFO:teuthology.orchestra.run.smithi172.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T08:35:05.499 INFO:teuthology.orchestra.run.smithi172.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T08:35:05.499 INFO:teuthology.orchestra.run.smithi172.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T08:35:05.499 INFO:teuthology.orchestra.run.smithi172.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T08:35:05.500 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T08:35:05.500 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T08:35:05.500 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T08:35:05.500 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:35:05.500 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-10T08:35:05.921 DEBUG:teuthology.parallel:result is None 2024-09-10T08:35:06.329 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T08:35:06.359 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T08:35:06.461 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T08:35:06.622 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T08:35:06.653 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T08:35:06.677 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T08:35:06.677 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:06.677 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T08:35:06.677 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T08:35:06.677 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T08:35:06.677 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:35:06.729 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T08:35:06.740 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T08:35:06.838 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T08:35:06.888 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T08:35:07.814 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T08:35:07.848 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T08:35:07.870 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T08:35:07.871 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:07.871 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T08:35:07.871 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T08:35:07.871 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T08:35:07.871 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:35:07.915 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T08:35:07.938 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T08:35:07.938 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:07.938 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T08:35:07.938 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:35:07.991 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T08:35:08.014 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T08:35:08.014 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:08.014 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T08:35:08.014 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T08:35:08.014 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T08:35:08.014 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:35:08.325 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T08:35:08.348 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T08:35:08.348 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:08.348 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T08:35:08.348 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T08:35:08.348 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T08:35:08.348 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:35:12.320 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T08:35:12.321 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /sys 2024-09-10T08:35:12.321 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /proc 2024-09-10T08:35:12.321 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /mnt 2024-09-10T08:35:12.321 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /var/tmp 2024-09-10T08:35:12.321 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /home 2024-09-10T08:35:12.321 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /root 2024-09-10T08:35:12.321 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /tmp 2024-09-10T08:35:12.321 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:35:12.518 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T08:35:12.533 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T08:35:12.533 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:12.533 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T08:35:12.533 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:35:12.928 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T08:35:12.970 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T08:35:13.010 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T08:35:13.059 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T08:35:13.086 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T08:35:13.275 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T08:35:13.325 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T08:35:13.355 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T08:35:13.385 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T08:35:13.423 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T08:35:14.001 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T08:35:14.006 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:35:14.027 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T08:35:14.027 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:14.027 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T08:35:14.027 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T08:35:14.027 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T08:35:14.027 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:35:14.062 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:35:14.082 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:35:14.251 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T08:35:14.275 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T08:35:14.275 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:14.275 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T08:35:14.275 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T08:35:14.275 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T08:35:14.275 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:35:14.711 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T08:35:14.733 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T08:35:14.734 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:14.734 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T08:35:14.734 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T08:35:14.734 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T08:35:14.734 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:35:14.832 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T08:35:14.880 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T08:35:14.919 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T08:35:14.936 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T08:35:15.009 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T08:35:15.058 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T08:35:15.130 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T08:35:15.157 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T08:35:15.186 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T08:35:15.186 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T08:35:15.220 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T08:35:15.255 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T08:35:15.255 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T08:35:15.265 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T08:35:15.330 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T08:35:15.331 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T08:35:15.353 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T08:35:15.406 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T08:35:15.439 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T08:35:15.444 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:35:15.445 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:35:15.485 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T08:35:15.529 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T08:35:15.568 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T08:35:15.595 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T08:35:15.633 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:35:15.634 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T08:35:15.709 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T08:35:15.776 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:35:16.033 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T08:35:16.060 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T08:35:16.100 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T08:35:16.172 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T08:35:16.208 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T08:35:17.246 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T08:35:17.277 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T08:35:18.233 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T08:35:18.277 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T08:35:18.378 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T08:35:18.538 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T08:35:18.571 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T08:35:18.594 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T08:35:18.594 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:18.594 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T08:35:18.595 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T08:35:18.595 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T08:35:18.595 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:35:18.646 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T08:35:18.657 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T08:35:18.756 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T08:35:18.810 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T08:35:19.749 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T08:35:19.774 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T08:35:19.797 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T08:35:19.798 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:19.798 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T08:35:19.798 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T08:35:19.798 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T08:35:19.798 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:35:19.841 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T08:35:19.864 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T08:35:19.864 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:19.864 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T08:35:19.864 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:35:19.908 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T08:35:19.931 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T08:35:19.931 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:19.931 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T08:35:19.931 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T08:35:19.931 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T08:35:19.932 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:35:20.242 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T08:35:20.265 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T08:35:20.266 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T08:35:20.266 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T08:35:20.266 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T08:35:20.266 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T08:35:20.266 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:35:22.075 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:35:22.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T08:35:22.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T08:35:22.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T08:35:22.075 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T08:35:22.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T08:35:22.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T08:35:22.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T08:35:22.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:35:22.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:35:22.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T08:35:22.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T08:35:22.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T08:35:22.076 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:35:22.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T08:35:22.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T08:35:22.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T08:35:22.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T08:35:22.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T08:35:22.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T08:35:22.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T08:35:22.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T08:35:22.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T08:35:22.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T08:35:22.077 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T08:35:22.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T08:35:22.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T08:35:22.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T08:35:22.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T08:35:22.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T08:35:22.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T08:35:22.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T08:35:22.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T08:35:22.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T08:35:22.078 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T08:35:22.079 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T08:35:22.079 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T08:35:22.079 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T08:35:22.079 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T08:35:22.079 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T08:35:22.079 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T08:35:22.079 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T08:35:22.079 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T08:35:22.079 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T08:35:22.080 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T08:35:22.080 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T08:35:22.080 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T08:35:22.080 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T08:35:22.080 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T08:35:22.080 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T08:35:22.080 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T08:35:22.080 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T08:35:22.080 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T08:35:22.080 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T08:35:22.081 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T08:35:22.081 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T08:35:22.081 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T08:35:22.081 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T08:35:22.081 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T08:35:22.081 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T08:35:22.081 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T08:35:22.081 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T08:35:22.081 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T08:35:22.081 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T08:35:22.081 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T08:35:22.082 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T08:35:22.082 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T08:35:22.082 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T08:35:22.082 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T08:35:22.082 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T08:35:22.082 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T08:35:22.082 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T08:35:22.082 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T08:35:22.082 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T08:35:22.082 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T08:35:22.083 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T08:35:22.083 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T08:35:22.083 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T08:35:22.083 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T08:35:22.083 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T08:35:22.083 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T08:35:22.083 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T08:35:22.083 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T08:35:22.083 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T08:35:22.083 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T08:35:22.083 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T08:35:22.084 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T08:35:22.084 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T08:35:22.084 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T08:35:22.084 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T08:35:22.084 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T08:35:22.084 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T08:35:22.084 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T08:35:22.084 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T08:35:22.084 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T08:35:22.084 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T08:35:22.084 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T08:35:22.084 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T08:35:22.085 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T08:35:22.085 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T08:35:22.085 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T08:35:22.085 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T08:35:22.085 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T08:35:22.085 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T08:35:22.085 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T08:35:22.085 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T08:35:22.085 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T08:35:22.085 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T08:35:22.085 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T08:35:22.086 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T08:35:22.086 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T08:35:22.086 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T08:35:22.086 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T08:35:22.086 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T08:35:22.086 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T08:35:22.086 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T08:35:22.086 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T08:35:22.086 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T08:35:22.086 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T08:35:22.086 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T08:35:22.087 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T08:35:22.087 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T08:35:22.087 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T08:35:22.087 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T08:35:22.087 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T08:35:22.087 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T08:35:22.087 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T08:35:22.087 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T08:35:22.087 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T08:35:22.087 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T08:35:22.087 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T08:35:22.088 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T08:35:22.088 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T08:35:22.088 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T08:35:22.088 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T08:35:22.088 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T08:35:22.088 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T08:35:22.088 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T08:35:22.088 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T08:35:22.088 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T08:35:22.088 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T08:35:22.089 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T08:35:22.089 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T08:35:22.089 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T08:35:22.089 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T08:35:22.089 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T08:35:22.089 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T08:35:22.089 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T08:35:22.089 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T08:35:22.089 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T08:35:22.089 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T08:35:22.090 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T08:35:22.090 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T08:35:22.090 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T08:35:22.090 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T08:35:22.090 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T08:35:22.090 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T08:35:22.090 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T08:35:22.090 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T08:35:22.090 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T08:35:22.090 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T08:35:22.091 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T08:35:22.091 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T08:35:22.091 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T08:35:24.587 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:35:24.588 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:35:24.588 INFO:teuthology.orchestra.run.smithi191.stdout:Upgraded: 2024-09-10T08:35:24.588 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T08:35:24.588 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T08:35:24.588 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T08:35:24.588 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T08:35:24.588 INFO:teuthology.orchestra.run.smithi191.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.588 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.588 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T08:35:24.588 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T08:35:24.588 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-09-10T08:35:24.589 INFO:teuthology.orchestra.run.smithi191.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T08:35:24.589 INFO:teuthology.orchestra.run.smithi191.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T08:35:24.589 INFO:teuthology.orchestra.run.smithi191.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T08:35:24.589 INFO:teuthology.orchestra.run.smithi191.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T08:35:24.589 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.589 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.589 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.589 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.589 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:24.589 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.589 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.589 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.590 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:24.590 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:24.590 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:24.590 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:24.590 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:24.590 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.590 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.590 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:24.590 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.590 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.590 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.590 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:24.591 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:24.591 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.591 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.591 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:24.591 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T08:35:24.591 INFO:teuthology.orchestra.run.smithi191.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T08:35:24.591 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T08:35:24.591 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T08:35:24.591 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T08:35:24.591 INFO:teuthology.orchestra.run.smithi191.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T08:35:24.591 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T08:35:24.591 INFO:teuthology.orchestra.run.smithi191.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T08:35:24.592 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T08:35:24.592 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T08:35:24.592 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T08:35:24.592 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.592 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.592 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.592 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T08:35:24.592 INFO:teuthology.orchestra.run.smithi191.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T08:35:24.592 INFO:teuthology.orchestra.run.smithi191.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T08:35:24.592 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T08:35:24.592 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T08:35:24.592 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.593 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.593 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T08:35:24.593 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.593 INFO:teuthology.orchestra.run.smithi191.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T08:35:24.593 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T08:35:24.593 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T08:35:24.593 INFO:teuthology.orchestra.run.smithi191.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T08:35:24.593 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T08:35:24.593 INFO:teuthology.orchestra.run.smithi191.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T08:35:24.593 INFO:teuthology.orchestra.run.smithi191.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T08:35:24.593 INFO:teuthology.orchestra.run.smithi191.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T08:35:24.593 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T08:35:24.594 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T08:35:24.594 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T08:35:24.594 INFO:teuthology.orchestra.run.smithi191.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T08:35:24.594 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T08:35:24.594 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T08:35:24.594 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T08:35:24.594 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T08:35:24.594 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T08:35:24.594 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T08:35:24.594 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T08:35:24.594 INFO:teuthology.orchestra.run.smithi191.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T08:35:24.594 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T08:35:24.594 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T08:35:24.595 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T08:35:24.595 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.595 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.595 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.595 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T08:35:24.595 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T08:35:24.595 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T08:35:24.595 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T08:35:24.595 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T08:35:24.595 INFO:teuthology.orchestra.run.smithi191.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T08:35:24.595 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T08:35:24.595 INFO:teuthology.orchestra.run.smithi191.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T08:35:24.596 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T08:35:24.596 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T08:35:24.596 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T08:35:24.596 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T08:35:24.596 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T08:35:24.596 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T08:35:24.596 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T08:35:24.596 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T08:35:24.596 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T08:35:24.596 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T08:35:24.596 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T08:35:24.596 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T08:35:24.596 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T08:35:24.597 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T08:35:24.597 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T08:35:24.597 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T08:35:24.597 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T08:35:24.597 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T08:35:24.597 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T08:35:24.597 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T08:35:24.597 INFO:teuthology.orchestra.run.smithi191.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T08:35:24.597 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T08:35:24.597 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T08:35:24.597 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T08:35:24.597 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T08:35:24.598 INFO:teuthology.orchestra.run.smithi191.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T08:35:24.598 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T08:35:24.598 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T08:35:24.598 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T08:35:24.598 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T08:35:24.598 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T08:35:24.598 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T08:35:24.598 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.598 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.598 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T08:35:24.598 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T08:35:24.598 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T08:35:24.598 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.599 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T08:35:24.599 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T08:35:24.599 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T08:35:24.599 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T08:35:24.599 INFO:teuthology.orchestra.run.smithi191.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T08:35:24.599 INFO:teuthology.orchestra.run.smithi191.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T08:35:24.599 INFO:teuthology.orchestra.run.smithi191.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T08:35:24.599 INFO:teuthology.orchestra.run.smithi191.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T08:35:24.599 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T08:35:24.599 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T08:35:24.599 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T08:35:24.599 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T08:35:24.600 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T08:35:24.600 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T08:35:24.600 INFO:teuthology.orchestra.run.smithi191.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T08:35:24.600 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.600 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.600 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:24.600 INFO:teuthology.orchestra.run.smithi191.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T08:35:24.600 INFO:teuthology.orchestra.run.smithi191.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T08:35:24.600 INFO:teuthology.orchestra.run.smithi191.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T08:35:24.600 INFO:teuthology.orchestra.run.smithi191.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T08:35:24.600 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T08:35:24.600 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T08:35:24.600 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T08:35:24.601 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:35:24.601 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-09-10T08:35:24.837 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T08:35:24.887 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T08:35:24.928 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T08:35:24.960 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T08:35:24.987 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T08:35:25.044 DEBUG:teuthology.parallel:result is None 2024-09-10T08:35:25.174 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T08:35:25.217 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T08:35:25.247 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T08:35:25.276 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T08:35:25.314 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T08:35:25.974 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:35:26.012 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:35:26.032 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T08:35:26.861 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T08:35:26.949 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T08:35:27.000 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T08:35:27.112 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T08:35:27.143 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T08:35:27.169 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T08:35:27.196 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T08:35:27.196 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T08:35:27.279 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T08:35:27.279 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T08:35:27.355 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T08:35:27.380 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T08:35:27.385 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:35:27.385 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:35:27.573 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T08:35:27.573 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T08:35:27.666 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T08:35:27.738 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:35:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:35:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T08:35:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T08:35:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T08:35:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T08:35:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T08:35:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T08:35:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T08:35:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T08:35:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T08:35:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T08:35:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T08:35:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T08:35:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T08:35:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T08:35:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T08:35:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T08:35:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T08:35:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T08:35:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T08:35:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T08:35:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T08:35:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T08:35:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T08:35:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T08:35:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T08:35:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T08:35:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T08:35:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T08:35:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T08:35:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T08:35:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T08:35:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T08:35:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T08:35:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T08:35:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T08:35:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T08:35:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T08:35:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T08:35:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T08:35:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T08:35:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T08:35:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T08:35:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T08:35:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T08:35:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T08:35:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T08:35:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T08:35:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T08:35:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T08:35:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T08:35:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T08:35:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T08:35:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T08:35:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T08:35:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T08:35:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T08:35:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T08:35:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T08:35:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T08:35:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T08:35:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T08:35:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T08:35:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T08:35:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T08:35:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T08:35:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T08:35:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T08:35:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T08:35:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T08:35:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T08:35:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T08:35:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T08:35:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T08:35:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T08:35:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T08:35:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T08:35:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T08:35:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T08:35:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T08:35:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T08:35:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T08:35:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T08:35:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T08:35:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T08:35:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T08:35:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T08:35:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T08:35:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T08:35:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T08:35:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T08:35:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T08:35:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T08:35:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T08:35:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T08:35:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T08:35:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T08:35:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T08:35:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T08:35:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T08:35:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T08:35:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T08:35:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T08:35:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T08:35:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T08:35:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T08:35:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T08:35:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T08:35:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T08:35:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T08:35:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T08:35:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T08:35:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T08:35:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T08:35:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T08:35:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T08:35:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T08:35:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T08:35:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T08:35:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T08:35:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T08:35:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T08:35:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T08:35:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T08:35:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T08:35:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T08:35:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T08:35:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T08:35:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T08:35:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T08:35:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T08:35:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T08:35:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T08:35:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T08:35:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T08:35:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T08:35:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T08:35:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T08:35:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T08:35:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T08:35:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T08:35:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T08:35:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T08:35:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T08:35:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T08:35:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T08:35:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T08:35:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T08:35:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T08:35:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T08:35:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T08:35:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T08:35:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T08:35:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T08:35:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T08:35:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T08:35:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T08:35:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T08:35:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T08:35:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T08:35:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T08:35:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T08:35:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T08:35:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T08:35:36.079 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T08:35:36.080 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:35:36.080 INFO:teuthology.orchestra.run.smithi174.stdout:Upgraded: 2024-09-10T08:35:36.080 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T08:35:36.080 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T08:35:36.080 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T08:35:36.080 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T08:35:36.080 INFO:teuthology.orchestra.run.smithi174.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.080 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.081 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T08:35:36.081 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T08:35:36.081 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-09-10T08:35:36.081 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T08:35:36.081 INFO:teuthology.orchestra.run.smithi174.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T08:35:36.081 INFO:teuthology.orchestra.run.smithi174.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T08:35:36.081 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T08:35:36.081 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.081 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.081 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.082 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.082 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:36.082 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.082 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.082 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.082 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:36.082 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:36.082 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:36.082 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:36.082 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:36.082 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.083 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.083 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:36.083 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.083 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.083 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.083 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:36.083 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:36.083 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.084 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.084 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T08:35:36.084 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T08:35:36.084 INFO:teuthology.orchestra.run.smithi174.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T08:35:36.084 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T08:35:36.084 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T08:35:36.084 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T08:35:36.084 INFO:teuthology.orchestra.run.smithi174.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T08:35:36.084 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T08:35:36.084 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T08:35:36.085 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T08:35:36.085 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T08:35:36.085 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T08:35:36.085 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.085 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.085 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.085 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T08:35:36.085 INFO:teuthology.orchestra.run.smithi174.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T08:35:36.085 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T08:35:36.085 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T08:35:36.085 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T08:35:36.085 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.085 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.086 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T08:35:36.086 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.086 INFO:teuthology.orchestra.run.smithi174.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T08:35:36.086 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T08:35:36.086 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T08:35:36.086 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T08:35:36.086 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T08:35:36.086 INFO:teuthology.orchestra.run.smithi174.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T08:35:36.086 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T08:35:36.086 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T08:35:36.086 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T08:35:36.086 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T08:35:36.087 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T08:35:36.087 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T08:35:36.087 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T08:35:36.087 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T08:35:36.087 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T08:35:36.087 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T08:35:36.088 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T08:35:36.088 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T08:35:36.088 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T08:35:36.088 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T08:35:36.088 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T08:35:36.088 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T08:35:36.088 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T08:35:36.088 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.088 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.088 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.089 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T08:35:36.089 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T08:35:36.089 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T08:35:36.089 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T08:35:36.089 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T08:35:36.089 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T08:35:36.089 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T08:35:36.089 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T08:35:36.089 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T08:35:36.089 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T08:35:36.089 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T08:35:36.089 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T08:35:36.090 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T08:35:36.090 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T08:35:36.090 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T08:35:36.090 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T08:35:36.090 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T08:35:36.090 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T08:35:36.090 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T08:35:36.090 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T08:35:36.090 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T08:35:36.090 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T08:35:36.090 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T08:35:36.090 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T08:35:36.091 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T08:35:36.091 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T08:35:36.091 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T08:35:36.091 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T08:35:36.091 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T08:35:36.091 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T08:35:36.091 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T08:35:36.091 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T08:35:36.091 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T08:35:36.091 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T08:35:36.091 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T08:35:36.091 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T08:35:36.092 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T08:35:36.092 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T08:35:36.092 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T08:35:36.092 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T08:35:36.092 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.092 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.092 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T08:35:36.092 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T08:35:36.092 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T08:35:36.092 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.092 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T08:35:36.092 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T08:35:36.093 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T08:35:36.093 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T08:35:36.093 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T08:35:36.093 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T08:35:36.093 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T08:35:36.093 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T08:35:36.093 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T08:35:36.093 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T08:35:36.093 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T08:35:36.093 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T08:35:36.093 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T08:35:36.093 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T08:35:36.094 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T08:35:36.094 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.094 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.094 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T08:35:36.094 INFO:teuthology.orchestra.run.smithi174.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T08:35:36.094 INFO:teuthology.orchestra.run.smithi174.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T08:35:36.094 INFO:teuthology.orchestra.run.smithi174.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T08:35:36.094 INFO:teuthology.orchestra.run.smithi174.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T08:35:36.094 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T08:35:36.094 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T08:35:36.095 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T08:35:36.095 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:35:36.095 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-09-10T08:35:36.555 DEBUG:teuthology.parallel:result is None 2024-09-10T08:35:36.556 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:35:36.699 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T08:35:36.726 INFO:teuthology.orchestra.run.smithi172.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T08:35:36.727 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T08:35:36.727 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T08:35:36.729 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:35:36.867 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T08:35:36.889 INFO:teuthology.orchestra.run.smithi174.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T08:35:36.889 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T08:35:36.889 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T08:35:36.891 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:35:37.030 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T08:35:37.056 INFO:teuthology.orchestra.run.smithi191.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T08:35:37.056 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T08:35:37.056 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T08:35:37.058 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T08:35:37.059 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:35:37.059 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T08:35:37.091 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T08:35:37.091 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T08:35:37.119 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-10T08:35:37.119 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T08:35:37.151 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T08:35:37.151 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:35:37.151 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T08:35:37.181 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T08:35:37.251 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T08:35:37.251 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T08:35:37.276 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T08:35:37.340 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-10T08:35:37.341 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T08:35:37.371 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T08:35:37.441 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T08:35:37.441 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:35:37.441 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T08:35:37.471 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T08:35:37.539 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T08:35:37.539 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T08:35:37.564 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T08:35:37.629 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-10T08:35:37.629 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T08:35:37.655 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T08:35:37.721 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T08:35:37.721 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:35:37.722 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T08:35:37.752 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T08:35:37.821 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T08:35:37.821 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T08:35:37.846 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T08:35:37.910 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-10T08:35:37.910 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T08:35:37.936 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T08:35:38.001 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T08:35:38.105 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'subvols': {'create': 2}} 2024-09-10T08:35:38.106 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:35:38.107 INFO:tasks.cephadm:Cluster fsid is a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:35:38.107 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T08:35:38.107 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.172', 'mon.b': '172.21.15.174', 'mon.c': '172.21.15.191'} 2024-09-10T08:35:38.107 INFO:tasks.cephadm:First mon is mon.a on smithi172 2024-09-10T08:35:38.107 INFO:tasks.cephadm:First mgr is x 2024-09-10T08:35:38.107 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T08:35:38.107 DEBUG:teuthology.orchestra.run.smithi172:> sudo hostname $(hostname -s) 2024-09-10T08:35:38.138 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname $(hostname -s) 2024-09-10T08:35:38.164 DEBUG:teuthology.orchestra.run.smithi191:> sudo hostname $(hostname -s) 2024-09-10T08:35:38.189 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-10T08:35:38.190 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:35:38.339 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82914/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:14:38.445020', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82912/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:05:03.996701', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-10T08:35:38.486 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240909.165402-squid, sha1 1cf819b7212a6048e92fc78c4eff963344f6d56b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:35:38.488 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T08:35:38.488 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T08:35:38.488 DEBUG:teuthology.orchestra.run.smithi172:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T08:35:38.855 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 08:35 /home/ubuntu/cephtest/cephadm 2024-09-10T08:35:38.855 DEBUG:teuthology.orchestra.run.smithi174:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T08:35:39.228 INFO:teuthology.orchestra.run.smithi174.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 08:35 /home/ubuntu/cephtest/cephadm 2024-09-10T08:35:39.228 DEBUG:teuthology.orchestra.run.smithi191:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T08:35:39.627 INFO:teuthology.orchestra.run.smithi191.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 08:35 /home/ubuntu/cephtest/cephadm 2024-09-10T08:35:39.627 DEBUG:teuthology.orchestra.run.smithi172:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T08:35:39.648 DEBUG:teuthology.orchestra.run.smithi174:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T08:35:39.667 DEBUG:teuthology.orchestra.run.smithi191:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T08:35:39.702 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b on all hosts... 2024-09-10T08:35:39.703 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T08:35:39.705 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T08:35:39.711 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T08:35:41.201 INFO:teuthology.orchestra.run.smithi172.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T08:35:41.278 INFO:teuthology.orchestra.run.smithi174.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T08:35:41.303 INFO:teuthology.orchestra.run.smithi191.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T08:36:08.480 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-10T08:36:08.480 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T08:36:08.481 INFO:teuthology.orchestra.run.smithi191.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T08:36:08.481 INFO:teuthology.orchestra.run.smithi191.stdout: "repo_digests": [ 2024-09-10T08:36:08.481 INFO:teuthology.orchestra.run.smithi191.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T08:36:08.481 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-09-10T08:36:08.481 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-10T08:36:10.252 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:36:10.253 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T08:36:10.253 INFO:teuthology.orchestra.run.smithi172.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T08:36:10.253 INFO:teuthology.orchestra.run.smithi172.stdout: "repo_digests": [ 2024-09-10T08:36:10.253 INFO:teuthology.orchestra.run.smithi172.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T08:36:10.253 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-09-10T08:36:10.253 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:36:10.956 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-10T08:36:11.041 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T08:36:11.041 INFO:teuthology.orchestra.run.smithi174.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T08:36:11.041 INFO:teuthology.orchestra.run.smithi174.stdout: "repo_digests": [ 2024-09-10T08:36:11.041 INFO:teuthology.orchestra.run.smithi174.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T08:36:11.041 INFO:teuthology.orchestra.run.smithi174.stdout: ] 2024-09-10T08:36:11.041 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-10T08:36:11.043 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph 2024-09-10T08:36:11.074 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph 2024-09-10T08:36:11.102 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph 2024-09-10T08:36:11.135 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 777 /etc/ceph 2024-09-10T08:36:11.162 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 777 /etc/ceph 2024-09-10T08:36:11.188 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 777 /etc/ceph 2024-09-10T08:36:11.218 INFO:tasks.cephadm:Writing seed config... 2024-09-10T08:36:11.219 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-10T08:36:11.219 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-10T08:36:11.219 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-10T08:36:11.219 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-10T08:36:11.219 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-10T08:36:11.219 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-10T08:36:11.220 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-10T08:36:11.220 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-10T08:36:11.220 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-10T08:36:11.220 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-10T08:36:11.220 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-10T08:36:11.220 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-10T08:36:11.220 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-10T08:36:11.220 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-10T08:36:11.220 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-10T08:36:11.220 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-10T08:36:11.220 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-10T08:36:11.220 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-10T08:36:11.220 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-10T08:36:11.221 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-10T08:36:11.221 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-10T08:36:11.221 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T08:36:11.221 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T08:36:11.221 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T08:36:11.221 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T08:36:11.221 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T08:36:11.221 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-10T08:36:11.221 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-10T08:36:11.222 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-10T08:36:11.222 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-10T08:36:11.222 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-10T08:36:11.222 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-10T08:36:11.222 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-10T08:36:11.222 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T08:36:11.222 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T08:36:11.222 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-10T08:36:11.222 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-09-10T08:36:11.222 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-10T08:36:11.222 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-10T08:36:11.222 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-10T08:36:11.222 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-10T08:36:11.223 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-09-10T08:36:11.223 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-10T08:36:11.223 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-10T08:36:11.223 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T08:36:11.223 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-10T08:36:11.224 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:36:11.224 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T08:36:11.242 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a7dba522-6f4f-11ef-bcea-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-10T08:36:11.242 DEBUG:teuthology.orchestra.run.smithi172:mon.a> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T08:36:11.285 DEBUG:teuthology.orchestra.run.smithi172:mgr.x> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T08:36:11.330 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T08:36:11.331 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b -v bootstrap --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.172 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:36:11.538 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-10T08:36:11.538 INFO:teuthology.orchestra.run.smithi172.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b', '-v', 'bootstrap', '--fsid', 'a7dba522-6f4f-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.172', '--skip-admin-label'] 2024-09-10T08:36:11.538 INFO:teuthology.orchestra.run.smithi172.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T08:36:11.538 INFO:teuthology.orchestra.run.smithi172.stdout:Verifying podman|docker is present... 2024-09-10T08:36:11.563 INFO:teuthology.orchestra.run.smithi172.stdout:/bin/podman: stdout 5.2.2 2024-09-10T08:36:11.563 INFO:teuthology.orchestra.run.smithi172.stdout:Verifying lvm2 is present... 2024-09-10T08:36:11.563 INFO:teuthology.orchestra.run.smithi172.stdout:Verifying time synchronization is in place... 2024-09-10T08:36:11.571 INFO:teuthology.orchestra.run.smithi172.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T08:36:11.571 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T08:36:11.577 INFO:teuthology.orchestra.run.smithi172.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T08:36:11.577 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stdout inactive 2024-09-10T08:36:11.584 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stdout enabled 2024-09-10T08:36:11.590 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stdout active 2024-09-10T08:36:11.590 INFO:teuthology.orchestra.run.smithi172.stdout:Unit chronyd.service is enabled and running 2024-09-10T08:36:11.590 INFO:teuthology.orchestra.run.smithi172.stdout:Repeating the final host check... 2024-09-10T08:36:11.612 INFO:teuthology.orchestra.run.smithi172.stdout:/bin/podman: stdout 5.2.2 2024-09-10T08:36:11.612 INFO:teuthology.orchestra.run.smithi172.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T08:36:11.613 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl is present 2024-09-10T08:36:11.613 INFO:teuthology.orchestra.run.smithi172.stdout:lvcreate is present 2024-09-10T08:36:11.619 INFO:teuthology.orchestra.run.smithi172.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T08:36:11.619 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T08:36:11.625 INFO:teuthology.orchestra.run.smithi172.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T08:36:11.625 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stdout inactive 2024-09-10T08:36:11.632 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stdout enabled 2024-09-10T08:36:11.638 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stdout active 2024-09-10T08:36:11.638 INFO:teuthology.orchestra.run.smithi172.stdout:Unit chronyd.service is enabled and running 2024-09-10T08:36:11.638 INFO:teuthology.orchestra.run.smithi172.stdout:Host looks OK 2024-09-10T08:36:11.638 INFO:teuthology.orchestra.run.smithi172.stdout:Cluster fsid: a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:36:11.639 INFO:teuthology.orchestra.run.smithi172.stdout:Acquiring lock 140717152039936 on /run/cephadm/a7dba522-6f4f-11ef-bcea-c7b262605968.lock 2024-09-10T08:36:11.639 INFO:teuthology.orchestra.run.smithi172.stdout:Lock 140717152039936 acquired on /run/cephadm/a7dba522-6f4f-11ef-bcea-c7b262605968.lock 2024-09-10T08:36:11.639 INFO:teuthology.orchestra.run.smithi172.stdout:Verifying IP 172.21.15.172 port 3300 ... 2024-09-10T08:36:11.639 INFO:teuthology.orchestra.run.smithi172.stdout:Verifying IP 172.21.15.172 port 6789 ... 2024-09-10T08:36:11.639 INFO:teuthology.orchestra.run.smithi172.stdout:Base mon IP(s) is [172.21.15.172:3300, 172.21.15.172:6789], mon addrv is [v2:172.21.15.172:3300,v1:172.21.15.172:6789] 2024-09-10T08:36:11.642 INFO:teuthology.orchestra.run.smithi172.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.172 metric 100 2024-09-10T08:36:11.643 INFO:teuthology.orchestra.run.smithi172.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.172 metric 100 2024-09-10T08:36:11.645 INFO:teuthology.orchestra.run.smithi172.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T08:36:11.645 INFO:teuthology.orchestra.run.smithi172.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-10T08:36:11.645 INFO:teuthology.orchestra.run.smithi172.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-10T08:36:11.648 INFO:teuthology.orchestra.run.smithi172.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T08:36:11.648 INFO:teuthology.orchestra.run.smithi172.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T08:36:11.648 INFO:teuthology.orchestra.run.smithi172.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T08:36:11.648 INFO:teuthology.orchestra.run.smithi172.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-10T08:36:11.648 INFO:teuthology.orchestra.run.smithi172.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7f91/64 scope link noprefixroute 2024-09-10T08:36:11.648 INFO:teuthology.orchestra.run.smithi172.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T08:36:11.649 INFO:teuthology.orchestra.run.smithi172.stdout:Mon IP `172.21.15.172` is in CIDR network `172.21.0.0/20` 2024-09-10T08:36:11.649 INFO:teuthology.orchestra.run.smithi172.stdout:Mon IP `172.21.15.172` is in CIDR network `172.21.0.0/20` 2024-09-10T08:36:11.649 INFO:teuthology.orchestra.run.smithi172.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T08:36:11.649 INFO:teuthology.orchestra.run.smithi172.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T08:36:11.650 INFO:teuthology.orchestra.run.smithi172.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T08:36:15.764 INFO:teuthology.orchestra.run.smithi172.stdout:/bin/podman: stdout 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T08:36:15.764 INFO:teuthology.orchestra.run.smithi172.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T08:36:15.765 INFO:teuthology.orchestra.run.smithi172.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T08:36:15.765 INFO:teuthology.orchestra.run.smithi172.stdout:/bin/podman: stderr Copying blob sha256:3965ca83761b6e221cd35b61f2ebac0e3b1f9d68a1ce39c1ada69153a1d73116 2024-09-10T08:36:15.765 INFO:teuthology.orchestra.run.smithi172.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T08:36:15.765 INFO:teuthology.orchestra.run.smithi172.stdout:/bin/podman: stderr Copying config sha256:41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T08:36:15.765 INFO:teuthology.orchestra.run.smithi172.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T08:36:25.210 INFO:teuthology.orchestra.run.smithi172.stdout:ceph: stdout ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T08:36:25.210 INFO:teuthology.orchestra.run.smithi172.stdout:Ceph version: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T08:36:25.211 INFO:teuthology.orchestra.run.smithi172.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T08:36:26.173 INFO:teuthology.orchestra.run.smithi172.stdout:stat: stdout 167 167 2024-09-10T08:36:26.173 INFO:teuthology.orchestra.run.smithi172.stdout:Creating initial keys... 2024-09-10T08:36:27.095 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph-authtool: stdout AQAKBeBmOqCxHRAAUkxND7EYgV7T2PmzZFMhlQ== 2024-09-10T08:36:28.134 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph-authtool: stdout AQALBeBmgtWGGBAAD0K+68xvAoSgLsGva0iWsw== 2024-09-10T08:36:29.023 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph-authtool: stdout AQAMBeBmxT/jGRAAlmA+9PytW0yuSpy8i5Dp3A== 2024-09-10T08:36:29.024 INFO:teuthology.orchestra.run.smithi172.stdout:Creating initial monmap... 2024-09-10T08:36:29.956 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T08:36:29.956 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-10T08:36:29.956 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:36:29.956 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T08:36:29.956 INFO:teuthology.orchestra.run.smithi172.stdout:monmaptool for a [v2:172.21.15.172:3300,v1:172.21.15.172:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T08:36:29.956 INFO:teuthology.orchestra.run.smithi172.stdout:setting min_mon_release = quincy 2024-09-10T08:36:29.956 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/monmaptool: set fsid to a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:36:29.956 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T08:36:29.956 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:36:29.957 INFO:teuthology.orchestra.run.smithi172.stdout:Creating mon... 2024-09-10T08:36:31.608 INFO:teuthology.orchestra.run.smithi172.stdout:create mon.a on 2024-09-10T08:36:31.812 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T08:36:31.946 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T08:36:32.162 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a7dba522-6f4f-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-a7dba522-6f4f-11ef-bcea-c7b262605968.target. 2024-09-10T08:36:32.162 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a7dba522-6f4f-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-a7dba522-6f4f-11ef-bcea-c7b262605968.target. 2024-09-10T08:36:32.396 INFO:teuthology.orchestra.run.smithi172.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mon.a 2024-09-10T08:36:32.396 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T08:36:32.565 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a7dba522-6f4f-11ef-bcea-c7b262605968.target.wants/ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@.service. 2024-09-10T08:36:32.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:32 smithi172 podman[30465]: 2024-09-10 08:36:32.724666195 +0000 UTC m=+0.015825651 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:36:33.085 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:32 smithi172 podman[30465]: 2024-09-10 08:36:32.833145508 +0000 UTC m=+0.124304964 container create f54d1064074e108f6657d1f95c38648b4a126f974cd76fa1cbc88b3774afee6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:36:33.086 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:33 smithi172 podman[30465]: 2024-09-10 08:36:33.044272707 +0000 UTC m=+0.335432163 container init f54d1064074e108f6657d1f95c38648b4a126f974cd76fa1cbc88b3774afee6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T08:36:33.086 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:33 smithi172 podman[30465]: 2024-09-10 08:36:33.047277743 +0000 UTC m=+0.338437200 container start f54d1064074e108f6657d1f95c38648b4a126f974cd76fa1cbc88b3774afee6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-10T08:36:33.142 INFO:teuthology.orchestra.run.smithi172.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T08:36:33.142 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stdout disabled 2024-09-10T08:36:33.149 INFO:teuthology.orchestra.run.smithi172.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T08:36:33.149 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stdout inactive 2024-09-10T08:36:33.149 INFO:teuthology.orchestra.run.smithi172.stdout:firewalld.service is not enabled 2024-09-10T08:36:33.149 INFO:teuthology.orchestra.run.smithi172.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T08:36:33.150 INFO:teuthology.orchestra.run.smithi172.stdout:Waiting for mon to start... 2024-09-10T08:36:33.150 INFO:teuthology.orchestra.run.smithi172.stdout:Waiting for mon... 2024-09-10T08:36:33.363 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:33 smithi172 ceph-mon[30485]: mkfs a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:36:33.363 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:33 smithi172 ceph-mon[30485]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T08:36:33.363 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:33 smithi172 bash[30465]: f54d1064074e108f6657d1f95c38648b4a126f974cd76fa1cbc88b3774afee6e 2024-09-10T08:36:33.363 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:33 smithi172 systemd[1]: Started Ceph mon.a for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:36:34.317 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:34 smithi172 ceph-mon[30485]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T08:36:34.318 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:34 smithi172 ceph-mon[30485]: monmap epoch 1 2024-09-10T08:36:34.318 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:34 smithi172 ceph-mon[30485]: fsid a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:36:34.318 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:34 smithi172 ceph-mon[30485]: last_changed 2024-09-10T08:36:29.338346+0000 2024-09-10T08:36:34.318 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:34 smithi172 ceph-mon[30485]: created 2024-09-10T08:36:29.338346+0000 2024-09-10T08:36:34.318 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:34 smithi172 ceph-mon[30485]: min_mon_release 19 (squid) 2024-09-10T08:36:34.318 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:34 smithi172 ceph-mon[30485]: election_strategy: 1 2024-09-10T08:36:34.318 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:34 smithi172 ceph-mon[30485]: 0: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.a 2024-09-10T08:36:34.318 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:34 smithi172 ceph-mon[30485]: fsmap 2024-09-10T08:36:34.319 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:34 smithi172 ceph-mon[30485]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T08:36:34.319 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:34 smithi172 ceph-mon[30485]: mgrmap e1: no daemons active 2024-09-10T08:36:34.319 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:34 smithi172 ceph-mon[30485]: from='client.? 172.21.15.172:0/3002252097' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T08:36:34.328 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T08:36:34.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout id: a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:36:34.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T08:36:34.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout 2024-09-10T08:36:34.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout services: 2024-09-10T08:36:34.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.525714s) 2024-09-10T08:36:34.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T08:36:34.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T08:36:34.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout 2024-09-10T08:36:34.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout data: 2024-09-10T08:36:34.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T08:36:34.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T08:36:34.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T08:36:34.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T08:36:34.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout 2024-09-10T08:36:34.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.540+0000 7f2001123640 1 Processor -- start 2024-09-10T08:36:34.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.540+0000 7f2001123640 1 -- start start 2024-09-10T08:36:34.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.540+0000 7f2001123640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ffc107ef0 0x7f1ffc10a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:34.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.540+0000 7f2001123640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc07be30 con 0x7f1ffc107ef0 2024-09-10T08:36:34.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.541+0000 7f1ffad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ffc107ef0 0x7f1ffc10a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:34.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.541+0000 7f1ffad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ffc107ef0 0x7f1ffc10a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36376/0 (socket says 172.21.15.172:36376) 2024-09-10T08:36:34.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.541+0000 7f1ffad9d640 1 -- 172.21.15.172:0/3505015947 learned_addr learned my addr 172.21.15.172:0/3505015947 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:34.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.541+0000 7f1ffad9d640 1 -- 172.21.15.172:0/3505015947 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ffc10a820 con 0x7f1ffc107ef0 2024-09-10T08:36:34.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.543+0000 7f1ffad9d640 1 --2- 172.21.15.172:0/3505015947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ffc107ef0 0x7f1ffc10a2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009820 tx=0x7f1fec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c1fa53db2d1430 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:34.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.543+0000 7f1ff9d9b640 1 -- 172.21.15.172:0/3505015947 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fec037070 con 0x7f1ffc107ef0 2024-09-10T08:36:34.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.543+0000 7f1ff9d9b640 1 -- 172.21.15.172:0/3505015947 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fec032440 con 0x7f1ffc107ef0 2024-09-10T08:36:34.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.543+0000 7f1ff9d9b640 1 -- 172.21.15.172:0/3505015947 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fec032740 con 0x7f1ffc107ef0 2024-09-10T08:36:34.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.543+0000 7f2001123640 1 -- 172.21.15.172:0/3505015947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ffc107ef0 msgr2=0x7f1ffc10a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:34.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.543+0000 7f2001123640 1 --2- 172.21.15.172:0/3505015947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ffc107ef0 0x7f1ffc10a2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009820 tx=0x7f1fec02a270 comp rx=0 tx=0).stop 2024-09-10T08:36:34.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.543+0000 7f2001123640 1 -- 172.21.15.172:0/3505015947 shutdown_connections 2024-09-10T08:36:34.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.543+0000 7f2001123640 1 --2- 172.21.15.172:0/3505015947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ffc107ef0 0x7f1ffc10a2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:34.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.543+0000 7f2001123640 1 -- 172.21.15.172:0/3505015947 >> 172.21.15.172:0/3505015947 conn(0x7f1ffc101b70 msgr2=0x7f1ffc103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:34.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.544+0000 7f2001123640 1 -- 172.21.15.172:0/3505015947 shutdown_connections 2024-09-10T08:36:34.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.544+0000 7f2001123640 1 -- 172.21.15.172:0/3505015947 wait complete. 2024-09-10T08:36:34.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.544+0000 7f2001123640 1 Processor -- start 2024-09-10T08:36:34.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.544+0000 7f2001123640 1 -- start start 2024-09-10T08:36:34.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.545+0000 7f2001123640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ffc107ef0 0x7f1ffc107770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:34.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.545+0000 7f2001123640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc10ad60 con 0x7f1ffc107ef0 2024-09-10T08:36:34.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.545+0000 7f1ffad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ffc107ef0 0x7f1ffc107770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:34.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.545+0000 7f1ffad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ffc107ef0 0x7f1ffc107770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36382/0 (socket says 172.21.15.172:36382) 2024-09-10T08:36:34.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.545+0000 7f1ffad9d640 1 -- 172.21.15.172:0/3002252097 learned_addr learned my addr 172.21.15.172:0/3002252097 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:34.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.545+0000 7f1ffad9d640 1 -- 172.21.15.172:0/3002252097 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ffc107cb0 con 0x7f1ffc107ef0 2024-09-10T08:36:34.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.545+0000 7f1ffad9d640 1 --2- 172.21.15.172:0/3002252097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ffc107ef0 0x7f1ffc107770 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0090b0 tx=0x7f1fec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:34.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.545+0000 7f1fe3fff640 1 -- 172.21.15.172:0/3002252097 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fec041070 con 0x7f1ffc107ef0 2024-09-10T08:36:34.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.545+0000 7f1fe3fff640 1 -- 172.21.15.172:0/3002252097 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fec03c7a0 con 0x7f1ffc107ef0 2024-09-10T08:36:34.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.545+0000 7f1fe3fff640 1 -- 172.21.15.172:0/3002252097 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fec037040 con 0x7f1ffc107ef0 2024-09-10T08:36:34.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.545+0000 7f2001123640 1 -- 172.21.15.172:0/3002252097 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ffc105ea0 con 0x7f1ffc107ef0 2024-09-10T08:36:34.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.545+0000 7f2001123640 1 -- 172.21.15.172:0/3002252097 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ffc1063e0 con 0x7f1ffc107ef0 2024-09-10T08:36:34.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.546+0000 7f1fe3fff640 1 -- 172.21.15.172:0/3002252097 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1fec044070 con 0x7f1ffc107ef0 2024-09-10T08:36:34.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.546+0000 7f1fe3fff640 1 -- 172.21.15.172:0/3002252097 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1fec03d3f0 con 0x7f1ffc107ef0 2024-09-10T08:36:34.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.546+0000 7f2001123640 1 -- 172.21.15.172:0/3002252097 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ffc108350 con 0x7f1ffc107ef0 2024-09-10T08:36:34.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.549+0000 7f1fe3fff640 1 -- 172.21.15.172:0/3002252097 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1fec046450 con 0x7f1ffc107ef0 2024-09-10T08:36:34.334 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.607+0000 7f2001123640 1 -- 172.21.15.172:0/3002252097 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f1ffc1945b0 con 0x7f1ffc107ef0 2024-09-10T08:36:34.334 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.607+0000 7f1fe3fff640 1 -- 172.21.15.172:0/3002252097 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f1fec046630 con 0x7f1ffc107ef0 2024-09-10T08:36:34.334 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.608+0000 7f2001123640 1 -- 172.21.15.172:0/3002252097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ffc107ef0 msgr2=0x7f1ffc107770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:34.334 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.608+0000 7f2001123640 1 --2- 172.21.15.172:0/3002252097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ffc107ef0 0x7f1ffc107770 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0090b0 tx=0x7f1fec0046c0 comp rx=0 tx=0).stop 2024-09-10T08:36:34.334 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.608+0000 7f2001123640 1 -- 172.21.15.172:0/3002252097 shutdown_connections 2024-09-10T08:36:34.334 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.608+0000 7f2001123640 1 --2- 172.21.15.172:0/3002252097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ffc107ef0 0x7f1ffc107770 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:34.334 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.608+0000 7f2001123640 1 -- 172.21.15.172:0/3002252097 >> 172.21.15.172:0/3002252097 conn(0x7f1ffc101b70 msgr2=0x7f1ffc102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:34.334 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.608+0000 7f2001123640 1 -- 172.21.15.172:0/3002252097 shutdown_connections 2024-09-10T08:36:34.334 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:33.608+0000 7f2001123640 1 -- 172.21.15.172:0/3002252097 wait complete. 2024-09-10T08:36:34.334 INFO:teuthology.orchestra.run.smithi172.stdout:mon is available 2024-09-10T08:36:34.335 INFO:teuthology.orchestra.run.smithi172.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T08:36:35.331 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:35 smithi172 ceph-mon[30485]: from='client.? 172.21.15.172:0/3575619565' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T08:36:35.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:35 smithi172 ceph-mon[30485]: from='client.? 172.21.15.172:0/3575619565' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T08:36:35.334 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout 2024-09-10T08:36:35.342 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T08:36:35.343 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout fsid = a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:36:35.343 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.172:3300,v1:172.21.15.172:6789] 2024-09-10T08:36:35.343 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T08:36:35.343 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T08:36:35.343 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T08:36:35.343 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T08:36:35.343 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout 2024-09-10T08:36:35.343 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T08:36:35.343 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T08:36:35.343 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout 2024-09-10T08:36:35.343 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T08:36:35.344 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-09-10T08:36:35.344 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T08:36:35.344 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T08:36:35.344 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T08:36:35.344 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T08:36:35.344 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T08:36:35.344 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.729+0000 7f44c99c9640 1 Processor -- start 2024-09-10T08:36:35.344 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.729+0000 7f44c99c9640 1 -- start start 2024-09-10T08:36:35.344 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.729+0000 7f44c99c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44c4107f70 0x7f44c410a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:35.344 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.729+0000 7f44c99c9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f44c40758a0 con 0x7f44c4107f70 2024-09-10T08:36:35.344 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.730+0000 7f44c2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44c4107f70 0x7f44c410a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:35.345 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.730+0000 7f44c2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44c4107f70 0x7f44c410a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36384/0 (socket says 172.21.15.172:36384) 2024-09-10T08:36:35.345 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.730+0000 7f44c2ffd640 1 -- 172.21.15.172:0/3663942146 learned_addr learned my addr 172.21.15.172:0/3663942146 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:35.345 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.730+0000 7f44c2ffd640 1 -- 172.21.15.172:0/3663942146 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c410a8a0 con 0x7f44c4107f70 2024-09-10T08:36:35.345 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.731+0000 7f44c2ffd640 1 --2- 172.21.15.172:0/3663942146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44c4107f70 0x7f44c410a360 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009e80 tx=0x7f44b402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719ca554ffe4065a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:35.345 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.731+0000 7f44c1ffb640 1 -- 172.21.15.172:0/3663942146 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44b4036070 con 0x7f44c4107f70 2024-09-10T08:36:35.345 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.731+0000 7f44c1ffb640 1 -- 172.21.15.172:0/3663942146 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44b4031400 con 0x7f44c4107f70 2024-09-10T08:36:35.345 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.731+0000 7f44c1ffb640 1 -- 172.21.15.172:0/3663942146 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44b4031700 con 0x7f44c4107f70 2024-09-10T08:36:35.345 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.732+0000 7f44c99c9640 1 -- 172.21.15.172:0/3663942146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44c4107f70 msgr2=0x7f44c410a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:35.345 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.732+0000 7f44c99c9640 1 --2- 172.21.15.172:0/3663942146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44c4107f70 0x7f44c410a360 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009e80 tx=0x7f44b402a7a0 comp rx=0 tx=0).stop 2024-09-10T08:36:35.345 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.732+0000 7f44c99c9640 1 -- 172.21.15.172:0/3663942146 shutdown_connections 2024-09-10T08:36:35.345 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.732+0000 7f44c99c9640 1 --2- 172.21.15.172:0/3663942146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44c4107f70 0x7f44c410a360 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:35.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.732+0000 7f44c99c9640 1 -- 172.21.15.172:0/3663942146 >> 172.21.15.172:0/3663942146 conn(0x7f44c4101b10 msgr2=0x7f44c4103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:35.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.732+0000 7f44c99c9640 1 -- 172.21.15.172:0/3663942146 shutdown_connections 2024-09-10T08:36:35.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.732+0000 7f44c99c9640 1 -- 172.21.15.172:0/3663942146 wait complete. 2024-09-10T08:36:35.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.732+0000 7f44c99c9640 1 Processor -- start 2024-09-10T08:36:35.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.732+0000 7f44c99c9640 1 -- start start 2024-09-10T08:36:35.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.733+0000 7f44c99c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44c4107f70 0x7f44c419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:35.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.733+0000 7f44c99c9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f44c410ad80 con 0x7f44c4107f70 2024-09-10T08:36:35.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.733+0000 7f44c2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44c4107f70 0x7f44c419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:35.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.733+0000 7f44c2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44c4107f70 0x7f44c419ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36394/0 (socket says 172.21.15.172:36394) 2024-09-10T08:36:35.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.733+0000 7f44c2ffd640 1 -- 172.21.15.172:0/3575619565 learned_addr learned my addr 172.21.15.172:0/3575619565 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:35.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.733+0000 7f44c2ffd640 1 -- 172.21.15.172:0/3575619565 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c419b160 con 0x7f44c4107f70 2024-09-10T08:36:35.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.733+0000 7f44c2ffd640 1 --2- 172.21.15.172:0/3575619565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44c4107f70 0x7f44c419ac20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f44b4034040 tx=0x7f44b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:35.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.733+0000 7f44c89c7640 1 -- 172.21.15.172:0/3575619565 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44b4041070 con 0x7f44c4107f70 2024-09-10T08:36:35.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.733+0000 7f44c89c7640 1 -- 172.21.15.172:0/3575619565 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44b403c650 con 0x7f44c4107f70 2024-09-10T08:36:35.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.733+0000 7f44c89c7640 1 -- 172.21.15.172:0/3575619565 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44b4036040 con 0x7f44c4107f70 2024-09-10T08:36:35.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.733+0000 7f44c99c9640 1 -- 172.21.15.172:0/3575619565 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c419b3f0 con 0x7f44c4107f70 2024-09-10T08:36:35.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.733+0000 7f44c99c9640 1 -- 172.21.15.172:0/3575619565 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c419e0e0 con 0x7f44c4107f70 2024-09-10T08:36:35.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.734+0000 7f44c89c7640 1 -- 172.21.15.172:0/3575619565 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f44b4044070 con 0x7f44c4107f70 2024-09-10T08:36:35.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.734+0000 7f44c89c7640 1 -- 172.21.15.172:0/3575619565 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f44b4046c10 con 0x7f44c4107f70 2024-09-10T08:36:35.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.734+0000 7f44c99c9640 1 -- 172.21.15.172:0/3575619565 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c41083d0 con 0x7f44c4107f70 2024-09-10T08:36:35.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.737+0000 7f44c89c7640 1 -- 172.21.15.172:0/3575619565 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f44b403c950 con 0x7f44c4107f70 2024-09-10T08:36:35.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.793+0000 7f44c99c9640 1 -- 172.21.15.172:0/3575619565 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f44c402d980 con 0x7f44c4107f70 2024-09-10T08:36:35.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.796+0000 7f44c89c7640 1 -- 172.21.15.172:0/3575619565 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+456 (secure 0 0 0) 0x7f44b403cb30 con 0x7f44c4107f70 2024-09-10T08:36:35.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.797+0000 7f44c89c7640 1 -- 172.21.15.172:0/3575619565 <== mon.0 v2:172.21.15.172:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f44b4031400 con 0x7f44c4107f70 2024-09-10T08:36:35.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.798+0000 7f44c99c9640 1 -- 172.21.15.172:0/3575619565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44c4107f70 msgr2=0x7f44c419ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:35.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.798+0000 7f44c99c9640 1 --2- 172.21.15.172:0/3575619565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44c4107f70 0x7f44c419ac20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f44b4034040 tx=0x7f44b4004a40 comp rx=0 tx=0).stop 2024-09-10T08:36:35.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.798+0000 7f44c99c9640 1 -- 172.21.15.172:0/3575619565 shutdown_connections 2024-09-10T08:36:35.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.798+0000 7f44c99c9640 1 --2- 172.21.15.172:0/3575619565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44c4107f70 0x7f44c419ac20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:35.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.798+0000 7f44c99c9640 1 -- 172.21.15.172:0/3575619565 >> 172.21.15.172:0/3575619565 conn(0x7f44c4101b10 msgr2=0x7f44c4102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:35.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.798+0000 7f44c99c9640 1 -- 172.21.15.172:0/3575619565 shutdown_connections 2024-09-10T08:36:35.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:34.798+0000 7f44c99c9640 1 -- 172.21.15.172:0/3575619565 wait complete. 2024-09-10T08:36:35.348 INFO:teuthology.orchestra.run.smithi172.stdout:Generating new minimal ceph.conf... 2024-09-10T08:36:36.331 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:36 smithi172 ceph-mon[30485]: from='client.? 172.21.15.172:0/2646725163' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:36:36.337 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.726+0000 7f3b7fc30640 1 Processor -- start 2024-09-10T08:36:36.338 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.726+0000 7f3b7fc30640 1 -- start start 2024-09-10T08:36:36.338 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.727+0000 7f3b7fc30640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b78105d00 0x7f3b78106100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:36.338 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.727+0000 7f3b7fc30640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78075900 con 0x7f3b78105d00 2024-09-10T08:36:36.338 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.727+0000 7f3b7d9cc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b78105d00 0x7f3b78106100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:36.338 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.727+0000 7f3b7d9cc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b78105d00 0x7f3b78106100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36402/0 (socket says 172.21.15.172:36402) 2024-09-10T08:36:36.338 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.727+0000 7f3b7d9cc640 1 -- 172.21.15.172:0/2847468609 learned_addr learned my addr 172.21.15.172:0/2847468609 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:36.338 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.727+0000 7f3b7d9cc640 1 -- 172.21.15.172:0/2847468609 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b78106640 con 0x7f3b78105d00 2024-09-10T08:36:36.338 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.728+0000 7f3b7d9cc640 1 --2- 172.21.15.172:0/2847468609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b78105d00 0x7f3b78106100 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009820 tx=0x7f3b7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9711c923c513f6e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:36.338 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.728+0000 7f3b7c9ca640 1 -- 172.21.15.172:0/2847468609 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b70037070 con 0x7f3b78105d00 2024-09-10T08:36:36.338 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.728+0000 7f3b7c9ca640 1 -- 172.21.15.172:0/2847468609 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3b700328d0 con 0x7f3b78105d00 2024-09-10T08:36:36.338 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.728+0000 7f3b7c9ca640 1 -- 172.21.15.172:0/2847468609 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b700307d0 con 0x7f3b78105d00 2024-09-10T08:36:36.339 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.728+0000 7f3b7fc30640 1 -- 172.21.15.172:0/2847468609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b78105d00 msgr2=0x7f3b78106100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:36.339 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.728+0000 7f3b7fc30640 1 --2- 172.21.15.172:0/2847468609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b78105d00 0x7f3b78106100 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009820 tx=0x7f3b7002a270 comp rx=0 tx=0).stop 2024-09-10T08:36:36.339 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.728+0000 7f3b7fc30640 1 -- 172.21.15.172:0/2847468609 shutdown_connections 2024-09-10T08:36:36.339 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.728+0000 7f3b7fc30640 1 --2- 172.21.15.172:0/2847468609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b78105d00 0x7f3b78106100 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:36.339 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.728+0000 7f3b7fc30640 1 -- 172.21.15.172:0/2847468609 >> 172.21.15.172:0/2847468609 conn(0x7f3b78101b10 msgr2=0x7f3b78103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:36.339 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.729+0000 7f3b7fc30640 1 -- 172.21.15.172:0/2847468609 shutdown_connections 2024-09-10T08:36:36.339 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.729+0000 7f3b7fc30640 1 -- 172.21.15.172:0/2847468609 wait complete. 2024-09-10T08:36:36.339 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.729+0000 7f3b7fc30640 1 Processor -- start 2024-09-10T08:36:36.339 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.729+0000 7f3b7fc30640 1 -- start start 2024-09-10T08:36:36.339 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.729+0000 7f3b7fc30640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b78105d00 0x7f3b7819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:36.339 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.729+0000 7f3b7fc30640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3b781076e0 con 0x7f3b78105d00 2024-09-10T08:36:36.339 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.730+0000 7f3b7d9cc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b78105d00 0x7f3b7819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:36.340 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.730+0000 7f3b7d9cc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b78105d00 0x7f3b7819d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36404/0 (socket says 172.21.15.172:36404) 2024-09-10T08:36:36.340 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.730+0000 7f3b7d9cc640 1 -- 172.21.15.172:0/2646725163 learned_addr learned my addr 172.21.15.172:0/2646725163 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:36.340 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.730+0000 7f3b7d9cc640 1 -- 172.21.15.172:0/2646725163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7819db50 con 0x7f3b78105d00 2024-09-10T08:36:36.340 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.730+0000 7f3b7d9cc640 1 --2- 172.21.15.172:0/2646725163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b78105d00 0x7f3b7819d610 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3b70030520 tx=0x7f3b70032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:36.340 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.730+0000 7f3b76ffd640 1 -- 172.21.15.172:0/2646725163 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b70042070 con 0x7f3b78105d00 2024-09-10T08:36:36.340 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.730+0000 7f3b76ffd640 1 -- 172.21.15.172:0/2646725163 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3b70030cc0 con 0x7f3b78105d00 2024-09-10T08:36:36.340 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.730+0000 7f3b76ffd640 1 -- 172.21.15.172:0/2646725163 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b70037040 con 0x7f3b78105d00 2024-09-10T08:36:36.340 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.730+0000 7f3b7fc30640 1 -- 172.21.15.172:0/2646725163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b7819dde0 con 0x7f3b78105d00 2024-09-10T08:36:36.340 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.730+0000 7f3b7fc30640 1 -- 172.21.15.172:0/2646725163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b781a0ad0 con 0x7f3b78105d00 2024-09-10T08:36:36.340 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.730+0000 7f3b76ffd640 1 -- 172.21.15.172:0/2646725163 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3b70045070 con 0x7f3b78105d00 2024-09-10T08:36:36.340 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.730+0000 7f3b76ffd640 1 -- 172.21.15.172:0/2646725163 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3b70047990 con 0x7f3b78105d00 2024-09-10T08:36:36.341 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.731+0000 7f3b7fc30640 1 -- 172.21.15.172:0/2646725163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b78106100 con 0x7f3b78105d00 2024-09-10T08:36:36.341 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.733+0000 7f3b76ffd640 1 -- 172.21.15.172:0/2646725163 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f3b7003d430 con 0x7f3b78105d00 2024-09-10T08:36:36.341 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.790+0000 7f3b7fc30640 1 -- 172.21.15.172:0/2646725163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f3b7802d940 con 0x7f3b78105d00 2024-09-10T08:36:36.341 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.790+0000 7f3b76ffd640 1 -- 172.21.15.172:0/2646725163 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f3b7003d610 con 0x7f3b78105d00 2024-09-10T08:36:36.341 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.791+0000 7f3b7fc30640 1 -- 172.21.15.172:0/2646725163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b78105d00 msgr2=0x7f3b7819d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:36.341 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.791+0000 7f3b7fc30640 1 --2- 172.21.15.172:0/2646725163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b78105d00 0x7f3b7819d610 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3b70030520 tx=0x7f3b70032dd0 comp rx=0 tx=0).stop 2024-09-10T08:36:36.341 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.791+0000 7f3b7fc30640 1 -- 172.21.15.172:0/2646725163 shutdown_connections 2024-09-10T08:36:36.341 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.791+0000 7f3b7fc30640 1 --2- 172.21.15.172:0/2646725163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b78105d00 0x7f3b7819d610 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:36.341 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.791+0000 7f3b7fc30640 1 -- 172.21.15.172:0/2646725163 >> 172.21.15.172:0/2646725163 conn(0x7f3b78101b10 msgr2=0x7f3b781942b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:36.341 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.791+0000 7f3b7fc30640 1 -- 172.21.15.172:0/2646725163 shutdown_connections 2024-09-10T08:36:36.342 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:35.791+0000 7f3b7fc30640 1 -- 172.21.15.172:0/2646725163 wait complete. 2024-09-10T08:36:36.342 INFO:teuthology.orchestra.run.smithi172.stdout:Restarting the monitor... 2024-09-10T08:36:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:36 smithi172 systemd[1]: Stopping Ceph mon.a for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T08:36:37.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:36 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a[30481]: 2024-09-10T08:36:36.616+0000 7f0eae13c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T08:36:37.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:36 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a[30481]: 2024-09-10T08:36:36.616+0000 7f0eae13c640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T08:36:37.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:36 smithi172 podman[30727]: 2024-09-10 08:36:36.781238836 +0000 UTC m=+0.283211090 container died f54d1064074e108f6657d1f95c38648b4a126f974cd76fa1cbc88b3774afee6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:36:37.493 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:37 smithi172 podman[30727]: 2024-09-10 08:36:37.284034404 +0000 UTC m=+0.786006657 container cleanup f54d1064074e108f6657d1f95c38648b4a126f974cd76fa1cbc88b3774afee6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T08:36:37.493 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:37 smithi172 bash[30727]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a 2024-09-10T08:36:37.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:37 smithi172 podman[30740]: 2024-09-10 08:36:37.493613222 +0000 UTC m=+0.716070769 container remove f54d1064074e108f6657d1f95c38648b4a126f974cd76fa1cbc88b3774afee6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:36:37.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:37 smithi172 systemd[1]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T08:36:37.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:37 smithi172 systemd[1]: Stopped Ceph mon.a for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:36:37.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:37 smithi172 systemd[1]: Starting Ceph mon.a for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T08:36:38.095 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:37 smithi172 podman[30835]: 2024-09-10 08:36:37.738269501 +0000 UTC m=+0.016069913 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:36:38.095 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:37 smithi172 podman[30835]: 2024-09-10 08:36:37.869638486 +0000 UTC m=+0.147438891 container create 4ca591a9b8a003ee6f1e3edc34e471234001816eb1d1bd40e33699195fa18d1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T08:36:38.095 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 podman[30835]: 2024-09-10 08:36:38.071884709 +0000 UTC m=+0.349685115 container init 4ca591a9b8a003ee6f1e3edc34e471234001816eb1d1bd40e33699195fa18d1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2) 2024-09-10T08:36:38.095 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 podman[30835]: 2024-09-10 08:36:38.074897497 +0000 UTC m=+0.352697903 container start 4ca591a9b8a003ee6f1e3edc34e471234001816eb1d1bd40e33699195fa18d1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:36:38.184 INFO:teuthology.orchestra.run.smithi172.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T08:36:38.432 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T08:36:38.432 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable), process ceph-mon, pid 2 2024-09-10T08:36:38.432 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: pidfile_write: ignore empty --pid-file 2024-09-10T08:36:38.432 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: load: jerasure load: lrc 2024-09-10T08:36:38.432 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: RocksDB version: 7.9.2 2024-09-10T08:36:38.432 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Git sha 0 2024-09-10T08:36:38.432 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Compile date 2024-09-09 20:43:47 2024-09-10T08:36:38.432 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: DB SUMMARY 2024-09-10T08:36:38.432 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: DB Session ID: 77ZT4CWPQD1VINIJFH53 2024-09-10T08:36:38.433 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: CURRENT file: CURRENT 2024-09-10T08:36:38.433 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T08:36:38.433 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T08:36:38.433 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T08:36:38.433 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104190 ; 2024-09-10T08:36:38.433 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.error_if_exists: 0 2024-09-10T08:36:38.433 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.create_if_missing: 0 2024-09-10T08:36:38.433 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.paranoid_checks: 1 2024-09-10T08:36:38.433 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T08:36:38.433 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T08:36:38.433 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T08:36:38.433 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.env: 0x55dc37515940 2024-09-10T08:36:38.433 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T08:36:38.434 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.info_log: 0x55dc39b496e0 2024-09-10T08:36:38.434 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T08:36:38.434 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.statistics: (nil) 2024-09-10T08:36:38.434 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.use_fsync: 0 2024-09-10T08:36:38.434 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_log_file_size: 0 2024-09-10T08:36:38.434 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T08:36:38.434 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T08:36:38.434 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T08:36:38.434 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T08:36:38.434 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.allow_fallocate: 1 2024-09-10T08:36:38.434 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T08:36:38.434 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T08:36:38.434 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.use_direct_reads: 0 2024-09-10T08:36:38.435 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T08:36:38.435 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T08:36:38.435 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.db_log_dir: 2024-09-10T08:36:38.435 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.wal_dir: 2024-09-10T08:36:38.435 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T08:36:38.436 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T08:36:38.436 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T08:36:38.436 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T08:36:38.437 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T08:36:38.437 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T08:36:38.437 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T08:36:38.437 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T08:36:38.437 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.write_buffer_manager: 0x55dc39b58dc0 2024-09-10T08:36:38.437 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T08:36:38.437 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T08:36:38.437 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T08:36:38.437 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T08:36:38.437 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T08:36:38.437 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T08:36:38.437 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T08:36:38.438 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T08:36:38.438 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.unordered_write: 0 2024-09-10T08:36:38.438 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T08:36:38.438 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T08:36:38.438 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T08:36:38.438 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T08:36:38.438 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.row_cache: None 2024-09-10T08:36:38.438 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.wal_filter: None 2024-09-10T08:36:38.438 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T08:36:38.438 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T08:36:38.438 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.two_write_queues: 0 2024-09-10T08:36:38.438 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T08:36:38.438 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.wal_compression: 0 2024-09-10T08:36:38.439 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.atomic_flush: 0 2024-09-10T08:36:38.439 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T08:36:38.439 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T08:36:38.439 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T08:36:38.439 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.log_readahead_size: 0 2024-09-10T08:36:38.439 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T08:36:38.439 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T08:36:38.439 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T08:36:38.439 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T08:36:38.439 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T08:36:38.439 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T08:36:38.439 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T08:36:38.439 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_background_jobs: 2 2024-09-10T08:36:38.440 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_background_compactions: -1 2024-09-10T08:36:38.440 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_subcompactions: 1 2024-09-10T08:36:38.440 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T08:36:38.440 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T08:36:38.440 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T08:36:38.440 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T08:36:38.440 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T08:36:38.440 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T08:36:38.440 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T08:36:38.440 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T08:36:38.440 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_open_files: -1 2024-09-10T08:36:38.440 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T08:36:38.440 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T08:36:38.441 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T08:36:38.441 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T08:36:38.441 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_background_flushes: -1 2024-09-10T08:36:38.441 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Compression algorithms supported: 2024-09-10T08:36:38.441 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: kZSTD supported: 0 2024-09-10T08:36:38.441 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: kXpressCompression supported: 0 2024-09-10T08:36:38.441 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: kBZip2Compression supported: 0 2024-09-10T08:36:38.441 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T08:36:38.441 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: kLZ4Compression supported: 1 2024-09-10T08:36:38.441 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: kZlibCompression supported: 1 2024-09-10T08:36:38.441 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T08:36:38.441 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: kSnappyCompression supported: 1 2024-09-10T08:36:38.441 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T08:36:38.442 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T08:36:38.442 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-10T08:36:38.442 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T08:36:38.442 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T08:36:38.442 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.merge_operator: 2024-09-10T08:36:38.442 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compaction_filter: None 2024-09-10T08:36:38.442 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T08:36:38.442 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T08:36:38.442 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T08:36:38.442 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T08:36:38.442 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55dc39b49320) 2024-09-10T08:36:38.442 INFO:journalctl@ceph.mon.a.smithi172.stdout: cache_index_and_filter_blocks: 1 2024-09-10T08:36:38.442 INFO:journalctl@ceph.mon.a.smithi172.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T08:36:38.443 INFO:journalctl@ceph.mon.a.smithi172.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T08:36:38.443 INFO:journalctl@ceph.mon.a.smithi172.stdout: pin_top_level_index_and_filter: 1 2024-09-10T08:36:38.443 INFO:journalctl@ceph.mon.a.smithi172.stdout: index_type: 0 2024-09-10T08:36:38.443 INFO:journalctl@ceph.mon.a.smithi172.stdout: data_block_index_type: 0 2024-09-10T08:36:38.443 INFO:journalctl@ceph.mon.a.smithi172.stdout: index_shortening: 1 2024-09-10T08:36:38.443 INFO:journalctl@ceph.mon.a.smithi172.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T08:36:38.443 INFO:journalctl@ceph.mon.a.smithi172.stdout: checksum: 4 2024-09-10T08:36:38.443 INFO:journalctl@ceph.mon.a.smithi172.stdout: no_block_cache: 0 2024-09-10T08:36:38.443 INFO:journalctl@ceph.mon.a.smithi172.stdout: block_cache: 0x55dc39b3d1f0 2024-09-10T08:36:38.443 INFO:journalctl@ceph.mon.a.smithi172.stdout: block_cache_name: BinnedLRUCache 2024-09-10T08:36:38.443 INFO:journalctl@ceph.mon.a.smithi172.stdout: block_cache_options: 2024-09-10T08:36:38.443 INFO:journalctl@ceph.mon.a.smithi172.stdout: capacity : 536870912 2024-09-10T08:36:38.443 INFO:journalctl@ceph.mon.a.smithi172.stdout: num_shard_bits : 4 2024-09-10T08:36:38.444 INFO:journalctl@ceph.mon.a.smithi172.stdout: strict_capacity_limit : 0 2024-09-10T08:36:38.444 INFO:journalctl@ceph.mon.a.smithi172.stdout: high_pri_pool_ratio: 0.000 2024-09-10T08:36:38.444 INFO:journalctl@ceph.mon.a.smithi172.stdout: block_cache_compressed: (nil) 2024-09-10T08:36:38.444 INFO:journalctl@ceph.mon.a.smithi172.stdout: persistent_cache: (nil) 2024-09-10T08:36:38.444 INFO:journalctl@ceph.mon.a.smithi172.stdout: block_size: 4096 2024-09-10T08:36:38.444 INFO:journalctl@ceph.mon.a.smithi172.stdout: block_size_deviation: 10 2024-09-10T08:36:38.444 INFO:journalctl@ceph.mon.a.smithi172.stdout: block_restart_interval: 16 2024-09-10T08:36:38.444 INFO:journalctl@ceph.mon.a.smithi172.stdout: index_block_restart_interval: 1 2024-09-10T08:36:38.444 INFO:journalctl@ceph.mon.a.smithi172.stdout: metadata_block_size: 4096 2024-09-10T08:36:38.444 INFO:journalctl@ceph.mon.a.smithi172.stdout: partition_filters: 0 2024-09-10T08:36:38.444 INFO:journalctl@ceph.mon.a.smithi172.stdout: use_delta_encoding: 1 2024-09-10T08:36:38.444 INFO:journalctl@ceph.mon.a.smithi172.stdout: filter_policy: bloomfilter 2024-09-10T08:36:38.444 INFO:journalctl@ceph.mon.a.smithi172.stdout: whole_key_filtering: 1 2024-09-10T08:36:38.445 INFO:journalctl@ceph.mon.a.smithi172.stdout: verify_compression: 0 2024-09-10T08:36:38.445 INFO:journalctl@ceph.mon.a.smithi172.stdout: read_amp_bytes_per_bit: 0 2024-09-10T08:36:38.445 INFO:journalctl@ceph.mon.a.smithi172.stdout: format_version: 5 2024-09-10T08:36:38.445 INFO:journalctl@ceph.mon.a.smithi172.stdout: enable_index_compression: 1 2024-09-10T08:36:38.445 INFO:journalctl@ceph.mon.a.smithi172.stdout: block_align: 0 2024-09-10T08:36:38.445 INFO:journalctl@ceph.mon.a.smithi172.stdout: max_auto_readahead_size: 262144 2024-09-10T08:36:38.445 INFO:journalctl@ceph.mon.a.smithi172.stdout: prepopulate_block_cache: 0 2024-09-10T08:36:38.445 INFO:journalctl@ceph.mon.a.smithi172.stdout: initial_auto_readahead_size: 8192 2024-09-10T08:36:38.445 INFO:journalctl@ceph.mon.a.smithi172.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T08:36:38.445 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T08:36:38.445 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T08:36:38.445 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compression: NoCompression 2024-09-10T08:36:38.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T08:36:38.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T08:36:38.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T08:36:38.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.num_levels: 7 2024-09-10T08:36:38.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T08:36:38.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T08:36:38.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T08:36:38.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T08:36:38.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T08:36:38.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T08:36:38.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T08:36:38.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T08:36:38.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T08:36:38.447 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T08:36:38.447 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T08:36:38.447 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T08:36:38.447 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T08:36:38.447 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T08:36:38.447 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T08:36:38.447 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T08:36:38.447 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T08:36:38.447 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T08:36:38.447 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T08:36:38.447 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T08:36:38.447 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T08:36:38.447 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T08:36:38.448 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T08:36:38.448 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T08:36:38.448 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T08:36:38.448 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T08:36:38.448 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T08:36:38.448 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T08:36:38.448 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T08:36:38.448 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T08:36:38.448 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T08:36:38.448 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T08:36:38.448 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T08:36:38.448 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T08:36:38.449 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T08:36:38.449 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T08:36:38.449 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T08:36:38.449 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T08:36:38.449 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T08:36:38.449 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T08:36:38.449 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T08:36:38.449 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T08:36:38.449 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T08:36:38.449 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T08:36:38.449 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T08:36:38.449 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T08:36:38.449 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T08:36:38.449 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T08:36:38.450 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T08:36:38.450 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T08:36:38.450 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T08:36:38.450 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T08:36:38.450 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T08:36:38.450 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T08:36:38.450 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.inplace_update_support: 0 2024-09-10T08:36:38.450 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T08:36:38.450 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T08:36:38.450 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T08:36:38.450 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T08:36:38.450 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.bloom_locality: 0 2024-09-10T08:36:38.450 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.max_successive_merges: 0 2024-09-10T08:36:38.451 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T08:36:38.451 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T08:36:38.451 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T08:36:38.451 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T08:36:38.451 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.ttl: 2592000 2024-09-10T08:36:38.451 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T08:36:38.451 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T08:36:38.451 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T08:36:38.451 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.enable_blob_files: false 2024-09-10T08:36:38.451 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.min_blob_size: 0 2024-09-10T08:36:38.451 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T08:36:38.451 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T08:36:38.451 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T08:36:38.452 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T08:36:38.452 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T08:36:38.452 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T08:36:38.452 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T08:36:38.452 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T08:36:38.452 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T08:36:38.452 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T08:36:38.452 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0fae2531-2e9e-4422-aa28-e4c490780a0f 2024-09-10T08:36:38.452 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725957398101611, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T08:36:38.452 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T08:36:38.452 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725957398103157, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99420, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97320, "index_size": 292, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14030, "raw_average_key_size": 49, "raw_value_size": 89655, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725957398, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0fae2531-2e9e-4422-aa28-e4c490780a0f", "db_session_id": "77ZT4CWPQD1VINIJFH53", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T08:36:38.452 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725957398103285, "job": 1, "event": "recovery_finished"} 2024-09-10T08:36:38.452 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T08:36:38.453 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T08:36:38.453 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55dc39b6ae00 2024-09-10T08:36:38.453 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: DB pointer 0x55dc39c66000 2024-09-10T08:36:38.453 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T08:36:38.453 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T08:36:38.453 INFO:journalctl@ceph.mon.a.smithi172.stdout: ** DB Stats ** 2024-09-10T08:36:38.453 INFO:journalctl@ceph.mon.a.smithi172.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T08:36:38.453 INFO:journalctl@ceph.mon.a.smithi172.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T08:36:38.453 INFO:journalctl@ceph.mon.a.smithi172.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T08:36:38.453 INFO:journalctl@ceph.mon.a.smithi172.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T08:36:38.453 INFO:journalctl@ceph.mon.a.smithi172.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T08:36:38.453 INFO:journalctl@ceph.mon.a.smithi172.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T08:36:38.453 INFO:journalctl@ceph.mon.a.smithi172.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T08:36:38.454 INFO:journalctl@ceph.mon.a.smithi172.stdout: 2024-09-10T08:36:38.454 INFO:journalctl@ceph.mon.a.smithi172.stdout: ** Compaction Stats [default] ** 2024-09-10T08:36:38.454 INFO:journalctl@ceph.mon.a.smithi172.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T08:36:38.454 INFO:journalctl@ceph.mon.a.smithi172.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T08:36:38.454 INFO:journalctl@ceph.mon.a.smithi172.stdout: L0 2/0 98.95 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 100.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T08:36:38.454 INFO:journalctl@ceph.mon.a.smithi172.stdout: Sum 2/0 98.95 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 100.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T08:36:38.454 INFO:journalctl@ceph.mon.a.smithi172.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 100.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T08:36:38.454 INFO:journalctl@ceph.mon.a.smithi172.stdout: 2024-09-10T08:36:38.454 INFO:journalctl@ceph.mon.a.smithi172.stdout: ** Compaction Stats [default] ** 2024-09-10T08:36:38.454 INFO:journalctl@ceph.mon.a.smithi172.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T08:36:38.454 INFO:journalctl@ceph.mon.a.smithi172.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T08:36:38.454 INFO:journalctl@ceph.mon.a.smithi172.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 100.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T08:36:38.454 INFO:journalctl@ceph.mon.a.smithi172.stdout: 2024-09-10T08:36:38.455 INFO:journalctl@ceph.mon.a.smithi172.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T08:36:38.455 INFO:journalctl@ceph.mon.a.smithi172.stdout: 2024-09-10T08:36:38.455 INFO:journalctl@ceph.mon.a.smithi172.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T08:36:38.455 INFO:journalctl@ceph.mon.a.smithi172.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T08:36:38.455 INFO:journalctl@ceph.mon.a.smithi172.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T08:36:38.455 INFO:journalctl@ceph.mon.a.smithi172.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T08:36:38.455 INFO:journalctl@ceph.mon.a.smithi172.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T08:36:38.455 INFO:journalctl@ceph.mon.a.smithi172.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T08:36:38.455 INFO:journalctl@ceph.mon.a.smithi172.stdout: Cumulative compaction: 0.00 GB write, 21.03 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T08:36:38.455 INFO:journalctl@ceph.mon.a.smithi172.stdout: Interval compaction: 0.00 GB write, 21.03 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T08:36:38.455 INFO:journalctl@ceph.mon.a.smithi172.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T08:36:38.455 INFO:journalctl@ceph.mon.a.smithi172.stdout: Block cache BinnedLRUCache@0x55dc39b3d1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-10T08:36:38.455 INFO:journalctl@ceph.mon.a.smithi172.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-10T08:36:38.456 INFO:journalctl@ceph.mon.a.smithi172.stdout: 2024-09-10T08:36:38.456 INFO:journalctl@ceph.mon.a.smithi172.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T08:36:38.456 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: starting mon.a rank 0 at public addrs [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] at bind addrs [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:36:38.456 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: mon.a@-1(???) e1 preinit fsid a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:36:38.456 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: mon.a@-1(???).mds e1 new map 2024-09-10T08:36:38.456 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: mon.a@-1(???).mds e1 print_map 2024-09-10T08:36:38.456 INFO:journalctl@ceph.mon.a.smithi172.stdout: e1 2024-09-10T08:36:38.456 INFO:journalctl@ceph.mon.a.smithi172.stdout: btime 2024-09-10T08:36:33:083320+0000 2024-09-10T08:36:38.456 INFO:journalctl@ceph.mon.a.smithi172.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T08:36:38.456 INFO:journalctl@ceph.mon.a.smithi172.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T08:36:38.456 INFO:journalctl@ceph.mon.a.smithi172.stdout: legacy client fscid: -1 2024-09-10T08:36:38.456 INFO:journalctl@ceph.mon.a.smithi172.stdout: 2024-09-10T08:36:38.457 INFO:journalctl@ceph.mon.a.smithi172.stdout: No filesystems configured 2024-09-10T08:36:38.457 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T08:36:38.457 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T08:36:38.457 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T08:36:38.457 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T08:36:38.457 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T08:36:38.457 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T08:36:38.457 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: monmap epoch 1 2024-09-10T08:36:38.457 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: fsid a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:36:38.457 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: last_changed 2024-09-10T08:36:29.338346+0000 2024-09-10T08:36:38.457 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: created 2024-09-10T08:36:29.338346+0000 2024-09-10T08:36:38.457 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: min_mon_release 19 (squid) 2024-09-10T08:36:38.457 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: election_strategy: 1 2024-09-10T08:36:38.458 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: 0: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.a 2024-09-10T08:36:38.458 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: fsmap 2024-09-10T08:36:38.458 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T08:36:38.458 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 ceph-mon[30854]: mgrmap e1: no daemons active 2024-09-10T08:36:38.458 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 bash[30835]: 4ca591a9b8a003ee6f1e3edc34e471234001816eb1d1bd40e33699195fa18d1e 2024-09-10T08:36:38.458 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:38 smithi172 systemd[1]: Started Ceph mon.a for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:36:39.310 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.621+0000 7f7de9773640 1 Processor -- start 2024-09-10T08:36:39.310 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.621+0000 7f7de9773640 1 -- start start 2024-09-10T08:36:39.310 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.621+0000 7f7de9773640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7de4107f10 0x7f7de410a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:39.310 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.621+0000 7f7de9773640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7de407be30 con 0x7f7de4107f10 2024-09-10T08:36:39.310 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.621+0000 7f7de2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7de4107f10 0x7f7de410a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:39.310 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.621+0000 7f7de2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7de4107f10 0x7f7de410a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36484/0 (socket says 172.21.15.172:36484) 2024-09-10T08:36:39.310 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.621+0000 7f7de2ffd640 1 -- 172.21.15.172:0/3634055482 learned_addr learned my addr 172.21.15.172:0/3634055482 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:39.310 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.622+0000 7f7de2ffd640 1 -- 172.21.15.172:0/3634055482 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de410a840 con 0x7f7de4107f10 2024-09-10T08:36:39.310 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.623+0000 7f7de2ffd640 1 --2- 172.21.15.172:0/3634055482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7de4107f10 0x7f7de410a300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009820 tx=0x7f7dcc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7428f5b709956b4b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:39.310 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.624+0000 7f7de1ffb640 1 -- 172.21.15.172:0/3634055482 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7dcc037070 con 0x7f7de4107f10 2024-09-10T08:36:39.311 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.624+0000 7f7de1ffb640 1 -- 172.21.15.172:0/3634055482 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7dcc0328d0 con 0x7f7de4107f10 2024-09-10T08:36:39.311 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.624+0000 7f7de1ffb640 1 -- 172.21.15.172:0/3634055482 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7dcc0307d0 con 0x7f7de4107f10 2024-09-10T08:36:39.311 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.624+0000 7f7de9773640 1 -- 172.21.15.172:0/3634055482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7de4107f10 msgr2=0x7f7de410a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:39.311 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.624+0000 7f7de9773640 1 --2- 172.21.15.172:0/3634055482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7de4107f10 0x7f7de410a300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009820 tx=0x7f7dcc02a270 comp rx=0 tx=0).stop 2024-09-10T08:36:39.311 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.624+0000 7f7de9773640 1 -- 172.21.15.172:0/3634055482 shutdown_connections 2024-09-10T08:36:39.311 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.624+0000 7f7de9773640 1 --2- 172.21.15.172:0/3634055482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7de4107f10 0x7f7de410a300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:39.311 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.624+0000 7f7de9773640 1 -- 172.21.15.172:0/3634055482 >> 172.21.15.172:0/3634055482 conn(0x7f7de4101b90 msgr2=0x7f7de4103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:39.311 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.624+0000 7f7de9773640 1 -- 172.21.15.172:0/3634055482 shutdown_connections 2024-09-10T08:36:39.311 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.624+0000 7f7de9773640 1 -- 172.21.15.172:0/3634055482 wait complete. 2024-09-10T08:36:39.311 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.625+0000 7f7de9773640 1 Processor -- start 2024-09-10T08:36:39.311 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.625+0000 7f7de9773640 1 -- start start 2024-09-10T08:36:39.311 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.626+0000 7f7de9773640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7de4107f10 0x7f7de419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:39.311 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.626+0000 7f7de9773640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7de410b880 con 0x7f7de4107f10 2024-09-10T08:36:39.312 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.626+0000 7f7de2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7de4107f10 0x7f7de419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:39.312 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.626+0000 7f7de2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7de4107f10 0x7f7de419b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36496/0 (socket says 172.21.15.172:36496) 2024-09-10T08:36:39.312 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.626+0000 7f7de2ffd640 1 -- 172.21.15.172:0/2328775958 learned_addr learned my addr 172.21.15.172:0/2328775958 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:39.312 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.626+0000 7f7de2ffd640 1 -- 172.21.15.172:0/2328775958 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de419b920 con 0x7f7de4107f10 2024-09-10T08:36:39.312 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.626+0000 7f7de2ffd640 1 --2- 172.21.15.172:0/2328775958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7de4107f10 0x7f7de419b3e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc032e10 tx=0x7f7dcc032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:39.312 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.626+0000 7f7ddbfff640 1 -- 172.21.15.172:0/2328775958 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7dcc042070 con 0x7f7de4107f10 2024-09-10T08:36:39.312 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.626+0000 7f7ddbfff640 1 -- 172.21.15.172:0/2328775958 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f7dcc030c80 con 0x7f7de4107f10 2024-09-10T08:36:39.312 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.626+0000 7f7ddbfff640 1 -- 172.21.15.172:0/2328775958 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7dcc037040 con 0x7f7de4107f10 2024-09-10T08:36:39.312 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.626+0000 7f7de9773640 1 -- 172.21.15.172:0/2328775958 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de419bbb0 con 0x7f7de4107f10 2024-09-10T08:36:39.312 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.627+0000 7f7de9773640 1 -- 172.21.15.172:0/2328775958 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de419e8a0 con 0x7f7de4107f10 2024-09-10T08:36:39.312 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.627+0000 7f7ddbfff640 1 -- 172.21.15.172:0/2328775958 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7dcc045070 con 0x7f7de4107f10 2024-09-10T08:36:39.312 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.627+0000 7f7ddbfff640 1 -- 172.21.15.172:0/2328775958 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7dcc0479f0 con 0x7f7de4107f10 2024-09-10T08:36:39.313 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.628+0000 7f7de9773640 1 -- 172.21.15.172:0/2328775958 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de4108370 con 0x7f7de4107f10 2024-09-10T08:36:39.313 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.630+0000 7f7ddbfff640 1 -- 172.21.15.172:0/2328775958 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7dcc03c5f0 con 0x7f7de4107f10 2024-09-10T08:36:39.313 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.686+0000 7f7de9773640 1 -- 172.21.15.172:0/2328775958 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f7de419ea70 con 0x7f7de4107f10 2024-09-10T08:36:39.313 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.688+0000 7f7ddbfff640 1 -- 172.21.15.172:0/2328775958 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f7dcc03c7d0 con 0x7f7de4107f10 2024-09-10T08:36:39.313 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.689+0000 7f7de9773640 1 -- 172.21.15.172:0/2328775958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7de4107f10 msgr2=0x7f7de419b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:39.313 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.689+0000 7f7de9773640 1 --2- 172.21.15.172:0/2328775958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7de4107f10 0x7f7de419b3e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc032e10 tx=0x7f7dcc032e40 comp rx=0 tx=0).stop 2024-09-10T08:36:39.313 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.689+0000 7f7de9773640 1 -- 172.21.15.172:0/2328775958 shutdown_connections 2024-09-10T08:36:39.313 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.689+0000 7f7de9773640 1 --2- 172.21.15.172:0/2328775958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7de4107f10 0x7f7de419b3e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:39.313 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.689+0000 7f7de9773640 1 -- 172.21.15.172:0/2328775958 >> 172.21.15.172:0/2328775958 conn(0x7f7de4101b90 msgr2=0x7f7de4102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:39.313 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.689+0000 7f7de9773640 1 -- 172.21.15.172:0/2328775958 shutdown_connections 2024-09-10T08:36:39.313 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:38.689+0000 7f7de9773640 1 -- 172.21.15.172:0/2328775958 wait complete. 2024-09-10T08:36:39.313 INFO:teuthology.orchestra.run.smithi172.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T08:36:39.328 INFO:teuthology.orchestra.run.smithi172.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:36:39.328 INFO:teuthology.orchestra.run.smithi172.stdout:Creating mgr... 2024-09-10T08:36:39.328 INFO:teuthology.orchestra.run.smithi172.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T08:36:39.328 INFO:teuthology.orchestra.run.smithi172.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T08:36:39.576 INFO:teuthology.orchestra.run.smithi172.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mgr.x 2024-09-10T08:36:39.576 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-10T08:36:39.727 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a7dba522-6f4f-11ef-bcea-c7b262605968.target.wants/ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@.service. 2024-09-10T08:36:39.956 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:39 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2328775958' entity='client.admin' 2024-09-10T08:36:39.957 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:39 smithi172 systemd[1]: Starting Ceph mgr.x for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T08:36:40.266 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:39 smithi172 podman[31064]: 2024-09-10 08:36:39.859190648 +0000 UTC m=+0.015722062 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:36:40.266 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:39 smithi172 podman[31064]: 2024-09-10 08:36:39.972521016 +0000 UTC m=+0.129052424 container create 381553d8692dd7997969ccd5686d607cbc12fd57bdc01b65d77572f57e130d63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T08:36:40.266 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:40 smithi172 podman[31064]: 2024-09-10 08:36:40.18319673 +0000 UTC m=+0.339728138 container init 381553d8692dd7997969ccd5686d607cbc12fd57bdc01b65d77572f57e130d63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T08:36:40.266 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:40 smithi172 podman[31064]: 2024-09-10 08:36:40.186245317 +0000 UTC m=+0.342776725 container start 381553d8692dd7997969ccd5686d607cbc12fd57bdc01b65d77572f57e130d63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:36:40.267 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:40 smithi172 bash[31064]: 381553d8692dd7997969ccd5686d607cbc12fd57bdc01b65d77572f57e130d63 2024-09-10T08:36:40.280 INFO:teuthology.orchestra.run.smithi172.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T08:36:40.280 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stdout disabled 2024-09-10T08:36:40.288 INFO:teuthology.orchestra.run.smithi172.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T08:36:40.288 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stdout inactive 2024-09-10T08:36:40.288 INFO:teuthology.orchestra.run.smithi172.stdout:firewalld.service is not enabled 2024-09-10T08:36:40.288 INFO:teuthology.orchestra.run.smithi172.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T08:36:40.296 INFO:teuthology.orchestra.run.smithi172.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T08:36:40.297 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stdout disabled 2024-09-10T08:36:40.304 INFO:teuthology.orchestra.run.smithi172.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T08:36:40.304 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stdout inactive 2024-09-10T08:36:40.304 INFO:teuthology.orchestra.run.smithi172.stdout:firewalld.service is not enabled 2024-09-10T08:36:40.304 INFO:teuthology.orchestra.run.smithi172.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T08:36:40.304 INFO:teuthology.orchestra.run.smithi172.stdout:Waiting for mgr to start... 2024-09-10T08:36:40.304 INFO:teuthology.orchestra.run.smithi172.stdout:Waiting for mgr... 2024-09-10T08:36:40.520 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:40 smithi172 systemd[1]: Started Ceph mgr.x for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:36:40.520 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:40 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:40.307+0000 7f18fd724040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T08:36:40.521 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:40 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:40.367+0000 7f18fd724040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T08:36:40.777 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:40 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:40.519+0000 7f18fd724040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T08:36:40.778 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:40 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:40.586+0000 7f18fd724040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T08:36:40.778 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:40 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:40.741+0000 7f18fd724040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:36:41.081 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:40 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3586944841' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T08:36:41.081 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:40 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:40.890+0000 7f18fd724040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T08:36:41.331 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:41 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:41.210+0000 7f18fd724040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T08:36:41.332 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:41 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:41.271+0000 7f18fd724040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T08:36:41.378 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout 2024-09-10T08:36:41.378 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout { 2024-09-10T08:36:41.378 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:36:41.378 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T08:36:41.378 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T08:36:41.379 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T08:36:41.379 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T08:36:41.379 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:41.379 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T08:36:41.379 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T08:36:41.379 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout 0 2024-09-10T08:36:41.379 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:36:41.379 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T08:36:41.379 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T08:36:41.379 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:36:41.379 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-10T08:36:41.379 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T08:36:41.380 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:36:41.380 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T08:36:41.380 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T08:36:41.380 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:41.380 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T08:36:41.380 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:36:41.380 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T08:36:41.380 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T08:36:41.380 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T08:36:41.390 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T08:36:41.390 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T08:36:41.390 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T08:36:41.390 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:41.390 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T08:36:41.390 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T08:36:41.390 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T08:36:41.391 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T08:36:41.391 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T08:36:41.391 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T08:36:41.391 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T08:36:41.391 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T08:36:41.391 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T08:36:41.391 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:41.391 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T08:36:41.391 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:36:41.391 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T08:36:33:083320+0000", 2024-09-10T08:36:41.391 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T08:36:41.392 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T08:36:41.392 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:41.392 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T08:36:41.392 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T08:36:41.392 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T08:36:41.392 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T08:36:41.392 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T08:36:41.392 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T08:36:41.392 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T08:36:41.392 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:36:41.392 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T08:36:41.393 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:41.393 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T08:36:41.393 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:36:41.393 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T08:36:33.084084+0000", 2024-09-10T08:36:41.393 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T08:36:41.393 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:41.393 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T08:36:41.393 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout } 2024-09-10T08:36:41.393 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.706+0000 7f959900c640 1 Processor -- start 2024-09-10T08:36:41.393 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.707+0000 7f959900c640 1 -- start start 2024-09-10T08:36:41.393 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.707+0000 7f959900c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9594107f50 0x7f959410a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:41.394 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.707+0000 7f959900c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f95940758c0 con 0x7f9594107f50 2024-09-10T08:36:41.394 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.707+0000 7f9592d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9594107f50 0x7f959410a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:41.394 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.707+0000 7f9592d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9594107f50 0x7f959410a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36514/0 (socket says 172.21.15.172:36514) 2024-09-10T08:36:41.394 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.707+0000 7f9592d9d640 1 -- 172.21.15.172:0/1342953704 learned_addr learned my addr 172.21.15.172:0/1342953704 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:41.394 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.707+0000 7f9592d9d640 1 -- 172.21.15.172:0/1342953704 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959410a880 con 0x7f9594107f50 2024-09-10T08:36:41.394 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.708+0000 7f9592d9d640 1 --2- 172.21.15.172:0/1342953704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9594107f50 0x7f959410a340 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9584009820 tx=0x7f958402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9a39235bd581f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:41.394 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.708+0000 7f9591d9b640 1 -- 172.21.15.172:0/1342953704 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9584037070 con 0x7f9594107f50 2024-09-10T08:36:41.394 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.709+0000 7f9591d9b640 1 -- 172.21.15.172:0/1342953704 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f95840328d0 con 0x7f9594107f50 2024-09-10T08:36:41.394 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.709+0000 7f9591d9b640 1 -- 172.21.15.172:0/1342953704 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95840307d0 con 0x7f9594107f50 2024-09-10T08:36:41.394 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.709+0000 7f959900c640 1 -- 172.21.15.172:0/1342953704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9594107f50 msgr2=0x7f959410a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:41.395 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.709+0000 7f959900c640 1 --2- 172.21.15.172:0/1342953704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9594107f50 0x7f959410a340 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9584009820 tx=0x7f958402a270 comp rx=0 tx=0).stop 2024-09-10T08:36:41.395 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.709+0000 7f959900c640 1 -- 172.21.15.172:0/1342953704 shutdown_connections 2024-09-10T08:36:41.395 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.709+0000 7f959900c640 1 --2- 172.21.15.172:0/1342953704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9594107f50 0x7f959410a340 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:41.395 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.709+0000 7f959900c640 1 -- 172.21.15.172:0/1342953704 >> 172.21.15.172:0/1342953704 conn(0x7f9594101b10 msgr2=0x7f9594103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:41.395 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.709+0000 7f959900c640 1 -- 172.21.15.172:0/1342953704 shutdown_connections 2024-09-10T08:36:41.395 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.709+0000 7f959900c640 1 -- 172.21.15.172:0/1342953704 wait complete. 2024-09-10T08:36:41.395 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.710+0000 7f959900c640 1 Processor -- start 2024-09-10T08:36:41.396 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.710+0000 7f959900c640 1 -- start start 2024-09-10T08:36:41.396 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.710+0000 7f959900c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9594107f50 0x7f959419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:41.396 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.711+0000 7f959900c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f959410b860 con 0x7f9594107f50 2024-09-10T08:36:41.396 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.711+0000 7f9592d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9594107f50 0x7f959419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:41.396 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.711+0000 7f9592d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9594107f50 0x7f959419b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36520/0 (socket says 172.21.15.172:36520) 2024-09-10T08:36:41.396 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.711+0000 7f9592d9d640 1 -- 172.21.15.172:0/3586944841 learned_addr learned my addr 172.21.15.172:0/3586944841 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:41.396 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.711+0000 7f9592d9d640 1 -- 172.21.15.172:0/3586944841 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959419b980 con 0x7f9594107f50 2024-09-10T08:36:41.396 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.711+0000 7f9592d9d640 1 --2- 172.21.15.172:0/3586944841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9594107f50 0x7f959419b440 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f95840097f0 tx=0x7f9584030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:41.396 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.711+0000 7f957bfff640 1 -- 172.21.15.172:0/3586944841 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9584037040 con 0x7f9594107f50 2024-09-10T08:36:41.396 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.711+0000 7f957bfff640 1 -- 172.21.15.172:0/3586944841 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f958403b3f0 con 0x7f9594107f50 2024-09-10T08:36:41.396 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.711+0000 7f959900c640 1 -- 172.21.15.172:0/3586944841 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959419bc10 con 0x7f9594107f50 2024-09-10T08:36:41.396 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.711+0000 7f957bfff640 1 -- 172.21.15.172:0/3586944841 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f958403a520 con 0x7f9594107f50 2024-09-10T08:36:41.396 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.711+0000 7f959900c640 1 -- 172.21.15.172:0/3586944841 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959419e900 con 0x7f9594107f50 2024-09-10T08:36:41.397 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.712+0000 7f957bfff640 1 -- 172.21.15.172:0/3586944841 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f958403a6c0 con 0x7f9594107f50 2024-09-10T08:36:41.397 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.712+0000 7f957bfff640 1 -- 172.21.15.172:0/3586944841 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9584044b00 con 0x7f9594107f50 2024-09-10T08:36:41.397 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.712+0000 7f959900c640 1 -- 172.21.15.172:0/3586944841 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95941083b0 con 0x7f9594107f50 2024-09-10T08:36:41.397 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.714+0000 7f957bfff640 1 -- 172.21.15.172:0/3586944841 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f958403b980 con 0x7f9594107f50 2024-09-10T08:36:41.397 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.776+0000 7f959900c640 1 -- 172.21.15.172:0/3586944841 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f959419ee70 con 0x7f9594107f50 2024-09-10T08:36:41.397 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.777+0000 7f957bfff640 1 -- 172.21.15.172:0/3586944841 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f958403b5a0 con 0x7f9594107f50 2024-09-10T08:36:41.397 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.778+0000 7f959900c640 1 -- 172.21.15.172:0/3586944841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9594107f50 msgr2=0x7f959419b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:41.397 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.778+0000 7f959900c640 1 --2- 172.21.15.172:0/3586944841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9594107f50 0x7f959419b440 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f95840097f0 tx=0x7f9584030d00 comp rx=0 tx=0).stop 2024-09-10T08:36:41.397 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.778+0000 7f959900c640 1 -- 172.21.15.172:0/3586944841 shutdown_connections 2024-09-10T08:36:41.397 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.778+0000 7f959900c640 1 --2- 172.21.15.172:0/3586944841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9594107f50 0x7f959419b440 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:41.397 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.778+0000 7f959900c640 1 -- 172.21.15.172:0/3586944841 >> 172.21.15.172:0/3586944841 conn(0x7f9594101b10 msgr2=0x7f9594102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:41.397 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.778+0000 7f959900c640 1 -- 172.21.15.172:0/3586944841 shutdown_connections 2024-09-10T08:36:41.397 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:40.778+0000 7f959900c640 1 -- 172.21.15.172:0/3586944841 wait complete. 2024-09-10T08:36:41.398 INFO:teuthology.orchestra.run.smithi172.stdout:mgr not available, waiting (1/15)... 2024-09-10T08:36:41.831 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:41 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:41.459+0000 7f18fd724040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T08:36:42.832 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:42 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:42.331+0000 7f18fd724040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T08:36:42.832 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:42 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:42.398+0000 7f18fd724040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T08:36:43.363 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:43 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:43.107+0000 7f18fd724040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T08:36:43.363 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:43 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:43.221+0000 7f18fd724040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T08:36:43.363 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:43 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:43.361+0000 7f18fd724040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:36:43.622 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:43 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:43.432+0000 7f18fd724040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T08:36:43.622 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:43 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:43.582+0000 7f18fd724040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T08:36:43.882 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:43 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:43.654+0000 7f18fd724040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T08:36:43.882 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:43 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:43.714+0000 7f18fd724040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T08:36:43.882 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:43 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:43.880+0000 7f18fd724040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T08:36:44.181 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:43 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/847104526' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T08:36:44.182 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:43 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:43.967+0000 7f18fd724040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T08:36:44.182 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:44 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:44.038+0000 7f18fd724040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T08:36:44.182 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:44 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:44.121+0000 7f18fd724040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T08:36:44.182 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:44 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:44.181+0000 7f18fd724040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T08:36:44.505 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:44 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:44.241+0000 7f18fd724040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T08:36:44.521 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout 2024-09-10T08:36:44.521 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout { 2024-09-10T08:36:44.521 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:36:44.521 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T08:36:44.522 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T08:36:44.522 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T08:36:44.522 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T08:36:44.522 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:44.522 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T08:36:44.522 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T08:36:44.522 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout 0 2024-09-10T08:36:44.522 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:36:44.522 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T08:36:44.523 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T08:36:44.523 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:36:44.523 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-10T08:36:44.523 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T08:36:44.523 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:36:44.523 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T08:36:44.523 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T08:36:44.523 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:44.524 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T08:36:44.524 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:36:44.524 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T08:36:44.524 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T08:36:44.524 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T08:36:44.524 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T08:36:44.524 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T08:36:44.524 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T08:36:44.524 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:44.524 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T08:36:44.524 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T08:36:44.524 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T08:36:44.524 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T08:36:44.525 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T08:36:44.525 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T08:36:44.525 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T08:36:44.525 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T08:36:44.525 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T08:36:44.525 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:44.525 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T08:36:44.525 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:36:44.525 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T08:36:33:083320+0000", 2024-09-10T08:36:44.525 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T08:36:44.525 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T08:36:44.525 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:44.526 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T08:36:44.526 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T08:36:44.526 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T08:36:44.526 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T08:36:44.526 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T08:36:44.526 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T08:36:44.526 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T08:36:44.526 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:36:44.526 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T08:36:44.526 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:44.526 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T08:36:44.526 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:36:44.527 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T08:36:33.084084+0000", 2024-09-10T08:36:44.527 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T08:36:44.527 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:44.527 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T08:36:44.527 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout } 2024-09-10T08:36:44.527 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.819+0000 7ff8b6db0640 1 Processor -- start 2024-09-10T08:36:44.527 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.819+0000 7ff8b6db0640 1 -- start start 2024-09-10T08:36:44.527 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.819+0000 7ff8b6db0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8b01063a0 0x7ff8b01067a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:44.527 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.819+0000 7ff8b6db0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0106d70 con 0x7ff8b01063a0 2024-09-10T08:36:44.527 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.819+0000 7ff8b4b4c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8b01063a0 0x7ff8b01067a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:44.527 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.819+0000 7ff8b4b4c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8b01063a0 0x7ff8b01067a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36524/0 (socket says 172.21.15.172:36524) 2024-09-10T08:36:44.527 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.819+0000 7ff8b4b4c640 1 -- 172.21.15.172:0/2835278526 learned_addr learned my addr 172.21.15.172:0/2835278526 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:44.528 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.820+0000 7ff8b4b4c640 1 -- 172.21.15.172:0/2835278526 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b01075a0 con 0x7ff8b01063a0 2024-09-10T08:36:44.528 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.820+0000 7ff8b4b4c640 1 --2- 172.21.15.172:0/2835278526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8b01063a0 0x7ff8b01067a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4009820 tx=0x7ff8a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4162e84b837711fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:44.528 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.820+0000 7ff8af7fe640 1 -- 172.21.15.172:0/2835278526 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8a4037070 con 0x7ff8b01063a0 2024-09-10T08:36:44.528 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.820+0000 7ff8af7fe640 1 -- 172.21.15.172:0/2835278526 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff8a40328d0 con 0x7ff8b01063a0 2024-09-10T08:36:44.528 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.820+0000 7ff8af7fe640 1 -- 172.21.15.172:0/2835278526 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8a40307d0 con 0x7ff8b01063a0 2024-09-10T08:36:44.528 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.821+0000 7ff8b6db0640 1 -- 172.21.15.172:0/2835278526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8b01063a0 msgr2=0x7ff8b01067a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:44.528 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.821+0000 7ff8b6db0640 1 --2- 172.21.15.172:0/2835278526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8b01063a0 0x7ff8b01067a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4009820 tx=0x7ff8a402a270 comp rx=0 tx=0).stop 2024-09-10T08:36:44.528 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.821+0000 7ff8b6db0640 1 -- 172.21.15.172:0/2835278526 shutdown_connections 2024-09-10T08:36:44.528 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.821+0000 7ff8b6db0640 1 --2- 172.21.15.172:0/2835278526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8b01063a0 0x7ff8b01067a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:44.528 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.821+0000 7ff8b6db0640 1 -- 172.21.15.172:0/2835278526 >> 172.21.15.172:0/2835278526 conn(0x7ff8b0101b10 msgr2=0x7ff8b0103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:44.528 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.821+0000 7ff8b6db0640 1 -- 172.21.15.172:0/2835278526 shutdown_connections 2024-09-10T08:36:44.528 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.821+0000 7ff8b6db0640 1 -- 172.21.15.172:0/2835278526 wait complete. 2024-09-10T08:36:44.529 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.822+0000 7ff8b6db0640 1 Processor -- start 2024-09-10T08:36:44.529 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.822+0000 7ff8b6db0640 1 -- start start 2024-09-10T08:36:44.529 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.822+0000 7ff8b6db0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8b01063a0 0x7ff8b019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:44.529 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.822+0000 7ff8b6db0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b010a950 con 0x7ff8b01063a0 2024-09-10T08:36:44.529 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.822+0000 7ff8b4b4c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8b01063a0 0x7ff8b019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:44.529 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.822+0000 7ff8b4b4c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8b01063a0 0x7ff8b019b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36530/0 (socket says 172.21.15.172:36530) 2024-09-10T08:36:44.529 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.822+0000 7ff8b4b4c640 1 -- 172.21.15.172:0/847104526 learned_addr learned my addr 172.21.15.172:0/847104526 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:44.529 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.822+0000 7ff8b4b4c640 1 -- 172.21.15.172:0/847104526 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b019b9f0 con 0x7ff8b01063a0 2024-09-10T08:36:44.529 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.823+0000 7ff8b4b4c640 1 --2- 172.21.15.172:0/847104526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8b01063a0 0x7ff8b019b4b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4030520 tx=0x7ff8a4032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:44.529 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.823+0000 7ff8adffb640 1 -- 172.21.15.172:0/847104526 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8a4042070 con 0x7ff8b01063a0 2024-09-10T08:36:44.529 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.823+0000 7ff8adffb640 1 -- 172.21.15.172:0/847104526 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff8a4030cc0 con 0x7ff8b01063a0 2024-09-10T08:36:44.530 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.823+0000 7ff8b6db0640 1 -- 172.21.15.172:0/847104526 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b019bc80 con 0x7ff8b01063a0 2024-09-10T08:36:44.530 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.823+0000 7ff8adffb640 1 -- 172.21.15.172:0/847104526 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8a4037040 con 0x7ff8b01063a0 2024-09-10T08:36:44.530 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.823+0000 7ff8b6db0640 1 -- 172.21.15.172:0/847104526 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b019e970 con 0x7ff8b01063a0 2024-09-10T08:36:44.530 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.823+0000 7ff8adffb640 1 -- 172.21.15.172:0/847104526 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff8a4045070 con 0x7ff8b01063a0 2024-09-10T08:36:44.530 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.823+0000 7ff8adffb640 1 -- 172.21.15.172:0/847104526 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff8a40479a0 con 0x7ff8b01063a0 2024-09-10T08:36:44.530 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.824+0000 7ff8b6db0640 1 -- 172.21.15.172:0/847104526 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b010aad0 con 0x7ff8b01063a0 2024-09-10T08:36:44.530 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.826+0000 7ff8adffb640 1 -- 172.21.15.172:0/847104526 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff8a403d9c0 con 0x7ff8b01063a0 2024-09-10T08:36:44.530 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.888+0000 7ff8b6db0640 1 -- 172.21.15.172:0/847104526 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff8b0195470 con 0x7ff8b01063a0 2024-09-10T08:36:44.530 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.888+0000 7ff8adffb640 1 -- 172.21.15.172:0/847104526 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7ff8a403d430 con 0x7ff8b01063a0 2024-09-10T08:36:44.530 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.889+0000 7ff8b6db0640 1 -- 172.21.15.172:0/847104526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8b01063a0 msgr2=0x7ff8b019b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:44.530 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.889+0000 7ff8b6db0640 1 --2- 172.21.15.172:0/847104526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8b01063a0 0x7ff8b019b4b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4030520 tx=0x7ff8a4032dd0 comp rx=0 tx=0).stop 2024-09-10T08:36:44.530 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.889+0000 7ff8b6db0640 1 -- 172.21.15.172:0/847104526 shutdown_connections 2024-09-10T08:36:44.531 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.889+0000 7ff8b6db0640 1 --2- 172.21.15.172:0/847104526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8b01063a0 0x7ff8b019b4b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:44.531 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.889+0000 7ff8b6db0640 1 -- 172.21.15.172:0/847104526 >> 172.21.15.172:0/847104526 conn(0x7ff8b0101b10 msgr2=0x7ff8b0102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:44.531 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.889+0000 7ff8b6db0640 1 -- 172.21.15.172:0/847104526 shutdown_connections 2024-09-10T08:36:44.531 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:43.890+0000 7ff8b6db0640 1 -- 172.21.15.172:0/847104526 wait complete. 2024-09-10T08:36:44.531 INFO:teuthology.orchestra.run.smithi172.stdout:mgr not available, waiting (2/15)... 2024-09-10T08:36:45.333 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:45 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:45.027+0000 7f18fd724040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T08:36:45.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:45 smithi172 ceph-mon[30854]: Activating manager daemon x 2024-09-10T08:36:45.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:45 smithi172 ceph-mon[30854]: mgrmap e2: x(active, starting, since 0.00315272s) 2024-09-10T08:36:45.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:45 smithi172 ceph-mon[30854]: from='mgr.14100 172.21.15.172:0/167329980' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T08:36:45.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:45 smithi172 ceph-mon[30854]: from='mgr.14100 172.21.15.172:0/167329980' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T08:36:45.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:45 smithi172 ceph-mon[30854]: from='mgr.14100 172.21.15.172:0/167329980' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T08:36:45.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:45 smithi172 ceph-mon[30854]: from='mgr.14100 172.21.15.172:0/167329980' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:36:45.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:45 smithi172 ceph-mon[30854]: from='mgr.14100 172.21.15.172:0/167329980' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T08:36:45.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:45 smithi172 ceph-mon[30854]: Manager daemon x is now available 2024-09-10T08:36:45.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:45 smithi172 ceph-mon[30854]: from='mgr.14100 172.21.15.172:0/167329980' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T08:36:45.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:45 smithi172 ceph-mon[30854]: from='mgr.14100 172.21.15.172:0/167329980' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T08:36:45.335 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:45 smithi172 ceph-mon[30854]: from='mgr.14100 172.21.15.172:0/167329980' entity='mgr.x' 2024-09-10T08:36:45.335 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:45 smithi172 ceph-mon[30854]: from='mgr.14100 172.21.15.172:0/167329980' entity='mgr.x' 2024-09-10T08:36:45.335 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:45 smithi172 ceph-mon[30854]: from='mgr.14100 172.21.15.172:0/167329980' entity='mgr.x' 2024-09-10T08:36:47.174 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:47 smithi172 ceph-mon[30854]: mgrmap e3: x(active, since 1.00672s) 2024-09-10T08:36:47.742 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout 2024-09-10T08:36:47.742 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout { 2024-09-10T08:36:47.742 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:36:47.742 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T08:36:47.742 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T08:36:47.742 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T08:36:47.742 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T08:36:47.743 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:47.743 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T08:36:47.743 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T08:36:47.743 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout 0 2024-09-10T08:36:47.743 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:36:47.743 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T08:36:47.743 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T08:36:47.743 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:36:47.743 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-10T08:36:47.743 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T08:36:47.743 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:36:47.743 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T08:36:47.743 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T08:36:47.744 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:47.744 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T08:36:47.744 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:36:47.744 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T08:36:47.744 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T08:36:47.744 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T08:36:47.744 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T08:36:47.744 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T08:36:47.744 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T08:36:47.744 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:47.744 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T08:36:47.744 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T08:36:47.745 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T08:36:47.745 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T08:36:47.745 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T08:36:47.745 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T08:36:47.745 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T08:36:47.745 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T08:36:47.745 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T08:36:47.745 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:47.745 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T08:36:47.745 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:36:47.745 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T08:36:33:083320+0000", 2024-09-10T08:36:47.745 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T08:36:47.745 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T08:36:47.746 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:47.746 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T08:36:47.746 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T08:36:47.746 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T08:36:47.746 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T08:36:47.746 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T08:36:47.746 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T08:36:47.746 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T08:36:47.746 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout ], 2024-09-10T08:36:47.746 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T08:36:47.746 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:47.746 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T08:36:47.746 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T08:36:47.747 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T08:36:33.084084+0000", 2024-09-10T08:36:47.747 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T08:36:47.747 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout }, 2024-09-10T08:36:47.747 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T08:36:47.747 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout } 2024-09-10T08:36:47.747 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.931+0000 7f621d33d640 1 Processor -- start 2024-09-10T08:36:47.747 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.931+0000 7f621d33d640 1 -- start start 2024-09-10T08:36:47.747 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.931+0000 7f621d33d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6218107f10 0x7f621810a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:47.747 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.931+0000 7f621d33d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6218075900 con 0x7f6218107f10 2024-09-10T08:36:47.747 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.932+0000 7f6216ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6218107f10 0x7f621810a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:47.747 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.932+0000 7f6216ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6218107f10 0x7f621810a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36406/0 (socket says 172.21.15.172:36406) 2024-09-10T08:36:47.747 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.932+0000 7f6216ffd640 1 -- 172.21.15.172:0/2407410278 learned_addr learned my addr 172.21.15.172:0/2407410278 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:47.748 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.932+0000 7f6216ffd640 1 -- 172.21.15.172:0/2407410278 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621810a840 con 0x7f6218107f10 2024-09-10T08:36:47.748 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.932+0000 7f6216ffd640 1 --2- 172.21.15.172:0/2407410278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6218107f10 0x7f621810a300 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6208009820 tx=0x7f620802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee9694840599e5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:47.748 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.933+0000 7f6215ffb640 1 -- 172.21.15.172:0/2407410278 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6208037070 con 0x7f6218107f10 2024-09-10T08:36:47.748 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.933+0000 7f6215ffb640 1 -- 172.21.15.172:0/2407410278 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f62080328d0 con 0x7f6218107f10 2024-09-10T08:36:47.748 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.933+0000 7f6215ffb640 1 -- 172.21.15.172:0/2407410278 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62080307d0 con 0x7f6218107f10 2024-09-10T08:36:47.748 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.933+0000 7f621d33d640 1 -- 172.21.15.172:0/2407410278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6218107f10 msgr2=0x7f621810a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:47.748 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.933+0000 7f621d33d640 1 --2- 172.21.15.172:0/2407410278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6218107f10 0x7f621810a300 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6208009820 tx=0x7f620802a270 comp rx=0 tx=0).stop 2024-09-10T08:36:47.748 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.933+0000 7f621d33d640 1 -- 172.21.15.172:0/2407410278 shutdown_connections 2024-09-10T08:36:47.748 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.933+0000 7f621d33d640 1 --2- 172.21.15.172:0/2407410278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6218107f10 0x7f621810a300 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:47.748 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.933+0000 7f621d33d640 1 -- 172.21.15.172:0/2407410278 >> 172.21.15.172:0/2407410278 conn(0x7f6218101b10 msgr2=0x7f6218103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:47.748 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.933+0000 7f621d33d640 1 -- 172.21.15.172:0/2407410278 shutdown_connections 2024-09-10T08:36:47.748 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.933+0000 7f621d33d640 1 -- 172.21.15.172:0/2407410278 wait complete. 2024-09-10T08:36:47.749 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.934+0000 7f621d33d640 1 Processor -- start 2024-09-10T08:36:47.749 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.934+0000 7f621d33d640 1 -- start start 2024-09-10T08:36:47.749 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.934+0000 7f621d33d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6218107f10 0x7f621819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:47.749 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.934+0000 7f621d33d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f621810b880 con 0x7f6218107f10 2024-09-10T08:36:47.749 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.935+0000 7f6216ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6218107f10 0x7f621819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:47.749 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.935+0000 7f6216ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6218107f10 0x7f621819b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36416/0 (socket says 172.21.15.172:36416) 2024-09-10T08:36:47.749 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.935+0000 7f6216ffd640 1 -- 172.21.15.172:0/922869672 learned_addr learned my addr 172.21.15.172:0/922869672 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:47.749 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.935+0000 7f6216ffd640 1 -- 172.21.15.172:0/922869672 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621819b980 con 0x7f6218107f10 2024-09-10T08:36:47.749 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.935+0000 7f6216ffd640 1 --2- 172.21.15.172:0/922869672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6218107f10 0x7f621819b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6208030520 tx=0x7f6208032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:47.749 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.935+0000 7f61fffff640 1 -- 172.21.15.172:0/922869672 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6208042070 con 0x7f6218107f10 2024-09-10T08:36:47.749 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.935+0000 7f621d33d640 1 -- 172.21.15.172:0/922869672 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621819bc10 con 0x7f6218107f10 2024-09-10T08:36:47.749 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.935+0000 7f61fffff640 1 -- 172.21.15.172:0/922869672 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6208030cc0 con 0x7f6218107f10 2024-09-10T08:36:47.750 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.935+0000 7f61fffff640 1 -- 172.21.15.172:0/922869672 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6208037040 con 0x7f6218107f10 2024-09-10T08:36:47.750 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.935+0000 7f621d33d640 1 -- 172.21.15.172:0/922869672 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621819e900 con 0x7f6218107f10 2024-09-10T08:36:47.750 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.936+0000 7f61fffff640 1 -- 172.21.15.172:0/922869672 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f6208045070 con 0x7f6218107f10 2024-09-10T08:36:47.750 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.936+0000 7f61fffff640 1 --2- 172.21.15.172:0/922869672 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f6200038e10 0x7f620003b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:47.750 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.936+0000 7f621d33d640 1 -- 172.21.15.172:0/922869672 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6218108370 con 0x7f6218107f10 2024-09-10T08:36:47.750 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.937+0000 7f61fffff640 1 -- 172.21.15.172:0/922869672 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f620806be90 con 0x7f6218107f10 2024-09-10T08:36:47.750 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.937+0000 7f62167fc640 1 --2- 172.21.15.172:0/922869672 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f6200038e10 0x7f620003b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:47.750 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.937+0000 7f62167fc640 1 --2- 172.21.15.172:0/922869672 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f6200038e10 0x7f620003b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f620c0098c0 tx=0x7f620c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:47.750 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:46.942+0000 7f61fffff640 1 -- 172.21.15.172:0/922869672 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6208045360 con 0x7f6218107f10 2024-09-10T08:36:47.750 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:47.036+0000 7f61fffff640 1 -- 172.21.15.172:0/922869672 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f620803b330 con 0x7f6218107f10 2024-09-10T08:36:47.750 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:47.173+0000 7f621d33d640 1 -- 172.21.15.172:0/922869672 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f621819eee0 con 0x7f6218107f10 2024-09-10T08:36:47.750 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:47.173+0000 7f61fffff640 1 -- 172.21.15.172:0/922869672 <== mon.0 v2:172.21.15.172:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f6208042de0 con 0x7f6218107f10 2024-09-10T08:36:47.751 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:47.176+0000 7f621d33d640 1 -- 172.21.15.172:0/922869672 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f6200038e10 msgr2=0x7f620003b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:47.751 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:47.176+0000 7f621d33d640 1 --2- 172.21.15.172:0/922869672 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f6200038e10 0x7f620003b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f620c0098c0 tx=0x7f620c006db0 comp rx=0 tx=0).stop 2024-09-10T08:36:47.751 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:47.176+0000 7f621d33d640 1 -- 172.21.15.172:0/922869672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6218107f10 msgr2=0x7f621819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:47.751 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:47.176+0000 7f621d33d640 1 --2- 172.21.15.172:0/922869672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6218107f10 0x7f621819b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6208030520 tx=0x7f6208032dd0 comp rx=0 tx=0).stop 2024-09-10T08:36:47.751 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:47.176+0000 7f621d33d640 1 -- 172.21.15.172:0/922869672 shutdown_connections 2024-09-10T08:36:47.751 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:47.176+0000 7f621d33d640 1 --2- 172.21.15.172:0/922869672 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f6200038e10 0x7f620003b2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:47.751 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:47.176+0000 7f621d33d640 1 --2- 172.21.15.172:0/922869672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6218107f10 0x7f621819b440 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:47.751 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:47.176+0000 7f621d33d640 1 -- 172.21.15.172:0/922869672 >> 172.21.15.172:0/922869672 conn(0x7f6218101b10 msgr2=0x7f62181026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:47.751 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:47.177+0000 7f621d33d640 1 -- 172.21.15.172:0/922869672 shutdown_connections 2024-09-10T08:36:47.751 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:47.177+0000 7f621d33d640 1 -- 172.21.15.172:0/922869672 wait complete. 2024-09-10T08:36:47.751 INFO:teuthology.orchestra.run.smithi172.stdout:mgr is available 2024-09-10T08:36:48.325 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:48 smithi172 ceph-mon[30854]: mgrmap e4: x(active, since 2s) 2024-09-10T08:36:48.325 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:48 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/922869672' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T08:36:49.027 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout 2024-09-10T08:36:49.027 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T08:36:49.027 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout fsid = a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:36:49.027 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.172:3300,v1:172.21.15.172:6789] 2024-09-10T08:36:49.027 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T08:36:49.027 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T08:36:49.027 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T08:36:49.028 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T08:36:49.028 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout 2024-09-10T08:36:49.028 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T08:36:49.028 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T08:36:49.028 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout 2024-09-10T08:36:49.028 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T08:36:49.029 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-09-10T08:36:49.029 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T08:36:49.029 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T08:36:49.029 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T08:36:49.029 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T08:36:49.029 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T08:36:49.029 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.150+0000 7f5ab8848640 1 Processor -- start 2024-09-10T08:36:49.029 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.150+0000 7f5ab8848640 1 -- start start 2024-09-10T08:36:49.029 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.150+0000 7f5ab8848640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab41074a0 0x7f5ab41078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:49.030 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.150+0000 7f5ab8848640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4107e70 con 0x7f5ab41074a0 2024-09-10T08:36:49.030 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.150+0000 7f5ab259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab41074a0 0x7f5ab41078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:49.030 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.150+0000 7f5ab259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab41074a0 0x7f5ab41078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36420/0 (socket says 172.21.15.172:36420) 2024-09-10T08:36:49.030 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.150+0000 7f5ab259c640 1 -- 172.21.15.172:0/2269343705 learned_addr learned my addr 172.21.15.172:0/2269343705 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:49.030 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.150+0000 7f5ab259c640 1 -- 172.21.15.172:0/2269343705 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab41086a0 con 0x7f5ab41074a0 2024-09-10T08:36:49.030 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.151+0000 7f5ab259c640 1 --2- 172.21.15.172:0/2269343705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab41074a0 0x7f5ab41078a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009820 tx=0x7f5aa402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1335e632226075cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:49.030 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.151+0000 7f5ab1d9b640 1 -- 172.21.15.172:0/2269343705 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aa4037070 con 0x7f5ab41074a0 2024-09-10T08:36:49.030 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.151+0000 7f5ab1d9b640 1 -- 172.21.15.172:0/2269343705 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5aa40328d0 con 0x7f5ab41074a0 2024-09-10T08:36:49.030 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.151+0000 7f5ab1d9b640 1 -- 172.21.15.172:0/2269343705 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aa40307d0 con 0x7f5ab41074a0 2024-09-10T08:36:49.030 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.152+0000 7f5ab8848640 1 -- 172.21.15.172:0/2269343705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab41074a0 msgr2=0x7f5ab41078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:49.030 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.152+0000 7f5ab8848640 1 --2- 172.21.15.172:0/2269343705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab41074a0 0x7f5ab41078a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009820 tx=0x7f5aa402a270 comp rx=0 tx=0).stop 2024-09-10T08:36:49.030 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.152+0000 7f5ab8848640 1 -- 172.21.15.172:0/2269343705 shutdown_connections 2024-09-10T08:36:49.031 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.152+0000 7f5ab8848640 1 --2- 172.21.15.172:0/2269343705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab41074a0 0x7f5ab41078a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:49.031 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.152+0000 7f5ab8848640 1 -- 172.21.15.172:0/2269343705 >> 172.21.15.172:0/2269343705 conn(0x7f5ab4102c50 msgr2=0x7f5ab4105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:49.031 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.152+0000 7f5ab8848640 1 -- 172.21.15.172:0/2269343705 shutdown_connections 2024-09-10T08:36:49.031 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.152+0000 7f5ab8848640 1 -- 172.21.15.172:0/2269343705 wait complete. 2024-09-10T08:36:49.031 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.153+0000 7f5ab8848640 1 Processor -- start 2024-09-10T08:36:49.031 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.153+0000 7f5ab8848640 1 -- start start 2024-09-10T08:36:49.031 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.153+0000 7f5ab8848640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab41074a0 0x7f5ab419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:49.031 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.153+0000 7f5ab8848640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab410ba50 con 0x7f5ab41074a0 2024-09-10T08:36:49.031 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.153+0000 7f5ab259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab41074a0 0x7f5ab419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:49.031 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.153+0000 7f5ab259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab41074a0 0x7f5ab419b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36436/0 (socket says 172.21.15.172:36436) 2024-09-10T08:36:49.031 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.153+0000 7f5ab259c640 1 -- 172.21.15.172:0/1882451508 learned_addr learned my addr 172.21.15.172:0/1882451508 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:49.031 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.153+0000 7f5ab259c640 1 -- 172.21.15.172:0/1882451508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab419b950 con 0x7f5ab41074a0 2024-09-10T08:36:49.032 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.153+0000 7f5ab259c640 1 --2- 172.21.15.172:0/1882451508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab41074a0 0x7f5ab419b410 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009940 tx=0x7f5aa4032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:49.032 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.154+0000 7f5aab7fe640 1 -- 172.21.15.172:0/1882451508 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aa4042070 con 0x7f5ab41074a0 2024-09-10T08:36:49.032 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.154+0000 7f5aab7fe640 1 -- 172.21.15.172:0/1882451508 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5aa4030c80 con 0x7f5ab41074a0 2024-09-10T08:36:49.032 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.154+0000 7f5aab7fe640 1 -- 172.21.15.172:0/1882451508 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aa4037040 con 0x7f5ab41074a0 2024-09-10T08:36:49.032 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.154+0000 7f5ab8848640 1 -- 172.21.15.172:0/1882451508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab419bbe0 con 0x7f5ab41074a0 2024-09-10T08:36:49.032 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.154+0000 7f5ab8848640 1 -- 172.21.15.172:0/1882451508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab419e8d0 con 0x7f5ab41074a0 2024-09-10T08:36:49.032 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.154+0000 7f5aab7fe640 1 -- 172.21.15.172:0/1882451508 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f5aa4045070 con 0x7f5ab41074a0 2024-09-10T08:36:49.032 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.155+0000 7f5ab8848640 1 -- 172.21.15.172:0/1882451508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab410bbd0 con 0x7f5ab41074a0 2024-09-10T08:36:49.032 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.155+0000 7f5aab7fe640 1 --2- 172.21.15.172:0/1882451508 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f5aa0038f20 0x7f5aa003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:49.032 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.155+0000 7f5aab7fe640 1 -- 172.21.15.172:0/1882451508 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5aa406bfe0 con 0x7f5ab41074a0 2024-09-10T08:36:49.032 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.155+0000 7f5aabfff640 1 --2- 172.21.15.172:0/1882451508 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f5aa0038f20 0x7f5aa003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:49.032 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.155+0000 7f5aabfff640 1 --2- 172.21.15.172:0/1882451508 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f5aa0038f20 0x7f5aa003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0098c0 tx=0x7f5a9c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:49.033 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.159+0000 7f5aab7fe640 1 -- 172.21.15.172:0/1882451508 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5aa4045360 con 0x7f5ab41074a0 2024-09-10T08:36:49.033 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.325+0000 7f5ab8848640 1 -- 172.21.15.172:0/1882451508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f5ab4107920 con 0x7f5ab41074a0 2024-09-10T08:36:49.033 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.326+0000 7f5aab7fe640 1 -- 172.21.15.172:0/1882451508 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+456 (secure 0 0 0) 0x7f5aa4050e00 con 0x7f5ab41074a0 2024-09-10T08:36:49.033 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.328+0000 7f5ab8848640 1 -- 172.21.15.172:0/1882451508 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f5aa0038f20 msgr2=0x7f5aa003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:49.033 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.328+0000 7f5ab8848640 1 --2- 172.21.15.172:0/1882451508 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f5aa0038f20 0x7f5aa003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0098c0 tx=0x7f5a9c006db0 comp rx=0 tx=0).stop 2024-09-10T08:36:49.033 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.328+0000 7f5ab8848640 1 -- 172.21.15.172:0/1882451508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab41074a0 msgr2=0x7f5ab419b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:49.033 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.328+0000 7f5ab8848640 1 --2- 172.21.15.172:0/1882451508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab41074a0 0x7f5ab419b410 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009940 tx=0x7f5aa4032e10 comp rx=0 tx=0).stop 2024-09-10T08:36:49.033 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.328+0000 7f5ab8848640 1 -- 172.21.15.172:0/1882451508 shutdown_connections 2024-09-10T08:36:49.033 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.328+0000 7f5ab8848640 1 --2- 172.21.15.172:0/1882451508 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f5aa0038f20 0x7f5aa003b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:49.033 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.328+0000 7f5ab8848640 1 --2- 172.21.15.172:0/1882451508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab41074a0 0x7f5ab419b410 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:49.033 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.328+0000 7f5ab8848640 1 -- 172.21.15.172:0/1882451508 >> 172.21.15.172:0/1882451508 conn(0x7f5ab4102c50 msgr2=0x7f5ab4103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:49.033 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.328+0000 7f5ab8848640 1 -- 172.21.15.172:0/1882451508 shutdown_connections 2024-09-10T08:36:49.033 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:48.328+0000 7f5ab8848640 1 -- 172.21.15.172:0/1882451508 wait complete. 2024-09-10T08:36:49.034 INFO:teuthology.orchestra.run.smithi172.stdout:Enabling cephadm module... 2024-09-10T08:36:49.320 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:49 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1882451508' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T08:36:50.331 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:50 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2548445726' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T08:36:50.332 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:50 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: ignoring --setuser ceph since I am not root 2024-09-10T08:36:50.332 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:50 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: ignoring --setgroup ceph since I am not root 2024-09-10T08:36:50.332 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:50 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:50.195+0000 7f7148f8d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T08:36:50.332 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:50 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:50.251+0000 7f7148f8d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T08:36:50.691 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:50 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:50.401+0000 7f7148f8d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T08:36:50.691 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:50 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:50.469+0000 7f7148f8d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T08:36:50.691 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:50 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:50.616+0000 7f7148f8d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:36:50.700 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.402+0000 7fb76f701640 1 Processor -- start 2024-09-10T08:36:50.700 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.403+0000 7fb76f701640 1 -- start start 2024-09-10T08:36:50.700 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.403+0000 7fb76f701640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb768105760 0x7fb768105b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:50.700 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.403+0000 7fb76f701640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb768106130 con 0x7fb768105760 2024-09-10T08:36:50.700 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.403+0000 7fb76d49d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb768105760 0x7fb768105b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:50.700 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.403+0000 7fb76d49d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb768105760 0x7fb768105b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36452/0 (socket says 172.21.15.172:36452) 2024-09-10T08:36:50.700 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.403+0000 7fb76d49d640 1 -- 172.21.15.172:0/3565935599 learned_addr learned my addr 172.21.15.172:0/3565935599 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:50.700 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.403+0000 7fb76d49d640 1 -- 172.21.15.172:0/3565935599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb768108740 con 0x7fb768105760 2024-09-10T08:36:50.701 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.404+0000 7fb76d49d640 1 --2- 172.21.15.172:0/3565935599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb768105760 0x7fb768105b60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb764009820 tx=0x7fb76402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce124a30ed7b4d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:50.701 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.404+0000 7fb75bfff640 1 -- 172.21.15.172:0/3565935599 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb764037070 con 0x7fb768105760 2024-09-10T08:36:50.701 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.404+0000 7fb75bfff640 1 -- 172.21.15.172:0/3565935599 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb7640328d0 con 0x7fb768105760 2024-09-10T08:36:50.701 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.404+0000 7fb75bfff640 1 -- 172.21.15.172:0/3565935599 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7640307d0 con 0x7fb768105760 2024-09-10T08:36:50.701 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.405+0000 7fb76f701640 1 -- 172.21.15.172:0/3565935599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb768105760 msgr2=0x7fb768105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:50.701 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.405+0000 7fb76f701640 1 --2- 172.21.15.172:0/3565935599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb768105760 0x7fb768105b60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb764009820 tx=0x7fb76402a270 comp rx=0 tx=0).stop 2024-09-10T08:36:50.701 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.405+0000 7fb76f701640 1 -- 172.21.15.172:0/3565935599 shutdown_connections 2024-09-10T08:36:50.701 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.405+0000 7fb76f701640 1 --2- 172.21.15.172:0/3565935599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb768105760 0x7fb768105b60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:50.701 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.405+0000 7fb76f701640 1 -- 172.21.15.172:0/3565935599 >> 172.21.15.172:0/3565935599 conn(0x7fb768101270 msgr2=0x7fb7681036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:50.701 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.405+0000 7fb76f701640 1 -- 172.21.15.172:0/3565935599 shutdown_connections 2024-09-10T08:36:50.701 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.405+0000 7fb76f701640 1 -- 172.21.15.172:0/3565935599 wait complete. 2024-09-10T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.406+0000 7fb76f701640 1 Processor -- start 2024-09-10T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.406+0000 7fb76f701640 1 -- start start 2024-09-10T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.406+0000 7fb76f701640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb768105760 0x7fb76819b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.406+0000 7fb76f701640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb76810b970 con 0x7fb768105760 2024-09-10T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.406+0000 7fb76d49d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb768105760 0x7fb76819b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.406+0000 7fb76d49d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb768105760 0x7fb76819b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36460/0 (socket says 172.21.15.172:36460) 2024-09-10T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.406+0000 7fb76d49d640 1 -- 172.21.15.172:0/2548445726 learned_addr learned my addr 172.21.15.172:0/2548445726 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.406+0000 7fb76d49d640 1 -- 172.21.15.172:0/2548445726 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76819ba00 con 0x7fb768105760 2024-09-10T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.406+0000 7fb76d49d640 1 --2- 172.21.15.172:0/2548445726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb768105760 0x7fb76819b4c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb764030520 tx=0x7fb764032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.406+0000 7fb75a7fc640 1 -- 172.21.15.172:0/2548445726 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb764042070 con 0x7fb768105760 2024-09-10T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.406+0000 7fb75a7fc640 1 -- 172.21.15.172:0/2548445726 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb764030cc0 con 0x7fb768105760 2024-09-10T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.406+0000 7fb76f701640 1 -- 172.21.15.172:0/2548445726 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76819bc90 con 0x7fb768105760 2024-09-10T08:36:50.702 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.406+0000 7fb75a7fc640 1 -- 172.21.15.172:0/2548445726 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb764037040 con 0x7fb768105760 2024-09-10T08:36:50.703 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.407+0000 7fb76f701640 1 -- 172.21.15.172:0/2548445726 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76819e980 con 0x7fb768105760 2024-09-10T08:36:50.703 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.407+0000 7fb75a7fc640 1 -- 172.21.15.172:0/2548445726 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fb764045070 con 0x7fb768105760 2024-09-10T08:36:50.703 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.407+0000 7fb76f701640 1 -- 172.21.15.172:0/2548445726 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb768105be0 con 0x7fb768105760 2024-09-10T08:36:50.703 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.408+0000 7fb75a7fc640 1 --2- 172.21.15.172:0/2548445726 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7fb754038f70 0x7fb75403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:50.703 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.408+0000 7fb75a7fc640 1 -- 172.21.15.172:0/2548445726 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb76406bfa0 con 0x7fb768105760 2024-09-10T08:36:50.703 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.408+0000 7fb76cc9c640 1 --2- 172.21.15.172:0/2548445726 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7fb754038f70 0x7fb75403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:50.703 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.408+0000 7fb76cc9c640 1 --2- 172.21.15.172:0/2548445726 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7fb754038f70 0x7fb75403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0098c0 tx=0x7fb75c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:50.703 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.413+0000 7fb75a7fc640 1 -- 172.21.15.172:0/2548445726 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb764046dd0 con 0x7fb768105760 2024-09-10T08:36:50.703 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:49.624+0000 7fb76f701640 1 -- 172.21.15.172:0/2548445726 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fb76819f000 con 0x7fb768105760 2024-09-10T08:36:50.703 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:50.039+0000 7fb75a7fc640 1 -- 172.21.15.172:0/2548445726 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fb76403cac0 con 0x7fb768105760 2024-09-10T08:36:50.703 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:50.041+0000 7fb75a7fc640 1 -- 172.21.15.172:0/2548445726 <== mon.0 v2:172.21.15.172:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fb76403c580 con 0x7fb768105760 2024-09-10T08:36:50.703 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:50.043+0000 7fb76f701640 1 -- 172.21.15.172:0/2548445726 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7fb754038f70 msgr2=0x7fb75403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:50.703 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:50.043+0000 7fb76f701640 1 --2- 172.21.15.172:0/2548445726 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7fb754038f70 0x7fb75403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0098c0 tx=0x7fb75c006db0 comp rx=0 tx=0).stop 2024-09-10T08:36:50.704 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:50.043+0000 7fb76f701640 1 -- 172.21.15.172:0/2548445726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb768105760 msgr2=0x7fb76819b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:50.704 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:50.043+0000 7fb76f701640 1 --2- 172.21.15.172:0/2548445726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb768105760 0x7fb76819b4c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb764030520 tx=0x7fb764032dd0 comp rx=0 tx=0).stop 2024-09-10T08:36:50.704 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:50.043+0000 7fb76f701640 1 -- 172.21.15.172:0/2548445726 shutdown_connections 2024-09-10T08:36:50.704 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:50.043+0000 7fb76f701640 1 --2- 172.21.15.172:0/2548445726 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7fb754038f70 0x7fb75403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:50.704 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:50.043+0000 7fb76f701640 1 --2- 172.21.15.172:0/2548445726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb768105760 0x7fb76819b4c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:50.704 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:50.043+0000 7fb76f701640 1 -- 172.21.15.172:0/2548445726 >> 172.21.15.172:0/2548445726 conn(0x7fb768101270 msgr2=0x7fb768101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:50.704 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:50.043+0000 7fb76f701640 1 -- 172.21.15.172:0/2548445726 shutdown_connections 2024-09-10T08:36:50.704 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:50.043+0000 7fb76f701640 1 -- 172.21.15.172:0/2548445726 wait complete. 2024-09-10T08:36:50.952 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:50 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:50.765+0000 7f7148f8d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T08:36:51.235 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:51 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2548445726' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T08:36:51.235 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:51 smithi172 ceph-mon[30854]: mgrmap e5: x(active, since 5s) 2024-09-10T08:36:51.235 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:51 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:51.091+0000 7f7148f8d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T08:36:51.235 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:51 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:51.151+0000 7f7148f8d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T08:36:51.581 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:51 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:51.342+0000 7f7148f8d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T08:36:51.816 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout { 2024-09-10T08:36:51.816 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T08:36:51.817 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T08:36:51.817 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T08:36:51.817 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T08:36:51.817 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout } 2024-09-10T08:36:51.817 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.028+0000 7f648ad20640 1 Processor -- start 2024-09-10T08:36:51.817 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.029+0000 7f648ad20640 1 -- start start 2024-09-10T08:36:51.817 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.029+0000 7f648ad20640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6484107ff0 0x7f648410a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:51.817 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.029+0000 7f648ad20640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6484075820 con 0x7f6484107ff0 2024-09-10T08:36:51.817 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.029+0000 7f6488abc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6484107ff0 0x7f648410a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:51.817 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.029+0000 7f6488abc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6484107ff0 0x7f648410a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36494/0 (socket says 172.21.15.172:36494) 2024-09-10T08:36:51.817 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.029+0000 7f6488abc640 1 -- 172.21.15.172:0/1268185333 learned_addr learned my addr 172.21.15.172:0/1268185333 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:51.818 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.029+0000 7f6488abc640 1 -- 172.21.15.172:0/1268185333 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648410a920 con 0x7f6484107ff0 2024-09-10T08:36:51.818 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.030+0000 7f6488abc640 1 --2- 172.21.15.172:0/1268185333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6484107ff0 0x7f648410a3e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6478009820 tx=0x7f647802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a4e7960e2f6b7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:51.818 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.030+0000 7f64837fe640 1 -- 172.21.15.172:0/1268185333 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6478037070 con 0x7f6484107ff0 2024-09-10T08:36:51.818 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.030+0000 7f64837fe640 1 -- 172.21.15.172:0/1268185333 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f64780328d0 con 0x7f6484107ff0 2024-09-10T08:36:51.818 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.030+0000 7f64837fe640 1 -- 172.21.15.172:0/1268185333 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64780307d0 con 0x7f6484107ff0 2024-09-10T08:36:51.818 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.031+0000 7f648ad20640 1 -- 172.21.15.172:0/1268185333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6484107ff0 msgr2=0x7f648410a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:51.818 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.031+0000 7f648ad20640 1 --2- 172.21.15.172:0/1268185333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6484107ff0 0x7f648410a3e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6478009820 tx=0x7f647802a270 comp rx=0 tx=0).stop 2024-09-10T08:36:51.818 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.031+0000 7f648ad20640 1 -- 172.21.15.172:0/1268185333 shutdown_connections 2024-09-10T08:36:51.818 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.031+0000 7f648ad20640 1 --2- 172.21.15.172:0/1268185333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6484107ff0 0x7f648410a3e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:51.818 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.031+0000 7f648ad20640 1 -- 172.21.15.172:0/1268185333 >> 172.21.15.172:0/1268185333 conn(0x7f6484101b10 msgr2=0x7f6484103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:51.818 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.031+0000 7f648ad20640 1 -- 172.21.15.172:0/1268185333 shutdown_connections 2024-09-10T08:36:51.818 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.031+0000 7f648ad20640 1 -- 172.21.15.172:0/1268185333 wait complete. 2024-09-10T08:36:51.818 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.032+0000 7f648ad20640 1 Processor -- start 2024-09-10T08:36:51.819 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.032+0000 7f648ad20640 1 -- start start 2024-09-10T08:36:51.819 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.032+0000 7f648ad20640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6484107ff0 0x7f6484107870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:51.819 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.032+0000 7f648ad20640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f648410b840 con 0x7f6484107ff0 2024-09-10T08:36:51.819 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.032+0000 7f6488abc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6484107ff0 0x7f6484107870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:51.819 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.032+0000 7f6488abc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6484107ff0 0x7f6484107870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36500/0 (socket says 172.21.15.172:36500) 2024-09-10T08:36:51.819 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.032+0000 7f6488abc640 1 -- 172.21.15.172:0/3984104545 learned_addr learned my addr 172.21.15.172:0/3984104545 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:51.819 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.032+0000 7f6488abc640 1 -- 172.21.15.172:0/3984104545 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6484107db0 con 0x7f6484107ff0 2024-09-10T08:36:51.819 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.032+0000 7f6488abc640 1 --2- 172.21.15.172:0/3984104545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6484107ff0 0x7f6484107870 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f64780097f0 tx=0x7f6478030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:51.819 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.032+0000 7f6481ffb640 1 -- 172.21.15.172:0/3984104545 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6478037040 con 0x7f6484107ff0 2024-09-10T08:36:51.819 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.032+0000 7f6481ffb640 1 -- 172.21.15.172:0/3984104545 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f647803b3f0 con 0x7f6484107ff0 2024-09-10T08:36:51.819 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.032+0000 7f6481ffb640 1 -- 172.21.15.172:0/3984104545 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f647803a520 con 0x7f6484107ff0 2024-09-10T08:36:51.819 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.032+0000 7f648ad20640 1 -- 172.21.15.172:0/3984104545 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6484105fa0 con 0x7f6484107ff0 2024-09-10T08:36:51.819 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.033+0000 7f648ad20640 1 -- 172.21.15.172:0/3984104545 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64841064e0 con 0x7f6484107ff0 2024-09-10T08:36:51.820 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.034+0000 7f6481ffb640 1 -- 172.21.15.172:0/3984104545 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f647803b5a0 con 0x7f6484107ff0 2024-09-10T08:36:51.820 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.034+0000 7f648ad20640 1 -- 172.21.15.172:0/3984104545 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6484108450 con 0x7f6484107ff0 2024-09-10T08:36:51.820 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.034+0000 7f6481ffb640 1 --2- 172.21.15.172:0/3984104545 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f6474038f70 0x7f647403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:51.820 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.034+0000 7f6481ffb640 1 -- 172.21.15.172:0/3984104545 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f647806c0b0 con 0x7f6484107ff0 2024-09-10T08:36:51.820 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.034+0000 7f6483fff640 1 -- 172.21.15.172:0/3984104545 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f6474038f70 msgr2=0x7f647403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.172:6800/2639090029 2024-09-10T08:36:51.820 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.034+0000 7f6483fff640 1 --2- 172.21.15.172:0/3984104545 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f6474038f70 0x7f647403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T08:36:51.820 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.039+0000 7f6481ffb640 1 -- 172.21.15.172:0/3984104545 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f64780328d0 con 0x7f6484107ff0 2024-09-10T08:36:51.820 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.235+0000 7f6483fff640 1 -- 172.21.15.172:0/3984104545 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f6474038f70 msgr2=0x7f647403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.172:6800/2639090029 2024-09-10T08:36:51.820 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.235+0000 7f6483fff640 1 --2- 172.21.15.172:0/3984104545 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f6474038f70 0x7f647403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T08:36:51.820 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.255+0000 7f648ad20640 1 -- 172.21.15.172:0/3984104545 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f6484106760 con 0x7f6484107ff0 2024-09-10T08:36:51.821 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.255+0000 7f6481ffb640 1 -- 172.21.15.172:0/3984104545 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f647803a7b0 con 0x7f6484107ff0 2024-09-10T08:36:51.821 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.257+0000 7f648ad20640 1 -- 172.21.15.172:0/3984104545 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f6474038f70 msgr2=0x7f647403b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:36:51.821 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.257+0000 7f648ad20640 1 --2- 172.21.15.172:0/3984104545 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f6474038f70 0x7f647403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:51.821 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.257+0000 7f648ad20640 1 -- 172.21.15.172:0/3984104545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6484107ff0 msgr2=0x7f6484107870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:51.821 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.257+0000 7f648ad20640 1 --2- 172.21.15.172:0/3984104545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6484107ff0 0x7f6484107870 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f64780097f0 tx=0x7f6478030d00 comp rx=0 tx=0).stop 2024-09-10T08:36:51.821 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.257+0000 7f648ad20640 1 -- 172.21.15.172:0/3984104545 shutdown_connections 2024-09-10T08:36:51.821 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.257+0000 7f648ad20640 1 --2- 172.21.15.172:0/3984104545 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f6474038f70 0x7f647403b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:51.821 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.257+0000 7f648ad20640 1 --2- 172.21.15.172:0/3984104545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6484107ff0 0x7f6484107870 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:51.821 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.257+0000 7f648ad20640 1 -- 172.21.15.172:0/3984104545 >> 172.21.15.172:0/3984104545 conn(0x7f6484101b10 msgr2=0x7f64841026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:51.821 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.257+0000 7f648ad20640 1 -- 172.21.15.172:0/3984104545 shutdown_connections 2024-09-10T08:36:51.821 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:51.257+0000 7f648ad20640 1 -- 172.21.15.172:0/3984104545 wait complete. 2024-09-10T08:36:51.821 INFO:teuthology.orchestra.run.smithi172.stdout:Waiting for the mgr to restart... 2024-09-10T08:36:51.821 INFO:teuthology.orchestra.run.smithi172.stdout:Waiting for mgr epoch 5... 2024-09-10T08:36:52.322 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:52 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3984104545' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T08:36:52.323 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:52 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:52.255+0000 7f7148f8d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T08:36:52.581 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:52 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:52.322+0000 7f7148f8d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T08:36:53.145 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:53 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:53.030+0000 7f7148f8d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T08:36:53.146 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:53 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:53.145+0000 7f7148f8d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T08:36:53.494 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:53 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:53.287+0000 7f7148f8d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:36:53.495 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:53 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:53.347+0000 7f7148f8d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T08:36:53.495 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:53 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:53.494+0000 7f7148f8d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T08:36:53.776 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:53 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:53.561+0000 7f7148f8d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T08:36:53.776 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:53 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:53.619+0000 7f7148f8d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T08:36:53.776 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:53 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:53.775+0000 7f7148f8d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T08:36:54.046 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:53 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:53.840+0000 7f7148f8d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T08:36:54.046 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:53 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:53.906+0000 7f7148f8d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T08:36:54.046 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:53 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:53.987+0000 7f7148f8d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T08:36:54.046 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:54.045+0000 7f7148f8d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T08:36:54.332 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:54.104+0000 7f7148f8d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T08:36:55.331 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:54.885+0000 7f7148f8d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T08:36:55.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-mon[30854]: Active manager daemon x restarted 2024-09-10T08:36:55.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-mon[30854]: Activating manager daemon x 2024-09-10T08:36:55.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-mon[30854]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T08:36:55.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-mon[30854]: mgrmap e6: x(active, starting, since 0.00337977s) 2024-09-10T08:36:55.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:36:55.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T08:36:55.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T08:36:55.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T08:36:55.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T08:36:55.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-mon[30854]: Manager daemon x is now available 2024-09-10T08:36:55.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' 2024-09-10T08:36:55.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' 2024-09-10T08:36:55.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:36:55.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:36:55.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T08:36:55.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:54 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T08:36:56.194 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:55 smithi172 ceph-mon[30854]: Found migration_current of "None". Setting to last migration. 2024-09-10T08:36:56.195 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:55 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' 2024-09-10T08:36:56.195 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:55 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' 2024-09-10T08:36:56.195 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:55 smithi172 ceph-mon[30854]: mgrmap e7: x(active, since 1.00584s) 2024-09-10T08:36:56.497 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout { 2024-09-10T08:36:56.498 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T08:36:56.498 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T08:36:56.498 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout } 2024-09-10T08:36:56.498 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.225+0000 7f2932c97640 1 Processor -- start 2024-09-10T08:36:56.498 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.226+0000 7f2932c97640 1 -- start start 2024-09-10T08:36:56.498 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.226+0000 7f2932c97640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c107f50 0x7f292c10a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:56.498 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.226+0000 7f2932c97640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f292c0758c0 con 0x7f292c107f50 2024-09-10T08:36:56.498 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.226+0000 7f2930a33640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c107f50 0x7f292c10a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:56.498 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.226+0000 7f2930a33640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c107f50 0x7f292c10a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36516/0 (socket says 172.21.15.172:36516) 2024-09-10T08:36:56.498 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.226+0000 7f2930a33640 1 -- 172.21.15.172:0/1645013944 learned_addr learned my addr 172.21.15.172:0/1645013944 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:56.498 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.226+0000 7f2930a33640 1 -- 172.21.15.172:0/1645013944 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292c10a880 con 0x7f292c107f50 2024-09-10T08:36:56.499 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.227+0000 7f2930a33640 1 --2- 172.21.15.172:0/1645013944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c107f50 0x7f292c10a340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2920009820 tx=0x7f292002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d1b7092d49dd19 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:56.499 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.227+0000 7f292b7fe640 1 -- 172.21.15.172:0/1645013944 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2920037070 con 0x7f292c107f50 2024-09-10T08:36:56.499 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.227+0000 7f292b7fe640 1 -- 172.21.15.172:0/1645013944 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f29200328d0 con 0x7f292c107f50 2024-09-10T08:36:56.499 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.227+0000 7f292b7fe640 1 -- 172.21.15.172:0/1645013944 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29200307d0 con 0x7f292c107f50 2024-09-10T08:36:56.499 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.228+0000 7f2932c97640 1 -- 172.21.15.172:0/1645013944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c107f50 msgr2=0x7f292c10a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:56.499 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.228+0000 7f2932c97640 1 --2- 172.21.15.172:0/1645013944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c107f50 0x7f292c10a340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2920009820 tx=0x7f292002a270 comp rx=0 tx=0).stop 2024-09-10T08:36:56.499 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.228+0000 7f2932c97640 1 -- 172.21.15.172:0/1645013944 shutdown_connections 2024-09-10T08:36:56.499 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.228+0000 7f2932c97640 1 --2- 172.21.15.172:0/1645013944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c107f50 0x7f292c10a340 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:56.500 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.228+0000 7f2932c97640 1 -- 172.21.15.172:0/1645013944 >> 172.21.15.172:0/1645013944 conn(0x7f292c101b10 msgr2=0x7f292c103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:56.500 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.228+0000 7f2932c97640 1 -- 172.21.15.172:0/1645013944 shutdown_connections 2024-09-10T08:36:56.500 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.228+0000 7f2932c97640 1 -- 172.21.15.172:0/1645013944 wait complete. 2024-09-10T08:36:56.500 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.229+0000 7f2932c97640 1 Processor -- start 2024-09-10T08:36:56.500 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.229+0000 7f2932c97640 1 -- start start 2024-09-10T08:36:56.500 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.229+0000 7f2932c97640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c107f50 0x7f292c19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:56.500 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.229+0000 7f2932c97640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f292c10b860 con 0x7f292c107f50 2024-09-10T08:36:56.500 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.229+0000 7f2930a33640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c107f50 0x7f292c19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:56.500 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.229+0000 7f2930a33640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c107f50 0x7f292c19b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36532/0 (socket says 172.21.15.172:36532) 2024-09-10T08:36:56.500 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.229+0000 7f2930a33640 1 -- 172.21.15.172:0/3420872379 learned_addr learned my addr 172.21.15.172:0/3420872379 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:56.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.229+0000 7f2930a33640 1 -- 172.21.15.172:0/3420872379 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292c19b980 con 0x7f292c107f50 2024-09-10T08:36:56.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.229+0000 7f2930a33640 1 --2- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c107f50 0x7f292c19b440 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29200097f0 tx=0x7f2920030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:56.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.230+0000 7f2929ffb640 1 -- 172.21.15.172:0/3420872379 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2920037040 con 0x7f292c107f50 2024-09-10T08:36:56.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.230+0000 7f2929ffb640 1 -- 172.21.15.172:0/3420872379 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f292003b3f0 con 0x7f292c107f50 2024-09-10T08:36:56.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.230+0000 7f2929ffb640 1 -- 172.21.15.172:0/3420872379 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f292003a520 con 0x7f292c107f50 2024-09-10T08:36:56.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.230+0000 7f2932c97640 1 -- 172.21.15.172:0/3420872379 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292c19bc10 con 0x7f292c107f50 2024-09-10T08:36:56.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.230+0000 7f2932c97640 1 -- 172.21.15.172:0/3420872379 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f292c19e900 con 0x7f292c107f50 2024-09-10T08:36:56.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.231+0000 7f2929ffb640 1 -- 172.21.15.172:0/3420872379 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f292003b5a0 con 0x7f292c107f50 2024-09-10T08:36:56.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.231+0000 7f2929ffb640 1 --2- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f291c038fc0 0x7f291c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:56.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.231+0000 7f2929ffb640 1 -- 172.21.15.172:0/3420872379 --> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f291c03bb90 con 0x7f291c038fc0 2024-09-10T08:36:56.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.231+0000 7f2929ffb640 1 -- 172.21.15.172:0/3420872379 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f292006c060 con 0x7f292c107f50 2024-09-10T08:36:56.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.231+0000 7f292bfff640 1 -- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f291c038fc0 msgr2=0x7f291c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.172:6800/2639090029 2024-09-10T08:36:56.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.231+0000 7f292bfff640 1 --2- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f291c038fc0 0x7f291c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T08:36:56.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.432+0000 7f292bfff640 1 -- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f291c038fc0 msgr2=0x7f291c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.172:6800/2639090029 2024-09-10T08:36:56.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.432+0000 7f292bfff640 1 --2- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f291c038fc0 0x7f291c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T08:36:56.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.833+0000 7f292bfff640 1 -- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f291c038fc0 msgr2=0x7f291c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.172:6800/2639090029 2024-09-10T08:36:56.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:52.833+0000 7f292bfff640 1 --2- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f291c038fc0 0x7f291c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T08:36:56.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:53.634+0000 7f292bfff640 1 -- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f291c038fc0 msgr2=0x7f291c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.172:6800/2639090029 2024-09-10T08:36:56.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:53.634+0000 7f292bfff640 1 --2- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f291c038fc0 0x7f291c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T08:36:56.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:54.888+0000 7f2929ffb640 1 -- 172.21.15.172:0/3420872379 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f292006abe0 con 0x7f292c107f50 2024-09-10T08:36:56.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:54.888+0000 7f2929ffb640 1 -- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f291c038fc0 msgr2=0x7f291c03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:36:56.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:54.888+0000 7f2929ffb640 1 --2- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f291c038fc0 0x7f291c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:56.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.890+0000 7f2929ffb640 1 -- 172.21.15.172:0/3420872379 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f2920032ba0 con 0x7f292c107f50 2024-09-10T08:36:56.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.890+0000 7f2929ffb640 1 --2- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f291c03c8f0 0x7f291c03ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:56.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.890+0000 7f2929ffb640 1 -- 172.21.15.172:0/3420872379 --> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f291c03bb90 con 0x7f291c03c8f0 2024-09-10T08:36:56.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.890+0000 7f292bfff640 1 --2- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f291c03c8f0 0x7f291c03ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:56.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.891+0000 7f292bfff640 1 --2- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f291c03c8f0 0x7f291c03ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2918003980 tx=0x7f29180092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:56.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.892+0000 7f2929ffb640 1 -- 172.21.15.172:0/3420872379 <== mgr.14118 v2:172.21.15.172:6800/2333617420 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f291c03bb90 con 0x7f291c03c8f0 2024-09-10T08:36:56.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.898+0000 7f2932c97640 1 -- 172.21.15.172:0/3420872379 --> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f292c1083b0 con 0x7f291c03c8f0 2024-09-10T08:36:56.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.899+0000 7f2929ffb640 1 -- 172.21.15.172:0/3420872379 <== mgr.14118 v2:172.21.15.172:6800/2333617420 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f292c1083b0 con 0x7f291c03c8f0 2024-09-10T08:36:56.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.899+0000 7f2932c97640 1 -- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f291c03c8f0 msgr2=0x7f291c03ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:56.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.899+0000 7f2932c97640 1 --2- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f291c03c8f0 0x7f291c03ece0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2918003980 tx=0x7f29180092b0 comp rx=0 tx=0).stop 2024-09-10T08:36:56.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.899+0000 7f2932c97640 1 -- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c107f50 msgr2=0x7f292c19b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:56.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.899+0000 7f2932c97640 1 --2- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c107f50 0x7f292c19b440 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29200097f0 tx=0x7f2920030d00 comp rx=0 tx=0).stop 2024-09-10T08:36:56.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.899+0000 7f2932c97640 1 -- 172.21.15.172:0/3420872379 shutdown_connections 2024-09-10T08:36:56.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.899+0000 7f2932c97640 1 --2- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f291c03c8f0 0x7f291c03ece0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:56.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.899+0000 7f2932c97640 1 --2- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:6800/2639090029,v1:172.21.15.172:6801/2639090029] conn(0x7f291c038fc0 0x7f291c03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:56.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.899+0000 7f2932c97640 1 --2- 172.21.15.172:0/3420872379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c107f50 0x7f292c19b440 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:56.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.899+0000 7f2932c97640 1 -- 172.21.15.172:0/3420872379 >> 172.21.15.172:0/3420872379 conn(0x7f292c101b10 msgr2=0x7f292c1026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:56.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.899+0000 7f2932c97640 1 -- 172.21.15.172:0/3420872379 shutdown_connections 2024-09-10T08:36:56.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:55.899+0000 7f2932c97640 1 -- 172.21.15.172:0/3420872379 wait complete. 2024-09-10T08:36:56.505 INFO:teuthology.orchestra.run.smithi172.stdout:mgr epoch 5 is available 2024-09-10T08:36:56.505 INFO:teuthology.orchestra.run.smithi172.stdout:Setting orchestrator backend to cephadm... 2024-09-10T08:36:57.081 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-mon[30854]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T08:36:57.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-mon[30854]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T08:36:57.082 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:56.897+0000 7f70b5af2640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:08:36:56] ENGINE Error in HTTPServer.serve 2024-09-10T08:36:57.082 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: Traceback (most recent call last): 2024-09-10T08:36:57.082 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T08:36:57.082 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: self._connections.run(self.expiration_interval) 2024-09-10T08:36:57.082 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T08:36:57.082 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: self._run(expiration_interval) 2024-09-10T08:36:57.083 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T08:36:57.083 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T08:36:57.083 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T08:36:57.083 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T08:36:57.083 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T08:36:57.083 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: s = self.context.wrap_socket( 2024-09-10T08:36:57.083 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T08:36:57.083 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: return self.sslsocket_class._create( 2024-09-10T08:36:57.083 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T08:36:57.083 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: self.do_handshake() 2024-09-10T08:36:57.083 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T08:36:57.083 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: self._sslobj.do_handshake() 2024-09-10T08:36:57.083 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T08:36:57.084 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:36:57.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.893+0000 7f83182c0640 1 Processor -- start 2024-09-10T08:36:57.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.893+0000 7f83182c0640 1 -- start start 2024-09-10T08:36:57.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.893+0000 7f83182c0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83101074a0 0x7f83101078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:57.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.893+0000 7f83182c0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8310107e70 con 0x7f83101074a0 2024-09-10T08:36:57.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.893+0000 7f831605c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83101074a0 0x7f83101078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:57.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.894+0000 7f831605c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83101074a0 0x7f83101078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50222/0 (socket says 172.21.15.172:50222) 2024-09-10T08:36:57.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.894+0000 7f831605c640 1 -- 172.21.15.172:0/241161512 learned_addr learned my addr 172.21.15.172:0/241161512 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:57.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.894+0000 7f831605c640 1 -- 172.21.15.172:0/241161512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83101086a0 con 0x7f83101074a0 2024-09-10T08:36:57.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.895+0000 7f831605c640 1 --2- 172.21.15.172:0/241161512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83101074a0 0x7f83101078a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8304009820 tx=0x7f830402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6444925c12b9d6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:57.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.895+0000 7f831505a640 1 -- 172.21.15.172:0/241161512 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8304037070 con 0x7f83101074a0 2024-09-10T08:36:57.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.895+0000 7f831505a640 1 -- 172.21.15.172:0/241161512 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f83040328d0 con 0x7f83101074a0 2024-09-10T08:36:57.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.895+0000 7f831505a640 1 -- 172.21.15.172:0/241161512 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83040307d0 con 0x7f83101074a0 2024-09-10T08:36:57.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.895+0000 7f83182c0640 1 -- 172.21.15.172:0/241161512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83101074a0 msgr2=0x7f83101078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:57.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.895+0000 7f83182c0640 1 --2- 172.21.15.172:0/241161512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83101074a0 0x7f83101078a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8304009820 tx=0x7f830402a270 comp rx=0 tx=0).stop 2024-09-10T08:36:57.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.896+0000 7f83182c0640 1 -- 172.21.15.172:0/241161512 shutdown_connections 2024-09-10T08:36:57.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.896+0000 7f83182c0640 1 --2- 172.21.15.172:0/241161512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83101074a0 0x7f83101078a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:57.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.896+0000 7f83182c0640 1 -- 172.21.15.172:0/241161512 >> 172.21.15.172:0/241161512 conn(0x7f8310102c50 msgr2=0x7f8310105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:57.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.896+0000 7f83182c0640 1 -- 172.21.15.172:0/241161512 shutdown_connections 2024-09-10T08:36:57.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.896+0000 7f83182c0640 1 -- 172.21.15.172:0/241161512 wait complete. 2024-09-10T08:36:57.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.896+0000 7f83182c0640 1 Processor -- start 2024-09-10T08:36:57.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.896+0000 7f83182c0640 1 -- start start 2024-09-10T08:36:57.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.896+0000 7f83182c0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83101074a0 0x7f831019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:57.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.896+0000 7f83182c0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f831010ba50 con 0x7f83101074a0 2024-09-10T08:36:57.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.896+0000 7f831605c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83101074a0 0x7f831019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:57.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.897+0000 7f831605c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83101074a0 0x7f831019b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50236/0 (socket says 172.21.15.172:50236) 2024-09-10T08:36:57.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.897+0000 7f831605c640 1 -- 172.21.15.172:0/3048885949 learned_addr learned my addr 172.21.15.172:0/3048885949 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:57.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.897+0000 7f831605c640 1 -- 172.21.15.172:0/3048885949 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831019b980 con 0x7f83101074a0 2024-09-10T08:36:57.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.897+0000 7f831605c640 1 --2- 172.21.15.172:0/3048885949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83101074a0 0x7f831019b440 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8304009940 tx=0x7f8304032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:57.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.897+0000 7f83037fe640 1 -- 172.21.15.172:0/3048885949 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8304042070 con 0x7f83101074a0 2024-09-10T08:36:57.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.897+0000 7f83037fe640 1 -- 172.21.15.172:0/3048885949 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8304030c80 con 0x7f83101074a0 2024-09-10T08:36:57.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.897+0000 7f83182c0640 1 -- 172.21.15.172:0/3048885949 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831019bc10 con 0x7f83101074a0 2024-09-10T08:36:57.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.897+0000 7f83037fe640 1 -- 172.21.15.172:0/3048885949 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8304037040 con 0x7f83101074a0 2024-09-10T08:36:57.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.897+0000 7f83182c0640 1 -- 172.21.15.172:0/3048885949 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831019e900 con 0x7f83101074a0 2024-09-10T08:36:57.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.898+0000 7f83037fe640 1 -- 172.21.15.172:0/3048885949 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f8304045070 con 0x7f83101074a0 2024-09-10T08:36:57.646 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.898+0000 7f83182c0640 1 -- 172.21.15.172:0/3048885949 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831010bbd0 con 0x7f83101074a0 2024-09-10T08:36:57.646 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.898+0000 7f83037fe640 1 --2- 172.21.15.172:0/3048885949 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f82fc038eb0 0x7f82fc03b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:57.646 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.898+0000 7f83037fe640 1 -- 172.21.15.172:0/3048885949 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f830406be60 con 0x7f83101074a0 2024-09-10T08:36:57.646 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.898+0000 7f831585b640 1 --2- 172.21.15.172:0/3048885949 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f82fc038eb0 0x7f82fc03b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:57.646 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.899+0000 7f831585b640 1 --2- 172.21.15.172:0/3048885949 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f82fc038eb0 0x7f82fc03b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f830c0098c0 tx=0x7f830c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:57.646 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.903+0000 7f83037fe640 1 -- 172.21.15.172:0/3048885949 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8304045360 con 0x7f83101074a0 2024-09-10T08:36:57.646 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:56.943+0000 7f83037fe640 1 -- 172.21.15.172:0/3048885949 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f830403cac0 con 0x7f83101074a0 2024-09-10T08:36:57.646 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:57.086+0000 7f83182c0640 1 -- 172.21.15.172:0/3048885949 --> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f8310103e60 con 0x7f82fc038eb0 2024-09-10T08:36:57.646 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:57.091+0000 7f83037fe640 1 -- 172.21.15.172:0/3048885949 <== mgr.14118 v2:172.21.15.172:6800/2333617420 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8310103e60 con 0x7f82fc038eb0 2024-09-10T08:36:57.646 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:57.093+0000 7f83182c0640 1 -- 172.21.15.172:0/3048885949 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f82fc038eb0 msgr2=0x7f82fc03b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:57.646 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:57.093+0000 7f83182c0640 1 --2- 172.21.15.172:0/3048885949 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f82fc038eb0 0x7f82fc03b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f830c0098c0 tx=0x7f830c006db0 comp rx=0 tx=0).stop 2024-09-10T08:36:57.646 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:57.093+0000 7f83182c0640 1 -- 172.21.15.172:0/3048885949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83101074a0 msgr2=0x7f831019b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:57.646 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:57.093+0000 7f83182c0640 1 --2- 172.21.15.172:0/3048885949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83101074a0 0x7f831019b440 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8304009940 tx=0x7f8304032e10 comp rx=0 tx=0).stop 2024-09-10T08:36:57.646 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:57.094+0000 7f83182c0640 1 -- 172.21.15.172:0/3048885949 shutdown_connections 2024-09-10T08:36:57.647 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:57.094+0000 7f83182c0640 1 --2- 172.21.15.172:0/3048885949 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f82fc038eb0 0x7f82fc03b370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:57.647 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:57.094+0000 7f83182c0640 1 --2- 172.21.15.172:0/3048885949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83101074a0 0x7f831019b440 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:57.647 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:57.094+0000 7f83182c0640 1 -- 172.21.15.172:0/3048885949 >> 172.21.15.172:0/3048885949 conn(0x7f8310102c50 msgr2=0x7f8310103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:57.647 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:57.094+0000 7f83182c0640 1 -- 172.21.15.172:0/3048885949 shutdown_connections 2024-09-10T08:36:57.647 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:57.094+0000 7f83182c0640 1 -- 172.21.15.172:0/3048885949 wait complete. 2024-09-10T08:36:58.219 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:57 smithi172 ceph-mon[30854]: [10/Sep/2024:08:36:56] ENGINE Bus STARTING 2024-09-10T08:36:58.220 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:57 smithi172 ceph-mon[30854]: [10/Sep/2024:08:36:56] ENGINE Serving on https://172.21.15.172:7150 2024-09-10T08:36:58.220 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:57 smithi172 ceph-mon[30854]: [10/Sep/2024:08:36:56] ENGINE Error in HTTPServer.serve 2024-09-10T08:36:58.220 INFO:journalctl@ceph.mon.a.smithi172.stdout: Traceback (most recent call last): 2024-09-10T08:36:58.220 INFO:journalctl@ceph.mon.a.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T08:36:58.220 INFO:journalctl@ceph.mon.a.smithi172.stdout: self._connections.run(self.expiration_interval) 2024-09-10T08:36:58.220 INFO:journalctl@ceph.mon.a.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T08:36:58.220 INFO:journalctl@ceph.mon.a.smithi172.stdout: self._run(expiration_interval) 2024-09-10T08:36:58.220 INFO:journalctl@ceph.mon.a.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T08:36:58.220 INFO:journalctl@ceph.mon.a.smithi172.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T08:36:58.220 INFO:journalctl@ceph.mon.a.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T08:36:58.220 INFO:journalctl@ceph.mon.a.smithi172.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T08:36:58.220 INFO:journalctl@ceph.mon.a.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T08:36:58.221 INFO:journalctl@ceph.mon.a.smithi172.stdout: s = self.context.wrap_socket( 2024-09-10T08:36:58.221 INFO:journalctl@ceph.mon.a.smithi172.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T08:36:58.221 INFO:journalctl@ceph.mon.a.smithi172.stdout: return self.sslsocket_class._create( 2024-09-10T08:36:58.221 INFO:journalctl@ceph.mon.a.smithi172.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T08:36:58.221 INFO:journalctl@ceph.mon.a.smithi172.stdout: self.do_handshake() 2024-09-10T08:36:58.221 INFO:journalctl@ceph.mon.a.smithi172.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T08:36:58.221 INFO:journalctl@ceph.mon.a.smithi172.stdout: self._sslobj.do_handshake() 2024-09-10T08:36:58.221 INFO:journalctl@ceph.mon.a.smithi172.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T08:36:58.221 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:57 smithi172 ceph-mon[30854]: mgrmap e8: x(active, since 2s) 2024-09-10T08:36:58.221 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:57 smithi172 ceph-mon[30854]: [10/Sep/2024:08:36:56] ENGINE Serving on http://172.21.15.172:8765 2024-09-10T08:36:58.221 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:57 smithi172 ceph-mon[30854]: [10/Sep/2024:08:36:56] ENGINE Bus STARTED 2024-09-10T08:36:58.221 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:57 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:36:58.221 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:57 smithi172 ceph-mon[30854]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:36:58.222 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:57 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' 2024-09-10T08:36:58.222 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:57 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:36:58.821 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T08:36:58.822 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.041+0000 7fe006972640 1 Processor -- start 2024-09-10T08:36:58.822 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.041+0000 7fe006972640 1 -- start start 2024-09-10T08:36:58.822 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.041+0000 7fe006972640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe000105760 0x7fe000105b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:58.822 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.041+0000 7fe006972640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe000106130 con 0x7fe000105760 2024-09-10T08:36:58.822 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.041+0000 7fdffffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe000105760 0x7fe000105b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:58.822 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.041+0000 7fdffffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe000105760 0x7fe000105b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50238/0 (socket says 172.21.15.172:50238) 2024-09-10T08:36:58.822 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.041+0000 7fdffffff640 1 -- 172.21.15.172:0/2225113131 learned_addr learned my addr 172.21.15.172:0/2225113131 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:58.822 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.042+0000 7fdffffff640 1 -- 172.21.15.172:0/2225113131 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe000108740 con 0x7fe000105760 2024-09-10T08:36:58.822 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.042+0000 7fdffffff640 1 --2- 172.21.15.172:0/2225113131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe000105760 0x7fe000105b60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009820 tx=0x7fdff002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575d43ca7aa7983f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:58.822 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.042+0000 7fdffeffd640 1 -- 172.21.15.172:0/2225113131 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdff0037070 con 0x7fe000105760 2024-09-10T08:36:58.822 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.043+0000 7fdffeffd640 1 -- 172.21.15.172:0/2225113131 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fdff00328d0 con 0x7fe000105760 2024-09-10T08:36:58.823 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.043+0000 7fdffeffd640 1 -- 172.21.15.172:0/2225113131 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdff00307d0 con 0x7fe000105760 2024-09-10T08:36:58.823 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.043+0000 7fe006972640 1 -- 172.21.15.172:0/2225113131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe000105760 msgr2=0x7fe000105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:58.823 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.043+0000 7fe006972640 1 --2- 172.21.15.172:0/2225113131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe000105760 0x7fe000105b60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009820 tx=0x7fdff002a270 comp rx=0 tx=0).stop 2024-09-10T08:36:58.823 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.043+0000 7fe006972640 1 -- 172.21.15.172:0/2225113131 shutdown_connections 2024-09-10T08:36:58.823 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.043+0000 7fe006972640 1 --2- 172.21.15.172:0/2225113131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe000105760 0x7fe000105b60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:58.823 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.043+0000 7fe006972640 1 -- 172.21.15.172:0/2225113131 >> 172.21.15.172:0/2225113131 conn(0x7fe000101270 msgr2=0x7fe0001036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:58.823 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.043+0000 7fe006972640 1 -- 172.21.15.172:0/2225113131 shutdown_connections 2024-09-10T08:36:58.823 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.043+0000 7fe006972640 1 -- 172.21.15.172:0/2225113131 wait complete. 2024-09-10T08:36:58.823 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.044+0000 7fe006972640 1 Processor -- start 2024-09-10T08:36:58.823 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.044+0000 7fe006972640 1 -- start start 2024-09-10T08:36:58.823 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.044+0000 7fe006972640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe000105760 0x7fe00019b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:58.823 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.044+0000 7fe006972640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe00010b970 con 0x7fe000105760 2024-09-10T08:36:58.824 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.044+0000 7fdffffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe000105760 0x7fe00019b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:58.824 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.044+0000 7fdffffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe000105760 0x7fe00019b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50252/0 (socket says 172.21.15.172:50252) 2024-09-10T08:36:58.824 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.044+0000 7fdffffff640 1 -- 172.21.15.172:0/2057279957 learned_addr learned my addr 172.21.15.172:0/2057279957 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:36:58.824 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.044+0000 7fdffffff640 1 -- 172.21.15.172:0/2057279957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00019ba10 con 0x7fe000105760 2024-09-10T08:36:58.824 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.044+0000 7fdffffff640 1 --2- 172.21.15.172:0/2057279957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe000105760 0x7fe00019b4d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdff00097f0 tx=0x7fdff0030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:58.824 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.045+0000 7fdffd7fa640 1 -- 172.21.15.172:0/2057279957 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdff0037040 con 0x7fe000105760 2024-09-10T08:36:58.824 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.045+0000 7fdffd7fa640 1 -- 172.21.15.172:0/2057279957 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fdff0032bb0 con 0x7fe000105760 2024-09-10T08:36:58.824 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.045+0000 7fe006972640 1 -- 172.21.15.172:0/2057279957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00019bca0 con 0x7fe000105760 2024-09-10T08:36:58.824 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.045+0000 7fdffd7fa640 1 -- 172.21.15.172:0/2057279957 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdff003a390 con 0x7fe000105760 2024-09-10T08:36:58.824 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.045+0000 7fe006972640 1 -- 172.21.15.172:0/2057279957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe00019e990 con 0x7fe000105760 2024-09-10T08:36:58.824 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.045+0000 7fdffd7fa640 1 -- 172.21.15.172:0/2057279957 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fdff003a580 con 0x7fe000105760 2024-09-10T08:36:58.824 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.046+0000 7fdffd7fa640 1 --2- 172.21.15.172:0/2057279957 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fdfe8038fc0 0x7fdfe803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:36:58.824 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.046+0000 7fdffd7fa640 1 -- 172.21.15.172:0/2057279957 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdff006c180 con 0x7fe000105760 2024-09-10T08:36:58.825 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.046+0000 7fdfff7fe640 1 --2- 172.21.15.172:0/2057279957 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fdfe8038fc0 0x7fdfe803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:36:58.825 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.046+0000 7fe006972640 1 -- 172.21.15.172:0/2057279957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe000105be0 con 0x7fe000105760 2024-09-10T08:36:58.825 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.046+0000 7fdfff7fe640 1 --2- 172.21.15.172:0/2057279957 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fdfe8038fc0 0x7fdfe803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdff40098c0 tx=0x7fdff4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:36:58.825 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.051+0000 7fdffd7fa640 1 -- 172.21.15.172:0/2057279957 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdff0040210 con 0x7fe000105760 2024-09-10T08:36:58.825 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.219+0000 7fe006972640 1 -- 172.21.15.172:0/2057279957 --> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fe0001025d0 con 0x7fdfe8038fc0 2024-09-10T08:36:58.825 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.220+0000 7fdffd7fa640 1 -- 172.21.15.172:0/2057279957 <== mgr.14118 v2:172.21.15.172:6800/2333617420 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fe0001025d0 con 0x7fdfe8038fc0 2024-09-10T08:36:58.825 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.222+0000 7fe006972640 1 -- 172.21.15.172:0/2057279957 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fdfe8038fc0 msgr2=0x7fdfe803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:58.825 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.222+0000 7fe006972640 1 --2- 172.21.15.172:0/2057279957 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fdfe8038fc0 0x7fdfe803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdff40098c0 tx=0x7fdff4006db0 comp rx=0 tx=0).stop 2024-09-10T08:36:58.825 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.222+0000 7fe006972640 1 -- 172.21.15.172:0/2057279957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe000105760 msgr2=0x7fe00019b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:36:58.825 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.222+0000 7fe006972640 1 --2- 172.21.15.172:0/2057279957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe000105760 0x7fe00019b4d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdff00097f0 tx=0x7fdff0030d00 comp rx=0 tx=0).stop 2024-09-10T08:36:58.825 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.222+0000 7fe006972640 1 -- 172.21.15.172:0/2057279957 shutdown_connections 2024-09-10T08:36:58.825 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.222+0000 7fe006972640 1 --2- 172.21.15.172:0/2057279957 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fdfe8038fc0 0x7fdfe803b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:58.826 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.222+0000 7fe006972640 1 --2- 172.21.15.172:0/2057279957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe000105760 0x7fe00019b4d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:36:58.826 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.222+0000 7fe006972640 1 -- 172.21.15.172:0/2057279957 >> 172.21.15.172:0/2057279957 conn(0x7fe000101270 msgr2=0x7fe000101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:36:58.826 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.222+0000 7fe006972640 1 -- 172.21.15.172:0/2057279957 shutdown_connections 2024-09-10T08:36:58.826 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:58.222+0000 7fe006972640 1 -- 172.21.15.172:0/2057279957 wait complete. 2024-09-10T08:36:58.826 INFO:teuthology.orchestra.run.smithi172.stdout:Generating ssh key... 2024-09-10T08:36:59.391 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-mon[30854]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:36:59.983 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: Generating public/private rsa key pair. 2024-09-10T08:36:59.984 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: Your identification has been saved in /tmp/tmpulddiaa4/key 2024-09-10T08:36:59.984 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: Your public key has been saved in /tmp/tmpulddiaa4/key.pub 2024-09-10T08:36:59.984 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: The key fingerprint is: 2024-09-10T08:36:59.984 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: SHA256:QlaxqGk93BRAYkmiNi5itn9L+NcvbQEfekSQlwnMC1Q ceph-a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:36:59.984 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: The key's randomart image is: 2024-09-10T08:36:59.984 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: +---[RSA 3072]----+ 2024-09-10T08:36:59.984 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: | ..+o+oBE+ o | 2024-09-10T08:36:59.984 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: | . o.. + * = | 2024-09-10T08:36:59.984 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: |.o + + + | 2024-09-10T08:36:59.984 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: |o . B o o o | 2024-09-10T08:36:59.984 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: |o+ + = S = . | 2024-09-10T08:36:59.985 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: |= ... o . + | 2024-09-10T08:36:59.985 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: | . . . . o . | 2024-09-10T08:36:59.985 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: | . o. . o o | 2024-09-10T08:36:59.985 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: | ..oo +. | 2024-09-10T08:36:59.985 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:36:59 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: +----[SHA256]-----+ 2024-09-10T08:37:00.241 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.213+0000 7f96ebe3b640 1 Processor -- start 2024-09-10T08:37:00.241 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.213+0000 7f96ebe3b640 1 -- start start 2024-09-10T08:37:00.242 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.213+0000 7f96ebe3b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e407de20 0x7f96e407c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:00.242 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.213+0000 7f96ebe3b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f96e407c850 con 0x7f96e407de20 2024-09-10T08:37:00.242 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.213+0000 7f96e9bd7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e407de20 0x7f96e407c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:00.242 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.213+0000 7f96e9bd7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e407de20 0x7f96e407c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50268/0 (socket says 172.21.15.172:50268) 2024-09-10T08:37:00.242 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.213+0000 7f96e9bd7640 1 -- 172.21.15.172:0/915964292 learned_addr learned my addr 172.21.15.172:0/915964292 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:00.242 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.214+0000 7f96e9bd7640 1 -- 172.21.15.172:0/915964292 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e407c9d0 con 0x7f96e407de20 2024-09-10T08:37:00.242 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.214+0000 7f96e9bd7640 1 --2- 172.21.15.172:0/915964292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e407de20 0x7f96e407c280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009820 tx=0x7f96e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2577631eec6d6a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:00.242 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.214+0000 7f96e8bd5640 1 -- 172.21.15.172:0/915964292 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96e0037070 con 0x7f96e407de20 2024-09-10T08:37:00.242 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.214+0000 7f96e8bd5640 1 -- 172.21.15.172:0/915964292 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f96e00328d0 con 0x7f96e407de20 2024-09-10T08:37:00.242 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.214+0000 7f96e8bd5640 1 -- 172.21.15.172:0/915964292 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96e00307d0 con 0x7f96e407de20 2024-09-10T08:37:00.242 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.215+0000 7f96ebe3b640 1 -- 172.21.15.172:0/915964292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e407de20 msgr2=0x7f96e407c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:00.242 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.215+0000 7f96ebe3b640 1 --2- 172.21.15.172:0/915964292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e407de20 0x7f96e407c280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009820 tx=0x7f96e002a270 comp rx=0 tx=0).stop 2024-09-10T08:37:00.242 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.215+0000 7f96ebe3b640 1 -- 172.21.15.172:0/915964292 shutdown_connections 2024-09-10T08:37:00.243 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.215+0000 7f96ebe3b640 1 --2- 172.21.15.172:0/915964292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e407de20 0x7f96e407c280 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:00.243 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.215+0000 7f96ebe3b640 1 -- 172.21.15.172:0/915964292 >> 172.21.15.172:0/915964292 conn(0x7f96e4102cb0 msgr2=0x7f96e41050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:00.243 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.215+0000 7f96ebe3b640 1 -- 172.21.15.172:0/915964292 shutdown_connections 2024-09-10T08:37:00.243 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.215+0000 7f96ebe3b640 1 -- 172.21.15.172:0/915964292 wait complete. 2024-09-10T08:37:00.243 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.216+0000 7f96ebe3b640 1 Processor -- start 2024-09-10T08:37:00.243 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.216+0000 7f96ebe3b640 1 -- start start 2024-09-10T08:37:00.243 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.216+0000 7f96ebe3b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e407de20 0x7f96e419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:00.243 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.216+0000 7f96ebe3b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f96e410dad0 con 0x7f96e407de20 2024-09-10T08:37:00.243 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.216+0000 7f96e9bd7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e407de20 0x7f96e419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:00.243 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.216+0000 7f96e9bd7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e407de20 0x7f96e419d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50272/0 (socket says 172.21.15.172:50272) 2024-09-10T08:37:00.243 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.216+0000 7f96e9bd7640 1 -- 172.21.15.172:0/901411147 learned_addr learned my addr 172.21.15.172:0/901411147 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:00.243 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.216+0000 7f96e9bd7640 1 -- 172.21.15.172:0/901411147 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e419db40 con 0x7f96e407de20 2024-09-10T08:37:00.243 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.217+0000 7f96e9bd7640 1 --2- 172.21.15.172:0/901411147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e407de20 0x7f96e419d600 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f96e0032e10 tx=0x7f96e0032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:00.244 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.217+0000 7f96daffd640 1 -- 172.21.15.172:0/901411147 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96e0042070 con 0x7f96e407de20 2024-09-10T08:37:00.244 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.217+0000 7f96ebe3b640 1 -- 172.21.15.172:0/901411147 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e419ddd0 con 0x7f96e407de20 2024-09-10T08:37:00.244 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.217+0000 7f96daffd640 1 -- 172.21.15.172:0/901411147 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f96e0030c80 con 0x7f96e407de20 2024-09-10T08:37:00.244 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.217+0000 7f96daffd640 1 -- 172.21.15.172:0/901411147 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96e0037040 con 0x7f96e407de20 2024-09-10T08:37:00.244 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.217+0000 7f96ebe3b640 1 -- 172.21.15.172:0/901411147 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e41a0ac0 con 0x7f96e407de20 2024-09-10T08:37:00.244 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.218+0000 7f96daffd640 1 -- 172.21.15.172:0/901411147 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f96e0045070 con 0x7f96e407de20 2024-09-10T08:37:00.244 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.218+0000 7f96daffd640 1 --2- 172.21.15.172:0/901411147 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f96d0038f70 0x7f96d003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:00.244 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.218+0000 7f96ebe3b640 1 -- 172.21.15.172:0/901411147 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e407c300 con 0x7f96e407de20 2024-09-10T08:37:00.244 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.218+0000 7f96daffd640 1 -- 172.21.15.172:0/901411147 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f96e006bf00 con 0x7f96e407de20 2024-09-10T08:37:00.244 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.218+0000 7f96e93d6640 1 --2- 172.21.15.172:0/901411147 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f96d0038f70 0x7f96d003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:00.244 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.218+0000 7f96e93d6640 1 --2- 172.21.15.172:0/901411147 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f96d0038f70 0x7f96d003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f96d40098c0 tx=0x7f96d4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:00.244 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.223+0000 7f96daffd640 1 -- 172.21.15.172:0/901411147 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f96e0045360 con 0x7f96e407de20 2024-09-10T08:37:00.245 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.390+0000 7f96ebe3b640 1 -- 172.21.15.172:0/901411147 --> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f96e4103ee0 con 0x7f96d0038f70 2024-09-10T08:37:00.245 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.664+0000 7f96daffd640 1 -- 172.21.15.172:0/901411147 <== mgr.14118 v2:172.21.15.172:6800/2333617420 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f96e4103ee0 con 0x7f96d0038f70 2024-09-10T08:37:00.245 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.666+0000 7f96ebe3b640 1 -- 172.21.15.172:0/901411147 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f96d0038f70 msgr2=0x7f96d003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:00.245 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.666+0000 7f96ebe3b640 1 --2- 172.21.15.172:0/901411147 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f96d0038f70 0x7f96d003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f96d40098c0 tx=0x7f96d4006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:00.245 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.666+0000 7f96ebe3b640 1 -- 172.21.15.172:0/901411147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e407de20 msgr2=0x7f96e419d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:00.245 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.666+0000 7f96ebe3b640 1 --2- 172.21.15.172:0/901411147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e407de20 0x7f96e419d600 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f96e0032e10 tx=0x7f96e0032e40 comp rx=0 tx=0).stop 2024-09-10T08:37:00.245 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.666+0000 7f96ebe3b640 1 -- 172.21.15.172:0/901411147 shutdown_connections 2024-09-10T08:37:00.245 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.666+0000 7f96ebe3b640 1 --2- 172.21.15.172:0/901411147 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f96d0038f70 0x7f96d003b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:00.245 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.666+0000 7f96ebe3b640 1 --2- 172.21.15.172:0/901411147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96e407de20 0x7f96e419d600 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:00.245 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.666+0000 7f96ebe3b640 1 -- 172.21.15.172:0/901411147 >> 172.21.15.172:0/901411147 conn(0x7f96e4102cb0 msgr2=0x7f96e41037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:00.245 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.666+0000 7f96ebe3b640 1 -- 172.21.15.172:0/901411147 shutdown_connections 2024-09-10T08:37:00.245 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:36:59.666+0000 7f96ebe3b640 1 -- 172.21.15.172:0/901411147 wait complete. 2024-09-10T08:37:00.831 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:00 smithi172 ceph-mon[30854]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:00.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:00 smithi172 ceph-mon[30854]: Generating ssh key... 2024-09-10T08:37:00.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:00 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' 2024-09-10T08:37:00.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:00 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' 2024-09-10T08:37:01.364 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDc6HsuEry07e68/ERoDsCX5EGlmKGRgv+iCYVILK8SfVLdG+HWaIyN5NyBWFHPG4KJdz/6ixrfbScmvXABtFwEkxjqQEhPXnkfHB3wDXEdEg22fEKRO/AutzLaDG0wty30ZQ6rabKhPnwa8Pl0QZ1uOkAyxbn2NVNgeyr+AuZJRm4sCe2i9naVVWcNUb94P8M9IhyxMhiyyVmc6Er+cEA6eM8IEx9I7ysndT8n7YjGusvG2CzxPCNTvzZkWdHiGyVifAu0Q6RiFlTV77zxAgxNNN+c+TqwL0jxgwpOsaPC/iRGJmYTM3I/6ejBR5q4sGpSjEhGsBmF+FlrRRTeGxX3SPb6xCB6kcKmk0dz/zLoq9vVY1I0R78P9XwFM4qd7eExgkcmoV5Cteg1mYgIfn8WHY4Gfk5TuNDXGSevCGzgMGTRcAit+iY87M3bkPeUhZ6FXvoZxcPYNHltg1mlVVqL+icW80C7NlgYDPYaBT8tSrW6eQGPrCMOQqBWMllICe0= ceph-a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:37:01.365 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.656+0000 7fe61055e640 1 Processor -- start 2024-09-10T08:37:01.365 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.656+0000 7fe61055e640 1 -- start start 2024-09-10T08:37:01.365 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.656+0000 7fe61055e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe60807de00 0x7fe60807c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:01.365 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.656+0000 7fe61055e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe60807c830 con 0x7fe60807de00 2024-09-10T08:37:01.365 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.657+0000 7fe60e2fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe60807de00 0x7fe60807c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:01.365 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.657+0000 7fe60e2fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe60807de00 0x7fe60807c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50276/0 (socket says 172.21.15.172:50276) 2024-09-10T08:37:01.365 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.657+0000 7fe60e2fa640 1 -- 172.21.15.172:0/3147811991 learned_addr learned my addr 172.21.15.172:0/3147811991 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:01.365 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.657+0000 7fe60e2fa640 1 -- 172.21.15.172:0/3147811991 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60807c9b0 con 0x7fe60807de00 2024-09-10T08:37:01.365 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.657+0000 7fe60e2fa640 1 --2- 172.21.15.172:0/3147811991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe60807de00 0x7fe60807c260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe604009820 tx=0x7fe60402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b42ac13ff5ccdd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:01.366 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.658+0000 7fe60d2f8640 1 -- 172.21.15.172:0/3147811991 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe604037070 con 0x7fe60807de00 2024-09-10T08:37:01.366 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.658+0000 7fe60d2f8640 1 -- 172.21.15.172:0/3147811991 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe6040328d0 con 0x7fe60807de00 2024-09-10T08:37:01.366 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.658+0000 7fe60d2f8640 1 -- 172.21.15.172:0/3147811991 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6040307d0 con 0x7fe60807de00 2024-09-10T08:37:01.366 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.658+0000 7fe61055e640 1 -- 172.21.15.172:0/3147811991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe60807de00 msgr2=0x7fe60807c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:01.366 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.658+0000 7fe61055e640 1 --2- 172.21.15.172:0/3147811991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe60807de00 0x7fe60807c260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe604009820 tx=0x7fe60402a270 comp rx=0 tx=0).stop 2024-09-10T08:37:01.366 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.658+0000 7fe61055e640 1 -- 172.21.15.172:0/3147811991 shutdown_connections 2024-09-10T08:37:01.366 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.658+0000 7fe61055e640 1 --2- 172.21.15.172:0/3147811991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe60807de00 0x7fe60807c260 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:01.366 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.658+0000 7fe61055e640 1 -- 172.21.15.172:0/3147811991 >> 172.21.15.172:0/3147811991 conn(0x7fe608102c50 msgr2=0x7fe608105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:01.366 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.658+0000 7fe61055e640 1 -- 172.21.15.172:0/3147811991 shutdown_connections 2024-09-10T08:37:01.366 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.658+0000 7fe61055e640 1 -- 172.21.15.172:0/3147811991 wait complete. 2024-09-10T08:37:01.366 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.659+0000 7fe61055e640 1 Processor -- start 2024-09-10T08:37:01.367 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.659+0000 7fe61055e640 1 -- start start 2024-09-10T08:37:01.367 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.659+0000 7fe61055e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe60807de00 0x7fe60819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:01.367 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.659+0000 7fe61055e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe60810da70 con 0x7fe60807de00 2024-09-10T08:37:01.367 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.659+0000 7fe60e2fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe60807de00 0x7fe60819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:01.367 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.659+0000 7fe60e2fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe60807de00 0x7fe60819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50280/0 (socket says 172.21.15.172:50280) 2024-09-10T08:37:01.367 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.659+0000 7fe60e2fa640 1 -- 172.21.15.172:0/719115395 learned_addr learned my addr 172.21.15.172:0/719115395 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:01.367 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.659+0000 7fe60e2fa640 1 -- 172.21.15.172:0/719115395 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60819db70 con 0x7fe60807de00 2024-09-10T08:37:01.367 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.660+0000 7fe60e2fa640 1 --2- 172.21.15.172:0/719115395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe60807de00 0x7fe60819d630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe604032e10 tx=0x7fe604032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:01.367 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.660+0000 7fe5ff7fe640 1 -- 172.21.15.172:0/719115395 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe604042070 con 0x7fe60807de00 2024-09-10T08:37:01.367 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.660+0000 7fe5ff7fe640 1 -- 172.21.15.172:0/719115395 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe604030c80 con 0x7fe60807de00 2024-09-10T08:37:01.367 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.660+0000 7fe5ff7fe640 1 -- 172.21.15.172:0/719115395 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe604037040 con 0x7fe60807de00 2024-09-10T08:37:01.368 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.660+0000 7fe61055e640 1 -- 172.21.15.172:0/719115395 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60819de00 con 0x7fe60807de00 2024-09-10T08:37:01.368 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.660+0000 7fe61055e640 1 -- 172.21.15.172:0/719115395 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6081a0af0 con 0x7fe60807de00 2024-09-10T08:37:01.368 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.661+0000 7fe61055e640 1 -- 172.21.15.172:0/719115395 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe60807c2e0 con 0x7fe60807de00 2024-09-10T08:37:01.368 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.663+0000 7fe5ff7fe640 1 -- 172.21.15.172:0/719115395 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe604045070 con 0x7fe60807de00 2024-09-10T08:37:01.368 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.663+0000 7fe5ff7fe640 1 --2- 172.21.15.172:0/719115395 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fe5f4038fc0 0x7fe5f403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:01.368 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.663+0000 7fe5ff7fe640 1 -- 172.21.15.172:0/719115395 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe60406c710 con 0x7fe60807de00 2024-09-10T08:37:01.368 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.663+0000 7fe60daf9640 1 --2- 172.21.15.172:0/719115395 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fe5f4038fc0 0x7fe5f403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:01.368 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.663+0000 7fe60daf9640 1 --2- 172.21.15.172:0/719115395 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fe5f4038fc0 0x7fe5f403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80098c0 tx=0x7fe5f8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:01.368 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.667+0000 7fe5ff7fe640 1 -- 172.21.15.172:0/719115395 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe60406b1c0 con 0x7fe60807de00 2024-09-10T08:37:01.368 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.832+0000 7fe61055e640 1 -- 172.21.15.172:0/719115395 --> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fe608103e90 con 0x7fe5f4038fc0 2024-09-10T08:37:01.369 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.833+0000 7fe5ff7fe640 1 -- 172.21.15.172:0/719115395 <== mgr.14118 v2:172.21.15.172:6800/2333617420 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fe608103e90 con 0x7fe5f4038fc0 2024-09-10T08:37:01.369 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.834+0000 7fe61055e640 1 -- 172.21.15.172:0/719115395 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fe5f4038fc0 msgr2=0x7fe5f403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:01.369 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.834+0000 7fe61055e640 1 --2- 172.21.15.172:0/719115395 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fe5f4038fc0 0x7fe5f403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80098c0 tx=0x7fe5f8006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:01.369 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.834+0000 7fe61055e640 1 -- 172.21.15.172:0/719115395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe60807de00 msgr2=0x7fe60819d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:01.369 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.834+0000 7fe61055e640 1 --2- 172.21.15.172:0/719115395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe60807de00 0x7fe60819d630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe604032e10 tx=0x7fe604032e40 comp rx=0 tx=0).stop 2024-09-10T08:37:01.369 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.835+0000 7fe61055e640 1 -- 172.21.15.172:0/719115395 shutdown_connections 2024-09-10T08:37:01.369 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.835+0000 7fe61055e640 1 --2- 172.21.15.172:0/719115395 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fe5f4038fc0 0x7fe5f403b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:01.369 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.835+0000 7fe61055e640 1 --2- 172.21.15.172:0/719115395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe60807de00 0x7fe60819d630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:01.369 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.835+0000 7fe61055e640 1 -- 172.21.15.172:0/719115395 >> 172.21.15.172:0/719115395 conn(0x7fe608102c50 msgr2=0x7fe608103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:01.369 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.835+0000 7fe61055e640 1 -- 172.21.15.172:0/719115395 shutdown_connections 2024-09-10T08:37:01.369 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:00.835+0000 7fe61055e640 1 -- 172.21.15.172:0/719115395 wait complete. 2024-09-10T08:37:01.370 INFO:teuthology.orchestra.run.smithi172.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T08:37:01.370 INFO:teuthology.orchestra.run.smithi172.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T08:37:01.370 INFO:teuthology.orchestra.run.smithi172.stdout:Adding host smithi172... 2024-09-10T08:37:01.928 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:01 smithi172 ceph-mon[30854]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:02.697 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:02 smithi172 ceph-mon[30854]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi172", "addr": "172.21.15.172", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:04.081 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:03 smithi172 ceph-mon[30854]: Deploying cephadm binary to smithi172 2024-09-10T08:37:05.624 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout Added host 'smithi172' with addr '172.21.15.172' 2024-09-10T08:37:05.624 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.747+0000 7fcecb940640 1 Processor -- start 2024-09-10T08:37:05.624 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.747+0000 7fcecb940640 1 -- start start 2024-09-10T08:37:05.625 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.747+0000 7fcecb940640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcec41074a0 0x7fcec41078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:05.625 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.747+0000 7fcecb940640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcec4107e70 con 0x7fcec41074a0 2024-09-10T08:37:05.625 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.748+0000 7fcec96dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcec41074a0 0x7fcec41078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:05.625 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.748+0000 7fcec96dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcec41074a0 0x7fcec41078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50288/0 (socket says 172.21.15.172:50288) 2024-09-10T08:37:05.625 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.748+0000 7fcec96dc640 1 -- 172.21.15.172:0/2226547662 learned_addr learned my addr 172.21.15.172:0/2226547662 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:05.625 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.748+0000 7fcec96dc640 1 -- 172.21.15.172:0/2226547662 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec41086a0 con 0x7fcec41074a0 2024-09-10T08:37:05.625 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.749+0000 7fcec96dc640 1 --2- 172.21.15.172:0/2226547662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcec41074a0 0x7fcec41078a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009820 tx=0x7fcec002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b549c29925fb45d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:05.625 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.749+0000 7fcebbfff640 1 -- 172.21.15.172:0/2226547662 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcec0037070 con 0x7fcec41074a0 2024-09-10T08:37:05.625 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.749+0000 7fcebbfff640 1 -- 172.21.15.172:0/2226547662 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fcec00328d0 con 0x7fcec41074a0 2024-09-10T08:37:05.626 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.749+0000 7fcebbfff640 1 -- 172.21.15.172:0/2226547662 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcec00307d0 con 0x7fcec41074a0 2024-09-10T08:37:05.626 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.749+0000 7fcecb940640 1 -- 172.21.15.172:0/2226547662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcec41074a0 msgr2=0x7fcec41078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:05.626 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.749+0000 7fcecb940640 1 --2- 172.21.15.172:0/2226547662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcec41074a0 0x7fcec41078a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009820 tx=0x7fcec002a270 comp rx=0 tx=0).stop 2024-09-10T08:37:05.626 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.750+0000 7fcecb940640 1 -- 172.21.15.172:0/2226547662 shutdown_connections 2024-09-10T08:37:05.626 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.750+0000 7fcecb940640 1 --2- 172.21.15.172:0/2226547662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcec41074a0 0x7fcec41078a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:05.626 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.750+0000 7fcecb940640 1 -- 172.21.15.172:0/2226547662 >> 172.21.15.172:0/2226547662 conn(0x7fcec4102c50 msgr2=0x7fcec4105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:05.626 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.750+0000 7fcecb940640 1 -- 172.21.15.172:0/2226547662 shutdown_connections 2024-09-10T08:37:05.626 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.750+0000 7fcecb940640 1 -- 172.21.15.172:0/2226547662 wait complete. 2024-09-10T08:37:05.626 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.750+0000 7fcecb940640 1 Processor -- start 2024-09-10T08:37:05.626 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.750+0000 7fcecb940640 1 -- start start 2024-09-10T08:37:05.626 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.750+0000 7fcecb940640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcec41074a0 0x7fcec419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:05.626 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.750+0000 7fcecb940640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcec410ba50 con 0x7fcec41074a0 2024-09-10T08:37:05.627 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.751+0000 7fcec96dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcec41074a0 0x7fcec419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:05.627 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.751+0000 7fcec96dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcec41074a0 0x7fcec419b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50298/0 (socket says 172.21.15.172:50298) 2024-09-10T08:37:05.627 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.751+0000 7fcec96dc640 1 -- 172.21.15.172:0/2087471621 learned_addr learned my addr 172.21.15.172:0/2087471621 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:05.627 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.751+0000 7fcec96dc640 1 -- 172.21.15.172:0/2087471621 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec419b950 con 0x7fcec41074a0 2024-09-10T08:37:05.627 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.751+0000 7fcec96dc640 1 --2- 172.21.15.172:0/2087471621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcec41074a0 0x7fcec419b410 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009940 tx=0x7fcec0032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:05.627 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.751+0000 7fceba7fc640 1 -- 172.21.15.172:0/2087471621 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcec0042070 con 0x7fcec41074a0 2024-09-10T08:37:05.627 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.751+0000 7fceba7fc640 1 -- 172.21.15.172:0/2087471621 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fcec0030c80 con 0x7fcec41074a0 2024-09-10T08:37:05.627 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.751+0000 7fceba7fc640 1 -- 172.21.15.172:0/2087471621 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcec0037040 con 0x7fcec41074a0 2024-09-10T08:37:05.627 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.751+0000 7fcecb940640 1 -- 172.21.15.172:0/2087471621 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec419bbe0 con 0x7fcec41074a0 2024-09-10T08:37:05.627 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.751+0000 7fcecb940640 1 -- 172.21.15.172:0/2087471621 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec419e8d0 con 0x7fcec41074a0 2024-09-10T08:37:05.627 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.752+0000 7fceba7fc640 1 -- 172.21.15.172:0/2087471621 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcec0045070 con 0x7fcec41074a0 2024-09-10T08:37:05.627 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.752+0000 7fceba7fc640 1 --2- 172.21.15.172:0/2087471621 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fceb0038f70 0x7fceb003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:05.628 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.753+0000 7fceba7fc640 1 -- 172.21.15.172:0/2087471621 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcec006bf40 con 0x7fcec41074a0 2024-09-10T08:37:05.628 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.753+0000 7fcecb940640 1 -- 172.21.15.172:0/2087471621 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcec410bbd0 con 0x7fcec41074a0 2024-09-10T08:37:05.628 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.753+0000 7fcec8edb640 1 --2- 172.21.15.172:0/2087471621 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fceb0038f70 0x7fceb003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:05.628 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.753+0000 7fcec8edb640 1 --2- 172.21.15.172:0/2087471621 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fceb0038f70 0x7fceb003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fceb40098c0 tx=0x7fceb4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:05.628 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.757+0000 7fceba7fc640 1 -- 172.21.15.172:0/2087471621 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcec0045360 con 0x7fcec41074a0 2024-09-10T08:37:05.628 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:01.928+0000 7fcecb940640 1 -- 172.21.15.172:0/2087471621 --> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi172", "addr": "172.21.15.172", "target": ["mon-mgr", ""]}) -- 0x7fcec4103e40 con 0x7fceb0038f70 2024-09-10T08:37:05.628 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.111+0000 7fceba7fc640 1 -- 172.21.15.172:0/2087471621 <== mgr.14118 v2:172.21.15.172:6800/2333617420 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fcec4103e40 con 0x7fceb0038f70 2024-09-10T08:37:05.628 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.114+0000 7fcecb940640 1 -- 172.21.15.172:0/2087471621 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fceb0038f70 msgr2=0x7fceb003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:05.628 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.114+0000 7fcecb940640 1 --2- 172.21.15.172:0/2087471621 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fceb0038f70 0x7fceb003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fceb40098c0 tx=0x7fceb4006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:05.628 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.114+0000 7fcecb940640 1 -- 172.21.15.172:0/2087471621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcec41074a0 msgr2=0x7fcec419b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:05.628 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.114+0000 7fcecb940640 1 --2- 172.21.15.172:0/2087471621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcec41074a0 0x7fcec419b410 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009940 tx=0x7fcec0032e10 comp rx=0 tx=0).stop 2024-09-10T08:37:05.628 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.114+0000 7fcecb940640 1 -- 172.21.15.172:0/2087471621 shutdown_connections 2024-09-10T08:37:05.628 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.114+0000 7fcecb940640 1 --2- 172.21.15.172:0/2087471621 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fceb0038f70 0x7fceb003b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:05.629 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.114+0000 7fcecb940640 1 --2- 172.21.15.172:0/2087471621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcec41074a0 0x7fcec419b410 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:05.629 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.114+0000 7fcecb940640 1 -- 172.21.15.172:0/2087471621 >> 172.21.15.172:0/2087471621 conn(0x7fcec4102c50 msgr2=0x7fcec4103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:05.629 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.114+0000 7fcecb940640 1 -- 172.21.15.172:0/2087471621 shutdown_connections 2024-09-10T08:37:05.629 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.114+0000 7fcecb940640 1 -- 172.21.15.172:0/2087471621 wait complete. 2024-09-10T08:37:05.629 INFO:teuthology.orchestra.run.smithi172.stdout:Deploying unmanaged mon service... 2024-09-10T08:37:06.189 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:06 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' 2024-09-10T08:37:06.189 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:06 smithi172 ceph-mon[30854]: Added host smithi172 2024-09-10T08:37:06.190 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:06 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:37:06.874 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T08:37:06.874 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.989+0000 7f85e0941640 1 Processor -- start 2024-09-10T08:37:06.874 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.989+0000 7f85e0941640 1 -- start start 2024-09-10T08:37:06.874 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.990+0000 7f85e0941640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85dc105760 0x7f85dc105b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:06.874 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.990+0000 7f85e0941640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc106130 con 0x7f85dc105760 2024-09-10T08:37:06.874 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.990+0000 7f85da59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85dc105760 0x7f85dc105b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:06.874 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.990+0000 7f85da59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85dc105760 0x7f85dc105b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50306/0 (socket says 172.21.15.172:50306) 2024-09-10T08:37:06.875 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.990+0000 7f85da59c640 1 -- 172.21.15.172:0/796727699 learned_addr learned my addr 172.21.15.172:0/796727699 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:06.875 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.990+0000 7f85da59c640 1 -- 172.21.15.172:0/796727699 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85dc108740 con 0x7f85dc105760 2024-09-10T08:37:06.875 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.991+0000 7f85da59c640 1 --2- 172.21.15.172:0/796727699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85dc105760 0x7f85dc105b60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009820 tx=0x7f85cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4035cbba196f40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:06.875 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.991+0000 7f85d959a640 1 -- 172.21.15.172:0/796727699 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85cc037070 con 0x7f85dc105760 2024-09-10T08:37:06.875 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.991+0000 7f85d959a640 1 -- 172.21.15.172:0/796727699 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f85cc0328d0 con 0x7f85dc105760 2024-09-10T08:37:06.875 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.991+0000 7f85d959a640 1 -- 172.21.15.172:0/796727699 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85cc0307d0 con 0x7f85dc105760 2024-09-10T08:37:06.875 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.991+0000 7f85e0941640 1 -- 172.21.15.172:0/796727699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85dc105760 msgr2=0x7f85dc105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:06.875 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.991+0000 7f85e0941640 1 --2- 172.21.15.172:0/796727699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85dc105760 0x7f85dc105b60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009820 tx=0x7f85cc02a270 comp rx=0 tx=0).stop 2024-09-10T08:37:06.875 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.992+0000 7f85e0941640 1 -- 172.21.15.172:0/796727699 shutdown_connections 2024-09-10T08:37:06.875 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.992+0000 7f85e0941640 1 --2- 172.21.15.172:0/796727699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85dc105760 0x7f85dc105b60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:06.875 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.992+0000 7f85e0941640 1 -- 172.21.15.172:0/796727699 >> 172.21.15.172:0/796727699 conn(0x7f85dc101270 msgr2=0x7f85dc1036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:06.875 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.992+0000 7f85e0941640 1 -- 172.21.15.172:0/796727699 shutdown_connections 2024-09-10T08:37:06.875 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.992+0000 7f85e0941640 1 -- 172.21.15.172:0/796727699 wait complete. 2024-09-10T08:37:06.876 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.992+0000 7f85e0941640 1 Processor -- start 2024-09-10T08:37:06.876 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.992+0000 7f85e0941640 1 -- start start 2024-09-10T08:37:06.876 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.993+0000 7f85e0941640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85dc105760 0x7f85dc19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:06.876 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.993+0000 7f85e0941640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc10b970 con 0x7f85dc105760 2024-09-10T08:37:06.876 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.993+0000 7f85da59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85dc105760 0x7f85dc19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:06.876 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.993+0000 7f85da59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85dc105760 0x7f85dc19b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50322/0 (socket says 172.21.15.172:50322) 2024-09-10T08:37:06.876 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.993+0000 7f85da59c640 1 -- 172.21.15.172:0/2514640057 learned_addr learned my addr 172.21.15.172:0/2514640057 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:06.876 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.993+0000 7f85da59c640 1 -- 172.21.15.172:0/2514640057 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85dc19ba00 con 0x7f85dc105760 2024-09-10T08:37:06.876 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.993+0000 7f85da59c640 1 --2- 172.21.15.172:0/2514640057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85dc105760 0x7f85dc19b4c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f85cc030520 tx=0x7f85cc032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:06.876 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.993+0000 7f85cb7fe640 1 -- 172.21.15.172:0/2514640057 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85cc042070 con 0x7f85dc105760 2024-09-10T08:37:06.876 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.993+0000 7f85cb7fe640 1 -- 172.21.15.172:0/2514640057 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f85cc030cc0 con 0x7f85dc105760 2024-09-10T08:37:06.876 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.993+0000 7f85e0941640 1 -- 172.21.15.172:0/2514640057 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85dc19bc90 con 0x7f85dc105760 2024-09-10T08:37:06.877 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.993+0000 7f85cb7fe640 1 -- 172.21.15.172:0/2514640057 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85cc037040 con 0x7f85dc105760 2024-09-10T08:37:06.877 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.993+0000 7f85e0941640 1 -- 172.21.15.172:0/2514640057 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85dc19e980 con 0x7f85dc105760 2024-09-10T08:37:06.877 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.994+0000 7f85cb7fe640 1 -- 172.21.15.172:0/2514640057 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f85cc045070 con 0x7f85dc105760 2024-09-10T08:37:06.877 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.994+0000 7f85cb7fe640 1 --2- 172.21.15.172:0/2514640057 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f85c4038fc0 0x7f85c403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:06.877 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.994+0000 7f85e0941640 1 -- 172.21.15.172:0/2514640057 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85dc105be0 con 0x7f85dc105760 2024-09-10T08:37:06.877 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.994+0000 7f85cb7fe640 1 -- 172.21.15.172:0/2514640057 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f85cc06c1a0 con 0x7f85dc105760 2024-09-10T08:37:06.877 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.995+0000 7f85d9d9b640 1 --2- 172.21.15.172:0/2514640057 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f85c4038fc0 0x7f85c403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:06.877 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.995+0000 7f85d9d9b640 1 --2- 172.21.15.172:0/2514640057 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f85c4038fc0 0x7f85c403b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f85d00098c0 tx=0x7f85d0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:06.877 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:05.999+0000 7f85cb7fe640 1 -- 172.21.15.172:0/2514640057 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85cc045360 con 0x7f85dc105760 2024-09-10T08:37:06.877 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:06.189+0000 7f85e0941640 1 -- 172.21.15.172:0/2514640057 --> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f85dc102590 con 0x7f85c4038fc0 2024-09-10T08:37:06.877 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:06.194+0000 7f85cb7fe640 1 -- 172.21.15.172:0/2514640057 <== mgr.14118 v2:172.21.15.172:6800/2333617420 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f85dc102590 con 0x7f85c4038fc0 2024-09-10T08:37:06.877 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:06.196+0000 7f85e0941640 1 -- 172.21.15.172:0/2514640057 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f85c4038fc0 msgr2=0x7f85c403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:06.877 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:06.196+0000 7f85e0941640 1 --2- 172.21.15.172:0/2514640057 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f85c4038fc0 0x7f85c403b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f85d00098c0 tx=0x7f85d0006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:06.878 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:06.196+0000 7f85e0941640 1 -- 172.21.15.172:0/2514640057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85dc105760 msgr2=0x7f85dc19b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:06.878 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:06.196+0000 7f85e0941640 1 --2- 172.21.15.172:0/2514640057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85dc105760 0x7f85dc19b4c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f85cc030520 tx=0x7f85cc032dd0 comp rx=0 tx=0).stop 2024-09-10T08:37:06.878 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:06.196+0000 7f85e0941640 1 -- 172.21.15.172:0/2514640057 shutdown_connections 2024-09-10T08:37:06.878 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:06.196+0000 7f85e0941640 1 --2- 172.21.15.172:0/2514640057 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f85c4038fc0 0x7f85c403b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:06.878 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:06.196+0000 7f85e0941640 1 --2- 172.21.15.172:0/2514640057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85dc105760 0x7f85dc19b4c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:06.878 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:06.196+0000 7f85e0941640 1 -- 172.21.15.172:0/2514640057 >> 172.21.15.172:0/2514640057 conn(0x7f85dc101270 msgr2=0x7f85dc101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:06.878 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:06.196+0000 7f85e0941640 1 -- 172.21.15.172:0/2514640057 shutdown_connections 2024-09-10T08:37:06.878 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:06.196+0000 7f85e0941640 1 -- 172.21.15.172:0/2514640057 wait complete. 2024-09-10T08:37:06.878 INFO:teuthology.orchestra.run.smithi172.stdout:Deploying unmanaged mgr service... 2024-09-10T08:37:07.453 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:07 smithi172 ceph-mon[30854]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:07.454 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:07 smithi172 ceph-mon[30854]: Saving service mon spec with placement count:5 2024-09-10T08:37:07.454 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:07 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' 2024-09-10T08:37:08.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T08:37:08.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.272+0000 7fe829cec640 1 Processor -- start 2024-09-10T08:37:08.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.272+0000 7fe829cec640 1 -- start start 2024-09-10T08:37:08.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.272+0000 7fe829cec640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8241063a0 0x7fe8241067a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:08.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.272+0000 7fe829cec640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe824106d70 con 0x7fe8241063a0 2024-09-10T08:37:08.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.273+0000 7fe8237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8241063a0 0x7fe8241067a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:08.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.273+0000 7fe8237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8241063a0 0x7fe8241067a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58612/0 (socket says 172.21.15.172:58612) 2024-09-10T08:37:08.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.273+0000 7fe8237fe640 1 -- 172.21.15.172:0/2060299058 learned_addr learned my addr 172.21.15.172:0/2060299058 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:08.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.273+0000 7fe8237fe640 1 -- 172.21.15.172:0/2060299058 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8241075a0 con 0x7fe8241063a0 2024-09-10T08:37:08.050 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.273+0000 7fe8237fe640 1 --2- 172.21.15.172:0/2060299058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8241063a0 0x7fe8241067a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe814009820 tx=0x7fe81402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fadd7951d94410d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:08.050 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.274+0000 7fe822ffd640 1 -- 172.21.15.172:0/2060299058 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe814037070 con 0x7fe8241063a0 2024-09-10T08:37:08.050 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.274+0000 7fe822ffd640 1 -- 172.21.15.172:0/2060299058 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe8140328d0 con 0x7fe8241063a0 2024-09-10T08:37:08.050 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.274+0000 7fe822ffd640 1 -- 172.21.15.172:0/2060299058 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8140307d0 con 0x7fe8241063a0 2024-09-10T08:37:08.050 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.274+0000 7fe829cec640 1 -- 172.21.15.172:0/2060299058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8241063a0 msgr2=0x7fe8241067a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:08.050 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.274+0000 7fe829cec640 1 --2- 172.21.15.172:0/2060299058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8241063a0 0x7fe8241067a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe814009820 tx=0x7fe81402a270 comp rx=0 tx=0).stop 2024-09-10T08:37:08.050 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.274+0000 7fe829cec640 1 -- 172.21.15.172:0/2060299058 shutdown_connections 2024-09-10T08:37:08.050 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.274+0000 7fe829cec640 1 --2- 172.21.15.172:0/2060299058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8241063a0 0x7fe8241067a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:08.050 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.274+0000 7fe829cec640 1 -- 172.21.15.172:0/2060299058 >> 172.21.15.172:0/2060299058 conn(0x7fe824101b10 msgr2=0x7fe824103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:08.050 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.274+0000 7fe829cec640 1 -- 172.21.15.172:0/2060299058 shutdown_connections 2024-09-10T08:37:08.050 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.274+0000 7fe829cec640 1 -- 172.21.15.172:0/2060299058 wait complete. 2024-09-10T08:37:08.050 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.275+0000 7fe829cec640 1 Processor -- start 2024-09-10T08:37:08.050 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.275+0000 7fe829cec640 1 -- start start 2024-09-10T08:37:08.051 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.275+0000 7fe829cec640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8241063a0 0x7fe8240754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:08.051 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.275+0000 7fe829cec640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe82410a950 con 0x7fe8241063a0 2024-09-10T08:37:08.051 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.275+0000 7fe8237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8241063a0 0x7fe8240754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:08.051 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.275+0000 7fe8237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8241063a0 0x7fe8240754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58620/0 (socket says 172.21.15.172:58620) 2024-09-10T08:37:08.051 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.275+0000 7fe8237fe640 1 -- 172.21.15.172:0/2061023688 learned_addr learned my addr 172.21.15.172:0/2061023688 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:08.051 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.276+0000 7fe8237fe640 1 -- 172.21.15.172:0/2061023688 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82419e1b0 con 0x7fe8241063a0 2024-09-10T08:37:08.051 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.276+0000 7fe8237fe640 1 --2- 172.21.15.172:0/2061023688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8241063a0 0x7fe8240754a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe814030520 tx=0x7fe814032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:08.051 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.276+0000 7fe8217fa640 1 -- 172.21.15.172:0/2061023688 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe814042070 con 0x7fe8241063a0 2024-09-10T08:37:08.051 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.276+0000 7fe8217fa640 1 -- 172.21.15.172:0/2061023688 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe814030cc0 con 0x7fe8241063a0 2024-09-10T08:37:08.051 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.276+0000 7fe829cec640 1 -- 172.21.15.172:0/2061023688 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe82419e380 con 0x7fe8241063a0 2024-09-10T08:37:08.051 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.276+0000 7fe8217fa640 1 -- 172.21.15.172:0/2061023688 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe814037040 con 0x7fe8241063a0 2024-09-10T08:37:08.051 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.276+0000 7fe829cec640 1 -- 172.21.15.172:0/2061023688 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe82419e720 con 0x7fe8241063a0 2024-09-10T08:37:08.051 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.277+0000 7fe8217fa640 1 -- 172.21.15.172:0/2061023688 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe814045070 con 0x7fe8241063a0 2024-09-10T08:37:08.052 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.277+0000 7fe8217fa640 1 --2- 172.21.15.172:0/2061023688 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fe810038fc0 0x7fe81003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:08.052 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.277+0000 7fe829cec640 1 -- 172.21.15.172:0/2061023688 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe82410aad0 con 0x7fe8241063a0 2024-09-10T08:37:08.052 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.277+0000 7fe8217fa640 1 -- 172.21.15.172:0/2061023688 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe81406bf10 con 0x7fe8241063a0 2024-09-10T08:37:08.052 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.277+0000 7fe81affd640 1 --2- 172.21.15.172:0/2061023688 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fe810038fc0 0x7fe81003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:08.052 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.278+0000 7fe81affd640 1 --2- 172.21.15.172:0/2061023688 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fe810038fc0 0x7fe81003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0098c0 tx=0x7fe80c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:08.052 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.282+0000 7fe8217fa640 1 -- 172.21.15.172:0/2061023688 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe814045360 con 0x7fe8241063a0 2024-09-10T08:37:08.052 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.466+0000 7fe829cec640 1 -- 172.21.15.172:0/2061023688 --> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe824102d90 con 0x7fe810038fc0 2024-09-10T08:37:08.052 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.471+0000 7fe8217fa640 1 -- 172.21.15.172:0/2061023688 <== mgr.14118 v2:172.21.15.172:6800/2333617420 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe824102d90 con 0x7fe810038fc0 2024-09-10T08:37:08.052 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.473+0000 7fe829cec640 1 -- 172.21.15.172:0/2061023688 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fe810038fc0 msgr2=0x7fe81003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:08.052 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.473+0000 7fe829cec640 1 --2- 172.21.15.172:0/2061023688 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fe810038fc0 0x7fe81003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0098c0 tx=0x7fe80c006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:08.052 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.473+0000 7fe829cec640 1 -- 172.21.15.172:0/2061023688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8241063a0 msgr2=0x7fe8240754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:08.052 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.473+0000 7fe829cec640 1 --2- 172.21.15.172:0/2061023688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8241063a0 0x7fe8240754a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe814030520 tx=0x7fe814032dd0 comp rx=0 tx=0).stop 2024-09-10T08:37:08.053 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.473+0000 7fe829cec640 1 -- 172.21.15.172:0/2061023688 shutdown_connections 2024-09-10T08:37:08.053 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.473+0000 7fe829cec640 1 --2- 172.21.15.172:0/2061023688 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fe810038fc0 0x7fe81003b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:08.053 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.473+0000 7fe829cec640 1 --2- 172.21.15.172:0/2061023688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8241063a0 0x7fe8240754a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:08.053 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.473+0000 7fe829cec640 1 -- 172.21.15.172:0/2061023688 >> 172.21.15.172:0/2061023688 conn(0x7fe824101b10 msgr2=0x7fe824102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:08.053 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.473+0000 7fe829cec640 1 -- 172.21.15.172:0/2061023688 shutdown_connections 2024-09-10T08:37:08.053 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:07.473+0000 7fe829cec640 1 -- 172.21.15.172:0/2061023688 wait complete. 2024-09-10T08:37:08.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:08 smithi172 ceph-mon[30854]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:08.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:08 smithi172 ceph-mon[30854]: Saving service mgr spec with placement count:2 2024-09-10T08:37:08.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:08 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' 2024-09-10T08:37:09.451 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.484+0000 7ff366507640 1 Processor -- start 2024-09-10T08:37:09.451 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.484+0000 7ff366507640 1 -- start start 2024-09-10T08:37:09.451 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.484+0000 7ff366507640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3601074b0 0x7ff3601078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:09.451 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.484+0000 7ff366507640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff360107e80 con 0x7ff3601074b0 2024-09-10T08:37:09.451 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.484+0000 7ff35ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3601074b0 0x7ff3601078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:09.451 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.484+0000 7ff35ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3601074b0 0x7ff3601078b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58634/0 (socket says 172.21.15.172:58634) 2024-09-10T08:37:09.451 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.484+0000 7ff35ffff640 1 -- 172.21.15.172:0/2206197914 learned_addr learned my addr 172.21.15.172:0/2206197914 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:09.452 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.485+0000 7ff35ffff640 1 -- 172.21.15.172:0/2206197914 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3601086b0 con 0x7ff3601074b0 2024-09-10T08:37:09.452 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.485+0000 7ff35ffff640 1 --2- 172.21.15.172:0/2206197914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3601074b0 0x7ff3601078b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff350009820 tx=0x7ff35002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5286ba72cc5c197e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:09.452 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.485+0000 7ff35effd640 1 -- 172.21.15.172:0/2206197914 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff350037070 con 0x7ff3601074b0 2024-09-10T08:37:09.452 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.485+0000 7ff35effd640 1 -- 172.21.15.172:0/2206197914 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff3500328d0 con 0x7ff3601074b0 2024-09-10T08:37:09.452 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.485+0000 7ff35effd640 1 -- 172.21.15.172:0/2206197914 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3500307d0 con 0x7ff3601074b0 2024-09-10T08:37:09.452 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.486+0000 7ff366507640 1 -- 172.21.15.172:0/2206197914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3601074b0 msgr2=0x7ff3601078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:09.452 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.486+0000 7ff366507640 1 --2- 172.21.15.172:0/2206197914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3601074b0 0x7ff3601078b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff350009820 tx=0x7ff35002a270 comp rx=0 tx=0).stop 2024-09-10T08:37:09.452 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.486+0000 7ff366507640 1 -- 172.21.15.172:0/2206197914 shutdown_connections 2024-09-10T08:37:09.452 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.486+0000 7ff366507640 1 --2- 172.21.15.172:0/2206197914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3601074b0 0x7ff3601078b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:09.452 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.486+0000 7ff366507640 1 -- 172.21.15.172:0/2206197914 >> 172.21.15.172:0/2206197914 conn(0x7ff360102c60 msgr2=0x7ff360105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:09.452 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.486+0000 7ff366507640 1 -- 172.21.15.172:0/2206197914 shutdown_connections 2024-09-10T08:37:09.453 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.486+0000 7ff366507640 1 -- 172.21.15.172:0/2206197914 wait complete. 2024-09-10T08:37:09.453 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.487+0000 7ff366507640 1 Processor -- start 2024-09-10T08:37:09.453 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.487+0000 7ff366507640 1 -- start start 2024-09-10T08:37:09.453 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.487+0000 7ff366507640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3601074b0 0x7ff36019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:09.453 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.487+0000 7ff366507640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff36010ba60 con 0x7ff3601074b0 2024-09-10T08:37:09.453 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.487+0000 7ff35ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3601074b0 0x7ff36019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:09.453 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.487+0000 7ff35ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3601074b0 0x7ff36019b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58638/0 (socket says 172.21.15.172:58638) 2024-09-10T08:37:09.453 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.487+0000 7ff35ffff640 1 -- 172.21.15.172:0/3940121260 learned_addr learned my addr 172.21.15.172:0/3940121260 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:09.453 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.487+0000 7ff35ffff640 1 -- 172.21.15.172:0/3940121260 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff36019b960 con 0x7ff3601074b0 2024-09-10T08:37:09.453 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.488+0000 7ff35ffff640 1 --2- 172.21.15.172:0/3940121260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3601074b0 0x7ff36019b420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff3500097f0 tx=0x7ff350030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:09.454 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.488+0000 7ff35d7fa640 1 -- 172.21.15.172:0/3940121260 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff350037040 con 0x7ff3601074b0 2024-09-10T08:37:09.454 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.488+0000 7ff366507640 1 -- 172.21.15.172:0/3940121260 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff36019bbf0 con 0x7ff3601074b0 2024-09-10T08:37:09.454 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.488+0000 7ff35d7fa640 1 -- 172.21.15.172:0/3940121260 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff350032bb0 con 0x7ff3601074b0 2024-09-10T08:37:09.454 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.488+0000 7ff35d7fa640 1 -- 172.21.15.172:0/3940121260 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff35003a390 con 0x7ff3601074b0 2024-09-10T08:37:09.454 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.488+0000 7ff366507640 1 -- 172.21.15.172:0/3940121260 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff36019e8e0 con 0x7ff3601074b0 2024-09-10T08:37:09.454 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.489+0000 7ff35d7fa640 1 -- 172.21.15.172:0/3940121260 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff35003a5d0 con 0x7ff3601074b0 2024-09-10T08:37:09.454 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.489+0000 7ff366507640 1 -- 172.21.15.172:0/3940121260 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff36010bbe0 con 0x7ff3601074b0 2024-09-10T08:37:09.454 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.489+0000 7ff35d7fa640 1 --2- 172.21.15.172:0/3940121260 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7ff348038f70 0x7ff34803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:09.454 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.489+0000 7ff35d7fa640 1 -- 172.21.15.172:0/3940121260 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff35006c330 con 0x7ff3601074b0 2024-09-10T08:37:09.454 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.489+0000 7ff35f7fe640 1 --2- 172.21.15.172:0/3940121260 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7ff348038f70 0x7ff34803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:09.455 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.490+0000 7ff35f7fe640 1 --2- 172.21.15.172:0/3940121260 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7ff348038f70 0x7ff34803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff3540098c0 tx=0x7ff354006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:09.455 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.494+0000 7ff35d7fa640 1 -- 172.21.15.172:0/3940121260 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff350032dd0 con 0x7ff3601074b0 2024-09-10T08:37:09.455 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.663+0000 7ff366507640 1 -- 172.21.15.172:0/3940121260 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7ff360107930 con 0x7ff3601074b0 2024-09-10T08:37:09.455 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.668+0000 7ff35d7fa640 1 -- 172.21.15.172:0/3940121260 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7ff350031560 con 0x7ff3601074b0 2024-09-10T08:37:09.455 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.671+0000 7ff366507640 1 -- 172.21.15.172:0/3940121260 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7ff348038f70 msgr2=0x7ff34803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:09.455 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.671+0000 7ff366507640 1 --2- 172.21.15.172:0/3940121260 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7ff348038f70 0x7ff34803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff3540098c0 tx=0x7ff354006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:09.455 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.671+0000 7ff366507640 1 -- 172.21.15.172:0/3940121260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3601074b0 msgr2=0x7ff36019b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:09.455 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.671+0000 7ff366507640 1 --2- 172.21.15.172:0/3940121260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3601074b0 0x7ff36019b420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff3500097f0 tx=0x7ff350030d00 comp rx=0 tx=0).stop 2024-09-10T08:37:09.455 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.671+0000 7ff366507640 1 -- 172.21.15.172:0/3940121260 shutdown_connections 2024-09-10T08:37:09.455 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.671+0000 7ff366507640 1 --2- 172.21.15.172:0/3940121260 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7ff348038f70 0x7ff34803b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:09.456 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.671+0000 7ff366507640 1 --2- 172.21.15.172:0/3940121260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3601074b0 0x7ff36019b420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:09.456 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.671+0000 7ff366507640 1 -- 172.21.15.172:0/3940121260 >> 172.21.15.172:0/3940121260 conn(0x7ff360102c60 msgr2=0x7ff360103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:09.456 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.671+0000 7ff366507640 1 -- 172.21.15.172:0/3940121260 shutdown_connections 2024-09-10T08:37:09.456 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:08.671+0000 7ff366507640 1 -- 172.21.15.172:0/3940121260 wait complete. 2024-09-10T08:37:09.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:09 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3940121260' entity='client.admin' 2024-09-10T08:37:11.194 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:11 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' 2024-09-10T08:37:11.195 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:11 smithi172 ceph-mon[30854]: from='mgr.14118 172.21.15.172:0/2378242087' entity='mgr.x' 2024-09-10T08:37:11.195 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:11 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2084726782' entity='client.admin' 2024-09-10T08:37:11.196 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.392+0000 7fbc96309640 1 Processor -- start 2024-09-10T08:37:11.201 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.393+0000 7fbc96309640 1 -- start start 2024-09-10T08:37:11.202 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.393+0000 7fbc96309640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc90107450 0x7fbc90107850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:11.202 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.393+0000 7fbc96309640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbc90107e20 con 0x7fbc90107450 2024-09-10T08:37:11.202 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.393+0000 7fbc8ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc90107450 0x7fbc90107850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:11.202 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.393+0000 7fbc8ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc90107450 0x7fbc90107850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58652/0 (socket says 172.21.15.172:58652) 2024-09-10T08:37:11.202 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.393+0000 7fbc8ffff640 1 -- 172.21.15.172:0/792290099 learned_addr learned my addr 172.21.15.172:0/792290099 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:11.202 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.393+0000 7fbc8ffff640 1 -- 172.21.15.172:0/792290099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc90108650 con 0x7fbc90107450 2024-09-10T08:37:11.202 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.394+0000 7fbc8ffff640 1 --2- 172.21.15.172:0/792290099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc90107450 0x7fbc90107850 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009820 tx=0x7fbc8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbf21561d5152d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:11.202 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.394+0000 7fbc8effd640 1 -- 172.21.15.172:0/792290099 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc80037070 con 0x7fbc90107450 2024-09-10T08:37:11.202 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.395+0000 7fbc8effd640 1 -- 172.21.15.172:0/792290099 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc800328f0 con 0x7fbc90107450 2024-09-10T08:37:11.202 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.395+0000 7fbc8effd640 1 -- 172.21.15.172:0/792290099 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc80030740 con 0x7fbc90107450 2024-09-10T08:37:11.203 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.395+0000 7fbc96309640 1 -- 172.21.15.172:0/792290099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc90107450 msgr2=0x7fbc90107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:11.203 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.395+0000 7fbc96309640 1 --2- 172.21.15.172:0/792290099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc90107450 0x7fbc90107850 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009820 tx=0x7fbc8002a270 comp rx=0 tx=0).stop 2024-09-10T08:37:11.203 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.395+0000 7fbc96309640 1 -- 172.21.15.172:0/792290099 shutdown_connections 2024-09-10T08:37:11.203 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.395+0000 7fbc96309640 1 --2- 172.21.15.172:0/792290099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc90107450 0x7fbc90107850 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:11.203 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.395+0000 7fbc96309640 1 -- 172.21.15.172:0/792290099 >> 172.21.15.172:0/792290099 conn(0x7fbc90102bc0 msgr2=0x7fbc90105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:11.203 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.395+0000 7fbc96309640 1 -- 172.21.15.172:0/792290099 shutdown_connections 2024-09-10T08:37:11.203 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.395+0000 7fbc96309640 1 -- 172.21.15.172:0/792290099 wait complete. 2024-09-10T08:37:11.203 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.396+0000 7fbc96309640 1 Processor -- start 2024-09-10T08:37:11.203 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.396+0000 7fbc96309640 1 -- start start 2024-09-10T08:37:11.203 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.397+0000 7fbc96309640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc90107450 0x7fbc9019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:11.203 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.397+0000 7fbc96309640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9010ba00 con 0x7fbc90107450 2024-09-10T08:37:11.203 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.397+0000 7fbc8ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc90107450 0x7fbc9019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:11.204 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.397+0000 7fbc8ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc90107450 0x7fbc9019b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58658/0 (socket says 172.21.15.172:58658) 2024-09-10T08:37:11.204 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.397+0000 7fbc8ffff640 1 -- 172.21.15.172:0/2084726782 learned_addr learned my addr 172.21.15.172:0/2084726782 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:11.204 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.397+0000 7fbc8ffff640 1 -- 172.21.15.172:0/2084726782 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9019b950 con 0x7fbc90107450 2024-09-10T08:37:11.204 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.397+0000 7fbc8ffff640 1 --2- 172.21.15.172:0/2084726782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc90107450 0x7fbc9019b410 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009940 tx=0x7fbc80030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:11.204 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.398+0000 7fbc8d7fa640 1 -- 172.21.15.172:0/2084726782 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc80042070 con 0x7fbc90107450 2024-09-10T08:37:11.204 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.398+0000 7fbc96309640 1 -- 172.21.15.172:0/2084726782 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9019bbe0 con 0x7fbc90107450 2024-09-10T08:37:11.204 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.398+0000 7fbc8d7fa640 1 -- 172.21.15.172:0/2084726782 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc80030e30 con 0x7fbc90107450 2024-09-10T08:37:11.204 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.398+0000 7fbc96309640 1 -- 172.21.15.172:0/2084726782 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc9019e8d0 con 0x7fbc90107450 2024-09-10T08:37:11.204 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.398+0000 7fbc8d7fa640 1 -- 172.21.15.172:0/2084726782 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc8004c060 con 0x7fbc90107450 2024-09-10T08:37:11.204 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.399+0000 7fbc96309640 1 -- 172.21.15.172:0/2084726782 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc9010bb80 con 0x7fbc90107450 2024-09-10T08:37:11.204 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.399+0000 7fbc8d7fa640 1 -- 172.21.15.172:0/2084726782 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbc80045070 con 0x7fbc90107450 2024-09-10T08:37:11.204 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.399+0000 7fbc8d7fa640 1 --2- 172.21.15.172:0/2084726782 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fbc78038f70 0x7fbc7803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:11.205 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.399+0000 7fbc8d7fa640 1 -- 172.21.15.172:0/2084726782 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbc8006c030 con 0x7fbc90107450 2024-09-10T08:37:11.205 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.399+0000 7fbc8f7fe640 1 --2- 172.21.15.172:0/2084726782 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fbc78038f70 0x7fbc7803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:11.205 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.400+0000 7fbc8f7fe640 1 --2- 172.21.15.172:0/2084726782 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fbc78038f70 0x7fbc7803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbc840098c0 tx=0x7fbc84006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:11.205 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.404+0000 7fbc8d7fa640 1 -- 172.21.15.172:0/2084726782 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc8003a380 con 0x7fbc90107450 2024-09-10T08:37:11.205 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.580+0000 7fbc96309640 1 -- 172.21.15.172:0/2084726782 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fbc901078d0 con 0x7fbc90107450 2024-09-10T08:37:11.205 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.583+0000 7fbc8d7fa640 1 -- 172.21.15.172:0/2084726782 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fbc8002e200 con 0x7fbc90107450 2024-09-10T08:37:11.205 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.586+0000 7fbc96309640 1 -- 172.21.15.172:0/2084726782 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fbc78038f70 msgr2=0x7fbc7803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:11.205 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.586+0000 7fbc96309640 1 --2- 172.21.15.172:0/2084726782 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fbc78038f70 0x7fbc7803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbc840098c0 tx=0x7fbc84006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:11.205 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.586+0000 7fbc96309640 1 -- 172.21.15.172:0/2084726782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc90107450 msgr2=0x7fbc9019b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:11.205 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.586+0000 7fbc96309640 1 --2- 172.21.15.172:0/2084726782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc90107450 0x7fbc9019b410 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009940 tx=0x7fbc80030cc0 comp rx=0 tx=0).stop 2024-09-10T08:37:11.205 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.586+0000 7fbc96309640 1 -- 172.21.15.172:0/2084726782 shutdown_connections 2024-09-10T08:37:11.205 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.587+0000 7fbc96309640 1 --2- 172.21.15.172:0/2084726782 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fbc78038f70 0x7fbc7803b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:11.206 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.587+0000 7fbc96309640 1 --2- 172.21.15.172:0/2084726782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc90107450 0x7fbc9019b410 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:11.206 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.587+0000 7fbc96309640 1 -- 172.21.15.172:0/2084726782 >> 172.21.15.172:0/2084726782 conn(0x7fbc90102bc0 msgr2=0x7fbc90103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:11.206 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.587+0000 7fbc96309640 1 -- 172.21.15.172:0/2084726782 shutdown_connections 2024-09-10T08:37:11.206 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:10.587+0000 7fbc96309640 1 -- 172.21.15.172:0/2084726782 wait complete. 2024-09-10T08:37:11.206 INFO:teuthology.orchestra.run.smithi172.stdout:Enabling the dashboard module... 2024-09-10T08:37:12.174 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:12 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1321771831' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T08:37:12.476 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:12 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: ignoring --setuser ceph since I am not root 2024-09-10T08:37:12.476 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:12 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: ignoring --setgroup ceph since I am not root 2024-09-10T08:37:12.476 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:12 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:12.298+0000 7f04ca0f4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T08:37:12.476 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:12 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:12.355+0000 7f04ca0f4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T08:37:12.730 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:12 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:12.514+0000 7f04ca0f4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T08:37:12.730 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:12 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:12.580+0000 7f04ca0f4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T08:37:12.730 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:12 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:12.729+0000 7f04ca0f4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:37:13.038 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:12 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:12.872+0000 7f04ca0f4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T08:37:13.046 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.650+0000 7fb2ec25b640 1 Processor -- start 2024-09-10T08:37:13.046 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.650+0000 7fb2ec25b640 1 -- start start 2024-09-10T08:37:13.046 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.650+0000 7fb2ec25b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2e4107490 0x7fb2e4107890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:13.046 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.650+0000 7fb2ec25b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4107e60 con 0x7fb2e4107490 2024-09-10T08:37:13.046 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.651+0000 7fb2e9ff7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2e4107490 0x7fb2e4107890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:13.046 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.651+0000 7fb2e9ff7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2e4107490 0x7fb2e4107890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58664/0 (socket says 172.21.15.172:58664) 2024-09-10T08:37:13.046 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.651+0000 7fb2e9ff7640 1 -- 172.21.15.172:0/3154573597 learned_addr learned my addr 172.21.15.172:0/3154573597 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:13.046 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.651+0000 7fb2e9ff7640 1 -- 172.21.15.172:0/3154573597 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e4108690 con 0x7fb2e4107490 2024-09-10T08:37:13.046 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.652+0000 7fb2e9ff7640 1 --2- 172.21.15.172:0/3154573597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2e4107490 0x7fb2e4107890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009e80 tx=0x7fb2dc02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698617921ed43ab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:13.046 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.652+0000 7fb2e8ff5640 1 -- 172.21.15.172:0/3154573597 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2dc036070 con 0x7fb2e4107490 2024-09-10T08:37:13.046 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.652+0000 7fb2e8ff5640 1 -- 172.21.15.172:0/3154573597 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2dc0318b0 con 0x7fb2e4107490 2024-09-10T08:37:13.047 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.652+0000 7fb2e8ff5640 1 -- 172.21.15.172:0/3154573597 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2dc007930 con 0x7fb2e4107490 2024-09-10T08:37:13.047 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.652+0000 7fb2ec25b640 1 -- 172.21.15.172:0/3154573597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2e4107490 msgr2=0x7fb2e4107890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:13.047 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.652+0000 7fb2ec25b640 1 --2- 172.21.15.172:0/3154573597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2e4107490 0x7fb2e4107890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009e80 tx=0x7fb2dc02a7a0 comp rx=0 tx=0).stop 2024-09-10T08:37:13.047 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.653+0000 7fb2ec25b640 1 -- 172.21.15.172:0/3154573597 shutdown_connections 2024-09-10T08:37:13.047 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.653+0000 7fb2ec25b640 1 --2- 172.21.15.172:0/3154573597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2e4107490 0x7fb2e4107890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:13.047 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.653+0000 7fb2ec25b640 1 -- 172.21.15.172:0/3154573597 >> 172.21.15.172:0/3154573597 conn(0x7fb2e4102ca0 msgr2=0x7fb2e41050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:13.047 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.653+0000 7fb2ec25b640 1 -- 172.21.15.172:0/3154573597 shutdown_connections 2024-09-10T08:37:13.047 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.653+0000 7fb2ec25b640 1 -- 172.21.15.172:0/3154573597 wait complete. 2024-09-10T08:37:13.047 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.653+0000 7fb2ec25b640 1 Processor -- start 2024-09-10T08:37:13.047 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.653+0000 7fb2ec25b640 1 -- start start 2024-09-10T08:37:13.047 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.653+0000 7fb2ec25b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2e4107490 0x7fb2e419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:13.047 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.653+0000 7fb2ec25b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e410ba40 con 0x7fb2e4107490 2024-09-10T08:37:13.048 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.654+0000 7fb2e9ff7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2e4107490 0x7fb2e419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:13.048 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.654+0000 7fb2e9ff7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2e4107490 0x7fb2e419b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58674/0 (socket says 172.21.15.172:58674) 2024-09-10T08:37:13.048 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.654+0000 7fb2e9ff7640 1 -- 172.21.15.172:0/1321771831 learned_addr learned my addr 172.21.15.172:0/1321771831 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:13.048 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.654+0000 7fb2e9ff7640 1 -- 172.21.15.172:0/1321771831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e419b940 con 0x7fb2e4107490 2024-09-10T08:37:13.048 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.654+0000 7fb2e9ff7640 1 --2- 172.21.15.172:0/1321771831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2e4107490 0x7fb2e419b400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc007b60 tx=0x7fb2dc031c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:13.048 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.654+0000 7fb2e2ffd640 1 -- 172.21.15.172:0/1321771831 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2dc036050 con 0x7fb2e4107490 2024-09-10T08:37:13.048 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.654+0000 7fb2e2ffd640 1 -- 172.21.15.172:0/1321771831 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2dc042070 con 0x7fb2e4107490 2024-09-10T08:37:13.048 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.654+0000 7fb2ec25b640 1 -- 172.21.15.172:0/1321771831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e419bbd0 con 0x7fb2e4107490 2024-09-10T08:37:13.048 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.654+0000 7fb2e2ffd640 1 -- 172.21.15.172:0/1321771831 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2dc03da30 con 0x7fb2e4107490 2024-09-10T08:37:13.048 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.654+0000 7fb2ec25b640 1 -- 172.21.15.172:0/1321771831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e419e8c0 con 0x7fb2e4107490 2024-09-10T08:37:13.048 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.655+0000 7fb2e2ffd640 1 -- 172.21.15.172:0/1321771831 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb2dc03dce0 con 0x7fb2e4107490 2024-09-10T08:37:13.048 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.655+0000 7fb2ec25b640 1 -- 172.21.15.172:0/1321771831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e410bbc0 con 0x7fb2e4107490 2024-09-10T08:37:13.048 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.655+0000 7fb2e2ffd640 1 --2- 172.21.15.172:0/1321771831 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fb2cc038fc0 0x7fb2cc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:13.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.655+0000 7fb2e2ffd640 1 -- 172.21.15.172:0/1321771831 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb2dc06d6e0 con 0x7fb2e4107490 2024-09-10T08:37:13.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.655+0000 7fb2e97f6640 1 --2- 172.21.15.172:0/1321771831 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fb2cc038fc0 0x7fb2cc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:13.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.656+0000 7fb2e97f6640 1 --2- 172.21.15.172:0/1321771831 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fb2cc038fc0 0x7fb2cc03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80098c0 tx=0x7fb2d8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:13.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.661+0000 7fb2e2ffd640 1 -- 172.21.15.172:0/1321771831 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2dc044310 con 0x7fb2e4107490 2024-09-10T08:37:13.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:11.910+0000 7fb2ec25b640 1 -- 172.21.15.172:0/1321771831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fb2e419ea90 con 0x7fb2e4107490 2024-09-10T08:37:13.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:12.146+0000 7fb2e2ffd640 1 -- 172.21.15.172:0/1321771831 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fb2dc039300 con 0x7fb2e4107490 2024-09-10T08:37:13.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:12.148+0000 7fb2e2ffd640 1 -- 172.21.15.172:0/1321771831 <== mon.0 v2:172.21.15.172:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fb2dc04dcb0 con 0x7fb2e4107490 2024-09-10T08:37:13.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:12.149+0000 7fb2ec25b640 1 -- 172.21.15.172:0/1321771831 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fb2cc038fc0 msgr2=0x7fb2cc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:13.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:12.149+0000 7fb2ec25b640 1 --2- 172.21.15.172:0/1321771831 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fb2cc038fc0 0x7fb2cc03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80098c0 tx=0x7fb2d8006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:13.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:12.149+0000 7fb2ec25b640 1 -- 172.21.15.172:0/1321771831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2e4107490 msgr2=0x7fb2e419b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:13.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:12.149+0000 7fb2ec25b640 1 --2- 172.21.15.172:0/1321771831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2e4107490 0x7fb2e419b400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc007b60 tx=0x7fb2dc031c90 comp rx=0 tx=0).stop 2024-09-10T08:37:13.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:12.149+0000 7fb2ec25b640 1 -- 172.21.15.172:0/1321771831 shutdown_connections 2024-09-10T08:37:13.049 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:12.149+0000 7fb2ec25b640 1 --2- 172.21.15.172:0/1321771831 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7fb2cc038fc0 0x7fb2cc03b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:13.050 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:12.149+0000 7fb2ec25b640 1 --2- 172.21.15.172:0/1321771831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2e4107490 0x7fb2e419b400 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:13.050 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:12.149+0000 7fb2ec25b640 1 -- 172.21.15.172:0/1321771831 >> 172.21.15.172:0/1321771831 conn(0x7fb2e4102ca0 msgr2=0x7fb2e4103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:13.050 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:12.149+0000 7fb2ec25b640 1 -- 172.21.15.172:0/1321771831 shutdown_connections 2024-09-10T08:37:13.050 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:12.149+0000 7fb2ec25b640 1 -- 172.21.15.172:0/1321771831 wait complete. 2024-09-10T08:37:13.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:13 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1321771831' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T08:37:13.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:13 smithi172 ceph-mon[30854]: mgrmap e9: x(active, since 17s) 2024-09-10T08:37:13.332 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:13 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:13.193+0000 7f04ca0f4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T08:37:13.332 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:13 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:13.256+0000 7f04ca0f4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T08:37:13.657 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:13 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:13.447+0000 7f04ca0f4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T08:37:14.321 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:14 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/4130482388' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T08:37:14.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout { 2024-09-10T08:37:14.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T08:37:14.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T08:37:14.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T08:37:14.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T08:37:14.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout } 2024-09-10T08:37:14.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.450+0000 7f46d12d0640 1 Processor -- start 2024-09-10T08:37:14.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.450+0000 7f46d12d0640 1 -- start start 2024-09-10T08:37:14.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.450+0000 7f46d12d0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46cc1074a0 0x7f46cc1078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:14.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.450+0000 7f46d12d0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc107e70 con 0x7f46cc1074a0 2024-09-10T08:37:14.329 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.450+0000 7f46caffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46cc1074a0 0x7f46cc1078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:14.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.450+0000 7f46caffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46cc1074a0 0x7f46cc1078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58696/0 (socket says 172.21.15.172:58696) 2024-09-10T08:37:14.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.450+0000 7f46caffd640 1 -- 172.21.15.172:0/1777351968 learned_addr learned my addr 172.21.15.172:0/1777351968 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:14.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.451+0000 7f46caffd640 1 -- 172.21.15.172:0/1777351968 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc1086a0 con 0x7f46cc1074a0 2024-09-10T08:37:14.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.451+0000 7f46caffd640 1 --2- 172.21.15.172:0/1777351968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46cc1074a0 0x7f46cc1078a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009820 tx=0x7f46bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b905cf5974e881a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:14.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.451+0000 7f46c9ffb640 1 -- 172.21.15.172:0/1777351968 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46bc037070 con 0x7f46cc1074a0 2024-09-10T08:37:14.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.452+0000 7f46c9ffb640 1 -- 172.21.15.172:0/1777351968 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46bc0328f0 con 0x7f46cc1074a0 2024-09-10T08:37:14.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.452+0000 7f46c9ffb640 1 -- 172.21.15.172:0/1777351968 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46bc030740 con 0x7f46cc1074a0 2024-09-10T08:37:14.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.452+0000 7f46d12d0640 1 -- 172.21.15.172:0/1777351968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46cc1074a0 msgr2=0x7f46cc1078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:14.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.452+0000 7f46d12d0640 1 --2- 172.21.15.172:0/1777351968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46cc1074a0 0x7f46cc1078a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009820 tx=0x7f46bc02a270 comp rx=0 tx=0).stop 2024-09-10T08:37:14.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.452+0000 7f46d12d0640 1 -- 172.21.15.172:0/1777351968 shutdown_connections 2024-09-10T08:37:14.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.452+0000 7f46d12d0640 1 --2- 172.21.15.172:0/1777351968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46cc1074a0 0x7f46cc1078a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:14.330 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.452+0000 7f46d12d0640 1 -- 172.21.15.172:0/1777351968 >> 172.21.15.172:0/1777351968 conn(0x7f46cc102c50 msgr2=0x7f46cc105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:14.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.452+0000 7f46d12d0640 1 -- 172.21.15.172:0/1777351968 shutdown_connections 2024-09-10T08:37:14.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.452+0000 7f46d12d0640 1 -- 172.21.15.172:0/1777351968 wait complete. 2024-09-10T08:37:14.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.453+0000 7f46d12d0640 1 Processor -- start 2024-09-10T08:37:14.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.453+0000 7f46d12d0640 1 -- start start 2024-09-10T08:37:14.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.453+0000 7f46d12d0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46cc1074a0 0x7f46cc19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:14.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.453+0000 7f46d12d0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc10ba50 con 0x7f46cc1074a0 2024-09-10T08:37:14.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.453+0000 7f46caffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46cc1074a0 0x7f46cc19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:14.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.453+0000 7f46caffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46cc1074a0 0x7f46cc19b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58700/0 (socket says 172.21.15.172:58700) 2024-09-10T08:37:14.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.453+0000 7f46caffd640 1 -- 172.21.15.172:0/4130482388 learned_addr learned my addr 172.21.15.172:0/4130482388 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:14.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.454+0000 7f46caffd640 1 -- 172.21.15.172:0/4130482388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc19b9a0 con 0x7f46cc1074a0 2024-09-10T08:37:14.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.454+0000 7f46caffd640 1 --2- 172.21.15.172:0/4130482388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46cc1074a0 0x7f46cc19b460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009940 tx=0x7f46bc030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:14.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.454+0000 7f46b3fff640 1 -- 172.21.15.172:0/4130482388 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46bc042070 con 0x7f46cc1074a0 2024-09-10T08:37:14.331 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.454+0000 7f46b3fff640 1 -- 172.21.15.172:0/4130482388 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46bc030e30 con 0x7f46cc1074a0 2024-09-10T08:37:14.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.454+0000 7f46d12d0640 1 -- 172.21.15.172:0/4130482388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46cc19bc30 con 0x7f46cc1074a0 2024-09-10T08:37:14.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.454+0000 7f46b3fff640 1 -- 172.21.15.172:0/4130482388 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46bc037040 con 0x7f46cc1074a0 2024-09-10T08:37:14.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.454+0000 7f46d12d0640 1 -- 172.21.15.172:0/4130482388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46cc19e920 con 0x7f46cc1074a0 2024-09-10T08:37:14.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.455+0000 7f46b3fff640 1 -- 172.21.15.172:0/4130482388 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f46bc045070 con 0x7f46cc1074a0 2024-09-10T08:37:14.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.455+0000 7f46d12d0640 1 -- 172.21.15.172:0/4130482388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46cc10bbd0 con 0x7f46cc1074a0 2024-09-10T08:37:14.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.455+0000 7f46b3fff640 1 --2- 172.21.15.172:0/4130482388 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f46b4038fc0 0x7f46b403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:14.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.455+0000 7f46b3fff640 1 -- 172.21.15.172:0/4130482388 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f46bc06c220 con 0x7f46cc1074a0 2024-09-10T08:37:14.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.455+0000 7f46ca7fc640 1 -- 172.21.15.172:0/4130482388 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f46b4038fc0 msgr2=0x7f46b403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.172:6800/2333617420 2024-09-10T08:37:14.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.455+0000 7f46ca7fc640 1 --2- 172.21.15.172:0/4130482388 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f46b4038fc0 0x7f46b403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T08:37:14.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.460+0000 7f46b3fff640 1 -- 172.21.15.172:0/4130482388 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46bc03add0 con 0x7f46cc1074a0 2024-09-10T08:37:14.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.655+0000 7f46ca7fc640 1 -- 172.21.15.172:0/4130482388 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f46b4038fc0 msgr2=0x7f46b403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.172:6800/2333617420 2024-09-10T08:37:14.332 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.655+0000 7f46ca7fc640 1 --2- 172.21.15.172:0/4130482388 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f46b4038fc0 0x7f46b403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T08:37:14.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.681+0000 7f46d12d0640 1 -- 172.21.15.172:0/4130482388 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f46cc19eaf0 con 0x7f46cc1074a0 2024-09-10T08:37:14.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.681+0000 7f46b3fff640 1 -- 172.21.15.172:0/4130482388 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f46bc045520 con 0x7f46cc1074a0 2024-09-10T08:37:14.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.684+0000 7f46d12d0640 1 -- 172.21.15.172:0/4130482388 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f46b4038fc0 msgr2=0x7f46b403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:37:14.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.684+0000 7f46d12d0640 1 --2- 172.21.15.172:0/4130482388 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f46b4038fc0 0x7f46b403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:14.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.684+0000 7f46d12d0640 1 -- 172.21.15.172:0/4130482388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46cc1074a0 msgr2=0x7f46cc19b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:14.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.684+0000 7f46d12d0640 1 --2- 172.21.15.172:0/4130482388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46cc1074a0 0x7f46cc19b460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009940 tx=0x7f46bc030cc0 comp rx=0 tx=0).stop 2024-09-10T08:37:14.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.684+0000 7f46d12d0640 1 -- 172.21.15.172:0/4130482388 shutdown_connections 2024-09-10T08:37:14.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.684+0000 7f46d12d0640 1 --2- 172.21.15.172:0/4130482388 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f46b4038fc0 0x7f46b403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:14.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.684+0000 7f46d12d0640 1 --2- 172.21.15.172:0/4130482388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46cc1074a0 0x7f46cc19b460 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:14.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.684+0000 7f46d12d0640 1 -- 172.21.15.172:0/4130482388 >> 172.21.15.172:0/4130482388 conn(0x7f46cc102c50 msgr2=0x7f46cc103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:14.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.684+0000 7f46d12d0640 1 -- 172.21.15.172:0/4130482388 shutdown_connections 2024-09-10T08:37:14.333 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:13.684+0000 7f46d12d0640 1 -- 172.21.15.172:0/4130482388 wait complete. 2024-09-10T08:37:14.333 INFO:teuthology.orchestra.run.smithi172.stdout:Waiting for the mgr to restart... 2024-09-10T08:37:14.334 INFO:teuthology.orchestra.run.smithi172.stdout:Waiting for mgr epoch 9... 2024-09-10T08:37:14.582 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:14 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:14.358+0000 7f04ca0f4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T08:37:14.582 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:14 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:14.428+0000 7f04ca0f4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T08:37:15.270 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:15 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:15.153+0000 7f04ca0f4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T08:37:15.270 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:15 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:15.268+0000 7f04ca0f4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T08:37:15.582 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:15 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:15.411+0000 7f04ca0f4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:37:15.582 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:15 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:15.472+0000 7f04ca0f4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T08:37:15.906 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:15 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:15.621+0000 7f04ca0f4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T08:37:15.906 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:15 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:15.689+0000 7f04ca0f4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T08:37:15.906 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:15 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:15.748+0000 7f04ca0f4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T08:37:15.906 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:15 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:15.905+0000 7f04ca0f4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T08:37:16.176 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:15 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:15.969+0000 7f04ca0f4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T08:37:16.176 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:16 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:16.034+0000 7f04ca0f4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T08:37:16.176 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:16 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:16.115+0000 7f04ca0f4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T08:37:16.176 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:16 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:16.175+0000 7f04ca0f4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T08:37:16.581 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:16 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:16.233+0000 7f04ca0f4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T08:37:17.332 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:17 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:17.016+0000 7f04ca0f4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T08:37:17.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:17 smithi172 ceph-mon[30854]: Active manager daemon x restarted 2024-09-10T08:37:17.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:17 smithi172 ceph-mon[30854]: Activating manager daemon x 2024-09-10T08:37:17.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:17 smithi172 ceph-mon[30854]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T08:37:17.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:17 smithi172 ceph-mon[30854]: mgrmap e10: x(active, starting, since 0.00350028s) 2024-09-10T08:37:17.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:37:17.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T08:37:17.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T08:37:17.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T08:37:17.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T08:37:17.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:17 smithi172 ceph-mon[30854]: Manager daemon x is now available 2024-09-10T08:37:17.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:37:17.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T08:37:17.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T08:37:18.332 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:18.015+0000 7f0435408640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:08:37:18] ENGINE Error in HTTPServer.serve 2024-09-10T08:37:18.332 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: Traceback (most recent call last): 2024-09-10T08:37:18.332 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T08:37:18.332 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: self._connections.run(self.expiration_interval) 2024-09-10T08:37:18.332 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T08:37:18.332 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: self._run(expiration_interval) 2024-09-10T08:37:18.332 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T08:37:18.333 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T08:37:18.333 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T08:37:18.333 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T08:37:18.333 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T08:37:18.333 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: s = self.context.wrap_socket( 2024-09-10T08:37:18.333 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T08:37:18.333 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: return self.sslsocket_class._create( 2024-09-10T08:37:18.333 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T08:37:18.333 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: self.do_handshake() 2024-09-10T08:37:18.333 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T08:37:18.333 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: self._sslobj.do_handshake() 2024-09-10T08:37:18.333 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T08:37:18.333 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:37:18 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:37:18.880 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout { 2024-09-10T08:37:18.880 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T08:37:18.880 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T08:37:18.881 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout } 2024-09-10T08:37:18.881 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.730+0000 7f8f1d268640 1 Processor -- start 2024-09-10T08:37:18.881 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.730+0000 7f8f1d268640 1 -- start start 2024-09-10T08:37:18.881 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.730+0000 7f8f1d268640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18107f80 0x7f8f1810a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:18.881 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.730+0000 7f8f1d268640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1807bdb0 con 0x7f8f18107f80 2024-09-10T08:37:18.881 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.730+0000 7f8f16ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18107f80 0x7f8f1810a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:18.881 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.730+0000 7f8f16ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18107f80 0x7f8f1810a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58716/0 (socket says 172.21.15.172:58716) 2024-09-10T08:37:18.881 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.730+0000 7f8f16ffd640 1 -- 172.21.15.172:0/596504583 learned_addr learned my addr 172.21.15.172:0/596504583 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:18.882 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.730+0000 7f8f16ffd640 1 -- 172.21.15.172:0/596504583 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1810a8b0 con 0x7f8f18107f80 2024-09-10T08:37:18.882 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.731+0000 7f8f16ffd640 1 --2- 172.21.15.172:0/596504583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18107f80 0x7f8f1810a370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009820 tx=0x7f8f0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378cb96b11901635 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:18.882 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.731+0000 7f8f167fc640 1 -- 172.21.15.172:0/596504583 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f08037070 con 0x7f8f18107f80 2024-09-10T08:37:18.882 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.731+0000 7f8f167fc640 1 -- 172.21.15.172:0/596504583 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f080328f0 con 0x7f8f18107f80 2024-09-10T08:37:18.882 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.731+0000 7f8f167fc640 1 -- 172.21.15.172:0/596504583 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f08030740 con 0x7f8f18107f80 2024-09-10T08:37:18.882 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.732+0000 7f8f1d268640 1 -- 172.21.15.172:0/596504583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18107f80 msgr2=0x7f8f1810a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:18.882 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.732+0000 7f8f1d268640 1 --2- 172.21.15.172:0/596504583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18107f80 0x7f8f1810a370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009820 tx=0x7f8f0802a270 comp rx=0 tx=0).stop 2024-09-10T08:37:18.882 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.732+0000 7f8f1d268640 1 -- 172.21.15.172:0/596504583 shutdown_connections 2024-09-10T08:37:18.882 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.732+0000 7f8f1d268640 1 --2- 172.21.15.172:0/596504583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18107f80 0x7f8f1810a370 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:18.882 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.732+0000 7f8f1d268640 1 -- 172.21.15.172:0/596504583 >> 172.21.15.172:0/596504583 conn(0x7f8f18101ba0 msgr2=0x7f8f18103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:18.882 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.732+0000 7f8f1d268640 1 -- 172.21.15.172:0/596504583 shutdown_connections 2024-09-10T08:37:18.883 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.733+0000 7f8f1d268640 1 -- 172.21.15.172:0/596504583 wait complete. 2024-09-10T08:37:18.883 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.733+0000 7f8f1d268640 1 Processor -- start 2024-09-10T08:37:18.883 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.733+0000 7f8f1d268640 1 -- start start 2024-09-10T08:37:18.883 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.734+0000 7f8f1d268640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18107f80 0x7f8f1819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:18.883 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.734+0000 7f8f1d268640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1810b870 con 0x7f8f18107f80 2024-09-10T08:37:18.883 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.734+0000 7f8f16ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18107f80 0x7f8f1819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:18.883 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.734+0000 7f8f16ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18107f80 0x7f8f1819b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58724/0 (socket says 172.21.15.172:58724) 2024-09-10T08:37:18.883 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.734+0000 7f8f16ffd640 1 -- 172.21.15.172:0/2303124073 learned_addr learned my addr 172.21.15.172:0/2303124073 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:18.883 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.734+0000 7f8f16ffd640 1 -- 172.21.15.172:0/2303124073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1819b960 con 0x7f8f18107f80 2024-09-10T08:37:18.883 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.734+0000 7f8f16ffd640 1 --2- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18107f80 0x7f8f1819b420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8f08030cc0 tx=0x7f8f08030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:18.883 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.734+0000 7f8f14ff9640 1 -- 172.21.15.172:0/2303124073 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f08042070 con 0x7f8f18107f80 2024-09-10T08:37:18.883 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.734+0000 7f8f14ff9640 1 -- 172.21.15.172:0/2303124073 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f08030e60 con 0x7f8f18107f80 2024-09-10T08:37:18.884 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.734+0000 7f8f1d268640 1 -- 172.21.15.172:0/2303124073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1819bbf0 con 0x7f8f18107f80 2024-09-10T08:37:18.884 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.734+0000 7f8f1d268640 1 -- 172.21.15.172:0/2303124073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f1819e8e0 con 0x7f8f18107f80 2024-09-10T08:37:18.884 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.734+0000 7f8f14ff9640 1 -- 172.21.15.172:0/2303124073 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f08037040 con 0x7f8f18107f80 2024-09-10T08:37:18.884 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.735+0000 7f8f14ff9640 1 -- 172.21.15.172:0/2303124073 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f8f08045070 con 0x7f8f18107f80 2024-09-10T08:37:18.884 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.735+0000 7f8f14ff9640 1 --2- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f8f04039020 0x7f8f0403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:18.884 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.735+0000 7f8f14ff9640 1 -- 172.21.15.172:0/2303124073 --> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8f0403bbf0 con 0x7f8f04039020 2024-09-10T08:37:18.884 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.735+0000 7f8f14ff9640 1 -- 172.21.15.172:0/2303124073 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8f0806c090 con 0x7f8f18107f80 2024-09-10T08:37:18.884 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.735+0000 7f8f0ffff640 1 -- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f8f04039020 msgr2=0x7f8f0403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.172:6800/2333617420 2024-09-10T08:37:18.884 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.735+0000 7f8f0ffff640 1 --2- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f8f04039020 0x7f8f0403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T08:37:18.884 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.936+0000 7f8f0ffff640 1 -- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f8f04039020 msgr2=0x7f8f0403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.172:6800/2333617420 2024-09-10T08:37:18.884 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:14.936+0000 7f8f0ffff640 1 --2- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f8f04039020 0x7f8f0403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T08:37:18.885 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:15.337+0000 7f8f0ffff640 1 -- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f8f04039020 msgr2=0x7f8f0403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.172:6800/2333617420 2024-09-10T08:37:18.885 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:15.337+0000 7f8f0ffff640 1 --2- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f8f04039020 0x7f8f0403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T08:37:18.885 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:16.138+0000 7f8f0ffff640 1 -- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f8f04039020 msgr2=0x7f8f0403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.172:6800/2333617420 2024-09-10T08:37:18.885 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:16.138+0000 7f8f0ffff640 1 --2- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f8f04039020 0x7f8f0403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T08:37:18.885 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:17.019+0000 7f8f14ff9640 1 -- 172.21.15.172:0/2303124073 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f8f0806ab40 con 0x7f8f18107f80 2024-09-10T08:37:18.885 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:17.019+0000 7f8f14ff9640 1 -- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f8f04039020 msgr2=0x7f8f0403b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:37:18.885 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:17.019+0000 7f8f14ff9640 1 --2- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f8f04039020 0x7f8f0403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:18.885 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.020+0000 7f8f14ff9640 1 -- 172.21.15.172:0/2303124073 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f8f0803b330 con 0x7f8f18107f80 2024-09-10T08:37:18.885 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.020+0000 7f8f14ff9640 1 --2- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8f0403cae0 0x7f8f0403eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:18.885 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.020+0000 7f8f14ff9640 1 -- 172.21.15.172:0/2303124073 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8f0403bbf0 con 0x7f8f0403cae0 2024-09-10T08:37:18.885 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.021+0000 7f8f0ffff640 1 --2- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8f0403cae0 0x7f8f0403eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:18.885 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.022+0000 7f8f0ffff640 1 --2- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8f0403cae0 0x7f8f0403eed0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8f00003980 tx=0x7f8f000092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:18.886 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.022+0000 7f8f14ff9640 1 -- 172.21.15.172:0/2303124073 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f8f0403bbf0 con 0x7f8f0403cae0 2024-09-10T08:37:18.886 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.029+0000 7f8f1d268640 1 -- 172.21.15.172:0/2303124073 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8f181083e0 con 0x7f8f0403cae0 2024-09-10T08:37:18.886 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.029+0000 7f8f14ff9640 1 -- 172.21.15.172:0/2303124073 <== mgr.14150 v2:172.21.15.172:6800/107199760 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f8f181083e0 con 0x7f8f0403cae0 2024-09-10T08:37:18.886 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.029+0000 7f8f1d268640 1 -- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8f0403cae0 msgr2=0x7f8f0403eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:18.886 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.029+0000 7f8f1d268640 1 --2- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8f0403cae0 0x7f8f0403eed0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8f00003980 tx=0x7f8f000092b0 comp rx=0 tx=0).stop 2024-09-10T08:37:18.886 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.029+0000 7f8f1d268640 1 -- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18107f80 msgr2=0x7f8f1819b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:18.886 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.029+0000 7f8f1d268640 1 --2- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18107f80 0x7f8f1819b420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8f08030cc0 tx=0x7f8f08030cf0 comp rx=0 tx=0).stop 2024-09-10T08:37:18.886 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.030+0000 7f8f1d268640 1 -- 172.21.15.172:0/2303124073 shutdown_connections 2024-09-10T08:37:18.886 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.030+0000 7f8f1d268640 1 --2- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8f0403cae0 0x7f8f0403eed0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:18.886 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.030+0000 7f8f1d268640 1 --2- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:6800/2333617420,v1:172.21.15.172:6801/2333617420] conn(0x7f8f04039020 0x7f8f0403b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:18.886 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.030+0000 7f8f1d268640 1 --2- 172.21.15.172:0/2303124073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18107f80 0x7f8f1819b420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:18.886 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.030+0000 7f8f1d268640 1 -- 172.21.15.172:0/2303124073 >> 172.21.15.172:0/2303124073 conn(0x7f8f18101ba0 msgr2=0x7f8f181026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:18.887 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.030+0000 7f8f1d268640 1 -- 172.21.15.172:0/2303124073 shutdown_connections 2024-09-10T08:37:18.887 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:18.030+0000 7f8f1d268640 1 -- 172.21.15.172:0/2303124073 wait complete. 2024-09-10T08:37:18.887 INFO:teuthology.orchestra.run.smithi172.stdout:mgr epoch 9 is available 2024-09-10T08:37:18.887 INFO:teuthology.orchestra.run.smithi172.stdout:Generating a dashboard self-signed certificate... 2024-09-10T08:37:19.124 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:19 smithi172 ceph-mon[30854]: [10/Sep/2024:08:37:17] ENGINE Bus STARTING 2024-09-10T08:37:19.124 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:19 smithi172 ceph-mon[30854]: [10/Sep/2024:08:37:18] ENGINE Serving on https://172.21.15.172:7150 2024-09-10T08:37:19.124 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:19 smithi172 ceph-mon[30854]: [10/Sep/2024:08:37:18] ENGINE Error in HTTPServer.serve 2024-09-10T08:37:19.124 INFO:journalctl@ceph.mon.a.smithi172.stdout: Traceback (most recent call last): 2024-09-10T08:37:19.124 INFO:journalctl@ceph.mon.a.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T08:37:19.125 INFO:journalctl@ceph.mon.a.smithi172.stdout: self._connections.run(self.expiration_interval) 2024-09-10T08:37:19.125 INFO:journalctl@ceph.mon.a.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T08:37:19.125 INFO:journalctl@ceph.mon.a.smithi172.stdout: self._run(expiration_interval) 2024-09-10T08:37:19.125 INFO:journalctl@ceph.mon.a.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T08:37:19.125 INFO:journalctl@ceph.mon.a.smithi172.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T08:37:19.125 INFO:journalctl@ceph.mon.a.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T08:37:19.125 INFO:journalctl@ceph.mon.a.smithi172.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T08:37:19.125 INFO:journalctl@ceph.mon.a.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T08:37:19.125 INFO:journalctl@ceph.mon.a.smithi172.stdout: s = self.context.wrap_socket( 2024-09-10T08:37:19.125 INFO:journalctl@ceph.mon.a.smithi172.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T08:37:19.125 INFO:journalctl@ceph.mon.a.smithi172.stdout: return self.sslsocket_class._create( 2024-09-10T08:37:19.125 INFO:journalctl@ceph.mon.a.smithi172.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T08:37:19.126 INFO:journalctl@ceph.mon.a.smithi172.stdout: self.do_handshake() 2024-09-10T08:37:19.126 INFO:journalctl@ceph.mon.a.smithi172.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T08:37:19.126 INFO:journalctl@ceph.mon.a.smithi172.stdout: self._sslobj.do_handshake() 2024-09-10T08:37:19.126 INFO:journalctl@ceph.mon.a.smithi172.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T08:37:19.126 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:19 smithi172 ceph-mon[30854]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T08:37:19.126 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:19 smithi172 ceph-mon[30854]: mgrmap e11: x(active, since 1.0052s) 2024-09-10T08:37:19.126 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:19 smithi172 ceph-mon[30854]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T08:37:19.126 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:19 smithi172 ceph-mon[30854]: [10/Sep/2024:08:37:18] ENGINE Serving on http://172.21.15.172:8765 2024-09-10T08:37:19.126 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:19 smithi172 ceph-mon[30854]: [10/Sep/2024:08:37:18] ENGINE Bus STARTED 2024-09-10T08:37:19.126 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:20.345 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T08:37:20.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.398+0000 7f61917e5640 1 Processor -- start 2024-09-10T08:37:20.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.398+0000 7f61917e5640 1 -- start start 2024-09-10T08:37:20.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.398+0000 7f61917e5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c105b70 0x7f618c105f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.398+0000 7f61917e5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f618c106540 con 0x7f618c105b70 2024-09-10T08:37:20.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.399+0000 7f618affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c105b70 0x7f618c105f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.399+0000 7f618affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c105b70 0x7f618c105f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39020/0 (socket says 172.21.15.172:39020) 2024-09-10T08:37:20.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.399+0000 7f618affd640 1 -- 172.21.15.172:0/2991922731 learned_addr learned my addr 172.21.15.172:0/2991922731 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:20.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.399+0000 7f618affd640 1 -- 172.21.15.172:0/2991922731 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618c106d70 con 0x7f618c105b70 2024-09-10T08:37:20.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.399+0000 7f618affd640 1 --2- 172.21.15.172:0/2991922731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c105b70 0x7f618c105f70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f617c009820 tx=0x7f617c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6898bc82e49f3a67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:20.346 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.400+0000 7f6189ffb640 1 -- 172.21.15.172:0/2991922731 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f617c037070 con 0x7f618c105b70 2024-09-10T08:37:20.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.400+0000 7f6189ffb640 1 -- 172.21.15.172:0/2991922731 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f617c0328f0 con 0x7f618c105b70 2024-09-10T08:37:20.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.400+0000 7f6189ffb640 1 -- 172.21.15.172:0/2991922731 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f617c030740 con 0x7f618c105b70 2024-09-10T08:37:20.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.400+0000 7f61917e5640 1 -- 172.21.15.172:0/2991922731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c105b70 msgr2=0x7f618c105f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:20.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.400+0000 7f61917e5640 1 --2- 172.21.15.172:0/2991922731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c105b70 0x7f618c105f70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f617c009820 tx=0x7f617c02a270 comp rx=0 tx=0).stop 2024-09-10T08:37:20.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.400+0000 7f61917e5640 1 -- 172.21.15.172:0/2991922731 shutdown_connections 2024-09-10T08:37:20.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.400+0000 7f61917e5640 1 --2- 172.21.15.172:0/2991922731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c105b70 0x7f618c105f70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.400+0000 7f61917e5640 1 -- 172.21.15.172:0/2991922731 >> 172.21.15.172:0/2991922731 conn(0x7f618c101340 msgr2=0x7f618c103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:20.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.400+0000 7f61917e5640 1 -- 172.21.15.172:0/2991922731 shutdown_connections 2024-09-10T08:37:20.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.400+0000 7f61917e5640 1 -- 172.21.15.172:0/2991922731 wait complete. 2024-09-10T08:37:20.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.401+0000 7f61917e5640 1 Processor -- start 2024-09-10T08:37:20.347 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.401+0000 7f61917e5640 1 -- start start 2024-09-10T08:37:20.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.401+0000 7f61917e5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c105b70 0x7f618c19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.401+0000 7f61917e5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f618c107550 con 0x7f618c105b70 2024-09-10T08:37:20.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.401+0000 7f618affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c105b70 0x7f618c19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.401+0000 7f618affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c105b70 0x7f618c19b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39022/0 (socket says 172.21.15.172:39022) 2024-09-10T08:37:20.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.401+0000 7f618affd640 1 -- 172.21.15.172:0/3615084308 learned_addr learned my addr 172.21.15.172:0/3615084308 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:20.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.401+0000 7f618affd640 1 -- 172.21.15.172:0/3615084308 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618c19ba00 con 0x7f618c105b70 2024-09-10T08:37:20.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.402+0000 7f618affd640 1 --2- 172.21.15.172:0/3615084308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c105b70 0x7f618c19b4c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f617c0097f0 tx=0x7f617c030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:20.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.402+0000 7f6183fff640 1 -- 172.21.15.172:0/3615084308 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f617c042070 con 0x7f618c105b70 2024-09-10T08:37:20.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.402+0000 7f6183fff640 1 -- 172.21.15.172:0/3615084308 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f617c030e80 con 0x7f618c105b70 2024-09-10T08:37:20.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.402+0000 7f61917e5640 1 -- 172.21.15.172:0/3615084308 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618c19bc90 con 0x7f618c105b70 2024-09-10T08:37:20.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.402+0000 7f6183fff640 1 -- 172.21.15.172:0/3615084308 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f617c037040 con 0x7f618c105b70 2024-09-10T08:37:20.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.402+0000 7f61917e5640 1 -- 172.21.15.172:0/3615084308 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f618c19e980 con 0x7f618c105b70 2024-09-10T08:37:20.348 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.403+0000 7f6183fff640 1 -- 172.21.15.172:0/3615084308 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f617c045070 con 0x7f618c105b70 2024-09-10T08:37:20.349 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.403+0000 7f61917e5640 1 -- 172.21.15.172:0/3615084308 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f618c107b70 con 0x7f618c105b70 2024-09-10T08:37:20.349 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.403+0000 7f6183fff640 1 --2- 172.21.15.172:0/3615084308 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6178038ef0 0x7f617803b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:20.349 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.403+0000 7f6183fff640 1 -- 172.21.15.172:0/3615084308 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f617c06bfb0 con 0x7f618c105b70 2024-09-10T08:37:20.349 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.403+0000 7f618a7fc640 1 --2- 172.21.15.172:0/3615084308 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6178038ef0 0x7f617803b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:20.349 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.403+0000 7f618a7fc640 1 --2- 172.21.15.172:0/3615084308 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6178038ef0 0x7f617803b3b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f61740098c0 tx=0x7f6174006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:20.349 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.408+0000 7f6183fff640 1 -- 172.21.15.172:0/3615084308 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f617c045520 con 0x7f618c105b70 2024-09-10T08:37:20.349 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.486+0000 7f6183fff640 1 -- 172.21.15.172:0/3615084308 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f617c03db70 con 0x7f618c105b70 2024-09-10T08:37:20.349 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.583+0000 7f61917e5640 1 -- 172.21.15.172:0/3615084308 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f618c02da10 con 0x7f6178038ef0 2024-09-10T08:37:20.349 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.717+0000 7f6183fff640 1 -- 172.21.15.172:0/3615084308 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f618c02da10 con 0x7f6178038ef0 2024-09-10T08:37:20.349 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.719+0000 7f61917e5640 1 -- 172.21.15.172:0/3615084308 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6178038ef0 msgr2=0x7f617803b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:20.349 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.719+0000 7f61917e5640 1 --2- 172.21.15.172:0/3615084308 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6178038ef0 0x7f617803b3b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f61740098c0 tx=0x7f6174006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.350 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.719+0000 7f61917e5640 1 -- 172.21.15.172:0/3615084308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c105b70 msgr2=0x7f618c19b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:20.350 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.719+0000 7f61917e5640 1 --2- 172.21.15.172:0/3615084308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c105b70 0x7f618c19b4c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f617c0097f0 tx=0x7f617c030c70 comp rx=0 tx=0).stop 2024-09-10T08:37:20.350 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.719+0000 7f61917e5640 1 -- 172.21.15.172:0/3615084308 shutdown_connections 2024-09-10T08:37:20.350 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.719+0000 7f61917e5640 1 --2- 172.21.15.172:0/3615084308 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6178038ef0 0x7f617803b3b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.350 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.719+0000 7f61917e5640 1 --2- 172.21.15.172:0/3615084308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f618c105b70 0x7f618c19b4c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:20.350 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.719+0000 7f61917e5640 1 -- 172.21.15.172:0/3615084308 >> 172.21.15.172:0/3615084308 conn(0x7f618c101340 msgr2=0x7f618c18ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:20.350 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.720+0000 7f61917e5640 1 -- 172.21.15.172:0/3615084308 shutdown_connections 2024-09-10T08:37:20.350 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:19.720+0000 7f61917e5640 1 -- 172.21.15.172:0/3615084308 wait complete. 2024-09-10T08:37:20.350 INFO:teuthology.orchestra.run.smithi172.stdout:Creating initial admin user... 2024-09-10T08:37:20.764 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:20 smithi172 ceph-mon[30854]: mgrmap e12: x(active, since 2s) 2024-09-10T08:37:20.764 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:20 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:20.764 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:20 smithi172 ceph-mon[30854]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:20.765 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:20 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:20.765 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:20 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:22.493 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:22 smithi172 ceph-mon[30854]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:22.494 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:22 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:22.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$ZQfYiC3YCXas.pENnUIj3uHQ3qVQc1sECjbiRrg1DxjVXMyLMv6NO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725957441, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T08:37:22.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.763+0000 7fe4a1b69640 1 Processor -- start 2024-09-10T08:37:22.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.763+0000 7fe4a1b69640 1 -- start start 2024-09-10T08:37:22.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.763+0000 7fe4a1b69640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe49c07de60 0x7fe49c07c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:22.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.763+0000 7fe4a1b69640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c07c890 con 0x7fe49c07de60 2024-09-10T08:37:22.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.763+0000 7fe49b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe49c07de60 0x7fe49c07c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:22.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.763+0000 7fe49b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe49c07de60 0x7fe49c07c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39034/0 (socket says 172.21.15.172:39034) 2024-09-10T08:37:22.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.763+0000 7fe49b7fe640 1 -- 172.21.15.172:0/3383802283 learned_addr learned my addr 172.21.15.172:0/3383802283 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:22.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.764+0000 7fe49b7fe640 1 -- 172.21.15.172:0/3383802283 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c07ca10 con 0x7fe49c07de60 2024-09-10T08:37:22.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.764+0000 7fe49b7fe640 1 --2- 172.21.15.172:0/3383802283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe49c07de60 0x7fe49c07c2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009820 tx=0x7fe48c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2f0604c9afc0d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:22.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.764+0000 7fe49a7fc640 1 -- 172.21.15.172:0/3383802283 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe48c037070 con 0x7fe49c07de60 2024-09-10T08:37:22.501 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.764+0000 7fe49a7fc640 1 -- 172.21.15.172:0/3383802283 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe48c0328f0 con 0x7fe49c07de60 2024-09-10T08:37:22.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.764+0000 7fe49a7fc640 1 -- 172.21.15.172:0/3383802283 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe48c030740 con 0x7fe49c07de60 2024-09-10T08:37:22.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.765+0000 7fe4a1b69640 1 -- 172.21.15.172:0/3383802283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe49c07de60 msgr2=0x7fe49c07c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:22.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.765+0000 7fe4a1b69640 1 --2- 172.21.15.172:0/3383802283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe49c07de60 0x7fe49c07c2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009820 tx=0x7fe48c02a270 comp rx=0 tx=0).stop 2024-09-10T08:37:22.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.765+0000 7fe4a1b69640 1 -- 172.21.15.172:0/3383802283 shutdown_connections 2024-09-10T08:37:22.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.765+0000 7fe4a1b69640 1 --2- 172.21.15.172:0/3383802283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe49c07de60 0x7fe49c07c2c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:22.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.765+0000 7fe4a1b69640 1 -- 172.21.15.172:0/3383802283 >> 172.21.15.172:0/3383802283 conn(0x7fe49c102c50 msgr2=0x7fe49c105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:22.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.765+0000 7fe4a1b69640 1 -- 172.21.15.172:0/3383802283 shutdown_connections 2024-09-10T08:37:22.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.765+0000 7fe4a1b69640 1 -- 172.21.15.172:0/3383802283 wait complete. 2024-09-10T08:37:22.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.766+0000 7fe4a1b69640 1 Processor -- start 2024-09-10T08:37:22.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.766+0000 7fe4a1b69640 1 -- start start 2024-09-10T08:37:22.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.766+0000 7fe4a1b69640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe49c07de60 0x7fe49c19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:22.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.766+0000 7fe4a1b69640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c10dad0 con 0x7fe49c07de60 2024-09-10T08:37:22.502 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.766+0000 7fe49b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe49c07de60 0x7fe49c19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:22.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.766+0000 7fe49b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe49c07de60 0x7fe49c19d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39040/0 (socket says 172.21.15.172:39040) 2024-09-10T08:37:22.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.766+0000 7fe49b7fe640 1 -- 172.21.15.172:0/2578738855 learned_addr learned my addr 172.21.15.172:0/2578738855 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:22.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.767+0000 7fe49b7fe640 1 -- 172.21.15.172:0/2578738855 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c19db70 con 0x7fe49c07de60 2024-09-10T08:37:22.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.767+0000 7fe49b7fe640 1 --2- 172.21.15.172:0/2578738855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe49c07de60 0x7fe49c19d630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0097f0 tx=0x7fe48c030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:22.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.767+0000 7fe498ff9640 1 -- 172.21.15.172:0/2578738855 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe48c042070 con 0x7fe49c07de60 2024-09-10T08:37:22.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.767+0000 7fe498ff9640 1 -- 172.21.15.172:0/2578738855 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe48c030d80 con 0x7fe49c07de60 2024-09-10T08:37:22.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.767+0000 7fe4a1b69640 1 -- 172.21.15.172:0/2578738855 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49c19de00 con 0x7fe49c07de60 2024-09-10T08:37:22.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.767+0000 7fe498ff9640 1 -- 172.21.15.172:0/2578738855 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe48c037040 con 0x7fe49c07de60 2024-09-10T08:37:22.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.767+0000 7fe4a1b69640 1 -- 172.21.15.172:0/2578738855 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe49c1a0af0 con 0x7fe49c07de60 2024-09-10T08:37:22.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.768+0000 7fe498ff9640 1 -- 172.21.15.172:0/2578738855 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fe48c045070 con 0x7fe49c07de60 2024-09-10T08:37:22.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.768+0000 7fe498ff9640 1 --2- 172.21.15.172:0/2578738855 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe484038fc0 0x7fe48403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:22.503 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.768+0000 7fe498ff9640 1 -- 172.21.15.172:0/2578738855 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe48c06c290 con 0x7fe49c07de60 2024-09-10T08:37:22.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.768+0000 7fe4a1b69640 1 -- 172.21.15.172:0/2578738855 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49c07c340 con 0x7fe49c07de60 2024-09-10T08:37:22.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.768+0000 7fe49affd640 1 --2- 172.21.15.172:0/2578738855 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe484038fc0 0x7fe48403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:22.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.769+0000 7fe49affd640 1 --2- 172.21.15.172:0/2578738855 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe484038fc0 0x7fe48403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe4900098c0 tx=0x7fe490006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:22.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.773+0000 7fe498ff9640 1 -- 172.21.15.172:0/2578738855 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe48c045360 con 0x7fe49c07de60 2024-09-10T08:37:22.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:20.962+0000 7fe4a1b69640 1 -- 172.21.15.172:0/2578738855 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fe49c1a0cc0 con 0x7fe484038fc0 2024-09-10T08:37:22.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:21.185+0000 7fe498ff9640 1 -- 172.21.15.172:0/2578738855 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fe49c1a0cc0 con 0x7fe484038fc0 2024-09-10T08:37:22.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:21.187+0000 7fe4a1b69640 1 -- 172.21.15.172:0/2578738855 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe484038fc0 msgr2=0x7fe48403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:22.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:21.187+0000 7fe4a1b69640 1 --2- 172.21.15.172:0/2578738855 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe484038fc0 0x7fe48403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe4900098c0 tx=0x7fe490006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:22.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:21.187+0000 7fe4a1b69640 1 -- 172.21.15.172:0/2578738855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe49c07de60 msgr2=0x7fe49c19d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:22.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:21.187+0000 7fe4a1b69640 1 --2- 172.21.15.172:0/2578738855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe49c07de60 0x7fe49c19d630 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0097f0 tx=0x7fe48c030c70 comp rx=0 tx=0).stop 2024-09-10T08:37:22.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:21.187+0000 7fe4a1b69640 1 -- 172.21.15.172:0/2578738855 shutdown_connections 2024-09-10T08:37:22.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:21.187+0000 7fe4a1b69640 1 --2- 172.21.15.172:0/2578738855 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe484038fc0 0x7fe48403b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:22.504 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:21.187+0000 7fe4a1b69640 1 --2- 172.21.15.172:0/2578738855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe49c07de60 0x7fe49c19d630 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:22.505 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:21.187+0000 7fe4a1b69640 1 -- 172.21.15.172:0/2578738855 >> 172.21.15.172:0/2578738855 conn(0x7fe49c102c50 msgr2=0x7fe49c1037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:22.505 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:21.187+0000 7fe4a1b69640 1 -- 172.21.15.172:0/2578738855 shutdown_connections 2024-09-10T08:37:22.505 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:21.187+0000 7fe4a1b69640 1 -- 172.21.15.172:0/2578738855 wait complete. 2024-09-10T08:37:22.505 INFO:teuthology.orchestra.run.smithi172.stdout:Fetching dashboard port number... 2024-09-10T08:37:23.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:23 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2051547621' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T08:37:23.641 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T08:37:23.641 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.884+0000 7f808ba5f640 1 Processor -- start 2024-09-10T08:37:23.641 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.884+0000 7f808ba5f640 1 -- start start 2024-09-10T08:37:23.641 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.884+0000 7f808ba5f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f808407ccf0 0x7f808407b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:23.641 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.884+0000 7f808ba5f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f808407b720 con 0x7f808407ccf0 2024-09-10T08:37:23.642 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.884+0000 7f80897fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f808407ccf0 0x7f808407b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:23.642 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.884+0000 7f80897fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f808407ccf0 0x7f808407b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39056/0 (socket says 172.21.15.172:39056) 2024-09-10T08:37:23.642 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.884+0000 7f80897fb640 1 -- 172.21.15.172:0/1213144222 learned_addr learned my addr 172.21.15.172:0/1213144222 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:23.642 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.885+0000 7f80897fb640 1 -- 172.21.15.172:0/1213144222 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808407b8a0 con 0x7f808407ccf0 2024-09-10T08:37:23.642 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.885+0000 7f80897fb640 1 --2- 172.21.15.172:0/1213144222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f808407ccf0 0x7f808407b150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f807c009e80 tx=0x7f807c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4091e72953b20a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:23.642 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.885+0000 7f807bfff640 1 -- 172.21.15.172:0/1213144222 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f807c036070 con 0x7f808407ccf0 2024-09-10T08:37:23.642 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.886+0000 7f807bfff640 1 -- 172.21.15.172:0/1213144222 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f807c0318b0 con 0x7f808407ccf0 2024-09-10T08:37:23.642 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.886+0000 7f807bfff640 1 -- 172.21.15.172:0/1213144222 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f807c007930 con 0x7f808407ccf0 2024-09-10T08:37:23.642 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.886+0000 7f808ba5f640 1 -- 172.21.15.172:0/1213144222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f808407ccf0 msgr2=0x7f808407b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:23.642 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.886+0000 7f808ba5f640 1 --2- 172.21.15.172:0/1213144222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f808407ccf0 0x7f808407b150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f807c009e80 tx=0x7f807c02a7a0 comp rx=0 tx=0).stop 2024-09-10T08:37:23.642 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.886+0000 7f808ba5f640 1 -- 172.21.15.172:0/1213144222 shutdown_connections 2024-09-10T08:37:23.642 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.886+0000 7f808ba5f640 1 --2- 172.21.15.172:0/1213144222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f808407ccf0 0x7f808407b150 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:23.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.886+0000 7f808ba5f640 1 -- 172.21.15.172:0/1213144222 >> 172.21.15.172:0/1213144222 conn(0x7f8084102ca0 msgr2=0x7f80841050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:23.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.886+0000 7f808ba5f640 1 -- 172.21.15.172:0/1213144222 shutdown_connections 2024-09-10T08:37:23.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.886+0000 7f808ba5f640 1 -- 172.21.15.172:0/1213144222 wait complete. 2024-09-10T08:37:23.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.887+0000 7f808ba5f640 1 Processor -- start 2024-09-10T08:37:23.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.887+0000 7f808ba5f640 1 -- start start 2024-09-10T08:37:23.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.887+0000 7f808ba5f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f808407ccf0 0x7f808419d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:23.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.887+0000 7f808ba5f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f808410dac0 con 0x7f808407ccf0 2024-09-10T08:37:23.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.887+0000 7f80897fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f808407ccf0 0x7f808419d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:23.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.887+0000 7f80897fb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f808407ccf0 0x7f808419d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39072/0 (socket says 172.21.15.172:39072) 2024-09-10T08:37:23.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.887+0000 7f80897fb640 1 -- 172.21.15.172:0/2051547621 learned_addr learned my addr 172.21.15.172:0/2051547621 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:23.643 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.887+0000 7f80897fb640 1 -- 172.21.15.172:0/2051547621 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808419db30 con 0x7f808407ccf0 2024-09-10T08:37:23.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.888+0000 7f80897fb640 1 --2- 172.21.15.172:0/2051547621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f808407ccf0 0x7f808419d5f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f807c031c90 tx=0x7f807c031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:23.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.888+0000 7f807a7fc640 1 -- 172.21.15.172:0/2051547621 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f807c036050 con 0x7f808407ccf0 2024-09-10T08:37:23.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.888+0000 7f807a7fc640 1 -- 172.21.15.172:0/2051547621 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f807c042070 con 0x7f808407ccf0 2024-09-10T08:37:23.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.888+0000 7f808ba5f640 1 -- 172.21.15.172:0/2051547621 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808419ddc0 con 0x7f808407ccf0 2024-09-10T08:37:23.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.888+0000 7f807a7fc640 1 -- 172.21.15.172:0/2051547621 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f807c03daa0 con 0x7f808407ccf0 2024-09-10T08:37:23.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.888+0000 7f808ba5f640 1 -- 172.21.15.172:0/2051547621 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80841a0ab0 con 0x7f808407ccf0 2024-09-10T08:37:23.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.889+0000 7f807a7fc640 1 -- 172.21.15.172:0/2051547621 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f807c047460 con 0x7f808407ccf0 2024-09-10T08:37:23.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.889+0000 7f807a7fc640 1 --2- 172.21.15.172:0/2051547621 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8070038fc0 0x7f807003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:23.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.889+0000 7f808ba5f640 1 -- 172.21.15.172:0/2051547621 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808407b1d0 con 0x7f808407ccf0 2024-09-10T08:37:23.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.889+0000 7f807a7fc640 1 -- 172.21.15.172:0/2051547621 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f807c06d7e0 con 0x7f808407ccf0 2024-09-10T08:37:23.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.889+0000 7f8088ffa640 1 --2- 172.21.15.172:0/2051547621 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8070038fc0 0x7f807003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:23.644 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.889+0000 7f8088ffa640 1 --2- 172.21.15.172:0/2051547621 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8070038fc0 0x7f807003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f80740098c0 tx=0x7f8074006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:23.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:22.894+0000 7f807a7fc640 1 -- 172.21.15.172:0/2051547621 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f807c03bac0 con 0x7f808407ccf0 2024-09-10T08:37:23.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:23.062+0000 7f808ba5f640 1 -- 172.21.15.172:0/2051547621 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f80841a0c80 con 0x7f808407ccf0 2024-09-10T08:37:23.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:23.062+0000 7f807a7fc640 1 -- 172.21.15.172:0/2051547621 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f807c03dd00 con 0x7f808407ccf0 2024-09-10T08:37:23.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:23.064+0000 7f808ba5f640 1 -- 172.21.15.172:0/2051547621 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8070038fc0 msgr2=0x7f807003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:23.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:23.064+0000 7f808ba5f640 1 --2- 172.21.15.172:0/2051547621 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8070038fc0 0x7f807003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f80740098c0 tx=0x7f8074006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:23.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:23.064+0000 7f808ba5f640 1 -- 172.21.15.172:0/2051547621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f808407ccf0 msgr2=0x7f808419d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:23.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:23.064+0000 7f808ba5f640 1 --2- 172.21.15.172:0/2051547621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f808407ccf0 0x7f808419d5f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f807c031c90 tx=0x7f807c031cc0 comp rx=0 tx=0).stop 2024-09-10T08:37:23.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:23.064+0000 7f808ba5f640 1 -- 172.21.15.172:0/2051547621 shutdown_connections 2024-09-10T08:37:23.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:23.064+0000 7f808ba5f640 1 --2- 172.21.15.172:0/2051547621 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8070038fc0 0x7f807003b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:23.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:23.064+0000 7f808ba5f640 1 --2- 172.21.15.172:0/2051547621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f808407ccf0 0x7f808419d5f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:23.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:23.064+0000 7f808ba5f640 1 -- 172.21.15.172:0/2051547621 >> 172.21.15.172:0/2051547621 conn(0x7f8084102ca0 msgr2=0x7f80841037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:23.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:23.064+0000 7f808ba5f640 1 -- 172.21.15.172:0/2051547621 shutdown_connections 2024-09-10T08:37:23.645 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:23.064+0000 7f808ba5f640 1 -- 172.21.15.172:0/2051547621 wait complete. 2024-09-10T08:37:23.648 INFO:teuthology.orchestra.run.smithi172.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T08:37:23.648 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stdout disabled 2024-09-10T08:37:23.655 INFO:teuthology.orchestra.run.smithi172.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T08:37:23.655 INFO:teuthology.orchestra.run.smithi172.stdout:systemctl: stdout inactive 2024-09-10T08:37:23.655 INFO:teuthology.orchestra.run.smithi172.stdout:firewalld.service is not enabled 2024-09-10T08:37:23.655 INFO:teuthology.orchestra.run.smithi172.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T08:37:23.657 INFO:teuthology.orchestra.run.smithi172.stdout:Ceph Dashboard is now available at: 2024-09-10T08:37:23.657 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:37:23.657 INFO:teuthology.orchestra.run.smithi172.stdout: URL: https://smithi172.front.sepia.ceph.com:8443/ 2024-09-10T08:37:23.657 INFO:teuthology.orchestra.run.smithi172.stdout: User: admin 2024-09-10T08:37:23.657 INFO:teuthology.orchestra.run.smithi172.stdout: Password: 6fd2hor02d 2024-09-10T08:37:23.657 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:37:23.657 INFO:teuthology.orchestra.run.smithi172.stdout:Saving cluster configuration to /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config directory 2024-09-10T08:37:24.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:24 smithi172 ceph-mon[30854]: mgrmap e13: x(active, since 6s) 2024-09-10T08:37:24.831 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.049+0000 7f438290d640 1 Processor -- start 2024-09-10T08:37:24.831 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.049+0000 7f438290d640 1 -- start start 2024-09-10T08:37:24.831 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.049+0000 7f438290d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437c107f60 0x7f437c10a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:24.831 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.049+0000 7f438290d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f437c07bdf0 con 0x7f437c107f60 2024-09-10T08:37:24.831 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.049+0000 7f437bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437c107f60 0x7f437c10a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:24.831 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.049+0000 7f437bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437c107f60 0x7f437c10a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39076/0 (socket says 172.21.15.172:39076) 2024-09-10T08:37:24.832 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.049+0000 7f437bfff640 1 -- 172.21.15.172:0/4265898408 learned_addr learned my addr 172.21.15.172:0/4265898408 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:24.832 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.049+0000 7f437bfff640 1 -- 172.21.15.172:0/4265898408 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c10a890 con 0x7f437c107f60 2024-09-10T08:37:24.832 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.050+0000 7f437bfff640 1 --2- 172.21.15.172:0/4265898408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437c107f60 0x7f437c10a350 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f436c009820 tx=0x7f436c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea75063ba8339a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:24.832 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.050+0000 7f437affd640 1 -- 172.21.15.172:0/4265898408 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f436c037070 con 0x7f437c107f60 2024-09-10T08:37:24.832 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.050+0000 7f437affd640 1 -- 172.21.15.172:0/4265898408 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f436c0328f0 con 0x7f437c107f60 2024-09-10T08:37:24.832 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.050+0000 7f437affd640 1 -- 172.21.15.172:0/4265898408 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f436c030740 con 0x7f437c107f60 2024-09-10T08:37:24.832 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.051+0000 7f438290d640 1 -- 172.21.15.172:0/4265898408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437c107f60 msgr2=0x7f437c10a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:24.832 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.051+0000 7f438290d640 1 --2- 172.21.15.172:0/4265898408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437c107f60 0x7f437c10a350 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f436c009820 tx=0x7f436c02a270 comp rx=0 tx=0).stop 2024-09-10T08:37:24.832 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.051+0000 7f438290d640 1 -- 172.21.15.172:0/4265898408 shutdown_connections 2024-09-10T08:37:24.832 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.051+0000 7f438290d640 1 --2- 172.21.15.172:0/4265898408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437c107f60 0x7f437c10a350 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:24.832 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.051+0000 7f438290d640 1 -- 172.21.15.172:0/4265898408 >> 172.21.15.172:0/4265898408 conn(0x7f437c101be0 msgr2=0x7f437c104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:24.832 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.051+0000 7f438290d640 1 -- 172.21.15.172:0/4265898408 shutdown_connections 2024-09-10T08:37:24.833 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.051+0000 7f438290d640 1 -- 172.21.15.172:0/4265898408 wait complete. 2024-09-10T08:37:24.833 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.052+0000 7f438290d640 1 Processor -- start 2024-09-10T08:37:24.833 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.052+0000 7f438290d640 1 -- start start 2024-09-10T08:37:24.833 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.052+0000 7f438290d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437c107f60 0x7f437c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:24.833 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.052+0000 7f438290d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f437c10b8b0 con 0x7f437c107f60 2024-09-10T08:37:24.833 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.052+0000 7f437bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437c107f60 0x7f437c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:24.833 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.052+0000 7f437bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437c107f60 0x7f437c19b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39082/0 (socket says 172.21.15.172:39082) 2024-09-10T08:37:24.833 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.052+0000 7f437bfff640 1 -- 172.21.15.172:0/1538587792 learned_addr learned my addr 172.21.15.172:0/1538587792 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:24.833 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.052+0000 7f437bfff640 1 -- 172.21.15.172:0/1538587792 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c19b950 con 0x7f437c107f60 2024-09-10T08:37:24.833 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.052+0000 7f437bfff640 1 --2- 172.21.15.172:0/1538587792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437c107f60 0x7f437c19b410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f436c0097f0 tx=0x7f436c030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:24.833 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.053+0000 7f43797fa640 1 -- 172.21.15.172:0/1538587792 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f436c042070 con 0x7f437c107f60 2024-09-10T08:37:24.833 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.053+0000 7f43797fa640 1 -- 172.21.15.172:0/1538587792 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f436c030d80 con 0x7f437c107f60 2024-09-10T08:37:24.834 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.053+0000 7f438290d640 1 -- 172.21.15.172:0/1538587792 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437c19bbe0 con 0x7f437c107f60 2024-09-10T08:37:24.834 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.053+0000 7f43797fa640 1 -- 172.21.15.172:0/1538587792 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f436c037040 con 0x7f437c107f60 2024-09-10T08:37:24.834 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.053+0000 7f438290d640 1 -- 172.21.15.172:0/1538587792 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437c19e8d0 con 0x7f437c107f60 2024-09-10T08:37:24.834 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.053+0000 7f43797fa640 1 -- 172.21.15.172:0/1538587792 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f436c045070 con 0x7f437c107f60 2024-09-10T08:37:24.834 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.054+0000 7f438290d640 1 -- 172.21.15.172:0/1538587792 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f437c1083c0 con 0x7f437c107f60 2024-09-10T08:37:24.834 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.054+0000 7f43797fa640 1 --2- 172.21.15.172:0/1538587792 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f43640390c0 0x7f436403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:24.834 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.054+0000 7f43797fa640 1 -- 172.21.15.172:0/1538587792 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f436c06c1e0 con 0x7f437c107f60 2024-09-10T08:37:24.834 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.054+0000 7f437b7fe640 1 --2- 172.21.15.172:0/1538587792 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f43640390c0 0x7f436403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:24.834 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.054+0000 7f437b7fe640 1 --2- 172.21.15.172:0/1538587792 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f43640390c0 0x7f436403b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f43700098c0 tx=0x7f4370006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:24.834 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.059+0000 7f43797fa640 1 -- 172.21.15.172:0/1538587792 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f436c045360 con 0x7f437c107f60 2024-09-10T08:37:24.834 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.313+0000 7f438290d640 1 -- 172.21.15.172:0/1538587792 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f437c19eaa0 con 0x7f437c107f60 2024-09-10T08:37:24.834 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.316+0000 7f43797fa640 1 -- 172.21.15.172:0/1538587792 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) ==== 153+0+0 (secure 0 0 0) 0x7f436c03ada0 con 0x7f437c107f60 2024-09-10T08:37:24.835 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T08:37:24.835 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.318+0000 7f438290d640 1 -- 172.21.15.172:0/1538587792 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f43640390c0 msgr2=0x7f436403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:24.835 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.318+0000 7f438290d640 1 --2- 172.21.15.172:0/1538587792 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f43640390c0 0x7f436403b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f43700098c0 tx=0x7f4370006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:24.835 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.318+0000 7f438290d640 1 -- 172.21.15.172:0/1538587792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437c107f60 msgr2=0x7f437c19b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:24.835 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.318+0000 7f438290d640 1 --2- 172.21.15.172:0/1538587792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437c107f60 0x7f437c19b410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f436c0097f0 tx=0x7f436c030c70 comp rx=0 tx=0).stop 2024-09-10T08:37:24.835 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.318+0000 7f438290d640 1 -- 172.21.15.172:0/1538587792 shutdown_connections 2024-09-10T08:37:24.835 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.318+0000 7f438290d640 1 --2- 172.21.15.172:0/1538587792 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f43640390c0 0x7f436403b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:24.835 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.318+0000 7f438290d640 1 --2- 172.21.15.172:0/1538587792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f437c107f60 0x7f437c19b410 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:24.835 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.318+0000 7f438290d640 1 -- 172.21.15.172:0/1538587792 >> 172.21.15.172:0/1538587792 conn(0x7f437c101be0 msgr2=0x7f437c102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:24.835 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.318+0000 7f438290d640 1 -- 172.21.15.172:0/1538587792 shutdown_connections 2024-09-10T08:37:24.835 INFO:teuthology.orchestra.run.smithi172.stdout:/usr/bin/ceph: stderr 2024-09-10T08:37:24.318+0000 7f438290d640 1 -- 172.21.15.172:0/1538587792 wait complete. 2024-09-10T08:37:24.835 INFO:teuthology.orchestra.run.smithi172.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T08:37:24.835 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:37:24.836 INFO:teuthology.orchestra.run.smithi172.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T08:37:24.836 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:37:24.836 INFO:teuthology.orchestra.run.smithi172.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T08:37:24.836 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:37:24.836 INFO:teuthology.orchestra.run.smithi172.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T08:37:24.836 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:37:24.836 INFO:teuthology.orchestra.run.smithi172.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T08:37:24.836 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:37:24.836 INFO:teuthology.orchestra.run.smithi172.stdout: ceph telemetry on 2024-09-10T08:37:24.836 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:37:24.836 INFO:teuthology.orchestra.run.smithi172.stdout:For more information see: 2024-09-10T08:37:24.836 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:37:24.836 INFO:teuthology.orchestra.run.smithi172.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T08:37:24.837 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:37:24.837 INFO:teuthology.orchestra.run.smithi172.stdout:Bootstrap complete. 2024-09-10T08:37:24.870 INFO:tasks.cephadm:Fetching config... 2024-09-10T08:37:24.871 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:37:24.871 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T08:37:24.886 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T08:37:24.887 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:37:24.887 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T08:37:24.942 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T08:37:24.943 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:37:24.943 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T08:37:25.008 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T08:37:25.008 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:37:25.008 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T08:37:25.064 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T08:37:25.064 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDc6HsuEry07e68/ERoDsCX5EGlmKGRgv+iCYVILK8SfVLdG+HWaIyN5NyBWFHPG4KJdz/6ixrfbScmvXABtFwEkxjqQEhPXnkfHB3wDXEdEg22fEKRO/AutzLaDG0wty30ZQ6rabKhPnwa8Pl0QZ1uOkAyxbn2NVNgeyr+AuZJRm4sCe2i9naVVWcNUb94P8M9IhyxMhiyyVmc6Er+cEA6eM8IEx9I7ysndT8n7YjGusvG2CzxPCNTvzZkWdHiGyVifAu0Q6RiFlTV77zxAgxNNN+c+TqwL0jxgwpOsaPC/iRGJmYTM3I/6ejBR5q4sGpSjEhGsBmF+FlrRRTeGxX3SPb6xCB6kcKmk0dz/zLoq9vVY1I0R78P9XwFM4qd7eExgkcmoV5Cteg1mYgIfn8WHY4Gfk5TuNDXGSevCGzgMGTRcAit+iY87M3bkPeUhZ6FXvoZxcPYNHltg1mlVVqL+icW80C7NlgYDPYaBT8tSrW6eQGPrCMOQqBWMllICe0= ceph-a7dba522-6f4f-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T08:37:25.139 INFO:teuthology.orchestra.run.smithi172.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDc6HsuEry07e68/ERoDsCX5EGlmKGRgv+iCYVILK8SfVLdG+HWaIyN5NyBWFHPG4KJdz/6ixrfbScmvXABtFwEkxjqQEhPXnkfHB3wDXEdEg22fEKRO/AutzLaDG0wty30ZQ6rabKhPnwa8Pl0QZ1uOkAyxbn2NVNgeyr+AuZJRm4sCe2i9naVVWcNUb94P8M9IhyxMhiyyVmc6Er+cEA6eM8IEx9I7ysndT8n7YjGusvG2CzxPCNTvzZkWdHiGyVifAu0Q6RiFlTV77zxAgxNNN+c+TqwL0jxgwpOsaPC/iRGJmYTM3I/6ejBR5q4sGpSjEhGsBmF+FlrRRTeGxX3SPb6xCB6kcKmk0dz/zLoq9vVY1I0R78P9XwFM4qd7eExgkcmoV5Cteg1mYgIfn8WHY4Gfk5TuNDXGSevCGzgMGTRcAit+iY87M3bkPeUhZ6FXvoZxcPYNHltg1mlVVqL+icW80C7NlgYDPYaBT8tSrW6eQGPrCMOQqBWMllICe0= ceph-a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:37:25.157 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDc6HsuEry07e68/ERoDsCX5EGlmKGRgv+iCYVILK8SfVLdG+HWaIyN5NyBWFHPG4KJdz/6ixrfbScmvXABtFwEkxjqQEhPXnkfHB3wDXEdEg22fEKRO/AutzLaDG0wty30ZQ6rabKhPnwa8Pl0QZ1uOkAyxbn2NVNgeyr+AuZJRm4sCe2i9naVVWcNUb94P8M9IhyxMhiyyVmc6Er+cEA6eM8IEx9I7ysndT8n7YjGusvG2CzxPCNTvzZkWdHiGyVifAu0Q6RiFlTV77zxAgxNNN+c+TqwL0jxgwpOsaPC/iRGJmYTM3I/6ejBR5q4sGpSjEhGsBmF+FlrRRTeGxX3SPb6xCB6kcKmk0dz/zLoq9vVY1I0R78P9XwFM4qd7eExgkcmoV5Cteg1mYgIfn8WHY4Gfk5TuNDXGSevCGzgMGTRcAit+iY87M3bkPeUhZ6FXvoZxcPYNHltg1mlVVqL+icW80C7NlgYDPYaBT8tSrW6eQGPrCMOQqBWMllICe0= ceph-a7dba522-6f4f-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T08:37:25.207 INFO:teuthology.orchestra.run.smithi174.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDc6HsuEry07e68/ERoDsCX5EGlmKGRgv+iCYVILK8SfVLdG+HWaIyN5NyBWFHPG4KJdz/6ixrfbScmvXABtFwEkxjqQEhPXnkfHB3wDXEdEg22fEKRO/AutzLaDG0wty30ZQ6rabKhPnwa8Pl0QZ1uOkAyxbn2NVNgeyr+AuZJRm4sCe2i9naVVWcNUb94P8M9IhyxMhiyyVmc6Er+cEA6eM8IEx9I7ysndT8n7YjGusvG2CzxPCNTvzZkWdHiGyVifAu0Q6RiFlTV77zxAgxNNN+c+TqwL0jxgwpOsaPC/iRGJmYTM3I/6ejBR5q4sGpSjEhGsBmF+FlrRRTeGxX3SPb6xCB6kcKmk0dz/zLoq9vVY1I0R78P9XwFM4qd7eExgkcmoV5Cteg1mYgIfn8WHY4Gfk5TuNDXGSevCGzgMGTRcAit+iY87M3bkPeUhZ6FXvoZxcPYNHltg1mlVVqL+icW80C7NlgYDPYaBT8tSrW6eQGPrCMOQqBWMllICe0= ceph-a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:37:25.219 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDc6HsuEry07e68/ERoDsCX5EGlmKGRgv+iCYVILK8SfVLdG+HWaIyN5NyBWFHPG4KJdz/6ixrfbScmvXABtFwEkxjqQEhPXnkfHB3wDXEdEg22fEKRO/AutzLaDG0wty30ZQ6rabKhPnwa8Pl0QZ1uOkAyxbn2NVNgeyr+AuZJRm4sCe2i9naVVWcNUb94P8M9IhyxMhiyyVmc6Er+cEA6eM8IEx9I7ysndT8n7YjGusvG2CzxPCNTvzZkWdHiGyVifAu0Q6RiFlTV77zxAgxNNN+c+TqwL0jxgwpOsaPC/iRGJmYTM3I/6ejBR5q4sGpSjEhGsBmF+FlrRRTeGxX3SPb6xCB6kcKmk0dz/zLoq9vVY1I0R78P9XwFM4qd7eExgkcmoV5Cteg1mYgIfn8WHY4Gfk5TuNDXGSevCGzgMGTRcAit+iY87M3bkPeUhZ6FXvoZxcPYNHltg1mlVVqL+icW80C7NlgYDPYaBT8tSrW6eQGPrCMOQqBWMllICe0= ceph-a7dba522-6f4f-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T08:37:25.285 INFO:teuthology.orchestra.run.smithi191.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDc6HsuEry07e68/ERoDsCX5EGlmKGRgv+iCYVILK8SfVLdG+HWaIyN5NyBWFHPG4KJdz/6ixrfbScmvXABtFwEkxjqQEhPXnkfHB3wDXEdEg22fEKRO/AutzLaDG0wty30ZQ6rabKhPnwa8Pl0QZ1uOkAyxbn2NVNgeyr+AuZJRm4sCe2i9naVVWcNUb94P8M9IhyxMhiyyVmc6Er+cEA6eM8IEx9I7ysndT8n7YjGusvG2CzxPCNTvzZkWdHiGyVifAu0Q6RiFlTV77zxAgxNNN+c+TqwL0jxgwpOsaPC/iRGJmYTM3I/6ejBR5q4sGpSjEhGsBmF+FlrRRTeGxX3SPb6xCB6kcKmk0dz/zLoq9vVY1I0R78P9XwFM4qd7eExgkcmoV5Cteg1mYgIfn8WHY4Gfk5TuNDXGSevCGzgMGTRcAit+iY87M3bkPeUhZ6FXvoZxcPYNHltg1mlVVqL+icW80C7NlgYDPYaBT8tSrW6eQGPrCMOQqBWMllICe0= ceph-a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:37:25.298 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T08:37:25.546 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:25.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:25 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1538587792' entity='client.admin' 2024-09-10T08:37:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.038+0000 7fefe6762640 1 -- 172.21.15.172:0/2530622359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe0101640 msgr2=0x7fefe0101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.038+0000 7fefe6762640 1 --2- 172.21.15.172:0/2530622359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe0101640 0x7fefe0101a40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009880 tx=0x7fefd002a5d0 comp rx=0 tx=0).stop 2024-09-10T08:37:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.038+0000 7fefe6762640 1 -- 172.21.15.172:0/2530622359 shutdown_connections 2024-09-10T08:37:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.038+0000 7fefe6762640 1 --2- 172.21.15.172:0/2530622359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe0101640 0x7fefe0101a40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.038+0000 7fefe6762640 1 -- 172.21.15.172:0/2530622359 >> 172.21.15.172:0/2530622359 conn(0x7fefe00fcdf0 msgr2=0x7fefe00ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.038+0000 7fefe6762640 1 -- 172.21.15.172:0/2530622359 shutdown_connections 2024-09-10T08:37:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.038+0000 7fefe6762640 1 -- 172.21.15.172:0/2530622359 wait complete. 2024-09-10T08:37:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.039+0000 7fefe6762640 1 Processor -- start 2024-09-10T08:37:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.039+0000 7fefe6762640 1 -- start start 2024-09-10T08:37:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.039+0000 7fefe6762640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe0101640 0x7fefe0196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.039+0000 7fefe6762640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0107600 con 0x7fefe0101640 2024-09-10T08:37:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.039+0000 7fefdffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe0101640 0x7fefe0196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.039+0000 7fefdffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe0101640 0x7fefe0196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39098/0 (socket says 172.21.15.172:39098) 2024-09-10T08:37:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.039+0000 7fefdffff640 1 -- 172.21.15.172:0/1319610448 learned_addr learned my addr 172.21.15.172:0/1319610448 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.040+0000 7fefdffff640 1 -- 172.21.15.172:0/1319610448 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe0197270 con 0x7fefe0101640 2024-09-10T08:37:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.040+0000 7fefdffff640 1 --2- 172.21.15.172:0/1319610448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe0101640 0x7fefe0196d30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009230 tx=0x7fefd00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.040+0000 7fefdd7fa640 1 -- 172.21.15.172:0/1319610448 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefd003f070 con 0x7fefe0101640 2024-09-10T08:37:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.040+0000 7fefdd7fa640 1 -- 172.21.15.172:0/1319610448 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefd0003aa0 con 0x7fefe0101640 2024-09-10T08:37:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.040+0000 7fefdd7fa640 1 -- 172.21.15.172:0/1319610448 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefd0035030 con 0x7fefe0101640 2024-09-10T08:37:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.040+0000 7fefe6762640 1 -- 172.21.15.172:0/1319610448 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe0197500 con 0x7fefe0101640 2024-09-10T08:37:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.040+0000 7fefe6762640 1 -- 172.21.15.172:0/1319610448 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe019a1f0 con 0x7fefe0101640 2024-09-10T08:37:26.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.041+0000 7fefdd7fa640 1 -- 172.21.15.172:0/1319610448 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fefd0042070 con 0x7fefe0101640 2024-09-10T08:37:26.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.041+0000 7fefdd7fa640 1 --2- 172.21.15.172:0/1319610448 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fefc8039060 0x7fefc803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:26.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.041+0000 7fefdd7fa640 1 -- 172.21.15.172:0/1319610448 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fefd006cdf0 con 0x7fefe0101640 2024-09-10T08:37:26.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.041+0000 7fefdf7fe640 1 --2- 172.21.15.172:0/1319610448 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fefc8039060 0x7fefc803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:26.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.041+0000 7fefe6762640 1 -- 172.21.15.172:0/1319610448 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe0107d80 con 0x7fefe0101640 2024-09-10T08:37:26.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.042+0000 7fefdf7fe640 1 --2- 172.21.15.172:0/1319610448 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fefc8039060 0x7fefc803b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fefd40098c0 tx=0x7fefd4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:26.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.046+0000 7fefdd7fa640 1 -- 172.21.15.172:0/1319610448 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fefd0039350 con 0x7fefe0101640 2024-09-10T08:37:26.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.215+0000 7fefe6762640 1 -- 172.21.15.172:0/1319610448 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fefe018fe30 con 0x7fefe0101640 2024-09-10T08:37:26.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.218+0000 7fefdd7fa640 1 -- 172.21.15.172:0/1319610448 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fefd002f300 con 0x7fefe0101640 2024-09-10T08:37:26.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.220+0000 7fefe6762640 1 -- 172.21.15.172:0/1319610448 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fefc8039060 msgr2=0x7fefc803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:26.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.220+0000 7fefe6762640 1 --2- 172.21.15.172:0/1319610448 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fefc8039060 0x7fefc803b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fefd40098c0 tx=0x7fefd4006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:26.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.220+0000 7fefe6762640 1 -- 172.21.15.172:0/1319610448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe0101640 msgr2=0x7fefe0196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:26.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.220+0000 7fefe6762640 1 --2- 172.21.15.172:0/1319610448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe0101640 0x7fefe0196d30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009230 tx=0x7fefd00048f0 comp rx=0 tx=0).stop 2024-09-10T08:37:26.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.220+0000 7fefe6762640 1 -- 172.21.15.172:0/1319610448 shutdown_connections 2024-09-10T08:37:26.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.220+0000 7fefe6762640 1 --2- 172.21.15.172:0/1319610448 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fefc8039060 0x7fefc803b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:26.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.220+0000 7fefe6762640 1 --2- 172.21.15.172:0/1319610448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefe0101640 0x7fefe0196d30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:26.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.220+0000 7fefe6762640 1 -- 172.21.15.172:0/1319610448 >> 172.21.15.172:0/1319610448 conn(0x7fefe00fcdf0 msgr2=0x7fefe00fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:26.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.220+0000 7fefe6762640 1 -- 172.21.15.172:0/1319610448 shutdown_connections 2024-09-10T08:37:26.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:26.220+0000 7fefe6762640 1 -- 172.21.15.172:0/1319610448 wait complete. 2024-09-10T08:37:26.832 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T08:37:26.832 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T08:37:27.332 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:27.515 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:27 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1319610448' entity='client.admin' 2024-09-10T08:37:27.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.753+0000 7f80c56d8640 1 -- 172.21.15.172:0/304814175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80c0101640 msgr2=0x7f80c0101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:27.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.753+0000 7f80c56d8640 1 --2- 172.21.15.172:0/304814175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80c0101640 0x7f80c0101a40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009c20 tx=0x7f80b002aa90 comp rx=0 tx=0).stop 2024-09-10T08:37:27.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.753+0000 7f80c56d8640 1 -- 172.21.15.172:0/304814175 shutdown_connections 2024-09-10T08:37:27.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.753+0000 7f80c56d8640 1 --2- 172.21.15.172:0/304814175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80c0101640 0x7f80c0101a40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:27.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.753+0000 7f80c56d8640 1 -- 172.21.15.172:0/304814175 >> 172.21.15.172:0/304814175 conn(0x7f80c00fcdf0 msgr2=0x7f80c00ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:27.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.753+0000 7f80c56d8640 1 -- 172.21.15.172:0/304814175 shutdown_connections 2024-09-10T08:37:27.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.753+0000 7f80c56d8640 1 -- 172.21.15.172:0/304814175 wait complete. 2024-09-10T08:37:27.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.753+0000 7f80c56d8640 1 Processor -- start 2024-09-10T08:37:27.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.753+0000 7f80c56d8640 1 -- start start 2024-09-10T08:37:27.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.754+0000 7f80c56d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80c0101640 0x7f80c0075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:27.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.754+0000 7f80c56d8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f80c0107600 con 0x7f80c0101640 2024-09-10T08:37:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.754+0000 7f80beffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80c0101640 0x7f80c0075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.754+0000 7f80beffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80c0101640 0x7f80c0075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38790/0 (socket says 172.21.15.172:38790) 2024-09-10T08:37:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.754+0000 7f80beffd640 1 -- 172.21.15.172:0/2908627069 learned_addr learned my addr 172.21.15.172:0/2908627069 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.754+0000 7f80beffd640 1 -- 172.21.15.172:0/2908627069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c00763f0 con 0x7f80c0101640 2024-09-10T08:37:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.754+0000 7f80beffd640 1 --2- 172.21.15.172:0/2908627069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80c0101640 0x7f80c0075eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f80b0006010 tx=0x7f80b0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.754+0000 7f80b7fff640 1 -- 172.21.15.172:0/2908627069 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80b0047070 con 0x7f80c0101640 2024-09-10T08:37:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.754+0000 7f80b7fff640 1 -- 172.21.15.172:0/2908627069 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80b0003e20 con 0x7f80c0101640 2024-09-10T08:37:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.754+0000 7f80c56d8640 1 -- 172.21.15.172:0/2908627069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c00745e0 con 0x7f80c0101640 2024-09-10T08:37:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.754+0000 7f80b7fff640 1 -- 172.21.15.172:0/2908627069 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80b0033050 con 0x7f80c0101640 2024-09-10T08:37:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.754+0000 7f80c56d8640 1 -- 172.21.15.172:0/2908627069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c0074b20 con 0x7f80c0101640 2024-09-10T08:37:27.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.755+0000 7f80b7fff640 1 -- 172.21.15.172:0/2908627069 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f80b004a070 con 0x7f80c0101640 2024-09-10T08:37:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.755+0000 7f80b7fff640 1 --2- 172.21.15.172:0/2908627069 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80ac0390c0 0x7f80ac03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.755+0000 7f80c56d8640 1 -- 172.21.15.172:0/2908627069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c0107d80 con 0x7f80c0101640 2024-09-10T08:37:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.756+0000 7f80b7fff640 1 -- 172.21.15.172:0/2908627069 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f80b003e030 con 0x7f80c0101640 2024-09-10T08:37:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.756+0000 7f80be7fc640 1 --2- 172.21.15.172:0/2908627069 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80ac0390c0 0x7f80ac03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:27.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.756+0000 7f80be7fc640 1 --2- 172.21.15.172:0/2908627069 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80ac0390c0 0x7f80ac03b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f80a80098c0 tx=0x7f80a8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:27.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.760+0000 7f80b7fff640 1 -- 172.21.15.172:0/2908627069 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f80b0038cf0 con 0x7f80c0101640 2024-09-10T08:37:27.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.932+0000 7f80c56d8640 1 -- 172.21.15.172:0/2908627069 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f80c01900c0 con 0x7f80ac0390c0 2024-09-10T08:37:27.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.935+0000 7f80b7fff640 1 -- 172.21.15.172:0/2908627069 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f80c01900c0 con 0x7f80ac0390c0 2024-09-10T08:37:27.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.938+0000 7f80c56d8640 1 -- 172.21.15.172:0/2908627069 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80ac0390c0 msgr2=0x7f80ac03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:27.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.938+0000 7f80c56d8640 1 --2- 172.21.15.172:0/2908627069 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80ac0390c0 0x7f80ac03b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f80a80098c0 tx=0x7f80a8006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:27.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.938+0000 7f80c56d8640 1 -- 172.21.15.172:0/2908627069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80c0101640 msgr2=0x7f80c0075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:27.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.938+0000 7f80c56d8640 1 --2- 172.21.15.172:0/2908627069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80c0101640 0x7f80c0075eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f80b0006010 tx=0x7f80b0030ec0 comp rx=0 tx=0).stop 2024-09-10T08:37:27.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.938+0000 7f80c56d8640 1 -- 172.21.15.172:0/2908627069 shutdown_connections 2024-09-10T08:37:27.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.938+0000 7f80c56d8640 1 --2- 172.21.15.172:0/2908627069 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80ac0390c0 0x7f80ac03b580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:27.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.938+0000 7f80c56d8640 1 --2- 172.21.15.172:0/2908627069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80c0101640 0x7f80c0075eb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:27.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.938+0000 7f80c56d8640 1 -- 172.21.15.172:0/2908627069 >> 172.21.15.172:0/2908627069 conn(0x7f80c00fcdf0 msgr2=0x7f80c00fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:27.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.938+0000 7f80c56d8640 1 -- 172.21.15.172:0/2908627069 shutdown_connections 2024-09-10T08:37:27.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:27.938+0000 7f80c56d8640 1 -- 172.21.15.172:0/2908627069 wait complete. 2024-09-10T08:37:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi172", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:37:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:37:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:28 smithi172 ceph-mon[30854]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:37:28.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:37:28.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:37:28.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:28 smithi172 ceph-mon[30854]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-10T08:37:28.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:28 smithi172 ceph-mon[30854]: Updating smithi172:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:37:28.694 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi174 2024-09-10T08:37:28.694 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T08:37:28.694 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.conf 2024-09-10T08:37:28.715 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T08:37:28.715 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:37:28.773 INFO:tasks.cephadm:Adding host smithi174 to orchestrator... 2024-09-10T08:37:28.773 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch host add smithi174 2024-09-10T08:37:29.016 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:29.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.451+0000 7fb08a61f640 1 -- 172.21.15.172:0/1564760170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb084102780 msgr2=0x7fb084102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:29.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.451+0000 7fb08a61f640 1 --2- 172.21.15.172:0/1564760170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb084102780 0x7fb084102b60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb074009880 tx=0x7fb07402a670 comp rx=0 tx=0).stop 2024-09-10T08:37:29.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.451+0000 7fb08a61f640 1 -- 172.21.15.172:0/1564760170 shutdown_connections 2024-09-10T08:37:29.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.451+0000 7fb08a61f640 1 --2- 172.21.15.172:0/1564760170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb084102780 0x7fb084102b60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:29.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.451+0000 7fb08a61f640 1 -- 172.21.15.172:0/1564760170 >> 172.21.15.172:0/1564760170 conn(0x7fb0840fc1a0 msgr2=0x7fb0840fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:29.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.451+0000 7fb08a61f640 1 -- 172.21.15.172:0/1564760170 shutdown_connections 2024-09-10T08:37:29.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.451+0000 7fb08a61f640 1 -- 172.21.15.172:0/1564760170 wait complete. 2024-09-10T08:37:29.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.452+0000 7fb08a61f640 1 Processor -- start 2024-09-10T08:37:29.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.452+0000 7fb08a61f640 1 -- start start 2024-09-10T08:37:29.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.452+0000 7fb08a61f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb084102780 0x7fb084197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:29.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.452+0000 7fb08a61f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb08410bba0 con 0x7fb084102780 2024-09-10T08:37:29.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.452+0000 7fb083fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb084102780 0x7fb084197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:29.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.452+0000 7fb083fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb084102780 0x7fb084197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38802/0 (socket says 172.21.15.172:38802) 2024-09-10T08:37:29.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.452+0000 7fb083fff640 1 -- 172.21.15.172:0/2312249629 learned_addr learned my addr 172.21.15.172:0/2312249629 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:29.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.453+0000 7fb083fff640 1 -- 172.21.15.172:0/2312249629 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb084197690 con 0x7fb084102780 2024-09-10T08:37:29.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.453+0000 7fb083fff640 1 --2- 172.21.15.172:0/2312249629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb084102780 0x7fb084197150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb074009230 tx=0x7fb0740048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:29.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.453+0000 7fb081ffb640 1 -- 172.21.15.172:0/2312249629 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb07403f070 con 0x7fb084102780 2024-09-10T08:37:29.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.453+0000 7fb081ffb640 1 -- 172.21.15.172:0/2312249629 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb074004d10 con 0x7fb084102780 2024-09-10T08:37:29.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.453+0000 7fb081ffb640 1 -- 172.21.15.172:0/2312249629 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb074035030 con 0x7fb084102780 2024-09-10T08:37:29.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.453+0000 7fb08a61f640 1 -- 172.21.15.172:0/2312249629 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb084197920 con 0x7fb084102780 2024-09-10T08:37:29.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.453+0000 7fb08a61f640 1 -- 172.21.15.172:0/2312249629 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb084197d80 con 0x7fb084102780 2024-09-10T08:37:29.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.454+0000 7fb081ffb640 1 -- 172.21.15.172:0/2312249629 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fb074042070 con 0x7fb084102780 2024-09-10T08:37:29.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.454+0000 7fb08a61f640 1 -- 172.21.15.172:0/2312249629 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb08406a7c0 con 0x7fb084102780 2024-09-10T08:37:29.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.454+0000 7fb081ffb640 1 --2- 172.21.15.172:0/2312249629 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb070039060 0x7fb07003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:29.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.454+0000 7fb081ffb640 1 -- 172.21.15.172:0/2312249629 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb07406ce20 con 0x7fb084102780 2024-09-10T08:37:29.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.454+0000 7fb07bfff640 1 --2- 172.21.15.172:0/2312249629 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb070039060 0x7fb07003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:29.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.455+0000 7fb07bfff640 1 --2- 172.21.15.172:0/2312249629 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb070039060 0x7fb07003b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb06c0098c0 tx=0x7fb06c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:29.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.459+0000 7fb081ffb640 1 -- 172.21.15.172:0/2312249629 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0740303a0 con 0x7fb084102780 2024-09-10T08:37:29.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:29.630+0000 7fb08a61f640 1 -- 172.21.15.172:0/2312249629 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}) -- 0x7fb0841984b0 con 0x7fb070039060 2024-09-10T08:37:30.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:29 smithi172 ceph-mon[30854]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:37:30.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:29 smithi172 ceph-mon[30854]: Updating smithi172:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T08:37:30.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:29 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:30.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:29 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:30.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:29 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:31.081 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:30 smithi172 ceph-mon[30854]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:32.081 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:31 smithi172 ceph-mon[30854]: Deploying cephadm binary to smithi174 2024-09-10T08:37:32.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:32.875+0000 7fb081ffb640 1 -- 172.21.15.172:0/2312249629 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb0841984b0 con 0x7fb070039060 2024-09-10T08:37:32.876 INFO:teuthology.orchestra.run.smithi172.stdout:Added host 'smithi174' with addr '172.21.15.174' 2024-09-10T08:37:32.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:32.878+0000 7fb08a61f640 1 -- 172.21.15.172:0/2312249629 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb070039060 msgr2=0x7fb07003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:32.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:32.878+0000 7fb08a61f640 1 --2- 172.21.15.172:0/2312249629 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb070039060 0x7fb07003b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb06c0098c0 tx=0x7fb06c006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:32.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:32.878+0000 7fb08a61f640 1 -- 172.21.15.172:0/2312249629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb084102780 msgr2=0x7fb084197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:32.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:32.878+0000 7fb08a61f640 1 --2- 172.21.15.172:0/2312249629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb084102780 0x7fb084197150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb074009230 tx=0x7fb0740048f0 comp rx=0 tx=0).stop 2024-09-10T08:37:32.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:32.878+0000 7fb08a61f640 1 -- 172.21.15.172:0/2312249629 shutdown_connections 2024-09-10T08:37:32.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:32.878+0000 7fb08a61f640 1 --2- 172.21.15.172:0/2312249629 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb070039060 0x7fb07003b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:32.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:32.878+0000 7fb08a61f640 1 --2- 172.21.15.172:0/2312249629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb084102780 0x7fb084197150 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:32.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:32.878+0000 7fb08a61f640 1 -- 172.21.15.172:0/2312249629 >> 172.21.15.172:0/2312249629 conn(0x7fb0840fc1a0 msgr2=0x7fb084105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:32.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:32.878+0000 7fb08a61f640 1 -- 172.21.15.172:0/2312249629 shutdown_connections 2024-09-10T08:37:32.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:32.878+0000 7fb08a61f640 1 -- 172.21.15.172:0/2312249629 wait complete. 2024-09-10T08:37:33.439 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T08:37:33.686 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:34.060 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:33 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:34.061 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:33 smithi172 ceph-mon[30854]: Added host smithi174 2024-09-10T08:37:34.061 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:33 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:37:34.061 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:33 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:34.061 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:33 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:34.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.140+0000 7faccdfc5640 1 -- 172.21.15.172:0/2118808367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc8106590 msgr2=0x7facc8106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:34.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.140+0000 7faccdfc5640 1 --2- 172.21.15.172:0/2118808367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc8106590 0x7facc8106970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7facb8009880 tx=0x7facb802a5d0 comp rx=0 tx=0).stop 2024-09-10T08:37:34.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.140+0000 7faccdfc5640 1 -- 172.21.15.172:0/2118808367 shutdown_connections 2024-09-10T08:37:34.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.140+0000 7faccdfc5640 1 --2- 172.21.15.172:0/2118808367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc8106590 0x7facc8106970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:34.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.140+0000 7faccdfc5640 1 -- 172.21.15.172:0/2118808367 >> 172.21.15.172:0/2118808367 conn(0x7facc80fc1a0 msgr2=0x7facc80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:34.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.140+0000 7faccdfc5640 1 -- 172.21.15.172:0/2118808367 shutdown_connections 2024-09-10T08:37:34.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.140+0000 7faccdfc5640 1 -- 172.21.15.172:0/2118808367 wait complete. 2024-09-10T08:37:34.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.141+0000 7faccdfc5640 1 Processor -- start 2024-09-10T08:37:34.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.141+0000 7faccdfc5640 1 -- start start 2024-09-10T08:37:34.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.141+0000 7faccdfc5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc8106590 0x7facc8197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:34.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.141+0000 7faccdfc5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7facc810aa70 con 0x7facc8106590 2024-09-10T08:37:34.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.141+0000 7facc77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc8106590 0x7facc8197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:34.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.141+0000 7facc77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc8106590 0x7facc8197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38816/0 (socket says 172.21.15.172:38816) 2024-09-10T08:37:34.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.141+0000 7facc77fe640 1 -- 172.21.15.172:0/646115585 learned_addr learned my addr 172.21.15.172:0/646115585 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:34.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.142+0000 7facc77fe640 1 -- 172.21.15.172:0/646115585 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc8198390 con 0x7facc8106590 2024-09-10T08:37:34.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.142+0000 7facc77fe640 1 --2- 172.21.15.172:0/646115585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc8106590 0x7facc8197e50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7facb8009230 tx=0x7facb80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:34.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.142+0000 7facc4ff9640 1 -- 172.21.15.172:0/646115585 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facb803f070 con 0x7facc8106590 2024-09-10T08:37:34.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.142+0000 7facc4ff9640 1 -- 172.21.15.172:0/646115585 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facb8004d10 con 0x7facc8106590 2024-09-10T08:37:34.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.142+0000 7faccdfc5640 1 -- 172.21.15.172:0/646115585 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc8198620 con 0x7facc8106590 2024-09-10T08:37:34.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.142+0000 7faccdfc5640 1 -- 172.21.15.172:0/646115585 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc8198a80 con 0x7facc8106590 2024-09-10T08:37:34.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.142+0000 7facc4ff9640 1 -- 172.21.15.172:0/646115585 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facb8035030 con 0x7facc8106590 2024-09-10T08:37:34.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.143+0000 7facc4ff9640 1 -- 172.21.15.172:0/646115585 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7facb8042070 con 0x7facc8106590 2024-09-10T08:37:34.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.143+0000 7faccdfc5640 1 -- 172.21.15.172:0/646115585 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc8101c00 con 0x7facc8106590 2024-09-10T08:37:34.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.143+0000 7facc4ff9640 1 --2- 172.21.15.172:0/646115585 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7facb0039060 0x7facb003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:34.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.143+0000 7facc4ff9640 1 -- 172.21.15.172:0/646115585 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7facb806ccf0 con 0x7facc8106590 2024-09-10T08:37:34.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.143+0000 7facc6ffd640 1 --2- 172.21.15.172:0/646115585 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7facb0039060 0x7facb003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:34.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.144+0000 7facc6ffd640 1 --2- 172.21.15.172:0/646115585 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7facb0039060 0x7facb003b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7facbc0098c0 tx=0x7facbc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:34.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.148+0000 7facc4ff9640 1 -- 172.21.15.172:0/646115585 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7facb8071070 con 0x7facc8106590 2024-09-10T08:37:34.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.320+0000 7faccdfc5640 1 -- 172.21.15.172:0/646115585 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7facc81069f0 con 0x7facb0039060 2024-09-10T08:37:34.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.322+0000 7facc4ff9640 1 -- 172.21.15.172:0/646115585 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7facc81069f0 con 0x7facb0039060 2024-09-10T08:37:34.322 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:37:34.322 INFO:teuthology.orchestra.run.smithi172.stdout:[{"addr": "172.21.15.172", "hostname": "smithi172", "labels": [], "status": ""}, {"addr": "172.21.15.174", "hostname": "smithi174", "labels": [], "status": ""}] 2024-09-10T08:37:34.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.324+0000 7faccdfc5640 1 -- 172.21.15.172:0/646115585 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7facb0039060 msgr2=0x7facb003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:34.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.324+0000 7faccdfc5640 1 --2- 172.21.15.172:0/646115585 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7facb0039060 0x7facb003b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7facbc0098c0 tx=0x7facbc006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:34.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.324+0000 7faccdfc5640 1 -- 172.21.15.172:0/646115585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc8106590 msgr2=0x7facc8197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:34.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.324+0000 7faccdfc5640 1 --2- 172.21.15.172:0/646115585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc8106590 0x7facc8197e50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7facb8009230 tx=0x7facb80048f0 comp rx=0 tx=0).stop 2024-09-10T08:37:34.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.324+0000 7faccdfc5640 1 -- 172.21.15.172:0/646115585 shutdown_connections 2024-09-10T08:37:34.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.324+0000 7faccdfc5640 1 --2- 172.21.15.172:0/646115585 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7facb0039060 0x7facb003b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:34.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.324+0000 7faccdfc5640 1 --2- 172.21.15.172:0/646115585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc8106590 0x7facc8197e50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:34.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.324+0000 7faccdfc5640 1 -- 172.21.15.172:0/646115585 >> 172.21.15.172:0/646115585 conn(0x7facc80fc1a0 msgr2=0x7facc80fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:34.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.324+0000 7faccdfc5640 1 -- 172.21.15.172:0/646115585 shutdown_connections 2024-09-10T08:37:34.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:34.324+0000 7faccdfc5640 1 -- 172.21.15.172:0/646115585 wait complete. 2024-09-10T08:37:34.915 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi191 2024-09-10T08:37:34.916 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-10T08:37:34.916 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/etc/ceph/ceph.conf 2024-09-10T08:37:34.935 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-10T08:37:34.935 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:37:34.993 INFO:tasks.cephadm:Adding host smithi191 to orchestrator... 2024-09-10T08:37:34.994 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch host add smithi191 2024-09-10T08:37:35.233 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:35.502 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:35 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:35.502 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:35 smithi172 ceph-mon[30854]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:37:35.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.706+0000 7f9918a11640 1 -- 172.21.15.172:0/2286448746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9914106620 msgr2=0x7f9914106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:35.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.706+0000 7f9918a11640 1 --2- 172.21.15.172:0/2286448746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9914106620 0x7f9914106a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9904009880 tx=0x7f990402a5d0 comp rx=0 tx=0).stop 2024-09-10T08:37:35.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.706+0000 7f9918a11640 1 -- 172.21.15.172:0/2286448746 shutdown_connections 2024-09-10T08:37:35.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.706+0000 7f9918a11640 1 --2- 172.21.15.172:0/2286448746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9914106620 0x7f9914106a00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:35.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.706+0000 7f9918a11640 1 -- 172.21.15.172:0/2286448746 >> 172.21.15.172:0/2286448746 conn(0x7f99140fb920 msgr2=0x7f99140fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:35.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.706+0000 7f9918a11640 1 -- 172.21.15.172:0/2286448746 shutdown_connections 2024-09-10T08:37:35.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.706+0000 7f9918a11640 1 -- 172.21.15.172:0/2286448746 wait complete. 2024-09-10T08:37:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.707+0000 7f9918a11640 1 Processor -- start 2024-09-10T08:37:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.707+0000 7f9918a11640 1 -- start start 2024-09-10T08:37:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.707+0000 7f9918a11640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9914106620 0x7f9914197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.707+0000 7f9918a11640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f991410aae0 con 0x7f9914106620 2024-09-10T08:37:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.707+0000 7f991259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9914106620 0x7f9914197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:35.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.707+0000 7f991259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9914106620 0x7f9914197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38836/0 (socket says 172.21.15.172:38836) 2024-09-10T08:37:35.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.707+0000 7f991259c640 1 -- 172.21.15.172:0/4016068790 learned_addr learned my addr 172.21.15.172:0/4016068790 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:35.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.707+0000 7f991259c640 1 -- 172.21.15.172:0/4016068790 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9914197570 con 0x7f9914106620 2024-09-10T08:37:35.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.708+0000 7f991259c640 1 --2- 172.21.15.172:0/4016068790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9914106620 0x7f9914197030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9904009230 tx=0x7f99040048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:35.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.708+0000 7f99037fe640 1 -- 172.21.15.172:0/4016068790 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f990403f070 con 0x7f9914106620 2024-09-10T08:37:35.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.708+0000 7f9918a11640 1 -- 172.21.15.172:0/4016068790 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9914197800 con 0x7f9914106620 2024-09-10T08:37:35.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.708+0000 7f99037fe640 1 -- 172.21.15.172:0/4016068790 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9904003aa0 con 0x7f9914106620 2024-09-10T08:37:35.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.708+0000 7f99037fe640 1 -- 172.21.15.172:0/4016068790 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9904035030 con 0x7f9914106620 2024-09-10T08:37:35.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.708+0000 7f9918a11640 1 -- 172.21.15.172:0/4016068790 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f991419b900 con 0x7f9914106620 2024-09-10T08:37:35.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.709+0000 7f99037fe640 1 -- 172.21.15.172:0/4016068790 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f9904042070 con 0x7f9914106620 2024-09-10T08:37:35.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.709+0000 7f9918a11640 1 -- 172.21.15.172:0/4016068790 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9914101c70 con 0x7f9914106620 2024-09-10T08:37:35.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.709+0000 7f99037fe640 1 --2- 172.21.15.172:0/4016068790 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98fc039060 0x7f98fc03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:35.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.709+0000 7f99037fe640 1 -- 172.21.15.172:0/4016068790 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f990406cc90 con 0x7f9914106620 2024-09-10T08:37:35.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.709+0000 7f9911d9b640 1 --2- 172.21.15.172:0/4016068790 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98fc039060 0x7f98fc03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:35.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.709+0000 7f9911d9b640 1 --2- 172.21.15.172:0/4016068790 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98fc039060 0x7f98fc03b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f99080098c0 tx=0x7f9908006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:35.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.714+0000 7f99037fe640 1 -- 172.21.15.172:0/4016068790 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99040433b0 con 0x7f9914106620 2024-09-10T08:37:35.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:35.883+0000 7f9918a11640 1 -- 172.21.15.172:0/4016068790 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi191", "target": ["mon-mgr", ""]}) -- 0x7f9914106a00 con 0x7f98fc039060 2024-09-10T08:37:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:36 smithi172 ceph-mon[30854]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi191", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:38.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:37 smithi172 ceph-mon[30854]: Deploying cephadm binary to smithi191 2024-09-10T08:37:38.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:38.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:38.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:38.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:38.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:37:38.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:37:38.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:37:38.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:37 smithi172 ceph-mon[30854]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-10T08:37:38.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:37 smithi172 ceph-mon[30854]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:38.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:37 smithi172 ceph-mon[30854]: Updating smithi174:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:37:39.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:39.065+0000 7f99037fe640 1 -- 172.21.15.172:0/4016068790 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9914106a00 con 0x7f98fc039060 2024-09-10T08:37:39.065 INFO:teuthology.orchestra.run.smithi172.stdout:Added host 'smithi191' with addr '172.21.15.191' 2024-09-10T08:37:39.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:39.067+0000 7f9918a11640 1 -- 172.21.15.172:0/4016068790 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98fc039060 msgr2=0x7f98fc03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:39.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:39.067+0000 7f9918a11640 1 --2- 172.21.15.172:0/4016068790 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98fc039060 0x7f98fc03b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f99080098c0 tx=0x7f9908006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:39.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:39.067+0000 7f9918a11640 1 -- 172.21.15.172:0/4016068790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9914106620 msgr2=0x7f9914197030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:39.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:39.067+0000 7f9918a11640 1 --2- 172.21.15.172:0/4016068790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9914106620 0x7f9914197030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9904009230 tx=0x7f99040048f0 comp rx=0 tx=0).stop 2024-09-10T08:37:39.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:39.067+0000 7f9918a11640 1 -- 172.21.15.172:0/4016068790 shutdown_connections 2024-09-10T08:37:39.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:39.067+0000 7f9918a11640 1 --2- 172.21.15.172:0/4016068790 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98fc039060 0x7f98fc03b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:39.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:39.067+0000 7f9918a11640 1 --2- 172.21.15.172:0/4016068790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9914106620 0x7f9914197030 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:39.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:39.067+0000 7f9918a11640 1 -- 172.21.15.172:0/4016068790 >> 172.21.15.172:0/4016068790 conn(0x7f99140fb920 msgr2=0x7f99140ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:39.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:39.067+0000 7f9918a11640 1 -- 172.21.15.172:0/4016068790 shutdown_connections 2024-09-10T08:37:39.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:39.067+0000 7f9918a11640 1 -- 172.21.15.172:0/4016068790 wait complete. 2024-09-10T08:37:39.233 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:38 smithi172 ceph-mon[30854]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:37:39.234 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:38 smithi172 ceph-mon[30854]: Updating smithi174:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T08:37:39.234 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:38 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:39.234 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:38 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:39.234 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:38 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:39.658 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T08:37:39.895 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:40.304 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:40 smithi172 ceph-mon[30854]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:40.304 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:40.304 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:40 smithi172 ceph-mon[30854]: Added host smithi191 2024-09-10T08:37:40.304 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:37:40.305 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:40.305 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:40.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.384+0000 7f5566277640 1 -- 172.21.15.172:0/322442678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5560107660 msgr2=0x7f5560107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:40.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.384+0000 7f5566277640 1 --2- 172.21.15.172:0/322442678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5560107660 0x7f5560107a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5550009880 tx=0x7f555002a5d0 comp rx=0 tx=0).stop 2024-09-10T08:37:40.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.384+0000 7f5566277640 1 -- 172.21.15.172:0/322442678 shutdown_connections 2024-09-10T08:37:40.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.384+0000 7f5566277640 1 --2- 172.21.15.172:0/322442678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5560107660 0x7f5560107a40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:40.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.384+0000 7f5566277640 1 -- 172.21.15.172:0/322442678 >> 172.21.15.172:0/322442678 conn(0x7f55600fd2b0 msgr2=0x7f55600ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:40.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.384+0000 7f5566277640 1 -- 172.21.15.172:0/322442678 shutdown_connections 2024-09-10T08:37:40.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.384+0000 7f5566277640 1 -- 172.21.15.172:0/322442678 wait complete. 2024-09-10T08:37:40.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.385+0000 7f5566277640 1 Processor -- start 2024-09-10T08:37:40.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.385+0000 7f5566277640 1 -- start start 2024-09-10T08:37:40.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.385+0000 7f5566277640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5560107660 0x7f5560197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:40.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.385+0000 7f5566277640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f556010bb20 con 0x7f5560107660 2024-09-10T08:37:40.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.385+0000 7f555ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5560107660 0x7f5560197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:40.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.385+0000 7f555ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5560107660 0x7f5560197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51284/0 (socket says 172.21.15.172:51284) 2024-09-10T08:37:40.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.385+0000 7f555ffff640 1 -- 172.21.15.172:0/1764356990 learned_addr learned my addr 172.21.15.172:0/1764356990 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:40.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.385+0000 7f555ffff640 1 -- 172.21.15.172:0/1764356990 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5560197540 con 0x7f5560107660 2024-09-10T08:37:40.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.386+0000 7f555ffff640 1 --2- 172.21.15.172:0/1764356990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5560107660 0x7f5560197000 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5550009230 tx=0x7f55500048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:40.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.386+0000 7f555d7fa640 1 -- 172.21.15.172:0/1764356990 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f555003f070 con 0x7f5560107660 2024-09-10T08:37:40.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.386+0000 7f5566277640 1 -- 172.21.15.172:0/1764356990 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5560197830 con 0x7f5560107660 2024-09-10T08:37:40.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.386+0000 7f555d7fa640 1 -- 172.21.15.172:0/1764356990 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5550004d10 con 0x7f5560107660 2024-09-10T08:37:40.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.386+0000 7f555d7fa640 1 -- 172.21.15.172:0/1764356990 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5550035030 con 0x7f5560107660 2024-09-10T08:37:40.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.386+0000 7f5566277640 1 -- 172.21.15.172:0/1764356990 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556019b8d0 con 0x7f5560107660 2024-09-10T08:37:40.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.387+0000 7f555d7fa640 1 -- 172.21.15.172:0/1764356990 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f5550042070 con 0x7f5560107660 2024-09-10T08:37:40.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.387+0000 7f555d7fa640 1 --2- 172.21.15.172:0/1764356990 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5548039060 0x7f554803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:40.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.387+0000 7f555d7fa640 1 -- 172.21.15.172:0/1764356990 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f555006cce0 con 0x7f5560107660 2024-09-10T08:37:40.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.387+0000 7f555f7fe640 1 --2- 172.21.15.172:0/1764356990 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5548039060 0x7f554803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:40.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.387+0000 7f5566277640 1 -- 172.21.15.172:0/1764356990 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5560102d10 con 0x7f5560107660 2024-09-10T08:37:40.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.387+0000 7f555f7fe640 1 --2- 172.21.15.172:0/1764356990 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5548039060 0x7f554803b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f55540098c0 tx=0x7f5554006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:40.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.392+0000 7f555d7fa640 1 -- 172.21.15.172:0/1764356990 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55500055a0 con 0x7f5560107660 2024-09-10T08:37:40.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.565+0000 7f5566277640 1 -- 172.21.15.172:0/1764356990 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5560107ac0 con 0x7f5548039060 2024-09-10T08:37:40.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.566+0000 7f555d7fa640 1 -- 172.21.15.172:0/1764356990 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f5560107ac0 con 0x7f5548039060 2024-09-10T08:37:40.571 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:37:40.571 INFO:teuthology.orchestra.run.smithi172.stdout:[{"addr": "172.21.15.172", "hostname": "smithi172", "labels": [], "status": ""}, {"addr": "172.21.15.174", "hostname": "smithi174", "labels": [], "status": ""}, {"addr": "172.21.15.191", "hostname": "smithi191", "labels": [], "status": ""}] 2024-09-10T08:37:40.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.568+0000 7f5566277640 1 -- 172.21.15.172:0/1764356990 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5548039060 msgr2=0x7f554803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:40.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.568+0000 7f5566277640 1 --2- 172.21.15.172:0/1764356990 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5548039060 0x7f554803b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f55540098c0 tx=0x7f5554006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:40.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.568+0000 7f5566277640 1 -- 172.21.15.172:0/1764356990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5560107660 msgr2=0x7f5560197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:40.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.568+0000 7f5566277640 1 --2- 172.21.15.172:0/1764356990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5560107660 0x7f5560197000 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5550009230 tx=0x7f55500048f0 comp rx=0 tx=0).stop 2024-09-10T08:37:40.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.568+0000 7f5566277640 1 -- 172.21.15.172:0/1764356990 shutdown_connections 2024-09-10T08:37:40.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.568+0000 7f5566277640 1 --2- 172.21.15.172:0/1764356990 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5548039060 0x7f554803b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:40.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.568+0000 7f5566277640 1 --2- 172.21.15.172:0/1764356990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5560107660 0x7f5560197000 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:40.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.568+0000 7f5566277640 1 -- 172.21.15.172:0/1764356990 >> 172.21.15.172:0/1764356990 conn(0x7f55600fd2b0 msgr2=0x7f55600fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:40.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.568+0000 7f5566277640 1 -- 172.21.15.172:0/1764356990 shutdown_connections 2024-09-10T08:37:40.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:40.568+0000 7f5566277640 1 -- 172.21.15.172:0/1764356990 wait complete. 2024-09-10T08:37:41.182 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T08:37:41.182 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T08:37:41.431 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:37:41.666 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:41 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:41.666 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:41 smithi172 ceph-mon[30854]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:37:41.666 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:41 smithi172 ceph-mon[30854]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:41.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.831+0000 7fb03520e640 1 -- 172.21.15.172:0/607647622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb030100990 msgr2=0x7fb030100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:41.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.831+0000 7fb03520e640 1 --2- 172.21.15.172:0/607647622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb030100990 0x7fb030100d70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb020009880 tx=0x7fb02002a5d0 comp rx=0 tx=0).stop 2024-09-10T08:37:41.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.831+0000 7fb03520e640 1 -- 172.21.15.172:0/607647622 shutdown_connections 2024-09-10T08:37:41.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.831+0000 7fb03520e640 1 --2- 172.21.15.172:0/607647622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb030100990 0x7fb030100d70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:41.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.831+0000 7fb03520e640 1 -- 172.21.15.172:0/607647622 >> 172.21.15.172:0/607647622 conn(0x7fb030076250 msgr2=0x7fb030076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:41.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.831+0000 7fb03520e640 1 -- 172.21.15.172:0/607647622 shutdown_connections 2024-09-10T08:37:41.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.831+0000 7fb03520e640 1 -- 172.21.15.172:0/607647622 wait complete. 2024-09-10T08:37:41.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.832+0000 7fb03520e640 1 Processor -- start 2024-09-10T08:37:41.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.832+0000 7fb03520e640 1 -- start start 2024-09-10T08:37:41.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.832+0000 7fb03520e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb030100990 0x7fb0300fd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:41.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.832+0000 7fb03520e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb03010aa40 con 0x7fb030100990 2024-09-10T08:37:41.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.832+0000 7fb02ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb030100990 0x7fb0300fd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:41.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.832+0000 7fb02ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb030100990 0x7fb0300fd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51306/0 (socket says 172.21.15.172:51306) 2024-09-10T08:37:41.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.832+0000 7fb02ed9d640 1 -- 172.21.15.172:0/1484222841 learned_addr learned my addr 172.21.15.172:0/1484222841 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:37:41.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.832+0000 7fb02ed9d640 1 -- 172.21.15.172:0/1484222841 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0300ff6d0 con 0x7fb030100990 2024-09-10T08:37:41.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.833+0000 7fb02ed9d640 1 --2- 172.21.15.172:0/1484222841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb030100990 0x7fb0300fd7e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb020009230 tx=0x7fb0200048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:41.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.833+0000 7fb017fff640 1 -- 172.21.15.172:0/1484222841 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb02003f070 con 0x7fb030100990 2024-09-10T08:37:41.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.833+0000 7fb017fff640 1 -- 172.21.15.172:0/1484222841 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb020004d10 con 0x7fb030100990 2024-09-10T08:37:41.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.833+0000 7fb03520e640 1 -- 172.21.15.172:0/1484222841 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0300fdd80 con 0x7fb030100990 2024-09-10T08:37:41.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.833+0000 7fb017fff640 1 -- 172.21.15.172:0/1484222841 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb020035030 con 0x7fb030100990 2024-09-10T08:37:41.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.833+0000 7fb03520e640 1 -- 172.21.15.172:0/1484222841 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0300fe240 con 0x7fb030100990 2024-09-10T08:37:41.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.834+0000 7fb017fff640 1 -- 172.21.15.172:0/1484222841 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fb020042070 con 0x7fb030100990 2024-09-10T08:37:41.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.834+0000 7fb017fff640 1 --2- 172.21.15.172:0/1484222841 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb018039060 0x7fb01803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:41.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.834+0000 7fb017fff640 1 -- 172.21.15.172:0/1484222841 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb020003ae0 con 0x7fb030100990 2024-09-10T08:37:41.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.834+0000 7fb02e59c640 1 --2- 172.21.15.172:0/1484222841 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb018039060 0x7fb01803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:41.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.834+0000 7fb03520e640 1 -- 172.21.15.172:0/1484222841 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb030101c40 con 0x7fb030100990 2024-09-10T08:37:41.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.834+0000 7fb02e59c640 1 --2- 172.21.15.172:0/1484222841 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb018039060 0x7fb01803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb0240098c0 tx=0x7fb024006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:41.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:41.839+0000 7fb017fff640 1 -- 172.21.15.172:0/1484222841 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0200055a0 con 0x7fb030100990 2024-09-10T08:37:42.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:42.011+0000 7fb03520e640 1 -- 172.21.15.172:0/1484222841 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb0300fe530 con 0x7fb030100990 2024-09-10T08:37:42.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:42.410+0000 7fb017fff640 1 -- 172.21.15.172:0/1484222841 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fb020042c60 con 0x7fb030100990 2024-09-10T08:37:42.411 INFO:teuthology.orchestra.run.smithi172.stderr:adjusted tunables profile to default 2024-09-10T08:37:42.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:42.412+0000 7fb03520e640 1 -- 172.21.15.172:0/1484222841 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb018039060 msgr2=0x7fb01803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:42.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:42.412+0000 7fb03520e640 1 --2- 172.21.15.172:0/1484222841 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb018039060 0x7fb01803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb0240098c0 tx=0x7fb024006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:42.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:42.412+0000 7fb03520e640 1 -- 172.21.15.172:0/1484222841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb030100990 msgr2=0x7fb0300fd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:42.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:42.412+0000 7fb03520e640 1 --2- 172.21.15.172:0/1484222841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb030100990 0x7fb0300fd7e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb020009230 tx=0x7fb0200048f0 comp rx=0 tx=0).stop 2024-09-10T08:37:42.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:42.412+0000 7fb03520e640 1 -- 172.21.15.172:0/1484222841 shutdown_connections 2024-09-10T08:37:42.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:42.412+0000 7fb03520e640 1 --2- 172.21.15.172:0/1484222841 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb018039060 0x7fb01803b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:42.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:42.412+0000 7fb03520e640 1 --2- 172.21.15.172:0/1484222841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb030100990 0x7fb0300fd7e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:42.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:42.412+0000 7fb03520e640 1 -- 172.21.15.172:0/1484222841 >> 172.21.15.172:0/1484222841 conn(0x7fb030076250 msgr2=0x7fb0300696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:42.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:42.413+0000 7fb03520e640 1 -- 172.21.15.172:0/1484222841 shutdown_connections 2024-09-10T08:37:42.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:37:42.413+0000 7fb03520e640 1 -- 172.21.15.172:0/1484222841 wait complete. 2024-09-10T08:37:42.709 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:42 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1484222841' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T08:37:43.030 INFO:tasks.cephadm:Adding mon.a on smithi172 2024-09-10T08:37:43.031 INFO:tasks.cephadm:Adding mon.b on smithi174 2024-09-10T08:37:43.031 INFO:tasks.cephadm:Adding mon.c on smithi191 2024-09-10T08:37:43.031 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi172:172.21.15.172=a;smithi174:172.21.15.174=b;smithi191:172.21.15.191=c' 2024-09-10T08:37:43.277 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T08:37:43.324 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T08:37:43.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.793+0000 7fc21ba64640 1 -- 172.21.15.191:0/140571842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc214101640 msgr2=0x7fc214101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:43.807 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.793+0000 7fc21ba64640 1 --2- 172.21.15.191:0/140571842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc214101640 0x7fc214101a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc210009880 tx=0x7fc21002a5d0 comp rx=0 tx=0).stop 2024-09-10T08:37:43.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.793+0000 7fc21ba64640 1 -- 172.21.15.191:0/140571842 shutdown_connections 2024-09-10T08:37:43.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.793+0000 7fc21ba64640 1 --2- 172.21.15.191:0/140571842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc214101640 0x7fc214101a40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:43.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.793+0000 7fc21ba64640 1 -- 172.21.15.191:0/140571842 >> 172.21.15.191:0/140571842 conn(0x7fc2140fcdf0 msgr2=0x7fc2140ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:43.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.793+0000 7fc21ba64640 1 -- 172.21.15.191:0/140571842 shutdown_connections 2024-09-10T08:37:43.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.793+0000 7fc21ba64640 1 -- 172.21.15.191:0/140571842 wait complete. 2024-09-10T08:37:43.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.794+0000 7fc21ba64640 1 Processor -- start 2024-09-10T08:37:43.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.794+0000 7fc21ba64640 1 -- start start 2024-09-10T08:37:43.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.794+0000 7fc21ba64640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc214101640 0x7fc214196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:43.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.794+0000 7fc21ba64640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc214107600 con 0x7fc214101640 2024-09-10T08:37:43.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.794+0000 7fc219800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc214101640 0x7fc214196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:43.808 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.794+0000 7fc219800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc214101640 0x7fc214196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.191:49114/0 (socket says 172.21.15.191:49114) 2024-09-10T08:37:43.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.794+0000 7fc219800640 1 -- 172.21.15.191:0/2454767341 learned_addr learned my addr 172.21.15.191:0/2454767341 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-10T08:37:43.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.794+0000 7fc219800640 1 -- 172.21.15.191:0/2454767341 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc214197270 con 0x7fc214101640 2024-09-10T08:37:43.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.795+0000 7fc219800640 1 --2- 172.21.15.191:0/2454767341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc214101640 0x7fc214196d30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc210009230 tx=0x7fc2100048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:43.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.795+0000 7fc20a7fc640 1 -- 172.21.15.191:0/2454767341 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc21003f070 con 0x7fc214101640 2024-09-10T08:37:43.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.795+0000 7fc20a7fc640 1 -- 172.21.15.191:0/2454767341 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc210003aa0 con 0x7fc214101640 2024-09-10T08:37:43.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.795+0000 7fc21ba64640 1 -- 172.21.15.191:0/2454767341 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc214197500 con 0x7fc214101640 2024-09-10T08:37:43.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.795+0000 7fc20a7fc640 1 -- 172.21.15.191:0/2454767341 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc210035030 con 0x7fc214101640 2024-09-10T08:37:43.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.795+0000 7fc21ba64640 1 -- 172.21.15.191:0/2454767341 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc21419a1f0 con 0x7fc214101640 2024-09-10T08:37:43.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.796+0000 7fc20a7fc640 1 -- 172.21.15.191:0/2454767341 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fc210042070 con 0x7fc214101640 2024-09-10T08:37:43.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.796+0000 7fc20a7fc640 1 --2- 172.21.15.191:0/2454767341 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc200039060 0x7fc20003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:43.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.796+0000 7fc20a7fc640 1 -- 172.21.15.191:0/2454767341 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc21006cea0 con 0x7fc214101640 2024-09-10T08:37:43.809 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.796+0000 7fc21ba64640 1 -- 172.21.15.191:0/2454767341 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc214107d80 con 0x7fc214101640 2024-09-10T08:37:43.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.796+0000 7fc218fff640 1 --2- 172.21.15.191:0/2454767341 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc200039060 0x7fc20003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:43.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.797+0000 7fc218fff640 1 --2- 172.21.15.191:0/2454767341 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc200039060 0x7fc20003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc2040098c0 tx=0x7fc204006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:43.810 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.801+0000 7fc20a7fc640 1 -- 172.21.15.191:0/2454767341 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc21002f300 con 0x7fc214101640 2024-09-10T08:37:43.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:43 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1484222841' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T08:37:43.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:43 smithi172 ceph-mon[30854]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:37:43.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:43 smithi172 ceph-mon[30854]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:43.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:43 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:43.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:43 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:43.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:43 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:43.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:43 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:43.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:43 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi191", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:37:43.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:43 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:37:43.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:43 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:37:43.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:43 smithi172 ceph-mon[30854]: Updating smithi191:/etc/ceph/ceph.conf 2024-09-10T08:37:43.989 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.989+0000 7fc21ba64640 1 -- 172.21.15.191:0/2454767341 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi172:172.21.15.172=a;smithi174:172.21.15.174=b;smithi191:172.21.15.191=c", "target": ["mon-mgr", ""]}) -- 0x7fc2140626f0 con 0x7fc200039060 2024-09-10T08:37:43.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.994+0000 7fc20a7fc640 1 -- 172.21.15.191:0/2454767341 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc2140626f0 con 0x7fc200039060 2024-09-10T08:37:43.994 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled mon update... 2024-09-10T08:37:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.996+0000 7fc21ba64640 1 -- 172.21.15.191:0/2454767341 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc200039060 msgr2=0x7fc20003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.996+0000 7fc21ba64640 1 --2- 172.21.15.191:0/2454767341 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc200039060 0x7fc20003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc2040098c0 tx=0x7fc204006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.996+0000 7fc21ba64640 1 -- 172.21.15.191:0/2454767341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc214101640 msgr2=0x7fc214196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.996+0000 7fc21ba64640 1 --2- 172.21.15.191:0/2454767341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc214101640 0x7fc214196d30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc210009230 tx=0x7fc2100048f0 comp rx=0 tx=0).stop 2024-09-10T08:37:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.996+0000 7fc21ba64640 1 -- 172.21.15.191:0/2454767341 shutdown_connections 2024-09-10T08:37:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.996+0000 7fc21ba64640 1 --2- 172.21.15.191:0/2454767341 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc200039060 0x7fc20003b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.996+0000 7fc21ba64640 1 --2- 172.21.15.191:0/2454767341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc214101640 0x7fc214196d30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.996+0000 7fc21ba64640 1 -- 172.21.15.191:0/2454767341 >> 172.21.15.191:0/2454767341 conn(0x7fc2140fcdf0 msgr2=0x7fc2140fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:43.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.996+0000 7fc21ba64640 1 -- 172.21.15.191:0/2454767341 shutdown_connections 2024-09-10T08:37:43.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:43.997+0000 7fc21ba64640 1 -- 172.21.15.191:0/2454767341 wait complete. 2024-09-10T08:37:44.574 DEBUG:teuthology.orchestra.run.smithi174:mon.b> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T08:37:44.576 DEBUG:teuthology.orchestra.run.smithi191:mon.c> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T08:37:44.579 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T08:37:44.579 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T08:37:44.847 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:37:45.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: Updating smithi191:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:37:45.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:37:45.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: Updating smithi191:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T08:37:45.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:45.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:45.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:45.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi172:172.21.15.172=a;smithi174:172.21.15.174=b;smithi191:172.21.15.191=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:45.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: Saving service mon spec with placement smithi172:172.21.15.172=a;smithi174:172.21.15.174=b;smithi191:172.21.15.191=c;count:3 2024-09-10T08:37:45.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:45.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:45.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:45.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:45.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:45.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T08:37:45.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:37:45.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:37:45.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:37:45.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:44 smithi172 ceph-mon[30854]: Reconfiguring daemon mon.a on smithi172 2024-09-10T08:37:45.266 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.261+0000 7f83002a2640 1 -- 172.21.15.191:0/4222007406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82f8107660 msgr2=0x7f82f8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:45.266 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.261+0000 7f83002a2640 1 --2- 172.21.15.191:0/4222007406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82f8107660 0x7f82f8107a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009880 tx=0x7f82f402a5d0 comp rx=0 tx=0).stop 2024-09-10T08:37:45.267 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.261+0000 7f83002a2640 1 -- 172.21.15.191:0/4222007406 shutdown_connections 2024-09-10T08:37:45.267 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.261+0000 7f83002a2640 1 --2- 172.21.15.191:0/4222007406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82f8107660 0x7f82f8107a40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:45.267 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.261+0000 7f83002a2640 1 -- 172.21.15.191:0/4222007406 >> 172.21.15.191:0/4222007406 conn(0x7f82f80fd2b0 msgr2=0x7f82f80ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:45.267 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.261+0000 7f83002a2640 1 -- 172.21.15.191:0/4222007406 shutdown_connections 2024-09-10T08:37:45.267 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.261+0000 7f83002a2640 1 -- 172.21.15.191:0/4222007406 wait complete. 2024-09-10T08:37:45.267 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.261+0000 7f83002a2640 1 Processor -- start 2024-09-10T08:37:45.267 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.261+0000 7f83002a2640 1 -- start start 2024-09-10T08:37:45.267 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.262+0000 7f83002a2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82f8107660 0x7f82f8196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:45.267 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.262+0000 7f83002a2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f82f810bb20 con 0x7f82f8107660 2024-09-10T08:37:45.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.262+0000 7f82fe03e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82f8107660 0x7f82f8196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:45.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.262+0000 7f82fe03e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82f8107660 0x7f82f8196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.191:44358/0 (socket says 172.21.15.191:44358) 2024-09-10T08:37:45.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.262+0000 7f82fe03e640 1 -- 172.21.15.191:0/1363935317 learned_addr learned my addr 172.21.15.191:0/1363935317 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-10T08:37:45.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.262+0000 7f82fe03e640 1 -- 172.21.15.191:0/1363935317 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f8197500 con 0x7f82f8107660 2024-09-10T08:37:45.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.262+0000 7f82fe03e640 1 --2- 172.21.15.191:0/1363935317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82f8107660 0x7f82f8196fc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009230 tx=0x7f82f40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:45.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.262+0000 7f82ef7fe640 1 -- 172.21.15.191:0/1363935317 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82f403f070 con 0x7f82f8107660 2024-09-10T08:37:45.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.263+0000 7f83002a2640 1 -- 172.21.15.191:0/1363935317 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f81977f0 con 0x7f82f8107660 2024-09-10T08:37:45.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.263+0000 7f82ef7fe640 1 -- 172.21.15.191:0/1363935317 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82f4004d10 con 0x7f82f8107660 2024-09-10T08:37:45.268 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.263+0000 7f82ef7fe640 1 -- 172.21.15.191:0/1363935317 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82f4035030 con 0x7f82f8107660 2024-09-10T08:37:45.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.263+0000 7f83002a2640 1 -- 172.21.15.191:0/1363935317 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f819b890 con 0x7f82f8107660 2024-09-10T08:37:45.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.263+0000 7f83002a2640 1 -- 172.21.15.191:0/1363935317 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f8102d10 con 0x7f82f8107660 2024-09-10T08:37:45.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.264+0000 7f82ef7fe640 1 -- 172.21.15.191:0/1363935317 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f82f4042070 con 0x7f82f8107660 2024-09-10T08:37:45.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.264+0000 7f82ef7fe640 1 --2- 172.21.15.191:0/1363935317 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f82e4039060 0x7f82e403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:45.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.264+0000 7f82ef7fe640 1 -- 172.21.15.191:0/1363935317 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f82f406cd30 con 0x7f82f8107660 2024-09-10T08:37:45.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.264+0000 7f82fd83d640 1 --2- 172.21.15.191:0/1363935317 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f82e4039060 0x7f82e403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:45.269 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.264+0000 7f82fd83d640 1 --2- 172.21.15.191:0/1363935317 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f82e4039060 0x7f82e403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f82e80098c0 tx=0x7f82e8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:45.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.268+0000 7f82ef7fe640 1 -- 172.21.15.191:0/1363935317 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82f4042360 con 0x7f82f8107660 2024-09-10T08:37:45.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.506+0000 7f83002a2640 1 -- 172.21.15.191:0/1363935317 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f82f8107af0 con 0x7f82f8107660 2024-09-10T08:37:45.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.507+0000 7f82ef7fe640 1 -- 172.21.15.191:0/1363935317 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f82f8107af0 con 0x7f82f8107660 2024-09-10T08:37:45.507 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:37:45.508 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":1,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","modified":"2024-09-10T08:36:29.338346Z","created":"2024-09-10T08:36:29.338346Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T08:37:45.508 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 1 2024-09-10T08:37:45.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.510+0000 7f83002a2640 1 -- 172.21.15.191:0/1363935317 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f82e4039060 msgr2=0x7f82e403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:45.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.510+0000 7f83002a2640 1 --2- 172.21.15.191:0/1363935317 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f82e4039060 0x7f82e403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f82e80098c0 tx=0x7f82e8006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:45.510 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.510+0000 7f83002a2640 1 -- 172.21.15.191:0/1363935317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82f8107660 msgr2=0x7f82f8196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:45.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.510+0000 7f83002a2640 1 --2- 172.21.15.191:0/1363935317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82f8107660 0x7f82f8196fc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009230 tx=0x7f82f40048f0 comp rx=0 tx=0).stop 2024-09-10T08:37:45.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.510+0000 7f83002a2640 1 -- 172.21.15.191:0/1363935317 shutdown_connections 2024-09-10T08:37:45.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.510+0000 7f83002a2640 1 --2- 172.21.15.191:0/1363935317 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f82e4039060 0x7f82e403b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:45.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.510+0000 7f83002a2640 1 --2- 172.21.15.191:0/1363935317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82f8107660 0x7f82f8196fc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:45.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.510+0000 7f83002a2640 1 -- 172.21.15.191:0/1363935317 >> 172.21.15.191:0/1363935317 conn(0x7f82f80fd2b0 msgr2=0x7f82f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:45.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.510+0000 7f83002a2640 1 -- 172.21.15.191:0/1363935317 shutdown_connections 2024-09-10T08:37:45.511 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:45.510+0000 7f83002a2640 1 -- 172.21.15.191:0/1363935317 wait complete. 2024-09-10T08:37:46.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:45 smithi172 ceph-mon[30854]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:46.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:46.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:46.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:37:46.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:37:46.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:37:46.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:46.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:37:46.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:37:46.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:45 smithi172 ceph-mon[30854]: from='client.? 172.21.15.191:0/1363935317' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:37:47.218 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T08:37:47.218 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T08:37:47.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:46 smithi172 ceph-mon[30854]: Deploying daemon mon.c on smithi191 2024-09-10T08:37:48.258 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:37:48.331 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:47 smithi172 ceph-mon[30854]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:48.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.803+0000 7f8dfa567640 1 -- 172.21.15.191:0/2633287438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8df4106660 msgr2=0x7f8df4106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:48.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.803+0000 7f8dfa567640 1 --2- 172.21.15.191:0/2633287438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8df4106660 0x7f8df4106a40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009880 tx=0x7f8de402a5d0 comp rx=0 tx=0).stop 2024-09-10T08:37:48.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.803+0000 7f8dfa567640 1 -- 172.21.15.191:0/2633287438 shutdown_connections 2024-09-10T08:37:48.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.803+0000 7f8dfa567640 1 --2- 172.21.15.191:0/2633287438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8df4106660 0x7f8df4106a40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:48.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.803+0000 7f8dfa567640 1 -- 172.21.15.191:0/2633287438 >> 172.21.15.191:0/2633287438 conn(0x7f8df40fc250 msgr2=0x7f8df40fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:48.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.803+0000 7f8dfa567640 1 -- 172.21.15.191:0/2633287438 shutdown_connections 2024-09-10T08:37:48.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.803+0000 7f8dfa567640 1 -- 172.21.15.191:0/2633287438 wait complete. 2024-09-10T08:37:48.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.803+0000 7f8dfa567640 1 Processor -- start 2024-09-10T08:37:48.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.803+0000 7f8dfa567640 1 -- start start 2024-09-10T08:37:48.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.804+0000 7f8dfa567640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8df4106660 0x7f8df4199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:48.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.804+0000 7f8dfa567640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8df410ab40 con 0x7f8df4106660 2024-09-10T08:37:48.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.804+0000 7f8df3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8df4106660 0x7f8df4199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:48.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.804+0000 7f8df3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8df4106660 0x7f8df4199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.191:44376/0 (socket says 172.21.15.191:44376) 2024-09-10T08:37:48.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.804+0000 7f8df3fff640 1 -- 172.21.15.191:0/1266788502 learned_addr learned my addr 172.21.15.191:0/1266788502 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-10T08:37:48.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.804+0000 7f8df3fff640 1 -- 172.21.15.191:0/1266788502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df4199950 con 0x7f8df4106660 2024-09-10T08:37:48.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.805+0000 7f8df3fff640 1 --2- 172.21.15.191:0/1266788502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8df4106660 0x7f8df4199410 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009230 tx=0x7f8de40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:48.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.805+0000 7f8df17fa640 1 -- 172.21.15.191:0/1266788502 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8de403f070 con 0x7f8df4106660 2024-09-10T08:37:48.821 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.805+0000 7f8df17fa640 1 -- 172.21.15.191:0/1266788502 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8de4004d10 con 0x7f8df4106660 2024-09-10T08:37:48.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.805+0000 7f8dfa567640 1 -- 172.21.15.191:0/1266788502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df4199be0 con 0x7f8df4106660 2024-09-10T08:37:48.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.805+0000 7f8df17fa640 1 -- 172.21.15.191:0/1266788502 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8de4035030 con 0x7f8df4106660 2024-09-10T08:37:48.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.805+0000 7f8dfa567640 1 -- 172.21.15.191:0/1266788502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df419a040 con 0x7f8df4106660 2024-09-10T08:37:48.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.806+0000 7f8df17fa640 1 -- 172.21.15.191:0/1266788502 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f8de4042070 con 0x7f8df4106660 2024-09-10T08:37:48.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.806+0000 7f8dfa567640 1 -- 172.21.15.191:0/1266788502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df4101cd0 con 0x7f8df4106660 2024-09-10T08:37:48.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.806+0000 7f8df17fa640 1 --2- 172.21.15.191:0/1266788502 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8ddc039060 0x7f8ddc03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:48.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.806+0000 7f8df17fa640 1 -- 172.21.15.191:0/1266788502 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8de406cd60 con 0x7f8df4106660 2024-09-10T08:37:48.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.806+0000 7f8df37fe640 1 --2- 172.21.15.191:0/1266788502 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8ddc039060 0x7f8ddc03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:48.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.807+0000 7f8df37fe640 1 --2- 172.21.15.191:0/1266788502 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8ddc039060 0x7f8ddc03b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8de80098c0 tx=0x7f8de8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:48.822 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:48.811+0000 7f8df17fa640 1 -- 172.21.15.191:0/1266788502 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8de40372d0 con 0x7f8df4106660 2024-09-10T08:37:49.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:49.066+0000 7f8dfa567640 1 -- 172.21.15.191:0/1266788502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8df419a770 con 0x7f8df4106660 2024-09-10T08:37:49.066 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:49.066+0000 7f8df17fa640 1 -- 172.21.15.191:0/1266788502 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f8de4003920 con 0x7f8df4106660 2024-09-10T08:37:49.066 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:37:49.067 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":1,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","modified":"2024-09-10T08:36:29.338346Z","created":"2024-09-10T08:36:29.338346Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T08:37:49.067 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 1 2024-09-10T08:37:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:49.069+0000 7f8dfa567640 1 -- 172.21.15.191:0/1266788502 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8ddc039060 msgr2=0x7f8ddc03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:49.069+0000 7f8dfa567640 1 --2- 172.21.15.191:0/1266788502 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8ddc039060 0x7f8ddc03b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8de80098c0 tx=0x7f8de8006db0 comp rx=0 tx=0).stop 2024-09-10T08:37:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:49.069+0000 7f8dfa567640 1 -- 172.21.15.191:0/1266788502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8df4106660 msgr2=0x7f8df4199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:49.069+0000 7f8dfa567640 1 --2- 172.21.15.191:0/1266788502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8df4106660 0x7f8df4199410 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009230 tx=0x7f8de40048f0 comp rx=0 tx=0).stop 2024-09-10T08:37:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:49.069+0000 7f8dfa567640 1 -- 172.21.15.191:0/1266788502 shutdown_connections 2024-09-10T08:37:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:49.069+0000 7f8dfa567640 1 --2- 172.21.15.191:0/1266788502 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8ddc039060 0x7f8ddc03b520 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:49.069 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:49.069+0000 7f8dfa567640 1 --2- 172.21.15.191:0/1266788502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8df4106660 0x7f8df4199410 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:49.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:49.069+0000 7f8dfa567640 1 -- 172.21.15.191:0/1266788502 >> 172.21.15.191:0/1266788502 conn(0x7f8df40fc250 msgr2=0x7f8df40fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:49.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:49.069+0000 7f8dfa567640 1 -- 172.21.15.191:0/1266788502 shutdown_connections 2024-09-10T08:37:49.070 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:49.069+0000 7f8dfa567640 1 -- 172.21.15.191:0/1266788502 wait complete. 2024-09-10T08:37:49.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:49 smithi172 ceph-mon[30854]: from='client.? 172.21.15.191:0/1266788502' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:37:50.299 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:50 smithi191 systemd[1]: Started Ceph mon.c for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:37:50.614 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T08:37:50.615 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T08:37:50.863 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:37:55.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:37:55.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:55.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: mon.a calling monitor election 2024-09-10T08:37:55.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:55.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:55.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:55.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: mon.c calling monitor election 2024-09-10T08:37:55.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:55.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:55.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:55.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:55.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T08:37:55.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: monmap epoch 2 2024-09-10T08:37:55.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: fsid a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:37:55.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: last_changed 2024-09-10T08:37:50.096260+0000 2024-09-10T08:37:55.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: created 2024-09-10T08:36:29.338346+0000 2024-09-10T08:37:55.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: min_mon_release 19 (squid) 2024-09-10T08:37:55.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: election_strategy: 1 2024-09-10T08:37:55.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: 0: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.a 2024-09-10T08:37:55.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: 1: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.c 2024-09-10T08:37:55.419 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: fsmap 2024-09-10T08:37:55.419 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:37:55.419 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: mgrmap e13: x(active, since 38s) 2024-09-10T08:37:55.419 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: overall HEALTH_OK 2024-09-10T08:37:55.419 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:55.419 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:55.419 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:55.419 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:37:55.419 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:55 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:37:55.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:37:55.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:55.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: mon.a calling monitor election 2024-09-10T08:37:55.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:55.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:55.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:55.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: mon.c calling monitor election 2024-09-10T08:37:55.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:55.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:55.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:55.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:55.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T08:37:55.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: monmap epoch 2 2024-09-10T08:37:55.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: fsid a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:37:55.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: last_changed 2024-09-10T08:37:50.096260+0000 2024-09-10T08:37:55.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: created 2024-09-10T08:36:29.338346+0000 2024-09-10T08:37:55.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: min_mon_release 19 (squid) 2024-09-10T08:37:55.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: election_strategy: 1 2024-09-10T08:37:55.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: 0: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.a 2024-09-10T08:37:55.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: 1: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.c 2024-09-10T08:37:55.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: fsmap 2024-09-10T08:37:55.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:37:55.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: mgrmap e13: x(active, since 38s) 2024-09-10T08:37:55.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: overall HEALTH_OK 2024-09-10T08:37:55.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:55.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:55.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:55.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:37:55.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:55 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:37:55.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.814+0000 7fa808fdd640 1 -- 172.21.15.191:0/3119605143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7f0003530 msgr2=0x7fa7f00059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:55.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.814+0000 7fa808fdd640 1 --2- 172.21.15.191:0/3119605143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7f0003530 0x7fa7f00059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa804069e40 tx=0x7fa7f402c490 comp rx=0 tx=0).stop 2024-09-10T08:37:55.815 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.814+0000 7fa808fdd640 1 -- 172.21.15.191:0/3119605143 shutdown_connections 2024-09-10T08:37:55.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.814+0000 7fa808fdd640 1 --2- 172.21.15.191:0/3119605143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7f0003530 0x7fa7f00059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:55.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.814+0000 7fa808fdd640 1 -- 172.21.15.191:0/3119605143 >> 172.21.15.191:0/3119605143 conn(0x7fa8040fb9a0 msgr2=0x7fa8040fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:55.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.814+0000 7fa808fdd640 1 -- 172.21.15.191:0/3119605143 shutdown_connections 2024-09-10T08:37:55.816 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.814+0000 7fa808fdd640 1 -- 172.21.15.191:0/3119605143 wait complete. 2024-09-10T08:37:55.817 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.815+0000 7fa808fdd640 1 Processor -- start 2024-09-10T08:37:55.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.816+0000 7fa808fdd640 1 -- start start 2024-09-10T08:37:55.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.816+0000 7fa808fdd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7f0003530 0x7fa804198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:55.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.816+0000 7fa808fdd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa804198570 0x7fa8041989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:55.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.816+0000 7fa808fdd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa804108520 con 0x7fa7f0003530 2024-09-10T08:37:55.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.816+0000 7fa808fdd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa8041083a0 con 0x7fa804198570 2024-09-10T08:37:55.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.816+0000 7fa7fbfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa804198570 0x7fa8041989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:55.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.816+0000 7fa7fbfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa804198570 0x7fa8041989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:43344/0 (socket says 172.21.15.191:43344) 2024-09-10T08:37:55.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.816+0000 7fa7fbfff640 1 -- 172.21.15.191:0/1378135791 learned_addr learned my addr 172.21.15.191:0/1378135791 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-10T08:37:55.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.816+0000 7fa80259c640 1 --2- 172.21.15.191:0/1378135791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7f0003530 0x7fa804198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:55.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.816+0000 7fa7fbfff640 1 -- 172.21.15.191:0/1378135791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa804198570 msgr2=0x7fa8041989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T08:37:55.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.816+0000 7fa7fbfff640 1 -- 172.21.15.191:0/1378135791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa804198570 msgr2=0x7fa8041989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T08:37:55.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.816+0000 7fa7fbfff640 1 --2- 172.21.15.191:0/1378135791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa804198570 0x7fa8041989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T08:37:55.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.816+0000 7fa7fbfff640 1 --2- 172.21.15.191:0/1378135791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa804198570 0x7fa8041989d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T08:37:55.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.817+0000 7fa80259c640 1 -- 172.21.15.191:0/1378135791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa804198570 msgr2=0x7fa8041989d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:37:55.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.817+0000 7fa80259c640 1 --2- 172.21.15.191:0/1378135791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa804198570 0x7fa8041989d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:55.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.817+0000 7fa80259c640 1 -- 172.21.15.191:0/1378135791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8041a10a0 con 0x7fa7f0003530 2024-09-10T08:37:55.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.817+0000 7fa80259c640 1 --2- 172.21.15.191:0/1378135791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7f0003530 0x7fa804198030 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa7f402ca80 tx=0x7fa7f40339a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:55.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.817+0000 7fa7fb7fe640 1 -- 172.21.15.191:0/1378135791 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa7f4005200 con 0x7fa7f0003530 2024-09-10T08:37:55.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.817+0000 7fa7fb7fe640 1 -- 172.21.15.191:0/1378135791 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7f4032ca0 con 0x7fa7f0003530 2024-09-10T08:37:55.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.817+0000 7fa7fb7fe640 1 -- 172.21.15.191:0/1378135791 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa7f4004b90 con 0x7fa7f0003530 2024-09-10T08:37:55.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.817+0000 7fa808fdd640 1 -- 172.21.15.191:0/1378135791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8041a1270 con 0x7fa7f0003530 2024-09-10T08:37:55.819 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.817+0000 7fa808fdd640 1 -- 172.21.15.191:0/1378135791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8041a1720 con 0x7fa7f0003530 2024-09-10T08:37:55.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.819+0000 7fa7fb7fe640 1 -- 172.21.15.191:0/1378135791 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fa7f4043020 con 0x7fa7f0003530 2024-09-10T08:37:55.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.819+0000 7fa808fdd640 1 -- 172.21.15.191:0/1378135791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa804101ea0 con 0x7fa7f0003530 2024-09-10T08:37:55.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.819+0000 7fa7fb7fe640 1 --2- 172.21.15.191:0/1378135791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7e4038e50 0x7fa7e403b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:55.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.819+0000 7fa7fb7fe640 1 -- 172.21.15.191:0/1378135791 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa7f4041020 con 0x7fa7f0003530 2024-09-10T08:37:55.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.819+0000 7fa7fbfff640 1 --2- 172.21.15.191:0/1378135791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7e4038e50 0x7fa7e403b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:55.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.820+0000 7fa7fbfff640 1 --2- 172.21.15.191:0/1378135791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7e4038e50 0x7fa7e403b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009100 tx=0x7fa7ec008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:55.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:55.824+0000 7fa7fb7fe640 1 -- 172.21.15.191:0/1378135791 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa7f404a020 con 0x7fa7f0003530 2024-09-10T08:37:56.057 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:56.057+0000 7fa808fdd640 1 -- 172.21.15.191:0/1378135791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa80418f6f0 con 0x7fa7f0003530 2024-09-10T08:37:56.058 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:56.058+0000 7fa7fb7fe640 1 -- 172.21.15.191:0/1378135791 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fa7f403ab60 con 0x7fa7f0003530 2024-09-10T08:37:56.058 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:37:56.059 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":2,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","modified":"2024-09-10T08:37:50.096260Z","created":"2024-09-10T08:36:29.338346Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:3300","nonce":0},{"type":"v1","addr":"172.21.15.191:6789","nonce":0}]},"addr":"172.21.15.191:6789/0","public_addr":"172.21.15.191:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T08:37:56.059 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 2 2024-09-10T08:37:56.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:56.060+0000 7fa808fdd640 1 -- 172.21.15.191:0/1378135791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7e4038e50 msgr2=0x7fa7e403b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:56.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:56.060+0000 7fa808fdd640 1 --2- 172.21.15.191:0/1378135791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7e4038e50 0x7fa7e403b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009100 tx=0x7fa7ec008040 comp rx=0 tx=0).stop 2024-09-10T08:37:56.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:56.060+0000 7fa808fdd640 1 -- 172.21.15.191:0/1378135791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7f0003530 msgr2=0x7fa804198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:56.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:56.060+0000 7fa808fdd640 1 --2- 172.21.15.191:0/1378135791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7f0003530 0x7fa804198030 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa7f402ca80 tx=0x7fa7f40339a0 comp rx=0 tx=0).stop 2024-09-10T08:37:56.060 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:56.060+0000 7fa808fdd640 1 -- 172.21.15.191:0/1378135791 shutdown_connections 2024-09-10T08:37:56.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:56.060+0000 7fa808fdd640 1 --2- 172.21.15.191:0/1378135791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7e4038e50 0x7fa7e403b310 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:56.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:56.060+0000 7fa808fdd640 1 --2- 172.21.15.191:0/1378135791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa804198570 0x7fa8041989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:56.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:56.060+0000 7fa808fdd640 1 --2- 172.21.15.191:0/1378135791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7f0003530 0x7fa804198030 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:56.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:56.060+0000 7fa808fdd640 1 -- 172.21.15.191:0/1378135791 >> 172.21.15.191:0/1378135791 conn(0x7fa8040fb9a0 msgr2=0x7fa8040fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:56.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:56.060+0000 7fa808fdd640 1 -- 172.21.15.191:0/1378135791 shutdown_connections 2024-09-10T08:37:56.061 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:56.060+0000 7fa808fdd640 1 -- 172.21.15.191:0/1378135791 wait complete. 2024-09-10T08:37:56.407 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:56 smithi191 ceph-mon[35314]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:56.407 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:56 smithi191 ceph-mon[35314]: Deploying daemon mon.b on smithi174 2024-09-10T08:37:56.407 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:56 smithi191 ceph-mon[35314]: from='client.? 172.21.15.191:0/1378135791' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:37:56.408 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:56 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:56.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:56 smithi172 ceph-mon[30854]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:56.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:56 smithi172 ceph-mon[30854]: Deploying daemon mon.b on smithi174 2024-09-10T08:37:56.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:56 smithi172 ceph-mon[30854]: from='client.? 172.21.15.191:0/1378135791' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:37:56.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:56 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:57.703 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T08:37:57.703 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T08:37:57.954 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:37:58.396 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:37:58 smithi191 ceph-mon[35314]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:58.479 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.475+0000 7f80971a0640 1 -- 172.21.15.191:0/2837369587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80901064f0 msgr2=0x7f80901068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:58.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.475+0000 7f80971a0640 1 --2- 172.21.15.191:0/2837369587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80901064f0 0x7f80901068d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f808c009880 tx=0x7f808c02a5d0 comp rx=0 tx=0).stop 2024-09-10T08:37:58.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.475+0000 7f80971a0640 1 -- 172.21.15.191:0/2837369587 shutdown_connections 2024-09-10T08:37:58.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.475+0000 7f80971a0640 1 --2- 172.21.15.191:0/2837369587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80901064f0 0x7f80901068d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:58.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.475+0000 7f80971a0640 1 -- 172.21.15.191:0/2837369587 >> 172.21.15.191:0/2837369587 conn(0x7f80900784e0 msgr2=0x7f80900788f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:58.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.475+0000 7f80971a0640 1 -- 172.21.15.191:0/2837369587 shutdown_connections 2024-09-10T08:37:58.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.475+0000 7f80971a0640 1 -- 172.21.15.191:0/2837369587 wait complete. 2024-09-10T08:37:58.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.476+0000 7f80971a0640 1 Processor -- start 2024-09-10T08:37:58.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.476+0000 7f80971a0640 1 -- start start 2024-09-10T08:37:58.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.476+0000 7f80971a0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80901064f0 0x7f8090197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:58.480 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.476+0000 7f80971a0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090197640 0x7f809019b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:58.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.476+0000 7f80971a0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f809010ba70 con 0x7f8090197640 2024-09-10T08:37:58.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.476+0000 7f80971a0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f809010b8f0 con 0x7f80901064f0 2024-09-10T08:37:58.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.477+0000 7f809619e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80901064f0 0x7f8090197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:58.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.477+0000 7f809619e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80901064f0 0x7f8090197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:43352/0 (socket says 172.21.15.191:43352) 2024-09-10T08:37:58.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.477+0000 7f809619e640 1 -- 172.21.15.191:0/1941882488 learned_addr learned my addr 172.21.15.191:0/1941882488 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-10T08:37:58.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.477+0000 7f809619e640 1 -- 172.21.15.191:0/1941882488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80901064f0 msgr2=0x7f8090197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-10T08:37:58.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.477+0000 7f809619e640 1 -- 172.21.15.191:0/1941882488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80901064f0 msgr2=0x7f8090197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T08:37:58.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.477+0000 7f809619e640 1 --2- 172.21.15.191:0/1941882488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80901064f0 0x7f8090197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T08:37:58.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.477+0000 7f809599d640 1 --2- 172.21.15.191:0/1941882488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090197640 0x7f809019b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:58.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.477+0000 7f809619e640 1 --2- 172.21.15.191:0/1941882488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80901064f0 0x7f8090197100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T08:37:58.481 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.477+0000 7f809599d640 1 -- 172.21.15.191:0/1941882488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80901064f0 msgr2=0x7f8090197100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:37:58.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.477+0000 7f809599d640 1 --2- 172.21.15.191:0/1941882488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80901064f0 0x7f8090197100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:58.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.477+0000 7f809599d640 1 -- 172.21.15.191:0/1941882488 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8090197c60 con 0x7f8090197640 2024-09-10T08:37:58.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.477+0000 7f809599d640 1 --2- 172.21.15.191:0/1941882488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090197640 0x7f809019b9d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f808000b3d0 tx=0x7f808000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:58.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.477+0000 7f80877fe640 1 -- 172.21.15.191:0/1941882488 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8080011630 con 0x7f8090197640 2024-09-10T08:37:58.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.477+0000 7f80877fe640 1 -- 172.21.15.191:0/1941882488 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8080011c80 con 0x7f8090197640 2024-09-10T08:37:58.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.477+0000 7f80971a0640 1 -- 172.21.15.191:0/1941882488 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809019bf10 con 0x7f8090197640 2024-09-10T08:37:58.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.477+0000 7f80877fe640 1 -- 172.21.15.191:0/1941882488 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8080010d80 con 0x7f8090197640 2024-09-10T08:37:58.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.477+0000 7f80971a0640 1 -- 172.21.15.191:0/1941882488 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f809019c450 con 0x7f8090197640 2024-09-10T08:37:58.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.478+0000 7f80877fe640 1 -- 172.21.15.191:0/1941882488 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f8080018020 con 0x7f8090197640 2024-09-10T08:37:58.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.478+0000 7f80971a0640 1 -- 172.21.15.191:0/1941882488 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8090102bf0 con 0x7f8090197640 2024-09-10T08:37:58.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.478+0000 7f80877fe640 1 --2- 172.21.15.191:0/1941882488 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f807c038df0 0x7f807c03b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:37:58.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.479+0000 7f80877fe640 1 -- 172.21.15.191:0/1941882488 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f808004bce0 con 0x7f8090197640 2024-09-10T08:37:58.482 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.479+0000 7f809619e640 1 --2- 172.21.15.191:0/1941882488 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f807c038df0 0x7f807c03b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:37:58.483 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.479+0000 7f809619e640 1 --2- 172.21.15.191:0/1941882488 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f807c038df0 0x7f807c03b2b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f808c02abc0 tx=0x7f808c02aad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:37:58.484 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.484+0000 7f80877fe640 1 -- 172.21.15.191:0/1941882488 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f808001f020 con 0x7f8090197640 2024-09-10T08:37:58.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:37:58 smithi172 ceph-mon[30854]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:58.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.711+0000 7f80971a0640 1 -- 172.21.15.191:0/1941882488 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f809019c760 con 0x7f8090197640 2024-09-10T08:37:58.712 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.712+0000 7f80877fe640 1 -- 172.21.15.191:0/1941882488 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f80800189a0 con 0x7f8090197640 2024-09-10T08:37:58.712 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:37:58.713 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":2,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","modified":"2024-09-10T08:37:50.096260Z","created":"2024-09-10T08:36:29.338346Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:3300","nonce":0},{"type":"v1","addr":"172.21.15.191:6789","nonce":0}]},"addr":"172.21.15.191:6789/0","public_addr":"172.21.15.191:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T08:37:58.713 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 2 2024-09-10T08:37:58.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.714+0000 7f80971a0640 1 -- 172.21.15.191:0/1941882488 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f807c038df0 msgr2=0x7f807c03b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:58.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.714+0000 7f80971a0640 1 --2- 172.21.15.191:0/1941882488 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f807c038df0 0x7f807c03b2b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f808c02abc0 tx=0x7f808c02aad0 comp rx=0 tx=0).stop 2024-09-10T08:37:58.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.714+0000 7f80971a0640 1 -- 172.21.15.191:0/1941882488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090197640 msgr2=0x7f809019b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:37:58.714 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.714+0000 7f80971a0640 1 --2- 172.21.15.191:0/1941882488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090197640 0x7f809019b9d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f808000b3d0 tx=0x7f808000b890 comp rx=0 tx=0).stop 2024-09-10T08:37:58.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.714+0000 7f80971a0640 1 -- 172.21.15.191:0/1941882488 shutdown_connections 2024-09-10T08:37:58.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.714+0000 7f80971a0640 1 --2- 172.21.15.191:0/1941882488 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f807c038df0 0x7f807c03b2b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:58.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.714+0000 7f80971a0640 1 --2- 172.21.15.191:0/1941882488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090197640 0x7f809019b9d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:58.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.714+0000 7f80971a0640 1 --2- 172.21.15.191:0/1941882488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80901064f0 0x7f8090197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:37:58.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.714+0000 7f80971a0640 1 -- 172.21.15.191:0/1941882488 >> 172.21.15.191:0/1941882488 conn(0x7f80900784e0 msgr2=0x7f8090103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:37:58.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.714+0000 7f80971a0640 1 -- 172.21.15.191:0/1941882488 shutdown_connections 2024-09-10T08:37:58.715 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:37:58.714+0000 7f80971a0640 1 -- 172.21.15.191:0/1941882488 wait complete. 2024-09-10T08:37:58.849 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T08:37:58.850 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T08:37:58.850 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T08:37:58.850 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T08:37:58.850 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T08:37:58.850 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T08:37:59.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T08:37:59.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T08:37:59.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T08:37:59.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T08:37:59.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1484222841' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1484222841' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi191", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:37:59.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:37:59.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: Updating smithi191:/etc/ceph/ceph.conf 2024-09-10T08:37:59.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: Updating smithi191:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:37:59.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:37:59.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: Updating smithi191:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T08:37:59.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi172:172.21.15.172=a;smithi174:172.21.15.174=b;smithi191:172.21.15.191=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:37:59.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: Saving service mon spec with placement smithi172:172.21.15.172=a;smithi174:172.21.15.174=b;smithi191:172.21.15.191=c;count:3 2024-09-10T08:37:59.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T08:37:59.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:37:59.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:37:59.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:37:59.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: Reconfiguring daemon mon.a on smithi172 2024-09-10T08:37:59.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:59.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:37:59.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:37:59.246 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:37:59.246 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.246 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:37:59.246 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:37:59.246 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='client.? 172.21.15.191:0/1363935317' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:37:59.246 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: Deploying daemon mon.c on smithi191 2024-09-10T08:37:59.246 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:59.246 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='client.? 172.21.15.191:0/1266788502' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:37:59.246 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:37:59.246 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:59.246 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: mon.a calling monitor election 2024-09-10T08:37:59.246 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:59.246 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:59.246 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:59.247 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: mon.c calling monitor election 2024-09-10T08:37:59.247 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:59.247 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:59.247 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:59.247 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:59.247 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T08:37:59.247 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: monmap epoch 2 2024-09-10T08:37:59.247 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: fsid a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:37:59.247 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: last_changed 2024-09-10T08:37:50.096260+0000 2024-09-10T08:37:59.247 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: created 2024-09-10T08:36:29.338346+0000 2024-09-10T08:37:59.247 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: min_mon_release 19 (squid) 2024-09-10T08:37:59.247 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: election_strategy: 1 2024-09-10T08:37:59.247 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: 0: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.a 2024-09-10T08:37:59.247 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: 1: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.c 2024-09-10T08:37:59.248 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: fsmap 2024-09-10T08:37:59.248 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:37:59.248 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: mgrmap e13: x(active, since 38s) 2024-09-10T08:37:59.248 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: overall HEALTH_OK 2024-09-10T08:37:59.248 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.248 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.248 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:37:59.248 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:37:59.248 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:37:59.248 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:59.248 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: Deploying daemon mon.b on smithi174 2024-09-10T08:37:59.248 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='client.? 172.21.15.191:0/1378135791' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:37:59.248 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:37:59.249 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:37:59.249 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 ceph-mon[34068]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T08:37:59.249 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 bash[34050]: 1732bb4b34821853f713868f7395977a6c69b292829fa44eb1ef692812610e1f 2024-09-10T08:37:59.249 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:37:58 smithi174 systemd[1]: Started Ceph mon.b for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:38:00.304 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T08:38:00.304 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T08:38:00.563 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:38:03.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.968+0000 7f928eb75640 1 -- 172.21.15.191:0/1114556423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9278003530 msgr2=0x7f92780059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:03.969 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.968+0000 7f928eb75640 1 --2- 172.21.15.191:0/1114556423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9278003530 0x7f92780059c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f92881016c0 tx=0x7f927c02bce0 comp rx=0 tx=0).stop 2024-09-10T08:38:03.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.968+0000 7f928eb75640 1 -- 172.21.15.191:0/1114556423 shutdown_connections 2024-09-10T08:38:03.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.968+0000 7f928eb75640 1 --2- 172.21.15.191:0/1114556423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9278003530 0x7f92780059c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:03.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.968+0000 7f928eb75640 1 -- 172.21.15.191:0/1114556423 >> 172.21.15.191:0/1114556423 conn(0x7f92880fc280 msgr2=0x7f92880fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:03.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.968+0000 7f928eb75640 1 -- 172.21.15.191:0/1114556423 shutdown_connections 2024-09-10T08:38:03.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.968+0000 7f928eb75640 1 -- 172.21.15.191:0/1114556423 wait complete. 2024-09-10T08:38:03.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.969+0000 7f928eb75640 1 Processor -- start 2024-09-10T08:38:03.970 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.969+0000 7f928eb75640 1 -- start start 2024-09-10T08:38:03.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.969+0000 7f928eb75640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9278003530 0x7f92881950c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:03.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.969+0000 7f928eb75640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9288195600 0x7f9288199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:03.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.969+0000 7f928eb75640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9288199ed0 0x7f928819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:03.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.969+0000 7f928eb75640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9288107570 con 0x7f9288195600 2024-09-10T08:38:03.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.969+0000 7f928eb75640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f92881073f0 con 0x7f9278003530 2024-09-10T08:38:03.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.969+0000 7f928eb75640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f92881076f0 con 0x7f9288199ed0 2024-09-10T08:38:03.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f928d112640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9288199ed0 0x7f928819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:03.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f9287fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9288195600 0x7f9288199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:03.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f928d112640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9288199ed0 0x7f928819a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:43364/0 (socket says 172.21.15.191:43364) 2024-09-10T08:38:03.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f9287fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9288195600 0x7f9288199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.191:37398/0 (socket says 172.21.15.191:37398) 2024-09-10T08:38:03.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f928d112640 1 -- 172.21.15.191:0/3327323025 learned_addr learned my addr 172.21.15.191:0/3327323025 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-10T08:38:03.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f928c911640 1 --2- 172.21.15.191:0/3327323025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9278003530 0x7f92881950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:03.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f928d112640 1 -- 172.21.15.191:0/3327323025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9288199ed0 msgr2=0x7f928819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-10T08:38:03.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f928d112640 1 -- 172.21.15.191:0/3327323025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9288199ed0 msgr2=0x7f928819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T08:38:03.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f928d112640 1 --2- 172.21.15.191:0/3327323025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9288199ed0 0x7f928819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T08:38:03.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f928d112640 1 --2- 172.21.15.191:0/3327323025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9288199ed0 0x7f928819a380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T08:38:03.974 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f928c911640 1 -- 172.21.15.191:0/3327323025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9278003530 msgr2=0x7f92881950c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-10T08:38:03.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f928c911640 1 -- 172.21.15.191:0/3327323025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9278003530 msgr2=0x7f92881950c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T08:38:03.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f928c911640 1 --2- 172.21.15.191:0/3327323025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9278003530 0x7f92881950c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T08:38:03.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f928c911640 1 --2- 172.21.15.191:0/3327323025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9278003530 0x7f92881950c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T08:38:03.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f9287fff640 1 -- 172.21.15.191:0/3327323025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9288199ed0 msgr2=0x7f928819a380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:38:03.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f9287fff640 1 --2- 172.21.15.191:0/3327323025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9288199ed0 0x7f928819a380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:03.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f9287fff640 1 -- 172.21.15.191:0/3327323025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9278003530 msgr2=0x7f92881950c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:38:03.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f9287fff640 1 --2- 172.21.15.191:0/3327323025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9278003530 0x7f92881950c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:03.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f9287fff640 1 -- 172.21.15.191:0/3327323025 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92881a1c40 con 0x7f9288195600 2024-09-10T08:38:03.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.970+0000 7f9287fff640 1 --2- 172.21.15.191:0/3327323025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9288195600 0x7f9288199990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f927400b450 tx=0x7f927400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:03.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.971+0000 7f9285ffb640 1 -- 172.21.15.191:0/3327323025 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274004230 con 0x7f9288195600 2024-09-10T08:38:03.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.971+0000 7f9285ffb640 1 -- 172.21.15.191:0/3327323025 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92740043d0 con 0x7f9288195600 2024-09-10T08:38:03.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.971+0000 7f928eb75640 1 -- 172.21.15.191:0/3327323025 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92881a1e70 con 0x7f9288195600 2024-09-10T08:38:03.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.971+0000 7f9285ffb640 1 -- 172.21.15.191:0/3327323025 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274003be0 con 0x7f9288195600 2024-09-10T08:38:03.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.971+0000 7f928eb75640 1 -- 172.21.15.191:0/3327323025 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92881a23f0 con 0x7f9288195600 2024-09-10T08:38:03.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.972+0000 7f9285ffb640 1 -- 172.21.15.191:0/3327323025 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f927400f050 con 0x7f9288195600 2024-09-10T08:38:03.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.972+0000 7f9285ffb640 1 --2- 172.21.15.191:0/3327323025 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f926c03b700 0x7f926c03dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:03.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.972+0000 7f9285ffb640 1 -- 172.21.15.191:0/3327323025 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9274051060 con 0x7f9288195600 2024-09-10T08:38:03.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.972+0000 7f928c911640 1 --2- 172.21.15.191:0/3327323025 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f926c03b700 0x7f926c03dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:03.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.973+0000 7f928eb75640 1 -- 172.21.15.191:0/3327323025 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9288108500 con 0x7f9288195600 2024-09-10T08:38:03.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.973+0000 7f928c911640 1 --2- 172.21.15.191:0/3327323025 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f926c03b700 0x7f926c03dbc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f927c011c00 tx=0x7f927c005f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:03.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:03.978+0000 7f9285ffb640 1 -- 172.21.15.191:0/3327323025 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f927400d040 con 0x7f9288195600 2024-09-10T08:38:04.221 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:04.221+0000 7f928eb75640 1 -- 172.21.15.191:0/3327323025 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f928802d7c0 con 0x7f9288195600 2024-09-10T08:38:04.222 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:38:04.222 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: mon.a calling monitor election 2024-09-10T08:38:04.222 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: mon.c calling monitor election 2024-09-10T08:38:04.222 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:04.222 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:38:04.222 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:04.222 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:04.222 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:04.222 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:04.222 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:04.222 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:04.223 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:04.223 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:04.223 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T08:38:04.223 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: monmap epoch 3 2024-09-10T08:38:04.223 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: fsid a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:38:04.223 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: last_changed 2024-09-10T08:37:58.855033+0000 2024-09-10T08:38:04.223 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: created 2024-09-10T08:36:29.338346+0000 2024-09-10T08:38:04.223 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: min_mon_release 19 (squid) 2024-09-10T08:38:04.223 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: election_strategy: 1 2024-09-10T08:38:04.223 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: 0: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.a 2024-09-10T08:38:04.223 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: 1: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.c 2024-09-10T08:38:04.223 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: 2: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.b 2024-09-10T08:38:04.223 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: fsmap 2024-09-10T08:38:04.223 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:38:04.224 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: mgrmap e13: x(active, since 46s) 2024-09-10T08:38:04.224 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: overall HEALTH_OK 2024-09-10T08:38:04.224 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:04.224 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:04.224 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:04.224 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:04.224 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:04.221+0000 7f9285ffb640 1 -- 172.21.15.191:0/3327323025 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f927400e2c0 con 0x7f9288195600 2024-09-10T08:38:04.225 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:38:04.225 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":3,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","modified":"2024-09-10T08:37:58.855033Z","created":"2024-09-10T08:36:29.338346Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:3300","nonce":0},{"type":"v1","addr":"172.21.15.191:6789","nonce":0}]},"addr":"172.21.15.191:6789/0","public_addr":"172.21.15.191:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T08:38:04.226 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 3 2024-09-10T08:38:04.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:04.224+0000 7f928eb75640 1 -- 172.21.15.191:0/3327323025 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f926c03b700 msgr2=0x7f926c03dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:04.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:04.224+0000 7f928eb75640 1 --2- 172.21.15.191:0/3327323025 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f926c03b700 0x7f926c03dbc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f927c011c00 tx=0x7f927c005f10 comp rx=0 tx=0).stop 2024-09-10T08:38:04.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:04.224+0000 7f928eb75640 1 -- 172.21.15.191:0/3327323025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9288195600 msgr2=0x7f9288199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:04.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:04.224+0000 7f928eb75640 1 --2- 172.21.15.191:0/3327323025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9288195600 0x7f9288199990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f927400b450 tx=0x7f927400b910 comp rx=0 tx=0).stop 2024-09-10T08:38:04.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:04.224+0000 7f928eb75640 1 -- 172.21.15.191:0/3327323025 shutdown_connections 2024-09-10T08:38:04.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:04.224+0000 7f928eb75640 1 --2- 172.21.15.191:0/3327323025 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f926c03b700 0x7f926c03dbc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:04.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:04.224+0000 7f928eb75640 1 --2- 172.21.15.191:0/3327323025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9288199ed0 0x7f928819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:04.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:04.224+0000 7f928eb75640 1 --2- 172.21.15.191:0/3327323025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9288195600 0x7f9288199990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:04.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:04.224+0000 7f928eb75640 1 --2- 172.21.15.191:0/3327323025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9278003530 0x7f92881950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:04.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:04.224+0000 7f928eb75640 1 -- 172.21.15.191:0/3327323025 >> 172.21.15.191:0/3327323025 conn(0x7f92880fc280 msgr2=0x7f92881051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:04.226 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:04.224+0000 7f928eb75640 1 -- 172.21.15.191:0/3327323025 shutdown_connections 2024-09-10T08:38:04.227 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:04.224+0000 7f928eb75640 1 -- 172.21.15.191:0/3327323025 wait complete. 2024-09-10T08:38:04.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:38:04.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: mon.a calling monitor election 2024-09-10T08:38:04.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: mon.c calling monitor election 2024-09-10T08:38:04.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:04.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:38:04.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:04.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:04.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:04.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:04.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:04.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:04.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:04.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:04.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T08:38:04.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: monmap epoch 3 2024-09-10T08:38:04.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: fsid a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:38:04.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: last_changed 2024-09-10T08:37:58.855033+0000 2024-09-10T08:38:04.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: created 2024-09-10T08:36:29.338346+0000 2024-09-10T08:38:04.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: min_mon_release 19 (squid) 2024-09-10T08:38:04.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: election_strategy: 1 2024-09-10T08:38:04.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: 0: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.a 2024-09-10T08:38:04.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: 1: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.c 2024-09-10T08:38:04.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: 2: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.b 2024-09-10T08:38:04.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: fsmap 2024-09-10T08:38:04.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:38:04.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: mgrmap e13: x(active, since 46s) 2024-09-10T08:38:04.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: overall HEALTH_OK 2024-09-10T08:38:04.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:04.335 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:04.335 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:04.335 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:04.804 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T08:38:04.805 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T08:38:05.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:04 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:05.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:04 smithi191 ceph-mon[35314]: from='client.? 172.21.15.191:0/3327323025' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:38:05.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:04 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:05.062 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:05.184 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:04 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:05.184 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:04 smithi172 ceph-mon[30854]: from='client.? 172.21.15.191:0/3327323025' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:38:05.184 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:04 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:05.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.550+0000 7fdb1b5cf640 1 -- 172.21.15.172:0/1809964790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb14106650 msgr2=0x7fdb14106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:05.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.550+0000 7fdb1b5cf640 1 --2- 172.21.15.172:0/1809964790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb14106650 0x7fdb14106a30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009880 tx=0x7fdb1002a5d0 comp rx=0 tx=0).stop 2024-09-10T08:38:05.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.550+0000 7fdb1b5cf640 1 -- 172.21.15.172:0/1809964790 shutdown_connections 2024-09-10T08:38:05.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.550+0000 7fdb1b5cf640 1 --2- 172.21.15.172:0/1809964790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb14106650 0x7fdb14106a30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:05.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.550+0000 7fdb1b5cf640 1 -- 172.21.15.172:0/1809964790 >> 172.21.15.172:0/1809964790 conn(0x7fdb140fc280 msgr2=0x7fdb140fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:05.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.550+0000 7fdb1b5cf640 1 -- 172.21.15.172:0/1809964790 shutdown_connections 2024-09-10T08:38:05.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.550+0000 7fdb1b5cf640 1 -- 172.21.15.172:0/1809964790 wait complete. 2024-09-10T08:38:05.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.551+0000 7fdb1b5cf640 1 Processor -- start 2024-09-10T08:38:05.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.551+0000 7fdb1b5cf640 1 -- start start 2024-09-10T08:38:05.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.551+0000 7fdb1b5cf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb14106650 0x7fdb140fc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:05.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.551+0000 7fdb1b5cf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb140fcc70 0x7fdb140fd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:05.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.551+0000 7fdb1b5cf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb141a3540 0x7fdb141a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:05.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.551+0000 7fdb1b5cf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1410aae0 con 0x7fdb14106650 2024-09-10T08:38:05.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.551+0000 7fdb1b5cf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1410a960 con 0x7fdb140fcc70 2024-09-10T08:38:05.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.551+0000 7fdb1b5cf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1410ac60 con 0x7fdb141a3540 2024-09-10T08:38:05.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.551+0000 7fdb1936b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb14106650 0x7fdb140fc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.551+0000 7fdb1936b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb14106650 0x7fdb140fc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53942/0 (socket says 172.21.15.172:53942) 2024-09-10T08:38:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.552+0000 7fdb1936b640 1 -- 172.21.15.172:0/1045280077 learned_addr learned my addr 172.21.15.172:0/1045280077 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:38:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.552+0000 7fdb18b6a640 1 --2- 172.21.15.172:0/1045280077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb140fcc70 0x7fdb140fd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.552+0000 7fdb1936b640 1 -- 172.21.15.172:0/1045280077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb141a3540 msgr2=0x7fdb141a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.552+0000 7fdb1936b640 1 --2- 172.21.15.172:0/1045280077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb141a3540 0x7fdb141a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.552+0000 7fdb1936b640 1 -- 172.21.15.172:0/1045280077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb140fcc70 msgr2=0x7fdb140fd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.552+0000 7fdb1936b640 1 --2- 172.21.15.172:0/1045280077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb140fcc70 0x7fdb140fd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.552+0000 7fdb1936b640 1 -- 172.21.15.172:0/1045280077 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb141a6000 con 0x7fdb14106650 2024-09-10T08:38:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.552+0000 7fdb1936b640 1 --2- 172.21.15.172:0/1045280077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb14106650 0x7fdb140fc730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdb10031440 tx=0x7fdb100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.552+0000 7fdb0a7fc640 1 -- 172.21.15.172:0/1045280077 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1003f030 con 0x7fdb14106650 2024-09-10T08:38:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.552+0000 7fdb0a7fc640 1 -- 172.21.15.172:0/1045280077 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb10003cb0 con 0x7fdb14106650 2024-09-10T08:38:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.552+0000 7fdb1b5cf640 1 -- 172.21.15.172:0/1045280077 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb141a6290 con 0x7fdb14106650 2024-09-10T08:38:05.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.552+0000 7fdb0a7fc640 1 -- 172.21.15.172:0/1045280077 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10035070 con 0x7fdb14106650 2024-09-10T08:38:05.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.552+0000 7fdb1b5cf640 1 -- 172.21.15.172:0/1045280077 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb141a67c0 con 0x7fdb14106650 2024-09-10T08:38:05.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.553+0000 7fdb0a7fc640 1 -- 172.21.15.172:0/1045280077 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fdb10042030 con 0x7fdb14106650 2024-09-10T08:38:05.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.553+0000 7fdb1b5cf640 1 -- 172.21.15.172:0/1045280077 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb14106ab0 con 0x7fdb14106650 2024-09-10T08:38:05.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.554+0000 7fdb0a7fc640 1 --2- 172.21.15.172:0/1045280077 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb000394f0 0x7fdb0003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:05.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.554+0000 7fdb0a7fc640 1 -- 172.21.15.172:0/1045280077 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdb1006ccd0 con 0x7fdb14106650 2024-09-10T08:38:05.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.554+0000 7fdb18b6a640 1 --2- 172.21.15.172:0/1045280077 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb000394f0 0x7fdb0003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:05.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.554+0000 7fdb18b6a640 1 --2- 172.21.15.172:0/1045280077 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb000394f0 0x7fdb0003b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdb04006fd0 tx=0x7fdb04008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:05.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.558+0000 7fdb0a7fc640 1 -- 172.21.15.172:0/1045280077 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb10072020 con 0x7fdb14106650 2024-09-10T08:38:05.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.759+0000 7fdb1b5cf640 1 -- 172.21.15.172:0/1045280077 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fdb141a6ae0 con 0x7fdb14106650 2024-09-10T08:38:05.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.760+0000 7fdb0a7fc640 1 -- 172.21.15.172:0/1045280077 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7fdb1004bd70 con 0x7fdb14106650 2024-09-10T08:38:05.761 INFO:teuthology.orchestra.run.smithi172.stdout:# minimal ceph.conf for a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:38:05.761 INFO:teuthology.orchestra.run.smithi172.stdout:[global] 2024-09-10T08:38:05.761 INFO:teuthology.orchestra.run.smithi172.stdout: fsid = a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:38:05.761 INFO:teuthology.orchestra.run.smithi172.stdout: mon_host = [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] 2024-09-10T08:38:05.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.763+0000 7fdb1b5cf640 1 -- 172.21.15.172:0/1045280077 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb000394f0 msgr2=0x7fdb0003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:05.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.763+0000 7fdb1b5cf640 1 --2- 172.21.15.172:0/1045280077 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb000394f0 0x7fdb0003b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdb04006fd0 tx=0x7fdb04008040 comp rx=0 tx=0).stop 2024-09-10T08:38:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.763+0000 7fdb1b5cf640 1 -- 172.21.15.172:0/1045280077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb14106650 msgr2=0x7fdb140fc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.763+0000 7fdb1b5cf640 1 --2- 172.21.15.172:0/1045280077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb14106650 0x7fdb140fc730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdb10031440 tx=0x7fdb100042f0 comp rx=0 tx=0).stop 2024-09-10T08:38:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.763+0000 7fdb1b5cf640 1 -- 172.21.15.172:0/1045280077 shutdown_connections 2024-09-10T08:38:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.764+0000 7fdb1b5cf640 1 --2- 172.21.15.172:0/1045280077 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb000394f0 0x7fdb0003b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.764+0000 7fdb1b5cf640 1 --2- 172.21.15.172:0/1045280077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb141a3540 0x7fdb141a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.764+0000 7fdb1b5cf640 1 --2- 172.21.15.172:0/1045280077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb140fcc70 0x7fdb140fd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.764+0000 7fdb1b5cf640 1 --2- 172.21.15.172:0/1045280077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb14106650 0x7fdb140fc730 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.764+0000 7fdb1b5cf640 1 -- 172.21.15.172:0/1045280077 >> 172.21.15.172:0/1045280077 conn(0x7fdb140fc280 msgr2=0x7fdb14069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:05.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.764+0000 7fdb1b5cf640 1 -- 172.21.15.172:0/1045280077 shutdown_connections 2024-09-10T08:38:05.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:05.764+0000 7fdb1b5cf640 1 -- 172.21.15.172:0/1045280077 wait complete. 2024-09-10T08:38:06.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:38:06.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: mon.a calling monitor election 2024-09-10T08:38:06.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: mon.c calling monitor election 2024-09-10T08:38:06.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:06.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:38:06.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:06.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:06.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:06.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:06.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:06.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:06.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:06.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:06.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T08:38:06.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: monmap epoch 3 2024-09-10T08:38:06.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: fsid a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:38:06.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: last_changed 2024-09-10T08:37:58.855033+0000 2024-09-10T08:38:06.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: created 2024-09-10T08:36:29.338346+0000 2024-09-10T08:38:06.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: min_mon_release 19 (squid) 2024-09-10T08:38:06.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: election_strategy: 1 2024-09-10T08:38:06.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: 0: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.a 2024-09-10T08:38:06.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: 1: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.c 2024-09-10T08:38:06.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: 2: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.b 2024-09-10T08:38:06.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: fsmap 2024-09-10T08:38:06.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:38:06.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: mgrmap e13: x(active, since 46s) 2024-09-10T08:38:06.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: overall HEALTH_OK 2024-09-10T08:38:06.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.244 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:06.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: from='client.? 172.21.15.191:0/3327323025' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T08:38:06.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:06.599 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T08:38:06.600 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:38:06.600 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T08:38:06.629 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:38:06.629 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:38:06.695 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T08:38:06.695 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T08:38:06.724 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T08:38:06.724 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:38:06.790 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-10T08:38:06.790 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T08:38:06.816 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-10T08:38:06.816 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T08:38:06.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: mon.b calling monitor election 2024-09-10T08:38:06.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-10T08:38:06.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-10T08:38:06.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: Updating smithi191:/etc/ceph/ceph.conf 2024-09-10T08:38:06.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: mon.b calling monitor election 2024-09-10T08:38:06.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: mon.a calling monitor election 2024-09-10T08:38:06.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: mon.c calling monitor election 2024-09-10T08:38:06.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T08:38:06.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: monmap epoch 3 2024-09-10T08:38:06.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: fsid a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:38:06.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: last_changed 2024-09-10T08:37:58.855033+0000 2024-09-10T08:38:06.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: created 2024-09-10T08:36:29.338346+0000 2024-09-10T08:38:06.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: min_mon_release 19 (squid) 2024-09-10T08:38:06.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: election_strategy: 1 2024-09-10T08:38:06.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: 0: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.a 2024-09-10T08:38:06.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: 1: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.c 2024-09-10T08:38:06.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: 2: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.b 2024-09-10T08:38:06.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: fsmap 2024-09-10T08:38:06.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:38:06.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: mgrmap e13: x(active, since 48s) 2024-09-10T08:38:06.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: overall HEALTH_OK 2024-09-10T08:38:06.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: Updating smithi191:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:38:06.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: Updating smithi172:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:38:06.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: Updating smithi174:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:38:06.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.835 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:38:06.835 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:38:06.835 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:06.883 INFO:tasks.cephadm:Adding mgr.x on smithi172 2024-09-10T08:38:06.883 INFO:tasks.cephadm:Adding mgr.y on smithi174 2024-09-10T08:38:06.884 INFO:tasks.cephadm:Adding mgr.z on smithi191 2024-09-10T08:38:06.884 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi172=x;smithi174=y;smithi191=z' 2024-09-10T08:38:06.946 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: mon.b calling monitor election 2024-09-10T08:38:06.946 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-10T08:38:06.946 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-10T08:38:06.946 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: Updating smithi191:/etc/ceph/ceph.conf 2024-09-10T08:38:06.946 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: mon.b calling monitor election 2024-09-10T08:38:06.946 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: mon.a calling monitor election 2024-09-10T08:38:06.946 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: mon.c calling monitor election 2024-09-10T08:38:06.946 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T08:38:06.946 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: monmap epoch 3 2024-09-10T08:38:06.947 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: fsid a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:38:06.947 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: last_changed 2024-09-10T08:37:58.855033+0000 2024-09-10T08:38:06.947 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: created 2024-09-10T08:36:29.338346+0000 2024-09-10T08:38:06.947 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: min_mon_release 19 (squid) 2024-09-10T08:38:06.947 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: election_strategy: 1 2024-09-10T08:38:06.947 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: 0: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.a 2024-09-10T08:38:06.947 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: 1: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.c 2024-09-10T08:38:06.947 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: 2: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.b 2024-09-10T08:38:06.947 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: fsmap 2024-09-10T08:38:06.947 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:38:06.947 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: mgrmap e13: x(active, since 48s) 2024-09-10T08:38:06.947 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: overall HEALTH_OK 2024-09-10T08:38:06.947 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: Updating smithi191:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:38:06.948 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: Updating smithi172:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:38:06.948 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: Updating smithi174:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:38:06.948 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.948 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.948 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.948 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.948 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.948 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.948 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.948 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:38:06.948 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:38:06.948 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:06.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: mon.b calling monitor election 2024-09-10T08:38:06.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-10T08:38:06.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-10T08:38:06.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: Updating smithi191:/etc/ceph/ceph.conf 2024-09-10T08:38:06.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: mon.b calling monitor election 2024-09-10T08:38:06.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: mon.a calling monitor election 2024-09-10T08:38:06.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: mon.c calling monitor election 2024-09-10T08:38:06.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T08:38:06.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: monmap epoch 3 2024-09-10T08:38:06.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: fsid a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:38:06.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: last_changed 2024-09-10T08:37:58.855033+0000 2024-09-10T08:38:06.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: created 2024-09-10T08:36:29.338346+0000 2024-09-10T08:38:06.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: min_mon_release 19 (squid) 2024-09-10T08:38:06.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: election_strategy: 1 2024-09-10T08:38:06.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: 0: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.a 2024-09-10T08:38:06.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: 1: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.c 2024-09-10T08:38:06.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: 2: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.b 2024-09-10T08:38:06.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: fsmap 2024-09-10T08:38:06.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T08:38:06.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: mgrmap e13: x(active, since 48s) 2024-09-10T08:38:06.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: overall HEALTH_OK 2024-09-10T08:38:06.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: Updating smithi191:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:38:06.994 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: Updating smithi172:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:38:06.994 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: Updating smithi174:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T08:38:06.994 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.994 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.994 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.994 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.994 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.994 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.994 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:06.994 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:38:06.994 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:38:06.994 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:07.159 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:38:07.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.587+0000 7fecef951640 1 -- 172.21.15.191:0/131119088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece81016e0 msgr2=0x7fece8101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:07.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.587+0000 7fecef951640 1 --2- 172.21.15.191:0/131119088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece81016e0 0x7fece8101ae0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fece4009880 tx=0x7fece402a5d0 comp rx=0 tx=0).stop 2024-09-10T08:38:07.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.588+0000 7fecef951640 1 -- 172.21.15.191:0/131119088 shutdown_connections 2024-09-10T08:38:07.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.588+0000 7fecef951640 1 --2- 172.21.15.191:0/131119088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece81016e0 0x7fece8101ae0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:07.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.588+0000 7fecef951640 1 -- 172.21.15.191:0/131119088 >> 172.21.15.191:0/131119088 conn(0x7fece80fce90 msgr2=0x7fece80ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:07.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.588+0000 7fecef951640 1 -- 172.21.15.191:0/131119088 shutdown_connections 2024-09-10T08:38:07.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.588+0000 7fecef951640 1 -- 172.21.15.191:0/131119088 wait complete. 2024-09-10T08:38:07.590 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.588+0000 7fecef951640 1 Processor -- start 2024-09-10T08:38:07.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.588+0000 7fecef951640 1 -- start start 2024-09-10T08:38:07.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.588+0000 7fecef951640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fece81016e0 0x7fece8075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:07.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.588+0000 7fecef951640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fece80745e0 0x7fece8074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:07.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.588+0000 7fecef951640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece8074fa0 0x7fece81a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:07.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.588+0000 7fecef951640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fece8107680 con 0x7fece8074fa0 2024-09-10T08:38:07.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.588+0000 7fecef951640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fece8107500 con 0x7fece80745e0 2024-09-10T08:38:07.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.588+0000 7fecef951640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fece8107800 con 0x7fece81016e0 2024-09-10T08:38:07.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7feced6ed640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fece81016e0 0x7fece8075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:07.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7feced6ed640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fece81016e0 0x7fece8075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:58796/0 (socket says 172.21.15.191:58796) 2024-09-10T08:38:07.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7feced6ed640 1 -- 172.21.15.191:0/4200762150 learned_addr learned my addr 172.21.15.191:0/4200762150 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-10T08:38:07.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7fecedeee640 1 --2- 172.21.15.191:0/4200762150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece8074fa0 0x7fece81a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:07.591 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7fececeec640 1 --2- 172.21.15.191:0/4200762150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fece80745e0 0x7fece8074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:07.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7feced6ed640 1 -- 172.21.15.191:0/4200762150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fece80745e0 msgr2=0x7fece8074a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:07.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7feced6ed640 1 --2- 172.21.15.191:0/4200762150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fece80745e0 0x7fece8074a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:07.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7feced6ed640 1 -- 172.21.15.191:0/4200762150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece8074fa0 msgr2=0x7fece81a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:07.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7feced6ed640 1 --2- 172.21.15.191:0/4200762150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece8074fa0 0x7fece81a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:07.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7feced6ed640 1 -- 172.21.15.191:0/4200762150 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece81a1500 con 0x7fece81016e0 2024-09-10T08:38:07.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7fecedeee640 1 --2- 172.21.15.191:0/4200762150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece8074fa0 0x7fece81a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:38:07.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7fececeec640 1 --2- 172.21.15.191:0/4200762150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fece80745e0 0x7fece8074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:38:07.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7feced6ed640 1 --2- 172.21.15.191:0/4200762150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fece81016e0 0x7fece8075f90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fece4006010 tx=0x7fece40042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:07.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7fecde7fc640 1 -- 172.21.15.191:0/4200762150 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece403f030 con 0x7fece81016e0 2024-09-10T08:38:07.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7fecef951640 1 -- 172.21.15.191:0/4200762150 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece81a1790 con 0x7fece81016e0 2024-09-10T08:38:07.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7fecde7fc640 1 -- 172.21.15.191:0/4200762150 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fece4003cb0 con 0x7fece81016e0 2024-09-10T08:38:07.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7fecde7fc640 1 -- 172.21.15.191:0/4200762150 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece4035070 con 0x7fece81016e0 2024-09-10T08:38:07.593 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.589+0000 7fecef951640 1 -- 172.21.15.191:0/4200762150 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece81a1cf0 con 0x7fece81016e0 2024-09-10T08:38:07.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.590+0000 7fecde7fc640 1 -- 172.21.15.191:0/4200762150 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fece4042030 con 0x7fece81016e0 2024-09-10T08:38:07.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.590+0000 7fecde7fc640 1 --2- 172.21.15.191:0/4200762150 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fecd40394f0 0x7fecd403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:07.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.590+0000 7fecde7fc640 1 -- 172.21.15.191:0/4200762150 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fece406ce20 con 0x7fece81016e0 2024-09-10T08:38:07.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.591+0000 7fececeec640 1 --2- 172.21.15.191:0/4200762150 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fecd40394f0 0x7fecd403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:07.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.591+0000 7fecef951640 1 -- 172.21.15.191:0/4200762150 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece8190440 con 0x7fece81016e0 2024-09-10T08:38:07.594 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.591+0000 7fececeec640 1 --2- 172.21.15.191:0/4200762150 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fecd40394f0 0x7fecd403b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fecd8006fd0 tx=0x7fecd8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:07.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.596+0000 7fecde7fc640 1 -- 172.21.15.191:0/4200762150 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fece40303a0 con 0x7fece81016e0 2024-09-10T08:38:07.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.775+0000 7fecef951640 1 -- 172.21.15.191:0/4200762150 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi172=x;smithi174=y;smithi191=z", "target": ["mon-mgr", ""]}) -- 0x7fece8062630 con 0x7fecd40394f0 2024-09-10T08:38:07.780 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.779+0000 7fecde7fc640 1 -- 172.21.15.191:0/4200762150 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fece8062630 con 0x7fecd40394f0 2024-09-10T08:38:07.780 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled mgr update... 2024-09-10T08:38:07.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.782+0000 7fecef951640 1 -- 172.21.15.191:0/4200762150 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fecd40394f0 msgr2=0x7fecd403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:07.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.782+0000 7fecef951640 1 --2- 172.21.15.191:0/4200762150 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fecd40394f0 0x7fecd403b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fecd8006fd0 tx=0x7fecd8008040 comp rx=0 tx=0).stop 2024-09-10T08:38:07.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.782+0000 7fecef951640 1 -- 172.21.15.191:0/4200762150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fece81016e0 msgr2=0x7fece8075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:07.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.782+0000 7fecef951640 1 --2- 172.21.15.191:0/4200762150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fece81016e0 0x7fece8075f90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fece4006010 tx=0x7fece40042f0 comp rx=0 tx=0).stop 2024-09-10T08:38:07.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.782+0000 7fecef951640 1 -- 172.21.15.191:0/4200762150 shutdown_connections 2024-09-10T08:38:07.782 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.782+0000 7fecef951640 1 --2- 172.21.15.191:0/4200762150 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fecd40394f0 0x7fecd403b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:07.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.782+0000 7fecef951640 1 --2- 172.21.15.191:0/4200762150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece8074fa0 0x7fece81a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:07.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.782+0000 7fecef951640 1 --2- 172.21.15.191:0/4200762150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fece80745e0 0x7fece8074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:07.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.782+0000 7fecef951640 1 --2- 172.21.15.191:0/4200762150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fece81016e0 0x7fece8075f90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:07.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.782+0000 7fecef951640 1 -- 172.21.15.191:0/4200762150 >> 172.21.15.191:0/4200762150 conn(0x7fece80fce90 msgr2=0x7fece80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:07.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.782+0000 7fecef951640 1 -- 172.21.15.191:0/4200762150 shutdown_connections 2024-09-10T08:38:07.783 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:38:07.782+0000 7fecef951640 1 -- 172.21.15.191:0/4200762150 wait complete. 2024-09-10T08:38:07.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:07 smithi174 ceph-mon[34068]: Reconfiguring mon.a (monmap changed)... 2024-09-10T08:38:07.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:07 smithi174 ceph-mon[34068]: Reconfiguring daemon mon.a on smithi172 2024-09-10T08:38:07.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:07 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:07.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:07 smithi174 ceph-mon[34068]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:07.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:07 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:08.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:07 smithi191 ceph-mon[35314]: Reconfiguring mon.a (monmap changed)... 2024-09-10T08:38:08.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:07 smithi191 ceph-mon[35314]: Reconfiguring daemon mon.a on smithi172 2024-09-10T08:38:08.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:07 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:08.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:07 smithi191 ceph-mon[35314]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:08.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:07 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:08.081 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:07 smithi172 ceph-mon[30854]: Reconfiguring mon.a (monmap changed)... 2024-09-10T08:38:08.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:07 smithi172 ceph-mon[30854]: Reconfiguring daemon mon.a on smithi172 2024-09-10T08:38:08.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:07 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:38:08.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:07 smithi172 ceph-mon[30854]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:08.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:07 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:08.396 DEBUG:teuthology.orchestra.run.smithi174:mgr.y> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T08:38:08.398 DEBUG:teuthology.orchestra.run.smithi191:mgr.z> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mgr.z.service 2024-09-10T08:38:08.439 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T08:38:08.439 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:38:08.439 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T08:38:08.457 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T08:38:08.458 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_1 2024-09-10T08:38:08.516 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T08:38:08.516 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:38:08.516 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-10T08:38:08.516 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:38:08.516 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:38:08.516 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-10 08:37:25.721696221 +0000 2024-09-10T08:38:08.516 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-10 08:33:06.419389896 +0000 2024-09-10T08:38:08.516 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-10 08:33:06.419389896 +0000 2024-09-10T08:38:08.517 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-10 08:33:06.419389896 +0000 2024-09-10T08:38:08.517 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T08:38:08.582 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-10T08:38:08.582 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-10T08:38:08.583 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000283174 s, 1.8 MB/s 2024-09-10T08:38:08.584 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T08:38:08.642 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_2 2024-09-10T08:38:08.699 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T08:38:08.699 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:38:08.699 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-10T08:38:08.699 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:38:08.699 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:38:08.700 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-10 08:37:25.747695569 +0000 2024-09-10T08:38:08.700 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-10 08:33:06.750381602 +0000 2024-09-10T08:38:08.700 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-10 08:33:06.750381602 +0000 2024-09-10T08:38:08.700 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-10 08:33:06.750381602 +0000 2024-09-10T08:38:08.700 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T08:38:08.764 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-10T08:38:08.765 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-10T08:38:08.765 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000229729 s, 2.2 MB/s 2024-09-10T08:38:08.766 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T08:38:08.823 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_3 2024-09-10T08:38:08.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:08 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:08.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:08 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:08.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:08 smithi172 ceph-mon[30854]: Reconfiguring mon.b (monmap changed)... 2024-09-10T08:38:08.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:08 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:38:08.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:08 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:38:08.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:08 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:08.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:08 smithi172 ceph-mon[30854]: Reconfiguring daemon mon.b on smithi174 2024-09-10T08:38:08.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:08 smithi172 ceph-mon[30854]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi172=x;smithi174=y;smithi191=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:38:08.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:08 smithi172 ceph-mon[30854]: Saving service mgr spec with placement smithi172=x;smithi174=y;smithi191=z;count:3 2024-09-10T08:38:08.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:08 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:08.851 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T08:38:08.852 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:38:08.852 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-10T08:38:08.852 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:38:08.852 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:38:08.852 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-10 08:37:25.772694943 +0000 2024-09-10T08:38:08.852 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-10 08:33:07.086373183 +0000 2024-09-10T08:38:08.852 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-10 08:33:07.086373183 +0000 2024-09-10T08:38:08.852 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-10 08:33:07.086373183 +0000 2024-09-10T08:38:08.852 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T08:38:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-10T08:38:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-10T08:38:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000199553 s, 2.6 MB/s 2024-09-10T08:38:08.918 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T08:38:08.974 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_4 2024-09-10T08:38:08.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:08 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:08.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:08 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:08.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:08 smithi174 ceph-mon[34068]: Reconfiguring mon.b (monmap changed)... 2024-09-10T08:38:08.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:08 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:38:08.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:08 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:38:08.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:08 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:08.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:08 smithi174 ceph-mon[34068]: Reconfiguring daemon mon.b on smithi174 2024-09-10T08:38:08.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:08 smithi174 ceph-mon[34068]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi172=x;smithi174=y;smithi191=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:38:08.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:08 smithi174 ceph-mon[34068]: Saving service mgr spec with placement smithi172=x;smithi174=y;smithi191=z;count:3 2024-09-10T08:38:08.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:08 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:09.031 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T08:38:09.031 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:38:09.031 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-10T08:38:09.032 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:38:09.032 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:38:09.032 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-10 08:37:25.800694240 +0000 2024-09-10T08:38:09.032 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-10 08:33:07.420364813 +0000 2024-09-10T08:38:09.032 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-10 08:33:07.420364813 +0000 2024-09-10T08:38:09.032 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-10 08:33:07.420364813 +0000 2024-09-10T08:38:09.032 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T08:38:09.049 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:08 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:09.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:08 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:09.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:08 smithi191 ceph-mon[35314]: Reconfiguring mon.b (monmap changed)... 2024-09-10T08:38:09.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:08 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:38:09.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:08 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:38:09.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:08 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:09.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:08 smithi191 ceph-mon[35314]: Reconfiguring daemon mon.b on smithi174 2024-09-10T08:38:09.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:08 smithi191 ceph-mon[35314]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi172=x;smithi174=y;smithi191=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:38:09.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:08 smithi191 ceph-mon[35314]: Saving service mgr spec with placement smithi172=x;smithi174=y;smithi191=z;count:3 2024-09-10T08:38:09.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:08 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:09.096 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-10T08:38:09.096 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-10T08:38:09.096 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.00027502 s, 1.9 MB/s 2024-09-10T08:38:09.097 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T08:38:09.154 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T08:38:09.154 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T08:38:09.172 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T08:38:09.172 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_1 2024-09-10T08:38:09.228 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T08:38:09.229 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:38:09.229 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-10T08:38:09.229 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:38:09.229 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:38:09.229 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-10 08:37:58.305481299 +0000 2024-09-10T08:38:09.229 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-10 08:33:08.649619960 +0000 2024-09-10T08:38:09.229 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-10 08:33:08.649619960 +0000 2024-09-10T08:38:09.229 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-10 08:33:08.649619960 +0000 2024-09-10T08:38:09.229 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T08:38:09.294 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-10T08:38:09.295 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-10T08:38:09.295 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000281101 s, 1.8 MB/s 2024-09-10T08:38:09.296 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T08:38:09.353 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_2 2024-09-10T08:38:09.408 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T08:38:09.409 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:38:09.409 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-10T08:38:09.409 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:38:09.409 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:38:09.409 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-10 08:37:58.305481299 +0000 2024-09-10T08:38:09.409 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-10 08:33:08.963615218 +0000 2024-09-10T08:38:09.409 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-10 08:33:08.963615218 +0000 2024-09-10T08:38:09.409 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-10 08:33:08.963615218 +0000 2024-09-10T08:38:09.409 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T08:38:09.472 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-10T08:38:09.473 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-10T08:38:09.473 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000219317 s, 2.3 MB/s 2024-09-10T08:38:09.474 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T08:38:09.530 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_3 2024-09-10T08:38:09.585 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T08:38:09.586 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:38:09.586 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-10T08:38:09.586 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:38:09.586 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:38:09.586 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-10 08:37:58.305481299 +0000 2024-09-10T08:38:09.586 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-10 08:33:09.285610355 +0000 2024-09-10T08:38:09.586 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-10 08:33:09.285610355 +0000 2024-09-10T08:38:09.586 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-10 08:33:09.285610355 +0000 2024-09-10T08:38:09.586 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T08:38:09.649 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-10T08:38:09.649 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-10T08:38:09.649 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000313456 s, 1.6 MB/s 2024-09-10T08:38:09.650 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T08:38:09.707 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_4 2024-09-10T08:38:09.762 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T08:38:09.762 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:38:09.762 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-10T08:38:09.762 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:38:09.763 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:38:09.763 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-10 08:37:58.305481299 +0000 2024-09-10T08:38:09.763 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-10 08:33:09.598605628 +0000 2024-09-10T08:38:09.763 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-10 08:33:09.598605628 +0000 2024-09-10T08:38:09.763 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-10 08:33:09.598605628 +0000 2024-09-10T08:38:09.763 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T08:38:09.826 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-10T08:38:09.826 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-10T08:38:09.826 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000198588 s, 2.6 MB/s 2024-09-10T08:38:09.828 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T08:38:09.884 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-10T08:38:09.884 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T08:38:09.901 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T08:38:09.902 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_1 2024-09-10T08:38:09.957 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T08:38:09.957 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:38:09.958 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-10T08:38:09.958 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:38:09.958 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:38:09.958 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-09-10 08:37:49.174645184 +0000 2024-09-10T08:38:09.958 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-09-10 08:32:59.088179378 +0000 2024-09-10T08:38:09.958 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-09-10 08:32:59.088179378 +0000 2024-09-10T08:38:09.958 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-09-10 08:32:59.088179378 +0000 2024-09-10T08:38:09.958 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T08:38:10.022 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-09-10T08:38:10.023 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-09-10T08:38:10.023 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000237259 s, 2.2 MB/s 2024-09-10T08:38:10.024 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T08:38:10.080 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_2 2024-09-10T08:38:10.136 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T08:38:10.136 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:38:10.136 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-10T08:38:10.136 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:38:10.136 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:38:10.136 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-09-10 08:37:49.175645166 +0000 2024-09-10T08:38:10.136 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-09-10 08:32:59.408173324 +0000 2024-09-10T08:38:10.136 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-09-10 08:32:59.408173324 +0000 2024-09-10T08:38:10.136 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-09-10 08:32:59.408173324 +0000 2024-09-10T08:38:10.137 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T08:38:10.200 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-09-10T08:38:10.200 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-09-10T08:38:10.200 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000193256 s, 2.6 MB/s 2024-09-10T08:38:10.201 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T08:38:10.257 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_3 2024-09-10T08:38:10.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:10 smithi191 ceph-mon[35314]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:10.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:10 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:10.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:10 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:10.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:10 smithi191 ceph-mon[35314]: Reconfiguring mon.c (monmap changed)... 2024-09-10T08:38:10.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:10 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:38:10.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:10 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:38:10.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:10 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:10.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:10 smithi191 ceph-mon[35314]: Reconfiguring daemon mon.c on smithi191 2024-09-10T08:38:10.312 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T08:38:10.312 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:38:10.312 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-10T08:38:10.313 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:38:10.313 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:38:10.313 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-09-10 08:37:49.175645166 +0000 2024-09-10T08:38:10.313 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-09-10 08:32:59.745166948 +0000 2024-09-10T08:38:10.313 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-09-10 08:32:59.745166948 +0000 2024-09-10T08:38:10.313 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-09-10 08:32:59.745166948 +0000 2024-09-10T08:38:10.313 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T08:38:10.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:10 smithi172 ceph-mon[30854]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:10.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:10 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:10.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:10 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:10.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:10 smithi172 ceph-mon[30854]: Reconfiguring mon.c (monmap changed)... 2024-09-10T08:38:10.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:10 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:38:10.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:10 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:38:10.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:10 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:10.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:10 smithi172 ceph-mon[30854]: Reconfiguring daemon mon.c on smithi191 2024-09-10T08:38:10.376 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-09-10T08:38:10.376 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-09-10T08:38:10.376 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000201301 s, 2.5 MB/s 2024-09-10T08:38:10.378 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T08:38:10.434 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_4 2024-09-10T08:38:10.491 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T08:38:10.491 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T08:38:10.491 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-10T08:38:10.492 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:38:10.492 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T08:38:10.492 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-09-10 08:37:49.175645166 +0000 2024-09-10T08:38:10.492 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-09-10 08:33:00.070160800 +0000 2024-09-10T08:38:10.492 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-09-10 08:33:00.070160800 +0000 2024-09-10T08:38:10.492 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-09-10 08:33:00.070160800 +0000 2024-09-10T08:38:10.492 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T08:38:10.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:10 smithi174 ceph-mon[34068]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:10.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:10 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:10.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:10 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:10.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:10 smithi174 ceph-mon[34068]: Reconfiguring mon.c (monmap changed)... 2024-09-10T08:38:10.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:10 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T08:38:10.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:10 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T08:38:10.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:10 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:10.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:10 smithi174 ceph-mon[34068]: Reconfiguring daemon mon.c on smithi191 2024-09-10T08:38:10.557 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-09-10T08:38:10.557 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-09-10T08:38:10.557 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000189148 s, 2.7 MB/s 2024-09-10T08:38:10.558 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T08:38:10.623 INFO:tasks.cephadm:Deploying osd.0 on smithi172 with /dev/vg_nvme/lv_4... 2024-09-10T08:38:10.623 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T08:38:10.892 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:11.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:11.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:11.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:11.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:11.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:38:11.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:11.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:38:11.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T08:38:11.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:38:11.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:11.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:11 smithi172 ceph-mon[30854]: Deploying daemon mgr.z on smithi191 2024-09-10T08:38:11.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:11 smithi172 ceph-mon[30854]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:11.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:38:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:38:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T08:38:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:38:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:11 smithi174 ceph-mon[34068]: Deploying daemon mgr.z on smithi191 2024-09-10T08:38:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:11 smithi174 ceph-mon[34068]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:11.781 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:11.781 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:11.781 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:11.781 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:11.781 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:38:11.781 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:11.781 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:38:11.781 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T08:38:11.781 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:38:11.782 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:11.782 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:11 smithi191 ceph-mon[35314]: Deploying daemon mgr.z on smithi191 2024-09-10T08:38:11.782 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:11 smithi191 ceph-mon[35314]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:12.712 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:12 smithi191 systemd[1]: Started Ceph mgr.z for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:38:12.988 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:12 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:12.711+0000 7f43de388040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T08:38:12.988 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:12 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:12.774+0000 7f43de388040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T08:38:12.988 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:12 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:12.922+0000 7f43de388040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T08:38:12.988 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:12 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:12.988+0000 7f43de388040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T08:38:13.274 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:13 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:13.134+0000 7f43de388040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:38:13.274 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:13 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:13.274+0000 7f43de388040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T08:38:13.391 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:38:13.416 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_4 2024-09-10T08:38:13.658 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:38:13.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:13.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:13.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:13.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:38:13.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T08:38:13.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:38:13.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:13.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:13 smithi174 ceph-mon[34068]: Deploying daemon mgr.y on smithi174 2024-09-10T08:38:13.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:13 smithi174 ceph-mon[34068]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:14.007 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:14.007 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:14.007 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:14.007 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:38:14.007 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T08:38:14.007 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:38:14.007 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:14.008 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:13 smithi172 ceph-mon[30854]: Deploying daemon mgr.y on smithi174 2024-09-10T08:38:14.008 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:13 smithi172 ceph-mon[30854]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:14.050 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:13 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:13.585+0000 7f43de388040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T08:38:14.050 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:13 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:13.646+0000 7f43de388040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T08:38:14.050 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:13 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:13.833+0000 7f43de388040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T08:38:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:38:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T08:38:14.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:38:14.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:14.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:13 smithi191 ceph-mon[35314]: Deploying daemon mgr.y on smithi174 2024-09-10T08:38:14.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:13 smithi191 ceph-mon[35314]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:14.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.090+0000 7efdf780a640 1 -- 172.21.15.172:0/1470385675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf0102690 msgr2=0x7efdf0108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:14.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.090+0000 7efdf780a640 1 --2- 172.21.15.172:0/1470385675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf0102690 0x7efdf0108f20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efde8009ca0 tx=0x7efde802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:38:14.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.090+0000 7efdf780a640 1 -- 172.21.15.172:0/1470385675 shutdown_connections 2024-09-10T08:38:14.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.090+0000 7efdf780a640 1 --2- 172.21.15.172:0/1470385675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf0102690 0x7efdf0108f20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:14.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.090+0000 7efdf780a640 1 --2- 172.21.15.172:0/1470385675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf0101cd0 0x7efdf0102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:14.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.090+0000 7efdf780a640 1 --2- 172.21.15.172:0/1470385675 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efdf0100ad0 0x7efdf0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:14.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.090+0000 7efdf780a640 1 -- 172.21.15.172:0/1470385675 >> 172.21.15.172:0/1470385675 conn(0x7efdf00fc280 msgr2=0x7efdf00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:14.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.090+0000 7efdf780a640 1 -- 172.21.15.172:0/1470385675 shutdown_connections 2024-09-10T08:38:14.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.090+0000 7efdf780a640 1 -- 172.21.15.172:0/1470385675 wait complete. 2024-09-10T08:38:14.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf780a640 1 Processor -- start 2024-09-10T08:38:14.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf780a640 1 -- start start 2024-09-10T08:38:14.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf780a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efdf0100ad0 0x7efdf0197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf780a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf0101cd0 0x7efdf0198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf780a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf0102690 0x7efdf019f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf780a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efdf010c600 con 0x7efdf0101cd0 2024-09-10T08:38:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf780a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efdf010c480 con 0x7efdf0102690 2024-09-10T08:38:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf780a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efdf010c780 con 0x7efdf0100ad0 2024-09-10T08:38:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf4da5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf0101cd0 0x7efdf0198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf4da5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf0101cd0 0x7efdf0198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58348/0 (socket says 172.21.15.172:58348) 2024-09-10T08:38:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf4da5640 1 -- 172.21.15.172:0/854006734 learned_addr learned my addr 172.21.15.172:0/854006734 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:38:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf55a6640 1 --2- 172.21.15.172:0/854006734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efdf0100ad0 0x7efdf0197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf5da7640 1 --2- 172.21.15.172:0/854006734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf0102690 0x7efdf019f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf4da5640 1 -- 172.21.15.172:0/854006734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efdf0100ad0 msgr2=0x7efdf0197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf4da5640 1 --2- 172.21.15.172:0/854006734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efdf0100ad0 0x7efdf0197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf4da5640 1 -- 172.21.15.172:0/854006734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf0102690 msgr2=0x7efdf019f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf4da5640 1 --2- 172.21.15.172:0/854006734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf0102690 0x7efdf019f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf4da5640 1 -- 172.21.15.172:0/854006734 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf019fa00 con 0x7efdf0101cd0 2024-09-10T08:38:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efdf4da5640 1 --2- 172.21.15.172:0/854006734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf0101cd0 0x7efdf0198310 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efde000d950 tx=0x7efde000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efde67fc640 1 -- 172.21.15.172:0/854006734 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0017030 con 0x7efdf0101cd0 2024-09-10T08:38:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.091+0000 7efde67fc640 1 -- 172.21.15.172:0/854006734 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efde0004020 con 0x7efdf0101cd0 2024-09-10T08:38:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.092+0000 7efdf780a640 1 -- 172.21.15.172:0/854006734 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf019fcf0 con 0x7efdf0101cd0 2024-09-10T08:38:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.092+0000 7efde67fc640 1 -- 172.21.15.172:0/854006734 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde00047a0 con 0x7efdf0101cd0 2024-09-10T08:38:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.092+0000 7efdf780a640 1 -- 172.21.15.172:0/854006734 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf01a0220 con 0x7efdf0101cd0 2024-09-10T08:38:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.093+0000 7efde67fc640 1 -- 172.21.15.172:0/854006734 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7efde0004940 con 0x7efdf0101cd0 2024-09-10T08:38:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.093+0000 7efdf780a640 1 -- 172.21.15.172:0/854006734 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf01a04b0 con 0x7efdf0101cd0 2024-09-10T08:38:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.093+0000 7efde67fc640 1 --2- 172.21.15.172:0/854006734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efddc0392c0 0x7efddc03b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.093+0000 7efde67fc640 1 -- 172.21.15.172:0/854006734 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efde004be30 con 0x7efdf0101cd0 2024-09-10T08:38:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.093+0000 7efdf55a6640 1 --2- 172.21.15.172:0/854006734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efddc0392c0 0x7efddc03b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.093+0000 7efdf55a6640 1 --2- 172.21.15.172:0/854006734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efddc0392c0 0x7efddc03b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efdf0101b30 tx=0x7efdec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.098+0000 7efde67fc640 1 -- 172.21.15.172:0/854006734 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efde0004d30 con 0x7efdf0101cd0 2024-09-10T08:38:14.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:14.272+0000 7efdf780a640 1 -- 172.21.15.172:0/854006734 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7efdf006a730 con 0x7efddc0392c0 2024-09-10T08:38:14.684 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:14 smithi174 systemd[1]: Starting Ceph mgr.y for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T08:38:14.935 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:14 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:38:14.935 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:14 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:38:14.935 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:14 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:14.936 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:14 smithi174 podman[35422]: 2024-09-10 08:38:14.587014645 +0000 UTC m=+0.015992799 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:38:14.936 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:14 smithi174 podman[35422]: 2024-09-10 08:38:14.700506593 +0000 UTC m=+0.129484771 container create 6ac7282b613dfc4170521b231a804e35c66bff364b7c0182b896f6552fe3b273 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T08:38:14.936 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:14 smithi174 podman[35422]: 2024-09-10 08:38:14.93587394 +0000 UTC m=+0.364852075 container init 6ac7282b613dfc4170521b231a804e35c66bff364b7c0182b896f6552fe3b273 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:38:14.952 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:14 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:38:14.952 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:14 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:38:14.952 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:14 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:15.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:14 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:38:15.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:14 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:38:15.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:14 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:15.050 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:14 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:14.716+0000 7f43de388040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T08:38:15.050 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:14 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:14.785+0000 7f43de388040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T08:38:15.241 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:14 smithi174 podman[35422]: 2024-09-10 08:38:14.938620425 +0000 UTC m=+0.367598560 container start 6ac7282b613dfc4170521b231a804e35c66bff364b7c0182b896f6552fe3b273 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:38:15.242 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:14 smithi174 ceph-mgr[35441]: -- 172.21.15.174:0/1010202604 <== mon.2 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b00d510e0 con 0x555b00d2f400 2024-09-10T08:38:15.242 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:15 smithi174 bash[35422]: 6ac7282b613dfc4170521b231a804e35c66bff364b7c0182b896f6552fe3b273 2024-09-10T08:38:15.242 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:15 smithi174 systemd[1]: Started Ceph mgr.y for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:38:15.242 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:15 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:15.060+0000 7fe57ed94040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T08:38:15.242 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:15 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:15.124+0000 7fe57ed94040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T08:38:15.669 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:15 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:15.279+0000 7fe57ed94040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T08:38:15.669 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:15 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:15.349+0000 7fe57ed94040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T08:38:15.669 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:15 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:15.509+0000 7fe57ed94040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:38:15.669 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:15 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:15.668+0000 7fe57ed94040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T08:38:15.800 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:15 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:15.532+0000 7f43de388040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T08:38:15.800 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:15 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:15.657+0000 7f43de388040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T08:38:16.096 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:15 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:15.800+0000 7f43de388040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:38:16.096 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:15 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:15.866+0000 7f43de388040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T08:38:16.096 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:16 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:16.025+0000 7f43de388040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T08:38:16.096 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:16 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:16.095+0000 7f43de388040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T08:38:16.241 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:15 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:15.991+0000 7fe57ed94040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T08:38:16.241 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:16 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:16.052+0000 7fe57ed94040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T08:38:16.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:16 smithi174 ceph-mon[34068]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:38:16.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:16 smithi174 ceph-mon[34068]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:16.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:16 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:16.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:16 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:16.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:16 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:16.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:16 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:16.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:16 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:16.252 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:16 smithi172 ceph-mon[30854]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:38:16.253 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:16 smithi172 ceph-mon[30854]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:16.253 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:16 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:16.253 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:16 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:16.253 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:16 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:16.253 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:16 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:16.253 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:16 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:16.400 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:16 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:16.159+0000 7f43de388040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T08:38:16.400 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:16 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:16.329+0000 7f43de388040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T08:38:16.400 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:16 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:16.400+0000 7f43de388040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T08:38:16.400 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:16 smithi191 ceph-mon[35314]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:38:16.400 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:16 smithi191 ceph-mon[35314]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:16.401 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:16 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:16.401 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:16 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:16.401 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:16 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:16.401 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:16 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:16.401 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:16 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:16.491 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:16 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:16.240+0000 7fe57ed94040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T08:38:16.672 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:16 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:16.470+0000 7f43de388040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T08:38:16.672 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:16 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:16.552+0000 7f43de388040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T08:38:16.672 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:16 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:16.612+0000 7f43de388040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T08:38:16.672 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:16 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:16.671+0000 7f43de388040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T08:38:17.491 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:17.134+0000 7fe57ed94040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T08:38:17.491 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:17.200+0000 7fe57ed94040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T08:38:17.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:17.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:38:17.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T08:38:17.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:38:17.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:38:17.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:17.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: Reconfiguring daemon mgr.x on smithi172 2024-09-10T08:38:17.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3083400192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12"}]: dispatch 2024-09-10T08:38:17.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3083400192' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12"}]': finished 2024-09-10T08:38:17.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T08:38:17.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:38:17.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:17.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/580170924' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:38:17.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:17.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:38:17.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T08:38:17.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:38:17.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:38:17.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:17.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: Reconfiguring daemon mgr.x on smithi172 2024-09-10T08:38:17.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3083400192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12"}]: dispatch 2024-09-10T08:38:17.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3083400192' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12"}]': finished 2024-09-10T08:38:17.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T08:38:17.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:38:17.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:17.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:17 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/580170924' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:38:17.800 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z[37118]: 2024-09-10T08:38:17.468+0000 7f43de388040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T08:38:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:17.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:38:17.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T08:38:17.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T08:38:17.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T08:38:17.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:17.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: Reconfiguring daemon mgr.x on smithi172 2024-09-10T08:38:17.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3083400192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12"}]: dispatch 2024-09-10T08:38:17.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3083400192' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12"}]': finished 2024-09-10T08:38:17.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T08:38:17.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:38:17.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:17.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:17.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:17 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/580170924' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:38:18.169 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:17 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:17.913+0000 7fe57ed94040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T08:38:18.169 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:18 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:18.027+0000 7fe57ed94040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T08:38:18.169 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:18 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:18.168+0000 7fe57ed94040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T08:38:18.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:18.314+0000 7efde67fc640 1 -- 172.21.15.172:0/854006734 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7efde00035f0 con 0x7efdf0101cd0 2024-09-10T08:38:18.442 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:18 smithi174 ceph-mon[34068]: Standby manager daemon z started 2024-09-10T08:38:18.442 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:18 smithi174 ceph-mon[34068]: from='mgr.? 172.21.15.191:0/3111293871' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T08:38:18.443 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:18 smithi174 ceph-mon[34068]: from='mgr.? 172.21.15.191:0/3111293871' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T08:38:18.443 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:18 smithi174 ceph-mon[34068]: from='mgr.? 172.21.15.191:0/3111293871' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T08:38:18.443 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:18 smithi174 ceph-mon[34068]: from='mgr.? 172.21.15.191:0/3111293871' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T08:38:18.443 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:18 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:18.228+0000 7fe57ed94040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T08:38:18.443 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:18 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:18.376+0000 7fe57ed94040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T08:38:18.443 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:18 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:18.442+0000 7fe57ed94040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T08:38:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:18 smithi172 ceph-mon[30854]: Standby manager daemon z started 2024-09-10T08:38:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:18 smithi172 ceph-mon[30854]: from='mgr.? 172.21.15.191:0/3111293871' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T08:38:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:18 smithi172 ceph-mon[30854]: from='mgr.? 172.21.15.191:0/3111293871' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T08:38:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:18 smithi172 ceph-mon[30854]: from='mgr.? 172.21.15.191:0/3111293871' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T08:38:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:18 smithi172 ceph-mon[30854]: from='mgr.? 172.21.15.191:0/3111293871' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T08:38:18.719 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:18 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:18.500+0000 7fe57ed94040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T08:38:18.720 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:18 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:18.655+0000 7fe57ed94040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T08:38:18.720 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:18 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:18.719+0000 7fe57ed94040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T08:38:18.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:18 smithi191 ceph-mon[35314]: Standby manager daemon z started 2024-09-10T08:38:18.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:18 smithi191 ceph-mon[35314]: from='mgr.? 172.21.15.191:0/3111293871' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T08:38:18.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:18 smithi191 ceph-mon[35314]: from='mgr.? 172.21.15.191:0/3111293871' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T08:38:18.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:18 smithi191 ceph-mon[35314]: from='mgr.? 172.21.15.191:0/3111293871' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T08:38:18.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:18 smithi191 ceph-mon[35314]: from='mgr.? 172.21.15.191:0/3111293871' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T08:38:18.982 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:18 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:18.784+0000 7fe57ed94040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T08:38:18.982 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:18 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:18.865+0000 7fe57ed94040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T08:38:18.982 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:18 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:18.923+0000 7fe57ed94040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T08:38:18.983 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:18 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:18.981+0000 7fe57ed94040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T08:38:19.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:19 smithi172 ceph-mon[30854]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-10T08:38:19.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T08:38:19.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:19 smithi172 ceph-mon[30854]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:19.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:19 smithi174 ceph-mon[34068]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-10T08:38:19.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T08:38:19.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:19 smithi174 ceph-mon[34068]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:19.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:19 smithi191 ceph-mon[35314]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-10T08:38:19.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T08:38:19.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:19 smithi191 ceph-mon[35314]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:20.241 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 08:38:19 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y[35437]: 2024-09-10T08:38:19.757+0000 7fe57ed94040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T08:38:20.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:20.326+0000 7efde67fc640 1 -- 172.21.15.172:0/854006734 <== mon.0 v2:172.21.15.172:3300/0 8 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7efde0009a30 con 0x7efdf0101cd0 2024-09-10T08:38:20.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:20 smithi172 ceph-mon[30854]: Standby manager daemon y started 2024-09-10T08:38:20.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:20 smithi172 ceph-mon[30854]: from='mgr.? 172.21.15.174:0/1072038981' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T08:38:20.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:20 smithi172 ceph-mon[30854]: from='mgr.? 172.21.15.174:0/1072038981' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T08:38:20.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:20 smithi172 ceph-mon[30854]: from='mgr.? 172.21.15.174:0/1072038981' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T08:38:20.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:20 smithi172 ceph-mon[30854]: from='mgr.? 172.21.15.174:0/1072038981' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T08:38:20.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:20 smithi174 ceph-mon[34068]: Standby manager daemon y started 2024-09-10T08:38:20.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:20 smithi174 ceph-mon[34068]: from='mgr.? 172.21.15.174:0/1072038981' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T08:38:20.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:20 smithi174 ceph-mon[34068]: from='mgr.? 172.21.15.174:0/1072038981' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T08:38:20.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:20 smithi174 ceph-mon[34068]: from='mgr.? 172.21.15.174:0/1072038981' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T08:38:20.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:20 smithi174 ceph-mon[34068]: from='mgr.? 172.21.15.174:0/1072038981' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T08:38:20.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:20 smithi191 ceph-mon[35314]: Standby manager daemon y started 2024-09-10T08:38:20.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:20 smithi191 ceph-mon[35314]: from='mgr.? 172.21.15.174:0/1072038981' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T08:38:20.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:20 smithi191 ceph-mon[35314]: from='mgr.? 172.21.15.174:0/1072038981' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T08:38:20.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:20 smithi191 ceph-mon[35314]: from='mgr.? 172.21.15.174:0/1072038981' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T08:38:20.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:20 smithi191 ceph-mon[35314]: from='mgr.? 172.21.15.174:0/1072038981' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T08:38:21.506 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:21 smithi172 ceph-mon[30854]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-10T08:38:21.506 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:21 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T08:38:21.506 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:21 smithi172 ceph-mon[30854]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:21.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:21 smithi174 ceph-mon[34068]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-10T08:38:21.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:21 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T08:38:21.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:21 smithi174 ceph-mon[34068]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:21.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:21 smithi191 ceph-mon[35314]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-10T08:38:21.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:21 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T08:38:21.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:21 smithi191 ceph-mon[35314]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:24.476 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:24 smithi172 ceph-mon[30854]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:24.476 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:24 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:24.476 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:24 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:24.476 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:24 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:24.476 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:24 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:24.477 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:24 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:38:24.477 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:24 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:24.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:24 smithi174 ceph-mon[34068]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:24.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:24 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:24.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:24 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:24.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:24 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:24.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:24 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:24.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:24 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:38:24.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:24 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:24 smithi191 ceph-mon[35314]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:24 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:24 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:24 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:24 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:24 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:38:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:24 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:26.352 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:26 smithi191 ceph-mon[35314]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:26.352 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:26 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T08:38:26.352 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:26 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:26.430 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:26 smithi172 ceph-mon[30854]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:26.430 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:26 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T08:38:26.430 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:26 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:26.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:26 smithi174 ceph-mon[34068]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:26.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:26 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T08:38:26.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:26 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:27.326 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:27 smithi172 ceph-mon[30854]: Deploying daemon osd.0 on smithi172 2024-09-10T08:38:27.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:27 smithi174 ceph-mon[34068]: Deploying daemon osd.0 on smithi172 2024-09-10T08:38:27.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:27 smithi191 ceph-mon[35314]: Deploying daemon osd.0 on smithi172 2024-09-10T08:38:28.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:28 smithi174 ceph-mon[34068]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:28.512 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:28 smithi172 ceph-mon[30854]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:28.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:28 smithi191 ceph-mon[35314]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:30.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:30 smithi174 ceph-mon[34068]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:30.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:30 smithi191 ceph-mon[35314]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:30.552 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:30 smithi172 ceph-mon[30854]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:32.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:32 smithi172 ceph-mon[30854]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:32.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:32 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:32.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:32 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:32.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:32 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:32.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:32 smithi174 ceph-mon[34068]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:32.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:32 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:32.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:32 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:32.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:32 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:32 smithi191 ceph-mon[35314]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:32 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:32 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:32 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:34.427 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:34 smithi172 ceph-mon[30854]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:34.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:34 smithi174 ceph-mon[34068]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:34.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:34 smithi191 ceph-mon[35314]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:35.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:35 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:35.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:35 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:35.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:35 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:35.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:35 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:38:35.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:35 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:35.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:35 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:35.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:35 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:35.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:35 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:35.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:35 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:38:35.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:35 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:35.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:35 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:35.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:35 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:35.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:35 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:35.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:35 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:38:35.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:35 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:36.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:36.258+0000 7efde67fc640 1 -- 172.21.15.172:0/854006734 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efdf006a730 con 0x7efddc0392c0 2024-09-10T08:38:36.260 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 0 on host 'smithi172' 2024-09-10T08:38:36.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:36.260+0000 7efdf780a640 1 -- 172.21.15.172:0/854006734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efddc0392c0 msgr2=0x7efddc03b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:36.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:36.260+0000 7efdf780a640 1 --2- 172.21.15.172:0/854006734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efddc0392c0 0x7efddc03b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efdf0101b30 tx=0x7efdec005f90 comp rx=0 tx=0).stop 2024-09-10T08:38:36.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:36.260+0000 7efdf780a640 1 -- 172.21.15.172:0/854006734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf0101cd0 msgr2=0x7efdf0198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:36.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:36.260+0000 7efdf780a640 1 --2- 172.21.15.172:0/854006734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf0101cd0 0x7efdf0198310 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efde000d950 tx=0x7efde000de10 comp rx=0 tx=0).stop 2024-09-10T08:38:36.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:36.260+0000 7efdf780a640 1 -- 172.21.15.172:0/854006734 shutdown_connections 2024-09-10T08:38:36.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:36.260+0000 7efdf780a640 1 --2- 172.21.15.172:0/854006734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efddc0392c0 0x7efddc03b780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:36.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:36.260+0000 7efdf780a640 1 --2- 172.21.15.172:0/854006734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf0102690 0x7efdf019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:36.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:36.260+0000 7efdf780a640 1 --2- 172.21.15.172:0/854006734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf0101cd0 0x7efdf0198310 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:36.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:36.260+0000 7efdf780a640 1 --2- 172.21.15.172:0/854006734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efdf0100ad0 0x7efdf0197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:36.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:36.260+0000 7efdf780a640 1 -- 172.21.15.172:0/854006734 >> 172.21.15.172:0/854006734 conn(0x7efdf00fc280 msgr2=0x7efdf006a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:36.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:36.260+0000 7efdf780a640 1 -- 172.21.15.172:0/854006734 shutdown_connections 2024-09-10T08:38:36.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:38:36.260+0000 7efdf780a640 1 -- 172.21.15.172:0/854006734 wait complete. 2024-09-10T08:38:36.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:36 smithi174 ceph-mon[34068]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:36 smithi191 ceph-mon[35314]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:36 smithi172 ceph-mon[30854]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:36.933 DEBUG:teuthology.orchestra.run.smithi172:osd.0> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T08:38:36.936 INFO:tasks.cephadm:Deploying osd.1 on smithi174 with /dev/vg_nvme/lv_4... 2024-09-10T08:38:36.936 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T08:38:37.199 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:38:37.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:37 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:37.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:37 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:37.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:37 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:37.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:37 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:37.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:37 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:37.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:37 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:37.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:37.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:37.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:38.074 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 08:38:37 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-0[42060]: 2024-09-10T08:38:37.814+0000 7fc950f3d640 -1 osd.0 0 log_to_monitors true 2024-09-10T08:38:38.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:38 smithi172 ceph-mon[30854]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:38.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:38 smithi172 ceph-mon[30854]: from='osd.0 [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T08:38:38.485 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:38 smithi174 ceph-mon[34068]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:38.486 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:38 smithi174 ceph-mon[34068]: from='osd.0 [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T08:38:38.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:38 smithi191 ceph-mon[35314]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:38.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:38 smithi191 ceph-mon[35314]: from='osd.0 [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T08:38:39.357 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:39 smithi174 ceph-mon[34068]: from='osd.0 [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T08:38:39.357 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:39 smithi174 ceph-mon[34068]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T08:38:39.358 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:39 smithi174 ceph-mon[34068]: from='osd.0 [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-10T08:38:39.358 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:39 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:38:39.358 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:39 smithi174 ceph-mon[34068]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:39.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:39 smithi191 ceph-mon[35314]: from='osd.0 [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T08:38:39.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:39 smithi191 ceph-mon[35314]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T08:38:39.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:39 smithi191 ceph-mon[35314]: from='osd.0 [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-10T08:38:39.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:39 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:38:39.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:39 smithi191 ceph-mon[35314]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:39.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:39 smithi172 ceph-mon[30854]: from='osd.0 [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T08:38:39.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:39 smithi172 ceph-mon[30854]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T08:38:39.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:39 smithi172 ceph-mon[30854]: from='osd.0 [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-10T08:38:39.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:39 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:38:39.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:39 smithi172 ceph-mon[30854]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T08:38:39.582 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 08:38:39 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-0[42060]: 2024-09-10T08:38:39.275+0000 7fc94cec0640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T08:38:39.582 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 08:38:39 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-0[42060]: 2024-09-10T08:38:39.278+0000 7fc948ce8640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:38:39.670 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:38:39.698 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_4 2024-09-10T08:38:39.946 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:38:40.328 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:40 smithi174 ceph-mon[34068]: from='osd.0 [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-10T08:38:40.328 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:40 smithi174 ceph-mon[34068]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T08:38:40.328 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:40 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:38:40.328 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:40 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:38:40.328 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:40 smithi174 ceph-mon[34068]: Detected new or changed devices on smithi172 2024-09-10T08:38:40.328 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:40 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:40.328 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:40 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:40.328 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:40 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:38:40.329 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:40 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi172 to 17088M 2024-09-10T08:38:40.329 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:40 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:40.329 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:40 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:40.329 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:40 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:38:40.329 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:40 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:40.420 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.411+0000 7feee007c640 1 -- 172.21.15.174:0/320222946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed8101cd0 msgr2=0x7feed8102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:40.420 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.411+0000 7feee007c640 1 --2- 172.21.15.174:0/320222946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed8101cd0 0x7feed8102150 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feecc009900 tx=0x7feecc02a410 comp rx=0 tx=0).stop 2024-09-10T08:38:40.420 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.411+0000 7feee007c640 1 -- 172.21.15.174:0/320222946 shutdown_connections 2024-09-10T08:38:40.420 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.411+0000 7feee007c640 1 --2- 172.21.15.174:0/320222946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feed8102690 0x7feed8108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:40.420 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.411+0000 7feee007c640 1 --2- 172.21.15.174:0/320222946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed8101cd0 0x7feed8102150 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:40.421 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.411+0000 7feee007c640 1 --2- 172.21.15.174:0/320222946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feed8100ad0 0x7feed8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:40.421 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.411+0000 7feee007c640 1 -- 172.21.15.174:0/320222946 >> 172.21.15.174:0/320222946 conn(0x7feed80fc280 msgr2=0x7feed80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:38:40.421 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.411+0000 7feee007c640 1 -- 172.21.15.174:0/320222946 shutdown_connections 2024-09-10T08:38:40.421 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.411+0000 7feee007c640 1 -- 172.21.15.174:0/320222946 wait complete. 2024-09-10T08:38:40.421 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.411+0000 7feee007c640 1 Processor -- start 2024-09-10T08:38:40.421 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.411+0000 7feee007c640 1 -- start start 2024-09-10T08:38:40.421 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feee007c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feed8100ad0 0x7feed8197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:40.421 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feee007c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed8101cd0 0x7feed81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:40.421 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feee007c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feed8102690 0x7feed819f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:40.421 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feee007c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feed810c600 con 0x7feed8102690 2024-09-10T08:38:40.421 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feee007c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feed810c480 con 0x7feed8101cd0 2024-09-10T08:38:40.421 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feee007c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7feed810c780 con 0x7feed8100ad0 2024-09-10T08:38:40.422 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feedd617640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed8101cd0 0x7feed81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:40.422 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feedd617640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed8101cd0 0x7feed81983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:37196/0 (socket says 172.21.15.174:37196) 2024-09-10T08:38:40.422 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feedd617640 1 -- 172.21.15.174:0/3716262344 learned_addr learned my addr 172.21.15.174:0/3716262344 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-10T08:38:40.422 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feede619640 1 --2- 172.21.15.174:0/3716262344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feed8102690 0x7feed819f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:40.422 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feedde18640 1 --2- 172.21.15.174:0/3716262344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feed8100ad0 0x7feed8197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:40.422 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feedd617640 1 -- 172.21.15.174:0/3716262344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feed8100ad0 msgr2=0x7feed8197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:40.422 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feedd617640 1 --2- 172.21.15.174:0/3716262344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feed8100ad0 0x7feed8197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:40.422 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feedd617640 1 -- 172.21.15.174:0/3716262344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feed8102690 msgr2=0x7feed819f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:38:40.422 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feedd617640 1 --2- 172.21.15.174:0/3716262344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feed8102690 0x7feed819f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:38:40.422 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feedd617640 1 -- 172.21.15.174:0/3716262344 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed819fae0 con 0x7feed8101cd0 2024-09-10T08:38:40.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feedd617640 1 --2- 172.21.15.174:0/3716262344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed8101cd0 0x7feed81983f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feecc004770 tx=0x7feecc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:40.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feed6ffd640 1 -- 172.21.15.174:0/3716262344 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc03f030 con 0x7feed8101cd0 2024-09-10T08:38:40.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feed6ffd640 1 -- 172.21.15.174:0/3716262344 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feecc003cb0 con 0x7feed8101cd0 2024-09-10T08:38:40.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feee007c640 1 -- 172.21.15.174:0/3716262344 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed819fd70 con 0x7feed8101cd0 2024-09-10T08:38:40.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feed6ffd640 1 -- 172.21.15.174:0/3716262344 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc035070 con 0x7feed8101cd0 2024-09-10T08:38:40.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.412+0000 7feee007c640 1 -- 172.21.15.174:0/3716262344 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed81a01f0 con 0x7feed8101cd0 2024-09-10T08:38:40.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.414+0000 7feee007c640 1 -- 172.21.15.174:0/3716262344 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed81a0420 con 0x7feed8101cd0 2024-09-10T08:38:40.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.414+0000 7feed6ffd640 1 -- 172.21.15.174:0/3716262344 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7feecc042030 con 0x7feed8101cd0 2024-09-10T08:38:40.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.415+0000 7feed6ffd640 1 --2- 172.21.15.174:0/3716262344 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feec00a2b20 0x7feec00a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:38:40.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.415+0000 7feed6ffd640 1 -- 172.21.15.174:0/3716262344 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7feecc0edbd0 con 0x7feed8101cd0 2024-09-10T08:38:40.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.415+0000 7feedde18640 1 --2- 172.21.15.174:0/3716262344 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feec00a2b20 0x7feec00a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:38:40.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.416+0000 7feedde18640 1 --2- 172.21.15.174:0/3716262344 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feec00a2b20 0x7feec00a4fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feec8005c40 tx=0x7feec800a780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:38:40.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.419+0000 7feed6ffd640 1 -- 172.21.15.174:0/3716262344 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feecc0bc5a0 con 0x7feed8101cd0 2024-09-10T08:38:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:40 smithi191 ceph-mon[35314]: from='osd.0 [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-10T08:38:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:40 smithi191 ceph-mon[35314]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T08:38:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:40 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:38:40.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:40 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:38:40.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:40 smithi191 ceph-mon[35314]: Detected new or changed devices on smithi172 2024-09-10T08:38:40.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:40 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:40.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:40 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:40.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:40 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:38:40.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:40 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi172 to 17088M 2024-09-10T08:38:40.552 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:40 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:40.552 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:40 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:40.552 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:40 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:38:40.552 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:40 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:40.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:40 smithi172 ceph-mon[30854]: from='osd.0 [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-10T08:38:40.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:40 smithi172 ceph-mon[30854]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T08:38:40.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:38:40.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:38:40.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:40 smithi172 ceph-mon[30854]: Detected new or changed devices on smithi172 2024-09-10T08:38:40.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:40.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:40.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:38:40.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:40 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi172 to 17088M 2024-09-10T08:38:40.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:40.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:40.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:38:40.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:40.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:38:40.592+0000 7feee007c640 1 -- 172.21.15.174:0/3716262344 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7feed806a730 con 0x7feec00a2b20 2024-09-10T08:38:41.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:41 smithi191 ceph-mon[35314]: purged_snaps scrub starts 2024-09-10T08:38:41.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:41 smithi191 ceph-mon[35314]: purged_snaps scrub ok 2024-09-10T08:38:41.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:41 smithi191 ceph-mon[35314]: osd.0 [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952] boot 2024-09-10T08:38:41.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:41 smithi191 ceph-mon[35314]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T08:38:41.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:41 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:38:41.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:41 smithi191 ceph-mon[35314]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:38:41.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:41 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:38:41.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:41 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:38:41.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:41 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:41.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:41 smithi191 ceph-mon[35314]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:41 smithi172 ceph-mon[30854]: purged_snaps scrub starts 2024-09-10T08:38:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:41 smithi172 ceph-mon[30854]: purged_snaps scrub ok 2024-09-10T08:38:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:41 smithi172 ceph-mon[30854]: osd.0 [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952] boot 2024-09-10T08:38:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:41 smithi172 ceph-mon[30854]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T08:38:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:41 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:38:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:41 smithi172 ceph-mon[30854]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:38:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:41 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:38:41.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:41 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:38:41.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:41 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:41.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:41 smithi172 ceph-mon[30854]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:41.641 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:41 smithi174 ceph-mon[34068]: purged_snaps scrub starts 2024-09-10T08:38:41.642 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:41 smithi174 ceph-mon[34068]: purged_snaps scrub ok 2024-09-10T08:38:41.642 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:41 smithi174 ceph-mon[34068]: osd.0 [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952] boot 2024-09-10T08:38:41.642 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:41 smithi174 ceph-mon[34068]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T08:38:41.642 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:41 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T08:38:41.642 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:41 smithi174 ceph-mon[34068]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:38:41.642 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:41 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:38:41.642 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:41 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:38:41.642 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:41 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:41.642 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:41 smithi174 ceph-mon[34068]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:42.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:42 smithi174 ceph-mon[34068]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T08:38:42.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:42 smithi191 ceph-mon[35314]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T08:38:42.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:42 smithi172 ceph-mon[30854]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T08:38:43.424 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:43 smithi174 ceph-mon[34068]: from='client.? 172.21.15.174:0/1648805323' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71"}]: dispatch 2024-09-10T08:38:43.424 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:43 smithi174 ceph-mon[34068]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71"}]: dispatch 2024-09-10T08:38:43.424 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:43 smithi174 ceph-mon[34068]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71"}]': finished 2024-09-10T08:38:43.424 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:43 smithi174 ceph-mon[34068]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T08:38:43.424 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:43 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:38:43.424 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:43 smithi174 ceph-mon[34068]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:43.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:43 smithi191 ceph-mon[35314]: from='client.? 172.21.15.174:0/1648805323' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71"}]: dispatch 2024-09-10T08:38:43.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:43 smithi191 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71"}]: dispatch 2024-09-10T08:38:43.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:43 smithi191 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71"}]': finished 2024-09-10T08:38:43.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:43 smithi191 ceph-mon[35314]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T08:38:43.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:43 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:38:43.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:43 smithi191 ceph-mon[35314]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:43.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:43 smithi172 ceph-mon[30854]: from='client.? 172.21.15.174:0/1648805323' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71"}]: dispatch 2024-09-10T08:38:43.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:43 smithi172 ceph-mon[30854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71"}]: dispatch 2024-09-10T08:38:43.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:43 smithi172 ceph-mon[30854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71"}]': finished 2024-09-10T08:38:43.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:43 smithi172 ceph-mon[30854]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T08:38:43.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:43 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:38:43.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:43 smithi172 ceph-mon[30854]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:44.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:44 smithi191 ceph-mon[35314]: from='client.? 172.21.15.174:0/1921215782' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:38:44.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:44 smithi172 ceph-mon[30854]: from='client.? 172.21.15.174:0/1921215782' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:38:44.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:44 smithi174 ceph-mon[34068]: from='client.? 172.21.15.174:0/1921215782' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:38:45.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:45 smithi191 ceph-mon[35314]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:45.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:45 smithi172 ceph-mon[30854]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:45.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:45 smithi174 ceph-mon[34068]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:48.391 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:48 smithi174 ceph-mon[34068]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:48.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:48 smithi191 ceph-mon[35314]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:48.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:48 smithi172 ceph-mon[30854]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:50.406 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:50 smithi172 ceph-mon[30854]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:50.408 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:50 smithi174 ceph-mon[34068]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:50.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:50 smithi191 ceph-mon[35314]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:51.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:51 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T08:38:51.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:51 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:51.489 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:51 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T08:38:51.489 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:51 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:51.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:51 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T08:38:51.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:51 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:38:52.384 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:52 smithi174 ceph-mon[34068]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:52.384 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:52 smithi174 ceph-mon[34068]: Deploying daemon osd.1 on smithi174 2024-09-10T08:38:52.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:52 smithi191 ceph-mon[35314]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:52 smithi191 ceph-mon[35314]: Deploying daemon osd.1 on smithi174 2024-09-10T08:38:52.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:52 smithi172 ceph-mon[30854]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:52 smithi172 ceph-mon[30854]: Deploying daemon osd.1 on smithi174 2024-09-10T08:38:54.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:54 smithi174 ceph-mon[34068]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:54.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:54 smithi191 ceph-mon[35314]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:54.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:54 smithi172 ceph-mon[30854]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:56.479 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:56 smithi174 ceph-mon[34068]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:56.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:56 smithi191 ceph-mon[35314]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:56.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:56 smithi172 ceph-mon[30854]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:58.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:58 smithi174 ceph-mon[34068]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:58.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:58.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:58.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:38:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:58.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:58 smithi191 ceph-mon[35314]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:58 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:58 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:38:58 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:58 smithi172 ceph-mon[30854]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:38:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:58 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:38:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:58 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:38:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:38:58 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:00.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:00 smithi174 ceph-mon[34068]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:39:00.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:00 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:00.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:00 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:00.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:00 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:00.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:00 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:00.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:00 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:00 smithi191 ceph-mon[35314]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:39:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:00 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:00 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:00 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:00 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:00 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:00 smithi172 ceph-mon[30854]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:39:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:00 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:00 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:00 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:00 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:00 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:01.620 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:01.619+0000 7feed6ffd640 1 -- 172.21.15.174:0/3716262344 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7feed806a730 con 0x7feec00a2b20 2024-09-10T08:39:01.620 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 1 on host 'smithi174' 2024-09-10T08:39:01.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:01.622+0000 7feee007c640 1 -- 172.21.15.174:0/3716262344 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feec00a2b20 msgr2=0x7feec00a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:01.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:01.622+0000 7feee007c640 1 --2- 172.21.15.174:0/3716262344 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feec00a2b20 0x7feec00a4fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feec8005c40 tx=0x7feec800a780 comp rx=0 tx=0).stop 2024-09-10T08:39:01.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:01.622+0000 7feee007c640 1 -- 172.21.15.174:0/3716262344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed8101cd0 msgr2=0x7feed81983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:01.622 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:01.622+0000 7feee007c640 1 --2- 172.21.15.174:0/3716262344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed8101cd0 0x7feed81983f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feecc004770 tx=0x7feecc0042f0 comp rx=0 tx=0).stop 2024-09-10T08:39:01.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:01.622+0000 7feee007c640 1 -- 172.21.15.174:0/3716262344 shutdown_connections 2024-09-10T08:39:01.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:01.622+0000 7feee007c640 1 --2- 172.21.15.174:0/3716262344 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feec00a2b20 0x7feec00a4fe0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:01.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:01.622+0000 7feee007c640 1 --2- 172.21.15.174:0/3716262344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feed8102690 0x7feed819f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:01.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:01.622+0000 7feee007c640 1 --2- 172.21.15.174:0/3716262344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed8101cd0 0x7feed81983f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:01.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:01.622+0000 7feee007c640 1 --2- 172.21.15.174:0/3716262344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feed8100ad0 0x7feed8197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:01.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:01.622+0000 7feee007c640 1 -- 172.21.15.174:0/3716262344 >> 172.21.15.174:0/3716262344 conn(0x7feed80fc280 msgr2=0x7feed806a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:01.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:01.622+0000 7feee007c640 1 -- 172.21.15.174:0/3716262344 shutdown_connections 2024-09-10T08:39:01.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:01.622+0000 7feee007c640 1 -- 172.21.15.174:0/3716262344 wait complete. 2024-09-10T08:39:02.235 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:02 smithi174 ceph-mon[34068]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:39:02.235 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:02 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:39:02.235 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:02 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:02.235 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:02 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:02.262 DEBUG:teuthology.orchestra.run.smithi174:osd.1> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T08:39:02.264 INFO:tasks.cephadm:Deploying osd.2 on smithi191 with /dev/vg_nvme/lv_4... 2024-09-10T08:39:02.265 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T08:39:02.517 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:39:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:02 smithi191 ceph-mon[35314]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:39:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:02 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:39:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:02 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:02 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:02.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:02 smithi172 ceph-mon[30854]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:39:02.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:02 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:39:02.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:02 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:02.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:02 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:03.992 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 08:39:03 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-1[38530]: 2024-09-10T08:39:03.799+0000 7f6de3f99640 -1 osd.1 0 log_to_monitors true 2024-09-10T08:39:04.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:04 smithi191 ceph-mon[35314]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:39:04.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:04 smithi191 ceph-mon[35314]: from='osd.1 [v2:172.21.15.174:6800/3279661439,v1:172.21.15.174:6801/3279661439]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T08:39:04.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:04 smithi191 ceph-mon[35314]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T08:39:04.413 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:04 smithi174 ceph-mon[34068]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:39:04.413 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:04 smithi174 ceph-mon[34068]: from='osd.1 [v2:172.21.15.174:6800/3279661439,v1:172.21.15.174:6801/3279661439]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T08:39:04.413 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:04 smithi174 ceph-mon[34068]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T08:39:04.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:04 smithi172 ceph-mon[30854]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:39:04.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:04 smithi172 ceph-mon[30854]: from='osd.1 [v2:172.21.15.174:6800/3279661439,v1:172.21.15.174:6801/3279661439]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T08:39:04.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:04 smithi172 ceph-mon[30854]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T08:39:05.062 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:39:05.087 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi191:vg_nvme/lv_4 2024-09-10T08:39:05.335 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:39:05.456 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:05 smithi191 ceph-mon[35314]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T08:39:05.456 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:05 smithi191 ceph-mon[35314]: from='osd.1 [v2:172.21.15.174:6800/3279661439,v1:172.21.15.174:6801/3279661439]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:39:05.456 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:05 smithi191 ceph-mon[35314]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T08:39:05.456 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:05 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:39:05.456 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:05 smithi191 ceph-mon[35314]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:39:05.456 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:05 smithi191 ceph-mon[35314]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T08:39:05.456 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:05 smithi191 ceph-mon[35314]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T08:39:05.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:05 smithi174 ceph-mon[34068]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T08:39:05.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:05 smithi174 ceph-mon[34068]: from='osd.1 [v2:172.21.15.174:6800/3279661439,v1:172.21.15.174:6801/3279661439]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:39:05.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:05 smithi174 ceph-mon[34068]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T08:39:05.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:39:05.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:05 smithi174 ceph-mon[34068]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:39:05.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:05 smithi174 ceph-mon[34068]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T08:39:05.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:05 smithi174 ceph-mon[34068]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T08:39:05.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:05 smithi172 ceph-mon[30854]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T08:39:05.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:05 smithi172 ceph-mon[30854]: from='osd.1 [v2:172.21.15.174:6800/3279661439,v1:172.21.15.174:6801/3279661439]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:39:05.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:05 smithi172 ceph-mon[30854]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T08:39:05.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:05 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:39:05.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:05 smithi172 ceph-mon[30854]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:39:05.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:05 smithi172 ceph-mon[30854]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T08:39:05.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:05 smithi172 ceph-mon[30854]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T08:39:05.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.769+0000 7fd76eb8a640 1 -- 172.21.15.191:0/1212736528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd768100b10 msgr2=0x7fd768100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:05.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.769+0000 7fd76eb8a640 1 --2- 172.21.15.191:0/1212736528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd768100b10 0x7fd768100f10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd758009900 tx=0x7fd75802a410 comp rx=0 tx=0).stop 2024-09-10T08:39:05.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.769+0000 7fd76eb8a640 1 -- 172.21.15.191:0/1212736528 shutdown_connections 2024-09-10T08:39:05.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.769+0000 7fd76eb8a640 1 --2- 172.21.15.191:0/1212736528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7681026d0 0x7fd768108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:05.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.769+0000 7fd76eb8a640 1 --2- 172.21.15.191:0/1212736528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd768101d10 0x7fd768102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:05.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.770+0000 7fd76eb8a640 1 --2- 172.21.15.191:0/1212736528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd768100b10 0x7fd768100f10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:05.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.770+0000 7fd76eb8a640 1 -- 172.21.15.191:0/1212736528 >> 172.21.15.191:0/1212736528 conn(0x7fd7680fc2a0 msgr2=0x7fd7680fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:05.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.770+0000 7fd76eb8a640 1 -- 172.21.15.191:0/1212736528 shutdown_connections 2024-09-10T08:39:05.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.770+0000 7fd76eb8a640 1 -- 172.21.15.191:0/1212736528 wait complete. 2024-09-10T08:39:05.772 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.770+0000 7fd76eb8a640 1 Processor -- start 2024-09-10T08:39:05.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.770+0000 7fd76eb8a640 1 -- start start 2024-09-10T08:39:05.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.770+0000 7fd76eb8a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd768100b10 0x7fd768197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:05.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.770+0000 7fd76eb8a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd768101d10 0x7fd768198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:05.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.770+0000 7fd76eb8a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7681026d0 0x7fd76819f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:05.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.770+0000 7fd76eb8a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd76810c670 con 0x7fd768100b10 2024-09-10T08:39:05.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.770+0000 7fd76eb8a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd76810c4f0 con 0x7fd7681026d0 2024-09-10T08:39:05.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.770+0000 7fd76eb8a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd76810c7f0 con 0x7fd768101d10 2024-09-10T08:39:05.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.770+0000 7fd767fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd768101d10 0x7fd768198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:05.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.771+0000 7fd767fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd768101d10 0x7fd768198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:52626/0 (socket says 172.21.15.191:52626) 2024-09-10T08:39:05.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.771+0000 7fd767fff640 1 -- 172.21.15.191:0/379413197 learned_addr learned my addr 172.21.15.191:0/379413197 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-10T08:39:05.773 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.771+0000 7fd76d127640 1 --2- 172.21.15.191:0/379413197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7681026d0 0x7fd76819f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:05.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.771+0000 7fd76c926640 1 --2- 172.21.15.191:0/379413197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd768100b10 0x7fd768197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:05.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.771+0000 7fd767fff640 1 -- 172.21.15.191:0/379413197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7681026d0 msgr2=0x7fd76819f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:05.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.771+0000 7fd767fff640 1 --2- 172.21.15.191:0/379413197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7681026d0 0x7fd76819f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:05.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.771+0000 7fd767fff640 1 -- 172.21.15.191:0/379413197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd768100b10 msgr2=0x7fd768197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:05.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.771+0000 7fd767fff640 1 --2- 172.21.15.191:0/379413197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd768100b10 0x7fd768197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:05.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.771+0000 7fd767fff640 1 -- 172.21.15.191:0/379413197 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76819fa80 con 0x7fd768101d10 2024-09-10T08:39:05.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.771+0000 7fd767fff640 1 --2- 172.21.15.191:0/379413197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd768101d10 0x7fd768198390 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00d950 tx=0x7fd75c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:05.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.771+0000 7fd765ffb640 1 -- 172.21.15.191:0/379413197 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c017070 con 0x7fd768101d10 2024-09-10T08:39:05.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.771+0000 7fd765ffb640 1 -- 172.21.15.191:0/379413197 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd75c009c30 con 0x7fd768101d10 2024-09-10T08:39:05.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.771+0000 7fd76eb8a640 1 -- 172.21.15.191:0/379413197 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd768102f90 con 0x7fd768101d10 2024-09-10T08:39:05.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.771+0000 7fd765ffb640 1 -- 172.21.15.191:0/379413197 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c00f620 con 0x7fd768101d10 2024-09-10T08:39:05.774 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.772+0000 7fd76eb8a640 1 -- 172.21.15.191:0/379413197 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7681034d0 con 0x7fd768101d10 2024-09-10T08:39:05.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.773+0000 7fd76eb8a640 1 -- 172.21.15.191:0/379413197 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7681037f0 con 0x7fd768101d10 2024-09-10T08:39:05.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.774+0000 7fd765ffb640 1 -- 172.21.15.191:0/379413197 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7fd75c0109a0 con 0x7fd768101d10 2024-09-10T08:39:05.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.775+0000 7fd765ffb640 1 --2- 172.21.15.191:0/379413197 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd7500a2b20 0x7fd7500a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:05.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.775+0000 7fd765ffb640 1 -- 172.21.15.191:0/379413197 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fd75c0cd1e0 con 0x7fd768101d10 2024-09-10T08:39:05.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.775+0000 7fd76c926640 1 --2- 172.21.15.191:0/379413197 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd7500a2b20 0x7fd7500a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:05.776 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.776+0000 7fd76c926640 1 --2- 172.21.15.191:0/379413197 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd7500a2b20 0x7fd7500a4fe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd758005ab0 tx=0x7fd758005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:05.778 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.778+0000 7fd765ffb640 1 -- 172.21.15.191:0/379413197 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd75c09b460 con 0x7fd768101d10 2024-09-10T08:39:05.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:05.953+0000 7fd76eb8a640 1 -- 172.21.15.191:0/379413197 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd76806aed0 con 0x7fd7500a2b20 2024-09-10T08:39:06.140 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 08:39:05 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-1[38530]: 2024-09-10T08:39:05.775+0000 7f6ddff1c640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T08:39:06.140 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 08:39:05 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-1[38530]: 2024-09-10T08:39:05.779+0000 7f6dd7d3c640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:39:06.447 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:06 smithi191 ceph-mon[35314]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:39:06.447 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:39:06.447 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:39:06.447 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:06 smithi191 ceph-mon[35314]: Detected new or changed devices on smithi174 2024-09-10T08:39:06.448 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:06.448 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:06.448 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:39:06.448 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:06.448 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:06.448 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:06.448 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:06.448 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:06 smithi191 ceph-mon[35314]: from='osd.1 ' entity='osd.1' 2024-09-10T08:39:06.448 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:39:06.448 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:39:06.448 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:06.448 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:39:06.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:06 smithi174 ceph-mon[34068]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:39:06.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:39:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:39:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:06 smithi174 ceph-mon[34068]: Detected new or changed devices on smithi174 2024-09-10T08:39:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:39:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:06 smithi174 ceph-mon[34068]: from='osd.1 ' entity='osd.1' 2024-09-10T08:39:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:39:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:39:06.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:06.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:39:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:06 smithi172 ceph-mon[30854]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:39:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:39:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:39:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:06 smithi172 ceph-mon[30854]: Detected new or changed devices on smithi174 2024-09-10T08:39:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:39:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:06.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:06.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:06.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:06 smithi172 ceph-mon[30854]: from='osd.1 ' entity='osd.1' 2024-09-10T08:39:06.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:39:06.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:39:06.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:06.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:39:07.428 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:07 smithi191 ceph-mon[35314]: purged_snaps scrub starts 2024-09-10T08:39:07.428 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:07 smithi191 ceph-mon[35314]: purged_snaps scrub ok 2024-09-10T08:39:07.428 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:07 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-10T08:39:07.429 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:07 smithi191 ceph-mon[35314]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:07.429 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:07 smithi191 ceph-mon[35314]: osd.1 [v2:172.21.15.174:6800/3279661439,v1:172.21.15.174:6801/3279661439] boot 2024-09-10T08:39:07.429 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:07 smithi191 ceph-mon[35314]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T08:39:07.429 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:07 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:39:07.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:07 smithi174 ceph-mon[34068]: purged_snaps scrub starts 2024-09-10T08:39:07.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:07 smithi174 ceph-mon[34068]: purged_snaps scrub ok 2024-09-10T08:39:07.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:07 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-10T08:39:07.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:07 smithi174 ceph-mon[34068]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:07.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:07 smithi174 ceph-mon[34068]: osd.1 [v2:172.21.15.174:6800/3279661439,v1:172.21.15.174:6801/3279661439] boot 2024-09-10T08:39:07.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:07 smithi174 ceph-mon[34068]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T08:39:07.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:07 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:39:07.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:07 smithi172 ceph-mon[30854]: purged_snaps scrub starts 2024-09-10T08:39:07.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:07 smithi172 ceph-mon[30854]: purged_snaps scrub ok 2024-09-10T08:39:07.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:07 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-10T08:39:07.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:07 smithi172 ceph-mon[30854]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:07.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:07 smithi172 ceph-mon[30854]: osd.1 [v2:172.21.15.174:6800/3279661439,v1:172.21.15.174:6801/3279661439] boot 2024-09-10T08:39:07.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:07 smithi172 ceph-mon[30854]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T08:39:07.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:07 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T08:39:08.452 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:08 smithi191 ceph-mon[35314]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:39:08.452 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:08 smithi191 ceph-mon[35314]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T08:39:08.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:08 smithi174 ceph-mon[34068]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:39:08.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:08 smithi174 ceph-mon[34068]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T08:39:08.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:08 smithi172 ceph-mon[30854]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T08:39:08.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:08 smithi172 ceph-mon[30854]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T08:39:09.483 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:09 smithi172 ceph-mon[30854]: from='client.? 172.21.15.191:0/3633948880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4"}]: dispatch 2024-09-10T08:39:09.483 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:09 smithi172 ceph-mon[30854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4"}]: dispatch 2024-09-10T08:39:09.483 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:09 smithi172 ceph-mon[30854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4"}]': finished 2024-09-10T08:39:09.483 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:09 smithi172 ceph-mon[30854]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T08:39:09.483 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:09 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:39:09.483 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:09 smithi172 ceph-mon[30854]: from='client.? 172.21.15.191:0/3589450004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:39:09.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:09 smithi174 ceph-mon[34068]: from='client.? 172.21.15.191:0/3633948880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4"}]: dispatch 2024-09-10T08:39:09.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:09 smithi174 ceph-mon[34068]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4"}]: dispatch 2024-09-10T08:39:09.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:09 smithi174 ceph-mon[34068]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4"}]': finished 2024-09-10T08:39:09.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:09 smithi174 ceph-mon[34068]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T08:39:09.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:09 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:39:09.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:09 smithi174 ceph-mon[34068]: from='client.? 172.21.15.191:0/3589450004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:39:09.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:09 smithi191 ceph-mon[35314]: from='client.? 172.21.15.191:0/3633948880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4"}]: dispatch 2024-09-10T08:39:09.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:09 smithi191 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4"}]: dispatch 2024-09-10T08:39:09.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:09 smithi191 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4"}]': finished 2024-09-10T08:39:09.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:09 smithi191 ceph-mon[35314]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T08:39:09.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:09 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:39:09.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:09 smithi191 ceph-mon[35314]: from='client.? 172.21.15.191:0/3589450004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:39:10.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:10 smithi174 ceph-mon[34068]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:10.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:10 smithi191 ceph-mon[35314]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:10.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:10 smithi172 ceph-mon[30854]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:12.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:12 smithi174 ceph-mon[34068]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:12.503 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:12 smithi191 ceph-mon[35314]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:12.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:12 smithi172 ceph-mon[30854]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:14.339 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:14 smithi191 ceph-mon[35314]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:14.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:14 smithi174 ceph-mon[34068]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:14.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:14 smithi172 ceph-mon[30854]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:16.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:16 smithi174 ceph-mon[34068]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:16.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:16 smithi191 ceph-mon[35314]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:16.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:16 smithi172 ceph-mon[30854]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:17.181 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T08:39:17.181 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:17.181 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:17 smithi191 ceph-mon[35314]: Deploying daemon osd.2 on smithi191 2024-09-10T08:39:17.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T08:39:17.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:17.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:17 smithi174 ceph-mon[34068]: Deploying daemon osd.2 on smithi191 2024-09-10T08:39:17.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T08:39:17.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:17.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:17 smithi172 ceph-mon[30854]: Deploying daemon osd.2 on smithi191 2024-09-10T08:39:18.404 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:18 smithi191 ceph-mon[35314]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:18.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:18 smithi174 ceph-mon[34068]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:18 smithi172 ceph-mon[30854]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:20.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:20 smithi174 ceph-mon[34068]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:20.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:20 smithi191 ceph-mon[35314]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:20.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:20 smithi172 ceph-mon[30854]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:22.442 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:22 smithi191 ceph-mon[35314]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:22.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:22 smithi174 ceph-mon[34068]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:22 smithi172 ceph-mon[30854]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:23.253 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:39:23.277 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:23.278 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:23.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:39:23.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:23.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:23.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:39:23.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:23.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:24.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:24 smithi174 ceph-mon[34068]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:24 smithi191 ceph-mon[35314]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:24.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:24 smithi172 ceph-mon[30854]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:26.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:26 smithi191 ceph-mon[35314]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:26.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:26 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:26.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:26 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:26.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:26 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:26.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:26 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:26.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:26 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:26.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:26 smithi174 ceph-mon[34068]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:26.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:26 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:26.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:26 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:26.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:26 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:26.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:26 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:26.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:26 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:26 smithi172 ceph-mon[30854]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:26 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:26 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:26 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:26 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:26 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:26.718 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:26.717+0000 7fd765ffb640 1 -- 172.21.15.191:0/379413197 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd76806aed0 con 0x7fd7500a2b20 2024-09-10T08:39:26.718 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 2 on host 'smithi191' 2024-09-10T08:39:26.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:26.720+0000 7fd76eb8a640 1 -- 172.21.15.191:0/379413197 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd7500a2b20 msgr2=0x7fd7500a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:26.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:26.720+0000 7fd76eb8a640 1 --2- 172.21.15.191:0/379413197 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd7500a2b20 0x7fd7500a4fe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd758005ab0 tx=0x7fd758005a20 comp rx=0 tx=0).stop 2024-09-10T08:39:26.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:26.720+0000 7fd76eb8a640 1 -- 172.21.15.191:0/379413197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd768101d10 msgr2=0x7fd768198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:26.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:26.720+0000 7fd76eb8a640 1 --2- 172.21.15.191:0/379413197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd768101d10 0x7fd768198390 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00d950 tx=0x7fd75c00de10 comp rx=0 tx=0).stop 2024-09-10T08:39:26.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:26.720+0000 7fd76eb8a640 1 -- 172.21.15.191:0/379413197 shutdown_connections 2024-09-10T08:39:26.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:26.720+0000 7fd76eb8a640 1 --2- 172.21.15.191:0/379413197 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd7500a2b20 0x7fd7500a4fe0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:26.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:26.720+0000 7fd76eb8a640 1 --2- 172.21.15.191:0/379413197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7681026d0 0x7fd76819f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:26.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:26.720+0000 7fd76eb8a640 1 --2- 172.21.15.191:0/379413197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd768101d10 0x7fd768198390 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:26.721 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:26.720+0000 7fd76eb8a640 1 --2- 172.21.15.191:0/379413197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd768100b10 0x7fd768197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:26.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:26.720+0000 7fd76eb8a640 1 -- 172.21.15.191:0/379413197 >> 172.21.15.191:0/379413197 conn(0x7fd7680fc2a0 msgr2=0x7fd76806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:26.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:26.720+0000 7fd76eb8a640 1 -- 172.21.15.191:0/379413197 shutdown_connections 2024-09-10T08:39:26.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:39:26.720+0000 7fd76eb8a640 1 -- 172.21.15.191:0/379413197 wait complete. 2024-09-10T08:39:27.299 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:39:27.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:27.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:27.349 DEBUG:teuthology.orchestra.run.smithi191:osd.2> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T08:39:27.351 INFO:tasks.cephadm:Deploying osd.3 on smithi172 with /dev/vg_nvme/lv_3... 2024-09-10T08:39:27.352 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T08:39:27.376 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:39:27.376 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:27.376 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:27.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:39:27.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:27.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:27.593 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:28.091 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 08:39:28 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-2[40272]: 2024-09-10T08:39:28.056+0000 7f329b144640 -1 osd.2 0 log_to_monitors true 2024-09-10T08:39:28.331 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:28 smithi172 ceph-mon[30854]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:28.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:28 smithi172 ceph-mon[30854]: from='osd.2 [v2:172.21.15.191:6800/1963796290,v1:172.21.15.191:6801/1963796290]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:39:28.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:28 smithi172 ceph-mon[30854]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:39:28.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:28 smithi174 ceph-mon[34068]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:28.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:28 smithi174 ceph-mon[34068]: from='osd.2 [v2:172.21.15.191:6800/1963796290,v1:172.21.15.191:6801/1963796290]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:39:28.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:28 smithi174 ceph-mon[34068]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:39:28.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:28 smithi191 ceph-mon[35314]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:28.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:28 smithi191 ceph-mon[35314]: from='osd.2 [v2:172.21.15.191:6800/1963796290,v1:172.21.15.191:6801/1963796290]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:39:28.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:28 smithi191 ceph-mon[35314]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T08:39:29.800 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 08:39:29 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-2[40272]: 2024-09-10T08:39:29.724+0000 7f32970c7640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T08:39:29.800 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 08:39:29 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-2[40272]: 2024-09-10T08:39:29.729+0000 7f3292eef640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:39:29.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:29 smithi191 ceph-mon[35314]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T08:39:29.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:29 smithi191 ceph-mon[35314]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T08:39:29.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:29 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:39:29.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:29 smithi191 ceph-mon[35314]: from='osd.2 [v2:172.21.15.191:6800/1963796290,v1:172.21.15.191:6801/1963796290]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:39:29.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:29 smithi191 ceph-mon[35314]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:39:29.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:29 smithi191 ceph-mon[35314]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:29.974 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:29 smithi172 ceph-mon[30854]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T08:39:29.974 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:29 smithi172 ceph-mon[30854]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T08:39:29.974 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:29 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:39:29.974 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:29 smithi172 ceph-mon[30854]: from='osd.2 [v2:172.21.15.191:6800/1963796290,v1:172.21.15.191:6801/1963796290]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:39:29.974 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:29 smithi172 ceph-mon[30854]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:39:29.974 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:29 smithi172 ceph-mon[30854]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:29.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:29 smithi174 ceph-mon[34068]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T08:39:29.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:29 smithi174 ceph-mon[34068]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T08:39:29.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:29 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:39:29.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:29 smithi174 ceph-mon[34068]: from='osd.2 [v2:172.21.15.191:6800/1963796290,v1:172.21.15.191:6801/1963796290]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:39:29.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:29 smithi174 ceph-mon[34068]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:39:29.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:29 smithi174 ceph-mon[34068]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:30.227 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:39:30.252 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_3 2024-09-10T08:39:30.550 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:39:30.822 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:30 smithi172 ceph-mon[30854]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-10T08:39:30.823 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:30 smithi172 ceph-mon[30854]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T08:39:30.823 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:39:30.823 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:39:30.823 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:30.823 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:30.823 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:39:30.823 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:30.823 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:30.823 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:30.824 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:30.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:30 smithi174 ceph-mon[34068]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-10T08:39:30.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:30 smithi174 ceph-mon[34068]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T08:39:30.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:39:30.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:39:30.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:30.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:30.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:39:30.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:30.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:30.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:30.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:31.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.047+0000 7f2e2ea2f640 1 -- 172.21.15.172:0/3605146515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28101c00 msgr2=0x7f2e28102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:31.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.047+0000 7f2e2ea2f640 1 --2- 172.21.15.172:0/3605146515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28101c00 0x7f2e28102000 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009900 tx=0x7f2e1802a410 comp rx=0 tx=0).stop 2024-09-10T08:39:31.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.047+0000 7f2e2ea2f640 1 -- 172.21.15.172:0/3605146515 shutdown_connections 2024-09-10T08:39:31.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.047+0000 7f2e2ea2f640 1 --2- 172.21.15.172:0/3605146515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e281037c0 0x7f2e2810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:31.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.047+0000 7f2e2ea2f640 1 --2- 172.21.15.172:0/3605146515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e28102e00 0x7f2e28103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:31.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.047+0000 7f2e2ea2f640 1 --2- 172.21.15.172:0/3605146515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28101c00 0x7f2e28102000 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:31.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.047+0000 7f2e2ea2f640 1 -- 172.21.15.172:0/3605146515 >> 172.21.15.172:0/3605146515 conn(0x7f2e280fd3b0 msgr2=0x7f2e280ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:31.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.047+0000 7f2e2ea2f640 1 -- 172.21.15.172:0/3605146515 shutdown_connections 2024-09-10T08:39:31.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.047+0000 7f2e2ea2f640 1 -- 172.21.15.172:0/3605146515 wait complete. 2024-09-10T08:39:31.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.048+0000 7f2e2ea2f640 1 Processor -- start 2024-09-10T08:39:31.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.048+0000 7f2e2ea2f640 1 -- start start 2024-09-10T08:39:31.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.048+0000 7f2e2ea2f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e28101c00 0x7f2e28197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:31.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.048+0000 7f2e2ea2f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28102e00 0x7f2e281982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:31.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.048+0000 7f2e2ea2f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e281037c0 0x7f2e2819f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:31.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.048+0000 7f2e2ea2f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2810d710 con 0x7f2e28102e00 2024-09-10T08:39:31.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.048+0000 7f2e2ea2f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2810d590 con 0x7f2e281037c0 2024-09-10T08:39:31.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.048+0000 7f2e2ea2f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2810d890 con 0x7f2e28101c00 2024-09-10T08:39:31.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.048+0000 7f2e1ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28102e00 0x7f2e281982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:31.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.048+0000 7f2e1ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28102e00 0x7f2e281982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33254/0 (socket says 172.21.15.172:33254) 2024-09-10T08:39:31.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.048+0000 7f2e1ffff640 1 -- 172.21.15.172:0/3731551201 learned_addr learned my addr 172.21.15.172:0/3731551201 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:39:31.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.049+0000 7f2e27fff640 1 --2- 172.21.15.172:0/3731551201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e28101c00 0x7f2e28197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:31.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.049+0000 7f2e2cfcc640 1 --2- 172.21.15.172:0/3731551201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e281037c0 0x7f2e2819f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:31.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.049+0000 7f2e1ffff640 1 -- 172.21.15.172:0/3731551201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e28101c00 msgr2=0x7f2e28197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:31.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.049+0000 7f2e1ffff640 1 --2- 172.21.15.172:0/3731551201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e28101c00 0x7f2e28197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:31.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.049+0000 7f2e1ffff640 1 -- 172.21.15.172:0/3731551201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e281037c0 msgr2=0x7f2e2819f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:31.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.049+0000 7f2e1ffff640 1 --2- 172.21.15.172:0/3731551201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e281037c0 0x7f2e2819f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:31.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.049+0000 7f2e1ffff640 1 -- 172.21.15.172:0/3731551201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2819f990 con 0x7f2e28102e00 2024-09-10T08:39:31.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.049+0000 7f2e1ffff640 1 --2- 172.21.15.172:0/3731551201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28102e00 0x7f2e281982a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000c920 tx=0x7f2e1000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:31.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.049+0000 7f2e25ffb640 1 -- 172.21.15.172:0/3731551201 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10007c70 con 0x7f2e28102e00 2024-09-10T08:39:31.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.049+0000 7f2e25ffb640 1 -- 172.21.15.172:0/3731551201 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e10007e10 con 0x7f2e28102e00 2024-09-10T08:39:31.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.049+0000 7f2e2ea2f640 1 -- 172.21.15.172:0/3731551201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e2819fc80 con 0x7f2e28102e00 2024-09-10T08:39:31.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.049+0000 7f2e25ffb640 1 -- 172.21.15.172:0/3731551201 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10018820 con 0x7f2e28102e00 2024-09-10T08:39:31.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.049+0000 7f2e2ea2f640 1 -- 172.21.15.172:0/3731551201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e281a01c0 con 0x7f2e28102e00 2024-09-10T08:39:31.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:30 smithi191 ceph-mon[35314]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-10T08:39:31.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:30 smithi191 ceph-mon[35314]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T08:39:31.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:39:31.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:39:31.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:31.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:31.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:39:31.053 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:31.054 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:31.054 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:31.054 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.050+0000 7f2e2ea2f640 1 -- 172.21.15.172:0/3731551201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e28109050 con 0x7f2e28102e00 2024-09-10T08:39:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.051+0000 7f2e25ffb640 1 -- 172.21.15.172:0/3731551201 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f2e100189c0 con 0x7f2e28102e00 2024-09-10T08:39:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.052+0000 7f2e25ffb640 1 --2- 172.21.15.172:0/3731551201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e140a2ba0 0x7f2e140a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.052+0000 7f2e27fff640 1 --2- 172.21.15.172:0/3731551201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e140a2ba0 0x7f2e140a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.052+0000 7f2e25ffb640 1 -- 172.21.15.172:0/3731551201 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7f2e100cdb40 con 0x7f2e28102e00 2024-09-10T08:39:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.053+0000 7f2e27fff640 1 --2- 172.21.15.172:0/3731551201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e140a2ba0 0x7f2e140a5060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2e18000c00 tx=0x7f2e180314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:31.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.057+0000 7f2e25ffb640 1 -- 172.21.15.172:0/3731551201 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e10014030 con 0x7f2e28102e00 2024-09-10T08:39:31.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:31.240+0000 7f2e2ea2f640 1 -- 172.21.15.172:0/3731551201 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2e28062630 con 0x7f2e140a2ba0 2024-09-10T08:39:31.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:31 smithi172 ceph-mon[30854]: purged_snaps scrub starts 2024-09-10T08:39:31.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:31 smithi172 ceph-mon[30854]: purged_snaps scrub ok 2024-09-10T08:39:31.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:31 smithi172 ceph-mon[30854]: Detected new or changed devices on smithi191 2024-09-10T08:39:31.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:31 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi191 to 17088M 2024-09-10T08:39:31.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:31 smithi172 ceph-mon[30854]: osd.2 [v2:172.21.15.191:6800/1963796290,v1:172.21.15.191:6801/1963796290] boot 2024-09-10T08:39:31.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:31 smithi172 ceph-mon[30854]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T08:39:31.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:31 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:39:31.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:31 smithi172 ceph-mon[30854]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:31.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:31 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:39:31.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:31 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:39:31.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:31 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:39:31.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:31 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:31.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:31 smithi174 ceph-mon[34068]: purged_snaps scrub starts 2024-09-10T08:39:31.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:31 smithi174 ceph-mon[34068]: purged_snaps scrub ok 2024-09-10T08:39:31.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:31 smithi174 ceph-mon[34068]: Detected new or changed devices on smithi191 2024-09-10T08:39:31.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:31 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi191 to 17088M 2024-09-10T08:39:31.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:31 smithi174 ceph-mon[34068]: osd.2 [v2:172.21.15.191:6800/1963796290,v1:172.21.15.191:6801/1963796290] boot 2024-09-10T08:39:31.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:31 smithi174 ceph-mon[34068]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T08:39:31.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:31 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:39:31.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:31 smithi174 ceph-mon[34068]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:31.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:31 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:39:31.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:31 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:39:31.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:31 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:39:31.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:31 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:32.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:31 smithi191 ceph-mon[35314]: purged_snaps scrub starts 2024-09-10T08:39:32.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:31 smithi191 ceph-mon[35314]: purged_snaps scrub ok 2024-09-10T08:39:32.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:31 smithi191 ceph-mon[35314]: Detected new or changed devices on smithi191 2024-09-10T08:39:32.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:31 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi191 to 17088M 2024-09-10T08:39:32.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:31 smithi191 ceph-mon[35314]: osd.2 [v2:172.21.15.191:6800/1963796290,v1:172.21.15.191:6801/1963796290] boot 2024-09-10T08:39:32.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:31 smithi191 ceph-mon[35314]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T08:39:32.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:31 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T08:39:32.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:31 smithi191 ceph-mon[35314]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T08:39:32.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:31 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:39:32.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:31 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:39:32.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:31 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:39:32.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:31 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:32.818 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:32 smithi172 ceph-mon[30854]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:32.818 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:32 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T08:39:32.818 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:32 smithi172 ceph-mon[30854]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T08:39:32.818 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:32 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:39:32.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:32 smithi174 ceph-mon[34068]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:32.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:32 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T08:39:32.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:32 smithi174 ceph-mon[34068]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T08:39:32.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:32 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:39:33.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:32 smithi191 ceph-mon[35314]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:33.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:32 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T08:39:33.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:32 smithi191 ceph-mon[35314]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T08:39:33.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:32 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T08:39:33.355 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 08:39:33 smithi172 sudo[46420]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:39:33.355 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 08:39:33 smithi172 sudo[46420]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:39:33.355 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 08:39:33 smithi172 sudo[46420]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:39:33.355 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 08:39:33 smithi172 sudo[46420]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:39:33.749 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 08:39:33 smithi172 sudo[46431]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:39:33.750 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 08:39:33 smithi172 sudo[46431]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:39:33.750 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 08:39:33 smithi172 sudo[46431]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:39:33.750 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 08:39:33 smithi172 sudo[46431]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:39:33.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:33.755+0000 7f2e25ffb640 1 -- 172.21.15.172:0/3731551201 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2e10098040 con 0x7f2e28102e00 2024-09-10T08:39:33.889 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 08:39:33 smithi174 sudo[42205]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:39:33.889 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 08:39:33 smithi174 sudo[42205]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:39:33.889 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 08:39:33 smithi174 sudo[42205]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:39:33.889 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 08:39:33 smithi174 sudo[42205]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:39:33.890 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:33 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T08:39:33.890 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:33 smithi174 ceph-mon[34068]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T08:39:33.890 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:33 smithi174 ceph-mon[34068]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:33.890 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:33 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:34.039 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:33 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T08:39:34.039 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:33 smithi172 ceph-mon[30854]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T08:39:34.039 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:33 smithi172 ceph-mon[30854]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:34.039 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:33 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:34.049 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:33 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T08:39:34.049 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:33 smithi191 ceph-mon[35314]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T08:39:34.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:33 smithi191 ceph-mon[35314]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:34.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:33 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:34.241 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 08:39:33 smithi174 sudo[42208]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:39:34.242 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 08:39:33 smithi174 sudo[42208]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:39:34.242 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 08:39:33 smithi174 sudo[42208]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:39:34.242 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 08:39:33 smithi174 sudo[42208]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:39:34.416 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 08:39:34 smithi191 sudo[43871]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:39:34.416 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 08:39:34 smithi191 sudo[43871]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:39:34.417 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 08:39:34 smithi191 sudo[43871]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:39:34.417 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 08:39:34 smithi191 sudo[43871]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:39:34.800 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 08:39:34 smithi191 sudo[43874]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:39:34.800 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 08:39:34 smithi191 sudo[43874]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:39:34.800 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 08:39:34 smithi191 sudo[43874]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:39:34.800 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 08:39:34 smithi191 sudo[43874]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:39:34.936 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 sudo[46529]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:39:34.936 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 sudo[46529]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:39:34.936 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 sudo[46529]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:39:34.936 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 sudo[46529]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:39:35.203 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:34 smithi191 ceph-mon[35314]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T08:39:35.203 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:34 smithi191 ceph-mon[35314]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T08:39:35.203 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:34 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3410237459' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15ecca3f-bc6b-4a06-b283-b515384beb19"}]: dispatch 2024-09-10T08:39:35.203 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:34 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3410237459' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15ecca3f-bc6b-4a06-b283-b515384beb19"}]': finished 2024-09-10T08:39:35.203 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:34 smithi191 ceph-mon[35314]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T08:39:35.204 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:34 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:39:35.204 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:34 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:35.204 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:34 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1056224697' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:39:35.204 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:34 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:35.204 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:34 smithi191 ceph-mon[35314]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:39:35.204 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 sudo[43877]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:39:35.204 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 sudo[43877]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:39:35.204 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 sudo[43877]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:39:35.204 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 sudo[43877]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:39:35.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:34 smithi174 ceph-mon[34068]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T08:39:35.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:34 smithi174 ceph-mon[34068]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T08:39:35.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:34 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3410237459' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15ecca3f-bc6b-4a06-b283-b515384beb19"}]: dispatch 2024-09-10T08:39:35.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:34 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3410237459' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15ecca3f-bc6b-4a06-b283-b515384beb19"}]': finished 2024-09-10T08:39:35.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:34 smithi174 ceph-mon[34068]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T08:39:35.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:34 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:39:35.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:34 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:35.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:34 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1056224697' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:39:35.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:34 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:35.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:34 smithi174 ceph-mon[34068]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:39:35.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 sudo[46556]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:39:35.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 sudo[46556]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:39:35.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 sudo[46556]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:39:35.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 sudo[46556]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:39:35.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 ceph-mon[30854]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T08:39:35.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 ceph-mon[30854]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T08:39:35.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3410237459' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "15ecca3f-bc6b-4a06-b283-b515384beb19"}]: dispatch 2024-09-10T08:39:35.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3410237459' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "15ecca3f-bc6b-4a06-b283-b515384beb19"}]': finished 2024-09-10T08:39:35.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 ceph-mon[30854]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T08:39:35.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:39:35.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:35.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1056224697' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:39:35.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:35.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:34 smithi172 ceph-mon[30854]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:39:35.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 sudo[43880]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:39:35.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 sudo[43880]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:39:35.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 sudo[43880]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:39:35.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 sudo[43880]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:39:35.941 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 sudo[42211]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T08:39:35.942 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 sudo[42211]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:39:35.942 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 sudo[42211]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:39:35.942 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 sudo[42211]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:39:36.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 sudo[42214]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T08:39:36.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 sudo[42214]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T08:39:36.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 sudo[42214]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T08:39:36.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 sudo[42214]: pam_unix(sudo:session): session closed for user root 2024-09-10T08:39:36.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 ceph-mon[34068]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:39:36.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:39:36.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:39:36.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:39:36.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:39:36.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 ceph-mon[34068]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:39:36.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:39:36.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:39:36.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 ceph-mon[34068]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:36.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 ceph-mon[34068]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:39:36.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:39:36.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:39:36.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:35 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:39:36.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 ceph-mon[35314]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:39:36.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:39:36.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:39:36.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:39:36.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:39:36.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 ceph-mon[35314]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:39:36.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:39:36.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:39:36.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 ceph-mon[35314]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:36.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 ceph-mon[35314]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:39:36.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:39:36.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:39:36.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:35 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:39:36.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:35 smithi172 ceph-mon[30854]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:39:36.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:35 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:39:36.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:35 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:39:36.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:35 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:39:36.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:35 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:39:36.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:35 smithi172 ceph-mon[30854]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:39:36.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:35 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:39:36.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:35 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:39:36.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:35 smithi172 ceph-mon[30854]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:36.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:35 smithi172 ceph-mon[30854]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:39:36.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:35 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T08:39:36.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:35 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T08:39:36.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:35 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T08:39:37.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:36 smithi174 ceph-mon[34068]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:39:37.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:36 smithi174 ceph-mon[34068]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:39:37.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:36 smithi191 ceph-mon[35314]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:39:37.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:36 smithi191 ceph-mon[35314]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:39:37.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:36 smithi172 ceph-mon[30854]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T08:39:37.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:36 smithi172 ceph-mon[30854]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T08:39:38.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:37 smithi174 ceph-mon[34068]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:38.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:37 smithi191 ceph-mon[35314]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:38.331 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:37 smithi172 ceph-mon[30854]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:40.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:40 smithi172 ceph-mon[30854]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:40.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:40 smithi174 ceph-mon[34068]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:40 smithi191 ceph-mon[35314]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:42.376 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:42 smithi172 ceph-mon[30854]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:42.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:42 smithi174 ceph-mon[34068]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:42.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:42 smithi191 ceph-mon[35314]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:43.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:43 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T08:39:43.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:43 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:43.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:43 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T08:39:43.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:43 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:43.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:43 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T08:39:43.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:43 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:44.251 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:44 smithi172 ceph-mon[30854]: Deploying daemon osd.3 on smithi172 2024-09-10T08:39:44.251 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:44 smithi172 ceph-mon[30854]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:44.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:44 smithi174 ceph-mon[34068]: Deploying daemon osd.3 on smithi172 2024-09-10T08:39:44.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:44 smithi174 ceph-mon[34068]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:44.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:44 smithi191 ceph-mon[35314]: Deploying daemon osd.3 on smithi172 2024-09-10T08:39:44.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:44 smithi191 ceph-mon[35314]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:46.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:46 smithi174 ceph-mon[34068]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:46.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:46 smithi191 ceph-mon[35314]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:46.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:46 smithi172 ceph-mon[30854]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:48.308 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:48 smithi172 ceph-mon[30854]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:48.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:48 smithi174 ceph-mon[34068]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:48.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:48 smithi191 ceph-mon[35314]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:49.413 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:49 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:39:49.413 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:49 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:49.413 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:49 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:49.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:49 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:39:49.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:49 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:49.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:49 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:49 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:39:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:49 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:49 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:50.411 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:50 smithi172 ceph-mon[30854]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:50.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:50 smithi174 ceph-mon[34068]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:50.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:50 smithi191 ceph-mon[35314]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:52.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:52 smithi174 ceph-mon[34068]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:52.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:52 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:52.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:52 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:52.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:52 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:52.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:52 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:52.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:52 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:52 smithi191 ceph-mon[35314]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:52 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:52 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:52 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:52 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:52 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:52.571 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:52 smithi172 ceph-mon[30854]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:52.571 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:52 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:52.571 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:52 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:52.572 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:52 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:52.572 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:52 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:52.572 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:52 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:52.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:52.918+0000 7f2e25ffb640 1 -- 172.21.15.172:0/3731551201 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2e28062630 con 0x7f2e140a2ba0 2024-09-10T08:39:52.919 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 3 on host 'smithi172' 2024-09-10T08:39:52.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:52.920+0000 7f2e2ea2f640 1 -- 172.21.15.172:0/3731551201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e140a2ba0 msgr2=0x7f2e140a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:52.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:52.920+0000 7f2e2ea2f640 1 --2- 172.21.15.172:0/3731551201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e140a2ba0 0x7f2e140a5060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2e18000c00 tx=0x7f2e180314a0 comp rx=0 tx=0).stop 2024-09-10T08:39:52.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:52.920+0000 7f2e2ea2f640 1 -- 172.21.15.172:0/3731551201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28102e00 msgr2=0x7f2e281982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:52.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:52.920+0000 7f2e2ea2f640 1 --2- 172.21.15.172:0/3731551201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28102e00 0x7f2e281982a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000c920 tx=0x7f2e1000cde0 comp rx=0 tx=0).stop 2024-09-10T08:39:52.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:52.920+0000 7f2e2ea2f640 1 -- 172.21.15.172:0/3731551201 shutdown_connections 2024-09-10T08:39:52.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:52.920+0000 7f2e2ea2f640 1 --2- 172.21.15.172:0/3731551201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e140a2ba0 0x7f2e140a5060 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:52.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:52.920+0000 7f2e2ea2f640 1 --2- 172.21.15.172:0/3731551201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e281037c0 0x7f2e2819f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:52.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:52.920+0000 7f2e2ea2f640 1 --2- 172.21.15.172:0/3731551201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28102e00 0x7f2e281982a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:52.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:52.920+0000 7f2e2ea2f640 1 --2- 172.21.15.172:0/3731551201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e28101c00 0x7f2e28197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:52.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:52.920+0000 7f2e2ea2f640 1 -- 172.21.15.172:0/3731551201 >> 172.21.15.172:0/3731551201 conn(0x7f2e280fd3b0 msgr2=0x7f2e28106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:52.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:52.920+0000 7f2e2ea2f640 1 -- 172.21.15.172:0/3731551201 shutdown_connections 2024-09-10T08:39:52.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:39:52.920+0000 7f2e2ea2f640 1 -- 172.21.15.172:0/3731551201 wait complete. 2024-09-10T08:39:53.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:53 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:39:53.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:53 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:53.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:53 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:53.492 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:53 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:39:53.492 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:53 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:53.492 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:53 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:53.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:53 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:39:53.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:53 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:53.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:53 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:53.612 DEBUG:teuthology.orchestra.run.smithi172:osd.3> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T08:39:53.615 INFO:tasks.cephadm:Deploying osd.4 on smithi174 with /dev/vg_nvme/lv_3... 2024-09-10T08:39:53.615 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T08:39:53.868 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:39:54.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:54 smithi174 ceph-mon[34068]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:54.331 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:54 smithi172 ceph-mon[30854]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:54 smithi191 ceph-mon[35314]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:55.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:55 smithi174 ceph-mon[34068]: from='osd.3 [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T08:39:55.295 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:55 smithi172 ceph-mon[30854]: from='osd.3 [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T08:39:55.296 INFO:journalctl@ceph.osd.3.smithi172.stdout:Sep 10 08:39:55 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-3[48485]: 2024-09-10T08:39:55.035+0000 7faeb95cb640 -1 osd.3 0 log_to_monitors true 2024-09-10T08:39:55.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:55 smithi191 ceph-mon[35314]: from='osd.3 [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T08:39:56.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:56 smithi174 ceph-mon[34068]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:56.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:56 smithi174 ceph-mon[34068]: from='osd.3 [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T08:39:56.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:56 smithi174 ceph-mon[34068]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T08:39:56.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:56 smithi174 ceph-mon[34068]: from='osd.3 [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-10T08:39:56.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:56 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:39:56.406 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:39:56.436 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_3 2024-09-10T08:39:56.450 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:56 smithi191 ceph-mon[35314]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:56.450 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:56 smithi191 ceph-mon[35314]: from='osd.3 [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T08:39:56.450 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:56 smithi191 ceph-mon[35314]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T08:39:56.450 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:56 smithi191 ceph-mon[35314]: from='osd.3 [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-10T08:39:56.450 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:56 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:39:56.475 INFO:journalctl@ceph.osd.3.smithi172.stdout:Sep 10 08:39:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-3[48485]: 2024-09-10T08:39:56.148+0000 7faeb554e640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T08:39:56.475 INFO:journalctl@ceph.osd.3.smithi172.stdout:Sep 10 08:39:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-3[48485]: 2024-09-10T08:39:56.155+0000 7faeb1376640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:39:56.475 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:56 smithi172 ceph-mon[30854]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:56.475 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:56 smithi172 ceph-mon[30854]: from='osd.3 [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T08:39:56.475 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:56 smithi172 ceph-mon[30854]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T08:39:56.476 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:56 smithi172 ceph-mon[30854]: from='osd.3 [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-10T08:39:56.476 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:56 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:39:56.692 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:39:57.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.166+0000 7f84ce74b640 1 -- 172.21.15.174:0/536405796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c8101c00 msgr2=0x7f84c8102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:57.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.166+0000 7f84ce74b640 1 --2- 172.21.15.174:0/536405796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c8101c00 0x7f84c8102000 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009900 tx=0x7f84b802a410 comp rx=0 tx=0).stop 2024-09-10T08:39:57.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.166+0000 7f84ce74b640 1 -- 172.21.15.174:0/536405796 shutdown_connections 2024-09-10T08:39:57.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.166+0000 7f84ce74b640 1 --2- 172.21.15.174:0/536405796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84c81037c0 0x7f84c810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:57.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.166+0000 7f84ce74b640 1 --2- 172.21.15.174:0/536405796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c8102e00 0x7f84c8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:57.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.166+0000 7f84ce74b640 1 --2- 172.21.15.174:0/536405796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c8101c00 0x7f84c8102000 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:57.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.166+0000 7f84ce74b640 1 -- 172.21.15.174:0/536405796 >> 172.21.15.174:0/536405796 conn(0x7f84c80fd390 msgr2=0x7f84c80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:39:57.189 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.166+0000 7f84ce74b640 1 -- 172.21.15.174:0/536405796 shutdown_connections 2024-09-10T08:39:57.189 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.166+0000 7f84ce74b640 1 -- 172.21.15.174:0/536405796 wait complete. 2024-09-10T08:39:57.189 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84ce74b640 1 Processor -- start 2024-09-10T08:39:57.189 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84ce74b640 1 -- start start 2024-09-10T08:39:57.189 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84ce74b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c8101c00 0x7f84c8197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:57.189 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84ce74b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84c8102e00 0x7f84c8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:57.189 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84ce74b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c81037c0 0x7f84c819f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:57.189 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84ce74b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84c810d740 con 0x7f84c8101c00 2024-09-10T08:39:57.189 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84ce74b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84c810d5c0 con 0x7f84c81037c0 2024-09-10T08:39:57.189 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84ce74b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f84c810d8c0 con 0x7f84c8102e00 2024-09-10T08:39:57.189 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84ccce8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c81037c0 0x7f84c819f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:57.190 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84ccce8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c81037c0 0x7f84c819f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:58790/0 (socket says 172.21.15.174:58790) 2024-09-10T08:39:57.190 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84ccce8640 1 -- 172.21.15.174:0/4251552549 learned_addr learned my addr 172.21.15.174:0/4251552549 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-10T08:39:57.190 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84bffff640 1 --2- 172.21.15.174:0/4251552549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84c8102e00 0x7f84c8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:57.190 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84c7fff640 1 --2- 172.21.15.174:0/4251552549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c8101c00 0x7f84c8197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:57.190 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84ccce8640 1 -- 172.21.15.174:0/4251552549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84c8102e00 msgr2=0x7f84c8198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:57.190 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84ccce8640 1 --2- 172.21.15.174:0/4251552549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84c8102e00 0x7f84c8198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:57.190 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84ccce8640 1 -- 172.21.15.174:0/4251552549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c8101c00 msgr2=0x7f84c8197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:39:57.190 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84ccce8640 1 --2- 172.21.15.174:0/4251552549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c8101c00 0x7f84c8197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:39:57.190 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84ccce8640 1 -- 172.21.15.174:0/4251552549 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c819fa50 con 0x7f84c81037c0 2024-09-10T08:39:57.190 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.167+0000 7f84ccce8640 1 --2- 172.21.15.174:0/4251552549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c81037c0 0x7f84c819f3e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f84c000eb10 tx=0x7f84c000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:57.190 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.168+0000 7f84c5ffb640 1 -- 172.21.15.174:0/4251552549 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0004b20 con 0x7f84c81037c0 2024-09-10T08:39:57.191 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.168+0000 7f84ce74b640 1 -- 172.21.15.174:0/4251552549 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c819fd40 con 0x7f84c81037c0 2024-09-10T08:39:57.191 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.168+0000 7f84c5ffb640 1 -- 172.21.15.174:0/4251552549 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84c0004cc0 con 0x7f84c81037c0 2024-09-10T08:39:57.191 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.168+0000 7f84c5ffb640 1 -- 172.21.15.174:0/4251552549 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0010690 con 0x7f84c81037c0 2024-09-10T08:39:57.191 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.168+0000 7f84ce74b640 1 -- 172.21.15.174:0/4251552549 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c81a0270 con 0x7f84c81037c0 2024-09-10T08:39:57.191 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.169+0000 7f84ce74b640 1 -- 172.21.15.174:0/4251552549 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c8109040 con 0x7f84c81037c0 2024-09-10T08:39:57.191 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.170+0000 7f84c5ffb640 1 -- 172.21.15.174:0/4251552549 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f84c0005000 con 0x7f84c81037c0 2024-09-10T08:39:57.191 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.171+0000 7f84c5ffb640 1 --2- 172.21.15.174:0/4251552549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84b40a2bb0 0x7f84b40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:39:57.191 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.171+0000 7f84c5ffb640 1 -- 172.21.15.174:0/4251552549 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3702+0+0 (secure 0 0 0) 0x7f84c00ce570 con 0x7f84c81037c0 2024-09-10T08:39:57.191 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.171+0000 7f84c7fff640 1 --2- 172.21.15.174:0/4251552549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84b40a2bb0 0x7f84b40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:39:57.191 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.171+0000 7f84c7fff640 1 --2- 172.21.15.174:0/4251552549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84b40a2bb0 0x7f84b40a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f84b8004770 tx=0x7f84b8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:39:57.192 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.175+0000 7f84c5ffb640 1 -- 172.21.15.174:0/4251552549 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84c009c630 con 0x7f84c81037c0 2024-09-10T08:39:57.356 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:39:57.356+0000 7f84ce74b640 1 -- 172.21.15.174:0/4251552549 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f84c8062630 con 0x7f84b40a2bb0 2024-09-10T08:39:57.357 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:57 smithi174 ceph-mon[34068]: from='osd.3 [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-10T08:39:57.357 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:57 smithi174 ceph-mon[34068]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T08:39:57.357 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:39:57.357 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:39:57.357 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:57.357 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:57.357 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:39:57.357 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:39:57.357 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:57.357 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:57.358 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:57.358 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:57.358 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:57 smithi174 ceph-mon[34068]: osd.3 [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480] boot 2024-09-10T08:39:57.358 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:57 smithi174 ceph-mon[34068]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T08:39:57.358 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:39:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:57 smithi191 ceph-mon[35314]: from='osd.3 [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-10T08:39:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:57 smithi191 ceph-mon[35314]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T08:39:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:39:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:39:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:39:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:39:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:57.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:57.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:57.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:57 smithi191 ceph-mon[35314]: osd.3 [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480] boot 2024-09-10T08:39:57.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:57 smithi191 ceph-mon[35314]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T08:39:57.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:39:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:57 smithi172 ceph-mon[30854]: from='osd.3 [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-10T08:39:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:57 smithi172 ceph-mon[30854]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T08:39:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:39:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:39:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:57.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:57.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:39:57.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:39:57.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:57.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:57.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:39:57.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:39:57.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:57 smithi172 ceph-mon[30854]: osd.3 [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480] boot 2024-09-10T08:39:57.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:57 smithi172 ceph-mon[30854]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T08:39:57.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T08:39:58.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:58 smithi174 ceph-mon[34068]: purged_snaps scrub starts 2024-09-10T08:39:58.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:58 smithi174 ceph-mon[34068]: purged_snaps scrub ok 2024-09-10T08:39:58.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:58 smithi174 ceph-mon[34068]: Detected new or changed devices on smithi172 2024-09-10T08:39:58.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:58 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi172 to 8544M 2024-09-10T08:39:58.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:58 smithi174 ceph-mon[34068]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:58.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:39:58.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:39:58.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:58.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:58 smithi174 ceph-mon[34068]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T08:39:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:58 smithi191 ceph-mon[35314]: purged_snaps scrub starts 2024-09-10T08:39:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:58 smithi191 ceph-mon[35314]: purged_snaps scrub ok 2024-09-10T08:39:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:58 smithi191 ceph-mon[35314]: Detected new or changed devices on smithi172 2024-09-10T08:39:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:58 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi172 to 8544M 2024-09-10T08:39:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:58 smithi191 ceph-mon[35314]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:58 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:39:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:58 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:39:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:58 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:58 smithi191 ceph-mon[35314]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T08:39:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:58 smithi172 ceph-mon[30854]: purged_snaps scrub starts 2024-09-10T08:39:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:58 smithi172 ceph-mon[30854]: purged_snaps scrub ok 2024-09-10T08:39:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:58 smithi172 ceph-mon[30854]: Detected new or changed devices on smithi172 2024-09-10T08:39:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:58 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi172 to 8544M 2024-09-10T08:39:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:58 smithi172 ceph-mon[30854]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-10T08:39:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:58 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:39:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:58 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:39:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:58 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:39:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:58 smithi172 ceph-mon[30854]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T08:39:59.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:59 smithi174 ceph-mon[34068]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:59.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:39:59 smithi174 ceph-mon[34068]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T08:39:59.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:59 smithi191 ceph-mon[35314]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:59.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:39:59 smithi191 ceph-mon[35314]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T08:39:59.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:59 smithi172 ceph-mon[30854]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:39:59.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:39:59 smithi172 ceph-mon[30854]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T08:40:00.400 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:00 smithi174 ceph-mon[34068]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:00.400 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:00 smithi174 ceph-mon[34068]: from='client.? 172.21.15.174:0/686579094' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c2c1d40-72ad-4a15-ab72-54871c7e2427"}]: dispatch 2024-09-10T08:40:00.400 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:00 smithi174 ceph-mon[34068]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c2c1d40-72ad-4a15-ab72-54871c7e2427"}]: dispatch 2024-09-10T08:40:00.400 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:00 smithi174 ceph-mon[34068]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c2c1d40-72ad-4a15-ab72-54871c7e2427"}]': finished 2024-09-10T08:40:00.401 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:00 smithi174 ceph-mon[34068]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T08:40:00.401 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:00 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:00.401 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:00 smithi174 ceph-mon[34068]: overall HEALTH_OK 2024-09-10T08:40:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:00 smithi191 ceph-mon[35314]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:00 smithi191 ceph-mon[35314]: from='client.? 172.21.15.174:0/686579094' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c2c1d40-72ad-4a15-ab72-54871c7e2427"}]: dispatch 2024-09-10T08:40:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:00 smithi191 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c2c1d40-72ad-4a15-ab72-54871c7e2427"}]: dispatch 2024-09-10T08:40:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:00 smithi191 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c2c1d40-72ad-4a15-ab72-54871c7e2427"}]': finished 2024-09-10T08:40:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:00 smithi191 ceph-mon[35314]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T08:40:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:00 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:00 smithi191 ceph-mon[35314]: overall HEALTH_OK 2024-09-10T08:40:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:00 smithi172 ceph-mon[30854]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:00 smithi172 ceph-mon[30854]: from='client.? 172.21.15.174:0/686579094' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c2c1d40-72ad-4a15-ab72-54871c7e2427"}]: dispatch 2024-09-10T08:40:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:00 smithi172 ceph-mon[30854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8c2c1d40-72ad-4a15-ab72-54871c7e2427"}]: dispatch 2024-09-10T08:40:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:00 smithi172 ceph-mon[30854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8c2c1d40-72ad-4a15-ab72-54871c7e2427"}]': finished 2024-09-10T08:40:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:00 smithi172 ceph-mon[30854]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T08:40:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:00 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:00 smithi172 ceph-mon[30854]: overall HEALTH_OK 2024-09-10T08:40:01.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:01 smithi174 ceph-mon[34068]: from='client.? 172.21.15.174:0/3098939111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:40:01.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:01 smithi174 ceph-mon[34068]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T08:40:01.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:01 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:01.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:01 smithi191 ceph-mon[35314]: from='client.? 172.21.15.174:0/3098939111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:40:01.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:01 smithi191 ceph-mon[35314]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T08:40:01.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:01 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:01.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:01 smithi172 ceph-mon[30854]: from='client.? 172.21.15.174:0/3098939111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:40:01.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:01 smithi172 ceph-mon[30854]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T08:40:01.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:01 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:02.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:02 smithi174 ceph-mon[34068]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:02.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:02 smithi174 ceph-mon[34068]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T08:40:02.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:02 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:02 smithi191 ceph-mon[35314]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:02 smithi191 ceph-mon[35314]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T08:40:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:02 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:02.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:02 smithi172 ceph-mon[30854]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:02.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:02 smithi172 ceph-mon[30854]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T08:40:02.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:02 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:04.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:04 smithi174 ceph-mon[34068]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T08:40:04.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:04 smithi191 ceph-mon[35314]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T08:40:04.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:04 smithi172 ceph-mon[30854]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T08:40:06.405 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:06 smithi174 ceph-mon[34068]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-10T08:40:06.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:06 smithi191 ceph-mon[35314]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-10T08:40:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:06 smithi172 ceph-mon[30854]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-10T08:40:08.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:08 smithi174 ceph-mon[34068]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T08:40:08.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:08 smithi191 ceph-mon[35314]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T08:40:08.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:08 smithi172 ceph-mon[30854]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T08:40:09.483 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:09 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T08:40:09.484 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:09 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:09.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:09 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T08:40:09.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:09 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:09.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:09 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T08:40:09.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:09 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:10.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:10 smithi174 ceph-mon[34068]: Deploying daemon osd.4 on smithi174 2024-09-10T08:40:10.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:10 smithi174 ceph-mon[34068]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-10T08:40:10.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:10 smithi191 ceph-mon[35314]: Deploying daemon osd.4 on smithi174 2024-09-10T08:40:10.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:10 smithi191 ceph-mon[35314]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-10T08:40:10.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:10 smithi172 ceph-mon[30854]: Deploying daemon osd.4 on smithi174 2024-09-10T08:40:10.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:10 smithi172 ceph-mon[30854]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-10T08:40:12.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:12 smithi174 ceph-mon[34068]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T08:40:12.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:12 smithi191 ceph-mon[35314]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T08:40:12.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:12 smithi172 ceph-mon[30854]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T08:40:14.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:14 smithi174 ceph-mon[34068]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T08:40:14.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:14 smithi191 ceph-mon[35314]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T08:40:14.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:14 smithi172 ceph-mon[30854]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T08:40:15.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:15 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:40:15.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:15 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:15.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:15 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:15.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:15 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:40:15.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:15 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:15.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:15 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:15.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:15 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:40:15.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:15 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:15.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:15 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:16.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:16 smithi174 ceph-mon[34068]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:16.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:16 smithi191 ceph-mon[35314]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:16.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:16 smithi172 ceph-mon[30854]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:18.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:18 smithi174 ceph-mon[34068]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:18.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:18.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:18.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:18.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:40:18.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:18.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:18 smithi191 ceph-mon[35314]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:18.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:18.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:18.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:18.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:40:18.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:18 smithi172 ceph-mon[30854]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:40:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:19.132 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:40:19.132+0000 7f84c5ffb640 1 -- 172.21.15.174:0/4251552549 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f84c8062630 con 0x7f84b40a2bb0 2024-09-10T08:40:19.133 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 4 on host 'smithi174' 2024-09-10T08:40:19.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:40:19.134+0000 7f84ce74b640 1 -- 172.21.15.174:0/4251552549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84b40a2bb0 msgr2=0x7f84b40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:19.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:40:19.134+0000 7f84ce74b640 1 --2- 172.21.15.174:0/4251552549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84b40a2bb0 0x7f84b40a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f84b8004770 tx=0x7f84b8000bc0 comp rx=0 tx=0).stop 2024-09-10T08:40:19.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:40:19.134+0000 7f84ce74b640 1 -- 172.21.15.174:0/4251552549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c81037c0 msgr2=0x7f84c819f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:19.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:40:19.134+0000 7f84ce74b640 1 --2- 172.21.15.174:0/4251552549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c81037c0 0x7f84c819f3e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f84c000eb10 tx=0x7f84c000c2d0 comp rx=0 tx=0).stop 2024-09-10T08:40:19.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:40:19.134+0000 7f84ce74b640 1 -- 172.21.15.174:0/4251552549 shutdown_connections 2024-09-10T08:40:19.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:40:19.134+0000 7f84ce74b640 1 --2- 172.21.15.174:0/4251552549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84b40a2bb0 0x7f84b40a5070 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:19.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:40:19.134+0000 7f84ce74b640 1 --2- 172.21.15.174:0/4251552549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c81037c0 0x7f84c819f3e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:19.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:40:19.134+0000 7f84ce74b640 1 --2- 172.21.15.174:0/4251552549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84c8102e00 0x7f84c8198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:19.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:40:19.134+0000 7f84ce74b640 1 --2- 172.21.15.174:0/4251552549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84c8101c00 0x7f84c8197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:19.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:40:19.134+0000 7f84ce74b640 1 -- 172.21.15.174:0/4251552549 >> 172.21.15.174:0/4251552549 conn(0x7f84c80fd390 msgr2=0x7f84c8106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:19.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:40:19.134+0000 7f84ce74b640 1 -- 172.21.15.174:0/4251552549 shutdown_connections 2024-09-10T08:40:19.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:40:19.134+0000 7f84ce74b640 1 -- 172.21.15.174:0/4251552549 wait complete. 2024-09-10T08:40:19.452 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:40:19.452 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:19.452 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:19.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:40:19.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:19.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:19.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:40:19.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:19.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:19.762 DEBUG:teuthology.orchestra.run.smithi174:osd.4> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T08:40:19.765 INFO:tasks.cephadm:Deploying osd.5 on smithi191 with /dev/vg_nvme/lv_3... 2024-09-10T08:40:19.765 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T08:40:20.019 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:40:20.330 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:20 smithi174 ceph-mon[34068]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:20.394 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:20 smithi191 ceph-mon[35314]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:20.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:20 smithi172 ceph-mon[30854]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:20.976 INFO:journalctl@ceph.osd.4.smithi174.stdout:Sep 10 08:40:20 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-4[44962]: 2024-09-10T08:40:20.683+0000 7fd5c0548640 -1 osd.4 0 log_to_monitors true 2024-09-10T08:40:21.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:21 smithi174 ceph-mon[34068]: from='osd.4 [v2:172.21.15.174:6808/961007949,v1:172.21.15.174:6809/961007949]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:40:21.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:21 smithi174 ceph-mon[34068]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:40:21.459 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:21 smithi191 ceph-mon[35314]: from='osd.4 [v2:172.21.15.174:6808/961007949,v1:172.21.15.174:6809/961007949]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:40:21.459 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:21 smithi191 ceph-mon[35314]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:40:21.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:21 smithi172 ceph-mon[30854]: from='osd.4 [v2:172.21.15.174:6808/961007949,v1:172.21.15.174:6809/961007949]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:40:21.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:21 smithi172 ceph-mon[30854]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T08:40:22.350 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:22 smithi174 ceph-mon[34068]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:22.350 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:22 smithi174 ceph-mon[34068]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T08:40:22.350 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:22 smithi174 ceph-mon[34068]: from='osd.4 [v2:172.21.15.174:6808/961007949,v1:172.21.15.174:6809/961007949]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:40:22.350 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:22 smithi174 ceph-mon[34068]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T08:40:22.351 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:22 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:22.351 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:22 smithi174 ceph-mon[34068]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:40:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:22 smithi191 ceph-mon[35314]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:22 smithi191 ceph-mon[35314]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T08:40:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:22 smithi191 ceph-mon[35314]: from='osd.4 [v2:172.21.15.174:6808/961007949,v1:172.21.15.174:6809/961007949]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:40:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:22 smithi191 ceph-mon[35314]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T08:40:22.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:22 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:22.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:22 smithi191 ceph-mon[35314]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:40:22.576 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:40:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:22 smithi172 ceph-mon[30854]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:22 smithi172 ceph-mon[30854]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T08:40:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:22 smithi172 ceph-mon[30854]: from='osd.4 [v2:172.21.15.174:6808/961007949,v1:172.21.15.174:6809/961007949]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:40:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:22 smithi172 ceph-mon[30854]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T08:40:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:22 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:22 smithi172 ceph-mon[30854]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:40:22.602 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi191:vg_nvme/lv_3 2024-09-10T08:40:22.856 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:40:22.991 INFO:journalctl@ceph.osd.4.smithi174.stdout:Sep 10 08:40:22 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-4[44962]: 2024-09-10T08:40:22.888+0000 7fd5bc4cb640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T08:40:22.991 INFO:journalctl@ceph.osd.4.smithi174.stdout:Sep 10 08:40:22 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-4[44962]: 2024-09-10T08:40:22.893+0000 7fd5b42eb640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:40:23.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.321+0000 7f4d80461640 1 -- 172.21.15.191:0/2983510841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d78102e00 msgr2=0x7f4d78103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:23.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.321+0000 7f4d80461640 1 --2- 172.21.15.191:0/2983510841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d78102e00 0x7f4d78103280 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009900 tx=0x7f4d7402a410 comp rx=0 tx=0).stop 2024-09-10T08:40:23.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.321+0000 7f4d80461640 1 -- 172.21.15.191:0/2983510841 shutdown_connections 2024-09-10T08:40:23.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.321+0000 7f4d80461640 1 --2- 172.21.15.191:0/2983510841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d781037c0 0x7f4d7810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:23.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.321+0000 7f4d80461640 1 --2- 172.21.15.191:0/2983510841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d78102e00 0x7f4d78103280 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:23.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.321+0000 7f4d80461640 1 --2- 172.21.15.191:0/2983510841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d78101c00 0x7f4d78102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:23.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.321+0000 7f4d80461640 1 -- 172.21.15.191:0/2983510841 >> 172.21.15.191:0/2983510841 conn(0x7f4d780fd390 msgr2=0x7f4d780ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:23.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.321+0000 7f4d80461640 1 -- 172.21.15.191:0/2983510841 shutdown_connections 2024-09-10T08:40:23.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.321+0000 7f4d80461640 1 -- 172.21.15.191:0/2983510841 wait complete. 2024-09-10T08:40:23.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d80461640 1 Processor -- start 2024-09-10T08:40:23.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d80461640 1 -- start start 2024-09-10T08:40:23.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d80461640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d78101c00 0x7f4d78075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:23.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d80461640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d78102e00 0x7f4d780745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:23.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d80461640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d781037c0 0x7f4d78074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:23.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d80461640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7810d730 con 0x7f4d78102e00 2024-09-10T08:40:23.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d80461640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7810d5b0 con 0x7f4d78101c00 2024-09-10T08:40:23.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d80461640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7810d8b0 con 0x7f4d781037c0 2024-09-10T08:40:23.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d7e9fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d781037c0 0x7f4d78074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:23.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d7e9fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d781037c0 0x7f4d78074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:39884/0 (socket says 172.21.15.191:39884) 2024-09-10T08:40:23.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d7e9fe640 1 -- 172.21.15.191:0/474927355 learned_addr learned my addr 172.21.15.191:0/474927355 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-10T08:40:23.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d7e1fd640 1 --2- 172.21.15.191:0/474927355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d78101c00 0x7f4d78075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:23.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d7d9fc640 1 --2- 172.21.15.191:0/474927355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d78102e00 0x7f4d780745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:23.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d7e9fe640 1 -- 172.21.15.191:0/474927355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d78101c00 msgr2=0x7f4d78075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:23.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d7e9fe640 1 --2- 172.21.15.191:0/474927355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d78101c00 0x7f4d78075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:23.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d7e9fe640 1 -- 172.21.15.191:0/474927355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d78102e00 msgr2=0x7f4d780745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:23.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d7e9fe640 1 --2- 172.21.15.191:0/474927355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d78102e00 0x7f4d780745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:23.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.322+0000 7f4d7e9fe640 1 -- 172.21.15.191:0/474927355 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d780753e0 con 0x7f4d781037c0 2024-09-10T08:40:23.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.323+0000 7f4d7e9fe640 1 --2- 172.21.15.191:0/474927355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d781037c0 0x7f4d78074b20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000eaa0 tx=0x7f4d7000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:23.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.323+0000 7f4d6f7fe640 1 -- 172.21.15.191:0/474927355 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7000cdb0 con 0x7f4d781037c0 2024-09-10T08:40:23.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.323+0000 7f4d6f7fe640 1 -- 172.21.15.191:0/474927355 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d70005020 con 0x7f4d781037c0 2024-09-10T08:40:23.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.323+0000 7f4d80461640 1 -- 172.21.15.191:0/474927355 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d781a1f50 con 0x7f4d781037c0 2024-09-10T08:40:23.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.323+0000 7f4d6f7fe640 1 -- 172.21.15.191:0/474927355 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d700116d0 con 0x7f4d781037c0 2024-09-10T08:40:23.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.323+0000 7f4d80461640 1 -- 172.21.15.191:0/474927355 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d781a2400 con 0x7f4d781037c0 2024-09-10T08:40:23.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.324+0000 7f4d80461640 1 -- 172.21.15.191:0/474927355 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d78109050 con 0x7f4d781037c0 2024-09-10T08:40:23.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.326+0000 7f4d6f7fe640 1 -- 172.21.15.191:0/474927355 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4d700051d0 con 0x7f4d781037c0 2024-09-10T08:40:23.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.326+0000 7f4d6f7fe640 1 --2- 172.21.15.191:0/474927355 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4d640a2b80 0x7f4d640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:23.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.327+0000 7f4d6f7fe640 1 -- 172.21.15.191:0/474927355 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 3821+0+0 (secure 0 0 0) 0x7f4d700ce3e0 con 0x7f4d781037c0 2024-09-10T08:40:23.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.327+0000 7f4d7e1fd640 1 --2- 172.21.15.191:0/474927355 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4d640a2b80 0x7f4d640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:23.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.327+0000 7f4d7e1fd640 1 --2- 172.21.15.191:0/474927355 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4d640a2b80 0x7f4d640a5040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d78102c60 tx=0x7f4d68005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:23.340 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.329+0000 7f4d6f7fe640 1 -- 172.21.15.191:0/474927355 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d70016030 con 0x7f4d781037c0 2024-09-10T08:40:23.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:23 smithi174 ceph-mon[34068]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T08:40:23.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:23 smithi174 ceph-mon[34068]: osdmap e32: 5 total, 4 up, 5 in 2024-09-10T08:40:23.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:23.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:23.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:23.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:23.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:40:23.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:40:23.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:23.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:23.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:40:23.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:23.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:23 smithi174 ceph-mon[34068]: from='osd.4 ' entity='osd.4' 2024-09-10T08:40:23.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:23.507 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:23.506+0000 7f4d80461640 1 -- 172.21.15.191:0/474927355 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4d78062630 con 0x7f4d640a2b80 2024-09-10T08:40:23.507 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:23 smithi191 ceph-mon[35314]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T08:40:23.507 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:23 smithi191 ceph-mon[35314]: osdmap e32: 5 total, 4 up, 5 in 2024-09-10T08:40:23.507 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:23.507 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:23.507 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:23.507 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:23.507 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:40:23.508 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:40:23.508 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:23.508 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:23.508 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:40:23.508 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:23.508 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:23 smithi191 ceph-mon[35314]: from='osd.4 ' entity='osd.4' 2024-09-10T08:40:23.508 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:23.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:23 smithi172 ceph-mon[30854]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T08:40:23.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:23 smithi172 ceph-mon[30854]: osdmap e32: 5 total, 4 up, 5 in 2024-09-10T08:40:23.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:23.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:23.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:23.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:23.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:40:23.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:40:23.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:23.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:23.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:40:23.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:23.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:23 smithi172 ceph-mon[30854]: from='osd.4 ' entity='osd.4' 2024-09-10T08:40:23.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:24.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:24 smithi174 ceph-mon[34068]: purged_snaps scrub starts 2024-09-10T08:40:24.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:24 smithi174 ceph-mon[34068]: purged_snaps scrub ok 2024-09-10T08:40:24.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:24 smithi174 ceph-mon[34068]: Detected new or changed devices on smithi174 2024-09-10T08:40:24.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:24 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-10T08:40:24.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:24 smithi174 ceph-mon[34068]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:24.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:24 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:40:24.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:24 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:40:24.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:24 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:24.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:24 smithi174 ceph-mon[34068]: osd.4 [v2:172.21.15.174:6808/961007949,v1:172.21.15.174:6809/961007949] boot 2024-09-10T08:40:24.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:24 smithi174 ceph-mon[34068]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T08:40:24.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:24 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:24 smithi191 ceph-mon[35314]: purged_snaps scrub starts 2024-09-10T08:40:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:24 smithi191 ceph-mon[35314]: purged_snaps scrub ok 2024-09-10T08:40:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:24 smithi191 ceph-mon[35314]: Detected new or changed devices on smithi174 2024-09-10T08:40:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:24 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-10T08:40:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:24 smithi191 ceph-mon[35314]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:24.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:24 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:40:24.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:24 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:40:24.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:24 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:24.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:24 smithi191 ceph-mon[35314]: osd.4 [v2:172.21.15.174:6808/961007949,v1:172.21.15.174:6809/961007949] boot 2024-09-10T08:40:24.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:24 smithi191 ceph-mon[35314]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T08:40:24.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:24 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:24.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:24 smithi172 ceph-mon[30854]: purged_snaps scrub starts 2024-09-10T08:40:24.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:24 smithi172 ceph-mon[30854]: purged_snaps scrub ok 2024-09-10T08:40:24.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:24 smithi172 ceph-mon[30854]: Detected new or changed devices on smithi174 2024-09-10T08:40:24.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:24 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-10T08:40:24.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:24 smithi172 ceph-mon[30854]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-10T08:40:24.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:24 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:40:24.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:24 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:40:24.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:24 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:24.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:24 smithi172 ceph-mon[30854]: osd.4 [v2:172.21.15.174:6808/961007949,v1:172.21.15.174:6809/961007949] boot 2024-09-10T08:40:24.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:24 smithi172 ceph-mon[30854]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T08:40:24.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:24 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T08:40:25.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:25 smithi174 ceph-mon[34068]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:40:25.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:25 smithi191 ceph-mon[35314]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:40:25.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:25 smithi172 ceph-mon[30854]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:40:26.479 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:26 smithi191 ceph-mon[35314]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:26.479 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:26 smithi191 ceph-mon[35314]: osdmap e34: 5 total, 5 up, 5 in 2024-09-10T08:40:26.479 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:26 smithi191 ceph-mon[35314]: from='client.? 172.21.15.191:0/3705801831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4de45a1d-ef0e-4bfe-ba15-5eeeb8806155"}]: dispatch 2024-09-10T08:40:26.479 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:26 smithi191 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4de45a1d-ef0e-4bfe-ba15-5eeeb8806155"}]: dispatch 2024-09-10T08:40:26.480 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:26 smithi191 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4de45a1d-ef0e-4bfe-ba15-5eeeb8806155"}]': finished 2024-09-10T08:40:26.480 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:26 smithi191 ceph-mon[35314]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T08:40:26.480 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:26 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:26.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:26 smithi174 ceph-mon[34068]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:26.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:26 smithi174 ceph-mon[34068]: osdmap e34: 5 total, 5 up, 5 in 2024-09-10T08:40:26.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:26 smithi174 ceph-mon[34068]: from='client.? 172.21.15.191:0/3705801831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4de45a1d-ef0e-4bfe-ba15-5eeeb8806155"}]: dispatch 2024-09-10T08:40:26.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:26 smithi174 ceph-mon[34068]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4de45a1d-ef0e-4bfe-ba15-5eeeb8806155"}]: dispatch 2024-09-10T08:40:26.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:26 smithi174 ceph-mon[34068]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4de45a1d-ef0e-4bfe-ba15-5eeeb8806155"}]': finished 2024-09-10T08:40:26.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:26 smithi174 ceph-mon[34068]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T08:40:26.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:26 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:26 smithi172 ceph-mon[30854]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:26 smithi172 ceph-mon[30854]: osdmap e34: 5 total, 5 up, 5 in 2024-09-10T08:40:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:26 smithi172 ceph-mon[30854]: from='client.? 172.21.15.191:0/3705801831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4de45a1d-ef0e-4bfe-ba15-5eeeb8806155"}]: dispatch 2024-09-10T08:40:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:26 smithi172 ceph-mon[30854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4de45a1d-ef0e-4bfe-ba15-5eeeb8806155"}]: dispatch 2024-09-10T08:40:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:26 smithi172 ceph-mon[30854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4de45a1d-ef0e-4bfe-ba15-5eeeb8806155"}]': finished 2024-09-10T08:40:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:26 smithi172 ceph-mon[30854]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T08:40:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:26 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:27.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:27 smithi191 ceph-mon[35314]: from='client.? 172.21.15.191:0/3804313242' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:40:27.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:27 smithi191 ceph-mon[35314]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:27.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:27 smithi172 ceph-mon[30854]: from='client.? 172.21.15.191:0/3804313242' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:40:27.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:27 smithi172 ceph-mon[30854]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:27.623 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:27 smithi174 ceph-mon[34068]: from='client.? 172.21.15.191:0/3804313242' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:40:27.624 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:27 smithi174 ceph-mon[34068]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:30.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:30 smithi174 ceph-mon[34068]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:30.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:30 smithi191 ceph-mon[35314]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:30.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:30 smithi172 ceph-mon[30854]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:32.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:32 smithi191 ceph-mon[35314]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:32.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:32 smithi174 ceph-mon[34068]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:32.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:32 smithi172 ceph-mon[30854]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:34.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:34 smithi191 ceph-mon[35314]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:34.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:34 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T08:40:34.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:34 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:34.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:34 smithi174 ceph-mon[34068]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:34.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:34 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T08:40:34.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:34 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:34.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:34 smithi172 ceph-mon[30854]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:34.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:34 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T08:40:34.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:34 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:35.424 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:35 smithi191 ceph-mon[35314]: Deploying daemon osd.5 on smithi191 2024-09-10T08:40:35.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:35 smithi174 ceph-mon[34068]: Deploying daemon osd.5 on smithi191 2024-09-10T08:40:35.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:35 smithi172 ceph-mon[30854]: Deploying daemon osd.5 on smithi191 2024-09-10T08:40:36.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:36 smithi191 ceph-mon[35314]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:36.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:36 smithi174 ceph-mon[34068]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:36 smithi172 ceph-mon[30854]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:38.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:38 smithi174 ceph-mon[34068]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:38.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:38 smithi191 ceph-mon[35314]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:38.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:38 smithi172 ceph-mon[30854]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:40.462 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:40 smithi191 ceph-mon[35314]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:40.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:40 smithi174 ceph-mon[34068]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:40.537 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:40 smithi172 ceph-mon[30854]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:41.224 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:41 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:40:41.224 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:41 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:41.224 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:41 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:41.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:41 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:40:41.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:41 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:41.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:41 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:41 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:40:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:41 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:41 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:42.233 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:42 smithi191 ceph-mon[35314]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:42.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:42 smithi174 ceph-mon[34068]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:42 smithi172 ceph-mon[30854]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:44.249 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:44 smithi191 ceph-mon[35314]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:44.249 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:44 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:44.249 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:44 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:44.249 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:44 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:44.249 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:44 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:40:44.249 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:44 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:44.291 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:44.291+0000 7f4d6f7fe640 1 -- 172.21.15.191:0/474927355 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4d78062630 con 0x7f4d640a2b80 2024-09-10T08:40:44.292 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 5 on host 'smithi191' 2024-09-10T08:40:44.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:44.294+0000 7f4d80461640 1 -- 172.21.15.191:0/474927355 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4d640a2b80 msgr2=0x7f4d640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:44.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:44.294+0000 7f4d80461640 1 --2- 172.21.15.191:0/474927355 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4d640a2b80 0x7f4d640a5040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d78102c60 tx=0x7f4d68005f90 comp rx=0 tx=0).stop 2024-09-10T08:40:44.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:44.294+0000 7f4d80461640 1 -- 172.21.15.191:0/474927355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d781037c0 msgr2=0x7f4d78074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:44.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:44.294+0000 7f4d80461640 1 --2- 172.21.15.191:0/474927355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d781037c0 0x7f4d78074b20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000eaa0 tx=0x7f4d7000ef60 comp rx=0 tx=0).stop 2024-09-10T08:40:44.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:44.294+0000 7f4d80461640 1 -- 172.21.15.191:0/474927355 shutdown_connections 2024-09-10T08:40:44.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:44.294+0000 7f4d80461640 1 --2- 172.21.15.191:0/474927355 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4d640a2b80 0x7f4d640a5040 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:44.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:44.294+0000 7f4d80461640 1 --2- 172.21.15.191:0/474927355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d781037c0 0x7f4d78074b20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:44.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:44.294+0000 7f4d80461640 1 --2- 172.21.15.191:0/474927355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d78102e00 0x7f4d780745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:44.294 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:44.294+0000 7f4d80461640 1 --2- 172.21.15.191:0/474927355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d78101c00 0x7f4d78075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:44.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:44.294+0000 7f4d80461640 1 -- 172.21.15.191:0/474927355 >> 172.21.15.191:0/474927355 conn(0x7f4d780fd390 msgr2=0x7f4d78106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:44.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:44.294+0000 7f4d80461640 1 -- 172.21.15.191:0/474927355 shutdown_connections 2024-09-10T08:40:44.295 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:40:44.294+0000 7f4d80461640 1 -- 172.21.15.191:0/474927355 wait complete. 2024-09-10T08:40:44.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:44 smithi174 ceph-mon[34068]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:44.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:44 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:44.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:44 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:44.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:44 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:44.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:44 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:40:44.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:44 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:44.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:44 smithi172 ceph-mon[30854]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:44.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:44.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:44.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:44.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:40:44.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:44.972 DEBUG:teuthology.orchestra.run.smithi191:osd.5> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T08:40:44.975 INFO:tasks.cephadm:Deploying osd.6 on smithi172 with /dev/vg_nvme/lv_2... 2024-09-10T08:40:44.975 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T08:40:45.233 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:45.255 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:40:45.255 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:45.255 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:45.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:40:45.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:45.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:45.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:40:45.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:45.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:46.378 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:46 smithi191 ceph-mon[35314]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:46.483 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:46 smithi172 ceph-mon[30854]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:46.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:46 smithi174 ceph-mon[34068]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:47.050 INFO:journalctl@ceph.osd.5.smithi191.stdout:Sep 10 08:40:46 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-5[46711]: 2024-09-10T08:40:46.774+0000 7fbd8d298640 -1 osd.5 0 log_to_monitors true 2024-09-10T08:40:47.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:47 smithi172 ceph-mon[30854]: from='osd.5 [v2:172.21.15.191:6808/4216938371,v1:172.21.15.191:6809/4216938371]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:40:47.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:47 smithi172 ceph-mon[30854]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:40:47.409 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:47 smithi191 ceph-mon[35314]: from='osd.5 [v2:172.21.15.191:6808/4216938371,v1:172.21.15.191:6809/4216938371]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:40:47.409 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:47 smithi191 ceph-mon[35314]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:40:47.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:47 smithi174 ceph-mon[34068]: from='osd.5 [v2:172.21.15.191:6808/4216938371,v1:172.21.15.191:6809/4216938371]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:40:47.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:47 smithi174 ceph-mon[34068]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T08:40:47.724 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:40:47.748 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_2 2024-09-10T08:40:48.005 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:40:48.392 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:48 smithi172 ceph-mon[30854]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:48.392 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:48 smithi172 ceph-mon[30854]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T08:40:48.392 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:48 smithi172 ceph-mon[30854]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T08:40:48.392 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:48.392 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:48 smithi172 ceph-mon[30854]: from='osd.5 [v2:172.21.15.191:6808/4216938371,v1:172.21.15.191:6809/4216938371]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:40:48.392 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:48 smithi172 ceph-mon[30854]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:40:48.392 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:48.392 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:48.392 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:40:48.393 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:40:48.393 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:48.393 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:48.393 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:40:48.393 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:48.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:48 smithi174 ceph-mon[34068]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:48.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:48 smithi174 ceph-mon[34068]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T08:40:48.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:48 smithi174 ceph-mon[34068]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T08:40:48.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:48.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:48 smithi174 ceph-mon[34068]: from='osd.5 [v2:172.21.15.191:6808/4216938371,v1:172.21.15.191:6809/4216938371]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:40:48.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:48 smithi174 ceph-mon[34068]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:40:48.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:48.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:48.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:40:48.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:40:48.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:48.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:48.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:40:48.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:48.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.475+0000 7f9785812640 1 -- 172.21.15.172:0/4226871721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9780100b10 msgr2=0x7f9780100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:48.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.475+0000 7f9785812640 1 --2- 172.21.15.172:0/4226871721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9780100b10 0x7f9780100f10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9770009900 tx=0x7f977002a410 comp rx=0 tx=0).stop 2024-09-10T08:40:48.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.475+0000 7f9785812640 1 -- 172.21.15.172:0/4226871721 shutdown_connections 2024-09-10T08:40:48.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.475+0000 7f9785812640 1 --2- 172.21.15.172:0/4226871721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97801026d0 0x7f9780108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:48.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.475+0000 7f9785812640 1 --2- 172.21.15.172:0/4226871721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9780101d10 0x7f9780102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:48.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.475+0000 7f9785812640 1 --2- 172.21.15.172:0/4226871721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9780100b10 0x7f9780100f10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:48.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.475+0000 7f9785812640 1 -- 172.21.15.172:0/4226871721 >> 172.21.15.172:0/4226871721 conn(0x7f97800fc2a0 msgr2=0x7f97800fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:40:48.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.475+0000 7f9785812640 1 -- 172.21.15.172:0/4226871721 shutdown_connections 2024-09-10T08:40:48.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.475+0000 7f9785812640 1 -- 172.21.15.172:0/4226871721 wait complete. 2024-09-10T08:40:48.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.476+0000 7f9785812640 1 Processor -- start 2024-09-10T08:40:48.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.476+0000 7f9785812640 1 -- start start 2024-09-10T08:40:48.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.476+0000 7f9785812640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9780100b10 0x7f9780197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:48.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.476+0000 7f9785812640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9780101d10 0x7f9780198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:48.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.476+0000 7f9785812640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97801026d0 0x7f978019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:48.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.476+0000 7f9785812640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f978010d7d0 con 0x7f9780100b10 2024-09-10T08:40:48.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.476+0000 7f9785812640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f978010d650 con 0x7f97801026d0 2024-09-10T08:40:48.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.476+0000 7f9785812640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f978010d950 con 0x7f9780101d10 2024-09-10T08:40:48.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.476+0000 7f977effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9780100b10 0x7f9780197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:48.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.476+0000 7f977effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9780100b10 0x7f9780197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46248/0 (socket says 172.21.15.172:46248) 2024-09-10T08:40:48.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.476+0000 7f977effd640 1 -- 172.21.15.172:0/3389667689 learned_addr learned my addr 172.21.15.172:0/3389667689 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:40:48.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.476+0000 7f977e7fc640 1 --2- 172.21.15.172:0/3389667689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9780101d10 0x7f9780198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:48.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.477+0000 7f977effd640 1 -- 172.21.15.172:0/3389667689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9780101d10 msgr2=0x7f9780198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:48.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.477+0000 7f977effd640 1 --2- 172.21.15.172:0/3389667689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9780101d10 0x7f9780198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:48.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.477+0000 7f977effd640 1 -- 172.21.15.172:0/3389667689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97801026d0 msgr2=0x7f978019f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:40:48.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.477+0000 7f977effd640 1 --2- 172.21.15.172:0/3389667689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97801026d0 0x7f978019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:40:48.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.477+0000 7f977effd640 1 -- 172.21.15.172:0/3389667689 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978019fb10 con 0x7f9780100b10 2024-09-10T08:40:48.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.477+0000 7f977effd640 1 --2- 172.21.15.172:0/3389667689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9780100b10 0x7f9780197ee0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f97700314e0 tx=0x7f9770031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:48.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.477+0000 7f9784810640 1 -- 172.21.15.172:0/3389667689 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977003f030 con 0x7f9780100b10 2024-09-10T08:40:48.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.477+0000 7f9785812640 1 -- 172.21.15.172:0/3389667689 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978019fda0 con 0x7f9780100b10 2024-09-10T08:40:48.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.477+0000 7f9784810640 1 -- 172.21.15.172:0/3389667689 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97700044b0 con 0x7f9780100b10 2024-09-10T08:40:48.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.477+0000 7f9784810640 1 -- 172.21.15.172:0/3389667689 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770035070 con 0x7f9780100b10 2024-09-10T08:40:48.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.477+0000 7f9785812640 1 -- 172.21.15.172:0/3389667689 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97801a0240 con 0x7f9780100b10 2024-09-10T08:40:48.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.478+0000 7f9785812640 1 -- 172.21.15.172:0/3389667689 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9780108000 con 0x7f9780100b10 2024-09-10T08:40:48.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.480+0000 7f9784810640 1 -- 172.21.15.172:0/3389667689 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9770042030 con 0x7f9780100b10 2024-09-10T08:40:48.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.480+0000 7f9784810640 1 --2- 172.21.15.172:0/3389667689 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97680a2bb0 0x7f97680a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:40:48.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.480+0000 7f977e7fc640 1 --2- 172.21.15.172:0/3389667689 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97680a2bb0 0x7f97680a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:40:48.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.480+0000 7f9784810640 1 -- 172.21.15.172:0/3389667689 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4253+0+0 (secure 0 0 0) 0x7f97700eeae0 con 0x7f9780100b10 2024-09-10T08:40:48.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.481+0000 7f977e7fc640 1 --2- 172.21.15.172:0/3389667689 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97680a2bb0 0x7f97680a5070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9780199400 tx=0x7f9774005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:40:48.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.484+0000 7f9784810640 1 -- 172.21.15.172:0/3389667689 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97700bc980 con 0x7f9780100b10 2024-09-10T08:40:48.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:48 smithi191 ceph-mon[35314]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:48.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:48 smithi191 ceph-mon[35314]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T08:40:48.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:48 smithi191 ceph-mon[35314]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T08:40:48.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:48.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:48 smithi191 ceph-mon[35314]: from='osd.5 [v2:172.21.15.191:6808/4216938371,v1:172.21.15.191:6809/4216938371]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:40:48.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:48 smithi191 ceph-mon[35314]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:40:48.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:48.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:48.552 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:40:48.552 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:40:48.552 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:48.552 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:48.552 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:40:48.552 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:40:48.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:40:48.658+0000 7f9785812640 1 -- 172.21.15.172:0/3389667689 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9780062630 con 0x7f97680a2bb0 2024-09-10T08:40:49.163 INFO:journalctl@ceph.osd.5.smithi191.stdout:Sep 10 08:40:48 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-5[46711]: 2024-09-10T08:40:48.812+0000 7fbd8921b640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T08:40:49.163 INFO:journalctl@ceph.osd.5.smithi191.stdout:Sep 10 08:40:48 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-5[46711]: 2024-09-10T08:40:48.820+0000 7fbd8103b640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:40:49.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:49 smithi172 ceph-mon[30854]: Detected new or changed devices on smithi191 2024-09-10T08:40:49.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:49 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi191 to 8544M 2024-09-10T08:40:49.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:49 smithi172 ceph-mon[30854]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-10T08:40:49.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:49 smithi172 ceph-mon[30854]: osdmap e37: 6 total, 5 up, 6 in 2024-09-10T08:40:49.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:49 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:49.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:49 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:49.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:49 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:40:49.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:49 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:40:49.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:49 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:49.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:49 smithi172 ceph-mon[30854]: from='osd.5 ' entity='osd.5' 2024-09-10T08:40:49.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:49 smithi174 ceph-mon[34068]: Detected new or changed devices on smithi191 2024-09-10T08:40:49.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:49 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi191 to 8544M 2024-09-10T08:40:49.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:49 smithi174 ceph-mon[34068]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-10T08:40:49.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:49 smithi174 ceph-mon[34068]: osdmap e37: 6 total, 5 up, 6 in 2024-09-10T08:40:49.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:49 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:49.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:49 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:49.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:49 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:40:49.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:49 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:40:49.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:49 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:49.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:49 smithi174 ceph-mon[34068]: from='osd.5 ' entity='osd.5' 2024-09-10T08:40:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:49 smithi191 ceph-mon[35314]: Detected new or changed devices on smithi191 2024-09-10T08:40:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:49 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi191 to 8544M 2024-09-10T08:40:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:49 smithi191 ceph-mon[35314]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-10T08:40:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:49 smithi191 ceph-mon[35314]: osdmap e37: 6 total, 5 up, 6 in 2024-09-10T08:40:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:49 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:49 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:49 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:40:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:49 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:40:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:49 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:49.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:49 smithi191 ceph-mon[35314]: from='osd.5 ' entity='osd.5' 2024-09-10T08:40:50.428 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:50 smithi172 ceph-mon[30854]: purged_snaps scrub starts 2024-09-10T08:40:50.429 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:50 smithi172 ceph-mon[30854]: purged_snaps scrub ok 2024-09-10T08:40:50.429 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:50 smithi172 ceph-mon[30854]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:40:50.429 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:50 smithi172 ceph-mon[30854]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:50.429 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:50 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:50.429 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:50 smithi172 ceph-mon[30854]: osd.5 [v2:172.21.15.191:6808/4216938371,v1:172.21.15.191:6809/4216938371] boot 2024-09-10T08:40:50.429 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:50 smithi172 ceph-mon[30854]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T08:40:50.429 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:50 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:50.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:50 smithi174 ceph-mon[34068]: purged_snaps scrub starts 2024-09-10T08:40:50.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:50 smithi174 ceph-mon[34068]: purged_snaps scrub ok 2024-09-10T08:40:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:50 smithi174 ceph-mon[34068]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:40:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:50 smithi174 ceph-mon[34068]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:50 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:50 smithi174 ceph-mon[34068]: osd.5 [v2:172.21.15.191:6808/4216938371,v1:172.21.15.191:6809/4216938371] boot 2024-09-10T08:40:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:50 smithi174 ceph-mon[34068]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T08:40:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:50 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:50.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:50 smithi191 ceph-mon[35314]: purged_snaps scrub starts 2024-09-10T08:40:50.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:50 smithi191 ceph-mon[35314]: purged_snaps scrub ok 2024-09-10T08:40:50.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:50 smithi191 ceph-mon[35314]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:40:50.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:50 smithi191 ceph-mon[35314]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-10T08:40:50.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:50 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:50.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:50 smithi191 ceph-mon[35314]: osd.5 [v2:172.21.15.191:6808/4216938371,v1:172.21.15.191:6809/4216938371] boot 2024-09-10T08:40:50.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:50 smithi191 ceph-mon[35314]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T08:40:50.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:50 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T08:40:51.474 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:51 smithi172 ceph-mon[30854]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T08:40:51.474 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:51 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2786393378' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60a9938a-f696-4db2-932c-402997543b6d"}]: dispatch 2024-09-10T08:40:51.474 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:51 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2786393378' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60a9938a-f696-4db2-932c-402997543b6d"}]': finished 2024-09-10T08:40:51.475 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:51 smithi172 ceph-mon[30854]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T08:40:51.475 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:51 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:40:51.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:51 smithi174 ceph-mon[34068]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T08:40:51.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:51 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2786393378' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60a9938a-f696-4db2-932c-402997543b6d"}]: dispatch 2024-09-10T08:40:51.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:51 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2786393378' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60a9938a-f696-4db2-932c-402997543b6d"}]': finished 2024-09-10T08:40:51.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:51 smithi174 ceph-mon[34068]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T08:40:51.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:51 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:40:51.552 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:51 smithi191 ceph-mon[35314]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T08:40:51.552 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:51 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2786393378' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60a9938a-f696-4db2-932c-402997543b6d"}]: dispatch 2024-09-10T08:40:51.552 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:51 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2786393378' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60a9938a-f696-4db2-932c-402997543b6d"}]': finished 2024-09-10T08:40:51.552 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:51 smithi191 ceph-mon[35314]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T08:40:51.552 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:51 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:40:52.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:52 smithi174 ceph-mon[34068]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:40:52.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:52 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2538713039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:40:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:52 smithi191 ceph-mon[35314]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:40:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:52 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2538713039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:40:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:52 smithi172 ceph-mon[30854]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:40:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:52 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2538713039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:40:53.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:53 smithi174 ceph-mon[34068]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T08:40:53.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:53 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:40:53.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:53 smithi191 ceph-mon[35314]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T08:40:53.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:53 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:40:53.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:53 smithi172 ceph-mon[30854]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T08:40:53.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:53 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:40:54.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:54 smithi174 ceph-mon[34068]: pgmap v139: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T08:40:54.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:54 smithi174 ceph-mon[34068]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T08:40:54.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:54 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:40:54.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:54 smithi191 ceph-mon[35314]: pgmap v139: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T08:40:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:54 smithi191 ceph-mon[35314]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T08:40:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:54 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:40:54.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:54 smithi172 ceph-mon[30854]: pgmap v139: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T08:40:54.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:54 smithi172 ceph-mon[30854]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T08:40:54.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:54 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:40:56.450 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:56 smithi172 ceph-mon[30854]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-10T08:40:56.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:56 smithi174 ceph-mon[34068]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-10T08:40:56.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:56 smithi191 ceph-mon[35314]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-10T08:40:58.489 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:58 smithi172 ceph-mon[30854]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:40:58.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:58 smithi174 ceph-mon[34068]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:40:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:58 smithi191 ceph-mon[35314]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:40:59.331 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:59 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T08:40:59.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:40:59 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:59.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:59 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T08:40:59.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:40:59 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:40:59.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:59 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T08:40:59.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:40:59 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:00.484 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:00 smithi172 ceph-mon[30854]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-10T08:41:00.484 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:00 smithi172 ceph-mon[30854]: Deploying daemon osd.6 on smithi172 2024-09-10T08:41:00.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:00 smithi174 ceph-mon[34068]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-10T08:41:00.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:00 smithi174 ceph-mon[34068]: Deploying daemon osd.6 on smithi172 2024-09-10T08:41:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:00 smithi191 ceph-mon[35314]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-10T08:41:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:00 smithi191 ceph-mon[35314]: Deploying daemon osd.6 on smithi172 2024-09-10T08:41:02.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:02 smithi174 ceph-mon[34068]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:02 smithi191 ceph-mon[35314]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:02.572 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:02 smithi172 ceph-mon[30854]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:04.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:04 smithi174 ceph-mon[34068]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:04.509 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:04 smithi172 ceph-mon[30854]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:04.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:04 smithi191 ceph-mon[35314]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:05.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:05.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:05.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:05.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:05 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:05.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:05 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:05.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:05 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:05.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:05 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:05.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:05 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:05.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:05 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:06.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:06 smithi174 ceph-mon[34068]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:06.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:06 smithi191 ceph-mon[35314]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:06 smithi172 ceph-mon[30854]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:08.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:08 smithi174 ceph-mon[34068]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:08.516 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:08 smithi172 ceph-mon[30854]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:08.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:08 smithi191 ceph-mon[35314]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:08.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:41:08.557+0000 7f9784810640 1 -- 172.21.15.172:0/3389667689 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9780062630 con 0x7f97680a2bb0 2024-09-10T08:41:08.558 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 6 on host 'smithi172' 2024-09-10T08:41:08.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:41:08.559+0000 7f9785812640 1 -- 172.21.15.172:0/3389667689 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97680a2bb0 msgr2=0x7f97680a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:08.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:41:08.559+0000 7f9785812640 1 --2- 172.21.15.172:0/3389667689 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97680a2bb0 0x7f97680a5070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9780199400 tx=0x7f9774005f90 comp rx=0 tx=0).stop 2024-09-10T08:41:08.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:41:08.559+0000 7f9785812640 1 -- 172.21.15.172:0/3389667689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9780100b10 msgr2=0x7f9780197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:08.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:41:08.559+0000 7f9785812640 1 --2- 172.21.15.172:0/3389667689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9780100b10 0x7f9780197ee0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f97700314e0 tx=0x7f9770031a30 comp rx=0 tx=0).stop 2024-09-10T08:41:08.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:41:08.559+0000 7f9785812640 1 -- 172.21.15.172:0/3389667689 shutdown_connections 2024-09-10T08:41:08.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:41:08.559+0000 7f9785812640 1 --2- 172.21.15.172:0/3389667689 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97680a2bb0 0x7f97680a5070 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:41:08.559+0000 7f9785812640 1 --2- 172.21.15.172:0/3389667689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97801026d0 0x7f978019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:41:08.559+0000 7f9785812640 1 --2- 172.21.15.172:0/3389667689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9780101d10 0x7f9780198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:41:08.559+0000 7f9785812640 1 --2- 172.21.15.172:0/3389667689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9780100b10 0x7f9780197ee0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:08.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:41:08.559+0000 7f9785812640 1 -- 172.21.15.172:0/3389667689 >> 172.21.15.172:0/3389667689 conn(0x7f97800fc2a0 msgr2=0x7f97801051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:08.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:41:08.559+0000 7f9785812640 1 -- 172.21.15.172:0/3389667689 shutdown_connections 2024-09-10T08:41:08.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:41:08.559+0000 7f9785812640 1 -- 172.21.15.172:0/3389667689 wait complete. 2024-09-10T08:41:09.199 DEBUG:teuthology.orchestra.run.smithi172:osd.6> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T08:41:09.202 INFO:tasks.cephadm:Deploying osd.7 on smithi174 with /dev/vg_nvme/lv_2... 2024-09-10T08:41:09.202 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T08:41:09.457 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:41:09.828 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:09 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:09.828 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:09 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:09.828 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:09 smithi172 ceph-mon[30854]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:09.829 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:09 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:09.829 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:09 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:09.858 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:09 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:09.859 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:09 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:09.859 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:09 smithi174 ceph-mon[34068]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:09.859 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:09 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:09.859 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:09 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:10.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:09 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:10.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:09 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:10.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:09 smithi191 ceph-mon[35314]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:10.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:09 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:10.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:09 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:10.681 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 10 08:41:10 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-6[55363]: 2024-09-10T08:41:10.621+0000 7f5a869ae640 -1 osd.6 0 log_to_monitors true 2024-09-10T08:41:10.896 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:10 smithi174 ceph-mon[34068]: from='osd.6 [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T08:41:10.997 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:10 smithi172 ceph-mon[30854]: from='osd.6 [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T08:41:11.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:10 smithi191 ceph-mon[35314]: from='osd.6 [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T08:41:11.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:11 smithi172 ceph-mon[30854]: from='osd.6 [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T08:41:11.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:11 smithi172 ceph-mon[30854]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T08:41:11.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:11 smithi172 ceph-mon[30854]: from='osd.6 [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-10T08:41:11.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:41:11.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:11 smithi172 ceph-mon[30854]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:11.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:11 smithi172 ceph-mon[30854]: from='osd.6 [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-10T08:41:11.833 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 10 08:41:11 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-6[55363]: 2024-09-10T08:41:11.685+0000 7f5a82931640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T08:41:11.833 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 10 08:41:11 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-6[55363]: 2024-09-10T08:41:11.697+0000 7f5a7e759640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:41:11.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:11 smithi174 ceph-mon[34068]: from='osd.6 [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T08:41:11.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:11 smithi174 ceph-mon[34068]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T08:41:11.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:11 smithi174 ceph-mon[34068]: from='osd.6 [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-10T08:41:11.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:41:11.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:11 smithi174 ceph-mon[34068]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:11.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:11 smithi174 ceph-mon[34068]: from='osd.6 [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-10T08:41:12.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:11 smithi191 ceph-mon[35314]: from='osd.6 [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T08:41:12.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:11 smithi191 ceph-mon[35314]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T08:41:12.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:11 smithi191 ceph-mon[35314]: from='osd.6 [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-10T08:41:12.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:41:12.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:11 smithi191 ceph-mon[35314]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:12.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:11 smithi191 ceph-mon[35314]: from='osd.6 [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-10T08:41:12.232 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:41:12.258 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_2 2024-09-10T08:41:12.507 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:41:12.908 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:12 smithi174 ceph-mon[34068]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T08:41:12.908 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:12 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:41:12.908 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:12 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:41:12.908 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:12 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:12.908 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:12 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:12.908 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:12 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:12.908 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:12 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:12.909 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:12 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:12.909 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:12 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:12.909 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:12 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:12.909 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:12 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:12.909 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:12 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:12.909 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:12 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:41:13.016 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.990+0000 7f4854bfc640 1 -- 172.21.15.174:0/3166587538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48501037c0 msgr2=0x7f485010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:13.016 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.990+0000 7f4854bfc640 1 --2- 172.21.15.174:0/3166587538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48501037c0 0x7f485010a050 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4848009ca0 tx=0x7f484802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:41:13.017 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.990+0000 7f4854bfc640 1 -- 172.21.15.174:0/3166587538 shutdown_connections 2024-09-10T08:41:13.017 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.990+0000 7f4854bfc640 1 --2- 172.21.15.174:0/3166587538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48501037c0 0x7f485010a050 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:13.017 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.990+0000 7f4854bfc640 1 --2- 172.21.15.174:0/3166587538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4850102e00 0x7f4850103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:13.017 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.990+0000 7f4854bfc640 1 --2- 172.21.15.174:0/3166587538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4850101c00 0x7f4850102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:13.017 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.990+0000 7f4854bfc640 1 -- 172.21.15.174:0/3166587538 >> 172.21.15.174:0/3166587538 conn(0x7f48500fd390 msgr2=0x7f48500ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:13.018 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.990+0000 7f4854bfc640 1 -- 172.21.15.174:0/3166587538 shutdown_connections 2024-09-10T08:41:13.018 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.990+0000 7f4854bfc640 1 -- 172.21.15.174:0/3166587538 wait complete. 2024-09-10T08:41:13.018 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.991+0000 7f4854bfc640 1 Processor -- start 2024-09-10T08:41:13.018 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.991+0000 7f4854bfc640 1 -- start start 2024-09-10T08:41:13.018 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.991+0000 7f4854bfc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4850101c00 0x7f4850197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:13.018 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.991+0000 7f4854bfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4850102e00 0x7f4850198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:13.019 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.991+0000 7f4854bfc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48501037c0 0x7f485019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:13.019 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.991+0000 7f4854bfc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f485010d730 con 0x7f4850102e00 2024-09-10T08:41:13.019 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.991+0000 7f4854bfc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f485010d5b0 con 0x7f4850101c00 2024-09-10T08:41:13.019 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.991+0000 7f4854bfc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f485010d8b0 con 0x7f48501037c0 2024-09-10T08:41:13.019 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.991+0000 7f484e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4850101c00 0x7f4850197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:13.019 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.992+0000 7f484e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4850101c00 0x7f4850197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:36516/0 (socket says 172.21.15.174:36516) 2024-09-10T08:41:13.019 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.992+0000 7f484e59c640 1 -- 172.21.15.174:0/2857855161 learned_addr learned my addr 172.21.15.174:0/2857855161 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-10T08:41:13.019 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.992+0000 7f484dd9b640 1 --2- 172.21.15.174:0/2857855161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4850102e00 0x7f4850198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:13.020 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.992+0000 7f484ed9d640 1 --2- 172.21.15.174:0/2857855161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48501037c0 0x7f485019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:13.020 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.992+0000 7f484e59c640 1 -- 172.21.15.174:0/2857855161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48501037c0 msgr2=0x7f485019f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:13.020 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.992+0000 7f484e59c640 1 --2- 172.21.15.174:0/2857855161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48501037c0 0x7f485019f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:13.020 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.992+0000 7f484e59c640 1 -- 172.21.15.174:0/2857855161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4850102e00 msgr2=0x7f4850198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:13.020 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.992+0000 7f484e59c640 1 --2- 172.21.15.174:0/2857855161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4850102e00 0x7f4850198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:13.020 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.992+0000 7f484e59c640 1 -- 172.21.15.174:0/2857855161 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485019fa20 con 0x7f4850101c00 2024-09-10T08:41:13.020 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.992+0000 7f484e59c640 1 --2- 172.21.15.174:0/2857855161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4850101c00 0x7f4850197df0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f484400d950 tx=0x7f484400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:13.020 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.992+0000 7f483f7fe640 1 -- 172.21.15.174:0/2857855161 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844017070 con 0x7f4850101c00 2024-09-10T08:41:13.020 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.992+0000 7f4854bfc640 1 -- 172.21.15.174:0/2857855161 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485019fd10 con 0x7f4850101c00 2024-09-10T08:41:13.021 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.992+0000 7f483f7fe640 1 -- 172.21.15.174:0/2857855161 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4844009c30 con 0x7f4850101c00 2024-09-10T08:41:13.021 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.992+0000 7f483f7fe640 1 -- 172.21.15.174:0/2857855161 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484400f620 con 0x7f4850101c00 2024-09-10T08:41:13.021 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.992+0000 7f4854bfc640 1 -- 172.21.15.174:0/2857855161 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48501a01a0 con 0x7f4850101c00 2024-09-10T08:41:13.021 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.993+0000 7f4854bfc640 1 -- 172.21.15.174:0/2857855161 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4850109020 con 0x7f4850101c00 2024-09-10T08:41:13.021 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.994+0000 7f483f7fe640 1 -- 172.21.15.174:0/2857855161 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f484401a070 con 0x7f4850101c00 2024-09-10T08:41:13.021 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.995+0000 7f483f7fe640 1 --2- 172.21.15.174:0/2857855161 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f48380a2bb0 0x7f48380a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:13.021 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.995+0000 7f483f7fe640 1 -- 172.21.15.174:0/2857855161 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4965+0+0 (secure 0 0 0) 0x7f48440cdcd0 con 0x7f4850101c00 2024-09-10T08:41:13.021 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.995+0000 7f484dd9b640 1 --2- 172.21.15.174:0/2857855161 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f48380a2bb0 0x7f48380a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:13.021 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.995+0000 7f484dd9b640 1 --2- 172.21.15.174:0/2857855161 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f48380a2bb0 0x7f48380a5070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4850199310 tx=0x7f484000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:13.022 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:12.999+0000 7f483f7fe640 1 -- 172.21.15.174:0/2857855161 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f484409b2f0 con 0x7f4850101c00 2024-09-10T08:41:13.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:12 smithi191 ceph-mon[35314]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T08:41:13.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:12 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:41:13.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:12 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:41:13.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:12 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:13.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:12 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:13.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:12 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:13.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:12 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:13.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:12 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:13.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:12 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:13.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:12 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:13.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:12 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:13.052 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:12 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:13.052 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:12 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:41:13.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:12 smithi172 ceph-mon[30854]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T08:41:13.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:12 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:41:13.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:12 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:41:13.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:12 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:13.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:12 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:13.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:12 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:13.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:12 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:13.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:12 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:13.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:12 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:13.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:12 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:13.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:12 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:13.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:12 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:13.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:12 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:41:13.173 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:13.172+0000 7f4854bfc640 1 -- 172.21.15.174:0/2857855161 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4850062630 con 0x7f48380a2bb0 2024-09-10T08:41:13.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:13 smithi174 ceph-mon[34068]: purged_snaps scrub starts 2024-09-10T08:41:13.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:13 smithi174 ceph-mon[34068]: purged_snaps scrub ok 2024-09-10T08:41:13.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:13 smithi174 ceph-mon[34068]: Detected new or changed devices on smithi172 2024-09-10T08:41:13.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:13 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi172 to 5696M 2024-09-10T08:41:13.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:13 smithi174 ceph-mon[34068]: osd.6 [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009] boot 2024-09-10T08:41:13.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:13 smithi174 ceph-mon[34068]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T08:41:13.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:41:13.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:13.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:13.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:13.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:13.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:13 smithi174 ceph-mon[34068]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:13.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:13 smithi174 ceph-mon[34068]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:41:13.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:41:13.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:41:13.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:13 smithi191 ceph-mon[35314]: purged_snaps scrub starts 2024-09-10T08:41:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:13 smithi191 ceph-mon[35314]: purged_snaps scrub ok 2024-09-10T08:41:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:13 smithi191 ceph-mon[35314]: Detected new or changed devices on smithi172 2024-09-10T08:41:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:13 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi172 to 5696M 2024-09-10T08:41:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:13 smithi191 ceph-mon[35314]: osd.6 [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009] boot 2024-09-10T08:41:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:13 smithi191 ceph-mon[35314]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T08:41:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:41:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:14.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:14.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:14.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:14.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:13 smithi191 ceph-mon[35314]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:14.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:13 smithi191 ceph-mon[35314]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:41:14.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:41:14.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:41:14.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:14.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:13 smithi172 ceph-mon[30854]: purged_snaps scrub starts 2024-09-10T08:41:14.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:13 smithi172 ceph-mon[30854]: purged_snaps scrub ok 2024-09-10T08:41:14.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:13 smithi172 ceph-mon[30854]: Detected new or changed devices on smithi172 2024-09-10T08:41:14.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:13 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi172 to 5696M 2024-09-10T08:41:14.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:13 smithi172 ceph-mon[30854]: osd.6 [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009] boot 2024-09-10T08:41:14.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:13 smithi172 ceph-mon[30854]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T08:41:14.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T08:41:14.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:14.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:14.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:14.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:14.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:13 smithi172 ceph-mon[30854]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-09-10T08:41:14.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:13 smithi172 ceph-mon[30854]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:41:14.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:41:14.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:41:14.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:14.867 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:14 smithi174 ceph-mon[34068]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T08:41:15.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:14 smithi191 ceph-mon[35314]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T08:41:15.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:14 smithi172 ceph-mon[30854]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T08:41:16.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:15 smithi191 ceph-mon[35314]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:16.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:15 smithi191 ceph-mon[35314]: from='client.? 172.21.15.174:0/791484211' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f0202f3-971e-472f-9c4d-ae3c24d39f2c"}]: dispatch 2024-09-10T08:41:16.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:15 smithi191 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f0202f3-971e-472f-9c4d-ae3c24d39f2c"}]: dispatch 2024-09-10T08:41:16.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:15 smithi191 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f0202f3-971e-472f-9c4d-ae3c24d39f2c"}]': finished 2024-09-10T08:41:16.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:15 smithi191 ceph-mon[35314]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T08:41:16.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:15 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:41:16.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:15 smithi172 ceph-mon[30854]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:16.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:15 smithi172 ceph-mon[30854]: from='client.? 172.21.15.174:0/791484211' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f0202f3-971e-472f-9c4d-ae3c24d39f2c"}]: dispatch 2024-09-10T08:41:16.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:15 smithi172 ceph-mon[30854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f0202f3-971e-472f-9c4d-ae3c24d39f2c"}]: dispatch 2024-09-10T08:41:16.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:15 smithi172 ceph-mon[30854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f0202f3-971e-472f-9c4d-ae3c24d39f2c"}]': finished 2024-09-10T08:41:16.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:15 smithi172 ceph-mon[30854]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T08:41:16.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:15 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:41:16.156 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:15 smithi174 ceph-mon[34068]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:16.156 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:15 smithi174 ceph-mon[34068]: from='client.? 172.21.15.174:0/791484211' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f0202f3-971e-472f-9c4d-ae3c24d39f2c"}]: dispatch 2024-09-10T08:41:16.156 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:15 smithi174 ceph-mon[34068]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f0202f3-971e-472f-9c4d-ae3c24d39f2c"}]: dispatch 2024-09-10T08:41:16.156 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:15 smithi174 ceph-mon[34068]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f0202f3-971e-472f-9c4d-ae3c24d39f2c"}]': finished 2024-09-10T08:41:16.156 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:15 smithi174 ceph-mon[34068]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T08:41:16.156 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:15 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:41:17.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:16 smithi191 ceph-mon[35314]: from='client.? 172.21.15.174:0/3055612249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:41:17.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:16 smithi172 ceph-mon[30854]: from='client.? 172.21.15.174:0/3055612249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:41:17.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:16 smithi174 ceph-mon[34068]: from='client.? 172.21.15.174:0/3055612249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:41:18.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:17 smithi172 ceph-mon[30854]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:18.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:17 smithi174 ceph-mon[34068]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:18.299 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:17 smithi191 ceph-mon[35314]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:20.357 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:20 smithi174 ceph-mon[34068]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:20.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:20 smithi191 ceph-mon[35314]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:20.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:20 smithi172 ceph-mon[30854]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:22.437 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:22 smithi172 ceph-mon[30854]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:22.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:22 smithi174 ceph-mon[34068]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:22.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:22 smithi191 ceph-mon[35314]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:24.228 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:24 smithi174 ceph-mon[34068]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:24.228 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:24 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T08:41:24.228 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:24 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:24 smithi191 ceph-mon[35314]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:24 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T08:41:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:24 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:24.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:24 smithi172 ceph-mon[30854]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:24.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:24 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T08:41:24.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:24 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:25.381 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:25 smithi174 ceph-mon[34068]: Deploying daemon osd.7 on smithi174 2024-09-10T08:41:25.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:25 smithi191 ceph-mon[35314]: Deploying daemon osd.7 on smithi174 2024-09-10T08:41:25.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:25 smithi172 ceph-mon[30854]: Deploying daemon osd.7 on smithi174 2024-09-10T08:41:26.286 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:26 smithi174 ceph-mon[34068]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:26.451 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:26 smithi191 ceph-mon[35314]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:26.669 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:26 smithi172 ceph-mon[30854]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:28.408 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:28 smithi174 ceph-mon[34068]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:28.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:28 smithi191 ceph-mon[35314]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:28 smithi172 ceph-mon[30854]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:30.441 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:30 smithi174 ceph-mon[34068]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:30.441 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:30.442 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:30.442 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:30.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:30 smithi191 ceph-mon[35314]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:30.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:30.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:30.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:30.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:30 smithi172 ceph-mon[30854]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:30.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:30.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:30.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:32.489 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:32 smithi174 ceph-mon[34068]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:32 smithi191 ceph-mon[35314]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:32.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:32 smithi172 ceph-mon[30854]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:33.642 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 7 on host 'smithi174' 2024-09-10T08:41:33.642 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:33.641+0000 7f483f7fe640 1 -- 172.21.15.174:0/2857855161 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4850062630 con 0x7f48380a2bb0 2024-09-10T08:41:33.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:33.643+0000 7f4854bfc640 1 -- 172.21.15.174:0/2857855161 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f48380a2bb0 msgr2=0x7f48380a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:33.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:33.643+0000 7f4854bfc640 1 --2- 172.21.15.174:0/2857855161 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f48380a2bb0 0x7f48380a5070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4850199310 tx=0x7f484000b390 comp rx=0 tx=0).stop 2024-09-10T08:41:33.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:33.643+0000 7f4854bfc640 1 -- 172.21.15.174:0/2857855161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4850101c00 msgr2=0x7f4850197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:33.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:33.643+0000 7f4854bfc640 1 --2- 172.21.15.174:0/2857855161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4850101c00 0x7f4850197df0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f484400d950 tx=0x7f484400de10 comp rx=0 tx=0).stop 2024-09-10T08:41:33.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:33.643+0000 7f4854bfc640 1 -- 172.21.15.174:0/2857855161 shutdown_connections 2024-09-10T08:41:33.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:33.643+0000 7f4854bfc640 1 --2- 172.21.15.174:0/2857855161 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f48380a2bb0 0x7f48380a5070 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:33.644 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:33.643+0000 7f4854bfc640 1 --2- 172.21.15.174:0/2857855161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48501037c0 0x7f485019f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:33.645 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:33.643+0000 7f4854bfc640 1 --2- 172.21.15.174:0/2857855161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4850102e00 0x7f4850198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:33.645 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:33.643+0000 7f4854bfc640 1 --2- 172.21.15.174:0/2857855161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4850101c00 0x7f4850197df0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:33.645 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:33.643+0000 7f4854bfc640 1 -- 172.21.15.174:0/2857855161 >> 172.21.15.174:0/2857855161 conn(0x7f48500fd390 msgr2=0x7f48501061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:33.645 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:33.643+0000 7f4854bfc640 1 -- 172.21.15.174:0/2857855161 shutdown_connections 2024-09-10T08:41:33.645 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:41:33.643+0000 7f4854bfc640 1 -- 172.21.15.174:0/2857855161 wait complete. 2024-09-10T08:41:33.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:33 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:33.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:33 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:33.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:33 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:33.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:33 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:33.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:33 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:33.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:33 smithi174 ceph-mon[34068]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:34.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:33 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:34.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:33 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:34.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:33 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:34.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:33 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:34.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:33 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:34.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:33 smithi191 ceph-mon[35314]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:34.084 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:33 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:34.085 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:33 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:34.085 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:33 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:34.085 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:33 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:34.085 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:33 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:34.085 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:33 smithi172 ceph-mon[30854]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:34.341 DEBUG:teuthology.orchestra.run.smithi174:osd.7> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T08:41:34.343 INFO:tasks.cephadm:Deploying osd.8 on smithi191 with /dev/vg_nvme/lv_2... 2024-09-10T08:41:34.343 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T08:41:34.596 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:41:34.717 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:34 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:34.718 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:34 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:34.718 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:34 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:34.858 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:34 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:34.858 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:34 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:34.859 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:34 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:35.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:34 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:35.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:34 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:35.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:34 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:35.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:35 smithi191 ceph-mon[35314]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:35.845 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:35 smithi174 ceph-mon[34068]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:35.846 INFO:journalctl@ceph.osd.7.smithi174.stdout:Sep 10 08:41:35 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-7[51746]: 2024-09-10T08:41:35.623+0000 7fc0f7887640 -1 osd.7 0 log_to_monitors true 2024-09-10T08:41:36.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:35 smithi172 ceph-mon[30854]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:36.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:36 smithi191 ceph-mon[35314]: from='osd.7 [v2:172.21.15.174:6816/1461923581,v1:172.21.15.174:6817/1461923581]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:41:36.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:36 smithi191 ceph-mon[35314]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:41:36.986 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:36 smithi174 ceph-mon[34068]: from='osd.7 [v2:172.21.15.174:6816/1461923581,v1:172.21.15.174:6817/1461923581]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:41:36.986 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:36 smithi174 ceph-mon[34068]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:41:37.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:36 smithi172 ceph-mon[30854]: from='osd.7 [v2:172.21.15.174:6816/1461923581,v1:172.21.15.174:6817/1461923581]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:41:37.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:36 smithi172 ceph-mon[30854]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T08:41:37.371 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:41:37.398 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi191:vg_nvme/lv_2 2024-09-10T08:41:37.647 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:41:37.901 INFO:journalctl@ceph.osd.7.smithi174.stdout:Sep 10 08:41:37 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-7[51746]: 2024-09-10T08:41:37.630+0000 7fc0f380a640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T08:41:37.901 INFO:journalctl@ceph.osd.7.smithi174.stdout:Sep 10 08:41:37 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-7[51746]: 2024-09-10T08:41:37.638+0000 7fc0ef632640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:41:37.901 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:37 smithi174 ceph-mon[34068]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T08:41:37.901 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:37 smithi174 ceph-mon[34068]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T08:41:37.901 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:37 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:41:37.902 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:37 smithi174 ceph-mon[34068]: from='osd.7 [v2:172.21.15.174:6816/1461923581,v1:172.21.15.174:6817/1461923581]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:41:37.902 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:37 smithi174 ceph-mon[34068]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:41:37.902 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:37 smithi174 ceph-mon[34068]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:37.902 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:37 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:37.902 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:37 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:37.902 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:37 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:37.902 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:37 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:37.902 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:37 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:37.902 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:37 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:37.902 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:37 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:37.902 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:37 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:37.902 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:37 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:37.924 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:37 smithi191 ceph-mon[35314]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T08:41:37.925 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:37 smithi191 ceph-mon[35314]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T08:41:37.925 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:37 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:41:37.925 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:37 smithi191 ceph-mon[35314]: from='osd.7 [v2:172.21.15.174:6816/1461923581,v1:172.21.15.174:6817/1461923581]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:41:37.925 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:37 smithi191 ceph-mon[35314]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:41:37.925 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:37 smithi191 ceph-mon[35314]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:37.925 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:37 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:37.925 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:37 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:37.925 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:37 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:37.925 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:37 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:37.925 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:37 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:37.925 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:37 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:37.926 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:37 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:37.926 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:37 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:37.926 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:37 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:38.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:37 smithi172 ceph-mon[30854]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T08:41:38.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:37 smithi172 ceph-mon[30854]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T08:41:38.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:41:38.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:37 smithi172 ceph-mon[30854]: from='osd.7 [v2:172.21.15.174:6816/1461923581,v1:172.21.15.174:6817/1461923581]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:41:38.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:37 smithi172 ceph-mon[30854]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:41:38.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:37 smithi172 ceph-mon[30854]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-10T08:41:38.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:38.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:38.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:38.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:38.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:41:38.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:38.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:38.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:38.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:38.147 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.129+0000 7f4678ef0640 1 -- 172.21.15.191:0/3695509149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4674102e00 msgr2=0x7f4674103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:38.147 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.129+0000 7f4678ef0640 1 --2- 172.21.15.191:0/3695509149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4674102e00 0x7f4674103280 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4668009900 tx=0x7f466802a410 comp rx=0 tx=0).stop 2024-09-10T08:41:38.148 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.129+0000 7f4678ef0640 1 -- 172.21.15.191:0/3695509149 shutdown_connections 2024-09-10T08:41:38.148 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.129+0000 7f4678ef0640 1 --2- 172.21.15.191:0/3695509149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46741037c0 0x7f467410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:38.148 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.129+0000 7f4678ef0640 1 --2- 172.21.15.191:0/3695509149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4674102e00 0x7f4674103280 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:38.148 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.129+0000 7f4678ef0640 1 --2- 172.21.15.191:0/3695509149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4674101c00 0x7f4674102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:38.148 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.129+0000 7f4678ef0640 1 -- 172.21.15.191:0/3695509149 >> 172.21.15.191:0/3695509149 conn(0x7f46740fd390 msgr2=0x7f46740ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:38.148 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.130+0000 7f4678ef0640 1 -- 172.21.15.191:0/3695509149 shutdown_connections 2024-09-10T08:41:38.148 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.130+0000 7f4678ef0640 1 -- 172.21.15.191:0/3695509149 wait complete. 2024-09-10T08:41:38.148 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.130+0000 7f4678ef0640 1 Processor -- start 2024-09-10T08:41:38.148 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.130+0000 7f4678ef0640 1 -- start start 2024-09-10T08:41:38.148 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4678ef0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4674101c00 0x7f4674197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:38.148 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4678ef0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4674102e00 0x7f46741982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:38.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4678ef0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46741037c0 0x7f467419f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:38.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4678ef0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f467410d710 con 0x7f4674101c00 2024-09-10T08:41:38.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4678ef0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f467410d590 con 0x7f4674102e00 2024-09-10T08:41:38.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4678ef0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f467410d890 con 0x7f46741037c0 2024-09-10T08:41:38.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4672d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46741037c0 0x7f467419f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:38.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4672d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46741037c0 0x7f467419f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:40944/0 (socket says 172.21.15.191:40944) 2024-09-10T08:41:38.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4672d9d640 1 -- 172.21.15.191:0/3619336984 learned_addr learned my addr 172.21.15.191:0/3619336984 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-10T08:41:38.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4671d9b640 1 --2- 172.21.15.191:0/3619336984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4674102e00 0x7f46741982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:38.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f467259c640 1 --2- 172.21.15.191:0/3619336984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4674101c00 0x7f4674197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:38.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4672d9d640 1 -- 172.21.15.191:0/3619336984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4674102e00 msgr2=0x7f46741982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:38.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4672d9d640 1 --2- 172.21.15.191:0/3619336984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4674102e00 0x7f46741982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:38.149 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4672d9d640 1 -- 172.21.15.191:0/3619336984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4674101c00 msgr2=0x7f4674197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:38.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4672d9d640 1 --2- 172.21.15.191:0/3619336984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4674101c00 0x7f4674197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:38.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4672d9d640 1 -- 172.21.15.191:0/3619336984 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467419f9e0 con 0x7f46741037c0 2024-09-10T08:41:38.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4672d9d640 1 --2- 172.21.15.191:0/3619336984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46741037c0 0x7f467419f370 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f466c00cce0 tx=0x7f466c00a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:38.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f46637fe640 1 -- 172.21.15.191:0/3619336984 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c00f450 con 0x7f46741037c0 2024-09-10T08:41:38.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f46637fe640 1 -- 172.21.15.191:0/3619336984 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f466c00faa0 con 0x7f46741037c0 2024-09-10T08:41:38.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4678ef0640 1 -- 172.21.15.191:0/3619336984 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467419fcd0 con 0x7f46741037c0 2024-09-10T08:41:38.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f46637fe640 1 -- 172.21.15.191:0/3619336984 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c018b70 con 0x7f46741037c0 2024-09-10T08:41:38.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.131+0000 7f4678ef0640 1 -- 172.21.15.191:0/3619336984 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46741a0190 con 0x7f46741037c0 2024-09-10T08:41:38.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.133+0000 7f4678ef0640 1 -- 172.21.15.191:0/3619336984 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4674109000 con 0x7f46741037c0 2024-09-10T08:41:38.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.134+0000 7f46637fe640 1 -- 172.21.15.191:0/3619336984 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f466c018d10 con 0x7f46741037c0 2024-09-10T08:41:38.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.134+0000 7f46637fe640 1 --2- 172.21.15.191:0/3619336984 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f465c0a2c30 0x7f465c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:41:38.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.135+0000 7f46637fe640 1 -- 172.21.15.191:0/3619336984 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5117+0+0 (secure 0 0 0) 0x7f466c0ce6f0 con 0x7f46741037c0 2024-09-10T08:41:38.150 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.135+0000 7f467259c640 1 --2- 172.21.15.191:0/3619336984 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f465c0a2c30 0x7f465c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:41:38.151 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.135+0000 7f467259c640 1 --2- 172.21.15.191:0/3619336984 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f465c0a2c30 0x7f465c0a50f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4664005d00 tx=0x7f4664005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:41:38.151 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.139+0000 7f46637fe640 1 -- 172.21.15.191:0/3619336984 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f466c014030 con 0x7f46741037c0 2024-09-10T08:41:38.317 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:38.317+0000 7f4678ef0640 1 -- 172.21.15.191:0/3619336984 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4674062630 con 0x7f465c0a2c30 2024-09-10T08:41:38.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:38 smithi174 ceph-mon[34068]: purged_snaps scrub starts 2024-09-10T08:41:38.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:38 smithi174 ceph-mon[34068]: purged_snaps scrub ok 2024-09-10T08:41:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:38 smithi174 ceph-mon[34068]: Detected new or changed devices on smithi174 2024-09-10T08:41:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:38 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-10T08:41:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:38 smithi174 ceph-mon[34068]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T08:41:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:38 smithi174 ceph-mon[34068]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T08:41:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:38 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:41:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:38 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:41:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:38 smithi174 ceph-mon[34068]: osd.7 [v2:172.21.15.174:6816/1461923581,v1:172.21.15.174:6817/1461923581] boot 2024-09-10T08:41:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:38 smithi174 ceph-mon[34068]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T08:41:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:38 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:41:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:38 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:41:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:38 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:41:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:38 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:38.983 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:38 smithi191 ceph-mon[35314]: purged_snaps scrub starts 2024-09-10T08:41:38.983 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:38 smithi191 ceph-mon[35314]: purged_snaps scrub ok 2024-09-10T08:41:38.984 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:38 smithi191 ceph-mon[35314]: Detected new or changed devices on smithi174 2024-09-10T08:41:38.984 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:38 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-10T08:41:38.984 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:38 smithi191 ceph-mon[35314]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T08:41:38.984 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:38 smithi191 ceph-mon[35314]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T08:41:38.984 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:38 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:41:38.984 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:38 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:41:38.984 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:38 smithi191 ceph-mon[35314]: osd.7 [v2:172.21.15.174:6816/1461923581,v1:172.21.15.174:6817/1461923581] boot 2024-09-10T08:41:38.984 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:38 smithi191 ceph-mon[35314]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T08:41:38.984 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:38 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:41:38.984 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:38 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:41:38.985 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:38 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:41:38.985 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:38 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:39.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:38 smithi172 ceph-mon[30854]: purged_snaps scrub starts 2024-09-10T08:41:39.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:38 smithi172 ceph-mon[30854]: purged_snaps scrub ok 2024-09-10T08:41:39.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:38 smithi172 ceph-mon[30854]: Detected new or changed devices on smithi174 2024-09-10T08:41:39.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:38 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-10T08:41:39.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:38 smithi172 ceph-mon[30854]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T08:41:39.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:38 smithi172 ceph-mon[30854]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T08:41:39.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:38 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:41:39.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:38 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:41:39.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:38 smithi172 ceph-mon[30854]: osd.7 [v2:172.21.15.174:6816/1461923581,v1:172.21.15.174:6817/1461923581] boot 2024-09-10T08:41:39.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:38 smithi172 ceph-mon[30854]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T08:41:39.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:38 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T08:41:39.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:38 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:41:39.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:38 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:41:39.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:38 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:40.440 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:40 smithi191 ceph-mon[35314]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:41:40.440 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:40 smithi191 ceph-mon[35314]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:40.440 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:40 smithi191 ceph-mon[35314]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T08:41:40.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:40 smithi174 ceph-mon[34068]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:41:40.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:40 smithi174 ceph-mon[34068]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:40.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:40 smithi174 ceph-mon[34068]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T08:41:40.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:40 smithi172 ceph-mon[30854]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:41:40.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:40 smithi172 ceph-mon[30854]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:40.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:40 smithi172 ceph-mon[30854]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T08:41:41.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:41 smithi191 ceph-mon[35314]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T08:41:41.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:41 smithi191 ceph-mon[35314]: from='client.? 172.21.15.191:0/3657884645' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4e7327f-ca9f-4833-9cd3-b8529b3d9761"}]: dispatch 2024-09-10T08:41:41.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:41 smithi191 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4e7327f-ca9f-4833-9cd3-b8529b3d9761"}]: dispatch 2024-09-10T08:41:41.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:41 smithi191 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b4e7327f-ca9f-4833-9cd3-b8529b3d9761"}]': finished 2024-09-10T08:41:41.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:41 smithi191 ceph-mon[35314]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T08:41:41.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:41 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:41:41.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:41 smithi174 ceph-mon[34068]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T08:41:41.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:41 smithi174 ceph-mon[34068]: from='client.? 172.21.15.191:0/3657884645' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4e7327f-ca9f-4833-9cd3-b8529b3d9761"}]: dispatch 2024-09-10T08:41:41.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:41 smithi174 ceph-mon[34068]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4e7327f-ca9f-4833-9cd3-b8529b3d9761"}]: dispatch 2024-09-10T08:41:41.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:41 smithi174 ceph-mon[34068]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b4e7327f-ca9f-4833-9cd3-b8529b3d9761"}]': finished 2024-09-10T08:41:41.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:41 smithi174 ceph-mon[34068]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T08:41:41.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:41 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:41:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:41 smithi172 ceph-mon[30854]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T08:41:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:41 smithi172 ceph-mon[30854]: from='client.? 172.21.15.191:0/3657884645' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4e7327f-ca9f-4833-9cd3-b8529b3d9761"}]: dispatch 2024-09-10T08:41:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:41 smithi172 ceph-mon[30854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4e7327f-ca9f-4833-9cd3-b8529b3d9761"}]: dispatch 2024-09-10T08:41:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:41 smithi172 ceph-mon[30854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b4e7327f-ca9f-4833-9cd3-b8529b3d9761"}]': finished 2024-09-10T08:41:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:41 smithi172 ceph-mon[30854]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T08:41:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:41 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:41:42.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:42 smithi174 ceph-mon[34068]: pgmap v175: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 2/6 objects misplaced (33.333%) 2024-09-10T08:41:42.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:42 smithi174 ceph-mon[34068]: from='client.? 172.21.15.191:0/1581317889' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:41:42.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:42 smithi174 ceph-mon[34068]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T08:41:42.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:42 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:41:42.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:42 smithi191 ceph-mon[35314]: pgmap v175: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 2/6 objects misplaced (33.333%) 2024-09-10T08:41:42.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:42 smithi191 ceph-mon[35314]: from='client.? 172.21.15.191:0/1581317889' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:41:42.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:42 smithi191 ceph-mon[35314]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T08:41:42.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:42 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:41:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:42 smithi172 ceph-mon[30854]: pgmap v175: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 2/6 objects misplaced (33.333%) 2024-09-10T08:41:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:42 smithi172 ceph-mon[30854]: from='client.? 172.21.15.191:0/1581317889' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:41:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:42 smithi172 ceph-mon[30854]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T08:41:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:42 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:41:44.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:44 smithi174 ceph-mon[34068]: pgmap v177: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail; 2/6 objects misplaced (33.333%) 2024-09-10T08:41:44.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:44 smithi191 ceph-mon[35314]: pgmap v177: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail; 2/6 objects misplaced (33.333%) 2024-09-10T08:41:44.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:44 smithi172 ceph-mon[30854]: pgmap v177: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail; 2/6 objects misplaced (33.333%) 2024-09-10T08:41:46.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:46 smithi191 ceph-mon[35314]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:46.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:46 smithi174 ceph-mon[34068]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:46.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:46 smithi172 ceph-mon[30854]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:48.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:48 smithi174 ceph-mon[34068]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:48.494 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:48 smithi191 ceph-mon[35314]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:48.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:48 smithi172 ceph-mon[30854]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:50.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:50 smithi191 ceph-mon[35314]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:50.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:50 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T08:41:50.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:50 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:50.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:50 smithi174 ceph-mon[34068]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:50.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:50 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T08:41:50.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:50 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:50.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:50 smithi172 ceph-mon[30854]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:50.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:50 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T08:41:50.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:50 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:51.269 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:51 smithi191 ceph-mon[35314]: Deploying daemon osd.8 on smithi191 2024-09-10T08:41:51.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:51 smithi174 ceph-mon[34068]: Deploying daemon osd.8 on smithi191 2024-09-10T08:41:51.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:51 smithi172 ceph-mon[30854]: Deploying daemon osd.8 on smithi191 2024-09-10T08:41:52.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:52 smithi174 ceph-mon[34068]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:52 smithi191 ceph-mon[35314]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:52 smithi172 ceph-mon[30854]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:54.473 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:54 smithi191 ceph-mon[35314]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:54.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:54 smithi174 ceph-mon[34068]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:54.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:54 smithi172 ceph-mon[30854]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:56.335 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:56 smithi191 ceph-mon[35314]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:56.335 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:56 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:56.335 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:56 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:56.335 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:56 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:56.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:56 smithi174 ceph-mon[34068]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:56.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:56 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:56.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:56 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:56.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:56 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:56.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:56 smithi172 ceph-mon[30854]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:56.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:56 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:41:56.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:56 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:56.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:56 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:58.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:58 smithi191 ceph-mon[35314]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:58.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:58 smithi174 ceph-mon[34068]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:58 smithi172 ceph-mon[30854]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:59.356 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 8 on host 'smithi191' 2024-09-10T08:41:59.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:59.355+0000 7f46637fe640 1 -- 172.21.15.191:0/3619336984 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4674062630 con 0x7f465c0a2c30 2024-09-10T08:41:59.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:59.357+0000 7f4678ef0640 1 -- 172.21.15.191:0/3619336984 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f465c0a2c30 msgr2=0x7f465c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:59.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:59.357+0000 7f4678ef0640 1 --2- 172.21.15.191:0/3619336984 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f465c0a2c30 0x7f465c0a50f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4664005d00 tx=0x7f4664005c50 comp rx=0 tx=0).stop 2024-09-10T08:41:59.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:59.357+0000 7f4678ef0640 1 -- 172.21.15.191:0/3619336984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46741037c0 msgr2=0x7f467419f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:41:59.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:59.357+0000 7f4678ef0640 1 --2- 172.21.15.191:0/3619336984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46741037c0 0x7f467419f370 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f466c00cce0 tx=0x7f466c00a620 comp rx=0 tx=0).stop 2024-09-10T08:41:59.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:59.357+0000 7f4678ef0640 1 -- 172.21.15.191:0/3619336984 shutdown_connections 2024-09-10T08:41:59.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:59.357+0000 7f4678ef0640 1 --2- 172.21.15.191:0/3619336984 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f465c0a2c30 0x7f465c0a50f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:59.357+0000 7f4678ef0640 1 --2- 172.21.15.191:0/3619336984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46741037c0 0x7f467419f370 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:59.357+0000 7f4678ef0640 1 --2- 172.21.15.191:0/3619336984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4674102e00 0x7f46741982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:59.357+0000 7f4678ef0640 1 --2- 172.21.15.191:0/3619336984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4674101c00 0x7f4674197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:41:59.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:59.357+0000 7f4678ef0640 1 -- 172.21.15.191:0/3619336984 >> 172.21.15.191:0/3619336984 conn(0x7f46740fd390 msgr2=0x7f46741061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:41:59.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:59.357+0000 7f4678ef0640 1 -- 172.21.15.191:0/3619336984 shutdown_connections 2024-09-10T08:41:59.359 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:41:59.357+0000 7f4678ef0640 1 -- 172.21.15.191:0/3619336984 wait complete. 2024-09-10T08:41:59.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:59 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:59.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:59 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:59.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:59 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:59.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:59 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:59.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:59 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:59.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:41:59 smithi191 ceph-mon[35314]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:59.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:59 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:59.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:59 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:59.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:59 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:59.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:59 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:59.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:59 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:59.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:41:59 smithi172 ceph-mon[30854]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:41:59.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:59 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:59.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:59 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:59.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:59 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:41:59.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:59 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:41:59.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:59 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:41:59.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:41:59 smithi174 ceph-mon[34068]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:42:00.076 DEBUG:teuthology.orchestra.run.smithi191:osd.8> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.8.service 2024-09-10T08:42:00.079 INFO:tasks.cephadm:Deploying osd.9 on smithi172 with /dev/vg_nvme/lv_1... 2024-09-10T08:42:00.079 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T08:42:00.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:00 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:00.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:00 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:00.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:00 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:00.334 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:42:00.455 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:00 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:00.456 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:00 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:00.456 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:00 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:00.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:00 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:00.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:00 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:00.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:00 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:01.508 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:01 smithi191 ceph-mon[35314]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:42:01.508 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:01 smithi191 ceph-mon[35314]: from='osd.8 [v2:172.21.15.191:6816/3675210430,v1:172.21.15.191:6817/3675210430]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:42:01.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:01 smithi191 ceph-mon[35314]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:42:01.509 INFO:journalctl@ceph.osd.8.smithi191.stdout:Sep 10 08:42:01 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-8[53344]: 2024-09-10T08:42:01.197+0000 7f1d6d2fe640 -1 osd.8 0 log_to_monitors true 2024-09-10T08:42:01.543 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:01 smithi172 ceph-mon[30854]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:42:01.543 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:01 smithi172 ceph-mon[30854]: from='osd.8 [v2:172.21.15.191:6816/3675210430,v1:172.21.15.191:6817/3675210430]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:42:01.543 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:01 smithi172 ceph-mon[30854]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:42:01.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:01 smithi174 ceph-mon[34068]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:42:01.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:01 smithi174 ceph-mon[34068]: from='osd.8 [v2:172.21.15.191:6816/3675210430,v1:172.21.15.191:6817/3675210430]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:42:01.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:01 smithi174 ceph-mon[34068]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T08:42:02.526 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:02 smithi172 ceph-mon[30854]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T08:42:02.527 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:02 smithi172 ceph-mon[30854]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T08:42:02.527 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:02 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:42:02.527 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:02 smithi172 ceph-mon[30854]: from='osd.8 [v2:172.21.15.191:6816/3675210430,v1:172.21.15.191:6817/3675210430]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:42:02.527 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:02 smithi172 ceph-mon[30854]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:42:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:02 smithi191 ceph-mon[35314]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T08:42:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:02 smithi191 ceph-mon[35314]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T08:42:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:02 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:42:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:02 smithi191 ceph-mon[35314]: from='osd.8 [v2:172.21.15.191:6816/3675210430,v1:172.21.15.191:6817/3675210430]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:42:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:02 smithi191 ceph-mon[35314]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:42:02.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:02 smithi174 ceph-mon[34068]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T08:42:02.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:02 smithi174 ceph-mon[34068]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T08:42:02.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:02 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:42:02.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:02 smithi174 ceph-mon[34068]: from='osd.8 [v2:172.21.15.191:6816/3675210430,v1:172.21.15.191:6817/3675210430]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:42:02.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:02 smithi174 ceph-mon[34068]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:42:03.021 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:42:03.047 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_1 2024-09-10T08:42:03.284 INFO:journalctl@ceph.osd.8.smithi191.stdout:Sep 10 08:42:02 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-8[53344]: 2024-09-10T08:42:02.987+0000 7f1d69281640 -1 osd.8 0 waiting for initial osdmap 2024-09-10T08:42:03.284 INFO:journalctl@ceph.osd.8.smithi191.stdout:Sep 10 08:42:02 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-8[53344]: 2024-09-10T08:42:02.995+0000 7f1d610a1640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:42:03.301 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:42:03.541 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-10T08:42:03.541 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T08:42:03.541 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:42:03.541 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:42:03.541 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: Detected new or changed devices on smithi191 2024-09-10T08:42:03.542 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:03.542 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:03.542 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:03.542 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:03.542 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:03.542 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi191 to 5696M 2024-09-10T08:42:03.542 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:03.542 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:03.542 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:03.542 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:03.542 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: from='osd.8 ' entity='osd.8' 2024-09-10T08:42:03.542 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:42:03.543 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: osd.8 [v2:172.21.15.191:6816/3675210430,v1:172.21.15.191:6817/3675210430] boot 2024-09-10T08:42:03.543 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T08:42:03.543 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:42:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-10T08:42:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T08:42:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:42:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:42:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: Detected new or changed devices on smithi191 2024-09-10T08:42:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi191 to 5696M 2024-09-10T08:42:03.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:03.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:03.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:03.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:03.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: from='osd.8 ' entity='osd.8' 2024-09-10T08:42:03.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:42:03.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: osd.8 [v2:172.21.15.191:6816/3675210430,v1:172.21.15.191:6817/3675210430] boot 2024-09-10T08:42:03.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T08:42:03.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:42:03.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-10T08:42:03.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T08:42:03.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:42:03.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:42:03.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: Detected new or changed devices on smithi191 2024-09-10T08:42:03.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:03.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:03.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:03.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:03.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:03.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi191 to 5696M 2024-09-10T08:42:03.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:03.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:03.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:03.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:03.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: from='osd.8 ' entity='osd.8' 2024-09-10T08:42:03.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-10T08:42:03.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: osd.8 [v2:172.21.15.191:6816/3675210430,v1:172.21.15.191:6817/3675210430] boot 2024-09-10T08:42:03.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T08:42:03.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:03 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T08:42:03.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.814+0000 7fb557eba640 1 -- 172.21.15.172:0/543016670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb550101d10 msgr2=0x7fb550102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:03.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.814+0000 7fb557eba640 1 --2- 172.21.15.172:0/543016670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb550101d10 0x7fb550102190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb540009900 tx=0x7fb54002a410 comp rx=0 tx=0).stop 2024-09-10T08:42:03.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.814+0000 7fb557eba640 1 -- 172.21.15.172:0/543016670 shutdown_connections 2024-09-10T08:42:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.814+0000 7fb557eba640 1 --2- 172.21.15.172:0/543016670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5501026d0 0x7fb550108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.814+0000 7fb557eba640 1 --2- 172.21.15.172:0/543016670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb550101d10 0x7fb550102190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.814+0000 7fb557eba640 1 --2- 172.21.15.172:0/543016670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb550100b10 0x7fb550100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.814+0000 7fb557eba640 1 -- 172.21.15.172:0/543016670 >> 172.21.15.172:0/543016670 conn(0x7fb5500fc2a0 msgr2=0x7fb5500fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.814+0000 7fb557eba640 1 -- 172.21.15.172:0/543016670 shutdown_connections 2024-09-10T08:42:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.814+0000 7fb557eba640 1 -- 172.21.15.172:0/543016670 wait complete. 2024-09-10T08:42:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.814+0000 7fb557eba640 1 Processor -- start 2024-09-10T08:42:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.814+0000 7fb557eba640 1 -- start start 2024-09-10T08:42:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb557eba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb550100b10 0x7fb55006b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb557eba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb550101d10 0x7fb55019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:03.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb557eba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5501026d0 0x7fb55019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb557eba640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb55010c6c0 con 0x7fb550101d10 2024-09-10T08:42:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb557eba640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb55010c540 con 0x7fb5501026d0 2024-09-10T08:42:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb557eba640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb55010c840 con 0x7fb550100b10 2024-09-10T08:42:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb555455640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb550101d10 0x7fb55019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb555455640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb550101d10 0x7fb55019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57870/0 (socket says 172.21.15.172:57870) 2024-09-10T08:42:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb555455640 1 -- 172.21.15.172:0/674630973 learned_addr learned my addr 172.21.15.172:0/674630973 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:42:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb556457640 1 --2- 172.21.15.172:0/674630973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5501026d0 0x7fb55019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb555c56640 1 --2- 172.21.15.172:0/674630973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb550100b10 0x7fb55006b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb555455640 1 -- 172.21.15.172:0/674630973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb550100b10 msgr2=0x7fb55006b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb555455640 1 --2- 172.21.15.172:0/674630973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb550100b10 0x7fb55006b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb555455640 1 -- 172.21.15.172:0/674630973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5501026d0 msgr2=0x7fb55019f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:03.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb555455640 1 --2- 172.21.15.172:0/674630973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5501026d0 0x7fb55019f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:03.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb555455640 1 -- 172.21.15.172:0/674630973 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55019fb30 con 0x7fb550101d10 2024-09-10T08:42:03.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb556457640 1 --2- 172.21.15.172:0/674630973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5501026d0 0x7fb55019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:42:03.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb555455640 1 --2- 172.21.15.172:0/674630973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb550101d10 0x7fb55019cc20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb540005940 tx=0x7fb540004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:03.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb546ffd640 1 -- 172.21.15.172:0/674630973 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54003f030 con 0x7fb550101d10 2024-09-10T08:42:03.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb546ffd640 1 -- 172.21.15.172:0/674630973 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb540003cb0 con 0x7fb550101d10 2024-09-10T08:42:03.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb557eba640 1 -- 172.21.15.172:0/674630973 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb55019fdc0 con 0x7fb550101d10 2024-09-10T08:42:03.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb546ffd640 1 -- 172.21.15.172:0/674630973 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540035070 con 0x7fb550101d10 2024-09-10T08:42:03.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.815+0000 7fb557eba640 1 -- 172.21.15.172:0/674630973 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5501a02e0 con 0x7fb550101d10 2024-09-10T08:42:03.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.817+0000 7fb557eba640 1 -- 172.21.15.172:0/674630973 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb550108000 con 0x7fb550101d10 2024-09-10T08:42:03.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.817+0000 7fb546ffd640 1 -- 172.21.15.172:0/674630973 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb540042030 con 0x7fb550101d10 2024-09-10T08:42:03.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.818+0000 7fb546ffd640 1 --2- 172.21.15.172:0/674630973 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb53c0a2c00 0x7fb53c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:03.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.818+0000 7fb555c56640 1 --2- 172.21.15.172:0/674630973 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb53c0a2c00 0x7fb53c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:03.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.818+0000 7fb546ffd640 1 -- 172.21.15.172:0/674630973 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5862+0+0 (secure 0 0 0) 0x7fb5400ef180 con 0x7fb550101d10 2024-09-10T08:42:03.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.819+0000 7fb555c56640 1 --2- 172.21.15.172:0/674630973 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb53c0a2c00 0x7fb53c0a50c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb55006b870 tx=0x7fb54c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:03.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:03.822+0000 7fb546ffd640 1 -- 172.21.15.172:0/674630973 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5400bca70 con 0x7fb550101d10 2024-09-10T08:42:04.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:04.001+0000 7fb557eba640 1 -- 172.21.15.172:0/674630973 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb550062630 con 0x7fb53c0a2c00 2024-09-10T08:42:05.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:04 smithi191 ceph-mon[35314]: purged_snaps scrub starts 2024-09-10T08:42:05.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:04 smithi191 ceph-mon[35314]: purged_snaps scrub ok 2024-09-10T08:42:05.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:04 smithi191 ceph-mon[35314]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:42:05.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:04 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:42:05.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:04 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:42:05.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:04 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:05.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:04 smithi191 ceph-mon[35314]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T08:42:05.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:04 smithi172 ceph-mon[30854]: purged_snaps scrub starts 2024-09-10T08:42:05.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:04 smithi172 ceph-mon[30854]: purged_snaps scrub ok 2024-09-10T08:42:05.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:04 smithi172 ceph-mon[30854]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:42:05.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:04 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:42:05.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:04 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:42:05.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:04 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:05.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:04 smithi172 ceph-mon[30854]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T08:42:05.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:04 smithi174 ceph-mon[34068]: purged_snaps scrub starts 2024-09-10T08:42:05.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:04 smithi174 ceph-mon[34068]: purged_snaps scrub ok 2024-09-10T08:42:05.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:04 smithi174 ceph-mon[34068]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:42:05.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:04 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:42:05.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:04 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:42:05.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:04 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:05.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:04 smithi174 ceph-mon[34068]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T08:42:06.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:06 smithi174 ceph-mon[34068]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:06 smithi174 ceph-mon[34068]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T08:42:06.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:06 smithi191 ceph-mon[35314]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:06.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:06 smithi191 ceph-mon[35314]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T08:42:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:06 smithi172 ceph-mon[30854]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:06 smithi172 ceph-mon[30854]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T08:42:07.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:07 smithi172 ceph-mon[30854]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T08:42:07.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:07 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/399865404' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f6153f2-f9db-4c00-a659-c932628016f4"}]: dispatch 2024-09-10T08:42:07.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:07 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/399865404' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f6153f2-f9db-4c00-a659-c932628016f4"}]': finished 2024-09-10T08:42:07.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:07 smithi172 ceph-mon[30854]: osdmap e61: 10 total, 9 up, 10 in 2024-09-10T08:42:07.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:07 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:07.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:07 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/840271727' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:42:07.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:07 smithi174 ceph-mon[34068]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T08:42:07.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:07 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/399865404' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f6153f2-f9db-4c00-a659-c932628016f4"}]: dispatch 2024-09-10T08:42:07.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:07 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/399865404' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f6153f2-f9db-4c00-a659-c932628016f4"}]': finished 2024-09-10T08:42:07.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:07 smithi174 ceph-mon[34068]: osdmap e61: 10 total, 9 up, 10 in 2024-09-10T08:42:07.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:07 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:07.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:07 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/840271727' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:42:07.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:07 smithi191 ceph-mon[35314]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T08:42:07.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:07 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/399865404' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f6153f2-f9db-4c00-a659-c932628016f4"}]: dispatch 2024-09-10T08:42:07.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:07 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/399865404' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f6153f2-f9db-4c00-a659-c932628016f4"}]': finished 2024-09-10T08:42:07.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:07 smithi191 ceph-mon[35314]: osdmap e61: 10 total, 9 up, 10 in 2024-09-10T08:42:07.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:07 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:07.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:07 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/840271727' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:42:08.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:08 smithi174 ceph-mon[34068]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:08.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:08 smithi191 ceph-mon[35314]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:08.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:08 smithi172 ceph-mon[30854]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:10.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:10 smithi174 ceph-mon[34068]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-10T08:42:10.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:10 smithi191 ceph-mon[35314]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-10T08:42:10.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:10 smithi172 ceph-mon[30854]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-10T08:42:12.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:12 smithi172 ceph-mon[30854]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:42:12.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:12 smithi174 ceph-mon[34068]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:42:12.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:12 smithi191 ceph-mon[35314]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T08:42:14.205 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:14 smithi172 ceph-mon[30854]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-10T08:42:14.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:14 smithi174 ceph-mon[34068]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-10T08:42:14.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:14 smithi191 ceph-mon[35314]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-10T08:42:16.331 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:16 smithi172 ceph-mon[30854]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-10T08:42:16.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:16 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T08:42:16.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:16 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:16.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:16 smithi174 ceph-mon[34068]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-10T08:42:16.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:16 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T08:42:16.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:16 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:16.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:16 smithi191 ceph-mon[35314]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-10T08:42:16.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:16 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T08:42:16.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:16 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:17.320 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:17 smithi172 ceph-mon[30854]: Deploying daemon osd.9 on smithi172 2024-09-10T08:42:17.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:17 smithi174 ceph-mon[34068]: Deploying daemon osd.9 on smithi172 2024-09-10T08:42:17.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:17 smithi191 ceph-mon[35314]: Deploying daemon osd.9 on smithi172 2024-09-10T08:42:18.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:18 smithi172 ceph-mon[30854]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T08:42:18.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:18 smithi174 ceph-mon[34068]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T08:42:18.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:18 smithi191 ceph-mon[35314]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T08:42:20.464 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:20 smithi172 ceph-mon[30854]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:42:20.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:20 smithi174 ceph-mon[34068]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:42:20.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:20 smithi191 ceph-mon[35314]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T08:42:22.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:22 smithi172 ceph-mon[30854]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:22.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:22 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:22.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:22 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:22.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:22 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:22.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:22 smithi174 ceph-mon[34068]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:22.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:22 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:22.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:22 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:22.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:22 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:22 smithi191 ceph-mon[35314]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:22 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:22 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:22 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:24.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:24 smithi172 ceph-mon[30854]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:24.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:24 smithi174 ceph-mon[34068]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:24 smithi191 ceph-mon[35314]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:24.892 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 9 on host 'smithi172' 2024-09-10T08:42:24.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:24.890+0000 7fb546ffd640 1 -- 172.21.15.172:0/674630973 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb550062630 con 0x7fb53c0a2c00 2024-09-10T08:42:24.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:24.894+0000 7fb557eba640 1 -- 172.21.15.172:0/674630973 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb53c0a2c00 msgr2=0x7fb53c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:24.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:24.894+0000 7fb557eba640 1 --2- 172.21.15.172:0/674630973 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb53c0a2c00 0x7fb53c0a50c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb55006b870 tx=0x7fb54c008040 comp rx=0 tx=0).stop 2024-09-10T08:42:24.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:24.894+0000 7fb557eba640 1 -- 172.21.15.172:0/674630973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb550101d10 msgr2=0x7fb55019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:24.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:24.894+0000 7fb557eba640 1 --2- 172.21.15.172:0/674630973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb550101d10 0x7fb55019cc20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb540005940 tx=0x7fb540004430 comp rx=0 tx=0).stop 2024-09-10T08:42:24.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:24.894+0000 7fb557eba640 1 -- 172.21.15.172:0/674630973 shutdown_connections 2024-09-10T08:42:24.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:24.894+0000 7fb557eba640 1 --2- 172.21.15.172:0/674630973 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb53c0a2c00 0x7fb53c0a50c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:24.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:24.894+0000 7fb557eba640 1 --2- 172.21.15.172:0/674630973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5501026d0 0x7fb55019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:24.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:24.894+0000 7fb557eba640 1 --2- 172.21.15.172:0/674630973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb550101d10 0x7fb55019cc20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:24.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:24.894+0000 7fb557eba640 1 --2- 172.21.15.172:0/674630973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb550100b10 0x7fb55006b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:24.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:24.894+0000 7fb557eba640 1 -- 172.21.15.172:0/674630973 >> 172.21.15.172:0/674630973 conn(0x7fb5500fc2a0 msgr2=0x7fb5501051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:24.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:24.894+0000 7fb557eba640 1 -- 172.21.15.172:0/674630973 shutdown_connections 2024-09-10T08:42:24.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:42:24.894+0000 7fb557eba640 1 -- 172.21.15.172:0/674630973 wait complete. 2024-09-10T08:42:25.569 DEBUG:teuthology.orchestra.run.smithi172:osd.9> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.9.service 2024-09-10T08:42:25.572 INFO:tasks.cephadm:Deploying osd.10 on smithi174 with /dev/vg_nvme/lv_1... 2024-09-10T08:42:25.572 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T08:42:25.830 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:42:26.167 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:25 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:26.167 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:25 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:26.167 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:25 smithi172 ceph-mon[30854]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:26.167 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:25 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:26.167 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:25 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:26.192 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:25 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:26.192 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:25 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:26.192 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:25 smithi174 ceph-mon[34068]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:26.193 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:25 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:26.193 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:25 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:26.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:25 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:26.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:25 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:26.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:25 smithi191 ceph-mon[35314]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:26.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:25 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:26.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:25 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:27.822 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 08:42:27 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T08:42:27.660+0000 7f551895e640 -1 osd.9 0 log_to_monitors true 2024-09-10T08:42:28.468 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:28 smithi174 ceph-mon[34068]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:28.468 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:28 smithi174 ceph-mon[34068]: from='osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T08:42:28.471 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T08:42:28.500 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_1 2024-09-10T08:42:28.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:28 smithi191 ceph-mon[35314]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:28.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:28 smithi191 ceph-mon[35314]: from='osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T08:42:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:28 smithi172 ceph-mon[30854]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:28 smithi172 ceph-mon[30854]: from='osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T08:42:28.744 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T08:42:29.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.222+0000 7f540b092640 1 -- 172.21.15.174:0/4131089497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5404101d10 msgr2=0x7f5404102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:29.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.222+0000 7f540b092640 1 --2- 172.21.15.174:0/4131089497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5404101d10 0x7f5404102190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009900 tx=0x7f53f002a410 comp rx=0 tx=0).stop 2024-09-10T08:42:29.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.222+0000 7f540b092640 1 -- 172.21.15.174:0/4131089497 shutdown_connections 2024-09-10T08:42:29.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.222+0000 7f540b092640 1 --2- 172.21.15.174:0/4131089497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54041026d0 0x7f5404108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:29.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.222+0000 7f540b092640 1 --2- 172.21.15.174:0/4131089497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5404101d10 0x7f5404102190 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:29.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.222+0000 7f540b092640 1 --2- 172.21.15.174:0/4131089497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5404100b10 0x7f5404100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:29.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.222+0000 7f540b092640 1 -- 172.21.15.174:0/4131089497 >> 172.21.15.174:0/4131089497 conn(0x7f54040fc2a0 msgr2=0x7f54040fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:29.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.222+0000 7f540b092640 1 -- 172.21.15.174:0/4131089497 shutdown_connections 2024-09-10T08:42:29.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.222+0000 7f540b092640 1 -- 172.21.15.174:0/4131089497 wait complete. 2024-09-10T08:42:29.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.222+0000 7f540b092640 1 Processor -- start 2024-09-10T08:42:29.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f540b092640 1 -- start start 2024-09-10T08:42:29.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f540b092640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5404100b10 0x7f5404197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:29.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f540b092640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5404101d10 0x7f5404198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:29.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f540b092640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54041026d0 0x7f540419f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:29.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f540b092640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f540410c6a0 con 0x7f54041026d0 2024-09-10T08:42:29.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f540b092640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f540410c520 con 0x7f5404101d10 2024-09-10T08:42:29.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f540b092640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f540410c820 con 0x7f5404100b10 2024-09-10T08:42:29.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f5403fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5404101d10 0x7f5404198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:29.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f5403fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5404101d10 0x7f5404198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:39640/0 (socket says 172.21.15.174:39640) 2024-09-10T08:42:29.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f5403fff640 1 -- 172.21.15.174:0/2807872656 learned_addr learned my addr 172.21.15.174:0/2807872656 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-10T08:42:29.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f540962f640 1 --2- 172.21.15.174:0/2807872656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54041026d0 0x7f540419f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:29.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f5408e2e640 1 --2- 172.21.15.174:0/2807872656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5404100b10 0x7f5404197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:29.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f5403fff640 1 -- 172.21.15.174:0/2807872656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5404100b10 msgr2=0x7f5404197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:29.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f5403fff640 1 --2- 172.21.15.174:0/2807872656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5404100b10 0x7f5404197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:29.241 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f5403fff640 1 -- 172.21.15.174:0/2807872656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54041026d0 msgr2=0x7f540419f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:29.242 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f5403fff640 1 --2- 172.21.15.174:0/2807872656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54041026d0 0x7f540419f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:29.242 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f5403fff640 1 -- 172.21.15.174:0/2807872656 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540419fb80 con 0x7f5404101d10 2024-09-10T08:42:29.242 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f540962f640 1 --2- 172.21.15.174:0/2807872656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54041026d0 0x7f540419f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:42:29.242 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f5408e2e640 1 --2- 172.21.15.174:0/2807872656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5404100b10 0x7f5404197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:42:29.242 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f5403fff640 1 --2- 172.21.15.174:0/2807872656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5404101d10 0x7f5404198490 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f53f0004770 tx=0x7f53f0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:29.242 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f5401ffb640 1 -- 172.21.15.174:0/2807872656 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f003f030 con 0x7f5404101d10 2024-09-10T08:42:29.242 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f5401ffb640 1 -- 172.21.15.174:0/2807872656 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53f0003aa0 con 0x7f5404101d10 2024-09-10T08:42:29.242 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f5401ffb640 1 -- 172.21.15.174:0/2807872656 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0035070 con 0x7f5404101d10 2024-09-10T08:42:29.242 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.223+0000 7f540b092640 1 -- 172.21.15.174:0/2807872656 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540419fe10 con 0x7f5404101d10 2024-09-10T08:42:29.242 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.224+0000 7f540b092640 1 -- 172.21.15.174:0/2807872656 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54041a0290 con 0x7f5404101d10 2024-09-10T08:42:29.242 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.225+0000 7f540b092640 1 -- 172.21.15.174:0/2807872656 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54041a04c0 con 0x7f5404101d10 2024-09-10T08:42:29.242 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.226+0000 7f5401ffb640 1 -- 172.21.15.174:0/2807872656 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f53f0042030 con 0x7f5404101d10 2024-09-10T08:42:29.242 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.227+0000 7f5401ffb640 1 --2- 172.21.15.174:0/2807872656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53f40a2c30 0x7f53f40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:29.243 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.227+0000 7f5401ffb640 1 -- 172.21.15.174:0/2807872656 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5981+0+0 (secure 0 0 0) 0x7f53f00ef4d0 con 0x7f5404101d10 2024-09-10T08:42:29.243 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.227+0000 7f5408e2e640 1 --2- 172.21.15.174:0/2807872656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53f40a2c30 0x7f53f40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:29.243 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.228+0000 7f5408e2e640 1 --2- 172.21.15.174:0/2807872656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53f40a2c30 0x7f53f40a50f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5404101b70 tx=0x7f53f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:29.243 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.230+0000 7f5401ffb640 1 -- 172.21.15.174:0/2807872656 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53f00bcd50 con 0x7f5404101d10 2024-09-10T08:42:29.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:29 smithi172 ceph-mon[30854]: from='osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T08:42:29.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:29 smithi172 ceph-mon[30854]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T08:42:29.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:29 smithi172 ceph-mon[30854]: from='osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-10T08:42:29.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:29 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:29.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:29 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:29.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:29 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:29.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:29 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:29.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:29 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:29.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:29 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:29.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:29 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:29.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:29 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:29.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:29 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:29.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:29 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:29.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:29 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:29.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:29 smithi172 ceph-mon[30854]: from='osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-10T08:42:29.411 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:29.410+0000 7f540b092640 1 -- 172.21.15.174:0/2807872656 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f540406af30 con 0x7f53f40a2c30 2024-09-10T08:42:29.411 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:29 smithi174 ceph-mon[34068]: from='osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T08:42:29.411 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:29 smithi174 ceph-mon[34068]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T08:42:29.411 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:29 smithi174 ceph-mon[34068]: from='osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-10T08:42:29.411 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:29 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:29.411 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:29 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:29.412 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:29 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:29.412 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:29 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:29.412 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:29 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:29.412 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:29 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:29.412 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:29 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:29.412 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:29 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:29.412 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:29 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:29.412 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:29 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:29.412 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:29 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:29.412 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:29 smithi174 ceph-mon[34068]: from='osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-10T08:42:29.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:29 smithi191 ceph-mon[35314]: from='osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T08:42:29.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:29 smithi191 ceph-mon[35314]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T08:42:29.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:29 smithi191 ceph-mon[35314]: from='osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-10T08:42:29.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:29 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:29.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:29 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:29.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:29 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:29.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:29 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:29.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:29 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:29.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:29 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:29.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:29 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:29.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:29 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:29.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:29 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:29.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:29 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:29.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:29 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:29.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:29 smithi191 ceph-mon[35314]: from='osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-10T08:42:30.082 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 08:42:29 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T08:42:29.778+0000 7f55148e1640 -1 osd.9 0 waiting for initial osdmap 2024-09-10T08:42:30.082 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 08:42:29 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T08:42:29.790+0000 7f550c701640 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:42:30.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:30 smithi174 ceph-mon[34068]: Detected new or changed devices on smithi172 2024-09-10T08:42:30.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:30 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi172 to 4272M 2024-09-10T08:42:30.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:30 smithi174 ceph-mon[34068]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:30.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:30 smithi174 ceph-mon[34068]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T08:42:30.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:30.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:30.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:30.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:30.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:30.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:30.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:42:30.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:42:30.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:30.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:30 smithi174 ceph-mon[34068]: from='osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619]' entity='osd.9' 2024-09-10T08:42:30.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:30.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:30 smithi191 ceph-mon[35314]: Detected new or changed devices on smithi172 2024-09-10T08:42:30.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:30 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi172 to 4272M 2024-09-10T08:42:30.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:30 smithi191 ceph-mon[35314]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:30.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:30 smithi191 ceph-mon[35314]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T08:42:30.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:30.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:30.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:30.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:30.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:30.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:30.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:42:30.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:42:30.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:30.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:30 smithi191 ceph-mon[35314]: from='osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619]' entity='osd.9' 2024-09-10T08:42:30.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:30.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:30 smithi172 ceph-mon[30854]: Detected new or changed devices on smithi172 2024-09-10T08:42:30.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:30 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi172 to 4272M 2024-09-10T08:42:30.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:30 smithi172 ceph-mon[30854]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:30.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:30 smithi172 ceph-mon[30854]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T08:42:30.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:30.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:30.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:30.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:30.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:30.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:30.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:42:30.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:42:30.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:30.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:30 smithi172 ceph-mon[30854]: from='osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619]' entity='osd.9' 2024-09-10T08:42:30.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:31.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:31 smithi174 ceph-mon[34068]: purged_snaps scrub starts 2024-09-10T08:42:31.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:31 smithi174 ceph-mon[34068]: purged_snaps scrub ok 2024-09-10T08:42:31.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:31 smithi174 ceph-mon[34068]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:42:31.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:31 smithi174 ceph-mon[34068]: osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619] boot 2024-09-10T08:42:31.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:31 smithi174 ceph-mon[34068]: osdmap e64: 10 total, 10 up, 10 in 2024-09-10T08:42:31.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:31 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:31.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:31 smithi191 ceph-mon[35314]: purged_snaps scrub starts 2024-09-10T08:42:31.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:31 smithi191 ceph-mon[35314]: purged_snaps scrub ok 2024-09-10T08:42:31.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:31 smithi191 ceph-mon[35314]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:42:31.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:31 smithi191 ceph-mon[35314]: osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619] boot 2024-09-10T08:42:31.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:31 smithi191 ceph-mon[35314]: osdmap e64: 10 total, 10 up, 10 in 2024-09-10T08:42:31.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:31 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:31.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:31 smithi172 ceph-mon[30854]: purged_snaps scrub starts 2024-09-10T08:42:31.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:31 smithi172 ceph-mon[30854]: purged_snaps scrub ok 2024-09-10T08:42:31.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:31 smithi172 ceph-mon[30854]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:42:31.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:31 smithi172 ceph-mon[30854]: osd.9 [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619] boot 2024-09-10T08:42:31.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:31 smithi172 ceph-mon[30854]: osdmap e64: 10 total, 10 up, 10 in 2024-09-10T08:42:31.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:31 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T08:42:32.485 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:32 smithi174 ceph-mon[34068]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:32.485 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:32 smithi174 ceph-mon[34068]: from='client.? 172.21.15.174:0/1718123786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "040a013a-4c3e-442f-bea6-12cbe517716d"}]: dispatch 2024-09-10T08:42:32.485 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:32 smithi174 ceph-mon[34068]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "040a013a-4c3e-442f-bea6-12cbe517716d"}]: dispatch 2024-09-10T08:42:32.485 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:32 smithi174 ceph-mon[34068]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "040a013a-4c3e-442f-bea6-12cbe517716d"}]': finished 2024-09-10T08:42:32.485 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:32 smithi174 ceph-mon[34068]: osdmap e65: 11 total, 10 up, 11 in 2024-09-10T08:42:32.485 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:32 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:32 smithi191 ceph-mon[35314]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:32 smithi191 ceph-mon[35314]: from='client.? 172.21.15.174:0/1718123786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "040a013a-4c3e-442f-bea6-12cbe517716d"}]: dispatch 2024-09-10T08:42:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:32 smithi191 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "040a013a-4c3e-442f-bea6-12cbe517716d"}]: dispatch 2024-09-10T08:42:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:32 smithi191 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "040a013a-4c3e-442f-bea6-12cbe517716d"}]': finished 2024-09-10T08:42:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:32 smithi191 ceph-mon[35314]: osdmap e65: 11 total, 10 up, 11 in 2024-09-10T08:42:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:32 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:32.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:32 smithi172 ceph-mon[30854]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T08:42:32.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:32 smithi172 ceph-mon[30854]: from='client.? 172.21.15.174:0/1718123786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "040a013a-4c3e-442f-bea6-12cbe517716d"}]: dispatch 2024-09-10T08:42:32.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:32 smithi172 ceph-mon[30854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "040a013a-4c3e-442f-bea6-12cbe517716d"}]: dispatch 2024-09-10T08:42:32.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:32 smithi172 ceph-mon[30854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "040a013a-4c3e-442f-bea6-12cbe517716d"}]': finished 2024-09-10T08:42:32.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:32 smithi172 ceph-mon[30854]: osdmap e65: 11 total, 10 up, 11 in 2024-09-10T08:42:32.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:32 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:33.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:33 smithi174 ceph-mon[34068]: from='client.? 172.21.15.174:0/4028913860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:42:33.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:33 smithi191 ceph-mon[35314]: from='client.? 172.21.15.174:0/4028913860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:42:33.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:33 smithi172 ceph-mon[30854]: from='client.? 172.21.15.174:0/4028913860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:42:34.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:34 smithi174 ceph-mon[34068]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:34.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:34 smithi191 ceph-mon[35314]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:34.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:34 smithi172 ceph-mon[30854]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:36.346 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:36 smithi174 ceph-mon[34068]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:36 smithi191 ceph-mon[35314]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:36 smithi172 ceph-mon[30854]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:38.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:38 smithi174 ceph-mon[34068]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:38.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:38 smithi191 ceph-mon[35314]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:38.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:38 smithi172 ceph-mon[30854]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:40.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:40 smithi174 ceph-mon[34068]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:40.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:40 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T08:42:40.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:40 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:40 smithi191 ceph-mon[35314]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:40 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T08:42:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:40 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:40.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:40 smithi172 ceph-mon[30854]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:40.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T08:42:40.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:41.302 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:41 smithi174 ceph-mon[34068]: Deploying daemon osd.10 on smithi174 2024-09-10T08:42:41.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:41 smithi191 ceph-mon[35314]: Deploying daemon osd.10 on smithi174 2024-09-10T08:42:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:41 smithi172 ceph-mon[30854]: Deploying daemon osd.10 on smithi174 2024-09-10T08:42:42.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:42 smithi174 ceph-mon[34068]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:42.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:42 smithi191 ceph-mon[35314]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:42 smithi172 ceph-mon[30854]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:44.437 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:44 smithi174 ceph-mon[34068]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:44.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:44 smithi191 ceph-mon[35314]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:44.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:44 smithi172 ceph-mon[30854]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:46.413 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:46 smithi174 ceph-mon[34068]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:46.413 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:46 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:46.413 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:46 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:46.413 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:46 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:46.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:46 smithi191 ceph-mon[35314]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:46.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:46 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:46.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:46 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:46.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:46 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:46.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:46 smithi172 ceph-mon[30854]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:46.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:46 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:46.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:46 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:46.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:46 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:48.377 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:48 smithi174 ceph-mon[34068]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:48.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:48 smithi191 ceph-mon[35314]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:48.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:48 smithi172 ceph-mon[30854]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:49.393 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:49.392+0000 7f5401ffb640 1 -- 172.21.15.174:0/2807872656 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f540406af30 con 0x7f53f40a2c30 2024-09-10T08:42:49.393 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 10 on host 'smithi174' 2024-09-10T08:42:49.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:49.394+0000 7f540b092640 1 -- 172.21.15.174:0/2807872656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53f40a2c30 msgr2=0x7f53f40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:49.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:49.394+0000 7f540b092640 1 --2- 172.21.15.174:0/2807872656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53f40a2c30 0x7f53f40a50f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5404101b70 tx=0x7f53f8005f90 comp rx=0 tx=0).stop 2024-09-10T08:42:49.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:49.395+0000 7f540b092640 1 -- 172.21.15.174:0/2807872656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5404101d10 msgr2=0x7f5404198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:49.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:49.395+0000 7f540b092640 1 --2- 172.21.15.174:0/2807872656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5404101d10 0x7f5404198490 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f53f0004770 tx=0x7f53f0004270 comp rx=0 tx=0).stop 2024-09-10T08:42:49.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:49.395+0000 7f540b092640 1 -- 172.21.15.174:0/2807872656 shutdown_connections 2024-09-10T08:42:49.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:49.395+0000 7f540b092640 1 --2- 172.21.15.174:0/2807872656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53f40a2c30 0x7f53f40a50f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:49.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:49.395+0000 7f540b092640 1 --2- 172.21.15.174:0/2807872656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54041026d0 0x7f540419f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:49.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:49.395+0000 7f540b092640 1 --2- 172.21.15.174:0/2807872656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5404101d10 0x7f5404198490 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:49.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:49.395+0000 7f540b092640 1 --2- 172.21.15.174:0/2807872656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5404100b10 0x7f5404197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:49.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:49.395+0000 7f540b092640 1 -- 172.21.15.174:0/2807872656 >> 172.21.15.174:0/2807872656 conn(0x7f54040fc2a0 msgr2=0x7f540406a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:49.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:49.395+0000 7f540b092640 1 -- 172.21.15.174:0/2807872656 shutdown_connections 2024-09-10T08:42:49.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T08:42:49.395+0000 7f540b092640 1 -- 172.21.15.174:0/2807872656 wait complete. 2024-09-10T08:42:49.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:49 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:49.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:49 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:49.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:49 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:49.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:49 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:49.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:49 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:49.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:49 smithi174 ceph-mon[34068]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:49.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:49 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:49.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:49 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:49.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:49 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:50.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:49 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:50.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:49 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:50.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:49 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:50.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:49 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:50.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:49 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:50.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:49 smithi191 ceph-mon[35314]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:50.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:49 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:50.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:49 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:50.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:49 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:50.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:49 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:50.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:49 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:50.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:49 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:50.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:49 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:50.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:49 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:50.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:49 smithi172 ceph-mon[30854]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:50.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:49 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:42:50.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:49 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:50.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:49 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:50.134 DEBUG:teuthology.orchestra.run.smithi174:osd.10> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.10.service 2024-09-10T08:42:50.136 INFO:tasks.cephadm:Deploying osd.11 on smithi191 with /dev/vg_nvme/lv_1... 2024-09-10T08:42:50.136 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T08:42:50.391 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:42:51.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 08:42:51 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T08:42:51.771+0000 7f983d6a1640 -1 osd.10 0 log_to_monitors true 2024-09-10T08:42:52.396 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:52 smithi191 ceph-mon[35314]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:52.396 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:52 smithi191 ceph-mon[35314]: from='osd.10 [v2:172.21.15.174:6824/1238812525,v1:172.21.15.174:6825/1238812525]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:42:52.396 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:52 smithi191 ceph-mon[35314]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:42:52.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:52 smithi174 ceph-mon[34068]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:52.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:52 smithi174 ceph-mon[34068]: from='osd.10 [v2:172.21.15.174:6824/1238812525,v1:172.21.15.174:6825/1238812525]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:42:52.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:52 smithi174 ceph-mon[34068]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:42:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:52 smithi172 ceph-mon[30854]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:52 smithi172 ceph-mon[30854]: from='osd.10 [v2:172.21.15.174:6824/1238812525,v1:172.21.15.174:6825/1238812525]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:42:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:52 smithi172 ceph-mon[30854]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T08:42:53.190 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-10T08:42:53.217 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi191:vg_nvme/lv_1 2024-09-10T08:42:53.467 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:42:53.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:53 smithi174 ceph-mon[34068]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T08:42:53.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:53 smithi174 ceph-mon[34068]: osdmap e66: 11 total, 10 up, 11 in 2024-09-10T08:42:53.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:53 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:53.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:53 smithi174 ceph-mon[34068]: from='osd.10 [v2:172.21.15.174:6824/1238812525,v1:172.21.15.174:6825/1238812525]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:42:53.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:53 smithi174 ceph-mon[34068]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:42:53.493 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 08:42:53 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T08:42:53.163+0000 7f9839624640 -1 osd.10 0 waiting for initial osdmap 2024-09-10T08:42:53.494 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 08:42:53 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T08:42:53.175+0000 7f983544c640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:42:53.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:53 smithi191 ceph-mon[35314]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T08:42:53.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:53 smithi191 ceph-mon[35314]: osdmap e66: 11 total, 10 up, 11 in 2024-09-10T08:42:53.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:53 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:53.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:53 smithi191 ceph-mon[35314]: from='osd.10 [v2:172.21.15.174:6824/1238812525,v1:172.21.15.174:6825/1238812525]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:42:53.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:53 smithi191 ceph-mon[35314]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:42:53.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:53 smithi172 ceph-mon[30854]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T08:42:53.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:53 smithi172 ceph-mon[30854]: osdmap e66: 11 total, 10 up, 11 in 2024-09-10T08:42:53.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:53 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:53.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:53 smithi172 ceph-mon[30854]: from='osd.10 [v2:172.21.15.174:6824/1238812525,v1:172.21.15.174:6825/1238812525]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:42:53.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:53 smithi172 ceph-mon[30854]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-10T08:42:54.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.982+0000 7f0eb54a6640 1 -- 172.21.15.191:0/2855693247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0eb0101d10 msgr2=0x7f0eb0102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:54.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.982+0000 7f0eb54a6640 1 --2- 172.21.15.191:0/2855693247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0eb0101d10 0x7f0eb0102190 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009900 tx=0x7f0ea402a410 comp rx=0 tx=0).stop 2024-09-10T08:42:54.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.982+0000 7f0eb54a6640 1 -- 172.21.15.191:0/2855693247 shutdown_connections 2024-09-10T08:42:54.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.982+0000 7f0eb54a6640 1 --2- 172.21.15.191:0/2855693247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb01026d0 0x7f0eb0108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:54.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.982+0000 7f0eb54a6640 1 --2- 172.21.15.191:0/2855693247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0eb0101d10 0x7f0eb0102190 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:54.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.982+0000 7f0eb54a6640 1 --2- 172.21.15.191:0/2855693247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0eb0100b10 0x7f0eb0100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:54.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.982+0000 7f0eb54a6640 1 -- 172.21.15.191:0/2855693247 >> 172.21.15.191:0/2855693247 conn(0x7f0eb00fc2a0 msgr2=0x7f0eb00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:42:54.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.982+0000 7f0eb54a6640 1 -- 172.21.15.191:0/2855693247 shutdown_connections 2024-09-10T08:42:54.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.982+0000 7f0eb54a6640 1 -- 172.21.15.191:0/2855693247 wait complete. 2024-09-10T08:42:54.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eb54a6640 1 Processor -- start 2024-09-10T08:42:54.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eb54a6640 1 -- start start 2024-09-10T08:42:54.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eb54a6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0eb0100b10 0x7f0eb0196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:54.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eb54a6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb0101d10 0x7f0eb0197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:54.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eb54a6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0eb01026d0 0x7f0eb019e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:54.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eb54a6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb010c640 con 0x7f0eb0101d10 2024-09-10T08:42:54.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eb54a6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb010c4c0 con 0x7f0eb0100b10 2024-09-10T08:42:54.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eb54a6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb010c7c0 con 0x7f0eb01026d0 2024-09-10T08:42:54.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eaf7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0eb01026d0 0x7f0eb019e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:54.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eaf7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0eb01026d0 0x7f0eb019e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:38528/0 (socket says 172.21.15.191:38528) 2024-09-10T08:42:54.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eaf7fe640 1 -- 172.21.15.191:0/2077033981 learned_addr learned my addr 172.21.15.191:0/2077033981 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-10T08:42:54.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eaeffd640 1 --2- 172.21.15.191:0/2077033981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0eb0100b10 0x7f0eb0196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:54.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eae7fc640 1 --2- 172.21.15.191:0/2077033981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb0101d10 0x7f0eb0197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:54.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eaeffd640 1 -- 172.21.15.191:0/2077033981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0eb01026d0 msgr2=0x7f0eb019e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:54.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eaeffd640 1 --2- 172.21.15.191:0/2077033981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0eb01026d0 0x7f0eb019e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:54.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eaeffd640 1 -- 172.21.15.191:0/2077033981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb0101d10 msgr2=0x7f0eb0197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:42:54.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eaeffd640 1 --2- 172.21.15.191:0/2077033981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb0101d10 0x7f0eb0197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:42:54.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eaeffd640 1 -- 172.21.15.191:0/2077033981 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb019e940 con 0x7f0eb0100b10 2024-09-10T08:42:54.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.983+0000 7f0eaf7fe640 1 --2- 172.21.15.191:0/2077033981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0eb01026d0 0x7f0eb019e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:42:54.019 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.984+0000 7f0eaeffd640 1 --2- 172.21.15.191:0/2077033981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0eb0100b10 0x7f0eb0196d20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000ead0 tx=0x7f0ea000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:54.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.984+0000 7f0e97fff640 1 -- 172.21.15.191:0/2077033981 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea001c070 con 0x7f0eb0100b10 2024-09-10T08:42:54.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.984+0000 7f0eb54a6640 1 -- 172.21.15.191:0/2077033981 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb006a7f0 con 0x7f0eb0100b10 2024-09-10T08:42:54.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.984+0000 7f0e97fff640 1 -- 172.21.15.191:0/2077033981 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ea000bb70 con 0x7f0eb0100b10 2024-09-10T08:42:54.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.984+0000 7f0e97fff640 1 -- 172.21.15.191:0/2077033981 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0003d10 con 0x7f0eb0100b10 2024-09-10T08:42:54.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.984+0000 7f0eb54a6640 1 -- 172.21.15.191:0/2077033981 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb006ace0 con 0x7f0eb0100b10 2024-09-10T08:42:54.020 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.985+0000 7f0eb54a6640 1 -- 172.21.15.191:0/2077033981 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb0100f10 con 0x7f0eb0100b10 2024-09-10T08:42:54.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.986+0000 7f0e97fff640 1 -- 172.21.15.191:0/2077033981 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0ea001f070 con 0x7f0eb0100b10 2024-09-10T08:42:54.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.986+0000 7f0e97fff640 1 --2- 172.21.15.191:0/2077033981 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0e980a2b80 0x7f0e980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:42:54.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.987+0000 7f0e97fff640 1 -- 172.21.15.191:0/2077033981 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6447+0+0 (secure 0 0 0) 0x7f0ea00d00c0 con 0x7f0eb0100b10 2024-09-10T08:42:54.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.987+0000 7f0eae7fc640 1 --2- 172.21.15.191:0/2077033981 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0e980a2b80 0x7f0e980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:42:54.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.987+0000 7f0eae7fc640 1 --2- 172.21.15.191:0/2077033981 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0e980a2b80 0x7f0e980a5040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4000c00 tx=0x7f0ea40314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:42:54.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:53.992+0000 7f0e97fff640 1 -- 172.21.15.191:0/2077033981 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ea009d6d0 con 0x7f0eb0100b10 2024-09-10T08:42:54.162 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:54.162 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T08:42:54.163 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T08:42:54.163 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:54.163 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: Detected new or changed devices on smithi174 2024-09-10T08:42:54.163 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:54.163 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:54.163 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:54.163 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:54.163 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T08:42:54.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:42:54.169+0000 7f0eb54a6640 1 -- 172.21.15.191:0/2077033981 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0eb006afa0 con 0x7f0e980a2b80 2024-09-10T08:42:54.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:54.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T08:42:54.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T08:42:54.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:54.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: Detected new or changed devices on smithi174 2024-09-10T08:42:54.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:54.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:54.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:54.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:54.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T08:42:54.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:54.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:54.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:54.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:54.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-10T08:42:54.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:54.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:54.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:54.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:54 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-10T08:42:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:54.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:54 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:54.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:54.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T08:42:54.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T08:42:54.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:54.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: Detected new or changed devices on smithi174 2024-09-10T08:42:54.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:54.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:54.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:54.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:54.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T08:42:54.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:54.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:54.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:54.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:42:54.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-10T08:42:54.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:54.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:54.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:42:54.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:54 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:42:55.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:55 smithi174 ceph-mon[34068]: purged_snaps scrub starts 2024-09-10T08:42:55.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:55 smithi174 ceph-mon[34068]: purged_snaps scrub ok 2024-09-10T08:42:55.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:55 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:55.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:55 smithi174 ceph-mon[34068]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:42:55.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:55 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:42:55.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:55 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:42:55.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:55 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:55.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:55 smithi174 ceph-mon[34068]: osd.10 [v2:172.21.15.174:6824/1238812525,v1:172.21.15.174:6825/1238812525] boot 2024-09-10T08:42:55.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:55 smithi174 ceph-mon[34068]: osdmap e69: 11 total, 11 up, 11 in 2024-09-10T08:42:55.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:55 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:55.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:55 smithi191 ceph-mon[35314]: purged_snaps scrub starts 2024-09-10T08:42:55.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:55 smithi191 ceph-mon[35314]: purged_snaps scrub ok 2024-09-10T08:42:55.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:55 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:55.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:55 smithi191 ceph-mon[35314]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:42:55.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:55 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:42:55.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:55 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:42:55.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:55 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:55.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:55 smithi191 ceph-mon[35314]: osd.10 [v2:172.21.15.174:6824/1238812525,v1:172.21.15.174:6825/1238812525] boot 2024-09-10T08:42:55.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:55 smithi191 ceph-mon[35314]: osdmap e69: 11 total, 11 up, 11 in 2024-09-10T08:42:55.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:55 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:55.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:55 smithi172 ceph-mon[30854]: purged_snaps scrub starts 2024-09-10T08:42:55.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:55 smithi172 ceph-mon[30854]: purged_snaps scrub ok 2024-09-10T08:42:55.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:55 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:55.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:55 smithi172 ceph-mon[30854]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:42:55.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:55 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T08:42:55.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:55 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T08:42:55.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:55 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:42:55.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:55 smithi172 ceph-mon[30854]: osd.10 [v2:172.21.15.174:6824/1238812525,v1:172.21.15.174:6825/1238812525] boot 2024-09-10T08:42:55.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:55 smithi172 ceph-mon[30854]: osdmap e69: 11 total, 11 up, 11 in 2024-09-10T08:42:55.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:55 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T08:42:56.445 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:56 smithi174 ceph-mon[34068]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:56.445 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:56 smithi174 ceph-mon[34068]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T08:42:56.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:56 smithi191 ceph-mon[35314]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:56.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:56 smithi191 ceph-mon[35314]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T08:42:56.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:56 smithi172 ceph-mon[30854]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-10T08:42:56.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:56 smithi172 ceph-mon[30854]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T08:42:57.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:57 smithi174 ceph-mon[34068]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T08:42:57.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:57 smithi174 ceph-mon[34068]: from='client.? 172.21.15.191:0/2359987397' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae6535bd-1510-4bf3-bf4b-1a2a256add43"}]: dispatch 2024-09-10T08:42:57.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:57 smithi174 ceph-mon[34068]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae6535bd-1510-4bf3-bf4b-1a2a256add43"}]: dispatch 2024-09-10T08:42:57.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:57 smithi174 ceph-mon[34068]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae6535bd-1510-4bf3-bf4b-1a2a256add43"}]': finished 2024-09-10T08:42:57.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:57 smithi174 ceph-mon[34068]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T08:42:57.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:42:57.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:57 smithi174 ceph-mon[34068]: from='client.? 172.21.15.191:0/3838860041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:42:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:57 smithi191 ceph-mon[35314]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T08:42:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:57 smithi191 ceph-mon[35314]: from='client.? 172.21.15.191:0/2359987397' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae6535bd-1510-4bf3-bf4b-1a2a256add43"}]: dispatch 2024-09-10T08:42:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:57 smithi191 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae6535bd-1510-4bf3-bf4b-1a2a256add43"}]: dispatch 2024-09-10T08:42:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:57 smithi191 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae6535bd-1510-4bf3-bf4b-1a2a256add43"}]': finished 2024-09-10T08:42:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:57 smithi191 ceph-mon[35314]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T08:42:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:42:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:57 smithi191 ceph-mon[35314]: from='client.? 172.21.15.191:0/3838860041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:42:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:57 smithi172 ceph-mon[30854]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T08:42:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:57 smithi172 ceph-mon[30854]: from='client.? 172.21.15.191:0/2359987397' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae6535bd-1510-4bf3-bf4b-1a2a256add43"}]: dispatch 2024-09-10T08:42:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:57 smithi172 ceph-mon[30854]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae6535bd-1510-4bf3-bf4b-1a2a256add43"}]: dispatch 2024-09-10T08:42:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:57 smithi172 ceph-mon[30854]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae6535bd-1510-4bf3-bf4b-1a2a256add43"}]': finished 2024-09-10T08:42:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:57 smithi172 ceph-mon[30854]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T08:42:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:42:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:57 smithi172 ceph-mon[30854]: from='client.? 172.21.15.191:0/3838860041' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T08:42:58.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:58 smithi174 ceph-mon[34068]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:42:58.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:58 smithi174 ceph-mon[34068]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T08:42:58.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:42:58 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:42:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:58 smithi191 ceph-mon[35314]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:42:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:58 smithi191 ceph-mon[35314]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T08:42:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:42:58 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:42:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:58 smithi172 ceph-mon[30854]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:42:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:58 smithi172 ceph-mon[30854]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T08:42:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:42:58 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:43:00.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:00 smithi174 ceph-mon[34068]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T08:43:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:00 smithi191 ceph-mon[35314]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T08:43:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:00 smithi172 ceph-mon[30854]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T08:43:02.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:02 smithi174 ceph-mon[34068]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-10T08:43:02.531 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:02 smithi191 ceph-mon[35314]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-10T08:43:02.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:02 smithi172 ceph-mon[30854]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-10T08:43:04.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:04 smithi191 ceph-mon[35314]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-10T08:43:04.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:04 smithi174 ceph-mon[34068]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-10T08:43:04.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:04 smithi172 ceph-mon[30854]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-10T08:43:05.466 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:05 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T08:43:05.467 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:05 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:05.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T08:43:05.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:05.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:05 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T08:43:05.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:05 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:06.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:06 smithi174 ceph-mon[34068]: Deploying daemon osd.11 on smithi191 2024-09-10T08:43:06.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:06 smithi174 ceph-mon[34068]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T08:43:06.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:06 smithi191 ceph-mon[35314]: Deploying daemon osd.11 on smithi191 2024-09-10T08:43:06.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:06 smithi191 ceph-mon[35314]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T08:43:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:06 smithi172 ceph-mon[30854]: Deploying daemon osd.11 on smithi191 2024-09-10T08:43:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:06 smithi172 ceph-mon[30854]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T08:43:08.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:08 smithi174 ceph-mon[34068]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T08:43:08.501 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:08 smithi191 ceph-mon[35314]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T08:43:08.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:08 smithi172 ceph-mon[30854]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T08:43:10.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:10 smithi174 ceph-mon[34068]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T08:43:10.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:10 smithi191 ceph-mon[35314]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T08:43:10.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:10 smithi172 ceph-mon[30854]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T08:43:11.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:43:11.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:11.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:11.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:43:11.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:11.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:11.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:43:11.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:11.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:12.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:12 smithi174 ceph-mon[34068]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:43:12.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:12 smithi191 ceph-mon[35314]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:43:12.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:12 smithi172 ceph-mon[30854]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:43:14.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:14 smithi174 ceph-mon[34068]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:43:14.544 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:14 smithi191 ceph-mon[35314]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:43:14.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:14 smithi172 ceph-mon[30854]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:43:14.592 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:14.592+0000 7f0e97fff640 1 -- 172.21.15.191:0/2077033981 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0eb006afa0 con 0x7f0e980a2b80 2024-09-10T08:43:14.593 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 11 on host 'smithi191' 2024-09-10T08:43:14.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:14.595+0000 7f0eb54a6640 1 -- 172.21.15.191:0/2077033981 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0e980a2b80 msgr2=0x7f0e980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:14.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:14.595+0000 7f0eb54a6640 1 --2- 172.21.15.191:0/2077033981 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0e980a2b80 0x7f0e980a5040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4000c00 tx=0x7f0ea40314a0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:14.595+0000 7f0eb54a6640 1 -- 172.21.15.191:0/2077033981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0eb0100b10 msgr2=0x7f0eb0196d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:14.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:14.595+0000 7f0eb54a6640 1 --2- 172.21.15.191:0/2077033981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0eb0100b10 0x7f0eb0196d20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000ead0 tx=0x7f0ea000ef90 comp rx=0 tx=0).stop 2024-09-10T08:43:14.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:14.595+0000 7f0eb54a6640 1 -- 172.21.15.191:0/2077033981 shutdown_connections 2024-09-10T08:43:14.595 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:14.595+0000 7f0eb54a6640 1 --2- 172.21.15.191:0/2077033981 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0e980a2b80 0x7f0e980a5040 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:14.595+0000 7f0eb54a6640 1 --2- 172.21.15.191:0/2077033981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0eb01026d0 0x7f0eb019e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:14.595+0000 7f0eb54a6640 1 --2- 172.21.15.191:0/2077033981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eb0101d10 0x7f0eb0197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:14.595+0000 7f0eb54a6640 1 --2- 172.21.15.191:0/2077033981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0eb0100b10 0x7f0eb0196d20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:14.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:14.595+0000 7f0eb54a6640 1 -- 172.21.15.191:0/2077033981 >> 172.21.15.191:0/2077033981 conn(0x7f0eb00fc2a0 msgr2=0x7f0eb0105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:14.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:14.595+0000 7f0eb54a6640 1 -- 172.21.15.191:0/2077033981 shutdown_connections 2024-09-10T08:43:14.596 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:14.595+0000 7f0eb54a6640 1 -- 172.21.15.191:0/2077033981 wait complete. 2024-09-10T08:43:15.319 DEBUG:teuthology.orchestra.run.smithi191:osd.11> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.11.service 2024-09-10T08:43:15.321 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-10T08:43:15.321 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T08:43:15.581 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:15.702 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:15 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:15.702 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:15 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:15.702 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:15 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:15.702 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:15 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:15.702 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:15 smithi172 ceph-mon[30854]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:43:15.853 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:15 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:15.853 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:15 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:15.853 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:15 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:15.853 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:15 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:15.853 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:15 smithi191 ceph-mon[35314]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:43:15.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:15 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:15.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:15 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:15.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:15 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:15.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:15 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:15.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:15 smithi174 ceph-mon[34068]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:43:16.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.076+0000 7f539b2e4640 1 -- 172.21.15.172:0/200412800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5394102e00 msgr2=0x7f5394103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:16.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.076+0000 7f539b2e4640 1 --2- 172.21.15.172:0/200412800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5394102e00 0x7f5394103280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5384009900 tx=0x7f538402a410 comp rx=0 tx=0).stop 2024-09-10T08:43:16.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.076+0000 7f539b2e4640 1 -- 172.21.15.172:0/200412800 shutdown_connections 2024-09-10T08:43:16.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.076+0000 7f539b2e4640 1 --2- 172.21.15.172:0/200412800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53941037c0 0x7f539410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:16.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.076+0000 7f539b2e4640 1 --2- 172.21.15.172:0/200412800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5394102e00 0x7f5394103280 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:16.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.076+0000 7f539b2e4640 1 --2- 172.21.15.172:0/200412800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5394101c00 0x7f5394102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:16.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.076+0000 7f539b2e4640 1 -- 172.21.15.172:0/200412800 >> 172.21.15.172:0/200412800 conn(0x7f53940fd390 msgr2=0x7f53940ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:16.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.076+0000 7f539b2e4640 1 -- 172.21.15.172:0/200412800 shutdown_connections 2024-09-10T08:43:16.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.076+0000 7f539b2e4640 1 -- 172.21.15.172:0/200412800 wait complete. 2024-09-10T08:43:16.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.077+0000 7f539b2e4640 1 Processor -- start 2024-09-10T08:43:16.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.077+0000 7f539b2e4640 1 -- start start 2024-09-10T08:43:16.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.077+0000 7f539b2e4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5394101c00 0x7f5394197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:16.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.077+0000 7f539b2e4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5394102e00 0x7f5394198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:16.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.077+0000 7f539b2e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53941037c0 0x7f539419f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:16.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.077+0000 7f539b2e4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f539410d710 con 0x7f5394101c00 2024-09-10T08:43:16.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.077+0000 7f539b2e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f539410d590 con 0x7f53941037c0 2024-09-10T08:43:16.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.077+0000 7f539b2e4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f539410d890 con 0x7f5394102e00 2024-09-10T08:43:16.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.077+0000 7f5399080640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5394101c00 0x7f5394197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:16.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.078+0000 7f5399080640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5394101c00 0x7f5394197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34138/0 (socket says 172.21.15.172:34138) 2024-09-10T08:43:16.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.078+0000 7f5399080640 1 -- 172.21.15.172:0/1728290784 learned_addr learned my addr 172.21.15.172:0/1728290784 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:43:16.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.078+0000 7f539887f640 1 --2- 172.21.15.172:0/1728290784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5394102e00 0x7f5394198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:16.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.078+0000 7f5399881640 1 --2- 172.21.15.172:0/1728290784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53941037c0 0x7f539419f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:16.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.078+0000 7f539887f640 1 -- 172.21.15.172:0/1728290784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53941037c0 msgr2=0x7f539419f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:16.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.078+0000 7f539887f640 1 --2- 172.21.15.172:0/1728290784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53941037c0 0x7f539419f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:16.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.078+0000 7f539887f640 1 -- 172.21.15.172:0/1728290784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5394101c00 msgr2=0x7f5394197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:16.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.078+0000 7f539887f640 1 --2- 172.21.15.172:0/1728290784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5394101c00 0x7f5394197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:16.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.078+0000 7f539887f640 1 -- 172.21.15.172:0/1728290784 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539419fa60 con 0x7f5394102e00 2024-09-10T08:43:16.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.078+0000 7f5399080640 1 --2- 172.21.15.172:0/1728290784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5394101c00 0x7f5394197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:43:16.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.078+0000 7f539887f640 1 --2- 172.21.15.172:0/1728290784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5394102e00 0x7f5394198370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f53840314e0 tx=0x7f5384031a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:16.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.078+0000 7f538a7fc640 1 -- 172.21.15.172:0/1728290784 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538403f030 con 0x7f5394102e00 2024-09-10T08:43:16.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.078+0000 7f538a7fc640 1 -- 172.21.15.172:0/1728290784 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5384003aa0 con 0x7f5394102e00 2024-09-10T08:43:16.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.078+0000 7f539b2e4640 1 -- 172.21.15.172:0/1728290784 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539419fcf0 con 0x7f5394102e00 2024-09-10T08:43:16.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.078+0000 7f538a7fc640 1 -- 172.21.15.172:0/1728290784 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384035070 con 0x7f5394102e00 2024-09-10T08:43:16.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.078+0000 7f539b2e4640 1 -- 172.21.15.172:0/1728290784 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53941a01c0 con 0x7f5394102e00 2024-09-10T08:43:16.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.079+0000 7f539b2e4640 1 -- 172.21.15.172:0/1728290784 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5394109140 con 0x7f5394102e00 2024-09-10T08:43:16.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.080+0000 7f538a7fc640 1 -- 172.21.15.172:0/1728290784 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5384042030 con 0x7f5394102e00 2024-09-10T08:43:16.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.081+0000 7f538a7fc640 1 --2- 172.21.15.172:0/1728290784 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53800a2c30 0x7f53800a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:16.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.081+0000 7f5399080640 1 --2- 172.21.15.172:0/1728290784 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53800a2c30 0x7f53800a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:16.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.082+0000 7f538a7fc640 1 -- 172.21.15.172:0/1728290784 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6809+0+0 (secure 0 0 0) 0x7f53840ef0f0 con 0x7f5394102e00 2024-09-10T08:43:16.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.082+0000 7f5399080640 1 --2- 172.21.15.172:0/1728290784 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53800a2c30 0x7f53800a50f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f53900095f0 tx=0x7f5390005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:16.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.084+0000 7f538a7fc640 1 -- 172.21.15.172:0/1728290784 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53840bbf40 con 0x7f5394102e00 2024-09-10T08:43:16.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.258+0000 7f539b2e4640 1 -- 172.21.15.172:0/1728290784 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f539402d990 con 0x7f5394102e00 2024-09-10T08:43:16.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.258+0000 7f538a7fc640 1 -- 172.21.15.172:0/1728290784 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f53840c0240 con 0x7f5394102e00 2024-09-10T08:43:16.260 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:43:16.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.261+0000 7f539b2e4640 1 -- 172.21.15.172:0/1728290784 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53800a2c30 msgr2=0x7f53800a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:16.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.261+0000 7f539b2e4640 1 --2- 172.21.15.172:0/1728290784 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53800a2c30 0x7f53800a50f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f53900095f0 tx=0x7f5390005f90 comp rx=0 tx=0).stop 2024-09-10T08:43:16.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.261+0000 7f539b2e4640 1 -- 172.21.15.172:0/1728290784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5394102e00 msgr2=0x7f5394198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:16.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.261+0000 7f539b2e4640 1 --2- 172.21.15.172:0/1728290784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5394102e00 0x7f5394198370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f53840314e0 tx=0x7f5384031a30 comp rx=0 tx=0).stop 2024-09-10T08:43:16.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.261+0000 7f539b2e4640 1 -- 172.21.15.172:0/1728290784 shutdown_connections 2024-09-10T08:43:16.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.261+0000 7f539b2e4640 1 --2- 172.21.15.172:0/1728290784 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53800a2c30 0x7f53800a50f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:16.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.261+0000 7f539b2e4640 1 --2- 172.21.15.172:0/1728290784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53941037c0 0x7f539419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:16.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.261+0000 7f539b2e4640 1 --2- 172.21.15.172:0/1728290784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5394102e00 0x7f5394198370 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:16.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.261+0000 7f539b2e4640 1 --2- 172.21.15.172:0/1728290784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5394101c00 0x7f5394197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:16.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.261+0000 7f539b2e4640 1 -- 172.21.15.172:0/1728290784 >> 172.21.15.172:0/1728290784 conn(0x7f53940fd390 msgr2=0x7f5394106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:16.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.261+0000 7f539b2e4640 1 -- 172.21.15.172:0/1728290784 shutdown_connections 2024-09-10T08:43:16.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:16.262+0000 7f539b2e4640 1 -- 172.21.15.172:0/1728290784 wait complete. 2024-09-10T08:43:17.074 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:16 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1728290784' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:43:17.074 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:16 smithi172 ceph-mon[30854]: from='osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T08:43:17.102 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1725957774,"num_in_osds":12,"osd_in_since":1725957776,"num_remapped_pgs":0} 2024-09-10T08:43:17.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:16 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1728290784' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:43:17.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:16 smithi174 ceph-mon[34068]: from='osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T08:43:17.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:16 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1728290784' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:43:17.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:16 smithi191 ceph-mon[35314]: from='osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T08:43:17.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 08:43:16 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T08:43:16.814+0000 7f2496e57640 -1 osd.11 0 log_to_monitors true 2024-09-10T08:43:18.103 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T08:43:18.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:17 smithi174 ceph-mon[34068]: from='osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T08:43:18.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:17 smithi174 ceph-mon[34068]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T08:43:18.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:17 smithi174 ceph-mon[34068]: from='osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:43:18.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:43:18.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:17 smithi174 ceph-mon[34068]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:43:18.253 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:17 smithi191 ceph-mon[35314]: from='osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T08:43:18.253 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:17 smithi191 ceph-mon[35314]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T08:43:18.253 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:17 smithi191 ceph-mon[35314]: from='osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:43:18.253 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:43:18.253 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:17 smithi191 ceph-mon[35314]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:43:18.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:17 smithi172 ceph-mon[30854]: from='osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T08:43:18.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:17 smithi172 ceph-mon[30854]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T08:43:18.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:17 smithi172 ceph-mon[30854]: from='osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-10T08:43:18.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:43:18.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:17 smithi172 ceph-mon[30854]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:43:18.373 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:18.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.849+0000 7fdd85d6c640 1 -- 172.21.15.172:0/3320940484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd80101c00 msgr2=0x7fdd80102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:18.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.849+0000 7fdd85d6c640 1 --2- 172.21.15.172:0/3320940484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd80101c00 0x7fdd80102000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009ca0 tx=0x7fdd7002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:43:18.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.849+0000 7fdd85d6c640 1 -- 172.21.15.172:0/3320940484 shutdown_connections 2024-09-10T08:43:18.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.849+0000 7fdd85d6c640 1 --2- 172.21.15.172:0/3320940484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd801037c0 0x7fdd8010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:18.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.849+0000 7fdd85d6c640 1 --2- 172.21.15.172:0/3320940484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd80102e00 0x7fdd80103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:18.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.849+0000 7fdd85d6c640 1 --2- 172.21.15.172:0/3320940484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd80101c00 0x7fdd80102000 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:18.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.849+0000 7fdd85d6c640 1 -- 172.21.15.172:0/3320940484 >> 172.21.15.172:0/3320940484 conn(0x7fdd800fd390 msgr2=0x7fdd800ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:18.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.849+0000 7fdd85d6c640 1 -- 172.21.15.172:0/3320940484 shutdown_connections 2024-09-10T08:43:18.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.849+0000 7fdd85d6c640 1 -- 172.21.15.172:0/3320940484 wait complete. 2024-09-10T08:43:18.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.850+0000 7fdd85d6c640 1 Processor -- start 2024-09-10T08:43:18.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.850+0000 7fdd85d6c640 1 -- start start 2024-09-10T08:43:18.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.850+0000 7fdd85d6c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd80101c00 0x7fdd80075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:18.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.850+0000 7fdd85d6c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd80102e00 0x7fdd800745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:18.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.850+0000 7fdd85d6c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd801037c0 0x7fdd80074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:18.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.850+0000 7fdd85d6c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8010d7b0 con 0x7fdd801037c0 2024-09-10T08:43:18.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.850+0000 7fdd85d6c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8010d630 con 0x7fdd80102e00 2024-09-10T08:43:18.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.850+0000 7fdd85d6c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8010d930 con 0x7fdd80101c00 2024-09-10T08:43:18.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.850+0000 7fdd7ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd801037c0 0x7fdd80074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:18.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.850+0000 7fdd7ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd801037c0 0x7fdd80074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59880/0 (socket says 172.21.15.172:59880) 2024-09-10T08:43:18.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.850+0000 7fdd7ffff640 1 -- 172.21.15.172:0/2344388667 learned_addr learned my addr 172.21.15.172:0/2344388667 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:43:18.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.851+0000 7fdd7f7fe640 1 --2- 172.21.15.172:0/2344388667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd80101c00 0x7fdd80075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:18.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.851+0000 7fdd7ffff640 1 -- 172.21.15.172:0/2344388667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd80101c00 msgr2=0x7fdd80075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:18.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.851+0000 7fdd7ffff640 1 --2- 172.21.15.172:0/2344388667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd80101c00 0x7fdd80075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:18.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.851+0000 7fdd7ffff640 1 -- 172.21.15.172:0/2344388667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd80102e00 msgr2=0x7fdd800745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:18.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.851+0000 7fdd7effd640 1 --2- 172.21.15.172:0/2344388667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd80102e00 0x7fdd800745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:18.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.851+0000 7fdd7ffff640 1 --2- 172.21.15.172:0/2344388667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd80102e00 0x7fdd800745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:18.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.851+0000 7fdd7ffff640 1 -- 172.21.15.172:0/2344388667 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd800753e0 con 0x7fdd801037c0 2024-09-10T08:43:18.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.851+0000 7fdd7effd640 1 --2- 172.21.15.172:0/2344388667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd80102e00 0x7fdd800745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:43:18.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.851+0000 7fdd7ffff640 1 --2- 172.21.15.172:0/2344388667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd801037c0 0x7fdd80074b20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800c970 tx=0x7fdd7800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:18.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.851+0000 7fdd7cff9640 1 -- 172.21.15.172:0/2344388667 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78007bb0 con 0x7fdd801037c0 2024-09-10T08:43:18.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.851+0000 7fdd85d6c640 1 -- 172.21.15.172:0/2344388667 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd801a1fd0 con 0x7fdd801037c0 2024-09-10T08:43:18.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.851+0000 7fdd7cff9640 1 -- 172.21.15.172:0/2344388667 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd78007d50 con 0x7fdd801037c0 2024-09-10T08:43:18.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.851+0000 7fdd7cff9640 1 -- 172.21.15.172:0/2344388667 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78004e10 con 0x7fdd801037c0 2024-09-10T08:43:18.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.851+0000 7fdd85d6c640 1 -- 172.21.15.172:0/2344388667 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd801a2530 con 0x7fdd801037c0 2024-09-10T08:43:18.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.852+0000 7fdd85d6c640 1 -- 172.21.15.172:0/2344388667 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd801090a0 con 0x7fdd801037c0 2024-09-10T08:43:18.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.854+0000 7fdd7cff9640 1 -- 172.21.15.172:0/2344388667 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdd78005000 con 0x7fdd801037c0 2024-09-10T08:43:18.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.855+0000 7fdd7cff9640 1 --2- 172.21.15.172:0/2344388667 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdd680a2b80 0x7fdd680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:18.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.855+0000 7fdd7f7fe640 1 --2- 172.21.15.172:0/2344388667 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdd680a2b80 0x7fdd680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:18.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.855+0000 7fdd7cff9640 1 -- 172.21.15.172:0/2344388667 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6847+0+0 (secure 0 0 0) 0x7fdd780ceb40 con 0x7fdd801037c0 2024-09-10T08:43:18.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.855+0000 7fdd7f7fe640 1 --2- 172.21.15.172:0/2344388667 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdd680a2b80 0x7fdd680a5040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdd70000f80 tx=0x7fdd70031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:18.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:18.858+0000 7fdd7cff9640 1 -- 172.21.15.172:0/2344388667 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdd78014030 con 0x7fdd801037c0 2024-09-10T08:43:18.918 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T08:43:18.612+0000 7f2492dda640 -1 osd.11 0 waiting for initial osdmap 2024-09-10T08:43:18.918 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T08:43:18.625+0000 7f248abfa640 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T08:43:19.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:19.028+0000 7fdd85d6c640 1 -- 172.21.15.172:0/2344388667 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fdd80062df0 con 0x7fdd801037c0 2024-09-10T08:43:19.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:19.028+0000 7fdd7cff9640 1 -- 172.21.15.172:0/2344388667 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7fdd7809c060 con 0x7fdd801037c0 2024-09-10T08:43:19.029 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:43:19.029 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-10T08:43:19.030 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T08:43:19.030 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:43:19.030 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:43:19.030 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:19.030 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:19.030 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:43:19.030 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:43:19.030 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:43:19.030 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:43:19.030 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:19.030 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:19.030 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:43:19.031 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:19.031 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:43:19.031 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:19.031 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:43:19.031 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:19.031 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:18 smithi172 ceph-mon[30854]: from='osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997]' entity='osd.11' 2024-09-10T08:43:19.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:19.030+0000 7fdd85d6c640 1 -- 172.21.15.172:0/2344388667 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdd680a2b80 msgr2=0x7fdd680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:19.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:19.030+0000 7fdd85d6c640 1 --2- 172.21.15.172:0/2344388667 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdd680a2b80 0x7fdd680a5040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdd70000f80 tx=0x7fdd70031000 comp rx=0 tx=0).stop 2024-09-10T08:43:19.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:19.030+0000 7fdd85d6c640 1 -- 172.21.15.172:0/2344388667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd801037c0 msgr2=0x7fdd80074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:19.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:19.030+0000 7fdd85d6c640 1 --2- 172.21.15.172:0/2344388667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd801037c0 0x7fdd80074b20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800c970 tx=0x7fdd7800ce30 comp rx=0 tx=0).stop 2024-09-10T08:43:19.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:19.031+0000 7fdd85d6c640 1 -- 172.21.15.172:0/2344388667 shutdown_connections 2024-09-10T08:43:19.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:19.031+0000 7fdd85d6c640 1 --2- 172.21.15.172:0/2344388667 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdd680a2b80 0x7fdd680a5040 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:19.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:19.031+0000 7fdd85d6c640 1 --2- 172.21.15.172:0/2344388667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd801037c0 0x7fdd80074b20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:19.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:19.031+0000 7fdd85d6c640 1 --2- 172.21.15.172:0/2344388667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd80102e00 0x7fdd800745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:19.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:19.031+0000 7fdd85d6c640 1 --2- 172.21.15.172:0/2344388667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd80101c00 0x7fdd80075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:19.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:19.031+0000 7fdd85d6c640 1 -- 172.21.15.172:0/2344388667 >> 172.21.15.172:0/2344388667 conn(0x7fdd800fd390 msgr2=0x7fdd80106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:19.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:19.031+0000 7fdd85d6c640 1 -- 172.21.15.172:0/2344388667 shutdown_connections 2024-09-10T08:43:19.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:19.031+0000 7fdd85d6c640 1 -- 172.21.15.172:0/2344388667 wait complete. 2024-09-10T08:43:19.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-10T08:43:19.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T08:43:19.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:43:19.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:43:19.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:19.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:19.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:43:19.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:43:19.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:43:19.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:43:19.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:19.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:19.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:43:19.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:19.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:43:19.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:19.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:43:19.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:19.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:18 smithi174 ceph-mon[34068]: from='osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997]' entity='osd.11' 2024-09-10T08:43:19.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-10T08:43:19.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T08:43:19.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:43:19.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:43:19.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:19.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:19.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:43:19.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:43:19.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:43:19.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:43:19.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:19.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:19.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:43:19.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:19.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:43:19.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:19.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:43:19.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:19.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:18 smithi191 ceph-mon[35314]: from='osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997]' entity='osd.11' 2024-09-10T08:43:19.646 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1725957774,"num_in_osds":12,"osd_in_since":1725957776,"num_remapped_pgs":0} 2024-09-10T08:43:20.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:19 smithi174 ceph-mon[34068]: purged_snaps scrub starts 2024-09-10T08:43:20.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:19 smithi174 ceph-mon[34068]: purged_snaps scrub ok 2024-09-10T08:43:20.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:19 smithi174 ceph-mon[34068]: Detected new or changed devices on smithi191 2024-09-10T08:43:20.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:19 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi191 to 4272M 2024-09-10T08:43:20.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:43:20.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:19 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2344388667' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:43:20.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:19 smithi174 ceph-mon[34068]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:43:20.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:19 smithi174 ceph-mon[34068]: osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997] boot 2024-09-10T08:43:20.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:19 smithi174 ceph-mon[34068]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T08:43:20.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:43:20.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:19 smithi191 ceph-mon[35314]: purged_snaps scrub starts 2024-09-10T08:43:20.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:19 smithi191 ceph-mon[35314]: purged_snaps scrub ok 2024-09-10T08:43:20.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:19 smithi191 ceph-mon[35314]: Detected new or changed devices on smithi191 2024-09-10T08:43:20.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:19 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi191 to 4272M 2024-09-10T08:43:20.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:43:20.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:19 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2344388667' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:43:20.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:19 smithi191 ceph-mon[35314]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:43:20.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:19 smithi191 ceph-mon[35314]: osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997] boot 2024-09-10T08:43:20.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:19 smithi191 ceph-mon[35314]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T08:43:20.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:43:20.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:19 smithi172 ceph-mon[30854]: purged_snaps scrub starts 2024-09-10T08:43:20.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:19 smithi172 ceph-mon[30854]: purged_snaps scrub ok 2024-09-10T08:43:20.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:19 smithi172 ceph-mon[30854]: Detected new or changed devices on smithi191 2024-09-10T08:43:20.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:19 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi191 to 4272M 2024-09-10T08:43:20.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:43:20.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:19 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2344388667' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:43:20.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:19 smithi172 ceph-mon[30854]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-09-10T08:43:20.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:19 smithi172 ceph-mon[30854]: osd.11 [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997] boot 2024-09-10T08:43:20.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:19 smithi172 ceph-mon[30854]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T08:43:20.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T08:43:20.647 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T08:43:20.904 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:21.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.391+0000 7fc8cf86f640 1 -- 172.21.15.172:0/3740451490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c81026d0 msgr2=0x7fc8c8108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:21.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.391+0000 7fc8cf86f640 1 --2- 172.21.15.172:0/3740451490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c81026d0 0x7fc8c8108f60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009ca0 tx=0x7fc8c002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:43:21.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.391+0000 7fc8cf86f640 1 -- 172.21.15.172:0/3740451490 shutdown_connections 2024-09-10T08:43:21.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.391+0000 7fc8cf86f640 1 --2- 172.21.15.172:0/3740451490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c81026d0 0x7fc8c8108f60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:21.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.391+0000 7fc8cf86f640 1 --2- 172.21.15.172:0/3740451490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c8101d10 0x7fc8c8102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:21.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.391+0000 7fc8cf86f640 1 --2- 172.21.15.172:0/3740451490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8c8100b10 0x7fc8c8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:21.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.391+0000 7fc8cf86f640 1 -- 172.21.15.172:0/3740451490 >> 172.21.15.172:0/3740451490 conn(0x7fc8c80fc2a0 msgr2=0x7fc8c80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:21.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.391+0000 7fc8cf86f640 1 -- 172.21.15.172:0/3740451490 shutdown_connections 2024-09-10T08:43:21.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.391+0000 7fc8cf86f640 1 -- 172.21.15.172:0/3740451490 wait complete. 2024-09-10T08:43:21.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cf86f640 1 Processor -- start 2024-09-10T08:43:21.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cf86f640 1 -- start start 2024-09-10T08:43:21.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cf86f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8c8100b10 0x7fc8c806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:21.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cf86f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c8101d10 0x7fc8c819cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:21.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cf86f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c81026d0 0x7fc8c819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:21.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cf86f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c810c640 con 0x7fc8c81026d0 2024-09-10T08:43:21.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cf86f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c810c4c0 con 0x7fc8c8101d10 2024-09-10T08:43:21.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cf86f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c810c7c0 con 0x7fc8c8100b10 2024-09-10T08:43:21.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cde0c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c81026d0 0x7fc8c819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:21.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cde0c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c81026d0 0x7fc8c819f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59894/0 (socket says 172.21.15.172:59894) 2024-09-10T08:43:21.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cde0c640 1 -- 172.21.15.172:0/4224325512 learned_addr learned my addr 172.21.15.172:0/4224325512 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:43:21.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cd60b640 1 --2- 172.21.15.172:0/4224325512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8c8100b10 0x7fc8c806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:21.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cce0a640 1 --2- 172.21.15.172:0/4224325512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c8101d10 0x7fc8c819cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:21.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cde0c640 1 -- 172.21.15.172:0/4224325512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8c8100b10 msgr2=0x7fc8c806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:21.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cde0c640 1 --2- 172.21.15.172:0/4224325512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8c8100b10 0x7fc8c806b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:21.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cde0c640 1 -- 172.21.15.172:0/4224325512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c8101d10 msgr2=0x7fc8c819cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:21.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cde0c640 1 --2- 172.21.15.172:0/4224325512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c8101d10 0x7fc8c819cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:21.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cde0c640 1 -- 172.21.15.172:0/4224325512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c819fa70 con 0x7fc8c81026d0 2024-09-10T08:43:21.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cce0a640 1 --2- 172.21.15.172:0/4224325512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c8101d10 0x7fc8c819cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:43:21.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.392+0000 7fc8cde0c640 1 --2- 172.21.15.172:0/4224325512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c81026d0 0x7fc8c819f160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009dc0 tx=0x7fc8c0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:21.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.393+0000 7fc8be7fc640 1 -- 172.21.15.172:0/4224325512 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0032ec0 con 0x7fc8c81026d0 2024-09-10T08:43:21.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.393+0000 7fc8be7fc640 1 -- 172.21.15.172:0/4224325512 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8c00044f0 con 0x7fc8c81026d0 2024-09-10T08:43:21.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.393+0000 7fc8be7fc640 1 -- 172.21.15.172:0/4224325512 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0030d40 con 0x7fc8c81026d0 2024-09-10T08:43:21.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.393+0000 7fc8cf86f640 1 -- 172.21.15.172:0/4224325512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c819fca0 con 0x7fc8c81026d0 2024-09-10T08:43:21.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.393+0000 7fc8cf86f640 1 -- 172.21.15.172:0/4224325512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c81a0190 con 0x7fc8c81026d0 2024-09-10T08:43:21.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.394+0000 7fc8cf86f640 1 -- 172.21.15.172:0/4224325512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c8107f10 con 0x7fc8c81026d0 2024-09-10T08:43:21.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.395+0000 7fc8be7fc640 1 -- 172.21.15.172:0/4224325512 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc8c0004040 con 0x7fc8c81026d0 2024-09-10T08:43:21.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.396+0000 7fc8be7fc640 1 --2- 172.21.15.172:0/4224325512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc8b40a2d20 0x7fc8b40a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:21.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.396+0000 7fc8cd60b640 1 --2- 172.21.15.172:0/4224325512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc8b40a2d20 0x7fc8b40a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:21.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.396+0000 7fc8be7fc640 1 -- 172.21.15.172:0/4224325512 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fc8c00f0470 con 0x7fc8c81026d0 2024-09-10T08:43:21.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.396+0000 7fc8cd60b640 1 --2- 172.21.15.172:0/4224325512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc8b40a2d20 0x7fc8b40a51e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80058c0 tx=0x7fc8b800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:21.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.399+0000 7fc8be7fc640 1 -- 172.21.15.172:0/4224325512 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8c00bd7d0 con 0x7fc8c81026d0 2024-09-10T08:43:21.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.565+0000 7fc8cf86f640 1 -- 172.21.15.172:0/4224325512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc8c802da10 con 0x7fc8c81026d0 2024-09-10T08:43:21.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.565+0000 7fc8be7fc640 1 -- 172.21.15.172:0/4224325512 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) ==== 74+0+133 (secure 0 0 0) 0x7fc8c00bd0d0 con 0x7fc8c81026d0 2024-09-10T08:43:21.566 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:43:21.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.567+0000 7fc8cf86f640 1 -- 172.21.15.172:0/4224325512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc8b40a2d20 msgr2=0x7fc8b40a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:21.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.567+0000 7fc8cf86f640 1 --2- 172.21.15.172:0/4224325512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc8b40a2d20 0x7fc8b40a51e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80058c0 tx=0x7fc8b800a4c0 comp rx=0 tx=0).stop 2024-09-10T08:43:21.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.567+0000 7fc8cf86f640 1 -- 172.21.15.172:0/4224325512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c81026d0 msgr2=0x7fc8c819f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:21.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.567+0000 7fc8cf86f640 1 --2- 172.21.15.172:0/4224325512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c81026d0 0x7fc8c819f160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009dc0 tx=0x7fc8c0003ba0 comp rx=0 tx=0).stop 2024-09-10T08:43:21.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.567+0000 7fc8cf86f640 1 -- 172.21.15.172:0/4224325512 shutdown_connections 2024-09-10T08:43:21.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.567+0000 7fc8cf86f640 1 --2- 172.21.15.172:0/4224325512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc8b40a2d20 0x7fc8b40a51e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:21.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.567+0000 7fc8cf86f640 1 --2- 172.21.15.172:0/4224325512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8c81026d0 0x7fc8c819f160 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:21.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.567+0000 7fc8cf86f640 1 --2- 172.21.15.172:0/4224325512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8c8101d10 0x7fc8c819cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:21.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.567+0000 7fc8cf86f640 1 --2- 172.21.15.172:0/4224325512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8c8100b10 0x7fc8c806b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:21.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.567+0000 7fc8cf86f640 1 -- 172.21.15.172:0/4224325512 >> 172.21.15.172:0/4224325512 conn(0x7fc8c80fc2a0 msgr2=0x7fc8c81050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:21.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.567+0000 7fc8cf86f640 1 -- 172.21.15.172:0/4224325512 shutdown_connections 2024-09-10T08:43:21.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:21.567+0000 7fc8cf86f640 1 -- 172.21.15.172:0/4224325512 wait complete. 2024-09-10T08:43:21.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:21 smithi172 ceph-mon[30854]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T08:43:21.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:21 smithi172 ceph-mon[30854]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:21.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:21 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/4224325512' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:43:21.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:21 smithi174 ceph-mon[34068]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T08:43:21.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:21 smithi174 ceph-mon[34068]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:21.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:21 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/4224325512' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:43:22.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:21 smithi191 ceph-mon[35314]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T08:43:22.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:21 smithi191 ceph-mon[35314]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:22.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:21 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/4224325512' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T08:43:22.371 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1725957799,"num_in_osds":12,"osd_in_since":1725957776,"num_remapped_pgs":0} 2024-09-10T08:43:22.371 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:43:22.614 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:23.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.127+0000 7f69b9fa4640 1 -- 172.21.15.172:0/1128717467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69b410c170 msgr2=0x7f69b410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:23.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.127+0000 7f69b9fa4640 1 --2- 172.21.15.172:0/1128717467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69b410c170 0x7f69b410e610 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009ca0 tx=0x7f69ac02a950 comp rx=0 tx=0).stop 2024-09-10T08:43:23.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.127+0000 7f69b9fa4640 1 -- 172.21.15.172:0/1128717467 shutdown_connections 2024-09-10T08:43:23.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.127+0000 7f69b9fa4640 1 --2- 172.21.15.172:0/1128717467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69b410c170 0x7f69b410e610 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:23.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.127+0000 7f69b9fa4640 1 --2- 172.21.15.172:0/1128717467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69b4103660 0x7f69b410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:23.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.127+0000 7f69b9fa4640 1 --2- 172.21.15.172:0/1128717467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69b4102d40 0x7f69b4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:23.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.127+0000 7f69b9fa4640 1 -- 172.21.15.172:0/1128717467 >> 172.21.15.172:0/1128717467 conn(0x7f69b40fc7a0 msgr2=0x7f69b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:23.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.127+0000 7f69b9fa4640 1 -- 172.21.15.172:0/1128717467 shutdown_connections 2024-09-10T08:43:23.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.127+0000 7f69b9fa4640 1 -- 172.21.15.172:0/1128717467 wait complete. 2024-09-10T08:43:23.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.128+0000 7f69b9fa4640 1 Processor -- start 2024-09-10T08:43:23.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.128+0000 7f69b9fa4640 1 -- start start 2024-09-10T08:43:23.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.128+0000 7f69b9fa4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69b4102d40 0x7f69b41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:23.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.128+0000 7f69b9fa4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69b4103660 0x7f69b4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:23.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.128+0000 7f69b9fa4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69b410c170 0x7f69b4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:23.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.128+0000 7f69b9fa4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f69b4111e20 con 0x7f69b4102d40 2024-09-10T08:43:23.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.128+0000 7f69b9fa4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f69b4111ca0 con 0x7f69b410c170 2024-09-10T08:43:23.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.128+0000 7f69b9fa4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f69b4111fa0 con 0x7f69b4103660 2024-09-10T08:43:23.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.128+0000 7f69b37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69b4102d40 0x7f69b41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:23.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.128+0000 7f69b37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69b4102d40 0x7f69b41009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59900/0 (socket says 172.21.15.172:59900) 2024-09-10T08:43:23.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.128+0000 7f69b37fe640 1 -- 172.21.15.172:0/3883694423 learned_addr learned my addr 172.21.15.172:0/3883694423 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:43:23.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.129+0000 7f69b3fff640 1 --2- 172.21.15.172:0/3883694423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69b410c170 0x7f69b4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:23.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.129+0000 7f69b2ffd640 1 --2- 172.21.15.172:0/3883694423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69b4103660 0x7f69b4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:23.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.129+0000 7f69b37fe640 1 -- 172.21.15.172:0/3883694423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69b4103660 msgr2=0x7f69b4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:23.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.129+0000 7f69b37fe640 1 --2- 172.21.15.172:0/3883694423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69b4103660 0x7f69b4100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:23.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.129+0000 7f69b37fe640 1 -- 172.21.15.172:0/3883694423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69b410c170 msgr2=0x7f69b4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:23.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.129+0000 7f69b37fe640 1 --2- 172.21.15.172:0/3883694423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69b410c170 0x7f69b4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:23.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.129+0000 7f69b37fe640 1 -- 172.21.15.172:0/3883694423 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b41a6800 con 0x7f69b4102d40 2024-09-10T08:43:23.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.129+0000 7f69b37fe640 1 --2- 172.21.15.172:0/3883694423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69b4102d40 0x7f69b41009a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f69a400d770 tx=0x7f69a400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:23.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.129+0000 7f69b0ff9640 1 -- 172.21.15.172:0/3883694423 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4017070 con 0x7f69b4102d40 2024-09-10T08:43:23.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.129+0000 7f69b0ff9640 1 -- 172.21.15.172:0/3883694423 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69a40054d0 con 0x7f69b4102d40 2024-09-10T08:43:23.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.129+0000 7f69b9fa4640 1 -- 172.21.15.172:0/3883694423 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b41a6af0 con 0x7f69b4102d40 2024-09-10T08:43:23.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.129+0000 7f69b0ff9640 1 -- 172.21.15.172:0/3883694423 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4004cf0 con 0x7f69b4102d40 2024-09-10T08:43:23.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.129+0000 7f69b9fa4640 1 -- 172.21.15.172:0/3883694423 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b41a6fe0 con 0x7f69b4102d40 2024-09-10T08:43:23.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.130+0000 7f69b9fa4640 1 -- 172.21.15.172:0/3883694423 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69b4103120 con 0x7f69b4102d40 2024-09-10T08:43:23.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.131+0000 7f69b0ff9640 1 -- 172.21.15.172:0/3883694423 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f69a4004e90 con 0x7f69b4102d40 2024-09-10T08:43:23.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.132+0000 7f69b0ff9640 1 --2- 172.21.15.172:0/3883694423 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f699c0a2b80 0x7f699c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:23.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.132+0000 7f69b2ffd640 1 --2- 172.21.15.172:0/3883694423 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f699c0a2b80 0x7f699c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:23.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.132+0000 7f69b0ff9640 1 -- 172.21.15.172:0/3883694423 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f69a40ce6c0 con 0x7f69b4102d40 2024-09-10T08:43:23.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.132+0000 7f69b2ffd640 1 --2- 172.21.15.172:0/3883694423 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f699c0a2b80 0x7f699c0a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f69a8005d90 tx=0x7f69a8005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:23.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.136+0000 7f69b0ff9640 1 -- 172.21.15.172:0/3883694423 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f69a409bac0 con 0x7f69b4102d40 2024-09-10T08:43:23.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.306+0000 7f69b9fa4640 1 -- 172.21.15.172:0/3883694423 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69b4107240 con 0x7f69b4102d40 2024-09-10T08:43:23.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.307+0000 7f69b0ff9640 1 -- 172.21.15.172:0/3883694423 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+18550 (secure 0 0 0) 0x7f69a409b3c0 con 0x7f69b4102d40 2024-09-10T08:43:23.308 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:43:23.309 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":77,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","created":"2024-09-10T08:36:33.083529+0000","modified":"2024-09-10T08:43:20.617292+0000","last_up_change":"2024-09-10T08:43:19.615640+0000","last_in_change":"2024-09-10T08:42:56.656115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:39:31.328306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6803","nonce":2678426952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6805","nonce":2678426952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6809","nonce":2678426952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6807","nonce":2678426952}]},"public_addr":"172.21.15.172:6803/2678426952","cluster_addr":"172.21.15.172:6805/2678426952","heartbeat_back_addr":"172.21.15.172:6809/2678426952","heartbeat_front_addr":"172.21.15.172:6807/2678426952","state":["exists","up"]},{"osd":1,"uuid":"55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6801","nonce":3279661439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6803","nonce":3279661439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6807","nonce":3279661439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6805","nonce":3279661439}]},"public_addr":"172.21.15.174:6801/3279661439","cluster_addr":"172.21.15.174:6803/3279661439","heartbeat_back_addr":"172.21.15.174:6807/3279661439","heartbeat_front_addr":"172.21.15.174:6805/3279661439","state":["exists","up"]},{"osd":2,"uuid":"f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6801","nonce":1963796290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6803","nonce":1963796290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6807","nonce":1963796290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6805","nonce":1963796290}]},"public_addr":"172.21.15.191:6801/1963796290","cluster_addr":"172.21.15.191:6803/1963796290","heartbeat_back_addr":"172.21.15.191:6807/1963796290","heartbeat_front_addr":"172.21.15.191:6805/1963796290","state":["exists","up"]},{"osd":3,"uuid":"15ecca3f-bc6b-4a06-b283-b515384beb19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6811","nonce":869546480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6813","nonce":869546480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6817","nonce":869546480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6815","nonce":869546480}]},"public_addr":"172.21.15.172:6811/869546480","cluster_addr":"172.21.15.172:6813/869546480","heartbeat_back_addr":"172.21.15.172:6817/869546480","heartbeat_front_addr":"172.21.15.172:6815/869546480","state":["exists","up"]},{"osd":4,"uuid":"8c2c1d40-72ad-4a15-ab72-54871c7e2427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6809","nonce":961007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6811","nonce":961007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6815","nonce":961007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6813","nonce":961007949}]},"public_addr":"172.21.15.174:6809/961007949","cluster_addr":"172.21.15.174:6811/961007949","heartbeat_back_addr":"172.21.15.174:6815/961007949","heartbeat_front_addr":"172.21.15.174:6813/961007949","state":["exists","up"]},{"osd":5,"uuid":"4de45a1d-ef0e-4bfe-ba15-5eeeb8806155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6809","nonce":4216938371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6811","nonce":4216938371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6815","nonce":4216938371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6813","nonce":4216938371}]},"public_addr":"172.21.15.191:6809/4216938371","cluster_addr":"172.21.15.191:6811/4216938371","heartbeat_back_addr":"172.21.15.191:6815/4216938371","heartbeat_front_addr":"172.21.15.191:6813/4216938371","state":["exists","up"]},{"osd":6,"uuid":"60a9938a-f696-4db2-932c-402997543b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6819","nonce":3210762009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6821","nonce":3210762009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6825","nonce":3210762009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6823","nonce":3210762009}]},"public_addr":"172.21.15.172:6819/3210762009","cluster_addr":"172.21.15.172:6821/3210762009","heartbeat_back_addr":"172.21.15.172:6825/3210762009","heartbeat_front_addr":"172.21.15.172:6823/3210762009","state":["exists","up"]},{"osd":7,"uuid":"7f0202f3-971e-472f-9c4d-ae3c24d39f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6817","nonce":1461923581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6819","nonce":1461923581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6823","nonce":1461923581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6821","nonce":1461923581}]},"public_addr":"172.21.15.174:6817/1461923581","cluster_addr":"172.21.15.174:6819/1461923581","heartbeat_back_addr":"172.21.15.174:6823/1461923581","heartbeat_front_addr":"172.21.15.174:6821/1461923581","state":["exists","up"]},{"osd":8,"uuid":"b4e7327f-ca9f-4833-9cd3-b8529b3d9761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6817","nonce":3675210430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6819","nonce":3675210430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6823","nonce":3675210430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6821","nonce":3675210430}]},"public_addr":"172.21.15.191:6817/3675210430","cluster_addr":"172.21.15.191:6819/3675210430","heartbeat_back_addr":"172.21.15.191:6823/3675210430","heartbeat_front_addr":"172.21.15.191:6821/3675210430","state":["exists","up"]},{"osd":9,"uuid":"9f6153f2-f9db-4c00-a659-c932628016f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6827","nonce":1181268619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6829","nonce":1181268619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6833","nonce":1181268619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6831","nonce":1181268619}]},"public_addr":"172.21.15.172:6827/1181268619","cluster_addr":"172.21.15.172:6829/1181268619","heartbeat_back_addr":"172.21.15.172:6833/1181268619","heartbeat_front_addr":"172.21.15.172:6831/1181268619","state":["exists","up"]},{"osd":10,"uuid":"040a013a-4c3e-442f-bea6-12cbe517716d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6825","nonce":1238812525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6827","nonce":1238812525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6831","nonce":1238812525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6829","nonce":1238812525}]},"public_addr":"172.21.15.174:6825/1238812525","cluster_addr":"172.21.15.174:6827/1238812525","heartbeat_back_addr":"172.21.15.174:6831/1238812525","heartbeat_front_addr":"172.21.15.174:6829/1238812525","state":["exists","up"]},{"osd":11,"uuid":"ae6535bd-1510-4bf3-bf4b-1a2a256add43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6825","nonce":2162719997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6827","nonce":2162719997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6831","nonce":2162719997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6829","nonce":2162719997}]},"public_addr":"172.21.15.191:6825/2162719997","cluster_addr":"172.21.15.191:6827/2162719997","heartbeat_back_addr":"172.21.15.191:6831/2162719997","heartbeat_front_addr":"172.21.15.191:6829/2162719997","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:38:38.812596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:04.765652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:29.103966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:56.085849+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:21.641784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:47.750376+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:11.640697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:36.638840+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:02.167446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:28.655804+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:52.749676+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:43:17.783908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:0/977245917":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/3919952878":"2024-09-11T08:36:54.887256+0000","172.21.15.172:6800/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/1525385361":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/2043423408":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/839344299":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6800/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/3251817286":"2024-09-11T08:36:54.887256+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:43:23.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.310+0000 7f69b9fa4640 1 -- 172.21.15.172:0/3883694423 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f699c0a2b80 msgr2=0x7f699c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:23.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.310+0000 7f69b9fa4640 1 --2- 172.21.15.172:0/3883694423 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f699c0a2b80 0x7f699c0a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f69a8005d90 tx=0x7f69a8005d20 comp rx=0 tx=0).stop 2024-09-10T08:43:23.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.310+0000 7f69b9fa4640 1 -- 172.21.15.172:0/3883694423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69b4102d40 msgr2=0x7f69b41009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:23.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.310+0000 7f69b9fa4640 1 --2- 172.21.15.172:0/3883694423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69b4102d40 0x7f69b41009a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f69a400d770 tx=0x7f69a400dc30 comp rx=0 tx=0).stop 2024-09-10T08:43:23.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.310+0000 7f69b9fa4640 1 -- 172.21.15.172:0/3883694423 shutdown_connections 2024-09-10T08:43:23.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.310+0000 7f69b9fa4640 1 --2- 172.21.15.172:0/3883694423 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f699c0a2b80 0x7f699c0a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:23.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.310+0000 7f69b9fa4640 1 --2- 172.21.15.172:0/3883694423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69b410c170 0x7f69b4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:23.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.310+0000 7f69b9fa4640 1 --2- 172.21.15.172:0/3883694423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69b4103660 0x7f69b4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:23.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.310+0000 7f69b9fa4640 1 --2- 172.21.15.172:0/3883694423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69b4102d40 0x7f69b41009a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:23.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.310+0000 7f69b9fa4640 1 -- 172.21.15.172:0/3883694423 >> 172.21.15.172:0/3883694423 conn(0x7f69b40fc7a0 msgr2=0x7f69b40fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:23.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.310+0000 7f69b9fa4640 1 -- 172.21.15.172:0/3883694423 shutdown_connections 2024-09-10T08:43:23.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:23.310+0000 7f69b9fa4640 1 -- 172.21.15.172:0/3883694423 wait complete. 2024-09-10T08:43:24.126 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T08:39:31.328306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T08:43:24.127 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T08:43:24.370 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:24.491 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:24 smithi172 ceph-mon[30854]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:24.491 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:24 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3883694423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:43:24.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:24 smithi174 ceph-mon[34068]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:24.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:24 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3883694423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:43:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:24 smithi191 ceph-mon[35314]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:24 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3883694423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:43:24.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.873+0000 7fcd0af96640 1 -- 172.21.15.172:0/38794273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd04100a80 msgr2=0x7fcd04100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:24.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.873+0000 7fcd0af96640 1 --2- 172.21.15.172:0/38794273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd04100a80 0x7fcd04100ee0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009930 tx=0x7fccf402a420 comp rx=0 tx=0).stop 2024-09-10T08:43:24.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.873+0000 7fcd0af96640 1 -- 172.21.15.172:0/38794273 shutdown_connections 2024-09-10T08:43:24.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.873+0000 7fcd0af96640 1 --2- 172.21.15.172:0/38794273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd04101420 0x7fcd0410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.873+0000 7fcd0af96640 1 --2- 172.21.15.172:0/38794273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd04100a80 0x7fcd04100ee0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.873+0000 7fcd0af96640 1 --2- 172.21.15.172:0/38794273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04106a80 0x7fcd04106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.873+0000 7fcd0af96640 1 -- 172.21.15.172:0/38794273 >> 172.21.15.172:0/38794273 conn(0x7fcd040fc7a0 msgr2=0x7fcd040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:24.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.873+0000 7fcd0af96640 1 -- 172.21.15.172:0/38794273 shutdown_connections 2024-09-10T08:43:24.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.873+0000 7fcd0af96640 1 -- 172.21.15.172:0/38794273 wait complete. 2024-09-10T08:43:24.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fcd0af96640 1 Processor -- start 2024-09-10T08:43:24.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fcd0af96640 1 -- start start 2024-09-10T08:43:24.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fcd0af96640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd04100a80 0x7fcd041973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fcd0af96640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd04101420 0x7fcd04197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fcd0af96640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04106a80 0x7fcd0419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fcd0af96640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcd04110dd0 con 0x7fcd04101420 2024-09-10T08:43:24.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fcd0af96640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd04110c50 con 0x7fcd04100a80 2024-09-10T08:43:24.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fcd0af96640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcd04110f50 con 0x7fcd04106a80 2024-09-10T08:43:24.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fccfbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd04101420 0x7fcd04197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fccfbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd04101420 0x7fcd04197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59922/0 (socket says 172.21.15.172:59922) 2024-09-10T08:43:24.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fccfbfff640 1 -- 172.21.15.172:0/1274389507 learned_addr learned my addr 172.21.15.172:0/1274389507 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:43:24.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fcd08d32640 1 --2- 172.21.15.172:0/1274389507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd04100a80 0x7fcd041973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fccfbfff640 1 -- 172.21.15.172:0/1274389507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04106a80 msgr2=0x7fcd0419bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:24.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fcd09533640 1 --2- 172.21.15.172:0/1274389507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04106a80 0x7fcd0419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fccfbfff640 1 --2- 172.21.15.172:0/1274389507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04106a80 0x7fcd0419bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fccfbfff640 1 -- 172.21.15.172:0/1274389507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd04100a80 msgr2=0x7fcd041973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:24.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fccfbfff640 1 --2- 172.21.15.172:0/1274389507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd04100a80 0x7fcd041973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:24.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fccfbfff640 1 -- 172.21.15.172:0/1274389507 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0419c3a0 con 0x7fcd04101420 2024-09-10T08:43:24.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fcd08d32640 1 --2- 172.21.15.172:0/1274389507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd04100a80 0x7fcd041973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:43:24.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.874+0000 7fcd09533640 1 --2- 172.21.15.172:0/1274389507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04106a80 0x7fcd0419bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:43:24.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.875+0000 7fccfbfff640 1 --2- 172.21.15.172:0/1274389507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd04101420 0x7fcd04197930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fccf4005940 tx=0x7fccf40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:24.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.875+0000 7fccf9ffb640 1 -- 172.21.15.172:0/1274389507 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf403f030 con 0x7fcd04101420 2024-09-10T08:43:24.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.875+0000 7fccf9ffb640 1 -- 172.21.15.172:0/1274389507 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fccf4003cb0 con 0x7fcd04101420 2024-09-10T08:43:24.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.875+0000 7fcd0af96640 1 -- 172.21.15.172:0/1274389507 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd0419c630 con 0x7fcd04101420 2024-09-10T08:43:24.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.875+0000 7fccf9ffb640 1 -- 172.21.15.172:0/1274389507 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4035070 con 0x7fcd04101420 2024-09-10T08:43:24.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.875+0000 7fcd0af96640 1 -- 172.21.15.172:0/1274389507 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd041a3f70 con 0x7fcd04101420 2024-09-10T08:43:24.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.876+0000 7fcd0af96640 1 -- 172.21.15.172:0/1274389507 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd041021c0 con 0x7fcd04101420 2024-09-10T08:43:24.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.877+0000 7fccf9ffb640 1 -- 172.21.15.172:0/1274389507 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fccf4042030 con 0x7fcd04101420 2024-09-10T08:43:24.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.878+0000 7fccf9ffb640 1 --2- 172.21.15.172:0/1274389507 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccec0a2b80 0x7fccec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:24.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.878+0000 7fcd08d32640 1 --2- 172.21.15.172:0/1274389507 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccec0a2b80 0x7fccec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:24.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.878+0000 7fccf9ffb640 1 -- 172.21.15.172:0/1274389507 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fccf40ef180 con 0x7fcd04101420 2024-09-10T08:43:24.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.878+0000 7fcd08d32640 1 --2- 172.21.15.172:0/1274389507 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccec0a2b80 0x7fccec0a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcd00006fd0 tx=0x7fcd00008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:24.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:24.882+0000 7fccf9ffb640 1 -- 172.21.15.172:0/1274389507 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fccf40bc580 con 0x7fcd04101420 2024-09-10T08:43:25.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:25.050+0000 7fcd0af96640 1 -- 172.21.15.172:0/1274389507 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd04109040 con 0x7fcd04101420 2024-09-10T08:43:25.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:25.050+0000 7fccf9ffb640 1 -- 172.21.15.172:0/1274389507 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7fccf40bbe80 con 0x7fcd04101420 2024-09-10T08:43:25.051 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-09-10T08:43:25.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:25.052+0000 7fcd0af96640 1 -- 172.21.15.172:0/1274389507 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccec0a2b80 msgr2=0x7fccec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:25.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:25.052+0000 7fcd0af96640 1 --2- 172.21.15.172:0/1274389507 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccec0a2b80 0x7fccec0a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcd00006fd0 tx=0x7fcd00008040 comp rx=0 tx=0).stop 2024-09-10T08:43:25.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:25.052+0000 7fcd0af96640 1 -- 172.21.15.172:0/1274389507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd04101420 msgr2=0x7fcd04197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:25.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:25.052+0000 7fcd0af96640 1 --2- 172.21.15.172:0/1274389507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd04101420 0x7fcd04197930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fccf4005940 tx=0x7fccf40042f0 comp rx=0 tx=0).stop 2024-09-10T08:43:25.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:25.052+0000 7fcd0af96640 1 -- 172.21.15.172:0/1274389507 shutdown_connections 2024-09-10T08:43:25.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:25.052+0000 7fcd0af96640 1 --2- 172.21.15.172:0/1274389507 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccec0a2b80 0x7fccec0a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:25.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:25.052+0000 7fcd0af96640 1 --2- 172.21.15.172:0/1274389507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd04106a80 0x7fcd0419bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:25.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:25.052+0000 7fcd0af96640 1 --2- 172.21.15.172:0/1274389507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd04101420 0x7fcd04197930 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:25.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:25.052+0000 7fcd0af96640 1 --2- 172.21.15.172:0/1274389507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd04100a80 0x7fcd041973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:25.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:25.052+0000 7fcd0af96640 1 -- 172.21.15.172:0/1274389507 >> 172.21.15.172:0/1274389507 conn(0x7fcd040fc7a0 msgr2=0x7fcd040fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:25.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:25.053+0000 7fcd0af96640 1 -- 172.21.15.172:0/1274389507 shutdown_connections 2024-09-10T08:43:25.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:25.053+0000 7fcd0af96640 1 -- 172.21.15.172:0/1274389507 wait complete. 2024-09-10T08:43:25.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:25 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1274389507' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:43:25.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:25 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1274389507' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:43:25.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:25 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1274389507' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:43:25.825 INFO:tasks.cephadm:Adding mds.a on smithi172 2024-09-10T08:43:25.825 INFO:tasks.cephadm:Adding mds.d on smithi172 2024-09-10T08:43:25.825 INFO:tasks.cephadm:Adding mds.g on smithi172 2024-09-10T08:43:25.825 INFO:tasks.cephadm:Adding mds.j on smithi172 2024-09-10T08:43:25.825 INFO:tasks.cephadm:Adding mds.b on smithi174 2024-09-10T08:43:25.825 INFO:tasks.cephadm:Adding mds.e on smithi174 2024-09-10T08:43:25.825 INFO:tasks.cephadm:Adding mds.h on smithi174 2024-09-10T08:43:25.825 INFO:tasks.cephadm:Adding mds.k on smithi174 2024-09-10T08:43:25.825 INFO:tasks.cephadm:Adding mds.c on smithi191 2024-09-10T08:43:25.825 INFO:tasks.cephadm:Adding mds.f on smithi191 2024-09-10T08:43:25.826 INFO:tasks.cephadm:Adding mds.i on smithi191 2024-09-10T08:43:25.826 INFO:tasks.cephadm:Adding mds.l on smithi191 2024-09-10T08:43:25.826 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi172=a;smithi172=d;smithi172=g;smithi172=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l' 2024-09-10T08:43:26.085 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T08:43:26.413 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:26 smithi174 ceph-mon[34068]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:26.430 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:26 smithi191 ceph-mon[35314]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:26 smithi172 ceph-mon[30854]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:26.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.670+0000 7f18803de640 1 -- 172.21.15.191:0/1832870901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1878102690 msgr2=0x7f1878108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:26.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.670+0000 7f18803de640 1 --2- 172.21.15.191:0/1832870901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1878102690 0x7f1878108f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1870009ca0 tx=0x7f187002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:43:26.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.670+0000 7f18803de640 1 -- 172.21.15.191:0/1832870901 shutdown_connections 2024-09-10T08:43:26.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.670+0000 7f18803de640 1 --2- 172.21.15.191:0/1832870901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1878102690 0x7f1878108f20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:26.693 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.670+0000 7f18803de640 1 --2- 172.21.15.191:0/1832870901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1878101cd0 0x7f1878102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:26.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.670+0000 7f18803de640 1 --2- 172.21.15.191:0/1832870901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1878100ad0 0x7f1878100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:26.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.670+0000 7f18803de640 1 -- 172.21.15.191:0/1832870901 >> 172.21.15.191:0/1832870901 conn(0x7f18780fc280 msgr2=0x7f18780fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:26.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.670+0000 7f18803de640 1 -- 172.21.15.191:0/1832870901 shutdown_connections 2024-09-10T08:43:26.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.670+0000 7f18803de640 1 -- 172.21.15.191:0/1832870901 wait complete. 2024-09-10T08:43:26.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.671+0000 7f18803de640 1 Processor -- start 2024-09-10T08:43:26.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.671+0000 7f18803de640 1 -- start start 2024-09-10T08:43:26.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.671+0000 7f18803de640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1878100ad0 0x7f1878197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:26.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.671+0000 7f18803de640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1878101cd0 0x7f18781982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:26.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.671+0000 7f18803de640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1878102690 0x7f187819f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:26.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.671+0000 7f18803de640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f187810c600 con 0x7f1878101cd0 2024-09-10T08:43:26.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.671+0000 7f18803de640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f187810c480 con 0x7f1878102690 2024-09-10T08:43:26.694 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.671+0000 7f18803de640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f187810c780 con 0x7f1878100ad0 2024-09-10T08:43:26.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.671+0000 7f187e17a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1878100ad0 0x7f1878197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:26.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.671+0000 7f187e17a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1878100ad0 0x7f1878197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:39870/0 (socket says 172.21.15.191:39870) 2024-09-10T08:43:26.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.671+0000 7f187e17a640 1 -- 172.21.15.191:0/291956056 learned_addr learned my addr 172.21.15.191:0/291956056 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-10T08:43:26.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.671+0000 7f187e97b640 1 --2- 172.21.15.191:0/291956056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1878102690 0x7f187819f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:26.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.671+0000 7f187d979640 1 --2- 172.21.15.191:0/291956056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1878101cd0 0x7f18781982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:26.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.672+0000 7f187e17a640 1 -- 172.21.15.191:0/291956056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1878102690 msgr2=0x7f187819f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:26.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.672+0000 7f187e17a640 1 --2- 172.21.15.191:0/291956056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1878102690 0x7f187819f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:26.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.672+0000 7f187e17a640 1 -- 172.21.15.191:0/291956056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1878101cd0 msgr2=0x7f18781982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:26.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.672+0000 7f187e17a640 1 --2- 172.21.15.191:0/291956056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1878101cd0 0x7f18781982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:26.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.672+0000 7f187e17a640 1 -- 172.21.15.191:0/291956056 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187819f9e0 con 0x7f1878100ad0 2024-09-10T08:43:26.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.672+0000 7f187e17a640 1 --2- 172.21.15.191:0/291956056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1878100ad0 0x7f1878197db0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f187400e9c0 tx=0x7f187400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:26.695 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.672+0000 7f186f7fe640 1 -- 172.21.15.191:0/291956056 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187400cd80 con 0x7f1878100ad0 2024-09-10T08:43:26.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.672+0000 7f18803de640 1 -- 172.21.15.191:0/291956056 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187819fcd0 con 0x7f1878100ad0 2024-09-10T08:43:26.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.672+0000 7f186f7fe640 1 -- 172.21.15.191:0/291956056 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18740035f0 con 0x7f1878100ad0 2024-09-10T08:43:26.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.672+0000 7f186f7fe640 1 -- 172.21.15.191:0/291956056 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874010690 con 0x7f1878100ad0 2024-09-10T08:43:26.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.672+0000 7f18803de640 1 -- 172.21.15.191:0/291956056 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18781a01b0 con 0x7f1878100ad0 2024-09-10T08:43:26.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.673+0000 7f18803de640 1 -- 172.21.15.191:0/291956056 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18781a0440 con 0x7f1878100ad0 2024-09-10T08:43:26.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.675+0000 7f186f7fe640 1 -- 172.21.15.191:0/291956056 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f18740037a0 con 0x7f1878100ad0 2024-09-10T08:43:26.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.676+0000 7f186f7fe640 1 --2- 172.21.15.191:0/291956056 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f18640a2bb0 0x7f18640a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:26.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.676+0000 7f186f7fe640 1 -- 172.21.15.191:0/291956056 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f18740cf0b0 con 0x7f1878100ad0 2024-09-10T08:43:26.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.676+0000 7f187d979640 1 --2- 172.21.15.191:0/291956056 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f18640a2bb0 0x7f18640a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:26.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.676+0000 7f187d979640 1 --2- 172.21.15.191:0/291956056 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f18640a2bb0 0x7f18640a5070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f18781992d0 tx=0x7f1868005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:26.696 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.678+0000 7f186f7fe640 1 -- 172.21.15.191:0/291956056 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f187409c4b0 con 0x7f1878100ad0 2024-09-10T08:43:26.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.859+0000 7f18803de640 1 -- 172.21.15.191:0/291956056 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi172=a;smithi172=d;smithi172=g;smithi172=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l", "target": ["mon-mgr", ""]}) -- 0x7f187806a6b0 con 0x7f18640a2bb0 2024-09-10T08:43:26.866 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.866+0000 7f186f7fe640 1 -- 172.21.15.191:0/291956056 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f187806a6b0 con 0x7f18640a2bb0 2024-09-10T08:43:26.866 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled mds.all update... 2024-09-10T08:43:26.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.868+0000 7f18803de640 1 -- 172.21.15.191:0/291956056 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f18640a2bb0 msgr2=0x7f18640a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:26.868 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.868+0000 7f18803de640 1 --2- 172.21.15.191:0/291956056 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f18640a2bb0 0x7f18640a5070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f18781992d0 tx=0x7f1868005f90 comp rx=0 tx=0).stop 2024-09-10T08:43:26.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.868+0000 7f18803de640 1 -- 172.21.15.191:0/291956056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1878100ad0 msgr2=0x7f1878197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:26.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.868+0000 7f18803de640 1 --2- 172.21.15.191:0/291956056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1878100ad0 0x7f1878197db0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f187400e9c0 tx=0x7f187400ee80 comp rx=0 tx=0).stop 2024-09-10T08:43:26.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.868+0000 7f18803de640 1 -- 172.21.15.191:0/291956056 shutdown_connections 2024-09-10T08:43:26.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.868+0000 7f18803de640 1 --2- 172.21.15.191:0/291956056 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f18640a2bb0 0x7f18640a5070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:26.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.868+0000 7f18803de640 1 --2- 172.21.15.191:0/291956056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1878102690 0x7f187819f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:26.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.868+0000 7f18803de640 1 --2- 172.21.15.191:0/291956056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1878101cd0 0x7f18781982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:26.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.868+0000 7f18803de640 1 --2- 172.21.15.191:0/291956056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1878100ad0 0x7f1878197db0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:26.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.868+0000 7f18803de640 1 -- 172.21.15.191:0/291956056 >> 172.21.15.191:0/291956056 conn(0x7f18780fc280 msgr2=0x7f1878069fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:26.869 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.868+0000 7f18803de640 1 -- 172.21.15.191:0/291956056 shutdown_connections 2024-09-10T08:43:26.870 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-10T08:43:26.868+0000 7f18803de640 1 -- 172.21.15.191:0/291956056 wait complete. 2024-09-10T08:43:27.589 DEBUG:teuthology.orchestra.run.smithi172:mds.a> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.a.service 2024-09-10T08:43:27.591 DEBUG:teuthology.orchestra.run.smithi172:mds.d> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.d.service 2024-09-10T08:43:27.635 DEBUG:teuthology.orchestra.run.smithi172:mds.g> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.g.service 2024-09-10T08:43:27.678 DEBUG:teuthology.orchestra.run.smithi172:mds.j> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.j.service 2024-09-10T08:43:27.722 DEBUG:teuthology.orchestra.run.smithi174:mds.b> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.b.service 2024-09-10T08:43:27.725 DEBUG:teuthology.orchestra.run.smithi174:mds.e> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.e.service 2024-09-10T08:43:27.768 DEBUG:teuthology.orchestra.run.smithi174:mds.h> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.h.service 2024-09-10T08:43:27.811 DEBUG:teuthology.orchestra.run.smithi174:mds.k> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.k.service 2024-09-10T08:43:27.854 DEBUG:teuthology.orchestra.run.smithi191:mds.c> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.c.service 2024-09-10T08:43:27.857 DEBUG:teuthology.orchestra.run.smithi191:mds.f> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.f.service 2024-09-10T08:43:27.900 DEBUG:teuthology.orchestra.run.smithi191:mds.i> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.i.service 2024-09-10T08:43:27.943 DEBUG:teuthology.orchestra.run.smithi191:mds.l> sudo journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.l.service 2024-09-10T08:43:27.986 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-10T08:43:27.986 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:43:27.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:27 smithi174 ceph-mon[34068]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi172=a;smithi172=d;smithi172=g;smithi172=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:43:27.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:27 smithi174 ceph-mon[34068]: Saving service mds.all spec with placement smithi172=a;smithi172=d;smithi172=g;smithi172=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l;count:12 2024-09-10T08:43:27.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:27.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:43:27.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:27 smithi174 ceph-mon[34068]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:27.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:27.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:43:27.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:27.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:27.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:27.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:27.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:27 smithi174 ceph-mon[34068]: Deploying daemon mds.l on smithi191 2024-09-10T08:43:28.228 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:28.238 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:27 smithi191 ceph-mon[35314]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi172=a;smithi172=d;smithi172=g;smithi172=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:43:28.239 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:27 smithi191 ceph-mon[35314]: Saving service mds.all spec with placement smithi172=a;smithi172=d;smithi172=g;smithi172=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l;count:12 2024-09-10T08:43:28.239 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:28.239 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:43:28.239 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:27 smithi191 ceph-mon[35314]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:28.239 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:28.239 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:43:28.239 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:28.239 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:28.239 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:28.239 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:28.239 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:27 smithi191 ceph-mon[35314]: Deploying daemon mds.l on smithi191 2024-09-10T08:43:28.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:27 smithi172 ceph-mon[30854]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi172=a;smithi172=d;smithi172=g;smithi172=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:43:28.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:27 smithi172 ceph-mon[30854]: Saving service mds.all spec with placement smithi172=a;smithi172=d;smithi172=g;smithi172=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l;count:12 2024-09-10T08:43:28.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:28.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:43:28.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:27 smithi172 ceph-mon[30854]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:28.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:28.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:43:28.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:28.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:28.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:28.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:28.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:27 smithi172 ceph-mon[30854]: Deploying daemon mds.l on smithi191 2024-09-10T08:43:28.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.731+0000 7f4b9c667640 1 -- 172.21.15.172:0/1604527549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b94106f10 msgr2=0x7f4b9410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:28.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.731+0000 7f4b9c667640 1 --2- 172.21.15.172:0/1604527549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b94106f10 0x7f4b9410ea90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009ca0 tx=0x7f4b8c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.731+0000 7f4b9c667640 1 -- 172.21.15.172:0/1604527549 shutdown_connections 2024-09-10T08:43:28.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.731+0000 7f4b9c667640 1 --2- 172.21.15.172:0/1604527549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b94106f10 0x7f4b9410ea90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.731+0000 7f4b9c667640 1 --2- 172.21.15.172:0/1604527549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b94102790 0x7f4b941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.731+0000 7f4b9c667640 1 --2- 172.21.15.172:0/1604527549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b94101de0 0x7f4b941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.731+0000 7f4b9c667640 1 -- 172.21.15.172:0/1604527549 >> 172.21.15.172:0/1604527549 conn(0x7f4b940fd8c0 msgr2=0x7f4b940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:28.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.731+0000 7f4b9c667640 1 -- 172.21.15.172:0/1604527549 shutdown_connections 2024-09-10T08:43:28.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.731+0000 7f4b9c667640 1 -- 172.21.15.172:0/1604527549 wait complete. 2024-09-10T08:43:28.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.731+0000 7f4b9c667640 1 Processor -- start 2024-09-10T08:43:28.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.731+0000 7f4b9c667640 1 -- start start 2024-09-10T08:43:28.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9c667640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b94101de0 0x7f4b940744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9c667640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b94102790 0x7f4b94074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9c667640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b94106f10 0x7f4b94074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9c667640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4b94111e70 con 0x7f4b94101de0 2024-09-10T08:43:28.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9c667640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b94111cf0 con 0x7f4b94106f10 2024-09-10T08:43:28.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9c667640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4b94111ff0 con 0x7f4b94102790 2024-09-10T08:43:28.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9a403640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b94101de0 0x7f4b940744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9a403640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b94101de0 0x7f4b940744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57168/0 (socket says 172.21.15.172:57168) 2024-09-10T08:43:28.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9a403640 1 -- 172.21.15.172:0/3523892727 learned_addr learned my addr 172.21.15.172:0/3523892727 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:43:28.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9ac04640 1 --2- 172.21.15.172:0/3523892727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b94106f10 0x7f4b94074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b99c02640 1 --2- 172.21.15.172:0/3523892727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b94102790 0x7f4b94074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9a403640 1 -- 172.21.15.172:0/3523892727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b94102790 msgr2=0x7f4b94074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:28.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9a403640 1 --2- 172.21.15.172:0/3523892727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b94102790 0x7f4b94074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9a403640 1 -- 172.21.15.172:0/3523892727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b94106f10 msgr2=0x7f4b94074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:28.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9a403640 1 --2- 172.21.15.172:0/3523892727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b94106f10 0x7f4b94074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9a403640 1 -- 172.21.15.172:0/3523892727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b941a67d0 con 0x7f4b94101de0 2024-09-10T08:43:28.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b99c02640 1 --2- 172.21.15.172:0/3523892727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b94102790 0x7f4b94074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:43:28.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9ac04640 1 --2- 172.21.15.172:0/3523892727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b94106f10 0x7f4b94074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:43:28.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9a403640 1 --2- 172.21.15.172:0/3523892727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b94101de0 0x7f4b940744f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000e9f0 tx=0x7f4b9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:28.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b8b7fe640 1 -- 172.21.15.172:0/3523892727 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9000cd80 con 0x7f4b94101de0 2024-09-10T08:43:28.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b8b7fe640 1 -- 172.21.15.172:0/3523892727 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b900108e0 con 0x7f4b94101de0 2024-09-10T08:43:28.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9c667640 1 -- 172.21.15.172:0/3523892727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b941a6ac0 con 0x7f4b94101de0 2024-09-10T08:43:28.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b8b7fe640 1 -- 172.21.15.172:0/3523892727 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90018920 con 0x7f4b94101de0 2024-09-10T08:43:28.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.732+0000 7f4b9c667640 1 -- 172.21.15.172:0/3523892727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b941a7000 con 0x7f4b94101de0 2024-09-10T08:43:28.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.734+0000 7f4b9c667640 1 -- 172.21.15.172:0/3523892727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b941021c0 con 0x7f4b94101de0 2024-09-10T08:43:28.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.734+0000 7f4b8b7fe640 1 -- 172.21.15.172:0/3523892727 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4b90010a90 con 0x7f4b94101de0 2024-09-10T08:43:28.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.735+0000 7f4b8b7fe640 1 --2- 172.21.15.172:0/3523892727 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4b800a2b80 0x7f4b800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:28.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.735+0000 7f4b99c02640 1 --2- 172.21.15.172:0/3523892727 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4b800a2b80 0x7f4b800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:28.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.735+0000 7f4b8b7fe640 1 -- 172.21.15.172:0/3523892727 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f4b900cefd0 con 0x7f4b94101de0 2024-09-10T08:43:28.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.735+0000 7f4b99c02640 1 --2- 172.21.15.172:0/3523892727 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4b800a2b80 0x7f4b800a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b94075f80 tx=0x7f4b84005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:28.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.739+0000 7f4b8b7fe640 1 -- 172.21.15.172:0/3523892727 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b9009c3d0 con 0x7f4b94101de0 2024-09-10T08:43:28.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.940+0000 7f4b9c667640 1 -- 172.21.15.172:0/3523892727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b9410a180 con 0x7f4b94101de0 2024-09-10T08:43:28.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.941+0000 7f4b8b7fe640 1 -- 172.21.15.172:0/3523892727 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f4b9009bcd0 con 0x7f4b94101de0 2024-09-10T08:43:28.942 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:43:28.942 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":1,"btime":"2024-09-10T08:36:33:083320+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-10T08:43:28.943 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 1 2024-09-10T08:43:28.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.943+0000 7f4b9c667640 1 -- 172.21.15.172:0/3523892727 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4b800a2b80 msgr2=0x7f4b800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:28.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.943+0000 7f4b9c667640 1 --2- 172.21.15.172:0/3523892727 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4b800a2b80 0x7f4b800a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b94075f80 tx=0x7f4b84005f90 comp rx=0 tx=0).stop 2024-09-10T08:43:28.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.943+0000 7f4b9c667640 1 -- 172.21.15.172:0/3523892727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b94101de0 msgr2=0x7f4b940744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:28.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.943+0000 7f4b9c667640 1 --2- 172.21.15.172:0/3523892727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b94101de0 0x7f4b940744f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000e9f0 tx=0x7f4b9000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.943+0000 7f4b9c667640 1 -- 172.21.15.172:0/3523892727 shutdown_connections 2024-09-10T08:43:28.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.943+0000 7f4b9c667640 1 --2- 172.21.15.172:0/3523892727 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4b800a2b80 0x7f4b800a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.943+0000 7f4b9c667640 1 --2- 172.21.15.172:0/3523892727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b94106f10 0x7f4b94074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.943+0000 7f4b9c667640 1 --2- 172.21.15.172:0/3523892727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b94102790 0x7f4b94074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.943+0000 7f4b9c667640 1 --2- 172.21.15.172:0/3523892727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b94101de0 0x7f4b940744f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:28.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.943+0000 7f4b9c667640 1 -- 172.21.15.172:0/3523892727 >> 172.21.15.172:0/3523892727 conn(0x7f4b940fd8c0 msgr2=0x7f4b940fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:28.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.943+0000 7f4b9c667640 1 -- 172.21.15.172:0/3523892727 shutdown_connections 2024-09-10T08:43:28.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:28.943+0000 7f4b9c667640 1 -- 172.21.15.172:0/3523892727 wait complete. 2024-09-10T08:43:29.170 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:28 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3523892727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:29.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:28 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3523892727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:29.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:28 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3523892727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:29.800 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 08:43:29 smithi191 systemd[1]: Starting Ceph mds.l for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T08:43:30.131 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:30 smithi191 ceph-mon[35314]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:30.131 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 08:43:29 smithi191 podman[64374]: 2024-09-10 08:43:29.763749244 +0000 UTC m=+0.015776112 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:43:30.132 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 08:43:29 smithi191 podman[64374]: 2024-09-10 08:43:29.998308703 +0000 UTC m=+0.250335571 container create b309f6acf4fc96b10f84c9a7d7a3c98a2f0d7d2d38ecb510321164a17384d4a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2) 2024-09-10T08:43:30.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:29 smithi172 ceph-mon[30854]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:30.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:29 smithi174 ceph-mon[34068]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:30.550 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 08:43:30 smithi191 podman[64374]: 2024-09-10 08:43:30.253982978 +0000 UTC m=+0.506009850 container init b309f6acf4fc96b10f84c9a7d7a3c98a2f0d7d2d38ecb510321164a17384d4a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-l, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T08:43:30.550 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 08:43:30 smithi191 podman[64374]: 2024-09-10 08:43:30.256727569 +0000 UTC m=+0.508754438 container start b309f6acf4fc96b10f84c9a7d7a3c98a2f0d7d2d38ecb510321164a17384d4a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T08:43:30.550 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 08:43:30 smithi191 ceph-mds[64393]: -- 172.21.15.191:0/4190428409 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a333ee9860 con 0x55a333ec6800 2024-09-10T08:43:30.550 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 08:43:30 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-l[64389]: starting mds.l at 2024-09-10T08:43:30.550 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 08:43:30 smithi191 bash[64374]: b309f6acf4fc96b10f84c9a7d7a3c98a2f0d7d2d38ecb510321164a17384d4a3 2024-09-10T08:43:30.550 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 08:43:30 smithi191 systemd[1]: Started Ceph mds.l for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:43:31.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:31 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] up:boot 2024-09-10T08:43:31.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:31 smithi174 ceph-mon[34068]: fsmap 1 up:standby 2024-09-10T08:43:31.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:31 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T08:43:31.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:31 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:31.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:31 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:31.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:31 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:31.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:31 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:31.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:31 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:31.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:31 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:31.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:31 smithi174 ceph-mon[34068]: Deploying daemon mds.k on smithi174 2024-09-10T08:43:31.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:31 smithi174 ceph-mon[34068]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:31.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:31 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] up:boot 2024-09-10T08:43:31.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:31 smithi191 ceph-mon[35314]: fsmap 1 up:standby 2024-09-10T08:43:31.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:31 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T08:43:31.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:31 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:31.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:31 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:31.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:31 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:31.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:31 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:31.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:31 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:31.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:31 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:31.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:31 smithi191 ceph-mon[35314]: Deploying daemon mds.k on smithi174 2024-09-10T08:43:31.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:31 smithi191 ceph-mon[35314]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:31.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:31 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] up:boot 2024-09-10T08:43:31.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:31 smithi172 ceph-mon[30854]: fsmap 1 up:standby 2024-09-10T08:43:31.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:31 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T08:43:31.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:31 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:31.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:31 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:31.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:31 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:31.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:31 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:31.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:31 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:31.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:31 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:31.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:31 smithi172 ceph-mon[30854]: Deploying daemon mds.k on smithi174 2024-09-10T08:43:31.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:31 smithi172 ceph-mon[30854]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:31.685 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:43:31.941 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:32.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.439+0000 7f5d126c2640 1 -- 172.21.15.172:0/1831682877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d0c101420 msgr2=0x7f5d0c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:32.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.439+0000 7f5d126c2640 1 --2- 172.21.15.172:0/1831682877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d0c101420 0x7f5d0c10d950 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009cd0 tx=0x7f5d0402a8e0 comp rx=0 tx=0).stop 2024-09-10T08:43:32.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.439+0000 7f5d126c2640 1 -- 172.21.15.172:0/1831682877 shutdown_connections 2024-09-10T08:43:32.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.439+0000 7f5d126c2640 1 --2- 172.21.15.172:0/1831682877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d0c101420 0x7f5d0c10d950 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:32.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.439+0000 7f5d126c2640 1 --2- 172.21.15.172:0/1831682877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d0c100a80 0x7f5d0c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:32.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.439+0000 7f5d126c2640 1 --2- 172.21.15.172:0/1831682877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c106a80 0x7f5d0c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:32.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.439+0000 7f5d126c2640 1 -- 172.21.15.172:0/1831682877 >> 172.21.15.172:0/1831682877 conn(0x7f5d0c0fc7a0 msgr2=0x7f5d0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:32.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.439+0000 7f5d126c2640 1 -- 172.21.15.172:0/1831682877 shutdown_connections 2024-09-10T08:43:32.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.439+0000 7f5d126c2640 1 -- 172.21.15.172:0/1831682877 wait complete. 2024-09-10T08:43:32.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d126c2640 1 Processor -- start 2024-09-10T08:43:32.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d126c2640 1 -- start start 2024-09-10T08:43:32.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d126c2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d0c100a80 0x7f5d0c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:32.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d126c2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d0c101420 0x7f5d0c197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:32.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d126c2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c106a80 0x7f5d0c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:32.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d126c2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c110d30 con 0x7f5d0c100a80 2024-09-10T08:43:32.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d126c2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c110bb0 con 0x7f5d0c106a80 2024-09-10T08:43:32.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d126c2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c110eb0 con 0x7f5d0c101420 2024-09-10T08:43:32.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d0c100a80 0x7f5d0c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:32.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d0bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d0c100a80 0x7f5d0c197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57190/0 (socket says 172.21.15.172:57190) 2024-09-10T08:43:32.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d0bfff640 1 -- 172.21.15.172:0/1983200669 learned_addr learned my addr 172.21.15.172:0/1983200669 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:43:32.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d10c5f640 1 --2- 172.21.15.172:0/1983200669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c106a80 0x7f5d0c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:32.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d0b7fe640 1 --2- 172.21.15.172:0/1983200669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d0c101420 0x7f5d0c197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:32.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d0bfff640 1 -- 172.21.15.172:0/1983200669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d0c101420 msgr2=0x7f5d0c197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:32.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d0bfff640 1 --2- 172.21.15.172:0/1983200669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d0c101420 0x7f5d0c197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d0bfff640 1 -- 172.21.15.172:0/1983200669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c106a80 msgr2=0x7f5d0c19bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d0bfff640 1 --2- 172.21.15.172:0/1983200669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c106a80 0x7f5d0c19bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.440+0000 7f5d0bfff640 1 -- 172.21.15.172:0/1983200669 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c19c1f0 con 0x7f5d0c100a80 2024-09-10T08:43:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.441+0000 7f5d0b7fe640 1 --2- 172.21.15.172:0/1983200669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d0c101420 0x7f5d0c197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:43:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.441+0000 7f5d10c5f640 1 --2- 172.21.15.172:0/1983200669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c106a80 0x7f5d0c19bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:43:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.441+0000 7f5d0bfff640 1 --2- 172.21.15.172:0/1983200669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d0c100a80 0x7f5d0c197240 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009970 tx=0x7f5cfc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.441+0000 7f5d097fa640 1 -- 172.21.15.172:0/1983200669 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc00d020 con 0x7f5d0c100a80 2024-09-10T08:43:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.441+0000 7f5d126c2640 1 -- 172.21.15.172:0/1983200669 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0c19c4e0 con 0x7f5d0c100a80 2024-09-10T08:43:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.441+0000 7f5d097fa640 1 -- 172.21.15.172:0/1983200669 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cfc00d1c0 con 0x7f5d0c100a80 2024-09-10T08:43:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.441+0000 7f5d097fa640 1 -- 172.21.15.172:0/1983200669 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc00cc50 con 0x7f5d0c100a80 2024-09-10T08:43:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.441+0000 7f5d126c2640 1 -- 172.21.15.172:0/1983200669 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0c06a790 con 0x7f5d0c100a80 2024-09-10T08:43:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.442+0000 7f5d126c2640 1 -- 172.21.15.172:0/1983200669 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d0c100ee0 con 0x7f5d0c100a80 2024-09-10T08:43:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.443+0000 7f5d097fa640 1 -- 172.21.15.172:0/1983200669 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5cfc00c460 con 0x7f5d0c100a80 2024-09-10T08:43:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.444+0000 7f5d097fa640 1 --2- 172.21.15.172:0/1983200669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5cf80a2b80 0x7f5cf80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.444+0000 7f5d0b7fe640 1 --2- 172.21.15.172:0/1983200669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5cf80a2b80 0x7f5cf80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.444+0000 7f5d097fa640 1 -- 172.21.15.172:0/1983200669 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f5cfc0d0020 con 0x7f5d0c100a80 2024-09-10T08:43:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.444+0000 7f5d0b7fe640 1 --2- 172.21.15.172:0/1983200669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5cf80a2b80 0x7f5cf80a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4005d00 tx=0x7f5cf4009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.448+0000 7f5d097fa640 1 -- 172.21.15.172:0/1983200669 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5cfc09d420 con 0x7f5d0c100a80 2024-09-10T08:43:32.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.648+0000 7f5d126c2640 1 -- 172.21.15.172:0/1983200669 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d0c06b270 con 0x7f5d0c100a80 2024-09-10T08:43:32.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.648+0000 7f5d097fa640 1 -- 172.21.15.172:0/1983200669 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f5cfc09cd20 con 0x7f5d0c100a80 2024-09-10T08:43:32.649 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:43:32.649 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":2,"btime":"2024-09-10T08:43:30:284141+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T08:43:32.649 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 2 2024-09-10T08:43:32.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.651+0000 7f5d126c2640 1 -- 172.21.15.172:0/1983200669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5cf80a2b80 msgr2=0x7f5cf80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:32.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.651+0000 7f5d126c2640 1 --2- 172.21.15.172:0/1983200669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5cf80a2b80 0x7f5cf80a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4005d00 tx=0x7f5cf4009550 comp rx=0 tx=0).stop 2024-09-10T08:43:32.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.651+0000 7f5d126c2640 1 -- 172.21.15.172:0/1983200669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d0c100a80 msgr2=0x7f5d0c197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:32.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.651+0000 7f5d126c2640 1 --2- 172.21.15.172:0/1983200669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d0c100a80 0x7f5d0c197240 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009970 tx=0x7f5cfc009e30 comp rx=0 tx=0).stop 2024-09-10T08:43:32.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.651+0000 7f5d126c2640 1 -- 172.21.15.172:0/1983200669 shutdown_connections 2024-09-10T08:43:32.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.651+0000 7f5d126c2640 1 --2- 172.21.15.172:0/1983200669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5cf80a2b80 0x7f5cf80a5040 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:32.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.651+0000 7f5d126c2640 1 --2- 172.21.15.172:0/1983200669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d0c106a80 0x7f5d0c19bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:32.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.651+0000 7f5d126c2640 1 --2- 172.21.15.172:0/1983200669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d0c101420 0x7f5d0c197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:32.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.651+0000 7f5d126c2640 1 --2- 172.21.15.172:0/1983200669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d0c100a80 0x7f5d0c197240 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:32.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.651+0000 7f5d126c2640 1 -- 172.21.15.172:0/1983200669 >> 172.21.15.172:0/1983200669 conn(0x7f5d0c0fc7a0 msgr2=0x7f5d0c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:32.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.651+0000 7f5d126c2640 1 -- 172.21.15.172:0/1983200669 shutdown_connections 2024-09-10T08:43:32.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:32.651+0000 7f5d126c2640 1 -- 172.21.15.172:0/1983200669 wait complete. 2024-09-10T08:43:32.701 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 08:43:32 smithi174 systemd[1]: Starting Ceph mds.k for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T08:43:32.948 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:32 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1983200669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:32.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:32 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1983200669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:32.992 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 08:43:32 smithi174 podman[62963]: 2024-09-10 08:43:32.678339489 +0000 UTC m=+0.018285035 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:43:32.992 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 08:43:32 smithi174 podman[62963]: 2024-09-10 08:43:32.798753961 +0000 UTC m=+0.138699558 container create 7e751f71648b913eb23b4c5ec898a63c3d0c7e7dd3f589c31d01944a655f93ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-k, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T08:43:33.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:32 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1983200669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:33.492 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 08:43:33 smithi174 podman[62963]: 2024-09-10 08:43:33.051485301 +0000 UTC m=+0.391430842 container init 7e751f71648b913eb23b4c5ec898a63c3d0c7e7dd3f589c31d01944a655f93ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-10T08:43:33.492 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 08:43:33 smithi174 podman[62963]: 2024-09-10 08:43:33.054432109 +0000 UTC m=+0.394377653 container start 7e751f71648b913eb23b4c5ec898a63c3d0c7e7dd3f589c31d01944a655f93ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-k, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T08:43:33.492 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 08:43:33 smithi174 ceph-mds[62982]: -- 172.21.15.174:0/3859659922 <== mon.2 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56050e38d860 con 0x56050e36b400 2024-09-10T08:43:33.492 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 08:43:33 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-k[62978]: starting mds.k at 2024-09-10T08:43:33.492 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 08:43:33 smithi174 bash[62963]: 7e751f71648b913eb23b4c5ec898a63c3d0c7e7dd3f589c31d01944a655f93ff 2024-09-10T08:43:33.493 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 08:43:33 smithi174 systemd[1]: Started Ceph mds.k for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:43:34.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:34 smithi172 ceph-mon[30854]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:34.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:34 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:34.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:34 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:34.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:34 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:34.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:34 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:34.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:34 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:34.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:34 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:34.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:34 smithi172 ceph-mon[30854]: Deploying daemon mds.j on smithi172 2024-09-10T08:43:34.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:34 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] up:boot 2024-09-10T08:43:34.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:34 smithi172 ceph-mon[30854]: fsmap 2 up:standby 2024-09-10T08:43:34.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:34 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T08:43:34.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:34 smithi174 ceph-mon[34068]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:34.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:34 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:34.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:34 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:34.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:34 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:34.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:34 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:34.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:34 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:34.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:34 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:34.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:34 smithi174 ceph-mon[34068]: Deploying daemon mds.j on smithi172 2024-09-10T08:43:34.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:34 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] up:boot 2024-09-10T08:43:34.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:34 smithi174 ceph-mon[34068]: fsmap 2 up:standby 2024-09-10T08:43:34.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:34 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T08:43:34.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:34 smithi191 ceph-mon[35314]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:34.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:34 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:34.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:34 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:34.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:34 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:34.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:34 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:34.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:34 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:34.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:34 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:34.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:34 smithi191 ceph-mon[35314]: Deploying daemon mds.j on smithi172 2024-09-10T08:43:34.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:34 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] up:boot 2024-09-10T08:43:34.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:34 smithi191 ceph-mon[35314]: fsmap 2 up:standby 2024-09-10T08:43:34.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:34 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T08:43:35.332 INFO:journalctl@ceph.mds.j.smithi172.stdout:Sep 10 08:43:35 smithi172 podman[67741]: 2024-09-10 08:43:35.310958816 +0000 UTC m=+0.136291650 container create 1278041f7ff316f6ef968d6f4507a932bb7dbfa946b20c378181f827ae99fef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T08:43:35.380 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:43:35.731 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:35.832 INFO:journalctl@ceph.mds.j.smithi172.stdout:Sep 10 08:43:35 smithi172 podman[67741]: 2024-09-10 08:43:35.555289738 +0000 UTC m=+0.380622502 container init 1278041f7ff316f6ef968d6f4507a932bb7dbfa946b20c378181f827ae99fef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-j, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T08:43:35.832 INFO:journalctl@ceph.mds.j.smithi172.stdout:Sep 10 08:43:35 smithi172 podman[67741]: 2024-09-10 08:43:35.558315955 +0000 UTC m=+0.383648732 container start 1278041f7ff316f6ef968d6f4507a932bb7dbfa946b20c378181f827ae99fef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-j, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T08:43:35.832 INFO:journalctl@ceph.mds.j.smithi172.stdout:Sep 10 08:43:35 smithi172 ceph-mds[67778]: -- 172.21.15.172:0/2758637654 <== mon.0 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad716bf860 con 0x55ad7169d000 2024-09-10T08:43:35.832 INFO:journalctl@ceph.mds.j.smithi172.stdout:Sep 10 08:43:35 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-j[67774]: starting mds.j at 2024-09-10T08:43:35.832 INFO:journalctl@ceph.mds.j.smithi172.stdout:Sep 10 08:43:35 smithi172 bash[67741]: 1278041f7ff316f6ef968d6f4507a932bb7dbfa946b20c378181f827ae99fef2 2024-09-10T08:43:35.832 INFO:journalctl@ceph.mds.j.smithi172.stdout:Sep 10 08:43:35 smithi172 systemd[1]: Started Ceph mds.j for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:43:36.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.257+0000 7f30d3198640 1 -- 172.21.15.172:0/2132969429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30cc101df0 msgr2=0x7f30cc1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:36.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.257+0000 7f30d3198640 1 --2- 172.21.15.172:0/2132969429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30cc101df0 0x7f30cc1021d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009900 tx=0x7f30c402a410 comp rx=0 tx=0).stop 2024-09-10T08:43:36.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.257+0000 7f30d3198640 1 -- 172.21.15.172:0/2132969429 shutdown_connections 2024-09-10T08:43:36.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.257+0000 7f30d3198640 1 --2- 172.21.15.172:0/2132969429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30cc106f20 0x7f30cc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:36.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.257+0000 7f30d3198640 1 --2- 172.21.15.172:0/2132969429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30cc1027a0 0x7f30cc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:36.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.257+0000 7f30d3198640 1 --2- 172.21.15.172:0/2132969429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30cc101df0 0x7f30cc1021d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:36.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.257+0000 7f30d3198640 1 -- 172.21.15.172:0/2132969429 >> 172.21.15.172:0/2132969429 conn(0x7f30cc0fd8f0 msgr2=0x7f30cc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:36.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.257+0000 7f30d3198640 1 -- 172.21.15.172:0/2132969429 shutdown_connections 2024-09-10T08:43:36.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.257+0000 7f30d3198640 1 -- 172.21.15.172:0/2132969429 wait complete. 2024-09-10T08:43:36.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.257+0000 7f30d3198640 1 Processor -- start 2024-09-10T08:43:36.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d3198640 1 -- start start 2024-09-10T08:43:36.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d3198640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30cc101df0 0x7f30cc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:36.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d3198640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30cc1027a0 0x7f30cc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:36.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d3198640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30cc106f20 0x7f30cc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:36.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d3198640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc111e50 con 0x7f30cc106f20 2024-09-10T08:43:36.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d3198640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc111cd0 con 0x7f30cc1027a0 2024-09-10T08:43:36.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d3198640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc111fd0 con 0x7f30cc101df0 2024-09-10T08:43:36.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d1735640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30cc106f20 0x7f30cc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:36.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d1735640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30cc106f20 0x7f30cc074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59062/0 (socket says 172.21.15.172:59062) 2024-09-10T08:43:36.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d1735640 1 -- 172.21.15.172:0/1533734329 learned_addr learned my addr 172.21.15.172:0/1533734329 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:43:36.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d0f34640 1 --2- 172.21.15.172:0/1533734329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30cc101df0 0x7f30cc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:36.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30cbfff640 1 --2- 172.21.15.172:0/1533734329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30cc1027a0 0x7f30cc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:36.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d1735640 1 -- 172.21.15.172:0/1533734329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30cc101df0 msgr2=0x7f30cc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:36.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d1735640 1 --2- 172.21.15.172:0/1533734329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30cc101df0 0x7f30cc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:36.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d1735640 1 -- 172.21.15.172:0/1533734329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30cc1027a0 msgr2=0x7f30cc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:36.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d1735640 1 --2- 172.21.15.172:0/1533734329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30cc1027a0 0x7f30cc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:36.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d1735640 1 -- 172.21.15.172:0/1533734329 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30cc1a6850 con 0x7f30cc106f20 2024-09-10T08:43:36.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d0f34640 1 --2- 172.21.15.172:0/1533734329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30cc101df0 0x7f30cc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:43:36.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30cbfff640 1 --2- 172.21.15.172:0/1533734329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30cc1027a0 0x7f30cc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:43:36.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.258+0000 7f30d1735640 1 --2- 172.21.15.172:0/1533734329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30cc106f20 0x7f30cc074f70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f30b4009fd0 tx=0x7f30b400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:36.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.259+0000 7f30c9ffb640 1 -- 172.21.15.172:0/1533734329 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4011830 con 0x7f30cc106f20 2024-09-10T08:43:36.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.259+0000 7f30c9ffb640 1 -- 172.21.15.172:0/1533734329 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30b4016070 con 0x7f30cc106f20 2024-09-10T08:43:36.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.259+0000 7f30d3198640 1 -- 172.21.15.172:0/1533734329 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30cc1a6b40 con 0x7f30cc106f20 2024-09-10T08:43:36.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.259+0000 7f30c9ffb640 1 -- 172.21.15.172:0/1533734329 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4010de0 con 0x7f30cc106f20 2024-09-10T08:43:36.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.259+0000 7f30d3198640 1 -- 172.21.15.172:0/1533734329 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30cc1a7080 con 0x7f30cc106f20 2024-09-10T08:43:36.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.260+0000 7f30d3198640 1 -- 172.21.15.172:0/1533734329 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30cc1021d0 con 0x7f30cc106f20 2024-09-10T08:43:36.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.261+0000 7f30c9ffb640 1 -- 172.21.15.172:0/1533734329 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f30b40119d0 con 0x7f30cc106f20 2024-09-10T08:43:36.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.261+0000 7f30c9ffb640 1 --2- 172.21.15.172:0/1533734329 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30b80a2b80 0x7f30b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:36.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.261+0000 7f30d0f34640 1 --2- 172.21.15.172:0/1533734329 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30b80a2b80 0x7f30b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:36.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.262+0000 7f30c9ffb640 1 -- 172.21.15.172:0/1533734329 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f30b40ce320 con 0x7f30cc106f20 2024-09-10T08:43:36.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.262+0000 7f30d0f34640 1 --2- 172.21.15.172:0/1533734329 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30b80a2b80 0x7f30b80a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f30c4031440 tx=0x7f30c40095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:36.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.266+0000 7f30c9ffb640 1 -- 172.21.15.172:0/1533734329 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30b409b720 con 0x7f30cc106f20 2024-09-10T08:43:36.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:36 smithi172 ceph-mon[30854]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:36.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:36 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:36.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:36 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:36.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:36 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:36.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:36 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:36.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:36 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:36.446 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:36 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:36.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.476+0000 7f30d3198640 1 -- 172.21.15.172:0/1533734329 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30cc10a100 con 0x7f30cc106f20 2024-09-10T08:43:36.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.477+0000 7f30c9ffb640 1 -- 172.21.15.172:0/1533734329 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3094 (secure 0 0 0) 0x7f30b409b020 con 0x7f30cc106f20 2024-09-10T08:43:36.478 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:43:36.478 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":4,"btime":"2024-09-10T08:43:36:167495+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T08:43:36.478 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 4 2024-09-10T08:43:36.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.479+0000 7f30d3198640 1 -- 172.21.15.172:0/1533734329 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30b80a2b80 msgr2=0x7f30b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:36.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.479+0000 7f30d3198640 1 --2- 172.21.15.172:0/1533734329 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30b80a2b80 0x7f30b80a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f30c4031440 tx=0x7f30c40095f0 comp rx=0 tx=0).stop 2024-09-10T08:43:36.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.479+0000 7f30d3198640 1 -- 172.21.15.172:0/1533734329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30cc106f20 msgr2=0x7f30cc074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:36.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.479+0000 7f30d3198640 1 --2- 172.21.15.172:0/1533734329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30cc106f20 0x7f30cc074f70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f30b4009fd0 tx=0x7f30b400b8e0 comp rx=0 tx=0).stop 2024-09-10T08:43:36.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.480+0000 7f30d3198640 1 -- 172.21.15.172:0/1533734329 shutdown_connections 2024-09-10T08:43:36.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.480+0000 7f30d3198640 1 --2- 172.21.15.172:0/1533734329 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30b80a2b80 0x7f30b80a5040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:36.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.480+0000 7f30d3198640 1 --2- 172.21.15.172:0/1533734329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30cc106f20 0x7f30cc074f70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:36.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.480+0000 7f30d3198640 1 --2- 172.21.15.172:0/1533734329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30cc1027a0 0x7f30cc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:36.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.480+0000 7f30d3198640 1 --2- 172.21.15.172:0/1533734329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30cc101df0 0x7f30cc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:36.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.480+0000 7f30d3198640 1 -- 172.21.15.172:0/1533734329 >> 172.21.15.172:0/1533734329 conn(0x7f30cc0fd8f0 msgr2=0x7f30cc0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:36.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.480+0000 7f30d3198640 1 -- 172.21.15.172:0/1533734329 shutdown_connections 2024-09-10T08:43:36.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:36.480+0000 7f30d3198640 1 -- 172.21.15.172:0/1533734329 wait complete. 2024-09-10T08:43:36.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:36 smithi174 ceph-mon[34068]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:36.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:36 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:36.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:36 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:36.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:36 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:36.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:36 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:36.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:36 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:36.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:36 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:36 smithi191 ceph-mon[35314]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:36 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:36 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:36 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:36 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:36 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:36 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:37.451 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:37 smithi172 ceph-mon[30854]: Deploying daemon mds.d on smithi172 2024-09-10T08:43:37.452 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:37 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.172:6834/2272815205,v1:172.21.15.172:6835/2272815205] up:boot 2024-09-10T08:43:37.452 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:37 smithi172 ceph-mon[30854]: fsmap 3 up:standby 2024-09-10T08:43:37.452 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:37 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T08:43:37.452 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:37 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1533734329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:37.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:37 smithi174 ceph-mon[34068]: Deploying daemon mds.d on smithi172 2024-09-10T08:43:37.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:37 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.172:6834/2272815205,v1:172.21.15.172:6835/2272815205] up:boot 2024-09-10T08:43:37.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:37 smithi174 ceph-mon[34068]: fsmap 3 up:standby 2024-09-10T08:43:37.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:37 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T08:43:37.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:37 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1533734329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:37.498 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:37 smithi191 ceph-mon[35314]: Deploying daemon mds.d on smithi172 2024-09-10T08:43:37.498 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:37 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.172:6834/2272815205,v1:172.21.15.172:6835/2272815205] up:boot 2024-09-10T08:43:37.498 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:37 smithi191 ceph-mon[35314]: fsmap 3 up:standby 2024-09-10T08:43:37.498 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:37 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T08:43:37.498 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:37 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1533734329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:38.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:38 smithi174 ceph-mon[34068]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:38.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:38 smithi191 ceph-mon[35314]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:38.555 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:38 smithi172 ceph-mon[30854]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:38.832 INFO:journalctl@ceph.mds.d.smithi172.stdout:Sep 10 08:43:38 smithi172 systemd[1]: Starting Ceph mds.d for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T08:43:39.229 INFO:journalctl@ceph.mds.d.smithi172.stdout:Sep 10 08:43:38 smithi172 podman[68155]: 2024-09-10 08:43:38.844781439 +0000 UTC m=+0.016414299 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:43:39.229 INFO:journalctl@ceph.mds.d.smithi172.stdout:Sep 10 08:43:38 smithi172 podman[68155]: 2024-09-10 08:43:38.968491853 +0000 UTC m=+0.140124704 container create 50cb24e942a546d1f0022ad9aae7418514c52c0f983d04166b736bf6eafb7024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T08:43:39.229 INFO:journalctl@ceph.mds.d.smithi172.stdout:Sep 10 08:43:39 smithi172 podman[68155]: 2024-09-10 08:43:39.229698668 +0000 UTC m=+0.401331518 container init 50cb24e942a546d1f0022ad9aae7418514c52c0f983d04166b736bf6eafb7024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-d, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T08:43:39.356 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:43:39.582 INFO:journalctl@ceph.mds.d.smithi172.stdout:Sep 10 08:43:39 smithi172 podman[68155]: 2024-09-10 08:43:39.23277379 +0000 UTC m=+0.404406641 container start 50cb24e942a546d1f0022ad9aae7418514c52c0f983d04166b736bf6eafb7024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T08:43:39.582 INFO:journalctl@ceph.mds.d.smithi172.stdout:Sep 10 08:43:39 smithi172 ceph-mds[68175]: -- 172.21.15.172:0/685685793 <== mon.0 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b79d9a5860 con 0x55b79d983000 2024-09-10T08:43:39.582 INFO:journalctl@ceph.mds.d.smithi172.stdout:Sep 10 08:43:39 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-d[68171]: starting mds.d at 2024-09-10T08:43:39.582 INFO:journalctl@ceph.mds.d.smithi172.stdout:Sep 10 08:43:39 smithi172 bash[68155]: 50cb24e942a546d1f0022ad9aae7418514c52c0f983d04166b736bf6eafb7024 2024-09-10T08:43:39.583 INFO:journalctl@ceph.mds.d.smithi172.stdout:Sep 10 08:43:39 smithi172 systemd[1]: Started Ceph mds.d for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:43:39.610 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:40.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.080+0000 7f3401452640 1 -- 172.21.15.172:0/1825526240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33fc106f20 msgr2=0x7f33fc10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:40.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.080+0000 7f3401452640 1 --2- 172.21.15.172:0/1825526240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33fc106f20 0x7f33fc10eaa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009900 tx=0x7f33f402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:43:40.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.080+0000 7f3401452640 1 -- 172.21.15.172:0/1825526240 shutdown_connections 2024-09-10T08:43:40.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.080+0000 7f3401452640 1 --2- 172.21.15.172:0/1825526240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33fc106f20 0x7f33fc10eaa0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:40.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.080+0000 7f3401452640 1 --2- 172.21.15.172:0/1825526240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33fc1027a0 0x7f33fc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:40.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.080+0000 7f3401452640 1 --2- 172.21.15.172:0/1825526240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33fc101df0 0x7f33fc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:40.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.080+0000 7f3401452640 1 -- 172.21.15.172:0/1825526240 >> 172.21.15.172:0/1825526240 conn(0x7f33fc0fd8f0 msgr2=0x7f33fc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:40.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.080+0000 7f3401452640 1 -- 172.21.15.172:0/1825526240 shutdown_connections 2024-09-10T08:43:40.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.080+0000 7f3401452640 1 -- 172.21.15.172:0/1825526240 wait complete. 2024-09-10T08:43:40.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.080+0000 7f3401452640 1 Processor -- start 2024-09-10T08:43:40.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f3401452640 1 -- start start 2024-09-10T08:43:40.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f3401452640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33fc101df0 0x7f33fc1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:40.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f3401452640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33fc1027a0 0x7f33fc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:40.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f3401452640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33fc106f20 0x7f33fc19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:40.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f3401452640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc111e30 con 0x7f33fc101df0 2024-09-10T08:43:40.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f3401452640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc111cb0 con 0x7f33fc106f20 2024-09-10T08:43:40.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f3401452640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc111fb0 con 0x7f33fc1027a0 2024-09-10T08:43:40.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f33faffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33fc101df0 0x7f33fc1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:40.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f33faffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33fc101df0 0x7f33fc1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59084/0 (socket says 172.21.15.172:59084) 2024-09-10T08:43:40.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f33faffd640 1 -- 172.21.15.172:0/990956976 learned_addr learned my addr 172.21.15.172:0/990956976 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:43:40.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f33fb7fe640 1 --2- 172.21.15.172:0/990956976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33fc106f20 0x7f33fc19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:40.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f33fa7fc640 1 --2- 172.21.15.172:0/990956976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33fc1027a0 0x7f33fc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:40.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f33faffd640 1 -- 172.21.15.172:0/990956976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33fc1027a0 msgr2=0x7f33fc198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:40.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f33faffd640 1 --2- 172.21.15.172:0/990956976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33fc1027a0 0x7f33fc198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:40.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f33faffd640 1 -- 172.21.15.172:0/990956976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33fc106f20 msgr2=0x7f33fc19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:40.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f33faffd640 1 --2- 172.21.15.172:0/990956976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33fc106f20 0x7f33fc19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:40.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f33faffd640 1 -- 172.21.15.172:0/990956976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33fc19d1a0 con 0x7f33fc101df0 2024-09-10T08:43:40.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f33fb7fe640 1 --2- 172.21.15.172:0/990956976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33fc106f20 0x7f33fc19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:43:40.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.081+0000 7f33faffd640 1 --2- 172.21.15.172:0/990956976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33fc101df0 0x7f33fc1981f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00cce0 tx=0x7f33ec00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:40.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.082+0000 7f33e3fff640 1 -- 172.21.15.172:0/990956976 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec00f650 con 0x7f33fc101df0 2024-09-10T08:43:40.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.082+0000 7f3401452640 1 -- 172.21.15.172:0/990956976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33fc19d490 con 0x7f33fc101df0 2024-09-10T08:43:40.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.082+0000 7f33e3fff640 1 -- 172.21.15.172:0/990956976 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33ec00fca0 con 0x7f33fc101df0 2024-09-10T08:43:40.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.082+0000 7f33e3fff640 1 -- 172.21.15.172:0/990956976 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec018b90 con 0x7f33fc101df0 2024-09-10T08:43:40.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.082+0000 7f3401452640 1 -- 172.21.15.172:0/990956976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33fc1a4d70 con 0x7f33fc101df0 2024-09-10T08:43:40.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.083+0000 7f3401452640 1 -- 172.21.15.172:0/990956976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33fc1021d0 con 0x7f33fc101df0 2024-09-10T08:43:40.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.084+0000 7f33e3fff640 1 -- 172.21.15.172:0/990956976 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f33ec00f7f0 con 0x7f33fc101df0 2024-09-10T08:43:40.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.084+0000 7f33e3fff640 1 --2- 172.21.15.172:0/990956976 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f33e40a2b80 0x7f33e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:40.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.084+0000 7f33fa7fc640 1 --2- 172.21.15.172:0/990956976 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f33e40a2b80 0x7f33e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:40.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.084+0000 7f33e3fff640 1 -- 172.21.15.172:0/990956976 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f33ec0cf130 con 0x7f33fc101df0 2024-09-10T08:43:40.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.085+0000 7f33fa7fc640 1 --2- 172.21.15.172:0/990956976 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f33e40a2b80 0x7f33e40a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0fe1a0 tx=0x7f33f0005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:40.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.089+0000 7f33e3fff640 1 -- 172.21.15.172:0/990956976 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f33ec014030 con 0x7f33fc101df0 2024-09-10T08:43:40.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.293+0000 7f3401452640 1 -- 172.21.15.172:0/990956976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33fc10a1c0 con 0x7f33fc101df0 2024-09-10T08:43:40.294 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:40 smithi172 ceph-mon[30854]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:40.294 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:40.294 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:40.294 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:40.294 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:40.294 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:40.294 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:40 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:40.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.293+0000 7f33e3fff640 1 -- 172.21.15.172:0/990956976 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3922 (secure 0 0 0) 0x7f33ec09c530 con 0x7f33fc101df0 2024-09-10T08:43:40.295 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:43:40.295 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":5,"btime":"2024-09-10T08:43:40:183439+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T08:43:40.295 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 5 2024-09-10T08:43:40.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.295+0000 7f3401452640 1 -- 172.21.15.172:0/990956976 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f33e40a2b80 msgr2=0x7f33e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:40.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.295+0000 7f3401452640 1 --2- 172.21.15.172:0/990956976 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f33e40a2b80 0x7f33e40a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0fe1a0 tx=0x7f33f0005c50 comp rx=0 tx=0).stop 2024-09-10T08:43:40.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.295+0000 7f3401452640 1 -- 172.21.15.172:0/990956976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33fc101df0 msgr2=0x7f33fc1981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:40.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.295+0000 7f3401452640 1 --2- 172.21.15.172:0/990956976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33fc101df0 0x7f33fc1981f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00cce0 tx=0x7f33ec00a620 comp rx=0 tx=0).stop 2024-09-10T08:43:40.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.296+0000 7f3401452640 1 -- 172.21.15.172:0/990956976 shutdown_connections 2024-09-10T08:43:40.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.296+0000 7f3401452640 1 --2- 172.21.15.172:0/990956976 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f33e40a2b80 0x7f33e40a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:40.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.296+0000 7f3401452640 1 --2- 172.21.15.172:0/990956976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33fc106f20 0x7f33fc19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:40.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.296+0000 7f3401452640 1 --2- 172.21.15.172:0/990956976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33fc1027a0 0x7f33fc198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:40.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.296+0000 7f3401452640 1 --2- 172.21.15.172:0/990956976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33fc101df0 0x7f33fc1981f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:40.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.296+0000 7f3401452640 1 -- 172.21.15.172:0/990956976 >> 172.21.15.172:0/990956976 conn(0x7f33fc0fd8f0 msgr2=0x7f33fc0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:40.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.296+0000 7f3401452640 1 -- 172.21.15.172:0/990956976 shutdown_connections 2024-09-10T08:43:40.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:40.296+0000 7f3401452640 1 -- 172.21.15.172:0/990956976 wait complete. 2024-09-10T08:43:40.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:40 smithi174 ceph-mon[34068]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:40.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:40 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:40.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:40 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:40.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:40 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:40.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:40 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:40.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:40 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:40.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:40 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:40 smithi191 ceph-mon[35314]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:40 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:40 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:40 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:40 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:40 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:40 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:41.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:41 smithi172 ceph-mon[30854]: Deploying daemon mds.e on smithi174 2024-09-10T08:43:41.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:41 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.172:6836/2122290739,v1:172.21.15.172:6837/2122290739] up:boot 2024-09-10T08:43:41.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:41 smithi172 ceph-mon[30854]: fsmap 4 up:standby 2024-09-10T08:43:41.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:41 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T08:43:41.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:41 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/990956976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:41.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:41 smithi174 ceph-mon[34068]: Deploying daemon mds.e on smithi174 2024-09-10T08:43:41.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:41 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.172:6836/2122290739,v1:172.21.15.172:6837/2122290739] up:boot 2024-09-10T08:43:41.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:41 smithi174 ceph-mon[34068]: fsmap 4 up:standby 2024-09-10T08:43:41.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:41 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T08:43:41.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:41 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/990956976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:41.493 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 08:43:41 smithi174 systemd[1]: Starting Ceph mds.e for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T08:43:41.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:41 smithi191 ceph-mon[35314]: Deploying daemon mds.e on smithi174 2024-09-10T08:43:41.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:41 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.172:6836/2122290739,v1:172.21.15.172:6837/2122290739] up:boot 2024-09-10T08:43:41.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:41 smithi191 ceph-mon[35314]: fsmap 4 up:standby 2024-09-10T08:43:41.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:41 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T08:43:41.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:41 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/990956976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:41.923 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 08:43:41 smithi174 podman[63268]: 2024-09-10 08:43:41.549936088 +0000 UTC m=+0.016190987 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:43:41.923 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 08:43:41 smithi174 podman[63268]: 2024-09-10 08:43:41.670769594 +0000 UTC m=+0.137024525 container create 0214c70d606fc6cbad0550b07b280a718b9a92f118ccecd09451a12406d51246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T08:43:41.923 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 08:43:41 smithi174 podman[63268]: 2024-09-10 08:43:41.92375611 +0000 UTC m=+0.390010999 container init 0214c70d606fc6cbad0550b07b280a718b9a92f118ccecd09451a12406d51246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T08:43:42.207 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 08:43:41 smithi174 podman[63268]: 2024-09-10 08:43:41.9267057 +0000 UTC m=+0.392960590 container start 0214c70d606fc6cbad0550b07b280a718b9a92f118ccecd09451a12406d51246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T08:43:42.207 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 08:43:41 smithi174 ceph-mds[63287]: -- 172.21.15.174:0/2075425629 <== mon.2 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d07b2e3860 con 0x55d07b2c0800 2024-09-10T08:43:42.207 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 08:43:41 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-e[63283]: starting mds.e at 2024-09-10T08:43:42.207 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 08:43:42 smithi174 bash[63268]: 0214c70d606fc6cbad0550b07b280a718b9a92f118ccecd09451a12406d51246 2024-09-10T08:43:42.208 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 08:43:42 smithi174 systemd[1]: Started Ceph mds.e for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:43:42.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:42 smithi174 ceph-mon[34068]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:42.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:42 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:42.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:42 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:42.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:42 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:42.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:42 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:42.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:42 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:42.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:42 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:42.511 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:42 smithi191 ceph-mon[35314]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:42.512 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:42 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:42.512 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:42 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:42.512 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:42 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:42.512 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:42 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:42.512 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:42 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:42.512 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:42 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:42 smithi172 ceph-mon[30854]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:42 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:42 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:42 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:42 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:42 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:42 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:43.045 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:43:43.310 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:43.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:43 smithi172 ceph-mon[30854]: Deploying daemon mds.f on smithi191 2024-09-10T08:43:43.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:43 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.174:6834/3197756201,v1:172.21.15.174:6835/3197756201] up:boot 2024-09-10T08:43:43.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:43 smithi172 ceph-mon[30854]: fsmap 5 up:standby 2024-09-10T08:43:43.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:43 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T08:43:43.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:43 smithi174 ceph-mon[34068]: Deploying daemon mds.f on smithi191 2024-09-10T08:43:43.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:43 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.174:6834/3197756201,v1:172.21.15.174:6835/3197756201] up:boot 2024-09-10T08:43:43.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:43 smithi174 ceph-mon[34068]: fsmap 5 up:standby 2024-09-10T08:43:43.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:43 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T08:43:43.553 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:43 smithi191 ceph-mon[35314]: Deploying daemon mds.f on smithi191 2024-09-10T08:43:43.553 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:43 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.174:6834/3197756201,v1:172.21.15.174:6835/3197756201] up:boot 2024-09-10T08:43:43.553 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:43 smithi191 ceph-mon[35314]: fsmap 5 up:standby 2024-09-10T08:43:43.553 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:43 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T08:43:43.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.772+0000 7f58299c0640 1 -- 172.21.15.172:0/3750925753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5824100460 msgr2=0x7f5824100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:43.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.772+0000 7f58299c0640 1 --2- 172.21.15.172:0/3750925753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5824100460 0x7f5824100840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5814009900 tx=0x7f581402a410 comp rx=0 tx=0).stop 2024-09-10T08:43:43.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.772+0000 7f58299c0640 1 -- 172.21.15.172:0/3750925753 shutdown_connections 2024-09-10T08:43:43.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.772+0000 7f58299c0640 1 --2- 172.21.15.172:0/3750925753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5824105590 0x7f582410d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:43.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.772+0000 7f58299c0640 1 --2- 172.21.15.172:0/3750925753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5824100e10 0x7f5824104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:43.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.772+0000 7f58299c0640 1 --2- 172.21.15.172:0/3750925753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5824100460 0x7f5824100840 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:43.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.772+0000 7f58299c0640 1 -- 172.21.15.172:0/3750925753 >> 172.21.15.172:0/3750925753 conn(0x7f58240fbee0 msgr2=0x7f58240fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:43.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.772+0000 7f58299c0640 1 -- 172.21.15.172:0/3750925753 shutdown_connections 2024-09-10T08:43:43.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.773+0000 7f58299c0640 1 -- 172.21.15.172:0/3750925753 wait complete. 2024-09-10T08:43:43.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.773+0000 7f58299c0640 1 Processor -- start 2024-09-10T08:43:43.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.773+0000 7f58299c0640 1 -- start start 2024-09-10T08:43:43.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.773+0000 7f58299c0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5824100460 0x7f5824199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:43.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.773+0000 7f58299c0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5824100e10 0x7f5824199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:43.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.773+0000 7f58299c0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5824105590 0x7f58241a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:43.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.773+0000 7f58299c0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5824110d20 con 0x7f5824100e10 2024-09-10T08:43:43.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.773+0000 7f58299c0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5824110ba0 con 0x7f5824100460 2024-09-10T08:43:43.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.773+0000 7f58299c0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5824110ea0 con 0x7f5824105590 2024-09-10T08:43:43.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.774+0000 7f58227fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5824100e10 0x7f5824199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:43.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.774+0000 7f58227fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5824100e10 0x7f5824199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:59114/0 (socket says 172.21.15.172:59114) 2024-09-10T08:43:43.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.774+0000 7f58227fc640 1 -- 172.21.15.172:0/1698944734 learned_addr learned my addr 172.21.15.172:0/1698944734 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:43:43.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.774+0000 7f58237fe640 1 --2- 172.21.15.172:0/1698944734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5824105590 0x7f58241a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:43.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.774+0000 7f5822ffd640 1 --2- 172.21.15.172:0/1698944734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5824100460 0x7f5824199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:43.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.774+0000 7f58227fc640 1 -- 172.21.15.172:0/1698944734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5824105590 msgr2=0x7f58241a41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:43.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.774+0000 7f58227fc640 1 --2- 172.21.15.172:0/1698944734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5824105590 0x7f58241a41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:43.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.774+0000 7f58227fc640 1 -- 172.21.15.172:0/1698944734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5824100460 msgr2=0x7f5824199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:43.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.774+0000 7f58227fc640 1 --2- 172.21.15.172:0/1698944734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5824100460 0x7f5824199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:43.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.774+0000 7f58227fc640 1 -- 172.21.15.172:0/1698944734 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58241a46e0 con 0x7f5824100e10 2024-09-10T08:43:43.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.774+0000 7f58227fc640 1 --2- 172.21.15.172:0/1698944734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5824100e10 0x7f5824199660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f581800eb90 tx=0x7f581800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:43.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.774+0000 7f58289be640 1 -- 172.21.15.172:0/1698944734 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5818004d20 con 0x7f5824100e10 2024-09-10T08:43:43.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.774+0000 7f58289be640 1 -- 172.21.15.172:0/1698944734 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f581800ee90 con 0x7f5824100e10 2024-09-10T08:43:43.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.774+0000 7f58299c0640 1 -- 172.21.15.172:0/1698944734 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58241a48b0 con 0x7f5824100e10 2024-09-10T08:43:43.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.774+0000 7f58289be640 1 -- 172.21.15.172:0/1698944734 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5818010670 con 0x7f5824100e10 2024-09-10T08:43:43.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.774+0000 7f58299c0640 1 -- 172.21.15.172:0/1698944734 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58241a4dc0 con 0x7f5824100e10 2024-09-10T08:43:43.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.775+0000 7f58299c0640 1 -- 172.21.15.172:0/1698944734 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5824100840 con 0x7f5824100e10 2024-09-10T08:43:43.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.777+0000 7f58289be640 1 -- 172.21.15.172:0/1698944734 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5818005020 con 0x7f5824100e10 2024-09-10T08:43:43.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.777+0000 7f58289be640 1 --2- 172.21.15.172:0/1698944734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f580c0a2c00 0x7f580c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:43.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.778+0000 7f5822ffd640 1 --2- 172.21.15.172:0/1698944734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f580c0a2c00 0x7f580c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:43.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.778+0000 7f58289be640 1 -- 172.21.15.172:0/1698944734 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f58180cf240 con 0x7f5824100e10 2024-09-10T08:43:43.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.778+0000 7f5822ffd640 1 --2- 172.21.15.172:0/1698944734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f580c0a2c00 0x7f580c0a50c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5814031440 tx=0x7f58140095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:43.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.782+0000 7f58289be640 1 -- 172.21.15.172:0/1698944734 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f581809c640 con 0x7f5824100e10 2024-09-10T08:43:43.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.980+0000 7f58299c0640 1 -- 172.21.15.172:0/1698944734 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5824108770 con 0x7f5824100e10 2024-09-10T08:43:43.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.981+0000 7f58289be640 1 -- 172.21.15.172:0/1698944734 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4750 (secure 0 0 0) 0x7f581809bf40 con 0x7f5824100e10 2024-09-10T08:43:43.982 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:43:43.982 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":6,"btime":"2024-09-10T08:43:42:201554+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-10T08:43:43.982 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 6 2024-09-10T08:43:43.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.984+0000 7f58299c0640 1 -- 172.21.15.172:0/1698944734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f580c0a2c00 msgr2=0x7f580c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:43.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.984+0000 7f58299c0640 1 --2- 172.21.15.172:0/1698944734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f580c0a2c00 0x7f580c0a50c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5814031440 tx=0x7f58140095f0 comp rx=0 tx=0).stop 2024-09-10T08:43:43.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.984+0000 7f58299c0640 1 -- 172.21.15.172:0/1698944734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5824100e10 msgr2=0x7f5824199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:43.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.984+0000 7f58299c0640 1 --2- 172.21.15.172:0/1698944734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5824100e10 0x7f5824199660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f581800eb90 tx=0x7f581800c6a0 comp rx=0 tx=0).stop 2024-09-10T08:43:43.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.984+0000 7f58299c0640 1 -- 172.21.15.172:0/1698944734 shutdown_connections 2024-09-10T08:43:43.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.984+0000 7f58299c0640 1 --2- 172.21.15.172:0/1698944734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f580c0a2c00 0x7f580c0a50c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:43.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.984+0000 7f58299c0640 1 --2- 172.21.15.172:0/1698944734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5824105590 0x7f58241a41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:43.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.984+0000 7f58299c0640 1 --2- 172.21.15.172:0/1698944734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5824100e10 0x7f5824199660 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:43.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.984+0000 7f58299c0640 1 --2- 172.21.15.172:0/1698944734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5824100460 0x7f5824199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:43.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.984+0000 7f58299c0640 1 -- 172.21.15.172:0/1698944734 >> 172.21.15.172:0/1698944734 conn(0x7f58240fbee0 msgr2=0x7f58240fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:43.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.984+0000 7f58299c0640 1 -- 172.21.15.172:0/1698944734 shutdown_connections 2024-09-10T08:43:43.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:43.984+0000 7f58299c0640 1 -- 172.21.15.172:0/1698944734 wait complete. 2024-09-10T08:43:44.214 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 08:43:44 smithi191 systemd[1]: Starting Ceph mds.f for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T08:43:44.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:44 smithi172 ceph-mon[30854]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:44.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:44 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1698944734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:44.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:44 smithi174 ceph-mon[34068]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:44.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:44 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1698944734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:44.548 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:44 smithi191 ceph-mon[35314]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:44.548 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:44 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1698944734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:44.549 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 08:43:44 smithi191 podman[64677]: 2024-09-10 08:43:44.163920803 +0000 UTC m=+0.015675748 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:43:44.549 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 08:43:44 smithi191 podman[64677]: 2024-09-10 08:43:44.295002094 +0000 UTC m=+0.146757025 container create 465ebd26bf991023b18ef3f5537d5365ab637e831fb4c92113cdeff23df94bff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T08:43:44.549 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 08:43:44 smithi191 podman[64677]: 2024-09-10 08:43:44.54896896 +0000 UTC m=+0.400723893 container init 465ebd26bf991023b18ef3f5537d5365ab637e831fb4c92113cdeff23df94bff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-f, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS) 2024-09-10T08:43:44.800 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 08:43:44 smithi191 podman[64677]: 2024-09-10 08:43:44.55168853 +0000 UTC m=+0.403443479 container start 465ebd26bf991023b18ef3f5537d5365ab637e831fb4c92113cdeff23df94bff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True) 2024-09-10T08:43:44.800 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 08:43:44 smithi191 ceph-mds[64697]: -- 172.21.15.191:0/2438288090 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55839b049860 con 0x55839b027400 2024-09-10T08:43:44.800 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 08:43:44 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-f[64693]: starting mds.f at 2024-09-10T08:43:44.800 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 08:43:44 smithi191 bash[64677]: 465ebd26bf991023b18ef3f5537d5365ab637e831fb4c92113cdeff23df94bff 2024-09-10T08:43:44.801 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 08:43:44 smithi191 systemd[1]: Started Ceph mds.f for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:43:45.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:45.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:45.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:45.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:45.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:45.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:45.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:45 smithi174 ceph-mon[34068]: Deploying daemon mds.c on smithi191 2024-09-10T08:43:45.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:45 smithi174 ceph-mon[34068]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:45.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:45 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.191:6834/1774553543,v1:172.21.15.191:6835/1774553543] up:boot 2024-09-10T08:43:45.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:45 smithi174 ceph-mon[34068]: fsmap 6 up:standby 2024-09-10T08:43:45.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T08:43:46.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:46.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:46.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:46.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:46.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:46.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:46.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:45 smithi191 ceph-mon[35314]: Deploying daemon mds.c on smithi191 2024-09-10T08:43:46.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:45 smithi191 ceph-mon[35314]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:46.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:45 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.191:6834/1774553543,v1:172.21.15.191:6835/1774553543] up:boot 2024-09-10T08:43:46.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:45 smithi191 ceph-mon[35314]: fsmap 6 up:standby 2024-09-10T08:43:46.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T08:43:46.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:46.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:46.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:46.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:46.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:46.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:46.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:45 smithi172 ceph-mon[30854]: Deploying daemon mds.c on smithi191 2024-09-10T08:43:46.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:45 smithi172 ceph-mon[30854]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:46.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:45 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.191:6834/1774553543,v1:172.21.15.191:6835/1774553543] up:boot 2024-09-10T08:43:46.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:45 smithi172 ceph-mon[30854]: fsmap 6 up:standby 2024-09-10T08:43:46.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T08:43:46.658 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:43:46.800 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 08:43:46 smithi191 systemd[1]: Starting Ceph mds.c for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T08:43:46.920 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:47.137 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 08:43:46 smithi191 podman[64975]: 2024-09-10 08:43:46.776932652 +0000 UTC m=+0.016055718 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:43:47.137 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 08:43:46 smithi191 podman[64975]: 2024-09-10 08:43:46.891824618 +0000 UTC m=+0.130947728 container create b8308ec924a8a0584993412fc3c47e807510c8717b37cc2a2c3f02c546439540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T08:43:47.137 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 08:43:47 smithi191 podman[64975]: 2024-09-10 08:43:47.137274931 +0000 UTC m=+0.376397996 container init b8308ec924a8a0584993412fc3c47e807510c8717b37cc2a2c3f02c546439540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-c, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T08:43:47.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.397+0000 7fca3f552640 1 -- 172.21.15.172:0/2122806601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38102570 msgr2=0x7fca381029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:47.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.397+0000 7fca3f552640 1 --2- 172.21.15.172:0/2122806601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38102570 0x7fca381029b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009900 tx=0x7fca2c02a480 comp rx=0 tx=0).stop 2024-09-10T08:43:47.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.397+0000 7fca3f552640 1 -- 172.21.15.172:0/2122806601 shutdown_connections 2024-09-10T08:43:47.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.397+0000 7fca3f552640 1 --2- 172.21.15.172:0/2122806601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca38102ef0 0x7fca3810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:47.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.397+0000 7fca3f552640 1 --2- 172.21.15.172:0/2122806601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38102570 0x7fca381029b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:47.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.397+0000 7fca3f552640 1 --2- 172.21.15.172:0/2122806601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca38100e20 0x7fca38101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:47.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.397+0000 7fca3f552640 1 -- 172.21.15.172:0/2122806601 >> 172.21.15.172:0/2122806601 conn(0x7fca380fc7d0 msgr2=0x7fca380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:47.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.397+0000 7fca3f552640 1 -- 172.21.15.172:0/2122806601 shutdown_connections 2024-09-10T08:43:47.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.397+0000 7fca3f552640 1 -- 172.21.15.172:0/2122806601 wait complete. 2024-09-10T08:43:47.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.397+0000 7fca3f552640 1 Processor -- start 2024-09-10T08:43:47.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.397+0000 7fca3f552640 1 -- start start 2024-09-10T08:43:47.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3f552640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca38100e20 0x7fca380fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:47.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3f552640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38102570 0x7fca380fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:47.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3f552640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca38102ef0 0x7fca380fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:47.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3f552640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fca38110e30 con 0x7fca38102570 2024-09-10T08:43:47.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3f552640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca38110cb0 con 0x7fca38102ef0 2024-09-10T08:43:47.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3f552640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fca38110fb0 con 0x7fca38100e20 2024-09-10T08:43:47.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3caed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38102570 0x7fca380fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:47.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3caed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38102570 0x7fca380fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43362/0 (socket says 172.21.15.172:43362) 2024-09-10T08:43:47.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3caed640 1 -- 172.21.15.172:0/370905682 learned_addr learned my addr 172.21.15.172:0/370905682 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:43:47.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3daef640 1 --2- 172.21.15.172:0/370905682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca38102ef0 0x7fca380fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:47.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3d2ee640 1 --2- 172.21.15.172:0/370905682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca38100e20 0x7fca380fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:47.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3caed640 1 -- 172.21.15.172:0/370905682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca38100e20 msgr2=0x7fca380fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:47.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3caed640 1 --2- 172.21.15.172:0/370905682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca38100e20 0x7fca380fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:47.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3caed640 1 -- 172.21.15.172:0/370905682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca38102ef0 msgr2=0x7fca380fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:47.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3caed640 1 --2- 172.21.15.172:0/370905682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca38102ef0 0x7fca380fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:47.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3caed640 1 -- 172.21.15.172:0/370905682 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca381a68e0 con 0x7fca38102570 2024-09-10T08:43:47.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3d2ee640 1 --2- 172.21.15.172:0/370905682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca38100e20 0x7fca380fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:43:47.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3caed640 1 --2- 172.21.15.172:0/370905682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38102570 0x7fca380fd1c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fca2c006010 tx=0x7fca2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:47.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca367fc640 1 -- 172.21.15.172:0/370905682 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c03f030 con 0x7fca38102570 2024-09-10T08:43:47.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca367fc640 1 -- 172.21.15.172:0/370905682 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca2c003cb0 con 0x7fca38102570 2024-09-10T08:43:47.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3f552640 1 -- 172.21.15.172:0/370905682 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca381a6b70 con 0x7fca38102570 2024-09-10T08:43:47.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca367fc640 1 -- 172.21.15.172:0/370905682 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c035070 con 0x7fca38102570 2024-09-10T08:43:47.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.398+0000 7fca3f552640 1 -- 172.21.15.172:0/370905682 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca381a7050 con 0x7fca38102570 2024-09-10T08:43:47.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.399+0000 7fca3f552640 1 -- 172.21.15.172:0/370905682 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca38101200 con 0x7fca38102570 2024-09-10T08:43:47.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.400+0000 7fca367fc640 1 -- 172.21.15.172:0/370905682 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fca2c042030 con 0x7fca38102570 2024-09-10T08:43:47.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.401+0000 7fca367fc640 1 --2- 172.21.15.172:0/370905682 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca200a2b80 0x7fca200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:47.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.401+0000 7fca3d2ee640 1 --2- 172.21.15.172:0/370905682 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca200a2b80 0x7fca200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:47.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.401+0000 7fca367fc640 1 -- 172.21.15.172:0/370905682 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fca2c0ef070 con 0x7fca38102570 2024-09-10T08:43:47.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.402+0000 7fca3d2ee640 1 --2- 172.21.15.172:0/370905682 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca200a2b80 0x7fca200a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fca30005fd0 tx=0x7fca3000b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:47.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.405+0000 7fca367fc640 1 -- 172.21.15.172:0/370905682 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca2c0bc470 con 0x7fca38102570 2024-09-10T08:43:47.550 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 08:43:47 smithi191 podman[64975]: 2024-09-10 08:43:47.140297163 +0000 UTC m=+0.379420237 container start b8308ec924a8a0584993412fc3c47e807510c8717b37cc2a2c3f02c546439540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-10T08:43:47.550 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 08:43:47 smithi191 ceph-mds[64994]: -- 172.21.15.191:0/2807938029 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f546f0f860 con 0x55f546eed400 2024-09-10T08:43:47.550 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 08:43:47 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-c[64990]: starting mds.c at 2024-09-10T08:43:47.550 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 08:43:47 smithi191 bash[64975]: b8308ec924a8a0584993412fc3c47e807510c8717b37cc2a2c3f02c546439540 2024-09-10T08:43:47.550 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 08:43:47 smithi191 systemd[1]: Started Ceph mds.c for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:43:47.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.607+0000 7fca3f552640 1 -- 172.21.15.172:0/370905682 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca381a7250 con 0x7fca38102570 2024-09-10T08:43:47.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.607+0000 7fca367fc640 1 -- 172.21.15.172:0/370905682 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5578 (secure 0 0 0) 0x7fca2c0bbd70 con 0x7fca38102570 2024-09-10T08:43:47.608 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:43:47.608 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":7,"btime":"2024-09-10T08:43:45:213381+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-10T08:43:47.608 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 7 2024-09-10T08:43:47.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.609+0000 7fca3f552640 1 -- 172.21.15.172:0/370905682 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca200a2b80 msgr2=0x7fca200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:47.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.609+0000 7fca3f552640 1 --2- 172.21.15.172:0/370905682 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca200a2b80 0x7fca200a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fca30005fd0 tx=0x7fca3000b5f0 comp rx=0 tx=0).stop 2024-09-10T08:43:47.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.609+0000 7fca3f552640 1 -- 172.21.15.172:0/370905682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38102570 msgr2=0x7fca380fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:47.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.609+0000 7fca3f552640 1 --2- 172.21.15.172:0/370905682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38102570 0x7fca380fd1c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fca2c006010 tx=0x7fca2c0042f0 comp rx=0 tx=0).stop 2024-09-10T08:43:47.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.609+0000 7fca3f552640 1 -- 172.21.15.172:0/370905682 shutdown_connections 2024-09-10T08:43:47.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.609+0000 7fca3f552640 1 --2- 172.21.15.172:0/370905682 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca200a2b80 0x7fca200a5040 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:47.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.609+0000 7fca3f552640 1 --2- 172.21.15.172:0/370905682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca38102ef0 0x7fca380fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:47.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.609+0000 7fca3f552640 1 --2- 172.21.15.172:0/370905682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38102570 0x7fca380fd1c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:47.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.609+0000 7fca3f552640 1 --2- 172.21.15.172:0/370905682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca38100e20 0x7fca380fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:47.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.609+0000 7fca3f552640 1 -- 172.21.15.172:0/370905682 >> 172.21.15.172:0/370905682 conn(0x7fca380fc7d0 msgr2=0x7fca38069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:47.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.609+0000 7fca3f552640 1 -- 172.21.15.172:0/370905682 shutdown_connections 2024-09-10T08:43:47.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:47.609+0000 7fca3f552640 1 -- 172.21.15.172:0/370905682 wait complete. 2024-09-10T08:43:48.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:48 smithi174 ceph-mon[34068]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:48.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:48.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:48.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:48.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:48.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:48.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:48.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:48 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/370905682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:48.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:48 smithi172 ceph-mon[30854]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:48.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:48.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:48.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:48.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:48.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:48.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:48.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:48 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/370905682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:48.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:48 smithi191 ceph-mon[35314]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:48.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:48.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:48.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:48.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:48.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:48.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:48.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:48 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/370905682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:49.455 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:49 smithi174 ceph-mon[34068]: Deploying daemon mds.h on smithi174 2024-09-10T08:43:49.455 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:49 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.191:6836/1802631256,v1:172.21.15.191:6837/1802631256] up:boot 2024-09-10T08:43:49.455 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:49 smithi174 ceph-mon[34068]: fsmap 7 up:standby 2024-09-10T08:43:49.455 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:49 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T08:43:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:49 smithi191 ceph-mon[35314]: Deploying daemon mds.h on smithi174 2024-09-10T08:43:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:49 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.191:6836/1802631256,v1:172.21.15.191:6837/1802631256] up:boot 2024-09-10T08:43:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:49 smithi191 ceph-mon[35314]: fsmap 7 up:standby 2024-09-10T08:43:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:49 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T08:43:49.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:49 smithi172 ceph-mon[30854]: Deploying daemon mds.h on smithi174 2024-09-10T08:43:49.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:49 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.191:6836/1802631256,v1:172.21.15.191:6837/1802631256] up:boot 2024-09-10T08:43:49.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:49 smithi172 ceph-mon[30854]: fsmap 7 up:standby 2024-09-10T08:43:49.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:49 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T08:43:49.742 INFO:journalctl@ceph.mds.h.smithi174.stdout:Sep 10 08:43:49 smithi174 podman[63569]: 2024-09-10 08:43:49.701283044 +0000 UTC m=+0.147262961 container create 809cc2da9db78ac8eed0b8a867389a3a37aa2dab470ac3dca575a0e55e6ec5f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-h, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD) 2024-09-10T08:43:50.175 INFO:journalctl@ceph.mds.h.smithi174.stdout:Sep 10 08:43:49 smithi174 podman[63569]: 2024-09-10 08:43:49.953816683 +0000 UTC m=+0.399796601 container init 809cc2da9db78ac8eed0b8a867389a3a37aa2dab470ac3dca575a0e55e6ec5f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-h, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T08:43:50.175 INFO:journalctl@ceph.mds.h.smithi174.stdout:Sep 10 08:43:49 smithi174 podman[63569]: 2024-09-10 08:43:49.956530158 +0000 UTC m=+0.402510076 container start 809cc2da9db78ac8eed0b8a867389a3a37aa2dab470ac3dca575a0e55e6ec5f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS) 2024-09-10T08:43:50.175 INFO:journalctl@ceph.mds.h.smithi174.stdout:Sep 10 08:43:49 smithi174 ceph-mds[63588]: -- 172.21.15.174:0/815926449 <== mon.2 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f0ee169c20 con 0x55f0ee147400 2024-09-10T08:43:50.176 INFO:journalctl@ceph.mds.h.smithi174.stdout:Sep 10 08:43:49 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-h[63584]: starting mds.h at 2024-09-10T08:43:50.176 INFO:journalctl@ceph.mds.h.smithi174.stdout:Sep 10 08:43:50 smithi174 bash[63569]: 809cc2da9db78ac8eed0b8a867389a3a37aa2dab470ac3dca575a0e55e6ec5f5 2024-09-10T08:43:50.176 INFO:journalctl@ceph.mds.h.smithi174.stdout:Sep 10 08:43:50 smithi174 systemd[1]: Started Ceph mds.h for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:43:50.297 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:43:50.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:50 smithi174 ceph-mon[34068]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:50 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:50 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:50 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:50 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:50 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:50 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:50.537 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:50 smithi191 ceph-mon[35314]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:50.538 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:50 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:50.538 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:50 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:50.538 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:50 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:50.538 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:50 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:50.538 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:50 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:50.538 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:50 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:50.555 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:50.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:50 smithi172 ceph-mon[30854]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:50.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:50 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:50.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:50 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:50.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:50 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:50.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:50 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:50.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:50 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:50.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:50 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.022+0000 7f3822825640 1 -- 172.21.15.172:0/4137485255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f381c101de0 msgr2=0x7f381c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.022+0000 7f3822825640 1 --2- 172.21.15.172:0/4137485255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f381c101de0 0x7f381c1021c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f380c009900 tx=0x7f380c02a410 comp rx=0 tx=0).stop 2024-09-10T08:43:51.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.022+0000 7f3822825640 1 -- 172.21.15.172:0/4137485255 shutdown_connections 2024-09-10T08:43:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.022+0000 7f3822825640 1 --2- 172.21.15.172:0/4137485255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f381c106f10 0x7f381c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.022+0000 7f3822825640 1 --2- 172.21.15.172:0/4137485255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f381c102790 0x7f381c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.022+0000 7f3822825640 1 --2- 172.21.15.172:0/4137485255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f381c101de0 0x7f381c1021c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.022+0000 7f3822825640 1 -- 172.21.15.172:0/4137485255 >> 172.21.15.172:0/4137485255 conn(0x7f381c0fd8c0 msgr2=0x7f381c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.022+0000 7f3822825640 1 -- 172.21.15.172:0/4137485255 shutdown_connections 2024-09-10T08:43:51.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.022+0000 7f3822825640 1 -- 172.21.15.172:0/4137485255 wait complete. 2024-09-10T08:43:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f3822825640 1 Processor -- start 2024-09-10T08:43:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f3822825640 1 -- start start 2024-09-10T08:43:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f3822825640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f381c101de0 0x7f381c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f3822825640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f381c102790 0x7f381c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f3822825640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f381c106f10 0x7f381c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:51.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f3822825640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f381c111e60 con 0x7f381c101de0 2024-09-10T08:43:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f3822825640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f381c111ce0 con 0x7f381c106f10 2024-09-10T08:43:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f3822825640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f381c111fe0 con 0x7f381c102790 2024-09-10T08:43:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f381bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f381c101de0 0x7f381c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f381bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f381c101de0 0x7f381c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43368/0 (socket says 172.21.15.172:43368) 2024-09-10T08:43:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f381bfff640 1 -- 172.21.15.172:0/245264851 learned_addr learned my addr 172.21.15.172:0/245264851 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:43:51.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f381b7fe640 1 --2- 172.21.15.172:0/245264851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f381c102790 0x7f381c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:51.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f3820dc2640 1 --2- 172.21.15.172:0/245264851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f381c106f10 0x7f381c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:51.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f381bfff640 1 -- 172.21.15.172:0/245264851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f381c102790 msgr2=0x7f381c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:51.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f381bfff640 1 --2- 172.21.15.172:0/245264851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f381c102790 0x7f381c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:51.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f381bfff640 1 -- 172.21.15.172:0/245264851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f381c106f10 msgr2=0x7f381c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:51.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f381bfff640 1 --2- 172.21.15.172:0/245264851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f381c106f10 0x7f381c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:51.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f381bfff640 1 -- 172.21.15.172:0/245264851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381c1a68c0 con 0x7f381c101de0 2024-09-10T08:43:51.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.023+0000 7f381b7fe640 1 --2- 172.21.15.172:0/245264851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f381c102790 0x7f381c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:43:51.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.024+0000 7f381bfff640 1 --2- 172.21.15.172:0/245264851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f381c101de0 0x7f381c0744f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f380c006010 tx=0x7f380c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:51.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.024+0000 7f38197fa640 1 -- 172.21.15.172:0/245264851 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c03f030 con 0x7f381c101de0 2024-09-10T08:43:51.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.024+0000 7f38197fa640 1 -- 172.21.15.172:0/245264851 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f380c0035f0 con 0x7f381c101de0 2024-09-10T08:43:51.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.024+0000 7f3822825640 1 -- 172.21.15.172:0/245264851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381c1a6b50 con 0x7f381c101de0 2024-09-10T08:43:51.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.024+0000 7f38197fa640 1 -- 172.21.15.172:0/245264851 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c035070 con 0x7f381c101de0 2024-09-10T08:43:51.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.024+0000 7f3822825640 1 -- 172.21.15.172:0/245264851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f381c1a7030 con 0x7f381c101de0 2024-09-10T08:43:51.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.025+0000 7f3822825640 1 -- 172.21.15.172:0/245264851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f381c1021c0 con 0x7f381c101de0 2024-09-10T08:43:51.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.026+0000 7f38197fa640 1 -- 172.21.15.172:0/245264851 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f380c003b10 con 0x7f381c101de0 2024-09-10T08:43:51.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.027+0000 7f38197fa640 1 --2- 172.21.15.172:0/245264851 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38040a2b80 0x7f38040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:51.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.027+0000 7f381b7fe640 1 --2- 172.21.15.172:0/245264851 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38040a2b80 0x7f38040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:51.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.027+0000 7f38197fa640 1 -- 172.21.15.172:0/245264851 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f380c0ef7d0 con 0x7f381c101de0 2024-09-10T08:43:51.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.027+0000 7f381b7fe640 1 --2- 172.21.15.172:0/245264851 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38040a2b80 0x7f38040a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3810005fd0 tx=0x7f3810005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:51.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.031+0000 7f38197fa640 1 -- 172.21.15.172:0/245264851 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f380c0bcbd0 con 0x7f381c101de0 2024-09-10T08:43:51.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.234+0000 7f3822825640 1 -- 172.21.15.172:0/245264851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f381c109b60 con 0x7f381c101de0 2024-09-10T08:43:51.236 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:51 smithi172 ceph-mon[30854]: Deploying daemon mds.i on smithi191 2024-09-10T08:43:51.236 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:51 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] up:boot 2024-09-10T08:43:51.236 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:51 smithi172 ceph-mon[30854]: fsmap 8 up:standby 2024-09-10T08:43:51.236 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:51 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T08:43:51.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.235+0000 7f38197fa640 1 -- 172.21.15.172:0/245264851 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7234 (secure 0 0 0) 0x7f380c0bc4d0 con 0x7f381c101de0 2024-09-10T08:43:51.237 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:43:51.237 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":9,"btime":"2024-09-10T08:43:50:174138+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T08:43:51.237 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 9 2024-09-10T08:43:51.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.237+0000 7f3822825640 1 -- 172.21.15.172:0/245264851 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38040a2b80 msgr2=0x7f38040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:51.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.237+0000 7f3822825640 1 --2- 172.21.15.172:0/245264851 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38040a2b80 0x7f38040a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3810005fd0 tx=0x7f3810005ea0 comp rx=0 tx=0).stop 2024-09-10T08:43:51.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.237+0000 7f3822825640 1 -- 172.21.15.172:0/245264851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f381c101de0 msgr2=0x7f381c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:51.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.237+0000 7f3822825640 1 --2- 172.21.15.172:0/245264851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f381c101de0 0x7f381c0744f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f380c006010 tx=0x7f380c004270 comp rx=0 tx=0).stop 2024-09-10T08:43:51.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.237+0000 7f3822825640 1 -- 172.21.15.172:0/245264851 shutdown_connections 2024-09-10T08:43:51.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.237+0000 7f3822825640 1 --2- 172.21.15.172:0/245264851 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38040a2b80 0x7f38040a5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:51.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.237+0000 7f3822825640 1 --2- 172.21.15.172:0/245264851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f381c106f10 0x7f381c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:51.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.237+0000 7f3822825640 1 --2- 172.21.15.172:0/245264851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f381c102790 0x7f381c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:51.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.237+0000 7f3822825640 1 --2- 172.21.15.172:0/245264851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f381c101de0 0x7f381c0744f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:51.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.237+0000 7f3822825640 1 -- 172.21.15.172:0/245264851 >> 172.21.15.172:0/245264851 conn(0x7f381c0fd8c0 msgr2=0x7f381c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:51.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.237+0000 7f3822825640 1 -- 172.21.15.172:0/245264851 shutdown_connections 2024-09-10T08:43:51.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:51.237+0000 7f3822825640 1 -- 172.21.15.172:0/245264851 wait complete. 2024-09-10T08:43:51.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:51 smithi174 ceph-mon[34068]: Deploying daemon mds.i on smithi191 2024-09-10T08:43:51.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:51 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] up:boot 2024-09-10T08:43:51.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:51 smithi174 ceph-mon[34068]: fsmap 8 up:standby 2024-09-10T08:43:51.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:51 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T08:43:51.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:51 smithi191 ceph-mon[35314]: Deploying daemon mds.i on smithi191 2024-09-10T08:43:51.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:51 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] up:boot 2024-09-10T08:43:51.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:51 smithi191 ceph-mon[35314]: fsmap 8 up:standby 2024-09-10T08:43:51.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:51 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T08:43:52.191 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:52 smithi191 ceph-mon[35314]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:52.191 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:52 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/245264851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:52.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:52 smithi174 ceph-mon[34068]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:52.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:52 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/245264851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:52 smithi172 ceph-mon[30854]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:52 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/245264851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:52.800 INFO:journalctl@ceph.mds.i.smithi191.stdout:Sep 10 08:43:52 smithi191 systemd[1]: Started Ceph mds.i for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:43:53.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:53 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:53.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:53 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:53.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:53 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:53.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:53 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:53.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:53 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:53.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:53 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:53.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:53 smithi172 ceph-mon[30854]: Deploying daemon mds.g on smithi172 2024-09-10T08:43:53.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:53 smithi172 ceph-mon[30854]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:53.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:53 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] up:boot 2024-09-10T08:43:53.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:53 smithi172 ceph-mon[30854]: fsmap 9 up:standby 2024-09-10T08:43:53.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:53 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T08:43:53.886 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:43:54.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:53 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:54.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:53 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:54.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:53 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:54.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:53 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:54.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:53 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:54.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:53 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:54.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:53 smithi191 ceph-mon[35314]: Deploying daemon mds.g on smithi172 2024-09-10T08:43:54.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:53 smithi191 ceph-mon[35314]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:54.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:53 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] up:boot 2024-09-10T08:43:54.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:53 smithi191 ceph-mon[35314]: fsmap 9 up:standby 2024-09-10T08:43:54.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:53 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T08:43:54.238 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:54.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:53 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:54.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:53 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:54.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:53 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:54.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:53 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:54.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:53 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:54.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:53 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:54.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:53 smithi174 ceph-mon[34068]: Deploying daemon mds.g on smithi172 2024-09-10T08:43:54.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:53 smithi174 ceph-mon[34068]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:54.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:53 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] up:boot 2024-09-10T08:43:54.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:53 smithi174 ceph-mon[34068]: fsmap 9 up:standby 2024-09-10T08:43:54.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:53 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T08:43:54.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.822+0000 7f2d9f4a1640 1 -- 172.21.15.172:0/1833525781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d981027a0 msgr2=0x7f2d981067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:54.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.822+0000 7f2d9f4a1640 1 --2- 172.21.15.172:0/1833525781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d981027a0 0x7f2d981067f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2d94009900 tx=0x7f2d9402a410 comp rx=0 tx=0).stop 2024-09-10T08:43:54.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.822+0000 7f2d9f4a1640 1 -- 172.21.15.172:0/1833525781 shutdown_connections 2024-09-10T08:43:54.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.822+0000 7f2d9f4a1640 1 --2- 172.21.15.172:0/1833525781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d98106f20 0x7f2d9810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:54.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.822+0000 7f2d9f4a1640 1 --2- 172.21.15.172:0/1833525781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d981027a0 0x7f2d981067f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:54.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.822+0000 7f2d9f4a1640 1 --2- 172.21.15.172:0/1833525781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d98101df0 0x7f2d981021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:54.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.822+0000 7f2d9f4a1640 1 -- 172.21.15.172:0/1833525781 >> 172.21.15.172:0/1833525781 conn(0x7f2d980fd8b0 msgr2=0x7f2d980ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:54.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.822+0000 7f2d9f4a1640 1 -- 172.21.15.172:0/1833525781 shutdown_connections 2024-09-10T08:43:54.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.822+0000 7f2d9f4a1640 1 -- 172.21.15.172:0/1833525781 wait complete. 2024-09-10T08:43:54.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.823+0000 7f2d9f4a1640 1 Processor -- start 2024-09-10T08:43:54.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.823+0000 7f2d9f4a1640 1 -- start start 2024-09-10T08:43:54.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.823+0000 7f2d9f4a1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d98101df0 0x7f2d981982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:54.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.823+0000 7f2d9f4a1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d981027a0 0x7f2d981987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:54.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.823+0000 7f2d9f4a1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d98106f20 0x7f2d9819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:54.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.823+0000 7f2d9f4a1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98111e30 con 0x7f2d98101df0 2024-09-10T08:43:54.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.823+0000 7f2d9f4a1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98111cb0 con 0x7f2d981027a0 2024-09-10T08:43:54.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.823+0000 7f2d9f4a1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98111fb0 con 0x7f2d98106f20 2024-09-10T08:43:54.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.823+0000 7f2d9d23d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d98101df0 0x7f2d981982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:54.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.823+0000 7f2d9d23d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d98101df0 0x7f2d981982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43374/0 (socket says 172.21.15.172:43374) 2024-09-10T08:43:54.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.823+0000 7f2d9d23d640 1 -- 172.21.15.172:0/2591841789 learned_addr learned my addr 172.21.15.172:0/2591841789 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:43:54.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.824+0000 7f2d9ca3c640 1 --2- 172.21.15.172:0/2591841789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d981027a0 0x7f2d981987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:54.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.824+0000 7f2d9d23d640 1 -- 172.21.15.172:0/2591841789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d98106f20 msgr2=0x7f2d9819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:54.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.824+0000 7f2d9d23d640 1 --2- 172.21.15.172:0/2591841789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d98106f20 0x7f2d9819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:54.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.824+0000 7f2d9d23d640 1 -- 172.21.15.172:0/2591841789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d981027a0 msgr2=0x7f2d981987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:54.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.824+0000 7f2d9d23d640 1 --2- 172.21.15.172:0/2591841789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d981027a0 0x7f2d981987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:54.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.824+0000 7f2d9d23d640 1 -- 172.21.15.172:0/2591841789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9819d260 con 0x7f2d98101df0 2024-09-10T08:43:54.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.824+0000 7f2d9ca3c640 1 --2- 172.21.15.172:0/2591841789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d981027a0 0x7f2d981987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:43:54.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.824+0000 7f2d9d23d640 1 --2- 172.21.15.172:0/2591841789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d98101df0 0x7f2d981982b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800d950 tx=0x7f2d8800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:54.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.824+0000 7f2d8e7fc640 1 -- 172.21.15.172:0/2591841789 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88017070 con 0x7f2d98101df0 2024-09-10T08:43:54.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.824+0000 7f2d9f4a1640 1 -- 172.21.15.172:0/2591841789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9819d550 con 0x7f2d98101df0 2024-09-10T08:43:54.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.824+0000 7f2d8e7fc640 1 -- 172.21.15.172:0/2591841789 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d88009c30 con 0x7f2d98101df0 2024-09-10T08:43:54.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.824+0000 7f2d8e7fc640 1 -- 172.21.15.172:0/2591841789 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8800f620 con 0x7f2d98101df0 2024-09-10T08:43:54.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.824+0000 7f2d9f4a1640 1 -- 172.21.15.172:0/2591841789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d981a4e30 con 0x7f2d98101df0 2024-09-10T08:43:54.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.825+0000 7f2d9f4a1640 1 -- 172.21.15.172:0/2591841789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d981021d0 con 0x7f2d98101df0 2024-09-10T08:43:54.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.827+0000 7f2d8e7fc640 1 -- 172.21.15.172:0/2591841789 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2d880109a0 con 0x7f2d98101df0 2024-09-10T08:43:54.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.827+0000 7f2d8e7fc640 1 --2- 172.21.15.172:0/2591841789 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2d840a2b80 0x7f2d840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:54.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.827+0000 7f2d9ca3c640 1 --2- 172.21.15.172:0/2591841789 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2d840a2b80 0x7f2d840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:54.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.828+0000 7f2d8e7fc640 1 -- 172.21.15.172:0/2591841789 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f2d880ce8a0 con 0x7f2d98101df0 2024-09-10T08:43:54.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.828+0000 7f2d9ca3c640 1 --2- 172.21.15.172:0/2591841789 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2d840a2b80 0x7f2d840a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d94000c00 tx=0x7f2d94031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:54.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:54.831+0000 7f2d8e7fc640 1 -- 172.21.15.172:0/2591841789 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d88009de0 con 0x7f2d98101df0 2024-09-10T08:43:55.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:55.049+0000 7f2d9f4a1640 1 -- 172.21.15.172:0/2591841789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d9802d7c0 con 0x7f2d98101df0 2024-09-10T08:43:55.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:55.050+0000 7f2d8e7fc640 1 -- 172.21.15.172:0/2591841789 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8064 (secure 0 0 0) 0x7f2d8809bca0 con 0x7f2d98101df0 2024-09-10T08:43:55.052 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:43:55.052 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":10,"btime":"2024-09-10T08:43:53:190853+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T08:43:55.052 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 10 2024-09-10T08:43:55.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:55.053+0000 7f2d9f4a1640 1 -- 172.21.15.172:0/2591841789 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2d840a2b80 msgr2=0x7f2d840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:55.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:55.053+0000 7f2d9f4a1640 1 --2- 172.21.15.172:0/2591841789 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2d840a2b80 0x7f2d840a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d94000c00 tx=0x7f2d94031400 comp rx=0 tx=0).stop 2024-09-10T08:43:55.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:55.053+0000 7f2d9f4a1640 1 -- 172.21.15.172:0/2591841789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d98101df0 msgr2=0x7f2d981982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:55.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:55.053+0000 7f2d9f4a1640 1 --2- 172.21.15.172:0/2591841789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d98101df0 0x7f2d981982b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800d950 tx=0x7f2d8800de10 comp rx=0 tx=0).stop 2024-09-10T08:43:55.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:55.053+0000 7f2d9f4a1640 1 -- 172.21.15.172:0/2591841789 shutdown_connections 2024-09-10T08:43:55.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:55.053+0000 7f2d9f4a1640 1 --2- 172.21.15.172:0/2591841789 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2d840a2b80 0x7f2d840a5040 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:55.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:55.053+0000 7f2d9f4a1640 1 --2- 172.21.15.172:0/2591841789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d98106f20 0x7f2d9819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:55.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:55.053+0000 7f2d9f4a1640 1 --2- 172.21.15.172:0/2591841789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d981027a0 0x7f2d981987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:55.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:55.053+0000 7f2d9f4a1640 1 --2- 172.21.15.172:0/2591841789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d98101df0 0x7f2d981982b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:55.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:55.053+0000 7f2d9f4a1640 1 -- 172.21.15.172:0/2591841789 >> 172.21.15.172:0/2591841789 conn(0x7f2d980fd8b0 msgr2=0x7f2d980fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:55.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:55.053+0000 7f2d9f4a1640 1 -- 172.21.15.172:0/2591841789 shutdown_connections 2024-09-10T08:43:55.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:55.053+0000 7f2d9f4a1640 1 -- 172.21.15.172:0/2591841789 wait complete. 2024-09-10T08:43:55.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:55 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2591841789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:55.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:55 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2591841789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:55.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:55 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2591841789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:56.110 INFO:journalctl@ceph.mds.g.smithi172.stdout:Sep 10 08:43:56 smithi172 systemd[1]: Started Ceph mds.g for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:43:56.393 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:56 smithi174 ceph-mon[34068]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:56.451 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:56 smithi191 ceph-mon[35314]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:56.545 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:56 smithi172 ceph-mon[30854]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:57.391 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:57 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.172:6838/1621056220,v1:172.21.15.172:6839/1621056220] up:boot 2024-09-10T08:43:57.391 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:57 smithi172 ceph-mon[30854]: fsmap 10 up:standby 2024-09-10T08:43:57.392 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T08:43:57.392 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:57.392 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:57.392 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:57.392 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:57.392 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:57.392 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:57 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:57.392 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:57 smithi172 ceph-mon[30854]: Deploying daemon mds.a on smithi172 2024-09-10T08:43:57.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:57 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.172:6838/1621056220,v1:172.21.15.172:6839/1621056220] up:boot 2024-09-10T08:43:57.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:57 smithi174 ceph-mon[34068]: fsmap 10 up:standby 2024-09-10T08:43:57.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T08:43:57.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:57.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:57.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:57.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:57.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:57.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:57 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:57.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:57 smithi174 ceph-mon[34068]: Deploying daemon mds.a on smithi172 2024-09-10T08:43:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:57 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.172:6838/1621056220,v1:172.21.15.172:6839/1621056220] up:boot 2024-09-10T08:43:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:57 smithi191 ceph-mon[35314]: fsmap 10 up:standby 2024-09-10T08:43:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T08:43:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:57.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:57.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:57.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:57 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:57.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:57 smithi191 ceph-mon[35314]: Deploying daemon mds.a on smithi172 2024-09-10T08:43:57.883 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:43:58.237 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:58 smithi172 ceph-mon[30854]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:58.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:58 smithi174 ceph-mon[34068]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:58 smithi191 ceph-mon[35314]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:58.662 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:43:58.786 INFO:journalctl@ceph.mds.a.smithi172.stdout:Sep 10 08:43:58 smithi172 systemd[1]: Started Ceph mds.a for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:43:59.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.163+0000 7fb3dfbb3640 1 -- 172.21.15.172:0/2523739953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d8102790 msgr2=0x7fb3d81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:59.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.163+0000 7fb3dfbb3640 1 --2- 172.21.15.172:0/2523739953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d8102790 0x7fb3d81067e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009900 tx=0x7fb3c802a410 comp rx=0 tx=0).stop 2024-09-10T08:43:59.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.163+0000 7fb3dfbb3640 1 -- 172.21.15.172:0/2523739953 shutdown_connections 2024-09-10T08:43:59.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.163+0000 7fb3dfbb3640 1 --2- 172.21.15.172:0/2523739953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3d8106f10 0x7fb3d810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:59.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.163+0000 7fb3dfbb3640 1 --2- 172.21.15.172:0/2523739953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d8102790 0x7fb3d81067e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:59.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.163+0000 7fb3dfbb3640 1 --2- 172.21.15.172:0/2523739953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3d8101de0 0x7fb3d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:59.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.163+0000 7fb3dfbb3640 1 -- 172.21.15.172:0/2523739953 >> 172.21.15.172:0/2523739953 conn(0x7fb3d80fd8c0 msgr2=0x7fb3d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:59.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.164+0000 7fb3dfbb3640 1 -- 172.21.15.172:0/2523739953 shutdown_connections 2024-09-10T08:43:59.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.164+0000 7fb3dfbb3640 1 -- 172.21.15.172:0/2523739953 wait complete. 2024-09-10T08:43:59.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.164+0000 7fb3dfbb3640 1 Processor -- start 2024-09-10T08:43:59.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.164+0000 7fb3dfbb3640 1 -- start start 2024-09-10T08:43:59.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.164+0000 7fb3dfbb3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3d8101de0 0x7fb3d8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:59.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.164+0000 7fb3dfbb3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d8102790 0x7fb3d8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:59.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.164+0000 7fb3dfbb3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3d8106f10 0x7fb3d819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:59.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.164+0000 7fb3dfbb3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8111e40 con 0x7fb3d8102790 2024-09-10T08:43:59.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.164+0000 7fb3dfbb3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8111cc0 con 0x7fb3d8106f10 2024-09-10T08:43:59.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.164+0000 7fb3dfbb3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8111fc0 con 0x7fb3d8101de0 2024-09-10T08:43:59.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.164+0000 7fb3dd14e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d8102790 0x7fb3d8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:59.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.164+0000 7fb3dd14e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d8102790 0x7fb3d8198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34022/0 (socket says 172.21.15.172:34022) 2024-09-10T08:43:59.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.165+0000 7fb3dd14e640 1 -- 172.21.15.172:0/1598948699 learned_addr learned my addr 172.21.15.172:0/1598948699 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:43:59.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.165+0000 7fb3dd94f640 1 --2- 172.21.15.172:0/1598948699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3d8101de0 0x7fb3d8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:59.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.165+0000 7fb3de150640 1 --2- 172.21.15.172:0/1598948699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3d8106f10 0x7fb3d819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:59.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.165+0000 7fb3dd14e640 1 -- 172.21.15.172:0/1598948699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3d8101de0 msgr2=0x7fb3d8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:59.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.165+0000 7fb3dd14e640 1 --2- 172.21.15.172:0/1598948699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3d8101de0 0x7fb3d8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:59.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.165+0000 7fb3dd14e640 1 -- 172.21.15.172:0/1598948699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3d8106f10 msgr2=0x7fb3d819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:59.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.165+0000 7fb3dd14e640 1 --2- 172.21.15.172:0/1598948699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3d8106f10 0x7fb3d819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:59.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.165+0000 7fb3dd14e640 1 -- 172.21.15.172:0/1598948699 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d819d3f0 con 0x7fb3d8102790 2024-09-10T08:43:59.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.165+0000 7fb3dd14e640 1 --2- 172.21.15.172:0/1598948699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d8102790 0x7fb3d8198980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8005940 tx=0x7fb3c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:59.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.165+0000 7fb3ceffd640 1 -- 172.21.15.172:0/1598948699 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c803f030 con 0x7fb3d8102790 2024-09-10T08:43:59.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.165+0000 7fb3ceffd640 1 -- 172.21.15.172:0/1598948699 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3c8003aa0 con 0x7fb3d8102790 2024-09-10T08:43:59.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.165+0000 7fb3dfbb3640 1 -- 172.21.15.172:0/1598948699 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d819d680 con 0x7fb3d8102790 2024-09-10T08:43:59.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.165+0000 7fb3ceffd640 1 -- 172.21.15.172:0/1598948699 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8035070 con 0x7fb3d8102790 2024-09-10T08:43:59.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.165+0000 7fb3dfbb3640 1 -- 172.21.15.172:0/1598948699 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d81a4fc0 con 0x7fb3d8102790 2024-09-10T08:43:59.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.166+0000 7fb3dfbb3640 1 -- 172.21.15.172:0/1598948699 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d81021c0 con 0x7fb3d8102790 2024-09-10T08:43:59.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.167+0000 7fb3ceffd640 1 -- 172.21.15.172:0/1598948699 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb3c8042030 con 0x7fb3d8102790 2024-09-10T08:43:59.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.168+0000 7fb3ceffd640 1 --2- 172.21.15.172:0/1598948699 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb3c40a2c00 0x7fb3c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:43:59.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.168+0000 7fb3dd94f640 1 --2- 172.21.15.172:0/1598948699 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb3c40a2c00 0x7fb3c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:43:59.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.168+0000 7fb3ceffd640 1 -- 172.21.15.172:0/1598948699 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fb3c80ef2e0 con 0x7fb3d8102790 2024-09-10T08:43:59.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.168+0000 7fb3dd94f640 1 --2- 172.21.15.172:0/1598948699 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb3c40a2c00 0x7fb3c40a50c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009990 tx=0x7fb3d4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:43:59.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.172+0000 7fb3ceffd640 1 -- 172.21.15.172:0/1598948699 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb3c80bc6e0 con 0x7fb3d8102790 2024-09-10T08:43:59.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.373+0000 7fb3dfbb3640 1 -- 172.21.15.172:0/1598948699 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3d8109b60 con 0x7fb3d8102790 2024-09-10T08:43:59.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.374+0000 7fb3ceffd640 1 -- 172.21.15.172:0/1598948699 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9719 (secure 0 0 0) 0x7fb3c80bbfe0 con 0x7fb3d8102790 2024-09-10T08:43:59.375 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:43:59.375 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":12,"btime":"2024-09-10T08:43:59:118348+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T08:43:59.376 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 12 2024-09-10T08:43:59.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.376+0000 7fb3dfbb3640 1 -- 172.21.15.172:0/1598948699 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb3c40a2c00 msgr2=0x7fb3c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:59.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.376+0000 7fb3dfbb3640 1 --2- 172.21.15.172:0/1598948699 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb3c40a2c00 0x7fb3c40a50c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009990 tx=0x7fb3d4009110 comp rx=0 tx=0).stop 2024-09-10T08:43:59.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.376+0000 7fb3dfbb3640 1 -- 172.21.15.172:0/1598948699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d8102790 msgr2=0x7fb3d8198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:43:59.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.376+0000 7fb3dfbb3640 1 --2- 172.21.15.172:0/1598948699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d8102790 0x7fb3d8198980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8005940 tx=0x7fb3c8004270 comp rx=0 tx=0).stop 2024-09-10T08:43:59.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.376+0000 7fb3dfbb3640 1 -- 172.21.15.172:0/1598948699 shutdown_connections 2024-09-10T08:43:59.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.376+0000 7fb3dfbb3640 1 --2- 172.21.15.172:0/1598948699 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb3c40a2c00 0x7fb3c40a50c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:59.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.376+0000 7fb3dfbb3640 1 --2- 172.21.15.172:0/1598948699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3d8106f10 0x7fb3d819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:59.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.376+0000 7fb3dfbb3640 1 --2- 172.21.15.172:0/1598948699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d8102790 0x7fb3d8198980 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:59.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.376+0000 7fb3dfbb3640 1 --2- 172.21.15.172:0/1598948699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3d8101de0 0x7fb3d8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:43:59.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.376+0000 7fb3dfbb3640 1 -- 172.21.15.172:0/1598948699 >> 172.21.15.172:0/1598948699 conn(0x7fb3d80fd8c0 msgr2=0x7fb3d80fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:43:59.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.376+0000 7fb3dfbb3640 1 -- 172.21.15.172:0/1598948699 shutdown_connections 2024-09-10T08:43:59.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:43:59.376+0000 7fb3dfbb3640 1 -- 172.21.15.172:0/1598948699 wait complete. 2024-09-10T08:43:59.959 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:59 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:59.960 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:59 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:59.960 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:59 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:59.960 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:59 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:59.960 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:59 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:59.960 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:59 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:59.960 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:59 smithi172 ceph-mon[30854]: Deploying daemon mds.b on smithi174 2024-09-10T08:43:59.960 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:59 smithi172 ceph-mon[30854]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:59.960 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:59 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.172:6840/176040463,v1:172.21.15.172:6841/176040463] up:boot 2024-09-10T08:43:59.960 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:59 smithi172 ceph-mon[30854]: fsmap 11 up:standby 2024-09-10T08:43:59.960 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:59 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T08:43:59.961 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:43:59 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1598948699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:43:59.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:59 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:59.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:59 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:59.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:59 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:43:59.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:59 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:43:59.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:59 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:43:59.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:59 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:43:59.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:59 smithi174 ceph-mon[34068]: Deploying daemon mds.b on smithi174 2024-09-10T08:43:59.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:59 smithi174 ceph-mon[34068]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:43:59.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:59 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.172:6840/176040463,v1:172.21.15.172:6841/176040463] up:boot 2024-09-10T08:43:59.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:59 smithi174 ceph-mon[34068]: fsmap 11 up:standby 2024-09-10T08:43:59.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:59 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T08:43:59.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:43:59 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1598948699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:00.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:59 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:00.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:59 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:00.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:59 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:00.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:59 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T08:44:00.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:59 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T08:44:00.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:59 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:44:00.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:59 smithi191 ceph-mon[35314]: Deploying daemon mds.b on smithi174 2024-09-10T08:44:00.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:59 smithi191 ceph-mon[35314]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:00.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:59 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.172:6840/176040463,v1:172.21.15.172:6841/176040463] up:boot 2024-09-10T08:44:00.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:59 smithi191 ceph-mon[35314]: fsmap 11 up:standby 2024-09-10T08:44:00.052 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:59 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T08:44:00.052 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:43:59 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1598948699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:00.780 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 08:44:00 smithi174 systemd[1]: Starting Ceph mds.b for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T08:44:01.137 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 08:44:00 smithi174 podman[63882]: 2024-09-10 08:44:00.893434165 +0000 UTC m=+0.016194783 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T08:44:01.137 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 08:44:01 smithi174 podman[63882]: 2024-09-10 08:44:01.015053355 +0000 UTC m=+0.137813982 container create 455496ffd6ee0a4e7c4e04eb9f0238088dae30678cebe2098b501f4e29b52c46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T08:44:01.741 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 08:44:01 smithi174 podman[63882]: 2024-09-10 08:44:01.259443524 +0000 UTC m=+0.382204142 container init 455496ffd6ee0a4e7c4e04eb9f0238088dae30678cebe2098b501f4e29b52c46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-b, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T08:44:01.742 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 08:44:01 smithi174 podman[63882]: 2024-09-10 08:44:01.262222481 +0000 UTC m=+0.384983100 container start 455496ffd6ee0a4e7c4e04eb9f0238088dae30678cebe2098b501f4e29b52c46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T08:44:01.742 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 08:44:01 smithi174 ceph-mds[63901]: -- 172.21.15.174:0/4081315696 <== mon.2 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bdae737860 con 0x55bdae714800 2024-09-10T08:44:01.742 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 08:44:01 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-b[63897]: starting mds.b at 2024-09-10T08:44:01.742 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 08:44:01 smithi174 bash[63882]: 455496ffd6ee0a4e7c4e04eb9f0238088dae30678cebe2098b501f4e29b52c46 2024-09-10T08:44:01.742 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 08:44:01 smithi174 systemd[1]: Started Ceph mds.b for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T08:44:01.986 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:44:02.268 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:02 smithi172 ceph-mon[30854]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:02.268 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:02 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.268 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:02 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.268 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:02 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.268 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:02 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.268 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:02 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.268 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:02 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:44:02.268 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:02 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:02 smithi174 ceph-mon[34068]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:02.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:02 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:02 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:02 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:02 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:02 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:02 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:44:02.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:02 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:02 smithi191 ceph-mon[35314]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:02.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:02 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:02 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:02 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:02 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:02 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:02 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:44:02.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:02 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:02.654 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:03.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.131+0000 7f2171007640 1 -- 172.21.15.172:0/1834153874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216c102d40 msgr2=0x7f216c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:03.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.131+0000 7f2171007640 1 --2- 172.21.15.172:0/1834153874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216c102d40 0x7f216c103120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2164009ca0 tx=0x7f216402a950 comp rx=0 tx=0).stop 2024-09-10T08:44:03.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.131+0000 7f2171007640 1 -- 172.21.15.172:0/1834153874 shutdown_connections 2024-09-10T08:44:03.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.131+0000 7f2171007640 1 --2- 172.21.15.172:0/1834153874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f216c10c170 0x7f216c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:03.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.131+0000 7f2171007640 1 --2- 172.21.15.172:0/1834153874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f216c103660 0x7f216c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:03.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.131+0000 7f2171007640 1 --2- 172.21.15.172:0/1834153874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216c102d40 0x7f216c103120 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:03.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.131+0000 7f2171007640 1 -- 172.21.15.172:0/1834153874 >> 172.21.15.172:0/1834153874 conn(0x7f216c0fc7a0 msgr2=0x7f216c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:03.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.131+0000 7f2171007640 1 -- 172.21.15.172:0/1834153874 shutdown_connections 2024-09-10T08:44:03.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.131+0000 7f2171007640 1 -- 172.21.15.172:0/1834153874 wait complete. 2024-09-10T08:44:03.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.131+0000 7f2171007640 1 Processor -- start 2024-09-10T08:44:03.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.132+0000 7f2171007640 1 -- start start 2024-09-10T08:44:03.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.132+0000 7f2171007640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216c102d40 0x7f216c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:03.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.132+0000 7f2171007640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f216c103660 0x7f216c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:03.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.132+0000 7f2171007640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f216c10c170 0x7f216c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:03.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.132+0000 7f2171007640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f216c111e20 con 0x7f216c102d40 2024-09-10T08:44:03.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.132+0000 7f2171007640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f216c111ca0 con 0x7f216c10c170 2024-09-10T08:44:03.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.132+0000 7f2171007640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f216c111fa0 con 0x7f216c103660 2024-09-10T08:44:03.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.132+0000 7f216ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216c102d40 0x7f216c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:03.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.132+0000 7f216ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216c102d40 0x7f216c1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34040/0 (socket says 172.21.15.172:34040) 2024-09-10T08:44:03.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.132+0000 7f216ad9d640 1 -- 172.21.15.172:0/3023287040 learned_addr learned my addr 172.21.15.172:0/3023287040 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:03.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.132+0000 7f216a59c640 1 --2- 172.21.15.172:0/3023287040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f216c103660 0x7f216c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:03.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.132+0000 7f216b59e640 1 --2- 172.21.15.172:0/3023287040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f216c10c170 0x7f216c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:03.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.132+0000 7f216ad9d640 1 -- 172.21.15.172:0/3023287040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f216c103660 msgr2=0x7f216c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:03.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.132+0000 7f216ad9d640 1 --2- 172.21.15.172:0/3023287040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f216c103660 0x7f216c1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:03.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.132+0000 7f216ad9d640 1 -- 172.21.15.172:0/3023287040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f216c10c170 msgr2=0x7f216c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:03.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.132+0000 7f216ad9d640 1 --2- 172.21.15.172:0/3023287040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f216c10c170 0x7f216c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:03.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.132+0000 7f216ad9d640 1 -- 172.21.15.172:0/3023287040 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216c19d450 con 0x7f216c102d40 2024-09-10T08:44:03.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.133+0000 7f216ad9d640 1 --2- 172.21.15.172:0/3023287040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216c102d40 0x7f216c1984a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2164005cc0 tx=0x7f216402f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:03.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.133+0000 7f2163fff640 1 -- 172.21.15.172:0/3023287040 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2164034040 con 0x7f216c102d40 2024-09-10T08:44:03.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.133+0000 7f2163fff640 1 -- 172.21.15.172:0/3023287040 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21640308f0 con 0x7f216c102d40 2024-09-10T08:44:03.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.133+0000 7f2171007640 1 -- 172.21.15.172:0/3023287040 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f216c19d6e0 con 0x7f216c102d40 2024-09-10T08:44:03.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.133+0000 7f2163fff640 1 -- 172.21.15.172:0/3023287040 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21640047c0 con 0x7f216c102d40 2024-09-10T08:44:03.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.133+0000 7f2171007640 1 -- 172.21.15.172:0/3023287040 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f216c1a5020 con 0x7f216c102d40 2024-09-10T08:44:03.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.134+0000 7f2171007640 1 -- 172.21.15.172:0/3023287040 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f216c06b070 con 0x7f216c102d40 2024-09-10T08:44:03.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.135+0000 7f2163fff640 1 -- 172.21.15.172:0/3023287040 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2164037020 con 0x7f216c102d40 2024-09-10T08:44:03.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.136+0000 7f2163fff640 1 --2- 172.21.15.172:0/3023287040 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f21580a2b80 0x7f21580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:03.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.136+0000 7f216a59c640 1 --2- 172.21.15.172:0/3023287040 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f21580a2b80 0x7f21580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:03.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.136+0000 7f2163fff640 1 -- 172.21.15.172:0/3023287040 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f21640efc00 con 0x7f216c102d40 2024-09-10T08:44:03.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.136+0000 7f216a59c640 1 --2- 172.21.15.172:0/3023287040 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f21580a2b80 0x7f21580a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f216c10d760 tx=0x7f215400b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:03.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.140+0000 7f2163fff640 1 -- 172.21.15.172:0/3023287040 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f21640f6080 con 0x7f216c102d40 2024-09-10T08:44:03.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:03 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] up:boot 2024-09-10T08:44:03.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:03 smithi172 ceph-mon[30854]: fsmap 12 up:standby 2024-09-10T08:44:03.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T08:44:03.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:03.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:03.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:03.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:03 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:03.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.365+0000 7f2171007640 1 -- 172.21.15.172:0/3023287040 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f216c1071e0 con 0x7f216c102d40 2024-09-10T08:44:03.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.366+0000 7f2163fff640 1 -- 172.21.15.172:0/3023287040 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10548 (secure 0 0 0) 0x7f21640d0e20 con 0x7f216c102d40 2024-09-10T08:44:03.366 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:44:03.366 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":13,"btime":"2024-09-10T08:44:02:165641+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-10T08:44:03.367 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 13 2024-09-10T08:44:03.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.368+0000 7f2171007640 1 -- 172.21.15.172:0/3023287040 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f21580a2b80 msgr2=0x7f21580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:03.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.368+0000 7f2171007640 1 --2- 172.21.15.172:0/3023287040 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f21580a2b80 0x7f21580a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f216c10d760 tx=0x7f215400b2e0 comp rx=0 tx=0).stop 2024-09-10T08:44:03.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.368+0000 7f2171007640 1 -- 172.21.15.172:0/3023287040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216c102d40 msgr2=0x7f216c1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:03.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.368+0000 7f2171007640 1 --2- 172.21.15.172:0/3023287040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216c102d40 0x7f216c1984a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2164005cc0 tx=0x7f216402f650 comp rx=0 tx=0).stop 2024-09-10T08:44:03.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.368+0000 7f2171007640 1 -- 172.21.15.172:0/3023287040 shutdown_connections 2024-09-10T08:44:03.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.368+0000 7f2171007640 1 --2- 172.21.15.172:0/3023287040 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f21580a2b80 0x7f21580a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:03.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.368+0000 7f2171007640 1 --2- 172.21.15.172:0/3023287040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f216c10c170 0x7f216c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:03.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.368+0000 7f2171007640 1 --2- 172.21.15.172:0/3023287040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f216c103660 0x7f216c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:03.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.368+0000 7f2171007640 1 --2- 172.21.15.172:0/3023287040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f216c102d40 0x7f216c1984a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:03.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.368+0000 7f2171007640 1 -- 172.21.15.172:0/3023287040 >> 172.21.15.172:0/3023287040 conn(0x7f216c0fc7a0 msgr2=0x7f216c0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:03.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.368+0000 7f2171007640 1 -- 172.21.15.172:0/3023287040 shutdown_connections 2024-09-10T08:44:03.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:03.368+0000 7f2171007640 1 -- 172.21.15.172:0/3023287040 wait complete. 2024-09-10T08:44:03.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:03 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] up:boot 2024-09-10T08:44:03.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:03 smithi174 ceph-mon[34068]: fsmap 12 up:standby 2024-09-10T08:44:03.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:03 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T08:44:03.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:03 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:03.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:03 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:03.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:03 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:03.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:03 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:03 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] up:boot 2024-09-10T08:44:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:03 smithi191 ceph-mon[35314]: fsmap 12 up:standby 2024-09-10T08:44:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T08:44:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:03 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:03.986 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-10T08:44:04.259 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:04.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:04 smithi172 ceph-mon[30854]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:04.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:04 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3023287040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:04.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:04 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:04.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:04 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:04.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:04 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:44:04.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:04 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:44:04.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:04 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:04.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:04 smithi174 ceph-mon[34068]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:04.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:04 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3023287040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:04.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:04 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:04.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:04 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:04.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:04 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:44:04.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:04 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:44:04.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:04 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:04.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:04 smithi191 ceph-mon[35314]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:04.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:04 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3023287040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:04.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:04 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:04.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:04 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:04.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:04 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:44:04.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:04 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:44:04.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:04 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:04.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.749+0000 7f98eb82e640 1 -- 172.21.15.172:0/1628436405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98e4100e20 msgr2=0x7f98e4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:04.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.749+0000 7f98eb82e640 1 --2- 172.21.15.172:0/1628436405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98e4100e20 0x7f98e4101200 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009cd0 tx=0x7f98d402a970 comp rx=0 tx=0).stop 2024-09-10T08:44:04.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.749+0000 7f98eb82e640 1 -- 172.21.15.172:0/1628436405 shutdown_connections 2024-09-10T08:44:04.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.749+0000 7f98eb82e640 1 --2- 172.21.15.172:0/1628436405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98e4102ef0 0x7f98e410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:04.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.749+0000 7f98eb82e640 1 --2- 172.21.15.172:0/1628436405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98e4102570 0x7f98e41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:04.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.749+0000 7f98eb82e640 1 --2- 172.21.15.172:0/1628436405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98e4100e20 0x7f98e4101200 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:04.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.749+0000 7f98eb82e640 1 -- 172.21.15.172:0/1628436405 >> 172.21.15.172:0/1628436405 conn(0x7f98e40fc7d0 msgr2=0x7f98e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:04.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.749+0000 7f98eb82e640 1 -- 172.21.15.172:0/1628436405 shutdown_connections 2024-09-10T08:44:04.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.749+0000 7f98eb82e640 1 -- 172.21.15.172:0/1628436405 wait complete. 2024-09-10T08:44:04.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.750+0000 7f98eb82e640 1 Processor -- start 2024-09-10T08:44:04.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.750+0000 7f98eb82e640 1 -- start start 2024-09-10T08:44:04.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.750+0000 7f98eb82e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98e4100e20 0x7f98e4197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:04.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.750+0000 7f98eb82e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98e4102570 0x7f98e4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:04.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.750+0000 7f98eb82e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98e4102ef0 0x7f98e419bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:04.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.750+0000 7f98eb82e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4110e30 con 0x7f98e4100e20 2024-09-10T08:44:04.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.750+0000 7f98eb82e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4110cb0 con 0x7f98e4102ef0 2024-09-10T08:44:04.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.750+0000 7f98eb82e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4110fb0 con 0x7f98e4102570 2024-09-10T08:44:04.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.750+0000 7f98e95ca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98e4100e20 0x7f98e4197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:04.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.750+0000 7f98e95ca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98e4100e20 0x7f98e4197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34064/0 (socket says 172.21.15.172:34064) 2024-09-10T08:44:04.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.750+0000 7f98e95ca640 1 -- 172.21.15.172:0/445239197 learned_addr learned my addr 172.21.15.172:0/445239197 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:04.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.750+0000 7f98e8dc9640 1 --2- 172.21.15.172:0/445239197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98e4102570 0x7f98e4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:04.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.751+0000 7f98e9dcb640 1 --2- 172.21.15.172:0/445239197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98e4102ef0 0x7f98e419bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:04.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.751+0000 7f98e95ca640 1 -- 172.21.15.172:0/445239197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98e4102570 msgr2=0x7f98e4197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:04.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.751+0000 7f98e95ca640 1 --2- 172.21.15.172:0/445239197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98e4102570 0x7f98e4197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:04.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.751+0000 7f98e95ca640 1 -- 172.21.15.172:0/445239197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98e4102ef0 msgr2=0x7f98e419bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:04.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.751+0000 7f98e95ca640 1 --2- 172.21.15.172:0/445239197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98e4102ef0 0x7f98e419bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:04.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.751+0000 7f98e95ca640 1 -- 172.21.15.172:0/445239197 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e419c2e0 con 0x7f98e4100e20 2024-09-10T08:44:04.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.751+0000 7f98e9dcb640 1 --2- 172.21.15.172:0/445239197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98e4102ef0 0x7f98e419bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:44:04.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.751+0000 7f98e95ca640 1 --2- 172.21.15.172:0/445239197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98e4100e20 0x7f98e4197330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f98d4005e30 tx=0x7f98d4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:04.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.751+0000 7f98e27fc640 1 -- 172.21.15.172:0/445239197 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4047070 con 0x7f98e4100e20 2024-09-10T08:44:04.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.751+0000 7f98eb82e640 1 -- 172.21.15.172:0/445239197 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e419c570 con 0x7f98e4100e20 2024-09-10T08:44:04.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.751+0000 7f98e27fc640 1 -- 172.21.15.172:0/445239197 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98d402f830 con 0x7f98e4100e20 2024-09-10T08:44:04.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.751+0000 7f98e27fc640 1 -- 172.21.15.172:0/445239197 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4034070 con 0x7f98e4100e20 2024-09-10T08:44:04.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.751+0000 7f98eb82e640 1 -- 172.21.15.172:0/445239197 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e41a3eb0 con 0x7f98e4100e20 2024-09-10T08:44:04.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.752+0000 7f98eb82e640 1 -- 172.21.15.172:0/445239197 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e41021b0 con 0x7f98e4100e20 2024-09-10T08:44:04.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.753+0000 7f98e27fc640 1 -- 172.21.15.172:0/445239197 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f98d404a070 con 0x7f98e4100e20 2024-09-10T08:44:04.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.754+0000 7f98e27fc640 1 --2- 172.21.15.172:0/445239197 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98cc0a2b80 0x7f98cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:04.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.754+0000 7f98e8dc9640 1 --2- 172.21.15.172:0/445239197 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98cc0a2b80 0x7f98cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:04.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.754+0000 7f98e27fc640 1 -- 172.21.15.172:0/445239197 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f98d40ef790 con 0x7f98e4100e20 2024-09-10T08:44:04.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.754+0000 7f98e8dc9640 1 --2- 172.21.15.172:0/445239197 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98cc0a2b80 0x7f98cc0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f98d80095f0 tx=0x7f98d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:04.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.757+0000 7f98e27fc640 1 -- 172.21.15.172:0/445239197 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98d4051020 con 0x7f98e4100e20 2024-09-10T08:44:04.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.950+0000 7f98eb82e640 1 -- 172.21.15.172:0/445239197 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f98e4198690 con 0x7f98e4100e20 2024-09-10T08:44:04.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.951+0000 7f98e27fc640 1 -- 172.21.15.172:0/445239197 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7f98d403f070 con 0x7f98e4100e20 2024-09-10T08:44:04.951 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:44:04.951 INFO:teuthology.orchestra.run.smithi172.stdout:[ 2024-09-10T08:44:04.951 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T08:44:04.952 INFO:teuthology.orchestra.run.smithi172.stdout: "poolnum": 1, 2024-09-10T08:44:04.952 INFO:teuthology.orchestra.run.smithi172.stdout: "poolname": ".mgr" 2024-09-10T08:44:04.952 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:44:04.952 INFO:teuthology.orchestra.run.smithi172.stdout:] 2024-09-10T08:44:04.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.953+0000 7f98eb82e640 1 -- 172.21.15.172:0/445239197 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98cc0a2b80 msgr2=0x7f98cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:04.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.953+0000 7f98eb82e640 1 --2- 172.21.15.172:0/445239197 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98cc0a2b80 0x7f98cc0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f98d80095f0 tx=0x7f98d8005f90 comp rx=0 tx=0).stop 2024-09-10T08:44:04.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.953+0000 7f98eb82e640 1 -- 172.21.15.172:0/445239197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98e4100e20 msgr2=0x7f98e4197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:04.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.953+0000 7f98eb82e640 1 --2- 172.21.15.172:0/445239197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98e4100e20 0x7f98e4197330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f98d4005e30 tx=0x7f98d4030000 comp rx=0 tx=0).stop 2024-09-10T08:44:04.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.953+0000 7f98eb82e640 1 -- 172.21.15.172:0/445239197 shutdown_connections 2024-09-10T08:44:04.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.953+0000 7f98eb82e640 1 --2- 172.21.15.172:0/445239197 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98cc0a2b80 0x7f98cc0a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:04.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.953+0000 7f98eb82e640 1 --2- 172.21.15.172:0/445239197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98e4102ef0 0x7f98e419bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:04.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.953+0000 7f98eb82e640 1 --2- 172.21.15.172:0/445239197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98e4102570 0x7f98e4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:04.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.953+0000 7f98eb82e640 1 --2- 172.21.15.172:0/445239197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98e4100e20 0x7f98e4197330 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:04.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.953+0000 7f98eb82e640 1 -- 172.21.15.172:0/445239197 >> 172.21.15.172:0/445239197 conn(0x7f98e40fc7d0 msgr2=0x7f98e41048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:04.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.953+0000 7f98eb82e640 1 -- 172.21.15.172:0/445239197 shutdown_connections 2024-09-10T08:44:04.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:04.953+0000 7f98eb82e640 1 -- 172.21.15.172:0/445239197 wait complete. 2024-09-10T08:44:05.293 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:05 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:44:05.293 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:05 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/445239197' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T08:44:05.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:05 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:44:05.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:05 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/445239197' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T08:44:05.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:05 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:44:05.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:05 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/445239197' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T08:44:06.253 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:06 smithi172 ceph-mon[30854]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:06.253 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:06.253 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:06.253 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:06.253 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:06 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:06.281 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T08:44:06.281 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T08:44:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:06 smithi174 ceph-mon[34068]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:06.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:06 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:06.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:06 smithi191 ceph-mon[35314]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:06.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:06.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:06.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:06.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:06 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:06.554 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:07.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.057+0000 7f19fbd2d640 1 -- 172.21.15.172:0/4250111901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f4103660 msgr2=0x7f19f410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:07.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.057+0000 7f19fbd2d640 1 --2- 172.21.15.172:0/4250111901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f4103660 0x7f19f410bb50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009900 tx=0x7f19e402a480 comp rx=0 tx=0).stop 2024-09-10T08:44:07.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.057+0000 7f19fbd2d640 1 -- 172.21.15.172:0/4250111901 shutdown_connections 2024-09-10T08:44:07.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.057+0000 7f19fbd2d640 1 --2- 172.21.15.172:0/4250111901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19f410c170 0x7f19f410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:07.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.057+0000 7f19fbd2d640 1 --2- 172.21.15.172:0/4250111901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f4103660 0x7f19f410bb50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:07.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.057+0000 7f19fbd2d640 1 --2- 172.21.15.172:0/4250111901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19f4102d40 0x7f19f4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:07.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.057+0000 7f19fbd2d640 1 -- 172.21.15.172:0/4250111901 >> 172.21.15.172:0/4250111901 conn(0x7f19f40fc7a0 msgr2=0x7f19f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:07.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.057+0000 7f19fbd2d640 1 -- 172.21.15.172:0/4250111901 shutdown_connections 2024-09-10T08:44:07.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.057+0000 7f19fbd2d640 1 -- 172.21.15.172:0/4250111901 wait complete. 2024-09-10T08:44:07.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.057+0000 7f19fbd2d640 1 Processor -- start 2024-09-10T08:44:07.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19fbd2d640 1 -- start start 2024-09-10T08:44:07.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19fbd2d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f4102d40 0x7f19f41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:07.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19fbd2d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19f4103660 0x7f19f4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:07.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19fbd2d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19f410c170 0x7f19f419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:07.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19fbd2d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f19f4111e70 con 0x7f19f4102d40 2024-09-10T08:44:07.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19fbd2d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f19f4111cf0 con 0x7f19f410c170 2024-09-10T08:44:07.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19fbd2d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f19f4111ff0 con 0x7f19f4103660 2024-09-10T08:44:07.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19f9ac9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f4102d40 0x7f19f41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:07.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19f9ac9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f4102d40 0x7f19f41983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53742/0 (socket says 172.21.15.172:53742) 2024-09-10T08:44:07.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19f9ac9640 1 -- 172.21.15.172:0/4277249325 learned_addr learned my addr 172.21.15.172:0/4277249325 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:07.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19f9ac9640 1 -- 172.21.15.172:0/4277249325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19f4103660 msgr2=0x7f19f4198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:07.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19f9ac9640 1 --2- 172.21.15.172:0/4277249325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19f4103660 0x7f19f4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:07.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19f9ac9640 1 -- 172.21.15.172:0/4277249325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19f410c170 msgr2=0x7f19f419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:07.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19f9ac9640 1 --2- 172.21.15.172:0/4277249325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19f410c170 0x7f19f419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:07.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19f9ac9640 1 -- 172.21.15.172:0/4277249325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f419d380 con 0x7f19f4102d40 2024-09-10T08:44:07.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19f9ac9640 1 --2- 172.21.15.172:0/4277249325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f4102d40 0x7f19f41983d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f19f000d770 tx=0x7f19f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:07.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19eaffd640 1 -- 172.21.15.172:0/4277249325 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0017070 con 0x7f19f4102d40 2024-09-10T08:44:07.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19eaffd640 1 -- 172.21.15.172:0/4277249325 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19f00054d0 con 0x7f19f4102d40 2024-09-10T08:44:07.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19fbd2d640 1 -- 172.21.15.172:0/4277249325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f419d670 con 0x7f19f4102d40 2024-09-10T08:44:07.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.058+0000 7f19eaffd640 1 -- 172.21.15.172:0/4277249325 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0004cf0 con 0x7f19f4102d40 2024-09-10T08:44:07.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.059+0000 7f19fbd2d640 1 -- 172.21.15.172:0/4277249325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f41a4f50 con 0x7f19f4102d40 2024-09-10T08:44:07.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.060+0000 7f19fbd2d640 1 -- 172.21.15.172:0/4277249325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f4103120 con 0x7f19f4102d40 2024-09-10T08:44:07.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.061+0000 7f19eaffd640 1 -- 172.21.15.172:0/4277249325 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f19f0004e90 con 0x7f19f4102d40 2024-09-10T08:44:07.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.061+0000 7f19eaffd640 1 --2- 172.21.15.172:0/4277249325 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f19e00a2c00 0x7f19e00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:07.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.061+0000 7f19f92c8640 1 --2- 172.21.15.172:0/4277249325 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f19e00a2c00 0x7f19e00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:07.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.061+0000 7f19eaffd640 1 -- 172.21.15.172:0/4277249325 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f19f00ce6a0 con 0x7f19f4102d40 2024-09-10T08:44:07.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.062+0000 7f19f92c8640 1 --2- 172.21.15.172:0/4277249325 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f19e00a2c00 0x7f19e00a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f19e402a9a0 tx=0x7f19e4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:07.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.065+0000 7f19eaffd640 1 -- 172.21.15.172:0/4277249325 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19f009ba40 con 0x7f19f4102d40 2024-09-10T08:44:07.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.233+0000 7f19fbd2d640 1 -- 172.21.15.172:0/4277249325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f19f402d7c0 con 0x7f19f4102d40 2024-09-10T08:44:07.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.642+0000 7f19eaffd640 1 -- 172.21.15.172:0/4277249325 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7f19f009b340 con 0x7f19f4102d40 2024-09-10T08:44:07.642 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' created 2024-09-10T08:44:07.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.644+0000 7f19fbd2d640 1 -- 172.21.15.172:0/4277249325 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f19e00a2c00 msgr2=0x7f19e00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:07.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.644+0000 7f19fbd2d640 1 --2- 172.21.15.172:0/4277249325 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f19e00a2c00 0x7f19e00a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f19e402a9a0 tx=0x7f19e4031400 comp rx=0 tx=0).stop 2024-09-10T08:44:07.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.644+0000 7f19fbd2d640 1 -- 172.21.15.172:0/4277249325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f4102d40 msgr2=0x7f19f41983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:07.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.644+0000 7f19fbd2d640 1 --2- 172.21.15.172:0/4277249325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f4102d40 0x7f19f41983d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f19f000d770 tx=0x7f19f000dc30 comp rx=0 tx=0).stop 2024-09-10T08:44:07.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.644+0000 7f19fbd2d640 1 -- 172.21.15.172:0/4277249325 shutdown_connections 2024-09-10T08:44:07.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.644+0000 7f19fbd2d640 1 --2- 172.21.15.172:0/4277249325 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f19e00a2c00 0x7f19e00a50c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:07.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.644+0000 7f19fbd2d640 1 --2- 172.21.15.172:0/4277249325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19f410c170 0x7f19f419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:07.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.644+0000 7f19fbd2d640 1 --2- 172.21.15.172:0/4277249325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19f4103660 0x7f19f4198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:07.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.644+0000 7f19fbd2d640 1 --2- 172.21.15.172:0/4277249325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f4102d40 0x7f19f41983d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:07.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.644+0000 7f19fbd2d640 1 -- 172.21.15.172:0/4277249325 >> 172.21.15.172:0/4277249325 conn(0x7f19f40fc7a0 msgr2=0x7f19f40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:07.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.645+0000 7f19fbd2d640 1 -- 172.21.15.172:0/4277249325 shutdown_connections 2024-09-10T08:44:07.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:07.645+0000 7f19fbd2d640 1 -- 172.21.15.172:0/4277249325 wait complete. 2024-09-10T08:44:07.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:07 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:07.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:07 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:07.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:07 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:44:07.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:07 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:44:07.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:07 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:07.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:07 smithi174 ceph-mon[34068]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:07.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:07 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/4277249325' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T08:44:07.994 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:07 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:07.994 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:07 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:07.994 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:07 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:44:07.994 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:07 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:44:07.994 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:07 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:07.994 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:07 smithi172 ceph-mon[30854]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:07.994 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:07 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/4277249325' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T08:44:08.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:07 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:08.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:07 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:08.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:07 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:44:08.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:07 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:44:08.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:07 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:44:08.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:07 smithi191 ceph-mon[35314]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:08.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:07 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/4277249325' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T08:44:08.357 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-10T08:44:08.603 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:08.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:08 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/4277249325' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T08:44:08.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:08 smithi174 ceph-mon[34068]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T08:44:09.023 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:08 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/4277249325' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T08:44:09.023 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:08 smithi172 ceph-mon[30854]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T08:44:09.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:08 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/4277249325' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T08:44:09.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:08 smithi191 ceph-mon[35314]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T08:44:09.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.113+0000 7f09d8440640 1 -- 172.21.15.172:0/2467402141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d0105a90 msgr2=0x7f09d0105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:09.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.113+0000 7f09d8440640 1 --2- 172.21.15.172:0/2467402141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d0105a90 0x7f09d0105e70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009900 tx=0x7f09cc02a410 comp rx=0 tx=0).stop 2024-09-10T08:44:09.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.113+0000 7f09d8440640 1 -- 172.21.15.172:0/2467402141 shutdown_connections 2024-09-10T08:44:09.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.113+0000 7f09d8440640 1 --2- 172.21.15.172:0/2467402141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09d0101bc0 0x7f09d010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:09.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.113+0000 7f09d8440640 1 --2- 172.21.15.172:0/2467402141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09d0101240 0x7f09d0101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:09.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.113+0000 7f09d8440640 1 --2- 172.21.15.172:0/2467402141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d0105a90 0x7f09d0105e70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:09.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.113+0000 7f09d8440640 1 -- 172.21.15.172:0/2467402141 >> 172.21.15.172:0/2467402141 conn(0x7f09d0077210 msgr2=0x7f09d00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:09.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.113+0000 7f09d8440640 1 -- 172.21.15.172:0/2467402141 shutdown_connections 2024-09-10T08:44:09.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.113+0000 7f09d8440640 1 -- 172.21.15.172:0/2467402141 wait complete. 2024-09-10T08:44:09.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.114+0000 7f09d8440640 1 Processor -- start 2024-09-10T08:44:09.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.114+0000 7f09d8440640 1 -- start start 2024-09-10T08:44:09.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.114+0000 7f09d8440640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09d0101240 0x7f09d0198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:09.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.114+0000 7f09d8440640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09d0101bc0 0x7f09d0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:09.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.114+0000 7f09d8440640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d0105a90 0x7f09d019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:09.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.114+0000 7f09d8440640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0110ee0 con 0x7f09d0105a90 2024-09-10T08:44:09.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.114+0000 7f09d8440640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0110d60 con 0x7f09d0101bc0 2024-09-10T08:44:09.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.114+0000 7f09d8440640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f09d0111060 con 0x7f09d0101240 2024-09-10T08:44:09.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.114+0000 7f09d69dd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d0105a90 0x7f09d019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:09.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.114+0000 7f09d69dd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d0105a90 0x7f09d019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53758/0 (socket says 172.21.15.172:53758) 2024-09-10T08:44:09.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.114+0000 7f09d69dd640 1 -- 172.21.15.172:0/384327819 learned_addr learned my addr 172.21.15.172:0/384327819 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:09.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.114+0000 7f09d61dc640 1 --2- 172.21.15.172:0/384327819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09d0101240 0x7f09d0198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:09.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.114+0000 7f09d59db640 1 --2- 172.21.15.172:0/384327819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09d0101bc0 0x7f09d0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:09.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.115+0000 7f09d69dd640 1 -- 172.21.15.172:0/384327819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09d0101240 msgr2=0x7f09d0198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:09.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.115+0000 7f09d69dd640 1 --2- 172.21.15.172:0/384327819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09d0101240 0x7f09d0198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:09.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.115+0000 7f09d69dd640 1 -- 172.21.15.172:0/384327819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09d0101bc0 msgr2=0x7f09d0198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:09.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.115+0000 7f09d69dd640 1 --2- 172.21.15.172:0/384327819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09d0101bc0 0x7f09d0198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:09.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.115+0000 7f09d69dd640 1 -- 172.21.15.172:0/384327819 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d019d4b0 con 0x7f09d0105a90 2024-09-10T08:44:09.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.115+0000 7f09d69dd640 1 --2- 172.21.15.172:0/384327819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d0105a90 0x7f09d019cdd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f09c800b970 tx=0x7f09c800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:09.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.115+0000 7f09c77fe640 1 -- 172.21.15.172:0/384327819 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8019070 con 0x7f09d0105a90 2024-09-10T08:44:09.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.115+0000 7f09d8440640 1 -- 172.21.15.172:0/384327819 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d019d7a0 con 0x7f09d0105a90 2024-09-10T08:44:09.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.115+0000 7f09c77fe640 1 -- 172.21.15.172:0/384327819 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09c8014440 con 0x7f09d0105a90 2024-09-10T08:44:09.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.115+0000 7f09c77fe640 1 -- 172.21.15.172:0/384327819 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c80075d0 con 0x7f09d0105a90 2024-09-10T08:44:09.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.115+0000 7f09d8440640 1 -- 172.21.15.172:0/384327819 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d01a5080 con 0x7f09d0105a90 2024-09-10T08:44:09.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.116+0000 7f09d8440640 1 -- 172.21.15.172:0/384327819 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d0101680 con 0x7f09d0105a90 2024-09-10T08:44:09.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.117+0000 7f09c77fe640 1 -- 172.21.15.172:0/384327819 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f09c801c070 con 0x7f09d0105a90 2024-09-10T08:44:09.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.118+0000 7f09c77fe640 1 --2- 172.21.15.172:0/384327819 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f09bc0a2b80 0x7f09bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:09.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.118+0000 7f09d61dc640 1 --2- 172.21.15.172:0/384327819 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f09bc0a2b80 0x7f09bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:09.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.118+0000 7f09c77fe640 1 -- 172.21.15.172:0/384327819 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7497+0+0 (secure 0 0 0) 0x7f09c80ce760 con 0x7f09d0105a90 2024-09-10T08:44:09.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.119+0000 7f09d61dc640 1 --2- 172.21.15.172:0/384327819 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f09bc0a2b80 0x7f09bc0a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009230 tx=0x7f09cc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:09.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.121+0000 7f09c77fe640 1 -- 172.21.15.172:0/384327819 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09c809b3a0 con 0x7f09d0105a90 2024-09-10T08:44:09.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.287+0000 7f09d8440640 1 -- 172.21.15.172:0/384327819 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f09d010a020 con 0x7f09d0105a90 2024-09-10T08:44:09.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.652+0000 7f09c77fe640 1 -- 172.21.15.172:0/384327819 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v80) ==== 171+0+0 (secure 0 0 0) 0x7f09c809f6a0 con 0x7f09d0105a90 2024-09-10T08:44:09.652 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' created 2024-09-10T08:44:09.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.654+0000 7f09d8440640 1 -- 172.21.15.172:0/384327819 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f09bc0a2b80 msgr2=0x7f09bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:09.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.654+0000 7f09d8440640 1 --2- 172.21.15.172:0/384327819 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f09bc0a2b80 0x7f09bc0a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009230 tx=0x7f09cc0058d0 comp rx=0 tx=0).stop 2024-09-10T08:44:09.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.654+0000 7f09d8440640 1 -- 172.21.15.172:0/384327819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d0105a90 msgr2=0x7f09d019cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:09.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.654+0000 7f09d8440640 1 --2- 172.21.15.172:0/384327819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d0105a90 0x7f09d019cdd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f09c800b970 tx=0x7f09c800be30 comp rx=0 tx=0).stop 2024-09-10T08:44:09.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.654+0000 7f09d8440640 1 -- 172.21.15.172:0/384327819 shutdown_connections 2024-09-10T08:44:09.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.654+0000 7f09d8440640 1 --2- 172.21.15.172:0/384327819 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f09bc0a2b80 0x7f09bc0a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:09.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.654+0000 7f09d8440640 1 --2- 172.21.15.172:0/384327819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09d0105a90 0x7f09d019cdd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:09.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.654+0000 7f09d8440640 1 --2- 172.21.15.172:0/384327819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09d0101bc0 0x7f09d0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:09.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.654+0000 7f09d8440640 1 --2- 172.21.15.172:0/384327819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09d0101240 0x7f09d0198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:09.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.654+0000 7f09d8440640 1 -- 172.21.15.172:0/384327819 >> 172.21.15.172:0/384327819 conn(0x7f09d0077210 msgr2=0x7f09d01004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:09.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.654+0000 7f09d8440640 1 -- 172.21.15.172:0/384327819 shutdown_connections 2024-09-10T08:44:09.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:09.655+0000 7f09d8440640 1 -- 172.21.15.172:0/384327819 wait complete. 2024-09-10T08:44:09.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:09 smithi174 ceph-mon[34068]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T08:44:09.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:09 smithi174 ceph-mon[34068]: pgmap v275: 33 pgs: 1 creating+activating, 9 creating+peering, 3 active+clean, 20 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:09.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:09 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/384327819' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T08:44:09.994 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:09 smithi172 ceph-mon[30854]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T08:44:09.994 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:09 smithi172 ceph-mon[30854]: pgmap v275: 33 pgs: 1 creating+activating, 9 creating+peering, 3 active+clean, 20 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:09.994 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:09 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/384327819' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T08:44:10.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:09 smithi191 ceph-mon[35314]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T08:44:10.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:09 smithi191 ceph-mon[35314]: pgmap v275: 33 pgs: 1 creating+activating, 9 creating+peering, 3 active+clean, 20 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:10.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:09 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/384327819' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T08:44:10.351 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T08:44:10.600 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:10.855 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:10 smithi172 ceph-mon[30854]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:44:10.855 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:10 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/384327819' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T08:44:10.856 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:10 smithi172 ceph-mon[30854]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T08:44:10.856 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:10 smithi172 ceph-mon[30854]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T08:44:10.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:10 smithi174 ceph-mon[34068]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:44:10.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:10 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/384327819' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T08:44:10.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:10 smithi174 ceph-mon[34068]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T08:44:10.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:10 smithi174 ceph-mon[34068]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T08:44:11.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:10 smithi191 ceph-mon[35314]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:44:11.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:10 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/384327819' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T08:44:11.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:10 smithi191 ceph-mon[35314]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T08:44:11.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:10 smithi191 ceph-mon[35314]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T08:44:11.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.098+0000 7f546f7c4640 1 -- 172.21.15.172:0/2438043380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5468106a80 msgr2=0x7f5468106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:11.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.098+0000 7f546f7c4640 1 --2- 172.21.15.172:0/2438043380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5468106a80 0x7f5468106e60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5464009930 tx=0x7f546402a420 comp rx=0 tx=0).stop 2024-09-10T08:44:11.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.098+0000 7f546f7c4640 1 -- 172.21.15.172:0/2438043380 shutdown_connections 2024-09-10T08:44:11.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.098+0000 7f546f7c4640 1 --2- 172.21.15.172:0/2438043380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5468101420 0x7f546810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:11.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.098+0000 7f546f7c4640 1 --2- 172.21.15.172:0/2438043380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468100a80 0x7f5468100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:11.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.098+0000 7f546f7c4640 1 --2- 172.21.15.172:0/2438043380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5468106a80 0x7f5468106e60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:11.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.098+0000 7f546f7c4640 1 -- 172.21.15.172:0/2438043380 >> 172.21.15.172:0/2438043380 conn(0x7f54680fc7a0 msgr2=0x7f54680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:11.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.098+0000 7f546f7c4640 1 -- 172.21.15.172:0/2438043380 shutdown_connections 2024-09-10T08:44:11.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.098+0000 7f546f7c4640 1 -- 172.21.15.172:0/2438043380 wait complete. 2024-09-10T08:44:11.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.099+0000 7f546f7c4640 1 Processor -- start 2024-09-10T08:44:11.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.099+0000 7f546f7c4640 1 -- start start 2024-09-10T08:44:11.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.099+0000 7f546f7c4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5468100a80 0x7f5468198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:11.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.099+0000 7f546f7c4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5468101420 0x7f54681988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:11.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.099+0000 7f546f7c4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468106a80 0x7f546819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:11.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f546f7c4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5468110d30 con 0x7f5468100a80 2024-09-10T08:44:11.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f546f7c4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5468110bb0 con 0x7f5468106a80 2024-09-10T08:44:11.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f546f7c4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5468110eb0 con 0x7f5468101420 2024-09-10T08:44:11.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f546d560640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5468100a80 0x7f5468198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:11.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f546d560640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5468100a80 0x7f5468198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53778/0 (socket says 172.21.15.172:53778) 2024-09-10T08:44:11.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f546d560640 1 -- 172.21.15.172:0/1295155902 learned_addr learned my addr 172.21.15.172:0/1295155902 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:11.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f546cd5f640 1 --2- 172.21.15.172:0/1295155902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5468101420 0x7f54681988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:11.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f546dd61640 1 --2- 172.21.15.172:0/1295155902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468106a80 0x7f546819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:11.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f546d560640 1 -- 172.21.15.172:0/1295155902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5468101420 msgr2=0x7f54681988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:11.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f546d560640 1 --2- 172.21.15.172:0/1295155902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5468101420 0x7f54681988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:11.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f546d560640 1 -- 172.21.15.172:0/1295155902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468106a80 msgr2=0x7f546819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:11.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f546d560640 1 --2- 172.21.15.172:0/1295155902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468106a80 0x7f546819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:11.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f546d560640 1 -- 172.21.15.172:0/1295155902 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546819d310 con 0x7f5468100a80 2024-09-10T08:44:11.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f546d560640 1 --2- 172.21.15.172:0/1295155902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5468100a80 0x7f5468198360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5464004770 tx=0x7f54640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:11.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f545e7fc640 1 -- 172.21.15.172:0/1295155902 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546403f070 con 0x7f5468100a80 2024-09-10T08:44:11.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f546f7c4640 1 -- 172.21.15.172:0/1295155902 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546819d5a0 con 0x7f5468100a80 2024-09-10T08:44:11.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f545e7fc640 1 -- 172.21.15.172:0/1295155902 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f546403b460 con 0x7f5468100a80 2024-09-10T08:44:11.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f545e7fc640 1 -- 172.21.15.172:0/1295155902 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464035070 con 0x7f5468100a80 2024-09-10T08:44:11.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.100+0000 7f546f7c4640 1 -- 172.21.15.172:0/1295155902 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54681a4ee0 con 0x7f5468100a80 2024-09-10T08:44:11.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.101+0000 7f546f7c4640 1 -- 172.21.15.172:0/1295155902 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54681021c0 con 0x7f5468100a80 2024-09-10T08:44:11.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.102+0000 7f545e7fc640 1 -- 172.21.15.172:0/1295155902 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f546403b610 con 0x7f5468100a80 2024-09-10T08:44:11.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.103+0000 7f545e7fc640 1 --2- 172.21.15.172:0/1295155902 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f54540a2b80 0x7f54540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:11.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.103+0000 7f546cd5f640 1 --2- 172.21.15.172:0/1295155902 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f54540a2b80 0x7f54540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:11.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.103+0000 7f545e7fc640 1 -- 172.21.15.172:0/1295155902 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7879+0+0 (secure 0 0 0) 0x7f54640ef3e0 con 0x7f5468100a80 2024-09-10T08:44:11.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.104+0000 7f546cd5f640 1 --2- 172.21.15.172:0/1295155902 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f54540a2b80 0x7f54540a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5468104a50 tx=0x7f545800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:11.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.106+0000 7f545e7fc640 1 -- 172.21.15.172:0/1295155902 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f54640bbea0 con 0x7f5468100a80 2024-09-10T08:44:11.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.309+0000 7f546f7c4640 1 -- 172.21.15.172:0/1295155902 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5468100ee0 con 0x7f5468100a80 2024-09-10T08:44:11.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.673+0000 7f545e7fc640 1 -- 172.21.15.172:0/1295155902 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f5468100ee0 con 0x7f5468100a80 2024-09-10T08:44:11.674 INFO:teuthology.orchestra.run.smithi172.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T08:44:11.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.676+0000 7f546f7c4640 1 -- 172.21.15.172:0/1295155902 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f54540a2b80 msgr2=0x7f54540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:11.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.676+0000 7f546f7c4640 1 --2- 172.21.15.172:0/1295155902 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f54540a2b80 0x7f54540a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5468104a50 tx=0x7f545800b390 comp rx=0 tx=0).stop 2024-09-10T08:44:11.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.676+0000 7f546f7c4640 1 -- 172.21.15.172:0/1295155902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5468100a80 msgr2=0x7f5468198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:11.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.676+0000 7f546f7c4640 1 --2- 172.21.15.172:0/1295155902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5468100a80 0x7f5468198360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5464004770 tx=0x7f54640042f0 comp rx=0 tx=0).stop 2024-09-10T08:44:11.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.676+0000 7f546f7c4640 1 -- 172.21.15.172:0/1295155902 shutdown_connections 2024-09-10T08:44:11.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.676+0000 7f546f7c4640 1 --2- 172.21.15.172:0/1295155902 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f54540a2b80 0x7f54540a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:11.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.676+0000 7f546f7c4640 1 --2- 172.21.15.172:0/1295155902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468106a80 0x7f546819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:11.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.676+0000 7f546f7c4640 1 --2- 172.21.15.172:0/1295155902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5468101420 0x7f54681988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:11.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.676+0000 7f546f7c4640 1 --2- 172.21.15.172:0/1295155902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5468100a80 0x7f5468198360 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:11.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.676+0000 7f546f7c4640 1 -- 172.21.15.172:0/1295155902 >> 172.21.15.172:0/1295155902 conn(0x7f54680fc7a0 msgr2=0x7f546806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:11.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.676+0000 7f546f7c4640 1 -- 172.21.15.172:0/1295155902 shutdown_connections 2024-09-10T08:44:11.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:11.676+0000 7f546f7c4640 1 -- 172.21.15.172:0/1295155902 wait complete. 2024-09-10T08:44:11.960 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:11 smithi172 ceph-mon[30854]: pgmap v278: 97 pgs: 1 creating+activating, 33 creating+peering, 14 active+clean, 49 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:11.960 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:11 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1295155902' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T08:44:11.960 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:11 smithi172 ceph-mon[30854]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T08:44:11.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:11 smithi174 ceph-mon[34068]: pgmap v278: 97 pgs: 1 creating+activating, 33 creating+peering, 14 active+clean, 49 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:11.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:11 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1295155902' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T08:44:11.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:11 smithi174 ceph-mon[34068]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T08:44:12.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:11 smithi191 ceph-mon[35314]: pgmap v278: 97 pgs: 1 creating+activating, 33 creating+peering, 14 active+clean, 49 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:12.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:11 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1295155902' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T08:44:12.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:11 smithi191 ceph-mon[35314]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T08:44:12.343 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-10T08:44:12.343 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-10T08:44:12.605 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:12.974 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:12 smithi172 ceph-mon[30854]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T08:44:12.974 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:12 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1295155902' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T08:44:12.974 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:12 smithi172 ceph-mon[30854]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T08:44:12.975 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:12 smithi172 ceph-mon[30854]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T08:44:12.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:12 smithi174 ceph-mon[34068]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T08:44:12.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:12 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1295155902' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T08:44:12.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:12 smithi174 ceph-mon[34068]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T08:44:12.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:12 smithi174 ceph-mon[34068]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T08:44:13.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:12 smithi191 ceph-mon[35314]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T08:44:13.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:12 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1295155902' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T08:44:13.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:12 smithi191 ceph-mon[35314]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T08:44:13.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:12 smithi191 ceph-mon[35314]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T08:44:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.111+0000 7f45310fd640 1 -- 172.21.15.172:0/3051694553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f452c106f20 msgr2=0x7f452c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.111+0000 7f45310fd640 1 --2- 172.21.15.172:0/3051694553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f452c106f20 0x7f452c10eaa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4524009900 tx=0x7f452402a410 comp rx=0 tx=0).stop 2024-09-10T08:44:13.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.111+0000 7f45310fd640 1 -- 172.21.15.172:0/3051694553 shutdown_connections 2024-09-10T08:44:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.111+0000 7f45310fd640 1 --2- 172.21.15.172:0/3051694553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f452c106f20 0x7f452c10eaa0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.111+0000 7f45310fd640 1 --2- 172.21.15.172:0/3051694553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f452c1027a0 0x7f452c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.111+0000 7f45310fd640 1 --2- 172.21.15.172:0/3051694553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c101df0 0x7f452c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.111+0000 7f45310fd640 1 -- 172.21.15.172:0/3051694553 >> 172.21.15.172:0/3051694553 conn(0x7f452c0fd8f0 msgr2=0x7f452c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.111+0000 7f45310fd640 1 -- 172.21.15.172:0/3051694553 shutdown_connections 2024-09-10T08:44:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.111+0000 7f45310fd640 1 -- 172.21.15.172:0/3051694553 wait complete. 2024-09-10T08:44:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.111+0000 7f45310fd640 1 Processor -- start 2024-09-10T08:44:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f45310fd640 1 -- start start 2024-09-10T08:44:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f45310fd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c101df0 0x7f452c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f45310fd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f452c1027a0 0x7f452c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f45310fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f452c106f20 0x7f452c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f45310fd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f452c111e30 con 0x7f452c106f20 2024-09-10T08:44:13.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f45310fd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f452c111cb0 con 0x7f452c1027a0 2024-09-10T08:44:13.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f45310fd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f452c111fb0 con 0x7f452c101df0 2024-09-10T08:44:13.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f452b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f452c106f20 0x7f452c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:13.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f452b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f452c106f20 0x7f452c19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53794/0 (socket says 172.21.15.172:53794) 2024-09-10T08:44:13.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f452b59e640 1 -- 172.21.15.172:0/916462214 learned_addr learned my addr 172.21.15.172:0/916462214 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:13.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f452a59c640 1 --2- 172.21.15.172:0/916462214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f452c1027a0 0x7f452c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:13.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f452ad9d640 1 --2- 172.21.15.172:0/916462214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c101df0 0x7f452c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:13.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f452b59e640 1 -- 172.21.15.172:0/916462214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c101df0 msgr2=0x7f452c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:13.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f452b59e640 1 --2- 172.21.15.172:0/916462214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c101df0 0x7f452c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:13.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f452b59e640 1 -- 172.21.15.172:0/916462214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f452c1027a0 msgr2=0x7f452c1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:13.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f452b59e640 1 --2- 172.21.15.172:0/916462214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f452c1027a0 0x7f452c1987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:13.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f452b59e640 1 -- 172.21.15.172:0/916462214 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c19d230 con 0x7f452c106f20 2024-09-10T08:44:13.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.112+0000 7f452ad9d640 1 --2- 172.21.15.172:0/916462214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c101df0 0x7f452c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:44:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.113+0000 7f452b59e640 1 --2- 172.21.15.172:0/916462214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f452c106f20 0x7f452c19cb50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4524009230 tx=0x7f4524004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.113+0000 7f4523fff640 1 -- 172.21.15.172:0/916462214 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452403f030 con 0x7f452c106f20 2024-09-10T08:44:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.113+0000 7f45310fd640 1 -- 172.21.15.172:0/916462214 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452c19d4c0 con 0x7f452c106f20 2024-09-10T08:44:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.113+0000 7f4523fff640 1 -- 172.21.15.172:0/916462214 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4524003aa0 con 0x7f452c106f20 2024-09-10T08:44:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.113+0000 7f4523fff640 1 -- 172.21.15.172:0/916462214 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524035070 con 0x7f452c106f20 2024-09-10T08:44:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.113+0000 7f45310fd640 1 -- 172.21.15.172:0/916462214 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452c1a4e00 con 0x7f452c106f20 2024-09-10T08:44:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.114+0000 7f45310fd640 1 -- 172.21.15.172:0/916462214 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452c1021d0 con 0x7f452c106f20 2024-09-10T08:44:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.115+0000 7f4523fff640 1 -- 172.21.15.172:0/916462214 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4524042030 con 0x7f452c106f20 2024-09-10T08:44:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.116+0000 7f4523fff640 1 --2- 172.21.15.172:0/916462214 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f45180a2b80 0x7f45180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.116+0000 7f452ad9d640 1 --2- 172.21.15.172:0/916462214 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f45180a2b80 0x7f45180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.116+0000 7f4523fff640 1 -- 172.21.15.172:0/916462214 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f45240ef470 con 0x7f452c106f20 2024-09-10T08:44:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.116+0000 7f452ad9d640 1 --2- 172.21.15.172:0/916462214 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f45180a2b80 0x7f45180a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f451c008e00 tx=0x7f451c008350 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:13.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.120+0000 7f4523fff640 1 -- 172.21.15.172:0/916462214 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f45240bc520 con 0x7f452c106f20 2024-09-10T08:44:13.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.298+0000 7f45310fd640 1 -- 172.21.15.172:0/916462214 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f452c10a100 con 0x7f452c106f20 2024-09-10T08:44:13.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.694+0000 7f4523fff640 1 -- 172.21.15.172:0/916462214 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v83) ==== 157+0+0 (secure 0 0 0) 0x7f45240bbe20 con 0x7f452c106f20 2024-09-10T08:44:13.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.696+0000 7f45310fd640 1 -- 172.21.15.172:0/916462214 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f45180a2b80 msgr2=0x7f45180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:13.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.696+0000 7f45310fd640 1 --2- 172.21.15.172:0/916462214 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f45180a2b80 0x7f45180a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f451c008e00 tx=0x7f451c008350 comp rx=0 tx=0).stop 2024-09-10T08:44:13.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.696+0000 7f45310fd640 1 -- 172.21.15.172:0/916462214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f452c106f20 msgr2=0x7f452c19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:13.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.696+0000 7f45310fd640 1 --2- 172.21.15.172:0/916462214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f452c106f20 0x7f452c19cb50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4524009230 tx=0x7f4524004270 comp rx=0 tx=0).stop 2024-09-10T08:44:13.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.696+0000 7f45310fd640 1 -- 172.21.15.172:0/916462214 shutdown_connections 2024-09-10T08:44:13.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.696+0000 7f45310fd640 1 --2- 172.21.15.172:0/916462214 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f45180a2b80 0x7f45180a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:13.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.696+0000 7f45310fd640 1 --2- 172.21.15.172:0/916462214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f452c106f20 0x7f452c19cb50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:13.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.696+0000 7f45310fd640 1 --2- 172.21.15.172:0/916462214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f452c1027a0 0x7f452c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:13.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.696+0000 7f45310fd640 1 --2- 172.21.15.172:0/916462214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c101df0 0x7f452c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:13.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.696+0000 7f45310fd640 1 -- 172.21.15.172:0/916462214 >> 172.21.15.172:0/916462214 conn(0x7f452c0fd8f0 msgr2=0x7f452c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:13.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.697+0000 7f45310fd640 1 -- 172.21.15.172:0/916462214 shutdown_connections 2024-09-10T08:44:13.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:13.697+0000 7f45310fd640 1 -- 172.21.15.172:0/916462214 wait complete. 2024-09-10T08:44:13.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:13 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] up:active 2024-09-10T08:44:13.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:13 smithi174 ceph-mon[34068]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:44:13.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:13 smithi174 ceph-mon[34068]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:44:13.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:13 smithi174 ceph-mon[34068]: pgmap v280: 97 pgs: 1 creating+activating, 33 creating+peering, 43 active+clean, 20 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 0 op/s 2024-09-10T08:44:13.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:13 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/916462214' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T08:44:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:13 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] up:active 2024-09-10T08:44:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:13 smithi191 ceph-mon[35314]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:44:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:13 smithi191 ceph-mon[35314]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:44:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:13 smithi191 ceph-mon[35314]: pgmap v280: 97 pgs: 1 creating+activating, 33 creating+peering, 43 active+clean, 20 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 0 op/s 2024-09-10T08:44:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:13 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/916462214' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T08:44:14.070 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:13 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] up:active 2024-09-10T08:44:14.071 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:13 smithi172 ceph-mon[30854]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:44:14.071 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:13 smithi172 ceph-mon[30854]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:44:14.071 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:13 smithi172 ceph-mon[30854]: pgmap v280: 97 pgs: 1 creating+activating, 33 creating+peering, 43 active+clean, 20 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 0 op/s 2024-09-10T08:44:14.071 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:13 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/916462214' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T08:44:14.393 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T08:44:14.642 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:14.939 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:14 smithi172 ceph-mon[30854]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T08:44:14.939 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:14 smithi172 ceph-mon[30854]: Cluster is now healthy 2024-09-10T08:44:14.939 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:14 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/916462214' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T08:44:14.939 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:14 smithi172 ceph-mon[30854]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T08:44:14.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:14 smithi174 ceph-mon[34068]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T08:44:14.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:14 smithi174 ceph-mon[34068]: Cluster is now healthy 2024-09-10T08:44:14.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:14 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/916462214' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T08:44:14.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:14 smithi174 ceph-mon[34068]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T08:44:15.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:14 smithi191 ceph-mon[35314]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T08:44:15.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:14 smithi191 ceph-mon[35314]: Cluster is now healthy 2024-09-10T08:44:15.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:14 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/916462214' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T08:44:15.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:14 smithi191 ceph-mon[35314]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T08:44:15.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.181+0000 7f04ce570640 1 -- 172.21.15.172:0/367568334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c8101de0 msgr2=0x7f04c81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:15.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.181+0000 7f04ce570640 1 --2- 172.21.15.172:0/367568334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c8101de0 0x7f04c81021c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009900 tx=0x7f04b802a410 comp rx=0 tx=0).stop 2024-09-10T08:44:15.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.181+0000 7f04ce570640 1 -- 172.21.15.172:0/367568334 shutdown_connections 2024-09-10T08:44:15.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.181+0000 7f04ce570640 1 --2- 172.21.15.172:0/367568334 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04c8106f10 0x7f04c810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:15.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.181+0000 7f04ce570640 1 --2- 172.21.15.172:0/367568334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04c8102790 0x7f04c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:15.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.181+0000 7f04ce570640 1 --2- 172.21.15.172:0/367568334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c8101de0 0x7f04c81021c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:15.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.181+0000 7f04ce570640 1 -- 172.21.15.172:0/367568334 >> 172.21.15.172:0/367568334 conn(0x7f04c80fd8c0 msgr2=0x7f04c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:15.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.181+0000 7f04ce570640 1 -- 172.21.15.172:0/367568334 shutdown_connections 2024-09-10T08:44:15.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.181+0000 7f04ce570640 1 -- 172.21.15.172:0/367568334 wait complete. 2024-09-10T08:44:15.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04ce570640 1 Processor -- start 2024-09-10T08:44:15.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04ce570640 1 -- start start 2024-09-10T08:44:15.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04ce570640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04c8101de0 0x7f04c8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:15.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04ce570640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04c8102790 0x7f04c81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:15.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04ce570640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c8106f10 0x7f04c819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:15.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04ce570640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f04c8111e40 con 0x7f04c8106f10 2024-09-10T08:44:15.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04ce570640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04c8111cc0 con 0x7f04c8101de0 2024-09-10T08:44:15.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04ce570640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f04c8111fc0 con 0x7f04c8102790 2024-09-10T08:44:15.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04ccb0d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c8106f10 0x7f04c819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:15.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04ccb0d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c8106f10 0x7f04c819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53820/0 (socket says 172.21.15.172:53820) 2024-09-10T08:44:15.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04ccb0d640 1 -- 172.21.15.172:0/4047380403 learned_addr learned my addr 172.21.15.172:0/4047380403 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:15.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04c7fff640 1 --2- 172.21.15.172:0/4047380403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04c8101de0 0x7f04c8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:15.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04bffff640 1 --2- 172.21.15.172:0/4047380403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04c8102790 0x7f04c81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:15.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04ccb0d640 1 -- 172.21.15.172:0/4047380403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04c8102790 msgr2=0x7f04c81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:15.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04ccb0d640 1 --2- 172.21.15.172:0/4047380403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04c8102790 0x7f04c81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:15.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04ccb0d640 1 -- 172.21.15.172:0/4047380403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04c8101de0 msgr2=0x7f04c8198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:15.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04ccb0d640 1 --2- 172.21.15.172:0/4047380403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04c8101de0 0x7f04c8198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:15.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04ccb0d640 1 -- 172.21.15.172:0/4047380403 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c819d410 con 0x7f04c8106f10 2024-09-10T08:44:15.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04c7fff640 1 --2- 172.21.15.172:0/4047380403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04c8101de0 0x7f04c8198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:15.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.182+0000 7f04ccb0d640 1 --2- 172.21.15.172:0/4047380403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c8106f10 0x7f04c819cd30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f04c000b970 tx=0x7f04c000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:15.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.183+0000 7f04c5ffb640 1 -- 172.21.15.172:0/4047380403 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0019070 con 0x7f04c8106f10 2024-09-10T08:44:15.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.183+0000 7f04c5ffb640 1 -- 172.21.15.172:0/4047380403 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04c0014440 con 0x7f04c8106f10 2024-09-10T08:44:15.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.183+0000 7f04ce570640 1 -- 172.21.15.172:0/4047380403 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c819d700 con 0x7f04c8106f10 2024-09-10T08:44:15.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.183+0000 7f04c5ffb640 1 -- 172.21.15.172:0/4047380403 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c00075d0 con 0x7f04c8106f10 2024-09-10T08:44:15.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.183+0000 7f04ce570640 1 -- 172.21.15.172:0/4047380403 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c81a4fe0 con 0x7f04c8106f10 2024-09-10T08:44:15.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.184+0000 7f04ce570640 1 -- 172.21.15.172:0/4047380403 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c81021c0 con 0x7f04c8106f10 2024-09-10T08:44:15.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.185+0000 7f04c5ffb640 1 -- 172.21.15.172:0/4047380403 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f04c001c070 con 0x7f04c8106f10 2024-09-10T08:44:15.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.185+0000 7f04c5ffb640 1 --2- 172.21.15.172:0/4047380403 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04b40a2c00 0x7f04b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:15.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.185+0000 7f04c7fff640 1 --2- 172.21.15.172:0/4047380403 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04b40a2c00 0x7f04b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:15.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.185+0000 7f04c5ffb640 1 -- 172.21.15.172:0/4047380403 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8277+0+0 (secure 0 0 0) 0x7f04c00ce5c0 con 0x7f04c8106f10 2024-09-10T08:44:15.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.186+0000 7f04c7fff640 1 --2- 172.21.15.172:0/4047380403 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04b40a2c00 0x7f04b40a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f04b8031440 tx=0x7f04b80095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:15.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.190+0000 7f04c5ffb640 1 -- 172.21.15.172:0/4047380403 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f04c009b4f0 con 0x7f04c8106f10 2024-09-10T08:44:15.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:15.358+0000 7f04ce570640 1 -- 172.21.15.172:0/4047380403 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f04c8109b60 con 0x7f04c8106f10 2024-09-10T08:44:15.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:15 smithi174 ceph-mon[34068]: pgmap v282: 97 pgs: 17 creating+peering, 80 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-10T08:44:15.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:15 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/4047380403' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T08:44:16.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:15 smithi191 ceph-mon[35314]: pgmap v282: 97 pgs: 17 creating+peering, 80 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-10T08:44:16.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:15 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/4047380403' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T08:44:16.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:15 smithi172 ceph-mon[30854]: pgmap v282: 97 pgs: 17 creating+peering, 80 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-10T08:44:16.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:15 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/4047380403' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T08:44:16.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:16.694+0000 7f04c5ffb640 1 -- 172.21.15.172:0/4047380403 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v85) ==== 228+0+0 (secure 0 0 0) 0x7f04c009adf0 con 0x7f04c8106f10 2024-09-10T08:44:16.695 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data_ec' created 2024-09-10T08:44:16.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:16.697+0000 7f04ce570640 1 -- 172.21.15.172:0/4047380403 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04b40a2c00 msgr2=0x7f04b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:16.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:16.697+0000 7f04ce570640 1 --2- 172.21.15.172:0/4047380403 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04b40a2c00 0x7f04b40a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f04b8031440 tx=0x7f04b80095f0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:16.697+0000 7f04ce570640 1 -- 172.21.15.172:0/4047380403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c8106f10 msgr2=0x7f04c819cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:16.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:16.697+0000 7f04ce570640 1 --2- 172.21.15.172:0/4047380403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c8106f10 0x7f04c819cd30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f04c000b970 tx=0x7f04c000be30 comp rx=0 tx=0).stop 2024-09-10T08:44:16.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:16.697+0000 7f04ce570640 1 -- 172.21.15.172:0/4047380403 shutdown_connections 2024-09-10T08:44:16.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:16.697+0000 7f04ce570640 1 --2- 172.21.15.172:0/4047380403 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04b40a2c00 0x7f04b40a50c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:16.697+0000 7f04ce570640 1 --2- 172.21.15.172:0/4047380403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f04c8106f10 0x7f04c819cd30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:16.697+0000 7f04ce570640 1 --2- 172.21.15.172:0/4047380403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04c8102790 0x7f04c81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:16.697+0000 7f04ce570640 1 --2- 172.21.15.172:0/4047380403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04c8101de0 0x7f04c8198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:16.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:16.697+0000 7f04ce570640 1 -- 172.21.15.172:0/4047380403 >> 172.21.15.172:0/4047380403 conn(0x7f04c80fd8c0 msgr2=0x7f04c80fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:16.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:16.698+0000 7f04ce570640 1 -- 172.21.15.172:0/4047380403 shutdown_connections 2024-09-10T08:44:16.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:16.698+0000 7f04ce570640 1 -- 172.21.15.172:0/4047380403 wait complete. 2024-09-10T08:44:16.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:16 smithi174 ceph-mon[34068]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T08:44:17.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:16 smithi191 ceph-mon[35314]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T08:44:17.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:16 smithi172 ceph-mon[30854]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T08:44:17.425 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-10T08:44:17.491 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 08:44:17 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-b[63897]: 2024-09-10T08:44:17.173+0000 7ff8b4274640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T08:44:17.679 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:17.703 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:17 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/4047380403' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T08:44:17.703 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:17 smithi172 ceph-mon[30854]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T08:44:17.703 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:17 smithi172 ceph-mon[30854]: pgmap v285: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-10T08:44:17.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:17 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/4047380403' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T08:44:17.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:17 smithi174 ceph-mon[34068]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T08:44:17.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:17 smithi174 ceph-mon[34068]: pgmap v285: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-10T08:44:18.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:17 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/4047380403' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T08:44:18.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:17 smithi191 ceph-mon[35314]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T08:44:18.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:17 smithi191 ceph-mon[35314]: pgmap v285: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-10T08:44:18.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.185+0000 7f8244df9640 1 -- 172.21.15.172:0/908631918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f824010c170 msgr2=0x7f824010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:18.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.185+0000 7f8244df9640 1 --2- 172.21.15.172:0/908631918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f824010c170 0x7f824010e610 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8238009ca0 tx=0x7f823802a950 comp rx=0 tx=0).stop 2024-09-10T08:44:18.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.185+0000 7f8244df9640 1 -- 172.21.15.172:0/908631918 shutdown_connections 2024-09-10T08:44:18.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.185+0000 7f8244df9640 1 --2- 172.21.15.172:0/908631918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f824010c170 0x7f824010e610 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:18.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.185+0000 7f8244df9640 1 --2- 172.21.15.172:0/908631918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8240103660 0x7f824010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:18.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.185+0000 7f8244df9640 1 --2- 172.21.15.172:0/908631918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8240102d40 0x7f8240103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:18.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.185+0000 7f8244df9640 1 -- 172.21.15.172:0/908631918 >> 172.21.15.172:0/908631918 conn(0x7f82400fc7a0 msgr2=0x7f82400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:18.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.185+0000 7f8244df9640 1 -- 172.21.15.172:0/908631918 shutdown_connections 2024-09-10T08:44:18.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.185+0000 7f8244df9640 1 -- 172.21.15.172:0/908631918 wait complete. 2024-09-10T08:44:18.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.186+0000 7f8244df9640 1 Processor -- start 2024-09-10T08:44:18.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.186+0000 7f8244df9640 1 -- start start 2024-09-10T08:44:18.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.186+0000 7f8244df9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8240102d40 0x7f824019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:18.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.186+0000 7f8244df9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8240103660 0x7f824019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:18.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.186+0000 7f8244df9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f824010c170 0x7f824019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:18.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.186+0000 7f8244df9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8240111e20 con 0x7f8240103660 2024-09-10T08:44:18.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.186+0000 7f8244df9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8240111ca0 con 0x7f8240102d40 2024-09-10T08:44:18.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.186+0000 7f8244df9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8240111fa0 con 0x7f824010c170 2024-09-10T08:44:18.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.186+0000 7f823dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8240103660 0x7f824019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:18.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.186+0000 7f823dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8240103660 0x7f824019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34926/0 (socket says 172.21.15.172:34926) 2024-09-10T08:44:18.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.186+0000 7f823dd9b640 1 -- 172.21.15.172:0/3438181002 learned_addr learned my addr 172.21.15.172:0/3438181002 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:18.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.187+0000 7f823e59c640 1 --2- 172.21.15.172:0/3438181002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8240102d40 0x7f824019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:18.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.187+0000 7f823ed9d640 1 --2- 172.21.15.172:0/3438181002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f824010c170 0x7f824019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:18.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.187+0000 7f823dd9b640 1 -- 172.21.15.172:0/3438181002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f824010c170 msgr2=0x7f824019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:18.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.187+0000 7f823dd9b640 1 --2- 172.21.15.172:0/3438181002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f824010c170 0x7f824019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:18.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.187+0000 7f823dd9b640 1 -- 172.21.15.172:0/3438181002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8240102d40 msgr2=0x7f824019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:18.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.187+0000 7f823dd9b640 1 --2- 172.21.15.172:0/3438181002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8240102d40 0x7f824019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:18.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.187+0000 7f823dd9b640 1 -- 172.21.15.172:0/3438181002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824019f4f0 con 0x7f8240103660 2024-09-10T08:44:18.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.187+0000 7f823e59c640 1 --2- 172.21.15.172:0/3438181002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8240102d40 0x7f824019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:18.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.187+0000 7f823dd9b640 1 --2- 172.21.15.172:0/3438181002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8240103660 0x7f824019aa80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f823400e9f0 tx=0x7f823400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:18.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.187+0000 7f822f7fe640 1 -- 172.21.15.172:0/3438181002 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823400ccc0 con 0x7f8240103660 2024-09-10T08:44:18.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.187+0000 7f822f7fe640 1 -- 172.21.15.172:0/3438181002 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82340108e0 con 0x7f8240103660 2024-09-10T08:44:18.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.187+0000 7f8244df9640 1 -- 172.21.15.172:0/3438181002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824019f7e0 con 0x7f8240103660 2024-09-10T08:44:18.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.187+0000 7f822f7fe640 1 -- 172.21.15.172:0/3438181002 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82340187c0 con 0x7f8240103660 2024-09-10T08:44:18.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.187+0000 7f8244df9640 1 -- 172.21.15.172:0/3438181002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82401a70c0 con 0x7f8240103660 2024-09-10T08:44:18.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.188+0000 7f8244df9640 1 -- 172.21.15.172:0/3438181002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8240103120 con 0x7f8240103660 2024-09-10T08:44:18.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.190+0000 7f822f7fe640 1 -- 172.21.15.172:0/3438181002 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8234010430 con 0x7f8240103660 2024-09-10T08:44:18.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.191+0000 7f822f7fe640 1 --2- 172.21.15.172:0/3438181002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f82280a2b80 0x7f82280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:18.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.191+0000 7f823e59c640 1 --2- 172.21.15.172:0/3438181002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f82280a2b80 0x7f82280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:18.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.191+0000 7f822f7fe640 1 -- 172.21.15.172:0/3438181002 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8770+0+0 (secure 0 0 0) 0x7f82340cfab0 con 0x7f8240103660 2024-09-10T08:44:18.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.191+0000 7f823e59c640 1 --2- 172.21.15.172:0/3438181002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f82280a2b80 0x7f82280a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f82300058c0 tx=0x7f8230005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:18.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.194+0000 7f822f7fe640 1 -- 172.21.15.172:0/3438181002 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f823409c7f0 con 0x7f8240103660 2024-09-10T08:44:18.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.362+0000 7f8244df9640 1 -- 172.21.15.172:0/3438181002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f82401a73f0 con 0x7f8240103660 2024-09-10T08:44:18.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.707+0000 7f822f7fe640 1 -- 172.21.15.172:0/3438181002 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) ==== 169+0+0 (secure 0 0 0) 0x7f823409c0f0 con 0x7f8240103660 2024-09-10T08:44:18.707 INFO:teuthology.orchestra.run.smithi172.stderr:set pool 4 allow_ec_overwrites to true 2024-09-10T08:44:18.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.710+0000 7f8244df9640 1 -- 172.21.15.172:0/3438181002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f82280a2b80 msgr2=0x7f82280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:18.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.710+0000 7f8244df9640 1 --2- 172.21.15.172:0/3438181002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f82280a2b80 0x7f82280a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f82300058c0 tx=0x7f8230005850 comp rx=0 tx=0).stop 2024-09-10T08:44:18.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.710+0000 7f8244df9640 1 -- 172.21.15.172:0/3438181002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8240103660 msgr2=0x7f824019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:18.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.710+0000 7f8244df9640 1 --2- 172.21.15.172:0/3438181002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8240103660 0x7f824019aa80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f823400e9f0 tx=0x7f823400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:44:18.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.710+0000 7f8244df9640 1 -- 172.21.15.172:0/3438181002 shutdown_connections 2024-09-10T08:44:18.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.710+0000 7f8244df9640 1 --2- 172.21.15.172:0/3438181002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f82280a2b80 0x7f82280a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:18.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.710+0000 7f8244df9640 1 --2- 172.21.15.172:0/3438181002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f824010c170 0x7f824019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:18.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.710+0000 7f8244df9640 1 --2- 172.21.15.172:0/3438181002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8240103660 0x7f824019aa80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:18.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.710+0000 7f8244df9640 1 --2- 172.21.15.172:0/3438181002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8240102d40 0x7f824019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:18.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.710+0000 7f8244df9640 1 -- 172.21.15.172:0/3438181002 >> 172.21.15.172:0/3438181002 conn(0x7f82400fc7a0 msgr2=0x7f8240101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:18.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.710+0000 7f8244df9640 1 -- 172.21.15.172:0/3438181002 shutdown_connections 2024-09-10T08:44:18.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:18.710+0000 7f8244df9640 1 -- 172.21.15.172:0/3438181002 wait complete. 2024-09-10T08:44:18.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:18 smithi174 ceph-mon[34068]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:44:18.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:18 smithi174 ceph-mon[34068]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T08:44:18.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:18 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3438181002' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T08:44:19.030 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:18 smithi172 ceph-mon[30854]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:44:19.030 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:18 smithi172 ceph-mon[30854]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T08:44:19.031 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:18 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3438181002' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T08:44:19.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:18 smithi191 ceph-mon[35314]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T08:44:19.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:18 smithi191 ceph-mon[35314]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T08:44:19.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:18 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3438181002' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T08:44:19.344 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-10T08:44:19.592 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:19.984 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:19 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3438181002' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T08:44:19.984 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:19 smithi172 ceph-mon[30854]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T08:44:19.984 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:19 smithi172 ceph-mon[30854]: pgmap v288: 129 pgs: 7 creating+peering, 20 unknown, 102 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 3 op/s 2024-09-10T08:44:19.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:19 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3438181002' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T08:44:19.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:19 smithi174 ceph-mon[34068]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T08:44:19.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:19 smithi174 ceph-mon[34068]: pgmap v288: 129 pgs: 7 creating+peering, 20 unknown, 102 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 3 op/s 2024-09-10T08:44:20.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:19 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3438181002' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T08:44:20.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:19 smithi191 ceph-mon[35314]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T08:44:20.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:19 smithi191 ceph-mon[35314]: pgmap v288: 129 pgs: 7 creating+peering, 20 unknown, 102 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 3 op/s 2024-09-10T08:44:20.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.065+0000 7fab6a764640 1 -- 172.21.15.172:0/739567012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab64102780 msgr2=0x7fab641067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:20.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.065+0000 7fab6a764640 1 --2- 172.21.15.172:0/739567012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab64102780 0x7fab641067d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fab58009900 tx=0x7fab5802a410 comp rx=0 tx=0).stop 2024-09-10T08:44:20.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.065+0000 7fab6a764640 1 -- 172.21.15.172:0/739567012 shutdown_connections 2024-09-10T08:44:20.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.065+0000 7fab6a764640 1 --2- 172.21.15.172:0/739567012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab64106f00 0x7fab6410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:20.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.065+0000 7fab6a764640 1 --2- 172.21.15.172:0/739567012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab64102780 0x7fab641067d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:20.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.065+0000 7fab6a764640 1 --2- 172.21.15.172:0/739567012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab64101dd0 0x7fab641021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:20.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.065+0000 7fab6a764640 1 -- 172.21.15.172:0/739567012 >> 172.21.15.172:0/739567012 conn(0x7fab640fd890 msgr2=0x7fab640ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:20.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.065+0000 7fab6a764640 1 -- 172.21.15.172:0/739567012 shutdown_connections 2024-09-10T08:44:20.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.065+0000 7fab6a764640 1 -- 172.21.15.172:0/739567012 wait complete. 2024-09-10T08:44:20.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.065+0000 7fab6a764640 1 Processor -- start 2024-09-10T08:44:20.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.065+0000 7fab6a764640 1 -- start start 2024-09-10T08:44:20.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.066+0000 7fab6a764640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab64101dd0 0x7fab64198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:20.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.066+0000 7fab6a764640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab64102780 0x7fab64198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:20.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.066+0000 7fab6a764640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab64106f00 0x7fab6419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:20.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.066+0000 7fab6a764640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fab64111eb0 con 0x7fab64106f00 2024-09-10T08:44:20.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.066+0000 7fab6a764640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fab64111d30 con 0x7fab64101dd0 2024-09-10T08:44:20.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.066+0000 7fab6a764640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fab64112030 con 0x7fab64102780 2024-09-10T08:44:20.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.066+0000 7fab68d01640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab64106f00 0x7fab6419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:20.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.066+0000 7fab68d01640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab64106f00 0x7fab6419cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34946/0 (socket says 172.21.15.172:34946) 2024-09-10T08:44:20.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.066+0000 7fab68d01640 1 -- 172.21.15.172:0/1211629216 learned_addr learned my addr 172.21.15.172:0/1211629216 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:20.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.066+0000 7fab637fe640 1 --2- 172.21.15.172:0/1211629216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab64102780 0x7fab64198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:20.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.066+0000 7fab68d01640 1 -- 172.21.15.172:0/1211629216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab64102780 msgr2=0x7fab64198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:20.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.066+0000 7fab63fff640 1 --2- 172.21.15.172:0/1211629216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab64101dd0 0x7fab64198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:20.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.066+0000 7fab68d01640 1 --2- 172.21.15.172:0/1211629216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab64102780 0x7fab64198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:20.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.066+0000 7fab68d01640 1 -- 172.21.15.172:0/1211629216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab64101dd0 msgr2=0x7fab64198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:20.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.066+0000 7fab68d01640 1 --2- 172.21.15.172:0/1211629216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab64101dd0 0x7fab64198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:20.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.066+0000 7fab68d01640 1 -- 172.21.15.172:0/1211629216 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6419d5c0 con 0x7fab64106f00 2024-09-10T08:44:20.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.066+0000 7fab68d01640 1 --2- 172.21.15.172:0/1211629216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab64106f00 0x7fab6419cee0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00b970 tx=0x7fab5c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:20.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.067+0000 7fab617fa640 1 -- 172.21.15.172:0/1211629216 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c019070 con 0x7fab64106f00 2024-09-10T08:44:20.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.067+0000 7fab617fa640 1 -- 172.21.15.172:0/1211629216 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab5c014440 con 0x7fab64106f00 2024-09-10T08:44:20.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.067+0000 7fab6a764640 1 -- 172.21.15.172:0/1211629216 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab6419d8b0 con 0x7fab64106f00 2024-09-10T08:44:20.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.067+0000 7fab617fa640 1 -- 172.21.15.172:0/1211629216 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0075d0 con 0x7fab64106f00 2024-09-10T08:44:20.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.067+0000 7fab6a764640 1 -- 172.21.15.172:0/1211629216 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab641a5190 con 0x7fab64106f00 2024-09-10T08:44:20.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.068+0000 7fab6a764640 1 -- 172.21.15.172:0/1211629216 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab64102230 con 0x7fab64106f00 2024-09-10T08:44:20.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.069+0000 7fab617fa640 1 -- 172.21.15.172:0/1211629216 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fab5c01c070 con 0x7fab64106f00 2024-09-10T08:44:20.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.069+0000 7fab617fa640 1 --2- 172.21.15.172:0/1211629216 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fab4c0a2c00 0x7fab4c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:20.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.070+0000 7fab63fff640 1 --2- 172.21.15.172:0/1211629216 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fab4c0a2c00 0x7fab4c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:20.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.070+0000 7fab617fa640 1 -- 172.21.15.172:0/1211629216 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7fab5c0ceca0 con 0x7fab64106f00 2024-09-10T08:44:20.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.070+0000 7fab63fff640 1 --2- 172.21.15.172:0/1211629216 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fab4c0a2c00 0x7fab4c0a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fab5400a810 tx=0x7fab54008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:20.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.073+0000 7fab617fa640 1 -- 172.21.15.172:0/1211629216 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab5c09b9e0 con 0x7fab64106f00 2024-09-10T08:44:20.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.274+0000 7fab6a764640 1 -- 172.21.15.172:0/1211629216 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fab6410a0e0 con 0x7fab64106f00 2024-09-10T08:44:20.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.719+0000 7fab617fa640 1 -- 172.21.15.172:0/1211629216 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fab5c09b2e0 con 0x7fab64106f00 2024-09-10T08:44:20.720 INFO:teuthology.orchestra.run.smithi172.stderr:added data pool 4 to fsmap 2024-09-10T08:44:20.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.722+0000 7fab6a764640 1 -- 172.21.15.172:0/1211629216 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fab4c0a2c00 msgr2=0x7fab4c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:20.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.722+0000 7fab6a764640 1 --2- 172.21.15.172:0/1211629216 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fab4c0a2c00 0x7fab4c0a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fab5400a810 tx=0x7fab54008040 comp rx=0 tx=0).stop 2024-09-10T08:44:20.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.722+0000 7fab6a764640 1 -- 172.21.15.172:0/1211629216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab64106f00 msgr2=0x7fab6419cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:20.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.722+0000 7fab6a764640 1 --2- 172.21.15.172:0/1211629216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab64106f00 0x7fab6419cee0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00b970 tx=0x7fab5c00be30 comp rx=0 tx=0).stop 2024-09-10T08:44:20.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.722+0000 7fab6a764640 1 -- 172.21.15.172:0/1211629216 shutdown_connections 2024-09-10T08:44:20.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.722+0000 7fab6a764640 1 --2- 172.21.15.172:0/1211629216 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fab4c0a2c00 0x7fab4c0a50c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:20.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.722+0000 7fab6a764640 1 --2- 172.21.15.172:0/1211629216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab64106f00 0x7fab6419cee0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:20.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.722+0000 7fab6a764640 1 --2- 172.21.15.172:0/1211629216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab64102780 0x7fab64198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:20.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.722+0000 7fab6a764640 1 --2- 172.21.15.172:0/1211629216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab64101dd0 0x7fab64198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:20.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.722+0000 7fab6a764640 1 -- 172.21.15.172:0/1211629216 >> 172.21.15.172:0/1211629216 conn(0x7fab640fd890 msgr2=0x7fab640fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:20.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.722+0000 7fab6a764640 1 -- 172.21.15.172:0/1211629216 shutdown_connections 2024-09-10T08:44:20.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:20.722+0000 7fab6a764640 1 -- 172.21.15.172:0/1211629216 wait complete. 2024-09-10T08:44:20.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:20 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1211629216' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T08:44:21.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:20 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1211629216' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T08:44:21.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:20 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1211629216' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T08:44:21.462 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:44:21.710 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:21.974 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:21 smithi172 ceph-mon[30854]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T08:44:21.974 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:21 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1211629216' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T08:44:21.974 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:21 smithi172 ceph-mon[30854]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:44:21.974 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:21 smithi172 ceph-mon[30854]: pgmap v290: 129 pgs: 7 creating+peering, 4 unknown, 118 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:21.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:21 smithi174 ceph-mon[34068]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T08:44:21.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:21 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1211629216' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T08:44:21.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:21 smithi174 ceph-mon[34068]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:44:21.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:21 smithi174 ceph-mon[34068]: pgmap v290: 129 pgs: 7 creating+peering, 4 unknown, 118 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:22.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:21 smithi191 ceph-mon[35314]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T08:44:22.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:21 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1211629216' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T08:44:22.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:21 smithi191 ceph-mon[35314]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:44:22.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:21 smithi191 ceph-mon[35314]: pgmap v290: 129 pgs: 7 creating+peering, 4 unknown, 118 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:22.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.206+0000 7f7d8d08d640 1 -- 172.21.15.172:0/3675889932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d88102790 msgr2=0x7f7d881067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:22.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.206+0000 7f7d8d08d640 1 --2- 172.21.15.172:0/3675889932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d88102790 0x7f7d881067e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009900 tx=0x7f7d7002a410 comp rx=0 tx=0).stop 2024-09-10T08:44:22.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.206+0000 7f7d8d08d640 1 -- 172.21.15.172:0/3675889932 shutdown_connections 2024-09-10T08:44:22.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.206+0000 7f7d8d08d640 1 --2- 172.21.15.172:0/3675889932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d88106f10 0x7f7d8810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:22.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.206+0000 7f7d8d08d640 1 --2- 172.21.15.172:0/3675889932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d88102790 0x7f7d881067e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:22.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.206+0000 7f7d8d08d640 1 --2- 172.21.15.172:0/3675889932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d88101de0 0x7f7d881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:22.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.206+0000 7f7d8d08d640 1 -- 172.21.15.172:0/3675889932 >> 172.21.15.172:0/3675889932 conn(0x7f7d880fd8c0 msgr2=0x7f7d880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:22.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.206+0000 7f7d8d08d640 1 -- 172.21.15.172:0/3675889932 shutdown_connections 2024-09-10T08:44:22.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.206+0000 7f7d8d08d640 1 -- 172.21.15.172:0/3675889932 wait complete. 2024-09-10T08:44:22.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.207+0000 7f7d8d08d640 1 Processor -- start 2024-09-10T08:44:22.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.207+0000 7f7d8d08d640 1 -- start start 2024-09-10T08:44:22.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.207+0000 7f7d8d08d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d88101de0 0x7f7d880744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:22.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.207+0000 7f7d8d08d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d88102790 0x7f7d88074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:22.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.207+0000 7f7d8d08d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d88106f10 0x7f7d88074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:22.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.207+0000 7f7d8d08d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7d88111e40 con 0x7f7d88106f10 2024-09-10T08:44:22.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.207+0000 7f7d8d08d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d88111cc0 con 0x7f7d88101de0 2024-09-10T08:44:22.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.207+0000 7f7d8d08d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7d88111fc0 con 0x7f7d88102790 2024-09-10T08:44:22.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.207+0000 7f7d8759e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d88106f10 0x7f7d88074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:22.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.207+0000 7f7d8759e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d88106f10 0x7f7d88074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34952/0 (socket says 172.21.15.172:34952) 2024-09-10T08:44:22.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.207+0000 7f7d8759e640 1 -- 172.21.15.172:0/2433317002 learned_addr learned my addr 172.21.15.172:0/2433317002 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:22.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.207+0000 7f7d86d9d640 1 --2- 172.21.15.172:0/2433317002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d88101de0 0x7f7d880744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:22.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.207+0000 7f7d8759e640 1 -- 172.21.15.172:0/2433317002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d88102790 msgr2=0x7f7d88074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:22.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.208+0000 7f7d8659c640 1 --2- 172.21.15.172:0/2433317002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d88102790 0x7f7d88074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:22.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.208+0000 7f7d8759e640 1 --2- 172.21.15.172:0/2433317002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d88102790 0x7f7d88074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:22.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.208+0000 7f7d8759e640 1 -- 172.21.15.172:0/2433317002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d88101de0 msgr2=0x7f7d880744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:22.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.208+0000 7f7d8759e640 1 --2- 172.21.15.172:0/2433317002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d88101de0 0x7f7d880744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:22.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.208+0000 7f7d8759e640 1 -- 172.21.15.172:0/2433317002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d881a6880 con 0x7f7d88106f10 2024-09-10T08:44:22.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.208+0000 7f7d86d9d640 1 --2- 172.21.15.172:0/2433317002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d88101de0 0x7f7d880744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:44:22.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.208+0000 7f7d8659c640 1 --2- 172.21.15.172:0/2433317002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d88102790 0x7f7d88074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:44:22.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.208+0000 7f7d8759e640 1 --2- 172.21.15.172:0/2433317002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d88106f10 0x7f7d88074f70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7d8000b970 tx=0x7f7d8000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:22.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.208+0000 7f7d7ffff640 1 -- 172.21.15.172:0/2433317002 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80019070 con 0x7f7d88106f10 2024-09-10T08:44:22.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.208+0000 7f7d8d08d640 1 -- 172.21.15.172:0/2433317002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d881a6b70 con 0x7f7d88106f10 2024-09-10T08:44:22.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.208+0000 7f7d7ffff640 1 -- 172.21.15.172:0/2433317002 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d80014440 con 0x7f7d88106f10 2024-09-10T08:44:22.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.208+0000 7f7d7ffff640 1 -- 172.21.15.172:0/2433317002 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d800075d0 con 0x7f7d88106f10 2024-09-10T08:44:22.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.208+0000 7f7d8d08d640 1 -- 172.21.15.172:0/2433317002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d881a7100 con 0x7f7d88106f10 2024-09-10T08:44:22.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.209+0000 7f7d8d08d640 1 -- 172.21.15.172:0/2433317002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d881021c0 con 0x7f7d88106f10 2024-09-10T08:44:22.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.210+0000 7f7d7ffff640 1 -- 172.21.15.172:0/2433317002 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f7d8001c070 con 0x7f7d88106f10 2024-09-10T08:44:22.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.211+0000 7f7d7ffff640 1 --2- 172.21.15.172:0/2433317002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7d740a2b80 0x7f7d740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:22.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.211+0000 7f7d86d9d640 1 --2- 172.21.15.172:0/2433317002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7d740a2b80 0x7f7d740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:22.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.211+0000 7f7d7ffff640 1 -- 172.21.15.172:0/2433317002 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f7d800ce8c0 con 0x7f7d88106f10 2024-09-10T08:44:22.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.211+0000 7f7d86d9d640 1 --2- 172.21.15.172:0/2433317002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7d740a2b80 0x7f7d740a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009b40 tx=0x7f7d78005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:22.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.215+0000 7f7d7ffff640 1 -- 172.21.15.172:0/2433317002 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d8009b5e0 con 0x7f7d88106f10 2024-09-10T08:44:22.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.417+0000 7f7d8d08d640 1 -- 172.21.15.172:0/2433317002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d8802d7c0 con 0x7f7d88106f10 2024-09-10T08:44:22.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.417+0000 7f7d7ffff640 1 -- 172.21.15.172:0/2433317002 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11866 (secure 0 0 0) 0x7f7d8009aee0 con 0x7f7d88106f10 2024-09-10T08:44:22.418 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:44:22.418 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":17,"btime":"2024-09-10T08:44:20:714228+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:44:20.276118+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24488,"qdb_cluster":[24488]},"id":1}]} 2024-09-10T08:44:22.418 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 17 2024-09-10T08:44:22.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.420+0000 7f7d8d08d640 1 -- 172.21.15.172:0/2433317002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7d740a2b80 msgr2=0x7f7d740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:22.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.420+0000 7f7d8d08d640 1 --2- 172.21.15.172:0/2433317002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7d740a2b80 0x7f7d740a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009b40 tx=0x7f7d78005e50 comp rx=0 tx=0).stop 2024-09-10T08:44:22.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.420+0000 7f7d8d08d640 1 -- 172.21.15.172:0/2433317002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d88106f10 msgr2=0x7f7d88074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:22.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.420+0000 7f7d8d08d640 1 --2- 172.21.15.172:0/2433317002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d88106f10 0x7f7d88074f70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7d8000b970 tx=0x7f7d8000be30 comp rx=0 tx=0).stop 2024-09-10T08:44:22.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.420+0000 7f7d8d08d640 1 -- 172.21.15.172:0/2433317002 shutdown_connections 2024-09-10T08:44:22.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.420+0000 7f7d8d08d640 1 --2- 172.21.15.172:0/2433317002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7d740a2b80 0x7f7d740a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:22.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.420+0000 7f7d8d08d640 1 --2- 172.21.15.172:0/2433317002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d88106f10 0x7f7d88074f70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:22.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.420+0000 7f7d8d08d640 1 --2- 172.21.15.172:0/2433317002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d88102790 0x7f7d88074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:22.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.420+0000 7f7d8d08d640 1 --2- 172.21.15.172:0/2433317002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d88101de0 0x7f7d880744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:22.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.420+0000 7f7d8d08d640 1 -- 172.21.15.172:0/2433317002 >> 172.21.15.172:0/2433317002 conn(0x7f7d880fd8c0 msgr2=0x7f7d880fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:22.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.420+0000 7f7d8d08d640 1 -- 172.21.15.172:0/2433317002 shutdown_connections 2024-09-10T08:44:22.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:22.420+0000 7f7d8d08d640 1 -- 172.21.15.172:0/2433317002 wait complete. 2024-09-10T08:44:22.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:22 smithi174 ceph-mon[34068]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T08:44:22.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:22 smithi174 ceph-mon[34068]: Cluster is now healthy 2024-09-10T08:44:22.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:22 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2433317002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:23.037 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:22 smithi172 ceph-mon[30854]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T08:44:23.037 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:22 smithi172 ceph-mon[30854]: Cluster is now healthy 2024-09-10T08:44:23.037 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:22 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2433317002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:23.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:22 smithi191 ceph-mon[35314]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T08:44:23.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:22 smithi191 ceph-mon[35314]: Cluster is now healthy 2024-09-10T08:44:23.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:22 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2433317002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:23.102 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:44:23.349 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:23.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.850+0000 7fbb01a1a640 1 -- 172.21.15.172:0/1849647015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbafc106a80 msgr2=0x7fbafc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.850+0000 7fbb01a1a640 1 --2- 172.21.15.172:0/1849647015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbafc106a80 0x7fbafc106e60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009930 tx=0x7fbaec02a420 comp rx=0 tx=0).stop 2024-09-10T08:44:23.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.850+0000 7fbb01a1a640 1 -- 172.21.15.172:0/1849647015 shutdown_connections 2024-09-10T08:44:23.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.850+0000 7fbb01a1a640 1 --2- 172.21.15.172:0/1849647015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbafc101420 0x7fbafc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.850+0000 7fbb01a1a640 1 --2- 172.21.15.172:0/1849647015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbafc100a80 0x7fbafc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.850+0000 7fbb01a1a640 1 --2- 172.21.15.172:0/1849647015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbafc106a80 0x7fbafc106e60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.850+0000 7fbb01a1a640 1 -- 172.21.15.172:0/1849647015 >> 172.21.15.172:0/1849647015 conn(0x7fbafc0fc7a0 msgr2=0x7fbafc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:23.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.850+0000 7fbb01a1a640 1 -- 172.21.15.172:0/1849647015 shutdown_connections 2024-09-10T08:44:23.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.850+0000 7fbb01a1a640 1 -- 172.21.15.172:0/1849647015 wait complete. 2024-09-10T08:44:23.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.851+0000 7fbb01a1a640 1 Processor -- start 2024-09-10T08:44:23.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.851+0000 7fbb01a1a640 1 -- start start 2024-09-10T08:44:23.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.851+0000 7fbb01a1a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbafc100a80 0x7fbafc197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.851+0000 7fbb01a1a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbafc101420 0x7fbafc197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.851+0000 7fbb01a1a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbafc106a80 0x7fbafc19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.851+0000 7fbb01a1a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc110d50 con 0x7fbafc101420 2024-09-10T08:44:23.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.851+0000 7fbb01a1a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc110bd0 con 0x7fbafc106a80 2024-09-10T08:44:23.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.851+0000 7fbb01a1a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbafc110ed0 con 0x7fbafc100a80 2024-09-10T08:44:23.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.851+0000 7fbaf3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbafc101420 0x7fbafc197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.851+0000 7fbaf3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbafc101420 0x7fbafc197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34966/0 (socket says 172.21.15.172:34966) 2024-09-10T08:44:23.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.851+0000 7fbaf3fff640 1 -- 172.21.15.172:0/3064285665 learned_addr learned my addr 172.21.15.172:0/3064285665 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:23.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.851+0000 7fbafaffd640 1 --2- 172.21.15.172:0/3064285665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbafc100a80 0x7fbafc197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.851+0000 7fbaf3fff640 1 -- 172.21.15.172:0/3064285665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbafc100a80 msgr2=0x7fbafc197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.851+0000 7fbaf3fff640 1 --2- 172.21.15.172:0/3064285665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbafc100a80 0x7fbafc197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.851+0000 7fbaf3fff640 1 -- 172.21.15.172:0/3064285665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbafc106a80 msgr2=0x7fbafc19bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:23.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.851+0000 7fbaf3fff640 1 --2- 172.21.15.172:0/3064285665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbafc106a80 0x7fbafc19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:23.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.851+0000 7fbaf3fff640 1 -- 172.21.15.172:0/3064285665 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbafc19c2e0 con 0x7fbafc101420 2024-09-10T08:44:23.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.852+0000 7fbaf3fff640 1 --2- 172.21.15.172:0/3064285665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbafc101420 0x7fbafc197870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbae400d950 tx=0x7fbae400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:23.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.852+0000 7fbb00a18640 1 -- 172.21.15.172:0/3064285665 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4017070 con 0x7fbafc101420 2024-09-10T08:44:23.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.852+0000 7fbb00a18640 1 -- 172.21.15.172:0/3064285665 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbae4009c30 con 0x7fbafc101420 2024-09-10T08:44:23.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.852+0000 7fbb01a1a640 1 -- 172.21.15.172:0/3064285665 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbafc19c5d0 con 0x7fbafc101420 2024-09-10T08:44:23.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.852+0000 7fbb00a18640 1 -- 172.21.15.172:0/3064285665 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae400f620 con 0x7fbafc101420 2024-09-10T08:44:23.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.852+0000 7fbb01a1a640 1 -- 172.21.15.172:0/3064285665 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbafc1a3eb0 con 0x7fbafc101420 2024-09-10T08:44:23.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.853+0000 7fbb01a1a640 1 -- 172.21.15.172:0/3064285665 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbafc100ee0 con 0x7fbafc101420 2024-09-10T08:44:23.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.855+0000 7fbb00a18640 1 -- 172.21.15.172:0/3064285665 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbae40109a0 con 0x7fbafc101420 2024-09-10T08:44:23.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.855+0000 7fbb00a18640 1 --2- 172.21.15.172:0/3064285665 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbae80a2c00 0x7fbae80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:23.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.856+0000 7fbafaffd640 1 --2- 172.21.15.172:0/3064285665 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbae80a2c00 0x7fbae80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:23.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.856+0000 7fbb00a18640 1 -- 172.21.15.172:0/3064285665 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fbae40cef00 con 0x7fbafc101420 2024-09-10T08:44:23.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.856+0000 7fbafaffd640 1 --2- 172.21.15.172:0/3064285665 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbae80a2c00 0x7fbae80a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbaec000c00 tx=0x7fbaec032c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:23.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:23.859+0000 7fbb00a18640 1 -- 172.21.15.172:0/3064285665 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbae4009de0 con 0x7fbafc101420 2024-09-10T08:44:23.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:23 smithi174 ceph-mon[34068]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:24.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:24.024+0000 7fbb01a1a640 1 -- 172.21.15.172:0/3064285665 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbafc198690 con 0x7fbafc101420 2024-09-10T08:44:24.025 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:23 smithi172 ceph-mon[30854]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:24.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:24.025+0000 7fbb00a18640 1 -- 172.21.15.172:0/3064285665 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24294 (secure 0 0 0) 0x7fbafc198690 con 0x7fbafc101420 2024-09-10T08:44:24.026 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:44:24.026 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":88,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","created":"2024-09-10T08:36:33.083529+0000","modified":"2024-09-10T08:44:20.275987+0000","last_up_change":"2024-09-10T08:43:19.615640+0000","last_in_change":"2024-09-10T08:42:56.656115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:39:31.328306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:44:07.466637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:44:09.520569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:44:15.926057+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6803","nonce":2678426952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6805","nonce":2678426952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6809","nonce":2678426952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6807","nonce":2678426952}]},"public_addr":"172.21.15.172:6803/2678426952","cluster_addr":"172.21.15.172:6805/2678426952","heartbeat_back_addr":"172.21.15.172:6809/2678426952","heartbeat_front_addr":"172.21.15.172:6807/2678426952","state":["exists","up"]},{"osd":1,"uuid":"55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6801","nonce":3279661439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6803","nonce":3279661439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6807","nonce":3279661439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6805","nonce":3279661439}]},"public_addr":"172.21.15.174:6801/3279661439","cluster_addr":"172.21.15.174:6803/3279661439","heartbeat_back_addr":"172.21.15.174:6807/3279661439","heartbeat_front_addr":"172.21.15.174:6805/3279661439","state":["exists","up"]},{"osd":2,"uuid":"f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6801","nonce":1963796290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6803","nonce":1963796290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6807","nonce":1963796290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6805","nonce":1963796290}]},"public_addr":"172.21.15.191:6801/1963796290","cluster_addr":"172.21.15.191:6803/1963796290","heartbeat_back_addr":"172.21.15.191:6807/1963796290","heartbeat_front_addr":"172.21.15.191:6805/1963796290","state":["exists","up"]},{"osd":3,"uuid":"15ecca3f-bc6b-4a06-b283-b515384beb19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6811","nonce":869546480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6813","nonce":869546480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6817","nonce":869546480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6815","nonce":869546480}]},"public_addr":"172.21.15.172:6811/869546480","cluster_addr":"172.21.15.172:6813/869546480","heartbeat_back_addr":"172.21.15.172:6817/869546480","heartbeat_front_addr":"172.21.15.172:6815/869546480","state":["exists","up"]},{"osd":4,"uuid":"8c2c1d40-72ad-4a15-ab72-54871c7e2427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6809","nonce":961007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6811","nonce":961007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6815","nonce":961007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6813","nonce":961007949}]},"public_addr":"172.21.15.174:6809/961007949","cluster_addr":"172.21.15.174:6811/961007949","heartbeat_back_addr":"172.21.15.174:6815/961007949","heartbeat_front_addr":"172.21.15.174:6813/961007949","state":["exists","up"]},{"osd":5,"uuid":"4de45a1d-ef0e-4bfe-ba15-5eeeb8806155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6809","nonce":4216938371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6811","nonce":4216938371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6815","nonce":4216938371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6813","nonce":4216938371}]},"public_addr":"172.21.15.191:6809/4216938371","cluster_addr":"172.21.15.191:6811/4216938371","heartbeat_back_addr":"172.21.15.191:6815/4216938371","heartbeat_front_addr":"172.21.15.191:6813/4216938371","state":["exists","up"]},{"osd":6,"uuid":"60a9938a-f696-4db2-932c-402997543b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6819","nonce":3210762009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6821","nonce":3210762009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6825","nonce":3210762009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6823","nonce":3210762009}]},"public_addr":"172.21.15.172:6819/3210762009","cluster_addr":"172.21.15.172:6821/3210762009","heartbeat_back_addr":"172.21.15.172:6825/3210762009","heartbeat_front_addr":"172.21.15.172:6823/3210762009","state":["exists","up"]},{"osd":7,"uuid":"7f0202f3-971e-472f-9c4d-ae3c24d39f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6817","nonce":1461923581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6819","nonce":1461923581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6823","nonce":1461923581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6821","nonce":1461923581}]},"public_addr":"172.21.15.174:6817/1461923581","cluster_addr":"172.21.15.174:6819/1461923581","heartbeat_back_addr":"172.21.15.174:6823/1461923581","heartbeat_front_addr":"172.21.15.174:6821/1461923581","state":["exists","up"]},{"osd":8,"uuid":"b4e7327f-ca9f-4833-9cd3-b8529b3d9761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6817","nonce":3675210430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6819","nonce":3675210430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6823","nonce":3675210430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6821","nonce":3675210430}]},"public_addr":"172.21.15.191:6817/3675210430","cluster_addr":"172.21.15.191:6819/3675210430","heartbeat_back_addr":"172.21.15.191:6823/3675210430","heartbeat_front_addr":"172.21.15.191:6821/3675210430","state":["exists","up"]},{"osd":9,"uuid":"9f6153f2-f9db-4c00-a659-c932628016f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6827","nonce":1181268619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6829","nonce":1181268619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6833","nonce":1181268619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6831","nonce":1181268619}]},"public_addr":"172.21.15.172:6827/1181268619","cluster_addr":"172.21.15.172:6829/1181268619","heartbeat_back_addr":"172.21.15.172:6833/1181268619","heartbeat_front_addr":"172.21.15.172:6831/1181268619","state":["exists","up"]},{"osd":10,"uuid":"040a013a-4c3e-442f-bea6-12cbe517716d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6825","nonce":1238812525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6827","nonce":1238812525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6831","nonce":1238812525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6829","nonce":1238812525}]},"public_addr":"172.21.15.174:6825/1238812525","cluster_addr":"172.21.15.174:6827/1238812525","heartbeat_back_addr":"172.21.15.174:6831/1238812525","heartbeat_front_addr":"172.21.15.174:6829/1238812525","state":["exists","up"]},{"osd":11,"uuid":"ae6535bd-1510-4bf3-bf4b-1a2a256add43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6825","nonce":2162719997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6827","nonce":2162719997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6831","nonce":2162719997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6829","nonce":2162719997}]},"public_addr":"172.21.15.191:6825/2162719997","cluster_addr":"172.21.15.191:6827/2162719997","heartbeat_back_addr":"172.21.15.191:6831/2162719997","heartbeat_front_addr":"172.21.15.191:6829/2162719997","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:38:38.812596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:04.765652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:29.103966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:56.085849+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:21.641784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:47.750376+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:11.640697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:36.638840+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:02.167446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:28.655804+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:52.749676+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:43:17.783908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:0/977245917":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/3919952878":"2024-09-11T08:36:54.887256+0000","172.21.15.172:6800/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/1525385361":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/2043423408":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/839344299":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6800/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/3251817286":"2024-09-11T08:36:54.887256+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:44:24.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:24.027+0000 7fbb01a1a640 1 -- 172.21.15.172:0/3064285665 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbae80a2c00 msgr2=0x7fbae80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:24.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:24.027+0000 7fbb01a1a640 1 --2- 172.21.15.172:0/3064285665 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbae80a2c00 0x7fbae80a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbaec000c00 tx=0x7fbaec032c80 comp rx=0 tx=0).stop 2024-09-10T08:44:24.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:24.027+0000 7fbb01a1a640 1 -- 172.21.15.172:0/3064285665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbafc101420 msgr2=0x7fbafc197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:24.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:24.027+0000 7fbb01a1a640 1 --2- 172.21.15.172:0/3064285665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbafc101420 0x7fbafc197870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbae400d950 tx=0x7fbae400de10 comp rx=0 tx=0).stop 2024-09-10T08:44:24.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:24.027+0000 7fbb01a1a640 1 -- 172.21.15.172:0/3064285665 shutdown_connections 2024-09-10T08:44:24.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:24.027+0000 7fbb01a1a640 1 --2- 172.21.15.172:0/3064285665 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbae80a2c00 0x7fbae80a50c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:24.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:24.027+0000 7fbb01a1a640 1 --2- 172.21.15.172:0/3064285665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbafc106a80 0x7fbafc19bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:24.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:24.027+0000 7fbb01a1a640 1 --2- 172.21.15.172:0/3064285665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbafc101420 0x7fbafc197870 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:24.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:24.027+0000 7fbb01a1a640 1 --2- 172.21.15.172:0/3064285665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbafc100a80 0x7fbafc197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:24.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:24.027+0000 7fbb01a1a640 1 -- 172.21.15.172:0/3064285665 >> 172.21.15.172:0/3064285665 conn(0x7fbafc0fc7a0 msgr2=0x7fbafc0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:24.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:24.028+0000 7fbb01a1a640 1 -- 172.21.15.172:0/3064285665 shutdown_connections 2024-09-10T08:44:24.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:24.028+0000 7fbb01a1a640 1 -- 172.21.15.172:0/3064285665 wait complete. 2024-09-10T08:44:24.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:23 smithi191 ceph-mon[35314]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:24.705 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:44:24.952 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:24.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:24 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3064285665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:44:25.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:24 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3064285665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:44:25.073 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:24 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3064285665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:44:25.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.401+0000 7f1f0dee7640 1 -- 172.21.15.172:0/3686472828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f08100e20 msgr2=0x7f1f08101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:25.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.401+0000 7f1f0dee7640 1 --2- 172.21.15.172:0/3686472828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f08100e20 0x7f1f08101200 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009900 tx=0x7f1ef802a480 comp rx=0 tx=0).stop 2024-09-10T08:44:25.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.401+0000 7f1f0dee7640 1 -- 172.21.15.172:0/3686472828 shutdown_connections 2024-09-10T08:44:25.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.401+0000 7f1f0dee7640 1 --2- 172.21.15.172:0/3686472828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f08102ef0 0x7f1f0810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:25.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.401+0000 7f1f0dee7640 1 --2- 172.21.15.172:0/3686472828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f08102570 0x7f1f081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:25.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.401+0000 7f1f0dee7640 1 --2- 172.21.15.172:0/3686472828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f08100e20 0x7f1f08101200 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:25.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.401+0000 7f1f0dee7640 1 -- 172.21.15.172:0/3686472828 >> 172.21.15.172:0/3686472828 conn(0x7f1f080fc7d0 msgr2=0x7f1f080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:25.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.401+0000 7f1f0dee7640 1 -- 172.21.15.172:0/3686472828 shutdown_connections 2024-09-10T08:44:25.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.401+0000 7f1f0dee7640 1 -- 172.21.15.172:0/3686472828 wait complete. 2024-09-10T08:44:25.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.401+0000 7f1f0dee7640 1 Processor -- start 2024-09-10T08:44:25.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.401+0000 7f1f0dee7640 1 -- start start 2024-09-10T08:44:25.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f0dee7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f08100e20 0x7f1f0819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:25.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f0dee7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f08102570 0x7f1f0819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:25.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f0dee7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f08102ef0 0x7f1f0819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:25.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f0dee7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08110ed0 con 0x7f1f08100e20 2024-09-10T08:44:25.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f0dee7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08110d50 con 0x7f1f08102570 2024-09-10T08:44:25.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f0dee7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08111050 con 0x7f1f08102ef0 2024-09-10T08:44:25.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f077fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f08100e20 0x7f1f0819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:25.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f077fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f08100e20 0x7f1f0819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34980/0 (socket says 172.21.15.172:34980) 2024-09-10T08:44:25.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f077fe640 1 -- 172.21.15.172:0/3645051444 learned_addr learned my addr 172.21.15.172:0/3645051444 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:25.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1efffff640 1 --2- 172.21.15.172:0/3645051444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f08102570 0x7f1f0819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:25.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f07fff640 1 --2- 172.21.15.172:0/3645051444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f08102ef0 0x7f1f0819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:25.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f077fe640 1 -- 172.21.15.172:0/3645051444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f08102ef0 msgr2=0x7f1f0819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:25.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f077fe640 1 --2- 172.21.15.172:0/3645051444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f08102ef0 0x7f1f0819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:25.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f077fe640 1 -- 172.21.15.172:0/3645051444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f08102570 msgr2=0x7f1f0819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:25.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f077fe640 1 --2- 172.21.15.172:0/3645051444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f08102570 0x7f1f0819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:25.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f077fe640 1 -- 172.21.15.172:0/3645051444 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0819f650 con 0x7f1f08100e20 2024-09-10T08:44:25.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f077fe640 1 --2- 172.21.15.172:0/3645051444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f08100e20 0x7f1f0819a6a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8006010 tx=0x7f1ef80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:25.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f057fa640 1 -- 172.21.15.172:0/3645051444 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef803f070 con 0x7f1f08100e20 2024-09-10T08:44:25.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f057fa640 1 -- 172.21.15.172:0/3645051444 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ef8003aa0 con 0x7f1f08100e20 2024-09-10T08:44:25.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f0dee7640 1 -- 172.21.15.172:0/3645051444 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0819f8e0 con 0x7f1f08100e20 2024-09-10T08:44:25.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f057fa640 1 -- 172.21.15.172:0/3645051444 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8035030 con 0x7f1f08100e20 2024-09-10T08:44:25.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.402+0000 7f1f0dee7640 1 -- 172.21.15.172:0/3645051444 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f081a7220 con 0x7f1f08100e20 2024-09-10T08:44:25.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.403+0000 7f1f0dee7640 1 -- 172.21.15.172:0/3645051444 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f08101200 con 0x7f1f08100e20 2024-09-10T08:44:25.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.405+0000 7f1f057fa640 1 -- 172.21.15.172:0/3645051444 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1ef8042070 con 0x7f1f08100e20 2024-09-10T08:44:25.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.405+0000 7f1f057fa640 1 --2- 172.21.15.172:0/3645051444 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1ef40a2c00 0x7f1ef40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:25.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.405+0000 7f1efffff640 1 --2- 172.21.15.172:0/3645051444 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1ef40a2c00 0x7f1ef40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:25.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.405+0000 7f1f057fa640 1 -- 172.21.15.172:0/3645051444 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f1ef80f01f0 con 0x7f1f08100e20 2024-09-10T08:44:25.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.406+0000 7f1efffff640 1 --2- 172.21.15.172:0/3645051444 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1ef40a2c00 0x7f1ef40a50c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1f08104c40 tx=0x7f1ef0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:25.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.408+0000 7f1f057fa640 1 -- 172.21.15.172:0/3645051444 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ef80bcf10 con 0x7f1f08100e20 2024-09-10T08:44:25.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.575+0000 7f1f0dee7640 1 -- 172.21.15.172:0/3645051444 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f081a73f0 con 0x7f1f08100e20 2024-09-10T08:44:25.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.577+0000 7f1f057fa640 1 -- 172.21.15.172:0/3645051444 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24294 (secure 0 0 0) 0x7f1ef80bc810 con 0x7f1f08100e20 2024-09-10T08:44:25.578 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:44:25.578 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":88,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","created":"2024-09-10T08:36:33.083529+0000","modified":"2024-09-10T08:44:20.275987+0000","last_up_change":"2024-09-10T08:43:19.615640+0000","last_in_change":"2024-09-10T08:42:56.656115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:39:31.328306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:44:07.466637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:44:09.520569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:44:15.926057+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6803","nonce":2678426952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6805","nonce":2678426952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6809","nonce":2678426952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6807","nonce":2678426952}]},"public_addr":"172.21.15.172:6803/2678426952","cluster_addr":"172.21.15.172:6805/2678426952","heartbeat_back_addr":"172.21.15.172:6809/2678426952","heartbeat_front_addr":"172.21.15.172:6807/2678426952","state":["exists","up"]},{"osd":1,"uuid":"55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6801","nonce":3279661439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6803","nonce":3279661439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6807","nonce":3279661439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6805","nonce":3279661439}]},"public_addr":"172.21.15.174:6801/3279661439","cluster_addr":"172.21.15.174:6803/3279661439","heartbeat_back_addr":"172.21.15.174:6807/3279661439","heartbeat_front_addr":"172.21.15.174:6805/3279661439","state":["exists","up"]},{"osd":2,"uuid":"f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6801","nonce":1963796290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6803","nonce":1963796290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6807","nonce":1963796290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6805","nonce":1963796290}]},"public_addr":"172.21.15.191:6801/1963796290","cluster_addr":"172.21.15.191:6803/1963796290","heartbeat_back_addr":"172.21.15.191:6807/1963796290","heartbeat_front_addr":"172.21.15.191:6805/1963796290","state":["exists","up"]},{"osd":3,"uuid":"15ecca3f-bc6b-4a06-b283-b515384beb19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6811","nonce":869546480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6813","nonce":869546480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6817","nonce":869546480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6815","nonce":869546480}]},"public_addr":"172.21.15.172:6811/869546480","cluster_addr":"172.21.15.172:6813/869546480","heartbeat_back_addr":"172.21.15.172:6817/869546480","heartbeat_front_addr":"172.21.15.172:6815/869546480","state":["exists","up"]},{"osd":4,"uuid":"8c2c1d40-72ad-4a15-ab72-54871c7e2427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6809","nonce":961007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6811","nonce":961007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6815","nonce":961007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6813","nonce":961007949}]},"public_addr":"172.21.15.174:6809/961007949","cluster_addr":"172.21.15.174:6811/961007949","heartbeat_back_addr":"172.21.15.174:6815/961007949","heartbeat_front_addr":"172.21.15.174:6813/961007949","state":["exists","up"]},{"osd":5,"uuid":"4de45a1d-ef0e-4bfe-ba15-5eeeb8806155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6809","nonce":4216938371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6811","nonce":4216938371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6815","nonce":4216938371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6813","nonce":4216938371}]},"public_addr":"172.21.15.191:6809/4216938371","cluster_addr":"172.21.15.191:6811/4216938371","heartbeat_back_addr":"172.21.15.191:6815/4216938371","heartbeat_front_addr":"172.21.15.191:6813/4216938371","state":["exists","up"]},{"osd":6,"uuid":"60a9938a-f696-4db2-932c-402997543b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6819","nonce":3210762009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6821","nonce":3210762009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6825","nonce":3210762009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6823","nonce":3210762009}]},"public_addr":"172.21.15.172:6819/3210762009","cluster_addr":"172.21.15.172:6821/3210762009","heartbeat_back_addr":"172.21.15.172:6825/3210762009","heartbeat_front_addr":"172.21.15.172:6823/3210762009","state":["exists","up"]},{"osd":7,"uuid":"7f0202f3-971e-472f-9c4d-ae3c24d39f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6817","nonce":1461923581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6819","nonce":1461923581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6823","nonce":1461923581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6821","nonce":1461923581}]},"public_addr":"172.21.15.174:6817/1461923581","cluster_addr":"172.21.15.174:6819/1461923581","heartbeat_back_addr":"172.21.15.174:6823/1461923581","heartbeat_front_addr":"172.21.15.174:6821/1461923581","state":["exists","up"]},{"osd":8,"uuid":"b4e7327f-ca9f-4833-9cd3-b8529b3d9761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6817","nonce":3675210430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6819","nonce":3675210430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6823","nonce":3675210430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6821","nonce":3675210430}]},"public_addr":"172.21.15.191:6817/3675210430","cluster_addr":"172.21.15.191:6819/3675210430","heartbeat_back_addr":"172.21.15.191:6823/3675210430","heartbeat_front_addr":"172.21.15.191:6821/3675210430","state":["exists","up"]},{"osd":9,"uuid":"9f6153f2-f9db-4c00-a659-c932628016f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6827","nonce":1181268619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6829","nonce":1181268619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6833","nonce":1181268619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6831","nonce":1181268619}]},"public_addr":"172.21.15.172:6827/1181268619","cluster_addr":"172.21.15.172:6829/1181268619","heartbeat_back_addr":"172.21.15.172:6833/1181268619","heartbeat_front_addr":"172.21.15.172:6831/1181268619","state":["exists","up"]},{"osd":10,"uuid":"040a013a-4c3e-442f-bea6-12cbe517716d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6825","nonce":1238812525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6827","nonce":1238812525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6831","nonce":1238812525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6829","nonce":1238812525}]},"public_addr":"172.21.15.174:6825/1238812525","cluster_addr":"172.21.15.174:6827/1238812525","heartbeat_back_addr":"172.21.15.174:6831/1238812525","heartbeat_front_addr":"172.21.15.174:6829/1238812525","state":["exists","up"]},{"osd":11,"uuid":"ae6535bd-1510-4bf3-bf4b-1a2a256add43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6825","nonce":2162719997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6827","nonce":2162719997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6831","nonce":2162719997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6829","nonce":2162719997}]},"public_addr":"172.21.15.191:6825/2162719997","cluster_addr":"172.21.15.191:6827/2162719997","heartbeat_back_addr":"172.21.15.191:6831/2162719997","heartbeat_front_addr":"172.21.15.191:6829/2162719997","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:38:38.812596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:04.765652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:29.103966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:56.085849+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:21.641784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:47.750376+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:11.640697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:36.638840+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:02.167446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:28.655804+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:52.749676+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:43:17.783908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:0/977245917":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/3919952878":"2024-09-11T08:36:54.887256+0000","172.21.15.172:6800/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/1525385361":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/2043423408":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/839344299":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6800/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/3251817286":"2024-09-11T08:36:54.887256+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:44:25.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.579+0000 7f1f0dee7640 1 -- 172.21.15.172:0/3645051444 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1ef40a2c00 msgr2=0x7f1ef40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:25.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.579+0000 7f1f0dee7640 1 --2- 172.21.15.172:0/3645051444 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1ef40a2c00 0x7f1ef40a50c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1f08104c40 tx=0x7f1ef0009110 comp rx=0 tx=0).stop 2024-09-10T08:44:25.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.579+0000 7f1f0dee7640 1 -- 172.21.15.172:0/3645051444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f08100e20 msgr2=0x7f1f0819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:25.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.579+0000 7f1f0dee7640 1 --2- 172.21.15.172:0/3645051444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f08100e20 0x7f1f0819a6a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8006010 tx=0x7f1ef80042f0 comp rx=0 tx=0).stop 2024-09-10T08:44:25.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.579+0000 7f1f0dee7640 1 -- 172.21.15.172:0/3645051444 shutdown_connections 2024-09-10T08:44:25.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.579+0000 7f1f0dee7640 1 --2- 172.21.15.172:0/3645051444 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1ef40a2c00 0x7f1ef40a50c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:25.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.579+0000 7f1f0dee7640 1 --2- 172.21.15.172:0/3645051444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f08102ef0 0x7f1f0819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:25.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.579+0000 7f1f0dee7640 1 --2- 172.21.15.172:0/3645051444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f08102570 0x7f1f0819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:25.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.579+0000 7f1f0dee7640 1 --2- 172.21.15.172:0/3645051444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f08100e20 0x7f1f0819a6a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:25.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.579+0000 7f1f0dee7640 1 -- 172.21.15.172:0/3645051444 >> 172.21.15.172:0/3645051444 conn(0x7f1f080fc7d0 msgr2=0x7f1f0806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:25.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.579+0000 7f1f0dee7640 1 -- 172.21.15.172:0/3645051444 shutdown_connections 2024-09-10T08:44:25.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:25.579+0000 7f1f0dee7640 1 -- 172.21.15.172:0/3645051444 wait complete. 2024-09-10T08:44:25.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:25 smithi172 ceph-mon[30854]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:25.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:25 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3645051444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:44:25.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:25 smithi174 ceph-mon[34068]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:25.991 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:25 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3645051444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:44:26.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:25 smithi191 ceph-mon[35314]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:26.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:25 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3645051444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:44:26.278 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:44:26.528 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:27.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.998+0000 7f491d9a4640 1 -- 172.21.15.172:0/2337026765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4918102dd0 msgr2=0x7f49181031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:27.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.998+0000 7f491d9a4640 1 --2- 172.21.15.172:0/2337026765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4918102dd0 0x7f49181031b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4910009930 tx=0x7f491002a420 comp rx=0 tx=0).stop 2024-09-10T08:44:27.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.998+0000 7f491d9a4640 1 -- 172.21.15.172:0/2337026765 shutdown_connections 2024-09-10T08:44:27.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.998+0000 7f491d9a4640 1 --2- 172.21.15.172:0/2337026765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4918108300 0x7f491810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:27.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.998+0000 7f491d9a4640 1 --2- 172.21.15.172:0/2337026765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4918103780 0x7f4918107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:27.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.998+0000 7f491d9a4640 1 --2- 172.21.15.172:0/2337026765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4918102dd0 0x7f49181031b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:27.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.998+0000 7f491d9a4640 1 -- 172.21.15.172:0/2337026765 >> 172.21.15.172:0/2337026765 conn(0x7f49180fc7d0 msgr2=0x7f49180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:27.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.998+0000 7f491d9a4640 1 -- 172.21.15.172:0/2337026765 shutdown_connections 2024-09-10T08:44:27.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.998+0000 7f491d9a4640 1 -- 172.21.15.172:0/2337026765 wait complete. 2024-09-10T08:44:27.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.998+0000 7f491d9a4640 1 Processor -- start 2024-09-10T08:44:27.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.998+0000 7f491d9a4640 1 -- start start 2024-09-10T08:44:27.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f491d9a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4918102dd0 0x7f4918100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:27.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f491d9a4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4918103780 0x7f4918100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:27.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f491d9a4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4918108300 0x7f49181014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:27.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f491d9a4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4918111f40 con 0x7f4918102dd0 2024-09-10T08:44:27.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f491d9a4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4918111dc0 con 0x7f4918108300 2024-09-10T08:44:27.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f491d9a4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f49181120c0 con 0x7f4918103780 2024-09-10T08:44:27.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f4916ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4918102dd0 0x7f4918100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:27.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f4916ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4918102dd0 0x7f4918100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48786/0 (socket says 172.21.15.172:48786) 2024-09-10T08:44:27.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f4916ffd640 1 -- 172.21.15.172:0/3128892160 learned_addr learned my addr 172.21.15.172:0/3128892160 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:27.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f4916ffd640 1 -- 172.21.15.172:0/3128892160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4918103780 msgr2=0x7f4918100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:27.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f49177fe640 1 --2- 172.21.15.172:0/3128892160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4918108300 0x7f49181014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:27.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f49167fc640 1 --2- 172.21.15.172:0/3128892160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4918103780 0x7f4918100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:27.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f4916ffd640 1 --2- 172.21.15.172:0/3128892160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4918103780 0x7f4918100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:27.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f4916ffd640 1 -- 172.21.15.172:0/3128892160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4918108300 msgr2=0x7f49181014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:27.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f4916ffd640 1 --2- 172.21.15.172:0/3128892160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4918108300 0x7f49181014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:27.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f4916ffd640 1 -- 172.21.15.172:0/3128892160 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49181a69c0 con 0x7f4918102dd0 2024-09-10T08:44:27.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f4916ffd640 1 --2- 172.21.15.172:0/3128892160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4918102dd0 0x7f4918100a30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4910004770 tx=0x7f49100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:27.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f491c9a2640 1 -- 172.21.15.172:0/3128892160 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491003f070 con 0x7f4918102dd0 2024-09-10T08:44:27.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f491c9a2640 1 -- 172.21.15.172:0/3128892160 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f491003b460 con 0x7f4918102dd0 2024-09-10T08:44:27.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f491c9a2640 1 -- 172.21.15.172:0/3128892160 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4910035070 con 0x7f4918102dd0 2024-09-10T08:44:27.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f491d9a4640 1 -- 172.21.15.172:0/3128892160 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49181a6c50 con 0x7f4918102dd0 2024-09-10T08:44:27.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:26.999+0000 7f491d9a4640 1 -- 172.21.15.172:0/3128892160 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49181a7130 con 0x7f4918102dd0 2024-09-10T08:44:27.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.001+0000 7f491d9a4640 1 -- 172.21.15.172:0/3128892160 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49181031b0 con 0x7f4918102dd0 2024-09-10T08:44:27.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.002+0000 7f491c9a2640 1 -- 172.21.15.172:0/3128892160 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f491003b610 con 0x7f4918102dd0 2024-09-10T08:44:27.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.002+0000 7f491c9a2640 1 --2- 172.21.15.172:0/3128892160 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49040a2b80 0x7f49040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:27.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.003+0000 7f49167fc640 1 --2- 172.21.15.172:0/3128892160 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49040a2b80 0x7f49040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:27.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.003+0000 7f491c9a2640 1 -- 172.21.15.172:0/3128892160 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f49100ef6c0 con 0x7f4918102dd0 2024-09-10T08:44:27.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.003+0000 7f49167fc640 1 --2- 172.21.15.172:0/3128892160 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49040a2b80 0x7f49040a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f49181024c0 tx=0x7f4900005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:27.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.007+0000 7f491c9a2640 1 -- 172.21.15.172:0/3128892160 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49100bc3e0 con 0x7f4918102dd0 2024-09-10T08:44:27.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.212+0000 7f491d9a4640 1 -- 172.21.15.172:0/3128892160 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4918101f60 con 0x7f4918102dd0 2024-09-10T08:44:27.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.213+0000 7f491c9a2640 1 -- 172.21.15.172:0/3128892160 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11866 (secure 0 0 0) 0x7f49100bbce0 con 0x7f4918102dd0 2024-09-10T08:44:27.215 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:44:27.215 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":17,"btime":"2024-09-10T08:44:20:714228+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:44:20.276118+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24488,"qdb_cluster":[24488]},"id":1}]} 2024-09-10T08:44:27.215 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 17 2024-09-10T08:44:27.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.215+0000 7f491d9a4640 1 -- 172.21.15.172:0/3128892160 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49040a2b80 msgr2=0x7f49040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:27.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.215+0000 7f491d9a4640 1 --2- 172.21.15.172:0/3128892160 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49040a2b80 0x7f49040a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f49181024c0 tx=0x7f4900005c60 comp rx=0 tx=0).stop 2024-09-10T08:44:27.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.215+0000 7f491d9a4640 1 -- 172.21.15.172:0/3128892160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4918102dd0 msgr2=0x7f4918100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:27.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.215+0000 7f491d9a4640 1 --2- 172.21.15.172:0/3128892160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4918102dd0 0x7f4918100a30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4910004770 tx=0x7f49100042f0 comp rx=0 tx=0).stop 2024-09-10T08:44:27.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.215+0000 7f491d9a4640 1 -- 172.21.15.172:0/3128892160 shutdown_connections 2024-09-10T08:44:27.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.215+0000 7f491d9a4640 1 --2- 172.21.15.172:0/3128892160 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49040a2b80 0x7f49040a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:27.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.215+0000 7f491d9a4640 1 --2- 172.21.15.172:0/3128892160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4918108300 0x7f49181014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:27.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.215+0000 7f491d9a4640 1 --2- 172.21.15.172:0/3128892160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4918103780 0x7f4918100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:27.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.215+0000 7f491d9a4640 1 --2- 172.21.15.172:0/3128892160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4918102dd0 0x7f4918100a30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:27.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.215+0000 7f491d9a4640 1 -- 172.21.15.172:0/3128892160 >> 172.21.15.172:0/3128892160 conn(0x7f49180fc7d0 msgr2=0x7f4918105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:27.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.215+0000 7f491d9a4640 1 -- 172.21.15.172:0/3128892160 shutdown_connections 2024-09-10T08:44:27.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:27.215+0000 7f491d9a4640 1 -- 172.21.15.172:0/3128892160 wait complete. 2024-09-10T08:44:27.923 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:44:28.167 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:28.430 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:28 smithi172 ceph-mon[30854]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:28.430 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:28 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3128892160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:28.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:28 smithi174 ceph-mon[34068]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:28.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:28 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3128892160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:28.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:28 smithi191 ceph-mon[35314]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:28.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:28 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3128892160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:44:28.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.616+0000 7f0117f36640 1 -- 172.21.15.172:0/1849862996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0110100a80 msgr2=0x7f0110100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:28.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.616+0000 7f0117f36640 1 --2- 172.21.15.172:0/1849862996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0110100a80 0x7f0110100ee0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0100009930 tx=0x7f010002a420 comp rx=0 tx=0).stop 2024-09-10T08:44:28.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.616+0000 7f0117f36640 1 -- 172.21.15.172:0/1849862996 shutdown_connections 2024-09-10T08:44:28.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.616+0000 7f0117f36640 1 --2- 172.21.15.172:0/1849862996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0110101420 0x7f011010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:28.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.616+0000 7f0117f36640 1 --2- 172.21.15.172:0/1849862996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0110100a80 0x7f0110100ee0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:28.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.616+0000 7f0117f36640 1 --2- 172.21.15.172:0/1849862996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0110106a80 0x7f0110106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:28.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.616+0000 7f0117f36640 1 -- 172.21.15.172:0/1849862996 >> 172.21.15.172:0/1849862996 conn(0x7f01100fc7a0 msgr2=0x7f01100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:28.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.616+0000 7f0117f36640 1 -- 172.21.15.172:0/1849862996 shutdown_connections 2024-09-10T08:44:28.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.616+0000 7f0117f36640 1 -- 172.21.15.172:0/1849862996 wait complete. 2024-09-10T08:44:28.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.617+0000 7f0117f36640 1 Processor -- start 2024-09-10T08:44:28.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.617+0000 7f0117f36640 1 -- start start 2024-09-10T08:44:28.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.617+0000 7f0117f36640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0110100a80 0x7f01101983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:28.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.617+0000 7f0117f36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0110101420 0x7f01101988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:28.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.617+0000 7f0117f36640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0110106a80 0x7f011019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:28.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.617+0000 7f0117f36640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0110110d30 con 0x7f0110101420 2024-09-10T08:44:28.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.617+0000 7f0117f36640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0110110bb0 con 0x7f0110100a80 2024-09-10T08:44:28.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.617+0000 7f0117f36640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0110110eb0 con 0x7f0110106a80 2024-09-10T08:44:28.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.617+0000 7f01154d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0110101420 0x7f01101988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:28.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.617+0000 7f01154d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0110101420 0x7f01101988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48808/0 (socket says 172.21.15.172:48808) 2024-09-10T08:44:28.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.617+0000 7f01154d1640 1 -- 172.21.15.172:0/4089633863 learned_addr learned my addr 172.21.15.172:0/4089633863 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:28.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.617+0000 7f0115cd2640 1 --2- 172.21.15.172:0/4089633863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0110100a80 0x7f01101983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:28.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.617+0000 7f01164d3640 1 --2- 172.21.15.172:0/4089633863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0110106a80 0x7f011019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:28.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.617+0000 7f01154d1640 1 -- 172.21.15.172:0/4089633863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0110106a80 msgr2=0x7f011019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:28.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.617+0000 7f01154d1640 1 --2- 172.21.15.172:0/4089633863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0110106a80 0x7f011019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:28.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.618+0000 7f01154d1640 1 -- 172.21.15.172:0/4089633863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0110100a80 msgr2=0x7f01101983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:28.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.618+0000 7f01154d1640 1 --2- 172.21.15.172:0/4089633863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0110100a80 0x7f01101983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:28.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.618+0000 7f01154d1640 1 -- 172.21.15.172:0/4089633863 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011019d350 con 0x7f0110101420 2024-09-10T08:44:28.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.618+0000 7f01164d3640 1 --2- 172.21.15.172:0/4089633863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0110106a80 0x7f011019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:28.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.618+0000 7f01154d1640 1 --2- 172.21.15.172:0/4089633863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0110101420 0x7f01101988e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0100004770 tx=0x7f01000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:28.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.618+0000 7f0106ffd640 1 -- 172.21.15.172:0/4089633863 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010003f070 con 0x7f0110101420 2024-09-10T08:44:28.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.618+0000 7f0106ffd640 1 -- 172.21.15.172:0/4089633863 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f010003b460 con 0x7f0110101420 2024-09-10T08:44:28.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.618+0000 7f0117f36640 1 -- 172.21.15.172:0/4089633863 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011019d5e0 con 0x7f0110101420 2024-09-10T08:44:28.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.618+0000 7f0106ffd640 1 -- 172.21.15.172:0/4089633863 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100035070 con 0x7f0110101420 2024-09-10T08:44:28.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.618+0000 7f0117f36640 1 -- 172.21.15.172:0/4089633863 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01101a4f20 con 0x7f0110101420 2024-09-10T08:44:28.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.619+0000 7f0117f36640 1 -- 172.21.15.172:0/4089633863 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0110100ee0 con 0x7f0110101420 2024-09-10T08:44:28.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.620+0000 7f0106ffd640 1 -- 172.21.15.172:0/4089633863 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f010003b610 con 0x7f0110101420 2024-09-10T08:44:28.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.621+0000 7f0106ffd640 1 --2- 172.21.15.172:0/4089633863 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f00fc0a2c00 0x7f00fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:28.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.621+0000 7f0115cd2640 1 --2- 172.21.15.172:0/4089633863 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f00fc0a2c00 0x7f00fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:28.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.621+0000 7f0106ffd640 1 -- 172.21.15.172:0/4089633863 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f01000ef5a0 con 0x7f0110101420 2024-09-10T08:44:28.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.621+0000 7f0115cd2640 1 --2- 172.21.15.172:0/4089633863 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f00fc0a2c00 0x7f00fc0a50c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f010c00bb60 tx=0x7f010c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:28.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.625+0000 7f0106ffd640 1 -- 172.21.15.172:0/4089633863 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01000bc2c0 con 0x7f0110101420 2024-09-10T08:44:28.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.791+0000 7f0117f36640 1 -- 172.21.15.172:0/4089633863 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01101a5180 con 0x7f0110101420 2024-09-10T08:44:28.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.792+0000 7f0106ffd640 1 -- 172.21.15.172:0/4089633863 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24294 (secure 0 0 0) 0x7f01000bbbc0 con 0x7f0110101420 2024-09-10T08:44:28.794 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:44:28.794 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":88,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","created":"2024-09-10T08:36:33.083529+0000","modified":"2024-09-10T08:44:20.275987+0000","last_up_change":"2024-09-10T08:43:19.615640+0000","last_in_change":"2024-09-10T08:42:56.656115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:39:31.328306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:44:07.466637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:44:09.520569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:44:15.926057+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6803","nonce":2678426952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6805","nonce":2678426952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6809","nonce":2678426952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6807","nonce":2678426952}]},"public_addr":"172.21.15.172:6803/2678426952","cluster_addr":"172.21.15.172:6805/2678426952","heartbeat_back_addr":"172.21.15.172:6809/2678426952","heartbeat_front_addr":"172.21.15.172:6807/2678426952","state":["exists","up"]},{"osd":1,"uuid":"55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6801","nonce":3279661439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6803","nonce":3279661439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6807","nonce":3279661439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6805","nonce":3279661439}]},"public_addr":"172.21.15.174:6801/3279661439","cluster_addr":"172.21.15.174:6803/3279661439","heartbeat_back_addr":"172.21.15.174:6807/3279661439","heartbeat_front_addr":"172.21.15.174:6805/3279661439","state":["exists","up"]},{"osd":2,"uuid":"f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6801","nonce":1963796290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6803","nonce":1963796290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6807","nonce":1963796290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6805","nonce":1963796290}]},"public_addr":"172.21.15.191:6801/1963796290","cluster_addr":"172.21.15.191:6803/1963796290","heartbeat_back_addr":"172.21.15.191:6807/1963796290","heartbeat_front_addr":"172.21.15.191:6805/1963796290","state":["exists","up"]},{"osd":3,"uuid":"15ecca3f-bc6b-4a06-b283-b515384beb19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6811","nonce":869546480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6813","nonce":869546480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6817","nonce":869546480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6815","nonce":869546480}]},"public_addr":"172.21.15.172:6811/869546480","cluster_addr":"172.21.15.172:6813/869546480","heartbeat_back_addr":"172.21.15.172:6817/869546480","heartbeat_front_addr":"172.21.15.172:6815/869546480","state":["exists","up"]},{"osd":4,"uuid":"8c2c1d40-72ad-4a15-ab72-54871c7e2427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6809","nonce":961007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6811","nonce":961007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6815","nonce":961007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6813","nonce":961007949}]},"public_addr":"172.21.15.174:6809/961007949","cluster_addr":"172.21.15.174:6811/961007949","heartbeat_back_addr":"172.21.15.174:6815/961007949","heartbeat_front_addr":"172.21.15.174:6813/961007949","state":["exists","up"]},{"osd":5,"uuid":"4de45a1d-ef0e-4bfe-ba15-5eeeb8806155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6809","nonce":4216938371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6811","nonce":4216938371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6815","nonce":4216938371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6813","nonce":4216938371}]},"public_addr":"172.21.15.191:6809/4216938371","cluster_addr":"172.21.15.191:6811/4216938371","heartbeat_back_addr":"172.21.15.191:6815/4216938371","heartbeat_front_addr":"172.21.15.191:6813/4216938371","state":["exists","up"]},{"osd":6,"uuid":"60a9938a-f696-4db2-932c-402997543b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6819","nonce":3210762009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6821","nonce":3210762009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6825","nonce":3210762009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6823","nonce":3210762009}]},"public_addr":"172.21.15.172:6819/3210762009","cluster_addr":"172.21.15.172:6821/3210762009","heartbeat_back_addr":"172.21.15.172:6825/3210762009","heartbeat_front_addr":"172.21.15.172:6823/3210762009","state":["exists","up"]},{"osd":7,"uuid":"7f0202f3-971e-472f-9c4d-ae3c24d39f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6817","nonce":1461923581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6819","nonce":1461923581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6823","nonce":1461923581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6821","nonce":1461923581}]},"public_addr":"172.21.15.174:6817/1461923581","cluster_addr":"172.21.15.174:6819/1461923581","heartbeat_back_addr":"172.21.15.174:6823/1461923581","heartbeat_front_addr":"172.21.15.174:6821/1461923581","state":["exists","up"]},{"osd":8,"uuid":"b4e7327f-ca9f-4833-9cd3-b8529b3d9761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6817","nonce":3675210430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6819","nonce":3675210430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6823","nonce":3675210430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6821","nonce":3675210430}]},"public_addr":"172.21.15.191:6817/3675210430","cluster_addr":"172.21.15.191:6819/3675210430","heartbeat_back_addr":"172.21.15.191:6823/3675210430","heartbeat_front_addr":"172.21.15.191:6821/3675210430","state":["exists","up"]},{"osd":9,"uuid":"9f6153f2-f9db-4c00-a659-c932628016f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6827","nonce":1181268619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6829","nonce":1181268619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6833","nonce":1181268619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6831","nonce":1181268619}]},"public_addr":"172.21.15.172:6827/1181268619","cluster_addr":"172.21.15.172:6829/1181268619","heartbeat_back_addr":"172.21.15.172:6833/1181268619","heartbeat_front_addr":"172.21.15.172:6831/1181268619","state":["exists","up"]},{"osd":10,"uuid":"040a013a-4c3e-442f-bea6-12cbe517716d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6825","nonce":1238812525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6827","nonce":1238812525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6831","nonce":1238812525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6829","nonce":1238812525}]},"public_addr":"172.21.15.174:6825/1238812525","cluster_addr":"172.21.15.174:6827/1238812525","heartbeat_back_addr":"172.21.15.174:6831/1238812525","heartbeat_front_addr":"172.21.15.174:6829/1238812525","state":["exists","up"]},{"osd":11,"uuid":"ae6535bd-1510-4bf3-bf4b-1a2a256add43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6825","nonce":2162719997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6827","nonce":2162719997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6831","nonce":2162719997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6829","nonce":2162719997}]},"public_addr":"172.21.15.191:6825/2162719997","cluster_addr":"172.21.15.191:6827/2162719997","heartbeat_back_addr":"172.21.15.191:6831/2162719997","heartbeat_front_addr":"172.21.15.191:6829/2162719997","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:38:38.812596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:04.765652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:29.103966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:56.085849+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:21.641784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:47.750376+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:11.640697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:36.638840+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:02.167446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:28.655804+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:52.749676+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:43:17.783908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:0/977245917":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/3919952878":"2024-09-11T08:36:54.887256+0000","172.21.15.172:6800/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/1525385361":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/2043423408":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/839344299":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6800/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/3251817286":"2024-09-11T08:36:54.887256+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:44:28.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.795+0000 7f0117f36640 1 -- 172.21.15.172:0/4089633863 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f00fc0a2c00 msgr2=0x7f00fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:28.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.795+0000 7f0117f36640 1 --2- 172.21.15.172:0/4089633863 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f00fc0a2c00 0x7f00fc0a50c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f010c00bb60 tx=0x7f010c00b390 comp rx=0 tx=0).stop 2024-09-10T08:44:28.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.795+0000 7f0117f36640 1 -- 172.21.15.172:0/4089633863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0110101420 msgr2=0x7f01101988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:28.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.795+0000 7f0117f36640 1 --2- 172.21.15.172:0/4089633863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0110101420 0x7f01101988e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0100004770 tx=0x7f01000042f0 comp rx=0 tx=0).stop 2024-09-10T08:44:28.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.795+0000 7f0117f36640 1 -- 172.21.15.172:0/4089633863 shutdown_connections 2024-09-10T08:44:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.795+0000 7f0117f36640 1 --2- 172.21.15.172:0/4089633863 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f00fc0a2c00 0x7f00fc0a50c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.795+0000 7f0117f36640 1 --2- 172.21.15.172:0/4089633863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0110106a80 0x7f011019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.795+0000 7f0117f36640 1 --2- 172.21.15.172:0/4089633863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0110101420 0x7f01101988e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.795+0000 7f0117f36640 1 --2- 172.21.15.172:0/4089633863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0110100a80 0x7f01101983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.795+0000 7f0117f36640 1 -- 172.21.15.172:0/4089633863 >> 172.21.15.172:0/4089633863 conn(0x7f01100fc7a0 msgr2=0x7f0110104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.796+0000 7f0117f36640 1 -- 172.21.15.172:0/4089633863 shutdown_connections 2024-09-10T08:44:28.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:28.796+0000 7f0117f36640 1 -- 172.21.15.172:0/4089633863 wait complete. 2024-09-10T08:44:29.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:29 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/4089633863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:44:29.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:29 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/4089633863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:44:29.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:29 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/4089633863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:44:29.630 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T08:44:11.310930+0000', 'modified': '2024-09-10T08:44:20.276118+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24488}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.174:6839/3029849701', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 3029849701}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 3029849701}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24488, 'qdb_cluster': [24488]} 2024-09-10T08:44:29.630 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T08:44:29.631 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T08:44:29.875 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:30.256 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:30 smithi172 ceph-mon[30854]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:30.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.335+0000 7ff9b371d640 1 -- 172.21.15.172:0/3919782087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac102570 msgr2=0x7ff9ac1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:30.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.335+0000 7ff9b371d640 1 --2- 172.21.15.172:0/3919782087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac102570 0x7ff9ac1029b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009900 tx=0x7ff99c02a480 comp rx=0 tx=0).stop 2024-09-10T08:44:30.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.335+0000 7ff9b371d640 1 -- 172.21.15.172:0/3919782087 shutdown_connections 2024-09-10T08:44:30.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.335+0000 7ff9b371d640 1 --2- 172.21.15.172:0/3919782087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac102ef0 0x7ff9ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.335+0000 7ff9b371d640 1 --2- 172.21.15.172:0/3919782087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac102570 0x7ff9ac1029b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.335+0000 7ff9b371d640 1 --2- 172.21.15.172:0/3919782087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9ac100e20 0x7ff9ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.335+0000 7ff9b371d640 1 -- 172.21.15.172:0/3919782087 >> 172.21.15.172:0/3919782087 conn(0x7ff9ac0fc7d0 msgr2=0x7ff9ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:30.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.335+0000 7ff9b371d640 1 -- 172.21.15.172:0/3919782087 shutdown_connections 2024-09-10T08:44:30.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.335+0000 7ff9b371d640 1 -- 172.21.15.172:0/3919782087 wait complete. 2024-09-10T08:44:30.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.335+0000 7ff9b371d640 1 Processor -- start 2024-09-10T08:44:30.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.335+0000 7ff9b371d640 1 -- start start 2024-09-10T08:44:30.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b371d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac100e20 0x7ff9ac10e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b371d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac102570 0x7ff9ac10ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b371d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9ac102ef0 0x7ff9ac10ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b371d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac111f50 con 0x7ff9ac102570 2024-09-10T08:44:30.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b371d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac111dd0 con 0x7ff9ac102ef0 2024-09-10T08:44:30.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b371d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1120d0 con 0x7ff9ac100e20 2024-09-10T08:44:30.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b0cb8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac102570 0x7ff9ac10ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b0cb8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac102570 0x7ff9ac10ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48822/0 (socket says 172.21.15.172:48822) 2024-09-10T08:44:30.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b0cb8640 1 -- 172.21.15.172:0/3109287754 learned_addr learned my addr 172.21.15.172:0/3109287754 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:30.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b14b9640 1 --2- 172.21.15.172:0/3109287754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac100e20 0x7ff9ac10e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b1cba640 1 --2- 172.21.15.172:0/3109287754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9ac102ef0 0x7ff9ac10ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b0cb8640 1 -- 172.21.15.172:0/3109287754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac100e20 msgr2=0x7ff9ac10e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:30.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b0cb8640 1 --2- 172.21.15.172:0/3109287754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac100e20 0x7ff9ac10e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b0cb8640 1 -- 172.21.15.172:0/3109287754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9ac102ef0 msgr2=0x7ff9ac10ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:30.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b0cb8640 1 --2- 172.21.15.172:0/3109287754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9ac102ef0 0x7ff9ac10ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b0cb8640 1 -- 172.21.15.172:0/3109287754 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ac1a68b0 con 0x7ff9ac102570 2024-09-10T08:44:30.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b14b9640 1 --2- 172.21.15.172:0/3109287754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac100e20 0x7ff9ac10e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:30.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b1cba640 1 --2- 172.21.15.172:0/3109287754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9ac102ef0 0x7ff9ac10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:30.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b0cb8640 1 --2- 172.21.15.172:0/3109287754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac102570 0x7ff9ac10ea10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff99c006010 tx=0x7ff99c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:30.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9a27fc640 1 -- 172.21.15.172:0/3109287754 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c03f030 con 0x7ff9ac102570 2024-09-10T08:44:30.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9b371d640 1 -- 172.21.15.172:0/3109287754 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ac1a6b40 con 0x7ff9ac102570 2024-09-10T08:44:30.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9a27fc640 1 -- 172.21.15.172:0/3109287754 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff99c003cb0 con 0x7ff9ac102570 2024-09-10T08:44:30.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.336+0000 7ff9a27fc640 1 -- 172.21.15.172:0/3109287754 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c035070 con 0x7ff9ac102570 2024-09-10T08:44:30.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.337+0000 7ff9b371d640 1 -- 172.21.15.172:0/3109287754 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ac1a7070 con 0x7ff9ac102570 2024-09-10T08:44:30.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.339+0000 7ff9a27fc640 1 -- 172.21.15.172:0/3109287754 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff99c042030 con 0x7ff9ac102570 2024-09-10T08:44:30.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.339+0000 7ff9b5af6740 1 Processor -- start 2024-09-10T08:44:30.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.339+0000 7ff9b5af6740 1 -- start start 2024-09-10T08:44:30.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.339+0000 7ff9b5af6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fc9d0 0x55f1977fcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.339+0000 7ff9b5af6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f1977fd2f0 0x55f197807670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.339+0000 7ff9b5af6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f197807bb0 0x55f197809fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.339+0000 7ff9b5af6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f197763090 con 0x55f1977fd2f0 2024-09-10T08:44:30.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.339+0000 7ff9b5af6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f19770aa70 con 0x55f197807bb0 2024-09-10T08:44:30.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.339+0000 7ff9b5af6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f19775f050 con 0x55f1977fc9d0 2024-09-10T08:44:30.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.339+0000 7ff9b14b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f1977fd2f0 0x55f197807670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.339+0000 7ff9b1cba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fc9d0 0x55f1977fcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.339+0000 7ff9b14b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f1977fd2f0 0x55f197807670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48828/0 (socket says 172.21.15.172:48828) 2024-09-10T08:44:30.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.339+0000 7ff9b14b9640 1 -- 172.21.15.172:0/4204844968 learned_addr learned my addr 172.21.15.172:0/4204844968 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:30.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.339+0000 7ff9b1cba640 1 --2- 172.21.15.172:0/4204844968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fc9d0 0x55f1977fcdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:51826/0 (socket says 172.21.15.172:51826) 2024-09-10T08:44:30.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.340+0000 7ff9b0cb8640 1 --2- 172.21.15.172:0/4204844968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f197807bb0 0x55f197809fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.340+0000 7ff9b14b9640 1 -- 172.21.15.172:0/4204844968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fc9d0 msgr2=0x55f1977fcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:30.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.340+0000 7ff9b14b9640 1 --2- 172.21.15.172:0/4204844968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fc9d0 0x55f1977fcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.340+0000 7ff9b14b9640 1 -- 172.21.15.172:0/4204844968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f197807bb0 msgr2=0x55f197809fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:30.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.340+0000 7ff9b14b9640 1 --2- 172.21.15.172:0/4204844968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f197807bb0 0x55f197809fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.340+0000 7ff9b14b9640 1 -- 172.21.15.172:0/4204844968 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f197717130 con 0x55f1977fd2f0 2024-09-10T08:44:30.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.340+0000 7ff9a27fc640 1 --2- 172.21.15.172:0/3109287754 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff9980a2b80 0x7ff9980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.340+0000 7ff9b1cba640 1 --2- 172.21.15.172:0/4204844968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fc9d0 0x55f1977fcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:44:30.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.340+0000 7ff9b0cb8640 1 --2- 172.21.15.172:0/4204844968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f197807bb0 0x55f197809fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:44:30.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.340+0000 7ff9b1cba640 1 --2- 172.21.15.172:0/3109287754 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff9980a2b80 0x7ff9980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.340+0000 7ff9b14b9640 1 --2- 172.21.15.172:0/4204844968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f1977fd2f0 0x55f197807670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800f3c0 tx=0x7ff9a800f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03a4812e3bda283 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:30.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.340+0000 7ff9a27fc640 1 -- 172.21.15.172:0/3109287754 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7ff99c0ef3e0 con 0x7ff9ac102570 2024-09-10T08:44:30.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.340+0000 7ff9977fe640 1 -- 172.21.15.172:0/4204844968 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a801d070 con 0x55f1977fd2f0 2024-09-10T08:44:30.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.340+0000 7ff9977fe640 1 -- 172.21.15.172:0/4204844968 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9a8010db0 con 0x55f1977fd2f0 2024-09-10T08:44:30.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.340+0000 7ff9977fe640 1 -- 172.21.15.172:0/4204844968 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8018e00 con 0x55f1977fd2f0 2024-09-10T08:44:30.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.341+0000 7ff9b5af6740 1 -- 172.21.15.172:0/4204844968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f1977fd2f0 msgr2=0x55f197807670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:30.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.341+0000 7ff9b1cba640 1 --2- 172.21.15.172:0/3109287754 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff9980a2b80 0x7ff9980a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4005fd0 tx=0x7ff9a4009780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:30.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.341+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/4204844968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f1977fd2f0 0x55f197807670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800f3c0 tx=0x7ff9a800f880 comp rx=0 tx=0).stop 2024-09-10T08:44:30.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.341+0000 7ff9b5af6740 1 -- 172.21.15.172:0/4204844968 shutdown_connections 2024-09-10T08:44:30.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.341+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/4204844968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f197807bb0 0x55f197809fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.341+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/4204844968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f1977fd2f0 0x55f197807670 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.341+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/4204844968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fc9d0 0x55f1977fcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.341+0000 7ff9b5af6740 1 -- 172.21.15.172:0/4204844968 >> 172.21.15.172:0/4204844968 conn(0x55f1977f8f60 msgr2=0x55f1977f9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:30.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.341+0000 7ff9b5af6740 1 -- 172.21.15.172:0/4204844968 shutdown_connections 2024-09-10T08:44:30.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.341+0000 7ff9b5af6740 1 -- 172.21.15.172:0/4204844968 wait complete. 2024-09-10T08:44:30.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.341+0000 7ff9b5af6740 1 -- start start 2024-09-10T08:44:30.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.341+0000 7ff9b5af6740 1 Processor -- start 2024-09-10T08:44:30.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b5af6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fc9d0 0x55f197807440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b5af6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1977fd2f0 0x55f197804790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b5af6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f197807bb0 0x55f197804cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b5af6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f19780a780 con 0x55f197807bb0 2024-09-10T08:44:30.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b5af6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f19771df20 con 0x55f1977fd2f0 2024-09-10T08:44:30.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b5af6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f19780a900 con 0x55f1977fc9d0 2024-09-10T08:44:30.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b5af6740 10 client.0 _ll_register_callbacks cb 0x55f1976e7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:30.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b1cba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f197807bb0 0x55f197804cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b1cba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f197807bb0 0x55f197804cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48844/0 (socket says 172.21.15.172:48844) 2024-09-10T08:44:30.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b1cba640 1 -- 172.21.15.172:0/3728287219 learned_addr learned my addr 172.21.15.172:0/3728287219 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:30.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b14b9640 1 --2- 172.21.15.172:0/3728287219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fc9d0 0x55f197807440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b0cb8640 1 --2- 172.21.15.172:0/3728287219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1977fd2f0 0x55f197804790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b1cba640 1 -- 172.21.15.172:0/3728287219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fc9d0 msgr2=0x55f197807440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:30.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b1cba640 1 --2- 172.21.15.172:0/3728287219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fc9d0 0x55f197807440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b1cba640 1 -- 172.21.15.172:0/3728287219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1977fd2f0 msgr2=0x55f197804790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:30.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b1cba640 1 --2- 172.21.15.172:0/3728287219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1977fd2f0 0x55f197804790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b1cba640 1 -- 172.21.15.172:0/3728287219 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1974d42b0 con 0x55f197807bb0 2024-09-10T08:44:30.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b0cb8640 1 --2- 172.21.15.172:0/3728287219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1977fd2f0 0x55f197804790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:30.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff9b1cba640 1 --2- 172.21.15.172:0/3728287219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f197807bb0 0x55f197804cd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4011df0 tx=0x7ff9a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d26b5727c0acc9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:30.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.342+0000 7ff994ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:44:30.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.343+0000 7ff994ff9640 1 -- 172.21.15.172:0/3728287219 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4020420 con 0x55f197807bb0 2024-09-10T08:44:30.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.343+0000 7ff994ff9640 1 -- 172.21.15.172:0/3728287219 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7ff9a40205c0 con 0x55f197807bb0 2024-09-10T08:44:30.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.343+0000 7ff9b5af6740 1 -- 172.21.15.172:0/3728287219 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a8010db0 con 0x55f197807bb0 2024-09-10T08:44:30.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.343+0000 7ff994ff9640 1 -- 172.21.15.172:0/3728287219 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ff9a4033050 con 0x55f197807bb0 2024-09-10T08:44:30.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.343+0000 7ff9b5af6740 10 client.14727 fetch_fsmap learned FSMap version 17 2024-09-10T08:44:30.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.343+0000 7ff9b5af6740 1 -- 172.21.15.172:0/3728287219 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55f197816090 con 0x55f197807bb0 2024-09-10T08:44:30.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.343+0000 7ff994ff9640 1 -- 172.21.15.172:0/3728287219 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9a8010db0 con 0x55f197807bb0 2024-09-10T08:44:30.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.343+0000 7ff994ff9640 1 -- 172.21.15.172:0/3728287219 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7ff9a400f040 con 0x55f197807bb0 2024-09-10T08:44:30.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.343+0000 7ff9b5af6740 10 client.14727 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T08:44:30.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.343+0000 7ff9b5af6740 10 client.14727 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:44:30.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.343+0000 7ff9b5af6740 20 client.14727 populate_metadata read hostname 'smithi172' 2024-09-10T08:44:30.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.343+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/3728287219 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f197818760 0x55f197819bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.343+0000 7ff9b5af6740 4 client.14727 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:44:30.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.343+0000 7ff9b5af6740 1 -- 172.21.15.172:0/3728287219 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f197770600 con 0x55f197818760 2024-09-10T08:44:30.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.343+0000 7ff9b14b9640 1 --2- 172.21.15.172:0/3728287219 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f197818760 0x55f197819bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.344+0000 7ff994ff9640 10 client.14727 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:44:30.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.344+0000 7ff9b14b9640 1 --2- 172.21.15.172:0/3728287219 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f197818760 0x55f197819bd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e2f2e541e5aad5 server_cookie=903452cf75827df1 in_seq=0 out_seq=0 2024-09-10T08:44:30.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.345+0000 7ff994ff9640 1 -- 172.21.15.172:0/3728287219 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f197770600 con 0x55f197818760 2024-09-10T08:44:30.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.345+0000 7ff994ff9640 10 client.14727 handle_command_reply: tid=0 2024-09-10T08:44:30.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.345+0000 7ff9b5af6740 1 client.14727 shutdown 2024-09-10T08:44:30.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.345+0000 7ff9b5af6740 1 -- 172.21.15.172:0/3728287219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f197807bb0 msgr2=0x55f197804cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:30.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.345+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/3728287219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f197807bb0 0x55f197804cd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4011df0 tx=0x7ff9a4004a40 comp rx=0 tx=0).stop 2024-09-10T08:44:30.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.345+0000 7ff9b5af6740 1 -- 172.21.15.172:0/3728287219 shutdown_connections 2024-09-10T08:44:30.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.345+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/3728287219 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f197818760 0x55f197819bd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.345+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/3728287219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f197807bb0 0x55f197804cd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.345+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/3728287219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1977fd2f0 0x55f197804790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.345+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/3728287219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fc9d0 0x55f197807440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.345+0000 7ff994ff9640 0 client.14727 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:44:30.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.345+0000 7ff9b5af6740 1 -- 172.21.15.172:0/3728287219 >> 172.21.15.172:0/3728287219 conn(0x55f1977f8f60 msgr2=0x55f1977fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:30.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.345+0000 7ff9b5af6740 1 -- 172.21.15.172:0/3728287219 shutdown_connections 2024-09-10T08:44:30.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.345+0000 7ff9b5af6740 1 -- 172.21.15.172:0/3728287219 wait complete. 2024-09-10T08:44:30.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.345+0000 7ff9b5af6740 20 client.14727 trim_cache size 0 max 16384 2024-09-10T08:44:30.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b5af6740 1 Processor -- start 2024-09-10T08:44:30.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b5af6740 1 -- start start 2024-09-10T08:44:30.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b5af6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fc9d0 0x55f197804070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b5af6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1977fd2f0 0x55f1978019f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b5af6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f197807bb0 0x55f197801f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b5af6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f197817f10 con 0x55f197807bb0 2024-09-10T08:44:30.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b5af6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f1978185e0 con 0x55f1977fd2f0 2024-09-10T08:44:30.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b5af6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f197818cb0 con 0x55f1977fc9d0 2024-09-10T08:44:30.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b1cba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f197807bb0 0x55f197801f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b1cba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f197807bb0 0x55f197801f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48848/0 (socket says 172.21.15.172:48848) 2024-09-10T08:44:30.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b1cba640 1 -- 172.21.15.172:0/699431395 learned_addr learned my addr 172.21.15.172:0/699431395 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b0cb8640 1 --2- 172.21.15.172:0/699431395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1977fd2f0 0x55f1978019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b14b9640 1 --2- 172.21.15.172:0/699431395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fc9d0 0x55f197804070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b1cba640 1 -- 172.21.15.172:0/699431395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fc9d0 msgr2=0x55f197804070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b1cba640 1 --2- 172.21.15.172:0/699431395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fc9d0 0x55f197804070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b1cba640 1 -- 172.21.15.172:0/699431395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1977fd2f0 msgr2=0x55f1978019f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b1cba640 1 --2- 172.21.15.172:0/699431395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1977fd2f0 0x55f1978019f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b1cba640 1 -- 172.21.15.172:0/699431395 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1977fa0f0 con 0x55f197807bb0 2024-09-10T08:44:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.363+0000 7ff9b1cba640 1 --2- 172.21.15.172:0/699431395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f197807bb0 0x55f197801f30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4018040 tx=0x7ff9a40284a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdadb41f576d863 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.364+0000 7ff996ffd640 1 -- 172.21.15.172:0/699431395 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a40119a0 con 0x55f197807bb0 2024-09-10T08:44:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.364+0000 7ff996ffd640 1 -- 172.21.15.172:0/699431395 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9a4020420 con 0x55f197807bb0 2024-09-10T08:44:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.364+0000 7ff996ffd640 1 -- 172.21.15.172:0/699431395 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4020c30 con 0x55f197807bb0 2024-09-10T08:44:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.364+0000 7ff9b5af6740 1 -- 172.21.15.172:0/699431395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f197807bb0 msgr2=0x55f197801f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:30.394 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:44:30.394 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:44:30.394 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-09-10T08:44:30.394 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 24488, 2024-09-10T08:44:30.394 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-09-10T08:44:30.394 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-09-10T08:44:30.394 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-09-10T08:44:30.394 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 18.693200519000001, 2024-09-10T08:44:30.394 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 17, 2024-09-10T08:44:30.395 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 88, 2024-09-10T08:44:30.395 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 88, 2024-09-10T08:44:30.395 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 29.081493763000001 2024-09-10T08:44:30.395 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:44:30.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.364+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/699431395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f197807bb0 0x55f197801f30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4018040 tx=0x7ff9a40284a0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.364+0000 7ff9b5af6740 1 -- 172.21.15.172:0/699431395 shutdown_connections 2024-09-10T08:44:30.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.364+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/699431395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f197807bb0 0x55f197801f30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.364+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/699431395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1977fd2f0 0x55f1978019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.364+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/699431395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fc9d0 0x55f197804070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.364+0000 7ff9b5af6740 1 -- 172.21.15.172:0/699431395 >> 172.21.15.172:0/699431395 conn(0x55f1977f8f60 msgr2=0x55f197806700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.364+0000 7ff9b5af6740 1 -- 172.21.15.172:0/699431395 shutdown_connections 2024-09-10T08:44:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.364+0000 7ff9b5af6740 1 -- 172.21.15.172:0/699431395 wait complete. 2024-09-10T08:44:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.365+0000 7ff9b5af6740 1 -- start start 2024-09-10T08:44:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.365+0000 7ff9b5af6740 1 Processor -- start 2024-09-10T08:44:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.365+0000 7ff9b5af6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f1977fc9d0 0x55f197822590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.365+0000 7ff9b5af6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fd2f0 0x55f19781d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.365+0000 7ff9b5af6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f197807bb0 0x55f19781de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.365+0000 7ff9b5af6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f197819a50 con 0x55f1977fc9d0 2024-09-10T08:44:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.365+0000 7ff9b5af6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f197819380 con 0x55f197807bb0 2024-09-10T08:44:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.365+0000 7ff9b5af6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f19781a120 con 0x55f1977fd2f0 2024-09-10T08:44:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.365+0000 7ff9b5af6740 10 client.0 _ll_register_callbacks cb 0x7ff9a4009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:44:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.365+0000 7ff9b14b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f1977fc9d0 0x55f197822590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.365+0000 7ff9b14b9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f1977fc9d0 0x55f197822590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48858/0 (socket says 172.21.15.172:48858) 2024-09-10T08:44:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.365+0000 7ff9b14b9640 1 -- 172.21.15.172:0/2134216584 learned_addr learned my addr 172.21.15.172:0/2134216584 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.365+0000 7ff9b0cb8640 1 --2- 172.21.15.172:0/2134216584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fd2f0 0x55f19781d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff9b1cba640 1 --2- 172.21.15.172:0/2134216584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f197807bb0 0x55f19781de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff9b14b9640 1 -- 172.21.15.172:0/2134216584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fd2f0 msgr2=0x55f19781d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff9b14b9640 1 --2- 172.21.15.172:0/2134216584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fd2f0 0x55f19781d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff9b14b9640 1 -- 172.21.15.172:0/2134216584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f197807bb0 msgr2=0x55f19781de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:44:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff9b14b9640 1 --2- 172.21.15.172:0/2134216584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f197807bb0 0x55f19781de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff9b14b9640 1 -- 172.21.15.172:0/2134216584 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f19781e5e0 con 0x55f1977fc9d0 2024-09-10T08:44:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff9b1cba640 1 --2- 172.21.15.172:0/2134216584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f197807bb0 0x55f19781de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:44:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff9b14b9640 1 --2- 172.21.15.172:0/2134216584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f1977fc9d0 0x55f197822590 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8000c00 tx=0x7ff9a800ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1daf0ce2a83242b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff994ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:44:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff994ff9640 1 -- 172.21.15.172:0/2134216584 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8031020 con 0x55f1977fc9d0 2024-09-10T08:44:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff994ff9640 1 -- 172.21.15.172:0/2134216584 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7ff9a8004220 con 0x55f1977fc9d0 2024-09-10T08:44:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff9b5af6740 1 -- 172.21.15.172:0/2134216584 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a4020420 con 0x55f1977fc9d0 2024-09-10T08:44:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff994ff9640 1 -- 172.21.15.172:0/2134216584 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ff9a8039050 con 0x55f1977fc9d0 2024-09-10T08:44:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff9b5af6740 10 client.14733 fetch_fsmap learned FSMap version 17 2024-09-10T08:44:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff9b5af6740 1 -- 172.21.15.172:0/2134216584 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55f197882e20 con 0x55f1977fc9d0 2024-09-10T08:44:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff994ff9640 1 -- 172.21.15.172:0/2134216584 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9a4020420 con 0x55f1977fc9d0 2024-09-10T08:44:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff994ff9640 1 -- 172.21.15.172:0/2134216584 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7ff9a80053e0 con 0x55f1977fc9d0 2024-09-10T08:44:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.366+0000 7ff9b5af6740 10 client.14733 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T08:44:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff9b5af6740 10 client.14733 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:44:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff9b5af6740 20 client.14733 populate_metadata read hostname 'smithi172' 2024-09-10T08:44:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/2134216584 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f1978876f0 0x55f197886400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff9b5af6740 4 client.14733 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:44:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff9b5af6740 1 -- 172.21.15.172:0/2134216584 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "status"}) -- 0x55f19776ab10 con 0x55f1978876f0 2024-09-10T08:44:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff9b0cb8640 1 --2- 172.21.15.172:0/2134216584 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f1978876f0 0x55f197886400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff9b0cb8640 1 --2- 172.21.15.172:0/2134216584 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f1978876f0 0x55f197886400 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4354f6fe70f4ab15 server_cookie=a5cb247e6067bdac in_seq=0 out_seq=0 2024-09-10T08:44:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff994ff9640 10 client.14733 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:44:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff994ff9640 1 -- 172.21.15.172:0/2134216584 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f19776ab10 con 0x55f1978876f0 2024-09-10T08:44:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff994ff9640 10 client.14733 handle_command_reply: tid=0 2024-09-10T08:44:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff9b5af6740 1 client.14733 shutdown 2024-09-10T08:44:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff9b5af6740 1 -- 172.21.15.172:0/2134216584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f1977fc9d0 msgr2=0x55f197822590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/2134216584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f1977fc9d0 0x55f197822590 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8000c00 tx=0x7ff9a800ab80 comp rx=0 tx=0).stop 2024-09-10T08:44:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff9b5af6740 1 -- 172.21.15.172:0/2134216584 shutdown_connections 2024-09-10T08:44:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/2134216584 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f1978876f0 0x55f197886400 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/2134216584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f197807bb0 0x55f19781de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/2134216584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1977fd2f0 0x55f19781d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff994ff9640 0 client.14733 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:44:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff9b5af6740 1 --2- 172.21.15.172:0/2134216584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f1977fc9d0 0x55f197822590 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff9b5af6740 1 -- 172.21.15.172:0/2134216584 >> 172.21.15.172:0/2134216584 conn(0x55f1977f8f60 msgr2=0x55f197812b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.367+0000 7ff9b5af6740 1 -- 172.21.15.172:0/2134216584 shutdown_connections 2024-09-10T08:44:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.368+0000 7ff9b5af6740 1 -- 172.21.15.172:0/2134216584 wait complete. 2024-09-10T08:44:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.368+0000 7ff9b5af6740 20 client.14733 trim_cache size 0 max 16384 2024-09-10T08:44:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.368+0000 7ff9b371d640 1 -- 172.21.15.172:0/3109287754 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff9980a2b80 msgr2=0x7ff9980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.368+0000 7ff9b371d640 1 --2- 172.21.15.172:0/3109287754 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff9980a2b80 0x7ff9980a5040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4005fd0 tx=0x7ff9a4009780 comp rx=0 tx=0).stop 2024-09-10T08:44:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.368+0000 7ff9b371d640 1 -- 172.21.15.172:0/3109287754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac102570 msgr2=0x7ff9ac10ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.368+0000 7ff9b371d640 1 --2- 172.21.15.172:0/3109287754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac102570 0x7ff9ac10ea10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff99c006010 tx=0x7ff99c0042f0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.368+0000 7ff9b371d640 1 -- 172.21.15.172:0/3109287754 shutdown_connections 2024-09-10T08:44:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.368+0000 7ff9b371d640 1 --2- 172.21.15.172:0/3109287754 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff9980a2b80 0x7ff9980a5040 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.368+0000 7ff9b371d640 1 --2- 172.21.15.172:0/3109287754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9ac102ef0 0x7ff9ac10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.368+0000 7ff9b371d640 1 --2- 172.21.15.172:0/3109287754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac102570 0x7ff9ac10ea10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.368+0000 7ff9b371d640 1 --2- 172.21.15.172:0/3109287754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac100e20 0x7ff9ac10e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.368+0000 7ff9b371d640 1 -- 172.21.15.172:0/3109287754 >> 172.21.15.172:0/3109287754 conn(0x7ff9ac0fc7d0 msgr2=0x7ff9ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.368+0000 7ff9b371d640 1 -- 172.21.15.172:0/3109287754 shutdown_connections 2024-09-10T08:44:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:30.368+0000 7ff9b371d640 1 -- 172.21.15.172:0/3109287754 wait complete. 2024-09-10T08:44:30.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:30 smithi174 ceph-mon[34068]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:30.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:30 smithi191 ceph-mon[35314]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:31.011 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T08:44:31.011 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:44:31.011 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T08:44:31.260 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:31.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.842+0000 7f08e265d640 1 -- 172.21.15.172:0/1677864412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08dc1027a0 msgr2=0x7f08dc1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:31.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.842+0000 7f08e265d640 1 --2- 172.21.15.172:0/1677864412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08dc1027a0 0x7f08dc1067f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f08d0009900 tx=0x7f08d002a410 comp rx=0 tx=0).stop 2024-09-10T08:44:31.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.842+0000 7f08e265d640 1 -- 172.21.15.172:0/1677864412 shutdown_connections 2024-09-10T08:44:31.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.842+0000 7f08e265d640 1 --2- 172.21.15.172:0/1677864412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08dc106f20 0x7f08dc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:31.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.842+0000 7f08e265d640 1 --2- 172.21.15.172:0/1677864412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08dc1027a0 0x7f08dc1067f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:31.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.842+0000 7f08e265d640 1 --2- 172.21.15.172:0/1677864412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08dc101df0 0x7f08dc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:31.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.842+0000 7f08e265d640 1 -- 172.21.15.172:0/1677864412 >> 172.21.15.172:0/1677864412 conn(0x7f08dc0fd8b0 msgr2=0x7f08dc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:31.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.842+0000 7f08e265d640 1 -- 172.21.15.172:0/1677864412 shutdown_connections 2024-09-10T08:44:31.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.842+0000 7f08e265d640 1 -- 172.21.15.172:0/1677864412 wait complete. 2024-09-10T08:44:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08e265d640 1 Processor -- start 2024-09-10T08:44:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08e265d640 1 -- start start 2024-09-10T08:44:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08e265d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08dc101df0 0x7f08dc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08e265d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08dc1027a0 0x7f08dc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08e265d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08dc106f20 0x7f08dc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08e265d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc111ed0 con 0x7f08dc101df0 2024-09-10T08:44:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08e265d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc111d50 con 0x7f08dc1027a0 2024-09-10T08:44:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08e265d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc112050 con 0x7f08dc106f20 2024-09-10T08:44:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08dc101df0 0x7f08dc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08dbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08dc101df0 0x7f08dc1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48880/0 (socket says 172.21.15.172:48880) 2024-09-10T08:44:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08dbfff640 1 -- 172.21.15.172:0/2828625593 learned_addr learned my addr 172.21.15.172:0/2828625593 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08db7fe640 1 --2- 172.21.15.172:0/2828625593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08dc1027a0 0x7f08dc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08dbfff640 1 -- 172.21.15.172:0/2828625593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08dc106f20 msgr2=0x7f08dc19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08dbfff640 1 --2- 172.21.15.172:0/2828625593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08dc106f20 0x7f08dc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08dbfff640 1 -- 172.21.15.172:0/2828625593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08dc1027a0 msgr2=0x7f08dc198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08dbfff640 1 --2- 172.21.15.172:0/2828625593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08dc1027a0 0x7f08dc198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08dbfff640 1 -- 172.21.15.172:0/2828625593 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc19d480 con 0x7f08dc101df0 2024-09-10T08:44:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08db7fe640 1 --2- 172.21.15.172:0/2828625593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08dc1027a0 0x7f08dc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:44:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.843+0000 7f08dbfff640 1 --2- 172.21.15.172:0/2828625593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08dc101df0 0x7f08dc1984d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00b4a0 tx=0x7f08cc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.844+0000 7f08d97fa640 1 -- 172.21.15.172:0/2828625593 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc014070 con 0x7f08dc101df0 2024-09-10T08:44:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.844+0000 7f08d97fa640 1 -- 172.21.15.172:0/2828625593 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08cc010ac0 con 0x7f08dc101df0 2024-09-10T08:44:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.844+0000 7f08e265d640 1 -- 172.21.15.172:0/2828625593 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08dc19d770 con 0x7f08dc101df0 2024-09-10T08:44:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.844+0000 7f08d97fa640 1 -- 172.21.15.172:0/2828625593 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc017c30 con 0x7f08dc101df0 2024-09-10T08:44:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.844+0000 7f08e265d640 1 -- 172.21.15.172:0/2828625593 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08dc1a5050 con 0x7f08dc101df0 2024-09-10T08:44:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.845+0000 7f08e265d640 1 -- 172.21.15.172:0/2828625593 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08dc102250 con 0x7f08dc101df0 2024-09-10T08:44:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.846+0000 7f08d97fa640 1 -- 172.21.15.172:0/2828625593 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f08cc017070 con 0x7f08dc101df0 2024-09-10T08:44:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.847+0000 7f08d97fa640 1 --2- 172.21.15.172:0/2828625593 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08c40a2c00 0x7f08c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.847+0000 7f08db7fe640 1 --2- 172.21.15.172:0/2828625593 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08c40a2c00 0x7f08c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.847+0000 7f08d97fa640 1 -- 172.21.15.172:0/2828625593 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f08cc0ce680 con 0x7f08dc101df0 2024-09-10T08:44:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.847+0000 7f08db7fe640 1 --2- 172.21.15.172:0/2828625593 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08c40a2c00 0x7f08c40a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f08d0000c00 tx=0x7f08d0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:31.851+0000 7f08d97fa640 1 -- 172.21.15.172:0/2828625593 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f08cc09b3a0 con 0x7f08dc101df0 2024-09-10T08:44:32.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:32.055+0000 7f08e265d640 1 -- 172.21.15.172:0/2828625593 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f08dc02d7c0 con 0x7f08dc101df0 2024-09-10T08:44:32.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:32.056+0000 7f08d97fa640 1 -- 172.21.15.172:0/2828625593 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f08cc09aca0 con 0x7f08dc101df0 2024-09-10T08:44:32.057 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T08:44:32.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:32.058+0000 7f08e265d640 1 -- 172.21.15.172:0/2828625593 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08c40a2c00 msgr2=0x7f08c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:32.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:32.058+0000 7f08e265d640 1 --2- 172.21.15.172:0/2828625593 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08c40a2c00 0x7f08c40a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f08d0000c00 tx=0x7f08d0031400 comp rx=0 tx=0).stop 2024-09-10T08:44:32.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:32.058+0000 7f08e265d640 1 -- 172.21.15.172:0/2828625593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08dc101df0 msgr2=0x7f08dc1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:32.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:32.058+0000 7f08e265d640 1 --2- 172.21.15.172:0/2828625593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08dc101df0 0x7f08dc1984d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00b4a0 tx=0x7f08cc00b960 comp rx=0 tx=0).stop 2024-09-10T08:44:32.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:32.058+0000 7f08e265d640 1 -- 172.21.15.172:0/2828625593 shutdown_connections 2024-09-10T08:44:32.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:32.058+0000 7f08e265d640 1 --2- 172.21.15.172:0/2828625593 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08c40a2c00 0x7f08c40a50c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:32.058+0000 7f08e265d640 1 --2- 172.21.15.172:0/2828625593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08dc106f20 0x7f08dc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:32.058+0000 7f08e265d640 1 --2- 172.21.15.172:0/2828625593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08dc1027a0 0x7f08dc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:32.058+0000 7f08e265d640 1 --2- 172.21.15.172:0/2828625593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08dc101df0 0x7f08dc1984d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:32.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:32.058+0000 7f08e265d640 1 -- 172.21.15.172:0/2828625593 >> 172.21.15.172:0/2828625593 conn(0x7f08dc0fd8b0 msgr2=0x7f08dc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:32.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:32.058+0000 7f08e265d640 1 -- 172.21.15.172:0/2828625593 shutdown_connections 2024-09-10T08:44:32.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:32.058+0000 7f08e265d640 1 -- 172.21.15.172:0/2828625593 wait complete. 2024-09-10T08:44:32.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:32 smithi172 ceph-mon[30854]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:32.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:32 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2828625593' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:44:32.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:32 smithi174 ceph-mon[34068]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:32.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:32 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2828625593' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:44:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:32 smithi191 ceph-mon[35314]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:32 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2828625593' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:44:32.801 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T08:44:32.802 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T08:44:32.802 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-09-10T08:44:32.802 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-10T08:44:32.802 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:44:32.802 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T08:44:32.802 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T08:44:32.802 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T08:44:32.802 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T08:44:32.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:44:32.803 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-09-10T08:44:32.820 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T08:44:32.820 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T08:44:32.820 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T08:44:32.820 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:44:32.820 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-09-10T08:44:32.820 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:44:32.820 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:44:32.820 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-09-10T08:44:32.821 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-09-10T08:44:32.821 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:44:32.821 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-09-10T08:44:32.821 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-09-10T08:44:32.821 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:44:32.821 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:44:32.821 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T08:44:32.821 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:44:32.821 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T08:44:32.821 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42258sec preferred_lft 42258sec 2024-09-10T08:44:32.821 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link noprefixroute 2024-09-10T08:44:32.821 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:44:32.822 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T08:44:32.822 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:44:32.822 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-09-10T08:44:32.822 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add name ceph-brx type bridge 2024-09-10T08:44:32.822 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr flush dev ceph-brx 2024-09-10T08:44:32.823 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set ceph-brx up 2024-09-10T08:44:32.823 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T08:44:32.823 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-09-10T08:44:32.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:44:33.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:44:33.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:44:33.025 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T08:44:33.051 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-09-10T08:44:33.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:44:33.053 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-09-10T08:44:33.118 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-09-10T08:44:33.118 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T08:44:33.118 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T08:44:33.118 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T08:44:33.118 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T08:44:33.119 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:44:33.119 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-09-10T08:44:33.119 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T08:44:33.119 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T08:44:33.119 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T08:44:33.119 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-09-10T08:44:33.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:44:33.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:44:33.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:44:33.473 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-09-10T08:44:33.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:44:33.489 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list-id 2024-09-10T08:44:33.545 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:44:33.545 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-09-10T08:44:33.545 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:44:33.545 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-10T08:44:33.546 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-09-10T08:44:33.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:44:33.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:44:33.664 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-10T08:44:33.664 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:44:33.664 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-09-10T08:44:33.664 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-10T08:44:33.664 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T08:44:33.664 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-10T08:44:33.665 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-10T08:44:33.665 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-10T08:44:33.665 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-09-10T08:44:33.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:44:33.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:44:33.786 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:44:33.787 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-09-10T08:44:33.787 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-09-10T08:44:33.787 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T08:44:33.787 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-09-10T08:44:33.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:44:33.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:44:33.863 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-10T08:44:33.863 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T08:44:33.863 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:44:33.878 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T08:44:33.879 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T08:44:33.879 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:44:33.934 DEBUG:teuthology.orchestra.run.smithi172:> sudo modprobe fuse 2024-09-10T08:44:34.000 DEBUG:teuthology.orchestra.run.smithi172:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T08:44:34.056 INFO:teuthology.orchestra.run.smithi172.stdout:/proc 2024-09-10T08:44:34.057 INFO:teuthology.orchestra.run.smithi172.stdout:/sys 2024-09-10T08:44:34.057 INFO:teuthology.orchestra.run.smithi172.stdout:/dev 2024-09-10T08:44:34.057 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/security 2024-09-10T08:44:34.057 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/shm 2024-09-10T08:44:34.057 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/pts 2024-09-10T08:44:34.057 INFO:teuthology.orchestra.run.smithi172.stdout:/run 2024-09-10T08:44:34.057 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/cgroup 2024-09-10T08:44:34.057 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/pstore 2024-09-10T08:44:34.057 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/bpf 2024-09-10T08:44:34.057 INFO:teuthology.orchestra.run.smithi172.stdout:/ 2024-09-10T08:44:34.057 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/selinux 2024-09-10T08:44:34.058 INFO:teuthology.orchestra.run.smithi172.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:44:34.058 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/debug 2024-09-10T08:44:34.058 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/mqueue 2024-09-10T08:44:34.058 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/hugepages 2024-09-10T08:44:34.058 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/tracing 2024-09-10T08:44:34.058 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/fuse/connections 2024-09-10T08:44:34.058 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T08:44:34.058 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/config 2024-09-10T08:44:34.058 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T08:44:34.058 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T08:44:34.058 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T08:44:34.058 INFO:teuthology.orchestra.run.smithi172.stdout:/run/user/1000 2024-09-10T08:44:34.058 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/ceph 2024-09-10T08:44:34.058 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay 2024-09-10T08:44:34.059 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/82e89ae6b50bdf753acd9f704569e643dc3c43dadf2c992d822fd3df2e2145a2/merged 2024-09-10T08:44:34.059 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/a42cc49770da75f41ece537546cf7211f25dee1fa59bf0e063d45dce66636908/merged 2024-09-10T08:44:34.059 INFO:teuthology.orchestra.run.smithi172.stdout:/run/user/0 2024-09-10T08:44:34.059 INFO:teuthology.orchestra.run.smithi172.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:44:34.059 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/c5a291b40f14d210658e79214dd8b0a20ab5c89b11b760d33358bc3eaaee4f07/merged 2024-09-10T08:44:34.059 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/9dfeb1b3da512da1821f7f773d5f0da825be3b645c8d588ef05e735a8f3179ce/merged 2024-09-10T08:44:34.059 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/fe3ade833ac302061953a0451b5f0bc6033afe2aab408dccdb42c5d7de392d73/merged 2024-09-10T08:44:34.059 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/7102ef80c772fb9c554e1496711136185f60e8668fe2ad8884b8006eef379002/merged 2024-09-10T08:44:34.059 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/a8be33b683701eaef8f6eebda60cfecf60cebf303fa9dacd6e08c920c0729ade/merged 2024-09-10T08:44:34.059 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/10e0103da8bd78312507fe36d16529f9090728d08bb98cf2ef03cbdbc8da26c2/merged 2024-09-10T08:44:34.059 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/d1a43694eb90671cd8405e144dc729b9f0db71eadbdc047d003177bbd777254c/merged 2024-09-10T08:44:34.059 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/ee332b36b97a9fb6adfb4e4de3e3dc938e9fc0a9bfe3952e9c04ed117956487c/merged 2024-09-10T08:44:34.059 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns 2024-09-10T08:44:34.060 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:44:34.060 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:44:34.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:44:34.060 DEBUG:teuthology.orchestra.run.smithi172:> ls /sys/fs/fuse/connections 2024-09-10T08:44:34.112 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T08:44:34.113 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-10T08:44:34.155 DEBUG:teuthology.orchestra.run.smithi172:> sudo modprobe fuse 2024-09-10T08:44:34.179 DEBUG:teuthology.orchestra.run.smithi172:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T08:44:34.235 INFO:teuthology.orchestra.run.smithi172.stdout:/proc 2024-09-10T08:44:34.235 INFO:teuthology.orchestra.run.smithi172.stdout:/sys 2024-09-10T08:44:34.236 INFO:teuthology.orchestra.run.smithi172.stdout:/dev 2024-09-10T08:44:34.236 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/security 2024-09-10T08:44:34.236 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/shm 2024-09-10T08:44:34.236 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/pts 2024-09-10T08:44:34.236 INFO:teuthology.orchestra.run.smithi172.stdout:/run 2024-09-10T08:44:34.236 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/cgroup 2024-09-10T08:44:34.236 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/pstore 2024-09-10T08:44:34.236 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/bpf 2024-09-10T08:44:34.236 INFO:teuthology.orchestra.run.smithi172.stdout:/ 2024-09-10T08:44:34.236 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/selinux 2024-09-10T08:44:34.236 INFO:teuthology.orchestra.run.smithi172.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:44:34.236 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/debug 2024-09-10T08:44:34.237 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/mqueue 2024-09-10T08:44:34.237 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/hugepages 2024-09-10T08:44:34.237 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/tracing 2024-09-10T08:44:34.237 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/fuse/connections 2024-09-10T08:44:34.237 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T08:44:34.237 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/config 2024-09-10T08:44:34.237 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T08:44:34.237 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T08:44:34.237 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T08:44:34.237 INFO:teuthology.orchestra.run.smithi172.stdout:/run/user/1000 2024-09-10T08:44:34.237 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/ceph 2024-09-10T08:44:34.237 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay 2024-09-10T08:44:34.237 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/82e89ae6b50bdf753acd9f704569e643dc3c43dadf2c992d822fd3df2e2145a2/merged 2024-09-10T08:44:34.238 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/a42cc49770da75f41ece537546cf7211f25dee1fa59bf0e063d45dce66636908/merged 2024-09-10T08:44:34.238 INFO:teuthology.orchestra.run.smithi172.stdout:/run/user/0 2024-09-10T08:44:34.238 INFO:teuthology.orchestra.run.smithi172.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:44:34.238 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/c5a291b40f14d210658e79214dd8b0a20ab5c89b11b760d33358bc3eaaee4f07/merged 2024-09-10T08:44:34.238 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/9dfeb1b3da512da1821f7f773d5f0da825be3b645c8d588ef05e735a8f3179ce/merged 2024-09-10T08:44:34.238 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/fe3ade833ac302061953a0451b5f0bc6033afe2aab408dccdb42c5d7de392d73/merged 2024-09-10T08:44:34.238 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/7102ef80c772fb9c554e1496711136185f60e8668fe2ad8884b8006eef379002/merged 2024-09-10T08:44:34.238 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/a8be33b683701eaef8f6eebda60cfecf60cebf303fa9dacd6e08c920c0729ade/merged 2024-09-10T08:44:34.238 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/10e0103da8bd78312507fe36d16529f9090728d08bb98cf2ef03cbdbc8da26c2/merged 2024-09-10T08:44:34.238 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/d1a43694eb90671cd8405e144dc729b9f0db71eadbdc047d003177bbd777254c/merged 2024-09-10T08:44:34.238 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/ee332b36b97a9fb6adfb4e4de3e3dc938e9fc0a9bfe3952e9c04ed117956487c/merged 2024-09-10T08:44:34.238 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns 2024-09-10T08:44:34.238 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:44:34.239 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:44:34.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:44:34.239 DEBUG:teuthology.orchestra.run.smithi172:> ls /sys/fs/fuse/connections 2024-09-10T08:44:34.257 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi172.stderr:2024-09-10T08:44:34.255+0000 7f9adbf1b480 -1 init, newargv = 0x560a22290a10 newargc=15 2024-09-10T08:44:34.257 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi172.stderr:ceph-fuse[72521]: starting ceph client 2024-09-10T08:44:34.263 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi172.stderr:ceph-fuse[72521]: starting fuse 2024-09-10T08:44:34.272 INFO:teuthology.orchestra.run.smithi172.stdout:120 2024-09-10T08:44:34.272 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-10T08:44:34.272 DEBUG:teuthology.orchestra.run.smithi172:> sudo stdin-killer -- python3 -c ' 2024-09-10T08:44:34.273 DEBUG:teuthology.orchestra.run.smithi172:> import glob 2024-09-10T08:44:34.273 DEBUG:teuthology.orchestra.run.smithi172:> import re 2024-09-10T08:44:34.273 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-09-10T08:44:34.273 DEBUG:teuthology.orchestra.run.smithi172:> import subprocess 2024-09-10T08:44:34.273 DEBUG:teuthology.orchestra.run.smithi172:> 2024-09-10T08:44:34.273 DEBUG:teuthology.orchestra.run.smithi172:> def _find_admin_socket(client_name): 2024-09-10T08:44:34.273 DEBUG:teuthology.orchestra.run.smithi172:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-10T08:44:34.273 DEBUG:teuthology.orchestra.run.smithi172:> files = glob.glob(asok_path) 2024-09-10T08:44:34.273 DEBUG:teuthology.orchestra.run.smithi172:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-10T08:44:34.273 DEBUG:teuthology.orchestra.run.smithi172:> 2024-09-10T08:44:34.273 DEBUG:teuthology.orchestra.run.smithi172:> # Given a non-glob path, it better be there 2024-09-10T08:44:34.273 DEBUG:teuthology.orchestra.run.smithi172:> if "*" not in asok_path: 2024-09-10T08:44:34.273 DEBUG:teuthology.orchestra.run.smithi172:> assert(len(files) == 1) 2024-09-10T08:44:34.273 DEBUG:teuthology.orchestra.run.smithi172:> return files[0] 2024-09-10T08:44:34.273 DEBUG:teuthology.orchestra.run.smithi172:> 2024-09-10T08:44:34.274 DEBUG:teuthology.orchestra.run.smithi172:> for f in files: 2024-09-10T08:44:34.274 DEBUG:teuthology.orchestra.run.smithi172:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T08:44:34.274 DEBUG:teuthology.orchestra.run.smithi172:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T08:44:34.274 DEBUG:teuthology.orchestra.run.smithi172:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T08:44:34.274 DEBUG:teuthology.orchestra.run.smithi172:> contents = proc_f.read() 2024-09-10T08:44:34.274 DEBUG:teuthology.orchestra.run.smithi172:> if mountpoint in contents: 2024-09-10T08:44:34.274 DEBUG:teuthology.orchestra.run.smithi172:> return f 2024-09-10T08:44:34.274 DEBUG:teuthology.orchestra.run.smithi172:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T08:44:34.274 DEBUG:teuthology.orchestra.run.smithi172:> 2024-09-10T08:44:34.274 DEBUG:teuthology.orchestra.run.smithi172:> print(_find_admin_socket("client.admin")) 2024-09-10T08:44:34.274 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-09-10T08:44:34.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:34 smithi172 ceph-mon[30854]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:34.391 INFO:teuthology.orchestra.run.smithi172.stdout:/var/run/ceph/ceph-client.admin.72521.asok 2024-09-10T08:44:34.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:44:34.402 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.72521.asok 2024-09-10T08:44:34.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:44:34.402 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.72521.asok status 2024-09-10T08:44:34.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:34 smithi174 ceph-mon[34068]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:34.535 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:44:34.535 INFO:teuthology.orchestra.run.smithi172.stdout: "metadata": { 2024-09-10T08:44:34.536 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T08:44:34.536 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T08:44:34.536 INFO:teuthology.orchestra.run.smithi172.stdout: "entity_id": "admin", 2024-09-10T08:44:34.536 INFO:teuthology.orchestra.run.smithi172.stdout: "hostname": "smithi172", 2024-09-10T08:44:34.536 INFO:teuthology.orchestra.run.smithi172.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-10T08:44:34.536 INFO:teuthology.orchestra.run.smithi172.stdout: "pid": "72521", 2024-09-10T08:44:34.536 INFO:teuthology.orchestra.run.smithi172.stdout: "root": "/" 2024-09-10T08:44:34.536 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:44:34.536 INFO:teuthology.orchestra.run.smithi172.stdout: "dentry_count": 0, 2024-09-10T08:44:34.536 INFO:teuthology.orchestra.run.smithi172.stdout: "dentry_pinned_count": 0, 2024-09-10T08:44:34.536 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 14745, 2024-09-10T08:44:34.536 INFO:teuthology.orchestra.run.smithi172.stdout: "inst": { 2024-09-10T08:44:34.537 INFO:teuthology.orchestra.run.smithi172.stdout: "name": { 2024-09-10T08:44:34.537 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "client", 2024-09-10T08:44:34.537 INFO:teuthology.orchestra.run.smithi172.stdout: "num": 14745 2024-09-10T08:44:34.537 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:44:34.537 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": { 2024-09-10T08:44:34.537 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v1", 2024-09-10T08:44:34.537 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "192.168.0.1:0", 2024-09-10T08:44:34.537 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 398742092 2024-09-10T08:44:34.537 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:44:34.537 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:44:34.537 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": { 2024-09-10T08:44:34.537 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v1", 2024-09-10T08:44:34.537 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "192.168.0.1:0", 2024-09-10T08:44:34.538 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 398742092 2024-09-10T08:44:34.538 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:44:34.538 INFO:teuthology.orchestra.run.smithi172.stdout: "inst_str": "client.14745 192.168.0.1:0/398742092", 2024-09-10T08:44:34.538 INFO:teuthology.orchestra.run.smithi172.stdout: "addr_str": "192.168.0.1:0/398742092", 2024-09-10T08:44:34.538 INFO:teuthology.orchestra.run.smithi172.stdout: "inode_count": 1, 2024-09-10T08:44:34.538 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_epoch": 17, 2024-09-10T08:44:34.538 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_epoch": 88, 2024-09-10T08:44:34.538 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_epoch_barrier": 0, 2024-09-10T08:44:34.538 INFO:teuthology.orchestra.run.smithi172.stdout: "blocklisted": false, 2024-09-10T08:44:34.538 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs" 2024-09-10T08:44:34.538 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:44:34.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:44:34.544 DEBUG:teuthology.orchestra.run.smithi172:> stat --file-system '--printf=%T 2024-09-10T08:44:34.544 DEBUG:teuthology.orchestra.run.smithi172:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:44:34.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:34 smithi191 ceph-mon[35314]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:34.560 INFO:teuthology.orchestra.run.smithi172.stdout:fuseblk 2024-09-10T08:44:34.560 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:44:34.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:44:34.561 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:44:34.626 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-10T08:44:34.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T08:44:34.749 INFO:teuthology.orchestra.run.smithi172.stdout:# file: . 2024-09-10T08:44:34.749 INFO:teuthology.orchestra.run.smithi172.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-10T08:44:34.749 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:44:34.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:44:34.755 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:44:34.755 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T08:44:34.772 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T08:44:35.060 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:35.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.599+0000 7f77e3c72640 1 -- 172.21.15.172:0/3951729388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77dc100e20 msgr2=0x7f77dc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:35.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.599+0000 7f77e3c72640 1 --2- 172.21.15.172:0/3951729388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77dc100e20 0x7f77dc101200 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009ca0 tx=0x7f77d402a950 comp rx=0 tx=0).stop 2024-09-10T08:44:35.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.599+0000 7f77e3c72640 1 -- 172.21.15.172:0/3951729388 shutdown_connections 2024-09-10T08:44:35.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.599+0000 7f77e3c72640 1 --2- 172.21.15.172:0/3951729388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77dc102ef0 0x7f77dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:35.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.599+0000 7f77e3c72640 1 --2- 172.21.15.172:0/3951729388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77dc102570 0x7f77dc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:35.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.599+0000 7f77e3c72640 1 --2- 172.21.15.172:0/3951729388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77dc100e20 0x7f77dc101200 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:35.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.599+0000 7f77e3c72640 1 -- 172.21.15.172:0/3951729388 >> 172.21.15.172:0/3951729388 conn(0x7f77dc0fc7d0 msgr2=0x7f77dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:35.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.599+0000 7f77e3c72640 1 -- 172.21.15.172:0/3951729388 shutdown_connections 2024-09-10T08:44:35.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.599+0000 7f77e3c72640 1 -- 172.21.15.172:0/3951729388 wait complete. 2024-09-10T08:44:35.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.599+0000 7f77e3c72640 1 Processor -- start 2024-09-10T08:44:35.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.599+0000 7f77e3c72640 1 -- start start 2024-09-10T08:44:35.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e3c72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77dc100e20 0x7f77dc1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:35.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e3c72640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77dc102570 0x7f77dc1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:35.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e3c72640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77dc102ef0 0x7f77dc19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:35.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e3c72640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc110e30 con 0x7f77dc100e20 2024-09-10T08:44:35.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e3c72640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc110cb0 con 0x7f77dc102570 2024-09-10T08:44:35.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e3c72640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc110fb0 con 0x7f77dc102ef0 2024-09-10T08:44:35.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e1a0e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77dc100e20 0x7f77dc1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:35.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e1a0e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77dc100e20 0x7f77dc1972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48910/0 (socket says 172.21.15.172:48910) 2024-09-10T08:44:35.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e1a0e640 1 -- 172.21.15.172:0/2438817576 learned_addr learned my addr 172.21.15.172:0/2438817576 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:35.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e120d640 1 --2- 172.21.15.172:0/2438817576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77dc102570 0x7f77dc1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:35.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e220f640 1 --2- 172.21.15.172:0/2438817576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77dc102ef0 0x7f77dc19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:35.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e1a0e640 1 -- 172.21.15.172:0/2438817576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77dc102ef0 msgr2=0x7f77dc19bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:35.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e1a0e640 1 --2- 172.21.15.172:0/2438817576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77dc102ef0 0x7f77dc19bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:35.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e1a0e640 1 -- 172.21.15.172:0/2438817576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77dc102570 msgr2=0x7f77dc1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:35.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e1a0e640 1 --2- 172.21.15.172:0/2438817576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77dc102570 0x7f77dc1977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:35.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e1a0e640 1 -- 172.21.15.172:0/2438817576 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77dc19c260 con 0x7f77dc100e20 2024-09-10T08:44:35.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e1a0e640 1 --2- 172.21.15.172:0/2438817576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77dc100e20 0x7f77dc1972b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f77d4005cc0 tx=0x7f77d402f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:35.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77d2ffd640 1 -- 172.21.15.172:0/2438817576 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d4034040 con 0x7f77dc100e20 2024-09-10T08:44:35.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77d2ffd640 1 -- 172.21.15.172:0/2438817576 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77d40308f0 con 0x7f77dc100e20 2024-09-10T08:44:35.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77e3c72640 1 -- 172.21.15.172:0/2438817576 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77dc19c4f0 con 0x7f77dc100e20 2024-09-10T08:44:35.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.600+0000 7f77d2ffd640 1 -- 172.21.15.172:0/2438817576 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d40047c0 con 0x7f77dc100e20 2024-09-10T08:44:35.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.601+0000 7f77e3c72640 1 -- 172.21.15.172:0/2438817576 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77dc1a3e30 con 0x7f77dc100e20 2024-09-10T08:44:35.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.601+0000 7f77e3c72640 1 -- 172.21.15.172:0/2438817576 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77dc1021b0 con 0x7f77dc100e20 2024-09-10T08:44:35.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.603+0000 7f77d2ffd640 1 -- 172.21.15.172:0/2438817576 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f77d4037020 con 0x7f77dc100e20 2024-09-10T08:44:35.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.604+0000 7f77d2ffd640 1 --2- 172.21.15.172:0/2438817576 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f77c80a2b80 0x7f77c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:35.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.604+0000 7f77e120d640 1 --2- 172.21.15.172:0/2438817576 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f77c80a2b80 0x7f77c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:35.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.604+0000 7f77d2ffd640 1 -- 172.21.15.172:0/2438817576 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f77d40f03b0 con 0x7f77dc100e20 2024-09-10T08:44:35.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.604+0000 7f77e120d640 1 --2- 172.21.15.172:0/2438817576 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f77c80a2b80 0x7f77c80a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0ffee0 tx=0x7f77cc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:35.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.607+0000 7f77d2ffd640 1 -- 172.21.15.172:0/2438817576 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77d40bd070 con 0x7f77dc100e20 2024-09-10T08:44:35.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.787+0000 7f77e3c72640 1 -- 172.21.15.172:0/2438817576 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f77dc101200 con 0x7f77dc100e20 2024-09-10T08:44:35.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.787+0000 7f77d2ffd640 1 -- 172.21.15.172:0/2438817576 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+600 (secure 0 0 0) 0x7f77dc101200 con 0x7f77dc100e20 2024-09-10T08:44:35.789 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/977245917 2024-09-11T08:37:17.019272+0000 2024-09-10T08:44:35.789 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6801/2333617420 2024-09-11T08:37:17.019272+0000 2024-09-10T08:44:35.789 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/3919952878 2024-09-11T08:36:54.887256+0000 2024-09-10T08:44:35.789 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6800/2333617420 2024-09-11T08:37:17.019272+0000 2024-09-10T08:44:35.790 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/1525385361 2024-09-11T08:36:54.887256+0000 2024-09-10T08:44:35.790 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/2043423408 2024-09-11T08:37:17.019272+0000 2024-09-10T08:44:35.790 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6801/2639090029 2024-09-11T08:36:54.887256+0000 2024-09-10T08:44:35.790 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/839344299 2024-09-11T08:37:17.019272+0000 2024-09-10T08:44:35.790 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6800/2639090029 2024-09-11T08:36:54.887256+0000 2024-09-10T08:44:35.790 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/3251817286 2024-09-11T08:36:54.887256+0000 2024-09-10T08:44:35.790 INFO:teuthology.orchestra.run.smithi172.stderr:listed 10 entries 2024-09-10T08:44:35.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.790+0000 7f77e3c72640 1 -- 172.21.15.172:0/2438817576 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f77c80a2b80 msgr2=0x7f77c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:35.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.790+0000 7f77e3c72640 1 --2- 172.21.15.172:0/2438817576 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f77c80a2b80 0x7f77c80a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0ffee0 tx=0x7f77cc00b390 comp rx=0 tx=0).stop 2024-09-10T08:44:35.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.790+0000 7f77e3c72640 1 -- 172.21.15.172:0/2438817576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77dc100e20 msgr2=0x7f77dc1972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:35.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.790+0000 7f77e3c72640 1 --2- 172.21.15.172:0/2438817576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77dc100e20 0x7f77dc1972b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f77d4005cc0 tx=0x7f77d402f650 comp rx=0 tx=0).stop 2024-09-10T08:44:35.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.790+0000 7f77e3c72640 1 -- 172.21.15.172:0/2438817576 shutdown_connections 2024-09-10T08:44:35.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.790+0000 7f77e3c72640 1 --2- 172.21.15.172:0/2438817576 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f77c80a2b80 0x7f77c80a5040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:35.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.790+0000 7f77e3c72640 1 --2- 172.21.15.172:0/2438817576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77dc102ef0 0x7f77dc19bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:35.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.790+0000 7f77e3c72640 1 --2- 172.21.15.172:0/2438817576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77dc102570 0x7f77dc1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:35.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.790+0000 7f77e3c72640 1 --2- 172.21.15.172:0/2438817576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77dc100e20 0x7f77dc1972b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:35.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.790+0000 7f77e3c72640 1 -- 172.21.15.172:0/2438817576 >> 172.21.15.172:0/2438817576 conn(0x7f77dc0fc7d0 msgr2=0x7f77dc0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:35.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.791+0000 7f77e3c72640 1 -- 172.21.15.172:0/2438817576 shutdown_connections 2024-09-10T08:44:35.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:35.791+0000 7f77e3c72640 1 -- 172.21.15.172:0/2438817576 wait complete. 2024-09-10T08:44:36.431 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:36 smithi172 ceph-mon[30854]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:36.431 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:36 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2438817576' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:44:36.460 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:44:36.460 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T08:44:36.479 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T08:44:36.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:36 smithi174 ceph-mon[34068]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:36.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:36 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2438817576' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:44:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:36 smithi191 ceph-mon[35314]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T08:44:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:36 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2438817576' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:44:36.768 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:37.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.249+0000 7facc4b6a640 1 -- 172.21.15.172:0/2351331495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc0102570 msgr2=0x7facc01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:37.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.249+0000 7facc4b6a640 1 --2- 172.21.15.172:0/2351331495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc0102570 0x7facc01029b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faca8009900 tx=0x7faca802a480 comp rx=0 tx=0).stop 2024-09-10T08:44:37.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.250+0000 7facc4b6a640 1 -- 172.21.15.172:0/2351331495 shutdown_connections 2024-09-10T08:44:37.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.250+0000 7facc4b6a640 1 --2- 172.21.15.172:0/2351331495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facc0102ef0 0x7facc010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:37.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.250+0000 7facc4b6a640 1 --2- 172.21.15.172:0/2351331495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc0102570 0x7facc01029b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:37.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.250+0000 7facc4b6a640 1 --2- 172.21.15.172:0/2351331495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facc0100e20 0x7facc0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:37.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.250+0000 7facc4b6a640 1 -- 172.21.15.172:0/2351331495 >> 172.21.15.172:0/2351331495 conn(0x7facc00fc7d0 msgr2=0x7facc00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:37.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.250+0000 7facc4b6a640 1 -- 172.21.15.172:0/2351331495 shutdown_connections 2024-09-10T08:44:37.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.250+0000 7facc4b6a640 1 -- 172.21.15.172:0/2351331495 wait complete. 2024-09-10T08:44:37.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facc4b6a640 1 Processor -- start 2024-09-10T08:44:37.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facc4b6a640 1 -- start start 2024-09-10T08:44:37.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facc4b6a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facc0100e20 0x7facc0197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:37.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facc4b6a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc0102570 0x7facc01978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:37.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facc4b6a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facc0102ef0 0x7facc019bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:37.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facc4b6a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7facc0110e30 con 0x7facc0102570 2024-09-10T08:44:37.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facc4b6a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7facc0110cb0 con 0x7facc0100e20 2024-09-10T08:44:37.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facc4b6a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7facc0110fb0 con 0x7facc0102ef0 2024-09-10T08:44:37.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facbdd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc0102570 0x7facc01978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:37.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facbdd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc0102570 0x7facc01978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51182/0 (socket says 172.21.15.172:51182) 2024-09-10T08:44:37.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facbdd9b640 1 -- 172.21.15.172:0/3291500442 learned_addr learned my addr 172.21.15.172:0/3291500442 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:37.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facbe59c640 1 --2- 172.21.15.172:0/3291500442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facc0100e20 0x7facc0197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:37.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facbdd9b640 1 -- 172.21.15.172:0/3291500442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facc0102ef0 msgr2=0x7facc019bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:37.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facbed9d640 1 --2- 172.21.15.172:0/3291500442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facc0102ef0 0x7facc019bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:37.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facbdd9b640 1 --2- 172.21.15.172:0/3291500442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facc0102ef0 0x7facc019bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:37.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facbdd9b640 1 -- 172.21.15.172:0/3291500442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facc0100e20 msgr2=0x7facc0197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:37.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facbdd9b640 1 --2- 172.21.15.172:0/3291500442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facc0100e20 0x7facc0197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:37.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facbdd9b640 1 -- 172.21.15.172:0/3291500442 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc019c340 con 0x7facc0102570 2024-09-10T08:44:37.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.251+0000 7facbe59c640 1 --2- 172.21.15.172:0/3291500442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facc0100e20 0x7facc0197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:37.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.252+0000 7facbdd9b640 1 --2- 172.21.15.172:0/3291500442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc0102570 0x7facc01978d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faca8006010 tx=0x7faca80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:37.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.252+0000 7facb77fe640 1 -- 172.21.15.172:0/3291500442 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca803f030 con 0x7facc0102570 2024-09-10T08:44:37.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.252+0000 7facb77fe640 1 -- 172.21.15.172:0/3291500442 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faca8003cb0 con 0x7facc0102570 2024-09-10T08:44:37.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.252+0000 7facc4b6a640 1 -- 172.21.15.172:0/3291500442 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc019c5d0 con 0x7facc0102570 2024-09-10T08:44:37.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.252+0000 7facb77fe640 1 -- 172.21.15.172:0/3291500442 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca8035070 con 0x7facc0102570 2024-09-10T08:44:37.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.252+0000 7facc4b6a640 1 -- 172.21.15.172:0/3291500442 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc01a3f10 con 0x7facc0102570 2024-09-10T08:44:37.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.253+0000 7facc4b6a640 1 -- 172.21.15.172:0/3291500442 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc01021b0 con 0x7facc0102570 2024-09-10T08:44:37.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.254+0000 7facb77fe640 1 -- 172.21.15.172:0/3291500442 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7faca8042030 con 0x7facc0102570 2024-09-10T08:44:37.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.255+0000 7facb77fe640 1 --2- 172.21.15.172:0/3291500442 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7facac0a2b80 0x7facac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:37.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.255+0000 7facbe59c640 1 --2- 172.21.15.172:0/3291500442 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7facac0a2b80 0x7facac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:37.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.255+0000 7facb77fe640 1 -- 172.21.15.172:0/3291500442 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7faca80efaa0 con 0x7facc0102570 2024-09-10T08:44:37.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.255+0000 7facbe59c640 1 --2- 172.21.15.172:0/3291500442 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7facac0a2b80 0x7facac0a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7facb80058c0 tx=0x7facb8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:37.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.258+0000 7facb77fe640 1 -- 172.21.15.172:0/3291500442 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faca80bc7c0 con 0x7facc0102570 2024-09-10T08:44:37.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.434+0000 7facc4b6a640 1 -- 172.21.15.172:0/3291500442 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7facc0109090 con 0x7facc0102570 2024-09-10T08:44:37.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.435+0000 7facb77fe640 1 -- 172.21.15.172:0/3291500442 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+600 (secure 0 0 0) 0x7faca80bc0c0 con 0x7facc0102570 2024-09-10T08:44:37.435 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/977245917 2024-09-11T08:37:17.019272+0000 2024-09-10T08:44:37.435 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6801/2333617420 2024-09-11T08:37:17.019272+0000 2024-09-10T08:44:37.436 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/3919952878 2024-09-11T08:36:54.887256+0000 2024-09-10T08:44:37.436 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6800/2333617420 2024-09-11T08:37:17.019272+0000 2024-09-10T08:44:37.436 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/1525385361 2024-09-11T08:36:54.887256+0000 2024-09-10T08:44:37.436 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/2043423408 2024-09-11T08:37:17.019272+0000 2024-09-10T08:44:37.436 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6801/2639090029 2024-09-11T08:36:54.887256+0000 2024-09-10T08:44:37.436 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/839344299 2024-09-11T08:37:17.019272+0000 2024-09-10T08:44:37.436 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6800/2639090029 2024-09-11T08:36:54.887256+0000 2024-09-10T08:44:37.436 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/3251817286 2024-09-11T08:36:54.887256+0000 2024-09-10T08:44:37.436 INFO:teuthology.orchestra.run.smithi172.stderr:listed 10 entries 2024-09-10T08:44:37.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.437+0000 7facc4b6a640 1 -- 172.21.15.172:0/3291500442 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7facac0a2b80 msgr2=0x7facac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:37.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.437+0000 7facc4b6a640 1 --2- 172.21.15.172:0/3291500442 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7facac0a2b80 0x7facac0a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7facb80058c0 tx=0x7facb8005850 comp rx=0 tx=0).stop 2024-09-10T08:44:37.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.437+0000 7facc4b6a640 1 -- 172.21.15.172:0/3291500442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc0102570 msgr2=0x7facc01978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:37.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.437+0000 7facc4b6a640 1 --2- 172.21.15.172:0/3291500442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc0102570 0x7facc01978d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faca8006010 tx=0x7faca80042f0 comp rx=0 tx=0).stop 2024-09-10T08:44:37.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.437+0000 7facc4b6a640 1 -- 172.21.15.172:0/3291500442 shutdown_connections 2024-09-10T08:44:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.437+0000 7facc4b6a640 1 --2- 172.21.15.172:0/3291500442 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7facac0a2b80 0x7facac0a5040 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.437+0000 7facc4b6a640 1 --2- 172.21.15.172:0/3291500442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facc0102ef0 0x7facc019bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.437+0000 7facc4b6a640 1 --2- 172.21.15.172:0/3291500442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7facc0102570 0x7facc01978d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.437+0000 7facc4b6a640 1 --2- 172.21.15.172:0/3291500442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facc0100e20 0x7facc0197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.437+0000 7facc4b6a640 1 -- 172.21.15.172:0/3291500442 >> 172.21.15.172:0/3291500442 conn(0x7facc00fc7d0 msgr2=0x7facc00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.437+0000 7facc4b6a640 1 -- 172.21.15.172:0/3291500442 shutdown_connections 2024-09-10T08:44:37.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:37.437+0000 7facc4b6a640 1 -- 172.21.15.172:0/3291500442 wait complete. 2024-09-10T08:44:38.062 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi172.front.sepia.ceph.com... 2024-09-10T08:44:38.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:44:38.062 DEBUG:teuthology.orchestra.run.smithi172:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-10T08:44:38.088 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi172.stderr:ceph-fuse[72521]: fuse finished with error 0 and tester_r 0 2024-09-10T08:44:38.089 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T08:44:38.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:38 smithi174 ceph-mon[34068]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T08:44:38.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:38 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3291500442' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:44:38.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:38 smithi191 ceph-mon[35314]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T08:44:38.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:38 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3291500442' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:44:38.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:38 smithi172 ceph-mon[30854]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T08:44:38.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:38 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3291500442' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T08:44:40.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:40 smithi174 ceph-mon[34068]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T08:44:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:40 smithi191 ceph-mon[35314]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T08:44:40.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:40 smithi172 ceph-mon[30854]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T08:44:42.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:42 smithi174 ceph-mon[34068]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:44:42.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:42 smithi191 ceph-mon[35314]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:44:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:42 smithi172 ceph-mon[30854]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:44:44.192 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-09-10T08:44:44.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:44:44.192 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-10T08:44:44.228 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:44:44.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:44 smithi174 ceph-mon[34068]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:44:44.530 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:44.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:44 smithi191 ceph-mon[35314]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:44:44.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:44 smithi172 ceph-mon[30854]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:44:45.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.036+0000 7f936993e640 1 -- 172.21.15.172:0/1930928477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9364100a70 msgr2=0x7f936410ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:45.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.036+0000 7f936993e640 1 --2- 172.21.15.172:0/1930928477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9364100a70 0x7f936410ea60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f935c009900 tx=0x7f935c02a410 comp rx=0 tx=0).stop 2024-09-10T08:44:45.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.036+0000 7f936993e640 1 -- 172.21.15.172:0/1930928477 shutdown_connections 2024-09-10T08:44:45.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.036+0000 7f936993e640 1 --2- 172.21.15.172:0/1930928477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9364100a70 0x7f936410ea60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:45.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.036+0000 7f936993e640 1 --2- 172.21.15.172:0/1930928477 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93641000d0 0x7f9364100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:45.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.036+0000 7f936993e640 1 --2- 172.21.15.172:0/1930928477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9364106ba0 0x7f9364106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:45.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.036+0000 7f936993e640 1 -- 172.21.15.172:0/1930928477 >> 172.21.15.172:0/1930928477 conn(0x7f9364077210 msgr2=0x7f93640fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:45.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.036+0000 7f936993e640 1 -- 172.21.15.172:0/1930928477 shutdown_connections 2024-09-10T08:44:45.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.036+0000 7f936993e640 1 -- 172.21.15.172:0/1930928477 wait complete. 2024-09-10T08:44:45.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.037+0000 7f936993e640 1 Processor -- start 2024-09-10T08:44:45.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.037+0000 7f936993e640 1 -- start start 2024-09-10T08:44:45.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.037+0000 7f936993e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93641000d0 0x7f93641012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:45.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.037+0000 7f936993e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9364100a70 0x7f93641017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:45.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.037+0000 7f936993e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9364106ba0 0x7f9364101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:45.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.037+0000 7f936993e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9364111f40 con 0x7f9364106ba0 2024-09-10T08:44:45.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.037+0000 7f936993e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9364111dc0 con 0x7f93641000d0 2024-09-10T08:44:45.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.037+0000 7f936993e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f93641120c0 con 0x7f9364100a70 2024-09-10T08:44:45.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.037+0000 7f93637fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9364106ba0 0x7f9364101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:45.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.037+0000 7f93637fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9364106ba0 0x7f9364101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51206/0 (socket says 172.21.15.172:51206) 2024-09-10T08:44:45.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.037+0000 7f93637fe640 1 -- 172.21.15.172:0/3672009256 learned_addr learned my addr 172.21.15.172:0/3672009256 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:45.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.038+0000 7f93627fc640 1 --2- 172.21.15.172:0/3672009256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9364100a70 0x7f93641017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:45.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.038+0000 7f9362ffd640 1 --2- 172.21.15.172:0/3672009256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93641000d0 0x7f93641012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:45.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.038+0000 7f93637fe640 1 -- 172.21.15.172:0/3672009256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9364100a70 msgr2=0x7f93641017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:45.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.038+0000 7f93637fe640 1 --2- 172.21.15.172:0/3672009256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9364100a70 0x7f93641017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:45.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.038+0000 7f93637fe640 1 -- 172.21.15.172:0/3672009256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93641000d0 msgr2=0x7f93641012b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:45.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.038+0000 7f93637fe640 1 --2- 172.21.15.172:0/3672009256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93641000d0 0x7f93641012b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:45.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.038+0000 7f93637fe640 1 -- 172.21.15.172:0/3672009256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93641a68a0 con 0x7f9364106ba0 2024-09-10T08:44:45.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.038+0000 7f93627fc640 1 --2- 172.21.15.172:0/3672009256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9364100a70 0x7f93641017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:45.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.038+0000 7f93637fe640 1 --2- 172.21.15.172:0/3672009256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9364106ba0 0x7f9364101d30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f935c005ab0 tx=0x7f935c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:45.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.038+0000 7f936893c640 1 -- 172.21.15.172:0/3672009256 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c03f030 con 0x7f9364106ba0 2024-09-10T08:44:45.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.038+0000 7f936893c640 1 -- 172.21.15.172:0/3672009256 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f935c003cb0 con 0x7f9364106ba0 2024-09-10T08:44:45.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.038+0000 7f936993e640 1 -- 172.21.15.172:0/3672009256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93641a6b30 con 0x7f9364106ba0 2024-09-10T08:44:45.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.038+0000 7f936893c640 1 -- 172.21.15.172:0/3672009256 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c035070 con 0x7f9364106ba0 2024-09-10T08:44:45.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.038+0000 7f936993e640 1 -- 172.21.15.172:0/3672009256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93641a7010 con 0x7f9364106ba0 2024-09-10T08:44:45.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.039+0000 7f936993e640 1 -- 172.21.15.172:0/3672009256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9364100530 con 0x7f9364106ba0 2024-09-10T08:44:45.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.040+0000 7f936893c640 1 -- 172.21.15.172:0/3672009256 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f935c042030 con 0x7f9364106ba0 2024-09-10T08:44:45.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.041+0000 7f936893c640 1 --2- 172.21.15.172:0/3672009256 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f934c0a2b80 0x7f934c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:45.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.041+0000 7f936893c640 1 -- 172.21.15.172:0/3672009256 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f935c0efd50 con 0x7f9364106ba0 2024-09-10T08:44:45.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.041+0000 7f9362ffd640 1 --2- 172.21.15.172:0/3672009256 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f934c0a2b80 0x7f934c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:45.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.041+0000 7f9362ffd640 1 --2- 172.21.15.172:0/3672009256 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f934c0a2b80 0x7f934c0a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9354005fd0 tx=0x7f9354009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:45.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.044+0000 7f936893c640 1 -- 172.21.15.172:0/3672009256 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f935c0bca70 con 0x7f9364106ba0 2024-09-10T08:44:45.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.212+0000 7f936993e640 1 -- 172.21.15.172:0/3672009256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f936410a0b0 con 0x7f9364106ba0 2024-09-10T08:44:45.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.213+0000 7f936893c640 1 -- 172.21.15.172:0/3672009256 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24294 (secure 0 0 0) 0x7f935c0bc370 con 0x7f9364106ba0 2024-09-10T08:44:45.214 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:44:45.214 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":88,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","created":"2024-09-10T08:36:33.083529+0000","modified":"2024-09-10T08:44:20.275987+0000","last_up_change":"2024-09-10T08:43:19.615640+0000","last_in_change":"2024-09-10T08:42:56.656115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:39:31.328306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:44:07.466637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:44:09.520569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:44:15.926057+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6803","nonce":2678426952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6805","nonce":2678426952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6809","nonce":2678426952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6807","nonce":2678426952}]},"public_addr":"172.21.15.172:6803/2678426952","cluster_addr":"172.21.15.172:6805/2678426952","heartbeat_back_addr":"172.21.15.172:6809/2678426952","heartbeat_front_addr":"172.21.15.172:6807/2678426952","state":["exists","up"]},{"osd":1,"uuid":"55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6801","nonce":3279661439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6803","nonce":3279661439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6807","nonce":3279661439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6805","nonce":3279661439}]},"public_addr":"172.21.15.174:6801/3279661439","cluster_addr":"172.21.15.174:6803/3279661439","heartbeat_back_addr":"172.21.15.174:6807/3279661439","heartbeat_front_addr":"172.21.15.174:6805/3279661439","state":["exists","up"]},{"osd":2,"uuid":"f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6801","nonce":1963796290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6803","nonce":1963796290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6807","nonce":1963796290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6805","nonce":1963796290}]},"public_addr":"172.21.15.191:6801/1963796290","cluster_addr":"172.21.15.191:6803/1963796290","heartbeat_back_addr":"172.21.15.191:6807/1963796290","heartbeat_front_addr":"172.21.15.191:6805/1963796290","state":["exists","up"]},{"osd":3,"uuid":"15ecca3f-bc6b-4a06-b283-b515384beb19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6811","nonce":869546480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6813","nonce":869546480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6817","nonce":869546480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6815","nonce":869546480}]},"public_addr":"172.21.15.172:6811/869546480","cluster_addr":"172.21.15.172:6813/869546480","heartbeat_back_addr":"172.21.15.172:6817/869546480","heartbeat_front_addr":"172.21.15.172:6815/869546480","state":["exists","up"]},{"osd":4,"uuid":"8c2c1d40-72ad-4a15-ab72-54871c7e2427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6809","nonce":961007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6811","nonce":961007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6815","nonce":961007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6813","nonce":961007949}]},"public_addr":"172.21.15.174:6809/961007949","cluster_addr":"172.21.15.174:6811/961007949","heartbeat_back_addr":"172.21.15.174:6815/961007949","heartbeat_front_addr":"172.21.15.174:6813/961007949","state":["exists","up"]},{"osd":5,"uuid":"4de45a1d-ef0e-4bfe-ba15-5eeeb8806155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6809","nonce":4216938371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6811","nonce":4216938371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6815","nonce":4216938371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6813","nonce":4216938371}]},"public_addr":"172.21.15.191:6809/4216938371","cluster_addr":"172.21.15.191:6811/4216938371","heartbeat_back_addr":"172.21.15.191:6815/4216938371","heartbeat_front_addr":"172.21.15.191:6813/4216938371","state":["exists","up"]},{"osd":6,"uuid":"60a9938a-f696-4db2-932c-402997543b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6819","nonce":3210762009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6821","nonce":3210762009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6825","nonce":3210762009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6823","nonce":3210762009}]},"public_addr":"172.21.15.172:6819/3210762009","cluster_addr":"172.21.15.172:6821/3210762009","heartbeat_back_addr":"172.21.15.172:6825/3210762009","heartbeat_front_addr":"172.21.15.172:6823/3210762009","state":["exists","up"]},{"osd":7,"uuid":"7f0202f3-971e-472f-9c4d-ae3c24d39f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6817","nonce":1461923581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6819","nonce":1461923581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6823","nonce":1461923581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6821","nonce":1461923581}]},"public_addr":"172.21.15.174:6817/1461923581","cluster_addr":"172.21.15.174:6819/1461923581","heartbeat_back_addr":"172.21.15.174:6823/1461923581","heartbeat_front_addr":"172.21.15.174:6821/1461923581","state":["exists","up"]},{"osd":8,"uuid":"b4e7327f-ca9f-4833-9cd3-b8529b3d9761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6817","nonce":3675210430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6819","nonce":3675210430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6823","nonce":3675210430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6821","nonce":3675210430}]},"public_addr":"172.21.15.191:6817/3675210430","cluster_addr":"172.21.15.191:6819/3675210430","heartbeat_back_addr":"172.21.15.191:6823/3675210430","heartbeat_front_addr":"172.21.15.191:6821/3675210430","state":["exists","up"]},{"osd":9,"uuid":"9f6153f2-f9db-4c00-a659-c932628016f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6827","nonce":1181268619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6829","nonce":1181268619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6833","nonce":1181268619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6831","nonce":1181268619}]},"public_addr":"172.21.15.172:6827/1181268619","cluster_addr":"172.21.15.172:6829/1181268619","heartbeat_back_addr":"172.21.15.172:6833/1181268619","heartbeat_front_addr":"172.21.15.172:6831/1181268619","state":["exists","up"]},{"osd":10,"uuid":"040a013a-4c3e-442f-bea6-12cbe517716d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6825","nonce":1238812525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6827","nonce":1238812525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6831","nonce":1238812525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6829","nonce":1238812525}]},"public_addr":"172.21.15.174:6825/1238812525","cluster_addr":"172.21.15.174:6827/1238812525","heartbeat_back_addr":"172.21.15.174:6831/1238812525","heartbeat_front_addr":"172.21.15.174:6829/1238812525","state":["exists","up"]},{"osd":11,"uuid":"ae6535bd-1510-4bf3-bf4b-1a2a256add43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6825","nonce":2162719997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6827","nonce":2162719997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6831","nonce":2162719997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6829","nonce":2162719997}]},"public_addr":"172.21.15.191:6825/2162719997","cluster_addr":"172.21.15.191:6827/2162719997","heartbeat_back_addr":"172.21.15.191:6831/2162719997","heartbeat_front_addr":"172.21.15.191:6829/2162719997","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:38:38.812596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:04.765652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:29.103966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:56.085849+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:21.641784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:47.750376+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:11.640697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:36.638840+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:02.167446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:28.655804+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:52.749676+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:43:17.783908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:0/977245917":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/3919952878":"2024-09-11T08:36:54.887256+0000","172.21.15.172:6800/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/1525385361":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/2043423408":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/839344299":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6800/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/3251817286":"2024-09-11T08:36:54.887256+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:44:45.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.216+0000 7f936993e640 1 -- 172.21.15.172:0/3672009256 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f934c0a2b80 msgr2=0x7f934c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:45.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.216+0000 7f936993e640 1 --2- 172.21.15.172:0/3672009256 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f934c0a2b80 0x7f934c0a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9354005fd0 tx=0x7f9354009260 comp rx=0 tx=0).stop 2024-09-10T08:44:45.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.216+0000 7f936993e640 1 -- 172.21.15.172:0/3672009256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9364106ba0 msgr2=0x7f9364101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:45.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.216+0000 7f936993e640 1 --2- 172.21.15.172:0/3672009256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9364106ba0 0x7f9364101d30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f935c005ab0 tx=0x7f935c0042f0 comp rx=0 tx=0).stop 2024-09-10T08:44:45.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.216+0000 7f936993e640 1 -- 172.21.15.172:0/3672009256 shutdown_connections 2024-09-10T08:44:45.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.216+0000 7f936993e640 1 --2- 172.21.15.172:0/3672009256 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f934c0a2b80 0x7f934c0a5040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:45.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.216+0000 7f936993e640 1 --2- 172.21.15.172:0/3672009256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9364106ba0 0x7f9364101d30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:45.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.216+0000 7f936993e640 1 --2- 172.21.15.172:0/3672009256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9364100a70 0x7f93641017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:45.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.216+0000 7f936993e640 1 --2- 172.21.15.172:0/3672009256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93641000d0 0x7f93641012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:45.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.216+0000 7f936993e640 1 -- 172.21.15.172:0/3672009256 >> 172.21.15.172:0/3672009256 conn(0x7f9364077210 msgr2=0x7f9364104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:45.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.216+0000 7f936993e640 1 -- 172.21.15.172:0/3672009256 shutdown_connections 2024-09-10T08:44:45.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:45.216+0000 7f936993e640 1 -- 172.21.15.172:0/3672009256 wait complete. 2024-09-10T08:44:45.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:45 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3672009256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:44:45.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:45 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3672009256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:44:45.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:45 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3672009256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:44:46.044 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:44:46.298 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:46.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:46 smithi172 ceph-mon[30854]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:44:46.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:46 smithi174 ceph-mon[34068]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:44:46.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:46 smithi191 ceph-mon[35314]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:44:46.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.816+0000 7f501f4f4640 1 -- 172.21.15.172:0/486578791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5018106a60 msgr2=0x7f5018106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:46.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.816+0000 7f501f4f4640 1 --2- 172.21.15.172:0/486578791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5018106a60 0x7f5018106e40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5008009930 tx=0x7f500802a420 comp rx=0 tx=0).stop 2024-09-10T08:44:46.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.816+0000 7f501f4f4640 1 -- 172.21.15.172:0/486578791 shutdown_connections 2024-09-10T08:44:46.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.816+0000 7f501f4f4640 1 --2- 172.21.15.172:0/486578791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5018101400 0x7f501810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:46.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.816+0000 7f501f4f4640 1 --2- 172.21.15.172:0/486578791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5018100a60 0x7f5018100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:46.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.816+0000 7f501f4f4640 1 --2- 172.21.15.172:0/486578791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5018106a60 0x7f5018106e40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:46.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.816+0000 7f501f4f4640 1 -- 172.21.15.172:0/486578791 >> 172.21.15.172:0/486578791 conn(0x7f50180fc780 msgr2=0x7f50180feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:46.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.816+0000 7f501f4f4640 1 -- 172.21.15.172:0/486578791 shutdown_connections 2024-09-10T08:44:46.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.816+0000 7f501f4f4640 1 -- 172.21.15.172:0/486578791 wait complete. 2024-09-10T08:44:46.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.816+0000 7f501f4f4640 1 Processor -- start 2024-09-10T08:44:46.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501f4f4640 1 -- start start 2024-09-10T08:44:46.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501f4f4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5018100a60 0x7f501819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:46.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501f4f4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5018101400 0x7f501819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:46.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501f4f4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5018106a60 0x7f501819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:46.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501f4f4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5018110d10 con 0x7f5018101400 2024-09-10T08:44:46.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501f4f4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5018110b90 con 0x7f5018106a60 2024-09-10T08:44:46.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501f4f4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5018110e90 con 0x7f5018100a60 2024-09-10T08:44:46.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501ca8f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5018101400 0x7f501819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:46.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501ca8f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5018101400 0x7f501819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41182/0 (socket says 172.21.15.172:41182) 2024-09-10T08:44:46.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501ca8f640 1 -- 172.21.15.172:0/1745356390 learned_addr learned my addr 172.21.15.172:0/1745356390 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:46.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501d290640 1 --2- 172.21.15.172:0/1745356390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5018100a60 0x7f501819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:46.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501da91640 1 --2- 172.21.15.172:0/1745356390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5018106a60 0x7f501819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:46.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501ca8f640 1 -- 172.21.15.172:0/1745356390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5018100a60 msgr2=0x7f501819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:46.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501ca8f640 1 --2- 172.21.15.172:0/1745356390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5018100a60 0x7f501819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:46.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501ca8f640 1 -- 172.21.15.172:0/1745356390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5018106a60 msgr2=0x7f501819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:46.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501ca8f640 1 --2- 172.21.15.172:0/1745356390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5018106a60 0x7f501819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:46.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501ca8f640 1 -- 172.21.15.172:0/1745356390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501819f4f0 con 0x7f5018101400 2024-09-10T08:44:46.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.817+0000 7f501ca8f640 1 --2- 172.21.15.172:0/1745356390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5018101400 0x7f501819aa80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f501400c890 tx=0x7f501400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:46.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.818+0000 7f500e7fc640 1 -- 172.21.15.172:0/1745356390 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014004220 con 0x7f5018101400 2024-09-10T08:44:46.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.818+0000 7f500e7fc640 1 -- 172.21.15.172:0/1745356390 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50140043c0 con 0x7f5018101400 2024-09-10T08:44:46.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.818+0000 7f501f4f4640 1 -- 172.21.15.172:0/1745356390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501819f7e0 con 0x7f5018101400 2024-09-10T08:44:46.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.818+0000 7f500e7fc640 1 -- 172.21.15.172:0/1745356390 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014004a00 con 0x7f5018101400 2024-09-10T08:44:46.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.818+0000 7f501f4f4640 1 -- 172.21.15.172:0/1745356390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50181a70c0 con 0x7f5018101400 2024-09-10T08:44:46.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.819+0000 7f501f4f4640 1 -- 172.21.15.172:0/1745356390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5018100ec0 con 0x7f5018101400 2024-09-10T08:44:46.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.820+0000 7f500e7fc640 1 -- 172.21.15.172:0/1745356390 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5014004ba0 con 0x7f5018101400 2024-09-10T08:44:46.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.820+0000 7f500e7fc640 1 --2- 172.21.15.172:0/1745356390 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f50040a2b80 0x7f50040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:46.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.820+0000 7f501d290640 1 --2- 172.21.15.172:0/1745356390 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f50040a2b80 0x7f50040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:46.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.820+0000 7f500e7fc640 1 -- 172.21.15.172:0/1745356390 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f50140cee90 con 0x7f5018101400 2024-09-10T08:44:46.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.821+0000 7f501d290640 1 --2- 172.21.15.172:0/1745356390 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f50040a2b80 0x7f50040a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5008006010 tx=0x7f5008000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:46.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.824+0000 7f500e7fc640 1 -- 172.21.15.172:0/1745356390 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5014014030 con 0x7f5018101400 2024-09-10T08:44:46.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.990+0000 7f501f4f4640 1 -- 172.21.15.172:0/1745356390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50181a7320 con 0x7f5018101400 2024-09-10T08:44:46.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.992+0000 7f500e7fc640 1 -- 172.21.15.172:0/1745356390 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24294 (secure 0 0 0) 0x7f501409b560 con 0x7f5018101400 2024-09-10T08:44:46.993 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:44:46.993 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":88,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","created":"2024-09-10T08:36:33.083529+0000","modified":"2024-09-10T08:44:20.275987+0000","last_up_change":"2024-09-10T08:43:19.615640+0000","last_in_change":"2024-09-10T08:42:56.656115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:39:31.328306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:44:07.466637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:44:09.520569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:44:15.926057+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6803","nonce":2678426952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6805","nonce":2678426952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6809","nonce":2678426952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6807","nonce":2678426952}]},"public_addr":"172.21.15.172:6803/2678426952","cluster_addr":"172.21.15.172:6805/2678426952","heartbeat_back_addr":"172.21.15.172:6809/2678426952","heartbeat_front_addr":"172.21.15.172:6807/2678426952","state":["exists","up"]},{"osd":1,"uuid":"55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6801","nonce":3279661439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6803","nonce":3279661439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6807","nonce":3279661439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6805","nonce":3279661439}]},"public_addr":"172.21.15.174:6801/3279661439","cluster_addr":"172.21.15.174:6803/3279661439","heartbeat_back_addr":"172.21.15.174:6807/3279661439","heartbeat_front_addr":"172.21.15.174:6805/3279661439","state":["exists","up"]},{"osd":2,"uuid":"f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6801","nonce":1963796290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6803","nonce":1963796290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6807","nonce":1963796290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6805","nonce":1963796290}]},"public_addr":"172.21.15.191:6801/1963796290","cluster_addr":"172.21.15.191:6803/1963796290","heartbeat_back_addr":"172.21.15.191:6807/1963796290","heartbeat_front_addr":"172.21.15.191:6805/1963796290","state":["exists","up"]},{"osd":3,"uuid":"15ecca3f-bc6b-4a06-b283-b515384beb19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6811","nonce":869546480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6813","nonce":869546480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6817","nonce":869546480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6815","nonce":869546480}]},"public_addr":"172.21.15.172:6811/869546480","cluster_addr":"172.21.15.172:6813/869546480","heartbeat_back_addr":"172.21.15.172:6817/869546480","heartbeat_front_addr":"172.21.15.172:6815/869546480","state":["exists","up"]},{"osd":4,"uuid":"8c2c1d40-72ad-4a15-ab72-54871c7e2427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6809","nonce":961007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6811","nonce":961007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6815","nonce":961007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6813","nonce":961007949}]},"public_addr":"172.21.15.174:6809/961007949","cluster_addr":"172.21.15.174:6811/961007949","heartbeat_back_addr":"172.21.15.174:6815/961007949","heartbeat_front_addr":"172.21.15.174:6813/961007949","state":["exists","up"]},{"osd":5,"uuid":"4de45a1d-ef0e-4bfe-ba15-5eeeb8806155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6809","nonce":4216938371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6811","nonce":4216938371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6815","nonce":4216938371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6813","nonce":4216938371}]},"public_addr":"172.21.15.191:6809/4216938371","cluster_addr":"172.21.15.191:6811/4216938371","heartbeat_back_addr":"172.21.15.191:6815/4216938371","heartbeat_front_addr":"172.21.15.191:6813/4216938371","state":["exists","up"]},{"osd":6,"uuid":"60a9938a-f696-4db2-932c-402997543b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6819","nonce":3210762009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6821","nonce":3210762009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6825","nonce":3210762009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6823","nonce":3210762009}]},"public_addr":"172.21.15.172:6819/3210762009","cluster_addr":"172.21.15.172:6821/3210762009","heartbeat_back_addr":"172.21.15.172:6825/3210762009","heartbeat_front_addr":"172.21.15.172:6823/3210762009","state":["exists","up"]},{"osd":7,"uuid":"7f0202f3-971e-472f-9c4d-ae3c24d39f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6817","nonce":1461923581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6819","nonce":1461923581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6823","nonce":1461923581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6821","nonce":1461923581}]},"public_addr":"172.21.15.174:6817/1461923581","cluster_addr":"172.21.15.174:6819/1461923581","heartbeat_back_addr":"172.21.15.174:6823/1461923581","heartbeat_front_addr":"172.21.15.174:6821/1461923581","state":["exists","up"]},{"osd":8,"uuid":"b4e7327f-ca9f-4833-9cd3-b8529b3d9761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6817","nonce":3675210430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6819","nonce":3675210430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6823","nonce":3675210430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6821","nonce":3675210430}]},"public_addr":"172.21.15.191:6817/3675210430","cluster_addr":"172.21.15.191:6819/3675210430","heartbeat_back_addr":"172.21.15.191:6823/3675210430","heartbeat_front_addr":"172.21.15.191:6821/3675210430","state":["exists","up"]},{"osd":9,"uuid":"9f6153f2-f9db-4c00-a659-c932628016f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6827","nonce":1181268619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6829","nonce":1181268619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6833","nonce":1181268619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6831","nonce":1181268619}]},"public_addr":"172.21.15.172:6827/1181268619","cluster_addr":"172.21.15.172:6829/1181268619","heartbeat_back_addr":"172.21.15.172:6833/1181268619","heartbeat_front_addr":"172.21.15.172:6831/1181268619","state":["exists","up"]},{"osd":10,"uuid":"040a013a-4c3e-442f-bea6-12cbe517716d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6825","nonce":1238812525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6827","nonce":1238812525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6831","nonce":1238812525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6829","nonce":1238812525}]},"public_addr":"172.21.15.174:6825/1238812525","cluster_addr":"172.21.15.174:6827/1238812525","heartbeat_back_addr":"172.21.15.174:6831/1238812525","heartbeat_front_addr":"172.21.15.174:6829/1238812525","state":["exists","up"]},{"osd":11,"uuid":"ae6535bd-1510-4bf3-bf4b-1a2a256add43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6825","nonce":2162719997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6827","nonce":2162719997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6831","nonce":2162719997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6829","nonce":2162719997}]},"public_addr":"172.21.15.191:6825/2162719997","cluster_addr":"172.21.15.191:6827/2162719997","heartbeat_back_addr":"172.21.15.191:6831/2162719997","heartbeat_front_addr":"172.21.15.191:6829/2162719997","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:38:38.812596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:04.765652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:29.103966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:56.085849+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:21.641784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:47.750376+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:11.640697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:36.638840+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:02.167446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:28.655804+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:52.749676+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:43:17.783908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:0/977245917":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/3919952878":"2024-09-11T08:36:54.887256+0000","172.21.15.172:6800/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/1525385361":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/2043423408":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/839344299":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6800/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/3251817286":"2024-09-11T08:36:54.887256+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:44:46.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.995+0000 7f501f4f4640 1 -- 172.21.15.172:0/1745356390 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f50040a2b80 msgr2=0x7f50040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:46.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.995+0000 7f501f4f4640 1 --2- 172.21.15.172:0/1745356390 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f50040a2b80 0x7f50040a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5008006010 tx=0x7f5008000bc0 comp rx=0 tx=0).stop 2024-09-10T08:44:46.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.995+0000 7f501f4f4640 1 -- 172.21.15.172:0/1745356390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5018101400 msgr2=0x7f501819aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:46.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.995+0000 7f501f4f4640 1 --2- 172.21.15.172:0/1745356390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5018101400 0x7f501819aa80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f501400c890 tx=0x7f501400cd50 comp rx=0 tx=0).stop 2024-09-10T08:44:46.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.995+0000 7f501f4f4640 1 -- 172.21.15.172:0/1745356390 shutdown_connections 2024-09-10T08:44:46.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.995+0000 7f501f4f4640 1 --2- 172.21.15.172:0/1745356390 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f50040a2b80 0x7f50040a5040 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:46.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.995+0000 7f501f4f4640 1 --2- 172.21.15.172:0/1745356390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5018106a60 0x7f501819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:46.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.995+0000 7f501f4f4640 1 --2- 172.21.15.172:0/1745356390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5018101400 0x7f501819aa80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:46.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.995+0000 7f501f4f4640 1 --2- 172.21.15.172:0/1745356390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5018100a60 0x7f501819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:46.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.995+0000 7f501f4f4640 1 -- 172.21.15.172:0/1745356390 >> 172.21.15.172:0/1745356390 conn(0x7f50180fc780 msgr2=0x7f501806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:46.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.995+0000 7f501f4f4640 1 -- 172.21.15.172:0/1745356390 shutdown_connections 2024-09-10T08:44:46.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:46.995+0000 7f501f4f4640 1 -- 172.21.15.172:0/1745356390 wait complete. 2024-09-10T08:44:47.312 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:47 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1745356390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:44:47.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:47 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1745356390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:44:47.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:47 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1745356390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:44:47.720 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-10T08:44:47.970 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:48.394 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:48 smithi172 ceph-mon[30854]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:44:48.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:48 smithi174 ceph-mon[34068]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:44:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.477+0000 7fc36b189640 1 -- 172.21.15.172:0/2102171419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc364106f20 msgr2=0x7fc36410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.477+0000 7fc36b189640 1 --2- 172.21.15.172:0/2102171419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc364106f20 0x7fc36410eaa0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009ca0 tx=0x7fc35c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:44:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.477+0000 7fc36b189640 1 -- 172.21.15.172:0/2102171419 shutdown_connections 2024-09-10T08:44:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.477+0000 7fc36b189640 1 --2- 172.21.15.172:0/2102171419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc364106f20 0x7fc36410eaa0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.477+0000 7fc36b189640 1 --2- 172.21.15.172:0/2102171419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3641027a0 0x7fc3641067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.477+0000 7fc36b189640 1 --2- 172.21.15.172:0/2102171419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc364101df0 0x7fc3641021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.477+0000 7fc36b189640 1 -- 172.21.15.172:0/2102171419 >> 172.21.15.172:0/2102171419 conn(0x7fc3640fd8b0 msgr2=0x7fc3640ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:48.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.477+0000 7fc36b189640 1 -- 172.21.15.172:0/2102171419 shutdown_connections 2024-09-10T08:44:48.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.477+0000 7fc36b189640 1 -- 172.21.15.172:0/2102171419 wait complete. 2024-09-10T08:44:48.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.478+0000 7fc36b189640 1 Processor -- start 2024-09-10T08:44:48.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.478+0000 7fc36b189640 1 -- start start 2024-09-10T08:44:48.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.478+0000 7fc36b189640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc364101df0 0x7fc3641983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:48.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.478+0000 7fc36b189640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3641027a0 0x7fc364198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:48.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.478+0000 7fc36b189640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc364106f20 0x7fc36419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:48.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.478+0000 7fc36b189640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc364111e30 con 0x7fc3641027a0 2024-09-10T08:44:48.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.478+0000 7fc36b189640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc364111cb0 con 0x7fc364101df0 2024-09-10T08:44:48.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.478+0000 7fc36b189640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc364111fb0 con 0x7fc364106f20 2024-09-10T08:44:48.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.478+0000 7fc363fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3641027a0 0x7fc364198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:48.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.478+0000 7fc363fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3641027a0 0x7fc364198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41198/0 (socket says 172.21.15.172:41198) 2024-09-10T08:44:48.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.478+0000 7fc363fff640 1 -- 172.21.15.172:0/1828527677 learned_addr learned my addr 172.21.15.172:0/1828527677 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:48.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.478+0000 7fc368f25640 1 --2- 172.21.15.172:0/1828527677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc364101df0 0x7fc3641983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:48.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.478+0000 7fc363fff640 1 -- 172.21.15.172:0/1828527677 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc364106f20 msgr2=0x7fc36419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:48.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.478+0000 7fc363fff640 1 --2- 172.21.15.172:0/1828527677 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc364106f20 0x7fc36419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:48.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.478+0000 7fc363fff640 1 -- 172.21.15.172:0/1828527677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc364101df0 msgr2=0x7fc3641983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:48.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.478+0000 7fc363fff640 1 --2- 172.21.15.172:0/1828527677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc364101df0 0x7fc3641983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:48.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.478+0000 7fc363fff640 1 -- 172.21.15.172:0/1828527677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36419d380 con 0x7fc3641027a0 2024-09-10T08:44:48.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.479+0000 7fc363fff640 1 --2- 172.21.15.172:0/1828527677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3641027a0 0x7fc364198910 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc35000d770 tx=0x7fc35000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:48.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.479+0000 7fc361ffb640 1 -- 172.21.15.172:0/1828527677 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc350017070 con 0x7fc3641027a0 2024-09-10T08:44:48.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.479+0000 7fc361ffb640 1 -- 172.21.15.172:0/1828527677 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc3500054d0 con 0x7fc3641027a0 2024-09-10T08:44:48.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.479+0000 7fc361ffb640 1 -- 172.21.15.172:0/1828527677 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc350004cf0 con 0x7fc3641027a0 2024-09-10T08:44:48.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.479+0000 7fc36b189640 1 -- 172.21.15.172:0/1828527677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36419d670 con 0x7fc3641027a0 2024-09-10T08:44:48.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.479+0000 7fc36b189640 1 -- 172.21.15.172:0/1828527677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3641a4f50 con 0x7fc3641027a0 2024-09-10T08:44:48.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.480+0000 7fc36b189640 1 -- 172.21.15.172:0/1828527677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3641021d0 con 0x7fc3641027a0 2024-09-10T08:44:48.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.481+0000 7fc361ffb640 1 -- 172.21.15.172:0/1828527677 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc350004e90 con 0x7fc3641027a0 2024-09-10T08:44:48.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.481+0000 7fc361ffb640 1 --2- 172.21.15.172:0/1828527677 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc3540a2b80 0x7fc3540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:48.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.482+0000 7fc368f25640 1 --2- 172.21.15.172:0/1828527677 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc3540a2b80 0x7fc3540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:48.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.482+0000 7fc361ffb640 1 -- 172.21.15.172:0/1828527677 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fc3500ced00 con 0x7fc3641027a0 2024-09-10T08:44:48.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.482+0000 7fc368f25640 1 --2- 172.21.15.172:0/1828527677 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc3540a2b80 0x7fc3540a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc358005fd0 tx=0x7fc358005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:48.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.486+0000 7fc361ffb640 1 -- 172.21.15.172:0/1828527677 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc35009ba20 con 0x7fc3641027a0 2024-09-10T08:44:48.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:48 smithi191 ceph-mon[35314]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T08:44:48.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:48.689+0000 7fc36b189640 1 -- 172.21.15.172:0/1828527677 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc36410a100 con 0x7fc3641027a0 2024-09-10T08:44:49.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:49.237+0000 7fc361ffb640 1 -- 172.21.15.172:0/1828527677 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fc35009b320 con 0x7fc3641027a0 2024-09-10T08:44:49.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:49.241+0000 7fc36b189640 1 -- 172.21.15.172:0/1828527677 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc3540a2b80 msgr2=0x7fc3540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:49.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:49.241+0000 7fc36b189640 1 --2- 172.21.15.172:0/1828527677 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc3540a2b80 0x7fc3540a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc358005fd0 tx=0x7fc358005850 comp rx=0 tx=0).stop 2024-09-10T08:44:49.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:49.241+0000 7fc36b189640 1 -- 172.21.15.172:0/1828527677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3641027a0 msgr2=0x7fc364198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:49.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:49.241+0000 7fc36b189640 1 --2- 172.21.15.172:0/1828527677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3641027a0 0x7fc364198910 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc35000d770 tx=0x7fc35000dc30 comp rx=0 tx=0).stop 2024-09-10T08:44:49.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:49.241+0000 7fc36b189640 1 -- 172.21.15.172:0/1828527677 shutdown_connections 2024-09-10T08:44:49.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:49.241+0000 7fc36b189640 1 --2- 172.21.15.172:0/1828527677 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc3540a2b80 0x7fc3540a5040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:49.241+0000 7fc36b189640 1 --2- 172.21.15.172:0/1828527677 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc364106f20 0x7fc36419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:49.241+0000 7fc36b189640 1 --2- 172.21.15.172:0/1828527677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3641027a0 0x7fc364198910 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:49.241+0000 7fc36b189640 1 --2- 172.21.15.172:0/1828527677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc364101df0 0x7fc3641983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:49.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:49.241+0000 7fc36b189640 1 -- 172.21.15.172:0/1828527677 >> 172.21.15.172:0/1828527677 conn(0x7fc3640fd8b0 msgr2=0x7fc3640febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:49.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:49.241+0000 7fc36b189640 1 -- 172.21.15.172:0/1828527677 shutdown_connections 2024-09-10T08:44:49.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:49.241+0000 7fc36b189640 1 -- 172.21.15.172:0/1828527677 wait complete. 2024-09-10T08:44:49.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:49 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1828527677' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T08:44:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:49 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1828527677' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T08:44:49.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:49 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1828527677' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T08:44:50.052 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-10T08:44:50.053 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-10T08:44:50.310 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:50.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:50 smithi172 ceph-mon[30854]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T08:44:50.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:50 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1828527677' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T08:44:50.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:50 smithi172 ceph-mon[30854]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:44:50.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:50 smithi174 ceph-mon[34068]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T08:44:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:50 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1828527677' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T08:44:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:50 smithi174 ceph-mon[34068]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:44:50.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:50 smithi191 ceph-mon[35314]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T08:44:50.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:50 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1828527677' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T08:44:50.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:50 smithi191 ceph-mon[35314]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:44:50.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.788+0000 7f670b200640 1 -- 172.21.15.172:0/2871194424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6704106a80 msgr2=0x7f6704106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:50.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.788+0000 7f670b200640 1 --2- 172.21.15.172:0/2871194424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6704106a80 0x7f6704106e60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009930 tx=0x7f66f802a420 comp rx=0 tx=0).stop 2024-09-10T08:44:50.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.788+0000 7f670b200640 1 -- 172.21.15.172:0/2871194424 shutdown_connections 2024-09-10T08:44:50.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.788+0000 7f670b200640 1 --2- 172.21.15.172:0/2871194424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6704101420 0x7f670410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:50.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.788+0000 7f670b200640 1 --2- 172.21.15.172:0/2871194424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6704100a80 0x7f6704100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.788+0000 7f670b200640 1 --2- 172.21.15.172:0/2871194424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6704106a80 0x7f6704106e60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.788+0000 7f670b200640 1 -- 172.21.15.172:0/2871194424 >> 172.21.15.172:0/2871194424 conn(0x7f67040fc7a0 msgr2=0x7f67040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.788+0000 7f670b200640 1 -- 172.21.15.172:0/2871194424 shutdown_connections 2024-09-10T08:44:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.788+0000 7f670b200640 1 -- 172.21.15.172:0/2871194424 wait complete. 2024-09-10T08:44:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.789+0000 7f670b200640 1 Processor -- start 2024-09-10T08:44:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.789+0000 7f670b200640 1 -- start start 2024-09-10T08:44:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.789+0000 7f670b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6704100a80 0x7f6704197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.789+0000 7f670b200640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6704101420 0x7f6704197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.789+0000 7f670b200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6704106a80 0x7f670419bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.789+0000 7f670b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6704110dd0 con 0x7f6704100a80 2024-09-10T08:44:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.789+0000 7f670b200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6704110c50 con 0x7f6704106a80 2024-09-10T08:44:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.790+0000 7f670b200640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6704110f50 con 0x7f6704101420 2024-09-10T08:44:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.790+0000 7f6708f9c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6704100a80 0x7f6704197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.790+0000 7f6708f9c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6704100a80 0x7f6704197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41206/0 (socket says 172.21.15.172:41206) 2024-09-10T08:44:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.790+0000 7f6708f9c640 1 -- 172.21.15.172:0/2834446914 learned_addr learned my addr 172.21.15.172:0/2834446914 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.790+0000 7f670979d640 1 --2- 172.21.15.172:0/2834446914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6704106a80 0x7f670419bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.790+0000 7f6703fff640 1 --2- 172.21.15.172:0/2834446914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6704101420 0x7f6704197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.790+0000 7f6708f9c640 1 -- 172.21.15.172:0/2834446914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6704101420 msgr2=0x7f6704197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.790+0000 7f6708f9c640 1 --2- 172.21.15.172:0/2834446914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6704101420 0x7f6704197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.790+0000 7f6708f9c640 1 -- 172.21.15.172:0/2834446914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6704106a80 msgr2=0x7f670419bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.790+0000 7f6708f9c640 1 --2- 172.21.15.172:0/2834446914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6704106a80 0x7f670419bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.790+0000 7f6708f9c640 1 -- 172.21.15.172:0/2834446914 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670419c3e0 con 0x7f6704100a80 2024-09-10T08:44:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.790+0000 7f6703fff640 1 --2- 172.21.15.172:0/2834446914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6704101420 0x7f6704197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.790+0000 7f6708f9c640 1 --2- 172.21.15.172:0/2834446914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6704100a80 0x7f6704197430 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f66f8005940 tx=0x7f66f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.790+0000 7f6701ffb640 1 -- 172.21.15.172:0/2834446914 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f803f030 con 0x7f6704100a80 2024-09-10T08:44:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.790+0000 7f6701ffb640 1 -- 172.21.15.172:0/2834446914 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66f8003cb0 con 0x7f6704100a80 2024-09-10T08:44:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.790+0000 7f670b200640 1 -- 172.21.15.172:0/2834446914 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670419c670 con 0x7f6704100a80 2024-09-10T08:44:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.790+0000 7f6701ffb640 1 -- 172.21.15.172:0/2834446914 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8035070 con 0x7f6704100a80 2024-09-10T08:44:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.791+0000 7f670b200640 1 -- 172.21.15.172:0/2834446914 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67041a3fb0 con 0x7f6704100a80 2024-09-10T08:44:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.792+0000 7f670b200640 1 -- 172.21.15.172:0/2834446914 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67041021c0 con 0x7f6704100a80 2024-09-10T08:44:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.793+0000 7f6701ffb640 1 -- 172.21.15.172:0/2834446914 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f66f8042030 con 0x7f6704100a80 2024-09-10T08:44:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.793+0000 7f6701ffb640 1 --2- 172.21.15.172:0/2834446914 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66f40a2b80 0x7f66f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.793+0000 7f6703fff640 1 --2- 172.21.15.172:0/2834446914 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66f40a2b80 0x7f66f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.793+0000 7f6701ffb640 1 -- 172.21.15.172:0/2834446914 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f66f80ef7d0 con 0x7f6704100a80 2024-09-10T08:44:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.794+0000 7f6703fff640 1 --2- 172.21.15.172:0/2834446914 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66f40a2b80 0x7f66f40a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f67040fef10 tx=0x7f66f0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:50.797+0000 7f6701ffb640 1 -- 172.21.15.172:0/2834446914 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f66f80bc4f0 con 0x7f6704100a80 2024-09-10T08:44:51.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:51.000+0000 7f670b200640 1 -- 172.21.15.172:0/2834446914 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f6704108ff0 con 0x7f6704100a80 2024-09-10T08:44:51.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:51.244+0000 7f6701ffb640 1 -- 172.21.15.172:0/2834446914 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f66f80bbdf0 con 0x7f6704100a80 2024-09-10T08:44:51.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:51.246+0000 7f670b200640 1 -- 172.21.15.172:0/2834446914 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66f40a2b80 msgr2=0x7f66f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:51.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:51.246+0000 7f670b200640 1 --2- 172.21.15.172:0/2834446914 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66f40a2b80 0x7f66f40a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f67040fef10 tx=0x7f66f0008040 comp rx=0 tx=0).stop 2024-09-10T08:44:51.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:51.246+0000 7f670b200640 1 -- 172.21.15.172:0/2834446914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6704100a80 msgr2=0x7f6704197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:51.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:51.246+0000 7f670b200640 1 --2- 172.21.15.172:0/2834446914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6704100a80 0x7f6704197430 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f66f8005940 tx=0x7f66f80042f0 comp rx=0 tx=0).stop 2024-09-10T08:44:51.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:51.246+0000 7f670b200640 1 -- 172.21.15.172:0/2834446914 shutdown_connections 2024-09-10T08:44:51.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:51.246+0000 7f670b200640 1 --2- 172.21.15.172:0/2834446914 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66f40a2b80 0x7f66f40a5040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:51.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:51.246+0000 7f670b200640 1 --2- 172.21.15.172:0/2834446914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6704106a80 0x7f670419bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:51.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:51.246+0000 7f670b200640 1 --2- 172.21.15.172:0/2834446914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6704101420 0x7f6704197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:51.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:51.246+0000 7f670b200640 1 --2- 172.21.15.172:0/2834446914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6704100a80 0x7f6704197430 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:51.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:51.246+0000 7f670b200640 1 -- 172.21.15.172:0/2834446914 >> 172.21.15.172:0/2834446914 conn(0x7f67040fc7a0 msgr2=0x7f67040fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:51.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:51.246+0000 7f670b200640 1 -- 172.21.15.172:0/2834446914 shutdown_connections 2024-09-10T08:44:51.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:51.247+0000 7f670b200640 1 -- 172.21.15.172:0/2834446914 wait complete. 2024-09-10T08:44:51.258 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:51 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2834446914' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T08:44:51.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:51 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2834446914' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T08:44:51.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:51 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2834446914' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T08:44:51.937 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-10T08:44:52.185 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:52 smithi191 ceph-mon[35314]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T08:44:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:52 smithi191 ceph-mon[35314]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T08:44:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:52 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2834446914' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T08:44:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:52 smithi191 ceph-mon[35314]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T08:44:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:52 smithi191 ceph-mon[35314]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:44:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:52 smithi191 ceph-mon[35314]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-10T08:44:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:52 smithi191 ceph-mon[35314]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-10T08:44:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:52 smithi172 ceph-mon[30854]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T08:44:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:52 smithi172 ceph-mon[30854]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T08:44:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:52 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2834446914' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T08:44:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:52 smithi172 ceph-mon[30854]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T08:44:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:52 smithi172 ceph-mon[30854]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:44:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:52 smithi172 ceph-mon[30854]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-10T08:44:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:52 smithi172 ceph-mon[30854]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-10T08:44:52.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.671+0000 7f12647e7640 1 -- 172.21.15.172:0/1584879428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f125c106a80 msgr2=0x7f125c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:52.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.671+0000 7f12647e7640 1 --2- 172.21.15.172:0/1584879428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f125c106a80 0x7f125c106e60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1258009930 tx=0x7f125802a420 comp rx=0 tx=0).stop 2024-09-10T08:44:52.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.671+0000 7f12647e7640 1 -- 172.21.15.172:0/1584879428 shutdown_connections 2024-09-10T08:44:52.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.671+0000 7f12647e7640 1 --2- 172.21.15.172:0/1584879428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f125c101420 0x7f125c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:52.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.671+0000 7f12647e7640 1 --2- 172.21.15.172:0/1584879428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f125c100a80 0x7f125c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:52.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.671+0000 7f12647e7640 1 --2- 172.21.15.172:0/1584879428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f125c106a80 0x7f125c106e60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:52.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.671+0000 7f12647e7640 1 -- 172.21.15.172:0/1584879428 >> 172.21.15.172:0/1584879428 conn(0x7f125c0fc7a0 msgr2=0x7f125c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:52.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.671+0000 7f12647e7640 1 -- 172.21.15.172:0/1584879428 shutdown_connections 2024-09-10T08:44:52.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.671+0000 7f12647e7640 1 -- 172.21.15.172:0/1584879428 wait complete. 2024-09-10T08:44:52.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.671+0000 7f12647e7640 1 Processor -- start 2024-09-10T08:44:52.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.671+0000 7f12647e7640 1 -- start start 2024-09-10T08:44:52.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.671+0000 7f12647e7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f125c100a80 0x7f125c102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:52.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f12647e7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f125c101420 0x7f125c102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:52.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f12647e7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f125c106a80 0x7f125c1030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:52.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f12647e7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f125c110d30 con 0x7f125c100a80 2024-09-10T08:44:52.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f12647e7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f125c110bb0 con 0x7f125c106a80 2024-09-10T08:44:52.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f12647e7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f125c110eb0 con 0x7f125c101420 2024-09-10T08:44:52.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f1262583640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f125c100a80 0x7f125c102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:52.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f1262583640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f125c100a80 0x7f125c102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41222/0 (socket says 172.21.15.172:41222) 2024-09-10T08:44:52.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f1262583640 1 -- 172.21.15.172:0/693657237 learned_addr learned my addr 172.21.15.172:0/693657237 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:52.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f1261d82640 1 --2- 172.21.15.172:0/693657237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f125c101420 0x7f125c102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:52.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f1262d84640 1 --2- 172.21.15.172:0/693657237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f125c106a80 0x7f125c1030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:52.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f1262583640 1 -- 172.21.15.172:0/693657237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f125c101420 msgr2=0x7f125c102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:52.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f1262583640 1 --2- 172.21.15.172:0/693657237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f125c101420 0x7f125c102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:52.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f1262583640 1 -- 172.21.15.172:0/693657237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f125c106a80 msgr2=0x7f125c1030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:52.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f1262583640 1 --2- 172.21.15.172:0/693657237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f125c106a80 0x7f125c1030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:52.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f1262583640 1 -- 172.21.15.172:0/693657237 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125c1a6880 con 0x7f125c100a80 2024-09-10T08:44:52.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f1262583640 1 --2- 172.21.15.172:0/693657237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f125c100a80 0x7f125c102650 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1258004770 tx=0x7f12580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:52.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f12537fe640 1 -- 172.21.15.172:0/693657237 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125803f070 con 0x7f125c100a80 2024-09-10T08:44:52.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f12537fe640 1 -- 172.21.15.172:0/693657237 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f125803b460 con 0x7f125c100a80 2024-09-10T08:44:52.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f12647e7640 1 -- 172.21.15.172:0/693657237 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125c1a6b10 con 0x7f125c100a80 2024-09-10T08:44:52.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f12537fe640 1 -- 172.21.15.172:0/693657237 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258035070 con 0x7f125c100a80 2024-09-10T08:44:52.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.672+0000 7f12647e7640 1 -- 172.21.15.172:0/693657237 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f125c1a6ff0 con 0x7f125c100a80 2024-09-10T08:44:52.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.673+0000 7f12647e7640 1 -- 172.21.15.172:0/693657237 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125c100ee0 con 0x7f125c100a80 2024-09-10T08:44:52.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.675+0000 7f12537fe640 1 -- 172.21.15.172:0/693657237 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f125803b610 con 0x7f125c100a80 2024-09-10T08:44:52.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.675+0000 7f12537fe640 1 --2- 172.21.15.172:0/693657237 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12480a2b80 0x7f12480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:52.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.675+0000 7f1261d82640 1 --2- 172.21.15.172:0/693657237 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12480a2b80 0x7f12480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:52.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.675+0000 7f12537fe640 1 -- 172.21.15.172:0/693657237 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f12580ef6f0 con 0x7f125c100a80 2024-09-10T08:44:52.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.676+0000 7f1261d82640 1 --2- 172.21.15.172:0/693657237 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12480a2b80 0x7f12480a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f124c005fd0 tx=0x7f124c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:52.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.679+0000 7f12537fe640 1 -- 172.21.15.172:0/693657237 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f12580bc410 con 0x7f125c100a80 2024-09-10T08:44:52.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:52 smithi174 ceph-mon[34068]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T08:44:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:52 smithi174 ceph-mon[34068]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T08:44:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:52 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2834446914' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T08:44:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:52 smithi174 ceph-mon[34068]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T08:44:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:52 smithi174 ceph-mon[34068]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T08:44:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:52 smithi174 ceph-mon[34068]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-10T08:44:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:52 smithi174 ceph-mon[34068]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-10T08:44:52.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:52.880+0000 7f12647e7640 1 -- 172.21.15.172:0/693657237 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f125c1a7290 con 0x7f125c100a80 2024-09-10T08:44:53.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:53.205+0000 7f12537fe640 1 -- 172.21.15.172:0/693657237 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f12580bbd10 con 0x7f125c100a80 2024-09-10T08:44:53.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:53.207+0000 7f12647e7640 1 -- 172.21.15.172:0/693657237 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12480a2b80 msgr2=0x7f12480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:53.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:53.207+0000 7f12647e7640 1 --2- 172.21.15.172:0/693657237 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12480a2b80 0x7f12480a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f124c005fd0 tx=0x7f124c005850 comp rx=0 tx=0).stop 2024-09-10T08:44:53.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:53.207+0000 7f12647e7640 1 -- 172.21.15.172:0/693657237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f125c100a80 msgr2=0x7f125c102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:53.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:53.207+0000 7f12647e7640 1 --2- 172.21.15.172:0/693657237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f125c100a80 0x7f125c102650 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1258004770 tx=0x7f12580042f0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:53.207+0000 7f12647e7640 1 -- 172.21.15.172:0/693657237 shutdown_connections 2024-09-10T08:44:53.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:53.207+0000 7f12647e7640 1 --2- 172.21.15.172:0/693657237 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12480a2b80 0x7f12480a5040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:53.207+0000 7f12647e7640 1 --2- 172.21.15.172:0/693657237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f125c106a80 0x7f125c1030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:53.207+0000 7f12647e7640 1 --2- 172.21.15.172:0/693657237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f125c101420 0x7f125c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:53.207+0000 7f12647e7640 1 --2- 172.21.15.172:0/693657237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f125c100a80 0x7f125c102650 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:53.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:53.207+0000 7f12647e7640 1 -- 172.21.15.172:0/693657237 >> 172.21.15.172:0/693657237 conn(0x7f125c0fc7a0 msgr2=0x7f125c06a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:53.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:53.207+0000 7f12647e7640 1 -- 172.21.15.172:0/693657237 shutdown_connections 2024-09-10T08:44:53.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:53.207+0000 7f12647e7640 1 -- 172.21.15.172:0/693657237 wait complete. 2024-09-10T08:44:53.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:53 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] up:active 2024-09-10T08:44:53.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:53 smithi191 ceph-mon[35314]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T08:44:53.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:53 smithi191 ceph-mon[35314]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-10T08:44:53.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:53 smithi191 ceph-mon[35314]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-10T08:44:53.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:53 smithi191 ceph-mon[35314]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-10T08:44:53.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:53 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/693657237' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T08:44:53.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:53 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/693657237' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T08:44:53.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:53 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] up:active 2024-09-10T08:44:53.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:53 smithi191 ceph-mon[35314]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T08:44:53.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:53 smithi191 ceph-mon[35314]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:44:53.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:53 smithi191 ceph-mon[35314]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-09-10T08:44:53.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:53 smithi191 ceph-mon[35314]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-09-10T08:44:53.552 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:53 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] up:active 2024-09-10T08:44:53.553 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:53 smithi172 ceph-mon[30854]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T08:44:53.553 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:53 smithi172 ceph-mon[30854]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-10T08:44:53.553 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:53 smithi172 ceph-mon[30854]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-10T08:44:53.553 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:53 smithi172 ceph-mon[30854]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-10T08:44:53.553 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:53 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/693657237' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T08:44:53.553 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:53 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/693657237' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T08:44:53.553 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:53 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] up:active 2024-09-10T08:44:53.553 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:53 smithi172 ceph-mon[30854]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T08:44:53.553 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:53 smithi172 ceph-mon[30854]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:44:53.553 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:53 smithi172 ceph-mon[30854]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-09-10T08:44:53.554 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:53 smithi172 ceph-mon[30854]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-09-10T08:44:53.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:53 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] up:active 2024-09-10T08:44:53.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:53 smithi174 ceph-mon[34068]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T08:44:53.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:53 smithi174 ceph-mon[34068]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-10T08:44:53.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:53 smithi174 ceph-mon[34068]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-10T08:44:53.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:53 smithi174 ceph-mon[34068]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-10T08:44:53.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:53 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/693657237' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T08:44:53.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:53 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/693657237' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T08:44:53.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:53 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] up:active 2024-09-10T08:44:53.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:53 smithi174 ceph-mon[34068]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T08:44:53.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:53 smithi174 ceph-mon[34068]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-10T08:44:53.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:53 smithi174 ceph-mon[34068]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-09-10T08:44:53.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:53 smithi174 ceph-mon[34068]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-09-10T08:44:54.037 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-10T08:44:54.290 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:54.513 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:54 smithi172 ceph-mon[30854]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T08:44:54.513 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:54 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] up:active 2024-09-10T08:44:54.513 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:54 smithi172 ceph-mon[30854]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T08:44:54.513 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:54 smithi172 ceph-mon[30854]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T08:44:54.514 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:54 smithi172 ceph-mon[30854]: Cluster is now healthy 2024-09-10T08:44:54.514 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:54 smithi172 ceph-mon[30854]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-09-10T08:44:54.514 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:54 smithi172 ceph-mon[30854]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-10T08:44:54.514 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:54 smithi172 ceph-mon[30854]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-09-10T08:44:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:54 smithi191 ceph-mon[35314]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T08:44:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:54 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] up:active 2024-09-10T08:44:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:54 smithi191 ceph-mon[35314]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T08:44:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:54 smithi191 ceph-mon[35314]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T08:44:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:54 smithi191 ceph-mon[35314]: Cluster is now healthy 2024-09-10T08:44:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:54 smithi191 ceph-mon[35314]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-09-10T08:44:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:54 smithi191 ceph-mon[35314]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-10T08:44:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:54 smithi191 ceph-mon[35314]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-09-10T08:44:54.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.714+0000 7fc2495ad640 1 -- 172.21.15.172:0/3368987188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2441011e0 msgr2=0x7fc244105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:54.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.714+0000 7fc2495ad640 1 --2- 172.21.15.172:0/3368987188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2441011e0 0x7fc244105980 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009900 tx=0x7fc22c02a410 comp rx=0 tx=0).stop 2024-09-10T08:44:54.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.714+0000 7fc2495ad640 1 -- 172.21.15.172:0/3368987188 shutdown_connections 2024-09-10T08:44:54.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.714+0000 7fc2495ad640 1 --2- 172.21.15.172:0/3368987188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc244105ec0 0x7fc244106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:54.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.714+0000 7fc2495ad640 1 --2- 172.21.15.172:0/3368987188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2441011e0 0x7fc244105980 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:54.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.714+0000 7fc2495ad640 1 --2- 172.21.15.172:0/3368987188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2441008c0 0x7fc244100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:54.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.714+0000 7fc2495ad640 1 -- 172.21.15.172:0/3368987188 >> 172.21.15.172:0/3368987188 conn(0x7fc2440fc7d0 msgr2=0x7fc2440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:54.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.714+0000 7fc2495ad640 1 -- 172.21.15.172:0/3368987188 shutdown_connections 2024-09-10T08:44:54.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.714+0000 7fc2495ad640 1 -- 172.21.15.172:0/3368987188 wait complete. 2024-09-10T08:44:54.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.715+0000 7fc2495ad640 1 Processor -- start 2024-09-10T08:44:54.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.715+0000 7fc2495ad640 1 -- start start 2024-09-10T08:44:54.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.715+0000 7fc2495ad640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2441008c0 0x7fc2441a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:54.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.715+0000 7fc2495ad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2441011e0 0x7fc2441a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:54.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.715+0000 7fc2495ad640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc244105ec0 0x7fc24419ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:54.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.715+0000 7fc2495ad640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441185c0 con 0x7fc2441011e0 2024-09-10T08:44:54.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.715+0000 7fc2495ad640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc244118440 con 0x7fc244105ec0 2024-09-10T08:44:54.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.715+0000 7fc2495ad640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc244118740 con 0x7fc2441008c0 2024-09-10T08:44:54.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.715+0000 7fc23bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2441011e0 0x7fc2441a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:54.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.716+0000 7fc23bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2441011e0 0x7fc2441a4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41240/0 (socket says 172.21.15.172:41240) 2024-09-10T08:44:54.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.716+0000 7fc23bfff640 1 -- 172.21.15.172:0/2991150546 learned_addr learned my addr 172.21.15.172:0/2991150546 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:54.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.716+0000 7fc242ffd640 1 --2- 172.21.15.172:0/2991150546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2441008c0 0x7fc2441a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:54.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.716+0000 7fc2437fe640 1 --2- 172.21.15.172:0/2991150546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc244105ec0 0x7fc24419ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:54.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.716+0000 7fc23bfff640 1 -- 172.21.15.172:0/2991150546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2441008c0 msgr2=0x7fc2441a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:54.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.716+0000 7fc23bfff640 1 --2- 172.21.15.172:0/2991150546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2441008c0 0x7fc2441a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:54.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.716+0000 7fc23bfff640 1 -- 172.21.15.172:0/2991150546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc244105ec0 msgr2=0x7fc24419ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:54.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.716+0000 7fc23bfff640 1 --2- 172.21.15.172:0/2991150546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc244105ec0 0x7fc24419ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:54.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.716+0000 7fc23bfff640 1 -- 172.21.15.172:0/2991150546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24419f140 con 0x7fc2441011e0 2024-09-10T08:44:54.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.716+0000 7fc2437fe640 1 --2- 172.21.15.172:0/2991150546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc244105ec0 0x7fc24419ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:44:54.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.716+0000 7fc23bfff640 1 --2- 172.21.15.172:0/2991150546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2441011e0 0x7fc2441a4d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc22c006010 tx=0x7fc22c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:54.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.716+0000 7fc240ff9640 1 -- 172.21.15.172:0/2991150546 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c03f070 con 0x7fc2441011e0 2024-09-10T08:44:54.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.716+0000 7fc240ff9640 1 -- 172.21.15.172:0/2991150546 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc22c003aa0 con 0x7fc2441011e0 2024-09-10T08:44:54.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.716+0000 7fc2495ad640 1 -- 172.21.15.172:0/2991150546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc24419f3d0 con 0x7fc2441011e0 2024-09-10T08:44:54.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.716+0000 7fc240ff9640 1 -- 172.21.15.172:0/2991150546 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c035030 con 0x7fc2441011e0 2024-09-10T08:44:54.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.716+0000 7fc2495ad640 1 -- 172.21.15.172:0/2991150546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2441ab720 con 0x7fc2441011e0 2024-09-10T08:44:54.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.717+0000 7fc2495ad640 1 -- 172.21.15.172:0/2991150546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc24406a850 con 0x7fc2441011e0 2024-09-10T08:44:54.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.718+0000 7fc240ff9640 1 -- 172.21.15.172:0/2991150546 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc22c042070 con 0x7fc2441011e0 2024-09-10T08:44:54.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.719+0000 7fc240ff9640 1 --2- 172.21.15.172:0/2991150546 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc2300a2c00 0x7fc2300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:54.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.719+0000 7fc242ffd640 1 --2- 172.21.15.172:0/2991150546 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc2300a2c00 0x7fc2300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:54.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.719+0000 7fc240ff9640 1 -- 172.21.15.172:0/2991150546 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fc22c0efbb0 con 0x7fc2441011e0 2024-09-10T08:44:54.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.720+0000 7fc242ffd640 1 --2- 172.21.15.172:0/2991150546 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc2300a2c00 0x7fc2300a50c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc24419fb50 tx=0x7fc234009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:54.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.723+0000 7fc240ff9640 1 -- 172.21.15.172:0/2991150546 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc22c0bc8d0 con 0x7fc2441011e0 2024-09-10T08:44:54.745 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:54 smithi174 ceph-mon[34068]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T08:44:54.746 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:54 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] up:active 2024-09-10T08:44:54.746 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:54 smithi174 ceph-mon[34068]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T08:44:54.746 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:54 smithi174 ceph-mon[34068]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T08:44:54.746 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:54 smithi174 ceph-mon[34068]: Cluster is now healthy 2024-09-10T08:44:54.746 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:54 smithi174 ceph-mon[34068]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-09-10T08:44:54.746 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:54 smithi174 ceph-mon[34068]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-10T08:44:54.746 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:54 smithi174 ceph-mon[34068]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-09-10T08:44:54.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:54.926+0000 7fc2495ad640 1 -- 172.21.15.172:0/2991150546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fc2441ab8f0 con 0x7fc2441011e0 2024-09-10T08:44:55.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:55.222+0000 7fc240ff9640 1 -- 172.21.15.172:0/2991150546 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v27) ==== 115+0+0 (secure 0 0 0) 0x7fc22c0bc1d0 con 0x7fc2441011e0 2024-09-10T08:44:55.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:55.225+0000 7fc2495ad640 1 -- 172.21.15.172:0/2991150546 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc2300a2c00 msgr2=0x7fc2300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:55.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:55.225+0000 7fc2495ad640 1 --2- 172.21.15.172:0/2991150546 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc2300a2c00 0x7fc2300a50c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc24419fb50 tx=0x7fc234009110 comp rx=0 tx=0).stop 2024-09-10T08:44:55.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:55.225+0000 7fc2495ad640 1 -- 172.21.15.172:0/2991150546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2441011e0 msgr2=0x7fc2441a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:55.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:55.225+0000 7fc2495ad640 1 --2- 172.21.15.172:0/2991150546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2441011e0 0x7fc2441a4d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc22c006010 tx=0x7fc22c0042f0 comp rx=0 tx=0).stop 2024-09-10T08:44:55.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:55.225+0000 7fc2495ad640 1 -- 172.21.15.172:0/2991150546 shutdown_connections 2024-09-10T08:44:55.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:55.225+0000 7fc2495ad640 1 --2- 172.21.15.172:0/2991150546 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc2300a2c00 0x7fc2300a50c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:55.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:55.225+0000 7fc2495ad640 1 --2- 172.21.15.172:0/2991150546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc244105ec0 0x7fc24419ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:55.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:55.225+0000 7fc2495ad640 1 --2- 172.21.15.172:0/2991150546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2441011e0 0x7fc2441a4d90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:55.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:55.225+0000 7fc2495ad640 1 --2- 172.21.15.172:0/2991150546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2441008c0 0x7fc2441a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:55.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:55.225+0000 7fc2495ad640 1 -- 172.21.15.172:0/2991150546 >> 172.21.15.172:0/2991150546 conn(0x7fc2440fc7d0 msgr2=0x7fc2440fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:55.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:55.225+0000 7fc2495ad640 1 -- 172.21.15.172:0/2991150546 shutdown_connections 2024-09-10T08:44:55.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:55.225+0000 7fc2495ad640 1 -- 172.21.15.172:0/2991150546 wait complete. 2024-09-10T08:44:55.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:55 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2991150546' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T08:44:55.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:55 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2991150546' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T08:44:55.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:55 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] up:active 2024-09-10T08:44:55.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:55 smithi191 ceph-mon[35314]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-10T08:44:55.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:55 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:44:55.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:55 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2991150546' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T08:44:55.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:55 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2991150546' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T08:44:55.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:55 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] up:active 2024-09-10T08:44:55.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:55 smithi172 ceph-mon[30854]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-10T08:44:55.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:55 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:44:55.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:55 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2991150546' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T08:44:55.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:55 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2991150546' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T08:44:55.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:55 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] up:active 2024-09-10T08:44:55.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:55 smithi174 ceph-mon[34068]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-10T08:44:55.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:55 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:44:56.060 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-10T08:44:56.060 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-10T08:44:56.309 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:56.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:56 smithi172 ceph-mon[30854]: pgmap v307: 129 pgs: 129 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T08:44:56.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:56 smithi191 ceph-mon[35314]: pgmap v307: 129 pgs: 129 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T08:44:56.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:56 smithi174 ceph-mon[34068]: pgmap v307: 129 pgs: 129 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T08:44:56.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.775+0000 7f041f6cb640 1 -- 172.21.15.172:0/3205566758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0418101420 msgr2=0x7f041810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:56.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.775+0000 7f041f6cb640 1 --2- 172.21.15.172:0/3205566758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0418101420 0x7f041810d950 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0410009cd0 tx=0x7f041002a8e0 comp rx=0 tx=0).stop 2024-09-10T08:44:56.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.775+0000 7f041f6cb640 1 -- 172.21.15.172:0/3205566758 shutdown_connections 2024-09-10T08:44:56.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.775+0000 7f041f6cb640 1 --2- 172.21.15.172:0/3205566758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0418101420 0x7f041810d950 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:56.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.775+0000 7f041f6cb640 1 --2- 172.21.15.172:0/3205566758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0418100a80 0x7f0418100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:56.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.775+0000 7f041f6cb640 1 --2- 172.21.15.172:0/3205566758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0418106a80 0x7f0418106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:56.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.775+0000 7f041f6cb640 1 -- 172.21.15.172:0/3205566758 >> 172.21.15.172:0/3205566758 conn(0x7f04180fc7a0 msgr2=0x7f04180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:56.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.775+0000 7f041f6cb640 1 -- 172.21.15.172:0/3205566758 shutdown_connections 2024-09-10T08:44:56.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.775+0000 7f041f6cb640 1 -- 172.21.15.172:0/3205566758 wait complete. 2024-09-10T08:44:56.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.775+0000 7f041f6cb640 1 Processor -- start 2024-09-10T08:44:56.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f041f6cb640 1 -- start start 2024-09-10T08:44:56.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f041f6cb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0418100a80 0x7f0418197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:56.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f041f6cb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0418101420 0x7f04181977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:56.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f041f6cb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0418106a80 0x7f041819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:56.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f041f6cb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0418110d30 con 0x7f0418101420 2024-09-10T08:44:56.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f041f6cb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0418110bb0 con 0x7f0418106a80 2024-09-10T08:44:56.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f041f6cb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0418110eb0 con 0x7f0418100a80 2024-09-10T08:44:56.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f041cc66640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0418101420 0x7f04181977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:56.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f041cc66640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0418101420 0x7f04181977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56992/0 (socket says 172.21.15.172:56992) 2024-09-10T08:44:56.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f041cc66640 1 -- 172.21.15.172:0/4238825240 learned_addr learned my addr 172.21.15.172:0/4238825240 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:56.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f041cc66640 1 -- 172.21.15.172:0/4238825240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0418100a80 msgr2=0x7f0418197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:56.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f041cc66640 1 --2- 172.21.15.172:0/4238825240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0418100a80 0x7f0418197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:56.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f041cc66640 1 -- 172.21.15.172:0/4238825240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0418106a80 msgr2=0x7f041819bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:56.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f041cc66640 1 --2- 172.21.15.172:0/4238825240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0418106a80 0x7f041819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:56.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f041cc66640 1 -- 172.21.15.172:0/4238825240 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041819c220 con 0x7f0418101420 2024-09-10T08:44:56.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f041cc66640 1 --2- 172.21.15.172:0/4238825240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0418101420 0x7f04181977b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f040c0078f0 tx=0x7f040c007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:56.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f040a7fc640 1 -- 172.21.15.172:0/4238825240 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c019020 con 0x7f0418101420 2024-09-10T08:44:56.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f040a7fc640 1 -- 172.21.15.172:0/4238825240 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f040c0054d0 con 0x7f0418101420 2024-09-10T08:44:56.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f041f6cb640 1 -- 172.21.15.172:0/4238825240 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041819c510 con 0x7f0418101420 2024-09-10T08:44:56.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.776+0000 7f040a7fc640 1 -- 172.21.15.172:0/4238825240 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c00e650 con 0x7f0418101420 2024-09-10T08:44:56.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.777+0000 7f041f6cb640 1 -- 172.21.15.172:0/4238825240 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041806a790 con 0x7f0418101420 2024-09-10T08:44:56.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.777+0000 7f041f6cb640 1 -- 172.21.15.172:0/4238825240 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04181021c0 con 0x7f0418101420 2024-09-10T08:44:56.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.779+0000 7f040a7fc640 1 -- 172.21.15.172:0/4238825240 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f040c005000 con 0x7f0418101420 2024-09-10T08:44:56.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.779+0000 7f040a7fc640 1 --2- 172.21.15.172:0/4238825240 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04040a2b80 0x7f04040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:56.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.779+0000 7f041d467640 1 --2- 172.21.15.172:0/4238825240 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04040a2b80 0x7f04040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:56.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.779+0000 7f040a7fc640 1 -- 172.21.15.172:0/4238825240 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f040c0d0fb0 con 0x7f0418101420 2024-09-10T08:44:56.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.780+0000 7f041d467640 1 --2- 172.21.15.172:0/4238825240 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04040a2b80 0x7f04040a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0414005d80 tx=0x7f0414005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:56.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.782+0000 7f040a7fc640 1 -- 172.21.15.172:0/4238825240 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f040c023020 con 0x7f0418101420 2024-09-10T08:44:56.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.972+0000 7f041f6cb640 1 -- 172.21.15.172:0/4238825240 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f041806b240 con 0x7f04040a2b80 2024-09-10T08:44:56.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.997+0000 7f040a7fc640 1 -- 172.21.15.172:0/4238825240 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f041806b240 con 0x7f04040a2b80 2024-09-10T08:44:56.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.999+0000 7f041f6cb640 1 -- 172.21.15.172:0/4238825240 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04040a2b80 msgr2=0x7f04040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:57.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.999+0000 7f041f6cb640 1 --2- 172.21.15.172:0/4238825240 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04040a2b80 0x7f04040a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0414005d80 tx=0x7f0414005c90 comp rx=0 tx=0).stop 2024-09-10T08:44:57.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.999+0000 7f041f6cb640 1 -- 172.21.15.172:0/4238825240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0418101420 msgr2=0x7f04181977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:57.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.999+0000 7f041f6cb640 1 --2- 172.21.15.172:0/4238825240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0418101420 0x7f04181977b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f040c0078f0 tx=0x7f040c007db0 comp rx=0 tx=0).stop 2024-09-10T08:44:57.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.999+0000 7f041f6cb640 1 -- 172.21.15.172:0/4238825240 shutdown_connections 2024-09-10T08:44:57.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.999+0000 7f041f6cb640 1 --2- 172.21.15.172:0/4238825240 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04040a2b80 0x7f04040a5040 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:57.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.999+0000 7f041f6cb640 1 --2- 172.21.15.172:0/4238825240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0418106a80 0x7f041819bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:57.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.999+0000 7f041f6cb640 1 --2- 172.21.15.172:0/4238825240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0418101420 0x7f04181977b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:57.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.999+0000 7f041f6cb640 1 --2- 172.21.15.172:0/4238825240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0418100a80 0x7f0418197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:57.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.999+0000 7f041f6cb640 1 -- 172.21.15.172:0/4238825240 >> 172.21.15.172:0/4238825240 conn(0x7f04180fc7a0 msgr2=0x7f04180fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:57.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.999+0000 7f041f6cb640 1 -- 172.21.15.172:0/4238825240 shutdown_connections 2024-09-10T08:44:57.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:56.999+0000 7f041f6cb640 1 -- 172.21.15.172:0/4238825240 wait complete. 2024-09-10T08:44:57.306 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:44:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:44:56.977+0000 7f0435c89640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:44:57.306 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:44:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:44:56.977+0000 7f0435c89640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:44:57.306 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:44:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:44:56.977+0000 7f0435c89640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:44:57.306 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:44:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:44:56.977+0000 7f0435c89640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:44:57.306 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 08:44:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T08:44:56.977+0000 7f0435c89640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T08:44:57.491 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 08:44:57 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-b[63897]: 2024-09-10T08:44:57.176+0000 7ff8b4274640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-10T08:44:57.492 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 08:44:57 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-b[63897]: 2024-09-10T08:44:57.176+0000 7ff8b4274640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-10T08:44:57.492 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 08:44:57 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-b[63897]: 2024-09-10T08:44:57.176+0000 7ff8b4274640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-10T08:44:57.643 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-10T08:44:57.890 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:44:58.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.403+0000 7f658a1d4640 1 -- 172.21.15.172:0/645840922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f658410c180 msgr2=0x7f658410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:58.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.403+0000 7f658a1d4640 1 --2- 172.21.15.172:0/645840922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f658410c180 0x7f658410e610 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f657c009ca0 tx=0x7f657c02a950 comp rx=0 tx=0).stop 2024-09-10T08:44:58.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.403+0000 7f658a1d4640 1 -- 172.21.15.172:0/645840922 shutdown_connections 2024-09-10T08:44:58.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.403+0000 7f658a1d4640 1 --2- 172.21.15.172:0/645840922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f658410c180 0x7f658410e610 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.403+0000 7f658a1d4640 1 --2- 172.21.15.172:0/645840922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6584103700 0x7f658410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.403+0000 7f658a1d4640 1 --2- 172.21.15.172:0/645840922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6584102de0 0x7f65841031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.403+0000 7f658a1d4640 1 -- 172.21.15.172:0/645840922 >> 172.21.15.172:0/645840922 conn(0x7f65840fc7a0 msgr2=0x7f65840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:58.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.403+0000 7f658a1d4640 1 -- 172.21.15.172:0/645840922 shutdown_connections 2024-09-10T08:44:58.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.403+0000 7f658a1d4640 1 -- 172.21.15.172:0/645840922 wait complete. 2024-09-10T08:44:58.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.404+0000 7f658a1d4640 1 Processor -- start 2024-09-10T08:44:58.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.404+0000 7f658a1d4640 1 -- start start 2024-09-10T08:44:58.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.404+0000 7f658a1d4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6584102de0 0x7f65841982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.404+0000 7f658a1d4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6584103700 0x7f65841987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.404+0000 7f658a1d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f658410c180 0x7f658419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.404+0000 7f658a1d4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6584111e50 con 0x7f6584103700 2024-09-10T08:44:58.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.404+0000 7f658a1d4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6584111cd0 con 0x7f6584102de0 2024-09-10T08:44:58.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.404+0000 7f658a1d4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6584111fd0 con 0x7f658410c180 2024-09-10T08:44:58.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.404+0000 7f6582ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6584103700 0x7f65841987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.404+0000 7f6582ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6584103700 0x7f65841987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57024/0 (socket says 172.21.15.172:57024) 2024-09-10T08:44:58.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.405+0000 7f6582ffd640 1 -- 172.21.15.172:0/903908927 learned_addr learned my addr 172.21.15.172:0/903908927 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:44:58.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.405+0000 7f65837fe640 1 --2- 172.21.15.172:0/903908927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6584102de0 0x7f65841982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.405+0000 7f6583fff640 1 --2- 172.21.15.172:0/903908927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f658410c180 0x7f658419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.405+0000 7f6582ffd640 1 -- 172.21.15.172:0/903908927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f658410c180 msgr2=0x7f658419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:58.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.405+0000 7f6582ffd640 1 --2- 172.21.15.172:0/903908927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f658410c180 0x7f658419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.405+0000 7f6582ffd640 1 -- 172.21.15.172:0/903908927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6584102de0 msgr2=0x7f65841982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:58.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.405+0000 7f6582ffd640 1 --2- 172.21.15.172:0/903908927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6584102de0 0x7f65841982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.405+0000 7f6582ffd640 1 -- 172.21.15.172:0/903908927 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658419d260 con 0x7f6584103700 2024-09-10T08:44:58.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.405+0000 7f6582ffd640 1 --2- 172.21.15.172:0/903908927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6584103700 0x7f65841987f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f657800d770 tx=0x7f657800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:58.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.405+0000 7f6580ff9640 1 -- 172.21.15.172:0/903908927 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578017070 con 0x7f6584103700 2024-09-10T08:44:58.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.405+0000 7f6580ff9640 1 -- 172.21.15.172:0/903908927 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6578009820 con 0x7f6584103700 2024-09-10T08:44:58.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.405+0000 7f658a1d4640 1 -- 172.21.15.172:0/903908927 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658419d550 con 0x7f6584103700 2024-09-10T08:44:58.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.405+0000 7f6580ff9640 1 -- 172.21.15.172:0/903908927 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578004ba0 con 0x7f6584103700 2024-09-10T08:44:58.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.405+0000 7f658a1d4640 1 -- 172.21.15.172:0/903908927 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65841a4e30 con 0x7f6584103700 2024-09-10T08:44:58.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.406+0000 7f658a1d4640 1 -- 172.21.15.172:0/903908927 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65841031c0 con 0x7f6584103700 2024-09-10T08:44:58.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.407+0000 7f6580ff9640 1 -- 172.21.15.172:0/903908927 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f65780099d0 con 0x7f6584103700 2024-09-10T08:44:58.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.408+0000 7f6580ff9640 1 --2- 172.21.15.172:0/903908927 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f656c0a2c10 0x7f656c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:44:58.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.408+0000 7f65837fe640 1 --2- 172.21.15.172:0/903908927 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f656c0a2c10 0x7f656c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:44:58.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.408+0000 7f6580ff9640 1 -- 172.21.15.172:0/903908927 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f65780cf150 con 0x7f6584103700 2024-09-10T08:44:58.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.408+0000 7f65837fe640 1 --2- 172.21.15.172:0/903908927 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f656c0a2c10 0x7f656c0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f657400abe0 tx=0x7f657400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:44:58.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.411+0000 7f6580ff9640 1 -- 172.21.15.172:0/903908927 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f657809bd10 con 0x7f6584103700 2024-09-10T08:44:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:58 smithi191 ceph-mon[35314]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:44:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:58 smithi191 ceph-mon[35314]: pgmap v308: 129 pgs: 129 active+clean; 585 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-10T08:44:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:44:58 smithi191 ceph-mon[35314]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T08:44:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:58 smithi172 ceph-mon[30854]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:44:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:58 smithi172 ceph-mon[30854]: pgmap v308: 129 pgs: 129 active+clean; 585 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-10T08:44:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:44:58 smithi172 ceph-mon[30854]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T08:44:58.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.603+0000 7f658a1d4640 1 -- 172.21.15.172:0/903908927 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6584062630 con 0x7f656c0a2c10 2024-09-10T08:44:58.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.626+0000 7f6580ff9640 1 -- 172.21.15.172:0/903908927 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6584062630 con 0x7f656c0a2c10 2024-09-10T08:44:58.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.628+0000 7f658a1d4640 1 -- 172.21.15.172:0/903908927 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f656c0a2c10 msgr2=0x7f656c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:58.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.628+0000 7f658a1d4640 1 --2- 172.21.15.172:0/903908927 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f656c0a2c10 0x7f656c0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f657400abe0 tx=0x7f657400a2d0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.628+0000 7f658a1d4640 1 -- 172.21.15.172:0/903908927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6584103700 msgr2=0x7f65841987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:44:58.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.628+0000 7f658a1d4640 1 --2- 172.21.15.172:0/903908927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6584103700 0x7f65841987f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f657800d770 tx=0x7f657800db70 comp rx=0 tx=0).stop 2024-09-10T08:44:58.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.628+0000 7f658a1d4640 1 -- 172.21.15.172:0/903908927 shutdown_connections 2024-09-10T08:44:58.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.628+0000 7f658a1d4640 1 --2- 172.21.15.172:0/903908927 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f656c0a2c10 0x7f656c0a50d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.628+0000 7f658a1d4640 1 --2- 172.21.15.172:0/903908927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f658410c180 0x7f658419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.628+0000 7f658a1d4640 1 --2- 172.21.15.172:0/903908927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6584103700 0x7f65841987f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.628+0000 7f658a1d4640 1 --2- 172.21.15.172:0/903908927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6584102de0 0x7f65841982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:44:58.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.628+0000 7f658a1d4640 1 -- 172.21.15.172:0/903908927 >> 172.21.15.172:0/903908927 conn(0x7f65840fc7a0 msgr2=0x7f65840fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:44:58.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.629+0000 7f658a1d4640 1 -- 172.21.15.172:0/903908927 shutdown_connections 2024-09-10T08:44:58.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:44:58.629+0000 7f658a1d4640 1 -- 172.21.15.172:0/903908927 wait complete. 2024-09-10T08:44:58.664 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:58 smithi174 ceph-mon[34068]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:44:58.664 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:58 smithi174 ceph-mon[34068]: pgmap v308: 129 pgs: 129 active+clean; 585 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-10T08:44:58.664 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:44:58 smithi174 ceph-mon[34068]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T08:44:59.255 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-10T08:44:59.513 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:00.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.024+0000 7f4eeec3a640 1 -- 172.21.15.172:0/2418384618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ee8101420 msgr2=0x7f4ee810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:00.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.024+0000 7f4eeec3a640 1 --2- 172.21.15.172:0/2418384618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ee8101420 0x7f4ee810d950 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009cd0 tx=0x7f4ee002a8e0 comp rx=0 tx=0).stop 2024-09-10T08:45:00.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.024+0000 7f4eeec3a640 1 -- 172.21.15.172:0/2418384618 shutdown_connections 2024-09-10T08:45:00.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.024+0000 7f4eeec3a640 1 --2- 172.21.15.172:0/2418384618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ee8101420 0x7f4ee810d950 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:00.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.024+0000 7f4eeec3a640 1 --2- 172.21.15.172:0/2418384618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ee8100a80 0x7f4ee8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:00.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.024+0000 7f4eeec3a640 1 --2- 172.21.15.172:0/2418384618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee8106a80 0x7f4ee8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:00.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.024+0000 7f4eeec3a640 1 -- 172.21.15.172:0/2418384618 >> 172.21.15.172:0/2418384618 conn(0x7f4ee80fc7a0 msgr2=0x7f4ee80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:00.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.024+0000 7f4eeec3a640 1 -- 172.21.15.172:0/2418384618 shutdown_connections 2024-09-10T08:45:00.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.024+0000 7f4eeec3a640 1 -- 172.21.15.172:0/2418384618 wait complete. 2024-09-10T08:45:00.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.025+0000 7f4eeec3a640 1 Processor -- start 2024-09-10T08:45:00.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.025+0000 7f4eeec3a640 1 -- start start 2024-09-10T08:45:00.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.025+0000 7f4eeec3a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee8100a80 0x7f4ee80fcee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:00.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.025+0000 7f4eeec3a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ee8101420 0x7f4ee80fd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:00.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.025+0000 7f4eeec3a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ee8106a80 0x7f4ee80fd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:00.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.025+0000 7f4eeec3a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8110d30 con 0x7f4ee8106a80 2024-09-10T08:45:00.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.025+0000 7f4eeec3a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8110bb0 con 0x7f4ee8100a80 2024-09-10T08:45:00.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.025+0000 7f4eeec3a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8110eb0 con 0x7f4ee8101420 2024-09-10T08:45:00.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.025+0000 7f4eed1d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ee8106a80 0x7f4ee80fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:00.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.025+0000 7f4eed1d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ee8106a80 0x7f4ee80fd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57032/0 (socket says 172.21.15.172:57032) 2024-09-10T08:45:00.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.025+0000 7f4eed1d7640 1 -- 172.21.15.172:0/1190139758 learned_addr learned my addr 172.21.15.172:0/1190139758 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:00.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.025+0000 7f4eec9d6640 1 --2- 172.21.15.172:0/1190139758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee8100a80 0x7f4ee80fcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:00.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.025+0000 7f4edffff640 1 --2- 172.21.15.172:0/1190139758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ee8101420 0x7f4ee80fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:00.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.026+0000 7f4eed1d7640 1 -- 172.21.15.172:0/1190139758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ee8101420 msgr2=0x7f4ee80fd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:00.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.026+0000 7f4eed1d7640 1 --2- 172.21.15.172:0/1190139758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ee8101420 0x7f4ee80fd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:00.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.026+0000 7f4eed1d7640 1 -- 172.21.15.172:0/1190139758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee8100a80 msgr2=0x7f4ee80fcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:00.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.026+0000 7f4eed1d7640 1 --2- 172.21.15.172:0/1190139758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee8100a80 0x7f4ee80fcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:00.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.026+0000 7f4eed1d7640 1 -- 172.21.15.172:0/1190139758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee81a6830 con 0x7f4ee8106a80 2024-09-10T08:45:00.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.026+0000 7f4edffff640 1 --2- 172.21.15.172:0/1190139758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ee8101420 0x7f4ee80fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:00.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.026+0000 7f4eed1d7640 1 --2- 172.21.15.172:0/1190139758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ee8106a80 0x7f4ee80fd960 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0033040 tx=0x7f4ee0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:00.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.026+0000 7f4eddffb640 1 -- 172.21.15.172:0/1190139758 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0004040 con 0x7f4ee8106a80 2024-09-10T08:45:00.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.026+0000 7f4eeec3a640 1 -- 172.21.15.172:0/1190139758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee81a6ac0 con 0x7f4ee8106a80 2024-09-10T08:45:00.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.026+0000 7f4eddffb640 1 -- 172.21.15.172:0/1190139758 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ee0004690 con 0x7f4ee8106a80 2024-09-10T08:45:00.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.026+0000 7f4eddffb640 1 -- 172.21.15.172:0/1190139758 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0030e10 con 0x7f4ee8106a80 2024-09-10T08:45:00.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.026+0000 7f4eeec3a640 1 -- 172.21.15.172:0/1190139758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee81a7000 con 0x7f4ee8106a80 2024-09-10T08:45:00.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.027+0000 7f4eeec3a640 1 -- 172.21.15.172:0/1190139758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee8100ee0 con 0x7f4ee8106a80 2024-09-10T08:45:00.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.028+0000 7f4eddffb640 1 -- 172.21.15.172:0/1190139758 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4ee00305a0 con 0x7f4ee8106a80 2024-09-10T08:45:00.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.029+0000 7f4eddffb640 1 --2- 172.21.15.172:0/1190139758 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4ed00a2c10 0x7f4ed00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:00.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.029+0000 7f4eec9d6640 1 --2- 172.21.15.172:0/1190139758 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4ed00a2c10 0x7f4ed00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:00.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.029+0000 7f4eddffb640 1 -- 172.21.15.172:0/1190139758 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f4ee002e030 con 0x7f4ee8106a80 2024-09-10T08:45:00.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.029+0000 7f4eec9d6640 1 --2- 172.21.15.172:0/1190139758 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4ed00a2c10 0x7f4ed00a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009b50 tx=0x7f4ee4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:00.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.033+0000 7f4eddffb640 1 -- 172.21.15.172:0/1190139758 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ee004b060 con 0x7f4ee8106a80 2024-09-10T08:45:00.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.224+0000 7f4eeec3a640 1 -- 172.21.15.172:0/1190139758 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f4ee810a070 con 0x7f4ed00a2c10 2024-09-10T08:45:00.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.229+0000 7f4eddffb640 1 -- 172.21.15.172:0/1190139758 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4ee4004050 con 0x7f4ed00a2c10 2024-09-10T08:45:00.229 INFO:teuthology.orchestra.run.smithi172.stdout:/volumes/qa/sv_0/1a98a737-45db-4cda-bd0b-58296d0e5397 2024-09-10T08:45:00.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.231+0000 7f4eeec3a640 1 -- 172.21.15.172:0/1190139758 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4ed00a2c10 msgr2=0x7f4ed00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:00.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.231+0000 7f4eeec3a640 1 --2- 172.21.15.172:0/1190139758 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4ed00a2c10 0x7f4ed00a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009b50 tx=0x7f4ee4005f90 comp rx=0 tx=0).stop 2024-09-10T08:45:00.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.231+0000 7f4eeec3a640 1 -- 172.21.15.172:0/1190139758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ee8106a80 msgr2=0x7f4ee80fd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:00.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.231+0000 7f4eeec3a640 1 --2- 172.21.15.172:0/1190139758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ee8106a80 0x7f4ee80fd960 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0033040 tx=0x7f4ee0003970 comp rx=0 tx=0).stop 2024-09-10T08:45:00.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.231+0000 7f4eeec3a640 1 -- 172.21.15.172:0/1190139758 shutdown_connections 2024-09-10T08:45:00.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.231+0000 7f4eeec3a640 1 --2- 172.21.15.172:0/1190139758 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4ed00a2c10 0x7f4ed00a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:00.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.231+0000 7f4eeec3a640 1 --2- 172.21.15.172:0/1190139758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ee8106a80 0x7f4ee80fd960 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:00.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.231+0000 7f4eeec3a640 1 --2- 172.21.15.172:0/1190139758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ee8101420 0x7f4ee80fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:00.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.231+0000 7f4eeec3a640 1 --2- 172.21.15.172:0/1190139758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee8100a80 0x7f4ee80fcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:00.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.231+0000 7f4eeec3a640 1 -- 172.21.15.172:0/1190139758 >> 172.21.15.172:0/1190139758 conn(0x7f4ee80fc7a0 msgr2=0x7f4ee806ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:00.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.231+0000 7f4eeec3a640 1 -- 172.21.15.172:0/1190139758 shutdown_connections 2024-09-10T08:45:00.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:00.231+0000 7f4eeec3a640 1 -- 172.21.15.172:0/1190139758 wait complete. 2024-09-10T08:45:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:00 smithi191 ceph-mon[35314]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:45:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:00 smithi191 ceph-mon[35314]: pgmap v309: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.1 KiB/s wr, 17 op/s 2024-09-10T08:45:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:00 smithi172 ceph-mon[30854]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:45:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:00 smithi172 ceph-mon[30854]: pgmap v309: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.1 KiB/s wr, 17 op/s 2024-09-10T08:45:00.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:00 smithi174 ceph-mon[34068]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:45:00.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:00 smithi174 ceph-mon[34068]: pgmap v309: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.1 KiB/s wr, 17 op/s 2024-09-10T08:45:00.933 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-10T08:45:01.182 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:01.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:01 smithi191 ceph-mon[35314]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:45:01.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:01 smithi172 ceph-mon[30854]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:45:01.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.712+0000 7f433c7c2640 1 -- 172.21.15.172:0/2935819959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4334102ef0 msgr2=0x7f433410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:01.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.712+0000 7f433c7c2640 1 --2- 172.21.15.172:0/2935819959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4334102ef0 0x7f433410da40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f432c009ca0 tx=0x7f432c02a950 comp rx=0 tx=0).stop 2024-09-10T08:45:01.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.712+0000 7f433c7c2640 1 -- 172.21.15.172:0/2935819959 shutdown_connections 2024-09-10T08:45:01.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.712+0000 7f433c7c2640 1 --2- 172.21.15.172:0/2935819959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4334102ef0 0x7f433410da40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.712+0000 7f433c7c2640 1 --2- 172.21.15.172:0/2935819959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4334102570 0x7f43341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.712+0000 7f433c7c2640 1 --2- 172.21.15.172:0/2935819959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4334100e20 0x7f4334101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.712+0000 7f433c7c2640 1 -- 172.21.15.172:0/2935819959 >> 172.21.15.172:0/2935819959 conn(0x7f43340fc7d0 msgr2=0x7f43340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:01.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.712+0000 7f433c7c2640 1 -- 172.21.15.172:0/2935819959 shutdown_connections 2024-09-10T08:45:01.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.712+0000 7f433c7c2640 1 -- 172.21.15.172:0/2935819959 wait complete. 2024-09-10T08:45:01.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433c7c2640 1 Processor -- start 2024-09-10T08:45:01.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433c7c2640 1 -- start start 2024-09-10T08:45:01.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433c7c2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4334100e20 0x7f43341982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:01.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433c7c2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4334102570 0x7f4334198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:01.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433c7c2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4334102ef0 0x7f433419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:01.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433c7c2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4334110e30 con 0x7f4334100e20 2024-09-10T08:45:01.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433c7c2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4334110cb0 con 0x7f4334102ef0 2024-09-10T08:45:01.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433c7c2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4334110fb0 con 0x7f4334102570 2024-09-10T08:45:01.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433a55e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4334100e20 0x7f43341982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:01.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433a55e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4334100e20 0x7f43341982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57058/0 (socket says 172.21.15.172:57058) 2024-09-10T08:45:01.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433a55e640 1 -- 172.21.15.172:0/2653212401 learned_addr learned my addr 172.21.15.172:0/2653212401 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:01.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433ad5f640 1 --2- 172.21.15.172:0/2653212401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4334102ef0 0x7f433419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:01.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f4339d5d640 1 --2- 172.21.15.172:0/2653212401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4334102570 0x7f4334198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:01.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433a55e640 1 -- 172.21.15.172:0/2653212401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4334102570 msgr2=0x7f4334198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:01.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433a55e640 1 --2- 172.21.15.172:0/2653212401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4334102570 0x7f4334198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433a55e640 1 -- 172.21.15.172:0/2653212401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4334102ef0 msgr2=0x7f433419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433a55e640 1 --2- 172.21.15.172:0/2653212401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4334102ef0 0x7f433419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433a55e640 1 -- 172.21.15.172:0/2653212401 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433419d2a0 con 0x7f4334100e20 2024-09-10T08:45:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f4339d5d640 1 --2- 172.21.15.172:0/2653212401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4334102570 0x7f4334198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f433a55e640 1 --2- 172.21.15.172:0/2653212401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4334100e20 0x7f43341982f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f433000d950 tx=0x7f433000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.713+0000 7f432b7fe640 1 -- 172.21.15.172:0/2653212401 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330017030 con 0x7f4334100e20 2024-09-10T08:45:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.714+0000 7f433c7c2640 1 -- 172.21.15.172:0/2653212401 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433419d590 con 0x7f4334100e20 2024-09-10T08:45:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.714+0000 7f432b7fe640 1 -- 172.21.15.172:0/2653212401 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4330004020 con 0x7f4334100e20 2024-09-10T08:45:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.714+0000 7f432b7fe640 1 -- 172.21.15.172:0/2653212401 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43300047a0 con 0x7f4334100e20 2024-09-10T08:45:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.714+0000 7f433c7c2640 1 -- 172.21.15.172:0/2653212401 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43341a4e70 con 0x7f4334100e20 2024-09-10T08:45:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.715+0000 7f433c7c2640 1 -- 172.21.15.172:0/2653212401 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4334101200 con 0x7f4334100e20 2024-09-10T08:45:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.716+0000 7f432b7fe640 1 -- 172.21.15.172:0/2653212401 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f43300041d0 con 0x7f4334100e20 2024-09-10T08:45:01.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.716+0000 7f432b7fe640 1 --2- 172.21.15.172:0/2653212401 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f43200a2c10 0x7f43200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:01.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.716+0000 7f4339d5d640 1 --2- 172.21.15.172:0/2653212401 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f43200a2c10 0x7f43200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:01.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.716+0000 7f432b7fe640 1 -- 172.21.15.172:0/2653212401 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f43300cefd0 con 0x7f4334100e20 2024-09-10T08:45:01.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.717+0000 7f4339d5d640 1 --2- 172.21.15.172:0/2653212401 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f43200a2c10 0x7f43200a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4334105180 tx=0x7f4324005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:01.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.720+0000 7f432b7fe640 1 -- 172.21.15.172:0/2653212401 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f433009bb90 con 0x7f4334100e20 2024-09-10T08:45:01.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:01 smithi174 ceph-mon[34068]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:45:01.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.911+0000 7f433c7c2640 1 -- 172.21.15.172:0/2653212401 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f433410a0d0 con 0x7f43200a2c10 2024-09-10T08:45:01.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.935+0000 7f432b7fe640 1 -- 172.21.15.172:0/2653212401 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f433410a0d0 con 0x7f43200a2c10 2024-09-10T08:45:01.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.937+0000 7f433c7c2640 1 -- 172.21.15.172:0/2653212401 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f43200a2c10 msgr2=0x7f43200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:01.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.937+0000 7f433c7c2640 1 --2- 172.21.15.172:0/2653212401 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f43200a2c10 0x7f43200a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4334105180 tx=0x7f4324005d20 comp rx=0 tx=0).stop 2024-09-10T08:45:01.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.937+0000 7f433c7c2640 1 -- 172.21.15.172:0/2653212401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4334100e20 msgr2=0x7f43341982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:01.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.937+0000 7f433c7c2640 1 --2- 172.21.15.172:0/2653212401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4334100e20 0x7f43341982f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f433000d950 tx=0x7f433000de10 comp rx=0 tx=0).stop 2024-09-10T08:45:01.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.937+0000 7f433c7c2640 1 -- 172.21.15.172:0/2653212401 shutdown_connections 2024-09-10T08:45:01.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.937+0000 7f433c7c2640 1 --2- 172.21.15.172:0/2653212401 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f43200a2c10 0x7f43200a50d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.937+0000 7f433c7c2640 1 --2- 172.21.15.172:0/2653212401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4334102ef0 0x7f433419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.937+0000 7f433c7c2640 1 --2- 172.21.15.172:0/2653212401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4334102570 0x7f4334198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.937+0000 7f433c7c2640 1 --2- 172.21.15.172:0/2653212401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4334100e20 0x7f43341982f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:01.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.937+0000 7f433c7c2640 1 -- 172.21.15.172:0/2653212401 >> 172.21.15.172:0/2653212401 conn(0x7f43340fc7d0 msgr2=0x7f433406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:01.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.937+0000 7f433c7c2640 1 -- 172.21.15.172:0/2653212401 shutdown_connections 2024-09-10T08:45:01.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:01.937+0000 7f433c7c2640 1 -- 172.21.15.172:0/2653212401 wait complete. 2024-09-10T08:45:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:02 smithi191 ceph-mon[35314]: pgmap v310: 129 pgs: 129 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.1 KiB/s wr, 21 op/s 2024-09-10T08:45:02.588 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:02 smithi172 ceph-mon[30854]: pgmap v310: 129 pgs: 129 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.1 KiB/s wr, 21 op/s 2024-09-10T08:45:02.629 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-10T08:45:02.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:02 smithi174 ceph-mon[34068]: pgmap v310: 129 pgs: 129 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.1 KiB/s wr, 21 op/s 2024-09-10T08:45:02.884 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:03.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.404+0000 7fb872940640 1 -- 172.21.15.172:0/534771973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c102790 msgr2=0x7fb86c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.404+0000 7fb872940640 1 --2- 172.21.15.172:0/534771973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c102790 0x7fb86c1067e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb860009900 tx=0x7fb86002a410 comp rx=0 tx=0).stop 2024-09-10T08:45:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.404+0000 7fb872940640 1 -- 172.21.15.172:0/534771973 shutdown_connections 2024-09-10T08:45:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.404+0000 7fb872940640 1 --2- 172.21.15.172:0/534771973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb86c106f10 0x7fb86c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.404+0000 7fb872940640 1 --2- 172.21.15.172:0/534771973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c102790 0x7fb86c1067e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.404+0000 7fb872940640 1 --2- 172.21.15.172:0/534771973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c101de0 0x7fb86c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.404+0000 7fb872940640 1 -- 172.21.15.172:0/534771973 >> 172.21.15.172:0/534771973 conn(0x7fb86c0fd8c0 msgr2=0x7fb86c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.404+0000 7fb872940640 1 -- 172.21.15.172:0/534771973 shutdown_connections 2024-09-10T08:45:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.404+0000 7fb872940640 1 -- 172.21.15.172:0/534771973 wait complete. 2024-09-10T08:45:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.405+0000 7fb872940640 1 Processor -- start 2024-09-10T08:45:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.405+0000 7fb872940640 1 -- start start 2024-09-10T08:45:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.405+0000 7fb872940640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb86c101de0 0x7fb86c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.405+0000 7fb872940640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c102790 0x7fb86c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.405+0000 7fb872940640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c106f10 0x7fb86c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.405+0000 7fb872940640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c111e20 con 0x7fb86c106f10 2024-09-10T08:45:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.405+0000 7fb872940640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c111ca0 con 0x7fb86c102790 2024-09-10T08:45:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.405+0000 7fb872940640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c111fa0 con 0x7fb86c101de0 2024-09-10T08:45:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.406+0000 7fb870edd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c106f10 0x7fb86c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.406+0000 7fb870edd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c106f10 0x7fb86c19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57072/0 (socket says 172.21.15.172:57072) 2024-09-10T08:45:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.406+0000 7fb870edd640 1 -- 172.21.15.172:0/2440979001 learned_addr learned my addr 172.21.15.172:0/2440979001 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.406+0000 7fb86b7fe640 1 --2- 172.21.15.172:0/2440979001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c102790 0x7fb86c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.406+0000 7fb86bfff640 1 --2- 172.21.15.172:0/2440979001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb86c101de0 0x7fb86c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.406+0000 7fb870edd640 1 -- 172.21.15.172:0/2440979001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb86c101de0 msgr2=0x7fb86c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:03.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.406+0000 7fb870edd640 1 --2- 172.21.15.172:0/2440979001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb86c101de0 0x7fb86c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.406+0000 7fb870edd640 1 -- 172.21.15.172:0/2440979001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c102790 msgr2=0x7fb86c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:03.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.406+0000 7fb870edd640 1 --2- 172.21.15.172:0/2440979001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c102790 0x7fb86c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.406+0000 7fb870edd640 1 -- 172.21.15.172:0/2440979001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c19d350 con 0x7fb86c106f10 2024-09-10T08:45:03.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.406+0000 7fb870edd640 1 --2- 172.21.15.172:0/2440979001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c106f10 0x7fb86c19cc70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb86400eaa0 tx=0x7fb86400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:03.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.406+0000 7fb8697fa640 1 -- 172.21.15.172:0/2440979001 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86400cdb0 con 0x7fb86c106f10 2024-09-10T08:45:03.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.406+0000 7fb872940640 1 -- 172.21.15.172:0/2440979001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86c19d640 con 0x7fb86c106f10 2024-09-10T08:45:03.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.406+0000 7fb8697fa640 1 -- 172.21.15.172:0/2440979001 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb864005020 con 0x7fb86c106f10 2024-09-10T08:45:03.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.406+0000 7fb8697fa640 1 -- 172.21.15.172:0/2440979001 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8640116d0 con 0x7fb86c106f10 2024-09-10T08:45:03.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.406+0000 7fb872940640 1 -- 172.21.15.172:0/2440979001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86c1a4f20 con 0x7fb86c106f10 2024-09-10T08:45:03.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.407+0000 7fb872940640 1 -- 172.21.15.172:0/2440979001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb86c1021c0 con 0x7fb86c106f10 2024-09-10T08:45:03.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.408+0000 7fb8697fa640 1 -- 172.21.15.172:0/2440979001 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb8640051d0 con 0x7fb86c106f10 2024-09-10T08:45:03.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.409+0000 7fb8697fa640 1 --2- 172.21.15.172:0/2440979001 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8540a2c90 0x7fb8540a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:03.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.409+0000 7fb8697fa640 1 -- 172.21.15.172:0/2440979001 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fb8640cf4d0 con 0x7fb86c106f10 2024-09-10T08:45:03.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.409+0000 7fb86bfff640 1 --2- 172.21.15.172:0/2440979001 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8540a2c90 0x7fb8540a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:03.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.409+0000 7fb86bfff640 1 --2- 172.21.15.172:0/2440979001 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8540a2c90 0x7fb8540a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb85c005fd0 tx=0x7fb85c005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:03.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.413+0000 7fb8697fa640 1 -- 172.21.15.172:0/2440979001 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb864016030 con 0x7fb86c106f10 2024-09-10T08:45:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:03 smithi191 ceph-mon[35314]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:45:03.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:03 smithi172 ceph-mon[30854]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:45:03.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.605+0000 7fb872940640 1 -- 172.21.15.172:0/2440979001 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fb86c062630 con 0x7fb8540a2c90 2024-09-10T08:45:03.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.610+0000 7fb8697fa640 1 -- 172.21.15.172:0/2440979001 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb86c062630 con 0x7fb8540a2c90 2024-09-10T08:45:03.611 INFO:teuthology.orchestra.run.smithi172.stdout:/volumes/qa/sv_1/5db8bac0-cdd8-4efe-8d44-d74821ce3690 2024-09-10T08:45:03.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.612+0000 7fb872940640 1 -- 172.21.15.172:0/2440979001 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8540a2c90 msgr2=0x7fb8540a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:03.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.612+0000 7fb872940640 1 --2- 172.21.15.172:0/2440979001 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8540a2c90 0x7fb8540a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb85c005fd0 tx=0x7fb85c005e20 comp rx=0 tx=0).stop 2024-09-10T08:45:03.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.612+0000 7fb872940640 1 -- 172.21.15.172:0/2440979001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c106f10 msgr2=0x7fb86c19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:03.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.612+0000 7fb872940640 1 --2- 172.21.15.172:0/2440979001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c106f10 0x7fb86c19cc70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb86400eaa0 tx=0x7fb86400ef60 comp rx=0 tx=0).stop 2024-09-10T08:45:03.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.613+0000 7fb872940640 1 -- 172.21.15.172:0/2440979001 shutdown_connections 2024-09-10T08:45:03.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.613+0000 7fb872940640 1 --2- 172.21.15.172:0/2440979001 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8540a2c90 0x7fb8540a5150 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.613+0000 7fb872940640 1 --2- 172.21.15.172:0/2440979001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86c106f10 0x7fb86c19cc70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.613+0000 7fb872940640 1 --2- 172.21.15.172:0/2440979001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c102790 0x7fb86c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.613+0000 7fb872940640 1 --2- 172.21.15.172:0/2440979001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb86c101de0 0x7fb86c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:03.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.613+0000 7fb872940640 1 -- 172.21.15.172:0/2440979001 >> 172.21.15.172:0/2440979001 conn(0x7fb86c0fd8c0 msgr2=0x7fb86c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:03.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.613+0000 7fb872940640 1 -- 172.21.15.172:0/2440979001 shutdown_connections 2024-09-10T08:45:03.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:03.613+0000 7fb872940640 1 -- 172.21.15.172:0/2440979001 wait complete. 2024-09-10T08:45:03.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:03 smithi174 ceph-mon[34068]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:45:04.287 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:45:04.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:04 smithi172 ceph-mon[30854]: pgmap v311: 129 pgs: 129 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-10T08:45:04.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:04 smithi191 ceph-mon[35314]: pgmap v311: 129 pgs: 129 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-10T08:45:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:04.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:04 smithi174 ceph-mon[34068]: pgmap v311: 129 pgs: 129 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-10T08:45:05.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.051+0000 7f1af844c640 1 -- 172.21.15.172:0/1221063903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af0106a60 msgr2=0x7f1af0106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.051+0000 7f1af844c640 1 --2- 172.21.15.172:0/1221063903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af0106a60 0x7f1af0106e40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009930 tx=0x7f1ae402a420 comp rx=0 tx=0).stop 2024-09-10T08:45:05.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.051+0000 7f1af844c640 1 -- 172.21.15.172:0/1221063903 shutdown_connections 2024-09-10T08:45:05.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.051+0000 7f1af844c640 1 --2- 172.21.15.172:0/1221063903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1af0101400 0x7f1af010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.051+0000 7f1af844c640 1 --2- 172.21.15.172:0/1221063903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1af0100a60 0x7f1af0100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.051+0000 7f1af844c640 1 --2- 172.21.15.172:0/1221063903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af0106a60 0x7f1af0106e40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.051+0000 7f1af844c640 1 -- 172.21.15.172:0/1221063903 >> 172.21.15.172:0/1221063903 conn(0x7f1af00fc780 msgr2=0x7f1af00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:05.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.051+0000 7f1af844c640 1 -- 172.21.15.172:0/1221063903 shutdown_connections 2024-09-10T08:45:05.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.051+0000 7f1af844c640 1 -- 172.21.15.172:0/1221063903 wait complete. 2024-09-10T08:45:05.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.051+0000 7f1af844c640 1 Processor -- start 2024-09-10T08:45:05.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.051+0000 7f1af844c640 1 -- start start 2024-09-10T08:45:05.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1af844c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af0100a60 0x7f1af01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1af844c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1af0101400 0x7f1af0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1af844c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1af0106a60 0x7f1af019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1af844c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1af0110cd0 con 0x7f1af0100a60 2024-09-10T08:45:05.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1af844c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1af0110b50 con 0x7f1af0101400 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1af844c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1af0110e50 con 0x7f1af0106a60 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1af61e8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af0100a60 0x7f1af01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1af61e8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af0100a60 0x7f1af01983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57090/0 (socket says 172.21.15.172:57090) 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1af61e8640 1 -- 172.21.15.172:0/2820521925 learned_addr learned my addr 172.21.15.172:0/2820521925 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1af59e7640 1 --2- 172.21.15.172:0/2820521925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1af0101400 0x7f1af0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1af69e9640 1 --2- 172.21.15.172:0/2820521925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1af0106a60 0x7f1af019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1af61e8640 1 -- 172.21.15.172:0/2820521925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1af0106a60 msgr2=0x7f1af019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1af61e8640 1 --2- 172.21.15.172:0/2820521925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1af0106a60 0x7f1af019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1af61e8640 1 -- 172.21.15.172:0/2820521925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1af0101400 msgr2=0x7f1af0198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1af61e8640 1 --2- 172.21.15.172:0/2820521925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1af0101400 0x7f1af0198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1af61e8640 1 -- 172.21.15.172:0/2820521925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af019d370 con 0x7f1af0100a60 2024-09-10T08:45:05.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1af61e8640 1 --2- 172.21.15.172:0/2820521925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af0100a60 0x7f1af01983c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4004770 tx=0x7f1ae40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:05.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.052+0000 7f1ae37fe640 1 -- 172.21.15.172:0/2820521925 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae403f070 con 0x7f1af0100a60 2024-09-10T08:45:05.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.053+0000 7f1ae37fe640 1 -- 172.21.15.172:0/2820521925 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ae403b460 con 0x7f1af0100a60 2024-09-10T08:45:05.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.053+0000 7f1af844c640 1 -- 172.21.15.172:0/2820521925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af019d600 con 0x7f1af0100a60 2024-09-10T08:45:05.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.053+0000 7f1ae37fe640 1 -- 172.21.15.172:0/2820521925 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4035070 con 0x7f1af0100a60 2024-09-10T08:45:05.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.053+0000 7f1af844c640 1 -- 172.21.15.172:0/2820521925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af01a4f40 con 0x7f1af0100a60 2024-09-10T08:45:05.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.054+0000 7f1af844c640 1 -- 172.21.15.172:0/2820521925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af01021a0 con 0x7f1af0100a60 2024-09-10T08:45:05.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.055+0000 7f1ae37fe640 1 -- 172.21.15.172:0/2820521925 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1ae403b610 con 0x7f1af0100a60 2024-09-10T08:45:05.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.055+0000 7f1ae37fe640 1 --2- 172.21.15.172:0/2820521925 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1adc0a2c90 0x7f1adc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:05.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.056+0000 7f1af59e7640 1 --2- 172.21.15.172:0/2820521925 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1adc0a2c90 0x7f1adc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:05.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.056+0000 7f1ae37fe640 1 -- 172.21.15.172:0/2820521925 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f1ae40ef720 con 0x7f1af0100a60 2024-09-10T08:45:05.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.056+0000 7f1af59e7640 1 --2- 172.21.15.172:0/2820521925 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1adc0a2c90 0x7f1adc0a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1af01062d0 tx=0x7f1aec00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:05.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.060+0000 7f1ae37fe640 1 -- 172.21.15.172:0/2820521925 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ae40bc2e0 con 0x7f1af0100a60 2024-09-10T08:45:05.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.261+0000 7f1af844c640 1 -- 172.21.15.172:0/2820521925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1af0109fe0 con 0x7f1af0100a60 2024-09-10T08:45:05.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.262+0000 7f1ae37fe640 1 -- 172.21.15.172:0/2820521925 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+12001 (secure 0 0 0) 0x7f1ae40bbbe0 con 0x7f1af0100a60 2024-09-10T08:45:05.263 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:45:05.263 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":28,"btime":"2024-09-10T08:44:55:226722+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:44:55.226705+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T08:45:05.263 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 28 2024-09-10T08:45:05.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.264+0000 7f1af844c640 1 -- 172.21.15.172:0/2820521925 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1adc0a2c90 msgr2=0x7f1adc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.264+0000 7f1af844c640 1 --2- 172.21.15.172:0/2820521925 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1adc0a2c90 0x7f1adc0a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1af01062d0 tx=0x7f1aec00b390 comp rx=0 tx=0).stop 2024-09-10T08:45:05.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.264+0000 7f1af844c640 1 -- 172.21.15.172:0/2820521925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af0100a60 msgr2=0x7f1af01983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:05.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.264+0000 7f1af844c640 1 --2- 172.21.15.172:0/2820521925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af0100a60 0x7f1af01983c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4004770 tx=0x7f1ae40042f0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.264+0000 7f1af844c640 1 -- 172.21.15.172:0/2820521925 shutdown_connections 2024-09-10T08:45:05.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.264+0000 7f1af844c640 1 --2- 172.21.15.172:0/2820521925 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1adc0a2c90 0x7f1adc0a5150 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.264+0000 7f1af844c640 1 --2- 172.21.15.172:0/2820521925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1af0106a60 0x7f1af019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.264+0000 7f1af844c640 1 --2- 172.21.15.172:0/2820521925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1af0101400 0x7f1af0198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.264+0000 7f1af844c640 1 --2- 172.21.15.172:0/2820521925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1af0100a60 0x7f1af01983c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:05.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.264+0000 7f1af844c640 1 -- 172.21.15.172:0/2820521925 >> 172.21.15.172:0/2820521925 conn(0x7f1af00fc780 msgr2=0x7f1af006ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:05.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.264+0000 7f1af844c640 1 -- 172.21.15.172:0/2820521925 shutdown_connections 2024-09-10T08:45:05.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:05.264+0000 7f1af844c640 1 -- 172.21.15.172:0/2820521925 wait complete. 2024-09-10T08:45:05.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:05 smithi191 ceph-mon[35314]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:45:05.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:05 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2820521925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:05.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:05 smithi172 ceph-mon[30854]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:45:05.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:05 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2820521925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:05.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:05 smithi174 ceph-mon[34068]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:45:05.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:05 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2820521925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:05.964 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:45:06.253 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:06.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:06 smithi191 ceph-mon[35314]: pgmap v312: 129 pgs: 129 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.3 KiB/s wr, 23 op/s 2024-09-10T08:45:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:06 smithi172 ceph-mon[30854]: pgmap v312: 129 pgs: 129 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.3 KiB/s wr, 23 op/s 2024-09-10T08:45:06.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:06 smithi174 ceph-mon[34068]: pgmap v312: 129 pgs: 129 active+clean; 607 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.3 KiB/s wr, 23 op/s 2024-09-10T08:45:06.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.743+0000 7f3a649d3640 1 -- 172.21.15.172:0/3885991989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a60106f10 msgr2=0x7f3a6010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:06.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.743+0000 7f3a649d3640 1 --2- 172.21.15.172:0/3885991989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a60106f10 0x7f3a6010ea90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3a58009ca0 tx=0x7f3a5802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.743+0000 7f3a649d3640 1 -- 172.21.15.172:0/3885991989 shutdown_connections 2024-09-10T08:45:06.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.743+0000 7f3a649d3640 1 --2- 172.21.15.172:0/3885991989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a60106f10 0x7f3a6010ea90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.743+0000 7f3a649d3640 1 --2- 172.21.15.172:0/3885991989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a60102790 0x7f3a601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.743+0000 7f3a649d3640 1 --2- 172.21.15.172:0/3885991989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a60101de0 0x7f3a601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.743+0000 7f3a649d3640 1 -- 172.21.15.172:0/3885991989 >> 172.21.15.172:0/3885991989 conn(0x7f3a600fd8c0 msgr2=0x7f3a600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:06.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.743+0000 7f3a649d3640 1 -- 172.21.15.172:0/3885991989 shutdown_connections 2024-09-10T08:45:06.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.743+0000 7f3a649d3640 1 -- 172.21.15.172:0/3885991989 wait complete. 2024-09-10T08:45:06.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.744+0000 7f3a649d3640 1 Processor -- start 2024-09-10T08:45:06.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.744+0000 7f3a649d3640 1 -- start start 2024-09-10T08:45:06.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.744+0000 7f3a649d3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a60101de0 0x7f3a60198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:06.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.744+0000 7f3a649d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a60102790 0x7f3a60198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:06.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.744+0000 7f3a649d3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a60106f10 0x7f3a6019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:06.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.744+0000 7f3a649d3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3a60111e20 con 0x7f3a60102790 2024-09-10T08:45:06.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.744+0000 7f3a649d3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a60111ca0 con 0x7f3a60101de0 2024-09-10T08:45:06.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.744+0000 7f3a649d3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3a60111fa0 con 0x7f3a60106f10 2024-09-10T08:45:06.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.744+0000 7f3a5dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a60102790 0x7f3a60198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:06.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.744+0000 7f3a5dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a60102790 0x7f3a60198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48356/0 (socket says 172.21.15.172:48356) 2024-09-10T08:45:06.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.744+0000 7f3a5dd9b640 1 -- 172.21.15.172:0/412625741 learned_addr learned my addr 172.21.15.172:0/412625741 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:06.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.744+0000 7f3a5e59c640 1 --2- 172.21.15.172:0/412625741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a60101de0 0x7f3a60198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:06.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.745+0000 7f3a5ed9d640 1 --2- 172.21.15.172:0/412625741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a60106f10 0x7f3a6019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:06.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.745+0000 7f3a5dd9b640 1 -- 172.21.15.172:0/412625741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a60106f10 msgr2=0x7f3a6019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:06.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.745+0000 7f3a5dd9b640 1 --2- 172.21.15.172:0/412625741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a60106f10 0x7f3a6019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.745+0000 7f3a5dd9b640 1 -- 172.21.15.172:0/412625741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a60101de0 msgr2=0x7f3a60198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:06.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.745+0000 7f3a5dd9b640 1 --2- 172.21.15.172:0/412625741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a60101de0 0x7f3a60198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.745+0000 7f3a5dd9b640 1 -- 172.21.15.172:0/412625741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6019d2d0 con 0x7f3a60102790 2024-09-10T08:45:06.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.745+0000 7f3a5dd9b640 1 --2- 172.21.15.172:0/412625741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a60102790 0x7f3a60198860 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000d950 tx=0x7f3a5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:06.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.745+0000 7f3a4f7fe640 1 -- 172.21.15.172:0/412625741 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a50017070 con 0x7f3a60102790 2024-09-10T08:45:06.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.745+0000 7f3a649d3640 1 -- 172.21.15.172:0/412625741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6019d5c0 con 0x7f3a60102790 2024-09-10T08:45:06.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.745+0000 7f3a4f7fe640 1 -- 172.21.15.172:0/412625741 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a50009c30 con 0x7f3a60102790 2024-09-10T08:45:06.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.745+0000 7f3a4f7fe640 1 -- 172.21.15.172:0/412625741 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5000f620 con 0x7f3a60102790 2024-09-10T08:45:06.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.745+0000 7f3a649d3640 1 -- 172.21.15.172:0/412625741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a601a4ea0 con 0x7f3a60102790 2024-09-10T08:45:06.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.746+0000 7f3a649d3640 1 -- 172.21.15.172:0/412625741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a601021c0 con 0x7f3a60102790 2024-09-10T08:45:06.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.748+0000 7f3a4f7fe640 1 -- 172.21.15.172:0/412625741 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3a500109a0 con 0x7f3a60102790 2024-09-10T08:45:06.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.749+0000 7f3a4f7fe640 1 --2- 172.21.15.172:0/412625741 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3a480a2c10 0x7f3a480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:06.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.750+0000 7f3a5e59c640 1 --2- 172.21.15.172:0/412625741 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3a480a2c10 0x7f3a480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:06.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.750+0000 7f3a4f7fe640 1 -- 172.21.15.172:0/412625741 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f3a500cef20 con 0x7f3a60102790 2024-09-10T08:45:06.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.750+0000 7f3a5e59c640 1 --2- 172.21.15.172:0/412625741 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3a480a2c10 0x7f3a480a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3a54005fd0 tx=0x7f3a5400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:06.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.752+0000 7f3a4f7fe640 1 -- 172.21.15.172:0/412625741 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a50009de0 con 0x7f3a60102790 2024-09-10T08:45:06.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.934+0000 7f3a649d3640 1 -- 172.21.15.172:0/412625741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a6002d7c0 con 0x7f3a60102790 2024-09-10T08:45:06.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.936+0000 7f3a4f7fe640 1 -- 172.21.15.172:0/412625741 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24294 (secure 0 0 0) 0x7f3a5009b490 con 0x7f3a60102790 2024-09-10T08:45:06.938 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:45:06.938 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":88,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","created":"2024-09-10T08:36:33.083529+0000","modified":"2024-09-10T08:44:20.275987+0000","last_up_change":"2024-09-10T08:43:19.615640+0000","last_in_change":"2024-09-10T08:42:56.656115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:39:31.328306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:44:07.466637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:44:09.520569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:44:15.926057+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6803","nonce":2678426952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6805","nonce":2678426952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6809","nonce":2678426952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6807","nonce":2678426952}]},"public_addr":"172.21.15.172:6803/2678426952","cluster_addr":"172.21.15.172:6805/2678426952","heartbeat_back_addr":"172.21.15.172:6809/2678426952","heartbeat_front_addr":"172.21.15.172:6807/2678426952","state":["exists","up"]},{"osd":1,"uuid":"55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6801","nonce":3279661439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6803","nonce":3279661439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6807","nonce":3279661439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6805","nonce":3279661439}]},"public_addr":"172.21.15.174:6801/3279661439","cluster_addr":"172.21.15.174:6803/3279661439","heartbeat_back_addr":"172.21.15.174:6807/3279661439","heartbeat_front_addr":"172.21.15.174:6805/3279661439","state":["exists","up"]},{"osd":2,"uuid":"f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6801","nonce":1963796290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6803","nonce":1963796290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6807","nonce":1963796290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6805","nonce":1963796290}]},"public_addr":"172.21.15.191:6801/1963796290","cluster_addr":"172.21.15.191:6803/1963796290","heartbeat_back_addr":"172.21.15.191:6807/1963796290","heartbeat_front_addr":"172.21.15.191:6805/1963796290","state":["exists","up"]},{"osd":3,"uuid":"15ecca3f-bc6b-4a06-b283-b515384beb19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6811","nonce":869546480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6813","nonce":869546480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6817","nonce":869546480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6815","nonce":869546480}]},"public_addr":"172.21.15.172:6811/869546480","cluster_addr":"172.21.15.172:6813/869546480","heartbeat_back_addr":"172.21.15.172:6817/869546480","heartbeat_front_addr":"172.21.15.172:6815/869546480","state":["exists","up"]},{"osd":4,"uuid":"8c2c1d40-72ad-4a15-ab72-54871c7e2427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6809","nonce":961007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6811","nonce":961007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6815","nonce":961007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6813","nonce":961007949}]},"public_addr":"172.21.15.174:6809/961007949","cluster_addr":"172.21.15.174:6811/961007949","heartbeat_back_addr":"172.21.15.174:6815/961007949","heartbeat_front_addr":"172.21.15.174:6813/961007949","state":["exists","up"]},{"osd":5,"uuid":"4de45a1d-ef0e-4bfe-ba15-5eeeb8806155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6809","nonce":4216938371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6811","nonce":4216938371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6815","nonce":4216938371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6813","nonce":4216938371}]},"public_addr":"172.21.15.191:6809/4216938371","cluster_addr":"172.21.15.191:6811/4216938371","heartbeat_back_addr":"172.21.15.191:6815/4216938371","heartbeat_front_addr":"172.21.15.191:6813/4216938371","state":["exists","up"]},{"osd":6,"uuid":"60a9938a-f696-4db2-932c-402997543b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6819","nonce":3210762009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6821","nonce":3210762009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6825","nonce":3210762009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6823","nonce":3210762009}]},"public_addr":"172.21.15.172:6819/3210762009","cluster_addr":"172.21.15.172:6821/3210762009","heartbeat_back_addr":"172.21.15.172:6825/3210762009","heartbeat_front_addr":"172.21.15.172:6823/3210762009","state":["exists","up"]},{"osd":7,"uuid":"7f0202f3-971e-472f-9c4d-ae3c24d39f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6817","nonce":1461923581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6819","nonce":1461923581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6823","nonce":1461923581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6821","nonce":1461923581}]},"public_addr":"172.21.15.174:6817/1461923581","cluster_addr":"172.21.15.174:6819/1461923581","heartbeat_back_addr":"172.21.15.174:6823/1461923581","heartbeat_front_addr":"172.21.15.174:6821/1461923581","state":["exists","up"]},{"osd":8,"uuid":"b4e7327f-ca9f-4833-9cd3-b8529b3d9761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6817","nonce":3675210430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6819","nonce":3675210430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6823","nonce":3675210430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6821","nonce":3675210430}]},"public_addr":"172.21.15.191:6817/3675210430","cluster_addr":"172.21.15.191:6819/3675210430","heartbeat_back_addr":"172.21.15.191:6823/3675210430","heartbeat_front_addr":"172.21.15.191:6821/3675210430","state":["exists","up"]},{"osd":9,"uuid":"9f6153f2-f9db-4c00-a659-c932628016f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6827","nonce":1181268619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6829","nonce":1181268619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6833","nonce":1181268619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6831","nonce":1181268619}]},"public_addr":"172.21.15.172:6827/1181268619","cluster_addr":"172.21.15.172:6829/1181268619","heartbeat_back_addr":"172.21.15.172:6833/1181268619","heartbeat_front_addr":"172.21.15.172:6831/1181268619","state":["exists","up"]},{"osd":10,"uuid":"040a013a-4c3e-442f-bea6-12cbe517716d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6825","nonce":1238812525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6827","nonce":1238812525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6831","nonce":1238812525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6829","nonce":1238812525}]},"public_addr":"172.21.15.174:6825/1238812525","cluster_addr":"172.21.15.174:6827/1238812525","heartbeat_back_addr":"172.21.15.174:6831/1238812525","heartbeat_front_addr":"172.21.15.174:6829/1238812525","state":["exists","up"]},{"osd":11,"uuid":"ae6535bd-1510-4bf3-bf4b-1a2a256add43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6825","nonce":2162719997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6827","nonce":2162719997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6831","nonce":2162719997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6829","nonce":2162719997}]},"public_addr":"172.21.15.191:6825/2162719997","cluster_addr":"172.21.15.191:6827/2162719997","heartbeat_back_addr":"172.21.15.191:6831/2162719997","heartbeat_front_addr":"172.21.15.191:6829/2162719997","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:38:38.812596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:04.765652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:29.103966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:56.085849+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:21.641784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:47.750376+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:11.640697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:36.638840+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:02.167446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:28.655804+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:52.749676+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:43:17.783908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:0/977245917":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/3919952878":"2024-09-11T08:36:54.887256+0000","172.21.15.172:6800/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/1525385361":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/2043423408":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/839344299":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6800/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/3251817286":"2024-09-11T08:36:54.887256+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:45:06.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.939+0000 7f3a649d3640 1 -- 172.21.15.172:0/412625741 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3a480a2c10 msgr2=0x7f3a480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:06.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.939+0000 7f3a649d3640 1 --2- 172.21.15.172:0/412625741 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3a480a2c10 0x7f3a480a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3a54005fd0 tx=0x7f3a5400b390 comp rx=0 tx=0).stop 2024-09-10T08:45:06.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.939+0000 7f3a649d3640 1 -- 172.21.15.172:0/412625741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a60102790 msgr2=0x7f3a60198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:06.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.939+0000 7f3a649d3640 1 --2- 172.21.15.172:0/412625741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a60102790 0x7f3a60198860 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000d950 tx=0x7f3a5000de10 comp rx=0 tx=0).stop 2024-09-10T08:45:06.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.939+0000 7f3a649d3640 1 -- 172.21.15.172:0/412625741 shutdown_connections 2024-09-10T08:45:06.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.939+0000 7f3a649d3640 1 --2- 172.21.15.172:0/412625741 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3a480a2c10 0x7f3a480a50d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.939+0000 7f3a649d3640 1 --2- 172.21.15.172:0/412625741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a60106f10 0x7f3a6019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.939+0000 7f3a649d3640 1 --2- 172.21.15.172:0/412625741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a60102790 0x7f3a60198860 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.939+0000 7f3a649d3640 1 --2- 172.21.15.172:0/412625741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a60101de0 0x7f3a60198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:06.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.939+0000 7f3a649d3640 1 -- 172.21.15.172:0/412625741 >> 172.21.15.172:0/412625741 conn(0x7f3a600fd8c0 msgr2=0x7f3a600fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:06.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.939+0000 7f3a649d3640 1 -- 172.21.15.172:0/412625741 shutdown_connections 2024-09-10T08:45:06.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:06.939+0000 7f3a649d3640 1 -- 172.21.15.172:0/412625741 wait complete. 2024-09-10T08:45:07.520 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:07 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:45:07.520 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:07 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/412625741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:45:07.520 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:07 smithi172 ceph-mon[30854]: pgmap v313: 129 pgs: 129 active+clean; 626 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-09-10T08:45:07.549 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T08:45:07.619 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:45:07.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:07 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:45:07.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:07 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/412625741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:45:07.621 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:07 smithi191 ceph-mon[35314]: pgmap v313: 129 pgs: 129 active+clean; 626 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-09-10T08:45:07.621 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:07 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:45:07.621 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:07 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/412625741' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:45:07.621 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:07 smithi174 ceph-mon[34068]: pgmap v313: 129 pgs: 129 active+clean; 626 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 KiB/s wr, 21 op/s 2024-09-10T08:45:08.545 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:09.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.024+0000 7fc693d0f640 1 -- 172.21.15.172:0/4174190772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc68c103660 msgr2=0x7fc68c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:09.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.024+0000 7fc693d0f640 1 --2- 172.21.15.172:0/4174190772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc68c103660 0x7fc68c10bb50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009900 tx=0x7fc67c02a480 comp rx=0 tx=0).stop 2024-09-10T08:45:09.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.025+0000 7fc693d0f640 1 -- 172.21.15.172:0/4174190772 shutdown_connections 2024-09-10T08:45:09.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.025+0000 7fc693d0f640 1 --2- 172.21.15.172:0/4174190772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc68c10c170 0x7fc68c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.025+0000 7fc693d0f640 1 --2- 172.21.15.172:0/4174190772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc68c103660 0x7fc68c10bb50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.025+0000 7fc693d0f640 1 --2- 172.21.15.172:0/4174190772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c102d40 0x7fc68c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.025+0000 7fc693d0f640 1 -- 172.21.15.172:0/4174190772 >> 172.21.15.172:0/4174190772 conn(0x7fc68c0fc7a0 msgr2=0x7fc68c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.025+0000 7fc693d0f640 1 -- 172.21.15.172:0/4174190772 shutdown_connections 2024-09-10T08:45:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.025+0000 7fc693d0f640 1 -- 172.21.15.172:0/4174190772 wait complete. 2024-09-10T08:45:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.025+0000 7fc693d0f640 1 Processor -- start 2024-09-10T08:45:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.025+0000 7fc693d0f640 1 -- start start 2024-09-10T08:45:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.025+0000 7fc693d0f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc68c102d40 0x7fc68c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.025+0000 7fc693d0f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c103660 0x7fc68c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.025+0000 7fc693d0f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc68c10c170 0x7fc68c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.025+0000 7fc693d0f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c111e20 con 0x7fc68c10c170 2024-09-10T08:45:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.025+0000 7fc693d0f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c111ca0 con 0x7fc68c103660 2024-09-10T08:45:09.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.025+0000 7fc693d0f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c111fa0 con 0x7fc68c102d40 2024-09-10T08:45:09.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc6922ac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc68c10c170 0x7fc68c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc691aab640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc68c102d40 0x7fc68c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc691aab640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc68c102d40 0x7fc68c198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:41160/0 (socket says 172.21.15.172:41160) 2024-09-10T08:45:09.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc691aab640 1 -- 172.21.15.172:0/645188410 learned_addr learned my addr 172.21.15.172:0/645188410 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:09.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc6922ac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc68c10c170 0x7fc68c19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48368/0 (socket says 172.21.15.172:48368) 2024-09-10T08:45:09.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc6912aa640 1 --2- 172.21.15.172:0/645188410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c103660 0x7fc68c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc6922ac640 1 -- 172.21.15.172:0/645188410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc68c102d40 msgr2=0x7fc68c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:09.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc6922ac640 1 --2- 172.21.15.172:0/645188410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc68c102d40 0x7fc68c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc6922ac640 1 -- 172.21.15.172:0/645188410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c103660 msgr2=0x7fc68c198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:09.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc6922ac640 1 --2- 172.21.15.172:0/645188410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c103660 0x7fc68c198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc6922ac640 1 -- 172.21.15.172:0/645188410 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68c19d2b0 con 0x7fc68c10c170 2024-09-10T08:45:09.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc691aab640 1 --2- 172.21.15.172:0/645188410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc68c102d40 0x7fc68c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:09.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc6922ac640 1 --2- 172.21.15.172:0/645188410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc68c10c170 0x7fc68c19cbd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc68400e940 tx=0x7fc68400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:09.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc682ffd640 1 -- 172.21.15.172:0/645188410 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684004d20 con 0x7fc68c10c170 2024-09-10T08:45:09.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc682ffd640 1 -- 172.21.15.172:0/645188410 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc684005020 con 0x7fc68c10c170 2024-09-10T08:45:09.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc693d0f640 1 -- 172.21.15.172:0/645188410 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68c19d5a0 con 0x7fc68c10c170 2024-09-10T08:45:09.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc682ffd640 1 -- 172.21.15.172:0/645188410 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6840116d0 con 0x7fc68c10c170 2024-09-10T08:45:09.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.026+0000 7fc693d0f640 1 -- 172.21.15.172:0/645188410 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68c1a4e80 con 0x7fc68c10c170 2024-09-10T08:45:09.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.027+0000 7fc693d0f640 1 -- 172.21.15.172:0/645188410 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc68c103120 con 0x7fc68c10c170 2024-09-10T08:45:09.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.028+0000 7fc682ffd640 1 -- 172.21.15.172:0/645188410 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc68400cb20 con 0x7fc68c10c170 2024-09-10T08:45:09.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.029+0000 7fc682ffd640 1 --2- 172.21.15.172:0/645188410 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc6780a2c10 0x7fc6780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:09.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.029+0000 7fc691aab640 1 --2- 172.21.15.172:0/645188410 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc6780a2c10 0x7fc6780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:09.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.029+0000 7fc682ffd640 1 -- 172.21.15.172:0/645188410 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fc6840cf450 con 0x7fc68c10c170 2024-09-10T08:45:09.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.030+0000 7fc691aab640 1 --2- 172.21.15.172:0/645188410 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc6780a2c10 0x7fc6780a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc68800f4d0 tx=0x7fc68800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:09.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.033+0000 7fc682ffd640 1 -- 172.21.15.172:0/645188410 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc684016030 con 0x7fc68c10c170 2024-09-10T08:45:09.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.222+0000 7fc693d0f640 1 -- 172.21.15.172:0/645188410 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc68c062630 con 0x7fc6780a2c10 2024-09-10T08:45:09.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.232+0000 7fc682ffd640 1 -- 172.21.15.172:0/645188410 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340379 (secure 0 0 0) 0x7fc68c062630 con 0x7fc6780a2c10 2024-09-10T08:45:09.238 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:45:09.240 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-09-10T08:45:09.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.235+0000 7fc693d0f640 1 -- 172.21.15.172:0/645188410 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc6780a2c10 msgr2=0x7fc6780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:09.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.235+0000 7fc693d0f640 1 --2- 172.21.15.172:0/645188410 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc6780a2c10 0x7fc6780a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc68800f4d0 tx=0x7fc68800a380 comp rx=0 tx=0).stop 2024-09-10T08:45:09.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.235+0000 7fc693d0f640 1 -- 172.21.15.172:0/645188410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc68c10c170 msgr2=0x7fc68c19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.235+0000 7fc693d0f640 1 --2- 172.21.15.172:0/645188410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc68c10c170 0x7fc68c19cbd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc68400e940 tx=0x7fc68400ee00 comp rx=0 tx=0).stop 2024-09-10T08:45:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.235+0000 7fc693d0f640 1 -- 172.21.15.172:0/645188410 shutdown_connections 2024-09-10T08:45:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.235+0000 7fc693d0f640 1 --2- 172.21.15.172:0/645188410 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc6780a2c10 0x7fc6780a50d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.235+0000 7fc693d0f640 1 --2- 172.21.15.172:0/645188410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc68c10c170 0x7fc68c19cbd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.235+0000 7fc693d0f640 1 --2- 172.21.15.172:0/645188410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c103660 0x7fc68c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.235+0000 7fc693d0f640 1 --2- 172.21.15.172:0/645188410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc68c102d40 0x7fc68c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.235+0000 7fc693d0f640 1 -- 172.21.15.172:0/645188410 >> 172.21.15.172:0/645188410 conn(0x7fc68c0fc7a0 msgr2=0x7fc68c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.235+0000 7fc693d0f640 1 -- 172.21.15.172:0/645188410 shutdown_connections 2024-09-10T08:45:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:09.235+0000 7fc693d0f640 1 -- 172.21.15.172:0/645188410 wait complete. 2024-09-10T08:45:09.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:09 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:09.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:09 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:09.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:09 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:09.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:09 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:09.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:09 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:09.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:09 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:09.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:09 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:09.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:09 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:09.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:09 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:09.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:09 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:09.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:09 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:09.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:09 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:09.937 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-10T08:45:09.131360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640762,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":285,"num_read_kb":246,"num_write":416,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375452,"kb_used_data":13724,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566372,"statfs":{"total":1151940427776,"available":1151555964928,"internally_reserved":0,"allocated":14053376,"data_stored":7734866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41410,"num_objects":16,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":123,"num_read_kb":106,"num_write":57,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003643"},"pg_stats":[{"pgid":"2.1f","version":"88'13","reported_seq":55,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:04.247638+0000","last_change":"2024-09-10T08:44:08.657737+0000","last_active":"2024-09-10T08:45:04.247638+0000","last_peered":"2024-09-10T08:45:04.247638+0000","last_clean":"2024-09-10T08:45:04.247638+0000","last_became_active":"2024-09-10T08:44:08.657473+0000","last_became_peered":"2024-09-10T08:44:08.657473+0000","last_unstale":"2024-09-10T08:45:04.247638+0000","last_undegraded":"2024-09-10T08:45:04.247638+0000","last_fullsized":"2024-09-10T08:45:04.247638+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:40.757343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990095+0000","last_change":"2024-09-10T08:44:10.675090+0000","last_active":"2024-09-10T08:44:20.990095+0000","last_peered":"2024-09-10T08:44:20.990095+0000","last_clean":"2024-09-10T08:44:20.990095+0000","last_became_active":"2024-09-10T08:44:10.674612+0000","last_became_peered":"2024-09-10T08:44:10.674612+0000","last_unstale":"2024-09-10T08:44:20.990095+0000","last_undegraded":"2024-09-10T08:44:20.990095+0000","last_fullsized":"2024-09-10T08:44:20.990095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:46.821080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717686+0000","last_change":"2024-09-10T08:44:17.721559+0000","last_active":"2024-09-10T08:44:20.717686+0000","last_peered":"2024-09-10T08:44:20.717686+0000","last_clean":"2024-09-10T08:44:20.717686+0000","last_became_active":"2024-09-10T08:44:17.721182+0000","last_became_peered":"2024-09-10T08:44:17.721182+0000","last_unstale":"2024-09-10T08:44:20.717686+0000","last_undegraded":"2024-09-10T08:44:20.717686+0000","last_fullsized":"2024-09-10T08:44:20.717686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:18.791489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990300+0000","last_change":"2024-09-10T08:44:09.049171+0000","last_active":"2024-09-10T08:44:20.990300+0000","last_peered":"2024-09-10T08:44:20.990300+0000","last_clean":"2024-09-10T08:44:20.990300+0000","last_became_active":"2024-09-10T08:44:09.048806+0000","last_became_peered":"2024-09-10T08:44:09.048806+0000","last_unstale":"2024-09-10T08:44:20.990300+0000","last_undegraded":"2024-09-10T08:44:20.990300+0000","last_fullsized":"2024-09-10T08:44:20.990300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:21.023460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990630+0000","last_change":"2024-09-10T08:44:10.677569+0000","last_active":"2024-09-10T08:44:20.990630+0000","last_peered":"2024-09-10T08:44:20.990630+0000","last_clean":"2024-09-10T08:44:20.990630+0000","last_became_active":"2024-09-10T08:44:10.677048+0000","last_became_peered":"2024-09-10T08:44:10.677048+0000","last_unstale":"2024-09-10T08:44:20.990630+0000","last_undegraded":"2024-09-10T08:44:20.990630+0000","last_fullsized":"2024-09-10T08:44:20.990630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:35.523063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990141+0000","last_change":"2024-09-10T08:44:17.720138+0000","last_active":"2024-09-10T08:44:20.990141+0000","last_peered":"2024-09-10T08:44:20.990141+0000","last_clean":"2024-09-10T08:44:20.990141+0000","last_became_active":"2024-09-10T08:44:17.719344+0000","last_became_peered":"2024-09-10T08:44:17.719344+0000","last_unstale":"2024-09-10T08:44:20.990141+0000","last_undegraded":"2024-09-10T08:44:20.990141+0000","last_fullsized":"2024-09-10T08:44:20.990141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:11.758058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:55.227631+0000","last_change":"2024-09-10T08:44:08.660099+0000","last_active":"2024-09-10T08:44:55.227631+0000","last_peered":"2024-09-10T08:44:55.227631+0000","last_clean":"2024-09-10T08:44:55.227631+0000","last_became_active":"2024-09-10T08:44:08.659745+0000","last_became_peered":"2024-09-10T08:44:08.659745+0000","last_unstale":"2024-09-10T08:44:55.227631+0000","last_undegraded":"2024-09-10T08:44:55.227631+0000","last_fullsized":"2024-09-10T08:44:55.227631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:43.960717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990344+0000","last_change":"2024-09-10T08:44:10.675583+0000","last_active":"2024-09-10T08:44:20.990344+0000","last_peered":"2024-09-10T08:44:20.990344+0000","last_clean":"2024-09-10T08:44:20.990344+0000","last_became_active":"2024-09-10T08:44:10.675296+0000","last_became_peered":"2024-09-10T08:44:10.675296+0000","last_unstale":"2024-09-10T08:44:20.990344+0000","last_undegraded":"2024-09-10T08:44:20.990344+0000","last_fullsized":"2024-09-10T08:44:20.990344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:31:51.147862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990657+0000","last_change":"2024-09-10T08:44:17.720104+0000","last_active":"2024-09-10T08:44:20.990657+0000","last_peered":"2024-09-10T08:44:20.990657+0000","last_clean":"2024-09-10T08:44:20.990657+0000","last_became_active":"2024-09-10T08:44:17.719607+0000","last_became_peered":"2024-09-10T08:44:17.719607+0000","last_unstale":"2024-09-10T08:44:20.990657+0000","last_undegraded":"2024-09-10T08:44:20.990657+0000","last_fullsized":"2024-09-10T08:44:20.990657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:52.976717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.233345+0000","last_change":"2024-09-10T08:44:08.658588+0000","last_active":"2024-09-10T08:44:54.233345+0000","last_peered":"2024-09-10T08:44:54.233345+0000","last_clean":"2024-09-10T08:44:54.233345+0000","last_became_active":"2024-09-10T08:44:08.658053+0000","last_became_peered":"2024-09-10T08:44:08.658053+0000","last_unstale":"2024-09-10T08:44:54.233345+0000","last_undegraded":"2024-09-10T08:44:54.233345+0000","last_fullsized":"2024-09-10T08:44:54.233345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:26.744943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991604+0000","last_change":"2024-09-10T08:44:10.681978+0000","last_active":"2024-09-10T08:44:20.991604+0000","last_peered":"2024-09-10T08:44:20.991604+0000","last_clean":"2024-09-10T08:44:20.991604+0000","last_became_active":"2024-09-10T08:44:10.681600+0000","last_became_peered":"2024-09-10T08:44:10.681600+0000","last_unstale":"2024-09-10T08:44:20.991604+0000","last_undegraded":"2024-09-10T08:44:20.991604+0000","last_fullsized":"2024-09-10T08:44:20.991604+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:37.190576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990299+0000","last_change":"2024-09-10T08:44:17.719784+0000","last_active":"2024-09-10T08:44:20.990299+0000","last_peered":"2024-09-10T08:44:20.990299+0000","last_clean":"2024-09-10T08:44:20.990299+0000","last_became_active":"2024-09-10T08:44:17.719182+0000","last_became_peered":"2024-09-10T08:44:17.719182+0000","last_unstale":"2024-09-10T08:44:20.990299+0000","last_undegraded":"2024-09-10T08:44:20.990299+0000","last_fullsized":"2024-09-10T08:44:20.990299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:15.237772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.250029+0000","last_change":"2024-09-10T08:44:10.679069+0000","last_active":"2024-09-10T08:44:21.250029+0000","last_peered":"2024-09-10T08:44:21.250029+0000","last_clean":"2024-09-10T08:44:21.250029+0000","last_became_active":"2024-09-10T08:44:10.678597+0000","last_became_peered":"2024-09-10T08:44:10.678597+0000","last_unstale":"2024-09-10T08:44:21.250029+0000","last_undegraded":"2024-09-10T08:44:21.250029+0000","last_fullsized":"2024-09-10T08:44:21.250029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:52.693906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:51.259703+0000","last_change":"2024-09-10T08:44:08.660886+0000","last_active":"2024-09-10T08:44:51.259703+0000","last_peered":"2024-09-10T08:44:51.259703+0000","last_clean":"2024-09-10T08:44:51.259703+0000","last_became_active":"2024-09-10T08:44:08.660478+0000","last_became_peered":"2024-09-10T08:44:08.660478+0000","last_unstale":"2024-09-10T08:44:51.259703+0000","last_undegraded":"2024-09-10T08:44:51.259703+0000","last_fullsized":"2024-09-10T08:44:51.259703+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:41.785531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990273+0000","last_change":"2024-09-10T08:44:17.761929+0000","last_active":"2024-09-10T08:44:20.990273+0000","last_peered":"2024-09-10T08:44:20.990273+0000","last_clean":"2024-09-10T08:44:20.990273+0000","last_became_active":"2024-09-10T08:44:17.761440+0000","last_became_peered":"2024-09-10T08:44:17.761440+0000","last_unstale":"2024-09-10T08:44:20.990273+0000","last_undegraded":"2024-09-10T08:44:20.990273+0000","last_fullsized":"2024-09-10T08:44:20.990273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:37.817759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989394+0000","last_change":"2024-09-10T08:44:10.674071+0000","last_active":"2024-09-10T08:44:20.989394+0000","last_peered":"2024-09-10T08:44:20.989394+0000","last_clean":"2024-09-10T08:44:20.989394+0000","last_became_active":"2024-09-10T08:44:10.673770+0000","last_became_peered":"2024-09-10T08:44:10.673770+0000","last_unstale":"2024-09-10T08:44:20.989394+0000","last_undegraded":"2024-09-10T08:44:20.989394+0000","last_fullsized":"2024-09-10T08:44:20.989394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:44:19.944557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:53.254468+0000","last_change":"2024-09-10T08:44:08.656636+0000","last_active":"2024-09-10T08:44:53.254468+0000","last_peered":"2024-09-10T08:44:53.254468+0000","last_clean":"2024-09-10T08:44:53.254468+0000","last_became_active":"2024-09-10T08:44:08.655813+0000","last_became_peered":"2024-09-10T08:44:08.655813+0000","last_unstale":"2024-09-10T08:44:53.254468+0000","last_undegraded":"2024-09-10T08:44:53.254468+0000","last_fullsized":"2024-09-10T08:44:53.254468+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:42.723053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990387+0000","last_change":"2024-09-10T08:44:17.722877+0000","last_active":"2024-09-10T08:44:20.990387+0000","last_peered":"2024-09-10T08:44:20.990387+0000","last_clean":"2024-09-10T08:44:20.990387+0000","last_became_active":"2024-09-10T08:44:17.722046+0000","last_became_peered":"2024-09-10T08:44:17.722046+0000","last_unstale":"2024-09-10T08:44:20.990387+0000","last_undegraded":"2024-09-10T08:44:20.990387+0000","last_fullsized":"2024-09-10T08:44:20.990387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:32.675598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990210+0000","last_change":"2024-09-10T08:44:10.671500+0000","last_active":"2024-09-10T08:44:20.990210+0000","last_peered":"2024-09-10T08:44:20.990210+0000","last_clean":"2024-09-10T08:44:20.990210+0000","last_became_active":"2024-09-10T08:44:10.671039+0000","last_became_peered":"2024-09-10T08:44:10.671039+0000","last_unstale":"2024-09-10T08:44:20.990210+0000","last_undegraded":"2024-09-10T08:44:20.990210+0000","last_fullsized":"2024-09-10T08:44:20.990210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:12.977373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.230652+0000","last_change":"2024-09-10T08:44:08.655467+0000","last_active":"2024-09-10T08:44:54.230652+0000","last_peered":"2024-09-10T08:44:54.230652+0000","last_clean":"2024-09-10T08:44:54.230652+0000","last_became_active":"2024-09-10T08:44:08.654991+0000","last_became_peered":"2024-09-10T08:44:08.654991+0000","last_unstale":"2024-09-10T08:44:54.230652+0000","last_undegraded":"2024-09-10T08:44:54.230652+0000","last_fullsized":"2024-09-10T08:44:54.230652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:33.124360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717671+0000","last_change":"2024-09-10T08:44:17.724250+0000","last_active":"2024-09-10T08:44:20.717671+0000","last_peered":"2024-09-10T08:44:20.717671+0000","last_clean":"2024-09-10T08:44:20.717671+0000","last_became_active":"2024-09-10T08:44:17.723733+0000","last_became_peered":"2024-09-10T08:44:17.723733+0000","last_unstale":"2024-09-10T08:44:20.717671+0000","last_undegraded":"2024-09-10T08:44:20.717671+0000","last_fullsized":"2024-09-10T08:44:20.717671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:40.098666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989901+0000","last_change":"2024-09-10T08:44:10.664159+0000","last_active":"2024-09-10T08:44:20.989901+0000","last_peered":"2024-09-10T08:44:20.989901+0000","last_clean":"2024-09-10T08:44:20.989901+0000","last_became_active":"2024-09-10T08:44:10.663674+0000","last_became_peered":"2024-09-10T08:44:10.663674+0000","last_unstale":"2024-09-10T08:44:20.989901+0000","last_undegraded":"2024-09-10T08:44:20.989901+0000","last_fullsized":"2024-09-10T08:44:20.989901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:24.234525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:51.258652+0000","last_change":"2024-09-10T08:44:09.050388+0000","last_active":"2024-09-10T08:44:51.258652+0000","last_peered":"2024-09-10T08:44:51.258652+0000","last_clean":"2024-09-10T08:44:51.258652+0000","last_became_active":"2024-09-10T08:44:09.049986+0000","last_became_peered":"2024-09-10T08:44:09.049986+0000","last_unstale":"2024-09-10T08:44:51.258652+0000","last_undegraded":"2024-09-10T08:44:51.258652+0000","last_fullsized":"2024-09-10T08:44:51.258652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:16:12.248883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990857+0000","last_change":"2024-09-10T08:44:17.761639+0000","last_active":"2024-09-10T08:44:20.990857+0000","last_peered":"2024-09-10T08:44:20.990857+0000","last_clean":"2024-09-10T08:44:20.990857+0000","last_became_active":"2024-09-10T08:44:17.761214+0000","last_became_peered":"2024-09-10T08:44:17.761214+0000","last_unstale":"2024-09-10T08:44:20.990857+0000","last_undegraded":"2024-09-10T08:44:20.990857+0000","last_fullsized":"2024-09-10T08:44:20.990857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:43.793195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990197+0000","last_change":"2024-09-10T08:44:10.672168+0000","last_active":"2024-09-10T08:44:20.990197+0000","last_peered":"2024-09-10T08:44:20.990197+0000","last_clean":"2024-09-10T08:44:20.990197+0000","last_became_active":"2024-09-10T08:44:10.671840+0000","last_became_peered":"2024-09-10T08:44:10.671840+0000","last_unstale":"2024-09-10T08:44:20.990197+0000","last_undegraded":"2024-09-10T08:44:20.990197+0000","last_fullsized":"2024-09-10T08:44:20.990197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:29.321232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.231076+0000","last_change":"2024-09-10T08:44:09.036632+0000","last_active":"2024-09-10T08:44:54.231076+0000","last_peered":"2024-09-10T08:44:54.231076+0000","last_clean":"2024-09-10T08:44:54.231076+0000","last_became_active":"2024-09-10T08:44:09.036254+0000","last_became_peered":"2024-09-10T08:44:09.036254+0000","last_unstale":"2024-09-10T08:44:54.231076+0000","last_undegraded":"2024-09-10T08:44:54.231076+0000","last_fullsized":"2024-09-10T08:44:54.231076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:59:40.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717686+0000","last_change":"2024-09-10T08:44:17.716215+0000","last_active":"2024-09-10T08:44:20.717686+0000","last_peered":"2024-09-10T08:44:20.717686+0000","last_clean":"2024-09-10T08:44:20.717686+0000","last_became_active":"2024-09-10T08:44:17.715809+0000","last_became_peered":"2024-09-10T08:44:17.715809+0000","last_unstale":"2024-09-10T08:44:20.717686+0000","last_undegraded":"2024-09-10T08:44:20.717686+0000","last_fullsized":"2024-09-10T08:44:20.717686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:52.896401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990018+0000","last_change":"2024-09-10T08:44:10.677871+0000","last_active":"2024-09-10T08:44:20.990018+0000","last_peered":"2024-09-10T08:44:20.990018+0000","last_clean":"2024-09-10T08:44:20.990018+0000","last_became_active":"2024-09-10T08:44:10.677101+0000","last_became_peered":"2024-09-10T08:44:10.677101+0000","last_unstale":"2024-09-10T08:44:20.990018+0000","last_undegraded":"2024-09-10T08:44:20.990018+0000","last_fullsized":"2024-09-10T08:44:20.990018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:46.137249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'12","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:02.690996+0000","last_change":"2024-09-10T08:44:08.654428+0000","last_active":"2024-09-10T08:45:02.690996+0000","last_peered":"2024-09-10T08:45:02.690996+0000","last_clean":"2024-09-10T08:45:02.690996+0000","last_became_active":"2024-09-10T08:44:08.654119+0000","last_became_peered":"2024-09-10T08:44:08.654119+0000","last_unstale":"2024-09-10T08:45:02.690996+0000","last_undegraded":"2024-09-10T08:45:02.690996+0000","last_fullsized":"2024-09-10T08:45:02.690996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:56.787721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989462+0000","last_change":"2024-09-10T08:44:17.760828+0000","last_active":"2024-09-10T08:44:20.989462+0000","last_peered":"2024-09-10T08:44:20.989462+0000","last_clean":"2024-09-10T08:44:20.989462+0000","last_became_active":"2024-09-10T08:44:17.760367+0000","last_became_peered":"2024-09-10T08:44:17.760367+0000","last_unstale":"2024-09-10T08:44:20.989462+0000","last_undegraded":"2024-09-10T08:44:20.989462+0000","last_fullsized":"2024-09-10T08:44:20.989462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:11.130480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991264+0000","last_change":"2024-09-10T08:44:10.670179+0000","last_active":"2024-09-10T08:44:20.991264+0000","last_peered":"2024-09-10T08:44:20.991264+0000","last_clean":"2024-09-10T08:44:20.991264+0000","last_became_active":"2024-09-10T08:44:10.669366+0000","last_became_peered":"2024-09-10T08:44:10.669366+0000","last_unstale":"2024-09-10T08:44:20.991264+0000","last_undegraded":"2024-09-10T08:44:20.991264+0000","last_fullsized":"2024-09-10T08:44:20.991264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:41.916365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:53.223607+0000","last_change":"2024-09-10T08:44:09.050441+0000","last_active":"2024-09-10T08:44:53.223607+0000","last_peered":"2024-09-10T08:44:53.223607+0000","last_clean":"2024-09-10T08:44:53.223607+0000","last_became_active":"2024-09-10T08:44:09.050070+0000","last_became_peered":"2024-09-10T08:44:09.050070+0000","last_unstale":"2024-09-10T08:44:53.223607+0000","last_undegraded":"2024-09-10T08:44:53.223607+0000","last_fullsized":"2024-09-10T08:44:53.223607+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:13.540021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990154+0000","last_change":"2024-09-10T08:44:17.720098+0000","last_active":"2024-09-10T08:44:20.990154+0000","last_peered":"2024-09-10T08:44:20.990154+0000","last_clean":"2024-09-10T08:44:20.990154+0000","last_became_active":"2024-09-10T08:44:17.719627+0000","last_became_peered":"2024-09-10T08:44:17.719627+0000","last_unstale":"2024-09-10T08:44:20.990154+0000","last_undegraded":"2024-09-10T08:44:20.990154+0000","last_fullsized":"2024-09-10T08:44:20.990154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:50.020919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717942+0000","last_change":"2024-09-10T08:44:10.669630+0000","last_active":"2024-09-10T08:44:20.717942+0000","last_peered":"2024-09-10T08:44:20.717942+0000","last_clean":"2024-09-10T08:44:20.717942+0000","last_became_active":"2024-09-10T08:44:10.669307+0000","last_became_peered":"2024-09-10T08:44:10.669307+0000","last_unstale":"2024-09-10T08:44:20.717942+0000","last_undegraded":"2024-09-10T08:44:20.717942+0000","last_fullsized":"2024-09-10T08:44:20.717942+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:06.320046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717501+0000","last_change":"2024-09-10T08:44:09.050715+0000","last_active":"2024-09-10T08:44:20.717501+0000","last_peered":"2024-09-10T08:44:20.717501+0000","last_clean":"2024-09-10T08:44:20.717501+0000","last_became_active":"2024-09-10T08:44:09.050347+0000","last_became_peered":"2024-09-10T08:44:09.050347+0000","last_unstale":"2024-09-10T08:44:20.717501+0000","last_undegraded":"2024-09-10T08:44:20.717501+0000","last_fullsized":"2024-09-10T08:44:20.717501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:18.689652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990046+0000","last_change":"2024-09-10T08:44:17.761503+0000","last_active":"2024-09-10T08:44:20.990046+0000","last_peered":"2024-09-10T08:44:20.990046+0000","last_clean":"2024-09-10T08:44:20.990046+0000","last_became_active":"2024-09-10T08:44:17.761042+0000","last_became_peered":"2024-09-10T08:44:17.761042+0000","last_unstale":"2024-09-10T08:44:20.990046+0000","last_undegraded":"2024-09-10T08:44:20.990046+0000","last_fullsized":"2024-09-10T08:44:20.990046+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:44.588850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.718217+0000","last_change":"2024-09-10T08:44:10.670881+0000","last_active":"2024-09-10T08:44:20.718217+0000","last_peered":"2024-09-10T08:44:20.718217+0000","last_clean":"2024-09-10T08:44:20.718217+0000","last_became_active":"2024-09-10T08:44:10.669791+0000","last_became_peered":"2024-09-10T08:44:10.669791+0000","last_unstale":"2024-09-10T08:44:20.718217+0000","last_undegraded":"2024-09-10T08:44:20.718217+0000","last_fullsized":"2024-09-10T08:44:20.718217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:33.644205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:07.252382+0000","last_change":"2024-09-10T08:44:08.657938+0000","last_active":"2024-09-10T08:45:07.252382+0000","last_peered":"2024-09-10T08:45:07.252382+0000","last_clean":"2024-09-10T08:45:07.252382+0000","last_became_active":"2024-09-10T08:44:08.657526+0000","last_became_peered":"2024-09-10T08:44:08.657526+0000","last_unstale":"2024-09-10T08:45:07.252382+0000","last_undegraded":"2024-09-10T08:45:07.252382+0000","last_fullsized":"2024-09-10T08:45:07.252382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:15.673561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990998+0000","last_change":"2024-09-10T08:44:17.721619+0000","last_active":"2024-09-10T08:44:20.990998+0000","last_peered":"2024-09-10T08:44:20.990998+0000","last_clean":"2024-09-10T08:44:20.990998+0000","last_became_active":"2024-09-10T08:44:17.721175+0000","last_became_peered":"2024-09-10T08:44:17.721175+0000","last_unstale":"2024-09-10T08:44:20.990998+0000","last_undegraded":"2024-09-10T08:44:20.990998+0000","last_fullsized":"2024-09-10T08:44:20.990998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:52.244120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990673+0000","last_change":"2024-09-10T08:44:10.676164+0000","last_active":"2024-09-10T08:44:20.990673+0000","last_peered":"2024-09-10T08:44:20.990673+0000","last_clean":"2024-09-10T08:44:20.990673+0000","last_became_active":"2024-09-10T08:44:10.675820+0000","last_became_peered":"2024-09-10T08:44:10.675820+0000","last_unstale":"2024-09-10T08:44:20.990673+0000","last_undegraded":"2024-09-10T08:44:20.990673+0000","last_fullsized":"2024-09-10T08:44:20.990673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:24.477584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:07.235617+0000","last_change":"2024-09-10T08:44:09.049124+0000","last_active":"2024-09-10T08:45:07.235617+0000","last_peered":"2024-09-10T08:45:07.235617+0000","last_clean":"2024-09-10T08:45:07.235617+0000","last_became_active":"2024-09-10T08:44:09.048799+0000","last_became_peered":"2024-09-10T08:44:09.048799+0000","last_unstale":"2024-09-10T08:45:07.235617+0000","last_undegraded":"2024-09-10T08:45:07.235617+0000","last_fullsized":"2024-09-10T08:45:07.235617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:43:39.849727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990940+0000","last_change":"2024-09-10T08:44:17.761130+0000","last_active":"2024-09-10T08:44:20.990940+0000","last_peered":"2024-09-10T08:44:20.990940+0000","last_clean":"2024-09-10T08:44:20.990940+0000","last_became_active":"2024-09-10T08:44:17.760753+0000","last_became_peered":"2024-09-10T08:44:17.760753+0000","last_unstale":"2024-09-10T08:44:20.990940+0000","last_undegraded":"2024-09-10T08:44:20.990940+0000","last_fullsized":"2024-09-10T08:44:20.990940+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:42.971962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990701+0000","last_change":"2024-09-10T08:44:10.669905+0000","last_active":"2024-09-10T08:44:20.990701+0000","last_peered":"2024-09-10T08:44:20.990701+0000","last_clean":"2024-09-10T08:44:20.990701+0000","last_became_active":"2024-09-10T08:44:10.669340+0000","last_became_peered":"2024-09-10T08:44:10.669340+0000","last_unstale":"2024-09-10T08:44:20.990701+0000","last_undegraded":"2024-09-10T08:44:20.990701+0000","last_fullsized":"2024-09-10T08:44:20.990701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:23.263179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:52.265666+0000","last_change":"2024-09-10T08:44:08.658862+0000","last_active":"2024-09-10T08:44:52.265666+0000","last_peered":"2024-09-10T08:44:52.265666+0000","last_clean":"2024-09-10T08:44:52.265666+0000","last_became_active":"2024-09-10T08:44:08.657880+0000","last_became_peered":"2024-09-10T08:44:08.657880+0000","last_unstale":"2024-09-10T08:44:52.265666+0000","last_undegraded":"2024-09-10T08:44:52.265666+0000","last_fullsized":"2024-09-10T08:44:52.265666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:34.224025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991130+0000","last_change":"2024-09-10T08:44:17.761517+0000","last_active":"2024-09-10T08:44:20.991130+0000","last_peered":"2024-09-10T08:44:20.991130+0000","last_clean":"2024-09-10T08:44:20.991130+0000","last_became_active":"2024-09-10T08:44:17.761081+0000","last_became_peered":"2024-09-10T08:44:17.761081+0000","last_unstale":"2024-09-10T08:44:20.991130+0000","last_undegraded":"2024-09-10T08:44:20.991130+0000","last_fullsized":"2024-09-10T08:44:20.991130+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:22.169689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989753+0000","last_change":"2024-09-10T08:44:10.673258+0000","last_active":"2024-09-10T08:44:20.989753+0000","last_peered":"2024-09-10T08:44:20.989753+0000","last_clean":"2024-09-10T08:44:20.989753+0000","last_became_active":"2024-09-10T08:44:10.672854+0000","last_became_peered":"2024-09-10T08:44:10.672854+0000","last_unstale":"2024-09-10T08:44:20.989753+0000","last_undegraded":"2024-09-10T08:44:20.989753+0000","last_fullsized":"2024-09-10T08:44:20.989753+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:35.913820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.232536+0000","last_change":"2024-09-10T08:44:08.658039+0000","last_active":"2024-09-10T08:44:54.232536+0000","last_peered":"2024-09-10T08:44:54.232536+0000","last_clean":"2024-09-10T08:44:54.232536+0000","last_became_active":"2024-09-10T08:44:08.657636+0000","last_became_peered":"2024-09-10T08:44:08.657636+0000","last_unstale":"2024-09-10T08:44:54.232536+0000","last_undegraded":"2024-09-10T08:44:54.232536+0000","last_fullsized":"2024-09-10T08:44:54.232536+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:09.615992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.249993+0000","last_change":"2024-09-10T08:42:57.666747+0000","last_active":"2024-09-10T08:44:21.249993+0000","last_peered":"2024-09-10T08:44:21.249993+0000","last_clean":"2024-09-10T08:44:21.249993+0000","last_became_active":"2024-09-10T08:42:57.666417+0000","last_became_peered":"2024-09-10T08:42:57.666417+0000","last_unstale":"2024-09-10T08:44:21.249993+0000","last_undegraded":"2024-09-10T08:44:21.249993+0000","last_fullsized":"2024-09-10T08:44:21.249993+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_clean_scrub_stamp":"2024-09-10T08:39:31.729794+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.547989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.250333+0000","last_change":"2024-09-10T08:44:17.723021+0000","last_active":"2024-09-10T08:44:21.250333+0000","last_peered":"2024-09-10T08:44:21.250333+0000","last_clean":"2024-09-10T08:44:21.250333+0000","last_became_active":"2024-09-10T08:44:17.722222+0000","last_became_peered":"2024-09-10T08:44:17.722222+0000","last_unstale":"2024-09-10T08:44:21.250333+0000","last_undegraded":"2024-09-10T08:44:21.250333+0000","last_fullsized":"2024-09-10T08:44:21.250333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:24.444427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.250381+0000","last_change":"2024-09-10T08:44:10.676651+0000","last_active":"2024-09-10T08:44:21.250381+0000","last_peered":"2024-09-10T08:44:21.250381+0000","last_clean":"2024-09-10T08:44:21.250381+0000","last_became_active":"2024-09-10T08:44:10.676342+0000","last_became_peered":"2024-09-10T08:44:10.676342+0000","last_unstale":"2024-09-10T08:44:21.250381+0000","last_undegraded":"2024-09-10T08:44:21.250381+0000","last_fullsized":"2024-09-10T08:44:21.250381+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:07.021273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:05.250329+0000","last_change":"2024-09-10T08:44:09.050305+0000","last_active":"2024-09-10T08:45:05.250329+0000","last_peered":"2024-09-10T08:45:05.250329+0000","last_clean":"2024-09-10T08:45:05.250329+0000","last_became_active":"2024-09-10T08:44:09.049958+0000","last_became_peered":"2024-09-10T08:44:09.049958+0000","last_unstale":"2024-09-10T08:45:05.250329+0000","last_undegraded":"2024-09-10T08:45:05.250329+0000","last_fullsized":"2024-09-10T08:45:05.250329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:42.219190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989775+0000","last_change":"2024-09-10T08:44:17.720219+0000","last_active":"2024-09-10T08:44:20.989775+0000","last_peered":"2024-09-10T08:44:20.989775+0000","last_clean":"2024-09-10T08:44:20.989775+0000","last_became_active":"2024-09-10T08:44:17.719756+0000","last_became_peered":"2024-09-10T08:44:17.719756+0000","last_unstale":"2024-09-10T08:44:20.989775+0000","last_undegraded":"2024-09-10T08:44:20.989775+0000","last_fullsized":"2024-09-10T08:44:20.989775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:01.070434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.250222+0000","last_change":"2024-09-10T08:44:10.678986+0000","last_active":"2024-09-10T08:44:21.250222+0000","last_peered":"2024-09-10T08:44:21.250222+0000","last_clean":"2024-09-10T08:44:21.250222+0000","last_became_active":"2024-09-10T08:44:10.678654+0000","last_became_peered":"2024-09-10T08:44:10.678654+0000","last_unstale":"2024-09-10T08:44:21.250222+0000","last_undegraded":"2024-09-10T08:44:21.250222+0000","last_fullsized":"2024-09-10T08:44:21.250222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:07.116802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:53.222413+0000","last_change":"2024-09-10T08:44:08.655025+0000","last_active":"2024-09-10T08:44:53.222413+0000","last_peered":"2024-09-10T08:44:53.222413+0000","last_clean":"2024-09-10T08:44:53.222413+0000","last_became_active":"2024-09-10T08:44:08.654728+0000","last_became_peered":"2024-09-10T08:44:08.654728+0000","last_unstale":"2024-09-10T08:44:53.222413+0000","last_undegraded":"2024-09-10T08:44:53.222413+0000","last_fullsized":"2024-09-10T08:44:53.222413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:31.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989360+0000","last_change":"2024-09-10T08:44:17.724226+0000","last_active":"2024-09-10T08:44:20.989360+0000","last_peered":"2024-09-10T08:44:20.989360+0000","last_clean":"2024-09-10T08:44:20.989360+0000","last_became_active":"2024-09-10T08:44:17.723927+0000","last_became_peered":"2024-09-10T08:44:17.723927+0000","last_unstale":"2024-09-10T08:44:20.989360+0000","last_undegraded":"2024-09-10T08:44:20.989360+0000","last_fullsized":"2024-09-10T08:44:20.989360+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:16.768671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.250297+0000","last_change":"2024-09-10T08:44:10.675072+0000","last_active":"2024-09-10T08:44:21.250297+0000","last_peered":"2024-09-10T08:44:21.250297+0000","last_clean":"2024-09-10T08:44:21.250297+0000","last_became_active":"2024-09-10T08:44:10.674636+0000","last_became_peered":"2024-09-10T08:44:10.674636+0000","last_unstale":"2024-09-10T08:44:21.250297+0000","last_undegraded":"2024-09-10T08:44:21.250297+0000","last_fullsized":"2024-09-10T08:44:21.250297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:55.198445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.230553+0000","last_change":"2024-09-10T08:44:08.656450+0000","last_active":"2024-09-10T08:44:54.230553+0000","last_peered":"2024-09-10T08:44:54.230553+0000","last_clean":"2024-09-10T08:44:54.230553+0000","last_became_active":"2024-09-10T08:44:08.655967+0000","last_became_peered":"2024-09-10T08:44:08.655967+0000","last_unstale":"2024-09-10T08:44:54.230553+0000","last_undegraded":"2024-09-10T08:44:54.230553+0000","last_fullsized":"2024-09-10T08:44:54.230553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:29.273167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990512+0000","last_change":"2024-09-10T08:44:17.722217+0000","last_active":"2024-09-10T08:44:20.990512+0000","last_peered":"2024-09-10T08:44:20.990512+0000","last_clean":"2024-09-10T08:44:20.990512+0000","last_became_active":"2024-09-10T08:44:17.721800+0000","last_became_peered":"2024-09-10T08:44:17.721800+0000","last_unstale":"2024-09-10T08:44:20.990512+0000","last_undegraded":"2024-09-10T08:44:20.990512+0000","last_fullsized":"2024-09-10T08:44:20.990512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:58.954397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991081+0000","last_change":"2024-09-10T08:44:10.676606+0000","last_active":"2024-09-10T08:44:20.991081+0000","last_peered":"2024-09-10T08:44:20.991081+0000","last_clean":"2024-09-10T08:44:20.991081+0000","last_became_active":"2024-09-10T08:44:10.676247+0000","last_became_peered":"2024-09-10T08:44:10.676247+0000","last_unstale":"2024-09-10T08:44:20.991081+0000","last_undegraded":"2024-09-10T08:44:20.991081+0000","last_fullsized":"2024-09-10T08:44:20.991081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:51.850917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:53.212635+0000","last_change":"2024-09-10T08:44:08.659003+0000","last_active":"2024-09-10T08:44:53.212635+0000","last_peered":"2024-09-10T08:44:53.212635+0000","last_clean":"2024-09-10T08:44:53.212635+0000","last_became_active":"2024-09-10T08:44:08.658324+0000","last_became_peered":"2024-09-10T08:44:08.658324+0000","last_unstale":"2024-09-10T08:44:53.212635+0000","last_undegraded":"2024-09-10T08:44:53.212635+0000","last_fullsized":"2024-09-10T08:44:53.212635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:44.743053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991074+0000","last_change":"2024-09-10T08:44:17.723363+0000","last_active":"2024-09-10T08:44:20.991074+0000","last_peered":"2024-09-10T08:44:20.991074+0000","last_clean":"2024-09-10T08:44:20.991074+0000","last_became_active":"2024-09-10T08:44:17.721799+0000","last_became_peered":"2024-09-10T08:44:17.721799+0000","last_unstale":"2024-09-10T08:44:20.991074+0000","last_undegraded":"2024-09-10T08:44:20.991074+0000","last_fullsized":"2024-09-10T08:44:20.991074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:08:28.772193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991225+0000","last_change":"2024-09-10T08:44:10.682952+0000","last_active":"2024-09-10T08:44:20.991225+0000","last_peered":"2024-09-10T08:44:20.991225+0000","last_clean":"2024-09-10T08:44:20.991225+0000","last_became_active":"2024-09-10T08:44:10.682249+0000","last_became_peered":"2024-09-10T08:44:10.682249+0000","last_unstale":"2024-09-10T08:44:20.991225+0000","last_undegraded":"2024-09-10T08:44:20.991225+0000","last_fullsized":"2024-09-10T08:44:20.991225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:32:21.285461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:53.223218+0000","last_change":"2024-09-10T08:44:09.050454+0000","last_active":"2024-09-10T08:44:53.223218+0000","last_peered":"2024-09-10T08:44:53.223218+0000","last_clean":"2024-09-10T08:44:53.223218+0000","last_became_active":"2024-09-10T08:44:09.050036+0000","last_became_peered":"2024-09-10T08:44:09.050036+0000","last_unstale":"2024-09-10T08:44:53.223218+0000","last_undegraded":"2024-09-10T08:44:53.223218+0000","last_fullsized":"2024-09-10T08:44:53.223218+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:55.141471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991151+0000","last_change":"2024-09-10T08:44:17.721203+0000","last_active":"2024-09-10T08:44:20.991151+0000","last_peered":"2024-09-10T08:44:20.991151+0000","last_clean":"2024-09-10T08:44:20.991151+0000","last_became_active":"2024-09-10T08:44:17.720737+0000","last_became_peered":"2024-09-10T08:44:17.720737+0000","last_unstale":"2024-09-10T08:44:20.991151+0000","last_undegraded":"2024-09-10T08:44:20.991151+0000","last_fullsized":"2024-09-10T08:44:20.991151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:32.081079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.249687+0000","last_change":"2024-09-10T08:44:10.680944+0000","last_active":"2024-09-10T08:44:21.249687+0000","last_peered":"2024-09-10T08:44:21.249687+0000","last_clean":"2024-09-10T08:44:21.249687+0000","last_became_active":"2024-09-10T08:44:10.680644+0000","last_became_peered":"2024-09-10T08:44:10.680644+0000","last_unstale":"2024-09-10T08:44:21.249687+0000","last_undegraded":"2024-09-10T08:44:21.249687+0000","last_fullsized":"2024-09-10T08:44:21.249687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:46.495894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:03.230735+0000","last_change":"2024-09-10T08:44:08.658135+0000","last_active":"2024-09-10T08:45:03.230735+0000","last_peered":"2024-09-10T08:45:03.230735+0000","last_clean":"2024-09-10T08:45:03.230735+0000","last_became_active":"2024-09-10T08:44:08.657793+0000","last_became_peered":"2024-09-10T08:44:08.657793+0000","last_unstale":"2024-09-10T08:45:03.230735+0000","last_undegraded":"2024-09-10T08:45:03.230735+0000","last_fullsized":"2024-09-10T08:45:03.230735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:16.376268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717723+0000","last_change":"2024-09-10T08:44:17.724342+0000","last_active":"2024-09-10T08:44:20.717723+0000","last_peered":"2024-09-10T08:44:20.717723+0000","last_clean":"2024-09-10T08:44:20.717723+0000","last_became_active":"2024-09-10T08:44:17.723958+0000","last_became_peered":"2024-09-10T08:44:17.723958+0000","last_unstale":"2024-09-10T08:44:20.717723+0000","last_undegraded":"2024-09-10T08:44:20.717723+0000","last_fullsized":"2024-09-10T08:44:20.717723+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:48.663820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.250127+0000","last_change":"2024-09-10T08:44:10.678254+0000","last_active":"2024-09-10T08:44:21.250127+0000","last_peered":"2024-09-10T08:44:21.250127+0000","last_clean":"2024-09-10T08:44:21.250127+0000","last_became_active":"2024-09-10T08:44:10.677891+0000","last_became_peered":"2024-09-10T08:44:10.677891+0000","last_unstale":"2024-09-10T08:44:21.250127+0000","last_undegraded":"2024-09-10T08:44:21.250127+0000","last_fullsized":"2024-09-10T08:44:21.250127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:32:43.399049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.232170+0000","last_change":"2024-09-10T08:44:08.656315+0000","last_active":"2024-09-10T08:44:54.232170+0000","last_peered":"2024-09-10T08:44:54.232170+0000","last_clean":"2024-09-10T08:44:54.232170+0000","last_became_active":"2024-09-10T08:44:08.655978+0000","last_became_peered":"2024-09-10T08:44:08.655978+0000","last_unstale":"2024-09-10T08:44:54.232170+0000","last_undegraded":"2024-09-10T08:44:54.232170+0000","last_fullsized":"2024-09-10T08:44:54.232170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:15.025354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990146+0000","last_change":"2024-09-10T08:44:17.726789+0000","last_active":"2024-09-10T08:44:20.990146+0000","last_peered":"2024-09-10T08:44:20.990146+0000","last_clean":"2024-09-10T08:44:20.990146+0000","last_became_active":"2024-09-10T08:44:17.726450+0000","last_became_peered":"2024-09-10T08:44:17.726450+0000","last_unstale":"2024-09-10T08:44:20.990146+0000","last_undegraded":"2024-09-10T08:44:20.990146+0000","last_fullsized":"2024-09-10T08:44:20.990146+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:03.899822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991128+0000","last_change":"2024-09-10T08:44:10.677359+0000","last_active":"2024-09-10T08:44:20.991128+0000","last_peered":"2024-09-10T08:44:20.991128+0000","last_clean":"2024-09-10T08:44:20.991128+0000","last_became_active":"2024-09-10T08:44:10.676782+0000","last_became_peered":"2024-09-10T08:44:10.676782+0000","last_unstale":"2024-09-10T08:44:20.991128+0000","last_undegraded":"2024-09-10T08:44:20.991128+0000","last_fullsized":"2024-09-10T08:44:20.991128+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:31.721714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.232479+0000","last_change":"2024-09-10T08:44:08.656993+0000","last_active":"2024-09-10T08:44:54.232479+0000","last_peered":"2024-09-10T08:44:54.232479+0000","last_clean":"2024-09-10T08:44:54.232479+0000","last_became_active":"2024-09-10T08:44:08.656571+0000","last_became_peered":"2024-09-10T08:44:08.656571+0000","last_unstale":"2024-09-10T08:44:54.232479+0000","last_undegraded":"2024-09-10T08:44:54.232479+0000","last_fullsized":"2024-09-10T08:44:54.232479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:32.202740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991605+0000","last_change":"2024-09-10T08:44:17.725674+0000","last_active":"2024-09-10T08:44:20.991605+0000","last_peered":"2024-09-10T08:44:20.991605+0000","last_clean":"2024-09-10T08:44:20.991605+0000","last_became_active":"2024-09-10T08:44:17.725263+0000","last_became_peered":"2024-09-10T08:44:17.725263+0000","last_unstale":"2024-09-10T08:44:20.991605+0000","last_undegraded":"2024-09-10T08:44:20.991605+0000","last_fullsized":"2024-09-10T08:44:20.991605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:18.183157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990960+0000","last_change":"2024-09-10T08:44:10.676871+0000","last_active":"2024-09-10T08:44:20.990960+0000","last_peered":"2024-09-10T08:44:20.990960+0000","last_clean":"2024-09-10T08:44:20.990960+0000","last_became_active":"2024-09-10T08:44:10.676540+0000","last_became_peered":"2024-09-10T08:44:10.676540+0000","last_unstale":"2024-09-10T08:44:20.990960+0000","last_undegraded":"2024-09-10T08:44:20.990960+0000","last_fullsized":"2024-09-10T08:44:20.990960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:50.964100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'27","reported_seq":86,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:04.232255+0000","last_change":"2024-09-10T08:44:08.660113+0000","last_active":"2024-09-10T08:45:04.232255+0000","last_peered":"2024-09-10T08:45:04.232255+0000","last_clean":"2024-09-10T08:45:04.232255+0000","last_became_active":"2024-09-10T08:44:08.659644+0000","last_became_peered":"2024-09-10T08:44:08.659644+0000","last_unstale":"2024-09-10T08:45:04.232255+0000","last_undegraded":"2024-09-10T08:45:04.232255+0000","last_fullsized":"2024-09-10T08:45:04.232255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:01.605344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":57,"num_write":29,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717369+0000","last_change":"2024-09-10T08:44:17.723653+0000","last_active":"2024-09-10T08:44:20.717369+0000","last_peered":"2024-09-10T08:44:20.717369+0000","last_clean":"2024-09-10T08:44:20.717369+0000","last_became_active":"2024-09-10T08:44:17.723284+0000","last_became_peered":"2024-09-10T08:44:17.723284+0000","last_unstale":"2024-09-10T08:44:20.717369+0000","last_undegraded":"2024-09-10T08:44:20.717369+0000","last_fullsized":"2024-09-10T08:44:20.717369+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:17.162526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990465+0000","last_change":"2024-09-10T08:44:10.681837+0000","last_active":"2024-09-10T08:44:20.990465+0000","last_peered":"2024-09-10T08:44:20.990465+0000","last_clean":"2024-09-10T08:44:20.990465+0000","last_became_active":"2024-09-10T08:44:10.681518+0000","last_became_peered":"2024-09-10T08:44:10.681518+0000","last_unstale":"2024-09-10T08:44:20.990465+0000","last_undegraded":"2024-09-10T08:44:20.990465+0000","last_fullsized":"2024-09-10T08:44:20.990465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:10.004215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:53.215205+0000","last_change":"2024-09-10T08:44:09.050394+0000","last_active":"2024-09-10T08:44:53.215205+0000","last_peered":"2024-09-10T08:44:53.215205+0000","last_clean":"2024-09-10T08:44:53.215205+0000","last_became_active":"2024-09-10T08:44:09.049969+0000","last_became_peered":"2024-09-10T08:44:09.049969+0000","last_unstale":"2024-09-10T08:44:53.215205+0000","last_undegraded":"2024-09-10T08:44:53.215205+0000","last_fullsized":"2024-09-10T08:44:53.215205+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:43.519246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990437+0000","last_change":"2024-09-10T08:44:17.720318+0000","last_active":"2024-09-10T08:44:20.990437+0000","last_peered":"2024-09-10T08:44:20.990437+0000","last_clean":"2024-09-10T08:44:20.990437+0000","last_became_active":"2024-09-10T08:44:17.719598+0000","last_became_peered":"2024-09-10T08:44:17.719598+0000","last_unstale":"2024-09-10T08:44:20.990437+0000","last_undegraded":"2024-09-10T08:44:20.990437+0000","last_fullsized":"2024-09-10T08:44:20.990437+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:15:49.520324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990679+0000","last_change":"2024-09-10T08:44:10.677340+0000","last_active":"2024-09-10T08:44:20.990679+0000","last_peered":"2024-09-10T08:44:20.990679+0000","last_clean":"2024-09-10T08:44:20.990679+0000","last_became_active":"2024-09-10T08:44:10.676628+0000","last_became_peered":"2024-09-10T08:44:10.676628+0000","last_unstale":"2024-09-10T08:44:20.990679+0000","last_undegraded":"2024-09-10T08:44:20.990679+0000","last_fullsized":"2024-09-10T08:44:20.990679+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:59:17.924217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.231825+0000","last_change":"2024-09-10T08:44:08.660254+0000","last_active":"2024-09-10T08:44:54.231825+0000","last_peered":"2024-09-10T08:44:54.231825+0000","last_clean":"2024-09-10T08:44:54.231825+0000","last_became_active":"2024-09-10T08:44:08.659667+0000","last_became_peered":"2024-09-10T08:44:08.659667+0000","last_unstale":"2024-09-10T08:44:54.231825+0000","last_undegraded":"2024-09-10T08:44:54.231825+0000","last_fullsized":"2024-09-10T08:44:54.231825+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:49:49.146867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717129+0000","last_change":"2024-09-10T08:44:17.722373+0000","last_active":"2024-09-10T08:44:20.717129+0000","last_peered":"2024-09-10T08:44:20.717129+0000","last_clean":"2024-09-10T08:44:20.717129+0000","last_became_active":"2024-09-10T08:44:17.722035+0000","last_became_peered":"2024-09-10T08:44:17.722035+0000","last_unstale":"2024-09-10T08:44:20.717129+0000","last_undegraded":"2024-09-10T08:44:20.717129+0000","last_fullsized":"2024-09-10T08:44:20.717129+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:30:36.187364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990791+0000","last_change":"2024-09-10T08:44:10.672585+0000","last_active":"2024-09-10T08:44:20.990791+0000","last_peered":"2024-09-10T08:44:20.990791+0000","last_clean":"2024-09-10T08:44:20.990791+0000","last_became_active":"2024-09-10T08:44:10.671987+0000","last_became_peered":"2024-09-10T08:44:10.671987+0000","last_unstale":"2024-09-10T08:44:20.990791+0000","last_undegraded":"2024-09-10T08:44:20.990791+0000","last_fullsized":"2024-09-10T08:44:20.990791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:48.660750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:04.230080+0000","last_change":"2024-09-10T08:44:08.658069+0000","last_active":"2024-09-10T08:45:04.230080+0000","last_peered":"2024-09-10T08:45:04.230080+0000","last_clean":"2024-09-10T08:45:04.230080+0000","last_became_active":"2024-09-10T08:44:08.657702+0000","last_became_peered":"2024-09-10T08:44:08.657702+0000","last_unstale":"2024-09-10T08:45:04.230080+0000","last_undegraded":"2024-09-10T08:45:04.230080+0000","last_fullsized":"2024-09-10T08:45:04.230080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:17.383267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990270+0000","last_change":"2024-09-10T08:44:17.717984+0000","last_active":"2024-09-10T08:44:20.990270+0000","last_peered":"2024-09-10T08:44:20.990270+0000","last_clean":"2024-09-10T08:44:20.990270+0000","last_became_active":"2024-09-10T08:44:17.717581+0000","last_became_peered":"2024-09-10T08:44:17.717581+0000","last_unstale":"2024-09-10T08:44:20.990270+0000","last_undegraded":"2024-09-10T08:44:20.990270+0000","last_fullsized":"2024-09-10T08:44:20.990270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:25.330732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989998+0000","last_change":"2024-09-10T08:44:17.721244+0000","last_active":"2024-09-10T08:44:20.989998+0000","last_peered":"2024-09-10T08:44:20.989998+0000","last_clean":"2024-09-10T08:44:20.989998+0000","last_became_active":"2024-09-10T08:44:17.720884+0000","last_became_peered":"2024-09-10T08:44:17.720884+0000","last_unstale":"2024-09-10T08:44:20.989998+0000","last_undegraded":"2024-09-10T08:44:20.989998+0000","last_fullsized":"2024-09-10T08:44:20.989998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.769513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717107+0000","last_change":"2024-09-10T08:44:10.667558+0000","last_active":"2024-09-10T08:44:20.717107+0000","last_peered":"2024-09-10T08:44:20.717107+0000","last_clean":"2024-09-10T08:44:20.717107+0000","last_became_active":"2024-09-10T08:44:10.667214+0000","last_became_peered":"2024-09-10T08:44:10.667214+0000","last_unstale":"2024-09-10T08:44:20.717107+0000","last_undegraded":"2024-09-10T08:44:20.717107+0000","last_fullsized":"2024-09-10T08:44:20.717107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:53.630790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.232150+0000","last_change":"2024-09-10T08:44:08.657930+0000","last_active":"2024-09-10T08:44:54.232150+0000","last_peered":"2024-09-10T08:44:54.232150+0000","last_clean":"2024-09-10T08:44:54.232150+0000","last_became_active":"2024-09-10T08:44:08.657611+0000","last_became_peered":"2024-09-10T08:44:08.657611+0000","last_unstale":"2024-09-10T08:44:54.232150+0000","last_undegraded":"2024-09-10T08:44:54.232150+0000","last_fullsized":"2024-09-10T08:44:54.232150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:36.923786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991280+0000","last_change":"2024-09-10T08:44:17.726106+0000","last_active":"2024-09-10T08:44:20.991280+0000","last_peered":"2024-09-10T08:44:20.991280+0000","last_clean":"2024-09-10T08:44:20.991280+0000","last_became_active":"2024-09-10T08:44:17.725683+0000","last_became_peered":"2024-09-10T08:44:17.725683+0000","last_unstale":"2024-09-10T08:44:20.991280+0000","last_undegraded":"2024-09-10T08:44:20.991280+0000","last_fullsized":"2024-09-10T08:44:20.991280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:23.392794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989651+0000","last_change":"2024-09-10T08:44:10.672204+0000","last_active":"2024-09-10T08:44:20.989651+0000","last_peered":"2024-09-10T08:44:20.989651+0000","last_clean":"2024-09-10T08:44:20.989651+0000","last_became_active":"2024-09-10T08:44:10.671898+0000","last_became_peered":"2024-09-10T08:44:10.671898+0000","last_unstale":"2024-09-10T08:44:20.989651+0000","last_undegraded":"2024-09-10T08:44:20.989651+0000","last_fullsized":"2024-09-10T08:44:20.989651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:03.737357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:52.265222+0000","last_change":"2024-09-10T08:44:08.656620+0000","last_active":"2024-09-10T08:44:52.265222+0000","last_peered":"2024-09-10T08:44:52.265222+0000","last_clean":"2024-09-10T08:44:52.265222+0000","last_became_active":"2024-09-10T08:44:08.656306+0000","last_became_peered":"2024-09-10T08:44:08.656306+0000","last_unstale":"2024-09-10T08:44:52.265222+0000","last_undegraded":"2024-09-10T08:44:52.265222+0000","last_fullsized":"2024-09-10T08:44:52.265222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:57.081909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:53.223364+0000","last_change":"2024-09-10T08:44:08.660178+0000","last_active":"2024-09-10T08:44:53.223364+0000","last_peered":"2024-09-10T08:44:53.223364+0000","last_clean":"2024-09-10T08:44:53.223364+0000","last_became_active":"2024-09-10T08:44:08.659839+0000","last_became_peered":"2024-09-10T08:44:08.659839+0000","last_unstale":"2024-09-10T08:44:53.223364+0000","last_undegraded":"2024-09-10T08:44:53.223364+0000","last_fullsized":"2024-09-10T08:44:53.223364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:17.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991349+0000","last_change":"2024-09-10T08:44:10.683683+0000","last_active":"2024-09-10T08:44:20.991349+0000","last_peered":"2024-09-10T08:44:20.991349+0000","last_clean":"2024-09-10T08:44:20.991349+0000","last_became_active":"2024-09-10T08:44:10.683337+0000","last_became_peered":"2024-09-10T08:44:10.683337+0000","last_unstale":"2024-09-10T08:44:20.991349+0000","last_undegraded":"2024-09-10T08:44:20.991349+0000","last_fullsized":"2024-09-10T08:44:20.991349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:54.863098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717904+0000","last_change":"2024-09-10T08:44:17.721602+0000","last_active":"2024-09-10T08:44:20.717904+0000","last_peered":"2024-09-10T08:44:20.717904+0000","last_clean":"2024-09-10T08:44:20.717904+0000","last_became_active":"2024-09-10T08:44:17.721194+0000","last_became_peered":"2024-09-10T08:44:17.721194+0000","last_unstale":"2024-09-10T08:44:20.717904+0000","last_undegraded":"2024-09-10T08:44:20.717904+0000","last_fullsized":"2024-09-10T08:44:20.717904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:39.298494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:03.229253+0000","last_change":"2024-09-10T08:44:08.661943+0000","last_active":"2024-09-10T08:45:03.229253+0000","last_peered":"2024-09-10T08:45:03.229253+0000","last_clean":"2024-09-10T08:45:03.229253+0000","last_became_active":"2024-09-10T08:44:08.661587+0000","last_became_peered":"2024-09-10T08:44:08.661587+0000","last_unstale":"2024-09-10T08:45:03.229253+0000","last_undegraded":"2024-09-10T08:45:03.229253+0000","last_fullsized":"2024-09-10T08:45:03.229253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:05.771641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990228+0000","last_change":"2024-09-10T08:44:10.673522+0000","last_active":"2024-09-10T08:44:20.990228+0000","last_peered":"2024-09-10T08:44:20.990228+0000","last_clean":"2024-09-10T08:44:20.990228+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:44:20.990228+0000","last_undegraded":"2024-09-10T08:44:20.990228+0000","last_fullsized":"2024-09-10T08:44:20.990228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:03.006144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.718054+0000","last_change":"2024-09-10T08:44:17.721306+0000","last_active":"2024-09-10T08:44:20.718054+0000","last_peered":"2024-09-10T08:44:20.718054+0000","last_clean":"2024-09-10T08:44:20.718054+0000","last_became_active":"2024-09-10T08:44:17.720401+0000","last_became_peered":"2024-09-10T08:44:17.720401+0000","last_unstale":"2024-09-10T08:44:20.718054+0000","last_undegraded":"2024-09-10T08:44:20.718054+0000","last_fullsized":"2024-09-10T08:44:20.718054+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:11.589371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990052+0000","last_change":"2024-09-10T08:44:10.678425+0000","last_active":"2024-09-10T08:44:20.990052+0000","last_peered":"2024-09-10T08:44:20.990052+0000","last_clean":"2024-09-10T08:44:20.990052+0000","last_became_active":"2024-09-10T08:44:10.677187+0000","last_became_peered":"2024-09-10T08:44:10.677187+0000","last_unstale":"2024-09-10T08:44:20.990052+0000","last_undegraded":"2024-09-10T08:44:20.990052+0000","last_fullsized":"2024-09-10T08:44:20.990052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:06.527530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.718245+0000","last_change":"2024-09-10T08:44:10.674178+0000","last_active":"2024-09-10T08:44:20.718245+0000","last_peered":"2024-09-10T08:44:20.718245+0000","last_clean":"2024-09-10T08:44:20.718245+0000","last_became_active":"2024-09-10T08:44:10.673843+0000","last_became_peered":"2024-09-10T08:44:10.673843+0000","last_unstale":"2024-09-10T08:44:20.718245+0000","last_undegraded":"2024-09-10T08:44:20.718245+0000","last_fullsized":"2024-09-10T08:44:20.718245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:35.023663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717659+0000","last_change":"2024-09-10T08:44:10.673271+0000","last_active":"2024-09-10T08:44:20.717659+0000","last_peered":"2024-09-10T08:44:20.717659+0000","last_clean":"2024-09-10T08:44:20.717659+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:44:20.717659+0000","last_undegraded":"2024-09-10T08:44:20.717659+0000","last_fullsized":"2024-09-10T08:44:20.717659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:21:26.606820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989465+0000","last_change":"2024-09-10T08:44:10.671348+0000","last_active":"2024-09-10T08:44:20.989465+0000","last_peered":"2024-09-10T08:44:20.989465+0000","last_clean":"2024-09-10T08:44:20.989465+0000","last_became_active":"2024-09-10T08:44:10.670623+0000","last_became_peered":"2024-09-10T08:44:10.670623+0000","last_unstale":"2024-09-10T08:44:20.989465+0000","last_undegraded":"2024-09-10T08:44:20.989465+0000","last_fullsized":"2024-09-10T08:44:20.989465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:06.145694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991259+0000","last_change":"2024-09-10T08:44:10.666011+0000","last_active":"2024-09-10T08:44:20.991259+0000","last_peered":"2024-09-10T08:44:20.991259+0000","last_clean":"2024-09-10T08:44:20.991259+0000","last_became_active":"2024-09-10T08:44:10.665664+0000","last_became_peered":"2024-09-10T08:44:10.665664+0000","last_unstale":"2024-09-10T08:44:20.991259+0000","last_undegraded":"2024-09-10T08:44:20.991259+0000","last_fullsized":"2024-09-10T08:44:20.991259+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:31.542102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990749+0000","last_change":"2024-09-10T08:44:10.675960+0000","last_active":"2024-09-10T08:44:20.990749+0000","last_peered":"2024-09-10T08:44:20.990749+0000","last_clean":"2024-09-10T08:44:20.990749+0000","last_became_active":"2024-09-10T08:44:10.675587+0000","last_became_peered":"2024-09-10T08:44:10.675587+0000","last_unstale":"2024-09-10T08:44:20.990749+0000","last_undegraded":"2024-09-10T08:44:20.990749+0000","last_fullsized":"2024-09-10T08:44:20.990749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:54.634619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.718259+0000","last_change":"2024-09-10T08:44:10.681094+0000","last_active":"2024-09-10T08:44:20.718259+0000","last_peered":"2024-09-10T08:44:20.718259+0000","last_clean":"2024-09-10T08:44:20.718259+0000","last_became_active":"2024-09-10T08:44:10.678585+0000","last_became_peered":"2024-09-10T08:44:10.678585+0000","last_unstale":"2024-09-10T08:44:20.718259+0000","last_undegraded":"2024-09-10T08:44:20.718259+0000","last_fullsized":"2024-09-10T08:44:20.718259+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:14.951949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989971+0000","last_change":"2024-09-10T08:44:10.675473+0000","last_active":"2024-09-10T08:44:20.989971+0000","last_peered":"2024-09-10T08:44:20.989971+0000","last_clean":"2024-09-10T08:44:20.989971+0000","last_became_active":"2024-09-10T08:44:10.674155+0000","last_became_peered":"2024-09-10T08:44:10.674155+0000","last_unstale":"2024-09-10T08:44:20.989971+0000","last_undegraded":"2024-09-10T08:44:20.989971+0000","last_fullsized":"2024-09-10T08:44:20.989971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:55.940235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990620+0000","last_change":"2024-09-10T08:44:10.680162+0000","last_active":"2024-09-10T08:44:20.990620+0000","last_peered":"2024-09-10T08:44:20.990620+0000","last_clean":"2024-09-10T08:44:20.990620+0000","last_became_active":"2024-09-10T08:44:10.679800+0000","last_became_peered":"2024-09-10T08:44:10.679800+0000","last_unstale":"2024-09-10T08:44:20.990620+0000","last_undegraded":"2024-09-10T08:44:20.990620+0000","last_fullsized":"2024-09-10T08:44:20.990620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:12.319813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990420+0000","last_change":"2024-09-10T08:44:10.674724+0000","last_active":"2024-09-10T08:44:20.990420+0000","last_peered":"2024-09-10T08:44:20.990420+0000","last_clean":"2024-09-10T08:44:20.990420+0000","last_became_active":"2024-09-10T08:44:10.674389+0000","last_became_peered":"2024-09-10T08:44:10.674389+0000","last_unstale":"2024-09-10T08:44:20.990420+0000","last_undegraded":"2024-09-10T08:44:20.990420+0000","last_fullsized":"2024-09-10T08:44:20.990420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:52.123186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989869+0000","last_change":"2024-09-10T08:44:10.672805+0000","last_active":"2024-09-10T08:44:20.989869+0000","last_peered":"2024-09-10T08:44:20.989869+0000","last_clean":"2024-09-10T08:44:20.989869+0000","last_became_active":"2024-09-10T08:44:10.672508+0000","last_became_peered":"2024-09-10T08:44:10.672508+0000","last_unstale":"2024-09-10T08:44:20.989869+0000","last_undegraded":"2024-09-10T08:44:20.989869+0000","last_fullsized":"2024-09-10T08:44:20.989869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:48.558633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990759+0000","last_change":"2024-09-10T08:44:10.677504+0000","last_active":"2024-09-10T08:44:20.990759+0000","last_peered":"2024-09-10T08:44:20.990759+0000","last_clean":"2024-09-10T08:44:20.990759+0000","last_became_active":"2024-09-10T08:44:10.676822+0000","last_became_peered":"2024-09-10T08:44:10.676822+0000","last_unstale":"2024-09-10T08:44:20.990759+0000","last_undegraded":"2024-09-10T08:44:20.990759+0000","last_fullsized":"2024-09-10T08:44:20.990759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:05:48.293333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990075+0000","last_change":"2024-09-10T08:44:10.673181+0000","last_active":"2024-09-10T08:44:20.990075+0000","last_peered":"2024-09-10T08:44:20.990075+0000","last_clean":"2024-09-10T08:44:20.990075+0000","last_became_active":"2024-09-10T08:44:10.672851+0000","last_became_peered":"2024-09-10T08:44:10.672851+0000","last_unstale":"2024-09-10T08:44:20.990075+0000","last_undegraded":"2024-09-10T08:44:20.990075+0000","last_fullsized":"2024-09-10T08:44:20.990075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:42.562229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.718225+0000","last_change":"2024-09-10T08:44:10.670615+0000","last_active":"2024-09-10T08:44:20.718225+0000","last_peered":"2024-09-10T08:44:20.718225+0000","last_clean":"2024-09-10T08:44:20.718225+0000","last_became_active":"2024-09-10T08:44:10.670032+0000","last_became_peered":"2024-09-10T08:44:10.670032+0000","last_unstale":"2024-09-10T08:44:20.718225+0000","last_undegraded":"2024-09-10T08:44:20.718225+0000","last_fullsized":"2024-09-10T08:44:20.718225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:31.078788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717634+0000","last_change":"2024-09-10T08:44:10.672367+0000","last_active":"2024-09-10T08:44:20.717634+0000","last_peered":"2024-09-10T08:44:20.717634+0000","last_clean":"2024-09-10T08:44:20.717634+0000","last_became_active":"2024-09-10T08:44:10.672040+0000","last_became_peered":"2024-09-10T08:44:10.672040+0000","last_unstale":"2024-09-10T08:44:20.717634+0000","last_undegraded":"2024-09-10T08:44:20.717634+0000","last_fullsized":"2024-09-10T08:44:20.717634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:10.634708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991593+0000","last_change":"2024-09-10T08:44:10.676375+0000","last_active":"2024-09-10T08:44:20.991593+0000","last_peered":"2024-09-10T08:44:20.991593+0000","last_clean":"2024-09-10T08:44:20.991593+0000","last_became_active":"2024-09-10T08:44:10.676022+0000","last_became_peered":"2024-09-10T08:44:10.676022+0000","last_unstale":"2024-09-10T08:44:20.991593+0000","last_undegraded":"2024-09-10T08:44:20.991593+0000","last_fullsized":"2024-09-10T08:44:20.991593+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:02.681954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989558+0000","last_change":"2024-09-10T08:44:10.673378+0000","last_active":"2024-09-10T08:44:20.989558+0000","last_peered":"2024-09-10T08:44:20.989558+0000","last_clean":"2024-09-10T08:44:20.989558+0000","last_became_active":"2024-09-10T08:44:10.673089+0000","last_became_peered":"2024-09-10T08:44:10.673089+0000","last_unstale":"2024-09-10T08:44:20.989558+0000","last_undegraded":"2024-09-10T08:44:20.989558+0000","last_fullsized":"2024-09-10T08:44:20.989558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:36:19.258651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989429+0000","last_change":"2024-09-10T08:44:10.671557+0000","last_active":"2024-09-10T08:44:20.989429+0000","last_peered":"2024-09-10T08:44:20.989429+0000","last_clean":"2024-09-10T08:44:20.989429+0000","last_became_active":"2024-09-10T08:44:10.671248+0000","last_became_peered":"2024-09-10T08:44:10.671248+0000","last_unstale":"2024-09-10T08:44:20.989429+0000","last_undegraded":"2024-09-10T08:44:20.989429+0000","last_fullsized":"2024-09-10T08:44:20.989429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:54.261525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989628+0000","last_change":"2024-09-10T08:44:10.674672+0000","last_active":"2024-09-10T08:44:20.989628+0000","last_peered":"2024-09-10T08:44:20.989628+0000","last_clean":"2024-09-10T08:44:20.989628+0000","last_became_active":"2024-09-10T08:44:10.673415+0000","last_became_peered":"2024-09-10T08:44:10.673415+0000","last_unstale":"2024-09-10T08:44:20.989628+0000","last_undegraded":"2024-09-10T08:44:20.989628+0000","last_fullsized":"2024-09-10T08:44:20.989628+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:59:14.179664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990654+0000","last_change":"2024-09-10T08:44:10.675293+0000","last_active":"2024-09-10T08:44:20.990654+0000","last_peered":"2024-09-10T08:44:20.990654+0000","last_clean":"2024-09-10T08:44:20.990654+0000","last_became_active":"2024-09-10T08:44:10.674924+0000","last_became_peered":"2024-09-10T08:44:10.674924+0000","last_unstale":"2024-09-10T08:44:20.990654+0000","last_undegraded":"2024-09-10T08:44:20.990654+0000","last_fullsized":"2024-09-10T08:44:20.990654+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:56.027320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990608+0000","last_change":"2024-09-10T08:44:10.674605+0000","last_active":"2024-09-10T08:44:20.990608+0000","last_peered":"2024-09-10T08:44:20.990608+0000","last_clean":"2024-09-10T08:44:20.990608+0000","last_became_active":"2024-09-10T08:44:10.674057+0000","last_became_peered":"2024-09-10T08:44:10.674057+0000","last_unstale":"2024-09-10T08:44:20.990608+0000","last_undegraded":"2024-09-10T08:44:20.990608+0000","last_fullsized":"2024-09-10T08:44:20.990608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:09.473003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.250288+0000","last_change":"2024-09-10T08:44:10.677410+0000","last_active":"2024-09-10T08:44:21.250288+0000","last_peered":"2024-09-10T08:44:21.250288+0000","last_clean":"2024-09-10T08:44:21.250288+0000","last_became_active":"2024-09-10T08:44:10.677109+0000","last_became_peered":"2024-09-10T08:44:10.677109+0000","last_unstale":"2024-09-10T08:44:21.250288+0000","last_undegraded":"2024-09-10T08:44:21.250288+0000","last_fullsized":"2024-09-10T08:44:21.250288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:45.455425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.718277+0000","last_change":"2024-09-10T08:44:10.669706+0000","last_active":"2024-09-10T08:44:20.718277+0000","last_peered":"2024-09-10T08:44:20.718277+0000","last_clean":"2024-09-10T08:44:20.718277+0000","last_became_active":"2024-09-10T08:44:10.669440+0000","last_became_peered":"2024-09-10T08:44:10.669440+0000","last_unstale":"2024-09-10T08:44:20.718277+0000","last_undegraded":"2024-09-10T08:44:20.718277+0000","last_fullsized":"2024-09-10T08:44:20.718277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:31:55.610651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990892+0000","last_change":"2024-09-10T08:44:10.676482+0000","last_active":"2024-09-10T08:44:20.990892+0000","last_peered":"2024-09-10T08:44:20.990892+0000","last_clean":"2024-09-10T08:44:20.990892+0000","last_became_active":"2024-09-10T08:44:10.675983+0000","last_became_peered":"2024-09-10T08:44:10.675983+0000","last_unstale":"2024-09-10T08:44:20.990892+0000","last_undegraded":"2024-09-10T08:44:20.990892+0000","last_fullsized":"2024-09-10T08:44:20.990892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:01.269283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990130+0000","last_change":"2024-09-10T08:44:10.675681+0000","last_active":"2024-09-10T08:44:20.990130+0000","last_peered":"2024-09-10T08:44:20.990130+0000","last_clean":"2024-09-10T08:44:20.990130+0000","last_became_active":"2024-09-10T08:44:10.675357+0000","last_became_peered":"2024-09-10T08:44:10.675357+0000","last_unstale":"2024-09-10T08:44:20.990130+0000","last_undegraded":"2024-09-10T08:44:20.990130+0000","last_fullsized":"2024-09-10T08:44:20.990130+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:26.138333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.249812+0000","last_change":"2024-09-10T08:44:10.680819+0000","last_active":"2024-09-10T08:44:21.249812+0000","last_peered":"2024-09-10T08:44:21.249812+0000","last_clean":"2024-09-10T08:44:21.249812+0000","last_became_active":"2024-09-10T08:44:10.680491+0000","last_became_peered":"2024-09-10T08:44:10.680491+0000","last_unstale":"2024-09-10T08:44:21.249812+0000","last_undegraded":"2024-09-10T08:44:21.249812+0000","last_fullsized":"2024-09-10T08:44:21.249812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:17.044776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717783+0000","last_change":"2024-09-10T08:44:10.667376+0000","last_active":"2024-09-10T08:44:20.717783+0000","last_peered":"2024-09-10T08:44:20.717783+0000","last_clean":"2024-09-10T08:44:20.717783+0000","last_became_active":"2024-09-10T08:44:10.666648+0000","last_became_peered":"2024-09-10T08:44:10.666648+0000","last_unstale":"2024-09-10T08:44:20.717783+0000","last_undegraded":"2024-09-10T08:44:20.717783+0000","last_fullsized":"2024-09-10T08:44:20.717783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:33:12.017550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989959+0000","last_change":"2024-09-10T08:44:10.668665+0000","last_active":"2024-09-10T08:44:20.989959+0000","last_peered":"2024-09-10T08:44:20.989959+0000","last_clean":"2024-09-10T08:44:20.989959+0000","last_became_active":"2024-09-10T08:44:10.666031+0000","last_became_peered":"2024-09-10T08:44:10.666031+0000","last_unstale":"2024-09-10T08:44:20.989959+0000","last_undegraded":"2024-09-10T08:44:20.989959+0000","last_fullsized":"2024-09-10T08:44:20.989959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:39:53.991463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990505+0000","last_change":"2024-09-10T08:44:10.676495+0000","last_active":"2024-09-10T08:44:20.990505+0000","last_peered":"2024-09-10T08:44:20.990505+0000","last_clean":"2024-09-10T08:44:20.990505+0000","last_became_active":"2024-09-10T08:44:10.676087+0000","last_became_peered":"2024-09-10T08:44:10.676087+0000","last_unstale":"2024-09-10T08:44:20.990505+0000","last_undegraded":"2024-09-10T08:44:20.990505+0000","last_fullsized":"2024-09-10T08:44:20.990505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:41.828157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989398+0000","last_change":"2024-09-10T08:44:10.672835+0000","last_active":"2024-09-10T08:44:20.989398+0000","last_peered":"2024-09-10T08:44:20.989398+0000","last_clean":"2024-09-10T08:44:20.989398+0000","last_became_active":"2024-09-10T08:44:10.672537+0000","last_became_peered":"2024-09-10T08:44:10.672537+0000","last_unstale":"2024-09-10T08:44:20.989398+0000","last_undegraded":"2024-09-10T08:44:20.989398+0000","last_fullsized":"2024-09-10T08:44:20.989398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:06.121460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990108+0000","last_change":"2024-09-10T08:44:10.675968+0000","last_active":"2024-09-10T08:44:20.990108+0000","last_peered":"2024-09-10T08:44:20.990108+0000","last_clean":"2024-09-10T08:44:20.990108+0000","last_became_active":"2024-09-10T08:44:10.675628+0000","last_became_peered":"2024-09-10T08:44:10.675628+0000","last_unstale":"2024-09-10T08:44:20.990108+0000","last_undegraded":"2024-09-10T08:44:20.990108+0000","last_fullsized":"2024-09-10T08:44:20.990108+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:40.261336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991537+0000","last_change":"2024-09-10T08:44:10.673151+0000","last_active":"2024-09-10T08:44:20.991537+0000","last_peered":"2024-09-10T08:44:20.991537+0000","last_clean":"2024-09-10T08:44:20.991537+0000","last_became_active":"2024-09-10T08:44:10.672833+0000","last_became_peered":"2024-09-10T08:44:10.672833+0000","last_unstale":"2024-09-10T08:44:20.991537+0000","last_undegraded":"2024-09-10T08:44:20.991537+0000","last_fullsized":"2024-09-10T08:44:20.991537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:38.556222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50160,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":159,"num_read_kb":137,"num_write":233,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":176118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514520,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":485254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743453,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1667072,"data_stored":1115022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906978,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":488887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":485169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1081275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":480295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36924,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708228,"statfs":{"total":95995035648,"available":95957225472,"internally_reserved":0,"allocated":1617920,"data_stored":1076857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920827,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":484369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":480003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":538102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":537489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":482144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":57916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":45081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:45:09.943 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:45:10.208 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:10.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:10 smithi174 ceph-mon[34068]: pgmap v314: 129 pgs: 129 active+clean; 626 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.1 KiB/s wr, 14 op/s 2024-09-10T08:45:10.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:10 smithi174 ceph-mon[34068]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:45:10.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:10 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:10.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:10 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:10.500 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:10 smithi172 ceph-mon[30854]: pgmap v314: 129 pgs: 129 active+clean; 626 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.1 KiB/s wr, 14 op/s 2024-09-10T08:45:10.500 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:10 smithi172 ceph-mon[30854]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:45:10.500 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:10 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:10.501 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:10 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:10.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:10 smithi191 ceph-mon[35314]: pgmap v314: 129 pgs: 129 active+clean; 626 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.1 KiB/s wr, 14 op/s 2024-09-10T08:45:10.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:10 smithi191 ceph-mon[35314]: from='client.14847 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:45:10.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:10 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:10.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:10 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:10.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.723+0000 7f6e36ec8640 1 -- 172.21.15.172:0/821860503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e30102d90 msgr2=0x7f6e30103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:10.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.723+0000 7f6e36ec8640 1 --2- 172.21.15.172:0/821860503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e30102d90 0x7f6e30103170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009930 tx=0x7f6e2402a420 comp rx=0 tx=0).stop 2024-09-10T08:45:10.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.724+0000 7f6e36ec8640 1 -- 172.21.15.172:0/821860503 shutdown_connections 2024-09-10T08:45:10.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.724+0000 7f6e36ec8640 1 --2- 172.21.15.172:0/821860503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e30108290 0x7f6e3010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.724+0000 7f6e36ec8640 1 --2- 172.21.15.172:0/821860503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e30103740 0x7f6e30107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.724+0000 7f6e36ec8640 1 --2- 172.21.15.172:0/821860503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e30102d90 0x7f6e30103170 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.724+0000 7f6e36ec8640 1 -- 172.21.15.172:0/821860503 >> 172.21.15.172:0/821860503 conn(0x7f6e300fc7d0 msgr2=0x7f6e300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:10.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.724+0000 7f6e36ec8640 1 -- 172.21.15.172:0/821860503 shutdown_connections 2024-09-10T08:45:10.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.724+0000 7f6e36ec8640 1 -- 172.21.15.172:0/821860503 wait complete. 2024-09-10T08:45:10.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.724+0000 7f6e36ec8640 1 Processor -- start 2024-09-10T08:45:10.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.724+0000 7f6e36ec8640 1 -- start start 2024-09-10T08:45:10.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.724+0000 7f6e36ec8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e30102d90 0x7f6e30198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:10.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.724+0000 7f6e36ec8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e30103740 0x7f6e30198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:10.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.724+0000 7f6e36ec8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e30108290 0x7f6e3019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:10.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.724+0000 7f6e36ec8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6e30111f40 con 0x7f6e30108290 2024-09-10T08:45:10.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.724+0000 7f6e36ec8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e30111dc0 con 0x7f6e30103740 2024-09-10T08:45:10.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.724+0000 7f6e36ec8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6e301120c0 con 0x7f6e30102d90 2024-09-10T08:45:10.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.725+0000 7f6e35465640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e30108290 0x7f6e3019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:10.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.725+0000 7f6e34c64640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e30102d90 0x7f6e30198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:10.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.725+0000 7f6e35465640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e30108290 0x7f6e3019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48380/0 (socket says 172.21.15.172:48380) 2024-09-10T08:45:10.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.725+0000 7f6e35465640 1 -- 172.21.15.172:0/3463104833 learned_addr learned my addr 172.21.15.172:0/3463104833 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:10.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.725+0000 7f6e2ffff640 1 --2- 172.21.15.172:0/3463104833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e30103740 0x7f6e30198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:10.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.725+0000 7f6e35465640 1 -- 172.21.15.172:0/3463104833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e30102d90 msgr2=0x7f6e30198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:10.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.725+0000 7f6e35465640 1 --2- 172.21.15.172:0/3463104833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e30102d90 0x7f6e30198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.725+0000 7f6e35465640 1 -- 172.21.15.172:0/3463104833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e30103740 msgr2=0x7f6e30198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:10.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.725+0000 7f6e35465640 1 --2- 172.21.15.172:0/3463104833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e30103740 0x7f6e30198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.725+0000 7f6e35465640 1 -- 172.21.15.172:0/3463104833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3019d520 con 0x7f6e30108290 2024-09-10T08:45:10.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.725+0000 7f6e35465640 1 --2- 172.21.15.172:0/3463104833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e30108290 0x7f6e3019ce40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009fd0 tx=0x7f6e2800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:10.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.725+0000 7f6e2dffb640 1 -- 172.21.15.172:0/3463104833 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28017070 con 0x7f6e30108290 2024-09-10T08:45:10.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.725+0000 7f6e36ec8640 1 -- 172.21.15.172:0/3463104833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3019d810 con 0x7f6e30108290 2024-09-10T08:45:10.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.725+0000 7f6e2dffb640 1 -- 172.21.15.172:0/3463104833 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e28003e20 con 0x7f6e30108290 2024-09-10T08:45:10.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.725+0000 7f6e2dffb640 1 -- 172.21.15.172:0/3463104833 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28010460 con 0x7f6e30108290 2024-09-10T08:45:10.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.726+0000 7f6e36ec8640 1 -- 172.21.15.172:0/3463104833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e301a50f0 con 0x7f6e30108290 2024-09-10T08:45:10.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.727+0000 7f6e36ec8640 1 -- 172.21.15.172:0/3463104833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e3006a7f0 con 0x7f6e30108290 2024-09-10T08:45:10.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.728+0000 7f6e2dffb640 1 -- 172.21.15.172:0/3463104833 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6e28003970 con 0x7f6e30108290 2024-09-10T08:45:10.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.728+0000 7f6e2dffb640 1 --2- 172.21.15.172:0/3463104833 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e200a2c90 0x7f6e200a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:10.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.728+0000 7f6e2dffb640 1 -- 172.21.15.172:0/3463104833 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f6e280cf190 con 0x7f6e30108290 2024-09-10T08:45:10.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.729+0000 7f6e34c64640 1 --2- 172.21.15.172:0/3463104833 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e200a2c90 0x7f6e200a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:10.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.729+0000 7f6e34c64640 1 --2- 172.21.15.172:0/3463104833 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e200a2c90 0x7f6e200a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009900 tx=0x7f6e24000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:10.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.732+0000 7f6e2dffb640 1 -- 172.21.15.172:0/3463104833 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e2809bd50 con 0x7f6e30108290 2024-09-10T08:45:10.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.898+0000 7f6e36ec8640 1 -- 172.21.15.172:0/3463104833 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6e30062630 con 0x7f6e200a2c90 2024-09-10T08:45:10.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.906+0000 7f6e2dffb640 1 -- 172.21.15.172:0/3463104833 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340379 (secure 0 0 0) 0x7f6e30062630 con 0x7f6e200a2c90 2024-09-10T08:45:10.911 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:45:10.913 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-09-10T08:45:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.909+0000 7f6e36ec8640 1 -- 172.21.15.172:0/3463104833 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e200a2c90 msgr2=0x7f6e200a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.909+0000 7f6e36ec8640 1 --2- 172.21.15.172:0/3463104833 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e200a2c90 0x7f6e200a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009900 tx=0x7f6e24000bc0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.909+0000 7f6e36ec8640 1 -- 172.21.15.172:0/3463104833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e30108290 msgr2=0x7f6e3019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.909+0000 7f6e36ec8640 1 --2- 172.21.15.172:0/3463104833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e30108290 0x7f6e3019ce40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009fd0 tx=0x7f6e2800b260 comp rx=0 tx=0).stop 2024-09-10T08:45:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.909+0000 7f6e36ec8640 1 -- 172.21.15.172:0/3463104833 shutdown_connections 2024-09-10T08:45:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.909+0000 7f6e36ec8640 1 --2- 172.21.15.172:0/3463104833 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e200a2c90 0x7f6e200a5150 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.909+0000 7f6e36ec8640 1 --2- 172.21.15.172:0/3463104833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e30108290 0x7f6e3019ce40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.909+0000 7f6e36ec8640 1 --2- 172.21.15.172:0/3463104833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e30103740 0x7f6e30198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.909+0000 7f6e36ec8640 1 --2- 172.21.15.172:0/3463104833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e30102d90 0x7f6e30198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.909+0000 7f6e36ec8640 1 -- 172.21.15.172:0/3463104833 >> 172.21.15.172:0/3463104833 conn(0x7f6e300fc7d0 msgr2=0x7f6e30105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.909+0000 7f6e36ec8640 1 -- 172.21.15.172:0/3463104833 shutdown_connections 2024-09-10T08:45:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:10.910+0000 7f6e36ec8640 1 -- 172.21.15.172:0/3463104833 wait complete. 2024-09-10T08:45:11.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:45:11.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:45:11.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:11.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:45:11.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:45:11.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:11.561 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:45:11.561 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:45:11.561 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:45:11.588 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-10T08:45:09.131360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640762,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":285,"num_read_kb":246,"num_write":416,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375452,"kb_used_data":13724,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566372,"statfs":{"total":1151940427776,"available":1151555964928,"internally_reserved":0,"allocated":14053376,"data_stored":7734866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41410,"num_objects":16,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":123,"num_read_kb":106,"num_write":57,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003643"},"pg_stats":[{"pgid":"2.1f","version":"88'13","reported_seq":55,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:04.247638+0000","last_change":"2024-09-10T08:44:08.657737+0000","last_active":"2024-09-10T08:45:04.247638+0000","last_peered":"2024-09-10T08:45:04.247638+0000","last_clean":"2024-09-10T08:45:04.247638+0000","last_became_active":"2024-09-10T08:44:08.657473+0000","last_became_peered":"2024-09-10T08:44:08.657473+0000","last_unstale":"2024-09-10T08:45:04.247638+0000","last_undegraded":"2024-09-10T08:45:04.247638+0000","last_fullsized":"2024-09-10T08:45:04.247638+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:40.757343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990095+0000","last_change":"2024-09-10T08:44:10.675090+0000","last_active":"2024-09-10T08:44:20.990095+0000","last_peered":"2024-09-10T08:44:20.990095+0000","last_clean":"2024-09-10T08:44:20.990095+0000","last_became_active":"2024-09-10T08:44:10.674612+0000","last_became_peered":"2024-09-10T08:44:10.674612+0000","last_unstale":"2024-09-10T08:44:20.990095+0000","last_undegraded":"2024-09-10T08:44:20.990095+0000","last_fullsized":"2024-09-10T08:44:20.990095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:46.821080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717686+0000","last_change":"2024-09-10T08:44:17.721559+0000","last_active":"2024-09-10T08:44:20.717686+0000","last_peered":"2024-09-10T08:44:20.717686+0000","last_clean":"2024-09-10T08:44:20.717686+0000","last_became_active":"2024-09-10T08:44:17.721182+0000","last_became_peered":"2024-09-10T08:44:17.721182+0000","last_unstale":"2024-09-10T08:44:20.717686+0000","last_undegraded":"2024-09-10T08:44:20.717686+0000","last_fullsized":"2024-09-10T08:44:20.717686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:18.791489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990300+0000","last_change":"2024-09-10T08:44:09.049171+0000","last_active":"2024-09-10T08:44:20.990300+0000","last_peered":"2024-09-10T08:44:20.990300+0000","last_clean":"2024-09-10T08:44:20.990300+0000","last_became_active":"2024-09-10T08:44:09.048806+0000","last_became_peered":"2024-09-10T08:44:09.048806+0000","last_unstale":"2024-09-10T08:44:20.990300+0000","last_undegraded":"2024-09-10T08:44:20.990300+0000","last_fullsized":"2024-09-10T08:44:20.990300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:21.023460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990630+0000","last_change":"2024-09-10T08:44:10.677569+0000","last_active":"2024-09-10T08:44:20.990630+0000","last_peered":"2024-09-10T08:44:20.990630+0000","last_clean":"2024-09-10T08:44:20.990630+0000","last_became_active":"2024-09-10T08:44:10.677048+0000","last_became_peered":"2024-09-10T08:44:10.677048+0000","last_unstale":"2024-09-10T08:44:20.990630+0000","last_undegraded":"2024-09-10T08:44:20.990630+0000","last_fullsized":"2024-09-10T08:44:20.990630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:35.523063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990141+0000","last_change":"2024-09-10T08:44:17.720138+0000","last_active":"2024-09-10T08:44:20.990141+0000","last_peered":"2024-09-10T08:44:20.990141+0000","last_clean":"2024-09-10T08:44:20.990141+0000","last_became_active":"2024-09-10T08:44:17.719344+0000","last_became_peered":"2024-09-10T08:44:17.719344+0000","last_unstale":"2024-09-10T08:44:20.990141+0000","last_undegraded":"2024-09-10T08:44:20.990141+0000","last_fullsized":"2024-09-10T08:44:20.990141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:11.758058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:55.227631+0000","last_change":"2024-09-10T08:44:08.660099+0000","last_active":"2024-09-10T08:44:55.227631+0000","last_peered":"2024-09-10T08:44:55.227631+0000","last_clean":"2024-09-10T08:44:55.227631+0000","last_became_active":"2024-09-10T08:44:08.659745+0000","last_became_peered":"2024-09-10T08:44:08.659745+0000","last_unstale":"2024-09-10T08:44:55.227631+0000","last_undegraded":"2024-09-10T08:44:55.227631+0000","last_fullsized":"2024-09-10T08:44:55.227631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:43.960717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990344+0000","last_change":"2024-09-10T08:44:10.675583+0000","last_active":"2024-09-10T08:44:20.990344+0000","last_peered":"2024-09-10T08:44:20.990344+0000","last_clean":"2024-09-10T08:44:20.990344+0000","last_became_active":"2024-09-10T08:44:10.675296+0000","last_became_peered":"2024-09-10T08:44:10.675296+0000","last_unstale":"2024-09-10T08:44:20.990344+0000","last_undegraded":"2024-09-10T08:44:20.990344+0000","last_fullsized":"2024-09-10T08:44:20.990344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:31:51.147862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990657+0000","last_change":"2024-09-10T08:44:17.720104+0000","last_active":"2024-09-10T08:44:20.990657+0000","last_peered":"2024-09-10T08:44:20.990657+0000","last_clean":"2024-09-10T08:44:20.990657+0000","last_became_active":"2024-09-10T08:44:17.719607+0000","last_became_peered":"2024-09-10T08:44:17.719607+0000","last_unstale":"2024-09-10T08:44:20.990657+0000","last_undegraded":"2024-09-10T08:44:20.990657+0000","last_fullsized":"2024-09-10T08:44:20.990657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:52.976717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.233345+0000","last_change":"2024-09-10T08:44:08.658588+0000","last_active":"2024-09-10T08:44:54.233345+0000","last_peered":"2024-09-10T08:44:54.233345+0000","last_clean":"2024-09-10T08:44:54.233345+0000","last_became_active":"2024-09-10T08:44:08.658053+0000","last_became_peered":"2024-09-10T08:44:08.658053+0000","last_unstale":"2024-09-10T08:44:54.233345+0000","last_undegraded":"2024-09-10T08:44:54.233345+0000","last_fullsized":"2024-09-10T08:44:54.233345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:26.744943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991604+0000","last_change":"2024-09-10T08:44:10.681978+0000","last_active":"2024-09-10T08:44:20.991604+0000","last_peered":"2024-09-10T08:44:20.991604+0000","last_clean":"2024-09-10T08:44:20.991604+0000","last_became_active":"2024-09-10T08:44:10.681600+0000","last_became_peered":"2024-09-10T08:44:10.681600+0000","last_unstale":"2024-09-10T08:44:20.991604+0000","last_undegraded":"2024-09-10T08:44:20.991604+0000","last_fullsized":"2024-09-10T08:44:20.991604+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:37.190576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990299+0000","last_change":"2024-09-10T08:44:17.719784+0000","last_active":"2024-09-10T08:44:20.990299+0000","last_peered":"2024-09-10T08:44:20.990299+0000","last_clean":"2024-09-10T08:44:20.990299+0000","last_became_active":"2024-09-10T08:44:17.719182+0000","last_became_peered":"2024-09-10T08:44:17.719182+0000","last_unstale":"2024-09-10T08:44:20.990299+0000","last_undegraded":"2024-09-10T08:44:20.990299+0000","last_fullsized":"2024-09-10T08:44:20.990299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:15.237772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.250029+0000","last_change":"2024-09-10T08:44:10.679069+0000","last_active":"2024-09-10T08:44:21.250029+0000","last_peered":"2024-09-10T08:44:21.250029+0000","last_clean":"2024-09-10T08:44:21.250029+0000","last_became_active":"2024-09-10T08:44:10.678597+0000","last_became_peered":"2024-09-10T08:44:10.678597+0000","last_unstale":"2024-09-10T08:44:21.250029+0000","last_undegraded":"2024-09-10T08:44:21.250029+0000","last_fullsized":"2024-09-10T08:44:21.250029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:52.693906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:51.259703+0000","last_change":"2024-09-10T08:44:08.660886+0000","last_active":"2024-09-10T08:44:51.259703+0000","last_peered":"2024-09-10T08:44:51.259703+0000","last_clean":"2024-09-10T08:44:51.259703+0000","last_became_active":"2024-09-10T08:44:08.660478+0000","last_became_peered":"2024-09-10T08:44:08.660478+0000","last_unstale":"2024-09-10T08:44:51.259703+0000","last_undegraded":"2024-09-10T08:44:51.259703+0000","last_fullsized":"2024-09-10T08:44:51.259703+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:41.785531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990273+0000","last_change":"2024-09-10T08:44:17.761929+0000","last_active":"2024-09-10T08:44:20.990273+0000","last_peered":"2024-09-10T08:44:20.990273+0000","last_clean":"2024-09-10T08:44:20.990273+0000","last_became_active":"2024-09-10T08:44:17.761440+0000","last_became_peered":"2024-09-10T08:44:17.761440+0000","last_unstale":"2024-09-10T08:44:20.990273+0000","last_undegraded":"2024-09-10T08:44:20.990273+0000","last_fullsized":"2024-09-10T08:44:20.990273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:37.817759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989394+0000","last_change":"2024-09-10T08:44:10.674071+0000","last_active":"2024-09-10T08:44:20.989394+0000","last_peered":"2024-09-10T08:44:20.989394+0000","last_clean":"2024-09-10T08:44:20.989394+0000","last_became_active":"2024-09-10T08:44:10.673770+0000","last_became_peered":"2024-09-10T08:44:10.673770+0000","last_unstale":"2024-09-10T08:44:20.989394+0000","last_undegraded":"2024-09-10T08:44:20.989394+0000","last_fullsized":"2024-09-10T08:44:20.989394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:44:19.944557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:53.254468+0000","last_change":"2024-09-10T08:44:08.656636+0000","last_active":"2024-09-10T08:44:53.254468+0000","last_peered":"2024-09-10T08:44:53.254468+0000","last_clean":"2024-09-10T08:44:53.254468+0000","last_became_active":"2024-09-10T08:44:08.655813+0000","last_became_peered":"2024-09-10T08:44:08.655813+0000","last_unstale":"2024-09-10T08:44:53.254468+0000","last_undegraded":"2024-09-10T08:44:53.254468+0000","last_fullsized":"2024-09-10T08:44:53.254468+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:42.723053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990387+0000","last_change":"2024-09-10T08:44:17.722877+0000","last_active":"2024-09-10T08:44:20.990387+0000","last_peered":"2024-09-10T08:44:20.990387+0000","last_clean":"2024-09-10T08:44:20.990387+0000","last_became_active":"2024-09-10T08:44:17.722046+0000","last_became_peered":"2024-09-10T08:44:17.722046+0000","last_unstale":"2024-09-10T08:44:20.990387+0000","last_undegraded":"2024-09-10T08:44:20.990387+0000","last_fullsized":"2024-09-10T08:44:20.990387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:32.675598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990210+0000","last_change":"2024-09-10T08:44:10.671500+0000","last_active":"2024-09-10T08:44:20.990210+0000","last_peered":"2024-09-10T08:44:20.990210+0000","last_clean":"2024-09-10T08:44:20.990210+0000","last_became_active":"2024-09-10T08:44:10.671039+0000","last_became_peered":"2024-09-10T08:44:10.671039+0000","last_unstale":"2024-09-10T08:44:20.990210+0000","last_undegraded":"2024-09-10T08:44:20.990210+0000","last_fullsized":"2024-09-10T08:44:20.990210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:12.977373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.230652+0000","last_change":"2024-09-10T08:44:08.655467+0000","last_active":"2024-09-10T08:44:54.230652+0000","last_peered":"2024-09-10T08:44:54.230652+0000","last_clean":"2024-09-10T08:44:54.230652+0000","last_became_active":"2024-09-10T08:44:08.654991+0000","last_became_peered":"2024-09-10T08:44:08.654991+0000","last_unstale":"2024-09-10T08:44:54.230652+0000","last_undegraded":"2024-09-10T08:44:54.230652+0000","last_fullsized":"2024-09-10T08:44:54.230652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:33.124360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717671+0000","last_change":"2024-09-10T08:44:17.724250+0000","last_active":"2024-09-10T08:44:20.717671+0000","last_peered":"2024-09-10T08:44:20.717671+0000","last_clean":"2024-09-10T08:44:20.717671+0000","last_became_active":"2024-09-10T08:44:17.723733+0000","last_became_peered":"2024-09-10T08:44:17.723733+0000","last_unstale":"2024-09-10T08:44:20.717671+0000","last_undegraded":"2024-09-10T08:44:20.717671+0000","last_fullsized":"2024-09-10T08:44:20.717671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:40.098666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989901+0000","last_change":"2024-09-10T08:44:10.664159+0000","last_active":"2024-09-10T08:44:20.989901+0000","last_peered":"2024-09-10T08:44:20.989901+0000","last_clean":"2024-09-10T08:44:20.989901+0000","last_became_active":"2024-09-10T08:44:10.663674+0000","last_became_peered":"2024-09-10T08:44:10.663674+0000","last_unstale":"2024-09-10T08:44:20.989901+0000","last_undegraded":"2024-09-10T08:44:20.989901+0000","last_fullsized":"2024-09-10T08:44:20.989901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:24.234525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:51.258652+0000","last_change":"2024-09-10T08:44:09.050388+0000","last_active":"2024-09-10T08:44:51.258652+0000","last_peered":"2024-09-10T08:44:51.258652+0000","last_clean":"2024-09-10T08:44:51.258652+0000","last_became_active":"2024-09-10T08:44:09.049986+0000","last_became_peered":"2024-09-10T08:44:09.049986+0000","last_unstale":"2024-09-10T08:44:51.258652+0000","last_undegraded":"2024-09-10T08:44:51.258652+0000","last_fullsized":"2024-09-10T08:44:51.258652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:16:12.248883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990857+0000","last_change":"2024-09-10T08:44:17.761639+0000","last_active":"2024-09-10T08:44:20.990857+0000","last_peered":"2024-09-10T08:44:20.990857+0000","last_clean":"2024-09-10T08:44:20.990857+0000","last_became_active":"2024-09-10T08:44:17.761214+0000","last_became_peered":"2024-09-10T08:44:17.761214+0000","last_unstale":"2024-09-10T08:44:20.990857+0000","last_undegraded":"2024-09-10T08:44:20.990857+0000","last_fullsized":"2024-09-10T08:44:20.990857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:43.793195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990197+0000","last_change":"2024-09-10T08:44:10.672168+0000","last_active":"2024-09-10T08:44:20.990197+0000","last_peered":"2024-09-10T08:44:20.990197+0000","last_clean":"2024-09-10T08:44:20.990197+0000","last_became_active":"2024-09-10T08:44:10.671840+0000","last_became_peered":"2024-09-10T08:44:10.671840+0000","last_unstale":"2024-09-10T08:44:20.990197+0000","last_undegraded":"2024-09-10T08:44:20.990197+0000","last_fullsized":"2024-09-10T08:44:20.990197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:29.321232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.231076+0000","last_change":"2024-09-10T08:44:09.036632+0000","last_active":"2024-09-10T08:44:54.231076+0000","last_peered":"2024-09-10T08:44:54.231076+0000","last_clean":"2024-09-10T08:44:54.231076+0000","last_became_active":"2024-09-10T08:44:09.036254+0000","last_became_peered":"2024-09-10T08:44:09.036254+0000","last_unstale":"2024-09-10T08:44:54.231076+0000","last_undegraded":"2024-09-10T08:44:54.231076+0000","last_fullsized":"2024-09-10T08:44:54.231076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:59:40.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717686+0000","last_change":"2024-09-10T08:44:17.716215+0000","last_active":"2024-09-10T08:44:20.717686+0000","last_peered":"2024-09-10T08:44:20.717686+0000","last_clean":"2024-09-10T08:44:20.717686+0000","last_became_active":"2024-09-10T08:44:17.715809+0000","last_became_peered":"2024-09-10T08:44:17.715809+0000","last_unstale":"2024-09-10T08:44:20.717686+0000","last_undegraded":"2024-09-10T08:44:20.717686+0000","last_fullsized":"2024-09-10T08:44:20.717686+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:52.896401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990018+0000","last_change":"2024-09-10T08:44:10.677871+0000","last_active":"2024-09-10T08:44:20.990018+0000","last_peered":"2024-09-10T08:44:20.990018+0000","last_clean":"2024-09-10T08:44:20.990018+0000","last_became_active":"2024-09-10T08:44:10.677101+0000","last_became_peered":"2024-09-10T08:44:10.677101+0000","last_unstale":"2024-09-10T08:44:20.990018+0000","last_undegraded":"2024-09-10T08:44:20.990018+0000","last_fullsized":"2024-09-10T08:44:20.990018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:46.137249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'12","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:02.690996+0000","last_change":"2024-09-10T08:44:08.654428+0000","last_active":"2024-09-10T08:45:02.690996+0000","last_peered":"2024-09-10T08:45:02.690996+0000","last_clean":"2024-09-10T08:45:02.690996+0000","last_became_active":"2024-09-10T08:44:08.654119+0000","last_became_peered":"2024-09-10T08:44:08.654119+0000","last_unstale":"2024-09-10T08:45:02.690996+0000","last_undegraded":"2024-09-10T08:45:02.690996+0000","last_fullsized":"2024-09-10T08:45:02.690996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:56.787721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989462+0000","last_change":"2024-09-10T08:44:17.760828+0000","last_active":"2024-09-10T08:44:20.989462+0000","last_peered":"2024-09-10T08:44:20.989462+0000","last_clean":"2024-09-10T08:44:20.989462+0000","last_became_active":"2024-09-10T08:44:17.760367+0000","last_became_peered":"2024-09-10T08:44:17.760367+0000","last_unstale":"2024-09-10T08:44:20.989462+0000","last_undegraded":"2024-09-10T08:44:20.989462+0000","last_fullsized":"2024-09-10T08:44:20.989462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:11.130480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991264+0000","last_change":"2024-09-10T08:44:10.670179+0000","last_active":"2024-09-10T08:44:20.991264+0000","last_peered":"2024-09-10T08:44:20.991264+0000","last_clean":"2024-09-10T08:44:20.991264+0000","last_became_active":"2024-09-10T08:44:10.669366+0000","last_became_peered":"2024-09-10T08:44:10.669366+0000","last_unstale":"2024-09-10T08:44:20.991264+0000","last_undegraded":"2024-09-10T08:44:20.991264+0000","last_fullsized":"2024-09-10T08:44:20.991264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:41.916365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:53.223607+0000","last_change":"2024-09-10T08:44:09.050441+0000","last_active":"2024-09-10T08:44:53.223607+0000","last_peered":"2024-09-10T08:44:53.223607+0000","last_clean":"2024-09-10T08:44:53.223607+0000","last_became_active":"2024-09-10T08:44:09.050070+0000","last_became_peered":"2024-09-10T08:44:09.050070+0000","last_unstale":"2024-09-10T08:44:53.223607+0000","last_undegraded":"2024-09-10T08:44:53.223607+0000","last_fullsized":"2024-09-10T08:44:53.223607+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:13.540021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990154+0000","last_change":"2024-09-10T08:44:17.720098+0000","last_active":"2024-09-10T08:44:20.990154+0000","last_peered":"2024-09-10T08:44:20.990154+0000","last_clean":"2024-09-10T08:44:20.990154+0000","last_became_active":"2024-09-10T08:44:17.719627+0000","last_became_peered":"2024-09-10T08:44:17.719627+0000","last_unstale":"2024-09-10T08:44:20.990154+0000","last_undegraded":"2024-09-10T08:44:20.990154+0000","last_fullsized":"2024-09-10T08:44:20.990154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:50.020919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717942+0000","last_change":"2024-09-10T08:44:10.669630+0000","last_active":"2024-09-10T08:44:20.717942+0000","last_peered":"2024-09-10T08:44:20.717942+0000","last_clean":"2024-09-10T08:44:20.717942+0000","last_became_active":"2024-09-10T08:44:10.669307+0000","last_became_peered":"2024-09-10T08:44:10.669307+0000","last_unstale":"2024-09-10T08:44:20.717942+0000","last_undegraded":"2024-09-10T08:44:20.717942+0000","last_fullsized":"2024-09-10T08:44:20.717942+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:06.320046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717501+0000","last_change":"2024-09-10T08:44:09.050715+0000","last_active":"2024-09-10T08:44:20.717501+0000","last_peered":"2024-09-10T08:44:20.717501+0000","last_clean":"2024-09-10T08:44:20.717501+0000","last_became_active":"2024-09-10T08:44:09.050347+0000","last_became_peered":"2024-09-10T08:44:09.050347+0000","last_unstale":"2024-09-10T08:44:20.717501+0000","last_undegraded":"2024-09-10T08:44:20.717501+0000","last_fullsized":"2024-09-10T08:44:20.717501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:18.689652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990046+0000","last_change":"2024-09-10T08:44:17.761503+0000","last_active":"2024-09-10T08:44:20.990046+0000","last_peered":"2024-09-10T08:44:20.990046+0000","last_clean":"2024-09-10T08:44:20.990046+0000","last_became_active":"2024-09-10T08:44:17.761042+0000","last_became_peered":"2024-09-10T08:44:17.761042+0000","last_unstale":"2024-09-10T08:44:20.990046+0000","last_undegraded":"2024-09-10T08:44:20.990046+0000","last_fullsized":"2024-09-10T08:44:20.990046+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:44.588850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.718217+0000","last_change":"2024-09-10T08:44:10.670881+0000","last_active":"2024-09-10T08:44:20.718217+0000","last_peered":"2024-09-10T08:44:20.718217+0000","last_clean":"2024-09-10T08:44:20.718217+0000","last_became_active":"2024-09-10T08:44:10.669791+0000","last_became_peered":"2024-09-10T08:44:10.669791+0000","last_unstale":"2024-09-10T08:44:20.718217+0000","last_undegraded":"2024-09-10T08:44:20.718217+0000","last_fullsized":"2024-09-10T08:44:20.718217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:33.644205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:07.252382+0000","last_change":"2024-09-10T08:44:08.657938+0000","last_active":"2024-09-10T08:45:07.252382+0000","last_peered":"2024-09-10T08:45:07.252382+0000","last_clean":"2024-09-10T08:45:07.252382+0000","last_became_active":"2024-09-10T08:44:08.657526+0000","last_became_peered":"2024-09-10T08:44:08.657526+0000","last_unstale":"2024-09-10T08:45:07.252382+0000","last_undegraded":"2024-09-10T08:45:07.252382+0000","last_fullsized":"2024-09-10T08:45:07.252382+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:15.673561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990998+0000","last_change":"2024-09-10T08:44:17.721619+0000","last_active":"2024-09-10T08:44:20.990998+0000","last_peered":"2024-09-10T08:44:20.990998+0000","last_clean":"2024-09-10T08:44:20.990998+0000","last_became_active":"2024-09-10T08:44:17.721175+0000","last_became_peered":"2024-09-10T08:44:17.721175+0000","last_unstale":"2024-09-10T08:44:20.990998+0000","last_undegraded":"2024-09-10T08:44:20.990998+0000","last_fullsized":"2024-09-10T08:44:20.990998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:52.244120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990673+0000","last_change":"2024-09-10T08:44:10.676164+0000","last_active":"2024-09-10T08:44:20.990673+0000","last_peered":"2024-09-10T08:44:20.990673+0000","last_clean":"2024-09-10T08:44:20.990673+0000","last_became_active":"2024-09-10T08:44:10.675820+0000","last_became_peered":"2024-09-10T08:44:10.675820+0000","last_unstale":"2024-09-10T08:44:20.990673+0000","last_undegraded":"2024-09-10T08:44:20.990673+0000","last_fullsized":"2024-09-10T08:44:20.990673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:24.477584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:07.235617+0000","last_change":"2024-09-10T08:44:09.049124+0000","last_active":"2024-09-10T08:45:07.235617+0000","last_peered":"2024-09-10T08:45:07.235617+0000","last_clean":"2024-09-10T08:45:07.235617+0000","last_became_active":"2024-09-10T08:44:09.048799+0000","last_became_peered":"2024-09-10T08:44:09.048799+0000","last_unstale":"2024-09-10T08:45:07.235617+0000","last_undegraded":"2024-09-10T08:45:07.235617+0000","last_fullsized":"2024-09-10T08:45:07.235617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:43:39.849727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990940+0000","last_change":"2024-09-10T08:44:17.761130+0000","last_active":"2024-09-10T08:44:20.990940+0000","last_peered":"2024-09-10T08:44:20.990940+0000","last_clean":"2024-09-10T08:44:20.990940+0000","last_became_active":"2024-09-10T08:44:17.760753+0000","last_became_peered":"2024-09-10T08:44:17.760753+0000","last_unstale":"2024-09-10T08:44:20.990940+0000","last_undegraded":"2024-09-10T08:44:20.990940+0000","last_fullsized":"2024-09-10T08:44:20.990940+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:42.971962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990701+0000","last_change":"2024-09-10T08:44:10.669905+0000","last_active":"2024-09-10T08:44:20.990701+0000","last_peered":"2024-09-10T08:44:20.990701+0000","last_clean":"2024-09-10T08:44:20.990701+0000","last_became_active":"2024-09-10T08:44:10.669340+0000","last_became_peered":"2024-09-10T08:44:10.669340+0000","last_unstale":"2024-09-10T08:44:20.990701+0000","last_undegraded":"2024-09-10T08:44:20.990701+0000","last_fullsized":"2024-09-10T08:44:20.990701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:23.263179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:52.265666+0000","last_change":"2024-09-10T08:44:08.658862+0000","last_active":"2024-09-10T08:44:52.265666+0000","last_peered":"2024-09-10T08:44:52.265666+0000","last_clean":"2024-09-10T08:44:52.265666+0000","last_became_active":"2024-09-10T08:44:08.657880+0000","last_became_peered":"2024-09-10T08:44:08.657880+0000","last_unstale":"2024-09-10T08:44:52.265666+0000","last_undegraded":"2024-09-10T08:44:52.265666+0000","last_fullsized":"2024-09-10T08:44:52.265666+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:34.224025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991130+0000","last_change":"2024-09-10T08:44:17.761517+0000","last_active":"2024-09-10T08:44:20.991130+0000","last_peered":"2024-09-10T08:44:20.991130+0000","last_clean":"2024-09-10T08:44:20.991130+0000","last_became_active":"2024-09-10T08:44:17.761081+0000","last_became_peered":"2024-09-10T08:44:17.761081+0000","last_unstale":"2024-09-10T08:44:20.991130+0000","last_undegraded":"2024-09-10T08:44:20.991130+0000","last_fullsized":"2024-09-10T08:44:20.991130+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:22.169689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989753+0000","last_change":"2024-09-10T08:44:10.673258+0000","last_active":"2024-09-10T08:44:20.989753+0000","last_peered":"2024-09-10T08:44:20.989753+0000","last_clean":"2024-09-10T08:44:20.989753+0000","last_became_active":"2024-09-10T08:44:10.672854+0000","last_became_peered":"2024-09-10T08:44:10.672854+0000","last_unstale":"2024-09-10T08:44:20.989753+0000","last_undegraded":"2024-09-10T08:44:20.989753+0000","last_fullsized":"2024-09-10T08:44:20.989753+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:35.913820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.232536+0000","last_change":"2024-09-10T08:44:08.658039+0000","last_active":"2024-09-10T08:44:54.232536+0000","last_peered":"2024-09-10T08:44:54.232536+0000","last_clean":"2024-09-10T08:44:54.232536+0000","last_became_active":"2024-09-10T08:44:08.657636+0000","last_became_peered":"2024-09-10T08:44:08.657636+0000","last_unstale":"2024-09-10T08:44:54.232536+0000","last_undegraded":"2024-09-10T08:44:54.232536+0000","last_fullsized":"2024-09-10T08:44:54.232536+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:09.615992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.249993+0000","last_change":"2024-09-10T08:42:57.666747+0000","last_active":"2024-09-10T08:44:21.249993+0000","last_peered":"2024-09-10T08:44:21.249993+0000","last_clean":"2024-09-10T08:44:21.249993+0000","last_became_active":"2024-09-10T08:42:57.666417+0000","last_became_peered":"2024-09-10T08:42:57.666417+0000","last_unstale":"2024-09-10T08:44:21.249993+0000","last_undegraded":"2024-09-10T08:44:21.249993+0000","last_fullsized":"2024-09-10T08:44:21.249993+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_clean_scrub_stamp":"2024-09-10T08:39:31.729794+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.547989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.250333+0000","last_change":"2024-09-10T08:44:17.723021+0000","last_active":"2024-09-10T08:44:21.250333+0000","last_peered":"2024-09-10T08:44:21.250333+0000","last_clean":"2024-09-10T08:44:21.250333+0000","last_became_active":"2024-09-10T08:44:17.722222+0000","last_became_peered":"2024-09-10T08:44:17.722222+0000","last_unstale":"2024-09-10T08:44:21.250333+0000","last_undegraded":"2024-09-10T08:44:21.250333+0000","last_fullsized":"2024-09-10T08:44:21.250333+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:24.444427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.250381+0000","last_change":"2024-09-10T08:44:10.676651+0000","last_active":"2024-09-10T08:44:21.250381+0000","last_peered":"2024-09-10T08:44:21.250381+0000","last_clean":"2024-09-10T08:44:21.250381+0000","last_became_active":"2024-09-10T08:44:10.676342+0000","last_became_peered":"2024-09-10T08:44:10.676342+0000","last_unstale":"2024-09-10T08:44:21.250381+0000","last_undegraded":"2024-09-10T08:44:21.250381+0000","last_fullsized":"2024-09-10T08:44:21.250381+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:07.021273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:05.250329+0000","last_change":"2024-09-10T08:44:09.050305+0000","last_active":"2024-09-10T08:45:05.250329+0000","last_peered":"2024-09-10T08:45:05.250329+0000","last_clean":"2024-09-10T08:45:05.250329+0000","last_became_active":"2024-09-10T08:44:09.049958+0000","last_became_peered":"2024-09-10T08:44:09.049958+0000","last_unstale":"2024-09-10T08:45:05.250329+0000","last_undegraded":"2024-09-10T08:45:05.250329+0000","last_fullsized":"2024-09-10T08:45:05.250329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:42.219190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989775+0000","last_change":"2024-09-10T08:44:17.720219+0000","last_active":"2024-09-10T08:44:20.989775+0000","last_peered":"2024-09-10T08:44:20.989775+0000","last_clean":"2024-09-10T08:44:20.989775+0000","last_became_active":"2024-09-10T08:44:17.719756+0000","last_became_peered":"2024-09-10T08:44:17.719756+0000","last_unstale":"2024-09-10T08:44:20.989775+0000","last_undegraded":"2024-09-10T08:44:20.989775+0000","last_fullsized":"2024-09-10T08:44:20.989775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:01.070434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.250222+0000","last_change":"2024-09-10T08:44:10.678986+0000","last_active":"2024-09-10T08:44:21.250222+0000","last_peered":"2024-09-10T08:44:21.250222+0000","last_clean":"2024-09-10T08:44:21.250222+0000","last_became_active":"2024-09-10T08:44:10.678654+0000","last_became_peered":"2024-09-10T08:44:10.678654+0000","last_unstale":"2024-09-10T08:44:21.250222+0000","last_undegraded":"2024-09-10T08:44:21.250222+0000","last_fullsized":"2024-09-10T08:44:21.250222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:07.116802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:53.222413+0000","last_change":"2024-09-10T08:44:08.655025+0000","last_active":"2024-09-10T08:44:53.222413+0000","last_peered":"2024-09-10T08:44:53.222413+0000","last_clean":"2024-09-10T08:44:53.222413+0000","last_became_active":"2024-09-10T08:44:08.654728+0000","last_became_peered":"2024-09-10T08:44:08.654728+0000","last_unstale":"2024-09-10T08:44:53.222413+0000","last_undegraded":"2024-09-10T08:44:53.222413+0000","last_fullsized":"2024-09-10T08:44:53.222413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:31.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989360+0000","last_change":"2024-09-10T08:44:17.724226+0000","last_active":"2024-09-10T08:44:20.989360+0000","last_peered":"2024-09-10T08:44:20.989360+0000","last_clean":"2024-09-10T08:44:20.989360+0000","last_became_active":"2024-09-10T08:44:17.723927+0000","last_became_peered":"2024-09-10T08:44:17.723927+0000","last_unstale":"2024-09-10T08:44:20.989360+0000","last_undegraded":"2024-09-10T08:44:20.989360+0000","last_fullsized":"2024-09-10T08:44:20.989360+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:16.768671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.250297+0000","last_change":"2024-09-10T08:44:10.675072+0000","last_active":"2024-09-10T08:44:21.250297+0000","last_peered":"2024-09-10T08:44:21.250297+0000","last_clean":"2024-09-10T08:44:21.250297+0000","last_became_active":"2024-09-10T08:44:10.674636+0000","last_became_peered":"2024-09-10T08:44:10.674636+0000","last_unstale":"2024-09-10T08:44:21.250297+0000","last_undegraded":"2024-09-10T08:44:21.250297+0000","last_fullsized":"2024-09-10T08:44:21.250297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:55.198445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.230553+0000","last_change":"2024-09-10T08:44:08.656450+0000","last_active":"2024-09-10T08:44:54.230553+0000","last_peered":"2024-09-10T08:44:54.230553+0000","last_clean":"2024-09-10T08:44:54.230553+0000","last_became_active":"2024-09-10T08:44:08.655967+0000","last_became_peered":"2024-09-10T08:44:08.655967+0000","last_unstale":"2024-09-10T08:44:54.230553+0000","last_undegraded":"2024-09-10T08:44:54.230553+0000","last_fullsized":"2024-09-10T08:44:54.230553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:29.273167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990512+0000","last_change":"2024-09-10T08:44:17.722217+0000","last_active":"2024-09-10T08:44:20.990512+0000","last_peered":"2024-09-10T08:44:20.990512+0000","last_clean":"2024-09-10T08:44:20.990512+0000","last_became_active":"2024-09-10T08:44:17.721800+0000","last_became_peered":"2024-09-10T08:44:17.721800+0000","last_unstale":"2024-09-10T08:44:20.990512+0000","last_undegraded":"2024-09-10T08:44:20.990512+0000","last_fullsized":"2024-09-10T08:44:20.990512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:58.954397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991081+0000","last_change":"2024-09-10T08:44:10.676606+0000","last_active":"2024-09-10T08:44:20.991081+0000","last_peered":"2024-09-10T08:44:20.991081+0000","last_clean":"2024-09-10T08:44:20.991081+0000","last_became_active":"2024-09-10T08:44:10.676247+0000","last_became_peered":"2024-09-10T08:44:10.676247+0000","last_unstale":"2024-09-10T08:44:20.991081+0000","last_undegraded":"2024-09-10T08:44:20.991081+0000","last_fullsized":"2024-09-10T08:44:20.991081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:51.850917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:53.212635+0000","last_change":"2024-09-10T08:44:08.659003+0000","last_active":"2024-09-10T08:44:53.212635+0000","last_peered":"2024-09-10T08:44:53.212635+0000","last_clean":"2024-09-10T08:44:53.212635+0000","last_became_active":"2024-09-10T08:44:08.658324+0000","last_became_peered":"2024-09-10T08:44:08.658324+0000","last_unstale":"2024-09-10T08:44:53.212635+0000","last_undegraded":"2024-09-10T08:44:53.212635+0000","last_fullsized":"2024-09-10T08:44:53.212635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:44.743053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991074+0000","last_change":"2024-09-10T08:44:17.723363+0000","last_active":"2024-09-10T08:44:20.991074+0000","last_peered":"2024-09-10T08:44:20.991074+0000","last_clean":"2024-09-10T08:44:20.991074+0000","last_became_active":"2024-09-10T08:44:17.721799+0000","last_became_peered":"2024-09-10T08:44:17.721799+0000","last_unstale":"2024-09-10T08:44:20.991074+0000","last_undegraded":"2024-09-10T08:44:20.991074+0000","last_fullsized":"2024-09-10T08:44:20.991074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:08:28.772193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991225+0000","last_change":"2024-09-10T08:44:10.682952+0000","last_active":"2024-09-10T08:44:20.991225+0000","last_peered":"2024-09-10T08:44:20.991225+0000","last_clean":"2024-09-10T08:44:20.991225+0000","last_became_active":"2024-09-10T08:44:10.682249+0000","last_became_peered":"2024-09-10T08:44:10.682249+0000","last_unstale":"2024-09-10T08:44:20.991225+0000","last_undegraded":"2024-09-10T08:44:20.991225+0000","last_fullsized":"2024-09-10T08:44:20.991225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:32:21.285461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:53.223218+0000","last_change":"2024-09-10T08:44:09.050454+0000","last_active":"2024-09-10T08:44:53.223218+0000","last_peered":"2024-09-10T08:44:53.223218+0000","last_clean":"2024-09-10T08:44:53.223218+0000","last_became_active":"2024-09-10T08:44:09.050036+0000","last_became_peered":"2024-09-10T08:44:09.050036+0000","last_unstale":"2024-09-10T08:44:53.223218+0000","last_undegraded":"2024-09-10T08:44:53.223218+0000","last_fullsized":"2024-09-10T08:44:53.223218+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:55.141471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991151+0000","last_change":"2024-09-10T08:44:17.721203+0000","last_active":"2024-09-10T08:44:20.991151+0000","last_peered":"2024-09-10T08:44:20.991151+0000","last_clean":"2024-09-10T08:44:20.991151+0000","last_became_active":"2024-09-10T08:44:17.720737+0000","last_became_peered":"2024-09-10T08:44:17.720737+0000","last_unstale":"2024-09-10T08:44:20.991151+0000","last_undegraded":"2024-09-10T08:44:20.991151+0000","last_fullsized":"2024-09-10T08:44:20.991151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:32.081079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.249687+0000","last_change":"2024-09-10T08:44:10.680944+0000","last_active":"2024-09-10T08:44:21.249687+0000","last_peered":"2024-09-10T08:44:21.249687+0000","last_clean":"2024-09-10T08:44:21.249687+0000","last_became_active":"2024-09-10T08:44:10.680644+0000","last_became_peered":"2024-09-10T08:44:10.680644+0000","last_unstale":"2024-09-10T08:44:21.249687+0000","last_undegraded":"2024-09-10T08:44:21.249687+0000","last_fullsized":"2024-09-10T08:44:21.249687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:46.495894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:03.230735+0000","last_change":"2024-09-10T08:44:08.658135+0000","last_active":"2024-09-10T08:45:03.230735+0000","last_peered":"2024-09-10T08:45:03.230735+0000","last_clean":"2024-09-10T08:45:03.230735+0000","last_became_active":"2024-09-10T08:44:08.657793+0000","last_became_peered":"2024-09-10T08:44:08.657793+0000","last_unstale":"2024-09-10T08:45:03.230735+0000","last_undegraded":"2024-09-10T08:45:03.230735+0000","last_fullsized":"2024-09-10T08:45:03.230735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:16.376268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717723+0000","last_change":"2024-09-10T08:44:17.724342+0000","last_active":"2024-09-10T08:44:20.717723+0000","last_peered":"2024-09-10T08:44:20.717723+0000","last_clean":"2024-09-10T08:44:20.717723+0000","last_became_active":"2024-09-10T08:44:17.723958+0000","last_became_peered":"2024-09-10T08:44:17.723958+0000","last_unstale":"2024-09-10T08:44:20.717723+0000","last_undegraded":"2024-09-10T08:44:20.717723+0000","last_fullsized":"2024-09-10T08:44:20.717723+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:48.663820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.250127+0000","last_change":"2024-09-10T08:44:10.678254+0000","last_active":"2024-09-10T08:44:21.250127+0000","last_peered":"2024-09-10T08:44:21.250127+0000","last_clean":"2024-09-10T08:44:21.250127+0000","last_became_active":"2024-09-10T08:44:10.677891+0000","last_became_peered":"2024-09-10T08:44:10.677891+0000","last_unstale":"2024-09-10T08:44:21.250127+0000","last_undegraded":"2024-09-10T08:44:21.250127+0000","last_fullsized":"2024-09-10T08:44:21.250127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:32:43.399049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.232170+0000","last_change":"2024-09-10T08:44:08.656315+0000","last_active":"2024-09-10T08:44:54.232170+0000","last_peered":"2024-09-10T08:44:54.232170+0000","last_clean":"2024-09-10T08:44:54.232170+0000","last_became_active":"2024-09-10T08:44:08.655978+0000","last_became_peered":"2024-09-10T08:44:08.655978+0000","last_unstale":"2024-09-10T08:44:54.232170+0000","last_undegraded":"2024-09-10T08:44:54.232170+0000","last_fullsized":"2024-09-10T08:44:54.232170+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:15.025354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990146+0000","last_change":"2024-09-10T08:44:17.726789+0000","last_active":"2024-09-10T08:44:20.990146+0000","last_peered":"2024-09-10T08:44:20.990146+0000","last_clean":"2024-09-10T08:44:20.990146+0000","last_became_active":"2024-09-10T08:44:17.726450+0000","last_became_peered":"2024-09-10T08:44:17.726450+0000","last_unstale":"2024-09-10T08:44:20.990146+0000","last_undegraded":"2024-09-10T08:44:20.990146+0000","last_fullsized":"2024-09-10T08:44:20.990146+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:03.899822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991128+0000","last_change":"2024-09-10T08:44:10.677359+0000","last_active":"2024-09-10T08:44:20.991128+0000","last_peered":"2024-09-10T08:44:20.991128+0000","last_clean":"2024-09-10T08:44:20.991128+0000","last_became_active":"2024-09-10T08:44:10.676782+0000","last_became_peered":"2024-09-10T08:44:10.676782+0000","last_unstale":"2024-09-10T08:44:20.991128+0000","last_undegraded":"2024-09-10T08:44:20.991128+0000","last_fullsized":"2024-09-10T08:44:20.991128+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:31.721714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.232479+0000","last_change":"2024-09-10T08:44:08.656993+0000","last_active":"2024-09-10T08:44:54.232479+0000","last_peered":"2024-09-10T08:44:54.232479+0000","last_clean":"2024-09-10T08:44:54.232479+0000","last_became_active":"2024-09-10T08:44:08.656571+0000","last_became_peered":"2024-09-10T08:44:08.656571+0000","last_unstale":"2024-09-10T08:44:54.232479+0000","last_undegraded":"2024-09-10T08:44:54.232479+0000","last_fullsized":"2024-09-10T08:44:54.232479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:32.202740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991605+0000","last_change":"2024-09-10T08:44:17.725674+0000","last_active":"2024-09-10T08:44:20.991605+0000","last_peered":"2024-09-10T08:44:20.991605+0000","last_clean":"2024-09-10T08:44:20.991605+0000","last_became_active":"2024-09-10T08:44:17.725263+0000","last_became_peered":"2024-09-10T08:44:17.725263+0000","last_unstale":"2024-09-10T08:44:20.991605+0000","last_undegraded":"2024-09-10T08:44:20.991605+0000","last_fullsized":"2024-09-10T08:44:20.991605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:18.183157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990960+0000","last_change":"2024-09-10T08:44:10.676871+0000","last_active":"2024-09-10T08:44:20.990960+0000","last_peered":"2024-09-10T08:44:20.990960+0000","last_clean":"2024-09-10T08:44:20.990960+0000","last_became_active":"2024-09-10T08:44:10.676540+0000","last_became_peered":"2024-09-10T08:44:10.676540+0000","last_unstale":"2024-09-10T08:44:20.990960+0000","last_undegraded":"2024-09-10T08:44:20.990960+0000","last_fullsized":"2024-09-10T08:44:20.990960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:50.964100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'27","reported_seq":86,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:04.232255+0000","last_change":"2024-09-10T08:44:08.660113+0000","last_active":"2024-09-10T08:45:04.232255+0000","last_peered":"2024-09-10T08:45:04.232255+0000","last_clean":"2024-09-10T08:45:04.232255+0000","last_became_active":"2024-09-10T08:44:08.659644+0000","last_became_peered":"2024-09-10T08:44:08.659644+0000","last_unstale":"2024-09-10T08:45:04.232255+0000","last_undegraded":"2024-09-10T08:45:04.232255+0000","last_fullsized":"2024-09-10T08:45:04.232255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:01.605344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":57,"num_write":29,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717369+0000","last_change":"2024-09-10T08:44:17.723653+0000","last_active":"2024-09-10T08:44:20.717369+0000","last_peered":"2024-09-10T08:44:20.717369+0000","last_clean":"2024-09-10T08:44:20.717369+0000","last_became_active":"2024-09-10T08:44:17.723284+0000","last_became_peered":"2024-09-10T08:44:17.723284+0000","last_unstale":"2024-09-10T08:44:20.717369+0000","last_undegraded":"2024-09-10T08:44:20.717369+0000","last_fullsized":"2024-09-10T08:44:20.717369+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:17.162526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990465+0000","last_change":"2024-09-10T08:44:10.681837+0000","last_active":"2024-09-10T08:44:20.990465+0000","last_peered":"2024-09-10T08:44:20.990465+0000","last_clean":"2024-09-10T08:44:20.990465+0000","last_became_active":"2024-09-10T08:44:10.681518+0000","last_became_peered":"2024-09-10T08:44:10.681518+0000","last_unstale":"2024-09-10T08:44:20.990465+0000","last_undegraded":"2024-09-10T08:44:20.990465+0000","last_fullsized":"2024-09-10T08:44:20.990465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:10.004215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:53.215205+0000","last_change":"2024-09-10T08:44:09.050394+0000","last_active":"2024-09-10T08:44:53.215205+0000","last_peered":"2024-09-10T08:44:53.215205+0000","last_clean":"2024-09-10T08:44:53.215205+0000","last_became_active":"2024-09-10T08:44:09.049969+0000","last_became_peered":"2024-09-10T08:44:09.049969+0000","last_unstale":"2024-09-10T08:44:53.215205+0000","last_undegraded":"2024-09-10T08:44:53.215205+0000","last_fullsized":"2024-09-10T08:44:53.215205+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:43.519246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990437+0000","last_change":"2024-09-10T08:44:17.720318+0000","last_active":"2024-09-10T08:44:20.990437+0000","last_peered":"2024-09-10T08:44:20.990437+0000","last_clean":"2024-09-10T08:44:20.990437+0000","last_became_active":"2024-09-10T08:44:17.719598+0000","last_became_peered":"2024-09-10T08:44:17.719598+0000","last_unstale":"2024-09-10T08:44:20.990437+0000","last_undegraded":"2024-09-10T08:44:20.990437+0000","last_fullsized":"2024-09-10T08:44:20.990437+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:15:49.520324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990679+0000","last_change":"2024-09-10T08:44:10.677340+0000","last_active":"2024-09-10T08:44:20.990679+0000","last_peered":"2024-09-10T08:44:20.990679+0000","last_clean":"2024-09-10T08:44:20.990679+0000","last_became_active":"2024-09-10T08:44:10.676628+0000","last_became_peered":"2024-09-10T08:44:10.676628+0000","last_unstale":"2024-09-10T08:44:20.990679+0000","last_undegraded":"2024-09-10T08:44:20.990679+0000","last_fullsized":"2024-09-10T08:44:20.990679+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:59:17.924217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.231825+0000","last_change":"2024-09-10T08:44:08.660254+0000","last_active":"2024-09-10T08:44:54.231825+0000","last_peered":"2024-09-10T08:44:54.231825+0000","last_clean":"2024-09-10T08:44:54.231825+0000","last_became_active":"2024-09-10T08:44:08.659667+0000","last_became_peered":"2024-09-10T08:44:08.659667+0000","last_unstale":"2024-09-10T08:44:54.231825+0000","last_undegraded":"2024-09-10T08:44:54.231825+0000","last_fullsized":"2024-09-10T08:44:54.231825+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:49:49.146867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717129+0000","last_change":"2024-09-10T08:44:17.722373+0000","last_active":"2024-09-10T08:44:20.717129+0000","last_peered":"2024-09-10T08:44:20.717129+0000","last_clean":"2024-09-10T08:44:20.717129+0000","last_became_active":"2024-09-10T08:44:17.722035+0000","last_became_peered":"2024-09-10T08:44:17.722035+0000","last_unstale":"2024-09-10T08:44:20.717129+0000","last_undegraded":"2024-09-10T08:44:20.717129+0000","last_fullsized":"2024-09-10T08:44:20.717129+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:30:36.187364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990791+0000","last_change":"2024-09-10T08:44:10.672585+0000","last_active":"2024-09-10T08:44:20.990791+0000","last_peered":"2024-09-10T08:44:20.990791+0000","last_clean":"2024-09-10T08:44:20.990791+0000","last_became_active":"2024-09-10T08:44:10.671987+0000","last_became_peered":"2024-09-10T08:44:10.671987+0000","last_unstale":"2024-09-10T08:44:20.990791+0000","last_undegraded":"2024-09-10T08:44:20.990791+0000","last_fullsized":"2024-09-10T08:44:20.990791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:48.660750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":56,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:04.230080+0000","last_change":"2024-09-10T08:44:08.658069+0000","last_active":"2024-09-10T08:45:04.230080+0000","last_peered":"2024-09-10T08:45:04.230080+0000","last_clean":"2024-09-10T08:45:04.230080+0000","last_became_active":"2024-09-10T08:44:08.657702+0000","last_became_peered":"2024-09-10T08:44:08.657702+0000","last_unstale":"2024-09-10T08:45:04.230080+0000","last_undegraded":"2024-09-10T08:45:04.230080+0000","last_fullsized":"2024-09-10T08:45:04.230080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:17.383267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990270+0000","last_change":"2024-09-10T08:44:17.717984+0000","last_active":"2024-09-10T08:44:20.990270+0000","last_peered":"2024-09-10T08:44:20.990270+0000","last_clean":"2024-09-10T08:44:20.990270+0000","last_became_active":"2024-09-10T08:44:17.717581+0000","last_became_peered":"2024-09-10T08:44:17.717581+0000","last_unstale":"2024-09-10T08:44:20.990270+0000","last_undegraded":"2024-09-10T08:44:20.990270+0000","last_fullsized":"2024-09-10T08:44:20.990270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:25.330732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989998+0000","last_change":"2024-09-10T08:44:17.721244+0000","last_active":"2024-09-10T08:44:20.989998+0000","last_peered":"2024-09-10T08:44:20.989998+0000","last_clean":"2024-09-10T08:44:20.989998+0000","last_became_active":"2024-09-10T08:44:17.720884+0000","last_became_peered":"2024-09-10T08:44:17.720884+0000","last_unstale":"2024-09-10T08:44:20.989998+0000","last_undegraded":"2024-09-10T08:44:20.989998+0000","last_fullsized":"2024-09-10T08:44:20.989998+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.769513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717107+0000","last_change":"2024-09-10T08:44:10.667558+0000","last_active":"2024-09-10T08:44:20.717107+0000","last_peered":"2024-09-10T08:44:20.717107+0000","last_clean":"2024-09-10T08:44:20.717107+0000","last_became_active":"2024-09-10T08:44:10.667214+0000","last_became_peered":"2024-09-10T08:44:10.667214+0000","last_unstale":"2024-09-10T08:44:20.717107+0000","last_undegraded":"2024-09-10T08:44:20.717107+0000","last_fullsized":"2024-09-10T08:44:20.717107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:53.630790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:54.232150+0000","last_change":"2024-09-10T08:44:08.657930+0000","last_active":"2024-09-10T08:44:54.232150+0000","last_peered":"2024-09-10T08:44:54.232150+0000","last_clean":"2024-09-10T08:44:54.232150+0000","last_became_active":"2024-09-10T08:44:08.657611+0000","last_became_peered":"2024-09-10T08:44:08.657611+0000","last_unstale":"2024-09-10T08:44:54.232150+0000","last_undegraded":"2024-09-10T08:44:54.232150+0000","last_fullsized":"2024-09-10T08:44:54.232150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:36.923786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991280+0000","last_change":"2024-09-10T08:44:17.726106+0000","last_active":"2024-09-10T08:44:20.991280+0000","last_peered":"2024-09-10T08:44:20.991280+0000","last_clean":"2024-09-10T08:44:20.991280+0000","last_became_active":"2024-09-10T08:44:17.725683+0000","last_became_peered":"2024-09-10T08:44:17.725683+0000","last_unstale":"2024-09-10T08:44:20.991280+0000","last_undegraded":"2024-09-10T08:44:20.991280+0000","last_fullsized":"2024-09-10T08:44:20.991280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:23.392794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989651+0000","last_change":"2024-09-10T08:44:10.672204+0000","last_active":"2024-09-10T08:44:20.989651+0000","last_peered":"2024-09-10T08:44:20.989651+0000","last_clean":"2024-09-10T08:44:20.989651+0000","last_became_active":"2024-09-10T08:44:10.671898+0000","last_became_peered":"2024-09-10T08:44:10.671898+0000","last_unstale":"2024-09-10T08:44:20.989651+0000","last_undegraded":"2024-09-10T08:44:20.989651+0000","last_fullsized":"2024-09-10T08:44:20.989651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:03.737357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:52.265222+0000","last_change":"2024-09-10T08:44:08.656620+0000","last_active":"2024-09-10T08:44:52.265222+0000","last_peered":"2024-09-10T08:44:52.265222+0000","last_clean":"2024-09-10T08:44:52.265222+0000","last_became_active":"2024-09-10T08:44:08.656306+0000","last_became_peered":"2024-09-10T08:44:08.656306+0000","last_unstale":"2024-09-10T08:44:52.265222+0000","last_undegraded":"2024-09-10T08:44:52.265222+0000","last_fullsized":"2024-09-10T08:44:52.265222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:57.081909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:53.223364+0000","last_change":"2024-09-10T08:44:08.660178+0000","last_active":"2024-09-10T08:44:53.223364+0000","last_peered":"2024-09-10T08:44:53.223364+0000","last_clean":"2024-09-10T08:44:53.223364+0000","last_became_active":"2024-09-10T08:44:08.659839+0000","last_became_peered":"2024-09-10T08:44:08.659839+0000","last_unstale":"2024-09-10T08:44:53.223364+0000","last_undegraded":"2024-09-10T08:44:53.223364+0000","last_fullsized":"2024-09-10T08:44:53.223364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:17.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991349+0000","last_change":"2024-09-10T08:44:10.683683+0000","last_active":"2024-09-10T08:44:20.991349+0000","last_peered":"2024-09-10T08:44:20.991349+0000","last_clean":"2024-09-10T08:44:20.991349+0000","last_became_active":"2024-09-10T08:44:10.683337+0000","last_became_peered":"2024-09-10T08:44:10.683337+0000","last_unstale":"2024-09-10T08:44:20.991349+0000","last_undegraded":"2024-09-10T08:44:20.991349+0000","last_fullsized":"2024-09-10T08:44:20.991349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:54.863098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717904+0000","last_change":"2024-09-10T08:44:17.721602+0000","last_active":"2024-09-10T08:44:20.717904+0000","last_peered":"2024-09-10T08:44:20.717904+0000","last_clean":"2024-09-10T08:44:20.717904+0000","last_became_active":"2024-09-10T08:44:17.721194+0000","last_became_peered":"2024-09-10T08:44:17.721194+0000","last_unstale":"2024-09-10T08:44:20.717904+0000","last_undegraded":"2024-09-10T08:44:20.717904+0000","last_fullsized":"2024-09-10T08:44:20.717904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:39.298494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:45:03.229253+0000","last_change":"2024-09-10T08:44:08.661943+0000","last_active":"2024-09-10T08:45:03.229253+0000","last_peered":"2024-09-10T08:45:03.229253+0000","last_clean":"2024-09-10T08:45:03.229253+0000","last_became_active":"2024-09-10T08:44:08.661587+0000","last_became_peered":"2024-09-10T08:44:08.661587+0000","last_unstale":"2024-09-10T08:45:03.229253+0000","last_undegraded":"2024-09-10T08:45:03.229253+0000","last_fullsized":"2024-09-10T08:45:03.229253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:05.771641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990228+0000","last_change":"2024-09-10T08:44:10.673522+0000","last_active":"2024-09-10T08:44:20.990228+0000","last_peered":"2024-09-10T08:44:20.990228+0000","last_clean":"2024-09-10T08:44:20.990228+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:44:20.990228+0000","last_undegraded":"2024-09-10T08:44:20.990228+0000","last_fullsized":"2024-09-10T08:44:20.990228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:03.006144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.718054+0000","last_change":"2024-09-10T08:44:17.721306+0000","last_active":"2024-09-10T08:44:20.718054+0000","last_peered":"2024-09-10T08:44:20.718054+0000","last_clean":"2024-09-10T08:44:20.718054+0000","last_became_active":"2024-09-10T08:44:17.720401+0000","last_became_peered":"2024-09-10T08:44:17.720401+0000","last_unstale":"2024-09-10T08:44:20.718054+0000","last_undegraded":"2024-09-10T08:44:20.718054+0000","last_fullsized":"2024-09-10T08:44:20.718054+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:11.589371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990052+0000","last_change":"2024-09-10T08:44:10.678425+0000","last_active":"2024-09-10T08:44:20.990052+0000","last_peered":"2024-09-10T08:44:20.990052+0000","last_clean":"2024-09-10T08:44:20.990052+0000","last_became_active":"2024-09-10T08:44:10.677187+0000","last_became_peered":"2024-09-10T08:44:10.677187+0000","last_unstale":"2024-09-10T08:44:20.990052+0000","last_undegraded":"2024-09-10T08:44:20.990052+0000","last_fullsized":"2024-09-10T08:44:20.990052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:06.527530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.718245+0000","last_change":"2024-09-10T08:44:10.674178+0000","last_active":"2024-09-10T08:44:20.718245+0000","last_peered":"2024-09-10T08:44:20.718245+0000","last_clean":"2024-09-10T08:44:20.718245+0000","last_became_active":"2024-09-10T08:44:10.673843+0000","last_became_peered":"2024-09-10T08:44:10.673843+0000","last_unstale":"2024-09-10T08:44:20.718245+0000","last_undegraded":"2024-09-10T08:44:20.718245+0000","last_fullsized":"2024-09-10T08:44:20.718245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:35.023663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717659+0000","last_change":"2024-09-10T08:44:10.673271+0000","last_active":"2024-09-10T08:44:20.717659+0000","last_peered":"2024-09-10T08:44:20.717659+0000","last_clean":"2024-09-10T08:44:20.717659+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:44:20.717659+0000","last_undegraded":"2024-09-10T08:44:20.717659+0000","last_fullsized":"2024-09-10T08:44:20.717659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:21:26.606820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989465+0000","last_change":"2024-09-10T08:44:10.671348+0000","last_active":"2024-09-10T08:44:20.989465+0000","last_peered":"2024-09-10T08:44:20.989465+0000","last_clean":"2024-09-10T08:44:20.989465+0000","last_became_active":"2024-09-10T08:44:10.670623+0000","last_became_peered":"2024-09-10T08:44:10.670623+0000","last_unstale":"2024-09-10T08:44:20.989465+0000","last_undegraded":"2024-09-10T08:44:20.989465+0000","last_fullsized":"2024-09-10T08:44:20.989465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:06.145694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991259+0000","last_change":"2024-09-10T08:44:10.666011+0000","last_active":"2024-09-10T08:44:20.991259+0000","last_peered":"2024-09-10T08:44:20.991259+0000","last_clean":"2024-09-10T08:44:20.991259+0000","last_became_active":"2024-09-10T08:44:10.665664+0000","last_became_peered":"2024-09-10T08:44:10.665664+0000","last_unstale":"2024-09-10T08:44:20.991259+0000","last_undegraded":"2024-09-10T08:44:20.991259+0000","last_fullsized":"2024-09-10T08:44:20.991259+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:31.542102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990749+0000","last_change":"2024-09-10T08:44:10.675960+0000","last_active":"2024-09-10T08:44:20.990749+0000","last_peered":"2024-09-10T08:44:20.990749+0000","last_clean":"2024-09-10T08:44:20.990749+0000","last_became_active":"2024-09-10T08:44:10.675587+0000","last_became_peered":"2024-09-10T08:44:10.675587+0000","last_unstale":"2024-09-10T08:44:20.990749+0000","last_undegraded":"2024-09-10T08:44:20.990749+0000","last_fullsized":"2024-09-10T08:44:20.990749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:54.634619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.718259+0000","last_change":"2024-09-10T08:44:10.681094+0000","last_active":"2024-09-10T08:44:20.718259+0000","last_peered":"2024-09-10T08:44:20.718259+0000","last_clean":"2024-09-10T08:44:20.718259+0000","last_became_active":"2024-09-10T08:44:10.678585+0000","last_became_peered":"2024-09-10T08:44:10.678585+0000","last_unstale":"2024-09-10T08:44:20.718259+0000","last_undegraded":"2024-09-10T08:44:20.718259+0000","last_fullsized":"2024-09-10T08:44:20.718259+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:14.951949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989971+0000","last_change":"2024-09-10T08:44:10.675473+0000","last_active":"2024-09-10T08:44:20.989971+0000","last_peered":"2024-09-10T08:44:20.989971+0000","last_clean":"2024-09-10T08:44:20.989971+0000","last_became_active":"2024-09-10T08:44:10.674155+0000","last_became_peered":"2024-09-10T08:44:10.674155+0000","last_unstale":"2024-09-10T08:44:20.989971+0000","last_undegraded":"2024-09-10T08:44:20.989971+0000","last_fullsized":"2024-09-10T08:44:20.989971+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:55.940235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990620+0000","last_change":"2024-09-10T08:44:10.680162+0000","last_active":"2024-09-10T08:44:20.990620+0000","last_peered":"2024-09-10T08:44:20.990620+0000","last_clean":"2024-09-10T08:44:20.990620+0000","last_became_active":"2024-09-10T08:44:10.679800+0000","last_became_peered":"2024-09-10T08:44:10.679800+0000","last_unstale":"2024-09-10T08:44:20.990620+0000","last_undegraded":"2024-09-10T08:44:20.990620+0000","last_fullsized":"2024-09-10T08:44:20.990620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:12.319813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990420+0000","last_change":"2024-09-10T08:44:10.674724+0000","last_active":"2024-09-10T08:44:20.990420+0000","last_peered":"2024-09-10T08:44:20.990420+0000","last_clean":"2024-09-10T08:44:20.990420+0000","last_became_active":"2024-09-10T08:44:10.674389+0000","last_became_peered":"2024-09-10T08:44:10.674389+0000","last_unstale":"2024-09-10T08:44:20.990420+0000","last_undegraded":"2024-09-10T08:44:20.990420+0000","last_fullsized":"2024-09-10T08:44:20.990420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:52.123186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989869+0000","last_change":"2024-09-10T08:44:10.672805+0000","last_active":"2024-09-10T08:44:20.989869+0000","last_peered":"2024-09-10T08:44:20.989869+0000","last_clean":"2024-09-10T08:44:20.989869+0000","last_became_active":"2024-09-10T08:44:10.672508+0000","last_became_peered":"2024-09-10T08:44:10.672508+0000","last_unstale":"2024-09-10T08:44:20.989869+0000","last_undegraded":"2024-09-10T08:44:20.989869+0000","last_fullsized":"2024-09-10T08:44:20.989869+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:48.558633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990759+0000","last_change":"2024-09-10T08:44:10.677504+0000","last_active":"2024-09-10T08:44:20.990759+0000","last_peered":"2024-09-10T08:44:20.990759+0000","last_clean":"2024-09-10T08:44:20.990759+0000","last_became_active":"2024-09-10T08:44:10.676822+0000","last_became_peered":"2024-09-10T08:44:10.676822+0000","last_unstale":"2024-09-10T08:44:20.990759+0000","last_undegraded":"2024-09-10T08:44:20.990759+0000","last_fullsized":"2024-09-10T08:44:20.990759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:05:48.293333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990075+0000","last_change":"2024-09-10T08:44:10.673181+0000","last_active":"2024-09-10T08:44:20.990075+0000","last_peered":"2024-09-10T08:44:20.990075+0000","last_clean":"2024-09-10T08:44:20.990075+0000","last_became_active":"2024-09-10T08:44:10.672851+0000","last_became_peered":"2024-09-10T08:44:10.672851+0000","last_unstale":"2024-09-10T08:44:20.990075+0000","last_undegraded":"2024-09-10T08:44:20.990075+0000","last_fullsized":"2024-09-10T08:44:20.990075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:42.562229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.718225+0000","last_change":"2024-09-10T08:44:10.670615+0000","last_active":"2024-09-10T08:44:20.718225+0000","last_peered":"2024-09-10T08:44:20.718225+0000","last_clean":"2024-09-10T08:44:20.718225+0000","last_became_active":"2024-09-10T08:44:10.670032+0000","last_became_peered":"2024-09-10T08:44:10.670032+0000","last_unstale":"2024-09-10T08:44:20.718225+0000","last_undegraded":"2024-09-10T08:44:20.718225+0000","last_fullsized":"2024-09-10T08:44:20.718225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:31.078788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717634+0000","last_change":"2024-09-10T08:44:10.672367+0000","last_active":"2024-09-10T08:44:20.717634+0000","last_peered":"2024-09-10T08:44:20.717634+0000","last_clean":"2024-09-10T08:44:20.717634+0000","last_became_active":"2024-09-10T08:44:10.672040+0000","last_became_peered":"2024-09-10T08:44:10.672040+0000","last_unstale":"2024-09-10T08:44:20.717634+0000","last_undegraded":"2024-09-10T08:44:20.717634+0000","last_fullsized":"2024-09-10T08:44:20.717634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:10.634708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991593+0000","last_change":"2024-09-10T08:44:10.676375+0000","last_active":"2024-09-10T08:44:20.991593+0000","last_peered":"2024-09-10T08:44:20.991593+0000","last_clean":"2024-09-10T08:44:20.991593+0000","last_became_active":"2024-09-10T08:44:10.676022+0000","last_became_peered":"2024-09-10T08:44:10.676022+0000","last_unstale":"2024-09-10T08:44:20.991593+0000","last_undegraded":"2024-09-10T08:44:20.991593+0000","last_fullsized":"2024-09-10T08:44:20.991593+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:02.681954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989558+0000","last_change":"2024-09-10T08:44:10.673378+0000","last_active":"2024-09-10T08:44:20.989558+0000","last_peered":"2024-09-10T08:44:20.989558+0000","last_clean":"2024-09-10T08:44:20.989558+0000","last_became_active":"2024-09-10T08:44:10.673089+0000","last_became_peered":"2024-09-10T08:44:10.673089+0000","last_unstale":"2024-09-10T08:44:20.989558+0000","last_undegraded":"2024-09-10T08:44:20.989558+0000","last_fullsized":"2024-09-10T08:44:20.989558+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:36:19.258651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989429+0000","last_change":"2024-09-10T08:44:10.671557+0000","last_active":"2024-09-10T08:44:20.989429+0000","last_peered":"2024-09-10T08:44:20.989429+0000","last_clean":"2024-09-10T08:44:20.989429+0000","last_became_active":"2024-09-10T08:44:10.671248+0000","last_became_peered":"2024-09-10T08:44:10.671248+0000","last_unstale":"2024-09-10T08:44:20.989429+0000","last_undegraded":"2024-09-10T08:44:20.989429+0000","last_fullsized":"2024-09-10T08:44:20.989429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:54.261525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989628+0000","last_change":"2024-09-10T08:44:10.674672+0000","last_active":"2024-09-10T08:44:20.989628+0000","last_peered":"2024-09-10T08:44:20.989628+0000","last_clean":"2024-09-10T08:44:20.989628+0000","last_became_active":"2024-09-10T08:44:10.673415+0000","last_became_peered":"2024-09-10T08:44:10.673415+0000","last_unstale":"2024-09-10T08:44:20.989628+0000","last_undegraded":"2024-09-10T08:44:20.989628+0000","last_fullsized":"2024-09-10T08:44:20.989628+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:59:14.179664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990654+0000","last_change":"2024-09-10T08:44:10.675293+0000","last_active":"2024-09-10T08:44:20.990654+0000","last_peered":"2024-09-10T08:44:20.990654+0000","last_clean":"2024-09-10T08:44:20.990654+0000","last_became_active":"2024-09-10T08:44:10.674924+0000","last_became_peered":"2024-09-10T08:44:10.674924+0000","last_unstale":"2024-09-10T08:44:20.990654+0000","last_undegraded":"2024-09-10T08:44:20.990654+0000","last_fullsized":"2024-09-10T08:44:20.990654+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:56.027320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990608+0000","last_change":"2024-09-10T08:44:10.674605+0000","last_active":"2024-09-10T08:44:20.990608+0000","last_peered":"2024-09-10T08:44:20.990608+0000","last_clean":"2024-09-10T08:44:20.990608+0000","last_became_active":"2024-09-10T08:44:10.674057+0000","last_became_peered":"2024-09-10T08:44:10.674057+0000","last_unstale":"2024-09-10T08:44:20.990608+0000","last_undegraded":"2024-09-10T08:44:20.990608+0000","last_fullsized":"2024-09-10T08:44:20.990608+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:09.473003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.250288+0000","last_change":"2024-09-10T08:44:10.677410+0000","last_active":"2024-09-10T08:44:21.250288+0000","last_peered":"2024-09-10T08:44:21.250288+0000","last_clean":"2024-09-10T08:44:21.250288+0000","last_became_active":"2024-09-10T08:44:10.677109+0000","last_became_peered":"2024-09-10T08:44:10.677109+0000","last_unstale":"2024-09-10T08:44:21.250288+0000","last_undegraded":"2024-09-10T08:44:21.250288+0000","last_fullsized":"2024-09-10T08:44:21.250288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:45.455425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.718277+0000","last_change":"2024-09-10T08:44:10.669706+0000","last_active":"2024-09-10T08:44:20.718277+0000","last_peered":"2024-09-10T08:44:20.718277+0000","last_clean":"2024-09-10T08:44:20.718277+0000","last_became_active":"2024-09-10T08:44:10.669440+0000","last_became_peered":"2024-09-10T08:44:10.669440+0000","last_unstale":"2024-09-10T08:44:20.718277+0000","last_undegraded":"2024-09-10T08:44:20.718277+0000","last_fullsized":"2024-09-10T08:44:20.718277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:31:55.610651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990892+0000","last_change":"2024-09-10T08:44:10.676482+0000","last_active":"2024-09-10T08:44:20.990892+0000","last_peered":"2024-09-10T08:44:20.990892+0000","last_clean":"2024-09-10T08:44:20.990892+0000","last_became_active":"2024-09-10T08:44:10.675983+0000","last_became_peered":"2024-09-10T08:44:10.675983+0000","last_unstale":"2024-09-10T08:44:20.990892+0000","last_undegraded":"2024-09-10T08:44:20.990892+0000","last_fullsized":"2024-09-10T08:44:20.990892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:01.269283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990130+0000","last_change":"2024-09-10T08:44:10.675681+0000","last_active":"2024-09-10T08:44:20.990130+0000","last_peered":"2024-09-10T08:44:20.990130+0000","last_clean":"2024-09-10T08:44:20.990130+0000","last_became_active":"2024-09-10T08:44:10.675357+0000","last_became_peered":"2024-09-10T08:44:10.675357+0000","last_unstale":"2024-09-10T08:44:20.990130+0000","last_undegraded":"2024-09-10T08:44:20.990130+0000","last_fullsized":"2024-09-10T08:44:20.990130+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:26.138333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:21.249812+0000","last_change":"2024-09-10T08:44:10.680819+0000","last_active":"2024-09-10T08:44:21.249812+0000","last_peered":"2024-09-10T08:44:21.249812+0000","last_clean":"2024-09-10T08:44:21.249812+0000","last_became_active":"2024-09-10T08:44:10.680491+0000","last_became_peered":"2024-09-10T08:44:10.680491+0000","last_unstale":"2024-09-10T08:44:21.249812+0000","last_undegraded":"2024-09-10T08:44:21.249812+0000","last_fullsized":"2024-09-10T08:44:21.249812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:17.044776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.717783+0000","last_change":"2024-09-10T08:44:10.667376+0000","last_active":"2024-09-10T08:44:20.717783+0000","last_peered":"2024-09-10T08:44:20.717783+0000","last_clean":"2024-09-10T08:44:20.717783+0000","last_became_active":"2024-09-10T08:44:10.666648+0000","last_became_peered":"2024-09-10T08:44:10.666648+0000","last_unstale":"2024-09-10T08:44:20.717783+0000","last_undegraded":"2024-09-10T08:44:20.717783+0000","last_fullsized":"2024-09-10T08:44:20.717783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:33:12.017550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989959+0000","last_change":"2024-09-10T08:44:10.668665+0000","last_active":"2024-09-10T08:44:20.989959+0000","last_peered":"2024-09-10T08:44:20.989959+0000","last_clean":"2024-09-10T08:44:20.989959+0000","last_became_active":"2024-09-10T08:44:10.666031+0000","last_became_peered":"2024-09-10T08:44:10.666031+0000","last_unstale":"2024-09-10T08:44:20.989959+0000","last_undegraded":"2024-09-10T08:44:20.989959+0000","last_fullsized":"2024-09-10T08:44:20.989959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:39:53.991463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990505+0000","last_change":"2024-09-10T08:44:10.676495+0000","last_active":"2024-09-10T08:44:20.990505+0000","last_peered":"2024-09-10T08:44:20.990505+0000","last_clean":"2024-09-10T08:44:20.990505+0000","last_became_active":"2024-09-10T08:44:10.676087+0000","last_became_peered":"2024-09-10T08:44:10.676087+0000","last_unstale":"2024-09-10T08:44:20.990505+0000","last_undegraded":"2024-09-10T08:44:20.990505+0000","last_fullsized":"2024-09-10T08:44:20.990505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:41.828157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.989398+0000","last_change":"2024-09-10T08:44:10.672835+0000","last_active":"2024-09-10T08:44:20.989398+0000","last_peered":"2024-09-10T08:44:20.989398+0000","last_clean":"2024-09-10T08:44:20.989398+0000","last_became_active":"2024-09-10T08:44:10.672537+0000","last_became_peered":"2024-09-10T08:44:10.672537+0000","last_unstale":"2024-09-10T08:44:20.989398+0000","last_undegraded":"2024-09-10T08:44:20.989398+0000","last_fullsized":"2024-09-10T08:44:20.989398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:06.121460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.990108+0000","last_change":"2024-09-10T08:44:10.675968+0000","last_active":"2024-09-10T08:44:20.990108+0000","last_peered":"2024-09-10T08:44:20.990108+0000","last_clean":"2024-09-10T08:44:20.990108+0000","last_became_active":"2024-09-10T08:44:10.675628+0000","last_became_peered":"2024-09-10T08:44:10.675628+0000","last_unstale":"2024-09-10T08:44:20.990108+0000","last_undegraded":"2024-09-10T08:44:20.990108+0000","last_fullsized":"2024-09-10T08:44:20.990108+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:40.261336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T08:44:20.991537+0000","last_change":"2024-09-10T08:44:10.673151+0000","last_active":"2024-09-10T08:44:20.991537+0000","last_peered":"2024-09-10T08:44:20.991537+0000","last_clean":"2024-09-10T08:44:20.991537+0000","last_became_active":"2024-09-10T08:44:10.672833+0000","last_became_peered":"2024-09-10T08:44:10.672833+0000","last_unstale":"2024-09-10T08:44:20.991537+0000","last_undegraded":"2024-09-10T08:44:20.991537+0000","last_fullsized":"2024-09-10T08:44:20.991537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:38.556222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50160,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":159,"num_read_kb":137,"num_write":233,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":176118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514520,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":485254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743453,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1667072,"data_stored":1115022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906978,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":488887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":485169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1081275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":480295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36924,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708228,"statfs":{"total":95995035648,"available":95957225472,"internally_reserved":0,"allocated":1617920,"data_stored":1076857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920827,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":484369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":480003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":538102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":537489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":482144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":57916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":45081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:45:11.594 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T08:45:11.594 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T08:45:11.595 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T08:45:11.852 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:12.373 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:12 smithi172 ceph-mon[30854]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:45:12.373 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:12 smithi172 ceph-mon[30854]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 14 op/s 2024-09-10T08:45:12.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.364+0000 7f2a78d98640 1 -- 172.21.15.172:0/2224872210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74107920 msgr2=0x7f2a74109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:12.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.364+0000 7f2a78d98640 1 --2- 172.21.15.172:0/2224872210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74107920 0x7f2a74109d10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009ca0 tx=0x7f2a6c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.365+0000 7f2a78d98640 1 -- 172.21.15.172:0/2224872210 shutdown_connections 2024-09-10T08:45:12.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.365+0000 7f2a78d98640 1 --2- 172.21.15.172:0/2224872210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74107920 0x7f2a74109d10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.365+0000 7f2a78d98640 1 --2- 172.21.15.172:0/2224872210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a74104ff0 0x7f2a741073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.365+0000 7f2a78d98640 1 --2- 172.21.15.172:0/2224872210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a741026c0 0x7f2a74104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.365+0000 7f2a78d98640 1 -- 172.21.15.172:0/2224872210 >> 172.21.15.172:0/2224872210 conn(0x7f2a740fc280 msgr2=0x7f2a740fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:12.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.365+0000 7f2a78d98640 1 -- 172.21.15.172:0/2224872210 shutdown_connections 2024-09-10T08:45:12.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.365+0000 7f2a78d98640 1 -- 172.21.15.172:0/2224872210 wait complete. 2024-09-10T08:45:12.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.365+0000 7f2a78d98640 1 Processor -- start 2024-09-10T08:45:12.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a78d98640 1 -- start start 2024-09-10T08:45:12.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a78d98640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a741026c0 0x7f2a74197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a78d98640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a74104ff0 0x7f2a741982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a78d98640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74107920 0x7f2a7419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a78d98640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7410d710 con 0x7f2a74107920 2024-09-10T08:45:12.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a78d98640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7410d590 con 0x7f2a741026c0 2024-09-10T08:45:12.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a78d98640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7410d890 con 0x7f2a74104ff0 2024-09-10T08:45:12.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a72d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74107920 0x7f2a7419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a72d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74107920 0x7f2a7419f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48398/0 (socket says 172.21.15.172:48398) 2024-09-10T08:45:12.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a72d9d640 1 -- 172.21.15.172:0/1697775603 learned_addr learned my addr 172.21.15.172:0/1697775603 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:12.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a7259c640 1 --2- 172.21.15.172:0/1697775603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a741026c0 0x7f2a74197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a71d9b640 1 --2- 172.21.15.172:0/1697775603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a74104ff0 0x7f2a741982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a72d9d640 1 -- 172.21.15.172:0/1697775603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a74104ff0 msgr2=0x7f2a741982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:12.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a72d9d640 1 --2- 172.21.15.172:0/1697775603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a74104ff0 0x7f2a741982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a72d9d640 1 -- 172.21.15.172:0/1697775603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a741026c0 msgr2=0x7f2a74197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:12.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a72d9d640 1 --2- 172.21.15.172:0/1697775603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a741026c0 0x7f2a74197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a72d9d640 1 -- 172.21.15.172:0/1697775603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7419f9b0 con 0x7f2a74107920 2024-09-10T08:45:12.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a7259c640 1 --2- 172.21.15.172:0/1697775603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a741026c0 0x7f2a74197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:12.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.366+0000 7f2a72d9d640 1 --2- 172.21.15.172:0/1697775603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74107920 0x7f2a7419f310 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009dc0 tx=0x7f2a6c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:12.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.367+0000 7f2a637fe640 1 -- 172.21.15.172:0/1697775603 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c032ec0 con 0x7f2a74107920 2024-09-10T08:45:12.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.367+0000 7f2a78d98640 1 -- 172.21.15.172:0/1697775603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7419fc40 con 0x7f2a74107920 2024-09-10T08:45:12.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.367+0000 7f2a637fe640 1 -- 172.21.15.172:0/1697775603 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a6c0044f0 con 0x7f2a74107920 2024-09-10T08:45:12.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.367+0000 7f2a637fe640 1 -- 172.21.15.172:0/1697775603 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c030d40 con 0x7f2a74107920 2024-09-10T08:45:12.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.367+0000 7f2a78d98640 1 -- 172.21.15.172:0/1697775603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a741a0180 con 0x7f2a74107920 2024-09-10T08:45:12.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.368+0000 7f2a78d98640 1 -- 172.21.15.172:0/1697775603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a740fdd70 con 0x7f2a74107920 2024-09-10T08:45:12.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.370+0000 7f2a637fe640 1 -- 172.21.15.172:0/1697775603 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2a6c004040 con 0x7f2a74107920 2024-09-10T08:45:12.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.371+0000 7f2a637fe640 1 --2- 172.21.15.172:0/1697775603 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2a5c0a2db0 0x7f2a5c0a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:12.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.371+0000 7f2a7259c640 1 --2- 172.21.15.172:0/1697775603 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2a5c0a2db0 0x7f2a5c0a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:12.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.371+0000 7f2a637fe640 1 -- 172.21.15.172:0/1697775603 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f2a6c0f0af0 con 0x7f2a74107920 2024-09-10T08:45:12.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.371+0000 7f2a7259c640 1 --2- 172.21.15.172:0/1697775603 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2a5c0a2db0 0x7f2a5c0a5270 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2a64005fd0 tx=0x7f2a64005f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:12.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.374+0000 7f2a637fe640 1 -- 172.21.15.172:0/1697775603 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a6c0bd7c0 con 0x7f2a74107920 2024-09-10T08:45:12.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:12 smithi174 ceph-mon[34068]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:45:12.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:12 smithi174 ceph-mon[34068]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 14 op/s 2024-09-10T08:45:12.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:12 smithi191 ceph-mon[35314]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:45:12.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:12 smithi191 ceph-mon[35314]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 14 op/s 2024-09-10T08:45:12.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.612+0000 7f2a78d98640 1 -- 172.21.15.172:0/1697775603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f2a7402d7c0 con 0x7f2a74107920 2024-09-10T08:45:12.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.616+0000 7f2a637fe640 1 -- 172.21.15.172:0/1697775603 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f2a6c0bd0c0 con 0x7f2a74107920 2024-09-10T08:45:12.618 INFO:teuthology.orchestra.run.smithi172.stdout:[client.0] 2024-09-10T08:45:12.618 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQAYB+Bmi5ifJBAAKVXA7FP1CbcADfQGG1zyZA== 2024-09-10T08:45:12.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.618+0000 7f2a78d98640 1 -- 172.21.15.172:0/1697775603 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2a5c0a2db0 msgr2=0x7f2a5c0a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:12.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.618+0000 7f2a78d98640 1 --2- 172.21.15.172:0/1697775603 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2a5c0a2db0 0x7f2a5c0a5270 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2a64005fd0 tx=0x7f2a64005f40 comp rx=0 tx=0).stop 2024-09-10T08:45:12.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.618+0000 7f2a78d98640 1 -- 172.21.15.172:0/1697775603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74107920 msgr2=0x7f2a7419f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:12.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.618+0000 7f2a78d98640 1 --2- 172.21.15.172:0/1697775603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74107920 0x7f2a7419f310 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009dc0 tx=0x7f2a6c003ba0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.619+0000 7f2a78d98640 1 -- 172.21.15.172:0/1697775603 shutdown_connections 2024-09-10T08:45:12.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.619+0000 7f2a78d98640 1 --2- 172.21.15.172:0/1697775603 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2a5c0a2db0 0x7f2a5c0a5270 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.619+0000 7f2a78d98640 1 --2- 172.21.15.172:0/1697775603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a74107920 0x7f2a7419f310 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.619+0000 7f2a78d98640 1 --2- 172.21.15.172:0/1697775603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a74104ff0 0x7f2a741982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.619+0000 7f2a78d98640 1 --2- 172.21.15.172:0/1697775603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a741026c0 0x7f2a74197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:12.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.619+0000 7f2a78d98640 1 -- 172.21.15.172:0/1697775603 >> 172.21.15.172:0/1697775603 conn(0x7f2a740fc280 msgr2=0x7f2a74102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:12.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.619+0000 7f2a78d98640 1 -- 172.21.15.172:0/1697775603 shutdown_connections 2024-09-10T08:45:12.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:12.619+0000 7f2a78d98640 1 -- 172.21.15.172:0/1697775603 wait complete. 2024-09-10T08:45:13.282 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:45:13.282 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T08:45:13.283 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T08:45:13.320 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T08:45:13.321 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T08:45:13.321 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T08:45:13.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:13 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1697775603' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T08:45:13.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:13 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1697775603' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T08:45:13.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:13 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1697775603' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T08:45:13.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:13 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1697775603' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T08:45:13.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:13 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1697775603' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T08:45:13.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:13 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1697775603' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T08:45:13.610 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:14.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.096+0000 7f5dbd7e1640 1 -- 172.21.15.172:0/175675631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db8103760 msgr2=0x7f5db8107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.096+0000 7f5dbd7e1640 1 --2- 172.21.15.172:0/175675631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db8103760 0x7f5db8107ba0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009930 tx=0x7f5da002a420 comp rx=0 tx=0).stop 2024-09-10T08:45:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.096+0000 7f5dbd7e1640 1 -- 172.21.15.172:0/175675631 shutdown_connections 2024-09-10T08:45:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.096+0000 7f5dbd7e1640 1 --2- 172.21.15.172:0/175675631 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5db81082e0 0x7f5db810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.096+0000 7f5dbd7e1640 1 --2- 172.21.15.172:0/175675631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db8103760 0x7f5db8107ba0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.096+0000 7f5dbd7e1640 1 --2- 172.21.15.172:0/175675631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db8102db0 0x7f5db8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.097+0000 7f5dbd7e1640 1 -- 172.21.15.172:0/175675631 >> 172.21.15.172:0/175675631 conn(0x7f5db80fc7d0 msgr2=0x7f5db80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.097+0000 7f5dbd7e1640 1 -- 172.21.15.172:0/175675631 shutdown_connections 2024-09-10T08:45:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.097+0000 7f5dbd7e1640 1 -- 172.21.15.172:0/175675631 wait complete. 2024-09-10T08:45:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.097+0000 7f5dbd7e1640 1 Processor -- start 2024-09-10T08:45:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.097+0000 7f5dbd7e1640 1 -- start start 2024-09-10T08:45:14.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.097+0000 7f5dbd7e1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5db8102db0 0x7f5db8100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.097+0000 7f5dbd7e1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db8103760 0x7f5db8100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.097+0000 7f5dbd7e1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db81082e0 0x7f5db8101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.097+0000 7f5dbd7e1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5db8111f00 con 0x7f5db8103760 2024-09-10T08:45:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.097+0000 7f5dbd7e1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5db8111d80 con 0x7f5db81082e0 2024-09-10T08:45:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.097+0000 7f5dbd7e1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5db8112080 con 0x7f5db8102db0 2024-09-10T08:45:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.097+0000 7f5daffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db8103760 0x7f5db8100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.097+0000 7f5daffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db8103760 0x7f5db8100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48408/0 (socket says 172.21.15.172:48408) 2024-09-10T08:45:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.097+0000 7f5daffff640 1 -- 172.21.15.172:0/2068290622 learned_addr learned my addr 172.21.15.172:0/2068290622 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:14.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.098+0000 7f5db6ffd640 1 --2- 172.21.15.172:0/2068290622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5db8102db0 0x7f5db8100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.098+0000 7f5db77fe640 1 --2- 172.21.15.172:0/2068290622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db81082e0 0x7f5db8101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.098+0000 7f5daffff640 1 -- 172.21.15.172:0/2068290622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5db8102db0 msgr2=0x7f5db8100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.098+0000 7f5daffff640 1 --2- 172.21.15.172:0/2068290622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5db8102db0 0x7f5db8100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.098+0000 7f5daffff640 1 -- 172.21.15.172:0/2068290622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db81082e0 msgr2=0x7f5db8101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.098+0000 7f5daffff640 1 --2- 172.21.15.172:0/2068290622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db81082e0 0x7f5db8101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.098+0000 7f5daffff640 1 -- 172.21.15.172:0/2068290622 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db81a6a60 con 0x7f5db8103760 2024-09-10T08:45:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.098+0000 7f5db77fe640 1 --2- 172.21.15.172:0/2068290622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db81082e0 0x7f5db8101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.098+0000 7f5daffff640 1 --2- 172.21.15.172:0/2068290622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db8103760 0x7f5db8100f50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5da0005940 tx=0x7f5da00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.098+0000 7f5db4ff9640 1 -- 172.21.15.172:0/2068290622 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da003f030 con 0x7f5db8103760 2024-09-10T08:45:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.098+0000 7f5db4ff9640 1 -- 172.21.15.172:0/2068290622 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5da0003cb0 con 0x7f5db8103760 2024-09-10T08:45:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.098+0000 7f5db4ff9640 1 -- 172.21.15.172:0/2068290622 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da0035070 con 0x7f5db8103760 2024-09-10T08:45:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.098+0000 7f5dbd7e1640 1 -- 172.21.15.172:0/2068290622 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db81a6cf0 con 0x7f5db8103760 2024-09-10T08:45:14.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.098+0000 7f5dbd7e1640 1 -- 172.21.15.172:0/2068290622 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db81a7180 con 0x7f5db8103760 2024-09-10T08:45:14.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.099+0000 7f5dbd7e1640 1 -- 172.21.15.172:0/2068290622 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db8103190 con 0x7f5db8103760 2024-09-10T08:45:14.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.100+0000 7f5db4ff9640 1 -- 172.21.15.172:0/2068290622 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5da0042030 con 0x7f5db8103760 2024-09-10T08:45:14.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.101+0000 7f5db4ff9640 1 --2- 172.21.15.172:0/2068290622 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5da40a2c10 0x7f5da40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:14.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.101+0000 7f5db6ffd640 1 --2- 172.21.15.172:0/2068290622 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5da40a2c10 0x7f5da40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:14.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.101+0000 7f5db4ff9640 1 -- 172.21.15.172:0/2068290622 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f5da00ef8e0 con 0x7f5db8103760 2024-09-10T08:45:14.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.101+0000 7f5db6ffd640 1 --2- 172.21.15.172:0/2068290622 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5da40a2c10 0x7f5da40a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5da800bbe0 tx=0x7f5da800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:14.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.105+0000 7f5db4ff9640 1 -- 172.21.15.172:0/2068290622 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5da00bc4a0 con 0x7f5db8103760 2024-09-10T08:45:14.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.322+0000 7f5dbd7e1640 1 -- 172.21.15.172:0/2068290622 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f5db810a2c0 con 0x7f5db8103760 2024-09-10T08:45:14.324 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:14 smithi172 ceph-mon[30854]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-10T08:45:14.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.326+0000 7f5db4ff9640 1 -- 172.21.15.172:0/2068290622 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263205 (secure 0 0 0) 0x7f5da00bbda0 con 0x7f5db8103760 2024-09-10T08:45:14.331 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:45:14.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.329+0000 7f5dbd7e1640 1 -- 172.21.15.172:0/2068290622 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5da40a2c10 msgr2=0x7f5da40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:14.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.329+0000 7f5dbd7e1640 1 --2- 172.21.15.172:0/2068290622 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5da40a2c10 0x7f5da40a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5da800bbe0 tx=0x7f5da800b390 comp rx=0 tx=0).stop 2024-09-10T08:45:14.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.329+0000 7f5dbd7e1640 1 -- 172.21.15.172:0/2068290622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db8103760 msgr2=0x7f5db8100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:14.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.329+0000 7f5dbd7e1640 1 --2- 172.21.15.172:0/2068290622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db8103760 0x7f5db8100f50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5da0005940 tx=0x7f5da00042f0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.329+0000 7f5dbd7e1640 1 -- 172.21.15.172:0/2068290622 shutdown_connections 2024-09-10T08:45:14.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.329+0000 7f5dbd7e1640 1 --2- 172.21.15.172:0/2068290622 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5da40a2c10 0x7f5da40a50d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.329+0000 7f5dbd7e1640 1 --2- 172.21.15.172:0/2068290622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db81082e0 0x7f5db8101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.329+0000 7f5dbd7e1640 1 --2- 172.21.15.172:0/2068290622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5db8103760 0x7f5db8100f50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.329+0000 7f5dbd7e1640 1 --2- 172.21.15.172:0/2068290622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5db8102db0 0x7f5db8100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:14.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.329+0000 7f5dbd7e1640 1 -- 172.21.15.172:0/2068290622 >> 172.21.15.172:0/2068290622 conn(0x7f5db80fc7d0 msgr2=0x7f5db8105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:14.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.329+0000 7f5dbd7e1640 1 -- 172.21.15.172:0/2068290622 shutdown_connections 2024-09-10T08:45:14.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:14.329+0000 7f5dbd7e1640 1 -- 172.21.15.172:0/2068290622 wait complete. 2024-09-10T08:45:14.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:14 smithi174 ceph-mon[34068]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-10T08:45:14.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:14 smithi191 ceph-mon[35314]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-10T08:45:14.992 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":107199760},{"type":"v1","addr":"172.21.15.172:6801","nonce":107199760}]},"active_addr":"172.21.15.172:6801/107199760","active_change":"2024-09-10T08:37:17.019379+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.172:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.172:0","nonce":3583026383}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.172:0","nonce":535101180}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.172:0","nonce":1155523672}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.172:0","nonce":3363064156}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.172:0","nonce":284774589}]}]} 2024-09-10T08:45:14.998 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T08:45:14.998 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T08:45:14.998 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:45:15.247 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:15.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:15 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2068290622' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T08:45:15.510 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:15 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2068290622' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T08:45:15.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:15 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2068290622' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T08:45:15.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.729+0000 7fdd9d4dd640 1 -- 172.21.15.172:0/3522770940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd98101420 msgr2=0x7fdd9810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:15.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.729+0000 7fdd9d4dd640 1 --2- 172.21.15.172:0/3522770940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd98101420 0x7fdd9810d950 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009cd0 tx=0x7fdd9002a8e0 comp rx=0 tx=0).stop 2024-09-10T08:45:15.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.729+0000 7fdd9d4dd640 1 -- 172.21.15.172:0/3522770940 shutdown_connections 2024-09-10T08:45:15.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.729+0000 7fdd9d4dd640 1 --2- 172.21.15.172:0/3522770940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd98101420 0x7fdd9810d950 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:15.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.729+0000 7fdd9d4dd640 1 --2- 172.21.15.172:0/3522770940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd98100a80 0x7fdd98100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:15.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.729+0000 7fdd9d4dd640 1 --2- 172.21.15.172:0/3522770940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd98106a80 0x7fdd98106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:15.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.729+0000 7fdd9d4dd640 1 -- 172.21.15.172:0/3522770940 >> 172.21.15.172:0/3522770940 conn(0x7fdd980fc7a0 msgr2=0x7fdd980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:15.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.729+0000 7fdd9d4dd640 1 -- 172.21.15.172:0/3522770940 shutdown_connections 2024-09-10T08:45:15.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.729+0000 7fdd9d4dd640 1 -- 172.21.15.172:0/3522770940 wait complete. 2024-09-10T08:45:15.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.730+0000 7fdd9d4dd640 1 Processor -- start 2024-09-10T08:45:15.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.730+0000 7fdd9d4dd640 1 -- start start 2024-09-10T08:45:15.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.730+0000 7fdd9d4dd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd98100a80 0x7fdd98199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:15.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.730+0000 7fdd9d4dd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd98101420 0x7fdd98199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:15.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.730+0000 7fdd9d4dd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd98106a80 0x7fdd981a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:15.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.730+0000 7fdd9d4dd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98110d50 con 0x7fdd98100a80 2024-09-10T08:45:15.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.730+0000 7fdd9d4dd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98110bd0 con 0x7fdd98101420 2024-09-10T08:45:15.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.730+0000 7fdd9d4dd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98110ed0 con 0x7fdd98106a80 2024-09-10T08:45:15.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.730+0000 7fdd96ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd98100a80 0x7fdd98199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:15.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.730+0000 7fdd96ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd98100a80 0x7fdd98199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48428/0 (socket says 172.21.15.172:48428) 2024-09-10T08:45:15.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.730+0000 7fdd96ffd640 1 -- 172.21.15.172:0/200054190 learned_addr learned my addr 172.21.15.172:0/200054190 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:15.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.730+0000 7fdd967fc640 1 --2- 172.21.15.172:0/200054190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd98101420 0x7fdd98199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:15.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.730+0000 7fdd96ffd640 1 -- 172.21.15.172:0/200054190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd98106a80 msgr2=0x7fdd981a42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:15.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.730+0000 7fdd96ffd640 1 --2- 172.21.15.172:0/200054190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd98106a80 0x7fdd981a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:15.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.730+0000 7fdd96ffd640 1 -- 172.21.15.172:0/200054190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd98101420 msgr2=0x7fdd98199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:15.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.730+0000 7fdd96ffd640 1 --2- 172.21.15.172:0/200054190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd98101420 0x7fdd98199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:15.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.730+0000 7fdd96ffd640 1 -- 172.21.15.172:0/200054190 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd981a4800 con 0x7fdd98100a80 2024-09-10T08:45:15.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.731+0000 7fdd96ffd640 1 --2- 172.21.15.172:0/200054190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd98100a80 0x7fdd98199240 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800b4a0 tx=0x7fdd8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:15.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.731+0000 7fdd7ffff640 1 -- 172.21.15.172:0/200054190 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88014070 con 0x7fdd98100a80 2024-09-10T08:45:15.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.731+0000 7fdd9d4dd640 1 -- 172.21.15.172:0/200054190 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd981a49d0 con 0x7fdd98100a80 2024-09-10T08:45:15.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.731+0000 7fdd7ffff640 1 -- 172.21.15.172:0/200054190 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdd88010ac0 con 0x7fdd98100a80 2024-09-10T08:45:15.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.731+0000 7fdd7ffff640 1 -- 172.21.15.172:0/200054190 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88017c30 con 0x7fdd98100a80 2024-09-10T08:45:15.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.731+0000 7fdd9d4dd640 1 -- 172.21.15.172:0/200054190 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd981a4ee0 con 0x7fdd98100a80 2024-09-10T08:45:15.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.732+0000 7fdd9d4dd640 1 -- 172.21.15.172:0/200054190 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd98100ee0 con 0x7fdd98100a80 2024-09-10T08:45:15.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.733+0000 7fdd7ffff640 1 -- 172.21.15.172:0/200054190 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fdd88017070 con 0x7fdd98100a80 2024-09-10T08:45:15.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.733+0000 7fdd7ffff640 1 --2- 172.21.15.172:0/200054190 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdd800a2c10 0x7fdd800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:15.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.734+0000 7fdd967fc640 1 --2- 172.21.15.172:0/200054190 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdd800a2c10 0x7fdd800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:15.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.734+0000 7fdd7ffff640 1 -- 172.21.15.172:0/200054190 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fdd880ceab0 con 0x7fdd98100a80 2024-09-10T08:45:15.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.734+0000 7fdd967fc640 1 --2- 172.21.15.172:0/200054190 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdd800a2c10 0x7fdd800a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdd9806b2e0 tx=0x7fdd8c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:15.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.737+0000 7fdd7ffff640 1 -- 172.21.15.172:0/200054190 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdd8809b780 con 0x7fdd98100a80 2024-09-10T08:45:15.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.905+0000 7fdd9d4dd640 1 -- 172.21.15.172:0/200054190 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd98109010 con 0x7fdd98100a80 2024-09-10T08:45:15.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.907+0000 7fdd7ffff640 1 -- 172.21.15.172:0/200054190 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24294 (secure 0 0 0) 0x7fdd8809b080 con 0x7fdd98100a80 2024-09-10T08:45:15.908 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:45:15.909 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":88,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","created":"2024-09-10T08:36:33.083529+0000","modified":"2024-09-10T08:44:20.275987+0000","last_up_change":"2024-09-10T08:43:19.615640+0000","last_in_change":"2024-09-10T08:42:56.656115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:39:31.328306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:44:07.466637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:44:09.520569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:44:15.926057+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6803","nonce":2678426952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6805","nonce":2678426952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6809","nonce":2678426952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6807","nonce":2678426952}]},"public_addr":"172.21.15.172:6803/2678426952","cluster_addr":"172.21.15.172:6805/2678426952","heartbeat_back_addr":"172.21.15.172:6809/2678426952","heartbeat_front_addr":"172.21.15.172:6807/2678426952","state":["exists","up"]},{"osd":1,"uuid":"55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6801","nonce":3279661439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6803","nonce":3279661439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6807","nonce":3279661439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6805","nonce":3279661439}]},"public_addr":"172.21.15.174:6801/3279661439","cluster_addr":"172.21.15.174:6803/3279661439","heartbeat_back_addr":"172.21.15.174:6807/3279661439","heartbeat_front_addr":"172.21.15.174:6805/3279661439","state":["exists","up"]},{"osd":2,"uuid":"f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6801","nonce":1963796290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6803","nonce":1963796290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6807","nonce":1963796290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6805","nonce":1963796290}]},"public_addr":"172.21.15.191:6801/1963796290","cluster_addr":"172.21.15.191:6803/1963796290","heartbeat_back_addr":"172.21.15.191:6807/1963796290","heartbeat_front_addr":"172.21.15.191:6805/1963796290","state":["exists","up"]},{"osd":3,"uuid":"15ecca3f-bc6b-4a06-b283-b515384beb19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6811","nonce":869546480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6813","nonce":869546480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6817","nonce":869546480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6815","nonce":869546480}]},"public_addr":"172.21.15.172:6811/869546480","cluster_addr":"172.21.15.172:6813/869546480","heartbeat_back_addr":"172.21.15.172:6817/869546480","heartbeat_front_addr":"172.21.15.172:6815/869546480","state":["exists","up"]},{"osd":4,"uuid":"8c2c1d40-72ad-4a15-ab72-54871c7e2427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6809","nonce":961007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6811","nonce":961007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6815","nonce":961007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6813","nonce":961007949}]},"public_addr":"172.21.15.174:6809/961007949","cluster_addr":"172.21.15.174:6811/961007949","heartbeat_back_addr":"172.21.15.174:6815/961007949","heartbeat_front_addr":"172.21.15.174:6813/961007949","state":["exists","up"]},{"osd":5,"uuid":"4de45a1d-ef0e-4bfe-ba15-5eeeb8806155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6809","nonce":4216938371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6811","nonce":4216938371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6815","nonce":4216938371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6813","nonce":4216938371}]},"public_addr":"172.21.15.191:6809/4216938371","cluster_addr":"172.21.15.191:6811/4216938371","heartbeat_back_addr":"172.21.15.191:6815/4216938371","heartbeat_front_addr":"172.21.15.191:6813/4216938371","state":["exists","up"]},{"osd":6,"uuid":"60a9938a-f696-4db2-932c-402997543b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6819","nonce":3210762009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6821","nonce":3210762009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6825","nonce":3210762009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6823","nonce":3210762009}]},"public_addr":"172.21.15.172:6819/3210762009","cluster_addr":"172.21.15.172:6821/3210762009","heartbeat_back_addr":"172.21.15.172:6825/3210762009","heartbeat_front_addr":"172.21.15.172:6823/3210762009","state":["exists","up"]},{"osd":7,"uuid":"7f0202f3-971e-472f-9c4d-ae3c24d39f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6817","nonce":1461923581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6819","nonce":1461923581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6823","nonce":1461923581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6821","nonce":1461923581}]},"public_addr":"172.21.15.174:6817/1461923581","cluster_addr":"172.21.15.174:6819/1461923581","heartbeat_back_addr":"172.21.15.174:6823/1461923581","heartbeat_front_addr":"172.21.15.174:6821/1461923581","state":["exists","up"]},{"osd":8,"uuid":"b4e7327f-ca9f-4833-9cd3-b8529b3d9761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6817","nonce":3675210430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6819","nonce":3675210430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6823","nonce":3675210430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6821","nonce":3675210430}]},"public_addr":"172.21.15.191:6817/3675210430","cluster_addr":"172.21.15.191:6819/3675210430","heartbeat_back_addr":"172.21.15.191:6823/3675210430","heartbeat_front_addr":"172.21.15.191:6821/3675210430","state":["exists","up"]},{"osd":9,"uuid":"9f6153f2-f9db-4c00-a659-c932628016f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6827","nonce":1181268619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6829","nonce":1181268619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6833","nonce":1181268619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6831","nonce":1181268619}]},"public_addr":"172.21.15.172:6827/1181268619","cluster_addr":"172.21.15.172:6829/1181268619","heartbeat_back_addr":"172.21.15.172:6833/1181268619","heartbeat_front_addr":"172.21.15.172:6831/1181268619","state":["exists","up"]},{"osd":10,"uuid":"040a013a-4c3e-442f-bea6-12cbe517716d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6825","nonce":1238812525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6827","nonce":1238812525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6831","nonce":1238812525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6829","nonce":1238812525}]},"public_addr":"172.21.15.174:6825/1238812525","cluster_addr":"172.21.15.174:6827/1238812525","heartbeat_back_addr":"172.21.15.174:6831/1238812525","heartbeat_front_addr":"172.21.15.174:6829/1238812525","state":["exists","up"]},{"osd":11,"uuid":"ae6535bd-1510-4bf3-bf4b-1a2a256add43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6825","nonce":2162719997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6827","nonce":2162719997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6831","nonce":2162719997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6829","nonce":2162719997}]},"public_addr":"172.21.15.191:6825/2162719997","cluster_addr":"172.21.15.191:6827/2162719997","heartbeat_back_addr":"172.21.15.191:6831/2162719997","heartbeat_front_addr":"172.21.15.191:6829/2162719997","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:38:38.812596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:04.765652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:29.103966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:56.085849+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:21.641784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:47.750376+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:11.640697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:36.638840+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:02.167446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:28.655804+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:52.749676+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:43:17.783908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:0/977245917":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/3919952878":"2024-09-11T08:36:54.887256+0000","172.21.15.172:6800/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/1525385361":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/2043423408":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/839344299":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6800/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/3251817286":"2024-09-11T08:36:54.887256+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:45:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.909+0000 7fdd9d4dd640 1 -- 172.21.15.172:0/200054190 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdd800a2c10 msgr2=0x7fdd800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.909+0000 7fdd9d4dd640 1 --2- 172.21.15.172:0/200054190 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdd800a2c10 0x7fdd800a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdd9806b2e0 tx=0x7fdd8c00b410 comp rx=0 tx=0).stop 2024-09-10T08:45:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.909+0000 7fdd9d4dd640 1 -- 172.21.15.172:0/200054190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd98100a80 msgr2=0x7fdd98199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.909+0000 7fdd9d4dd640 1 --2- 172.21.15.172:0/200054190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd98100a80 0x7fdd98199240 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800b4a0 tx=0x7fdd8800b960 comp rx=0 tx=0).stop 2024-09-10T08:45:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.909+0000 7fdd9d4dd640 1 -- 172.21.15.172:0/200054190 shutdown_connections 2024-09-10T08:45:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.909+0000 7fdd9d4dd640 1 --2- 172.21.15.172:0/200054190 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdd800a2c10 0x7fdd800a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.909+0000 7fdd9d4dd640 1 --2- 172.21.15.172:0/200054190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd98106a80 0x7fdd981a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.909+0000 7fdd9d4dd640 1 --2- 172.21.15.172:0/200054190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd98101420 0x7fdd98199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.909+0000 7fdd9d4dd640 1 --2- 172.21.15.172:0/200054190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd98100a80 0x7fdd98199240 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.909+0000 7fdd9d4dd640 1 -- 172.21.15.172:0/200054190 >> 172.21.15.172:0/200054190 conn(0x7fdd980fc7a0 msgr2=0x7fdd980fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.909+0000 7fdd9d4dd640 1 -- 172.21.15.172:0/200054190 shutdown_connections 2024-09-10T08:45:15.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:15.909+0000 7fdd9d4dd640 1 -- 172.21.15.172:0/200054190 wait complete. 2024-09-10T08:45:16.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:16 smithi174 ceph-mon[34068]: pgmap v317: 129 pgs: 129 active+clean; 638 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-10T08:45:16.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:16 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/200054190' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:45:16.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:16 smithi191 ceph-mon[35314]: pgmap v317: 129 pgs: 129 active+clean; 638 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-10T08:45:16.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:16 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/200054190' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:45:16.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:16 smithi172 ceph-mon[30854]: pgmap v317: 129 pgs: 129 active+clean; 638 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-10T08:45:16.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:16 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/200054190' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:45:16.702 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T08:45:16.702 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:45:16.950 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:17.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-10T08:45:17.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]: dispatch 2024-09-10T08:45:17.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-09-10T08:45:17.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-10T08:45:17.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-10T08:45:17.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-09-10T08:45:17.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-09-10T08:45:17.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-09-10T08:45:17.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-09-10T08:45:17.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.450+0000 7f2cf7c31640 1 -- 172.21.15.172:0/590262938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cf0100e20 msgr2=0x7f2cf0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:17.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.450+0000 7f2cf7c31640 1 --2- 172.21.15.172:0/590262938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cf0100e20 0x7f2cf0101200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009900 tx=0x7f2cec02a480 comp rx=0 tx=0).stop 2024-09-10T08:45:17.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.450+0000 7f2cf7c31640 1 -- 172.21.15.172:0/590262938 shutdown_connections 2024-09-10T08:45:17.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.450+0000 7f2cf7c31640 1 --2- 172.21.15.172:0/590262938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cf0102ef0 0x7f2cf010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.450+0000 7f2cf7c31640 1 --2- 172.21.15.172:0/590262938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cf0102570 0x7f2cf01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.450+0000 7f2cf7c31640 1 --2- 172.21.15.172:0/590262938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cf0100e20 0x7f2cf0101200 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.450+0000 7f2cf7c31640 1 -- 172.21.15.172:0/590262938 >> 172.21.15.172:0/590262938 conn(0x7f2cf00fc7d0 msgr2=0x7f2cf00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:17.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.450+0000 7f2cf7c31640 1 -- 172.21.15.172:0/590262938 shutdown_connections 2024-09-10T08:45:17.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.450+0000 7f2cf7c31640 1 -- 172.21.15.172:0/590262938 wait complete. 2024-09-10T08:45:17.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.451+0000 7f2cf7c31640 1 Processor -- start 2024-09-10T08:45:17.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.451+0000 7f2cf7c31640 1 -- start start 2024-09-10T08:45:17.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.451+0000 7f2cf7c31640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cf0100e20 0x7f2cf0197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.451+0000 7f2cf7c31640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cf0102570 0x7f2cf01978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.451+0000 7f2cf7c31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cf0102ef0 0x7f2cf019bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.451+0000 7f2cf7c31640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0110e70 con 0x7f2cf0102ef0 2024-09-10T08:45:17.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.451+0000 7f2cf7c31640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0110cf0 con 0x7f2cf0102570 2024-09-10T08:45:17.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.451+0000 7f2cf7c31640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf0110ff0 con 0x7f2cf0100e20 2024-09-10T08:45:17.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.451+0000 7f2cf61ce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cf0102ef0 0x7f2cf019bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.451+0000 7f2cf61ce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cf0102ef0 0x7f2cf019bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42316/0 (socket says 172.21.15.172:42316) 2024-09-10T08:45:17.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.451+0000 7f2cf61ce640 1 -- 172.21.15.172:0/3536538397 learned_addr learned my addr 172.21.15.172:0/3536538397 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:17.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.451+0000 7f2cf59cd640 1 --2- 172.21.15.172:0/3536538397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cf0100e20 0x7f2cf0197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.451+0000 7f2cf51cc640 1 --2- 172.21.15.172:0/3536538397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cf0102570 0x7f2cf01978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.452+0000 7f2cf61ce640 1 -- 172.21.15.172:0/3536538397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cf0100e20 msgr2=0x7f2cf0197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:17.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.452+0000 7f2cf61ce640 1 --2- 172.21.15.172:0/3536538397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cf0100e20 0x7f2cf0197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.452+0000 7f2cf61ce640 1 -- 172.21.15.172:0/3536538397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cf0102570 msgr2=0x7f2cf01978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:17.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.452+0000 7f2cf61ce640 1 --2- 172.21.15.172:0/3536538397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cf0102570 0x7f2cf01978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.452+0000 7f2cf61ce640 1 -- 172.21.15.172:0/3536538397 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf019c320 con 0x7f2cf0102ef0 2024-09-10T08:45:17.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.452+0000 7f2cf61ce640 1 --2- 172.21.15.172:0/3536538397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cf0102ef0 0x7f2cf019bc40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800b970 tx=0x7f2ce800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:17.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.452+0000 7f2ce6ffd640 1 -- 172.21.15.172:0/3536538397 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8019070 con 0x7f2cf0102ef0 2024-09-10T08:45:17.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.452+0000 7f2ce6ffd640 1 -- 172.21.15.172:0/3536538397 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ce8014440 con 0x7f2cf0102ef0 2024-09-10T08:45:17.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.452+0000 7f2cf7c31640 1 -- 172.21.15.172:0/3536538397 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf019c610 con 0x7f2cf0102ef0 2024-09-10T08:45:17.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.452+0000 7f2ce6ffd640 1 -- 172.21.15.172:0/3536538397 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce80075d0 con 0x7f2cf0102ef0 2024-09-10T08:45:17.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.452+0000 7f2cf7c31640 1 -- 172.21.15.172:0/3536538397 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf01a3ef0 con 0x7f2cf0102ef0 2024-09-10T08:45:17.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.453+0000 7f2cf7c31640 1 -- 172.21.15.172:0/3536538397 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf01021b0 con 0x7f2cf0102ef0 2024-09-10T08:45:17.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.454+0000 7f2ce6ffd640 1 -- 172.21.15.172:0/3536538397 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2ce801c070 con 0x7f2cf0102ef0 2024-09-10T08:45:17.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.455+0000 7f2ce6ffd640 1 --2- 172.21.15.172:0/3536538397 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2cdc0a2c10 0x7f2cdc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:17.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.455+0000 7f2cf59cd640 1 --2- 172.21.15.172:0/3536538397 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2cdc0a2c10 0x7f2cdc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:17.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.455+0000 7f2ce6ffd640 1 -- 172.21.15.172:0/3536538397 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9071+0+0 (secure 0 0 0) 0x7f2ce80ceee0 con 0x7f2cf0102ef0 2024-09-10T08:45:17.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.456+0000 7f2cf59cd640 1 --2- 172.21.15.172:0/3536538397 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2cdc0a2c10 0x7f2cdc0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009230 tx=0x7f2cec0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:17.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.458+0000 7f2ce6ffd640 1 -- 172.21.15.172:0/3536538397 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ce809b400 con 0x7f2cf0102ef0 2024-09-10T08:45:17.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-10T08:45:17.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]: dispatch 2024-09-10T08:45:17.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-09-10T08:45:17.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-10T08:45:17.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-10T08:45:17.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-09-10T08:45:17.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-09-10T08:45:17.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-09-10T08:45:17.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-09-10T08:45:17.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.627+0000 7f2cf7c31640 1 -- 172.21.15.172:0/3536538397 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2cf0101200 con 0x7f2cf0102ef0 2024-09-10T08:45:17.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.629+0000 7f2ce6ffd640 1 -- 172.21.15.172:0/3536538397 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24732 (secure 0 0 0) 0x7f2ce809f700 con 0x7f2cf0102ef0 2024-09-10T08:45:17.630 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:45:17.630 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":89,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","created":"2024-09-10T08:36:33.083529+0000","modified":"2024-09-10T08:45:17.239006+0000","last_up_change":"2024-09-10T08:43:19.615640+0000","last_in_change":"2024-09-10T08:42:56.656115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:39:31.328306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:44:07.466637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:44:09.520569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:44:15.926057+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6803","nonce":2678426952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6805","nonce":2678426952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6809","nonce":2678426952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6807","nonce":2678426952}]},"public_addr":"172.21.15.172:6803/2678426952","cluster_addr":"172.21.15.172:6805/2678426952","heartbeat_back_addr":"172.21.15.172:6809/2678426952","heartbeat_front_addr":"172.21.15.172:6807/2678426952","state":["exists","up"]},{"osd":1,"uuid":"55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6801","nonce":3279661439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6803","nonce":3279661439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6807","nonce":3279661439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6805","nonce":3279661439}]},"public_addr":"172.21.15.174:6801/3279661439","cluster_addr":"172.21.15.174:6803/3279661439","heartbeat_back_addr":"172.21.15.174:6807/3279661439","heartbeat_front_addr":"172.21.15.174:6805/3279661439","state":["exists","up"]},{"osd":2,"uuid":"f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6801","nonce":1963796290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6803","nonce":1963796290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6807","nonce":1963796290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6805","nonce":1963796290}]},"public_addr":"172.21.15.191:6801/1963796290","cluster_addr":"172.21.15.191:6803/1963796290","heartbeat_back_addr":"172.21.15.191:6807/1963796290","heartbeat_front_addr":"172.21.15.191:6805/1963796290","state":["exists","up"]},{"osd":3,"uuid":"15ecca3f-bc6b-4a06-b283-b515384beb19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6811","nonce":869546480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6813","nonce":869546480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6817","nonce":869546480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6815","nonce":869546480}]},"public_addr":"172.21.15.172:6811/869546480","cluster_addr":"172.21.15.172:6813/869546480","heartbeat_back_addr":"172.21.15.172:6817/869546480","heartbeat_front_addr":"172.21.15.172:6815/869546480","state":["exists","up"]},{"osd":4,"uuid":"8c2c1d40-72ad-4a15-ab72-54871c7e2427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6809","nonce":961007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6811","nonce":961007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6815","nonce":961007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6813","nonce":961007949}]},"public_addr":"172.21.15.174:6809/961007949","cluster_addr":"172.21.15.174:6811/961007949","heartbeat_back_addr":"172.21.15.174:6815/961007949","heartbeat_front_addr":"172.21.15.174:6813/961007949","state":["exists","up"]},{"osd":5,"uuid":"4de45a1d-ef0e-4bfe-ba15-5eeeb8806155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6809","nonce":4216938371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6811","nonce":4216938371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6815","nonce":4216938371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6813","nonce":4216938371}]},"public_addr":"172.21.15.191:6809/4216938371","cluster_addr":"172.21.15.191:6811/4216938371","heartbeat_back_addr":"172.21.15.191:6815/4216938371","heartbeat_front_addr":"172.21.15.191:6813/4216938371","state":["exists","up"]},{"osd":6,"uuid":"60a9938a-f696-4db2-932c-402997543b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6819","nonce":3210762009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6821","nonce":3210762009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6825","nonce":3210762009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6823","nonce":3210762009}]},"public_addr":"172.21.15.172:6819/3210762009","cluster_addr":"172.21.15.172:6821/3210762009","heartbeat_back_addr":"172.21.15.172:6825/3210762009","heartbeat_front_addr":"172.21.15.172:6823/3210762009","state":["exists","up"]},{"osd":7,"uuid":"7f0202f3-971e-472f-9c4d-ae3c24d39f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6817","nonce":1461923581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6819","nonce":1461923581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6823","nonce":1461923581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6821","nonce":1461923581}]},"public_addr":"172.21.15.174:6817/1461923581","cluster_addr":"172.21.15.174:6819/1461923581","heartbeat_back_addr":"172.21.15.174:6823/1461923581","heartbeat_front_addr":"172.21.15.174:6821/1461923581","state":["exists","up"]},{"osd":8,"uuid":"b4e7327f-ca9f-4833-9cd3-b8529b3d9761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6817","nonce":3675210430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6819","nonce":3675210430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6823","nonce":3675210430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6821","nonce":3675210430}]},"public_addr":"172.21.15.191:6817/3675210430","cluster_addr":"172.21.15.191:6819/3675210430","heartbeat_back_addr":"172.21.15.191:6823/3675210430","heartbeat_front_addr":"172.21.15.191:6821/3675210430","state":["exists","up"]},{"osd":9,"uuid":"9f6153f2-f9db-4c00-a659-c932628016f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6827","nonce":1181268619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6829","nonce":1181268619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6833","nonce":1181268619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6831","nonce":1181268619}]},"public_addr":"172.21.15.172:6827/1181268619","cluster_addr":"172.21.15.172:6829/1181268619","heartbeat_back_addr":"172.21.15.172:6833/1181268619","heartbeat_front_addr":"172.21.15.172:6831/1181268619","state":["exists","up"]},{"osd":10,"uuid":"040a013a-4c3e-442f-bea6-12cbe517716d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6825","nonce":1238812525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6827","nonce":1238812525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6831","nonce":1238812525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6829","nonce":1238812525}]},"public_addr":"172.21.15.174:6825/1238812525","cluster_addr":"172.21.15.174:6827/1238812525","heartbeat_back_addr":"172.21.15.174:6831/1238812525","heartbeat_front_addr":"172.21.15.174:6829/1238812525","state":["exists","up"]},{"osd":11,"uuid":"ae6535bd-1510-4bf3-bf4b-1a2a256add43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6825","nonce":2162719997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6827","nonce":2162719997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6831","nonce":2162719997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6829","nonce":2162719997}]},"public_addr":"172.21.15.191:6825/2162719997","cluster_addr":"172.21.15.191:6827/2162719997","heartbeat_back_addr":"172.21.15.191:6831/2162719997","heartbeat_front_addr":"172.21.15.191:6829/2162719997","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:38:38.812596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:04.765652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:29.103966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:56.085849+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:21.641784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:47.750376+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:11.640697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:36.638840+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:02.167446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:28.655804+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:52.749676+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:43:17.783908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:0/977245917":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/3919952878":"2024-09-11T08:36:54.887256+0000","172.21.15.172:6800/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/1525385361":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/2043423408":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/839344299":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6800/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/3251817286":"2024-09-11T08:36:54.887256+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:45:17.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.632+0000 7f2cf7c31640 1 -- 172.21.15.172:0/3536538397 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2cdc0a2c10 msgr2=0x7f2cdc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:17.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.632+0000 7f2cf7c31640 1 --2- 172.21.15.172:0/3536538397 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2cdc0a2c10 0x7f2cdc0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009230 tx=0x7f2cec0058d0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.632+0000 7f2cf7c31640 1 -- 172.21.15.172:0/3536538397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cf0102ef0 msgr2=0x7f2cf019bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:17.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.632+0000 7f2cf7c31640 1 --2- 172.21.15.172:0/3536538397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cf0102ef0 0x7f2cf019bc40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800b970 tx=0x7f2ce800be30 comp rx=0 tx=0).stop 2024-09-10T08:45:17.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.632+0000 7f2cf7c31640 1 -- 172.21.15.172:0/3536538397 shutdown_connections 2024-09-10T08:45:17.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.632+0000 7f2cf7c31640 1 --2- 172.21.15.172:0/3536538397 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2cdc0a2c10 0x7f2cdc0a50d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.632+0000 7f2cf7c31640 1 --2- 172.21.15.172:0/3536538397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2cf0102ef0 0x7f2cf019bc40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.632+0000 7f2cf7c31640 1 --2- 172.21.15.172:0/3536538397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cf0102570 0x7f2cf01978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.632+0000 7f2cf7c31640 1 --2- 172.21.15.172:0/3536538397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2cf0100e20 0x7f2cf0197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:17.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.632+0000 7f2cf7c31640 1 -- 172.21.15.172:0/3536538397 >> 172.21.15.172:0/3536538397 conn(0x7f2cf00fc7d0 msgr2=0x7f2cf00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:17.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.632+0000 7f2cf7c31640 1 -- 172.21.15.172:0/3536538397 shutdown_connections 2024-09-10T08:45:17.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:17.632+0000 7f2cf7c31640 1 -- 172.21.15.172:0/3536538397 wait complete. 2024-09-10T08:45:17.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-10T08:45:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]: dispatch 2024-09-10T08:45:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-09-10T08:45:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-10T08:45:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-10T08:45:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-09-10T08:45:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-09-10T08:45:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-09-10T08:45:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-09-10T08:45:18.329 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T08:45:18.330 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T08:45:18.330 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T08:45:18.331 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T08:45:18.331 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T08:45:18.332 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T08:45:18.332 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T08:45:18.332 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T08:45:18.333 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-10T08:45:18.333 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-10T08:45:18.334 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-10T08:45:18.334 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-10T08:45:18.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:18 smithi191 ceph-mon[35314]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-09-10T08:45:18.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-10T08:45:18.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]': finished 2024-09-10T08:45:18.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-10T08:45:18.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-10T08:45:18.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-10T08:45:18.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-09-10T08:45:18.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-09-10T08:45:18.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-10T08:45:18.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-10T08:45:18.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:18 smithi191 ceph-mon[35314]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T08:45:18.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:18 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3536538397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:45:18.585 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:18 smithi172 ceph-mon[30854]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-09-10T08:45:18.585 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-10T08:45:18.585 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]': finished 2024-09-10T08:45:18.585 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-10T08:45:18.585 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-10T08:45:18.585 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-10T08:45:18.585 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-09-10T08:45:18.585 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-09-10T08:45:18.585 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-10T08:45:18.586 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-10T08:45:18.586 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:18 smithi172 ceph-mon[30854]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T08:45:18.586 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:18 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3536538397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:45:18.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:18 smithi174 ceph-mon[34068]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-09-10T08:45:18.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-10T08:45:18.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]': finished 2024-09-10T08:45:18.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-10T08:45:18.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-10T08:45:18.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-10T08:45:18.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-09-10T08:45:18.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-09-10T08:45:18.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-09-10T08:45:18.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-10T08:45:18.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:18 smithi174 ceph-mon[34068]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T08:45:18.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:18 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3536538397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:45:19.059 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:19.059 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:19.063 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:19.065 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:19.067 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:19.070 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:19.073 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:19.073 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:19.080 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:19.136 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:19.533 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:19 smithi172 ceph-mon[30854]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T08:45:19.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:19 smithi191 ceph-mon[35314]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T08:45:19.669 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:19.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:19 smithi174 ceph-mon[34068]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T08:45:19.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.839+0000 7f4bda310640 1 -- 172.21.15.172:0/701104758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd4101dd0 msgr2=0x7f4bd41021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.839+0000 7f4bda310640 1 --2- 172.21.15.172:0/701104758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd4101dd0 0x7f4bd41021b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009900 tx=0x7f4bc402a410 comp rx=0 tx=0).stop 2024-09-10T08:45:19.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.839+0000 7f4bda310640 1 -- 172.21.15.172:0/701104758 shutdown_connections 2024-09-10T08:45:19.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.839+0000 7f4bda310640 1 --2- 172.21.15.172:0/701104758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bd4106f00 0x7f4bd410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.839+0000 7f4bda310640 1 --2- 172.21.15.172:0/701104758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bd4102780 0x7f4bd41067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.839+0000 7f4bda310640 1 --2- 172.21.15.172:0/701104758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd4101dd0 0x7f4bd41021b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.839+0000 7f4bda310640 1 -- 172.21.15.172:0/701104758 >> 172.21.15.172:0/701104758 conn(0x7f4bd40fd8b0 msgr2=0x7f4bd40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.839+0000 7f4bda310640 1 -- 172.21.15.172:0/701104758 shutdown_connections 2024-09-10T08:45:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.839+0000 7f4bda310640 1 -- 172.21.15.172:0/701104758 wait complete. 2024-09-10T08:45:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.840+0000 7f4bda310640 1 Processor -- start 2024-09-10T08:45:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.840+0000 7f4bda310640 1 -- start start 2024-09-10T08:45:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.840+0000 7f4bda310640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd4101dd0 0x7f4bd41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.840+0000 7f4bda310640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bd4102780 0x7f4bd41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.840+0000 7f4bda310640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bd4106f00 0x7f4bd419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.840+0000 7f4bda310640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd4111e30 con 0x7f4bd4101dd0 2024-09-10T08:45:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.840+0000 7f4bda310640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd4111cb0 con 0x7f4bd4102780 2024-09-10T08:45:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.840+0000 7f4bda310640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd4111fb0 con 0x7f4bd4106f00 2024-09-10T08:45:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.840+0000 7f4bd3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd4101dd0 0x7f4bd41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.840+0000 7f4bd3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd4101dd0 0x7f4bd41983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42336/0 (socket says 172.21.15.172:42336) 2024-09-10T08:45:19.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.840+0000 7f4bd3fff640 1 -- 172.21.15.172:0/3430489431 learned_addr learned my addr 172.21.15.172:0/3430489431 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:19.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.841+0000 7f4bd88ad640 1 --2- 172.21.15.172:0/3430489431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bd4106f00 0x7f4bd419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.841+0000 7f4bd37fe640 1 --2- 172.21.15.172:0/3430489431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bd4102780 0x7f4bd41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.841+0000 7f4bd3fff640 1 -- 172.21.15.172:0/3430489431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bd4106f00 msgr2=0x7f4bd419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.841+0000 7f4bd3fff640 1 --2- 172.21.15.172:0/3430489431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bd4106f00 0x7f4bd419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.841+0000 7f4bd3fff640 1 -- 172.21.15.172:0/3430489431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bd4102780 msgr2=0x7f4bd41988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.841+0000 7f4bd3fff640 1 --2- 172.21.15.172:0/3430489431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bd4102780 0x7f4bd41988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.841+0000 7f4bd3fff640 1 -- 172.21.15.172:0/3430489431 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd419d350 con 0x7f4bd4101dd0 2024-09-10T08:45:19.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.841+0000 7f4bd3fff640 1 --2- 172.21.15.172:0/3430489431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd4101dd0 0x7f4bd41983a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4006010 tx=0x7f4bc4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:19.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.841+0000 7f4bd17fa640 1 -- 172.21.15.172:0/3430489431 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc403f030 con 0x7f4bd4101dd0 2024-09-10T08:45:19.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.841+0000 7f4bda310640 1 -- 172.21.15.172:0/3430489431 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd419d5e0 con 0x7f4bd4101dd0 2024-09-10T08:45:19.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.841+0000 7f4bd17fa640 1 -- 172.21.15.172:0/3430489431 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4bc40035f0 con 0x7f4bd4101dd0 2024-09-10T08:45:19.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.841+0000 7f4bd17fa640 1 -- 172.21.15.172:0/3430489431 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4035070 con 0x7f4bd4101dd0 2024-09-10T08:45:19.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.841+0000 7f4bda310640 1 -- 172.21.15.172:0/3430489431 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd41a4f20 con 0x7f4bd4101dd0 2024-09-10T08:45:19.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.842+0000 7f4bda310640 1 -- 172.21.15.172:0/3430489431 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4bd4062900 con 0x7f4bd4101dd0 2024-09-10T08:45:19.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.843+0000 7f4bd17fa640 1 -- 172.21.15.172:0/3430489431 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4bc4003b10 con 0x7f4bd4101dd0 2024-09-10T08:45:19.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.844+0000 7f4bd17fa640 1 --2- 172.21.15.172:0/3430489431 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4bc00a2c10 0x7f4bc00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.844+0000 7f4bd37fe640 1 --2- 172.21.15.172:0/3430489431 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4bc00a2c10 0x7f4bc00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.844+0000 7f4bd17fa640 1 -- 172.21.15.172:0/3430489431 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f4bc40f0680 con 0x7f4bd4101dd0 2024-09-10T08:45:19.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.844+0000 7f4bd17fa640 1 --2- 172.21.15.172:0/3430489431 >> [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952] conn(0x7f4bc00adce0 0x7f4bc00b0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:19.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.844+0000 7f4bd17fa640 1 -- 172.21.15.172:0/3430489431 --> [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4bc4001d30 con 0x7f4bc00adce0 2024-09-10T08:45:19.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.844+0000 7f4bd17fa640 1 -- 172.21.15.172:0/3430489431 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f4bc40f0ad0 con 0x7f4bd4101dd0 2024-09-10T08:45:19.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.844+0000 7f4bd88ad640 1 --2- 172.21.15.172:0/3430489431 >> [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952] conn(0x7f4bc00adce0 0x7f4bc00b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:19.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.845+0000 7f4bd37fe640 1 --2- 172.21.15.172:0/3430489431 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4bc00a2c10 0x7f4bc00a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40ffe00 tx=0x7f4bbc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:19.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.845+0000 7f4bd88ad640 1 --2- 172.21.15.172:0/3430489431 >> [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952] conn(0x7f4bc00adce0 0x7f4bc00b0180 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:19.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.845+0000 7f4bd17fa640 1 -- 172.21.15.172:0/3430489431 <== osd.0 v2:172.21.15.172:6802/2678426952 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4bc4001d30 con 0x7f4bc00adce0 2024-09-10T08:45:19.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.859+0000 7f4bda310640 1 -- 172.21.15.172:0/3430489431 --> [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4bd4062d00 con 0x7f4bc00adce0 2024-09-10T08:45:19.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.859+0000 7f4bd17fa640 1 -- 172.21.15.172:0/3430489431 <== osd.0 v2:172.21.15.172:6802/2678426952 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4bd4062d00 con 0x7f4bc00adce0 2024-09-10T08:45:19.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.859+0000 7f4bda310640 1 -- 172.21.15.172:0/3430489431 >> [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952] conn(0x7f4bc00adce0 msgr2=0x7f4bc00b0180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.859+0000 7f4bda310640 1 --2- 172.21.15.172:0/3430489431 >> [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952] conn(0x7f4bc00adce0 0x7f4bc00b0180 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.859+0000 7f4bda310640 1 -- 172.21.15.172:0/3430489431 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4bc00a2c10 msgr2=0x7f4bc00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.859+0000 7f4bda310640 1 --2- 172.21.15.172:0/3430489431 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4bc00a2c10 0x7f4bc00a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40ffe00 tx=0x7f4bbc00a2d0 comp rx=0 tx=0).stop 2024-09-10T08:45:19.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.859+0000 7f4bda310640 1 -- 172.21.15.172:0/3430489431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd4101dd0 msgr2=0x7f4bd41983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:19.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.859+0000 7f4bda310640 1 --2- 172.21.15.172:0/3430489431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bd4101dd0 0x7f4bd41983a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4006010 tx=0x7f4bc4004270 comp rx=0 tx=0).stop 2024-09-10T08:45:19.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.859+0000 7f4bd88ad640 1 -- 172.21.15.172:0/3430489431 reap_dead start 2024-09-10T08:45:19.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.860+0000 7f4bda310640 1 -- 172.21.15.172:0/3430489431 shutdown_connections 2024-09-10T08:45:19.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.860+0000 7f4bda310640 1 -- 172.21.15.172:0/3430489431 >> 172.21.15.172:0/3430489431 conn(0x7f4bd40fd8b0 msgr2=0x7f4bd40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:19.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.860+0000 7f4bda310640 1 -- 172.21.15.172:0/3430489431 shutdown_connections 2024-09-10T08:45:19.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:19.860+0000 7f4bda310640 1 -- 172.21.15.172:0/3430489431 wait complete. 2024-09-10T08:45:20.415 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:20 smithi172 ceph-mon[30854]: pgmap v321: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T08:45:20.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:20 smithi191 ceph-mon[35314]: pgmap v321: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T08:45:20.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:20 smithi174 ceph-mon[34068]: pgmap v321: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T08:45:20.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.764+0000 7fcc92106640 1 -- 172.21.15.172:0/1400642803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc8c102d40 msgr2=0x7fcc8c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:20.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.764+0000 7fcc92106640 1 --2- 172.21.15.172:0/1400642803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc8c102d40 0x7fcc8c103120 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009900 tx=0x7fcc7c02a480 comp rx=0 tx=0).stop 2024-09-10T08:45:20.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.764+0000 7fcc92106640 1 -- 172.21.15.172:0/1400642803 shutdown_connections 2024-09-10T08:45:20.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.764+0000 7fcc92106640 1 --2- 172.21.15.172:0/1400642803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc8c10c170 0x7fcc8c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:20.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.764+0000 7fcc92106640 1 --2- 172.21.15.172:0/1400642803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc8c103660 0x7fcc8c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:20.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.764+0000 7fcc92106640 1 --2- 172.21.15.172:0/1400642803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc8c102d40 0x7fcc8c103120 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:20.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.764+0000 7fcc92106640 1 -- 172.21.15.172:0/1400642803 >> 172.21.15.172:0/1400642803 conn(0x7fcc8c0fc7a0 msgr2=0x7fcc8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:20.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.764+0000 7fcc92106640 1 -- 172.21.15.172:0/1400642803 shutdown_connections 2024-09-10T08:45:20.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.764+0000 7fcc92106640 1 -- 172.21.15.172:0/1400642803 wait complete. 2024-09-10T08:45:20.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.765+0000 7fcc92106640 1 Processor -- start 2024-09-10T08:45:20.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.765+0000 7fcc92106640 1 -- start start 2024-09-10T08:45:20.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.765+0000 7fcc92106640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc8c102d40 0x7fcc8c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:20.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.765+0000 7fcc92106640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc8c103660 0x7fcc8c198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:20.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.765+0000 7fcc92106640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc8c10c170 0x7fcc8c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:20.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.765+0000 7fcc92106640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c111e20 con 0x7fcc8c103660 2024-09-10T08:45:20.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.765+0000 7fcc92106640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c111ca0 con 0x7fcc8c10c170 2024-09-10T08:45:20.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.765+0000 7fcc92106640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c111fa0 con 0x7fcc8c102d40 2024-09-10T08:45:20.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.765+0000 7fcc8affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc8c103660 0x7fcc8c198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:20.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.765+0000 7fcc8affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc8c103660 0x7fcc8c198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42356/0 (socket says 172.21.15.172:42356) 2024-09-10T08:45:20.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.765+0000 7fcc8affd640 1 -- 172.21.15.172:0/1996641846 learned_addr learned my addr 172.21.15.172:0/1996641846 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:20.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.765+0000 7fcc8b7fe640 1 --2- 172.21.15.172:0/1996641846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc8c102d40 0x7fcc8c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:20.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.765+0000 7fcc8bfff640 1 --2- 172.21.15.172:0/1996641846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc8c10c170 0x7fcc8c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:20.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.766+0000 7fcc8affd640 1 -- 172.21.15.172:0/1996641846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc8c102d40 msgr2=0x7fcc8c1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:20.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.766+0000 7fcc8affd640 1 --2- 172.21.15.172:0/1996641846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc8c102d40 0x7fcc8c1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:20.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.766+0000 7fcc8affd640 1 -- 172.21.15.172:0/1996641846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc8c10c170 msgr2=0x7fcc8c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:20.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.766+0000 7fcc8affd640 1 --2- 172.21.15.172:0/1996641846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc8c10c170 0x7fcc8c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:20.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.766+0000 7fcc8affd640 1 -- 172.21.15.172:0/1996641846 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8c19d290 con 0x7fcc8c103660 2024-09-10T08:45:20.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.766+0000 7fcc8affd640 1 --2- 172.21.15.172:0/1996641846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc8c103660 0x7fcc8c198820 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000c970 tx=0x7fcc8000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:20.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.766+0000 7fcc88ff9640 1 -- 172.21.15.172:0/1996641846 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80007bb0 con 0x7fcc8c103660 2024-09-10T08:45:20.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.766+0000 7fcc88ff9640 1 -- 172.21.15.172:0/1996641846 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc80007d50 con 0x7fcc8c103660 2024-09-10T08:45:20.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.766+0000 7fcc92106640 1 -- 172.21.15.172:0/1996641846 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc8c19d580 con 0x7fcc8c103660 2024-09-10T08:45:20.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.766+0000 7fcc88ff9640 1 -- 172.21.15.172:0/1996641846 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80018780 con 0x7fcc8c103660 2024-09-10T08:45:20.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.766+0000 7fcc92106640 1 -- 172.21.15.172:0/1996641846 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc8c1a4e60 con 0x7fcc8c103660 2024-09-10T08:45:20.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.767+0000 7fcc92106640 1 -- 172.21.15.172:0/1996641846 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcc8c062940 con 0x7fcc8c103660 2024-09-10T08:45:20.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.768+0000 7fcc88ff9640 1 -- 172.21.15.172:0/1996641846 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fcc8000f450 con 0x7fcc8c103660 2024-09-10T08:45:20.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.769+0000 7fcc88ff9640 1 --2- 172.21.15.172:0/1996641846 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcc740a2c10 0x7fcc740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:20.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.769+0000 7fcc88ff9640 1 -- 172.21.15.172:0/1996641846 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fcc800cf930 con 0x7fcc8c103660 2024-09-10T08:45:20.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.769+0000 7fcc8b7fe640 1 --2- 172.21.15.172:0/1996641846 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcc740a2c10 0x7fcc740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:20.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.769+0000 7fcc88ff9640 1 --2- 172.21.15.172:0/1996641846 >> [v2:172.21.15.191:6808/4216938371,v1:172.21.15.191:6809/4216938371] conn(0x7fcc740add60 0x7fcc740b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:20.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.769+0000 7fcc88ff9640 1 -- 172.21.15.172:0/1996641846 --> [v2:172.21.15.191:6808/4216938371,v1:172.21.15.191:6809/4216938371] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcc740b08d0 con 0x7fcc740add60 2024-09-10T08:45:20.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.769+0000 7fcc88ff9640 1 -- 172.21.15.172:0/1996641846 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fcc800cfd80 con 0x7fcc8c103660 2024-09-10T08:45:20.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.770+0000 7fcc8b7fe640 1 --2- 172.21.15.172:0/1996641846 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcc740a2c10 0x7fcc740a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c031440 tx=0x7fcc7c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:20.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.770+0000 7fcc8bfff640 1 --2- 172.21.15.172:0/1996641846 >> [v2:172.21.15.191:6808/4216938371,v1:172.21.15.191:6809/4216938371] conn(0x7fcc740add60 0x7fcc740b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:20.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.770+0000 7fcc8bfff640 1 --2- 172.21.15.172:0/1996641846 >> [v2:172.21.15.191:6808/4216938371,v1:172.21.15.191:6809/4216938371] conn(0x7fcc740add60 0x7fcc740b0200 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:20.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.771+0000 7fcc88ff9640 1 -- 172.21.15.172:0/1996641846 <== osd.5 v2:172.21.15.191:6808/4216938371 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcc740b08d0 con 0x7fcc740add60 2024-09-10T08:45:20.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.785+0000 7fcc92106640 1 -- 172.21.15.172:0/1996641846 --> [v2:172.21.15.191:6808/4216938371,v1:172.21.15.191:6809/4216938371] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcc8c062dc0 con 0x7fcc740add60 2024-09-10T08:45:20.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.785+0000 7fcc88ff9640 1 -- 172.21.15.172:0/1996641846 <== osd.5 v2:172.21.15.191:6808/4216938371 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcc8c062dc0 con 0x7fcc740add60 2024-09-10T08:45:20.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.786+0000 7fcc92106640 1 -- 172.21.15.172:0/1996641846 >> [v2:172.21.15.191:6808/4216938371,v1:172.21.15.191:6809/4216938371] conn(0x7fcc740add60 msgr2=0x7fcc740b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:20.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.786+0000 7fcc92106640 1 --2- 172.21.15.172:0/1996641846 >> [v2:172.21.15.191:6808/4216938371,v1:172.21.15.191:6809/4216938371] conn(0x7fcc740add60 0x7fcc740b0200 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:20.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.786+0000 7fcc92106640 1 -- 172.21.15.172:0/1996641846 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcc740a2c10 msgr2=0x7fcc740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:20.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.786+0000 7fcc92106640 1 --2- 172.21.15.172:0/1996641846 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcc740a2c10 0x7fcc740a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c031440 tx=0x7fcc7c005b50 comp rx=0 tx=0).stop 2024-09-10T08:45:20.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.786+0000 7fcc92106640 1 -- 172.21.15.172:0/1996641846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc8c103660 msgr2=0x7fcc8c198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:20.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.786+0000 7fcc92106640 1 --2- 172.21.15.172:0/1996641846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc8c103660 0x7fcc8c198820 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000c970 tx=0x7fcc8000ce30 comp rx=0 tx=0).stop 2024-09-10T08:45:20.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.786+0000 7fcc8bfff640 1 -- 172.21.15.172:0/1996641846 reap_dead start 2024-09-10T08:45:20.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.786+0000 7fcc92106640 1 -- 172.21.15.172:0/1996641846 shutdown_connections 2024-09-10T08:45:20.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.786+0000 7fcc92106640 1 -- 172.21.15.172:0/1996641846 >> 172.21.15.172:0/1996641846 conn(0x7fcc8c0fc7a0 msgr2=0x7fcc8c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:20.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.786+0000 7fcc92106640 1 -- 172.21.15.172:0/1996641846 shutdown_connections 2024-09-10T08:45:20.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:20.786+0000 7fcc92106640 1 -- 172.21.15.172:0/1996641846 wait complete. 2024-09-10T08:45:21.236 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:21.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.699+0000 7faaba508640 1 -- 172.21.15.172:0/2325887131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faab4106f40 msgr2=0x7faab410eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:21.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.699+0000 7faaba508640 1 --2- 172.21.15.172:0/2325887131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faab4106f40 0x7faab410eac0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faaac009ca0 tx=0x7faaac02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:45:21.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.699+0000 7faaba508640 1 -- 172.21.15.172:0/2325887131 shutdown_connections 2024-09-10T08:45:21.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.699+0000 7faaba508640 1 --2- 172.21.15.172:0/2325887131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faab4106f40 0x7faab410eac0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:21.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.699+0000 7faaba508640 1 --2- 172.21.15.172:0/2325887131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faab41027c0 0x7faab4106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:21.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.699+0000 7faaba508640 1 --2- 172.21.15.172:0/2325887131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab4101e10 0x7faab41021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:21.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.699+0000 7faaba508640 1 -- 172.21.15.172:0/2325887131 >> 172.21.15.172:0/2325887131 conn(0x7faab40fd8d0 msgr2=0x7faab40ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:21.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.699+0000 7faaba508640 1 -- 172.21.15.172:0/2325887131 shutdown_connections 2024-09-10T08:45:21.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.699+0000 7faaba508640 1 -- 172.21.15.172:0/2325887131 wait complete. 2024-09-10T08:45:21.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.700+0000 7faaba508640 1 Processor -- start 2024-09-10T08:45:21.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.700+0000 7faaba508640 1 -- start start 2024-09-10T08:45:21.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.700+0000 7faaba508640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faab4101e10 0x7faab40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:21.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.700+0000 7faaba508640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faab41027c0 0x7faab4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:21.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.700+0000 7faaba508640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab4106f40 0x7faab4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:21.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.700+0000 7faaba508640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faab4111e50 con 0x7faab41027c0 2024-09-10T08:45:21.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.700+0000 7faaba508640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faab4111cd0 con 0x7faab4106f40 2024-09-10T08:45:21.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.700+0000 7faaba508640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7faab4111fd0 con 0x7faab4101e10 2024-09-10T08:45:21.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.700+0000 7faab37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faab41027c0 0x7faab4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:21.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.700+0000 7faab37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faab41027c0 0x7faab4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42398/0 (socket says 172.21.15.172:42398) 2024-09-10T08:45:21.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.700+0000 7faab37fe640 1 -- 172.21.15.172:0/1019849696 learned_addr learned my addr 172.21.15.172:0/1019849696 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:21.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.701+0000 7faab8aa5640 1 --2- 172.21.15.172:0/1019849696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab4106f40 0x7faab4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:21.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.701+0000 7faab3fff640 1 --2- 172.21.15.172:0/1019849696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faab4101e10 0x7faab40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:21.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.701+0000 7faab37fe640 1 -- 172.21.15.172:0/1019849696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faab4101e10 msgr2=0x7faab40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:21.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.701+0000 7faab37fe640 1 --2- 172.21.15.172:0/1019849696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faab4101e10 0x7faab40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:21.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.701+0000 7faab37fe640 1 -- 172.21.15.172:0/1019849696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab4106f40 msgr2=0x7faab4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:21.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.701+0000 7faab37fe640 1 --2- 172.21.15.172:0/1019849696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab4106f40 0x7faab4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:21.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.701+0000 7faab37fe640 1 -- 172.21.15.172:0/1019849696 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab41a6740 con 0x7faab41027c0 2024-09-10T08:45:21.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.701+0000 7faab8aa5640 1 --2- 172.21.15.172:0/1019849696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab4106f40 0x7faab4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:21.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.701+0000 7faab37fe640 1 --2- 172.21.15.172:0/1019849696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faab41027c0 0x7faab4074a30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faaa800c920 tx=0x7faaa800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:21.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.701+0000 7faab17fa640 1 -- 172.21.15.172:0/1019849696 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8007c70 con 0x7faab41027c0 2024-09-10T08:45:21.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.701+0000 7faab17fa640 1 -- 172.21.15.172:0/1019849696 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaa8007e10 con 0x7faab41027c0 2024-09-10T08:45:21.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.701+0000 7faaba508640 1 -- 172.21.15.172:0/1019849696 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab41a6a30 con 0x7faab41027c0 2024-09-10T08:45:21.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.701+0000 7faab17fa640 1 -- 172.21.15.172:0/1019849696 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8018820 con 0x7faab41027c0 2024-09-10T08:45:21.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.701+0000 7faaba508640 1 -- 172.21.15.172:0/1019849696 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab41a6f70 con 0x7faab41027c0 2024-09-10T08:45:21.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.702+0000 7faaba508640 1 -- 172.21.15.172:0/1019849696 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faab4062900 con 0x7faab41027c0 2024-09-10T08:45:21.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.703+0000 7faab17fa640 1 -- 172.21.15.172:0/1019849696 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7faaa80189c0 con 0x7faab41027c0 2024-09-10T08:45:21.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.704+0000 7faab17fa640 1 --2- 172.21.15.172:0/1019849696 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7faa9c0a2c10 0x7faa9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:21.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.704+0000 7faab3fff640 1 --2- 172.21.15.172:0/1019849696 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7faa9c0a2c10 0x7faa9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:21.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.704+0000 7faab17fa640 1 -- 172.21.15.172:0/1019849696 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7faaa80cfa60 con 0x7faab41027c0 2024-09-10T08:45:21.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.704+0000 7faab17fa640 1 --2- 172.21.15.172:0/1019849696 >> [v2:172.21.15.174:6808/961007949,v1:172.21.15.174:6809/961007949] conn(0x7faa9c0add60 0x7faa9c0b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:21.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.704+0000 7faab17fa640 1 -- 172.21.15.172:0/1019849696 --> [v2:172.21.15.174:6808/961007949,v1:172.21.15.174:6809/961007949] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faaa8015e70 con 0x7faa9c0add60 2024-09-10T08:45:21.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.704+0000 7faab17fa640 1 -- 172.21.15.172:0/1019849696 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7faaa80cfeb0 con 0x7faab41027c0 2024-09-10T08:45:21.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.704+0000 7faab3fff640 1 --2- 172.21.15.172:0/1019849696 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7faa9c0a2c10 0x7faa9c0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faaa400bbe0 tx=0x7faaa400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:21.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.705+0000 7faab8aa5640 1 --2- 172.21.15.172:0/1019849696 >> [v2:172.21.15.174:6808/961007949,v1:172.21.15.174:6809/961007949] conn(0x7faa9c0add60 0x7faa9c0b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:21.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.705+0000 7faab8aa5640 1 --2- 172.21.15.172:0/1019849696 >> [v2:172.21.15.174:6808/961007949,v1:172.21.15.174:6809/961007949] conn(0x7faa9c0add60 0x7faa9c0b0200 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:21.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.706+0000 7faab17fa640 1 -- 172.21.15.172:0/1019849696 <== osd.4 v2:172.21.15.174:6808/961007949 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faaa8015e70 con 0x7faa9c0add60 2024-09-10T08:45:21.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.719+0000 7faaba508640 1 -- 172.21.15.172:0/1019849696 --> [v2:172.21.15.174:6808/961007949,v1:172.21.15.174:6809/961007949] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faab4062d80 con 0x7faa9c0add60 2024-09-10T08:45:21.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.720+0000 7faab17fa640 1 -- 172.21.15.172:0/1019849696 <== osd.4 v2:172.21.15.174:6808/961007949 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7faab4062d80 con 0x7faa9c0add60 2024-09-10T08:45:21.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.720+0000 7faaba508640 1 -- 172.21.15.172:0/1019849696 >> [v2:172.21.15.174:6808/961007949,v1:172.21.15.174:6809/961007949] conn(0x7faa9c0add60 msgr2=0x7faa9c0b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:21.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.720+0000 7faaba508640 1 --2- 172.21.15.172:0/1019849696 >> [v2:172.21.15.174:6808/961007949,v1:172.21.15.174:6809/961007949] conn(0x7faa9c0add60 0x7faa9c0b0200 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:21.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.720+0000 7faaba508640 1 -- 172.21.15.172:0/1019849696 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7faa9c0a2c10 msgr2=0x7faa9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:21.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.720+0000 7faaba508640 1 --2- 172.21.15.172:0/1019849696 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7faa9c0a2c10 0x7faa9c0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faaa400bbe0 tx=0x7faaa400b390 comp rx=0 tx=0).stop 2024-09-10T08:45:21.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.720+0000 7faaba508640 1 -- 172.21.15.172:0/1019849696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faab41027c0 msgr2=0x7faab4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:21.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.720+0000 7faaba508640 1 --2- 172.21.15.172:0/1019849696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faab41027c0 0x7faab4074a30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faaa800c920 tx=0x7faaa800cde0 comp rx=0 tx=0).stop 2024-09-10T08:45:21.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.720+0000 7faab8aa5640 1 -- 172.21.15.172:0/1019849696 reap_dead start 2024-09-10T08:45:21.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.720+0000 7faaba508640 1 -- 172.21.15.172:0/1019849696 shutdown_connections 2024-09-10T08:45:21.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.720+0000 7faaba508640 1 -- 172.21.15.172:0/1019849696 >> 172.21.15.172:0/1019849696 conn(0x7faab40fd8d0 msgr2=0x7faab40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:21.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.720+0000 7faaba508640 1 -- 172.21.15.172:0/1019849696 shutdown_connections 2024-09-10T08:45:21.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.720+0000 7faaba508640 1 -- 172.21.15.172:0/1019849696 wait complete. 2024-09-10T08:45:22.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.228+0000 7f5e50f58640 1 -- 172.21.15.172:0/3154162879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c100e20 msgr2=0x7f5e4c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:22.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.228+0000 7f5e50f58640 1 --2- 172.21.15.172:0/3154162879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c100e20 0x7f5e4c101200 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009900 tx=0x7f5e3c02a480 comp rx=0 tx=0).stop 2024-09-10T08:45:22.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.228+0000 7f5e50f58640 1 -- 172.21.15.172:0/3154162879 shutdown_connections 2024-09-10T08:45:22.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.228+0000 7f5e50f58640 1 --2- 172.21.15.172:0/3154162879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e4c102ef0 0x7f5e4c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:22.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.228+0000 7f5e50f58640 1 --2- 172.21.15.172:0/3154162879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e4c102570 0x7f5e4c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.228+0000 7f5e50f58640 1 --2- 172.21.15.172:0/3154162879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c100e20 0x7f5e4c101200 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.228+0000 7f5e50f58640 1 -- 172.21.15.172:0/3154162879 >> 172.21.15.172:0/3154162879 conn(0x7f5e4c0fc7d0 msgr2=0x7f5e4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.228+0000 7f5e50f58640 1 -- 172.21.15.172:0/3154162879 shutdown_connections 2024-09-10T08:45:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.228+0000 7f5e50f58640 1 -- 172.21.15.172:0/3154162879 wait complete. 2024-09-10T08:45:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e50f58640 1 Processor -- start 2024-09-10T08:45:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e50f58640 1 -- start start 2024-09-10T08:45:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e50f58640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e4c100e20 0x7f5e4c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e50f58640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e4c102570 0x7f5e4c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e50f58640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c102ef0 0x7f5e4c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e50f58640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c110e30 con 0x7f5e4c102ef0 2024-09-10T08:45:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e50f58640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c110cb0 con 0x7f5e4c100e20 2024-09-10T08:45:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e50f58640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c110fb0 con 0x7f5e4c102570 2024-09-10T08:45:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e4ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c102ef0 0x7f5e4c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e4ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c102ef0 0x7f5e4c19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42372/0 (socket says 172.21.15.172:42372) 2024-09-10T08:45:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e4ad9d640 1 -- 172.21.15.172:0/1951583039 learned_addr learned my addr 172.21.15.172:0/1951583039 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e49d9b640 1 --2- 172.21.15.172:0/1951583039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e4c102570 0x7f5e4c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e4a59c640 1 --2- 172.21.15.172:0/1951583039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e4c100e20 0x7f5e4c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e4ad9d640 1 -- 172.21.15.172:0/1951583039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e4c102570 msgr2=0x7f5e4c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e4ad9d640 1 --2- 172.21.15.172:0/1951583039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e4c102570 0x7f5e4c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e4ad9d640 1 -- 172.21.15.172:0/1951583039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e4c100e20 msgr2=0x7f5e4c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:22.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e4ad9d640 1 --2- 172.21.15.172:0/1951583039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e4c100e20 0x7f5e4c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:22.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.229+0000 7f5e4ad9d640 1 -- 172.21.15.172:0/1951583039 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4c19d3e0 con 0x7f5e4c102ef0 2024-09-10T08:45:22.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.230+0000 7f5e4ad9d640 1 --2- 172.21.15.172:0/1951583039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c102ef0 0x7f5e4c19cd00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400eb10 tx=0x7f5e4400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:22.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.230+0000 7f5e3b7fe640 1 -- 172.21.15.172:0/1951583039 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e44004b20 con 0x7f5e4c102ef0 2024-09-10T08:45:22.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.230+0000 7f5e3b7fe640 1 -- 172.21.15.172:0/1951583039 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e44004cc0 con 0x7f5e4c102ef0 2024-09-10T08:45:22.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.230+0000 7f5e50f58640 1 -- 172.21.15.172:0/1951583039 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4c19d6d0 con 0x7f5e4c102ef0 2024-09-10T08:45:22.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.230+0000 7f5e3b7fe640 1 -- 172.21.15.172:0/1951583039 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e44010690 con 0x7f5e4c102ef0 2024-09-10T08:45:22.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.230+0000 7f5e50f58640 1 -- 172.21.15.172:0/1951583039 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e4c1a4fb0 con 0x7f5e4c102ef0 2024-09-10T08:45:22.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.231+0000 7f5e50f58640 1 -- 172.21.15.172:0/1951583039 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5e4c106bf0 con 0x7f5e4c102ef0 2024-09-10T08:45:22.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.232+0000 7f5e3b7fe640 1 -- 172.21.15.172:0/1951583039 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5e44005020 con 0x7f5e4c102ef0 2024-09-10T08:45:22.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.233+0000 7f5e3b7fe640 1 --2- 172.21.15.172:0/1951583039 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5e340a2c10 0x7f5e340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:22.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.233+0000 7f5e4a59c640 1 --2- 172.21.15.172:0/1951583039 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5e340a2c10 0x7f5e340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:22.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.233+0000 7f5e3b7fe640 1 -- 172.21.15.172:0/1951583039 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f5e440cff50 con 0x7f5e4c102ef0 2024-09-10T08:45:22.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.233+0000 7f5e3b7fe640 1 --2- 172.21.15.172:0/1951583039 >> [v2:172.21.15.191:6800/1963796290,v1:172.21.15.191:6801/1963796290] conn(0x7f5e340adcd0 0x7f5e340b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:22.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.233+0000 7f5e3b7fe640 1 -- 172.21.15.172:0/1951583039 --> [v2:172.21.15.191:6800/1963796290,v1:172.21.15.191:6801/1963796290] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e440047f0 con 0x7f5e340adcd0 2024-09-10T08:45:22.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.233+0000 7f5e3b7fe640 1 -- 172.21.15.172:0/1951583039 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f5e440d03a0 con 0x7f5e4c102ef0 2024-09-10T08:45:22.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.233+0000 7f5e4a59c640 1 --2- 172.21.15.172:0/1951583039 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5e340a2c10 0x7f5e340a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c005ab0 tx=0x7f5e3c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:22.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.233+0000 7f5e49d9b640 1 --2- 172.21.15.172:0/1951583039 >> [v2:172.21.15.191:6800/1963796290,v1:172.21.15.191:6801/1963796290] conn(0x7f5e340adcd0 0x7f5e340b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:22.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.234+0000 7f5e49d9b640 1 --2- 172.21.15.172:0/1951583039 >> [v2:172.21.15.191:6800/1963796290,v1:172.21.15.191:6801/1963796290] conn(0x7f5e340adcd0 0x7f5e340b0170 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:22.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.235+0000 7f5e3b7fe640 1 -- 172.21.15.172:0/1951583039 <== osd.2 v2:172.21.15.191:6800/1963796290 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e440047f0 con 0x7f5e340adcd0 2024-09-10T08:45:22.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.248+0000 7f5e50f58640 1 -- 172.21.15.172:0/1951583039 --> [v2:172.21.15.191:6800/1963796290,v1:172.21.15.191:6801/1963796290] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5e4c1021b0 con 0x7f5e340adcd0 2024-09-10T08:45:22.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.249+0000 7f5e3b7fe640 1 -- 172.21.15.172:0/1951583039 <== osd.2 v2:172.21.15.191:6800/1963796290 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5e4c1021b0 con 0x7f5e340adcd0 2024-09-10T08:45:22.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.249+0000 7f5e50f58640 1 -- 172.21.15.172:0/1951583039 >> [v2:172.21.15.191:6800/1963796290,v1:172.21.15.191:6801/1963796290] conn(0x7f5e340adcd0 msgr2=0x7f5e340b0170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:22.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.249+0000 7f5e50f58640 1 --2- 172.21.15.172:0/1951583039 >> [v2:172.21.15.191:6800/1963796290,v1:172.21.15.191:6801/1963796290] conn(0x7f5e340adcd0 0x7f5e340b0170 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:22.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.249+0000 7f5e50f58640 1 -- 172.21.15.172:0/1951583039 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5e340a2c10 msgr2=0x7f5e340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:22.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.249+0000 7f5e50f58640 1 --2- 172.21.15.172:0/1951583039 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5e340a2c10 0x7f5e340a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c005ab0 tx=0x7f5e3c005a20 comp rx=0 tx=0).stop 2024-09-10T08:45:22.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.249+0000 7f5e50f58640 1 -- 172.21.15.172:0/1951583039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c102ef0 msgr2=0x7f5e4c19cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:22.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.249+0000 7f5e50f58640 1 --2- 172.21.15.172:0/1951583039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c102ef0 0x7f5e4c19cd00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400eb10 tx=0x7f5e4400c2d0 comp rx=0 tx=0).stop 2024-09-10T08:45:22.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.249+0000 7f5e4ad9d640 1 -- 172.21.15.172:0/1951583039 reap_dead start 2024-09-10T08:45:22.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.250+0000 7f5e50f58640 1 -- 172.21.15.172:0/1951583039 shutdown_connections 2024-09-10T08:45:22.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.250+0000 7f5e50f58640 1 -- 172.21.15.172:0/1951583039 >> 172.21.15.172:0/1951583039 conn(0x7f5e4c0fc7d0 msgr2=0x7f5e4c06a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:22.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.250+0000 7f5e50f58640 1 -- 172.21.15.172:0/1951583039 shutdown_connections 2024-09-10T08:45:22.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.250+0000 7f5e50f58640 1 -- 172.21.15.172:0/1951583039 wait complete. 2024-09-10T08:45:22.411 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:22 smithi172 ceph-mon[30854]: pgmap v322: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:22 smithi191 ceph-mon[35314]: pgmap v322: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:22.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:22 smithi174 ceph-mon[34068]: pgmap v322: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:23.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.231+0000 7ff40064d640 1 -- 172.21.15.172:0/950706867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3f8102ef0 msgr2=0x7ff3f810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:23.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.231+0000 7ff40064d640 1 --2- 172.21.15.172:0/950706867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3f8102ef0 0x7ff3f810da40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009ca0 tx=0x7ff3f002a950 comp rx=0 tx=0).stop 2024-09-10T08:45:23.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.231+0000 7ff40064d640 1 -- 172.21.15.172:0/950706867 shutdown_connections 2024-09-10T08:45:23.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.231+0000 7ff40064d640 1 --2- 172.21.15.172:0/950706867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3f8102ef0 0x7ff3f810da40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:23.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.231+0000 7ff40064d640 1 --2- 172.21.15.172:0/950706867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3f8102570 0x7ff3f81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:23.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.231+0000 7ff40064d640 1 --2- 172.21.15.172:0/950706867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f8100e20 0x7ff3f8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:23.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.231+0000 7ff40064d640 1 -- 172.21.15.172:0/950706867 >> 172.21.15.172:0/950706867 conn(0x7ff3f80fc7d0 msgr2=0x7ff3f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:23.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.231+0000 7ff40064d640 1 -- 172.21.15.172:0/950706867 shutdown_connections 2024-09-10T08:45:23.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.231+0000 7ff40064d640 1 -- 172.21.15.172:0/950706867 wait complete. 2024-09-10T08:45:23.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.231+0000 7ff40064d640 1 Processor -- start 2024-09-10T08:45:23.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.231+0000 7ff40064d640 1 -- start start 2024-09-10T08:45:23.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff40064d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3f8100e20 0x7ff3f8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:23.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff40064d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f8102570 0x7ff3f81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:23.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff40064d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3f8102ef0 0x7ff3f819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:23.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff40064d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f8110e30 con 0x7ff3f8102ef0 2024-09-10T08:45:23.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff40064d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f8110cb0 con 0x7ff3f8102570 2024-09-10T08:45:23.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff40064d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f8110fb0 con 0x7ff3f8100e20 2024-09-10T08:45:23.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff3febea640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3f8102ef0 0x7ff3f819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:23.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff3febea640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3f8102ef0 0x7ff3f819cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42414/0 (socket says 172.21.15.172:42414) 2024-09-10T08:45:23.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff3febea640 1 -- 172.21.15.172:0/3810793477 learned_addr learned my addr 172.21.15.172:0/3810793477 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:23.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff3fdbe8640 1 --2- 172.21.15.172:0/3810793477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f8102570 0x7ff3f81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:23.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff3fe3e9640 1 --2- 172.21.15.172:0/3810793477 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3f8100e20 0x7ff3f8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:23.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff3febea640 1 -- 172.21.15.172:0/3810793477 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3f8100e20 msgr2=0x7ff3f8198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:23.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff3febea640 1 --2- 172.21.15.172:0/3810793477 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3f8100e20 0x7ff3f8198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:23.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff3febea640 1 -- 172.21.15.172:0/3810793477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f8102570 msgr2=0x7ff3f81988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:23.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff3febea640 1 --2- 172.21.15.172:0/3810793477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3f8102570 0x7ff3f81988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:23.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff3febea640 1 -- 172.21.15.172:0/3810793477 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f819d340 con 0x7ff3f8102ef0 2024-09-10T08:45:23.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff3febea640 1 --2- 172.21.15.172:0/3810793477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3f8102ef0 0x7ff3f819cc60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0004af0 tx=0x7ff3f002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:23.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff3ef7fe640 1 -- 172.21.15.172:0/3810793477 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0034040 con 0x7ff3f8102ef0 2024-09-10T08:45:23.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff3ef7fe640 1 -- 172.21.15.172:0/3810793477 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3f00308f0 con 0x7ff3f8102ef0 2024-09-10T08:45:23.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff40064d640 1 -- 172.21.15.172:0/3810793477 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f819d5d0 con 0x7ff3f8102ef0 2024-09-10T08:45:23.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.232+0000 7ff3ef7fe640 1 -- 172.21.15.172:0/3810793477 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0004670 con 0x7ff3f8102ef0 2024-09-10T08:45:23.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.233+0000 7ff40064d640 1 -- 172.21.15.172:0/3810793477 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f81a4f10 con 0x7ff3f8102ef0 2024-09-10T08:45:23.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.233+0000 7ff40064d640 1 -- 172.21.15.172:0/3810793477 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff3f8062940 con 0x7ff3f8102ef0 2024-09-10T08:45:23.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.235+0000 7ff3ef7fe640 1 -- 172.21.15.172:0/3810793477 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff3f0037020 con 0x7ff3f8102ef0 2024-09-10T08:45:23.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.235+0000 7ff3ef7fe640 1 --2- 172.21.15.172:0/3810793477 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff3e40a2c90 0x7ff3e40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:23.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.235+0000 7ff3fe3e9640 1 --2- 172.21.15.172:0/3810793477 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff3e40a2c90 0x7ff3e40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:23.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.235+0000 7ff3ef7fe640 1 -- 172.21.15.172:0/3810793477 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7ff3f00f0670 con 0x7ff3f8102ef0 2024-09-10T08:45:23.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.236+0000 7ff3ef7fe640 1 --2- 172.21.15.172:0/3810793477 >> [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619] conn(0x7ff3e40adde0 0x7ff3e40b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:23.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.236+0000 7ff3ef7fe640 1 -- 172.21.15.172:0/3810793477 --> [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff3f00020b0 con 0x7ff3e40adde0 2024-09-10T08:45:23.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.236+0000 7ff3ef7fe640 1 -- 172.21.15.172:0/3810793477 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff3f00f0ac0 con 0x7ff3f8102ef0 2024-09-10T08:45:23.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.236+0000 7ff3fdbe8640 1 --2- 172.21.15.172:0/3810793477 >> [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619] conn(0x7ff3e40adde0 0x7ff3e40b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:23.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.236+0000 7ff3fe3e9640 1 --2- 172.21.15.172:0/3810793477 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff3e40a2c90 0x7ff3e40a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009bd0 tx=0x7ff3f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:23.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.236+0000 7ff3fdbe8640 1 --2- 172.21.15.172:0/3810793477 >> [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619] conn(0x7ff3e40adde0 0x7ff3e40b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:23.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.237+0000 7ff3ef7fe640 1 -- 172.21.15.172:0/3810793477 <== osd.9 v2:172.21.15.172:6826/1181268619 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff3f00020b0 con 0x7ff3e40adde0 2024-09-10T08:45:23.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.250+0000 7ff40064d640 1 -- 172.21.15.172:0/3810793477 --> [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff3f8062d40 con 0x7ff3e40adde0 2024-09-10T08:45:23.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.251+0000 7ff3ef7fe640 1 -- 172.21.15.172:0/3810793477 <== osd.9 v2:172.21.15.172:6826/1181268619 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff3f8062d40 con 0x7ff3e40adde0 2024-09-10T08:45:23.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.251+0000 7ff40064d640 1 -- 172.21.15.172:0/3810793477 >> [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619] conn(0x7ff3e40adde0 msgr2=0x7ff3e40b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:23.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.251+0000 7ff40064d640 1 --2- 172.21.15.172:0/3810793477 >> [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619] conn(0x7ff3e40adde0 0x7ff3e40b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:23.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.251+0000 7ff40064d640 1 -- 172.21.15.172:0/3810793477 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff3e40a2c90 msgr2=0x7ff3e40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:23.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.251+0000 7ff40064d640 1 --2- 172.21.15.172:0/3810793477 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff3e40a2c90 0x7ff3e40a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009bd0 tx=0x7ff3f4005f90 comp rx=0 tx=0).stop 2024-09-10T08:45:23.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.251+0000 7ff40064d640 1 -- 172.21.15.172:0/3810793477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3f8102ef0 msgr2=0x7ff3f819cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:23.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.251+0000 7ff40064d640 1 --2- 172.21.15.172:0/3810793477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3f8102ef0 0x7ff3f819cc60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0004af0 tx=0x7ff3f002f5d0 comp rx=0 tx=0).stop 2024-09-10T08:45:23.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.251+0000 7ff3febea640 1 -- 172.21.15.172:0/3810793477 reap_dead start 2024-09-10T08:45:23.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.251+0000 7ff40064d640 1 -- 172.21.15.172:0/3810793477 shutdown_connections 2024-09-10T08:45:23.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.251+0000 7ff40064d640 1 -- 172.21.15.172:0/3810793477 >> 172.21.15.172:0/3810793477 conn(0x7ff3f80fc7d0 msgr2=0x7ff3f806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:23.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.251+0000 7ff40064d640 1 -- 172.21.15.172:0/3810793477 shutdown_connections 2024-09-10T08:45:23.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:23.251+0000 7ff40064d640 1 -- 172.21.15.172:0/3810793477 wait complete. 2024-09-10T08:45:23.875 INFO:teuthology.orchestra.run.smithi172.stdout:77309411401 2024-09-10T08:45:23.875 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T08:45:24.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.239+0000 7fcb1b190640 1 -- 172.21.15.172:0/386912783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb14106a80 msgr2=0x7fcb14106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:24.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.239+0000 7fcb1b190640 1 --2- 172.21.15.172:0/386912783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb14106a80 0x7fcb14106e60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009930 tx=0x7fcb0802a420 comp rx=0 tx=0).stop 2024-09-10T08:45:24.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.239+0000 7fcb1b190640 1 -- 172.21.15.172:0/386912783 shutdown_connections 2024-09-10T08:45:24.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.239+0000 7fcb1b190640 1 --2- 172.21.15.172:0/386912783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb14101420 0x7fcb1410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.239+0000 7fcb1b190640 1 --2- 172.21.15.172:0/386912783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb14100a80 0x7fcb14100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.239+0000 7fcb1b190640 1 --2- 172.21.15.172:0/386912783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb14106a80 0x7fcb14106e60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.239+0000 7fcb1b190640 1 -- 172.21.15.172:0/386912783 >> 172.21.15.172:0/386912783 conn(0x7fcb140fc7a0 msgr2=0x7fcb140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:24.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.239+0000 7fcb1b190640 1 -- 172.21.15.172:0/386912783 shutdown_connections 2024-09-10T08:45:24.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.239+0000 7fcb1b190640 1 -- 172.21.15.172:0/386912783 wait complete. 2024-09-10T08:45:24.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.239+0000 7fcb1b190640 1 Processor -- start 2024-09-10T08:45:24.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.239+0000 7fcb1b190640 1 -- start start 2024-09-10T08:45:24.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb1b190640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb14100a80 0x7fcb14199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:24.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb1b190640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb14101420 0x7fcb141997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:24.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb1b190640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb14106a80 0x7fcb141a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:24.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb1b190640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcb14110d50 con 0x7fcb14100a80 2024-09-10T08:45:24.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb1b190640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcb14110bd0 con 0x7fcb14101420 2024-09-10T08:45:24.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb1b190640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcb14110ed0 con 0x7fcb14106a80 2024-09-10T08:45:24.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb18f2c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb14100a80 0x7fcb14199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:24.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb18f2c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb14100a80 0x7fcb14199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42436/0 (socket says 172.21.15.172:42436) 2024-09-10T08:45:24.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb18f2c640 1 -- 172.21.15.172:0/2801363757 learned_addr learned my addr 172.21.15.172:0/2801363757 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:24.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb13fff640 1 --2- 172.21.15.172:0/2801363757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb14101420 0x7fcb141997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:24.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb18f2c640 1 -- 172.21.15.172:0/2801363757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb14106a80 msgr2=0x7fcb141a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:24.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb1972d640 1 --2- 172.21.15.172:0/2801363757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb14106a80 0x7fcb141a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:24.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb18f2c640 1 --2- 172.21.15.172:0/2801363757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb14106a80 0x7fcb141a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb18f2c640 1 -- 172.21.15.172:0/2801363757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb14101420 msgr2=0x7fcb141997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:24.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb18f2c640 1 --2- 172.21.15.172:0/2801363757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb14101420 0x7fcb141997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb18f2c640 1 -- 172.21.15.172:0/2801363757 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb141a4840 con 0x7fcb14100a80 2024-09-10T08:45:24.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb1972d640 1 --2- 172.21.15.172:0/2801363757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb14106a80 0x7fcb141a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:24.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb13fff640 1 --2- 172.21.15.172:0/2801363757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb14101420 0x7fcb141997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:24.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.240+0000 7fcb18f2c640 1 --2- 172.21.15.172:0/2801363757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb14100a80 0x7fcb14199280 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcb08005940 tx=0x7fcb080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:24.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.241+0000 7fcb11ffb640 1 -- 172.21.15.172:0/2801363757 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0803f030 con 0x7fcb14100a80 2024-09-10T08:45:24.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.241+0000 7fcb11ffb640 1 -- 172.21.15.172:0/2801363757 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb08003cb0 con 0x7fcb14100a80 2024-09-10T08:45:24.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.241+0000 7fcb1b190640 1 -- 172.21.15.172:0/2801363757 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb141a4a10 con 0x7fcb14100a80 2024-09-10T08:45:24.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.241+0000 7fcb11ffb640 1 -- 172.21.15.172:0/2801363757 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08035070 con 0x7fcb14100a80 2024-09-10T08:45:24.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.241+0000 7fcb1b190640 1 -- 172.21.15.172:0/2801363757 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb141a4e60 con 0x7fcb14100a80 2024-09-10T08:45:24.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.242+0000 7fcb1b190640 1 -- 172.21.15.172:0/2801363757 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcb14062900 con 0x7fcb14100a80 2024-09-10T08:45:24.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.243+0000 7fcb11ffb640 1 -- 172.21.15.172:0/2801363757 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fcb08042030 con 0x7fcb14100a80 2024-09-10T08:45:24.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.244+0000 7fcb11ffb640 1 --2- 172.21.15.172:0/2801363757 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcb040a2c10 0x7fcb040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:24.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.244+0000 7fcb13fff640 1 --2- 172.21.15.172:0/2801363757 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcb040a2c10 0x7fcb040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:24.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.244+0000 7fcb11ffb640 1 -- 172.21.15.172:0/2801363757 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fcb080f0230 con 0x7fcb14100a80 2024-09-10T08:45:24.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.244+0000 7fcb11ffb640 1 --2- 172.21.15.172:0/2801363757 >> [v2:172.21.15.191:6816/3675210430,v1:172.21.15.191:6817/3675210430] conn(0x7fcb040add60 0x7fcb040b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:24.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.244+0000 7fcb11ffb640 1 -- 172.21.15.172:0/2801363757 --> [v2:172.21.15.191:6816/3675210430,v1:172.21.15.191:6817/3675210430] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcb040b08d0 con 0x7fcb040add60 2024-09-10T08:45:24.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.244+0000 7fcb11ffb640 1 -- 172.21.15.172:0/2801363757 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fcb080f0680 con 0x7fcb14100a80 2024-09-10T08:45:24.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.244+0000 7fcb13fff640 1 --2- 172.21.15.172:0/2801363757 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcb040a2c10 0x7fcb040a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcb1406b2e0 tx=0x7fcb00008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:24.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.244+0000 7fcb1972d640 1 --2- 172.21.15.172:0/2801363757 >> [v2:172.21.15.191:6816/3675210430,v1:172.21.15.191:6817/3675210430] conn(0x7fcb040add60 0x7fcb040b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:24.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.245+0000 7fcb1972d640 1 --2- 172.21.15.172:0/2801363757 >> [v2:172.21.15.191:6816/3675210430,v1:172.21.15.191:6817/3675210430] conn(0x7fcb040add60 0x7fcb040b0200 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:24.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.245+0000 7fcb11ffb640 1 -- 172.21.15.172:0/2801363757 <== osd.8 v2:172.21.15.191:6816/3675210430 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcb040b08d0 con 0x7fcb040add60 2024-09-10T08:45:24.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.259+0000 7fcb1b190640 1 -- 172.21.15.172:0/2801363757 --> [v2:172.21.15.191:6816/3675210430,v1:172.21.15.191:6817/3675210430] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcb14062d00 con 0x7fcb040add60 2024-09-10T08:45:24.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.259+0000 7fcb11ffb640 1 -- 172.21.15.172:0/2801363757 <== osd.8 v2:172.21.15.191:6816/3675210430 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcb14062d00 con 0x7fcb040add60 2024-09-10T08:45:24.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.260+0000 7fcb1b190640 1 -- 172.21.15.172:0/2801363757 >> [v2:172.21.15.191:6816/3675210430,v1:172.21.15.191:6817/3675210430] conn(0x7fcb040add60 msgr2=0x7fcb040b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:24.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.260+0000 7fcb1b190640 1 --2- 172.21.15.172:0/2801363757 >> [v2:172.21.15.191:6816/3675210430,v1:172.21.15.191:6817/3675210430] conn(0x7fcb040add60 0x7fcb040b0200 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.260+0000 7fcb1b190640 1 -- 172.21.15.172:0/2801363757 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcb040a2c10 msgr2=0x7fcb040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:24.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.260+0000 7fcb1b190640 1 --2- 172.21.15.172:0/2801363757 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcb040a2c10 0x7fcb040a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcb1406b2e0 tx=0x7fcb00008040 comp rx=0 tx=0).stop 2024-09-10T08:45:24.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.260+0000 7fcb1b190640 1 -- 172.21.15.172:0/2801363757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb14100a80 msgr2=0x7fcb14199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:24.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.260+0000 7fcb1b190640 1 --2- 172.21.15.172:0/2801363757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb14100a80 0x7fcb14199280 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcb08005940 tx=0x7fcb080042f0 comp rx=0 tx=0).stop 2024-09-10T08:45:24.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.260+0000 7fcb1972d640 1 -- 172.21.15.172:0/2801363757 reap_dead start 2024-09-10T08:45:24.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.260+0000 7fcb1b190640 1 -- 172.21.15.172:0/2801363757 shutdown_connections 2024-09-10T08:45:24.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.260+0000 7fcb1b190640 1 -- 172.21.15.172:0/2801363757 >> 172.21.15.172:0/2801363757 conn(0x7fcb140fc7a0 msgr2=0x7fcb140fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:24.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.260+0000 7fcb1b190640 1 -- 172.21.15.172:0/2801363757 shutdown_connections 2024-09-10T08:45:24.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:24.260+0000 7fcb1b190640 1 -- 172.21.15.172:0/2801363757 wait complete. 2024-09-10T08:45:24.475 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:24 smithi172 ceph-mon[30854]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:24 smithi191 ceph-mon[35314]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:24.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:24 smithi174 ceph-mon[34068]: pgmap v323: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:25.223 INFO:teuthology.orchestra.run.smithi172.stdout:141733920831 2024-09-10T08:45:25.224 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T08:45:25.810 INFO:teuthology.orchestra.run.smithi172.stdout:34359738450 2024-09-10T08:45:25.810 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T08:45:26.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.151+0000 7fbfd6f41640 1 -- 172.21.15.172:0/2717993687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfd0100a80 msgr2=0x7fbfd0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.151+0000 7fbfd6f41640 1 --2- 172.21.15.172:0/2717993687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfd0100a80 0x7fbfd0100ee0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009930 tx=0x7fbfbc02a420 comp rx=0 tx=0).stop 2024-09-10T08:45:26.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.151+0000 7fbfd6f41640 1 -- 172.21.15.172:0/2717993687 shutdown_connections 2024-09-10T08:45:26.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.151+0000 7fbfd6f41640 1 --2- 172.21.15.172:0/2717993687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfd0101420 0x7fbfd010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.151+0000 7fbfd6f41640 1 --2- 172.21.15.172:0/2717993687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfd0100a80 0x7fbfd0100ee0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.151+0000 7fbfd6f41640 1 --2- 172.21.15.172:0/2717993687 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfd0106a80 0x7fbfd0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.151+0000 7fbfd6f41640 1 -- 172.21.15.172:0/2717993687 >> 172.21.15.172:0/2717993687 conn(0x7fbfd00fc7a0 msgr2=0x7fbfd00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:26.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.151+0000 7fbfd6f41640 1 -- 172.21.15.172:0/2717993687 shutdown_connections 2024-09-10T08:45:26.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.151+0000 7fbfd6f41640 1 -- 172.21.15.172:0/2717993687 wait complete. 2024-09-10T08:45:26.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.152+0000 7fbfd6f41640 1 Processor -- start 2024-09-10T08:45:26.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.152+0000 7fbfd6f41640 1 -- start start 2024-09-10T08:45:26.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.152+0000 7fbfd6f41640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfd0100a80 0x7fbfd0197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.152+0000 7fbfd6f41640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfd0101420 0x7fbfd01977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.152+0000 7fbfd6f41640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfd0106a80 0x7fbfd019bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.152+0000 7fbfd6f41640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0110d50 con 0x7fbfd0101420 2024-09-10T08:45:26.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.152+0000 7fbfd6f41640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0110bd0 con 0x7fbfd0100a80 2024-09-10T08:45:26.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.152+0000 7fbfd6f41640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0110ed0 con 0x7fbfd0106a80 2024-09-10T08:45:26.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.152+0000 7fbfcffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfd0101420 0x7fbfd01977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.152+0000 7fbfcffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfd0101420 0x7fbfd01977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42448/0 (socket says 172.21.15.172:42448) 2024-09-10T08:45:26.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.152+0000 7fbfcffff640 1 -- 172.21.15.172:0/2792725268 learned_addr learned my addr 172.21.15.172:0/2792725268 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:26.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.152+0000 7fbfd4cdd640 1 --2- 172.21.15.172:0/2792725268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfd0100a80 0x7fbfd0197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.153+0000 7fbfd54de640 1 --2- 172.21.15.172:0/2792725268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfd0106a80 0x7fbfd019bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.153+0000 7fbfcffff640 1 -- 172.21.15.172:0/2792725268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfd0106a80 msgr2=0x7fbfd019bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.153+0000 7fbfcffff640 1 --2- 172.21.15.172:0/2792725268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfd0106a80 0x7fbfd019bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.153+0000 7fbfcffff640 1 -- 172.21.15.172:0/2792725268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfd0100a80 msgr2=0x7fbfd0197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.153+0000 7fbfcffff640 1 --2- 172.21.15.172:0/2792725268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfd0100a80 0x7fbfd0197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.153+0000 7fbfcffff640 1 -- 172.21.15.172:0/2792725268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd019c240 con 0x7fbfd0101420 2024-09-10T08:45:26.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.153+0000 7fbfd54de640 1 --2- 172.21.15.172:0/2792725268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbfd0106a80 0x7fbfd019bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:26.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.153+0000 7fbfcffff640 1 --2- 172.21.15.172:0/2792725268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfd0101420 0x7fbfd01977d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc005940 tx=0x7fbfbc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:26.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.153+0000 7fbfcdffb640 1 -- 172.21.15.172:0/2792725268 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc03f030 con 0x7fbfd0101420 2024-09-10T08:45:26.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.153+0000 7fbfcdffb640 1 -- 172.21.15.172:0/2792725268 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfbc003cb0 con 0x7fbfd0101420 2024-09-10T08:45:26.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.153+0000 7fbfcdffb640 1 -- 172.21.15.172:0/2792725268 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc035070 con 0x7fbfd0101420 2024-09-10T08:45:26.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.153+0000 7fbfd6f41640 1 -- 172.21.15.172:0/2792725268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd019c4d0 con 0x7fbfd0101420 2024-09-10T08:45:26.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.153+0000 7fbfd6f41640 1 -- 172.21.15.172:0/2792725268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd01a3e10 con 0x7fbfd0101420 2024-09-10T08:45:26.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.154+0000 7fbfd6f41640 1 -- 172.21.15.172:0/2792725268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbfd0062980 con 0x7fbfd0101420 2024-09-10T08:45:26.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.155+0000 7fbfcdffb640 1 -- 172.21.15.172:0/2792725268 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fbfbc042030 con 0x7fbfd0101420 2024-09-10T08:45:26.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.156+0000 7fbfcdffb640 1 --2- 172.21.15.172:0/2792725268 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbfc00a2c10 0x7fbfc00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.156+0000 7fbfd4cdd640 1 --2- 172.21.15.172:0/2792725268 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbfc00a2c10 0x7fbfc00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.156+0000 7fbfcdffb640 1 -- 172.21.15.172:0/2792725268 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fbfbc0f01d0 con 0x7fbfd0101420 2024-09-10T08:45:26.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.156+0000 7fbfcdffb640 1 --2- 172.21.15.172:0/2792725268 >> [v2:172.21.15.174:6816/1461923581,v1:172.21.15.174:6817/1461923581] conn(0x7fbfc00add60 0x7fbfc00b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.156+0000 7fbfcdffb640 1 -- 172.21.15.172:0/2792725268 --> [v2:172.21.15.174:6816/1461923581,v1:172.21.15.174:6817/1461923581] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbfc00b08d0 con 0x7fbfc00add60 2024-09-10T08:45:26.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.156+0000 7fbfcdffb640 1 -- 172.21.15.172:0/2792725268 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fbfbc0f0620 con 0x7fbfd0101420 2024-09-10T08:45:26.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.157+0000 7fbfd4cdd640 1 --2- 172.21.15.172:0/2792725268 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbfc00a2c10 0x7fbfc00a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009990 tx=0x7fbfc4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:26.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.157+0000 7fbfd54de640 1 --2- 172.21.15.172:0/2792725268 >> [v2:172.21.15.174:6816/1461923581,v1:172.21.15.174:6817/1461923581] conn(0x7fbfc00add60 0x7fbfc00b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.157+0000 7fbfd54de640 1 --2- 172.21.15.172:0/2792725268 >> [v2:172.21.15.174:6816/1461923581,v1:172.21.15.174:6817/1461923581] conn(0x7fbfc00add60 0x7fbfc00b0200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:26.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.158+0000 7fbfcdffb640 1 -- 172.21.15.172:0/2792725268 <== osd.7 v2:172.21.15.174:6816/1461923581 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbfc00b08d0 con 0x7fbfc00add60 2024-09-10T08:45:26.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.172+0000 7fbfd6f41640 1 -- 172.21.15.172:0/2792725268 --> [v2:172.21.15.174:6816/1461923581,v1:172.21.15.174:6817/1461923581] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbfd0062e00 con 0x7fbfc00add60 2024-09-10T08:45:26.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.172+0000 7fbfcdffb640 1 -- 172.21.15.172:0/2792725268 <== osd.7 v2:172.21.15.174:6816/1461923581 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbfd0062e00 con 0x7fbfc00add60 2024-09-10T08:45:26.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.172+0000 7fbfd6f41640 1 -- 172.21.15.172:0/2792725268 >> [v2:172.21.15.174:6816/1461923581,v1:172.21.15.174:6817/1461923581] conn(0x7fbfc00add60 msgr2=0x7fbfc00b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.172+0000 7fbfd6f41640 1 --2- 172.21.15.172:0/2792725268 >> [v2:172.21.15.174:6816/1461923581,v1:172.21.15.174:6817/1461923581] conn(0x7fbfc00add60 0x7fbfc00b0200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.172+0000 7fbfd6f41640 1 -- 172.21.15.172:0/2792725268 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbfc00a2c10 msgr2=0x7fbfc00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.173+0000 7fbfd6f41640 1 --2- 172.21.15.172:0/2792725268 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbfc00a2c10 0x7fbfc00a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009990 tx=0x7fbfc4009110 comp rx=0 tx=0).stop 2024-09-10T08:45:26.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.173+0000 7fbfd6f41640 1 -- 172.21.15.172:0/2792725268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfd0101420 msgr2=0x7fbfd01977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.173+0000 7fbfd6f41640 1 --2- 172.21.15.172:0/2792725268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbfd0101420 0x7fbfd01977d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc005940 tx=0x7fbfbc0042f0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.173+0000 7fbfd54de640 1 -- 172.21.15.172:0/2792725268 reap_dead start 2024-09-10T08:45:26.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.173+0000 7fbfd6f41640 1 -- 172.21.15.172:0/2792725268 shutdown_connections 2024-09-10T08:45:26.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.173+0000 7fbfd6f41640 1 -- 172.21.15.172:0/2792725268 >> 172.21.15.172:0/2792725268 conn(0x7fbfd00fc7a0 msgr2=0x7fbfd00fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:26.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.173+0000 7fbfd6f41640 1 -- 172.21.15.172:0/2792725268 shutdown_connections 2024-09-10T08:45:26.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.173+0000 7fbfd6f41640 1 -- 172.21.15.172:0/2792725268 wait complete. 2024-09-10T08:45:26.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:26 smithi172 ceph-mon[30854]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 639 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:26.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:26 smithi191 ceph-mon[35314]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 639 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:26.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.611+0000 7f2eddb78640 1 -- 172.21.15.172:0/2675906637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ed8102790 msgr2=0x7f2ed81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.611+0000 7f2eddb78640 1 --2- 172.21.15.172:0/2675906637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ed8102790 0x7f2ed81067e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009900 tx=0x7f2ecc02a410 comp rx=0 tx=0).stop 2024-09-10T08:45:26.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.611+0000 7f2eddb78640 1 -- 172.21.15.172:0/2675906637 shutdown_connections 2024-09-10T08:45:26.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.611+0000 7f2eddb78640 1 --2- 172.21.15.172:0/2675906637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ed8106f10 0x7f2ed810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.611+0000 7f2eddb78640 1 --2- 172.21.15.172:0/2675906637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ed8102790 0x7f2ed81067e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.611+0000 7f2eddb78640 1 --2- 172.21.15.172:0/2675906637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ed8101de0 0x7f2ed81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.611+0000 7f2eddb78640 1 -- 172.21.15.172:0/2675906637 >> 172.21.15.172:0/2675906637 conn(0x7f2ed80fd8c0 msgr2=0x7f2ed80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:26.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.611+0000 7f2eddb78640 1 -- 172.21.15.172:0/2675906637 shutdown_connections 2024-09-10T08:45:26.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.611+0000 7f2eddb78640 1 -- 172.21.15.172:0/2675906637 wait complete. 2024-09-10T08:45:26.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.611+0000 7f2eddb78640 1 Processor -- start 2024-09-10T08:45:26.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2eddb78640 1 -- start start 2024-09-10T08:45:26.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2eddb78640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ed8101de0 0x7f2ed8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2eddb78640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ed8102790 0x7f2ed81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2eddb78640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ed8106f10 0x7f2ed819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2eddb78640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed8111e40 con 0x7f2ed8102790 2024-09-10T08:45:26.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2eddb78640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed8111cc0 con 0x7f2ed8101de0 2024-09-10T08:45:26.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2eddb78640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed8111fc0 con 0x7f2ed8106f10 2024-09-10T08:45:26.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2ed6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ed8102790 0x7f2ed81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2ed6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ed8102790 0x7f2ed81989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45910/0 (socket says 172.21.15.172:45910) 2024-09-10T08:45:26.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2ed6ffd640 1 -- 172.21.15.172:0/2893512723 learned_addr learned my addr 172.21.15.172:0/2893512723 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:26.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2ed77fe640 1 --2- 172.21.15.172:0/2893512723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ed8101de0 0x7f2ed8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2ed7fff640 1 --2- 172.21.15.172:0/2893512723 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ed8106f10 0x7f2ed819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2ed6ffd640 1 -- 172.21.15.172:0/2893512723 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ed8106f10 msgr2=0x7f2ed819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2ed6ffd640 1 --2- 172.21.15.172:0/2893512723 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ed8106f10 0x7f2ed819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2ed6ffd640 1 -- 172.21.15.172:0/2893512723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ed8101de0 msgr2=0x7f2ed8198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2ed6ffd640 1 --2- 172.21.15.172:0/2893512723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ed8101de0 0x7f2ed8198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2ed6ffd640 1 -- 172.21.15.172:0/2893512723 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed819d410 con 0x7f2ed8102790 2024-09-10T08:45:26.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.612+0000 7f2ed7fff640 1 --2- 172.21.15.172:0/2893512723 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ed8106f10 0x7f2ed819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:26.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.613+0000 7f2ed6ffd640 1 --2- 172.21.15.172:0/2893512723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ed8102790 0x7f2ed81989a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc005940 tx=0x7f2ecc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:26.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.613+0000 7f2ed4ff9640 1 -- 172.21.15.172:0/2893512723 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc03f030 con 0x7f2ed8102790 2024-09-10T08:45:26.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.613+0000 7f2ed4ff9640 1 -- 172.21.15.172:0/2893512723 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ecc003aa0 con 0x7f2ed8102790 2024-09-10T08:45:26.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.613+0000 7f2eddb78640 1 -- 172.21.15.172:0/2893512723 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed819d6a0 con 0x7f2ed8102790 2024-09-10T08:45:26.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.613+0000 7f2ed4ff9640 1 -- 172.21.15.172:0/2893512723 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc035070 con 0x7f2ed8102790 2024-09-10T08:45:26.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.613+0000 7f2eddb78640 1 -- 172.21.15.172:0/2893512723 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed81a4fe0 con 0x7f2ed8102790 2024-09-10T08:45:26.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.614+0000 7f2eddb78640 1 -- 172.21.15.172:0/2893512723 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2ed8062900 con 0x7f2ed8102790 2024-09-10T08:45:26.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.615+0000 7f2ed4ff9640 1 -- 172.21.15.172:0/2893512723 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2ecc042030 con 0x7f2ed8102790 2024-09-10T08:45:26.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.616+0000 7f2ed4ff9640 1 --2- 172.21.15.172:0/2893512723 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2ec00a2c90 0x7f2ec00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.616+0000 7f2ed77fe640 1 --2- 172.21.15.172:0/2893512723 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2ec00a2c90 0x7f2ec00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.616+0000 7f2ed4ff9640 1 -- 172.21.15.172:0/2893512723 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f2ecc0f02e0 con 0x7f2ed8102790 2024-09-10T08:45:26.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.616+0000 7f2ed4ff9640 1 --2- 172.21.15.172:0/2893512723 >> [v2:172.21.15.174:6824/1238812525,v1:172.21.15.174:6825/1238812525] conn(0x7f2ec00adde0 0x7f2ec00b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.616+0000 7f2ed4ff9640 1 -- 172.21.15.172:0/2893512723 --> [v2:172.21.15.174:6824/1238812525,v1:172.21.15.174:6825/1238812525] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ec00b0950 con 0x7f2ec00adde0 2024-09-10T08:45:26.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.616+0000 7f2ed4ff9640 1 -- 172.21.15.172:0/2893512723 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f2ecc0f07c0 con 0x7f2ed8102790 2024-09-10T08:45:26.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.616+0000 7f2ed77fe640 1 --2- 172.21.15.172:0/2893512723 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2ec00a2c90 0x7f2ec00a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800bbe0 tx=0x7f2ec800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:26.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.616+0000 7f2ed7fff640 1 --2- 172.21.15.172:0/2893512723 >> [v2:172.21.15.174:6824/1238812525,v1:172.21.15.174:6825/1238812525] conn(0x7f2ec00adde0 0x7f2ec00b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.617+0000 7f2ed7fff640 1 --2- 172.21.15.172:0/2893512723 >> [v2:172.21.15.174:6824/1238812525,v1:172.21.15.174:6825/1238812525] conn(0x7f2ec00adde0 0x7f2ec00b0280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:26.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.618+0000 7f2ed4ff9640 1 -- 172.21.15.172:0/2893512723 <== osd.10 v2:172.21.15.174:6824/1238812525 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2ec00b0950 con 0x7f2ec00adde0 2024-09-10T08:45:26.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.632+0000 7f2eddb78640 1 -- 172.21.15.172:0/2893512723 --> [v2:172.21.15.174:6824/1238812525,v1:172.21.15.174:6825/1238812525] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2ed8062d00 con 0x7f2ec00adde0 2024-09-10T08:45:26.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.633+0000 7f2ed4ff9640 1 -- 172.21.15.172:0/2893512723 <== osd.10 v2:172.21.15.174:6824/1238812525 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2ed8062d00 con 0x7f2ec00adde0 2024-09-10T08:45:26.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.633+0000 7f2eddb78640 1 -- 172.21.15.172:0/2893512723 >> [v2:172.21.15.174:6824/1238812525,v1:172.21.15.174:6825/1238812525] conn(0x7f2ec00adde0 msgr2=0x7f2ec00b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.633+0000 7f2eddb78640 1 --2- 172.21.15.172:0/2893512723 >> [v2:172.21.15.174:6824/1238812525,v1:172.21.15.174:6825/1238812525] conn(0x7f2ec00adde0 0x7f2ec00b0280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.634+0000 7f2eddb78640 1 -- 172.21.15.172:0/2893512723 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2ec00a2c90 msgr2=0x7f2ec00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.634+0000 7f2eddb78640 1 --2- 172.21.15.172:0/2893512723 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2ec00a2c90 0x7f2ec00a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800bbe0 tx=0x7f2ec800b390 comp rx=0 tx=0).stop 2024-09-10T08:45:26.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.634+0000 7f2eddb78640 1 -- 172.21.15.172:0/2893512723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ed8102790 msgr2=0x7f2ed81989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.634+0000 7f2eddb78640 1 --2- 172.21.15.172:0/2893512723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ed8102790 0x7f2ed81989a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc005940 tx=0x7f2ecc004270 comp rx=0 tx=0).stop 2024-09-10T08:45:26.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.634+0000 7f2ed7fff640 1 -- 172.21.15.172:0/2893512723 reap_dead start 2024-09-10T08:45:26.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.634+0000 7f2eddb78640 1 -- 172.21.15.172:0/2893512723 shutdown_connections 2024-09-10T08:45:26.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.634+0000 7f2eddb78640 1 -- 172.21.15.172:0/2893512723 >> 172.21.15.172:0/2893512723 conn(0x7f2ed80fd8c0 msgr2=0x7f2ed80fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:26.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.634+0000 7f2eddb78640 1 -- 172.21.15.172:0/2893512723 shutdown_connections 2024-09-10T08:45:26.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.634+0000 7f2eddb78640 1 -- 172.21.15.172:0/2893512723 wait complete. 2024-09-10T08:45:26.656 INFO:teuthology.orchestra.run.smithi172.stdout:163208757306 2024-09-10T08:45:26.656 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T08:45:26.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:26 smithi174 ceph-mon[34068]: pgmap v324: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 639 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:26.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.899+0000 7fbcf45b7640 1 -- 172.21.15.172:0/926327908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcec103780 msgr2=0x7fbcec107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.899+0000 7fbcf45b7640 1 --2- 172.21.15.172:0/926327908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcec103780 0x7fbcec107bc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc009930 tx=0x7fbcdc02a420 comp rx=0 tx=0).stop 2024-09-10T08:45:26.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.899+0000 7fbcf45b7640 1 -- 172.21.15.172:0/926327908 shutdown_connections 2024-09-10T08:45:26.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.899+0000 7fbcf45b7640 1 --2- 172.21.15.172:0/926327908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcec108300 0x7fbcec10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.899+0000 7fbcf45b7640 1 --2- 172.21.15.172:0/926327908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcec103780 0x7fbcec107bc0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.899+0000 7fbcf45b7640 1 --2- 172.21.15.172:0/926327908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbcec102dd0 0x7fbcec1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.899+0000 7fbcf45b7640 1 -- 172.21.15.172:0/926327908 >> 172.21.15.172:0/926327908 conn(0x7fbcec0fc7d0 msgr2=0x7fbcec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:26.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.899+0000 7fbcf45b7640 1 -- 172.21.15.172:0/926327908 shutdown_connections 2024-09-10T08:45:26.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.899+0000 7fbcf45b7640 1 -- 172.21.15.172:0/926327908 wait complete. 2024-09-10T08:45:26.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.899+0000 7fbcf45b7640 1 Processor -- start 2024-09-10T08:45:26.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.899+0000 7fbcf45b7640 1 -- start start 2024-09-10T08:45:26.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf45b7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcec102dd0 0x7fbcec19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf45b7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcec103780 0x7fbcec19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf45b7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbcec108300 0x7fbcec19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf45b7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec111f20 con 0x7fbcec103780 2024-09-10T08:45:26.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf45b7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec111da0 con 0x7fbcec102dd0 2024-09-10T08:45:26.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf45b7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1120a0 con 0x7fbcec108300 2024-09-10T08:45:26.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf1b52640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcec103780 0x7fbcec19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf1b52640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcec103780 0x7fbcec19ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45922/0 (socket says 172.21.15.172:45922) 2024-09-10T08:45:26.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf1b52640 1 -- 172.21.15.172:0/4018758922 learned_addr learned my addr 172.21.15.172:0/4018758922 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:26.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf2353640 1 --2- 172.21.15.172:0/4018758922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcec102dd0 0x7fbcec19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf2b54640 1 --2- 172.21.15.172:0/4018758922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbcec108300 0x7fbcec19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf1b52640 1 -- 172.21.15.172:0/4018758922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbcec108300 msgr2=0x7fbcec19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf1b52640 1 --2- 172.21.15.172:0/4018758922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbcec108300 0x7fbcec19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf1b52640 1 -- 172.21.15.172:0/4018758922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcec102dd0 msgr2=0x7fbcec19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf1b52640 1 --2- 172.21.15.172:0/4018758922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcec102dd0 0x7fbcec19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf1b52640 1 -- 172.21.15.172:0/4018758922 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcec19f6d0 con 0x7fbcec103780 2024-09-10T08:45:26.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf1b52640 1 --2- 172.21.15.172:0/4018758922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcec103780 0x7fbcec19ac60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc031440 tx=0x7fbcdc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:26.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbce37fe640 1 -- 172.21.15.172:0/4018758922 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc03f030 con 0x7fbcec103780 2024-09-10T08:45:26.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf45b7640 1 -- 172.21.15.172:0/4018758922 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcec19f960 con 0x7fbcec103780 2024-09-10T08:45:26.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbce37fe640 1 -- 172.21.15.172:0/4018758922 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcdc003aa0 con 0x7fbcec103780 2024-09-10T08:45:26.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbce37fe640 1 -- 172.21.15.172:0/4018758922 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc035070 con 0x7fbcec103780 2024-09-10T08:45:26.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.900+0000 7fbcf45b7640 1 -- 172.21.15.172:0/4018758922 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcec1a72a0 con 0x7fbcec103780 2024-09-10T08:45:26.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.901+0000 7fbcf45b7640 1 -- 172.21.15.172:0/4018758922 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbcec102350 con 0x7fbcec103780 2024-09-10T08:45:26.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.903+0000 7fbce37fe640 1 -- 172.21.15.172:0/4018758922 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fbcdc042030 con 0x7fbcec103780 2024-09-10T08:45:26.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.903+0000 7fbce37fe640 1 --2- 172.21.15.172:0/4018758922 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbcd80a2c10 0x7fbcd80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.903+0000 7fbcf2353640 1 --2- 172.21.15.172:0/4018758922 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbcd80a2c10 0x7fbcd80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.903+0000 7fbce37fe640 1 -- 172.21.15.172:0/4018758922 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fbcdc0f0120 con 0x7fbcec103780 2024-09-10T08:45:26.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.903+0000 7fbce37fe640 1 --2- 172.21.15.172:0/4018758922 >> [v2:172.21.15.174:6800/3279661439,v1:172.21.15.174:6801/3279661439] conn(0x7fbcd80add60 0x7fbcd80b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:26.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.903+0000 7fbce37fe640 1 -- 172.21.15.172:0/4018758922 --> [v2:172.21.15.174:6800/3279661439,v1:172.21.15.174:6801/3279661439] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbcdc00b0b0 con 0x7fbcd80add60 2024-09-10T08:45:26.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.903+0000 7fbce37fe640 1 -- 172.21.15.172:0/4018758922 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fbcdc0f0570 con 0x7fbcec103780 2024-09-10T08:45:26.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.904+0000 7fbcf2353640 1 --2- 172.21.15.172:0/4018758922 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbcd80a2c10 0x7fbcd80a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbce8005fd0 tx=0x7fbce8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:26.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.904+0000 7fbcf2b54640 1 --2- 172.21.15.172:0/4018758922 >> [v2:172.21.15.174:6800/3279661439,v1:172.21.15.174:6801/3279661439] conn(0x7fbcd80add60 0x7fbcd80b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:26.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.904+0000 7fbcf2b54640 1 --2- 172.21.15.172:0/4018758922 >> [v2:172.21.15.174:6800/3279661439,v1:172.21.15.174:6801/3279661439] conn(0x7fbcd80add60 0x7fbcd80b0200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:26.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.905+0000 7fbce37fe640 1 -- 172.21.15.172:0/4018758922 <== osd.1 v2:172.21.15.174:6800/3279661439 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbcdc00b0b0 con 0x7fbcd80add60 2024-09-10T08:45:26.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.918+0000 7fbcf45b7640 1 -- 172.21.15.172:0/4018758922 --> [v2:172.21.15.174:6800/3279661439,v1:172.21.15.174:6801/3279661439] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbcec06a7b0 con 0x7fbcd80add60 2024-09-10T08:45:26.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.919+0000 7fbce37fe640 1 -- 172.21.15.172:0/4018758922 <== osd.1 v2:172.21.15.174:6800/3279661439 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbcec06a7b0 con 0x7fbcd80add60 2024-09-10T08:45:26.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.919+0000 7fbcf45b7640 1 -- 172.21.15.172:0/4018758922 >> [v2:172.21.15.174:6800/3279661439,v1:172.21.15.174:6801/3279661439] conn(0x7fbcd80add60 msgr2=0x7fbcd80b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.919+0000 7fbcf45b7640 1 --2- 172.21.15.172:0/4018758922 >> [v2:172.21.15.174:6800/3279661439,v1:172.21.15.174:6801/3279661439] conn(0x7fbcd80add60 0x7fbcd80b0200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:26.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.919+0000 7fbcf45b7640 1 -- 172.21.15.172:0/4018758922 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbcd80a2c10 msgr2=0x7fbcd80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.919+0000 7fbcf45b7640 1 --2- 172.21.15.172:0/4018758922 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbcd80a2c10 0x7fbcd80a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbce8005fd0 tx=0x7fbce8005850 comp rx=0 tx=0).stop 2024-09-10T08:45:26.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.919+0000 7fbcf45b7640 1 -- 172.21.15.172:0/4018758922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcec103780 msgr2=0x7fbcec19ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:26.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.919+0000 7fbcf45b7640 1 --2- 172.21.15.172:0/4018758922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcec103780 0x7fbcec19ac60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc031440 tx=0x7fbcdc004270 comp rx=0 tx=0).stop 2024-09-10T08:45:26.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.919+0000 7fbcf2b54640 1 -- 172.21.15.172:0/4018758922 reap_dead start 2024-09-10T08:45:26.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.919+0000 7fbcf45b7640 1 -- 172.21.15.172:0/4018758922 shutdown_connections 2024-09-10T08:45:26.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.919+0000 7fbcf45b7640 1 -- 172.21.15.172:0/4018758922 >> 172.21.15.172:0/4018758922 conn(0x7fbcec0fc7d0 msgr2=0x7fbcec100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:26.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.920+0000 7fbcf45b7640 1 -- 172.21.15.172:0/4018758922 shutdown_connections 2024-09-10T08:45:26.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:26.920+0000 7fbcf45b7640 1 -- 172.21.15.172:0/4018758922 wait complete. 2024-09-10T08:45:27.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.590+0000 7f84c3a63640 1 -- 172.21.15.172:0/2179759286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84bc103720 msgr2=0x7f84bc107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:27.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.590+0000 7f84c3a63640 1 --2- 172.21.15.172:0/2179759286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84bc103720 0x7f84bc107b60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009930 tx=0x7f84ac02a420 comp rx=0 tx=0).stop 2024-09-10T08:45:27.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.590+0000 7f84c3a63640 1 -- 172.21.15.172:0/2179759286 shutdown_connections 2024-09-10T08:45:27.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.590+0000 7f84c3a63640 1 --2- 172.21.15.172:0/2179759286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84bc1082a0 0x7f84bc10ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:27.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.590+0000 7f84c3a63640 1 --2- 172.21.15.172:0/2179759286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84bc103720 0x7f84bc107b60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:27.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.590+0000 7f84c3a63640 1 --2- 172.21.15.172:0/2179759286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84bc102d70 0x7f84bc103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:27.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.590+0000 7f84c3a63640 1 -- 172.21.15.172:0/2179759286 >> 172.21.15.172:0/2179759286 conn(0x7f84bc0fc7d0 msgr2=0x7f84bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:27.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.590+0000 7f84c3a63640 1 -- 172.21.15.172:0/2179759286 shutdown_connections 2024-09-10T08:45:27.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.590+0000 7f84c3a63640 1 -- 172.21.15.172:0/2179759286 wait complete. 2024-09-10T08:45:27.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.591+0000 7f84c3a63640 1 Processor -- start 2024-09-10T08:45:27.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.591+0000 7f84c3a63640 1 -- start start 2024-09-10T08:45:27.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.591+0000 7f84c3a63640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84bc102d70 0x7f84bc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:27.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.591+0000 7f84c3a63640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84bc103720 0x7f84bc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:27.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.591+0000 7f84c3a63640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84bc1082a0 0x7f84bc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:27.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.591+0000 7f84c3a63640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc111f10 con 0x7f84bc1082a0 2024-09-10T08:45:27.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.591+0000 7f84c3a63640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc111d90 con 0x7f84bc103720 2024-09-10T08:45:27.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.591+0000 7f84c3a63640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc112090 con 0x7f84bc102d70 2024-09-10T08:45:27.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.591+0000 7f84c2000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84bc1082a0 0x7f84bc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:27.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.591+0000 7f84c2000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84bc1082a0 0x7f84bc19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42388/0 (socket says 172.21.15.172:42388) 2024-09-10T08:45:27.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.591+0000 7f84c2000640 1 -- 172.21.15.172:0/202797485 learned_addr learned my addr 172.21.15.172:0/202797485 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:27.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.591+0000 7f84c0ffe640 1 --2- 172.21.15.172:0/202797485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84bc103720 0x7f84bc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:27.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.592+0000 7f84c17ff640 1 --2- 172.21.15.172:0/202797485 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84bc102d70 0x7f84bc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:27.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.592+0000 7f84c2000640 1 -- 172.21.15.172:0/202797485 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84bc102d70 msgr2=0x7f84bc198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:27.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.592+0000 7f84c2000640 1 --2- 172.21.15.172:0/202797485 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84bc102d70 0x7f84bc198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:27.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.592+0000 7f84c2000640 1 -- 172.21.15.172:0/202797485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84bc103720 msgr2=0x7f84bc1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:27.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.592+0000 7f84c2000640 1 --2- 172.21.15.172:0/202797485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84bc103720 0x7f84bc1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:27.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.592+0000 7f84c2000640 1 -- 172.21.15.172:0/202797485 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84bc19d440 con 0x7f84bc1082a0 2024-09-10T08:45:27.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.592+0000 7f84c2000640 1 --2- 172.21.15.172:0/202797485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84bc1082a0 0x7f84bc19cd60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f84b400cce0 tx=0x7f84b400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:27.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.592+0000 7f84b27fc640 1 -- 172.21.15.172:0/202797485 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b400f450 con 0x7f84bc1082a0 2024-09-10T08:45:27.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.592+0000 7f84b27fc640 1 -- 172.21.15.172:0/202797485 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84b400faa0 con 0x7f84bc1082a0 2024-09-10T08:45:27.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.592+0000 7f84b27fc640 1 -- 172.21.15.172:0/202797485 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b4018b70 con 0x7f84bc1082a0 2024-09-10T08:45:27.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.592+0000 7f84c3a63640 1 -- 172.21.15.172:0/202797485 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84bc19d730 con 0x7f84bc1082a0 2024-09-10T08:45:27.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.592+0000 7f84c3a63640 1 -- 172.21.15.172:0/202797485 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84bc1a5010 con 0x7f84bc1082a0 2024-09-10T08:45:27.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.593+0000 7f84c3a63640 1 -- 172.21.15.172:0/202797485 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f84bc062980 con 0x7f84bc1082a0 2024-09-10T08:45:27.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.594+0000 7f84b27fc640 1 -- 172.21.15.172:0/202797485 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f84b4018d10 con 0x7f84bc1082a0 2024-09-10T08:45:27.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.595+0000 7f84b27fc640 1 --2- 172.21.15.172:0/202797485 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84a80a2c10 0x7f84a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:27.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.595+0000 7f84b27fc640 1 -- 172.21.15.172:0/202797485 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f84b40cfaf0 con 0x7f84bc1082a0 2024-09-10T08:45:27.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.595+0000 7f84c17ff640 1 --2- 172.21.15.172:0/202797485 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84a80a2c10 0x7f84a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:27.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.595+0000 7f84b27fc640 1 --2- 172.21.15.172:0/202797485 >> [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480] conn(0x7f84a80add60 0x7f84a80b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:27.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.595+0000 7f84b27fc640 1 -- 172.21.15.172:0/202797485 --> [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f84b4015e70 con 0x7f84a80add60 2024-09-10T08:45:27.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.595+0000 7f84b27fc640 1 -- 172.21.15.172:0/202797485 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f84b40cff40 con 0x7f84bc1082a0 2024-09-10T08:45:27.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.595+0000 7f84c0ffe640 1 --2- 172.21.15.172:0/202797485 >> [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480] conn(0x7f84a80add60 0x7f84a80b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:27.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.595+0000 7f84c17ff640 1 --2- 172.21.15.172:0/202797485 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84a80a2c10 0x7f84a80a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f84b80058c0 tx=0x7f84b800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:27.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.596+0000 7f84c0ffe640 1 --2- 172.21.15.172:0/202797485 >> [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480] conn(0x7f84a80add60 0x7f84a80b0200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:27.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.596+0000 7f84b27fc640 1 -- 172.21.15.172:0/202797485 <== osd.3 v2:172.21.15.172:6810/869546480 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f84b4015e70 con 0x7f84a80add60 2024-09-10T08:45:27.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.610+0000 7f84c3a63640 1 -- 172.21.15.172:0/202797485 --> [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f84bc062d80 con 0x7f84a80add60 2024-09-10T08:45:27.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.610+0000 7f84b27fc640 1 -- 172.21.15.172:0/202797485 <== osd.3 v2:172.21.15.172:6810/869546480 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f84bc062d80 con 0x7f84a80add60 2024-09-10T08:45:27.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.611+0000 7f84c3a63640 1 -- 172.21.15.172:0/202797485 >> [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480] conn(0x7f84a80add60 msgr2=0x7f84a80b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:27.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.611+0000 7f84c3a63640 1 --2- 172.21.15.172:0/202797485 >> [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480] conn(0x7f84a80add60 0x7f84a80b0200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:27.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.611+0000 7f84c3a63640 1 -- 172.21.15.172:0/202797485 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84a80a2c10 msgr2=0x7f84a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:27.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.611+0000 7f84c3a63640 1 --2- 172.21.15.172:0/202797485 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84a80a2c10 0x7f84a80a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f84b80058c0 tx=0x7f84b800a250 comp rx=0 tx=0).stop 2024-09-10T08:45:27.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.611+0000 7f84c3a63640 1 -- 172.21.15.172:0/202797485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84bc1082a0 msgr2=0x7f84bc19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:27.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.611+0000 7f84c3a63640 1 --2- 172.21.15.172:0/202797485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84bc1082a0 0x7f84bc19cd60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f84b400cce0 tx=0x7f84b400a620 comp rx=0 tx=0).stop 2024-09-10T08:45:27.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.611+0000 7f84c2000640 1 -- 172.21.15.172:0/202797485 reap_dead start 2024-09-10T08:45:27.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.611+0000 7f84c3a63640 1 -- 172.21.15.172:0/202797485 shutdown_connections 2024-09-10T08:45:27.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.611+0000 7f84c3a63640 1 -- 172.21.15.172:0/202797485 >> 172.21.15.172:0/202797485 conn(0x7f84bc0fc7d0 msgr2=0x7f84bc105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:27.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.611+0000 7f84c3a63640 1 -- 172.21.15.172:0/202797485 shutdown_connections 2024-09-10T08:45:27.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:21.611+0000 7f84c3a63640 1 -- 172.21.15.172:0/202797485 wait complete. 2024-09-10T08:45:27.869 INFO:teuthology.orchestra.run.smithi172.stdout:107374182467 2024-09-10T08:45:27.869 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T08:45:28.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.024+0000 7f7aa3f00640 1 -- 172.21.15.172:0/3825965934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a9c102570 msgr2=0x7f7a9c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:28.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.024+0000 7f7aa3f00640 1 --2- 172.21.15.172:0/3825965934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a9c102570 0x7f7a9c1029b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009900 tx=0x7f7a9002a480 comp rx=0 tx=0).stop 2024-09-10T08:45:28.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.024+0000 7f7aa3f00640 1 -- 172.21.15.172:0/3825965934 shutdown_connections 2024-09-10T08:45:28.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.024+0000 7f7aa3f00640 1 --2- 172.21.15.172:0/3825965934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a9c102ef0 0x7f7a9c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:28.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.024+0000 7f7aa3f00640 1 --2- 172.21.15.172:0/3825965934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a9c102570 0x7f7a9c1029b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:28.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.024+0000 7f7aa3f00640 1 --2- 172.21.15.172:0/3825965934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a9c100e20 0x7f7a9c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:28.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.024+0000 7f7aa3f00640 1 -- 172.21.15.172:0/3825965934 >> 172.21.15.172:0/3825965934 conn(0x7f7a9c0fc7d0 msgr2=0x7f7a9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:28.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.024+0000 7f7aa3f00640 1 -- 172.21.15.172:0/3825965934 shutdown_connections 2024-09-10T08:45:28.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.024+0000 7f7aa3f00640 1 -- 172.21.15.172:0/3825965934 wait complete. 2024-09-10T08:45:28.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.025+0000 7f7aa3f00640 1 Processor -- start 2024-09-10T08:45:28.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.025+0000 7f7aa3f00640 1 -- start start 2024-09-10T08:45:28.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.025+0000 7f7aa3f00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a9c100e20 0x7f7a9c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:28.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.025+0000 7f7aa3f00640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a9c102570 0x7f7a9c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:28.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.025+0000 7f7aa3f00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a9c102ef0 0x7f7a9c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:28.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.025+0000 7f7aa3f00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c110e30 con 0x7f7a9c102ef0 2024-09-10T08:45:28.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.025+0000 7f7aa3f00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c110cb0 con 0x7f7a9c100e20 2024-09-10T08:45:28.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.025+0000 7f7aa3f00640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c110fb0 con 0x7f7a9c102570 2024-09-10T08:45:28.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.025+0000 7f7aa249d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a9c102ef0 0x7f7a9c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:28.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.025+0000 7f7aa249d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a9c102ef0 0x7f7a9c19cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45938/0 (socket says 172.21.15.172:45938) 2024-09-10T08:45:28.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.025+0000 7f7aa249d640 1 -- 172.21.15.172:0/2528889806 learned_addr learned my addr 172.21.15.172:0/2528889806 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:28.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.025+0000 7f7aa149b640 1 --2- 172.21.15.172:0/2528889806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a9c102570 0x7f7a9c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:28.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.025+0000 7f7aa1c9c640 1 --2- 172.21.15.172:0/2528889806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a9c100e20 0x7f7a9c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:28.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.025+0000 7f7aa249d640 1 -- 172.21.15.172:0/2528889806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a9c102570 msgr2=0x7f7a9c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:28.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.025+0000 7f7aa249d640 1 --2- 172.21.15.172:0/2528889806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a9c102570 0x7f7a9c198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:28.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.025+0000 7f7aa249d640 1 -- 172.21.15.172:0/2528889806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a9c100e20 msgr2=0x7f7a9c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:28.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.026+0000 7f7aa249d640 1 --2- 172.21.15.172:0/2528889806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a9c100e20 0x7f7a9c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:28.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.026+0000 7f7aa249d640 1 -- 172.21.15.172:0/2528889806 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9c19d470 con 0x7f7a9c102ef0 2024-09-10T08:45:28.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.026+0000 7f7aa249d640 1 --2- 172.21.15.172:0/2528889806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a9c102ef0 0x7f7a9c19cd90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7a9400eb10 tx=0x7f7a9400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:28.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.026+0000 7f7a8effd640 1 -- 172.21.15.172:0/2528889806 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a94004b20 con 0x7f7a9c102ef0 2024-09-10T08:45:28.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.026+0000 7f7a8effd640 1 -- 172.21.15.172:0/2528889806 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a94004cc0 con 0x7f7a9c102ef0 2024-09-10T08:45:28.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.026+0000 7f7aa3f00640 1 -- 172.21.15.172:0/2528889806 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a9c19d760 con 0x7f7a9c102ef0 2024-09-10T08:45:28.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.026+0000 7f7a8effd640 1 -- 172.21.15.172:0/2528889806 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a94010690 con 0x7f7a9c102ef0 2024-09-10T08:45:28.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.026+0000 7f7aa3f00640 1 -- 172.21.15.172:0/2528889806 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a9c0fcbb0 con 0x7f7a9c102ef0 2024-09-10T08:45:28.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.027+0000 7f7aa3f00640 1 -- 172.21.15.172:0/2528889806 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a9c062980 con 0x7f7a9c102ef0 2024-09-10T08:45:28.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.028+0000 7f7a8effd640 1 -- 172.21.15.172:0/2528889806 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7a94005000 con 0x7f7a9c102ef0 2024-09-10T08:45:28.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.029+0000 7f7a8effd640 1 --2- 172.21.15.172:0/2528889806 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7a880a2c10 0x7f7a880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:28.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.029+0000 7f7aa1c9c640 1 --2- 172.21.15.172:0/2528889806 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7a880a2c10 0x7f7a880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:28.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.029+0000 7f7a8effd640 1 -- 172.21.15.172:0/2528889806 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f7a940cfe50 con 0x7f7a9c102ef0 2024-09-10T08:45:28.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.029+0000 7f7a8effd640 1 --2- 172.21.15.172:0/2528889806 >> [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009] conn(0x7f7a880add60 0x7f7a880b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:28.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.029+0000 7f7a8effd640 1 -- 172.21.15.172:0/2528889806 --> [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a880b08d0 con 0x7f7a880add60 2024-09-10T08:45:28.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.029+0000 7f7a8effd640 1 -- 172.21.15.172:0/2528889806 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f7a940d02a0 con 0x7f7a9c102ef0 2024-09-10T08:45:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.029+0000 7f7aa149b640 1 --2- 172.21.15.172:0/2528889806 >> [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009] conn(0x7f7a880add60 0x7f7a880b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.030+0000 7f7aa1c9c640 1 --2- 172.21.15.172:0/2528889806 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7a880a2c10 0x7f7a880a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a980058c0 tx=0x7f7a9800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.030+0000 7f7aa149b640 1 --2- 172.21.15.172:0/2528889806 >> [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009] conn(0x7f7a880add60 0x7f7a880b0200 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.031+0000 7f7a8effd640 1 -- 172.21.15.172:0/2528889806 <== osd.6 v2:172.21.15.172:6818/3210762009 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a880b08d0 con 0x7f7a880add60 2024-09-10T08:45:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.044+0000 7f7aa3f00640 1 -- 172.21.15.172:0/2528889806 --> [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7a9c062e00 con 0x7f7a880add60 2024-09-10T08:45:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.044+0000 7f7a8effd640 1 -- 172.21.15.172:0/2528889806 <== osd.6 v2:172.21.15.172:6818/3210762009 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7a9c062e00 con 0x7f7a880add60 2024-09-10T08:45:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.045+0000 7f7aa3f00640 1 -- 172.21.15.172:0/2528889806 >> [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009] conn(0x7f7a880add60 msgr2=0x7f7a880b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.045+0000 7f7aa3f00640 1 --2- 172.21.15.172:0/2528889806 >> [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009] conn(0x7f7a880add60 0x7f7a880b0200 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.045+0000 7f7aa3f00640 1 -- 172.21.15.172:0/2528889806 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7a880a2c10 msgr2=0x7f7a880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.045+0000 7f7aa3f00640 1 --2- 172.21.15.172:0/2528889806 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7a880a2c10 0x7f7a880a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a980058c0 tx=0x7f7a9800a250 comp rx=0 tx=0).stop 2024-09-10T08:45:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.045+0000 7f7aa3f00640 1 -- 172.21.15.172:0/2528889806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a9c102ef0 msgr2=0x7f7a9c19cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.045+0000 7f7aa3f00640 1 --2- 172.21.15.172:0/2528889806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a9c102ef0 0x7f7a9c19cd90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7a9400eb10 tx=0x7f7a9400c2d0 comp rx=0 tx=0).stop 2024-09-10T08:45:28.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.045+0000 7f7aa249d640 1 -- 172.21.15.172:0/2528889806 reap_dead start 2024-09-10T08:45:28.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.045+0000 7f7aa3f00640 1 -- 172.21.15.172:0/2528889806 shutdown_connections 2024-09-10T08:45:28.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.045+0000 7f7aa3f00640 1 -- 172.21.15.172:0/2528889806 >> 172.21.15.172:0/2528889806 conn(0x7f7a9c0fc7d0 msgr2=0x7f7a9c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:28.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.045+0000 7f7aa3f00640 1 -- 172.21.15.172:0/2528889806 shutdown_connections 2024-09-10T08:45:28.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:28.045+0000 7f7aa3f00640 1 -- 172.21.15.172:0/2528889806 wait complete. 2024-09-10T08:45:28.544 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:28 smithi174 ceph-mon[34068]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 517 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:28.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:28 smithi191 ceph-mon[35314]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 517 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:28 smithi172 ceph-mon[30854]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 517 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:28.938 INFO:teuthology.orchestra.run.smithi172.stdout:55834574927 2024-09-10T08:45:28.939 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T08:45:29.342 INFO:teuthology.orchestra.run.smithi172.stdout:296352743458 2024-09-10T08:45:29.342 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-10T08:45:29.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.665+0000 7ff5260a0640 1 -- 172.21.15.172:0/2667645551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff520074590 msgr2=0x7ff520074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:29.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.665+0000 7ff5260a0640 1 --2- 172.21.15.172:0/2667645551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff520074590 0x7ff520074970 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff510009900 tx=0x7ff51002a480 comp rx=0 tx=0).stop 2024-09-10T08:45:29.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.665+0000 7ff5260a0640 1 -- 172.21.15.172:0/2667645551 shutdown_connections 2024-09-10T08:45:29.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.665+0000 7ff5260a0640 1 --2- 172.21.15.172:0/2667645551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff520112b90 0x7ff520114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:29.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.665+0000 7ff5260a0640 1 --2- 172.21.15.172:0/2667645551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff520074eb0 0x7ff520112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:29.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.665+0000 7ff5260a0640 1 --2- 172.21.15.172:0/2667645551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff520074590 0x7ff520074970 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:29.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.665+0000 7ff5260a0640 1 -- 172.21.15.172:0/2667645551 >> 172.21.15.172:0/2667645551 conn(0x7ff5200fd8c0 msgr2=0x7ff5200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:29.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.665+0000 7ff5260a0640 1 -- 172.21.15.172:0/2667645551 shutdown_connections 2024-09-10T08:45:29.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.665+0000 7ff5260a0640 1 -- 172.21.15.172:0/2667645551 wait complete. 2024-09-10T08:45:29.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.666+0000 7ff5260a0640 1 Processor -- start 2024-09-10T08:45:29.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.666+0000 7ff5260a0640 1 -- start start 2024-09-10T08:45:29.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.666+0000 7ff5260a0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff520074590 0x7ff5201a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:29.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.666+0000 7ff5260a0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff520074eb0 0x7ff5201a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:29.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.666+0000 7ff5260a0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff520112b90 0x7ff52019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:29.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.666+0000 7ff5260a0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff520118420 con 0x7ff520074eb0 2024-09-10T08:45:29.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.666+0000 7ff5260a0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201182a0 con 0x7ff520074590 2024-09-10T08:45:29.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.666+0000 7ff5260a0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201185a0 con 0x7ff520112b90 2024-09-10T08:45:29.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.666+0000 7ff51effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff520074eb0 0x7ff5201a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:29.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.666+0000 7ff51effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff520074eb0 0x7ff5201a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45954/0 (socket says 172.21.15.172:45954) 2024-09-10T08:45:29.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.666+0000 7ff51effd640 1 -- 172.21.15.172:0/4291183599 learned_addr learned my addr 172.21.15.172:0/4291183599 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:29.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.666+0000 7ff51f7fe640 1 --2- 172.21.15.172:0/4291183599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff520074590 0x7ff5201a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:29.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.666+0000 7ff51ffff640 1 --2- 172.21.15.172:0/4291183599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff520112b90 0x7ff52019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:29.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.667+0000 7ff51effd640 1 -- 172.21.15.172:0/4291183599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff520112b90 msgr2=0x7ff52019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:29.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.667+0000 7ff51effd640 1 --2- 172.21.15.172:0/4291183599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff520112b90 0x7ff52019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:29.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.667+0000 7ff51effd640 1 -- 172.21.15.172:0/4291183599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff520074590 msgr2=0x7ff5201a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:29.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.667+0000 7ff51effd640 1 --2- 172.21.15.172:0/4291183599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff520074590 0x7ff5201a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:29.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.667+0000 7ff51effd640 1 -- 172.21.15.172:0/4291183599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52019efc0 con 0x7ff520074eb0 2024-09-10T08:45:29.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.667+0000 7ff51ffff640 1 --2- 172.21.15.172:0/4291183599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff520112b90 0x7ff52019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:29.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.667+0000 7ff51effd640 1 --2- 172.21.15.172:0/4291183599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff520074eb0 0x7ff5201a4c10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff51400d950 tx=0x7ff51400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:29.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.667+0000 7ff51cff9640 1 -- 172.21.15.172:0/4291183599 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514017070 con 0x7ff520074eb0 2024-09-10T08:45:29.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.667+0000 7ff5260a0640 1 -- 172.21.15.172:0/4291183599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff52019f2b0 con 0x7ff520074eb0 2024-09-10T08:45:29.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.667+0000 7ff51cff9640 1 -- 172.21.15.172:0/4291183599 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff514009c30 con 0x7ff520074eb0 2024-09-10T08:45:29.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.667+0000 7ff51cff9640 1 -- 172.21.15.172:0/4291183599 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51400f620 con 0x7ff520074eb0 2024-09-10T08:45:29.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.667+0000 7ff5260a0640 1 -- 172.21.15.172:0/4291183599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5201ab5a0 con 0x7ff520074eb0 2024-09-10T08:45:29.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.668+0000 7ff5260a0640 1 -- 172.21.15.172:0/4291183599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff520062900 con 0x7ff520074eb0 2024-09-10T08:45:29.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.669+0000 7ff51cff9640 1 -- 172.21.15.172:0/4291183599 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff5140109a0 con 0x7ff520074eb0 2024-09-10T08:45:29.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.670+0000 7ff51cff9640 1 --2- 172.21.15.172:0/4291183599 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff5080a2c10 0x7ff5080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:29.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.670+0000 7ff51f7fe640 1 --2- 172.21.15.172:0/4291183599 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff5080a2c10 0x7ff5080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:29.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.670+0000 7ff51cff9640 1 -- 172.21.15.172:0/4291183599 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7ff5140cf4e0 con 0x7ff520074eb0 2024-09-10T08:45:29.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.670+0000 7ff51cff9640 1 --2- 172.21.15.172:0/4291183599 >> [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997] conn(0x7ff5080add60 0x7ff5080b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:29.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.670+0000 7ff51cff9640 1 -- 172.21.15.172:0/4291183599 --> [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff5080b08d0 con 0x7ff5080add60 2024-09-10T08:45:29.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.670+0000 7ff51cff9640 1 -- 172.21.15.172:0/4291183599 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff5140cf930 con 0x7ff520074eb0 2024-09-10T08:45:29.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.670+0000 7ff51f7fe640 1 --2- 172.21.15.172:0/4291183599 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff5080a2c10 0x7ff5080a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff510000c00 tx=0x7ff510031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:29.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.670+0000 7ff51ffff640 1 --2- 172.21.15.172:0/4291183599 >> [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997] conn(0x7ff5080add60 0x7ff5080b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:29.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.671+0000 7ff51ffff640 1 --2- 172.21.15.172:0/4291183599 >> [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997] conn(0x7ff5080add60 0x7ff5080b0200 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:29.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.671+0000 7ff51cff9640 1 -- 172.21.15.172:0/4291183599 <== osd.11 v2:172.21.15.191:6824/2162719997 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff5080b08d0 con 0x7ff5080add60 2024-09-10T08:45:29.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.685+0000 7ff5260a0640 1 -- 172.21.15.172:0/4291183599 --> [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff520062d80 con 0x7ff5080add60 2024-09-10T08:45:29.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.685+0000 7ff51cff9640 1 -- 172.21.15.172:0/4291183599 <== osd.11 v2:172.21.15.191:6824/2162719997 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff520062d80 con 0x7ff5080add60 2024-09-10T08:45:29.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.686+0000 7ff5260a0640 1 -- 172.21.15.172:0/4291183599 >> [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997] conn(0x7ff5080add60 msgr2=0x7ff5080b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:29.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.686+0000 7ff5260a0640 1 --2- 172.21.15.172:0/4291183599 >> [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997] conn(0x7ff5080add60 0x7ff5080b0200 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:29.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.686+0000 7ff5260a0640 1 -- 172.21.15.172:0/4291183599 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff5080a2c10 msgr2=0x7ff5080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:29.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.686+0000 7ff5260a0640 1 --2- 172.21.15.172:0/4291183599 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff5080a2c10 0x7ff5080a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff510000c00 tx=0x7ff510031400 comp rx=0 tx=0).stop 2024-09-10T08:45:29.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.686+0000 7ff5260a0640 1 -- 172.21.15.172:0/4291183599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff520074eb0 msgr2=0x7ff5201a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:29.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.686+0000 7ff5260a0640 1 --2- 172.21.15.172:0/4291183599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff520074eb0 0x7ff5201a4c10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff51400d950 tx=0x7ff51400de10 comp rx=0 tx=0).stop 2024-09-10T08:45:29.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.686+0000 7ff51ffff640 1 -- 172.21.15.172:0/4291183599 reap_dead start 2024-09-10T08:45:29.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.686+0000 7ff5260a0640 1 -- 172.21.15.172:0/4291183599 shutdown_connections 2024-09-10T08:45:29.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.686+0000 7ff5260a0640 1 -- 172.21.15.172:0/4291183599 >> 172.21.15.172:0/4291183599 conn(0x7ff5200fd8c0 msgr2=0x7ff5200ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:29.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.686+0000 7ff5260a0640 1 -- 172.21.15.172:0/4291183599 shutdown_connections 2024-09-10T08:45:29.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:29.686+0000 7ff5260a0640 1 -- 172.21.15.172:0/4291183599 wait complete. 2024-09-10T08:45:30.136 INFO:teuthology.orchestra.run.smithi172.stdout:214748364848 2024-09-10T08:45:30.136 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T08:45:30.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:30 smithi191 ceph-mon[35314]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 470 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:30.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:30 smithi172 ceph-mon[30854]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 470 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:30.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:30 smithi174 ceph-mon[34068]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 470 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:30.782 INFO:teuthology.orchestra.run.smithi172.stdout:326417514525 2024-09-10T08:45:30.782 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-10T08:45:31.408 INFO:teuthology.orchestra.run.smithi172.stdout:274877906982 2024-09-10T08:45:31.409 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-10T08:45:31.884 INFO:teuthology.orchestra.run.smithi172.stdout:193273528374 2024-09-10T08:45:31.884 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T08:45:32.504 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:32 smithi172 ceph-mon[30854]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:32.533 INFO:teuthology.orchestra.run.smithi172.stdout:244813135915 2024-09-10T08:45:32.533 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-10T08:45:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:32 smithi191 ceph-mon[35314]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:32.592 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:32.600 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:32.633 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:32.633 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:32.633 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:32.634 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:32.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:32 smithi174 ceph-mon[34068]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:33.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.460+0000 7fa6b1f1b640 1 -- 172.21.15.172:0/930324176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac100e20 msgr2=0x7fa6ac101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:33.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.460+0000 7fa6b1f1b640 1 --2- 172.21.15.172:0/930324176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac100e20 0x7fa6ac101200 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009900 tx=0x7fa69c02a480 comp rx=0 tx=0).stop 2024-09-10T08:45:33.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.460+0000 7fa6b1f1b640 1 -- 172.21.15.172:0/930324176 shutdown_connections 2024-09-10T08:45:33.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.460+0000 7fa6b1f1b640 1 --2- 172.21.15.172:0/930324176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6ac102ef0 0x7fa6ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.460+0000 7fa6b1f1b640 1 --2- 172.21.15.172:0/930324176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6ac102570 0x7fa6ac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.460+0000 7fa6b1f1b640 1 --2- 172.21.15.172:0/930324176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac100e20 0x7fa6ac101200 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.460+0000 7fa6b1f1b640 1 -- 172.21.15.172:0/930324176 >> 172.21.15.172:0/930324176 conn(0x7fa6ac0fc7d0 msgr2=0x7fa6ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:33.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.460+0000 7fa6b1f1b640 1 -- 172.21.15.172:0/930324176 shutdown_connections 2024-09-10T08:45:33.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.460+0000 7fa6b1f1b640 1 -- 172.21.15.172:0/930324176 wait complete. 2024-09-10T08:45:33.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.461+0000 7fa6b1f1b640 1 Processor -- start 2024-09-10T08:45:33.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.461+0000 7fa6b1f1b640 1 -- start start 2024-09-10T08:45:33.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.461+0000 7fa6b1f1b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6ac100e20 0x7fa6ac197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.461+0000 7fa6b1f1b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac102570 0x7fa6ac1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.461+0000 7fa6b1f1b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6ac102ef0 0x7fa6ac19bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.461+0000 7fa6b1f1b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac110e50 con 0x7fa6ac102570 2024-09-10T08:45:33.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.461+0000 7fa6b1f1b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac110cd0 con 0x7fa6ac102ef0 2024-09-10T08:45:33.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.461+0000 7fa6b1f1b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac110fd0 con 0x7fa6ac100e20 2024-09-10T08:45:33.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.461+0000 7fa6aaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac102570 0x7fa6ac1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.461+0000 7fa6aaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac102570 0x7fa6ac1978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45990/0 (socket says 172.21.15.172:45990) 2024-09-10T08:45:33.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.461+0000 7fa6aaffd640 1 -- 172.21.15.172:0/2617088645 learned_addr learned my addr 172.21.15.172:0/2617088645 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:33.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.461+0000 7fa6ab7fe640 1 --2- 172.21.15.172:0/2617088645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6ac100e20 0x7fa6ac197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.462+0000 7fa6abfff640 1 --2- 172.21.15.172:0/2617088645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6ac102ef0 0x7fa6ac19bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.462+0000 7fa6aaffd640 1 -- 172.21.15.172:0/2617088645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6ac100e20 msgr2=0x7fa6ac197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:33.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.462+0000 7fa6aaffd640 1 --2- 172.21.15.172:0/2617088645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6ac100e20 0x7fa6ac197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.462+0000 7fa6aaffd640 1 -- 172.21.15.172:0/2617088645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6ac102ef0 msgr2=0x7fa6ac19bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:33.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.462+0000 7fa6aaffd640 1 --2- 172.21.15.172:0/2617088645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6ac102ef0 0x7fa6ac19bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.462+0000 7fa6aaffd640 1 -- 172.21.15.172:0/2617088645 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac19c320 con 0x7fa6ac102570 2024-09-10T08:45:33.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.462+0000 7fa6aaffd640 1 --2- 172.21.15.172:0/2617088645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac102570 0x7fa6ac1978b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000b4a0 tx=0x7fa6a000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:33.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.462+0000 7fa6a8ff9640 1 -- 172.21.15.172:0/2617088645 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0014070 con 0x7fa6ac102570 2024-09-10T08:45:33.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.462+0000 7fa6a8ff9640 1 -- 172.21.15.172:0/2617088645 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6a0010ac0 con 0x7fa6ac102570 2024-09-10T08:45:33.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.462+0000 7fa6b1f1b640 1 -- 172.21.15.172:0/2617088645 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ac19c610 con 0x7fa6ac102570 2024-09-10T08:45:33.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.462+0000 7fa6a8ff9640 1 -- 172.21.15.172:0/2617088645 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0017c30 con 0x7fa6ac102570 2024-09-10T08:45:33.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.462+0000 7fa6b1f1b640 1 -- 172.21.15.172:0/2617088645 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ac1a3ef0 con 0x7fa6ac102570 2024-09-10T08:45:33.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.463+0000 7fa6b1f1b640 1 -- 172.21.15.172:0/2617088645 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ac101200 con 0x7fa6ac102570 2024-09-10T08:45:33.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.464+0000 7fa6a8ff9640 1 -- 172.21.15.172:0/2617088645 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa6a0017070 con 0x7fa6ac102570 2024-09-10T08:45:33.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.465+0000 7fa6a8ff9640 1 --2- 172.21.15.172:0/2617088645 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa6940a2c90 0x7fa6940a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:33.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.465+0000 7fa6ab7fe640 1 --2- 172.21.15.172:0/2617088645 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa6940a2c90 0x7fa6940a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:33.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.465+0000 7fa6a8ff9640 1 -- 172.21.15.172:0/2617088645 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fa6a00cea30 con 0x7fa6ac102570 2024-09-10T08:45:33.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.466+0000 7fa6ab7fe640 1 --2- 172.21.15.172:0/2617088645 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa6940a2c90 0x7fa6940a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009230 tx=0x7fa69c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:33.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.472+0000 7fa6a8ff9640 1 -- 172.21.15.172:0/2617088645 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6a009afb0 con 0x7fa6ac102570 2024-09-10T08:45:33.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.660+0000 7fa6b1f1b640 1 -- 172.21.15.172:0/2617088645 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa6ac1986d0 con 0x7fa6ac102570 2024-09-10T08:45:33.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.660+0000 7fa6a8ff9640 1 -- 172.21.15.172:0/2617088645 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa6ac1986d0 con 0x7fa6ac102570 2024-09-10T08:45:33.660 INFO:teuthology.orchestra.run.smithi172.stdout:77309411404 2024-09-10T08:45:33.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.662+0000 7fa6b1f1b640 1 -- 172.21.15.172:0/2617088645 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa6940a2c90 msgr2=0x7fa6940a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:33.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.662+0000 7fa6b1f1b640 1 --2- 172.21.15.172:0/2617088645 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa6940a2c90 0x7fa6940a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009230 tx=0x7fa69c0058d0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.662+0000 7fa6b1f1b640 1 -- 172.21.15.172:0/2617088645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac102570 msgr2=0x7fa6ac1978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:33.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.662+0000 7fa6b1f1b640 1 --2- 172.21.15.172:0/2617088645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac102570 0x7fa6ac1978b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000b4a0 tx=0x7fa6a000b960 comp rx=0 tx=0).stop 2024-09-10T08:45:33.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.662+0000 7fa6b1f1b640 1 -- 172.21.15.172:0/2617088645 shutdown_connections 2024-09-10T08:45:33.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.662+0000 7fa6b1f1b640 1 --2- 172.21.15.172:0/2617088645 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa6940a2c90 0x7fa6940a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.662+0000 7fa6b1f1b640 1 --2- 172.21.15.172:0/2617088645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6ac102ef0 0x7fa6ac19bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.662+0000 7fa6b1f1b640 1 --2- 172.21.15.172:0/2617088645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac102570 0x7fa6ac1978b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.662+0000 7fa6b1f1b640 1 --2- 172.21.15.172:0/2617088645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6ac100e20 0x7fa6ac197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:33.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.662+0000 7fa6b1f1b640 1 -- 172.21.15.172:0/2617088645 >> 172.21.15.172:0/2617088645 conn(0x7fa6ac0fc7d0 msgr2=0x7fa6ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:33.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.662+0000 7fa6b1f1b640 1 -- 172.21.15.172:0/2617088645 shutdown_connections 2024-09-10T08:45:33.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.662+0000 7fa6b1f1b640 1 -- 172.21.15.172:0/2617088645 wait complete. 2024-09-10T08:45:34.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.351+0000 7f856f7cf640 1 -- 172.21.15.172:0/3402110117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f856810c170 msgr2=0x7f856810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:34.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.351+0000 7f856f7cf640 1 --2- 172.21.15.172:0/3402110117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f856810c170 0x7f856810e610 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8560009ca0 tx=0x7f856002a950 comp rx=0 tx=0).stop 2024-09-10T08:45:34.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.351+0000 7f856f7cf640 1 -- 172.21.15.172:0/3402110117 shutdown_connections 2024-09-10T08:45:34.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.351+0000 7f856f7cf640 1 --2- 172.21.15.172:0/3402110117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f856810c170 0x7f856810e610 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:34.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.351+0000 7f856f7cf640 1 --2- 172.21.15.172:0/3402110117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8568103660 0x7f856810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:34.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.351+0000 7f856f7cf640 1 --2- 172.21.15.172:0/3402110117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8568102d40 0x7f8568103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:34.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.351+0000 7f856f7cf640 1 -- 172.21.15.172:0/3402110117 >> 172.21.15.172:0/3402110117 conn(0x7f85680fc7a0 msgr2=0x7f85680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.351+0000 7f856f7cf640 1 -- 172.21.15.172:0/3402110117 shutdown_connections 2024-09-10T08:45:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.351+0000 7f856f7cf640 1 -- 172.21.15.172:0/3402110117 wait complete. 2024-09-10T08:45:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.352+0000 7f856f7cf640 1 Processor -- start 2024-09-10T08:45:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.352+0000 7f856f7cf640 1 -- start start 2024-09-10T08:45:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.352+0000 7f856f7cf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8568102d40 0x7f85681009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.352+0000 7f856f7cf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8568103660 0x7f8568100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.352+0000 7f856f7cf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f856810c170 0x7f8568101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.352+0000 7f856f7cf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8568111e20 con 0x7f8568103660 2024-09-10T08:45:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.352+0000 7f856f7cf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8568111ca0 con 0x7f856810c170 2024-09-10T08:45:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.352+0000 7f856f7cf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8568111fa0 con 0x7f8568102d40 2024-09-10T08:45:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.352+0000 7f856cd6a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8568103660 0x7f8568100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.352+0000 7f856cd6a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8568103660 0x7f8568100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45974/0 (socket says 172.21.15.172:45974) 2024-09-10T08:45:34.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.352+0000 7f856cd6a640 1 -- 172.21.15.172:0/3837786595 learned_addr learned my addr 172.21.15.172:0/3837786595 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:34.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.352+0000 7f856d56b640 1 --2- 172.21.15.172:0/3837786595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8568102d40 0x7f85681009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:34.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.352+0000 7f856cd6a640 1 -- 172.21.15.172:0/3837786595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8568102d40 msgr2=0x7f85681009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:34.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.352+0000 7f856cd6a640 1 --2- 172.21.15.172:0/3837786595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8568102d40 0x7f85681009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:34.194 INFO:teuthology.orchestra.run.smithi172.stdout:296352743459 2024-09-10T08:45:34.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.352+0000 7f856cd6a640 1 -- 172.21.15.172:0/3837786595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f856810c170 msgr2=0x7f8568101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:34.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.352+0000 7f856cd6a640 1 --2- 172.21.15.172:0/3837786595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f856810c170 0x7f8568101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:34.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.352+0000 7f856cd6a640 1 -- 172.21.15.172:0/3837786595 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85681a6840 con 0x7f8568103660 2024-09-10T08:45:34.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.353+0000 7f856cd6a640 1 --2- 172.21.15.172:0/3837786595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8568103660 0x7f8568100ee0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f856400c940 tx=0x7f856400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:34.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.353+0000 7f855e7fc640 1 -- 172.21.15.172:0/3837786595 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8564004220 con 0x7f8568103660 2024-09-10T08:45:34.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.353+0000 7f855e7fc640 1 -- 172.21.15.172:0/3837786595 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85640043c0 con 0x7f8568103660 2024-09-10T08:45:34.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.353+0000 7f856f7cf640 1 -- 172.21.15.172:0/3837786595 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85681a6b30 con 0x7f8568103660 2024-09-10T08:45:34.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.353+0000 7f855e7fc640 1 -- 172.21.15.172:0/3837786595 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8564004a00 con 0x7f8568103660 2024-09-10T08:45:34.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.353+0000 7f856f7cf640 1 -- 172.21.15.172:0/3837786595 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85681a70c0 con 0x7f8568103660 2024-09-10T08:45:34.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.354+0000 7f856f7cf640 1 -- 172.21.15.172:0/3837786595 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8568103120 con 0x7f8568103660 2024-09-10T08:45:34.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.355+0000 7f855e7fc640 1 -- 172.21.15.172:0/3837786595 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f85640079b0 con 0x7f8568103660 2024-09-10T08:45:34.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.355+0000 7f855e7fc640 1 --2- 172.21.15.172:0/3837786595 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f85540a2c90 0x7f85540a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:34.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.356+0000 7f856d56b640 1 --2- 172.21.15.172:0/3837786595 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f85540a2c90 0x7f85540a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:34.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.356+0000 7f855e7fc640 1 -- 172.21.15.172:0/3837786595 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f85640cf310 con 0x7f8568103660 2024-09-10T08:45:34.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.356+0000 7f856d56b640 1 --2- 172.21.15.172:0/3837786595 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f85540a2c90 0x7f85540a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8558005fd0 tx=0x7f855800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:34.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.360+0000 7f855e7fc640 1 -- 172.21.15.172:0/3837786595 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8564014030 con 0x7f8568103660 2024-09-10T08:45:34.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.551+0000 7f856f7cf640 1 -- 172.21.15.172:0/3837786595 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f8568101ed0 con 0x7f8568103660 2024-09-10T08:45:34.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.551+0000 7f855e7fc640 1 -- 172.21.15.172:0/3837786595 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f856409be70 con 0x7f8568103660 2024-09-10T08:45:34.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.553+0000 7f856f7cf640 1 -- 172.21.15.172:0/3837786595 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f85540a2c90 msgr2=0x7f85540a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:34.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.553+0000 7f856f7cf640 1 --2- 172.21.15.172:0/3837786595 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f85540a2c90 0x7f85540a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8558005fd0 tx=0x7f855800b390 comp rx=0 tx=0).stop 2024-09-10T08:45:34.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.553+0000 7f856f7cf640 1 -- 172.21.15.172:0/3837786595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8568103660 msgr2=0x7f8568100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:34.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.553+0000 7f856f7cf640 1 --2- 172.21.15.172:0/3837786595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8568103660 0x7f8568100ee0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f856400c940 tx=0x7f856400ce00 comp rx=0 tx=0).stop 2024-09-10T08:45:34.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.554+0000 7f856f7cf640 1 -- 172.21.15.172:0/3837786595 shutdown_connections 2024-09-10T08:45:34.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.554+0000 7f856f7cf640 1 --2- 172.21.15.172:0/3837786595 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f85540a2c90 0x7f85540a5150 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:34.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.554+0000 7f856f7cf640 1 --2- 172.21.15.172:0/3837786595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f856810c170 0x7f8568101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:34.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.554+0000 7f856f7cf640 1 --2- 172.21.15.172:0/3837786595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8568103660 0x7f8568100ee0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:34.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.554+0000 7f856f7cf640 1 --2- 172.21.15.172:0/3837786595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8568102d40 0x7f85681009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:34.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.554+0000 7f856f7cf640 1 -- 172.21.15.172:0/3837786595 >> 172.21.15.172:0/3837786595 conn(0x7f85680fc7a0 msgr2=0x7f85680fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:34.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.554+0000 7f856f7cf640 1 -- 172.21.15.172:0/3837786595 shutdown_connections 2024-09-10T08:45:34.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:33.554+0000 7f856f7cf640 1 -- 172.21.15.172:0/3837786595 wait complete. 2024-09-10T08:45:34.359 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:34 smithi172 ceph-mon[30854]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:34.360 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:34 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3837786595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T08:45:34.360 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:34 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2617088645' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T08:45:34.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:34 smithi191 ceph-mon[35314]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:34.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:34 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3837786595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T08:45:34.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:34 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2617088645' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T08:45:34.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:34 smithi174 ceph-mon[34068]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T08:45:34.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:34 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3837786595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T08:45:34.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:34 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2617088645' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T08:45:34.886 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743458 got 296352743459 for osd.10 2024-09-10T08:45:34.886 DEBUG:teuthology.parallel:result is None 2024-09-10T08:45:35.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.398+0000 7fd1f5d3e640 1 -- 172.21.15.172:0/2585358235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f0102570 msgr2=0x7fd1f01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:35.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.398+0000 7fd1f5d3e640 1 --2- 172.21.15.172:0/2585358235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f0102570 0x7fd1f01029b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009900 tx=0x7fd1e402a480 comp rx=0 tx=0).stop 2024-09-10T08:45:35.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.398+0000 7fd1f5d3e640 1 -- 172.21.15.172:0/2585358235 shutdown_connections 2024-09-10T08:45:35.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.398+0000 7fd1f5d3e640 1 --2- 172.21.15.172:0/2585358235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1f0102ef0 0x7fd1f010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:35.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.398+0000 7fd1f5d3e640 1 --2- 172.21.15.172:0/2585358235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f0102570 0x7fd1f01029b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:35.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.398+0000 7fd1f5d3e640 1 --2- 172.21.15.172:0/2585358235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1f0100e20 0x7fd1f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:35.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.398+0000 7fd1f5d3e640 1 -- 172.21.15.172:0/2585358235 >> 172.21.15.172:0/2585358235 conn(0x7fd1f00fc7d0 msgr2=0x7fd1f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:35.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.398+0000 7fd1f5d3e640 1 -- 172.21.15.172:0/2585358235 shutdown_connections 2024-09-10T08:45:35.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.398+0000 7fd1f5d3e640 1 -- 172.21.15.172:0/2585358235 wait complete. 2024-09-10T08:45:35.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1f5d3e640 1 Processor -- start 2024-09-10T08:45:35.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1f5d3e640 1 -- start start 2024-09-10T08:45:35.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1f5d3e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1f0100e20 0x7fd1f01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:35.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1f5d3e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1f0102570 0x7fd1f0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:35.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1f5d3e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f0102ef0 0x7fd1f019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:35.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1f5d3e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f0110e50 con 0x7fd1f0102ef0 2024-09-10T08:45:35.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1f5d3e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f0110cd0 con 0x7fd1f0102570 2024-09-10T08:45:35.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1f5d3e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f0110fd0 con 0x7fd1f0100e20 2024-09-10T08:45:35.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1effff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f0102ef0 0x7fd1f019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:35.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1effff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f0102ef0 0x7fd1f019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46050/0 (socket says 172.21.15.172:46050) 2024-09-10T08:45:35.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1effff640 1 -- 172.21.15.172:0/1745211852 learned_addr learned my addr 172.21.15.172:0/1745211852 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:35.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1eeffd640 1 --2- 172.21.15.172:0/1745211852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1f0102570 0x7fd1f0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:35.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1ef7fe640 1 --2- 172.21.15.172:0/1745211852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1f0100e20 0x7fd1f01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:35.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1effff640 1 -- 172.21.15.172:0/1745211852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1f0100e20 msgr2=0x7fd1f01984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:35.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1effff640 1 --2- 172.21.15.172:0/1745211852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1f0100e20 0x7fd1f01984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:35.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1effff640 1 -- 172.21.15.172:0/1745211852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1f0102570 msgr2=0x7fd1f0198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:35.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1effff640 1 --2- 172.21.15.172:0/1745211852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1f0102570 0x7fd1f0198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:35.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.399+0000 7fd1effff640 1 -- 172.21.15.172:0/1745211852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f019d400 con 0x7fd1f0102ef0 2024-09-10T08:45:35.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.400+0000 7fd1effff640 1 --2- 172.21.15.172:0/1745211852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f0102ef0 0x7fd1f019cd20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800b970 tx=0x7fd1e800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:35.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.400+0000 7fd1ecff9640 1 -- 172.21.15.172:0/1745211852 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8019070 con 0x7fd1f0102ef0 2024-09-10T08:45:35.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.400+0000 7fd1ecff9640 1 -- 172.21.15.172:0/1745211852 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1e8014440 con 0x7fd1f0102ef0 2024-09-10T08:45:35.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.400+0000 7fd1f5d3e640 1 -- 172.21.15.172:0/1745211852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f019d6f0 con 0x7fd1f0102ef0 2024-09-10T08:45:35.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.400+0000 7fd1ecff9640 1 -- 172.21.15.172:0/1745211852 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e80075d0 con 0x7fd1f0102ef0 2024-09-10T08:45:35.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.400+0000 7fd1f5d3e640 1 -- 172.21.15.172:0/1745211852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f01a4fd0 con 0x7fd1f0102ef0 2024-09-10T08:45:35.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.401+0000 7fd1f5d3e640 1 -- 172.21.15.172:0/1745211852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f0101200 con 0x7fd1f0102ef0 2024-09-10T08:45:35.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.402+0000 7fd1ecff9640 1 -- 172.21.15.172:0/1745211852 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd1e801c070 con 0x7fd1f0102ef0 2024-09-10T08:45:35.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.403+0000 7fd1ecff9640 1 --2- 172.21.15.172:0/1745211852 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1d80a2c10 0x7fd1d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:35.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.403+0000 7fd1ef7fe640 1 --2- 172.21.15.172:0/1745211852 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1d80a2c10 0x7fd1d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:35.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.403+0000 7fd1ecff9640 1 -- 172.21.15.172:0/1745211852 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fd1e80cee80 con 0x7fd1f0102ef0 2024-09-10T08:45:35.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.403+0000 7fd1ef7fe640 1 --2- 172.21.15.172:0/1745211852 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1d80a2c10 0x7fd1d80a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00058c0 tx=0x7fd1e000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:35.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.406+0000 7fd1ecff9640 1 -- 172.21.15.172:0/1745211852 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1e809b400 con 0x7fd1f0102ef0 2024-09-10T08:45:35.536 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:35 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2162953534' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T08:45:35.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:35 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2162953534' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T08:45:35.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.582+0000 7fd1f5d3e640 1 -- 172.21.15.172:0/1745211852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd1f01a51a0 con 0x7fd1f0102ef0 2024-09-10T08:45:35.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.582+0000 7fd1ecff9640 1 -- 172.21.15.172:0/1745211852 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd1e809f700 con 0x7fd1f0102ef0 2024-09-10T08:45:35.583 INFO:teuthology.orchestra.run.smithi172.stdout:163208757308 2024-09-10T08:45:35.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.584+0000 7fd1f5d3e640 1 -- 172.21.15.172:0/1745211852 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1d80a2c10 msgr2=0x7fd1d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:35.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.584+0000 7fd1f5d3e640 1 --2- 172.21.15.172:0/1745211852 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1d80a2c10 0x7fd1d80a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00058c0 tx=0x7fd1e000a2d0 comp rx=0 tx=0).stop 2024-09-10T08:45:35.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.584+0000 7fd1f5d3e640 1 -- 172.21.15.172:0/1745211852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f0102ef0 msgr2=0x7fd1f019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:35.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.584+0000 7fd1f5d3e640 1 --2- 172.21.15.172:0/1745211852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f0102ef0 0x7fd1f019cd20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800b970 tx=0x7fd1e800be30 comp rx=0 tx=0).stop 2024-09-10T08:45:35.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.584+0000 7fd1f5d3e640 1 -- 172.21.15.172:0/1745211852 shutdown_connections 2024-09-10T08:45:35.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.584+0000 7fd1f5d3e640 1 --2- 172.21.15.172:0/1745211852 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1d80a2c10 0x7fd1d80a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:35.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.584+0000 7fd1f5d3e640 1 --2- 172.21.15.172:0/1745211852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f0102ef0 0x7fd1f019cd20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:35.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.584+0000 7fd1f5d3e640 1 --2- 172.21.15.172:0/1745211852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1f0102570 0x7fd1f0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:35.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.584+0000 7fd1f5d3e640 1 --2- 172.21.15.172:0/1745211852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1f0100e20 0x7fd1f01984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:35.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.584+0000 7fd1f5d3e640 1 -- 172.21.15.172:0/1745211852 >> 172.21.15.172:0/1745211852 conn(0x7fd1f00fc7d0 msgr2=0x7fd1f0069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:35.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.584+0000 7fd1f5d3e640 1 -- 172.21.15.172:0/1745211852 shutdown_connections 2024-09-10T08:45:35.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:35.584+0000 7fd1f5d3e640 1 -- 172.21.15.172:0/1745211852 wait complete. 2024-09-10T08:45:35.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:35 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2162953534' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T08:45:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.004+0000 7f80e01a4640 1 -- 172.21.15.172:0/1033386115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80d8102570 msgr2=0x7f80d81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.004+0000 7f80e01a4640 1 --2- 172.21.15.172:0/1033386115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80d8102570 0x7f80d81029b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009900 tx=0x7f80c802a480 comp rx=0 tx=0).stop 2024-09-10T08:45:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.004+0000 7f80e01a4640 1 -- 172.21.15.172:0/1033386115 shutdown_connections 2024-09-10T08:45:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.004+0000 7f80e01a4640 1 --2- 172.21.15.172:0/1033386115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80d8102ef0 0x7f80d810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.004+0000 7f80e01a4640 1 --2- 172.21.15.172:0/1033386115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80d8102570 0x7f80d81029b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.005+0000 7f80e01a4640 1 --2- 172.21.15.172:0/1033386115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80d8100e20 0x7f80d8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.005+0000 7f80e01a4640 1 -- 172.21.15.172:0/1033386115 >> 172.21.15.172:0/1033386115 conn(0x7f80d80fc7d0 msgr2=0x7f80d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.005+0000 7f80e01a4640 1 -- 172.21.15.172:0/1033386115 shutdown_connections 2024-09-10T08:45:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.005+0000 7f80e01a4640 1 -- 172.21.15.172:0/1033386115 wait complete. 2024-09-10T08:45:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.005+0000 7f80e01a4640 1 Processor -- start 2024-09-10T08:45:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.005+0000 7f80e01a4640 1 -- start start 2024-09-10T08:45:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.005+0000 7f80e01a4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80d8100e20 0x7f80d8198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.005+0000 7f80e01a4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80d8102570 0x7f80d8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.005+0000 7f80e01a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80d8102ef0 0x7f80d819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.005+0000 7f80e01a4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f80d8110e80 con 0x7f80d8102ef0 2024-09-10T08:45:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.005+0000 7f80e01a4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80d8110d00 con 0x7f80d8102570 2024-09-10T08:45:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.005+0000 7f80e01a4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f80d8111000 con 0x7f80d8100e20 2024-09-10T08:45:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.006+0000 7f80de741640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80d8102ef0 0x7f80d819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.006+0000 7f80de741640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80d8102ef0 0x7f80d819ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46082/0 (socket says 172.21.15.172:46082) 2024-09-10T08:45:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.006+0000 7f80de741640 1 -- 172.21.15.172:0/1858084502 learned_addr learned my addr 172.21.15.172:0/1858084502 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.006+0000 7f80dd73f640 1 --2- 172.21.15.172:0/1858084502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80d8102570 0x7f80d8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.006+0000 7f80ddf40640 1 --2- 172.21.15.172:0/1858084502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80d8100e20 0x7f80d8198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.006+0000 7f80de741640 1 -- 172.21.15.172:0/1858084502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80d8100e20 msgr2=0x7f80d8198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.006+0000 7f80de741640 1 --2- 172.21.15.172:0/1858084502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80d8100e20 0x7f80d8198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.006+0000 7f80de741640 1 -- 172.21.15.172:0/1858084502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80d8102570 msgr2=0x7f80d8198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.006+0000 7f80de741640 1 --2- 172.21.15.172:0/1858084502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80d8102570 0x7f80d8198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.006+0000 7f80de741640 1 -- 172.21.15.172:0/1858084502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d819d4e0 con 0x7f80d8102ef0 2024-09-10T08:45:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.006+0000 7f80de741640 1 --2- 172.21.15.172:0/1858084502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80d8102ef0 0x7f80d819ce00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f80d000cad0 tx=0x7f80d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.006+0000 7f80ceffd640 1 -- 172.21.15.172:0/1858084502 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d000ae20 con 0x7f80d8102ef0 2024-09-10T08:45:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.006+0000 7f80e01a4640 1 -- 172.21.15.172:0/1858084502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d819d7d0 con 0x7f80d8102ef0 2024-09-10T08:45:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.006+0000 7f80ceffd640 1 -- 172.21.15.172:0/1858084502 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80d0005020 con 0x7f80d8102ef0 2024-09-10T08:45:36.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.006+0000 7f80ceffd640 1 -- 172.21.15.172:0/1858084502 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d000f6b0 con 0x7f80d8102ef0 2024-09-10T08:45:36.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.006+0000 7f80e01a4640 1 -- 172.21.15.172:0/1858084502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d81a50b0 con 0x7f80d8102ef0 2024-09-10T08:45:36.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.007+0000 7f80e01a4640 1 -- 172.21.15.172:0/1858084502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d8101200 con 0x7f80d8102ef0 2024-09-10T08:45:36.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.008+0000 7f80ceffd640 1 -- 172.21.15.172:0/1858084502 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f80d00051d0 con 0x7f80d8102ef0 2024-09-10T08:45:36.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.009+0000 7f80ceffd640 1 --2- 172.21.15.172:0/1858084502 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80c40a2c10 0x7f80c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:36.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.009+0000 7f80ddf40640 1 --2- 172.21.15.172:0/1858084502 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80c40a2c10 0x7f80c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:36.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.009+0000 7f80ceffd640 1 -- 172.21.15.172:0/1858084502 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f80d00cf420 con 0x7f80d8102ef0 2024-09-10T08:45:36.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.010+0000 7f80ddf40640 1 --2- 172.21.15.172:0/1858084502 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80c40a2c10 0x7f80c40a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f80d4009bd0 tx=0x7f80d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:36.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.013+0000 7f80ceffd640 1 -- 172.21.15.172:0/1858084502 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f80d0014030 con 0x7f80d8102ef0 2024-09-10T08:45:36.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.186+0000 7f80e01a4640 1 -- 172.21.15.172:0/1858084502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f80d810a0a0 con 0x7f80d8102ef0 2024-09-10T08:45:36.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.186+0000 7f80ceffd640 1 -- 172.21.15.172:0/1858084502 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f80d0084e20 con 0x7f80d8102ef0 2024-09-10T08:45:36.187 INFO:teuthology.orchestra.run.smithi172.stdout:34359738453 2024-09-10T08:45:36.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.188+0000 7f80e01a4640 1 -- 172.21.15.172:0/1858084502 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80c40a2c10 msgr2=0x7f80c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:36.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.188+0000 7f80e01a4640 1 --2- 172.21.15.172:0/1858084502 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80c40a2c10 0x7f80c40a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f80d4009bd0 tx=0x7f80d4005f90 comp rx=0 tx=0).stop 2024-09-10T08:45:36.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.188+0000 7f80e01a4640 1 -- 172.21.15.172:0/1858084502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80d8102ef0 msgr2=0x7f80d819ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:36.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.188+0000 7f80e01a4640 1 --2- 172.21.15.172:0/1858084502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80d8102ef0 0x7f80d819ce00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f80d000cad0 tx=0x7f80d000cf90 comp rx=0 tx=0).stop 2024-09-10T08:45:36.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.188+0000 7f80e01a4640 1 -- 172.21.15.172:0/1858084502 shutdown_connections 2024-09-10T08:45:36.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.188+0000 7f80e01a4640 1 --2- 172.21.15.172:0/1858084502 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80c40a2c10 0x7f80c40a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.188+0000 7f80e01a4640 1 --2- 172.21.15.172:0/1858084502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80d8102ef0 0x7f80d819ce00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.188+0000 7f80e01a4640 1 --2- 172.21.15.172:0/1858084502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80d8102570 0x7f80d8198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.188+0000 7f80e01a4640 1 --2- 172.21.15.172:0/1858084502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80d8100e20 0x7f80d8198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.188+0000 7f80e01a4640 1 -- 172.21.15.172:0/1858084502 >> 172.21.15.172:0/1858084502 conn(0x7f80d80fc7d0 msgr2=0x7f80d806a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:36.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.188+0000 7f80e01a4640 1 -- 172.21.15.172:0/1858084502 shutdown_connections 2024-09-10T08:45:36.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:36.188+0000 7f80e01a4640 1 -- 172.21.15.172:0/1858084502 wait complete. 2024-09-10T08:45:36.389 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757306 got 163208757308 for osd.5 2024-09-10T08:45:36.389 DEBUG:teuthology.parallel:result is None 2024-09-10T08:45:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:36 smithi191 ceph-mon[35314]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:36 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1745211852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T08:45:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:36 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1858084502' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T08:45:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:36 smithi172 ceph-mon[30854]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:36 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1745211852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T08:45:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:36 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1858084502' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T08:45:36.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:36 smithi174 ceph-mon[34068]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:36.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:36 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1745211852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T08:45:36.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:36 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1858084502' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T08:45:36.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.180+0000 7f1796a7c640 1 -- 172.21.15.172:0/1030142272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1790101de0 msgr2=0x7f17901021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:36.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.180+0000 7f1796a7c640 1 --2- 172.21.15.172:0/1030142272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1790101de0 0x7f17901021c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1780009900 tx=0x7f178002a410 comp rx=0 tx=0).stop 2024-09-10T08:45:36.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.180+0000 7f1796a7c640 1 -- 172.21.15.172:0/1030142272 shutdown_connections 2024-09-10T08:45:36.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.180+0000 7f1796a7c640 1 --2- 172.21.15.172:0/1030142272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1790106f10 0x7f179010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.180+0000 7f1796a7c640 1 --2- 172.21.15.172:0/1030142272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1790102790 0x7f17901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.180+0000 7f1796a7c640 1 --2- 172.21.15.172:0/1030142272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1790101de0 0x7f17901021c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.180+0000 7f1796a7c640 1 -- 172.21.15.172:0/1030142272 >> 172.21.15.172:0/1030142272 conn(0x7f17900fd8c0 msgr2=0x7f17900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:36.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.180+0000 7f1796a7c640 1 -- 172.21.15.172:0/1030142272 shutdown_connections 2024-09-10T08:45:36.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.180+0000 7f1796a7c640 1 -- 172.21.15.172:0/1030142272 wait complete. 2024-09-10T08:45:36.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.181+0000 7f1796a7c640 1 Processor -- start 2024-09-10T08:45:36.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.181+0000 7f1796a7c640 1 -- start start 2024-09-10T08:45:36.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.181+0000 7f1796a7c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1790101de0 0x7f17900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:36.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.181+0000 7f1796a7c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1790102790 0x7f1790074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:36.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.181+0000 7f1796a7c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1790106f10 0x7f1790074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:36.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.181+0000 7f1796a7c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1790111e20 con 0x7f1790101de0 2024-09-10T08:45:36.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.181+0000 7f1796a7c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1790111ca0 con 0x7f1790106f10 2024-09-10T08:45:36.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.181+0000 7f1796a7c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1790111fa0 con 0x7f1790102790 2024-09-10T08:45:36.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.181+0000 7f1794818640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1790101de0 0x7f17900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:36.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.181+0000 7f1794818640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1790101de0 0x7f17900744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46020/0 (socket says 172.21.15.172:46020) 2024-09-10T08:45:36.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.181+0000 7f1794818640 1 -- 172.21.15.172:0/2162953534 learned_addr learned my addr 172.21.15.172:0/2162953534 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:36.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.181+0000 7f1795019640 1 --2- 172.21.15.172:0/2162953534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1790106f10 0x7f1790074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:36.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.182+0000 7f1794818640 1 -- 172.21.15.172:0/2162953534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1790102790 msgr2=0x7f1790074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:36.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.182+0000 7f1794818640 1 --2- 172.21.15.172:0/2162953534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1790102790 0x7f1790074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.182+0000 7f1794818640 1 -- 172.21.15.172:0/2162953534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1790106f10 msgr2=0x7f1790074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:36.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.182+0000 7f1794818640 1 --2- 172.21.15.172:0/2162953534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1790106f10 0x7f1790074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.928 INFO:teuthology.orchestra.run.smithi172.stdout:274877906984 2024-09-10T08:45:36.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.182+0000 7f1794818640 1 -- 172.21.15.172:0/2162953534 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17901a6860 con 0x7f1790101de0 2024-09-10T08:45:36.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.182+0000 7f1795019640 1 --2- 172.21.15.172:0/2162953534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1790106f10 0x7f1790074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:36.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.182+0000 7f1794818640 1 --2- 172.21.15.172:0/2162953534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1790101de0 0x7f17900744f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1780006010 tx=0x7f1780004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:36.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.182+0000 7f178dffb640 1 -- 172.21.15.172:0/2162953534 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178003f030 con 0x7f1790101de0 2024-09-10T08:45:36.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.182+0000 7f178dffb640 1 -- 172.21.15.172:0/2162953534 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17800035f0 con 0x7f1790101de0 2024-09-10T08:45:36.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.182+0000 7f178dffb640 1 -- 172.21.15.172:0/2162953534 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780035070 con 0x7f1790101de0 2024-09-10T08:45:36.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.182+0000 7f1796a7c640 1 -- 172.21.15.172:0/2162953534 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17901a6af0 con 0x7f1790101de0 2024-09-10T08:45:36.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.182+0000 7f1796a7c640 1 -- 172.21.15.172:0/2162953534 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17901a6fd0 con 0x7f1790101de0 2024-09-10T08:45:36.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.183+0000 7f1796a7c640 1 -- 172.21.15.172:0/2162953534 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17901021c0 con 0x7f1790101de0 2024-09-10T08:45:36.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.184+0000 7f178dffb640 1 -- 172.21.15.172:0/2162953534 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1780003b10 con 0x7f1790101de0 2024-09-10T08:45:36.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.185+0000 7f178dffb640 1 --2- 172.21.15.172:0/2162953534 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f17780a2c10 0x7f17780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:36.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.185+0000 7f178dffb640 1 -- 172.21.15.172:0/2162953534 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f17800f0350 con 0x7f1790101de0 2024-09-10T08:45:36.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.185+0000 7f178ffff640 1 --2- 172.21.15.172:0/2162953534 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f17780a2c10 0x7f17780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:36.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.185+0000 7f178ffff640 1 --2- 172.21.15.172:0/2162953534 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f17780a2c10 0x7f17780a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1790075f80 tx=0x7f1784005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:36.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.188+0000 7f178dffb640 1 -- 172.21.15.172:0/2162953534 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17800bce10 con 0x7f1790101de0 2024-09-10T08:45:36.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.359+0000 7f1796a7c640 1 -- 172.21.15.172:0/2162953534 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f1790075a20 con 0x7f1790101de0 2024-09-10T08:45:36.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.359+0000 7f178dffb640 1 -- 172.21.15.172:0/2162953534 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f17800bc710 con 0x7f1790101de0 2024-09-10T08:45:36.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.362+0000 7f1796a7c640 1 -- 172.21.15.172:0/2162953534 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f17780a2c10 msgr2=0x7f17780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:36.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.362+0000 7f1796a7c640 1 --2- 172.21.15.172:0/2162953534 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f17780a2c10 0x7f17780a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1790075f80 tx=0x7f1784005bf0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.362+0000 7f1796a7c640 1 -- 172.21.15.172:0/2162953534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1790101de0 msgr2=0x7f17900744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:36.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.362+0000 7f1796a7c640 1 --2- 172.21.15.172:0/2162953534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1790101de0 0x7f17900744f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1780006010 tx=0x7f1780004270 comp rx=0 tx=0).stop 2024-09-10T08:45:36.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.362+0000 7f1796a7c640 1 -- 172.21.15.172:0/2162953534 shutdown_connections 2024-09-10T08:45:36.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.362+0000 7f1796a7c640 1 --2- 172.21.15.172:0/2162953534 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f17780a2c10 0x7f17780a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.362+0000 7f1796a7c640 1 --2- 172.21.15.172:0/2162953534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1790106f10 0x7f1790074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.362+0000 7f1796a7c640 1 --2- 172.21.15.172:0/2162953534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1790102790 0x7f1790074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.362+0000 7f1796a7c640 1 --2- 172.21.15.172:0/2162953534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1790101de0 0x7f17900744f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:36.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.362+0000 7f1796a7c640 1 -- 172.21.15.172:0/2162953534 >> 172.21.15.172:0/2162953534 conn(0x7f17900fd8c0 msgr2=0x7f17900fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:36.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.362+0000 7f1796a7c640 1 -- 172.21.15.172:0/2162953534 shutdown_connections 2024-09-10T08:45:36.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:34.362+0000 7f1796a7c640 1 -- 172.21.15.172:0/2162953534 wait complete. 2024-09-10T08:45:37.518 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906982 got 274877906984 for osd.9 2024-09-10T08:45:37.519 DEBUG:teuthology.parallel:result is None 2024-09-10T08:45:37.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.747+0000 7fd774926640 1 -- 172.21.15.172:0/3276545443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd77010c180 msgr2=0x7fd77010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:37.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.747+0000 7fd774926640 1 --2- 172.21.15.172:0/3276545443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd77010c180 0x7fd77010e600 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd768009ca0 tx=0x7fd76802a950 comp rx=0 tx=0).stop 2024-09-10T08:45:37.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.747+0000 7fd774926640 1 -- 172.21.15.172:0/3276545443 shutdown_connections 2024-09-10T08:45:37.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.747+0000 7fd774926640 1 --2- 172.21.15.172:0/3276545443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd77010c180 0x7fd77010e600 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:37.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.747+0000 7fd774926640 1 --2- 172.21.15.172:0/3276545443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7701036c0 0x7fd77010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:37.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.747+0000 7fd774926640 1 --2- 172.21.15.172:0/3276545443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd770102da0 0x7fd770103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:37.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.747+0000 7fd774926640 1 -- 172.21.15.172:0/3276545443 >> 172.21.15.172:0/3276545443 conn(0x7fd7700fc7a0 msgr2=0x7fd7700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:37.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.747+0000 7fd774926640 1 -- 172.21.15.172:0/3276545443 shutdown_connections 2024-09-10T08:45:37.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.747+0000 7fd774926640 1 -- 172.21.15.172:0/3276545443 wait complete. 2024-09-10T08:45:37.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.748+0000 7fd774926640 1 Processor -- start 2024-09-10T08:45:37.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.748+0000 7fd774926640 1 -- start start 2024-09-10T08:45:37.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.748+0000 7fd774926640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd770102da0 0x7fd7701982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:37.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.748+0000 7fd774926640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7701036c0 0x7fd770198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:37.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.748+0000 7fd774926640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd77010c180 0x7fd77019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:37.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.748+0000 7fd774926640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd770111e50 con 0x7fd7701036c0 2024-09-10T08:45:37.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.748+0000 7fd774926640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd770111cd0 con 0x7fd77010c180 2024-09-10T08:45:37.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.748+0000 7fd774926640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd770111fd0 con 0x7fd770102da0 2024-09-10T08:45:37.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.748+0000 7fd76dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7701036c0 0x7fd770198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:37.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.749+0000 7fd76dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7701036c0 0x7fd770198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42942/0 (socket says 172.21.15.172:42942) 2024-09-10T08:45:37.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.749+0000 7fd76dd9b640 1 -- 172.21.15.172:0/1740756798 learned_addr learned my addr 172.21.15.172:0/1740756798 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:37.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.749+0000 7fd76e59c640 1 --2- 172.21.15.172:0/1740756798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd770102da0 0x7fd7701982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:37.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.749+0000 7fd76ed9d640 1 --2- 172.21.15.172:0/1740756798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd77010c180 0x7fd77019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:37.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.749+0000 7fd76dd9b640 1 -- 172.21.15.172:0/1740756798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd770102da0 msgr2=0x7fd7701982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:37.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.749+0000 7fd76dd9b640 1 --2- 172.21.15.172:0/1740756798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd770102da0 0x7fd7701982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:37.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.749+0000 7fd76dd9b640 1 -- 172.21.15.172:0/1740756798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd77010c180 msgr2=0x7fd77019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:37.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.749+0000 7fd76dd9b640 1 --2- 172.21.15.172:0/1740756798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd77010c180 0x7fd77019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:37.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.749+0000 7fd76dd9b640 1 -- 172.21.15.172:0/1740756798 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77019d2a0 con 0x7fd7701036c0 2024-09-10T08:45:37.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.749+0000 7fd76ed9d640 1 --2- 172.21.15.172:0/1740756798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd77010c180 0x7fd77019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:37.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.749+0000 7fd76dd9b640 1 --2- 172.21.15.172:0/1740756798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7701036c0 0x7fd770198830 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c920 tx=0x7fd76400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:37.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.749+0000 7fd75f7fe640 1 -- 172.21.15.172:0/1740756798 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764007c70 con 0x7fd7701036c0 2024-09-10T08:45:37.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.749+0000 7fd75f7fe640 1 -- 172.21.15.172:0/1740756798 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd764007e10 con 0x7fd7701036c0 2024-09-10T08:45:37.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.749+0000 7fd774926640 1 -- 172.21.15.172:0/1740756798 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77019d590 con 0x7fd7701036c0 2024-09-10T08:45:37.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.749+0000 7fd75f7fe640 1 -- 172.21.15.172:0/1740756798 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764018820 con 0x7fd7701036c0 2024-09-10T08:45:37.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.749+0000 7fd774926640 1 -- 172.21.15.172:0/1740756798 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7701a4e70 con 0x7fd7701036c0 2024-09-10T08:45:37.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.750+0000 7fd774926640 1 -- 172.21.15.172:0/1740756798 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd770103180 con 0x7fd7701036c0 2024-09-10T08:45:37.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.751+0000 7fd75f7fe640 1 -- 172.21.15.172:0/1740756798 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd7640189c0 con 0x7fd7701036c0 2024-09-10T08:45:37.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.752+0000 7fd75f7fe640 1 --2- 172.21.15.172:0/1740756798 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd7580a2c10 0x7fd7580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:37.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.752+0000 7fd76e59c640 1 --2- 172.21.15.172:0/1740756798 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd7580a2c10 0x7fd7580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:37.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.752+0000 7fd75f7fe640 1 -- 172.21.15.172:0/1740756798 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fd7640cf9c0 con 0x7fd7701036c0 2024-09-10T08:45:37.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.753+0000 7fd76e59c640 1 --2- 172.21.15.172:0/1740756798 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd7580a2c10 0x7fd7580a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd760009890 tx=0x7fd760008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:37.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.755+0000 7fd75f7fe640 1 -- 172.21.15.172:0/1740756798 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd764014030 con 0x7fd7701036c0 2024-09-10T08:45:37.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.931+0000 7fd774926640 1 -- 172.21.15.172:0/1740756798 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd770199650 con 0x7fd7701036c0 2024-09-10T08:45:37.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.931+0000 7fd75f7fe640 1 -- 172.21.15.172:0/1740756798 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd770199650 con 0x7fd7701036c0 2024-09-10T08:45:37.932 INFO:teuthology.orchestra.run.smithi172.stdout:55834574929 2024-09-10T08:45:37.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.933+0000 7fd774926640 1 -- 172.21.15.172:0/1740756798 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd7580a2c10 msgr2=0x7fd7580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:37.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.933+0000 7fd774926640 1 --2- 172.21.15.172:0/1740756798 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd7580a2c10 0x7fd7580a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd760009890 tx=0x7fd760008040 comp rx=0 tx=0).stop 2024-09-10T08:45:37.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.933+0000 7fd774926640 1 -- 172.21.15.172:0/1740756798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7701036c0 msgr2=0x7fd770198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:37.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.933+0000 7fd774926640 1 --2- 172.21.15.172:0/1740756798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7701036c0 0x7fd770198830 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd76400c920 tx=0x7fd76400cde0 comp rx=0 tx=0).stop 2024-09-10T08:45:37.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.933+0000 7fd774926640 1 -- 172.21.15.172:0/1740756798 shutdown_connections 2024-09-10T08:45:37.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.933+0000 7fd774926640 1 --2- 172.21.15.172:0/1740756798 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd7580a2c10 0x7fd7580a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:37.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.933+0000 7fd774926640 1 --2- 172.21.15.172:0/1740756798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd77010c180 0x7fd77019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:37.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.933+0000 7fd774926640 1 --2- 172.21.15.172:0/1740756798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7701036c0 0x7fd770198830 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:37.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.933+0000 7fd774926640 1 --2- 172.21.15.172:0/1740756798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd770102da0 0x7fd7701982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:37.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.933+0000 7fd774926640 1 -- 172.21.15.172:0/1740756798 >> 172.21.15.172:0/1740756798 conn(0x7fd7700fc7a0 msgr2=0x7fd7700fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:37.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.934+0000 7fd774926640 1 -- 172.21.15.172:0/1740756798 shutdown_connections 2024-09-10T08:45:37.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:37.934+0000 7fd774926640 1 -- 172.21.15.172:0/1740756798 wait complete. 2024-09-10T08:45:38.171 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411401 got 77309411404 for osd.2 2024-09-10T08:45:38.171 DEBUG:teuthology.parallel:result is None 2024-09-10T08:45:38.403 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:38.403 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:38.404 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:38.404 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:38.491 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:38 smithi172 ceph-mon[30854]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-10T08:45:38.491 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:38 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1740756798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T08:45:38.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:38 smithi191 ceph-mon[35314]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-10T08:45:38.698 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:38 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1740756798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T08:45:38.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:38 smithi174 ceph-mon[34068]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-10T08:45:38.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:38 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1740756798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T08:45:38.794 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574927 got 55834574929 for osd.1 2024-09-10T08:45:38.794 DEBUG:teuthology.parallel:result is None 2024-09-10T08:45:39.250 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738450 got 34359738453 for osd.0 2024-09-10T08:45:39.251 DEBUG:teuthology.parallel:result is None 2024-09-10T08:45:39.499 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:39.499 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.815+0000 7f6b917b4640 1 -- 172.21.15.172:0/2832510565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b8c100eb0 msgr2=0x7f6b8c106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.815+0000 7f6b917b4640 1 --2- 172.21.15.172:0/2832510565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b8c100eb0 0x7f6b8c106840 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009900 tx=0x7f6b7402a410 comp rx=0 tx=0).stop 2024-09-10T08:45:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.816+0000 7f6b917b4640 1 -- 172.21.15.172:0/2832510565 shutdown_connections 2024-09-10T08:45:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.816+0000 7f6b917b4640 1 --2- 172.21.15.172:0/2832510565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b8c106f70 0x7f6b8c10eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.816+0000 7f6b917b4640 1 --2- 172.21.15.172:0/2832510565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b8c100eb0 0x7f6b8c106840 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.816+0000 7f6b917b4640 1 --2- 172.21.15.172:0/2832510565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b8c100500 0x7f6b8c1008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.816+0000 7f6b917b4640 1 -- 172.21.15.172:0/2832510565 >> 172.21.15.172:0/2832510565 conn(0x7f6b8c0fc5c0 msgr2=0x7f6b8c0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.816+0000 7f6b917b4640 1 -- 172.21.15.172:0/2832510565 shutdown_connections 2024-09-10T08:45:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.816+0000 7f6b917b4640 1 -- 172.21.15.172:0/2832510565 wait complete. 2024-09-10T08:45:39.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.816+0000 7f6b917b4640 1 Processor -- start 2024-09-10T08:45:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.816+0000 7f6b917b4640 1 -- start start 2024-09-10T08:45:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b917b4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b8c100500 0x7f6b8c101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b917b4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b8c100eb0 0x7f6b8c1019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b917b4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b8c106f70 0x7f6b8c101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b917b4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c111e40 con 0x7f6b8c100eb0 2024-09-10T08:45:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b917b4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c111cc0 con 0x7f6b8c100500 2024-09-10T08:45:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b917b4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c111fc0 con 0x7f6b8c106f70 2024-09-10T08:45:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b8a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b8c100eb0 0x7f6b8c1019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b8a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b8c100eb0 0x7f6b8c1019c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42966/0 (socket says 172.21.15.172:42966) 2024-09-10T08:45:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b8a7fc640 1 -- 172.21.15.172:0/140100382 learned_addr learned my addr 172.21.15.172:0/140100382 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b8b7fe640 1 --2- 172.21.15.172:0/140100382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b8c106f70 0x7f6b8c101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b8affd640 1 --2- 172.21.15.172:0/140100382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b8c100500 0x7f6b8c101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b8a7fc640 1 -- 172.21.15.172:0/140100382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b8c106f70 msgr2=0x7f6b8c101f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b8a7fc640 1 --2- 172.21.15.172:0/140100382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b8c106f70 0x7f6b8c101f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b8a7fc640 1 -- 172.21.15.172:0/140100382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b8c100500 msgr2=0x7f6b8c101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b8a7fc640 1 --2- 172.21.15.172:0/140100382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b8c100500 0x7f6b8c101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b8a7fc640 1 -- 172.21.15.172:0/140100382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c1a6840 con 0x7f6b8c100eb0 2024-09-10T08:45:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b8a7fc640 1 --2- 172.21.15.172:0/140100382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b8c100eb0 0x7f6b8c1019c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6b7402a930 tx=0x7f6b74004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b83fff640 1 -- 172.21.15.172:0/140100382 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7403f030 con 0x7f6b8c100eb0 2024-09-10T08:45:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.817+0000 7f6b83fff640 1 -- 172.21.15.172:0/140100382 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b74004490 con 0x7f6b8c100eb0 2024-09-10T08:45:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.818+0000 7f6b917b4640 1 -- 172.21.15.172:0/140100382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8c1a6ad0 con 0x7f6b8c100eb0 2024-09-10T08:45:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.818+0000 7f6b83fff640 1 -- 172.21.15.172:0/140100382 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74035070 con 0x7f6b8c100eb0 2024-09-10T08:45:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.818+0000 7f6b917b4640 1 -- 172.21.15.172:0/140100382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b8c1a7000 con 0x7f6b8c100eb0 2024-09-10T08:45:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.819+0000 7f6b917b4640 1 -- 172.21.15.172:0/140100382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b8c1008e0 con 0x7f6b8c100eb0 2024-09-10T08:45:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.820+0000 7f6b83fff640 1 -- 172.21.15.172:0/140100382 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6b74042030 con 0x7f6b8c100eb0 2024-09-10T08:45:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.821+0000 7f6b83fff640 1 --2- 172.21.15.172:0/140100382 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6b780a2c90 0x7f6b780a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.821+0000 7f6b8affd640 1 --2- 172.21.15.172:0/140100382 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6b780a2c90 0x7f6b780a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.821+0000 7f6b83fff640 1 -- 172.21.15.172:0/140100382 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f6b740efc80 con 0x7f6b8c100eb0 2024-09-10T08:45:39.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.821+0000 7f6b8affd640 1 --2- 172.21.15.172:0/140100382 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6b780a2c90 0x7f6b780a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c069fc0 tx=0x7f6b7c009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:39.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.824+0000 7f6b83fff640 1 -- 172.21.15.172:0/140100382 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b740bc0f0 con 0x7f6b8c100eb0 2024-09-10T08:45:39.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.998+0000 7f6b917b4640 1 -- 172.21.15.172:0/140100382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6b8c10a0e0 con 0x7f6b8c100eb0 2024-09-10T08:45:39.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:39.998+0000 7f6b83fff640 1 -- 172.21.15.172:0/140100382 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6b740c03f0 con 0x7f6b8c100eb0 2024-09-10T08:45:39.999 INFO:teuthology.orchestra.run.smithi172.stdout:107374182471 2024-09-10T08:45:40.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:40.000+0000 7f6b917b4640 1 -- 172.21.15.172:0/140100382 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6b780a2c90 msgr2=0x7f6b780a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:40.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:40.000+0000 7f6b917b4640 1 --2- 172.21.15.172:0/140100382 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6b780a2c90 0x7f6b780a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c069fc0 tx=0x7f6b7c009680 comp rx=0 tx=0).stop 2024-09-10T08:45:40.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:40.000+0000 7f6b917b4640 1 -- 172.21.15.172:0/140100382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b8c100eb0 msgr2=0x7f6b8c1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:40.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:40.000+0000 7f6b917b4640 1 --2- 172.21.15.172:0/140100382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b8c100eb0 0x7f6b8c1019c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6b7402a930 tx=0x7f6b74004270 comp rx=0 tx=0).stop 2024-09-10T08:45:40.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:40.000+0000 7f6b917b4640 1 -- 172.21.15.172:0/140100382 shutdown_connections 2024-09-10T08:45:40.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:40.000+0000 7f6b917b4640 1 --2- 172.21.15.172:0/140100382 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6b780a2c90 0x7f6b780a5150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:40.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:40.000+0000 7f6b917b4640 1 --2- 172.21.15.172:0/140100382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b8c106f70 0x7f6b8c101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:40.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:40.000+0000 7f6b917b4640 1 --2- 172.21.15.172:0/140100382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b8c100eb0 0x7f6b8c1019c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:40.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:40.000+0000 7f6b917b4640 1 --2- 172.21.15.172:0/140100382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b8c100500 0x7f6b8c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:40.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:40.000+0000 7f6b917b4640 1 -- 172.21.15.172:0/140100382 >> 172.21.15.172:0/140100382 conn(0x7f6b8c0fc5c0 msgr2=0x7f6b8c1053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:40.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:40.000+0000 7f6b917b4640 1 -- 172.21.15.172:0/140100382 shutdown_connections 2024-09-10T08:45:40.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:40.000+0000 7f6b917b4640 1 -- 172.21.15.172:0/140100382 wait complete. 2024-09-10T08:45:40.544 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:40 smithi172 ceph-mon[30854]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-10T08:45:40.544 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:40 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/140100382' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T08:45:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:40 smithi191 ceph-mon[35314]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-10T08:45:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:40 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/140100382' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T08:45:40.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:40 smithi174 ceph-mon[34068]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-10T08:45:40.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:40 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/140100382' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T08:45:40.846 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182467 got 107374182471 for osd.3 2024-09-10T08:45:40.847 DEBUG:teuthology.parallel:result is None 2024-09-10T08:45:41.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.633+0000 7f7337b7b640 1 -- 172.21.15.172:0/2018461881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7330102ef0 msgr2=0x7f733010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:41.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.633+0000 7f7337b7b640 1 --2- 172.21.15.172:0/2018461881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7330102ef0 0x7f733010da40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7328009900 tx=0x7f732802a480 comp rx=0 tx=0).stop 2024-09-10T08:45:41.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.633+0000 7f7337b7b640 1 -- 172.21.15.172:0/2018461881 shutdown_connections 2024-09-10T08:45:41.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.633+0000 7f7337b7b640 1 --2- 172.21.15.172:0/2018461881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7330102ef0 0x7f733010da40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.633+0000 7f7337b7b640 1 --2- 172.21.15.172:0/2018461881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7330102570 0x7f73301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.633+0000 7f7337b7b640 1 --2- 172.21.15.172:0/2018461881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7330100e20 0x7f7330101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.633+0000 7f7337b7b640 1 -- 172.21.15.172:0/2018461881 >> 172.21.15.172:0/2018461881 conn(0x7f73300fc7d0 msgr2=0x7f73300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:41.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.633+0000 7f7337b7b640 1 -- 172.21.15.172:0/2018461881 shutdown_connections 2024-09-10T08:45:41.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.633+0000 7f7337b7b640 1 -- 172.21.15.172:0/2018461881 wait complete. 2024-09-10T08:45:41.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.633+0000 7f7337b7b640 1 Processor -- start 2024-09-10T08:45:41.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.633+0000 7f7337b7b640 1 -- start start 2024-09-10T08:45:41.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.633+0000 7f7337b7b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7330100e20 0x7f73301983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:41.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7337b7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7330102570 0x7f7330198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:41.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7337b7b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7330102ef0 0x7f733019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:41.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7337b7b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7330110e30 con 0x7f7330102570 2024-09-10T08:45:41.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7337b7b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7330110cb0 con 0x7f7330102ef0 2024-09-10T08:45:41.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7337b7b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7330110fb0 con 0x7f7330100e20 2024-09-10T08:45:41.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7335116640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7330102570 0x7f7330198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:41.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7335116640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7330102570 0x7f7330198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43000/0 (socket says 172.21.15.172:43000) 2024-09-10T08:45:41.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7335116640 1 -- 172.21.15.172:0/2905380686 learned_addr learned my addr 172.21.15.172:0/2905380686 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:41.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7335917640 1 --2- 172.21.15.172:0/2905380686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7330100e20 0x7f73301983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:41.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7336118640 1 --2- 172.21.15.172:0/2905380686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7330102ef0 0x7f733019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:41.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7335116640 1 -- 172.21.15.172:0/2905380686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7330100e20 msgr2=0x7f73301983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:41.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7335116640 1 --2- 172.21.15.172:0/2905380686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7330100e20 0x7f73301983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7335116640 1 -- 172.21.15.172:0/2905380686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7330102ef0 msgr2=0x7f733019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:41.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7335116640 1 --2- 172.21.15.172:0/2905380686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7330102ef0 0x7f733019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7335116640 1 -- 172.21.15.172:0/2905380686 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733019d370 con 0x7f7330102570 2024-09-10T08:45:41.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7336118640 1 --2- 172.21.15.172:0/2905380686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7330102ef0 0x7f733019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:41.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7335116640 1 --2- 172.21.15.172:0/2905380686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7330102570 0x7f7330198900 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f732400c890 tx=0x7f732400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:41.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f732effd640 1 -- 172.21.15.172:0/2905380686 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324004220 con 0x7f7330102570 2024-09-10T08:45:41.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f732effd640 1 -- 172.21.15.172:0/2905380686 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73240043c0 con 0x7f7330102570 2024-09-10T08:45:41.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7337b7b640 1 -- 172.21.15.172:0/2905380686 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733019d660 con 0x7f7330102570 2024-09-10T08:45:41.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f732effd640 1 -- 172.21.15.172:0/2905380686 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324004a00 con 0x7f7330102570 2024-09-10T08:45:41.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.634+0000 7f7337b7b640 1 -- 172.21.15.172:0/2905380686 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73301a4f40 con 0x7f7330102570 2024-09-10T08:45:41.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.636+0000 7f7337b7b640 1 -- 172.21.15.172:0/2905380686 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7330101200 con 0x7f7330102570 2024-09-10T08:45:41.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.637+0000 7f732effd640 1 -- 172.21.15.172:0/2905380686 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7324004ba0 con 0x7f7330102570 2024-09-10T08:45:41.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.637+0000 7f732effd640 1 --2- 172.21.15.172:0/2905380686 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f73180a2c90 0x7f73180a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:41.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.637+0000 7f7335917640 1 --2- 172.21.15.172:0/2905380686 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f73180a2c90 0x7f73180a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:41.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.637+0000 7f732effd640 1 -- 172.21.15.172:0/2905380686 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f73240cf150 con 0x7f7330102570 2024-09-10T08:45:41.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.638+0000 7f7335917640 1 --2- 172.21.15.172:0/2905380686 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f73180a2c90 0x7f73180a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7320005fd0 tx=0x7f7320009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:41.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.641+0000 7f732effd640 1 -- 172.21.15.172:0/2905380686 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7324014030 con 0x7f7330102570 2024-09-10T08:45:41.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.279+0000 7fee7c544640 1 -- 172.21.15.172:0/4204005527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee74102790 msgr2=0x7fee741067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:41.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.279+0000 7fee7c544640 1 --2- 172.21.15.172:0/4204005527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee74102790 0x7fee741067e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fee64009900 tx=0x7fee6402a410 comp rx=0 tx=0).stop 2024-09-10T08:45:41.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.279+0000 7fee7c544640 1 -- 172.21.15.172:0/4204005527 shutdown_connections 2024-09-10T08:45:41.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.279+0000 7fee7c544640 1 --2- 172.21.15.172:0/4204005527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee74106f10 0x7fee7410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.279+0000 7fee7c544640 1 --2- 172.21.15.172:0/4204005527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee74102790 0x7fee741067e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.279+0000 7fee7c544640 1 --2- 172.21.15.172:0/4204005527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee74101de0 0x7fee741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.279+0000 7fee7c544640 1 -- 172.21.15.172:0/4204005527 >> 172.21.15.172:0/4204005527 conn(0x7fee740fd8c0 msgr2=0x7fee740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:41.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.279+0000 7fee7c544640 1 -- 172.21.15.172:0/4204005527 shutdown_connections 2024-09-10T08:45:41.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.279+0000 7fee7c544640 1 -- 172.21.15.172:0/4204005527 wait complete. 2024-09-10T08:45:41.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.279+0000 7fee7c544640 1 Processor -- start 2024-09-10T08:45:41.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee7c544640 1 -- start start 2024-09-10T08:45:41.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee7c544640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee74101de0 0x7fee74198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:41.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee7c544640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee74102790 0x7fee741988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:41.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee7c544640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee74106f10 0x7fee7419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:41.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee7c544640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fee74111e40 con 0x7fee74102790 2024-09-10T08:45:41.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee7c544640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee74111cc0 con 0x7fee74106f10 2024-09-10T08:45:41.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee7c544640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fee74111fc0 con 0x7fee74101de0 2024-09-10T08:45:41.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee79adf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee74102790 0x7fee741988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:41.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee79adf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee74102790 0x7fee741988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42974/0 (socket says 172.21.15.172:42974) 2024-09-10T08:45:41.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee79adf640 1 -- 172.21.15.172:0/868561845 learned_addr learned my addr 172.21.15.172:0/868561845 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:41.723 INFO:teuthology.orchestra.run.smithi172.stdout:214748364851 2024-09-10T08:45:41.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee7a2e0640 1 --2- 172.21.15.172:0/868561845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee74101de0 0x7fee74198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:41.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee7aae1640 1 --2- 172.21.15.172:0/868561845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee74106f10 0x7fee7419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:41.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee79adf640 1 -- 172.21.15.172:0/868561845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee74101de0 msgr2=0x7fee74198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:41.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee79adf640 1 --2- 172.21.15.172:0/868561845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee74101de0 0x7fee74198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee79adf640 1 -- 172.21.15.172:0/868561845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee74106f10 msgr2=0x7fee7419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:41.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee79adf640 1 --2- 172.21.15.172:0/868561845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee74106f10 0x7fee7419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee79adf640 1 -- 172.21.15.172:0/868561845 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7419d330 con 0x7fee74102790 2024-09-10T08:45:41.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee7aae1640 1 --2- 172.21.15.172:0/868561845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee74106f10 0x7fee7419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:41.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.280+0000 7fee79adf640 1 --2- 172.21.15.172:0/868561845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee74102790 0x7fee741988c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fee64006010 tx=0x7fee64004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:41.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.281+0000 7fee6b7fe640 1 -- 172.21.15.172:0/868561845 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6403f030 con 0x7fee74102790 2024-09-10T08:45:41.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.281+0000 7fee6b7fe640 1 -- 172.21.15.172:0/868561845 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee640035f0 con 0x7fee74102790 2024-09-10T08:45:41.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.281+0000 7fee7c544640 1 -- 172.21.15.172:0/868561845 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7419d5c0 con 0x7fee74102790 2024-09-10T08:45:41.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.281+0000 7fee6b7fe640 1 -- 172.21.15.172:0/868561845 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64035070 con 0x7fee74102790 2024-09-10T08:45:41.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.281+0000 7fee7c544640 1 -- 172.21.15.172:0/868561845 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee741a4f00 con 0x7fee74102790 2024-09-10T08:45:41.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.282+0000 7fee7c544640 1 -- 172.21.15.172:0/868561845 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee741021c0 con 0x7fee74102790 2024-09-10T08:45:41.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.283+0000 7fee6b7fe640 1 -- 172.21.15.172:0/868561845 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fee64003b10 con 0x7fee74102790 2024-09-10T08:45:41.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.283+0000 7fee6b7fe640 1 --2- 172.21.15.172:0/868561845 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee600a2c10 0x7fee600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:41.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.283+0000 7fee7a2e0640 1 --2- 172.21.15.172:0/868561845 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee600a2c10 0x7fee600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:41.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.283+0000 7fee6b7fe640 1 -- 172.21.15.172:0/868561845 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fee640efe80 con 0x7fee74102790 2024-09-10T08:45:41.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.284+0000 7fee7a2e0640 1 --2- 172.21.15.172:0/868561845 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee600a2c10 0x7fee600a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fee70009990 tx=0x7fee70009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:41.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.288+0000 7fee6b7fe640 1 -- 172.21.15.172:0/868561845 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee640bc940 con 0x7fee74102790 2024-09-10T08:45:41.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.463+0000 7fee7c544640 1 -- 172.21.15.172:0/868561845 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fee7410a0f0 con 0x7fee74102790 2024-09-10T08:45:41.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.464+0000 7fee6b7fe640 1 -- 172.21.15.172:0/868561845 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fee640bc240 con 0x7fee74102790 2024-09-10T08:45:41.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.466+0000 7fee7c544640 1 -- 172.21.15.172:0/868561845 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee600a2c10 msgr2=0x7fee600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:41.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.466+0000 7fee7c544640 1 --2- 172.21.15.172:0/868561845 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee600a2c10 0x7fee600a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fee70009990 tx=0x7fee70009110 comp rx=0 tx=0).stop 2024-09-10T08:45:41.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.466+0000 7fee7c544640 1 -- 172.21.15.172:0/868561845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee74102790 msgr2=0x7fee741988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:41.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.466+0000 7fee7c544640 1 --2- 172.21.15.172:0/868561845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee74102790 0x7fee741988c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fee64006010 tx=0x7fee64004270 comp rx=0 tx=0).stop 2024-09-10T08:45:41.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.466+0000 7fee7c544640 1 -- 172.21.15.172:0/868561845 shutdown_connections 2024-09-10T08:45:41.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.466+0000 7fee7c544640 1 --2- 172.21.15.172:0/868561845 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee600a2c10 0x7fee600a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.466+0000 7fee7c544640 1 --2- 172.21.15.172:0/868561845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee74106f10 0x7fee7419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.466+0000 7fee7c544640 1 --2- 172.21.15.172:0/868561845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee74102790 0x7fee741988c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.466+0000 7fee7c544640 1 --2- 172.21.15.172:0/868561845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee74101de0 0x7fee74198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.466+0000 7fee7c544640 1 -- 172.21.15.172:0/868561845 >> 172.21.15.172:0/868561845 conn(0x7fee740fd8c0 msgr2=0x7fee740fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:41.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.466+0000 7fee7c544640 1 -- 172.21.15.172:0/868561845 shutdown_connections 2024-09-10T08:45:41.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.466+0000 7fee7c544640 1 -- 172.21.15.172:0/868561845 wait complete. 2024-09-10T08:45:41.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.812+0000 7f7337b7b640 1 -- 172.21.15.172:0/2905380686 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f73301a51d0 con 0x7f7330102570 2024-09-10T08:45:41.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.812+0000 7f732effd640 1 -- 172.21.15.172:0/2905380686 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f732409bc10 con 0x7f7330102570 2024-09-10T08:45:41.813 INFO:teuthology.orchestra.run.smithi172.stdout:141733920835 2024-09-10T08:45:41.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.815+0000 7f7337b7b640 1 -- 172.21.15.172:0/2905380686 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f73180a2c90 msgr2=0x7f73180a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:41.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.815+0000 7f7337b7b640 1 --2- 172.21.15.172:0/2905380686 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f73180a2c90 0x7f73180a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7320005fd0 tx=0x7f7320009680 comp rx=0 tx=0).stop 2024-09-10T08:45:41.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.815+0000 7f7337b7b640 1 -- 172.21.15.172:0/2905380686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7330102570 msgr2=0x7f7330198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:41.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.815+0000 7f7337b7b640 1 --2- 172.21.15.172:0/2905380686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7330102570 0x7f7330198900 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f732400c890 tx=0x7f732400cd50 comp rx=0 tx=0).stop 2024-09-10T08:45:41.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.815+0000 7f7337b7b640 1 -- 172.21.15.172:0/2905380686 shutdown_connections 2024-09-10T08:45:41.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.815+0000 7f7337b7b640 1 --2- 172.21.15.172:0/2905380686 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f73180a2c90 0x7f73180a5150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.815+0000 7f7337b7b640 1 --2- 172.21.15.172:0/2905380686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7330102ef0 0x7f733019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.815+0000 7f7337b7b640 1 --2- 172.21.15.172:0/2905380686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7330102570 0x7f7330198900 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.815+0000 7f7337b7b640 1 --2- 172.21.15.172:0/2905380686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7330100e20 0x7f73301983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:41.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.815+0000 7f7337b7b640 1 -- 172.21.15.172:0/2905380686 >> 172.21.15.172:0/2905380686 conn(0x7f73300fc7d0 msgr2=0x7f7330069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:41.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.815+0000 7f7337b7b640 1 -- 172.21.15.172:0/2905380686 shutdown_connections 2024-09-10T08:45:41.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:41.815+0000 7f7337b7b640 1 -- 172.21.15.172:0/2905380686 wait complete. 2024-09-10T08:45:42.511 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:42 smithi172 ceph-mon[30854]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-09-10T08:45:42.512 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:42 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/868561845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T08:45:42.512 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:42 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2905380686' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T08:45:42.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:42 smithi191 ceph-mon[35314]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-09-10T08:45:42.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:42 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/868561845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T08:45:42.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:42 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2905380686' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T08:45:42.667 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920831 got 141733920835 for osd.4 2024-09-10T08:45:42.668 DEBUG:teuthology.parallel:result is None 2024-09-10T08:45:42.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:42 smithi174 ceph-mon[34068]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 10 op/s 2024-09-10T08:45:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:42 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/868561845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T08:45:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:42 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2905380686' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T08:45:42.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.863+0000 7fc59f6f4640 1 -- 172.21.15.172:0/3711988741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5981036e0 msgr2=0x7fc598107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:42.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.863+0000 7fc59f6f4640 1 --2- 172.21.15.172:0/3711988741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5981036e0 0x7fc598107b90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc594009930 tx=0x7fc59402a420 comp rx=0 tx=0).stop 2024-09-10T08:45:42.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.863+0000 7fc59f6f4640 1 -- 172.21.15.172:0/3711988741 shutdown_connections 2024-09-10T08:45:42.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.863+0000 7fc59f6f4640 1 --2- 172.21.15.172:0/3711988741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5981082a0 0x7fc59810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:42.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.863+0000 7fc59f6f4640 1 --2- 172.21.15.172:0/3711988741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5981036e0 0x7fc598107b90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:42.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.863+0000 7fc59f6f4640 1 --2- 172.21.15.172:0/3711988741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc598102d30 0x7fc598103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:42.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.863+0000 7fc59f6f4640 1 -- 172.21.15.172:0/3711988741 >> 172.21.15.172:0/3711988741 conn(0x7fc5980fc7d0 msgr2=0x7fc5980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:42.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.863+0000 7fc59f6f4640 1 -- 172.21.15.172:0/3711988741 shutdown_connections 2024-09-10T08:45:42.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.863+0000 7fc59f6f4640 1 -- 172.21.15.172:0/3711988741 wait complete. 2024-09-10T08:45:42.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.864+0000 7fc59f6f4640 1 Processor -- start 2024-09-10T08:45:42.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.864+0000 7fc59f6f4640 1 -- start start 2024-09-10T08:45:42.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.864+0000 7fc59f6f4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc598102d30 0x7fc598198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:42.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.864+0000 7fc59f6f4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5981036e0 0x7fc598198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:42.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.864+0000 7fc59f6f4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5981082a0 0x7fc59819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:42.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.864+0000 7fc59f6f4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc598111f50 con 0x7fc5981036e0 2024-09-10T08:45:42.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.864+0000 7fc59f6f4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc598111dd0 con 0x7fc5981082a0 2024-09-10T08:45:42.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.864+0000 7fc59f6f4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc5981120d0 con 0x7fc598102d30 2024-09-10T08:45:42.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.864+0000 7fc59cc8f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5981036e0 0x7fc598198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:42.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.864+0000 7fc59cc8f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5981036e0 0x7fc598198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43038/0 (socket says 172.21.15.172:43038) 2024-09-10T08:45:42.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.864+0000 7fc59cc8f640 1 -- 172.21.15.172:0/153733038 learned_addr learned my addr 172.21.15.172:0/153733038 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:42.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.865+0000 7fc59d490640 1 --2- 172.21.15.172:0/153733038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc598102d30 0x7fc598198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:42.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.865+0000 7fc59dc91640 1 --2- 172.21.15.172:0/153733038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5981082a0 0x7fc59819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:42.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.865+0000 7fc59cc8f640 1 -- 172.21.15.172:0/153733038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc598102d30 msgr2=0x7fc598198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:42.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.865+0000 7fc59cc8f640 1 --2- 172.21.15.172:0/153733038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc598102d30 0x7fc598198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:42.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.865+0000 7fc59cc8f640 1 -- 172.21.15.172:0/153733038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5981082a0 msgr2=0x7fc59819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:42.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.865+0000 7fc59cc8f640 1 --2- 172.21.15.172:0/153733038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5981082a0 0x7fc59819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:42.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.865+0000 7fc59cc8f640 1 -- 172.21.15.172:0/153733038 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59819d4c0 con 0x7fc5981036e0 2024-09-10T08:45:42.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.865+0000 7fc59dc91640 1 --2- 172.21.15.172:0/153733038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5981082a0 0x7fc59819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:42.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.865+0000 7fc59cc8f640 1 --2- 172.21.15.172:0/153733038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5981036e0 0x7fc598198a50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc594031440 tx=0x7fc594004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:42.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.865+0000 7fc58e7fc640 1 -- 172.21.15.172:0/153733038 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59403f030 con 0x7fc5981036e0 2024-09-10T08:45:42.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.865+0000 7fc59f6f4640 1 -- 172.21.15.172:0/153733038 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59819d750 con 0x7fc5981036e0 2024-09-10T08:45:42.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.865+0000 7fc58e7fc640 1 -- 172.21.15.172:0/153733038 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc594003aa0 con 0x7fc5981036e0 2024-09-10T08:45:42.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.865+0000 7fc58e7fc640 1 -- 172.21.15.172:0/153733038 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594035070 con 0x7fc5981036e0 2024-09-10T08:45:42.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.865+0000 7fc59f6f4640 1 -- 172.21.15.172:0/153733038 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5981a5090 con 0x7fc5981036e0 2024-09-10T08:45:42.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.866+0000 7fc59f6f4640 1 -- 172.21.15.172:0/153733038 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc598103110 con 0x7fc5981036e0 2024-09-10T08:45:42.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.867+0000 7fc58e7fc640 1 -- 172.21.15.172:0/153733038 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc594042030 con 0x7fc5981036e0 2024-09-10T08:45:42.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.868+0000 7fc58e7fc640 1 --2- 172.21.15.172:0/153733038 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5840a2c10 0x7fc5840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:42.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.868+0000 7fc59d490640 1 --2- 172.21.15.172:0/153733038 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5840a2c10 0x7fc5840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:42.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.868+0000 7fc58e7fc640 1 -- 172.21.15.172:0/153733038 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fc5940ef900 con 0x7fc5981036e0 2024-09-10T08:45:42.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.868+0000 7fc59d490640 1 --2- 172.21.15.172:0/153733038 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5840a2c10 0x7fc5840a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc588005fd0 tx=0x7fc58800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:42.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.876+0000 7fc58e7fc640 1 -- 172.21.15.172:0/153733038 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5940bc3c0 con 0x7fc5981036e0 2024-09-10T08:45:42.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.344+0000 7f2798866640 1 -- 172.21.15.172:0/3511781365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2794108300 msgr2=0x7f279410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:42.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.344+0000 7f2798866640 1 --2- 172.21.15.172:0/3511781365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2794108300 0x7f279410ec70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2784009cd0 tx=0x7f278402a8d0 comp rx=0 tx=0).stop 2024-09-10T08:45:42.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.344+0000 7f2798866640 1 -- 172.21.15.172:0/3511781365 shutdown_connections 2024-09-10T08:45:42.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.344+0000 7f2798866640 1 --2- 172.21.15.172:0/3511781365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2794108300 0x7f279410ec70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:42.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.344+0000 7f2798866640 1 --2- 172.21.15.172:0/3511781365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2794103780 0x7f2794107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:42.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.344+0000 7f2798866640 1 --2- 172.21.15.172:0/3511781365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2794102dd0 0x7f27941031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:42.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.344+0000 7f2798866640 1 -- 172.21.15.172:0/3511781365 >> 172.21.15.172:0/3511781365 conn(0x7f27940fc7d0 msgr2=0x7f27940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:42.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.344+0000 7f2798866640 1 -- 172.21.15.172:0/3511781365 shutdown_connections 2024-09-10T08:45:42.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.344+0000 7f2798866640 1 -- 172.21.15.172:0/3511781365 wait complete. 2024-09-10T08:45:42.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.344+0000 7f2798866640 1 Processor -- start 2024-09-10T08:45:42.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f2798866640 1 -- start start 2024-09-10T08:45:42.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f2798866640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2794102dd0 0x7f2794198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:42.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f2798866640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2794103780 0x7f2794198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:42.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f2798866640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2794108300 0x7f279419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:42.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f2798866640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2794111f80 con 0x7f2794103780 2024-09-10T08:45:42.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f2798866640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2794111e00 con 0x7f2794102dd0 2024-09-10T08:45:42.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f2798866640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2794112100 con 0x7f2794108300 2024-09-10T08:45:42.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f2791d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2794103780 0x7f2794198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:42.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f2791d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2794103780 0x7f2794198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43016/0 (socket says 172.21.15.172:43016) 2024-09-10T08:45:42.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f2791d9b640 1 -- 172.21.15.172:0/1067763306 learned_addr learned my addr 172.21.15.172:0/1067763306 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:42.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f2792d9d640 1 --2- 172.21.15.172:0/1067763306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2794108300 0x7f279419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:42.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f279259c640 1 --2- 172.21.15.172:0/1067763306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2794102dd0 0x7f2794198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:42.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f2791d9b640 1 -- 172.21.15.172:0/1067763306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2794108300 msgr2=0x7f279419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:42.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f2791d9b640 1 --2- 172.21.15.172:0/1067763306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2794108300 0x7f279419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:42.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f2791d9b640 1 -- 172.21.15.172:0/1067763306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2794102dd0 msgr2=0x7f2794198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:42.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f2791d9b640 1 --2- 172.21.15.172:0/1067763306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2794102dd0 0x7f2794198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:42.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f2791d9b640 1 -- 172.21.15.172:0/1067763306 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279419d3d0 con 0x7f2794103780 2024-09-10T08:45:42.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f279259c640 1 --2- 172.21.15.172:0/1067763306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2794102dd0 0x7f2794198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:42.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f2791d9b640 1 --2- 172.21.15.172:0/1067763306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2794103780 0x7f2794198960 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f277c00c8c0 tx=0x7f277c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:42.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.345+0000 7f278b7fe640 1 -- 172.21.15.172:0/1067763306 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c007bb0 con 0x7f2794103780 2024-09-10T08:45:42.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.346+0000 7f278b7fe640 1 -- 172.21.15.172:0/1067763306 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f277c007d50 con 0x7f2794103780 2024-09-10T08:45:42.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.346+0000 7f2798866640 1 -- 172.21.15.172:0/1067763306 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279419d6c0 con 0x7f2794103780 2024-09-10T08:45:42.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.346+0000 7f278b7fe640 1 -- 172.21.15.172:0/1067763306 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c0187c0 con 0x7f2794103780 2024-09-10T08:45:42.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.346+0000 7f2798866640 1 -- 172.21.15.172:0/1067763306 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27941a4fa0 con 0x7f2794103780 2024-09-10T08:45:42.984 INFO:teuthology.orchestra.run.smithi172.stdout:326417514528 2024-09-10T08:45:42.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.347+0000 7f2798866640 1 -- 172.21.15.172:0/1067763306 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27941031b0 con 0x7f2794103780 2024-09-10T08:45:42.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.348+0000 7f278b7fe640 1 -- 172.21.15.172:0/1067763306 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f277c00f450 con 0x7f2794103780 2024-09-10T08:45:42.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.348+0000 7f278b7fe640 1 --2- 172.21.15.172:0/1067763306 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f27800a2c10 0x7f27800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:42.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.348+0000 7f278b7fe640 1 -- 172.21.15.172:0/1067763306 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f277c0cf570 con 0x7f2794103780 2024-09-10T08:45:42.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.348+0000 7f279259c640 1 --2- 172.21.15.172:0/1067763306 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f27800a2c10 0x7f27800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:42.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.349+0000 7f279259c640 1 --2- 172.21.15.172:0/1067763306 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f27800a2c10 0x7f27800a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f278c009b40 tx=0x7f278c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:42.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.352+0000 7f278b7fe640 1 -- 172.21.15.172:0/1067763306 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f277c014030 con 0x7f2794103780 2024-09-10T08:45:42.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.526+0000 7f2798866640 1 -- 172.21.15.172:0/1067763306 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f279410a290 con 0x7f2794103780 2024-09-10T08:45:42.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.526+0000 7f278b7fe640 1 -- 172.21.15.172:0/1067763306 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f277c09c030 con 0x7f2794103780 2024-09-10T08:45:42.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.528+0000 7f2798866640 1 -- 172.21.15.172:0/1067763306 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f27800a2c10 msgr2=0x7f27800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:42.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.528+0000 7f2798866640 1 --2- 172.21.15.172:0/1067763306 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f27800a2c10 0x7f27800a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f278c009b40 tx=0x7f278c005e50 comp rx=0 tx=0).stop 2024-09-10T08:45:42.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.528+0000 7f2798866640 1 -- 172.21.15.172:0/1067763306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2794103780 msgr2=0x7f2794198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:42.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.528+0000 7f2798866640 1 --2- 172.21.15.172:0/1067763306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2794103780 0x7f2794198960 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f277c00c8c0 tx=0x7f277c00cd80 comp rx=0 tx=0).stop 2024-09-10T08:45:42.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.528+0000 7f2798866640 1 -- 172.21.15.172:0/1067763306 shutdown_connections 2024-09-10T08:45:42.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.528+0000 7f2798866640 1 --2- 172.21.15.172:0/1067763306 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f27800a2c10 0x7f27800a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:42.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.528+0000 7f2798866640 1 --2- 172.21.15.172:0/1067763306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2794108300 0x7f279419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:42.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.528+0000 7f2798866640 1 --2- 172.21.15.172:0/1067763306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2794103780 0x7f2794198960 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:42.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.528+0000 7f2798866640 1 --2- 172.21.15.172:0/1067763306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2794102dd0 0x7f2794198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:42.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.528+0000 7f2798866640 1 -- 172.21.15.172:0/1067763306 >> 172.21.15.172:0/1067763306 conn(0x7f27940fc7d0 msgr2=0x7f2794105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:42.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.528+0000 7f2798866640 1 -- 172.21.15.172:0/1067763306 shutdown_connections 2024-09-10T08:45:42.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:42.529+0000 7f2798866640 1 -- 172.21.15.172:0/1067763306 wait complete. 2024-09-10T08:45:43.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:43.052+0000 7fc59f6f4640 1 -- 172.21.15.172:0/153733038 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fc59810a2b0 con 0x7fc5981036e0 2024-09-10T08:45:43.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:43.052+0000 7fc58e7fc640 1 -- 172.21.15.172:0/153733038 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc5940bbcc0 con 0x7fc5981036e0 2024-09-10T08:45:43.053 INFO:teuthology.orchestra.run.smithi172.stdout:193273528377 2024-09-10T08:45:43.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:43.054+0000 7fc59f6f4640 1 -- 172.21.15.172:0/153733038 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5840a2c10 msgr2=0x7fc5840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:43.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:43.054+0000 7fc59f6f4640 1 --2- 172.21.15.172:0/153733038 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5840a2c10 0x7fc5840a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc588005fd0 tx=0x7fc58800b390 comp rx=0 tx=0).stop 2024-09-10T08:45:43.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:43.054+0000 7fc59f6f4640 1 -- 172.21.15.172:0/153733038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5981036e0 msgr2=0x7fc598198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:43.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:43.054+0000 7fc59f6f4640 1 --2- 172.21.15.172:0/153733038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5981036e0 0x7fc598198a50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc594031440 tx=0x7fc594004270 comp rx=0 tx=0).stop 2024-09-10T08:45:43.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:43.054+0000 7fc59f6f4640 1 -- 172.21.15.172:0/153733038 shutdown_connections 2024-09-10T08:45:43.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:43.054+0000 7fc59f6f4640 1 --2- 172.21.15.172:0/153733038 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5840a2c10 0x7fc5840a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:43.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:43.054+0000 7fc59f6f4640 1 --2- 172.21.15.172:0/153733038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5981082a0 0x7fc59819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:43.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:43.054+0000 7fc59f6f4640 1 --2- 172.21.15.172:0/153733038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5981036e0 0x7fc598198a50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:43.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:43.054+0000 7fc59f6f4640 1 --2- 172.21.15.172:0/153733038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc598102d30 0x7fc598198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:43.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:43.054+0000 7fc59f6f4640 1 -- 172.21.15.172:0/153733038 >> 172.21.15.172:0/153733038 conn(0x7fc5980fc7d0 msgr2=0x7fc598105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:43.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:43.055+0000 7fc59f6f4640 1 -- 172.21.15.172:0/153733038 shutdown_connections 2024-09-10T08:45:43.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:43.055+0000 7fc59f6f4640 1 -- 172.21.15.172:0/153733038 wait complete. 2024-09-10T08:45:43.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:43 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1067763306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T08:45:43.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:43 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/153733038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T08:45:43.622 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514525 got 326417514528 for osd.11 2024-09-10T08:45:43.623 DEBUG:teuthology.parallel:result is None 2024-09-10T08:45:43.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:43 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1067763306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T08:45:43.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:43 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/153733038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T08:45:43.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:43 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1067763306' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T08:45:43.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:43 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/153733038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T08:45:44.075 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364848 got 214748364851 for osd.7 2024-09-10T08:45:44.075 DEBUG:teuthology.parallel:result is None 2024-09-10T08:45:44.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:44 smithi172 ceph-mon[30854]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:44.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:44 smithi174 ceph-mon[34068]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:44.789 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528374 got 193273528377 for osd.6 2024-09-10T08:45:44.790 DEBUG:teuthology.parallel:result is None 2024-09-10T08:45:44.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:44 smithi191 ceph-mon[35314]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:45.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.325+0000 7fa115216640 1 -- 172.21.15.172:0/4085391131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa110100a80 msgr2=0x7fa110100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:45.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.325+0000 7fa115216640 1 --2- 172.21.15.172:0/4085391131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa110100a80 0x7fa110100ee0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa104009930 tx=0x7fa10402a420 comp rx=0 tx=0).stop 2024-09-10T08:45:45.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.325+0000 7fa115216640 1 -- 172.21.15.172:0/4085391131 shutdown_connections 2024-09-10T08:45:45.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.325+0000 7fa115216640 1 --2- 172.21.15.172:0/4085391131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa110101420 0x7fa11010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:45.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.325+0000 7fa115216640 1 --2- 172.21.15.172:0/4085391131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa110100a80 0x7fa110100ee0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:45.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.325+0000 7fa115216640 1 --2- 172.21.15.172:0/4085391131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa110106a80 0x7fa110106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:45.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.325+0000 7fa115216640 1 -- 172.21.15.172:0/4085391131 >> 172.21.15.172:0/4085391131 conn(0x7fa1100fc7a0 msgr2=0x7fa1100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:45.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.325+0000 7fa115216640 1 -- 172.21.15.172:0/4085391131 shutdown_connections 2024-09-10T08:45:45.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.325+0000 7fa115216640 1 -- 172.21.15.172:0/4085391131 wait complete. 2024-09-10T08:45:45.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa115216640 1 Processor -- start 2024-09-10T08:45:45.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa115216640 1 -- start start 2024-09-10T08:45:45.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa115216640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa110100a80 0x7fa1101983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:45.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa115216640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa110101420 0x7fa110198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:45.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa115216640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa110106a80 0x7fa11019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:45.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa115216640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa110110da0 con 0x7fa110106a80 2024-09-10T08:45:45.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa115216640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa110110c20 con 0x7fa110100a80 2024-09-10T08:45:45.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa115216640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa110110f20 con 0x7fa110101420 2024-09-10T08:45:45.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa10f59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa110106a80 0x7fa11019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:45.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa10f59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa110106a80 0x7fa11019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43060/0 (socket says 172.21.15.172:43060) 2024-09-10T08:45:45.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa10f59e640 1 -- 172.21.15.172:0/467809293 learned_addr learned my addr 172.21.15.172:0/467809293 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:45.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa10e59c640 1 --2- 172.21.15.172:0/467809293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa110101420 0x7fa110198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:45.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa10ed9d640 1 --2- 172.21.15.172:0/467809293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa110100a80 0x7fa1101983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:45.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa10f59e640 1 -- 172.21.15.172:0/467809293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa110101420 msgr2=0x7fa110198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:45.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa10f59e640 1 --2- 172.21.15.172:0/467809293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa110101420 0x7fa110198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:45.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa10f59e640 1 -- 172.21.15.172:0/467809293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa110100a80 msgr2=0x7fa1101983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:45.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa10f59e640 1 --2- 172.21.15.172:0/467809293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa110100a80 0x7fa1101983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:45.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa10f59e640 1 -- 172.21.15.172:0/467809293 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11019d390 con 0x7fa110106a80 2024-09-10T08:45:45.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.326+0000 7fa10ed9d640 1 --2- 172.21.15.172:0/467809293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa110100a80 0x7fa1101983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:45.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.327+0000 7fa10f59e640 1 --2- 172.21.15.172:0/467809293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa110106a80 0x7fa11019ccb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa10800b970 tx=0x7fa10800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:45.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.327+0000 7fa0f7fff640 1 -- 172.21.15.172:0/467809293 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108019070 con 0x7fa110106a80 2024-09-10T08:45:45.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.327+0000 7fa0f7fff640 1 -- 172.21.15.172:0/467809293 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa108014440 con 0x7fa110106a80 2024-09-10T08:45:45.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.327+0000 7fa115216640 1 -- 172.21.15.172:0/467809293 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa11019d680 con 0x7fa110106a80 2024-09-10T08:45:45.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.327+0000 7fa0f7fff640 1 -- 172.21.15.172:0/467809293 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1080075d0 con 0x7fa110106a80 2024-09-10T08:45:45.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.327+0000 7fa115216640 1 -- 172.21.15.172:0/467809293 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1101a4f60 con 0x7fa110106a80 2024-09-10T08:45:45.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.328+0000 7fa115216640 1 -- 172.21.15.172:0/467809293 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa110100ee0 con 0x7fa110106a80 2024-09-10T08:45:45.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.329+0000 7fa0f7fff640 1 -- 172.21.15.172:0/467809293 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa10801c070 con 0x7fa110106a80 2024-09-10T08:45:45.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.330+0000 7fa0f7fff640 1 --2- 172.21.15.172:0/467809293 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0f80a2c10 0x7fa0f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:45.354 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:45 smithi172 ceph-mon[30854]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:45.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.330+0000 7fa10ed9d640 1 --2- 172.21.15.172:0/467809293 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0f80a2c10 0x7fa0f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:45.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.330+0000 7fa0f7fff640 1 -- 172.21.15.172:0/467809293 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fa1080ce9d0 con 0x7fa110106a80 2024-09-10T08:45:45.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.330+0000 7fa10ed9d640 1 --2- 172.21.15.172:0/467809293 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0f80a2c10 0x7fa0f80a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa1000058c0 tx=0x7fa100005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:45.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.334+0000 7fa0f7fff640 1 -- 172.21.15.172:0/467809293 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa10809af50 con 0x7fa110106a80 2024-09-10T08:45:45.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.501+0000 7fa115216640 1 -- 172.21.15.172:0/467809293 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fa110109a90 con 0x7fa110106a80 2024-09-10T08:45:45.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.501+0000 7fa0f7fff640 1 -- 172.21.15.172:0/467809293 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa10809f250 con 0x7fa110106a80 2024-09-10T08:45:45.503 INFO:teuthology.orchestra.run.smithi172.stdout:244813135919 2024-09-10T08:45:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.504+0000 7fa115216640 1 -- 172.21.15.172:0/467809293 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0f80a2c10 msgr2=0x7fa0f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.504+0000 7fa115216640 1 --2- 172.21.15.172:0/467809293 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0f80a2c10 0x7fa0f80a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa1000058c0 tx=0x7fa100005ec0 comp rx=0 tx=0).stop 2024-09-10T08:45:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.504+0000 7fa115216640 1 -- 172.21.15.172:0/467809293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa110106a80 msgr2=0x7fa11019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.504+0000 7fa115216640 1 --2- 172.21.15.172:0/467809293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa110106a80 0x7fa11019ccb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa10800b970 tx=0x7fa10800be30 comp rx=0 tx=0).stop 2024-09-10T08:45:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.504+0000 7fa115216640 1 -- 172.21.15.172:0/467809293 shutdown_connections 2024-09-10T08:45:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.504+0000 7fa115216640 1 --2- 172.21.15.172:0/467809293 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0f80a2c10 0x7fa0f80a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.504+0000 7fa115216640 1 --2- 172.21.15.172:0/467809293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa110106a80 0x7fa11019ccb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.504+0000 7fa115216640 1 --2- 172.21.15.172:0/467809293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa110101420 0x7fa110198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.504+0000 7fa115216640 1 --2- 172.21.15.172:0/467809293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa110100a80 0x7fa1101983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.504+0000 7fa115216640 1 -- 172.21.15.172:0/467809293 >> 172.21.15.172:0/467809293 conn(0x7fa1100fc7a0 msgr2=0x7fa11006aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.504+0000 7fa115216640 1 -- 172.21.15.172:0/467809293 shutdown_connections 2024-09-10T08:45:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:45.504+0000 7fa115216640 1 -- 172.21.15.172:0/467809293 wait complete. 2024-09-10T08:45:45.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:45 smithi174 ceph-mon[34068]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:45.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:45 smithi191 ceph-mon[35314]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:46.253 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135915 got 244813135919 for osd.8 2024-09-10T08:45:46.253 DEBUG:teuthology.parallel:result is None 2024-09-10T08:45:46.253 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T08:45:46.253 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:45:46.508 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:46.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:46 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/467809293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T08:45:46.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:46 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/467809293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T08:45:46.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:46 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/467809293' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T08:45:47.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.104+0000 7efd75541640 1 -- 172.21.15.172:0/1199028328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd70101cf0 msgr2=0x7efd70105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:47.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.104+0000 7efd75541640 1 --2- 172.21.15.172:0/1199028328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd70101cf0 0x7efd70105730 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efd58009900 tx=0x7efd5802a410 comp rx=0 tx=0).stop 2024-09-10T08:45:47.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.104+0000 7efd75541640 1 -- 172.21.15.172:0/1199028328 shutdown_connections 2024-09-10T08:45:47.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.104+0000 7efd75541640 1 --2- 172.21.15.172:0/1199028328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd70105e60 0x7efd7010da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:47.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.104+0000 7efd75541640 1 --2- 172.21.15.172:0/1199028328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd70101cf0 0x7efd70105730 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:47.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.104+0000 7efd75541640 1 --2- 172.21.15.172:0/1199028328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd70101340 0x7efd70101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:47.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.104+0000 7efd75541640 1 -- 172.21.15.172:0/1199028328 >> 172.21.15.172:0/1199028328 conn(0x7efd700fbee0 msgr2=0x7efd700fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:47.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.104+0000 7efd75541640 1 -- 172.21.15.172:0/1199028328 shutdown_connections 2024-09-10T08:45:47.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.104+0000 7efd75541640 1 -- 172.21.15.172:0/1199028328 wait complete. 2024-09-10T08:45:47.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.105+0000 7efd75541640 1 Processor -- start 2024-09-10T08:45:47.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.105+0000 7efd75541640 1 -- start start 2024-09-10T08:45:47.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.105+0000 7efd75541640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd70101340 0x7efd701982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:47.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.105+0000 7efd75541640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd70101cf0 0x7efd701987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:47.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.105+0000 7efd75541640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd70105e60 0x7efd7019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:47.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.105+0000 7efd75541640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efd70110db0 con 0x7efd70105e60 2024-09-10T08:45:47.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.105+0000 7efd75541640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efd70110c30 con 0x7efd70101340 2024-09-10T08:45:47.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.105+0000 7efd75541640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efd70110f30 con 0x7efd70101cf0 2024-09-10T08:45:47.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.106+0000 7efd6f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd70105e60 0x7efd7019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:47.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.106+0000 7efd6f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd70105e60 0x7efd7019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52450/0 (socket says 172.21.15.172:52450) 2024-09-10T08:45:47.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.106+0000 7efd6f7fe640 1 -- 172.21.15.172:0/2885802826 learned_addr learned my addr 172.21.15.172:0/2885802826 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:47.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.106+0000 7efd6effd640 1 --2- 172.21.15.172:0/2885802826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd70101340 0x7efd701982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:47.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.106+0000 7efd667fc640 1 --2- 172.21.15.172:0/2885802826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd70101cf0 0x7efd701987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:47.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.106+0000 7efd6f7fe640 1 -- 172.21.15.172:0/2885802826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd70101cf0 msgr2=0x7efd701987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:47.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.106+0000 7efd6f7fe640 1 --2- 172.21.15.172:0/2885802826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd70101cf0 0x7efd701987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:47.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.106+0000 7efd6f7fe640 1 -- 172.21.15.172:0/2885802826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd70101340 msgr2=0x7efd701982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:47.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.106+0000 7efd6f7fe640 1 --2- 172.21.15.172:0/2885802826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd70101340 0x7efd701982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:47.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.106+0000 7efd6f7fe640 1 -- 172.21.15.172:0/2885802826 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7019d250 con 0x7efd70105e60 2024-09-10T08:45:47.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.106+0000 7efd6f7fe640 1 --2- 172.21.15.172:0/2885802826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd70105e60 0x7efd7019cb70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efd6800cce0 tx=0x7efd6800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:47.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.106+0000 7efd6cff9640 1 -- 172.21.15.172:0/2885802826 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6800f450 con 0x7efd70105e60 2024-09-10T08:45:47.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.106+0000 7efd6cff9640 1 -- 172.21.15.172:0/2885802826 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd6800faa0 con 0x7efd70105e60 2024-09-10T08:45:47.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.106+0000 7efd6cff9640 1 -- 172.21.15.172:0/2885802826 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68018b70 con 0x7efd70105e60 2024-09-10T08:45:47.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.106+0000 7efd75541640 1 -- 172.21.15.172:0/2885802826 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7019d540 con 0x7efd70105e60 2024-09-10T08:45:47.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.106+0000 7efd75541640 1 -- 172.21.15.172:0/2885802826 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd701a4e20 con 0x7efd70105e60 2024-09-10T08:45:47.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.107+0000 7efd75541640 1 -- 172.21.15.172:0/2885802826 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd70101720 con 0x7efd70105e60 2024-09-10T08:45:47.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.108+0000 7efd6cff9640 1 -- 172.21.15.172:0/2885802826 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7efd68018d10 con 0x7efd70105e60 2024-09-10T08:45:47.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.109+0000 7efd6cff9640 1 --2- 172.21.15.172:0/2885802826 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efd5c0a2c10 0x7efd5c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:47.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.109+0000 7efd6effd640 1 --2- 172.21.15.172:0/2885802826 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efd5c0a2c10 0x7efd5c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:47.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.109+0000 7efd6cff9640 1 -- 172.21.15.172:0/2885802826 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7efd680cf990 con 0x7efd70105e60 2024-09-10T08:45:47.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.109+0000 7efd6effd640 1 --2- 172.21.15.172:0/2885802826 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efd5c0a2c10 0x7efd5c0a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efd60005e10 tx=0x7efd60005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:47.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.113+0000 7efd6cff9640 1 -- 172.21.15.172:0/2885802826 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd68014030 con 0x7efd70105e60 2024-09-10T08:45:47.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.277+0000 7efd75541640 1 -- 172.21.15.172:0/2885802826 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efd7010a150 con 0x7efd5c0a2c10 2024-09-10T08:45:47.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.286+0000 7efd6cff9640 1 -- 172.21.15.172:0/2885802826 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340394 (secure 0 0 0) 0x7efd7010a150 con 0x7efd5c0a2c10 2024-09-10T08:45:47.291 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:45:47.293 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-09-10T08:45:47.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.289+0000 7efd75541640 1 -- 172.21.15.172:0/2885802826 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efd5c0a2c10 msgr2=0x7efd5c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:47.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.289+0000 7efd75541640 1 --2- 172.21.15.172:0/2885802826 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efd5c0a2c10 0x7efd5c0a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efd60005e10 tx=0x7efd60005da0 comp rx=0 tx=0).stop 2024-09-10T08:45:47.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.289+0000 7efd75541640 1 -- 172.21.15.172:0/2885802826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd70105e60 msgr2=0x7efd7019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:47.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.289+0000 7efd75541640 1 --2- 172.21.15.172:0/2885802826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd70105e60 0x7efd7019cb70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efd6800cce0 tx=0x7efd6800a620 comp rx=0 tx=0).stop 2024-09-10T08:45:47.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.289+0000 7efd75541640 1 -- 172.21.15.172:0/2885802826 shutdown_connections 2024-09-10T08:45:47.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.289+0000 7efd75541640 1 --2- 172.21.15.172:0/2885802826 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efd5c0a2c10 0x7efd5c0a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:47.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.289+0000 7efd75541640 1 --2- 172.21.15.172:0/2885802826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd70105e60 0x7efd7019cb70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:47.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.289+0000 7efd75541640 1 --2- 172.21.15.172:0/2885802826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd70101cf0 0x7efd701987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:47.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.289+0000 7efd75541640 1 --2- 172.21.15.172:0/2885802826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd70101340 0x7efd701982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:47.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.289+0000 7efd75541640 1 -- 172.21.15.172:0/2885802826 >> 172.21.15.172:0/2885802826 conn(0x7efd700fbee0 msgr2=0x7efd701001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:47.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.289+0000 7efd75541640 1 -- 172.21.15.172:0/2885802826 shutdown_connections 2024-09-10T08:45:47.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:47.289+0000 7efd75541640 1 -- 172.21.15.172:0/2885802826 wait complete. 2024-09-10T08:45:47.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:47 smithi172 ceph-mon[30854]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T08:45:47.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:47 smithi172 ceph-mon[30854]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:45:47.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:47 smithi174 ceph-mon[34068]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T08:45:47.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:47 smithi174 ceph-mon[34068]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:45:47.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:47 smithi191 ceph-mon[35314]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T08:45:47.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:47 smithi191 ceph-mon[35314]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:45:48.079 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-10T08:45:47.150041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660901,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":677,"num_read_kb":462,"num_write":431,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":352,"ondisk_log_size":352,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375944,"kb_used_data":14144,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565880,"statfs":{"total":1151940427776,"available":1151555461120,"internally_reserved":0,"allocated":14483456,"data_stored":7995761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011219"},"pg_stats":[{"pgid":"2.1f","version":"88'13","reported_seq":99,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:44.299573+0000","last_change":"2024-09-10T08:44:08.657737+0000","last_active":"2024-09-10T08:45:44.299573+0000","last_peered":"2024-09-10T08:45:44.299573+0000","last_clean":"2024-09-10T08:45:44.299573+0000","last_became_active":"2024-09-10T08:44:08.657473+0000","last_became_peered":"2024-09-10T08:44:08.657473+0000","last_unstale":"2024-09-10T08:45:44.299573+0000","last_undegraded":"2024-09-10T08:45:44.299573+0000","last_fullsized":"2024-09-10T08:45:44.299573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:40.757343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254491+0000","last_change":"2024-09-10T08:44:10.675090+0000","last_active":"2024-09-10T08:45:18.254491+0000","last_peered":"2024-09-10T08:45:18.254491+0000","last_clean":"2024-09-10T08:45:18.254491+0000","last_became_active":"2024-09-10T08:44:10.674612+0000","last_became_peered":"2024-09-10T08:44:10.674612+0000","last_unstale":"2024-09-10T08:45:18.254491+0000","last_undegraded":"2024-09-10T08:45:18.254491+0000","last_fullsized":"2024-09-10T08:45:18.254491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:46.821080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.251619+0000","last_change":"2024-09-10T08:44:17.721559+0000","last_active":"2024-09-10T08:45:18.251619+0000","last_peered":"2024-09-10T08:45:18.251619+0000","last_clean":"2024-09-10T08:45:18.251619+0000","last_became_active":"2024-09-10T08:44:17.721182+0000","last_became_peered":"2024-09-10T08:44:17.721182+0000","last_unstale":"2024-09-10T08:45:18.251619+0000","last_undegraded":"2024-09-10T08:45:18.251619+0000","last_fullsized":"2024-09-10T08:45:18.251619+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:18.791489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254739+0000","last_change":"2024-09-10T08:44:09.049171+0000","last_active":"2024-09-10T08:45:18.254739+0000","last_peered":"2024-09-10T08:45:18.254739+0000","last_clean":"2024-09-10T08:45:18.254739+0000","last_became_active":"2024-09-10T08:44:09.048806+0000","last_became_peered":"2024-09-10T08:44:09.048806+0000","last_unstale":"2024-09-10T08:45:18.254739+0000","last_undegraded":"2024-09-10T08:45:18.254739+0000","last_fullsized":"2024-09-10T08:45:18.254739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:21.023460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248482+0000","last_change":"2024-09-10T08:44:10.677569+0000","last_active":"2024-09-10T08:45:18.248482+0000","last_peered":"2024-09-10T08:45:18.248482+0000","last_clean":"2024-09-10T08:45:18.248482+0000","last_became_active":"2024-09-10T08:44:10.677048+0000","last_became_peered":"2024-09-10T08:44:10.677048+0000","last_unstale":"2024-09-10T08:45:18.248482+0000","last_undegraded":"2024-09-10T08:45:18.248482+0000","last_fullsized":"2024-09-10T08:45:18.248482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:35.523063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.260396+0000","last_change":"2024-09-10T08:44:17.720138+0000","last_active":"2024-09-10T08:45:18.260396+0000","last_peered":"2024-09-10T08:45:18.260396+0000","last_clean":"2024-09-10T08:45:18.260396+0000","last_became_active":"2024-09-10T08:44:17.719344+0000","last_became_peered":"2024-09-10T08:44:17.719344+0000","last_unstale":"2024-09-10T08:45:18.260396+0000","last_undegraded":"2024-09-10T08:45:18.260396+0000","last_fullsized":"2024-09-10T08:45:18.260396+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:11.758058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.249040+0000","last_change":"2024-09-10T08:44:08.660099+0000","last_active":"2024-09-10T08:45:18.249040+0000","last_peered":"2024-09-10T08:45:18.249040+0000","last_clean":"2024-09-10T08:45:18.249040+0000","last_became_active":"2024-09-10T08:44:08.659745+0000","last_became_peered":"2024-09-10T08:44:08.659745+0000","last_unstale":"2024-09-10T08:45:18.249040+0000","last_undegraded":"2024-09-10T08:45:18.249040+0000","last_fullsized":"2024-09-10T08:45:18.249040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:43.960717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248650+0000","last_change":"2024-09-10T08:44:10.675583+0000","last_active":"2024-09-10T08:45:18.248650+0000","last_peered":"2024-09-10T08:45:18.248650+0000","last_clean":"2024-09-10T08:45:18.248650+0000","last_became_active":"2024-09-10T08:44:10.675296+0000","last_became_peered":"2024-09-10T08:44:10.675296+0000","last_unstale":"2024-09-10T08:45:18.248650+0000","last_undegraded":"2024-09-10T08:45:18.248650+0000","last_fullsized":"2024-09-10T08:45:18.248650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:31:51.147862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261671+0000","last_change":"2024-09-10T08:44:17.720104+0000","last_active":"2024-09-10T08:45:18.261671+0000","last_peered":"2024-09-10T08:45:18.261671+0000","last_clean":"2024-09-10T08:45:18.261671+0000","last_became_active":"2024-09-10T08:44:17.719607+0000","last_became_peered":"2024-09-10T08:44:17.719607+0000","last_unstale":"2024-09-10T08:45:18.261671+0000","last_undegraded":"2024-09-10T08:45:18.261671+0000","last_fullsized":"2024-09-10T08:45:18.261671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:52.976717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248702+0000","last_change":"2024-09-10T08:44:08.658588+0000","last_active":"2024-09-10T08:45:18.248702+0000","last_peered":"2024-09-10T08:45:18.248702+0000","last_clean":"2024-09-10T08:45:18.248702+0000","last_became_active":"2024-09-10T08:44:08.658053+0000","last_became_peered":"2024-09-10T08:44:08.658053+0000","last_unstale":"2024-09-10T08:45:18.248702+0000","last_undegraded":"2024-09-10T08:45:18.248702+0000","last_fullsized":"2024-09-10T08:45:18.248702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:26.744943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261968+0000","last_change":"2024-09-10T08:44:10.681978+0000","last_active":"2024-09-10T08:45:18.261968+0000","last_peered":"2024-09-10T08:45:18.261968+0000","last_clean":"2024-09-10T08:45:18.261968+0000","last_became_active":"2024-09-10T08:44:10.681600+0000","last_became_peered":"2024-09-10T08:44:10.681600+0000","last_unstale":"2024-09-10T08:45:18.261968+0000","last_undegraded":"2024-09-10T08:45:18.261968+0000","last_fullsized":"2024-09-10T08:45:18.261968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:37.190576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.260926+0000","last_change":"2024-09-10T08:44:17.719784+0000","last_active":"2024-09-10T08:45:18.260926+0000","last_peered":"2024-09-10T08:45:18.260926+0000","last_clean":"2024-09-10T08:45:18.260926+0000","last_became_active":"2024-09-10T08:44:17.719182+0000","last_became_peered":"2024-09-10T08:44:17.719182+0000","last_unstale":"2024-09-10T08:45:18.260926+0000","last_undegraded":"2024-09-10T08:45:18.260926+0000","last_fullsized":"2024-09-10T08:45:18.260926+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:15.237772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270233+0000","last_change":"2024-09-10T08:44:10.679069+0000","last_active":"2024-09-10T08:45:18.270233+0000","last_peered":"2024-09-10T08:45:18.270233+0000","last_clean":"2024-09-10T08:45:18.270233+0000","last_became_active":"2024-09-10T08:44:10.678597+0000","last_became_peered":"2024-09-10T08:44:10.678597+0000","last_unstale":"2024-09-10T08:45:18.270233+0000","last_undegraded":"2024-09-10T08:45:18.270233+0000","last_fullsized":"2024-09-10T08:45:18.270233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:52.693906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248551+0000","last_change":"2024-09-10T08:44:08.660886+0000","last_active":"2024-09-10T08:45:18.248551+0000","last_peered":"2024-09-10T08:45:18.248551+0000","last_clean":"2024-09-10T08:45:18.248551+0000","last_became_active":"2024-09-10T08:44:08.660478+0000","last_became_peered":"2024-09-10T08:44:08.660478+0000","last_unstale":"2024-09-10T08:45:18.248551+0000","last_undegraded":"2024-09-10T08:45:18.248551+0000","last_fullsized":"2024-09-10T08:45:18.248551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:41.785531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261582+0000","last_change":"2024-09-10T08:44:17.761929+0000","last_active":"2024-09-10T08:45:18.261582+0000","last_peered":"2024-09-10T08:45:18.261582+0000","last_clean":"2024-09-10T08:45:18.261582+0000","last_became_active":"2024-09-10T08:44:17.761440+0000","last_became_peered":"2024-09-10T08:44:17.761440+0000","last_unstale":"2024-09-10T08:45:18.261582+0000","last_undegraded":"2024-09-10T08:45:18.261582+0000","last_fullsized":"2024-09-10T08:45:18.261582+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:37.817759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254646+0000","last_change":"2024-09-10T08:44:10.674071+0000","last_active":"2024-09-10T08:45:18.254646+0000","last_peered":"2024-09-10T08:45:18.254646+0000","last_clean":"2024-09-10T08:45:18.254646+0000","last_became_active":"2024-09-10T08:44:10.673770+0000","last_became_peered":"2024-09-10T08:44:10.673770+0000","last_unstale":"2024-09-10T08:45:18.254646+0000","last_undegraded":"2024-09-10T08:45:18.254646+0000","last_fullsized":"2024-09-10T08:45:18.254646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:44:19.944557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.264475+0000","last_change":"2024-09-10T08:45:18.264060+0000","last_active":"2024-09-10T08:45:18.264475+0000","last_peered":"2024-09-10T08:45:18.264475+0000","last_clean":"2024-09-10T08:45:18.264475+0000","last_became_active":"2024-09-10T08:45:18.255983+0000","last_became_peered":"2024-09-10T08:45:18.255983+0000","last_unstale":"2024-09-10T08:45:18.264475+0000","last_undegraded":"2024-09-10T08:45:18.264475+0000","last_fullsized":"2024-09-10T08:45:18.264475+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:10.810491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261509+0000","last_change":"2024-09-10T08:44:17.722877+0000","last_active":"2024-09-10T08:45:18.261509+0000","last_peered":"2024-09-10T08:45:18.261509+0000","last_clean":"2024-09-10T08:45:18.261509+0000","last_became_active":"2024-09-10T08:44:17.722046+0000","last_became_peered":"2024-09-10T08:44:17.722046+0000","last_unstale":"2024-09-10T08:45:18.261509+0000","last_undegraded":"2024-09-10T08:45:18.261509+0000","last_fullsized":"2024-09-10T08:45:18.261509+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:32.675598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252361+0000","last_change":"2024-09-10T08:44:10.671500+0000","last_active":"2024-09-10T08:45:18.252361+0000","last_peered":"2024-09-10T08:45:18.252361+0000","last_clean":"2024-09-10T08:45:18.252361+0000","last_became_active":"2024-09-10T08:44:10.671039+0000","last_became_peered":"2024-09-10T08:44:10.671039+0000","last_unstale":"2024-09-10T08:45:18.252361+0000","last_undegraded":"2024-09-10T08:45:18.252361+0000","last_fullsized":"2024-09-10T08:45:18.252361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:12.977373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.255013+0000","last_change":"2024-09-10T08:44:08.655467+0000","last_active":"2024-09-10T08:45:18.255013+0000","last_peered":"2024-09-10T08:45:18.255013+0000","last_clean":"2024-09-10T08:45:18.255013+0000","last_became_active":"2024-09-10T08:44:08.654991+0000","last_became_peered":"2024-09-10T08:44:08.654991+0000","last_unstale":"2024-09-10T08:45:18.255013+0000","last_undegraded":"2024-09-10T08:45:18.255013+0000","last_fullsized":"2024-09-10T08:45:18.255013+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:33.124360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.267039+0000","last_change":"2024-09-10T08:44:17.724250+0000","last_active":"2024-09-10T08:45:18.267039+0000","last_peered":"2024-09-10T08:45:18.267039+0000","last_clean":"2024-09-10T08:45:18.267039+0000","last_became_active":"2024-09-10T08:44:17.723733+0000","last_became_peered":"2024-09-10T08:44:17.723733+0000","last_unstale":"2024-09-10T08:45:18.267039+0000","last_undegraded":"2024-09-10T08:45:18.267039+0000","last_fullsized":"2024-09-10T08:45:18.267039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:40.098666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248261+0000","last_change":"2024-09-10T08:44:10.664159+0000","last_active":"2024-09-10T08:45:18.248261+0000","last_peered":"2024-09-10T08:45:18.248261+0000","last_clean":"2024-09-10T08:45:18.248261+0000","last_became_active":"2024-09-10T08:44:10.663674+0000","last_became_peered":"2024-09-10T08:44:10.663674+0000","last_unstale":"2024-09-10T08:45:18.248261+0000","last_undegraded":"2024-09-10T08:45:18.248261+0000","last_fullsized":"2024-09-10T08:45:18.248261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:24.234525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257226+0000","last_change":"2024-09-10T08:45:18.257226+0000","last_active":"2024-09-10T08:45:18.257226+0000","last_peered":"2024-09-10T08:45:18.257226+0000","last_clean":"2024-09-10T08:45:18.257226+0000","last_became_active":"2024-09-10T08:45:18.254113+0000","last_became_peered":"2024-09-10T08:45:18.254113+0000","last_unstale":"2024-09-10T08:45:18.257226+0000","last_undegraded":"2024-09-10T08:45:18.257226+0000","last_fullsized":"2024-09-10T08:45:18.257226+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:19.370105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261558+0000","last_change":"2024-09-10T08:44:17.761639+0000","last_active":"2024-09-10T08:45:18.261558+0000","last_peered":"2024-09-10T08:45:18.261558+0000","last_clean":"2024-09-10T08:45:18.261558+0000","last_became_active":"2024-09-10T08:44:17.761214+0000","last_became_peered":"2024-09-10T08:44:17.761214+0000","last_unstale":"2024-09-10T08:45:18.261558+0000","last_undegraded":"2024-09-10T08:45:18.261558+0000","last_fullsized":"2024-09-10T08:45:18.261558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:43.793195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.258661+0000","last_change":"2024-09-10T08:45:18.258661+0000","last_active":"2024-09-10T08:45:18.258661+0000","last_peered":"2024-09-10T08:45:18.258661+0000","last_clean":"2024-09-10T08:45:18.258661+0000","last_became_active":"2024-09-10T08:45:18.258225+0000","last_became_peered":"2024-09-10T08:45:18.258225+0000","last_unstale":"2024-09-10T08:45:18.258661+0000","last_undegraded":"2024-09-10T08:45:18.258661+0000","last_fullsized":"2024-09-10T08:45:18.258661+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:28:37.184766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.262282+0000","last_change":"2024-09-10T08:44:09.036632+0000","last_active":"2024-09-10T08:45:18.262282+0000","last_peered":"2024-09-10T08:45:18.262282+0000","last_clean":"2024-09-10T08:45:18.262282+0000","last_became_active":"2024-09-10T08:44:09.036254+0000","last_became_peered":"2024-09-10T08:44:09.036254+0000","last_unstale":"2024-09-10T08:45:18.262282+0000","last_undegraded":"2024-09-10T08:45:18.262282+0000","last_fullsized":"2024-09-10T08:45:18.262282+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:59:40.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252129+0000","last_change":"2024-09-10T08:44:17.716215+0000","last_active":"2024-09-10T08:45:18.252129+0000","last_peered":"2024-09-10T08:45:18.252129+0000","last_clean":"2024-09-10T08:45:18.252129+0000","last_became_active":"2024-09-10T08:44:17.715809+0000","last_became_peered":"2024-09-10T08:44:17.715809+0000","last_unstale":"2024-09-10T08:45:18.252129+0000","last_undegraded":"2024-09-10T08:45:18.252129+0000","last_fullsized":"2024-09-10T08:45:18.252129+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:52.896401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254431+0000","last_change":"2024-09-10T08:44:10.677871+0000","last_active":"2024-09-10T08:45:18.254431+0000","last_peered":"2024-09-10T08:45:18.254431+0000","last_clean":"2024-09-10T08:45:18.254431+0000","last_became_active":"2024-09-10T08:44:10.677101+0000","last_became_peered":"2024-09-10T08:44:10.677101+0000","last_unstale":"2024-09-10T08:45:18.254431+0000","last_undegraded":"2024-09-10T08:45:18.254431+0000","last_fullsized":"2024-09-10T08:45:18.254431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:46.137249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.267182+0000","last_change":"2024-09-10T08:44:08.654428+0000","last_active":"2024-09-10T08:45:18.267182+0000","last_peered":"2024-09-10T08:45:18.267182+0000","last_clean":"2024-09-10T08:45:18.267182+0000","last_became_active":"2024-09-10T08:44:08.654119+0000","last_became_peered":"2024-09-10T08:44:08.654119+0000","last_unstale":"2024-09-10T08:45:18.267182+0000","last_undegraded":"2024-09-10T08:45:18.267182+0000","last_fullsized":"2024-09-10T08:45:18.267182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:56.787721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.253964+0000","last_change":"2024-09-10T08:44:17.760828+0000","last_active":"2024-09-10T08:45:18.253964+0000","last_peered":"2024-09-10T08:45:18.253964+0000","last_clean":"2024-09-10T08:45:18.253964+0000","last_became_active":"2024-09-10T08:44:17.760367+0000","last_became_peered":"2024-09-10T08:44:17.760367+0000","last_unstale":"2024-09-10T08:45:18.253964+0000","last_undegraded":"2024-09-10T08:45:18.253964+0000","last_fullsized":"2024-09-10T08:45:18.253964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:11.130480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257638+0000","last_change":"2024-09-10T08:44:10.670179+0000","last_active":"2024-09-10T08:45:18.257638+0000","last_peered":"2024-09-10T08:45:18.257638+0000","last_clean":"2024-09-10T08:45:18.257638+0000","last_became_active":"2024-09-10T08:44:10.669366+0000","last_became_peered":"2024-09-10T08:44:10.669366+0000","last_unstale":"2024-09-10T08:45:18.257638+0000","last_undegraded":"2024-09-10T08:45:18.257638+0000","last_fullsized":"2024-09-10T08:45:18.257638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:41.916365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.249082+0000","last_change":"2024-09-10T08:44:09.050441+0000","last_active":"2024-09-10T08:45:18.249082+0000","last_peered":"2024-09-10T08:45:18.249082+0000","last_clean":"2024-09-10T08:45:18.249082+0000","last_became_active":"2024-09-10T08:44:09.050070+0000","last_became_peered":"2024-09-10T08:44:09.050070+0000","last_unstale":"2024-09-10T08:45:18.249082+0000","last_undegraded":"2024-09-10T08:45:18.249082+0000","last_fullsized":"2024-09-10T08:45:18.249082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:13.540021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.251158+0000","last_change":"2024-09-10T08:44:17.720098+0000","last_active":"2024-09-10T08:45:18.251158+0000","last_peered":"2024-09-10T08:45:18.251158+0000","last_clean":"2024-09-10T08:45:18.251158+0000","last_became_active":"2024-09-10T08:44:17.719627+0000","last_became_peered":"2024-09-10T08:44:17.719627+0000","last_unstale":"2024-09-10T08:45:18.251158+0000","last_undegraded":"2024-09-10T08:45:18.251158+0000","last_fullsized":"2024-09-10T08:45:18.251158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:50.020919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.258587+0000","last_change":"2024-09-10T08:44:10.669630+0000","last_active":"2024-09-10T08:45:18.258587+0000","last_peered":"2024-09-10T08:45:18.258587+0000","last_clean":"2024-09-10T08:45:18.258587+0000","last_became_active":"2024-09-10T08:44:10.669307+0000","last_became_peered":"2024-09-10T08:44:10.669307+0000","last_unstale":"2024-09-10T08:45:18.258587+0000","last_undegraded":"2024-09-10T08:45:18.258587+0000","last_fullsized":"2024-09-10T08:45:18.258587+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:06.320046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.267039+0000","last_change":"2024-09-10T08:44:09.050715+0000","last_active":"2024-09-10T08:45:18.267039+0000","last_peered":"2024-09-10T08:45:18.267039+0000","last_clean":"2024-09-10T08:45:18.267039+0000","last_became_active":"2024-09-10T08:44:09.050347+0000","last_became_peered":"2024-09-10T08:44:09.050347+0000","last_unstale":"2024-09-10T08:45:18.267039+0000","last_undegraded":"2024-09-10T08:45:18.267039+0000","last_fullsized":"2024-09-10T08:45:18.267039+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:18.689652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.260444+0000","last_change":"2024-09-10T08:45:18.260444+0000","last_active":"2024-09-10T08:45:18.260444+0000","last_peered":"2024-09-10T08:45:18.260444+0000","last_clean":"2024-09-10T08:45:18.260444+0000","last_became_active":"2024-09-10T08:45:18.259949+0000","last_became_peered":"2024-09-10T08:45:18.259949+0000","last_unstale":"2024-09-10T08:45:18.260444+0000","last_undegraded":"2024-09-10T08:45:18.260444+0000","last_fullsized":"2024-09-10T08:45:18.260444+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:56.391253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.258585+0000","last_change":"2024-09-10T08:44:10.670881+0000","last_active":"2024-09-10T08:45:18.258585+0000","last_peered":"2024-09-10T08:45:18.258585+0000","last_clean":"2024-09-10T08:45:18.258585+0000","last_became_active":"2024-09-10T08:44:10.669791+0000","last_became_peered":"2024-09-10T08:44:10.669791+0000","last_unstale":"2024-09-10T08:45:18.258585+0000","last_undegraded":"2024-09-10T08:45:18.258585+0000","last_fullsized":"2024-09-10T08:45:18.258585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:33.644205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":89,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:42.298071+0000","last_change":"2024-09-10T08:44:08.657938+0000","last_active":"2024-09-10T08:45:42.298071+0000","last_peered":"2024-09-10T08:45:42.298071+0000","last_clean":"2024-09-10T08:45:42.298071+0000","last_became_active":"2024-09-10T08:44:08.657526+0000","last_became_peered":"2024-09-10T08:44:08.657526+0000","last_unstale":"2024-09-10T08:45:42.298071+0000","last_undegraded":"2024-09-10T08:45:42.298071+0000","last_fullsized":"2024-09-10T08:45:42.298071+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:15.673561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257344+0000","last_change":"2024-09-10T08:44:17.721619+0000","last_active":"2024-09-10T08:45:18.257344+0000","last_peered":"2024-09-10T08:45:18.257344+0000","last_clean":"2024-09-10T08:45:18.257344+0000","last_became_active":"2024-09-10T08:44:17.721175+0000","last_became_peered":"2024-09-10T08:44:17.721175+0000","last_unstale":"2024-09-10T08:45:18.257344+0000","last_undegraded":"2024-09-10T08:45:18.257344+0000","last_fullsized":"2024-09-10T08:45:18.257344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:52.244120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261384+0000","last_change":"2024-09-10T08:44:10.676164+0000","last_active":"2024-09-10T08:45:18.261384+0000","last_peered":"2024-09-10T08:45:18.261384+0000","last_clean":"2024-09-10T08:45:18.261384+0000","last_became_active":"2024-09-10T08:44:10.675820+0000","last_became_peered":"2024-09-10T08:44:10.675820+0000","last_unstale":"2024-09-10T08:45:18.261384+0000","last_undegraded":"2024-09-10T08:45:18.261384+0000","last_fullsized":"2024-09-10T08:45:18.261384+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:24.477584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":91,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:42.293033+0000","last_change":"2024-09-10T08:44:09.049124+0000","last_active":"2024-09-10T08:45:42.293033+0000","last_peered":"2024-09-10T08:45:42.293033+0000","last_clean":"2024-09-10T08:45:42.293033+0000","last_became_active":"2024-09-10T08:44:09.048799+0000","last_became_peered":"2024-09-10T08:44:09.048799+0000","last_unstale":"2024-09-10T08:45:42.293033+0000","last_undegraded":"2024-09-10T08:45:42.293033+0000","last_fullsized":"2024-09-10T08:45:42.293033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:43:39.849727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257766+0000","last_change":"2024-09-10T08:44:17.761130+0000","last_active":"2024-09-10T08:45:18.257766+0000","last_peered":"2024-09-10T08:45:18.257766+0000","last_clean":"2024-09-10T08:45:18.257766+0000","last_became_active":"2024-09-10T08:44:17.760753+0000","last_became_peered":"2024-09-10T08:44:17.760753+0000","last_unstale":"2024-09-10T08:45:18.257766+0000","last_undegraded":"2024-09-10T08:45:18.257766+0000","last_fullsized":"2024-09-10T08:45:18.257766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:42.971962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.256832+0000","last_change":"2024-09-10T08:44:10.669905+0000","last_active":"2024-09-10T08:45:18.256832+0000","last_peered":"2024-09-10T08:45:18.256832+0000","last_clean":"2024-09-10T08:45:18.256832+0000","last_became_active":"2024-09-10T08:44:10.669340+0000","last_became_peered":"2024-09-10T08:44:10.669340+0000","last_unstale":"2024-09-10T08:45:18.256832+0000","last_undegraded":"2024-09-10T08:45:18.256832+0000","last_fullsized":"2024-09-10T08:45:18.256832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:23.263179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270000+0000","last_change":"2024-09-10T08:44:08.658862+0000","last_active":"2024-09-10T08:45:18.270000+0000","last_peered":"2024-09-10T08:45:18.270000+0000","last_clean":"2024-09-10T08:45:18.270000+0000","last_became_active":"2024-09-10T08:44:08.657880+0000","last_became_peered":"2024-09-10T08:44:08.657880+0000","last_unstale":"2024-09-10T08:45:18.270000+0000","last_undegraded":"2024-09-10T08:45:18.270000+0000","last_fullsized":"2024-09-10T08:45:18.270000+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:34.224025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.269022+0000","last_change":"2024-09-10T08:45:18.268446+0000","last_active":"2024-09-10T08:45:18.269022+0000","last_peered":"2024-09-10T08:45:18.269022+0000","last_clean":"2024-09-10T08:45:18.269022+0000","last_became_active":"2024-09-10T08:45:18.267988+0000","last_became_peered":"2024-09-10T08:45:18.267988+0000","last_unstale":"2024-09-10T08:45:18.269022+0000","last_undegraded":"2024-09-10T08:45:18.269022+0000","last_fullsized":"2024-09-10T08:45:18.269022+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:42:35.990794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254358+0000","last_change":"2024-09-10T08:44:10.673258+0000","last_active":"2024-09-10T08:45:18.254358+0000","last_peered":"2024-09-10T08:45:18.254358+0000","last_clean":"2024-09-10T08:45:18.254358+0000","last_became_active":"2024-09-10T08:44:10.672854+0000","last_became_peered":"2024-09-10T08:44:10.672854+0000","last_unstale":"2024-09-10T08:45:18.254358+0000","last_undegraded":"2024-09-10T08:45:18.254358+0000","last_fullsized":"2024-09-10T08:45:18.254358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:35.913820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261847+0000","last_change":"2024-09-10T08:44:08.658039+0000","last_active":"2024-09-10T08:45:18.261847+0000","last_peered":"2024-09-10T08:45:18.261847+0000","last_clean":"2024-09-10T08:45:18.261847+0000","last_became_active":"2024-09-10T08:44:08.657636+0000","last_became_peered":"2024-09-10T08:44:08.657636+0000","last_unstale":"2024-09-10T08:45:18.261847+0000","last_undegraded":"2024-09-10T08:45:18.261847+0000","last_fullsized":"2024-09-10T08:45:18.261847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:09.615992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270530+0000","last_change":"2024-09-10T08:42:57.666747+0000","last_active":"2024-09-10T08:45:18.270530+0000","last_peered":"2024-09-10T08:45:18.270530+0000","last_clean":"2024-09-10T08:45:18.270530+0000","last_became_active":"2024-09-10T08:42:57.666417+0000","last_became_peered":"2024-09-10T08:42:57.666417+0000","last_unstale":"2024-09-10T08:45:18.270530+0000","last_undegraded":"2024-09-10T08:45:18.270530+0000","last_fullsized":"2024-09-10T08:45:18.270530+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_clean_scrub_stamp":"2024-09-10T08:39:31.729794+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.547989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270574+0000","last_change":"2024-09-10T08:44:17.723021+0000","last_active":"2024-09-10T08:45:18.270574+0000","last_peered":"2024-09-10T08:45:18.270574+0000","last_clean":"2024-09-10T08:45:18.270574+0000","last_became_active":"2024-09-10T08:44:17.722222+0000","last_became_peered":"2024-09-10T08:44:17.722222+0000","last_unstale":"2024-09-10T08:45:18.270574+0000","last_undegraded":"2024-09-10T08:45:18.270574+0000","last_fullsized":"2024-09-10T08:45:18.270574+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:24.444427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270662+0000","last_change":"2024-09-10T08:44:10.676651+0000","last_active":"2024-09-10T08:45:18.270662+0000","last_peered":"2024-09-10T08:45:18.270662+0000","last_clean":"2024-09-10T08:45:18.270662+0000","last_became_active":"2024-09-10T08:44:10.676342+0000","last_became_peered":"2024-09-10T08:44:10.676342+0000","last_unstale":"2024-09-10T08:45:18.270662+0000","last_undegraded":"2024-09-10T08:45:18.270662+0000","last_fullsized":"2024-09-10T08:45:18.270662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:07.021273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":95,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:45.302623+0000","last_change":"2024-09-10T08:44:09.050305+0000","last_active":"2024-09-10T08:45:45.302623+0000","last_peered":"2024-09-10T08:45:45.302623+0000","last_clean":"2024-09-10T08:45:45.302623+0000","last_became_active":"2024-09-10T08:44:09.049958+0000","last_became_peered":"2024-09-10T08:44:09.049958+0000","last_unstale":"2024-09-10T08:45:45.302623+0000","last_undegraded":"2024-09-10T08:45:45.302623+0000","last_fullsized":"2024-09-10T08:45:45.302623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:42.219190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248307+0000","last_change":"2024-09-10T08:44:17.720219+0000","last_active":"2024-09-10T08:45:18.248307+0000","last_peered":"2024-09-10T08:45:18.248307+0000","last_clean":"2024-09-10T08:45:18.248307+0000","last_became_active":"2024-09-10T08:44:17.719756+0000","last_became_peered":"2024-09-10T08:44:17.719756+0000","last_unstale":"2024-09-10T08:45:18.248307+0000","last_undegraded":"2024-09-10T08:45:18.248307+0000","last_fullsized":"2024-09-10T08:45:18.248307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:01.070434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270237+0000","last_change":"2024-09-10T08:44:10.678986+0000","last_active":"2024-09-10T08:45:18.270237+0000","last_peered":"2024-09-10T08:45:18.270237+0000","last_clean":"2024-09-10T08:45:18.270237+0000","last_became_active":"2024-09-10T08:44:10.678654+0000","last_became_peered":"2024-09-10T08:44:10.678654+0000","last_unstale":"2024-09-10T08:45:18.270237+0000","last_undegraded":"2024-09-10T08:45:18.270237+0000","last_fullsized":"2024-09-10T08:45:18.270237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:07.116802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.262374+0000","last_change":"2024-09-10T08:44:08.655025+0000","last_active":"2024-09-10T08:45:18.262374+0000","last_peered":"2024-09-10T08:45:18.262374+0000","last_clean":"2024-09-10T08:45:18.262374+0000","last_became_active":"2024-09-10T08:44:08.654728+0000","last_became_peered":"2024-09-10T08:44:08.654728+0000","last_unstale":"2024-09-10T08:45:18.262374+0000","last_undegraded":"2024-09-10T08:45:18.262374+0000","last_fullsized":"2024-09-10T08:45:18.262374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:31.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261492+0000","last_change":"2024-09-10T08:45:18.261492+0000","last_active":"2024-09-10T08:45:18.261492+0000","last_peered":"2024-09-10T08:45:18.261492+0000","last_clean":"2024-09-10T08:45:18.261492+0000","last_became_active":"2024-09-10T08:45:18.261111+0000","last_became_peered":"2024-09-10T08:45:18.261111+0000","last_unstale":"2024-09-10T08:45:18.261492+0000","last_undegraded":"2024-09-10T08:45:18.261492+0000","last_fullsized":"2024-09-10T08:45:18.261492+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:48.709459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270355+0000","last_change":"2024-09-10T08:44:10.675072+0000","last_active":"2024-09-10T08:45:18.270355+0000","last_peered":"2024-09-10T08:45:18.270355+0000","last_clean":"2024-09-10T08:45:18.270355+0000","last_became_active":"2024-09-10T08:44:10.674636+0000","last_became_peered":"2024-09-10T08:44:10.674636+0000","last_unstale":"2024-09-10T08:45:18.270355+0000","last_undegraded":"2024-09-10T08:45:18.270355+0000","last_fullsized":"2024-09-10T08:45:18.270355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:55.198445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257848+0000","last_change":"2024-09-10T08:44:08.656450+0000","last_active":"2024-09-10T08:45:18.257848+0000","last_peered":"2024-09-10T08:45:18.257848+0000","last_clean":"2024-09-10T08:45:18.257848+0000","last_became_active":"2024-09-10T08:44:08.655967+0000","last_became_peered":"2024-09-10T08:44:08.655967+0000","last_unstale":"2024-09-10T08:45:18.257848+0000","last_undegraded":"2024-09-10T08:45:18.257848+0000","last_fullsized":"2024-09-10T08:45:18.257848+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:29.273167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261136+0000","last_change":"2024-09-10T08:44:17.722217+0000","last_active":"2024-09-10T08:45:18.261136+0000","last_peered":"2024-09-10T08:45:18.261136+0000","last_clean":"2024-09-10T08:45:18.261136+0000","last_became_active":"2024-09-10T08:44:17.721800+0000","last_became_peered":"2024-09-10T08:44:17.721800+0000","last_unstale":"2024-09-10T08:45:18.261136+0000","last_undegraded":"2024-09-10T08:45:18.261136+0000","last_fullsized":"2024-09-10T08:45:18.261136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:58.954397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.267857+0000","last_change":"2024-09-10T08:45:18.267857+0000","last_active":"2024-09-10T08:45:18.267857+0000","last_peered":"2024-09-10T08:45:18.267857+0000","last_clean":"2024-09-10T08:45:18.267857+0000","last_became_active":"2024-09-10T08:45:18.267440+0000","last_became_peered":"2024-09-10T08:45:18.267440+0000","last_unstale":"2024-09-10T08:45:18.267857+0000","last_undegraded":"2024-09-10T08:45:18.267857+0000","last_fullsized":"2024-09-10T08:45:18.267857+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:28.659915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.249034+0000","last_change":"2024-09-10T08:44:08.659003+0000","last_active":"2024-09-10T08:45:18.249034+0000","last_peered":"2024-09-10T08:45:18.249034+0000","last_clean":"2024-09-10T08:45:18.249034+0000","last_became_active":"2024-09-10T08:44:08.658324+0000","last_became_peered":"2024-09-10T08:44:08.658324+0000","last_unstale":"2024-09-10T08:45:18.249034+0000","last_undegraded":"2024-09-10T08:45:18.249034+0000","last_fullsized":"2024-09-10T08:45:18.249034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:44.743053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261510+0000","last_change":"2024-09-10T08:44:17.723363+0000","last_active":"2024-09-10T08:45:18.261510+0000","last_peered":"2024-09-10T08:45:18.261510+0000","last_clean":"2024-09-10T08:45:18.261510+0000","last_became_active":"2024-09-10T08:44:17.721799+0000","last_became_peered":"2024-09-10T08:44:17.721799+0000","last_unstale":"2024-09-10T08:45:18.261510+0000","last_undegraded":"2024-09-10T08:45:18.261510+0000","last_fullsized":"2024-09-10T08:45:18.261510+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:08:28.772193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261712+0000","last_change":"2024-09-10T08:44:10.682952+0000","last_active":"2024-09-10T08:45:18.261712+0000","last_peered":"2024-09-10T08:45:18.261712+0000","last_clean":"2024-09-10T08:45:18.261712+0000","last_became_active":"2024-09-10T08:44:10.682249+0000","last_became_peered":"2024-09-10T08:44:10.682249+0000","last_unstale":"2024-09-10T08:45:18.261712+0000","last_undegraded":"2024-09-10T08:45:18.261712+0000","last_fullsized":"2024-09-10T08:45:18.261712+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:32:21.285461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248258+0000","last_change":"2024-09-10T08:44:09.050454+0000","last_active":"2024-09-10T08:45:18.248258+0000","last_peered":"2024-09-10T08:45:18.248258+0000","last_clean":"2024-09-10T08:45:18.248258+0000","last_became_active":"2024-09-10T08:44:09.050036+0000","last_became_peered":"2024-09-10T08:44:09.050036+0000","last_unstale":"2024-09-10T08:45:18.248258+0000","last_undegraded":"2024-09-10T08:45:18.248258+0000","last_fullsized":"2024-09-10T08:45:18.248258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:55.141471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257803+0000","last_change":"2024-09-10T08:44:17.721203+0000","last_active":"2024-09-10T08:45:18.257803+0000","last_peered":"2024-09-10T08:45:18.257803+0000","last_clean":"2024-09-10T08:45:18.257803+0000","last_became_active":"2024-09-10T08:44:17.720737+0000","last_became_peered":"2024-09-10T08:44:17.720737+0000","last_unstale":"2024-09-10T08:45:18.257803+0000","last_undegraded":"2024-09-10T08:45:18.257803+0000","last_fullsized":"2024-09-10T08:45:18.257803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:32.081079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.269917+0000","last_change":"2024-09-10T08:44:10.680944+0000","last_active":"2024-09-10T08:45:18.269917+0000","last_peered":"2024-09-10T08:45:18.269917+0000","last_clean":"2024-09-10T08:45:18.269917+0000","last_became_active":"2024-09-10T08:44:10.680644+0000","last_became_peered":"2024-09-10T08:44:10.680644+0000","last_unstale":"2024-09-10T08:45:18.269917+0000","last_undegraded":"2024-09-10T08:45:18.269917+0000","last_fullsized":"2024-09-10T08:45:18.269917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:46.495894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:42.293794+0000","last_change":"2024-09-10T08:44:08.658135+0000","last_active":"2024-09-10T08:45:42.293794+0000","last_peered":"2024-09-10T08:45:42.293794+0000","last_clean":"2024-09-10T08:45:42.293794+0000","last_became_active":"2024-09-10T08:44:08.657793+0000","last_became_peered":"2024-09-10T08:44:08.657793+0000","last_unstale":"2024-09-10T08:45:42.293794+0000","last_undegraded":"2024-09-10T08:45:42.293794+0000","last_fullsized":"2024-09-10T08:45:42.293794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:16.376268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.272086+0000","last_change":"2024-09-10T08:45:18.271678+0000","last_active":"2024-09-10T08:45:18.272086+0000","last_peered":"2024-09-10T08:45:18.272086+0000","last_clean":"2024-09-10T08:45:18.272086+0000","last_became_active":"2024-09-10T08:45:18.271209+0000","last_became_peered":"2024-09-10T08:45:18.271209+0000","last_unstale":"2024-09-10T08:45:18.272086+0000","last_undegraded":"2024-09-10T08:45:18.272086+0000","last_fullsized":"2024-09-10T08:45:18.272086+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:17.730839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270358+0000","last_change":"2024-09-10T08:44:10.678254+0000","last_active":"2024-09-10T08:45:18.270358+0000","last_peered":"2024-09-10T08:45:18.270358+0000","last_clean":"2024-09-10T08:45:18.270358+0000","last_became_active":"2024-09-10T08:44:10.677891+0000","last_became_peered":"2024-09-10T08:44:10.677891+0000","last_unstale":"2024-09-10T08:45:18.270358+0000","last_undegraded":"2024-09-10T08:45:18.270358+0000","last_fullsized":"2024-09-10T08:45:18.270358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:32:43.399049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261698+0000","last_change":"2024-09-10T08:44:08.656315+0000","last_active":"2024-09-10T08:45:18.261698+0000","last_peered":"2024-09-10T08:45:18.261698+0000","last_clean":"2024-09-10T08:45:18.261698+0000","last_became_active":"2024-09-10T08:44:08.655978+0000","last_became_peered":"2024-09-10T08:44:08.655978+0000","last_unstale":"2024-09-10T08:45:18.261698+0000","last_undegraded":"2024-09-10T08:45:18.261698+0000","last_fullsized":"2024-09-10T08:45:18.261698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:15.025354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252245+0000","last_change":"2024-09-10T08:44:17.726789+0000","last_active":"2024-09-10T08:45:18.252245+0000","last_peered":"2024-09-10T08:45:18.252245+0000","last_clean":"2024-09-10T08:45:18.252245+0000","last_became_active":"2024-09-10T08:44:17.726450+0000","last_became_peered":"2024-09-10T08:44:17.726450+0000","last_unstale":"2024-09-10T08:45:18.252245+0000","last_undegraded":"2024-09-10T08:45:18.252245+0000","last_fullsized":"2024-09-10T08:45:18.252245+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:03.899822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257056+0000","last_change":"2024-09-10T08:44:10.677359+0000","last_active":"2024-09-10T08:45:18.257056+0000","last_peered":"2024-09-10T08:45:18.257056+0000","last_clean":"2024-09-10T08:45:18.257056+0000","last_became_active":"2024-09-10T08:44:10.676782+0000","last_became_peered":"2024-09-10T08:44:10.676782+0000","last_unstale":"2024-09-10T08:45:18.257056+0000","last_undegraded":"2024-09-10T08:45:18.257056+0000","last_fullsized":"2024-09-10T08:45:18.257056+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:31.721714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252007+0000","last_change":"2024-09-10T08:44:08.656993+0000","last_active":"2024-09-10T08:45:18.252007+0000","last_peered":"2024-09-10T08:45:18.252007+0000","last_clean":"2024-09-10T08:45:18.252007+0000","last_became_active":"2024-09-10T08:44:08.656571+0000","last_became_peered":"2024-09-10T08:44:08.656571+0000","last_unstale":"2024-09-10T08:45:18.252007+0000","last_undegraded":"2024-09-10T08:45:18.252007+0000","last_fullsized":"2024-09-10T08:45:18.252007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:32.202740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261935+0000","last_change":"2024-09-10T08:44:17.725674+0000","last_active":"2024-09-10T08:45:18.261935+0000","last_peered":"2024-09-10T08:45:18.261935+0000","last_clean":"2024-09-10T08:45:18.261935+0000","last_became_active":"2024-09-10T08:44:17.725263+0000","last_became_peered":"2024-09-10T08:44:17.725263+0000","last_unstale":"2024-09-10T08:45:18.261935+0000","last_undegraded":"2024-09-10T08:45:18.261935+0000","last_fullsized":"2024-09-10T08:45:18.261935+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:18.183157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248867+0000","last_change":"2024-09-10T08:44:10.676871+0000","last_active":"2024-09-10T08:45:18.248867+0000","last_peered":"2024-09-10T08:45:18.248867+0000","last_clean":"2024-09-10T08:45:18.248867+0000","last_became_active":"2024-09-10T08:44:10.676540+0000","last_became_peered":"2024-09-10T08:44:10.676540+0000","last_unstale":"2024-09-10T08:45:18.248867+0000","last_undegraded":"2024-09-10T08:45:18.248867+0000","last_fullsized":"2024-09-10T08:45:18.248867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:50.964100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'35","reported_seq":181,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:44.295955+0000","last_change":"2024-09-10T08:44:08.660113+0000","last_active":"2024-09-10T08:45:44.295955+0000","last_peered":"2024-09-10T08:45:44.295955+0000","last_clean":"2024-09-10T08:45:44.295955+0000","last_became_active":"2024-09-10T08:44:08.659644+0000","last_became_peered":"2024-09-10T08:44:08.659644+0000","last_unstale":"2024-09-10T08:45:44.295955+0000","last_undegraded":"2024-09-10T08:45:44.295955+0000","last_fullsized":"2024-09-10T08:45:44.295955+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:01.605344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63484,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":112,"num_read_kb":118,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.266627+0000","last_change":"2024-09-10T08:44:17.723653+0000","last_active":"2024-09-10T08:45:18.266627+0000","last_peered":"2024-09-10T08:45:18.266627+0000","last_clean":"2024-09-10T08:45:18.266627+0000","last_became_active":"2024-09-10T08:44:17.723284+0000","last_became_peered":"2024-09-10T08:44:17.723284+0000","last_unstale":"2024-09-10T08:45:18.266627+0000","last_undegraded":"2024-09-10T08:45:18.266627+0000","last_fullsized":"2024-09-10T08:45:18.266627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:17.162526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248747+0000","last_change":"2024-09-10T08:44:10.681837+0000","last_active":"2024-09-10T08:45:18.248747+0000","last_peered":"2024-09-10T08:45:18.248747+0000","last_clean":"2024-09-10T08:45:18.248747+0000","last_became_active":"2024-09-10T08:44:10.681518+0000","last_became_peered":"2024-09-10T08:44:10.681518+0000","last_unstale":"2024-09-10T08:45:18.248747+0000","last_undegraded":"2024-09-10T08:45:18.248747+0000","last_fullsized":"2024-09-10T08:45:18.248747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:10.004215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.262072+0000","last_change":"2024-09-10T08:44:09.050394+0000","last_active":"2024-09-10T08:45:18.262072+0000","last_peered":"2024-09-10T08:45:18.262072+0000","last_clean":"2024-09-10T08:45:18.262072+0000","last_became_active":"2024-09-10T08:44:09.049969+0000","last_became_peered":"2024-09-10T08:44:09.049969+0000","last_unstale":"2024-09-10T08:45:18.262072+0000","last_undegraded":"2024-09-10T08:45:18.262072+0000","last_fullsized":"2024-09-10T08:45:18.262072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:43.519246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261264+0000","last_change":"2024-09-10T08:44:17.720318+0000","last_active":"2024-09-10T08:45:18.261264+0000","last_peered":"2024-09-10T08:45:18.261264+0000","last_clean":"2024-09-10T08:45:18.261264+0000","last_became_active":"2024-09-10T08:44:17.719598+0000","last_became_peered":"2024-09-10T08:44:17.719598+0000","last_unstale":"2024-09-10T08:45:18.261264+0000","last_undegraded":"2024-09-10T08:45:18.261264+0000","last_fullsized":"2024-09-10T08:45:18.261264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:15:49.520324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248538+0000","last_change":"2024-09-10T08:44:10.677340+0000","last_active":"2024-09-10T08:45:18.248538+0000","last_peered":"2024-09-10T08:45:18.248538+0000","last_clean":"2024-09-10T08:45:18.248538+0000","last_became_active":"2024-09-10T08:44:10.676628+0000","last_became_peered":"2024-09-10T08:44:10.676628+0000","last_unstale":"2024-09-10T08:45:18.248538+0000","last_undegraded":"2024-09-10T08:45:18.248538+0000","last_fullsized":"2024-09-10T08:45:18.248538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:59:17.924217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270431+0000","last_change":"2024-09-10T08:44:08.660254+0000","last_active":"2024-09-10T08:45:18.270431+0000","last_peered":"2024-09-10T08:45:18.270431+0000","last_clean":"2024-09-10T08:45:18.270431+0000","last_became_active":"2024-09-10T08:44:08.659667+0000","last_became_peered":"2024-09-10T08:44:08.659667+0000","last_unstale":"2024-09-10T08:45:18.270431+0000","last_undegraded":"2024-09-10T08:45:18.270431+0000","last_fullsized":"2024-09-10T08:45:18.270431+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:49:49.146867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.266387+0000","last_change":"2024-09-10T08:44:17.722373+0000","last_active":"2024-09-10T08:45:18.266387+0000","last_peered":"2024-09-10T08:45:18.266387+0000","last_clean":"2024-09-10T08:45:18.266387+0000","last_became_active":"2024-09-10T08:44:17.722035+0000","last_became_peered":"2024-09-10T08:44:17.722035+0000","last_unstale":"2024-09-10T08:45:18.266387+0000","last_undegraded":"2024-09-10T08:45:18.266387+0000","last_fullsized":"2024-09-10T08:45:18.266387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:30:36.187364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.249176+0000","last_change":"2024-09-10T08:44:10.672585+0000","last_active":"2024-09-10T08:45:18.249176+0000","last_peered":"2024-09-10T08:45:18.249176+0000","last_clean":"2024-09-10T08:45:18.249176+0000","last_became_active":"2024-09-10T08:44:10.671987+0000","last_became_peered":"2024-09-10T08:44:10.671987+0000","last_unstale":"2024-09-10T08:45:18.249176+0000","last_undegraded":"2024-09-10T08:45:18.249176+0000","last_fullsized":"2024-09-10T08:45:18.249176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:48.660750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":140,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:44.295983+0000","last_change":"2024-09-10T08:44:08.658069+0000","last_active":"2024-09-10T08:45:44.295983+0000","last_peered":"2024-09-10T08:45:44.295983+0000","last_clean":"2024-09-10T08:45:44.295983+0000","last_became_active":"2024-09-10T08:44:08.657702+0000","last_became_peered":"2024-09-10T08:44:08.657702+0000","last_unstale":"2024-09-10T08:45:44.295983+0000","last_undegraded":"2024-09-10T08:45:44.295983+0000","last_fullsized":"2024-09-10T08:45:44.295983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:17.383267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254196+0000","last_change":"2024-09-10T08:44:17.717984+0000","last_active":"2024-09-10T08:45:18.254196+0000","last_peered":"2024-09-10T08:45:18.254196+0000","last_clean":"2024-09-10T08:45:18.254196+0000","last_became_active":"2024-09-10T08:44:17.717581+0000","last_became_peered":"2024-09-10T08:44:17.717581+0000","last_unstale":"2024-09-10T08:45:18.254196+0000","last_undegraded":"2024-09-10T08:45:18.254196+0000","last_fullsized":"2024-09-10T08:45:18.254196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:25.330732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248233+0000","last_change":"2024-09-10T08:44:17.721244+0000","last_active":"2024-09-10T08:45:18.248233+0000","last_peered":"2024-09-10T08:45:18.248233+0000","last_clean":"2024-09-10T08:45:18.248233+0000","last_became_active":"2024-09-10T08:44:17.720884+0000","last_became_peered":"2024-09-10T08:44:17.720884+0000","last_unstale":"2024-09-10T08:45:18.248233+0000","last_undegraded":"2024-09-10T08:45:18.248233+0000","last_fullsized":"2024-09-10T08:45:18.248233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.769513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.266636+0000","last_change":"2024-09-10T08:44:10.667558+0000","last_active":"2024-09-10T08:45:18.266636+0000","last_peered":"2024-09-10T08:45:18.266636+0000","last_clean":"2024-09-10T08:45:18.266636+0000","last_became_active":"2024-09-10T08:44:10.667214+0000","last_became_peered":"2024-09-10T08:44:10.667214+0000","last_unstale":"2024-09-10T08:45:18.266636+0000","last_undegraded":"2024-09-10T08:45:18.266636+0000","last_fullsized":"2024-09-10T08:45:18.266636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:53.630790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261521+0000","last_change":"2024-09-10T08:44:08.657930+0000","last_active":"2024-09-10T08:45:18.261521+0000","last_peered":"2024-09-10T08:45:18.261521+0000","last_clean":"2024-09-10T08:45:18.261521+0000","last_became_active":"2024-09-10T08:44:08.657611+0000","last_became_peered":"2024-09-10T08:44:08.657611+0000","last_unstale":"2024-09-10T08:45:18.261521+0000","last_undegraded":"2024-09-10T08:45:18.261521+0000","last_fullsized":"2024-09-10T08:45:18.261521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:36.923786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252655+0000","last_change":"2024-09-10T08:44:17.726106+0000","last_active":"2024-09-10T08:45:18.252655+0000","last_peered":"2024-09-10T08:45:18.252655+0000","last_clean":"2024-09-10T08:45:18.252655+0000","last_became_active":"2024-09-10T08:44:17.725683+0000","last_became_peered":"2024-09-10T08:44:17.725683+0000","last_unstale":"2024-09-10T08:45:18.252655+0000","last_undegraded":"2024-09-10T08:45:18.252655+0000","last_fullsized":"2024-09-10T08:45:18.252655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:23.392794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254634+0000","last_change":"2024-09-10T08:44:10.672204+0000","last_active":"2024-09-10T08:45:18.254634+0000","last_peered":"2024-09-10T08:45:18.254634+0000","last_clean":"2024-09-10T08:45:18.254634+0000","last_became_active":"2024-09-10T08:44:10.671898+0000","last_became_peered":"2024-09-10T08:44:10.671898+0000","last_unstale":"2024-09-10T08:45:18.254634+0000","last_undegraded":"2024-09-10T08:45:18.254634+0000","last_fullsized":"2024-09-10T08:45:18.254634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:03.737357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.256851+0000","last_change":"2024-09-10T08:44:08.656620+0000","last_active":"2024-09-10T08:45:18.256851+0000","last_peered":"2024-09-10T08:45:18.256851+0000","last_clean":"2024-09-10T08:45:18.256851+0000","last_became_active":"2024-09-10T08:44:08.656306+0000","last_became_peered":"2024-09-10T08:44:08.656306+0000","last_unstale":"2024-09-10T08:45:18.256851+0000","last_undegraded":"2024-09-10T08:45:18.256851+0000","last_fullsized":"2024-09-10T08:45:18.256851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:57.081909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.258679+0000","last_change":"2024-09-10T08:44:08.660178+0000","last_active":"2024-09-10T08:45:18.258679+0000","last_peered":"2024-09-10T08:45:18.258679+0000","last_clean":"2024-09-10T08:45:18.258679+0000","last_became_active":"2024-09-10T08:44:08.659839+0000","last_became_peered":"2024-09-10T08:44:08.659839+0000","last_unstale":"2024-09-10T08:45:18.258679+0000","last_undegraded":"2024-09-10T08:45:18.258679+0000","last_fullsized":"2024-09-10T08:45:18.258679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:17.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.262119+0000","last_change":"2024-09-10T08:44:10.683683+0000","last_active":"2024-09-10T08:45:18.262119+0000","last_peered":"2024-09-10T08:45:18.262119+0000","last_clean":"2024-09-10T08:45:18.262119+0000","last_became_active":"2024-09-10T08:44:10.683337+0000","last_became_peered":"2024-09-10T08:44:10.683337+0000","last_unstale":"2024-09-10T08:45:18.262119+0000","last_undegraded":"2024-09-10T08:45:18.262119+0000","last_fullsized":"2024-09-10T08:45:18.262119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:54.863098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252195+0000","last_change":"2024-09-10T08:44:17.721602+0000","last_active":"2024-09-10T08:45:18.252195+0000","last_peered":"2024-09-10T08:45:18.252195+0000","last_clean":"2024-09-10T08:45:18.252195+0000","last_became_active":"2024-09-10T08:44:17.721194+0000","last_became_peered":"2024-09-10T08:44:17.721194+0000","last_unstale":"2024-09-10T08:45:18.252195+0000","last_undegraded":"2024-09-10T08:45:18.252195+0000","last_fullsized":"2024-09-10T08:45:18.252195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:39.298494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":93,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:43.294961+0000","last_change":"2024-09-10T08:44:08.661943+0000","last_active":"2024-09-10T08:45:43.294961+0000","last_peered":"2024-09-10T08:45:43.294961+0000","last_clean":"2024-09-10T08:45:43.294961+0000","last_became_active":"2024-09-10T08:44:08.661587+0000","last_became_peered":"2024-09-10T08:44:08.661587+0000","last_unstale":"2024-09-10T08:45:43.294961+0000","last_undegraded":"2024-09-10T08:45:43.294961+0000","last_fullsized":"2024-09-10T08:45:43.294961+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:05.771641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248419+0000","last_change":"2024-09-10T08:44:10.673522+0000","last_active":"2024-09-10T08:45:18.248419+0000","last_peered":"2024-09-10T08:45:18.248419+0000","last_clean":"2024-09-10T08:45:18.248419+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:45:18.248419+0000","last_undegraded":"2024-09-10T08:45:18.248419+0000","last_fullsized":"2024-09-10T08:45:18.248419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:03.006144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.251983+0000","last_change":"2024-09-10T08:44:17.721306+0000","last_active":"2024-09-10T08:45:18.251983+0000","last_peered":"2024-09-10T08:45:18.251983+0000","last_clean":"2024-09-10T08:45:18.251983+0000","last_became_active":"2024-09-10T08:44:17.720401+0000","last_became_peered":"2024-09-10T08:44:17.720401+0000","last_unstale":"2024-09-10T08:45:18.251983+0000","last_undegraded":"2024-09-10T08:45:18.251983+0000","last_fullsized":"2024-09-10T08:45:18.251983+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:11.589371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252336+0000","last_change":"2024-09-10T08:44:10.678425+0000","last_active":"2024-09-10T08:45:18.252336+0000","last_peered":"2024-09-10T08:45:18.252336+0000","last_clean":"2024-09-10T08:45:18.252336+0000","last_became_active":"2024-09-10T08:44:10.677187+0000","last_became_peered":"2024-09-10T08:44:10.677187+0000","last_unstale":"2024-09-10T08:45:18.252336+0000","last_undegraded":"2024-09-10T08:45:18.252336+0000","last_fullsized":"2024-09-10T08:45:18.252336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:06.527530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252431+0000","last_change":"2024-09-10T08:44:10.674178+0000","last_active":"2024-09-10T08:45:18.252431+0000","last_peered":"2024-09-10T08:45:18.252431+0000","last_clean":"2024-09-10T08:45:18.252431+0000","last_became_active":"2024-09-10T08:44:10.673843+0000","last_became_peered":"2024-09-10T08:44:10.673843+0000","last_unstale":"2024-09-10T08:45:18.252431+0000","last_undegraded":"2024-09-10T08:45:18.252431+0000","last_fullsized":"2024-09-10T08:45:18.252431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:35.023663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.251565+0000","last_change":"2024-09-10T08:44:10.673271+0000","last_active":"2024-09-10T08:45:18.251565+0000","last_peered":"2024-09-10T08:45:18.251565+0000","last_clean":"2024-09-10T08:45:18.251565+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:45:18.251565+0000","last_undegraded":"2024-09-10T08:45:18.251565+0000","last_fullsized":"2024-09-10T08:45:18.251565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:21:26.606820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248019+0000","last_change":"2024-09-10T08:44:10.671348+0000","last_active":"2024-09-10T08:45:18.248019+0000","last_peered":"2024-09-10T08:45:18.248019+0000","last_clean":"2024-09-10T08:45:18.248019+0000","last_became_active":"2024-09-10T08:44:10.670623+0000","last_became_peered":"2024-09-10T08:44:10.670623+0000","last_unstale":"2024-09-10T08:45:18.248019+0000","last_undegraded":"2024-09-10T08:45:18.248019+0000","last_fullsized":"2024-09-10T08:45:18.248019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:06.145694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257532+0000","last_change":"2024-09-10T08:44:10.666011+0000","last_active":"2024-09-10T08:45:18.257532+0000","last_peered":"2024-09-10T08:45:18.257532+0000","last_clean":"2024-09-10T08:45:18.257532+0000","last_became_active":"2024-09-10T08:44:10.665664+0000","last_became_peered":"2024-09-10T08:44:10.665664+0000","last_unstale":"2024-09-10T08:45:18.257532+0000","last_undegraded":"2024-09-10T08:45:18.257532+0000","last_fullsized":"2024-09-10T08:45:18.257532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:31.542102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.251543+0000","last_change":"2024-09-10T08:44:10.675960+0000","last_active":"2024-09-10T08:45:18.251543+0000","last_peered":"2024-09-10T08:45:18.251543+0000","last_clean":"2024-09-10T08:45:18.251543+0000","last_became_active":"2024-09-10T08:44:10.675587+0000","last_became_peered":"2024-09-10T08:44:10.675587+0000","last_unstale":"2024-09-10T08:45:18.251543+0000","last_undegraded":"2024-09-10T08:45:18.251543+0000","last_fullsized":"2024-09-10T08:45:18.251543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:54.634619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252919+0000","last_change":"2024-09-10T08:44:10.681094+0000","last_active":"2024-09-10T08:45:18.252919+0000","last_peered":"2024-09-10T08:45:18.252919+0000","last_clean":"2024-09-10T08:45:18.252919+0000","last_became_active":"2024-09-10T08:44:10.678585+0000","last_became_peered":"2024-09-10T08:44:10.678585+0000","last_unstale":"2024-09-10T08:45:18.252919+0000","last_undegraded":"2024-09-10T08:45:18.252919+0000","last_fullsized":"2024-09-10T08:45:18.252919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:14.951949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254333+0000","last_change":"2024-09-10T08:44:10.675473+0000","last_active":"2024-09-10T08:45:18.254333+0000","last_peered":"2024-09-10T08:45:18.254333+0000","last_clean":"2024-09-10T08:45:18.254333+0000","last_became_active":"2024-09-10T08:44:10.674155+0000","last_became_peered":"2024-09-10T08:44:10.674155+0000","last_unstale":"2024-09-10T08:45:18.254333+0000","last_undegraded":"2024-09-10T08:45:18.254333+0000","last_fullsized":"2024-09-10T08:45:18.254333+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:55.940235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261299+0000","last_change":"2024-09-10T08:44:10.680162+0000","last_active":"2024-09-10T08:45:18.261299+0000","last_peered":"2024-09-10T08:45:18.261299+0000","last_clean":"2024-09-10T08:45:18.261299+0000","last_became_active":"2024-09-10T08:44:10.679800+0000","last_became_peered":"2024-09-10T08:44:10.679800+0000","last_unstale":"2024-09-10T08:45:18.261299+0000","last_undegraded":"2024-09-10T08:45:18.261299+0000","last_fullsized":"2024-09-10T08:45:18.261299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:12.319813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254847+0000","last_change":"2024-09-10T08:44:10.674724+0000","last_active":"2024-09-10T08:45:18.254847+0000","last_peered":"2024-09-10T08:45:18.254847+0000","last_clean":"2024-09-10T08:45:18.254847+0000","last_became_active":"2024-09-10T08:44:10.674389+0000","last_became_peered":"2024-09-10T08:44:10.674389+0000","last_unstale":"2024-09-10T08:45:18.254847+0000","last_undegraded":"2024-09-10T08:45:18.254847+0000","last_fullsized":"2024-09-10T08:45:18.254847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:52.123186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.249100+0000","last_change":"2024-09-10T08:44:10.672805+0000","last_active":"2024-09-10T08:45:18.249100+0000","last_peered":"2024-09-10T08:45:18.249100+0000","last_clean":"2024-09-10T08:45:18.249100+0000","last_became_active":"2024-09-10T08:44:10.672508+0000","last_became_peered":"2024-09-10T08:44:10.672508+0000","last_unstale":"2024-09-10T08:45:18.249100+0000","last_undegraded":"2024-09-10T08:45:18.249100+0000","last_fullsized":"2024-09-10T08:45:18.249100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:48.558633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257040+0000","last_change":"2024-09-10T08:44:10.677504+0000","last_active":"2024-09-10T08:45:18.257040+0000","last_peered":"2024-09-10T08:45:18.257040+0000","last_clean":"2024-09-10T08:45:18.257040+0000","last_became_active":"2024-09-10T08:44:10.676822+0000","last_became_peered":"2024-09-10T08:44:10.676822+0000","last_unstale":"2024-09-10T08:45:18.257040+0000","last_undegraded":"2024-09-10T08:45:18.257040+0000","last_fullsized":"2024-09-10T08:45:18.257040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:05:48.293333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.268208+0000","last_change":"2024-09-10T08:45:18.268208+0000","last_active":"2024-09-10T08:45:18.268208+0000","last_peered":"2024-09-10T08:45:18.268208+0000","last_clean":"2024-09-10T08:45:18.268208+0000","last_became_active":"2024-09-10T08:45:18.267700+0000","last_became_peered":"2024-09-10T08:45:18.267700+0000","last_unstale":"2024-09-10T08:45:18.268208+0000","last_undegraded":"2024-09-10T08:45:18.268208+0000","last_fullsized":"2024-09-10T08:45:18.268208+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:54.487624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252556+0000","last_change":"2024-09-10T08:44:10.670615+0000","last_active":"2024-09-10T08:45:18.252556+0000","last_peered":"2024-09-10T08:45:18.252556+0000","last_clean":"2024-09-10T08:45:18.252556+0000","last_became_active":"2024-09-10T08:44:10.670032+0000","last_became_peered":"2024-09-10T08:44:10.670032+0000","last_unstale":"2024-09-10T08:45:18.252556+0000","last_undegraded":"2024-09-10T08:45:18.252556+0000","last_fullsized":"2024-09-10T08:45:18.252556+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:31.078788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.267175+0000","last_change":"2024-09-10T08:44:10.672367+0000","last_active":"2024-09-10T08:45:18.267175+0000","last_peered":"2024-09-10T08:45:18.267175+0000","last_clean":"2024-09-10T08:45:18.267175+0000","last_became_active":"2024-09-10T08:44:10.672040+0000","last_became_peered":"2024-09-10T08:44:10.672040+0000","last_unstale":"2024-09-10T08:45:18.267175+0000","last_undegraded":"2024-09-10T08:45:18.267175+0000","last_fullsized":"2024-09-10T08:45:18.267175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:10.634708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252491+0000","last_change":"2024-09-10T08:44:10.676375+0000","last_active":"2024-09-10T08:45:18.252491+0000","last_peered":"2024-09-10T08:45:18.252491+0000","last_clean":"2024-09-10T08:45:18.252491+0000","last_became_active":"2024-09-10T08:44:10.676022+0000","last_became_peered":"2024-09-10T08:44:10.676022+0000","last_unstale":"2024-09-10T08:45:18.252491+0000","last_undegraded":"2024-09-10T08:45:18.252491+0000","last_fullsized":"2024-09-10T08:45:18.252491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:02.681954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248238+0000","last_change":"2024-09-10T08:44:10.673378+0000","last_active":"2024-09-10T08:45:18.248238+0000","last_peered":"2024-09-10T08:45:18.248238+0000","last_clean":"2024-09-10T08:45:18.248238+0000","last_became_active":"2024-09-10T08:44:10.673089+0000","last_became_peered":"2024-09-10T08:44:10.673089+0000","last_unstale":"2024-09-10T08:45:18.248238+0000","last_undegraded":"2024-09-10T08:45:18.248238+0000","last_fullsized":"2024-09-10T08:45:18.248238+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:36:19.258651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.247840+0000","last_change":"2024-09-10T08:44:10.671557+0000","last_active":"2024-09-10T08:45:18.247840+0000","last_peered":"2024-09-10T08:45:18.247840+0000","last_clean":"2024-09-10T08:45:18.247840+0000","last_became_active":"2024-09-10T08:44:10.671248+0000","last_became_peered":"2024-09-10T08:44:10.671248+0000","last_unstale":"2024-09-10T08:45:18.247840+0000","last_undegraded":"2024-09-10T08:45:18.247840+0000","last_fullsized":"2024-09-10T08:45:18.247840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:54.261525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.247887+0000","last_change":"2024-09-10T08:44:10.674672+0000","last_active":"2024-09-10T08:45:18.247887+0000","last_peered":"2024-09-10T08:45:18.247887+0000","last_clean":"2024-09-10T08:45:18.247887+0000","last_became_active":"2024-09-10T08:44:10.673415+0000","last_became_peered":"2024-09-10T08:44:10.673415+0000","last_unstale":"2024-09-10T08:45:18.247887+0000","last_undegraded":"2024-09-10T08:45:18.247887+0000","last_fullsized":"2024-09-10T08:45:18.247887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:59:14.179664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257050+0000","last_change":"2024-09-10T08:44:10.675293+0000","last_active":"2024-09-10T08:45:18.257050+0000","last_peered":"2024-09-10T08:45:18.257050+0000","last_clean":"2024-09-10T08:45:18.257050+0000","last_became_active":"2024-09-10T08:44:10.674924+0000","last_became_peered":"2024-09-10T08:44:10.674924+0000","last_unstale":"2024-09-10T08:45:18.257050+0000","last_undegraded":"2024-09-10T08:45:18.257050+0000","last_fullsized":"2024-09-10T08:45:18.257050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:56.027320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261968+0000","last_change":"2024-09-10T08:44:10.674605+0000","last_active":"2024-09-10T08:45:18.261968+0000","last_peered":"2024-09-10T08:45:18.261968+0000","last_clean":"2024-09-10T08:45:18.261968+0000","last_became_active":"2024-09-10T08:44:10.674057+0000","last_became_peered":"2024-09-10T08:44:10.674057+0000","last_unstale":"2024-09-10T08:45:18.261968+0000","last_undegraded":"2024-09-10T08:45:18.261968+0000","last_fullsized":"2024-09-10T08:45:18.261968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:09.473003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270641+0000","last_change":"2024-09-10T08:44:10.677410+0000","last_active":"2024-09-10T08:45:18.270641+0000","last_peered":"2024-09-10T08:45:18.270641+0000","last_clean":"2024-09-10T08:45:18.270641+0000","last_became_active":"2024-09-10T08:44:10.677109+0000","last_became_peered":"2024-09-10T08:44:10.677109+0000","last_unstale":"2024-09-10T08:45:18.270641+0000","last_undegraded":"2024-09-10T08:45:18.270641+0000","last_fullsized":"2024-09-10T08:45:18.270641+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:45.455425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.259103+0000","last_change":"2024-09-10T08:44:10.669706+0000","last_active":"2024-09-10T08:45:18.259103+0000","last_peered":"2024-09-10T08:45:18.259103+0000","last_clean":"2024-09-10T08:45:18.259103+0000","last_became_active":"2024-09-10T08:44:10.669440+0000","last_became_peered":"2024-09-10T08:44:10.669440+0000","last_unstale":"2024-09-10T08:45:18.259103+0000","last_undegraded":"2024-09-10T08:45:18.259103+0000","last_fullsized":"2024-09-10T08:45:18.259103+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:31:55.610651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.249193+0000","last_change":"2024-09-10T08:44:10.676482+0000","last_active":"2024-09-10T08:45:18.249193+0000","last_peered":"2024-09-10T08:45:18.249193+0000","last_clean":"2024-09-10T08:45:18.249193+0000","last_became_active":"2024-09-10T08:44:10.675983+0000","last_became_peered":"2024-09-10T08:44:10.675983+0000","last_unstale":"2024-09-10T08:45:18.249193+0000","last_undegraded":"2024-09-10T08:45:18.249193+0000","last_fullsized":"2024-09-10T08:45:18.249193+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:01.269283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254722+0000","last_change":"2024-09-10T08:44:10.675681+0000","last_active":"2024-09-10T08:45:18.254722+0000","last_peered":"2024-09-10T08:45:18.254722+0000","last_clean":"2024-09-10T08:45:18.254722+0000","last_became_active":"2024-09-10T08:44:10.675357+0000","last_became_peered":"2024-09-10T08:44:10.675357+0000","last_unstale":"2024-09-10T08:45:18.254722+0000","last_undegraded":"2024-09-10T08:45:18.254722+0000","last_fullsized":"2024-09-10T08:45:18.254722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:26.138333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270056+0000","last_change":"2024-09-10T08:44:10.680819+0000","last_active":"2024-09-10T08:45:18.270056+0000","last_peered":"2024-09-10T08:45:18.270056+0000","last_clean":"2024-09-10T08:45:18.270056+0000","last_became_active":"2024-09-10T08:44:10.680491+0000","last_became_peered":"2024-09-10T08:44:10.680491+0000","last_unstale":"2024-09-10T08:45:18.270056+0000","last_undegraded":"2024-09-10T08:45:18.270056+0000","last_fullsized":"2024-09-10T08:45:18.270056+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:17.044776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.258679+0000","last_change":"2024-09-10T08:44:10.667376+0000","last_active":"2024-09-10T08:45:18.258679+0000","last_peered":"2024-09-10T08:45:18.258679+0000","last_clean":"2024-09-10T08:45:18.258679+0000","last_became_active":"2024-09-10T08:44:10.666648+0000","last_became_peered":"2024-09-10T08:44:10.666648+0000","last_unstale":"2024-09-10T08:45:18.258679+0000","last_undegraded":"2024-09-10T08:45:18.258679+0000","last_fullsized":"2024-09-10T08:45:18.258679+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:33:12.017550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248402+0000","last_change":"2024-09-10T08:44:10.668665+0000","last_active":"2024-09-10T08:45:18.248402+0000","last_peered":"2024-09-10T08:45:18.248402+0000","last_clean":"2024-09-10T08:45:18.248402+0000","last_became_active":"2024-09-10T08:44:10.666031+0000","last_became_peered":"2024-09-10T08:44:10.666031+0000","last_unstale":"2024-09-10T08:45:18.248402+0000","last_undegraded":"2024-09-10T08:45:18.248402+0000","last_fullsized":"2024-09-10T08:45:18.248402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:39:53.991463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261851+0000","last_change":"2024-09-10T08:44:10.676495+0000","last_active":"2024-09-10T08:45:18.261851+0000","last_peered":"2024-09-10T08:45:18.261851+0000","last_clean":"2024-09-10T08:45:18.261851+0000","last_became_active":"2024-09-10T08:44:10.676087+0000","last_became_peered":"2024-09-10T08:44:10.676087+0000","last_unstale":"2024-09-10T08:45:18.261851+0000","last_undegraded":"2024-09-10T08:45:18.261851+0000","last_fullsized":"2024-09-10T08:45:18.261851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:41.828157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248048+0000","last_change":"2024-09-10T08:44:10.672835+0000","last_active":"2024-09-10T08:45:18.248048+0000","last_peered":"2024-09-10T08:45:18.248048+0000","last_clean":"2024-09-10T08:45:18.248048+0000","last_became_active":"2024-09-10T08:44:10.672537+0000","last_became_peered":"2024-09-10T08:44:10.672537+0000","last_unstale":"2024-09-10T08:45:18.248048+0000","last_undegraded":"2024-09-10T08:45:18.248048+0000","last_fullsized":"2024-09-10T08:45:18.248048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:06.121460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248222+0000","last_change":"2024-09-10T08:44:10.675968+0000","last_active":"2024-09-10T08:45:18.248222+0000","last_peered":"2024-09-10T08:45:18.248222+0000","last_clean":"2024-09-10T08:45:18.248222+0000","last_became_active":"2024-09-10T08:44:10.675628+0000","last_became_peered":"2024-09-10T08:44:10.675628+0000","last_unstale":"2024-09-10T08:45:18.248222+0000","last_undegraded":"2024-09-10T08:45:18.248222+0000","last_fullsized":"2024-09-10T08:45:18.248222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:40.261336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252266+0000","last_change":"2024-09-10T08:44:10.673151+0000","last_active":"2024-09-10T08:45:18.252266+0000","last_peered":"2024-09-10T08:45:18.252266+0000","last_clean":"2024-09-10T08:45:18.252266+0000","last_became_active":"2024-09-10T08:44:10.672833+0000","last_became_peered":"2024-09-10T08:44:10.672833+0000","last_unstale":"2024-09-10T08:45:18.252266+0000","last_undegraded":"2024-09-10T08:45:18.252266+0000","last_fullsized":"2024-09-10T08:45:18.252266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:38.556222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70299,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":551,"num_read_kb":353,"num_write":248,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":210897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514529,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1024000,"data_stored":504097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743462,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28712,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716440,"statfs":{"total":95995035648,"available":95965634560,"internally_reserved":0,"allocated":1720320,"data_stored":1154004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906987,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1052672,"data_stored":507730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135919,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1044480,"data_stored":504012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364852,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1642496,"data_stored":1100118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528378,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1024000,"data_stored":499138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36964,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708188,"statfs":{"total":95995035648,"available":95957184512,"internally_reserved":0,"allocated":1650688,"data_stored":1095700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1032192,"data_stored":503212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1036288,"data_stored":498846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":564265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1093632,"data_stored":563652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1048576,"data_stored":500987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:45:48.085 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:45:48.334 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:48.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.848+0000 7f0d1065c640 1 -- 172.21.15.172:0/337407003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d08101de0 msgr2=0x7f0d081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:48.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.848+0000 7f0d1065c640 1 --2- 172.21.15.172:0/337407003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d08101de0 0x7f0d081021c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009900 tx=0x7f0d0402a410 comp rx=0 tx=0).stop 2024-09-10T08:45:48.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.848+0000 7f0d1065c640 1 -- 172.21.15.172:0/337407003 shutdown_connections 2024-09-10T08:45:48.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.848+0000 7f0d1065c640 1 --2- 172.21.15.172:0/337407003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d08106f10 0x7f0d0810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:48.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.848+0000 7f0d1065c640 1 --2- 172.21.15.172:0/337407003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d08102790 0x7f0d081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:48.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.848+0000 7f0d1065c640 1 --2- 172.21.15.172:0/337407003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d08101de0 0x7f0d081021c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:48.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.848+0000 7f0d1065c640 1 -- 172.21.15.172:0/337407003 >> 172.21.15.172:0/337407003 conn(0x7f0d080fd880 msgr2=0x7f0d080ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:48.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.848+0000 7f0d1065c640 1 -- 172.21.15.172:0/337407003 shutdown_connections 2024-09-10T08:45:48.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.848+0000 7f0d1065c640 1 -- 172.21.15.172:0/337407003 wait complete. 2024-09-10T08:45:48.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.848+0000 7f0d1065c640 1 Processor -- start 2024-09-10T08:45:48.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d1065c640 1 -- start start 2024-09-10T08:45:48.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d1065c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d08101de0 0x7f0d08198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:48.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d1065c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d08102790 0x7f0d08198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:48.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d1065c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d08106f10 0x7f0d0819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:48.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d1065c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08111e20 con 0x7f0d08102790 2024-09-10T08:45:48.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d1065c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08111ca0 con 0x7f0d08106f10 2024-09-10T08:45:48.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d1065c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08111fa0 con 0x7f0d08101de0 2024-09-10T08:45:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d0dbf7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d08102790 0x7f0d08198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d0dbf7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d08102790 0x7f0d08198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52472/0 (socket says 172.21.15.172:52472) 2024-09-10T08:45:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d0dbf7640 1 -- 172.21.15.172:0/981550829 learned_addr learned my addr 172.21.15.172:0/981550829 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d0ebf9640 1 --2- 172.21.15.172:0/981550829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d08106f10 0x7f0d0819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d0e3f8640 1 --2- 172.21.15.172:0/981550829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d08101de0 0x7f0d08198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d0dbf7640 1 -- 172.21.15.172:0/981550829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d08101de0 msgr2=0x7f0d08198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d0dbf7640 1 --2- 172.21.15.172:0/981550829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d08101de0 0x7f0d08198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d0dbf7640 1 -- 172.21.15.172:0/981550829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d08106f10 msgr2=0x7f0d0819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d0dbf7640 1 --2- 172.21.15.172:0/981550829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d08106f10 0x7f0d0819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:48.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d0dbf7640 1 -- 172.21.15.172:0/981550829 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0819d2f0 con 0x7f0d08102790 2024-09-10T08:45:48.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.849+0000 7f0d0dbf7640 1 --2- 172.21.15.172:0/981550829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d08102790 0x7f0d08198880 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800d950 tx=0x7f0cf800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:48.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.850+0000 7f0cff7fe640 1 -- 172.21.15.172:0/981550829 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8017030 con 0x7f0d08102790 2024-09-10T08:45:48.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.850+0000 7f0cff7fe640 1 -- 172.21.15.172:0/981550829 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cf8004020 con 0x7f0d08102790 2024-09-10T08:45:48.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.850+0000 7f0d1065c640 1 -- 172.21.15.172:0/981550829 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0819d5e0 con 0x7f0d08102790 2024-09-10T08:45:48.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.850+0000 7f0cff7fe640 1 -- 172.21.15.172:0/981550829 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80047a0 con 0x7f0d08102790 2024-09-10T08:45:48.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.850+0000 7f0d1065c640 1 -- 172.21.15.172:0/981550829 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d081a4ec0 con 0x7f0d08102790 2024-09-10T08:45:48.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.851+0000 7f0d1065c640 1 -- 172.21.15.172:0/981550829 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d081021c0 con 0x7f0d08102790 2024-09-10T08:45:48.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.852+0000 7f0cff7fe640 1 -- 172.21.15.172:0/981550829 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0cf80041d0 con 0x7f0d08102790 2024-09-10T08:45:48.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.852+0000 7f0cff7fe640 1 --2- 172.21.15.172:0/981550829 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0cf40a2c90 0x7f0cf40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:48.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.853+0000 7f0d0e3f8640 1 --2- 172.21.15.172:0/981550829 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0cf40a2c90 0x7f0cf40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:48.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.853+0000 7f0cff7fe640 1 -- 172.21.15.172:0/981550829 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f0cf80cee70 con 0x7f0d08102790 2024-09-10T08:45:48.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.853+0000 7f0d0e3f8640 1 --2- 172.21.15.172:0/981550829 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0cf40a2c90 0x7f0cf40a5150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0d04031440 tx=0x7f0d04005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:48.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:48.857+0000 7f0cff7fe640 1 -- 172.21.15.172:0/981550829 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0cf809b930 con 0x7f0d08102790 2024-09-10T08:45:49.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:49.020+0000 7f0d1065c640 1 -- 172.21.15.172:0/981550829 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0d08062630 con 0x7f0cf40a2c90 2024-09-10T08:45:49.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:49.029+0000 7f0cff7fe640 1 -- 172.21.15.172:0/981550829 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340394 (secure 0 0 0) 0x7f0d08062630 con 0x7f0cf40a2c90 2024-09-10T08:45:49.031 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:45:49.035 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-09-10T08:45:49.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:49.032+0000 7f0d1065c640 1 -- 172.21.15.172:0/981550829 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0cf40a2c90 msgr2=0x7f0cf40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:49.032+0000 7f0d1065c640 1 --2- 172.21.15.172:0/981550829 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0cf40a2c90 0x7f0cf40a5150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0d04031440 tx=0x7f0d04005b50 comp rx=0 tx=0).stop 2024-09-10T08:45:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:49.032+0000 7f0d1065c640 1 -- 172.21.15.172:0/981550829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d08102790 msgr2=0x7f0d08198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:49.032+0000 7f0d1065c640 1 --2- 172.21.15.172:0/981550829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d08102790 0x7f0d08198880 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800d950 tx=0x7f0cf800de10 comp rx=0 tx=0).stop 2024-09-10T08:45:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:49.032+0000 7f0d1065c640 1 -- 172.21.15.172:0/981550829 shutdown_connections 2024-09-10T08:45:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:49.032+0000 7f0d1065c640 1 --2- 172.21.15.172:0/981550829 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0cf40a2c90 0x7f0cf40a5150 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:49.032+0000 7f0d1065c640 1 --2- 172.21.15.172:0/981550829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d08106f10 0x7f0d0819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:49.032+0000 7f0d1065c640 1 --2- 172.21.15.172:0/981550829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d08102790 0x7f0d08198880 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:49.032+0000 7f0d1065c640 1 --2- 172.21.15.172:0/981550829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d08101de0 0x7f0d08198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:49.032+0000 7f0d1065c640 1 -- 172.21.15.172:0/981550829 >> 172.21.15.172:0/981550829 conn(0x7f0d080fd880 msgr2=0x7f0d080fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:49.032+0000 7f0d1065c640 1 -- 172.21.15.172:0/981550829 shutdown_connections 2024-09-10T08:45:49.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:49.032+0000 7f0d1065c640 1 -- 172.21.15.172:0/981550829 wait complete. 2024-09-10T08:45:49.795 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-10T08:45:47.150041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660901,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":677,"num_read_kb":462,"num_write":431,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":352,"ondisk_log_size":352,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375944,"kb_used_data":14144,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565880,"statfs":{"total":1151940427776,"available":1151555461120,"internally_reserved":0,"allocated":14483456,"data_stored":7995761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011219"},"pg_stats":[{"pgid":"2.1f","version":"88'13","reported_seq":99,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:44.299573+0000","last_change":"2024-09-10T08:44:08.657737+0000","last_active":"2024-09-10T08:45:44.299573+0000","last_peered":"2024-09-10T08:45:44.299573+0000","last_clean":"2024-09-10T08:45:44.299573+0000","last_became_active":"2024-09-10T08:44:08.657473+0000","last_became_peered":"2024-09-10T08:44:08.657473+0000","last_unstale":"2024-09-10T08:45:44.299573+0000","last_undegraded":"2024-09-10T08:45:44.299573+0000","last_fullsized":"2024-09-10T08:45:44.299573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:40.757343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254491+0000","last_change":"2024-09-10T08:44:10.675090+0000","last_active":"2024-09-10T08:45:18.254491+0000","last_peered":"2024-09-10T08:45:18.254491+0000","last_clean":"2024-09-10T08:45:18.254491+0000","last_became_active":"2024-09-10T08:44:10.674612+0000","last_became_peered":"2024-09-10T08:44:10.674612+0000","last_unstale":"2024-09-10T08:45:18.254491+0000","last_undegraded":"2024-09-10T08:45:18.254491+0000","last_fullsized":"2024-09-10T08:45:18.254491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:46.821080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.251619+0000","last_change":"2024-09-10T08:44:17.721559+0000","last_active":"2024-09-10T08:45:18.251619+0000","last_peered":"2024-09-10T08:45:18.251619+0000","last_clean":"2024-09-10T08:45:18.251619+0000","last_became_active":"2024-09-10T08:44:17.721182+0000","last_became_peered":"2024-09-10T08:44:17.721182+0000","last_unstale":"2024-09-10T08:45:18.251619+0000","last_undegraded":"2024-09-10T08:45:18.251619+0000","last_fullsized":"2024-09-10T08:45:18.251619+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:18.791489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254739+0000","last_change":"2024-09-10T08:44:09.049171+0000","last_active":"2024-09-10T08:45:18.254739+0000","last_peered":"2024-09-10T08:45:18.254739+0000","last_clean":"2024-09-10T08:45:18.254739+0000","last_became_active":"2024-09-10T08:44:09.048806+0000","last_became_peered":"2024-09-10T08:44:09.048806+0000","last_unstale":"2024-09-10T08:45:18.254739+0000","last_undegraded":"2024-09-10T08:45:18.254739+0000","last_fullsized":"2024-09-10T08:45:18.254739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:21.023460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248482+0000","last_change":"2024-09-10T08:44:10.677569+0000","last_active":"2024-09-10T08:45:18.248482+0000","last_peered":"2024-09-10T08:45:18.248482+0000","last_clean":"2024-09-10T08:45:18.248482+0000","last_became_active":"2024-09-10T08:44:10.677048+0000","last_became_peered":"2024-09-10T08:44:10.677048+0000","last_unstale":"2024-09-10T08:45:18.248482+0000","last_undegraded":"2024-09-10T08:45:18.248482+0000","last_fullsized":"2024-09-10T08:45:18.248482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:35.523063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.260396+0000","last_change":"2024-09-10T08:44:17.720138+0000","last_active":"2024-09-10T08:45:18.260396+0000","last_peered":"2024-09-10T08:45:18.260396+0000","last_clean":"2024-09-10T08:45:18.260396+0000","last_became_active":"2024-09-10T08:44:17.719344+0000","last_became_peered":"2024-09-10T08:44:17.719344+0000","last_unstale":"2024-09-10T08:45:18.260396+0000","last_undegraded":"2024-09-10T08:45:18.260396+0000","last_fullsized":"2024-09-10T08:45:18.260396+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:11.758058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.249040+0000","last_change":"2024-09-10T08:44:08.660099+0000","last_active":"2024-09-10T08:45:18.249040+0000","last_peered":"2024-09-10T08:45:18.249040+0000","last_clean":"2024-09-10T08:45:18.249040+0000","last_became_active":"2024-09-10T08:44:08.659745+0000","last_became_peered":"2024-09-10T08:44:08.659745+0000","last_unstale":"2024-09-10T08:45:18.249040+0000","last_undegraded":"2024-09-10T08:45:18.249040+0000","last_fullsized":"2024-09-10T08:45:18.249040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:43.960717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248650+0000","last_change":"2024-09-10T08:44:10.675583+0000","last_active":"2024-09-10T08:45:18.248650+0000","last_peered":"2024-09-10T08:45:18.248650+0000","last_clean":"2024-09-10T08:45:18.248650+0000","last_became_active":"2024-09-10T08:44:10.675296+0000","last_became_peered":"2024-09-10T08:44:10.675296+0000","last_unstale":"2024-09-10T08:45:18.248650+0000","last_undegraded":"2024-09-10T08:45:18.248650+0000","last_fullsized":"2024-09-10T08:45:18.248650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:31:51.147862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261671+0000","last_change":"2024-09-10T08:44:17.720104+0000","last_active":"2024-09-10T08:45:18.261671+0000","last_peered":"2024-09-10T08:45:18.261671+0000","last_clean":"2024-09-10T08:45:18.261671+0000","last_became_active":"2024-09-10T08:44:17.719607+0000","last_became_peered":"2024-09-10T08:44:17.719607+0000","last_unstale":"2024-09-10T08:45:18.261671+0000","last_undegraded":"2024-09-10T08:45:18.261671+0000","last_fullsized":"2024-09-10T08:45:18.261671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:52.976717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248702+0000","last_change":"2024-09-10T08:44:08.658588+0000","last_active":"2024-09-10T08:45:18.248702+0000","last_peered":"2024-09-10T08:45:18.248702+0000","last_clean":"2024-09-10T08:45:18.248702+0000","last_became_active":"2024-09-10T08:44:08.658053+0000","last_became_peered":"2024-09-10T08:44:08.658053+0000","last_unstale":"2024-09-10T08:45:18.248702+0000","last_undegraded":"2024-09-10T08:45:18.248702+0000","last_fullsized":"2024-09-10T08:45:18.248702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:26.744943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261968+0000","last_change":"2024-09-10T08:44:10.681978+0000","last_active":"2024-09-10T08:45:18.261968+0000","last_peered":"2024-09-10T08:45:18.261968+0000","last_clean":"2024-09-10T08:45:18.261968+0000","last_became_active":"2024-09-10T08:44:10.681600+0000","last_became_peered":"2024-09-10T08:44:10.681600+0000","last_unstale":"2024-09-10T08:45:18.261968+0000","last_undegraded":"2024-09-10T08:45:18.261968+0000","last_fullsized":"2024-09-10T08:45:18.261968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:37.190576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.260926+0000","last_change":"2024-09-10T08:44:17.719784+0000","last_active":"2024-09-10T08:45:18.260926+0000","last_peered":"2024-09-10T08:45:18.260926+0000","last_clean":"2024-09-10T08:45:18.260926+0000","last_became_active":"2024-09-10T08:44:17.719182+0000","last_became_peered":"2024-09-10T08:44:17.719182+0000","last_unstale":"2024-09-10T08:45:18.260926+0000","last_undegraded":"2024-09-10T08:45:18.260926+0000","last_fullsized":"2024-09-10T08:45:18.260926+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:15.237772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270233+0000","last_change":"2024-09-10T08:44:10.679069+0000","last_active":"2024-09-10T08:45:18.270233+0000","last_peered":"2024-09-10T08:45:18.270233+0000","last_clean":"2024-09-10T08:45:18.270233+0000","last_became_active":"2024-09-10T08:44:10.678597+0000","last_became_peered":"2024-09-10T08:44:10.678597+0000","last_unstale":"2024-09-10T08:45:18.270233+0000","last_undegraded":"2024-09-10T08:45:18.270233+0000","last_fullsized":"2024-09-10T08:45:18.270233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:52.693906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248551+0000","last_change":"2024-09-10T08:44:08.660886+0000","last_active":"2024-09-10T08:45:18.248551+0000","last_peered":"2024-09-10T08:45:18.248551+0000","last_clean":"2024-09-10T08:45:18.248551+0000","last_became_active":"2024-09-10T08:44:08.660478+0000","last_became_peered":"2024-09-10T08:44:08.660478+0000","last_unstale":"2024-09-10T08:45:18.248551+0000","last_undegraded":"2024-09-10T08:45:18.248551+0000","last_fullsized":"2024-09-10T08:45:18.248551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:41.785531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261582+0000","last_change":"2024-09-10T08:44:17.761929+0000","last_active":"2024-09-10T08:45:18.261582+0000","last_peered":"2024-09-10T08:45:18.261582+0000","last_clean":"2024-09-10T08:45:18.261582+0000","last_became_active":"2024-09-10T08:44:17.761440+0000","last_became_peered":"2024-09-10T08:44:17.761440+0000","last_unstale":"2024-09-10T08:45:18.261582+0000","last_undegraded":"2024-09-10T08:45:18.261582+0000","last_fullsized":"2024-09-10T08:45:18.261582+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:37.817759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254646+0000","last_change":"2024-09-10T08:44:10.674071+0000","last_active":"2024-09-10T08:45:18.254646+0000","last_peered":"2024-09-10T08:45:18.254646+0000","last_clean":"2024-09-10T08:45:18.254646+0000","last_became_active":"2024-09-10T08:44:10.673770+0000","last_became_peered":"2024-09-10T08:44:10.673770+0000","last_unstale":"2024-09-10T08:45:18.254646+0000","last_undegraded":"2024-09-10T08:45:18.254646+0000","last_fullsized":"2024-09-10T08:45:18.254646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:44:19.944557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.264475+0000","last_change":"2024-09-10T08:45:18.264060+0000","last_active":"2024-09-10T08:45:18.264475+0000","last_peered":"2024-09-10T08:45:18.264475+0000","last_clean":"2024-09-10T08:45:18.264475+0000","last_became_active":"2024-09-10T08:45:18.255983+0000","last_became_peered":"2024-09-10T08:45:18.255983+0000","last_unstale":"2024-09-10T08:45:18.264475+0000","last_undegraded":"2024-09-10T08:45:18.264475+0000","last_fullsized":"2024-09-10T08:45:18.264475+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:10.810491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261509+0000","last_change":"2024-09-10T08:44:17.722877+0000","last_active":"2024-09-10T08:45:18.261509+0000","last_peered":"2024-09-10T08:45:18.261509+0000","last_clean":"2024-09-10T08:45:18.261509+0000","last_became_active":"2024-09-10T08:44:17.722046+0000","last_became_peered":"2024-09-10T08:44:17.722046+0000","last_unstale":"2024-09-10T08:45:18.261509+0000","last_undegraded":"2024-09-10T08:45:18.261509+0000","last_fullsized":"2024-09-10T08:45:18.261509+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:32.675598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252361+0000","last_change":"2024-09-10T08:44:10.671500+0000","last_active":"2024-09-10T08:45:18.252361+0000","last_peered":"2024-09-10T08:45:18.252361+0000","last_clean":"2024-09-10T08:45:18.252361+0000","last_became_active":"2024-09-10T08:44:10.671039+0000","last_became_peered":"2024-09-10T08:44:10.671039+0000","last_unstale":"2024-09-10T08:45:18.252361+0000","last_undegraded":"2024-09-10T08:45:18.252361+0000","last_fullsized":"2024-09-10T08:45:18.252361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:12.977373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.255013+0000","last_change":"2024-09-10T08:44:08.655467+0000","last_active":"2024-09-10T08:45:18.255013+0000","last_peered":"2024-09-10T08:45:18.255013+0000","last_clean":"2024-09-10T08:45:18.255013+0000","last_became_active":"2024-09-10T08:44:08.654991+0000","last_became_peered":"2024-09-10T08:44:08.654991+0000","last_unstale":"2024-09-10T08:45:18.255013+0000","last_undegraded":"2024-09-10T08:45:18.255013+0000","last_fullsized":"2024-09-10T08:45:18.255013+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:33.124360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.267039+0000","last_change":"2024-09-10T08:44:17.724250+0000","last_active":"2024-09-10T08:45:18.267039+0000","last_peered":"2024-09-10T08:45:18.267039+0000","last_clean":"2024-09-10T08:45:18.267039+0000","last_became_active":"2024-09-10T08:44:17.723733+0000","last_became_peered":"2024-09-10T08:44:17.723733+0000","last_unstale":"2024-09-10T08:45:18.267039+0000","last_undegraded":"2024-09-10T08:45:18.267039+0000","last_fullsized":"2024-09-10T08:45:18.267039+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:40.098666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248261+0000","last_change":"2024-09-10T08:44:10.664159+0000","last_active":"2024-09-10T08:45:18.248261+0000","last_peered":"2024-09-10T08:45:18.248261+0000","last_clean":"2024-09-10T08:45:18.248261+0000","last_became_active":"2024-09-10T08:44:10.663674+0000","last_became_peered":"2024-09-10T08:44:10.663674+0000","last_unstale":"2024-09-10T08:45:18.248261+0000","last_undegraded":"2024-09-10T08:45:18.248261+0000","last_fullsized":"2024-09-10T08:45:18.248261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:24.234525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257226+0000","last_change":"2024-09-10T08:45:18.257226+0000","last_active":"2024-09-10T08:45:18.257226+0000","last_peered":"2024-09-10T08:45:18.257226+0000","last_clean":"2024-09-10T08:45:18.257226+0000","last_became_active":"2024-09-10T08:45:18.254113+0000","last_became_peered":"2024-09-10T08:45:18.254113+0000","last_unstale":"2024-09-10T08:45:18.257226+0000","last_undegraded":"2024-09-10T08:45:18.257226+0000","last_fullsized":"2024-09-10T08:45:18.257226+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:19.370105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261558+0000","last_change":"2024-09-10T08:44:17.761639+0000","last_active":"2024-09-10T08:45:18.261558+0000","last_peered":"2024-09-10T08:45:18.261558+0000","last_clean":"2024-09-10T08:45:18.261558+0000","last_became_active":"2024-09-10T08:44:17.761214+0000","last_became_peered":"2024-09-10T08:44:17.761214+0000","last_unstale":"2024-09-10T08:45:18.261558+0000","last_undegraded":"2024-09-10T08:45:18.261558+0000","last_fullsized":"2024-09-10T08:45:18.261558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:43.793195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.258661+0000","last_change":"2024-09-10T08:45:18.258661+0000","last_active":"2024-09-10T08:45:18.258661+0000","last_peered":"2024-09-10T08:45:18.258661+0000","last_clean":"2024-09-10T08:45:18.258661+0000","last_became_active":"2024-09-10T08:45:18.258225+0000","last_became_peered":"2024-09-10T08:45:18.258225+0000","last_unstale":"2024-09-10T08:45:18.258661+0000","last_undegraded":"2024-09-10T08:45:18.258661+0000","last_fullsized":"2024-09-10T08:45:18.258661+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:28:37.184766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.262282+0000","last_change":"2024-09-10T08:44:09.036632+0000","last_active":"2024-09-10T08:45:18.262282+0000","last_peered":"2024-09-10T08:45:18.262282+0000","last_clean":"2024-09-10T08:45:18.262282+0000","last_became_active":"2024-09-10T08:44:09.036254+0000","last_became_peered":"2024-09-10T08:44:09.036254+0000","last_unstale":"2024-09-10T08:45:18.262282+0000","last_undegraded":"2024-09-10T08:45:18.262282+0000","last_fullsized":"2024-09-10T08:45:18.262282+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:59:40.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252129+0000","last_change":"2024-09-10T08:44:17.716215+0000","last_active":"2024-09-10T08:45:18.252129+0000","last_peered":"2024-09-10T08:45:18.252129+0000","last_clean":"2024-09-10T08:45:18.252129+0000","last_became_active":"2024-09-10T08:44:17.715809+0000","last_became_peered":"2024-09-10T08:44:17.715809+0000","last_unstale":"2024-09-10T08:45:18.252129+0000","last_undegraded":"2024-09-10T08:45:18.252129+0000","last_fullsized":"2024-09-10T08:45:18.252129+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:52.896401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254431+0000","last_change":"2024-09-10T08:44:10.677871+0000","last_active":"2024-09-10T08:45:18.254431+0000","last_peered":"2024-09-10T08:45:18.254431+0000","last_clean":"2024-09-10T08:45:18.254431+0000","last_became_active":"2024-09-10T08:44:10.677101+0000","last_became_peered":"2024-09-10T08:44:10.677101+0000","last_unstale":"2024-09-10T08:45:18.254431+0000","last_undegraded":"2024-09-10T08:45:18.254431+0000","last_fullsized":"2024-09-10T08:45:18.254431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:46.137249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.267182+0000","last_change":"2024-09-10T08:44:08.654428+0000","last_active":"2024-09-10T08:45:18.267182+0000","last_peered":"2024-09-10T08:45:18.267182+0000","last_clean":"2024-09-10T08:45:18.267182+0000","last_became_active":"2024-09-10T08:44:08.654119+0000","last_became_peered":"2024-09-10T08:44:08.654119+0000","last_unstale":"2024-09-10T08:45:18.267182+0000","last_undegraded":"2024-09-10T08:45:18.267182+0000","last_fullsized":"2024-09-10T08:45:18.267182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:56.787721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.253964+0000","last_change":"2024-09-10T08:44:17.760828+0000","last_active":"2024-09-10T08:45:18.253964+0000","last_peered":"2024-09-10T08:45:18.253964+0000","last_clean":"2024-09-10T08:45:18.253964+0000","last_became_active":"2024-09-10T08:44:17.760367+0000","last_became_peered":"2024-09-10T08:44:17.760367+0000","last_unstale":"2024-09-10T08:45:18.253964+0000","last_undegraded":"2024-09-10T08:45:18.253964+0000","last_fullsized":"2024-09-10T08:45:18.253964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:11.130480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257638+0000","last_change":"2024-09-10T08:44:10.670179+0000","last_active":"2024-09-10T08:45:18.257638+0000","last_peered":"2024-09-10T08:45:18.257638+0000","last_clean":"2024-09-10T08:45:18.257638+0000","last_became_active":"2024-09-10T08:44:10.669366+0000","last_became_peered":"2024-09-10T08:44:10.669366+0000","last_unstale":"2024-09-10T08:45:18.257638+0000","last_undegraded":"2024-09-10T08:45:18.257638+0000","last_fullsized":"2024-09-10T08:45:18.257638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:41.916365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.249082+0000","last_change":"2024-09-10T08:44:09.050441+0000","last_active":"2024-09-10T08:45:18.249082+0000","last_peered":"2024-09-10T08:45:18.249082+0000","last_clean":"2024-09-10T08:45:18.249082+0000","last_became_active":"2024-09-10T08:44:09.050070+0000","last_became_peered":"2024-09-10T08:44:09.050070+0000","last_unstale":"2024-09-10T08:45:18.249082+0000","last_undegraded":"2024-09-10T08:45:18.249082+0000","last_fullsized":"2024-09-10T08:45:18.249082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:13.540021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.251158+0000","last_change":"2024-09-10T08:44:17.720098+0000","last_active":"2024-09-10T08:45:18.251158+0000","last_peered":"2024-09-10T08:45:18.251158+0000","last_clean":"2024-09-10T08:45:18.251158+0000","last_became_active":"2024-09-10T08:44:17.719627+0000","last_became_peered":"2024-09-10T08:44:17.719627+0000","last_unstale":"2024-09-10T08:45:18.251158+0000","last_undegraded":"2024-09-10T08:45:18.251158+0000","last_fullsized":"2024-09-10T08:45:18.251158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:50.020919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.258587+0000","last_change":"2024-09-10T08:44:10.669630+0000","last_active":"2024-09-10T08:45:18.258587+0000","last_peered":"2024-09-10T08:45:18.258587+0000","last_clean":"2024-09-10T08:45:18.258587+0000","last_became_active":"2024-09-10T08:44:10.669307+0000","last_became_peered":"2024-09-10T08:44:10.669307+0000","last_unstale":"2024-09-10T08:45:18.258587+0000","last_undegraded":"2024-09-10T08:45:18.258587+0000","last_fullsized":"2024-09-10T08:45:18.258587+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:06.320046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.267039+0000","last_change":"2024-09-10T08:44:09.050715+0000","last_active":"2024-09-10T08:45:18.267039+0000","last_peered":"2024-09-10T08:45:18.267039+0000","last_clean":"2024-09-10T08:45:18.267039+0000","last_became_active":"2024-09-10T08:44:09.050347+0000","last_became_peered":"2024-09-10T08:44:09.050347+0000","last_unstale":"2024-09-10T08:45:18.267039+0000","last_undegraded":"2024-09-10T08:45:18.267039+0000","last_fullsized":"2024-09-10T08:45:18.267039+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:18.689652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.260444+0000","last_change":"2024-09-10T08:45:18.260444+0000","last_active":"2024-09-10T08:45:18.260444+0000","last_peered":"2024-09-10T08:45:18.260444+0000","last_clean":"2024-09-10T08:45:18.260444+0000","last_became_active":"2024-09-10T08:45:18.259949+0000","last_became_peered":"2024-09-10T08:45:18.259949+0000","last_unstale":"2024-09-10T08:45:18.260444+0000","last_undegraded":"2024-09-10T08:45:18.260444+0000","last_fullsized":"2024-09-10T08:45:18.260444+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:56.391253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.258585+0000","last_change":"2024-09-10T08:44:10.670881+0000","last_active":"2024-09-10T08:45:18.258585+0000","last_peered":"2024-09-10T08:45:18.258585+0000","last_clean":"2024-09-10T08:45:18.258585+0000","last_became_active":"2024-09-10T08:44:10.669791+0000","last_became_peered":"2024-09-10T08:44:10.669791+0000","last_unstale":"2024-09-10T08:45:18.258585+0000","last_undegraded":"2024-09-10T08:45:18.258585+0000","last_fullsized":"2024-09-10T08:45:18.258585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:33.644205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":89,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:42.298071+0000","last_change":"2024-09-10T08:44:08.657938+0000","last_active":"2024-09-10T08:45:42.298071+0000","last_peered":"2024-09-10T08:45:42.298071+0000","last_clean":"2024-09-10T08:45:42.298071+0000","last_became_active":"2024-09-10T08:44:08.657526+0000","last_became_peered":"2024-09-10T08:44:08.657526+0000","last_unstale":"2024-09-10T08:45:42.298071+0000","last_undegraded":"2024-09-10T08:45:42.298071+0000","last_fullsized":"2024-09-10T08:45:42.298071+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:15.673561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257344+0000","last_change":"2024-09-10T08:44:17.721619+0000","last_active":"2024-09-10T08:45:18.257344+0000","last_peered":"2024-09-10T08:45:18.257344+0000","last_clean":"2024-09-10T08:45:18.257344+0000","last_became_active":"2024-09-10T08:44:17.721175+0000","last_became_peered":"2024-09-10T08:44:17.721175+0000","last_unstale":"2024-09-10T08:45:18.257344+0000","last_undegraded":"2024-09-10T08:45:18.257344+0000","last_fullsized":"2024-09-10T08:45:18.257344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:52.244120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261384+0000","last_change":"2024-09-10T08:44:10.676164+0000","last_active":"2024-09-10T08:45:18.261384+0000","last_peered":"2024-09-10T08:45:18.261384+0000","last_clean":"2024-09-10T08:45:18.261384+0000","last_became_active":"2024-09-10T08:44:10.675820+0000","last_became_peered":"2024-09-10T08:44:10.675820+0000","last_unstale":"2024-09-10T08:45:18.261384+0000","last_undegraded":"2024-09-10T08:45:18.261384+0000","last_fullsized":"2024-09-10T08:45:18.261384+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:24.477584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":91,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:42.293033+0000","last_change":"2024-09-10T08:44:09.049124+0000","last_active":"2024-09-10T08:45:42.293033+0000","last_peered":"2024-09-10T08:45:42.293033+0000","last_clean":"2024-09-10T08:45:42.293033+0000","last_became_active":"2024-09-10T08:44:09.048799+0000","last_became_peered":"2024-09-10T08:44:09.048799+0000","last_unstale":"2024-09-10T08:45:42.293033+0000","last_undegraded":"2024-09-10T08:45:42.293033+0000","last_fullsized":"2024-09-10T08:45:42.293033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:43:39.849727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257766+0000","last_change":"2024-09-10T08:44:17.761130+0000","last_active":"2024-09-10T08:45:18.257766+0000","last_peered":"2024-09-10T08:45:18.257766+0000","last_clean":"2024-09-10T08:45:18.257766+0000","last_became_active":"2024-09-10T08:44:17.760753+0000","last_became_peered":"2024-09-10T08:44:17.760753+0000","last_unstale":"2024-09-10T08:45:18.257766+0000","last_undegraded":"2024-09-10T08:45:18.257766+0000","last_fullsized":"2024-09-10T08:45:18.257766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:42.971962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.256832+0000","last_change":"2024-09-10T08:44:10.669905+0000","last_active":"2024-09-10T08:45:18.256832+0000","last_peered":"2024-09-10T08:45:18.256832+0000","last_clean":"2024-09-10T08:45:18.256832+0000","last_became_active":"2024-09-10T08:44:10.669340+0000","last_became_peered":"2024-09-10T08:44:10.669340+0000","last_unstale":"2024-09-10T08:45:18.256832+0000","last_undegraded":"2024-09-10T08:45:18.256832+0000","last_fullsized":"2024-09-10T08:45:18.256832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:23.263179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270000+0000","last_change":"2024-09-10T08:44:08.658862+0000","last_active":"2024-09-10T08:45:18.270000+0000","last_peered":"2024-09-10T08:45:18.270000+0000","last_clean":"2024-09-10T08:45:18.270000+0000","last_became_active":"2024-09-10T08:44:08.657880+0000","last_became_peered":"2024-09-10T08:44:08.657880+0000","last_unstale":"2024-09-10T08:45:18.270000+0000","last_undegraded":"2024-09-10T08:45:18.270000+0000","last_fullsized":"2024-09-10T08:45:18.270000+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:34.224025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.269022+0000","last_change":"2024-09-10T08:45:18.268446+0000","last_active":"2024-09-10T08:45:18.269022+0000","last_peered":"2024-09-10T08:45:18.269022+0000","last_clean":"2024-09-10T08:45:18.269022+0000","last_became_active":"2024-09-10T08:45:18.267988+0000","last_became_peered":"2024-09-10T08:45:18.267988+0000","last_unstale":"2024-09-10T08:45:18.269022+0000","last_undegraded":"2024-09-10T08:45:18.269022+0000","last_fullsized":"2024-09-10T08:45:18.269022+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:42:35.990794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254358+0000","last_change":"2024-09-10T08:44:10.673258+0000","last_active":"2024-09-10T08:45:18.254358+0000","last_peered":"2024-09-10T08:45:18.254358+0000","last_clean":"2024-09-10T08:45:18.254358+0000","last_became_active":"2024-09-10T08:44:10.672854+0000","last_became_peered":"2024-09-10T08:44:10.672854+0000","last_unstale":"2024-09-10T08:45:18.254358+0000","last_undegraded":"2024-09-10T08:45:18.254358+0000","last_fullsized":"2024-09-10T08:45:18.254358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:35.913820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261847+0000","last_change":"2024-09-10T08:44:08.658039+0000","last_active":"2024-09-10T08:45:18.261847+0000","last_peered":"2024-09-10T08:45:18.261847+0000","last_clean":"2024-09-10T08:45:18.261847+0000","last_became_active":"2024-09-10T08:44:08.657636+0000","last_became_peered":"2024-09-10T08:44:08.657636+0000","last_unstale":"2024-09-10T08:45:18.261847+0000","last_undegraded":"2024-09-10T08:45:18.261847+0000","last_fullsized":"2024-09-10T08:45:18.261847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:09.615992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270530+0000","last_change":"2024-09-10T08:42:57.666747+0000","last_active":"2024-09-10T08:45:18.270530+0000","last_peered":"2024-09-10T08:45:18.270530+0000","last_clean":"2024-09-10T08:45:18.270530+0000","last_became_active":"2024-09-10T08:42:57.666417+0000","last_became_peered":"2024-09-10T08:42:57.666417+0000","last_unstale":"2024-09-10T08:45:18.270530+0000","last_undegraded":"2024-09-10T08:45:18.270530+0000","last_fullsized":"2024-09-10T08:45:18.270530+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_clean_scrub_stamp":"2024-09-10T08:39:31.729794+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.547989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270574+0000","last_change":"2024-09-10T08:44:17.723021+0000","last_active":"2024-09-10T08:45:18.270574+0000","last_peered":"2024-09-10T08:45:18.270574+0000","last_clean":"2024-09-10T08:45:18.270574+0000","last_became_active":"2024-09-10T08:44:17.722222+0000","last_became_peered":"2024-09-10T08:44:17.722222+0000","last_unstale":"2024-09-10T08:45:18.270574+0000","last_undegraded":"2024-09-10T08:45:18.270574+0000","last_fullsized":"2024-09-10T08:45:18.270574+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:24.444427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270662+0000","last_change":"2024-09-10T08:44:10.676651+0000","last_active":"2024-09-10T08:45:18.270662+0000","last_peered":"2024-09-10T08:45:18.270662+0000","last_clean":"2024-09-10T08:45:18.270662+0000","last_became_active":"2024-09-10T08:44:10.676342+0000","last_became_peered":"2024-09-10T08:44:10.676342+0000","last_unstale":"2024-09-10T08:45:18.270662+0000","last_undegraded":"2024-09-10T08:45:18.270662+0000","last_fullsized":"2024-09-10T08:45:18.270662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:07.021273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":95,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:45.302623+0000","last_change":"2024-09-10T08:44:09.050305+0000","last_active":"2024-09-10T08:45:45.302623+0000","last_peered":"2024-09-10T08:45:45.302623+0000","last_clean":"2024-09-10T08:45:45.302623+0000","last_became_active":"2024-09-10T08:44:09.049958+0000","last_became_peered":"2024-09-10T08:44:09.049958+0000","last_unstale":"2024-09-10T08:45:45.302623+0000","last_undegraded":"2024-09-10T08:45:45.302623+0000","last_fullsized":"2024-09-10T08:45:45.302623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:42.219190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248307+0000","last_change":"2024-09-10T08:44:17.720219+0000","last_active":"2024-09-10T08:45:18.248307+0000","last_peered":"2024-09-10T08:45:18.248307+0000","last_clean":"2024-09-10T08:45:18.248307+0000","last_became_active":"2024-09-10T08:44:17.719756+0000","last_became_peered":"2024-09-10T08:44:17.719756+0000","last_unstale":"2024-09-10T08:45:18.248307+0000","last_undegraded":"2024-09-10T08:45:18.248307+0000","last_fullsized":"2024-09-10T08:45:18.248307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:01.070434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270237+0000","last_change":"2024-09-10T08:44:10.678986+0000","last_active":"2024-09-10T08:45:18.270237+0000","last_peered":"2024-09-10T08:45:18.270237+0000","last_clean":"2024-09-10T08:45:18.270237+0000","last_became_active":"2024-09-10T08:44:10.678654+0000","last_became_peered":"2024-09-10T08:44:10.678654+0000","last_unstale":"2024-09-10T08:45:18.270237+0000","last_undegraded":"2024-09-10T08:45:18.270237+0000","last_fullsized":"2024-09-10T08:45:18.270237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:07.116802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.262374+0000","last_change":"2024-09-10T08:44:08.655025+0000","last_active":"2024-09-10T08:45:18.262374+0000","last_peered":"2024-09-10T08:45:18.262374+0000","last_clean":"2024-09-10T08:45:18.262374+0000","last_became_active":"2024-09-10T08:44:08.654728+0000","last_became_peered":"2024-09-10T08:44:08.654728+0000","last_unstale":"2024-09-10T08:45:18.262374+0000","last_undegraded":"2024-09-10T08:45:18.262374+0000","last_fullsized":"2024-09-10T08:45:18.262374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:31.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261492+0000","last_change":"2024-09-10T08:45:18.261492+0000","last_active":"2024-09-10T08:45:18.261492+0000","last_peered":"2024-09-10T08:45:18.261492+0000","last_clean":"2024-09-10T08:45:18.261492+0000","last_became_active":"2024-09-10T08:45:18.261111+0000","last_became_peered":"2024-09-10T08:45:18.261111+0000","last_unstale":"2024-09-10T08:45:18.261492+0000","last_undegraded":"2024-09-10T08:45:18.261492+0000","last_fullsized":"2024-09-10T08:45:18.261492+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:48.709459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270355+0000","last_change":"2024-09-10T08:44:10.675072+0000","last_active":"2024-09-10T08:45:18.270355+0000","last_peered":"2024-09-10T08:45:18.270355+0000","last_clean":"2024-09-10T08:45:18.270355+0000","last_became_active":"2024-09-10T08:44:10.674636+0000","last_became_peered":"2024-09-10T08:44:10.674636+0000","last_unstale":"2024-09-10T08:45:18.270355+0000","last_undegraded":"2024-09-10T08:45:18.270355+0000","last_fullsized":"2024-09-10T08:45:18.270355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:55.198445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257848+0000","last_change":"2024-09-10T08:44:08.656450+0000","last_active":"2024-09-10T08:45:18.257848+0000","last_peered":"2024-09-10T08:45:18.257848+0000","last_clean":"2024-09-10T08:45:18.257848+0000","last_became_active":"2024-09-10T08:44:08.655967+0000","last_became_peered":"2024-09-10T08:44:08.655967+0000","last_unstale":"2024-09-10T08:45:18.257848+0000","last_undegraded":"2024-09-10T08:45:18.257848+0000","last_fullsized":"2024-09-10T08:45:18.257848+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:29.273167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261136+0000","last_change":"2024-09-10T08:44:17.722217+0000","last_active":"2024-09-10T08:45:18.261136+0000","last_peered":"2024-09-10T08:45:18.261136+0000","last_clean":"2024-09-10T08:45:18.261136+0000","last_became_active":"2024-09-10T08:44:17.721800+0000","last_became_peered":"2024-09-10T08:44:17.721800+0000","last_unstale":"2024-09-10T08:45:18.261136+0000","last_undegraded":"2024-09-10T08:45:18.261136+0000","last_fullsized":"2024-09-10T08:45:18.261136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:58.954397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.267857+0000","last_change":"2024-09-10T08:45:18.267857+0000","last_active":"2024-09-10T08:45:18.267857+0000","last_peered":"2024-09-10T08:45:18.267857+0000","last_clean":"2024-09-10T08:45:18.267857+0000","last_became_active":"2024-09-10T08:45:18.267440+0000","last_became_peered":"2024-09-10T08:45:18.267440+0000","last_unstale":"2024-09-10T08:45:18.267857+0000","last_undegraded":"2024-09-10T08:45:18.267857+0000","last_fullsized":"2024-09-10T08:45:18.267857+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:28.659915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.249034+0000","last_change":"2024-09-10T08:44:08.659003+0000","last_active":"2024-09-10T08:45:18.249034+0000","last_peered":"2024-09-10T08:45:18.249034+0000","last_clean":"2024-09-10T08:45:18.249034+0000","last_became_active":"2024-09-10T08:44:08.658324+0000","last_became_peered":"2024-09-10T08:44:08.658324+0000","last_unstale":"2024-09-10T08:45:18.249034+0000","last_undegraded":"2024-09-10T08:45:18.249034+0000","last_fullsized":"2024-09-10T08:45:18.249034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:44.743053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261510+0000","last_change":"2024-09-10T08:44:17.723363+0000","last_active":"2024-09-10T08:45:18.261510+0000","last_peered":"2024-09-10T08:45:18.261510+0000","last_clean":"2024-09-10T08:45:18.261510+0000","last_became_active":"2024-09-10T08:44:17.721799+0000","last_became_peered":"2024-09-10T08:44:17.721799+0000","last_unstale":"2024-09-10T08:45:18.261510+0000","last_undegraded":"2024-09-10T08:45:18.261510+0000","last_fullsized":"2024-09-10T08:45:18.261510+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:08:28.772193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261712+0000","last_change":"2024-09-10T08:44:10.682952+0000","last_active":"2024-09-10T08:45:18.261712+0000","last_peered":"2024-09-10T08:45:18.261712+0000","last_clean":"2024-09-10T08:45:18.261712+0000","last_became_active":"2024-09-10T08:44:10.682249+0000","last_became_peered":"2024-09-10T08:44:10.682249+0000","last_unstale":"2024-09-10T08:45:18.261712+0000","last_undegraded":"2024-09-10T08:45:18.261712+0000","last_fullsized":"2024-09-10T08:45:18.261712+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:32:21.285461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248258+0000","last_change":"2024-09-10T08:44:09.050454+0000","last_active":"2024-09-10T08:45:18.248258+0000","last_peered":"2024-09-10T08:45:18.248258+0000","last_clean":"2024-09-10T08:45:18.248258+0000","last_became_active":"2024-09-10T08:44:09.050036+0000","last_became_peered":"2024-09-10T08:44:09.050036+0000","last_unstale":"2024-09-10T08:45:18.248258+0000","last_undegraded":"2024-09-10T08:45:18.248258+0000","last_fullsized":"2024-09-10T08:45:18.248258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:55.141471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257803+0000","last_change":"2024-09-10T08:44:17.721203+0000","last_active":"2024-09-10T08:45:18.257803+0000","last_peered":"2024-09-10T08:45:18.257803+0000","last_clean":"2024-09-10T08:45:18.257803+0000","last_became_active":"2024-09-10T08:44:17.720737+0000","last_became_peered":"2024-09-10T08:44:17.720737+0000","last_unstale":"2024-09-10T08:45:18.257803+0000","last_undegraded":"2024-09-10T08:45:18.257803+0000","last_fullsized":"2024-09-10T08:45:18.257803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:32.081079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.269917+0000","last_change":"2024-09-10T08:44:10.680944+0000","last_active":"2024-09-10T08:45:18.269917+0000","last_peered":"2024-09-10T08:45:18.269917+0000","last_clean":"2024-09-10T08:45:18.269917+0000","last_became_active":"2024-09-10T08:44:10.680644+0000","last_became_peered":"2024-09-10T08:44:10.680644+0000","last_unstale":"2024-09-10T08:45:18.269917+0000","last_undegraded":"2024-09-10T08:45:18.269917+0000","last_fullsized":"2024-09-10T08:45:18.269917+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:46.495894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:42.293794+0000","last_change":"2024-09-10T08:44:08.658135+0000","last_active":"2024-09-10T08:45:42.293794+0000","last_peered":"2024-09-10T08:45:42.293794+0000","last_clean":"2024-09-10T08:45:42.293794+0000","last_became_active":"2024-09-10T08:44:08.657793+0000","last_became_peered":"2024-09-10T08:44:08.657793+0000","last_unstale":"2024-09-10T08:45:42.293794+0000","last_undegraded":"2024-09-10T08:45:42.293794+0000","last_fullsized":"2024-09-10T08:45:42.293794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:16.376268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.272086+0000","last_change":"2024-09-10T08:45:18.271678+0000","last_active":"2024-09-10T08:45:18.272086+0000","last_peered":"2024-09-10T08:45:18.272086+0000","last_clean":"2024-09-10T08:45:18.272086+0000","last_became_active":"2024-09-10T08:45:18.271209+0000","last_became_peered":"2024-09-10T08:45:18.271209+0000","last_unstale":"2024-09-10T08:45:18.272086+0000","last_undegraded":"2024-09-10T08:45:18.272086+0000","last_fullsized":"2024-09-10T08:45:18.272086+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:17.730839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270358+0000","last_change":"2024-09-10T08:44:10.678254+0000","last_active":"2024-09-10T08:45:18.270358+0000","last_peered":"2024-09-10T08:45:18.270358+0000","last_clean":"2024-09-10T08:45:18.270358+0000","last_became_active":"2024-09-10T08:44:10.677891+0000","last_became_peered":"2024-09-10T08:44:10.677891+0000","last_unstale":"2024-09-10T08:45:18.270358+0000","last_undegraded":"2024-09-10T08:45:18.270358+0000","last_fullsized":"2024-09-10T08:45:18.270358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:32:43.399049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261698+0000","last_change":"2024-09-10T08:44:08.656315+0000","last_active":"2024-09-10T08:45:18.261698+0000","last_peered":"2024-09-10T08:45:18.261698+0000","last_clean":"2024-09-10T08:45:18.261698+0000","last_became_active":"2024-09-10T08:44:08.655978+0000","last_became_peered":"2024-09-10T08:44:08.655978+0000","last_unstale":"2024-09-10T08:45:18.261698+0000","last_undegraded":"2024-09-10T08:45:18.261698+0000","last_fullsized":"2024-09-10T08:45:18.261698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:15.025354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252245+0000","last_change":"2024-09-10T08:44:17.726789+0000","last_active":"2024-09-10T08:45:18.252245+0000","last_peered":"2024-09-10T08:45:18.252245+0000","last_clean":"2024-09-10T08:45:18.252245+0000","last_became_active":"2024-09-10T08:44:17.726450+0000","last_became_peered":"2024-09-10T08:44:17.726450+0000","last_unstale":"2024-09-10T08:45:18.252245+0000","last_undegraded":"2024-09-10T08:45:18.252245+0000","last_fullsized":"2024-09-10T08:45:18.252245+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:03.899822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257056+0000","last_change":"2024-09-10T08:44:10.677359+0000","last_active":"2024-09-10T08:45:18.257056+0000","last_peered":"2024-09-10T08:45:18.257056+0000","last_clean":"2024-09-10T08:45:18.257056+0000","last_became_active":"2024-09-10T08:44:10.676782+0000","last_became_peered":"2024-09-10T08:44:10.676782+0000","last_unstale":"2024-09-10T08:45:18.257056+0000","last_undegraded":"2024-09-10T08:45:18.257056+0000","last_fullsized":"2024-09-10T08:45:18.257056+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:31.721714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252007+0000","last_change":"2024-09-10T08:44:08.656993+0000","last_active":"2024-09-10T08:45:18.252007+0000","last_peered":"2024-09-10T08:45:18.252007+0000","last_clean":"2024-09-10T08:45:18.252007+0000","last_became_active":"2024-09-10T08:44:08.656571+0000","last_became_peered":"2024-09-10T08:44:08.656571+0000","last_unstale":"2024-09-10T08:45:18.252007+0000","last_undegraded":"2024-09-10T08:45:18.252007+0000","last_fullsized":"2024-09-10T08:45:18.252007+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:32.202740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261935+0000","last_change":"2024-09-10T08:44:17.725674+0000","last_active":"2024-09-10T08:45:18.261935+0000","last_peered":"2024-09-10T08:45:18.261935+0000","last_clean":"2024-09-10T08:45:18.261935+0000","last_became_active":"2024-09-10T08:44:17.725263+0000","last_became_peered":"2024-09-10T08:44:17.725263+0000","last_unstale":"2024-09-10T08:45:18.261935+0000","last_undegraded":"2024-09-10T08:45:18.261935+0000","last_fullsized":"2024-09-10T08:45:18.261935+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:18.183157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248867+0000","last_change":"2024-09-10T08:44:10.676871+0000","last_active":"2024-09-10T08:45:18.248867+0000","last_peered":"2024-09-10T08:45:18.248867+0000","last_clean":"2024-09-10T08:45:18.248867+0000","last_became_active":"2024-09-10T08:44:10.676540+0000","last_became_peered":"2024-09-10T08:44:10.676540+0000","last_unstale":"2024-09-10T08:45:18.248867+0000","last_undegraded":"2024-09-10T08:45:18.248867+0000","last_fullsized":"2024-09-10T08:45:18.248867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:50.964100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'35","reported_seq":181,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:44.295955+0000","last_change":"2024-09-10T08:44:08.660113+0000","last_active":"2024-09-10T08:45:44.295955+0000","last_peered":"2024-09-10T08:45:44.295955+0000","last_clean":"2024-09-10T08:45:44.295955+0000","last_became_active":"2024-09-10T08:44:08.659644+0000","last_became_peered":"2024-09-10T08:44:08.659644+0000","last_unstale":"2024-09-10T08:45:44.295955+0000","last_undegraded":"2024-09-10T08:45:44.295955+0000","last_fullsized":"2024-09-10T08:45:44.295955+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:01.605344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63484,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":112,"num_read_kb":118,"num_write":37,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.266627+0000","last_change":"2024-09-10T08:44:17.723653+0000","last_active":"2024-09-10T08:45:18.266627+0000","last_peered":"2024-09-10T08:45:18.266627+0000","last_clean":"2024-09-10T08:45:18.266627+0000","last_became_active":"2024-09-10T08:44:17.723284+0000","last_became_peered":"2024-09-10T08:44:17.723284+0000","last_unstale":"2024-09-10T08:45:18.266627+0000","last_undegraded":"2024-09-10T08:45:18.266627+0000","last_fullsized":"2024-09-10T08:45:18.266627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:17.162526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248747+0000","last_change":"2024-09-10T08:44:10.681837+0000","last_active":"2024-09-10T08:45:18.248747+0000","last_peered":"2024-09-10T08:45:18.248747+0000","last_clean":"2024-09-10T08:45:18.248747+0000","last_became_active":"2024-09-10T08:44:10.681518+0000","last_became_peered":"2024-09-10T08:44:10.681518+0000","last_unstale":"2024-09-10T08:45:18.248747+0000","last_undegraded":"2024-09-10T08:45:18.248747+0000","last_fullsized":"2024-09-10T08:45:18.248747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:10.004215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.262072+0000","last_change":"2024-09-10T08:44:09.050394+0000","last_active":"2024-09-10T08:45:18.262072+0000","last_peered":"2024-09-10T08:45:18.262072+0000","last_clean":"2024-09-10T08:45:18.262072+0000","last_became_active":"2024-09-10T08:44:09.049969+0000","last_became_peered":"2024-09-10T08:44:09.049969+0000","last_unstale":"2024-09-10T08:45:18.262072+0000","last_undegraded":"2024-09-10T08:45:18.262072+0000","last_fullsized":"2024-09-10T08:45:18.262072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:43.519246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261264+0000","last_change":"2024-09-10T08:44:17.720318+0000","last_active":"2024-09-10T08:45:18.261264+0000","last_peered":"2024-09-10T08:45:18.261264+0000","last_clean":"2024-09-10T08:45:18.261264+0000","last_became_active":"2024-09-10T08:44:17.719598+0000","last_became_peered":"2024-09-10T08:44:17.719598+0000","last_unstale":"2024-09-10T08:45:18.261264+0000","last_undegraded":"2024-09-10T08:45:18.261264+0000","last_fullsized":"2024-09-10T08:45:18.261264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:15:49.520324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248538+0000","last_change":"2024-09-10T08:44:10.677340+0000","last_active":"2024-09-10T08:45:18.248538+0000","last_peered":"2024-09-10T08:45:18.248538+0000","last_clean":"2024-09-10T08:45:18.248538+0000","last_became_active":"2024-09-10T08:44:10.676628+0000","last_became_peered":"2024-09-10T08:44:10.676628+0000","last_unstale":"2024-09-10T08:45:18.248538+0000","last_undegraded":"2024-09-10T08:45:18.248538+0000","last_fullsized":"2024-09-10T08:45:18.248538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:59:17.924217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270431+0000","last_change":"2024-09-10T08:44:08.660254+0000","last_active":"2024-09-10T08:45:18.270431+0000","last_peered":"2024-09-10T08:45:18.270431+0000","last_clean":"2024-09-10T08:45:18.270431+0000","last_became_active":"2024-09-10T08:44:08.659667+0000","last_became_peered":"2024-09-10T08:44:08.659667+0000","last_unstale":"2024-09-10T08:45:18.270431+0000","last_undegraded":"2024-09-10T08:45:18.270431+0000","last_fullsized":"2024-09-10T08:45:18.270431+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:49:49.146867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.266387+0000","last_change":"2024-09-10T08:44:17.722373+0000","last_active":"2024-09-10T08:45:18.266387+0000","last_peered":"2024-09-10T08:45:18.266387+0000","last_clean":"2024-09-10T08:45:18.266387+0000","last_became_active":"2024-09-10T08:44:17.722035+0000","last_became_peered":"2024-09-10T08:44:17.722035+0000","last_unstale":"2024-09-10T08:45:18.266387+0000","last_undegraded":"2024-09-10T08:45:18.266387+0000","last_fullsized":"2024-09-10T08:45:18.266387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:30:36.187364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.249176+0000","last_change":"2024-09-10T08:44:10.672585+0000","last_active":"2024-09-10T08:45:18.249176+0000","last_peered":"2024-09-10T08:45:18.249176+0000","last_clean":"2024-09-10T08:45:18.249176+0000","last_became_active":"2024-09-10T08:44:10.671987+0000","last_became_peered":"2024-09-10T08:44:10.671987+0000","last_unstale":"2024-09-10T08:45:18.249176+0000","last_undegraded":"2024-09-10T08:45:18.249176+0000","last_fullsized":"2024-09-10T08:45:18.249176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:48.660750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":140,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:44.295983+0000","last_change":"2024-09-10T08:44:08.658069+0000","last_active":"2024-09-10T08:45:44.295983+0000","last_peered":"2024-09-10T08:45:44.295983+0000","last_clean":"2024-09-10T08:45:44.295983+0000","last_became_active":"2024-09-10T08:44:08.657702+0000","last_became_peered":"2024-09-10T08:44:08.657702+0000","last_unstale":"2024-09-10T08:45:44.295983+0000","last_undegraded":"2024-09-10T08:45:44.295983+0000","last_fullsized":"2024-09-10T08:45:44.295983+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:17.383267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254196+0000","last_change":"2024-09-10T08:44:17.717984+0000","last_active":"2024-09-10T08:45:18.254196+0000","last_peered":"2024-09-10T08:45:18.254196+0000","last_clean":"2024-09-10T08:45:18.254196+0000","last_became_active":"2024-09-10T08:44:17.717581+0000","last_became_peered":"2024-09-10T08:44:17.717581+0000","last_unstale":"2024-09-10T08:45:18.254196+0000","last_undegraded":"2024-09-10T08:45:18.254196+0000","last_fullsized":"2024-09-10T08:45:18.254196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:25.330732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248233+0000","last_change":"2024-09-10T08:44:17.721244+0000","last_active":"2024-09-10T08:45:18.248233+0000","last_peered":"2024-09-10T08:45:18.248233+0000","last_clean":"2024-09-10T08:45:18.248233+0000","last_became_active":"2024-09-10T08:44:17.720884+0000","last_became_peered":"2024-09-10T08:44:17.720884+0000","last_unstale":"2024-09-10T08:45:18.248233+0000","last_undegraded":"2024-09-10T08:45:18.248233+0000","last_fullsized":"2024-09-10T08:45:18.248233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.769513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.266636+0000","last_change":"2024-09-10T08:44:10.667558+0000","last_active":"2024-09-10T08:45:18.266636+0000","last_peered":"2024-09-10T08:45:18.266636+0000","last_clean":"2024-09-10T08:45:18.266636+0000","last_became_active":"2024-09-10T08:44:10.667214+0000","last_became_peered":"2024-09-10T08:44:10.667214+0000","last_unstale":"2024-09-10T08:45:18.266636+0000","last_undegraded":"2024-09-10T08:45:18.266636+0000","last_fullsized":"2024-09-10T08:45:18.266636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:53.630790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261521+0000","last_change":"2024-09-10T08:44:08.657930+0000","last_active":"2024-09-10T08:45:18.261521+0000","last_peered":"2024-09-10T08:45:18.261521+0000","last_clean":"2024-09-10T08:45:18.261521+0000","last_became_active":"2024-09-10T08:44:08.657611+0000","last_became_peered":"2024-09-10T08:44:08.657611+0000","last_unstale":"2024-09-10T08:45:18.261521+0000","last_undegraded":"2024-09-10T08:45:18.261521+0000","last_fullsized":"2024-09-10T08:45:18.261521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:36.923786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252655+0000","last_change":"2024-09-10T08:44:17.726106+0000","last_active":"2024-09-10T08:45:18.252655+0000","last_peered":"2024-09-10T08:45:18.252655+0000","last_clean":"2024-09-10T08:45:18.252655+0000","last_became_active":"2024-09-10T08:44:17.725683+0000","last_became_peered":"2024-09-10T08:44:17.725683+0000","last_unstale":"2024-09-10T08:45:18.252655+0000","last_undegraded":"2024-09-10T08:45:18.252655+0000","last_fullsized":"2024-09-10T08:45:18.252655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:23.392794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254634+0000","last_change":"2024-09-10T08:44:10.672204+0000","last_active":"2024-09-10T08:45:18.254634+0000","last_peered":"2024-09-10T08:45:18.254634+0000","last_clean":"2024-09-10T08:45:18.254634+0000","last_became_active":"2024-09-10T08:44:10.671898+0000","last_became_peered":"2024-09-10T08:44:10.671898+0000","last_unstale":"2024-09-10T08:45:18.254634+0000","last_undegraded":"2024-09-10T08:45:18.254634+0000","last_fullsized":"2024-09-10T08:45:18.254634+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:03.737357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.256851+0000","last_change":"2024-09-10T08:44:08.656620+0000","last_active":"2024-09-10T08:45:18.256851+0000","last_peered":"2024-09-10T08:45:18.256851+0000","last_clean":"2024-09-10T08:45:18.256851+0000","last_became_active":"2024-09-10T08:44:08.656306+0000","last_became_peered":"2024-09-10T08:44:08.656306+0000","last_unstale":"2024-09-10T08:45:18.256851+0000","last_undegraded":"2024-09-10T08:45:18.256851+0000","last_fullsized":"2024-09-10T08:45:18.256851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:57.081909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.258679+0000","last_change":"2024-09-10T08:44:08.660178+0000","last_active":"2024-09-10T08:45:18.258679+0000","last_peered":"2024-09-10T08:45:18.258679+0000","last_clean":"2024-09-10T08:45:18.258679+0000","last_became_active":"2024-09-10T08:44:08.659839+0000","last_became_peered":"2024-09-10T08:44:08.659839+0000","last_unstale":"2024-09-10T08:45:18.258679+0000","last_undegraded":"2024-09-10T08:45:18.258679+0000","last_fullsized":"2024-09-10T08:45:18.258679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:17.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.262119+0000","last_change":"2024-09-10T08:44:10.683683+0000","last_active":"2024-09-10T08:45:18.262119+0000","last_peered":"2024-09-10T08:45:18.262119+0000","last_clean":"2024-09-10T08:45:18.262119+0000","last_became_active":"2024-09-10T08:44:10.683337+0000","last_became_peered":"2024-09-10T08:44:10.683337+0000","last_unstale":"2024-09-10T08:45:18.262119+0000","last_undegraded":"2024-09-10T08:45:18.262119+0000","last_fullsized":"2024-09-10T08:45:18.262119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:54.863098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252195+0000","last_change":"2024-09-10T08:44:17.721602+0000","last_active":"2024-09-10T08:45:18.252195+0000","last_peered":"2024-09-10T08:45:18.252195+0000","last_clean":"2024-09-10T08:45:18.252195+0000","last_became_active":"2024-09-10T08:44:17.721194+0000","last_became_peered":"2024-09-10T08:44:17.721194+0000","last_unstale":"2024-09-10T08:45:18.252195+0000","last_undegraded":"2024-09-10T08:45:18.252195+0000","last_fullsized":"2024-09-10T08:45:18.252195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:39.298494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":93,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:43.294961+0000","last_change":"2024-09-10T08:44:08.661943+0000","last_active":"2024-09-10T08:45:43.294961+0000","last_peered":"2024-09-10T08:45:43.294961+0000","last_clean":"2024-09-10T08:45:43.294961+0000","last_became_active":"2024-09-10T08:44:08.661587+0000","last_became_peered":"2024-09-10T08:44:08.661587+0000","last_unstale":"2024-09-10T08:45:43.294961+0000","last_undegraded":"2024-09-10T08:45:43.294961+0000","last_fullsized":"2024-09-10T08:45:43.294961+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:05.771641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248419+0000","last_change":"2024-09-10T08:44:10.673522+0000","last_active":"2024-09-10T08:45:18.248419+0000","last_peered":"2024-09-10T08:45:18.248419+0000","last_clean":"2024-09-10T08:45:18.248419+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:45:18.248419+0000","last_undegraded":"2024-09-10T08:45:18.248419+0000","last_fullsized":"2024-09-10T08:45:18.248419+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:03.006144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.251983+0000","last_change":"2024-09-10T08:44:17.721306+0000","last_active":"2024-09-10T08:45:18.251983+0000","last_peered":"2024-09-10T08:45:18.251983+0000","last_clean":"2024-09-10T08:45:18.251983+0000","last_became_active":"2024-09-10T08:44:17.720401+0000","last_became_peered":"2024-09-10T08:44:17.720401+0000","last_unstale":"2024-09-10T08:45:18.251983+0000","last_undegraded":"2024-09-10T08:45:18.251983+0000","last_fullsized":"2024-09-10T08:45:18.251983+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:11.589371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252336+0000","last_change":"2024-09-10T08:44:10.678425+0000","last_active":"2024-09-10T08:45:18.252336+0000","last_peered":"2024-09-10T08:45:18.252336+0000","last_clean":"2024-09-10T08:45:18.252336+0000","last_became_active":"2024-09-10T08:44:10.677187+0000","last_became_peered":"2024-09-10T08:44:10.677187+0000","last_unstale":"2024-09-10T08:45:18.252336+0000","last_undegraded":"2024-09-10T08:45:18.252336+0000","last_fullsized":"2024-09-10T08:45:18.252336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:06.527530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252431+0000","last_change":"2024-09-10T08:44:10.674178+0000","last_active":"2024-09-10T08:45:18.252431+0000","last_peered":"2024-09-10T08:45:18.252431+0000","last_clean":"2024-09-10T08:45:18.252431+0000","last_became_active":"2024-09-10T08:44:10.673843+0000","last_became_peered":"2024-09-10T08:44:10.673843+0000","last_unstale":"2024-09-10T08:45:18.252431+0000","last_undegraded":"2024-09-10T08:45:18.252431+0000","last_fullsized":"2024-09-10T08:45:18.252431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:35.023663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.251565+0000","last_change":"2024-09-10T08:44:10.673271+0000","last_active":"2024-09-10T08:45:18.251565+0000","last_peered":"2024-09-10T08:45:18.251565+0000","last_clean":"2024-09-10T08:45:18.251565+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:45:18.251565+0000","last_undegraded":"2024-09-10T08:45:18.251565+0000","last_fullsized":"2024-09-10T08:45:18.251565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:21:26.606820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248019+0000","last_change":"2024-09-10T08:44:10.671348+0000","last_active":"2024-09-10T08:45:18.248019+0000","last_peered":"2024-09-10T08:45:18.248019+0000","last_clean":"2024-09-10T08:45:18.248019+0000","last_became_active":"2024-09-10T08:44:10.670623+0000","last_became_peered":"2024-09-10T08:44:10.670623+0000","last_unstale":"2024-09-10T08:45:18.248019+0000","last_undegraded":"2024-09-10T08:45:18.248019+0000","last_fullsized":"2024-09-10T08:45:18.248019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:06.145694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257532+0000","last_change":"2024-09-10T08:44:10.666011+0000","last_active":"2024-09-10T08:45:18.257532+0000","last_peered":"2024-09-10T08:45:18.257532+0000","last_clean":"2024-09-10T08:45:18.257532+0000","last_became_active":"2024-09-10T08:44:10.665664+0000","last_became_peered":"2024-09-10T08:44:10.665664+0000","last_unstale":"2024-09-10T08:45:18.257532+0000","last_undegraded":"2024-09-10T08:45:18.257532+0000","last_fullsized":"2024-09-10T08:45:18.257532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:31.542102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.251543+0000","last_change":"2024-09-10T08:44:10.675960+0000","last_active":"2024-09-10T08:45:18.251543+0000","last_peered":"2024-09-10T08:45:18.251543+0000","last_clean":"2024-09-10T08:45:18.251543+0000","last_became_active":"2024-09-10T08:44:10.675587+0000","last_became_peered":"2024-09-10T08:44:10.675587+0000","last_unstale":"2024-09-10T08:45:18.251543+0000","last_undegraded":"2024-09-10T08:45:18.251543+0000","last_fullsized":"2024-09-10T08:45:18.251543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:54.634619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252919+0000","last_change":"2024-09-10T08:44:10.681094+0000","last_active":"2024-09-10T08:45:18.252919+0000","last_peered":"2024-09-10T08:45:18.252919+0000","last_clean":"2024-09-10T08:45:18.252919+0000","last_became_active":"2024-09-10T08:44:10.678585+0000","last_became_peered":"2024-09-10T08:44:10.678585+0000","last_unstale":"2024-09-10T08:45:18.252919+0000","last_undegraded":"2024-09-10T08:45:18.252919+0000","last_fullsized":"2024-09-10T08:45:18.252919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:14.951949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254333+0000","last_change":"2024-09-10T08:44:10.675473+0000","last_active":"2024-09-10T08:45:18.254333+0000","last_peered":"2024-09-10T08:45:18.254333+0000","last_clean":"2024-09-10T08:45:18.254333+0000","last_became_active":"2024-09-10T08:44:10.674155+0000","last_became_peered":"2024-09-10T08:44:10.674155+0000","last_unstale":"2024-09-10T08:45:18.254333+0000","last_undegraded":"2024-09-10T08:45:18.254333+0000","last_fullsized":"2024-09-10T08:45:18.254333+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:55.940235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261299+0000","last_change":"2024-09-10T08:44:10.680162+0000","last_active":"2024-09-10T08:45:18.261299+0000","last_peered":"2024-09-10T08:45:18.261299+0000","last_clean":"2024-09-10T08:45:18.261299+0000","last_became_active":"2024-09-10T08:44:10.679800+0000","last_became_peered":"2024-09-10T08:44:10.679800+0000","last_unstale":"2024-09-10T08:45:18.261299+0000","last_undegraded":"2024-09-10T08:45:18.261299+0000","last_fullsized":"2024-09-10T08:45:18.261299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:12.319813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254847+0000","last_change":"2024-09-10T08:44:10.674724+0000","last_active":"2024-09-10T08:45:18.254847+0000","last_peered":"2024-09-10T08:45:18.254847+0000","last_clean":"2024-09-10T08:45:18.254847+0000","last_became_active":"2024-09-10T08:44:10.674389+0000","last_became_peered":"2024-09-10T08:44:10.674389+0000","last_unstale":"2024-09-10T08:45:18.254847+0000","last_undegraded":"2024-09-10T08:45:18.254847+0000","last_fullsized":"2024-09-10T08:45:18.254847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:52.123186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.249100+0000","last_change":"2024-09-10T08:44:10.672805+0000","last_active":"2024-09-10T08:45:18.249100+0000","last_peered":"2024-09-10T08:45:18.249100+0000","last_clean":"2024-09-10T08:45:18.249100+0000","last_became_active":"2024-09-10T08:44:10.672508+0000","last_became_peered":"2024-09-10T08:44:10.672508+0000","last_unstale":"2024-09-10T08:45:18.249100+0000","last_undegraded":"2024-09-10T08:45:18.249100+0000","last_fullsized":"2024-09-10T08:45:18.249100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:48.558633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257040+0000","last_change":"2024-09-10T08:44:10.677504+0000","last_active":"2024-09-10T08:45:18.257040+0000","last_peered":"2024-09-10T08:45:18.257040+0000","last_clean":"2024-09-10T08:45:18.257040+0000","last_became_active":"2024-09-10T08:44:10.676822+0000","last_became_peered":"2024-09-10T08:44:10.676822+0000","last_unstale":"2024-09-10T08:45:18.257040+0000","last_undegraded":"2024-09-10T08:45:18.257040+0000","last_fullsized":"2024-09-10T08:45:18.257040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:05:48.293333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.268208+0000","last_change":"2024-09-10T08:45:18.268208+0000","last_active":"2024-09-10T08:45:18.268208+0000","last_peered":"2024-09-10T08:45:18.268208+0000","last_clean":"2024-09-10T08:45:18.268208+0000","last_became_active":"2024-09-10T08:45:18.267700+0000","last_became_peered":"2024-09-10T08:45:18.267700+0000","last_unstale":"2024-09-10T08:45:18.268208+0000","last_undegraded":"2024-09-10T08:45:18.268208+0000","last_fullsized":"2024-09-10T08:45:18.268208+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:54.487624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252556+0000","last_change":"2024-09-10T08:44:10.670615+0000","last_active":"2024-09-10T08:45:18.252556+0000","last_peered":"2024-09-10T08:45:18.252556+0000","last_clean":"2024-09-10T08:45:18.252556+0000","last_became_active":"2024-09-10T08:44:10.670032+0000","last_became_peered":"2024-09-10T08:44:10.670032+0000","last_unstale":"2024-09-10T08:45:18.252556+0000","last_undegraded":"2024-09-10T08:45:18.252556+0000","last_fullsized":"2024-09-10T08:45:18.252556+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:31.078788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.267175+0000","last_change":"2024-09-10T08:44:10.672367+0000","last_active":"2024-09-10T08:45:18.267175+0000","last_peered":"2024-09-10T08:45:18.267175+0000","last_clean":"2024-09-10T08:45:18.267175+0000","last_became_active":"2024-09-10T08:44:10.672040+0000","last_became_peered":"2024-09-10T08:44:10.672040+0000","last_unstale":"2024-09-10T08:45:18.267175+0000","last_undegraded":"2024-09-10T08:45:18.267175+0000","last_fullsized":"2024-09-10T08:45:18.267175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:10.634708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252491+0000","last_change":"2024-09-10T08:44:10.676375+0000","last_active":"2024-09-10T08:45:18.252491+0000","last_peered":"2024-09-10T08:45:18.252491+0000","last_clean":"2024-09-10T08:45:18.252491+0000","last_became_active":"2024-09-10T08:44:10.676022+0000","last_became_peered":"2024-09-10T08:44:10.676022+0000","last_unstale":"2024-09-10T08:45:18.252491+0000","last_undegraded":"2024-09-10T08:45:18.252491+0000","last_fullsized":"2024-09-10T08:45:18.252491+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:02.681954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248238+0000","last_change":"2024-09-10T08:44:10.673378+0000","last_active":"2024-09-10T08:45:18.248238+0000","last_peered":"2024-09-10T08:45:18.248238+0000","last_clean":"2024-09-10T08:45:18.248238+0000","last_became_active":"2024-09-10T08:44:10.673089+0000","last_became_peered":"2024-09-10T08:44:10.673089+0000","last_unstale":"2024-09-10T08:45:18.248238+0000","last_undegraded":"2024-09-10T08:45:18.248238+0000","last_fullsized":"2024-09-10T08:45:18.248238+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:36:19.258651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.247840+0000","last_change":"2024-09-10T08:44:10.671557+0000","last_active":"2024-09-10T08:45:18.247840+0000","last_peered":"2024-09-10T08:45:18.247840+0000","last_clean":"2024-09-10T08:45:18.247840+0000","last_became_active":"2024-09-10T08:44:10.671248+0000","last_became_peered":"2024-09-10T08:44:10.671248+0000","last_unstale":"2024-09-10T08:45:18.247840+0000","last_undegraded":"2024-09-10T08:45:18.247840+0000","last_fullsized":"2024-09-10T08:45:18.247840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:54.261525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.247887+0000","last_change":"2024-09-10T08:44:10.674672+0000","last_active":"2024-09-10T08:45:18.247887+0000","last_peered":"2024-09-10T08:45:18.247887+0000","last_clean":"2024-09-10T08:45:18.247887+0000","last_became_active":"2024-09-10T08:44:10.673415+0000","last_became_peered":"2024-09-10T08:44:10.673415+0000","last_unstale":"2024-09-10T08:45:18.247887+0000","last_undegraded":"2024-09-10T08:45:18.247887+0000","last_fullsized":"2024-09-10T08:45:18.247887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:59:14.179664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.257050+0000","last_change":"2024-09-10T08:44:10.675293+0000","last_active":"2024-09-10T08:45:18.257050+0000","last_peered":"2024-09-10T08:45:18.257050+0000","last_clean":"2024-09-10T08:45:18.257050+0000","last_became_active":"2024-09-10T08:44:10.674924+0000","last_became_peered":"2024-09-10T08:44:10.674924+0000","last_unstale":"2024-09-10T08:45:18.257050+0000","last_undegraded":"2024-09-10T08:45:18.257050+0000","last_fullsized":"2024-09-10T08:45:18.257050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:56.027320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261968+0000","last_change":"2024-09-10T08:44:10.674605+0000","last_active":"2024-09-10T08:45:18.261968+0000","last_peered":"2024-09-10T08:45:18.261968+0000","last_clean":"2024-09-10T08:45:18.261968+0000","last_became_active":"2024-09-10T08:44:10.674057+0000","last_became_peered":"2024-09-10T08:44:10.674057+0000","last_unstale":"2024-09-10T08:45:18.261968+0000","last_undegraded":"2024-09-10T08:45:18.261968+0000","last_fullsized":"2024-09-10T08:45:18.261968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:09.473003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270641+0000","last_change":"2024-09-10T08:44:10.677410+0000","last_active":"2024-09-10T08:45:18.270641+0000","last_peered":"2024-09-10T08:45:18.270641+0000","last_clean":"2024-09-10T08:45:18.270641+0000","last_became_active":"2024-09-10T08:44:10.677109+0000","last_became_peered":"2024-09-10T08:44:10.677109+0000","last_unstale":"2024-09-10T08:45:18.270641+0000","last_undegraded":"2024-09-10T08:45:18.270641+0000","last_fullsized":"2024-09-10T08:45:18.270641+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:45.455425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.259103+0000","last_change":"2024-09-10T08:44:10.669706+0000","last_active":"2024-09-10T08:45:18.259103+0000","last_peered":"2024-09-10T08:45:18.259103+0000","last_clean":"2024-09-10T08:45:18.259103+0000","last_became_active":"2024-09-10T08:44:10.669440+0000","last_became_peered":"2024-09-10T08:44:10.669440+0000","last_unstale":"2024-09-10T08:45:18.259103+0000","last_undegraded":"2024-09-10T08:45:18.259103+0000","last_fullsized":"2024-09-10T08:45:18.259103+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:31:55.610651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.249193+0000","last_change":"2024-09-10T08:44:10.676482+0000","last_active":"2024-09-10T08:45:18.249193+0000","last_peered":"2024-09-10T08:45:18.249193+0000","last_clean":"2024-09-10T08:45:18.249193+0000","last_became_active":"2024-09-10T08:44:10.675983+0000","last_became_peered":"2024-09-10T08:44:10.675983+0000","last_unstale":"2024-09-10T08:45:18.249193+0000","last_undegraded":"2024-09-10T08:45:18.249193+0000","last_fullsized":"2024-09-10T08:45:18.249193+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:01.269283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.254722+0000","last_change":"2024-09-10T08:44:10.675681+0000","last_active":"2024-09-10T08:45:18.254722+0000","last_peered":"2024-09-10T08:45:18.254722+0000","last_clean":"2024-09-10T08:45:18.254722+0000","last_became_active":"2024-09-10T08:44:10.675357+0000","last_became_peered":"2024-09-10T08:44:10.675357+0000","last_unstale":"2024-09-10T08:45:18.254722+0000","last_undegraded":"2024-09-10T08:45:18.254722+0000","last_fullsized":"2024-09-10T08:45:18.254722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:26.138333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.270056+0000","last_change":"2024-09-10T08:44:10.680819+0000","last_active":"2024-09-10T08:45:18.270056+0000","last_peered":"2024-09-10T08:45:18.270056+0000","last_clean":"2024-09-10T08:45:18.270056+0000","last_became_active":"2024-09-10T08:44:10.680491+0000","last_became_peered":"2024-09-10T08:44:10.680491+0000","last_unstale":"2024-09-10T08:45:18.270056+0000","last_undegraded":"2024-09-10T08:45:18.270056+0000","last_fullsized":"2024-09-10T08:45:18.270056+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:17.044776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.258679+0000","last_change":"2024-09-10T08:44:10.667376+0000","last_active":"2024-09-10T08:45:18.258679+0000","last_peered":"2024-09-10T08:45:18.258679+0000","last_clean":"2024-09-10T08:45:18.258679+0000","last_became_active":"2024-09-10T08:44:10.666648+0000","last_became_peered":"2024-09-10T08:44:10.666648+0000","last_unstale":"2024-09-10T08:45:18.258679+0000","last_undegraded":"2024-09-10T08:45:18.258679+0000","last_fullsized":"2024-09-10T08:45:18.258679+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:33:12.017550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248402+0000","last_change":"2024-09-10T08:44:10.668665+0000","last_active":"2024-09-10T08:45:18.248402+0000","last_peered":"2024-09-10T08:45:18.248402+0000","last_clean":"2024-09-10T08:45:18.248402+0000","last_became_active":"2024-09-10T08:44:10.666031+0000","last_became_peered":"2024-09-10T08:44:10.666031+0000","last_unstale":"2024-09-10T08:45:18.248402+0000","last_undegraded":"2024-09-10T08:45:18.248402+0000","last_fullsized":"2024-09-10T08:45:18.248402+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:39:53.991463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.261851+0000","last_change":"2024-09-10T08:44:10.676495+0000","last_active":"2024-09-10T08:45:18.261851+0000","last_peered":"2024-09-10T08:45:18.261851+0000","last_clean":"2024-09-10T08:45:18.261851+0000","last_became_active":"2024-09-10T08:44:10.676087+0000","last_became_peered":"2024-09-10T08:44:10.676087+0000","last_unstale":"2024-09-10T08:45:18.261851+0000","last_undegraded":"2024-09-10T08:45:18.261851+0000","last_fullsized":"2024-09-10T08:45:18.261851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:41.828157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248048+0000","last_change":"2024-09-10T08:44:10.672835+0000","last_active":"2024-09-10T08:45:18.248048+0000","last_peered":"2024-09-10T08:45:18.248048+0000","last_clean":"2024-09-10T08:45:18.248048+0000","last_became_active":"2024-09-10T08:44:10.672537+0000","last_became_peered":"2024-09-10T08:44:10.672537+0000","last_unstale":"2024-09-10T08:45:18.248048+0000","last_undegraded":"2024-09-10T08:45:18.248048+0000","last_fullsized":"2024-09-10T08:45:18.248048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:06.121460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.248222+0000","last_change":"2024-09-10T08:44:10.675968+0000","last_active":"2024-09-10T08:45:18.248222+0000","last_peered":"2024-09-10T08:45:18.248222+0000","last_clean":"2024-09-10T08:45:18.248222+0000","last_became_active":"2024-09-10T08:44:10.675628+0000","last_became_peered":"2024-09-10T08:44:10.675628+0000","last_unstale":"2024-09-10T08:45:18.248222+0000","last_undegraded":"2024-09-10T08:45:18.248222+0000","last_fullsized":"2024-09-10T08:45:18.248222+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:40.261336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T08:45:18.252266+0000","last_change":"2024-09-10T08:44:10.673151+0000","last_active":"2024-09-10T08:45:18.252266+0000","last_peered":"2024-09-10T08:45:18.252266+0000","last_clean":"2024-09-10T08:45:18.252266+0000","last_became_active":"2024-09-10T08:44:10.672833+0000","last_became_peered":"2024-09-10T08:44:10.672833+0000","last_unstale":"2024-09-10T08:45:18.252266+0000","last_undegraded":"2024-09-10T08:45:18.252266+0000","last_fullsized":"2024-09-10T08:45:18.252266+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:38.556222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70299,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":551,"num_read_kb":353,"num_write":248,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":210897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514529,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1024000,"data_stored":504097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743462,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28712,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716440,"statfs":{"total":95995035648,"available":95965634560,"internally_reserved":0,"allocated":1720320,"data_stored":1154004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906987,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1052672,"data_stored":507730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135919,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1044480,"data_stored":504012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364852,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1642496,"data_stored":1100118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528378,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1024000,"data_stored":499138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36964,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708188,"statfs":{"total":95995035648,"available":95957184512,"internally_reserved":0,"allocated":1650688,"data_stored":1095700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1032192,"data_stored":503212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1036288,"data_stored":498846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":564265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1093632,"data_stored":563652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1048576,"data_stored":500987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:45:49.800 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T08:45:49.800 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T08:45:49.800 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T08:45:49.801 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T08:45:50.047 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:50.455 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:50 smithi172 ceph-mon[30854]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:45:50.455 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:50 smithi172 ceph-mon[30854]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:50.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:50 smithi174 ceph-mon[34068]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:45:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:50 smithi174 ceph-mon[34068]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:50.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:50 smithi191 ceph-mon[35314]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:45:50.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:50 smithi191 ceph-mon[35314]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:50.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.532+0000 7fc5f7477640 1 -- 172.21.15.172:0/3724441540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f01082a0 msgr2=0x7fc5f010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:50.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.532+0000 7fc5f7477640 1 --2- 172.21.15.172:0/3724441540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f01082a0 0x7fc5f010ec40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009930 tx=0x7fc5e002a440 comp rx=0 tx=0).stop 2024-09-10T08:45:50.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.532+0000 7fc5f7477640 1 -- 172.21.15.172:0/3724441540 shutdown_connections 2024-09-10T08:45:50.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.532+0000 7fc5f7477640 1 --2- 172.21.15.172:0/3724441540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f01082a0 0x7fc5f010ec40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:50.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.532+0000 7fc5f7477640 1 --2- 172.21.15.172:0/3724441540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5f01036e0 0x7fc5f0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:50.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.533+0000 7fc5f7477640 1 --2- 172.21.15.172:0/3724441540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5f0102d30 0x7fc5f0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:50.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.533+0000 7fc5f7477640 1 -- 172.21.15.172:0/3724441540 >> 172.21.15.172:0/3724441540 conn(0x7fc5f00fc7d0 msgr2=0x7fc5f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:50.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.533+0000 7fc5f7477640 1 -- 172.21.15.172:0/3724441540 shutdown_connections 2024-09-10T08:45:50.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.533+0000 7fc5f7477640 1 -- 172.21.15.172:0/3724441540 wait complete. 2024-09-10T08:45:50.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.533+0000 7fc5f7477640 1 Processor -- start 2024-09-10T08:45:50.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.533+0000 7fc5f7477640 1 -- start start 2024-09-10T08:45:50.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.533+0000 7fc5f7477640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5f0102d30 0x7fc5f0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:50.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.533+0000 7fc5f7477640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f01036e0 0x7fc5f01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:50.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.533+0000 7fc5f7477640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5f01082a0 0x7fc5f019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:50.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.533+0000 7fc5f7477640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f0111f90 con 0x7fc5f01036e0 2024-09-10T08:45:50.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.533+0000 7fc5f7477640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f0111e10 con 0x7fc5f0102d30 2024-09-10T08:45:50.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.533+0000 7fc5f7477640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f0112110 con 0x7fc5f01082a0 2024-09-10T08:45:50.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5f4a12640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f01036e0 0x7fc5f01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:50.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5f4a12640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f01036e0 0x7fc5f01989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52500/0 (socket says 172.21.15.172:52500) 2024-09-10T08:45:50.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5f4a12640 1 -- 172.21.15.172:0/1665842368 learned_addr learned my addr 172.21.15.172:0/1665842368 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:50.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5f5a14640 1 --2- 172.21.15.172:0/1665842368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5f01082a0 0x7fc5f019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:50.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5f5213640 1 --2- 172.21.15.172:0/1665842368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5f0102d30 0x7fc5f0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:50.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5f4a12640 1 -- 172.21.15.172:0/1665842368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5f01082a0 msgr2=0x7fc5f019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:50.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5f4a12640 1 --2- 172.21.15.172:0/1665842368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5f01082a0 0x7fc5f019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:50.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5f4a12640 1 -- 172.21.15.172:0/1665842368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5f0102d30 msgr2=0x7fc5f0198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:50.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5f4a12640 1 --2- 172.21.15.172:0/1665842368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5f0102d30 0x7fc5f0198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:50.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5f4a12640 1 -- 172.21.15.172:0/1665842368 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f019d430 con 0x7fc5f01036e0 2024-09-10T08:45:50.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5f5213640 1 --2- 172.21.15.172:0/1665842368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5f0102d30 0x7fc5f0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:50.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5f5a14640 1 --2- 172.21.15.172:0/1665842368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5f01082a0 0x7fc5f019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:50.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5f4a12640 1 --2- 172.21.15.172:0/1665842368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f01036e0 0x7fc5f01989c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400b990 tx=0x7fc5e400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:50.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5ee7fc640 1 -- 172.21.15.172:0/1665842368 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4019070 con 0x7fc5f01036e0 2024-09-10T08:45:50.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5ee7fc640 1 -- 172.21.15.172:0/1665842368 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5e400cba0 con 0x7fc5f01036e0 2024-09-10T08:45:50.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5f7477640 1 -- 172.21.15.172:0/1665842368 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f019d720 con 0x7fc5f01036e0 2024-09-10T08:45:50.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5ee7fc640 1 -- 172.21.15.172:0/1665842368 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e40125a0 con 0x7fc5f01036e0 2024-09-10T08:45:50.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.534+0000 7fc5f7477640 1 -- 172.21.15.172:0/1665842368 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f01a5000 con 0x7fc5f01036e0 2024-09-10T08:45:50.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.535+0000 7fc5f7477640 1 -- 172.21.15.172:0/1665842368 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f0103110 con 0x7fc5f01036e0 2024-09-10T08:45:50.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.536+0000 7fc5ee7fc640 1 -- 172.21.15.172:0/1665842368 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc5e401c070 con 0x7fc5f01036e0 2024-09-10T08:45:50.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.537+0000 7fc5ee7fc640 1 --2- 172.21.15.172:0/1665842368 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5d80a2c10 0x7fc5d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:50.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.537+0000 7fc5f5213640 1 --2- 172.21.15.172:0/1665842368 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5d80a2c10 0x7fc5d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:50.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.537+0000 7fc5ee7fc640 1 -- 172.21.15.172:0/1665842368 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fc5e40cea80 con 0x7fc5f01036e0 2024-09-10T08:45:50.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.538+0000 7fc5f5213640 1 --2- 172.21.15.172:0/1665842368 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5d80a2c10 0x7fc5d80a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80058c0 tx=0x7fc5e8005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:50.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.541+0000 7fc5ee7fc640 1 -- 172.21.15.172:0/1665842368 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5e409b650 con 0x7fc5f01036e0 2024-09-10T08:45:50.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.743+0000 7fc5f7477640 1 -- 172.21.15.172:0/1665842368 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc5f010a2e0 con 0x7fc5f01036e0 2024-09-10T08:45:50.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.743+0000 7fc5ee7fc640 1 -- 172.21.15.172:0/1665842368 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc5e409af50 con 0x7fc5f01036e0 2024-09-10T08:45:50.745 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:45:50.745 INFO:teuthology.orchestra.run.smithi172.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T08:45:50.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.745+0000 7fc5f7477640 1 -- 172.21.15.172:0/1665842368 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5d80a2c10 msgr2=0x7fc5d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:50.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.745+0000 7fc5f7477640 1 --2- 172.21.15.172:0/1665842368 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5d80a2c10 0x7fc5d80a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80058c0 tx=0x7fc5e8005d30 comp rx=0 tx=0).stop 2024-09-10T08:45:50.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.745+0000 7fc5f7477640 1 -- 172.21.15.172:0/1665842368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f01036e0 msgr2=0x7fc5f01989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:50.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.745+0000 7fc5f7477640 1 --2- 172.21.15.172:0/1665842368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f01036e0 0x7fc5f01989c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400b990 tx=0x7fc5e400be50 comp rx=0 tx=0).stop 2024-09-10T08:45:50.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.745+0000 7fc5f7477640 1 -- 172.21.15.172:0/1665842368 shutdown_connections 2024-09-10T08:45:50.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.745+0000 7fc5f7477640 1 --2- 172.21.15.172:0/1665842368 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5d80a2c10 0x7fc5d80a50d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:50.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.745+0000 7fc5f7477640 1 --2- 172.21.15.172:0/1665842368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5f01082a0 0x7fc5f019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:50.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.745+0000 7fc5f7477640 1 --2- 172.21.15.172:0/1665842368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f01036e0 0x7fc5f01989c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:50.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.745+0000 7fc5f7477640 1 --2- 172.21.15.172:0/1665842368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5f0102d30 0x7fc5f0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:50.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.745+0000 7fc5f7477640 1 -- 172.21.15.172:0/1665842368 >> 172.21.15.172:0/1665842368 conn(0x7fc5f00fc7d0 msgr2=0x7fc5f0105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:50.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.745+0000 7fc5f7477640 1 -- 172.21.15.172:0/1665842368 shutdown_connections 2024-09-10T08:45:50.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:50.745+0000 7fc5f7477640 1 -- 172.21.15.172:0/1665842368 wait complete. 2024-09-10T08:45:51.436 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:51 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1665842368' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T08:45:51.463 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T08:45:51.464 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:45:51.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:51 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1665842368' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T08:45:51.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:51 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1665842368' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T08:45:51.715 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:52.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.244+0000 7f4557f3d640 1 -- 172.21.15.172:0/1677188336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4550101de0 msgr2=0x7f45501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:52.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.244+0000 7f4557f3d640 1 --2- 172.21.15.172:0/1677188336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4550101de0 0x7f45501021c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4540009900 tx=0x7f454002a410 comp rx=0 tx=0).stop 2024-09-10T08:45:52.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.244+0000 7f4557f3d640 1 -- 172.21.15.172:0/1677188336 shutdown_connections 2024-09-10T08:45:52.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.244+0000 7f4557f3d640 1 --2- 172.21.15.172:0/1677188336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4550106f10 0x7f455010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:52.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.244+0000 7f4557f3d640 1 --2- 172.21.15.172:0/1677188336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4550102790 0x7f45501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:52.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.244+0000 7f4557f3d640 1 --2- 172.21.15.172:0/1677188336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4550101de0 0x7f45501021c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:52.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.244+0000 7f4557f3d640 1 -- 172.21.15.172:0/1677188336 >> 172.21.15.172:0/1677188336 conn(0x7f45500fd8c0 msgr2=0x7f45500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:52.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.244+0000 7f4557f3d640 1 -- 172.21.15.172:0/1677188336 shutdown_connections 2024-09-10T08:45:52.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.244+0000 7f4557f3d640 1 -- 172.21.15.172:0/1677188336 wait complete. 2024-09-10T08:45:52.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.245+0000 7f4557f3d640 1 Processor -- start 2024-09-10T08:45:52.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.245+0000 7f4557f3d640 1 -- start start 2024-09-10T08:45:52.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.245+0000 7f4557f3d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4550101de0 0x7f45501983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:52.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.245+0000 7f4557f3d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4550102790 0x7f4550198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:52.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.245+0000 7f4557f3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4550106f10 0x7f455019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:52.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.245+0000 7f4557f3d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4550111e40 con 0x7f4550106f10 2024-09-10T08:45:52.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.245+0000 7f4557f3d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4550111cc0 con 0x7f4550101de0 2024-09-10T08:45:52.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.245+0000 7f4557f3d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4550111fc0 con 0x7f4550102790 2024-09-10T08:45:52.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.245+0000 7f45564da640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4550106f10 0x7f455019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:52.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.245+0000 7f45564da640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4550106f10 0x7f455019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52532/0 (socket says 172.21.15.172:52532) 2024-09-10T08:45:52.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.245+0000 7f45564da640 1 -- 172.21.15.172:0/1012323181 learned_addr learned my addr 172.21.15.172:0/1012323181 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:52.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.246+0000 7f4555cd9640 1 --2- 172.21.15.172:0/1012323181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4550101de0 0x7f45501983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:52.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.246+0000 7f45554d8640 1 --2- 172.21.15.172:0/1012323181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4550102790 0x7f4550198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:52.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.246+0000 7f45564da640 1 -- 172.21.15.172:0/1012323181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4550102790 msgr2=0x7f4550198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:52.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.246+0000 7f45564da640 1 --2- 172.21.15.172:0/1012323181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4550102790 0x7f4550198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:52.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.246+0000 7f45564da640 1 -- 172.21.15.172:0/1012323181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4550101de0 msgr2=0x7f45501983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:52.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.246+0000 7f45564da640 1 --2- 172.21.15.172:0/1012323181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4550101de0 0x7f45501983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:52.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.246+0000 7f45564da640 1 -- 172.21.15.172:0/1012323181 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455019d390 con 0x7f4550106f10 2024-09-10T08:45:52.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.246+0000 7f45554d8640 1 --2- 172.21.15.172:0/1012323181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4550102790 0x7f4550198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:52.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.246+0000 7f45564da640 1 --2- 172.21.15.172:0/1012323181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4550106f10 0x7f455019ccb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f454800b970 tx=0x7f454800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:52.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.246+0000 7f4546ffd640 1 -- 172.21.15.172:0/1012323181 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548019070 con 0x7f4550106f10 2024-09-10T08:45:52.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.246+0000 7f4546ffd640 1 -- 172.21.15.172:0/1012323181 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4548014440 con 0x7f4550106f10 2024-09-10T08:45:52.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.246+0000 7f4557f3d640 1 -- 172.21.15.172:0/1012323181 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455019d680 con 0x7f4550106f10 2024-09-10T08:45:52.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.246+0000 7f4546ffd640 1 -- 172.21.15.172:0/1012323181 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45480075d0 con 0x7f4550106f10 2024-09-10T08:45:52.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.246+0000 7f4557f3d640 1 -- 172.21.15.172:0/1012323181 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45501a4f60 con 0x7f4550106f10 2024-09-10T08:45:52.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.247+0000 7f4557f3d640 1 -- 172.21.15.172:0/1012323181 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45501021c0 con 0x7f4550106f10 2024-09-10T08:45:52.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.248+0000 7f4546ffd640 1 -- 172.21.15.172:0/1012323181 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f454801c070 con 0x7f4550106f10 2024-09-10T08:45:52.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.249+0000 7f4546ffd640 1 --2- 172.21.15.172:0/1012323181 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f453c0a2c10 0x7f453c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:52.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.249+0000 7f4555cd9640 1 --2- 172.21.15.172:0/1012323181 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f453c0a2c10 0x7f453c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:52.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.249+0000 7f4546ffd640 1 -- 172.21.15.172:0/1012323181 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f45480cee00 con 0x7f4550106f10 2024-09-10T08:45:52.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.250+0000 7f4555cd9640 1 --2- 172.21.15.172:0/1012323181 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f453c0a2c10 0x7f453c0a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4540031440 tx=0x7f4540005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:52.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.252+0000 7f4546ffd640 1 -- 172.21.15.172:0/1012323181 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f454809b380 con 0x7f4550106f10 2024-09-10T08:45:52.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.454+0000 7f4557f3d640 1 -- 172.21.15.172:0/1012323181 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f455010a150 con 0x7f4550106f10 2024-09-10T08:45:52.456 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:52 smithi172 ceph-mon[30854]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T08:45:52.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.455+0000 7f4546ffd640 1 -- 172.21.15.172:0/1012323181 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+12001 (secure 0 0 0) 0x7f454809f680 con 0x7f4550106f10 2024-09-10T08:45:52.457 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:45:52.457 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":28,"btime":"2024-09-10T08:44:55:226722+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:44:55.226705+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T08:45:52.458 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 28 2024-09-10T08:45:52.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.458+0000 7f4557f3d640 1 -- 172.21.15.172:0/1012323181 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f453c0a2c10 msgr2=0x7f453c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:52.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.458+0000 7f4557f3d640 1 --2- 172.21.15.172:0/1012323181 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f453c0a2c10 0x7f453c0a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4540031440 tx=0x7f4540005e80 comp rx=0 tx=0).stop 2024-09-10T08:45:52.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.458+0000 7f4557f3d640 1 -- 172.21.15.172:0/1012323181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4550106f10 msgr2=0x7f455019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:52.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.458+0000 7f4557f3d640 1 --2- 172.21.15.172:0/1012323181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4550106f10 0x7f455019ccb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f454800b970 tx=0x7f454800be30 comp rx=0 tx=0).stop 2024-09-10T08:45:52.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.458+0000 7f4557f3d640 1 -- 172.21.15.172:0/1012323181 shutdown_connections 2024-09-10T08:45:52.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.458+0000 7f4557f3d640 1 --2- 172.21.15.172:0/1012323181 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f453c0a2c10 0x7f453c0a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:52.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.458+0000 7f4557f3d640 1 --2- 172.21.15.172:0/1012323181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4550106f10 0x7f455019ccb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:52.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.458+0000 7f4557f3d640 1 --2- 172.21.15.172:0/1012323181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4550102790 0x7f4550198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:52.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.458+0000 7f4557f3d640 1 --2- 172.21.15.172:0/1012323181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4550101de0 0x7f45501983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:52.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.458+0000 7f4557f3d640 1 -- 172.21.15.172:0/1012323181 >> 172.21.15.172:0/1012323181 conn(0x7f45500fd8c0 msgr2=0x7f45500fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:52.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.458+0000 7f4557f3d640 1 -- 172.21.15.172:0/1012323181 shutdown_connections 2024-09-10T08:45:52.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:52.458+0000 7f4557f3d640 1 -- 172.21.15.172:0/1012323181 wait complete. 2024-09-10T08:45:52.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:52 smithi174 ceph-mon[34068]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T08:45:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:52 smithi191 ceph-mon[35314]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T08:45:53.140 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:45:53.394 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:53.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:53 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1012323181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:53.515 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:53 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1012323181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:53.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:53 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1012323181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:53.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.903+0000 7fa7a5e7d640 1 -- 172.21.15.172:0/276229549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7a0103660 msgr2=0x7fa7a010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:53.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.903+0000 7fa7a5e7d640 1 --2- 172.21.15.172:0/276229549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7a0103660 0x7fa7a010bb50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa794009900 tx=0x7fa79402a480 comp rx=0 tx=0).stop 2024-09-10T08:45:53.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.903+0000 7fa7a5e7d640 1 -- 172.21.15.172:0/276229549 shutdown_connections 2024-09-10T08:45:53.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.903+0000 7fa7a5e7d640 1 --2- 172.21.15.172:0/276229549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7a010c170 0x7fa7a010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:53.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.903+0000 7fa7a5e7d640 1 --2- 172.21.15.172:0/276229549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7a0103660 0x7fa7a010bb50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:53.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.903+0000 7fa7a5e7d640 1 --2- 172.21.15.172:0/276229549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7a0102d40 0x7fa7a0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:53.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.903+0000 7fa7a5e7d640 1 -- 172.21.15.172:0/276229549 >> 172.21.15.172:0/276229549 conn(0x7fa7a00fc7a0 msgr2=0x7fa7a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:53.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.903+0000 7fa7a5e7d640 1 -- 172.21.15.172:0/276229549 shutdown_connections 2024-09-10T08:45:53.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.903+0000 7fa7a5e7d640 1 -- 172.21.15.172:0/276229549 wait complete. 2024-09-10T08:45:53.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.904+0000 7fa7a5e7d640 1 Processor -- start 2024-09-10T08:45:53.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.904+0000 7fa7a5e7d640 1 -- start start 2024-09-10T08:45:53.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa7a5e7d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7a0102d40 0x7fa7a0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:53.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa7a5e7d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7a0103660 0x7fa7a01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:53.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa7a5e7d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7a010c170 0x7fa7a019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:53.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa7a5e7d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a0111e20 con 0x7fa7a010c170 2024-09-10T08:45:53.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa7a5e7d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a0111ca0 con 0x7fa7a0103660 2024-09-10T08:45:53.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa7a5e7d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a0111fa0 con 0x7fa7a0102d40 2024-09-10T08:45:53.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa79ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7a010c170 0x7fa7a019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:53.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa79ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7a010c170 0x7fa7a019cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52552/0 (socket says 172.21.15.172:52552) 2024-09-10T08:45:53.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa79ffff640 1 -- 172.21.15.172:0/2228774249 learned_addr learned my addr 172.21.15.172:0/2228774249 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:53.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa79f7fe640 1 --2- 172.21.15.172:0/2228774249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7a0102d40 0x7fa7a0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:53.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa79effd640 1 --2- 172.21.15.172:0/2228774249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7a0103660 0x7fa7a01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:53.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa79ffff640 1 -- 172.21.15.172:0/2228774249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7a0102d40 msgr2=0x7fa7a0198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:53.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa79ffff640 1 --2- 172.21.15.172:0/2228774249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7a0102d40 0x7fa7a0198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:53.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa79ffff640 1 -- 172.21.15.172:0/2228774249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7a0103660 msgr2=0x7fa7a01988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:53.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa79ffff640 1 --2- 172.21.15.172:0/2228774249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7a0103660 0x7fa7a01988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:53.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa79ffff640 1 -- 172.21.15.172:0/2228774249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a019d320 con 0x7fa7a010c170 2024-09-10T08:45:53.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa79effd640 1 --2- 172.21.15.172:0/2228774249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7a0103660 0x7fa7a01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:53.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa79ffff640 1 --2- 172.21.15.172:0/2228774249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7a010c170 0x7fa7a019cc40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa79800d750 tx=0x7fa79800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:53.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa79cff9640 1 -- 172.21.15.172:0/2228774249 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798017070 con 0x7fa7a010c170 2024-09-10T08:45:53.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa79cff9640 1 -- 172.21.15.172:0/2228774249 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7980054d0 con 0x7fa7a010c170 2024-09-10T08:45:53.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa7a5e7d640 1 -- 172.21.15.172:0/2228774249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a019d610 con 0x7fa7a010c170 2024-09-10T08:45:53.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa79cff9640 1 -- 172.21.15.172:0/2228774249 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798004c60 con 0x7fa7a010c170 2024-09-10T08:45:53.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.905+0000 7fa7a5e7d640 1 -- 172.21.15.172:0/2228774249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a01a4ef0 con 0x7fa7a010c170 2024-09-10T08:45:53.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.907+0000 7fa7a5e7d640 1 -- 172.21.15.172:0/2228774249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7a0103120 con 0x7fa7a010c170 2024-09-10T08:45:53.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.908+0000 7fa79cff9640 1 -- 172.21.15.172:0/2228774249 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa798004e00 con 0x7fa7a010c170 2024-09-10T08:45:53.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.908+0000 7fa79cff9640 1 --2- 172.21.15.172:0/2228774249 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7880a2c10 0x7fa7880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:53.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.908+0000 7fa79cff9640 1 -- 172.21.15.172:0/2228774249 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fa7980cee50 con 0x7fa7a010c170 2024-09-10T08:45:53.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.908+0000 7fa79f7fe640 1 --2- 172.21.15.172:0/2228774249 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7880a2c10 0x7fa7880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:53.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.909+0000 7fa79f7fe640 1 --2- 172.21.15.172:0/2228774249 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7880a2c10 0x7fa7880a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa790005fd0 tx=0x7fa790009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:53.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:53.913+0000 7fa79cff9640 1 -- 172.21.15.172:0/2228774249 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa79809b910 con 0x7fa7a010c170 2024-09-10T08:45:54.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:54.114+0000 7fa7a5e7d640 1 -- 172.21.15.172:0/2228774249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7a01071e0 con 0x7fa7a010c170 2024-09-10T08:45:54.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:54.114+0000 7fa79cff9640 1 -- 172.21.15.172:0/2228774249 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+12001 (secure 0 0 0) 0x7fa79809b210 con 0x7fa7a010c170 2024-09-10T08:45:54.117 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:45:54.117 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":28,"btime":"2024-09-10T08:44:55:226722+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:44:55.226705+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T08:45:54.117 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 28 2024-09-10T08:45:54.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:54.117+0000 7fa7a5e7d640 1 -- 172.21.15.172:0/2228774249 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7880a2c10 msgr2=0x7fa7880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:54.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:54.117+0000 7fa7a5e7d640 1 --2- 172.21.15.172:0/2228774249 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7880a2c10 0x7fa7880a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa790005fd0 tx=0x7fa790009680 comp rx=0 tx=0).stop 2024-09-10T08:45:54.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:54.117+0000 7fa7a5e7d640 1 -- 172.21.15.172:0/2228774249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7a010c170 msgr2=0x7fa7a019cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:54.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:54.117+0000 7fa7a5e7d640 1 --2- 172.21.15.172:0/2228774249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7a010c170 0x7fa7a019cc40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa79800d750 tx=0x7fa79800dc10 comp rx=0 tx=0).stop 2024-09-10T08:45:54.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:54.117+0000 7fa7a5e7d640 1 -- 172.21.15.172:0/2228774249 shutdown_connections 2024-09-10T08:45:54.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:54.117+0000 7fa7a5e7d640 1 --2- 172.21.15.172:0/2228774249 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7880a2c10 0x7fa7880a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:54.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:54.117+0000 7fa7a5e7d640 1 --2- 172.21.15.172:0/2228774249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7a010c170 0x7fa7a019cc40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:54.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:54.117+0000 7fa7a5e7d640 1 --2- 172.21.15.172:0/2228774249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7a0103660 0x7fa7a01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:54.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:54.117+0000 7fa7a5e7d640 1 --2- 172.21.15.172:0/2228774249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7a0102d40 0x7fa7a0198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:54.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:54.117+0000 7fa7a5e7d640 1 -- 172.21.15.172:0/2228774249 >> 172.21.15.172:0/2228774249 conn(0x7fa7a00fc7a0 msgr2=0x7fa7a00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:54.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:54.117+0000 7fa7a5e7d640 1 -- 172.21.15.172:0/2228774249 shutdown_connections 2024-09-10T08:45:54.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:54.117+0000 7fa7a5e7d640 1 -- 172.21.15.172:0/2228774249 wait complete. 2024-09-10T08:45:54.479 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:54 smithi172 ceph-mon[30854]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:54.479 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:54 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2228774249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:54.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:54 smithi174 ceph-mon[34068]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:54.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:54 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2228774249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:54 smithi191 ceph-mon[35314]: pgmap v338: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:54 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2228774249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:54.810 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:45:55.058 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:55.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.530+0000 7f199dd6c640 1 -- 172.21.15.172:0/1969274182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1998106a80 msgr2=0x7f1998106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:55.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.530+0000 7f199dd6c640 1 --2- 172.21.15.172:0/1969274182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1998106a80 0x7f1998106e60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1988009930 tx=0x7f198802a420 comp rx=0 tx=0).stop 2024-09-10T08:45:55.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.530+0000 7f199dd6c640 1 -- 172.21.15.172:0/1969274182 shutdown_connections 2024-09-10T08:45:55.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.530+0000 7f199dd6c640 1 --2- 172.21.15.172:0/1969274182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1998101420 0x7f199810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:55.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.530+0000 7f199dd6c640 1 --2- 172.21.15.172:0/1969274182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1998100a80 0x7f1998100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:55.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.530+0000 7f199dd6c640 1 --2- 172.21.15.172:0/1969274182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1998106a80 0x7f1998106e60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:55.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.530+0000 7f199dd6c640 1 -- 172.21.15.172:0/1969274182 >> 172.21.15.172:0/1969274182 conn(0x7f19980fc7a0 msgr2=0x7f19980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:55.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.530+0000 7f199dd6c640 1 -- 172.21.15.172:0/1969274182 shutdown_connections 2024-09-10T08:45:55.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.531+0000 7f199dd6c640 1 -- 172.21.15.172:0/1969274182 wait complete. 2024-09-10T08:45:55.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.531+0000 7f199dd6c640 1 Processor -- start 2024-09-10T08:45:55.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.531+0000 7f199dd6c640 1 -- start start 2024-09-10T08:45:55.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.531+0000 7f199dd6c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1998100a80 0x7f1998197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:55.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.531+0000 7f199dd6c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1998101420 0x7f1998197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:55.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.531+0000 7f199dd6c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1998106a80 0x7f199819bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:55.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.531+0000 7f199dd6c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1998110d50 con 0x7f1998100a80 2024-09-10T08:45:55.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.531+0000 7f199dd6c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1998110bd0 con 0x7f1998106a80 2024-09-10T08:45:55.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.531+0000 7f199dd6c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1998110ed0 con 0x7f1998101420 2024-09-10T08:45:55.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.531+0000 7f19977fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1998100a80 0x7f1998197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:55.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.531+0000 7f19977fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1998100a80 0x7f1998197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52580/0 (socket says 172.21.15.172:52580) 2024-09-10T08:45:55.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.531+0000 7f19977fe640 1 -- 172.21.15.172:0/1698250972 learned_addr learned my addr 172.21.15.172:0/1698250972 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:55.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.532+0000 7f19977fe640 1 -- 172.21.15.172:0/1698250972 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1998101420 msgr2=0x7f1998197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:55.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.532+0000 7f1997fff640 1 --2- 172.21.15.172:0/1698250972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1998106a80 0x7f199819bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:55.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.532+0000 7f19977fe640 1 --2- 172.21.15.172:0/1698250972 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1998101420 0x7f1998197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:55.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.532+0000 7f19977fe640 1 -- 172.21.15.172:0/1698250972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1998106a80 msgr2=0x7f199819bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:55.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.532+0000 7f19977fe640 1 --2- 172.21.15.172:0/1698250972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1998106a80 0x7f199819bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:55.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.532+0000 7f19977fe640 1 -- 172.21.15.172:0/1698250972 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199819c300 con 0x7f1998100a80 2024-09-10T08:45:55.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.532+0000 7f19977fe640 1 --2- 172.21.15.172:0/1698250972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1998100a80 0x7f1998197350 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1988004770 tx=0x7f1988004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:55.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.532+0000 7f19957fa640 1 -- 172.21.15.172:0/1698250972 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988035050 con 0x7f1998100a80 2024-09-10T08:45:55.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.532+0000 7f19957fa640 1 -- 172.21.15.172:0/1698250972 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1988003aa0 con 0x7f1998100a80 2024-09-10T08:45:55.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.532+0000 7f199dd6c640 1 -- 172.21.15.172:0/1698250972 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199819c590 con 0x7f1998100a80 2024-09-10T08:45:55.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.532+0000 7f19957fa640 1 -- 172.21.15.172:0/1698250972 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988038ae0 con 0x7f1998100a80 2024-09-10T08:45:55.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.532+0000 7f199dd6c640 1 -- 172.21.15.172:0/1698250972 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199806a790 con 0x7f1998100a80 2024-09-10T08:45:55.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.533+0000 7f199dd6c640 1 -- 172.21.15.172:0/1698250972 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19981021c0 con 0x7f1998100a80 2024-09-10T08:45:55.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.534+0000 7f19957fa640 1 -- 172.21.15.172:0/1698250972 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1988003c50 con 0x7f1998100a80 2024-09-10T08:45:55.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.535+0000 7f19957fa640 1 --2- 172.21.15.172:0/1698250972 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f19840a2c90 0x7f19840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:55.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.535+0000 7f198ffff640 1 --2- 172.21.15.172:0/1698250972 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f19840a2c90 0x7f19840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:55.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.535+0000 7f19957fa640 1 -- 172.21.15.172:0/1698250972 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f19880f16b0 con 0x7f1998100a80 2024-09-10T08:45:55.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.535+0000 7f198ffff640 1 --2- 172.21.15.172:0/1698250972 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f19840a2c90 0x7f19840a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f19980fee90 tx=0x7f1980006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:55.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.539+0000 7f19957fa640 1 -- 172.21.15.172:0/1698250972 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19880be170 con 0x7f1998100a80 2024-09-10T08:45:55.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.707+0000 7f199dd6c640 1 -- 172.21.15.172:0/1698250972 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f199806b270 con 0x7f1998100a80 2024-09-10T08:45:55.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.708+0000 7f19957fa640 1 -- 172.21.15.172:0/1698250972 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24732 (secure 0 0 0) 0x7f19880bda70 con 0x7f1998100a80 2024-09-10T08:45:55.709 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:45:55.709 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":90,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","created":"2024-09-10T08:36:33.083529+0000","modified":"2024-09-10T08:45:18.243953+0000","last_up_change":"2024-09-10T08:43:19.615640+0000","last_in_change":"2024-09-10T08:42:56.656115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:39:31.328306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:44:07.466637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:44:09.520569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:44:15.926057+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6803","nonce":2678426952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6805","nonce":2678426952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6809","nonce":2678426952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6807","nonce":2678426952}]},"public_addr":"172.21.15.172:6803/2678426952","cluster_addr":"172.21.15.172:6805/2678426952","heartbeat_back_addr":"172.21.15.172:6809/2678426952","heartbeat_front_addr":"172.21.15.172:6807/2678426952","state":["exists","up"]},{"osd":1,"uuid":"55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6801","nonce":3279661439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6803","nonce":3279661439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6807","nonce":3279661439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6805","nonce":3279661439}]},"public_addr":"172.21.15.174:6801/3279661439","cluster_addr":"172.21.15.174:6803/3279661439","heartbeat_back_addr":"172.21.15.174:6807/3279661439","heartbeat_front_addr":"172.21.15.174:6805/3279661439","state":["exists","up"]},{"osd":2,"uuid":"f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6801","nonce":1963796290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6803","nonce":1963796290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6807","nonce":1963796290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6805","nonce":1963796290}]},"public_addr":"172.21.15.191:6801/1963796290","cluster_addr":"172.21.15.191:6803/1963796290","heartbeat_back_addr":"172.21.15.191:6807/1963796290","heartbeat_front_addr":"172.21.15.191:6805/1963796290","state":["exists","up"]},{"osd":3,"uuid":"15ecca3f-bc6b-4a06-b283-b515384beb19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6811","nonce":869546480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6813","nonce":869546480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6817","nonce":869546480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6815","nonce":869546480}]},"public_addr":"172.21.15.172:6811/869546480","cluster_addr":"172.21.15.172:6813/869546480","heartbeat_back_addr":"172.21.15.172:6817/869546480","heartbeat_front_addr":"172.21.15.172:6815/869546480","state":["exists","up"]},{"osd":4,"uuid":"8c2c1d40-72ad-4a15-ab72-54871c7e2427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6809","nonce":961007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6811","nonce":961007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6815","nonce":961007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6813","nonce":961007949}]},"public_addr":"172.21.15.174:6809/961007949","cluster_addr":"172.21.15.174:6811/961007949","heartbeat_back_addr":"172.21.15.174:6815/961007949","heartbeat_front_addr":"172.21.15.174:6813/961007949","state":["exists","up"]},{"osd":5,"uuid":"4de45a1d-ef0e-4bfe-ba15-5eeeb8806155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6809","nonce":4216938371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6811","nonce":4216938371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6815","nonce":4216938371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6813","nonce":4216938371}]},"public_addr":"172.21.15.191:6809/4216938371","cluster_addr":"172.21.15.191:6811/4216938371","heartbeat_back_addr":"172.21.15.191:6815/4216938371","heartbeat_front_addr":"172.21.15.191:6813/4216938371","state":["exists","up"]},{"osd":6,"uuid":"60a9938a-f696-4db2-932c-402997543b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6819","nonce":3210762009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6821","nonce":3210762009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6825","nonce":3210762009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6823","nonce":3210762009}]},"public_addr":"172.21.15.172:6819/3210762009","cluster_addr":"172.21.15.172:6821/3210762009","heartbeat_back_addr":"172.21.15.172:6825/3210762009","heartbeat_front_addr":"172.21.15.172:6823/3210762009","state":["exists","up"]},{"osd":7,"uuid":"7f0202f3-971e-472f-9c4d-ae3c24d39f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6817","nonce":1461923581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6819","nonce":1461923581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6823","nonce":1461923581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6821","nonce":1461923581}]},"public_addr":"172.21.15.174:6817/1461923581","cluster_addr":"172.21.15.174:6819/1461923581","heartbeat_back_addr":"172.21.15.174:6823/1461923581","heartbeat_front_addr":"172.21.15.174:6821/1461923581","state":["exists","up"]},{"osd":8,"uuid":"b4e7327f-ca9f-4833-9cd3-b8529b3d9761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6817","nonce":3675210430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6819","nonce":3675210430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6823","nonce":3675210430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6821","nonce":3675210430}]},"public_addr":"172.21.15.191:6817/3675210430","cluster_addr":"172.21.15.191:6819/3675210430","heartbeat_back_addr":"172.21.15.191:6823/3675210430","heartbeat_front_addr":"172.21.15.191:6821/3675210430","state":["exists","up"]},{"osd":9,"uuid":"9f6153f2-f9db-4c00-a659-c932628016f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6827","nonce":1181268619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6829","nonce":1181268619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6833","nonce":1181268619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6831","nonce":1181268619}]},"public_addr":"172.21.15.172:6827/1181268619","cluster_addr":"172.21.15.172:6829/1181268619","heartbeat_back_addr":"172.21.15.172:6833/1181268619","heartbeat_front_addr":"172.21.15.172:6831/1181268619","state":["exists","up"]},{"osd":10,"uuid":"040a013a-4c3e-442f-bea6-12cbe517716d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6825","nonce":1238812525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6827","nonce":1238812525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6831","nonce":1238812525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6829","nonce":1238812525}]},"public_addr":"172.21.15.174:6825/1238812525","cluster_addr":"172.21.15.174:6827/1238812525","heartbeat_back_addr":"172.21.15.174:6831/1238812525","heartbeat_front_addr":"172.21.15.174:6829/1238812525","state":["exists","up"]},{"osd":11,"uuid":"ae6535bd-1510-4bf3-bf4b-1a2a256add43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6825","nonce":2162719997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6827","nonce":2162719997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6831","nonce":2162719997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6829","nonce":2162719997}]},"public_addr":"172.21.15.191:6825/2162719997","cluster_addr":"172.21.15.191:6827/2162719997","heartbeat_back_addr":"172.21.15.191:6831/2162719997","heartbeat_front_addr":"172.21.15.191:6829/2162719997","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:38:38.812596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:04.765652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:29.103966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:56.085849+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:21.641784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:47.750376+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:11.640697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:36.638840+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:02.167446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:28.655804+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:52.749676+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:43:17.783908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:0/977245917":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/3919952878":"2024-09-11T08:36:54.887256+0000","172.21.15.172:6800/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/1525385361":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/2043423408":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/839344299":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6800/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/3251817286":"2024-09-11T08:36:54.887256+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:45:55.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.711+0000 7f199dd6c640 1 -- 172.21.15.172:0/1698250972 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f19840a2c90 msgr2=0x7f19840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:55.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.711+0000 7f199dd6c640 1 --2- 172.21.15.172:0/1698250972 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f19840a2c90 0x7f19840a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f19980fee90 tx=0x7f1980006c90 comp rx=0 tx=0).stop 2024-09-10T08:45:55.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.711+0000 7f199dd6c640 1 -- 172.21.15.172:0/1698250972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1998100a80 msgr2=0x7f1998197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:55.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.711+0000 7f199dd6c640 1 --2- 172.21.15.172:0/1698250972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1998100a80 0x7f1998197350 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1988004770 tx=0x7f1988004270 comp rx=0 tx=0).stop 2024-09-10T08:45:55.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.711+0000 7f199dd6c640 1 -- 172.21.15.172:0/1698250972 shutdown_connections 2024-09-10T08:45:55.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.711+0000 7f199dd6c640 1 --2- 172.21.15.172:0/1698250972 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f19840a2c90 0x7f19840a5150 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:55.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.711+0000 7f199dd6c640 1 --2- 172.21.15.172:0/1698250972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1998106a80 0x7f199819bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:55.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.711+0000 7f199dd6c640 1 --2- 172.21.15.172:0/1698250972 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1998101420 0x7f1998197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:55.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.711+0000 7f199dd6c640 1 --2- 172.21.15.172:0/1698250972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1998100a80 0x7f1998197350 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:55.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.711+0000 7f199dd6c640 1 -- 172.21.15.172:0/1698250972 >> 172.21.15.172:0/1698250972 conn(0x7f19980fc7a0 msgr2=0x7f19980fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:55.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.711+0000 7f199dd6c640 1 -- 172.21.15.172:0/1698250972 shutdown_connections 2024-09-10T08:45:55.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:55.711+0000 7f199dd6c640 1 -- 172.21.15.172:0/1698250972 wait complete. 2024-09-10T08:45:56.451 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:45:56.474 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:56 smithi172 ceph-mon[30854]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:56.474 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:56 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1698250972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:45:56.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:56 smithi174 ceph-mon[34068]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:56.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:56 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1698250972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:45:56.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:56 smithi191 ceph-mon[35314]: pgmap v339: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:45:56.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:56 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1698250972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:45:56.699 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:57.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.214+0000 7fa09d7f2640 1 -- 172.21.15.172:0/3325421922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098101de0 msgr2=0x7fa0981021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.214+0000 7fa09d7f2640 1 --2- 172.21.15.172:0/3325421922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098101de0 0x7fa0981021c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa088009900 tx=0x7fa08802a410 comp rx=0 tx=0).stop 2024-09-10T08:45:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.214+0000 7fa09d7f2640 1 -- 172.21.15.172:0/3325421922 shutdown_connections 2024-09-10T08:45:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.214+0000 7fa09d7f2640 1 --2- 172.21.15.172:0/3325421922 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa098106f10 0x7fa09810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.214+0000 7fa09d7f2640 1 --2- 172.21.15.172:0/3325421922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa098102790 0x7fa0981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.214+0000 7fa09d7f2640 1 --2- 172.21.15.172:0/3325421922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098101de0 0x7fa0981021c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.214+0000 7fa09d7f2640 1 -- 172.21.15.172:0/3325421922 >> 172.21.15.172:0/3325421922 conn(0x7fa0980fd8c0 msgr2=0x7fa0980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.214+0000 7fa09d7f2640 1 -- 172.21.15.172:0/3325421922 shutdown_connections 2024-09-10T08:45:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.214+0000 7fa09d7f2640 1 -- 172.21.15.172:0/3325421922 wait complete. 2024-09-10T08:45:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.215+0000 7fa09d7f2640 1 Processor -- start 2024-09-10T08:45:57.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.215+0000 7fa09d7f2640 1 -- start start 2024-09-10T08:45:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.215+0000 7fa09d7f2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa098101de0 0x7fa0981984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.215+0000 7fa09d7f2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa098102790 0x7fa0981989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.215+0000 7fa09d7f2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098106f10 0x7fa09819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.215+0000 7fa09d7f2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa098111e40 con 0x7fa098106f10 2024-09-10T08:45:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.215+0000 7fa09d7f2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa098111cc0 con 0x7fa098101de0 2024-09-10T08:45:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.215+0000 7fa09d7f2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa098111fc0 con 0x7fa098102790 2024-09-10T08:45:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.215+0000 7fa0977fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098106f10 0x7fa09819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.215+0000 7fa0977fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098106f10 0x7fa09819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56896/0 (socket says 172.21.15.172:56896) 2024-09-10T08:45:57.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.215+0000 7fa0977fe640 1 -- 172.21.15.172:0/3832861820 learned_addr learned my addr 172.21.15.172:0/3832861820 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:57.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.215+0000 7fa096ffd640 1 --2- 172.21.15.172:0/3832861820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa098101de0 0x7fa0981984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:57.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.215+0000 7fa0967fc640 1 --2- 172.21.15.172:0/3832861820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa098102790 0x7fa0981989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:57.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.216+0000 7fa0977fe640 1 -- 172.21.15.172:0/3832861820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa098102790 msgr2=0x7fa0981989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:57.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.216+0000 7fa0977fe640 1 --2- 172.21.15.172:0/3832861820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa098102790 0x7fa0981989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:57.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.216+0000 7fa0977fe640 1 -- 172.21.15.172:0/3832861820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa098101de0 msgr2=0x7fa0981984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:57.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.216+0000 7fa0977fe640 1 --2- 172.21.15.172:0/3832861820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa098101de0 0x7fa0981984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:57.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.216+0000 7fa0977fe640 1 -- 172.21.15.172:0/3832861820 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09819d460 con 0x7fa098106f10 2024-09-10T08:45:57.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.216+0000 7fa0967fc640 1 --2- 172.21.15.172:0/3832861820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa098102790 0x7fa0981989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:57.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.216+0000 7fa0977fe640 1 --2- 172.21.15.172:0/3832861820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098106f10 0x7fa09819cd80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa09000b970 tx=0x7fa09000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:57.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.216+0000 7fa08ffff640 1 -- 172.21.15.172:0/3832861820 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa090019070 con 0x7fa098106f10 2024-09-10T08:45:57.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.216+0000 7fa09d7f2640 1 -- 172.21.15.172:0/3832861820 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa09819d750 con 0x7fa098106f10 2024-09-10T08:45:57.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.216+0000 7fa08ffff640 1 -- 172.21.15.172:0/3832861820 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa090014440 con 0x7fa098106f10 2024-09-10T08:45:57.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.216+0000 7fa08ffff640 1 -- 172.21.15.172:0/3832861820 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0900075d0 con 0x7fa098106f10 2024-09-10T08:45:57.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.216+0000 7fa09d7f2640 1 -- 172.21.15.172:0/3832861820 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0981a5030 con 0x7fa098106f10 2024-09-10T08:45:57.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.217+0000 7fa09d7f2640 1 -- 172.21.15.172:0/3832861820 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0981021c0 con 0x7fa098106f10 2024-09-10T08:45:57.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.218+0000 7fa08ffff640 1 -- 172.21.15.172:0/3832861820 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa09001c070 con 0x7fa098106f10 2024-09-10T08:45:57.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.219+0000 7fa08ffff640 1 --2- 172.21.15.172:0/3832861820 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0840a2c10 0x7fa0840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:57.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.219+0000 7fa096ffd640 1 --2- 172.21.15.172:0/3832861820 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0840a2c10 0x7fa0840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:57.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.219+0000 7fa08ffff640 1 -- 172.21.15.172:0/3832861820 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fa0900cee20 con 0x7fa098106f10 2024-09-10T08:45:57.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.219+0000 7fa096ffd640 1 --2- 172.21.15.172:0/3832861820 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0840a2c10 0x7fa0840a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa088031440 tx=0x7fa088005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:57.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.222+0000 7fa08ffff640 1 -- 172.21.15.172:0/3832861820 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa09009b3a0 con 0x7fa098106f10 2024-09-10T08:45:57.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.423+0000 7fa09d7f2640 1 -- 172.21.15.172:0/3832861820 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa098199810 con 0x7fa098106f10 2024-09-10T08:45:57.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.424+0000 7fa08ffff640 1 -- 172.21.15.172:0/3832861820 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+12001 (secure 0 0 0) 0x7fa09009f6a0 con 0x7fa098106f10 2024-09-10T08:45:57.425 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:45:57.425 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":28,"btime":"2024-09-10T08:44:55:226722+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:44:55.226705+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T08:45:57.425 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 28 2024-09-10T08:45:57.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.426+0000 7fa09d7f2640 1 -- 172.21.15.172:0/3832861820 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0840a2c10 msgr2=0x7fa0840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:57.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.426+0000 7fa09d7f2640 1 --2- 172.21.15.172:0/3832861820 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0840a2c10 0x7fa0840a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa088031440 tx=0x7fa088005b50 comp rx=0 tx=0).stop 2024-09-10T08:45:57.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.426+0000 7fa09d7f2640 1 -- 172.21.15.172:0/3832861820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098106f10 msgr2=0x7fa09819cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:57.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.426+0000 7fa09d7f2640 1 --2- 172.21.15.172:0/3832861820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098106f10 0x7fa09819cd80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa09000b970 tx=0x7fa09000be30 comp rx=0 tx=0).stop 2024-09-10T08:45:57.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.426+0000 7fa09d7f2640 1 -- 172.21.15.172:0/3832861820 shutdown_connections 2024-09-10T08:45:57.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.426+0000 7fa09d7f2640 1 --2- 172.21.15.172:0/3832861820 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0840a2c10 0x7fa0840a50d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:57.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.426+0000 7fa09d7f2640 1 --2- 172.21.15.172:0/3832861820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa098106f10 0x7fa09819cd80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:57.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.426+0000 7fa09d7f2640 1 --2- 172.21.15.172:0/3832861820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa098102790 0x7fa0981989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:57.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.426+0000 7fa09d7f2640 1 --2- 172.21.15.172:0/3832861820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa098101de0 0x7fa0981984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:57.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.426+0000 7fa09d7f2640 1 -- 172.21.15.172:0/3832861820 >> 172.21.15.172:0/3832861820 conn(0x7fa0980fd8c0 msgr2=0x7fa0980feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:57.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.426+0000 7fa09d7f2640 1 -- 172.21.15.172:0/3832861820 shutdown_connections 2024-09-10T08:45:57.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:57.426+0000 7fa09d7f2640 1 -- 172.21.15.172:0/3832861820 wait complete. 2024-09-10T08:45:58.214 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T08:44:11.310930+0000', 'modified': '2024-09-10T08:44:55.226705+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24488, 'mds_1': 24458, 'mds_2': 24457, 'mds_3': 24416, 'mds_4': 24406}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6839/1621056220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6838', 'nonce': 1621056220}, {'type': 'v1', 'addr': '172.21.15.172:6839', 'nonce': 1621056220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14622': {'gid': 14622, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6841/176040463', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6840', 'nonce': 176040463}, {'type': 'v1', 'addr': '172.21.15.172:6841', 'nonce': 176040463}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24406': {'gid': 24406, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.191:6833/1662718523', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6832', 'nonce': 1662718523}, {'type': 'v1', 'addr': '172.21.15.191:6833', 'nonce': 1662718523}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.174:6833/3418766024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 3418766024}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 3418766024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6835/1774553543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6834', 'nonce': 1774553543}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 1774553543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/3197756201', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 3197756201}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 3197756201}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6837/1802631256', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6836', 'nonce': 1802631256}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 1802631256}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.191:6839/1702936476', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6838', 'nonce': 1702936476}, {'type': 'v1', 'addr': '172.21.15.191:6839', 'nonce': 1702936476}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.174:6837/2399337052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 2399337052}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 2399337052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.174:6839/3029849701', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 3029849701}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 3029849701}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24488, 24457, 24416, 24458, 24406]} 2024-09-10T08:45:58.214 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T08:45:58.214 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.l status 2024-09-10T08:45:58.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:58 smithi174 ceph-mon[34068]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T08:45:58.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:45:58 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3832861820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:58.503 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:45:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:58 smithi191 ceph-mon[35314]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T08:45:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:45:58 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3832861820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:58 smithi172 ceph-mon[30854]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T08:45:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:45:58 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3832861820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:45:58.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.976+0000 7f81f498f640 1 -- 172.21.15.172:0/1237133135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f0100480 msgr2=0x7f81f0100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:58.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.976+0000 7f81f498f640 1 --2- 172.21.15.172:0/1237133135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f0100480 0x7f81f0100860 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009900 tx=0x7f81e002a410 comp rx=0 tx=0).stop 2024-09-10T08:45:58.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.976+0000 7f81f498f640 1 -- 172.21.15.172:0/1237133135 shutdown_connections 2024-09-10T08:45:58.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.976+0000 7f81f498f640 1 --2- 172.21.15.172:0/1237133135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81f01055b0 0x7f81f010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.976+0000 7f81f498f640 1 --2- 172.21.15.172:0/1237133135 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81f0100e30 0x7f81f0104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.976+0000 7f81f498f640 1 --2- 172.21.15.172:0/1237133135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f0100480 0x7f81f0100860 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.976+0000 7f81f498f640 1 -- 172.21.15.172:0/1237133135 >> 172.21.15.172:0/1237133135 conn(0x7f81f0077210 msgr2=0x7f81f00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:58.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.976+0000 7f81f498f640 1 -- 172.21.15.172:0/1237133135 shutdown_connections 2024-09-10T08:45:58.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.976+0000 7f81f498f640 1 -- 172.21.15.172:0/1237133135 wait complete. 2024-09-10T08:45:58.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.977+0000 7f81f498f640 1 Processor -- start 2024-09-10T08:45:58.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.977+0000 7f81f498f640 1 -- start start 2024-09-10T08:45:58.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.977+0000 7f81f498f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81f0100480 0x7f81f01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.977+0000 7f81f498f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81f0100e30 0x7f81f01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.977+0000 7f81f498f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f01055b0 0x7f81f019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.977+0000 7f81f498f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81f0110db0 con 0x7f81f01055b0 2024-09-10T08:45:58.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.977+0000 7f81f498f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81f0110c30 con 0x7f81f0100e30 2024-09-10T08:45:58.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.977+0000 7f81f498f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f81f0110f30 con 0x7f81f0100480 2024-09-10T08:45:58.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.977+0000 7f81eed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f01055b0 0x7f81f019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.978+0000 7f81eed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f01055b0 0x7f81f019cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56918/0 (socket says 172.21.15.172:56918) 2024-09-10T08:45:58.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.978+0000 7f81eed9d640 1 -- 172.21.15.172:0/530336871 learned_addr learned my addr 172.21.15.172:0/530336871 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:58.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.978+0000 7f81ee59c640 1 --2- 172.21.15.172:0/530336871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81f0100480 0x7f81f01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.978+0000 7f81edd9b640 1 --2- 172.21.15.172:0/530336871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81f0100e30 0x7f81f01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.978+0000 7f81eed9d640 1 -- 172.21.15.172:0/530336871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81f0100480 msgr2=0x7f81f01983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:58.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.978+0000 7f81eed9d640 1 --2- 172.21.15.172:0/530336871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81f0100480 0x7f81f01983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.978+0000 7f81eed9d640 1 -- 172.21.15.172:0/530336871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81f0100e30 msgr2=0x7f81f01988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:58.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.978+0000 7f81eed9d640 1 --2- 172.21.15.172:0/530336871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81f0100e30 0x7f81f01988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.978+0000 7f81eed9d640 1 -- 172.21.15.172:0/530336871 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f019d360 con 0x7f81f01055b0 2024-09-10T08:45:58.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.978+0000 7f81ee59c640 1 --2- 172.21.15.172:0/530336871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81f0100480 0x7f81f01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:45:58.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.978+0000 7f81eed9d640 1 --2- 172.21.15.172:0/530336871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f01055b0 0x7f81f019cc80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f81e800eaa0 tx=0x7f81e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:58.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.978+0000 7f81df7fe640 1 -- 172.21.15.172:0/530336871 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e800cdb0 con 0x7f81f01055b0 2024-09-10T08:45:58.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.978+0000 7f81f498f640 1 -- 172.21.15.172:0/530336871 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f019d650 con 0x7f81f01055b0 2024-09-10T08:45:58.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.978+0000 7f81df7fe640 1 -- 172.21.15.172:0/530336871 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81e8005020 con 0x7f81f01055b0 2024-09-10T08:45:58.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.978+0000 7f81df7fe640 1 -- 172.21.15.172:0/530336871 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e80116d0 con 0x7f81f01055b0 2024-09-10T08:45:58.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.978+0000 7f81f498f640 1 -- 172.21.15.172:0/530336871 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f01a4f30 con 0x7f81f01055b0 2024-09-10T08:45:58.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.980+0000 7f81df7fe640 1 -- 172.21.15.172:0/530336871 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f81e80051d0 con 0x7f81f01055b0 2024-09-10T08:45:58.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.981+0000 7f81f6d68740 1 Processor -- start 2024-09-10T08:45:58.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.981+0000 7f81f6d68740 1 -- start start 2024-09-10T08:45:58.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.981+0000 7f81f6d68740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dadf9d0 0x55679dadfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.981+0000 7f81f6d68740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dae02f0 0x55679daea670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.981+0000 7f81f6d68740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679daeabb0 0x55679daecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.981+0000 7f81f6d68740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55679da46090 con 0x55679dae02f0 2024-09-10T08:45:58.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.981+0000 7f81f6d68740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55679d9eda70 con 0x55679dadf9d0 2024-09-10T08:45:58.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.981+0000 7f81f6d68740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55679da42050 con 0x55679daeabb0 2024-09-10T08:45:58.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.981+0000 7f81df7fe640 1 --2- 172.21.15.172:0/530336871 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f81d80a2c90 0x7f81d80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.982+0000 7f81ee59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dae02f0 0x55679daea670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.982+0000 7f81ee59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dae02f0 0x55679daea670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56924/0 (socket says 172.21.15.172:56924) 2024-09-10T08:45:58.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.982+0000 7f81ee59c640 1 -- 172.21.15.172:0/3479240642 learned_addr learned my addr 172.21.15.172:0/3479240642 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:58.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.982+0000 7f81edd9b640 1 --2- 172.21.15.172:0/3479240642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dadf9d0 0x55679dadfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.982+0000 7f81eed9d640 1 --2- 172.21.15.172:0/530336871 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f81d80a2c90 0x7f81d80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.982+0000 7f81edd9b640 1 --2- 172.21.15.172:0/3479240642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679daeabb0 0x55679daecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.982+0000 7f81df7fe640 1 -- 172.21.15.172:0/530336871 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f81e80cf7c0 con 0x7f81f01055b0 2024-09-10T08:45:58.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.982+0000 7f81ee59c640 1 -- 172.21.15.172:0/3479240642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679daeabb0 msgr2=0x55679daecfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:58.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.982+0000 7f81ee59c640 1 --2- 172.21.15.172:0/3479240642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679daeabb0 0x55679daecfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.982+0000 7f81ee59c640 1 -- 172.21.15.172:0/3479240642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dadf9d0 msgr2=0x55679dadfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:58.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.982+0000 7f81ee59c640 1 --2- 172.21.15.172:0/3479240642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dadf9d0 0x55679dadfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.982+0000 7f81ee59c640 1 -- 172.21.15.172:0/3479240642 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55679d9fa130 con 0x55679dae02f0 2024-09-10T08:45:58.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.982+0000 7f81ee59c640 1 --2- 172.21.15.172:0/3479240642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dae02f0 0x55679daea670 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009900 tx=0x7f81e00042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9cb09a89ad7d81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:58.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.982+0000 7f81eed9d640 1 --2- 172.21.15.172:0/530336871 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f81d80a2c90 0x7f81d80a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f81e80cce00 tx=0x7f81e8091000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:58.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.982+0000 7f81dcff9640 1 -- 172.21.15.172:0/3479240642 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e0035030 con 0x55679dae02f0 2024-09-10T08:45:58.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.982+0000 7f81dcff9640 1 -- 172.21.15.172:0/3479240642 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81e00054f0 con 0x55679dae02f0 2024-09-10T08:45:58.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.982+0000 7f81dcff9640 1 -- 172.21.15.172:0/3479240642 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e00037f0 con 0x55679dae02f0 2024-09-10T08:45:58.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.983+0000 7f81f6d68740 1 -- 172.21.15.172:0/3479240642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dae02f0 msgr2=0x55679daea670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:58.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.983+0000 7f81f6d68740 1 --2- 172.21.15.172:0/3479240642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dae02f0 0x55679daea670 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009900 tx=0x7f81e00042e0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.983+0000 7f81f6d68740 1 -- 172.21.15.172:0/3479240642 shutdown_connections 2024-09-10T08:45:58.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.983+0000 7f81f6d68740 1 --2- 172.21.15.172:0/3479240642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679daeabb0 0x55679daecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.983+0000 7f81f6d68740 1 --2- 172.21.15.172:0/3479240642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dae02f0 0x55679daea670 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.983+0000 7f81f6d68740 1 --2- 172.21.15.172:0/3479240642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dadf9d0 0x55679dadfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.983+0000 7f81f6d68740 1 -- 172.21.15.172:0/3479240642 >> 172.21.15.172:0/3479240642 conn(0x55679dadbf60 msgr2=0x55679dadc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:58.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.983+0000 7f81f6d68740 1 -- 172.21.15.172:0/3479240642 shutdown_connections 2024-09-10T08:45:58.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.983+0000 7f81f6d68740 1 -- 172.21.15.172:0/3479240642 wait complete. 2024-09-10T08:45:58.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.983+0000 7f81f6d68740 1 -- start start 2024-09-10T08:45:58.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81f6d68740 1 Processor -- start 2024-09-10T08:45:58.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81f6d68740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dadf9d0 0x55679daea470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81f6d68740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dae02f0 0x55679dae77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81f6d68740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679daeabb0 0x55679dae7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81f6d68740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55679daed780 con 0x55679dadf9d0 2024-09-10T08:45:58.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81f6d68740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55679da00f20 con 0x55679dae02f0 2024-09-10T08:45:58.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81f6d68740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55679daed900 con 0x55679daeabb0 2024-09-10T08:45:58.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81f6d68740 10 client.0 _ll_register_callbacks cb 0x55679d9ca030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:58.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81edd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dadf9d0 0x55679daea470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81edd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dadf9d0 0x55679daea470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56932/0 (socket says 172.21.15.172:56932) 2024-09-10T08:45:58.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81edd9b640 1 -- 172.21.15.172:0/2154413589 learned_addr learned my addr 172.21.15.172:0/2154413589 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:58.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81ee59c640 1 --2- 172.21.15.172:0/2154413589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dae02f0 0x55679dae77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:58.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81edd9b640 1 -- 172.21.15.172:0/2154413589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679daeabb0 msgr2=0x55679dae7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:58.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81edd9b640 1 --2- 172.21.15.172:0/2154413589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679daeabb0 0x55679dae7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81edd9b640 1 -- 172.21.15.172:0/2154413589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dae02f0 msgr2=0x55679dae77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:58.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81edd9b640 1 --2- 172.21.15.172:0/2154413589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dae02f0 0x55679dae77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:58.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81edd9b640 1 -- 172.21.15.172:0/2154413589 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55679d7b72b0 con 0x55679dadf9d0 2024-09-10T08:45:58.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81edd9b640 1 --2- 172.21.15.172:0/2154413589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dadf9d0 0x55679daea470 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f81e400ca70 tx=0x7f81e40146c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a69721a97e136bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:58.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.984+0000 7f81d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:45:58.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.985+0000 7f81d5ffb640 1 -- 172.21.15.172:0/2154413589 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e400f040 con 0x55679dadf9d0 2024-09-10T08:45:58.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.985+0000 7f81d5ffb640 1 -- 172.21.15.172:0/2154413589 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f81e4015d30 con 0x55679dadf9d0 2024-09-10T08:45:58.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.985+0000 7f81f6d68740 1 -- 172.21.15.172:0/2154413589 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81e00054f0 con 0x55679dadf9d0 2024-09-10T08:45:58.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.985+0000 7f81d5ffb640 1 -- 172.21.15.172:0/2154413589 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81e4024050 con 0x55679dadf9d0 2024-09-10T08:45:58.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.985+0000 7f81f6d68740 10 client.15075 fetch_fsmap learned FSMap version 28 2024-09-10T08:45:58.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.985+0000 7f81f6d68740 1 -- 172.21.15.172:0/2154413589 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55679daf9120 con 0x55679dadf9d0 2024-09-10T08:45:58.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.985+0000 7f81d5ffb640 1 -- 172.21.15.172:0/2154413589 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81e00054f0 con 0x55679dadf9d0 2024-09-10T08:45:58.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.985+0000 7f81d5ffb640 1 -- 172.21.15.172:0/2154413589 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f81e4015040 con 0x55679dadf9d0 2024-09-10T08:45:58.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.985+0000 7f81f6d68740 10 client.15075 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:45:58.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.985+0000 7f81f6d68740 10 client.15075 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-10T08:45:58.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.985+0000 7f81f6d68740 20 client.15075 populate_metadata read hostname 'smithi172' 2024-09-10T08:45:58.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.985+0000 7f81f6d68740 1 --2- 172.21.15.172:0/2154413589 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55679dafb750 0x55679dafcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:58.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.985+0000 7f81f6d68740 4 client.15075 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:45:58.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.985+0000 7f81f6d68740 1 -- 172.21.15.172:0/2154413589 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55679da53600 con 0x55679dafb750 2024-09-10T08:45:58.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.985+0000 7f81ee59c640 1 --2- 172.21.15.172:0/2154413589 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55679dafb750 0x55679dafcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:59.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.986+0000 7f81ee59c640 1 --2- 172.21.15.172:0/2154413589 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55679dafb750 0x55679dafcbc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ca5b129e18dd9bba server_cookie=99a90a1deb10055 in_seq=0 out_seq=0 2024-09-10T08:45:59.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.986+0000 7f81d5ffb640 10 client.15075 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:45:59.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.987+0000 7f81d5ffb640 1 -- 172.21.15.172:0/2154413589 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55679da53600 con 0x55679dafb750 2024-09-10T08:45:59.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.987+0000 7f81d5ffb640 10 client.15075 handle_command_reply: tid=0 2024-09-10T08:45:59.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.987+0000 7f81f6d68740 1 client.15075 shutdown 2024-09-10T08:45:59.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.987+0000 7f81f6d68740 1 -- 172.21.15.172:0/2154413589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dadf9d0 msgr2=0x55679daea470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:59.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.987+0000 7f81f6d68740 1 --2- 172.21.15.172:0/2154413589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dadf9d0 0x55679daea470 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f81e400ca70 tx=0x7f81e40146c0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.987+0000 7f81f6d68740 1 -- 172.21.15.172:0/2154413589 shutdown_connections 2024-09-10T08:45:59.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.987+0000 7f81f6d68740 1 --2- 172.21.15.172:0/2154413589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679daeabb0 0x55679dae7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.987+0000 7f81f6d68740 1 --2- 172.21.15.172:0/2154413589 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55679dafb750 0x55679dafcbc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.987+0000 7f81f6d68740 1 --2- 172.21.15.172:0/2154413589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dae02f0 0x55679dae77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.987+0000 7f81f6d68740 1 --2- 172.21.15.172:0/2154413589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dadf9d0 0x55679daea470 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.987+0000 7f81f6d68740 1 -- 172.21.15.172:0/2154413589 >> 172.21.15.172:0/2154413589 conn(0x55679dadbf60 msgr2=0x55679dae1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:59.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.987+0000 7f81d5ffb640 0 client.15075 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:45:59.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.987+0000 7f81f6d68740 1 -- 172.21.15.172:0/2154413589 shutdown_connections 2024-09-10T08:45:59.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.987+0000 7f81f6d68740 1 -- 172.21.15.172:0/2154413589 wait complete. 2024-09-10T08:45:59.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:58.987+0000 7f81f6d68740 20 client.15075 trim_cache size 0 max 16384 2024-09-10T08:45:59.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.004+0000 7f81f6d68740 1 Processor -- start 2024-09-10T08:45:59.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.004+0000 7f81f6d68740 1 -- start start 2024-09-10T08:45:59.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81f6d68740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dadf9d0 0x55679dafb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:59.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81f6d68740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dae02f0 0x55679dafc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:59.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81f6d68740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679daeabb0 0x55679db04ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:59.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81f6d68740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55679dafaf00 con 0x55679dae02f0 2024-09-10T08:45:59.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81f6d68740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55679dafb5d0 con 0x55679dadf9d0 2024-09-10T08:45:59.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81f6d68740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55679dafbca0 con 0x55679daeabb0 2024-09-10T08:45:59.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81ee59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dae02f0 0x55679dafc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:59.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81ee59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dae02f0 0x55679dafc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56936/0 (socket says 172.21.15.172:56936) 2024-09-10T08:45:59.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81ee59c640 1 -- 172.21.15.172:0/3176618777 learned_addr learned my addr 172.21.15.172:0/3176618777 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:59.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81edd9b640 1 --2- 172.21.15.172:0/3176618777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679daeabb0 0x55679db04ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:59.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81ee59c640 1 -- 172.21.15.172:0/3176618777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679daeabb0 msgr2=0x55679db04ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:59.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81ee59c640 1 --2- 172.21.15.172:0/3176618777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679daeabb0 0x55679db04ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81ee59c640 1 -- 172.21.15.172:0/3176618777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dadf9d0 msgr2=0x55679dafb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:59.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81ee59c640 1 --2- 172.21.15.172:0/3176618777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dadf9d0 0x55679dafb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81ee59c640 1 -- 172.21.15.172:0/3176618777 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55679daea7e0 con 0x55679dae02f0 2024-09-10T08:45:59.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81edd9b640 1 --2- 172.21.15.172:0/3176618777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679daeabb0 0x55679db04ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:45:59.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81ee59c640 1 --2- 172.21.15.172:0/3176618777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dae02f0 0x55679dafc4f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55679dadcdc0 tx=0x7f81e0004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95261f8b9f2be55b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:59.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81d7fff640 1 -- 172.21.15.172:0/3176618777 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e0053070 con 0x55679dae02f0 2024-09-10T08:45:59.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81d7fff640 1 -- 172.21.15.172:0/3176618777 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81e0031b50 con 0x55679dae02f0 2024-09-10T08:45:59.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.005+0000 7f81d7fff640 1 -- 172.21.15.172:0/3176618777 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e0035040 con 0x55679dae02f0 2024-09-10T08:45:59.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.006+0000 7f81f6d68740 1 -- 172.21.15.172:0/3176618777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dae02f0 msgr2=0x55679dafc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:59.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.006+0000 7f81f6d68740 1 --2- 172.21.15.172:0/3176618777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dae02f0 0x55679dafc4f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55679dadcdc0 tx=0x7f81e0004970 comp rx=0 tx=0).stop 2024-09-10T08:45:59.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.006+0000 7f81f6d68740 1 -- 172.21.15.172:0/3176618777 shutdown_connections 2024-09-10T08:45:59.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.006+0000 7f81f6d68740 1 --2- 172.21.15.172:0/3176618777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679daeabb0 0x55679db04ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.006+0000 7f81f6d68740 1 --2- 172.21.15.172:0/3176618777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679dae02f0 0x55679dafc4f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.006+0000 7f81f6d68740 1 --2- 172.21.15.172:0/3176618777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dadf9d0 0x55679dafb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.006+0000 7f81f6d68740 1 -- 172.21.15.172:0/3176618777 >> 172.21.15.172:0/3176618777 conn(0x55679dadbf60 msgr2=0x55679dae6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:59.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.006+0000 7f81f6d68740 1 -- 172.21.15.172:0/3176618777 shutdown_connections 2024-09-10T08:45:59.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.006+0000 7f81f6d68740 1 -- 172.21.15.172:0/3176618777 wait complete. 2024-09-10T08:45:59.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.006+0000 7f81f6d68740 1 -- start start 2024-09-10T08:45:59.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.006+0000 7f81f6d68740 1 Processor -- start 2024-09-10T08:45:59.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81f6d68740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679dadf9d0 0x55679db056b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:59.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81f6d68740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dae02f0 0x55679daff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:59.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81f6d68740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679daeabb0 0x55679db00030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:59.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81f6d68740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55679dafca40 con 0x55679daeabb0 2024-09-10T08:45:59.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81f6d68740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55679dafc370 con 0x55679dae02f0 2024-09-10T08:45:59.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81f6d68740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55679dafd110 con 0x55679dadf9d0 2024-09-10T08:45:59.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81f6d68740 10 client.0 _ll_register_callbacks cb 0x7f81e400c4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:45:59.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81edd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679daeabb0 0x55679db00030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:59.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81edd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679daeabb0 0x55679db00030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56952/0 (socket says 172.21.15.172:56952) 2024-09-10T08:45:59.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81edd9b640 1 -- 172.21.15.172:0/3158908571 learned_addr learned my addr 172.21.15.172:0/3158908571 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:45:59.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81ee59c640 1 --2- 172.21.15.172:0/3158908571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dae02f0 0x55679daff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:59.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81edd9b640 1 --2- 172.21.15.172:0/3158908571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679dadf9d0 0x55679db056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:59.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81edd9b640 1 -- 172.21.15.172:0/3158908571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679dadf9d0 msgr2=0x55679db056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:59.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81edd9b640 1 --2- 172.21.15.172:0/3158908571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679dadf9d0 0x55679db056b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81edd9b640 1 -- 172.21.15.172:0/3158908571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dae02f0 msgr2=0x55679daff9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:45:59.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81edd9b640 1 --2- 172.21.15.172:0/3158908571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dae02f0 0x55679daff9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81edd9b640 1 -- 172.21.15.172:0/3158908571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55679db63e50 con 0x55679daeabb0 2024-09-10T08:45:59.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81ee59c640 1 --2- 172.21.15.172:0/3158908571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dae02f0 0x55679daff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:45:59.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81edd9b640 1 --2- 172.21.15.172:0/3158908571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679daeabb0 0x55679db00030 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f81e4004690 tx=0x7f81e400caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521576a6b5ecda6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:45:59.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.007+0000 7f81d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:45:59.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.008+0000 7f81d5ffb640 1 -- 172.21.15.172:0/3158908571 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e40059c0 con 0x55679daeabb0 2024-09-10T08:45:59.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.008+0000 7f81d5ffb640 1 -- 172.21.15.172:0/3158908571 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f81e4005b60 con 0x55679daeabb0 2024-09-10T08:45:59.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.008+0000 7f81f6d68740 1 -- 172.21.15.172:0/3158908571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81e0031b50 con 0x55679daeabb0 2024-09-10T08:45:59.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.008+0000 7f81d5ffb640 1 -- 172.21.15.172:0/3158908571 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81e4005d60 con 0x55679daeabb0 2024-09-10T08:45:59.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.008+0000 7f81f6d68740 10 client.15081 fetch_fsmap learned FSMap version 28 2024-09-10T08:45:59.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.008+0000 7f81f6d68740 1 -- 172.21.15.172:0/3158908571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55679db64f00 con 0x55679daeabb0 2024-09-10T08:45:59.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.008+0000 7f81d5ffb640 1 -- 172.21.15.172:0/3158908571 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81e0031b50 con 0x55679daeabb0 2024-09-10T08:45:59.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.008+0000 7f81d5ffb640 1 -- 172.21.15.172:0/3158908571 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f81e4006390 con 0x55679daeabb0 2024-09-10T08:45:59.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.008+0000 7f81f6d68740 10 client.15081 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:45:59.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.008+0000 7f81f6d68740 10 client.15081 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-10T08:45:59.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.008+0000 7f81f6d68740 20 client.15081 populate_metadata read hostname 'smithi172' 2024-09-10T08:45:59.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.008+0000 7f81f6d68740 1 --2- 172.21.15.172:0/3158908571 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55679db69b30 0x55679db68840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:45:59.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.008+0000 7f81f6d68740 4 client.15081 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:45:59.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.008+0000 7f81f6d68740 1 -- 172.21.15.172:0/3158908571 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "status"}) -- 0x55679da4db10 con 0x55679db69b30 2024-09-10T08:45:59.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.009+0000 7f81ee59c640 1 --2- 172.21.15.172:0/3158908571 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55679db69b30 0x55679db68840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:45:59.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.009+0000 7f81d5ffb640 10 client.15081 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:45:59.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.009+0000 7f81ee59c640 1 --2- 172.21.15.172:0/3158908571 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55679db69b30 0x55679db68840 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7eaac1537e4b66cf server_cookie=989cfc26726afeb7 in_seq=0 out_seq=0 2024-09-10T08:45:59.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.009+0000 7f81d5ffb640 1 -- 172.21.15.172:0/3158908571 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55679da4db10 con 0x55679db69b30 2024-09-10T08:45:59.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.009+0000 7f81d5ffb640 10 client.15081 handle_command_reply: tid=0 2024-09-10T08:45:59.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.009+0000 7f81f6d68740 1 client.15081 shutdown 2024-09-10T08:45:59.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.009+0000 7f81f6d68740 1 -- 172.21.15.172:0/3158908571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679daeabb0 msgr2=0x55679db00030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:59.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.009+0000 7f81f6d68740 1 --2- 172.21.15.172:0/3158908571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679daeabb0 0x55679db00030 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f81e4004690 tx=0x7f81e400caa0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.009+0000 7f81f6d68740 1 -- 172.21.15.172:0/3158908571 shutdown_connections 2024-09-10T08:45:59.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.009+0000 7f81f6d68740 1 --2- 172.21.15.172:0/3158908571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55679daeabb0 0x55679db00030 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.009+0000 7f81f6d68740 1 --2- 172.21.15.172:0/3158908571 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55679db69b30 0x55679db68840 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.009+0000 7f81f6d68740 1 --2- 172.21.15.172:0/3158908571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55679dae02f0 0x55679daff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.009+0000 7f81f6d68740 1 --2- 172.21.15.172:0/3158908571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55679dadf9d0 0x55679db056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.009+0000 7f81d5ffb640 0 client.15081 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:45:59.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.009+0000 7f81f6d68740 1 -- 172.21.15.172:0/3158908571 >> 172.21.15.172:0/3158908571 conn(0x55679dadbf60 msgr2=0x55679dae6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:59.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.010+0000 7f81f6d68740 1 -- 172.21.15.172:0/3158908571 shutdown_connections 2024-09-10T08:45:59.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.010+0000 7f81f6d68740 1 -- 172.21.15.172:0/3158908571 wait complete. 2024-09-10T08:45:59.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.010+0000 7f81f6d68740 20 client.15081 trim_cache size 0 max 16384 2024-09-10T08:45:59.033 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:45:59.033 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:45:59.033 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 4, 2024-09-10T08:45:59.033 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 24406, 2024-09-10T08:45:59.033 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-09-10T08:45:59.033 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-09-10T08:45:59.034 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-09-10T08:45:59.034 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 64.786868181000003, 2024-09-10T08:45:59.034 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 28, 2024-09-10T08:45:59.034 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 90, 2024-09-10T08:45:59.034 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 88, 2024-09-10T08:45:59.035 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 148.72825664300001 2024-09-10T08:45:59.035 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:45:59.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.010+0000 7f81f498f640 1 -- 172.21.15.172:0/530336871 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f81d80a2c90 msgr2=0x7f81d80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:59.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.010+0000 7f81f498f640 1 --2- 172.21.15.172:0/530336871 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f81d80a2c90 0x7f81d80a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f81e80cce00 tx=0x7f81e8091000 comp rx=0 tx=0).stop 2024-09-10T08:45:59.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.010+0000 7f81f498f640 1 -- 172.21.15.172:0/530336871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f01055b0 msgr2=0x7f81f019cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:45:59.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.010+0000 7f81f498f640 1 --2- 172.21.15.172:0/530336871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f01055b0 0x7f81f019cc80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f81e800eaa0 tx=0x7f81e800ef60 comp rx=0 tx=0).stop 2024-09-10T08:45:59.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.010+0000 7f81f498f640 1 -- 172.21.15.172:0/530336871 shutdown_connections 2024-09-10T08:45:59.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.010+0000 7f81f498f640 1 --2- 172.21.15.172:0/530336871 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f81d80a2c90 0x7f81d80a5150 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.010+0000 7f81f498f640 1 --2- 172.21.15.172:0/530336871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f01055b0 0x7f81f019cc80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.010+0000 7f81f498f640 1 --2- 172.21.15.172:0/530336871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81f0100e30 0x7f81f01988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.010+0000 7f81f498f640 1 --2- 172.21.15.172:0/530336871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81f0100480 0x7f81f01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:45:59.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.010+0000 7f81f498f640 1 -- 172.21.15.172:0/530336871 >> 172.21.15.172:0/530336871 conn(0x7f81f0077210 msgr2=0x7f81f010dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:45:59.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.011+0000 7f81f498f640 1 -- 172.21.15.172:0/530336871 shutdown_connections 2024-09-10T08:45:59.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:45:59.011+0000 7f81f498f640 1 -- 172.21.15.172:0/530336871 wait complete. 2024-09-10T08:45:59.806 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-10T08:46:00.055 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:00.439 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:00 smithi172 ceph-mon[30854]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:00.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:00 smithi174 ceph-mon[34068]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:00 smithi191 ceph-mon[35314]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:00.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.520+0000 7ffa9fcc7640 1 -- 172.21.15.172:0/4181562198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa98102db0 msgr2=0x7ffa98103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:00.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.520+0000 7ffa9fcc7640 1 --2- 172.21.15.172:0/4181562198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa98102db0 0x7ffa98103190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ffa90009930 tx=0x7ffa9002a420 comp rx=0 tx=0).stop 2024-09-10T08:46:00.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.520+0000 7ffa9fcc7640 1 -- 172.21.15.172:0/4181562198 shutdown_connections 2024-09-10T08:46:00.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.520+0000 7ffa9fcc7640 1 --2- 172.21.15.172:0/4181562198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa981082e0 0x7ffa9810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.520+0000 7ffa9fcc7640 1 --2- 172.21.15.172:0/4181562198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa98103760 0x7ffa98107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.520+0000 7ffa9fcc7640 1 --2- 172.21.15.172:0/4181562198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa98102db0 0x7ffa98103190 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.520+0000 7ffa9fcc7640 1 -- 172.21.15.172:0/4181562198 >> 172.21.15.172:0/4181562198 conn(0x7ffa980fc7d0 msgr2=0x7ffa980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:00.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.521+0000 7ffa9fcc7640 1 -- 172.21.15.172:0/4181562198 shutdown_connections 2024-09-10T08:46:00.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.521+0000 7ffa9fcc7640 1 -- 172.21.15.172:0/4181562198 wait complete. 2024-09-10T08:46:00.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.521+0000 7ffa9fcc7640 1 Processor -- start 2024-09-10T08:46:00.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.521+0000 7ffa9fcc7640 1 -- start start 2024-09-10T08:46:00.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.521+0000 7ffa9fcc7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa98102db0 0x7ffa9819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.521+0000 7ffa9fcc7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa98103760 0x7ffa9819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.521+0000 7ffa9fcc7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa981082e0 0x7ffa9819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.521+0000 7ffa9fcc7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffa98111f60 con 0x7ffa98103760 2024-09-10T08:46:00.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.521+0000 7ffa9fcc7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa98111de0 con 0x7ffa981082e0 2024-09-10T08:46:00.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.521+0000 7ffa9fcc7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffa981120e0 con 0x7ffa98102db0 2024-09-10T08:46:00.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.521+0000 7ffa9d262640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa98103760 0x7ffa9819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.521+0000 7ffa9d262640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa98103760 0x7ffa9819ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56974/0 (socket says 172.21.15.172:56974) 2024-09-10T08:46:00.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.521+0000 7ffa9d262640 1 -- 172.21.15.172:0/1211785282 learned_addr learned my addr 172.21.15.172:0/1211785282 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:00.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.522+0000 7ffa9da63640 1 --2- 172.21.15.172:0/1211785282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa98102db0 0x7ffa9819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.522+0000 7ffa9e264640 1 --2- 172.21.15.172:0/1211785282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa981082e0 0x7ffa9819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.522+0000 7ffa9d262640 1 -- 172.21.15.172:0/1211785282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa98102db0 msgr2=0x7ffa9819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:00.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.522+0000 7ffa9d262640 1 --2- 172.21.15.172:0/1211785282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa98102db0 0x7ffa9819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.522+0000 7ffa9d262640 1 -- 172.21.15.172:0/1211785282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa981082e0 msgr2=0x7ffa9819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:00.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.522+0000 7ffa9d262640 1 --2- 172.21.15.172:0/1211785282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa981082e0 0x7ffa9819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.522+0000 7ffa9d262640 1 -- 172.21.15.172:0/1211785282 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa9819f6b0 con 0x7ffa98103760 2024-09-10T08:46:00.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.522+0000 7ffa9d262640 1 --2- 172.21.15.172:0/1211785282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa98103760 0x7ffa9819ac40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00e9c0 tx=0x7ffa8c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:00.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.522+0000 7ffa96ffd640 1 -- 172.21.15.172:0/1211785282 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c00ccc0 con 0x7ffa98103760 2024-09-10T08:46:00.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.522+0000 7ffa96ffd640 1 -- 172.21.15.172:0/1211785282 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa8c0054d0 con 0x7ffa98103760 2024-09-10T08:46:00.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.522+0000 7ffa9fcc7640 1 -- 172.21.15.172:0/1211785282 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa9819f9a0 con 0x7ffa98103760 2024-09-10T08:46:00.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.522+0000 7ffa96ffd640 1 -- 172.21.15.172:0/1211785282 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c010430 con 0x7ffa98103760 2024-09-10T08:46:00.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.522+0000 7ffa9fcc7640 1 -- 172.21.15.172:0/1211785282 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa981a7280 con 0x7ffa98103760 2024-09-10T08:46:00.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.524+0000 7ffa96ffd640 1 -- 172.21.15.172:0/1211785282 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ffa8c005020 con 0x7ffa98103760 2024-09-10T08:46:00.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.525+0000 7ffaa20a0740 1 Processor -- start 2024-09-10T08:46:00.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.525+0000 7ffaa20a0740 1 -- start start 2024-09-10T08:46:00.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffaa20a0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536269d0 0x559953626db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffaa20a0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536272f0 0x559953631670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffaa20a0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559953631bb0 0x559953633fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffaa20a0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55995358d090 con 0x5599536269d0 2024-09-10T08:46:00.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffaa20a0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559953534a70 con 0x5599536272f0 2024-09-10T08:46:00.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffaa20a0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559953589050 con 0x559953631bb0 2024-09-10T08:46:00.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffa96ffd640 1 --2- 172.21.15.172:0/1211785282 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa800a2c90 0x7ffa800a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffa9e264640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536269d0 0x559953626db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffa9e264640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536269d0 0x559953626db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56984/0 (socket says 172.21.15.172:56984) 2024-09-10T08:46:00.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffa9e264640 1 -- 172.21.15.172:0/1599655343 learned_addr learned my addr 172.21.15.172:0/1599655343 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:00.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffa9e264640 1 --2- 172.21.15.172:0/1211785282 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa800a2c90 0x7ffa800a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffa9da63640 1 --2- 172.21.15.172:0/1599655343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536272f0 0x559953631670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffa96ffd640 1 -- 172.21.15.172:0/1211785282 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7ffa8c0cfa20 con 0x7ffa98103760 2024-09-10T08:46:00.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffa9da63640 1 -- 172.21.15.172:0/1599655343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559953631bb0 msgr2=0x559953633fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:00.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffa9d262640 1 --2- 172.21.15.172:0/1599655343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559953631bb0 0x559953633fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffa9da63640 1 --2- 172.21.15.172:0/1599655343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559953631bb0 0x559953633fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffa9da63640 1 -- 172.21.15.172:0/1599655343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536269d0 msgr2=0x559953626db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:00.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffa9da63640 1 --2- 172.21.15.172:0/1599655343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536269d0 0x559953626db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffa9da63640 1 -- 172.21.15.172:0/1599655343 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559953541130 con 0x5599536272f0 2024-09-10T08:46:00.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffa9d262640 1 --2- 172.21.15.172:0/1599655343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559953631bb0 0x559953633fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:00.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffa9e264640 1 --2- 172.21.15.172:0/1599655343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536269d0 0x559953626db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:46:00.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffa9e264640 1 --2- 172.21.15.172:0/1211785282 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa800a2c90 0x7ffa800a5150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa880098b0 tx=0x7ffa88009b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:00.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.526+0000 7ffa9da63640 1 --2- 172.21.15.172:0/1599655343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536272f0 0x559953631670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa90005940 tx=0x7ffa90005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8d8e9ca4e59b35e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:00.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.527+0000 7ffa7ffff640 1 -- 172.21.15.172:0/1599655343 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa90040070 con 0x5599536272f0 2024-09-10T08:46:00.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.527+0000 7ffa7ffff640 1 -- 172.21.15.172:0/1599655343 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa90004030 con 0x5599536272f0 2024-09-10T08:46:00.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.527+0000 7ffa7ffff640 1 -- 172.21.15.172:0/1599655343 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa90035040 con 0x5599536272f0 2024-09-10T08:46:00.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.527+0000 7ffaa20a0740 1 -- 172.21.15.172:0/1599655343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536272f0 msgr2=0x559953631670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:00.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.527+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/1599655343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536272f0 0x559953631670 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ffa90005940 tx=0x7ffa90005c00 comp rx=0 tx=0).stop 2024-09-10T08:46:00.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.528+0000 7ffaa20a0740 1 -- 172.21.15.172:0/1599655343 shutdown_connections 2024-09-10T08:46:00.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.528+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/1599655343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559953631bb0 0x559953633fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.528+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/1599655343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536272f0 0x559953631670 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.528+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/1599655343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536269d0 0x559953626db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.528+0000 7ffaa20a0740 1 -- 172.21.15.172:0/1599655343 >> 172.21.15.172:0/1599655343 conn(0x559953622f60 msgr2=0x559953623340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:00.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.528+0000 7ffaa20a0740 1 -- 172.21.15.172:0/1599655343 shutdown_connections 2024-09-10T08:46:00.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.528+0000 7ffaa20a0740 1 -- 172.21.15.172:0/1599655343 wait complete. 2024-09-10T08:46:00.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.528+0000 7ffaa20a0740 1 -- start start 2024-09-10T08:46:00.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.528+0000 7ffaa20a0740 1 Processor -- start 2024-09-10T08:46:00.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.528+0000 7ffaa20a0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536269d0 0x5599536313e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.528+0000 7ffaa20a0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536272f0 0x55995362e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.528+0000 7ffaa20a0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559953631bb0 0x55995362ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.528+0000 7ffaa20a0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559953634780 con 0x5599536272f0 2024-09-10T08:46:00.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.528+0000 7ffaa20a0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559953547f20 con 0x5599536269d0 2024-09-10T08:46:00.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.528+0000 7ffaa20a0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559953634900 con 0x559953631bb0 2024-09-10T08:46:00.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffaa20a0740 10 client.0 _ll_register_callbacks cb 0x559953511030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:00.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffa9d262640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536272f0 0x55995362e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffa9d262640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536272f0 0x55995362e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56988/0 (socket says 172.21.15.172:56988) 2024-09-10T08:46:00.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffa9d262640 1 -- 172.21.15.172:0/2298870043 learned_addr learned my addr 172.21.15.172:0/2298870043 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:00.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffa9da63640 1 --2- 172.21.15.172:0/2298870043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536269d0 0x5599536313e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffa9e264640 1 --2- 172.21.15.172:0/2298870043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559953631bb0 0x55995362ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffa9d262640 1 -- 172.21.15.172:0/2298870043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559953631bb0 msgr2=0x55995362ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:00.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffa9d262640 1 --2- 172.21.15.172:0/2298870043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559953631bb0 0x55995362ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffa9d262640 1 -- 172.21.15.172:0/2298870043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536269d0 msgr2=0x5599536313e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:00.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffa9d262640 1 --2- 172.21.15.172:0/2298870043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536269d0 0x5599536313e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffa9d262640 1 -- 172.21.15.172:0/2298870043 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599532fe2b0 con 0x5599536272f0 2024-09-10T08:46:00.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffa9d262640 1 --2- 172.21.15.172:0/2298870043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536272f0 0x55995362e730 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0981e0 tx=0x7ffa8c09a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa08398c7c0cb1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:00.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffa7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:00.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffa7d7fa640 1 -- 172.21.15.172:0/2298870043 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c005310 con 0x5599536272f0 2024-09-10T08:46:00.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffa7d7fa640 1 -- 172.21.15.172:0/2298870043 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7ffa8c09b1d0 con 0x5599536272f0 2024-09-10T08:46:00.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffaa20a0740 1 -- 172.21.15.172:0/2298870043 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa90004030 con 0x5599536272f0 2024-09-10T08:46:00.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffa7d7fa640 1 -- 172.21.15.172:0/2298870043 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa8c005680 con 0x5599536272f0 2024-09-10T08:46:00.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffaa20a0740 10 client.15093 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:00.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.529+0000 7ffaa20a0740 1 -- 172.21.15.172:0/2298870043 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55995363ffa0 con 0x5599536272f0 2024-09-10T08:46:00.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.530+0000 7ffa7d7fa640 1 -- 172.21.15.172:0/2298870043 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa90004030 con 0x5599536272f0 2024-09-10T08:46:00.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.530+0000 7ffa7d7fa640 1 -- 172.21.15.172:0/2298870043 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ffa8c097ad0 con 0x5599536272f0 2024-09-10T08:46:00.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.530+0000 7ffaa20a0740 10 client.15093 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:00.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.530+0000 7ffaa20a0740 10 client.15093 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T08:46:00.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.530+0000 7ffaa20a0740 20 client.15093 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:00.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.530+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/2298870043 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5599536425b0 0x559953643a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.530+0000 7ffaa20a0740 4 client.15093 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:00.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.530+0000 7ffaa20a0740 1 -- 172.21.15.172:0/2298870043 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55995359a600 con 0x5599536425b0 2024-09-10T08:46:00.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.530+0000 7ffa9da63640 1 --2- 172.21.15.172:0/2298870043 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5599536425b0 0x559953643a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.530+0000 7ffa9da63640 1 --2- 172.21.15.172:0/2298870043 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5599536425b0 0x559953643a20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c06a486381c72d1f server_cookie=9ded369d89871136 in_seq=0 out_seq=0 2024-09-10T08:46:00.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.530+0000 7ffa7d7fa640 10 client.15093 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:46:00.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.531+0000 7ffa7d7fa640 1 -- 172.21.15.172:0/2298870043 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55995359a600 con 0x5599536425b0 2024-09-10T08:46:00.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.531+0000 7ffa7d7fa640 10 client.15093 handle_command_reply: tid=0 2024-09-10T08:46:00.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.531+0000 7ffaa20a0740 1 client.15093 shutdown 2024-09-10T08:46:00.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.531+0000 7ffaa20a0740 1 -- 172.21.15.172:0/2298870043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536272f0 msgr2=0x55995362e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:00.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.531+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/2298870043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536272f0 0x55995362e730 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0981e0 tx=0x7ffa8c09a530 comp rx=0 tx=0).stop 2024-09-10T08:46:00.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.531+0000 7ffaa20a0740 1 -- 172.21.15.172:0/2298870043 shutdown_connections 2024-09-10T08:46:00.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.531+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/2298870043 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5599536425b0 0x559953643a20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.531+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/2298870043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559953631bb0 0x55995362ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.531+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/2298870043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536272f0 0x55995362e730 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.531+0000 7ffa7d7fa640 0 client.15093 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:46:00.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.531+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/2298870043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536269d0 0x5599536313e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.532+0000 7ffaa20a0740 1 -- 172.21.15.172:0/2298870043 >> 172.21.15.172:0/2298870043 conn(0x559953622f60 msgr2=0x559953628c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:00.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.532+0000 7ffaa20a0740 1 -- 172.21.15.172:0/2298870043 shutdown_connections 2024-09-10T08:46:00.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.532+0000 7ffaa20a0740 1 -- 172.21.15.172:0/2298870043 wait complete. 2024-09-10T08:46:00.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.532+0000 7ffaa20a0740 20 client.15093 trim_cache size 0 max 16384 2024-09-10T08:46:00.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.549+0000 7ffaa20a0740 1 Processor -- start 2024-09-10T08:46:00.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.549+0000 7ffaa20a0740 1 -- start start 2024-09-10T08:46:00.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.549+0000 7ffaa20a0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536269d0 0x559953641ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.549+0000 7ffaa20a0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599536272f0 0x559953643a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.549+0000 7ffaa20a0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559953631bb0 0x55995362dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.549+0000 7ffaa20a0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559953641d60 con 0x559953631bb0 2024-09-10T08:46:00.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.549+0000 7ffaa20a0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559953642430 con 0x5599536269d0 2024-09-10T08:46:00.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.549+0000 7ffaa20a0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559953642b00 con 0x5599536272f0 2024-09-10T08:46:00.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.550+0000 7ffa9e264640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559953631bb0 0x55995362dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.550+0000 7ffa9e264640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559953631bb0 0x55995362dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57000/0 (socket says 172.21.15.172:57000) 2024-09-10T08:46:00.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.550+0000 7ffa9d262640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599536272f0 0x559953643a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.550+0000 7ffa9e264640 1 -- 172.21.15.172:0/2606758136 learned_addr learned my addr 172.21.15.172:0/2606758136 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:00.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.550+0000 7ffa9da63640 1 --2- 172.21.15.172:0/2606758136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536269d0 0x559953641ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.550+0000 7ffa9e264640 1 -- 172.21.15.172:0/2606758136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599536272f0 msgr2=0x559953643a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:00.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.550+0000 7ffa9e264640 1 --2- 172.21.15.172:0/2606758136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599536272f0 0x559953643a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.550+0000 7ffa9e264640 1 -- 172.21.15.172:0/2606758136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536269d0 msgr2=0x559953641ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:00.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.550+0000 7ffa9e264640 1 --2- 172.21.15.172:0/2606758136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536269d0 0x559953641ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.550+0000 7ffa9e264640 1 -- 172.21.15.172:0/2606758136 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559953627120 con 0x559953631bb0 2024-09-10T08:46:00.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.550+0000 7ffa9e264640 1 --2- 172.21.15.172:0/2606758136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559953631bb0 0x55995362dff0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffa88009c90 tx=0x7ffa8801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1630983f4820f4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:00.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.550+0000 7ffa7f7fe640 1 -- 172.21.15.172:0/2606758136 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88009da0 con 0x559953631bb0 2024-09-10T08:46:00.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.550+0000 7ffa7f7fe640 1 -- 172.21.15.172:0/2606758136 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa88012c60 con 0x559953631bb0 2024-09-10T08:46:00.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.550+0000 7ffa7f7fe640 1 -- 172.21.15.172:0/2606758136 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa880057e0 con 0x559953631bb0 2024-09-10T08:46:00.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.551+0000 7ffaa20a0740 1 -- 172.21.15.172:0/2606758136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559953631bb0 msgr2=0x55995362dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:00.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.551+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/2606758136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559953631bb0 0x55995362dff0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffa88009c90 tx=0x7ffa8801f000 comp rx=0 tx=0).stop 2024-09-10T08:46:00.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.551+0000 7ffaa20a0740 1 -- 172.21.15.172:0/2606758136 shutdown_connections 2024-09-10T08:46:00.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.551+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/2606758136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559953631bb0 0x55995362dff0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.551+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/2606758136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599536272f0 0x559953643a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.551+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/2606758136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536269d0 0x559953641ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.551+0000 7ffaa20a0740 1 -- 172.21.15.172:0/2606758136 >> 172.21.15.172:0/2606758136 conn(0x559953622f60 msgr2=0x55995362e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:00.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.551+0000 7ffaa20a0740 1 -- 172.21.15.172:0/2606758136 shutdown_connections 2024-09-10T08:46:00.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.551+0000 7ffaa20a0740 1 -- 172.21.15.172:0/2606758136 wait complete. 2024-09-10T08:46:00.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.551+0000 7ffaa20a0740 1 -- start start 2024-09-10T08:46:00.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.551+0000 7ffaa20a0740 1 Processor -- start 2024-09-10T08:46:00.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffaa20a0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536269d0 0x5599536454e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffaa20a0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536272f0 0x559953645a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffaa20a0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559953631bb0 0x559953649920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffaa20a0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5599536438a0 con 0x5599536272f0 2024-09-10T08:46:00.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffaa20a0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5599536431d0 con 0x5599536269d0 2024-09-10T08:46:00.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffaa20a0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559953643f70 con 0x559953631bb0 2024-09-10T08:46:00.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffaa20a0740 10 client.0 _ll_register_callbacks cb 0x7ffa8c0096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:00.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffa9d262640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536272f0 0x559953645a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffa9d262640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536272f0 0x559953645a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57008/0 (socket says 172.21.15.172:57008) 2024-09-10T08:46:00.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffa9d262640 1 -- 172.21.15.172:0/1461901689 learned_addr learned my addr 172.21.15.172:0/1461901689 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:00.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffa9e264640 1 --2- 172.21.15.172:0/1461901689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559953631bb0 0x559953649920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffa9da63640 1 --2- 172.21.15.172:0/1461901689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536269d0 0x5599536454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.580 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:46:00.580 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:46:00.580 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 3, 2024-09-10T08:46:00.580 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 24416, 2024-09-10T08:46:00.580 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-09-10T08:46:00.580 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-09-10T08:46:00.580 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-09-10T08:46:00.580 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 67.342980819000005, 2024-09-10T08:46:00.580 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 28, 2024-09-10T08:46:00.580 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 90, 2024-09-10T08:46:00.580 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 88, 2024-09-10T08:46:00.581 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 147.47453857400001 2024-09-10T08:46:00.581 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:46:00.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffa9d262640 1 -- 172.21.15.172:0/1461901689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559953631bb0 msgr2=0x559953649920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:00.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffa9d262640 1 --2- 172.21.15.172:0/1461901689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559953631bb0 0x559953649920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffa9d262640 1 -- 172.21.15.172:0/1461901689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536269d0 msgr2=0x5599536454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:00.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffa9d262640 1 --2- 172.21.15.172:0/1461901689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536269d0 0x5599536454e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffa9d262640 1 -- 172.21.15.172:0/1461901689 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55995364a150 con 0x5599536272f0 2024-09-10T08:46:00.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffa9d262640 1 --2- 172.21.15.172:0/1461901689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536272f0 0x559953645a20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c004650 tx=0x7ffa8c097f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0f2a0e4aeb03db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:00.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffa7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:00.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffa7d7fa640 1 -- 172.21.15.172:0/1461901689 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c005310 con 0x5599536272f0 2024-09-10T08:46:00.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffa7d7fa640 1 -- 172.21.15.172:0/1461901689 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7ffa8c09ac40 con 0x5599536272f0 2024-09-10T08:46:00.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.552+0000 7ffaa20a0740 1 -- 172.21.15.172:0/1461901689 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa88012c60 con 0x5599536272f0 2024-09-10T08:46:00.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.553+0000 7ffa7d7fa640 1 -- 172.21.15.172:0/1461901689 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa8c09c480 con 0x5599536272f0 2024-09-10T08:46:00.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.553+0000 7ffaa20a0740 10 client.15099 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:00.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.553+0000 7ffaa20a0740 1 -- 172.21.15.172:0/1461901689 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55995364c510 con 0x5599536272f0 2024-09-10T08:46:00.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.553+0000 7ffa7d7fa640 1 -- 172.21.15.172:0/1461901689 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa88012c60 con 0x5599536272f0 2024-09-10T08:46:00.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.553+0000 7ffa7d7fa640 1 -- 172.21.15.172:0/1461901689 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ffa8c09ae40 con 0x5599536272f0 2024-09-10T08:46:00.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.553+0000 7ffaa20a0740 10 client.15099 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:00.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.553+0000 7ffaa20a0740 10 client.15099 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T08:46:00.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.553+0000 7ffaa20a0740 20 client.15099 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:00.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.553+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/1461901689 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5599536b1060 0x5599536afd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:00.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.553+0000 7ffaa20a0740 4 client.15099 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:46:00.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.553+0000 7ffaa20a0740 1 -- 172.21.15.172:0/1461901689 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "status"}) -- 0x559953594b10 con 0x5599536b1060 2024-09-10T08:46:00.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.553+0000 7ffa9da63640 1 --2- 172.21.15.172:0/1461901689 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5599536b1060 0x5599536afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:00.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.554+0000 7ffa9da63640 1 --2- 172.21.15.172:0/1461901689 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5599536b1060 0x5599536afd70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=86d4e8a7ec9b8e69 server_cookie=138ed8453ff66a4e in_seq=0 out_seq=0 2024-09-10T08:46:00.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.554+0000 7ffa7d7fa640 10 client.15099 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:46:00.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.554+0000 7ffa7d7fa640 1 -- 172.21.15.172:0/1461901689 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559953594b10 con 0x5599536b1060 2024-09-10T08:46:00.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.554+0000 7ffa7d7fa640 10 client.15099 handle_command_reply: tid=0 2024-09-10T08:46:00.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.554+0000 7ffaa20a0740 1 client.15099 shutdown 2024-09-10T08:46:00.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.554+0000 7ffaa20a0740 1 -- 172.21.15.172:0/1461901689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536272f0 msgr2=0x559953645a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:00.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.554+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/1461901689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536272f0 0x559953645a20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c004650 tx=0x7ffa8c097f80 comp rx=0 tx=0).stop 2024-09-10T08:46:00.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.554+0000 7ffaa20a0740 1 -- 172.21.15.172:0/1461901689 shutdown_connections 2024-09-10T08:46:00.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.554+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/1461901689 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5599536b1060 0x5599536afd70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.554+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/1461901689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559953631bb0 0x559953649920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.554+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/1461901689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599536272f0 0x559953645a20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.554+0000 7ffa7d7fa640 0 client.15099 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:46:00.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.554+0000 7ffaa20a0740 1 --2- 172.21.15.172:0/1461901689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599536269d0 0x5599536454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.554+0000 7ffaa20a0740 1 -- 172.21.15.172:0/1461901689 >> 172.21.15.172:0/1461901689 conn(0x559953622f60 msgr2=0x55995362cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:00.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.554+0000 7ffaa20a0740 1 -- 172.21.15.172:0/1461901689 shutdown_connections 2024-09-10T08:46:00.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.554+0000 7ffaa20a0740 1 -- 172.21.15.172:0/1461901689 wait complete. 2024-09-10T08:46:00.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.554+0000 7ffaa20a0740 20 client.15099 trim_cache size 0 max 16384 2024-09-10T08:46:00.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.555+0000 7ffa9fcc7640 1 -- 172.21.15.172:0/1211785282 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa800a2c90 msgr2=0x7ffa800a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:00.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.555+0000 7ffa9fcc7640 1 --2- 172.21.15.172:0/1211785282 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa800a2c90 0x7ffa800a5150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa880098b0 tx=0x7ffa88009b50 comp rx=0 tx=0).stop 2024-09-10T08:46:00.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.555+0000 7ffa9fcc7640 1 -- 172.21.15.172:0/1211785282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa98103760 msgr2=0x7ffa9819ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:00.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.555+0000 7ffa9fcc7640 1 --2- 172.21.15.172:0/1211785282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa98103760 0x7ffa9819ac40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00e9c0 tx=0x7ffa8c00ee80 comp rx=0 tx=0).stop 2024-09-10T08:46:00.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.555+0000 7ffa9fcc7640 1 -- 172.21.15.172:0/1211785282 shutdown_connections 2024-09-10T08:46:00.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.555+0000 7ffa9fcc7640 1 --2- 172.21.15.172:0/1211785282 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa800a2c90 0x7ffa800a5150 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.555+0000 7ffa9fcc7640 1 --2- 172.21.15.172:0/1211785282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa981082e0 0x7ffa9819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.555+0000 7ffa9fcc7640 1 --2- 172.21.15.172:0/1211785282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa98103760 0x7ffa9819ac40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.555+0000 7ffa9fcc7640 1 --2- 172.21.15.172:0/1211785282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa98102db0 0x7ffa9819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:00.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.555+0000 7ffa9fcc7640 1 -- 172.21.15.172:0/1211785282 >> 172.21.15.172:0/1211785282 conn(0x7ffa980fc7d0 msgr2=0x7ffa98100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:00.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.555+0000 7ffa9fcc7640 1 -- 172.21.15.172:0/1211785282 shutdown_connections 2024-09-10T08:46:00.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:00.555+0000 7ffa9fcc7640 1 -- 172.21.15.172:0/1211785282 wait complete. 2024-09-10T08:46:01.433 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T08:46:01.706 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:02.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.180+0000 7fc2111c0640 1 -- 172.21.15.172:0/2036513627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc20c102d40 msgr2=0x7fc20c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.180+0000 7fc2111c0640 1 --2- 172.21.15.172:0/2036513627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc20c102d40 0x7fc20c103120 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009900 tx=0x7fc1fc02a480 comp rx=0 tx=0).stop 2024-09-10T08:46:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.180+0000 7fc2111c0640 1 -- 172.21.15.172:0/2036513627 shutdown_connections 2024-09-10T08:46:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.180+0000 7fc2111c0640 1 --2- 172.21.15.172:0/2036513627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc20c10c170 0x7fc20c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.180+0000 7fc2111c0640 1 --2- 172.21.15.172:0/2036513627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc20c103660 0x7fc20c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.180+0000 7fc2111c0640 1 --2- 172.21.15.172:0/2036513627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc20c102d40 0x7fc20c103120 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.180+0000 7fc2111c0640 1 -- 172.21.15.172:0/2036513627 >> 172.21.15.172:0/2036513627 conn(0x7fc20c0fc7a0 msgr2=0x7fc20c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.180+0000 7fc2111c0640 1 -- 172.21.15.172:0/2036513627 shutdown_connections 2024-09-10T08:46:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.181+0000 7fc2111c0640 1 -- 172.21.15.172:0/2036513627 wait complete. 2024-09-10T08:46:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.181+0000 7fc2111c0640 1 Processor -- start 2024-09-10T08:46:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.181+0000 7fc2111c0640 1 -- start start 2024-09-10T08:46:02.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.181+0000 7fc2111c0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc20c102d40 0x7fc20c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.181+0000 7fc2111c0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc20c103660 0x7fc20c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.181+0000 7fc2111c0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc20c10c170 0x7fc20c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc2111c0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c111e20 con 0x7fc20c103660 2024-09-10T08:46:02.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc2111c0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c111ca0 con 0x7fc20c10c170 2024-09-10T08:46:02.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc2111c0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc20c111fa0 con 0x7fc20c102d40 2024-09-10T08:46:02.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc203fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc20c103660 0x7fc20c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc203fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc20c103660 0x7fc20c198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57034/0 (socket says 172.21.15.172:57034) 2024-09-10T08:46:02.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc203fff640 1 -- 172.21.15.172:0/1563062078 learned_addr learned my addr 172.21.15.172:0/1563062078 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:02.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc20b59e640 1 --2- 172.21.15.172:0/1563062078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc20c10c170 0x7fc20c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc20ad9d640 1 --2- 172.21.15.172:0/1563062078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc20c102d40 0x7fc20c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc203fff640 1 -- 172.21.15.172:0/1563062078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc20c102d40 msgr2=0x7fc20c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:02.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc203fff640 1 --2- 172.21.15.172:0/1563062078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc20c102d40 0x7fc20c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc203fff640 1 -- 172.21.15.172:0/1563062078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc20c10c170 msgr2=0x7fc20c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:02.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc203fff640 1 --2- 172.21.15.172:0/1563062078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc20c10c170 0x7fc20c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc203fff640 1 -- 172.21.15.172:0/1563062078 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20c19d300 con 0x7fc20c103660 2024-09-10T08:46:02.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc203fff640 1 --2- 172.21.15.172:0/1563062078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc20c103660 0x7fc20c198890 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400c940 tx=0x7fc1f400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:02.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc208d99640 1 -- 172.21.15.172:0/1563062078 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4004220 con 0x7fc20c103660 2024-09-10T08:46:02.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc208d99640 1 -- 172.21.15.172:0/1563062078 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1f40043c0 con 0x7fc20c103660 2024-09-10T08:46:02.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc2111c0640 1 -- 172.21.15.172:0/1563062078 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20c19d5f0 con 0x7fc20c103660 2024-09-10T08:46:02.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc208d99640 1 -- 172.21.15.172:0/1563062078 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4004a00 con 0x7fc20c103660 2024-09-10T08:46:02.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.182+0000 7fc2111c0640 1 -- 172.21.15.172:0/1563062078 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc20c1008d0 con 0x7fc20c103660 2024-09-10T08:46:02.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.185+0000 7fc208d99640 1 -- 172.21.15.172:0/1563062078 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc1f40079b0 con 0x7fc20c103660 2024-09-10T08:46:02.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.185+0000 7fc213599740 1 Processor -- start 2024-09-10T08:46:02.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.185+0000 7fc213599740 1 -- start start 2024-09-10T08:46:02.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.185+0000 7fc213599740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 0x55e01bae5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.185+0000 7fc213599740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01bae62f0 0x55e01baf0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.185+0000 7fc213599740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01baf0bb0 0x55e01baf2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.185+0000 7fc213599740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e01ba4c090 con 0x55e01baf0bb0 2024-09-10T08:46:02.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.185+0000 7fc213599740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e01b9f3a70 con 0x55e01bae62f0 2024-09-10T08:46:02.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.185+0000 7fc213599740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e01ba48050 con 0x55e01bae59d0 2024-09-10T08:46:02.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.185+0000 7fc208d99640 1 --2- 172.21.15.172:0/1563062078 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc1f80a2c10 0x7fc1f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc203fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01baf0bb0 0x55e01baf2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc20b59e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 0x55e01bae5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc203fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01baf0bb0 0x55e01baf2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57036/0 (socket says 172.21.15.172:57036) 2024-09-10T08:46:02.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc203fff640 1 -- 172.21.15.172:0/3961102384 learned_addr learned my addr 172.21.15.172:0/3961102384 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:02.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc20b59e640 1 --2- 172.21.15.172:0/1563062078 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc1f80a2c10 0x7fc1f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc20ad9d640 1 --2- 172.21.15.172:0/3961102384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01bae62f0 0x55e01baf0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc208d99640 1 -- 172.21.15.172:0/1563062078 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fc1f40cf430 con 0x7fc20c103660 2024-09-10T08:46:02.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc20b59e640 1 -- 172.21.15.172:0/3961102384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01bae62f0 msgr2=0x55e01baf0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:02.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc20b59e640 1 --2- 172.21.15.172:0/3961102384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01bae62f0 0x55e01baf0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc20b59e640 1 -- 172.21.15.172:0/3961102384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01baf0bb0 msgr2=0x55e01baf2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:02.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc20b59e640 1 --2- 172.21.15.172:0/3961102384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01baf0bb0 0x55e01baf2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc20b59e640 1 -- 172.21.15.172:0/3961102384 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e01ba00130 con 0x55e01bae59d0 2024-09-10T08:46:02.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc203fff640 1 --2- 172.21.15.172:0/3961102384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01baf0bb0 0x55e01baf2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:46:02.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc20b59e640 1 --2- 172.21.15.172:0/1563062078 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc1f80a2c10 0x7fc1f80a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc204011cb0 tx=0x7fc204016000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:02.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc20b59e640 1 --2- 172.21.15.172:0/3961102384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 0x55e01bae5db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc204014db0 tx=0x7fc2040117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3db04790099463d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc2017fa640 1 -- 172.21.15.172:0/3961102384 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20400cd70 con 0x55e01bae59d0 2024-09-10T08:46:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc2017fa640 1 -- 172.21.15.172:0/3961102384 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc20400abb0 con 0x55e01bae59d0 2024-09-10T08:46:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.186+0000 7fc2017fa640 1 -- 172.21.15.172:0/3961102384 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20400f450 con 0x55e01bae59d0 2024-09-10T08:46:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.187+0000 7fc213599740 1 -- 172.21.15.172:0/3961102384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 msgr2=0x55e01bae5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.187+0000 7fc213599740 1 --2- 172.21.15.172:0/3961102384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 0x55e01bae5db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc204014db0 tx=0x7fc2040117b0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.187+0000 7fc213599740 1 -- 172.21.15.172:0/3961102384 shutdown_connections 2024-09-10T08:46:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.187+0000 7fc213599740 1 --2- 172.21.15.172:0/3961102384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01baf0bb0 0x55e01baf2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.187+0000 7fc213599740 1 --2- 172.21.15.172:0/3961102384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01bae62f0 0x55e01baf0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.187+0000 7fc213599740 1 --2- 172.21.15.172:0/3961102384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 0x55e01bae5db0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.187+0000 7fc213599740 1 -- 172.21.15.172:0/3961102384 >> 172.21.15.172:0/3961102384 conn(0x55e01bae1f60 msgr2=0x55e01bae2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.187+0000 7fc213599740 1 -- 172.21.15.172:0/3961102384 shutdown_connections 2024-09-10T08:46:02.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.187+0000 7fc213599740 1 -- 172.21.15.172:0/3961102384 wait complete. 2024-09-10T08:46:02.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc213599740 1 -- start start 2024-09-10T08:46:02.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc213599740 1 Processor -- start 2024-09-10T08:46:02.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc213599740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 0x55e01baf0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc213599740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01bae62f0 0x55e01baed7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc213599740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01baf0bb0 0x55e01baedd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc213599740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e01baf3780 con 0x55e01bae62f0 2024-09-10T08:46:02.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc213599740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e01ba06f20 con 0x55e01baf0bb0 2024-09-10T08:46:02.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc213599740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e01baf3900 con 0x55e01bae59d0 2024-09-10T08:46:02.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc213599740 10 client.0 _ll_register_callbacks cb 0x55e01b9d0030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:02.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc203fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01bae62f0 0x55e01baed7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc203fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01bae62f0 0x55e01baed7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57050/0 (socket says 172.21.15.172:57050) 2024-09-10T08:46:02.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc203fff640 1 -- 172.21.15.172:0/1810367934 learned_addr learned my addr 172.21.15.172:0/1810367934 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc20ad9d640 1 --2- 172.21.15.172:0/1810367934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 0x55e01baf0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc20b59e640 1 --2- 172.21.15.172:0/1810367934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01baf0bb0 0x55e01baedd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc203fff640 1 -- 172.21.15.172:0/1810367934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 msgr2=0x55e01baf0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc203fff640 1 --2- 172.21.15.172:0/1810367934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 0x55e01baf0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc203fff640 1 -- 172.21.15.172:0/1810367934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01baf0bb0 msgr2=0x55e01baedd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc203fff640 1 --2- 172.21.15.172:0/1810367934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01baf0bb0 0x55e01baedd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.188+0000 7fc203fff640 1 -- 172.21.15.172:0/1810367934 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e01b7bd2b0 con 0x55e01bae62f0 2024-09-10T08:46:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.189+0000 7fc203fff640 1 --2- 172.21.15.172:0/1810367934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01bae62f0 0x55e01baed7d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4005fd0 tx=0x7fc1f400a4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f3c323dd5282bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.189+0000 7fc1f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.189+0000 7fc1f27fc640 1 -- 172.21.15.172:0/1810367934 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f401d040 con 0x55e01bae62f0 2024-09-10T08:46:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.189+0000 7fc1f27fc640 1 -- 172.21.15.172:0/1810367934 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fc1f40cccf0 con 0x55e01bae62f0 2024-09-10T08:46:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.189+0000 7fc213599740 1 -- 172.21.15.172:0/1810367934 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e01ba59600 con 0x55e01bae62f0 2024-09-10T08:46:02.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.189+0000 7fc1f27fc640 1 -- 172.21.15.172:0/1810367934 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc1f40056a0 con 0x55e01bae62f0 2024-09-10T08:46:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.189+0000 7fc213599740 10 client.15111 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.189+0000 7fc213599740 1 -- 172.21.15.172:0/1810367934 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e01baff130 con 0x55e01bae62f0 2024-09-10T08:46:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.189+0000 7fc1f27fc640 1 -- 172.21.15.172:0/1810367934 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55e01ba59600 con 0x55e01bae62f0 2024-09-10T08:46:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.189+0000 7fc1f27fc640 1 -- 172.21.15.172:0/1810367934 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fc1f40a3fd0 con 0x55e01bae62f0 2024-09-10T08:46:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.189+0000 7fc213599740 10 client.15111 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.189+0000 7fc213599740 10 client.15111 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:46:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.189+0000 7fc213599740 20 client.15111 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.189+0000 7fc213599740 1 --2- 172.21.15.172:0/1810367934 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e01bb01740 0x55e01bb02bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.189+0000 7fc213599740 4 client.15111 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.189+0000 7fc213599740 1 -- 172.21.15.172:0/1810367934 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e01ba53b10 con 0x55e01bb01740 2024-09-10T08:46:02.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.190+0000 7fc20ad9d640 1 --2- 172.21.15.172:0/1810367934 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e01bb01740 0x55e01bb02bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.190+0000 7fc1f27fc640 10 client.15111 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:46:02.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.190+0000 7fc20ad9d640 1 --2- 172.21.15.172:0/1810367934 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e01bb01740 0x55e01bb02bb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9cce78987b6751f8 server_cookie=df3dc3b823057250 in_seq=0 out_seq=0 2024-09-10T08:46:02.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.191+0000 7fc1f27fc640 1 -- 172.21.15.172:0/1810367934 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e01ba53b10 con 0x55e01bb01740 2024-09-10T08:46:02.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.191+0000 7fc1f27fc640 10 client.15111 handle_command_reply: tid=0 2024-09-10T08:46:02.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.191+0000 7fc213599740 1 client.15111 shutdown 2024-09-10T08:46:02.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.191+0000 7fc213599740 1 -- 172.21.15.172:0/1810367934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01bae62f0 msgr2=0x55e01baed7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:02.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.191+0000 7fc213599740 1 --2- 172.21.15.172:0/1810367934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01bae62f0 0x55e01baed7d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4005fd0 tx=0x7fc1f400a4f0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.191+0000 7fc213599740 1 -- 172.21.15.172:0/1810367934 shutdown_connections 2024-09-10T08:46:02.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.191+0000 7fc213599740 1 --2- 172.21.15.172:0/1810367934 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e01bb01740 0x55e01bb02bb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.191+0000 7fc213599740 1 --2- 172.21.15.172:0/1810367934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01baf0bb0 0x55e01baedd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.191+0000 7fc213599740 1 --2- 172.21.15.172:0/1810367934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01bae62f0 0x55e01baed7d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.191+0000 7fc213599740 1 --2- 172.21.15.172:0/1810367934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 0x55e01baf0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.191+0000 7fc1f27fc640 0 client.15111 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:46:02.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.191+0000 7fc213599740 1 -- 172.21.15.172:0/1810367934 >> 172.21.15.172:0/1810367934 conn(0x55e01bae1f60 msgr2=0x55e01bae7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:02.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.191+0000 7fc213599740 1 -- 172.21.15.172:0/1810367934 shutdown_connections 2024-09-10T08:46:02.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.191+0000 7fc213599740 1 -- 172.21.15.172:0/1810367934 wait complete. 2024-09-10T08:46:02.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.191+0000 7fc213599740 20 client.15111 trim_cache size 0 max 16384 2024-09-10T08:46:02.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.209+0000 7fc213599740 1 Processor -- start 2024-09-10T08:46:02.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.209+0000 7fc213599740 1 -- start start 2024-09-10T08:46:02.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.209+0000 7fc213599740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 0x55e01baed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.209+0000 7fc213599740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01bae62f0 0x55e01bb0ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.209+0000 7fc213599740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01baf0bb0 0x55e01bb0b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.209+0000 7fc213599740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e01bb00ef0 con 0x55e01baf0bb0 2024-09-10T08:46:02.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.209+0000 7fc213599740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e01bb015c0 con 0x55e01bae62f0 2024-09-10T08:46:02.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.209+0000 7fc213599740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e01bb01c90 con 0x55e01bae59d0 2024-09-10T08:46:02.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.209+0000 7fc20b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01baf0bb0 0x55e01bb0b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.209+0000 7fc20b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01baf0bb0 0x55e01bb0b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57064/0 (socket says 172.21.15.172:57064) 2024-09-10T08:46:02.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.209+0000 7fc20b59e640 1 -- 172.21.15.172:0/657175799 learned_addr learned my addr 172.21.15.172:0/657175799 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:02.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.209+0000 7fc203fff640 1 --2- 172.21.15.172:0/657175799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01bae62f0 0x55e01bb0ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.209+0000 7fc20ad9d640 1 --2- 172.21.15.172:0/657175799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 0x55e01baed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.210+0000 7fc20b59e640 1 -- 172.21.15.172:0/657175799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 msgr2=0x55e01baed090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:02.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.210+0000 7fc20b59e640 1 --2- 172.21.15.172:0/657175799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 0x55e01baed090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.210+0000 7fc20b59e640 1 -- 172.21.15.172:0/657175799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01bae62f0 msgr2=0x55e01bb0ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:02.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.210+0000 7fc20b59e640 1 --2- 172.21.15.172:0/657175799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01bae62f0 0x55e01bb0ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.210+0000 7fc20b59e640 1 -- 172.21.15.172:0/657175799 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e01baf00a0 con 0x55e01baf0bb0 2024-09-10T08:46:02.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.210+0000 7fc203fff640 1 --2- 172.21.15.172:0/657175799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01bae62f0 0x55e01bb0ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:02.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.210+0000 7fc20b59e640 1 --2- 172.21.15.172:0/657175799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01baf0bb0 0x55e01bb0b3d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc20400ae70 tx=0x7fc20400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b773721e9ac761ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:02.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.210+0000 7fc200ff9640 1 -- 172.21.15.172:0/657175799 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204014050 con 0x55e01baf0bb0 2024-09-10T08:46:02.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.210+0000 7fc200ff9640 1 -- 172.21.15.172:0/657175799 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2040141f0 con 0x55e01baf0bb0 2024-09-10T08:46:02.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.210+0000 7fc200ff9640 1 -- 172.21.15.172:0/657175799 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204004000 con 0x55e01baf0bb0 2024-09-10T08:46:02.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.211+0000 7fc213599740 1 -- 172.21.15.172:0/657175799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01baf0bb0 msgr2=0x55e01bb0b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:02.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.211+0000 7fc213599740 1 --2- 172.21.15.172:0/657175799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01baf0bb0 0x55e01bb0b3d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc20400ae70 tx=0x7fc20400cf80 comp rx=0 tx=0).stop 2024-09-10T08:46:02.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.211+0000 7fc213599740 1 -- 172.21.15.172:0/657175799 shutdown_connections 2024-09-10T08:46:02.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.211+0000 7fc213599740 1 --2- 172.21.15.172:0/657175799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01baf0bb0 0x55e01bb0b3d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.211+0000 7fc213599740 1 --2- 172.21.15.172:0/657175799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01bae62f0 0x55e01bb0ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.211+0000 7fc213599740 1 --2- 172.21.15.172:0/657175799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 0x55e01baed090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.211+0000 7fc213599740 1 -- 172.21.15.172:0/657175799 >> 172.21.15.172:0/657175799 conn(0x55e01bae1f60 msgr2=0x55e01bae2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:02.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.211+0000 7fc213599740 1 -- 172.21.15.172:0/657175799 shutdown_connections 2024-09-10T08:46:02.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.211+0000 7fc213599740 1 -- 172.21.15.172:0/657175799 wait complete. 2024-09-10T08:46:02.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.211+0000 7fc213599740 1 -- start start 2024-09-10T08:46:02.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.211+0000 7fc213599740 1 Processor -- start 2024-09-10T08:46:02.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.211+0000 7fc213599740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 0x55e01baea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.211+0000 7fc213599740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01bae62f0 0x55e01baea720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.211+0000 7fc213599740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01baf0bb0 0x55e01bb09e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.211+0000 7fc213599740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e01bb02a30 con 0x55e01bae62f0 2024-09-10T08:46:02.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.211+0000 7fc213599740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e01bb02360 con 0x55e01baf0bb0 2024-09-10T08:46:02.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.211+0000 7fc213599740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e01bb03100 con 0x55e01bae59d0 2024-09-10T08:46:02.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc213599740 10 client.0 _ll_register_callbacks cb 0x7fc1f40058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:02.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc203fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01bae62f0 0x55e01baea720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc203fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01bae62f0 0x55e01baea720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57076/0 (socket says 172.21.15.172:57076) 2024-09-10T08:46:02.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc203fff640 1 -- 172.21.15.172:0/3358979648 learned_addr learned my addr 172.21.15.172:0/3358979648 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:02.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc20ad9d640 1 --2- 172.21.15.172:0/3358979648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 0x55e01baea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc20b59e640 1 --2- 172.21.15.172:0/3358979648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01baf0bb0 0x55e01bb09e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc203fff640 1 -- 172.21.15.172:0/3358979648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 msgr2=0x55e01baea1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:02.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc203fff640 1 --2- 172.21.15.172:0/3358979648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 0x55e01baea1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc203fff640 1 -- 172.21.15.172:0/3358979648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01baf0bb0 msgr2=0x55e01bb09e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:02.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc203fff640 1 --2- 172.21.15.172:0/3358979648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01baf0bb0 0x55e01bb09e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc203fff640 1 -- 172.21.15.172:0/3358979648 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e01bb0a340 con 0x55e01bae62f0 2024-09-10T08:46:02.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc20b59e640 1 --2- 172.21.15.172:0/3358979648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01baf0bb0 0x55e01bb09e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:02.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc203fff640 1 --2- 172.21.15.172:0/3358979648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01bae62f0 0x55e01baea720 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc1f409b5a0 tx=0x7fc1f40cf760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ab88b78c8cb687 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:02.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc1f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:02.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc1f27fc640 1 -- 172.21.15.172:0/3358979648 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f401d040 con 0x55e01bae62f0 2024-09-10T08:46:02.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc1f27fc640 1 -- 172.21.15.172:0/3358979648 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fc1f4097040 con 0x55e01bae62f0 2024-09-10T08:46:02.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc213599740 1 -- 172.21.15.172:0/3358979648 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2040141f0 con 0x55e01bae62f0 2024-09-10T08:46:02.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc1f27fc640 1 -- 172.21.15.172:0/3358979648 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc1f409b200 con 0x55e01bae62f0 2024-09-10T08:46:02.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc213599740 10 client.15117 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:02.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.212+0000 7fc213599740 1 -- 172.21.15.172:0/3358979648 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e01bb0a340 con 0x55e01bae62f0 2024-09-10T08:46:02.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.213+0000 7fc1f27fc640 1 -- 172.21.15.172:0/3358979648 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2040141f0 con 0x55e01bae62f0 2024-09-10T08:46:02.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.213+0000 7fc1f27fc640 1 -- 172.21.15.172:0/3358979648 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fc1f409acf0 con 0x55e01bae62f0 2024-09-10T08:46:02.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.213+0000 7fc213599740 10 client.15117 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:02.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.213+0000 7fc213599740 10 client.15117 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:46:02.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.213+0000 7fc213599740 20 client.15117 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:02.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.213+0000 7fc213599740 1 --2- 172.21.15.172:0/3358979648 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e01bb6ff30 0x55e01bb6ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:02.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.213+0000 7fc213599740 4 client.15117 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:46:02.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.213+0000 7fc213599740 1 -- 172.21.15.172:0/3358979648 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "status"}) -- 0x55e01b9d00e0 con 0x55e01bb6ff30 2024-09-10T08:46:02.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.213+0000 7fc20ad9d640 1 --2- 172.21.15.172:0/3358979648 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e01bb6ff30 0x55e01bb6ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:02.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.213+0000 7fc1f27fc640 10 client.15117 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:46:02.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.213+0000 7fc20ad9d640 1 --2- 172.21.15.172:0/3358979648 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e01bb6ff30 0x55e01bb6ec40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a00a1f1d70f08df server_cookie=2ce45b9a54e33b82 in_seq=0 out_seq=0 2024-09-10T08:46:02.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.214+0000 7fc1f27fc640 1 -- 172.21.15.172:0/3358979648 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55e01b9d00e0 con 0x55e01bb6ff30 2024-09-10T08:46:02.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.214+0000 7fc1f27fc640 10 client.15117 handle_command_reply: tid=0 2024-09-10T08:46:02.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.214+0000 7fc213599740 1 client.15117 shutdown 2024-09-10T08:46:02.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.214+0000 7fc213599740 1 -- 172.21.15.172:0/3358979648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01bae62f0 msgr2=0x55e01baea720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:02.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.214+0000 7fc213599740 1 --2- 172.21.15.172:0/3358979648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01bae62f0 0x55e01baea720 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc1f409b5a0 tx=0x7fc1f40cf760 comp rx=0 tx=0).stop 2024-09-10T08:46:02.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.214+0000 7fc213599740 1 -- 172.21.15.172:0/3358979648 shutdown_connections 2024-09-10T08:46:02.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.214+0000 7fc213599740 1 --2- 172.21.15.172:0/3358979648 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e01bb6ff30 0x55e01bb6ec40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.214+0000 7fc213599740 1 --2- 172.21.15.172:0/3358979648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e01baf0bb0 0x55e01bb09e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.214+0000 7fc213599740 1 --2- 172.21.15.172:0/3358979648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e01bae62f0 0x55e01baea720 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.214+0000 7fc213599740 1 --2- 172.21.15.172:0/3358979648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e01bae59d0 0x55e01baea1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.214+0000 7fc213599740 1 -- 172.21.15.172:0/3358979648 >> 172.21.15.172:0/3358979648 conn(0x55e01bae1f60 msgr2=0x55e01baec0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:02.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.214+0000 7fc1f27fc640 0 client.15117 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:46:02.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.214+0000 7fc213599740 1 -- 172.21.15.172:0/3358979648 shutdown_connections 2024-09-10T08:46:02.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.214+0000 7fc213599740 1 -- 172.21.15.172:0/3358979648 wait complete. 2024-09-10T08:46:02.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.214+0000 7fc213599740 20 client.15117 trim_cache size 0 max 16384 2024-09-10T08:46:02.221 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:46:02.221 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:46:02.221 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 2, 2024-09-10T08:46:02.221 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 24457, 2024-09-10T08:46:02.222 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-09-10T08:46:02.222 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-09-10T08:46:02.222 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-09-10T08:46:02.222 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 69.957287244, 2024-09-10T08:46:02.222 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 28, 2024-09-10T08:46:02.222 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 90, 2024-09-10T08:46:02.222 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 88, 2024-09-10T08:46:02.222 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 129.57001090899999 2024-09-10T08:46:02.222 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:46:02.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.214+0000 7fc2111c0640 1 -- 172.21.15.172:0/1563062078 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc1f80a2c10 msgr2=0x7fc1f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:02.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.215+0000 7fc2111c0640 1 --2- 172.21.15.172:0/1563062078 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc1f80a2c10 0x7fc1f80a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc204011cb0 tx=0x7fc204016000 comp rx=0 tx=0).stop 2024-09-10T08:46:02.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.215+0000 7fc2111c0640 1 -- 172.21.15.172:0/1563062078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc20c103660 msgr2=0x7fc20c198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:02.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.215+0000 7fc2111c0640 1 --2- 172.21.15.172:0/1563062078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc20c103660 0x7fc20c198890 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400c940 tx=0x7fc1f400ce00 comp rx=0 tx=0).stop 2024-09-10T08:46:02.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.215+0000 7fc2111c0640 1 -- 172.21.15.172:0/1563062078 shutdown_connections 2024-09-10T08:46:02.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.215+0000 7fc2111c0640 1 --2- 172.21.15.172:0/1563062078 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc1f80a2c10 0x7fc1f80a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.215+0000 7fc2111c0640 1 --2- 172.21.15.172:0/1563062078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc20c10c170 0x7fc20c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.215+0000 7fc2111c0640 1 --2- 172.21.15.172:0/1563062078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc20c103660 0x7fc20c198890 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.215+0000 7fc2111c0640 1 --2- 172.21.15.172:0/1563062078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc20c102d40 0x7fc20c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:02.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.215+0000 7fc2111c0640 1 -- 172.21.15.172:0/1563062078 >> 172.21.15.172:0/1563062078 conn(0x7fc20c0fc7a0 msgr2=0x7fc20c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:02.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.215+0000 7fc2111c0640 1 -- 172.21.15.172:0/1563062078 shutdown_connections 2024-09-10T08:46:02.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:02.215+0000 7fc2111c0640 1 -- 172.21.15.172:0/1563062078 wait complete. 2024-09-10T08:46:02.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:02 smithi174 ceph-mon[34068]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T08:46:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:02 smithi191 ceph-mon[35314]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T08:46:02.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:02 smithi172 ceph-mon[30854]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T08:46:02.918 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T08:46:03.163 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:03.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.659+0000 7fa6a9d4d640 1 -- 172.21.15.172:0/4109052591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6a4102790 msgr2=0x7fa6a41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:03.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.659+0000 7fa6a9d4d640 1 --2- 172.21.15.172:0/4109052591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6a4102790 0x7fa6a41067e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa698009900 tx=0x7fa69802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.659+0000 7fa6a9d4d640 1 -- 172.21.15.172:0/4109052591 shutdown_connections 2024-09-10T08:46:03.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.659+0000 7fa6a9d4d640 1 --2- 172.21.15.172:0/4109052591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6a4106f10 0x7fa6a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.659+0000 7fa6a9d4d640 1 --2- 172.21.15.172:0/4109052591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6a4102790 0x7fa6a41067e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.659+0000 7fa6a9d4d640 1 --2- 172.21.15.172:0/4109052591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6a4101de0 0x7fa6a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.659+0000 7fa6a9d4d640 1 -- 172.21.15.172:0/4109052591 >> 172.21.15.172:0/4109052591 conn(0x7fa6a40fd8c0 msgr2=0x7fa6a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:03.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.659+0000 7fa6a9d4d640 1 -- 172.21.15.172:0/4109052591 shutdown_connections 2024-09-10T08:46:03.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.659+0000 7fa6a9d4d640 1 -- 172.21.15.172:0/4109052591 wait complete. 2024-09-10T08:46:03.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.660+0000 7fa6a9d4d640 1 Processor -- start 2024-09-10T08:46:03.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.660+0000 7fa6a9d4d640 1 -- start start 2024-09-10T08:46:03.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.660+0000 7fa6a9d4d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6a4101de0 0x7fa6a4198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.660+0000 7fa6a9d4d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6a4102790 0x7fa6a4198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.660+0000 7fa6a9d4d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6a4106f10 0x7fa6a419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.660+0000 7fa6a9d4d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a4111e40 con 0x7fa6a4106f10 2024-09-10T08:46:03.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.660+0000 7fa6a9d4d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a4111cc0 con 0x7fa6a4102790 2024-09-10T08:46:03.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.660+0000 7fa6a9d4d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a4111fc0 con 0x7fa6a4101de0 2024-09-10T08:46:03.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.660+0000 7fa6a3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6a4106f10 0x7fa6a419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:03.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.660+0000 7fa6a3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6a4106f10 0x7fa6a419cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57098/0 (socket says 172.21.15.172:57098) 2024-09-10T08:46:03.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.660+0000 7fa6a3fff640 1 -- 172.21.15.172:0/1443270337 learned_addr learned my addr 172.21.15.172:0/1443270337 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:03.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.661+0000 7fa6a37fe640 1 --2- 172.21.15.172:0/1443270337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6a4101de0 0x7fa6a4198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:03.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.661+0000 7fa6a2ffd640 1 --2- 172.21.15.172:0/1443270337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6a4102790 0x7fa6a4198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:03.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.661+0000 7fa6a3fff640 1 -- 172.21.15.172:0/1443270337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6a4101de0 msgr2=0x7fa6a4198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:03.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.661+0000 7fa6a3fff640 1 --2- 172.21.15.172:0/1443270337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6a4101de0 0x7fa6a4198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.661+0000 7fa6a3fff640 1 -- 172.21.15.172:0/1443270337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6a4102790 msgr2=0x7fa6a4198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:03.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.661+0000 7fa6a3fff640 1 --2- 172.21.15.172:0/1443270337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6a4102790 0x7fa6a4198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.661+0000 7fa6a3fff640 1 -- 172.21.15.172:0/1443270337 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a419d2d0 con 0x7fa6a4106f10 2024-09-10T08:46:03.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.661+0000 7fa6a2ffd640 1 --2- 172.21.15.172:0/1443270337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6a4102790 0x7fa6a4198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:03.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.661+0000 7fa6a3fff640 1 --2- 172.21.15.172:0/1443270337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6a4106f10 0x7fa6a419cbf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00b970 tx=0x7fa69c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:03.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.661+0000 7fa6a0ff9640 1 -- 172.21.15.172:0/1443270337 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c019070 con 0x7fa6a4106f10 2024-09-10T08:46:03.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.661+0000 7fa6a0ff9640 1 -- 172.21.15.172:0/1443270337 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa69c014440 con 0x7fa6a4106f10 2024-09-10T08:46:03.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.661+0000 7fa6a9d4d640 1 -- 172.21.15.172:0/1443270337 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a419d5c0 con 0x7fa6a4106f10 2024-09-10T08:46:03.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.661+0000 7fa6a0ff9640 1 -- 172.21.15.172:0/1443270337 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c0075d0 con 0x7fa6a4106f10 2024-09-10T08:46:03.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.661+0000 7fa6a9d4d640 1 -- 172.21.15.172:0/1443270337 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a41a4ea0 con 0x7fa6a4106f10 2024-09-10T08:46:03.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.663+0000 7fa6a0ff9640 1 -- 172.21.15.172:0/1443270337 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa69c01c070 con 0x7fa6a4106f10 2024-09-10T08:46:03.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6ac126740 1 Processor -- start 2024-09-10T08:46:03.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6ac126740 1 -- start start 2024-09-10T08:46:03.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6ac126740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2e5890 0x5578da2e5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6ac126740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2e61b0 0x5578da2f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6ac126740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2f4fc0 0x5578da2f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6ac126740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5578da24c090 con 0x5578da2e5890 2024-09-10T08:46:03.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6ac126740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578da1f3a70 con 0x5578da2f4fc0 2024-09-10T08:46:03.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6ac126740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5578da248050 con 0x5578da2e61b0 2024-09-10T08:46:03.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6a0ff9640 1 --2- 172.21.15.172:0/1443270337 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa68c0a2c90 0x7fa68c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6a2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2e5890 0x5578da2e5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:03.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6a2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2e5890 0x5578da2e5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57100/0 (socket says 172.21.15.172:57100) 2024-09-10T08:46:03.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6a2ffd640 1 -- 172.21.15.172:0/3388344097 learned_addr learned my addr 172.21.15.172:0/3388344097 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:03.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6a0ff9640 1 -- 172.21.15.172:0/1443270337 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fa69c0ce940 con 0x7fa6a4106f10 2024-09-10T08:46:03.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6a3fff640 1 --2- 172.21.15.172:0/1443270337 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa68c0a2c90 0x7fa68c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:03.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6a37fe640 1 --2- 172.21.15.172:0/3388344097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2e61b0 0x5578da2f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:03.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6a2ffd640 1 -- 172.21.15.172:0/3388344097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2e61b0 msgr2=0x5578da2f4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:03.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6a2ffd640 1 --2- 172.21.15.172:0/3388344097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2e61b0 0x5578da2f4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6a2ffd640 1 -- 172.21.15.172:0/3388344097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2f4fc0 msgr2=0x5578da2f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:03.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6a2ffd640 1 --2- 172.21.15.172:0/3388344097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2f4fc0 0x5578da2f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.664+0000 7fa6a2ffd640 1 -- 172.21.15.172:0/3388344097 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578da200130 con 0x5578da2e5890 2024-09-10T08:46:03.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.665+0000 7fa6a2ffd640 1 --2- 172.21.15.172:0/3388344097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2e5890 0x5578da2e5c70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa69802a910 tx=0x7fa698031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca882104344cbca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:03.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.665+0000 7fa6a3fff640 1 --2- 172.21.15.172:0/1443270337 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa68c0a2c90 0x7fa68c0a5150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0cc0f0 tx=0x7fa69c0bb040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:03.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.665+0000 7fa689ffb640 1 -- 172.21.15.172:0/3388344097 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698035050 con 0x5578da2e5890 2024-09-10T08:46:03.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.665+0000 7fa689ffb640 1 -- 172.21.15.172:0/3388344097 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6980316a0 con 0x5578da2e5890 2024-09-10T08:46:03.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.665+0000 7fa689ffb640 1 -- 172.21.15.172:0/3388344097 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6980047a0 con 0x5578da2e5890 2024-09-10T08:46:03.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.665+0000 7fa6ac126740 1 -- 172.21.15.172:0/3388344097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2e5890 msgr2=0x5578da2e5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:03.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.665+0000 7fa6ac126740 1 --2- 172.21.15.172:0/3388344097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2e5890 0x5578da2e5c70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa69802a910 tx=0x7fa698031dc0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.665+0000 7fa6ac126740 1 -- 172.21.15.172:0/3388344097 shutdown_connections 2024-09-10T08:46:03.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.665+0000 7fa6ac126740 1 --2- 172.21.15.172:0/3388344097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2f4fc0 0x5578da2f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.665+0000 7fa6ac126740 1 --2- 172.21.15.172:0/3388344097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2e61b0 0x5578da2f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.665+0000 7fa6ac126740 1 --2- 172.21.15.172:0/3388344097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2e5890 0x5578da2e5c70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.665+0000 7fa6ac126740 1 -- 172.21.15.172:0/3388344097 >> 172.21.15.172:0/3388344097 conn(0x5578da2e1f60 msgr2=0x5578da2e2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:03.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.665+0000 7fa6ac126740 1 -- 172.21.15.172:0/3388344097 shutdown_connections 2024-09-10T08:46:03.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.666+0000 7fa6ac126740 1 -- 172.21.15.172:0/3388344097 wait complete. 2024-09-10T08:46:03.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.666+0000 7fa6ac126740 1 -- start start 2024-09-10T08:46:03.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.666+0000 7fa6ac126740 1 Processor -- start 2024-09-10T08:46:03.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.666+0000 7fa6ac126740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2e5890 0x5578da2f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.666+0000 7fa6ac126740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2e61b0 0x5578da2f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.666+0000 7fa6ac126740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2f4fc0 0x5578da2f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.666+0000 7fa6ac126740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5578da2f7b30 con 0x5578da2f4fc0 2024-09-10T08:46:03.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.666+0000 7fa6ac126740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578da206f20 con 0x5578da2e61b0 2024-09-10T08:46:03.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.666+0000 7fa6ac126740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5578da2f7cb0 con 0x5578da2e5890 2024-09-10T08:46:03.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.666+0000 7fa6ac126740 10 client.0 _ll_register_callbacks cb 0x5578da1d0030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:03.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.666+0000 7fa6a2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2f4fc0 0x5578da2f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:03.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.666+0000 7fa6a2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2f4fc0 0x5578da2f21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57114/0 (socket says 172.21.15.172:57114) 2024-09-10T08:46:03.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.666+0000 7fa6a2ffd640 1 -- 172.21.15.172:0/3313243875 learned_addr learned my addr 172.21.15.172:0/3313243875 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:03.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.666+0000 7fa6a2ffd640 1 --2- 172.21.15.172:0/3313243875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2e5890 0x5578da2f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:03.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.667+0000 7fa6a37fe640 1 --2- 172.21.15.172:0/3313243875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2e61b0 0x5578da2f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:03.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.667+0000 7fa6a2ffd640 1 -- 172.21.15.172:0/3313243875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2e5890 msgr2=0x5578da2f48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:03.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.667+0000 7fa6a2ffd640 1 --2- 172.21.15.172:0/3313243875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2e5890 0x5578da2f48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.667+0000 7fa6a2ffd640 1 -- 172.21.15.172:0/3313243875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2e61b0 msgr2=0x5578da2f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:03.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.667+0000 7fa6a2ffd640 1 --2- 172.21.15.172:0/3313243875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2e61b0 0x5578da2f1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.667+0000 7fa6a2ffd640 1 -- 172.21.15.172:0/3313243875 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d9fbd2b0 con 0x5578da2f4fc0 2024-09-10T08:46:03.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.667+0000 7fa6a2ffd640 1 --2- 172.21.15.172:0/3313243875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2f4fc0 0x5578da2f21c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa698031fd0 tx=0x7fa69802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8bd571cca6c39e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:03.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.667+0000 7fa682ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:03.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.667+0000 7fa682ffd640 1 -- 172.21.15.172:0/3313243875 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698035070 con 0x5578da2f4fc0 2024-09-10T08:46:03.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.667+0000 7fa6ac126740 1 -- 172.21.15.172:0/3313243875 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6980316a0 con 0x5578da2f4fc0 2024-09-10T08:46:03.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.667+0000 7fa682ffd640 1 -- 172.21.15.172:0/3313243875 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fa698004be0 con 0x5578da2f4fc0 2024-09-10T08:46:03.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.667+0000 7fa682ffd640 1 -- 172.21.15.172:0/3313243875 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa698058050 con 0x5578da2f4fc0 2024-09-10T08:46:03.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.667+0000 7fa6ac126740 10 client.15129 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:03.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.667+0000 7fa6ac126740 1 -- 172.21.15.172:0/3313243875 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578da303540 con 0x5578da2f4fc0 2024-09-10T08:46:03.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.667+0000 7fa682ffd640 1 -- 172.21.15.172:0/3313243875 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6980316a0 con 0x5578da2f4fc0 2024-09-10T08:46:03.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.668+0000 7fa682ffd640 1 -- 172.21.15.172:0/3313243875 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fa698046420 con 0x5578da2f4fc0 2024-09-10T08:46:03.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.668+0000 7fa6ac126740 10 client.15129 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:03.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.668+0000 7fa6ac126740 10 client.15129 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:46:03.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.668+0000 7fa6ac126740 20 client.15129 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:03.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.668+0000 7fa6ac126740 1 --2- 172.21.15.172:0/3313243875 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5578da304890 0x5578da306a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.668+0000 7fa6ac126740 4 client.15129 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:03.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.668+0000 7fa6ac126740 1 -- 172.21.15.172:0/3313243875 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578da259600 con 0x5578da304890 2024-09-10T08:46:03.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.668+0000 7fa6a37fe640 1 --2- 172.21.15.172:0/3313243875 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5578da304890 0x5578da306a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:03.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.668+0000 7fa682ffd640 10 client.15129 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:46:03.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.668+0000 7fa6a37fe640 1 --2- 172.21.15.172:0/3313243875 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5578da304890 0x5578da306a80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a05aaf4bc1ecffff server_cookie=e9eaaf6e359fc1df in_seq=0 out_seq=0 2024-09-10T08:46:03.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.669+0000 7fa682ffd640 1 -- 172.21.15.172:0/3313243875 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578da259600 con 0x5578da304890 2024-09-10T08:46:03.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.669+0000 7fa682ffd640 10 client.15129 handle_command_reply: tid=0 2024-09-10T08:46:03.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.669+0000 7fa6ac126740 1 client.15129 shutdown 2024-09-10T08:46:03.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.669+0000 7fa6ac126740 1 -- 172.21.15.172:0/3313243875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2f4fc0 msgr2=0x5578da2f21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:03.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.669+0000 7fa6ac126740 1 --2- 172.21.15.172:0/3313243875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2f4fc0 0x5578da2f21c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa698031fd0 tx=0x7fa69802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.669+0000 7fa6ac126740 1 -- 172.21.15.172:0/3313243875 shutdown_connections 2024-09-10T08:46:03.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.669+0000 7fa6ac126740 1 --2- 172.21.15.172:0/3313243875 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5578da304890 0x5578da306a80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.669+0000 7fa6ac126740 1 --2- 172.21.15.172:0/3313243875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2f4fc0 0x5578da2f21c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.669+0000 7fa6ac126740 1 --2- 172.21.15.172:0/3313243875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2e61b0 0x5578da2f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.669+0000 7fa6ac126740 1 --2- 172.21.15.172:0/3313243875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2e5890 0x5578da2f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.669+0000 7fa682ffd640 0 client.15129 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:46:03.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.669+0000 7fa6ac126740 1 -- 172.21.15.172:0/3313243875 >> 172.21.15.172:0/3313243875 conn(0x5578da2e1f60 msgr2=0x5578da2e7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:03.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.670+0000 7fa6ac126740 1 -- 172.21.15.172:0/3313243875 shutdown_connections 2024-09-10T08:46:03.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.670+0000 7fa6ac126740 1 -- 172.21.15.172:0/3313243875 wait complete. 2024-09-10T08:46:03.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.670+0000 7fa6ac126740 20 client.15129 trim_cache size 0 max 16384 2024-09-10T08:46:03.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.687+0000 7fa6ac126740 1 Processor -- start 2024-09-10T08:46:03.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.687+0000 7fa6ac126740 1 -- start start 2024-09-10T08:46:03.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.687+0000 7fa6ac126740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2e5890 0x5578da306a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.687+0000 7fa6ac126740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2e61b0 0x5578da2f0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.687+0000 7fa6ac126740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2f4fc0 0x5578da2f0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.687+0000 7fa6ac126740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5578da304dc0 con 0x5578da2e61b0 2024-09-10T08:46:03.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.687+0000 7fa6ac126740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578da305490 con 0x5578da2e5890 2024-09-10T08:46:03.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.687+0000 7fa6ac126740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5578da305b60 con 0x5578da2f4fc0 2024-09-10T08:46:03.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.687+0000 7fa6a37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2e61b0 0x5578da2f0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:03.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.687+0000 7fa6a37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2e61b0 0x5578da2f0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57124/0 (socket says 172.21.15.172:57124) 2024-09-10T08:46:03.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.687+0000 7fa6a37fe640 1 -- 172.21.15.172:0/3034994477 learned_addr learned my addr 172.21.15.172:0/3034994477 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:03.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.688+0000 7fa6a37fe640 1 -- 172.21.15.172:0/3034994477 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2f4fc0 msgr2=0x5578da2f0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:03.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.688+0000 7fa6a2ffd640 1 --2- 172.21.15.172:0/3034994477 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2f4fc0 0x5578da2f0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:03.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.688+0000 7fa6a37fe640 1 --2- 172.21.15.172:0/3034994477 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2f4fc0 0x5578da2f0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.688+0000 7fa6a37fe640 1 -- 172.21.15.172:0/3034994477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2e5890 msgr2=0x5578da306a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:03.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.688+0000 7fa6a37fe640 1 --2- 172.21.15.172:0/3034994477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2e5890 0x5578da306a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.688+0000 7fa6a37fe640 1 -- 172.21.15.172:0/3034994477 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578da2e5fe0 con 0x5578da2e61b0 2024-09-10T08:46:03.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.688+0000 7fa6a2ffd640 1 --2- 172.21.15.172:0/3034994477 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2f4fc0 0x5578da2f0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:03.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.688+0000 7fa6a37fe640 1 --2- 172.21.15.172:0/3034994477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2e61b0 0x5578da2f0530 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x5578da307220 tx=0x7fa694014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311c15c991451e6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:03.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.688+0000 7fa6897fa640 1 -- 172.21.15.172:0/3034994477 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6940140d0 con 0x5578da2e61b0 2024-09-10T08:46:03.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.688+0000 7fa6897fa640 1 -- 172.21.15.172:0/3034994477 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa694014270 con 0x5578da2e61b0 2024-09-10T08:46:03.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.688+0000 7fa6897fa640 1 -- 172.21.15.172:0/3034994477 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694004e50 con 0x5578da2e61b0 2024-09-10T08:46:03.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.689+0000 7fa6ac126740 1 -- 172.21.15.172:0/3034994477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2e61b0 msgr2=0x5578da2f0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:03.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.689+0000 7fa6ac126740 1 --2- 172.21.15.172:0/3034994477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2e61b0 0x5578da2f0530 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x5578da307220 tx=0x7fa694014840 comp rx=0 tx=0).stop 2024-09-10T08:46:03.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.689+0000 7fa6ac126740 1 -- 172.21.15.172:0/3034994477 shutdown_connections 2024-09-10T08:46:03.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.689+0000 7fa6ac126740 1 --2- 172.21.15.172:0/3034994477 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2f4fc0 0x5578da2f0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.689+0000 7fa6ac126740 1 --2- 172.21.15.172:0/3034994477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2e61b0 0x5578da2f0530 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.689+0000 7fa6ac126740 1 --2- 172.21.15.172:0/3034994477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2e5890 0x5578da306a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.689+0000 7fa6ac126740 1 -- 172.21.15.172:0/3034994477 >> 172.21.15.172:0/3034994477 conn(0x5578da2e1f60 msgr2=0x5578da2dcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:03.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.689+0000 7fa6ac126740 1 -- 172.21.15.172:0/3034994477 shutdown_connections 2024-09-10T08:46:03.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.689+0000 7fa6ac126740 1 -- 172.21.15.172:0/3034994477 wait complete. 2024-09-10T08:46:03.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.689+0000 7fa6ac126740 1 -- start start 2024-09-10T08:46:03.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.689+0000 7fa6ac126740 1 Processor -- start 2024-09-10T08:46:03.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.689+0000 7fa6ac126740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2e5890 0x5578da3084e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6ac126740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2e61b0 0x5578da308a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6ac126740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2f4fc0 0x5578da30eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6ac126740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5578da306900 con 0x5578da2f4fc0 2024-09-10T08:46:03.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6ac126740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578da306230 con 0x5578da2e5890 2024-09-10T08:46:03.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6ac126740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5578da306fd0 con 0x5578da2e61b0 2024-09-10T08:46:03.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6ac126740 10 client.0 _ll_register_callbacks cb 0x7fa698031d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:03.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6a2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2f4fc0 0x5578da30eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:03.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6a2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2f4fc0 0x5578da30eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57136/0 (socket says 172.21.15.172:57136) 2024-09-10T08:46:03.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6a2ffd640 1 -- 172.21.15.172:0/2321371404 learned_addr learned my addr 172.21.15.172:0/2321371404 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:03.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6a2ffd640 1 --2- 172.21.15.172:0/2321371404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2e5890 0x5578da3084e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:03.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6a37fe640 1 --2- 172.21.15.172:0/2321371404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2e61b0 0x5578da308a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:03.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6a2ffd640 1 -- 172.21.15.172:0/2321371404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2e61b0 msgr2=0x5578da308a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:03.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6a2ffd640 1 --2- 172.21.15.172:0/2321371404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2e61b0 0x5578da308a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6a2ffd640 1 -- 172.21.15.172:0/2321371404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2e5890 msgr2=0x5578da3084e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:03.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6a2ffd640 1 --2- 172.21.15.172:0/2321371404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2e5890 0x5578da3084e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6a2ffd640 1 -- 172.21.15.172:0/2321371404 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578da30f2a0 con 0x5578da2f4fc0 2024-09-10T08:46:03.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6a2ffd640 1 --2- 172.21.15.172:0/2321371404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2f4fc0 0x5578da30eb60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa69802a910 tx=0x7fa69802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110e21e5a96e8ffc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:03.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa682ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:03.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa682ffd640 1 -- 172.21.15.172:0/2321371404 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698035070 con 0x5578da2f4fc0 2024-09-10T08:46:03.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa682ffd640 1 -- 172.21.15.172:0/2321371404 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fa698046420 con 0x5578da2f4fc0 2024-09-10T08:46:03.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.690+0000 7fa6ac126740 1 -- 172.21.15.172:0/2321371404 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa694014270 con 0x5578da2f4fc0 2024-09-10T08:46:03.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.691+0000 7fa682ffd640 1 -- 172.21.15.172:0/2321371404 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa698046a60 con 0x5578da2f4fc0 2024-09-10T08:46:03.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.691+0000 7fa6ac126740 10 client.15135 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:03.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.691+0000 7fa6ac126740 1 -- 172.21.15.172:0/2321371404 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578da30d2c0 con 0x5578da2f4fc0 2024-09-10T08:46:03.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.691+0000 7fa682ffd640 1 -- 172.21.15.172:0/2321371404 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa694014270 con 0x5578da2f4fc0 2024-09-10T08:46:03.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.691+0000 7fa682ffd640 1 -- 172.21.15.172:0/2321371404 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fa698004020 con 0x5578da2f4fc0 2024-09-10T08:46:03.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.691+0000 7fa6ac126740 10 client.15135 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:03.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.691+0000 7fa6ac126740 10 client.15135 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:46:03.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.691+0000 7fa6ac126740 20 client.15135 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:03.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.691+0000 7fa6ac126740 1 --2- 172.21.15.172:0/2321371404 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5578da374240 0x5578da372f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:03.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.691+0000 7fa6ac126740 4 client.15135 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:46:03.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.691+0000 7fa6ac126740 1 -- 172.21.15.172:0/2321371404 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "status"}) -- 0x5578da253b10 con 0x5578da374240 2024-09-10T08:46:03.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.691+0000 7fa6a37fe640 1 --2- 172.21.15.172:0/2321371404 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5578da374240 0x5578da372f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:03.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.692+0000 7fa6a37fe640 1 --2- 172.21.15.172:0/2321371404 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5578da374240 0x5578da372f50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d7d3826bdf51 server_cookie=a2207fe0fed7cd88 in_seq=0 out_seq=0 2024-09-10T08:46:03.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.692+0000 7fa682ffd640 10 client.15135 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:46:03.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.692+0000 7fa682ffd640 1 -- 172.21.15.172:0/2321371404 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5578da253b10 con 0x5578da374240 2024-09-10T08:46:03.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.692+0000 7fa682ffd640 10 client.15135 handle_command_reply: tid=0 2024-09-10T08:46:03.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.692+0000 7fa6ac126740 1 client.15135 shutdown 2024-09-10T08:46:03.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.692+0000 7fa6ac126740 1 -- 172.21.15.172:0/2321371404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2f4fc0 msgr2=0x5578da30eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:03.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.692+0000 7fa6ac126740 1 --2- 172.21.15.172:0/2321371404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2f4fc0 0x5578da30eb60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa69802a910 tx=0x7fa69802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.692+0000 7fa6ac126740 1 -- 172.21.15.172:0/2321371404 shutdown_connections 2024-09-10T08:46:03.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.692+0000 7fa6ac126740 1 --2- 172.21.15.172:0/2321371404 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5578da374240 0x5578da372f50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.692+0000 7fa6ac126740 1 --2- 172.21.15.172:0/2321371404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578da2f4fc0 0x5578da30eb60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.692+0000 7fa682ffd640 0 client.15135 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:46:03.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.692+0000 7fa6ac126740 1 --2- 172.21.15.172:0/2321371404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578da2e61b0 0x5578da308a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.692+0000 7fa6ac126740 1 --2- 172.21.15.172:0/2321371404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578da2e5890 0x5578da3084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.692+0000 7fa6ac126740 1 -- 172.21.15.172:0/2321371404 >> 172.21.15.172:0/2321371404 conn(0x5578da2e1f60 msgr2=0x5578da2f04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:03.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.692+0000 7fa6ac126740 1 -- 172.21.15.172:0/2321371404 shutdown_connections 2024-09-10T08:46:03.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.692+0000 7fa6ac126740 1 -- 172.21.15.172:0/2321371404 wait complete. 2024-09-10T08:46:03.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.692+0000 7fa6ac126740 20 client.15135 trim_cache size 0 max 16384 2024-09-10T08:46:03.699 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:46:03.700 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:46:03.700 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 1, 2024-09-10T08:46:03.700 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 24458, 2024-09-10T08:46:03.700 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-09-10T08:46:03.700 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-09-10T08:46:03.700 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-09-10T08:46:03.700 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 72.443038580000007, 2024-09-10T08:46:03.700 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 28, 2024-09-10T08:46:03.700 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 90, 2024-09-10T08:46:03.700 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 88, 2024-09-10T08:46:03.700 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 133.71092034200001 2024-09-10T08:46:03.700 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:46:03.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.693+0000 7fa6a9d4d640 1 -- 172.21.15.172:0/1443270337 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa68c0a2c90 msgr2=0x7fa68c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:03.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.693+0000 7fa6a9d4d640 1 --2- 172.21.15.172:0/1443270337 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa68c0a2c90 0x7fa68c0a5150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0cc0f0 tx=0x7fa69c0bb040 comp rx=0 tx=0).stop 2024-09-10T08:46:03.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.693+0000 7fa6a9d4d640 1 -- 172.21.15.172:0/1443270337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6a4106f10 msgr2=0x7fa6a419cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:03.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.693+0000 7fa6a9d4d640 1 --2- 172.21.15.172:0/1443270337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6a4106f10 0x7fa6a419cbf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00b970 tx=0x7fa69c00bd70 comp rx=0 tx=0).stop 2024-09-10T08:46:03.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.693+0000 7fa6a9d4d640 1 -- 172.21.15.172:0/1443270337 shutdown_connections 2024-09-10T08:46:03.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.693+0000 7fa6a9d4d640 1 --2- 172.21.15.172:0/1443270337 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa68c0a2c90 0x7fa68c0a5150 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.693+0000 7fa6a9d4d640 1 --2- 172.21.15.172:0/1443270337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6a4106f10 0x7fa6a419cbf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.693+0000 7fa6a9d4d640 1 --2- 172.21.15.172:0/1443270337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6a4102790 0x7fa6a4198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.693+0000 7fa6a9d4d640 1 --2- 172.21.15.172:0/1443270337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6a4101de0 0x7fa6a4198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:03.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.693+0000 7fa6a9d4d640 1 -- 172.21.15.172:0/1443270337 >> 172.21.15.172:0/1443270337 conn(0x7fa6a40fd8c0 msgr2=0x7fa6a40fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:03.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.693+0000 7fa6a9d4d640 1 -- 172.21.15.172:0/1443270337 shutdown_connections 2024-09-10T08:46:03.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:03.693+0000 7fa6a9d4d640 1 -- 172.21.15.172:0/1443270337 wait complete. 2024-09-10T08:46:04.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:04 smithi172 ceph-mon[30854]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:04.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:04 smithi174 ceph-mon[34068]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:04.525 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T08:46:04.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:04 smithi191 ceph-mon[35314]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:04.783 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:05.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.270+0000 7f93a116e640 1 -- 172.21.15.172:0/29795163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f939c101420 msgr2=0x7f939c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:05.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.270+0000 7f93a116e640 1 --2- 172.21.15.172:0/29795163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f939c101420 0x7f939c10d950 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9394009900 tx=0x7f939402a410 comp rx=0 tx=0).stop 2024-09-10T08:46:05.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.270+0000 7f93a116e640 1 -- 172.21.15.172:0/29795163 shutdown_connections 2024-09-10T08:46:05.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.270+0000 7f93a116e640 1 --2- 172.21.15.172:0/29795163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f939c101420 0x7f939c10d950 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.270+0000 7f93a116e640 1 --2- 172.21.15.172:0/29795163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f939c100a80 0x7f939c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.270+0000 7f93a116e640 1 --2- 172.21.15.172:0/29795163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f939c106a80 0x7f939c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.270+0000 7f93a116e640 1 -- 172.21.15.172:0/29795163 >> 172.21.15.172:0/29795163 conn(0x7f939c0fc7a0 msgr2=0x7f939c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:05.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.270+0000 7f93a116e640 1 -- 172.21.15.172:0/29795163 shutdown_connections 2024-09-10T08:46:05.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.270+0000 7f93a116e640 1 -- 172.21.15.172:0/29795163 wait complete. 2024-09-10T08:46:05.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.270+0000 7f93a116e640 1 Processor -- start 2024-09-10T08:46:05.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.271+0000 7f93a116e640 1 -- start start 2024-09-10T08:46:05.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.271+0000 7f93a116e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f939c100a80 0x7f939c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.271+0000 7f93a116e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f939c101420 0x7f939c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.271+0000 7f93a116e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f939c106a80 0x7f939c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.271+0000 7f93a116e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f939c111e50 con 0x7f939c101420 2024-09-10T08:46:05.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.271+0000 7f93a116e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f939c111cd0 con 0x7f939c100a80 2024-09-10T08:46:05.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.271+0000 7f93a116e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f939c111fd0 con 0x7f939c106a80 2024-09-10T08:46:05.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.271+0000 7f939a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f939c101420 0x7f939c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.271+0000 7f939a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f939c101420 0x7f939c1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57154/0 (socket says 172.21.15.172:57154) 2024-09-10T08:46:05.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.271+0000 7f939a59c640 1 -- 172.21.15.172:0/3466512796 learned_addr learned my addr 172.21.15.172:0/3466512796 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:05.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.271+0000 7f939ad9d640 1 --2- 172.21.15.172:0/3466512796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f939c100a80 0x7f939c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.271+0000 7f939b59e640 1 --2- 172.21.15.172:0/3466512796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f939c106a80 0x7f939c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.271+0000 7f939a59c640 1 -- 172.21.15.172:0/3466512796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f939c106a80 msgr2=0x7f939c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:05.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.271+0000 7f939a59c640 1 --2- 172.21.15.172:0/3466512796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f939c106a80 0x7f939c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.271+0000 7f939a59c640 1 -- 172.21.15.172:0/3466512796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f939c100a80 msgr2=0x7f939c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:05.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.271+0000 7f939a59c640 1 --2- 172.21.15.172:0/3466512796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f939c100a80 0x7f939c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.271+0000 7f939a59c640 1 -- 172.21.15.172:0/3466512796 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939c19d320 con 0x7f939c101420 2024-09-10T08:46:05.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.272+0000 7f939a59c640 1 --2- 172.21.15.172:0/3466512796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f939c101420 0x7f939c1988b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f938c00d950 tx=0x7f938c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:05.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.272+0000 7f9383fff640 1 -- 172.21.15.172:0/3466512796 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c017070 con 0x7f939c101420 2024-09-10T08:46:05.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.272+0000 7f93a116e640 1 -- 172.21.15.172:0/3466512796 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939c19d610 con 0x7f939c101420 2024-09-10T08:46:05.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.272+0000 7f9383fff640 1 -- 172.21.15.172:0/3466512796 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f938c009c30 con 0x7f939c101420 2024-09-10T08:46:05.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.272+0000 7f9383fff640 1 -- 172.21.15.172:0/3466512796 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c00f620 con 0x7f939c101420 2024-09-10T08:46:05.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.272+0000 7f93a116e640 1 -- 172.21.15.172:0/3466512796 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f939c1a4ef0 con 0x7f939c101420 2024-09-10T08:46:05.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.274+0000 7f9383fff640 1 -- 172.21.15.172:0/3466512796 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f938c0109a0 con 0x7f939c101420 2024-09-10T08:46:05.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.274+0000 7f93a3547740 1 Processor -- start 2024-09-10T08:46:05.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f93a3547740 1 -- start start 2024-09-10T08:46:05.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f9383fff640 1 --2- 172.21.15.172:0/3466512796 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f93840a2c10 0x7f93840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f939b59e640 1 --2- 172.21.15.172:0/3466512796 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f93840a2c10 0x7f93840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f93a3547740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e135de0 0x55890e1361c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f93a3547740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e136700 0x55890e140a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f93a3547740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e140fc0 0x55890e1433b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f93a3547740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55890e098090 con 0x55890e135de0 2024-09-10T08:46:05.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f93a3547740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55890e03fa70 con 0x55890e140fc0 2024-09-10T08:46:05.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f9383fff640 1 -- 172.21.15.172:0/3466512796 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f938c0cf1c0 con 0x7f939c101420 2024-09-10T08:46:05.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f93a3547740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55890e094050 con 0x55890e136700 2024-09-10T08:46:05.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f939ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e135de0 0x55890e1361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f939ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e135de0 0x55890e1361c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57156/0 (socket says 172.21.15.172:57156) 2024-09-10T08:46:05.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f939ad9d640 1 -- 172.21.15.172:0/4174840340 learned_addr learned my addr 172.21.15.172:0/4174840340 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:05.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f939b59e640 1 --2- 172.21.15.172:0/4174840340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e140fc0 0x55890e1433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f939a59c640 1 --2- 172.21.15.172:0/4174840340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e136700 0x55890e140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f939ad9d640 1 -- 172.21.15.172:0/4174840340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e136700 msgr2=0x55890e140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:05.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f939ad9d640 1 --2- 172.21.15.172:0/4174840340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e136700 0x55890e140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f939ad9d640 1 -- 172.21.15.172:0/4174840340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e140fc0 msgr2=0x55890e1433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:05.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f939ad9d640 1 --2- 172.21.15.172:0/4174840340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e140fc0 0x55890e1433b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f939ad9d640 1 -- 172.21.15.172:0/4174840340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890e04c130 con 0x55890e135de0 2024-09-10T08:46:05.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f939a59c640 1 --2- 172.21.15.172:0/4174840340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e136700 0x55890e140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:05.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f939b59e640 1 --2- 172.21.15.172:0/4174840340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e140fc0 0x55890e1433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:05.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f939b59e640 1 --2- 172.21.15.172:0/3466512796 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f93840a2c10 0x7f93840a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9394031440 tx=0x7f93940314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:05.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.275+0000 7f939ad9d640 1 --2- 172.21.15.172:0/4174840340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e135de0 0x55890e1361c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f939000a4b0 tx=0x7f93900096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb78716332da3349 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:05.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.276+0000 7f93817fa640 1 -- 172.21.15.172:0/4174840340 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390018020 con 0x55890e135de0 2024-09-10T08:46:05.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.276+0000 7f93817fa640 1 -- 172.21.15.172:0/4174840340 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9390014cc0 con 0x55890e135de0 2024-09-10T08:46:05.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.276+0000 7f93817fa640 1 -- 172.21.15.172:0/4174840340 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390013cb0 con 0x55890e135de0 2024-09-10T08:46:05.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.276+0000 7f93a3547740 1 -- 172.21.15.172:0/4174840340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e135de0 msgr2=0x55890e1361c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:05.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.276+0000 7f93a3547740 1 --2- 172.21.15.172:0/4174840340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e135de0 0x55890e1361c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f939000a4b0 tx=0x7f93900096d0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.276+0000 7f93a3547740 1 -- 172.21.15.172:0/4174840340 shutdown_connections 2024-09-10T08:46:05.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.276+0000 7f93a3547740 1 --2- 172.21.15.172:0/4174840340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e140fc0 0x55890e1433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.276+0000 7f93a3547740 1 --2- 172.21.15.172:0/4174840340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e136700 0x55890e140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.276+0000 7f93a3547740 1 --2- 172.21.15.172:0/4174840340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e135de0 0x55890e1361c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.276+0000 7f93a3547740 1 -- 172.21.15.172:0/4174840340 >> 172.21.15.172:0/4174840340 conn(0x55890e132370 msgr2=0x55890e132750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:05.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.276+0000 7f93a3547740 1 -- 172.21.15.172:0/4174840340 shutdown_connections 2024-09-10T08:46:05.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.276+0000 7f93a3547740 1 -- 172.21.15.172:0/4174840340 wait complete. 2024-09-10T08:46:05.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f93a3547740 1 -- start start 2024-09-10T08:46:05.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f93a3547740 1 Processor -- start 2024-09-10T08:46:05.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f93a3547740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e135de0 0x55890e140830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f93a3547740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e136700 0x55890e13db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f93a3547740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e140fc0 0x55890e13e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f93a3547740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55890e143b90 con 0x55890e140fc0 2024-09-10T08:46:05.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f93a3547740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55890e052f20 con 0x55890e136700 2024-09-10T08:46:05.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f93a3547740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55890e143d10 con 0x55890e135de0 2024-09-10T08:46:05.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f93a3547740 10 client.0 _ll_register_callbacks cb 0x55890e01c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:05.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f939b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e140fc0 0x55890e13e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f939b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e140fc0 0x55890e13e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57162/0 (socket says 172.21.15.172:57162) 2024-09-10T08:46:05.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f939b59e640 1 -- 172.21.15.172:0/4046808879 learned_addr learned my addr 172.21.15.172:0/4046808879 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:05.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f939a59c640 1 --2- 172.21.15.172:0/4046808879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e136700 0x55890e13db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f939b59e640 1 -- 172.21.15.172:0/4046808879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e135de0 msgr2=0x55890e140830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:05.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f939b59e640 1 --2- 172.21.15.172:0/4046808879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e135de0 0x55890e140830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f939b59e640 1 -- 172.21.15.172:0/4046808879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e136700 msgr2=0x55890e13db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:05.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f939b59e640 1 --2- 172.21.15.172:0/4046808879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e136700 0x55890e13db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.277+0000 7f939b59e640 1 -- 172.21.15.172:0/4046808879 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55890de092b0 con 0x55890e140fc0 2024-09-10T08:46:05.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.278+0000 7f939b59e640 1 --2- 172.21.15.172:0/4046808879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e140fc0 0x55890e13e0c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9394033000 tx=0x7f939402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1290b3096971f1f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:05.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.278+0000 7f937eff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:05.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.278+0000 7f937eff5640 1 -- 172.21.15.172:0/4046808879 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394035050 con 0x55890e140fc0 2024-09-10T08:46:05.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.278+0000 7f937eff5640 1 -- 172.21.15.172:0/4046808879 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f9394005450 con 0x55890e140fc0 2024-09-10T08:46:05.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.278+0000 7f93a3547740 1 -- 172.21.15.172:0/4046808879 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9390014cc0 con 0x55890e140fc0 2024-09-10T08:46:05.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.278+0000 7f937eff5640 1 -- 172.21.15.172:0/4046808879 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9394057050 con 0x55890e140fc0 2024-09-10T08:46:05.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.278+0000 7f93a3547740 10 client.15147 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:05.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.278+0000 7f93a3547740 1 -- 172.21.15.172:0/4046808879 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55890e14f3f0 con 0x55890e140fc0 2024-09-10T08:46:05.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.278+0000 7f937eff5640 1 -- 172.21.15.172:0/4046808879 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9390014cc0 con 0x55890e140fc0 2024-09-10T08:46:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.278+0000 7f937eff5640 1 -- 172.21.15.172:0/4046808879 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f93940035f0 con 0x55890e140fc0 2024-09-10T08:46:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.278+0000 7f93a3547740 10 client.15147 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.278+0000 7f93a3547740 10 client.15147 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:46:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.278+0000 7f93a3547740 20 client.15147 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.278+0000 7f93a3547740 1 --2- 172.21.15.172:0/4046808879 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55890e151af0 0x55890e152f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.278+0000 7f93a3547740 4 client.15147 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.278+0000 7f93a3547740 1 -- 172.21.15.172:0/4046808879 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55890e0a5600 con 0x55890e151af0 2024-09-10T08:46:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.279+0000 7f939ad9d640 1 --2- 172.21.15.172:0/4046808879 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55890e151af0 0x55890e152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.279+0000 7f937eff5640 10 client.15147 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.279+0000 7f939ad9d640 1 --2- 172.21.15.172:0/4046808879 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55890e151af0 0x55890e152f60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8ea10c7ba430346 server_cookie=62e6ce843266ebd9 in_seq=0 out_seq=0 2024-09-10T08:46:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.280+0000 7f937eff5640 1 -- 172.21.15.172:0/4046808879 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55890e0a5600 con 0x55890e151af0 2024-09-10T08:46:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.280+0000 7f937eff5640 10 client.15147 handle_command_reply: tid=0 2024-09-10T08:46:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.280+0000 7f93a3547740 1 client.15147 shutdown 2024-09-10T08:46:05.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.280+0000 7f93a3547740 1 -- 172.21.15.172:0/4046808879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e140fc0 msgr2=0x55890e13e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.280+0000 7f93a3547740 1 --2- 172.21.15.172:0/4046808879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e140fc0 0x55890e13e0c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9394033000 tx=0x7f939402a410 comp rx=0 tx=0).stop 2024-09-10T08:46:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.280+0000 7f93a3547740 1 -- 172.21.15.172:0/4046808879 shutdown_connections 2024-09-10T08:46:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.280+0000 7f93a3547740 1 --2- 172.21.15.172:0/4046808879 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55890e151af0 0x55890e152f60 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.280+0000 7f93a3547740 1 --2- 172.21.15.172:0/4046808879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e140fc0 0x55890e13e0c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.280+0000 7f93a3547740 1 --2- 172.21.15.172:0/4046808879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e136700 0x55890e13db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.280+0000 7f93a3547740 1 --2- 172.21.15.172:0/4046808879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e135de0 0x55890e140830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.280+0000 7f93a3547740 1 -- 172.21.15.172:0/4046808879 >> 172.21.15.172:0/4046808879 conn(0x55890e132370 msgr2=0x55890e138050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.280+0000 7f937eff5640 0 client.15147 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.280+0000 7f93a3547740 1 -- 172.21.15.172:0/4046808879 shutdown_connections 2024-09-10T08:46:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.280+0000 7f93a3547740 1 -- 172.21.15.172:0/4046808879 wait complete. 2024-09-10T08:46:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.280+0000 7f93a3547740 20 client.15147 trim_cache size 0 max 16384 2024-09-10T08:46:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.297+0000 7f93a3547740 1 Processor -- start 2024-09-10T08:46:05.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.297+0000 7f93a3547740 1 -- start start 2024-09-10T08:46:05.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.297+0000 7f93a3547740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e135de0 0x55890e151420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.297+0000 7f93a3547740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e136700 0x55890e152890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.297+0000 7f93a3547740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e140fc0 0x55890e13db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.297+0000 7f93a3547740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55890e1512a0 con 0x55890e135de0 2024-09-10T08:46:05.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.297+0000 7f93a3547740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55890e151970 con 0x55890e140fc0 2024-09-10T08:46:05.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.297+0000 7f93a3547740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55890e152040 con 0x55890e136700 2024-09-10T08:46:05.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.298+0000 7f939ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e135de0 0x55890e151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.298+0000 7f939ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e135de0 0x55890e151420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57172/0 (socket says 172.21.15.172:57172) 2024-09-10T08:46:05.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.298+0000 7f939ad9d640 1 -- 172.21.15.172:0/1313616331 learned_addr learned my addr 172.21.15.172:0/1313616331 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:05.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.298+0000 7f939a59c640 1 --2- 172.21.15.172:0/1313616331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e136700 0x55890e152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.298+0000 7f939b59e640 1 --2- 172.21.15.172:0/1313616331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e140fc0 0x55890e13db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.298+0000 7f939ad9d640 1 -- 172.21.15.172:0/1313616331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e136700 msgr2=0x55890e152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.298+0000 7f939ad9d640 1 --2- 172.21.15.172:0/1313616331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e136700 0x55890e152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.298+0000 7f939ad9d640 1 -- 172.21.15.172:0/1313616331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e140fc0 msgr2=0x55890e13db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.298+0000 7f939ad9d640 1 --2- 172.21.15.172:0/1313616331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e140fc0 0x55890e13db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.298+0000 7f939ad9d640 1 -- 172.21.15.172:0/1313616331 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890e13ef00 con 0x55890e135de0 2024-09-10T08:46:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.298+0000 7f939a59c640 1 --2- 172.21.15.172:0/1313616331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e136700 0x55890e152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.298+0000 7f939ad9d640 1 --2- 172.21.15.172:0/1313616331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e135de0 0x55890e151420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9390013920 tx=0x7f9390004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7da7a49d1c45a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.298+0000 7f9380ff9640 1 -- 172.21.15.172:0/1313616331 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939002a070 con 0x55890e135de0 2024-09-10T08:46:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.298+0000 7f9380ff9640 1 -- 172.21.15.172:0/1313616331 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9390013e70 con 0x55890e135de0 2024-09-10T08:46:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.298+0000 7f9380ff9640 1 -- 172.21.15.172:0/1313616331 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390018040 con 0x55890e135de0 2024-09-10T08:46:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.299+0000 7f93a3547740 1 -- 172.21.15.172:0/1313616331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e135de0 msgr2=0x55890e151420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.299+0000 7f93a3547740 1 --2- 172.21.15.172:0/1313616331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e135de0 0x55890e151420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9390013920 tx=0x7f9390004a40 comp rx=0 tx=0).stop 2024-09-10T08:46:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.299+0000 7f93a3547740 1 -- 172.21.15.172:0/1313616331 shutdown_connections 2024-09-10T08:46:05.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.299+0000 7f93a3547740 1 --2- 172.21.15.172:0/1313616331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e140fc0 0x55890e13db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.299+0000 7f93a3547740 1 --2- 172.21.15.172:0/1313616331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e136700 0x55890e152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.299+0000 7f93a3547740 1 --2- 172.21.15.172:0/1313616331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e135de0 0x55890e151420 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.299+0000 7f93a3547740 1 -- 172.21.15.172:0/1313616331 >> 172.21.15.172:0/1313616331 conn(0x55890e132370 msgr2=0x55890e132750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.299+0000 7f93a3547740 1 -- 172.21.15.172:0/1313616331 shutdown_connections 2024-09-10T08:46:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.299+0000 7f93a3547740 1 -- 172.21.15.172:0/1313616331 wait complete. 2024-09-10T08:46:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.299+0000 7f93a3547740 1 -- start start 2024-09-10T08:46:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.299+0000 7f93a3547740 1 Processor -- start 2024-09-10T08:46:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f93a3547740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e135de0 0x55890e156240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f93a3547740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e136700 0x55890e156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f93a3547740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e140fc0 0x55890e156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f93a3547740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55890e152de0 con 0x55890e136700 2024-09-10T08:46:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f93a3547740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55890e152710 con 0x55890e135de0 2024-09-10T08:46:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f93a3547740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55890e1534b0 con 0x55890e140fc0 2024-09-10T08:46:05.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f93a3547740 10 client.0 _ll_register_callbacks cb 0x7f9394009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:05.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f939a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e136700 0x55890e156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f939a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e136700 0x55890e156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57182/0 (socket says 172.21.15.172:57182) 2024-09-10T08:46:05.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f939a59c640 1 -- 172.21.15.172:0/3681332156 learned_addr learned my addr 172.21.15.172:0/3681332156 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:05.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f939b59e640 1 --2- 172.21.15.172:0/3681332156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e140fc0 0x55890e156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f939ad9d640 1 --2- 172.21.15.172:0/3681332156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e135de0 0x55890e156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f939a59c640 1 -- 172.21.15.172:0/3681332156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e140fc0 msgr2=0x55890e156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:05.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f939a59c640 1 --2- 172.21.15.172:0/3681332156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e140fc0 0x55890e156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f939a59c640 1 -- 172.21.15.172:0/3681332156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e135de0 msgr2=0x55890e156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:05.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f939a59c640 1 --2- 172.21.15.172:0/3681332156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e135de0 0x55890e156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f939a59c640 1 -- 172.21.15.172:0/3681332156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55890e1ba490 con 0x55890e136700 2024-09-10T08:46:05.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f939a59c640 1 --2- 172.21.15.172:0/3681332156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e136700 0x55890e156780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f938c097980 tx=0x7f938c09b150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d37d04761c783f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:05.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f937eff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:05.320 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:46:05.320 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:46:05.320 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-09-10T08:46:05.320 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 24488, 2024-09-10T08:46:05.320 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-09-10T08:46:05.320 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-09-10T08:46:05.321 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-09-10T08:46:05.321 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 113.626716696, 2024-09-10T08:46:05.321 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 28, 2024-09-10T08:46:05.321 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 90, 2024-09-10T08:46:05.321 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 88, 2024-09-10T08:46:05.321 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 124.015003558 2024-09-10T08:46:05.321 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:46:05.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f937eff5640 1 -- 172.21.15.172:0/3681332156 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c020040 con 0x55890e136700 2024-09-10T08:46:05.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f937eff5640 1 -- 172.21.15.172:0/3681332156 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f938c09a970 con 0x55890e136700 2024-09-10T08:46:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f93a3547740 1 -- 172.21.15.172:0/3681332156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9390013e70 con 0x55890e136700 2024-09-10T08:46:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f937eff5640 1 -- 172.21.15.172:0/3681332156 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f938c09b520 con 0x55890e136700 2024-09-10T08:46:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f93a3547740 10 client.15153 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.300+0000 7f93a3547740 1 -- 172.21.15.172:0/3681332156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55890e1bb790 con 0x55890e136700 2024-09-10T08:46:05.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.301+0000 7f937eff5640 1 -- 172.21.15.172:0/3681332156 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9390013e70 con 0x55890e136700 2024-09-10T08:46:05.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.301+0000 7f937eff5640 1 -- 172.21.15.172:0/3681332156 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f938c0b42a0 con 0x55890e136700 2024-09-10T08:46:05.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.301+0000 7f93a3547740 10 client.15153 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:05.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.301+0000 7f93a3547740 10 client.15153 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:46:05.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.301+0000 7f93a3547740 20 client.15153 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:05.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.301+0000 7f93a3547740 1 --2- 172.21.15.172:0/3681332156 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55890e1c03b0 0x55890e1bf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:05.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.301+0000 7f93a3547740 4 client.15153 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:46:05.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.301+0000 7f93a3547740 1 -- 172.21.15.172:0/3681332156 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "status"}) -- 0x55890e09fb10 con 0x55890e1c03b0 2024-09-10T08:46:05.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.301+0000 7f939ad9d640 1 --2- 172.21.15.172:0/3681332156 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55890e1c03b0 0x55890e1bf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:05.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.301+0000 7f937eff5640 10 client.15153 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:05.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.301+0000 7f939ad9d640 1 --2- 172.21.15.172:0/3681332156 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55890e1c03b0 0x55890e1bf0c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e17c06f2a56bd306 server_cookie=ed947986271e7891 in_seq=0 out_seq=0 2024-09-10T08:46:05.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.302+0000 7f937eff5640 1 -- 172.21.15.172:0/3681332156 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55890e09fb10 con 0x55890e1c03b0 2024-09-10T08:46:05.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.302+0000 7f937eff5640 10 client.15153 handle_command_reply: tid=0 2024-09-10T08:46:05.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.302+0000 7f93a3547740 1 client.15153 shutdown 2024-09-10T08:46:05.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.302+0000 7f93a3547740 1 -- 172.21.15.172:0/3681332156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e136700 msgr2=0x55890e156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:05.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.302+0000 7f93a3547740 1 --2- 172.21.15.172:0/3681332156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e136700 0x55890e156780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f938c097980 tx=0x7f938c09b150 comp rx=0 tx=0).stop 2024-09-10T08:46:05.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.302+0000 7f93a3547740 1 -- 172.21.15.172:0/3681332156 shutdown_connections 2024-09-10T08:46:05.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.302+0000 7f93a3547740 1 --2- 172.21.15.172:0/3681332156 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55890e1c03b0 0x55890e1bf0c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.302+0000 7f937eff5640 0 client.15153 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:05.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.302+0000 7f93a3547740 1 --2- 172.21.15.172:0/3681332156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55890e140fc0 0x55890e156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.302+0000 7f93a3547740 1 --2- 172.21.15.172:0/3681332156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55890e136700 0x55890e156780 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.302+0000 7f93a3547740 1 --2- 172.21.15.172:0/3681332156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55890e135de0 0x55890e156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.302+0000 7f93a3547740 1 -- 172.21.15.172:0/3681332156 >> 172.21.15.172:0/3681332156 conn(0x55890e132370 msgr2=0x55890e13db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:05.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.302+0000 7f93a3547740 1 -- 172.21.15.172:0/3681332156 shutdown_connections 2024-09-10T08:46:05.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.302+0000 7f93a3547740 1 -- 172.21.15.172:0/3681332156 wait complete. 2024-09-10T08:46:05.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.302+0000 7f93a3547740 20 client.15153 trim_cache size 0 max 16384 2024-09-10T08:46:05.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.303+0000 7f93a116e640 1 -- 172.21.15.172:0/3466512796 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f93840a2c10 msgr2=0x7f93840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:05.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.303+0000 7f93a116e640 1 --2- 172.21.15.172:0/3466512796 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f93840a2c10 0x7f93840a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9394031440 tx=0x7f93940314a0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.303+0000 7f93a116e640 1 -- 172.21.15.172:0/3466512796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f939c101420 msgr2=0x7f939c1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:05.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.303+0000 7f93a116e640 1 --2- 172.21.15.172:0/3466512796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f939c101420 0x7f939c1988b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f938c00d950 tx=0x7f938c00de10 comp rx=0 tx=0).stop 2024-09-10T08:46:05.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.303+0000 7f93a116e640 1 -- 172.21.15.172:0/3466512796 shutdown_connections 2024-09-10T08:46:05.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.303+0000 7f93a116e640 1 --2- 172.21.15.172:0/3466512796 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f93840a2c10 0x7f93840a50d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.303+0000 7f93a116e640 1 --2- 172.21.15.172:0/3466512796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f939c106a80 0x7f939c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.303+0000 7f93a116e640 1 --2- 172.21.15.172:0/3466512796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f939c101420 0x7f939c1988b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.303+0000 7f93a116e640 1 --2- 172.21.15.172:0/3466512796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f939c100a80 0x7f939c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:05.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.303+0000 7f93a116e640 1 -- 172.21.15.172:0/3466512796 >> 172.21.15.172:0/3466512796 conn(0x7f939c0fc7a0 msgr2=0x7f939c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:05.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.303+0000 7f93a116e640 1 -- 172.21.15.172:0/3466512796 shutdown_connections 2024-09-10T08:46:05.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:05.303+0000 7f93a116e640 1 -- 172.21.15.172:0/3466512796 wait complete. 2024-09-10T08:46:06.168 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T08:46:06.168 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T08:46:06.181 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi172.front.sepia.ceph.com 2024-09-10T08:46:06.181 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T08:46:06.435 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:06.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:06 smithi174 ceph-mon[34068]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:06.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:06 smithi191 ceph-mon[35314]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:06.555 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:06 smithi172 ceph-mon[30854]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:06.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.954+0000 7ff7db0c4640 1 -- 172.21.15.172:0/1887938096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7d4102740 msgr2=0x7ff7d4104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:06.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.954+0000 7ff7db0c4640 1 --2- 172.21.15.172:0/1887938096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7d4102740 0x7ff7d4104b30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8009900 tx=0x7ff7c802a410 comp rx=0 tx=0).stop 2024-09-10T08:46:06.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.955+0000 7ff7db0c4640 1 -- 172.21.15.172:0/1887938096 shutdown_connections 2024-09-10T08:46:06.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.955+0000 7ff7db0c4640 1 --2- 172.21.15.172:0/1887938096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7d41079a0 0x7ff7d4109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:06.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.955+0000 7ff7db0c4640 1 --2- 172.21.15.172:0/1887938096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7d4105070 0x7ff7d4107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:06.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.955+0000 7ff7db0c4640 1 --2- 172.21.15.172:0/1887938096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7d4102740 0x7ff7d4104b30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:06.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.955+0000 7ff7db0c4640 1 -- 172.21.15.172:0/1887938096 >> 172.21.15.172:0/1887938096 conn(0x7ff7d40fc2a0 msgr2=0x7ff7d40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:06.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.955+0000 7ff7db0c4640 1 -- 172.21.15.172:0/1887938096 shutdown_connections 2024-09-10T08:46:06.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.955+0000 7ff7db0c4640 1 -- 172.21.15.172:0/1887938096 wait complete. 2024-09-10T08:46:06.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.955+0000 7ff7db0c4640 1 Processor -- start 2024-09-10T08:46:06.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.955+0000 7ff7db0c4640 1 -- start start 2024-09-10T08:46:06.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.955+0000 7ff7db0c4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7d4102740 0x7ff7d4197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:06.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.955+0000 7ff7db0c4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7d4105070 0x7ff7d41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:06.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.955+0000 7ff7db0c4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7d41079a0 0x7ff7d419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:06.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.955+0000 7ff7db0c4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d410d810 con 0x7ff7d4102740 2024-09-10T08:46:06.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.955+0000 7ff7db0c4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d410d690 con 0x7ff7d4105070 2024-09-10T08:46:06.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.955+0000 7ff7db0c4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d410d990 con 0x7ff7d41079a0 2024-09-10T08:46:06.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.955+0000 7ff7d8e60640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7d4102740 0x7ff7d4197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:06.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.956+0000 7ff7d8e60640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7d4102740 0x7ff7d4197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48734/0 (socket says 172.21.15.172:48734) 2024-09-10T08:46:06.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.956+0000 7ff7d8e60640 1 -- 172.21.15.172:0/3819784228 learned_addr learned my addr 172.21.15.172:0/3819784228 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:06.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.956+0000 7ff7d9661640 1 --2- 172.21.15.172:0/3819784228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7d41079a0 0x7ff7d419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:06.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.956+0000 7ff7d3fff640 1 --2- 172.21.15.172:0/3819784228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7d4105070 0x7ff7d41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:06.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.956+0000 7ff7d8e60640 1 -- 172.21.15.172:0/3819784228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7d41079a0 msgr2=0x7ff7d419f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:06.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.956+0000 7ff7d8e60640 1 --2- 172.21.15.172:0/3819784228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7d41079a0 0x7ff7d419f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:06.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.956+0000 7ff7d8e60640 1 -- 172.21.15.172:0/3819784228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7d4105070 msgr2=0x7ff7d41983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:06.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.956+0000 7ff7d8e60640 1 --2- 172.21.15.172:0/3819784228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7d4105070 0x7ff7d41983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:06.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.956+0000 7ff7d8e60640 1 -- 172.21.15.172:0/3819784228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d419fa90 con 0x7ff7d4102740 2024-09-10T08:46:06.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.956+0000 7ff7d3fff640 1 --2- 172.21.15.172:0/3819784228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7d4105070 0x7ff7d41983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:06.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.956+0000 7ff7d8e60640 1 --2- 172.21.15.172:0/3819784228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7d4102740 0x7ff7d4197e60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80314e0 tx=0x7ff7c8003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:06.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.956+0000 7ff7d1ffb640 1 -- 172.21.15.172:0/3819784228 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c803f070 con 0x7ff7d4102740 2024-09-10T08:46:06.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.956+0000 7ff7db0c4640 1 -- 172.21.15.172:0/3819784228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d419fd20 con 0x7ff7d4102740 2024-09-10T08:46:06.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.956+0000 7ff7d1ffb640 1 -- 172.21.15.172:0/3819784228 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7c8031e40 con 0x7ff7d4102740 2024-09-10T08:46:06.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.956+0000 7ff7d1ffb640 1 -- 172.21.15.172:0/3819784228 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8035030 con 0x7ff7d4102740 2024-09-10T08:46:06.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.957+0000 7ff7db0c4640 1 -- 172.21.15.172:0/3819784228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d41a01a0 con 0x7ff7d4102740 2024-09-10T08:46:06.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.957+0000 7ff7db0c4640 1 -- 172.21.15.172:0/3819784228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d40fdd90 con 0x7ff7d4102740 2024-09-10T08:46:06.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.959+0000 7ff7d1ffb640 1 -- 172.21.15.172:0/3819784228 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff7c8042070 con 0x7ff7d4102740 2024-09-10T08:46:06.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.959+0000 7ff7d1ffb640 1 --2- 172.21.15.172:0/3819784228 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7c40a2c40 0x7ff7c40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:06.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.959+0000 7ff7d3fff640 1 --2- 172.21.15.172:0/3819784228 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7c40a2c40 0x7ff7c40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:06.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.960+0000 7ff7d1ffb640 1 -- 172.21.15.172:0/3819784228 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7ff7c80f02e0 con 0x7ff7d4102740 2024-09-10T08:46:06.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.960+0000 7ff7d3fff640 1 --2- 172.21.15.172:0/3819784228 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7c40a2c40 0x7ff7c40a5100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4199380 tx=0x7ff7c0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:06.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:06.963+0000 7ff7d1ffb640 1 -- 172.21.15.172:0/3819784228 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff7c80bc750 con 0x7ff7d4102740 2024-09-10T08:46:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:07.146+0000 7ff7db0c4640 1 -- 172.21.15.172:0/3819784228 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7ff7d4062630 con 0x7ff7c40a2c40 2024-09-10T08:46:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:07.147+0000 7ff7d1ffb640 1 -- 172.21.15.172:0/3819784228 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7ff7d4062630 con 0x7ff7c40a2c40 2024-09-10T08:46:07.147 INFO:teuthology.orchestra.run.smithi172.stdout:Backend: cephadm 2024-09-10T08:46:07.147 INFO:teuthology.orchestra.run.smithi172.stdout:Available: Yes 2024-09-10T08:46:07.147 INFO:teuthology.orchestra.run.smithi172.stdout:Paused: No 2024-09-10T08:46:07.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:07.150+0000 7ff7db0c4640 1 -- 172.21.15.172:0/3819784228 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7c40a2c40 msgr2=0x7ff7c40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:07.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:07.150+0000 7ff7db0c4640 1 --2- 172.21.15.172:0/3819784228 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7c40a2c40 0x7ff7c40a5100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4199380 tx=0x7ff7c0008040 comp rx=0 tx=0).stop 2024-09-10T08:46:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:07.150+0000 7ff7db0c4640 1 -- 172.21.15.172:0/3819784228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7d4102740 msgr2=0x7ff7d4197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:07.150+0000 7ff7db0c4640 1 --2- 172.21.15.172:0/3819784228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7d4102740 0x7ff7d4197e60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80314e0 tx=0x7ff7c8003d50 comp rx=0 tx=0).stop 2024-09-10T08:46:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:07.150+0000 7ff7db0c4640 1 -- 172.21.15.172:0/3819784228 shutdown_connections 2024-09-10T08:46:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:07.150+0000 7ff7db0c4640 1 --2- 172.21.15.172:0/3819784228 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7c40a2c40 0x7ff7c40a5100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:07.150+0000 7ff7db0c4640 1 --2- 172.21.15.172:0/3819784228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7d41079a0 0x7ff7d419f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:07.150+0000 7ff7db0c4640 1 --2- 172.21.15.172:0/3819784228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7d4105070 0x7ff7d41983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:07.150+0000 7ff7db0c4640 1 --2- 172.21.15.172:0/3819784228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7d4102740 0x7ff7d4197e60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:07.150+0000 7ff7db0c4640 1 -- 172.21.15.172:0/3819784228 >> 172.21.15.172:0/3819784228 conn(0x7ff7d40fc2a0 msgr2=0x7ff7d41030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:07.150+0000 7ff7db0c4640 1 -- 172.21.15.172:0/3819784228 shutdown_connections 2024-09-10T08:46:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:07.150+0000 7ff7db0c4640 1 -- 172.21.15.172:0/3819784228 wait complete. 2024-09-10T08:46:07.884 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T08:46:08.132 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:08.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:08 smithi174 ceph-mon[34068]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:46:08.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:08 smithi174 ceph-mon[34068]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T08:46:08.526 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:08 smithi172 ceph-mon[30854]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:46:08.526 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:08 smithi172 ceph-mon[30854]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T08:46:08.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:08 smithi191 ceph-mon[35314]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:46:08.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:08 smithi191 ceph-mon[35314]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T08:46:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.605+0000 7f81a02fc640 1 -- 172.21.15.172:0/1310421085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81981026d0 msgr2=0x7f8198108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.605+0000 7f81a02fc640 1 --2- 172.21.15.172:0/1310421085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81981026d0 0x7f8198108f60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8190009ca0 tx=0x7f819002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:46:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.605+0000 7f81a02fc640 1 -- 172.21.15.172:0/1310421085 shutdown_connections 2024-09-10T08:46:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.605+0000 7f81a02fc640 1 --2- 172.21.15.172:0/1310421085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81981026d0 0x7f8198108f60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.605+0000 7f81a02fc640 1 --2- 172.21.15.172:0/1310421085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8198101d10 0x7f8198102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.605+0000 7f81a02fc640 1 --2- 172.21.15.172:0/1310421085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8198100b10 0x7f8198100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.605+0000 7f81a02fc640 1 -- 172.21.15.172:0/1310421085 >> 172.21.15.172:0/1310421085 conn(0x7f81980fc2a0 msgr2=0x7f81980fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.605+0000 7f81a02fc640 1 -- 172.21.15.172:0/1310421085 shutdown_connections 2024-09-10T08:46:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.605+0000 7f81a02fc640 1 -- 172.21.15.172:0/1310421085 wait complete. 2024-09-10T08:46:08.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f81a02fc640 1 Processor -- start 2024-09-10T08:46:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f81a02fc640 1 -- start start 2024-09-10T08:46:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f81a02fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8198100b10 0x7f819806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f81a02fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8198101d10 0x7f819819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f81a02fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81981026d0 0x7f819819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f81a02fc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f819810c6a0 con 0x7f8198101d10 2024-09-10T08:46:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f81a02fc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f819810c520 con 0x7f81981026d0 2024-09-10T08:46:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f81a02fc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f819810c820 con 0x7f8198100b10 2024-09-10T08:46:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f819d897640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8198101d10 0x7f819819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f819d897640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8198101d10 0x7f819819cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48748/0 (socket says 172.21.15.172:48748) 2024-09-10T08:46:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f819d897640 1 -- 172.21.15.172:0/2930377628 learned_addr learned my addr 172.21.15.172:0/2930377628 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f819e098640 1 --2- 172.21.15.172:0/2930377628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8198100b10 0x7f819806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:08.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f819d897640 1 -- 172.21.15.172:0/2930377628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8198100b10 msgr2=0x7f819806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:08.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f819d897640 1 --2- 172.21.15.172:0/2930377628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8198100b10 0x7f819806b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:08.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f819e899640 1 --2- 172.21.15.172:0/2930377628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81981026d0 0x7f819819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:08.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f819d897640 1 -- 172.21.15.172:0/2930377628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81981026d0 msgr2=0x7f819819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:08.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f819d897640 1 --2- 172.21.15.172:0/2930377628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81981026d0 0x7f819819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:08.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f819d897640 1 -- 172.21.15.172:0/2930377628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819819fa50 con 0x7f8198101d10 2024-09-10T08:46:08.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.606+0000 7f819e899640 1 --2- 172.21.15.172:0/2930377628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81981026d0 0x7f819819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:08.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.607+0000 7f819d897640 1 --2- 172.21.15.172:0/2930377628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8198101d10 0x7f819819cb20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f818800b4a0 tx=0x7f818800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:08.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.607+0000 7f818f7fe640 1 -- 172.21.15.172:0/2930377628 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188014070 con 0x7f8198101d10 2024-09-10T08:46:08.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.607+0000 7f818f7fe640 1 -- 172.21.15.172:0/2930377628 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8188010ac0 con 0x7f8198101d10 2024-09-10T08:46:08.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.607+0000 7f81a02fc640 1 -- 172.21.15.172:0/2930377628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819819fce0 con 0x7f8198101d10 2024-09-10T08:46:08.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.607+0000 7f818f7fe640 1 -- 172.21.15.172:0/2930377628 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188017c30 con 0x7f8198101d10 2024-09-10T08:46:08.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.607+0000 7f81a02fc640 1 -- 172.21.15.172:0/2930377628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81981a01c0 con 0x7f8198101d10 2024-09-10T08:46:08.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.608+0000 7f81a02fc640 1 -- 172.21.15.172:0/2930377628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8198107f60 con 0x7f8198101d10 2024-09-10T08:46:08.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.609+0000 7f818f7fe640 1 -- 172.21.15.172:0/2930377628 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8188017070 con 0x7f8198101d10 2024-09-10T08:46:08.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.610+0000 7f818f7fe640 1 --2- 172.21.15.172:0/2930377628 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f81840a2c10 0x7f81840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:08.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.610+0000 7f819e098640 1 --2- 172.21.15.172:0/2930377628 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f81840a2c10 0x7f81840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:08.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.610+0000 7f818f7fe640 1 -- 172.21.15.172:0/2930377628 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f81880cebc0 con 0x7f8198101d10 2024-09-10T08:46:08.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.610+0000 7f819e098640 1 --2- 172.21.15.172:0/2930377628 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f81840a2c10 0x7f81840a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8194005fd0 tx=0x7f8194005ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:08.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.613+0000 7f818f7fe640 1 -- 172.21.15.172:0/2930377628 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f818809b790 con 0x7f8198101d10 2024-09-10T08:46:08.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.789+0000 7f81a02fc640 1 -- 172.21.15.172:0/2930377628 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f8198062630 con 0x7f81840a2c10 2024-09-10T08:46:08.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.801+0000 7f818f7fe640 1 -- 172.21.15.172:0/2930377628 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7f8198062630 con 0x7f81840a2c10 2024-09-10T08:46:08.801 INFO:teuthology.orchestra.run.smithi172.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T08:46:08.802 INFO:teuthology.orchestra.run.smithi172.stdout:mds.a smithi172 running (2m) 58s ago 2m 17.0M - 19.1.1-305-g1cf819b7 41d671128357 d653c3296f85 2024-09-10T08:46:08.802 INFO:teuthology.orchestra.run.smithi172.stdout:mds.b smithi174 running (2m) 60s ago 2m 26.6M - 19.1.1-305-g1cf819b7 41d671128357 455496ffd6ee 2024-09-10T08:46:08.802 INFO:teuthology.orchestra.run.smithi172.stdout:mds.c smithi191 running (2m) 60s ago 2m 15.6M - 19.1.1-305-g1cf819b7 41d671128357 b8308ec924a8 2024-09-10T08:46:08.802 INFO:teuthology.orchestra.run.smithi172.stdout:mds.d smithi172 running (2m) 58s ago 2m 14.4M - 19.1.1-305-g1cf819b7 41d671128357 50cb24e942a5 2024-09-10T08:46:08.802 INFO:teuthology.orchestra.run.smithi172.stdout:mds.e smithi174 running (2m) 60s ago 2m 14.9M - 19.1.1-305-g1cf819b7 41d671128357 0214c70d606f 2024-09-10T08:46:08.802 INFO:teuthology.orchestra.run.smithi172.stdout:mds.f smithi191 running (2m) 60s ago 2m 15.4M - 19.1.1-305-g1cf819b7 41d671128357 465ebd26bf99 2024-09-10T08:46:08.802 INFO:teuthology.orchestra.run.smithi172.stdout:mds.g smithi172 running (2m) 58s ago 2m 14.5M - 19.1.1-305-g1cf819b7 41d671128357 c1224c865ee0 2024-09-10T08:46:08.802 INFO:teuthology.orchestra.run.smithi172.stdout:mds.h smithi174 running (2m) 60s ago 2m 17.2M - 19.1.1-305-g1cf819b7 41d671128357 809cc2da9db7 2024-09-10T08:46:08.802 INFO:teuthology.orchestra.run.smithi172.stdout:mds.i smithi191 running (2m) 60s ago 2m 19.3M - 19.1.1-305-g1cf819b7 41d671128357 68bd573d32e4 2024-09-10T08:46:08.802 INFO:teuthology.orchestra.run.smithi172.stdout:mds.j smithi172 running (2m) 58s ago 2m 15.9M - 19.1.1-305-g1cf819b7 41d671128357 1278041f7ff3 2024-09-10T08:46:08.802 INFO:teuthology.orchestra.run.smithi172.stdout:mds.k smithi174 running (2m) 60s ago 2m 17.2M - 19.1.1-305-g1cf819b7 41d671128357 7e751f71648b 2024-09-10T08:46:08.802 INFO:teuthology.orchestra.run.smithi172.stdout:mds.l smithi191 running (2m) 60s ago 2m 16.1M - 19.1.1-305-g1cf819b7 41d671128357 b309f6acf4fc 2024-09-10T08:46:08.803 INFO:teuthology.orchestra.run.smithi172.stdout:mgr.x smithi172 *:9283,8765 running (9m) 58s ago 9m 507M - 19.1.1-305-g1cf819b7 41d671128357 381553d8692d 2024-09-10T08:46:08.803 INFO:teuthology.orchestra.run.smithi172.stdout:mgr.y smithi174 *:8443,8765 running (7m) 60s ago 7m 442M - 19.1.1-305-g1cf819b7 41d671128357 6ac7282b613d 2024-09-10T08:46:08.803 INFO:teuthology.orchestra.run.smithi172.stdout:mgr.z smithi191 *:8443,8765 running (7m) 60s ago 7m 444M - 19.1.1-305-g1cf819b7 41d671128357 26426940635a 2024-09-10T08:46:08.803 INFO:teuthology.orchestra.run.smithi172.stdout:mon.a smithi172 running (9m) 58s ago 9m 50.6M 2048M 19.1.1-305-g1cf819b7 41d671128357 4ca591a9b8a0 2024-09-10T08:46:08.803 INFO:teuthology.orchestra.run.smithi172.stdout:mon.b smithi174 running (8m) 60s ago 8m 40.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 1732bb4b3482 2024-09-10T08:46:08.803 INFO:teuthology.orchestra.run.smithi172.stdout:mon.c smithi191 running (8m) 60s ago 8m 39.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 319a9d1e0516 2024-09-10T08:46:08.803 INFO:teuthology.orchestra.run.smithi172.stdout:osd.0 smithi172 running (7m) 58s ago 7m 47.2M 4272M 19.1.1-305-g1cf819b7 41d671128357 62944d5e6d01 2024-09-10T08:46:08.803 INFO:teuthology.orchestra.run.smithi172.stdout:osd.1 smithi174 running (7m) 60s ago 7m 56.1M 4272M 19.1.1-305-g1cf819b7 41d671128357 d9dc945be20d 2024-09-10T08:46:08.803 INFO:teuthology.orchestra.run.smithi172.stdout:osd.2 smithi191 running (6m) 60s ago 6m 46.4M 4272M 19.1.1-305-g1cf819b7 41d671128357 09fb1ee4b8c2 2024-09-10T08:46:08.803 INFO:teuthology.orchestra.run.smithi172.stdout:osd.3 smithi172 running (6m) 58s ago 6m 43.9M 4272M 19.1.1-305-g1cf819b7 41d671128357 e7784c241613 2024-09-10T08:46:08.803 INFO:teuthology.orchestra.run.smithi172.stdout:osd.4 smithi174 running (5m) 60s ago 5m 50.8M 4272M 19.1.1-305-g1cf819b7 41d671128357 aa043b0b0408 2024-09-10T08:46:08.803 INFO:teuthology.orchestra.run.smithi172.stdout:osd.5 smithi191 running (5m) 60s ago 5m 53.0M 4272M 19.1.1-305-g1cf819b7 41d671128357 ae1af52f1859 2024-09-10T08:46:08.803 INFO:teuthology.orchestra.run.smithi172.stdout:osd.6 smithi172 running (5m) 58s ago 5m 42.5M 4272M 19.1.1-305-g1cf819b7 41d671128357 e5723f647f88 2024-09-10T08:46:08.804 INFO:teuthology.orchestra.run.smithi172.stdout:osd.7 smithi174 running (4m) 60s ago 4m 45.4M 4272M 19.1.1-305-g1cf819b7 41d671128357 6dc02288f487 2024-09-10T08:46:08.804 INFO:teuthology.orchestra.run.smithi172.stdout:osd.8 smithi191 running (4m) 60s ago 4m 54.3M 4272M 19.1.1-305-g1cf819b7 41d671128357 00e7d3dc6041 2024-09-10T08:46:08.804 INFO:teuthology.orchestra.run.smithi172.stdout:osd.9 smithi172 running (3m) 58s ago 3m 51.2M 4272M 19.1.1-305-g1cf819b7 41d671128357 abb43fcacbac 2024-09-10T08:46:08.804 INFO:teuthology.orchestra.run.smithi172.stdout:osd.10 smithi174 running (3m) 60s ago 3m 44.4M 4272M 19.1.1-305-g1cf819b7 41d671128357 3b7183de11dd 2024-09-10T08:46:08.804 INFO:teuthology.orchestra.run.smithi172.stdout:osd.11 smithi191 running (2m) 60s ago 2m 53.6M 4272M 19.1.1-305-g1cf819b7 41d671128357 f3d58ec1b6df 2024-09-10T08:46:08.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.803+0000 7f81a02fc640 1 -- 172.21.15.172:0/2930377628 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f81840a2c10 msgr2=0x7f81840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:08.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.803+0000 7f81a02fc640 1 --2- 172.21.15.172:0/2930377628 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f81840a2c10 0x7f81840a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8194005fd0 tx=0x7f8194005ce0 comp rx=0 tx=0).stop 2024-09-10T08:46:08.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.803+0000 7f81a02fc640 1 -- 172.21.15.172:0/2930377628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8198101d10 msgr2=0x7f819819cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:08.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.803+0000 7f81a02fc640 1 --2- 172.21.15.172:0/2930377628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8198101d10 0x7f819819cb20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f818800b4a0 tx=0x7f818800b960 comp rx=0 tx=0).stop 2024-09-10T08:46:08.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.803+0000 7f81a02fc640 1 -- 172.21.15.172:0/2930377628 shutdown_connections 2024-09-10T08:46:08.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.803+0000 7f81a02fc640 1 --2- 172.21.15.172:0/2930377628 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f81840a2c10 0x7f81840a50d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:08.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.803+0000 7f81a02fc640 1 --2- 172.21.15.172:0/2930377628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81981026d0 0x7f819819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:08.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.803+0000 7f81a02fc640 1 --2- 172.21.15.172:0/2930377628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8198101d10 0x7f819819cb20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:08.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.803+0000 7f81a02fc640 1 --2- 172.21.15.172:0/2930377628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8198100b10 0x7f819806b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:08.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.803+0000 7f81a02fc640 1 -- 172.21.15.172:0/2930377628 >> 172.21.15.172:0/2930377628 conn(0x7f81980fc2a0 msgr2=0x7f8198105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:08.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.803+0000 7f81a02fc640 1 -- 172.21.15.172:0/2930377628 shutdown_connections 2024-09-10T08:46:08.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:08.803+0000 7f81a02fc640 1 -- 172.21.15.172:0/2930377628 wait complete. 2024-09-10T08:46:09.577 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T08:46:09.824 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:10.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.301+0000 7f73feb0e640 1 -- 172.21.15.172:0/1427736496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73f81026a0 msgr2=0x7f73f8104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:10.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.301+0000 7f73feb0e640 1 --2- 172.21.15.172:0/1427736496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73f81026a0 0x7f73f8104a90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009900 tx=0x7f73ec02a410 comp rx=0 tx=0).stop 2024-09-10T08:46:10.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.301+0000 7f73feb0e640 1 -- 172.21.15.172:0/1427736496 shutdown_connections 2024-09-10T08:46:10.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.301+0000 7f73feb0e640 1 --2- 172.21.15.172:0/1427736496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73f8107900 0x7f73f8109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:10.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.301+0000 7f73feb0e640 1 --2- 172.21.15.172:0/1427736496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73f8104fd0 0x7f73f81073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:10.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.301+0000 7f73feb0e640 1 --2- 172.21.15.172:0/1427736496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73f81026a0 0x7f73f8104a90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:10.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.301+0000 7f73feb0e640 1 -- 172.21.15.172:0/1427736496 >> 172.21.15.172:0/1427736496 conn(0x7f73f80fc2a0 msgr2=0x7f73f80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:10.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.301+0000 7f73feb0e640 1 -- 172.21.15.172:0/1427736496 shutdown_connections 2024-09-10T08:46:10.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.301+0000 7f73feb0e640 1 -- 172.21.15.172:0/1427736496 wait complete. 2024-09-10T08:46:10.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.301+0000 7f73feb0e640 1 Processor -- start 2024-09-10T08:46:10.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.301+0000 7f73feb0e640 1 -- start start 2024-09-10T08:46:10.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73feb0e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73f81026a0 0x7f73f8197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:10.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73feb0e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73f8104fd0 0x7f73f81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:10.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73feb0e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73f8107900 0x7f73f819f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:10.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73feb0e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f73f810d7f0 con 0x7f73f8107900 2024-09-10T08:46:10.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73feb0e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f73f810d670 con 0x7f73f8104fd0 2024-09-10T08:46:10.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73feb0e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f73f810d970 con 0x7f73f81026a0 2024-09-10T08:46:10.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73fd0ab640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73f8107900 0x7f73f819f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:10.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73fd0ab640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73f8107900 0x7f73f819f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48756/0 (socket says 172.21.15.172:48756) 2024-09-10T08:46:10.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73fd0ab640 1 -- 172.21.15.172:0/1859744426 learned_addr learned my addr 172.21.15.172:0/1859744426 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:10.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73f7fff640 1 --2- 172.21.15.172:0/1859744426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73f8104fd0 0x7f73f81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:10.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73fc8aa640 1 --2- 172.21.15.172:0/1859744426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73f81026a0 0x7f73f8197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:10.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73fd0ab640 1 -- 172.21.15.172:0/1859744426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73f81026a0 msgr2=0x7f73f8197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:10.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73fd0ab640 1 --2- 172.21.15.172:0/1859744426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73f81026a0 0x7f73f8197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:10.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73fd0ab640 1 -- 172.21.15.172:0/1859744426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73f8104fd0 msgr2=0x7f73f81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:10.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73fd0ab640 1 --2- 172.21.15.172:0/1859744426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73f8104fd0 0x7f73f81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:10.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73fd0ab640 1 -- 172.21.15.172:0/1859744426 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f819fae0 con 0x7f73f8107900 2024-09-10T08:46:10.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73fc8aa640 1 --2- 172.21.15.172:0/1859744426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73f81026a0 0x7f73f8197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:10.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73fd0ab640 1 --2- 172.21.15.172:0/1859744426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73f8107900 0x7f73f819f470 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f73f000eaa0 tx=0x7f73f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:10.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73f5ffb640 1 -- 172.21.15.172:0/1859744426 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f000cdb0 con 0x7f73f8107900 2024-09-10T08:46:10.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73f5ffb640 1 -- 172.21.15.172:0/1859744426 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73f0005020 con 0x7f73f8107900 2024-09-10T08:46:10.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73feb0e640 1 -- 172.21.15.172:0/1859744426 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f819fdd0 con 0x7f73f8107900 2024-09-10T08:46:10.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.302+0000 7f73f5ffb640 1 -- 172.21.15.172:0/1859744426 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f00116d0 con 0x7f73f8107900 2024-09-10T08:46:10.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.303+0000 7f73feb0e640 1 -- 172.21.15.172:0/1859744426 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f81a0300 con 0x7f73f8107900 2024-09-10T08:46:10.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.304+0000 7f73feb0e640 1 -- 172.21.15.172:0/1859744426 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f80fdd90 con 0x7f73f8107900 2024-09-10T08:46:10.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.305+0000 7f73f5ffb640 1 -- 172.21.15.172:0/1859744426 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f73f00051d0 con 0x7f73f8107900 2024-09-10T08:46:10.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.305+0000 7f73f5ffb640 1 --2- 172.21.15.172:0/1859744426 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f73e80a2c40 0x7f73e80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:10.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.305+0000 7f73fc8aa640 1 --2- 172.21.15.172:0/1859744426 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f73e80a2c40 0x7f73e80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:10.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.306+0000 7f73f5ffb640 1 -- 172.21.15.172:0/1859744426 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f73f00cf700 con 0x7f73f8107900 2024-09-10T08:46:10.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.306+0000 7f73fc8aa640 1 --2- 172.21.15.172:0/1859744426 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f73e80a2c40 0x7f73e80a5100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f73ec006010 tx=0x7f73ec000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:10.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.309+0000 7f73f5ffb640 1 -- 172.21.15.172:0/1859744426 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73f0016030 con 0x7f73f8107900 2024-09-10T08:46:10.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.485+0000 7f73feb0e640 1 -- 172.21.15.172:0/1859744426 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f73f8062630 con 0x7f73e80a2c40 2024-09-10T08:46:10.486 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:10 smithi172 ceph-mon[30854]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:46:10.486 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:10 smithi172 ceph-mon[30854]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:10.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.491+0000 7f73f5ffb640 1 -- 172.21.15.172:0/1859744426 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f73f8062630 con 0x7f73e80a2c40 2024-09-10T08:46:10.492 INFO:teuthology.orchestra.run.smithi172.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T08:46:10.492 INFO:teuthology.orchestra.run.smithi172.stdout:mds.all 12/12 62s ago 2m smithi172=a;smithi172=d;smithi172=g;smithi172=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l;count:12 2024-09-10T08:46:10.492 INFO:teuthology.orchestra.run.smithi172.stdout:mgr 3/3 62s ago 8m smithi172=x;smithi174=y;smithi191=z;count:3 2024-09-10T08:46:10.492 INFO:teuthology.orchestra.run.smithi172.stdout:mon 3/3 62s ago 8m smithi172:172.21.15.172=a;smithi174:172.21.15.174=b;smithi191:172.21.15.191=c;count:3 2024-09-10T08:46:10.492 INFO:teuthology.orchestra.run.smithi172.stdout:osd 12 62s ago - 2024-09-10T08:46:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.493+0000 7f73feb0e640 1 -- 172.21.15.172:0/1859744426 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f73e80a2c40 msgr2=0x7f73e80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.493+0000 7f73feb0e640 1 --2- 172.21.15.172:0/1859744426 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f73e80a2c40 0x7f73e80a5100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f73ec006010 tx=0x7f73ec000bc0 comp rx=0 tx=0).stop 2024-09-10T08:46:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.493+0000 7f73feb0e640 1 -- 172.21.15.172:0/1859744426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73f8107900 msgr2=0x7f73f819f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.493+0000 7f73feb0e640 1 --2- 172.21.15.172:0/1859744426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73f8107900 0x7f73f819f470 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f73f000eaa0 tx=0x7f73f000ef60 comp rx=0 tx=0).stop 2024-09-10T08:46:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.493+0000 7f73feb0e640 1 -- 172.21.15.172:0/1859744426 shutdown_connections 2024-09-10T08:46:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.493+0000 7f73feb0e640 1 --2- 172.21.15.172:0/1859744426 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f73e80a2c40 0x7f73e80a5100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.493+0000 7f73feb0e640 1 --2- 172.21.15.172:0/1859744426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73f8107900 0x7f73f819f470 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.493+0000 7f73feb0e640 1 --2- 172.21.15.172:0/1859744426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73f8104fd0 0x7f73f81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:10.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.493+0000 7f73feb0e640 1 --2- 172.21.15.172:0/1859744426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73f81026a0 0x7f73f8197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:10.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.493+0000 7f73feb0e640 1 -- 172.21.15.172:0/1859744426 >> 172.21.15.172:0/1859744426 conn(0x7f73f80fc2a0 msgr2=0x7f73f8102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:10.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.493+0000 7f73feb0e640 1 -- 172.21.15.172:0/1859744426 shutdown_connections 2024-09-10T08:46:10.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:10.493+0000 7f73feb0e640 1 -- 172.21.15.172:0/1859744426 wait complete. 2024-09-10T08:46:10.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:10 smithi191 ceph-mon[35314]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:46:10.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:10 smithi191 ceph-mon[35314]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:10.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:10 smithi174 ceph-mon[34068]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:46:10.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:10 smithi174 ceph-mon[34068]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:11.314 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T08:46:11.532 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:11 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:46:11.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:11 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:46:11.604 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:11 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:46:12.219 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:12.504 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:12 smithi172 ceph-mon[30854]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:46:12.505 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:12 smithi172 ceph-mon[30854]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T08:46:12.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:12 smithi191 ceph-mon[35314]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:46:12.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:12 smithi191 ceph-mon[35314]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T08:46:12.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.714+0000 7f5b83f5a640 1 -- 172.21.15.172:0/2019026615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b7c102e00 msgr2=0x7f5b7c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:12.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.714+0000 7f5b83f5a640 1 --2- 172.21.15.172:0/2019026615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b7c102e00 0x7f5b7c103280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009900 tx=0x7f5b7002a410 comp rx=0 tx=0).stop 2024-09-10T08:46:12.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.714+0000 7f5b83f5a640 1 -- 172.21.15.172:0/2019026615 shutdown_connections 2024-09-10T08:46:12.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.714+0000 7f5b83f5a640 1 --2- 172.21.15.172:0/2019026615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b7c1037c0 0x7f5b7c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.714+0000 7f5b83f5a640 1 --2- 172.21.15.172:0/2019026615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b7c102e00 0x7f5b7c103280 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.714+0000 7f5b83f5a640 1 --2- 172.21.15.172:0/2019026615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b7c101c00 0x7f5b7c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.714+0000 7f5b83f5a640 1 -- 172.21.15.172:0/2019026615 >> 172.21.15.172:0/2019026615 conn(0x7f5b7c0fd390 msgr2=0x7f5b7c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:12.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.715+0000 7f5b83f5a640 1 -- 172.21.15.172:0/2019026615 shutdown_connections 2024-09-10T08:46:12.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.715+0000 7f5b83f5a640 1 -- 172.21.15.172:0/2019026615 wait complete. 2024-09-10T08:46:12.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.715+0000 7f5b83f5a640 1 Processor -- start 2024-09-10T08:46:12.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.715+0000 7f5b83f5a640 1 -- start start 2024-09-10T08:46:12.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.715+0000 7f5b83f5a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b7c101c00 0x7f5b7c197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.715+0000 7f5b83f5a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b7c102e00 0x7f5b7c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.715+0000 7f5b83f5a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b7c1037c0 0x7f5b7c19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.715+0000 7f5b83f5a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c10d730 con 0x7f5b7c102e00 2024-09-10T08:46:12.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.715+0000 7f5b83f5a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c10d5b0 con 0x7f5b7c1037c0 2024-09-10T08:46:12.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.715+0000 7f5b83f5a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7c10d8b0 con 0x7f5b7c101c00 2024-09-10T08:46:12.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.715+0000 7f5b814f5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b7c102e00 0x7f5b7c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:12.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.716+0000 7f5b81cf6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b7c101c00 0x7f5b7c197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:12.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.716+0000 7f5b81cf6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b7c101c00 0x7f5b7c197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:34452/0 (socket says 172.21.15.172:34452) 2024-09-10T08:46:12.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.716+0000 7f5b814f5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b7c102e00 0x7f5b7c198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48780/0 (socket says 172.21.15.172:48780) 2024-09-10T08:46:12.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.716+0000 7f5b814f5640 1 -- 172.21.15.172:0/3609074643 learned_addr learned my addr 172.21.15.172:0/3609074643 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:12.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.716+0000 7f5b824f7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b7c1037c0 0x7f5b7c19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:12.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.716+0000 7f5b81cf6640 1 -- 172.21.15.172:0/3609074643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b7c1037c0 msgr2=0x7f5b7c19f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:12.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.716+0000 7f5b81cf6640 1 --2- 172.21.15.172:0/3609074643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b7c1037c0 0x7f5b7c19f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.716+0000 7f5b81cf6640 1 -- 172.21.15.172:0/3609074643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b7c102e00 msgr2=0x7f5b7c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:12.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.716+0000 7f5b81cf6640 1 --2- 172.21.15.172:0/3609074643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b7c102e00 0x7f5b7c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.716+0000 7f5b81cf6640 1 -- 172.21.15.172:0/3609074643 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7c19fa80 con 0x7f5b7c101c00 2024-09-10T08:46:12.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.716+0000 7f5b814f5640 1 --2- 172.21.15.172:0/3609074643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b7c102e00 0x7f5b7c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:46:12.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.716+0000 7f5b81cf6640 1 --2- 172.21.15.172:0/3609074643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b7c101c00 0x7f5b7c197e50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400e9c0 tx=0x7f5b7400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:12.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.716+0000 7f5b7affd640 1 -- 172.21.15.172:0/3609074643 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7400ccc0 con 0x7f5b7c101c00 2024-09-10T08:46:12.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.716+0000 7f5b7affd640 1 -- 172.21.15.172:0/3609074643 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b740035f0 con 0x7f5b7c101c00 2024-09-10T08:46:12.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.716+0000 7f5b83f5a640 1 -- 172.21.15.172:0/3609074643 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7c19fd70 con 0x7f5b7c101c00 2024-09-10T08:46:12.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.716+0000 7f5b7affd640 1 -- 172.21.15.172:0/3609074643 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7400ce60 con 0x7f5b7c101c00 2024-09-10T08:46:12.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.716+0000 7f5b83f5a640 1 -- 172.21.15.172:0/3609074643 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b7c1a01c0 con 0x7f5b7c101c00 2024-09-10T08:46:12.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.718+0000 7f5b83f5a640 1 -- 172.21.15.172:0/3609074643 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b7c1090a0 con 0x7f5b7c101c00 2024-09-10T08:46:12.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.718+0000 7f5b7affd640 1 -- 172.21.15.172:0/3609074643 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5b740037a0 con 0x7f5b7c101c00 2024-09-10T08:46:12.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.719+0000 7f5b7affd640 1 --2- 172.21.15.172:0/3609074643 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b640a2cc0 0x7f5b640a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:12.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.719+0000 7f5b814f5640 1 --2- 172.21.15.172:0/3609074643 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b640a2cc0 0x7f5b640a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:12.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.719+0000 7f5b7affd640 1 -- 172.21.15.172:0/3609074643 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f5b740cf7b0 con 0x7f5b7c101c00 2024-09-10T08:46:12.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.720+0000 7f5b814f5640 1 --2- 172.21.15.172:0/3609074643 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b640a2cc0 0x7f5b640a5180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5b700314e0 tx=0x7f5b700058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:12.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.724+0000 7f5b7affd640 1 -- 172.21.15.172:0/3609074643 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b7409c270 con 0x7f5b7c101c00 2024-09-10T08:46:12.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:12 smithi174 ceph-mon[34068]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:46:12.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:12 smithi174 ceph-mon[34068]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T08:46:12.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.947+0000 7f5b83f5a640 1 -- 172.21.15.172:0/3609074643 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f5b7c062630 con 0x7f5b640a2cc0 2024-09-10T08:46:12.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.948+0000 7f5b7affd640 1 -- 172.21.15.172:0/3609074643 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f5b7c062630 con 0x7f5b640a2cc0 2024-09-10T08:46:12.949 INFO:teuthology.orchestra.run.smithi172.stdout:HOST ADDR LABELS STATUS 2024-09-10T08:46:12.949 INFO:teuthology.orchestra.run.smithi172.stdout:smithi172 172.21.15.172 2024-09-10T08:46:12.949 INFO:teuthology.orchestra.run.smithi172.stdout:smithi174 172.21.15.174 2024-09-10T08:46:12.949 INFO:teuthology.orchestra.run.smithi172.stdout:smithi191 172.21.15.191 2024-09-10T08:46:12.949 INFO:teuthology.orchestra.run.smithi172.stdout:3 hosts in cluster 2024-09-10T08:46:12.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.951+0000 7f5b83f5a640 1 -- 172.21.15.172:0/3609074643 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b640a2cc0 msgr2=0x7f5b640a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:12.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.951+0000 7f5b83f5a640 1 --2- 172.21.15.172:0/3609074643 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b640a2cc0 0x7f5b640a5180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5b700314e0 tx=0x7f5b700058d0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.951+0000 7f5b83f5a640 1 -- 172.21.15.172:0/3609074643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b7c101c00 msgr2=0x7f5b7c197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:12.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.951+0000 7f5b83f5a640 1 --2- 172.21.15.172:0/3609074643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b7c101c00 0x7f5b7c197e50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5b7400e9c0 tx=0x7f5b7400ee80 comp rx=0 tx=0).stop 2024-09-10T08:46:12.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.951+0000 7f5b83f5a640 1 -- 172.21.15.172:0/3609074643 shutdown_connections 2024-09-10T08:46:12.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.951+0000 7f5b83f5a640 1 --2- 172.21.15.172:0/3609074643 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b640a2cc0 0x7f5b640a5180 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.951+0000 7f5b83f5a640 1 --2- 172.21.15.172:0/3609074643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b7c1037c0 0x7f5b7c19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.951+0000 7f5b83f5a640 1 --2- 172.21.15.172:0/3609074643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b7c102e00 0x7f5b7c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.951+0000 7f5b83f5a640 1 --2- 172.21.15.172:0/3609074643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b7c101c00 0x7f5b7c197e50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:12.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.951+0000 7f5b83f5a640 1 -- 172.21.15.172:0/3609074643 >> 172.21.15.172:0/3609074643 conn(0x7f5b7c0fd390 msgr2=0x7f5b7c106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:12.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.951+0000 7f5b83f5a640 1 -- 172.21.15.172:0/3609074643 shutdown_connections 2024-09-10T08:46:12.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:12.951+0000 7f5b83f5a640 1 -- 172.21.15.172:0/3609074643 wait complete. 2024-09-10T08:46:13.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:13.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:13.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:13.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:13 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:13.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:13 smithi172 ceph-mon[30854]: from='client.24799 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:46:13.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:13 smithi172 ceph-mon[30854]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:13.711 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T08:46:13.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:13.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:13.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:13.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:13 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:13.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:13 smithi174 ceph-mon[34068]: from='client.24799 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:46:13.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:13 smithi174 ceph-mon[34068]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:13.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:13.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:13.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:13.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:13 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:13.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:13 smithi191 ceph-mon[35314]: from='client.24799 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:46:13.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:13 smithi191 ceph-mon[35314]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:13.974 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:14.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.479+0000 7f16aaa9e640 1 -- 172.21.15.172:0/2983547591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a4102e00 msgr2=0x7f16a4103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:14.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.479+0000 7f16aaa9e640 1 --2- 172.21.15.172:0/2983547591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a4102e00 0x7f16a4103280 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1690009900 tx=0x7f169002a410 comp rx=0 tx=0).stop 2024-09-10T08:46:14.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.479+0000 7f16aaa9e640 1 -- 172.21.15.172:0/2983547591 shutdown_connections 2024-09-10T08:46:14.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.479+0000 7f16aaa9e640 1 --2- 172.21.15.172:0/2983547591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16a41037c0 0x7f16a410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:14.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.479+0000 7f16aaa9e640 1 --2- 172.21.15.172:0/2983547591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a4102e00 0x7f16a4103280 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:14.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.479+0000 7f16aaa9e640 1 --2- 172.21.15.172:0/2983547591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a4101c00 0x7f16a4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:14.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.479+0000 7f16aaa9e640 1 -- 172.21.15.172:0/2983547591 >> 172.21.15.172:0/2983547591 conn(0x7f16a40fd390 msgr2=0x7f16a40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:14.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.479+0000 7f16aaa9e640 1 -- 172.21.15.172:0/2983547591 shutdown_connections 2024-09-10T08:46:14.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.479+0000 7f16aaa9e640 1 -- 172.21.15.172:0/2983547591 wait complete. 2024-09-10T08:46:14.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.479+0000 7f16aaa9e640 1 Processor -- start 2024-09-10T08:46:14.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.479+0000 7f16aaa9e640 1 -- start start 2024-09-10T08:46:14.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16aaa9e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16a4101c00 0x7f16a4075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:14.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16aaa9e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a4102e00 0x7f16a40745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:14.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16aaa9e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a41037c0 0x7f16a4074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:14.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16aaa9e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16a410d710 con 0x7f16a41037c0 2024-09-10T08:46:14.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16aaa9e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16a410d590 con 0x7f16a4102e00 2024-09-10T08:46:14.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16aaa9e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f16a410d890 con 0x7f16a4101c00 2024-09-10T08:46:14.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16a903b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a41037c0 0x7f16a4074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:14.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16a883a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16a4101c00 0x7f16a4075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:14.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16a903b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a41037c0 0x7f16a4074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48800/0 (socket says 172.21.15.172:48800) 2024-09-10T08:46:14.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16a903b640 1 -- 172.21.15.172:0/1057706490 learned_addr learned my addr 172.21.15.172:0/1057706490 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:14.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16a883a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16a4101c00 0x7f16a4075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:34480/0 (socket says 172.21.15.172:34480) 2024-09-10T08:46:14.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16a3fff640 1 --2- 172.21.15.172:0/1057706490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a4102e00 0x7f16a40745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:14.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16a903b640 1 -- 172.21.15.172:0/1057706490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16a4101c00 msgr2=0x7f16a4075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:14.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16a903b640 1 --2- 172.21.15.172:0/1057706490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16a4101c00 0x7f16a4075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:14.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16a903b640 1 -- 172.21.15.172:0/1057706490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a4102e00 msgr2=0x7f16a40745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:14.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16a903b640 1 --2- 172.21.15.172:0/1057706490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a4102e00 0x7f16a40745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:14.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16a903b640 1 -- 172.21.15.172:0/1057706490 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a40753e0 con 0x7f16a41037c0 2024-09-10T08:46:14.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16a903b640 1 --2- 172.21.15.172:0/1057706490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a41037c0 0x7f16a4074b20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f169c00cce0 tx=0x7f169c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:14.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16a1ffb640 1 -- 172.21.15.172:0/1057706490 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c00f450 con 0x7f16a41037c0 2024-09-10T08:46:14.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16a1ffb640 1 -- 172.21.15.172:0/1057706490 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f169c00faa0 con 0x7f16a41037c0 2024-09-10T08:46:14.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16aaa9e640 1 -- 172.21.15.172:0/1057706490 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a41a1f20 con 0x7f16a41037c0 2024-09-10T08:46:14.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.480+0000 7f16a1ffb640 1 -- 172.21.15.172:0/1057706490 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c018b70 con 0x7f16a41037c0 2024-09-10T08:46:14.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.481+0000 7f16aaa9e640 1 -- 172.21.15.172:0/1057706490 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a41a2380 con 0x7f16a41037c0 2024-09-10T08:46:14.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.482+0000 7f16aaa9e640 1 -- 172.21.15.172:0/1057706490 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a4109020 con 0x7f16a41037c0 2024-09-10T08:46:14.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.483+0000 7f16a1ffb640 1 -- 172.21.15.172:0/1057706490 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f169c018d10 con 0x7f16a41037c0 2024-09-10T08:46:14.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.483+0000 7f16a1ffb640 1 --2- 172.21.15.172:0/1057706490 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16940a2c10 0x7f16940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:14.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.484+0000 7f16a883a640 1 --2- 172.21.15.172:0/1057706490 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16940a2c10 0x7f16940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:14.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.484+0000 7f16a1ffb640 1 -- 172.21.15.172:0/1057706490 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f169c0cf920 con 0x7f16a41037c0 2024-09-10T08:46:14.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.484+0000 7f16a883a640 1 --2- 172.21.15.172:0/1057706490 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16940a2c10 0x7f16940a50d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f16a4102c60 tx=0x7f169800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:14.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.487+0000 7f16a1ffb640 1 -- 172.21.15.172:0/1057706490 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f169c014030 con 0x7f16a41037c0 2024-09-10T08:46:14.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.660+0000 7f16aaa9e640 1 -- 172.21.15.172:0/1057706490 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f16a4062630 con 0x7f16940a2c10 2024-09-10T08:46:14.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.662+0000 7f16a1ffb640 1 -- 172.21.15.172:0/1057706490 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f16a4062630 con 0x7f16940a2c10 2024-09-10T08:46:14.663 INFO:teuthology.orchestra.run.smithi172.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T08:46:14.663 INFO:teuthology.orchestra.run.smithi172.stdout:smithi172 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233005M400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T08:46:14.663 INFO:teuthology.orchestra.run.smithi172.stdout:smithi174 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050074400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T08:46:14.663 INFO:teuthology.orchestra.run.smithi172.stdout:smithi191 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303008M400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-10T08:46:14.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.664+0000 7f16aaa9e640 1 -- 172.21.15.172:0/1057706490 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16940a2c10 msgr2=0x7f16940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:14.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.664+0000 7f16aaa9e640 1 --2- 172.21.15.172:0/1057706490 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16940a2c10 0x7f16940a50d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f16a4102c60 tx=0x7f169800b390 comp rx=0 tx=0).stop 2024-09-10T08:46:14.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.664+0000 7f16aaa9e640 1 -- 172.21.15.172:0/1057706490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a41037c0 msgr2=0x7f16a4074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:14.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.664+0000 7f16aaa9e640 1 --2- 172.21.15.172:0/1057706490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a41037c0 0x7f16a4074b20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f169c00cce0 tx=0x7f169c00a620 comp rx=0 tx=0).stop 2024-09-10T08:46:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.664+0000 7f16aaa9e640 1 -- 172.21.15.172:0/1057706490 shutdown_connections 2024-09-10T08:46:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.664+0000 7f16aaa9e640 1 --2- 172.21.15.172:0/1057706490 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16940a2c10 0x7f16940a50d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.664+0000 7f16aaa9e640 1 --2- 172.21.15.172:0/1057706490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a41037c0 0x7f16a4074b20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.664+0000 7f16aaa9e640 1 --2- 172.21.15.172:0/1057706490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a4102e00 0x7f16a40745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.664+0000 7f16aaa9e640 1 --2- 172.21.15.172:0/1057706490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16a4101c00 0x7f16a4075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.664+0000 7f16aaa9e640 1 -- 172.21.15.172:0/1057706490 >> 172.21.15.172:0/1057706490 conn(0x7f16a40fd390 msgr2=0x7f16a41061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.665+0000 7f16aaa9e640 1 -- 172.21.15.172:0/1057706490 shutdown_connections 2024-09-10T08:46:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:14.665+0000 7f16aaa9e640 1 -- 172.21.15.172:0/1057706490 wait complete. 2024-09-10T08:46:15.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:14 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:15.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:14 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:15.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:14 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:46:15.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:14 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:46:15.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:14 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:15.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:14 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:15.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:14 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:15.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:14 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:46:15.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:14 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:46:15.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:14 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:15.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:14 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:15.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:14 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:15.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:14 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:46:15.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:14 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:46:15.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:14 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:46:15.285 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T08:46:15.294 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi172.front.sepia.ceph.com 2024-09-10T08:46:15.294 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T08:46:15.544 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:15.933 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:15 smithi172 ceph-mon[30854]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:46:15.934 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:15 smithi172 ceph-mon[30854]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:16.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.012+0000 7f8f1f363640 1 -- 172.21.15.172:0/3829932246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f181026c0 msgr2=0x7f8f18108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:16.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.012+0000 7f8f1f363640 1 --2- 172.21.15.172:0/3829932246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f181026c0 0x7f8f18108f50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009900 tx=0x7f8f1002a410 comp rx=0 tx=0).stop 2024-09-10T08:46:16.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.012+0000 7f8f1f363640 1 -- 172.21.15.172:0/3829932246 shutdown_connections 2024-09-10T08:46:16.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.012+0000 7f8f1f363640 1 --2- 172.21.15.172:0/3829932246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f181026c0 0x7f8f18108f50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:16.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.012+0000 7f8f1f363640 1 --2- 172.21.15.172:0/3829932246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f18101d00 0x7f8f18102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:16.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.012+0000 7f8f1f363640 1 --2- 172.21.15.172:0/3829932246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f18100b00 0x7f8f18100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:16.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.012+0000 7f8f1f363640 1 -- 172.21.15.172:0/3829932246 >> 172.21.15.172:0/3829932246 conn(0x7f8f180fc290 msgr2=0x7f8f180fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:16.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.012+0000 7f8f1f363640 1 -- 172.21.15.172:0/3829932246 shutdown_connections 2024-09-10T08:46:16.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.012+0000 7f8f1f363640 1 -- 172.21.15.172:0/3829932246 wait complete. 2024-09-10T08:46:16.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1f363640 1 Processor -- start 2024-09-10T08:46:16.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1f363640 1 -- start start 2024-09-10T08:46:16.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1f363640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f18100b00 0x7f8f18197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:16.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1f363640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18101d00 0x7f8f18198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:16.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1f363640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f181026c0 0x7f8f1819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:16.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1f363640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1810c630 con 0x7f8f18101d00 2024-09-10T08:46:16.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1f363640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1810c4b0 con 0x7f8f18100b00 2024-09-10T08:46:16.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1f363640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1810c7b0 con 0x7f8f181026c0 2024-09-10T08:46:16.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1c8fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18101d00 0x7f8f18198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:16.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1c8fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18101d00 0x7f8f18198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48826/0 (socket says 172.21.15.172:48826) 2024-09-10T08:46:16.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1c8fe640 1 -- 172.21.15.172:0/2007651601 learned_addr learned my addr 172.21.15.172:0/2007651601 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:16.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1d0ff640 1 --2- 172.21.15.172:0/2007651601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f18100b00 0x7f8f18197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:16.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1d900640 1 --2- 172.21.15.172:0/2007651601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f181026c0 0x7f8f1819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:16.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1c8fe640 1 -- 172.21.15.172:0/2007651601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f181026c0 msgr2=0x7f8f1819f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:16.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1c8fe640 1 --2- 172.21.15.172:0/2007651601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f181026c0 0x7f8f1819f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:16.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1c8fe640 1 -- 172.21.15.172:0/2007651601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f18100b00 msgr2=0x7f8f18197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:16.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1c8fe640 1 --2- 172.21.15.172:0/2007651601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f18100b00 0x7f8f18197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:16.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1c8fe640 1 -- 172.21.15.172:0/2007651601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1819fa00 con 0x7f8f18101d00 2024-09-10T08:46:16.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.013+0000 7f8f1d0ff640 1 --2- 172.21.15.172:0/2007651601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f18100b00 0x7f8f18197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:16.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.014+0000 7f8f1c8fe640 1 --2- 172.21.15.172:0/2007651601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18101d00 0x7f8f18198310 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00c890 tx=0x7f8f0c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:16.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.014+0000 7f8f167fc640 1 -- 172.21.15.172:0/2007651601 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c004220 con 0x7f8f18101d00 2024-09-10T08:46:16.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.014+0000 7f8f167fc640 1 -- 172.21.15.172:0/2007651601 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f0c0043c0 con 0x7f8f18101d00 2024-09-10T08:46:16.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.014+0000 7f8f1f363640 1 -- 172.21.15.172:0/2007651601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1819fcf0 con 0x7f8f18101d00 2024-09-10T08:46:16.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.014+0000 7f8f167fc640 1 -- 172.21.15.172:0/2007651601 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c004a00 con 0x7f8f18101d00 2024-09-10T08:46:16.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.014+0000 7f8f1f363640 1 -- 172.21.15.172:0/2007651601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f181a01d0 con 0x7f8f18101d00 2024-09-10T08:46:16.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.015+0000 7f8f1f363640 1 -- 172.21.15.172:0/2007651601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f181a0460 con 0x7f8f18101d00 2024-09-10T08:46:16.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.016+0000 7f8f167fc640 1 -- 172.21.15.172:0/2007651601 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8f0c004ba0 con 0x7f8f18101d00 2024-09-10T08:46:16.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.017+0000 7f8f167fc640 1 --2- 172.21.15.172:0/2007651601 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8f000a2c40 0x7f8f000a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:16.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.017+0000 7f8f1d0ff640 1 --2- 172.21.15.172:0/2007651601 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8f000a2c40 0x7f8f000a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:16.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.017+0000 7f8f167fc640 1 -- 172.21.15.172:0/2007651601 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f8f0c0cf660 con 0x7f8f18101d00 2024-09-10T08:46:16.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.017+0000 7f8f1d0ff640 1 --2- 172.21.15.172:0/2007651601 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8f000a2c40 0x7f8f000a5100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8f18101b60 tx=0x7f8f08009400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:16.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.020+0000 7f8f167fc640 1 -- 172.21.15.172:0/2007651601 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f0c014030 con 0x7f8f18101d00 2024-09-10T08:46:16.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:15 smithi191 ceph-mon[35314]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:46:16.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:15 smithi191 ceph-mon[35314]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:16.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.219+0000 7f8f1f363640 1 -- 172.21.15.172:0/2007651601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f8f181a0630 con 0x7f8f18101d00 2024-09-10T08:46:16.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.220+0000 7f8f167fc640 1 -- 172.21.15.172:0/2007651601 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3148 (secure 0 0 0) 0x7f8f0c09c120 con 0x7f8f18101d00 2024-09-10T08:46:16.221 INFO:teuthology.orchestra.run.smithi172.stdout:e28 2024-09-10T08:46:16.221 INFO:teuthology.orchestra.run.smithi172.stdout:btime 2024-09-10T08:44:55:226722+0000 2024-09-10T08:46:16.221 INFO:teuthology.orchestra.run.smithi172.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T08:46:16.221 INFO:teuthology.orchestra.run.smithi172.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T08:46:16.221 INFO:teuthology.orchestra.run.smithi172.stdout:legacy client fscid: 1 2024-09-10T08:46:16.221 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:16.221 INFO:teuthology.orchestra.run.smithi172.stdout:Filesystem 'cephfs' (1) 2024-09-10T08:46:16.222 INFO:teuthology.orchestra.run.smithi172.stdout:fs_name cephfs 2024-09-10T08:46:16.222 INFO:teuthology.orchestra.run.smithi172.stdout:epoch 28 2024-09-10T08:46:16.222 INFO:teuthology.orchestra.run.smithi172.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T08:46:16.222 INFO:teuthology.orchestra.run.smithi172.stdout:created 2024-09-10T08:44:11.310930+0000 2024-09-10T08:46:16.222 INFO:teuthology.orchestra.run.smithi172.stdout:modified 2024-09-10T08:44:55.226705+0000 2024-09-10T08:46:16.222 INFO:teuthology.orchestra.run.smithi172.stdout:tableserver 0 2024-09-10T08:46:16.222 INFO:teuthology.orchestra.run.smithi172.stdout:root 0 2024-09-10T08:46:16.222 INFO:teuthology.orchestra.run.smithi172.stdout:session_timeout 300 2024-09-10T08:46:16.222 INFO:teuthology.orchestra.run.smithi172.stdout:session_autoclose 300 2024-09-10T08:46:16.222 INFO:teuthology.orchestra.run.smithi172.stdout:max_file_size 1099511627776 2024-09-10T08:46:16.222 INFO:teuthology.orchestra.run.smithi172.stdout:max_xattr_size 65536 2024-09-10T08:46:16.222 INFO:teuthology.orchestra.run.smithi172.stdout:required_client_features {} 2024-09-10T08:46:16.222 INFO:teuthology.orchestra.run.smithi172.stdout:last_failure 0 2024-09-10T08:46:16.223 INFO:teuthology.orchestra.run.smithi172.stdout:last_failure_osd_epoch 0 2024-09-10T08:46:16.223 INFO:teuthology.orchestra.run.smithi172.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T08:46:16.223 INFO:teuthology.orchestra.run.smithi172.stdout:max_mds 5 2024-09-10T08:46:16.223 INFO:teuthology.orchestra.run.smithi172.stdout:in 0,1,2,3,4 2024-09-10T08:46:16.223 INFO:teuthology.orchestra.run.smithi172.stdout:up {0=24488,1=24458,2=24457,3=24416,4=24406} 2024-09-10T08:46:16.223 INFO:teuthology.orchestra.run.smithi172.stdout:failed 2024-09-10T08:46:16.223 INFO:teuthology.orchestra.run.smithi172.stdout:damaged 2024-09-10T08:46:16.223 INFO:teuthology.orchestra.run.smithi172.stdout:stopped 2024-09-10T08:46:16.223 INFO:teuthology.orchestra.run.smithi172.stdout:data_pools [3,4] 2024-09-10T08:46:16.223 INFO:teuthology.orchestra.run.smithi172.stdout:metadata_pool 2 2024-09-10T08:46:16.223 INFO:teuthology.orchestra.run.smithi172.stdout:inline_data disabled 2024-09-10T08:46:16.223 INFO:teuthology.orchestra.run.smithi172.stdout:balancer 2024-09-10T08:46:16.223 INFO:teuthology.orchestra.run.smithi172.stdout:bal_rank_mask -1 2024-09-10T08:46:16.224 INFO:teuthology.orchestra.run.smithi172.stdout:standby_count_wanted 0 2024-09-10T08:46:16.224 INFO:teuthology.orchestra.run.smithi172.stdout:qdb_cluster leader: 24488 members: 24488,24457,24416,24458,24406 2024-09-10T08:46:16.224 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b{0:24488} state up:active seq 4 addr [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:46:16.224 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c{0:24439} state up:standby-replay seq 1 addr [v2:172.21.15.191:6836/1802631256,v1:172.21.15.191:6837/1802631256] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:46:16.224 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.h{1:24458} state up:active seq 17 addr [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:46:16.225 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.e{1:24434} state up:standby-replay seq 1 addr [v2:172.21.15.174:6834/3197756201,v1:172.21.15.174:6835/3197756201] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:46:16.225 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.i{2:24457} state up:active seq 16 addr [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:46:16.225 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.f{2:24433} state up:standby-replay seq 1 addr [v2:172.21.15.191:6834/1774553543,v1:172.21.15.191:6835/1774553543] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:46:16.225 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.k{3:24416} state up:active seq 22 addr [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:46:16.225 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a{3:14622} state up:standby-replay seq 1 addr [v2:172.21.15.172:6840/176040463,v1:172.21.15.172:6841/176040463] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:46:16.225 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.l{4:24406} state up:active seq 22 addr [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:46:16.226 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.g{4:14616} state up:standby-replay seq 1 addr [v2:172.21.15.172:6838/1621056220,v1:172.21.15.172:6839/1621056220] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:46:16.226 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:16.226 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:16.226 INFO:teuthology.orchestra.run.smithi172.stdout:Standby daemons: 2024-09-10T08:46:16.226 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:16.226 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.j{-1:14547} state up:standby seq 1 addr [v2:172.21.15.172:6834/2272815205,v1:172.21.15.172:6835/2272815205] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:46:16.226 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d{-1:14559} state up:standby seq 1 addr [v2:172.21.15.172:6836/2122290739,v1:172.21.15.172:6837/2122290739] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T08:46:16.227 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 28 2024-09-10T08:46:16.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.223+0000 7f8f1f363640 1 -- 172.21.15.172:0/2007651601 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8f000a2c40 msgr2=0x7f8f000a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:16.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.223+0000 7f8f1f363640 1 --2- 172.21.15.172:0/2007651601 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8f000a2c40 0x7f8f000a5100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8f18101b60 tx=0x7f8f08009400 comp rx=0 tx=0).stop 2024-09-10T08:46:16.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.223+0000 7f8f1f363640 1 -- 172.21.15.172:0/2007651601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18101d00 msgr2=0x7f8f18198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:16.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.223+0000 7f8f1f363640 1 --2- 172.21.15.172:0/2007651601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18101d00 0x7f8f18198310 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00c890 tx=0x7f8f0c00cd50 comp rx=0 tx=0).stop 2024-09-10T08:46:16.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.223+0000 7f8f1f363640 1 -- 172.21.15.172:0/2007651601 shutdown_connections 2024-09-10T08:46:16.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.223+0000 7f8f1f363640 1 --2- 172.21.15.172:0/2007651601 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8f000a2c40 0x7f8f000a5100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:16.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.223+0000 7f8f1f363640 1 --2- 172.21.15.172:0/2007651601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f181026c0 0x7f8f1819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:16.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.223+0000 7f8f1f363640 1 --2- 172.21.15.172:0/2007651601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8f18101d00 0x7f8f18198310 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:16.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.223+0000 7f8f1f363640 1 --2- 172.21.15.172:0/2007651601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f18100b00 0x7f8f18197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:16.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.223+0000 7f8f1f363640 1 -- 172.21.15.172:0/2007651601 >> 172.21.15.172:0/2007651601 conn(0x7f8f180fc290 msgr2=0x7f8f1806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:16.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.223+0000 7f8f1f363640 1 -- 172.21.15.172:0/2007651601 shutdown_connections 2024-09-10T08:46:16.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:16.223+0000 7f8f1f363640 1 -- 172.21.15.172:0/2007651601 wait complete. 2024-09-10T08:46:16.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:15 smithi174 ceph-mon[34068]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T08:46:16.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:15 smithi174 ceph-mon[34068]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T08:46:16.907 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:16 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2007651601' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T08:46:16.936 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-10T08:46:17.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:16 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2007651601' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T08:46:17.210 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:17.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:16 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2007651601' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T08:46:17.863 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:17 smithi172 ceph-mon[30854]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T08:46:17.863 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-10T08:46:17.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.873+0000 7f3753333640 1 -- 172.21.15.172:0/3580295359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f374c107920 msgr2=0x7f374c109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:17.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.873+0000 7f3753333640 1 --2- 172.21.15.172:0/3580295359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f374c107920 0x7f374c109d10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3744009900 tx=0x7f374402a410 comp rx=0 tx=0).stop 2024-09-10T08:46:17.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.873+0000 7f3753333640 1 -- 172.21.15.172:0/3580295359 shutdown_connections 2024-09-10T08:46:17.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.873+0000 7f3753333640 1 --2- 172.21.15.172:0/3580295359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f374c107920 0x7f374c109d10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.873+0000 7f3753333640 1 --2- 172.21.15.172:0/3580295359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f374c104ff0 0x7f374c1073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.873+0000 7f3753333640 1 --2- 172.21.15.172:0/3580295359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f374c1026c0 0x7f374c104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.873+0000 7f3753333640 1 -- 172.21.15.172:0/3580295359 >> 172.21.15.172:0/3580295359 conn(0x7f374c0fc280 msgr2=0x7f374c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:17.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.873+0000 7f3753333640 1 -- 172.21.15.172:0/3580295359 shutdown_connections 2024-09-10T08:46:17.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.873+0000 7f3753333640 1 -- 172.21.15.172:0/3580295359 wait complete. 2024-09-10T08:46:17.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.873+0000 7f3753333640 1 Processor -- start 2024-09-10T08:46:17.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.873+0000 7f3753333640 1 -- start start 2024-09-10T08:46:17.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.874+0000 7f3753333640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f374c1026c0 0x7f374c197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.874+0000 7f3753333640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f374c104ff0 0x7f374c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.874+0000 7f3753333640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f374c107920 0x7f374c19f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.874+0000 7f3753333640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f374c10d730 con 0x7f374c107920 2024-09-10T08:46:17.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.874+0000 7f3753333640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f374c10d5b0 con 0x7f374c1026c0 2024-09-10T08:46:17.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.874+0000 7f3753333640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f374c10d8b0 con 0x7f374c104ff0 2024-09-10T08:46:17.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.874+0000 7f37518d0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f374c107920 0x7f374c19f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.874+0000 7f37518d0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f374c107920 0x7f374c19f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37880/0 (socket says 172.21.15.172:37880) 2024-09-10T08:46:17.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.874+0000 7f37518d0640 1 -- 172.21.15.172:0/1625527503 learned_addr learned my addr 172.21.15.172:0/1625527503 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:17.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.874+0000 7f37508ce640 1 --2- 172.21.15.172:0/1625527503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f374c104ff0 0x7f374c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.874+0000 7f37510cf640 1 --2- 172.21.15.172:0/1625527503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f374c1026c0 0x7f374c197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.874+0000 7f37518d0640 1 -- 172.21.15.172:0/1625527503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f374c104ff0 msgr2=0x7f374c1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:17.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.874+0000 7f37518d0640 1 --2- 172.21.15.172:0/1625527503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f374c104ff0 0x7f374c1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.874+0000 7f37518d0640 1 -- 172.21.15.172:0/1625527503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f374c1026c0 msgr2=0x7f374c197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:17.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.874+0000 7f37518d0640 1 --2- 172.21.15.172:0/1625527503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f374c1026c0 0x7f374c197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:17.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.874+0000 7f37518d0640 1 -- 172.21.15.172:0/1625527503 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374c19f9d0 con 0x7f374c107920 2024-09-10T08:46:17.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.874+0000 7f37518d0640 1 --2- 172.21.15.172:0/1625527503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f374c107920 0x7f374c19f330 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3744005940 tx=0x7f37440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:17.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.875+0000 7f374a7fc640 1 -- 172.21.15.172:0/1625527503 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374403f030 con 0x7f374c107920 2024-09-10T08:46:17.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.875+0000 7f374a7fc640 1 -- 172.21.15.172:0/1625527503 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37440318d0 con 0x7f374c107920 2024-09-10T08:46:17.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.875+0000 7f3753333640 1 -- 172.21.15.172:0/1625527503 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374c19fc60 con 0x7f374c107920 2024-09-10T08:46:17.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.875+0000 7f374a7fc640 1 -- 172.21.15.172:0/1625527503 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3744035070 con 0x7f374c107920 2024-09-10T08:46:17.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.875+0000 7f3753333640 1 -- 172.21.15.172:0/1625527503 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f374c1a0140 con 0x7f374c107920 2024-09-10T08:46:17.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.876+0000 7f3753333640 1 -- 172.21.15.172:0/1625527503 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f374c0fdd70 con 0x7f374c107920 2024-09-10T08:46:17.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.877+0000 7f374a7fc640 1 -- 172.21.15.172:0/1625527503 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3744042030 con 0x7f374c107920 2024-09-10T08:46:17.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.878+0000 7f374a7fc640 1 --2- 172.21.15.172:0/1625527503 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f37340a2b80 0x7f37340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:17.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.878+0000 7f37510cf640 1 --2- 172.21.15.172:0/1625527503 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f37340a2b80 0x7f37340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:17.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.878+0000 7f374a7fc640 1 -- 172.21.15.172:0/1625527503 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9100+0+0 (secure 0 0 0) 0x7f37440ef8a0 con 0x7f374c107920 2024-09-10T08:46:17.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.878+0000 7f37510cf640 1 --2- 172.21.15.172:0/1625527503 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f37340a2b80 0x7f37340a5040 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f374c199290 tx=0x7f373c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:17.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:17.882+0000 7f374a7fc640 1 -- 172.21.15.172:0/1625527503 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37440bc4a0 con 0x7f374c107920 2024-09-10T08:46:18.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:17 smithi191 ceph-mon[35314]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T08:46:18.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-10T08:46:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:18.051+0000 7f3753333640 1 -- 172.21.15.172:0/1625527503 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f374c062df0 con 0x7f374c107920 2024-09-10T08:46:18.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:18.052+0000 7f374a7fc640 1 -- 172.21.15.172:0/1625527503 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) ==== 56+0+5631 (secure 0 0 0) 0x7f37440bbda0 con 0x7f374c107920 2024-09-10T08:46:18.053 INFO:teuthology.orchestra.run.smithi172.stdout:epoch 91 2024-09-10T08:46:18.053 INFO:teuthology.orchestra.run.smithi172.stdout:fsid a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T08:46:18.053 INFO:teuthology.orchestra.run.smithi172.stdout:created 2024-09-10T08:36:33.083529+0000 2024-09-10T08:46:18.053 INFO:teuthology.orchestra.run.smithi172.stdout:modified 2024-09-10T08:46:17.793498+0000 2024-09-10T08:46:18.053 INFO:teuthology.orchestra.run.smithi172.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T08:46:18.053 INFO:teuthology.orchestra.run.smithi172.stdout:crush_version 27 2024-09-10T08:46:18.053 INFO:teuthology.orchestra.run.smithi172.stdout:full_ratio 0.95 2024-09-10T08:46:18.053 INFO:teuthology.orchestra.run.smithi172.stdout:backfillfull_ratio 0.9 2024-09-10T08:46:18.054 INFO:teuthology.orchestra.run.smithi172.stdout:nearfull_ratio 0.85 2024-09-10T08:46:18.054 INFO:teuthology.orchestra.run.smithi172.stdout:require_min_compat_client luminous 2024-09-10T08:46:18.054 INFO:teuthology.orchestra.run.smithi172.stdout:min_compat_client luminous 2024-09-10T08:46:18.054 INFO:teuthology.orchestra.run.smithi172.stdout:require_osd_release squid 2024-09-10T08:46:18.054 INFO:teuthology.orchestra.run.smithi172.stdout:stretch_mode_enabled false 2024-09-10T08:46:18.054 INFO:teuthology.orchestra.run.smithi172.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-10T08:46:18.054 INFO:teuthology.orchestra.run.smithi172.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-10T08:46:18.054 INFO:teuthology.orchestra.run.smithi172.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-10T08:46:18.054 INFO:teuthology.orchestra.run.smithi172.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-10T08:46:18.054 INFO:teuthology.orchestra.run.smithi172.stdout:max_osd 12 2024-09-10T08:46:18.054 INFO:teuthology.orchestra.run.smithi172.stdout:osd.0 up in weight 1 up_from 8 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6802/2678426952,v1:172.21.15.172:6803/2678426952] [v2:172.21.15.172:6804/2678426952,v1:172.21.15.172:6805/2678426952] exists,up c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12 2024-09-10T08:46:18.054 INFO:teuthology.orchestra.run.smithi172.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6800/3279661439,v1:172.21.15.174:6801/3279661439] [v2:172.21.15.174:6802/3279661439,v1:172.21.15.174:6803/3279661439] exists,up 55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71 2024-09-10T08:46:18.055 INFO:teuthology.orchestra.run.smithi172.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6800/1963796290,v1:172.21.15.191:6801/1963796290] [v2:172.21.15.191:6802/1963796290,v1:172.21.15.191:6803/1963796290] exists,up f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4 2024-09-10T08:46:18.055 INFO:teuthology.orchestra.run.smithi172.stdout:osd.3 up in weight 1 up_from 25 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6810/869546480,v1:172.21.15.172:6811/869546480] [v2:172.21.15.172:6812/869546480,v1:172.21.15.172:6813/869546480] exists,up 15ecca3f-bc6b-4a06-b283-b515384beb19 2024-09-10T08:46:18.055 INFO:teuthology.orchestra.run.smithi172.stdout:osd.4 up in weight 1 up_from 33 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6808/961007949,v1:172.21.15.174:6809/961007949] [v2:172.21.15.174:6810/961007949,v1:172.21.15.174:6811/961007949] exists,up 8c2c1d40-72ad-4a15-ab72-54871c7e2427 2024-09-10T08:46:18.055 INFO:teuthology.orchestra.run.smithi172.stdout:osd.5 up in weight 1 up_from 38 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6808/4216938371,v1:172.21.15.191:6809/4216938371] [v2:172.21.15.191:6810/4216938371,v1:172.21.15.191:6811/4216938371] exists,up 4de45a1d-ef0e-4bfe-ba15-5eeeb8806155 2024-09-10T08:46:18.055 INFO:teuthology.orchestra.run.smithi172.stdout:osd.6 up in weight 1 up_from 45 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6818/3210762009,v1:172.21.15.172:6819/3210762009] [v2:172.21.15.172:6820/3210762009,v1:172.21.15.172:6821/3210762009] exists,up 60a9938a-f696-4db2-932c-402997543b6d 2024-09-10T08:46:18.055 INFO:teuthology.orchestra.run.smithi172.stdout:osd.7 up in weight 1 up_from 50 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6816/1461923581,v1:172.21.15.174:6817/1461923581] [v2:172.21.15.174:6818/1461923581,v1:172.21.15.174:6819/1461923581] exists,up 7f0202f3-971e-472f-9c4d-ae3c24d39f2c 2024-09-10T08:46:18.055 INFO:teuthology.orchestra.run.smithi172.stdout:osd.8 up in weight 1 up_from 57 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6816/3675210430,v1:172.21.15.191:6817/3675210430] [v2:172.21.15.191:6818/3675210430,v1:172.21.15.191:6819/3675210430] exists,up b4e7327f-ca9f-4833-9cd3-b8529b3d9761 2024-09-10T08:46:18.055 INFO:teuthology.orchestra.run.smithi172.stdout:osd.9 up in weight 1 up_from 64 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6826/1181268619,v1:172.21.15.172:6827/1181268619] [v2:172.21.15.172:6828/1181268619,v1:172.21.15.172:6829/1181268619] exists,up 9f6153f2-f9db-4c00-a659-c932628016f4 2024-09-10T08:46:18.055 INFO:teuthology.orchestra.run.smithi172.stdout:osd.10 up in weight 1 up_from 69 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6824/1238812525,v1:172.21.15.174:6825/1238812525] [v2:172.21.15.174:6826/1238812525,v1:172.21.15.174:6827/1238812525] exists,up 040a013a-4c3e-442f-bea6-12cbe517716d 2024-09-10T08:46:18.055 INFO:teuthology.orchestra.run.smithi172.stdout:osd.11 up in weight 1 up_from 76 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6824/2162719997,v1:172.21.15.191:6825/2162719997] [v2:172.21.15.191:6826/2162719997,v1:172.21.15.191:6827/2162719997] exists,up ae6535bd-1510-4bf3-bf4b-1a2a256add43 2024-09-10T08:46:18.055 INFO:teuthology.orchestra.run.smithi172.stdout:pg_upmap_items 2.8 [2,11] 2024-09-10T08:46:18.055 INFO:teuthology.orchestra.run.smithi172.stdout:pg_upmap_items 2.a [2,11,8,1] 2024-09-10T08:46:18.056 INFO:teuthology.orchestra.run.smithi172.stdout:pg_upmap_items 3.1 [4,9] 2024-09-10T08:46:18.056 INFO:teuthology.orchestra.run.smithi172.stdout:pg_upmap_items 3.6 [4,6] 2024-09-10T08:46:18.056 INFO:teuthology.orchestra.run.smithi172.stdout:pg_upmap_items 3.e [5,9] 2024-09-10T08:46:18.056 INFO:teuthology.orchestra.run.smithi172.stdout:pg_upmap_items 3.2c [4,9] 2024-09-10T08:46:18.056 INFO:teuthology.orchestra.run.smithi172.stdout:pg_upmap_items 4.2 [9,1] 2024-09-10T08:46:18.056 INFO:teuthology.orchestra.run.smithi172.stdout:pg_upmap_items 4.6 [5,8] 2024-09-10T08:46:18.056 INFO:teuthology.orchestra.run.smithi172.stdout:pg_upmap_items 4.b [9,8] 2024-09-10T08:46:18.056 INFO:teuthology.orchestra.run.smithi172.stdout:pg_upmap_items 4.17 [9,8] 2024-09-10T08:46:18.056 INFO:teuthology.orchestra.run.smithi172.stdout:blocklist 172.21.15.172:0/977245917 expires 2024-09-11T08:37:17.019272+0000 2024-09-10T08:46:18.056 INFO:teuthology.orchestra.run.smithi172.stdout:blocklist 172.21.15.172:6801/2333617420 expires 2024-09-11T08:37:17.019272+0000 2024-09-10T08:46:18.056 INFO:teuthology.orchestra.run.smithi172.stdout:blocklist 172.21.15.172:0/3919952878 expires 2024-09-11T08:36:54.887256+0000 2024-09-10T08:46:18.056 INFO:teuthology.orchestra.run.smithi172.stdout:blocklist 172.21.15.172:6800/2333617420 expires 2024-09-11T08:37:17.019272+0000 2024-09-10T08:46:18.056 INFO:teuthology.orchestra.run.smithi172.stdout:blocklist 172.21.15.172:0/1525385361 expires 2024-09-11T08:36:54.887256+0000 2024-09-10T08:46:18.056 INFO:teuthology.orchestra.run.smithi172.stdout:blocklist 172.21.15.172:0/2043423408 expires 2024-09-11T08:37:17.019272+0000 2024-09-10T08:46:18.057 INFO:teuthology.orchestra.run.smithi172.stdout:blocklist 172.21.15.172:6801/2639090029 expires 2024-09-11T08:36:54.887256+0000 2024-09-10T08:46:18.057 INFO:teuthology.orchestra.run.smithi172.stdout:blocklist 172.21.15.172:0/839344299 expires 2024-09-11T08:37:17.019272+0000 2024-09-10T08:46:18.057 INFO:teuthology.orchestra.run.smithi172.stdout:blocklist 172.21.15.172:6800/2639090029 expires 2024-09-11T08:36:54.887256+0000 2024-09-10T08:46:18.057 INFO:teuthology.orchestra.run.smithi172.stdout:blocklist 172.21.15.172:0/3251817286 expires 2024-09-11T08:36:54.887256+0000 2024-09-10T08:46:18.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:18.054+0000 7f3753333640 1 -- 172.21.15.172:0/1625527503 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f37340a2b80 msgr2=0x7f37340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:18.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:18.054+0000 7f3753333640 1 --2- 172.21.15.172:0/1625527503 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f37340a2b80 0x7f37340a5040 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f374c199290 tx=0x7f373c009500 comp rx=0 tx=0).stop 2024-09-10T08:46:18.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:18.054+0000 7f3753333640 1 -- 172.21.15.172:0/1625527503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f374c107920 msgr2=0x7f374c19f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:18.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:18.054+0000 7f3753333640 1 --2- 172.21.15.172:0/1625527503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f374c107920 0x7f374c19f330 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3744005940 tx=0x7f37440042f0 comp rx=0 tx=0).stop 2024-09-10T08:46:18.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:18.055+0000 7f3753333640 1 -- 172.21.15.172:0/1625527503 shutdown_connections 2024-09-10T08:46:18.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:18.055+0000 7f3753333640 1 --2- 172.21.15.172:0/1625527503 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f37340a2b80 0x7f37340a5040 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:18.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:18.055+0000 7f3753333640 1 --2- 172.21.15.172:0/1625527503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f374c107920 0x7f374c19f330 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:18.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:18.055+0000 7f3753333640 1 --2- 172.21.15.172:0/1625527503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f374c104ff0 0x7f374c1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:18.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:18.055+0000 7f3753333640 1 --2- 172.21.15.172:0/1625527503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f374c1026c0 0x7f374c197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:18.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:18.055+0000 7f3753333640 1 -- 172.21.15.172:0/1625527503 >> 172.21.15.172:0/1625527503 conn(0x7f374c0fc280 msgr2=0x7f374c102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:18.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:18.055+0000 7f3753333640 1 -- 172.21.15.172:0/1625527503 shutdown_connections 2024-09-10T08:46:18.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:18.055+0000 7f3753333640 1 -- 172.21.15.172:0/1625527503 wait complete. 2024-09-10T08:46:18.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:17 smithi174 ceph-mon[34068]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T08:46:18.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-09-10T08:46:18.706 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-10T08:46:18.718 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:46:18.971 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:19.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-10T08:46:19.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:18 smithi172 ceph-mon[30854]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T08:46:19.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:18 smithi172 ceph-mon[30854]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T08:46:19.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:18 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1625527503' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T08:46:19.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-10T08:46:19.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:18 smithi174 ceph-mon[34068]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T08:46:19.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:18 smithi174 ceph-mon[34068]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T08:46:19.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:18 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1625527503' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T08:46:19.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-09-10T08:46:19.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:18 smithi191 ceph-mon[35314]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T08:46:19.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:18 smithi191 ceph-mon[35314]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T08:46:19.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:18 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1625527503' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T08:46:19.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.455+0000 7f975386e640 1 -- 172.21.15.172:0/3647723721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f974c100e20 msgr2=0x7f974c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:19.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.455+0000 7f975386e640 1 --2- 172.21.15.172:0/3647723721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f974c100e20 0x7f974c101200 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9744009900 tx=0x7f974402a480 comp rx=0 tx=0).stop 2024-09-10T08:46:19.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.455+0000 7f975386e640 1 -- 172.21.15.172:0/3647723721 shutdown_connections 2024-09-10T08:46:19.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.455+0000 7f975386e640 1 --2- 172.21.15.172:0/3647723721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f974c102ef0 0x7f974c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:19.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.455+0000 7f975386e640 1 --2- 172.21.15.172:0/3647723721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f974c102570 0x7f974c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:19.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.455+0000 7f975386e640 1 --2- 172.21.15.172:0/3647723721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f974c100e20 0x7f974c101200 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:19.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.455+0000 7f975386e640 1 -- 172.21.15.172:0/3647723721 >> 172.21.15.172:0/3647723721 conn(0x7f974c0fc7d0 msgr2=0x7f974c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:19.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.455+0000 7f975386e640 1 -- 172.21.15.172:0/3647723721 shutdown_connections 2024-09-10T08:46:19.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.455+0000 7f975386e640 1 -- 172.21.15.172:0/3647723721 wait complete. 2024-09-10T08:46:19.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f975386e640 1 Processor -- start 2024-09-10T08:46:19.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f975386e640 1 -- start start 2024-09-10T08:46:19.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f975386e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f974c100e20 0x7f974c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:19.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f975386e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f974c102570 0x7f974c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:19.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f975386e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f974c102ef0 0x7f974c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:19.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f975386e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f974c111f40 con 0x7f974c102570 2024-09-10T08:46:19.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f975386e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f974c111dc0 con 0x7f974c100e20 2024-09-10T08:46:19.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f975386e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f974c1120c0 con 0x7f974c102ef0 2024-09-10T08:46:19.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f9750e09640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f974c102570 0x7f974c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:19.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f9750e09640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f974c102570 0x7f974c1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37896/0 (socket says 172.21.15.172:37896) 2024-09-10T08:46:19.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f9750e09640 1 -- 172.21.15.172:0/467839039 learned_addr learned my addr 172.21.15.172:0/467839039 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:19.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f9750e09640 1 -- 172.21.15.172:0/467839039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f974c102ef0 msgr2=0x7f974c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:19.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f9750e09640 1 --2- 172.21.15.172:0/467839039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f974c102ef0 0x7f974c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:19.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f975160a640 1 --2- 172.21.15.172:0/467839039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f974c100e20 0x7f974c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:19.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f9750e09640 1 -- 172.21.15.172:0/467839039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f974c100e20 msgr2=0x7f974c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:19.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f9750e09640 1 --2- 172.21.15.172:0/467839039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f974c100e20 0x7f974c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:19.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f9750e09640 1 -- 172.21.15.172:0/467839039 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974c19d320 con 0x7f974c102570 2024-09-10T08:46:19.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.456+0000 7f9750e09640 1 --2- 172.21.15.172:0/467839039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f974c102570 0x7f974c1988b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f974000c890 tx=0x7f974000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:19.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.457+0000 7f974a7fc640 1 -- 172.21.15.172:0/467839039 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740004220 con 0x7f974c102570 2024-09-10T08:46:19.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.457+0000 7f974a7fc640 1 -- 172.21.15.172:0/467839039 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97400043c0 con 0x7f974c102570 2024-09-10T08:46:19.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.457+0000 7f975386e640 1 -- 172.21.15.172:0/467839039 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974c19d610 con 0x7f974c102570 2024-09-10T08:46:19.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.457+0000 7f974a7fc640 1 -- 172.21.15.172:0/467839039 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740004a00 con 0x7f974c102570 2024-09-10T08:46:19.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.457+0000 7f975386e640 1 -- 172.21.15.172:0/467839039 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f974c1a4ef0 con 0x7f974c102570 2024-09-10T08:46:19.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.458+0000 7f975386e640 1 -- 172.21.15.172:0/467839039 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f974c101200 con 0x7f974c102570 2024-09-10T08:46:19.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.459+0000 7f974a7fc640 1 -- 172.21.15.172:0/467839039 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9740004ba0 con 0x7f974c102570 2024-09-10T08:46:19.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.459+0000 7f974a7fc640 1 --2- 172.21.15.172:0/467839039 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97340a2b80 0x7f97340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:19.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.459+0000 7f975160a640 1 --2- 172.21.15.172:0/467839039 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97340a2b80 0x7f97340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:19.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.459+0000 7f974a7fc640 1 -- 172.21.15.172:0/467839039 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f97400cf110 con 0x7f974c102570 2024-09-10T08:46:19.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.460+0000 7f975160a640 1 --2- 172.21.15.172:0/467839039 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97340a2b80 0x7f97340a5040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9744006010 tx=0x7f9744000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:19.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.464+0000 7f974a7fc640 1 -- 172.21.15.172:0/467839039 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9740014030 con 0x7f974c102570 2024-09-10T08:46:19.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.667+0000 7f975386e640 1 -- 172.21.15.172:0/467839039 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f974c1090c0 con 0x7f974c102570 2024-09-10T08:46:19.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.667+0000 7f974a7fc640 1 -- 172.21.15.172:0/467839039 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+12001 (secure 0 0 0) 0x7f974009bd10 con 0x7f974c102570 2024-09-10T08:46:19.668 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:19.668 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":28,"btime":"2024-09-10T08:44:55:226722+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:44:55.226705+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T08:46:19.669 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 28 2024-09-10T08:46:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.670+0000 7f975386e640 1 -- 172.21.15.172:0/467839039 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97340a2b80 msgr2=0x7f97340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.670+0000 7f975386e640 1 --2- 172.21.15.172:0/467839039 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97340a2b80 0x7f97340a5040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9744006010 tx=0x7f9744000bc0 comp rx=0 tx=0).stop 2024-09-10T08:46:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.670+0000 7f975386e640 1 -- 172.21.15.172:0/467839039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f974c102570 msgr2=0x7f974c1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.670+0000 7f975386e640 1 --2- 172.21.15.172:0/467839039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f974c102570 0x7f974c1988b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f974000c890 tx=0x7f974000cd50 comp rx=0 tx=0).stop 2024-09-10T08:46:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.670+0000 7f975386e640 1 -- 172.21.15.172:0/467839039 shutdown_connections 2024-09-10T08:46:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.670+0000 7f975386e640 1 --2- 172.21.15.172:0/467839039 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97340a2b80 0x7f97340a5040 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.670+0000 7f975386e640 1 --2- 172.21.15.172:0/467839039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f974c102ef0 0x7f974c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:19.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.670+0000 7f975386e640 1 --2- 172.21.15.172:0/467839039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f974c102570 0x7f974c1988b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:19.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.670+0000 7f975386e640 1 --2- 172.21.15.172:0/467839039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f974c100e20 0x7f974c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:19.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.670+0000 7f975386e640 1 -- 172.21.15.172:0/467839039 >> 172.21.15.172:0/467839039 conn(0x7f974c0fc7d0 msgr2=0x7f974c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:19.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.670+0000 7f975386e640 1 -- 172.21.15.172:0/467839039 shutdown_connections 2024-09-10T08:46:19.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:19.671+0000 7f975386e640 1 -- 172.21.15.172:0/467839039 wait complete. 2024-09-10T08:46:20.066 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:19 smithi172 ceph-mon[30854]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T08:46:20.066 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:19 smithi172 ceph-mon[30854]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-10T08:46:20.066 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:19 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/467839039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:20.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:19 smithi174 ceph-mon[34068]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T08:46:20.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:19 smithi174 ceph-mon[34068]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-10T08:46:20.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:19 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/467839039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:20.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:19 smithi191 ceph-mon[35314]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T08:46:20.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:19 smithi191 ceph-mon[35314]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-10T08:46:20.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:19 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/467839039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:20.402 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:46:20.652 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:21.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.148+0000 7f7d8b3a5640 1 -- 172.21.15.172:0/4095687238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d84102570 msgr2=0x7f7d841029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:21.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.148+0000 7f7d8b3a5640 1 --2- 172.21.15.172:0/4095687238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d84102570 0x7f7d841029b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009900 tx=0x7f7d8002a480 comp rx=0 tx=0).stop 2024-09-10T08:46:21.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.148+0000 7f7d8b3a5640 1 -- 172.21.15.172:0/4095687238 shutdown_connections 2024-09-10T08:46:21.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.148+0000 7f7d8b3a5640 1 --2- 172.21.15.172:0/4095687238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d84102ef0 0x7f7d8410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:21.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.148+0000 7f7d8b3a5640 1 --2- 172.21.15.172:0/4095687238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d84102570 0x7f7d841029b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:21.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.148+0000 7f7d8b3a5640 1 --2- 172.21.15.172:0/4095687238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d84100e20 0x7f7d84101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:21.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.148+0000 7f7d8b3a5640 1 -- 172.21.15.172:0/4095687238 >> 172.21.15.172:0/4095687238 conn(0x7f7d840fc7d0 msgr2=0x7f7d840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:21.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.148+0000 7f7d8b3a5640 1 -- 172.21.15.172:0/4095687238 shutdown_connections 2024-09-10T08:46:21.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.148+0000 7f7d8b3a5640 1 -- 172.21.15.172:0/4095687238 wait complete. 2024-09-10T08:46:21.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.148+0000 7f7d8b3a5640 1 Processor -- start 2024-09-10T08:46:21.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.148+0000 7f7d8b3a5640 1 -- start start 2024-09-10T08:46:21.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d8b3a5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d84100e20 0x7f7d8419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:21.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d8b3a5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d84102570 0x7f7d8419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:21.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d8b3a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d84102ef0 0x7f7d8419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:21.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d8b3a5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84111f50 con 0x7f7d84102ef0 2024-09-10T08:46:21.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d8b3a5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84111dd0 con 0x7f7d84100e20 2024-09-10T08:46:21.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d8b3a5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7d841120d0 con 0x7f7d84102570 2024-09-10T08:46:21.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d89942640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d84102ef0 0x7f7d8419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:21.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d89942640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d84102ef0 0x7f7d8419eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37906/0 (socket says 172.21.15.172:37906) 2024-09-10T08:46:21.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d89942640 1 -- 172.21.15.172:0/2066885751 learned_addr learned my addr 172.21.15.172:0/2066885751 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:21.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d88940640 1 --2- 172.21.15.172:0/2066885751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d84102570 0x7f7d8419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:21.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d89141640 1 --2- 172.21.15.172:0/2066885751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d84100e20 0x7f7d8419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:21.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d89942640 1 -- 172.21.15.172:0/2066885751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d84102570 msgr2=0x7f7d8419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:21.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d89942640 1 --2- 172.21.15.172:0/2066885751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d84102570 0x7f7d8419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:21.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d89942640 1 -- 172.21.15.172:0/2066885751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d84100e20 msgr2=0x7f7d8419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:21.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d89942640 1 --2- 172.21.15.172:0/2066885751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d84100e20 0x7f7d8419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:21.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d89942640 1 -- 172.21.15.172:0/2066885751 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8419f580 con 0x7f7d84102ef0 2024-09-10T08:46:21.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d88940640 1 --2- 172.21.15.172:0/2066885751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d84102570 0x7f7d8419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:21.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d89942640 1 --2- 172.21.15.172:0/2066885751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d84102ef0 0x7f7d8419eea0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00b970 tx=0x7f7d7c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:21.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d767fc640 1 -- 172.21.15.172:0/2066885751 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c019070 con 0x7f7d84102ef0 2024-09-10T08:46:21.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d767fc640 1 -- 172.21.15.172:0/2066885751 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d7c014440 con 0x7f7d84102ef0 2024-09-10T08:46:21.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d8b3a5640 1 -- 172.21.15.172:0/2066885751 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8419f870 con 0x7f7d84102ef0 2024-09-10T08:46:21.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.149+0000 7f7d767fc640 1 -- 172.21.15.172:0/2066885751 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c0075d0 con 0x7f7d84102ef0 2024-09-10T08:46:21.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.150+0000 7f7d8b3a5640 1 -- 172.21.15.172:0/2066885751 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d841a7150 con 0x7f7d84102ef0 2024-09-10T08:46:21.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.150+0000 7f7d8b3a5640 1 -- 172.21.15.172:0/2066885751 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d84101200 con 0x7f7d84102ef0 2024-09-10T08:46:21.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.151+0000 7f7d767fc640 1 -- 172.21.15.172:0/2066885751 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7d7c01c070 con 0x7f7d84102ef0 2024-09-10T08:46:21.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.152+0000 7f7d767fc640 1 --2- 172.21.15.172:0/2066885751 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7d700a2b80 0x7f7d700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:21.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.152+0000 7f7d89141640 1 --2- 172.21.15.172:0/2066885751 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7d700a2b80 0x7f7d700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:21.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.152+0000 7f7d767fc640 1 -- 172.21.15.172:0/2066885751 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f7d7c0cea20 con 0x7f7d84102ef0 2024-09-10T08:46:21.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.153+0000 7f7d89141640 1 --2- 172.21.15.172:0/2066885751 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7d700a2b80 0x7f7d700a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7d78005e10 tx=0x7f7d78005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:21.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.156+0000 7f7d767fc640 1 -- 172.21.15.172:0/2066885751 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d7c09b620 con 0x7f7d84102ef0 2024-09-10T08:46:21.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.390+0000 7f7d8b3a5640 1 -- 172.21.15.172:0/2066885751 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d8410d4a0 con 0x7f7d84102ef0 2024-09-10T08:46:21.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.391+0000 7f7d767fc640 1 -- 172.21.15.172:0/2066885751 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+12001 (secure 0 0 0) 0x7f7d7c09af20 con 0x7f7d84102ef0 2024-09-10T08:46:21.392 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 28 2024-09-10T08:46:21.392 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:21.393 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":28,"btime":"2024-09-10T08:44:55:226722+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:44:55.226705+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T08:46:21.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.393+0000 7f7d8b3a5640 1 -- 172.21.15.172:0/2066885751 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7d700a2b80 msgr2=0x7f7d700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:21.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.393+0000 7f7d8b3a5640 1 --2- 172.21.15.172:0/2066885751 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7d700a2b80 0x7f7d700a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7d78005e10 tx=0x7f7d78005da0 comp rx=0 tx=0).stop 2024-09-10T08:46:21.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.393+0000 7f7d8b3a5640 1 -- 172.21.15.172:0/2066885751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d84102ef0 msgr2=0x7f7d8419eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:21.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.393+0000 7f7d8b3a5640 1 --2- 172.21.15.172:0/2066885751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d84102ef0 0x7f7d8419eea0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00b970 tx=0x7f7d7c00be30 comp rx=0 tx=0).stop 2024-09-10T08:46:21.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.393+0000 7f7d8b3a5640 1 -- 172.21.15.172:0/2066885751 shutdown_connections 2024-09-10T08:46:21.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.393+0000 7f7d8b3a5640 1 --2- 172.21.15.172:0/2066885751 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7d700a2b80 0x7f7d700a5040 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:21.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.393+0000 7f7d8b3a5640 1 --2- 172.21.15.172:0/2066885751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d84102ef0 0x7f7d8419eea0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:21.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.393+0000 7f7d8b3a5640 1 --2- 172.21.15.172:0/2066885751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d84102570 0x7f7d8419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:21.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.393+0000 7f7d8b3a5640 1 --2- 172.21.15.172:0/2066885751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d84100e20 0x7f7d8419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:21.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.393+0000 7f7d8b3a5640 1 -- 172.21.15.172:0/2066885751 >> 172.21.15.172:0/2066885751 conn(0x7f7d840fc7d0 msgr2=0x7f7d84105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:21.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.394+0000 7f7d8b3a5640 1 -- 172.21.15.172:0/2066885751 shutdown_connections 2024-09-10T08:46:21.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:21.394+0000 7f7d8b3a5640 1 -- 172.21.15.172:0/2066885751 wait complete. 2024-09-10T08:46:22.096 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:46:22.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:22 smithi172 ceph-mon[30854]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-10T08:46:22.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:22 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2066885751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:22.347 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:22.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:22 smithi174 ceph-mon[34068]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-10T08:46:22.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:22 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2066885751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:22 smithi191 ceph-mon[35314]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-10T08:46:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:22 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2066885751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:22.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.858+0000 7f35530c1640 1 -- 172.21.15.172:0/3300313299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f354c101450 msgr2=0x7f354c105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:22.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.858+0000 7f35530c1640 1 --2- 172.21.15.172:0/3300313299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f354c101450 0x7f354c105920 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3538009900 tx=0x7f353802a470 comp rx=0 tx=0).stop 2024-09-10T08:46:22.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.858+0000 7f35530c1640 1 -- 172.21.15.172:0/3300313299 shutdown_connections 2024-09-10T08:46:22.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.858+0000 7f35530c1640 1 --2- 172.21.15.172:0/3300313299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f354c105f40 0x7f354c1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:22.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.858+0000 7f35530c1640 1 --2- 172.21.15.172:0/3300313299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f354c101450 0x7f354c105920 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:22.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.858+0000 7f35530c1640 1 --2- 172.21.15.172:0/3300313299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f354c100b30 0x7f354c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:22.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.858+0000 7f35530c1640 1 -- 172.21.15.172:0/3300313299 >> 172.21.15.172:0/3300313299 conn(0x7f354c0fc7a0 msgr2=0x7f354c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:22.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.858+0000 7f35530c1640 1 -- 172.21.15.172:0/3300313299 shutdown_connections 2024-09-10T08:46:22.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.858+0000 7f35530c1640 1 -- 172.21.15.172:0/3300313299 wait complete. 2024-09-10T08:46:22.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.859+0000 7f35530c1640 1 Processor -- start 2024-09-10T08:46:22.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.859+0000 7f35530c1640 1 -- start start 2024-09-10T08:46:22.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.859+0000 7f35530c1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f354c100b30 0x7f354c1a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.859+0000 7f35530c1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f354c101450 0x7f354c1a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.859+0000 7f35530c1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f354c105f40 0x7f354c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.859+0000 7f35530c1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1183f0 con 0x7f354c105f40 2024-09-10T08:46:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.859+0000 7f35530c1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f354c118270 con 0x7f354c100b30 2024-09-10T08:46:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.859+0000 7f35530c1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f354c118570 con 0x7f354c101450 2024-09-10T08:46:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.859+0000 7f355165e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f354c105f40 0x7f354c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.859+0000 7f355165e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f354c105f40 0x7f354c19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37924/0 (socket says 172.21.15.172:37924) 2024-09-10T08:46:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.859+0000 7f355165e640 1 -- 172.21.15.172:0/2238607427 learned_addr learned my addr 172.21.15.172:0/2238607427 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.859+0000 7f3550e5d640 1 --2- 172.21.15.172:0/2238607427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f354c100b30 0x7f354c1a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.860+0000 7f354bfff640 1 --2- 172.21.15.172:0/2238607427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f354c101450 0x7f354c1a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:22.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.860+0000 7f355165e640 1 -- 172.21.15.172:0/2238607427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f354c101450 msgr2=0x7f354c1a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:22.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.860+0000 7f355165e640 1 --2- 172.21.15.172:0/2238607427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f354c101450 0x7f354c1a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:22.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.860+0000 7f355165e640 1 -- 172.21.15.172:0/2238607427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f354c100b30 msgr2=0x7f354c1a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:22.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.860+0000 7f355165e640 1 --2- 172.21.15.172:0/2238607427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f354c100b30 0x7f354c1a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:22.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.860+0000 7f355165e640 1 -- 172.21.15.172:0/2238607427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c19ef40 con 0x7f354c105f40 2024-09-10T08:46:22.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.860+0000 7f3550e5d640 1 --2- 172.21.15.172:0/2238607427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f354c100b30 0x7f354c1a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:22.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.860+0000 7f355165e640 1 --2- 172.21.15.172:0/2238607427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f354c105f40 0x7f354c19e7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f354400cce0 tx=0x7f354400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:22.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.860+0000 7f3549ffb640 1 -- 172.21.15.172:0/2238607427 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354400f450 con 0x7f354c105f40 2024-09-10T08:46:22.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.860+0000 7f35530c1640 1 -- 172.21.15.172:0/2238607427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354c19f230 con 0x7f354c105f40 2024-09-10T08:46:22.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.860+0000 7f3549ffb640 1 -- 172.21.15.172:0/2238607427 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f354400faa0 con 0x7f354c105f40 2024-09-10T08:46:22.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.860+0000 7f3549ffb640 1 -- 172.21.15.172:0/2238607427 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544018b70 con 0x7f354c105f40 2024-09-10T08:46:22.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.860+0000 7f35530c1640 1 -- 172.21.15.172:0/2238607427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354c1ab480 con 0x7f354c105f40 2024-09-10T08:46:22.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.861+0000 7f35530c1640 1 -- 172.21.15.172:0/2238607427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f354c06b070 con 0x7f354c105f40 2024-09-10T08:46:22.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.862+0000 7f3549ffb640 1 -- 172.21.15.172:0/2238607427 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3544018d10 con 0x7f354c105f40 2024-09-10T08:46:22.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.863+0000 7f3549ffb640 1 --2- 172.21.15.172:0/2238607427 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f353c0a2b80 0x7f353c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:22.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.863+0000 7f3550e5d640 1 --2- 172.21.15.172:0/2238607427 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f353c0a2b80 0x7f353c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:22.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.863+0000 7f3549ffb640 1 -- 172.21.15.172:0/2238607427 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f35440cf820 con 0x7f354c105f40 2024-09-10T08:46:22.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.863+0000 7f3550e5d640 1 --2- 172.21.15.172:0/2238607427 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f353c0a2b80 0x7f353c0a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3540005fd0 tx=0x7f354000b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:22.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:22.867+0000 7f3549ffb640 1 -- 172.21.15.172:0/2238607427 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3544014030 con 0x7f354c105f40 2024-09-10T08:46:23.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:23.034+0000 7f35530c1640 1 -- 172.21.15.172:0/2238607427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f354c1ab650 con 0x7f354c105f40 2024-09-10T08:46:23.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:23.036+0000 7f3549ffb640 1 -- 172.21.15.172:0/2238607427 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24778 (secure 0 0 0) 0x7f354409c420 con 0x7f354c105f40 2024-09-10T08:46:23.037 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:23.037 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":92,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","created":"2024-09-10T08:36:33.083529+0000","modified":"2024-09-10T08:46:18.799844+0000","last_up_change":"2024-09-10T08:43:19.615640+0000","last_in_change":"2024-09-10T08:42:56.656115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:39:31.328306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:44:07.466637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:44:09.520569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:44:15.926057+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6803","nonce":2678426952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6805","nonce":2678426952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6809","nonce":2678426952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6807","nonce":2678426952}]},"public_addr":"172.21.15.172:6803/2678426952","cluster_addr":"172.21.15.172:6805/2678426952","heartbeat_back_addr":"172.21.15.172:6809/2678426952","heartbeat_front_addr":"172.21.15.172:6807/2678426952","state":["exists","up"]},{"osd":1,"uuid":"55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6801","nonce":3279661439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6803","nonce":3279661439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6807","nonce":3279661439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6805","nonce":3279661439}]},"public_addr":"172.21.15.174:6801/3279661439","cluster_addr":"172.21.15.174:6803/3279661439","heartbeat_back_addr":"172.21.15.174:6807/3279661439","heartbeat_front_addr":"172.21.15.174:6805/3279661439","state":["exists","up"]},{"osd":2,"uuid":"f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6801","nonce":1963796290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6803","nonce":1963796290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6807","nonce":1963796290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6805","nonce":1963796290}]},"public_addr":"172.21.15.191:6801/1963796290","cluster_addr":"172.21.15.191:6803/1963796290","heartbeat_back_addr":"172.21.15.191:6807/1963796290","heartbeat_front_addr":"172.21.15.191:6805/1963796290","state":["exists","up"]},{"osd":3,"uuid":"15ecca3f-bc6b-4a06-b283-b515384beb19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6811","nonce":869546480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6813","nonce":869546480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6817","nonce":869546480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6815","nonce":869546480}]},"public_addr":"172.21.15.172:6811/869546480","cluster_addr":"172.21.15.172:6813/869546480","heartbeat_back_addr":"172.21.15.172:6817/869546480","heartbeat_front_addr":"172.21.15.172:6815/869546480","state":["exists","up"]},{"osd":4,"uuid":"8c2c1d40-72ad-4a15-ab72-54871c7e2427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6809","nonce":961007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6811","nonce":961007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6815","nonce":961007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6813","nonce":961007949}]},"public_addr":"172.21.15.174:6809/961007949","cluster_addr":"172.21.15.174:6811/961007949","heartbeat_back_addr":"172.21.15.174:6815/961007949","heartbeat_front_addr":"172.21.15.174:6813/961007949","state":["exists","up"]},{"osd":5,"uuid":"4de45a1d-ef0e-4bfe-ba15-5eeeb8806155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6809","nonce":4216938371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6811","nonce":4216938371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6815","nonce":4216938371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6813","nonce":4216938371}]},"public_addr":"172.21.15.191:6809/4216938371","cluster_addr":"172.21.15.191:6811/4216938371","heartbeat_back_addr":"172.21.15.191:6815/4216938371","heartbeat_front_addr":"172.21.15.191:6813/4216938371","state":["exists","up"]},{"osd":6,"uuid":"60a9938a-f696-4db2-932c-402997543b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6819","nonce":3210762009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6821","nonce":3210762009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6825","nonce":3210762009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6823","nonce":3210762009}]},"public_addr":"172.21.15.172:6819/3210762009","cluster_addr":"172.21.15.172:6821/3210762009","heartbeat_back_addr":"172.21.15.172:6825/3210762009","heartbeat_front_addr":"172.21.15.172:6823/3210762009","state":["exists","up"]},{"osd":7,"uuid":"7f0202f3-971e-472f-9c4d-ae3c24d39f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6817","nonce":1461923581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6819","nonce":1461923581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6823","nonce":1461923581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6821","nonce":1461923581}]},"public_addr":"172.21.15.174:6817/1461923581","cluster_addr":"172.21.15.174:6819/1461923581","heartbeat_back_addr":"172.21.15.174:6823/1461923581","heartbeat_front_addr":"172.21.15.174:6821/1461923581","state":["exists","up"]},{"osd":8,"uuid":"b4e7327f-ca9f-4833-9cd3-b8529b3d9761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6817","nonce":3675210430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6819","nonce":3675210430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6823","nonce":3675210430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6821","nonce":3675210430}]},"public_addr":"172.21.15.191:6817/3675210430","cluster_addr":"172.21.15.191:6819/3675210430","heartbeat_back_addr":"172.21.15.191:6823/3675210430","heartbeat_front_addr":"172.21.15.191:6821/3675210430","state":["exists","up"]},{"osd":9,"uuid":"9f6153f2-f9db-4c00-a659-c932628016f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6827","nonce":1181268619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6829","nonce":1181268619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6833","nonce":1181268619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6831","nonce":1181268619}]},"public_addr":"172.21.15.172:6827/1181268619","cluster_addr":"172.21.15.172:6829/1181268619","heartbeat_back_addr":"172.21.15.172:6833/1181268619","heartbeat_front_addr":"172.21.15.172:6831/1181268619","state":["exists","up"]},{"osd":10,"uuid":"040a013a-4c3e-442f-bea6-12cbe517716d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6825","nonce":1238812525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6827","nonce":1238812525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6831","nonce":1238812525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6829","nonce":1238812525}]},"public_addr":"172.21.15.174:6825/1238812525","cluster_addr":"172.21.15.174:6827/1238812525","heartbeat_back_addr":"172.21.15.174:6831/1238812525","heartbeat_front_addr":"172.21.15.174:6829/1238812525","state":["exists","up"]},{"osd":11,"uuid":"ae6535bd-1510-4bf3-bf4b-1a2a256add43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6825","nonce":2162719997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6827","nonce":2162719997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6831","nonce":2162719997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6829","nonce":2162719997}]},"public_addr":"172.21.15.191:6825/2162719997","cluster_addr":"172.21.15.191:6827/2162719997","heartbeat_back_addr":"172.21.15.191:6831/2162719997","heartbeat_front_addr":"172.21.15.191:6829/2162719997","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:38:38.812596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:04.765652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:29.103966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:56.085849+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:21.641784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:47.750376+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:11.640697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:36.638840+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:02.167446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:28.655804+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:52.749676+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:43:17.783908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:0/977245917":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/3919952878":"2024-09-11T08:36:54.887256+0000","172.21.15.172:6800/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/1525385361":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/2043423408":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/839344299":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6800/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/3251817286":"2024-09-11T08:36:54.887256+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:46:23.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:23.038+0000 7f35530c1640 1 -- 172.21.15.172:0/2238607427 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f353c0a2b80 msgr2=0x7f353c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:23.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:23.038+0000 7f35530c1640 1 --2- 172.21.15.172:0/2238607427 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f353c0a2b80 0x7f353c0a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3540005fd0 tx=0x7f354000b5f0 comp rx=0 tx=0).stop 2024-09-10T08:46:23.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:23.038+0000 7f35530c1640 1 -- 172.21.15.172:0/2238607427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f354c105f40 msgr2=0x7f354c19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:23.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:23.038+0000 7f35530c1640 1 --2- 172.21.15.172:0/2238607427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f354c105f40 0x7f354c19e7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f354400cce0 tx=0x7f354400a620 comp rx=0 tx=0).stop 2024-09-10T08:46:23.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:23.038+0000 7f35530c1640 1 -- 172.21.15.172:0/2238607427 shutdown_connections 2024-09-10T08:46:23.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:23.038+0000 7f35530c1640 1 --2- 172.21.15.172:0/2238607427 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f353c0a2b80 0x7f353c0a5040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:23.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:23.038+0000 7f35530c1640 1 --2- 172.21.15.172:0/2238607427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f354c105f40 0x7f354c19e7a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:23.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:23.038+0000 7f35530c1640 1 --2- 172.21.15.172:0/2238607427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f354c101450 0x7f354c1a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:23.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:23.038+0000 7f35530c1640 1 --2- 172.21.15.172:0/2238607427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f354c100b30 0x7f354c1a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:23.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:23.038+0000 7f35530c1640 1 -- 172.21.15.172:0/2238607427 >> 172.21.15.172:0/2238607427 conn(0x7f354c0fc7a0 msgr2=0x7f354c1038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:23.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:23.038+0000 7f35530c1640 1 -- 172.21.15.172:0/2238607427 shutdown_connections 2024-09-10T08:46:23.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:23.038+0000 7f35530c1640 1 -- 172.21.15.172:0/2238607427 wait complete. 2024-09-10T08:46:23.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:23 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2238607427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:46:23.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:23 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2238607427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:46:23.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:23 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2238607427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:46:23.671 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T08:44:11.310930+0000', 'modified': '2024-09-10T08:44:55.226705+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24488, 'mds_1': 24458, 'mds_2': 24457, 'mds_3': 24416, 'mds_4': 24406}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6839/1621056220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6838', 'nonce': 1621056220}, {'type': 'v1', 'addr': '172.21.15.172:6839', 'nonce': 1621056220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14622': {'gid': 14622, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6841/176040463', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6840', 'nonce': 176040463}, {'type': 'v1', 'addr': '172.21.15.172:6841', 'nonce': 176040463}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24406': {'gid': 24406, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.191:6833/1662718523', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6832', 'nonce': 1662718523}, {'type': 'v1', 'addr': '172.21.15.191:6833', 'nonce': 1662718523}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.174:6833/3418766024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 3418766024}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 3418766024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6835/1774553543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6834', 'nonce': 1774553543}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 1774553543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/3197756201', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 3197756201}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 3197756201}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6837/1802631256', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6836', 'nonce': 1802631256}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 1802631256}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.191:6839/1702936476', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6838', 'nonce': 1702936476}, {'type': 'v1', 'addr': '172.21.15.191:6839', 'nonce': 1702936476}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.174:6837/2399337052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 2399337052}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 2399337052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.174:6839/3029849701', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 3029849701}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 3029849701}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24488, 24457, 24416, 24458, 24406]} 2024-09-10T08:46:23.672 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T08:46:23.672 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.l status 2024-09-10T08:46:23.920 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:24.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:24 smithi172 ceph-mon[30854]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 639 B/s wr, 11 op/s 2024-09-10T08:46:24.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.426+0000 7f5f2b5f9640 1 -- 172.21.15.172:0/537778023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f24100a80 msgr2=0x7f5f24100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:24.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.426+0000 7f5f2b5f9640 1 --2- 172.21.15.172:0/537778023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f24100a80 0x7f5f24100ee0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5f14009930 tx=0x7f5f1402a420 comp rx=0 tx=0).stop 2024-09-10T08:46:24.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.426+0000 7f5f2b5f9640 1 -- 172.21.15.172:0/537778023 shutdown_connections 2024-09-10T08:46:24.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.426+0000 7f5f2b5f9640 1 --2- 172.21.15.172:0/537778023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f24101420 0x7f5f2410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.426+0000 7f5f2b5f9640 1 --2- 172.21.15.172:0/537778023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f24100a80 0x7f5f24100ee0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.426+0000 7f5f2b5f9640 1 --2- 172.21.15.172:0/537778023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f24106a80 0x7f5f24106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.426+0000 7f5f2b5f9640 1 -- 172.21.15.172:0/537778023 >> 172.21.15.172:0/537778023 conn(0x7f5f240fc7a0 msgr2=0x7f5f240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:24.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.426+0000 7f5f2b5f9640 1 -- 172.21.15.172:0/537778023 shutdown_connections 2024-09-10T08:46:24.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.427+0000 7f5f2b5f9640 1 -- 172.21.15.172:0/537778023 wait complete. 2024-09-10T08:46:24.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.427+0000 7f5f2b5f9640 1 Processor -- start 2024-09-10T08:46:24.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.427+0000 7f5f2b5f9640 1 -- start start 2024-09-10T08:46:24.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.427+0000 7f5f2b5f9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f24100a80 0x7f5f24197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.427+0000 7f5f2b5f9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f24101420 0x7f5f24197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.427+0000 7f5f2b5f9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f24106a80 0x7f5f2419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.427+0000 7f5f2b5f9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5f24110d50 con 0x7f5f24100a80 2024-09-10T08:46:24.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.427+0000 7f5f2b5f9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f24110bd0 con 0x7f5f24106a80 2024-09-10T08:46:24.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.427+0000 7f5f2b5f9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5f24110ed0 con 0x7f5f24101420 2024-09-10T08:46:24.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f29395640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f24100a80 0x7f5f24197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:24.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f29395640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f24100a80 0x7f5f24197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37944/0 (socket says 172.21.15.172:37944) 2024-09-10T08:46:24.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f29395640 1 -- 172.21.15.172:0/3302561610 learned_addr learned my addr 172.21.15.172:0/3302561610 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:24.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f28b94640 1 --2- 172.21.15.172:0/3302561610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f24101420 0x7f5f24197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:24.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f29b96640 1 --2- 172.21.15.172:0/3302561610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f24106a80 0x7f5f2419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:24.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f29395640 1 -- 172.21.15.172:0/3302561610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f24101420 msgr2=0x7f5f24197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f29395640 1 --2- 172.21.15.172:0/3302561610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f24101420 0x7f5f24197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f29395640 1 -- 172.21.15.172:0/3302561610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f24106a80 msgr2=0x7f5f2419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f29395640 1 --2- 172.21.15.172:0/3302561610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f24106a80 0x7f5f2419bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f29395640 1 -- 172.21.15.172:0/3302561610 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2419c2b0 con 0x7f5f24100a80 2024-09-10T08:46:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f29b96640 1 --2- 172.21.15.172:0/3302561610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f24106a80 0x7f5f2419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f28b94640 1 --2- 172.21.15.172:0/3302561610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f24101420 0x7f5f24197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f29395640 1 --2- 172.21.15.172:0/3302561610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f24100a80 0x7f5f24197300 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5f20007910 tx=0x7f5f20007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f1a7fc640 1 -- 172.21.15.172:0/3302561610 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20019070 con 0x7f5f24100a80 2024-09-10T08:46:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f1a7fc640 1 -- 172.21.15.172:0/3302561610 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f20005020 con 0x7f5f24100a80 2024-09-10T08:46:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f2b5f9640 1 -- 172.21.15.172:0/3302561610 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2419c5a0 con 0x7f5f24100a80 2024-09-10T08:46:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f1a7fc640 1 -- 172.21.15.172:0/3302561610 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20004a00 con 0x7f5f24100a80 2024-09-10T08:46:24.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.428+0000 7f5f2b5f9640 1 -- 172.21.15.172:0/3302561610 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2406a790 con 0x7f5f24100a80 2024-09-10T08:46:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.430+0000 7f5f1a7fc640 1 -- 172.21.15.172:0/3302561610 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5f2001c070 con 0x7f5f24100a80 2024-09-10T08:46:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.431+0000 7f5f2d9d2740 1 Processor -- start 2024-09-10T08:46:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.431+0000 7f5f2d9d2740 1 -- start start 2024-09-10T08:46:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.431+0000 7f5f2d9d2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02eb9d0 0x55ffe02ebdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.431+0000 7f5f2d9d2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02ec2f0 0x55ffe02f6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.431+0000 7f5f2d9d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02f6bb0 0x55ffe02f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.431+0000 7f5f2d9d2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ffe0252090 con 0x55ffe02ec2f0 2024-09-10T08:46:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.431+0000 7f5f2d9d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ffe01f9a70 con 0x55ffe02f6bb0 2024-09-10T08:46:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.431+0000 7f5f2d9d2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ffe024e050 con 0x55ffe02eb9d0 2024-09-10T08:46:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.431+0000 7f5f1a7fc640 1 --2- 172.21.15.172:0/3302561610 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f100a2c00 0x7f5f100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.431+0000 7f5f29395640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02ec2f0 0x55ffe02f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:24.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.431+0000 7f5f29395640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02ec2f0 0x55ffe02f6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37948/0 (socket says 172.21.15.172:37948) 2024-09-10T08:46:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.431+0000 7f5f29395640 1 -- 172.21.15.172:0/766896996 learned_addr learned my addr 172.21.15.172:0/766896996 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.431+0000 7f5f28b94640 1 --2- 172.21.15.172:0/766896996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02f6bb0 0x55ffe02f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.431+0000 7f5f29b96640 1 --2- 172.21.15.172:0/3302561610 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f100a2c00 0x7f5f100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.432+0000 7f5f29b96640 1 --2- 172.21.15.172:0/766896996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02eb9d0 0x55ffe02ebdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.432+0000 7f5f1a7fc640 1 -- 172.21.15.172:0/3302561610 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5f200d02f0 con 0x7f5f24100a80 2024-09-10T08:46:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.432+0000 7f5f29395640 1 -- 172.21.15.172:0/766896996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02eb9d0 msgr2=0x55ffe02ebdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.432+0000 7f5f29395640 1 --2- 172.21.15.172:0/766896996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02eb9d0 0x55ffe02ebdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.432+0000 7f5f29395640 1 -- 172.21.15.172:0/766896996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02f6bb0 msgr2=0x55ffe02f8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.432+0000 7f5f29395640 1 --2- 172.21.15.172:0/766896996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02f6bb0 0x55ffe02f8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.432+0000 7f5f29395640 1 -- 172.21.15.172:0/766896996 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe0206130 con 0x55ffe02ec2f0 2024-09-10T08:46:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.432+0000 7f5f29b96640 1 --2- 172.21.15.172:0/3302561610 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f100a2c00 0x7f5f100a50c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009440 tx=0x7f5f1c009610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.432+0000 7f5f29395640 1 --2- 172.21.15.172:0/766896996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02ec2f0 0x55ffe02f6670 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5f20005fd0 tx=0x7f5f20000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69dcb9de0cec0901 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:24.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.432+0000 7f5f0f7fe640 1 -- 172.21.15.172:0/766896996 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2009cef0 con 0x55ffe02ec2f0 2024-09-10T08:46:24.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.432+0000 7f5f0f7fe640 1 -- 172.21.15.172:0/766896996 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f2009c7f0 con 0x55ffe02ec2f0 2024-09-10T08:46:24.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.432+0000 7f5f0f7fe640 1 -- 172.21.15.172:0/766896996 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2009ab80 con 0x55ffe02ec2f0 2024-09-10T08:46:24.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.433+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/766896996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02ec2f0 msgr2=0x55ffe02f6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:24.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.433+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/766896996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02ec2f0 0x55ffe02f6670 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5f20005fd0 tx=0x7f5f20000bc0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.433+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/766896996 shutdown_connections 2024-09-10T08:46:24.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.433+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/766896996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02f6bb0 0x55ffe02f8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.433+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/766896996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02ec2f0 0x55ffe02f6670 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.433+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/766896996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02eb9d0 0x55ffe02ebdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.433+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/766896996 >> 172.21.15.172:0/766896996 conn(0x55ffe02e7f60 msgr2=0x55ffe02e8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:24.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.433+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/766896996 shutdown_connections 2024-09-10T08:46:24.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.433+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/766896996 wait complete. 2024-09-10T08:46:24.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.433+0000 7f5f2d9d2740 1 -- start start 2024-09-10T08:46:24.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.433+0000 7f5f2d9d2740 1 Processor -- start 2024-09-10T08:46:24.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f2d9d2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02eb9d0 0x55ffe02f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f2d9d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02ec2f0 0x55ffe02f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f2d9d2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02f6bb0 0x55ffe02f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f2d9d2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ffe02f9780 con 0x55ffe02eb9d0 2024-09-10T08:46:24.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f2d9d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ffe020cf20 con 0x55ffe02ec2f0 2024-09-10T08:46:24.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f2d9d2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ffe02f9900 con 0x55ffe02f6bb0 2024-09-10T08:46:24.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f2d9d2740 10 client.0 _ll_register_callbacks cb 0x55ffe01d6030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:24.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f29395640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02eb9d0 0x55ffe02f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:24.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f29395640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02eb9d0 0x55ffe02f64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37964/0 (socket says 172.21.15.172:37964) 2024-09-10T08:46:24.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f29395640 1 -- 172.21.15.172:0/3191519640 learned_addr learned my addr 172.21.15.172:0/3191519640 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:24.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f28b94640 1 --2- 172.21.15.172:0/3191519640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02ec2f0 0x55ffe02f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:24.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f29b96640 1 --2- 172.21.15.172:0/3191519640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02f6bb0 0x55ffe02f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:24.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f29395640 1 -- 172.21.15.172:0/3191519640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02f6bb0 msgr2=0x55ffe02f3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:24.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f29395640 1 --2- 172.21.15.172:0/3191519640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02f6bb0 0x55ffe02f3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f29395640 1 -- 172.21.15.172:0/3191519640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02ec2f0 msgr2=0x55ffe02f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:24.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f29395640 1 --2- 172.21.15.172:0/3191519640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02ec2f0 0x55ffe02f37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f29395640 1 -- 172.21.15.172:0/3191519640 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffdffc32b0 con 0x55ffe02eb9d0 2024-09-10T08:46:24.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f28b94640 1 --2- 172.21.15.172:0/3191519640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02ec2f0 0x55ffe02f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:24.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f29b96640 1 --2- 172.21.15.172:0/3191519640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02f6bb0 0x55ffe02f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:24.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f29395640 1 --2- 172.21.15.172:0/3191519640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02eb9d0 0x55ffe02f64a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5f200991c0 tx=0x7f5f20098b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4398b442ae9676c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:24.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.434+0000 7f5f0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:24.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.435+0000 7f5f0cff9640 1 -- 172.21.15.172:0/3191519640 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f200a5460 con 0x55ffe02eb9d0 2024-09-10T08:46:24.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.435+0000 7f5f0cff9640 1 -- 172.21.15.172:0/3191519640 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5f200cdba0 con 0x55ffe02eb9d0 2024-09-10T08:46:24.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.435+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/3191519640 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f2009c7f0 con 0x55ffe02eb9d0 2024-09-10T08:46:24.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.435+0000 7f5f0cff9640 1 -- 172.21.15.172:0/3191519640 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f2009cef0 con 0x55ffe02eb9d0 2024-09-10T08:46:24.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.435+0000 7f5f2d9d2740 10 client.15225 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:24.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.435+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/3191519640 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ffe03050f0 con 0x55ffe02eb9d0 2024-09-10T08:46:24.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.435+0000 7f5f0cff9640 1 -- 172.21.15.172:0/3191519640 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f2009c7f0 con 0x55ffe02eb9d0 2024-09-10T08:46:24.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.435+0000 7f5f0cff9640 1 -- 172.21.15.172:0/3191519640 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f5f20014070 con 0x55ffe02eb9d0 2024-09-10T08:46:24.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.435+0000 7f5f2d9d2740 10 client.15225 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:24.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.435+0000 7f5f2d9d2740 10 client.15225 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-10T08:46:24.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.435+0000 7f5f2d9d2740 20 client.15225 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:24.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.435+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/3191519640 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55ffe0306440 0x55ffe03085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.435+0000 7f5f2d9d2740 4 client.15225 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:24.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.435+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/3191519640 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffe025f600 con 0x55ffe0306440 2024-09-10T08:46:24.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.436+0000 7f5f28b94640 1 --2- 172.21.15.172:0/3191519640 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55ffe0306440 0x55ffe03085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:24.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.436+0000 7f5f0cff9640 10 client.15225 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:46:24.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.436+0000 7f5f28b94640 1 --2- 172.21.15.172:0/3191519640 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55ffe0306440 0x55ffe03085c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=66d844d165bccacc server_cookie=8540bb2ea4844f0 in_seq=0 out_seq=0 2024-09-10T08:46:24.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.437+0000 7f5f0cff9640 1 -- 172.21.15.172:0/3191519640 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffe025f600 con 0x55ffe0306440 2024-09-10T08:46:24.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.437+0000 7f5f0cff9640 10 client.15225 handle_command_reply: tid=0 2024-09-10T08:46:24.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.437+0000 7f5f2d9d2740 1 client.15225 shutdown 2024-09-10T08:46:24.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.437+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/3191519640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02eb9d0 msgr2=0x55ffe02f64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:24.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.437+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/3191519640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02eb9d0 0x55ffe02f64a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5f200991c0 tx=0x7f5f20098b30 comp rx=0 tx=0).stop 2024-09-10T08:46:24.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.437+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/3191519640 shutdown_connections 2024-09-10T08:46:24.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.437+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/3191519640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02f6bb0 0x55ffe02f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.437+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/3191519640 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55ffe0306440 0x55ffe03085c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.437+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/3191519640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02ec2f0 0x55ffe02f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.437+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/3191519640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02eb9d0 0x55ffe02f64a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.437+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/3191519640 >> 172.21.15.172:0/3191519640 conn(0x55ffe02e7f60 msgr2=0x55ffe02edc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:24.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.437+0000 7f5f0cff9640 0 client.15225 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:46:24.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.437+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/3191519640 shutdown_connections 2024-09-10T08:46:24.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.437+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/3191519640 wait complete. 2024-09-10T08:46:24.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.437+0000 7f5f2d9d2740 20 client.15225 trim_cache size 0 max 16384 2024-09-10T08:46:24.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f2d9d2740 1 Processor -- start 2024-09-10T08:46:24.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f2d9d2740 1 -- start start 2024-09-10T08:46:24.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f2d9d2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02eb9d0 0x55ffe0306a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f2d9d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02ec2f0 0x55ffe02e6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f2d9d2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02f6bb0 0x55ffe02f0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f2d9d2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ffe0306900 con 0x55ffe02f6bb0 2024-09-10T08:46:24.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f2d9d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ffe0306fd0 con 0x55ffe02ec2f0 2024-09-10T08:46:24.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f2d9d2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ffe03076a0 con 0x55ffe02eb9d0 2024-09-10T08:46:24.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f29b96640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02f6bb0 0x55ffe02f0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:24.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f29b96640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02f6bb0 0x55ffe02f0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37974/0 (socket says 172.21.15.172:37974) 2024-09-10T08:46:24.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f29b96640 1 -- 172.21.15.172:0/1255717878 learned_addr learned my addr 172.21.15.172:0/1255717878 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:24.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f29395640 1 --2- 172.21.15.172:0/1255717878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02eb9d0 0x55ffe0306a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:24.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f29b96640 1 -- 172.21.15.172:0/1255717878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02eb9d0 msgr2=0x55ffe0306a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:24.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f29b96640 1 --2- 172.21.15.172:0/1255717878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02eb9d0 0x55ffe0306a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f29b96640 1 -- 172.21.15.172:0/1255717878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02ec2f0 msgr2=0x55ffe02e6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:24.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f29b96640 1 --2- 172.21.15.172:0/1255717878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02ec2f0 0x55ffe02e6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f29b96640 1 -- 172.21.15.172:0/1255717878 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe02f6810 con 0x55ffe02f6bb0 2024-09-10T08:46:24.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.455+0000 7f5f29395640 1 --2- 172.21.15.172:0/1255717878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02eb9d0 0x55ffe0306a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:24.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.456+0000 7f5f29b96640 1 --2- 172.21.15.172:0/1255717878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02f6bb0 0x55ffe02f0a70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00ad80 tx=0x7f5f1c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e54b1a0bb612cb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:24.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.456+0000 7f5f0effd640 1 -- 172.21.15.172:0/1255717878 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c02a070 con 0x55ffe02f6bb0 2024-09-10T08:46:24.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.456+0000 7f5f0effd640 1 -- 172.21.15.172:0/1255717878 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f1c00ae70 con 0x55ffe02f6bb0 2024-09-10T08:46:24.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.456+0000 7f5f0effd640 1 -- 172.21.15.172:0/1255717878 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c013040 con 0x55ffe02f6bb0 2024-09-10T08:46:24.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.456+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/1255717878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02f6bb0 msgr2=0x55ffe02f0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:24.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.456+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/1255717878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02f6bb0 0x55ffe02f0a70 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00ad80 tx=0x7f5f1c0093d0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.456+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/1255717878 shutdown_connections 2024-09-10T08:46:24.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.456+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/1255717878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02f6bb0 0x55ffe02f0a70 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.456+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/1255717878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02ec2f0 0x55ffe02e6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.456+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/1255717878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02eb9d0 0x55ffe0306a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.456+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/1255717878 >> 172.21.15.172:0/1255717878 conn(0x55ffe02e7f60 msgr2=0x55ffe02e67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:24.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.457+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/1255717878 shutdown_connections 2024-09-10T08:46:24.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.457+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/1255717878 wait complete. 2024-09-10T08:46:24.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.457+0000 7f5f2d9d2740 1 -- start start 2024-09-10T08:46:24.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.457+0000 7f5f2d9d2740 1 Processor -- start 2024-09-10T08:46:24.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.457+0000 7f5f2d9d2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02eb9d0 0x55ffe030bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.457+0000 7f5f2d9d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02ec2f0 0x55ffe030c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.457+0000 7f5f2d9d2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02f6bb0 0x55ffe0310630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.457+0000 7f5f2d9d2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ffe0308440 con 0x55ffe02f6bb0 2024-09-10T08:46:24.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.457+0000 7f5f2d9d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ffe0307d70 con 0x55ffe02ec2f0 2024-09-10T08:46:24.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.457+0000 7f5f2d9d2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ffe0308b10 con 0x55ffe02eb9d0 2024-09-10T08:46:24.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.457+0000 7f5f2d9d2740 10 client.0 _ll_register_callbacks cb 0x7f5f200cdf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:24.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f29b96640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02f6bb0 0x55ffe0310630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:24.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f29b96640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02f6bb0 0x55ffe0310630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37980/0 (socket says 172.21.15.172:37980) 2024-09-10T08:46:24.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f29b96640 1 -- 172.21.15.172:0/4277369807 learned_addr learned my addr 172.21.15.172:0/4277369807 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:24.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f29395640 1 --2- 172.21.15.172:0/4277369807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02eb9d0 0x55ffe030bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:24.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f28b94640 1 --2- 172.21.15.172:0/4277369807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02ec2f0 0x55ffe030c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:24.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f29b96640 1 -- 172.21.15.172:0/4277369807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02eb9d0 msgr2=0x55ffe030bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:24.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f29b96640 1 --2- 172.21.15.172:0/4277369807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02eb9d0 0x55ffe030bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f29b96640 1 -- 172.21.15.172:0/4277369807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02ec2f0 msgr2=0x55ffe030c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:24.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f29b96640 1 --2- 172.21.15.172:0/4277369807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02ec2f0 0x55ffe030c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f29b96640 1 -- 172.21.15.172:0/4277369807 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffe0310d20 con 0x55ffe02f6bb0 2024-09-10T08:46:24.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f29b96640 1 --2- 172.21.15.172:0/4277369807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02f6bb0 0x55ffe0310630 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c014a50 tx=0x7f5f1c019d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5c4f2ced34a47b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:24.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:24.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f0cff9640 1 -- 172.21.15.172:0/4277369807 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c02a070 con 0x55ffe02f6bb0 2024-09-10T08:46:24.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f0cff9640 1 -- 172.21.15.172:0/4277369807 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5f1c02f460 con 0x55ffe02f6bb0 2024-09-10T08:46:24.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/4277369807 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f1c00ae70 con 0x55ffe02f6bb0 2024-09-10T08:46:24.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f0cff9640 1 -- 172.21.15.172:0/4277369807 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f1c038050 con 0x55ffe02f6bb0 2024-09-10T08:46:24.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f2d9d2740 10 client.15231 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:24.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.458+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/4277369807 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ffe0371300 con 0x55ffe02f6bb0 2024-09-10T08:46:24.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.459+0000 7f5f0cff9640 1 -- 172.21.15.172:0/4277369807 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f1c00ae70 con 0x55ffe02f6bb0 2024-09-10T08:46:24.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.459+0000 7f5f0cff9640 1 -- 172.21.15.172:0/4277369807 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f5f1c024070 con 0x55ffe02f6bb0 2024-09-10T08:46:24.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.459+0000 7f5f2d9d2740 10 client.15231 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:24.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.459+0000 7f5f2d9d2740 10 client.15231 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-10T08:46:24.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.459+0000 7f5f2d9d2740 20 client.15231 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:24.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.459+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/4277369807 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55ffe0375a90 0x55ffe03747a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:24.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.459+0000 7f5f2d9d2740 4 client.15231 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:46:24.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.459+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/4277369807 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "status"}) -- 0x55ffe0259b10 con 0x55ffe0375a90 2024-09-10T08:46:24.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.459+0000 7f5f29395640 1 --2- 172.21.15.172:0/4277369807 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55ffe0375a90 0x55ffe03747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:24.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.460+0000 7f5f29395640 1 --2- 172.21.15.172:0/4277369807 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55ffe0375a90 0x55ffe03747a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cca8f8c0202589eb server_cookie=bab601501b4b5f7b in_seq=0 out_seq=0 2024-09-10T08:46:24.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.460+0000 7f5f0cff9640 10 client.15231 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:46:24.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.460+0000 7f5f0cff9640 1 -- 172.21.15.172:0/4277369807 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ffe0259b10 con 0x55ffe0375a90 2024-09-10T08:46:24.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.460+0000 7f5f0cff9640 10 client.15231 handle_command_reply: tid=0 2024-09-10T08:46:24.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.460+0000 7f5f2d9d2740 1 client.15231 shutdown 2024-09-10T08:46:24.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.460+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/4277369807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02f6bb0 msgr2=0x55ffe0310630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:24.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.460+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/4277369807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02f6bb0 0x55ffe0310630 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c014a50 tx=0x7f5f1c019d30 comp rx=0 tx=0).stop 2024-09-10T08:46:24.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.460+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/4277369807 shutdown_connections 2024-09-10T08:46:24.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.460+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/4277369807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe02f6bb0 0x55ffe0310630 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.460+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/4277369807 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55ffe0375a90 0x55ffe03747a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.460+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/4277369807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe02ec2f0 0x55ffe030c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.460+0000 7f5f2d9d2740 1 --2- 172.21.15.172:0/4277369807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe02eb9d0 0x55ffe030bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.460+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/4277369807 >> 172.21.15.172:0/4277369807 conn(0x55ffe02e7f60 msgr2=0x55ffe02e7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:24.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.460+0000 7f5f0cff9640 0 client.15231 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:46:24.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.460+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/4277369807 shutdown_connections 2024-09-10T08:46:24.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.460+0000 7f5f2d9d2740 1 -- 172.21.15.172:0/4277369807 wait complete. 2024-09-10T08:46:24.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.460+0000 7f5f2d9d2740 20 client.15231 trim_cache size 0 max 16384 2024-09-10T08:46:24.469 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:46:24.469 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:46:24.469 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 4, 2024-09-10T08:46:24.469 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 24406, 2024-09-10T08:46:24.469 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-09-10T08:46:24.469 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-09-10T08:46:24.470 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-09-10T08:46:24.470 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 90.237563172999998, 2024-09-10T08:46:24.470 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 28, 2024-09-10T08:46:24.470 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 92, 2024-09-10T08:46:24.470 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 88, 2024-09-10T08:46:24.470 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 174.178951329 2024-09-10T08:46:24.470 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:46:24.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.461+0000 7f5f2b5f9640 1 -- 172.21.15.172:0/3302561610 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f100a2c00 msgr2=0x7f5f100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:24.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.461+0000 7f5f2b5f9640 1 --2- 172.21.15.172:0/3302561610 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f100a2c00 0x7f5f100a50c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009440 tx=0x7f5f1c009610 comp rx=0 tx=0).stop 2024-09-10T08:46:24.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.461+0000 7f5f2b5f9640 1 -- 172.21.15.172:0/3302561610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f24100a80 msgr2=0x7f5f24197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:24.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.461+0000 7f5f2b5f9640 1 --2- 172.21.15.172:0/3302561610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f24100a80 0x7f5f24197300 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5f20007910 tx=0x7f5f20007dd0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.461+0000 7f5f2b5f9640 1 -- 172.21.15.172:0/3302561610 shutdown_connections 2024-09-10T08:46:24.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.461+0000 7f5f2b5f9640 1 --2- 172.21.15.172:0/3302561610 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f100a2c00 0x7f5f100a50c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.461+0000 7f5f2b5f9640 1 --2- 172.21.15.172:0/3302561610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f24106a80 0x7f5f2419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.461+0000 7f5f2b5f9640 1 --2- 172.21.15.172:0/3302561610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f24101420 0x7f5f24197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.461+0000 7f5f2b5f9640 1 --2- 172.21.15.172:0/3302561610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f24100a80 0x7f5f24197300 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:24.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.461+0000 7f5f2b5f9640 1 -- 172.21.15.172:0/3302561610 >> 172.21.15.172:0/3302561610 conn(0x7f5f240fc7a0 msgr2=0x7f5f240fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:24.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.461+0000 7f5f2b5f9640 1 -- 172.21.15.172:0/3302561610 shutdown_connections 2024-09-10T08:46:24.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:24.461+0000 7f5f2b5f9640 1 -- 172.21.15.172:0/3302561610 wait complete. 2024-09-10T08:46:24.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:24 smithi174 ceph-mon[34068]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 639 B/s wr, 11 op/s 2024-09-10T08:46:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:24 smithi191 ceph-mon[35314]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 639 B/s wr, 11 op/s 2024-09-10T08:46:25.138 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-10T08:46:25.387 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:25.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.920+0000 7f1c50393640 1 -- 172.21.15.172:0/2486531961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c48106f10 msgr2=0x7f1c4810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:25.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.920+0000 7f1c50393640 1 --2- 172.21.15.172:0/2486531961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c48106f10 0x7f1c4810ea90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009ca0 tx=0x7f1c4002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.921+0000 7f1c50393640 1 -- 172.21.15.172:0/2486531961 shutdown_connections 2024-09-10T08:46:25.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.921+0000 7f1c50393640 1 --2- 172.21.15.172:0/2486531961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c48106f10 0x7f1c4810ea90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.921+0000 7f1c50393640 1 --2- 172.21.15.172:0/2486531961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c48102790 0x7f1c481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.921+0000 7f1c50393640 1 --2- 172.21.15.172:0/2486531961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c48101de0 0x7f1c481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.921+0000 7f1c50393640 1 -- 172.21.15.172:0/2486531961 >> 172.21.15.172:0/2486531961 conn(0x7f1c480fd8c0 msgr2=0x7f1c480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:25.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.921+0000 7f1c50393640 1 -- 172.21.15.172:0/2486531961 shutdown_connections 2024-09-10T08:46:25.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.921+0000 7f1c50393640 1 -- 172.21.15.172:0/2486531961 wait complete. 2024-09-10T08:46:25.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.921+0000 7f1c50393640 1 Processor -- start 2024-09-10T08:46:25.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.921+0000 7f1c50393640 1 -- start start 2024-09-10T08:46:25.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c50393640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c48101de0 0x7f1c480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c50393640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c48102790 0x7f1c48074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c50393640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c48106f10 0x7f1c48074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c50393640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1c48111e40 con 0x7f1c48102790 2024-09-10T08:46:25.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c50393640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c48111cc0 con 0x7f1c48106f10 2024-09-10T08:46:25.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c50393640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1c48111fc0 con 0x7f1c48101de0 2024-09-10T08:46:25.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c4d92e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c48102790 0x7f1c48074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:25.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c4d92e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c48102790 0x7f1c48074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38010/0 (socket says 172.21.15.172:38010) 2024-09-10T08:46:25.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c4d92e640 1 -- 172.21.15.172:0/4259747657 learned_addr learned my addr 172.21.15.172:0/4259747657 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:25.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c4e930640 1 --2- 172.21.15.172:0/4259747657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c48106f10 0x7f1c48074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:25.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c4e12f640 1 --2- 172.21.15.172:0/4259747657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c48101de0 0x7f1c480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:25.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c4d92e640 1 -- 172.21.15.172:0/4259747657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c48101de0 msgr2=0x7f1c480744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:25.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c4d92e640 1 --2- 172.21.15.172:0/4259747657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c48101de0 0x7f1c480744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c4d92e640 1 -- 172.21.15.172:0/4259747657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c48106f10 msgr2=0x7f1c48074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:25.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c4d92e640 1 --2- 172.21.15.172:0/4259747657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c48106f10 0x7f1c48074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c4d92e640 1 -- 172.21.15.172:0/4259747657 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c481a6880 con 0x7f1c48102790 2024-09-10T08:46:25.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c4e930640 1 --2- 172.21.15.172:0/4259747657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c48106f10 0x7f1c48074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:25.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c4d92e640 1 --2- 172.21.15.172:0/4259747657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c48102790 0x7f1c48074a30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800d950 tx=0x7f1c3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:25.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.922+0000 7f1c3f7fe640 1 -- 172.21.15.172:0/4259747657 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38017070 con 0x7f1c48102790 2024-09-10T08:46:25.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.923+0000 7f1c50393640 1 -- 172.21.15.172:0/4259747657 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c481a6b70 con 0x7f1c48102790 2024-09-10T08:46:25.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.923+0000 7f1c3f7fe640 1 -- 172.21.15.172:0/4259747657 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c38009c30 con 0x7f1c48102790 2024-09-10T08:46:25.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.923+0000 7f1c3f7fe640 1 -- 172.21.15.172:0/4259747657 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3800f620 con 0x7f1c48102790 2024-09-10T08:46:25.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.923+0000 7f1c50393640 1 -- 172.21.15.172:0/4259747657 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c481a70b0 con 0x7f1c48102790 2024-09-10T08:46:25.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.925+0000 7f1c3f7fe640 1 -- 172.21.15.172:0/4259747657 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1c380109a0 con 0x7f1c48102790 2024-09-10T08:46:25.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.925+0000 7f1c5276c740 1 Processor -- start 2024-09-10T08:46:25.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.925+0000 7f1c5276c740 1 -- start start 2024-09-10T08:46:25.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.925+0000 7f1c5276c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628656469d0 0x562865646db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.925+0000 7f1c5276c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628656472f0 0x562865651670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c5276c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562865651bb0 0x562865653fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c5276c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5628655ad090 con 0x5628656472f0 2024-09-10T08:46:25.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c5276c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562865554a70 con 0x562865651bb0 2024-09-10T08:46:25.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c5276c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5628655a9050 con 0x5628656469d0 2024-09-10T08:46:25.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c3f7fe640 1 --2- 172.21.15.172:0/4259747657 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1c340a2b80 0x7f1c340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c4e12f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628656472f0 0x562865651670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:25.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c4e12f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628656472f0 0x562865651670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38014/0 (socket says 172.21.15.172:38014) 2024-09-10T08:46:25.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c4e12f640 1 -- 172.21.15.172:0/4180529603 learned_addr learned my addr 172.21.15.172:0/4180529603 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:25.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c4e930640 1 --2- 172.21.15.172:0/4259747657 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1c340a2b80 0x7f1c340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:25.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c4e930640 1 --2- 172.21.15.172:0/4180529603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628656469d0 0x562865646db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:25.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c3f7fe640 1 -- 172.21.15.172:0/4259747657 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1c380cf080 con 0x7f1c48102790 2024-09-10T08:46:25.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c4d92e640 1 --2- 172.21.15.172:0/4180529603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562865651bb0 0x562865653fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:25.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c4e12f640 1 -- 172.21.15.172:0/4180529603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628656469d0 msgr2=0x562865646db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:25.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c4e12f640 1 --2- 172.21.15.172:0/4180529603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628656469d0 0x562865646db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c4e12f640 1 -- 172.21.15.172:0/4180529603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562865651bb0 msgr2=0x562865653fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:25.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c4e12f640 1 --2- 172.21.15.172:0/4180529603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562865651bb0 0x562865653fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c4e12f640 1 -- 172.21.15.172:0/4180529603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562865561130 con 0x5628656472f0 2024-09-10T08:46:25.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c4e930640 1 --2- 172.21.15.172:0/4180529603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628656469d0 0x562865646db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:25.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c4e930640 1 --2- 172.21.15.172:0/4259747657 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1c340a2b80 0x7f1c340a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009430 tx=0x7f1c40005c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:25.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c4e12f640 1 --2- 172.21.15.172:0/4180529603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628656472f0 0x562865651670 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400b820 tx=0x7f1c4400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e50399770a9df19 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:25.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.926+0000 7f1c3cdea640 1 -- 172.21.15.172:0/4180529603 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4400d950 con 0x5628656472f0 2024-09-10T08:46:25.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.927+0000 7f1c3cdea640 1 -- 172.21.15.172:0/4180529603 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c44019070 con 0x5628656472f0 2024-09-10T08:46:25.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.927+0000 7f1c3cdea640 1 -- 172.21.15.172:0/4180529603 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c440123e0 con 0x5628656472f0 2024-09-10T08:46:25.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.927+0000 7f1c5276c740 1 -- 172.21.15.172:0/4180529603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628656472f0 msgr2=0x562865651670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:25.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.927+0000 7f1c5276c740 1 --2- 172.21.15.172:0/4180529603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628656472f0 0x562865651670 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400b820 tx=0x7f1c4400bce0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.927+0000 7f1c5276c740 1 -- 172.21.15.172:0/4180529603 shutdown_connections 2024-09-10T08:46:25.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.927+0000 7f1c5276c740 1 --2- 172.21.15.172:0/4180529603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562865651bb0 0x562865653fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.927+0000 7f1c5276c740 1 --2- 172.21.15.172:0/4180529603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628656472f0 0x562865651670 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.927+0000 7f1c5276c740 1 --2- 172.21.15.172:0/4180529603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628656469d0 0x562865646db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.927+0000 7f1c5276c740 1 -- 172.21.15.172:0/4180529603 >> 172.21.15.172:0/4180529603 conn(0x562865642f60 msgr2=0x562865643340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:25.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.927+0000 7f1c5276c740 1 -- 172.21.15.172:0/4180529603 shutdown_connections 2024-09-10T08:46:25.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.927+0000 7f1c5276c740 1 -- 172.21.15.172:0/4180529603 wait complete. 2024-09-10T08:46:25.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.928+0000 7f1c5276c740 1 -- start start 2024-09-10T08:46:25.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.928+0000 7f1c5276c740 1 Processor -- start 2024-09-10T08:46:25.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.928+0000 7f1c5276c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628656469d0 0x5628656513f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.928+0000 7f1c5276c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628656472f0 0x56286564e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.928+0000 7f1c5276c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562865651bb0 0x56286564ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.928+0000 7f1c5276c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562865654780 con 0x562865651bb0 2024-09-10T08:46:25.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.928+0000 7f1c5276c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562865567f20 con 0x5628656469d0 2024-09-10T08:46:25.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.928+0000 7f1c5276c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562865654900 con 0x5628656472f0 2024-09-10T08:46:25.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.928+0000 7f1c5276c740 10 client.0 _ll_register_callbacks cb 0x562865531030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:25.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.928+0000 7f1c4e930640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562865651bb0 0x56286564ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:25.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.928+0000 7f1c4e930640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562865651bb0 0x56286564ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38024/0 (socket says 172.21.15.172:38024) 2024-09-10T08:46:25.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.928+0000 7f1c4e930640 1 -- 172.21.15.172:0/1284770875 learned_addr learned my addr 172.21.15.172:0/1284770875 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:25.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.928+0000 7f1c4e12f640 1 --2- 172.21.15.172:0/1284770875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628656469d0 0x5628656513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:25.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.928+0000 7f1c4d92e640 1 --2- 172.21.15.172:0/1284770875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628656472f0 0x56286564e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:25.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.929+0000 7f1c4e930640 1 -- 172.21.15.172:0/1284770875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628656472f0 msgr2=0x56286564e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:25.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.929+0000 7f1c4e930640 1 --2- 172.21.15.172:0/1284770875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628656472f0 0x56286564e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.929+0000 7f1c4e930640 1 -- 172.21.15.172:0/1284770875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628656469d0 msgr2=0x5628656513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:25.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.929+0000 7f1c4e930640 1 --2- 172.21.15.172:0/1284770875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628656469d0 0x5628656513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.929+0000 7f1c4e930640 1 -- 172.21.15.172:0/1284770875 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56286531e2b0 con 0x562865651bb0 2024-09-10T08:46:25.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.929+0000 7f1c4e930640 1 --2- 172.21.15.172:0/1284770875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562865651bb0 0x56286564ec80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1c40035000 tx=0x7f1c4002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef512cdfb778d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:25.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.929+0000 7f1c31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:25.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.929+0000 7f1c31ffb640 1 -- 172.21.15.172:0/1284770875 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c400045c0 con 0x562865651bb0 2024-09-10T08:46:25.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.929+0000 7f1c31ffb640 1 -- 172.21.15.172:0/1284770875 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1c40004760 con 0x562865651bb0 2024-09-10T08:46:25.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.929+0000 7f1c5276c740 1 -- 172.21.15.172:0/1284770875 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c44019070 con 0x562865651bb0 2024-09-10T08:46:25.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.929+0000 7f1c31ffb640 1 -- 172.21.15.172:0/1284770875 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1c4005b050 con 0x562865651bb0 2024-09-10T08:46:25.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.929+0000 7f1c5276c740 10 client.15243 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:25.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.929+0000 7f1c5276c740 1 -- 172.21.15.172:0/1284770875 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562865660040 con 0x562865651bb0 2024-09-10T08:46:25.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.929+0000 7f1c31ffb640 1 -- 172.21.15.172:0/1284770875 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c44019070 con 0x562865651bb0 2024-09-10T08:46:25.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.929+0000 7f1c31ffb640 1 -- 172.21.15.172:0/1284770875 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f1c4002f4c0 con 0x562865651bb0 2024-09-10T08:46:25.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.930+0000 7f1c5276c740 10 client.15243 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:25.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.930+0000 7f1c5276c740 10 client.15243 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T08:46:25.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.930+0000 7f1c5276c740 20 client.15243 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:25.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.930+0000 7f1c5276c740 1 --2- 172.21.15.172:0/1284770875 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5628656626f0 0x562865663b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.930+0000 7f1c5276c740 4 client.15243 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:25.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.930+0000 7f1c5276c740 1 -- 172.21.15.172:0/1284770875 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628655ba600 con 0x5628656626f0 2024-09-10T08:46:25.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.930+0000 7f1c4e12f640 1 --2- 172.21.15.172:0/1284770875 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5628656626f0 0x562865663b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:25.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.930+0000 7f1c4e12f640 1 --2- 172.21.15.172:0/1284770875 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5628656626f0 0x562865663b60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a383a2bda056b831 server_cookie=c3aa4c78345edbe1 in_seq=0 out_seq=0 2024-09-10T08:46:25.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.930+0000 7f1c31ffb640 10 client.15243 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:46:25.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.931+0000 7f1c31ffb640 1 -- 172.21.15.172:0/1284770875 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628655ba600 con 0x5628656626f0 2024-09-10T08:46:25.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.931+0000 7f1c31ffb640 10 client.15243 handle_command_reply: tid=0 2024-09-10T08:46:25.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.931+0000 7f1c5276c740 1 client.15243 shutdown 2024-09-10T08:46:25.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.931+0000 7f1c5276c740 1 -- 172.21.15.172:0/1284770875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562865651bb0 msgr2=0x56286564ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.931+0000 7f1c5276c740 1 --2- 172.21.15.172:0/1284770875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562865651bb0 0x56286564ec80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1c40035000 tx=0x7f1c4002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.932+0000 7f1c5276c740 1 -- 172.21.15.172:0/1284770875 shutdown_connections 2024-09-10T08:46:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.932+0000 7f1c5276c740 1 --2- 172.21.15.172:0/1284770875 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5628656626f0 0x562865663b60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.932+0000 7f1c5276c740 1 --2- 172.21.15.172:0/1284770875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562865651bb0 0x56286564ec80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.932+0000 7f1c5276c740 1 --2- 172.21.15.172:0/1284770875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628656472f0 0x56286564e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.932+0000 7f1c5276c740 1 --2- 172.21.15.172:0/1284770875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628656469d0 0x5628656513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.932+0000 7f1c5276c740 1 -- 172.21.15.172:0/1284770875 >> 172.21.15.172:0/1284770875 conn(0x562865642f60 msgr2=0x562865648c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.932+0000 7f1c31ffb640 0 client.15243 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:46:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.932+0000 7f1c5276c740 1 -- 172.21.15.172:0/1284770875 shutdown_connections 2024-09-10T08:46:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.932+0000 7f1c5276c740 1 -- 172.21.15.172:0/1284770875 wait complete. 2024-09-10T08:46:25.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.932+0000 7f1c5276c740 20 client.15243 trim_cache size 0 max 16384 2024-09-10T08:46:25.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.949+0000 7f1c5276c740 1 Processor -- start 2024-09-10T08:46:25.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.949+0000 7f1c5276c740 1 -- start start 2024-09-10T08:46:25.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.949+0000 7f1c5276c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628656469d0 0x5628656626f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.949+0000 7f1c5276c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628656472f0 0x562865642880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.949+0000 7f1c5276c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562865651bb0 0x562865661390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.949+0000 7f1c5276c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562865661ea0 con 0x5628656472f0 2024-09-10T08:46:25.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.949+0000 7f1c5276c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562865662570 con 0x5628656469d0 2024-09-10T08:46:25.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.949+0000 7f1c5276c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562865662c40 con 0x562865651bb0 2024-09-10T08:46:25.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.950+0000 7f1c4d92e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628656472f0 0x562865642880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:25.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.950+0000 7f1c4d92e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628656472f0 0x562865642880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38028/0 (socket says 172.21.15.172:38028) 2024-09-10T08:46:25.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.950+0000 7f1c4d92e640 1 -- 172.21.15.172:0/3044139309 learned_addr learned my addr 172.21.15.172:0/3044139309 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:25.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.950+0000 7f1c4d92e640 1 -- 172.21.15.172:0/3044139309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562865651bb0 msgr2=0x562865661390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:25.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.950+0000 7f1c4d92e640 1 --2- 172.21.15.172:0/3044139309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562865651bb0 0x562865661390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.950+0000 7f1c4d92e640 1 -- 172.21.15.172:0/3044139309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628656469d0 msgr2=0x5628656626f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:25.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.950+0000 7f1c4e12f640 1 --2- 172.21.15.172:0/3044139309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628656469d0 0x5628656626f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:25.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.950+0000 7f1c4d92e640 1 --2- 172.21.15.172:0/3044139309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628656469d0 0x5628656626f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.950+0000 7f1c4d92e640 1 -- 172.21.15.172:0/3044139309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562865651760 con 0x5628656472f0 2024-09-10T08:46:25.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.950+0000 7f1c4e12f640 1 --2- 172.21.15.172:0/3044139309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628656469d0 0x5628656626f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:25.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.950+0000 7f1c4d92e640 1 --2- 172.21.15.172:0/3044139309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628656472f0 0x562865642880 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1c380979e0 tx=0x7f1c380ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5dd9004330d85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:25.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.950+0000 7f1c33fff640 1 -- 172.21.15.172:0/3044139309 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c380b4070 con 0x5628656472f0 2024-09-10T08:46:25.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.950+0000 7f1c33fff640 1 -- 172.21.15.172:0/3044139309 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c38009de0 con 0x5628656472f0 2024-09-10T08:46:25.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.950+0000 7f1c33fff640 1 -- 172.21.15.172:0/3044139309 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38020040 con 0x5628656472f0 2024-09-10T08:46:25.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.951+0000 7f1c5276c740 1 -- 172.21.15.172:0/3044139309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628656472f0 msgr2=0x562865642880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:25.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.951+0000 7f1c5276c740 1 --2- 172.21.15.172:0/3044139309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628656472f0 0x562865642880 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1c380979e0 tx=0x7f1c380ce040 comp rx=0 tx=0).stop 2024-09-10T08:46:25.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.951+0000 7f1c5276c740 1 -- 172.21.15.172:0/3044139309 shutdown_connections 2024-09-10T08:46:25.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.951+0000 7f1c5276c740 1 --2- 172.21.15.172:0/3044139309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562865651bb0 0x562865661390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.951+0000 7f1c5276c740 1 --2- 172.21.15.172:0/3044139309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5628656472f0 0x562865642880 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.951+0000 7f1c5276c740 1 --2- 172.21.15.172:0/3044139309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628656469d0 0x5628656626f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.951+0000 7f1c5276c740 1 -- 172.21.15.172:0/3044139309 >> 172.21.15.172:0/3044139309 conn(0x562865642f60 msgr2=0x56286564e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:25.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.951+0000 7f1c5276c740 1 -- 172.21.15.172:0/3044139309 shutdown_connections 2024-09-10T08:46:25.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.951+0000 7f1c5276c740 1 -- 172.21.15.172:0/3044139309 wait complete. 2024-09-10T08:46:25.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.951+0000 7f1c5276c740 1 -- start start 2024-09-10T08:46:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.951+0000 7f1c5276c740 1 Processor -- start 2024-09-10T08:46:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.951+0000 7f1c5276c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628656469d0 0x562865664d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.951+0000 7f1c5276c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628656472f0 0x5628656652a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.951+0000 7f1c5276c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562865651bb0 0x5628656657e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.951+0000 7f1c5276c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5628656639e0 con 0x562865651bb0 2024-09-10T08:46:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.951+0000 7f1c5276c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562865663310 con 0x5628656469d0 2024-09-10T08:46:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.951+0000 7f1c5276c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5628656640b0 con 0x5628656472f0 2024-09-10T08:46:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c5276c740 10 client.0 _ll_register_callbacks cb 0x7f1c40009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c4e930640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562865651bb0 0x5628656657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c4e12f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628656469d0 0x562865664d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c4e930640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562865651bb0 0x5628656657e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38038/0 (socket says 172.21.15.172:38038) 2024-09-10T08:46:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c4e930640 1 -- 172.21.15.172:0/2349934578 learned_addr learned my addr 172.21.15.172:0/2349934578 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:25.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c4d92e640 1 --2- 172.21.15.172:0/2349934578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628656472f0 0x5628656652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:25.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c4e930640 1 -- 172.21.15.172:0/2349934578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628656472f0 msgr2=0x5628656652a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:25.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c4e930640 1 --2- 172.21.15.172:0/2349934578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628656472f0 0x5628656652a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c4e930640 1 -- 172.21.15.172:0/2349934578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628656469d0 msgr2=0x562865664d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:25.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c4e930640 1 --2- 172.21.15.172:0/2349934578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628656469d0 0x562865664d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c4e930640 1 -- 172.21.15.172:0/2349934578 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628656cadf0 con 0x562865651bb0 2024-09-10T08:46:25.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c4e930640 1 --2- 172.21.15.172:0/2349934578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562865651bb0 0x5628656657e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1c40003a10 tx=0x7f1c4002f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31dd9f21d27f1c47 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:25.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:25.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c31ffb640 1 -- 172.21.15.172:0/2349934578 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40031650 con 0x562865651bb0 2024-09-10T08:46:25.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c31ffb640 1 -- 172.21.15.172:0/2349934578 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1c40004410 con 0x562865651bb0 2024-09-10T08:46:25.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c5276c740 1 -- 172.21.15.172:0/2349934578 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c38009de0 con 0x562865651bb0 2024-09-10T08:46:25.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c31ffb640 1 -- 172.21.15.172:0/2349934578 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1c4006c050 con 0x562865651bb0 2024-09-10T08:46:25.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c5276c740 10 client.15249 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:25.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.952+0000 7f1c5276c740 1 -- 172.21.15.172:0/2349934578 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5628656cbf60 con 0x562865651bb0 2024-09-10T08:46:25.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.953+0000 7f1c31ffb640 1 -- 172.21.15.172:0/2349934578 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c38009de0 con 0x562865651bb0 2024-09-10T08:46:25.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.953+0000 7f1c31ffb640 1 -- 172.21.15.172:0/2349934578 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f1c40067030 con 0x562865651bb0 2024-09-10T08:46:25.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.953+0000 7f1c5276c740 10 client.15249 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:25.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.953+0000 7f1c5276c740 10 client.15249 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T08:46:25.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.953+0000 7f1c5276c740 20 client.15249 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:25.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.953+0000 7f1c5276c740 1 --2- 172.21.15.172:0/2349934578 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5628656d0b00 0x5628656cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:25.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.953+0000 7f1c5276c740 4 client.15249 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:46:25.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.953+0000 7f1c5276c740 1 -- 172.21.15.172:0/2349934578 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "status"}) -- 0x5628655b4b10 con 0x5628656d0b00 2024-09-10T08:46:25.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.953+0000 7f1c4e12f640 1 --2- 172.21.15.172:0/2349934578 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5628656d0b00 0x5628656cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:25.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.953+0000 7f1c4e12f640 1 --2- 172.21.15.172:0/2349934578 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5628656d0b00 0x5628656cf810 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3282f8c29f2826d5 server_cookie=a7f251008b81651b in_seq=0 out_seq=0 2024-09-10T08:46:25.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.953+0000 7f1c31ffb640 10 client.15249 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:46:25.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c31ffb640 1 -- 172.21.15.172:0/2349934578 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5628655b4b10 con 0x5628656d0b00 2024-09-10T08:46:25.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c31ffb640 10 client.15249 handle_command_reply: tid=0 2024-09-10T08:46:25.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c5276c740 1 client.15249 shutdown 2024-09-10T08:46:25.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c5276c740 1 -- 172.21.15.172:0/2349934578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562865651bb0 msgr2=0x5628656657e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:25.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c5276c740 1 --2- 172.21.15.172:0/2349934578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562865651bb0 0x5628656657e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1c40003a10 tx=0x7f1c4002f6f0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c5276c740 1 -- 172.21.15.172:0/2349934578 shutdown_connections 2024-09-10T08:46:25.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c5276c740 1 --2- 172.21.15.172:0/2349934578 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5628656d0b00 0x5628656cf810 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c5276c740 1 --2- 172.21.15.172:0/2349934578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562865651bb0 0x5628656657e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c5276c740 1 --2- 172.21.15.172:0/2349934578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628656472f0 0x5628656652a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c5276c740 1 --2- 172.21.15.172:0/2349934578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628656469d0 0x562865664d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c5276c740 1 -- 172.21.15.172:0/2349934578 >> 172.21.15.172:0/2349934578 conn(0x562865642f60 msgr2=0x562865661390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:25.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c31ffb640 0 client.15249 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:46:25.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c5276c740 1 -- 172.21.15.172:0/2349934578 shutdown_connections 2024-09-10T08:46:25.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c5276c740 1 -- 172.21.15.172:0/2349934578 wait complete. 2024-09-10T08:46:25.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c5276c740 20 client.15249 trim_cache size 0 max 16384 2024-09-10T08:46:25.962 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:46:25.962 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:46:25.962 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 3, 2024-09-10T08:46:25.962 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 24416, 2024-09-10T08:46:25.962 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-09-10T08:46:25.962 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-09-10T08:46:25.962 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-09-10T08:46:25.962 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 92.743880059000006, 2024-09-10T08:46:25.962 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 28, 2024-09-10T08:46:25.963 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 92, 2024-09-10T08:46:25.963 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 88, 2024-09-10T08:46:25.963 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 172.87543751999999 2024-09-10T08:46:25.963 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:46:25.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c50393640 1 -- 172.21.15.172:0/4259747657 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1c340a2b80 msgr2=0x7f1c340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:25.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c50393640 1 --2- 172.21.15.172:0/4259747657 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1c340a2b80 0x7f1c340a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009430 tx=0x7f1c40005c30 comp rx=0 tx=0).stop 2024-09-10T08:46:25.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c50393640 1 -- 172.21.15.172:0/4259747657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c48102790 msgr2=0x7f1c48074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:25.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c50393640 1 --2- 172.21.15.172:0/4259747657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c48102790 0x7f1c48074a30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800d950 tx=0x7f1c3800de10 comp rx=0 tx=0).stop 2024-09-10T08:46:25.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.954+0000 7f1c50393640 1 -- 172.21.15.172:0/4259747657 shutdown_connections 2024-09-10T08:46:25.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.955+0000 7f1c50393640 1 --2- 172.21.15.172:0/4259747657 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1c340a2b80 0x7f1c340a5040 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.955+0000 7f1c50393640 1 --2- 172.21.15.172:0/4259747657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c48106f10 0x7f1c48074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.955+0000 7f1c50393640 1 --2- 172.21.15.172:0/4259747657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c48102790 0x7f1c48074a30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.955+0000 7f1c50393640 1 --2- 172.21.15.172:0/4259747657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c48101de0 0x7f1c480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:25.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.955+0000 7f1c50393640 1 -- 172.21.15.172:0/4259747657 >> 172.21.15.172:0/4259747657 conn(0x7f1c480fd8c0 msgr2=0x7f1c480fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:25.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.955+0000 7f1c50393640 1 -- 172.21.15.172:0/4259747657 shutdown_connections 2024-09-10T08:46:25.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:25.955+0000 7f1c50393640 1 -- 172.21.15.172:0/4259747657 wait complete. 2024-09-10T08:46:26.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:26 smithi174 ceph-mon[34068]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 9 op/s 2024-09-10T08:46:26.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:26 smithi191 ceph-mon[35314]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 9 op/s 2024-09-10T08:46:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:26 smithi172 ceph-mon[30854]: pgmap v356: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 9 op/s 2024-09-10T08:46:26.639 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T08:46:26.884 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:27.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.406+0000 7f363b2e3640 1 -- 172.21.15.172:0/4165761497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3634105f40 msgr2=0x7f36341063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:27.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.406+0000 7f363b2e3640 1 --2- 172.21.15.172:0/4165761497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3634105f40 0x7f36341063d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3624009900 tx=0x7f362402a470 comp rx=0 tx=0).stop 2024-09-10T08:46:27.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.406+0000 7f363b2e3640 1 -- 172.21.15.172:0/4165761497 shutdown_connections 2024-09-10T08:46:27.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.406+0000 7f363b2e3640 1 --2- 172.21.15.172:0/4165761497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3634105f40 0x7f36341063d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.406+0000 7f363b2e3640 1 --2- 172.21.15.172:0/4165761497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3634101450 0x7f3634105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.406+0000 7f363b2e3640 1 --2- 172.21.15.172:0/4165761497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3634100b30 0x7f3634100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.406+0000 7f363b2e3640 1 -- 172.21.15.172:0/4165761497 >> 172.21.15.172:0/4165761497 conn(0x7f36340fc7a0 msgr2=0x7f36340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:27.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.406+0000 7f363b2e3640 1 -- 172.21.15.172:0/4165761497 shutdown_connections 2024-09-10T08:46:27.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.406+0000 7f363b2e3640 1 -- 172.21.15.172:0/4165761497 wait complete. 2024-09-10T08:46:27.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363b2e3640 1 Processor -- start 2024-09-10T08:46:27.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363b2e3640 1 -- start start 2024-09-10T08:46:27.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363b2e3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3634100b30 0x7f36341a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363b2e3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3634101450 0x7f36341a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363b2e3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3634105f40 0x7f363419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363b2e3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3634118470 con 0x7f3634100b30 2024-09-10T08:46:27.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363b2e3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36341182f0 con 0x7f3634105f40 2024-09-10T08:46:27.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363b2e3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f36341185f0 con 0x7f3634101450 2024-09-10T08:46:27.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363907f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3634100b30 0x7f36341a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363907f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3634100b30 0x7f36341a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33078/0 (socket says 172.21.15.172:33078) 2024-09-10T08:46:27.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363907f640 1 -- 172.21.15.172:0/1801978641 learned_addr learned my addr 172.21.15.172:0/1801978641 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:27.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363887e640 1 --2- 172.21.15.172:0/1801978641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3634101450 0x7f36341a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f3639880640 1 --2- 172.21.15.172:0/1801978641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3634105f40 0x7f363419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363907f640 1 -- 172.21.15.172:0/1801978641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3634101450 msgr2=0x7f36341a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:27.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363907f640 1 --2- 172.21.15.172:0/1801978641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3634101450 0x7f36341a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363907f640 1 -- 172.21.15.172:0/1801978641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3634105f40 msgr2=0x7f363419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:27.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363907f640 1 --2- 172.21.15.172:0/1801978641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3634105f40 0x7f363419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363907f640 1 -- 172.21.15.172:0/1801978641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363419efa0 con 0x7f3634100b30 2024-09-10T08:46:27.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363887e640 1 --2- 172.21.15.172:0/1801978641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3634101450 0x7f36341a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:27.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f3639880640 1 --2- 172.21.15.172:0/1801978641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3634105f40 0x7f363419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:27.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.407+0000 7f363907f640 1 --2- 172.21.15.172:0/1801978641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3634100b30 0x7f36341a4680 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f362c00c890 tx=0x7f362c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:27.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.408+0000 7f36327fc640 1 -- 172.21.15.172:0/1801978641 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c004220 con 0x7f3634100b30 2024-09-10T08:46:27.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.408+0000 7f36327fc640 1 -- 172.21.15.172:0/1801978641 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f362c0043c0 con 0x7f3634100b30 2024-09-10T08:46:27.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.408+0000 7f36327fc640 1 -- 172.21.15.172:0/1801978641 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c004a00 con 0x7f3634100b30 2024-09-10T08:46:27.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.408+0000 7f363b2e3640 1 -- 172.21.15.172:0/1801978641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363419f290 con 0x7f3634100b30 2024-09-10T08:46:27.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.408+0000 7f363b2e3640 1 -- 172.21.15.172:0/1801978641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36341ab4e0 con 0x7f3634100b30 2024-09-10T08:46:27.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.410+0000 7f36327fc640 1 -- 172.21.15.172:0/1801978641 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f362c004ba0 con 0x7f3634100b30 2024-09-10T08:46:27.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.410+0000 7f363d6bc740 1 Processor -- start 2024-09-10T08:46:27.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f363d6bc740 1 -- start start 2024-09-10T08:46:27.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f363d6bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba1c9d0 0x560bfba1cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f363d6bc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1d2f0 0x560bfba27670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f363d6bc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba29fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f363d6bc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560bfb983090 con 0x560bfba27bb0 2024-09-10T08:46:27.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f363d6bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560bfb92aa70 con 0x560bfba1c9d0 2024-09-10T08:46:27.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f363d6bc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560bfb97f050 con 0x560bfba1d2f0 2024-09-10T08:46:27.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f363887e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba29fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f36327fc640 1 --2- 172.21.15.172:0/1801978641 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f361c0a2b80 0x7f361c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f363887e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba29fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33090/0 (socket says 172.21.15.172:33090) 2024-09-10T08:46:27.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f363887e640 1 -- 172.21.15.172:0/3149690375 learned_addr learned my addr 172.21.15.172:0/3149690375 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:27.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f363907f640 1 --2- 172.21.15.172:0/3149690375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1d2f0 0x560bfba27670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f3639880640 1 --2- 172.21.15.172:0/1801978641 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f361c0a2b80 0x7f361c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f3639880640 1 --2- 172.21.15.172:0/3149690375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba1c9d0 0x560bfba1cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f363887e640 1 -- 172.21.15.172:0/3149690375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1d2f0 msgr2=0x560bfba27670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:27.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f363887e640 1 --2- 172.21.15.172:0/3149690375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1d2f0 0x560bfba27670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f363887e640 1 -- 172.21.15.172:0/3149690375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba1c9d0 msgr2=0x560bfba1cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:27.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f363887e640 1 --2- 172.21.15.172:0/3149690375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba1c9d0 0x560bfba1cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f363887e640 1 -- 172.21.15.172:0/3149690375 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bfb937130 con 0x560bfba27bb0 2024-09-10T08:46:27.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.411+0000 7f36327fc640 1 -- 172.21.15.172:0/1801978641 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f362c0cf170 con 0x7f3634100b30 2024-09-10T08:46:27.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.412+0000 7f363887e640 1 --2- 172.21.15.172:0/3149690375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba29fa0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3628007f40 tx=0x7f362800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6978c36b49229eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:27.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.412+0000 7f3639880640 1 --2- 172.21.15.172:0/1801978641 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f361c0a2b80 0x7f361c0a5040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f36240096c0 tx=0x7f3624031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:27.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.412+0000 7f361b7fe640 1 -- 172.21.15.172:0/3149690375 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362800e850 con 0x560bfba27bb0 2024-09-10T08:46:27.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.412+0000 7f361b7fe640 1 -- 172.21.15.172:0/3149690375 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3628017480 con 0x560bfba27bb0 2024-09-10T08:46:27.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.412+0000 7f361b7fe640 1 -- 172.21.15.172:0/3149690375 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362800d610 con 0x560bfba27bb0 2024-09-10T08:46:27.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.413+0000 7f363d6bc740 1 -- 172.21.15.172:0/3149690375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 msgr2=0x560bfba29fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:27.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.413+0000 7f363d6bc740 1 --2- 172.21.15.172:0/3149690375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba29fa0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3628007f40 tx=0x7f362800c7f0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.413+0000 7f363d6bc740 1 -- 172.21.15.172:0/3149690375 shutdown_connections 2024-09-10T08:46:27.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.413+0000 7f363d6bc740 1 --2- 172.21.15.172:0/3149690375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba29fa0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.413+0000 7f363d6bc740 1 --2- 172.21.15.172:0/3149690375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1d2f0 0x560bfba27670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.413+0000 7f363d6bc740 1 --2- 172.21.15.172:0/3149690375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba1c9d0 0x560bfba1cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.413+0000 7f363d6bc740 1 -- 172.21.15.172:0/3149690375 >> 172.21.15.172:0/3149690375 conn(0x560bfba18f60 msgr2=0x560bfba19340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:27.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.413+0000 7f363d6bc740 1 -- 172.21.15.172:0/3149690375 shutdown_connections 2024-09-10T08:46:27.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.413+0000 7f363d6bc740 1 -- 172.21.15.172:0/3149690375 wait complete. 2024-09-10T08:46:27.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.413+0000 7f363d6bc740 1 -- start start 2024-09-10T08:46:27.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.413+0000 7f363d6bc740 1 Processor -- start 2024-09-10T08:46:27.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.413+0000 7f363d6bc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1c9d0 0x560bfba274c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.413+0000 7f363d6bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba1d2f0 0x560bfba24810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.413+0000 7f363d6bc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba24d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f363d6bc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560bfba2a780 con 0x560bfba27bb0 2024-09-10T08:46:27.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f363d6bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560bfb93df20 con 0x560bfba1d2f0 2024-09-10T08:46:27.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f363d6bc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560bfba2a900 con 0x560bfba1c9d0 2024-09-10T08:46:27.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f363d6bc740 10 client.0 _ll_register_callbacks cb 0x560bfb907030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:27.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f3639880640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba24d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f363907f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1c9d0 0x560bfba274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f3639880640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba24d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33096/0 (socket says 172.21.15.172:33096) 2024-09-10T08:46:27.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f3639880640 1 -- 172.21.15.172:0/3278745563 learned_addr learned my addr 172.21.15.172:0/3278745563 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:27.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f363887e640 1 --2- 172.21.15.172:0/3278745563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba1d2f0 0x560bfba24810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f3639880640 1 -- 172.21.15.172:0/3278745563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1c9d0 msgr2=0x560bfba274c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:27.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f3639880640 1 --2- 172.21.15.172:0/3278745563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1c9d0 0x560bfba274c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f3639880640 1 -- 172.21.15.172:0/3278745563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba1d2f0 msgr2=0x560bfba24810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:27.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f3639880640 1 --2- 172.21.15.172:0/3278745563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba1d2f0 0x560bfba24810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f3639880640 1 -- 172.21.15.172:0/3278745563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bfb6f42b0 con 0x560bfba27bb0 2024-09-10T08:46:27.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f3639880640 1 --2- 172.21.15.172:0/3278745563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba24d50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3624031fa0 tx=0x7f3624009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f987294a4d0fc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:27.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f3618ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:27.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f3618ff9640 1 -- 172.21.15.172:0/3278745563 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3624035040 con 0x560bfba27bb0 2024-09-10T08:46:27.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f3618ff9640 1 -- 172.21.15.172:0/3278745563 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f36240053b0 con 0x560bfba27bb0 2024-09-10T08:46:27.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f363d6bc740 1 -- 172.21.15.172:0/3278745563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3628017480 con 0x560bfba27bb0 2024-09-10T08:46:27.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f3618ff9640 1 -- 172.21.15.172:0/3278745563 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3624057050 con 0x560bfba27bb0 2024-09-10T08:46:27.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f363d6bc740 10 client.15261 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:27.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.414+0000 7f363d6bc740 1 -- 172.21.15.172:0/3278745563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560bfba36100 con 0x560bfba27bb0 2024-09-10T08:46:27.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.415+0000 7f3618ff9640 1 -- 172.21.15.172:0/3278745563 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3628017480 con 0x560bfba27bb0 2024-09-10T08:46:27.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.415+0000 7f3618ff9640 1 -- 172.21.15.172:0/3278745563 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f36240035f0 con 0x560bfba27bb0 2024-09-10T08:46:27.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.415+0000 7f363d6bc740 10 client.15261 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:27.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.415+0000 7f363d6bc740 10 client.15261 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:46:27.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.415+0000 7f363d6bc740 20 client.15261 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:27.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.415+0000 7f363d6bc740 1 --2- 172.21.15.172:0/3278745563 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560bfba387b0 0x560bfba39c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.415+0000 7f363d6bc740 4 client.15261 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:27.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.415+0000 7f363d6bc740 1 -- 172.21.15.172:0/3278745563 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bfb990600 con 0x560bfba387b0 2024-09-10T08:46:27.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.415+0000 7f363907f640 1 --2- 172.21.15.172:0/3278745563 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560bfba387b0 0x560bfba39c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.416+0000 7f363907f640 1 --2- 172.21.15.172:0/3278745563 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560bfba387b0 0x560bfba39c20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c4d42d8b78abc69 server_cookie=6932509b27cc51f4 in_seq=0 out_seq=0 2024-09-10T08:46:27.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.416+0000 7f3618ff9640 10 client.15261 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:46:27.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.416+0000 7f3618ff9640 1 -- 172.21.15.172:0/3278745563 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560bfb990600 con 0x560bfba387b0 2024-09-10T08:46:27.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.416+0000 7f3618ff9640 10 client.15261 handle_command_reply: tid=0 2024-09-10T08:46:27.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.416+0000 7f363d6bc740 1 client.15261 shutdown 2024-09-10T08:46:27.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.416+0000 7f363d6bc740 1 -- 172.21.15.172:0/3278745563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 msgr2=0x560bfba24d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:27.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.416+0000 7f363d6bc740 1 --2- 172.21.15.172:0/3278745563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba24d50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3624031fa0 tx=0x7f3624009260 comp rx=0 tx=0).stop 2024-09-10T08:46:27.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.416+0000 7f363d6bc740 1 -- 172.21.15.172:0/3278745563 shutdown_connections 2024-09-10T08:46:27.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.416+0000 7f363d6bc740 1 --2- 172.21.15.172:0/3278745563 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560bfba387b0 0x560bfba39c20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.416+0000 7f363d6bc740 1 --2- 172.21.15.172:0/3278745563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba24d50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.416+0000 7f363d6bc740 1 --2- 172.21.15.172:0/3278745563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba1d2f0 0x560bfba24810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.416+0000 7f363d6bc740 1 --2- 172.21.15.172:0/3278745563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1c9d0 0x560bfba274c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.416+0000 7f3618ff9640 0 client.15261 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:46:27.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.416+0000 7f363d6bc740 1 -- 172.21.15.172:0/3278745563 >> 172.21.15.172:0/3278745563 conn(0x560bfba18f60 msgr2=0x560bfba1ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:27.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.417+0000 7f363d6bc740 1 -- 172.21.15.172:0/3278745563 shutdown_connections 2024-09-10T08:46:27.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.417+0000 7f363d6bc740 1 -- 172.21.15.172:0/3278745563 wait complete. 2024-09-10T08:46:27.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.417+0000 7f363d6bc740 20 client.15261 trim_cache size 0 max 16384 2024-09-10T08:46:27.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.434+0000 7f363d6bc740 1 Processor -- start 2024-09-10T08:46:27.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.434+0000 7f363d6bc740 1 -- start start 2024-09-10T08:46:27.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.434+0000 7f363d6bc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba1c9d0 0x560bfba19340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.434+0000 7f363d6bc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1d2f0 0x560bfba19880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.434+0000 7f363d6bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba27bb0 0x560bfba23100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.434+0000 7f363d6bc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560bfba37f60 con 0x560bfba1c9d0 2024-09-10T08:46:27.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.434+0000 7f363d6bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560bfba38630 con 0x560bfba27bb0 2024-09-10T08:46:27.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.434+0000 7f363d6bc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560bfba38d00 con 0x560bfba1d2f0 2024-09-10T08:46:27.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.434+0000 7f363907f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba1c9d0 0x560bfba19340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.434+0000 7f363907f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba1c9d0 0x560bfba19340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33108/0 (socket says 172.21.15.172:33108) 2024-09-10T08:46:27.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.434+0000 7f363907f640 1 -- 172.21.15.172:0/2559952519 learned_addr learned my addr 172.21.15.172:0/2559952519 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:27.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.435+0000 7f363887e640 1 --2- 172.21.15.172:0/2559952519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1d2f0 0x560bfba19880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.435+0000 7f3639880640 1 --2- 172.21.15.172:0/2559952519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba27bb0 0x560bfba23100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.435+0000 7f363907f640 1 -- 172.21.15.172:0/2559952519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1d2f0 msgr2=0x560bfba19880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:27.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.435+0000 7f363907f640 1 --2- 172.21.15.172:0/2559952519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1d2f0 0x560bfba19880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.435+0000 7f363907f640 1 -- 172.21.15.172:0/2559952519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba27bb0 msgr2=0x560bfba23100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:27.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.435+0000 7f363907f640 1 --2- 172.21.15.172:0/2559952519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba27bb0 0x560bfba23100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.435+0000 7f363907f640 1 -- 172.21.15.172:0/2559952519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bfba1a0d0 con 0x560bfba1c9d0 2024-09-10T08:46:27.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.435+0000 7f3639880640 1 --2- 172.21.15.172:0/2559952519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba27bb0 0x560bfba23100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:27.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.435+0000 7f363907f640 1 --2- 172.21.15.172:0/2559952519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba1c9d0 0x560bfba19340 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f362c09b5a0 tx=0x7f362c0cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc806a0ed391c5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:27.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.435+0000 7f361affd640 1 -- 172.21.15.172:0/2559952519 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c0a4280 con 0x560bfba1c9d0 2024-09-10T08:46:27.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.435+0000 7f361affd640 1 -- 172.21.15.172:0/2559952519 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f362c0996b0 con 0x560bfba1c9d0 2024-09-10T08:46:27.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.435+0000 7f361affd640 1 -- 172.21.15.172:0/2559952519 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c084ba0 con 0x560bfba1c9d0 2024-09-10T08:46:27.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.436+0000 7f363d6bc740 1 -- 172.21.15.172:0/2559952519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba1c9d0 msgr2=0x560bfba19340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:27.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.436+0000 7f363d6bc740 1 --2- 172.21.15.172:0/2559952519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba1c9d0 0x560bfba19340 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f362c09b5a0 tx=0x7f362c0cc040 comp rx=0 tx=0).stop 2024-09-10T08:46:27.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.436+0000 7f363d6bc740 1 -- 172.21.15.172:0/2559952519 shutdown_connections 2024-09-10T08:46:27.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.436+0000 7f363d6bc740 1 --2- 172.21.15.172:0/2559952519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba27bb0 0x560bfba23100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.436+0000 7f363d6bc740 1 --2- 172.21.15.172:0/2559952519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1d2f0 0x560bfba19880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.436+0000 7f363d6bc740 1 --2- 172.21.15.172:0/2559952519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba1c9d0 0x560bfba19340 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.436+0000 7f363d6bc740 1 -- 172.21.15.172:0/2559952519 >> 172.21.15.172:0/2559952519 conn(0x560bfba18f60 msgr2=0x560bfba23d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:27.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.436+0000 7f363d6bc740 1 -- 172.21.15.172:0/2559952519 shutdown_connections 2024-09-10T08:46:27.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.436+0000 7f363d6bc740 1 -- 172.21.15.172:0/2559952519 wait complete. 2024-09-10T08:46:27.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.436+0000 7f363d6bc740 1 -- start start 2024-09-10T08:46:27.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.436+0000 7f363d6bc740 1 Processor -- start 2024-09-10T08:46:27.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f363d6bc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1c9d0 0x560bfba41fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f363d6bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba1d2f0 0x560bfba42520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f363d6bc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba21060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f363d6bc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560bfba39aa0 con 0x560bfba27bb0 2024-09-10T08:46:27.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f363d6bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560bfba393d0 con 0x560bfba1d2f0 2024-09-10T08:46:27.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f363d6bc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560bfba3a170 con 0x560bfba1c9d0 2024-09-10T08:46:27.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f3639880640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba21060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f363d6bc740 10 client.0 _ll_register_callbacks cb 0x7f3624009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:27.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f3639880640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba21060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33118/0 (socket says 172.21.15.172:33118) 2024-09-10T08:46:27.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f3639880640 1 -- 172.21.15.172:0/787973383 learned_addr learned my addr 172.21.15.172:0/787973383 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:27.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f363907f640 1 --2- 172.21.15.172:0/787973383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1c9d0 0x560bfba41fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f363887e640 1 --2- 172.21.15.172:0/787973383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba1d2f0 0x560bfba42520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f3639880640 1 -- 172.21.15.172:0/787973383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1c9d0 msgr2=0x560bfba41fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:27.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f3639880640 1 --2- 172.21.15.172:0/787973383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1c9d0 0x560bfba41fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f3639880640 1 -- 172.21.15.172:0/787973383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba1d2f0 msgr2=0x560bfba42520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:27.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f3639880640 1 --2- 172.21.15.172:0/787973383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba1d2f0 0x560bfba42520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f3639880640 1 -- 172.21.15.172:0/787973383 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bfba21750 con 0x560bfba27bb0 2024-09-10T08:46:27.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f363887e640 1 --2- 172.21.15.172:0/787973383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba1d2f0 0x560bfba42520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:27.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f3639880640 1 --2- 172.21.15.172:0/787973383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba21060 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3624003bd0 tx=0x7f3624003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce9f945d632a89c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:27.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.437+0000 7f3618ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:27.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.438+0000 7f3618ff9640 1 -- 172.21.15.172:0/787973383 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3624035070 con 0x560bfba27bb0 2024-09-10T08:46:27.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.438+0000 7f3618ff9640 1 -- 172.21.15.172:0/787973383 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f36240046c0 con 0x560bfba27bb0 2024-09-10T08:46:27.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.438+0000 7f363d6bc740 1 -- 172.21.15.172:0/787973383 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f362c0996b0 con 0x560bfba27bb0 2024-09-10T08:46:27.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.438+0000 7f3618ff9640 1 -- 172.21.15.172:0/787973383 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f362406a050 con 0x560bfba27bb0 2024-09-10T08:46:27.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.438+0000 7f363d6bc740 10 client.15267 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:27.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.438+0000 7f363d6bc740 1 -- 172.21.15.172:0/787973383 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560bfba41670 con 0x560bfba27bb0 2024-09-10T08:46:27.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.438+0000 7f3618ff9640 1 -- 172.21.15.172:0/787973383 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f362c0996b0 con 0x560bfba27bb0 2024-09-10T08:46:27.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.438+0000 7f3618ff9640 1 -- 172.21.15.172:0/787973383 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f3624003d00 con 0x560bfba27bb0 2024-09-10T08:46:27.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.438+0000 7f363d6bc740 10 client.15267 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:27.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.438+0000 7f363d6bc740 10 client.15267 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:46:27.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.438+0000 7f363d6bc740 20 client.15267 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:27.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.438+0000 7f363d6bc740 1 --2- 172.21.15.172:0/787973383 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560bfbaa73c0 0x560bfbaa60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:27.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.438+0000 7f363d6bc740 4 client.15267 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:46:27.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.438+0000 7f363d6bc740 1 -- 172.21.15.172:0/787973383 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "status"}) -- 0x560bfb98ab10 con 0x560bfbaa73c0 2024-09-10T08:46:27.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.439+0000 7f363907f640 1 --2- 172.21.15.172:0/787973383 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560bfbaa73c0 0x560bfbaa60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:27.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.439+0000 7f363907f640 1 --2- 172.21.15.172:0/787973383 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560bfbaa73c0 0x560bfbaa60d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=189c67bf264d56c1 server_cookie=faee2e37ea83d7b7 in_seq=0 out_seq=0 2024-09-10T08:46:27.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.439+0000 7f3618ff9640 10 client.15267 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:46:27.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.439+0000 7f3618ff9640 1 -- 172.21.15.172:0/787973383 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560bfb98ab10 con 0x560bfbaa73c0 2024-09-10T08:46:27.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.439+0000 7f3618ff9640 10 client.15267 handle_command_reply: tid=0 2024-09-10T08:46:27.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.439+0000 7f363d6bc740 1 client.15267 shutdown 2024-09-10T08:46:27.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.439+0000 7f363d6bc740 1 -- 172.21.15.172:0/787973383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 msgr2=0x560bfba21060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:27.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.439+0000 7f363d6bc740 1 --2- 172.21.15.172:0/787973383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba21060 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3624003bd0 tx=0x7f3624003920 comp rx=0 tx=0).stop 2024-09-10T08:46:27.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.439+0000 7f363d6bc740 1 -- 172.21.15.172:0/787973383 shutdown_connections 2024-09-10T08:46:27.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.439+0000 7f363d6bc740 1 --2- 172.21.15.172:0/787973383 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560bfbaa73c0 0x560bfbaa60d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.439+0000 7f363d6bc740 1 --2- 172.21.15.172:0/787973383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bfba27bb0 0x560bfba21060 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.439+0000 7f363d6bc740 1 --2- 172.21.15.172:0/787973383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfba1d2f0 0x560bfba42520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.439+0000 7f363d6bc740 1 --2- 172.21.15.172:0/787973383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560bfba1c9d0 0x560bfba41fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.439+0000 7f363d6bc740 1 -- 172.21.15.172:0/787973383 >> 172.21.15.172:0/787973383 conn(0x560bfba18f60 msgr2=0x560bfba247d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:27.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.439+0000 7f3618ff9640 0 client.15267 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:46:27.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.439+0000 7f363d6bc740 1 -- 172.21.15.172:0/787973383 shutdown_connections 2024-09-10T08:46:27.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.439+0000 7f363d6bc740 1 -- 172.21.15.172:0/787973383 wait complete. 2024-09-10T08:46:27.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.440+0000 7f363d6bc740 20 client.15267 trim_cache size 0 max 16384 2024-09-10T08:46:27.448 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:46:27.448 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:46:27.448 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 2, 2024-09-10T08:46:27.448 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 24457, 2024-09-10T08:46:27.449 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-09-10T08:46:27.449 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-09-10T08:46:27.449 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-09-10T08:46:27.449 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 95.183794195000004, 2024-09-10T08:46:27.449 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 28, 2024-09-10T08:46:27.449 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 92, 2024-09-10T08:46:27.449 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 88, 2024-09-10T08:46:27.449 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 154.796517155 2024-09-10T08:46:27.449 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:46:27.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.440+0000 7f363b2e3640 1 -- 172.21.15.172:0/1801978641 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f361c0a2b80 msgr2=0x7f361c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:27.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.440+0000 7f363b2e3640 1 --2- 172.21.15.172:0/1801978641 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f361c0a2b80 0x7f361c0a5040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f36240096c0 tx=0x7f3624031400 comp rx=0 tx=0).stop 2024-09-10T08:46:27.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.440+0000 7f363b2e3640 1 -- 172.21.15.172:0/1801978641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3634100b30 msgr2=0x7f36341a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:27.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.440+0000 7f363b2e3640 1 --2- 172.21.15.172:0/1801978641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3634100b30 0x7f36341a4680 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f362c00c890 tx=0x7f362c00cd50 comp rx=0 tx=0).stop 2024-09-10T08:46:27.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.440+0000 7f363b2e3640 1 -- 172.21.15.172:0/1801978641 shutdown_connections 2024-09-10T08:46:27.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.440+0000 7f363b2e3640 1 --2- 172.21.15.172:0/1801978641 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f361c0a2b80 0x7f361c0a5040 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.440+0000 7f363b2e3640 1 --2- 172.21.15.172:0/1801978641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3634105f40 0x7f363419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.440+0000 7f363b2e3640 1 --2- 172.21.15.172:0/1801978641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3634101450 0x7f36341a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.440+0000 7f363b2e3640 1 --2- 172.21.15.172:0/1801978641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3634100b30 0x7f36341a4680 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:27.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.440+0000 7f363b2e3640 1 -- 172.21.15.172:0/1801978641 >> 172.21.15.172:0/1801978641 conn(0x7f36340fc7a0 msgr2=0x7f363410f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:27.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.440+0000 7f363b2e3640 1 -- 172.21.15.172:0/1801978641 shutdown_connections 2024-09-10T08:46:27.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:27.440+0000 7f363b2e3640 1 -- 172.21.15.172:0/1801978641 wait complete. 2024-09-10T08:46:28.125 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T08:46:28.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:28 smithi172 ceph-mon[30854]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 546 B/s wr, 11 op/s 2024-09-10T08:46:28.376 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:28.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:28 smithi174 ceph-mon[34068]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 546 B/s wr, 11 op/s 2024-09-10T08:46:28.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:28 smithi191 ceph-mon[35314]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 546 B/s wr, 11 op/s 2024-09-10T08:46:28.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.923+0000 7ff61b69f640 1 -- 172.21.15.172:0/3668969253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6141082a0 msgr2=0x7ff61410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:28.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.923+0000 7ff61b69f640 1 --2- 172.21.15.172:0/3668969253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6141082a0 0x7ff61410ec40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009930 tx=0x7ff60c02a440 comp rx=0 tx=0).stop 2024-09-10T08:46:28.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.923+0000 7ff61b69f640 1 -- 172.21.15.172:0/3668969253 shutdown_connections 2024-09-10T08:46:28.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.923+0000 7ff61b69f640 1 --2- 172.21.15.172:0/3668969253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6141082a0 0x7ff61410ec40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.923+0000 7ff61b69f640 1 --2- 172.21.15.172:0/3668969253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6141036e0 0x7ff614107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.923+0000 7ff61b69f640 1 --2- 172.21.15.172:0/3668969253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff614102d30 0x7ff614103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.923+0000 7ff61b69f640 1 -- 172.21.15.172:0/3668969253 >> 172.21.15.172:0/3668969253 conn(0x7ff6140fc7d0 msgr2=0x7ff6140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:28.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.923+0000 7ff61b69f640 1 -- 172.21.15.172:0/3668969253 shutdown_connections 2024-09-10T08:46:28.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.923+0000 7ff61b69f640 1 -- 172.21.15.172:0/3668969253 wait complete. 2024-09-10T08:46:28.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.924+0000 7ff61b69f640 1 Processor -- start 2024-09-10T08:46:28.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.924+0000 7ff61b69f640 1 -- start start 2024-09-10T08:46:28.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.924+0000 7ff61b69f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff614102d30 0x7ff6141984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.924+0000 7ff61b69f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6141036e0 0x7ff614198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.924+0000 7ff61b69f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6141082a0 0x7ff61419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.924+0000 7ff61b69f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff614111f90 con 0x7ff6141036e0 2024-09-10T08:46:28.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.924+0000 7ff61b69f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff614111e10 con 0x7ff614102d30 2024-09-10T08:46:28.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.924+0000 7ff61b69f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff614112110 con 0x7ff6141082a0 2024-09-10T08:46:28.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.924+0000 7ff618c3a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6141036e0 0x7ff614198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:28.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.924+0000 7ff618c3a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6141036e0 0x7ff614198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33148/0 (socket says 172.21.15.172:33148) 2024-09-10T08:46:28.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.924+0000 7ff618c3a640 1 -- 172.21.15.172:0/3321565280 learned_addr learned my addr 172.21.15.172:0/3321565280 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:28.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.924+0000 7ff61943b640 1 --2- 172.21.15.172:0/3321565280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff614102d30 0x7ff6141984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:28.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.924+0000 7ff619c3c640 1 --2- 172.21.15.172:0/3321565280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6141082a0 0x7ff61419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:28.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.924+0000 7ff618c3a640 1 -- 172.21.15.172:0/3321565280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6141082a0 msgr2=0x7ff61419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:28.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.924+0000 7ff618c3a640 1 --2- 172.21.15.172:0/3321565280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6141082a0 0x7ff61419cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.924+0000 7ff618c3a640 1 -- 172.21.15.172:0/3321565280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff614102d30 msgr2=0x7ff6141984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:28.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.925+0000 7ff618c3a640 1 --2- 172.21.15.172:0/3321565280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff614102d30 0x7ff6141984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.925+0000 7ff618c3a640 1 -- 172.21.15.172:0/3321565280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61419d490 con 0x7ff6141036e0 2024-09-10T08:46:28.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.925+0000 7ff618c3a640 1 --2- 172.21.15.172:0/3321565280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6141036e0 0x7ff614198a20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff608006fd0 tx=0x7ff60800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:28.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.925+0000 7ff6127fc640 1 -- 172.21.15.172:0/3321565280 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608011830 con 0x7ff6141036e0 2024-09-10T08:46:28.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.925+0000 7ff6127fc640 1 -- 172.21.15.172:0/3321565280 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff608016070 con 0x7ff6141036e0 2024-09-10T08:46:28.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.925+0000 7ff61b69f640 1 -- 172.21.15.172:0/3321565280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61419d780 con 0x7ff6141036e0 2024-09-10T08:46:28.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.925+0000 7ff6127fc640 1 -- 172.21.15.172:0/3321565280 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608010df0 con 0x7ff6141036e0 2024-09-10T08:46:28.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.925+0000 7ff61b69f640 1 -- 172.21.15.172:0/3321565280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6141a5060 con 0x7ff6141036e0 2024-09-10T08:46:28.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.927+0000 7ff6127fc640 1 -- 172.21.15.172:0/3321565280 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff6080119d0 con 0x7ff6141036e0 2024-09-10T08:46:28.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff61da78740 1 Processor -- start 2024-09-10T08:46:28.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff61da78740 1 -- start start 2024-09-10T08:46:28.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff6127fc640 1 --2- 172.21.15.172:0/3321565280 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff5fc0a2b80 0x7ff5fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff61da78740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e198b0 0x55a607e19c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff61da78740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e1a1d0 0x55a607e28a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff61da78740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e28fc0 0x55a607e2b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff61da78740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a607d80090 con 0x55a607e1a1d0 2024-09-10T08:46:28.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff61da78740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a607d27a70 con 0x55a607e28fc0 2024-09-10T08:46:28.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff61da78740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a607d7c050 con 0x55a607e198b0 2024-09-10T08:46:28.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff6127fc640 1 -- 172.21.15.172:0/3321565280 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff6080cea30 con 0x7ff6141036e0 2024-09-10T08:46:28.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff619c3c640 1 --2- 172.21.15.172:0/3321565280 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff5fc0a2b80 0x7ff5fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:28.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff618c3a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e1a1d0 0x55a607e28a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:28.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff61943b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e198b0 0x55a607e19c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:28.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff618c3a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e1a1d0 0x55a607e28a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33164/0 (socket says 172.21.15.172:33164) 2024-09-10T08:46:28.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff618c3a640 1 -- 172.21.15.172:0/2700864124 learned_addr learned my addr 172.21.15.172:0/2700864124 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:28.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff61943b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e198b0 0x55a607e19c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:34476/0 (socket says 172.21.15.172:34476) 2024-09-10T08:46:28.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff619c3c640 1 --2- 172.21.15.172:0/2700864124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e28fc0 0x55a607e2b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:28.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff618c3a640 1 -- 172.21.15.172:0/2700864124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e198b0 msgr2=0x55a607e19c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:28.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff618c3a640 1 --2- 172.21.15.172:0/2700864124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e198b0 0x55a607e19c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff618c3a640 1 -- 172.21.15.172:0/2700864124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e28fc0 msgr2=0x55a607e2b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:28.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff618c3a640 1 --2- 172.21.15.172:0/2700864124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e28fc0 0x55a607e2b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.928+0000 7ff618c3a640 1 -- 172.21.15.172:0/2700864124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a607d34130 con 0x55a607e1a1d0 2024-09-10T08:46:28.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.929+0000 7ff619c3c640 1 --2- 172.21.15.172:0/3321565280 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff5fc0a2b80 0x7ff5fc0a5040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff60c02a960 tx=0x7ff60c032f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:28.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.929+0000 7ff618c3a640 1 --2- 172.21.15.172:0/2700864124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e1a1d0 0x55a607e28a80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff608097390 tx=0x7ff60809ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f373e4a4afbe64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:28.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.929+0000 7ff5fb7fe640 1 -- 172.21.15.172:0/2700864124 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60809af30 con 0x55a607e1a1d0 2024-09-10T08:46:28.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.929+0000 7ff5fb7fe640 1 -- 172.21.15.172:0/2700864124 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff60809f2e0 con 0x55a607e1a1d0 2024-09-10T08:46:28.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.929+0000 7ff5fb7fe640 1 -- 172.21.15.172:0/2700864124 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608099300 con 0x55a607e1a1d0 2024-09-10T08:46:28.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.930+0000 7ff61da78740 1 -- 172.21.15.172:0/2700864124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e1a1d0 msgr2=0x55a607e28a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:28.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.930+0000 7ff61da78740 1 --2- 172.21.15.172:0/2700864124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e1a1d0 0x55a607e28a80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff608097390 tx=0x7ff60809ab30 comp rx=0 tx=0).stop 2024-09-10T08:46:28.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.930+0000 7ff61da78740 1 -- 172.21.15.172:0/2700864124 shutdown_connections 2024-09-10T08:46:28.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.930+0000 7ff61da78740 1 --2- 172.21.15.172:0/2700864124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e28fc0 0x55a607e2b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.930+0000 7ff61da78740 1 --2- 172.21.15.172:0/2700864124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e1a1d0 0x55a607e28a80 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.930+0000 7ff61da78740 1 --2- 172.21.15.172:0/2700864124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e198b0 0x55a607e19c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.930+0000 7ff61da78740 1 -- 172.21.15.172:0/2700864124 >> 172.21.15.172:0/2700864124 conn(0x55a607e15f60 msgr2=0x55a607e16340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:28.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.930+0000 7ff61da78740 1 -- 172.21.15.172:0/2700864124 shutdown_connections 2024-09-10T08:46:28.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.930+0000 7ff61da78740 1 -- 172.21.15.172:0/2700864124 wait complete. 2024-09-10T08:46:28.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.930+0000 7ff61da78740 1 -- start start 2024-09-10T08:46:28.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.930+0000 7ff61da78740 1 Processor -- start 2024-09-10T08:46:28.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.930+0000 7ff61da78740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e198b0 0x55a607e28830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.930+0000 7ff61da78740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e1a1d0 0x55a607e25b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff61da78740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e28fc0 0x55a607e260c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff61da78740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a607e2bb90 con 0x55a607e28fc0 2024-09-10T08:46:28.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff61da78740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a607d3af20 con 0x55a607e198b0 2024-09-10T08:46:28.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff61da78740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a607e2bd10 con 0x55a607e1a1d0 2024-09-10T08:46:28.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff61da78740 10 client.0 _ll_register_callbacks cb 0x55a607d04030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:28.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff619c3c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e28fc0 0x55a607e260c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:28.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff619c3c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e28fc0 0x55a607e260c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33180/0 (socket says 172.21.15.172:33180) 2024-09-10T08:46:28.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff619c3c640 1 -- 172.21.15.172:0/3430830377 learned_addr learned my addr 172.21.15.172:0/3430830377 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:28.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff61943b640 1 --2- 172.21.15.172:0/3430830377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e198b0 0x55a607e28830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:28.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff618c3a640 1 --2- 172.21.15.172:0/3430830377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e1a1d0 0x55a607e25b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:28.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff619c3c640 1 -- 172.21.15.172:0/3430830377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e1a1d0 msgr2=0x55a607e25b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:28.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff619c3c640 1 --2- 172.21.15.172:0/3430830377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e1a1d0 0x55a607e25b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff619c3c640 1 -- 172.21.15.172:0/3430830377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e198b0 msgr2=0x55a607e28830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:28.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff619c3c640 1 --2- 172.21.15.172:0/3430830377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e198b0 0x55a607e28830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff619c3c640 1 -- 172.21.15.172:0/3430830377 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a607af12b0 con 0x55a607e28fc0 2024-09-10T08:46:28.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff618c3a640 1 --2- 172.21.15.172:0/3430830377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e1a1d0 0x55a607e25b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:28.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff619c3c640 1 --2- 172.21.15.172:0/3430830377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e28fc0 0x55a607e260c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff60c033000 tx=0x7ff60c02a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4249dff0f6f45942 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:28.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff5f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:28.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff5f8ff9640 1 -- 172.21.15.172:0/3430830377 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c04e060 con 0x55a607e28fc0 2024-09-10T08:46:28.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff5f8ff9640 1 -- 172.21.15.172:0/3430830377 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff60c005510 con 0x55a607e28fc0 2024-09-10T08:46:28.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.931+0000 7ff61da78740 1 -- 172.21.15.172:0/3430830377 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff60809f2e0 con 0x55a607e28fc0 2024-09-10T08:46:28.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.932+0000 7ff5f8ff9640 1 -- 172.21.15.172:0/3430830377 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff60c055050 con 0x55a607e28fc0 2024-09-10T08:46:28.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.932+0000 7ff61da78740 10 client.15279 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:28.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.932+0000 7ff61da78740 1 -- 172.21.15.172:0/3430830377 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a607e374c0 con 0x55a607e28fc0 2024-09-10T08:46:28.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.932+0000 7ff5f8ff9640 1 -- 172.21.15.172:0/3430830377 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff60809f2e0 con 0x55a607e28fc0 2024-09-10T08:46:28.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.932+0000 7ff5f8ff9640 1 -- 172.21.15.172:0/3430830377 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ff60c003920 con 0x55a607e28fc0 2024-09-10T08:46:28.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.932+0000 7ff61da78740 10 client.15279 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:28.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.932+0000 7ff61da78740 10 client.15279 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:46:28.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.932+0000 7ff61da78740 20 client.15279 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:28.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.932+0000 7ff61da78740 1 --2- 172.21.15.172:0/3430830377 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a607e39b00 0x55a607e3af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.932+0000 7ff61da78740 4 client.15279 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:28.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.932+0000 7ff61da78740 1 -- 172.21.15.172:0/3430830377 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a607d8d600 con 0x55a607e39b00 2024-09-10T08:46:28.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.932+0000 7ff61943b640 1 --2- 172.21.15.172:0/3430830377 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a607e39b00 0x55a607e3af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:28.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.933+0000 7ff61943b640 1 --2- 172.21.15.172:0/3430830377 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a607e39b00 0x55a607e3af70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f05331208a339d8b server_cookie=aae4a6ec6785c0c in_seq=0 out_seq=0 2024-09-10T08:46:28.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.933+0000 7ff5f8ff9640 10 client.15279 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:46:28.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.934+0000 7ff5f8ff9640 1 -- 172.21.15.172:0/3430830377 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a607d8d600 con 0x55a607e39b00 2024-09-10T08:46:28.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.934+0000 7ff5f8ff9640 10 client.15279 handle_command_reply: tid=0 2024-09-10T08:46:28.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.934+0000 7ff61da78740 1 client.15279 shutdown 2024-09-10T08:46:28.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.934+0000 7ff61da78740 1 -- 172.21.15.172:0/3430830377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e28fc0 msgr2=0x55a607e260c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:28.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.934+0000 7ff61da78740 1 --2- 172.21.15.172:0/3430830377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e28fc0 0x55a607e260c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff60c033000 tx=0x7ff60c02a440 comp rx=0 tx=0).stop 2024-09-10T08:46:28.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.934+0000 7ff61da78740 1 -- 172.21.15.172:0/3430830377 shutdown_connections 2024-09-10T08:46:28.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.934+0000 7ff61da78740 1 --2- 172.21.15.172:0/3430830377 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a607e39b00 0x55a607e3af70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.934+0000 7ff61da78740 1 --2- 172.21.15.172:0/3430830377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e28fc0 0x55a607e260c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.934+0000 7ff61da78740 1 --2- 172.21.15.172:0/3430830377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e1a1d0 0x55a607e25b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.934+0000 7ff61da78740 1 --2- 172.21.15.172:0/3430830377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e198b0 0x55a607e28830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.934+0000 7ff61da78740 1 -- 172.21.15.172:0/3430830377 >> 172.21.15.172:0/3430830377 conn(0x55a607e15f60 msgr2=0x55a607e1bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:28.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.934+0000 7ff5f8ff9640 0 client.15279 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:46:28.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.934+0000 7ff61da78740 1 -- 172.21.15.172:0/3430830377 shutdown_connections 2024-09-10T08:46:28.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.934+0000 7ff61da78740 1 -- 172.21.15.172:0/3430830377 wait complete. 2024-09-10T08:46:28.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.934+0000 7ff61da78740 20 client.15279 trim_cache size 0 max 16384 2024-09-10T08:46:28.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff61da78740 1 Processor -- start 2024-09-10T08:46:28.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff61da78740 1 -- start start 2024-09-10T08:46:28.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff61da78740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e198b0 0x55a607e39430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff61da78740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e1a1d0 0x55a607e3a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff61da78740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e28fc0 0x55a607e25b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff61da78740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a607e392b0 con 0x55a607e1a1d0 2024-09-10T08:46:28.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff61da78740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a607e39980 con 0x55a607e28fc0 2024-09-10T08:46:28.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff61da78740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a607e3a050 con 0x55a607e198b0 2024-09-10T08:46:28.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff618c3a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e1a1d0 0x55a607e3a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:28.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff618c3a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e1a1d0 0x55a607e3a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33196/0 (socket says 172.21.15.172:33196) 2024-09-10T08:46:28.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff618c3a640 1 -- 172.21.15.172:0/210311180 learned_addr learned my addr 172.21.15.172:0/210311180 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:28.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff618c3a640 1 -- 172.21.15.172:0/210311180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e198b0 msgr2=0x55a607e39430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:28.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff618c3a640 1 --2- 172.21.15.172:0/210311180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e198b0 0x55a607e39430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff619c3c640 1 --2- 172.21.15.172:0/210311180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e28fc0 0x55a607e25b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:28.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff618c3a640 1 -- 172.21.15.172:0/210311180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e28fc0 msgr2=0x55a607e25b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:28.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff618c3a640 1 --2- 172.21.15.172:0/210311180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e28fc0 0x55a607e25b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff618c3a640 1 -- 172.21.15.172:0/210311180 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a607e1a000 con 0x55a607e1a1d0 2024-09-10T08:46:28.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff619c3c640 1 --2- 172.21.15.172:0/210311180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e28fc0 0x55a607e25b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:28.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.952+0000 7ff618c3a640 1 --2- 172.21.15.172:0/210311180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e1a1d0 0x55a607e3a8a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff60809aec0 tx=0x7ff608010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362056d331c80e75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:28.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.953+0000 7ff5faffd640 1 -- 172.21.15.172:0/210311180 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6080a3ba0 con 0x55a607e1a1d0 2024-09-10T08:46:28.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.953+0000 7ff5faffd640 1 -- 172.21.15.172:0/210311180 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff608098fd0 con 0x55a607e1a1d0 2024-09-10T08:46:28.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.953+0000 7ff5faffd640 1 -- 172.21.15.172:0/210311180 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608097740 con 0x55a607e1a1d0 2024-09-10T08:46:28.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.953+0000 7ff61da78740 1 -- 172.21.15.172:0/210311180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e1a1d0 msgr2=0x55a607e3a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:28.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.953+0000 7ff61da78740 1 --2- 172.21.15.172:0/210311180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e1a1d0 0x55a607e3a8a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff60809aec0 tx=0x7ff608010710 comp rx=0 tx=0).stop 2024-09-10T08:46:28.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.953+0000 7ff61da78740 1 -- 172.21.15.172:0/210311180 shutdown_connections 2024-09-10T08:46:28.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.953+0000 7ff61da78740 1 --2- 172.21.15.172:0/210311180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e28fc0 0x55a607e25b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.953+0000 7ff61da78740 1 --2- 172.21.15.172:0/210311180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e1a1d0 0x55a607e3a8a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.953+0000 7ff61da78740 1 --2- 172.21.15.172:0/210311180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e198b0 0x55a607e39430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.953+0000 7ff61da78740 1 -- 172.21.15.172:0/210311180 >> 172.21.15.172:0/210311180 conn(0x55a607e15f60 msgr2=0x55a607e10ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:28.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.953+0000 7ff61da78740 1 -- 172.21.15.172:0/210311180 shutdown_connections 2024-09-10T08:46:28.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.953+0000 7ff61da78740 1 -- 172.21.15.172:0/210311180 wait complete. 2024-09-10T08:46:28.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.954+0000 7ff61da78740 1 -- start start 2024-09-10T08:46:28.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.954+0000 7ff61da78740 1 Processor -- start 2024-09-10T08:46:28.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.954+0000 7ff61da78740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e198b0 0x55a607e43840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.954+0000 7ff61da78740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e1a1d0 0x55a607e3c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.954+0000 7ff61da78740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e28fc0 0x55a607e3c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.954+0000 7ff61da78740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a607e3adf0 con 0x55a607e198b0 2024-09-10T08:46:28.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.954+0000 7ff61da78740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a607e3a720 con 0x55a607e1a1d0 2024-09-10T08:46:28.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.954+0000 7ff61da78740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a607e3b4c0 con 0x55a607e28fc0 2024-09-10T08:46:28.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.954+0000 7ff61da78740 10 client.0 _ll_register_callbacks cb 0x7ff60c0316c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:28.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.954+0000 7ff61943b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e198b0 0x55a607e43840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:28.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.954+0000 7ff61943b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e198b0 0x55a607e43840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33200/0 (socket says 172.21.15.172:33200) 2024-09-10T08:46:28.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.954+0000 7ff61943b640 1 -- 172.21.15.172:0/597579319 learned_addr learned my addr 172.21.15.172:0/597579319 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:28.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.954+0000 7ff61943b640 1 -- 172.21.15.172:0/597579319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e28fc0 msgr2=0x55a607e3c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:28.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.954+0000 7ff61943b640 1 --2- 172.21.15.172:0/597579319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e28fc0 0x55a607e3c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.954+0000 7ff61943b640 1 -- 172.21.15.172:0/597579319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e1a1d0 msgr2=0x55a607e3c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:28.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.955+0000 7ff618c3a640 1 --2- 172.21.15.172:0/597579319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e1a1d0 0x55a607e3c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:28.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.955+0000 7ff61943b640 1 --2- 172.21.15.172:0/597579319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e1a1d0 0x55a607e3c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.955+0000 7ff61943b640 1 -- 172.21.15.172:0/597579319 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a607e3cd20 con 0x55a607e198b0 2024-09-10T08:46:28.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.955+0000 7ff618c3a640 1 --2- 172.21.15.172:0/597579319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e1a1d0 0x55a607e3c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:28.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.955+0000 7ff61943b640 1 --2- 172.21.15.172:0/597579319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e198b0 0x55a607e43840 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff604010870 tx=0x7ff604010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5bd31b2f927731 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:28.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.955+0000 7ff5f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:28.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.955+0000 7ff5f8ff9640 1 -- 172.21.15.172:0/597579319 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60402b060 con 0x55a607e198b0 2024-09-10T08:46:28.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.955+0000 7ff5f8ff9640 1 -- 172.21.15.172:0/597579319 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff60400fd90 con 0x55a607e198b0 2024-09-10T08:46:28.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.955+0000 7ff61da78740 1 -- 172.21.15.172:0/597579319 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff608098fd0 con 0x55a607e198b0 2024-09-10T08:46:28.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.955+0000 7ff5f8ff9640 1 -- 172.21.15.172:0/597579319 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff604032050 con 0x55a607e198b0 2024-09-10T08:46:28.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.955+0000 7ff61da78740 10 client.15285 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:28.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.955+0000 7ff61da78740 1 -- 172.21.15.172:0/597579319 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a607ea3040 con 0x55a607e198b0 2024-09-10T08:46:28.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.955+0000 7ff5f8ff9640 1 -- 172.21.15.172:0/597579319 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff608098fd0 con 0x55a607e198b0 2024-09-10T08:46:28.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.955+0000 7ff5f8ff9640 1 -- 172.21.15.172:0/597579319 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ff60401d420 con 0x55a607e198b0 2024-09-10T08:46:28.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.955+0000 7ff61da78740 10 client.15285 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:28.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.956+0000 7ff61da78740 10 client.15285 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:46:28.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.956+0000 7ff61da78740 20 client.15285 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:28.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.956+0000 7ff61da78740 1 --2- 172.21.15.172:0/597579319 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a607ea7cd0 0x55a607ea69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:28.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.956+0000 7ff61da78740 4 client.15285 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:46:28.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.956+0000 7ff61da78740 1 -- 172.21.15.172:0/597579319 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "status"}) -- 0x55a607d87b10 con 0x55a607ea7cd0 2024-09-10T08:46:28.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.956+0000 7ff618c3a640 1 --2- 172.21.15.172:0/597579319 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a607ea7cd0 0x55a607ea69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:28.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.956+0000 7ff5f8ff9640 10 client.15285 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:46:28.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.956+0000 7ff618c3a640 1 --2- 172.21.15.172:0/597579319 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a607ea7cd0 0x55a607ea69e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c68dcc8cf4aca3ea server_cookie=ff0401d7c41de828 in_seq=0 out_seq=0 2024-09-10T08:46:28.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.956+0000 7ff5f8ff9640 1 -- 172.21.15.172:0/597579319 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x55a607d87b10 con 0x55a607ea7cd0 2024-09-10T08:46:28.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.956+0000 7ff5f8ff9640 10 client.15285 handle_command_reply: tid=0 2024-09-10T08:46:28.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.956+0000 7ff61da78740 1 client.15285 shutdown 2024-09-10T08:46:28.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.956+0000 7ff61da78740 1 -- 172.21.15.172:0/597579319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e198b0 msgr2=0x55a607e43840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:28.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.956+0000 7ff61da78740 1 --2- 172.21.15.172:0/597579319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e198b0 0x55a607e43840 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff604010870 tx=0x7ff604010c70 comp rx=0 tx=0).stop 2024-09-10T08:46:28.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.956+0000 7ff61da78740 1 -- 172.21.15.172:0/597579319 shutdown_connections 2024-09-10T08:46:28.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61da78740 1 --2- 172.21.15.172:0/597579319 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a607ea7cd0 0x55a607ea69e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61da78740 1 --2- 172.21.15.172:0/597579319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a607e28fc0 0x55a607e3c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61da78740 1 --2- 172.21.15.172:0/597579319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a607e1a1d0 0x55a607e3c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61da78740 1 --2- 172.21.15.172:0/597579319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a607e198b0 0x55a607e43840 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61da78740 1 -- 172.21.15.172:0/597579319 >> 172.21.15.172:0/597579319 conn(0x55a607e15f60 msgr2=0x55a607e39250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:28.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff5f8ff9640 0 client.15285 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:46:28.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61da78740 1 -- 172.21.15.172:0/597579319 shutdown_connections 2024-09-10T08:46:28.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61da78740 1 -- 172.21.15.172:0/597579319 wait complete. 2024-09-10T08:46:28.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61da78740 20 client.15285 trim_cache size 0 max 16384 2024-09-10T08:46:28.965 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:46:28.965 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:46:28.965 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 1, 2024-09-10T08:46:28.965 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 24458, 2024-09-10T08:46:28.965 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-09-10T08:46:28.965 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-09-10T08:46:28.965 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-09-10T08:46:28.965 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 97.70847277, 2024-09-10T08:46:28.965 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 28, 2024-09-10T08:46:28.965 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 92, 2024-09-10T08:46:28.965 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 88, 2024-09-10T08:46:28.966 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 158.976353786 2024-09-10T08:46:28.966 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:46:28.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61b69f640 1 -- 172.21.15.172:0/3321565280 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff5fc0a2b80 msgr2=0x7ff5fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:28.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61b69f640 1 --2- 172.21.15.172:0/3321565280 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff5fc0a2b80 0x7ff5fc0a5040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff60c02a960 tx=0x7ff60c032f30 comp rx=0 tx=0).stop 2024-09-10T08:46:28.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61b69f640 1 -- 172.21.15.172:0/3321565280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6141036e0 msgr2=0x7ff614198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:28.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61b69f640 1 --2- 172.21.15.172:0/3321565280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6141036e0 0x7ff614198a20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff608006fd0 tx=0x7ff60800b8e0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61b69f640 1 -- 172.21.15.172:0/3321565280 shutdown_connections 2024-09-10T08:46:28.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61b69f640 1 --2- 172.21.15.172:0/3321565280 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff5fc0a2b80 0x7ff5fc0a5040 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61b69f640 1 --2- 172.21.15.172:0/3321565280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6141082a0 0x7ff61419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61b69f640 1 --2- 172.21.15.172:0/3321565280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6141036e0 0x7ff614198a20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61b69f640 1 --2- 172.21.15.172:0/3321565280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff614102d30 0x7ff6141984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:28.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61b69f640 1 -- 172.21.15.172:0/3321565280 >> 172.21.15.172:0/3321565280 conn(0x7ff6140fc7d0 msgr2=0x7ff614105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:28.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.957+0000 7ff61b69f640 1 -- 172.21.15.172:0/3321565280 shutdown_connections 2024-09-10T08:46:28.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:28.958+0000 7ff61b69f640 1 -- 172.21.15.172:0/3321565280 wait complete. 2024-09-10T08:46:29.627 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T08:46:29.874 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:30.311 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:30 smithi172 ceph-mon[30854]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 494 B/s wr, 10 op/s 2024-09-10T08:46:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.390+0000 7f5b3a6d5640 1 -- 172.21.15.172:0/3215177205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b34101420 msgr2=0x7f5b3410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.390+0000 7f5b3a6d5640 1 --2- 172.21.15.172:0/3215177205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b34101420 0x7f5b3410d950 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009cd0 tx=0x7f5b2c02a8e0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.390+0000 7f5b3a6d5640 1 -- 172.21.15.172:0/3215177205 shutdown_connections 2024-09-10T08:46:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.390+0000 7f5b3a6d5640 1 --2- 172.21.15.172:0/3215177205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b34101420 0x7f5b3410d950 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.390+0000 7f5b3a6d5640 1 --2- 172.21.15.172:0/3215177205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34100a80 0x7f5b34100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.390+0000 7f5b3a6d5640 1 --2- 172.21.15.172:0/3215177205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b34106a80 0x7f5b34106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.390+0000 7f5b3a6d5640 1 -- 172.21.15.172:0/3215177205 >> 172.21.15.172:0/3215177205 conn(0x7f5b340fc7a0 msgr2=0x7f5b340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.390+0000 7f5b3a6d5640 1 -- 172.21.15.172:0/3215177205 shutdown_connections 2024-09-10T08:46:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.390+0000 7f5b3a6d5640 1 -- 172.21.15.172:0/3215177205 wait complete. 2024-09-10T08:46:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.390+0000 7f5b3a6d5640 1 Processor -- start 2024-09-10T08:46:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.390+0000 7f5b3a6d5640 1 -- start start 2024-09-10T08:46:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.391+0000 7f5b3a6d5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b34100a80 0x7f5b34198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.391+0000 7f5b3a6d5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34101420 0x7f5b341987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.391+0000 7f5b3a6d5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b34106a80 0x7f5b3419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.391+0000 7f5b3a6d5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5b34110d30 con 0x7f5b34100a80 2024-09-10T08:46:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.391+0000 7f5b3a6d5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b34110bb0 con 0x7f5b34101420 2024-09-10T08:46:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.391+0000 7f5b3a6d5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5b34110eb0 con 0x7f5b34106a80 2024-09-10T08:46:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.391+0000 7f5b33fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b34100a80 0x7f5b34198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.391+0000 7f5b33fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b34100a80 0x7f5b34198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33210/0 (socket says 172.21.15.172:33210) 2024-09-10T08:46:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.391+0000 7f5b33fff640 1 -- 172.21.15.172:0/1736168043 learned_addr learned my addr 172.21.15.172:0/1736168043 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.391+0000 7f5b337fe640 1 --2- 172.21.15.172:0/1736168043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34101420 0x7f5b341987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.391+0000 7f5b33fff640 1 -- 172.21.15.172:0/1736168043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b34106a80 msgr2=0x7f5b3419cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.391+0000 7f5b33fff640 1 --2- 172.21.15.172:0/1736168043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b34106a80 0x7f5b3419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.391+0000 7f5b33fff640 1 -- 172.21.15.172:0/1736168043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34101420 msgr2=0x7f5b341987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.391+0000 7f5b33fff640 1 --2- 172.21.15.172:0/1736168043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34101420 0x7f5b341987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.391+0000 7f5b33fff640 1 -- 172.21.15.172:0/1736168043 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3419d210 con 0x7f5b34100a80 2024-09-10T08:46:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.391+0000 7f5b337fe640 1 --2- 172.21.15.172:0/1736168043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34101420 0x7f5b341987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.391+0000 7f5b33fff640 1 --2- 172.21.15.172:0/1736168043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b34100a80 0x7f5b34198260 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400c920 tx=0x7f5b2400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.392+0000 7f5b317fa640 1 -- 172.21.15.172:0/1736168043 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b24007c70 con 0x7f5b34100a80 2024-09-10T08:46:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.392+0000 7f5b317fa640 1 -- 172.21.15.172:0/1736168043 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b24007e10 con 0x7f5b34100a80 2024-09-10T08:46:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.392+0000 7f5b317fa640 1 -- 172.21.15.172:0/1736168043 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b24018820 con 0x7f5b34100a80 2024-09-10T08:46:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.392+0000 7f5b3a6d5640 1 -- 172.21.15.172:0/1736168043 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3419d500 con 0x7f5b34100a80 2024-09-10T08:46:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.392+0000 7f5b3a6d5640 1 -- 172.21.15.172:0/1736168043 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b341a4de0 con 0x7f5b34100a80 2024-09-10T08:46:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.394+0000 7f5b317fa640 1 -- 172.21.15.172:0/1736168043 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5b240189c0 con 0x7f5b34100a80 2024-09-10T08:46:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.394+0000 7f5b3caae740 1 Processor -- start 2024-09-10T08:46:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.394+0000 7f5b3caae740 1 -- start start 2024-09-10T08:46:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.394+0000 7f5b3caae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc41309d0 0x557bc4130db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.394+0000 7f5b3caae740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc41312f0 0x557bc413b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.394+0000 7f5b3caae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc413bbb0 0x557bc413dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.394+0000 7f5b3caae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557bc4097090 con 0x557bc413bbb0 2024-09-10T08:46:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.394+0000 7f5b3caae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557bc403ea70 con 0x557bc41309d0 2024-09-10T08:46:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.394+0000 7f5b3caae740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557bc4093050 con 0x557bc41312f0 2024-09-10T08:46:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.394+0000 7f5b317fa640 1 --2- 172.21.15.172:0/1736168043 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b200a2b80 0x7f5b200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b337fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc413bbb0 0x557bc413dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b38c72640 1 --2- 172.21.15.172:0/1736168043 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b200a2b80 0x7f5b200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b337fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc413bbb0 0x557bc413dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33222/0 (socket says 172.21.15.172:33222) 2024-09-10T08:46:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b337fe640 1 -- 172.21.15.172:0/32811429 learned_addr learned my addr 172.21.15.172:0/32811429 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b33fff640 1 --2- 172.21.15.172:0/32811429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc41312f0 0x557bc413b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b38c72640 1 --2- 172.21.15.172:0/32811429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc41309d0 0x557bc4130db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b317fa640 1 -- 172.21.15.172:0/1736168043 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5b240cf560 con 0x7f5b34100a80 2024-09-10T08:46:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b337fe640 1 -- 172.21.15.172:0/32811429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc41312f0 msgr2=0x557bc413b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b337fe640 1 --2- 172.21.15.172:0/32811429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc41312f0 0x557bc413b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b337fe640 1 -- 172.21.15.172:0/32811429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc41309d0 msgr2=0x557bc4130db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b337fe640 1 --2- 172.21.15.172:0/32811429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc41309d0 0x557bc4130db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b337fe640 1 -- 172.21.15.172:0/32811429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bc404b130 con 0x557bc413bbb0 2024-09-10T08:46:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b33fff640 1 --2- 172.21.15.172:0/32811429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc41312f0 0x557bc413b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b38c72640 1 --2- 172.21.15.172:0/1736168043 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b200a2b80 0x7f5b200a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0317f0 tx=0x7f5b2c0093a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b337fe640 1 --2- 172.21.15.172:0/32811429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc413bbb0 0x557bc413dfa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00cc50 tx=0x7f5b1c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b5c743ff851848 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b2a7fc640 1 -- 172.21.15.172:0/32811429 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c014070 con 0x557bc413bbb0 2024-09-10T08:46:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b2a7fc640 1 -- 172.21.15.172:0/32811429 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b1c019b10 con 0x557bc413bbb0 2024-09-10T08:46:30.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.395+0000 7f5b2a7fc640 1 -- 172.21.15.172:0/32811429 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c010ae0 con 0x557bc413bbb0 2024-09-10T08:46:30.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.396+0000 7f5b3caae740 1 -- 172.21.15.172:0/32811429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc413bbb0 msgr2=0x557bc413dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.396+0000 7f5b3caae740 1 --2- 172.21.15.172:0/32811429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc413bbb0 0x557bc413dfa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00cc50 tx=0x7f5b1c00e720 comp rx=0 tx=0).stop 2024-09-10T08:46:30.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.396+0000 7f5b3caae740 1 -- 172.21.15.172:0/32811429 shutdown_connections 2024-09-10T08:46:30.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.396+0000 7f5b3caae740 1 --2- 172.21.15.172:0/32811429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc413bbb0 0x557bc413dfa0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.396+0000 7f5b3caae740 1 --2- 172.21.15.172:0/32811429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc41312f0 0x557bc413b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.396+0000 7f5b3caae740 1 --2- 172.21.15.172:0/32811429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc41309d0 0x557bc4130db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.396+0000 7f5b3caae740 1 -- 172.21.15.172:0/32811429 >> 172.21.15.172:0/32811429 conn(0x557bc412cf60 msgr2=0x557bc412d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:30.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.396+0000 7f5b3caae740 1 -- 172.21.15.172:0/32811429 shutdown_connections 2024-09-10T08:46:30.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.396+0000 7f5b3caae740 1 -- 172.21.15.172:0/32811429 wait complete. 2024-09-10T08:46:30.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.396+0000 7f5b3caae740 1 -- start start 2024-09-10T08:46:30.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b3caae740 1 Processor -- start 2024-09-10T08:46:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b3caae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc41309d0 0x557bc413b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b3caae740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc41312f0 0x557bc4138750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b3caae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc413bbb0 0x557bc4138c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b3caae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557bc413e780 con 0x557bc41309d0 2024-09-10T08:46:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b3caae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557bc4051f20 con 0x557bc413bbb0 2024-09-10T08:46:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b3caae740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557bc413e900 con 0x557bc41312f0 2024-09-10T08:46:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b3caae740 10 client.0 _ll_register_callbacks cb 0x557bc401b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b33fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc41309d0 0x557bc413b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b33fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc41309d0 0x557bc413b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33226/0 (socket says 172.21.15.172:33226) 2024-09-10T08:46:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b33fff640 1 -- 172.21.15.172:0/1247208165 learned_addr learned my addr 172.21.15.172:0/1247208165 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b337fe640 1 --2- 172.21.15.172:0/1247208165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc41312f0 0x557bc4138750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b38c72640 1 --2- 172.21.15.172:0/1247208165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc413bbb0 0x557bc4138c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b33fff640 1 -- 172.21.15.172:0/1247208165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc41312f0 msgr2=0x557bc4138750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b33fff640 1 --2- 172.21.15.172:0/1247208165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc41312f0 0x557bc4138750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b33fff640 1 -- 172.21.15.172:0/1247208165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc413bbb0 msgr2=0x557bc4138c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b33fff640 1 --2- 172.21.15.172:0/1247208165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc413bbb0 0x557bc4138c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b33fff640 1 -- 172.21.15.172:0/1247208165 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bc3e082b0 con 0x557bc41309d0 2024-09-10T08:46:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b337fe640 1 --2- 172.21.15.172:0/1247208165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc41312f0 0x557bc4138750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b33fff640 1 --2- 172.21.15.172:0/1247208165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc41309d0 0x557bc413b400 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5b24007590 tx=0x7f5b24015d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecbb376c3b93da5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.397+0000 7f5b137fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.398+0000 7f5b137fe640 1 -- 172.21.15.172:0/1247208165 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b240b3070 con 0x557bc41309d0 2024-09-10T08:46:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.398+0000 7f5b137fe640 1 -- 172.21.15.172:0/1247208165 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5b240cf2a0 con 0x557bc41309d0 2024-09-10T08:46:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.398+0000 7f5b3caae740 1 -- 172.21.15.172:0/1247208165 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b1c019b10 con 0x557bc41309d0 2024-09-10T08:46:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.398+0000 7f5b137fe640 1 -- 172.21.15.172:0/1247208165 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b24065050 con 0x557bc41309d0 2024-09-10T08:46:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.398+0000 7f5b3caae740 10 client.15297 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.398+0000 7f5b3caae740 1 -- 172.21.15.172:0/1247208165 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557bc414a0a0 con 0x557bc41309d0 2024-09-10T08:46:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.398+0000 7f5b137fe640 1 -- 172.21.15.172:0/1247208165 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b1c019b10 con 0x557bc41309d0 2024-09-10T08:46:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.398+0000 7f5b137fe640 1 -- 172.21.15.172:0/1247208165 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f5b24066250 con 0x557bc41309d0 2024-09-10T08:46:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.398+0000 7f5b3caae740 10 client.15297 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.398+0000 7f5b3caae740 10 client.15297 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:46:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.398+0000 7f5b3caae740 20 client.15297 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.398+0000 7f5b3caae740 1 --2- 172.21.15.172:0/1247208165 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557bc414c720 0x557bc414db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.398+0000 7f5b3caae740 4 client.15297 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.398+0000 7f5b3caae740 1 -- 172.21.15.172:0/1247208165 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bc40a4600 con 0x557bc414c720 2024-09-10T08:46:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.398+0000 7f5b337fe640 1 --2- 172.21.15.172:0/1247208165 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557bc414c720 0x557bc414db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.399+0000 7f5b137fe640 10 client.15297 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.399+0000 7f5b337fe640 1 --2- 172.21.15.172:0/1247208165 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557bc414c720 0x557bc414db90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcbbf0213a375cf8 server_cookie=48012ad6694375f6 in_seq=0 out_seq=0 2024-09-10T08:46:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.399+0000 7f5b137fe640 1 -- 172.21.15.172:0/1247208165 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557bc40a4600 con 0x557bc414c720 2024-09-10T08:46:30.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.399+0000 7f5b137fe640 10 client.15297 handle_command_reply: tid=0 2024-09-10T08:46:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.400+0000 7f5b3caae740 1 client.15297 shutdown 2024-09-10T08:46:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.400+0000 7f5b3caae740 1 -- 172.21.15.172:0/1247208165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc41309d0 msgr2=0x557bc413b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.400+0000 7f5b3caae740 1 --2- 172.21.15.172:0/1247208165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc41309d0 0x557bc413b400 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5b24007590 tx=0x7f5b24015d00 comp rx=0 tx=0).stop 2024-09-10T08:46:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.400+0000 7f5b3caae740 1 -- 172.21.15.172:0/1247208165 shutdown_connections 2024-09-10T08:46:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.400+0000 7f5b3caae740 1 --2- 172.21.15.172:0/1247208165 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557bc414c720 0x557bc414db90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.400+0000 7f5b3caae740 1 --2- 172.21.15.172:0/1247208165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc413bbb0 0x557bc4138c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.400+0000 7f5b3caae740 1 --2- 172.21.15.172:0/1247208165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc41312f0 0x557bc4138750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.400+0000 7f5b137fe640 0 client.15297 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.400+0000 7f5b3caae740 1 --2- 172.21.15.172:0/1247208165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc41309d0 0x557bc413b400 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.400+0000 7f5b3caae740 1 -- 172.21.15.172:0/1247208165 >> 172.21.15.172:0/1247208165 conn(0x557bc412cf60 msgr2=0x557bc4132c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.400+0000 7f5b3caae740 1 -- 172.21.15.172:0/1247208165 shutdown_connections 2024-09-10T08:46:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.400+0000 7f5b3caae740 1 -- 172.21.15.172:0/1247208165 wait complete. 2024-09-10T08:46:30.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.400+0000 7f5b3caae740 20 client.15297 trim_cache size 0 max 16384 2024-09-10T08:46:30.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.417+0000 7f5b3caae740 1 Processor -- start 2024-09-10T08:46:30.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.417+0000 7f5b3caae740 1 -- start start 2024-09-10T08:46:30.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.418+0000 7f5b3caae740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc41309d0 0x557bc414db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.418+0000 7f5b3caae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc41312f0 0x557bc4138010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.418+0000 7f5b3caae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc413bbb0 0x557bc4156260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.418+0000 7f5b3caae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557bc414bed0 con 0x557bc413bbb0 2024-09-10T08:46:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.418+0000 7f5b3caae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557bc414c5a0 con 0x557bc41312f0 2024-09-10T08:46:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.418+0000 7f5b3caae740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557bc414cc70 con 0x557bc41309d0 2024-09-10T08:46:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.418+0000 7f5b38c72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc413bbb0 0x557bc4156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.418+0000 7f5b38c72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc413bbb0 0x557bc4156260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33242/0 (socket says 172.21.15.172:33242) 2024-09-10T08:46:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.418+0000 7f5b38c72640 1 -- 172.21.15.172:0/1772681676 learned_addr learned my addr 172.21.15.172:0/1772681676 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.418+0000 7f5b337fe640 1 --2- 172.21.15.172:0/1772681676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc41312f0 0x557bc4138010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.418+0000 7f5b33fff640 1 --2- 172.21.15.172:0/1772681676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc41309d0 0x557bc414db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.418+0000 7f5b38c72640 1 -- 172.21.15.172:0/1772681676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc41309d0 msgr2=0x557bc414db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.418+0000 7f5b38c72640 1 --2- 172.21.15.172:0/1772681676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc41309d0 0x557bc414db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.418+0000 7f5b38c72640 1 -- 172.21.15.172:0/1772681676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc41312f0 msgr2=0x557bc4138010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:30.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.418+0000 7f5b38c72640 1 --2- 172.21.15.172:0/1772681676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc41312f0 0x557bc4138010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.418+0000 7f5b38c72640 1 -- 172.21.15.172:0/1772681676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bc412e0f0 con 0x557bc413bbb0 2024-09-10T08:46:30.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.418+0000 7f5b38c72640 1 --2- 172.21.15.172:0/1772681676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc413bbb0 0x557bc4156260 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009cd0 tx=0x7f5b2c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997419f9c4dcd937 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:30.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.419+0000 7f5b29ffb640 1 -- 172.21.15.172:0/1772681676 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c049620 con 0x557bc413bbb0 2024-09-10T08:46:30.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.419+0000 7f5b29ffb640 1 -- 172.21.15.172:0/1772681676 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b2c036050 con 0x557bc413bbb0 2024-09-10T08:46:30.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.419+0000 7f5b29ffb640 1 -- 172.21.15.172:0/1772681676 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0537f0 con 0x557bc413bbb0 2024-09-10T08:46:30.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.419+0000 7f5b3caae740 1 -- 172.21.15.172:0/1772681676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc413bbb0 msgr2=0x557bc4156260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.419+0000 7f5b3caae740 1 --2- 172.21.15.172:0/1772681676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc413bbb0 0x557bc4156260 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009cd0 tx=0x7f5b2c041000 comp rx=0 tx=0).stop 2024-09-10T08:46:30.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.419+0000 7f5b3caae740 1 -- 172.21.15.172:0/1772681676 shutdown_connections 2024-09-10T08:46:30.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.419+0000 7f5b3caae740 1 --2- 172.21.15.172:0/1772681676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc413bbb0 0x557bc4156260 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.419+0000 7f5b3caae740 1 --2- 172.21.15.172:0/1772681676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc41312f0 0x557bc4138010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.419+0000 7f5b3caae740 1 --2- 172.21.15.172:0/1772681676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc41309d0 0x557bc414db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.419+0000 7f5b3caae740 1 -- 172.21.15.172:0/1772681676 >> 172.21.15.172:0/1772681676 conn(0x557bc412cf60 msgr2=0x557bc412d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:30.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.419+0000 7f5b3caae740 1 -- 172.21.15.172:0/1772681676 shutdown_connections 2024-09-10T08:46:30.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.419+0000 7f5b3caae740 1 -- 172.21.15.172:0/1772681676 wait complete. 2024-09-10T08:46:30.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.420+0000 7f5b3caae740 1 -- start start 2024-09-10T08:46:30.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.420+0000 7f5b3caae740 1 Processor -- start 2024-09-10T08:46:30.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.420+0000 7f5b3caae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc41309d0 0x557bc4155420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.420+0000 7f5b3caae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc41312f0 0x557bc4155960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.420+0000 7f5b3caae740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc413bbb0 0x557bc41b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.420+0000 7f5b3caae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557bc414da10 con 0x557bc41312f0 2024-09-10T08:46:30.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.420+0000 7f5b3caae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557bc414d340 con 0x557bc41309d0 2024-09-10T08:46:30.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.420+0000 7f5b3caae740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557bc414e0e0 con 0x557bc413bbb0 2024-09-10T08:46:30.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.420+0000 7f5b3caae740 10 client.0 _ll_register_callbacks cb 0x7f5b2400c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:30.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.420+0000 7f5b337fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc41312f0 0x557bc4155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.420+0000 7f5b337fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc41312f0 0x557bc4155960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33258/0 (socket says 172.21.15.172:33258) 2024-09-10T08:46:30.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.420+0000 7f5b337fe640 1 -- 172.21.15.172:0/3726726641 learned_addr learned my addr 172.21.15.172:0/3726726641 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:30.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.420+0000 7f5b33fff640 1 --2- 172.21.15.172:0/3726726641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc41309d0 0x557bc4155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.420+0000 7f5b38c72640 1 --2- 172.21.15.172:0/3726726641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc413bbb0 0x557bc41b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b337fe640 1 -- 172.21.15.172:0/3726726641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc413bbb0 msgr2=0x557bc41b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:30.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b337fe640 1 --2- 172.21.15.172:0/3726726641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc413bbb0 0x557bc41b8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b337fe640 1 -- 172.21.15.172:0/3726726641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc41309d0 msgr2=0x557bc4155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:30.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b337fe640 1 --2- 172.21.15.172:0/3726726641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc41309d0 0x557bc4155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b337fe640 1 -- 172.21.15.172:0/3726726641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bc41b9500 con 0x557bc41312f0 2024-09-10T08:46:30.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b337fe640 1 --2- 172.21.15.172:0/3726726641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc41312f0 0x557bc4155960 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00f010 tx=0x7f5b1c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331414230a1c72cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:30.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b137fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:30.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b137fe640 1 -- 172.21.15.172:0/3726726641 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c020040 con 0x557bc41312f0 2024-09-10T08:46:30.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b137fe640 1 -- 172.21.15.172:0/3726726641 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5b1c004220 con 0x557bc41312f0 2024-09-10T08:46:30.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b3caae740 1 -- 172.21.15.172:0/3726726641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b2c036050 con 0x557bc41312f0 2024-09-10T08:46:30.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b137fe640 1 -- 172.21.15.172:0/3726726641 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b1c036050 con 0x557bc41312f0 2024-09-10T08:46:30.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b3caae740 10 client.15303 fetch_fsmap learned FSMap version 28 2024-09-10T08:46:30.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b3caae740 1 -- 172.21.15.172:0/3726726641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557bc41ba800 con 0x557bc41312f0 2024-09-10T08:46:30.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b137fe640 1 -- 172.21.15.172:0/3726726641 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b2c036050 con 0x557bc41312f0 2024-09-10T08:46:30.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b137fe640 1 -- 172.21.15.172:0/3726726641 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f5b1c0047a0 con 0x557bc41312f0 2024-09-10T08:46:30.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b3caae740 10 client.15303 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T08:46:30.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b3caae740 10 client.15303 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:46:30.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.421+0000 7f5b3caae740 20 client.15303 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:30.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.422+0000 7f5b3caae740 1 --2- 172.21.15.172:0/3726726641 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557bc41bf0e0 0x557bc41bddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:30.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.422+0000 7f5b3caae740 4 client.15303 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:46:30.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.422+0000 7f5b3caae740 1 -- 172.21.15.172:0/3726726641 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "status"}) -- 0x557bc409eb10 con 0x557bc41bf0e0 2024-09-10T08:46:30.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.422+0000 7f5b33fff640 1 --2- 172.21.15.172:0/3726726641 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557bc41bf0e0 0x557bc41bddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:30.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.422+0000 7f5b33fff640 1 --2- 172.21.15.172:0/3726726641 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557bc41bf0e0 0x557bc41bddf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f11beb26fb8d565 server_cookie=d8f27a874ace30b1 in_seq=0 out_seq=0 2024-09-10T08:46:30.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.422+0000 7f5b137fe640 10 client.15303 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:30.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.422+0000 7f5b137fe640 1 -- 172.21.15.172:0/3726726641 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557bc409eb10 con 0x557bc41bf0e0 2024-09-10T08:46:30.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.422+0000 7f5b137fe640 10 client.15303 handle_command_reply: tid=0 2024-09-10T08:46:30.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.422+0000 7f5b3caae740 1 client.15303 shutdown 2024-09-10T08:46:30.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.423+0000 7f5b3caae740 1 -- 172.21.15.172:0/3726726641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc41312f0 msgr2=0x557bc4155960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.423+0000 7f5b3caae740 1 --2- 172.21.15.172:0/3726726641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc41312f0 0x557bc4155960 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00f010 tx=0x7f5b1c00c910 comp rx=0 tx=0).stop 2024-09-10T08:46:30.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.423+0000 7f5b3caae740 1 -- 172.21.15.172:0/3726726641 shutdown_connections 2024-09-10T08:46:30.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.423+0000 7f5b3caae740 1 --2- 172.21.15.172:0/3726726641 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557bc41bf0e0 0x557bc41bddf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.423+0000 7f5b3caae740 1 --2- 172.21.15.172:0/3726726641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557bc413bbb0 0x557bc41b8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.423+0000 7f5b3caae740 1 --2- 172.21.15.172:0/3726726641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bc41312f0 0x557bc4155960 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.423+0000 7f5b137fe640 0 client.15303 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:30.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.423+0000 7f5b3caae740 1 --2- 172.21.15.172:0/3726726641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bc41309d0 0x557bc4155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.423+0000 7f5b3caae740 1 -- 172.21.15.172:0/3726726641 >> 172.21.15.172:0/3726726641 conn(0x557bc412cf60 msgr2=0x557bc4137020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:30.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.423+0000 7f5b3caae740 1 -- 172.21.15.172:0/3726726641 shutdown_connections 2024-09-10T08:46:30.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.423+0000 7f5b3caae740 1 -- 172.21.15.172:0/3726726641 wait complete. 2024-09-10T08:46:30.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.423+0000 7f5b3caae740 20 client.15303 trim_cache size 0 max 16384 2024-09-10T08:46:30.433 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:46:30.433 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:46:30.433 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-09-10T08:46:30.433 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 24488, 2024-09-10T08:46:30.433 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-09-10T08:46:30.433 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-09-10T08:46:30.434 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-09-10T08:46:30.434 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 138.74839859599999, 2024-09-10T08:46:30.434 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 28, 2024-09-10T08:46:30.434 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 92, 2024-09-10T08:46:30.434 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 88, 2024-09-10T08:46:30.434 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 149.136685696 2024-09-10T08:46:30.434 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:46:30.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.423+0000 7f5b3a6d5640 1 -- 172.21.15.172:0/1736168043 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b200a2b80 msgr2=0x7f5b200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.423+0000 7f5b3a6d5640 1 --2- 172.21.15.172:0/1736168043 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b200a2b80 0x7f5b200a5040 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0317f0 tx=0x7f5b2c0093a0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.423+0000 7f5b3a6d5640 1 -- 172.21.15.172:0/1736168043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b34100a80 msgr2=0x7f5b34198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:30.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.423+0000 7f5b3a6d5640 1 --2- 172.21.15.172:0/1736168043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b34100a80 0x7f5b34198260 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400c920 tx=0x7f5b2400cde0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.424+0000 7f5b3a6d5640 1 -- 172.21.15.172:0/1736168043 shutdown_connections 2024-09-10T08:46:30.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.424+0000 7f5b3a6d5640 1 --2- 172.21.15.172:0/1736168043 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b200a2b80 0x7f5b200a5040 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.424+0000 7f5b3a6d5640 1 --2- 172.21.15.172:0/1736168043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b34106a80 0x7f5b3419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.424+0000 7f5b3a6d5640 1 --2- 172.21.15.172:0/1736168043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34101420 0x7f5b341987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.424+0000 7f5b3a6d5640 1 --2- 172.21.15.172:0/1736168043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b34100a80 0x7f5b34198260 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:30.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.424+0000 7f5b3a6d5640 1 -- 172.21.15.172:0/1736168043 >> 172.21.15.172:0/1736168043 conn(0x7f5b340fc7a0 msgr2=0x7f5b3406a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:30.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.424+0000 7f5b3a6d5640 1 -- 172.21.15.172:0/1736168043 shutdown_connections 2024-09-10T08:46:30.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:30.424+0000 7f5b3a6d5640 1 -- 172.21.15.172:0/1736168043 wait complete. 2024-09-10T08:46:30.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:30 smithi174 ceph-mon[34068]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 494 B/s wr, 10 op/s 2024-09-10T08:46:30.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:30 smithi191 ceph-mon[35314]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 494 B/s wr, 10 op/s 2024-09-10T08:46:31.094 INFO:teuthology.run_tasks:Running task sequential... 2024-09-10T08:46:31.104 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-10T08:46:31.104 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-10T08:46:31.104 INFO:teuthology.task.print:Enabling mgr modules 2024-09-10T08:46:31.104 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-10T08:46:31.114 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-10T08:46:31.114 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-10T08:46:31.115 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-10T08:46:31.115 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T08:46:31.115 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T08:46:31.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:46:31.115 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-09-10T08:46:31.132 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-10T08:46:31.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:46:31.133 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T08:46:31.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:46:31.199 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2024-09-10T08:46:31.300 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-10T08:46:31.300 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T08:46:31.301 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T08:46:31.553 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:32.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.051+0000 7f56f9c1c640 1 -- 172.21.15.172:0/1256046213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f4101420 msgr2=0x7f56f410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:32.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.051+0000 7f56f9c1c640 1 --2- 172.21.15.172:0/1256046213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f4101420 0x7f56f410d950 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009930 tx=0x7f56ec02a420 comp rx=0 tx=0).stop 2024-09-10T08:46:32.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.051+0000 7f56f9c1c640 1 -- 172.21.15.172:0/1256046213 shutdown_connections 2024-09-10T08:46:32.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.051+0000 7f56f9c1c640 1 --2- 172.21.15.172:0/1256046213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f4101420 0x7f56f410d950 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.051+0000 7f56f9c1c640 1 --2- 172.21.15.172:0/1256046213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56f4100a80 0x7f56f4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.051+0000 7f56f9c1c640 1 --2- 172.21.15.172:0/1256046213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56f4106a80 0x7f56f4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.051+0000 7f56f9c1c640 1 -- 172.21.15.172:0/1256046213 >> 172.21.15.172:0/1256046213 conn(0x7f56f40fc7a0 msgr2=0x7f56f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.051+0000 7f56f9c1c640 1 -- 172.21.15.172:0/1256046213 shutdown_connections 2024-09-10T08:46:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.051+0000 7f56f9c1c640 1 -- 172.21.15.172:0/1256046213 wait complete. 2024-09-10T08:46:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.052+0000 7f56f9c1c640 1 Processor -- start 2024-09-10T08:46:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.052+0000 7f56f9c1c640 1 -- start start 2024-09-10T08:46:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.052+0000 7f56f9c1c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f4100a80 0x7f56f4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:32.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.052+0000 7f56f9c1c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56f4101420 0x7f56f41996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:32.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.052+0000 7f56f9c1c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56f4106a80 0x7f56f41a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:32.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.052+0000 7f56f9c1c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f56f4110d80 con 0x7f56f4100a80 2024-09-10T08:46:32.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.052+0000 7f56f9c1c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56f4110c00 con 0x7f56f4106a80 2024-09-10T08:46:32.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.052+0000 7f56f9c1c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f56f4110f00 con 0x7f56f4101420 2024-09-10T08:46:32.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.052+0000 7f56f37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f4100a80 0x7f56f4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.052+0000 7f56f37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f4100a80 0x7f56f4199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33270/0 (socket says 172.21.15.172:33270) 2024-09-10T08:46:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.052+0000 7f56f37fe640 1 -- 172.21.15.172:0/1380503743 learned_addr learned my addr 172.21.15.172:0/1380503743 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.053+0000 7f56f2ffd640 1 --2- 172.21.15.172:0/1380503743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56f4101420 0x7f56f41996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.053+0000 7f56f3fff640 1 --2- 172.21.15.172:0/1380503743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56f4106a80 0x7f56f41a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.053+0000 7f56f37fe640 1 -- 172.21.15.172:0/1380503743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56f4101420 msgr2=0x7f56f41996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.053+0000 7f56f37fe640 1 --2- 172.21.15.172:0/1380503743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56f4101420 0x7f56f41996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.053+0000 7f56f37fe640 1 -- 172.21.15.172:0/1380503743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56f4106a80 msgr2=0x7f56f41a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.053+0000 7f56f37fe640 1 --2- 172.21.15.172:0/1380503743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56f4106a80 0x7f56f41a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.053+0000 7f56f37fe640 1 -- 172.21.15.172:0/1380503743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f41a4750 con 0x7f56f4100a80 2024-09-10T08:46:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.053+0000 7f56f37fe640 1 --2- 172.21.15.172:0/1380503743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f4100a80 0x7f56f4199190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f56e400eab0 tx=0x7f56e400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:32.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.053+0000 7f56f0ff9640 1 -- 172.21.15.172:0/1380503743 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e400ce40 con 0x7f56f4100a80 2024-09-10T08:46:32.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.053+0000 7f56f9c1c640 1 -- 172.21.15.172:0/1380503743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f41a4920 con 0x7f56f4100a80 2024-09-10T08:46:32.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.053+0000 7f56f0ff9640 1 -- 172.21.15.172:0/1380503743 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56e4011920 con 0x7f56f4100a80 2024-09-10T08:46:32.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.053+0000 7f56f0ff9640 1 -- 172.21.15.172:0/1380503743 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e4010a00 con 0x7f56f4100a80 2024-09-10T08:46:32.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.053+0000 7f56f9c1c640 1 -- 172.21.15.172:0/1380503743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f41a4e10 con 0x7f56f4100a80 2024-09-10T08:46:32.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.054+0000 7f56f9c1c640 1 -- 172.21.15.172:0/1380503743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f4100ee0 con 0x7f56f4100a80 2024-09-10T08:46:32.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.055+0000 7f56f0ff9640 1 -- 172.21.15.172:0/1380503743 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f56e4011ad0 con 0x7f56f4100a80 2024-09-10T08:46:32.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.056+0000 7f56f0ff9640 1 --2- 172.21.15.172:0/1380503743 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56dc0a2b80 0x7f56dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:32.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.056+0000 7f56f0ff9640 1 -- 172.21.15.172:0/1380503743 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f56e40cf8f0 con 0x7f56f4100a80 2024-09-10T08:46:32.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.056+0000 7f56f2ffd640 1 --2- 172.21.15.172:0/1380503743 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56dc0a2b80 0x7f56dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:32.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.056+0000 7f56f2ffd640 1 --2- 172.21.15.172:0/1380503743 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56dc0a2b80 0x7f56dc0a5040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f56f406b2e0 tx=0x7f56e800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:32.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.060+0000 7f56f0ff9640 1 -- 172.21.15.172:0/1380503743 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56e4016030 con 0x7f56f4100a80 2024-09-10T08:46:32.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.271+0000 7f56f9c1c640 1 -- 172.21.15.172:0/1380503743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f56f4108ff0 con 0x7f56f4100a80 2024-09-10T08:46:32.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.271+0000 7f56f0ff9640 1 -- 172.21.15.172:0/1380503743 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7f56e409c4f0 con 0x7f56f4100a80 2024-09-10T08:46:32.273 INFO:teuthology.orchestra.run.smithi172.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T08:46:32.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.273+0000 7f56f9c1c640 1 -- 172.21.15.172:0/1380503743 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56dc0a2b80 msgr2=0x7f56dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:32.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.273+0000 7f56f9c1c640 1 --2- 172.21.15.172:0/1380503743 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56dc0a2b80 0x7f56dc0a5040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f56f406b2e0 tx=0x7f56e800a2d0 comp rx=0 tx=0).stop 2024-09-10T08:46:32.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.273+0000 7f56f9c1c640 1 -- 172.21.15.172:0/1380503743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f4100a80 msgr2=0x7f56f4199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:32.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.273+0000 7f56f9c1c640 1 --2- 172.21.15.172:0/1380503743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f4100a80 0x7f56f4199190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f56e400eab0 tx=0x7f56e400ef70 comp rx=0 tx=0).stop 2024-09-10T08:46:32.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.273+0000 7f56f9c1c640 1 -- 172.21.15.172:0/1380503743 shutdown_connections 2024-09-10T08:46:32.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.273+0000 7f56f9c1c640 1 --2- 172.21.15.172:0/1380503743 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56dc0a2b80 0x7f56dc0a5040 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:32.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.273+0000 7f56f9c1c640 1 --2- 172.21.15.172:0/1380503743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56f4106a80 0x7f56f41a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:32.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.273+0000 7f56f9c1c640 1 --2- 172.21.15.172:0/1380503743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56f4101420 0x7f56f41996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:32.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.273+0000 7f56f9c1c640 1 --2- 172.21.15.172:0/1380503743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f4100a80 0x7f56f4199190 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:32.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.273+0000 7f56f9c1c640 1 -- 172.21.15.172:0/1380503743 >> 172.21.15.172:0/1380503743 conn(0x7f56f40fc7a0 msgr2=0x7f56f40fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:32.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.274+0000 7f56f9c1c640 1 -- 172.21.15.172:0/1380503743 shutdown_connections 2024-09-10T08:46:32.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:32.274+0000 7f56f9c1c640 1 -- 172.21.15.172:0/1380503743 wait complete. 2024-09-10T08:46:32.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:32 smithi174 ceph-mon[34068]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-10T08:46:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:32 smithi191 ceph-mon[35314]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-10T08:46:32.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:32 smithi172 ceph-mon[30854]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-10T08:46:32.981 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T08:46:32.981 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T08:46:32.981 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-09-10T08:46:32.981 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T08:46:32.981 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T08:46:32.981 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T08:46:32.981 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T08:46:32.982 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T08:46:32.982 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T08:46:32.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:46:32.982 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-09-10T08:46:32.999 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T08:46:32.999 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T08:46:32.999 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T08:46:32.999 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:46:32.999 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-09-10T08:46:32.999 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:46:32.999 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:46:33.000 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-09-10T08:46:33.000 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-09-10T08:46:33.000 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:46:33.000 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-09-10T08:46:33.000 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-09-10T08:46:33.000 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T08:46:33.000 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:46:33.000 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T08:46:33.000 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-09-10T08:46:33.000 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T08:46:33.000 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42138sec preferred_lft 42138sec 2024-09-10T08:46:33.000 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link noprefixroute 2024-09-10T08:46:33.000 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-09-10T08:46:33.001 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T08:46:33.001 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:46:33.001 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-09-10T08:46:33.001 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add name ceph-brx type bridge 2024-09-10T08:46:33.001 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr flush dev ceph-brx 2024-09-10T08:46:33.001 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set ceph-brx up 2024-09-10T08:46:33.001 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T08:46:33.001 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-09-10T08:46:33.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:46:33.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:46:33.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:46:33.140 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T08:46:33.164 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-09-10T08:46:33.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:46:33.165 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-09-10T08:46:33.222 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-09-10T08:46:33.222 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T08:46:33.222 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T08:46:33.222 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T08:46:33.222 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T08:46:33.222 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:46:33.223 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-09-10T08:46:33.223 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T08:46:33.223 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T08:46:33.223 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T08:46:33.223 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-09-10T08:46:33.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:46:33.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:46:33.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:46:33.350 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-09-10T08:46:33.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:46:33.366 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list-id 2024-09-10T08:46:33.422 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:46:33.422 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-09-10T08:46:33.422 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T08:46:33.422 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T08:46:33.422 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-09-10T08:46:33.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:33 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1380503743' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:46:33.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:46:33.518 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:33 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1380503743' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:46:33.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:46:33.540 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T08:46:33.540 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:46:33.540 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-09-10T08:46:33.540 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T08:46:33.540 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T08:46:33.540 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T08:46:33.540 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T08:46:33.540 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T08:46:33.541 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-09-10T08:46:33.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:33 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1380503743' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T08:46:33.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:46:33.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:46:33.654 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T08:46:33.655 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-09-10T08:46:33.655 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-09-10T08:46:33.655 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T08:46:33.655 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-09-10T08:46:33.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T08:46:33.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:46:33.732 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-10T08:46:33.732 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T08:46:33.732 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T08:46:33.747 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T08:46:33.748 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T08:46:33.748 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T08:46:33.803 DEBUG:teuthology.orchestra.run.smithi172:> sudo modprobe fuse 2024-09-10T08:46:33.870 DEBUG:teuthology.orchestra.run.smithi172:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T08:46:33.927 INFO:teuthology.orchestra.run.smithi172.stdout:/proc 2024-09-10T08:46:33.927 INFO:teuthology.orchestra.run.smithi172.stdout:/sys 2024-09-10T08:46:33.927 INFO:teuthology.orchestra.run.smithi172.stdout:/dev 2024-09-10T08:46:33.927 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/security 2024-09-10T08:46:33.927 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/shm 2024-09-10T08:46:33.927 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/pts 2024-09-10T08:46:33.927 INFO:teuthology.orchestra.run.smithi172.stdout:/run 2024-09-10T08:46:33.927 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/cgroup 2024-09-10T08:46:33.928 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/pstore 2024-09-10T08:46:33.928 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/bpf 2024-09-10T08:46:33.928 INFO:teuthology.orchestra.run.smithi172.stdout:/ 2024-09-10T08:46:33.928 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/selinux 2024-09-10T08:46:33.928 INFO:teuthology.orchestra.run.smithi172.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:46:33.928 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/debug 2024-09-10T08:46:33.928 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/mqueue 2024-09-10T08:46:33.928 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/hugepages 2024-09-10T08:46:33.928 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/tracing 2024-09-10T08:46:33.928 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/fuse/connections 2024-09-10T08:46:33.928 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T08:46:33.928 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/config 2024-09-10T08:46:33.928 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T08:46:33.929 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T08:46:33.929 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T08:46:33.929 INFO:teuthology.orchestra.run.smithi172.stdout:/run/user/1000 2024-09-10T08:46:33.929 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/ceph 2024-09-10T08:46:33.929 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay 2024-09-10T08:46:33.929 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/82e89ae6b50bdf753acd9f704569e643dc3c43dadf2c992d822fd3df2e2145a2/merged 2024-09-10T08:46:33.929 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/a42cc49770da75f41ece537546cf7211f25dee1fa59bf0e063d45dce66636908/merged 2024-09-10T08:46:33.929 INFO:teuthology.orchestra.run.smithi172.stdout:/run/user/0 2024-09-10T08:46:33.929 INFO:teuthology.orchestra.run.smithi172.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:46:33.929 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/c5a291b40f14d210658e79214dd8b0a20ab5c89b11b760d33358bc3eaaee4f07/merged 2024-09-10T08:46:33.929 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/9dfeb1b3da512da1821f7f773d5f0da825be3b645c8d588ef05e735a8f3179ce/merged 2024-09-10T08:46:33.929 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/fe3ade833ac302061953a0451b5f0bc6033afe2aab408dccdb42c5d7de392d73/merged 2024-09-10T08:46:33.930 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/7102ef80c772fb9c554e1496711136185f60e8668fe2ad8884b8006eef379002/merged 2024-09-10T08:46:33.930 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/a8be33b683701eaef8f6eebda60cfecf60cebf303fa9dacd6e08c920c0729ade/merged 2024-09-10T08:46:33.930 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/10e0103da8bd78312507fe36d16529f9090728d08bb98cf2ef03cbdbc8da26c2/merged 2024-09-10T08:46:33.930 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/d1a43694eb90671cd8405e144dc729b9f0db71eadbdc047d003177bbd777254c/merged 2024-09-10T08:46:33.930 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/ee332b36b97a9fb6adfb4e4de3e3dc938e9fc0a9bfe3952e9c04ed117956487c/merged 2024-09-10T08:46:33.930 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns 2024-09-10T08:46:33.930 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T08:46:33.930 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T08:46:33.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:46:33.930 DEBUG:teuthology.orchestra.run.smithi172:> ls /sys/fs/fuse/connections 2024-09-10T08:46:33.983 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T08:46:33.983 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/1a98a737-45db-4cda-bd0b-58296d0e5397 --client_fs=cephfs) 2024-09-10T08:46:34.026 DEBUG:teuthology.orchestra.run.smithi172:> sudo modprobe fuse 2024-09-10T08:46:34.050 DEBUG:teuthology.orchestra.run.smithi172:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T08:46:34.106 INFO:teuthology.orchestra.run.smithi172.stdout:/proc 2024-09-10T08:46:34.107 INFO:teuthology.orchestra.run.smithi172.stdout:/sys 2024-09-10T08:46:34.107 INFO:teuthology.orchestra.run.smithi172.stdout:/dev 2024-09-10T08:46:34.107 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/security 2024-09-10T08:46:34.107 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/shm 2024-09-10T08:46:34.107 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/pts 2024-09-10T08:46:34.107 INFO:teuthology.orchestra.run.smithi172.stdout:/run 2024-09-10T08:46:34.107 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/cgroup 2024-09-10T08:46:34.107 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/pstore 2024-09-10T08:46:34.107 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/bpf 2024-09-10T08:46:34.107 INFO:teuthology.orchestra.run.smithi172.stdout:/ 2024-09-10T08:46:34.107 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/selinux 2024-09-10T08:46:34.108 INFO:teuthology.orchestra.run.smithi172.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:46:34.108 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/debug 2024-09-10T08:46:34.108 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/mqueue 2024-09-10T08:46:34.108 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/hugepages 2024-09-10T08:46:34.108 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/tracing 2024-09-10T08:46:34.108 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/fuse/connections 2024-09-10T08:46:34.108 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T08:46:34.108 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/config 2024-09-10T08:46:34.108 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T08:46:34.108 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T08:46:34.108 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T08:46:34.108 INFO:teuthology.orchestra.run.smithi172.stdout:/run/user/1000 2024-09-10T08:46:34.109 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/ceph 2024-09-10T08:46:34.109 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay 2024-09-10T08:46:34.109 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/82e89ae6b50bdf753acd9f704569e643dc3c43dadf2c992d822fd3df2e2145a2/merged 2024-09-10T08:46:34.109 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/a42cc49770da75f41ece537546cf7211f25dee1fa59bf0e063d45dce66636908/merged 2024-09-10T08:46:34.109 INFO:teuthology.orchestra.run.smithi172.stdout:/run/user/0 2024-09-10T08:46:34.109 INFO:teuthology.orchestra.run.smithi172.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:46:34.109 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/c5a291b40f14d210658e79214dd8b0a20ab5c89b11b760d33358bc3eaaee4f07/merged 2024-09-10T08:46:34.109 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/9dfeb1b3da512da1821f7f773d5f0da825be3b645c8d588ef05e735a8f3179ce/merged 2024-09-10T08:46:34.109 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/fe3ade833ac302061953a0451b5f0bc6033afe2aab408dccdb42c5d7de392d73/merged 2024-09-10T08:46:34.109 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/7102ef80c772fb9c554e1496711136185f60e8668fe2ad8884b8006eef379002/merged 2024-09-10T08:46:34.109 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/a8be33b683701eaef8f6eebda60cfecf60cebf303fa9dacd6e08c920c0729ade/merged 2024-09-10T08:46:34.109 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/10e0103da8bd78312507fe36d16529f9090728d08bb98cf2ef03cbdbc8da26c2/merged 2024-09-10T08:46:34.110 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/d1a43694eb90671cd8405e144dc729b9f0db71eadbdc047d003177bbd777254c/merged 2024-09-10T08:46:34.110 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/ee332b36b97a9fb6adfb4e4de3e3dc938e9fc0a9bfe3952e9c04ed117956487c/merged 2024-09-10T08:46:34.110 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns 2024-09-10T08:46:34.110 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T08:46:34.110 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T08:46:34.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:46:34.110 DEBUG:teuthology.orchestra.run.smithi172:> ls /sys/fs/fuse/connections 2024-09-10T08:46:34.111 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi172.stderr:2024-09-10T08:46:34.107+0000 7f924d674480 -1 init, newargv = 0x5654a22588f0 newargc=15 2024-09-10T08:46:34.111 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi172.stderr:ceph-fuse[82703]: starting ceph client 2024-09-10T08:46:34.132 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi172.stderr:ceph-fuse[82703]: starting fuse 2024-09-10T08:46:34.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:34 smithi174 ceph-mon[34068]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T08:46:34.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:34 smithi191 ceph-mon[35314]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T08:46:34.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:34 smithi172 ceph-mon[30854]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T08:46:35.228 DEBUG:teuthology.orchestra.run.smithi172:> sudo modprobe fuse 2024-09-10T08:46:35.256 DEBUG:teuthology.orchestra.run.smithi172:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T08:46:35.315 INFO:teuthology.orchestra.run.smithi172.stdout:/proc 2024-09-10T08:46:35.315 INFO:teuthology.orchestra.run.smithi172.stdout:/sys 2024-09-10T08:46:35.315 INFO:teuthology.orchestra.run.smithi172.stdout:/dev 2024-09-10T08:46:35.315 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/security 2024-09-10T08:46:35.315 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/shm 2024-09-10T08:46:35.315 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/pts 2024-09-10T08:46:35.315 INFO:teuthology.orchestra.run.smithi172.stdout:/run 2024-09-10T08:46:35.315 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/cgroup 2024-09-10T08:46:35.315 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/pstore 2024-09-10T08:46:35.315 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/bpf 2024-09-10T08:46:35.316 INFO:teuthology.orchestra.run.smithi172.stdout:/ 2024-09-10T08:46:35.316 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/selinux 2024-09-10T08:46:35.316 INFO:teuthology.orchestra.run.smithi172.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:46:35.316 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/debug 2024-09-10T08:46:35.316 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/mqueue 2024-09-10T08:46:35.316 INFO:teuthology.orchestra.run.smithi172.stdout:/dev/hugepages 2024-09-10T08:46:35.316 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/tracing 2024-09-10T08:46:35.316 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/fs/fuse/connections 2024-09-10T08:46:35.316 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T08:46:35.316 INFO:teuthology.orchestra.run.smithi172.stdout:/sys/kernel/config 2024-09-10T08:46:35.316 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T08:46:35.316 INFO:teuthology.orchestra.run.smithi172.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T08:46:35.317 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T08:46:35.317 INFO:teuthology.orchestra.run.smithi172.stdout:/run/user/1000 2024-09-10T08:46:35.317 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/ceph 2024-09-10T08:46:35.317 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay 2024-09-10T08:46:35.317 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/82e89ae6b50bdf753acd9f704569e643dc3c43dadf2c992d822fd3df2e2145a2/merged 2024-09-10T08:46:35.317 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/a42cc49770da75f41ece537546cf7211f25dee1fa59bf0e063d45dce66636908/merged 2024-09-10T08:46:35.317 INFO:teuthology.orchestra.run.smithi172.stdout:/run/user/0 2024-09-10T08:46:35.317 INFO:teuthology.orchestra.run.smithi172.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T08:46:35.317 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/c5a291b40f14d210658e79214dd8b0a20ab5c89b11b760d33358bc3eaaee4f07/merged 2024-09-10T08:46:35.317 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/9dfeb1b3da512da1821f7f773d5f0da825be3b645c8d588ef05e735a8f3179ce/merged 2024-09-10T08:46:35.317 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/fe3ade833ac302061953a0451b5f0bc6033afe2aab408dccdb42c5d7de392d73/merged 2024-09-10T08:46:35.317 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/7102ef80c772fb9c554e1496711136185f60e8668fe2ad8884b8006eef379002/merged 2024-09-10T08:46:35.317 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/a8be33b683701eaef8f6eebda60cfecf60cebf303fa9dacd6e08c920c0729ade/merged 2024-09-10T08:46:35.318 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/10e0103da8bd78312507fe36d16529f9090728d08bb98cf2ef03cbdbc8da26c2/merged 2024-09-10T08:46:35.318 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/d1a43694eb90671cd8405e144dc729b9f0db71eadbdc047d003177bbd777254c/merged 2024-09-10T08:46:35.318 INFO:teuthology.orchestra.run.smithi172.stdout:/var/lib/containers/storage/overlay/ee332b36b97a9fb6adfb4e4de3e3dc938e9fc0a9bfe3952e9c04ed117956487c/merged 2024-09-10T08:46:35.318 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns 2024-09-10T08:46:35.318 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T08:46:35.318 INFO:teuthology.orchestra.run.smithi172.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T08:46:35.318 INFO:teuthology.orchestra.run.smithi172.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-10T08:46:35.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:46:35.318 DEBUG:teuthology.orchestra.run.smithi172:> ls /sys/fs/fuse/connections 2024-09-10T08:46:35.373 INFO:teuthology.orchestra.run.smithi172.stdout:120 2024-09-10T08:46:35.374 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-10T08:46:35.374 DEBUG:teuthology.orchestra.run.smithi172:> sudo stdin-killer -- python3 -c ' 2024-09-10T08:46:35.374 DEBUG:teuthology.orchestra.run.smithi172:> import glob 2024-09-10T08:46:35.374 DEBUG:teuthology.orchestra.run.smithi172:> import re 2024-09-10T08:46:35.374 DEBUG:teuthology.orchestra.run.smithi172:> import os 2024-09-10T08:46:35.374 DEBUG:teuthology.orchestra.run.smithi172:> import subprocess 2024-09-10T08:46:35.374 DEBUG:teuthology.orchestra.run.smithi172:> 2024-09-10T08:46:35.374 DEBUG:teuthology.orchestra.run.smithi172:> def _find_admin_socket(client_name): 2024-09-10T08:46:35.374 DEBUG:teuthology.orchestra.run.smithi172:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T08:46:35.375 DEBUG:teuthology.orchestra.run.smithi172:> files = glob.glob(asok_path) 2024-09-10T08:46:35.375 DEBUG:teuthology.orchestra.run.smithi172:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T08:46:35.375 DEBUG:teuthology.orchestra.run.smithi172:> 2024-09-10T08:46:35.375 DEBUG:teuthology.orchestra.run.smithi172:> # Given a non-glob path, it better be there 2024-09-10T08:46:35.375 DEBUG:teuthology.orchestra.run.smithi172:> if "*" not in asok_path: 2024-09-10T08:46:35.375 DEBUG:teuthology.orchestra.run.smithi172:> assert(len(files) == 1) 2024-09-10T08:46:35.375 DEBUG:teuthology.orchestra.run.smithi172:> return files[0] 2024-09-10T08:46:35.375 DEBUG:teuthology.orchestra.run.smithi172:> 2024-09-10T08:46:35.375 DEBUG:teuthology.orchestra.run.smithi172:> for f in files: 2024-09-10T08:46:35.375 DEBUG:teuthology.orchestra.run.smithi172:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T08:46:35.375 DEBUG:teuthology.orchestra.run.smithi172:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T08:46:35.375 DEBUG:teuthology.orchestra.run.smithi172:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T08:46:35.375 DEBUG:teuthology.orchestra.run.smithi172:> contents = proc_f.read() 2024-09-10T08:46:35.375 DEBUG:teuthology.orchestra.run.smithi172:> if mountpoint in contents: 2024-09-10T08:46:35.375 DEBUG:teuthology.orchestra.run.smithi172:> return f 2024-09-10T08:46:35.376 DEBUG:teuthology.orchestra.run.smithi172:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T08:46:35.376 DEBUG:teuthology.orchestra.run.smithi172:> 2024-09-10T08:46:35.376 DEBUG:teuthology.orchestra.run.smithi172:> print(_find_admin_socket("client.0")) 2024-09-10T08:46:35.376 DEBUG:teuthology.orchestra.run.smithi172:> ' 2024-09-10T08:46:35.495 INFO:teuthology.orchestra.run.smithi172.stdout:/var/run/ceph/ceph-client.0.82703.asok 2024-09-10T08:46:35.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T08:46:35.505 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.82703.asok 2024-09-10T08:46:35.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:46:35.506 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.82703.asok status 2024-09-10T08:46:35.639 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:46:35.640 INFO:teuthology.orchestra.run.smithi172.stdout: "metadata": { 2024-09-10T08:46:35.640 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T08:46:35.640 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T08:46:35.640 INFO:teuthology.orchestra.run.smithi172.stdout: "entity_id": "0", 2024-09-10T08:46:35.640 INFO:teuthology.orchestra.run.smithi172.stdout: "hostname": "smithi172", 2024-09-10T08:46:35.640 INFO:teuthology.orchestra.run.smithi172.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T08:46:35.640 INFO:teuthology.orchestra.run.smithi172.stdout: "pid": "82703", 2024-09-10T08:46:35.640 INFO:teuthology.orchestra.run.smithi172.stdout: "root": "/volumes/qa/sv_0/1a98a737-45db-4cda-bd0b-58296d0e5397" 2024-09-10T08:46:35.640 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:46:35.640 INFO:teuthology.orchestra.run.smithi172.stdout: "dentry_count": 0, 2024-09-10T08:46:35.640 INFO:teuthology.orchestra.run.smithi172.stdout: "dentry_pinned_count": 0, 2024-09-10T08:46:35.640 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 15315, 2024-09-10T08:46:35.641 INFO:teuthology.orchestra.run.smithi172.stdout: "inst": { 2024-09-10T08:46:35.641 INFO:teuthology.orchestra.run.smithi172.stdout: "name": { 2024-09-10T08:46:35.641 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "client", 2024-09-10T08:46:35.641 INFO:teuthology.orchestra.run.smithi172.stdout: "num": 15315 2024-09-10T08:46:35.641 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:46:35.641 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": { 2024-09-10T08:46:35.641 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v1", 2024-09-10T08:46:35.641 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "192.168.0.1:0", 2024-09-10T08:46:35.641 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 3539461266 2024-09-10T08:46:35.641 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:46:35.641 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:46:35.641 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": { 2024-09-10T08:46:35.641 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v1", 2024-09-10T08:46:35.642 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "192.168.0.1:0", 2024-09-10T08:46:35.642 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 3539461266 2024-09-10T08:46:35.642 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:46:35.642 INFO:teuthology.orchestra.run.smithi172.stdout: "inst_str": "client.15315 192.168.0.1:0/3539461266", 2024-09-10T08:46:35.642 INFO:teuthology.orchestra.run.smithi172.stdout: "addr_str": "192.168.0.1:0/3539461266", 2024-09-10T08:46:35.642 INFO:teuthology.orchestra.run.smithi172.stdout: "inode_count": 5, 2024-09-10T08:46:35.642 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_epoch": 28, 2024-09-10T08:46:35.642 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_epoch": 92, 2024-09-10T08:46:35.642 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_epoch_barrier": 0, 2024-09-10T08:46:35.642 INFO:teuthology.orchestra.run.smithi172.stdout: "blocklisted": false, 2024-09-10T08:46:35.642 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs" 2024-09-10T08:46:35.642 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:46:35.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:46:35.648 DEBUG:teuthology.orchestra.run.smithi172:> stat --file-system '--printf=%T 2024-09-10T08:46:35.648 DEBUG:teuthology.orchestra.run.smithi172:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T08:46:35.664 INFO:teuthology.orchestra.run.smithi172.stdout:fuseblk 2024-09-10T08:46:35.665 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T08:46:35.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T08:46:35.665 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T08:46:35.734 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T08:46:35.949 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T08:46:35.949 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi172.front.sepia.ceph.com 2024-09-10T08:46:35.949 DEBUG:teuthology.orchestra.run.smithi172:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-10T08:46:36.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.052+0000 7f0beba42640 1 -- 172.21.15.172:0/1288880430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0be4102d20 msgr2=0x7f0be4105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:36.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.052+0000 7f0beba42640 1 --2- 172.21.15.172:0/1288880430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0be4102d20 0x7f0be4105190 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8009900 tx=0x7f0bd802a410 comp rx=0 tx=0).stop 2024-09-10T08:46:36.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.052+0000 7f0beba42640 1 -- 172.21.15.172:0/1288880430 shutdown_connections 2024-09-10T08:46:36.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.052+0000 7f0beba42640 1 --2- 172.21.15.172:0/1288880430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0be41056d0 0x7f0be4105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:36.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.052+0000 7f0beba42640 1 --2- 172.21.15.172:0/1288880430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0be4102d20 0x7f0be4105190 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:36.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.052+0000 7f0beba42640 1 --2- 172.21.15.172:0/1288880430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0be41003d0 0x7f0be41027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:36.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.052+0000 7f0beba42640 1 -- 172.21.15.172:0/1288880430 >> 172.21.15.172:0/1288880430 conn(0x7f0be40fc1e0 msgr2=0x7f0be40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:36.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.053+0000 7f0beba42640 1 -- 172.21.15.172:0/1288880430 shutdown_connections 2024-09-10T08:46:36.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.053+0000 7f0beba42640 1 -- 172.21.15.172:0/1288880430 wait complete. 2024-09-10T08:46:36.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.053+0000 7f0beba42640 1 Processor -- start 2024-09-10T08:46:36.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.053+0000 7f0beba42640 1 -- start start 2024-09-10T08:46:36.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.053+0000 7f0beba42640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0be41003d0 0x7f0be419e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:36.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.053+0000 7f0beba42640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0be4102d20 0x7f0be419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:36.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.053+0000 7f0beba42640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0be41056d0 0x7f0be41a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:36.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.053+0000 7f0beba42640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0be4113cc0 con 0x7f0be41056d0 2024-09-10T08:46:36.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.053+0000 7f0beba42640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0be4113b40 con 0x7f0be4102d20 2024-09-10T08:46:36.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.053+0000 7f0beba42640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0be4113e40 con 0x7f0be41003d0 2024-09-10T08:46:36.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.053+0000 7f0be9fdf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0be41056d0 0x7f0be41a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:36.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.053+0000 7f0be9fdf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0be41056d0 0x7f0be41a58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33284/0 (socket says 172.21.15.172:33284) 2024-09-10T08:46:36.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.053+0000 7f0be9fdf640 1 -- 172.21.15.172:0/1724421556 learned_addr learned my addr 172.21.15.172:0/1724421556 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:36.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.054+0000 7f0be97de640 1 --2- 172.21.15.172:0/1724421556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0be41003d0 0x7f0be419e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:36.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.054+0000 7f0be8fdd640 1 --2- 172.21.15.172:0/1724421556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0be4102d20 0x7f0be419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:36.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.054+0000 7f0be9fdf640 1 -- 172.21.15.172:0/1724421556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0be41003d0 msgr2=0x7f0be419e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:36.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.054+0000 7f0be9fdf640 1 --2- 172.21.15.172:0/1724421556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0be41003d0 0x7f0be419e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:36.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.054+0000 7f0be9fdf640 1 -- 172.21.15.172:0/1724421556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0be4102d20 msgr2=0x7f0be419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:36.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.054+0000 7f0be9fdf640 1 --2- 172.21.15.172:0/1724421556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0be4102d20 0x7f0be419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:36.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.054+0000 7f0be9fdf640 1 -- 172.21.15.172:0/1724421556 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be41a5f20 con 0x7f0be41056d0 2024-09-10T08:46:36.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.054+0000 7f0be8fdd640 1 --2- 172.21.15.172:0/1724421556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0be4102d20 0x7f0be419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:36.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.054+0000 7f0be97de640 1 --2- 172.21.15.172:0/1724421556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0be41003d0 0x7f0be419e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:36.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.054+0000 7f0be9fdf640 1 --2- 172.21.15.172:0/1724421556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0be41056d0 0x7f0be41a58b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400cad0 tx=0x7f0bd400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:36.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.054+0000 7f0be27fc640 1 -- 172.21.15.172:0/1724421556 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd400ae20 con 0x7f0be41056d0 2024-09-10T08:46:36.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.054+0000 7f0beba42640 1 -- 172.21.15.172:0/1724421556 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be41a6210 con 0x7f0be41056d0 2024-09-10T08:46:36.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.054+0000 7f0be27fc640 1 -- 172.21.15.172:0/1724421556 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bd4005020 con 0x7f0be41056d0 2024-09-10T08:46:36.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.054+0000 7f0be27fc640 1 -- 172.21.15.172:0/1724421556 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd400f6b0 con 0x7f0be41056d0 2024-09-10T08:46:36.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.054+0000 7f0beba42640 1 -- 172.21.15.172:0/1724421556 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be41a6710 con 0x7f0be41056d0 2024-09-10T08:46:36.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.055+0000 7f0beba42640 1 -- 172.21.15.172:0/1724421556 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be40fdcd0 con 0x7f0be41056d0 2024-09-10T08:46:36.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.056+0000 7f0be27fc640 1 -- 172.21.15.172:0/1724421556 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0bd40051d0 con 0x7f0be41056d0 2024-09-10T08:46:36.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.057+0000 7f0be27fc640 1 --2- 172.21.15.172:0/1724421556 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0bcc0a2c30 0x7f0bcc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:36.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.057+0000 7f0be97de640 1 --2- 172.21.15.172:0/1724421556 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0bcc0a2c30 0x7f0bcc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:36.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.058+0000 7f0be27fc640 1 -- 172.21.15.172:0/1724421556 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0bd40cf4f0 con 0x7f0be41056d0 2024-09-10T08:46:36.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.058+0000 7f0be97de640 1 --2- 172.21.15.172:0/1724421556 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0bcc0a2c30 0x7f0bcc0a50f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009b40 tx=0x7f0bdc005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:36.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.062+0000 7f0be27fc640 1 -- 172.21.15.172:0/1724421556 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0bd4014030 con 0x7f0be41056d0 2024-09-10T08:46:36.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:36.266+0000 7f0beba42640 1 -- 172.21.15.172:0/1724421556 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f0be4062df0 con 0x7f0be41056d0 2024-09-10T08:46:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:36 smithi191 ceph-mon[35314]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:46:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:36 smithi172 ceph-mon[30854]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:46:36.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:36 smithi174 ceph-mon[34068]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:46:37.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.250+0000 7f0be27fc640 1 -- 172.21.15.172:0/1724421556 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7f0bd409c0f0 con 0x7f0be41056d0 2024-09-10T08:46:37.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.253+0000 7f0beba42640 1 -- 172.21.15.172:0/1724421556 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0bcc0a2c30 msgr2=0x7f0bcc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:37.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.253+0000 7f0beba42640 1 --2- 172.21.15.172:0/1724421556 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0bcc0a2c30 0x7f0bcc0a50f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009b40 tx=0x7f0bdc005e50 comp rx=0 tx=0).stop 2024-09-10T08:46:37.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.253+0000 7f0beba42640 1 -- 172.21.15.172:0/1724421556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0be41056d0 msgr2=0x7f0be41a58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:37.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.253+0000 7f0beba42640 1 --2- 172.21.15.172:0/1724421556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0be41056d0 0x7f0be41a58b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400cad0 tx=0x7f0bd400cf90 comp rx=0 tx=0).stop 2024-09-10T08:46:37.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.253+0000 7f0beba42640 1 -- 172.21.15.172:0/1724421556 shutdown_connections 2024-09-10T08:46:37.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.253+0000 7f0beba42640 1 --2- 172.21.15.172:0/1724421556 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0bcc0a2c30 0x7f0bcc0a50f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.253+0000 7f0beba42640 1 --2- 172.21.15.172:0/1724421556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0be41056d0 0x7f0be41a58b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.253+0000 7f0beba42640 1 --2- 172.21.15.172:0/1724421556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0be4102d20 0x7f0be419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.253+0000 7f0beba42640 1 --2- 172.21.15.172:0/1724421556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0be41003d0 0x7f0be419e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.253+0000 7f0beba42640 1 -- 172.21.15.172:0/1724421556 >> 172.21.15.172:0/1724421556 conn(0x7f0be40fc1e0 msgr2=0x7f0be4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:37.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.253+0000 7f0beba42640 1 -- 172.21.15.172:0/1724421556 shutdown_connections 2024-09-10T08:46:37.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.253+0000 7f0beba42640 1 -- 172.21.15.172:0/1724421556 wait complete. 2024-09-10T08:46:37.267 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-10T08:46:37.278 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-10T08:46:37.292 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T08:46:37.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.397+0000 7f0423d03640 1 -- 172.21.15.172:0/1972767219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f041c104f40 msgr2=0x7f041c107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:37.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.397+0000 7f0423d03640 1 --2- 172.21.15.172:0/1972767219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f041c104f40 0x7f041c107330 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f040c009900 tx=0x7f040c02a410 comp rx=0 tx=0).stop 2024-09-10T08:46:37.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.397+0000 7f0423d03640 1 -- 172.21.15.172:0/1972767219 shutdown_connections 2024-09-10T08:46:37.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.397+0000 7f0423d03640 1 --2- 172.21.15.172:0/1972767219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f041c107870 0x7f041c109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.397+0000 7f0423d03640 1 --2- 172.21.15.172:0/1972767219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f041c104f40 0x7f041c107330 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.397+0000 7f0423d03640 1 --2- 172.21.15.172:0/1972767219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f041c102610 0x7f041c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.397+0000 7f0423d03640 1 -- 172.21.15.172:0/1972767219 >> 172.21.15.172:0/1972767219 conn(0x7f041c0fc1b0 msgr2=0x7f041c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:37.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.398+0000 7f0423d03640 1 -- 172.21.15.172:0/1972767219 shutdown_connections 2024-09-10T08:46:37.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.398+0000 7f0423d03640 1 -- 172.21.15.172:0/1972767219 wait complete. 2024-09-10T08:46:37.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.398+0000 7f0423d03640 1 Processor -- start 2024-09-10T08:46:37.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.398+0000 7f0423d03640 1 -- start start 2024-09-10T08:46:37.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.398+0000 7f0423d03640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f041c102610 0x7f041c197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:37.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.398+0000 7f0423d03640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f041c104f40 0x7f041c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:37.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.398+0000 7f0423d03640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f041c107870 0x7f041c19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:37.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.398+0000 7f0423d03640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f041c10d6e0 con 0x7f041c104f40 2024-09-10T08:46:37.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.398+0000 7f0423d03640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f041c10d560 con 0x7f041c107870 2024-09-10T08:46:37.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.398+0000 7f0423d03640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f041c10d860 con 0x7f041c102610 2024-09-10T08:46:37.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.398+0000 7f042129e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f041c104f40 0x7f041c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:37.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.398+0000 7f042129e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f041c104f40 0x7f041c198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33158/0 (socket says 172.21.15.172:33158) 2024-09-10T08:46:37.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.398+0000 7f042129e640 1 -- 172.21.15.172:0/1753712744 learned_addr learned my addr 172.21.15.172:0/1753712744 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:37.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.399+0000 7f0421a9f640 1 --2- 172.21.15.172:0/1753712744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f041c102610 0x7f041c197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:37.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.399+0000 7f042129e640 1 -- 172.21.15.172:0/1753712744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f041c102610 msgr2=0x7f041c197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:37.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.399+0000 7f042129e640 1 --2- 172.21.15.172:0/1753712744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f041c102610 0x7f041c197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.399+0000 7f042129e640 1 -- 172.21.15.172:0/1753712744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f041c107870 msgr2=0x7f041c19f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:37.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.399+0000 7f042129e640 1 --2- 172.21.15.172:0/1753712744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f041c107870 0x7f041c19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.399+0000 7f042129e640 1 -- 172.21.15.172:0/1753712744 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c19fa40 con 0x7f041c104f40 2024-09-10T08:46:37.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.399+0000 7f042129e640 1 --2- 172.21.15.172:0/1753712744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f041c104f40 0x7f041c198360 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f040c0314e0 tx=0x7f040c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:37.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.399+0000 7f0412ffd640 1 -- 172.21.15.172:0/1753712744 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c03f070 con 0x7f041c104f40 2024-09-10T08:46:37.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.399+0000 7f0423d03640 1 -- 172.21.15.172:0/1753712744 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041c19fcd0 con 0x7f041c104f40 2024-09-10T08:46:37.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.399+0000 7f0412ffd640 1 -- 172.21.15.172:0/1753712744 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f040c031e40 con 0x7f041c104f40 2024-09-10T08:46:37.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.399+0000 7f0412ffd640 1 -- 172.21.15.172:0/1753712744 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c035030 con 0x7f041c104f40 2024-09-10T08:46:37.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.399+0000 7f0423d03640 1 -- 172.21.15.172:0/1753712744 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041c1a01c0 con 0x7f041c104f40 2024-09-10T08:46:37.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.400+0000 7f0423d03640 1 -- 172.21.15.172:0/1753712744 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f041c0fdca0 con 0x7f041c104f40 2024-09-10T08:46:37.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.401+0000 7f0412ffd640 1 -- 172.21.15.172:0/1753712744 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f040c042070 con 0x7f041c104f40 2024-09-10T08:46:37.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.402+0000 7f0412ffd640 1 --2- 172.21.15.172:0/1753712744 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04080a2b90 0x7f04080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:37.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.402+0000 7f0421a9f640 1 --2- 172.21.15.172:0/1753712744 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04080a2b90 0x7f04080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:37.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.403+0000 7f0412ffd640 1 -- 172.21.15.172:0/1753712744 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f040c0f02b0 con 0x7f041c104f40 2024-09-10T08:46:37.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.403+0000 7f0421a9f640 1 --2- 172.21.15.172:0/1753712744 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04080a2b90 0x7f04080a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0418005fd0 tx=0x7f0418005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:37.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.406+0000 7f0412ffd640 1 -- 172.21.15.172:0/1753712744 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f040c0bce50 con 0x7f041c104f40 2024-09-10T08:46:37.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:37 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1724421556' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-10T08:46:37.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.572+0000 7f0423d03640 1 -- 172.21.15.172:0/1753712744 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f041c02d9c0 con 0x7f041c104f40 2024-09-10T08:46:37.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.573+0000 7f0412ffd640 1 -- 172.21.15.172:0/1753712744 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24778 (secure 0 0 0) 0x7f040c0bc750 con 0x7f041c104f40 2024-09-10T08:46:37.574 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:37.574 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":92,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","created":"2024-09-10T08:36:33.083529+0000","modified":"2024-09-10T08:46:18.799844+0000","last_up_change":"2024-09-10T08:43:19.615640+0000","last_in_change":"2024-09-10T08:42:56.656115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:39:31.328306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:44:07.466637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:44:09.520569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:44:15.926057+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6803","nonce":2678426952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6805","nonce":2678426952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6809","nonce":2678426952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6807","nonce":2678426952}]},"public_addr":"172.21.15.172:6803/2678426952","cluster_addr":"172.21.15.172:6805/2678426952","heartbeat_back_addr":"172.21.15.172:6809/2678426952","heartbeat_front_addr":"172.21.15.172:6807/2678426952","state":["exists","up"]},{"osd":1,"uuid":"55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6801","nonce":3279661439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6803","nonce":3279661439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6807","nonce":3279661439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6805","nonce":3279661439}]},"public_addr":"172.21.15.174:6801/3279661439","cluster_addr":"172.21.15.174:6803/3279661439","heartbeat_back_addr":"172.21.15.174:6807/3279661439","heartbeat_front_addr":"172.21.15.174:6805/3279661439","state":["exists","up"]},{"osd":2,"uuid":"f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6801","nonce":1963796290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6803","nonce":1963796290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6807","nonce":1963796290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6805","nonce":1963796290}]},"public_addr":"172.21.15.191:6801/1963796290","cluster_addr":"172.21.15.191:6803/1963796290","heartbeat_back_addr":"172.21.15.191:6807/1963796290","heartbeat_front_addr":"172.21.15.191:6805/1963796290","state":["exists","up"]},{"osd":3,"uuid":"15ecca3f-bc6b-4a06-b283-b515384beb19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6811","nonce":869546480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6813","nonce":869546480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6817","nonce":869546480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6815","nonce":869546480}]},"public_addr":"172.21.15.172:6811/869546480","cluster_addr":"172.21.15.172:6813/869546480","heartbeat_back_addr":"172.21.15.172:6817/869546480","heartbeat_front_addr":"172.21.15.172:6815/869546480","state":["exists","up"]},{"osd":4,"uuid":"8c2c1d40-72ad-4a15-ab72-54871c7e2427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6809","nonce":961007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6811","nonce":961007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6815","nonce":961007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6813","nonce":961007949}]},"public_addr":"172.21.15.174:6809/961007949","cluster_addr":"172.21.15.174:6811/961007949","heartbeat_back_addr":"172.21.15.174:6815/961007949","heartbeat_front_addr":"172.21.15.174:6813/961007949","state":["exists","up"]},{"osd":5,"uuid":"4de45a1d-ef0e-4bfe-ba15-5eeeb8806155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6809","nonce":4216938371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6811","nonce":4216938371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6815","nonce":4216938371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6813","nonce":4216938371}]},"public_addr":"172.21.15.191:6809/4216938371","cluster_addr":"172.21.15.191:6811/4216938371","heartbeat_back_addr":"172.21.15.191:6815/4216938371","heartbeat_front_addr":"172.21.15.191:6813/4216938371","state":["exists","up"]},{"osd":6,"uuid":"60a9938a-f696-4db2-932c-402997543b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6819","nonce":3210762009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6821","nonce":3210762009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6825","nonce":3210762009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6823","nonce":3210762009}]},"public_addr":"172.21.15.172:6819/3210762009","cluster_addr":"172.21.15.172:6821/3210762009","heartbeat_back_addr":"172.21.15.172:6825/3210762009","heartbeat_front_addr":"172.21.15.172:6823/3210762009","state":["exists","up"]},{"osd":7,"uuid":"7f0202f3-971e-472f-9c4d-ae3c24d39f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6817","nonce":1461923581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6819","nonce":1461923581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6823","nonce":1461923581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6821","nonce":1461923581}]},"public_addr":"172.21.15.174:6817/1461923581","cluster_addr":"172.21.15.174:6819/1461923581","heartbeat_back_addr":"172.21.15.174:6823/1461923581","heartbeat_front_addr":"172.21.15.174:6821/1461923581","state":["exists","up"]},{"osd":8,"uuid":"b4e7327f-ca9f-4833-9cd3-b8529b3d9761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6817","nonce":3675210430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6819","nonce":3675210430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6823","nonce":3675210430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6821","nonce":3675210430}]},"public_addr":"172.21.15.191:6817/3675210430","cluster_addr":"172.21.15.191:6819/3675210430","heartbeat_back_addr":"172.21.15.191:6823/3675210430","heartbeat_front_addr":"172.21.15.191:6821/3675210430","state":["exists","up"]},{"osd":9,"uuid":"9f6153f2-f9db-4c00-a659-c932628016f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6827","nonce":1181268619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6829","nonce":1181268619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6833","nonce":1181268619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6831","nonce":1181268619}]},"public_addr":"172.21.15.172:6827/1181268619","cluster_addr":"172.21.15.172:6829/1181268619","heartbeat_back_addr":"172.21.15.172:6833/1181268619","heartbeat_front_addr":"172.21.15.172:6831/1181268619","state":["exists","up"]},{"osd":10,"uuid":"040a013a-4c3e-442f-bea6-12cbe517716d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6825","nonce":1238812525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6827","nonce":1238812525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6831","nonce":1238812525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6829","nonce":1238812525}]},"public_addr":"172.21.15.174:6825/1238812525","cluster_addr":"172.21.15.174:6827/1238812525","heartbeat_back_addr":"172.21.15.174:6831/1238812525","heartbeat_front_addr":"172.21.15.174:6829/1238812525","state":["exists","up"]},{"osd":11,"uuid":"ae6535bd-1510-4bf3-bf4b-1a2a256add43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6825","nonce":2162719997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6827","nonce":2162719997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6831","nonce":2162719997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6829","nonce":2162719997}]},"public_addr":"172.21.15.191:6825/2162719997","cluster_addr":"172.21.15.191:6827/2162719997","heartbeat_back_addr":"172.21.15.191:6831/2162719997","heartbeat_front_addr":"172.21.15.191:6829/2162719997","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:38:38.812596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:04.765652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:29.103966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:56.085849+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:21.641784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:47.750376+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:11.640697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:36.638840+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:02.167446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:28.655804+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:52.749676+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:43:17.783908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:0/977245917":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/3919952878":"2024-09-11T08:36:54.887256+0000","172.21.15.172:6800/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/1525385361":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/2043423408":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/839344299":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6800/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/3251817286":"2024-09-11T08:36:54.887256+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:46:37.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.575+0000 7f0423d03640 1 -- 172.21.15.172:0/1753712744 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04080a2b90 msgr2=0x7f04080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:37.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.575+0000 7f0423d03640 1 --2- 172.21.15.172:0/1753712744 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04080a2b90 0x7f04080a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0418005fd0 tx=0x7f0418005850 comp rx=0 tx=0).stop 2024-09-10T08:46:37.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.575+0000 7f0423d03640 1 -- 172.21.15.172:0/1753712744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f041c104f40 msgr2=0x7f041c198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:37.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.575+0000 7f0423d03640 1 --2- 172.21.15.172:0/1753712744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f041c104f40 0x7f041c198360 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f040c0314e0 tx=0x7f040c003c80 comp rx=0 tx=0).stop 2024-09-10T08:46:37.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.576+0000 7f0423d03640 1 -- 172.21.15.172:0/1753712744 shutdown_connections 2024-09-10T08:46:37.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.576+0000 7f0423d03640 1 --2- 172.21.15.172:0/1753712744 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04080a2b90 0x7f04080a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.576+0000 7f0423d03640 1 --2- 172.21.15.172:0/1753712744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f041c107870 0x7f041c19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.576+0000 7f0423d03640 1 --2- 172.21.15.172:0/1753712744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f041c104f40 0x7f041c198360 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.576+0000 7f0423d03640 1 --2- 172.21.15.172:0/1753712744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f041c102610 0x7f041c197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.576+0000 7f0423d03640 1 -- 172.21.15.172:0/1753712744 >> 172.21.15.172:0/1753712744 conn(0x7f041c0fc1b0 msgr2=0x7f041c103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:37.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.576+0000 7f0423d03640 1 -- 172.21.15.172:0/1753712744 shutdown_connections 2024-09-10T08:46:37.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.576+0000 7f0423d03640 1 -- 172.21.15.172:0/1753712744 wait complete. 2024-09-10T08:46:37.581 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:37 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1724421556' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-10T08:46:37.587 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T08:39:31.328306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T08:44:07.466637+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T08:44:09.520569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T08:44:15.926057+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T08:46:37.588 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T08:46:37.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.691+0000 7ff264fc4640 1 -- 172.21.15.172:0/4064240526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2601077f0 msgr2=0x7ff260109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:37.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.691+0000 7ff264fc4640 1 --2- 172.21.15.172:0/4064240526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2601077f0 0x7ff260109c60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff258009ca0 tx=0x7ff25802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.691+0000 7ff264fc4640 1 -- 172.21.15.172:0/4064240526 shutdown_connections 2024-09-10T08:46:37.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.691+0000 7ff264fc4640 1 --2- 172.21.15.172:0/4064240526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2601077f0 0x7ff260109c60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.691+0000 7ff264fc4640 1 --2- 172.21.15.172:0/4064240526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff260104ec0 0x7ff2601072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.691+0000 7ff264fc4640 1 --2- 172.21.15.172:0/4064240526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff260102590 0x7ff260104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.691+0000 7ff264fc4640 1 -- 172.21.15.172:0/4064240526 >> 172.21.15.172:0/4064240526 conn(0x7ff2600fc1b0 msgr2=0x7ff2600fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:37.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.691+0000 7ff264fc4640 1 -- 172.21.15.172:0/4064240526 shutdown_connections 2024-09-10T08:46:37.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.691+0000 7ff264fc4640 1 -- 172.21.15.172:0/4064240526 wait complete. 2024-09-10T08:46:37.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.692+0000 7ff264fc4640 1 Processor -- start 2024-09-10T08:46:37.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.692+0000 7ff264fc4640 1 -- start start 2024-09-10T08:46:37.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.692+0000 7ff264fc4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff260102590 0x7ff260197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:37.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.692+0000 7ff264fc4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff260104ec0 0x7ff260198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:37.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.692+0000 7ff264fc4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2601077f0 0x7ff26019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:37.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.692+0000 7ff264fc4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff26010d680 con 0x7ff260102590 2024-09-10T08:46:37.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.692+0000 7ff264fc4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff26010d500 con 0x7ff260104ec0 2024-09-10T08:46:37.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.692+0000 7ff264fc4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff26010d800 con 0x7ff2601077f0 2024-09-10T08:46:37.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.692+0000 7ff25e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff260102590 0x7ff260197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:37.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.692+0000 7ff25e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff260102590 0x7ff260197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33182/0 (socket says 172.21.15.172:33182) 2024-09-10T08:46:37.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.692+0000 7ff25e59c640 1 -- 172.21.15.172:0/126867871 learned_addr learned my addr 172.21.15.172:0/126867871 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:37.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.692+0000 7ff25ed9d640 1 --2- 172.21.15.172:0/126867871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2601077f0 0x7ff26019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:37.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.692+0000 7ff25dd9b640 1 --2- 172.21.15.172:0/126867871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff260104ec0 0x7ff260198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:37.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.693+0000 7ff25e59c640 1 -- 172.21.15.172:0/126867871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2601077f0 msgr2=0x7ff26019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:37.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.693+0000 7ff25e59c640 1 --2- 172.21.15.172:0/126867871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2601077f0 0x7ff26019f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.693+0000 7ff25e59c640 1 -- 172.21.15.172:0/126867871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff260104ec0 msgr2=0x7ff260198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:37.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.693+0000 7ff25e59c640 1 --2- 172.21.15.172:0/126867871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff260104ec0 0x7ff260198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.693+0000 7ff25e59c640 1 -- 172.21.15.172:0/126867871 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26019f9f0 con 0x7ff260102590 2024-09-10T08:46:37.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.693+0000 7ff25dd9b640 1 --2- 172.21.15.172:0/126867871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff260104ec0 0x7ff260198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:37.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.693+0000 7ff25e59c640 1 --2- 172.21.15.172:0/126867871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff260102590 0x7ff260197dd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff25000c970 tx=0x7ff25000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:37.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.693+0000 7ff24f7fe640 1 -- 172.21.15.172:0/126867871 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250007bb0 con 0x7ff260102590 2024-09-10T08:46:37.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.693+0000 7ff24f7fe640 1 -- 172.21.15.172:0/126867871 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff250007d50 con 0x7ff260102590 2024-09-10T08:46:37.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.693+0000 7ff264fc4640 1 -- 172.21.15.172:0/126867871 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26019fce0 con 0x7ff260102590 2024-09-10T08:46:37.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.693+0000 7ff24f7fe640 1 -- 172.21.15.172:0/126867871 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25000f450 con 0x7ff260102590 2024-09-10T08:46:37.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.693+0000 7ff264fc4640 1 -- 172.21.15.172:0/126867871 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2601a0180 con 0x7ff260102590 2024-09-10T08:46:37.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.694+0000 7ff264fc4640 1 -- 172.21.15.172:0/126867871 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2600fdca0 con 0x7ff260102590 2024-09-10T08:46:37.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.696+0000 7ff24f7fe640 1 -- 172.21.15.172:0/126867871 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff250004f00 con 0x7ff260102590 2024-09-10T08:46:37.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.696+0000 7ff24f7fe640 1 --2- 172.21.15.172:0/126867871 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2480a2b90 0x7ff2480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:37.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.697+0000 7ff25dd9b640 1 --2- 172.21.15.172:0/126867871 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2480a2b90 0x7ff2480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:37.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.697+0000 7ff24f7fe640 1 -- 172.21.15.172:0/126867871 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff2500cf5a0 con 0x7ff260102590 2024-09-10T08:46:37.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.697+0000 7ff25dd9b640 1 --2- 172.21.15.172:0/126867871 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2480a2b90 0x7ff2480a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff2601992e0 tx=0x7ff254005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:37.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.700+0000 7ff24f7fe640 1 -- 172.21.15.172:0/126867871 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff250014030 con 0x7ff260102590 2024-09-10T08:46:37.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:37 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1724421556' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-10T08:46:37.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.867+0000 7ff264fc4640 1 -- 172.21.15.172:0/126867871 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff260062ef0 con 0x7ff260102590 2024-09-10T08:46:37.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.867+0000 7ff24f7fe640 1 -- 172.21.15.172:0/126867871 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7ff25009c1a0 con 0x7ff260102590 2024-09-10T08:46:37.868 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-09-10T08:46:37.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.869+0000 7ff264fc4640 1 -- 172.21.15.172:0/126867871 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2480a2b90 msgr2=0x7ff2480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:37.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.869+0000 7ff264fc4640 1 --2- 172.21.15.172:0/126867871 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2480a2b90 0x7ff2480a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff2601992e0 tx=0x7ff254005f90 comp rx=0 tx=0).stop 2024-09-10T08:46:37.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.869+0000 7ff264fc4640 1 -- 172.21.15.172:0/126867871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff260102590 msgr2=0x7ff260197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:37.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.869+0000 7ff264fc4640 1 --2- 172.21.15.172:0/126867871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff260102590 0x7ff260197dd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff25000c970 tx=0x7ff25000ce30 comp rx=0 tx=0).stop 2024-09-10T08:46:37.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.869+0000 7ff264fc4640 1 -- 172.21.15.172:0/126867871 shutdown_connections 2024-09-10T08:46:37.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.869+0000 7ff264fc4640 1 --2- 172.21.15.172:0/126867871 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2480a2b90 0x7ff2480a5050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.869+0000 7ff264fc4640 1 --2- 172.21.15.172:0/126867871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2601077f0 0x7ff26019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.869+0000 7ff264fc4640 1 --2- 172.21.15.172:0/126867871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff260104ec0 0x7ff260198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.869+0000 7ff264fc4640 1 --2- 172.21.15.172:0/126867871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff260102590 0x7ff260197dd0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.870+0000 7ff264fc4640 1 -- 172.21.15.172:0/126867871 >> 172.21.15.172:0/126867871 conn(0x7ff2600fc1b0 msgr2=0x7ff260102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:37.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.870+0000 7ff264fc4640 1 -- 172.21.15.172:0/126867871 shutdown_connections 2024-09-10T08:46:37.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.870+0000 7ff264fc4640 1 -- 172.21.15.172:0/126867871 wait complete. 2024-09-10T08:46:37.881 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T08:46:37.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.985+0000 7fc5bdc45640 1 -- 172.21.15.172:0/2116121281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b8107850 msgr2=0x7fc5b8109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:37.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.985+0000 7fc5bdc45640 1 --2- 172.21.15.172:0/2116121281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b8107850 0x7fc5b8109c60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009ca0 tx=0x7fc5b002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.985+0000 7fc5bdc45640 1 -- 172.21.15.172:0/2116121281 shutdown_connections 2024-09-10T08:46:37.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.985+0000 7fc5bdc45640 1 --2- 172.21.15.172:0/2116121281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b8107850 0x7fc5b8109c60 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.985+0000 7fc5bdc45640 1 --2- 172.21.15.172:0/2116121281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5b8104f20 0x7fc5b8107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.985+0000 7fc5bdc45640 1 --2- 172.21.15.172:0/2116121281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5b81025f0 0x7fc5b81049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.985+0000 7fc5bdc45640 1 -- 172.21.15.172:0/2116121281 >> 172.21.15.172:0/2116121281 conn(0x7fc5b80fc190 msgr2=0x7fc5b80fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:37.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.985+0000 7fc5bdc45640 1 -- 172.21.15.172:0/2116121281 shutdown_connections 2024-09-10T08:46:37.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.985+0000 7fc5bdc45640 1 -- 172.21.15.172:0/2116121281 wait complete. 2024-09-10T08:46:37.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5bdc45640 1 Processor -- start 2024-09-10T08:46:37.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5bdc45640 1 -- start start 2024-09-10T08:46:37.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5bdc45640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5b81025f0 0x7fc5b8101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:37.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5bdc45640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b8104f20 0x7fc5b81005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:37.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5bdc45640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5b8107850 0x7fc5b8100ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:37.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5bdc45640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b810d6e0 con 0x7fc5b8104f20 2024-09-10T08:46:37.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5bdc45640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b810d560 con 0x7fc5b8107850 2024-09-10T08:46:37.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5bdc45640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b810d860 con 0x7fc5b81025f0 2024-09-10T08:46:37.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5b6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b8104f20 0x7fc5b81005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:37.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5b6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b8104f20 0x7fc5b81005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33192/0 (socket says 172.21.15.172:33192) 2024-09-10T08:46:37.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5b6ffd640 1 -- 172.21.15.172:0/2248482049 learned_addr learned my addr 172.21.15.172:0/2248482049 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:37.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5b77fe640 1 --2- 172.21.15.172:0/2248482049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5b81025f0 0x7fc5b8101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:37.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5b7fff640 1 --2- 172.21.15.172:0/2248482049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5b8107850 0x7fc5b8100ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:37.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5b6ffd640 1 -- 172.21.15.172:0/2248482049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5b81025f0 msgr2=0x7fc5b8101f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:37.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5b6ffd640 1 --2- 172.21.15.172:0/2248482049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5b81025f0 0x7fc5b8101f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5b6ffd640 1 -- 172.21.15.172:0/2248482049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5b8107850 msgr2=0x7fc5b8100ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:37.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5b6ffd640 1 --2- 172.21.15.172:0/2248482049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5b8107850 0x7fc5b8100ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:37.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5b6ffd640 1 -- 172.21.15.172:0/2248482049 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b8101370 con 0x7fc5b8104f20 2024-09-10T08:46:37.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5b77fe640 1 --2- 172.21.15.172:0/2248482049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5b81025f0 0x7fc5b8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:37.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.986+0000 7fc5b7fff640 1 --2- 172.21.15.172:0/2248482049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5b8107850 0x7fc5b8100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:37.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.987+0000 7fc5b6ffd640 1 --2- 172.21.15.172:0/2248482049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b8104f20 0x7fc5b81005a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00c970 tx=0x7fc5ac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:37.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.987+0000 7fc5b4ff9640 1 -- 172.21.15.172:0/2248482049 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac007bb0 con 0x7fc5b8104f20 2024-09-10T08:46:37.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.987+0000 7fc5b4ff9640 1 -- 172.21.15.172:0/2248482049 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc5ac007d50 con 0x7fc5b8104f20 2024-09-10T08:46:37.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.987+0000 7fc5bdc45640 1 -- 172.21.15.172:0/2248482049 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b81a1f60 con 0x7fc5b8104f20 2024-09-10T08:46:37.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.987+0000 7fc5b4ff9640 1 -- 172.21.15.172:0/2248482049 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac00f450 con 0x7fc5b8104f20 2024-09-10T08:46:37.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.987+0000 7fc5bdc45640 1 -- 172.21.15.172:0/2248482049 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b81a23c0 con 0x7fc5b8104f20 2024-09-10T08:46:37.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.988+0000 7fc5bdc45640 1 -- 172.21.15.172:0/2248482049 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b80fdc80 con 0x7fc5b8104f20 2024-09-10T08:46:37.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.989+0000 7fc5b4ff9640 1 -- 172.21.15.172:0/2248482049 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc5ac004eb0 con 0x7fc5b8104f20 2024-09-10T08:46:37.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.990+0000 7fc5b4ff9640 1 --2- 172.21.15.172:0/2248482049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5a00a2b90 0x7fc5a00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:37.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.990+0000 7fc5b77fe640 1 --2- 172.21.15.172:0/2248482049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5a00a2b90 0x7fc5a00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:37.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.990+0000 7fc5b4ff9640 1 -- 172.21.15.172:0/2248482049 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fc5ac0cf100 con 0x7fc5b8104f20 2024-09-10T08:46:37.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.990+0000 7fc5b77fe640 1 --2- 172.21.15.172:0/2248482049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5a00a2b90 0x7fc5a00a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8006fd0 tx=0x7fc5a8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:37.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:37.994+0000 7fc5b4ff9640 1 -- 172.21.15.172:0/2248482049 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5ac014030 con 0x7fc5b8104f20 2024-09-10T08:46:38.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.161+0000 7fc5bdc45640 1 -- 172.21.15.172:0/2248482049 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc5b8062f30 con 0x7fc5b8104f20 2024-09-10T08:46:38.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.162+0000 7fc5b4ff9640 1 -- 172.21.15.172:0/2248482049 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fc5ac09bd00 con 0x7fc5b8104f20 2024-09-10T08:46:38.162 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-09-10T08:46:38.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.164+0000 7fc5bdc45640 1 -- 172.21.15.172:0/2248482049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5a00a2b90 msgr2=0x7fc5a00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:38.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.164+0000 7fc5bdc45640 1 --2- 172.21.15.172:0/2248482049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5a00a2b90 0x7fc5a00a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8006fd0 tx=0x7fc5a8008040 comp rx=0 tx=0).stop 2024-09-10T08:46:38.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.164+0000 7fc5bdc45640 1 -- 172.21.15.172:0/2248482049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b8104f20 msgr2=0x7fc5b81005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:38.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.164+0000 7fc5bdc45640 1 --2- 172.21.15.172:0/2248482049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b8104f20 0x7fc5b81005a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00c970 tx=0x7fc5ac00ce30 comp rx=0 tx=0).stop 2024-09-10T08:46:38.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.164+0000 7fc5bdc45640 1 -- 172.21.15.172:0/2248482049 shutdown_connections 2024-09-10T08:46:38.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.164+0000 7fc5bdc45640 1 --2- 172.21.15.172:0/2248482049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5a00a2b90 0x7fc5a00a5050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.164+0000 7fc5bdc45640 1 --2- 172.21.15.172:0/2248482049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5b8107850 0x7fc5b8100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.164+0000 7fc5bdc45640 1 --2- 172.21.15.172:0/2248482049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b8104f20 0x7fc5b81005a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.164+0000 7fc5bdc45640 1 --2- 172.21.15.172:0/2248482049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5b81025f0 0x7fc5b8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.164+0000 7fc5bdc45640 1 -- 172.21.15.172:0/2248482049 >> 172.21.15.172:0/2248482049 conn(0x7fc5b80fc190 msgr2=0x7fc5b8103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:38.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.164+0000 7fc5bdc45640 1 -- 172.21.15.172:0/2248482049 shutdown_connections 2024-09-10T08:46:38.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.164+0000 7fc5bdc45640 1 -- 172.21.15.172:0/2248482049 wait complete. 2024-09-10T08:46:38.176 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T08:46:38.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.280+0000 7f89a4581640 1 -- 172.21.15.172:0/3166264532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f899c1025b0 msgr2=0x7f899c108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:38.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.280+0000 7f89a4581640 1 --2- 172.21.15.172:0/3166264532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f899c1025b0 0x7f899c108e40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8994009ca0 tx=0x7f899402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.280+0000 7f89a4581640 1 -- 172.21.15.172:0/3166264532 shutdown_connections 2024-09-10T08:46:38.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.280+0000 7f89a4581640 1 --2- 172.21.15.172:0/3166264532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f899c1025b0 0x7f899c108e40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.280+0000 7f89a4581640 1 --2- 172.21.15.172:0/3166264532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f899c101bf0 0x7f899c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.280+0000 7f89a4581640 1 --2- 172.21.15.172:0/3166264532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f899c100a00 0x7f899c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.280+0000 7f89a4581640 1 -- 172.21.15.172:0/3166264532 >> 172.21.15.172:0/3166264532 conn(0x7f899c0fc180 msgr2=0x7f899c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:38.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.280+0000 7f89a4581640 1 -- 172.21.15.172:0/3166264532 shutdown_connections 2024-09-10T08:46:38.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.280+0000 7f89a4581640 1 -- 172.21.15.172:0/3166264532 wait complete. 2024-09-10T08:46:38.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.280+0000 7f89a4581640 1 Processor -- start 2024-09-10T08:46:38.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.280+0000 7f89a4581640 1 -- start start 2024-09-10T08:46:38.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a4581640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f899c100a00 0x7f899c06aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:38.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a4581640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f899c101bf0 0x7f899c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:38.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a4581640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f899c1025b0 0x7f899c19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:38.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a4581640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f899c10c530 con 0x7f899c101bf0 2024-09-10T08:46:38.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a4581640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f899c10c3b0 con 0x7f899c100a00 2024-09-10T08:46:38.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a4581640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f899c10c6b0 con 0x7f899c1025b0 2024-09-10T08:46:38.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a1b1c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f899c101bf0 0x7f899c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:38.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a1b1c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f899c101bf0 0x7f899c19cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33212/0 (socket says 172.21.15.172:33212) 2024-09-10T08:46:38.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a1b1c640 1 -- 172.21.15.172:0/1266478760 learned_addr learned my addr 172.21.15.172:0/1266478760 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:38.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a231d640 1 --2- 172.21.15.172:0/1266478760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f899c100a00 0x7f899c06aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:38.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a2b1e640 1 --2- 172.21.15.172:0/1266478760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f899c1025b0 0x7f899c19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:38.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a1b1c640 1 -- 172.21.15.172:0/1266478760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f899c1025b0 msgr2=0x7f899c19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:38.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a1b1c640 1 --2- 172.21.15.172:0/1266478760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f899c1025b0 0x7f899c19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a1b1c640 1 -- 172.21.15.172:0/1266478760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f899c100a00 msgr2=0x7f899c06aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:38.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a1b1c640 1 --2- 172.21.15.172:0/1266478760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f899c100a00 0x7f899c06aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a1b1c640 1 -- 172.21.15.172:0/1266478760 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899c19f9f0 con 0x7f899c101bf0 2024-09-10T08:46:38.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a2b1e640 1 --2- 172.21.15.172:0/1266478760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f899c1025b0 0x7f899c19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:38.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a1b1c640 1 --2- 172.21.15.172:0/1266478760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f899c101bf0 0x7f899c19cac0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f898c00e9f0 tx=0x7f898c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:38.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89937fe640 1 -- 172.21.15.172:0/1266478760 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c00cd80 con 0x7f899c101bf0 2024-09-10T08:46:38.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89937fe640 1 -- 172.21.15.172:0/1266478760 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f898c0108e0 con 0x7f899c101bf0 2024-09-10T08:46:38.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a4581640 1 -- 172.21.15.172:0/1266478760 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f899c19fc80 con 0x7f899c101bf0 2024-09-10T08:46:38.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89937fe640 1 -- 172.21.15.172:0/1266478760 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c018920 con 0x7f899c101bf0 2024-09-10T08:46:38.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.281+0000 7f89a4581640 1 -- 172.21.15.172:0/1266478760 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f899c1a01d0 con 0x7f899c101bf0 2024-09-10T08:46:38.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.282+0000 7f89a4581640 1 -- 172.21.15.172:0/1266478760 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f899c107e60 con 0x7f899c101bf0 2024-09-10T08:46:38.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.283+0000 7f89937fe640 1 -- 172.21.15.172:0/1266478760 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f898c010a90 con 0x7f899c101bf0 2024-09-10T08:46:38.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.284+0000 7f89937fe640 1 --2- 172.21.15.172:0/1266478760 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f89880a2b90 0x7f89880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:38.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.284+0000 7f89a231d640 1 --2- 172.21.15.172:0/1266478760 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f89880a2b90 0x7f89880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:38.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.284+0000 7f89937fe640 1 -- 172.21.15.172:0/1266478760 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f898c0cf720 con 0x7f899c101bf0 2024-09-10T08:46:38.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.285+0000 7f89a231d640 1 --2- 172.21.15.172:0/1266478760 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f89880a2b90 0x7f89880a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8998005fd0 tx=0x7f899800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:38.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.288+0000 7f89937fe640 1 -- 172.21.15.172:0/1266478760 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f898c09c320 con 0x7f899c101bf0 2024-09-10T08:46:38.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.458+0000 7f89a4581640 1 -- 172.21.15.172:0/1266478760 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f899c062e90 con 0x7f899c101bf0 2024-09-10T08:46:38.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.458+0000 7f89937fe640 1 -- 172.21.15.172:0/1266478760 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f898c0860a0 con 0x7f899c101bf0 2024-09-10T08:46:38.459 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-09-10T08:46:38.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.460+0000 7f89a4581640 1 -- 172.21.15.172:0/1266478760 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f89880a2b90 msgr2=0x7f89880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:38.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.460+0000 7f89a4581640 1 --2- 172.21.15.172:0/1266478760 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f89880a2b90 0x7f89880a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8998005fd0 tx=0x7f899800b390 comp rx=0 tx=0).stop 2024-09-10T08:46:38.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.460+0000 7f89a4581640 1 -- 172.21.15.172:0/1266478760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f899c101bf0 msgr2=0x7f899c19cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:38.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.460+0000 7f89a4581640 1 --2- 172.21.15.172:0/1266478760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f899c101bf0 0x7f899c19cac0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f898c00e9f0 tx=0x7f898c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.460+0000 7f89a4581640 1 -- 172.21.15.172:0/1266478760 shutdown_connections 2024-09-10T08:46:38.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.460+0000 7f89a4581640 1 --2- 172.21.15.172:0/1266478760 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f89880a2b90 0x7f89880a5050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.460+0000 7f89a4581640 1 --2- 172.21.15.172:0/1266478760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f899c1025b0 0x7f899c19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.460+0000 7f89a4581640 1 --2- 172.21.15.172:0/1266478760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f899c101bf0 0x7f899c19cac0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.460+0000 7f89a4581640 1 --2- 172.21.15.172:0/1266478760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f899c100a00 0x7f899c06aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.460+0000 7f89a4581640 1 -- 172.21.15.172:0/1266478760 >> 172.21.15.172:0/1266478760 conn(0x7f899c0fc180 msgr2=0x7f899c105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:38.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.460+0000 7f89a4581640 1 -- 172.21.15.172:0/1266478760 shutdown_connections 2024-09-10T08:46:38.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.460+0000 7f89a4581640 1 -- 172.21.15.172:0/1266478760 wait complete. 2024-09-10T08:46:38.471 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:38 smithi172 ceph-mon[30854]: pgmap v362: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T08:46:38.471 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:38 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1724421556' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T08:46:38.471 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:38 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:46:38.471 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:38 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1753712744' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:46:38.471 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:38 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/126867871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:46:38.471 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:38 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2248482049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T08:46:38.472 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T08:46:38.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:38 smithi191 ceph-mon[35314]: pgmap v362: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T08:46:38.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:38 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1724421556' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T08:46:38.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:38 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:46:38.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:38 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1753712744' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:46:38.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:38 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/126867871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:46:38.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:38 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2248482049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T08:46:38.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.576+0000 7f8a79d4b640 1 -- 172.21.15.172:0/425098571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a741025c0 msgr2=0x7f8a74108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:38.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.576+0000 7f8a79d4b640 1 --2- 172.21.15.172:0/425098571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a741025c0 0x7f8a74108e50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009900 tx=0x7f8a6c02a410 comp rx=0 tx=0).stop 2024-09-10T08:46:38.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.576+0000 7f8a79d4b640 1 -- 172.21.15.172:0/425098571 shutdown_connections 2024-09-10T08:46:38.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.576+0000 7f8a79d4b640 1 --2- 172.21.15.172:0/425098571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a741025c0 0x7f8a74108e50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.576+0000 7f8a79d4b640 1 --2- 172.21.15.172:0/425098571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74101c00 0x7f8a74102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.576+0000 7f8a79d4b640 1 --2- 172.21.15.172:0/425098571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a74100a10 0x7f8a74100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.576+0000 7f8a79d4b640 1 -- 172.21.15.172:0/425098571 >> 172.21.15.172:0/425098571 conn(0x7f8a740fc1b0 msgr2=0x7f8a740fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:38.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.576+0000 7f8a79d4b640 1 -- 172.21.15.172:0/425098571 shutdown_connections 2024-09-10T08:46:38.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.576+0000 7f8a79d4b640 1 -- 172.21.15.172:0/425098571 wait complete. 2024-09-10T08:46:38.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.576+0000 7f8a79d4b640 1 Processor -- start 2024-09-10T08:46:38.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.576+0000 7f8a79d4b640 1 -- start start 2024-09-10T08:46:38.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a79d4b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74100a10 0x7f8a7406af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:38.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a79d4b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a74101c00 0x7f8a7419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:38.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a79d4b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a741025c0 0x7f8a7419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:38.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a79d4b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7410c5a0 con 0x7f8a74101c00 2024-09-10T08:46:38.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a79d4b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7410c420 con 0x7f8a741025c0 2024-09-10T08:46:38.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a79d4b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7410c720 con 0x7f8a74100a10 2024-09-10T08:46:38.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a72ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a74101c00 0x7f8a7419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:38.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a72ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a74101c00 0x7f8a7419cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33228/0 (socket says 172.21.15.172:33228) 2024-09-10T08:46:38.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a737fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74100a10 0x7f8a7406af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:38.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a72ffd640 1 -- 172.21.15.172:0/271933058 learned_addr learned my addr 172.21.15.172:0/271933058 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:38.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a73fff640 1 --2- 172.21.15.172:0/271933058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a741025c0 0x7f8a7419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:38.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a72ffd640 1 -- 172.21.15.172:0/271933058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74100a10 msgr2=0x7f8a7406af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:38.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a72ffd640 1 --2- 172.21.15.172:0/271933058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74100a10 0x7f8a7406af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a72ffd640 1 -- 172.21.15.172:0/271933058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a741025c0 msgr2=0x7f8a7419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:38.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a72ffd640 1 --2- 172.21.15.172:0/271933058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a741025c0 0x7f8a7419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a72ffd640 1 -- 172.21.15.172:0/271933058 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7419fa10 con 0x7f8a74101c00 2024-09-10T08:46:38.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a72ffd640 1 --2- 172.21.15.172:0/271933058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a74101c00 0x7f8a7419cae0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800b4a0 tx=0x7f8a6800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:38.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a70ff9640 1 -- 172.21.15.172:0/271933058 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68014070 con 0x7f8a74101c00 2024-09-10T08:46:38.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a70ff9640 1 -- 172.21.15.172:0/271933058 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a68010ac0 con 0x7f8a74101c00 2024-09-10T08:46:38.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a79d4b640 1 -- 172.21.15.172:0/271933058 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7419fca0 con 0x7f8a74101c00 2024-09-10T08:46:38.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a70ff9640 1 -- 172.21.15.172:0/271933058 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68017c30 con 0x7f8a74101c00 2024-09-10T08:46:38.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.577+0000 7f8a79d4b640 1 -- 172.21.15.172:0/271933058 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a741a0220 con 0x7f8a74101c00 2024-09-10T08:46:38.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.578+0000 7f8a79d4b640 1 -- 172.21.15.172:0/271933058 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a74107e20 con 0x7f8a74101c00 2024-09-10T08:46:38.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.580+0000 7f8a70ff9640 1 -- 172.21.15.172:0/271933058 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8a68017070 con 0x7f8a74101c00 2024-09-10T08:46:38.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.580+0000 7f8a70ff9640 1 --2- 172.21.15.172:0/271933058 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8a5c0a2b90 0x7f8a5c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:38.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.580+0000 7f8a737fe640 1 --2- 172.21.15.172:0/271933058 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8a5c0a2b90 0x7f8a5c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:38.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.580+0000 7f8a70ff9640 1 -- 172.21.15.172:0/271933058 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8a680ce9c0 con 0x7f8a74101c00 2024-09-10T08:46:38.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.581+0000 7f8a737fe640 1 --2- 172.21.15.172:0/271933058 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8a5c0a2b90 0x7f8a5c0a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8a74101a60 tx=0x7f8a64009670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:38.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.584+0000 7f8a70ff9640 1 -- 172.21.15.172:0/271933058 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a680d0e20 con 0x7f8a74101c00 2024-09-10T08:46:38.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:38 smithi174 ceph-mon[34068]: pgmap v362: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T08:46:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:38 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1724421556' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T08:46:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:38 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:46:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:38 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1753712744' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:46:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:38 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/126867871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T08:46:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:38 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2248482049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T08:46:38.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.752+0000 7f8a79d4b640 1 -- 172.21.15.172:0/271933058 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8a74062df0 con 0x7f8a74101c00 2024-09-10T08:46:38.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.752+0000 7f8a70ff9640 1 -- 172.21.15.172:0/271933058 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f8a6809af70 con 0x7f8a74101c00 2024-09-10T08:46:38.753 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-09-10T08:46:38.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.755+0000 7f8a79d4b640 1 -- 172.21.15.172:0/271933058 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8a5c0a2b90 msgr2=0x7f8a5c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:38.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.755+0000 7f8a79d4b640 1 --2- 172.21.15.172:0/271933058 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8a5c0a2b90 0x7f8a5c0a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8a74101a60 tx=0x7f8a64009670 comp rx=0 tx=0).stop 2024-09-10T08:46:38.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.755+0000 7f8a79d4b640 1 -- 172.21.15.172:0/271933058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a74101c00 msgr2=0x7f8a7419cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:38.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.755+0000 7f8a79d4b640 1 --2- 172.21.15.172:0/271933058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a74101c00 0x7f8a7419cae0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8a6800b4a0 tx=0x7f8a6800b960 comp rx=0 tx=0).stop 2024-09-10T08:46:38.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.755+0000 7f8a79d4b640 1 -- 172.21.15.172:0/271933058 shutdown_connections 2024-09-10T08:46:38.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.755+0000 7f8a79d4b640 1 --2- 172.21.15.172:0/271933058 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8a5c0a2b90 0x7f8a5c0a5050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.755+0000 7f8a79d4b640 1 --2- 172.21.15.172:0/271933058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a741025c0 0x7f8a7419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.755+0000 7f8a79d4b640 1 --2- 172.21.15.172:0/271933058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a74101c00 0x7f8a7419cae0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.755+0000 7f8a79d4b640 1 --2- 172.21.15.172:0/271933058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74100a10 0x7f8a7406af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:38.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.755+0000 7f8a79d4b640 1 -- 172.21.15.172:0/271933058 >> 172.21.15.172:0/271933058 conn(0x7f8a740fc1b0 msgr2=0x7f8a74104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:38.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.755+0000 7f8a79d4b640 1 -- 172.21.15.172:0/271933058 shutdown_connections 2024-09-10T08:46:38.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:38.755+0000 7f8a79d4b640 1 -- 172.21.15.172:0/271933058 wait complete. 2024-09-10T08:46:38.766 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-10T08:46:38.766 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:46:39.013 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:39.414 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:39 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1266478760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T08:46:39.414 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:39 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/271933058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T08:46:39.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.493+0000 7fd4c8cc8640 1 -- 172.21.15.172:0/1566793623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4c4100e20 msgr2=0x7fd4c4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:39.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.493+0000 7fd4c8cc8640 1 --2- 172.21.15.172:0/1566793623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4c4100e20 0x7fd4c4101200 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009900 tx=0x7fd4ac02a480 comp rx=0 tx=0).stop 2024-09-10T08:46:39.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.493+0000 7fd4c8cc8640 1 -- 172.21.15.172:0/1566793623 shutdown_connections 2024-09-10T08:46:39.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.493+0000 7fd4c8cc8640 1 --2- 172.21.15.172:0/1566793623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4c4102ef0 0x7fd4c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:39.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.493+0000 7fd4c8cc8640 1 --2- 172.21.15.172:0/1566793623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c4102570 0x7fd4c41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:39.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.493+0000 7fd4c8cc8640 1 --2- 172.21.15.172:0/1566793623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4c4100e20 0x7fd4c4101200 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:39.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.493+0000 7fd4c8cc8640 1 -- 172.21.15.172:0/1566793623 >> 172.21.15.172:0/1566793623 conn(0x7fd4c40fc7d0 msgr2=0x7fd4c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:39.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.493+0000 7fd4c8cc8640 1 -- 172.21.15.172:0/1566793623 shutdown_connections 2024-09-10T08:46:39.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.493+0000 7fd4c8cc8640 1 -- 172.21.15.172:0/1566793623 wait complete. 2024-09-10T08:46:39.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c8cc8640 1 Processor -- start 2024-09-10T08:46:39.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c8cc8640 1 -- start start 2024-09-10T08:46:39.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c8cc8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4c4100e20 0x7fd4c4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:39.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c8cc8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c4102570 0x7fd4c4198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:39.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c8cc8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4c4102ef0 0x7fd4c419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:39.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c8cc8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c4110e00 con 0x7fd4c4102ef0 2024-09-10T08:46:39.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c8cc8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c4110c80 con 0x7fd4c4102570 2024-09-10T08:46:39.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c8cc8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c4110f80 con 0x7fd4c4100e20 2024-09-10T08:46:39.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4c4102ef0 0x7fd4c419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:39.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4c4102ef0 0x7fd4c419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33240/0 (socket says 172.21.15.172:33240) 2024-09-10T08:46:39.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c2d9d640 1 -- 172.21.15.172:0/17995658 learned_addr learned my addr 172.21.15.172:0/17995658 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:39.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c259c640 1 --2- 172.21.15.172:0/17995658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4c4100e20 0x7fd4c4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:39.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c1d9b640 1 --2- 172.21.15.172:0/17995658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c4102570 0x7fd4c4198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:39.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c2d9d640 1 -- 172.21.15.172:0/17995658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4c4100e20 msgr2=0x7fd4c4198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:39.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c2d9d640 1 --2- 172.21.15.172:0/17995658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4c4100e20 0x7fd4c4198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:39.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c2d9d640 1 -- 172.21.15.172:0/17995658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c4102570 msgr2=0x7fd4c4198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:39.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c2d9d640 1 --2- 172.21.15.172:0/17995658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c4102570 0x7fd4c4198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:39.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.494+0000 7fd4c2d9d640 1 -- 172.21.15.172:0/17995658 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c419d300 con 0x7fd4c4102ef0 2024-09-10T08:46:39.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.495+0000 7fd4c2d9d640 1 --2- 172.21.15.172:0/17995658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4c4102ef0 0x7fd4c419cc20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00eaa0 tx=0x7fd4bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:39.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.495+0000 7fd4bb7fe640 1 -- 172.21.15.172:0/17995658 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc00cdb0 con 0x7fd4c4102ef0 2024-09-10T08:46:39.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.495+0000 7fd4bb7fe640 1 -- 172.21.15.172:0/17995658 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4bc005020 con 0x7fd4c4102ef0 2024-09-10T08:46:39.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.495+0000 7fd4c8cc8640 1 -- 172.21.15.172:0/17995658 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c419d5f0 con 0x7fd4c4102ef0 2024-09-10T08:46:39.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.495+0000 7fd4bb7fe640 1 -- 172.21.15.172:0/17995658 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0116d0 con 0x7fd4c4102ef0 2024-09-10T08:46:39.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.495+0000 7fd4c8cc8640 1 -- 172.21.15.172:0/17995658 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c41a4ed0 con 0x7fd4c4102ef0 2024-09-10T08:46:39.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.496+0000 7fd4c8cc8640 1 -- 172.21.15.172:0/17995658 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c4101200 con 0x7fd4c4102ef0 2024-09-10T08:46:39.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.497+0000 7fd4bb7fe640 1 -- 172.21.15.172:0/17995658 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd4bc0051d0 con 0x7fd4c4102ef0 2024-09-10T08:46:39.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.498+0000 7fd4bb7fe640 1 --2- 172.21.15.172:0/17995658 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd4b00a2b80 0x7fd4b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:39.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.498+0000 7fd4c259c640 1 --2- 172.21.15.172:0/17995658 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd4b00a2b80 0x7fd4b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:39.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.498+0000 7fd4bb7fe640 1 -- 172.21.15.172:0/17995658 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd4bc0cf550 con 0x7fd4c4102ef0 2024-09-10T08:46:39.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.498+0000 7fd4c259c640 1 --2- 172.21.15.172:0/17995658 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd4b00a2b80 0x7fd4b00a5040 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009230 tx=0x7fd4ac0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:39.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.502+0000 7fd4bb7fe640 1 -- 172.21.15.172:0/17995658 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd4bc016030 con 0x7fd4c4102ef0 2024-09-10T08:46:39.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:39 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1266478760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T08:46:39.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:39 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/271933058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T08:46:39.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.703+0000 7fd4c8cc8640 1 -- 172.21.15.172:0/17995658 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4c41a5100 con 0x7fd4c4102ef0 2024-09-10T08:46:39.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.703+0000 7fd4bb7fe640 1 -- 172.21.15.172:0/17995658 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7fd4bc09c150 con 0x7fd4c4102ef0 2024-09-10T08:46:39.704 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:39.704 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":29,"btime":"2024-09-10T08:46:37:242916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:46:36.267985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:46:39.704 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 29 2024-09-10T08:46:39.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.706+0000 7fd4c8cc8640 1 -- 172.21.15.172:0/17995658 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd4b00a2b80 msgr2=0x7fd4b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:39.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.706+0000 7fd4c8cc8640 1 --2- 172.21.15.172:0/17995658 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd4b00a2b80 0x7fd4b00a5040 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009230 tx=0x7fd4ac0058d0 comp rx=0 tx=0).stop 2024-09-10T08:46:39.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.706+0000 7fd4c8cc8640 1 -- 172.21.15.172:0/17995658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4c4102ef0 msgr2=0x7fd4c419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:39.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.706+0000 7fd4c8cc8640 1 --2- 172.21.15.172:0/17995658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4c4102ef0 0x7fd4c419cc20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00eaa0 tx=0x7fd4bc00ef60 comp rx=0 tx=0).stop 2024-09-10T08:46:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.706+0000 7fd4c8cc8640 1 -- 172.21.15.172:0/17995658 shutdown_connections 2024-09-10T08:46:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.706+0000 7fd4c8cc8640 1 --2- 172.21.15.172:0/17995658 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd4b00a2b80 0x7fd4b00a5040 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.706+0000 7fd4c8cc8640 1 --2- 172.21.15.172:0/17995658 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4c4102ef0 0x7fd4c419cc20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.706+0000 7fd4c8cc8640 1 --2- 172.21.15.172:0/17995658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c4102570 0x7fd4c4198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.706+0000 7fd4c8cc8640 1 --2- 172.21.15.172:0/17995658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4c4100e20 0x7fd4c4198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.706+0000 7fd4c8cc8640 1 -- 172.21.15.172:0/17995658 >> 172.21.15.172:0/17995658 conn(0x7fd4c40fc7d0 msgr2=0x7fd4c4069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.706+0000 7fd4c8cc8640 1 -- 172.21.15.172:0/17995658 shutdown_connections 2024-09-10T08:46:39.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:39.706+0000 7fd4c8cc8640 1 -- 172.21.15.172:0/17995658 wait complete. 2024-09-10T08:46:39.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:39 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1266478760' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T08:46:39.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:39 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/271933058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T08:46:40.354 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:40 smithi172 ceph-mon[30854]: pgmap v363: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T08:46:40.354 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:40 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/17995658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:40.356 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-10T08:46:40.356 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-10T08:46:40.356 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T08:46:40.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.462+0000 7f537bcd0640 1 -- 172.21.15.172:0/2714801993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374101b30 msgr2=0x7f5374101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:40.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.462+0000 7f537bcd0640 1 --2- 172.21.15.172:0/2714801993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374101b30 0x7f5374101f30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5370009900 tx=0x7f537002a410 comp rx=0 tx=0).stop 2024-09-10T08:46:40.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.462+0000 7f537bcd0640 1 -- 172.21.15.172:0/2714801993 shutdown_connections 2024-09-10T08:46:40.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.462+0000 7f537bcd0640 1 --2- 172.21.15.172:0/2714801993 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53741036e0 0x7f5374109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.462+0000 7f537bcd0640 1 --2- 172.21.15.172:0/2714801993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5374102d20 0x7f53741031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.462+0000 7f537bcd0640 1 --2- 172.21.15.172:0/2714801993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374101b30 0x7f5374101f30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.462+0000 7f537bcd0640 1 -- 172.21.15.172:0/2714801993 >> 172.21.15.172:0/2714801993 conn(0x7f53740fd2b0 msgr2=0x7f53740ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:40.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.462+0000 7f537bcd0640 1 -- 172.21.15.172:0/2714801993 shutdown_connections 2024-09-10T08:46:40.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.462+0000 7f537bcd0640 1 -- 172.21.15.172:0/2714801993 wait complete. 2024-09-10T08:46:40.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.463+0000 7f537bcd0640 1 Processor -- start 2024-09-10T08:46:40.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.463+0000 7f537bcd0640 1 -- start start 2024-09-10T08:46:40.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.463+0000 7f537bcd0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5374101b30 0x7f5374197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.463+0000 7f537bcd0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374102d20 0x7f5374198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.463+0000 7f537bcd0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53741036e0 0x7f537419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.463+0000 7f537bcd0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f537410d6b0 con 0x7f5374102d20 2024-09-10T08:46:40.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.463+0000 7f537bcd0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f537410d530 con 0x7f53741036e0 2024-09-10T08:46:40.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.463+0000 7f537bcd0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f537410d830 con 0x7f5374101b30 2024-09-10T08:46:40.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.463+0000 7f537926b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374102d20 0x7f5374198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.463+0000 7f537926b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374102d20 0x7f5374198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33268/0 (socket says 172.21.15.172:33268) 2024-09-10T08:46:40.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.463+0000 7f537926b640 1 -- 172.21.15.172:0/606456926 learned_addr learned my addr 172.21.15.172:0/606456926 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:40.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.463+0000 7f5379a6c640 1 --2- 172.21.15.172:0/606456926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5374101b30 0x7f5374197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.463+0000 7f537a26d640 1 --2- 172.21.15.172:0/606456926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53741036e0 0x7f537419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.464+0000 7f537926b640 1 -- 172.21.15.172:0/606456926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5374101b30 msgr2=0x7f5374197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:40.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.464+0000 7f537926b640 1 --2- 172.21.15.172:0/606456926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5374101b30 0x7f5374197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.464+0000 7f537926b640 1 -- 172.21.15.172:0/606456926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53741036e0 msgr2=0x7f537419f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:40.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.464+0000 7f537926b640 1 --2- 172.21.15.172:0/606456926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53741036e0 0x7f537419f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.464+0000 7f537926b640 1 -- 172.21.15.172:0/606456926 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537419f9d0 con 0x7f5374102d20 2024-09-10T08:46:40.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.464+0000 7f5379a6c640 1 --2- 172.21.15.172:0/606456926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5374101b30 0x7f5374197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:40.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.464+0000 7f537926b640 1 --2- 172.21.15.172:0/606456926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374102d20 0x7f5374198300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f536400d950 tx=0x7f536400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:40.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.464+0000 7f536affd640 1 -- 172.21.15.172:0/606456926 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364017070 con 0x7f5374102d20 2024-09-10T08:46:40.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.464+0000 7f536affd640 1 -- 172.21.15.172:0/606456926 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5364009c30 con 0x7f5374102d20 2024-09-10T08:46:40.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.464+0000 7f536affd640 1 -- 172.21.15.172:0/606456926 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536400f620 con 0x7f5374102d20 2024-09-10T08:46:40.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.464+0000 7f537bcd0640 1 -- 172.21.15.172:0/606456926 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537419fcc0 con 0x7f5374102d20 2024-09-10T08:46:40.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.464+0000 7f537bcd0640 1 -- 172.21.15.172:0/606456926 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53741a01c0 con 0x7f5374102d20 2024-09-10T08:46:40.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.465+0000 7f537bcd0640 1 -- 172.21.15.172:0/606456926 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5374108f70 con 0x7f5374102d20 2024-09-10T08:46:40.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.466+0000 7f536affd640 1 -- 172.21.15.172:0/606456926 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f53640109a0 con 0x7f5374102d20 2024-09-10T08:46:40.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.467+0000 7f536affd640 1 --2- 172.21.15.172:0/606456926 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53600a2b90 0x7f53600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.467+0000 7f5379a6c640 1 --2- 172.21.15.172:0/606456926 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53600a2b90 0x7f53600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.467+0000 7f536affd640 1 -- 172.21.15.172:0/606456926 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f53640cf080 con 0x7f5374102d20 2024-09-10T08:46:40.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.467+0000 7f5379a6c640 1 --2- 172.21.15.172:0/606456926 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53600a2b90 0x7f53600a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5374199290 tx=0x7f5370000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:40.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.471+0000 7f536affd640 1 -- 172.21.15.172:0/606456926 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5364009de0 con 0x7f5374102d20 2024-09-10T08:46:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:40 smithi191 ceph-mon[35314]: pgmap v363: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T08:46:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:40 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/17995658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:40.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.637+0000 7f537bcd0640 1 -- 172.21.15.172:0/606456926 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5374062630 con 0x7f53600a2b90 2024-09-10T08:46:40.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.645+0000 7f536affd640 1 -- 172.21.15.172:0/606456926 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340551 (secure 0 0 0) 0x7f5374062630 con 0x7f53600a2b90 2024-09-10T08:46:40.650 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:40.652 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-09-10T08:46:40.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.649+0000 7f537bcd0640 1 -- 172.21.15.172:0/606456926 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53600a2b90 msgr2=0x7f53600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:40.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.649+0000 7f537bcd0640 1 --2- 172.21.15.172:0/606456926 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53600a2b90 0x7f53600a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5374199290 tx=0x7f5370000bc0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.649+0000 7f537bcd0640 1 -- 172.21.15.172:0/606456926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374102d20 msgr2=0x7f5374198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:40.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.649+0000 7f537bcd0640 1 --2- 172.21.15.172:0/606456926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374102d20 0x7f5374198300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f536400d950 tx=0x7f536400de10 comp rx=0 tx=0).stop 2024-09-10T08:46:40.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.649+0000 7f537bcd0640 1 -- 172.21.15.172:0/606456926 shutdown_connections 2024-09-10T08:46:40.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.649+0000 7f537bcd0640 1 --2- 172.21.15.172:0/606456926 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53600a2b90 0x7f53600a5050 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.649+0000 7f537bcd0640 1 --2- 172.21.15.172:0/606456926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53741036e0 0x7f537419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.649+0000 7f537bcd0640 1 --2- 172.21.15.172:0/606456926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374102d20 0x7f5374198300 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.649+0000 7f537bcd0640 1 --2- 172.21.15.172:0/606456926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5374101b30 0x7f5374197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.649+0000 7f537bcd0640 1 -- 172.21.15.172:0/606456926 >> 172.21.15.172:0/606456926 conn(0x7f53740fd2b0 msgr2=0x7f5374106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:40.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.649+0000 7f537bcd0640 1 -- 172.21.15.172:0/606456926 shutdown_connections 2024-09-10T08:46:40.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.649+0000 7f537bcd0640 1 -- 172.21.15.172:0/606456926 wait complete. 2024-09-10T08:46:40.661 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-10T08:46:39.164177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663766,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1193,"num_read_kb":723,"num_write":456,"num_write_kb":3101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376264,"kb_used_data":14528,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565560,"statfs":{"total":1151940427776,"available":1151555133440,"internally_reserved":0,"allocated":14876672,"data_stored":8227505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2475,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":59,"num_write":18,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003231"},"pg_stats":[{"pgid":"2.1f","version":"92'14","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.363708+0000","last_change":"2024-09-10T08:44:08.657737+0000","last_active":"2024-09-10T08:46:34.363708+0000","last_peered":"2024-09-10T08:46:34.363708+0000","last_clean":"2024-09-10T08:46:34.363708+0000","last_became_active":"2024-09-10T08:44:08.657473+0000","last_became_peered":"2024-09-10T08:44:08.657473+0000","last_unstale":"2024-09-10T08:46:34.363708+0000","last_undegraded":"2024-09-10T08:46:34.363708+0000","last_fullsized":"2024-09-10T08:46:34.363708+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:40.757343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193599+0000","last_change":"2024-09-10T08:44:10.675090+0000","last_active":"2024-09-10T08:46:19.193599+0000","last_peered":"2024-09-10T08:46:19.193599+0000","last_clean":"2024-09-10T08:46:19.193599+0000","last_became_active":"2024-09-10T08:44:10.674612+0000","last_became_peered":"2024-09-10T08:44:10.674612+0000","last_unstale":"2024-09-10T08:46:19.193599+0000","last_undegraded":"2024-09-10T08:46:19.193599+0000","last_fullsized":"2024-09-10T08:46:19.193599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:46.821080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.807904+0000","last_change":"2024-09-10T08:44:17.721559+0000","last_active":"2024-09-10T08:46:18.807904+0000","last_peered":"2024-09-10T08:46:18.807904+0000","last_clean":"2024-09-10T08:46:18.807904+0000","last_became_active":"2024-09-10T08:44:17.721182+0000","last_became_peered":"2024-09-10T08:44:17.721182+0000","last_unstale":"2024-09-10T08:46:18.807904+0000","last_undegraded":"2024-09-10T08:46:18.807904+0000","last_fullsized":"2024-09-10T08:46:18.807904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:18.791489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193663+0000","last_change":"2024-09-10T08:44:09.049171+0000","last_active":"2024-09-10T08:46:19.193663+0000","last_peered":"2024-09-10T08:46:19.193663+0000","last_clean":"2024-09-10T08:46:19.193663+0000","last_became_active":"2024-09-10T08:44:09.048806+0000","last_became_peered":"2024-09-10T08:44:09.048806+0000","last_unstale":"2024-09-10T08:46:19.193663+0000","last_undegraded":"2024-09-10T08:46:19.193663+0000","last_fullsized":"2024-09-10T08:46:19.193663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:21.023460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193866+0000","last_change":"2024-09-10T08:44:10.677569+0000","last_active":"2024-09-10T08:46:19.193866+0000","last_peered":"2024-09-10T08:46:19.193866+0000","last_clean":"2024-09-10T08:46:19.193866+0000","last_became_active":"2024-09-10T08:44:10.677048+0000","last_became_peered":"2024-09-10T08:44:10.677048+0000","last_unstale":"2024-09-10T08:46:19.193866+0000","last_undegraded":"2024-09-10T08:46:19.193866+0000","last_fullsized":"2024-09-10T08:46:19.193866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:35.523063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193330+0000","last_change":"2024-09-10T08:44:17.720138+0000","last_active":"2024-09-10T08:46:19.193330+0000","last_peered":"2024-09-10T08:46:19.193330+0000","last_clean":"2024-09-10T08:46:19.193330+0000","last_became_active":"2024-09-10T08:44:17.719344+0000","last_became_peered":"2024-09-10T08:44:17.719344+0000","last_unstale":"2024-09-10T08:46:19.193330+0000","last_undegraded":"2024-09-10T08:46:19.193330+0000","last_fullsized":"2024-09-10T08:46:19.193330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:11.758058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194115+0000","last_change":"2024-09-10T08:44:08.660099+0000","last_active":"2024-09-10T08:46:19.194115+0000","last_peered":"2024-09-10T08:46:19.194115+0000","last_clean":"2024-09-10T08:46:19.194115+0000","last_became_active":"2024-09-10T08:44:08.659745+0000","last_became_peered":"2024-09-10T08:44:08.659745+0000","last_unstale":"2024-09-10T08:46:19.194115+0000","last_undegraded":"2024-09-10T08:46:19.194115+0000","last_fullsized":"2024-09-10T08:46:19.194115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:43.960717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193698+0000","last_change":"2024-09-10T08:44:10.675583+0000","last_active":"2024-09-10T08:46:19.193698+0000","last_peered":"2024-09-10T08:46:19.193698+0000","last_clean":"2024-09-10T08:46:19.193698+0000","last_became_active":"2024-09-10T08:44:10.675296+0000","last_became_peered":"2024-09-10T08:44:10.675296+0000","last_unstale":"2024-09-10T08:46:19.193698+0000","last_undegraded":"2024-09-10T08:46:19.193698+0000","last_fullsized":"2024-09-10T08:46:19.193698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:31:51.147862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194363+0000","last_change":"2024-09-10T08:44:17.720104+0000","last_active":"2024-09-10T08:46:19.194363+0000","last_peered":"2024-09-10T08:46:19.194363+0000","last_clean":"2024-09-10T08:46:19.194363+0000","last_became_active":"2024-09-10T08:44:17.719607+0000","last_became_peered":"2024-09-10T08:44:17.719607+0000","last_unstale":"2024-09-10T08:46:19.194363+0000","last_undegraded":"2024-09-10T08:46:19.194363+0000","last_fullsized":"2024-09-10T08:46:19.194363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:52.976717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193719+0000","last_change":"2024-09-10T08:44:08.658588+0000","last_active":"2024-09-10T08:46:19.193719+0000","last_peered":"2024-09-10T08:46:19.193719+0000","last_clean":"2024-09-10T08:46:19.193719+0000","last_became_active":"2024-09-10T08:44:08.658053+0000","last_became_peered":"2024-09-10T08:44:08.658053+0000","last_unstale":"2024-09-10T08:46:19.193719+0000","last_undegraded":"2024-09-10T08:46:19.193719+0000","last_fullsized":"2024-09-10T08:46:19.193719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:26.744943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804707+0000","last_change":"2024-09-10T08:44:10.681978+0000","last_active":"2024-09-10T08:46:18.804707+0000","last_peered":"2024-09-10T08:46:18.804707+0000","last_clean":"2024-09-10T08:46:18.804707+0000","last_became_active":"2024-09-10T08:44:10.681600+0000","last_became_peered":"2024-09-10T08:44:10.681600+0000","last_unstale":"2024-09-10T08:46:18.804707+0000","last_undegraded":"2024-09-10T08:46:18.804707+0000","last_fullsized":"2024-09-10T08:46:18.804707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:37.190576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193813+0000","last_change":"2024-09-10T08:44:17.719784+0000","last_active":"2024-09-10T08:46:19.193813+0000","last_peered":"2024-09-10T08:46:19.193813+0000","last_clean":"2024-09-10T08:46:19.193813+0000","last_became_active":"2024-09-10T08:44:17.719182+0000","last_became_peered":"2024-09-10T08:44:17.719182+0000","last_unstale":"2024-09-10T08:46:19.193813+0000","last_undegraded":"2024-09-10T08:46:19.193813+0000","last_fullsized":"2024-09-10T08:46:19.193813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:15.237772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873709+0000","last_change":"2024-09-10T08:44:10.679069+0000","last_active":"2024-09-10T08:46:18.873709+0000","last_peered":"2024-09-10T08:46:18.873709+0000","last_clean":"2024-09-10T08:46:18.873709+0000","last_became_active":"2024-09-10T08:44:10.678597+0000","last_became_peered":"2024-09-10T08:44:10.678597+0000","last_unstale":"2024-09-10T08:46:18.873709+0000","last_undegraded":"2024-09-10T08:46:18.873709+0000","last_fullsized":"2024-09-10T08:46:18.873709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:52.693906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194153+0000","last_change":"2024-09-10T08:44:08.660886+0000","last_active":"2024-09-10T08:46:19.194153+0000","last_peered":"2024-09-10T08:46:19.194153+0000","last_clean":"2024-09-10T08:46:19.194153+0000","last_became_active":"2024-09-10T08:44:08.660478+0000","last_became_peered":"2024-09-10T08:44:08.660478+0000","last_unstale":"2024-09-10T08:46:19.194153+0000","last_undegraded":"2024-09-10T08:46:19.194153+0000","last_fullsized":"2024-09-10T08:46:19.194153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:41.785531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194323+0000","last_change":"2024-09-10T08:44:17.761929+0000","last_active":"2024-09-10T08:46:19.194323+0000","last_peered":"2024-09-10T08:46:19.194323+0000","last_clean":"2024-09-10T08:46:19.194323+0000","last_became_active":"2024-09-10T08:44:17.761440+0000","last_became_peered":"2024-09-10T08:44:17.761440+0000","last_unstale":"2024-09-10T08:46:19.194323+0000","last_undegraded":"2024-09-10T08:46:19.194323+0000","last_fullsized":"2024-09-10T08:46:19.194323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:37.817759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192895+0000","last_change":"2024-09-10T08:44:10.674071+0000","last_active":"2024-09-10T08:46:19.192895+0000","last_peered":"2024-09-10T08:46:19.192895+0000","last_clean":"2024-09-10T08:46:19.192895+0000","last_became_active":"2024-09-10T08:44:10.673770+0000","last_became_peered":"2024-09-10T08:44:10.673770+0000","last_unstale":"2024-09-10T08:46:19.192895+0000","last_undegraded":"2024-09-10T08:46:19.192895+0000","last_fullsized":"2024-09-10T08:46:19.192895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:44:19.944557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.231938+0000","last_change":"2024-09-10T08:45:18.264060+0000","last_active":"2024-09-10T08:46:34.231938+0000","last_peered":"2024-09-10T08:46:34.231938+0000","last_clean":"2024-09-10T08:46:34.231938+0000","last_became_active":"2024-09-10T08:45:18.255983+0000","last_became_peered":"2024-09-10T08:45:18.255983+0000","last_unstale":"2024-09-10T08:46:34.231938+0000","last_undegraded":"2024-09-10T08:46:34.231938+0000","last_fullsized":"2024-09-10T08:46:34.231938+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:10.810491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194248+0000","last_change":"2024-09-10T08:44:17.722877+0000","last_active":"2024-09-10T08:46:19.194248+0000","last_peered":"2024-09-10T08:46:19.194248+0000","last_clean":"2024-09-10T08:46:19.194248+0000","last_became_active":"2024-09-10T08:44:17.722046+0000","last_became_peered":"2024-09-10T08:44:17.722046+0000","last_unstale":"2024-09-10T08:46:19.194248+0000","last_undegraded":"2024-09-10T08:46:19.194248+0000","last_fullsized":"2024-09-10T08:46:19.194248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:32.675598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193398+0000","last_change":"2024-09-10T08:44:10.671500+0000","last_active":"2024-09-10T08:46:19.193398+0000","last_peered":"2024-09-10T08:46:19.193398+0000","last_clean":"2024-09-10T08:46:19.193398+0000","last_became_active":"2024-09-10T08:44:10.671039+0000","last_became_peered":"2024-09-10T08:44:10.671039+0000","last_unstale":"2024-09-10T08:46:19.193398+0000","last_undegraded":"2024-09-10T08:46:19.193398+0000","last_fullsized":"2024-09-10T08:46:19.193398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:12.977373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193090+0000","last_change":"2024-09-10T08:44:08.655467+0000","last_active":"2024-09-10T08:46:19.193090+0000","last_peered":"2024-09-10T08:46:19.193090+0000","last_clean":"2024-09-10T08:46:19.193090+0000","last_became_active":"2024-09-10T08:44:08.654991+0000","last_became_peered":"2024-09-10T08:44:08.654991+0000","last_unstale":"2024-09-10T08:46:19.193090+0000","last_undegraded":"2024-09-10T08:46:19.193090+0000","last_fullsized":"2024-09-10T08:46:19.193090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:33.124360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803728+0000","last_change":"2024-09-10T08:44:17.724250+0000","last_active":"2024-09-10T08:46:18.803728+0000","last_peered":"2024-09-10T08:46:18.803728+0000","last_clean":"2024-09-10T08:46:18.803728+0000","last_became_active":"2024-09-10T08:44:17.723733+0000","last_became_peered":"2024-09-10T08:44:17.723733+0000","last_unstale":"2024-09-10T08:46:18.803728+0000","last_undegraded":"2024-09-10T08:46:18.803728+0000","last_fullsized":"2024-09-10T08:46:18.803728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:40.098666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193144+0000","last_change":"2024-09-10T08:44:10.664159+0000","last_active":"2024-09-10T08:46:19.193144+0000","last_peered":"2024-09-10T08:46:19.193144+0000","last_clean":"2024-09-10T08:46:19.193144+0000","last_became_active":"2024-09-10T08:44:10.663674+0000","last_became_peered":"2024-09-10T08:44:10.663674+0000","last_unstale":"2024-09-10T08:46:19.193144+0000","last_undegraded":"2024-09-10T08:46:19.193144+0000","last_fullsized":"2024-09-10T08:46:19.193144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:24.234525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193535+0000","last_change":"2024-09-10T08:45:18.257226+0000","last_active":"2024-09-10T08:46:19.193535+0000","last_peered":"2024-09-10T08:46:19.193535+0000","last_clean":"2024-09-10T08:46:19.193535+0000","last_became_active":"2024-09-10T08:45:18.254113+0000","last_became_peered":"2024-09-10T08:45:18.254113+0000","last_unstale":"2024-09-10T08:46:19.193535+0000","last_undegraded":"2024-09-10T08:46:19.193535+0000","last_fullsized":"2024-09-10T08:46:19.193535+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:19.370105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194467+0000","last_change":"2024-09-10T08:44:17.761639+0000","last_active":"2024-09-10T08:46:19.194467+0000","last_peered":"2024-09-10T08:46:19.194467+0000","last_clean":"2024-09-10T08:46:19.194467+0000","last_became_active":"2024-09-10T08:44:17.761214+0000","last_became_peered":"2024-09-10T08:44:17.761214+0000","last_unstale":"2024-09-10T08:46:19.194467+0000","last_undegraded":"2024-09-10T08:46:19.194467+0000","last_fullsized":"2024-09-10T08:46:19.194467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:43.793195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193666+0000","last_change":"2024-09-10T08:45:18.258661+0000","last_active":"2024-09-10T08:46:19.193666+0000","last_peered":"2024-09-10T08:46:19.193666+0000","last_clean":"2024-09-10T08:46:19.193666+0000","last_became_active":"2024-09-10T08:45:18.258225+0000","last_became_peered":"2024-09-10T08:45:18.258225+0000","last_unstale":"2024-09-10T08:46:19.193666+0000","last_undegraded":"2024-09-10T08:46:19.193666+0000","last_fullsized":"2024-09-10T08:46:19.193666+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:28:37.184766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194660+0000","last_change":"2024-09-10T08:44:09.036632+0000","last_active":"2024-09-10T08:46:19.194660+0000","last_peered":"2024-09-10T08:46:19.194660+0000","last_clean":"2024-09-10T08:46:19.194660+0000","last_became_active":"2024-09-10T08:44:09.036254+0000","last_became_peered":"2024-09-10T08:44:09.036254+0000","last_unstale":"2024-09-10T08:46:19.194660+0000","last_undegraded":"2024-09-10T08:46:19.194660+0000","last_fullsized":"2024-09-10T08:46:19.194660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:59:40.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808140+0000","last_change":"2024-09-10T08:44:17.716215+0000","last_active":"2024-09-10T08:46:18.808140+0000","last_peered":"2024-09-10T08:46:18.808140+0000","last_clean":"2024-09-10T08:46:18.808140+0000","last_became_active":"2024-09-10T08:44:17.715809+0000","last_became_peered":"2024-09-10T08:44:17.715809+0000","last_unstale":"2024-09-10T08:46:18.808140+0000","last_undegraded":"2024-09-10T08:46:18.808140+0000","last_fullsized":"2024-09-10T08:46:18.808140+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:52.896401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193713+0000","last_change":"2024-09-10T08:44:10.677871+0000","last_active":"2024-09-10T08:46:19.193713+0000","last_peered":"2024-09-10T08:46:19.193713+0000","last_clean":"2024-09-10T08:46:19.193713+0000","last_became_active":"2024-09-10T08:44:10.677101+0000","last_became_peered":"2024-09-10T08:44:10.677101+0000","last_unstale":"2024-09-10T08:46:19.193713+0000","last_undegraded":"2024-09-10T08:46:19.193713+0000","last_fullsized":"2024-09-10T08:46:19.193713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:46.137249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'17","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:21.176388+0000","last_change":"2024-09-10T08:44:08.654428+0000","last_active":"2024-09-10T08:46:21.176388+0000","last_peered":"2024-09-10T08:46:21.176388+0000","last_clean":"2024-09-10T08:46:21.176388+0000","last_became_active":"2024-09-10T08:44:08.654119+0000","last_became_peered":"2024-09-10T08:44:08.654119+0000","last_unstale":"2024-09-10T08:46:21.176388+0000","last_undegraded":"2024-09-10T08:46:21.176388+0000","last_fullsized":"2024-09-10T08:46:21.176388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:56.787721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192538+0000","last_change":"2024-09-10T08:44:17.760828+0000","last_active":"2024-09-10T08:46:19.192538+0000","last_peered":"2024-09-10T08:46:19.192538+0000","last_clean":"2024-09-10T08:46:19.192538+0000","last_became_active":"2024-09-10T08:44:17.760367+0000","last_became_peered":"2024-09-10T08:44:17.760367+0000","last_unstale":"2024-09-10T08:46:19.192538+0000","last_undegraded":"2024-09-10T08:46:19.192538+0000","last_fullsized":"2024-09-10T08:46:19.192538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:11.130480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806446+0000","last_change":"2024-09-10T08:44:10.670179+0000","last_active":"2024-09-10T08:46:18.806446+0000","last_peered":"2024-09-10T08:46:18.806446+0000","last_clean":"2024-09-10T08:46:18.806446+0000","last_became_active":"2024-09-10T08:44:10.669366+0000","last_became_peered":"2024-09-10T08:44:10.669366+0000","last_unstale":"2024-09-10T08:46:18.806446+0000","last_undegraded":"2024-09-10T08:46:18.806446+0000","last_fullsized":"2024-09-10T08:46:18.806446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:41.916365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194210+0000","last_change":"2024-09-10T08:44:09.050441+0000","last_active":"2024-09-10T08:46:19.194210+0000","last_peered":"2024-09-10T08:46:19.194210+0000","last_clean":"2024-09-10T08:46:19.194210+0000","last_became_active":"2024-09-10T08:44:09.050070+0000","last_became_peered":"2024-09-10T08:44:09.050070+0000","last_unstale":"2024-09-10T08:46:19.194210+0000","last_undegraded":"2024-09-10T08:46:19.194210+0000","last_fullsized":"2024-09-10T08:46:19.194210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:13.540021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193573+0000","last_change":"2024-09-10T08:44:17.720098+0000","last_active":"2024-09-10T08:46:19.193573+0000","last_peered":"2024-09-10T08:46:19.193573+0000","last_clean":"2024-09-10T08:46:19.193573+0000","last_became_active":"2024-09-10T08:44:17.719627+0000","last_became_peered":"2024-09-10T08:44:17.719627+0000","last_unstale":"2024-09-10T08:46:19.193573+0000","last_undegraded":"2024-09-10T08:46:19.193573+0000","last_fullsized":"2024-09-10T08:46:19.193573+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:50.020919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804677+0000","last_change":"2024-09-10T08:44:10.669630+0000","last_active":"2024-09-10T08:46:18.804677+0000","last_peered":"2024-09-10T08:46:18.804677+0000","last_clean":"2024-09-10T08:46:18.804677+0000","last_became_active":"2024-09-10T08:44:10.669307+0000","last_became_peered":"2024-09-10T08:44:10.669307+0000","last_unstale":"2024-09-10T08:46:18.804677+0000","last_undegraded":"2024-09-10T08:46:18.804677+0000","last_fullsized":"2024-09-10T08:46:18.804677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:06.320046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803568+0000","last_change":"2024-09-10T08:44:09.050715+0000","last_active":"2024-09-10T08:46:18.803568+0000","last_peered":"2024-09-10T08:46:18.803568+0000","last_clean":"2024-09-10T08:46:18.803568+0000","last_became_active":"2024-09-10T08:44:09.050347+0000","last_became_peered":"2024-09-10T08:44:09.050347+0000","last_unstale":"2024-09-10T08:46:18.803568+0000","last_undegraded":"2024-09-10T08:46:18.803568+0000","last_fullsized":"2024-09-10T08:46:18.803568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:18.689652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194038+0000","last_change":"2024-09-10T08:45:18.260444+0000","last_active":"2024-09-10T08:46:19.194038+0000","last_peered":"2024-09-10T08:46:19.194038+0000","last_clean":"2024-09-10T08:46:19.194038+0000","last_became_active":"2024-09-10T08:45:18.259949+0000","last_became_peered":"2024-09-10T08:45:18.259949+0000","last_unstale":"2024-09-10T08:46:19.194038+0000","last_undegraded":"2024-09-10T08:46:19.194038+0000","last_fullsized":"2024-09-10T08:46:19.194038+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:56.391253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804630+0000","last_change":"2024-09-10T08:44:10.670881+0000","last_active":"2024-09-10T08:46:18.804630+0000","last_peered":"2024-09-10T08:46:18.804630+0000","last_clean":"2024-09-10T08:46:18.804630+0000","last_became_active":"2024-09-10T08:44:10.669791+0000","last_became_peered":"2024-09-10T08:44:10.669791+0000","last_unstale":"2024-09-10T08:46:18.804630+0000","last_undegraded":"2024-09-10T08:46:18.804630+0000","last_fullsized":"2024-09-10T08:46:18.804630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:33.644205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'6","reported_seq":151,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:37.369215+0000","last_change":"2024-09-10T08:44:08.657938+0000","last_active":"2024-09-10T08:46:37.369215+0000","last_peered":"2024-09-10T08:46:37.369215+0000","last_clean":"2024-09-10T08:46:37.369215+0000","last_became_active":"2024-09-10T08:44:08.657526+0000","last_became_peered":"2024-09-10T08:44:08.657526+0000","last_unstale":"2024-09-10T08:46:37.369215+0000","last_undegraded":"2024-09-10T08:46:37.369215+0000","last_fullsized":"2024-09-10T08:46:37.369215+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:15.673561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806147+0000","last_change":"2024-09-10T08:44:17.721619+0000","last_active":"2024-09-10T08:46:18.806147+0000","last_peered":"2024-09-10T08:46:18.806147+0000","last_clean":"2024-09-10T08:46:18.806147+0000","last_became_active":"2024-09-10T08:44:17.721175+0000","last_became_peered":"2024-09-10T08:44:17.721175+0000","last_unstale":"2024-09-10T08:46:18.806147+0000","last_undegraded":"2024-09-10T08:46:18.806147+0000","last_fullsized":"2024-09-10T08:46:18.806147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:52.244120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804320+0000","last_change":"2024-09-10T08:44:10.676164+0000","last_active":"2024-09-10T08:46:18.804320+0000","last_peered":"2024-09-10T08:46:18.804320+0000","last_clean":"2024-09-10T08:46:18.804320+0000","last_became_active":"2024-09-10T08:44:10.675820+0000","last_became_peered":"2024-09-10T08:44:10.675820+0000","last_unstale":"2024-09-10T08:46:18.804320+0000","last_undegraded":"2024-09-10T08:46:18.804320+0000","last_fullsized":"2024-09-10T08:46:18.804320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:24.477584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":152,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:37.369105+0000","last_change":"2024-09-10T08:44:09.049124+0000","last_active":"2024-09-10T08:46:37.369105+0000","last_peered":"2024-09-10T08:46:37.369105+0000","last_clean":"2024-09-10T08:46:37.369105+0000","last_became_active":"2024-09-10T08:44:09.048799+0000","last_became_peered":"2024-09-10T08:44:09.048799+0000","last_unstale":"2024-09-10T08:46:37.369105+0000","last_undegraded":"2024-09-10T08:46:37.369105+0000","last_fullsized":"2024-09-10T08:46:37.369105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:43:39.849727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806891+0000","last_change":"2024-09-10T08:44:17.761130+0000","last_active":"2024-09-10T08:46:18.806891+0000","last_peered":"2024-09-10T08:46:18.806891+0000","last_clean":"2024-09-10T08:46:18.806891+0000","last_became_active":"2024-09-10T08:44:17.760753+0000","last_became_peered":"2024-09-10T08:44:17.760753+0000","last_unstale":"2024-09-10T08:46:18.806891+0000","last_undegraded":"2024-09-10T08:46:18.806891+0000","last_fullsized":"2024-09-10T08:46:18.806891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:42.971962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.809258+0000","last_change":"2024-09-10T08:46:18.809258+0000","last_active":"2024-09-10T08:46:18.809258+0000","last_peered":"2024-09-10T08:46:18.809258+0000","last_clean":"2024-09-10T08:46:18.809258+0000","last_became_active":"2024-09-10T08:46:18.808908+0000","last_became_peered":"2024-09-10T08:46:18.808908+0000","last_unstale":"2024-09-10T08:46:18.809258+0000","last_undegraded":"2024-09-10T08:46:18.809258+0000","last_fullsized":"2024-09-10T08:46:18.809258+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:10.305672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873500+0000","last_change":"2024-09-10T08:44:08.658862+0000","last_active":"2024-09-10T08:46:18.873500+0000","last_peered":"2024-09-10T08:46:18.873500+0000","last_clean":"2024-09-10T08:46:18.873500+0000","last_became_active":"2024-09-10T08:44:08.657880+0000","last_became_peered":"2024-09-10T08:44:08.657880+0000","last_unstale":"2024-09-10T08:46:18.873500+0000","last_undegraded":"2024-09-10T08:46:18.873500+0000","last_fullsized":"2024-09-10T08:46:18.873500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:34.224025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194185+0000","last_change":"2024-09-10T08:45:18.268446+0000","last_active":"2024-09-10T08:46:19.194185+0000","last_peered":"2024-09-10T08:46:19.194185+0000","last_clean":"2024-09-10T08:46:19.194185+0000","last_became_active":"2024-09-10T08:45:18.267988+0000","last_became_peered":"2024-09-10T08:45:18.267988+0000","last_unstale":"2024-09-10T08:46:19.194185+0000","last_undegraded":"2024-09-10T08:46:19.194185+0000","last_fullsized":"2024-09-10T08:46:19.194185+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:42:35.990794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193214+0000","last_change":"2024-09-10T08:44:10.673258+0000","last_active":"2024-09-10T08:46:19.193214+0000","last_peered":"2024-09-10T08:46:19.193214+0000","last_clean":"2024-09-10T08:46:19.193214+0000","last_became_active":"2024-09-10T08:44:10.672854+0000","last_became_peered":"2024-09-10T08:44:10.672854+0000","last_unstale":"2024-09-10T08:46:19.193214+0000","last_undegraded":"2024-09-10T08:46:19.193214+0000","last_fullsized":"2024-09-10T08:46:19.193214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:35.913820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194588+0000","last_change":"2024-09-10T08:44:08.658039+0000","last_active":"2024-09-10T08:46:19.194588+0000","last_peered":"2024-09-10T08:46:19.194588+0000","last_clean":"2024-09-10T08:46:19.194588+0000","last_became_active":"2024-09-10T08:44:08.657636+0000","last_became_peered":"2024-09-10T08:44:08.657636+0000","last_unstale":"2024-09-10T08:46:19.194588+0000","last_undegraded":"2024-09-10T08:46:19.194588+0000","last_fullsized":"2024-09-10T08:46:19.194588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:09.615992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873591+0000","last_change":"2024-09-10T08:42:57.666747+0000","last_active":"2024-09-10T08:46:18.873591+0000","last_peered":"2024-09-10T08:46:18.873591+0000","last_clean":"2024-09-10T08:46:18.873591+0000","last_became_active":"2024-09-10T08:42:57.666417+0000","last_became_peered":"2024-09-10T08:42:57.666417+0000","last_unstale":"2024-09-10T08:46:18.873591+0000","last_undegraded":"2024-09-10T08:46:18.873591+0000","last_fullsized":"2024-09-10T08:46:18.873591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_clean_scrub_stamp":"2024-09-10T08:39:31.729794+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.547989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874074+0000","last_change":"2024-09-10T08:44:17.723021+0000","last_active":"2024-09-10T08:46:18.874074+0000","last_peered":"2024-09-10T08:46:18.874074+0000","last_clean":"2024-09-10T08:46:18.874074+0000","last_became_active":"2024-09-10T08:44:17.722222+0000","last_became_peered":"2024-09-10T08:44:17.722222+0000","last_unstale":"2024-09-10T08:46:18.874074+0000","last_undegraded":"2024-09-10T08:46:18.874074+0000","last_fullsized":"2024-09-10T08:46:18.874074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:24.444427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874129+0000","last_change":"2024-09-10T08:44:10.676651+0000","last_active":"2024-09-10T08:46:18.874129+0000","last_peered":"2024-09-10T08:46:18.874129+0000","last_clean":"2024-09-10T08:46:18.874129+0000","last_became_active":"2024-09-10T08:44:10.676342+0000","last_became_peered":"2024-09-10T08:44:10.676342+0000","last_unstale":"2024-09-10T08:46:18.874129+0000","last_undegraded":"2024-09-10T08:46:18.874129+0000","last_fullsized":"2024-09-10T08:46:18.874129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:07.021273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'8","reported_seq":154,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:35.366830+0000","last_change":"2024-09-10T08:44:09.050305+0000","last_active":"2024-09-10T08:46:35.366830+0000","last_peered":"2024-09-10T08:46:35.366830+0000","last_clean":"2024-09-10T08:46:35.366830+0000","last_became_active":"2024-09-10T08:44:09.049958+0000","last_became_peered":"2024-09-10T08:44:09.049958+0000","last_unstale":"2024-09-10T08:46:35.366830+0000","last_undegraded":"2024-09-10T08:46:35.366830+0000","last_fullsized":"2024-09-10T08:46:35.366830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:42.219190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2199,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193720+0000","last_change":"2024-09-10T08:44:17.720219+0000","last_active":"2024-09-10T08:46:19.193720+0000","last_peered":"2024-09-10T08:46:19.193720+0000","last_clean":"2024-09-10T08:46:19.193720+0000","last_became_active":"2024-09-10T08:44:17.719756+0000","last_became_peered":"2024-09-10T08:44:17.719756+0000","last_unstale":"2024-09-10T08:46:19.193720+0000","last_undegraded":"2024-09-10T08:46:19.193720+0000","last_fullsized":"2024-09-10T08:46:19.193720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:01.070434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873929+0000","last_change":"2024-09-10T08:44:10.678986+0000","last_active":"2024-09-10T08:46:18.873929+0000","last_peered":"2024-09-10T08:46:18.873929+0000","last_clean":"2024-09-10T08:46:18.873929+0000","last_became_active":"2024-09-10T08:44:10.678654+0000","last_became_peered":"2024-09-10T08:44:10.678654+0000","last_unstale":"2024-09-10T08:46:18.873929+0000","last_undegraded":"2024-09-10T08:46:18.873929+0000","last_fullsized":"2024-09-10T08:46:18.873929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:07.116802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805026+0000","last_change":"2024-09-10T08:44:08.655025+0000","last_active":"2024-09-10T08:46:18.805026+0000","last_peered":"2024-09-10T08:46:18.805026+0000","last_clean":"2024-09-10T08:46:18.805026+0000","last_became_active":"2024-09-10T08:44:08.654728+0000","last_became_peered":"2024-09-10T08:44:08.654728+0000","last_unstale":"2024-09-10T08:46:18.805026+0000","last_undegraded":"2024-09-10T08:46:18.805026+0000","last_fullsized":"2024-09-10T08:46:18.805026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:31.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193534+0000","last_change":"2024-09-10T08:45:18.261492+0000","last_active":"2024-09-10T08:46:19.193534+0000","last_peered":"2024-09-10T08:46:19.193534+0000","last_clean":"2024-09-10T08:46:19.193534+0000","last_became_active":"2024-09-10T08:45:18.261111+0000","last_became_peered":"2024-09-10T08:45:18.261111+0000","last_unstale":"2024-09-10T08:46:19.193534+0000","last_undegraded":"2024-09-10T08:46:19.193534+0000","last_fullsized":"2024-09-10T08:46:19.193534+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:48.709459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874006+0000","last_change":"2024-09-10T08:44:10.675072+0000","last_active":"2024-09-10T08:46:18.874006+0000","last_peered":"2024-09-10T08:46:18.874006+0000","last_clean":"2024-09-10T08:46:18.874006+0000","last_became_active":"2024-09-10T08:44:10.674636+0000","last_became_peered":"2024-09-10T08:44:10.674636+0000","last_unstale":"2024-09-10T08:46:18.874006+0000","last_undegraded":"2024-09-10T08:46:18.874006+0000","last_fullsized":"2024-09-10T08:46:18.874006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:55.198445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806864+0000","last_change":"2024-09-10T08:44:08.656450+0000","last_active":"2024-09-10T08:46:18.806864+0000","last_peered":"2024-09-10T08:46:18.806864+0000","last_clean":"2024-09-10T08:46:18.806864+0000","last_became_active":"2024-09-10T08:44:08.655967+0000","last_became_peered":"2024-09-10T08:44:08.655967+0000","last_unstale":"2024-09-10T08:46:18.806864+0000","last_undegraded":"2024-09-10T08:46:18.806864+0000","last_fullsized":"2024-09-10T08:46:18.806864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:29.273167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804195+0000","last_change":"2024-09-10T08:44:17.722217+0000","last_active":"2024-09-10T08:46:18.804195+0000","last_peered":"2024-09-10T08:46:18.804195+0000","last_clean":"2024-09-10T08:46:18.804195+0000","last_became_active":"2024-09-10T08:44:17.721800+0000","last_became_peered":"2024-09-10T08:44:17.721800+0000","last_unstale":"2024-09-10T08:46:18.804195+0000","last_undegraded":"2024-09-10T08:46:18.804195+0000","last_fullsized":"2024-09-10T08:46:18.804195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:58.954397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194230+0000","last_change":"2024-09-10T08:45:18.267857+0000","last_active":"2024-09-10T08:46:19.194230+0000","last_peered":"2024-09-10T08:46:19.194230+0000","last_clean":"2024-09-10T08:46:19.194230+0000","last_became_active":"2024-09-10T08:45:18.267440+0000","last_became_peered":"2024-09-10T08:45:18.267440+0000","last_unstale":"2024-09-10T08:46:19.194230+0000","last_undegraded":"2024-09-10T08:46:19.194230+0000","last_fullsized":"2024-09-10T08:46:19.194230+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:28.659915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:23.312217+0000","last_change":"2024-09-10T08:44:08.659003+0000","last_active":"2024-09-10T08:46:23.312217+0000","last_peered":"2024-09-10T08:46:23.312217+0000","last_clean":"2024-09-10T08:46:23.312217+0000","last_became_active":"2024-09-10T08:44:08.658324+0000","last_became_peered":"2024-09-10T08:44:08.658324+0000","last_unstale":"2024-09-10T08:46:23.312217+0000","last_undegraded":"2024-09-10T08:46:23.312217+0000","last_fullsized":"2024-09-10T08:46:23.312217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:44.743053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804596+0000","last_change":"2024-09-10T08:44:17.723363+0000","last_active":"2024-09-10T08:46:18.804596+0000","last_peered":"2024-09-10T08:46:18.804596+0000","last_clean":"2024-09-10T08:46:18.804596+0000","last_became_active":"2024-09-10T08:44:17.721799+0000","last_became_peered":"2024-09-10T08:44:17.721799+0000","last_unstale":"2024-09-10T08:46:18.804596+0000","last_undegraded":"2024-09-10T08:46:18.804596+0000","last_fullsized":"2024-09-10T08:46:18.804596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:08:28.772193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804709+0000","last_change":"2024-09-10T08:44:10.682952+0000","last_active":"2024-09-10T08:46:18.804709+0000","last_peered":"2024-09-10T08:46:18.804709+0000","last_clean":"2024-09-10T08:46:18.804709+0000","last_became_active":"2024-09-10T08:44:10.682249+0000","last_became_peered":"2024-09-10T08:44:10.682249+0000","last_unstale":"2024-09-10T08:46:18.804709+0000","last_undegraded":"2024-09-10T08:46:18.804709+0000","last_fullsized":"2024-09-10T08:46:18.804709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:32:21.285461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193239+0000","last_change":"2024-09-10T08:44:09.050454+0000","last_active":"2024-09-10T08:46:19.193239+0000","last_peered":"2024-09-10T08:46:19.193239+0000","last_clean":"2024-09-10T08:46:19.193239+0000","last_became_active":"2024-09-10T08:44:09.050036+0000","last_became_peered":"2024-09-10T08:44:09.050036+0000","last_unstale":"2024-09-10T08:46:19.193239+0000","last_undegraded":"2024-09-10T08:46:19.193239+0000","last_fullsized":"2024-09-10T08:46:19.193239+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:55.141471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806678+0000","last_change":"2024-09-10T08:44:17.721203+0000","last_active":"2024-09-10T08:46:18.806678+0000","last_peered":"2024-09-10T08:46:18.806678+0000","last_clean":"2024-09-10T08:46:18.806678+0000","last_became_active":"2024-09-10T08:44:17.720737+0000","last_became_peered":"2024-09-10T08:44:17.720737+0000","last_unstale":"2024-09-10T08:46:18.806678+0000","last_undegraded":"2024-09-10T08:46:18.806678+0000","last_fullsized":"2024-09-10T08:46:18.806678+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:32.081079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873435+0000","last_change":"2024-09-10T08:44:10.680944+0000","last_active":"2024-09-10T08:46:18.873435+0000","last_peered":"2024-09-10T08:46:18.873435+0000","last_clean":"2024-09-10T08:46:18.873435+0000","last_became_active":"2024-09-10T08:44:10.680644+0000","last_became_peered":"2024-09-10T08:44:10.680644+0000","last_unstale":"2024-09-10T08:46:18.873435+0000","last_undegraded":"2024-09-10T08:46:18.873435+0000","last_fullsized":"2024-09-10T08:46:18.873435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:46.495894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":148,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:33.363751+0000","last_change":"2024-09-10T08:44:08.658135+0000","last_active":"2024-09-10T08:46:33.363751+0000","last_peered":"2024-09-10T08:46:33.363751+0000","last_clean":"2024-09-10T08:46:33.363751+0000","last_became_active":"2024-09-10T08:44:08.657793+0000","last_became_peered":"2024-09-10T08:44:08.657793+0000","last_unstale":"2024-09-10T08:46:33.363751+0000","last_undegraded":"2024-09-10T08:46:33.363751+0000","last_fullsized":"2024-09-10T08:46:33.363751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:16.376268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193715+0000","last_change":"2024-09-10T08:45:18.271678+0000","last_active":"2024-09-10T08:46:19.193715+0000","last_peered":"2024-09-10T08:46:19.193715+0000","last_clean":"2024-09-10T08:46:19.193715+0000","last_became_active":"2024-09-10T08:45:18.271209+0000","last_became_peered":"2024-09-10T08:45:18.271209+0000","last_unstale":"2024-09-10T08:46:19.193715+0000","last_undegraded":"2024-09-10T08:46:19.193715+0000","last_fullsized":"2024-09-10T08:46:19.193715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:17.730839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873847+0000","last_change":"2024-09-10T08:44:10.678254+0000","last_active":"2024-09-10T08:46:18.873847+0000","last_peered":"2024-09-10T08:46:18.873847+0000","last_clean":"2024-09-10T08:46:18.873847+0000","last_became_active":"2024-09-10T08:44:10.677891+0000","last_became_peered":"2024-09-10T08:44:10.677891+0000","last_unstale":"2024-09-10T08:46:18.873847+0000","last_undegraded":"2024-09-10T08:46:18.873847+0000","last_fullsized":"2024-09-10T08:46:18.873847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:32:43.399049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804719+0000","last_change":"2024-09-10T08:44:08.656315+0000","last_active":"2024-09-10T08:46:18.804719+0000","last_peered":"2024-09-10T08:46:18.804719+0000","last_clean":"2024-09-10T08:46:18.804719+0000","last_became_active":"2024-09-10T08:44:08.655978+0000","last_became_peered":"2024-09-10T08:44:08.655978+0000","last_unstale":"2024-09-10T08:46:18.804719+0000","last_undegraded":"2024-09-10T08:46:18.804719+0000","last_fullsized":"2024-09-10T08:46:18.804719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:15.025354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193370+0000","last_change":"2024-09-10T08:44:17.726789+0000","last_active":"2024-09-10T08:46:19.193370+0000","last_peered":"2024-09-10T08:46:19.193370+0000","last_clean":"2024-09-10T08:46:19.193370+0000","last_became_active":"2024-09-10T08:44:17.726450+0000","last_became_peered":"2024-09-10T08:44:17.726450+0000","last_unstale":"2024-09-10T08:46:19.193370+0000","last_undegraded":"2024-09-10T08:46:19.193370+0000","last_fullsized":"2024-09-10T08:46:19.193370+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:03.899822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806240+0000","last_change":"2024-09-10T08:44:10.677359+0000","last_active":"2024-09-10T08:46:18.806240+0000","last_peered":"2024-09-10T08:46:18.806240+0000","last_clean":"2024-09-10T08:46:18.806240+0000","last_became_active":"2024-09-10T08:44:10.676782+0000","last_became_peered":"2024-09-10T08:44:10.676782+0000","last_unstale":"2024-09-10T08:46:18.806240+0000","last_undegraded":"2024-09-10T08:46:18.806240+0000","last_fullsized":"2024-09-10T08:46:18.806240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:31.721714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808306+0000","last_change":"2024-09-10T08:44:08.656993+0000","last_active":"2024-09-10T08:46:18.808306+0000","last_peered":"2024-09-10T08:46:18.808306+0000","last_clean":"2024-09-10T08:46:18.808306+0000","last_became_active":"2024-09-10T08:44:08.656571+0000","last_became_peered":"2024-09-10T08:44:08.656571+0000","last_unstale":"2024-09-10T08:46:18.808306+0000","last_undegraded":"2024-09-10T08:46:18.808306+0000","last_fullsized":"2024-09-10T08:46:18.808306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:32.202740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804870+0000","last_change":"2024-09-10T08:44:17.725674+0000","last_active":"2024-09-10T08:46:18.804870+0000","last_peered":"2024-09-10T08:46:18.804870+0000","last_clean":"2024-09-10T08:46:18.804870+0000","last_became_active":"2024-09-10T08:44:17.725263+0000","last_became_peered":"2024-09-10T08:44:17.725263+0000","last_unstale":"2024-09-10T08:46:18.804870+0000","last_undegraded":"2024-09-10T08:46:18.804870+0000","last_fullsized":"2024-09-10T08:46:18.804870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:18.183157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193937+0000","last_change":"2024-09-10T08:44:10.676871+0000","last_active":"2024-09-10T08:46:19.193937+0000","last_peered":"2024-09-10T08:46:19.193937+0000","last_clean":"2024-09-10T08:46:19.193937+0000","last_became_active":"2024-09-10T08:44:10.676540+0000","last_became_peered":"2024-09-10T08:44:10.676540+0000","last_unstale":"2024-09-10T08:46:19.193937+0000","last_undegraded":"2024-09-10T08:46:19.193937+0000","last_fullsized":"2024-09-10T08:46:19.193937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:50.964100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'40","reported_seq":293,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.365469+0000","last_change":"2024-09-10T08:44:08.660113+0000","last_active":"2024-09-10T08:46:34.365469+0000","last_peered":"2024-09-10T08:46:34.365469+0000","last_clean":"2024-09-10T08:46:34.365469+0000","last_became_active":"2024-09-10T08:44:08.659644+0000","last_became_peered":"2024-09-10T08:44:08.659644+0000","last_unstale":"2024-09-10T08:46:34.365469+0000","last_undegraded":"2024-09-10T08:46:34.365469+0000","last_fullsized":"2024-09-10T08:46:34.365469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:01.605344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64228,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":214,"num_read_kb":170,"num_write":42,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803296+0000","last_change":"2024-09-10T08:44:17.723653+0000","last_active":"2024-09-10T08:46:18.803296+0000","last_peered":"2024-09-10T08:46:18.803296+0000","last_clean":"2024-09-10T08:46:18.803296+0000","last_became_active":"2024-09-10T08:44:17.723284+0000","last_became_peered":"2024-09-10T08:44:17.723284+0000","last_unstale":"2024-09-10T08:46:18.803296+0000","last_undegraded":"2024-09-10T08:46:18.803296+0000","last_fullsized":"2024-09-10T08:46:18.803296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:17.162526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193970+0000","last_change":"2024-09-10T08:44:10.681837+0000","last_active":"2024-09-10T08:46:19.193970+0000","last_peered":"2024-09-10T08:46:19.193970+0000","last_clean":"2024-09-10T08:46:19.193970+0000","last_became_active":"2024-09-10T08:44:10.681518+0000","last_became_peered":"2024-09-10T08:44:10.681518+0000","last_unstale":"2024-09-10T08:46:19.193970+0000","last_undegraded":"2024-09-10T08:46:19.193970+0000","last_fullsized":"2024-09-10T08:46:19.193970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:10.004215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804988+0000","last_change":"2024-09-10T08:44:09.050394+0000","last_active":"2024-09-10T08:46:18.804988+0000","last_peered":"2024-09-10T08:46:18.804988+0000","last_clean":"2024-09-10T08:46:18.804988+0000","last_became_active":"2024-09-10T08:44:09.049969+0000","last_became_peered":"2024-09-10T08:44:09.049969+0000","last_unstale":"2024-09-10T08:46:18.804988+0000","last_undegraded":"2024-09-10T08:46:18.804988+0000","last_fullsized":"2024-09-10T08:46:18.804988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:43.519246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194127+0000","last_change":"2024-09-10T08:44:17.720318+0000","last_active":"2024-09-10T08:46:19.194127+0000","last_peered":"2024-09-10T08:46:19.194127+0000","last_clean":"2024-09-10T08:46:19.194127+0000","last_became_active":"2024-09-10T08:44:17.719598+0000","last_became_peered":"2024-09-10T08:44:17.719598+0000","last_unstale":"2024-09-10T08:46:19.194127+0000","last_undegraded":"2024-09-10T08:46:19.194127+0000","last_fullsized":"2024-09-10T08:46:19.194127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:15:49.520324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193946+0000","last_change":"2024-09-10T08:44:10.677340+0000","last_active":"2024-09-10T08:46:19.193946+0000","last_peered":"2024-09-10T08:46:19.193946+0000","last_clean":"2024-09-10T08:46:19.193946+0000","last_became_active":"2024-09-10T08:44:10.676628+0000","last_became_peered":"2024-09-10T08:44:10.676628+0000","last_unstale":"2024-09-10T08:46:19.193946+0000","last_undegraded":"2024-09-10T08:46:19.193946+0000","last_fullsized":"2024-09-10T08:46:19.193946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:59:17.924217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874027+0000","last_change":"2024-09-10T08:44:08.660254+0000","last_active":"2024-09-10T08:46:18.874027+0000","last_peered":"2024-09-10T08:46:18.874027+0000","last_clean":"2024-09-10T08:46:18.874027+0000","last_became_active":"2024-09-10T08:44:08.659667+0000","last_became_peered":"2024-09-10T08:44:08.659667+0000","last_unstale":"2024-09-10T08:46:18.874027+0000","last_undegraded":"2024-09-10T08:46:18.874027+0000","last_fullsized":"2024-09-10T08:46:18.874027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:49:49.146867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803135+0000","last_change":"2024-09-10T08:44:17.722373+0000","last_active":"2024-09-10T08:46:18.803135+0000","last_peered":"2024-09-10T08:46:18.803135+0000","last_clean":"2024-09-10T08:46:18.803135+0000","last_became_active":"2024-09-10T08:44:17.722035+0000","last_became_peered":"2024-09-10T08:44:17.722035+0000","last_unstale":"2024-09-10T08:46:18.803135+0000","last_undegraded":"2024-09-10T08:46:18.803135+0000","last_fullsized":"2024-09-10T08:46:18.803135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:30:36.187364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194365+0000","last_change":"2024-09-10T08:44:10.672585+0000","last_active":"2024-09-10T08:46:19.194365+0000","last_peered":"2024-09-10T08:46:19.194365+0000","last_clean":"2024-09-10T08:46:19.194365+0000","last_became_active":"2024-09-10T08:44:10.671987+0000","last_became_peered":"2024-09-10T08:44:10.671987+0000","last_unstale":"2024-09-10T08:46:19.194365+0000","last_undegraded":"2024-09-10T08:46:19.194365+0000","last_fullsized":"2024-09-10T08:46:19.194365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:48.660750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'7","reported_seq":248,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.365000+0000","last_change":"2024-09-10T08:44:08.658069+0000","last_active":"2024-09-10T08:46:34.365000+0000","last_peered":"2024-09-10T08:46:34.365000+0000","last_clean":"2024-09-10T08:46:34.365000+0000","last_became_active":"2024-09-10T08:44:08.657702+0000","last_became_peered":"2024-09-10T08:44:08.657702+0000","last_unstale":"2024-09-10T08:46:34.365000+0000","last_undegraded":"2024-09-10T08:46:34.365000+0000","last_fullsized":"2024-09-10T08:46:34.365000+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:17.383267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":202,"num_read_kb":202,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192772+0000","last_change":"2024-09-10T08:44:17.717984+0000","last_active":"2024-09-10T08:46:19.192772+0000","last_peered":"2024-09-10T08:46:19.192772+0000","last_clean":"2024-09-10T08:46:19.192772+0000","last_became_active":"2024-09-10T08:44:17.717581+0000","last_became_peered":"2024-09-10T08:44:17.717581+0000","last_unstale":"2024-09-10T08:46:19.192772+0000","last_undegraded":"2024-09-10T08:46:19.192772+0000","last_fullsized":"2024-09-10T08:46:19.192772+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:25.330732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193566+0000","last_change":"2024-09-10T08:44:17.721244+0000","last_active":"2024-09-10T08:46:19.193566+0000","last_peered":"2024-09-10T08:46:19.193566+0000","last_clean":"2024-09-10T08:46:19.193566+0000","last_became_active":"2024-09-10T08:44:17.720884+0000","last_became_peered":"2024-09-10T08:44:17.720884+0000","last_unstale":"2024-09-10T08:46:19.193566+0000","last_undegraded":"2024-09-10T08:46:19.193566+0000","last_fullsized":"2024-09-10T08:46:19.193566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.769513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803279+0000","last_change":"2024-09-10T08:44:10.667558+0000","last_active":"2024-09-10T08:46:18.803279+0000","last_peered":"2024-09-10T08:46:18.803279+0000","last_clean":"2024-09-10T08:46:18.803279+0000","last_became_active":"2024-09-10T08:44:10.667214+0000","last_became_peered":"2024-09-10T08:44:10.667214+0000","last_unstale":"2024-09-10T08:46:18.803279+0000","last_undegraded":"2024-09-10T08:46:18.803279+0000","last_fullsized":"2024-09-10T08:46:18.803279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:53.630790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804697+0000","last_change":"2024-09-10T08:44:08.657930+0000","last_active":"2024-09-10T08:46:18.804697+0000","last_peered":"2024-09-10T08:46:18.804697+0000","last_clean":"2024-09-10T08:46:18.804697+0000","last_became_active":"2024-09-10T08:44:08.657611+0000","last_became_peered":"2024-09-10T08:44:08.657611+0000","last_unstale":"2024-09-10T08:46:18.804697+0000","last_undegraded":"2024-09-10T08:46:18.804697+0000","last_fullsized":"2024-09-10T08:46:18.804697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:36.923786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193910+0000","last_change":"2024-09-10T08:44:17.726106+0000","last_active":"2024-09-10T08:46:19.193910+0000","last_peered":"2024-09-10T08:46:19.193910+0000","last_clean":"2024-09-10T08:46:19.193910+0000","last_became_active":"2024-09-10T08:44:17.725683+0000","last_became_peered":"2024-09-10T08:44:17.725683+0000","last_unstale":"2024-09-10T08:46:19.193910+0000","last_undegraded":"2024-09-10T08:46:19.193910+0000","last_fullsized":"2024-09-10T08:46:19.193910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:23.392794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192727+0000","last_change":"2024-09-10T08:44:10.672204+0000","last_active":"2024-09-10T08:46:19.192727+0000","last_peered":"2024-09-10T08:46:19.192727+0000","last_clean":"2024-09-10T08:46:19.192727+0000","last_became_active":"2024-09-10T08:44:10.671898+0000","last_became_peered":"2024-09-10T08:44:10.671898+0000","last_unstale":"2024-09-10T08:46:19.192727+0000","last_undegraded":"2024-09-10T08:46:19.192727+0000","last_fullsized":"2024-09-10T08:46:19.192727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:03.737357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805979+0000","last_change":"2024-09-10T08:44:08.656620+0000","last_active":"2024-09-10T08:46:18.805979+0000","last_peered":"2024-09-10T08:46:18.805979+0000","last_clean":"2024-09-10T08:46:18.805979+0000","last_became_active":"2024-09-10T08:44:08.656306+0000","last_became_peered":"2024-09-10T08:44:08.656306+0000","last_unstale":"2024-09-10T08:46:18.805979+0000","last_undegraded":"2024-09-10T08:46:18.805979+0000","last_fullsized":"2024-09-10T08:46:18.805979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:57.081909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804682+0000","last_change":"2024-09-10T08:44:08.660178+0000","last_active":"2024-09-10T08:46:18.804682+0000","last_peered":"2024-09-10T08:46:18.804682+0000","last_clean":"2024-09-10T08:46:18.804682+0000","last_became_active":"2024-09-10T08:44:08.659839+0000","last_became_peered":"2024-09-10T08:44:08.659839+0000","last_unstale":"2024-09-10T08:46:18.804682+0000","last_undegraded":"2024-09-10T08:46:18.804682+0000","last_fullsized":"2024-09-10T08:46:18.804682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:17.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804773+0000","last_change":"2024-09-10T08:44:10.683683+0000","last_active":"2024-09-10T08:46:18.804773+0000","last_peered":"2024-09-10T08:46:18.804773+0000","last_clean":"2024-09-10T08:46:18.804773+0000","last_became_active":"2024-09-10T08:44:10.683337+0000","last_became_peered":"2024-09-10T08:44:10.683337+0000","last_unstale":"2024-09-10T08:46:18.804773+0000","last_undegraded":"2024-09-10T08:46:18.804773+0000","last_fullsized":"2024-09-10T08:46:18.804773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:54.863098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808044+0000","last_change":"2024-09-10T08:44:17.721602+0000","last_active":"2024-09-10T08:46:18.808044+0000","last_peered":"2024-09-10T08:46:18.808044+0000","last_clean":"2024-09-10T08:46:18.808044+0000","last_became_active":"2024-09-10T08:44:17.721194+0000","last_became_peered":"2024-09-10T08:44:17.721194+0000","last_unstale":"2024-09-10T08:46:18.808044+0000","last_undegraded":"2024-09-10T08:46:18.808044+0000","last_fullsized":"2024-09-10T08:46:18.808044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:39.298494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":149,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.365494+0000","last_change":"2024-09-10T08:44:08.661943+0000","last_active":"2024-09-10T08:46:34.365494+0000","last_peered":"2024-09-10T08:46:34.365494+0000","last_clean":"2024-09-10T08:46:34.365494+0000","last_became_active":"2024-09-10T08:44:08.661587+0000","last_became_peered":"2024-09-10T08:44:08.661587+0000","last_unstale":"2024-09-10T08:46:34.365494+0000","last_undegraded":"2024-09-10T08:46:34.365494+0000","last_fullsized":"2024-09-10T08:46:34.365494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:05.771641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193718+0000","last_change":"2024-09-10T08:44:10.673522+0000","last_active":"2024-09-10T08:46:19.193718+0000","last_peered":"2024-09-10T08:46:19.193718+0000","last_clean":"2024-09-10T08:46:19.193718+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:46:19.193718+0000","last_undegraded":"2024-09-10T08:46:19.193718+0000","last_fullsized":"2024-09-10T08:46:19.193718+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:03.006144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808419+0000","last_change":"2024-09-10T08:44:17.721306+0000","last_active":"2024-09-10T08:46:18.808419+0000","last_peered":"2024-09-10T08:46:18.808419+0000","last_clean":"2024-09-10T08:46:18.808419+0000","last_became_active":"2024-09-10T08:44:17.720401+0000","last_became_peered":"2024-09-10T08:44:17.720401+0000","last_unstale":"2024-09-10T08:46:18.808419+0000","last_undegraded":"2024-09-10T08:46:18.808419+0000","last_fullsized":"2024-09-10T08:46:18.808419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:11.589371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193166+0000","last_change":"2024-09-10T08:44:10.678425+0000","last_active":"2024-09-10T08:46:19.193166+0000","last_peered":"2024-09-10T08:46:19.193166+0000","last_clean":"2024-09-10T08:46:19.193166+0000","last_became_active":"2024-09-10T08:44:10.677187+0000","last_became_peered":"2024-09-10T08:44:10.677187+0000","last_unstale":"2024-09-10T08:46:19.193166+0000","last_undegraded":"2024-09-10T08:46:19.193166+0000","last_fullsized":"2024-09-10T08:46:19.193166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:06.527530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808431+0000","last_change":"2024-09-10T08:44:10.674178+0000","last_active":"2024-09-10T08:46:18.808431+0000","last_peered":"2024-09-10T08:46:18.808431+0000","last_clean":"2024-09-10T08:46:18.808431+0000","last_became_active":"2024-09-10T08:44:10.673843+0000","last_became_peered":"2024-09-10T08:44:10.673843+0000","last_unstale":"2024-09-10T08:46:18.808431+0000","last_undegraded":"2024-09-10T08:46:18.808431+0000","last_fullsized":"2024-09-10T08:46:18.808431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:35.023663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.807988+0000","last_change":"2024-09-10T08:44:10.673271+0000","last_active":"2024-09-10T08:46:18.807988+0000","last_peered":"2024-09-10T08:46:18.807988+0000","last_clean":"2024-09-10T08:46:18.807988+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:46:18.807988+0000","last_undegraded":"2024-09-10T08:46:18.807988+0000","last_fullsized":"2024-09-10T08:46:18.807988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:21:26.606820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193502+0000","last_change":"2024-09-10T08:44:10.671348+0000","last_active":"2024-09-10T08:46:19.193502+0000","last_peered":"2024-09-10T08:46:19.193502+0000","last_clean":"2024-09-10T08:46:19.193502+0000","last_became_active":"2024-09-10T08:44:10.670623+0000","last_became_peered":"2024-09-10T08:44:10.670623+0000","last_unstale":"2024-09-10T08:46:19.193502+0000","last_undegraded":"2024-09-10T08:46:19.193502+0000","last_fullsized":"2024-09-10T08:46:19.193502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:06.145694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806400+0000","last_change":"2024-09-10T08:44:10.666011+0000","last_active":"2024-09-10T08:46:18.806400+0000","last_peered":"2024-09-10T08:46:18.806400+0000","last_clean":"2024-09-10T08:46:18.806400+0000","last_became_active":"2024-09-10T08:44:10.665664+0000","last_became_peered":"2024-09-10T08:44:10.665664+0000","last_unstale":"2024-09-10T08:46:18.806400+0000","last_undegraded":"2024-09-10T08:46:18.806400+0000","last_fullsized":"2024-09-10T08:46:18.806400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:31.542102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193792+0000","last_change":"2024-09-10T08:44:10.675960+0000","last_active":"2024-09-10T08:46:19.193792+0000","last_peered":"2024-09-10T08:46:19.193792+0000","last_clean":"2024-09-10T08:46:19.193792+0000","last_became_active":"2024-09-10T08:44:10.675587+0000","last_became_peered":"2024-09-10T08:44:10.675587+0000","last_unstale":"2024-09-10T08:46:19.193792+0000","last_undegraded":"2024-09-10T08:46:19.193792+0000","last_fullsized":"2024-09-10T08:46:19.193792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:54.634619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808734+0000","last_change":"2024-09-10T08:44:10.681094+0000","last_active":"2024-09-10T08:46:18.808734+0000","last_peered":"2024-09-10T08:46:18.808734+0000","last_clean":"2024-09-10T08:46:18.808734+0000","last_became_active":"2024-09-10T08:44:10.678585+0000","last_became_peered":"2024-09-10T08:44:10.678585+0000","last_unstale":"2024-09-10T08:46:18.808734+0000","last_undegraded":"2024-09-10T08:46:18.808734+0000","last_fullsized":"2024-09-10T08:46:18.808734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:14.951949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193627+0000","last_change":"2024-09-10T08:44:10.675473+0000","last_active":"2024-09-10T08:46:19.193627+0000","last_peered":"2024-09-10T08:46:19.193627+0000","last_clean":"2024-09-10T08:46:19.193627+0000","last_became_active":"2024-09-10T08:44:10.674155+0000","last_became_peered":"2024-09-10T08:44:10.674155+0000","last_unstale":"2024-09-10T08:46:19.193627+0000","last_undegraded":"2024-09-10T08:46:19.193627+0000","last_fullsized":"2024-09-10T08:46:19.193627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:55.940235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804316+0000","last_change":"2024-09-10T08:44:10.680162+0000","last_active":"2024-09-10T08:46:18.804316+0000","last_peered":"2024-09-10T08:46:18.804316+0000","last_clean":"2024-09-10T08:46:18.804316+0000","last_became_active":"2024-09-10T08:44:10.679800+0000","last_became_peered":"2024-09-10T08:44:10.679800+0000","last_unstale":"2024-09-10T08:46:18.804316+0000","last_undegraded":"2024-09-10T08:46:18.804316+0000","last_fullsized":"2024-09-10T08:46:18.804316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:12.319813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192932+0000","last_change":"2024-09-10T08:44:10.674724+0000","last_active":"2024-09-10T08:46:19.192932+0000","last_peered":"2024-09-10T08:46:19.192932+0000","last_clean":"2024-09-10T08:46:19.192932+0000","last_became_active":"2024-09-10T08:44:10.674389+0000","last_became_peered":"2024-09-10T08:44:10.674389+0000","last_unstale":"2024-09-10T08:46:19.192932+0000","last_undegraded":"2024-09-10T08:46:19.192932+0000","last_fullsized":"2024-09-10T08:46:19.192932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:52.123186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193888+0000","last_change":"2024-09-10T08:44:10.672805+0000","last_active":"2024-09-10T08:46:19.193888+0000","last_peered":"2024-09-10T08:46:19.193888+0000","last_clean":"2024-09-10T08:46:19.193888+0000","last_became_active":"2024-09-10T08:44:10.672508+0000","last_became_peered":"2024-09-10T08:44:10.672508+0000","last_unstale":"2024-09-10T08:46:19.193888+0000","last_undegraded":"2024-09-10T08:46:19.193888+0000","last_fullsized":"2024-09-10T08:46:19.193888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:48.558633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806228+0000","last_change":"2024-09-10T08:44:10.677504+0000","last_active":"2024-09-10T08:46:18.806228+0000","last_peered":"2024-09-10T08:46:18.806228+0000","last_clean":"2024-09-10T08:46:18.806228+0000","last_became_active":"2024-09-10T08:44:10.676822+0000","last_became_peered":"2024-09-10T08:44:10.676822+0000","last_unstale":"2024-09-10T08:46:18.806228+0000","last_undegraded":"2024-09-10T08:46:18.806228+0000","last_fullsized":"2024-09-10T08:46:18.806228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:05:48.293333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193457+0000","last_change":"2024-09-10T08:45:18.268208+0000","last_active":"2024-09-10T08:46:19.193457+0000","last_peered":"2024-09-10T08:46:19.193457+0000","last_clean":"2024-09-10T08:46:19.193457+0000","last_became_active":"2024-09-10T08:45:18.267700+0000","last_became_peered":"2024-09-10T08:45:18.267700+0000","last_unstale":"2024-09-10T08:46:19.193457+0000","last_undegraded":"2024-09-10T08:46:19.193457+0000","last_fullsized":"2024-09-10T08:46:19.193457+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:54.487624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808678+0000","last_change":"2024-09-10T08:44:10.670615+0000","last_active":"2024-09-10T08:46:18.808678+0000","last_peered":"2024-09-10T08:46:18.808678+0000","last_clean":"2024-09-10T08:46:18.808678+0000","last_became_active":"2024-09-10T08:44:10.670032+0000","last_became_peered":"2024-09-10T08:44:10.670032+0000","last_unstale":"2024-09-10T08:46:18.808678+0000","last_undegraded":"2024-09-10T08:46:18.808678+0000","last_fullsized":"2024-09-10T08:46:18.808678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:31.078788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803799+0000","last_change":"2024-09-10T08:44:10.672367+0000","last_active":"2024-09-10T08:46:18.803799+0000","last_peered":"2024-09-10T08:46:18.803799+0000","last_clean":"2024-09-10T08:46:18.803799+0000","last_became_active":"2024-09-10T08:44:10.672040+0000","last_became_peered":"2024-09-10T08:44:10.672040+0000","last_unstale":"2024-09-10T08:46:18.803799+0000","last_undegraded":"2024-09-10T08:46:18.803799+0000","last_fullsized":"2024-09-10T08:46:18.803799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:10.634708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194226+0000","last_change":"2024-09-10T08:44:10.676375+0000","last_active":"2024-09-10T08:46:19.194226+0000","last_peered":"2024-09-10T08:46:19.194226+0000","last_clean":"2024-09-10T08:46:19.194226+0000","last_became_active":"2024-09-10T08:44:10.676022+0000","last_became_peered":"2024-09-10T08:44:10.676022+0000","last_unstale":"2024-09-10T08:46:19.194226+0000","last_undegraded":"2024-09-10T08:46:19.194226+0000","last_fullsized":"2024-09-10T08:46:19.194226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:02.681954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193408+0000","last_change":"2024-09-10T08:44:10.673378+0000","last_active":"2024-09-10T08:46:19.193408+0000","last_peered":"2024-09-10T08:46:19.193408+0000","last_clean":"2024-09-10T08:46:19.193408+0000","last_became_active":"2024-09-10T08:44:10.673089+0000","last_became_peered":"2024-09-10T08:44:10.673089+0000","last_unstale":"2024-09-10T08:46:19.193408+0000","last_undegraded":"2024-09-10T08:46:19.193408+0000","last_fullsized":"2024-09-10T08:46:19.193408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:36:19.258651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192898+0000","last_change":"2024-09-10T08:44:10.671557+0000","last_active":"2024-09-10T08:46:19.192898+0000","last_peered":"2024-09-10T08:46:19.192898+0000","last_clean":"2024-09-10T08:46:19.192898+0000","last_became_active":"2024-09-10T08:44:10.671248+0000","last_became_peered":"2024-09-10T08:44:10.671248+0000","last_unstale":"2024-09-10T08:46:19.192898+0000","last_undegraded":"2024-09-10T08:46:19.192898+0000","last_fullsized":"2024-09-10T08:46:19.192898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:54.261525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193104+0000","last_change":"2024-09-10T08:44:10.674672+0000","last_active":"2024-09-10T08:46:19.193104+0000","last_peered":"2024-09-10T08:46:19.193104+0000","last_clean":"2024-09-10T08:46:19.193104+0000","last_became_active":"2024-09-10T08:44:10.673415+0000","last_became_peered":"2024-09-10T08:44:10.673415+0000","last_unstale":"2024-09-10T08:46:19.193104+0000","last_undegraded":"2024-09-10T08:46:19.193104+0000","last_fullsized":"2024-09-10T08:46:19.193104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:59:14.179664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806192+0000","last_change":"2024-09-10T08:44:10.675293+0000","last_active":"2024-09-10T08:46:18.806192+0000","last_peered":"2024-09-10T08:46:18.806192+0000","last_clean":"2024-09-10T08:46:18.806192+0000","last_became_active":"2024-09-10T08:44:10.674924+0000","last_became_peered":"2024-09-10T08:44:10.674924+0000","last_unstale":"2024-09-10T08:46:18.806192+0000","last_undegraded":"2024-09-10T08:46:18.806192+0000","last_fullsized":"2024-09-10T08:46:18.806192+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:56.027320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194671+0000","last_change":"2024-09-10T08:44:10.674605+0000","last_active":"2024-09-10T08:46:19.194671+0000","last_peered":"2024-09-10T08:46:19.194671+0000","last_clean":"2024-09-10T08:46:19.194671+0000","last_became_active":"2024-09-10T08:44:10.674057+0000","last_became_peered":"2024-09-10T08:44:10.674057+0000","last_unstale":"2024-09-10T08:46:19.194671+0000","last_undegraded":"2024-09-10T08:46:19.194671+0000","last_fullsized":"2024-09-10T08:46:19.194671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:09.473003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874065+0000","last_change":"2024-09-10T08:44:10.677410+0000","last_active":"2024-09-10T08:46:18.874065+0000","last_peered":"2024-09-10T08:46:18.874065+0000","last_clean":"2024-09-10T08:46:18.874065+0000","last_became_active":"2024-09-10T08:44:10.677109+0000","last_became_peered":"2024-09-10T08:44:10.677109+0000","last_unstale":"2024-09-10T08:46:18.874065+0000","last_undegraded":"2024-09-10T08:46:18.874065+0000","last_fullsized":"2024-09-10T08:46:18.874065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:45.455425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805091+0000","last_change":"2024-09-10T08:44:10.669706+0000","last_active":"2024-09-10T08:46:18.805091+0000","last_peered":"2024-09-10T08:46:18.805091+0000","last_clean":"2024-09-10T08:46:18.805091+0000","last_became_active":"2024-09-10T08:44:10.669440+0000","last_became_peered":"2024-09-10T08:44:10.669440+0000","last_unstale":"2024-09-10T08:46:18.805091+0000","last_undegraded":"2024-09-10T08:46:18.805091+0000","last_fullsized":"2024-09-10T08:46:18.805091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:31:55.610651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194758+0000","last_change":"2024-09-10T08:44:10.676482+0000","last_active":"2024-09-10T08:46:19.194758+0000","last_peered":"2024-09-10T08:46:19.194758+0000","last_clean":"2024-09-10T08:46:19.194758+0000","last_became_active":"2024-09-10T08:44:10.675983+0000","last_became_peered":"2024-09-10T08:44:10.675983+0000","last_unstale":"2024-09-10T08:46:19.194758+0000","last_undegraded":"2024-09-10T08:46:19.194758+0000","last_fullsized":"2024-09-10T08:46:19.194758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:01.269283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192653+0000","last_change":"2024-09-10T08:44:10.675681+0000","last_active":"2024-09-10T08:46:19.192653+0000","last_peered":"2024-09-10T08:46:19.192653+0000","last_clean":"2024-09-10T08:46:19.192653+0000","last_became_active":"2024-09-10T08:44:10.675357+0000","last_became_peered":"2024-09-10T08:44:10.675357+0000","last_unstale":"2024-09-10T08:46:19.192653+0000","last_undegraded":"2024-09-10T08:46:19.192653+0000","last_fullsized":"2024-09-10T08:46:19.192653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:26.138333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873467+0000","last_change":"2024-09-10T08:44:10.680819+0000","last_active":"2024-09-10T08:46:18.873467+0000","last_peered":"2024-09-10T08:46:18.873467+0000","last_clean":"2024-09-10T08:46:18.873467+0000","last_became_active":"2024-09-10T08:44:10.680491+0000","last_became_peered":"2024-09-10T08:44:10.680491+0000","last_unstale":"2024-09-10T08:46:18.873467+0000","last_undegraded":"2024-09-10T08:46:18.873467+0000","last_fullsized":"2024-09-10T08:46:18.873467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:17.044776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804562+0000","last_change":"2024-09-10T08:44:10.667376+0000","last_active":"2024-09-10T08:46:18.804562+0000","last_peered":"2024-09-10T08:46:18.804562+0000","last_clean":"2024-09-10T08:46:18.804562+0000","last_became_active":"2024-09-10T08:44:10.666648+0000","last_became_peered":"2024-09-10T08:44:10.666648+0000","last_unstale":"2024-09-10T08:46:18.804562+0000","last_undegraded":"2024-09-10T08:46:18.804562+0000","last_fullsized":"2024-09-10T08:46:18.804562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:33:12.017550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193207+0000","last_change":"2024-09-10T08:44:10.668665+0000","last_active":"2024-09-10T08:46:19.193207+0000","last_peered":"2024-09-10T08:46:19.193207+0000","last_clean":"2024-09-10T08:46:19.193207+0000","last_became_active":"2024-09-10T08:44:10.666031+0000","last_became_peered":"2024-09-10T08:44:10.666031+0000","last_unstale":"2024-09-10T08:46:19.193207+0000","last_undegraded":"2024-09-10T08:46:19.193207+0000","last_fullsized":"2024-09-10T08:46:19.193207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:39:53.991463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194458+0000","last_change":"2024-09-10T08:44:10.676495+0000","last_active":"2024-09-10T08:46:19.194458+0000","last_peered":"2024-09-10T08:46:19.194458+0000","last_clean":"2024-09-10T08:46:19.194458+0000","last_became_active":"2024-09-10T08:44:10.676087+0000","last_became_peered":"2024-09-10T08:44:10.676087+0000","last_unstale":"2024-09-10T08:46:19.194458+0000","last_undegraded":"2024-09-10T08:46:19.194458+0000","last_fullsized":"2024-09-10T08:46:19.194458+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:41.828157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193138+0000","last_change":"2024-09-10T08:44:10.672835+0000","last_active":"2024-09-10T08:46:19.193138+0000","last_peered":"2024-09-10T08:46:19.193138+0000","last_clean":"2024-09-10T08:46:19.193138+0000","last_became_active":"2024-09-10T08:44:10.672537+0000","last_became_peered":"2024-09-10T08:44:10.672537+0000","last_unstale":"2024-09-10T08:46:19.193138+0000","last_undegraded":"2024-09-10T08:46:19.193138+0000","last_fullsized":"2024-09-10T08:46:19.193138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:06.121460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193568+0000","last_change":"2024-09-10T08:44:10.675968+0000","last_active":"2024-09-10T08:46:19.193568+0000","last_peered":"2024-09-10T08:46:19.193568+0000","last_clean":"2024-09-10T08:46:19.193568+0000","last_became_active":"2024-09-10T08:44:10.675628+0000","last_became_peered":"2024-09-10T08:44:10.675628+0000","last_unstale":"2024-09-10T08:46:19.193568+0000","last_undegraded":"2024-09-10T08:46:19.193568+0000","last_fullsized":"2024-09-10T08:46:19.193568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:40.261336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194091+0000","last_change":"2024-09-10T08:44:10.673151+0000","last_active":"2024-09-10T08:46:19.194091+0000","last_peered":"2024-09-10T08:46:19.194091+0000","last_clean":"2024-09-10T08:46:19.194091+0000","last_became_active":"2024-09-10T08:44:10.672833+0000","last_became_peered":"2024-09-10T08:44:10.672833+0000","last_unstale":"2024-09-10T08:46:19.194091+0000","last_undegraded":"2024-09-10T08:46:19.194091+0000","last_fullsized":"2024-09-10T08:46:19.194091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:38.556222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73164,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1067,"num_read_kb":614,"num_write":273,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":228120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514539,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":523370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743472,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1753088,"data_stored":1173880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906997,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":526367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135930,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":522649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364863,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1120240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528389,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":518270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36988,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708164,"statfs":{"total":95995035648,"available":95957159936,"internally_reserved":0,"allocated":1683456,"data_stored":1114832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1064960,"data_stored":521849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":517483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":584777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1126400,"data_stored":583528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":520260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:46:40.722 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T08:46:40.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:40 smithi174 ceph-mon[34068]: pgmap v363: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T08:46:40.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:40 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/17995658' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:40.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.836+0000 7fac4c61e640 1 -- 172.21.15.172:0/2614506238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac441036e0 msgr2=0x7fac44109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:40.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.836+0000 7fac4c61e640 1 --2- 172.21.15.172:0/2614506238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac441036e0 0x7fac44109f70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009900 tx=0x7fac3c02a410 comp rx=0 tx=0).stop 2024-09-10T08:46:40.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.836+0000 7fac4c61e640 1 -- 172.21.15.172:0/2614506238 shutdown_connections 2024-09-10T08:46:40.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.836+0000 7fac4c61e640 1 --2- 172.21.15.172:0/2614506238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac441036e0 0x7fac44109f70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.836+0000 7fac4c61e640 1 --2- 172.21.15.172:0/2614506238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac44102d20 0x7fac441031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.836+0000 7fac4c61e640 1 --2- 172.21.15.172:0/2614506238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac44101b30 0x7fac44101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.836+0000 7fac4c61e640 1 -- 172.21.15.172:0/2614506238 >> 172.21.15.172:0/2614506238 conn(0x7fac440fd2b0 msgr2=0x7fac440ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:40.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.836+0000 7fac4c61e640 1 -- 172.21.15.172:0/2614506238 shutdown_connections 2024-09-10T08:46:40.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.836+0000 7fac4c61e640 1 -- 172.21.15.172:0/2614506238 wait complete. 2024-09-10T08:46:40.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4c61e640 1 Processor -- start 2024-09-10T08:46:40.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4c61e640 1 -- start start 2024-09-10T08:46:40.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4c61e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac44101b30 0x7fac44075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4c61e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac44102d20 0x7fac440745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4c61e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac441036e0 0x7fac44074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4c61e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fac4410d660 con 0x7fac44101b30 2024-09-10T08:46:40.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4c61e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac4410d4e0 con 0x7fac441036e0 2024-09-10T08:46:40.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4c61e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fac4410d7e0 con 0x7fac44102d20 2024-09-10T08:46:40.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4a3ba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac44101b30 0x7fac44075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4a3ba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac44101b30 0x7fac44075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33278/0 (socket says 172.21.15.172:33278) 2024-09-10T08:46:40.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4a3ba640 1 -- 172.21.15.172:0/338018409 learned_addr learned my addr 172.21.15.172:0/338018409 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:40.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4abbb640 1 --2- 172.21.15.172:0/338018409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac441036e0 0x7fac44074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac49bb9640 1 --2- 172.21.15.172:0/338018409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac44102d20 0x7fac440745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4a3ba640 1 -- 172.21.15.172:0/338018409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac44102d20 msgr2=0x7fac440745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:40.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4a3ba640 1 --2- 172.21.15.172:0/338018409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac44102d20 0x7fac440745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4a3ba640 1 -- 172.21.15.172:0/338018409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac441036e0 msgr2=0x7fac44074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:40.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4a3ba640 1 --2- 172.21.15.172:0/338018409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac441036e0 0x7fac44074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:40.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4a3ba640 1 -- 172.21.15.172:0/338018409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac440753b0 con 0x7fac44101b30 2024-09-10T08:46:40.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.837+0000 7fac4a3ba640 1 --2- 172.21.15.172:0/338018409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac44101b30 0x7fac44075fa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fac3400e9c0 tx=0x7fac3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:40.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.838+0000 7fac437fe640 1 -- 172.21.15.172:0/338018409 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3400cd80 con 0x7fac44101b30 2024-09-10T08:46:40.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.838+0000 7fac437fe640 1 -- 172.21.15.172:0/338018409 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac340035f0 con 0x7fac44101b30 2024-09-10T08:46:40.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.838+0000 7fac437fe640 1 -- 172.21.15.172:0/338018409 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac34010690 con 0x7fac44101b30 2024-09-10T08:46:40.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.838+0000 7fac4c61e640 1 -- 172.21.15.172:0/338018409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac441a1e60 con 0x7fac44101b30 2024-09-10T08:46:40.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.838+0000 7fac4c61e640 1 -- 172.21.15.172:0/338018409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac441a2270 con 0x7fac44101b30 2024-09-10T08:46:40.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.839+0000 7fac4c61e640 1 -- 172.21.15.172:0/338018409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac44108f20 con 0x7fac44101b30 2024-09-10T08:46:40.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.840+0000 7fac437fe640 1 -- 172.21.15.172:0/338018409 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fac340037a0 con 0x7fac44101b30 2024-09-10T08:46:40.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.840+0000 7fac437fe640 1 --2- 172.21.15.172:0/338018409 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fac2c0a2c10 0x7fac2c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:40.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.840+0000 7fac49bb9640 1 --2- 172.21.15.172:0/338018409 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fac2c0a2c10 0x7fac2c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:40.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.840+0000 7fac437fe640 1 -- 172.21.15.172:0/338018409 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fac340cf8c0 con 0x7fac44101b30 2024-09-10T08:46:40.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.841+0000 7fac49bb9640 1 --2- 172.21.15.172:0/338018409 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fac2c0a2c10 0x7fac2c0a50d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fac44075d10 tx=0x7fac38009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:40.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:40.844+0000 7fac437fe640 1 -- 172.21.15.172:0/338018409 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac3409c4c0 con 0x7fac44101b30 2024-09-10T08:46:41.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.007+0000 7fac4c61e640 1 -- 172.21.15.172:0/338018409 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac44062630 con 0x7fac2c0a2c10 2024-09-10T08:46:41.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.015+0000 7fac437fe640 1 -- 172.21.15.172:0/338018409 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340551 (secure 0 0 0) 0x7fac44062630 con 0x7fac2c0a2c10 2024-09-10T08:46:41.020 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:41.022 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-09-10T08:46:41.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.018+0000 7fac4c61e640 1 -- 172.21.15.172:0/338018409 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fac2c0a2c10 msgr2=0x7fac2c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:41.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.018+0000 7fac4c61e640 1 --2- 172.21.15.172:0/338018409 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fac2c0a2c10 0x7fac2c0a50d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fac44075d10 tx=0x7fac38009500 comp rx=0 tx=0).stop 2024-09-10T08:46:41.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.018+0000 7fac4c61e640 1 -- 172.21.15.172:0/338018409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac44101b30 msgr2=0x7fac44075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:41.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.018+0000 7fac4c61e640 1 --2- 172.21.15.172:0/338018409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac44101b30 0x7fac44075fa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fac3400e9c0 tx=0x7fac3400ee80 comp rx=0 tx=0).stop 2024-09-10T08:46:41.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.018+0000 7fac4c61e640 1 -- 172.21.15.172:0/338018409 shutdown_connections 2024-09-10T08:46:41.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.018+0000 7fac4c61e640 1 --2- 172.21.15.172:0/338018409 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fac2c0a2c10 0x7fac2c0a50d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:41.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.018+0000 7fac4c61e640 1 --2- 172.21.15.172:0/338018409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac441036e0 0x7fac44074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:41.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.019+0000 7fac4c61e640 1 --2- 172.21.15.172:0/338018409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac44102d20 0x7fac440745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:41.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.019+0000 7fac4c61e640 1 --2- 172.21.15.172:0/338018409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac44101b30 0x7fac44075fa0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:41.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.019+0000 7fac4c61e640 1 -- 172.21.15.172:0/338018409 >> 172.21.15.172:0/338018409 conn(0x7fac440fd2b0 msgr2=0x7fac441060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:41.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.019+0000 7fac4c61e640 1 -- 172.21.15.172:0/338018409 shutdown_connections 2024-09-10T08:46:41.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.019+0000 7fac4c61e640 1 -- 172.21.15.172:0/338018409 wait complete. 2024-09-10T08:46:41.030 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-10T08:46:39.164177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663766,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1193,"num_read_kb":723,"num_write":456,"num_write_kb":3101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376264,"kb_used_data":14528,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565560,"statfs":{"total":1151940427776,"available":1151555133440,"internally_reserved":0,"allocated":14876672,"data_stored":8227505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2475,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":59,"num_write":18,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003231"},"pg_stats":[{"pgid":"2.1f","version":"92'14","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.363708+0000","last_change":"2024-09-10T08:44:08.657737+0000","last_active":"2024-09-10T08:46:34.363708+0000","last_peered":"2024-09-10T08:46:34.363708+0000","last_clean":"2024-09-10T08:46:34.363708+0000","last_became_active":"2024-09-10T08:44:08.657473+0000","last_became_peered":"2024-09-10T08:44:08.657473+0000","last_unstale":"2024-09-10T08:46:34.363708+0000","last_undegraded":"2024-09-10T08:46:34.363708+0000","last_fullsized":"2024-09-10T08:46:34.363708+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:40.757343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193599+0000","last_change":"2024-09-10T08:44:10.675090+0000","last_active":"2024-09-10T08:46:19.193599+0000","last_peered":"2024-09-10T08:46:19.193599+0000","last_clean":"2024-09-10T08:46:19.193599+0000","last_became_active":"2024-09-10T08:44:10.674612+0000","last_became_peered":"2024-09-10T08:44:10.674612+0000","last_unstale":"2024-09-10T08:46:19.193599+0000","last_undegraded":"2024-09-10T08:46:19.193599+0000","last_fullsized":"2024-09-10T08:46:19.193599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:46.821080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.807904+0000","last_change":"2024-09-10T08:44:17.721559+0000","last_active":"2024-09-10T08:46:18.807904+0000","last_peered":"2024-09-10T08:46:18.807904+0000","last_clean":"2024-09-10T08:46:18.807904+0000","last_became_active":"2024-09-10T08:44:17.721182+0000","last_became_peered":"2024-09-10T08:44:17.721182+0000","last_unstale":"2024-09-10T08:46:18.807904+0000","last_undegraded":"2024-09-10T08:46:18.807904+0000","last_fullsized":"2024-09-10T08:46:18.807904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:18.791489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193663+0000","last_change":"2024-09-10T08:44:09.049171+0000","last_active":"2024-09-10T08:46:19.193663+0000","last_peered":"2024-09-10T08:46:19.193663+0000","last_clean":"2024-09-10T08:46:19.193663+0000","last_became_active":"2024-09-10T08:44:09.048806+0000","last_became_peered":"2024-09-10T08:44:09.048806+0000","last_unstale":"2024-09-10T08:46:19.193663+0000","last_undegraded":"2024-09-10T08:46:19.193663+0000","last_fullsized":"2024-09-10T08:46:19.193663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:21.023460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193866+0000","last_change":"2024-09-10T08:44:10.677569+0000","last_active":"2024-09-10T08:46:19.193866+0000","last_peered":"2024-09-10T08:46:19.193866+0000","last_clean":"2024-09-10T08:46:19.193866+0000","last_became_active":"2024-09-10T08:44:10.677048+0000","last_became_peered":"2024-09-10T08:44:10.677048+0000","last_unstale":"2024-09-10T08:46:19.193866+0000","last_undegraded":"2024-09-10T08:46:19.193866+0000","last_fullsized":"2024-09-10T08:46:19.193866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:35.523063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193330+0000","last_change":"2024-09-10T08:44:17.720138+0000","last_active":"2024-09-10T08:46:19.193330+0000","last_peered":"2024-09-10T08:46:19.193330+0000","last_clean":"2024-09-10T08:46:19.193330+0000","last_became_active":"2024-09-10T08:44:17.719344+0000","last_became_peered":"2024-09-10T08:44:17.719344+0000","last_unstale":"2024-09-10T08:46:19.193330+0000","last_undegraded":"2024-09-10T08:46:19.193330+0000","last_fullsized":"2024-09-10T08:46:19.193330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:11.758058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194115+0000","last_change":"2024-09-10T08:44:08.660099+0000","last_active":"2024-09-10T08:46:19.194115+0000","last_peered":"2024-09-10T08:46:19.194115+0000","last_clean":"2024-09-10T08:46:19.194115+0000","last_became_active":"2024-09-10T08:44:08.659745+0000","last_became_peered":"2024-09-10T08:44:08.659745+0000","last_unstale":"2024-09-10T08:46:19.194115+0000","last_undegraded":"2024-09-10T08:46:19.194115+0000","last_fullsized":"2024-09-10T08:46:19.194115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:43.960717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193698+0000","last_change":"2024-09-10T08:44:10.675583+0000","last_active":"2024-09-10T08:46:19.193698+0000","last_peered":"2024-09-10T08:46:19.193698+0000","last_clean":"2024-09-10T08:46:19.193698+0000","last_became_active":"2024-09-10T08:44:10.675296+0000","last_became_peered":"2024-09-10T08:44:10.675296+0000","last_unstale":"2024-09-10T08:46:19.193698+0000","last_undegraded":"2024-09-10T08:46:19.193698+0000","last_fullsized":"2024-09-10T08:46:19.193698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:31:51.147862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194363+0000","last_change":"2024-09-10T08:44:17.720104+0000","last_active":"2024-09-10T08:46:19.194363+0000","last_peered":"2024-09-10T08:46:19.194363+0000","last_clean":"2024-09-10T08:46:19.194363+0000","last_became_active":"2024-09-10T08:44:17.719607+0000","last_became_peered":"2024-09-10T08:44:17.719607+0000","last_unstale":"2024-09-10T08:46:19.194363+0000","last_undegraded":"2024-09-10T08:46:19.194363+0000","last_fullsized":"2024-09-10T08:46:19.194363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:52.976717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193719+0000","last_change":"2024-09-10T08:44:08.658588+0000","last_active":"2024-09-10T08:46:19.193719+0000","last_peered":"2024-09-10T08:46:19.193719+0000","last_clean":"2024-09-10T08:46:19.193719+0000","last_became_active":"2024-09-10T08:44:08.658053+0000","last_became_peered":"2024-09-10T08:44:08.658053+0000","last_unstale":"2024-09-10T08:46:19.193719+0000","last_undegraded":"2024-09-10T08:46:19.193719+0000","last_fullsized":"2024-09-10T08:46:19.193719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:26.744943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804707+0000","last_change":"2024-09-10T08:44:10.681978+0000","last_active":"2024-09-10T08:46:18.804707+0000","last_peered":"2024-09-10T08:46:18.804707+0000","last_clean":"2024-09-10T08:46:18.804707+0000","last_became_active":"2024-09-10T08:44:10.681600+0000","last_became_peered":"2024-09-10T08:44:10.681600+0000","last_unstale":"2024-09-10T08:46:18.804707+0000","last_undegraded":"2024-09-10T08:46:18.804707+0000","last_fullsized":"2024-09-10T08:46:18.804707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:37.190576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193813+0000","last_change":"2024-09-10T08:44:17.719784+0000","last_active":"2024-09-10T08:46:19.193813+0000","last_peered":"2024-09-10T08:46:19.193813+0000","last_clean":"2024-09-10T08:46:19.193813+0000","last_became_active":"2024-09-10T08:44:17.719182+0000","last_became_peered":"2024-09-10T08:44:17.719182+0000","last_unstale":"2024-09-10T08:46:19.193813+0000","last_undegraded":"2024-09-10T08:46:19.193813+0000","last_fullsized":"2024-09-10T08:46:19.193813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:15.237772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873709+0000","last_change":"2024-09-10T08:44:10.679069+0000","last_active":"2024-09-10T08:46:18.873709+0000","last_peered":"2024-09-10T08:46:18.873709+0000","last_clean":"2024-09-10T08:46:18.873709+0000","last_became_active":"2024-09-10T08:44:10.678597+0000","last_became_peered":"2024-09-10T08:44:10.678597+0000","last_unstale":"2024-09-10T08:46:18.873709+0000","last_undegraded":"2024-09-10T08:46:18.873709+0000","last_fullsized":"2024-09-10T08:46:18.873709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:52.693906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194153+0000","last_change":"2024-09-10T08:44:08.660886+0000","last_active":"2024-09-10T08:46:19.194153+0000","last_peered":"2024-09-10T08:46:19.194153+0000","last_clean":"2024-09-10T08:46:19.194153+0000","last_became_active":"2024-09-10T08:44:08.660478+0000","last_became_peered":"2024-09-10T08:44:08.660478+0000","last_unstale":"2024-09-10T08:46:19.194153+0000","last_undegraded":"2024-09-10T08:46:19.194153+0000","last_fullsized":"2024-09-10T08:46:19.194153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:41.785531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194323+0000","last_change":"2024-09-10T08:44:17.761929+0000","last_active":"2024-09-10T08:46:19.194323+0000","last_peered":"2024-09-10T08:46:19.194323+0000","last_clean":"2024-09-10T08:46:19.194323+0000","last_became_active":"2024-09-10T08:44:17.761440+0000","last_became_peered":"2024-09-10T08:44:17.761440+0000","last_unstale":"2024-09-10T08:46:19.194323+0000","last_undegraded":"2024-09-10T08:46:19.194323+0000","last_fullsized":"2024-09-10T08:46:19.194323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:37.817759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192895+0000","last_change":"2024-09-10T08:44:10.674071+0000","last_active":"2024-09-10T08:46:19.192895+0000","last_peered":"2024-09-10T08:46:19.192895+0000","last_clean":"2024-09-10T08:46:19.192895+0000","last_became_active":"2024-09-10T08:44:10.673770+0000","last_became_peered":"2024-09-10T08:44:10.673770+0000","last_unstale":"2024-09-10T08:46:19.192895+0000","last_undegraded":"2024-09-10T08:46:19.192895+0000","last_fullsized":"2024-09-10T08:46:19.192895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:44:19.944557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.231938+0000","last_change":"2024-09-10T08:45:18.264060+0000","last_active":"2024-09-10T08:46:34.231938+0000","last_peered":"2024-09-10T08:46:34.231938+0000","last_clean":"2024-09-10T08:46:34.231938+0000","last_became_active":"2024-09-10T08:45:18.255983+0000","last_became_peered":"2024-09-10T08:45:18.255983+0000","last_unstale":"2024-09-10T08:46:34.231938+0000","last_undegraded":"2024-09-10T08:46:34.231938+0000","last_fullsized":"2024-09-10T08:46:34.231938+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:10.810491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194248+0000","last_change":"2024-09-10T08:44:17.722877+0000","last_active":"2024-09-10T08:46:19.194248+0000","last_peered":"2024-09-10T08:46:19.194248+0000","last_clean":"2024-09-10T08:46:19.194248+0000","last_became_active":"2024-09-10T08:44:17.722046+0000","last_became_peered":"2024-09-10T08:44:17.722046+0000","last_unstale":"2024-09-10T08:46:19.194248+0000","last_undegraded":"2024-09-10T08:46:19.194248+0000","last_fullsized":"2024-09-10T08:46:19.194248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:32.675598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193398+0000","last_change":"2024-09-10T08:44:10.671500+0000","last_active":"2024-09-10T08:46:19.193398+0000","last_peered":"2024-09-10T08:46:19.193398+0000","last_clean":"2024-09-10T08:46:19.193398+0000","last_became_active":"2024-09-10T08:44:10.671039+0000","last_became_peered":"2024-09-10T08:44:10.671039+0000","last_unstale":"2024-09-10T08:46:19.193398+0000","last_undegraded":"2024-09-10T08:46:19.193398+0000","last_fullsized":"2024-09-10T08:46:19.193398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:12.977373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193090+0000","last_change":"2024-09-10T08:44:08.655467+0000","last_active":"2024-09-10T08:46:19.193090+0000","last_peered":"2024-09-10T08:46:19.193090+0000","last_clean":"2024-09-10T08:46:19.193090+0000","last_became_active":"2024-09-10T08:44:08.654991+0000","last_became_peered":"2024-09-10T08:44:08.654991+0000","last_unstale":"2024-09-10T08:46:19.193090+0000","last_undegraded":"2024-09-10T08:46:19.193090+0000","last_fullsized":"2024-09-10T08:46:19.193090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:33.124360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803728+0000","last_change":"2024-09-10T08:44:17.724250+0000","last_active":"2024-09-10T08:46:18.803728+0000","last_peered":"2024-09-10T08:46:18.803728+0000","last_clean":"2024-09-10T08:46:18.803728+0000","last_became_active":"2024-09-10T08:44:17.723733+0000","last_became_peered":"2024-09-10T08:44:17.723733+0000","last_unstale":"2024-09-10T08:46:18.803728+0000","last_undegraded":"2024-09-10T08:46:18.803728+0000","last_fullsized":"2024-09-10T08:46:18.803728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:40.098666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193144+0000","last_change":"2024-09-10T08:44:10.664159+0000","last_active":"2024-09-10T08:46:19.193144+0000","last_peered":"2024-09-10T08:46:19.193144+0000","last_clean":"2024-09-10T08:46:19.193144+0000","last_became_active":"2024-09-10T08:44:10.663674+0000","last_became_peered":"2024-09-10T08:44:10.663674+0000","last_unstale":"2024-09-10T08:46:19.193144+0000","last_undegraded":"2024-09-10T08:46:19.193144+0000","last_fullsized":"2024-09-10T08:46:19.193144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:24.234525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193535+0000","last_change":"2024-09-10T08:45:18.257226+0000","last_active":"2024-09-10T08:46:19.193535+0000","last_peered":"2024-09-10T08:46:19.193535+0000","last_clean":"2024-09-10T08:46:19.193535+0000","last_became_active":"2024-09-10T08:45:18.254113+0000","last_became_peered":"2024-09-10T08:45:18.254113+0000","last_unstale":"2024-09-10T08:46:19.193535+0000","last_undegraded":"2024-09-10T08:46:19.193535+0000","last_fullsized":"2024-09-10T08:46:19.193535+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:19.370105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194467+0000","last_change":"2024-09-10T08:44:17.761639+0000","last_active":"2024-09-10T08:46:19.194467+0000","last_peered":"2024-09-10T08:46:19.194467+0000","last_clean":"2024-09-10T08:46:19.194467+0000","last_became_active":"2024-09-10T08:44:17.761214+0000","last_became_peered":"2024-09-10T08:44:17.761214+0000","last_unstale":"2024-09-10T08:46:19.194467+0000","last_undegraded":"2024-09-10T08:46:19.194467+0000","last_fullsized":"2024-09-10T08:46:19.194467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:43.793195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193666+0000","last_change":"2024-09-10T08:45:18.258661+0000","last_active":"2024-09-10T08:46:19.193666+0000","last_peered":"2024-09-10T08:46:19.193666+0000","last_clean":"2024-09-10T08:46:19.193666+0000","last_became_active":"2024-09-10T08:45:18.258225+0000","last_became_peered":"2024-09-10T08:45:18.258225+0000","last_unstale":"2024-09-10T08:46:19.193666+0000","last_undegraded":"2024-09-10T08:46:19.193666+0000","last_fullsized":"2024-09-10T08:46:19.193666+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:28:37.184766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194660+0000","last_change":"2024-09-10T08:44:09.036632+0000","last_active":"2024-09-10T08:46:19.194660+0000","last_peered":"2024-09-10T08:46:19.194660+0000","last_clean":"2024-09-10T08:46:19.194660+0000","last_became_active":"2024-09-10T08:44:09.036254+0000","last_became_peered":"2024-09-10T08:44:09.036254+0000","last_unstale":"2024-09-10T08:46:19.194660+0000","last_undegraded":"2024-09-10T08:46:19.194660+0000","last_fullsized":"2024-09-10T08:46:19.194660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:59:40.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808140+0000","last_change":"2024-09-10T08:44:17.716215+0000","last_active":"2024-09-10T08:46:18.808140+0000","last_peered":"2024-09-10T08:46:18.808140+0000","last_clean":"2024-09-10T08:46:18.808140+0000","last_became_active":"2024-09-10T08:44:17.715809+0000","last_became_peered":"2024-09-10T08:44:17.715809+0000","last_unstale":"2024-09-10T08:46:18.808140+0000","last_undegraded":"2024-09-10T08:46:18.808140+0000","last_fullsized":"2024-09-10T08:46:18.808140+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:52.896401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193713+0000","last_change":"2024-09-10T08:44:10.677871+0000","last_active":"2024-09-10T08:46:19.193713+0000","last_peered":"2024-09-10T08:46:19.193713+0000","last_clean":"2024-09-10T08:46:19.193713+0000","last_became_active":"2024-09-10T08:44:10.677101+0000","last_became_peered":"2024-09-10T08:44:10.677101+0000","last_unstale":"2024-09-10T08:46:19.193713+0000","last_undegraded":"2024-09-10T08:46:19.193713+0000","last_fullsized":"2024-09-10T08:46:19.193713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:46.137249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'17","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:21.176388+0000","last_change":"2024-09-10T08:44:08.654428+0000","last_active":"2024-09-10T08:46:21.176388+0000","last_peered":"2024-09-10T08:46:21.176388+0000","last_clean":"2024-09-10T08:46:21.176388+0000","last_became_active":"2024-09-10T08:44:08.654119+0000","last_became_peered":"2024-09-10T08:44:08.654119+0000","last_unstale":"2024-09-10T08:46:21.176388+0000","last_undegraded":"2024-09-10T08:46:21.176388+0000","last_fullsized":"2024-09-10T08:46:21.176388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:56.787721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192538+0000","last_change":"2024-09-10T08:44:17.760828+0000","last_active":"2024-09-10T08:46:19.192538+0000","last_peered":"2024-09-10T08:46:19.192538+0000","last_clean":"2024-09-10T08:46:19.192538+0000","last_became_active":"2024-09-10T08:44:17.760367+0000","last_became_peered":"2024-09-10T08:44:17.760367+0000","last_unstale":"2024-09-10T08:46:19.192538+0000","last_undegraded":"2024-09-10T08:46:19.192538+0000","last_fullsized":"2024-09-10T08:46:19.192538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:11.130480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806446+0000","last_change":"2024-09-10T08:44:10.670179+0000","last_active":"2024-09-10T08:46:18.806446+0000","last_peered":"2024-09-10T08:46:18.806446+0000","last_clean":"2024-09-10T08:46:18.806446+0000","last_became_active":"2024-09-10T08:44:10.669366+0000","last_became_peered":"2024-09-10T08:44:10.669366+0000","last_unstale":"2024-09-10T08:46:18.806446+0000","last_undegraded":"2024-09-10T08:46:18.806446+0000","last_fullsized":"2024-09-10T08:46:18.806446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:41.916365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194210+0000","last_change":"2024-09-10T08:44:09.050441+0000","last_active":"2024-09-10T08:46:19.194210+0000","last_peered":"2024-09-10T08:46:19.194210+0000","last_clean":"2024-09-10T08:46:19.194210+0000","last_became_active":"2024-09-10T08:44:09.050070+0000","last_became_peered":"2024-09-10T08:44:09.050070+0000","last_unstale":"2024-09-10T08:46:19.194210+0000","last_undegraded":"2024-09-10T08:46:19.194210+0000","last_fullsized":"2024-09-10T08:46:19.194210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:13.540021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193573+0000","last_change":"2024-09-10T08:44:17.720098+0000","last_active":"2024-09-10T08:46:19.193573+0000","last_peered":"2024-09-10T08:46:19.193573+0000","last_clean":"2024-09-10T08:46:19.193573+0000","last_became_active":"2024-09-10T08:44:17.719627+0000","last_became_peered":"2024-09-10T08:44:17.719627+0000","last_unstale":"2024-09-10T08:46:19.193573+0000","last_undegraded":"2024-09-10T08:46:19.193573+0000","last_fullsized":"2024-09-10T08:46:19.193573+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:50.020919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804677+0000","last_change":"2024-09-10T08:44:10.669630+0000","last_active":"2024-09-10T08:46:18.804677+0000","last_peered":"2024-09-10T08:46:18.804677+0000","last_clean":"2024-09-10T08:46:18.804677+0000","last_became_active":"2024-09-10T08:44:10.669307+0000","last_became_peered":"2024-09-10T08:44:10.669307+0000","last_unstale":"2024-09-10T08:46:18.804677+0000","last_undegraded":"2024-09-10T08:46:18.804677+0000","last_fullsized":"2024-09-10T08:46:18.804677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:06.320046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803568+0000","last_change":"2024-09-10T08:44:09.050715+0000","last_active":"2024-09-10T08:46:18.803568+0000","last_peered":"2024-09-10T08:46:18.803568+0000","last_clean":"2024-09-10T08:46:18.803568+0000","last_became_active":"2024-09-10T08:44:09.050347+0000","last_became_peered":"2024-09-10T08:44:09.050347+0000","last_unstale":"2024-09-10T08:46:18.803568+0000","last_undegraded":"2024-09-10T08:46:18.803568+0000","last_fullsized":"2024-09-10T08:46:18.803568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:18.689652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194038+0000","last_change":"2024-09-10T08:45:18.260444+0000","last_active":"2024-09-10T08:46:19.194038+0000","last_peered":"2024-09-10T08:46:19.194038+0000","last_clean":"2024-09-10T08:46:19.194038+0000","last_became_active":"2024-09-10T08:45:18.259949+0000","last_became_peered":"2024-09-10T08:45:18.259949+0000","last_unstale":"2024-09-10T08:46:19.194038+0000","last_undegraded":"2024-09-10T08:46:19.194038+0000","last_fullsized":"2024-09-10T08:46:19.194038+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:56.391253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804630+0000","last_change":"2024-09-10T08:44:10.670881+0000","last_active":"2024-09-10T08:46:18.804630+0000","last_peered":"2024-09-10T08:46:18.804630+0000","last_clean":"2024-09-10T08:46:18.804630+0000","last_became_active":"2024-09-10T08:44:10.669791+0000","last_became_peered":"2024-09-10T08:44:10.669791+0000","last_unstale":"2024-09-10T08:46:18.804630+0000","last_undegraded":"2024-09-10T08:46:18.804630+0000","last_fullsized":"2024-09-10T08:46:18.804630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:33.644205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'6","reported_seq":151,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:37.369215+0000","last_change":"2024-09-10T08:44:08.657938+0000","last_active":"2024-09-10T08:46:37.369215+0000","last_peered":"2024-09-10T08:46:37.369215+0000","last_clean":"2024-09-10T08:46:37.369215+0000","last_became_active":"2024-09-10T08:44:08.657526+0000","last_became_peered":"2024-09-10T08:44:08.657526+0000","last_unstale":"2024-09-10T08:46:37.369215+0000","last_undegraded":"2024-09-10T08:46:37.369215+0000","last_fullsized":"2024-09-10T08:46:37.369215+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:15.673561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806147+0000","last_change":"2024-09-10T08:44:17.721619+0000","last_active":"2024-09-10T08:46:18.806147+0000","last_peered":"2024-09-10T08:46:18.806147+0000","last_clean":"2024-09-10T08:46:18.806147+0000","last_became_active":"2024-09-10T08:44:17.721175+0000","last_became_peered":"2024-09-10T08:44:17.721175+0000","last_unstale":"2024-09-10T08:46:18.806147+0000","last_undegraded":"2024-09-10T08:46:18.806147+0000","last_fullsized":"2024-09-10T08:46:18.806147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:52.244120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804320+0000","last_change":"2024-09-10T08:44:10.676164+0000","last_active":"2024-09-10T08:46:18.804320+0000","last_peered":"2024-09-10T08:46:18.804320+0000","last_clean":"2024-09-10T08:46:18.804320+0000","last_became_active":"2024-09-10T08:44:10.675820+0000","last_became_peered":"2024-09-10T08:44:10.675820+0000","last_unstale":"2024-09-10T08:46:18.804320+0000","last_undegraded":"2024-09-10T08:46:18.804320+0000","last_fullsized":"2024-09-10T08:46:18.804320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:24.477584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":152,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:37.369105+0000","last_change":"2024-09-10T08:44:09.049124+0000","last_active":"2024-09-10T08:46:37.369105+0000","last_peered":"2024-09-10T08:46:37.369105+0000","last_clean":"2024-09-10T08:46:37.369105+0000","last_became_active":"2024-09-10T08:44:09.048799+0000","last_became_peered":"2024-09-10T08:44:09.048799+0000","last_unstale":"2024-09-10T08:46:37.369105+0000","last_undegraded":"2024-09-10T08:46:37.369105+0000","last_fullsized":"2024-09-10T08:46:37.369105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:43:39.849727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806891+0000","last_change":"2024-09-10T08:44:17.761130+0000","last_active":"2024-09-10T08:46:18.806891+0000","last_peered":"2024-09-10T08:46:18.806891+0000","last_clean":"2024-09-10T08:46:18.806891+0000","last_became_active":"2024-09-10T08:44:17.760753+0000","last_became_peered":"2024-09-10T08:44:17.760753+0000","last_unstale":"2024-09-10T08:46:18.806891+0000","last_undegraded":"2024-09-10T08:46:18.806891+0000","last_fullsized":"2024-09-10T08:46:18.806891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:42.971962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.809258+0000","last_change":"2024-09-10T08:46:18.809258+0000","last_active":"2024-09-10T08:46:18.809258+0000","last_peered":"2024-09-10T08:46:18.809258+0000","last_clean":"2024-09-10T08:46:18.809258+0000","last_became_active":"2024-09-10T08:46:18.808908+0000","last_became_peered":"2024-09-10T08:46:18.808908+0000","last_unstale":"2024-09-10T08:46:18.809258+0000","last_undegraded":"2024-09-10T08:46:18.809258+0000","last_fullsized":"2024-09-10T08:46:18.809258+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:10.305672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873500+0000","last_change":"2024-09-10T08:44:08.658862+0000","last_active":"2024-09-10T08:46:18.873500+0000","last_peered":"2024-09-10T08:46:18.873500+0000","last_clean":"2024-09-10T08:46:18.873500+0000","last_became_active":"2024-09-10T08:44:08.657880+0000","last_became_peered":"2024-09-10T08:44:08.657880+0000","last_unstale":"2024-09-10T08:46:18.873500+0000","last_undegraded":"2024-09-10T08:46:18.873500+0000","last_fullsized":"2024-09-10T08:46:18.873500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:34.224025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194185+0000","last_change":"2024-09-10T08:45:18.268446+0000","last_active":"2024-09-10T08:46:19.194185+0000","last_peered":"2024-09-10T08:46:19.194185+0000","last_clean":"2024-09-10T08:46:19.194185+0000","last_became_active":"2024-09-10T08:45:18.267988+0000","last_became_peered":"2024-09-10T08:45:18.267988+0000","last_unstale":"2024-09-10T08:46:19.194185+0000","last_undegraded":"2024-09-10T08:46:19.194185+0000","last_fullsized":"2024-09-10T08:46:19.194185+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:42:35.990794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193214+0000","last_change":"2024-09-10T08:44:10.673258+0000","last_active":"2024-09-10T08:46:19.193214+0000","last_peered":"2024-09-10T08:46:19.193214+0000","last_clean":"2024-09-10T08:46:19.193214+0000","last_became_active":"2024-09-10T08:44:10.672854+0000","last_became_peered":"2024-09-10T08:44:10.672854+0000","last_unstale":"2024-09-10T08:46:19.193214+0000","last_undegraded":"2024-09-10T08:46:19.193214+0000","last_fullsized":"2024-09-10T08:46:19.193214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:35.913820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194588+0000","last_change":"2024-09-10T08:44:08.658039+0000","last_active":"2024-09-10T08:46:19.194588+0000","last_peered":"2024-09-10T08:46:19.194588+0000","last_clean":"2024-09-10T08:46:19.194588+0000","last_became_active":"2024-09-10T08:44:08.657636+0000","last_became_peered":"2024-09-10T08:44:08.657636+0000","last_unstale":"2024-09-10T08:46:19.194588+0000","last_undegraded":"2024-09-10T08:46:19.194588+0000","last_fullsized":"2024-09-10T08:46:19.194588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:09.615992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873591+0000","last_change":"2024-09-10T08:42:57.666747+0000","last_active":"2024-09-10T08:46:18.873591+0000","last_peered":"2024-09-10T08:46:18.873591+0000","last_clean":"2024-09-10T08:46:18.873591+0000","last_became_active":"2024-09-10T08:42:57.666417+0000","last_became_peered":"2024-09-10T08:42:57.666417+0000","last_unstale":"2024-09-10T08:46:18.873591+0000","last_undegraded":"2024-09-10T08:46:18.873591+0000","last_fullsized":"2024-09-10T08:46:18.873591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_clean_scrub_stamp":"2024-09-10T08:39:31.729794+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.547989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874074+0000","last_change":"2024-09-10T08:44:17.723021+0000","last_active":"2024-09-10T08:46:18.874074+0000","last_peered":"2024-09-10T08:46:18.874074+0000","last_clean":"2024-09-10T08:46:18.874074+0000","last_became_active":"2024-09-10T08:44:17.722222+0000","last_became_peered":"2024-09-10T08:44:17.722222+0000","last_unstale":"2024-09-10T08:46:18.874074+0000","last_undegraded":"2024-09-10T08:46:18.874074+0000","last_fullsized":"2024-09-10T08:46:18.874074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:24.444427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874129+0000","last_change":"2024-09-10T08:44:10.676651+0000","last_active":"2024-09-10T08:46:18.874129+0000","last_peered":"2024-09-10T08:46:18.874129+0000","last_clean":"2024-09-10T08:46:18.874129+0000","last_became_active":"2024-09-10T08:44:10.676342+0000","last_became_peered":"2024-09-10T08:44:10.676342+0000","last_unstale":"2024-09-10T08:46:18.874129+0000","last_undegraded":"2024-09-10T08:46:18.874129+0000","last_fullsized":"2024-09-10T08:46:18.874129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:07.021273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'8","reported_seq":154,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:35.366830+0000","last_change":"2024-09-10T08:44:09.050305+0000","last_active":"2024-09-10T08:46:35.366830+0000","last_peered":"2024-09-10T08:46:35.366830+0000","last_clean":"2024-09-10T08:46:35.366830+0000","last_became_active":"2024-09-10T08:44:09.049958+0000","last_became_peered":"2024-09-10T08:44:09.049958+0000","last_unstale":"2024-09-10T08:46:35.366830+0000","last_undegraded":"2024-09-10T08:46:35.366830+0000","last_fullsized":"2024-09-10T08:46:35.366830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:42.219190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2199,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193720+0000","last_change":"2024-09-10T08:44:17.720219+0000","last_active":"2024-09-10T08:46:19.193720+0000","last_peered":"2024-09-10T08:46:19.193720+0000","last_clean":"2024-09-10T08:46:19.193720+0000","last_became_active":"2024-09-10T08:44:17.719756+0000","last_became_peered":"2024-09-10T08:44:17.719756+0000","last_unstale":"2024-09-10T08:46:19.193720+0000","last_undegraded":"2024-09-10T08:46:19.193720+0000","last_fullsized":"2024-09-10T08:46:19.193720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:01.070434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873929+0000","last_change":"2024-09-10T08:44:10.678986+0000","last_active":"2024-09-10T08:46:18.873929+0000","last_peered":"2024-09-10T08:46:18.873929+0000","last_clean":"2024-09-10T08:46:18.873929+0000","last_became_active":"2024-09-10T08:44:10.678654+0000","last_became_peered":"2024-09-10T08:44:10.678654+0000","last_unstale":"2024-09-10T08:46:18.873929+0000","last_undegraded":"2024-09-10T08:46:18.873929+0000","last_fullsized":"2024-09-10T08:46:18.873929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:07.116802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805026+0000","last_change":"2024-09-10T08:44:08.655025+0000","last_active":"2024-09-10T08:46:18.805026+0000","last_peered":"2024-09-10T08:46:18.805026+0000","last_clean":"2024-09-10T08:46:18.805026+0000","last_became_active":"2024-09-10T08:44:08.654728+0000","last_became_peered":"2024-09-10T08:44:08.654728+0000","last_unstale":"2024-09-10T08:46:18.805026+0000","last_undegraded":"2024-09-10T08:46:18.805026+0000","last_fullsized":"2024-09-10T08:46:18.805026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:31.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193534+0000","last_change":"2024-09-10T08:45:18.261492+0000","last_active":"2024-09-10T08:46:19.193534+0000","last_peered":"2024-09-10T08:46:19.193534+0000","last_clean":"2024-09-10T08:46:19.193534+0000","last_became_active":"2024-09-10T08:45:18.261111+0000","last_became_peered":"2024-09-10T08:45:18.261111+0000","last_unstale":"2024-09-10T08:46:19.193534+0000","last_undegraded":"2024-09-10T08:46:19.193534+0000","last_fullsized":"2024-09-10T08:46:19.193534+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:48.709459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874006+0000","last_change":"2024-09-10T08:44:10.675072+0000","last_active":"2024-09-10T08:46:18.874006+0000","last_peered":"2024-09-10T08:46:18.874006+0000","last_clean":"2024-09-10T08:46:18.874006+0000","last_became_active":"2024-09-10T08:44:10.674636+0000","last_became_peered":"2024-09-10T08:44:10.674636+0000","last_unstale":"2024-09-10T08:46:18.874006+0000","last_undegraded":"2024-09-10T08:46:18.874006+0000","last_fullsized":"2024-09-10T08:46:18.874006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:55.198445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806864+0000","last_change":"2024-09-10T08:44:08.656450+0000","last_active":"2024-09-10T08:46:18.806864+0000","last_peered":"2024-09-10T08:46:18.806864+0000","last_clean":"2024-09-10T08:46:18.806864+0000","last_became_active":"2024-09-10T08:44:08.655967+0000","last_became_peered":"2024-09-10T08:44:08.655967+0000","last_unstale":"2024-09-10T08:46:18.806864+0000","last_undegraded":"2024-09-10T08:46:18.806864+0000","last_fullsized":"2024-09-10T08:46:18.806864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:29.273167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804195+0000","last_change":"2024-09-10T08:44:17.722217+0000","last_active":"2024-09-10T08:46:18.804195+0000","last_peered":"2024-09-10T08:46:18.804195+0000","last_clean":"2024-09-10T08:46:18.804195+0000","last_became_active":"2024-09-10T08:44:17.721800+0000","last_became_peered":"2024-09-10T08:44:17.721800+0000","last_unstale":"2024-09-10T08:46:18.804195+0000","last_undegraded":"2024-09-10T08:46:18.804195+0000","last_fullsized":"2024-09-10T08:46:18.804195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:58.954397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194230+0000","last_change":"2024-09-10T08:45:18.267857+0000","last_active":"2024-09-10T08:46:19.194230+0000","last_peered":"2024-09-10T08:46:19.194230+0000","last_clean":"2024-09-10T08:46:19.194230+0000","last_became_active":"2024-09-10T08:45:18.267440+0000","last_became_peered":"2024-09-10T08:45:18.267440+0000","last_unstale":"2024-09-10T08:46:19.194230+0000","last_undegraded":"2024-09-10T08:46:19.194230+0000","last_fullsized":"2024-09-10T08:46:19.194230+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:28.659915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:23.312217+0000","last_change":"2024-09-10T08:44:08.659003+0000","last_active":"2024-09-10T08:46:23.312217+0000","last_peered":"2024-09-10T08:46:23.312217+0000","last_clean":"2024-09-10T08:46:23.312217+0000","last_became_active":"2024-09-10T08:44:08.658324+0000","last_became_peered":"2024-09-10T08:44:08.658324+0000","last_unstale":"2024-09-10T08:46:23.312217+0000","last_undegraded":"2024-09-10T08:46:23.312217+0000","last_fullsized":"2024-09-10T08:46:23.312217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:44.743053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804596+0000","last_change":"2024-09-10T08:44:17.723363+0000","last_active":"2024-09-10T08:46:18.804596+0000","last_peered":"2024-09-10T08:46:18.804596+0000","last_clean":"2024-09-10T08:46:18.804596+0000","last_became_active":"2024-09-10T08:44:17.721799+0000","last_became_peered":"2024-09-10T08:44:17.721799+0000","last_unstale":"2024-09-10T08:46:18.804596+0000","last_undegraded":"2024-09-10T08:46:18.804596+0000","last_fullsized":"2024-09-10T08:46:18.804596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:08:28.772193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804709+0000","last_change":"2024-09-10T08:44:10.682952+0000","last_active":"2024-09-10T08:46:18.804709+0000","last_peered":"2024-09-10T08:46:18.804709+0000","last_clean":"2024-09-10T08:46:18.804709+0000","last_became_active":"2024-09-10T08:44:10.682249+0000","last_became_peered":"2024-09-10T08:44:10.682249+0000","last_unstale":"2024-09-10T08:46:18.804709+0000","last_undegraded":"2024-09-10T08:46:18.804709+0000","last_fullsized":"2024-09-10T08:46:18.804709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:32:21.285461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193239+0000","last_change":"2024-09-10T08:44:09.050454+0000","last_active":"2024-09-10T08:46:19.193239+0000","last_peered":"2024-09-10T08:46:19.193239+0000","last_clean":"2024-09-10T08:46:19.193239+0000","last_became_active":"2024-09-10T08:44:09.050036+0000","last_became_peered":"2024-09-10T08:44:09.050036+0000","last_unstale":"2024-09-10T08:46:19.193239+0000","last_undegraded":"2024-09-10T08:46:19.193239+0000","last_fullsized":"2024-09-10T08:46:19.193239+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:55.141471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806678+0000","last_change":"2024-09-10T08:44:17.721203+0000","last_active":"2024-09-10T08:46:18.806678+0000","last_peered":"2024-09-10T08:46:18.806678+0000","last_clean":"2024-09-10T08:46:18.806678+0000","last_became_active":"2024-09-10T08:44:17.720737+0000","last_became_peered":"2024-09-10T08:44:17.720737+0000","last_unstale":"2024-09-10T08:46:18.806678+0000","last_undegraded":"2024-09-10T08:46:18.806678+0000","last_fullsized":"2024-09-10T08:46:18.806678+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:32.081079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873435+0000","last_change":"2024-09-10T08:44:10.680944+0000","last_active":"2024-09-10T08:46:18.873435+0000","last_peered":"2024-09-10T08:46:18.873435+0000","last_clean":"2024-09-10T08:46:18.873435+0000","last_became_active":"2024-09-10T08:44:10.680644+0000","last_became_peered":"2024-09-10T08:44:10.680644+0000","last_unstale":"2024-09-10T08:46:18.873435+0000","last_undegraded":"2024-09-10T08:46:18.873435+0000","last_fullsized":"2024-09-10T08:46:18.873435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:46.495894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":148,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:33.363751+0000","last_change":"2024-09-10T08:44:08.658135+0000","last_active":"2024-09-10T08:46:33.363751+0000","last_peered":"2024-09-10T08:46:33.363751+0000","last_clean":"2024-09-10T08:46:33.363751+0000","last_became_active":"2024-09-10T08:44:08.657793+0000","last_became_peered":"2024-09-10T08:44:08.657793+0000","last_unstale":"2024-09-10T08:46:33.363751+0000","last_undegraded":"2024-09-10T08:46:33.363751+0000","last_fullsized":"2024-09-10T08:46:33.363751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:16.376268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193715+0000","last_change":"2024-09-10T08:45:18.271678+0000","last_active":"2024-09-10T08:46:19.193715+0000","last_peered":"2024-09-10T08:46:19.193715+0000","last_clean":"2024-09-10T08:46:19.193715+0000","last_became_active":"2024-09-10T08:45:18.271209+0000","last_became_peered":"2024-09-10T08:45:18.271209+0000","last_unstale":"2024-09-10T08:46:19.193715+0000","last_undegraded":"2024-09-10T08:46:19.193715+0000","last_fullsized":"2024-09-10T08:46:19.193715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:17.730839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873847+0000","last_change":"2024-09-10T08:44:10.678254+0000","last_active":"2024-09-10T08:46:18.873847+0000","last_peered":"2024-09-10T08:46:18.873847+0000","last_clean":"2024-09-10T08:46:18.873847+0000","last_became_active":"2024-09-10T08:44:10.677891+0000","last_became_peered":"2024-09-10T08:44:10.677891+0000","last_unstale":"2024-09-10T08:46:18.873847+0000","last_undegraded":"2024-09-10T08:46:18.873847+0000","last_fullsized":"2024-09-10T08:46:18.873847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:32:43.399049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804719+0000","last_change":"2024-09-10T08:44:08.656315+0000","last_active":"2024-09-10T08:46:18.804719+0000","last_peered":"2024-09-10T08:46:18.804719+0000","last_clean":"2024-09-10T08:46:18.804719+0000","last_became_active":"2024-09-10T08:44:08.655978+0000","last_became_peered":"2024-09-10T08:44:08.655978+0000","last_unstale":"2024-09-10T08:46:18.804719+0000","last_undegraded":"2024-09-10T08:46:18.804719+0000","last_fullsized":"2024-09-10T08:46:18.804719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:15.025354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193370+0000","last_change":"2024-09-10T08:44:17.726789+0000","last_active":"2024-09-10T08:46:19.193370+0000","last_peered":"2024-09-10T08:46:19.193370+0000","last_clean":"2024-09-10T08:46:19.193370+0000","last_became_active":"2024-09-10T08:44:17.726450+0000","last_became_peered":"2024-09-10T08:44:17.726450+0000","last_unstale":"2024-09-10T08:46:19.193370+0000","last_undegraded":"2024-09-10T08:46:19.193370+0000","last_fullsized":"2024-09-10T08:46:19.193370+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:03.899822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806240+0000","last_change":"2024-09-10T08:44:10.677359+0000","last_active":"2024-09-10T08:46:18.806240+0000","last_peered":"2024-09-10T08:46:18.806240+0000","last_clean":"2024-09-10T08:46:18.806240+0000","last_became_active":"2024-09-10T08:44:10.676782+0000","last_became_peered":"2024-09-10T08:44:10.676782+0000","last_unstale":"2024-09-10T08:46:18.806240+0000","last_undegraded":"2024-09-10T08:46:18.806240+0000","last_fullsized":"2024-09-10T08:46:18.806240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:31.721714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808306+0000","last_change":"2024-09-10T08:44:08.656993+0000","last_active":"2024-09-10T08:46:18.808306+0000","last_peered":"2024-09-10T08:46:18.808306+0000","last_clean":"2024-09-10T08:46:18.808306+0000","last_became_active":"2024-09-10T08:44:08.656571+0000","last_became_peered":"2024-09-10T08:44:08.656571+0000","last_unstale":"2024-09-10T08:46:18.808306+0000","last_undegraded":"2024-09-10T08:46:18.808306+0000","last_fullsized":"2024-09-10T08:46:18.808306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:32.202740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804870+0000","last_change":"2024-09-10T08:44:17.725674+0000","last_active":"2024-09-10T08:46:18.804870+0000","last_peered":"2024-09-10T08:46:18.804870+0000","last_clean":"2024-09-10T08:46:18.804870+0000","last_became_active":"2024-09-10T08:44:17.725263+0000","last_became_peered":"2024-09-10T08:44:17.725263+0000","last_unstale":"2024-09-10T08:46:18.804870+0000","last_undegraded":"2024-09-10T08:46:18.804870+0000","last_fullsized":"2024-09-10T08:46:18.804870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:18.183157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193937+0000","last_change":"2024-09-10T08:44:10.676871+0000","last_active":"2024-09-10T08:46:19.193937+0000","last_peered":"2024-09-10T08:46:19.193937+0000","last_clean":"2024-09-10T08:46:19.193937+0000","last_became_active":"2024-09-10T08:44:10.676540+0000","last_became_peered":"2024-09-10T08:44:10.676540+0000","last_unstale":"2024-09-10T08:46:19.193937+0000","last_undegraded":"2024-09-10T08:46:19.193937+0000","last_fullsized":"2024-09-10T08:46:19.193937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:50.964100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'40","reported_seq":293,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.365469+0000","last_change":"2024-09-10T08:44:08.660113+0000","last_active":"2024-09-10T08:46:34.365469+0000","last_peered":"2024-09-10T08:46:34.365469+0000","last_clean":"2024-09-10T08:46:34.365469+0000","last_became_active":"2024-09-10T08:44:08.659644+0000","last_became_peered":"2024-09-10T08:44:08.659644+0000","last_unstale":"2024-09-10T08:46:34.365469+0000","last_undegraded":"2024-09-10T08:46:34.365469+0000","last_fullsized":"2024-09-10T08:46:34.365469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:01.605344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64228,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":214,"num_read_kb":170,"num_write":42,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803296+0000","last_change":"2024-09-10T08:44:17.723653+0000","last_active":"2024-09-10T08:46:18.803296+0000","last_peered":"2024-09-10T08:46:18.803296+0000","last_clean":"2024-09-10T08:46:18.803296+0000","last_became_active":"2024-09-10T08:44:17.723284+0000","last_became_peered":"2024-09-10T08:44:17.723284+0000","last_unstale":"2024-09-10T08:46:18.803296+0000","last_undegraded":"2024-09-10T08:46:18.803296+0000","last_fullsized":"2024-09-10T08:46:18.803296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:17.162526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193970+0000","last_change":"2024-09-10T08:44:10.681837+0000","last_active":"2024-09-10T08:46:19.193970+0000","last_peered":"2024-09-10T08:46:19.193970+0000","last_clean":"2024-09-10T08:46:19.193970+0000","last_became_active":"2024-09-10T08:44:10.681518+0000","last_became_peered":"2024-09-10T08:44:10.681518+0000","last_unstale":"2024-09-10T08:46:19.193970+0000","last_undegraded":"2024-09-10T08:46:19.193970+0000","last_fullsized":"2024-09-10T08:46:19.193970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:10.004215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804988+0000","last_change":"2024-09-10T08:44:09.050394+0000","last_active":"2024-09-10T08:46:18.804988+0000","last_peered":"2024-09-10T08:46:18.804988+0000","last_clean":"2024-09-10T08:46:18.804988+0000","last_became_active":"2024-09-10T08:44:09.049969+0000","last_became_peered":"2024-09-10T08:44:09.049969+0000","last_unstale":"2024-09-10T08:46:18.804988+0000","last_undegraded":"2024-09-10T08:46:18.804988+0000","last_fullsized":"2024-09-10T08:46:18.804988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:43.519246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194127+0000","last_change":"2024-09-10T08:44:17.720318+0000","last_active":"2024-09-10T08:46:19.194127+0000","last_peered":"2024-09-10T08:46:19.194127+0000","last_clean":"2024-09-10T08:46:19.194127+0000","last_became_active":"2024-09-10T08:44:17.719598+0000","last_became_peered":"2024-09-10T08:44:17.719598+0000","last_unstale":"2024-09-10T08:46:19.194127+0000","last_undegraded":"2024-09-10T08:46:19.194127+0000","last_fullsized":"2024-09-10T08:46:19.194127+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:15:49.520324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193946+0000","last_change":"2024-09-10T08:44:10.677340+0000","last_active":"2024-09-10T08:46:19.193946+0000","last_peered":"2024-09-10T08:46:19.193946+0000","last_clean":"2024-09-10T08:46:19.193946+0000","last_became_active":"2024-09-10T08:44:10.676628+0000","last_became_peered":"2024-09-10T08:44:10.676628+0000","last_unstale":"2024-09-10T08:46:19.193946+0000","last_undegraded":"2024-09-10T08:46:19.193946+0000","last_fullsized":"2024-09-10T08:46:19.193946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:59:17.924217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874027+0000","last_change":"2024-09-10T08:44:08.660254+0000","last_active":"2024-09-10T08:46:18.874027+0000","last_peered":"2024-09-10T08:46:18.874027+0000","last_clean":"2024-09-10T08:46:18.874027+0000","last_became_active":"2024-09-10T08:44:08.659667+0000","last_became_peered":"2024-09-10T08:44:08.659667+0000","last_unstale":"2024-09-10T08:46:18.874027+0000","last_undegraded":"2024-09-10T08:46:18.874027+0000","last_fullsized":"2024-09-10T08:46:18.874027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:49:49.146867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803135+0000","last_change":"2024-09-10T08:44:17.722373+0000","last_active":"2024-09-10T08:46:18.803135+0000","last_peered":"2024-09-10T08:46:18.803135+0000","last_clean":"2024-09-10T08:46:18.803135+0000","last_became_active":"2024-09-10T08:44:17.722035+0000","last_became_peered":"2024-09-10T08:44:17.722035+0000","last_unstale":"2024-09-10T08:46:18.803135+0000","last_undegraded":"2024-09-10T08:46:18.803135+0000","last_fullsized":"2024-09-10T08:46:18.803135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:30:36.187364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194365+0000","last_change":"2024-09-10T08:44:10.672585+0000","last_active":"2024-09-10T08:46:19.194365+0000","last_peered":"2024-09-10T08:46:19.194365+0000","last_clean":"2024-09-10T08:46:19.194365+0000","last_became_active":"2024-09-10T08:44:10.671987+0000","last_became_peered":"2024-09-10T08:44:10.671987+0000","last_unstale":"2024-09-10T08:46:19.194365+0000","last_undegraded":"2024-09-10T08:46:19.194365+0000","last_fullsized":"2024-09-10T08:46:19.194365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:48.660750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'7","reported_seq":248,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.365000+0000","last_change":"2024-09-10T08:44:08.658069+0000","last_active":"2024-09-10T08:46:34.365000+0000","last_peered":"2024-09-10T08:46:34.365000+0000","last_clean":"2024-09-10T08:46:34.365000+0000","last_became_active":"2024-09-10T08:44:08.657702+0000","last_became_peered":"2024-09-10T08:44:08.657702+0000","last_unstale":"2024-09-10T08:46:34.365000+0000","last_undegraded":"2024-09-10T08:46:34.365000+0000","last_fullsized":"2024-09-10T08:46:34.365000+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:17.383267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":202,"num_read_kb":202,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192772+0000","last_change":"2024-09-10T08:44:17.717984+0000","last_active":"2024-09-10T08:46:19.192772+0000","last_peered":"2024-09-10T08:46:19.192772+0000","last_clean":"2024-09-10T08:46:19.192772+0000","last_became_active":"2024-09-10T08:44:17.717581+0000","last_became_peered":"2024-09-10T08:44:17.717581+0000","last_unstale":"2024-09-10T08:46:19.192772+0000","last_undegraded":"2024-09-10T08:46:19.192772+0000","last_fullsized":"2024-09-10T08:46:19.192772+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:25.330732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193566+0000","last_change":"2024-09-10T08:44:17.721244+0000","last_active":"2024-09-10T08:46:19.193566+0000","last_peered":"2024-09-10T08:46:19.193566+0000","last_clean":"2024-09-10T08:46:19.193566+0000","last_became_active":"2024-09-10T08:44:17.720884+0000","last_became_peered":"2024-09-10T08:44:17.720884+0000","last_unstale":"2024-09-10T08:46:19.193566+0000","last_undegraded":"2024-09-10T08:46:19.193566+0000","last_fullsized":"2024-09-10T08:46:19.193566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.769513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803279+0000","last_change":"2024-09-10T08:44:10.667558+0000","last_active":"2024-09-10T08:46:18.803279+0000","last_peered":"2024-09-10T08:46:18.803279+0000","last_clean":"2024-09-10T08:46:18.803279+0000","last_became_active":"2024-09-10T08:44:10.667214+0000","last_became_peered":"2024-09-10T08:44:10.667214+0000","last_unstale":"2024-09-10T08:46:18.803279+0000","last_undegraded":"2024-09-10T08:46:18.803279+0000","last_fullsized":"2024-09-10T08:46:18.803279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:53.630790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804697+0000","last_change":"2024-09-10T08:44:08.657930+0000","last_active":"2024-09-10T08:46:18.804697+0000","last_peered":"2024-09-10T08:46:18.804697+0000","last_clean":"2024-09-10T08:46:18.804697+0000","last_became_active":"2024-09-10T08:44:08.657611+0000","last_became_peered":"2024-09-10T08:44:08.657611+0000","last_unstale":"2024-09-10T08:46:18.804697+0000","last_undegraded":"2024-09-10T08:46:18.804697+0000","last_fullsized":"2024-09-10T08:46:18.804697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:36.923786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193910+0000","last_change":"2024-09-10T08:44:17.726106+0000","last_active":"2024-09-10T08:46:19.193910+0000","last_peered":"2024-09-10T08:46:19.193910+0000","last_clean":"2024-09-10T08:46:19.193910+0000","last_became_active":"2024-09-10T08:44:17.725683+0000","last_became_peered":"2024-09-10T08:44:17.725683+0000","last_unstale":"2024-09-10T08:46:19.193910+0000","last_undegraded":"2024-09-10T08:46:19.193910+0000","last_fullsized":"2024-09-10T08:46:19.193910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:23.392794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192727+0000","last_change":"2024-09-10T08:44:10.672204+0000","last_active":"2024-09-10T08:46:19.192727+0000","last_peered":"2024-09-10T08:46:19.192727+0000","last_clean":"2024-09-10T08:46:19.192727+0000","last_became_active":"2024-09-10T08:44:10.671898+0000","last_became_peered":"2024-09-10T08:44:10.671898+0000","last_unstale":"2024-09-10T08:46:19.192727+0000","last_undegraded":"2024-09-10T08:46:19.192727+0000","last_fullsized":"2024-09-10T08:46:19.192727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:03.737357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805979+0000","last_change":"2024-09-10T08:44:08.656620+0000","last_active":"2024-09-10T08:46:18.805979+0000","last_peered":"2024-09-10T08:46:18.805979+0000","last_clean":"2024-09-10T08:46:18.805979+0000","last_became_active":"2024-09-10T08:44:08.656306+0000","last_became_peered":"2024-09-10T08:44:08.656306+0000","last_unstale":"2024-09-10T08:46:18.805979+0000","last_undegraded":"2024-09-10T08:46:18.805979+0000","last_fullsized":"2024-09-10T08:46:18.805979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:57.081909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804682+0000","last_change":"2024-09-10T08:44:08.660178+0000","last_active":"2024-09-10T08:46:18.804682+0000","last_peered":"2024-09-10T08:46:18.804682+0000","last_clean":"2024-09-10T08:46:18.804682+0000","last_became_active":"2024-09-10T08:44:08.659839+0000","last_became_peered":"2024-09-10T08:44:08.659839+0000","last_unstale":"2024-09-10T08:46:18.804682+0000","last_undegraded":"2024-09-10T08:46:18.804682+0000","last_fullsized":"2024-09-10T08:46:18.804682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:17.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804773+0000","last_change":"2024-09-10T08:44:10.683683+0000","last_active":"2024-09-10T08:46:18.804773+0000","last_peered":"2024-09-10T08:46:18.804773+0000","last_clean":"2024-09-10T08:46:18.804773+0000","last_became_active":"2024-09-10T08:44:10.683337+0000","last_became_peered":"2024-09-10T08:44:10.683337+0000","last_unstale":"2024-09-10T08:46:18.804773+0000","last_undegraded":"2024-09-10T08:46:18.804773+0000","last_fullsized":"2024-09-10T08:46:18.804773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:54.863098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808044+0000","last_change":"2024-09-10T08:44:17.721602+0000","last_active":"2024-09-10T08:46:18.808044+0000","last_peered":"2024-09-10T08:46:18.808044+0000","last_clean":"2024-09-10T08:46:18.808044+0000","last_became_active":"2024-09-10T08:44:17.721194+0000","last_became_peered":"2024-09-10T08:44:17.721194+0000","last_unstale":"2024-09-10T08:46:18.808044+0000","last_undegraded":"2024-09-10T08:46:18.808044+0000","last_fullsized":"2024-09-10T08:46:18.808044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:39.298494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":149,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.365494+0000","last_change":"2024-09-10T08:44:08.661943+0000","last_active":"2024-09-10T08:46:34.365494+0000","last_peered":"2024-09-10T08:46:34.365494+0000","last_clean":"2024-09-10T08:46:34.365494+0000","last_became_active":"2024-09-10T08:44:08.661587+0000","last_became_peered":"2024-09-10T08:44:08.661587+0000","last_unstale":"2024-09-10T08:46:34.365494+0000","last_undegraded":"2024-09-10T08:46:34.365494+0000","last_fullsized":"2024-09-10T08:46:34.365494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:05.771641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193718+0000","last_change":"2024-09-10T08:44:10.673522+0000","last_active":"2024-09-10T08:46:19.193718+0000","last_peered":"2024-09-10T08:46:19.193718+0000","last_clean":"2024-09-10T08:46:19.193718+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:46:19.193718+0000","last_undegraded":"2024-09-10T08:46:19.193718+0000","last_fullsized":"2024-09-10T08:46:19.193718+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:03.006144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808419+0000","last_change":"2024-09-10T08:44:17.721306+0000","last_active":"2024-09-10T08:46:18.808419+0000","last_peered":"2024-09-10T08:46:18.808419+0000","last_clean":"2024-09-10T08:46:18.808419+0000","last_became_active":"2024-09-10T08:44:17.720401+0000","last_became_peered":"2024-09-10T08:44:17.720401+0000","last_unstale":"2024-09-10T08:46:18.808419+0000","last_undegraded":"2024-09-10T08:46:18.808419+0000","last_fullsized":"2024-09-10T08:46:18.808419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:11.589371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193166+0000","last_change":"2024-09-10T08:44:10.678425+0000","last_active":"2024-09-10T08:46:19.193166+0000","last_peered":"2024-09-10T08:46:19.193166+0000","last_clean":"2024-09-10T08:46:19.193166+0000","last_became_active":"2024-09-10T08:44:10.677187+0000","last_became_peered":"2024-09-10T08:44:10.677187+0000","last_unstale":"2024-09-10T08:46:19.193166+0000","last_undegraded":"2024-09-10T08:46:19.193166+0000","last_fullsized":"2024-09-10T08:46:19.193166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:06.527530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808431+0000","last_change":"2024-09-10T08:44:10.674178+0000","last_active":"2024-09-10T08:46:18.808431+0000","last_peered":"2024-09-10T08:46:18.808431+0000","last_clean":"2024-09-10T08:46:18.808431+0000","last_became_active":"2024-09-10T08:44:10.673843+0000","last_became_peered":"2024-09-10T08:44:10.673843+0000","last_unstale":"2024-09-10T08:46:18.808431+0000","last_undegraded":"2024-09-10T08:46:18.808431+0000","last_fullsized":"2024-09-10T08:46:18.808431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:35.023663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.807988+0000","last_change":"2024-09-10T08:44:10.673271+0000","last_active":"2024-09-10T08:46:18.807988+0000","last_peered":"2024-09-10T08:46:18.807988+0000","last_clean":"2024-09-10T08:46:18.807988+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:46:18.807988+0000","last_undegraded":"2024-09-10T08:46:18.807988+0000","last_fullsized":"2024-09-10T08:46:18.807988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:21:26.606820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193502+0000","last_change":"2024-09-10T08:44:10.671348+0000","last_active":"2024-09-10T08:46:19.193502+0000","last_peered":"2024-09-10T08:46:19.193502+0000","last_clean":"2024-09-10T08:46:19.193502+0000","last_became_active":"2024-09-10T08:44:10.670623+0000","last_became_peered":"2024-09-10T08:44:10.670623+0000","last_unstale":"2024-09-10T08:46:19.193502+0000","last_undegraded":"2024-09-10T08:46:19.193502+0000","last_fullsized":"2024-09-10T08:46:19.193502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:06.145694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806400+0000","last_change":"2024-09-10T08:44:10.666011+0000","last_active":"2024-09-10T08:46:18.806400+0000","last_peered":"2024-09-10T08:46:18.806400+0000","last_clean":"2024-09-10T08:46:18.806400+0000","last_became_active":"2024-09-10T08:44:10.665664+0000","last_became_peered":"2024-09-10T08:44:10.665664+0000","last_unstale":"2024-09-10T08:46:18.806400+0000","last_undegraded":"2024-09-10T08:46:18.806400+0000","last_fullsized":"2024-09-10T08:46:18.806400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:31.542102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193792+0000","last_change":"2024-09-10T08:44:10.675960+0000","last_active":"2024-09-10T08:46:19.193792+0000","last_peered":"2024-09-10T08:46:19.193792+0000","last_clean":"2024-09-10T08:46:19.193792+0000","last_became_active":"2024-09-10T08:44:10.675587+0000","last_became_peered":"2024-09-10T08:44:10.675587+0000","last_unstale":"2024-09-10T08:46:19.193792+0000","last_undegraded":"2024-09-10T08:46:19.193792+0000","last_fullsized":"2024-09-10T08:46:19.193792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:54.634619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808734+0000","last_change":"2024-09-10T08:44:10.681094+0000","last_active":"2024-09-10T08:46:18.808734+0000","last_peered":"2024-09-10T08:46:18.808734+0000","last_clean":"2024-09-10T08:46:18.808734+0000","last_became_active":"2024-09-10T08:44:10.678585+0000","last_became_peered":"2024-09-10T08:44:10.678585+0000","last_unstale":"2024-09-10T08:46:18.808734+0000","last_undegraded":"2024-09-10T08:46:18.808734+0000","last_fullsized":"2024-09-10T08:46:18.808734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:14.951949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193627+0000","last_change":"2024-09-10T08:44:10.675473+0000","last_active":"2024-09-10T08:46:19.193627+0000","last_peered":"2024-09-10T08:46:19.193627+0000","last_clean":"2024-09-10T08:46:19.193627+0000","last_became_active":"2024-09-10T08:44:10.674155+0000","last_became_peered":"2024-09-10T08:44:10.674155+0000","last_unstale":"2024-09-10T08:46:19.193627+0000","last_undegraded":"2024-09-10T08:46:19.193627+0000","last_fullsized":"2024-09-10T08:46:19.193627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:55.940235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804316+0000","last_change":"2024-09-10T08:44:10.680162+0000","last_active":"2024-09-10T08:46:18.804316+0000","last_peered":"2024-09-10T08:46:18.804316+0000","last_clean":"2024-09-10T08:46:18.804316+0000","last_became_active":"2024-09-10T08:44:10.679800+0000","last_became_peered":"2024-09-10T08:44:10.679800+0000","last_unstale":"2024-09-10T08:46:18.804316+0000","last_undegraded":"2024-09-10T08:46:18.804316+0000","last_fullsized":"2024-09-10T08:46:18.804316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:12.319813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192932+0000","last_change":"2024-09-10T08:44:10.674724+0000","last_active":"2024-09-10T08:46:19.192932+0000","last_peered":"2024-09-10T08:46:19.192932+0000","last_clean":"2024-09-10T08:46:19.192932+0000","last_became_active":"2024-09-10T08:44:10.674389+0000","last_became_peered":"2024-09-10T08:44:10.674389+0000","last_unstale":"2024-09-10T08:46:19.192932+0000","last_undegraded":"2024-09-10T08:46:19.192932+0000","last_fullsized":"2024-09-10T08:46:19.192932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:52.123186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193888+0000","last_change":"2024-09-10T08:44:10.672805+0000","last_active":"2024-09-10T08:46:19.193888+0000","last_peered":"2024-09-10T08:46:19.193888+0000","last_clean":"2024-09-10T08:46:19.193888+0000","last_became_active":"2024-09-10T08:44:10.672508+0000","last_became_peered":"2024-09-10T08:44:10.672508+0000","last_unstale":"2024-09-10T08:46:19.193888+0000","last_undegraded":"2024-09-10T08:46:19.193888+0000","last_fullsized":"2024-09-10T08:46:19.193888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:48.558633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806228+0000","last_change":"2024-09-10T08:44:10.677504+0000","last_active":"2024-09-10T08:46:18.806228+0000","last_peered":"2024-09-10T08:46:18.806228+0000","last_clean":"2024-09-10T08:46:18.806228+0000","last_became_active":"2024-09-10T08:44:10.676822+0000","last_became_peered":"2024-09-10T08:44:10.676822+0000","last_unstale":"2024-09-10T08:46:18.806228+0000","last_undegraded":"2024-09-10T08:46:18.806228+0000","last_fullsized":"2024-09-10T08:46:18.806228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:05:48.293333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193457+0000","last_change":"2024-09-10T08:45:18.268208+0000","last_active":"2024-09-10T08:46:19.193457+0000","last_peered":"2024-09-10T08:46:19.193457+0000","last_clean":"2024-09-10T08:46:19.193457+0000","last_became_active":"2024-09-10T08:45:18.267700+0000","last_became_peered":"2024-09-10T08:45:18.267700+0000","last_unstale":"2024-09-10T08:46:19.193457+0000","last_undegraded":"2024-09-10T08:46:19.193457+0000","last_fullsized":"2024-09-10T08:46:19.193457+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:54.487624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808678+0000","last_change":"2024-09-10T08:44:10.670615+0000","last_active":"2024-09-10T08:46:18.808678+0000","last_peered":"2024-09-10T08:46:18.808678+0000","last_clean":"2024-09-10T08:46:18.808678+0000","last_became_active":"2024-09-10T08:44:10.670032+0000","last_became_peered":"2024-09-10T08:44:10.670032+0000","last_unstale":"2024-09-10T08:46:18.808678+0000","last_undegraded":"2024-09-10T08:46:18.808678+0000","last_fullsized":"2024-09-10T08:46:18.808678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:31.078788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803799+0000","last_change":"2024-09-10T08:44:10.672367+0000","last_active":"2024-09-10T08:46:18.803799+0000","last_peered":"2024-09-10T08:46:18.803799+0000","last_clean":"2024-09-10T08:46:18.803799+0000","last_became_active":"2024-09-10T08:44:10.672040+0000","last_became_peered":"2024-09-10T08:44:10.672040+0000","last_unstale":"2024-09-10T08:46:18.803799+0000","last_undegraded":"2024-09-10T08:46:18.803799+0000","last_fullsized":"2024-09-10T08:46:18.803799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:10.634708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194226+0000","last_change":"2024-09-10T08:44:10.676375+0000","last_active":"2024-09-10T08:46:19.194226+0000","last_peered":"2024-09-10T08:46:19.194226+0000","last_clean":"2024-09-10T08:46:19.194226+0000","last_became_active":"2024-09-10T08:44:10.676022+0000","last_became_peered":"2024-09-10T08:44:10.676022+0000","last_unstale":"2024-09-10T08:46:19.194226+0000","last_undegraded":"2024-09-10T08:46:19.194226+0000","last_fullsized":"2024-09-10T08:46:19.194226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:02.681954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193408+0000","last_change":"2024-09-10T08:44:10.673378+0000","last_active":"2024-09-10T08:46:19.193408+0000","last_peered":"2024-09-10T08:46:19.193408+0000","last_clean":"2024-09-10T08:46:19.193408+0000","last_became_active":"2024-09-10T08:44:10.673089+0000","last_became_peered":"2024-09-10T08:44:10.673089+0000","last_unstale":"2024-09-10T08:46:19.193408+0000","last_undegraded":"2024-09-10T08:46:19.193408+0000","last_fullsized":"2024-09-10T08:46:19.193408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:36:19.258651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192898+0000","last_change":"2024-09-10T08:44:10.671557+0000","last_active":"2024-09-10T08:46:19.192898+0000","last_peered":"2024-09-10T08:46:19.192898+0000","last_clean":"2024-09-10T08:46:19.192898+0000","last_became_active":"2024-09-10T08:44:10.671248+0000","last_became_peered":"2024-09-10T08:44:10.671248+0000","last_unstale":"2024-09-10T08:46:19.192898+0000","last_undegraded":"2024-09-10T08:46:19.192898+0000","last_fullsized":"2024-09-10T08:46:19.192898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:54.261525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193104+0000","last_change":"2024-09-10T08:44:10.674672+0000","last_active":"2024-09-10T08:46:19.193104+0000","last_peered":"2024-09-10T08:46:19.193104+0000","last_clean":"2024-09-10T08:46:19.193104+0000","last_became_active":"2024-09-10T08:44:10.673415+0000","last_became_peered":"2024-09-10T08:44:10.673415+0000","last_unstale":"2024-09-10T08:46:19.193104+0000","last_undegraded":"2024-09-10T08:46:19.193104+0000","last_fullsized":"2024-09-10T08:46:19.193104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:59:14.179664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806192+0000","last_change":"2024-09-10T08:44:10.675293+0000","last_active":"2024-09-10T08:46:18.806192+0000","last_peered":"2024-09-10T08:46:18.806192+0000","last_clean":"2024-09-10T08:46:18.806192+0000","last_became_active":"2024-09-10T08:44:10.674924+0000","last_became_peered":"2024-09-10T08:44:10.674924+0000","last_unstale":"2024-09-10T08:46:18.806192+0000","last_undegraded":"2024-09-10T08:46:18.806192+0000","last_fullsized":"2024-09-10T08:46:18.806192+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:56.027320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194671+0000","last_change":"2024-09-10T08:44:10.674605+0000","last_active":"2024-09-10T08:46:19.194671+0000","last_peered":"2024-09-10T08:46:19.194671+0000","last_clean":"2024-09-10T08:46:19.194671+0000","last_became_active":"2024-09-10T08:44:10.674057+0000","last_became_peered":"2024-09-10T08:44:10.674057+0000","last_unstale":"2024-09-10T08:46:19.194671+0000","last_undegraded":"2024-09-10T08:46:19.194671+0000","last_fullsized":"2024-09-10T08:46:19.194671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:09.473003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874065+0000","last_change":"2024-09-10T08:44:10.677410+0000","last_active":"2024-09-10T08:46:18.874065+0000","last_peered":"2024-09-10T08:46:18.874065+0000","last_clean":"2024-09-10T08:46:18.874065+0000","last_became_active":"2024-09-10T08:44:10.677109+0000","last_became_peered":"2024-09-10T08:44:10.677109+0000","last_unstale":"2024-09-10T08:46:18.874065+0000","last_undegraded":"2024-09-10T08:46:18.874065+0000","last_fullsized":"2024-09-10T08:46:18.874065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:45.455425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805091+0000","last_change":"2024-09-10T08:44:10.669706+0000","last_active":"2024-09-10T08:46:18.805091+0000","last_peered":"2024-09-10T08:46:18.805091+0000","last_clean":"2024-09-10T08:46:18.805091+0000","last_became_active":"2024-09-10T08:44:10.669440+0000","last_became_peered":"2024-09-10T08:44:10.669440+0000","last_unstale":"2024-09-10T08:46:18.805091+0000","last_undegraded":"2024-09-10T08:46:18.805091+0000","last_fullsized":"2024-09-10T08:46:18.805091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:31:55.610651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194758+0000","last_change":"2024-09-10T08:44:10.676482+0000","last_active":"2024-09-10T08:46:19.194758+0000","last_peered":"2024-09-10T08:46:19.194758+0000","last_clean":"2024-09-10T08:46:19.194758+0000","last_became_active":"2024-09-10T08:44:10.675983+0000","last_became_peered":"2024-09-10T08:44:10.675983+0000","last_unstale":"2024-09-10T08:46:19.194758+0000","last_undegraded":"2024-09-10T08:46:19.194758+0000","last_fullsized":"2024-09-10T08:46:19.194758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:01.269283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192653+0000","last_change":"2024-09-10T08:44:10.675681+0000","last_active":"2024-09-10T08:46:19.192653+0000","last_peered":"2024-09-10T08:46:19.192653+0000","last_clean":"2024-09-10T08:46:19.192653+0000","last_became_active":"2024-09-10T08:44:10.675357+0000","last_became_peered":"2024-09-10T08:44:10.675357+0000","last_unstale":"2024-09-10T08:46:19.192653+0000","last_undegraded":"2024-09-10T08:46:19.192653+0000","last_fullsized":"2024-09-10T08:46:19.192653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:26.138333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873467+0000","last_change":"2024-09-10T08:44:10.680819+0000","last_active":"2024-09-10T08:46:18.873467+0000","last_peered":"2024-09-10T08:46:18.873467+0000","last_clean":"2024-09-10T08:46:18.873467+0000","last_became_active":"2024-09-10T08:44:10.680491+0000","last_became_peered":"2024-09-10T08:44:10.680491+0000","last_unstale":"2024-09-10T08:46:18.873467+0000","last_undegraded":"2024-09-10T08:46:18.873467+0000","last_fullsized":"2024-09-10T08:46:18.873467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:17.044776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804562+0000","last_change":"2024-09-10T08:44:10.667376+0000","last_active":"2024-09-10T08:46:18.804562+0000","last_peered":"2024-09-10T08:46:18.804562+0000","last_clean":"2024-09-10T08:46:18.804562+0000","last_became_active":"2024-09-10T08:44:10.666648+0000","last_became_peered":"2024-09-10T08:44:10.666648+0000","last_unstale":"2024-09-10T08:46:18.804562+0000","last_undegraded":"2024-09-10T08:46:18.804562+0000","last_fullsized":"2024-09-10T08:46:18.804562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:33:12.017550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193207+0000","last_change":"2024-09-10T08:44:10.668665+0000","last_active":"2024-09-10T08:46:19.193207+0000","last_peered":"2024-09-10T08:46:19.193207+0000","last_clean":"2024-09-10T08:46:19.193207+0000","last_became_active":"2024-09-10T08:44:10.666031+0000","last_became_peered":"2024-09-10T08:44:10.666031+0000","last_unstale":"2024-09-10T08:46:19.193207+0000","last_undegraded":"2024-09-10T08:46:19.193207+0000","last_fullsized":"2024-09-10T08:46:19.193207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:39:53.991463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194458+0000","last_change":"2024-09-10T08:44:10.676495+0000","last_active":"2024-09-10T08:46:19.194458+0000","last_peered":"2024-09-10T08:46:19.194458+0000","last_clean":"2024-09-10T08:46:19.194458+0000","last_became_active":"2024-09-10T08:44:10.676087+0000","last_became_peered":"2024-09-10T08:44:10.676087+0000","last_unstale":"2024-09-10T08:46:19.194458+0000","last_undegraded":"2024-09-10T08:46:19.194458+0000","last_fullsized":"2024-09-10T08:46:19.194458+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:41.828157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193138+0000","last_change":"2024-09-10T08:44:10.672835+0000","last_active":"2024-09-10T08:46:19.193138+0000","last_peered":"2024-09-10T08:46:19.193138+0000","last_clean":"2024-09-10T08:46:19.193138+0000","last_became_active":"2024-09-10T08:44:10.672537+0000","last_became_peered":"2024-09-10T08:44:10.672537+0000","last_unstale":"2024-09-10T08:46:19.193138+0000","last_undegraded":"2024-09-10T08:46:19.193138+0000","last_fullsized":"2024-09-10T08:46:19.193138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:06.121460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193568+0000","last_change":"2024-09-10T08:44:10.675968+0000","last_active":"2024-09-10T08:46:19.193568+0000","last_peered":"2024-09-10T08:46:19.193568+0000","last_clean":"2024-09-10T08:46:19.193568+0000","last_became_active":"2024-09-10T08:44:10.675628+0000","last_became_peered":"2024-09-10T08:44:10.675628+0000","last_unstale":"2024-09-10T08:46:19.193568+0000","last_undegraded":"2024-09-10T08:46:19.193568+0000","last_fullsized":"2024-09-10T08:46:19.193568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:40.261336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194091+0000","last_change":"2024-09-10T08:44:10.673151+0000","last_active":"2024-09-10T08:46:19.194091+0000","last_peered":"2024-09-10T08:46:19.194091+0000","last_clean":"2024-09-10T08:46:19.194091+0000","last_became_active":"2024-09-10T08:44:10.672833+0000","last_became_peered":"2024-09-10T08:44:10.672833+0000","last_unstale":"2024-09-10T08:46:19.194091+0000","last_undegraded":"2024-09-10T08:46:19.194091+0000","last_fullsized":"2024-09-10T08:46:19.194091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:38.556222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73164,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1067,"num_read_kb":614,"num_write":273,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":228120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514539,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":523370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743472,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28736,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716416,"statfs":{"total":95995035648,"available":95965609984,"internally_reserved":0,"allocated":1753088,"data_stored":1173880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906997,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":526367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135930,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":522649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364863,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1120240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528389,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":518270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36988,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708164,"statfs":{"total":95995035648,"available":95957159936,"internally_reserved":0,"allocated":1683456,"data_stored":1114832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1064960,"data_stored":521849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":517483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":584777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1126400,"data_stored":583528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":520260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:46:41.035 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-10T08:46:41.035 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T08:46:41.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.172+0000 7f11899fd640 1 -- 172.21.15.172:0/515281934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11840ffdf0 msgr2=0x7f11841001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:41.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.172+0000 7f11899fd640 1 --2- 172.21.15.172:0/515281934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11840ffdf0 0x7f11841001f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1174009900 tx=0x7f117402a410 comp rx=0 tx=0).stop 2024-09-10T08:46:41.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.172+0000 7f11899fd640 1 -- 172.21.15.172:0/515281934 shutdown_connections 2024-09-10T08:46:41.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.172+0000 7f11899fd640 1 --2- 172.21.15.172:0/515281934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1184103780 0x7f118410a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:41.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.172+0000 7f11899fd640 1 --2- 172.21.15.172:0/515281934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1184102dc0 0x7f1184103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:41.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.172+0000 7f11899fd640 1 --2- 172.21.15.172:0/515281934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11840ffdf0 0x7f11841001f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:41.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.172+0000 7f11899fd640 1 -- 172.21.15.172:0/515281934 >> 172.21.15.172:0/515281934 conn(0x7f11840fb950 msgr2=0x7f11840fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:41.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.172+0000 7f11899fd640 1 -- 172.21.15.172:0/515281934 shutdown_connections 2024-09-10T08:46:41.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.172+0000 7f11899fd640 1 -- 172.21.15.172:0/515281934 wait complete. 2024-09-10T08:46:41.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f11899fd640 1 Processor -- start 2024-09-10T08:46:41.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f11899fd640 1 -- start start 2024-09-10T08:46:41.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f11899fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11840ffdf0 0x7f1184197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:41.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f11899fd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1184102dc0 0x7f11841982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:41.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f11899fd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1184103780 0x7f118419f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:41.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f11899fd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f118410d650 con 0x7f11840ffdf0 2024-09-10T08:46:41.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f11899fd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f118410d4d0 con 0x7f1184103780 2024-09-10T08:46:41.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f11899fd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f118410d7d0 con 0x7f1184102dc0 2024-09-10T08:46:41.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f1182ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11840ffdf0 0x7f1184197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:41.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f1182ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11840ffdf0 0x7f1184197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33290/0 (socket says 172.21.15.172:33290) 2024-09-10T08:46:41.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f1182ffd640 1 -- 172.21.15.172:0/2636504156 learned_addr learned my addr 172.21.15.172:0/2636504156 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:41.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f117a7fc640 1 --2- 172.21.15.172:0/2636504156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1184102dc0 0x7f11841982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:41.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f1182ffd640 1 -- 172.21.15.172:0/2636504156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1184102dc0 msgr2=0x7f11841982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:41.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f1182ffd640 1 --2- 172.21.15.172:0/2636504156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1184102dc0 0x7f11841982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:41.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f1182ffd640 1 -- 172.21.15.172:0/2636504156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1184103780 msgr2=0x7f118419f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:41.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f11837fe640 1 --2- 172.21.15.172:0/2636504156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1184103780 0x7f118419f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:41.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f1182ffd640 1 --2- 172.21.15.172:0/2636504156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1184103780 0x7f118419f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:41.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f1182ffd640 1 -- 172.21.15.172:0/2636504156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118419f9c0 con 0x7f11840ffdf0 2024-09-10T08:46:41.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.173+0000 7f11837fe640 1 --2- 172.21.15.172:0/2636504156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1184103780 0x7f118419f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:41.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.174+0000 7f1182ffd640 1 --2- 172.21.15.172:0/2636504156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11840ffdf0 0x7f1184197db0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f11740314e0 tx=0x7f1174004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:41.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.174+0000 7f11889fb640 1 -- 172.21.15.172:0/2636504156 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117403f030 con 0x7f11840ffdf0 2024-09-10T08:46:41.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.174+0000 7f11889fb640 1 -- 172.21.15.172:0/2636504156 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1174003cb0 con 0x7f11840ffdf0 2024-09-10T08:46:41.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.174+0000 7f11899fd640 1 -- 172.21.15.172:0/2636504156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f118419fc50 con 0x7f11840ffdf0 2024-09-10T08:46:41.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.174+0000 7f11889fb640 1 -- 172.21.15.172:0/2636504156 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1174035070 con 0x7f11840ffdf0 2024-09-10T08:46:41.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.174+0000 7f11899fd640 1 -- 172.21.15.172:0/2636504156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11841a0140 con 0x7f11840ffdf0 2024-09-10T08:46:41.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.175+0000 7f11899fd640 1 -- 172.21.15.172:0/2636504156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1184109050 con 0x7f11840ffdf0 2024-09-10T08:46:41.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.176+0000 7f11889fb640 1 -- 172.21.15.172:0/2636504156 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1174042030 con 0x7f11840ffdf0 2024-09-10T08:46:41.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.177+0000 7f11889fb640 1 --2- 172.21.15.172:0/2636504156 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11700a2b90 0x7f11700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:41.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.177+0000 7f117a7fc640 1 --2- 172.21.15.172:0/2636504156 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11700a2b90 0x7f11700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:41.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.177+0000 7f11889fb640 1 -- 172.21.15.172:0/2636504156 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f11740efa40 con 0x7f11840ffdf0 2024-09-10T08:46:41.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.178+0000 7f117a7fc640 1 --2- 172.21.15.172:0/2636504156 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11700a2b90 0x7f11700a5050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f116c005d00 tx=0x7f116c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:41.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.181+0000 7f11889fb640 1 -- 172.21.15.172:0/2636504156 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11740bc640 con 0x7f11840ffdf0 2024-09-10T08:46:41.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.353+0000 7f11899fd640 1 -- 172.21.15.172:0/2636504156 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1184062630 con 0x7f11700a2b90 2024-09-10T08:46:41.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.361+0000 7f11889fb640 1 -- 172.21.15.172:0/2636504156 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340551 (secure 0 0 0) 0x7f1184062630 con 0x7f11700a2b90 2024-09-10T08:46:41.365 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:41.367 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-09-10T08:46:41.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.364+0000 7f11899fd640 1 -- 172.21.15.172:0/2636504156 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11700a2b90 msgr2=0x7f11700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:41.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.364+0000 7f11899fd640 1 --2- 172.21.15.172:0/2636504156 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11700a2b90 0x7f11700a5050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f116c005d00 tx=0x7f116c00b410 comp rx=0 tx=0).stop 2024-09-10T08:46:41.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.364+0000 7f11899fd640 1 -- 172.21.15.172:0/2636504156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11840ffdf0 msgr2=0x7f1184197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:41.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.364+0000 7f11899fd640 1 --2- 172.21.15.172:0/2636504156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11840ffdf0 0x7f1184197db0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f11740314e0 tx=0x7f1174004270 comp rx=0 tx=0).stop 2024-09-10T08:46:41.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.364+0000 7f11899fd640 1 -- 172.21.15.172:0/2636504156 shutdown_connections 2024-09-10T08:46:41.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.364+0000 7f11899fd640 1 --2- 172.21.15.172:0/2636504156 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11700a2b90 0x7f11700a5050 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:41.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.364+0000 7f11899fd640 1 --2- 172.21.15.172:0/2636504156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1184103780 0x7f118419f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:41.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.364+0000 7f11899fd640 1 --2- 172.21.15.172:0/2636504156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1184102dc0 0x7f11841982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:41.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.364+0000 7f11899fd640 1 --2- 172.21.15.172:0/2636504156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11840ffdf0 0x7f1184197db0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:41.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.364+0000 7f11899fd640 1 -- 172.21.15.172:0/2636504156 >> 172.21.15.172:0/2636504156 conn(0x7f11840fb950 msgr2=0x7f1184106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:41.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.364+0000 7f11899fd640 1 -- 172.21.15.172:0/2636504156 shutdown_connections 2024-09-10T08:46:41.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:41.364+0000 7f11899fd640 1 -- 172.21.15.172:0/2636504156 wait complete. 2024-09-10T08:46:41.376 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-10T08:46:41.164771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668080,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1225,"num_read_kb":744,"num_write":457,"num_write_kb":3106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":873,"ondisk_log_size":873,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376260,"kb_used_data":14532,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565564,"statfs":{"total":1151940427776,"available":1151555137536,"internally_reserved":0,"allocated":14880768,"data_stored":8232314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6789,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":136,"num_read_kb":75,"num_write":19,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003254"},"pg_stats":[{"pgid":"2.1f","version":"92'14","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.363708+0000","last_change":"2024-09-10T08:44:08.657737+0000","last_active":"2024-09-10T08:46:34.363708+0000","last_peered":"2024-09-10T08:46:34.363708+0000","last_clean":"2024-09-10T08:46:34.363708+0000","last_became_active":"2024-09-10T08:44:08.657473+0000","last_became_peered":"2024-09-10T08:44:08.657473+0000","last_unstale":"2024-09-10T08:46:34.363708+0000","last_undegraded":"2024-09-10T08:46:34.363708+0000","last_fullsized":"2024-09-10T08:46:34.363708+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:40.757343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193599+0000","last_change":"2024-09-10T08:44:10.675090+0000","last_active":"2024-09-10T08:46:19.193599+0000","last_peered":"2024-09-10T08:46:19.193599+0000","last_clean":"2024-09-10T08:46:19.193599+0000","last_became_active":"2024-09-10T08:44:10.674612+0000","last_became_peered":"2024-09-10T08:44:10.674612+0000","last_unstale":"2024-09-10T08:46:19.193599+0000","last_undegraded":"2024-09-10T08:46:19.193599+0000","last_fullsized":"2024-09-10T08:46:19.193599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:46.821080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.807904+0000","last_change":"2024-09-10T08:44:17.721559+0000","last_active":"2024-09-10T08:46:18.807904+0000","last_peered":"2024-09-10T08:46:18.807904+0000","last_clean":"2024-09-10T08:46:18.807904+0000","last_became_active":"2024-09-10T08:44:17.721182+0000","last_became_peered":"2024-09-10T08:44:17.721182+0000","last_unstale":"2024-09-10T08:46:18.807904+0000","last_undegraded":"2024-09-10T08:46:18.807904+0000","last_fullsized":"2024-09-10T08:46:18.807904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:18.791489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193663+0000","last_change":"2024-09-10T08:44:09.049171+0000","last_active":"2024-09-10T08:46:19.193663+0000","last_peered":"2024-09-10T08:46:19.193663+0000","last_clean":"2024-09-10T08:46:19.193663+0000","last_became_active":"2024-09-10T08:44:09.048806+0000","last_became_peered":"2024-09-10T08:44:09.048806+0000","last_unstale":"2024-09-10T08:46:19.193663+0000","last_undegraded":"2024-09-10T08:46:19.193663+0000","last_fullsized":"2024-09-10T08:46:19.193663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:21.023460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193866+0000","last_change":"2024-09-10T08:44:10.677569+0000","last_active":"2024-09-10T08:46:19.193866+0000","last_peered":"2024-09-10T08:46:19.193866+0000","last_clean":"2024-09-10T08:46:19.193866+0000","last_became_active":"2024-09-10T08:44:10.677048+0000","last_became_peered":"2024-09-10T08:44:10.677048+0000","last_unstale":"2024-09-10T08:46:19.193866+0000","last_undegraded":"2024-09-10T08:46:19.193866+0000","last_fullsized":"2024-09-10T08:46:19.193866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:35.523063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193330+0000","last_change":"2024-09-10T08:44:17.720138+0000","last_active":"2024-09-10T08:46:19.193330+0000","last_peered":"2024-09-10T08:46:19.193330+0000","last_clean":"2024-09-10T08:46:19.193330+0000","last_became_active":"2024-09-10T08:44:17.719344+0000","last_became_peered":"2024-09-10T08:44:17.719344+0000","last_unstale":"2024-09-10T08:46:19.193330+0000","last_undegraded":"2024-09-10T08:46:19.193330+0000","last_fullsized":"2024-09-10T08:46:19.193330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:11.758058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194115+0000","last_change":"2024-09-10T08:44:08.660099+0000","last_active":"2024-09-10T08:46:19.194115+0000","last_peered":"2024-09-10T08:46:19.194115+0000","last_clean":"2024-09-10T08:46:19.194115+0000","last_became_active":"2024-09-10T08:44:08.659745+0000","last_became_peered":"2024-09-10T08:44:08.659745+0000","last_unstale":"2024-09-10T08:46:19.194115+0000","last_undegraded":"2024-09-10T08:46:19.194115+0000","last_fullsized":"2024-09-10T08:46:19.194115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:43.960717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193698+0000","last_change":"2024-09-10T08:44:10.675583+0000","last_active":"2024-09-10T08:46:19.193698+0000","last_peered":"2024-09-10T08:46:19.193698+0000","last_clean":"2024-09-10T08:46:19.193698+0000","last_became_active":"2024-09-10T08:44:10.675296+0000","last_became_peered":"2024-09-10T08:44:10.675296+0000","last_unstale":"2024-09-10T08:46:19.193698+0000","last_undegraded":"2024-09-10T08:46:19.193698+0000","last_fullsized":"2024-09-10T08:46:19.193698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:31:51.147862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194363+0000","last_change":"2024-09-10T08:44:17.720104+0000","last_active":"2024-09-10T08:46:19.194363+0000","last_peered":"2024-09-10T08:46:19.194363+0000","last_clean":"2024-09-10T08:46:19.194363+0000","last_became_active":"2024-09-10T08:44:17.719607+0000","last_became_peered":"2024-09-10T08:44:17.719607+0000","last_unstale":"2024-09-10T08:46:19.194363+0000","last_undegraded":"2024-09-10T08:46:19.194363+0000","last_fullsized":"2024-09-10T08:46:19.194363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:52.976717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193719+0000","last_change":"2024-09-10T08:44:08.658588+0000","last_active":"2024-09-10T08:46:19.193719+0000","last_peered":"2024-09-10T08:46:19.193719+0000","last_clean":"2024-09-10T08:46:19.193719+0000","last_became_active":"2024-09-10T08:44:08.658053+0000","last_became_peered":"2024-09-10T08:44:08.658053+0000","last_unstale":"2024-09-10T08:46:19.193719+0000","last_undegraded":"2024-09-10T08:46:19.193719+0000","last_fullsized":"2024-09-10T08:46:19.193719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:26.744943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804775+0000","last_change":"2024-09-10T08:44:10.681978+0000","last_active":"2024-09-10T08:46:18.804775+0000","last_peered":"2024-09-10T08:46:18.804775+0000","last_clean":"2024-09-10T08:46:18.804775+0000","last_became_active":"2024-09-10T08:44:10.681600+0000","last_became_peered":"2024-09-10T08:44:10.681600+0000","last_unstale":"2024-09-10T08:46:18.804775+0000","last_undegraded":"2024-09-10T08:46:18.804775+0000","last_fullsized":"2024-09-10T08:46:18.804775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:37.190576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193813+0000","last_change":"2024-09-10T08:44:17.719784+0000","last_active":"2024-09-10T08:46:19.193813+0000","last_peered":"2024-09-10T08:46:19.193813+0000","last_clean":"2024-09-10T08:46:19.193813+0000","last_became_active":"2024-09-10T08:44:17.719182+0000","last_became_peered":"2024-09-10T08:44:17.719182+0000","last_unstale":"2024-09-10T08:46:19.193813+0000","last_undegraded":"2024-09-10T08:46:19.193813+0000","last_fullsized":"2024-09-10T08:46:19.193813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:15.237772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873709+0000","last_change":"2024-09-10T08:44:10.679069+0000","last_active":"2024-09-10T08:46:18.873709+0000","last_peered":"2024-09-10T08:46:18.873709+0000","last_clean":"2024-09-10T08:46:18.873709+0000","last_became_active":"2024-09-10T08:44:10.678597+0000","last_became_peered":"2024-09-10T08:44:10.678597+0000","last_unstale":"2024-09-10T08:46:18.873709+0000","last_undegraded":"2024-09-10T08:46:18.873709+0000","last_fullsized":"2024-09-10T08:46:18.873709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:52.693906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194153+0000","last_change":"2024-09-10T08:44:08.660886+0000","last_active":"2024-09-10T08:46:19.194153+0000","last_peered":"2024-09-10T08:46:19.194153+0000","last_clean":"2024-09-10T08:46:19.194153+0000","last_became_active":"2024-09-10T08:44:08.660478+0000","last_became_peered":"2024-09-10T08:44:08.660478+0000","last_unstale":"2024-09-10T08:46:19.194153+0000","last_undegraded":"2024-09-10T08:46:19.194153+0000","last_fullsized":"2024-09-10T08:46:19.194153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:41.785531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194323+0000","last_change":"2024-09-10T08:44:17.761929+0000","last_active":"2024-09-10T08:46:19.194323+0000","last_peered":"2024-09-10T08:46:19.194323+0000","last_clean":"2024-09-10T08:46:19.194323+0000","last_became_active":"2024-09-10T08:44:17.761440+0000","last_became_peered":"2024-09-10T08:44:17.761440+0000","last_unstale":"2024-09-10T08:46:19.194323+0000","last_undegraded":"2024-09-10T08:46:19.194323+0000","last_fullsized":"2024-09-10T08:46:19.194323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:37.817759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192895+0000","last_change":"2024-09-10T08:44:10.674071+0000","last_active":"2024-09-10T08:46:19.192895+0000","last_peered":"2024-09-10T08:46:19.192895+0000","last_clean":"2024-09-10T08:46:19.192895+0000","last_became_active":"2024-09-10T08:44:10.673770+0000","last_became_peered":"2024-09-10T08:44:10.673770+0000","last_unstale":"2024-09-10T08:46:19.192895+0000","last_undegraded":"2024-09-10T08:46:19.192895+0000","last_fullsized":"2024-09-10T08:46:19.192895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:44:19.944557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.231938+0000","last_change":"2024-09-10T08:45:18.264060+0000","last_active":"2024-09-10T08:46:34.231938+0000","last_peered":"2024-09-10T08:46:34.231938+0000","last_clean":"2024-09-10T08:46:34.231938+0000","last_became_active":"2024-09-10T08:45:18.255983+0000","last_became_peered":"2024-09-10T08:45:18.255983+0000","last_unstale":"2024-09-10T08:46:34.231938+0000","last_undegraded":"2024-09-10T08:46:34.231938+0000","last_fullsized":"2024-09-10T08:46:34.231938+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:10.810491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194248+0000","last_change":"2024-09-10T08:44:17.722877+0000","last_active":"2024-09-10T08:46:19.194248+0000","last_peered":"2024-09-10T08:46:19.194248+0000","last_clean":"2024-09-10T08:46:19.194248+0000","last_became_active":"2024-09-10T08:44:17.722046+0000","last_became_peered":"2024-09-10T08:44:17.722046+0000","last_unstale":"2024-09-10T08:46:19.194248+0000","last_undegraded":"2024-09-10T08:46:19.194248+0000","last_fullsized":"2024-09-10T08:46:19.194248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:32.675598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193398+0000","last_change":"2024-09-10T08:44:10.671500+0000","last_active":"2024-09-10T08:46:19.193398+0000","last_peered":"2024-09-10T08:46:19.193398+0000","last_clean":"2024-09-10T08:46:19.193398+0000","last_became_active":"2024-09-10T08:44:10.671039+0000","last_became_peered":"2024-09-10T08:44:10.671039+0000","last_unstale":"2024-09-10T08:46:19.193398+0000","last_undegraded":"2024-09-10T08:46:19.193398+0000","last_fullsized":"2024-09-10T08:46:19.193398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:12.977373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193090+0000","last_change":"2024-09-10T08:44:08.655467+0000","last_active":"2024-09-10T08:46:19.193090+0000","last_peered":"2024-09-10T08:46:19.193090+0000","last_clean":"2024-09-10T08:46:19.193090+0000","last_became_active":"2024-09-10T08:44:08.654991+0000","last_became_peered":"2024-09-10T08:44:08.654991+0000","last_unstale":"2024-09-10T08:46:19.193090+0000","last_undegraded":"2024-09-10T08:46:19.193090+0000","last_fullsized":"2024-09-10T08:46:19.193090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:33.124360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803728+0000","last_change":"2024-09-10T08:44:17.724250+0000","last_active":"2024-09-10T08:46:18.803728+0000","last_peered":"2024-09-10T08:46:18.803728+0000","last_clean":"2024-09-10T08:46:18.803728+0000","last_became_active":"2024-09-10T08:44:17.723733+0000","last_became_peered":"2024-09-10T08:44:17.723733+0000","last_unstale":"2024-09-10T08:46:18.803728+0000","last_undegraded":"2024-09-10T08:46:18.803728+0000","last_fullsized":"2024-09-10T08:46:18.803728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:40.098666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193144+0000","last_change":"2024-09-10T08:44:10.664159+0000","last_active":"2024-09-10T08:46:19.193144+0000","last_peered":"2024-09-10T08:46:19.193144+0000","last_clean":"2024-09-10T08:46:19.193144+0000","last_became_active":"2024-09-10T08:44:10.663674+0000","last_became_peered":"2024-09-10T08:44:10.663674+0000","last_unstale":"2024-09-10T08:46:19.193144+0000","last_undegraded":"2024-09-10T08:46:19.193144+0000","last_fullsized":"2024-09-10T08:46:19.193144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:24.234525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193535+0000","last_change":"2024-09-10T08:45:18.257226+0000","last_active":"2024-09-10T08:46:19.193535+0000","last_peered":"2024-09-10T08:46:19.193535+0000","last_clean":"2024-09-10T08:46:19.193535+0000","last_became_active":"2024-09-10T08:45:18.254113+0000","last_became_peered":"2024-09-10T08:45:18.254113+0000","last_unstale":"2024-09-10T08:46:19.193535+0000","last_undegraded":"2024-09-10T08:46:19.193535+0000","last_fullsized":"2024-09-10T08:46:19.193535+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:19.370105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194467+0000","last_change":"2024-09-10T08:44:17.761639+0000","last_active":"2024-09-10T08:46:19.194467+0000","last_peered":"2024-09-10T08:46:19.194467+0000","last_clean":"2024-09-10T08:46:19.194467+0000","last_became_active":"2024-09-10T08:44:17.761214+0000","last_became_peered":"2024-09-10T08:44:17.761214+0000","last_unstale":"2024-09-10T08:46:19.194467+0000","last_undegraded":"2024-09-10T08:46:19.194467+0000","last_fullsized":"2024-09-10T08:46:19.194467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:43.793195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193666+0000","last_change":"2024-09-10T08:45:18.258661+0000","last_active":"2024-09-10T08:46:19.193666+0000","last_peered":"2024-09-10T08:46:19.193666+0000","last_clean":"2024-09-10T08:46:19.193666+0000","last_became_active":"2024-09-10T08:45:18.258225+0000","last_became_peered":"2024-09-10T08:45:18.258225+0000","last_unstale":"2024-09-10T08:46:19.193666+0000","last_undegraded":"2024-09-10T08:46:19.193666+0000","last_fullsized":"2024-09-10T08:46:19.193666+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:28:37.184766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194660+0000","last_change":"2024-09-10T08:44:09.036632+0000","last_active":"2024-09-10T08:46:19.194660+0000","last_peered":"2024-09-10T08:46:19.194660+0000","last_clean":"2024-09-10T08:46:19.194660+0000","last_became_active":"2024-09-10T08:44:09.036254+0000","last_became_peered":"2024-09-10T08:44:09.036254+0000","last_unstale":"2024-09-10T08:46:19.194660+0000","last_undegraded":"2024-09-10T08:46:19.194660+0000","last_fullsized":"2024-09-10T08:46:19.194660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:59:40.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808140+0000","last_change":"2024-09-10T08:44:17.716215+0000","last_active":"2024-09-10T08:46:18.808140+0000","last_peered":"2024-09-10T08:46:18.808140+0000","last_clean":"2024-09-10T08:46:18.808140+0000","last_became_active":"2024-09-10T08:44:17.715809+0000","last_became_peered":"2024-09-10T08:44:17.715809+0000","last_unstale":"2024-09-10T08:46:18.808140+0000","last_undegraded":"2024-09-10T08:46:18.808140+0000","last_fullsized":"2024-09-10T08:46:18.808140+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:52.896401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193713+0000","last_change":"2024-09-10T08:44:10.677871+0000","last_active":"2024-09-10T08:46:19.193713+0000","last_peered":"2024-09-10T08:46:19.193713+0000","last_clean":"2024-09-10T08:46:19.193713+0000","last_became_active":"2024-09-10T08:44:10.677101+0000","last_became_peered":"2024-09-10T08:44:10.677101+0000","last_unstale":"2024-09-10T08:46:19.193713+0000","last_undegraded":"2024-09-10T08:46:19.193713+0000","last_fullsized":"2024-09-10T08:46:19.193713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:46.137249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'17","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:21.176388+0000","last_change":"2024-09-10T08:44:08.654428+0000","last_active":"2024-09-10T08:46:21.176388+0000","last_peered":"2024-09-10T08:46:21.176388+0000","last_clean":"2024-09-10T08:46:21.176388+0000","last_became_active":"2024-09-10T08:44:08.654119+0000","last_became_peered":"2024-09-10T08:44:08.654119+0000","last_unstale":"2024-09-10T08:46:21.176388+0000","last_undegraded":"2024-09-10T08:46:21.176388+0000","last_fullsized":"2024-09-10T08:46:21.176388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:56.787721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192538+0000","last_change":"2024-09-10T08:44:17.760828+0000","last_active":"2024-09-10T08:46:19.192538+0000","last_peered":"2024-09-10T08:46:19.192538+0000","last_clean":"2024-09-10T08:46:19.192538+0000","last_became_active":"2024-09-10T08:44:17.760367+0000","last_became_peered":"2024-09-10T08:44:17.760367+0000","last_unstale":"2024-09-10T08:46:19.192538+0000","last_undegraded":"2024-09-10T08:46:19.192538+0000","last_fullsized":"2024-09-10T08:46:19.192538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:11.130480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806446+0000","last_change":"2024-09-10T08:44:10.670179+0000","last_active":"2024-09-10T08:46:18.806446+0000","last_peered":"2024-09-10T08:46:18.806446+0000","last_clean":"2024-09-10T08:46:18.806446+0000","last_became_active":"2024-09-10T08:44:10.669366+0000","last_became_peered":"2024-09-10T08:44:10.669366+0000","last_unstale":"2024-09-10T08:46:18.806446+0000","last_undegraded":"2024-09-10T08:46:18.806446+0000","last_fullsized":"2024-09-10T08:46:18.806446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:41.916365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194210+0000","last_change":"2024-09-10T08:44:09.050441+0000","last_active":"2024-09-10T08:46:19.194210+0000","last_peered":"2024-09-10T08:46:19.194210+0000","last_clean":"2024-09-10T08:46:19.194210+0000","last_became_active":"2024-09-10T08:44:09.050070+0000","last_became_peered":"2024-09-10T08:44:09.050070+0000","last_unstale":"2024-09-10T08:46:19.194210+0000","last_undegraded":"2024-09-10T08:46:19.194210+0000","last_fullsized":"2024-09-10T08:46:19.194210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:13.540021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193573+0000","last_change":"2024-09-10T08:44:17.720098+0000","last_active":"2024-09-10T08:46:19.193573+0000","last_peered":"2024-09-10T08:46:19.193573+0000","last_clean":"2024-09-10T08:46:19.193573+0000","last_became_active":"2024-09-10T08:44:17.719627+0000","last_became_peered":"2024-09-10T08:44:17.719627+0000","last_unstale":"2024-09-10T08:46:19.193573+0000","last_undegraded":"2024-09-10T08:46:19.193573+0000","last_fullsized":"2024-09-10T08:46:19.193573+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:50.020919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804677+0000","last_change":"2024-09-10T08:44:10.669630+0000","last_active":"2024-09-10T08:46:18.804677+0000","last_peered":"2024-09-10T08:46:18.804677+0000","last_clean":"2024-09-10T08:46:18.804677+0000","last_became_active":"2024-09-10T08:44:10.669307+0000","last_became_peered":"2024-09-10T08:44:10.669307+0000","last_unstale":"2024-09-10T08:46:18.804677+0000","last_undegraded":"2024-09-10T08:46:18.804677+0000","last_fullsized":"2024-09-10T08:46:18.804677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:06.320046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803568+0000","last_change":"2024-09-10T08:44:09.050715+0000","last_active":"2024-09-10T08:46:18.803568+0000","last_peered":"2024-09-10T08:46:18.803568+0000","last_clean":"2024-09-10T08:46:18.803568+0000","last_became_active":"2024-09-10T08:44:09.050347+0000","last_became_peered":"2024-09-10T08:44:09.050347+0000","last_unstale":"2024-09-10T08:46:18.803568+0000","last_undegraded":"2024-09-10T08:46:18.803568+0000","last_fullsized":"2024-09-10T08:46:18.803568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:18.689652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194038+0000","last_change":"2024-09-10T08:45:18.260444+0000","last_active":"2024-09-10T08:46:19.194038+0000","last_peered":"2024-09-10T08:46:19.194038+0000","last_clean":"2024-09-10T08:46:19.194038+0000","last_became_active":"2024-09-10T08:45:18.259949+0000","last_became_peered":"2024-09-10T08:45:18.259949+0000","last_unstale":"2024-09-10T08:46:19.194038+0000","last_undegraded":"2024-09-10T08:46:19.194038+0000","last_fullsized":"2024-09-10T08:46:19.194038+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:56.391253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804630+0000","last_change":"2024-09-10T08:44:10.670881+0000","last_active":"2024-09-10T08:46:18.804630+0000","last_peered":"2024-09-10T08:46:18.804630+0000","last_clean":"2024-09-10T08:46:18.804630+0000","last_became_active":"2024-09-10T08:44:10.669791+0000","last_became_peered":"2024-09-10T08:44:10.669791+0000","last_unstale":"2024-09-10T08:46:18.804630+0000","last_undegraded":"2024-09-10T08:46:18.804630+0000","last_fullsized":"2024-09-10T08:46:18.804630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:33.644205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'6","reported_seq":151,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:37.369215+0000","last_change":"2024-09-10T08:44:08.657938+0000","last_active":"2024-09-10T08:46:37.369215+0000","last_peered":"2024-09-10T08:46:37.369215+0000","last_clean":"2024-09-10T08:46:37.369215+0000","last_became_active":"2024-09-10T08:44:08.657526+0000","last_became_peered":"2024-09-10T08:44:08.657526+0000","last_unstale":"2024-09-10T08:46:37.369215+0000","last_undegraded":"2024-09-10T08:46:37.369215+0000","last_fullsized":"2024-09-10T08:46:37.369215+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:15.673561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806147+0000","last_change":"2024-09-10T08:44:17.721619+0000","last_active":"2024-09-10T08:46:18.806147+0000","last_peered":"2024-09-10T08:46:18.806147+0000","last_clean":"2024-09-10T08:46:18.806147+0000","last_became_active":"2024-09-10T08:44:17.721175+0000","last_became_peered":"2024-09-10T08:44:17.721175+0000","last_unstale":"2024-09-10T08:46:18.806147+0000","last_undegraded":"2024-09-10T08:46:18.806147+0000","last_fullsized":"2024-09-10T08:46:18.806147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:52.244120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804320+0000","last_change":"2024-09-10T08:44:10.676164+0000","last_active":"2024-09-10T08:46:18.804320+0000","last_peered":"2024-09-10T08:46:18.804320+0000","last_clean":"2024-09-10T08:46:18.804320+0000","last_became_active":"2024-09-10T08:44:10.675820+0000","last_became_peered":"2024-09-10T08:44:10.675820+0000","last_unstale":"2024-09-10T08:46:18.804320+0000","last_undegraded":"2024-09-10T08:46:18.804320+0000","last_fullsized":"2024-09-10T08:46:18.804320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:24.477584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":152,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:37.369105+0000","last_change":"2024-09-10T08:44:09.049124+0000","last_active":"2024-09-10T08:46:37.369105+0000","last_peered":"2024-09-10T08:46:37.369105+0000","last_clean":"2024-09-10T08:46:37.369105+0000","last_became_active":"2024-09-10T08:44:09.048799+0000","last_became_peered":"2024-09-10T08:44:09.048799+0000","last_unstale":"2024-09-10T08:46:37.369105+0000","last_undegraded":"2024-09-10T08:46:37.369105+0000","last_fullsized":"2024-09-10T08:46:37.369105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:43:39.849727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806891+0000","last_change":"2024-09-10T08:44:17.761130+0000","last_active":"2024-09-10T08:46:18.806891+0000","last_peered":"2024-09-10T08:46:18.806891+0000","last_clean":"2024-09-10T08:46:18.806891+0000","last_became_active":"2024-09-10T08:44:17.760753+0000","last_became_peered":"2024-09-10T08:44:17.760753+0000","last_unstale":"2024-09-10T08:46:18.806891+0000","last_undegraded":"2024-09-10T08:46:18.806891+0000","last_fullsized":"2024-09-10T08:46:18.806891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:42.971962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.809258+0000","last_change":"2024-09-10T08:46:18.809258+0000","last_active":"2024-09-10T08:46:18.809258+0000","last_peered":"2024-09-10T08:46:18.809258+0000","last_clean":"2024-09-10T08:46:18.809258+0000","last_became_active":"2024-09-10T08:46:18.808908+0000","last_became_peered":"2024-09-10T08:46:18.808908+0000","last_unstale":"2024-09-10T08:46:18.809258+0000","last_undegraded":"2024-09-10T08:46:18.809258+0000","last_fullsized":"2024-09-10T08:46:18.809258+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:10.305672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873500+0000","last_change":"2024-09-10T08:44:08.658862+0000","last_active":"2024-09-10T08:46:18.873500+0000","last_peered":"2024-09-10T08:46:18.873500+0000","last_clean":"2024-09-10T08:46:18.873500+0000","last_became_active":"2024-09-10T08:44:08.657880+0000","last_became_peered":"2024-09-10T08:44:08.657880+0000","last_unstale":"2024-09-10T08:46:18.873500+0000","last_undegraded":"2024-09-10T08:46:18.873500+0000","last_fullsized":"2024-09-10T08:46:18.873500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:34.224025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194185+0000","last_change":"2024-09-10T08:45:18.268446+0000","last_active":"2024-09-10T08:46:19.194185+0000","last_peered":"2024-09-10T08:46:19.194185+0000","last_clean":"2024-09-10T08:46:19.194185+0000","last_became_active":"2024-09-10T08:45:18.267988+0000","last_became_peered":"2024-09-10T08:45:18.267988+0000","last_unstale":"2024-09-10T08:46:19.194185+0000","last_undegraded":"2024-09-10T08:46:19.194185+0000","last_fullsized":"2024-09-10T08:46:19.194185+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:42:35.990794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193214+0000","last_change":"2024-09-10T08:44:10.673258+0000","last_active":"2024-09-10T08:46:19.193214+0000","last_peered":"2024-09-10T08:46:19.193214+0000","last_clean":"2024-09-10T08:46:19.193214+0000","last_became_active":"2024-09-10T08:44:10.672854+0000","last_became_peered":"2024-09-10T08:44:10.672854+0000","last_unstale":"2024-09-10T08:46:19.193214+0000","last_undegraded":"2024-09-10T08:46:19.193214+0000","last_fullsized":"2024-09-10T08:46:19.193214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:35.913820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194588+0000","last_change":"2024-09-10T08:44:08.658039+0000","last_active":"2024-09-10T08:46:19.194588+0000","last_peered":"2024-09-10T08:46:19.194588+0000","last_clean":"2024-09-10T08:46:19.194588+0000","last_became_active":"2024-09-10T08:44:08.657636+0000","last_became_peered":"2024-09-10T08:44:08.657636+0000","last_unstale":"2024-09-10T08:46:19.194588+0000","last_undegraded":"2024-09-10T08:46:19.194588+0000","last_fullsized":"2024-09-10T08:46:19.194588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:09.615992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873591+0000","last_change":"2024-09-10T08:42:57.666747+0000","last_active":"2024-09-10T08:46:18.873591+0000","last_peered":"2024-09-10T08:46:18.873591+0000","last_clean":"2024-09-10T08:46:18.873591+0000","last_became_active":"2024-09-10T08:42:57.666417+0000","last_became_peered":"2024-09-10T08:42:57.666417+0000","last_unstale":"2024-09-10T08:46:18.873591+0000","last_undegraded":"2024-09-10T08:46:18.873591+0000","last_fullsized":"2024-09-10T08:46:18.873591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_clean_scrub_stamp":"2024-09-10T08:39:31.729794+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.547989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874074+0000","last_change":"2024-09-10T08:44:17.723021+0000","last_active":"2024-09-10T08:46:18.874074+0000","last_peered":"2024-09-10T08:46:18.874074+0000","last_clean":"2024-09-10T08:46:18.874074+0000","last_became_active":"2024-09-10T08:44:17.722222+0000","last_became_peered":"2024-09-10T08:44:17.722222+0000","last_unstale":"2024-09-10T08:46:18.874074+0000","last_undegraded":"2024-09-10T08:46:18.874074+0000","last_fullsized":"2024-09-10T08:46:18.874074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:24.444427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874129+0000","last_change":"2024-09-10T08:44:10.676651+0000","last_active":"2024-09-10T08:46:18.874129+0000","last_peered":"2024-09-10T08:46:18.874129+0000","last_clean":"2024-09-10T08:46:18.874129+0000","last_became_active":"2024-09-10T08:44:10.676342+0000","last_became_peered":"2024-09-10T08:44:10.676342+0000","last_unstale":"2024-09-10T08:46:18.874129+0000","last_undegraded":"2024-09-10T08:46:18.874129+0000","last_fullsized":"2024-09-10T08:46:18.874129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:07.021273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'8","reported_seq":154,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:35.366830+0000","last_change":"2024-09-10T08:44:09.050305+0000","last_active":"2024-09-10T08:46:35.366830+0000","last_peered":"2024-09-10T08:46:35.366830+0000","last_clean":"2024-09-10T08:46:35.366830+0000","last_became_active":"2024-09-10T08:44:09.049958+0000","last_became_peered":"2024-09-10T08:44:09.049958+0000","last_unstale":"2024-09-10T08:46:35.366830+0000","last_undegraded":"2024-09-10T08:46:35.366830+0000","last_fullsized":"2024-09-10T08:46:35.366830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:42.219190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2199,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193720+0000","last_change":"2024-09-10T08:44:17.720219+0000","last_active":"2024-09-10T08:46:19.193720+0000","last_peered":"2024-09-10T08:46:19.193720+0000","last_clean":"2024-09-10T08:46:19.193720+0000","last_became_active":"2024-09-10T08:44:17.719756+0000","last_became_peered":"2024-09-10T08:44:17.719756+0000","last_unstale":"2024-09-10T08:46:19.193720+0000","last_undegraded":"2024-09-10T08:46:19.193720+0000","last_fullsized":"2024-09-10T08:46:19.193720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:01.070434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873929+0000","last_change":"2024-09-10T08:44:10.678986+0000","last_active":"2024-09-10T08:46:18.873929+0000","last_peered":"2024-09-10T08:46:18.873929+0000","last_clean":"2024-09-10T08:46:18.873929+0000","last_became_active":"2024-09-10T08:44:10.678654+0000","last_became_peered":"2024-09-10T08:44:10.678654+0000","last_unstale":"2024-09-10T08:46:18.873929+0000","last_undegraded":"2024-09-10T08:46:18.873929+0000","last_fullsized":"2024-09-10T08:46:18.873929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:07.116802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805026+0000","last_change":"2024-09-10T08:44:08.655025+0000","last_active":"2024-09-10T08:46:18.805026+0000","last_peered":"2024-09-10T08:46:18.805026+0000","last_clean":"2024-09-10T08:46:18.805026+0000","last_became_active":"2024-09-10T08:44:08.654728+0000","last_became_peered":"2024-09-10T08:44:08.654728+0000","last_unstale":"2024-09-10T08:46:18.805026+0000","last_undegraded":"2024-09-10T08:46:18.805026+0000","last_fullsized":"2024-09-10T08:46:18.805026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:31.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193534+0000","last_change":"2024-09-10T08:45:18.261492+0000","last_active":"2024-09-10T08:46:19.193534+0000","last_peered":"2024-09-10T08:46:19.193534+0000","last_clean":"2024-09-10T08:46:19.193534+0000","last_became_active":"2024-09-10T08:45:18.261111+0000","last_became_peered":"2024-09-10T08:45:18.261111+0000","last_unstale":"2024-09-10T08:46:19.193534+0000","last_undegraded":"2024-09-10T08:46:19.193534+0000","last_fullsized":"2024-09-10T08:46:19.193534+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:48.709459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874006+0000","last_change":"2024-09-10T08:44:10.675072+0000","last_active":"2024-09-10T08:46:18.874006+0000","last_peered":"2024-09-10T08:46:18.874006+0000","last_clean":"2024-09-10T08:46:18.874006+0000","last_became_active":"2024-09-10T08:44:10.674636+0000","last_became_peered":"2024-09-10T08:44:10.674636+0000","last_unstale":"2024-09-10T08:46:18.874006+0000","last_undegraded":"2024-09-10T08:46:18.874006+0000","last_fullsized":"2024-09-10T08:46:18.874006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:55.198445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806864+0000","last_change":"2024-09-10T08:44:08.656450+0000","last_active":"2024-09-10T08:46:18.806864+0000","last_peered":"2024-09-10T08:46:18.806864+0000","last_clean":"2024-09-10T08:46:18.806864+0000","last_became_active":"2024-09-10T08:44:08.655967+0000","last_became_peered":"2024-09-10T08:44:08.655967+0000","last_unstale":"2024-09-10T08:46:18.806864+0000","last_undegraded":"2024-09-10T08:46:18.806864+0000","last_fullsized":"2024-09-10T08:46:18.806864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:29.273167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804195+0000","last_change":"2024-09-10T08:44:17.722217+0000","last_active":"2024-09-10T08:46:18.804195+0000","last_peered":"2024-09-10T08:46:18.804195+0000","last_clean":"2024-09-10T08:46:18.804195+0000","last_became_active":"2024-09-10T08:44:17.721800+0000","last_became_peered":"2024-09-10T08:44:17.721800+0000","last_unstale":"2024-09-10T08:46:18.804195+0000","last_undegraded":"2024-09-10T08:46:18.804195+0000","last_fullsized":"2024-09-10T08:46:18.804195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:58.954397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194230+0000","last_change":"2024-09-10T08:45:18.267857+0000","last_active":"2024-09-10T08:46:19.194230+0000","last_peered":"2024-09-10T08:46:19.194230+0000","last_clean":"2024-09-10T08:46:19.194230+0000","last_became_active":"2024-09-10T08:45:18.267440+0000","last_became_peered":"2024-09-10T08:45:18.267440+0000","last_unstale":"2024-09-10T08:46:19.194230+0000","last_undegraded":"2024-09-10T08:46:19.194230+0000","last_fullsized":"2024-09-10T08:46:19.194230+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:28.659915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:23.312217+0000","last_change":"2024-09-10T08:44:08.659003+0000","last_active":"2024-09-10T08:46:23.312217+0000","last_peered":"2024-09-10T08:46:23.312217+0000","last_clean":"2024-09-10T08:46:23.312217+0000","last_became_active":"2024-09-10T08:44:08.658324+0000","last_became_peered":"2024-09-10T08:44:08.658324+0000","last_unstale":"2024-09-10T08:46:23.312217+0000","last_undegraded":"2024-09-10T08:46:23.312217+0000","last_fullsized":"2024-09-10T08:46:23.312217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:44.743053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804596+0000","last_change":"2024-09-10T08:44:17.723363+0000","last_active":"2024-09-10T08:46:18.804596+0000","last_peered":"2024-09-10T08:46:18.804596+0000","last_clean":"2024-09-10T08:46:18.804596+0000","last_became_active":"2024-09-10T08:44:17.721799+0000","last_became_peered":"2024-09-10T08:44:17.721799+0000","last_unstale":"2024-09-10T08:46:18.804596+0000","last_undegraded":"2024-09-10T08:46:18.804596+0000","last_fullsized":"2024-09-10T08:46:18.804596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:08:28.772193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804709+0000","last_change":"2024-09-10T08:44:10.682952+0000","last_active":"2024-09-10T08:46:18.804709+0000","last_peered":"2024-09-10T08:46:18.804709+0000","last_clean":"2024-09-10T08:46:18.804709+0000","last_became_active":"2024-09-10T08:44:10.682249+0000","last_became_peered":"2024-09-10T08:44:10.682249+0000","last_unstale":"2024-09-10T08:46:18.804709+0000","last_undegraded":"2024-09-10T08:46:18.804709+0000","last_fullsized":"2024-09-10T08:46:18.804709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:32:21.285461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193239+0000","last_change":"2024-09-10T08:44:09.050454+0000","last_active":"2024-09-10T08:46:19.193239+0000","last_peered":"2024-09-10T08:46:19.193239+0000","last_clean":"2024-09-10T08:46:19.193239+0000","last_became_active":"2024-09-10T08:44:09.050036+0000","last_became_peered":"2024-09-10T08:44:09.050036+0000","last_unstale":"2024-09-10T08:46:19.193239+0000","last_undegraded":"2024-09-10T08:46:19.193239+0000","last_fullsized":"2024-09-10T08:46:19.193239+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:55.141471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806678+0000","last_change":"2024-09-10T08:44:17.721203+0000","last_active":"2024-09-10T08:46:18.806678+0000","last_peered":"2024-09-10T08:46:18.806678+0000","last_clean":"2024-09-10T08:46:18.806678+0000","last_became_active":"2024-09-10T08:44:17.720737+0000","last_became_peered":"2024-09-10T08:44:17.720737+0000","last_unstale":"2024-09-10T08:46:18.806678+0000","last_undegraded":"2024-09-10T08:46:18.806678+0000","last_fullsized":"2024-09-10T08:46:18.806678+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:32.081079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873435+0000","last_change":"2024-09-10T08:44:10.680944+0000","last_active":"2024-09-10T08:46:18.873435+0000","last_peered":"2024-09-10T08:46:18.873435+0000","last_clean":"2024-09-10T08:46:18.873435+0000","last_became_active":"2024-09-10T08:44:10.680644+0000","last_became_peered":"2024-09-10T08:44:10.680644+0000","last_unstale":"2024-09-10T08:46:18.873435+0000","last_undegraded":"2024-09-10T08:46:18.873435+0000","last_fullsized":"2024-09-10T08:46:18.873435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:46.495894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":154,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:37.369880+0000","last_change":"2024-09-10T08:44:08.658135+0000","last_active":"2024-09-10T08:46:37.369880+0000","last_peered":"2024-09-10T08:46:37.369880+0000","last_clean":"2024-09-10T08:46:37.369880+0000","last_became_active":"2024-09-10T08:44:08.657793+0000","last_became_peered":"2024-09-10T08:44:08.657793+0000","last_unstale":"2024-09-10T08:46:37.369880+0000","last_undegraded":"2024-09-10T08:46:37.369880+0000","last_fullsized":"2024-09-10T08:46:37.369880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:16.376268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193715+0000","last_change":"2024-09-10T08:45:18.271678+0000","last_active":"2024-09-10T08:46:19.193715+0000","last_peered":"2024-09-10T08:46:19.193715+0000","last_clean":"2024-09-10T08:46:19.193715+0000","last_became_active":"2024-09-10T08:45:18.271209+0000","last_became_peered":"2024-09-10T08:45:18.271209+0000","last_unstale":"2024-09-10T08:46:19.193715+0000","last_undegraded":"2024-09-10T08:46:19.193715+0000","last_fullsized":"2024-09-10T08:46:19.193715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:17.730839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873847+0000","last_change":"2024-09-10T08:44:10.678254+0000","last_active":"2024-09-10T08:46:18.873847+0000","last_peered":"2024-09-10T08:46:18.873847+0000","last_clean":"2024-09-10T08:46:18.873847+0000","last_became_active":"2024-09-10T08:44:10.677891+0000","last_became_peered":"2024-09-10T08:44:10.677891+0000","last_unstale":"2024-09-10T08:46:18.873847+0000","last_undegraded":"2024-09-10T08:46:18.873847+0000","last_fullsized":"2024-09-10T08:46:18.873847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:32:43.399049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804719+0000","last_change":"2024-09-10T08:44:08.656315+0000","last_active":"2024-09-10T08:46:18.804719+0000","last_peered":"2024-09-10T08:46:18.804719+0000","last_clean":"2024-09-10T08:46:18.804719+0000","last_became_active":"2024-09-10T08:44:08.655978+0000","last_became_peered":"2024-09-10T08:44:08.655978+0000","last_unstale":"2024-09-10T08:46:18.804719+0000","last_undegraded":"2024-09-10T08:46:18.804719+0000","last_fullsized":"2024-09-10T08:46:18.804719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:15.025354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193456+0000","last_change":"2024-09-10T08:44:17.726789+0000","last_active":"2024-09-10T08:46:19.193456+0000","last_peered":"2024-09-10T08:46:19.193456+0000","last_clean":"2024-09-10T08:46:19.193456+0000","last_became_active":"2024-09-10T08:44:17.726450+0000","last_became_peered":"2024-09-10T08:44:17.726450+0000","last_unstale":"2024-09-10T08:46:19.193456+0000","last_undegraded":"2024-09-10T08:46:19.193456+0000","last_fullsized":"2024-09-10T08:46:19.193456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:03.899822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806240+0000","last_change":"2024-09-10T08:44:10.677359+0000","last_active":"2024-09-10T08:46:18.806240+0000","last_peered":"2024-09-10T08:46:18.806240+0000","last_clean":"2024-09-10T08:46:18.806240+0000","last_became_active":"2024-09-10T08:44:10.676782+0000","last_became_peered":"2024-09-10T08:44:10.676782+0000","last_unstale":"2024-09-10T08:46:18.806240+0000","last_undegraded":"2024-09-10T08:46:18.806240+0000","last_fullsized":"2024-09-10T08:46:18.806240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:31.721714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808306+0000","last_change":"2024-09-10T08:44:08.656993+0000","last_active":"2024-09-10T08:46:18.808306+0000","last_peered":"2024-09-10T08:46:18.808306+0000","last_clean":"2024-09-10T08:46:18.808306+0000","last_became_active":"2024-09-10T08:44:08.656571+0000","last_became_peered":"2024-09-10T08:44:08.656571+0000","last_unstale":"2024-09-10T08:46:18.808306+0000","last_undegraded":"2024-09-10T08:46:18.808306+0000","last_fullsized":"2024-09-10T08:46:18.808306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:32.202740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804870+0000","last_change":"2024-09-10T08:44:17.725674+0000","last_active":"2024-09-10T08:46:18.804870+0000","last_peered":"2024-09-10T08:46:18.804870+0000","last_clean":"2024-09-10T08:46:18.804870+0000","last_became_active":"2024-09-10T08:44:17.725263+0000","last_became_peered":"2024-09-10T08:44:17.725263+0000","last_unstale":"2024-09-10T08:46:18.804870+0000","last_undegraded":"2024-09-10T08:46:18.804870+0000","last_fullsized":"2024-09-10T08:46:18.804870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:18.183157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193937+0000","last_change":"2024-09-10T08:44:10.676871+0000","last_active":"2024-09-10T08:46:19.193937+0000","last_peered":"2024-09-10T08:46:19.193937+0000","last_clean":"2024-09-10T08:46:19.193937+0000","last_became_active":"2024-09-10T08:44:10.676540+0000","last_became_peered":"2024-09-10T08:44:10.676540+0000","last_unstale":"2024-09-10T08:46:19.193937+0000","last_undegraded":"2024-09-10T08:46:19.193937+0000","last_fullsized":"2024-09-10T08:46:19.193937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:50.964100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'41","reported_seq":305,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:39.372295+0000","last_change":"2024-09-10T08:44:08.660113+0000","last_active":"2024-09-10T08:46:39.372295+0000","last_peered":"2024-09-10T08:46:39.372295+0000","last_clean":"2024-09-10T08:46:39.372295+0000","last_became_active":"2024-09-10T08:44:08.659644+0000","last_became_peered":"2024-09-10T08:44:08.659644+0000","last_unstale":"2024-09-10T08:46:39.372295+0000","last_undegraded":"2024-09-10T08:46:39.372295+0000","last_fullsized":"2024-09-10T08:46:39.372295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:01.605344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68542,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":225,"num_read_kb":180,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803296+0000","last_change":"2024-09-10T08:44:17.723653+0000","last_active":"2024-09-10T08:46:18.803296+0000","last_peered":"2024-09-10T08:46:18.803296+0000","last_clean":"2024-09-10T08:46:18.803296+0000","last_became_active":"2024-09-10T08:44:17.723284+0000","last_became_peered":"2024-09-10T08:44:17.723284+0000","last_unstale":"2024-09-10T08:46:18.803296+0000","last_undegraded":"2024-09-10T08:46:18.803296+0000","last_fullsized":"2024-09-10T08:46:18.803296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:17.162526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193970+0000","last_change":"2024-09-10T08:44:10.681837+0000","last_active":"2024-09-10T08:46:19.193970+0000","last_peered":"2024-09-10T08:46:19.193970+0000","last_clean":"2024-09-10T08:46:19.193970+0000","last_became_active":"2024-09-10T08:44:10.681518+0000","last_became_peered":"2024-09-10T08:44:10.681518+0000","last_unstale":"2024-09-10T08:46:19.193970+0000","last_undegraded":"2024-09-10T08:46:19.193970+0000","last_fullsized":"2024-09-10T08:46:19.193970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:10.004215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804988+0000","last_change":"2024-09-10T08:44:09.050394+0000","last_active":"2024-09-10T08:46:18.804988+0000","last_peered":"2024-09-10T08:46:18.804988+0000","last_clean":"2024-09-10T08:46:18.804988+0000","last_became_active":"2024-09-10T08:44:09.049969+0000","last_became_peered":"2024-09-10T08:44:09.049969+0000","last_unstale":"2024-09-10T08:46:18.804988+0000","last_undegraded":"2024-09-10T08:46:18.804988+0000","last_fullsized":"2024-09-10T08:46:18.804988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:43.519246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194174+0000","last_change":"2024-09-10T08:44:17.720318+0000","last_active":"2024-09-10T08:46:19.194174+0000","last_peered":"2024-09-10T08:46:19.194174+0000","last_clean":"2024-09-10T08:46:19.194174+0000","last_became_active":"2024-09-10T08:44:17.719598+0000","last_became_peered":"2024-09-10T08:44:17.719598+0000","last_unstale":"2024-09-10T08:46:19.194174+0000","last_undegraded":"2024-09-10T08:46:19.194174+0000","last_fullsized":"2024-09-10T08:46:19.194174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:15:49.520324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193946+0000","last_change":"2024-09-10T08:44:10.677340+0000","last_active":"2024-09-10T08:46:19.193946+0000","last_peered":"2024-09-10T08:46:19.193946+0000","last_clean":"2024-09-10T08:46:19.193946+0000","last_became_active":"2024-09-10T08:44:10.676628+0000","last_became_peered":"2024-09-10T08:44:10.676628+0000","last_unstale":"2024-09-10T08:46:19.193946+0000","last_undegraded":"2024-09-10T08:46:19.193946+0000","last_fullsized":"2024-09-10T08:46:19.193946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:59:17.924217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874027+0000","last_change":"2024-09-10T08:44:08.660254+0000","last_active":"2024-09-10T08:46:18.874027+0000","last_peered":"2024-09-10T08:46:18.874027+0000","last_clean":"2024-09-10T08:46:18.874027+0000","last_became_active":"2024-09-10T08:44:08.659667+0000","last_became_peered":"2024-09-10T08:44:08.659667+0000","last_unstale":"2024-09-10T08:46:18.874027+0000","last_undegraded":"2024-09-10T08:46:18.874027+0000","last_fullsized":"2024-09-10T08:46:18.874027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:49:49.146867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803135+0000","last_change":"2024-09-10T08:44:17.722373+0000","last_active":"2024-09-10T08:46:18.803135+0000","last_peered":"2024-09-10T08:46:18.803135+0000","last_clean":"2024-09-10T08:46:18.803135+0000","last_became_active":"2024-09-10T08:44:17.722035+0000","last_became_peered":"2024-09-10T08:44:17.722035+0000","last_unstale":"2024-09-10T08:46:18.803135+0000","last_undegraded":"2024-09-10T08:46:18.803135+0000","last_fullsized":"2024-09-10T08:46:18.803135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:30:36.187364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194365+0000","last_change":"2024-09-10T08:44:10.672585+0000","last_active":"2024-09-10T08:46:19.194365+0000","last_peered":"2024-09-10T08:46:19.194365+0000","last_clean":"2024-09-10T08:46:19.194365+0000","last_became_active":"2024-09-10T08:44:10.671987+0000","last_became_peered":"2024-09-10T08:44:10.671987+0000","last_unstale":"2024-09-10T08:46:19.194365+0000","last_undegraded":"2024-09-10T08:46:19.194365+0000","last_fullsized":"2024-09-10T08:46:19.194365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:48.660750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'7","reported_seq":258,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:39.371955+0000","last_change":"2024-09-10T08:44:08.658069+0000","last_active":"2024-09-10T08:46:39.371955+0000","last_peered":"2024-09-10T08:46:39.371955+0000","last_clean":"2024-09-10T08:46:39.371955+0000","last_became_active":"2024-09-10T08:44:08.657702+0000","last_became_peered":"2024-09-10T08:44:08.657702+0000","last_unstale":"2024-09-10T08:46:39.371955+0000","last_undegraded":"2024-09-10T08:46:39.371955+0000","last_fullsized":"2024-09-10T08:46:39.371955+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:17.383267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":212,"num_read_kb":212,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192772+0000","last_change":"2024-09-10T08:44:17.717984+0000","last_active":"2024-09-10T08:46:19.192772+0000","last_peered":"2024-09-10T08:46:19.192772+0000","last_clean":"2024-09-10T08:46:19.192772+0000","last_became_active":"2024-09-10T08:44:17.717581+0000","last_became_peered":"2024-09-10T08:44:17.717581+0000","last_unstale":"2024-09-10T08:46:19.192772+0000","last_undegraded":"2024-09-10T08:46:19.192772+0000","last_fullsized":"2024-09-10T08:46:19.192772+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:25.330732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193566+0000","last_change":"2024-09-10T08:44:17.721244+0000","last_active":"2024-09-10T08:46:19.193566+0000","last_peered":"2024-09-10T08:46:19.193566+0000","last_clean":"2024-09-10T08:46:19.193566+0000","last_became_active":"2024-09-10T08:44:17.720884+0000","last_became_peered":"2024-09-10T08:44:17.720884+0000","last_unstale":"2024-09-10T08:46:19.193566+0000","last_undegraded":"2024-09-10T08:46:19.193566+0000","last_fullsized":"2024-09-10T08:46:19.193566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.769513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803279+0000","last_change":"2024-09-10T08:44:10.667558+0000","last_active":"2024-09-10T08:46:18.803279+0000","last_peered":"2024-09-10T08:46:18.803279+0000","last_clean":"2024-09-10T08:46:18.803279+0000","last_became_active":"2024-09-10T08:44:10.667214+0000","last_became_peered":"2024-09-10T08:44:10.667214+0000","last_unstale":"2024-09-10T08:46:18.803279+0000","last_undegraded":"2024-09-10T08:46:18.803279+0000","last_fullsized":"2024-09-10T08:46:18.803279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:53.630790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804697+0000","last_change":"2024-09-10T08:44:08.657930+0000","last_active":"2024-09-10T08:46:18.804697+0000","last_peered":"2024-09-10T08:46:18.804697+0000","last_clean":"2024-09-10T08:46:18.804697+0000","last_became_active":"2024-09-10T08:44:08.657611+0000","last_became_peered":"2024-09-10T08:44:08.657611+0000","last_unstale":"2024-09-10T08:46:18.804697+0000","last_undegraded":"2024-09-10T08:46:18.804697+0000","last_fullsized":"2024-09-10T08:46:18.804697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:36.923786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193910+0000","last_change":"2024-09-10T08:44:17.726106+0000","last_active":"2024-09-10T08:46:19.193910+0000","last_peered":"2024-09-10T08:46:19.193910+0000","last_clean":"2024-09-10T08:46:19.193910+0000","last_became_active":"2024-09-10T08:44:17.725683+0000","last_became_peered":"2024-09-10T08:44:17.725683+0000","last_unstale":"2024-09-10T08:46:19.193910+0000","last_undegraded":"2024-09-10T08:46:19.193910+0000","last_fullsized":"2024-09-10T08:46:19.193910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:23.392794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192727+0000","last_change":"2024-09-10T08:44:10.672204+0000","last_active":"2024-09-10T08:46:19.192727+0000","last_peered":"2024-09-10T08:46:19.192727+0000","last_clean":"2024-09-10T08:46:19.192727+0000","last_became_active":"2024-09-10T08:44:10.671898+0000","last_became_peered":"2024-09-10T08:44:10.671898+0000","last_unstale":"2024-09-10T08:46:19.192727+0000","last_undegraded":"2024-09-10T08:46:19.192727+0000","last_fullsized":"2024-09-10T08:46:19.192727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:03.737357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805979+0000","last_change":"2024-09-10T08:44:08.656620+0000","last_active":"2024-09-10T08:46:18.805979+0000","last_peered":"2024-09-10T08:46:18.805979+0000","last_clean":"2024-09-10T08:46:18.805979+0000","last_became_active":"2024-09-10T08:44:08.656306+0000","last_became_peered":"2024-09-10T08:44:08.656306+0000","last_unstale":"2024-09-10T08:46:18.805979+0000","last_undegraded":"2024-09-10T08:46:18.805979+0000","last_fullsized":"2024-09-10T08:46:18.805979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:57.081909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804743+0000","last_change":"2024-09-10T08:44:08.660178+0000","last_active":"2024-09-10T08:46:18.804743+0000","last_peered":"2024-09-10T08:46:18.804743+0000","last_clean":"2024-09-10T08:46:18.804743+0000","last_became_active":"2024-09-10T08:44:08.659839+0000","last_became_peered":"2024-09-10T08:44:08.659839+0000","last_unstale":"2024-09-10T08:46:18.804743+0000","last_undegraded":"2024-09-10T08:46:18.804743+0000","last_fullsized":"2024-09-10T08:46:18.804743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:17.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804848+0000","last_change":"2024-09-10T08:44:10.683683+0000","last_active":"2024-09-10T08:46:18.804848+0000","last_peered":"2024-09-10T08:46:18.804848+0000","last_clean":"2024-09-10T08:46:18.804848+0000","last_became_active":"2024-09-10T08:44:10.683337+0000","last_became_peered":"2024-09-10T08:44:10.683337+0000","last_unstale":"2024-09-10T08:46:18.804848+0000","last_undegraded":"2024-09-10T08:46:18.804848+0000","last_fullsized":"2024-09-10T08:46:18.804848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:54.863098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808044+0000","last_change":"2024-09-10T08:44:17.721602+0000","last_active":"2024-09-10T08:46:18.808044+0000","last_peered":"2024-09-10T08:46:18.808044+0000","last_clean":"2024-09-10T08:46:18.808044+0000","last_became_active":"2024-09-10T08:44:17.721194+0000","last_became_peered":"2024-09-10T08:44:17.721194+0000","last_unstale":"2024-09-10T08:46:18.808044+0000","last_undegraded":"2024-09-10T08:46:18.808044+0000","last_fullsized":"2024-09-10T08:46:18.808044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:39.298494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:39.372628+0000","last_change":"2024-09-10T08:44:08.661943+0000","last_active":"2024-09-10T08:46:39.372628+0000","last_peered":"2024-09-10T08:46:39.372628+0000","last_clean":"2024-09-10T08:46:39.372628+0000","last_became_active":"2024-09-10T08:44:08.661587+0000","last_became_peered":"2024-09-10T08:44:08.661587+0000","last_unstale":"2024-09-10T08:46:39.372628+0000","last_undegraded":"2024-09-10T08:46:39.372628+0000","last_fullsized":"2024-09-10T08:46:39.372628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:05.771641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193718+0000","last_change":"2024-09-10T08:44:10.673522+0000","last_active":"2024-09-10T08:46:19.193718+0000","last_peered":"2024-09-10T08:46:19.193718+0000","last_clean":"2024-09-10T08:46:19.193718+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:46:19.193718+0000","last_undegraded":"2024-09-10T08:46:19.193718+0000","last_fullsized":"2024-09-10T08:46:19.193718+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:03.006144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808419+0000","last_change":"2024-09-10T08:44:17.721306+0000","last_active":"2024-09-10T08:46:18.808419+0000","last_peered":"2024-09-10T08:46:18.808419+0000","last_clean":"2024-09-10T08:46:18.808419+0000","last_became_active":"2024-09-10T08:44:17.720401+0000","last_became_peered":"2024-09-10T08:44:17.720401+0000","last_unstale":"2024-09-10T08:46:18.808419+0000","last_undegraded":"2024-09-10T08:46:18.808419+0000","last_fullsized":"2024-09-10T08:46:18.808419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:11.589371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193166+0000","last_change":"2024-09-10T08:44:10.678425+0000","last_active":"2024-09-10T08:46:19.193166+0000","last_peered":"2024-09-10T08:46:19.193166+0000","last_clean":"2024-09-10T08:46:19.193166+0000","last_became_active":"2024-09-10T08:44:10.677187+0000","last_became_peered":"2024-09-10T08:44:10.677187+0000","last_unstale":"2024-09-10T08:46:19.193166+0000","last_undegraded":"2024-09-10T08:46:19.193166+0000","last_fullsized":"2024-09-10T08:46:19.193166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:06.527530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808431+0000","last_change":"2024-09-10T08:44:10.674178+0000","last_active":"2024-09-10T08:46:18.808431+0000","last_peered":"2024-09-10T08:46:18.808431+0000","last_clean":"2024-09-10T08:46:18.808431+0000","last_became_active":"2024-09-10T08:44:10.673843+0000","last_became_peered":"2024-09-10T08:44:10.673843+0000","last_unstale":"2024-09-10T08:46:18.808431+0000","last_undegraded":"2024-09-10T08:46:18.808431+0000","last_fullsized":"2024-09-10T08:46:18.808431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:35.023663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.807988+0000","last_change":"2024-09-10T08:44:10.673271+0000","last_active":"2024-09-10T08:46:18.807988+0000","last_peered":"2024-09-10T08:46:18.807988+0000","last_clean":"2024-09-10T08:46:18.807988+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:46:18.807988+0000","last_undegraded":"2024-09-10T08:46:18.807988+0000","last_fullsized":"2024-09-10T08:46:18.807988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:21:26.606820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193502+0000","last_change":"2024-09-10T08:44:10.671348+0000","last_active":"2024-09-10T08:46:19.193502+0000","last_peered":"2024-09-10T08:46:19.193502+0000","last_clean":"2024-09-10T08:46:19.193502+0000","last_became_active":"2024-09-10T08:44:10.670623+0000","last_became_peered":"2024-09-10T08:44:10.670623+0000","last_unstale":"2024-09-10T08:46:19.193502+0000","last_undegraded":"2024-09-10T08:46:19.193502+0000","last_fullsized":"2024-09-10T08:46:19.193502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:06.145694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806400+0000","last_change":"2024-09-10T08:44:10.666011+0000","last_active":"2024-09-10T08:46:18.806400+0000","last_peered":"2024-09-10T08:46:18.806400+0000","last_clean":"2024-09-10T08:46:18.806400+0000","last_became_active":"2024-09-10T08:44:10.665664+0000","last_became_peered":"2024-09-10T08:44:10.665664+0000","last_unstale":"2024-09-10T08:46:18.806400+0000","last_undegraded":"2024-09-10T08:46:18.806400+0000","last_fullsized":"2024-09-10T08:46:18.806400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:31.542102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193792+0000","last_change":"2024-09-10T08:44:10.675960+0000","last_active":"2024-09-10T08:46:19.193792+0000","last_peered":"2024-09-10T08:46:19.193792+0000","last_clean":"2024-09-10T08:46:19.193792+0000","last_became_active":"2024-09-10T08:44:10.675587+0000","last_became_peered":"2024-09-10T08:44:10.675587+0000","last_unstale":"2024-09-10T08:46:19.193792+0000","last_undegraded":"2024-09-10T08:46:19.193792+0000","last_fullsized":"2024-09-10T08:46:19.193792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:54.634619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808734+0000","last_change":"2024-09-10T08:44:10.681094+0000","last_active":"2024-09-10T08:46:18.808734+0000","last_peered":"2024-09-10T08:46:18.808734+0000","last_clean":"2024-09-10T08:46:18.808734+0000","last_became_active":"2024-09-10T08:44:10.678585+0000","last_became_peered":"2024-09-10T08:44:10.678585+0000","last_unstale":"2024-09-10T08:46:18.808734+0000","last_undegraded":"2024-09-10T08:46:18.808734+0000","last_fullsized":"2024-09-10T08:46:18.808734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:14.951949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193627+0000","last_change":"2024-09-10T08:44:10.675473+0000","last_active":"2024-09-10T08:46:19.193627+0000","last_peered":"2024-09-10T08:46:19.193627+0000","last_clean":"2024-09-10T08:46:19.193627+0000","last_became_active":"2024-09-10T08:44:10.674155+0000","last_became_peered":"2024-09-10T08:44:10.674155+0000","last_unstale":"2024-09-10T08:46:19.193627+0000","last_undegraded":"2024-09-10T08:46:19.193627+0000","last_fullsized":"2024-09-10T08:46:19.193627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:55.940235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804316+0000","last_change":"2024-09-10T08:44:10.680162+0000","last_active":"2024-09-10T08:46:18.804316+0000","last_peered":"2024-09-10T08:46:18.804316+0000","last_clean":"2024-09-10T08:46:18.804316+0000","last_became_active":"2024-09-10T08:44:10.679800+0000","last_became_peered":"2024-09-10T08:44:10.679800+0000","last_unstale":"2024-09-10T08:46:18.804316+0000","last_undegraded":"2024-09-10T08:46:18.804316+0000","last_fullsized":"2024-09-10T08:46:18.804316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:12.319813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192932+0000","last_change":"2024-09-10T08:44:10.674724+0000","last_active":"2024-09-10T08:46:19.192932+0000","last_peered":"2024-09-10T08:46:19.192932+0000","last_clean":"2024-09-10T08:46:19.192932+0000","last_became_active":"2024-09-10T08:44:10.674389+0000","last_became_peered":"2024-09-10T08:44:10.674389+0000","last_unstale":"2024-09-10T08:46:19.192932+0000","last_undegraded":"2024-09-10T08:46:19.192932+0000","last_fullsized":"2024-09-10T08:46:19.192932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:52.123186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193888+0000","last_change":"2024-09-10T08:44:10.672805+0000","last_active":"2024-09-10T08:46:19.193888+0000","last_peered":"2024-09-10T08:46:19.193888+0000","last_clean":"2024-09-10T08:46:19.193888+0000","last_became_active":"2024-09-10T08:44:10.672508+0000","last_became_peered":"2024-09-10T08:44:10.672508+0000","last_unstale":"2024-09-10T08:46:19.193888+0000","last_undegraded":"2024-09-10T08:46:19.193888+0000","last_fullsized":"2024-09-10T08:46:19.193888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:48.558633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806228+0000","last_change":"2024-09-10T08:44:10.677504+0000","last_active":"2024-09-10T08:46:18.806228+0000","last_peered":"2024-09-10T08:46:18.806228+0000","last_clean":"2024-09-10T08:46:18.806228+0000","last_became_active":"2024-09-10T08:44:10.676822+0000","last_became_peered":"2024-09-10T08:44:10.676822+0000","last_unstale":"2024-09-10T08:46:18.806228+0000","last_undegraded":"2024-09-10T08:46:18.806228+0000","last_fullsized":"2024-09-10T08:46:18.806228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:05:48.293333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193457+0000","last_change":"2024-09-10T08:45:18.268208+0000","last_active":"2024-09-10T08:46:19.193457+0000","last_peered":"2024-09-10T08:46:19.193457+0000","last_clean":"2024-09-10T08:46:19.193457+0000","last_became_active":"2024-09-10T08:45:18.267700+0000","last_became_peered":"2024-09-10T08:45:18.267700+0000","last_unstale":"2024-09-10T08:46:19.193457+0000","last_undegraded":"2024-09-10T08:46:19.193457+0000","last_fullsized":"2024-09-10T08:46:19.193457+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:54.487624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808678+0000","last_change":"2024-09-10T08:44:10.670615+0000","last_active":"2024-09-10T08:46:18.808678+0000","last_peered":"2024-09-10T08:46:18.808678+0000","last_clean":"2024-09-10T08:46:18.808678+0000","last_became_active":"2024-09-10T08:44:10.670032+0000","last_became_peered":"2024-09-10T08:44:10.670032+0000","last_unstale":"2024-09-10T08:46:18.808678+0000","last_undegraded":"2024-09-10T08:46:18.808678+0000","last_fullsized":"2024-09-10T08:46:18.808678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:31.078788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803799+0000","last_change":"2024-09-10T08:44:10.672367+0000","last_active":"2024-09-10T08:46:18.803799+0000","last_peered":"2024-09-10T08:46:18.803799+0000","last_clean":"2024-09-10T08:46:18.803799+0000","last_became_active":"2024-09-10T08:44:10.672040+0000","last_became_peered":"2024-09-10T08:44:10.672040+0000","last_unstale":"2024-09-10T08:46:18.803799+0000","last_undegraded":"2024-09-10T08:46:18.803799+0000","last_fullsized":"2024-09-10T08:46:18.803799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:10.634708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194226+0000","last_change":"2024-09-10T08:44:10.676375+0000","last_active":"2024-09-10T08:46:19.194226+0000","last_peered":"2024-09-10T08:46:19.194226+0000","last_clean":"2024-09-10T08:46:19.194226+0000","last_became_active":"2024-09-10T08:44:10.676022+0000","last_became_peered":"2024-09-10T08:44:10.676022+0000","last_unstale":"2024-09-10T08:46:19.194226+0000","last_undegraded":"2024-09-10T08:46:19.194226+0000","last_fullsized":"2024-09-10T08:46:19.194226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:02.681954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193408+0000","last_change":"2024-09-10T08:44:10.673378+0000","last_active":"2024-09-10T08:46:19.193408+0000","last_peered":"2024-09-10T08:46:19.193408+0000","last_clean":"2024-09-10T08:46:19.193408+0000","last_became_active":"2024-09-10T08:44:10.673089+0000","last_became_peered":"2024-09-10T08:44:10.673089+0000","last_unstale":"2024-09-10T08:46:19.193408+0000","last_undegraded":"2024-09-10T08:46:19.193408+0000","last_fullsized":"2024-09-10T08:46:19.193408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:36:19.258651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192898+0000","last_change":"2024-09-10T08:44:10.671557+0000","last_active":"2024-09-10T08:46:19.192898+0000","last_peered":"2024-09-10T08:46:19.192898+0000","last_clean":"2024-09-10T08:46:19.192898+0000","last_became_active":"2024-09-10T08:44:10.671248+0000","last_became_peered":"2024-09-10T08:44:10.671248+0000","last_unstale":"2024-09-10T08:46:19.192898+0000","last_undegraded":"2024-09-10T08:46:19.192898+0000","last_fullsized":"2024-09-10T08:46:19.192898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:54.261525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193104+0000","last_change":"2024-09-10T08:44:10.674672+0000","last_active":"2024-09-10T08:46:19.193104+0000","last_peered":"2024-09-10T08:46:19.193104+0000","last_clean":"2024-09-10T08:46:19.193104+0000","last_became_active":"2024-09-10T08:44:10.673415+0000","last_became_peered":"2024-09-10T08:44:10.673415+0000","last_unstale":"2024-09-10T08:46:19.193104+0000","last_undegraded":"2024-09-10T08:46:19.193104+0000","last_fullsized":"2024-09-10T08:46:19.193104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:59:14.179664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806192+0000","last_change":"2024-09-10T08:44:10.675293+0000","last_active":"2024-09-10T08:46:18.806192+0000","last_peered":"2024-09-10T08:46:18.806192+0000","last_clean":"2024-09-10T08:46:18.806192+0000","last_became_active":"2024-09-10T08:44:10.674924+0000","last_became_peered":"2024-09-10T08:44:10.674924+0000","last_unstale":"2024-09-10T08:46:18.806192+0000","last_undegraded":"2024-09-10T08:46:18.806192+0000","last_fullsized":"2024-09-10T08:46:18.806192+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:56.027320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194722+0000","last_change":"2024-09-10T08:44:10.674605+0000","last_active":"2024-09-10T08:46:19.194722+0000","last_peered":"2024-09-10T08:46:19.194722+0000","last_clean":"2024-09-10T08:46:19.194722+0000","last_became_active":"2024-09-10T08:44:10.674057+0000","last_became_peered":"2024-09-10T08:44:10.674057+0000","last_unstale":"2024-09-10T08:46:19.194722+0000","last_undegraded":"2024-09-10T08:46:19.194722+0000","last_fullsized":"2024-09-10T08:46:19.194722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:09.473003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874065+0000","last_change":"2024-09-10T08:44:10.677410+0000","last_active":"2024-09-10T08:46:18.874065+0000","last_peered":"2024-09-10T08:46:18.874065+0000","last_clean":"2024-09-10T08:46:18.874065+0000","last_became_active":"2024-09-10T08:44:10.677109+0000","last_became_peered":"2024-09-10T08:44:10.677109+0000","last_unstale":"2024-09-10T08:46:18.874065+0000","last_undegraded":"2024-09-10T08:46:18.874065+0000","last_fullsized":"2024-09-10T08:46:18.874065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:45.455425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805091+0000","last_change":"2024-09-10T08:44:10.669706+0000","last_active":"2024-09-10T08:46:18.805091+0000","last_peered":"2024-09-10T08:46:18.805091+0000","last_clean":"2024-09-10T08:46:18.805091+0000","last_became_active":"2024-09-10T08:44:10.669440+0000","last_became_peered":"2024-09-10T08:44:10.669440+0000","last_unstale":"2024-09-10T08:46:18.805091+0000","last_undegraded":"2024-09-10T08:46:18.805091+0000","last_fullsized":"2024-09-10T08:46:18.805091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:31:55.610651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194758+0000","last_change":"2024-09-10T08:44:10.676482+0000","last_active":"2024-09-10T08:46:19.194758+0000","last_peered":"2024-09-10T08:46:19.194758+0000","last_clean":"2024-09-10T08:46:19.194758+0000","last_became_active":"2024-09-10T08:44:10.675983+0000","last_became_peered":"2024-09-10T08:44:10.675983+0000","last_unstale":"2024-09-10T08:46:19.194758+0000","last_undegraded":"2024-09-10T08:46:19.194758+0000","last_fullsized":"2024-09-10T08:46:19.194758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:01.269283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192653+0000","last_change":"2024-09-10T08:44:10.675681+0000","last_active":"2024-09-10T08:46:19.192653+0000","last_peered":"2024-09-10T08:46:19.192653+0000","last_clean":"2024-09-10T08:46:19.192653+0000","last_became_active":"2024-09-10T08:44:10.675357+0000","last_became_peered":"2024-09-10T08:44:10.675357+0000","last_unstale":"2024-09-10T08:46:19.192653+0000","last_undegraded":"2024-09-10T08:46:19.192653+0000","last_fullsized":"2024-09-10T08:46:19.192653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:26.138333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873467+0000","last_change":"2024-09-10T08:44:10.680819+0000","last_active":"2024-09-10T08:46:18.873467+0000","last_peered":"2024-09-10T08:46:18.873467+0000","last_clean":"2024-09-10T08:46:18.873467+0000","last_became_active":"2024-09-10T08:44:10.680491+0000","last_became_peered":"2024-09-10T08:44:10.680491+0000","last_unstale":"2024-09-10T08:46:18.873467+0000","last_undegraded":"2024-09-10T08:46:18.873467+0000","last_fullsized":"2024-09-10T08:46:18.873467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:17.044776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804621+0000","last_change":"2024-09-10T08:44:10.667376+0000","last_active":"2024-09-10T08:46:18.804621+0000","last_peered":"2024-09-10T08:46:18.804621+0000","last_clean":"2024-09-10T08:46:18.804621+0000","last_became_active":"2024-09-10T08:44:10.666648+0000","last_became_peered":"2024-09-10T08:44:10.666648+0000","last_unstale":"2024-09-10T08:46:18.804621+0000","last_undegraded":"2024-09-10T08:46:18.804621+0000","last_fullsized":"2024-09-10T08:46:18.804621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:33:12.017550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193207+0000","last_change":"2024-09-10T08:44:10.668665+0000","last_active":"2024-09-10T08:46:19.193207+0000","last_peered":"2024-09-10T08:46:19.193207+0000","last_clean":"2024-09-10T08:46:19.193207+0000","last_became_active":"2024-09-10T08:44:10.666031+0000","last_became_peered":"2024-09-10T08:44:10.666031+0000","last_unstale":"2024-09-10T08:46:19.193207+0000","last_undegraded":"2024-09-10T08:46:19.193207+0000","last_fullsized":"2024-09-10T08:46:19.193207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:39:53.991463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194458+0000","last_change":"2024-09-10T08:44:10.676495+0000","last_active":"2024-09-10T08:46:19.194458+0000","last_peered":"2024-09-10T08:46:19.194458+0000","last_clean":"2024-09-10T08:46:19.194458+0000","last_became_active":"2024-09-10T08:44:10.676087+0000","last_became_peered":"2024-09-10T08:44:10.676087+0000","last_unstale":"2024-09-10T08:46:19.194458+0000","last_undegraded":"2024-09-10T08:46:19.194458+0000","last_fullsized":"2024-09-10T08:46:19.194458+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:41.828157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193138+0000","last_change":"2024-09-10T08:44:10.672835+0000","last_active":"2024-09-10T08:46:19.193138+0000","last_peered":"2024-09-10T08:46:19.193138+0000","last_clean":"2024-09-10T08:46:19.193138+0000","last_became_active":"2024-09-10T08:44:10.672537+0000","last_became_peered":"2024-09-10T08:44:10.672537+0000","last_unstale":"2024-09-10T08:46:19.193138+0000","last_undegraded":"2024-09-10T08:46:19.193138+0000","last_fullsized":"2024-09-10T08:46:19.193138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:06.121460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193568+0000","last_change":"2024-09-10T08:44:10.675968+0000","last_active":"2024-09-10T08:46:19.193568+0000","last_peered":"2024-09-10T08:46:19.193568+0000","last_clean":"2024-09-10T08:46:19.193568+0000","last_became_active":"2024-09-10T08:44:10.675628+0000","last_became_peered":"2024-09-10T08:44:10.675628+0000","last_unstale":"2024-09-10T08:46:19.193568+0000","last_undegraded":"2024-09-10T08:46:19.193568+0000","last_fullsized":"2024-09-10T08:46:19.193568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:40.261336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194091+0000","last_change":"2024-09-10T08:44:10.673151+0000","last_active":"2024-09-10T08:46:19.194091+0000","last_peered":"2024-09-10T08:46:19.194091+0000","last_clean":"2024-09-10T08:46:19.194091+0000","last_became_active":"2024-09-10T08:44:10.672833+0000","last_became_peered":"2024-09-10T08:44:10.672833+0000","last_unstale":"2024-09-10T08:46:19.194091+0000","last_undegraded":"2024-09-10T08:46:19.194091+0000","last_fullsized":"2024-09-10T08:46:19.194091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:38.556222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77478,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1099,"num_read_kb":635,"num_write":274,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":232434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514540,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":523370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743473,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28740,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716412,"statfs":{"total":95995035648,"available":95965605888,"internally_reserved":0,"allocated":1757184,"data_stored":1178194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906997,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":526367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135930,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":522649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364863,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1120240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528389,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":518270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757322,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36988,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708164,"statfs":{"total":95995035648,"available":95957159936,"internally_reserved":0,"allocated":1683456,"data_stored":1114832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":522344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":517483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":584777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1126400,"data_stored":583528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":520260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:46:41.381 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:46:41.683 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:42.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.189+0000 7f065ce4c640 1 -- 172.21.15.172:0/2960948213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0658100e20 msgr2=0x7f0658101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:42.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.189+0000 7f065ce4c640 1 --2- 172.21.15.172:0/2960948213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0658100e20 0x7f0658101200 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0648009900 tx=0x7f064802a480 comp rx=0 tx=0).stop 2024-09-10T08:46:42.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.189+0000 7f065ce4c640 1 -- 172.21.15.172:0/2960948213 shutdown_connections 2024-09-10T08:46:42.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.189+0000 7f065ce4c640 1 --2- 172.21.15.172:0/2960948213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0658102ef0 0x7f065810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:42.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.189+0000 7f065ce4c640 1 --2- 172.21.15.172:0/2960948213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0658102570 0x7f06581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:42.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.189+0000 7f065ce4c640 1 --2- 172.21.15.172:0/2960948213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0658100e20 0x7f0658101200 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:42.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.189+0000 7f065ce4c640 1 -- 172.21.15.172:0/2960948213 >> 172.21.15.172:0/2960948213 conn(0x7f06580fc7d0 msgr2=0x7f06580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:42.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.189+0000 7f065ce4c640 1 -- 172.21.15.172:0/2960948213 shutdown_connections 2024-09-10T08:46:42.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.189+0000 7f065ce4c640 1 -- 172.21.15.172:0/2960948213 wait complete. 2024-09-10T08:46:42.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.190+0000 7f065ce4c640 1 Processor -- start 2024-09-10T08:46:42.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.190+0000 7f065ce4c640 1 -- start start 2024-09-10T08:46:42.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.190+0000 7f065ce4c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0658100e20 0x7f065819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:42.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.190+0000 7f065ce4c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0658102570 0x7f065819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:42.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.190+0000 7f065ce4c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0658102ef0 0x7f065819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:42.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.190+0000 7f065ce4c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0658110e30 con 0x7f0658100e20 2024-09-10T08:46:42.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.190+0000 7f065ce4c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0658110cb0 con 0x7f0658102570 2024-09-10T08:46:42.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.190+0000 7f065ce4c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0658110fb0 con 0x7f0658102ef0 2024-09-10T08:46:42.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.190+0000 7f065659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0658100e20 0x7f065819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:42.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.190+0000 7f065659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0658100e20 0x7f065819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33308/0 (socket says 172.21.15.172:33308) 2024-09-10T08:46:42.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.190+0000 7f065659c640 1 -- 172.21.15.172:0/4145974130 learned_addr learned my addr 172.21.15.172:0/4145974130 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:42.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.190+0000 7f0656d9d640 1 --2- 172.21.15.172:0/4145974130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0658102ef0 0x7f065819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:42.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.190+0000 7f0655d9b640 1 --2- 172.21.15.172:0/4145974130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0658102570 0x7f065819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:42.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.191+0000 7f065659c640 1 -- 172.21.15.172:0/4145974130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0658102ef0 msgr2=0x7f065819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:42.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.191+0000 7f065659c640 1 --2- 172.21.15.172:0/4145974130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0658102ef0 0x7f065819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:42.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.191+0000 7f065659c640 1 -- 172.21.15.172:0/4145974130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0658102570 msgr2=0x7f065819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:42.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.191+0000 7f065659c640 1 --2- 172.21.15.172:0/4145974130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0658102570 0x7f065819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:42.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.191+0000 7f065659c640 1 -- 172.21.15.172:0/4145974130 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065819f5b0 con 0x7f0658100e20 2024-09-10T08:46:42.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.191+0000 7f0655d9b640 1 --2- 172.21.15.172:0/4145974130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0658102570 0x7f065819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:42.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.191+0000 7f065659c640 1 --2- 172.21.15.172:0/4145974130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0658100e20 0x7f065819a600 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0648006010 tx=0x7f06480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:42.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.191+0000 7f06477fe640 1 -- 172.21.15.172:0/4145974130 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064803f030 con 0x7f0658100e20 2024-09-10T08:46:42.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.191+0000 7f065ce4c640 1 -- 172.21.15.172:0/4145974130 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065819f840 con 0x7f0658100e20 2024-09-10T08:46:42.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.191+0000 7f06477fe640 1 -- 172.21.15.172:0/4145974130 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0648003cb0 con 0x7f0658100e20 2024-09-10T08:46:42.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.191+0000 7f06477fe640 1 -- 172.21.15.172:0/4145974130 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648035070 con 0x7f0658100e20 2024-09-10T08:46:42.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.191+0000 7f065ce4c640 1 -- 172.21.15.172:0/4145974130 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06581a7180 con 0x7f0658100e20 2024-09-10T08:46:42.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.192+0000 7f065ce4c640 1 -- 172.21.15.172:0/4145974130 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06581021b0 con 0x7f0658100e20 2024-09-10T08:46:42.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.193+0000 7f06477fe640 1 -- 172.21.15.172:0/4145974130 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0648042030 con 0x7f0658100e20 2024-09-10T08:46:42.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.194+0000 7f06477fe640 1 --2- 172.21.15.172:0/4145974130 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f06400a2b80 0x7f06400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:42.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.194+0000 7f0655d9b640 1 --2- 172.21.15.172:0/4145974130 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f06400a2b80 0x7f06400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:42.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.194+0000 7f06477fe640 1 -- 172.21.15.172:0/4145974130 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f06480ef8a0 con 0x7f0658100e20 2024-09-10T08:46:42.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.194+0000 7f0655d9b640 1 --2- 172.21.15.172:0/4145974130 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f06400a2b80 0x7f06400a5040 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f064c005d00 tx=0x7f064c005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:42.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.202+0000 7f06477fe640 1 -- 172.21.15.172:0/4145974130 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f06480bc4a0 con 0x7f0658100e20 2024-09-10T08:46:42.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.410+0000 7f065ce4c640 1 -- 172.21.15.172:0/4145974130 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06581a7350 con 0x7f0658100e20 2024-09-10T08:46:42.411 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:42 smithi172 ceph-mon[30854]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:42.411 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:42 smithi172 ceph-mon[30854]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:42.411 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:42 smithi172 ceph-mon[30854]: pgmap v364: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-10T08:46:42.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.410+0000 7f06477fe640 1 -- 172.21.15.172:0/4145974130 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7f06480bbda0 con 0x7f0658100e20 2024-09-10T08:46:42.412 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:42.412 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":29,"btime":"2024-09-10T08:46:37:242916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:46:36.267985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:46:42.412 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 29 2024-09-10T08:46:42.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.412+0000 7f065ce4c640 1 -- 172.21.15.172:0/4145974130 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f06400a2b80 msgr2=0x7f06400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:42.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.412+0000 7f065ce4c640 1 --2- 172.21.15.172:0/4145974130 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f06400a2b80 0x7f06400a5040 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f064c005d00 tx=0x7f064c005c70 comp rx=0 tx=0).stop 2024-09-10T08:46:42.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.412+0000 7f065ce4c640 1 -- 172.21.15.172:0/4145974130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0658100e20 msgr2=0x7f065819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:42.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.413+0000 7f065ce4c640 1 --2- 172.21.15.172:0/4145974130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0658100e20 0x7f065819a600 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0648006010 tx=0x7f06480042f0 comp rx=0 tx=0).stop 2024-09-10T08:46:42.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.413+0000 7f065ce4c640 1 -- 172.21.15.172:0/4145974130 shutdown_connections 2024-09-10T08:46:42.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.413+0000 7f065ce4c640 1 --2- 172.21.15.172:0/4145974130 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f06400a2b80 0x7f06400a5040 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:42.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.413+0000 7f065ce4c640 1 --2- 172.21.15.172:0/4145974130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0658102ef0 0x7f065819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:42.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.413+0000 7f065ce4c640 1 --2- 172.21.15.172:0/4145974130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0658102570 0x7f065819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:42.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.413+0000 7f065ce4c640 1 --2- 172.21.15.172:0/4145974130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0658100e20 0x7f065819a600 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:42.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.413+0000 7f065ce4c640 1 -- 172.21.15.172:0/4145974130 >> 172.21.15.172:0/4145974130 conn(0x7f06580fc7d0 msgr2=0x7f065806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:42.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.413+0000 7f065ce4c640 1 -- 172.21.15.172:0/4145974130 shutdown_connections 2024-09-10T08:46:42.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:42.413+0000 7f065ce4c640 1 -- 172.21.15.172:0/4145974130 wait complete. 2024-09-10T08:46:42.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:42 smithi191 ceph-mon[35314]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:42.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:42 smithi191 ceph-mon[35314]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:42.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:42 smithi191 ceph-mon[35314]: pgmap v364: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-10T08:46:42.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:42 smithi174 ceph-mon[34068]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:42 smithi174 ceph-mon[34068]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:42 smithi174 ceph-mon[34068]: pgmap v364: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-10T08:46:43.128 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:46:43.382 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:43.502 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:43 smithi172 ceph-mon[30854]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:43.503 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:43 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/4145974130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:43.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:43 smithi191 ceph-mon[35314]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:43.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:43 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/4145974130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:43.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:43 smithi174 ceph-mon[34068]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:43.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:43 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/4145974130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:43.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.866+0000 7f843d1d1640 1 -- 172.21.15.172:0/153191746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8438101410 msgr2=0x7f843810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:43.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.866+0000 7f843d1d1640 1 --2- 172.21.15.172:0/153191746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8438101410 0x7f843810d940 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8430009cd0 tx=0x7f843002a8e0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.867+0000 7f843d1d1640 1 -- 172.21.15.172:0/153191746 shutdown_connections 2024-09-10T08:46:43.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.867+0000 7f843d1d1640 1 --2- 172.21.15.172:0/153191746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8438101410 0x7f843810d940 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.867+0000 7f843d1d1640 1 --2- 172.21.15.172:0/153191746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8438100a70 0x7f8438100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.867+0000 7f843d1d1640 1 --2- 172.21.15.172:0/153191746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8438106a70 0x7f8438106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.867+0000 7f843d1d1640 1 -- 172.21.15.172:0/153191746 >> 172.21.15.172:0/153191746 conn(0x7f84380fc790 msgr2=0x7f84380febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:43.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.867+0000 7f843d1d1640 1 -- 172.21.15.172:0/153191746 shutdown_connections 2024-09-10T08:46:43.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.867+0000 7f843d1d1640 1 -- 172.21.15.172:0/153191746 wait complete. 2024-09-10T08:46:43.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.867+0000 7f843d1d1640 1 Processor -- start 2024-09-10T08:46:43.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f843d1d1640 1 -- start start 2024-09-10T08:46:43.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f843d1d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8438100a70 0x7f843819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f843d1d1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8438101410 0x7f843819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f843d1d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8438106a70 0x7f843819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f843d1d1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8438110d20 con 0x7f8438100a70 2024-09-10T08:46:43.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f843d1d1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8438110ba0 con 0x7f8438106a70 2024-09-10T08:46:43.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f843d1d1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8438110ea0 con 0x7f8438101410 2024-09-10T08:46:43.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f8436d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8438100a70 0x7f843819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:43.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f8436d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8438100a70 0x7f843819a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33312/0 (socket says 172.21.15.172:33312) 2024-09-10T08:46:43.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f8436d9d640 1 -- 172.21.15.172:0/1655882784 learned_addr learned my addr 172.21.15.172:0/1655882784 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:43.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f843759e640 1 --2- 172.21.15.172:0/1655882784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8438106a70 0x7f843819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:43.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f843659c640 1 --2- 172.21.15.172:0/1655882784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8438101410 0x7f843819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:43.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f8436d9d640 1 -- 172.21.15.172:0/1655882784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8438101410 msgr2=0x7f843819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:43.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f8436d9d640 1 --2- 172.21.15.172:0/1655882784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8438101410 0x7f843819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f8436d9d640 1 -- 172.21.15.172:0/1655882784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8438106a70 msgr2=0x7f843819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:43.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f8436d9d640 1 --2- 172.21.15.172:0/1655882784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8438106a70 0x7f843819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:43.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f8436d9d640 1 -- 172.21.15.172:0/1655882784 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843819f4d0 con 0x7f8438100a70 2024-09-10T08:46:43.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.868+0000 7f8436d9d640 1 --2- 172.21.15.172:0/1655882784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8438100a70 0x7f843819a520 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f842c00d770 tx=0x7f842c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:43.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.869+0000 7f841ffff640 1 -- 172.21.15.172:0/1655882784 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c017070 con 0x7f8438100a70 2024-09-10T08:46:43.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.869+0000 7f843d1d1640 1 -- 172.21.15.172:0/1655882784 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843819f7c0 con 0x7f8438100a70 2024-09-10T08:46:43.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.869+0000 7f841ffff640 1 -- 172.21.15.172:0/1655882784 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f842c0054d0 con 0x7f8438100a70 2024-09-10T08:46:43.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.869+0000 7f841ffff640 1 -- 172.21.15.172:0/1655882784 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c004cf0 con 0x7f8438100a70 2024-09-10T08:46:43.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.869+0000 7f843d1d1640 1 -- 172.21.15.172:0/1655882784 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84381a70a0 con 0x7f8438100a70 2024-09-10T08:46:43.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.870+0000 7f843d1d1640 1 -- 172.21.15.172:0/1655882784 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8438106e50 con 0x7f8438100a70 2024-09-10T08:46:43.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.871+0000 7f841ffff640 1 -- 172.21.15.172:0/1655882784 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f842c004e90 con 0x7f8438100a70 2024-09-10T08:46:43.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.871+0000 7f841ffff640 1 --2- 172.21.15.172:0/1655882784 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84200a2c00 0x7f84200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:43.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.871+0000 7f843659c640 1 --2- 172.21.15.172:0/1655882784 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84200a2c00 0x7f84200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:43.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.871+0000 7f841ffff640 1 -- 172.21.15.172:0/1655882784 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f842c0cec60 con 0x7f8438100a70 2024-09-10T08:46:43.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.872+0000 7f843659c640 1 --2- 172.21.15.172:0/1655882784 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84200a2c00 0x7f84200a50c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f843806ade0 tx=0x7f842800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:43.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:43.876+0000 7f841ffff640 1 -- 172.21.15.172:0/1655882784 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f842c09b210 con 0x7f8438100a70 2024-09-10T08:46:44.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:44.042+0000 7f843d1d1640 1 -- 172.21.15.172:0/1655882784 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84380fe280 con 0x7f8438100a70 2024-09-10T08:46:44.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:44.044+0000 7f841ffff640 1 -- 172.21.15.172:0/1655882784 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24778 (secure 0 0 0) 0x7f842c09f510 con 0x7f8438100a70 2024-09-10T08:46:44.045 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:44.045 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":92,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","created":"2024-09-10T08:36:33.083529+0000","modified":"2024-09-10T08:46:18.799844+0000","last_up_change":"2024-09-10T08:43:19.615640+0000","last_in_change":"2024-09-10T08:42:56.656115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:39:31.328306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:44:07.466637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:44:09.520569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:44:15.926057+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6803","nonce":2678426952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6805","nonce":2678426952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6809","nonce":2678426952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6807","nonce":2678426952}]},"public_addr":"172.21.15.172:6803/2678426952","cluster_addr":"172.21.15.172:6805/2678426952","heartbeat_back_addr":"172.21.15.172:6809/2678426952","heartbeat_front_addr":"172.21.15.172:6807/2678426952","state":["exists","up"]},{"osd":1,"uuid":"55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6801","nonce":3279661439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6803","nonce":3279661439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6807","nonce":3279661439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6805","nonce":3279661439}]},"public_addr":"172.21.15.174:6801/3279661439","cluster_addr":"172.21.15.174:6803/3279661439","heartbeat_back_addr":"172.21.15.174:6807/3279661439","heartbeat_front_addr":"172.21.15.174:6805/3279661439","state":["exists","up"]},{"osd":2,"uuid":"f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6801","nonce":1963796290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6803","nonce":1963796290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6807","nonce":1963796290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6805","nonce":1963796290}]},"public_addr":"172.21.15.191:6801/1963796290","cluster_addr":"172.21.15.191:6803/1963796290","heartbeat_back_addr":"172.21.15.191:6807/1963796290","heartbeat_front_addr":"172.21.15.191:6805/1963796290","state":["exists","up"]},{"osd":3,"uuid":"15ecca3f-bc6b-4a06-b283-b515384beb19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6811","nonce":869546480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6813","nonce":869546480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6817","nonce":869546480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6815","nonce":869546480}]},"public_addr":"172.21.15.172:6811/869546480","cluster_addr":"172.21.15.172:6813/869546480","heartbeat_back_addr":"172.21.15.172:6817/869546480","heartbeat_front_addr":"172.21.15.172:6815/869546480","state":["exists","up"]},{"osd":4,"uuid":"8c2c1d40-72ad-4a15-ab72-54871c7e2427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6809","nonce":961007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6811","nonce":961007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6815","nonce":961007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6813","nonce":961007949}]},"public_addr":"172.21.15.174:6809/961007949","cluster_addr":"172.21.15.174:6811/961007949","heartbeat_back_addr":"172.21.15.174:6815/961007949","heartbeat_front_addr":"172.21.15.174:6813/961007949","state":["exists","up"]},{"osd":5,"uuid":"4de45a1d-ef0e-4bfe-ba15-5eeeb8806155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6809","nonce":4216938371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6811","nonce":4216938371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6815","nonce":4216938371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6813","nonce":4216938371}]},"public_addr":"172.21.15.191:6809/4216938371","cluster_addr":"172.21.15.191:6811/4216938371","heartbeat_back_addr":"172.21.15.191:6815/4216938371","heartbeat_front_addr":"172.21.15.191:6813/4216938371","state":["exists","up"]},{"osd":6,"uuid":"60a9938a-f696-4db2-932c-402997543b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6819","nonce":3210762009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6821","nonce":3210762009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6825","nonce":3210762009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6823","nonce":3210762009}]},"public_addr":"172.21.15.172:6819/3210762009","cluster_addr":"172.21.15.172:6821/3210762009","heartbeat_back_addr":"172.21.15.172:6825/3210762009","heartbeat_front_addr":"172.21.15.172:6823/3210762009","state":["exists","up"]},{"osd":7,"uuid":"7f0202f3-971e-472f-9c4d-ae3c24d39f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6817","nonce":1461923581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6819","nonce":1461923581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6823","nonce":1461923581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6821","nonce":1461923581}]},"public_addr":"172.21.15.174:6817/1461923581","cluster_addr":"172.21.15.174:6819/1461923581","heartbeat_back_addr":"172.21.15.174:6823/1461923581","heartbeat_front_addr":"172.21.15.174:6821/1461923581","state":["exists","up"]},{"osd":8,"uuid":"b4e7327f-ca9f-4833-9cd3-b8529b3d9761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6817","nonce":3675210430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6819","nonce":3675210430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6823","nonce":3675210430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6821","nonce":3675210430}]},"public_addr":"172.21.15.191:6817/3675210430","cluster_addr":"172.21.15.191:6819/3675210430","heartbeat_back_addr":"172.21.15.191:6823/3675210430","heartbeat_front_addr":"172.21.15.191:6821/3675210430","state":["exists","up"]},{"osd":9,"uuid":"9f6153f2-f9db-4c00-a659-c932628016f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6827","nonce":1181268619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6829","nonce":1181268619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6833","nonce":1181268619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6831","nonce":1181268619}]},"public_addr":"172.21.15.172:6827/1181268619","cluster_addr":"172.21.15.172:6829/1181268619","heartbeat_back_addr":"172.21.15.172:6833/1181268619","heartbeat_front_addr":"172.21.15.172:6831/1181268619","state":["exists","up"]},{"osd":10,"uuid":"040a013a-4c3e-442f-bea6-12cbe517716d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6825","nonce":1238812525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6827","nonce":1238812525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6831","nonce":1238812525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6829","nonce":1238812525}]},"public_addr":"172.21.15.174:6825/1238812525","cluster_addr":"172.21.15.174:6827/1238812525","heartbeat_back_addr":"172.21.15.174:6831/1238812525","heartbeat_front_addr":"172.21.15.174:6829/1238812525","state":["exists","up"]},{"osd":11,"uuid":"ae6535bd-1510-4bf3-bf4b-1a2a256add43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6825","nonce":2162719997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6827","nonce":2162719997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6831","nonce":2162719997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6829","nonce":2162719997}]},"public_addr":"172.21.15.191:6825/2162719997","cluster_addr":"172.21.15.191:6827/2162719997","heartbeat_back_addr":"172.21.15.191:6831/2162719997","heartbeat_front_addr":"172.21.15.191:6829/2162719997","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:38:38.812596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:04.765652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:29.103966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:56.085849+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:21.641784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:47.750376+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:11.640697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:36.638840+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:02.167446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:28.655804+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:52.749676+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:43:17.783908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:0/977245917":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/3919952878":"2024-09-11T08:36:54.887256+0000","172.21.15.172:6800/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/1525385361":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/2043423408":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/839344299":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6800/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/3251817286":"2024-09-11T08:36:54.887256+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:46:44.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:44.046+0000 7f843d1d1640 1 -- 172.21.15.172:0/1655882784 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84200a2c00 msgr2=0x7f84200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:44.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:44.046+0000 7f843d1d1640 1 --2- 172.21.15.172:0/1655882784 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84200a2c00 0x7f84200a50c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f843806ade0 tx=0x7f842800b390 comp rx=0 tx=0).stop 2024-09-10T08:46:44.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:44.046+0000 7f843d1d1640 1 -- 172.21.15.172:0/1655882784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8438100a70 msgr2=0x7f843819a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:44.046+0000 7f843d1d1640 1 --2- 172.21.15.172:0/1655882784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8438100a70 0x7f843819a520 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f842c00d770 tx=0x7f842c00dc30 comp rx=0 tx=0).stop 2024-09-10T08:46:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:44.046+0000 7f843d1d1640 1 -- 172.21.15.172:0/1655882784 shutdown_connections 2024-09-10T08:46:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:44.046+0000 7f843d1d1640 1 --2- 172.21.15.172:0/1655882784 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84200a2c00 0x7f84200a50c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:44.046+0000 7f843d1d1640 1 --2- 172.21.15.172:0/1655882784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8438106a70 0x7f843819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:44.046+0000 7f843d1d1640 1 --2- 172.21.15.172:0/1655882784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8438101410 0x7f843819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:44.046+0000 7f843d1d1640 1 --2- 172.21.15.172:0/1655882784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8438100a70 0x7f843819a520 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:44.046+0000 7f843d1d1640 1 -- 172.21.15.172:0/1655882784 >> 172.21.15.172:0/1655882784 conn(0x7f84380fc790 msgr2=0x7f843810c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:44.046+0000 7f843d1d1640 1 -- 172.21.15.172:0/1655882784 shutdown_connections 2024-09-10T08:46:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:44.046+0000 7f843d1d1640 1 -- 172.21.15.172:0/1655882784 wait complete. 2024-09-10T08:46:44.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:44 smithi172 ceph-mon[30854]: pgmap v365: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-10T08:46:44.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:44 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1655882784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:46:44.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:44 smithi191 ceph-mon[35314]: pgmap v365: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-10T08:46:44.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:44 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1655882784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:46:44.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:44 smithi174 ceph-mon[34068]: pgmap v365: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-10T08:46:44.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:44 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1655882784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:46:44.773 DEBUG:tasks.fwd_scrub:Yielding 2024-09-10T08:46:44.773 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-10T08:46:44.784 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi172.front.sepia.ceph.com 2024-09-10T08:46:44.784 DEBUG:teuthology.orchestra.run.smithi172:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-10T08:46:44.785 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-10T08:46:44.785 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:46:44.785 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:46:44.788 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-10T08:46:44.808 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T08:46:44.809 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:46:45.047 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:45.069 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:45.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.719+0000 7f3cd564a640 1 -- 172.21.15.172:0/2629355355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd0106f10 msgr2=0x7f3cd010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:45.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.719+0000 7f3cd564a640 1 --2- 172.21.15.172:0/2629355355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd0106f10 0x7f3cd010ea90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009900 tx=0x7f3cc802a410 comp rx=0 tx=0).stop 2024-09-10T08:46:45.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.719+0000 7f3cd564a640 1 -- 172.21.15.172:0/2629355355 shutdown_connections 2024-09-10T08:46:45.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.719+0000 7f3cd564a640 1 --2- 172.21.15.172:0/2629355355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd0106f10 0x7f3cd010ea90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.719+0000 7f3cd564a640 1 --2- 172.21.15.172:0/2629355355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd0102790 0x7f3cd01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.719+0000 7f3cd564a640 1 --2- 172.21.15.172:0/2629355355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd0101de0 0x7f3cd01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.719+0000 7f3cd564a640 1 -- 172.21.15.172:0/2629355355 >> 172.21.15.172:0/2629355355 conn(0x7f3cd00fd8c0 msgr2=0x7f3cd00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:45.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.719+0000 7f3cd564a640 1 -- 172.21.15.172:0/2629355355 shutdown_connections 2024-09-10T08:46:45.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.719+0000 7f3cd564a640 1 -- 172.21.15.172:0/2629355355 wait complete. 2024-09-10T08:46:45.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.719+0000 7f3cd564a640 1 Processor -- start 2024-09-10T08:46:45.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.719+0000 7f3cd564a640 1 -- start start 2024-09-10T08:46:45.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.720+0000 7f3cd564a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd0101de0 0x7f3cd0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.720+0000 7f3cd564a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd0102790 0x7f3cd0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.720+0000 7f3cd564a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd0106f10 0x7f3cd019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.720+0000 7f3cd564a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd0111e20 con 0x7f3cd0106f10 2024-09-10T08:46:45.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.720+0000 7f3cd564a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd0111ca0 con 0x7f3cd0101de0 2024-09-10T08:46:45.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.720+0000 7f3cd564a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd0111fa0 con 0x7f3cd0102790 2024-09-10T08:46:45.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.720+0000 7f3ccf7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd0106f10 0x7f3cd019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.720+0000 7f3ccf7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd0106f10 0x7f3cd019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33326/0 (socket says 172.21.15.172:33326) 2024-09-10T08:46:45.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.720+0000 7f3ccf7fe640 1 -- 172.21.15.172:0/1816623287 learned_addr learned my addr 172.21.15.172:0/1816623287 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:45.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.720+0000 7f3cceffd640 1 --2- 172.21.15.172:0/1816623287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd0101de0 0x7f3cd0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.720+0000 7f3cce7fc640 1 --2- 172.21.15.172:0/1816623287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd0102790 0x7f3cd0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.720+0000 7f3ccf7fe640 1 -- 172.21.15.172:0/1816623287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd0102790 msgr2=0x7f3cd0198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:45.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.720+0000 7f3ccf7fe640 1 --2- 172.21.15.172:0/1816623287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd0102790 0x7f3cd0198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.720+0000 7f3ccf7fe640 1 -- 172.21.15.172:0/1816623287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd0101de0 msgr2=0x7f3cd0198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:45.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.720+0000 7f3ccf7fe640 1 --2- 172.21.15.172:0/1816623287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd0101de0 0x7f3cd0198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.720+0000 7f3ccf7fe640 1 -- 172.21.15.172:0/1816623287 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd019d2f0 con 0x7f3cd0106f10 2024-09-10T08:46:45.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.720+0000 7f3ccf7fe640 1 --2- 172.21.15.172:0/1816623287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd0106f10 0x7f3cd019cc10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009230 tx=0x7f3cc8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:45.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.721+0000 7f3cc7fff640 1 -- 172.21.15.172:0/1816623287 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc803f030 con 0x7f3cd0106f10 2024-09-10T08:46:45.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.721+0000 7f3cd564a640 1 -- 172.21.15.172:0/1816623287 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd019d580 con 0x7f3cd0106f10 2024-09-10T08:46:45.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.721+0000 7f3cc7fff640 1 -- 172.21.15.172:0/1816623287 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cc8003aa0 con 0x7f3cd0106f10 2024-09-10T08:46:45.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.721+0000 7f3cc7fff640 1 -- 172.21.15.172:0/1816623287 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8035070 con 0x7f3cd0106f10 2024-09-10T08:46:45.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.721+0000 7f3cd564a640 1 -- 172.21.15.172:0/1816623287 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd01a4ec0 con 0x7f3cd0106f10 2024-09-10T08:46:45.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.723+0000 7f3cc7fff640 1 -- 172.21.15.172:0/1816623287 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3cc8042030 con 0x7f3cd0106f10 2024-09-10T08:46:45.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.723+0000 7f3cd7a23740 1 Processor -- start 2024-09-10T08:46:45.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.723+0000 7f3cd7a23740 1 -- start start 2024-09-10T08:46:45.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cd7a23740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d504698b60 0x55d504698f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cd7a23740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504699480 0x55d5046a3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cd7a23740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5046a3d40 0x55d5046a6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cd7a23740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d5045fe280 con 0x55d504699480 2024-09-10T08:46:45.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cd7a23740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d5045a5b10 con 0x55d504698b60 2024-09-10T08:46:45.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cd7a23740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d5045fa240 con 0x55d5046a3d40 2024-09-10T08:46:45.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cceffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504699480 0x55d5046a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cceffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504699480 0x55d5046a3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33334/0 (socket says 172.21.15.172:33334) 2024-09-10T08:46:45.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cceffd640 1 -- 172.21.15.172:0/3405401518 learned_addr learned my addr 172.21.15.172:0/3405401518 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:45.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cce7fc640 1 --2- 172.21.15.172:0/3405401518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d504698b60 0x55d504698f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cce7fc640 1 --2- 172.21.15.172:0/3405401518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5046a3d40 0x55d5046a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cceffd640 1 -- 172.21.15.172:0/3405401518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5046a3d40 msgr2=0x55d5046a6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:45.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cceffd640 1 --2- 172.21.15.172:0/3405401518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5046a3d40 0x55d5046a6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cceffd640 1 -- 172.21.15.172:0/3405401518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d504698b60 msgr2=0x55d504698f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:45.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cceffd640 1 --2- 172.21.15.172:0/3405401518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d504698b60 0x55d504698f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cceffd640 1 -- 172.21.15.172:0/3405401518 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5045b21f0 con 0x55d504699480 2024-09-10T08:46:45.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cce7fc640 1 --2- 172.21.15.172:0/3405401518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d504698b60 0x55d504698f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:46:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cc7fff640 1 --2- 172.21.15.172:0/1816623287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cbc0a2b80 0x7f3cbc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.724+0000 7f3cceffd640 1 --2- 172.21.15.172:0/3405401518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504699480 0x55d5046a3800 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000af30 tx=0x7f3cc000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba2cbed8b6b00a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.725+0000 7f3cce7fc640 1 --2- 172.21.15.172:0/1816623287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cbc0a2b80 0x7f3cbc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.725+0000 7f3cc7fff640 1 -- 172.21.15.172:0/1816623287 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3cc80efab0 con 0x7f3cd0106f10 2024-09-10T08:46:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.725+0000 7f3cc57fa640 1 -- 172.21.15.172:0/3405401518 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0010900 con 0x55d504699480 2024-09-10T08:46:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.725+0000 7f3cc57fa640 1 -- 172.21.15.172:0/3405401518 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cc0018430 con 0x55d504699480 2024-09-10T08:46:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.725+0000 7f3cc57fa640 1 -- 172.21.15.172:0/3405401518 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0017640 con 0x55d504699480 2024-09-10T08:46:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.725+0000 7f3cd7a23740 1 -- 172.21.15.172:0/3405401518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504699480 msgr2=0x55d5046a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.725+0000 7f3cce7fc640 1 --2- 172.21.15.172:0/1816623287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cbc0a2b80 0x7f3cbc0a5040 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55d504696470 tx=0x7f3cb8015040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.725+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/3405401518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504699480 0x55d5046a3800 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000af30 tx=0x7f3cc000f850 comp rx=0 tx=0).stop 2024-09-10T08:46:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cd7a23740 1 -- 172.21.15.172:0/3405401518 shutdown_connections 2024-09-10T08:46:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/3405401518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5046a3d40 0x55d5046a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/3405401518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504699480 0x55d5046a3800 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/3405401518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d504698b60 0x55d504698f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cd7a23740 1 -- 172.21.15.172:0/3405401518 >> 172.21.15.172:0/3405401518 conn(0x55d5046950f0 msgr2=0x55d5046954d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cd7a23740 1 -- 172.21.15.172:0/3405401518 shutdown_connections 2024-09-10T08:46:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cd7a23740 1 -- 172.21.15.172:0/3405401518 wait complete. 2024-09-10T08:46:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cd7a23740 1 -- start start 2024-09-10T08:46:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cd7a23740 1 Processor -- start 2024-09-10T08:46:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cd7a23740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504698b60 0x55d5046a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cd7a23740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d504699480 0x55d5046a1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cd7a23740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5046a3d40 0x55d5046a1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cd7a23740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d5046a6910 con 0x55d504698b60 2024-09-10T08:46:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cd7a23740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d5045b8f10 con 0x55d5046a3d40 2024-09-10T08:46:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cd7a23740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d5046a6a90 con 0x55d504699480 2024-09-10T08:46:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cceffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504698b60 0x55d5046a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cd7a23740 10 client.0 _ll_register_callbacks cb 0x55d504581a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cceffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504698b60 0x55d5046a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33344/0 (socket says 172.21.15.172:33344) 2024-09-10T08:46:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.726+0000 7f3cceffd640 1 -- 172.21.15.172:0/2554268504 learned_addr learned my addr 172.21.15.172:0/2554268504 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.727+0000 7f3cce7fc640 1 --2- 172.21.15.172:0/2554268504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d504699480 0x55d5046a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.727+0000 7f3ccf7fe640 1 --2- 172.21.15.172:0/2554268504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5046a3d40 0x55d5046a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.727+0000 7f3cceffd640 1 -- 172.21.15.172:0/2554268504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d504699480 msgr2=0x55d5046a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.727+0000 7f3cceffd640 1 --2- 172.21.15.172:0/2554268504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d504699480 0x55d5046a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.727+0000 7f3cceffd640 1 -- 172.21.15.172:0/2554268504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5046a3d40 msgr2=0x55d5046a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.727+0000 7f3cceffd640 1 --2- 172.21.15.172:0/2554268504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5046a3d40 0x55d5046a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.727+0000 7f3cceffd640 1 -- 172.21.15.172:0/2554268504 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d50436f6a0 con 0x55d504698b60 2024-09-10T08:46:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.727+0000 7f3cceffd640 1 --2- 172.21.15.172:0/2554268504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504698b60 0x55d5046a0c00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0018e10 tx=0x7f3cc000ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d88819743d0cf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.727+0000 7f3cb67fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.727+0000 7f3cb67fc640 1 -- 172.21.15.172:0/2554268504 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0026070 con 0x55d504698b60 2024-09-10T08:46:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.727+0000 7f3cb67fc640 1 -- 172.21.15.172:0/2554268504 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3cc0010700 con 0x55d504698b60 2024-09-10T08:46:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.727+0000 7f3cd7a23740 1 -- 172.21.15.172:0/2554268504 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc0018430 con 0x55d504698b60 2024-09-10T08:46:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.727+0000 7f3cb67fc640 1 -- 172.21.15.172:0/2554268504 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3cc0032050 con 0x55d504698b60 2024-09-10T08:46:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.727+0000 7f3cd7a23740 10 client.15399 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.727+0000 7f3cd7a23740 1 -- 172.21.15.172:0/2554268504 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d5046b24f0 con 0x55d504698b60 2024-09-10T08:46:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.727+0000 7f3cb67fc640 1 -- 172.21.15.172:0/2554268504 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cc0018430 con 0x55d504698b60 2024-09-10T08:46:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.728+0000 7f3cb67fc640 1 -- 172.21.15.172:0/2554268504 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3cc0005360 con 0x55d504698b60 2024-09-10T08:46:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.728+0000 7f3cd7a23740 10 client.15399 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.728+0000 7f3cd7a23740 10 client.15399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:46:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.728+0000 7f3cd7a23740 20 client.15399 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.728+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/2554268504 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d5046b38a0 0x55d5046b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.728+0000 7f3cd7a23740 4 client.15399 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:45.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.728+0000 7f3cd7a23740 1 -- 172.21.15.172:0/2554268504 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d50460b7f0 con 0x55d5046b38a0 2024-09-10T08:46:45.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.728+0000 7f3cce7fc640 1 --2- 172.21.15.172:0/2554268504 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d5046b38a0 0x55d5046b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.728+0000 7f3cce7fc640 1 --2- 172.21.15.172:0/2554268504 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d5046b38a0 0x55d5046b5c90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0e224a69a233261 server_cookie=2cda24fc17fc5626 in_seq=0 out_seq=0 2024-09-10T08:46:45.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.728+0000 7f3cb67fc640 10 client.15399 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:45.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.729+0000 7f3cb67fc640 1 -- 172.21.15.172:0/2554268504 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d50460b7f0 con 0x55d5046b38a0 2024-09-10T08:46:45.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.729+0000 7f3cb67fc640 10 client.15399 handle_command_reply: tid=0 2024-09-10T08:46:45.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.729+0000 7f3cd7a23740 1 client.15399 shutdown 2024-09-10T08:46:45.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.729+0000 7f3cd7a23740 1 -- 172.21.15.172:0/2554268504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504698b60 msgr2=0x55d5046a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:45.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.729+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/2554268504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504698b60 0x55d5046a0c00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0018e10 tx=0x7f3cc000ac50 comp rx=0 tx=0).stop 2024-09-10T08:46:45.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.729+0000 7f3cd7a23740 1 -- 172.21.15.172:0/2554268504 shutdown_connections 2024-09-10T08:46:45.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.729+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/2554268504 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d5046b38a0 0x55d5046b5c90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.729+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/2554268504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5046a3d40 0x55d5046a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.729+0000 7f3cb67fc640 0 client.15399 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:45.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.729+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/2554268504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d504699480 0x55d5046a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.729+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/2554268504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504698b60 0x55d5046a0c00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.729+0000 7f3cd7a23740 1 -- 172.21.15.172:0/2554268504 >> 172.21.15.172:0/2554268504 conn(0x55d5046950f0 msgr2=0x55d504694270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:45.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.729+0000 7f3cd7a23740 1 -- 172.21.15.172:0/2554268504 shutdown_connections 2024-09-10T08:46:45.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.730+0000 7f3cd7a23740 1 -- 172.21.15.172:0/2554268504 wait complete. 2024-09-10T08:46:45.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.730+0000 7f3cd7a23740 20 client.15399 trim_cache size 0 max 16384 2024-09-10T08:46:45.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.746+0000 7f3cd7a23740 1 Processor -- start 2024-09-10T08:46:45.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.746+0000 7f3cd7a23740 1 -- start start 2024-09-10T08:46:45.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3cd7a23740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504698b60 0x55d50469b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3cd7a23740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d504699480 0x55d50469fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3cd7a23740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5046a3d40 0x55d5046bb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3cd7a23740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01aeb10 con 0x55d504698b60 2024-09-10T08:46:45.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3cd7a23740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01b0650 con 0x55d504699480 2024-09-10T08:46:45.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3cd7a23740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01af1e0 con 0x55d5046a3d40 2024-09-10T08:46:45.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3cceffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504698b60 0x55d50469b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3cceffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504698b60 0x55d50469b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33352/0 (socket says 172.21.15.172:33352) 2024-09-10T08:46:45.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3cceffd640 1 -- 172.21.15.172:0/1344082051 learned_addr learned my addr 172.21.15.172:0/1344082051 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:45.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3cce7fc640 1 --2- 172.21.15.172:0/1344082051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d504699480 0x55d50469fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3ccf7fe640 1 --2- 172.21.15.172:0/1344082051 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5046a3d40 0x55d5046bb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3cceffd640 1 -- 172.21.15.172:0/1344082051 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5046a3d40 msgr2=0x55d5046bb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:45.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3cceffd640 1 --2- 172.21.15.172:0/1344082051 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5046a3d40 0x55d5046bb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3cceffd640 1 -- 172.21.15.172:0/1344082051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d504699480 msgr2=0x55d50469fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:45.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3cceffd640 1 --2- 172.21.15.172:0/1344082051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d504699480 0x55d50469fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3cceffd640 1 -- 172.21.15.172:0/1344082051 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5046988e0 con 0x55d504698b60 2024-09-10T08:46:45.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3cce7fc640 1 --2- 172.21.15.172:0/1344082051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d504699480 0x55d50469fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:45.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3ccf7fe640 1 --2- 172.21.15.172:0/1344082051 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5046a3d40 0x55d5046bb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:45.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.747+0000 7f3cceffd640 1 --2- 172.21.15.172:0/1344082051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504698b60 0x55d50469b5c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000a9f0 tx=0x7f3cc000ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc13c9447dea045 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:45.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.748+0000 7f3cc4ff9640 1 -- 172.21.15.172:0/1344082051 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc003b070 con 0x55d504698b60 2024-09-10T08:46:45.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.748+0000 7f3cc4ff9640 1 -- 172.21.15.172:0/1344082051 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cc0018430 con 0x55d504698b60 2024-09-10T08:46:45.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.748+0000 7f3cc4ff9640 1 -- 172.21.15.172:0/1344082051 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0026070 con 0x55d504698b60 2024-09-10T08:46:45.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.748+0000 7f3cd7a23740 1 -- 172.21.15.172:0/1344082051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504698b60 msgr2=0x55d50469b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:45.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.748+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/1344082051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504698b60 0x55d50469b5c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000a9f0 tx=0x7f3cc000ad80 comp rx=0 tx=0).stop 2024-09-10T08:46:45.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.748+0000 7f3cd7a23740 1 -- 172.21.15.172:0/1344082051 shutdown_connections 2024-09-10T08:46:45.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.748+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/1344082051 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5046a3d40 0x55d5046bb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.748+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/1344082051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d504699480 0x55d50469fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.748+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/1344082051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d504698b60 0x55d50469b5c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.748+0000 7f3cd7a23740 1 -- 172.21.15.172:0/1344082051 >> 172.21.15.172:0/1344082051 conn(0x55d5046950f0 msgr2=0x55d50469f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:45.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.748+0000 7f3cd7a23740 1 -- 172.21.15.172:0/1344082051 shutdown_connections 2024-09-10T08:46:45.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.748+0000 7f3cd7a23740 1 -- 172.21.15.172:0/1344082051 wait complete. 2024-09-10T08:46:45.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.749+0000 7f3cd7a23740 1 -- start start 2024-09-10T08:46:45.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.749+0000 7f3cd7a23740 1 Processor -- start 2024-09-10T08:46:45.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.749+0000 7f3cd7a23740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d504698b60 0x55d5046ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.749+0000 7f3cd7a23740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d504699480 0x55d5046a7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.749+0000 7f3cd7a23740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5046a3d40 0x55d5046a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.749+0000 7f3cd7a23740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01aff80 con 0x55d5046a3d40 2024-09-10T08:46:45.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.749+0000 7f3cd7a23740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01af8b0 con 0x55d504698b60 2024-09-10T08:46:45.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.749+0000 7f3cd7a23740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01acaf0 con 0x55d504699480 2024-09-10T08:46:45.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.749+0000 7f3cd7a23740 10 client.0 _ll_register_callbacks cb 0x7f3cb800f4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:45.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.749+0000 7f3ccf7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5046a3d40 0x55d5046a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.749+0000 7f3ccf7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5046a3d40 0x55d5046a7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33364/0 (socket says 172.21.15.172:33364) 2024-09-10T08:46:45.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.749+0000 7f3ccf7fe640 1 -- 172.21.15.172:0/4060370897 learned_addr learned my addr 172.21.15.172:0/4060370897 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:45.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.749+0000 7f3cceffd640 1 --2- 172.21.15.172:0/4060370897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d504698b60 0x55d5046ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.749+0000 7f3cce7fc640 1 --2- 172.21.15.172:0/4060370897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d504699480 0x55d5046a7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3ccf7fe640 1 -- 172.21.15.172:0/4060370897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d504699480 msgr2=0x55d5046a7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:45.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3ccf7fe640 1 --2- 172.21.15.172:0/4060370897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d504699480 0x55d5046a7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3ccf7fe640 1 -- 172.21.15.172:0/4060370897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d504698b60 msgr2=0x55d5046ac2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:45.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3ccf7fe640 1 --2- 172.21.15.172:0/4060370897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d504698b60 0x55d5046ac2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3ccf7fe640 1 -- 172.21.15.172:0/4060370897 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5046a8220 con 0x55d5046a3d40 2024-09-10T08:46:45.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3cce7fc640 1 --2- 172.21.15.172:0/4060370897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d504699480 0x55d5046a7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:45.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3ccf7fe640 1 --2- 172.21.15.172:0/4060370897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5046a3d40 0x55d5046a7b80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8031440 tx=0x7f3cc80dc170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb49ab2555e276a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:45.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3cb67fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:45.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3cb67fc640 1 -- 172.21.15.172:0/4060370897 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8049050 con 0x55d5046a3d40 2024-09-10T08:46:45.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3cb67fc640 1 -- 172.21.15.172:0/4060370897 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3cc80ba2b0 con 0x55d5046a3d40 2024-09-10T08:46:45.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3cd7a23740 1 -- 172.21.15.172:0/4060370897 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc0018430 con 0x55d5046a3d40 2024-09-10T08:46:45.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3cb67fc640 1 -- 172.21.15.172:0/4060370897 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3cc80c0360 con 0x55d5046a3d40 2024-09-10T08:46:45.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3cd7a23740 10 client.15405 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:45.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3cd7a23740 1 -- 172.21.15.172:0/4060370897 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d50471e160 con 0x55d5046a3d40 2024-09-10T08:46:45.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3cb67fc640 1 -- 172.21.15.172:0/4060370897 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cc0018430 con 0x55d5046a3d40 2024-09-10T08:46:45.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3cb67fc640 1 -- 172.21.15.172:0/4060370897 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3cc80bc210 con 0x55d5046a3d40 2024-09-10T08:46:45.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3cd7a23740 10 client.15405 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:45.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3cd7a23740 10 client.15405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:46:45.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3cd7a23740 20 client.15405 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:45.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/4060370897 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d50471e880 0x55d504720c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3cd7a23740 4 client.15405 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:46:45.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.750+0000 7f3cd7a23740 1 -- 172.21.15.172:0/4060370897 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d504605d00 con 0x55d50471e880 2024-09-10T08:46:45.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.751+0000 7f3cceffd640 1 --2- 172.21.15.172:0/4060370897 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d50471e880 0x55d504720c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.751+0000 7f3cb67fc640 10 client.15405 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:45.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.751+0000 7f3cceffd640 1 --2- 172.21.15.172:0/4060370897 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d50471e880 0x55d504720c70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6d77887dcf9dc2d server_cookie=c8824e9ea3e9337a in_seq=0 out_seq=0 2024-09-10T08:46:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.752+0000 7f3cb67fc640 1 -- 172.21.15.172:0/4060370897 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d504605d00 con 0x55d50471e880 2024-09-10T08:46:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.752+0000 7f3cb67fc640 10 client.15405 handle_command_reply: tid=0 2024-09-10T08:46:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.752+0000 7f3cd7a23740 1 client.15405 shutdown 2024-09-10T08:46:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.752+0000 7f3cd7a23740 1 -- 172.21.15.172:0/4060370897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5046a3d40 msgr2=0x55d5046a7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.752+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/4060370897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5046a3d40 0x55d5046a7b80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8031440 tx=0x7f3cc80dc170 comp rx=0 tx=0).stop 2024-09-10T08:46:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.752+0000 7f3cd7a23740 1 -- 172.21.15.172:0/4060370897 shutdown_connections 2024-09-10T08:46:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.752+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/4060370897 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d50471e880 0x55d504720c70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.752+0000 7f3cb67fc640 0 client.15405 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.752+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/4060370897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5046a3d40 0x55d5046a7b80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.752+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/4060370897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d504699480 0x55d5046a7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.752+0000 7f3cd7a23740 1 --2- 172.21.15.172:0/4060370897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d504698b60 0x55d5046ac2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.752+0000 7f3cd7a23740 1 -- 172.21.15.172:0/4060370897 >> 172.21.15.172:0/4060370897 conn(0x55d5046950f0 msgr2=0x55d504694270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:45.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.753+0000 7f3cd7a23740 1 -- 172.21.15.172:0/4060370897 shutdown_connections 2024-09-10T08:46:45.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.753+0000 7f3cd7a23740 1 -- 172.21.15.172:0/4060370897 wait complete. 2024-09-10T08:46:45.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.753+0000 7f3cd7a23740 20 client.15405 trim_cache size 0 max 16384 2024-09-10T08:46:45.759 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:46:45.759 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T08:46:45.759 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "e61817eb-66d0-4ac0-808a-76f60fbc6235", 2024-09-10T08:46:45.759 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T08:46:45.759 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:46:45.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.753+0000 7f3cd564a640 1 -- 172.21.15.172:0/1816623287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cbc0a2b80 msgr2=0x7f3cbc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:45.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.753+0000 7f3cd564a640 1 --2- 172.21.15.172:0/1816623287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cbc0a2b80 0x7f3cbc0a5040 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55d504696470 tx=0x7f3cb8015040 comp rx=0 tx=0).stop 2024-09-10T08:46:45.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.753+0000 7f3cd564a640 1 -- 172.21.15.172:0/1816623287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd0106f10 msgr2=0x7f3cd019cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:45.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.753+0000 7f3cd564a640 1 --2- 172.21.15.172:0/1816623287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd0106f10 0x7f3cd019cc10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009230 tx=0x7f3cc8004270 comp rx=0 tx=0).stop 2024-09-10T08:46:45.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.753+0000 7f3cd564a640 1 -- 172.21.15.172:0/1816623287 shutdown_connections 2024-09-10T08:46:45.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.753+0000 7f3cd564a640 1 --2- 172.21.15.172:0/1816623287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cbc0a2b80 0x7f3cbc0a5040 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.753+0000 7f3cd564a640 1 --2- 172.21.15.172:0/1816623287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd0106f10 0x7f3cd019cc10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.753+0000 7f3cd564a640 1 --2- 172.21.15.172:0/1816623287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd0102790 0x7f3cd0198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.753+0000 7f3cd564a640 1 --2- 172.21.15.172:0/1816623287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd0101de0 0x7f3cd0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.753+0000 7f3cd564a640 1 -- 172.21.15.172:0/1816623287 >> 172.21.15.172:0/1816623287 conn(0x7f3cd00fd8c0 msgr2=0x7f3cd00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:45.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.753+0000 7f3cd564a640 1 -- 172.21.15.172:0/1816623287 shutdown_connections 2024-09-10T08:46:45.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.754+0000 7f3cd564a640 1 -- 172.21.15.172:0/1816623287 wait complete. 2024-09-10T08:46:45.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.922+0000 7fb812aa1640 1 -- 172.21.15.172:0/2900822518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb80c100a70 msgr2=0x7fb80c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:45.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.922+0000 7fb812aa1640 1 --2- 172.21.15.172:0/2900822518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb80c100a70 0x7fb80c100ed0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8009930 tx=0x7fb7f802a420 comp rx=0 tx=0).stop 2024-09-10T08:46:45.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.923+0000 7fb812aa1640 1 -- 172.21.15.172:0/2900822518 shutdown_connections 2024-09-10T08:46:45.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.923+0000 7fb812aa1640 1 --2- 172.21.15.172:0/2900822518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb80c101410 0x7fb80c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.923+0000 7fb812aa1640 1 --2- 172.21.15.172:0/2900822518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb80c100a70 0x7fb80c100ed0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.923+0000 7fb812aa1640 1 --2- 172.21.15.172:0/2900822518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb80c106a70 0x7fb80c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.923+0000 7fb812aa1640 1 -- 172.21.15.172:0/2900822518 >> 172.21.15.172:0/2900822518 conn(0x7fb80c0fc790 msgr2=0x7fb80c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:45.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.923+0000 7fb812aa1640 1 -- 172.21.15.172:0/2900822518 shutdown_connections 2024-09-10T08:46:45.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.923+0000 7fb812aa1640 1 -- 172.21.15.172:0/2900822518 wait complete. 2024-09-10T08:46:45.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.923+0000 7fb812aa1640 1 Processor -- start 2024-09-10T08:46:45.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.923+0000 7fb812aa1640 1 -- start start 2024-09-10T08:46:45.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.923+0000 7fb812aa1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb80c100a70 0x7fb80c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.923+0000 7fb812aa1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb80c101410 0x7fb80c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.923+0000 7fb812aa1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb80c106a70 0x7fb80c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.923+0000 7fb812aa1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c110d20 con 0x7fb80c106a70 2024-09-10T08:46:45.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.923+0000 7fb812aa1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c110ba0 con 0x7fb80c100a70 2024-09-10T08:46:45.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.923+0000 7fb812aa1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c110ea0 con 0x7fb80c101410 2024-09-10T08:46:45.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.924+0000 7fb81103e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb80c106a70 0x7fb80c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.924+0000 7fb81103e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb80c106a70 0x7fb80c19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33390/0 (socket says 172.21.15.172:33390) 2024-09-10T08:46:45.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.924+0000 7fb81103e640 1 -- 172.21.15.172:0/2216931186 learned_addr learned my addr 172.21.15.172:0/2216931186 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:45.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.924+0000 7fb81083d640 1 --2- 172.21.15.172:0/2216931186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb80c100a70 0x7fb80c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.924+0000 7fb80bfff640 1 --2- 172.21.15.172:0/2216931186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb80c101410 0x7fb80c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.924+0000 7fb81103e640 1 -- 172.21.15.172:0/2216931186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb80c101410 msgr2=0x7fb80c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:45.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.924+0000 7fb81103e640 1 --2- 172.21.15.172:0/2216931186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb80c101410 0x7fb80c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.924+0000 7fb81103e640 1 -- 172.21.15.172:0/2216931186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb80c100a70 msgr2=0x7fb80c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:45.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.924+0000 7fb81103e640 1 --2- 172.21.15.172:0/2216931186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb80c100a70 0x7fb80c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:45.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.924+0000 7fb81103e640 1 -- 172.21.15.172:0/2216931186 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80c19d380 con 0x7fb80c106a70 2024-09-10T08:46:45.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.924+0000 7fb81103e640 1 --2- 172.21.15.172:0/2216931186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb80c106a70 0x7fb80c19cca0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb80400eb10 tx=0x7fb80400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:45.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.924+0000 7fb809ffb640 1 -- 172.21.15.172:0/2216931186 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804004b20 con 0x7fb80c106a70 2024-09-10T08:46:45.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.924+0000 7fb809ffb640 1 -- 172.21.15.172:0/2216931186 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb804004cc0 con 0x7fb80c106a70 2024-09-10T08:46:45.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.924+0000 7fb812aa1640 1 -- 172.21.15.172:0/2216931186 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb80c19d670 con 0x7fb80c106a70 2024-09-10T08:46:45.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.924+0000 7fb809ffb640 1 -- 172.21.15.172:0/2216931186 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804010690 con 0x7fb80c106a70 2024-09-10T08:46:45.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.925+0000 7fb812aa1640 1 -- 172.21.15.172:0/2216931186 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb80c1a4f50 con 0x7fb80c106a70 2024-09-10T08:46:45.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.926+0000 7fb812aa1640 1 -- 172.21.15.172:0/2216931186 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb80c100ed0 con 0x7fb80c106a70 2024-09-10T08:46:45.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.926+0000 7fb809ffb640 1 -- 172.21.15.172:0/2216931186 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb804005020 con 0x7fb80c106a70 2024-09-10T08:46:45.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.927+0000 7fb809ffb640 1 --2- 172.21.15.172:0/2216931186 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb7fc0a2c00 0x7fb7fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:45.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.927+0000 7fb81083d640 1 --2- 172.21.15.172:0/2216931186 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb7fc0a2c00 0x7fb7fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:45.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.927+0000 7fb809ffb640 1 -- 172.21.15.172:0/2216931186 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb8040cf8b0 con 0x7fb80c106a70 2024-09-10T08:46:45.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.927+0000 7fb81083d640 1 --2- 172.21.15.172:0/2216931186 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb7fc0a2c00 0x7fb7fc0a50c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb800009b50 tx=0x7fb800005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:45.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:45.931+0000 7fb809ffb640 1 -- 172.21.15.172:0/2216931186 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb80409c4b0 con 0x7fb80c106a70 2024-09-10T08:46:46.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:46.097+0000 7fb812aa1640 1 -- 172.21.15.172:0/2216931186 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb80c10a070 con 0x7fb7fc0a2c00 2024-09-10T08:46:46.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:46.105+0000 7fb809ffb640 1 -- 172.21.15.172:0/2216931186 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340527 (secure 0 0 0) 0x7fb80c10a070 con 0x7fb7fc0a2c00 2024-09-10T08:46:46.109 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:46.111 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-09-10T08:46:46.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:46.108+0000 7fb812aa1640 1 -- 172.21.15.172:0/2216931186 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb7fc0a2c00 msgr2=0x7fb7fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:46.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:46.108+0000 7fb812aa1640 1 --2- 172.21.15.172:0/2216931186 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb7fc0a2c00 0x7fb7fc0a50c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb800009b50 tx=0x7fb800005f90 comp rx=0 tx=0).stop 2024-09-10T08:46:46.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:46.108+0000 7fb812aa1640 1 -- 172.21.15.172:0/2216931186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb80c106a70 msgr2=0x7fb80c19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:46.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:46.108+0000 7fb812aa1640 1 --2- 172.21.15.172:0/2216931186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb80c106a70 0x7fb80c19cca0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb80400eb10 tx=0x7fb80400c2d0 comp rx=0 tx=0).stop 2024-09-10T08:46:46.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:46.108+0000 7fb812aa1640 1 -- 172.21.15.172:0/2216931186 shutdown_connections 2024-09-10T08:46:46.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:46.108+0000 7fb812aa1640 1 --2- 172.21.15.172:0/2216931186 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb7fc0a2c00 0x7fb7fc0a50c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:46.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:46.108+0000 7fb812aa1640 1 --2- 172.21.15.172:0/2216931186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb80c106a70 0x7fb80c19cca0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:46.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:46.108+0000 7fb812aa1640 1 --2- 172.21.15.172:0/2216931186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb80c101410 0x7fb80c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:46.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:46.108+0000 7fb812aa1640 1 --2- 172.21.15.172:0/2216931186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb80c100a70 0x7fb80c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:46.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:46.108+0000 7fb812aa1640 1 -- 172.21.15.172:0/2216931186 >> 172.21.15.172:0/2216931186 conn(0x7fb80c0fc790 msgr2=0x7fb80c06ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:46.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:46.109+0000 7fb812aa1640 1 -- 172.21.15.172:0/2216931186 shutdown_connections 2024-09-10T08:46:46.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:46.109+0000 7fb812aa1640 1 -- 172.21.15.172:0/2216931186 wait complete. 2024-09-10T08:46:46.493 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:46 smithi172 ceph-mon[30854]: pgmap v366: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-10T08:46:46.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:46 smithi191 ceph-mon[35314]: pgmap v366: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-10T08:46:46.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:46 smithi174 ceph-mon[34068]: pgmap v366: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-10T08:46:46.814 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-10T08:46:45.165717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668080,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1250,"num_read_kb":754,"num_write":459,"num_write_kb":3107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":874,"ondisk_log_size":874,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376268,"kb_used_data":14540,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565556,"statfs":{"total":1151940427776,"available":1151555129344,"internally_reserved":0,"allocated":14888960,"data_stored":8240942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6789,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":65,"num_write":21,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003152"},"pg_stats":[{"pgid":"2.1f","version":"92'14","reported_seq":160,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:39.371312+0000","last_change":"2024-09-10T08:44:08.657737+0000","last_active":"2024-09-10T08:46:39.371312+0000","last_peered":"2024-09-10T08:46:39.371312+0000","last_clean":"2024-09-10T08:46:39.371312+0000","last_became_active":"2024-09-10T08:44:08.657473+0000","last_became_peered":"2024-09-10T08:44:08.657473+0000","last_unstale":"2024-09-10T08:46:39.371312+0000","last_undegraded":"2024-09-10T08:46:39.371312+0000","last_fullsized":"2024-09-10T08:46:39.371312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:40.757343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193599+0000","last_change":"2024-09-10T08:44:10.675090+0000","last_active":"2024-09-10T08:46:19.193599+0000","last_peered":"2024-09-10T08:46:19.193599+0000","last_clean":"2024-09-10T08:46:19.193599+0000","last_became_active":"2024-09-10T08:44:10.674612+0000","last_became_peered":"2024-09-10T08:44:10.674612+0000","last_unstale":"2024-09-10T08:46:19.193599+0000","last_undegraded":"2024-09-10T08:46:19.193599+0000","last_fullsized":"2024-09-10T08:46:19.193599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:46.821080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.807967+0000","last_change":"2024-09-10T08:44:17.721559+0000","last_active":"2024-09-10T08:46:18.807967+0000","last_peered":"2024-09-10T08:46:18.807967+0000","last_clean":"2024-09-10T08:46:18.807967+0000","last_became_active":"2024-09-10T08:44:17.721182+0000","last_became_peered":"2024-09-10T08:44:17.721182+0000","last_unstale":"2024-09-10T08:46:18.807967+0000","last_undegraded":"2024-09-10T08:46:18.807967+0000","last_fullsized":"2024-09-10T08:46:18.807967+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:18.791489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193663+0000","last_change":"2024-09-10T08:44:09.049171+0000","last_active":"2024-09-10T08:46:19.193663+0000","last_peered":"2024-09-10T08:46:19.193663+0000","last_clean":"2024-09-10T08:46:19.193663+0000","last_became_active":"2024-09-10T08:44:09.048806+0000","last_became_peered":"2024-09-10T08:44:09.048806+0000","last_unstale":"2024-09-10T08:46:19.193663+0000","last_undegraded":"2024-09-10T08:46:19.193663+0000","last_fullsized":"2024-09-10T08:46:19.193663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:21.023460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193866+0000","last_change":"2024-09-10T08:44:10.677569+0000","last_active":"2024-09-10T08:46:19.193866+0000","last_peered":"2024-09-10T08:46:19.193866+0000","last_clean":"2024-09-10T08:46:19.193866+0000","last_became_active":"2024-09-10T08:44:10.677048+0000","last_became_peered":"2024-09-10T08:44:10.677048+0000","last_unstale":"2024-09-10T08:46:19.193866+0000","last_undegraded":"2024-09-10T08:46:19.193866+0000","last_fullsized":"2024-09-10T08:46:19.193866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:35.523063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193389+0000","last_change":"2024-09-10T08:44:17.720138+0000","last_active":"2024-09-10T08:46:19.193389+0000","last_peered":"2024-09-10T08:46:19.193389+0000","last_clean":"2024-09-10T08:46:19.193389+0000","last_became_active":"2024-09-10T08:44:17.719344+0000","last_became_peered":"2024-09-10T08:44:17.719344+0000","last_unstale":"2024-09-10T08:46:19.193389+0000","last_undegraded":"2024-09-10T08:46:19.193389+0000","last_fullsized":"2024-09-10T08:46:19.193389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:11.758058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194115+0000","last_change":"2024-09-10T08:44:08.660099+0000","last_active":"2024-09-10T08:46:19.194115+0000","last_peered":"2024-09-10T08:46:19.194115+0000","last_clean":"2024-09-10T08:46:19.194115+0000","last_became_active":"2024-09-10T08:44:08.659745+0000","last_became_peered":"2024-09-10T08:44:08.659745+0000","last_unstale":"2024-09-10T08:46:19.194115+0000","last_undegraded":"2024-09-10T08:46:19.194115+0000","last_fullsized":"2024-09-10T08:46:19.194115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:43.960717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193791+0000","last_change":"2024-09-10T08:44:10.675583+0000","last_active":"2024-09-10T08:46:19.193791+0000","last_peered":"2024-09-10T08:46:19.193791+0000","last_clean":"2024-09-10T08:46:19.193791+0000","last_became_active":"2024-09-10T08:44:10.675296+0000","last_became_peered":"2024-09-10T08:44:10.675296+0000","last_unstale":"2024-09-10T08:46:19.193791+0000","last_undegraded":"2024-09-10T08:46:19.193791+0000","last_fullsized":"2024-09-10T08:46:19.193791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:31:51.147862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194363+0000","last_change":"2024-09-10T08:44:17.720104+0000","last_active":"2024-09-10T08:46:19.194363+0000","last_peered":"2024-09-10T08:46:19.194363+0000","last_clean":"2024-09-10T08:46:19.194363+0000","last_became_active":"2024-09-10T08:44:17.719607+0000","last_became_peered":"2024-09-10T08:44:17.719607+0000","last_unstale":"2024-09-10T08:46:19.194363+0000","last_undegraded":"2024-09-10T08:46:19.194363+0000","last_fullsized":"2024-09-10T08:46:19.194363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:52.976717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193719+0000","last_change":"2024-09-10T08:44:08.658588+0000","last_active":"2024-09-10T08:46:19.193719+0000","last_peered":"2024-09-10T08:46:19.193719+0000","last_clean":"2024-09-10T08:46:19.193719+0000","last_became_active":"2024-09-10T08:44:08.658053+0000","last_became_peered":"2024-09-10T08:44:08.658053+0000","last_unstale":"2024-09-10T08:46:19.193719+0000","last_undegraded":"2024-09-10T08:46:19.193719+0000","last_fullsized":"2024-09-10T08:46:19.193719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:26.744943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804775+0000","last_change":"2024-09-10T08:44:10.681978+0000","last_active":"2024-09-10T08:46:18.804775+0000","last_peered":"2024-09-10T08:46:18.804775+0000","last_clean":"2024-09-10T08:46:18.804775+0000","last_became_active":"2024-09-10T08:44:10.681600+0000","last_became_peered":"2024-09-10T08:44:10.681600+0000","last_unstale":"2024-09-10T08:46:18.804775+0000","last_undegraded":"2024-09-10T08:46:18.804775+0000","last_fullsized":"2024-09-10T08:46:18.804775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:37.190576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193813+0000","last_change":"2024-09-10T08:44:17.719784+0000","last_active":"2024-09-10T08:46:19.193813+0000","last_peered":"2024-09-10T08:46:19.193813+0000","last_clean":"2024-09-10T08:46:19.193813+0000","last_became_active":"2024-09-10T08:44:17.719182+0000","last_became_peered":"2024-09-10T08:44:17.719182+0000","last_unstale":"2024-09-10T08:46:19.193813+0000","last_undegraded":"2024-09-10T08:46:19.193813+0000","last_fullsized":"2024-09-10T08:46:19.193813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:15.237772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873709+0000","last_change":"2024-09-10T08:44:10.679069+0000","last_active":"2024-09-10T08:46:18.873709+0000","last_peered":"2024-09-10T08:46:18.873709+0000","last_clean":"2024-09-10T08:46:18.873709+0000","last_became_active":"2024-09-10T08:44:10.678597+0000","last_became_peered":"2024-09-10T08:44:10.678597+0000","last_unstale":"2024-09-10T08:46:18.873709+0000","last_undegraded":"2024-09-10T08:46:18.873709+0000","last_fullsized":"2024-09-10T08:46:18.873709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:52.693906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194153+0000","last_change":"2024-09-10T08:44:08.660886+0000","last_active":"2024-09-10T08:46:19.194153+0000","last_peered":"2024-09-10T08:46:19.194153+0000","last_clean":"2024-09-10T08:46:19.194153+0000","last_became_active":"2024-09-10T08:44:08.660478+0000","last_became_peered":"2024-09-10T08:44:08.660478+0000","last_unstale":"2024-09-10T08:46:19.194153+0000","last_undegraded":"2024-09-10T08:46:19.194153+0000","last_fullsized":"2024-09-10T08:46:19.194153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:41.785531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194323+0000","last_change":"2024-09-10T08:44:17.761929+0000","last_active":"2024-09-10T08:46:19.194323+0000","last_peered":"2024-09-10T08:46:19.194323+0000","last_clean":"2024-09-10T08:46:19.194323+0000","last_became_active":"2024-09-10T08:44:17.761440+0000","last_became_peered":"2024-09-10T08:44:17.761440+0000","last_unstale":"2024-09-10T08:46:19.194323+0000","last_undegraded":"2024-09-10T08:46:19.194323+0000","last_fullsized":"2024-09-10T08:46:19.194323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:37.817759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192895+0000","last_change":"2024-09-10T08:44:10.674071+0000","last_active":"2024-09-10T08:46:19.192895+0000","last_peered":"2024-09-10T08:46:19.192895+0000","last_clean":"2024-09-10T08:46:19.192895+0000","last_became_active":"2024-09-10T08:44:10.673770+0000","last_became_peered":"2024-09-10T08:44:10.673770+0000","last_unstale":"2024-09-10T08:46:19.192895+0000","last_undegraded":"2024-09-10T08:46:19.192895+0000","last_fullsized":"2024-09-10T08:46:19.192895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:44:19.944557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.231938+0000","last_change":"2024-09-10T08:45:18.264060+0000","last_active":"2024-09-10T08:46:34.231938+0000","last_peered":"2024-09-10T08:46:34.231938+0000","last_clean":"2024-09-10T08:46:34.231938+0000","last_became_active":"2024-09-10T08:45:18.255983+0000","last_became_peered":"2024-09-10T08:45:18.255983+0000","last_unstale":"2024-09-10T08:46:34.231938+0000","last_undegraded":"2024-09-10T08:46:34.231938+0000","last_fullsized":"2024-09-10T08:46:34.231938+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:10.810491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194248+0000","last_change":"2024-09-10T08:44:17.722877+0000","last_active":"2024-09-10T08:46:19.194248+0000","last_peered":"2024-09-10T08:46:19.194248+0000","last_clean":"2024-09-10T08:46:19.194248+0000","last_became_active":"2024-09-10T08:44:17.722046+0000","last_became_peered":"2024-09-10T08:44:17.722046+0000","last_unstale":"2024-09-10T08:46:19.194248+0000","last_undegraded":"2024-09-10T08:46:19.194248+0000","last_fullsized":"2024-09-10T08:46:19.194248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:32.675598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193398+0000","last_change":"2024-09-10T08:44:10.671500+0000","last_active":"2024-09-10T08:46:19.193398+0000","last_peered":"2024-09-10T08:46:19.193398+0000","last_clean":"2024-09-10T08:46:19.193398+0000","last_became_active":"2024-09-10T08:44:10.671039+0000","last_became_peered":"2024-09-10T08:44:10.671039+0000","last_unstale":"2024-09-10T08:46:19.193398+0000","last_undegraded":"2024-09-10T08:46:19.193398+0000","last_fullsized":"2024-09-10T08:46:19.193398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:12.977373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193090+0000","last_change":"2024-09-10T08:44:08.655467+0000","last_active":"2024-09-10T08:46:19.193090+0000","last_peered":"2024-09-10T08:46:19.193090+0000","last_clean":"2024-09-10T08:46:19.193090+0000","last_became_active":"2024-09-10T08:44:08.654991+0000","last_became_peered":"2024-09-10T08:44:08.654991+0000","last_unstale":"2024-09-10T08:46:19.193090+0000","last_undegraded":"2024-09-10T08:46:19.193090+0000","last_fullsized":"2024-09-10T08:46:19.193090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:33.124360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803728+0000","last_change":"2024-09-10T08:44:17.724250+0000","last_active":"2024-09-10T08:46:18.803728+0000","last_peered":"2024-09-10T08:46:18.803728+0000","last_clean":"2024-09-10T08:46:18.803728+0000","last_became_active":"2024-09-10T08:44:17.723733+0000","last_became_peered":"2024-09-10T08:44:17.723733+0000","last_unstale":"2024-09-10T08:46:18.803728+0000","last_undegraded":"2024-09-10T08:46:18.803728+0000","last_fullsized":"2024-09-10T08:46:18.803728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:40.098666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193144+0000","last_change":"2024-09-10T08:44:10.664159+0000","last_active":"2024-09-10T08:46:19.193144+0000","last_peered":"2024-09-10T08:46:19.193144+0000","last_clean":"2024-09-10T08:46:19.193144+0000","last_became_active":"2024-09-10T08:44:10.663674+0000","last_became_peered":"2024-09-10T08:44:10.663674+0000","last_unstale":"2024-09-10T08:46:19.193144+0000","last_undegraded":"2024-09-10T08:46:19.193144+0000","last_fullsized":"2024-09-10T08:46:19.193144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:24.234525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193535+0000","last_change":"2024-09-10T08:45:18.257226+0000","last_active":"2024-09-10T08:46:19.193535+0000","last_peered":"2024-09-10T08:46:19.193535+0000","last_clean":"2024-09-10T08:46:19.193535+0000","last_became_active":"2024-09-10T08:45:18.254113+0000","last_became_peered":"2024-09-10T08:45:18.254113+0000","last_unstale":"2024-09-10T08:46:19.193535+0000","last_undegraded":"2024-09-10T08:46:19.193535+0000","last_fullsized":"2024-09-10T08:46:19.193535+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:19.370105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194467+0000","last_change":"2024-09-10T08:44:17.761639+0000","last_active":"2024-09-10T08:46:19.194467+0000","last_peered":"2024-09-10T08:46:19.194467+0000","last_clean":"2024-09-10T08:46:19.194467+0000","last_became_active":"2024-09-10T08:44:17.761214+0000","last_became_peered":"2024-09-10T08:44:17.761214+0000","last_unstale":"2024-09-10T08:46:19.194467+0000","last_undegraded":"2024-09-10T08:46:19.194467+0000","last_fullsized":"2024-09-10T08:46:19.194467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:43.793195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193666+0000","last_change":"2024-09-10T08:45:18.258661+0000","last_active":"2024-09-10T08:46:19.193666+0000","last_peered":"2024-09-10T08:46:19.193666+0000","last_clean":"2024-09-10T08:46:19.193666+0000","last_became_active":"2024-09-10T08:45:18.258225+0000","last_became_peered":"2024-09-10T08:45:18.258225+0000","last_unstale":"2024-09-10T08:46:19.193666+0000","last_undegraded":"2024-09-10T08:46:19.193666+0000","last_fullsized":"2024-09-10T08:46:19.193666+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:28:37.184766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194660+0000","last_change":"2024-09-10T08:44:09.036632+0000","last_active":"2024-09-10T08:46:19.194660+0000","last_peered":"2024-09-10T08:46:19.194660+0000","last_clean":"2024-09-10T08:46:19.194660+0000","last_became_active":"2024-09-10T08:44:09.036254+0000","last_became_peered":"2024-09-10T08:44:09.036254+0000","last_unstale":"2024-09-10T08:46:19.194660+0000","last_undegraded":"2024-09-10T08:46:19.194660+0000","last_fullsized":"2024-09-10T08:46:19.194660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:59:40.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808140+0000","last_change":"2024-09-10T08:44:17.716215+0000","last_active":"2024-09-10T08:46:18.808140+0000","last_peered":"2024-09-10T08:46:18.808140+0000","last_clean":"2024-09-10T08:46:18.808140+0000","last_became_active":"2024-09-10T08:44:17.715809+0000","last_became_peered":"2024-09-10T08:44:17.715809+0000","last_unstale":"2024-09-10T08:46:18.808140+0000","last_undegraded":"2024-09-10T08:46:18.808140+0000","last_fullsized":"2024-09-10T08:46:18.808140+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:52.896401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193713+0000","last_change":"2024-09-10T08:44:10.677871+0000","last_active":"2024-09-10T08:46:19.193713+0000","last_peered":"2024-09-10T08:46:19.193713+0000","last_clean":"2024-09-10T08:46:19.193713+0000","last_became_active":"2024-09-10T08:44:10.677101+0000","last_became_peered":"2024-09-10T08:44:10.677101+0000","last_unstale":"2024-09-10T08:46:19.193713+0000","last_undegraded":"2024-09-10T08:46:19.193713+0000","last_fullsized":"2024-09-10T08:46:19.193713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:46.137249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'18","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.704451+0000","last_change":"2024-09-10T08:44:08.654428+0000","last_active":"2024-09-10T08:46:34.704451+0000","last_peered":"2024-09-10T08:46:34.704451+0000","last_clean":"2024-09-10T08:46:34.704451+0000","last_became_active":"2024-09-10T08:44:08.654119+0000","last_became_peered":"2024-09-10T08:44:08.654119+0000","last_unstale":"2024-09-10T08:46:34.704451+0000","last_undegraded":"2024-09-10T08:46:34.704451+0000","last_fullsized":"2024-09-10T08:46:34.704451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:56.787721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192538+0000","last_change":"2024-09-10T08:44:17.760828+0000","last_active":"2024-09-10T08:46:19.192538+0000","last_peered":"2024-09-10T08:46:19.192538+0000","last_clean":"2024-09-10T08:46:19.192538+0000","last_became_active":"2024-09-10T08:44:17.760367+0000","last_became_peered":"2024-09-10T08:44:17.760367+0000","last_unstale":"2024-09-10T08:46:19.192538+0000","last_undegraded":"2024-09-10T08:46:19.192538+0000","last_fullsized":"2024-09-10T08:46:19.192538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:11.130480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806446+0000","last_change":"2024-09-10T08:44:10.670179+0000","last_active":"2024-09-10T08:46:18.806446+0000","last_peered":"2024-09-10T08:46:18.806446+0000","last_clean":"2024-09-10T08:46:18.806446+0000","last_became_active":"2024-09-10T08:44:10.669366+0000","last_became_peered":"2024-09-10T08:44:10.669366+0000","last_unstale":"2024-09-10T08:46:18.806446+0000","last_undegraded":"2024-09-10T08:46:18.806446+0000","last_fullsized":"2024-09-10T08:46:18.806446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:41.916365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194210+0000","last_change":"2024-09-10T08:44:09.050441+0000","last_active":"2024-09-10T08:46:19.194210+0000","last_peered":"2024-09-10T08:46:19.194210+0000","last_clean":"2024-09-10T08:46:19.194210+0000","last_became_active":"2024-09-10T08:44:09.050070+0000","last_became_peered":"2024-09-10T08:44:09.050070+0000","last_unstale":"2024-09-10T08:46:19.194210+0000","last_undegraded":"2024-09-10T08:46:19.194210+0000","last_fullsized":"2024-09-10T08:46:19.194210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:13.540021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193573+0000","last_change":"2024-09-10T08:44:17.720098+0000","last_active":"2024-09-10T08:46:19.193573+0000","last_peered":"2024-09-10T08:46:19.193573+0000","last_clean":"2024-09-10T08:46:19.193573+0000","last_became_active":"2024-09-10T08:44:17.719627+0000","last_became_peered":"2024-09-10T08:44:17.719627+0000","last_unstale":"2024-09-10T08:46:19.193573+0000","last_undegraded":"2024-09-10T08:46:19.193573+0000","last_fullsized":"2024-09-10T08:46:19.193573+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:50.020919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804677+0000","last_change":"2024-09-10T08:44:10.669630+0000","last_active":"2024-09-10T08:46:18.804677+0000","last_peered":"2024-09-10T08:46:18.804677+0000","last_clean":"2024-09-10T08:46:18.804677+0000","last_became_active":"2024-09-10T08:44:10.669307+0000","last_became_peered":"2024-09-10T08:44:10.669307+0000","last_unstale":"2024-09-10T08:46:18.804677+0000","last_undegraded":"2024-09-10T08:46:18.804677+0000","last_fullsized":"2024-09-10T08:46:18.804677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:06.320046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803568+0000","last_change":"2024-09-10T08:44:09.050715+0000","last_active":"2024-09-10T08:46:18.803568+0000","last_peered":"2024-09-10T08:46:18.803568+0000","last_clean":"2024-09-10T08:46:18.803568+0000","last_became_active":"2024-09-10T08:44:09.050347+0000","last_became_peered":"2024-09-10T08:44:09.050347+0000","last_unstale":"2024-09-10T08:46:18.803568+0000","last_undegraded":"2024-09-10T08:46:18.803568+0000","last_fullsized":"2024-09-10T08:46:18.803568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:18.689652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194038+0000","last_change":"2024-09-10T08:45:18.260444+0000","last_active":"2024-09-10T08:46:19.194038+0000","last_peered":"2024-09-10T08:46:19.194038+0000","last_clean":"2024-09-10T08:46:19.194038+0000","last_became_active":"2024-09-10T08:45:18.259949+0000","last_became_peered":"2024-09-10T08:45:18.259949+0000","last_unstale":"2024-09-10T08:46:19.194038+0000","last_undegraded":"2024-09-10T08:46:19.194038+0000","last_fullsized":"2024-09-10T08:46:19.194038+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:56.391253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804630+0000","last_change":"2024-09-10T08:44:10.670881+0000","last_active":"2024-09-10T08:46:18.804630+0000","last_peered":"2024-09-10T08:46:18.804630+0000","last_clean":"2024-09-10T08:46:18.804630+0000","last_became_active":"2024-09-10T08:44:10.669791+0000","last_became_peered":"2024-09-10T08:44:10.669791+0000","last_unstale":"2024-09-10T08:46:18.804630+0000","last_undegraded":"2024-09-10T08:46:18.804630+0000","last_fullsized":"2024-09-10T08:46:18.804630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:33.644205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'6","reported_seq":156,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:42.375958+0000","last_change":"2024-09-10T08:44:08.657938+0000","last_active":"2024-09-10T08:46:42.375958+0000","last_peered":"2024-09-10T08:46:42.375958+0000","last_clean":"2024-09-10T08:46:42.375958+0000","last_became_active":"2024-09-10T08:44:08.657526+0000","last_became_peered":"2024-09-10T08:44:08.657526+0000","last_unstale":"2024-09-10T08:46:42.375958+0000","last_undegraded":"2024-09-10T08:46:42.375958+0000","last_fullsized":"2024-09-10T08:46:42.375958+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:15.673561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806147+0000","last_change":"2024-09-10T08:44:17.721619+0000","last_active":"2024-09-10T08:46:18.806147+0000","last_peered":"2024-09-10T08:46:18.806147+0000","last_clean":"2024-09-10T08:46:18.806147+0000","last_became_active":"2024-09-10T08:44:17.721175+0000","last_became_peered":"2024-09-10T08:44:17.721175+0000","last_unstale":"2024-09-10T08:46:18.806147+0000","last_undegraded":"2024-09-10T08:46:18.806147+0000","last_fullsized":"2024-09-10T08:46:18.806147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:52.244120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804320+0000","last_change":"2024-09-10T08:44:10.676164+0000","last_active":"2024-09-10T08:46:18.804320+0000","last_peered":"2024-09-10T08:46:18.804320+0000","last_clean":"2024-09-10T08:46:18.804320+0000","last_became_active":"2024-09-10T08:44:10.675820+0000","last_became_peered":"2024-09-10T08:44:10.675820+0000","last_unstale":"2024-09-10T08:46:18.804320+0000","last_undegraded":"2024-09-10T08:46:18.804320+0000","last_fullsized":"2024-09-10T08:46:18.804320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:24.477584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":157,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:42.375891+0000","last_change":"2024-09-10T08:44:09.049124+0000","last_active":"2024-09-10T08:46:42.375891+0000","last_peered":"2024-09-10T08:46:42.375891+0000","last_clean":"2024-09-10T08:46:42.375891+0000","last_became_active":"2024-09-10T08:44:09.048799+0000","last_became_peered":"2024-09-10T08:44:09.048799+0000","last_unstale":"2024-09-10T08:46:42.375891+0000","last_undegraded":"2024-09-10T08:46:42.375891+0000","last_fullsized":"2024-09-10T08:46:42.375891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:43:39.849727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806891+0000","last_change":"2024-09-10T08:44:17.761130+0000","last_active":"2024-09-10T08:46:18.806891+0000","last_peered":"2024-09-10T08:46:18.806891+0000","last_clean":"2024-09-10T08:46:18.806891+0000","last_became_active":"2024-09-10T08:44:17.760753+0000","last_became_peered":"2024-09-10T08:44:17.760753+0000","last_unstale":"2024-09-10T08:46:18.806891+0000","last_undegraded":"2024-09-10T08:46:18.806891+0000","last_fullsized":"2024-09-10T08:46:18.806891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:42.971962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.809258+0000","last_change":"2024-09-10T08:46:18.809258+0000","last_active":"2024-09-10T08:46:18.809258+0000","last_peered":"2024-09-10T08:46:18.809258+0000","last_clean":"2024-09-10T08:46:18.809258+0000","last_became_active":"2024-09-10T08:46:18.808908+0000","last_became_peered":"2024-09-10T08:46:18.808908+0000","last_unstale":"2024-09-10T08:46:18.809258+0000","last_undegraded":"2024-09-10T08:46:18.809258+0000","last_fullsized":"2024-09-10T08:46:18.809258+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:10.305672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873500+0000","last_change":"2024-09-10T08:44:08.658862+0000","last_active":"2024-09-10T08:46:18.873500+0000","last_peered":"2024-09-10T08:46:18.873500+0000","last_clean":"2024-09-10T08:46:18.873500+0000","last_became_active":"2024-09-10T08:44:08.657880+0000","last_became_peered":"2024-09-10T08:44:08.657880+0000","last_unstale":"2024-09-10T08:46:18.873500+0000","last_undegraded":"2024-09-10T08:46:18.873500+0000","last_fullsized":"2024-09-10T08:46:18.873500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:34.224025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194185+0000","last_change":"2024-09-10T08:45:18.268446+0000","last_active":"2024-09-10T08:46:19.194185+0000","last_peered":"2024-09-10T08:46:19.194185+0000","last_clean":"2024-09-10T08:46:19.194185+0000","last_became_active":"2024-09-10T08:45:18.267988+0000","last_became_peered":"2024-09-10T08:45:18.267988+0000","last_unstale":"2024-09-10T08:46:19.194185+0000","last_undegraded":"2024-09-10T08:46:19.194185+0000","last_fullsized":"2024-09-10T08:46:19.194185+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:42:35.990794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193214+0000","last_change":"2024-09-10T08:44:10.673258+0000","last_active":"2024-09-10T08:46:19.193214+0000","last_peered":"2024-09-10T08:46:19.193214+0000","last_clean":"2024-09-10T08:46:19.193214+0000","last_became_active":"2024-09-10T08:44:10.672854+0000","last_became_peered":"2024-09-10T08:44:10.672854+0000","last_unstale":"2024-09-10T08:46:19.193214+0000","last_undegraded":"2024-09-10T08:46:19.193214+0000","last_fullsized":"2024-09-10T08:46:19.193214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:35.913820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194588+0000","last_change":"2024-09-10T08:44:08.658039+0000","last_active":"2024-09-10T08:46:19.194588+0000","last_peered":"2024-09-10T08:46:19.194588+0000","last_clean":"2024-09-10T08:46:19.194588+0000","last_became_active":"2024-09-10T08:44:08.657636+0000","last_became_peered":"2024-09-10T08:44:08.657636+0000","last_unstale":"2024-09-10T08:46:19.194588+0000","last_undegraded":"2024-09-10T08:46:19.194588+0000","last_fullsized":"2024-09-10T08:46:19.194588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:09.615992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873591+0000","last_change":"2024-09-10T08:42:57.666747+0000","last_active":"2024-09-10T08:46:18.873591+0000","last_peered":"2024-09-10T08:46:18.873591+0000","last_clean":"2024-09-10T08:46:18.873591+0000","last_became_active":"2024-09-10T08:42:57.666417+0000","last_became_peered":"2024-09-10T08:42:57.666417+0000","last_unstale":"2024-09-10T08:46:18.873591+0000","last_undegraded":"2024-09-10T08:46:18.873591+0000","last_fullsized":"2024-09-10T08:46:18.873591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_clean_scrub_stamp":"2024-09-10T08:39:31.729794+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.547989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874074+0000","last_change":"2024-09-10T08:44:17.723021+0000","last_active":"2024-09-10T08:46:18.874074+0000","last_peered":"2024-09-10T08:46:18.874074+0000","last_clean":"2024-09-10T08:46:18.874074+0000","last_became_active":"2024-09-10T08:44:17.722222+0000","last_became_peered":"2024-09-10T08:44:17.722222+0000","last_unstale":"2024-09-10T08:46:18.874074+0000","last_undegraded":"2024-09-10T08:46:18.874074+0000","last_fullsized":"2024-09-10T08:46:18.874074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:24.444427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874129+0000","last_change":"2024-09-10T08:44:10.676651+0000","last_active":"2024-09-10T08:46:18.874129+0000","last_peered":"2024-09-10T08:46:18.874129+0000","last_clean":"2024-09-10T08:46:18.874129+0000","last_became_active":"2024-09-10T08:44:10.676342+0000","last_became_peered":"2024-09-10T08:44:10.676342+0000","last_unstale":"2024-09-10T08:46:18.874129+0000","last_undegraded":"2024-09-10T08:46:18.874129+0000","last_fullsized":"2024-09-10T08:46:18.874129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:07.021273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'8","reported_seq":159,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:40.373644+0000","last_change":"2024-09-10T08:44:09.050305+0000","last_active":"2024-09-10T08:46:40.373644+0000","last_peered":"2024-09-10T08:46:40.373644+0000","last_clean":"2024-09-10T08:46:40.373644+0000","last_became_active":"2024-09-10T08:44:09.049958+0000","last_became_peered":"2024-09-10T08:44:09.049958+0000","last_unstale":"2024-09-10T08:46:40.373644+0000","last_undegraded":"2024-09-10T08:46:40.373644+0000","last_fullsized":"2024-09-10T08:46:40.373644+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:42.219190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2199,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193720+0000","last_change":"2024-09-10T08:44:17.720219+0000","last_active":"2024-09-10T08:46:19.193720+0000","last_peered":"2024-09-10T08:46:19.193720+0000","last_clean":"2024-09-10T08:46:19.193720+0000","last_became_active":"2024-09-10T08:44:17.719756+0000","last_became_peered":"2024-09-10T08:44:17.719756+0000","last_unstale":"2024-09-10T08:46:19.193720+0000","last_undegraded":"2024-09-10T08:46:19.193720+0000","last_fullsized":"2024-09-10T08:46:19.193720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:01.070434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873929+0000","last_change":"2024-09-10T08:44:10.678986+0000","last_active":"2024-09-10T08:46:18.873929+0000","last_peered":"2024-09-10T08:46:18.873929+0000","last_clean":"2024-09-10T08:46:18.873929+0000","last_became_active":"2024-09-10T08:44:10.678654+0000","last_became_peered":"2024-09-10T08:44:10.678654+0000","last_unstale":"2024-09-10T08:46:18.873929+0000","last_undegraded":"2024-09-10T08:46:18.873929+0000","last_fullsized":"2024-09-10T08:46:18.873929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:07.116802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805026+0000","last_change":"2024-09-10T08:44:08.655025+0000","last_active":"2024-09-10T08:46:18.805026+0000","last_peered":"2024-09-10T08:46:18.805026+0000","last_clean":"2024-09-10T08:46:18.805026+0000","last_became_active":"2024-09-10T08:44:08.654728+0000","last_became_peered":"2024-09-10T08:44:08.654728+0000","last_unstale":"2024-09-10T08:46:18.805026+0000","last_undegraded":"2024-09-10T08:46:18.805026+0000","last_fullsized":"2024-09-10T08:46:18.805026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:31.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193534+0000","last_change":"2024-09-10T08:45:18.261492+0000","last_active":"2024-09-10T08:46:19.193534+0000","last_peered":"2024-09-10T08:46:19.193534+0000","last_clean":"2024-09-10T08:46:19.193534+0000","last_became_active":"2024-09-10T08:45:18.261111+0000","last_became_peered":"2024-09-10T08:45:18.261111+0000","last_unstale":"2024-09-10T08:46:19.193534+0000","last_undegraded":"2024-09-10T08:46:19.193534+0000","last_fullsized":"2024-09-10T08:46:19.193534+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:48.709459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874006+0000","last_change":"2024-09-10T08:44:10.675072+0000","last_active":"2024-09-10T08:46:18.874006+0000","last_peered":"2024-09-10T08:46:18.874006+0000","last_clean":"2024-09-10T08:46:18.874006+0000","last_became_active":"2024-09-10T08:44:10.674636+0000","last_became_peered":"2024-09-10T08:44:10.674636+0000","last_unstale":"2024-09-10T08:46:18.874006+0000","last_undegraded":"2024-09-10T08:46:18.874006+0000","last_fullsized":"2024-09-10T08:46:18.874006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:55.198445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806864+0000","last_change":"2024-09-10T08:44:08.656450+0000","last_active":"2024-09-10T08:46:18.806864+0000","last_peered":"2024-09-10T08:46:18.806864+0000","last_clean":"2024-09-10T08:46:18.806864+0000","last_became_active":"2024-09-10T08:44:08.655967+0000","last_became_peered":"2024-09-10T08:44:08.655967+0000","last_unstale":"2024-09-10T08:46:18.806864+0000","last_undegraded":"2024-09-10T08:46:18.806864+0000","last_fullsized":"2024-09-10T08:46:18.806864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:29.273167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804195+0000","last_change":"2024-09-10T08:44:17.722217+0000","last_active":"2024-09-10T08:46:18.804195+0000","last_peered":"2024-09-10T08:46:18.804195+0000","last_clean":"2024-09-10T08:46:18.804195+0000","last_became_active":"2024-09-10T08:44:17.721800+0000","last_became_peered":"2024-09-10T08:44:17.721800+0000","last_unstale":"2024-09-10T08:46:18.804195+0000","last_undegraded":"2024-09-10T08:46:18.804195+0000","last_fullsized":"2024-09-10T08:46:18.804195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:58.954397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194230+0000","last_change":"2024-09-10T08:45:18.267857+0000","last_active":"2024-09-10T08:46:19.194230+0000","last_peered":"2024-09-10T08:46:19.194230+0000","last_clean":"2024-09-10T08:46:19.194230+0000","last_became_active":"2024-09-10T08:45:18.267440+0000","last_became_peered":"2024-09-10T08:45:18.267440+0000","last_unstale":"2024-09-10T08:46:19.194230+0000","last_undegraded":"2024-09-10T08:46:19.194230+0000","last_fullsized":"2024-09-10T08:46:19.194230+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:28.659915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:23.312217+0000","last_change":"2024-09-10T08:44:08.659003+0000","last_active":"2024-09-10T08:46:23.312217+0000","last_peered":"2024-09-10T08:46:23.312217+0000","last_clean":"2024-09-10T08:46:23.312217+0000","last_became_active":"2024-09-10T08:44:08.658324+0000","last_became_peered":"2024-09-10T08:44:08.658324+0000","last_unstale":"2024-09-10T08:46:23.312217+0000","last_undegraded":"2024-09-10T08:46:23.312217+0000","last_fullsized":"2024-09-10T08:46:23.312217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:44.743053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804596+0000","last_change":"2024-09-10T08:44:17.723363+0000","last_active":"2024-09-10T08:46:18.804596+0000","last_peered":"2024-09-10T08:46:18.804596+0000","last_clean":"2024-09-10T08:46:18.804596+0000","last_became_active":"2024-09-10T08:44:17.721799+0000","last_became_peered":"2024-09-10T08:44:17.721799+0000","last_unstale":"2024-09-10T08:46:18.804596+0000","last_undegraded":"2024-09-10T08:46:18.804596+0000","last_fullsized":"2024-09-10T08:46:18.804596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:08:28.772193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804709+0000","last_change":"2024-09-10T08:44:10.682952+0000","last_active":"2024-09-10T08:46:18.804709+0000","last_peered":"2024-09-10T08:46:18.804709+0000","last_clean":"2024-09-10T08:46:18.804709+0000","last_became_active":"2024-09-10T08:44:10.682249+0000","last_became_peered":"2024-09-10T08:44:10.682249+0000","last_unstale":"2024-09-10T08:46:18.804709+0000","last_undegraded":"2024-09-10T08:46:18.804709+0000","last_fullsized":"2024-09-10T08:46:18.804709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:32:21.285461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193239+0000","last_change":"2024-09-10T08:44:09.050454+0000","last_active":"2024-09-10T08:46:19.193239+0000","last_peered":"2024-09-10T08:46:19.193239+0000","last_clean":"2024-09-10T08:46:19.193239+0000","last_became_active":"2024-09-10T08:44:09.050036+0000","last_became_peered":"2024-09-10T08:44:09.050036+0000","last_unstale":"2024-09-10T08:46:19.193239+0000","last_undegraded":"2024-09-10T08:46:19.193239+0000","last_fullsized":"2024-09-10T08:46:19.193239+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:55.141471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806725+0000","last_change":"2024-09-10T08:44:17.721203+0000","last_active":"2024-09-10T08:46:18.806725+0000","last_peered":"2024-09-10T08:46:18.806725+0000","last_clean":"2024-09-10T08:46:18.806725+0000","last_became_active":"2024-09-10T08:44:17.720737+0000","last_became_peered":"2024-09-10T08:44:17.720737+0000","last_unstale":"2024-09-10T08:46:18.806725+0000","last_undegraded":"2024-09-10T08:46:18.806725+0000","last_fullsized":"2024-09-10T08:46:18.806725+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:32.081079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873435+0000","last_change":"2024-09-10T08:44:10.680944+0000","last_active":"2024-09-10T08:46:18.873435+0000","last_peered":"2024-09-10T08:46:18.873435+0000","last_clean":"2024-09-10T08:46:18.873435+0000","last_became_active":"2024-09-10T08:44:10.680644+0000","last_became_peered":"2024-09-10T08:44:10.680644+0000","last_unstale":"2024-09-10T08:46:18.873435+0000","last_undegraded":"2024-09-10T08:46:18.873435+0000","last_fullsized":"2024-09-10T08:46:18.873435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:46.495894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":159,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:42.376726+0000","last_change":"2024-09-10T08:44:08.658135+0000","last_active":"2024-09-10T08:46:42.376726+0000","last_peered":"2024-09-10T08:46:42.376726+0000","last_clean":"2024-09-10T08:46:42.376726+0000","last_became_active":"2024-09-10T08:44:08.657793+0000","last_became_peered":"2024-09-10T08:44:08.657793+0000","last_unstale":"2024-09-10T08:46:42.376726+0000","last_undegraded":"2024-09-10T08:46:42.376726+0000","last_fullsized":"2024-09-10T08:46:42.376726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:16.376268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193715+0000","last_change":"2024-09-10T08:45:18.271678+0000","last_active":"2024-09-10T08:46:19.193715+0000","last_peered":"2024-09-10T08:46:19.193715+0000","last_clean":"2024-09-10T08:46:19.193715+0000","last_became_active":"2024-09-10T08:45:18.271209+0000","last_became_peered":"2024-09-10T08:45:18.271209+0000","last_unstale":"2024-09-10T08:46:19.193715+0000","last_undegraded":"2024-09-10T08:46:19.193715+0000","last_fullsized":"2024-09-10T08:46:19.193715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:17.730839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873847+0000","last_change":"2024-09-10T08:44:10.678254+0000","last_active":"2024-09-10T08:46:18.873847+0000","last_peered":"2024-09-10T08:46:18.873847+0000","last_clean":"2024-09-10T08:46:18.873847+0000","last_became_active":"2024-09-10T08:44:10.677891+0000","last_became_peered":"2024-09-10T08:44:10.677891+0000","last_unstale":"2024-09-10T08:46:18.873847+0000","last_undegraded":"2024-09-10T08:46:18.873847+0000","last_fullsized":"2024-09-10T08:46:18.873847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:32:43.399049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804719+0000","last_change":"2024-09-10T08:44:08.656315+0000","last_active":"2024-09-10T08:46:18.804719+0000","last_peered":"2024-09-10T08:46:18.804719+0000","last_clean":"2024-09-10T08:46:18.804719+0000","last_became_active":"2024-09-10T08:44:08.655978+0000","last_became_peered":"2024-09-10T08:44:08.655978+0000","last_unstale":"2024-09-10T08:46:18.804719+0000","last_undegraded":"2024-09-10T08:46:18.804719+0000","last_fullsized":"2024-09-10T08:46:18.804719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:15.025354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193456+0000","last_change":"2024-09-10T08:44:17.726789+0000","last_active":"2024-09-10T08:46:19.193456+0000","last_peered":"2024-09-10T08:46:19.193456+0000","last_clean":"2024-09-10T08:46:19.193456+0000","last_became_active":"2024-09-10T08:44:17.726450+0000","last_became_peered":"2024-09-10T08:44:17.726450+0000","last_unstale":"2024-09-10T08:46:19.193456+0000","last_undegraded":"2024-09-10T08:46:19.193456+0000","last_fullsized":"2024-09-10T08:46:19.193456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:03.899822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806240+0000","last_change":"2024-09-10T08:44:10.677359+0000","last_active":"2024-09-10T08:46:18.806240+0000","last_peered":"2024-09-10T08:46:18.806240+0000","last_clean":"2024-09-10T08:46:18.806240+0000","last_became_active":"2024-09-10T08:44:10.676782+0000","last_became_peered":"2024-09-10T08:44:10.676782+0000","last_unstale":"2024-09-10T08:46:18.806240+0000","last_undegraded":"2024-09-10T08:46:18.806240+0000","last_fullsized":"2024-09-10T08:46:18.806240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:31.721714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808306+0000","last_change":"2024-09-10T08:44:08.656993+0000","last_active":"2024-09-10T08:46:18.808306+0000","last_peered":"2024-09-10T08:46:18.808306+0000","last_clean":"2024-09-10T08:46:18.808306+0000","last_became_active":"2024-09-10T08:44:08.656571+0000","last_became_peered":"2024-09-10T08:44:08.656571+0000","last_unstale":"2024-09-10T08:46:18.808306+0000","last_undegraded":"2024-09-10T08:46:18.808306+0000","last_fullsized":"2024-09-10T08:46:18.808306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:32.202740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804870+0000","last_change":"2024-09-10T08:44:17.725674+0000","last_active":"2024-09-10T08:46:18.804870+0000","last_peered":"2024-09-10T08:46:18.804870+0000","last_clean":"2024-09-10T08:46:18.804870+0000","last_became_active":"2024-09-10T08:44:17.725263+0000","last_became_peered":"2024-09-10T08:44:17.725263+0000","last_unstale":"2024-09-10T08:46:18.804870+0000","last_undegraded":"2024-09-10T08:46:18.804870+0000","last_fullsized":"2024-09-10T08:46:18.804870+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:18.183157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193937+0000","last_change":"2024-09-10T08:44:10.676871+0000","last_active":"2024-09-10T08:46:19.193937+0000","last_peered":"2024-09-10T08:46:19.193937+0000","last_clean":"2024-09-10T08:46:19.193937+0000","last_became_active":"2024-09-10T08:44:10.676540+0000","last_became_peered":"2024-09-10T08:44:10.676540+0000","last_unstale":"2024-09-10T08:46:19.193937+0000","last_undegraded":"2024-09-10T08:46:19.193937+0000","last_fullsized":"2024-09-10T08:46:19.193937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:50.964100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'41","reported_seq":305,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:39.372295+0000","last_change":"2024-09-10T08:44:08.660113+0000","last_active":"2024-09-10T08:46:39.372295+0000","last_peered":"2024-09-10T08:46:39.372295+0000","last_clean":"2024-09-10T08:46:39.372295+0000","last_became_active":"2024-09-10T08:44:08.659644+0000","last_became_peered":"2024-09-10T08:44:08.659644+0000","last_unstale":"2024-09-10T08:46:39.372295+0000","last_undegraded":"2024-09-10T08:46:39.372295+0000","last_fullsized":"2024-09-10T08:46:39.372295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:01.605344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68542,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":225,"num_read_kb":180,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803342+0000","last_change":"2024-09-10T08:44:17.723653+0000","last_active":"2024-09-10T08:46:18.803342+0000","last_peered":"2024-09-10T08:46:18.803342+0000","last_clean":"2024-09-10T08:46:18.803342+0000","last_became_active":"2024-09-10T08:44:17.723284+0000","last_became_peered":"2024-09-10T08:44:17.723284+0000","last_unstale":"2024-09-10T08:46:18.803342+0000","last_undegraded":"2024-09-10T08:46:18.803342+0000","last_fullsized":"2024-09-10T08:46:18.803342+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:17.162526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193970+0000","last_change":"2024-09-10T08:44:10.681837+0000","last_active":"2024-09-10T08:46:19.193970+0000","last_peered":"2024-09-10T08:46:19.193970+0000","last_clean":"2024-09-10T08:46:19.193970+0000","last_became_active":"2024-09-10T08:44:10.681518+0000","last_became_peered":"2024-09-10T08:44:10.681518+0000","last_unstale":"2024-09-10T08:46:19.193970+0000","last_undegraded":"2024-09-10T08:46:19.193970+0000","last_fullsized":"2024-09-10T08:46:19.193970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:10.004215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804988+0000","last_change":"2024-09-10T08:44:09.050394+0000","last_active":"2024-09-10T08:46:18.804988+0000","last_peered":"2024-09-10T08:46:18.804988+0000","last_clean":"2024-09-10T08:46:18.804988+0000","last_became_active":"2024-09-10T08:44:09.049969+0000","last_became_peered":"2024-09-10T08:44:09.049969+0000","last_unstale":"2024-09-10T08:46:18.804988+0000","last_undegraded":"2024-09-10T08:46:18.804988+0000","last_fullsized":"2024-09-10T08:46:18.804988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:43.519246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194174+0000","last_change":"2024-09-10T08:44:17.720318+0000","last_active":"2024-09-10T08:46:19.194174+0000","last_peered":"2024-09-10T08:46:19.194174+0000","last_clean":"2024-09-10T08:46:19.194174+0000","last_became_active":"2024-09-10T08:44:17.719598+0000","last_became_peered":"2024-09-10T08:44:17.719598+0000","last_unstale":"2024-09-10T08:46:19.194174+0000","last_undegraded":"2024-09-10T08:46:19.194174+0000","last_fullsized":"2024-09-10T08:46:19.194174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:15:49.520324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193946+0000","last_change":"2024-09-10T08:44:10.677340+0000","last_active":"2024-09-10T08:46:19.193946+0000","last_peered":"2024-09-10T08:46:19.193946+0000","last_clean":"2024-09-10T08:46:19.193946+0000","last_became_active":"2024-09-10T08:44:10.676628+0000","last_became_peered":"2024-09-10T08:44:10.676628+0000","last_unstale":"2024-09-10T08:46:19.193946+0000","last_undegraded":"2024-09-10T08:46:19.193946+0000","last_fullsized":"2024-09-10T08:46:19.193946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:59:17.924217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874027+0000","last_change":"2024-09-10T08:44:08.660254+0000","last_active":"2024-09-10T08:46:18.874027+0000","last_peered":"2024-09-10T08:46:18.874027+0000","last_clean":"2024-09-10T08:46:18.874027+0000","last_became_active":"2024-09-10T08:44:08.659667+0000","last_became_peered":"2024-09-10T08:44:08.659667+0000","last_unstale":"2024-09-10T08:46:18.874027+0000","last_undegraded":"2024-09-10T08:46:18.874027+0000","last_fullsized":"2024-09-10T08:46:18.874027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:49:49.146867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803198+0000","last_change":"2024-09-10T08:44:17.722373+0000","last_active":"2024-09-10T08:46:18.803198+0000","last_peered":"2024-09-10T08:46:18.803198+0000","last_clean":"2024-09-10T08:46:18.803198+0000","last_became_active":"2024-09-10T08:44:17.722035+0000","last_became_peered":"2024-09-10T08:44:17.722035+0000","last_unstale":"2024-09-10T08:46:18.803198+0000","last_undegraded":"2024-09-10T08:46:18.803198+0000","last_fullsized":"2024-09-10T08:46:18.803198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:30:36.187364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194365+0000","last_change":"2024-09-10T08:44:10.672585+0000","last_active":"2024-09-10T08:46:19.194365+0000","last_peered":"2024-09-10T08:46:19.194365+0000","last_clean":"2024-09-10T08:46:19.194365+0000","last_became_active":"2024-09-10T08:44:10.671987+0000","last_became_peered":"2024-09-10T08:44:10.671987+0000","last_unstale":"2024-09-10T08:46:19.194365+0000","last_undegraded":"2024-09-10T08:46:19.194365+0000","last_fullsized":"2024-09-10T08:46:19.194365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:48.660750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'7","reported_seq":258,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:39.371955+0000","last_change":"2024-09-10T08:44:08.658069+0000","last_active":"2024-09-10T08:46:39.371955+0000","last_peered":"2024-09-10T08:46:39.371955+0000","last_clean":"2024-09-10T08:46:39.371955+0000","last_became_active":"2024-09-10T08:44:08.657702+0000","last_became_peered":"2024-09-10T08:44:08.657702+0000","last_unstale":"2024-09-10T08:46:39.371955+0000","last_undegraded":"2024-09-10T08:46:39.371955+0000","last_fullsized":"2024-09-10T08:46:39.371955+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:17.383267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":212,"num_read_kb":212,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192772+0000","last_change":"2024-09-10T08:44:17.717984+0000","last_active":"2024-09-10T08:46:19.192772+0000","last_peered":"2024-09-10T08:46:19.192772+0000","last_clean":"2024-09-10T08:46:19.192772+0000","last_became_active":"2024-09-10T08:44:17.717581+0000","last_became_peered":"2024-09-10T08:44:17.717581+0000","last_unstale":"2024-09-10T08:46:19.192772+0000","last_undegraded":"2024-09-10T08:46:19.192772+0000","last_fullsized":"2024-09-10T08:46:19.192772+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:25.330732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193566+0000","last_change":"2024-09-10T08:44:17.721244+0000","last_active":"2024-09-10T08:46:19.193566+0000","last_peered":"2024-09-10T08:46:19.193566+0000","last_clean":"2024-09-10T08:46:19.193566+0000","last_became_active":"2024-09-10T08:44:17.720884+0000","last_became_peered":"2024-09-10T08:44:17.720884+0000","last_unstale":"2024-09-10T08:46:19.193566+0000","last_undegraded":"2024-09-10T08:46:19.193566+0000","last_fullsized":"2024-09-10T08:46:19.193566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.769513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803279+0000","last_change":"2024-09-10T08:44:10.667558+0000","last_active":"2024-09-10T08:46:18.803279+0000","last_peered":"2024-09-10T08:46:18.803279+0000","last_clean":"2024-09-10T08:46:18.803279+0000","last_became_active":"2024-09-10T08:44:10.667214+0000","last_became_peered":"2024-09-10T08:44:10.667214+0000","last_unstale":"2024-09-10T08:46:18.803279+0000","last_undegraded":"2024-09-10T08:46:18.803279+0000","last_fullsized":"2024-09-10T08:46:18.803279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:53.630790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804697+0000","last_change":"2024-09-10T08:44:08.657930+0000","last_active":"2024-09-10T08:46:18.804697+0000","last_peered":"2024-09-10T08:46:18.804697+0000","last_clean":"2024-09-10T08:46:18.804697+0000","last_became_active":"2024-09-10T08:44:08.657611+0000","last_became_peered":"2024-09-10T08:44:08.657611+0000","last_unstale":"2024-09-10T08:46:18.804697+0000","last_undegraded":"2024-09-10T08:46:18.804697+0000","last_fullsized":"2024-09-10T08:46:18.804697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:36.923786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193910+0000","last_change":"2024-09-10T08:44:17.726106+0000","last_active":"2024-09-10T08:46:19.193910+0000","last_peered":"2024-09-10T08:46:19.193910+0000","last_clean":"2024-09-10T08:46:19.193910+0000","last_became_active":"2024-09-10T08:44:17.725683+0000","last_became_peered":"2024-09-10T08:44:17.725683+0000","last_unstale":"2024-09-10T08:46:19.193910+0000","last_undegraded":"2024-09-10T08:46:19.193910+0000","last_fullsized":"2024-09-10T08:46:19.193910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:23.392794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192727+0000","last_change":"2024-09-10T08:44:10.672204+0000","last_active":"2024-09-10T08:46:19.192727+0000","last_peered":"2024-09-10T08:46:19.192727+0000","last_clean":"2024-09-10T08:46:19.192727+0000","last_became_active":"2024-09-10T08:44:10.671898+0000","last_became_peered":"2024-09-10T08:44:10.671898+0000","last_unstale":"2024-09-10T08:46:19.192727+0000","last_undegraded":"2024-09-10T08:46:19.192727+0000","last_fullsized":"2024-09-10T08:46:19.192727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:03.737357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805979+0000","last_change":"2024-09-10T08:44:08.656620+0000","last_active":"2024-09-10T08:46:18.805979+0000","last_peered":"2024-09-10T08:46:18.805979+0000","last_clean":"2024-09-10T08:46:18.805979+0000","last_became_active":"2024-09-10T08:44:08.656306+0000","last_became_peered":"2024-09-10T08:44:08.656306+0000","last_unstale":"2024-09-10T08:46:18.805979+0000","last_undegraded":"2024-09-10T08:46:18.805979+0000","last_fullsized":"2024-09-10T08:46:18.805979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:57.081909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804743+0000","last_change":"2024-09-10T08:44:08.660178+0000","last_active":"2024-09-10T08:46:18.804743+0000","last_peered":"2024-09-10T08:46:18.804743+0000","last_clean":"2024-09-10T08:46:18.804743+0000","last_became_active":"2024-09-10T08:44:08.659839+0000","last_became_peered":"2024-09-10T08:44:08.659839+0000","last_unstale":"2024-09-10T08:46:18.804743+0000","last_undegraded":"2024-09-10T08:46:18.804743+0000","last_fullsized":"2024-09-10T08:46:18.804743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:17.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804848+0000","last_change":"2024-09-10T08:44:10.683683+0000","last_active":"2024-09-10T08:46:18.804848+0000","last_peered":"2024-09-10T08:46:18.804848+0000","last_clean":"2024-09-10T08:46:18.804848+0000","last_became_active":"2024-09-10T08:44:10.683337+0000","last_became_peered":"2024-09-10T08:44:10.683337+0000","last_unstale":"2024-09-10T08:46:18.804848+0000","last_undegraded":"2024-09-10T08:46:18.804848+0000","last_fullsized":"2024-09-10T08:46:18.804848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:54.863098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808044+0000","last_change":"2024-09-10T08:44:17.721602+0000","last_active":"2024-09-10T08:46:18.808044+0000","last_peered":"2024-09-10T08:46:18.808044+0000","last_clean":"2024-09-10T08:46:18.808044+0000","last_became_active":"2024-09-10T08:44:17.721194+0000","last_became_peered":"2024-09-10T08:44:17.721194+0000","last_unstale":"2024-09-10T08:46:18.808044+0000","last_undegraded":"2024-09-10T08:46:18.808044+0000","last_fullsized":"2024-09-10T08:46:18.808044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:39.298494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:39.372628+0000","last_change":"2024-09-10T08:44:08.661943+0000","last_active":"2024-09-10T08:46:39.372628+0000","last_peered":"2024-09-10T08:46:39.372628+0000","last_clean":"2024-09-10T08:46:39.372628+0000","last_became_active":"2024-09-10T08:44:08.661587+0000","last_became_peered":"2024-09-10T08:44:08.661587+0000","last_unstale":"2024-09-10T08:46:39.372628+0000","last_undegraded":"2024-09-10T08:46:39.372628+0000","last_fullsized":"2024-09-10T08:46:39.372628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:05.771641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193844+0000","last_change":"2024-09-10T08:44:10.673522+0000","last_active":"2024-09-10T08:46:19.193844+0000","last_peered":"2024-09-10T08:46:19.193844+0000","last_clean":"2024-09-10T08:46:19.193844+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:46:19.193844+0000","last_undegraded":"2024-09-10T08:46:19.193844+0000","last_fullsized":"2024-09-10T08:46:19.193844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:03.006144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808419+0000","last_change":"2024-09-10T08:44:17.721306+0000","last_active":"2024-09-10T08:46:18.808419+0000","last_peered":"2024-09-10T08:46:18.808419+0000","last_clean":"2024-09-10T08:46:18.808419+0000","last_became_active":"2024-09-10T08:44:17.720401+0000","last_became_peered":"2024-09-10T08:44:17.720401+0000","last_unstale":"2024-09-10T08:46:18.808419+0000","last_undegraded":"2024-09-10T08:46:18.808419+0000","last_fullsized":"2024-09-10T08:46:18.808419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:11.589371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193166+0000","last_change":"2024-09-10T08:44:10.678425+0000","last_active":"2024-09-10T08:46:19.193166+0000","last_peered":"2024-09-10T08:46:19.193166+0000","last_clean":"2024-09-10T08:46:19.193166+0000","last_became_active":"2024-09-10T08:44:10.677187+0000","last_became_peered":"2024-09-10T08:44:10.677187+0000","last_unstale":"2024-09-10T08:46:19.193166+0000","last_undegraded":"2024-09-10T08:46:19.193166+0000","last_fullsized":"2024-09-10T08:46:19.193166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:06.527530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808431+0000","last_change":"2024-09-10T08:44:10.674178+0000","last_active":"2024-09-10T08:46:18.808431+0000","last_peered":"2024-09-10T08:46:18.808431+0000","last_clean":"2024-09-10T08:46:18.808431+0000","last_became_active":"2024-09-10T08:44:10.673843+0000","last_became_peered":"2024-09-10T08:44:10.673843+0000","last_unstale":"2024-09-10T08:46:18.808431+0000","last_undegraded":"2024-09-10T08:46:18.808431+0000","last_fullsized":"2024-09-10T08:46:18.808431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:35.023663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.807988+0000","last_change":"2024-09-10T08:44:10.673271+0000","last_active":"2024-09-10T08:46:18.807988+0000","last_peered":"2024-09-10T08:46:18.807988+0000","last_clean":"2024-09-10T08:46:18.807988+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:46:18.807988+0000","last_undegraded":"2024-09-10T08:46:18.807988+0000","last_fullsized":"2024-09-10T08:46:18.807988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:21:26.606820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193502+0000","last_change":"2024-09-10T08:44:10.671348+0000","last_active":"2024-09-10T08:46:19.193502+0000","last_peered":"2024-09-10T08:46:19.193502+0000","last_clean":"2024-09-10T08:46:19.193502+0000","last_became_active":"2024-09-10T08:44:10.670623+0000","last_became_peered":"2024-09-10T08:44:10.670623+0000","last_unstale":"2024-09-10T08:46:19.193502+0000","last_undegraded":"2024-09-10T08:46:19.193502+0000","last_fullsized":"2024-09-10T08:46:19.193502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:06.145694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806400+0000","last_change":"2024-09-10T08:44:10.666011+0000","last_active":"2024-09-10T08:46:18.806400+0000","last_peered":"2024-09-10T08:46:18.806400+0000","last_clean":"2024-09-10T08:46:18.806400+0000","last_became_active":"2024-09-10T08:44:10.665664+0000","last_became_peered":"2024-09-10T08:44:10.665664+0000","last_unstale":"2024-09-10T08:46:18.806400+0000","last_undegraded":"2024-09-10T08:46:18.806400+0000","last_fullsized":"2024-09-10T08:46:18.806400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:31.542102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193792+0000","last_change":"2024-09-10T08:44:10.675960+0000","last_active":"2024-09-10T08:46:19.193792+0000","last_peered":"2024-09-10T08:46:19.193792+0000","last_clean":"2024-09-10T08:46:19.193792+0000","last_became_active":"2024-09-10T08:44:10.675587+0000","last_became_peered":"2024-09-10T08:44:10.675587+0000","last_unstale":"2024-09-10T08:46:19.193792+0000","last_undegraded":"2024-09-10T08:46:19.193792+0000","last_fullsized":"2024-09-10T08:46:19.193792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:54.634619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808734+0000","last_change":"2024-09-10T08:44:10.681094+0000","last_active":"2024-09-10T08:46:18.808734+0000","last_peered":"2024-09-10T08:46:18.808734+0000","last_clean":"2024-09-10T08:46:18.808734+0000","last_became_active":"2024-09-10T08:44:10.678585+0000","last_became_peered":"2024-09-10T08:44:10.678585+0000","last_unstale":"2024-09-10T08:46:18.808734+0000","last_undegraded":"2024-09-10T08:46:18.808734+0000","last_fullsized":"2024-09-10T08:46:18.808734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:14.951949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193627+0000","last_change":"2024-09-10T08:44:10.675473+0000","last_active":"2024-09-10T08:46:19.193627+0000","last_peered":"2024-09-10T08:46:19.193627+0000","last_clean":"2024-09-10T08:46:19.193627+0000","last_became_active":"2024-09-10T08:44:10.674155+0000","last_became_peered":"2024-09-10T08:44:10.674155+0000","last_unstale":"2024-09-10T08:46:19.193627+0000","last_undegraded":"2024-09-10T08:46:19.193627+0000","last_fullsized":"2024-09-10T08:46:19.193627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:55.940235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804316+0000","last_change":"2024-09-10T08:44:10.680162+0000","last_active":"2024-09-10T08:46:18.804316+0000","last_peered":"2024-09-10T08:46:18.804316+0000","last_clean":"2024-09-10T08:46:18.804316+0000","last_became_active":"2024-09-10T08:44:10.679800+0000","last_became_peered":"2024-09-10T08:44:10.679800+0000","last_unstale":"2024-09-10T08:46:18.804316+0000","last_undegraded":"2024-09-10T08:46:18.804316+0000","last_fullsized":"2024-09-10T08:46:18.804316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:12.319813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192932+0000","last_change":"2024-09-10T08:44:10.674724+0000","last_active":"2024-09-10T08:46:19.192932+0000","last_peered":"2024-09-10T08:46:19.192932+0000","last_clean":"2024-09-10T08:46:19.192932+0000","last_became_active":"2024-09-10T08:44:10.674389+0000","last_became_peered":"2024-09-10T08:44:10.674389+0000","last_unstale":"2024-09-10T08:46:19.192932+0000","last_undegraded":"2024-09-10T08:46:19.192932+0000","last_fullsized":"2024-09-10T08:46:19.192932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:52.123186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193888+0000","last_change":"2024-09-10T08:44:10.672805+0000","last_active":"2024-09-10T08:46:19.193888+0000","last_peered":"2024-09-10T08:46:19.193888+0000","last_clean":"2024-09-10T08:46:19.193888+0000","last_became_active":"2024-09-10T08:44:10.672508+0000","last_became_peered":"2024-09-10T08:44:10.672508+0000","last_unstale":"2024-09-10T08:46:19.193888+0000","last_undegraded":"2024-09-10T08:46:19.193888+0000","last_fullsized":"2024-09-10T08:46:19.193888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:48.558633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806228+0000","last_change":"2024-09-10T08:44:10.677504+0000","last_active":"2024-09-10T08:46:18.806228+0000","last_peered":"2024-09-10T08:46:18.806228+0000","last_clean":"2024-09-10T08:46:18.806228+0000","last_became_active":"2024-09-10T08:44:10.676822+0000","last_became_peered":"2024-09-10T08:44:10.676822+0000","last_unstale":"2024-09-10T08:46:18.806228+0000","last_undegraded":"2024-09-10T08:46:18.806228+0000","last_fullsized":"2024-09-10T08:46:18.806228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:05:48.293333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193457+0000","last_change":"2024-09-10T08:45:18.268208+0000","last_active":"2024-09-10T08:46:19.193457+0000","last_peered":"2024-09-10T08:46:19.193457+0000","last_clean":"2024-09-10T08:46:19.193457+0000","last_became_active":"2024-09-10T08:45:18.267700+0000","last_became_peered":"2024-09-10T08:45:18.267700+0000","last_unstale":"2024-09-10T08:46:19.193457+0000","last_undegraded":"2024-09-10T08:46:19.193457+0000","last_fullsized":"2024-09-10T08:46:19.193457+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:54.487624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808678+0000","last_change":"2024-09-10T08:44:10.670615+0000","last_active":"2024-09-10T08:46:18.808678+0000","last_peered":"2024-09-10T08:46:18.808678+0000","last_clean":"2024-09-10T08:46:18.808678+0000","last_became_active":"2024-09-10T08:44:10.670032+0000","last_became_peered":"2024-09-10T08:44:10.670032+0000","last_unstale":"2024-09-10T08:46:18.808678+0000","last_undegraded":"2024-09-10T08:46:18.808678+0000","last_fullsized":"2024-09-10T08:46:18.808678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:31.078788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803799+0000","last_change":"2024-09-10T08:44:10.672367+0000","last_active":"2024-09-10T08:46:18.803799+0000","last_peered":"2024-09-10T08:46:18.803799+0000","last_clean":"2024-09-10T08:46:18.803799+0000","last_became_active":"2024-09-10T08:44:10.672040+0000","last_became_peered":"2024-09-10T08:44:10.672040+0000","last_unstale":"2024-09-10T08:46:18.803799+0000","last_undegraded":"2024-09-10T08:46:18.803799+0000","last_fullsized":"2024-09-10T08:46:18.803799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:10.634708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194226+0000","last_change":"2024-09-10T08:44:10.676375+0000","last_active":"2024-09-10T08:46:19.194226+0000","last_peered":"2024-09-10T08:46:19.194226+0000","last_clean":"2024-09-10T08:46:19.194226+0000","last_became_active":"2024-09-10T08:44:10.676022+0000","last_became_peered":"2024-09-10T08:44:10.676022+0000","last_unstale":"2024-09-10T08:46:19.194226+0000","last_undegraded":"2024-09-10T08:46:19.194226+0000","last_fullsized":"2024-09-10T08:46:19.194226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:02.681954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193408+0000","last_change":"2024-09-10T08:44:10.673378+0000","last_active":"2024-09-10T08:46:19.193408+0000","last_peered":"2024-09-10T08:46:19.193408+0000","last_clean":"2024-09-10T08:46:19.193408+0000","last_became_active":"2024-09-10T08:44:10.673089+0000","last_became_peered":"2024-09-10T08:44:10.673089+0000","last_unstale":"2024-09-10T08:46:19.193408+0000","last_undegraded":"2024-09-10T08:46:19.193408+0000","last_fullsized":"2024-09-10T08:46:19.193408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:36:19.258651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192898+0000","last_change":"2024-09-10T08:44:10.671557+0000","last_active":"2024-09-10T08:46:19.192898+0000","last_peered":"2024-09-10T08:46:19.192898+0000","last_clean":"2024-09-10T08:46:19.192898+0000","last_became_active":"2024-09-10T08:44:10.671248+0000","last_became_peered":"2024-09-10T08:44:10.671248+0000","last_unstale":"2024-09-10T08:46:19.192898+0000","last_undegraded":"2024-09-10T08:46:19.192898+0000","last_fullsized":"2024-09-10T08:46:19.192898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:54.261525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193104+0000","last_change":"2024-09-10T08:44:10.674672+0000","last_active":"2024-09-10T08:46:19.193104+0000","last_peered":"2024-09-10T08:46:19.193104+0000","last_clean":"2024-09-10T08:46:19.193104+0000","last_became_active":"2024-09-10T08:44:10.673415+0000","last_became_peered":"2024-09-10T08:44:10.673415+0000","last_unstale":"2024-09-10T08:46:19.193104+0000","last_undegraded":"2024-09-10T08:46:19.193104+0000","last_fullsized":"2024-09-10T08:46:19.193104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:59:14.179664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806192+0000","last_change":"2024-09-10T08:44:10.675293+0000","last_active":"2024-09-10T08:46:18.806192+0000","last_peered":"2024-09-10T08:46:18.806192+0000","last_clean":"2024-09-10T08:46:18.806192+0000","last_became_active":"2024-09-10T08:44:10.674924+0000","last_became_peered":"2024-09-10T08:44:10.674924+0000","last_unstale":"2024-09-10T08:46:18.806192+0000","last_undegraded":"2024-09-10T08:46:18.806192+0000","last_fullsized":"2024-09-10T08:46:18.806192+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:56.027320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194722+0000","last_change":"2024-09-10T08:44:10.674605+0000","last_active":"2024-09-10T08:46:19.194722+0000","last_peered":"2024-09-10T08:46:19.194722+0000","last_clean":"2024-09-10T08:46:19.194722+0000","last_became_active":"2024-09-10T08:44:10.674057+0000","last_became_peered":"2024-09-10T08:44:10.674057+0000","last_unstale":"2024-09-10T08:46:19.194722+0000","last_undegraded":"2024-09-10T08:46:19.194722+0000","last_fullsized":"2024-09-10T08:46:19.194722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:09.473003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874065+0000","last_change":"2024-09-10T08:44:10.677410+0000","last_active":"2024-09-10T08:46:18.874065+0000","last_peered":"2024-09-10T08:46:18.874065+0000","last_clean":"2024-09-10T08:46:18.874065+0000","last_became_active":"2024-09-10T08:44:10.677109+0000","last_became_peered":"2024-09-10T08:44:10.677109+0000","last_unstale":"2024-09-10T08:46:18.874065+0000","last_undegraded":"2024-09-10T08:46:18.874065+0000","last_fullsized":"2024-09-10T08:46:18.874065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:45.455425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805091+0000","last_change":"2024-09-10T08:44:10.669706+0000","last_active":"2024-09-10T08:46:18.805091+0000","last_peered":"2024-09-10T08:46:18.805091+0000","last_clean":"2024-09-10T08:46:18.805091+0000","last_became_active":"2024-09-10T08:44:10.669440+0000","last_became_peered":"2024-09-10T08:44:10.669440+0000","last_unstale":"2024-09-10T08:46:18.805091+0000","last_undegraded":"2024-09-10T08:46:18.805091+0000","last_fullsized":"2024-09-10T08:46:18.805091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:31:55.610651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194758+0000","last_change":"2024-09-10T08:44:10.676482+0000","last_active":"2024-09-10T08:46:19.194758+0000","last_peered":"2024-09-10T08:46:19.194758+0000","last_clean":"2024-09-10T08:46:19.194758+0000","last_became_active":"2024-09-10T08:44:10.675983+0000","last_became_peered":"2024-09-10T08:44:10.675983+0000","last_unstale":"2024-09-10T08:46:19.194758+0000","last_undegraded":"2024-09-10T08:46:19.194758+0000","last_fullsized":"2024-09-10T08:46:19.194758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:01.269283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192699+0000","last_change":"2024-09-10T08:44:10.675681+0000","last_active":"2024-09-10T08:46:19.192699+0000","last_peered":"2024-09-10T08:46:19.192699+0000","last_clean":"2024-09-10T08:46:19.192699+0000","last_became_active":"2024-09-10T08:44:10.675357+0000","last_became_peered":"2024-09-10T08:44:10.675357+0000","last_unstale":"2024-09-10T08:46:19.192699+0000","last_undegraded":"2024-09-10T08:46:19.192699+0000","last_fullsized":"2024-09-10T08:46:19.192699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:26.138333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873522+0000","last_change":"2024-09-10T08:44:10.680819+0000","last_active":"2024-09-10T08:46:18.873522+0000","last_peered":"2024-09-10T08:46:18.873522+0000","last_clean":"2024-09-10T08:46:18.873522+0000","last_became_active":"2024-09-10T08:44:10.680491+0000","last_became_peered":"2024-09-10T08:44:10.680491+0000","last_unstale":"2024-09-10T08:46:18.873522+0000","last_undegraded":"2024-09-10T08:46:18.873522+0000","last_fullsized":"2024-09-10T08:46:18.873522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:17.044776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804621+0000","last_change":"2024-09-10T08:44:10.667376+0000","last_active":"2024-09-10T08:46:18.804621+0000","last_peered":"2024-09-10T08:46:18.804621+0000","last_clean":"2024-09-10T08:46:18.804621+0000","last_became_active":"2024-09-10T08:44:10.666648+0000","last_became_peered":"2024-09-10T08:44:10.666648+0000","last_unstale":"2024-09-10T08:46:18.804621+0000","last_undegraded":"2024-09-10T08:46:18.804621+0000","last_fullsized":"2024-09-10T08:46:18.804621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:33:12.017550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193270+0000","last_change":"2024-09-10T08:44:10.668665+0000","last_active":"2024-09-10T08:46:19.193270+0000","last_peered":"2024-09-10T08:46:19.193270+0000","last_clean":"2024-09-10T08:46:19.193270+0000","last_became_active":"2024-09-10T08:44:10.666031+0000","last_became_peered":"2024-09-10T08:44:10.666031+0000","last_unstale":"2024-09-10T08:46:19.193270+0000","last_undegraded":"2024-09-10T08:46:19.193270+0000","last_fullsized":"2024-09-10T08:46:19.193270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:39:53.991463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194458+0000","last_change":"2024-09-10T08:44:10.676495+0000","last_active":"2024-09-10T08:46:19.194458+0000","last_peered":"2024-09-10T08:46:19.194458+0000","last_clean":"2024-09-10T08:46:19.194458+0000","last_became_active":"2024-09-10T08:44:10.676087+0000","last_became_peered":"2024-09-10T08:44:10.676087+0000","last_unstale":"2024-09-10T08:46:19.194458+0000","last_undegraded":"2024-09-10T08:46:19.194458+0000","last_fullsized":"2024-09-10T08:46:19.194458+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:41.828157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193138+0000","last_change":"2024-09-10T08:44:10.672835+0000","last_active":"2024-09-10T08:46:19.193138+0000","last_peered":"2024-09-10T08:46:19.193138+0000","last_clean":"2024-09-10T08:46:19.193138+0000","last_became_active":"2024-09-10T08:44:10.672537+0000","last_became_peered":"2024-09-10T08:44:10.672537+0000","last_unstale":"2024-09-10T08:46:19.193138+0000","last_undegraded":"2024-09-10T08:46:19.193138+0000","last_fullsized":"2024-09-10T08:46:19.193138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:06.121460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193568+0000","last_change":"2024-09-10T08:44:10.675968+0000","last_active":"2024-09-10T08:46:19.193568+0000","last_peered":"2024-09-10T08:46:19.193568+0000","last_clean":"2024-09-10T08:46:19.193568+0000","last_became_active":"2024-09-10T08:44:10.675628+0000","last_became_peered":"2024-09-10T08:44:10.675628+0000","last_unstale":"2024-09-10T08:46:19.193568+0000","last_undegraded":"2024-09-10T08:46:19.193568+0000","last_fullsized":"2024-09-10T08:46:19.193568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:40.261336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194091+0000","last_change":"2024-09-10T08:44:10.673151+0000","last_active":"2024-09-10T08:46:19.194091+0000","last_peered":"2024-09-10T08:46:19.194091+0000","last_clean":"2024-09-10T08:46:19.194091+0000","last_became_active":"2024-09-10T08:44:10.672833+0000","last_became_peered":"2024-09-10T08:44:10.672833+0000","last_unstale":"2024-09-10T08:46:19.194091+0000","last_undegraded":"2024-09-10T08:46:19.194091+0000","last_fullsized":"2024-09-10T08:46:19.194091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:38.556222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77478,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1124,"num_read_kb":645,"num_write":276,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":232434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514541,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":523370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743473,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28740,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716412,"statfs":{"total":95995035648,"available":95965605888,"internally_reserved":0,"allocated":1757184,"data_stored":1178194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906998,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":526367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135931,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":522649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364864,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1120240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528390,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":518270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757322,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36988,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708164,"statfs":{"total":95995035648,"available":95957159936,"internally_reserved":0,"allocated":1683456,"data_stored":1114832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":522344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":517483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":589091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1130496,"data_stored":587842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":520260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:46:46.820 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:46:47.252 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:46:47.304 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:47.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:47 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T08:46:47.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:47 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:47.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:47 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:46:47.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:47 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:47.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:47 smithi191 ceph-mon[35314]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:47.551 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:47 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T08:46:47.551 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:47 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:47.551 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:47 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:46:47.551 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:47 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:47.551 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:47 smithi172 ceph-mon[30854]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:47.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:47 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T08:46:47.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:47 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:47.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:47 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:46:47.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:47 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:46:47.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:47 smithi174 ceph-mon[34068]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:47.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.782+0000 7f0937c39640 1 -- 172.21.15.172:0/2462520904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0930101420 msgr2=0x7f093010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:47.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.782+0000 7f0937c39640 1 --2- 172.21.15.172:0/2462520904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0930101420 0x7f093010d950 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0928009cd0 tx=0x7f092802a8e0 comp rx=0 tx=0).stop 2024-09-10T08:46:47.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.782+0000 7f0937c39640 1 -- 172.21.15.172:0/2462520904 shutdown_connections 2024-09-10T08:46:47.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.782+0000 7f0937c39640 1 --2- 172.21.15.172:0/2462520904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0930101420 0x7f093010d950 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:47.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.782+0000 7f0937c39640 1 --2- 172.21.15.172:0/2462520904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0930100a80 0x7f0930100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:47.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.782+0000 7f0937c39640 1 --2- 172.21.15.172:0/2462520904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0930106a80 0x7f0930106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:47.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.782+0000 7f0937c39640 1 -- 172.21.15.172:0/2462520904 >> 172.21.15.172:0/2462520904 conn(0x7f09300fc7a0 msgr2=0x7f09300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:47.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.782+0000 7f0937c39640 1 -- 172.21.15.172:0/2462520904 shutdown_connections 2024-09-10T08:46:47.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.782+0000 7f0937c39640 1 -- 172.21.15.172:0/2462520904 wait complete. 2024-09-10T08:46:47.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.782+0000 7f0937c39640 1 Processor -- start 2024-09-10T08:46:47.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.782+0000 7f0937c39640 1 -- start start 2024-09-10T08:46:47.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f0937c39640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0930100a80 0x7f0930102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:47.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f0937c39640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0930101420 0x7f0930102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:47.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f0937c39640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0930106a80 0x7f09301030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:47.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f0937c39640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0930110d30 con 0x7f0930101420 2024-09-10T08:46:47.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f0937c39640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0930110bb0 con 0x7f0930106a80 2024-09-10T08:46:47.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f0937c39640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0930110eb0 con 0x7f0930100a80 2024-09-10T08:46:47.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f09351d4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0930101420 0x7f0930102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:47.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f09351d4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0930101420 0x7f0930102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57762/0 (socket says 172.21.15.172:57762) 2024-09-10T08:46:47.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f09351d4640 1 -- 172.21.15.172:0/1219156198 learned_addr learned my addr 172.21.15.172:0/1219156198 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:47.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f09359d5640 1 --2- 172.21.15.172:0/1219156198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0930100a80 0x7f0930102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:47.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f09361d6640 1 --2- 172.21.15.172:0/1219156198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0930106a80 0x7f09301030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:47.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f09351d4640 1 -- 172.21.15.172:0/1219156198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0930100a80 msgr2=0x7f0930102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:47.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f09351d4640 1 --2- 172.21.15.172:0/1219156198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0930100a80 0x7f0930102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:47.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f09351d4640 1 -- 172.21.15.172:0/1219156198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0930106a80 msgr2=0x7f09301030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:47.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f09351d4640 1 --2- 172.21.15.172:0/1219156198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0930106a80 0x7f09301030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:47.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f09351d4640 1 -- 172.21.15.172:0/1219156198 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09301a6800 con 0x7f0930101420 2024-09-10T08:46:47.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f09351d4640 1 --2- 172.21.15.172:0/1219156198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0930101420 0x7f0930102b90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f092c00d950 tx=0x7f092c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:47.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.783+0000 7f0922ffd640 1 -- 172.21.15.172:0/1219156198 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c017070 con 0x7f0930101420 2024-09-10T08:46:47.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.784+0000 7f0937c39640 1 -- 172.21.15.172:0/1219156198 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09301a6af0 con 0x7f0930101420 2024-09-10T08:46:47.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.784+0000 7f0922ffd640 1 -- 172.21.15.172:0/1219156198 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f092c009c30 con 0x7f0930101420 2024-09-10T08:46:47.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.784+0000 7f0922ffd640 1 -- 172.21.15.172:0/1219156198 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c00f620 con 0x7f0930101420 2024-09-10T08:46:47.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.784+0000 7f0937c39640 1 -- 172.21.15.172:0/1219156198 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09301a7080 con 0x7f0930101420 2024-09-10T08:46:47.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.785+0000 7f0937c39640 1 -- 172.21.15.172:0/1219156198 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0930100ee0 con 0x7f0930101420 2024-09-10T08:46:47.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.786+0000 7f0922ffd640 1 -- 172.21.15.172:0/1219156198 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f092c0109a0 con 0x7f0930101420 2024-09-10T08:46:47.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.786+0000 7f0922ffd640 1 --2- 172.21.15.172:0/1219156198 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f091c0a2b80 0x7f091c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:47.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.786+0000 7f09359d5640 1 --2- 172.21.15.172:0/1219156198 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f091c0a2b80 0x7f091c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:47.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.787+0000 7f0922ffd640 1 -- 172.21.15.172:0/1219156198 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f092c0ceec0 con 0x7f0930101420 2024-09-10T08:46:47.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.787+0000 7f09359d5640 1 --2- 172.21.15.172:0/1219156198 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f091c0a2b80 0x7f091c0a5040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f09240058c0 tx=0x7f092400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:47.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.789+0000 7f0922ffd640 1 -- 172.21.15.172:0/1219156198 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f092c009de0 con 0x7f0930101420 2024-09-10T08:46:47.853 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:47.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.971+0000 7f0937c39640 1 -- 172.21.15.172:0/1219156198 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f093006b020 con 0x7f091c0a2b80 2024-09-10T08:46:47.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.980+0000 7f0922ffd640 1 -- 172.21.15.172:0/1219156198 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340527 (secure 0 0 0) 0x7f093006b020 con 0x7f091c0a2b80 2024-09-10T08:46:47.983 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:47.985 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-09-10T08:46:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.983+0000 7f0937c39640 1 -- 172.21.15.172:0/1219156198 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f091c0a2b80 msgr2=0x7f091c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.983+0000 7f0937c39640 1 --2- 172.21.15.172:0/1219156198 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f091c0a2b80 0x7f091c0a5040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f09240058c0 tx=0x7f092400a2d0 comp rx=0 tx=0).stop 2024-09-10T08:46:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.983+0000 7f0937c39640 1 -- 172.21.15.172:0/1219156198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0930101420 msgr2=0x7f0930102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.983+0000 7f0937c39640 1 --2- 172.21.15.172:0/1219156198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0930101420 0x7f0930102b90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f092c00d950 tx=0x7f092c00de10 comp rx=0 tx=0).stop 2024-09-10T08:46:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.983+0000 7f0937c39640 1 -- 172.21.15.172:0/1219156198 shutdown_connections 2024-09-10T08:46:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.983+0000 7f0937c39640 1 --2- 172.21.15.172:0/1219156198 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f091c0a2b80 0x7f091c0a5040 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.983+0000 7f0937c39640 1 --2- 172.21.15.172:0/1219156198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0930106a80 0x7f09301030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.983+0000 7f0937c39640 1 --2- 172.21.15.172:0/1219156198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0930101420 0x7f0930102b90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.983+0000 7f0937c39640 1 --2- 172.21.15.172:0/1219156198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0930100a80 0x7f0930102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.983+0000 7f0937c39640 1 -- 172.21.15.172:0/1219156198 >> 172.21.15.172:0/1219156198 conn(0x7f09300fc7a0 msgr2=0x7f093006a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:47.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.983+0000 7f0937c39640 1 -- 172.21.15.172:0/1219156198 shutdown_connections 2024-09-10T08:46:47.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:47.983+0000 7f0937c39640 1 -- 172.21.15.172:0/1219156198 wait complete. 2024-09-10T08:46:48.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:48 smithi191 ceph-mon[35314]: pgmap v367: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-10T08:46:48.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:48 smithi172 ceph-mon[30854]: pgmap v367: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-10T08:46:48.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:48 smithi174 ceph-mon[34068]: pgmap v367: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-10T08:46:48.880 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-09-10T08:46:47.166273+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668080,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1286,"num_read_kb":775,"num_write":459,"num_write_kb":3107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":874,"ondisk_log_size":874,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376268,"kb_used_data":14540,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565556,"statfs":{"total":1151940427776,"available":1151555129344,"internally_reserved":0,"allocated":14888960,"data_stored":8242080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6294,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":142,"num_read_kb":81,"num_write":16,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003260"},"pg_stats":[{"pgid":"2.1f","version":"92'14","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:45.379095+0000","last_change":"2024-09-10T08:44:08.657737+0000","last_active":"2024-09-10T08:46:45.379095+0000","last_peered":"2024-09-10T08:46:45.379095+0000","last_clean":"2024-09-10T08:46:45.379095+0000","last_became_active":"2024-09-10T08:44:08.657473+0000","last_became_peered":"2024-09-10T08:44:08.657473+0000","last_unstale":"2024-09-10T08:46:45.379095+0000","last_undegraded":"2024-09-10T08:46:45.379095+0000","last_fullsized":"2024-09-10T08:46:45.379095+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:40.757343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193599+0000","last_change":"2024-09-10T08:44:10.675090+0000","last_active":"2024-09-10T08:46:19.193599+0000","last_peered":"2024-09-10T08:46:19.193599+0000","last_clean":"2024-09-10T08:46:19.193599+0000","last_became_active":"2024-09-10T08:44:10.674612+0000","last_became_peered":"2024-09-10T08:44:10.674612+0000","last_unstale":"2024-09-10T08:46:19.193599+0000","last_undegraded":"2024-09-10T08:46:19.193599+0000","last_fullsized":"2024-09-10T08:46:19.193599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:46.821080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.807967+0000","last_change":"2024-09-10T08:44:17.721559+0000","last_active":"2024-09-10T08:46:18.807967+0000","last_peered":"2024-09-10T08:46:18.807967+0000","last_clean":"2024-09-10T08:46:18.807967+0000","last_became_active":"2024-09-10T08:44:17.721182+0000","last_became_peered":"2024-09-10T08:44:17.721182+0000","last_unstale":"2024-09-10T08:46:18.807967+0000","last_undegraded":"2024-09-10T08:46:18.807967+0000","last_fullsized":"2024-09-10T08:46:18.807967+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:18.791489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193663+0000","last_change":"2024-09-10T08:44:09.049171+0000","last_active":"2024-09-10T08:46:19.193663+0000","last_peered":"2024-09-10T08:46:19.193663+0000","last_clean":"2024-09-10T08:46:19.193663+0000","last_became_active":"2024-09-10T08:44:09.048806+0000","last_became_peered":"2024-09-10T08:44:09.048806+0000","last_unstale":"2024-09-10T08:46:19.193663+0000","last_undegraded":"2024-09-10T08:46:19.193663+0000","last_fullsized":"2024-09-10T08:46:19.193663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:21.023460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193866+0000","last_change":"2024-09-10T08:44:10.677569+0000","last_active":"2024-09-10T08:46:19.193866+0000","last_peered":"2024-09-10T08:46:19.193866+0000","last_clean":"2024-09-10T08:46:19.193866+0000","last_became_active":"2024-09-10T08:44:10.677048+0000","last_became_peered":"2024-09-10T08:44:10.677048+0000","last_unstale":"2024-09-10T08:46:19.193866+0000","last_undegraded":"2024-09-10T08:46:19.193866+0000","last_fullsized":"2024-09-10T08:46:19.193866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:35.523063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193389+0000","last_change":"2024-09-10T08:44:17.720138+0000","last_active":"2024-09-10T08:46:19.193389+0000","last_peered":"2024-09-10T08:46:19.193389+0000","last_clean":"2024-09-10T08:46:19.193389+0000","last_became_active":"2024-09-10T08:44:17.719344+0000","last_became_peered":"2024-09-10T08:44:17.719344+0000","last_unstale":"2024-09-10T08:46:19.193389+0000","last_undegraded":"2024-09-10T08:46:19.193389+0000","last_fullsized":"2024-09-10T08:46:19.193389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:11.758058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194115+0000","last_change":"2024-09-10T08:44:08.660099+0000","last_active":"2024-09-10T08:46:19.194115+0000","last_peered":"2024-09-10T08:46:19.194115+0000","last_clean":"2024-09-10T08:46:19.194115+0000","last_became_active":"2024-09-10T08:44:08.659745+0000","last_became_peered":"2024-09-10T08:44:08.659745+0000","last_unstale":"2024-09-10T08:46:19.194115+0000","last_undegraded":"2024-09-10T08:46:19.194115+0000","last_fullsized":"2024-09-10T08:46:19.194115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:43.960717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193791+0000","last_change":"2024-09-10T08:44:10.675583+0000","last_active":"2024-09-10T08:46:19.193791+0000","last_peered":"2024-09-10T08:46:19.193791+0000","last_clean":"2024-09-10T08:46:19.193791+0000","last_became_active":"2024-09-10T08:44:10.675296+0000","last_became_peered":"2024-09-10T08:44:10.675296+0000","last_unstale":"2024-09-10T08:46:19.193791+0000","last_undegraded":"2024-09-10T08:46:19.193791+0000","last_fullsized":"2024-09-10T08:46:19.193791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:31:51.147862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194363+0000","last_change":"2024-09-10T08:44:17.720104+0000","last_active":"2024-09-10T08:46:19.194363+0000","last_peered":"2024-09-10T08:46:19.194363+0000","last_clean":"2024-09-10T08:46:19.194363+0000","last_became_active":"2024-09-10T08:44:17.719607+0000","last_became_peered":"2024-09-10T08:44:17.719607+0000","last_unstale":"2024-09-10T08:46:19.194363+0000","last_undegraded":"2024-09-10T08:46:19.194363+0000","last_fullsized":"2024-09-10T08:46:19.194363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:52.976717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193719+0000","last_change":"2024-09-10T08:44:08.658588+0000","last_active":"2024-09-10T08:46:19.193719+0000","last_peered":"2024-09-10T08:46:19.193719+0000","last_clean":"2024-09-10T08:46:19.193719+0000","last_became_active":"2024-09-10T08:44:08.658053+0000","last_became_peered":"2024-09-10T08:44:08.658053+0000","last_unstale":"2024-09-10T08:46:19.193719+0000","last_undegraded":"2024-09-10T08:46:19.193719+0000","last_fullsized":"2024-09-10T08:46:19.193719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:26.744943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804775+0000","last_change":"2024-09-10T08:44:10.681978+0000","last_active":"2024-09-10T08:46:18.804775+0000","last_peered":"2024-09-10T08:46:18.804775+0000","last_clean":"2024-09-10T08:46:18.804775+0000","last_became_active":"2024-09-10T08:44:10.681600+0000","last_became_peered":"2024-09-10T08:44:10.681600+0000","last_unstale":"2024-09-10T08:46:18.804775+0000","last_undegraded":"2024-09-10T08:46:18.804775+0000","last_fullsized":"2024-09-10T08:46:18.804775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:37.190576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193813+0000","last_change":"2024-09-10T08:44:17.719784+0000","last_active":"2024-09-10T08:46:19.193813+0000","last_peered":"2024-09-10T08:46:19.193813+0000","last_clean":"2024-09-10T08:46:19.193813+0000","last_became_active":"2024-09-10T08:44:17.719182+0000","last_became_peered":"2024-09-10T08:44:17.719182+0000","last_unstale":"2024-09-10T08:46:19.193813+0000","last_undegraded":"2024-09-10T08:46:19.193813+0000","last_fullsized":"2024-09-10T08:46:19.193813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:15.237772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873709+0000","last_change":"2024-09-10T08:44:10.679069+0000","last_active":"2024-09-10T08:46:18.873709+0000","last_peered":"2024-09-10T08:46:18.873709+0000","last_clean":"2024-09-10T08:46:18.873709+0000","last_became_active":"2024-09-10T08:44:10.678597+0000","last_became_peered":"2024-09-10T08:44:10.678597+0000","last_unstale":"2024-09-10T08:46:18.873709+0000","last_undegraded":"2024-09-10T08:46:18.873709+0000","last_fullsized":"2024-09-10T08:46:18.873709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:52.693906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194153+0000","last_change":"2024-09-10T08:44:08.660886+0000","last_active":"2024-09-10T08:46:19.194153+0000","last_peered":"2024-09-10T08:46:19.194153+0000","last_clean":"2024-09-10T08:46:19.194153+0000","last_became_active":"2024-09-10T08:44:08.660478+0000","last_became_peered":"2024-09-10T08:44:08.660478+0000","last_unstale":"2024-09-10T08:46:19.194153+0000","last_undegraded":"2024-09-10T08:46:19.194153+0000","last_fullsized":"2024-09-10T08:46:19.194153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:41.785531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194323+0000","last_change":"2024-09-10T08:44:17.761929+0000","last_active":"2024-09-10T08:46:19.194323+0000","last_peered":"2024-09-10T08:46:19.194323+0000","last_clean":"2024-09-10T08:46:19.194323+0000","last_became_active":"2024-09-10T08:44:17.761440+0000","last_became_peered":"2024-09-10T08:44:17.761440+0000","last_unstale":"2024-09-10T08:46:19.194323+0000","last_undegraded":"2024-09-10T08:46:19.194323+0000","last_fullsized":"2024-09-10T08:46:19.194323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:37.817759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192895+0000","last_change":"2024-09-10T08:44:10.674071+0000","last_active":"2024-09-10T08:46:19.192895+0000","last_peered":"2024-09-10T08:46:19.192895+0000","last_clean":"2024-09-10T08:46:19.192895+0000","last_became_active":"2024-09-10T08:44:10.673770+0000","last_became_peered":"2024-09-10T08:44:10.673770+0000","last_unstale":"2024-09-10T08:46:19.192895+0000","last_undegraded":"2024-09-10T08:46:19.192895+0000","last_fullsized":"2024-09-10T08:46:19.192895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:44:19.944557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.231938+0000","last_change":"2024-09-10T08:45:18.264060+0000","last_active":"2024-09-10T08:46:34.231938+0000","last_peered":"2024-09-10T08:46:34.231938+0000","last_clean":"2024-09-10T08:46:34.231938+0000","last_became_active":"2024-09-10T08:45:18.255983+0000","last_became_peered":"2024-09-10T08:45:18.255983+0000","last_unstale":"2024-09-10T08:46:34.231938+0000","last_undegraded":"2024-09-10T08:46:34.231938+0000","last_fullsized":"2024-09-10T08:46:34.231938+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:10.810491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194248+0000","last_change":"2024-09-10T08:44:17.722877+0000","last_active":"2024-09-10T08:46:19.194248+0000","last_peered":"2024-09-10T08:46:19.194248+0000","last_clean":"2024-09-10T08:46:19.194248+0000","last_became_active":"2024-09-10T08:44:17.722046+0000","last_became_peered":"2024-09-10T08:44:17.722046+0000","last_unstale":"2024-09-10T08:46:19.194248+0000","last_undegraded":"2024-09-10T08:46:19.194248+0000","last_fullsized":"2024-09-10T08:46:19.194248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:32.675598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193398+0000","last_change":"2024-09-10T08:44:10.671500+0000","last_active":"2024-09-10T08:46:19.193398+0000","last_peered":"2024-09-10T08:46:19.193398+0000","last_clean":"2024-09-10T08:46:19.193398+0000","last_became_active":"2024-09-10T08:44:10.671039+0000","last_became_peered":"2024-09-10T08:44:10.671039+0000","last_unstale":"2024-09-10T08:46:19.193398+0000","last_undegraded":"2024-09-10T08:46:19.193398+0000","last_fullsized":"2024-09-10T08:46:19.193398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:12.977373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193090+0000","last_change":"2024-09-10T08:44:08.655467+0000","last_active":"2024-09-10T08:46:19.193090+0000","last_peered":"2024-09-10T08:46:19.193090+0000","last_clean":"2024-09-10T08:46:19.193090+0000","last_became_active":"2024-09-10T08:44:08.654991+0000","last_became_peered":"2024-09-10T08:44:08.654991+0000","last_unstale":"2024-09-10T08:46:19.193090+0000","last_undegraded":"2024-09-10T08:46:19.193090+0000","last_fullsized":"2024-09-10T08:46:19.193090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:33.124360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803728+0000","last_change":"2024-09-10T08:44:17.724250+0000","last_active":"2024-09-10T08:46:18.803728+0000","last_peered":"2024-09-10T08:46:18.803728+0000","last_clean":"2024-09-10T08:46:18.803728+0000","last_became_active":"2024-09-10T08:44:17.723733+0000","last_became_peered":"2024-09-10T08:44:17.723733+0000","last_unstale":"2024-09-10T08:46:18.803728+0000","last_undegraded":"2024-09-10T08:46:18.803728+0000","last_fullsized":"2024-09-10T08:46:18.803728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:40.098666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193144+0000","last_change":"2024-09-10T08:44:10.664159+0000","last_active":"2024-09-10T08:46:19.193144+0000","last_peered":"2024-09-10T08:46:19.193144+0000","last_clean":"2024-09-10T08:46:19.193144+0000","last_became_active":"2024-09-10T08:44:10.663674+0000","last_became_peered":"2024-09-10T08:44:10.663674+0000","last_unstale":"2024-09-10T08:46:19.193144+0000","last_undegraded":"2024-09-10T08:46:19.193144+0000","last_fullsized":"2024-09-10T08:46:19.193144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:24.234525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193535+0000","last_change":"2024-09-10T08:45:18.257226+0000","last_active":"2024-09-10T08:46:19.193535+0000","last_peered":"2024-09-10T08:46:19.193535+0000","last_clean":"2024-09-10T08:46:19.193535+0000","last_became_active":"2024-09-10T08:45:18.254113+0000","last_became_peered":"2024-09-10T08:45:18.254113+0000","last_unstale":"2024-09-10T08:46:19.193535+0000","last_undegraded":"2024-09-10T08:46:19.193535+0000","last_fullsized":"2024-09-10T08:46:19.193535+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:19.370105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194467+0000","last_change":"2024-09-10T08:44:17.761639+0000","last_active":"2024-09-10T08:46:19.194467+0000","last_peered":"2024-09-10T08:46:19.194467+0000","last_clean":"2024-09-10T08:46:19.194467+0000","last_became_active":"2024-09-10T08:44:17.761214+0000","last_became_peered":"2024-09-10T08:44:17.761214+0000","last_unstale":"2024-09-10T08:46:19.194467+0000","last_undegraded":"2024-09-10T08:46:19.194467+0000","last_fullsized":"2024-09-10T08:46:19.194467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:43.793195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193666+0000","last_change":"2024-09-10T08:45:18.258661+0000","last_active":"2024-09-10T08:46:19.193666+0000","last_peered":"2024-09-10T08:46:19.193666+0000","last_clean":"2024-09-10T08:46:19.193666+0000","last_became_active":"2024-09-10T08:45:18.258225+0000","last_became_peered":"2024-09-10T08:45:18.258225+0000","last_unstale":"2024-09-10T08:46:19.193666+0000","last_undegraded":"2024-09-10T08:46:19.193666+0000","last_fullsized":"2024-09-10T08:46:19.193666+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:28:37.184766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194660+0000","last_change":"2024-09-10T08:44:09.036632+0000","last_active":"2024-09-10T08:46:19.194660+0000","last_peered":"2024-09-10T08:46:19.194660+0000","last_clean":"2024-09-10T08:46:19.194660+0000","last_became_active":"2024-09-10T08:44:09.036254+0000","last_became_peered":"2024-09-10T08:44:09.036254+0000","last_unstale":"2024-09-10T08:46:19.194660+0000","last_undegraded":"2024-09-10T08:46:19.194660+0000","last_fullsized":"2024-09-10T08:46:19.194660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:59:40.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808140+0000","last_change":"2024-09-10T08:44:17.716215+0000","last_active":"2024-09-10T08:46:18.808140+0000","last_peered":"2024-09-10T08:46:18.808140+0000","last_clean":"2024-09-10T08:46:18.808140+0000","last_became_active":"2024-09-10T08:44:17.715809+0000","last_became_peered":"2024-09-10T08:44:17.715809+0000","last_unstale":"2024-09-10T08:46:18.808140+0000","last_undegraded":"2024-09-10T08:46:18.808140+0000","last_fullsized":"2024-09-10T08:46:18.808140+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:52.896401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193713+0000","last_change":"2024-09-10T08:44:10.677871+0000","last_active":"2024-09-10T08:46:19.193713+0000","last_peered":"2024-09-10T08:46:19.193713+0000","last_clean":"2024-09-10T08:46:19.193713+0000","last_became_active":"2024-09-10T08:44:10.677101+0000","last_became_peered":"2024-09-10T08:44:10.677101+0000","last_unstale":"2024-09-10T08:46:19.193713+0000","last_undegraded":"2024-09-10T08:46:19.193713+0000","last_fullsized":"2024-09-10T08:46:19.193713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:46.137249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'18","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.704451+0000","last_change":"2024-09-10T08:44:08.654428+0000","last_active":"2024-09-10T08:46:34.704451+0000","last_peered":"2024-09-10T08:46:34.704451+0000","last_clean":"2024-09-10T08:46:34.704451+0000","last_became_active":"2024-09-10T08:44:08.654119+0000","last_became_peered":"2024-09-10T08:44:08.654119+0000","last_unstale":"2024-09-10T08:46:34.704451+0000","last_undegraded":"2024-09-10T08:46:34.704451+0000","last_fullsized":"2024-09-10T08:46:34.704451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:56.787721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192538+0000","last_change":"2024-09-10T08:44:17.760828+0000","last_active":"2024-09-10T08:46:19.192538+0000","last_peered":"2024-09-10T08:46:19.192538+0000","last_clean":"2024-09-10T08:46:19.192538+0000","last_became_active":"2024-09-10T08:44:17.760367+0000","last_became_peered":"2024-09-10T08:44:17.760367+0000","last_unstale":"2024-09-10T08:46:19.192538+0000","last_undegraded":"2024-09-10T08:46:19.192538+0000","last_fullsized":"2024-09-10T08:46:19.192538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:11.130480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806446+0000","last_change":"2024-09-10T08:44:10.670179+0000","last_active":"2024-09-10T08:46:18.806446+0000","last_peered":"2024-09-10T08:46:18.806446+0000","last_clean":"2024-09-10T08:46:18.806446+0000","last_became_active":"2024-09-10T08:44:10.669366+0000","last_became_peered":"2024-09-10T08:44:10.669366+0000","last_unstale":"2024-09-10T08:46:18.806446+0000","last_undegraded":"2024-09-10T08:46:18.806446+0000","last_fullsized":"2024-09-10T08:46:18.806446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:41.916365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194210+0000","last_change":"2024-09-10T08:44:09.050441+0000","last_active":"2024-09-10T08:46:19.194210+0000","last_peered":"2024-09-10T08:46:19.194210+0000","last_clean":"2024-09-10T08:46:19.194210+0000","last_became_active":"2024-09-10T08:44:09.050070+0000","last_became_peered":"2024-09-10T08:44:09.050070+0000","last_unstale":"2024-09-10T08:46:19.194210+0000","last_undegraded":"2024-09-10T08:46:19.194210+0000","last_fullsized":"2024-09-10T08:46:19.194210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:13.540021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193573+0000","last_change":"2024-09-10T08:44:17.720098+0000","last_active":"2024-09-10T08:46:19.193573+0000","last_peered":"2024-09-10T08:46:19.193573+0000","last_clean":"2024-09-10T08:46:19.193573+0000","last_became_active":"2024-09-10T08:44:17.719627+0000","last_became_peered":"2024-09-10T08:44:17.719627+0000","last_unstale":"2024-09-10T08:46:19.193573+0000","last_undegraded":"2024-09-10T08:46:19.193573+0000","last_fullsized":"2024-09-10T08:46:19.193573+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:50.020919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804677+0000","last_change":"2024-09-10T08:44:10.669630+0000","last_active":"2024-09-10T08:46:18.804677+0000","last_peered":"2024-09-10T08:46:18.804677+0000","last_clean":"2024-09-10T08:46:18.804677+0000","last_became_active":"2024-09-10T08:44:10.669307+0000","last_became_peered":"2024-09-10T08:44:10.669307+0000","last_unstale":"2024-09-10T08:46:18.804677+0000","last_undegraded":"2024-09-10T08:46:18.804677+0000","last_fullsized":"2024-09-10T08:46:18.804677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:06.320046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803568+0000","last_change":"2024-09-10T08:44:09.050715+0000","last_active":"2024-09-10T08:46:18.803568+0000","last_peered":"2024-09-10T08:46:18.803568+0000","last_clean":"2024-09-10T08:46:18.803568+0000","last_became_active":"2024-09-10T08:44:09.050347+0000","last_became_peered":"2024-09-10T08:44:09.050347+0000","last_unstale":"2024-09-10T08:46:18.803568+0000","last_undegraded":"2024-09-10T08:46:18.803568+0000","last_fullsized":"2024-09-10T08:46:18.803568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:18.689652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194038+0000","last_change":"2024-09-10T08:45:18.260444+0000","last_active":"2024-09-10T08:46:19.194038+0000","last_peered":"2024-09-10T08:46:19.194038+0000","last_clean":"2024-09-10T08:46:19.194038+0000","last_became_active":"2024-09-10T08:45:18.259949+0000","last_became_peered":"2024-09-10T08:45:18.259949+0000","last_unstale":"2024-09-10T08:46:19.194038+0000","last_undegraded":"2024-09-10T08:46:19.194038+0000","last_fullsized":"2024-09-10T08:46:19.194038+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:56.391253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804630+0000","last_change":"2024-09-10T08:44:10.670881+0000","last_active":"2024-09-10T08:46:18.804630+0000","last_peered":"2024-09-10T08:46:18.804630+0000","last_clean":"2024-09-10T08:46:18.804630+0000","last_became_active":"2024-09-10T08:44:10.669791+0000","last_became_peered":"2024-09-10T08:44:10.669791+0000","last_unstale":"2024-09-10T08:46:18.804630+0000","last_undegraded":"2024-09-10T08:46:18.804630+0000","last_fullsized":"2024-09-10T08:46:18.804630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:33.644205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'6","reported_seq":156,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:42.375958+0000","last_change":"2024-09-10T08:44:08.657938+0000","last_active":"2024-09-10T08:46:42.375958+0000","last_peered":"2024-09-10T08:46:42.375958+0000","last_clean":"2024-09-10T08:46:42.375958+0000","last_became_active":"2024-09-10T08:44:08.657526+0000","last_became_peered":"2024-09-10T08:44:08.657526+0000","last_unstale":"2024-09-10T08:46:42.375958+0000","last_undegraded":"2024-09-10T08:46:42.375958+0000","last_fullsized":"2024-09-10T08:46:42.375958+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:15.673561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806147+0000","last_change":"2024-09-10T08:44:17.721619+0000","last_active":"2024-09-10T08:46:18.806147+0000","last_peered":"2024-09-10T08:46:18.806147+0000","last_clean":"2024-09-10T08:46:18.806147+0000","last_became_active":"2024-09-10T08:44:17.721175+0000","last_became_peered":"2024-09-10T08:44:17.721175+0000","last_unstale":"2024-09-10T08:46:18.806147+0000","last_undegraded":"2024-09-10T08:46:18.806147+0000","last_fullsized":"2024-09-10T08:46:18.806147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:52.244120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804320+0000","last_change":"2024-09-10T08:44:10.676164+0000","last_active":"2024-09-10T08:46:18.804320+0000","last_peered":"2024-09-10T08:46:18.804320+0000","last_clean":"2024-09-10T08:46:18.804320+0000","last_became_active":"2024-09-10T08:44:10.675820+0000","last_became_peered":"2024-09-10T08:44:10.675820+0000","last_unstale":"2024-09-10T08:46:18.804320+0000","last_undegraded":"2024-09-10T08:46:18.804320+0000","last_fullsized":"2024-09-10T08:46:18.804320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:24.477584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":157,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:42.375891+0000","last_change":"2024-09-10T08:44:09.049124+0000","last_active":"2024-09-10T08:46:42.375891+0000","last_peered":"2024-09-10T08:46:42.375891+0000","last_clean":"2024-09-10T08:46:42.375891+0000","last_became_active":"2024-09-10T08:44:09.048799+0000","last_became_peered":"2024-09-10T08:44:09.048799+0000","last_unstale":"2024-09-10T08:46:42.375891+0000","last_undegraded":"2024-09-10T08:46:42.375891+0000","last_fullsized":"2024-09-10T08:46:42.375891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:43:39.849727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806891+0000","last_change":"2024-09-10T08:44:17.761130+0000","last_active":"2024-09-10T08:46:18.806891+0000","last_peered":"2024-09-10T08:46:18.806891+0000","last_clean":"2024-09-10T08:46:18.806891+0000","last_became_active":"2024-09-10T08:44:17.760753+0000","last_became_peered":"2024-09-10T08:44:17.760753+0000","last_unstale":"2024-09-10T08:46:18.806891+0000","last_undegraded":"2024-09-10T08:46:18.806891+0000","last_fullsized":"2024-09-10T08:46:18.806891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:42.971962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.809258+0000","last_change":"2024-09-10T08:46:18.809258+0000","last_active":"2024-09-10T08:46:18.809258+0000","last_peered":"2024-09-10T08:46:18.809258+0000","last_clean":"2024-09-10T08:46:18.809258+0000","last_became_active":"2024-09-10T08:46:18.808908+0000","last_became_peered":"2024-09-10T08:46:18.808908+0000","last_unstale":"2024-09-10T08:46:18.809258+0000","last_undegraded":"2024-09-10T08:46:18.809258+0000","last_fullsized":"2024-09-10T08:46:18.809258+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:10.305672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873500+0000","last_change":"2024-09-10T08:44:08.658862+0000","last_active":"2024-09-10T08:46:18.873500+0000","last_peered":"2024-09-10T08:46:18.873500+0000","last_clean":"2024-09-10T08:46:18.873500+0000","last_became_active":"2024-09-10T08:44:08.657880+0000","last_became_peered":"2024-09-10T08:44:08.657880+0000","last_unstale":"2024-09-10T08:46:18.873500+0000","last_undegraded":"2024-09-10T08:46:18.873500+0000","last_fullsized":"2024-09-10T08:46:18.873500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:34.224025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194185+0000","last_change":"2024-09-10T08:45:18.268446+0000","last_active":"2024-09-10T08:46:19.194185+0000","last_peered":"2024-09-10T08:46:19.194185+0000","last_clean":"2024-09-10T08:46:19.194185+0000","last_became_active":"2024-09-10T08:45:18.267988+0000","last_became_peered":"2024-09-10T08:45:18.267988+0000","last_unstale":"2024-09-10T08:46:19.194185+0000","last_undegraded":"2024-09-10T08:46:19.194185+0000","last_fullsized":"2024-09-10T08:46:19.194185+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:42:35.990794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193214+0000","last_change":"2024-09-10T08:44:10.673258+0000","last_active":"2024-09-10T08:46:19.193214+0000","last_peered":"2024-09-10T08:46:19.193214+0000","last_clean":"2024-09-10T08:46:19.193214+0000","last_became_active":"2024-09-10T08:44:10.672854+0000","last_became_peered":"2024-09-10T08:44:10.672854+0000","last_unstale":"2024-09-10T08:46:19.193214+0000","last_undegraded":"2024-09-10T08:46:19.193214+0000","last_fullsized":"2024-09-10T08:46:19.193214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:35.913820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194588+0000","last_change":"2024-09-10T08:44:08.658039+0000","last_active":"2024-09-10T08:46:19.194588+0000","last_peered":"2024-09-10T08:46:19.194588+0000","last_clean":"2024-09-10T08:46:19.194588+0000","last_became_active":"2024-09-10T08:44:08.657636+0000","last_became_peered":"2024-09-10T08:44:08.657636+0000","last_unstale":"2024-09-10T08:46:19.194588+0000","last_undegraded":"2024-09-10T08:46:19.194588+0000","last_fullsized":"2024-09-10T08:46:19.194588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:09.615992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873591+0000","last_change":"2024-09-10T08:42:57.666747+0000","last_active":"2024-09-10T08:46:18.873591+0000","last_peered":"2024-09-10T08:46:18.873591+0000","last_clean":"2024-09-10T08:46:18.873591+0000","last_became_active":"2024-09-10T08:42:57.666417+0000","last_became_peered":"2024-09-10T08:42:57.666417+0000","last_unstale":"2024-09-10T08:46:18.873591+0000","last_undegraded":"2024-09-10T08:46:18.873591+0000","last_fullsized":"2024-09-10T08:46:18.873591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_clean_scrub_stamp":"2024-09-10T08:39:31.729794+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.547989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874074+0000","last_change":"2024-09-10T08:44:17.723021+0000","last_active":"2024-09-10T08:46:18.874074+0000","last_peered":"2024-09-10T08:46:18.874074+0000","last_clean":"2024-09-10T08:46:18.874074+0000","last_became_active":"2024-09-10T08:44:17.722222+0000","last_became_peered":"2024-09-10T08:44:17.722222+0000","last_unstale":"2024-09-10T08:46:18.874074+0000","last_undegraded":"2024-09-10T08:46:18.874074+0000","last_fullsized":"2024-09-10T08:46:18.874074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:24.444427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874129+0000","last_change":"2024-09-10T08:44:10.676651+0000","last_active":"2024-09-10T08:46:18.874129+0000","last_peered":"2024-09-10T08:46:18.874129+0000","last_clean":"2024-09-10T08:46:18.874129+0000","last_became_active":"2024-09-10T08:44:10.676342+0000","last_became_peered":"2024-09-10T08:44:10.676342+0000","last_unstale":"2024-09-10T08:46:18.874129+0000","last_undegraded":"2024-09-10T08:46:18.874129+0000","last_fullsized":"2024-09-10T08:46:18.874129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:07.021273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'8","reported_seq":164,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:45.380424+0000","last_change":"2024-09-10T08:44:09.050305+0000","last_active":"2024-09-10T08:46:45.380424+0000","last_peered":"2024-09-10T08:46:45.380424+0000","last_clean":"2024-09-10T08:46:45.380424+0000","last_became_active":"2024-09-10T08:44:09.049958+0000","last_became_peered":"2024-09-10T08:44:09.049958+0000","last_unstale":"2024-09-10T08:46:45.380424+0000","last_undegraded":"2024-09-10T08:46:45.380424+0000","last_fullsized":"2024-09-10T08:46:45.380424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:42.219190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2199,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193720+0000","last_change":"2024-09-10T08:44:17.720219+0000","last_active":"2024-09-10T08:46:19.193720+0000","last_peered":"2024-09-10T08:46:19.193720+0000","last_clean":"2024-09-10T08:46:19.193720+0000","last_became_active":"2024-09-10T08:44:17.719756+0000","last_became_peered":"2024-09-10T08:44:17.719756+0000","last_unstale":"2024-09-10T08:46:19.193720+0000","last_undegraded":"2024-09-10T08:46:19.193720+0000","last_fullsized":"2024-09-10T08:46:19.193720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:01.070434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873929+0000","last_change":"2024-09-10T08:44:10.678986+0000","last_active":"2024-09-10T08:46:18.873929+0000","last_peered":"2024-09-10T08:46:18.873929+0000","last_clean":"2024-09-10T08:46:18.873929+0000","last_became_active":"2024-09-10T08:44:10.678654+0000","last_became_peered":"2024-09-10T08:44:10.678654+0000","last_unstale":"2024-09-10T08:46:18.873929+0000","last_undegraded":"2024-09-10T08:46:18.873929+0000","last_fullsized":"2024-09-10T08:46:18.873929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:07.116802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805026+0000","last_change":"2024-09-10T08:44:08.655025+0000","last_active":"2024-09-10T08:46:18.805026+0000","last_peered":"2024-09-10T08:46:18.805026+0000","last_clean":"2024-09-10T08:46:18.805026+0000","last_became_active":"2024-09-10T08:44:08.654728+0000","last_became_peered":"2024-09-10T08:44:08.654728+0000","last_unstale":"2024-09-10T08:46:18.805026+0000","last_undegraded":"2024-09-10T08:46:18.805026+0000","last_fullsized":"2024-09-10T08:46:18.805026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:31.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193534+0000","last_change":"2024-09-10T08:45:18.261492+0000","last_active":"2024-09-10T08:46:19.193534+0000","last_peered":"2024-09-10T08:46:19.193534+0000","last_clean":"2024-09-10T08:46:19.193534+0000","last_became_active":"2024-09-10T08:45:18.261111+0000","last_became_peered":"2024-09-10T08:45:18.261111+0000","last_unstale":"2024-09-10T08:46:19.193534+0000","last_undegraded":"2024-09-10T08:46:19.193534+0000","last_fullsized":"2024-09-10T08:46:19.193534+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:48.709459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874006+0000","last_change":"2024-09-10T08:44:10.675072+0000","last_active":"2024-09-10T08:46:18.874006+0000","last_peered":"2024-09-10T08:46:18.874006+0000","last_clean":"2024-09-10T08:46:18.874006+0000","last_became_active":"2024-09-10T08:44:10.674636+0000","last_became_peered":"2024-09-10T08:44:10.674636+0000","last_unstale":"2024-09-10T08:46:18.874006+0000","last_undegraded":"2024-09-10T08:46:18.874006+0000","last_fullsized":"2024-09-10T08:46:18.874006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:55.198445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806864+0000","last_change":"2024-09-10T08:44:08.656450+0000","last_active":"2024-09-10T08:46:18.806864+0000","last_peered":"2024-09-10T08:46:18.806864+0000","last_clean":"2024-09-10T08:46:18.806864+0000","last_became_active":"2024-09-10T08:44:08.655967+0000","last_became_peered":"2024-09-10T08:44:08.655967+0000","last_unstale":"2024-09-10T08:46:18.806864+0000","last_undegraded":"2024-09-10T08:46:18.806864+0000","last_fullsized":"2024-09-10T08:46:18.806864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:29.273167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804195+0000","last_change":"2024-09-10T08:44:17.722217+0000","last_active":"2024-09-10T08:46:18.804195+0000","last_peered":"2024-09-10T08:46:18.804195+0000","last_clean":"2024-09-10T08:46:18.804195+0000","last_became_active":"2024-09-10T08:44:17.721800+0000","last_became_peered":"2024-09-10T08:44:17.721800+0000","last_unstale":"2024-09-10T08:46:18.804195+0000","last_undegraded":"2024-09-10T08:46:18.804195+0000","last_fullsized":"2024-09-10T08:46:18.804195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:58.954397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194230+0000","last_change":"2024-09-10T08:45:18.267857+0000","last_active":"2024-09-10T08:46:19.194230+0000","last_peered":"2024-09-10T08:46:19.194230+0000","last_clean":"2024-09-10T08:46:19.194230+0000","last_became_active":"2024-09-10T08:45:18.267440+0000","last_became_peered":"2024-09-10T08:45:18.267440+0000","last_unstale":"2024-09-10T08:46:19.194230+0000","last_undegraded":"2024-09-10T08:46:19.194230+0000","last_fullsized":"2024-09-10T08:46:19.194230+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:28.659915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:23.312217+0000","last_change":"2024-09-10T08:44:08.659003+0000","last_active":"2024-09-10T08:46:23.312217+0000","last_peered":"2024-09-10T08:46:23.312217+0000","last_clean":"2024-09-10T08:46:23.312217+0000","last_became_active":"2024-09-10T08:44:08.658324+0000","last_became_peered":"2024-09-10T08:44:08.658324+0000","last_unstale":"2024-09-10T08:46:23.312217+0000","last_undegraded":"2024-09-10T08:46:23.312217+0000","last_fullsized":"2024-09-10T08:46:23.312217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:44.743053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804596+0000","last_change":"2024-09-10T08:44:17.723363+0000","last_active":"2024-09-10T08:46:18.804596+0000","last_peered":"2024-09-10T08:46:18.804596+0000","last_clean":"2024-09-10T08:46:18.804596+0000","last_became_active":"2024-09-10T08:44:17.721799+0000","last_became_peered":"2024-09-10T08:44:17.721799+0000","last_unstale":"2024-09-10T08:46:18.804596+0000","last_undegraded":"2024-09-10T08:46:18.804596+0000","last_fullsized":"2024-09-10T08:46:18.804596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:08:28.772193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804709+0000","last_change":"2024-09-10T08:44:10.682952+0000","last_active":"2024-09-10T08:46:18.804709+0000","last_peered":"2024-09-10T08:46:18.804709+0000","last_clean":"2024-09-10T08:46:18.804709+0000","last_became_active":"2024-09-10T08:44:10.682249+0000","last_became_peered":"2024-09-10T08:44:10.682249+0000","last_unstale":"2024-09-10T08:46:18.804709+0000","last_undegraded":"2024-09-10T08:46:18.804709+0000","last_fullsized":"2024-09-10T08:46:18.804709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:32:21.285461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193239+0000","last_change":"2024-09-10T08:44:09.050454+0000","last_active":"2024-09-10T08:46:19.193239+0000","last_peered":"2024-09-10T08:46:19.193239+0000","last_clean":"2024-09-10T08:46:19.193239+0000","last_became_active":"2024-09-10T08:44:09.050036+0000","last_became_peered":"2024-09-10T08:44:09.050036+0000","last_unstale":"2024-09-10T08:46:19.193239+0000","last_undegraded":"2024-09-10T08:46:19.193239+0000","last_fullsized":"2024-09-10T08:46:19.193239+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:55.141471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806725+0000","last_change":"2024-09-10T08:44:17.721203+0000","last_active":"2024-09-10T08:46:18.806725+0000","last_peered":"2024-09-10T08:46:18.806725+0000","last_clean":"2024-09-10T08:46:18.806725+0000","last_became_active":"2024-09-10T08:44:17.720737+0000","last_became_peered":"2024-09-10T08:44:17.720737+0000","last_unstale":"2024-09-10T08:46:18.806725+0000","last_undegraded":"2024-09-10T08:46:18.806725+0000","last_fullsized":"2024-09-10T08:46:18.806725+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:32.081079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873435+0000","last_change":"2024-09-10T08:44:10.680944+0000","last_active":"2024-09-10T08:46:18.873435+0000","last_peered":"2024-09-10T08:46:18.873435+0000","last_clean":"2024-09-10T08:46:18.873435+0000","last_became_active":"2024-09-10T08:44:10.680644+0000","last_became_peered":"2024-09-10T08:44:10.680644+0000","last_unstale":"2024-09-10T08:46:18.873435+0000","last_undegraded":"2024-09-10T08:46:18.873435+0000","last_fullsized":"2024-09-10T08:46:18.873435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:46.495894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":159,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:42.376726+0000","last_change":"2024-09-10T08:44:08.658135+0000","last_active":"2024-09-10T08:46:42.376726+0000","last_peered":"2024-09-10T08:46:42.376726+0000","last_clean":"2024-09-10T08:46:42.376726+0000","last_became_active":"2024-09-10T08:44:08.657793+0000","last_became_peered":"2024-09-10T08:44:08.657793+0000","last_unstale":"2024-09-10T08:46:42.376726+0000","last_undegraded":"2024-09-10T08:46:42.376726+0000","last_fullsized":"2024-09-10T08:46:42.376726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:16.376268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193715+0000","last_change":"2024-09-10T08:45:18.271678+0000","last_active":"2024-09-10T08:46:19.193715+0000","last_peered":"2024-09-10T08:46:19.193715+0000","last_clean":"2024-09-10T08:46:19.193715+0000","last_became_active":"2024-09-10T08:45:18.271209+0000","last_became_peered":"2024-09-10T08:45:18.271209+0000","last_unstale":"2024-09-10T08:46:19.193715+0000","last_undegraded":"2024-09-10T08:46:19.193715+0000","last_fullsized":"2024-09-10T08:46:19.193715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:17.730839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873847+0000","last_change":"2024-09-10T08:44:10.678254+0000","last_active":"2024-09-10T08:46:18.873847+0000","last_peered":"2024-09-10T08:46:18.873847+0000","last_clean":"2024-09-10T08:46:18.873847+0000","last_became_active":"2024-09-10T08:44:10.677891+0000","last_became_peered":"2024-09-10T08:44:10.677891+0000","last_unstale":"2024-09-10T08:46:18.873847+0000","last_undegraded":"2024-09-10T08:46:18.873847+0000","last_fullsized":"2024-09-10T08:46:18.873847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:32:43.399049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804719+0000","last_change":"2024-09-10T08:44:08.656315+0000","last_active":"2024-09-10T08:46:18.804719+0000","last_peered":"2024-09-10T08:46:18.804719+0000","last_clean":"2024-09-10T08:46:18.804719+0000","last_became_active":"2024-09-10T08:44:08.655978+0000","last_became_peered":"2024-09-10T08:44:08.655978+0000","last_unstale":"2024-09-10T08:46:18.804719+0000","last_undegraded":"2024-09-10T08:46:18.804719+0000","last_fullsized":"2024-09-10T08:46:18.804719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:15.025354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193456+0000","last_change":"2024-09-10T08:44:17.726789+0000","last_active":"2024-09-10T08:46:19.193456+0000","last_peered":"2024-09-10T08:46:19.193456+0000","last_clean":"2024-09-10T08:46:19.193456+0000","last_became_active":"2024-09-10T08:44:17.726450+0000","last_became_peered":"2024-09-10T08:44:17.726450+0000","last_unstale":"2024-09-10T08:46:19.193456+0000","last_undegraded":"2024-09-10T08:46:19.193456+0000","last_fullsized":"2024-09-10T08:46:19.193456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:03.899822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806240+0000","last_change":"2024-09-10T08:44:10.677359+0000","last_active":"2024-09-10T08:46:18.806240+0000","last_peered":"2024-09-10T08:46:18.806240+0000","last_clean":"2024-09-10T08:46:18.806240+0000","last_became_active":"2024-09-10T08:44:10.676782+0000","last_became_peered":"2024-09-10T08:44:10.676782+0000","last_unstale":"2024-09-10T08:46:18.806240+0000","last_undegraded":"2024-09-10T08:46:18.806240+0000","last_fullsized":"2024-09-10T08:46:18.806240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:31.721714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808306+0000","last_change":"2024-09-10T08:44:08.656993+0000","last_active":"2024-09-10T08:46:18.808306+0000","last_peered":"2024-09-10T08:46:18.808306+0000","last_clean":"2024-09-10T08:46:18.808306+0000","last_became_active":"2024-09-10T08:44:08.656571+0000","last_became_peered":"2024-09-10T08:44:08.656571+0000","last_unstale":"2024-09-10T08:46:18.808306+0000","last_undegraded":"2024-09-10T08:46:18.808306+0000","last_fullsized":"2024-09-10T08:46:18.808306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:32.202740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804919+0000","last_change":"2024-09-10T08:44:17.725674+0000","last_active":"2024-09-10T08:46:18.804919+0000","last_peered":"2024-09-10T08:46:18.804919+0000","last_clean":"2024-09-10T08:46:18.804919+0000","last_became_active":"2024-09-10T08:44:17.725263+0000","last_became_peered":"2024-09-10T08:44:17.725263+0000","last_unstale":"2024-09-10T08:46:18.804919+0000","last_undegraded":"2024-09-10T08:46:18.804919+0000","last_fullsized":"2024-09-10T08:46:18.804919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:18.183157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193937+0000","last_change":"2024-09-10T08:44:10.676871+0000","last_active":"2024-09-10T08:46:19.193937+0000","last_peered":"2024-09-10T08:46:19.193937+0000","last_clean":"2024-09-10T08:46:19.193937+0000","last_became_active":"2024-09-10T08:44:10.676540+0000","last_became_peered":"2024-09-10T08:44:10.676540+0000","last_unstale":"2024-09-10T08:46:19.193937+0000","last_undegraded":"2024-09-10T08:46:19.193937+0000","last_fullsized":"2024-09-10T08:46:19.193937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:50.964100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'41","reported_seq":315,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:44.378613+0000","last_change":"2024-09-10T08:44:08.660113+0000","last_active":"2024-09-10T08:46:44.378613+0000","last_peered":"2024-09-10T08:46:44.378613+0000","last_clean":"2024-09-10T08:46:44.378613+0000","last_became_active":"2024-09-10T08:44:08.659644+0000","last_became_peered":"2024-09-10T08:44:08.659644+0000","last_unstale":"2024-09-10T08:46:44.378613+0000","last_undegraded":"2024-09-10T08:46:44.378613+0000","last_fullsized":"2024-09-10T08:46:44.378613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:01.605344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68542,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":235,"num_read_kb":185,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803342+0000","last_change":"2024-09-10T08:44:17.723653+0000","last_active":"2024-09-10T08:46:18.803342+0000","last_peered":"2024-09-10T08:46:18.803342+0000","last_clean":"2024-09-10T08:46:18.803342+0000","last_became_active":"2024-09-10T08:44:17.723284+0000","last_became_peered":"2024-09-10T08:44:17.723284+0000","last_unstale":"2024-09-10T08:46:18.803342+0000","last_undegraded":"2024-09-10T08:46:18.803342+0000","last_fullsized":"2024-09-10T08:46:18.803342+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:17.162526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193970+0000","last_change":"2024-09-10T08:44:10.681837+0000","last_active":"2024-09-10T08:46:19.193970+0000","last_peered":"2024-09-10T08:46:19.193970+0000","last_clean":"2024-09-10T08:46:19.193970+0000","last_became_active":"2024-09-10T08:44:10.681518+0000","last_became_peered":"2024-09-10T08:44:10.681518+0000","last_unstale":"2024-09-10T08:46:19.193970+0000","last_undegraded":"2024-09-10T08:46:19.193970+0000","last_fullsized":"2024-09-10T08:46:19.193970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:10.004215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804988+0000","last_change":"2024-09-10T08:44:09.050394+0000","last_active":"2024-09-10T08:46:18.804988+0000","last_peered":"2024-09-10T08:46:18.804988+0000","last_clean":"2024-09-10T08:46:18.804988+0000","last_became_active":"2024-09-10T08:44:09.049969+0000","last_became_peered":"2024-09-10T08:44:09.049969+0000","last_unstale":"2024-09-10T08:46:18.804988+0000","last_undegraded":"2024-09-10T08:46:18.804988+0000","last_fullsized":"2024-09-10T08:46:18.804988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:43.519246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194174+0000","last_change":"2024-09-10T08:44:17.720318+0000","last_active":"2024-09-10T08:46:19.194174+0000","last_peered":"2024-09-10T08:46:19.194174+0000","last_clean":"2024-09-10T08:46:19.194174+0000","last_became_active":"2024-09-10T08:44:17.719598+0000","last_became_peered":"2024-09-10T08:44:17.719598+0000","last_unstale":"2024-09-10T08:46:19.194174+0000","last_undegraded":"2024-09-10T08:46:19.194174+0000","last_fullsized":"2024-09-10T08:46:19.194174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:15:49.520324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193946+0000","last_change":"2024-09-10T08:44:10.677340+0000","last_active":"2024-09-10T08:46:19.193946+0000","last_peered":"2024-09-10T08:46:19.193946+0000","last_clean":"2024-09-10T08:46:19.193946+0000","last_became_active":"2024-09-10T08:44:10.676628+0000","last_became_peered":"2024-09-10T08:44:10.676628+0000","last_unstale":"2024-09-10T08:46:19.193946+0000","last_undegraded":"2024-09-10T08:46:19.193946+0000","last_fullsized":"2024-09-10T08:46:19.193946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:59:17.924217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874027+0000","last_change":"2024-09-10T08:44:08.660254+0000","last_active":"2024-09-10T08:46:18.874027+0000","last_peered":"2024-09-10T08:46:18.874027+0000","last_clean":"2024-09-10T08:46:18.874027+0000","last_became_active":"2024-09-10T08:44:08.659667+0000","last_became_peered":"2024-09-10T08:44:08.659667+0000","last_unstale":"2024-09-10T08:46:18.874027+0000","last_undegraded":"2024-09-10T08:46:18.874027+0000","last_fullsized":"2024-09-10T08:46:18.874027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:49:49.146867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803198+0000","last_change":"2024-09-10T08:44:17.722373+0000","last_active":"2024-09-10T08:46:18.803198+0000","last_peered":"2024-09-10T08:46:18.803198+0000","last_clean":"2024-09-10T08:46:18.803198+0000","last_became_active":"2024-09-10T08:44:17.722035+0000","last_became_peered":"2024-09-10T08:44:17.722035+0000","last_unstale":"2024-09-10T08:46:18.803198+0000","last_undegraded":"2024-09-10T08:46:18.803198+0000","last_fullsized":"2024-09-10T08:46:18.803198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:30:36.187364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194365+0000","last_change":"2024-09-10T08:44:10.672585+0000","last_active":"2024-09-10T08:46:19.194365+0000","last_peered":"2024-09-10T08:46:19.194365+0000","last_clean":"2024-09-10T08:46:19.194365+0000","last_became_active":"2024-09-10T08:44:10.671987+0000","last_became_peered":"2024-09-10T08:44:10.671987+0000","last_unstale":"2024-09-10T08:46:19.194365+0000","last_undegraded":"2024-09-10T08:46:19.194365+0000","last_fullsized":"2024-09-10T08:46:19.194365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:48.660750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'7","reported_seq":268,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:44.378619+0000","last_change":"2024-09-10T08:44:08.658069+0000","last_active":"2024-09-10T08:46:44.378619+0000","last_peered":"2024-09-10T08:46:44.378619+0000","last_clean":"2024-09-10T08:46:44.378619+0000","last_became_active":"2024-09-10T08:44:08.657702+0000","last_became_peered":"2024-09-10T08:44:08.657702+0000","last_unstale":"2024-09-10T08:46:44.378619+0000","last_undegraded":"2024-09-10T08:46:44.378619+0000","last_fullsized":"2024-09-10T08:46:44.378619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:17.383267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":222,"num_read_kb":222,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192772+0000","last_change":"2024-09-10T08:44:17.717984+0000","last_active":"2024-09-10T08:46:19.192772+0000","last_peered":"2024-09-10T08:46:19.192772+0000","last_clean":"2024-09-10T08:46:19.192772+0000","last_became_active":"2024-09-10T08:44:17.717581+0000","last_became_peered":"2024-09-10T08:44:17.717581+0000","last_unstale":"2024-09-10T08:46:19.192772+0000","last_undegraded":"2024-09-10T08:46:19.192772+0000","last_fullsized":"2024-09-10T08:46:19.192772+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:25.330732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193566+0000","last_change":"2024-09-10T08:44:17.721244+0000","last_active":"2024-09-10T08:46:19.193566+0000","last_peered":"2024-09-10T08:46:19.193566+0000","last_clean":"2024-09-10T08:46:19.193566+0000","last_became_active":"2024-09-10T08:44:17.720884+0000","last_became_peered":"2024-09-10T08:44:17.720884+0000","last_unstale":"2024-09-10T08:46:19.193566+0000","last_undegraded":"2024-09-10T08:46:19.193566+0000","last_fullsized":"2024-09-10T08:46:19.193566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.769513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803279+0000","last_change":"2024-09-10T08:44:10.667558+0000","last_active":"2024-09-10T08:46:18.803279+0000","last_peered":"2024-09-10T08:46:18.803279+0000","last_clean":"2024-09-10T08:46:18.803279+0000","last_became_active":"2024-09-10T08:44:10.667214+0000","last_became_peered":"2024-09-10T08:44:10.667214+0000","last_unstale":"2024-09-10T08:46:18.803279+0000","last_undegraded":"2024-09-10T08:46:18.803279+0000","last_fullsized":"2024-09-10T08:46:18.803279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:53.630790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804697+0000","last_change":"2024-09-10T08:44:08.657930+0000","last_active":"2024-09-10T08:46:18.804697+0000","last_peered":"2024-09-10T08:46:18.804697+0000","last_clean":"2024-09-10T08:46:18.804697+0000","last_became_active":"2024-09-10T08:44:08.657611+0000","last_became_peered":"2024-09-10T08:44:08.657611+0000","last_unstale":"2024-09-10T08:46:18.804697+0000","last_undegraded":"2024-09-10T08:46:18.804697+0000","last_fullsized":"2024-09-10T08:46:18.804697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:36.923786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193910+0000","last_change":"2024-09-10T08:44:17.726106+0000","last_active":"2024-09-10T08:46:19.193910+0000","last_peered":"2024-09-10T08:46:19.193910+0000","last_clean":"2024-09-10T08:46:19.193910+0000","last_became_active":"2024-09-10T08:44:17.725683+0000","last_became_peered":"2024-09-10T08:44:17.725683+0000","last_unstale":"2024-09-10T08:46:19.193910+0000","last_undegraded":"2024-09-10T08:46:19.193910+0000","last_fullsized":"2024-09-10T08:46:19.193910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:23.392794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192727+0000","last_change":"2024-09-10T08:44:10.672204+0000","last_active":"2024-09-10T08:46:19.192727+0000","last_peered":"2024-09-10T08:46:19.192727+0000","last_clean":"2024-09-10T08:46:19.192727+0000","last_became_active":"2024-09-10T08:44:10.671898+0000","last_became_peered":"2024-09-10T08:44:10.671898+0000","last_unstale":"2024-09-10T08:46:19.192727+0000","last_undegraded":"2024-09-10T08:46:19.192727+0000","last_fullsized":"2024-09-10T08:46:19.192727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:03.737357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805979+0000","last_change":"2024-09-10T08:44:08.656620+0000","last_active":"2024-09-10T08:46:18.805979+0000","last_peered":"2024-09-10T08:46:18.805979+0000","last_clean":"2024-09-10T08:46:18.805979+0000","last_became_active":"2024-09-10T08:44:08.656306+0000","last_became_peered":"2024-09-10T08:44:08.656306+0000","last_unstale":"2024-09-10T08:46:18.805979+0000","last_undegraded":"2024-09-10T08:46:18.805979+0000","last_fullsized":"2024-09-10T08:46:18.805979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:57.081909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804743+0000","last_change":"2024-09-10T08:44:08.660178+0000","last_active":"2024-09-10T08:46:18.804743+0000","last_peered":"2024-09-10T08:46:18.804743+0000","last_clean":"2024-09-10T08:46:18.804743+0000","last_became_active":"2024-09-10T08:44:08.659839+0000","last_became_peered":"2024-09-10T08:44:08.659839+0000","last_unstale":"2024-09-10T08:46:18.804743+0000","last_undegraded":"2024-09-10T08:46:18.804743+0000","last_fullsized":"2024-09-10T08:46:18.804743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:17.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804848+0000","last_change":"2024-09-10T08:44:10.683683+0000","last_active":"2024-09-10T08:46:18.804848+0000","last_peered":"2024-09-10T08:46:18.804848+0000","last_clean":"2024-09-10T08:46:18.804848+0000","last_became_active":"2024-09-10T08:44:10.683337+0000","last_became_peered":"2024-09-10T08:44:10.683337+0000","last_unstale":"2024-09-10T08:46:18.804848+0000","last_undegraded":"2024-09-10T08:46:18.804848+0000","last_fullsized":"2024-09-10T08:46:18.804848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:54.863098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808113+0000","last_change":"2024-09-10T08:44:17.721602+0000","last_active":"2024-09-10T08:46:18.808113+0000","last_peered":"2024-09-10T08:46:18.808113+0000","last_clean":"2024-09-10T08:46:18.808113+0000","last_became_active":"2024-09-10T08:44:17.721194+0000","last_became_peered":"2024-09-10T08:44:17.721194+0000","last_unstale":"2024-09-10T08:46:18.808113+0000","last_undegraded":"2024-09-10T08:46:18.808113+0000","last_fullsized":"2024-09-10T08:46:18.808113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:39.298494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":160,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:43.378005+0000","last_change":"2024-09-10T08:44:08.661943+0000","last_active":"2024-09-10T08:46:43.378005+0000","last_peered":"2024-09-10T08:46:43.378005+0000","last_clean":"2024-09-10T08:46:43.378005+0000","last_became_active":"2024-09-10T08:44:08.661587+0000","last_became_peered":"2024-09-10T08:44:08.661587+0000","last_unstale":"2024-09-10T08:46:43.378005+0000","last_undegraded":"2024-09-10T08:46:43.378005+0000","last_fullsized":"2024-09-10T08:46:43.378005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:05.771641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193844+0000","last_change":"2024-09-10T08:44:10.673522+0000","last_active":"2024-09-10T08:46:19.193844+0000","last_peered":"2024-09-10T08:46:19.193844+0000","last_clean":"2024-09-10T08:46:19.193844+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:46:19.193844+0000","last_undegraded":"2024-09-10T08:46:19.193844+0000","last_fullsized":"2024-09-10T08:46:19.193844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:03.006144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808482+0000","last_change":"2024-09-10T08:44:17.721306+0000","last_active":"2024-09-10T08:46:18.808482+0000","last_peered":"2024-09-10T08:46:18.808482+0000","last_clean":"2024-09-10T08:46:18.808482+0000","last_became_active":"2024-09-10T08:44:17.720401+0000","last_became_peered":"2024-09-10T08:44:17.720401+0000","last_unstale":"2024-09-10T08:46:18.808482+0000","last_undegraded":"2024-09-10T08:46:18.808482+0000","last_fullsized":"2024-09-10T08:46:18.808482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:11.589371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193166+0000","last_change":"2024-09-10T08:44:10.678425+0000","last_active":"2024-09-10T08:46:19.193166+0000","last_peered":"2024-09-10T08:46:19.193166+0000","last_clean":"2024-09-10T08:46:19.193166+0000","last_became_active":"2024-09-10T08:44:10.677187+0000","last_became_peered":"2024-09-10T08:44:10.677187+0000","last_unstale":"2024-09-10T08:46:19.193166+0000","last_undegraded":"2024-09-10T08:46:19.193166+0000","last_fullsized":"2024-09-10T08:46:19.193166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:06.527530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808431+0000","last_change":"2024-09-10T08:44:10.674178+0000","last_active":"2024-09-10T08:46:18.808431+0000","last_peered":"2024-09-10T08:46:18.808431+0000","last_clean":"2024-09-10T08:46:18.808431+0000","last_became_active":"2024-09-10T08:44:10.673843+0000","last_became_peered":"2024-09-10T08:44:10.673843+0000","last_unstale":"2024-09-10T08:46:18.808431+0000","last_undegraded":"2024-09-10T08:46:18.808431+0000","last_fullsized":"2024-09-10T08:46:18.808431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:35.023663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.807988+0000","last_change":"2024-09-10T08:44:10.673271+0000","last_active":"2024-09-10T08:46:18.807988+0000","last_peered":"2024-09-10T08:46:18.807988+0000","last_clean":"2024-09-10T08:46:18.807988+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:46:18.807988+0000","last_undegraded":"2024-09-10T08:46:18.807988+0000","last_fullsized":"2024-09-10T08:46:18.807988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:21:26.606820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193502+0000","last_change":"2024-09-10T08:44:10.671348+0000","last_active":"2024-09-10T08:46:19.193502+0000","last_peered":"2024-09-10T08:46:19.193502+0000","last_clean":"2024-09-10T08:46:19.193502+0000","last_became_active":"2024-09-10T08:44:10.670623+0000","last_became_peered":"2024-09-10T08:44:10.670623+0000","last_unstale":"2024-09-10T08:46:19.193502+0000","last_undegraded":"2024-09-10T08:46:19.193502+0000","last_fullsized":"2024-09-10T08:46:19.193502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:06.145694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806400+0000","last_change":"2024-09-10T08:44:10.666011+0000","last_active":"2024-09-10T08:46:18.806400+0000","last_peered":"2024-09-10T08:46:18.806400+0000","last_clean":"2024-09-10T08:46:18.806400+0000","last_became_active":"2024-09-10T08:44:10.665664+0000","last_became_peered":"2024-09-10T08:44:10.665664+0000","last_unstale":"2024-09-10T08:46:18.806400+0000","last_undegraded":"2024-09-10T08:46:18.806400+0000","last_fullsized":"2024-09-10T08:46:18.806400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:31.542102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193792+0000","last_change":"2024-09-10T08:44:10.675960+0000","last_active":"2024-09-10T08:46:19.193792+0000","last_peered":"2024-09-10T08:46:19.193792+0000","last_clean":"2024-09-10T08:46:19.193792+0000","last_became_active":"2024-09-10T08:44:10.675587+0000","last_became_peered":"2024-09-10T08:44:10.675587+0000","last_unstale":"2024-09-10T08:46:19.193792+0000","last_undegraded":"2024-09-10T08:46:19.193792+0000","last_fullsized":"2024-09-10T08:46:19.193792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:54.634619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808734+0000","last_change":"2024-09-10T08:44:10.681094+0000","last_active":"2024-09-10T08:46:18.808734+0000","last_peered":"2024-09-10T08:46:18.808734+0000","last_clean":"2024-09-10T08:46:18.808734+0000","last_became_active":"2024-09-10T08:44:10.678585+0000","last_became_peered":"2024-09-10T08:44:10.678585+0000","last_unstale":"2024-09-10T08:46:18.808734+0000","last_undegraded":"2024-09-10T08:46:18.808734+0000","last_fullsized":"2024-09-10T08:46:18.808734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:14.951949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193627+0000","last_change":"2024-09-10T08:44:10.675473+0000","last_active":"2024-09-10T08:46:19.193627+0000","last_peered":"2024-09-10T08:46:19.193627+0000","last_clean":"2024-09-10T08:46:19.193627+0000","last_became_active":"2024-09-10T08:44:10.674155+0000","last_became_peered":"2024-09-10T08:44:10.674155+0000","last_unstale":"2024-09-10T08:46:19.193627+0000","last_undegraded":"2024-09-10T08:46:19.193627+0000","last_fullsized":"2024-09-10T08:46:19.193627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:55.940235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804316+0000","last_change":"2024-09-10T08:44:10.680162+0000","last_active":"2024-09-10T08:46:18.804316+0000","last_peered":"2024-09-10T08:46:18.804316+0000","last_clean":"2024-09-10T08:46:18.804316+0000","last_became_active":"2024-09-10T08:44:10.679800+0000","last_became_peered":"2024-09-10T08:44:10.679800+0000","last_unstale":"2024-09-10T08:46:18.804316+0000","last_undegraded":"2024-09-10T08:46:18.804316+0000","last_fullsized":"2024-09-10T08:46:18.804316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:12.319813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192932+0000","last_change":"2024-09-10T08:44:10.674724+0000","last_active":"2024-09-10T08:46:19.192932+0000","last_peered":"2024-09-10T08:46:19.192932+0000","last_clean":"2024-09-10T08:46:19.192932+0000","last_became_active":"2024-09-10T08:44:10.674389+0000","last_became_peered":"2024-09-10T08:44:10.674389+0000","last_unstale":"2024-09-10T08:46:19.192932+0000","last_undegraded":"2024-09-10T08:46:19.192932+0000","last_fullsized":"2024-09-10T08:46:19.192932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:52.123186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193888+0000","last_change":"2024-09-10T08:44:10.672805+0000","last_active":"2024-09-10T08:46:19.193888+0000","last_peered":"2024-09-10T08:46:19.193888+0000","last_clean":"2024-09-10T08:46:19.193888+0000","last_became_active":"2024-09-10T08:44:10.672508+0000","last_became_peered":"2024-09-10T08:44:10.672508+0000","last_unstale":"2024-09-10T08:46:19.193888+0000","last_undegraded":"2024-09-10T08:46:19.193888+0000","last_fullsized":"2024-09-10T08:46:19.193888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:48.558633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806228+0000","last_change":"2024-09-10T08:44:10.677504+0000","last_active":"2024-09-10T08:46:18.806228+0000","last_peered":"2024-09-10T08:46:18.806228+0000","last_clean":"2024-09-10T08:46:18.806228+0000","last_became_active":"2024-09-10T08:44:10.676822+0000","last_became_peered":"2024-09-10T08:44:10.676822+0000","last_unstale":"2024-09-10T08:46:18.806228+0000","last_undegraded":"2024-09-10T08:46:18.806228+0000","last_fullsized":"2024-09-10T08:46:18.806228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:05:48.293333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193457+0000","last_change":"2024-09-10T08:45:18.268208+0000","last_active":"2024-09-10T08:46:19.193457+0000","last_peered":"2024-09-10T08:46:19.193457+0000","last_clean":"2024-09-10T08:46:19.193457+0000","last_became_active":"2024-09-10T08:45:18.267700+0000","last_became_peered":"2024-09-10T08:45:18.267700+0000","last_unstale":"2024-09-10T08:46:19.193457+0000","last_undegraded":"2024-09-10T08:46:19.193457+0000","last_fullsized":"2024-09-10T08:46:19.193457+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:54.487624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808678+0000","last_change":"2024-09-10T08:44:10.670615+0000","last_active":"2024-09-10T08:46:18.808678+0000","last_peered":"2024-09-10T08:46:18.808678+0000","last_clean":"2024-09-10T08:46:18.808678+0000","last_became_active":"2024-09-10T08:44:10.670032+0000","last_became_peered":"2024-09-10T08:44:10.670032+0000","last_unstale":"2024-09-10T08:46:18.808678+0000","last_undegraded":"2024-09-10T08:46:18.808678+0000","last_fullsized":"2024-09-10T08:46:18.808678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:31.078788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803799+0000","last_change":"2024-09-10T08:44:10.672367+0000","last_active":"2024-09-10T08:46:18.803799+0000","last_peered":"2024-09-10T08:46:18.803799+0000","last_clean":"2024-09-10T08:46:18.803799+0000","last_became_active":"2024-09-10T08:44:10.672040+0000","last_became_peered":"2024-09-10T08:44:10.672040+0000","last_unstale":"2024-09-10T08:46:18.803799+0000","last_undegraded":"2024-09-10T08:46:18.803799+0000","last_fullsized":"2024-09-10T08:46:18.803799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:10.634708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194226+0000","last_change":"2024-09-10T08:44:10.676375+0000","last_active":"2024-09-10T08:46:19.194226+0000","last_peered":"2024-09-10T08:46:19.194226+0000","last_clean":"2024-09-10T08:46:19.194226+0000","last_became_active":"2024-09-10T08:44:10.676022+0000","last_became_peered":"2024-09-10T08:44:10.676022+0000","last_unstale":"2024-09-10T08:46:19.194226+0000","last_undegraded":"2024-09-10T08:46:19.194226+0000","last_fullsized":"2024-09-10T08:46:19.194226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:02.681954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193408+0000","last_change":"2024-09-10T08:44:10.673378+0000","last_active":"2024-09-10T08:46:19.193408+0000","last_peered":"2024-09-10T08:46:19.193408+0000","last_clean":"2024-09-10T08:46:19.193408+0000","last_became_active":"2024-09-10T08:44:10.673089+0000","last_became_peered":"2024-09-10T08:44:10.673089+0000","last_unstale":"2024-09-10T08:46:19.193408+0000","last_undegraded":"2024-09-10T08:46:19.193408+0000","last_fullsized":"2024-09-10T08:46:19.193408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:36:19.258651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192898+0000","last_change":"2024-09-10T08:44:10.671557+0000","last_active":"2024-09-10T08:46:19.192898+0000","last_peered":"2024-09-10T08:46:19.192898+0000","last_clean":"2024-09-10T08:46:19.192898+0000","last_became_active":"2024-09-10T08:44:10.671248+0000","last_became_peered":"2024-09-10T08:44:10.671248+0000","last_unstale":"2024-09-10T08:46:19.192898+0000","last_undegraded":"2024-09-10T08:46:19.192898+0000","last_fullsized":"2024-09-10T08:46:19.192898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:54.261525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193104+0000","last_change":"2024-09-10T08:44:10.674672+0000","last_active":"2024-09-10T08:46:19.193104+0000","last_peered":"2024-09-10T08:46:19.193104+0000","last_clean":"2024-09-10T08:46:19.193104+0000","last_became_active":"2024-09-10T08:44:10.673415+0000","last_became_peered":"2024-09-10T08:44:10.673415+0000","last_unstale":"2024-09-10T08:46:19.193104+0000","last_undegraded":"2024-09-10T08:46:19.193104+0000","last_fullsized":"2024-09-10T08:46:19.193104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:59:14.179664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806192+0000","last_change":"2024-09-10T08:44:10.675293+0000","last_active":"2024-09-10T08:46:18.806192+0000","last_peered":"2024-09-10T08:46:18.806192+0000","last_clean":"2024-09-10T08:46:18.806192+0000","last_became_active":"2024-09-10T08:44:10.674924+0000","last_became_peered":"2024-09-10T08:44:10.674924+0000","last_unstale":"2024-09-10T08:46:18.806192+0000","last_undegraded":"2024-09-10T08:46:18.806192+0000","last_fullsized":"2024-09-10T08:46:18.806192+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:56.027320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194722+0000","last_change":"2024-09-10T08:44:10.674605+0000","last_active":"2024-09-10T08:46:19.194722+0000","last_peered":"2024-09-10T08:46:19.194722+0000","last_clean":"2024-09-10T08:46:19.194722+0000","last_became_active":"2024-09-10T08:44:10.674057+0000","last_became_peered":"2024-09-10T08:44:10.674057+0000","last_unstale":"2024-09-10T08:46:19.194722+0000","last_undegraded":"2024-09-10T08:46:19.194722+0000","last_fullsized":"2024-09-10T08:46:19.194722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:09.473003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874065+0000","last_change":"2024-09-10T08:44:10.677410+0000","last_active":"2024-09-10T08:46:18.874065+0000","last_peered":"2024-09-10T08:46:18.874065+0000","last_clean":"2024-09-10T08:46:18.874065+0000","last_became_active":"2024-09-10T08:44:10.677109+0000","last_became_peered":"2024-09-10T08:44:10.677109+0000","last_unstale":"2024-09-10T08:46:18.874065+0000","last_undegraded":"2024-09-10T08:46:18.874065+0000","last_fullsized":"2024-09-10T08:46:18.874065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:45.455425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805091+0000","last_change":"2024-09-10T08:44:10.669706+0000","last_active":"2024-09-10T08:46:18.805091+0000","last_peered":"2024-09-10T08:46:18.805091+0000","last_clean":"2024-09-10T08:46:18.805091+0000","last_became_active":"2024-09-10T08:44:10.669440+0000","last_became_peered":"2024-09-10T08:44:10.669440+0000","last_unstale":"2024-09-10T08:46:18.805091+0000","last_undegraded":"2024-09-10T08:46:18.805091+0000","last_fullsized":"2024-09-10T08:46:18.805091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:31:55.610651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194758+0000","last_change":"2024-09-10T08:44:10.676482+0000","last_active":"2024-09-10T08:46:19.194758+0000","last_peered":"2024-09-10T08:46:19.194758+0000","last_clean":"2024-09-10T08:46:19.194758+0000","last_became_active":"2024-09-10T08:44:10.675983+0000","last_became_peered":"2024-09-10T08:44:10.675983+0000","last_unstale":"2024-09-10T08:46:19.194758+0000","last_undegraded":"2024-09-10T08:46:19.194758+0000","last_fullsized":"2024-09-10T08:46:19.194758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:01.269283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192699+0000","last_change":"2024-09-10T08:44:10.675681+0000","last_active":"2024-09-10T08:46:19.192699+0000","last_peered":"2024-09-10T08:46:19.192699+0000","last_clean":"2024-09-10T08:46:19.192699+0000","last_became_active":"2024-09-10T08:44:10.675357+0000","last_became_peered":"2024-09-10T08:44:10.675357+0000","last_unstale":"2024-09-10T08:46:19.192699+0000","last_undegraded":"2024-09-10T08:46:19.192699+0000","last_fullsized":"2024-09-10T08:46:19.192699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:26.138333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873522+0000","last_change":"2024-09-10T08:44:10.680819+0000","last_active":"2024-09-10T08:46:18.873522+0000","last_peered":"2024-09-10T08:46:18.873522+0000","last_clean":"2024-09-10T08:46:18.873522+0000","last_became_active":"2024-09-10T08:44:10.680491+0000","last_became_peered":"2024-09-10T08:44:10.680491+0000","last_unstale":"2024-09-10T08:46:18.873522+0000","last_undegraded":"2024-09-10T08:46:18.873522+0000","last_fullsized":"2024-09-10T08:46:18.873522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:17.044776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804621+0000","last_change":"2024-09-10T08:44:10.667376+0000","last_active":"2024-09-10T08:46:18.804621+0000","last_peered":"2024-09-10T08:46:18.804621+0000","last_clean":"2024-09-10T08:46:18.804621+0000","last_became_active":"2024-09-10T08:44:10.666648+0000","last_became_peered":"2024-09-10T08:44:10.666648+0000","last_unstale":"2024-09-10T08:46:18.804621+0000","last_undegraded":"2024-09-10T08:46:18.804621+0000","last_fullsized":"2024-09-10T08:46:18.804621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:33:12.017550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193270+0000","last_change":"2024-09-10T08:44:10.668665+0000","last_active":"2024-09-10T08:46:19.193270+0000","last_peered":"2024-09-10T08:46:19.193270+0000","last_clean":"2024-09-10T08:46:19.193270+0000","last_became_active":"2024-09-10T08:44:10.666031+0000","last_became_peered":"2024-09-10T08:44:10.666031+0000","last_unstale":"2024-09-10T08:46:19.193270+0000","last_undegraded":"2024-09-10T08:46:19.193270+0000","last_fullsized":"2024-09-10T08:46:19.193270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:39:53.991463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194458+0000","last_change":"2024-09-10T08:44:10.676495+0000","last_active":"2024-09-10T08:46:19.194458+0000","last_peered":"2024-09-10T08:46:19.194458+0000","last_clean":"2024-09-10T08:46:19.194458+0000","last_became_active":"2024-09-10T08:44:10.676087+0000","last_became_peered":"2024-09-10T08:44:10.676087+0000","last_unstale":"2024-09-10T08:46:19.194458+0000","last_undegraded":"2024-09-10T08:46:19.194458+0000","last_fullsized":"2024-09-10T08:46:19.194458+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:41.828157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193138+0000","last_change":"2024-09-10T08:44:10.672835+0000","last_active":"2024-09-10T08:46:19.193138+0000","last_peered":"2024-09-10T08:46:19.193138+0000","last_clean":"2024-09-10T08:46:19.193138+0000","last_became_active":"2024-09-10T08:44:10.672537+0000","last_became_peered":"2024-09-10T08:44:10.672537+0000","last_unstale":"2024-09-10T08:46:19.193138+0000","last_undegraded":"2024-09-10T08:46:19.193138+0000","last_fullsized":"2024-09-10T08:46:19.193138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:06.121460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193568+0000","last_change":"2024-09-10T08:44:10.675968+0000","last_active":"2024-09-10T08:46:19.193568+0000","last_peered":"2024-09-10T08:46:19.193568+0000","last_clean":"2024-09-10T08:46:19.193568+0000","last_became_active":"2024-09-10T08:44:10.675628+0000","last_became_peered":"2024-09-10T08:44:10.675628+0000","last_unstale":"2024-09-10T08:46:19.193568+0000","last_undegraded":"2024-09-10T08:46:19.193568+0000","last_fullsized":"2024-09-10T08:46:19.193568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:40.261336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194091+0000","last_change":"2024-09-10T08:44:10.673151+0000","last_active":"2024-09-10T08:46:19.194091+0000","last_peered":"2024-09-10T08:46:19.194091+0000","last_clean":"2024-09-10T08:46:19.194091+0000","last_became_active":"2024-09-10T08:44:10.672833+0000","last_became_peered":"2024-09-10T08:44:10.672833+0000","last_unstale":"2024-09-10T08:46:19.194091+0000","last_undegraded":"2024-09-10T08:46:19.194091+0000","last_fullsized":"2024-09-10T08:46:19.194091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:38.556222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77478,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1160,"num_read_kb":666,"num_write":276,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":233572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514541,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":523370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743474,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28740,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716412,"statfs":{"total":95995035648,"available":95965605888,"internally_reserved":0,"allocated":1757184,"data_stored":1178194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906999,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":526367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135931,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":522649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364864,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1120240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528390,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":518270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757323,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36988,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708164,"statfs":{"total":95995035648,"available":95957159936,"internally_reserved":0,"allocated":1683456,"data_stored":1114832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":522344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":517483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":590229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1130496,"data_stored":587842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":520260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:46:48.890 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T08:46:48.891 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T08:46:49.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.061+0000 7f459fd09640 1 -- 172.21.15.172:0/468963788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4598101430 msgr2=0x7f459810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.061+0000 7f459fd09640 1 --2- 172.21.15.172:0/468963788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4598101430 0x7f459810d960 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4590009cd0 tx=0x7f459002a8e0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.061+0000 7f459fd09640 1 -- 172.21.15.172:0/468963788 shutdown_connections 2024-09-10T08:46:49.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.061+0000 7f459fd09640 1 --2- 172.21.15.172:0/468963788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4598101430 0x7f459810d960 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.061+0000 7f459fd09640 1 --2- 172.21.15.172:0/468963788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4598100a90 0x7f4598100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.061+0000 7f459fd09640 1 --2- 172.21.15.172:0/468963788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4598106a90 0x7f4598106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.061+0000 7f459fd09640 1 -- 172.21.15.172:0/468963788 >> 172.21.15.172:0/468963788 conn(0x7f45980fc7d0 msgr2=0x7f45980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:49.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.061+0000 7f459fd09640 1 -- 172.21.15.172:0/468963788 shutdown_connections 2024-09-10T08:46:49.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.061+0000 7f459fd09640 1 -- 172.21.15.172:0/468963788 wait complete. 2024-09-10T08:46:49.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459fd09640 1 Processor -- start 2024-09-10T08:46:49.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459fd09640 1 -- start start 2024-09-10T08:46:49.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459fd09640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4598100a90 0x7f4598198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459fd09640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4598101430 0x7f45981988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459fd09640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4598106a90 0x7f459819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459fd09640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4598110d40 con 0x7f4598101430 2024-09-10T08:46:49.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459fd09640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4598110bc0 con 0x7f4598100a90 2024-09-10T08:46:49.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459fd09640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4598110ec0 con 0x7f4598106a90 2024-09-10T08:46:49.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459d2a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4598101430 0x7f45981988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459d2a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4598101430 0x7f45981988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57780/0 (socket says 172.21.15.172:57780) 2024-09-10T08:46:49.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459d2a4640 1 -- 172.21.15.172:0/3427193481 learned_addr learned my addr 172.21.15.172:0/3427193481 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:49.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459daa5640 1 --2- 172.21.15.172:0/3427193481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4598100a90 0x7f4598198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459e2a6640 1 --2- 172.21.15.172:0/3427193481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4598106a90 0x7f459819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459d2a4640 1 -- 172.21.15.172:0/3427193481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4598106a90 msgr2=0x7f459819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459d2a4640 1 --2- 172.21.15.172:0/3427193481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4598106a90 0x7f459819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459d2a4640 1 -- 172.21.15.172:0/3427193481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4598100a90 msgr2=0x7f4598198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459d2a4640 1 --2- 172.21.15.172:0/3427193481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4598100a90 0x7f4598198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459d2a4640 1 -- 172.21.15.172:0/3427193481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459819d310 con 0x7f4598101430 2024-09-10T08:46:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.062+0000 7f459daa5640 1 --2- 172.21.15.172:0/3427193481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4598100a90 0x7f4598198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.063+0000 7f459d2a4640 1 --2- 172.21.15.172:0/3427193481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4598101430 0x7f45981988a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f459400d950 tx=0x7f459400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.063+0000 7f458affd640 1 -- 172.21.15.172:0/3427193481 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594017070 con 0x7f4598101430 2024-09-10T08:46:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.063+0000 7f459fd09640 1 -- 172.21.15.172:0/3427193481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f459819d600 con 0x7f4598101430 2024-09-10T08:46:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.063+0000 7f458affd640 1 -- 172.21.15.172:0/3427193481 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4594009c30 con 0x7f4598101430 2024-09-10T08:46:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.063+0000 7f458affd640 1 -- 172.21.15.172:0/3427193481 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459400f620 con 0x7f4598101430 2024-09-10T08:46:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.063+0000 7f459fd09640 1 -- 172.21.15.172:0/3427193481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45981a4ee0 con 0x7f4598101430 2024-09-10T08:46:49.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.065+0000 7f458affd640 1 -- 172.21.15.172:0/3427193481 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f45940109a0 con 0x7f4598101430 2024-09-10T08:46:49.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.066+0000 7f45a20e2740 1 Processor -- start 2024-09-10T08:46:49.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.066+0000 7f45a20e2740 1 -- start start 2024-09-10T08:46:49.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.066+0000 7f458affd640 1 --2- 172.21.15.172:0/3427193481 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f45840a2b80 0x7f45840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.066+0000 7f45a20e2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3251584a0 0x55e325158880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.066+0000 7f45a20e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325158dc0 0x55e325163140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.066+0000 7f45a20e2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e325163680 0x55e325165a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.066+0000 7f45a20e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e3250be2e0 con 0x55e325158dc0 2024-09-10T08:46:49.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.066+0000 7f45a20e2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e325065ce0 con 0x55e325163680 2024-09-10T08:46:49.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.066+0000 7f45a20e2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e3250ba2a0 con 0x55e3251584a0 2024-09-10T08:46:49.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.066+0000 7f459e2a6640 1 --2- 172.21.15.172:0/3427193481 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f45840a2b80 0x7f45840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.066+0000 7f459d2a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325158dc0 0x55e325163140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.066+0000 7f458affd640 1 -- 172.21.15.172:0/3427193481 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f45940cf0a0 con 0x7f4598101430 2024-09-10T08:46:49.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.066+0000 7f459d2a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325158dc0 0x55e325163140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57784/0 (socket says 172.21.15.172:57784) 2024-09-10T08:46:49.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.066+0000 7f459d2a4640 1 -- 172.21.15.172:0/3368783689 learned_addr learned my addr 172.21.15.172:0/3368783689 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:49.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.067+0000 7f459d2a4640 1 -- 172.21.15.172:0/3368783689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3251584a0 msgr2=0x55e325158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:49.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.067+0000 7f459d2a4640 1 --2- 172.21.15.172:0/3368783689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3251584a0 0x55e325158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.067+0000 7f459d2a4640 1 -- 172.21.15.172:0/3368783689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e325163680 msgr2=0x55e325165a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:49.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.067+0000 7f459d2a4640 1 --2- 172.21.15.172:0/3368783689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e325163680 0x55e325165a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.067+0000 7f459d2a4640 1 -- 172.21.15.172:0/3368783689 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e32504a440 con 0x55e325158dc0 2024-09-10T08:46:49.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.067+0000 7f459e2a6640 1 --2- 172.21.15.172:0/3427193481 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f45840a2b80 0x7f45840a5040 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4590006010 tx=0x7f4590009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:49.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.067+0000 7f459d2a4640 1 --2- 172.21.15.172:0/3368783689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325158dc0 0x55e325163140 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4594097a00 tx=0x7f459409af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b150142d3f2de2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:49.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.067+0000 7f4583fff640 1 -- 172.21.15.172:0/3368783689 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594020040 con 0x55e325158dc0 2024-09-10T08:46:49.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.067+0000 7f4583fff640 1 -- 172.21.15.172:0/3368783689 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f459409bca0 con 0x55e325158dc0 2024-09-10T08:46:49.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.067+0000 7f4583fff640 1 -- 172.21.15.172:0/3368783689 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45940a4210 con 0x55e325158dc0 2024-09-10T08:46:49.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.068+0000 7f45a20e2740 1 -- 172.21.15.172:0/3368783689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325158dc0 msgr2=0x55e325163140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.068+0000 7f45a20e2740 1 --2- 172.21.15.172:0/3368783689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325158dc0 0x55e325163140 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4594097a00 tx=0x7f459409af00 comp rx=0 tx=0).stop 2024-09-10T08:46:49.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.068+0000 7f45a20e2740 1 -- 172.21.15.172:0/3368783689 shutdown_connections 2024-09-10T08:46:49.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.068+0000 7f45a20e2740 1 --2- 172.21.15.172:0/3368783689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e325163680 0x55e325165a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.068+0000 7f45a20e2740 1 --2- 172.21.15.172:0/3368783689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325158dc0 0x55e325163140 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.068+0000 7f45a20e2740 1 --2- 172.21.15.172:0/3368783689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3251584a0 0x55e325158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.068+0000 7f45a20e2740 1 -- 172.21.15.172:0/3368783689 >> 172.21.15.172:0/3368783689 conn(0x55e325154a30 msgr2=0x55e325154e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:49.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.068+0000 7f45a20e2740 1 -- 172.21.15.172:0/3368783689 shutdown_connections 2024-09-10T08:46:49.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.068+0000 7f45a20e2740 1 -- 172.21.15.172:0/3368783689 wait complete. 2024-09-10T08:46:49.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.068+0000 7f45a20e2740 1 -- start start 2024-09-10T08:46:49.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.069+0000 7f45a20e2740 1 Processor -- start 2024-09-10T08:46:49.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.069+0000 7f45a20e2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3251584a0 0x55e325160490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.069+0000 7f45a20e2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e325163680 0x55e3251609d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.069+0000 7f45a20e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325160f10 0x55e32516f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.069+0000 7f45a20e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e325166250 con 0x55e325160f10 2024-09-10T08:46:49.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.069+0000 7f45a20e2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e325079050 con 0x55e3251584a0 2024-09-10T08:46:49.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.069+0000 7f45a20e2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e3251663d0 con 0x55e325163680 2024-09-10T08:46:49.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.069+0000 7f45a20e2740 10 client.0 _ll_register_callbacks cb 0x55e3250b9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:49.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.069+0000 7f459e2a6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325160f10 0x55e32516f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.069+0000 7f459e2a6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325160f10 0x55e32516f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57796/0 (socket says 172.21.15.172:57796) 2024-09-10T08:46:49.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.069+0000 7f459e2a6640 1 -- 172.21.15.172:0/3098776722 learned_addr learned my addr 172.21.15.172:0/3098776722 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:49.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.069+0000 7f459d2a4640 1 --2- 172.21.15.172:0/3098776722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e325163680 0x55e3251609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.069+0000 7f459daa5640 1 --2- 172.21.15.172:0/3098776722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3251584a0 0x55e325160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.070+0000 7f459e2a6640 1 -- 172.21.15.172:0/3098776722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e325163680 msgr2=0x55e3251609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:49.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.070+0000 7f459e2a6640 1 --2- 172.21.15.172:0/3098776722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e325163680 0x55e3251609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.070+0000 7f459e2a6640 1 -- 172.21.15.172:0/3098776722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3251584a0 msgr2=0x55e325160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:49.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.070+0000 7f459e2a6640 1 --2- 172.21.15.172:0/3098776722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3251584a0 0x55e325160490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.070+0000 7f459e2a6640 1 -- 172.21.15.172:0/3098776722 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e324e2f3c0 con 0x55e325160f10 2024-09-10T08:46:49.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.070+0000 7f459e2a6640 1 --2- 172.21.15.172:0/3098776722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325160f10 0x55e32516f6c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4590031760 tx=0x7f45900095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea060548ba7c90d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:49.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.070+0000 7f45817fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:49.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.070+0000 7f45817fa640 1 -- 172.21.15.172:0/3098776722 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459004c070 con 0x55e325160f10 2024-09-10T08:46:49.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.070+0000 7f45817fa640 1 -- 172.21.15.172:0/3098776722 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4590004740 con 0x55e325160f10 2024-09-10T08:46:49.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.070+0000 7f45a20e2740 1 -- 172.21.15.172:0/3098776722 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f459409bca0 con 0x55e325160f10 2024-09-10T08:46:49.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.070+0000 7f45817fa640 1 -- 172.21.15.172:0/3098776722 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4590058050 con 0x55e325160f10 2024-09-10T08:46:49.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.070+0000 7f45a20e2740 10 client.15429 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:49.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.070+0000 7f45a20e2740 1 -- 172.21.15.172:0/3098776722 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e325176600 con 0x55e325160f10 2024-09-10T08:46:49.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.071+0000 7f45817fa640 1 -- 172.21.15.172:0/3098776722 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f459409bca0 con 0x55e325160f10 2024-09-10T08:46:49.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.071+0000 7f45817fa640 1 -- 172.21.15.172:0/3098776722 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4590030070 con 0x55e325160f10 2024-09-10T08:46:49.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.071+0000 7f45a20e2740 10 client.15429 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:49.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.071+0000 7f45a20e2740 10 client.15429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:46:49.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.071+0000 7f45a20e2740 20 client.15429 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:49.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.071+0000 7f45a20e2740 1 --2- 172.21.15.172:0/3098776722 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e325177a10 0x55e325179e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.071+0000 7f45a20e2740 4 client.15429 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:49.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.071+0000 7f45a20e2740 1 -- 172.21.15.172:0/3098776722 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3250cb850 con 0x55e325177a10 2024-09-10T08:46:49.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.071+0000 7f459daa5640 1 --2- 172.21.15.172:0/3098776722 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e325177a10 0x55e325179e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.072+0000 7f45817fa640 10 client.15429 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:49.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.072+0000 7f459daa5640 1 --2- 172.21.15.172:0/3098776722 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e325177a10 0x55e325179e20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9bf98a75c8501f server_cookie=61ac769f7c8f950a in_seq=0 out_seq=0 2024-09-10T08:46:49.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.072+0000 7f45817fa640 1 -- 172.21.15.172:0/3098776722 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3250cb850 con 0x55e325177a10 2024-09-10T08:46:49.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.072+0000 7f45817fa640 10 client.15429 handle_command_reply: tid=0 2024-09-10T08:46:49.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.072+0000 7f45a20e2740 1 client.15429 shutdown 2024-09-10T08:46:49.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.072+0000 7f45a20e2740 1 -- 172.21.15.172:0/3098776722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325160f10 msgr2=0x55e32516f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.072+0000 7f45a20e2740 1 --2- 172.21.15.172:0/3098776722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325160f10 0x55e32516f6c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4590031760 tx=0x7f45900095c0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.073+0000 7f45a20e2740 1 -- 172.21.15.172:0/3098776722 shutdown_connections 2024-09-10T08:46:49.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.073+0000 7f45a20e2740 1 --2- 172.21.15.172:0/3098776722 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e325177a10 0x55e325179e20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.073+0000 7f45a20e2740 1 --2- 172.21.15.172:0/3098776722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325160f10 0x55e32516f6c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.073+0000 7f45a20e2740 1 --2- 172.21.15.172:0/3098776722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e325163680 0x55e3251609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.073+0000 7f45817fa640 0 client.15429 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:49.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.073+0000 7f45a20e2740 1 --2- 172.21.15.172:0/3098776722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3251584a0 0x55e325160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.073+0000 7f45a20e2740 1 -- 172.21.15.172:0/3098776722 >> 172.21.15.172:0/3098776722 conn(0x55e325154a30 msgr2=0x55e32515a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:49.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.073+0000 7f45a20e2740 1 -- 172.21.15.172:0/3098776722 shutdown_connections 2024-09-10T08:46:49.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.073+0000 7f45a20e2740 1 -- 172.21.15.172:0/3098776722 wait complete. 2024-09-10T08:46:49.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.073+0000 7f45a20e2740 20 client.15429 trim_cache size 0 max 16384 2024-09-10T08:46:49.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f45a20e2740 1 Processor -- start 2024-09-10T08:46:49.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f45a20e2740 1 -- start start 2024-09-10T08:46:49.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f45a20e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325154a30 0x55e32516f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f45a20e2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3251584a0 0x55e32516f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f45a20e2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e325160f10 0x55e32517f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f45a20e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f45981ac8a0 con 0x55e325154a30 2024-09-10T08:46:49.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f45a20e2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f45981ae3e0 con 0x55e3251584a0 2024-09-10T08:46:49.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f45a20e2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f45981acf70 con 0x55e325160f10 2024-09-10T08:46:49.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f459daa5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325154a30 0x55e32516f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f459daa5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325154a30 0x55e32516f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57804/0 (socket says 172.21.15.172:57804) 2024-09-10T08:46:49.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f459daa5640 1 -- 172.21.15.172:0/1988716456 learned_addr learned my addr 172.21.15.172:0/1988716456 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:49.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f459d2a4640 1 --2- 172.21.15.172:0/1988716456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3251584a0 0x55e32516f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f459daa5640 1 -- 172.21.15.172:0/1988716456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e325160f10 msgr2=0x55e32517f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:49.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f459daa5640 1 --2- 172.21.15.172:0/1988716456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e325160f10 0x55e32517f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f459daa5640 1 -- 172.21.15.172:0/1988716456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3251584a0 msgr2=0x55e32516f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:49.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f459daa5640 1 --2- 172.21.15.172:0/1988716456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3251584a0 0x55e32516f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f459daa5640 1 -- 172.21.15.172:0/1988716456 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3251556c0 con 0x55e325154a30 2024-09-10T08:46:49.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.096+0000 7f459d2a4640 1 --2- 172.21.15.172:0/1988716456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3251584a0 0x55e32516f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:49.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.097+0000 7f459daa5640 1 --2- 172.21.15.172:0/1988716456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325154a30 0x55e32516f4b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f458c00b780 tx=0x7f458c005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc601e05a3288e27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:49.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.097+0000 7f45837fe640 1 -- 172.21.15.172:0/1988716456 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c005030 con 0x55e325154a30 2024-09-10T08:46:49.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.097+0000 7f45837fe640 1 -- 172.21.15.172:0/1988716456 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f458c004730 con 0x55e325154a30 2024-09-10T08:46:49.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.097+0000 7f45837fe640 1 -- 172.21.15.172:0/1988716456 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c01a590 con 0x55e325154a30 2024-09-10T08:46:49.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.098+0000 7f45a20e2740 1 -- 172.21.15.172:0/1988716456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325154a30 msgr2=0x55e32516f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.098+0000 7f45a20e2740 1 --2- 172.21.15.172:0/1988716456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325154a30 0x55e32516f4b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f458c00b780 tx=0x7f458c005d40 comp rx=0 tx=0).stop 2024-09-10T08:46:49.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.098+0000 7f45a20e2740 1 -- 172.21.15.172:0/1988716456 shutdown_connections 2024-09-10T08:46:49.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.098+0000 7f45a20e2740 1 --2- 172.21.15.172:0/1988716456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e325160f10 0x55e32517f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.098+0000 7f45a20e2740 1 --2- 172.21.15.172:0/1988716456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3251584a0 0x55e32516f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.098+0000 7f45a20e2740 1 --2- 172.21.15.172:0/1988716456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e325154a30 0x55e32516f4b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.098+0000 7f45a20e2740 1 -- 172.21.15.172:0/1988716456 >> 172.21.15.172:0/1988716456 conn(0x55e325160b30 msgr2=0x55e325162510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:49.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.098+0000 7f45a20e2740 1 -- 172.21.15.172:0/1988716456 shutdown_connections 2024-09-10T08:46:49.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.098+0000 7f45a20e2740 1 -- 172.21.15.172:0/1988716456 wait complete. 2024-09-10T08:46:49.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.098+0000 7f45a20e2740 1 -- start start 2024-09-10T08:46:49.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.098+0000 7f45a20e2740 1 Processor -- start 2024-09-10T08:46:49.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f45a20e2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e325154a30 0x55e325167630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f45a20e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3251584a0 0x55e325167b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f45a20e2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e325160f10 0x55e3251680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f45a20e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f45981add10 con 0x55e3251584a0 2024-09-10T08:46:49.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f45a20e2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f45981ad640 con 0x55e325154a30 2024-09-10T08:46:49.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f45a20e2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f45981aa880 con 0x55e325160f10 2024-09-10T08:46:49.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f45a20e2740 10 client.0 _ll_register_callbacks cb 0x7f4590041050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:49.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f459d2a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3251584a0 0x55e325167b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f459d2a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3251584a0 0x55e325167b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57806/0 (socket says 172.21.15.172:57806) 2024-09-10T08:46:49.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f459d2a4640 1 -- 172.21.15.172:0/480238879 learned_addr learned my addr 172.21.15.172:0/480238879 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:49.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f459daa5640 1 --2- 172.21.15.172:0/480238879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e325154a30 0x55e325167630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f459e2a6640 1 --2- 172.21.15.172:0/480238879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e325160f10 0x55e3251680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f459d2a4640 1 -- 172.21.15.172:0/480238879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e325160f10 msgr2=0x55e3251680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:49.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f459d2a4640 1 --2- 172.21.15.172:0/480238879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e325160f10 0x55e3251680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f459d2a4640 1 -- 172.21.15.172:0/480238879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e325154a30 msgr2=0x55e325167630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:49.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f459d2a4640 1 --2- 172.21.15.172:0/480238879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e325154a30 0x55e325167630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f459d2a4640 1 -- 172.21.15.172:0/480238879 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3251e19b0 con 0x55e3251584a0 2024-09-10T08:46:49.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f459d2a4640 1 --2- 172.21.15.172:0/480238879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3251584a0 0x55e325167b70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f459409b2a0 tx=0x7f459409b030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93db4392f0f824b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:49.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.099+0000 7f45817fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:49.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.100+0000 7f45817fa640 1 -- 172.21.15.172:0/480238879 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594020040 con 0x55e3251584a0 2024-09-10T08:46:49.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.100+0000 7f45817fa640 1 -- 172.21.15.172:0/480238879 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f45940998a0 con 0x55e3251584a0 2024-09-10T08:46:49.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.100+0000 7f45a20e2740 1 -- 172.21.15.172:0/480238879 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f458c004730 con 0x55e3251584a0 2024-09-10T08:46:49.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.100+0000 7f45817fa640 1 -- 172.21.15.172:0/480238879 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4594013050 con 0x55e3251584a0 2024-09-10T08:46:49.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.100+0000 7f45a20e2740 10 client.15435 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:49.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.100+0000 7f45a20e2740 1 -- 172.21.15.172:0/480238879 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e3251e2c30 con 0x55e3251584a0 2024-09-10T08:46:49.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.100+0000 7f45817fa640 1 -- 172.21.15.172:0/480238879 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f458c004730 con 0x55e3251584a0 2024-09-10T08:46:49.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.100+0000 7f45817fa640 1 -- 172.21.15.172:0/480238879 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f459409ab80 con 0x55e3251584a0 2024-09-10T08:46:49.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.100+0000 7f45a20e2740 10 client.15435 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:49.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.100+0000 7f45a20e2740 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:46:49.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.100+0000 7f45a20e2740 20 client.15435 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:49.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.100+0000 7f45a20e2740 1 --2- 172.21.15.172:0/480238879 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e325163680 0x55e3251e4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:49.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.100+0000 7f45a20e2740 4 client.15435 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:46:49.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.100+0000 7f45a20e2740 1 -- 172.21.15.172:0/480238879 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e3250c5d60 con 0x55e325163680 2024-09-10T08:46:49.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.101+0000 7f459daa5640 1 --2- 172.21.15.172:0/480238879 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e325163680 0x55e3251e4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:49.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.101+0000 7f45817fa640 10 client.15435 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:49.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.101+0000 7f459daa5640 1 --2- 172.21.15.172:0/480238879 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e325163680 0x55e3251e4e10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa2718d65106c43c server_cookie=8280e8beb05f13f9 in_seq=0 out_seq=0 2024-09-10T08:46:49.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.101+0000 7f45817fa640 1 -- 172.21.15.172:0/480238879 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e3250c5d60 con 0x55e325163680 2024-09-10T08:46:49.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.101+0000 7f45817fa640 10 client.15435 handle_command_reply: tid=0 2024-09-10T08:46:49.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.101+0000 7f45a20e2740 1 client.15435 shutdown 2024-09-10T08:46:49.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.102+0000 7f45a20e2740 1 -- 172.21.15.172:0/480238879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3251584a0 msgr2=0x55e325167b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.102+0000 7f45a20e2740 1 --2- 172.21.15.172:0/480238879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3251584a0 0x55e325167b70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f459409b2a0 tx=0x7f459409b030 comp rx=0 tx=0).stop 2024-09-10T08:46:49.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.102+0000 7f45a20e2740 1 -- 172.21.15.172:0/480238879 shutdown_connections 2024-09-10T08:46:49.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.102+0000 7f45a20e2740 1 --2- 172.21.15.172:0/480238879 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e325163680 0x55e3251e4e10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.102+0000 7f45a20e2740 1 --2- 172.21.15.172:0/480238879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e325160f10 0x55e3251680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.102+0000 7f45a20e2740 1 --2- 172.21.15.172:0/480238879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3251584a0 0x55e325167b70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.102+0000 7f45a20e2740 1 --2- 172.21.15.172:0/480238879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e325154a30 0x55e325167630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.102+0000 7f45817fa640 0 client.15435 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:49.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.102+0000 7f45a20e2740 1 -- 172.21.15.172:0/480238879 >> 172.21.15.172:0/480238879 conn(0x55e325160b30 msgr2=0x55e325153350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:49.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.102+0000 7f45a20e2740 1 -- 172.21.15.172:0/480238879 shutdown_connections 2024-09-10T08:46:49.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.102+0000 7f45a20e2740 1 -- 172.21.15.172:0/480238879 wait complete. 2024-09-10T08:46:49.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.102+0000 7f45a20e2740 20 client.15435 trim_cache size 0 max 16384 2024-09-10T08:46:49.108 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:46:49.109 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:46:49.109 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T08:46:49.109 INFO:teuthology.orchestra.run.smithi172.stdout: "e61817eb-66d0-4ac0-808a-76f60fbc6235": { 2024-09-10T08:46:49.109 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T08:46:49.109 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "e61817eb-66d0-4ac0-808a-76f60fbc6235", 2024-09-10T08:46:49.109 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T08:46:49.109 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:46:49.109 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:46:49.109 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:46:49.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.103+0000 7f459fd09640 1 -- 172.21.15.172:0/3427193481 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f45840a2b80 msgr2=0x7f45840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.103+0000 7f459fd09640 1 --2- 172.21.15.172:0/3427193481 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f45840a2b80 0x7f45840a5040 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4590006010 tx=0x7f4590009920 comp rx=0 tx=0).stop 2024-09-10T08:46:49.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.103+0000 7f459fd09640 1 -- 172.21.15.172:0/3427193481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4598101430 msgr2=0x7f45981988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:49.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.103+0000 7f459fd09640 1 --2- 172.21.15.172:0/3427193481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4598101430 0x7f45981988a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f459400d950 tx=0x7f459400de10 comp rx=0 tx=0).stop 2024-09-10T08:46:49.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.103+0000 7f459fd09640 1 -- 172.21.15.172:0/3427193481 shutdown_connections 2024-09-10T08:46:49.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.103+0000 7f459fd09640 1 --2- 172.21.15.172:0/3427193481 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f45840a2b80 0x7f45840a5040 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.103+0000 7f459fd09640 1 --2- 172.21.15.172:0/3427193481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4598106a90 0x7f459819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.103+0000 7f459fd09640 1 --2- 172.21.15.172:0/3427193481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4598101430 0x7f45981988a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.103+0000 7f459fd09640 1 --2- 172.21.15.172:0/3427193481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4598100a90 0x7f4598198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:49.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.103+0000 7f459fd09640 1 -- 172.21.15.172:0/3427193481 >> 172.21.15.172:0/3427193481 conn(0x7f45980fc7d0 msgr2=0x7f459806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:49.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.103+0000 7f459fd09640 1 -- 172.21.15.172:0/3427193481 shutdown_connections 2024-09-10T08:46:49.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:49.103+0000 7f459fd09640 1 -- 172.21.15.172:0/3427193481 wait complete. 2024-09-10T08:46:49.474 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:49 smithi172 ceph-mon[30854]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:49 smithi191 ceph-mon[35314]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:49.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:49 smithi174 ceph-mon[34068]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:49.816 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:49.824 INFO:tasks.cephfs.filesystem:scrub status for tag:e61817eb-66d0-4ac0-808a-76f60fbc6235 - {'path': '/', 'tag': 'e61817eb-66d0-4ac0-808a-76f60fbc6235', 'options': 'recursive,force'} 2024-09-10T08:46:50.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.275+0000 7f1625cce640 1 -- 172.21.15.172:0/4281935466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1620100e20 msgr2=0x7f1620101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:50.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.275+0000 7f1625cce640 1 --2- 172.21.15.172:0/4281935466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1620100e20 0x7f1620101200 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1610009900 tx=0x7f161002a480 comp rx=0 tx=0).stop 2024-09-10T08:46:50.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.275+0000 7f1625cce640 1 -- 172.21.15.172:0/4281935466 shutdown_connections 2024-09-10T08:46:50.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.275+0000 7f1625cce640 1 --2- 172.21.15.172:0/4281935466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1620102ef0 0x7f162010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:50.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.275+0000 7f1625cce640 1 --2- 172.21.15.172:0/4281935466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1620102570 0x7f16201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:50.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.275+0000 7f1625cce640 1 --2- 172.21.15.172:0/4281935466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1620100e20 0x7f1620101200 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:50.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.275+0000 7f1625cce640 1 -- 172.21.15.172:0/4281935466 >> 172.21.15.172:0/4281935466 conn(0x7f16200fc7d0 msgr2=0x7f16200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:50.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.275+0000 7f1625cce640 1 -- 172.21.15.172:0/4281935466 shutdown_connections 2024-09-10T08:46:50.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.275+0000 7f1625cce640 1 -- 172.21.15.172:0/4281935466 wait complete. 2024-09-10T08:46:50.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.276+0000 7f1625cce640 1 Processor -- start 2024-09-10T08:46:50.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.276+0000 7f1625cce640 1 -- start start 2024-09-10T08:46:50.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.276+0000 7f1625cce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1620100e20 0x7f16201972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:50.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.276+0000 7f1625cce640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1620102570 0x7f16201977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:50.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.276+0000 7f1625cce640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1620102ef0 0x7f162019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:50.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.276+0000 7f1625cce640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1620110e30 con 0x7f1620100e20 2024-09-10T08:46:50.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.276+0000 7f1625cce640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1620110cb0 con 0x7f1620102ef0 2024-09-10T08:46:50.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.276+0000 7f1625cce640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1620110fb0 con 0x7f1620102570 2024-09-10T08:46:50.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.276+0000 7f161f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1620100e20 0x7f16201972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:50.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.276+0000 7f161f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1620100e20 0x7f16201972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57836/0 (socket says 172.21.15.172:57836) 2024-09-10T08:46:50.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.276+0000 7f161f7fe640 1 -- 172.21.15.172:0/725862111 learned_addr learned my addr 172.21.15.172:0/725862111 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:50.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.276+0000 7f1616ffd640 1 --2- 172.21.15.172:0/725862111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1620102570 0x7f16201977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:50.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.276+0000 7f161f7fe640 1 -- 172.21.15.172:0/725862111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1620102570 msgr2=0x7f16201977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:50.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.276+0000 7f161f7fe640 1 --2- 172.21.15.172:0/725862111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1620102570 0x7f16201977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:50.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.276+0000 7f161f7fe640 1 -- 172.21.15.172:0/725862111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1620102ef0 msgr2=0x7f162019bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:50.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.276+0000 7f161f7fe640 1 --2- 172.21.15.172:0/725862111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1620102ef0 0x7f162019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:50.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.276+0000 7f161f7fe640 1 -- 172.21.15.172:0/725862111 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162019c250 con 0x7f1620100e20 2024-09-10T08:46:50.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.277+0000 7f161f7fe640 1 --2- 172.21.15.172:0/725862111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1620100e20 0x7f16201972a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1610006010 tx=0x7f16100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:50.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.277+0000 7f161d7fa640 1 -- 172.21.15.172:0/725862111 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161003f070 con 0x7f1620100e20 2024-09-10T08:46:50.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.277+0000 7f161d7fa640 1 -- 172.21.15.172:0/725862111 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1610003aa0 con 0x7f1620100e20 2024-09-10T08:46:50.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.277+0000 7f1625cce640 1 -- 172.21.15.172:0/725862111 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162019c4e0 con 0x7f1620100e20 2024-09-10T08:46:50.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.277+0000 7f161d7fa640 1 -- 172.21.15.172:0/725862111 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610035030 con 0x7f1620100e20 2024-09-10T08:46:50.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.277+0000 7f1625cce640 1 -- 172.21.15.172:0/725862111 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16201a3e20 con 0x7f1620100e20 2024-09-10T08:46:50.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.278+0000 7f1625cce640 1 -- 172.21.15.172:0/725862111 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16201021b0 con 0x7f1620100e20 2024-09-10T08:46:50.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.279+0000 7f161d7fa640 1 -- 172.21.15.172:0/725862111 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1610042070 con 0x7f1620100e20 2024-09-10T08:46:50.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.280+0000 7f161d7fa640 1 --2- 172.21.15.172:0/725862111 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f160c0a2b80 0x7f160c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:50.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.281+0000 7f1616ffd640 1 --2- 172.21.15.172:0/725862111 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f160c0a2b80 0x7f160c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:50.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.281+0000 7f161d7fa640 1 -- 172.21.15.172:0/725862111 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f16100efff0 con 0x7f1620100e20 2024-09-10T08:46:50.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.281+0000 7f1616ffd640 1 --2- 172.21.15.172:0/725862111 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f160c0a2b80 0x7f160c0a5040 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f16200ffea0 tx=0x7f160800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:50.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.284+0000 7f161d7fa640 1 -- 172.21.15.172:0/725862111 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16100bcbf0 con 0x7f1620100e20 2024-09-10T08:46:50.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:50 smithi172 ceph-mon[30854]: pgmap v368: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T08:46:50.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.450+0000 7f1625cce640 1 -- 172.21.15.172:0/725862111 --> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1620062630 con 0x7f160c0a2b80 2024-09-10T08:46:50.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.458+0000 7f161d7fa640 1 -- 172.21.15.172:0/725862111 <== mgr.14150 v2:172.21.15.172:6800/107199760 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340525 (secure 0 0 0) 0x7f1620062630 con 0x7f160c0a2b80 2024-09-10T08:46:50.462 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:50.463 INFO:teuthology.orchestra.run.smithi172.stderr:dumped all 2024-09-10T08:46:50.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.461+0000 7f1625cce640 1 -- 172.21.15.172:0/725862111 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f160c0a2b80 msgr2=0x7f160c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:50.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.461+0000 7f1625cce640 1 --2- 172.21.15.172:0/725862111 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f160c0a2b80 0x7f160c0a5040 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f16200ffea0 tx=0x7f160800a4c0 comp rx=0 tx=0).stop 2024-09-10T08:46:50.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.461+0000 7f1625cce640 1 -- 172.21.15.172:0/725862111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1620100e20 msgr2=0x7f16201972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:50.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.461+0000 7f1625cce640 1 --2- 172.21.15.172:0/725862111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1620100e20 0x7f16201972a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1610006010 tx=0x7f16100042f0 comp rx=0 tx=0).stop 2024-09-10T08:46:50.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.461+0000 7f1625cce640 1 -- 172.21.15.172:0/725862111 shutdown_connections 2024-09-10T08:46:50.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.461+0000 7f1625cce640 1 --2- 172.21.15.172:0/725862111 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f160c0a2b80 0x7f160c0a5040 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:50.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.461+0000 7f1625cce640 1 --2- 172.21.15.172:0/725862111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1620102ef0 0x7f162019bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:50.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.461+0000 7f1625cce640 1 --2- 172.21.15.172:0/725862111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1620102570 0x7f16201977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:50.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.461+0000 7f1625cce640 1 --2- 172.21.15.172:0/725862111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1620100e20 0x7f16201972a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:50.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.461+0000 7f1625cce640 1 -- 172.21.15.172:0/725862111 >> 172.21.15.172:0/725862111 conn(0x7f16200fc7d0 msgr2=0x7f16200fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:50.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.461+0000 7f1625cce640 1 -- 172.21.15.172:0/725862111 shutdown_connections 2024-09-10T08:46:50.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:50.461+0000 7f1625cce640 1 -- 172.21.15.172:0/725862111 wait complete. 2024-09-10T08:46:50.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:50 smithi191 ceph-mon[35314]: pgmap v368: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T08:46:50.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:50 smithi174 ceph-mon[34068]: pgmap v368: 129 pgs: 129 active+clean; 652 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T08:46:51.132 INFO:teuthology.orchestra.run.smithi172.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-10T08:46:49.166802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668080,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1296,"num_read_kb":780,"num_write":459,"num_write_kb":3107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":874,"ondisk_log_size":874,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376268,"kb_used_data":14540,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565556,"statfs":{"total":1151940427776,"available":1151555129344,"internally_reserved":0,"allocated":14888960,"data_stored":8243218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4809,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":63,"num_write":7,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003220"},"pg_stats":[{"pgid":"2.1f","version":"92'14","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:45.379095+0000","last_change":"2024-09-10T08:44:08.657737+0000","last_active":"2024-09-10T08:46:45.379095+0000","last_peered":"2024-09-10T08:46:45.379095+0000","last_clean":"2024-09-10T08:46:45.379095+0000","last_became_active":"2024-09-10T08:44:08.657473+0000","last_became_peered":"2024-09-10T08:44:08.657473+0000","last_unstale":"2024-09-10T08:46:45.379095+0000","last_undegraded":"2024-09-10T08:46:45.379095+0000","last_fullsized":"2024-09-10T08:46:45.379095+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:40.757343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193599+0000","last_change":"2024-09-10T08:44:10.675090+0000","last_active":"2024-09-10T08:46:19.193599+0000","last_peered":"2024-09-10T08:46:19.193599+0000","last_clean":"2024-09-10T08:46:19.193599+0000","last_became_active":"2024-09-10T08:44:10.674612+0000","last_became_peered":"2024-09-10T08:44:10.674612+0000","last_unstale":"2024-09-10T08:46:19.193599+0000","last_undegraded":"2024-09-10T08:46:19.193599+0000","last_fullsized":"2024-09-10T08:46:19.193599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:46.821080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.807967+0000","last_change":"2024-09-10T08:44:17.721559+0000","last_active":"2024-09-10T08:46:18.807967+0000","last_peered":"2024-09-10T08:46:18.807967+0000","last_clean":"2024-09-10T08:46:18.807967+0000","last_became_active":"2024-09-10T08:44:17.721182+0000","last_became_peered":"2024-09-10T08:44:17.721182+0000","last_unstale":"2024-09-10T08:46:18.807967+0000","last_undegraded":"2024-09-10T08:46:18.807967+0000","last_fullsized":"2024-09-10T08:46:18.807967+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:18.791489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193663+0000","last_change":"2024-09-10T08:44:09.049171+0000","last_active":"2024-09-10T08:46:19.193663+0000","last_peered":"2024-09-10T08:46:19.193663+0000","last_clean":"2024-09-10T08:46:19.193663+0000","last_became_active":"2024-09-10T08:44:09.048806+0000","last_became_peered":"2024-09-10T08:44:09.048806+0000","last_unstale":"2024-09-10T08:46:19.193663+0000","last_undegraded":"2024-09-10T08:46:19.193663+0000","last_fullsized":"2024-09-10T08:46:19.193663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:21.023460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193866+0000","last_change":"2024-09-10T08:44:10.677569+0000","last_active":"2024-09-10T08:46:19.193866+0000","last_peered":"2024-09-10T08:46:19.193866+0000","last_clean":"2024-09-10T08:46:19.193866+0000","last_became_active":"2024-09-10T08:44:10.677048+0000","last_became_peered":"2024-09-10T08:44:10.677048+0000","last_unstale":"2024-09-10T08:46:19.193866+0000","last_undegraded":"2024-09-10T08:46:19.193866+0000","last_fullsized":"2024-09-10T08:46:19.193866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:35.523063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193389+0000","last_change":"2024-09-10T08:44:17.720138+0000","last_active":"2024-09-10T08:46:19.193389+0000","last_peered":"2024-09-10T08:46:19.193389+0000","last_clean":"2024-09-10T08:46:19.193389+0000","last_became_active":"2024-09-10T08:44:17.719344+0000","last_became_peered":"2024-09-10T08:44:17.719344+0000","last_unstale":"2024-09-10T08:46:19.193389+0000","last_undegraded":"2024-09-10T08:46:19.193389+0000","last_fullsized":"2024-09-10T08:46:19.193389+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:11.758058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194115+0000","last_change":"2024-09-10T08:44:08.660099+0000","last_active":"2024-09-10T08:46:19.194115+0000","last_peered":"2024-09-10T08:46:19.194115+0000","last_clean":"2024-09-10T08:46:19.194115+0000","last_became_active":"2024-09-10T08:44:08.659745+0000","last_became_peered":"2024-09-10T08:44:08.659745+0000","last_unstale":"2024-09-10T08:46:19.194115+0000","last_undegraded":"2024-09-10T08:46:19.194115+0000","last_fullsized":"2024-09-10T08:46:19.194115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:43.960717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193791+0000","last_change":"2024-09-10T08:44:10.675583+0000","last_active":"2024-09-10T08:46:19.193791+0000","last_peered":"2024-09-10T08:46:19.193791+0000","last_clean":"2024-09-10T08:46:19.193791+0000","last_became_active":"2024-09-10T08:44:10.675296+0000","last_became_peered":"2024-09-10T08:44:10.675296+0000","last_unstale":"2024-09-10T08:46:19.193791+0000","last_undegraded":"2024-09-10T08:46:19.193791+0000","last_fullsized":"2024-09-10T08:46:19.193791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:31:51.147862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194363+0000","last_change":"2024-09-10T08:44:17.720104+0000","last_active":"2024-09-10T08:46:19.194363+0000","last_peered":"2024-09-10T08:46:19.194363+0000","last_clean":"2024-09-10T08:46:19.194363+0000","last_became_active":"2024-09-10T08:44:17.719607+0000","last_became_peered":"2024-09-10T08:44:17.719607+0000","last_unstale":"2024-09-10T08:46:19.194363+0000","last_undegraded":"2024-09-10T08:46:19.194363+0000","last_fullsized":"2024-09-10T08:46:19.194363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:52.976717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193719+0000","last_change":"2024-09-10T08:44:08.658588+0000","last_active":"2024-09-10T08:46:19.193719+0000","last_peered":"2024-09-10T08:46:19.193719+0000","last_clean":"2024-09-10T08:46:19.193719+0000","last_became_active":"2024-09-10T08:44:08.658053+0000","last_became_peered":"2024-09-10T08:44:08.658053+0000","last_unstale":"2024-09-10T08:46:19.193719+0000","last_undegraded":"2024-09-10T08:46:19.193719+0000","last_fullsized":"2024-09-10T08:46:19.193719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:26.744943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804775+0000","last_change":"2024-09-10T08:44:10.681978+0000","last_active":"2024-09-10T08:46:18.804775+0000","last_peered":"2024-09-10T08:46:18.804775+0000","last_clean":"2024-09-10T08:46:18.804775+0000","last_became_active":"2024-09-10T08:44:10.681600+0000","last_became_peered":"2024-09-10T08:44:10.681600+0000","last_unstale":"2024-09-10T08:46:18.804775+0000","last_undegraded":"2024-09-10T08:46:18.804775+0000","last_fullsized":"2024-09-10T08:46:18.804775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:37.190576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193813+0000","last_change":"2024-09-10T08:44:17.719784+0000","last_active":"2024-09-10T08:46:19.193813+0000","last_peered":"2024-09-10T08:46:19.193813+0000","last_clean":"2024-09-10T08:46:19.193813+0000","last_became_active":"2024-09-10T08:44:17.719182+0000","last_became_peered":"2024-09-10T08:44:17.719182+0000","last_unstale":"2024-09-10T08:46:19.193813+0000","last_undegraded":"2024-09-10T08:46:19.193813+0000","last_fullsized":"2024-09-10T08:46:19.193813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:21:15.237772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873709+0000","last_change":"2024-09-10T08:44:10.679069+0000","last_active":"2024-09-10T08:46:18.873709+0000","last_peered":"2024-09-10T08:46:18.873709+0000","last_clean":"2024-09-10T08:46:18.873709+0000","last_became_active":"2024-09-10T08:44:10.678597+0000","last_became_peered":"2024-09-10T08:44:10.678597+0000","last_unstale":"2024-09-10T08:46:18.873709+0000","last_undegraded":"2024-09-10T08:46:18.873709+0000","last_fullsized":"2024-09-10T08:46:18.873709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:52.693906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194153+0000","last_change":"2024-09-10T08:44:08.660886+0000","last_active":"2024-09-10T08:46:19.194153+0000","last_peered":"2024-09-10T08:46:19.194153+0000","last_clean":"2024-09-10T08:46:19.194153+0000","last_became_active":"2024-09-10T08:44:08.660478+0000","last_became_peered":"2024-09-10T08:44:08.660478+0000","last_unstale":"2024-09-10T08:46:19.194153+0000","last_undegraded":"2024-09-10T08:46:19.194153+0000","last_fullsized":"2024-09-10T08:46:19.194153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:41.785531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194323+0000","last_change":"2024-09-10T08:44:17.761929+0000","last_active":"2024-09-10T08:46:19.194323+0000","last_peered":"2024-09-10T08:46:19.194323+0000","last_clean":"2024-09-10T08:46:19.194323+0000","last_became_active":"2024-09-10T08:44:17.761440+0000","last_became_peered":"2024-09-10T08:44:17.761440+0000","last_unstale":"2024-09-10T08:46:19.194323+0000","last_undegraded":"2024-09-10T08:46:19.194323+0000","last_fullsized":"2024-09-10T08:46:19.194323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:37.817759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192895+0000","last_change":"2024-09-10T08:44:10.674071+0000","last_active":"2024-09-10T08:46:19.192895+0000","last_peered":"2024-09-10T08:46:19.192895+0000","last_clean":"2024-09-10T08:46:19.192895+0000","last_became_active":"2024-09-10T08:44:10.673770+0000","last_became_peered":"2024-09-10T08:44:10.673770+0000","last_unstale":"2024-09-10T08:46:19.192895+0000","last_undegraded":"2024-09-10T08:46:19.192895+0000","last_fullsized":"2024-09-10T08:46:19.192895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:44:19.944557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.231938+0000","last_change":"2024-09-10T08:45:18.264060+0000","last_active":"2024-09-10T08:46:34.231938+0000","last_peered":"2024-09-10T08:46:34.231938+0000","last_clean":"2024-09-10T08:46:34.231938+0000","last_became_active":"2024-09-10T08:45:18.255983+0000","last_became_peered":"2024-09-10T08:45:18.255983+0000","last_unstale":"2024-09-10T08:46:34.231938+0000","last_undegraded":"2024-09-10T08:46:34.231938+0000","last_fullsized":"2024-09-10T08:46:34.231938+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:10.810491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194248+0000","last_change":"2024-09-10T08:44:17.722877+0000","last_active":"2024-09-10T08:46:19.194248+0000","last_peered":"2024-09-10T08:46:19.194248+0000","last_clean":"2024-09-10T08:46:19.194248+0000","last_became_active":"2024-09-10T08:44:17.722046+0000","last_became_peered":"2024-09-10T08:44:17.722046+0000","last_unstale":"2024-09-10T08:46:19.194248+0000","last_undegraded":"2024-09-10T08:46:19.194248+0000","last_fullsized":"2024-09-10T08:46:19.194248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:32.675598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193398+0000","last_change":"2024-09-10T08:44:10.671500+0000","last_active":"2024-09-10T08:46:19.193398+0000","last_peered":"2024-09-10T08:46:19.193398+0000","last_clean":"2024-09-10T08:46:19.193398+0000","last_became_active":"2024-09-10T08:44:10.671039+0000","last_became_peered":"2024-09-10T08:44:10.671039+0000","last_unstale":"2024-09-10T08:46:19.193398+0000","last_undegraded":"2024-09-10T08:46:19.193398+0000","last_fullsized":"2024-09-10T08:46:19.193398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:12.977373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193090+0000","last_change":"2024-09-10T08:44:08.655467+0000","last_active":"2024-09-10T08:46:19.193090+0000","last_peered":"2024-09-10T08:46:19.193090+0000","last_clean":"2024-09-10T08:46:19.193090+0000","last_became_active":"2024-09-10T08:44:08.654991+0000","last_became_peered":"2024-09-10T08:44:08.654991+0000","last_unstale":"2024-09-10T08:46:19.193090+0000","last_undegraded":"2024-09-10T08:46:19.193090+0000","last_fullsized":"2024-09-10T08:46:19.193090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:16:33.124360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803728+0000","last_change":"2024-09-10T08:44:17.724250+0000","last_active":"2024-09-10T08:46:18.803728+0000","last_peered":"2024-09-10T08:46:18.803728+0000","last_clean":"2024-09-10T08:46:18.803728+0000","last_became_active":"2024-09-10T08:44:17.723733+0000","last_became_peered":"2024-09-10T08:44:17.723733+0000","last_unstale":"2024-09-10T08:46:18.803728+0000","last_undegraded":"2024-09-10T08:46:18.803728+0000","last_fullsized":"2024-09-10T08:46:18.803728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:40.098666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193144+0000","last_change":"2024-09-10T08:44:10.664159+0000","last_active":"2024-09-10T08:46:19.193144+0000","last_peered":"2024-09-10T08:46:19.193144+0000","last_clean":"2024-09-10T08:46:19.193144+0000","last_became_active":"2024-09-10T08:44:10.663674+0000","last_became_peered":"2024-09-10T08:44:10.663674+0000","last_unstale":"2024-09-10T08:46:19.193144+0000","last_undegraded":"2024-09-10T08:46:19.193144+0000","last_fullsized":"2024-09-10T08:46:19.193144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:24.234525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193535+0000","last_change":"2024-09-10T08:45:18.257226+0000","last_active":"2024-09-10T08:46:19.193535+0000","last_peered":"2024-09-10T08:46:19.193535+0000","last_clean":"2024-09-10T08:46:19.193535+0000","last_became_active":"2024-09-10T08:45:18.254113+0000","last_became_peered":"2024-09-10T08:45:18.254113+0000","last_unstale":"2024-09-10T08:46:19.193535+0000","last_undegraded":"2024-09-10T08:46:19.193535+0000","last_fullsized":"2024-09-10T08:46:19.193535+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:03:19.370105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194467+0000","last_change":"2024-09-10T08:44:17.761639+0000","last_active":"2024-09-10T08:46:19.194467+0000","last_peered":"2024-09-10T08:46:19.194467+0000","last_clean":"2024-09-10T08:46:19.194467+0000","last_became_active":"2024-09-10T08:44:17.761214+0000","last_became_peered":"2024-09-10T08:44:17.761214+0000","last_unstale":"2024-09-10T08:46:19.194467+0000","last_undegraded":"2024-09-10T08:46:19.194467+0000","last_fullsized":"2024-09-10T08:46:19.194467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:43.793195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193666+0000","last_change":"2024-09-10T08:45:18.258661+0000","last_active":"2024-09-10T08:46:19.193666+0000","last_peered":"2024-09-10T08:46:19.193666+0000","last_clean":"2024-09-10T08:46:19.193666+0000","last_became_active":"2024-09-10T08:45:18.258225+0000","last_became_peered":"2024-09-10T08:45:18.258225+0000","last_unstale":"2024-09-10T08:46:19.193666+0000","last_undegraded":"2024-09-10T08:46:19.193666+0000","last_fullsized":"2024-09-10T08:46:19.193666+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:28:37.184766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194660+0000","last_change":"2024-09-10T08:44:09.036632+0000","last_active":"2024-09-10T08:46:19.194660+0000","last_peered":"2024-09-10T08:46:19.194660+0000","last_clean":"2024-09-10T08:46:19.194660+0000","last_became_active":"2024-09-10T08:44:09.036254+0000","last_became_peered":"2024-09-10T08:44:09.036254+0000","last_unstale":"2024-09-10T08:46:19.194660+0000","last_undegraded":"2024-09-10T08:46:19.194660+0000","last_fullsized":"2024-09-10T08:46:19.194660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:59:40.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808140+0000","last_change":"2024-09-10T08:44:17.716215+0000","last_active":"2024-09-10T08:46:18.808140+0000","last_peered":"2024-09-10T08:46:18.808140+0000","last_clean":"2024-09-10T08:46:18.808140+0000","last_became_active":"2024-09-10T08:44:17.715809+0000","last_became_peered":"2024-09-10T08:44:17.715809+0000","last_unstale":"2024-09-10T08:46:18.808140+0000","last_undegraded":"2024-09-10T08:46:18.808140+0000","last_fullsized":"2024-09-10T08:46:18.808140+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:52.896401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193713+0000","last_change":"2024-09-10T08:44:10.677871+0000","last_active":"2024-09-10T08:46:19.193713+0000","last_peered":"2024-09-10T08:46:19.193713+0000","last_clean":"2024-09-10T08:46:19.193713+0000","last_became_active":"2024-09-10T08:44:10.677101+0000","last_became_peered":"2024-09-10T08:44:10.677101+0000","last_unstale":"2024-09-10T08:46:19.193713+0000","last_undegraded":"2024-09-10T08:46:19.193713+0000","last_fullsized":"2024-09-10T08:46:19.193713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:46.137249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'18","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:34.704451+0000","last_change":"2024-09-10T08:44:08.654428+0000","last_active":"2024-09-10T08:46:34.704451+0000","last_peered":"2024-09-10T08:46:34.704451+0000","last_clean":"2024-09-10T08:46:34.704451+0000","last_became_active":"2024-09-10T08:44:08.654119+0000","last_became_peered":"2024-09-10T08:44:08.654119+0000","last_unstale":"2024-09-10T08:46:34.704451+0000","last_undegraded":"2024-09-10T08:46:34.704451+0000","last_fullsized":"2024-09-10T08:46:34.704451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:16:56.787721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192538+0000","last_change":"2024-09-10T08:44:17.760828+0000","last_active":"2024-09-10T08:46:19.192538+0000","last_peered":"2024-09-10T08:46:19.192538+0000","last_clean":"2024-09-10T08:46:19.192538+0000","last_became_active":"2024-09-10T08:44:17.760367+0000","last_became_peered":"2024-09-10T08:44:17.760367+0000","last_unstale":"2024-09-10T08:46:19.192538+0000","last_undegraded":"2024-09-10T08:46:19.192538+0000","last_fullsized":"2024-09-10T08:46:19.192538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:11.130480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806446+0000","last_change":"2024-09-10T08:44:10.670179+0000","last_active":"2024-09-10T08:46:18.806446+0000","last_peered":"2024-09-10T08:46:18.806446+0000","last_clean":"2024-09-10T08:46:18.806446+0000","last_became_active":"2024-09-10T08:44:10.669366+0000","last_became_peered":"2024-09-10T08:44:10.669366+0000","last_unstale":"2024-09-10T08:46:18.806446+0000","last_undegraded":"2024-09-10T08:46:18.806446+0000","last_fullsized":"2024-09-10T08:46:18.806446+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:41.916365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194210+0000","last_change":"2024-09-10T08:44:09.050441+0000","last_active":"2024-09-10T08:46:19.194210+0000","last_peered":"2024-09-10T08:46:19.194210+0000","last_clean":"2024-09-10T08:46:19.194210+0000","last_became_active":"2024-09-10T08:44:09.050070+0000","last_became_peered":"2024-09-10T08:44:09.050070+0000","last_unstale":"2024-09-10T08:46:19.194210+0000","last_undegraded":"2024-09-10T08:46:19.194210+0000","last_fullsized":"2024-09-10T08:46:19.194210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:13.540021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193573+0000","last_change":"2024-09-10T08:44:17.720098+0000","last_active":"2024-09-10T08:46:19.193573+0000","last_peered":"2024-09-10T08:46:19.193573+0000","last_clean":"2024-09-10T08:46:19.193573+0000","last_became_active":"2024-09-10T08:44:17.719627+0000","last_became_peered":"2024-09-10T08:44:17.719627+0000","last_unstale":"2024-09-10T08:46:19.193573+0000","last_undegraded":"2024-09-10T08:46:19.193573+0000","last_fullsized":"2024-09-10T08:46:19.193573+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:50.020919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804677+0000","last_change":"2024-09-10T08:44:10.669630+0000","last_active":"2024-09-10T08:46:18.804677+0000","last_peered":"2024-09-10T08:46:18.804677+0000","last_clean":"2024-09-10T08:46:18.804677+0000","last_became_active":"2024-09-10T08:44:10.669307+0000","last_became_peered":"2024-09-10T08:44:10.669307+0000","last_unstale":"2024-09-10T08:46:18.804677+0000","last_undegraded":"2024-09-10T08:46:18.804677+0000","last_fullsized":"2024-09-10T08:46:18.804677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:06.320046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803568+0000","last_change":"2024-09-10T08:44:09.050715+0000","last_active":"2024-09-10T08:46:18.803568+0000","last_peered":"2024-09-10T08:46:18.803568+0000","last_clean":"2024-09-10T08:46:18.803568+0000","last_became_active":"2024-09-10T08:44:09.050347+0000","last_became_peered":"2024-09-10T08:44:09.050347+0000","last_unstale":"2024-09-10T08:46:18.803568+0000","last_undegraded":"2024-09-10T08:46:18.803568+0000","last_fullsized":"2024-09-10T08:46:18.803568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:18.689652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194038+0000","last_change":"2024-09-10T08:45:18.260444+0000","last_active":"2024-09-10T08:46:19.194038+0000","last_peered":"2024-09-10T08:46:19.194038+0000","last_clean":"2024-09-10T08:46:19.194038+0000","last_became_active":"2024-09-10T08:45:18.259949+0000","last_became_peered":"2024-09-10T08:45:18.259949+0000","last_unstale":"2024-09-10T08:46:19.194038+0000","last_undegraded":"2024-09-10T08:46:19.194038+0000","last_fullsized":"2024-09-10T08:46:19.194038+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:56.391253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804630+0000","last_change":"2024-09-10T08:44:10.670881+0000","last_active":"2024-09-10T08:46:18.804630+0000","last_peered":"2024-09-10T08:46:18.804630+0000","last_clean":"2024-09-10T08:46:18.804630+0000","last_became_active":"2024-09-10T08:44:10.669791+0000","last_became_peered":"2024-09-10T08:44:10.669791+0000","last_unstale":"2024-09-10T08:46:18.804630+0000","last_undegraded":"2024-09-10T08:46:18.804630+0000","last_fullsized":"2024-09-10T08:46:18.804630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:33.644205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'6","reported_seq":161,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:47.382562+0000","last_change":"2024-09-10T08:44:08.657938+0000","last_active":"2024-09-10T08:46:47.382562+0000","last_peered":"2024-09-10T08:46:47.382562+0000","last_clean":"2024-09-10T08:46:47.382562+0000","last_became_active":"2024-09-10T08:44:08.657526+0000","last_became_peered":"2024-09-10T08:44:08.657526+0000","last_unstale":"2024-09-10T08:46:47.382562+0000","last_undegraded":"2024-09-10T08:46:47.382562+0000","last_fullsized":"2024-09-10T08:46:47.382562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:15.673561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806147+0000","last_change":"2024-09-10T08:44:17.721619+0000","last_active":"2024-09-10T08:46:18.806147+0000","last_peered":"2024-09-10T08:46:18.806147+0000","last_clean":"2024-09-10T08:46:18.806147+0000","last_became_active":"2024-09-10T08:44:17.721175+0000","last_became_peered":"2024-09-10T08:44:17.721175+0000","last_unstale":"2024-09-10T08:46:18.806147+0000","last_undegraded":"2024-09-10T08:46:18.806147+0000","last_fullsized":"2024-09-10T08:46:18.806147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:52.244120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804320+0000","last_change":"2024-09-10T08:44:10.676164+0000","last_active":"2024-09-10T08:46:18.804320+0000","last_peered":"2024-09-10T08:46:18.804320+0000","last_clean":"2024-09-10T08:46:18.804320+0000","last_became_active":"2024-09-10T08:44:10.675820+0000","last_became_peered":"2024-09-10T08:44:10.675820+0000","last_unstale":"2024-09-10T08:46:18.804320+0000","last_undegraded":"2024-09-10T08:46:18.804320+0000","last_fullsized":"2024-09-10T08:46:18.804320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:24.477584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'6","reported_seq":162,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:47.382222+0000","last_change":"2024-09-10T08:44:09.049124+0000","last_active":"2024-09-10T08:46:47.382222+0000","last_peered":"2024-09-10T08:46:47.382222+0000","last_clean":"2024-09-10T08:46:47.382222+0000","last_became_active":"2024-09-10T08:44:09.048799+0000","last_became_peered":"2024-09-10T08:44:09.048799+0000","last_unstale":"2024-09-10T08:46:47.382222+0000","last_undegraded":"2024-09-10T08:46:47.382222+0000","last_fullsized":"2024-09-10T08:46:47.382222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:43:39.849727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806891+0000","last_change":"2024-09-10T08:44:17.761130+0000","last_active":"2024-09-10T08:46:18.806891+0000","last_peered":"2024-09-10T08:46:18.806891+0000","last_clean":"2024-09-10T08:46:18.806891+0000","last_became_active":"2024-09-10T08:44:17.760753+0000","last_became_peered":"2024-09-10T08:44:17.760753+0000","last_unstale":"2024-09-10T08:46:18.806891+0000","last_undegraded":"2024-09-10T08:46:18.806891+0000","last_fullsized":"2024-09-10T08:46:18.806891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:42.971962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.809258+0000","last_change":"2024-09-10T08:46:18.809258+0000","last_active":"2024-09-10T08:46:18.809258+0000","last_peered":"2024-09-10T08:46:18.809258+0000","last_clean":"2024-09-10T08:46:18.809258+0000","last_became_active":"2024-09-10T08:46:18.808908+0000","last_became_peered":"2024-09-10T08:46:18.808908+0000","last_unstale":"2024-09-10T08:46:18.809258+0000","last_undegraded":"2024-09-10T08:46:18.809258+0000","last_fullsized":"2024-09-10T08:46:18.809258+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:10.305672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873500+0000","last_change":"2024-09-10T08:44:08.658862+0000","last_active":"2024-09-10T08:46:18.873500+0000","last_peered":"2024-09-10T08:46:18.873500+0000","last_clean":"2024-09-10T08:46:18.873500+0000","last_became_active":"2024-09-10T08:44:08.657880+0000","last_became_peered":"2024-09-10T08:44:08.657880+0000","last_unstale":"2024-09-10T08:46:18.873500+0000","last_undegraded":"2024-09-10T08:46:18.873500+0000","last_fullsized":"2024-09-10T08:46:18.873500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:34.224025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194185+0000","last_change":"2024-09-10T08:45:18.268446+0000","last_active":"2024-09-10T08:46:19.194185+0000","last_peered":"2024-09-10T08:46:19.194185+0000","last_clean":"2024-09-10T08:46:19.194185+0000","last_became_active":"2024-09-10T08:45:18.267988+0000","last_became_peered":"2024-09-10T08:45:18.267988+0000","last_unstale":"2024-09-10T08:46:19.194185+0000","last_undegraded":"2024-09-10T08:46:19.194185+0000","last_fullsized":"2024-09-10T08:46:19.194185+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:42:35.990794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193214+0000","last_change":"2024-09-10T08:44:10.673258+0000","last_active":"2024-09-10T08:46:19.193214+0000","last_peered":"2024-09-10T08:46:19.193214+0000","last_clean":"2024-09-10T08:46:19.193214+0000","last_became_active":"2024-09-10T08:44:10.672854+0000","last_became_peered":"2024-09-10T08:44:10.672854+0000","last_unstale":"2024-09-10T08:46:19.193214+0000","last_undegraded":"2024-09-10T08:46:19.193214+0000","last_fullsized":"2024-09-10T08:46:19.193214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:35.913820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194588+0000","last_change":"2024-09-10T08:44:08.658039+0000","last_active":"2024-09-10T08:46:19.194588+0000","last_peered":"2024-09-10T08:46:19.194588+0000","last_clean":"2024-09-10T08:46:19.194588+0000","last_became_active":"2024-09-10T08:44:08.657636+0000","last_became_peered":"2024-09-10T08:44:08.657636+0000","last_unstale":"2024-09-10T08:46:19.194588+0000","last_undegraded":"2024-09-10T08:46:19.194588+0000","last_fullsized":"2024-09-10T08:46:19.194588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:09.615992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873591+0000","last_change":"2024-09-10T08:42:57.666747+0000","last_active":"2024-09-10T08:46:18.873591+0000","last_peered":"2024-09-10T08:46:18.873591+0000","last_clean":"2024-09-10T08:46:18.873591+0000","last_became_active":"2024-09-10T08:42:57.666417+0000","last_became_peered":"2024-09-10T08:42:57.666417+0000","last_unstale":"2024-09-10T08:46:18.873591+0000","last_undegraded":"2024-09-10T08:46:18.873591+0000","last_fullsized":"2024-09-10T08:46:18.873591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:39:31.729794+0000","last_clean_scrub_stamp":"2024-09-10T08:39:31.729794+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:39:48.547989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874074+0000","last_change":"2024-09-10T08:44:17.723021+0000","last_active":"2024-09-10T08:46:18.874074+0000","last_peered":"2024-09-10T08:46:18.874074+0000","last_clean":"2024-09-10T08:46:18.874074+0000","last_became_active":"2024-09-10T08:44:17.722222+0000","last_became_peered":"2024-09-10T08:44:17.722222+0000","last_unstale":"2024-09-10T08:46:18.874074+0000","last_undegraded":"2024-09-10T08:46:18.874074+0000","last_fullsized":"2024-09-10T08:46:18.874074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:24.444427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874129+0000","last_change":"2024-09-10T08:44:10.676651+0000","last_active":"2024-09-10T08:46:18.874129+0000","last_peered":"2024-09-10T08:46:18.874129+0000","last_clean":"2024-09-10T08:46:18.874129+0000","last_became_active":"2024-09-10T08:44:10.676342+0000","last_became_peered":"2024-09-10T08:44:10.676342+0000","last_unstale":"2024-09-10T08:46:18.874129+0000","last_undegraded":"2024-09-10T08:46:18.874129+0000","last_fullsized":"2024-09-10T08:46:18.874129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:07.021273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'8","reported_seq":164,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:45.380424+0000","last_change":"2024-09-10T08:44:09.050305+0000","last_active":"2024-09-10T08:46:45.380424+0000","last_peered":"2024-09-10T08:46:45.380424+0000","last_clean":"2024-09-10T08:46:45.380424+0000","last_became_active":"2024-09-10T08:44:09.049958+0000","last_became_peered":"2024-09-10T08:44:09.049958+0000","last_unstale":"2024-09-10T08:46:45.380424+0000","last_undegraded":"2024-09-10T08:46:45.380424+0000","last_fullsized":"2024-09-10T08:46:45.380424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:42.219190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2199,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193720+0000","last_change":"2024-09-10T08:44:17.720219+0000","last_active":"2024-09-10T08:46:19.193720+0000","last_peered":"2024-09-10T08:46:19.193720+0000","last_clean":"2024-09-10T08:46:19.193720+0000","last_became_active":"2024-09-10T08:44:17.719756+0000","last_became_peered":"2024-09-10T08:44:17.719756+0000","last_unstale":"2024-09-10T08:46:19.193720+0000","last_undegraded":"2024-09-10T08:46:19.193720+0000","last_fullsized":"2024-09-10T08:46:19.193720+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:01.070434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873929+0000","last_change":"2024-09-10T08:44:10.678986+0000","last_active":"2024-09-10T08:46:18.873929+0000","last_peered":"2024-09-10T08:46:18.873929+0000","last_clean":"2024-09-10T08:46:18.873929+0000","last_became_active":"2024-09-10T08:44:10.678654+0000","last_became_peered":"2024-09-10T08:44:10.678654+0000","last_unstale":"2024-09-10T08:46:18.873929+0000","last_undegraded":"2024-09-10T08:46:18.873929+0000","last_fullsized":"2024-09-10T08:46:18.873929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:07.116802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805026+0000","last_change":"2024-09-10T08:44:08.655025+0000","last_active":"2024-09-10T08:46:18.805026+0000","last_peered":"2024-09-10T08:46:18.805026+0000","last_clean":"2024-09-10T08:46:18.805026+0000","last_became_active":"2024-09-10T08:44:08.654728+0000","last_became_peered":"2024-09-10T08:44:08.654728+0000","last_unstale":"2024-09-10T08:46:18.805026+0000","last_undegraded":"2024-09-10T08:46:18.805026+0000","last_fullsized":"2024-09-10T08:46:18.805026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:31.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193534+0000","last_change":"2024-09-10T08:45:18.261492+0000","last_active":"2024-09-10T08:46:19.193534+0000","last_peered":"2024-09-10T08:46:19.193534+0000","last_clean":"2024-09-10T08:46:19.193534+0000","last_became_active":"2024-09-10T08:45:18.261111+0000","last_became_peered":"2024-09-10T08:45:18.261111+0000","last_unstale":"2024-09-10T08:46:19.193534+0000","last_undegraded":"2024-09-10T08:46:19.193534+0000","last_fullsized":"2024-09-10T08:46:19.193534+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:48.709459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874006+0000","last_change":"2024-09-10T08:44:10.675072+0000","last_active":"2024-09-10T08:46:18.874006+0000","last_peered":"2024-09-10T08:46:18.874006+0000","last_clean":"2024-09-10T08:46:18.874006+0000","last_became_active":"2024-09-10T08:44:10.674636+0000","last_became_peered":"2024-09-10T08:44:10.674636+0000","last_unstale":"2024-09-10T08:46:18.874006+0000","last_undegraded":"2024-09-10T08:46:18.874006+0000","last_fullsized":"2024-09-10T08:46:18.874006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:55.198445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806864+0000","last_change":"2024-09-10T08:44:08.656450+0000","last_active":"2024-09-10T08:46:18.806864+0000","last_peered":"2024-09-10T08:46:18.806864+0000","last_clean":"2024-09-10T08:46:18.806864+0000","last_became_active":"2024-09-10T08:44:08.655967+0000","last_became_peered":"2024-09-10T08:44:08.655967+0000","last_unstale":"2024-09-10T08:46:18.806864+0000","last_undegraded":"2024-09-10T08:46:18.806864+0000","last_fullsized":"2024-09-10T08:46:18.806864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:29.273167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804195+0000","last_change":"2024-09-10T08:44:17.722217+0000","last_active":"2024-09-10T08:46:18.804195+0000","last_peered":"2024-09-10T08:46:18.804195+0000","last_clean":"2024-09-10T08:46:18.804195+0000","last_became_active":"2024-09-10T08:44:17.721800+0000","last_became_peered":"2024-09-10T08:44:17.721800+0000","last_unstale":"2024-09-10T08:46:18.804195+0000","last_undegraded":"2024-09-10T08:46:18.804195+0000","last_fullsized":"2024-09-10T08:46:18.804195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:58.954397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194230+0000","last_change":"2024-09-10T08:45:18.267857+0000","last_active":"2024-09-10T08:46:19.194230+0000","last_peered":"2024-09-10T08:46:19.194230+0000","last_clean":"2024-09-10T08:46:19.194230+0000","last_became_active":"2024-09-10T08:45:18.267440+0000","last_became_peered":"2024-09-10T08:45:18.267440+0000","last_unstale":"2024-09-10T08:46:19.194230+0000","last_undegraded":"2024-09-10T08:46:19.194230+0000","last_fullsized":"2024-09-10T08:46:19.194230+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:28.659915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"92'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:23.312217+0000","last_change":"2024-09-10T08:44:08.659003+0000","last_active":"2024-09-10T08:46:23.312217+0000","last_peered":"2024-09-10T08:46:23.312217+0000","last_clean":"2024-09-10T08:46:23.312217+0000","last_became_active":"2024-09-10T08:44:08.658324+0000","last_became_peered":"2024-09-10T08:44:08.658324+0000","last_unstale":"2024-09-10T08:46:23.312217+0000","last_undegraded":"2024-09-10T08:46:23.312217+0000","last_fullsized":"2024-09-10T08:46:23.312217+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:13:44.743053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804596+0000","last_change":"2024-09-10T08:44:17.723363+0000","last_active":"2024-09-10T08:46:18.804596+0000","last_peered":"2024-09-10T08:46:18.804596+0000","last_clean":"2024-09-10T08:46:18.804596+0000","last_became_active":"2024-09-10T08:44:17.721799+0000","last_became_peered":"2024-09-10T08:44:17.721799+0000","last_unstale":"2024-09-10T08:46:18.804596+0000","last_undegraded":"2024-09-10T08:46:18.804596+0000","last_fullsized":"2024-09-10T08:46:18.804596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:08:28.772193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804709+0000","last_change":"2024-09-10T08:44:10.682952+0000","last_active":"2024-09-10T08:46:18.804709+0000","last_peered":"2024-09-10T08:46:18.804709+0000","last_clean":"2024-09-10T08:46:18.804709+0000","last_became_active":"2024-09-10T08:44:10.682249+0000","last_became_peered":"2024-09-10T08:44:10.682249+0000","last_unstale":"2024-09-10T08:46:18.804709+0000","last_undegraded":"2024-09-10T08:46:18.804709+0000","last_fullsized":"2024-09-10T08:46:18.804709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:32:21.285461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193239+0000","last_change":"2024-09-10T08:44:09.050454+0000","last_active":"2024-09-10T08:46:19.193239+0000","last_peered":"2024-09-10T08:46:19.193239+0000","last_clean":"2024-09-10T08:46:19.193239+0000","last_became_active":"2024-09-10T08:44:09.050036+0000","last_became_peered":"2024-09-10T08:44:09.050036+0000","last_unstale":"2024-09-10T08:46:19.193239+0000","last_undegraded":"2024-09-10T08:46:19.193239+0000","last_fullsized":"2024-09-10T08:46:19.193239+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:55.141471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806725+0000","last_change":"2024-09-10T08:44:17.721203+0000","last_active":"2024-09-10T08:46:18.806725+0000","last_peered":"2024-09-10T08:46:18.806725+0000","last_clean":"2024-09-10T08:46:18.806725+0000","last_became_active":"2024-09-10T08:44:17.720737+0000","last_became_peered":"2024-09-10T08:44:17.720737+0000","last_unstale":"2024-09-10T08:46:18.806725+0000","last_undegraded":"2024-09-10T08:46:18.806725+0000","last_fullsized":"2024-09-10T08:46:18.806725+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:32.081079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873435+0000","last_change":"2024-09-10T08:44:10.680944+0000","last_active":"2024-09-10T08:46:18.873435+0000","last_peered":"2024-09-10T08:46:18.873435+0000","last_clean":"2024-09-10T08:46:18.873435+0000","last_became_active":"2024-09-10T08:44:10.680644+0000","last_became_peered":"2024-09-10T08:44:10.680644+0000","last_unstale":"2024-09-10T08:46:18.873435+0000","last_undegraded":"2024-09-10T08:46:18.873435+0000","last_fullsized":"2024-09-10T08:46:18.873435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:46.495894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'6","reported_seq":159,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:42.376726+0000","last_change":"2024-09-10T08:44:08.658135+0000","last_active":"2024-09-10T08:46:42.376726+0000","last_peered":"2024-09-10T08:46:42.376726+0000","last_clean":"2024-09-10T08:46:42.376726+0000","last_became_active":"2024-09-10T08:44:08.657793+0000","last_became_peered":"2024-09-10T08:44:08.657793+0000","last_unstale":"2024-09-10T08:46:42.376726+0000","last_undegraded":"2024-09-10T08:46:42.376726+0000","last_fullsized":"2024-09-10T08:46:42.376726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:56:16.376268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1083,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193715+0000","last_change":"2024-09-10T08:45:18.271678+0000","last_active":"2024-09-10T08:46:19.193715+0000","last_peered":"2024-09-10T08:46:19.193715+0000","last_clean":"2024-09-10T08:46:19.193715+0000","last_became_active":"2024-09-10T08:45:18.271209+0000","last_became_peered":"2024-09-10T08:45:18.271209+0000","last_unstale":"2024-09-10T08:46:19.193715+0000","last_undegraded":"2024-09-10T08:46:19.193715+0000","last_fullsized":"2024-09-10T08:46:19.193715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:17.730839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873847+0000","last_change":"2024-09-10T08:44:10.678254+0000","last_active":"2024-09-10T08:46:18.873847+0000","last_peered":"2024-09-10T08:46:18.873847+0000","last_clean":"2024-09-10T08:46:18.873847+0000","last_became_active":"2024-09-10T08:44:10.677891+0000","last_became_peered":"2024-09-10T08:44:10.677891+0000","last_unstale":"2024-09-10T08:46:18.873847+0000","last_undegraded":"2024-09-10T08:46:18.873847+0000","last_fullsized":"2024-09-10T08:46:18.873847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:32:43.399049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804719+0000","last_change":"2024-09-10T08:44:08.656315+0000","last_active":"2024-09-10T08:46:18.804719+0000","last_peered":"2024-09-10T08:46:18.804719+0000","last_clean":"2024-09-10T08:46:18.804719+0000","last_became_active":"2024-09-10T08:44:08.655978+0000","last_became_peered":"2024-09-10T08:44:08.655978+0000","last_unstale":"2024-09-10T08:46:18.804719+0000","last_undegraded":"2024-09-10T08:46:18.804719+0000","last_fullsized":"2024-09-10T08:46:18.804719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:15.025354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193456+0000","last_change":"2024-09-10T08:44:17.726789+0000","last_active":"2024-09-10T08:46:19.193456+0000","last_peered":"2024-09-10T08:46:19.193456+0000","last_clean":"2024-09-10T08:46:19.193456+0000","last_became_active":"2024-09-10T08:44:17.726450+0000","last_became_peered":"2024-09-10T08:44:17.726450+0000","last_unstale":"2024-09-10T08:46:19.193456+0000","last_undegraded":"2024-09-10T08:46:19.193456+0000","last_fullsized":"2024-09-10T08:46:19.193456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:03.899822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806240+0000","last_change":"2024-09-10T08:44:10.677359+0000","last_active":"2024-09-10T08:46:18.806240+0000","last_peered":"2024-09-10T08:46:18.806240+0000","last_clean":"2024-09-10T08:46:18.806240+0000","last_became_active":"2024-09-10T08:44:10.676782+0000","last_became_peered":"2024-09-10T08:44:10.676782+0000","last_unstale":"2024-09-10T08:46:18.806240+0000","last_undegraded":"2024-09-10T08:46:18.806240+0000","last_fullsized":"2024-09-10T08:46:18.806240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:31.721714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808306+0000","last_change":"2024-09-10T08:44:08.656993+0000","last_active":"2024-09-10T08:46:18.808306+0000","last_peered":"2024-09-10T08:46:18.808306+0000","last_clean":"2024-09-10T08:46:18.808306+0000","last_became_active":"2024-09-10T08:44:08.656571+0000","last_became_peered":"2024-09-10T08:44:08.656571+0000","last_unstale":"2024-09-10T08:46:18.808306+0000","last_undegraded":"2024-09-10T08:46:18.808306+0000","last_fullsized":"2024-09-10T08:46:18.808306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:32.202740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804919+0000","last_change":"2024-09-10T08:44:17.725674+0000","last_active":"2024-09-10T08:46:18.804919+0000","last_peered":"2024-09-10T08:46:18.804919+0000","last_clean":"2024-09-10T08:46:18.804919+0000","last_became_active":"2024-09-10T08:44:17.725263+0000","last_became_peered":"2024-09-10T08:44:17.725263+0000","last_unstale":"2024-09-10T08:46:18.804919+0000","last_undegraded":"2024-09-10T08:46:18.804919+0000","last_fullsized":"2024-09-10T08:46:18.804919+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:18.183157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193937+0000","last_change":"2024-09-10T08:44:10.676871+0000","last_active":"2024-09-10T08:46:19.193937+0000","last_peered":"2024-09-10T08:46:19.193937+0000","last_clean":"2024-09-10T08:46:19.193937+0000","last_became_active":"2024-09-10T08:44:10.676540+0000","last_became_peered":"2024-09-10T08:44:10.676540+0000","last_unstale":"2024-09-10T08:46:19.193937+0000","last_undegraded":"2024-09-10T08:46:19.193937+0000","last_fullsized":"2024-09-10T08:46:19.193937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:50.964100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'41","reported_seq":315,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:44.378613+0000","last_change":"2024-09-10T08:44:08.660113+0000","last_active":"2024-09-10T08:46:44.378613+0000","last_peered":"2024-09-10T08:46:44.378613+0000","last_clean":"2024-09-10T08:46:44.378613+0000","last_became_active":"2024-09-10T08:44:08.659644+0000","last_became_peered":"2024-09-10T08:44:08.659644+0000","last_unstale":"2024-09-10T08:46:44.378613+0000","last_undegraded":"2024-09-10T08:46:44.378613+0000","last_fullsized":"2024-09-10T08:46:44.378613+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:01.605344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68542,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":235,"num_read_kb":185,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803342+0000","last_change":"2024-09-10T08:44:17.723653+0000","last_active":"2024-09-10T08:46:18.803342+0000","last_peered":"2024-09-10T08:46:18.803342+0000","last_clean":"2024-09-10T08:46:18.803342+0000","last_became_active":"2024-09-10T08:44:17.723284+0000","last_became_peered":"2024-09-10T08:44:17.723284+0000","last_unstale":"2024-09-10T08:46:18.803342+0000","last_undegraded":"2024-09-10T08:46:18.803342+0000","last_fullsized":"2024-09-10T08:46:18.803342+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:17.162526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193970+0000","last_change":"2024-09-10T08:44:10.681837+0000","last_active":"2024-09-10T08:46:19.193970+0000","last_peered":"2024-09-10T08:46:19.193970+0000","last_clean":"2024-09-10T08:46:19.193970+0000","last_became_active":"2024-09-10T08:44:10.681518+0000","last_became_peered":"2024-09-10T08:44:10.681518+0000","last_unstale":"2024-09-10T08:46:19.193970+0000","last_undegraded":"2024-09-10T08:46:19.193970+0000","last_fullsized":"2024-09-10T08:46:19.193970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:10.004215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804988+0000","last_change":"2024-09-10T08:44:09.050394+0000","last_active":"2024-09-10T08:46:18.804988+0000","last_peered":"2024-09-10T08:46:18.804988+0000","last_clean":"2024-09-10T08:46:18.804988+0000","last_became_active":"2024-09-10T08:44:09.049969+0000","last_became_peered":"2024-09-10T08:44:09.049969+0000","last_unstale":"2024-09-10T08:46:18.804988+0000","last_undegraded":"2024-09-10T08:46:18.804988+0000","last_fullsized":"2024-09-10T08:46:18.804988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:43.519246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194174+0000","last_change":"2024-09-10T08:44:17.720318+0000","last_active":"2024-09-10T08:46:19.194174+0000","last_peered":"2024-09-10T08:46:19.194174+0000","last_clean":"2024-09-10T08:46:19.194174+0000","last_became_active":"2024-09-10T08:44:17.719598+0000","last_became_peered":"2024-09-10T08:44:17.719598+0000","last_unstale":"2024-09-10T08:46:19.194174+0000","last_undegraded":"2024-09-10T08:46:19.194174+0000","last_fullsized":"2024-09-10T08:46:19.194174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:15:49.520324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193946+0000","last_change":"2024-09-10T08:44:10.677340+0000","last_active":"2024-09-10T08:46:19.193946+0000","last_peered":"2024-09-10T08:46:19.193946+0000","last_clean":"2024-09-10T08:46:19.193946+0000","last_became_active":"2024-09-10T08:44:10.676628+0000","last_became_peered":"2024-09-10T08:44:10.676628+0000","last_unstale":"2024-09-10T08:46:19.193946+0000","last_undegraded":"2024-09-10T08:46:19.193946+0000","last_fullsized":"2024-09-10T08:46:19.193946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:59:17.924217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874027+0000","last_change":"2024-09-10T08:44:08.660254+0000","last_active":"2024-09-10T08:46:18.874027+0000","last_peered":"2024-09-10T08:46:18.874027+0000","last_clean":"2024-09-10T08:46:18.874027+0000","last_became_active":"2024-09-10T08:44:08.659667+0000","last_became_peered":"2024-09-10T08:44:08.659667+0000","last_unstale":"2024-09-10T08:46:18.874027+0000","last_undegraded":"2024-09-10T08:46:18.874027+0000","last_fullsized":"2024-09-10T08:46:18.874027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:49:49.146867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803198+0000","last_change":"2024-09-10T08:44:17.722373+0000","last_active":"2024-09-10T08:46:18.803198+0000","last_peered":"2024-09-10T08:46:18.803198+0000","last_clean":"2024-09-10T08:46:18.803198+0000","last_became_active":"2024-09-10T08:44:17.722035+0000","last_became_peered":"2024-09-10T08:44:17.722035+0000","last_unstale":"2024-09-10T08:46:18.803198+0000","last_undegraded":"2024-09-10T08:46:18.803198+0000","last_fullsized":"2024-09-10T08:46:18.803198+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:30:36.187364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194365+0000","last_change":"2024-09-10T08:44:10.672585+0000","last_active":"2024-09-10T08:46:19.194365+0000","last_peered":"2024-09-10T08:46:19.194365+0000","last_clean":"2024-09-10T08:46:19.194365+0000","last_became_active":"2024-09-10T08:44:10.671987+0000","last_became_peered":"2024-09-10T08:44:10.671987+0000","last_unstale":"2024-09-10T08:46:19.194365+0000","last_undegraded":"2024-09-10T08:46:19.194365+0000","last_fullsized":"2024-09-10T08:46:19.194365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:48.660750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'7","reported_seq":268,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:44.378619+0000","last_change":"2024-09-10T08:44:08.658069+0000","last_active":"2024-09-10T08:46:44.378619+0000","last_peered":"2024-09-10T08:46:44.378619+0000","last_clean":"2024-09-10T08:46:44.378619+0000","last_became_active":"2024-09-10T08:44:08.657702+0000","last_became_peered":"2024-09-10T08:44:08.657702+0000","last_unstale":"2024-09-10T08:46:44.378619+0000","last_undegraded":"2024-09-10T08:46:44.378619+0000","last_fullsized":"2024-09-10T08:46:44.378619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:17.383267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":222,"num_read_kb":222,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192772+0000","last_change":"2024-09-10T08:44:17.717984+0000","last_active":"2024-09-10T08:46:19.192772+0000","last_peered":"2024-09-10T08:46:19.192772+0000","last_clean":"2024-09-10T08:46:19.192772+0000","last_became_active":"2024-09-10T08:44:17.717581+0000","last_became_peered":"2024-09-10T08:44:17.717581+0000","last_unstale":"2024-09-10T08:46:19.192772+0000","last_undegraded":"2024-09-10T08:46:19.192772+0000","last_fullsized":"2024-09-10T08:46:19.192772+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:25.330732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193566+0000","last_change":"2024-09-10T08:44:17.721244+0000","last_active":"2024-09-10T08:46:19.193566+0000","last_peered":"2024-09-10T08:46:19.193566+0000","last_clean":"2024-09-10T08:46:19.193566+0000","last_became_active":"2024-09-10T08:44:17.720884+0000","last_became_peered":"2024-09-10T08:44:17.720884+0000","last_unstale":"2024-09-10T08:46:19.193566+0000","last_undegraded":"2024-09-10T08:46:19.193566+0000","last_fullsized":"2024-09-10T08:46:19.193566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:28.769513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803279+0000","last_change":"2024-09-10T08:44:10.667558+0000","last_active":"2024-09-10T08:46:18.803279+0000","last_peered":"2024-09-10T08:46:18.803279+0000","last_clean":"2024-09-10T08:46:18.803279+0000","last_became_active":"2024-09-10T08:44:10.667214+0000","last_became_peered":"2024-09-10T08:44:10.667214+0000","last_unstale":"2024-09-10T08:46:18.803279+0000","last_undegraded":"2024-09-10T08:46:18.803279+0000","last_fullsized":"2024-09-10T08:46:18.803279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:49:53.630790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804697+0000","last_change":"2024-09-10T08:44:08.657930+0000","last_active":"2024-09-10T08:46:18.804697+0000","last_peered":"2024-09-10T08:46:18.804697+0000","last_clean":"2024-09-10T08:46:18.804697+0000","last_became_active":"2024-09-10T08:44:08.657611+0000","last_became_peered":"2024-09-10T08:44:08.657611+0000","last_unstale":"2024-09-10T08:46:18.804697+0000","last_undegraded":"2024-09-10T08:46:18.804697+0000","last_fullsized":"2024-09-10T08:46:18.804697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:36.923786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193910+0000","last_change":"2024-09-10T08:44:17.726106+0000","last_active":"2024-09-10T08:46:19.193910+0000","last_peered":"2024-09-10T08:46:19.193910+0000","last_clean":"2024-09-10T08:46:19.193910+0000","last_became_active":"2024-09-10T08:44:17.725683+0000","last_became_peered":"2024-09-10T08:44:17.725683+0000","last_unstale":"2024-09-10T08:46:19.193910+0000","last_undegraded":"2024-09-10T08:46:19.193910+0000","last_fullsized":"2024-09-10T08:46:19.193910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:23.392794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192727+0000","last_change":"2024-09-10T08:44:10.672204+0000","last_active":"2024-09-10T08:46:19.192727+0000","last_peered":"2024-09-10T08:46:19.192727+0000","last_clean":"2024-09-10T08:46:19.192727+0000","last_became_active":"2024-09-10T08:44:10.671898+0000","last_became_peered":"2024-09-10T08:44:10.671898+0000","last_unstale":"2024-09-10T08:46:19.192727+0000","last_undegraded":"2024-09-10T08:46:19.192727+0000","last_fullsized":"2024-09-10T08:46:19.192727+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:03.737357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805979+0000","last_change":"2024-09-10T08:44:08.656620+0000","last_active":"2024-09-10T08:46:18.805979+0000","last_peered":"2024-09-10T08:46:18.805979+0000","last_clean":"2024-09-10T08:46:18.805979+0000","last_became_active":"2024-09-10T08:44:08.656306+0000","last_became_peered":"2024-09-10T08:44:08.656306+0000","last_unstale":"2024-09-10T08:46:18.805979+0000","last_undegraded":"2024-09-10T08:46:18.805979+0000","last_fullsized":"2024-09-10T08:46:18.805979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:57.081909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804743+0000","last_change":"2024-09-10T08:44:08.660178+0000","last_active":"2024-09-10T08:46:18.804743+0000","last_peered":"2024-09-10T08:46:18.804743+0000","last_clean":"2024-09-10T08:46:18.804743+0000","last_became_active":"2024-09-10T08:44:08.659839+0000","last_became_peered":"2024-09-10T08:44:08.659839+0000","last_unstale":"2024-09-10T08:46:18.804743+0000","last_undegraded":"2024-09-10T08:46:18.804743+0000","last_fullsized":"2024-09-10T08:46:18.804743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:17.315167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804848+0000","last_change":"2024-09-10T08:44:10.683683+0000","last_active":"2024-09-10T08:46:18.804848+0000","last_peered":"2024-09-10T08:46:18.804848+0000","last_clean":"2024-09-10T08:46:18.804848+0000","last_became_active":"2024-09-10T08:44:10.683337+0000","last_became_peered":"2024-09-10T08:44:10.683337+0000","last_unstale":"2024-09-10T08:46:18.804848+0000","last_undegraded":"2024-09-10T08:46:18.804848+0000","last_fullsized":"2024-09-10T08:46:18.804848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:54.863098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808113+0000","last_change":"2024-09-10T08:44:17.721602+0000","last_active":"2024-09-10T08:46:18.808113+0000","last_peered":"2024-09-10T08:46:18.808113+0000","last_clean":"2024-09-10T08:46:18.808113+0000","last_became_active":"2024-09-10T08:44:17.721194+0000","last_became_peered":"2024-09-10T08:44:17.721194+0000","last_unstale":"2024-09-10T08:46:18.808113+0000","last_undegraded":"2024-09-10T08:46:18.808113+0000","last_fullsized":"2024-09-10T08:46:18.808113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:39.298494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":160,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:43.378005+0000","last_change":"2024-09-10T08:44:08.661943+0000","last_active":"2024-09-10T08:46:43.378005+0000","last_peered":"2024-09-10T08:46:43.378005+0000","last_clean":"2024-09-10T08:46:43.378005+0000","last_became_active":"2024-09-10T08:44:08.661587+0000","last_became_peered":"2024-09-10T08:44:08.661587+0000","last_unstale":"2024-09-10T08:46:43.378005+0000","last_undegraded":"2024-09-10T08:46:43.378005+0000","last_fullsized":"2024-09-10T08:46:43.378005+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:07.637064+0000","last_clean_scrub_stamp":"2024-09-10T08:44:07.637064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:05.771641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1117,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193844+0000","last_change":"2024-09-10T08:44:10.673522+0000","last_active":"2024-09-10T08:46:19.193844+0000","last_peered":"2024-09-10T08:46:19.193844+0000","last_clean":"2024-09-10T08:46:19.193844+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:46:19.193844+0000","last_undegraded":"2024-09-10T08:46:19.193844+0000","last_fullsized":"2024-09-10T08:46:19.193844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:03.006144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808482+0000","last_change":"2024-09-10T08:44:17.721306+0000","last_active":"2024-09-10T08:46:18.808482+0000","last_peered":"2024-09-10T08:46:18.808482+0000","last_clean":"2024-09-10T08:46:18.808482+0000","last_became_active":"2024-09-10T08:44:17.720401+0000","last_became_peered":"2024-09-10T08:44:17.720401+0000","last_unstale":"2024-09-10T08:46:18.808482+0000","last_undegraded":"2024-09-10T08:46:18.808482+0000","last_fullsized":"2024-09-10T08:46:18.808482+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:16.689636+0000","last_clean_scrub_stamp":"2024-09-10T08:44:16.689636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:11.589371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193166+0000","last_change":"2024-09-10T08:44:10.678425+0000","last_active":"2024-09-10T08:46:19.193166+0000","last_peered":"2024-09-10T08:46:19.193166+0000","last_clean":"2024-09-10T08:46:19.193166+0000","last_became_active":"2024-09-10T08:44:10.677187+0000","last_became_peered":"2024-09-10T08:44:10.677187+0000","last_unstale":"2024-09-10T08:46:19.193166+0000","last_undegraded":"2024-09-10T08:46:19.193166+0000","last_fullsized":"2024-09-10T08:46:19.193166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:06.527530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808431+0000","last_change":"2024-09-10T08:44:10.674178+0000","last_active":"2024-09-10T08:46:18.808431+0000","last_peered":"2024-09-10T08:46:18.808431+0000","last_clean":"2024-09-10T08:46:18.808431+0000","last_became_active":"2024-09-10T08:44:10.673843+0000","last_became_peered":"2024-09-10T08:44:10.673843+0000","last_unstale":"2024-09-10T08:46:18.808431+0000","last_undegraded":"2024-09-10T08:46:18.808431+0000","last_fullsized":"2024-09-10T08:46:18.808431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:35.023663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.807988+0000","last_change":"2024-09-10T08:44:10.673271+0000","last_active":"2024-09-10T08:46:18.807988+0000","last_peered":"2024-09-10T08:46:18.807988+0000","last_clean":"2024-09-10T08:46:18.807988+0000","last_became_active":"2024-09-10T08:44:10.672935+0000","last_became_peered":"2024-09-10T08:44:10.672935+0000","last_unstale":"2024-09-10T08:46:18.807988+0000","last_undegraded":"2024-09-10T08:46:18.807988+0000","last_fullsized":"2024-09-10T08:46:18.807988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:21:26.606820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193502+0000","last_change":"2024-09-10T08:44:10.671348+0000","last_active":"2024-09-10T08:46:19.193502+0000","last_peered":"2024-09-10T08:46:19.193502+0000","last_clean":"2024-09-10T08:46:19.193502+0000","last_became_active":"2024-09-10T08:44:10.670623+0000","last_became_peered":"2024-09-10T08:44:10.670623+0000","last_unstale":"2024-09-10T08:46:19.193502+0000","last_undegraded":"2024-09-10T08:46:19.193502+0000","last_fullsized":"2024-09-10T08:46:19.193502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:06.145694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806400+0000","last_change":"2024-09-10T08:44:10.666011+0000","last_active":"2024-09-10T08:46:18.806400+0000","last_peered":"2024-09-10T08:46:18.806400+0000","last_clean":"2024-09-10T08:46:18.806400+0000","last_became_active":"2024-09-10T08:44:10.665664+0000","last_became_peered":"2024-09-10T08:44:10.665664+0000","last_unstale":"2024-09-10T08:46:18.806400+0000","last_undegraded":"2024-09-10T08:46:18.806400+0000","last_fullsized":"2024-09-10T08:46:18.806400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:31.542102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193792+0000","last_change":"2024-09-10T08:44:10.675960+0000","last_active":"2024-09-10T08:46:19.193792+0000","last_peered":"2024-09-10T08:46:19.193792+0000","last_clean":"2024-09-10T08:46:19.193792+0000","last_became_active":"2024-09-10T08:44:10.675587+0000","last_became_peered":"2024-09-10T08:44:10.675587+0000","last_unstale":"2024-09-10T08:46:19.193792+0000","last_undegraded":"2024-09-10T08:46:19.193792+0000","last_fullsized":"2024-09-10T08:46:19.193792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:54.634619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808734+0000","last_change":"2024-09-10T08:44:10.681094+0000","last_active":"2024-09-10T08:46:18.808734+0000","last_peered":"2024-09-10T08:46:18.808734+0000","last_clean":"2024-09-10T08:46:18.808734+0000","last_became_active":"2024-09-10T08:44:10.678585+0000","last_became_peered":"2024-09-10T08:44:10.678585+0000","last_unstale":"2024-09-10T08:46:18.808734+0000","last_undegraded":"2024-09-10T08:46:18.808734+0000","last_fullsized":"2024-09-10T08:46:18.808734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:14.951949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193627+0000","last_change":"2024-09-10T08:44:10.675473+0000","last_active":"2024-09-10T08:46:19.193627+0000","last_peered":"2024-09-10T08:46:19.193627+0000","last_clean":"2024-09-10T08:46:19.193627+0000","last_became_active":"2024-09-10T08:44:10.674155+0000","last_became_peered":"2024-09-10T08:44:10.674155+0000","last_unstale":"2024-09-10T08:46:19.193627+0000","last_undegraded":"2024-09-10T08:46:19.193627+0000","last_fullsized":"2024-09-10T08:46:19.193627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:55.940235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804316+0000","last_change":"2024-09-10T08:44:10.680162+0000","last_active":"2024-09-10T08:46:18.804316+0000","last_peered":"2024-09-10T08:46:18.804316+0000","last_clean":"2024-09-10T08:46:18.804316+0000","last_became_active":"2024-09-10T08:44:10.679800+0000","last_became_peered":"2024-09-10T08:44:10.679800+0000","last_unstale":"2024-09-10T08:46:18.804316+0000","last_undegraded":"2024-09-10T08:46:18.804316+0000","last_fullsized":"2024-09-10T08:46:18.804316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:12.319813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192932+0000","last_change":"2024-09-10T08:44:10.674724+0000","last_active":"2024-09-10T08:46:19.192932+0000","last_peered":"2024-09-10T08:46:19.192932+0000","last_clean":"2024-09-10T08:46:19.192932+0000","last_became_active":"2024-09-10T08:44:10.674389+0000","last_became_peered":"2024-09-10T08:44:10.674389+0000","last_unstale":"2024-09-10T08:46:19.192932+0000","last_undegraded":"2024-09-10T08:46:19.192932+0000","last_fullsized":"2024-09-10T08:46:19.192932+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:52.123186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193888+0000","last_change":"2024-09-10T08:44:10.672805+0000","last_active":"2024-09-10T08:46:19.193888+0000","last_peered":"2024-09-10T08:46:19.193888+0000","last_clean":"2024-09-10T08:46:19.193888+0000","last_became_active":"2024-09-10T08:44:10.672508+0000","last_became_peered":"2024-09-10T08:44:10.672508+0000","last_unstale":"2024-09-10T08:46:19.193888+0000","last_undegraded":"2024-09-10T08:46:19.193888+0000","last_fullsized":"2024-09-10T08:46:19.193888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:48.558633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806228+0000","last_change":"2024-09-10T08:44:10.677504+0000","last_active":"2024-09-10T08:46:18.806228+0000","last_peered":"2024-09-10T08:46:18.806228+0000","last_clean":"2024-09-10T08:46:18.806228+0000","last_became_active":"2024-09-10T08:44:10.676822+0000","last_became_peered":"2024-09-10T08:44:10.676822+0000","last_unstale":"2024-09-10T08:46:18.806228+0000","last_undegraded":"2024-09-10T08:46:18.806228+0000","last_fullsized":"2024-09-10T08:46:18.806228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:05:48.293333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193457+0000","last_change":"2024-09-10T08:45:18.268208+0000","last_active":"2024-09-10T08:46:19.193457+0000","last_peered":"2024-09-10T08:46:19.193457+0000","last_clean":"2024-09-10T08:46:19.193457+0000","last_became_active":"2024-09-10T08:45:18.267700+0000","last_became_peered":"2024-09-10T08:45:18.267700+0000","last_unstale":"2024-09-10T08:46:19.193457+0000","last_undegraded":"2024-09-10T08:46:19.193457+0000","last_fullsized":"2024-09-10T08:46:19.193457+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:54.487624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.808678+0000","last_change":"2024-09-10T08:44:10.670615+0000","last_active":"2024-09-10T08:46:18.808678+0000","last_peered":"2024-09-10T08:46:18.808678+0000","last_clean":"2024-09-10T08:46:18.808678+0000","last_became_active":"2024-09-10T08:44:10.670032+0000","last_became_peered":"2024-09-10T08:44:10.670032+0000","last_unstale":"2024-09-10T08:46:18.808678+0000","last_undegraded":"2024-09-10T08:46:18.808678+0000","last_fullsized":"2024-09-10T08:46:18.808678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:31.078788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.803799+0000","last_change":"2024-09-10T08:44:10.672367+0000","last_active":"2024-09-10T08:46:18.803799+0000","last_peered":"2024-09-10T08:46:18.803799+0000","last_clean":"2024-09-10T08:46:18.803799+0000","last_became_active":"2024-09-10T08:44:10.672040+0000","last_became_peered":"2024-09-10T08:44:10.672040+0000","last_unstale":"2024-09-10T08:46:18.803799+0000","last_undegraded":"2024-09-10T08:46:18.803799+0000","last_fullsized":"2024-09-10T08:46:18.803799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:10.634708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194226+0000","last_change":"2024-09-10T08:44:10.676375+0000","last_active":"2024-09-10T08:46:19.194226+0000","last_peered":"2024-09-10T08:46:19.194226+0000","last_clean":"2024-09-10T08:46:19.194226+0000","last_became_active":"2024-09-10T08:44:10.676022+0000","last_became_peered":"2024-09-10T08:44:10.676022+0000","last_unstale":"2024-09-10T08:46:19.194226+0000","last_undegraded":"2024-09-10T08:46:19.194226+0000","last_fullsized":"2024-09-10T08:46:19.194226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:02.681954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193408+0000","last_change":"2024-09-10T08:44:10.673378+0000","last_active":"2024-09-10T08:46:19.193408+0000","last_peered":"2024-09-10T08:46:19.193408+0000","last_clean":"2024-09-10T08:46:19.193408+0000","last_became_active":"2024-09-10T08:44:10.673089+0000","last_became_peered":"2024-09-10T08:44:10.673089+0000","last_unstale":"2024-09-10T08:46:19.193408+0000","last_undegraded":"2024-09-10T08:46:19.193408+0000","last_fullsized":"2024-09-10T08:46:19.193408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:36:19.258651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192898+0000","last_change":"2024-09-10T08:44:10.671557+0000","last_active":"2024-09-10T08:46:19.192898+0000","last_peered":"2024-09-10T08:46:19.192898+0000","last_clean":"2024-09-10T08:46:19.192898+0000","last_became_active":"2024-09-10T08:44:10.671248+0000","last_became_peered":"2024-09-10T08:44:10.671248+0000","last_unstale":"2024-09-10T08:46:19.192898+0000","last_undegraded":"2024-09-10T08:46:19.192898+0000","last_fullsized":"2024-09-10T08:46:19.192898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:54.261525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193104+0000","last_change":"2024-09-10T08:44:10.674672+0000","last_active":"2024-09-10T08:46:19.193104+0000","last_peered":"2024-09-10T08:46:19.193104+0000","last_clean":"2024-09-10T08:46:19.193104+0000","last_became_active":"2024-09-10T08:44:10.673415+0000","last_became_peered":"2024-09-10T08:44:10.673415+0000","last_unstale":"2024-09-10T08:46:19.193104+0000","last_undegraded":"2024-09-10T08:46:19.193104+0000","last_fullsized":"2024-09-10T08:46:19.193104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:59:14.179664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.806192+0000","last_change":"2024-09-10T08:44:10.675293+0000","last_active":"2024-09-10T08:46:18.806192+0000","last_peered":"2024-09-10T08:46:18.806192+0000","last_clean":"2024-09-10T08:46:18.806192+0000","last_became_active":"2024-09-10T08:44:10.674924+0000","last_became_peered":"2024-09-10T08:44:10.674924+0000","last_unstale":"2024-09-10T08:46:18.806192+0000","last_undegraded":"2024-09-10T08:46:18.806192+0000","last_fullsized":"2024-09-10T08:46:18.806192+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:56.027320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194722+0000","last_change":"2024-09-10T08:44:10.674605+0000","last_active":"2024-09-10T08:46:19.194722+0000","last_peered":"2024-09-10T08:46:19.194722+0000","last_clean":"2024-09-10T08:46:19.194722+0000","last_became_active":"2024-09-10T08:44:10.674057+0000","last_became_peered":"2024-09-10T08:44:10.674057+0000","last_unstale":"2024-09-10T08:46:19.194722+0000","last_undegraded":"2024-09-10T08:46:19.194722+0000","last_fullsized":"2024-09-10T08:46:19.194722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:09.473003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.874065+0000","last_change":"2024-09-10T08:44:10.677410+0000","last_active":"2024-09-10T08:46:18.874065+0000","last_peered":"2024-09-10T08:46:18.874065+0000","last_clean":"2024-09-10T08:46:18.874065+0000","last_became_active":"2024-09-10T08:44:10.677109+0000","last_became_peered":"2024-09-10T08:44:10.677109+0000","last_unstale":"2024-09-10T08:46:18.874065+0000","last_undegraded":"2024-09-10T08:46:18.874065+0000","last_fullsized":"2024-09-10T08:46:18.874065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:45.455425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.805091+0000","last_change":"2024-09-10T08:44:10.669706+0000","last_active":"2024-09-10T08:46:18.805091+0000","last_peered":"2024-09-10T08:46:18.805091+0000","last_clean":"2024-09-10T08:46:18.805091+0000","last_became_active":"2024-09-10T08:44:10.669440+0000","last_became_peered":"2024-09-10T08:44:10.669440+0000","last_unstale":"2024-09-10T08:46:18.805091+0000","last_undegraded":"2024-09-10T08:46:18.805091+0000","last_fullsized":"2024-09-10T08:46:18.805091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:31:55.610651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194758+0000","last_change":"2024-09-10T08:44:10.676482+0000","last_active":"2024-09-10T08:46:19.194758+0000","last_peered":"2024-09-10T08:46:19.194758+0000","last_clean":"2024-09-10T08:46:19.194758+0000","last_became_active":"2024-09-10T08:44:10.675983+0000","last_became_peered":"2024-09-10T08:44:10.675983+0000","last_unstale":"2024-09-10T08:46:19.194758+0000","last_undegraded":"2024-09-10T08:46:19.194758+0000","last_fullsized":"2024-09-10T08:46:19.194758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:01.269283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.192699+0000","last_change":"2024-09-10T08:44:10.675681+0000","last_active":"2024-09-10T08:46:19.192699+0000","last_peered":"2024-09-10T08:46:19.192699+0000","last_clean":"2024-09-10T08:46:19.192699+0000","last_became_active":"2024-09-10T08:44:10.675357+0000","last_became_peered":"2024-09-10T08:44:10.675357+0000","last_unstale":"2024-09-10T08:46:19.192699+0000","last_undegraded":"2024-09-10T08:46:19.192699+0000","last_fullsized":"2024-09-10T08:46:19.192699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:25:26.138333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.873522+0000","last_change":"2024-09-10T08:44:10.680819+0000","last_active":"2024-09-10T08:46:18.873522+0000","last_peered":"2024-09-10T08:46:18.873522+0000","last_clean":"2024-09-10T08:46:18.873522+0000","last_became_active":"2024-09-10T08:44:10.680491+0000","last_became_peered":"2024-09-10T08:44:10.680491+0000","last_unstale":"2024-09-10T08:46:18.873522+0000","last_undegraded":"2024-09-10T08:46:18.873522+0000","last_fullsized":"2024-09-10T08:46:18.873522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:06:17.044776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:18.804621+0000","last_change":"2024-09-10T08:44:10.667376+0000","last_active":"2024-09-10T08:46:18.804621+0000","last_peered":"2024-09-10T08:46:18.804621+0000","last_clean":"2024-09-10T08:46:18.804621+0000","last_became_active":"2024-09-10T08:44:10.666648+0000","last_became_peered":"2024-09-10T08:44:10.666648+0000","last_unstale":"2024-09-10T08:46:18.804621+0000","last_undegraded":"2024-09-10T08:46:18.804621+0000","last_fullsized":"2024-09-10T08:46:18.804621+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:33:12.017550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193270+0000","last_change":"2024-09-10T08:44:10.668665+0000","last_active":"2024-09-10T08:46:19.193270+0000","last_peered":"2024-09-10T08:46:19.193270+0000","last_clean":"2024-09-10T08:46:19.193270+0000","last_became_active":"2024-09-10T08:44:10.666031+0000","last_became_peered":"2024-09-10T08:44:10.666031+0000","last_unstale":"2024-09-10T08:46:19.193270+0000","last_undegraded":"2024-09-10T08:46:19.193270+0000","last_fullsized":"2024-09-10T08:46:19.193270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:39:53.991463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194458+0000","last_change":"2024-09-10T08:44:10.676495+0000","last_active":"2024-09-10T08:46:19.194458+0000","last_peered":"2024-09-10T08:46:19.194458+0000","last_clean":"2024-09-10T08:46:19.194458+0000","last_became_active":"2024-09-10T08:44:10.676087+0000","last_became_peered":"2024-09-10T08:44:10.676087+0000","last_unstale":"2024-09-10T08:46:19.194458+0000","last_undegraded":"2024-09-10T08:46:19.194458+0000","last_fullsized":"2024-09-10T08:46:19.194458+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:41.828157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193138+0000","last_change":"2024-09-10T08:44:10.672835+0000","last_active":"2024-09-10T08:46:19.193138+0000","last_peered":"2024-09-10T08:46:19.193138+0000","last_clean":"2024-09-10T08:46:19.193138+0000","last_became_active":"2024-09-10T08:44:10.672537+0000","last_became_peered":"2024-09-10T08:44:10.672537+0000","last_unstale":"2024-09-10T08:46:19.193138+0000","last_undegraded":"2024-09-10T08:46:19.193138+0000","last_fullsized":"2024-09-10T08:46:19.193138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:06.121460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.193568+0000","last_change":"2024-09-10T08:44:10.675968+0000","last_active":"2024-09-10T08:46:19.193568+0000","last_peered":"2024-09-10T08:46:19.193568+0000","last_clean":"2024-09-10T08:46:19.193568+0000","last_became_active":"2024-09-10T08:44:10.675628+0000","last_became_peered":"2024-09-10T08:44:10.675628+0000","last_unstale":"2024-09-10T08:46:19.193568+0000","last_undegraded":"2024-09-10T08:46:19.193568+0000","last_fullsized":"2024-09-10T08:46:19.193568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:40.261336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-10T08:46:19.194091+0000","last_change":"2024-09-10T08:44:10.673151+0000","last_active":"2024-09-10T08:46:19.194091+0000","last_peered":"2024-09-10T08:46:19.194091+0000","last_clean":"2024-09-10T08:46:19.194091+0000","last_became_active":"2024-09-10T08:44:10.672833+0000","last_became_peered":"2024-09-10T08:44:10.672833+0000","last_unstale":"2024-09-10T08:46:19.194091+0000","last_undegraded":"2024-09-10T08:46:19.194091+0000","last_fullsized":"2024-09-10T08:46:19.194091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T08:44:09.650640+0000","last_clean_scrub_stamp":"2024-09-10T08:44:09.650640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:47:38.556222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77478,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1170,"num_read_kb":671,"num_write":276,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":234710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":235,"ondisk_log_size":235,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514541,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":523370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743474,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28740,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716412,"statfs":{"total":95995035648,"available":95965605888,"internally_reserved":0,"allocated":1757184,"data_stored":1178194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906999,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":526367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135932,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":522649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364865,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28660,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716492,"statfs":{"total":95995035648,"available":95965687808,"internally_reserved":0,"allocated":1675264,"data_stored":1120240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528391,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":518270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757323,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36988,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708164,"statfs":{"total":95995035648,"available":95957159936,"internally_reserved":0,"allocated":1683456,"data_stored":1114832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920848,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":522344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182485,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":517483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":590229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1130496,"data_stored":588980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":520260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T08:46:51.138 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:46:51.392 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:51.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.876+0000 7fee3d5d7640 1 -- 172.21.15.172:0/2982815623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee38106f10 msgr2=0x7fee3810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:51.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.876+0000 7fee3d5d7640 1 --2- 172.21.15.172:0/2982815623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee38106f10 0x7fee3810ea90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fee30009ca0 tx=0x7fee3002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:46:51.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.876+0000 7fee3d5d7640 1 -- 172.21.15.172:0/2982815623 shutdown_connections 2024-09-10T08:46:51.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.876+0000 7fee3d5d7640 1 --2- 172.21.15.172:0/2982815623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee38106f10 0x7fee3810ea90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:51.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.876+0000 7fee3d5d7640 1 --2- 172.21.15.172:0/2982815623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee38102790 0x7fee381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:51.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.876+0000 7fee3d5d7640 1 --2- 172.21.15.172:0/2982815623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee38101de0 0x7fee381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:51.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.876+0000 7fee3d5d7640 1 -- 172.21.15.172:0/2982815623 >> 172.21.15.172:0/2982815623 conn(0x7fee380fd8c0 msgr2=0x7fee380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:51.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.876+0000 7fee3d5d7640 1 -- 172.21.15.172:0/2982815623 shutdown_connections 2024-09-10T08:46:51.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.876+0000 7fee3d5d7640 1 -- 172.21.15.172:0/2982815623 wait complete. 2024-09-10T08:46:51.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee3d5d7640 1 Processor -- start 2024-09-10T08:46:51.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee3d5d7640 1 -- start start 2024-09-10T08:46:51.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee3d5d7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee38101de0 0x7fee38198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:51.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee3d5d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee38102790 0x7fee38198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:51.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee3d5d7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee38106f10 0x7fee3819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:51.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee3d5d7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fee38111e40 con 0x7fee38102790 2024-09-10T08:46:51.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee3d5d7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee38111cc0 con 0x7fee38106f10 2024-09-10T08:46:51.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee3d5d7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fee38111fc0 con 0x7fee38101de0 2024-09-10T08:46:51.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee367fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee38102790 0x7fee38198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:51.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee367fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee38102790 0x7fee38198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57842/0 (socket says 172.21.15.172:57842) 2024-09-10T08:46:51.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee367fc640 1 -- 172.21.15.172:0/1237071932 learned_addr learned my addr 172.21.15.172:0/1237071932 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:51.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee36ffd640 1 --2- 172.21.15.172:0/1237071932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee38101de0 0x7fee38198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:51.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee377fe640 1 --2- 172.21.15.172:0/1237071932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee38106f10 0x7fee3819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:51.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee367fc640 1 -- 172.21.15.172:0/1237071932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee38101de0 msgr2=0x7fee38198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:51.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee367fc640 1 --2- 172.21.15.172:0/1237071932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee38101de0 0x7fee38198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:51.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee367fc640 1 -- 172.21.15.172:0/1237071932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee38106f10 msgr2=0x7fee3819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:51.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee367fc640 1 --2- 172.21.15.172:0/1237071932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee38106f10 0x7fee3819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:51.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.877+0000 7fee367fc640 1 -- 172.21.15.172:0/1237071932 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3819d3b0 con 0x7fee38102790 2024-09-10T08:46:51.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.878+0000 7fee367fc640 1 --2- 172.21.15.172:0/1237071932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee38102790 0x7fee38198940 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00d950 tx=0x7fee2c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:51.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.878+0000 7fee1ffff640 1 -- 172.21.15.172:0/1237071932 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c017070 con 0x7fee38102790 2024-09-10T08:46:51.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.878+0000 7fee3d5d7640 1 -- 172.21.15.172:0/1237071932 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3819d6a0 con 0x7fee38102790 2024-09-10T08:46:51.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.878+0000 7fee1ffff640 1 -- 172.21.15.172:0/1237071932 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee2c00de80 con 0x7fee38102790 2024-09-10T08:46:51.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.878+0000 7fee1ffff640 1 -- 172.21.15.172:0/1237071932 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c00f620 con 0x7fee38102790 2024-09-10T08:46:51.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.878+0000 7fee3d5d7640 1 -- 172.21.15.172:0/1237071932 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee381a4f80 con 0x7fee38102790 2024-09-10T08:46:51.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.879+0000 7fee3d5d7640 1 -- 172.21.15.172:0/1237071932 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee381021c0 con 0x7fee38102790 2024-09-10T08:46:51.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.880+0000 7fee1ffff640 1 -- 172.21.15.172:0/1237071932 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fee2c009c30 con 0x7fee38102790 2024-09-10T08:46:51.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.881+0000 7fee1ffff640 1 --2- 172.21.15.172:0/1237071932 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee200a2b80 0x7fee200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:51.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.881+0000 7fee36ffd640 1 --2- 172.21.15.172:0/1237071932 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee200a2b80 0x7fee200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:51.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.881+0000 7fee1ffff640 1 -- 172.21.15.172:0/1237071932 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fee2c0cefd0 con 0x7fee38102790 2024-09-10T08:46:51.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.881+0000 7fee36ffd640 1 --2- 172.21.15.172:0/1237071932 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee200a2b80 0x7fee200a5040 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fee28009990 tx=0x7fee28009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:51.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:51.884+0000 7fee1ffff640 1 -- 172.21.15.172:0/1237071932 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee2c09bbd0 con 0x7fee38102790 2024-09-10T08:46:52.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:52.082+0000 7fee3d5d7640 1 -- 172.21.15.172:0/1237071932 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee38199760 con 0x7fee38102790 2024-09-10T08:46:52.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:52.083+0000 7fee1ffff640 1 -- 172.21.15.172:0/1237071932 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7fee38199760 con 0x7fee38102790 2024-09-10T08:46:52.084 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:52.084 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":29,"btime":"2024-09-10T08:46:37:242916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:46:36.267985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:46:52.084 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 29 2024-09-10T08:46:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:52.085+0000 7fee3d5d7640 1 -- 172.21.15.172:0/1237071932 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee200a2b80 msgr2=0x7fee200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:52.085+0000 7fee3d5d7640 1 --2- 172.21.15.172:0/1237071932 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee200a2b80 0x7fee200a5040 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fee28009990 tx=0x7fee28009110 comp rx=0 tx=0).stop 2024-09-10T08:46:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:52.085+0000 7fee3d5d7640 1 -- 172.21.15.172:0/1237071932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee38102790 msgr2=0x7fee38198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:52.085+0000 7fee3d5d7640 1 --2- 172.21.15.172:0/1237071932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee38102790 0x7fee38198940 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00d950 tx=0x7fee2c00dd50 comp rx=0 tx=0).stop 2024-09-10T08:46:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:52.085+0000 7fee3d5d7640 1 -- 172.21.15.172:0/1237071932 shutdown_connections 2024-09-10T08:46:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:52.086+0000 7fee3d5d7640 1 --2- 172.21.15.172:0/1237071932 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee200a2b80 0x7fee200a5040 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:52.086+0000 7fee3d5d7640 1 --2- 172.21.15.172:0/1237071932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee38106f10 0x7fee3819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:52.086+0000 7fee3d5d7640 1 --2- 172.21.15.172:0/1237071932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee38102790 0x7fee38198940 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:52.086+0000 7fee3d5d7640 1 --2- 172.21.15.172:0/1237071932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee38101de0 0x7fee38198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:52.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:52.086+0000 7fee3d5d7640 1 -- 172.21.15.172:0/1237071932 >> 172.21.15.172:0/1237071932 conn(0x7fee380fd8c0 msgr2=0x7fee380fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:52.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:52.086+0000 7fee3d5d7640 1 -- 172.21.15.172:0/1237071932 shutdown_connections 2024-09-10T08:46:52.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:52.086+0000 7fee3d5d7640 1 -- 172.21.15.172:0/1237071932 wait complete. 2024-09-10T08:46:52.539 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:52 smithi172 ceph-mon[30854]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:52.539 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:52 smithi172 ceph-mon[30854]: pgmap v369: 129 pgs: 129 active+clean; 654 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T08:46:52.539 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:52 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1237071932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:52 smithi191 ceph-mon[35314]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:52 smithi191 ceph-mon[35314]: pgmap v369: 129 pgs: 129 active+clean; 654 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T08:46:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:52 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1237071932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:52 smithi174 ceph-mon[34068]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T08:46:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:52 smithi174 ceph-mon[34068]: pgmap v369: 129 pgs: 129 active+clean; 654 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T08:46:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:52 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1237071932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:52.805 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:46:53.058 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:53.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.587+0000 7f77c55f0640 1 -- 172.21.15.172:0/3092266001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77c0100a80 msgr2=0x7f77c0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:53.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.587+0000 7f77c55f0640 1 --2- 172.21.15.172:0/3092266001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77c0100a80 0x7f77c0100ee0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009930 tx=0x7f77a802a420 comp rx=0 tx=0).stop 2024-09-10T08:46:53.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.587+0000 7f77c55f0640 1 -- 172.21.15.172:0/3092266001 shutdown_connections 2024-09-10T08:46:53.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.587+0000 7f77c55f0640 1 --2- 172.21.15.172:0/3092266001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c0101420 0x7f77c010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.587+0000 7f77c55f0640 1 --2- 172.21.15.172:0/3092266001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77c0100a80 0x7f77c0100ee0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.587+0000 7f77c55f0640 1 --2- 172.21.15.172:0/3092266001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77c0106a80 0x7f77c0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.587+0000 7f77c55f0640 1 -- 172.21.15.172:0/3092266001 >> 172.21.15.172:0/3092266001 conn(0x7f77c00fc7a0 msgr2=0x7f77c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:53.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.587+0000 7f77c55f0640 1 -- 172.21.15.172:0/3092266001 shutdown_connections 2024-09-10T08:46:53.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.587+0000 7f77c55f0640 1 -- 172.21.15.172:0/3092266001 wait complete. 2024-09-10T08:46:53.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.588+0000 7f77c55f0640 1 Processor -- start 2024-09-10T08:46:53.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.588+0000 7f77c55f0640 1 -- start start 2024-09-10T08:46:53.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.588+0000 7f77c55f0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77c0100a80 0x7f77c0199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:53.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.588+0000 7f77c55f0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c0101420 0x7f77c01997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:53.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.588+0000 7f77c55f0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77c0106a80 0x7f77c01a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:53.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.588+0000 7f77c55f0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f77c0110cf0 con 0x7f77c0100a80 2024-09-10T08:46:53.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.588+0000 7f77c55f0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f77c0110b70 con 0x7f77c0106a80 2024-09-10T08:46:53.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.588+0000 7f77c55f0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f77c0110e70 con 0x7f77c0101420 2024-09-10T08:46:53.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.588+0000 7f77beffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77c0100a80 0x7f77c0199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:53.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.589+0000 7f77beffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77c0100a80 0x7f77c0199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57860/0 (socket says 172.21.15.172:57860) 2024-09-10T08:46:53.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.589+0000 7f77beffd640 1 -- 172.21.15.172:0/4195129417 learned_addr learned my addr 172.21.15.172:0/4195129417 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:53.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.589+0000 7f77b67fc640 1 --2- 172.21.15.172:0/4195129417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c0101420 0x7f77c01997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:53.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.589+0000 7f77beffd640 1 -- 172.21.15.172:0/4195129417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c0101420 msgr2=0x7f77c01997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:53.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.589+0000 7f77beffd640 1 --2- 172.21.15.172:0/4195129417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c0101420 0x7f77c01997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.589+0000 7f77beffd640 1 -- 172.21.15.172:0/4195129417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77c0106a80 msgr2=0x7f77c01a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:53.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.589+0000 7f77beffd640 1 --2- 172.21.15.172:0/4195129417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77c0106a80 0x7f77c01a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.589+0000 7f77beffd640 1 -- 172.21.15.172:0/4195129417 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c01a4820 con 0x7f77c0100a80 2024-09-10T08:46:53.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.589+0000 7f77beffd640 1 --2- 172.21.15.172:0/4195129417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77c0100a80 0x7f77c0199260 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f77b000c890 tx=0x7f77b000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:53.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.589+0000 7f77bcff9640 1 -- 172.21.15.172:0/4195129417 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b0004220 con 0x7f77c0100a80 2024-09-10T08:46:53.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.589+0000 7f77bcff9640 1 -- 172.21.15.172:0/4195129417 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77b00043c0 con 0x7f77c0100a80 2024-09-10T08:46:53.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.589+0000 7f77bcff9640 1 -- 172.21.15.172:0/4195129417 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b0004a00 con 0x7f77c0100a80 2024-09-10T08:46:53.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.589+0000 7f77c55f0640 1 -- 172.21.15.172:0/4195129417 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c01a49f0 con 0x7f77c0100a80 2024-09-10T08:46:53.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.589+0000 7f77c55f0640 1 -- 172.21.15.172:0/4195129417 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c01a4f30 con 0x7f77c0100a80 2024-09-10T08:46:53.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.590+0000 7f77c55f0640 1 -- 172.21.15.172:0/4195129417 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c0100ee0 con 0x7f77c0100a80 2024-09-10T08:46:53.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.591+0000 7f77bcff9640 1 -- 172.21.15.172:0/4195129417 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f77b0004ba0 con 0x7f77c0100a80 2024-09-10T08:46:53.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.592+0000 7f77bcff9640 1 --2- 172.21.15.172:0/4195129417 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f77ac0a2c00 0x7f77ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:53.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.592+0000 7f77b67fc640 1 --2- 172.21.15.172:0/4195129417 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f77ac0a2c00 0x7f77ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:53.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.592+0000 7f77bcff9640 1 -- 172.21.15.172:0/4195129417 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f77b00cf020 con 0x7f77c0100a80 2024-09-10T08:46:53.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.593+0000 7f77b67fc640 1 --2- 172.21.15.172:0/4195129417 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f77ac0a2c00 0x7f77ac0a50c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f77a8006010 tx=0x7f77a8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:53.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.596+0000 7f77bcff9640 1 -- 172.21.15.172:0/4195129417 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77b0014030 con 0x7f77c0100a80 2024-09-10T08:46:53.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.798+0000 7f77c55f0640 1 -- 172.21.15.172:0/4195129417 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77c0108a80 con 0x7f77c0100a80 2024-09-10T08:46:53.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.798+0000 7f77bcff9640 1 -- 172.21.15.172:0/4195129417 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7f77b009bc20 con 0x7f77c0100a80 2024-09-10T08:46:53.799 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:53.799 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":29,"btime":"2024-09-10T08:46:37:242916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:46:36.267985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:46:53.800 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 29 2024-09-10T08:46:53.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.800+0000 7f77c55f0640 1 -- 172.21.15.172:0/4195129417 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f77ac0a2c00 msgr2=0x7f77ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:53.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.800+0000 7f77c55f0640 1 --2- 172.21.15.172:0/4195129417 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f77ac0a2c00 0x7f77ac0a50c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f77a8006010 tx=0x7f77a8000bc0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.800+0000 7f77c55f0640 1 -- 172.21.15.172:0/4195129417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77c0100a80 msgr2=0x7f77c0199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:53.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.800+0000 7f77c55f0640 1 --2- 172.21.15.172:0/4195129417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77c0100a80 0x7f77c0199260 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f77b000c890 tx=0x7f77b000cd50 comp rx=0 tx=0).stop 2024-09-10T08:46:53.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.801+0000 7f77c55f0640 1 -- 172.21.15.172:0/4195129417 shutdown_connections 2024-09-10T08:46:53.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.801+0000 7f77c55f0640 1 --2- 172.21.15.172:0/4195129417 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f77ac0a2c00 0x7f77ac0a50c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.801+0000 7f77c55f0640 1 --2- 172.21.15.172:0/4195129417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77c0106a80 0x7f77c01a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.801+0000 7f77c55f0640 1 --2- 172.21.15.172:0/4195129417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77c0101420 0x7f77c01997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.801+0000 7f77c55f0640 1 --2- 172.21.15.172:0/4195129417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77c0100a80 0x7f77c0199260 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:53.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.801+0000 7f77c55f0640 1 -- 172.21.15.172:0/4195129417 >> 172.21.15.172:0/4195129417 conn(0x7f77c00fc7a0 msgr2=0x7f77c00fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:53.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.801+0000 7f77c55f0640 1 -- 172.21.15.172:0/4195129417 shutdown_connections 2024-09-10T08:46:53.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:53.801+0000 7f77c55f0640 1 -- 172.21.15.172:0/4195129417 wait complete. 2024-09-10T08:46:54.429 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:54 smithi172 ceph-mon[30854]: pgmap v370: 129 pgs: 129 active+clean; 654 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:46:54.429 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:54 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/4195129417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:54.459 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T08:46:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:54 smithi191 ceph-mon[35314]: pgmap v370: 129 pgs: 129 active+clean; 654 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:46:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:54 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/4195129417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:54.712 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:54.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:54 smithi174 ceph-mon[34068]: pgmap v370: 129 pgs: 129 active+clean; 654 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:46:54.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:54 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/4195129417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:55.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.221+0000 7fa8d62f0640 1 -- 172.21.15.172:0/2841887135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8d0106a80 msgr2=0x7fa8d0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:55.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.221+0000 7fa8d62f0640 1 --2- 172.21.15.172:0/2841887135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8d0106a80 0x7fa8d0106e60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009930 tx=0x7fa8c002a420 comp rx=0 tx=0).stop 2024-09-10T08:46:55.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.221+0000 7fa8d62f0640 1 -- 172.21.15.172:0/2841887135 shutdown_connections 2024-09-10T08:46:55.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.221+0000 7fa8d62f0640 1 --2- 172.21.15.172:0/2841887135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8d0101420 0x7fa8d010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:55.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.221+0000 7fa8d62f0640 1 --2- 172.21.15.172:0/2841887135 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8d0100a80 0x7fa8d0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:55.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.221+0000 7fa8d62f0640 1 --2- 172.21.15.172:0/2841887135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8d0106a80 0x7fa8d0106e60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:55.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.221+0000 7fa8d62f0640 1 -- 172.21.15.172:0/2841887135 >> 172.21.15.172:0/2841887135 conn(0x7fa8d00fc7a0 msgr2=0x7fa8d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:55.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.221+0000 7fa8d62f0640 1 -- 172.21.15.172:0/2841887135 shutdown_connections 2024-09-10T08:46:55.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.221+0000 7fa8d62f0640 1 -- 172.21.15.172:0/2841887135 wait complete. 2024-09-10T08:46:55.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.221+0000 7fa8d62f0640 1 Processor -- start 2024-09-10T08:46:55.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.221+0000 7fa8d62f0640 1 -- start start 2024-09-10T08:46:55.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8d62f0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8d0100a80 0x7fa8d01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:55.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8d62f0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8d0101420 0x7fa8d0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:55.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8d62f0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8d0106a80 0x7fa8d019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:55.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8d62f0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0110d50 con 0x7fa8d0101420 2024-09-10T08:46:55.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8d62f0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0110bd0 con 0x7fa8d0106a80 2024-09-10T08:46:55.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8d62f0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0110ed0 con 0x7fa8d0100a80 2024-09-10T08:46:55.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8cf7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8d0101420 0x7fa8d0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8cf7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8d0101420 0x7fa8d0198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57880/0 (socket says 172.21.15.172:57880) 2024-09-10T08:46:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8cf7fe640 1 -- 172.21.15.172:0/1610533193 learned_addr learned my addr 172.21.15.172:0/1610533193 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8cffff640 1 --2- 172.21.15.172:0/1610533193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8d0100a80 0x7fa8d01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:55.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8d488d640 1 --2- 172.21.15.172:0/1610533193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8d0106a80 0x7fa8d019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:55.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8cf7fe640 1 -- 172.21.15.172:0/1610533193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8d0100a80 msgr2=0x7fa8d01983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:55.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8cf7fe640 1 --2- 172.21.15.172:0/1610533193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8d0100a80 0x7fa8d01983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:55.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8cf7fe640 1 -- 172.21.15.172:0/1610533193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8d0106a80 msgr2=0x7fa8d019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:55.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8cf7fe640 1 --2- 172.21.15.172:0/1610533193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8d0106a80 0x7fa8d019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:55.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8cf7fe640 1 -- 172.21.15.172:0/1610533193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d019d3a0 con 0x7fa8d0101420 2024-09-10T08:46:55.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8cffff640 1 --2- 172.21.15.172:0/1610533193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8d0100a80 0x7fa8d01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:55.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8d488d640 1 --2- 172.21.15.172:0/1610533193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8d0106a80 0x7fa8d019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:55.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.222+0000 7fa8cf7fe640 1 --2- 172.21.15.172:0/1610533193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8d0101420 0x7fa8d0198930 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400b4a0 tx=0x7fa8c400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:55.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.223+0000 7fa8cd7fa640 1 -- 172.21.15.172:0/1610533193 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4014070 con 0x7fa8d0101420 2024-09-10T08:46:55.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.223+0000 7fa8d62f0640 1 -- 172.21.15.172:0/1610533193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d019d690 con 0x7fa8d0101420 2024-09-10T08:46:55.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.223+0000 7fa8cd7fa640 1 -- 172.21.15.172:0/1610533193 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8c4010ac0 con 0x7fa8d0101420 2024-09-10T08:46:55.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.223+0000 7fa8cd7fa640 1 -- 172.21.15.172:0/1610533193 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4017c30 con 0x7fa8d0101420 2024-09-10T08:46:55.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.223+0000 7fa8d62f0640 1 -- 172.21.15.172:0/1610533193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d01a4f70 con 0x7fa8d0101420 2024-09-10T08:46:55.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.224+0000 7fa8d62f0640 1 -- 172.21.15.172:0/1610533193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8d0100ee0 con 0x7fa8d0101420 2024-09-10T08:46:55.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.225+0000 7fa8cd7fa640 1 -- 172.21.15.172:0/1610533193 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa8c4017070 con 0x7fa8d0101420 2024-09-10T08:46:55.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.225+0000 7fa8cd7fa640 1 --2- 172.21.15.172:0/1610533193 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa8b80a2b80 0x7fa8b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:55.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.226+0000 7fa8cffff640 1 --2- 172.21.15.172:0/1610533193 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa8b80a2b80 0x7fa8b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:55.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.226+0000 7fa8cd7fa640 1 -- 172.21.15.172:0/1610533193 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa8c40ce860 con 0x7fa8d0101420 2024-09-10T08:46:55.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.226+0000 7fa8cffff640 1 --2- 172.21.15.172:0/1610533193 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa8b80a2b80 0x7fa8b80a5040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0000c00 tx=0x7fa8c0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:55.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.230+0000 7fa8cd7fa640 1 -- 172.21.15.172:0/1610533193 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa8c409b400 con 0x7fa8d0101420 2024-09-10T08:46:55.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.398+0000 7fa8d62f0640 1 -- 172.21.15.172:0/1610533193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8d002d7c0 con 0x7fa8d0101420 2024-09-10T08:46:55.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.400+0000 7fa8cd7fa640 1 -- 172.21.15.172:0/1610533193 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24778 (secure 0 0 0) 0x7fa8c409ad00 con 0x7fa8d0101420 2024-09-10T08:46:55.401 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:55.401 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":92,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","created":"2024-09-10T08:36:33.083529+0000","modified":"2024-09-10T08:46:18.799844+0000","last_up_change":"2024-09-10T08:43:19.615640+0000","last_in_change":"2024-09-10T08:42:56.656115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:39:31.328306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:44:07.466637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:44:09.520569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:44:15.926057+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6803","nonce":2678426952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6805","nonce":2678426952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6809","nonce":2678426952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6807","nonce":2678426952}]},"public_addr":"172.21.15.172:6803/2678426952","cluster_addr":"172.21.15.172:6805/2678426952","heartbeat_back_addr":"172.21.15.172:6809/2678426952","heartbeat_front_addr":"172.21.15.172:6807/2678426952","state":["exists","up"]},{"osd":1,"uuid":"55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6801","nonce":3279661439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6803","nonce":3279661439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6807","nonce":3279661439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6805","nonce":3279661439}]},"public_addr":"172.21.15.174:6801/3279661439","cluster_addr":"172.21.15.174:6803/3279661439","heartbeat_back_addr":"172.21.15.174:6807/3279661439","heartbeat_front_addr":"172.21.15.174:6805/3279661439","state":["exists","up"]},{"osd":2,"uuid":"f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6801","nonce":1963796290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6803","nonce":1963796290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6807","nonce":1963796290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6805","nonce":1963796290}]},"public_addr":"172.21.15.191:6801/1963796290","cluster_addr":"172.21.15.191:6803/1963796290","heartbeat_back_addr":"172.21.15.191:6807/1963796290","heartbeat_front_addr":"172.21.15.191:6805/1963796290","state":["exists","up"]},{"osd":3,"uuid":"15ecca3f-bc6b-4a06-b283-b515384beb19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6811","nonce":869546480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6813","nonce":869546480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6817","nonce":869546480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6815","nonce":869546480}]},"public_addr":"172.21.15.172:6811/869546480","cluster_addr":"172.21.15.172:6813/869546480","heartbeat_back_addr":"172.21.15.172:6817/869546480","heartbeat_front_addr":"172.21.15.172:6815/869546480","state":["exists","up"]},{"osd":4,"uuid":"8c2c1d40-72ad-4a15-ab72-54871c7e2427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6809","nonce":961007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6811","nonce":961007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6815","nonce":961007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6813","nonce":961007949}]},"public_addr":"172.21.15.174:6809/961007949","cluster_addr":"172.21.15.174:6811/961007949","heartbeat_back_addr":"172.21.15.174:6815/961007949","heartbeat_front_addr":"172.21.15.174:6813/961007949","state":["exists","up"]},{"osd":5,"uuid":"4de45a1d-ef0e-4bfe-ba15-5eeeb8806155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6809","nonce":4216938371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6811","nonce":4216938371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6815","nonce":4216938371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6813","nonce":4216938371}]},"public_addr":"172.21.15.191:6809/4216938371","cluster_addr":"172.21.15.191:6811/4216938371","heartbeat_back_addr":"172.21.15.191:6815/4216938371","heartbeat_front_addr":"172.21.15.191:6813/4216938371","state":["exists","up"]},{"osd":6,"uuid":"60a9938a-f696-4db2-932c-402997543b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6819","nonce":3210762009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6821","nonce":3210762009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6825","nonce":3210762009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6823","nonce":3210762009}]},"public_addr":"172.21.15.172:6819/3210762009","cluster_addr":"172.21.15.172:6821/3210762009","heartbeat_back_addr":"172.21.15.172:6825/3210762009","heartbeat_front_addr":"172.21.15.172:6823/3210762009","state":["exists","up"]},{"osd":7,"uuid":"7f0202f3-971e-472f-9c4d-ae3c24d39f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6817","nonce":1461923581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6819","nonce":1461923581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6823","nonce":1461923581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6821","nonce":1461923581}]},"public_addr":"172.21.15.174:6817/1461923581","cluster_addr":"172.21.15.174:6819/1461923581","heartbeat_back_addr":"172.21.15.174:6823/1461923581","heartbeat_front_addr":"172.21.15.174:6821/1461923581","state":["exists","up"]},{"osd":8,"uuid":"b4e7327f-ca9f-4833-9cd3-b8529b3d9761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6817","nonce":3675210430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6819","nonce":3675210430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6823","nonce":3675210430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6821","nonce":3675210430}]},"public_addr":"172.21.15.191:6817/3675210430","cluster_addr":"172.21.15.191:6819/3675210430","heartbeat_back_addr":"172.21.15.191:6823/3675210430","heartbeat_front_addr":"172.21.15.191:6821/3675210430","state":["exists","up"]},{"osd":9,"uuid":"9f6153f2-f9db-4c00-a659-c932628016f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6827","nonce":1181268619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6829","nonce":1181268619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6833","nonce":1181268619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6831","nonce":1181268619}]},"public_addr":"172.21.15.172:6827/1181268619","cluster_addr":"172.21.15.172:6829/1181268619","heartbeat_back_addr":"172.21.15.172:6833/1181268619","heartbeat_front_addr":"172.21.15.172:6831/1181268619","state":["exists","up"]},{"osd":10,"uuid":"040a013a-4c3e-442f-bea6-12cbe517716d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6825","nonce":1238812525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6827","nonce":1238812525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6831","nonce":1238812525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6829","nonce":1238812525}]},"public_addr":"172.21.15.174:6825/1238812525","cluster_addr":"172.21.15.174:6827/1238812525","heartbeat_back_addr":"172.21.15.174:6831/1238812525","heartbeat_front_addr":"172.21.15.174:6829/1238812525","state":["exists","up"]},{"osd":11,"uuid":"ae6535bd-1510-4bf3-bf4b-1a2a256add43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6825","nonce":2162719997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6827","nonce":2162719997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6831","nonce":2162719997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6829","nonce":2162719997}]},"public_addr":"172.21.15.191:6825/2162719997","cluster_addr":"172.21.15.191:6827/2162719997","heartbeat_back_addr":"172.21.15.191:6831/2162719997","heartbeat_front_addr":"172.21.15.191:6829/2162719997","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:38:38.812596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:04.765652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:29.103966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:56.085849+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:21.641784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:47.750376+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:11.640697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:36.638840+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:02.167446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:28.655804+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:52.749676+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:43:17.783908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:0/977245917":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/3919952878":"2024-09-11T08:36:54.887256+0000","172.21.15.172:6800/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/1525385361":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/2043423408":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/839344299":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6800/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/3251817286":"2024-09-11T08:36:54.887256+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T08:46:55.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.402+0000 7fa8d62f0640 1 -- 172.21.15.172:0/1610533193 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa8b80a2b80 msgr2=0x7fa8b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:55.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.402+0000 7fa8d62f0640 1 --2- 172.21.15.172:0/1610533193 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa8b80a2b80 0x7fa8b80a5040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0000c00 tx=0x7fa8c0031400 comp rx=0 tx=0).stop 2024-09-10T08:46:55.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.402+0000 7fa8d62f0640 1 -- 172.21.15.172:0/1610533193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8d0101420 msgr2=0x7fa8d0198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:55.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.402+0000 7fa8d62f0640 1 --2- 172.21.15.172:0/1610533193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8d0101420 0x7fa8d0198930 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400b4a0 tx=0x7fa8c400b960 comp rx=0 tx=0).stop 2024-09-10T08:46:55.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.403+0000 7fa8d62f0640 1 -- 172.21.15.172:0/1610533193 shutdown_connections 2024-09-10T08:46:55.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.403+0000 7fa8d62f0640 1 --2- 172.21.15.172:0/1610533193 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa8b80a2b80 0x7fa8b80a5040 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:55.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.403+0000 7fa8d62f0640 1 --2- 172.21.15.172:0/1610533193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8d0106a80 0x7fa8d019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:55.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.403+0000 7fa8d62f0640 1 --2- 172.21.15.172:0/1610533193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8d0101420 0x7fa8d0198930 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:55.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.403+0000 7fa8d62f0640 1 --2- 172.21.15.172:0/1610533193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8d0100a80 0x7fa8d01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:55.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.403+0000 7fa8d62f0640 1 -- 172.21.15.172:0/1610533193 >> 172.21.15.172:0/1610533193 conn(0x7fa8d00fc7a0 msgr2=0x7fa8d006ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:55.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.403+0000 7fa8d62f0640 1 -- 172.21.15.172:0/1610533193 shutdown_connections 2024-09-10T08:46:55.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:55.403+0000 7fa8d62f0640 1 -- 172.21.15.172:0/1610533193 wait complete. 2024-09-10T08:46:56.051 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-10T08:46:56.301 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:56.545 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:56 smithi172 ceph-mon[30854]: pgmap v371: 129 pgs: 129 active+clean; 654 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T08:46:56.545 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:56 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1610533193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:46:56.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:56 smithi191 ceph-mon[35314]: pgmap v371: 129 pgs: 129 active+clean; 654 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T08:46:56.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:56 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1610533193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:46:56.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:56 smithi174 ceph-mon[34068]: pgmap v371: 129 pgs: 129 active+clean; 654 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T08:46:56.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:56 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1610533193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T08:46:57.397 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:57 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:46:57.423 INFO:teuthology.orchestra.run.smithi172.stdout:{"run_dir":"/var/run/ceph"} 2024-09-10T08:46:57.424 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-10T08:46:57.424 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 9893 2024-09-10T08:46:57.424 DEBUG:tasks.quiescer:Yielding 2024-09-10T08:46:57.424 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T08:46:57.439 INFO:tasks.workunit:Pulling workunits from ref 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:46:57.439 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T08:46:57.439 INFO:tasks.workunit:timeout=3h 2024-09-10T08:46:57.439 INFO:tasks.workunit:cleanup=True 2024-09-10T08:46:57.439 DEBUG:teuthology.orchestra.run.smithi172:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T08:46:57.440 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:46:57.456 INFO:teuthology.orchestra.run.smithi172.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-10T08:46:57.456 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-10T08:46:57.456 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 78h/120d Inode: 1 Links: 2 2024-09-10T08:46:57.456 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T08:46:57.456 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-10T08:46:57.457 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-10 08:44:58.608992027 +0000 2024-09-10T08:46:57.457 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-10 08:44:58.608992027 +0000 2024-09-10T08:46:57.457 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-10 08:46:35.730938503 +0000 2024-09-10T08:46:57.457 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: - 2024-09-10T08:46:57.457 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-10T08:46:57.457 DEBUG:teuthology.orchestra.run.smithi172:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-10T08:46:57.530 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T08:46:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:57 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:46:57.586 INFO:tasks.workunit.client.0.smithi172.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T08:46:57.699 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:57.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:57 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:46:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.919+0000 7f15482dd640 1 -- 172.21.15.172:0/270054571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1540102ea0 msgr2=0x7f1540103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.919+0000 7f15482dd640 1 --2- 172.21.15.172:0/270054571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1540102ea0 0x7f1540103320 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1534009900 tx=0x7f153402a410 comp rx=0 tx=0).stop 2024-09-10T08:46:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.919+0000 7f15482dd640 1 -- 172.21.15.172:0/270054571 shutdown_connections 2024-09-10T08:46:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.919+0000 7f15482dd640 1 --2- 172.21.15.172:0/270054571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1540103860 0x7f154010a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.919+0000 7f15482dd640 1 --2- 172.21.15.172:0/270054571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1540102ea0 0x7f1540103320 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.919+0000 7f15482dd640 1 --2- 172.21.15.172:0/270054571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15400ffe30 0x7f1540100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.919+0000 7f15482dd640 1 -- 172.21.15.172:0/270054571 >> 172.21.15.172:0/270054571 conn(0x7f15400fb970 msgr2=0x7f15400fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.919+0000 7f15482dd640 1 -- 172.21.15.172:0/270054571 shutdown_connections 2024-09-10T08:46:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.919+0000 7f15482dd640 1 -- 172.21.15.172:0/270054571 wait complete. 2024-09-10T08:46:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.919+0000 7f15482dd640 1 Processor -- start 2024-09-10T08:46:57.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.919+0000 7f15482dd640 1 -- start start 2024-09-10T08:46:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f15482dd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15400ffe30 0x7f1540197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f15482dd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1540102ea0 0x7f1540198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f15482dd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1540103860 0x7f154019f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f15482dd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f154010d690 con 0x7f1540102ea0 2024-09-10T08:46:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f15482dd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f154010d510 con 0x7f1540103860 2024-09-10T08:46:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f15482dd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f154010d810 con 0x7f15400ffe30 2024-09-10T08:46:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f1545878640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1540102ea0 0x7f1540198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f1545878640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1540102ea0 0x7f1540198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55634/0 (socket says 172.21.15.172:55634) 2024-09-10T08:46:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f1545878640 1 -- 172.21.15.172:0/2904836431 learned_addr learned my addr 172.21.15.172:0/2904836431 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f154687a640 1 --2- 172.21.15.172:0/2904836431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1540103860 0x7f154019f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f1546079640 1 --2- 172.21.15.172:0/2904836431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15400ffe30 0x7f1540197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f1545878640 1 -- 172.21.15.172:0/2904836431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15400ffe30 msgr2=0x7f1540197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f1545878640 1 --2- 172.21.15.172:0/2904836431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15400ffe30 0x7f1540197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f1545878640 1 -- 172.21.15.172:0/2904836431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1540103860 msgr2=0x7f154019f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f1545878640 1 --2- 172.21.15.172:0/2904836431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1540103860 0x7f154019f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f1545878640 1 -- 172.21.15.172:0/2904836431 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154019f940 con 0x7f1540102ea0 2024-09-10T08:46:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f154687a640 1 --2- 172.21.15.172:0/2904836431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1540103860 0x7f154019f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:46:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f1545878640 1 --2- 172.21.15.172:0/2904836431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1540102ea0 0x7f1540198270 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1534004770 tx=0x7f15340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f15337fe640 1 -- 172.21.15.172:0/2904836431 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153403f030 con 0x7f1540102ea0 2024-09-10T08:46:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f15337fe640 1 -- 172.21.15.172:0/2904836431 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1534003cb0 con 0x7f1540102ea0 2024-09-10T08:46:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f15482dd640 1 -- 172.21.15.172:0/2904836431 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f154019fbd0 con 0x7f1540102ea0 2024-09-10T08:46:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f15337fe640 1 -- 172.21.15.172:0/2904836431 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534035070 con 0x7f1540102ea0 2024-09-10T08:46:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.920+0000 7f15482dd640 1 -- 172.21.15.172:0/2904836431 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15401a0110 con 0x7f1540102ea0 2024-09-10T08:46:57.930 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.923+0000 7f15337fe640 1 -- 172.21.15.172:0/2904836431 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1534042030 con 0x7f1540102ea0 2024-09-10T08:46:57.930 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.923+0000 7f154b07b740 1 Processor -- start 2024-09-10T08:46:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f154b07b740 1 -- start start 2024-09-10T08:46:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f154b07b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd09a0 0x55d3e6fd0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f154b07b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd12e0 0x55d3e6fd5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f154b07b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd6370 0x55d3e6fd8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f154b07b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d3e6f342f0 con 0x55d3e6fd12e0 2024-09-10T08:46:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f154b07b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d3e6edc270 con 0x55d3e6fd6370 2024-09-10T08:46:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f154b07b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d3e6f302b0 con 0x55d3e6fd09a0 2024-09-10T08:46:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f1546079640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd12e0 0x55d3e6fd5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f1546079640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd12e0 0x55d3e6fd5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55646/0 (socket says 172.21.15.172:55646) 2024-09-10T08:46:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f1546079640 1 -- 172.21.15.172:0/3934579731 learned_addr learned my addr 172.21.15.172:0/3934579731 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f15337fe640 1 --2- 172.21.15.172:0/2904836431 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f152c0a2cb0 0x7f152c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f154687a640 1 --2- 172.21.15.172:0/3934579731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd09a0 0x55d3e6fd0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f1546079640 1 -- 172.21.15.172:0/3934579731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd09a0 msgr2=0x55d3e6fd0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f1545878640 1 --2- 172.21.15.172:0/3934579731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd6370 0x55d3e6fd8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f1546079640 1 --2- 172.21.15.172:0/3934579731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd09a0 0x55d3e6fd0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f1546079640 1 -- 172.21.15.172:0/3934579731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd6370 msgr2=0x55d3e6fd8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f1546079640 1 --2- 172.21.15.172:0/3934579731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd6370 0x55d3e6fd8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f1546079640 1 -- 172.21.15.172:0/3934579731 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e6edb340 con 0x55d3e6fd12e0 2024-09-10T08:46:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f154687a640 1 --2- 172.21.15.172:0/3934579731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd09a0 0x55d3e6fd0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f1545878640 1 --2- 172.21.15.172:0/3934579731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd6370 0x55d3e6fd8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f154687a640 1 --2- 172.21.15.172:0/2904836431 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f152c0a2cb0 0x7f152c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.924+0000 7f15337fe640 1 -- 172.21.15.172:0/2904836431 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f15340efda0 con 0x7f1540102ea0 2024-09-10T08:46:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.925+0000 7f1546079640 1 --2- 172.21.15.172:0/3934579731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd12e0 0x55d3e6fd5da0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f153c00bc60 tx=0x7f153c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ff2d4a1df1a121 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.925+0000 7f1530bfe640 1 -- 172.21.15.172:0/3934579731 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c015070 con 0x55d3e6fd12e0 2024-09-10T08:46:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.925+0000 7f1530bfe640 1 -- 172.21.15.172:0/3934579731 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f153c01a910 con 0x55d3e6fd12e0 2024-09-10T08:46:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.925+0000 7f1530bfe640 1 -- 172.21.15.172:0/3934579731 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c0118e0 con 0x55d3e6fd12e0 2024-09-10T08:46:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 1 -- 172.21.15.172:0/3934579731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd12e0 msgr2=0x55d3e6fd5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 1 --2- 172.21.15.172:0/3934579731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd12e0 0x55d3e6fd5da0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f153c00bc60 tx=0x7f153c00f730 comp rx=0 tx=0).stop 2024-09-10T08:46:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154687a640 1 --2- 172.21.15.172:0/2904836431 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f152c0a2cb0 0x7f152c0a5170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1538005fd0 tx=0x7f1538004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 1 -- 172.21.15.172:0/3934579731 shutdown_connections 2024-09-10T08:46:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 1 --2- 172.21.15.172:0/3934579731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd6370 0x55d3e6fd8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 1 --2- 172.21.15.172:0/3934579731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd12e0 0x55d3e6fd5da0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 1 --2- 172.21.15.172:0/3934579731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd09a0 0x55d3e6fd0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 1 -- 172.21.15.172:0/3934579731 >> 172.21.15.172:0/3934579731 conn(0x55d3e6fcc010 msgr2=0x55d3e6fcc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 1 -- 172.21.15.172:0/3934579731 shutdown_connections 2024-09-10T08:46:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 1 -- 172.21.15.172:0/3934579731 wait complete. 2024-09-10T08:46:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 1 -- start start 2024-09-10T08:46:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 1 Processor -- start 2024-09-10T08:46:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd09a0 0x55d3e6fdaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd12e0 0x55d3e6fdb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd6370 0x55d3e6fe2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d3e6fd9340 con 0x55d3e6fd12e0 2024-09-10T08:46:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d3e6ebd290 con 0x55d3e6fd09a0 2024-09-10T08:46:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d3e6fd94c0 con 0x55d3e6fd6370 2024-09-10T08:46:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f154b07b740 10 client.0 _ll_register_callbacks cb 0x55d3e6eee140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f1545878640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd12e0 0x55d3e6fdb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f1545878640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd12e0 0x55d3e6fdb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55660/0 (socket says 172.21.15.172:55660) 2024-09-10T08:46:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.926+0000 7f1545878640 1 -- 172.21.15.172:0/594935220 learned_addr learned my addr 172.21.15.172:0/594935220 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f1546079640 1 --2- 172.21.15.172:0/594935220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd09a0 0x55d3e6fdaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f1545878640 1 -- 172.21.15.172:0/594935220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd6370 msgr2=0x55d3e6fe2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f1545878640 1 --2- 172.21.15.172:0/594935220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd6370 0x55d3e6fe2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f1545878640 1 -- 172.21.15.172:0/594935220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd09a0 msgr2=0x55d3e6fdaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f1545878640 1 --2- 172.21.15.172:0/594935220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd09a0 0x55d3e6fdaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f1545878640 1 -- 172.21.15.172:0/594935220 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e6c47210 con 0x55d3e6fd12e0 2024-09-10T08:46:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f1545878640 1 --2- 172.21.15.172:0/594935220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd12e0 0x55d3e6fdb0e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f15340ed540 tx=0x7f153408b190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef628a789f62a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f1529ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f1529ffb640 1 -- 172.21.15.172:0/594935220 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534049070 con 0x55d3e6fd12e0 2024-09-10T08:46:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f1529ffb640 1 -- 172.21.15.172:0/594935220 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f15340b8880 con 0x55d3e6fd12e0 2024-09-10T08:46:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f154b07b740 1 -- 172.21.15.172:0/594935220 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f153c01a910 con 0x55d3e6fd12e0 2024-09-10T08:46:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f1529ffb640 1 -- 172.21.15.172:0/594935220 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f15340ba3e0 con 0x55d3e6fd12e0 2024-09-10T08:46:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f154b07b740 10 client.15471 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f154b07b740 1 -- 172.21.15.172:0/594935220 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3e6d13620 con 0x55d3e6fd12e0 2024-09-10T08:46:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f1529ffb640 1 -- 172.21.15.172:0/594935220 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f153c01a910 con 0x55d3e6fd12e0 2024-09-10T08:46:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f1529ffb640 1 -- 172.21.15.172:0/594935220 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f153403bc90 con 0x55d3e6fd12e0 2024-09-10T08:46:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f154b07b740 10 client.15471 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f154b07b740 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:46:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f154b07b740 20 client.15471 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.927+0000 7f154b07b740 1 --2- 172.21.15.172:0/594935220 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d3e6fe66d0 0x55d3e6fe8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.928+0000 7f154b07b740 4 client.15471 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.928+0000 7f154b07b740 1 -- 172.21.15.172:0/594935220 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3e6f41a20 con 0x55d3e6fe66d0 2024-09-10T08:46:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.928+0000 7f1546079640 1 --2- 172.21.15.172:0/594935220 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d3e6fe66d0 0x55d3e6fe8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.928+0000 7f1546079640 1 --2- 172.21.15.172:0/594935220 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d3e6fe66d0 0x55d3e6fe8b40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45b6dd9e3a4537c7 server_cookie=40f19789666231d0 in_seq=0 out_seq=0 2024-09-10T08:46:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.928+0000 7f1529ffb640 10 client.15471 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.929+0000 7f1529ffb640 1 -- 172.21.15.172:0/594935220 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3e6f41a20 con 0x55d3e6fe66d0 2024-09-10T08:46:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.929+0000 7f1529ffb640 10 client.15471 handle_command_reply: tid=0 2024-09-10T08:46:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.929+0000 7f154b07b740 1 client.15471 shutdown 2024-09-10T08:46:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.929+0000 7f154b07b740 1 -- 172.21.15.172:0/594935220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd12e0 msgr2=0x55d3e6fdb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.929+0000 7f154b07b740 1 --2- 172.21.15.172:0/594935220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd12e0 0x55d3e6fdb0e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f15340ed540 tx=0x7f153408b190 comp rx=0 tx=0).stop 2024-09-10T08:46:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.929+0000 7f154b07b740 1 -- 172.21.15.172:0/594935220 shutdown_connections 2024-09-10T08:46:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.929+0000 7f154b07b740 1 --2- 172.21.15.172:0/594935220 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d3e6fe66d0 0x55d3e6fe8b40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.929+0000 7f154b07b740 1 --2- 172.21.15.172:0/594935220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd6370 0x55d3e6fe2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.929+0000 7f154b07b740 1 --2- 172.21.15.172:0/594935220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd12e0 0x55d3e6fdb0e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.929+0000 7f1529ffb640 0 client.15471 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.929+0000 7f154b07b740 1 --2- 172.21.15.172:0/594935220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd09a0 0x55d3e6fdaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.929+0000 7f154b07b740 1 -- 172.21.15.172:0/594935220 >> 172.21.15.172:0/594935220 conn(0x55d3e6fcc010 msgr2=0x55d3e6fcd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.929+0000 7f154b07b740 1 -- 172.21.15.172:0/594935220 shutdown_connections 2024-09-10T08:46:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.929+0000 7f154b07b740 1 -- 172.21.15.172:0/594935220 wait complete. 2024-09-10T08:46:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.929+0000 7f154b07b740 20 client.15471 trim_cache size 0 max 16384 2024-09-10T08:46:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.943+0000 7f154b07b740 1 Processor -- start 2024-09-10T08:46:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f154b07b740 1 -- start start 2024-09-10T08:46:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f154b07b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd09a0 0x55d3e6fcb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f154b07b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd12e0 0x55d3e6fed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f154b07b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd6370 0x55d3e6fed980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f154b07b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f15401aa3f0 con 0x55d3e6fd09a0 2024-09-10T08:46:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f154b07b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f15401abf30 con 0x55d3e6fd12e0 2024-09-10T08:46:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f154b07b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f15401aaac0 con 0x55d3e6fd6370 2024-09-10T08:46:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f1546079640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd09a0 0x55d3e6fcb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f1546079640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd09a0 0x55d3e6fcb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55672/0 (socket says 172.21.15.172:55672) 2024-09-10T08:46:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f1546079640 1 -- 172.21.15.172:0/3898211900 learned_addr learned my addr 172.21.15.172:0/3898211900 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f1545878640 1 --2- 172.21.15.172:0/3898211900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd12e0 0x55d3e6fed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:57.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f154687a640 1 --2- 172.21.15.172:0/3898211900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd6370 0x55d3e6fed980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f1546079640 1 -- 172.21.15.172:0/3898211900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd6370 msgr2=0x55d3e6fed980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f1546079640 1 --2- 172.21.15.172:0/3898211900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd6370 0x55d3e6fed980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f1546079640 1 -- 172.21.15.172:0/3898211900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd12e0 msgr2=0x55d3e6fed440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f1546079640 1 --2- 172.21.15.172:0/3898211900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd12e0 0x55d3e6fed440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f1546079640 1 -- 172.21.15.172:0/3898211900 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e6fd9580 con 0x55d3e6fd09a0 2024-09-10T08:46:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f1545878640 1 --2- 172.21.15.172:0/3898211900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd12e0 0x55d3e6fed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f154687a640 1 --2- 172.21.15.172:0/3898211900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd6370 0x55d3e6fed980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f1546079640 1 --2- 172.21.15.172:0/3898211900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd09a0 0x55d3e6fcb0b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f153c00bc60 tx=0x7f153c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09b212e6ed62be1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f152bfff640 1 -- 172.21.15.172:0/3898211900 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c015040 con 0x55d3e6fd09a0 2024-09-10T08:46:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f152bfff640 1 -- 172.21.15.172:0/3898211900 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f153c005580 con 0x55d3e6fd09a0 2024-09-10T08:46:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.944+0000 7f152bfff640 1 -- 172.21.15.172:0/3898211900 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c004ac0 con 0x55d3e6fd09a0 2024-09-10T08:46:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.945+0000 7f154b07b740 1 -- 172.21.15.172:0/3898211900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd09a0 msgr2=0x55d3e6fcb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.945+0000 7f154b07b740 1 --2- 172.21.15.172:0/3898211900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd09a0 0x55d3e6fcb0b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f153c00bc60 tx=0x7f153c005850 comp rx=0 tx=0).stop 2024-09-10T08:46:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.945+0000 7f154b07b740 1 -- 172.21.15.172:0/3898211900 shutdown_connections 2024-09-10T08:46:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.945+0000 7f154b07b740 1 --2- 172.21.15.172:0/3898211900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd6370 0x55d3e6fed980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.945+0000 7f154b07b740 1 --2- 172.21.15.172:0/3898211900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd12e0 0x55d3e6fed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.945+0000 7f154b07b740 1 --2- 172.21.15.172:0/3898211900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd09a0 0x55d3e6fcb0b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.945+0000 7f154b07b740 1 -- 172.21.15.172:0/3898211900 >> 172.21.15.172:0/3898211900 conn(0x55d3e6fcc010 msgr2=0x55d3e6fca9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.945+0000 7f154b07b740 1 -- 172.21.15.172:0/3898211900 shutdown_connections 2024-09-10T08:46:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.945+0000 7f154b07b740 1 -- 172.21.15.172:0/3898211900 wait complete. 2024-09-10T08:46:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.945+0000 7f154b07b740 1 -- start start 2024-09-10T08:46:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.945+0000 7f154b07b740 1 Processor -- start 2024-09-10T08:46:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f154b07b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd09a0 0x55d3e704c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f154b07b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd12e0 0x55d3e704ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f154b07b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd6370 0x55d3e7053e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f154b07b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f15401ab860 con 0x55d3e6fd09a0 2024-09-10T08:46:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f154b07b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f15401ab190 con 0x55d3e6fd6370 2024-09-10T08:46:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f154b07b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f15401a83d0 con 0x55d3e6fd12e0 2024-09-10T08:46:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f154b07b740 10 client.0 _ll_register_callbacks cb 0x7f1534037eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f1546079640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd09a0 0x55d3e704c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f1546079640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd09a0 0x55d3e704c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55680/0 (socket says 172.21.15.172:55680) 2024-09-10T08:46:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f1546079640 1 -- 172.21.15.172:0/1276308571 learned_addr learned my addr 172.21.15.172:0/1276308571 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f154687a640 1 --2- 172.21.15.172:0/1276308571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd6370 0x55d3e7053e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f1545878640 1 --2- 172.21.15.172:0/1276308571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd12e0 0x55d3e704ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f1546079640 1 -- 172.21.15.172:0/1276308571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd12e0 msgr2=0x55d3e704ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f1546079640 1 --2- 172.21.15.172:0/1276308571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd12e0 0x55d3e704ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f1546079640 1 -- 172.21.15.172:0/1276308571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd6370 msgr2=0x55d3e7053e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f1546079640 1 --2- 172.21.15.172:0/1276308571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd6370 0x55d3e7053e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f1546079640 1 -- 172.21.15.172:0/1276308571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e7054470 con 0x55d3e6fd09a0 2024-09-10T08:46:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f1545878640 1 --2- 172.21.15.172:0/1276308571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd12e0 0x55d3e704ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f1546079640 1 --2- 172.21.15.172:0/1276308571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd09a0 0x55d3e704c930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f153c0059b0 tx=0x7f153c005490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6254594f9794c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f1529ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f1529ffb640 1 -- 172.21.15.172:0/1276308571 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c004c10 con 0x55d3e6fd09a0 2024-09-10T08:46:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f1529ffb640 1 -- 172.21.15.172:0/1276308571 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f153c004db0 con 0x55d3e6fd09a0 2024-09-10T08:46:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f154b07b740 1 -- 172.21.15.172:0/1276308571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f153c005580 con 0x55d3e6fd09a0 2024-09-10T08:46:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f1529ffb640 1 -- 172.21.15.172:0/1276308571 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f153c004fd0 con 0x55d3e6fd09a0 2024-09-10T08:46:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f154b07b740 10 client.15477 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.946+0000 7f154b07b740 1 -- 172.21.15.172:0/1276308571 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3e70556f0 con 0x55d3e6fd09a0 2024-09-10T08:46:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.947+0000 7f1529ffb640 1 -- 172.21.15.172:0/1276308571 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f153c005580 con 0x55d3e6fd09a0 2024-09-10T08:46:57.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.947+0000 7f1529ffb640 1 -- 172.21.15.172:0/1276308571 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f153c0114d0 con 0x55d3e6fd09a0 2024-09-10T08:46:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.947+0000 7f154b07b740 10 client.15477 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.947+0000 7f154b07b740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:46:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.947+0000 7f154b07b740 20 client.15477 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.947+0000 7f154b07b740 1 --2- 172.21.15.172:0/1276308571 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d3e7055ed0 0x55d3e70582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.947+0000 7f154b07b740 4 client.15477 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:46:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.947+0000 7f154b07b740 1 -- 172.21.15.172:0/1276308571 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d3e6f3bd70 con 0x55d3e7055ed0 2024-09-10T08:46:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.947+0000 7f1545878640 1 --2- 172.21.15.172:0/1276308571 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d3e7055ed0 0x55d3e70582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.947+0000 7f1545878640 1 --2- 172.21.15.172:0/1276308571 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d3e7055ed0 0x55d3e70582f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15e92eab2849568e server_cookie=40a3b94e7727c38 in_seq=0 out_seq=0 2024-09-10T08:46:57.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.947+0000 7f1529ffb640 10 client.15477 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.959+0000 7f1529ffb640 1 -- 172.21.15.172:0/1276308571 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d3e6f3bd70 con 0x55d3e7055ed0 2024-09-10T08:46:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.959+0000 7f1529ffb640 10 client.15477 handle_command_reply: tid=0 2024-09-10T08:46:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.959+0000 7f154b07b740 1 client.15477 shutdown 2024-09-10T08:46:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.959+0000 7f154b07b740 1 -- 172.21.15.172:0/1276308571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd09a0 msgr2=0x55d3e704c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.959+0000 7f154b07b740 1 --2- 172.21.15.172:0/1276308571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd09a0 0x55d3e704c930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f153c0059b0 tx=0x7f153c005490 comp rx=0 tx=0).stop 2024-09-10T08:46:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.959+0000 7f154b07b740 1 -- 172.21.15.172:0/1276308571 shutdown_connections 2024-09-10T08:46:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.959+0000 7f154b07b740 1 --2- 172.21.15.172:0/1276308571 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d3e7055ed0 0x55d3e70582f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.959+0000 7f154b07b740 1 --2- 172.21.15.172:0/1276308571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e6fd6370 0x55d3e7053e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.959+0000 7f1529ffb640 0 client.15477 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:46:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.959+0000 7f154b07b740 1 --2- 172.21.15.172:0/1276308571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d3e6fd12e0 0x55d3e704ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.959+0000 7f154b07b740 1 --2- 172.21.15.172:0/1276308571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3e6fd09a0 0x55d3e704c930 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.959+0000 7f154b07b740 1 -- 172.21.15.172:0/1276308571 >> 172.21.15.172:0/1276308571 conn(0x55d3e6fcc010 msgr2=0x55d3e6fca9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.960+0000 7f154b07b740 1 -- 172.21.15.172:0/1276308571 shutdown_connections 2024-09-10T08:46:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.960+0000 7f154b07b740 1 -- 172.21.15.172:0/1276308571 wait complete. 2024-09-10T08:46:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.960+0000 7f154b07b740 20 client.15477 trim_cache size 0 max 16384 2024-09-10T08:46:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:46:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:46:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:46:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:46:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.960+0000 7f15482dd640 1 -- 172.21.15.172:0/2904836431 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f152c0a2cb0 msgr2=0x7f152c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.960+0000 7f15482dd640 1 --2- 172.21.15.172:0/2904836431 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f152c0a2cb0 0x7f152c0a5170 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1538005fd0 tx=0x7f1538004a40 comp rx=0 tx=0).stop 2024-09-10T08:46:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.960+0000 7f15482dd640 1 -- 172.21.15.172:0/2904836431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1540102ea0 msgr2=0x7f1540198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.960+0000 7f15482dd640 1 --2- 172.21.15.172:0/2904836431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1540102ea0 0x7f1540198270 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1534004770 tx=0x7f15340042f0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.960+0000 7f15482dd640 1 -- 172.21.15.172:0/2904836431 shutdown_connections 2024-09-10T08:46:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.960+0000 7f15482dd640 1 --2- 172.21.15.172:0/2904836431 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f152c0a2cb0 0x7f152c0a5170 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.960+0000 7f15482dd640 1 --2- 172.21.15.172:0/2904836431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1540103860 0x7f154019f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.960+0000 7f15482dd640 1 --2- 172.21.15.172:0/2904836431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1540102ea0 0x7f1540198270 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.960+0000 7f15482dd640 1 --2- 172.21.15.172:0/2904836431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15400ffe30 0x7f1540197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.960+0000 7f15482dd640 1 -- 172.21.15.172:0/2904836431 >> 172.21.15.172:0/2904836431 conn(0x7f15400fb970 msgr2=0x7f1540106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.960+0000 7f15482dd640 1 -- 172.21.15.172:0/2904836431 shutdown_connections 2024-09-10T08:46:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:46:57.960+0000 7f15482dd640 1 -- 172.21.15.172:0/2904836431 wait complete. 2024-09-10T08:46:58.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.249+0000 7ff81fe9a640 1 -- 172.21.15.172:0/1277171072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff818101420 msgr2=0x7ff81810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:58.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.249+0000 7ff81fe9a640 1 --2- 172.21.15.172:0/1277171072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff818101420 0x7ff81810d950 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff810009cd0 tx=0x7ff81002a8e0 comp rx=0 tx=0).stop 2024-09-10T08:46:58.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.249+0000 7ff81fe9a640 1 -- 172.21.15.172:0/1277171072 shutdown_connections 2024-09-10T08:46:58.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.249+0000 7ff81fe9a640 1 --2- 172.21.15.172:0/1277171072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff818101420 0x7ff81810d950 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:58.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.249+0000 7ff81fe9a640 1 --2- 172.21.15.172:0/1277171072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff818100a80 0x7ff818100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:58.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.249+0000 7ff81fe9a640 1 --2- 172.21.15.172:0/1277171072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff818106a80 0x7ff818106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:58.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.249+0000 7ff81fe9a640 1 -- 172.21.15.172:0/1277171072 >> 172.21.15.172:0/1277171072 conn(0x7ff8180fc7a0 msgr2=0x7ff8180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:58.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.249+0000 7ff81fe9a640 1 -- 172.21.15.172:0/1277171072 shutdown_connections 2024-09-10T08:46:58.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.249+0000 7ff81fe9a640 1 -- 172.21.15.172:0/1277171072 wait complete. 2024-09-10T08:46:58.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81fe9a640 1 Processor -- start 2024-09-10T08:46:58.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81fe9a640 1 -- start start 2024-09-10T08:46:58.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81fe9a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff818100a80 0x7ff818198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:58.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81fe9a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff818101420 0x7ff8181988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:58.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81fe9a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff818106a80 0x7ff81819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:58.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81fe9a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff818110d30 con 0x7ff818100a80 2024-09-10T08:46:58.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81fe9a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff818110bb0 con 0x7ff818106a80 2024-09-10T08:46:58.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81fe9a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff818110eb0 con 0x7ff818101420 2024-09-10T08:46:58.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81dc36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff818100a80 0x7ff818198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:58.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81dc36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff818100a80 0x7ff818198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55702/0 (socket says 172.21.15.172:55702) 2024-09-10T08:46:58.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81dc36640 1 -- 172.21.15.172:0/1815751834 learned_addr learned my addr 172.21.15.172:0/1815751834 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:58.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81e437640 1 --2- 172.21.15.172:0/1815751834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff818106a80 0x7ff81819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:58.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81d435640 1 --2- 172.21.15.172:0/1815751834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff818101420 0x7ff8181988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:58.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81dc36640 1 -- 172.21.15.172:0/1815751834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff818101420 msgr2=0x7ff8181988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:58.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81dc36640 1 --2- 172.21.15.172:0/1815751834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff818101420 0x7ff8181988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:58.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81dc36640 1 -- 172.21.15.172:0/1815751834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff818106a80 msgr2=0x7ff81819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:58.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81dc36640 1 --2- 172.21.15.172:0/1815751834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff818106a80 0x7ff81819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:58.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81dc36640 1 -- 172.21.15.172:0/1815751834 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81819d310 con 0x7ff818100a80 2024-09-10T08:46:58.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.250+0000 7ff81dc36640 1 --2- 172.21.15.172:0/1815751834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff818100a80 0x7ff818198360 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff80800d770 tx=0x7ff80800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:58.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.251+0000 7ff80effd640 1 -- 172.21.15.172:0/1815751834 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808017070 con 0x7ff818100a80 2024-09-10T08:46:58.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.251+0000 7ff80effd640 1 -- 172.21.15.172:0/1815751834 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8080054d0 con 0x7ff818100a80 2024-09-10T08:46:58.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.251+0000 7ff81fe9a640 1 -- 172.21.15.172:0/1815751834 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81819d600 con 0x7ff818100a80 2024-09-10T08:46:58.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.251+0000 7ff80effd640 1 -- 172.21.15.172:0/1815751834 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808004cf0 con 0x7ff818100a80 2024-09-10T08:46:58.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.251+0000 7ff81fe9a640 1 -- 172.21.15.172:0/1815751834 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8181a4ee0 con 0x7ff818100a80 2024-09-10T08:46:58.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.252+0000 7ff81fe9a640 1 -- 172.21.15.172:0/1815751834 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff818100ee0 con 0x7ff818100a80 2024-09-10T08:46:58.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.253+0000 7ff80effd640 1 -- 172.21.15.172:0/1815751834 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff808004e90 con 0x7ff818100a80 2024-09-10T08:46:58.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.254+0000 7ff80effd640 1 --2- 172.21.15.172:0/1815751834 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff8040a2b80 0x7ff8040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:58.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.254+0000 7ff81d435640 1 --2- 172.21.15.172:0/1815751834 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff8040a2b80 0x7ff8040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:58.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.254+0000 7ff80effd640 1 -- 172.21.15.172:0/1815751834 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff8080cf080 con 0x7ff818100a80 2024-09-10T08:46:58.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.254+0000 7ff81d435640 1 --2- 172.21.15.172:0/1815751834 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff8040a2b80 0x7ff8040a5040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff81806b870 tx=0x7ff81400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:58.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.257+0000 7ff80effd640 1 -- 172.21.15.172:0/1815751834 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff80809b630 con 0x7ff818100a80 2024-09-10T08:46:58.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.460+0000 7ff81fe9a640 1 -- 172.21.15.172:0/1815751834 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8181a50b0 con 0x7ff818100a80 2024-09-10T08:46:58.460 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:58 smithi172 ceph-mon[30854]: pgmap v372: 129 pgs: 129 active+clean; 654 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-10T08:46:58.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.461+0000 7ff80effd640 1 -- 172.21.15.172:0/1815751834 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7ff80809f930 con 0x7ff818100a80 2024-09-10T08:46:58.461 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:46:58.462 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":29,"btime":"2024-09-10T08:46:37:242916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:46:36.267985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:46:58.462 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 29 2024-09-10T08:46:58.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.463+0000 7ff81fe9a640 1 -- 172.21.15.172:0/1815751834 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff8040a2b80 msgr2=0x7ff8040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:58.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.463+0000 7ff81fe9a640 1 --2- 172.21.15.172:0/1815751834 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff8040a2b80 0x7ff8040a5040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff81806b870 tx=0x7ff81400b390 comp rx=0 tx=0).stop 2024-09-10T08:46:58.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.463+0000 7ff81fe9a640 1 -- 172.21.15.172:0/1815751834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff818100a80 msgr2=0x7ff818198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:58.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.463+0000 7ff81fe9a640 1 --2- 172.21.15.172:0/1815751834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff818100a80 0x7ff818198360 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff80800d770 tx=0x7ff80800dc30 comp rx=0 tx=0).stop 2024-09-10T08:46:58.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.463+0000 7ff81fe9a640 1 -- 172.21.15.172:0/1815751834 shutdown_connections 2024-09-10T08:46:58.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.463+0000 7ff81fe9a640 1 --2- 172.21.15.172:0/1815751834 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff8040a2b80 0x7ff8040a5040 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:58.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.463+0000 7ff81fe9a640 1 --2- 172.21.15.172:0/1815751834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff818106a80 0x7ff81819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:58.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.463+0000 7ff81fe9a640 1 --2- 172.21.15.172:0/1815751834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff818101420 0x7ff8181988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:58.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.463+0000 7ff81fe9a640 1 --2- 172.21.15.172:0/1815751834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff818100a80 0x7ff818198360 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:58.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.463+0000 7ff81fe9a640 1 -- 172.21.15.172:0/1815751834 >> 172.21.15.172:0/1815751834 conn(0x7ff8180fc7a0 msgr2=0x7ff8181048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:58.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.463+0000 7ff81fe9a640 1 -- 172.21.15.172:0/1815751834 shutdown_connections 2024-09-10T08:46:58.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:58.463+0000 7ff81fe9a640 1 -- 172.21.15.172:0/1815751834 wait complete. 2024-09-10T08:46:58.643 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:58 smithi174 ceph-mon[34068]: pgmap v372: 129 pgs: 129 active+clean; 654 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-10T08:46:58.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:58 smithi191 ceph-mon[35314]: pgmap v372: 129 pgs: 129 active+clean; 654 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-10T08:46:59.146 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T08:44:11.310930+0000', 'modified': '2024-09-10T08:46:36.267985+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24488, 'mds_1': 24458, 'mds_2': 24457, 'mds_3': 24416, 'mds_4': 24406}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6839/1621056220', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6838', 'nonce': 1621056220}, {'type': 'v1', 'addr': '172.21.15.172:6839', 'nonce': 1621056220}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14622': {'gid': 14622, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6841/176040463', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6840', 'nonce': 176040463}, {'type': 'v1', 'addr': '172.21.15.172:6841', 'nonce': 176040463}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24406': {'gid': 24406, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.191:6833/1662718523', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6832', 'nonce': 1662718523}, {'type': 'v1', 'addr': '172.21.15.191:6833', 'nonce': 1662718523}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.174:6833/3418766024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 3418766024}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 3418766024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6835/1774553543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6834', 'nonce': 1774553543}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 1774553543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/3197756201', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 3197756201}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 3197756201}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6837/1802631256', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6836', 'nonce': 1802631256}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 1802631256}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.191:6839/1702936476', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6838', 'nonce': 1702936476}, {'type': 'v1', 'addr': '172.21.15.191:6839', 'nonce': 1702936476}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.174:6837/2399337052', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 2399337052}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 2399337052}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.174:6839/3029849701', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 3029849701}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 3029849701}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24406, 24458, 24416, 24457, 24488]} 2024-09-10T08:46:59.146 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T08:46:59.147 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.l status 2024-09-10T08:46:59.439 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:46:59.559 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:46:59 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1815751834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:59.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:46:59 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1815751834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:59.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:46:59 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1815751834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:46:59.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.956+0000 7f414908e640 1 -- 172.21.15.172:0/823895058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4144103660 msgr2=0x7f414410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:59.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.956+0000 7f414908e640 1 --2- 172.21.15.172:0/823895058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4144103660 0x7f414410bb50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4138009900 tx=0x7f413802a480 comp rx=0 tx=0).stop 2024-09-10T08:46:59.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.956+0000 7f414908e640 1 -- 172.21.15.172:0/823895058 shutdown_connections 2024-09-10T08:46:59.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.956+0000 7f414908e640 1 --2- 172.21.15.172:0/823895058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f414410c170 0x7f414410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.956+0000 7f414908e640 1 --2- 172.21.15.172:0/823895058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4144103660 0x7f414410bb50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.956+0000 7f414908e640 1 --2- 172.21.15.172:0/823895058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4144102d40 0x7f4144103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.956+0000 7f414908e640 1 -- 172.21.15.172:0/823895058 >> 172.21.15.172:0/823895058 conn(0x7f41440fc7a0 msgr2=0x7f41440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:59.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.956+0000 7f414908e640 1 -- 172.21.15.172:0/823895058 shutdown_connections 2024-09-10T08:46:59.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.956+0000 7f414908e640 1 -- 172.21.15.172:0/823895058 wait complete. 2024-09-10T08:46:59.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.957+0000 7f414908e640 1 Processor -- start 2024-09-10T08:46:59.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.957+0000 7f414908e640 1 -- start start 2024-09-10T08:46:59.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.957+0000 7f414908e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4144102d40 0x7f41441983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.957+0000 7f414908e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4144103660 0x7f4144198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.957+0000 7f414908e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f414410c170 0x7f414419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.957+0000 7f414908e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4144111e60 con 0x7f4144102d40 2024-09-10T08:46:59.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.957+0000 7f414908e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4144111ce0 con 0x7f414410c170 2024-09-10T08:46:59.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.957+0000 7f414908e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4144111fe0 con 0x7f4144103660 2024-09-10T08:46:59.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.957+0000 7f4142d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4144102d40 0x7f41441983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:59.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.958+0000 7f4142d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4144102d40 0x7f41441983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55712/0 (socket says 172.21.15.172:55712) 2024-09-10T08:46:59.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.958+0000 7f4142d9d640 1 -- 172.21.15.172:0/4197337726 learned_addr learned my addr 172.21.15.172:0/4197337726 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:59.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.958+0000 7f414359e640 1 --2- 172.21.15.172:0/4197337726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f414410c170 0x7f414419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:59.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.958+0000 7f414259c640 1 --2- 172.21.15.172:0/4197337726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4144103660 0x7f4144198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:59.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.958+0000 7f4142d9d640 1 -- 172.21.15.172:0/4197337726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4144103660 msgr2=0x7f4144198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:59.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.958+0000 7f4142d9d640 1 --2- 172.21.15.172:0/4197337726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4144103660 0x7f4144198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.958+0000 7f4142d9d640 1 -- 172.21.15.172:0/4197337726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f414410c170 msgr2=0x7f414419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:59.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.958+0000 7f4142d9d640 1 --2- 172.21.15.172:0/4197337726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f414410c170 0x7f414419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.958+0000 7f4142d9d640 1 -- 172.21.15.172:0/4197337726 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414419d380 con 0x7f4144102d40 2024-09-10T08:46:59.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.958+0000 7f4142d9d640 1 --2- 172.21.15.172:0/4197337726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4144102d40 0x7f41441983d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f413400b4a0 tx=0x7f413400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:59.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.958+0000 7f412bfff640 1 -- 172.21.15.172:0/4197337726 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134014070 con 0x7f4144102d40 2024-09-10T08:46:59.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.958+0000 7f414908e640 1 -- 172.21.15.172:0/4197337726 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414419d670 con 0x7f4144102d40 2024-09-10T08:46:59.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.958+0000 7f412bfff640 1 -- 172.21.15.172:0/4197337726 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4134010ac0 con 0x7f4144102d40 2024-09-10T08:46:59.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.958+0000 7f412bfff640 1 -- 172.21.15.172:0/4197337726 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134017c30 con 0x7f4144102d40 2024-09-10T08:46:59.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.958+0000 7f414908e640 1 -- 172.21.15.172:0/4197337726 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41441a4f50 con 0x7f4144102d40 2024-09-10T08:46:59.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.960+0000 7f412bfff640 1 -- 172.21.15.172:0/4197337726 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4134017070 con 0x7f4144102d40 2024-09-10T08:46:59.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.961+0000 7f414b467740 1 Processor -- start 2024-09-10T08:46:59.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.961+0000 7f414b467740 1 -- start start 2024-09-10T08:46:59.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.961+0000 7f414b467740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e6890 0x562bbf0e6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.961+0000 7f414b467740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0e71b0 0x562bbf0f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.961+0000 7f414b467740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0f5fc0 0x562bbf0f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.961+0000 7f414b467740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562bbf04d090 con 0x562bbf0e6890 2024-09-10T08:46:59.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.961+0000 7f414b467740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562bbeff4a70 con 0x562bbf0e71b0 2024-09-10T08:46:59.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.961+0000 7f414b467740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562bbf049050 con 0x562bbf0f5fc0 2024-09-10T08:46:59.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.961+0000 7f412bfff640 1 --2- 172.21.15.172:0/4197337726 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f412c0a2b80 0x7f412c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.961+0000 7f414359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e6890 0x562bbf0e6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:59.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.961+0000 7f414359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e6890 0x562bbf0e6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55716/0 (socket says 172.21.15.172:55716) 2024-09-10T08:46:59.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.961+0000 7f414359e640 1 -- 172.21.15.172:0/1205381649 learned_addr learned my addr 172.21.15.172:0/1205381649 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:59.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.961+0000 7f414359e640 1 --2- 172.21.15.172:0/4197337726 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f412c0a2b80 0x7f412c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:59.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.961+0000 7f414259c640 1 --2- 172.21.15.172:0/1205381649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0f5fc0 0x562bbf0f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:59.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.961+0000 7f412bfff640 1 -- 172.21.15.172:0/4197337726 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f41340ce9b0 con 0x7f4144102d40 2024-09-10T08:46:59.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.961+0000 7f414359e640 1 -- 172.21.15.172:0/1205381649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0f5fc0 msgr2=0x562bbf0f83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:59.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.962+0000 7f414359e640 1 --2- 172.21.15.172:0/1205381649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0f5fc0 0x562bbf0f83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.962+0000 7f4142d9d640 1 --2- 172.21.15.172:0/1205381649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0e71b0 0x562bbf0f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:59.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.962+0000 7f414359e640 1 -- 172.21.15.172:0/1205381649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0e71b0 msgr2=0x562bbf0f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:59.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.962+0000 7f414359e640 1 --2- 172.21.15.172:0/1205381649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0e71b0 0x562bbf0f5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.962+0000 7f414359e640 1 -- 172.21.15.172:0/1205381649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bbf001130 con 0x562bbf0e6890 2024-09-10T08:46:59.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.962+0000 7f4142d9d640 1 --2- 172.21.15.172:0/1205381649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0e71b0 0x562bbf0f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:59.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.962+0000 7f414359e640 1 --2- 172.21.15.172:0/1205381649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e6890 0x562bbf0e6c70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f413c005f40 tx=0x7f413c014f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b1210ed5209e60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:59.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.962+0000 7f41297fa640 1 -- 172.21.15.172:0/1205381649 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c012d90 con 0x562bbf0e6890 2024-09-10T08:46:59.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.962+0000 7f41297fa640 1 -- 172.21.15.172:0/1205381649 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f413c023070 con 0x562bbf0e6890 2024-09-10T08:46:59.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.962+0000 7f41297fa640 1 -- 172.21.15.172:0/1205381649 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c01de60 con 0x562bbf0e6890 2024-09-10T08:46:59.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.963+0000 7f414b467740 1 -- 172.21.15.172:0/1205381649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e6890 msgr2=0x562bbf0e6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:59.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.963+0000 7f414359e640 1 --2- 172.21.15.172:0/4197337726 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f412c0a2b80 0x7f412c0a5040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f413c022000 tx=0x7f413c020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:59.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.963+0000 7f414b467740 1 --2- 172.21.15.172:0/1205381649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e6890 0x562bbf0e6c70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f413c005f40 tx=0x7f413c014f70 comp rx=0 tx=0).stop 2024-09-10T08:46:59.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.963+0000 7f414b467740 1 -- 172.21.15.172:0/1205381649 shutdown_connections 2024-09-10T08:46:59.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.963+0000 7f414b467740 1 --2- 172.21.15.172:0/1205381649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0f5fc0 0x562bbf0f83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.963+0000 7f414b467740 1 --2- 172.21.15.172:0/1205381649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0e71b0 0x562bbf0f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.963+0000 7f414b467740 1 --2- 172.21.15.172:0/1205381649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e6890 0x562bbf0e6c70 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.963+0000 7f414b467740 1 -- 172.21.15.172:0/1205381649 >> 172.21.15.172:0/1205381649 conn(0x562bbf0e2f60 msgr2=0x562bbf0e3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:59.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.963+0000 7f414b467740 1 -- 172.21.15.172:0/1205381649 shutdown_connections 2024-09-10T08:46:59.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.963+0000 7f414b467740 1 -- 172.21.15.172:0/1205381649 wait complete. 2024-09-10T08:46:59.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.963+0000 7f414b467740 1 -- start start 2024-09-10T08:46:59.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.963+0000 7f414b467740 1 Processor -- start 2024-09-10T08:46:59.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414b467740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0e6890 0x562bbf0f5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414b467740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0e71b0 0x562bbf0f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414b467740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0f5fc0 0x562bbf0f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414b467740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562bbf0f8b90 con 0x562bbf0f5fc0 2024-09-10T08:46:59.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414b467740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562bbf007f20 con 0x562bbf0e71b0 2024-09-10T08:46:59.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414b467740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562bbf0f8d10 con 0x562bbf0e6890 2024-09-10T08:46:59.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414b467740 10 client.0 _ll_register_callbacks cb 0x562bbefd1030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:59.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0f5fc0 0x562bbf0f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:59.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0f5fc0 0x562bbf0f30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55730/0 (socket says 172.21.15.172:55730) 2024-09-10T08:46:59.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414359e640 1 -- 172.21.15.172:0/1036373124 learned_addr learned my addr 172.21.15.172:0/1036373124 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:59.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f4142d9d640 1 --2- 172.21.15.172:0/1036373124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0e6890 0x562bbf0f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:59.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414259c640 1 --2- 172.21.15.172:0/1036373124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0e71b0 0x562bbf0f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:59.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414359e640 1 -- 172.21.15.172:0/1036373124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0e6890 msgr2=0x562bbf0f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:59.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414359e640 1 --2- 172.21.15.172:0/1036373124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0e6890 0x562bbf0f5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414359e640 1 -- 172.21.15.172:0/1036373124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0e71b0 msgr2=0x562bbf0f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:59.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414359e640 1 --2- 172.21.15.172:0/1036373124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0e71b0 0x562bbf0f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414359e640 1 -- 172.21.15.172:0/1036373124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bbedbe2b0 con 0x562bbf0f5fc0 2024-09-10T08:46:59.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f4142d9d640 1 --2- 172.21.15.172:0/1036373124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0e6890 0x562bbf0f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:46:59.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414359e640 1 --2- 172.21.15.172:0/1036373124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0f5fc0 0x562bbf0f30e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f413c0148a0 tx=0x7f413c009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5513f675cf1c621 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:59.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f4126ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:59.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f4126ff5640 1 -- 172.21.15.172:0/1036373124 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c012d90 con 0x562bbf0f5fc0 2024-09-10T08:46:59.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f4126ff5640 1 -- 172.21.15.172:0/1036373124 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f413c01ec50 con 0x562bbf0f5fc0 2024-09-10T08:46:59.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.964+0000 7f414b467740 1 -- 172.21.15.172:0/1036373124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f413c023070 con 0x562bbf0f5fc0 2024-09-10T08:46:59.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.965+0000 7f4126ff5640 1 -- 172.21.15.172:0/1036373124 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f413c014050 con 0x562bbf0f5fc0 2024-09-10T08:46:59.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.965+0000 7f414b467740 10 client.15495 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:59.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.965+0000 7f414b467740 1 -- 172.21.15.172:0/1036373124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562bbf104430 con 0x562bbf0f5fc0 2024-09-10T08:46:59.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.965+0000 7f4126ff5640 1 -- 172.21.15.172:0/1036373124 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f413c023070 con 0x562bbf0f5fc0 2024-09-10T08:46:59.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.965+0000 7f4126ff5640 1 -- 172.21.15.172:0/1036373124 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f413c00f040 con 0x562bbf0f5fc0 2024-09-10T08:46:59.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.965+0000 7f414b467740 10 client.15495 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:59.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.965+0000 7f414b467740 10 client.15495 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-10T08:46:59.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.965+0000 7f414b467740 20 client.15495 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:59.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.965+0000 7f414b467740 1 --2- 172.21.15.172:0/1036373124 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x562bbf106710 0x562bbf107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.965+0000 7f414b467740 4 client.15495 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:46:59.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.965+0000 7f414b467740 1 -- 172.21.15.172:0/1036373124 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bbf05a600 con 0x562bbf106710 2024-09-10T08:46:59.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.966+0000 7f4142d9d640 1 --2- 172.21.15.172:0/1036373124 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x562bbf106710 0x562bbf107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:59.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.966+0000 7f4126ff5640 10 client.15495 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:46:59.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.966+0000 7f4142d9d640 1 --2- 172.21.15.172:0/1036373124 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x562bbf106710 0x562bbf107b80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2a43d6036e20562a server_cookie=21d858a07de465e9 in_seq=0 out_seq=0 2024-09-10T08:46:59.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.967+0000 7f4126ff5640 1 -- 172.21.15.172:0/1036373124 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562bbf05a600 con 0x562bbf106710 2024-09-10T08:46:59.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.967+0000 7f4126ff5640 10 client.15495 handle_command_reply: tid=0 2024-09-10T08:46:59.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.967+0000 7f414b467740 1 client.15495 shutdown 2024-09-10T08:46:59.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.967+0000 7f414b467740 1 -- 172.21.15.172:0/1036373124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0f5fc0 msgr2=0x562bbf0f30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:59.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.967+0000 7f414b467740 1 --2- 172.21.15.172:0/1036373124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0f5fc0 0x562bbf0f30e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f413c0148a0 tx=0x7f413c009b40 comp rx=0 tx=0).stop 2024-09-10T08:46:59.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.967+0000 7f414b467740 1 -- 172.21.15.172:0/1036373124 shutdown_connections 2024-09-10T08:46:59.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.967+0000 7f414b467740 1 --2- 172.21.15.172:0/1036373124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0f5fc0 0x562bbf0f30e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.967+0000 7f414b467740 1 --2- 172.21.15.172:0/1036373124 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x562bbf106710 0x562bbf107b80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.967+0000 7f414b467740 1 --2- 172.21.15.172:0/1036373124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0e71b0 0x562bbf0f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.967+0000 7f414b467740 1 --2- 172.21.15.172:0/1036373124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0e6890 0x562bbf0f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.967+0000 7f414b467740 1 -- 172.21.15.172:0/1036373124 >> 172.21.15.172:0/1036373124 conn(0x562bbf0e2f60 msgr2=0x562bbf0e8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:59.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.967+0000 7f4126ff5640 0 client.15495 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:46:59.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.967+0000 7f414b467740 1 -- 172.21.15.172:0/1036373124 shutdown_connections 2024-09-10T08:46:59.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.967+0000 7f414b467740 1 -- 172.21.15.172:0/1036373124 wait complete. 2024-09-10T08:46:59.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.967+0000 7f414b467740 20 client.15495 trim_cache size 0 max 16384 2024-09-10T08:46:59.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.984+0000 7f414b467740 1 Processor -- start 2024-09-10T08:46:59.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.984+0000 7f414b467740 1 -- start start 2024-09-10T08:46:59.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f414b467740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0e6890 0x562bbf107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f414b467740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e71b0 0x562bbf1074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f414b467740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0f5fc0 0x562bbf106040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f414b467740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562bbf105ec0 con 0x562bbf0e71b0 2024-09-10T08:46:59.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f414b467740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562bbf106590 con 0x562bbf0e6890 2024-09-10T08:46:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f414b467740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562bbf106c60 con 0x562bbf0f5fc0 2024-09-10T08:46:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f414259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e71b0 0x562bbf1074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f414259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e71b0 0x562bbf1074b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55736/0 (socket says 172.21.15.172:55736) 2024-09-10T08:46:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f414259c640 1 -- 172.21.15.172:0/1014318447 learned_addr learned my addr 172.21.15.172:0/1014318447 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f414359e640 1 --2- 172.21.15.172:0/1014318447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0f5fc0 0x562bbf106040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f414259c640 1 -- 172.21.15.172:0/1014318447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0f5fc0 msgr2=0x562bbf106040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f414259c640 1 --2- 172.21.15.172:0/1014318447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0f5fc0 0x562bbf106040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f414259c640 1 -- 172.21.15.172:0/1014318447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0e6890 msgr2=0x562bbf107b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f414259c640 1 --2- 172.21.15.172:0/1014318447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0e6890 0x562bbf107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f414259c640 1 -- 172.21.15.172:0/1014318447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bbf0e6fe0 con 0x562bbf0e71b0 2024-09-10T08:46:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f414359e640 1 --2- 172.21.15.172:0/1014318447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0f5fc0 0x562bbf106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:46:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f414259c640 1 --2- 172.21.15.172:0/1014318447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e71b0 0x562bbf1074b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4138033000 tx=0x7f4138032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a20c903b4583825 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:59.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f4128ff9640 1 -- 172.21.15.172:0/1014318447 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138035040 con 0x562bbf0e71b0 2024-09-10T08:46:59.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f4128ff9640 1 -- 172.21.15.172:0/1014318447 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41380316a0 con 0x562bbf0e71b0 2024-09-10T08:46:59.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.985+0000 7f4128ff9640 1 -- 172.21.15.172:0/1014318447 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138004750 con 0x562bbf0e71b0 2024-09-10T08:46:59.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.986+0000 7f414b467740 1 -- 172.21.15.172:0/1014318447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e71b0 msgr2=0x562bbf1074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:59.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.986+0000 7f414b467740 1 --2- 172.21.15.172:0/1014318447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e71b0 0x562bbf1074b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4138033000 tx=0x7f4138032dd0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.986+0000 7f414b467740 1 -- 172.21.15.172:0/1014318447 shutdown_connections 2024-09-10T08:46:59.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.986+0000 7f414b467740 1 --2- 172.21.15.172:0/1014318447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0f5fc0 0x562bbf106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.986+0000 7f414b467740 1 --2- 172.21.15.172:0/1014318447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e71b0 0x562bbf1074b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.986+0000 7f414b467740 1 --2- 172.21.15.172:0/1014318447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0e6890 0x562bbf107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.986+0000 7f414b467740 1 -- 172.21.15.172:0/1014318447 >> 172.21.15.172:0/1014318447 conn(0x562bbf0e2f60 msgr2=0x562bbf0e3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:59.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.986+0000 7f414b467740 1 -- 172.21.15.172:0/1014318447 shutdown_connections 2024-09-10T08:46:59.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.986+0000 7f414b467740 1 -- 172.21.15.172:0/1014318447 wait complete. 2024-09-10T08:46:59.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.986+0000 7f414b467740 1 -- start start 2024-09-10T08:46:59.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f414b467740 1 Processor -- start 2024-09-10T08:46:59.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f414b467740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e6890 0x562bbf110580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f414b467740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0e71b0 0x562bbf10e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f414b467740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0f5fc0 0x562bbf10e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f414b467740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562bbf107a00 con 0x562bbf0e6890 2024-09-10T08:46:59.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f414b467740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562bbf107330 con 0x562bbf0f5fc0 2024-09-10T08:46:59.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f414b467740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562bbf1080d0 con 0x562bbf0e71b0 2024-09-10T08:46:59.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f414b467740 10 client.0 _ll_register_callbacks cb 0x7f413c0148e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:46:59.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f4142d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e6890 0x562bbf110580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:59.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f4142d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e6890 0x562bbf110580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55746/0 (socket says 172.21.15.172:55746) 2024-09-10T08:46:59.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f4142d9d640 1 -- 172.21.15.172:0/1378989855 learned_addr learned my addr 172.21.15.172:0/1378989855 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:46:59.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f414359e640 1 --2- 172.21.15.172:0/1378989855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0f5fc0 0x562bbf10e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:59.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f414259c640 1 --2- 172.21.15.172:0/1378989855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0e71b0 0x562bbf10e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:59.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f4142d9d640 1 -- 172.21.15.172:0/1378989855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0e71b0 msgr2=0x562bbf10e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:59.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f4142d9d640 1 --2- 172.21.15.172:0/1378989855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0e71b0 0x562bbf10e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f4142d9d640 1 -- 172.21.15.172:0/1378989855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0f5fc0 msgr2=0x562bbf10e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:46:59.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f4142d9d640 1 --2- 172.21.15.172:0/1378989855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0f5fc0 0x562bbf10e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f4142d9d640 1 -- 172.21.15.172:0/1378989855 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bbf10eff0 con 0x562bbf0e6890 2024-09-10T08:46:59.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f4142d9d640 1 --2- 172.21.15.172:0/1378989855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e6890 0x562bbf110580 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f41340cc150 tx=0x7f4134000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638c356885dd9810 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:46:59.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f4126ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:46:59.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.987+0000 7f4126ff5640 1 -- 172.21.15.172:0/1378989855 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413401e040 con 0x562bbf0e6890 2024-09-10T08:46:59.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.988+0000 7f4126ff5640 1 -- 172.21.15.172:0/1378989855 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4134099000 con 0x562bbf0e6890 2024-09-10T08:46:59.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.988+0000 7f414b467740 1 -- 172.21.15.172:0/1378989855 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41380316a0 con 0x562bbf0e6890 2024-09-10T08:46:59.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.988+0000 7f4126ff5640 1 -- 172.21.15.172:0/1378989855 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4134099260 con 0x562bbf0e6890 2024-09-10T08:46:59.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.988+0000 7f414b467740 10 client.15501 fetch_fsmap learned FSMap version 29 2024-09-10T08:46:59.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.988+0000 7f414b467740 1 -- 172.21.15.172:0/1378989855 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562bbf10d0c0 con 0x562bbf0e6890 2024-09-10T08:46:59.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.988+0000 7f4126ff5640 1 -- 172.21.15.172:0/1378989855 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41380316a0 con 0x562bbf0e6890 2024-09-10T08:46:59.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.988+0000 7f4126ff5640 1 -- 172.21.15.172:0/1378989855 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f41340a3690 con 0x562bbf0e6890 2024-09-10T08:46:59.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.988+0000 7f414b467740 10 client.15501 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:46:59.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.988+0000 7f414b467740 10 client.15501 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-10T08:46:59.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.988+0000 7f414b467740 20 client.15501 populate_metadata read hostname 'smithi172' 2024-09-10T08:46:59.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.988+0000 7f414b467740 1 --2- 172.21.15.172:0/1378989855 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x562bbf174990 0x562bbf1736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:46:59.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.988+0000 7f414b467740 4 client.15501 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:46:59.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.988+0000 7f414b467740 1 -- 172.21.15.172:0/1378989855 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "status"}) -- 0x562bbf054b10 con 0x562bbf174990 2024-09-10T08:46:59.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.988+0000 7f414259c640 1 --2- 172.21.15.172:0/1378989855 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x562bbf174990 0x562bbf1736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:46:59.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.989+0000 7f414259c640 1 --2- 172.21.15.172:0/1378989855 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x562bbf174990 0x562bbf1736a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=97cb8d628a32ad38 server_cookie=50f4f65ec0e32e1a in_seq=0 out_seq=0 2024-09-10T08:46:59.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.989+0000 7f4126ff5640 10 client.15501 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:46:59.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.989+0000 7f4126ff5640 1 -- 172.21.15.172:0/1378989855 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562bbf054b10 con 0x562bbf174990 2024-09-10T08:46:59.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.989+0000 7f4126ff5640 10 client.15501 handle_command_reply: tid=0 2024-09-10T08:46:59.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.989+0000 7f414b467740 1 client.15501 shutdown 2024-09-10T08:46:59.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.989+0000 7f414b467740 1 -- 172.21.15.172:0/1378989855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e6890 msgr2=0x562bbf110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:46:59.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.989+0000 7f414b467740 1 --2- 172.21.15.172:0/1378989855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e6890 0x562bbf110580 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f41340cc150 tx=0x7f4134000bc0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.989+0000 7f414b467740 1 -- 172.21.15.172:0/1378989855 shutdown_connections 2024-09-10T08:46:59.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.989+0000 7f414b467740 1 --2- 172.21.15.172:0/1378989855 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x562bbf174990 0x562bbf1736a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.989+0000 7f414b467740 1 --2- 172.21.15.172:0/1378989855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562bbf0f5fc0 0x562bbf10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.989+0000 7f4126ff5640 0 client.15501 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:46:59.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.989+0000 7f414b467740 1 --2- 172.21.15.172:0/1378989855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562bbf0e71b0 0x562bbf10e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.989+0000 7f414b467740 1 --2- 172.21.15.172:0/1378989855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562bbf0e6890 0x562bbf110580 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:46:59.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.989+0000 7f414b467740 1 -- 172.21.15.172:0/1378989855 >> 172.21.15.172:0/1378989855 conn(0x562bbf0e2f60 msgr2=0x562bbf0ee240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:46:59.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.989+0000 7f414b467740 1 -- 172.21.15.172:0/1378989855 shutdown_connections 2024-09-10T08:47:00.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.989+0000 7f414b467740 1 -- 172.21.15.172:0/1378989855 wait complete. 2024-09-10T08:47:00.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.989+0000 7f414b467740 20 client.15501 trim_cache size 0 max 16384 2024-09-10T08:47:00.000 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:47:00.000 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:47:00.000 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 4, 2024-09-10T08:47:00.000 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 24406, 2024-09-10T08:47:00.000 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-09-10T08:47:00.001 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-09-10T08:47:00.001 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-09-10T08:47:00.001 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 125.766638653, 2024-09-10T08:47:00.001 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 29, 2024-09-10T08:47:00.001 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 92, 2024-09-10T08:47:00.001 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 92, 2024-09-10T08:47:00.001 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 209.708027297 2024-09-10T08:47:00.001 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:47:00.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.990+0000 7f414908e640 1 -- 172.21.15.172:0/4197337726 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f412c0a2b80 msgr2=0x7f412c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:00.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.990+0000 7f414908e640 1 --2- 172.21.15.172:0/4197337726 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f412c0a2b80 0x7f412c0a5040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f413c022000 tx=0x7f413c020000 comp rx=0 tx=0).stop 2024-09-10T08:47:00.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.990+0000 7f414908e640 1 -- 172.21.15.172:0/4197337726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4144102d40 msgr2=0x7f41441983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:00.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.990+0000 7f414908e640 1 --2- 172.21.15.172:0/4197337726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4144102d40 0x7f41441983d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f413400b4a0 tx=0x7f413400b960 comp rx=0 tx=0).stop 2024-09-10T08:47:00.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.990+0000 7f414908e640 1 -- 172.21.15.172:0/4197337726 shutdown_connections 2024-09-10T08:47:00.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.990+0000 7f414908e640 1 --2- 172.21.15.172:0/4197337726 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f412c0a2b80 0x7f412c0a5040 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:00.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.990+0000 7f414908e640 1 --2- 172.21.15.172:0/4197337726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f414410c170 0x7f414419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:00.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.990+0000 7f414908e640 1 --2- 172.21.15.172:0/4197337726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4144103660 0x7f4144198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:00.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.990+0000 7f414908e640 1 --2- 172.21.15.172:0/4197337726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4144102d40 0x7f41441983d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:00.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.990+0000 7f414908e640 1 -- 172.21.15.172:0/4197337726 >> 172.21.15.172:0/4197337726 conn(0x7f41440fc7a0 msgr2=0x7f41440fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:00.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.990+0000 7f414908e640 1 -- 172.21.15.172:0/4197337726 shutdown_connections 2024-09-10T08:47:00.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:46:59.990+0000 7f414908e640 1 -- 172.21.15.172:0/4197337726 wait complete. 2024-09-10T08:47:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:00 smithi172 ceph-mon[30854]: pgmap v373: 129 pgs: 129 active+clean; 654 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-10T08:47:00.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:00 smithi174 ceph-mon[34068]: pgmap v373: 129 pgs: 129 active+clean; 654 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-10T08:47:00.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:00 smithi191 ceph-mon[35314]: pgmap v373: 129 pgs: 129 active+clean; 654 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-10T08:47:00.855 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-10T08:47:01.100 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:01.488 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:01 smithi172 ceph-mon[30854]: pgmap v374: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-09-10T08:47:01.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.568+0000 7f2e51631640 1 -- 172.21.15.172:0/735521579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e4c106f10 msgr2=0x7f2e4c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:01.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.568+0000 7f2e51631640 1 --2- 172.21.15.172:0/735521579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e4c106f10 0x7f2e4c10ea90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009ca0 tx=0x7f2e4402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.568+0000 7f2e51631640 1 -- 172.21.15.172:0/735521579 shutdown_connections 2024-09-10T08:47:01.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.568+0000 7f2e51631640 1 --2- 172.21.15.172:0/735521579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e4c106f10 0x7f2e4c10ea90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.568+0000 7f2e51631640 1 --2- 172.21.15.172:0/735521579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e4c102790 0x7f2e4c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.568+0000 7f2e51631640 1 --2- 172.21.15.172:0/735521579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e4c101de0 0x7f2e4c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.568+0000 7f2e51631640 1 -- 172.21.15.172:0/735521579 >> 172.21.15.172:0/735521579 conn(0x7f2e4c0fd8c0 msgr2=0x7f2e4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:01.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.568+0000 7f2e51631640 1 -- 172.21.15.172:0/735521579 shutdown_connections 2024-09-10T08:47:01.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.568+0000 7f2e51631640 1 -- 172.21.15.172:0/735521579 wait complete. 2024-09-10T08:47:01.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.569+0000 7f2e51631640 1 Processor -- start 2024-09-10T08:47:01.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.569+0000 7f2e51631640 1 -- start start 2024-09-10T08:47:01.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.569+0000 7f2e51631640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e4c101de0 0x7f2e4c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.569+0000 7f2e51631640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e4c102790 0x7f2e4c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.569+0000 7f2e51631640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e4c106f10 0x7f2e4c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.569+0000 7f2e51631640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c111e60 con 0x7f2e4c101de0 2024-09-10T08:47:01.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.569+0000 7f2e51631640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c111ce0 con 0x7f2e4c106f10 2024-09-10T08:47:01.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.569+0000 7f2e51631640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c111fe0 con 0x7f2e4c102790 2024-09-10T08:47:01.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.569+0000 7f2e4affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e4c101de0 0x7f2e4c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:01.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.569+0000 7f2e4affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e4c101de0 0x7f2e4c19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55758/0 (socket says 172.21.15.172:55758) 2024-09-10T08:47:01.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.569+0000 7f2e4affd640 1 -- 172.21.15.172:0/794646349 learned_addr learned my addr 172.21.15.172:0/794646349 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:01.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.569+0000 7f2e4b7fe640 1 --2- 172.21.15.172:0/794646349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e4c106f10 0x7f2e4c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:01.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.569+0000 7f2e43fff640 1 --2- 172.21.15.172:0/794646349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e4c102790 0x7f2e4c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:01.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.569+0000 7f2e4affd640 1 -- 172.21.15.172:0/794646349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e4c102790 msgr2=0x7f2e4c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:01.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.570+0000 7f2e4affd640 1 --2- 172.21.15.172:0/794646349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e4c102790 0x7f2e4c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.570+0000 7f2e4affd640 1 -- 172.21.15.172:0/794646349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e4c106f10 msgr2=0x7f2e4c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:01.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.570+0000 7f2e4affd640 1 --2- 172.21.15.172:0/794646349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e4c106f10 0x7f2e4c19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.570+0000 7f2e4affd640 1 -- 172.21.15.172:0/794646349 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c19f5d0 con 0x7f2e4c101de0 2024-09-10T08:47:01.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.570+0000 7f2e43fff640 1 --2- 172.21.15.172:0/794646349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e4c102790 0x7f2e4c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:01.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.570+0000 7f2e4affd640 1 --2- 172.21.15.172:0/794646349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e4c101de0 0x7f2e4c19a620 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00b4a0 tx=0x7f2e3c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:01.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.570+0000 7f2e48ff9640 1 -- 172.21.15.172:0/794646349 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c014070 con 0x7f2e4c101de0 2024-09-10T08:47:01.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.570+0000 7f2e48ff9640 1 -- 172.21.15.172:0/794646349 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e3c010ac0 con 0x7f2e4c101de0 2024-09-10T08:47:01.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.570+0000 7f2e51631640 1 -- 172.21.15.172:0/794646349 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4c19f8c0 con 0x7f2e4c101de0 2024-09-10T08:47:01.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.570+0000 7f2e48ff9640 1 -- 172.21.15.172:0/794646349 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c017c30 con 0x7f2e4c101de0 2024-09-10T08:47:01.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.570+0000 7f2e51631640 1 -- 172.21.15.172:0/794646349 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4c1a71a0 con 0x7f2e4c101de0 2024-09-10T08:47:01.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.572+0000 7f2e48ff9640 1 -- 172.21.15.172:0/794646349 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2e3c017070 con 0x7f2e4c101de0 2024-09-10T08:47:01.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.572+0000 7f2e53a0a740 1 Processor -- start 2024-09-10T08:47:01.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.573+0000 7f2e53a0a740 1 -- start start 2024-09-10T08:47:01.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.573+0000 7f2e53a0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d89b49d0 0x5578d89b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.573+0000 7f2e53a0a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b52f0 0x5578d89bf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.573+0000 7f2e53a0a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89bfbb0 0x5578d89c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.573+0000 7f2e53a0a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5578d891b090 con 0x5578d89b52f0 2024-09-10T08:47:01.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.573+0000 7f2e53a0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578d88c2a70 con 0x5578d89b49d0 2024-09-10T08:47:01.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.573+0000 7f2e53a0a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5578d8917050 con 0x5578d89bfbb0 2024-09-10T08:47:01.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.573+0000 7f2e4affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b52f0 0x5578d89bf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.573+0000 7f2e48ff9640 1 --2- 172.21.15.172:0/794646349 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e380a2b80 0x7f2e380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.573+0000 7f2e4affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b52f0 0x5578d89bf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55768/0 (socket says 172.21.15.172:55768) 2024-09-10T08:47:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.573+0000 7f2e4affd640 1 -- 172.21.15.172:0/2821834430 learned_addr learned my addr 172.21.15.172:0/2821834430 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.573+0000 7f2e4b7fe640 1 --2- 172.21.15.172:0/2821834430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d89b49d0 0x5578d89b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.573+0000 7f2e4b7fe640 1 --2- 172.21.15.172:0/794646349 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e380a2b80 0x7f2e380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.573+0000 7f2e43fff640 1 --2- 172.21.15.172:0/2821834430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89bfbb0 0x5578d89c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.573+0000 7f2e48ff9640 1 -- 172.21.15.172:0/794646349 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2e3c0ceb20 con 0x7f2e4c101de0 2024-09-10T08:47:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.574+0000 7f2e4affd640 1 -- 172.21.15.172:0/2821834430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89bfbb0 msgr2=0x5578d89c1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.574+0000 7f2e4affd640 1 --2- 172.21.15.172:0/2821834430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89bfbb0 0x5578d89c1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.574+0000 7f2e4affd640 1 -- 172.21.15.172:0/2821834430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d89b49d0 msgr2=0x5578d89b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.574+0000 7f2e4affd640 1 --2- 172.21.15.172:0/2821834430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d89b49d0 0x5578d89b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.574+0000 7f2e4affd640 1 -- 172.21.15.172:0/2821834430 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d88cf130 con 0x5578d89b52f0 2024-09-10T08:47:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.574+0000 7f2e4b7fe640 1 --2- 172.21.15.172:0/794646349 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e380a2b80 0x7f2e380a5040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009430 tx=0x7f2e44005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.574+0000 7f2e4affd640 1 --2- 172.21.15.172:0/2821834430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b52f0 0x5578d89bf670 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c09b1e0 tx=0x7f2e3c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd882e2ff926e1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.574+0000 7f2e417fa640 1 -- 172.21.15.172:0/2821834430 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c01e070 con 0x5578d89b52f0 2024-09-10T08:47:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.574+0000 7f2e417fa640 1 -- 172.21.15.172:0/2821834430 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e3c0bb110 con 0x5578d89b52f0 2024-09-10T08:47:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.574+0000 7f2e417fa640 1 -- 172.21.15.172:0/2821834430 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c099320 con 0x5578d89b52f0 2024-09-10T08:47:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.575+0000 7f2e53a0a740 1 -- 172.21.15.172:0/2821834430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b52f0 msgr2=0x5578d89bf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.575+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/2821834430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b52f0 0x5578d89bf670 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c09b1e0 tx=0x7f2e3c00bf90 comp rx=0 tx=0).stop 2024-09-10T08:47:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.575+0000 7f2e53a0a740 1 -- 172.21.15.172:0/2821834430 shutdown_connections 2024-09-10T08:47:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.575+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/2821834430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89bfbb0 0x5578d89c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.575+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/2821834430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b52f0 0x5578d89bf670 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.575+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/2821834430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d89b49d0 0x5578d89b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.575+0000 7f2e53a0a740 1 -- 172.21.15.172:0/2821834430 >> 172.21.15.172:0/2821834430 conn(0x5578d89b0f60 msgr2=0x5578d89b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.575+0000 7f2e53a0a740 1 -- 172.21.15.172:0/2821834430 shutdown_connections 2024-09-10T08:47:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.575+0000 7f2e53a0a740 1 -- 172.21.15.172:0/2821834430 wait complete. 2024-09-10T08:47:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.575+0000 7f2e53a0a740 1 -- start start 2024-09-10T08:47:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.575+0000 7f2e53a0a740 1 Processor -- start 2024-09-10T08:47:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e53a0a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89b49d0 0x5578d89bf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e53a0a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b52f0 0x5578d89bc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e53a0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d89bfbb0 0x5578d89bcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e53a0a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5578d89c2780 con 0x5578d89b52f0 2024-09-10T08:47:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e53a0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578d88d5f20 con 0x5578d89bfbb0 2024-09-10T08:47:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e53a0a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5578d89c2900 con 0x5578d89b49d0 2024-09-10T08:47:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e53a0a740 10 client.0 _ll_register_callbacks cb 0x5578d889f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e43fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b52f0 0x5578d89bc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e43fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b52f0 0x5578d89bc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55776/0 (socket says 172.21.15.172:55776) 2024-09-10T08:47:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e43fff640 1 -- 172.21.15.172:0/702459303 learned_addr learned my addr 172.21.15.172:0/702459303 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e4affd640 1 --2- 172.21.15.172:0/702459303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89b49d0 0x5578d89bf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e43fff640 1 -- 172.21.15.172:0/702459303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89b49d0 msgr2=0x5578d89bf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e43fff640 1 --2- 172.21.15.172:0/702459303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89b49d0 0x5578d89bf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e43fff640 1 -- 172.21.15.172:0/702459303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d89bfbb0 msgr2=0x5578d89bcd30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:47:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e43fff640 1 --2- 172.21.15.172:0/702459303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d89bfbb0 0x5578d89bcd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e43fff640 1 -- 172.21.15.172:0/702459303 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d868c2b0 con 0x5578d89b52f0 2024-09-10T08:47:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e4affd640 1 --2- 172.21.15.172:0/702459303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89b49d0 0x5578d89bf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e43fff640 1 --2- 172.21.15.172:0/702459303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b52f0 0x5578d89bc7f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2e340106e0 tx=0x7f2e34010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edfcc26722917f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e327fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e327fc640 1 -- 172.21.15.172:0/702459303 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3401d060 con 0x5578d89b52f0 2024-09-10T08:47:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e53a0a740 1 -- 172.21.15.172:0/702459303 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e3c0bb110 con 0x5578d89b52f0 2024-09-10T08:47:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.576+0000 7f2e327fc640 1 -- 172.21.15.172:0/702459303 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2e3400eb90 con 0x5578d89b52f0 2024-09-10T08:47:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.577+0000 7f2e327fc640 1 -- 172.21.15.172:0/702459303 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e34024050 con 0x5578d89b52f0 2024-09-10T08:47:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.577+0000 7f2e53a0a740 10 client.15513 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.577+0000 7f2e53a0a740 1 -- 172.21.15.172:0/702459303 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5578d89ce1a0 con 0x5578d89b52f0 2024-09-10T08:47:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.577+0000 7f2e327fc640 1 -- 172.21.15.172:0/702459303 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e3c0bb110 con 0x5578d89b52f0 2024-09-10T08:47:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.577+0000 7f2e327fc640 1 -- 172.21.15.172:0/702459303 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2e34016330 con 0x5578d89b52f0 2024-09-10T08:47:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.577+0000 7f2e53a0a740 10 client.15513 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.577+0000 7f2e53a0a740 10 client.15513 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T08:47:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.577+0000 7f2e53a0a740 20 client.15513 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.577+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/702459303 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5578d89d0800 0x5578d89d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.577+0000 7f2e53a0a740 4 client.15513 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.577+0000 7f2e53a0a740 1 -- 172.21.15.172:0/702459303 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578d8928600 con 0x5578d89d0800 2024-09-10T08:47:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.577+0000 7f2e4affd640 1 --2- 172.21.15.172:0/702459303 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5578d89d0800 0x5578d89d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.578+0000 7f2e4affd640 1 --2- 172.21.15.172:0/702459303 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5578d89d0800 0x5578d89d1c70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f20714ee15c5a131 server_cookie=963c2dee38e8b817 in_seq=0 out_seq=0 2024-09-10T08:47:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.578+0000 7f2e327fc640 10 client.15513 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:47:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.578+0000 7f2e327fc640 1 -- 172.21.15.172:0/702459303 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578d8928600 con 0x5578d89d0800 2024-09-10T08:47:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.579+0000 7f2e327fc640 10 client.15513 handle_command_reply: tid=0 2024-09-10T08:47:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.579+0000 7f2e53a0a740 1 client.15513 shutdown 2024-09-10T08:47:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.579+0000 7f2e53a0a740 1 -- 172.21.15.172:0/702459303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b52f0 msgr2=0x5578d89bc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.579+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/702459303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b52f0 0x5578d89bc7f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2e340106e0 tx=0x7f2e34010ba0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.579+0000 7f2e53a0a740 1 -- 172.21.15.172:0/702459303 shutdown_connections 2024-09-10T08:47:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.579+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/702459303 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5578d89d0800 0x5578d89d1c70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.579+0000 7f2e327fc640 0 client.15513 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:47:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.579+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/702459303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d89bfbb0 0x5578d89bcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.579+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/702459303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b52f0 0x5578d89bc7f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.579+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/702459303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89b49d0 0x5578d89bf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.579+0000 7f2e53a0a740 1 -- 172.21.15.172:0/702459303 >> 172.21.15.172:0/702459303 conn(0x5578d89b0f60 msgr2=0x5578d89b6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:01.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.579+0000 7f2e53a0a740 1 -- 172.21.15.172:0/702459303 shutdown_connections 2024-09-10T08:47:01.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.579+0000 7f2e53a0a740 1 -- 172.21.15.172:0/702459303 wait complete. 2024-09-10T08:47:01.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.579+0000 7f2e53a0a740 20 client.15513 trim_cache size 0 max 16384 2024-09-10T08:47:01.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.596+0000 7f2e53a0a740 1 Processor -- start 2024-09-10T08:47:01.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.596+0000 7f2e53a0a740 1 -- start start 2024-09-10T08:47:01.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.597+0000 7f2e53a0a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b49d0 0x5578d89be650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.597+0000 7f2e53a0a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89b52f0 0x5578d89b9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.597+0000 7f2e53a0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d89bfbb0 0x5578d89b9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.597+0000 7f2e53a0a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5578d89cffb0 con 0x5578d89b49d0 2024-09-10T08:47:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.597+0000 7f2e53a0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578d89d0680 con 0x5578d89bfbb0 2024-09-10T08:47:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.597+0000 7f2e53a0a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5578d89d0d50 con 0x5578d89b52f0 2024-09-10T08:47:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.597+0000 7f2e4affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b49d0 0x5578d89be650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.597+0000 7f2e4affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b49d0 0x5578d89be650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55786/0 (socket says 172.21.15.172:55786) 2024-09-10T08:47:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.597+0000 7f2e4affd640 1 -- 172.21.15.172:0/989996037 learned_addr learned my addr 172.21.15.172:0/989996037 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.597+0000 7f2e4affd640 1 -- 172.21.15.172:0/989996037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89b52f0 msgr2=0x5578d89b9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.597+0000 7f2e4affd640 1 --2- 172.21.15.172:0/989996037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89b52f0 0x5578d89b9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.597+0000 7f2e4affd640 1 -- 172.21.15.172:0/989996037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d89bfbb0 msgr2=0x5578d89b9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.597+0000 7f2e4affd640 1 --2- 172.21.15.172:0/989996037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d89bfbb0 0x5578d89b9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.597+0000 7f2e4affd640 1 -- 172.21.15.172:0/989996037 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d89cfde0 con 0x5578d89b49d0 2024-09-10T08:47:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.597+0000 7f2e4affd640 1 --2- 172.21.15.172:0/989996037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b49d0 0x5578d89be650 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0979f0 tx=0x7f2e3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2133bcd512d3c59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.597+0000 7f2e40ff9640 1 -- 172.21.15.172:0/989996037 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c01e040 con 0x5578d89b49d0 2024-09-10T08:47:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.598+0000 7f2e40ff9640 1 -- 172.21.15.172:0/989996037 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e3c09b720 con 0x5578d89b49d0 2024-09-10T08:47:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.598+0000 7f2e40ff9640 1 -- 172.21.15.172:0/989996037 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c010c70 con 0x5578d89b49d0 2024-09-10T08:47:01.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.598+0000 7f2e53a0a740 1 -- 172.21.15.172:0/989996037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b49d0 msgr2=0x5578d89be650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:01.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.598+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/989996037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b49d0 0x5578d89be650 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0979f0 tx=0x7f2e3c0046c0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.598+0000 7f2e53a0a740 1 -- 172.21.15.172:0/989996037 shutdown_connections 2024-09-10T08:47:01.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.598+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/989996037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d89bfbb0 0x5578d89b9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.598+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/989996037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89b52f0 0x5578d89b9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.598+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/989996037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b49d0 0x5578d89be650 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.598+0000 7f2e53a0a740 1 -- 172.21.15.172:0/989996037 >> 172.21.15.172:0/989996037 conn(0x5578d89b0f60 msgr2=0x5578d89bdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:01.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.598+0000 7f2e53a0a740 1 -- 172.21.15.172:0/989996037 shutdown_connections 2024-09-10T08:47:01.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.598+0000 7f2e53a0a740 1 -- 172.21.15.172:0/989996037 wait complete. 2024-09-10T08:47:01.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e53a0a740 1 -- start start 2024-09-10T08:47:01.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e53a0a740 1 Processor -- start 2024-09-10T08:47:01.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e53a0a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b49d0 0x5578d89d55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e53a0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d89b52f0 0x5578d89d5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e53a0a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89bfbb0 0x5578d8a32f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e53a0a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5578d89d1af0 con 0x5578d89b49d0 2024-09-10T08:47:01.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e53a0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578d89d1420 con 0x5578d89b52f0 2024-09-10T08:47:01.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e53a0a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5578d89d21c0 con 0x5578d89bfbb0 2024-09-10T08:47:01.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e53a0a740 10 client.0 _ll_register_callbacks cb 0x7f2e3400b630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:01.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e4affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b49d0 0x5578d89d55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:01.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e4affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b49d0 0x5578d89d55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55796/0 (socket says 172.21.15.172:55796) 2024-09-10T08:47:01.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e4affd640 1 -- 172.21.15.172:0/2094997230 learned_addr learned my addr 172.21.15.172:0/2094997230 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:01.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e4affd640 1 -- 172.21.15.172:0/2094997230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89bfbb0 msgr2=0x5578d8a32f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:01.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e4affd640 1 --2- 172.21.15.172:0/2094997230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89bfbb0 0x5578d8a32f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e4affd640 1 -- 172.21.15.172:0/2094997230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d89b52f0 msgr2=0x5578d89d5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:01.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e4affd640 1 --2- 172.21.15.172:0/2094997230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d89b52f0 0x5578d89d5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e4affd640 1 -- 172.21.15.172:0/2094997230 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d8a33580 con 0x5578d89b49d0 2024-09-10T08:47:01.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e4affd640 1 --2- 172.21.15.172:0/2094997230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b49d0 0x5578d89d55a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0cee60 tx=0x7f2e3c009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efeb9690e8298126 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:01.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.599+0000 7f2e327fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:01.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.600+0000 7f2e327fc640 1 -- 172.21.15.172:0/2094997230 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c01e050 con 0x5578d89b49d0 2024-09-10T08:47:01.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.600+0000 7f2e327fc640 1 -- 172.21.15.172:0/2094997230 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2e3c010c70 con 0x5578d89b49d0 2024-09-10T08:47:01.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.600+0000 7f2e53a0a740 1 -- 172.21.15.172:0/2094997230 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e3c09b720 con 0x5578d89b49d0 2024-09-10T08:47:01.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.600+0000 7f2e327fc640 1 -- 172.21.15.172:0/2094997230 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e3c0bb110 con 0x5578d89b49d0 2024-09-10T08:47:01.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.600+0000 7f2e53a0a740 10 client.15519 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:01.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.600+0000 7f2e53a0a740 1 -- 172.21.15.172:0/2094997230 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5578d8a3a930 con 0x5578d89b49d0 2024-09-10T08:47:01.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.600+0000 7f2e327fc640 1 -- 172.21.15.172:0/2094997230 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e3c09b720 con 0x5578d89b49d0 2024-09-10T08:47:01.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.600+0000 7f2e327fc640 1 -- 172.21.15.172:0/2094997230 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2e3c09b2f0 con 0x5578d89b49d0 2024-09-10T08:47:01.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.600+0000 7f2e53a0a740 10 client.15519 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:01.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.600+0000 7f2e53a0a740 10 client.15519 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T08:47:01.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.600+0000 7f2e53a0a740 20 client.15519 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:01.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.600+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/2094997230 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5578d89d0800 0x5578d8a3db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:01.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.600+0000 7f2e53a0a740 4 client.15519 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:47:01.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.600+0000 7f2e53a0a740 1 -- 172.21.15.172:0/2094997230 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "status"}) -- 0x5578d8922b10 con 0x5578d89d0800 2024-09-10T08:47:01.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.601+0000 7f2e43fff640 1 --2- 172.21.15.172:0/2094997230 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5578d89d0800 0x5578d8a3db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:01.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.601+0000 7f2e327fc640 10 client.15519 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:47:01.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.601+0000 7f2e43fff640 1 --2- 172.21.15.172:0/2094997230 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5578d89d0800 0x5578d8a3db40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b01367cdd3bd2d59 server_cookie=d164417ad48045d4 in_seq=0 out_seq=0 2024-09-10T08:47:01.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.601+0000 7f2e327fc640 1 -- 172.21.15.172:0/2094997230 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5578d8922b10 con 0x5578d89d0800 2024-09-10T08:47:01.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.601+0000 7f2e327fc640 10 client.15519 handle_command_reply: tid=0 2024-09-10T08:47:01.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.601+0000 7f2e53a0a740 1 client.15519 shutdown 2024-09-10T08:47:01.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.601+0000 7f2e53a0a740 1 -- 172.21.15.172:0/2094997230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b49d0 msgr2=0x5578d89d55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:01.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.601+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/2094997230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b49d0 0x5578d89d55a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0cee60 tx=0x7f2e3c009320 comp rx=0 tx=0).stop 2024-09-10T08:47:01.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.601+0000 7f2e53a0a740 1 -- 172.21.15.172:0/2094997230 shutdown_connections 2024-09-10T08:47:01.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.601+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/2094997230 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5578d89d0800 0x5578d8a3db40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.601+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/2094997230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578d89bfbb0 0x5578d8a32f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.601+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/2094997230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d89b52f0 0x5578d89d5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.601+0000 7f2e53a0a740 1 --2- 172.21.15.172:0/2094997230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578d89b49d0 0x5578d89d55a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.601+0000 7f2e53a0a740 1 -- 172.21.15.172:0/2094997230 >> 172.21.15.172:0/2094997230 conn(0x5578d89b0f60 msgr2=0x5578d89be680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:01.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.601+0000 7f2e327fc640 0 client.15519 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:47:01.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.602+0000 7f2e53a0a740 1 -- 172.21.15.172:0/2094997230 shutdown_connections 2024-09-10T08:47:01.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.602+0000 7f2e53a0a740 1 -- 172.21.15.172:0/2094997230 wait complete. 2024-09-10T08:47:01.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.602+0000 7f2e53a0a740 20 client.15519 trim_cache size 0 max 16384 2024-09-10T08:47:01.610 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:47:01.611 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:47:01.611 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 3, 2024-09-10T08:47:01.611 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 24416, 2024-09-10T08:47:01.611 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-09-10T08:47:01.611 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-09-10T08:47:01.611 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-09-10T08:47:01.611 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 128.39049637900001, 2024-09-10T08:47:01.611 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 29, 2024-09-10T08:47:01.611 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 92, 2024-09-10T08:47:01.611 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 92, 2024-09-10T08:47:01.611 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 208.52205409600001 2024-09-10T08:47:01.612 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:47:01.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.602+0000 7f2e51631640 1 -- 172.21.15.172:0/794646349 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e380a2b80 msgr2=0x7f2e380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:01.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.602+0000 7f2e51631640 1 --2- 172.21.15.172:0/794646349 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e380a2b80 0x7f2e380a5040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009430 tx=0x7f2e44005f50 comp rx=0 tx=0).stop 2024-09-10T08:47:01.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.602+0000 7f2e51631640 1 -- 172.21.15.172:0/794646349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e4c101de0 msgr2=0x7f2e4c19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:01.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.602+0000 7f2e51631640 1 --2- 172.21.15.172:0/794646349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e4c101de0 0x7f2e4c19a620 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00b4a0 tx=0x7f2e3c00b960 comp rx=0 tx=0).stop 2024-09-10T08:47:01.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.602+0000 7f2e51631640 1 -- 172.21.15.172:0/794646349 shutdown_connections 2024-09-10T08:47:01.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.602+0000 7f2e51631640 1 --2- 172.21.15.172:0/794646349 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e380a2b80 0x7f2e380a5040 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.602+0000 7f2e51631640 1 --2- 172.21.15.172:0/794646349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e4c106f10 0x7f2e4c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.602+0000 7f2e51631640 1 --2- 172.21.15.172:0/794646349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e4c102790 0x7f2e4c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.602+0000 7f2e51631640 1 --2- 172.21.15.172:0/794646349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e4c101de0 0x7f2e4c19a620 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:01.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.602+0000 7f2e51631640 1 -- 172.21.15.172:0/794646349 >> 172.21.15.172:0/794646349 conn(0x7f2e4c0fd8c0 msgr2=0x7f2e4c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:01.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.602+0000 7f2e51631640 1 -- 172.21.15.172:0/794646349 shutdown_connections 2024-09-10T08:47:01.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:01.602+0000 7f2e51631640 1 -- 172.21.15.172:0/794646349 wait complete. 2024-09-10T08:47:01.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:01 smithi174 ceph-mon[34068]: pgmap v374: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-09-10T08:47:01.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:01 smithi191 ceph-mon[35314]: pgmap v374: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-09-10T08:47:02.356 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T08:47:02.607 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:03.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.070+0000 7f970404b640 1 -- 172.21.15.172:0/1794193557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96fc101450 msgr2=0x7f96fc105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:03.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.070+0000 7f970404b640 1 --2- 172.21.15.172:0/1794193557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96fc101450 0x7f96fc105920 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009900 tx=0x7f96ec02a470 comp rx=0 tx=0).stop 2024-09-10T08:47:03.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.070+0000 7f970404b640 1 -- 172.21.15.172:0/1794193557 shutdown_connections 2024-09-10T08:47:03.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.070+0000 7f970404b640 1 --2- 172.21.15.172:0/1794193557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96fc105f40 0x7f96fc1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.070+0000 7f970404b640 1 --2- 172.21.15.172:0/1794193557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96fc101450 0x7f96fc105920 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.070+0000 7f970404b640 1 --2- 172.21.15.172:0/1794193557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96fc100b30 0x7f96fc100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.070+0000 7f970404b640 1 -- 172.21.15.172:0/1794193557 >> 172.21.15.172:0/1794193557 conn(0x7f96fc0fc7a0 msgr2=0x7f96fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:03.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.070+0000 7f970404b640 1 -- 172.21.15.172:0/1794193557 shutdown_connections 2024-09-10T08:47:03.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.070+0000 7f970404b640 1 -- 172.21.15.172:0/1794193557 wait complete. 2024-09-10T08:47:03.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f970404b640 1 Processor -- start 2024-09-10T08:47:03.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f970404b640 1 -- start start 2024-09-10T08:47:03.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f970404b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96fc100b30 0x7f96fc1a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f970404b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96fc101450 0x7f96fc1a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f970404b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96fc105f40 0x7f96fc19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f970404b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc1183f0 con 0x7f96fc105f40 2024-09-10T08:47:03.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f970404b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc118270 con 0x7f96fc100b30 2024-09-10T08:47:03.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f970404b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc118570 con 0x7f96fc101450 2024-09-10T08:47:03.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f97025e8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96fc105f40 0x7f96fc19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:03.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f97025e8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96fc105f40 0x7f96fc19e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55818/0 (socket says 172.21.15.172:55818) 2024-09-10T08:47:03.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f97025e8640 1 -- 172.21.15.172:0/297991723 learned_addr learned my addr 172.21.15.172:0/297991723 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:03.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f9701de7640 1 --2- 172.21.15.172:0/297991723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96fc100b30 0x7f96fc1a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:03.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f97015e6640 1 --2- 172.21.15.172:0/297991723 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96fc101450 0x7f96fc1a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:03.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f97025e8640 1 -- 172.21.15.172:0/297991723 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96fc101450 msgr2=0x7f96fc1a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:03.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f97025e8640 1 --2- 172.21.15.172:0/297991723 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96fc101450 0x7f96fc1a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f97025e8640 1 -- 172.21.15.172:0/297991723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96fc100b30 msgr2=0x7f96fc1a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:03.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f97025e8640 1 --2- 172.21.15.172:0/297991723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96fc100b30 0x7f96fc1a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.071+0000 7f97025e8640 1 -- 172.21.15.172:0/297991723 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96fc19efd0 con 0x7f96fc105f40 2024-09-10T08:47:03.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.072+0000 7f97025e8640 1 --2- 172.21.15.172:0/297991723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96fc105f40 0x7f96fc19e760 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f96f400cce0 tx=0x7f96f400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:03.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.072+0000 7f96f2ffd640 1 -- 172.21.15.172:0/297991723 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f400f450 con 0x7f96fc105f40 2024-09-10T08:47:03.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.072+0000 7f96f2ffd640 1 -- 172.21.15.172:0/297991723 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96f400faa0 con 0x7f96fc105f40 2024-09-10T08:47:03.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.072+0000 7f970404b640 1 -- 172.21.15.172:0/297991723 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96fc19f2c0 con 0x7f96fc105f40 2024-09-10T08:47:03.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.072+0000 7f96f2ffd640 1 -- 172.21.15.172:0/297991723 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4018b70 con 0x7f96fc105f40 2024-09-10T08:47:03.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.072+0000 7f970404b640 1 -- 172.21.15.172:0/297991723 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96fc1ab610 con 0x7f96fc105f40 2024-09-10T08:47:03.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.074+0000 7f96f2ffd640 1 -- 172.21.15.172:0/297991723 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f96f4018d10 con 0x7f96fc105f40 2024-09-10T08:47:03.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.075+0000 7f96f2ffd640 1 --2- 172.21.15.172:0/297991723 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f96e80a2c80 0x7f96e80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.075+0000 7f9706424740 1 Processor -- start 2024-09-10T08:47:03.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.075+0000 7f9701de7640 1 --2- 172.21.15.172:0/297991723 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f96e80a2c80 0x7f96e80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:03.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.075+0000 7f96f2ffd640 1 -- 172.21.15.172:0/297991723 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f96f40cf770 con 0x7f96fc105f40 2024-09-10T08:47:03.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.075+0000 7f9706424740 1 -- start start 2024-09-10T08:47:03.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.075+0000 7f9706424740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601744879d0 0x560174487db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.075+0000 7f9706424740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744882f0 0x560174492670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.075+0000 7f9706424740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560174492bb0 0x560174494fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.075+0000 7f9706424740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5601743ee090 con 0x5601744879d0 2024-09-10T08:47:03.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.075+0000 7f9706424740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560174395a70 con 0x5601744882f0 2024-09-10T08:47:03.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.075+0000 7f9706424740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5601743ea050 con 0x560174492bb0 2024-09-10T08:47:03.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.075+0000 7f9701de7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601744879d0 0x560174487db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:03.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.075+0000 7f9701de7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601744879d0 0x560174487db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55832/0 (socket says 172.21.15.172:55832) 2024-09-10T08:47:03.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.075+0000 7f9701de7640 1 -- 172.21.15.172:0/3209399914 learned_addr learned my addr 172.21.15.172:0/3209399914 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:03.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.075+0000 7f9701de7640 1 --2- 172.21.15.172:0/297991723 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f96e80a2c80 0x7f96e80a5140 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f96f800bbe0 tx=0x7f96f800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:03.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.075+0000 7f97015e6640 1 --2- 172.21.15.172:0/3209399914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744882f0 0x560174492670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:03.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.076+0000 7f9701de7640 1 -- 172.21.15.172:0/3209399914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560174492bb0 msgr2=0x560174494fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:03.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.076+0000 7f9701de7640 1 --2- 172.21.15.172:0/3209399914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560174492bb0 0x560174494fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.076+0000 7f9701de7640 1 -- 172.21.15.172:0/3209399914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744882f0 msgr2=0x560174492670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:03.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.076+0000 7f9701de7640 1 --2- 172.21.15.172:0/3209399914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744882f0 0x560174492670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.076+0000 7f9701de7640 1 -- 172.21.15.172:0/3209399914 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601743a2130 con 0x5601744879d0 2024-09-10T08:47:03.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.076+0000 7f97015e6640 1 --2- 172.21.15.172:0/3209399914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744882f0 0x560174492670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:03.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.076+0000 7f9701de7640 1 --2- 172.21.15.172:0/3209399914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601744879d0 0x560174487db0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f96f8010fe0 tx=0x7f96f8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe00554997edb94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:03.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.076+0000 7f96e7fff640 1 -- 172.21.15.172:0/3209399914 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f801e620 con 0x5601744879d0 2024-09-10T08:47:03.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.076+0000 7f96e7fff640 1 -- 172.21.15.172:0/3209399914 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96f801ec70 con 0x5601744879d0 2024-09-10T08:47:03.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.076+0000 7f96e7fff640 1 -- 172.21.15.172:0/3209399914 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f801dd00 con 0x5601744879d0 2024-09-10T08:47:03.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.077+0000 7f9706424740 1 -- 172.21.15.172:0/3209399914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601744879d0 msgr2=0x560174487db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:03.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.077+0000 7f9706424740 1 --2- 172.21.15.172:0/3209399914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601744879d0 0x560174487db0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f96f8010fe0 tx=0x7f96f8013000 comp rx=0 tx=0).stop 2024-09-10T08:47:03.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.077+0000 7f9706424740 1 -- 172.21.15.172:0/3209399914 shutdown_connections 2024-09-10T08:47:03.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.077+0000 7f9706424740 1 --2- 172.21.15.172:0/3209399914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560174492bb0 0x560174494fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.077+0000 7f9706424740 1 --2- 172.21.15.172:0/3209399914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744882f0 0x560174492670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.077+0000 7f9706424740 1 --2- 172.21.15.172:0/3209399914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601744879d0 0x560174487db0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.077+0000 7f9706424740 1 -- 172.21.15.172:0/3209399914 >> 172.21.15.172:0/3209399914 conn(0x560174483f60 msgr2=0x560174484340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:03.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.077+0000 7f9706424740 1 -- 172.21.15.172:0/3209399914 shutdown_connections 2024-09-10T08:47:03.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.077+0000 7f9706424740 1 -- 172.21.15.172:0/3209399914 wait complete. 2024-09-10T08:47:03.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.077+0000 7f9706424740 1 -- start start 2024-09-10T08:47:03.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.077+0000 7f9706424740 1 Processor -- start 2024-09-10T08:47:03.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f9706424740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601744879d0 0x560174492470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f9706424740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744882f0 0x56017448f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f9706424740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560174492bb0 0x56017448fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f9706424740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560174495780 con 0x560174492bb0 2024-09-10T08:47:03.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f9706424740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5601743a8f20 con 0x5601744882f0 2024-09-10T08:47:03.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f9706424740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560174495900 con 0x5601744879d0 2024-09-10T08:47:03.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f9706424740 10 client.0 _ll_register_callbacks cb 0x560174372030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:03.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f97025e8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560174492bb0 0x56017448fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:03.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f97025e8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560174492bb0 0x56017448fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55848/0 (socket says 172.21.15.172:55848) 2024-09-10T08:47:03.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f97025e8640 1 -- 172.21.15.172:0/3825623316 learned_addr learned my addr 172.21.15.172:0/3825623316 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:03.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f9701de7640 1 --2- 172.21.15.172:0/3825623316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601744879d0 0x560174492470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:03.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f97015e6640 1 --2- 172.21.15.172:0/3825623316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744882f0 0x56017448f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:03.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f97025e8640 1 -- 172.21.15.172:0/3825623316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601744879d0 msgr2=0x560174492470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:03.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f97025e8640 1 --2- 172.21.15.172:0/3825623316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601744879d0 0x560174492470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f97025e8640 1 -- 172.21.15.172:0/3825623316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744882f0 msgr2=0x56017448f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:03.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f97025e8640 1 --2- 172.21.15.172:0/3825623316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744882f0 0x56017448f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f97025e8640 1 -- 172.21.15.172:0/3825623316 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56017415f2b0 con 0x560174492bb0 2024-09-10T08:47:03.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f97025e8640 1 --2- 172.21.15.172:0/3825623316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560174492bb0 0x56017448fd00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f96f40985e0 tx=0x7f96f409b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543dca63f870109 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:03.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f96e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:03.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f96e57fa640 1 -- 172.21.15.172:0/3825623316 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f409c310 con 0x560174492bb0 2024-09-10T08:47:03.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.078+0000 7f96e57fa640 1 -- 172.21.15.172:0/3825623316 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f96f40cf470 con 0x560174492bb0 2024-09-10T08:47:03.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.079+0000 7f9706424740 1 -- 172.21.15.172:0/3825623316 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f801ec70 con 0x560174492bb0 2024-09-10T08:47:03.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.079+0000 7f96e57fa640 1 -- 172.21.15.172:0/3825623316 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96f409bc10 con 0x560174492bb0 2024-09-10T08:47:03.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.079+0000 7f9706424740 10 client.15531 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:03.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.079+0000 7f9706424740 1 -- 172.21.15.172:0/3825623316 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5601744a10a0 con 0x560174492bb0 2024-09-10T08:47:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.079+0000 7f96e57fa640 1 -- 172.21.15.172:0/3825623316 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96f801ec70 con 0x560174492bb0 2024-09-10T08:47:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.079+0000 7f96e57fa640 1 -- 172.21.15.172:0/3825623316 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f96f409a6f0 con 0x560174492bb0 2024-09-10T08:47:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.079+0000 7f9706424740 10 client.15531 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.079+0000 7f9706424740 10 client.15531 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:47:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.079+0000 7f9706424740 20 client.15531 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.079+0000 7f9706424740 1 --2- 172.21.15.172:0/3825623316 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x5601744a35c0 0x5601744a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.079+0000 7f9706424740 4 client.15531 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.079+0000 7f9706424740 1 -- 172.21.15.172:0/3825623316 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601743fb600 con 0x5601744a35c0 2024-09-10T08:47:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.079+0000 7f9701de7640 1 --2- 172.21.15.172:0/3825623316 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x5601744a35c0 0x5601744a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.080+0000 7f96e57fa640 10 client.15531 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:47:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.080+0000 7f9701de7640 1 --2- 172.21.15.172:0/3825623316 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x5601744a35c0 0x5601744a4a30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1ab3f23fcd26fd7 server_cookie=8326d61848ca7d02 in_seq=0 out_seq=0 2024-09-10T08:47:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.081+0000 7f96e57fa640 1 -- 172.21.15.172:0/3825623316 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5601743fb600 con 0x5601744a35c0 2024-09-10T08:47:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.081+0000 7f96e57fa640 10 client.15531 handle_command_reply: tid=0 2024-09-10T08:47:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.081+0000 7f9706424740 1 client.15531 shutdown 2024-09-10T08:47:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.081+0000 7f9706424740 1 -- 172.21.15.172:0/3825623316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560174492bb0 msgr2=0x56017448fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.081+0000 7f9706424740 1 --2- 172.21.15.172:0/3825623316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560174492bb0 0x56017448fd00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f96f40985e0 tx=0x7f96f409b630 comp rx=0 tx=0).stop 2024-09-10T08:47:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.081+0000 7f9706424740 1 -- 172.21.15.172:0/3825623316 shutdown_connections 2024-09-10T08:47:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.081+0000 7f9706424740 1 --2- 172.21.15.172:0/3825623316 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x5601744a35c0 0x5601744a4a30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.081+0000 7f9706424740 1 --2- 172.21.15.172:0/3825623316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560174492bb0 0x56017448fd00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.081+0000 7f9706424740 1 --2- 172.21.15.172:0/3825623316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744882f0 0x56017448f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.081+0000 7f96e57fa640 0 client.15531 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:47:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.081+0000 7f9706424740 1 --2- 172.21.15.172:0/3825623316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601744879d0 0x560174492470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.081+0000 7f9706424740 1 -- 172.21.15.172:0/3825623316 >> 172.21.15.172:0/3825623316 conn(0x560174483f60 msgr2=0x560174489c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.081+0000 7f9706424740 1 -- 172.21.15.172:0/3825623316 shutdown_connections 2024-09-10T08:47:03.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.081+0000 7f9706424740 1 -- 172.21.15.172:0/3825623316 wait complete. 2024-09-10T08:47:03.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.081+0000 7f9706424740 20 client.15531 trim_cache size 0 max 16384 2024-09-10T08:47:03.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.098+0000 7f9706424740 1 Processor -- start 2024-09-10T08:47:03.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.098+0000 7f9706424740 1 -- start start 2024-09-10T08:47:03.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.098+0000 7f9706424740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601744879d0 0x56017448ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.098+0000 7f9706424740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744882f0 0x56017448ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.098+0000 7f9706424740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560174492bb0 0x56017448cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.098+0000 7f9706424740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5601744a2d70 con 0x560174492bb0 2024-09-10T08:47:03.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.098+0000 7f9706424740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5601744a3440 con 0x5601744882f0 2024-09-10T08:47:03.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.098+0000 7f9706424740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5601744a3b10 con 0x5601744879d0 2024-09-10T08:47:03.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.099+0000 7f97025e8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560174492bb0 0x56017448cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:03.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.099+0000 7f97025e8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560174492bb0 0x56017448cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55852/0 (socket says 172.21.15.172:55852) 2024-09-10T08:47:03.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.099+0000 7f97025e8640 1 -- 172.21.15.172:0/511463689 learned_addr learned my addr 172.21.15.172:0/511463689 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:03.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.099+0000 7f9701de7640 1 --2- 172.21.15.172:0/511463689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601744879d0 0x56017448ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:03.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.099+0000 7f97025e8640 1 -- 172.21.15.172:0/511463689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601744879d0 msgr2=0x56017448ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:03.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.099+0000 7f97015e6640 1 --2- 172.21.15.172:0/511463689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744882f0 0x56017448ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:03.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.099+0000 7f97025e8640 1 --2- 172.21.15.172:0/511463689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601744879d0 0x56017448ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.099+0000 7f97025e8640 1 -- 172.21.15.172:0/511463689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744882f0 msgr2=0x56017448ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:03.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.099+0000 7f97025e8640 1 --2- 172.21.15.172:0/511463689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744882f0 0x56017448ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.099+0000 7f97025e8640 1 -- 172.21.15.172:0/511463689 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601744927e0 con 0x560174492bb0 2024-09-10T08:47:03.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.099+0000 7f97025e8640 1 --2- 172.21.15.172:0/511463689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560174492bb0 0x56017448cf80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f96f400ab20 tx=0x7f96f4018510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f84b7ff06bb2a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:03.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.099+0000 7f96e77fe640 1 -- 172.21.15.172:0/511463689 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f409ffc0 con 0x560174492bb0 2024-09-10T08:47:03.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.099+0000 7f96e77fe640 1 -- 172.21.15.172:0/511463689 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96f40a4880 con 0x560174492bb0 2024-09-10T08:47:03.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.099+0000 7f96e77fe640 1 -- 172.21.15.172:0/511463689 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f409a6a0 con 0x560174492bb0 2024-09-10T08:47:03.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.100+0000 7f9706424740 1 -- 172.21.15.172:0/511463689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560174492bb0 msgr2=0x56017448cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:03.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.100+0000 7f9706424740 1 --2- 172.21.15.172:0/511463689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560174492bb0 0x56017448cf80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f96f400ab20 tx=0x7f96f4018510 comp rx=0 tx=0).stop 2024-09-10T08:47:03.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.100+0000 7f9706424740 1 -- 172.21.15.172:0/511463689 shutdown_connections 2024-09-10T08:47:03.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.100+0000 7f9706424740 1 --2- 172.21.15.172:0/511463689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560174492bb0 0x56017448cf80 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.100+0000 7f9706424740 1 --2- 172.21.15.172:0/511463689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744882f0 0x56017448ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.100+0000 7f9706424740 1 --2- 172.21.15.172:0/511463689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601744879d0 0x56017448ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.100+0000 7f9706424740 1 -- 172.21.15.172:0/511463689 >> 172.21.15.172:0/511463689 conn(0x560174483f60 msgr2=0x560174484340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:03.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.100+0000 7f9706424740 1 -- 172.21.15.172:0/511463689 shutdown_connections 2024-09-10T08:47:03.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.100+0000 7f9706424740 1 -- 172.21.15.172:0/511463689 wait complete. 2024-09-10T08:47:03.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.100+0000 7f9706424740 1 -- start start 2024-09-10T08:47:03.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.100+0000 7f9706424740 1 Processor -- start 2024-09-10T08:47:03.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f9706424740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744879d0 0x5601744a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f9706424740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601744882f0 0x5601744a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f9706424740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560174492bb0 0x5601744aa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f9706424740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5601744a48b0 con 0x5601744882f0 2024-09-10T08:47:03.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f9706424740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5601744a41e0 con 0x5601744879d0 2024-09-10T08:47:03.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f9706424740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5601744a4f80 con 0x560174492bb0 2024-09-10T08:47:03.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f9706424740 10 client.0 _ll_register_callbacks cb 0x7f96f40185b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:03.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f97015e6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601744882f0 0x5601744a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:03.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f97015e6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601744882f0 0x5601744a9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55868/0 (socket says 172.21.15.172:55868) 2024-09-10T08:47:03.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f97015e6640 1 -- 172.21.15.172:0/3688211624 learned_addr learned my addr 172.21.15.172:0/3688211624 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:03.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f9701de7640 1 --2- 172.21.15.172:0/3688211624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744879d0 0x5601744a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:03.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f97025e8640 1 --2- 172.21.15.172:0/3688211624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560174492bb0 0x5601744aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:03.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f97015e6640 1 -- 172.21.15.172:0/3688211624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560174492bb0 msgr2=0x5601744aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:03.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f97015e6640 1 --2- 172.21.15.172:0/3688211624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560174492bb0 0x5601744aa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f97015e6640 1 -- 172.21.15.172:0/3688211624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744879d0 msgr2=0x5601744a99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:03.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f97015e6640 1 --2- 172.21.15.172:0/3688211624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744879d0 0x5601744a99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f97015e6640 1 -- 172.21.15.172:0/3688211624 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601744aaec0 con 0x5601744882f0 2024-09-10T08:47:03.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f9701de7640 1 --2- 172.21.15.172:0/3688211624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744879d0 0x5601744a99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:03.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f97015e6640 1 --2- 172.21.15.172:0/3688211624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601744882f0 0x5601744a9ee0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f96ec005ab0 tx=0x7f96ec031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834f79d4a0534729 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:03.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f96e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:03.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f96e57fa640 1 -- 172.21.15.172:0/3688211624 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec035040 con 0x5601744882f0 2024-09-10T08:47:03.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f96e57fa640 1 -- 172.21.15.172:0/3688211624 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f96ec004240 con 0x5601744882f0 2024-09-10T08:47:03.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.101+0000 7f9706424740 1 -- 172.21.15.172:0/3688211624 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f40a4880 con 0x5601744882f0 2024-09-10T08:47:03.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.102+0000 7f96e57fa640 1 -- 172.21.15.172:0/3688211624 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96ec0318b0 con 0x5601744882f0 2024-09-10T08:47:03.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.102+0000 7f9706424740 10 client.15537 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:03.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.102+0000 7f9706424740 1 -- 172.21.15.172:0/3688211624 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5601744ac250 con 0x5601744882f0 2024-09-10T08:47:03.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.102+0000 7f96e57fa640 1 -- 172.21.15.172:0/3688211624 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96f40a4880 con 0x5601744882f0 2024-09-10T08:47:03.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.102+0000 7f96e57fa640 1 -- 172.21.15.172:0/3688211624 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f96ec004b10 con 0x5601744882f0 2024-09-10T08:47:03.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.102+0000 7f9706424740 10 client.15537 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:03.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.102+0000 7f9706424740 10 client.15537 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T08:47:03.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.102+0000 7f9706424740 20 client.15537 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:03.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.102+0000 7f9706424740 1 --2- 172.21.15.172:0/3688211624 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560174512140 0x560174510e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:03.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.102+0000 7f9706424740 4 client.15537 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:47:03.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.102+0000 7f9706424740 1 -- 172.21.15.172:0/3688211624 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "status"}) -- 0x5601743f5b10 con 0x560174512140 2024-09-10T08:47:03.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.102+0000 7f9701de7640 1 --2- 172.21.15.172:0/3688211624 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560174512140 0x560174510e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:03.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.103+0000 7f96e57fa640 10 client.15537 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:47:03.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.103+0000 7f9701de7640 1 --2- 172.21.15.172:0/3688211624 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560174512140 0x560174510e50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4860a57c60ccbcd server_cookie=487ca854be2dfbe in_seq=0 out_seq=0 2024-09-10T08:47:03.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.103+0000 7f96e57fa640 1 -- 172.21.15.172:0/3688211624 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5601743f5b10 con 0x560174512140 2024-09-10T08:47:03.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.103+0000 7f96e57fa640 10 client.15537 handle_command_reply: tid=0 2024-09-10T08:47:03.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.103+0000 7f9706424740 1 client.15537 shutdown 2024-09-10T08:47:03.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.103+0000 7f9706424740 1 -- 172.21.15.172:0/3688211624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601744882f0 msgr2=0x5601744a9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:03.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.103+0000 7f9706424740 1 --2- 172.21.15.172:0/3688211624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601744882f0 0x5601744a9ee0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f96ec005ab0 tx=0x7f96ec031740 comp rx=0 tx=0).stop 2024-09-10T08:47:03.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.103+0000 7f9706424740 1 -- 172.21.15.172:0/3688211624 shutdown_connections 2024-09-10T08:47:03.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.103+0000 7f9706424740 1 --2- 172.21.15.172:0/3688211624 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560174512140 0x560174510e50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.103+0000 7f9706424740 1 --2- 172.21.15.172:0/3688211624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560174492bb0 0x5601744aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.103+0000 7f9706424740 1 --2- 172.21.15.172:0/3688211624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601744882f0 0x5601744a9ee0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.103+0000 7f96e57fa640 0 client.15537 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:47:03.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.103+0000 7f9706424740 1 --2- 172.21.15.172:0/3688211624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601744879d0 0x5601744a99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.103+0000 7f9706424740 1 -- 172.21.15.172:0/3688211624 >> 172.21.15.172:0/3688211624 conn(0x560174483f60 msgr2=0x5601744827e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:03.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.103+0000 7f9706424740 1 -- 172.21.15.172:0/3688211624 shutdown_connections 2024-09-10T08:47:03.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.103+0000 7f9706424740 1 -- 172.21.15.172:0/3688211624 wait complete. 2024-09-10T08:47:03.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.103+0000 7f9706424740 20 client.15537 trim_cache size 0 max 16384 2024-09-10T08:47:03.113 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:47:03.113 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:47:03.113 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 2, 2024-09-10T08:47:03.113 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 24457, 2024-09-10T08:47:03.113 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-09-10T08:47:03.114 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-09-10T08:47:03.114 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-09-10T08:47:03.114 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 130.84662960899999, 2024-09-10T08:47:03.114 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 29, 2024-09-10T08:47:03.114 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 92, 2024-09-10T08:47:03.114 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 92, 2024-09-10T08:47:03.114 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 190.459353354 2024-09-10T08:47:03.114 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:47:03.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.104+0000 7f970404b640 1 -- 172.21.15.172:0/297991723 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f96e80a2c80 msgr2=0x7f96e80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:03.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.104+0000 7f970404b640 1 --2- 172.21.15.172:0/297991723 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f96e80a2c80 0x7f96e80a5140 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f96f800bbe0 tx=0x7f96f800b390 comp rx=0 tx=0).stop 2024-09-10T08:47:03.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.104+0000 7f970404b640 1 -- 172.21.15.172:0/297991723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96fc105f40 msgr2=0x7f96fc19e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:03.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.104+0000 7f970404b640 1 --2- 172.21.15.172:0/297991723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96fc105f40 0x7f96fc19e760 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f96f400cce0 tx=0x7f96f400a620 comp rx=0 tx=0).stop 2024-09-10T08:47:03.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.104+0000 7f970404b640 1 -- 172.21.15.172:0/297991723 shutdown_connections 2024-09-10T08:47:03.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.104+0000 7f970404b640 1 --2- 172.21.15.172:0/297991723 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f96e80a2c80 0x7f96e80a5140 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.104+0000 7f970404b640 1 --2- 172.21.15.172:0/297991723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96fc105f40 0x7f96fc19e760 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.104+0000 7f970404b640 1 --2- 172.21.15.172:0/297991723 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96fc101450 0x7f96fc1a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.104+0000 7f970404b640 1 --2- 172.21.15.172:0/297991723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96fc100b30 0x7f96fc1a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:03.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.104+0000 7f970404b640 1 -- 172.21.15.172:0/297991723 >> 172.21.15.172:0/297991723 conn(0x7f96fc0fc7a0 msgr2=0x7f96fc0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:03.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.104+0000 7f970404b640 1 -- 172.21.15.172:0/297991723 shutdown_connections 2024-09-10T08:47:03.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:03.104+0000 7f970404b640 1 -- 172.21.15.172:0/297991723 wait complete. 2024-09-10T08:47:03.816 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T08:47:04.065 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:04.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:04 smithi174 ceph-mon[34068]: pgmap v375: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-09-10T08:47:04.508 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:04 smithi172 ceph-mon[30854]: pgmap v375: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-09-10T08:47:04.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:04 smithi191 ceph-mon[35314]: pgmap v375: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-09-10T08:47:04.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.588+0000 7f4818534640 1 -- 172.21.15.172:0/2379157965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4810102da0 msgr2=0x7f4810103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:04.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.588+0000 7f4818534640 1 --2- 172.21.15.172:0/2379157965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4810102da0 0x7f4810103180 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f480c009900 tx=0x7f480c02a480 comp rx=0 tx=0).stop 2024-09-10T08:47:04.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.588+0000 7f4818534640 1 -- 172.21.15.172:0/2379157965 shutdown_connections 2024-09-10T08:47:04.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.588+0000 7f4818534640 1 --2- 172.21.15.172:0/2379157965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f481010c180 0x7f481010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.588+0000 7f4818534640 1 --2- 172.21.15.172:0/2379157965 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48101036c0 0x7f481010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.588+0000 7f4818534640 1 --2- 172.21.15.172:0/2379157965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4810102da0 0x7f4810103180 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.588+0000 7f4818534640 1 -- 172.21.15.172:0/2379157965 >> 172.21.15.172:0/2379157965 conn(0x7f48100fc7a0 msgr2=0x7f48100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:04.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.588+0000 7f4818534640 1 -- 172.21.15.172:0/2379157965 shutdown_connections 2024-09-10T08:47:04.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.588+0000 7f4818534640 1 -- 172.21.15.172:0/2379157965 wait complete. 2024-09-10T08:47:04.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.588+0000 7f4818534640 1 Processor -- start 2024-09-10T08:47:04.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f4818534640 1 -- start start 2024-09-10T08:47:04.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f4818534640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4810102da0 0x7f48101983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f4818534640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48101036c0 0x7f48101988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f4818534640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f481010c180 0x7f481019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f4818534640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4810111e50 con 0x7f4810102da0 2024-09-10T08:47:04.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f4818534640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4810111cd0 con 0x7f481010c180 2024-09-10T08:47:04.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f4818534640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4810111fd0 con 0x7f48101036c0 2024-09-10T08:47:04.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f48162d0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4810102da0 0x7f48101983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:04.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f48162d0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4810102da0 0x7f48101983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55892/0 (socket says 172.21.15.172:55892) 2024-09-10T08:47:04.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f48162d0640 1 -- 172.21.15.172:0/3022513079 learned_addr learned my addr 172.21.15.172:0/3022513079 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:04.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f4816ad1640 1 --2- 172.21.15.172:0/3022513079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f481010c180 0x7f481019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:04.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f48162d0640 1 -- 172.21.15.172:0/3022513079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48101036c0 msgr2=0x7f48101988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:04.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f48162d0640 1 --2- 172.21.15.172:0/3022513079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48101036c0 0x7f48101988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f48162d0640 1 -- 172.21.15.172:0/3022513079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f481010c180 msgr2=0x7f481019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:04.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f48162d0640 1 --2- 172.21.15.172:0/3022513079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f481010c180 0x7f481019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f48162d0640 1 -- 172.21.15.172:0/3022513079 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481019d360 con 0x7f4810102da0 2024-09-10T08:47:04.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f48162d0640 1 --2- 172.21.15.172:0/3022513079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4810102da0 0x7f48101983b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f480c005940 tx=0x7f480c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:04.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f48077fe640 1 -- 172.21.15.172:0/3022513079 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c03f030 con 0x7f4810102da0 2024-09-10T08:47:04.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f4818534640 1 -- 172.21.15.172:0/3022513079 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481019d5f0 con 0x7f4810102da0 2024-09-10T08:47:04.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f48077fe640 1 -- 172.21.15.172:0/3022513079 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f480c003aa0 con 0x7f4810102da0 2024-09-10T08:47:04.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.589+0000 7f48077fe640 1 -- 172.21.15.172:0/3022513079 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c035070 con 0x7f4810102da0 2024-09-10T08:47:04.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.590+0000 7f4818534640 1 -- 172.21.15.172:0/3022513079 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48101a4f30 con 0x7f4810102da0 2024-09-10T08:47:04.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.592+0000 7f48077fe640 1 -- 172.21.15.172:0/3022513079 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f480c042030 con 0x7f4810102da0 2024-09-10T08:47:04.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.592+0000 7f481a90d740 1 Processor -- start 2024-09-10T08:47:04.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.592+0000 7f481a90d740 1 -- start start 2024-09-10T08:47:04.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.592+0000 7f481a90d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fb4890 0x555858fb4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.592+0000 7f481a90d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb51b0 0x555858fc3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.592+0000 7f481a90d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fc3fc0 0x555858fc63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.592+0000 7f481a90d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555858f1b090 con 0x555858fb4890 2024-09-10T08:47:04.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.592+0000 7f481a90d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555858ec2a70 con 0x555858fc3fc0 2024-09-10T08:47:04.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.592+0000 7f481a90d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555858f17050 con 0x555858fb51b0 2024-09-10T08:47:04.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.592+0000 7f4816ad1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fb4890 0x555858fb4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:04.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.593+0000 7f4816ad1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fb4890 0x555858fb4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55906/0 (socket says 172.21.15.172:55906) 2024-09-10T08:47:04.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.593+0000 7f4816ad1640 1 -- 172.21.15.172:0/416499688 learned_addr learned my addr 172.21.15.172:0/416499688 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:04.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.593+0000 7f48162d0640 1 --2- 172.21.15.172:0/416499688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb51b0 0x555858fc3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:04.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.593+0000 7f4815acf640 1 --2- 172.21.15.172:0/416499688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fc3fc0 0x555858fc63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:04.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.593+0000 7f48077fe640 1 --2- 172.21.15.172:0/3022513079 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f47fc0a2b80 0x7f47fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.593+0000 7f4816ad1640 1 -- 172.21.15.172:0/416499688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb51b0 msgr2=0x555858fc3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:04.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.593+0000 7f4816ad1640 1 --2- 172.21.15.172:0/416499688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb51b0 0x555858fc3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.593+0000 7f4816ad1640 1 -- 172.21.15.172:0/416499688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fc3fc0 msgr2=0x555858fc63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:04.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.593+0000 7f4816ad1640 1 --2- 172.21.15.172:0/416499688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fc3fc0 0x555858fc63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.593+0000 7f4816ad1640 1 -- 172.21.15.172:0/416499688 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555858ecf130 con 0x555858fb4890 2024-09-10T08:47:04.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.593+0000 7f48077fe640 1 -- 172.21.15.172:0/3022513079 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f480c0efc00 con 0x7f4810102da0 2024-09-10T08:47:04.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.593+0000 7f4816ad1640 1 --2- 172.21.15.172:0/3022513079 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f47fc0a2b80 0x7f47fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:04.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.593+0000 7f4816ad1640 1 --2- 172.21.15.172:0/416499688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fb4890 0x555858fb4c70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f48080103f0 tx=0x7f48080108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e52b19974cb4067 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:04.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.593+0000 7f4804ff9640 1 -- 172.21.15.172:0/416499688 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480801c900 con 0x555858fb4890 2024-09-10T08:47:04.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.593+0000 7f4804ff9640 1 -- 172.21.15.172:0/416499688 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f480801b3d0 con 0x555858fb4890 2024-09-10T08:47:04.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.593+0000 7f4804ff9640 1 -- 172.21.15.172:0/416499688 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48080255e0 con 0x555858fb4890 2024-09-10T08:47:04.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.594+0000 7f481a90d740 1 -- 172.21.15.172:0/416499688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fb4890 msgr2=0x555858fb4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:04.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.594+0000 7f4816ad1640 1 --2- 172.21.15.172:0/3022513079 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f47fc0a2b80 0x7f47fc0a5040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4808020000 tx=0x7f480801d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:04.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.594+0000 7f481a90d740 1 --2- 172.21.15.172:0/416499688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fb4890 0x555858fb4c70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f48080103f0 tx=0x7f48080108b0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.594+0000 7f481a90d740 1 -- 172.21.15.172:0/416499688 shutdown_connections 2024-09-10T08:47:04.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.594+0000 7f481a90d740 1 --2- 172.21.15.172:0/416499688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fc3fc0 0x555858fc63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.594+0000 7f481a90d740 1 --2- 172.21.15.172:0/416499688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb51b0 0x555858fc3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.594+0000 7f481a90d740 1 --2- 172.21.15.172:0/416499688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fb4890 0x555858fb4c70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.594+0000 7f481a90d740 1 -- 172.21.15.172:0/416499688 >> 172.21.15.172:0/416499688 conn(0x555858fb0f60 msgr2=0x555858fb1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:04.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.594+0000 7f481a90d740 1 -- 172.21.15.172:0/416499688 shutdown_connections 2024-09-10T08:47:04.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.594+0000 7f481a90d740 1 -- 172.21.15.172:0/416499688 wait complete. 2024-09-10T08:47:04.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.595+0000 7f481a90d740 1 -- start start 2024-09-10T08:47:04.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.595+0000 7f481a90d740 1 Processor -- start 2024-09-10T08:47:04.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.595+0000 7f481a90d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fb4890 0x555858fc3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.595+0000 7f481a90d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb51b0 0x555858fc0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.595+0000 7f481a90d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fc3fc0 0x555858fc10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.595+0000 7f481a90d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555858fc6b90 con 0x555858fc3fc0 2024-09-10T08:47:04.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.595+0000 7f481a90d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555858ed5f20 con 0x555858fb4890 2024-09-10T08:47:04.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.595+0000 7f481a90d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555858fc6d10 con 0x555858fb51b0 2024-09-10T08:47:04.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.595+0000 7f481a90d740 10 client.0 _ll_register_callbacks cb 0x555858e9f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:04.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.595+0000 7f4816ad1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fc3fc0 0x555858fc10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:04.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.595+0000 7f4816ad1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fc3fc0 0x555858fc10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55918/0 (socket says 172.21.15.172:55918) 2024-09-10T08:47:04.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.595+0000 7f4816ad1640 1 -- 172.21.15.172:0/2647452901 learned_addr learned my addr 172.21.15.172:0/2647452901 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:04.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.595+0000 7f48162d0640 1 --2- 172.21.15.172:0/2647452901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fb4890 0x555858fc3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:04.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.595+0000 7f4815acf640 1 --2- 172.21.15.172:0/2647452901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb51b0 0x555858fc0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:04.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.596+0000 7f4816ad1640 1 -- 172.21.15.172:0/2647452901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb51b0 msgr2=0x555858fc0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:04.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.596+0000 7f4816ad1640 1 --2- 172.21.15.172:0/2647452901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb51b0 0x555858fc0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.596+0000 7f4816ad1640 1 -- 172.21.15.172:0/2647452901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fb4890 msgr2=0x555858fc3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:04.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.596+0000 7f4816ad1640 1 --2- 172.21.15.172:0/2647452901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fb4890 0x555858fc3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.596+0000 7f4816ad1640 1 -- 172.21.15.172:0/2647452901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555858c8c2b0 con 0x555858fc3fc0 2024-09-10T08:47:04.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.596+0000 7f48162d0640 1 --2- 172.21.15.172:0/2647452901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fb4890 0x555858fc3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:04.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.596+0000 7f4816ad1640 1 --2- 172.21.15.172:0/2647452901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fc3fc0 0x555858fc10f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f480801ba50 tx=0x7f480801b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b22aa17f535726 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:04.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.596+0000 7f47f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:04.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.596+0000 7f47f9ffb640 1 -- 172.21.15.172:0/2647452901 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808009b90 con 0x555858fc3fc0 2024-09-10T08:47:04.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.596+0000 7f481a90d740 1 -- 172.21.15.172:0/2647452901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f480801b3d0 con 0x555858fc3fc0 2024-09-10T08:47:04.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.596+0000 7f47f9ffb640 1 -- 172.21.15.172:0/2647452901 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f480801ca90 con 0x555858fc3fc0 2024-09-10T08:47:04.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.596+0000 7f47f9ffb640 1 -- 172.21.15.172:0/2647452901 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f480803c050 con 0x555858fc3fc0 2024-09-10T08:47:04.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.596+0000 7f481a90d740 10 client.15549 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:04.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.596+0000 7f481a90d740 1 -- 172.21.15.172:0/2647452901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555858fd24e0 con 0x555858fc3fc0 2024-09-10T08:47:04.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.596+0000 7f47f9ffb640 1 -- 172.21.15.172:0/2647452901 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f480801b3d0 con 0x555858fc3fc0 2024-09-10T08:47:04.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.596+0000 7f47f9ffb640 1 -- 172.21.15.172:0/2647452901 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4808037030 con 0x555858fc3fc0 2024-09-10T08:47:04.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.597+0000 7f481a90d740 10 client.15549 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:04.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.597+0000 7f481a90d740 10 client.15549 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:47:04.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.597+0000 7f481a90d740 20 client.15549 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:04.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.597+0000 7f481a90d740 1 --2- 172.21.15.172:0/2647452901 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x555858fd49c0 0x555858fd5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.597+0000 7f481a90d740 4 client.15549 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:04.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.597+0000 7f481a90d740 1 -- 172.21.15.172:0/2647452901 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555858f28600 con 0x555858fd49c0 2024-09-10T08:47:04.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.597+0000 7f48162d0640 1 --2- 172.21.15.172:0/2647452901 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x555858fd49c0 0x555858fd5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:04.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.597+0000 7f47f9ffb640 10 client.15549 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:47:04.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.597+0000 7f48162d0640 1 --2- 172.21.15.172:0/2647452901 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x555858fd49c0 0x555858fd5e30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae582e7985485ba5 server_cookie=57f0f27c37bd2e4b in_seq=0 out_seq=0 2024-09-10T08:47:04.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.598+0000 7f47f9ffb640 1 -- 172.21.15.172:0/2647452901 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555858f28600 con 0x555858fd49c0 2024-09-10T08:47:04.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.598+0000 7f47f9ffb640 10 client.15549 handle_command_reply: tid=0 2024-09-10T08:47:04.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.598+0000 7f481a90d740 1 client.15549 shutdown 2024-09-10T08:47:04.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.598+0000 7f481a90d740 1 -- 172.21.15.172:0/2647452901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fc3fc0 msgr2=0x555858fc10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:04.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.598+0000 7f481a90d740 1 --2- 172.21.15.172:0/2647452901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fc3fc0 0x555858fc10f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f480801ba50 tx=0x7f480801b830 comp rx=0 tx=0).stop 2024-09-10T08:47:04.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.599+0000 7f481a90d740 1 -- 172.21.15.172:0/2647452901 shutdown_connections 2024-09-10T08:47:04.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.599+0000 7f481a90d740 1 --2- 172.21.15.172:0/2647452901 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x555858fd49c0 0x555858fd5e30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.599+0000 7f481a90d740 1 --2- 172.21.15.172:0/2647452901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fc3fc0 0x555858fc10f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.599+0000 7f481a90d740 1 --2- 172.21.15.172:0/2647452901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb51b0 0x555858fc0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.599+0000 7f481a90d740 1 --2- 172.21.15.172:0/2647452901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fb4890 0x555858fc3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.599+0000 7f481a90d740 1 -- 172.21.15.172:0/2647452901 >> 172.21.15.172:0/2647452901 conn(0x555858fb0f60 msgr2=0x555858fb6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:04.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.599+0000 7f47f9ffb640 0 client.15549 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:47:04.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.599+0000 7f481a90d740 1 -- 172.21.15.172:0/2647452901 shutdown_connections 2024-09-10T08:47:04.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.599+0000 7f481a90d740 1 -- 172.21.15.172:0/2647452901 wait complete. 2024-09-10T08:47:04.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.599+0000 7f481a90d740 20 client.15549 trim_cache size 0 max 16384 2024-09-10T08:47:04.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.616+0000 7f481a90d740 1 Processor -- start 2024-09-10T08:47:04.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.616+0000 7f481a90d740 1 -- start start 2024-09-10T08:47:04.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.616+0000 7f481a90d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fb4890 0x555858fd42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.616+0000 7f481a90d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb51b0 0x555858fd5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.616+0000 7f481a90d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fc3fc0 0x555858fbde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.616+0000 7f481a90d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555858fd4170 con 0x555858fc3fc0 2024-09-10T08:47:04.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.616+0000 7f481a90d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555858fd4840 con 0x555858fb4890 2024-09-10T08:47:04.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.616+0000 7f481a90d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555858fd4f10 con 0x555858fb51b0 2024-09-10T08:47:04.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.617+0000 7f4816ad1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fc3fc0 0x555858fbde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:04.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.617+0000 7f4816ad1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fc3fc0 0x555858fbde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55924/0 (socket says 172.21.15.172:55924) 2024-09-10T08:47:04.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.617+0000 7f4816ad1640 1 -- 172.21.15.172:0/4176043662 learned_addr learned my addr 172.21.15.172:0/4176043662 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:04.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.617+0000 7f48162d0640 1 --2- 172.21.15.172:0/4176043662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fb4890 0x555858fd42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:04.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.617+0000 7f4816ad1640 1 -- 172.21.15.172:0/4176043662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb51b0 msgr2=0x555858fd5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:04.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.617+0000 7f4816ad1640 1 --2- 172.21.15.172:0/4176043662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb51b0 0x555858fd5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.617+0000 7f4816ad1640 1 -- 172.21.15.172:0/4176043662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fb4890 msgr2=0x555858fd42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:04.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.617+0000 7f4816ad1640 1 --2- 172.21.15.172:0/4176043662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fb4890 0x555858fd42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.617+0000 7f4816ad1640 1 -- 172.21.15.172:0/4176043662 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555858fbf100 con 0x555858fc3fc0 2024-09-10T08:47:04.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.617+0000 7f48162d0640 1 --2- 172.21.15.172:0/4176043662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fb4890 0x555858fd42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:04.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.617+0000 7f4816ad1640 1 --2- 172.21.15.172:0/4176043662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fc3fc0 0x555858fbde10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f480801cef0 tx=0x7f4808009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0e8be28bcc17ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:04.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.617+0000 7f47fbfff640 1 -- 172.21.15.172:0/4176043662 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480801b8a0 con 0x555858fc3fc0 2024-09-10T08:47:04.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.617+0000 7f47fbfff640 1 -- 172.21.15.172:0/4176043662 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f480802a050 con 0x555858fc3fc0 2024-09-10T08:47:04.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.617+0000 7f47fbfff640 1 -- 172.21.15.172:0/4176043662 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480801bb70 con 0x555858fc3fc0 2024-09-10T08:47:04.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 1 -- 172.21.15.172:0/4176043662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fc3fc0 msgr2=0x555858fbde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:04.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 1 --2- 172.21.15.172:0/4176043662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fc3fc0 0x555858fbde10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f480801cef0 tx=0x7f4808009650 comp rx=0 tx=0).stop 2024-09-10T08:47:04.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 1 -- 172.21.15.172:0/4176043662 shutdown_connections 2024-09-10T08:47:04.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 1 --2- 172.21.15.172:0/4176043662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fc3fc0 0x555858fbde10 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 1 --2- 172.21.15.172:0/4176043662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb51b0 0x555858fd5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 1 --2- 172.21.15.172:0/4176043662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fb4890 0x555858fd42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 1 -- 172.21.15.172:0/4176043662 >> 172.21.15.172:0/4176043662 conn(0x555858fb0f60 msgr2=0x555858fabce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:04.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 1 -- 172.21.15.172:0/4176043662 shutdown_connections 2024-09-10T08:47:04.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 1 -- 172.21.15.172:0/4176043662 wait complete. 2024-09-10T08:47:04.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 1 -- start start 2024-09-10T08:47:04.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 1 Processor -- start 2024-09-10T08:47:04.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb4890 0x555858fd70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fb51b0 0x555858fd7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fc3fc0 0x555858fd7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555858fd5cb0 con 0x555858fb51b0 2024-09-10T08:47:04.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555858fd55e0 con 0x555858fc3fc0 2024-09-10T08:47:04.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555858fd6380 con 0x555858fb4890 2024-09-10T08:47:04.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.618+0000 7f481a90d740 10 client.0 _ll_register_callbacks cb 0x7f480801f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:04.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f4815acf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fb51b0 0x555858fd7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:04.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f4815acf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fb51b0 0x555858fd7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55928/0 (socket says 172.21.15.172:55928) 2024-09-10T08:47:04.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f4815acf640 1 -- 172.21.15.172:0/3514037632 learned_addr learned my addr 172.21.15.172:0/3514037632 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:04.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f48162d0640 1 --2- 172.21.15.172:0/3514037632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb4890 0x555858fd70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:04.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f4816ad1640 1 --2- 172.21.15.172:0/3514037632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fc3fc0 0x555858fd7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:04.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f4815acf640 1 -- 172.21.15.172:0/3514037632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb4890 msgr2=0x555858fd70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:04.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f4815acf640 1 --2- 172.21.15.172:0/3514037632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb4890 0x555858fd70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f4815acf640 1 -- 172.21.15.172:0/3514037632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fc3fc0 msgr2=0x555858fd7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:04.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f4815acf640 1 --2- 172.21.15.172:0/3514037632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fc3fc0 0x555858fd7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f4815acf640 1 -- 172.21.15.172:0/3514037632 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55585903d390 con 0x555858fb51b0 2024-09-10T08:47:04.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f4816ad1640 1 --2- 172.21.15.172:0/3514037632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fc3fc0 0x555858fd7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:04.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f4815acf640 1 --2- 172.21.15.172:0/3514037632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fb51b0 0x555858fd7610 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x555858fb2050 tx=0x7f48000105c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a150ca76fc5645 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:04.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f47f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:04.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f47f9ffb640 1 -- 172.21.15.172:0/3514037632 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800018070 con 0x555858fb51b0 2024-09-10T08:47:04.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f47f9ffb640 1 -- 172.21.15.172:0/3514037632 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f48000143f0 con 0x555858fb51b0 2024-09-10T08:47:04.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f481a90d740 1 -- 172.21.15.172:0/3514037632 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f480802a050 con 0x555858fb51b0 2024-09-10T08:47:04.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f47f9ffb640 1 -- 172.21.15.172:0/3514037632 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4800024050 con 0x555858fb51b0 2024-09-10T08:47:04.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f481a90d740 10 client.15555 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:04.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.619+0000 7f481a90d740 1 -- 172.21.15.172:0/3514037632 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55585903e690 con 0x555858fb51b0 2024-09-10T08:47:04.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.620+0000 7f47f9ffb640 1 -- 172.21.15.172:0/3514037632 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f480802a050 con 0x555858fb51b0 2024-09-10T08:47:04.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.620+0000 7f47f9ffb640 1 -- 172.21.15.172:0/3514037632 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f480001c6b0 con 0x555858fb51b0 2024-09-10T08:47:04.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.620+0000 7f481a90d740 10 client.15555 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:04.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.620+0000 7f481a90d740 10 client.15555 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T08:47:04.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.620+0000 7f481a90d740 20 client.15555 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:04.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.620+0000 7f481a90d740 1 --2- 172.21.15.172:0/3514037632 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5558590432a0 0x555859041fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:04.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.620+0000 7f481a90d740 4 client.15555 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:47:04.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.620+0000 7f481a90d740 1 -- 172.21.15.172:0/3514037632 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "status"}) -- 0x555858f22b10 con 0x5558590432a0 2024-09-10T08:47:04.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.620+0000 7f48162d0640 1 --2- 172.21.15.172:0/3514037632 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5558590432a0 0x555859041fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:04.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.620+0000 7f47f9ffb640 10 client.15555 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:47:04.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.620+0000 7f48162d0640 1 --2- 172.21.15.172:0/3514037632 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5558590432a0 0x555859041fb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c398d1523d4cd0d3 server_cookie=390af254172bc617 in_seq=0 out_seq=0 2024-09-10T08:47:04.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f47f9ffb640 1 -- 172.21.15.172:0/3514037632 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555858f22b10 con 0x5558590432a0 2024-09-10T08:47:04.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f47f9ffb640 10 client.15555 handle_command_reply: tid=0 2024-09-10T08:47:04.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f481a90d740 1 client.15555 shutdown 2024-09-10T08:47:04.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f481a90d740 1 -- 172.21.15.172:0/3514037632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fb51b0 msgr2=0x555858fd7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:04.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f481a90d740 1 --2- 172.21.15.172:0/3514037632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fb51b0 0x555858fd7610 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x555858fb2050 tx=0x7f48000105c0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f481a90d740 1 -- 172.21.15.172:0/3514037632 shutdown_connections 2024-09-10T08:47:04.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f481a90d740 1 --2- 172.21.15.172:0/3514037632 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5558590432a0 0x555859041fb0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f481a90d740 1 --2- 172.21.15.172:0/3514037632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555858fc3fc0 0x555858fd7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f47f9ffb640 0 client.15555 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:47:04.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f481a90d740 1 --2- 172.21.15.172:0/3514037632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555858fb51b0 0x555858fd7610 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f481a90d740 1 --2- 172.21.15.172:0/3514037632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555858fb4890 0x555858fd70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f481a90d740 1 -- 172.21.15.172:0/3514037632 >> 172.21.15.172:0/3514037632 conn(0x555858fb0f60 msgr2=0x555858fbba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:04.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f481a90d740 1 -- 172.21.15.172:0/3514037632 shutdown_connections 2024-09-10T08:47:04.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f481a90d740 1 -- 172.21.15.172:0/3514037632 wait complete. 2024-09-10T08:47:04.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f481a90d740 20 client.15555 trim_cache size 0 max 16384 2024-09-10T08:47:04.631 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:47:04.631 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:47:04.631 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 1, 2024-09-10T08:47:04.631 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 24458, 2024-09-10T08:47:04.631 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-09-10T08:47:04.631 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-09-10T08:47:04.631 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-09-10T08:47:04.631 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 133.37284848600001, 2024-09-10T08:47:04.631 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 29, 2024-09-10T08:47:04.631 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 92, 2024-09-10T08:47:04.632 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 92, 2024-09-10T08:47:04.632 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 194.640730684 2024-09-10T08:47:04.632 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:47:04.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f4818534640 1 -- 172.21.15.172:0/3022513079 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f47fc0a2b80 msgr2=0x7f47fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:04.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f4818534640 1 --2- 172.21.15.172:0/3022513079 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f47fc0a2b80 0x7f47fc0a5040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4808020000 tx=0x7f480801d000 comp rx=0 tx=0).stop 2024-09-10T08:47:04.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f4818534640 1 -- 172.21.15.172:0/3022513079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4810102da0 msgr2=0x7f48101983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:04.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.621+0000 7f4818534640 1 --2- 172.21.15.172:0/3022513079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4810102da0 0x7f48101983b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f480c005940 tx=0x7f480c004270 comp rx=0 tx=0).stop 2024-09-10T08:47:04.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.622+0000 7f4818534640 1 -- 172.21.15.172:0/3022513079 shutdown_connections 2024-09-10T08:47:04.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.622+0000 7f4818534640 1 --2- 172.21.15.172:0/3022513079 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f47fc0a2b80 0x7f47fc0a5040 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.622+0000 7f4818534640 1 --2- 172.21.15.172:0/3022513079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f481010c180 0x7f481019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.622+0000 7f4818534640 1 --2- 172.21.15.172:0/3022513079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48101036c0 0x7f48101988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.622+0000 7f4818534640 1 --2- 172.21.15.172:0/3022513079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4810102da0 0x7f48101983b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:04.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.622+0000 7f4818534640 1 -- 172.21.15.172:0/3022513079 >> 172.21.15.172:0/3022513079 conn(0x7f48100fc7a0 msgr2=0x7f48100fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:04.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.622+0000 7f4818534640 1 -- 172.21.15.172:0/3022513079 shutdown_connections 2024-09-10T08:47:04.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:04.622+0000 7f4818534640 1 -- 172.21.15.172:0/3022513079 wait complete. 2024-09-10T08:47:05.368 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T08:47:05.614 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:06.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.140+0000 7fbd941f0640 1 -- 172.21.15.172:0/663797378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd8c101420 msgr2=0x7fbd8c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.140+0000 7fbd941f0640 1 --2- 172.21.15.172:0/663797378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd8c101420 0x7fbd8c10d950 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009cd0 tx=0x7fbd8402a8e0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.140+0000 7fbd941f0640 1 -- 172.21.15.172:0/663797378 shutdown_connections 2024-09-10T08:47:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.140+0000 7fbd941f0640 1 --2- 172.21.15.172:0/663797378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd8c101420 0x7fbd8c10d950 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.140+0000 7fbd941f0640 1 --2- 172.21.15.172:0/663797378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd8c100a80 0x7fbd8c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.140+0000 7fbd941f0640 1 --2- 172.21.15.172:0/663797378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd8c106a80 0x7fbd8c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.140+0000 7fbd941f0640 1 -- 172.21.15.172:0/663797378 >> 172.21.15.172:0/663797378 conn(0x7fbd8c0fc7a0 msgr2=0x7fbd8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.141+0000 7fbd941f0640 1 -- 172.21.15.172:0/663797378 shutdown_connections 2024-09-10T08:47:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.141+0000 7fbd941f0640 1 -- 172.21.15.172:0/663797378 wait complete. 2024-09-10T08:47:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.141+0000 7fbd941f0640 1 Processor -- start 2024-09-10T08:47:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.141+0000 7fbd941f0640 1 -- start start 2024-09-10T08:47:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.141+0000 7fbd941f0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd8c100a80 0x7fbd8c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.141+0000 7fbd941f0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd8c101420 0x7fbd8c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.141+0000 7fbd941f0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd8c106a80 0x7fbd8c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.141+0000 7fbd941f0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c110d50 con 0x7fbd8c106a80 2024-09-10T08:47:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.141+0000 7fbd941f0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c110bd0 con 0x7fbd8c100a80 2024-09-10T08:47:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.141+0000 7fbd941f0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8c110ed0 con 0x7fbd8c101420 2024-09-10T08:47:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.141+0000 7fbd9278d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd8c106a80 0x7fbd8c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.141+0000 7fbd9278d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd8c106a80 0x7fbd8c19cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55952/0 (socket says 172.21.15.172:55952) 2024-09-10T08:47:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.141+0000 7fbd9278d640 1 -- 172.21.15.172:0/4123819173 learned_addr learned my addr 172.21.15.172:0/4123819173 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.142+0000 7fbd91f8c640 1 --2- 172.21.15.172:0/4123819173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd8c100a80 0x7fbd8c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.142+0000 7fbd9178b640 1 --2- 172.21.15.172:0/4123819173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd8c101420 0x7fbd8c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.142+0000 7fbd9278d640 1 -- 172.21.15.172:0/4123819173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd8c101420 msgr2=0x7fbd8c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.142+0000 7fbd9278d640 1 --2- 172.21.15.172:0/4123819173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd8c101420 0x7fbd8c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.142+0000 7fbd9278d640 1 -- 172.21.15.172:0/4123819173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd8c100a80 msgr2=0x7fbd8c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.142+0000 7fbd9278d640 1 --2- 172.21.15.172:0/4123819173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd8c100a80 0x7fbd8c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.142+0000 7fbd9278d640 1 -- 172.21.15.172:0/4123819173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8c19d370 con 0x7fbd8c106a80 2024-09-10T08:47:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.142+0000 7fbd91f8c640 1 --2- 172.21.15.172:0/4123819173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd8c100a80 0x7fbd8c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.142+0000 7fbd9278d640 1 --2- 172.21.15.172:0/4123819173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd8c106a80 0x7fbd8c19cc90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbd84005cc0 tx=0x7fbd84003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.142+0000 7fbd82ffd640 1 -- 172.21.15.172:0/4123819173 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84004240 con 0x7fbd8c106a80 2024-09-10T08:47:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.142+0000 7fbd82ffd640 1 -- 172.21.15.172:0/4123819173 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd840305a0 con 0x7fbd8c106a80 2024-09-10T08:47:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.142+0000 7fbd941f0640 1 -- 172.21.15.172:0/4123819173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd8c19d600 con 0x7fbd8c106a80 2024-09-10T08:47:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.142+0000 7fbd82ffd640 1 -- 172.21.15.172:0/4123819173 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd840466c0 con 0x7fbd8c106a80 2024-09-10T08:47:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.142+0000 7fbd941f0640 1 -- 172.21.15.172:0/4123819173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd8c1a4f40 con 0x7fbd8c106a80 2024-09-10T08:47:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.144+0000 7fbd82ffd640 1 -- 172.21.15.172:0/4123819173 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbd84030ac0 con 0x7fbd8c106a80 2024-09-10T08:47:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.144+0000 7fbd965c9740 1 Processor -- start 2024-09-10T08:47:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.145+0000 7fbd965c9740 1 -- start start 2024-09-10T08:47:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.145+0000 7fbd965c9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f57705db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.145+0000 7fbd965c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f577062f0 0x562f57710670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.145+0000 7fbd965c9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f57710bb0 0x562f57712fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.145+0000 7fbd965c9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562f5766c090 con 0x562f577059d0 2024-09-10T08:47:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.145+0000 7fbd965c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562f57613a70 con 0x562f577062f0 2024-09-10T08:47:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.145+0000 7fbd965c9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562f57668050 con 0x562f57710bb0 2024-09-10T08:47:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.145+0000 7fbd82ffd640 1 --2- 172.21.15.172:0/4123819173 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbd780a2b80 0x7fbd780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.145+0000 7fbd9178b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f57705db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.145+0000 7fbd9178b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f57705db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55958/0 (socket says 172.21.15.172:55958) 2024-09-10T08:47:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.145+0000 7fbd9178b640 1 -- 172.21.15.172:0/3572600105 learned_addr learned my addr 172.21.15.172:0/3572600105 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.145+0000 7fbd9278d640 1 --2- 172.21.15.172:0/4123819173 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbd780a2b80 0x7fbd780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.145+0000 7fbd91f8c640 1 --2- 172.21.15.172:0/3572600105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f577062f0 0x562f57710670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.145+0000 7fbd82ffd640 1 -- 172.21.15.172:0/4123819173 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fbd8402e030 con 0x7fbd8c106a80 2024-09-10T08:47:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.145+0000 7fbd9178b640 1 -- 172.21.15.172:0/3572600105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f57710bb0 msgr2=0x562f57712fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.145+0000 7fbd9178b640 1 --2- 172.21.15.172:0/3572600105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f57710bb0 0x562f57712fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.145+0000 7fbd9178b640 1 -- 172.21.15.172:0/3572600105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f577062f0 msgr2=0x562f57710670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.146+0000 7fbd9178b640 1 --2- 172.21.15.172:0/3572600105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f577062f0 0x562f57710670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.146+0000 7fbd9178b640 1 -- 172.21.15.172:0/3572600105 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f57620130 con 0x562f577059d0 2024-09-10T08:47:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.146+0000 7fbd91f8c640 1 --2- 172.21.15.172:0/3572600105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f577062f0 0x562f57710670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.146+0000 7fbd9178b640 1 --2- 172.21.15.172:0/3572600105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f57705db0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c005d90 tx=0x7fbd7c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04579282117d346 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.146+0000 7fbd9278d640 1 --2- 172.21.15.172:0/4123819173 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbd780a2b80 0x7fbd780a5040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbd840bca10 tx=0x7fbd840878e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.146+0000 7fbd77fff640 1 -- 172.21.15.172:0/3572600105 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c016070 con 0x562f577059d0 2024-09-10T08:47:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.146+0000 7fbd77fff640 1 -- 172.21.15.172:0/3572600105 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd7c012b30 con 0x562f577059d0 2024-09-10T08:47:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.146+0000 7fbd77fff640 1 -- 172.21.15.172:0/3572600105 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c011b30 con 0x562f577059d0 2024-09-10T08:47:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.147+0000 7fbd965c9740 1 -- 172.21.15.172:0/3572600105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 msgr2=0x562f57705db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.147+0000 7fbd965c9740 1 --2- 172.21.15.172:0/3572600105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f57705db0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c005d90 tx=0x7fbd7c010740 comp rx=0 tx=0).stop 2024-09-10T08:47:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.147+0000 7fbd965c9740 1 -- 172.21.15.172:0/3572600105 shutdown_connections 2024-09-10T08:47:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.147+0000 7fbd965c9740 1 --2- 172.21.15.172:0/3572600105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f57710bb0 0x562f57712fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.147+0000 7fbd965c9740 1 --2- 172.21.15.172:0/3572600105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f577062f0 0x562f57710670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.147+0000 7fbd965c9740 1 --2- 172.21.15.172:0/3572600105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f57705db0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.147+0000 7fbd965c9740 1 -- 172.21.15.172:0/3572600105 >> 172.21.15.172:0/3572600105 conn(0x562f57701f60 msgr2=0x562f57702340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.147+0000 7fbd965c9740 1 -- 172.21.15.172:0/3572600105 shutdown_connections 2024-09-10T08:47:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.147+0000 7fbd965c9740 1 -- 172.21.15.172:0/3572600105 wait complete. 2024-09-10T08:47:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.147+0000 7fbd965c9740 1 -- start start 2024-09-10T08:47:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.147+0000 7fbd965c9740 1 Processor -- start 2024-09-10T08:47:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd965c9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f577104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd965c9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f577062f0 0x562f5770d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd965c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f57710bb0 0x562f5770dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd965c9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562f57713780 con 0x562f577059d0 2024-09-10T08:47:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd965c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562f57626f20 con 0x562f57710bb0 2024-09-10T08:47:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd965c9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562f57713900 con 0x562f577062f0 2024-09-10T08:47:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd965c9740 10 client.0 _ll_register_callbacks cb 0x562f575f0030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd9178b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f577104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd9178b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f577104a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55966/0 (socket says 172.21.15.172:55966) 2024-09-10T08:47:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd9178b640 1 -- 172.21.15.172:0/2985245472 learned_addr learned my addr 172.21.15.172:0/2985245472 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd91f8c640 1 --2- 172.21.15.172:0/2985245472 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f577062f0 0x562f5770d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd9178b640 1 -- 172.21.15.172:0/2985245472 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f577062f0 msgr2=0x562f5770d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd9178b640 1 --2- 172.21.15.172:0/2985245472 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f577062f0 0x562f5770d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd9178b640 1 -- 172.21.15.172:0/2985245472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f57710bb0 msgr2=0x562f5770dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd9178b640 1 --2- 172.21.15.172:0/2985245472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f57710bb0 0x562f5770dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd9178b640 1 -- 172.21.15.172:0/2985245472 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f573dd2b0 con 0x562f577059d0 2024-09-10T08:47:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd9178b640 1 --2- 172.21.15.172:0/2985245472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f577104a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c011810 tx=0x7fbd7c00ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a02969957e4303 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd757fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd757fa640 1 -- 172.21.15.172:0/2985245472 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c016070 con 0x562f577059d0 2024-09-10T08:47:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd757fa640 1 -- 172.21.15.172:0/2985245472 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fbd7c005460 con 0x562f577059d0 2024-09-10T08:47:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.148+0000 7fbd965c9740 1 -- 172.21.15.172:0/2985245472 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd7c012b30 con 0x562f577059d0 2024-09-10T08:47:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.149+0000 7fbd757fa640 1 -- 172.21.15.172:0/2985245472 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbd7c02f050 con 0x562f577059d0 2024-09-10T08:47:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.149+0000 7fbd965c9740 10 client.15567 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.149+0000 7fbd965c9740 1 -- 172.21.15.172:0/2985245472 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562f5771f0f0 con 0x562f577059d0 2024-09-10T08:47:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.149+0000 7fbd757fa640 1 -- 172.21.15.172:0/2985245472 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd7c012b30 con 0x562f577059d0 2024-09-10T08:47:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.149+0000 7fbd757fa640 1 -- 172.21.15.172:0/2985245472 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbd7c012ce0 con 0x562f577059d0 2024-09-10T08:47:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.149+0000 7fbd965c9740 10 client.15567 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.149+0000 7fbd965c9740 10 client.15567 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:47:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.149+0000 7fbd965c9740 20 client.15567 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.149+0000 7fbd965c9740 1 --2- 172.21.15.172:0/2985245472 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562f577213d0 0x562f57722840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.149+0000 7fbd965c9740 4 client.15567 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.149+0000 7fbd965c9740 1 -- 172.21.15.172:0/2985245472 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f57679600 con 0x562f577213d0 2024-09-10T08:47:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.149+0000 7fbd91f8c640 1 --2- 172.21.15.172:0/2985245472 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562f577213d0 0x562f57722840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.150+0000 7fbd757fa640 10 client.15567 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.150+0000 7fbd91f8c640 1 --2- 172.21.15.172:0/2985245472 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562f577213d0 0x562f57722840 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a00f0ef87905ed0e server_cookie=1b75c605b460ad8e in_seq=0 out_seq=0 2024-09-10T08:47:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.150+0000 7fbd757fa640 1 -- 172.21.15.172:0/2985245472 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f57679600 con 0x562f577213d0 2024-09-10T08:47:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.150+0000 7fbd757fa640 10 client.15567 handle_command_reply: tid=0 2024-09-10T08:47:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.150+0000 7fbd965c9740 1 client.15567 shutdown 2024-09-10T08:47:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.151+0000 7fbd965c9740 1 -- 172.21.15.172:0/2985245472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 msgr2=0x562f577104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.151+0000 7fbd965c9740 1 --2- 172.21.15.172:0/2985245472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f577104a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c011810 tx=0x7fbd7c00ca50 comp rx=0 tx=0).stop 2024-09-10T08:47:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.151+0000 7fbd965c9740 1 -- 172.21.15.172:0/2985245472 shutdown_connections 2024-09-10T08:47:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.151+0000 7fbd965c9740 1 --2- 172.21.15.172:0/2985245472 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562f577213d0 0x562f57722840 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.151+0000 7fbd965c9740 1 --2- 172.21.15.172:0/2985245472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f57710bb0 0x562f5770dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.151+0000 7fbd965c9740 1 --2- 172.21.15.172:0/2985245472 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f577062f0 0x562f5770d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.151+0000 7fbd757fa640 0 client.15567 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.151+0000 7fbd965c9740 1 --2- 172.21.15.172:0/2985245472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f577104a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.151+0000 7fbd965c9740 1 -- 172.21.15.172:0/2985245472 >> 172.21.15.172:0/2985245472 conn(0x562f57701f60 msgr2=0x562f57707c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:06.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.151+0000 7fbd965c9740 1 -- 172.21.15.172:0/2985245472 shutdown_connections 2024-09-10T08:47:06.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.151+0000 7fbd965c9740 1 -- 172.21.15.172:0/2985245472 wait complete. 2024-09-10T08:47:06.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.151+0000 7fbd965c9740 20 client.15567 trim_cache size 0 max 16384 2024-09-10T08:47:06.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.169+0000 7fbd965c9740 1 Processor -- start 2024-09-10T08:47:06.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.169+0000 7fbd965c9740 1 -- start start 2024-09-10T08:47:06.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.169+0000 7fbd965c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f577059d0 0x562f57721aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.169+0000 7fbd965c9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f577062f0 0x562f576fcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.169+0000 7fbd965c9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f57710bb0 0x562f5770aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.169+0000 7fbd965c9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562f57720b80 con 0x562f57710bb0 2024-09-10T08:47:06.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.169+0000 7fbd965c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562f57721250 con 0x562f577059d0 2024-09-10T08:47:06.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.169+0000 7fbd965c9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562f57721920 con 0x562f577062f0 2024-09-10T08:47:06.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.169+0000 7fbd9178b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f57710bb0 0x562f5770aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.169+0000 7fbd9178b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f57710bb0 0x562f5770aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44620/0 (socket says 172.21.15.172:44620) 2024-09-10T08:47:06.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.169+0000 7fbd9178b640 1 -- 172.21.15.172:0/409339164 learned_addr learned my addr 172.21.15.172:0/409339164 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:06.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.170+0000 7fbd9178b640 1 --2- 172.21.15.172:0/409339164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f577059d0 0x562f57721aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.170+0000 7fbd91f8c640 1 --2- 172.21.15.172:0/409339164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f577062f0 0x562f576fcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.170+0000 7fbd9178b640 1 -- 172.21.15.172:0/409339164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f577062f0 msgr2=0x562f576fcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:06.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.170+0000 7fbd9178b640 1 --2- 172.21.15.172:0/409339164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f577062f0 0x562f576fcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.170+0000 7fbd9178b640 1 -- 172.21.15.172:0/409339164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f577059d0 msgr2=0x562f57721aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:06.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.170+0000 7fbd9178b640 1 --2- 172.21.15.172:0/409339164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f577059d0 0x562f57721aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.170+0000 7fbd9178b640 1 -- 172.21.15.172:0/409339164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f5770e310 con 0x562f57710bb0 2024-09-10T08:47:06.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.170+0000 7fbd9178b640 1 --2- 172.21.15.172:0/409339164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f57710bb0 0x562f5770aab0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c005020 tx=0x7fbd7c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f68f8894b879cb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:06.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.170+0000 7fbd777fe640 1 -- 172.21.15.172:0/409339164 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c016070 con 0x562f57710bb0 2024-09-10T08:47:06.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.170+0000 7fbd777fe640 1 -- 172.21.15.172:0/409339164 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd7c022070 con 0x562f57710bb0 2024-09-10T08:47:06.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.170+0000 7fbd777fe640 1 -- 172.21.15.172:0/409339164 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c010fe0 con 0x562f57710bb0 2024-09-10T08:47:06.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.171+0000 7fbd965c9740 1 -- 172.21.15.172:0/409339164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f57710bb0 msgr2=0x562f5770aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:06.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.171+0000 7fbd965c9740 1 --2- 172.21.15.172:0/409339164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f57710bb0 0x562f5770aab0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c005020 tx=0x7fbd7c014040 comp rx=0 tx=0).stop 2024-09-10T08:47:06.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.171+0000 7fbd965c9740 1 -- 172.21.15.172:0/409339164 shutdown_connections 2024-09-10T08:47:06.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.171+0000 7fbd965c9740 1 --2- 172.21.15.172:0/409339164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f57710bb0 0x562f5770aab0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.171+0000 7fbd965c9740 1 --2- 172.21.15.172:0/409339164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f577062f0 0x562f576fcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.171+0000 7fbd965c9740 1 --2- 172.21.15.172:0/409339164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f577059d0 0x562f57721aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.171+0000 7fbd965c9740 1 -- 172.21.15.172:0/409339164 >> 172.21.15.172:0/409339164 conn(0x562f57701f60 msgr2=0x562f5770c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:06.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.171+0000 7fbd965c9740 1 -- 172.21.15.172:0/409339164 shutdown_connections 2024-09-10T08:47:06.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.171+0000 7fbd965c9740 1 -- 172.21.15.172:0/409339164 wait complete. 2024-09-10T08:47:06.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.171+0000 7fbd965c9740 1 -- start start 2024-09-10T08:47:06.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.171+0000 7fbd965c9740 1 Processor -- start 2024-09-10T08:47:06.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd965c9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f5772b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd965c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f577062f0 0x562f57724570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd965c9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f57710bb0 0x562f57724ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd965c9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562f577226c0 con 0x562f577059d0 2024-09-10T08:47:06.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd965c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562f57721ff0 con 0x562f577062f0 2024-09-10T08:47:06.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd965c9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562f57722d90 con 0x562f57710bb0 2024-09-10T08:47:06.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd965c9740 10 client.0 _ll_register_callbacks cb 0x7fbd7c00cb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:06.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd9178b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f5772b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd9178b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f5772b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44632/0 (socket says 172.21.15.172:44632) 2024-09-10T08:47:06.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd9178b640 1 -- 172.21.15.172:0/2363987896 learned_addr learned my addr 172.21.15.172:0/2363987896 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:06.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd91f8c640 1 --2- 172.21.15.172:0/2363987896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f577062f0 0x562f57724570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd9178b640 1 --2- 172.21.15.172:0/2363987896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f57710bb0 0x562f57724ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd9178b640 1 -- 172.21.15.172:0/2363987896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f57710bb0 msgr2=0x562f57724ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:06.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd9178b640 1 --2- 172.21.15.172:0/2363987896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f57710bb0 0x562f57724ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd9178b640 1 -- 172.21.15.172:0/2363987896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f577062f0 msgr2=0x562f57724570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:06.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd9178b640 1 --2- 172.21.15.172:0/2363987896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f577062f0 0x562f57724570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd9178b640 1 -- 172.21.15.172:0/2363987896 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f57725300 con 0x562f577059d0 2024-09-10T08:47:06.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd9178b640 1 --2- 172.21.15.172:0/2363987896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f5772b240 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c005a90 tx=0x7fbd7c0053e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d20a575e032479 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:06.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.172+0000 7fbd757fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:06.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.173+0000 7fbd757fa640 1 -- 172.21.15.172:0/2363987896 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c016040 con 0x562f577059d0 2024-09-10T08:47:06.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.173+0000 7fbd757fa640 1 -- 172.21.15.172:0/2363987896 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fbd7c0128d0 con 0x562f577059d0 2024-09-10T08:47:06.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.173+0000 7fbd965c9740 1 -- 172.21.15.172:0/2363987896 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd7c022070 con 0x562f577059d0 2024-09-10T08:47:06.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.173+0000 7fbd757fa640 1 -- 172.21.15.172:0/2363987896 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbd7c005680 con 0x562f577059d0 2024-09-10T08:47:06.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.173+0000 7fbd965c9740 10 client.15573 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:06.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.173+0000 7fbd965c9740 1 -- 172.21.15.172:0/2363987896 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562f5778bba0 con 0x562f577059d0 2024-09-10T08:47:06.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.173+0000 7fbd757fa640 1 -- 172.21.15.172:0/2363987896 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd7c022070 con 0x562f577059d0 2024-09-10T08:47:06.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.173+0000 7fbd757fa640 1 -- 172.21.15.172:0/2363987896 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbd7c011810 con 0x562f577059d0 2024-09-10T08:47:06.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.173+0000 7fbd965c9740 10 client.15573 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:06.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.173+0000 7fbd965c9740 10 client.15573 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T08:47:06.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.173+0000 7fbd965c9740 20 client.15573 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:06.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.173+0000 7fbd965c9740 1 --2- 172.21.15.172:0/2363987896 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562f57790490 0x562f5778f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:06.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.173+0000 7fbd965c9740 4 client.15573 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T08:47:06.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.173+0000 7fbd965c9740 1 -- 172.21.15.172:0/2363987896 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "status"}) -- 0x562f57673b10 con 0x562f57790490 2024-09-10T08:47:06.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.174+0000 7fbd91f8c640 1 --2- 172.21.15.172:0/2363987896 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562f57790490 0x562f5778f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:06.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.174+0000 7fbd757fa640 10 client.15573 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:06.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.174+0000 7fbd91f8c640 1 --2- 172.21.15.172:0/2363987896 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562f57790490 0x562f5778f1a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=928c3f85210dcd4f server_cookie=523cf071f4a931e8 in_seq=0 out_seq=0 2024-09-10T08:47:06.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.174+0000 7fbd757fa640 1 -- 172.21.15.172:0/2363987896 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562f57673b10 con 0x562f57790490 2024-09-10T08:47:06.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.174+0000 7fbd757fa640 10 client.15573 handle_command_reply: tid=0 2024-09-10T08:47:06.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.174+0000 7fbd965c9740 1 client.15573 shutdown 2024-09-10T08:47:06.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.174+0000 7fbd965c9740 1 -- 172.21.15.172:0/2363987896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 msgr2=0x562f5772b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:06.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.174+0000 7fbd965c9740 1 --2- 172.21.15.172:0/2363987896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f5772b240 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c005a90 tx=0x7fbd7c0053e0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.174+0000 7fbd965c9740 1 -- 172.21.15.172:0/2363987896 shutdown_connections 2024-09-10T08:47:06.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.174+0000 7fbd965c9740 1 --2- 172.21.15.172:0/2363987896 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562f57790490 0x562f5778f1a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.174+0000 7fbd965c9740 1 --2- 172.21.15.172:0/2363987896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f57710bb0 0x562f57724ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.174+0000 7fbd965c9740 1 --2- 172.21.15.172:0/2363987896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f577062f0 0x562f57724570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.174+0000 7fbd965c9740 1 --2- 172.21.15.172:0/2363987896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f577059d0 0x562f5772b240 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.174+0000 7fbd965c9740 1 -- 172.21.15.172:0/2363987896 >> 172.21.15.172:0/2363987896 conn(0x562f57701f60 msgr2=0x562f57700800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:06.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.174+0000 7fbd757fa640 0 client.15573 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:06.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.174+0000 7fbd965c9740 1 -- 172.21.15.172:0/2363987896 shutdown_connections 2024-09-10T08:47:06.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.174+0000 7fbd965c9740 1 -- 172.21.15.172:0/2363987896 wait complete. 2024-09-10T08:47:06.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.175+0000 7fbd965c9740 20 client.15573 trim_cache size 0 max 16384 2024-09-10T08:47:06.184 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:47:06.184 INFO:teuthology.orchestra.run.smithi172.stdout: "cluster_fsid": "a7dba522-6f4f-11ef-bcea-c7b262605968", 2024-09-10T08:47:06.184 INFO:teuthology.orchestra.run.smithi172.stdout: "whoami": 0, 2024-09-10T08:47:06.184 INFO:teuthology.orchestra.run.smithi172.stdout: "id": 24488, 2024-09-10T08:47:06.184 INFO:teuthology.orchestra.run.smithi172.stdout: "want_state": "up:active", 2024-09-10T08:47:06.184 INFO:teuthology.orchestra.run.smithi172.stdout: "state": "up:active", 2024-09-10T08:47:06.184 INFO:teuthology.orchestra.run.smithi172.stdout: "fs_name": "cephfs", 2024-09-10T08:47:06.184 INFO:teuthology.orchestra.run.smithi172.stdout: "rank_uptime": 174.500175579, 2024-09-10T08:47:06.184 INFO:teuthology.orchestra.run.smithi172.stdout: "mdsmap_epoch": 29, 2024-09-10T08:47:06.184 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch": 92, 2024-09-10T08:47:06.184 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_epoch_barrier": 92, 2024-09-10T08:47:06.184 INFO:teuthology.orchestra.run.smithi172.stdout: "uptime": 184.88846225500001 2024-09-10T08:47:06.184 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:47:06.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.175+0000 7fbd941f0640 1 -- 172.21.15.172:0/4123819173 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbd780a2b80 msgr2=0x7fbd780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:06.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.175+0000 7fbd941f0640 1 --2- 172.21.15.172:0/4123819173 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbd780a2b80 0x7fbd780a5040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbd840bca10 tx=0x7fbd840878e0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.175+0000 7fbd941f0640 1 -- 172.21.15.172:0/4123819173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd8c106a80 msgr2=0x7fbd8c19cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:06.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.175+0000 7fbd941f0640 1 --2- 172.21.15.172:0/4123819173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd8c106a80 0x7fbd8c19cc90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbd84005cc0 tx=0x7fbd84003ba0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.175+0000 7fbd941f0640 1 -- 172.21.15.172:0/4123819173 shutdown_connections 2024-09-10T08:47:06.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.175+0000 7fbd941f0640 1 --2- 172.21.15.172:0/4123819173 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbd780a2b80 0x7fbd780a5040 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.175+0000 7fbd941f0640 1 --2- 172.21.15.172:0/4123819173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd8c106a80 0x7fbd8c19cc90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.175+0000 7fbd941f0640 1 --2- 172.21.15.172:0/4123819173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd8c101420 0x7fbd8c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.175+0000 7fbd941f0640 1 --2- 172.21.15.172:0/4123819173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd8c100a80 0x7fbd8c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:06.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.175+0000 7fbd941f0640 1 -- 172.21.15.172:0/4123819173 >> 172.21.15.172:0/4123819173 conn(0x7fbd8c0fc7a0 msgr2=0x7fbd8c1048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:06.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.175+0000 7fbd941f0640 1 -- 172.21.15.172:0/4123819173 shutdown_connections 2024-09-10T08:47:06.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:06.175+0000 7fbd941f0640 1 -- 172.21.15.172:0/4123819173 wait complete. 2024-09-10T08:47:06.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:06 smithi172 ceph-mon[30854]: pgmap v376: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-09-10T08:47:06.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:06 smithi174 ceph-mon[34068]: pgmap v376: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-09-10T08:47:06.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:06 smithi191 ceph-mon[35314]: pgmap v376: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-09-10T08:47:06.868 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-10T08:47:08.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:08 smithi174 ceph-mon[34068]: pgmap v377: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 16 op/s 2024-09-10T08:47:08.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:08 smithi191 ceph-mon[35314]: pgmap v377: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 16 op/s 2024-09-10T08:47:08.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:08 smithi172 ceph-mon[30854]: pgmap v377: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.2 KiB/s wr, 16 op/s 2024-09-10T08:47:10.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:10 smithi174 ceph-mon[34068]: pgmap v378: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-09-10T08:47:10.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:10 smithi191 ceph-mon[35314]: pgmap v378: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-09-10T08:47:10.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:10 smithi172 ceph-mon[30854]: pgmap v378: 129 pgs: 129 active+clean; 660 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-09-10T08:47:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.048+0000 7f8fbd001640 1 -- 172.21.15.172:0/2584615621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb8100af0 msgr2=0x7f8fb8100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.048+0000 7f8fbd001640 1 --2- 172.21.15.172:0/2584615621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb8100af0 0x7f8fb8100f70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009900 tx=0x7f8fa002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.049+0000 7f8fbd001640 1 -- 172.21.15.172:0/2584615621 shutdown_connections 2024-09-10T08:47:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.049+0000 7f8fbd001640 1 --2- 172.21.15.172:0/2584615621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fb81014b0 0x7f8fb8108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.049+0000 7f8fbd001640 1 --2- 172.21.15.172:0/2584615621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb8100af0 0x7f8fb8100f70 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.049+0000 7f8fbd001640 1 --2- 172.21.15.172:0/2584615621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fb8101ea0 0x7f8fb81022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.049+0000 7f8fbd001640 1 -- 172.21.15.172:0/2584615621 >> 172.21.15.172:0/2584615621 conn(0x7f8fb80fb790 msgr2=0x7f8fb80fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.049+0000 7f8fbd001640 1 -- 172.21.15.172:0/2584615621 shutdown_connections 2024-09-10T08:47:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.049+0000 7f8fbd001640 1 -- 172.21.15.172:0/2584615621 wait complete. 2024-09-10T08:47:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.049+0000 7f8fbd001640 1 Processor -- start 2024-09-10T08:47:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.049+0000 7f8fbd001640 1 -- start start 2024-09-10T08:47:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.049+0000 7f8fbd001640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fb8100af0 0x7f8fb8197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.049+0000 7f8fbd001640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fb81014b0 0x7f8fb8198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fbd001640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb8101ea0 0x7f8fb819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fbd001640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb810c3b0 con 0x7f8fb8101ea0 2024-09-10T08:47:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fbd001640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb810c230 con 0x7f8fb81014b0 2024-09-10T08:47:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fbd001640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb810c530 con 0x7f8fb8100af0 2024-09-10T08:47:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fb759e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb8101ea0 0x7f8fb819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fb6d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fb8100af0 0x7f8fb8197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fb759e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb8101ea0 0x7f8fb819f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44642/0 (socket says 172.21.15.172:44642) 2024-09-10T08:47:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fb759e640 1 -- 172.21.15.172:0/1025523049 learned_addr learned my addr 172.21.15.172:0/1025523049 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8faffff640 1 --2- 172.21.15.172:0/1025523049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fb81014b0 0x7f8fb8198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fb759e640 1 -- 172.21.15.172:0/1025523049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fb8100af0 msgr2=0x7f8fb8197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fb759e640 1 --2- 172.21.15.172:0/1025523049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fb8100af0 0x7f8fb8197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fb759e640 1 -- 172.21.15.172:0/1025523049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fb81014b0 msgr2=0x7f8fb8198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fb759e640 1 --2- 172.21.15.172:0/1025523049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fb81014b0 0x7f8fb8198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fb759e640 1 -- 172.21.15.172:0/1025523049 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb819f750 con 0x7f8fb8101ea0 2024-09-10T08:47:11.055 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fb759e640 1 --2- 172.21.15.172:0/1025523049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb8101ea0 0x7f8fb819f0e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000cce0 tx=0x7f8fb000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:11.055 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fb4d99640 1 -- 172.21.15.172:0/1025523049 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb000f450 con 0x7f8fb8101ea0 2024-09-10T08:47:11.055 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fb4d99640 1 -- 172.21.15.172:0/1025523049 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fb000faa0 con 0x7f8fb8101ea0 2024-09-10T08:47:11.055 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fbd001640 1 -- 172.21.15.172:0/1025523049 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb819fa40 con 0x7f8fb8101ea0 2024-09-10T08:47:11.055 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fb4d99640 1 -- 172.21.15.172:0/1025523049 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0018b70 con 0x7f8fb8101ea0 2024-09-10T08:47:11.055 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.050+0000 7f8fbd001640 1 -- 172.21.15.172:0/1025523049 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb819ff60 con 0x7f8fb8101ea0 2024-09-10T08:47:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.053+0000 7f8fb4d99640 1 -- 172.21.15.172:0/1025523049 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8fb0018d10 con 0x7f8fb8101ea0 2024-09-10T08:47:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.053+0000 7f8fb4d99640 1 --2- 172.21.15.172:0/1025523049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8fa40a2c00 0x7f8fa40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.053+0000 7f8fb4d99640 1 -- 172.21.15.172:0/1025523049 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8fb00cf740 con 0x7f8fb8101ea0 2024-09-10T08:47:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.053+0000 7f8fb6d9d640 1 --2- 172.21.15.172:0/1025523049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8fa40a2c00 0x7f8fa40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.054+0000 7f8fbfe70740 1 Processor -- start 2024-09-10T08:47:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.054+0000 7f8fbfe70740 1 -- start start 2024-09-10T08:47:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.054+0000 7f8fb6d9d640 1 --2- 172.21.15.172:0/1025523049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8fa40a2c00 0x7f8fa40a50c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8069fa0 tx=0x7f8fa8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.054+0000 7f8fbfe70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f62b70 0x559ad1f62f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.054+0000 7f8fbfe70740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f634b0 0x559ad1f67f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.054+0000 7f8fbfe70740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f68540 0x559ad1f6aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.054+0000 7f8fbfe70740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559ad1ec62f0 con 0x559ad1f634b0 2024-09-10T08:47:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.054+0000 7f8fbfe70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559ad1e6e270 con 0x559ad1f62b70 2024-09-10T08:47:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.054+0000 7f8fbfe70740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559ad1ec22b0 con 0x559ad1f68540 2024-09-10T08:47:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.054+0000 7f8faffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f634b0 0x559ad1f67f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.054+0000 7f8faffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f634b0 0x559ad1f67f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44652/0 (socket says 172.21.15.172:44652) 2024-09-10T08:47:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.054+0000 7f8faffff640 1 -- 172.21.15.172:0/1168824860 learned_addr learned my addr 172.21.15.172:0/1168824860 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.054+0000 7f8fb6d9d640 1 --2- 172.21.15.172:0/1168824860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f62b70 0x559ad1f62f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.054+0000 7f8faffff640 1 -- 172.21.15.172:0/1168824860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f68540 msgr2=0x559ad1f6aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.055+0000 7f8faffff640 1 --2- 172.21.15.172:0/1168824860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f68540 0x559ad1f6aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.055+0000 7f8faffff640 1 -- 172.21.15.172:0/1168824860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f62b70 msgr2=0x559ad1f62f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.055+0000 7f8faffff640 1 --2- 172.21.15.172:0/1168824860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f62b70 0x559ad1f62f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.055+0000 7f8faffff640 1 -- 172.21.15.172:0/1168824860 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ad1e6d340 con 0x559ad1f634b0 2024-09-10T08:47:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.055+0000 7f8faffff640 1 --2- 172.21.15.172:0/1168824860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f634b0 0x559ad1f67f70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8fb806a190 tx=0x7f8fa0032d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de29b97811fbbab5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.055+0000 7f8fad7fa640 1 -- 172.21.15.172:0/1168824860 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0035070 con 0x559ad1f634b0 2024-09-10T08:47:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.055+0000 7f8fad7fa640 1 -- 172.21.15.172:0/1168824860 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fa0031780 con 0x559ad1f634b0 2024-09-10T08:47:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.055+0000 7f8fad7fa640 1 -- 172.21.15.172:0/1168824860 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0045470 con 0x559ad1f634b0 2024-09-10T08:47:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 1 -- 172.21.15.172:0/1168824860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f634b0 msgr2=0x559ad1f67f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/1168824860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f634b0 0x559ad1f67f70 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8fb806a190 tx=0x7f8fa0032d80 comp rx=0 tx=0).stop 2024-09-10T08:47:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 1 -- 172.21.15.172:0/1168824860 shutdown_connections 2024-09-10T08:47:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/1168824860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f68540 0x559ad1f6aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/1168824860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f634b0 0x559ad1f67f70 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/1168824860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f62b70 0x559ad1f62f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 1 -- 172.21.15.172:0/1168824860 >> 172.21.15.172:0/1168824860 conn(0x559ad1f5e1e0 msgr2=0x559ad1f5e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 1 -- 172.21.15.172:0/1168824860 shutdown_connections 2024-09-10T08:47:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 1 -- 172.21.15.172:0/1168824860 wait complete. 2024-09-10T08:47:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 1 -- start start 2024-09-10T08:47:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 1 Processor -- start 2024-09-10T08:47:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f62b70 0x559ad1f6ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f634b0 0x559ad1f6d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f68540 0x559ad1f743a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559ad1f6b4b0 con 0x559ad1f68540 2024-09-10T08:47:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559ad1e4f290 con 0x559ad1f634b0 2024-09-10T08:47:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559ad1f6b630 con 0x559ad1f62b70 2024-09-10T08:47:11.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fbfe70740 10 client.0 _ll_register_callbacks cb 0x559ad1e80140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fb759e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f68540 0x559ad1f743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fb759e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f68540 0x559ad1f743a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44654/0 (socket says 172.21.15.172:44654) 2024-09-10T08:47:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.056+0000 7f8fb759e640 1 -- 172.21.15.172:0/3046536194 learned_addr learned my addr 172.21.15.172:0/3046536194 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.057+0000 7f8faffff640 1 --2- 172.21.15.172:0/3046536194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f634b0 0x559ad1f6d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.057+0000 7f8fb6d9d640 1 --2- 172.21.15.172:0/3046536194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f62b70 0x559ad1f6ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.057+0000 7f8fb759e640 1 -- 172.21.15.172:0/3046536194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f62b70 msgr2=0x559ad1f6ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.057+0000 7f8fb759e640 1 --2- 172.21.15.172:0/3046536194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f62b70 0x559ad1f6ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.057+0000 7f8fb759e640 1 -- 172.21.15.172:0/3046536194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f634b0 msgr2=0x559ad1f6d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.057+0000 7f8fb759e640 1 --2- 172.21.15.172:0/3046536194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f634b0 0x559ad1f6d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.057+0000 7f8fb759e640 1 -- 172.21.15.172:0/3046536194 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ad1bd9210 con 0x559ad1f68540 2024-09-10T08:47:11.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.057+0000 7f8fb759e640 1 --2- 172.21.15.172:0/3046536194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f68540 0x559ad1f743a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00a4320 tx=0x7f8fb000fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635ad5f033bd5588 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.057+0000 7f8f9e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.057+0000 7f8f9e7fc640 1 -- 172.21.15.172:0/3046536194 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0086040 con 0x559ad1f68540 2024-09-10T08:47:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.057+0000 7f8f9e7fc640 1 -- 172.21.15.172:0/3046536194 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8fb00a43c0 con 0x559ad1f68540 2024-09-10T08:47:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.057+0000 7f8fbfe70740 1 -- 172.21.15.172:0/3046536194 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fa0031780 con 0x559ad1f68540 2024-09-10T08:47:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.057+0000 7f8f9e7fc640 1 -- 172.21.15.172:0/3046536194 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8fb009ff90 con 0x559ad1f68540 2024-09-10T08:47:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.057+0000 7f8fbfe70740 10 client.15585 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.057+0000 7f8fbfe70740 1 -- 172.21.15.172:0/3046536194 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559ad1ca5620 con 0x559ad1f68540 2024-09-10T08:47:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.057+0000 7f8f9e7fc640 1 -- 172.21.15.172:0/3046536194 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fa0031780 con 0x559ad1f68540 2024-09-10T08:47:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.058+0000 7f8f9e7fc640 1 -- 172.21.15.172:0/3046536194 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8fb0097930 con 0x559ad1f68540 2024-09-10T08:47:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.058+0000 7f8fbfe70740 10 client.15585 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.058+0000 7f8fbfe70740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.058+0000 7f8fbfe70740 20 client.15585 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:11.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.058+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/3046536194 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ad1f78af0 0x559ad1f7aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.058+0000 7f8fbfe70740 4 client.15585 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.058+0000 7f8fbfe70740 1 -- 172.21.15.172:0/3046536194 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ad1ed3a20 con 0x559ad1f78af0 2024-09-10T08:47:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.058+0000 7f8fb6d9d640 1 --2- 172.21.15.172:0/3046536194 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ad1f78af0 0x559ad1f7aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.058+0000 7f8fb6d9d640 1 --2- 172.21.15.172:0/3046536194 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ad1f78af0 0x559ad1f7aee0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e174cbb1f0026f34 server_cookie=84578d7f0ebc093 in_seq=0 out_seq=0 2024-09-10T08:47:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.058+0000 7f8f9e7fc640 10 client.15585 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.059+0000 7f8f9e7fc640 1 -- 172.21.15.172:0/3046536194 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ad1ed3a20 con 0x559ad1f78af0 2024-09-10T08:47:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.059+0000 7f8f9e7fc640 10 client.15585 handle_command_reply: tid=0 2024-09-10T08:47:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.059+0000 7f8fbfe70740 1 client.15585 shutdown 2024-09-10T08:47:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.059+0000 7f8fbfe70740 1 -- 172.21.15.172:0/3046536194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f68540 msgr2=0x559ad1f743a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.059+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/3046536194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f68540 0x559ad1f743a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00a4320 tx=0x7f8fb000fc50 comp rx=0 tx=0).stop 2024-09-10T08:47:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.059+0000 7f8fbfe70740 1 -- 172.21.15.172:0/3046536194 shutdown_connections 2024-09-10T08:47:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.059+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/3046536194 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ad1f78af0 0x559ad1f7aee0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.059+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/3046536194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f68540 0x559ad1f743a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.059+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/3046536194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f634b0 0x559ad1f6d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.059+0000 7f8f9e7fc640 0 client.15585 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:11.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.059+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/3046536194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f62b70 0x559ad1f6ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.059+0000 7f8fbfe70740 1 -- 172.21.15.172:0/3046536194 >> 172.21.15.172:0/3046536194 conn(0x559ad1f5e1e0 msgr2=0x559ad1f5f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:11.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.059+0000 7f8fbfe70740 1 -- 172.21.15.172:0/3046536194 shutdown_connections 2024-09-10T08:47:11.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.059+0000 7f8fbfe70740 1 -- 172.21.15.172:0/3046536194 wait complete. 2024-09-10T08:47:11.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.059+0000 7f8fbfe70740 20 client.15585 trim_cache size 0 max 16384 2024-09-10T08:47:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.073+0000 7f8fbfe70740 1 Processor -- start 2024-09-10T08:47:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8fbfe70740 1 -- start start 2024-09-10T08:47:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8fbfe70740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f62b70 0x559ad1f5f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8fbfe70740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f634b0 0x559ad1f5fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8fbfe70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f68540 0x559ad1f74360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8fbfe70740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb81aa210 con 0x559ad1f634b0 2024-09-10T08:47:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8fbfe70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb81abd50 con 0x559ad1f68540 2024-09-10T08:47:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8fbfe70740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb81aa8e0 con 0x559ad1f62b70 2024-09-10T08:47:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8faffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f634b0 0x559ad1f5fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8faffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f634b0 0x559ad1f5fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44670/0 (socket says 172.21.15.172:44670) 2024-09-10T08:47:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8faffff640 1 -- 172.21.15.172:0/4010910497 learned_addr learned my addr 172.21.15.172:0/4010910497 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8fb759e640 1 --2- 172.21.15.172:0/4010910497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f68540 0x559ad1f74360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8fb6d9d640 1 --2- 172.21.15.172:0/4010910497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f62b70 0x559ad1f5f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8faffff640 1 -- 172.21.15.172:0/4010910497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f62b70 msgr2=0x559ad1f5f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8faffff640 1 --2- 172.21.15.172:0/4010910497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f62b70 0x559ad1f5f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8faffff640 1 -- 172.21.15.172:0/4010910497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f68540 msgr2=0x559ad1f74360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8faffff640 1 --2- 172.21.15.172:0/4010910497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f68540 0x559ad1f74360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8faffff640 1 -- 172.21.15.172:0/4010910497 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ad1f7b510 con 0x559ad1f634b0 2024-09-10T08:47:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.074+0000 7f8faffff640 1 --2- 172.21.15.172:0/4010910497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f634b0 0x559ad1f5fce0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0044000 tx=0x7f8fa002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60043434e7ef9f68 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.075+0000 7f8facff9640 1 -- 172.21.15.172:0/4010910497 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0035070 con 0x559ad1f634b0 2024-09-10T08:47:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.075+0000 7f8facff9640 1 -- 172.21.15.172:0/4010910497 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fa004b070 con 0x559ad1f634b0 2024-09-10T08:47:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.075+0000 7f8facff9640 1 -- 172.21.15.172:0/4010910497 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0004920 con 0x559ad1f634b0 2024-09-10T08:47:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.075+0000 7f8fbfe70740 1 -- 172.21.15.172:0/4010910497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f634b0 msgr2=0x559ad1f5fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.075+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/4010910497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f634b0 0x559ad1f5fce0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0044000 tx=0x7f8fa002a3f0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.075+0000 7f8fbfe70740 1 -- 172.21.15.172:0/4010910497 shutdown_connections 2024-09-10T08:47:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/4010910497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f68540 0x559ad1f74360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/4010910497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f634b0 0x559ad1f5fce0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/4010910497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f62b70 0x559ad1f5f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fbfe70740 1 -- 172.21.15.172:0/4010910497 >> 172.21.15.172:0/4010910497 conn(0x559ad1f5e1e0 msgr2=0x559ad1f5f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fbfe70740 1 -- 172.21.15.172:0/4010910497 shutdown_connections 2024-09-10T08:47:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fbfe70740 1 -- 172.21.15.172:0/4010910497 wait complete. 2024-09-10T08:47:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fbfe70740 1 -- start start 2024-09-10T08:47:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fbfe70740 1 Processor -- start 2024-09-10T08:47:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fbfe70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f62b70 0x559ad1f720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fbfe70740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f634b0 0x559ad1f72610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fbfe70740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f68540 0x559ad1f73620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fbfe70740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb81ab680 con 0x559ad1f68540 2024-09-10T08:47:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fbfe70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb81aafb0 con 0x559ad1f62b70 2024-09-10T08:47:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fbfe70740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb81a81f0 con 0x559ad1f634b0 2024-09-10T08:47:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fbfe70740 10 client.0 _ll_register_callbacks cb 0x7f8fb00185b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fb759e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f68540 0x559ad1f73620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:11.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fb759e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f68540 0x559ad1f73620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44678/0 (socket says 172.21.15.172:44678) 2024-09-10T08:47:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fb759e640 1 -- 172.21.15.172:0/568632163 learned_addr learned my addr 172.21.15.172:0/568632163 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fb6d9d640 1 --2- 172.21.15.172:0/568632163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f62b70 0x559ad1f720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fb759e640 1 -- 172.21.15.172:0/568632163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f634b0 msgr2=0x559ad1f72610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fb759e640 1 --2- 172.21.15.172:0/568632163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f634b0 0x559ad1f72610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.076+0000 7f8fb759e640 1 -- 172.21.15.172:0/568632163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f62b70 msgr2=0x559ad1f720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8fb759e640 1 --2- 172.21.15.172:0/568632163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f62b70 0x559ad1f720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8fb759e640 1 -- 172.21.15.172:0/568632163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ad1f73c30 con 0x559ad1f68540 2024-09-10T08:47:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8fb759e640 1 --2- 172.21.15.172:0/568632163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f68540 0x559ad1f73620 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0099e90 tx=0x7f8fb009baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62051d1bf361293c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8f9e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8f9e7fc640 1 -- 172.21.15.172:0/568632163 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0086070 con 0x559ad1f68540 2024-09-10T08:47:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8f9e7fc640 1 -- 172.21.15.172:0/568632163 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8fb00a43c0 con 0x559ad1f68540 2024-09-10T08:47:11.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8fbfe70740 1 -- 172.21.15.172:0/568632163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fa004b070 con 0x559ad1f68540 2024-09-10T08:47:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8f9e7fc640 1 -- 172.21.15.172:0/568632163 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8fb00ab050 con 0x559ad1f68540 2024-09-10T08:47:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8fbfe70740 10 client.15591 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8fbfe70740 1 -- 172.21.15.172:0/568632163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559ad1fe3e20 con 0x559ad1f68540 2024-09-10T08:47:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8f9e7fc640 1 -- 172.21.15.172:0/568632163 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8fa004b070 con 0x559ad1f68540 2024-09-10T08:47:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8f9e7fc640 1 -- 172.21.15.172:0/568632163 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8fb00ba3e0 con 0x559ad1f68540 2024-09-10T08:47:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8fbfe70740 10 client.15591 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8fbfe70740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8fbfe70740 20 client.15591 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/568632163 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ad1fe4610 0x559ad1fe6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8fbfe70740 4 client.15591 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:47:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.077+0000 7f8fbfe70740 1 -- 172.21.15.172:0/568632163 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ad1ecdd70 con 0x559ad1fe4610 2024-09-10T08:47:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.078+0000 7f8fb6d9d640 1 --2- 172.21.15.172:0/568632163 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ad1fe4610 0x559ad1fe6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:11.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.078+0000 7f8fb6d9d640 1 --2- 172.21.15.172:0/568632163 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ad1fe4610 0x559ad1fe6a30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54609c5a53f0decf server_cookie=49185ea29979d40c in_seq=0 out_seq=0 2024-09-10T08:47:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.078+0000 7f8f9e7fc640 10 client.15591 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.085+0000 7f8f9e7fc640 1 -- 172.21.15.172:0/568632163 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ad1ecdd70 con 0x559ad1fe4610 2024-09-10T08:47:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.085+0000 7f8f9e7fc640 10 client.15591 handle_command_reply: tid=0 2024-09-10T08:47:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.085+0000 7f8fbfe70740 1 client.15591 shutdown 2024-09-10T08:47:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.085+0000 7f8fbfe70740 1 -- 172.21.15.172:0/568632163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f68540 msgr2=0x559ad1f73620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.085+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/568632163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f68540 0x559ad1f73620 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0099e90 tx=0x7f8fb009baa0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.085+0000 7f8fbfe70740 1 -- 172.21.15.172:0/568632163 shutdown_connections 2024-09-10T08:47:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.085+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/568632163 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ad1fe4610 0x559ad1fe6a30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.085+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/568632163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad1f68540 0x559ad1f73620 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.085+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/568632163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ad1f634b0 0x559ad1f72610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.085+0000 7f8f9e7fc640 0 client.15591 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.085+0000 7f8fbfe70740 1 --2- 172.21.15.172:0/568632163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ad1f62b70 0x559ad1f720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.085+0000 7f8fbfe70740 1 -- 172.21.15.172:0/568632163 >> 172.21.15.172:0/568632163 conn(0x559ad1f5e1e0 msgr2=0x559ad1f5d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.085+0000 7f8fbfe70740 1 -- 172.21.15.172:0/568632163 shutdown_connections 2024-09-10T08:47:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.085+0000 7f8fbfe70740 1 -- 172.21.15.172:0/568632163 wait complete. 2024-09-10T08:47:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.085+0000 7f8fbfe70740 20 client.15591 trim_cache size 0 max 16384 2024-09-10T08:47:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:47:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:47:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:47:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:47:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.086+0000 7f8fbd001640 1 -- 172.21.15.172:0/1025523049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8fa40a2c00 msgr2=0x7f8fa40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.086+0000 7f8fbd001640 1 --2- 172.21.15.172:0/1025523049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8fa40a2c00 0x7f8fa40a50c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8069fa0 tx=0x7f8fa8005f90 comp rx=0 tx=0).stop 2024-09-10T08:47:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.086+0000 7f8fbd001640 1 -- 172.21.15.172:0/1025523049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb8101ea0 msgr2=0x7f8fb819f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.086+0000 7f8fbd001640 1 --2- 172.21.15.172:0/1025523049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb8101ea0 0x7f8fb819f0e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000cce0 tx=0x7f8fb000a620 comp rx=0 tx=0).stop 2024-09-10T08:47:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.086+0000 7f8fbd001640 1 -- 172.21.15.172:0/1025523049 shutdown_connections 2024-09-10T08:47:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.086+0000 7f8fbd001640 1 --2- 172.21.15.172:0/1025523049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8fa40a2c00 0x7f8fa40a50c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.086+0000 7f8fbd001640 1 --2- 172.21.15.172:0/1025523049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb8101ea0 0x7f8fb819f0e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.086+0000 7f8fbd001640 1 --2- 172.21.15.172:0/1025523049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fb81014b0 0x7f8fb8198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.086+0000 7f8fbd001640 1 --2- 172.21.15.172:0/1025523049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8fb8100af0 0x7f8fb8197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.086+0000 7f8fbd001640 1 -- 172.21.15.172:0/1025523049 >> 172.21.15.172:0/1025523049 conn(0x7f8fb80fb790 msgr2=0x7f8fb80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.086+0000 7f8fbd001640 1 -- 172.21.15.172:0/1025523049 shutdown_connections 2024-09-10T08:47:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:11.086+0000 7f8fbd001640 1 -- 172.21.15.172:0/1025523049 wait complete. 2024-09-10T08:47:12.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:12 smithi174 ceph-mon[34068]: pgmap v379: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-10T08:47:12.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:12 smithi191 ceph-mon[35314]: pgmap v379: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-10T08:47:12.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:12 smithi172 ceph-mon[30854]: pgmap v379: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.7 KiB/s wr, 15 op/s 2024-09-10T08:47:14.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:14 smithi174 ceph-mon[34068]: pgmap v380: 129 pgs: 129 active+clean; 669 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-10T08:47:14.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:14 smithi191 ceph-mon[35314]: pgmap v380: 129 pgs: 129 active+clean; 669 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-10T08:47:14.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:14 smithi172 ceph-mon[30854]: pgmap v380: 129 pgs: 129 active+clean; 669 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-10T08:47:15.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:15 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:47:15.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:15 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:47:15.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:15 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:47:15.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:15 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:47:15.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:15 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:47:15.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:15 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:47:15.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:15 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:47:15.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:15 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:47:15.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:15 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:47:15.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:15 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:47:15.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:15 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:47:15.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:15 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:47:15.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:15 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:47:15.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:15 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:47:15.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:15 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:47:15.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:15 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:47:15.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:15 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:47:15.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:15 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:47:16.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:16 smithi191 ceph-mon[35314]: pgmap v381: 129 pgs: 129 active+clean; 669 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T08:47:16.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:16 smithi172 ceph-mon[30854]: pgmap v381: 129 pgs: 129 active+clean; 669 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T08:47:16.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:16 smithi174 ceph-mon[34068]: pgmap v381: 129 pgs: 129 active+clean; 669 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T08:47:18.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:18 smithi191 ceph-mon[35314]: pgmap v382: 129 pgs: 129 active+clean; 670 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-10T08:47:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:18 smithi172 ceph-mon[30854]: pgmap v382: 129 pgs: 129 active+clean; 670 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-10T08:47:18.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:18 smithi174 ceph-mon[34068]: pgmap v382: 129 pgs: 129 active+clean; 670 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-10T08:47:19.825 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:47:20.084 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:20.486 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:20 smithi172 ceph-mon[30854]: pgmap v383: 129 pgs: 129 active+clean; 670 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-10T08:47:20.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:20 smithi191 ceph-mon[35314]: pgmap v383: 129 pgs: 129 active+clean; 670 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-10T08:47:20.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.570+0000 7fd01b494640 1 -- 172.21.15.172:0/2896672299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd014102ef0 msgr2=0x7fd01410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:20.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.570+0000 7fd01b494640 1 --2- 172.21.15.172:0/2896672299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd014102ef0 0x7fd01410da40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009ca0 tx=0x7fd00c02a950 comp rx=0 tx=0).stop 2024-09-10T08:47:20.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.570+0000 7fd01b494640 1 -- 172.21.15.172:0/2896672299 shutdown_connections 2024-09-10T08:47:20.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.570+0000 7fd01b494640 1 --2- 172.21.15.172:0/2896672299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd014102ef0 0x7fd01410da40 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.570+0000 7fd01b494640 1 --2- 172.21.15.172:0/2896672299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd014102570 0x7fd0141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.570+0000 7fd01b494640 1 --2- 172.21.15.172:0/2896672299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd014100e20 0x7fd014101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.570+0000 7fd01b494640 1 -- 172.21.15.172:0/2896672299 >> 172.21.15.172:0/2896672299 conn(0x7fd0140fc7d0 msgr2=0x7fd0140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:20.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.570+0000 7fd01b494640 1 -- 172.21.15.172:0/2896672299 shutdown_connections 2024-09-10T08:47:20.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.570+0000 7fd01b494640 1 -- 172.21.15.172:0/2896672299 wait complete. 2024-09-10T08:47:20.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.571+0000 7fd01b494640 1 Processor -- start 2024-09-10T08:47:20.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.571+0000 7fd01b494640 1 -- start start 2024-09-10T08:47:20.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.571+0000 7fd01b494640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd014100e20 0x7fd0141972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.571+0000 7fd01b494640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd014102570 0x7fd0141977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.571+0000 7fd01b494640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd014102ef0 0x7fd01419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.571+0000 7fd01b494640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd014110e30 con 0x7fd014102ef0 2024-09-10T08:47:20.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.571+0000 7fd01b494640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd014110cb0 con 0x7fd014100e20 2024-09-10T08:47:20.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.571+0000 7fd01b494640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd014110fb0 con 0x7fd014102570 2024-09-10T08:47:20.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.571+0000 7fd019a31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd014102ef0 0x7fd01419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.571+0000 7fd019a31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd014102ef0 0x7fd01419bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60170/0 (socket says 172.21.15.172:60170) 2024-09-10T08:47:20.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.571+0000 7fd019a31640 1 -- 172.21.15.172:0/3081807540 learned_addr learned my addr 172.21.15.172:0/3081807540 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:20.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.571+0000 7fd019230640 1 --2- 172.21.15.172:0/3081807540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd014100e20 0x7fd0141972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.571+0000 7fd019a31640 1 -- 172.21.15.172:0/3081807540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd014102570 msgr2=0x7fd0141977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:20.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.571+0000 7fd019a31640 1 --2- 172.21.15.172:0/3081807540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd014102570 0x7fd0141977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.571+0000 7fd019a31640 1 -- 172.21.15.172:0/3081807540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd014100e20 msgr2=0x7fd0141972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:20.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.571+0000 7fd019a31640 1 --2- 172.21.15.172:0/3081807540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd014100e20 0x7fd0141972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.571+0000 7fd019a31640 1 -- 172.21.15.172:0/3081807540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01419c260 con 0x7fd014102ef0 2024-09-10T08:47:20.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.572+0000 7fd019230640 1 --2- 172.21.15.172:0/3081807540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd014100e20 0x7fd0141972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:20.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.572+0000 7fd019a31640 1 --2- 172.21.15.172:0/3081807540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd014102ef0 0x7fd01419bb80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd00c004af0 tx=0x7fd00c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:20.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.572+0000 7fd0067fc640 1 -- 172.21.15.172:0/3081807540 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c034040 con 0x7fd014102ef0 2024-09-10T08:47:20.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.572+0000 7fd01b494640 1 -- 172.21.15.172:0/3081807540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01419c4f0 con 0x7fd014102ef0 2024-09-10T08:47:20.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.572+0000 7fd0067fc640 1 -- 172.21.15.172:0/3081807540 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd00c0308f0 con 0x7fd014102ef0 2024-09-10T08:47:20.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.572+0000 7fd0067fc640 1 -- 172.21.15.172:0/3081807540 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c004670 con 0x7fd014102ef0 2024-09-10T08:47:20.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.572+0000 7fd01b494640 1 -- 172.21.15.172:0/3081807540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd01419c9c0 con 0x7fd014102ef0 2024-09-10T08:47:20.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.574+0000 7fd0067fc640 1 -- 172.21.15.172:0/3081807540 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd00c037020 con 0x7fd014102ef0 2024-09-10T08:47:20.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd01d86d740 1 Processor -- start 2024-09-10T08:47:20.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd01d86d740 1 -- start start 2024-09-10T08:47:20.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd0067fc640 1 --2- 172.21.15.172:0/3081807540 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd0000a2b80 0x7fd0000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd01d86d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81e54a0 0x55cfd81e5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd01d86d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 0x55cfd81f0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd01d86d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 0x55cfd81f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd01d86d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cfd814b2e0 con 0x55cfd81f0680 2024-09-10T08:47:20.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd01d86d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cfd80f2ce0 con 0x55cfd81e5dc0 2024-09-10T08:47:20.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd01d86d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55cfd81472a0 con 0x55cfd81e54a0 2024-09-10T08:47:20.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd018a2f640 1 --2- 172.21.15.172:0/3081807540 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd0000a2b80 0x7fd0000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd0067fc640 1 -- 172.21.15.172:0/3081807540 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd00c0f1c90 con 0x7fd014102ef0 2024-09-10T08:47:20.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd019230640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81e54a0 0x55cfd81e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd019a31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 0x55cfd81f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd019230640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81e54a0 0x55cfd81e5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:49116/0 (socket says 172.21.15.172:49116) 2024-09-10T08:47:20.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd019230640 1 -- 172.21.15.172:0/1776791637 learned_addr learned my addr 172.21.15.172:0/1776791637 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:20.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd018a2f640 1 --2- 172.21.15.172:0/1776791637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 0x55cfd81f0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd019a31640 1 -- 172.21.15.172:0/1776791637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81e54a0 msgr2=0x55cfd81e5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:20.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd019a31640 1 --2- 172.21.15.172:0/1776791637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81e54a0 0x55cfd81e5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd019a31640 1 -- 172.21.15.172:0/1776791637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 msgr2=0x55cfd81f0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:20.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd019a31640 1 --2- 172.21.15.172:0/1776791637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 0x55cfd81f0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd019a31640 1 -- 172.21.15.172:0/1776791637 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfd80d7440 con 0x55cfd81f0680 2024-09-10T08:47:20.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.575+0000 7fd019230640 1 --2- 172.21.15.172:0/1776791637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81e54a0 0x55cfd81e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:20.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.576+0000 7fd018a2f640 1 --2- 172.21.15.172:0/3081807540 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd0000a2b80 0x7fd0000a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd010009460 tx=0x7fd0100127c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:20.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.576+0000 7fd019a31640 1 --2- 172.21.15.172:0/1776791637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 0x55cfd81f2a70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0ba590 tx=0x7fd00c0b9dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0712d4f0ae9dbb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:20.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.576+0000 7fcfff7fe640 1 -- 172.21.15.172:0/1776791637 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c02c040 con 0x55cfd81f0680 2024-09-10T08:47:20.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.576+0000 7fcfff7fe640 1 -- 172.21.15.172:0/1776791637 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd00c0098d0 con 0x55cfd81f0680 2024-09-10T08:47:20.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.576+0000 7fcfff7fe640 1 -- 172.21.15.172:0/1776791637 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c004810 con 0x55cfd81f0680 2024-09-10T08:47:20.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.577+0000 7fd01d86d740 1 -- 172.21.15.172:0/1776791637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 msgr2=0x55cfd81f2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:20.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.577+0000 7fd01d86d740 1 --2- 172.21.15.172:0/1776791637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 0x55cfd81f2a70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0ba590 tx=0x7fd00c0b9dd0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.577+0000 7fd01d86d740 1 -- 172.21.15.172:0/1776791637 shutdown_connections 2024-09-10T08:47:20.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.577+0000 7fd01d86d740 1 --2- 172.21.15.172:0/1776791637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 0x55cfd81f2a70 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.577+0000 7fd01d86d740 1 --2- 172.21.15.172:0/1776791637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 0x55cfd81f0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.577+0000 7fd01d86d740 1 --2- 172.21.15.172:0/1776791637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81e54a0 0x55cfd81e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.577+0000 7fd01d86d740 1 -- 172.21.15.172:0/1776791637 >> 172.21.15.172:0/1776791637 conn(0x55cfd81e1a30 msgr2=0x55cfd81e1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:20.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.577+0000 7fd01d86d740 1 -- 172.21.15.172:0/1776791637 shutdown_connections 2024-09-10T08:47:20.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.577+0000 7fd01d86d740 1 -- 172.21.15.172:0/1776791637 wait complete. 2024-09-10T08:47:20.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.577+0000 7fd01d86d740 1 -- start start 2024-09-10T08:47:20.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.577+0000 7fd01d86d740 1 Processor -- start 2024-09-10T08:47:20.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd01d86d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81e54a0 0x55cfd81effd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd01d86d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 0x55cfd81ed280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd01d86d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81f0680 0x55cfd81ed7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd01d86d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cfd814b2e0 con 0x55cfd81e54a0 2024-09-10T08:47:20.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd01d86d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cfd8106050 con 0x55cfd81e5dc0 2024-09-10T08:47:20.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd01d86d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55cfd81f3010 con 0x55cfd81f0680 2024-09-10T08:47:20.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd01d86d740 10 client.0 _ll_register_callbacks cb 0x55cfd8146160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:20.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd019230640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81e54a0 0x55cfd81effd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd019230640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81e54a0 0x55cfd81effd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60194/0 (socket says 172.21.15.172:60194) 2024-09-10T08:47:20.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd019230640 1 -- 172.21.15.172:0/127964840 learned_addr learned my addr 172.21.15.172:0/127964840 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:20.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd018a2f640 1 --2- 172.21.15.172:0/127964840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 0x55cfd81ed280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd019a31640 1 --2- 172.21.15.172:0/127964840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81f0680 0x55cfd81ed7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd019230640 1 -- 172.21.15.172:0/127964840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81f0680 msgr2=0x55cfd81ed7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:20.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd019230640 1 --2- 172.21.15.172:0/127964840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81f0680 0x55cfd81ed7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd019230640 1 -- 172.21.15.172:0/127964840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 msgr2=0x55cfd81ed280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:20.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd019230640 1 --2- 172.21.15.172:0/127964840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 0x55cfd81ed280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd019230640 1 -- 172.21.15.172:0/127964840 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfd7ebc3c0 con 0x55cfd81e54a0 2024-09-10T08:47:20.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd019230640 1 --2- 172.21.15.172:0/127964840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81e54a0 0x55cfd81effd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd008010840 tx=0x7fd008010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8c1cb0e7324285 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:20.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fcffcff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:20.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fcffcff9640 1 -- 172.21.15.172:0/127964840 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00800eda0 con 0x55cfd81e54a0 2024-09-10T08:47:20.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fcffcff9640 1 -- 172.21.15.172:0/127964840 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd008005020 con 0x55cfd81e54a0 2024-09-10T08:47:20.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.578+0000 7fd01d86d740 1 -- 172.21.15.172:0/127964840 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cfd8158850 con 0x55cfd81e54a0 2024-09-10T08:47:20.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.579+0000 7fcffcff9640 1 -- 172.21.15.172:0/127964840 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd008023050 con 0x55cfd81e54a0 2024-09-10T08:47:20.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.579+0000 7fd01d86d740 10 client.15603 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:20.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.579+0000 7fd01d86d740 1 -- 172.21.15.172:0/127964840 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cfd81fe830 con 0x55cfd81e54a0 2024-09-10T08:47:20.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.579+0000 7fcffcff9640 1 -- 172.21.15.172:0/127964840 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55cfd8158850 con 0x55cfd81e54a0 2024-09-10T08:47:20.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.579+0000 7fcffcff9640 1 -- 172.21.15.172:0/127964840 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd008018430 con 0x55cfd81e54a0 2024-09-10T08:47:20.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.579+0000 7fd01d86d740 10 client.15603 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:20.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.579+0000 7fd01d86d740 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:20.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.579+0000 7fd01d86d740 20 client.15603 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:20.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.579+0000 7fd01d86d740 1 --2- 172.21.15.172:0/127964840 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cfd81ffd90 0x55cfd82021a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.579+0000 7fd01d86d740 4 client.15603 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:20.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.579+0000 7fd01d86d740 1 -- 172.21.15.172:0/127964840 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfd8152d60 con 0x55cfd81ffd90 2024-09-10T08:47:20.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.579+0000 7fd018a2f640 1 --2- 172.21.15.172:0/127964840 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cfd81ffd90 0x55cfd82021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.580+0000 7fcffcff9640 10 client.15603 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:20.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.580+0000 7fd018a2f640 1 --2- 172.21.15.172:0/127964840 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cfd81ffd90 0x55cfd82021a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef111461d861d1b0 server_cookie=241169f75642aeb1 in_seq=0 out_seq=0 2024-09-10T08:47:20.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.580+0000 7fcffcff9640 1 -- 172.21.15.172:0/127964840 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cfd8152d60 con 0x55cfd81ffd90 2024-09-10T08:47:20.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.580+0000 7fcffcff9640 10 client.15603 handle_command_reply: tid=0 2024-09-10T08:47:20.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.580+0000 7fd01d86d740 1 client.15603 shutdown 2024-09-10T08:47:20.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.581+0000 7fd01d86d740 1 -- 172.21.15.172:0/127964840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81e54a0 msgr2=0x55cfd81effd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:20.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.581+0000 7fd01d86d740 1 --2- 172.21.15.172:0/127964840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81e54a0 0x55cfd81effd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd008010840 tx=0x7fd008010d00 comp rx=0 tx=0).stop 2024-09-10T08:47:20.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.581+0000 7fd01d86d740 1 -- 172.21.15.172:0/127964840 shutdown_connections 2024-09-10T08:47:20.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.581+0000 7fd01d86d740 1 --2- 172.21.15.172:0/127964840 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cfd81ffd90 0x55cfd82021a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.581+0000 7fd01d86d740 1 --2- 172.21.15.172:0/127964840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81f0680 0x55cfd81ed7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.581+0000 7fd01d86d740 1 --2- 172.21.15.172:0/127964840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 0x55cfd81ed280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.581+0000 7fd01d86d740 1 --2- 172.21.15.172:0/127964840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81e54a0 0x55cfd81effd0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.581+0000 7fd01d86d740 1 -- 172.21.15.172:0/127964840 >> 172.21.15.172:0/127964840 conn(0x55cfd81e1a30 msgr2=0x55cfd81e7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:20.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.581+0000 7fcffcff9640 0 client.15603 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:20.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.581+0000 7fd01d86d740 1 -- 172.21.15.172:0/127964840 shutdown_connections 2024-09-10T08:47:20.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.581+0000 7fd01d86d740 1 -- 172.21.15.172:0/127964840 wait complete. 2024-09-10T08:47:20.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.581+0000 7fd01d86d740 20 client.15603 trim_cache size 0 max 16384 2024-09-10T08:47:20.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd01d86d740 1 Processor -- start 2024-09-10T08:47:20.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd01d86d740 1 -- start start 2024-09-10T08:47:20.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd01d86d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81e54a0 0x55cfd81fc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd01d86d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 0x55cfd81e1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd01d86d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 0x55cfd8207ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd01d86d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd0141ac870 con 0x55cfd81f0680 2024-09-10T08:47:20.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd01d86d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0141ae3b0 con 0x55cfd81e5dc0 2024-09-10T08:47:20.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd01d86d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd0141acf40 con 0x55cfd81e54a0 2024-09-10T08:47:20.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd019a31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 0x55cfd8207ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd019a31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 0x55cfd8207ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60204/0 (socket says 172.21.15.172:60204) 2024-09-10T08:47:20.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd019a31640 1 -- 172.21.15.172:0/3391652719 learned_addr learned my addr 172.21.15.172:0/3391652719 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:20.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd019230640 1 --2- 172.21.15.172:0/3391652719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81e54a0 0x55cfd81fc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd019a31640 1 -- 172.21.15.172:0/3391652719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81e54a0 msgr2=0x55cfd81fc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:20.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd019a31640 1 --2- 172.21.15.172:0/3391652719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81e54a0 0x55cfd81fc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd019a31640 1 -- 172.21.15.172:0/3391652719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 msgr2=0x55cfd81e1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:20.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd019a31640 1 --2- 172.21.15.172:0/3391652719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 0x55cfd81e1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd019a31640 1 -- 172.21.15.172:0/3391652719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfd81edb50 con 0x55cfd81f0680 2024-09-10T08:47:20.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.598+0000 7fd019a31640 1 --2- 172.21.15.172:0/3391652719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 0x55cfd8207ab0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd00c02ae70 tx=0x7fd00c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232001c50828f946 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:20.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.599+0000 7fcffeffd640 1 -- 172.21.15.172:0/3391652719 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c02c040 con 0x55cfd81f0680 2024-09-10T08:47:20.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.599+0000 7fcffeffd640 1 -- 172.21.15.172:0/3391652719 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd00c004810 con 0x55cfd81f0680 2024-09-10T08:47:20.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.599+0000 7fcffeffd640 1 -- 172.21.15.172:0/3391652719 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c0ef450 con 0x55cfd81f0680 2024-09-10T08:47:20.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.599+0000 7fd01d86d740 1 -- 172.21.15.172:0/3391652719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 msgr2=0x55cfd8207ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:20.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.599+0000 7fd01d86d740 1 --2- 172.21.15.172:0/3391652719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 0x55cfd8207ab0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd00c02ae70 tx=0x7fd00c005ef0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.599+0000 7fd01d86d740 1 -- 172.21.15.172:0/3391652719 shutdown_connections 2024-09-10T08:47:20.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.599+0000 7fd01d86d740 1 --2- 172.21.15.172:0/3391652719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 0x55cfd8207ab0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.599+0000 7fd01d86d740 1 --2- 172.21.15.172:0/3391652719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 0x55cfd81e1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.599+0000 7fd01d86d740 1 --2- 172.21.15.172:0/3391652719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81e54a0 0x55cfd81fc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.599+0000 7fd01d86d740 1 -- 172.21.15.172:0/3391652719 >> 172.21.15.172:0/3391652719 conn(0x55cfd81e1a30 msgr2=0x55cfd81ec370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:20.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.599+0000 7fd01d86d740 1 -- 172.21.15.172:0/3391652719 shutdown_connections 2024-09-10T08:47:20.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.599+0000 7fd01d86d740 1 -- 172.21.15.172:0/3391652719 wait complete. 2024-09-10T08:47:20.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd01d86d740 1 -- start start 2024-09-10T08:47:20.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd01d86d740 1 Processor -- start 2024-09-10T08:47:20.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd01d86d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81e54a0 0x55cfd81f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd01d86d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 0x55cfd81f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd01d86d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 0x55cfd8268a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd01d86d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd0141adce0 con 0x55cfd81f0680 2024-09-10T08:47:20.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd01d86d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0141ad610 con 0x55cfd81e5dc0 2024-09-10T08:47:20.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd01d86d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd0141aa850 con 0x55cfd81e54a0 2024-09-10T08:47:20.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd01d86d740 10 client.0 _ll_register_callbacks cb 0x7fd00800b750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:20.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd019a31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 0x55cfd8268a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd019a31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 0x55cfd8268a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60218/0 (socket says 172.21.15.172:60218) 2024-09-10T08:47:20.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd019a31640 1 -- 172.21.15.172:0/385635795 learned_addr learned my addr 172.21.15.172:0/385635795 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:20.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd019230640 1 --2- 172.21.15.172:0/385635795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81e54a0 0x55cfd81f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd019a31640 1 -- 172.21.15.172:0/385635795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81e54a0 msgr2=0x55cfd81f37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:20.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd019a31640 1 --2- 172.21.15.172:0/385635795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81e54a0 0x55cfd81f37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd019a31640 1 -- 172.21.15.172:0/385635795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 msgr2=0x55cfd81f3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:20.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd019a31640 1 --2- 172.21.15.172:0/385635795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 0x55cfd81f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.600+0000 7fd019a31640 1 -- 172.21.15.172:0/385635795 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfd8268fe0 con 0x55cfd81f0680 2024-09-10T08:47:20.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.601+0000 7fd019a31640 1 --2- 172.21.15.172:0/385635795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 0x55cfd8268a70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0be0c0 tx=0x7fd00c0c2550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc813a309e3cceb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:20.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.601+0000 7fcffcff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:20.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.601+0000 7fcffcff9640 1 -- 172.21.15.172:0/385635795 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c02c040 con 0x55cfd81f0680 2024-09-10T08:47:20.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.601+0000 7fcffcff9640 1 -- 172.21.15.172:0/385635795 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd00c0ef450 con 0x55cfd81f0680 2024-09-10T08:47:20.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.601+0000 7fd01d86d740 1 -- 172.21.15.172:0/385635795 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd00c004810 con 0x55cfd81f0680 2024-09-10T08:47:20.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.601+0000 7fcffcff9640 1 -- 172.21.15.172:0/385635795 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd00c0be130 con 0x55cfd81f0680 2024-09-10T08:47:20.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.601+0000 7fd01d86d740 10 client.15609 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:20.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.601+0000 7fd01d86d740 1 -- 172.21.15.172:0/385635795 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cfd826a2d0 con 0x55cfd81f0680 2024-09-10T08:47:20.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.601+0000 7fcffcff9640 1 -- 172.21.15.172:0/385635795 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd00c004810 con 0x55cfd81f0680 2024-09-10T08:47:20.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.601+0000 7fcffcff9640 1 -- 172.21.15.172:0/385635795 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd00c0be390 con 0x55cfd81f0680 2024-09-10T08:47:20.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.601+0000 7fd01d86d740 10 client.15609 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:20.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.601+0000 7fd01d86d740 10 client.15609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:20.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.601+0000 7fd01d86d740 20 client.15609 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:20.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.601+0000 7fd01d86d740 1 --2- 172.21.15.172:0/385635795 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cfd826a6b0 0x55cfd826caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:20.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.601+0000 7fd01d86d740 4 client.15609 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:47:20.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.601+0000 7fd01d86d740 1 -- 172.21.15.172:0/385635795 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cfd80cf210 con 0x55cfd826a6b0 2024-09-10T08:47:20.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.602+0000 7fd019230640 1 --2- 172.21.15.172:0/385635795 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cfd826a6b0 0x55cfd826caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:20.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.602+0000 7fd019230640 1 --2- 172.21.15.172:0/385635795 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cfd826a6b0 0x55cfd826caa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da3a17044e956523 server_cookie=217e444b4b72c455 in_seq=0 out_seq=0 2024-09-10T08:47:20.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.602+0000 7fcffcff9640 10 client.15609 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:20.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.602+0000 7fcffcff9640 1 -- 172.21.15.172:0/385635795 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cfd80cf210 con 0x55cfd826a6b0 2024-09-10T08:47:20.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.602+0000 7fcffcff9640 10 client.15609 handle_command_reply: tid=0 2024-09-10T08:47:20.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.602+0000 7fd01d86d740 1 client.15609 shutdown 2024-09-10T08:47:20.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.602+0000 7fd01d86d740 1 -- 172.21.15.172:0/385635795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 msgr2=0x55cfd8268a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:20.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.602+0000 7fd01d86d740 1 --2- 172.21.15.172:0/385635795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 0x55cfd8268a70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0be0c0 tx=0x7fd00c0c2550 comp rx=0 tx=0).stop 2024-09-10T08:47:20.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.602+0000 7fd01d86d740 1 -- 172.21.15.172:0/385635795 shutdown_connections 2024-09-10T08:47:20.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.602+0000 7fd01d86d740 1 --2- 172.21.15.172:0/385635795 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cfd826a6b0 0x55cfd826caa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.602+0000 7fd01d86d740 1 --2- 172.21.15.172:0/385635795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfd81f0680 0x55cfd8268a70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.602+0000 7fd01d86d740 1 --2- 172.21.15.172:0/385635795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfd81e5dc0 0x55cfd81f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.602+0000 7fd01d86d740 1 --2- 172.21.15.172:0/385635795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cfd81e54a0 0x55cfd81f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.602+0000 7fd01d86d740 1 -- 172.21.15.172:0/385635795 >> 172.21.15.172:0/385635795 conn(0x55cfd81e1a30 msgr2=0x55cfd81e8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:20.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.602+0000 7fcffcff9640 0 client.15609 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:20.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.602+0000 7fd01d86d740 1 -- 172.21.15.172:0/385635795 shutdown_connections 2024-09-10T08:47:20.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.603+0000 7fd01d86d740 1 -- 172.21.15.172:0/385635795 wait complete. 2024-09-10T08:47:20.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.603+0000 7fd01d86d740 20 client.15609 trim_cache size 0 max 16384 2024-09-10T08:47:20.609 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:47:20.609 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T08:47:20.609 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T08:47:20.609 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:47:20.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.603+0000 7fd01b494640 1 -- 172.21.15.172:0/3081807540 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd0000a2b80 msgr2=0x7fd0000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:20.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.603+0000 7fd01b494640 1 --2- 172.21.15.172:0/3081807540 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd0000a2b80 0x7fd0000a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd010009460 tx=0x7fd0100127c0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.603+0000 7fd01b494640 1 -- 172.21.15.172:0/3081807540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd014102ef0 msgr2=0x7fd01419bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:20.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.603+0000 7fd01b494640 1 --2- 172.21.15.172:0/3081807540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd014102ef0 0x7fd01419bb80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd00c004af0 tx=0x7fd00c02f5d0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.603+0000 7fd01b494640 1 -- 172.21.15.172:0/3081807540 shutdown_connections 2024-09-10T08:47:20.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.603+0000 7fd01b494640 1 --2- 172.21.15.172:0/3081807540 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd0000a2b80 0x7fd0000a5040 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.603+0000 7fd01b494640 1 --2- 172.21.15.172:0/3081807540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd014102ef0 0x7fd01419bb80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.603+0000 7fd01b494640 1 --2- 172.21.15.172:0/3081807540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd014102570 0x7fd0141977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.603+0000 7fd01b494640 1 --2- 172.21.15.172:0/3081807540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd014100e20 0x7fd0141972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:20.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.603+0000 7fd01b494640 1 -- 172.21.15.172:0/3081807540 >> 172.21.15.172:0/3081807540 conn(0x7fd0140fc7d0 msgr2=0x7fd014106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:20.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.603+0000 7fd01b494640 1 -- 172.21.15.172:0/3081807540 shutdown_connections 2024-09-10T08:47:20.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:20.603+0000 7fd01b494640 1 -- 172.21.15.172:0/3081807540 wait complete. 2024-09-10T08:47:20.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:20 smithi174 ceph-mon[34068]: pgmap v383: 129 pgs: 129 active+clean; 670 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-10T08:47:21.387 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:47:21.388 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:47:21.639 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:21.869 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T08:47:21.869 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T08:47:21.869 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:47:22.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.127+0000 7f536eb9a640 1 -- 172.21.15.172:0/3877761532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f536810c170 msgr2=0x7f536810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:22.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.127+0000 7f536eb9a640 1 --2- 172.21.15.172:0/3877761532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f536810c170 0x7f536810e610 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5360009ca0 tx=0x7f536002a950 comp rx=0 tx=0).stop 2024-09-10T08:47:22.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.127+0000 7f536eb9a640 1 -- 172.21.15.172:0/3877761532 shutdown_connections 2024-09-10T08:47:22.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.127+0000 7f536eb9a640 1 --2- 172.21.15.172:0/3877761532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f536810c170 0x7f536810e610 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:22.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.127+0000 7f536eb9a640 1 --2- 172.21.15.172:0/3877761532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5368103660 0x7f536810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:22.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.127+0000 7f536eb9a640 1 --2- 172.21.15.172:0/3877761532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5368102d40 0x7f5368103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:22.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.127+0000 7f536eb9a640 1 -- 172.21.15.172:0/3877761532 >> 172.21.15.172:0/3877761532 conn(0x7f53680fc7a0 msgr2=0x7f53680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:22.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.127+0000 7f536eb9a640 1 -- 172.21.15.172:0/3877761532 shutdown_connections 2024-09-10T08:47:22.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.127+0000 7f536eb9a640 1 -- 172.21.15.172:0/3877761532 wait complete. 2024-09-10T08:47:22.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f536eb9a640 1 Processor -- start 2024-09-10T08:47:22.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f536eb9a640 1 -- start start 2024-09-10T08:47:22.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f536eb9a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5368102d40 0x7f53681983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:22.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f536eb9a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5368103660 0x7f53681988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:22.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f536eb9a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f536810c170 0x7f536819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:22.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f536eb9a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5368111e20 con 0x7f536810c170 2024-09-10T08:47:22.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f536eb9a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5368111ca0 con 0x7f5368103660 2024-09-10T08:47:22.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f536eb9a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5368111fa0 con 0x7f5368102d40 2024-09-10T08:47:22.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f536d137640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f536810c170 0x7f536819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:22.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f536d137640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f536810c170 0x7f536819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60226/0 (socket says 172.21.15.172:60226) 2024-09-10T08:47:22.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f536d137640 1 -- 172.21.15.172:0/1010835821 learned_addr learned my addr 172.21.15.172:0/1010835821 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:22.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f536c936640 1 --2- 172.21.15.172:0/1010835821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5368102d40 0x7f53681983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:22.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f5367fff640 1 --2- 172.21.15.172:0/1010835821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5368103660 0x7f53681988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:22.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f536d137640 1 -- 172.21.15.172:0/1010835821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5368102d40 msgr2=0x7f53681983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:22.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f536d137640 1 --2- 172.21.15.172:0/1010835821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5368102d40 0x7f53681983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:22.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f536d137640 1 -- 172.21.15.172:0/1010835821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5368103660 msgr2=0x7f53681988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:22.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f536d137640 1 --2- 172.21.15.172:0/1010835821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5368103660 0x7f53681988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:22.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f536d137640 1 -- 172.21.15.172:0/1010835821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536819d350 con 0x7f536810c170 2024-09-10T08:47:22.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.128+0000 7f5367fff640 1 --2- 172.21.15.172:0/1010835821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5368103660 0x7f53681988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:22.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.129+0000 7f536d137640 1 --2- 172.21.15.172:0/1010835821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f536810c170 0x7f536819cc70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5360004af0 tx=0x7f536002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:22.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.129+0000 7f5365ffb640 1 -- 172.21.15.172:0/1010835821 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360034040 con 0x7f536810c170 2024-09-10T08:47:22.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.129+0000 7f5365ffb640 1 -- 172.21.15.172:0/1010835821 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f536002fe80 con 0x7f536810c170 2024-09-10T08:47:22.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.129+0000 7f5365ffb640 1 -- 172.21.15.172:0/1010835821 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53600046a0 con 0x7f536810c170 2024-09-10T08:47:22.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.129+0000 7f536eb9a640 1 -- 172.21.15.172:0/1010835821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536819d5e0 con 0x7f536810c170 2024-09-10T08:47:22.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.129+0000 7f536eb9a640 1 -- 172.21.15.172:0/1010835821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53681a4f20 con 0x7f536810c170 2024-09-10T08:47:22.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.130+0000 7f536eb9a640 1 -- 172.21.15.172:0/1010835821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5368103120 con 0x7f536810c170 2024-09-10T08:47:22.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.131+0000 7f5365ffb640 1 -- 172.21.15.172:0/1010835821 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5360037020 con 0x7f536810c170 2024-09-10T08:47:22.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.132+0000 7f5365ffb640 1 --2- 172.21.15.172:0/1010835821 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53580a2c00 0x7f53580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:22.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.132+0000 7f536c936640 1 --2- 172.21.15.172:0/1010835821 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53580a2c00 0x7f53580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:22.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.132+0000 7f5365ffb640 1 -- 172.21.15.172:0/1010835821 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f53600f0270 con 0x7f536810c170 2024-09-10T08:47:22.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.132+0000 7f536c936640 1 --2- 172.21.15.172:0/1010835821 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53580a2c00 0x7f53580a50c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f535c00bbe0 tx=0x7f535c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:22.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.136+0000 7f5365ffb640 1 -- 172.21.15.172:0/1010835821 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53600bce10 con 0x7f536810c170 2024-09-10T08:47:22.231 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:22.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:22 smithi172 ceph-mon[30854]: pgmap v384: 129 pgs: 129 active+clean; 670 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-10T08:47:22.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.358+0000 7f536eb9a640 1 -- 172.21.15.172:0/1010835821 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53681071e0 con 0x7f536810c170 2024-09-10T08:47:22.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.359+0000 7f5365ffb640 1 -- 172.21.15.172:0/1010835821 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7f53600bc710 con 0x7f536810c170 2024-09-10T08:47:22.360 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:47:22.360 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":29,"btime":"2024-09-10T08:46:37:242916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:46:36.267985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:47:22.360 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 29 2024-09-10T08:47:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.361+0000 7f536eb9a640 1 -- 172.21.15.172:0/1010835821 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53580a2c00 msgr2=0x7f53580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.361+0000 7f536eb9a640 1 --2- 172.21.15.172:0/1010835821 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53580a2c00 0x7f53580a50c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f535c00bbe0 tx=0x7f535c00b390 comp rx=0 tx=0).stop 2024-09-10T08:47:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.361+0000 7f536eb9a640 1 -- 172.21.15.172:0/1010835821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f536810c170 msgr2=0x7f536819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.361+0000 7f536eb9a640 1 --2- 172.21.15.172:0/1010835821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f536810c170 0x7f536819cc70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5360004af0 tx=0x7f536002f5d0 comp rx=0 tx=0).stop 2024-09-10T08:47:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.361+0000 7f536eb9a640 1 -- 172.21.15.172:0/1010835821 shutdown_connections 2024-09-10T08:47:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.361+0000 7f536eb9a640 1 --2- 172.21.15.172:0/1010835821 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53580a2c00 0x7f53580a50c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.361+0000 7f536eb9a640 1 --2- 172.21.15.172:0/1010835821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f536810c170 0x7f536819cc70 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.361+0000 7f536eb9a640 1 --2- 172.21.15.172:0/1010835821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5368103660 0x7f53681988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.361+0000 7f536eb9a640 1 --2- 172.21.15.172:0/1010835821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5368102d40 0x7f53681983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:22.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.361+0000 7f536eb9a640 1 -- 172.21.15.172:0/1010835821 >> 172.21.15.172:0/1010835821 conn(0x7f53680fc7a0 msgr2=0x7f53680fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:22.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.361+0000 7f536eb9a640 1 -- 172.21.15.172:0/1010835821 shutdown_connections 2024-09-10T08:47:22.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.361+0000 7f536eb9a640 1 -- 172.21.15.172:0/1010835821 wait complete. 2024-09-10T08:47:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:22 smithi191 ceph-mon[35314]: pgmap v384: 129 pgs: 129 active+clean; 670 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-10T08:47:22.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:22 smithi174 ceph-mon[34068]: pgmap v384: 129 pgs: 129 active+clean; 670 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-10T08:47:22.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.959+0000 7fd6cfb69640 1 -- 172.21.15.172:0/1280551410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6c81036e0 msgr2=0x7fd6c8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.959+0000 7fd6cfb69640 1 --2- 172.21.15.172:0/1280551410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6c81036e0 0x7fd6c8107b90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8009930 tx=0x7fd6b802a420 comp rx=0 tx=0).stop 2024-09-10T08:47:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.960+0000 7fd6cfb69640 1 -- 172.21.15.172:0/1280551410 shutdown_connections 2024-09-10T08:47:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.960+0000 7fd6cfb69640 1 --2- 172.21.15.172:0/1280551410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6c81082a0 0x7fd6c810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.960+0000 7fd6cfb69640 1 --2- 172.21.15.172:0/1280551410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6c81036e0 0x7fd6c8107b90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.960+0000 7fd6cfb69640 1 --2- 172.21.15.172:0/1280551410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6c8102d30 0x7fd6c8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.960+0000 7fd6cfb69640 1 -- 172.21.15.172:0/1280551410 >> 172.21.15.172:0/1280551410 conn(0x7fd6c80fc7d0 msgr2=0x7fd6c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.960+0000 7fd6cfb69640 1 -- 172.21.15.172:0/1280551410 shutdown_connections 2024-09-10T08:47:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.960+0000 7fd6cfb69640 1 -- 172.21.15.172:0/1280551410 wait complete. 2024-09-10T08:47:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.960+0000 7fd6cfb69640 1 Processor -- start 2024-09-10T08:47:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.960+0000 7fd6cfb69640 1 -- start start 2024-09-10T08:47:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6cfb69640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6c8102d30 0x7fd6c81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6cfb69640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6c81036e0 0x7fd6c8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6cfb69640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6c81082a0 0x7fd6c819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6cfb69640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c8111f50 con 0x7fd6c8102d30 2024-09-10T08:47:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6cfb69640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c8111dd0 con 0x7fd6c81082a0 2024-09-10T08:47:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6cfb69640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c81120d0 con 0x7fd6c81036e0 2024-09-10T08:47:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6cd905640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6c8102d30 0x7fd6c81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6cd905640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6c8102d30 0x7fd6c81984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60246/0 (socket says 172.21.15.172:60246) 2024-09-10T08:47:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6cd905640 1 -- 172.21.15.172:0/443932467 learned_addr learned my addr 172.21.15.172:0/443932467 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6cd104640 1 --2- 172.21.15.172:0/443932467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6c81036e0 0x7fd6c8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6ce106640 1 --2- 172.21.15.172:0/443932467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6c81082a0 0x7fd6c819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6cd905640 1 -- 172.21.15.172:0/443932467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6c81036e0 msgr2=0x7fd6c8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6cd905640 1 --2- 172.21.15.172:0/443932467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6c81036e0 0x7fd6c8198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6cd905640 1 -- 172.21.15.172:0/443932467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6c81082a0 msgr2=0x7fd6c819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6cd905640 1 --2- 172.21.15.172:0/443932467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6c81082a0 0x7fd6c819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6cd905640 1 -- 172.21.15.172:0/443932467 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c819d4a0 con 0x7fd6c8102d30 2024-09-10T08:47:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6ce106640 1 --2- 172.21.15.172:0/443932467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6c81082a0 0x7fd6c819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6cd905640 1 --2- 172.21.15.172:0/443932467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6c8102d30 0x7fd6c81984f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400e9f0 tx=0x7fd6c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6beffd640 1 -- 172.21.15.172:0/443932467 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c400cd80 con 0x7fd6c8102d30 2024-09-10T08:47:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6beffd640 1 -- 172.21.15.172:0/443932467 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6c40054d0 con 0x7fd6c8102d30 2024-09-10T08:47:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.961+0000 7fd6cfb69640 1 -- 172.21.15.172:0/443932467 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c819d790 con 0x7fd6c8102d30 2024-09-10T08:47:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.962+0000 7fd6beffd640 1 -- 172.21.15.172:0/443932467 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4010490 con 0x7fd6c8102d30 2024-09-10T08:47:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.962+0000 7fd6cfb69640 1 -- 172.21.15.172:0/443932467 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c81a5070 con 0x7fd6c8102d30 2024-09-10T08:47:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.963+0000 7fd6cfb69640 1 -- 172.21.15.172:0/443932467 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c8103110 con 0x7fd6c8102d30 2024-09-10T08:47:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.964+0000 7fd6beffd640 1 -- 172.21.15.172:0/443932467 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd6c4005020 con 0x7fd6c8102d30 2024-09-10T08:47:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.965+0000 7fd6beffd640 1 --2- 172.21.15.172:0/443932467 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd6b40a2b80 0x7fd6b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.965+0000 7fd6cd104640 1 --2- 172.21.15.172:0/443932467 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd6b40a2b80 0x7fd6b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.965+0000 7fd6beffd640 1 -- 172.21.15.172:0/443932467 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd6c40cf800 con 0x7fd6c8102d30 2024-09-10T08:47:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.965+0000 7fd6cd104640 1 --2- 172.21.15.172:0/443932467 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd6b40a2b80 0x7fd6b40a5040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8031440 tx=0x7fd6b8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:22.969+0000 7fd6beffd640 1 -- 172.21.15.172:0/443932467 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd6c409c400 con 0x7fd6c8102d30 2024-09-10T08:47:23.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:23.180+0000 7fd6cfb69640 1 -- 172.21.15.172:0/443932467 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6c810a2b0 con 0x7fd6c8102d30 2024-09-10T08:47:23.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:23.180+0000 7fd6beffd640 1 -- 172.21.15.172:0/443932467 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7fd6c409bd00 con 0x7fd6c8102d30 2024-09-10T08:47:23.181 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:47:23.181 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":29,"btime":"2024-09-10T08:46:37:242916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:46:36.267985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:47:23.182 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 29 2024-09-10T08:47:23.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:23.183+0000 7fd6cfb69640 1 -- 172.21.15.172:0/443932467 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd6b40a2b80 msgr2=0x7fd6b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:23.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:23.183+0000 7fd6cfb69640 1 --2- 172.21.15.172:0/443932467 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd6b40a2b80 0x7fd6b40a5040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8031440 tx=0x7fd6b8000bc0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:23.183+0000 7fd6cfb69640 1 -- 172.21.15.172:0/443932467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6c8102d30 msgr2=0x7fd6c81984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:23.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:23.183+0000 7fd6cfb69640 1 --2- 172.21.15.172:0/443932467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6c8102d30 0x7fd6c81984f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400e9f0 tx=0x7fd6c400edf0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:23.183+0000 7fd6cfb69640 1 -- 172.21.15.172:0/443932467 shutdown_connections 2024-09-10T08:47:23.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:23.183+0000 7fd6cfb69640 1 --2- 172.21.15.172:0/443932467 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd6b40a2b80 0x7fd6b40a5040 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:23.183+0000 7fd6cfb69640 1 --2- 172.21.15.172:0/443932467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6c81082a0 0x7fd6c819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:23.183+0000 7fd6cfb69640 1 --2- 172.21.15.172:0/443932467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6c81036e0 0x7fd6c8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:23.183+0000 7fd6cfb69640 1 --2- 172.21.15.172:0/443932467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6c8102d30 0x7fd6c81984f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:23.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:23.183+0000 7fd6cfb69640 1 -- 172.21.15.172:0/443932467 >> 172.21.15.172:0/443932467 conn(0x7fd6c80fc7d0 msgr2=0x7fd6c8105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:23.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:23.183+0000 7fd6cfb69640 1 -- 172.21.15.172:0/443932467 shutdown_connections 2024-09-10T08:47:23.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:23.183+0000 7fd6cfb69640 1 -- 172.21.15.172:0/443932467 wait complete. 2024-09-10T08:47:23.508 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:23 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1010835821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:23.508 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:23 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/443932467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:23.536 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T08:47:23.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:23 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1010835821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:23.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:23 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/443932467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:23.744 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:23 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1010835821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:23.744 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:23 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/443932467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:24.151 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T08:47:24.152 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T08:47:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.180+0000 7ff9b23fd640 1 -- 172.21.15.172:0/329295617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac1025f0 msgr2=0x7ff9ac108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.180+0000 7ff9b23fd640 1 --2- 172.21.15.172:0/329295617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac1025f0 0x7ff9ac108e80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009900 tx=0x7ff99c02a410 comp rx=0 tx=0).stop 2024-09-10T08:47:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.180+0000 7ff9b23fd640 1 -- 172.21.15.172:0/329295617 shutdown_connections 2024-09-10T08:47:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.180+0000 7ff9b23fd640 1 --2- 172.21.15.172:0/329295617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac1025f0 0x7ff9ac108e80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.180+0000 7ff9b23fd640 1 --2- 172.21.15.172:0/329295617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9ac101c30 0x7ff9ac1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.180+0000 7ff9b23fd640 1 --2- 172.21.15.172:0/329295617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac100a40 0x7ff9ac100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.180+0000 7ff9b23fd640 1 -- 172.21.15.172:0/329295617 >> 172.21.15.172:0/329295617 conn(0x7ff9ac0fc1e0 msgr2=0x7ff9ac0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.180+0000 7ff9b23fd640 1 -- 172.21.15.172:0/329295617 shutdown_connections 2024-09-10T08:47:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.180+0000 7ff9b23fd640 1 -- 172.21.15.172:0/329295617 wait complete. 2024-09-10T08:47:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9b23fd640 1 Processor -- start 2024-09-10T08:47:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9b23fd640 1 -- start start 2024-09-10T08:47:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9b23fd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9ac100a40 0x7ff9ac197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9b23fd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac101c30 0x7ff9ac198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9b23fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac1025f0 0x7ff9ac19f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9b23fd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac10c570 con 0x7ff9ac1025f0 2024-09-10T08:47:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9b23fd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac10c3f0 con 0x7ff9ac100a40 2024-09-10T08:47:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9b23fd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac10c6f0 con 0x7ff9ac101c30 2024-09-10T08:47:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9b099a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac1025f0 0x7ff9ac19f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9b099a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac1025f0 0x7ff9ac19f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60268/0 (socket says 172.21.15.172:60268) 2024-09-10T08:47:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9b099a640 1 -- 172.21.15.172:0/2678052408 learned_addr learned my addr 172.21.15.172:0/2678052408 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9abfff640 1 --2- 172.21.15.172:0/2678052408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9ac100a40 0x7ff9ac197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9a37fe640 1 --2- 172.21.15.172:0/2678052408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac101c30 0x7ff9ac198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9b099a640 1 -- 172.21.15.172:0/2678052408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac101c30 msgr2=0x7ff9ac198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9b099a640 1 --2- 172.21.15.172:0/2678052408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac101c30 0x7ff9ac198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9b099a640 1 -- 172.21.15.172:0/2678052408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9ac100a40 msgr2=0x7ff9ac197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9b099a640 1 --2- 172.21.15.172:0/2678052408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9ac100a40 0x7ff9ac197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9b099a640 1 -- 172.21.15.172:0/2678052408 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ac19f910 con 0x7ff9ac1025f0 2024-09-10T08:47:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9abfff640 1 --2- 172.21.15.172:0/2678052408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9ac100a40 0x7ff9ac197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.181+0000 7ff9b099a640 1 --2- 172.21.15.172:0/2678052408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac1025f0 0x7ff9ac19f270 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff99c005940 tx=0x7ff99c031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.182+0000 7ff9a9ffb640 1 -- 172.21.15.172:0/2678052408 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c03f030 con 0x7ff9ac1025f0 2024-09-10T08:47:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.182+0000 7ff9a9ffb640 1 -- 172.21.15.172:0/2678052408 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff99c003aa0 con 0x7ff9ac1025f0 2024-09-10T08:47:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.182+0000 7ff9b23fd640 1 -- 172.21.15.172:0/2678052408 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ac19fba0 con 0x7ff9ac1025f0 2024-09-10T08:47:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.182+0000 7ff9a9ffb640 1 -- 172.21.15.172:0/2678052408 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c035070 con 0x7ff9ac1025f0 2024-09-10T08:47:24.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.182+0000 7ff9b23fd640 1 -- 172.21.15.172:0/2678052408 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ac1a0090 con 0x7ff9ac1025f0 2024-09-10T08:47:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.184+0000 7ff9a9ffb640 1 -- 172.21.15.172:0/2678052408 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff99c042030 con 0x7ff9ac1025f0 2024-09-10T08:47:24.187 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.185+0000 7ff9a9ffb640 1 --2- 172.21.15.172:0/2678052408 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff9980a2c30 0x7ff9980a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.185+0000 7ff9abfff640 1 --2- 172.21.15.172:0/2678052408 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff9980a2c30 0x7ff9980a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.185+0000 7ff9a9ffb640 1 -- 172.21.15.172:0/2678052408 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff99c0efb40 con 0x7ff9ac1025f0 2024-09-10T08:47:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.185+0000 7ff9b527b740 1 Processor -- start 2024-09-10T08:47:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.185+0000 7ff9b527b740 1 -- start start 2024-09-10T08:47:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9b527b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b4f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9abfff640 1 --2- 172.21.15.172:0/2678052408 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff9980a2c30 0x7ff9980a50f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac1991d0 tx=0x7ff9a4009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9b527b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4f92e0 0x56151b4fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9b527b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4fe370 0x56151b500830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9b527b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56151b45c2f0 con 0x56151b4f89a0 2024-09-10T08:47:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9b527b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56151b404270 con 0x56151b4fe370 2024-09-10T08:47:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9b527b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56151b4582b0 con 0x56151b4f92e0 2024-09-10T08:47:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9abfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b4f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9abfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b4f8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60280/0 (socket says 172.21.15.172:60280) 2024-09-10T08:47:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9abfff640 1 -- 172.21.15.172:0/1612135220 learned_addr learned my addr 172.21.15.172:0/1612135220 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9a37fe640 1 --2- 172.21.15.172:0/1612135220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4f92e0 0x56151b4fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9abfff640 1 -- 172.21.15.172:0/1612135220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4f92e0 msgr2=0x56151b4fdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9abfff640 1 --2- 172.21.15.172:0/1612135220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4f92e0 0x56151b4fdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9abfff640 1 -- 172.21.15.172:0/1612135220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4fe370 msgr2=0x56151b500830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9b099a640 1 --2- 172.21.15.172:0/1612135220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4fe370 0x56151b500830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9abfff640 1 --2- 172.21.15.172:0/1612135220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4fe370 0x56151b500830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9abfff640 1 -- 172.21.15.172:0/1612135220 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56151b403340 con 0x56151b4f89a0 2024-09-10T08:47:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9a37fe640 1 --2- 172.21.15.172:0/1612135220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4f92e0 0x56151b4fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9b099a640 1 --2- 172.21.15.172:0/1612135220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4fe370 0x56151b500830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9abfff640 1 --2- 172.21.15.172:0/1612135220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b4f8da0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff9a40125e0 tx=0x7ff9a4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7ef0ab2fdcda3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.186+0000 7ff9a27fc640 1 -- 172.21.15.172:0/1612135220 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4023070 con 0x56151b4f89a0 2024-09-10T08:47:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.187+0000 7ff9a27fc640 1 -- 172.21.15.172:0/1612135220 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9a4011bd0 con 0x56151b4f89a0 2024-09-10T08:47:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.187+0000 7ff9a27fc640 1 -- 172.21.15.172:0/1612135220 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a401fd30 con 0x56151b4f89a0 2024-09-10T08:47:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.187+0000 7ff9b527b740 1 -- 172.21.15.172:0/1612135220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 msgr2=0x56151b4f8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.187+0000 7ff9b527b740 1 --2- 172.21.15.172:0/1612135220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b4f8da0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff9a40125e0 tx=0x7ff9a4013000 comp rx=0 tx=0).stop 2024-09-10T08:47:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.188+0000 7ff9b527b740 1 -- 172.21.15.172:0/1612135220 shutdown_connections 2024-09-10T08:47:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.188+0000 7ff9b527b740 1 --2- 172.21.15.172:0/1612135220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4fe370 0x56151b500830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.188+0000 7ff9b527b740 1 --2- 172.21.15.172:0/1612135220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4f92e0 0x56151b4fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.188+0000 7ff9b527b740 1 --2- 172.21.15.172:0/1612135220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b4f8da0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.188+0000 7ff9b527b740 1 -- 172.21.15.172:0/1612135220 >> 172.21.15.172:0/1612135220 conn(0x56151b4f4010 msgr2=0x56151b4f4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.188+0000 7ff9b527b740 1 -- 172.21.15.172:0/1612135220 shutdown_connections 2024-09-10T08:47:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.188+0000 7ff9b527b740 1 -- 172.21.15.172:0/1612135220 wait complete. 2024-09-10T08:47:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.188+0000 7ff9b527b740 1 -- start start 2024-09-10T08:47:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.188+0000 7ff9b527b740 1 Processor -- start 2024-09-10T08:47:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.188+0000 7ff9b527b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b502bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.188+0000 7ff9b527b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4f92e0 0x56151b503100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.188+0000 7ff9b527b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4fe370 0x56151b50a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.188+0000 7ff9b527b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56151b501340 con 0x56151b4f89a0 2024-09-10T08:47:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.188+0000 7ff9b527b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56151b3e5290 con 0x56151b4f92e0 2024-09-10T08:47:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.188+0000 7ff9b527b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56151b5014c0 con 0x56151b4fe370 2024-09-10T08:47:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.188+0000 7ff9b527b740 10 client.0 _ll_register_callbacks cb 0x56151b416140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.189+0000 7ff9abfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b502bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.189+0000 7ff9abfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b502bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60290/0 (socket says 172.21.15.172:60290) 2024-09-10T08:47:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.189+0000 7ff9abfff640 1 -- 172.21.15.172:0/1758467278 learned_addr learned my addr 172.21.15.172:0/1758467278 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:24.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.189+0000 7ff9b099a640 1 --2- 172.21.15.172:0/1758467278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4fe370 0x56151b50a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.189+0000 7ff9abfff640 1 -- 172.21.15.172:0/1758467278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4fe370 msgr2=0x56151b50a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.189+0000 7ff9abfff640 1 --2- 172.21.15.172:0/1758467278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4fe370 0x56151b50a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.189+0000 7ff9abfff640 1 -- 172.21.15.172:0/1758467278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4f92e0 msgr2=0x56151b503100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.189+0000 7ff9a37fe640 1 --2- 172.21.15.172:0/1758467278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4f92e0 0x56151b503100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.189+0000 7ff9abfff640 1 --2- 172.21.15.172:0/1758467278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4f92e0 0x56151b503100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.189+0000 7ff9abfff640 1 -- 172.21.15.172:0/1758467278 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56151b16f210 con 0x56151b4f89a0 2024-09-10T08:47:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.189+0000 7ff9b099a640 1 --2- 172.21.15.172:0/1758467278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4fe370 0x56151b50a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.190+0000 7ff9abfff640 1 --2- 172.21.15.172:0/1758467278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b502bc0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff9a401f5b0 tx=0x7ff9a401fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab5eed5d1cfef81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.190+0000 7ff98b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.190+0000 7ff98b7fe640 1 -- 172.21.15.172:0/1758467278 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4023050 con 0x56151b4f89a0 2024-09-10T08:47:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.190+0000 7ff98b7fe640 1 -- 172.21.15.172:0/1758467278 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff9a401fdf0 con 0x56151b4f89a0 2024-09-10T08:47:24.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.190+0000 7ff9b527b740 1 -- 172.21.15.172:0/1758467278 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a4011bd0 con 0x56151b4f89a0 2024-09-10T08:47:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.190+0000 7ff98b7fe640 1 -- 172.21.15.172:0/1758467278 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9a403c050 con 0x56151b4f89a0 2024-09-10T08:47:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.190+0000 7ff9b527b740 10 client.15633 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.190+0000 7ff9b527b740 1 -- 172.21.15.172:0/1758467278 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56151b23b620 con 0x56151b4f89a0 2024-09-10T08:47:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.190+0000 7ff98b7fe640 1 -- 172.21.15.172:0/1758467278 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9a4011bd0 con 0x56151b4f89a0 2024-09-10T08:47:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.190+0000 7ff98b7fe640 1 -- 172.21.15.172:0/1758467278 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff9a400f040 con 0x56151b4f89a0 2024-09-10T08:47:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.191+0000 7ff9b527b740 10 client.15633 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.191+0000 7ff9b527b740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.191+0000 7ff9b527b740 20 client.15633 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.191+0000 7ff9b527b740 1 --2- 172.21.15.172:0/1758467278 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56151b50e4c0 0x56151b5108b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.191+0000 7ff9b527b740 4 client.15633 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.191+0000 7ff9b527b740 1 -- 172.21.15.172:0/1758467278 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56151b469a20 con 0x56151b50e4c0 2024-09-10T08:47:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.191+0000 7ff9a37fe640 1 --2- 172.21.15.172:0/1758467278 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56151b50e4c0 0x56151b5108b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.191+0000 7ff98b7fe640 10 client.15633 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.191+0000 7ff9a37fe640 1 --2- 172.21.15.172:0/1758467278 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56151b50e4c0 0x56151b5108b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6aadae9bb6bce34 server_cookie=ab7b279d96e1c91d in_seq=0 out_seq=0 2024-09-10T08:47:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.192+0000 7ff98b7fe640 1 -- 172.21.15.172:0/1758467278 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56151b469a20 con 0x56151b50e4c0 2024-09-10T08:47:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.192+0000 7ff98b7fe640 10 client.15633 handle_command_reply: tid=0 2024-09-10T08:47:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.192+0000 7ff9b527b740 1 client.15633 shutdown 2024-09-10T08:47:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.192+0000 7ff9b527b740 1 -- 172.21.15.172:0/1758467278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 msgr2=0x56151b502bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.192+0000 7ff9b527b740 1 --2- 172.21.15.172:0/1758467278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b502bc0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff9a401f5b0 tx=0x7ff9a401fa70 comp rx=0 tx=0).stop 2024-09-10T08:47:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.192+0000 7ff9b527b740 1 -- 172.21.15.172:0/1758467278 shutdown_connections 2024-09-10T08:47:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.192+0000 7ff9b527b740 1 --2- 172.21.15.172:0/1758467278 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56151b50e4c0 0x56151b5108b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.192+0000 7ff9b527b740 1 --2- 172.21.15.172:0/1758467278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4fe370 0x56151b50a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.192+0000 7ff9b527b740 1 --2- 172.21.15.172:0/1758467278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4f92e0 0x56151b503100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.192+0000 7ff98b7fe640 0 client.15633 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.192+0000 7ff9b527b740 1 --2- 172.21.15.172:0/1758467278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b502bc0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.192+0000 7ff9b527b740 1 -- 172.21.15.172:0/1758467278 >> 172.21.15.172:0/1758467278 conn(0x56151b4f4010 msgr2=0x56151b4f5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.192+0000 7ff9b527b740 1 -- 172.21.15.172:0/1758467278 shutdown_connections 2024-09-10T08:47:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.193+0000 7ff9b527b740 1 -- 172.21.15.172:0/1758467278 wait complete. 2024-09-10T08:47:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.193+0000 7ff9b527b740 20 client.15633 trim_cache size 0 max 16384 2024-09-10T08:47:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.207+0000 7ff9b527b740 1 Processor -- start 2024-09-10T08:47:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.207+0000 7ff9b527b740 1 -- start start 2024-09-10T08:47:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.207+0000 7ff9b527b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b4f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.207+0000 7ff9b527b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4f92e0 0x56151b50a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.207+0000 7ff9b527b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4fe370 0x56151b502fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.207+0000 7ff9b527b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1aa500 con 0x56151b4f89a0 2024-09-10T08:47:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.207+0000 7ff9b527b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1ac040 con 0x56151b4fe370 2024-09-10T08:47:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.207+0000 7ff9b527b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1aabd0 con 0x56151b4f92e0 2024-09-10T08:47:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.207+0000 7ff9abfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b4f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.207+0000 7ff9abfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b4f39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60304/0 (socket says 172.21.15.172:60304) 2024-09-10T08:47:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.207+0000 7ff9abfff640 1 -- 172.21.15.172:0/689375873 learned_addr learned my addr 172.21.15.172:0/689375873 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.207+0000 7ff9a37fe640 1 --2- 172.21.15.172:0/689375873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4f92e0 0x56151b50a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.208+0000 7ff9b099a640 1 --2- 172.21.15.172:0/689375873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4fe370 0x56151b502fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.208+0000 7ff9abfff640 1 -- 172.21.15.172:0/689375873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4f92e0 msgr2=0x56151b50a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.208+0000 7ff9abfff640 1 --2- 172.21.15.172:0/689375873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4f92e0 0x56151b50a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.208+0000 7ff9abfff640 1 -- 172.21.15.172:0/689375873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4fe370 msgr2=0x56151b502fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.208+0000 7ff9abfff640 1 --2- 172.21.15.172:0/689375873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4fe370 0x56151b502fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.208+0000 7ff9abfff640 1 -- 172.21.15.172:0/689375873 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56151b4f9f60 con 0x56151b4f89a0 2024-09-10T08:47:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.208+0000 7ff9b099a640 1 --2- 172.21.15.172:0/689375873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4fe370 0x56151b502fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.208+0000 7ff9abfff640 1 --2- 172.21.15.172:0/689375873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b4f39b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff9a40315c0 tx=0x7ff9a40315f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9873308d8326d952 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.208+0000 7ff9a1ffb640 1 -- 172.21.15.172:0/689375873 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4016070 con 0x56151b4f89a0 2024-09-10T08:47:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.208+0000 7ff9a1ffb640 1 -- 172.21.15.172:0/689375873 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9a4003e20 con 0x56151b4f89a0 2024-09-10T08:47:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.208+0000 7ff9a1ffb640 1 -- 172.21.15.172:0/689375873 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4023040 con 0x56151b4f89a0 2024-09-10T08:47:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.209+0000 7ff9b527b740 1 -- 172.21.15.172:0/689375873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 msgr2=0x56151b4f39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.209+0000 7ff9b527b740 1 --2- 172.21.15.172:0/689375873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b4f39b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff9a40315c0 tx=0x7ff9a40315f0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.209+0000 7ff9b527b740 1 -- 172.21.15.172:0/689375873 shutdown_connections 2024-09-10T08:47:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.209+0000 7ff9b527b740 1 --2- 172.21.15.172:0/689375873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4fe370 0x56151b502fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.209+0000 7ff9b527b740 1 --2- 172.21.15.172:0/689375873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4f92e0 0x56151b50a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.209+0000 7ff9b527b740 1 --2- 172.21.15.172:0/689375873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4f89a0 0x56151b4f39b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.209+0000 7ff9b527b740 1 -- 172.21.15.172:0/689375873 >> 172.21.15.172:0/689375873 conn(0x56151b4f4010 msgr2=0x56151b4f56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.209+0000 7ff9b527b740 1 -- 172.21.15.172:0/689375873 shutdown_connections 2024-09-10T08:47:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.209+0000 7ff9b527b740 1 -- 172.21.15.172:0/689375873 wait complete. 2024-09-10T08:47:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.209+0000 7ff9b527b740 1 -- start start 2024-09-10T08:47:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.209+0000 7ff9b527b740 1 Processor -- start 2024-09-10T08:47:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.210+0000 7ff9b527b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4f89a0 0x56151b507940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.210+0000 7ff9b527b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4f92e0 0x56151b507e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.210+0000 7ff9b527b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4fe370 0x56151b508e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.210+0000 7ff9b527b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1ab970 con 0x56151b4fe370 2024-09-10T08:47:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.210+0000 7ff9b527b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1ab2a0 con 0x56151b4f92e0 2024-09-10T08:47:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.210+0000 7ff9b527b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1a84e0 con 0x56151b4f89a0 2024-09-10T08:47:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.210+0000 7ff9b527b740 10 client.0 _ll_register_callbacks cb 0x7ff9a40124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.210+0000 7ff9b099a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4fe370 0x56151b508e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.210+0000 7ff9b099a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4fe370 0x56151b508e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60308/0 (socket says 172.21.15.172:60308) 2024-09-10T08:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.210+0000 7ff9b099a640 1 -- 172.21.15.172:0/3017867052 learned_addr learned my addr 172.21.15.172:0/3017867052 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.210+0000 7ff9abfff640 1 --2- 172.21.15.172:0/3017867052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4f89a0 0x56151b507940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.210+0000 7ff9a37fe640 1 --2- 172.21.15.172:0/3017867052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4f92e0 0x56151b507e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.210+0000 7ff9abfff640 1 -- 172.21.15.172:0/3017867052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4f92e0 msgr2=0x56151b507e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.210+0000 7ff9abfff640 1 --2- 172.21.15.172:0/3017867052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4f92e0 0x56151b507e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.210+0000 7ff9abfff640 1 -- 172.21.15.172:0/3017867052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4fe370 msgr2=0x56151b508e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.210+0000 7ff9abfff640 1 --2- 172.21.15.172:0/3017867052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4fe370 0x56151b508e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.210+0000 7ff9abfff640 1 -- 172.21.15.172:0/3017867052 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56151b5094a0 con 0x56151b4f89a0 2024-09-10T08:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.211+0000 7ff9b099a640 1 --2- 172.21.15.172:0/3017867052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4fe370 0x56151b508e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.211+0000 7ff9abfff640 1 --2- 172.21.15.172:0/3017867052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4f89a0 0x56151b507940 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff9a402a040 tx=0x7ff9a40313e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9220c7dc03c1bb07 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.211+0000 7ff98b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.211+0000 7ff98b7fe640 1 -- 172.21.15.172:0/3017867052 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4044020 con 0x56151b4f89a0 2024-09-10T08:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.211+0000 7ff9b527b740 1 -- 172.21.15.172:0/3017867052 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a4003e20 con 0x56151b4f89a0 2024-09-10T08:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.211+0000 7ff98b7fe640 1 -- 172.21.15.172:0/3017867052 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff9a401f8d0 con 0x56151b4f89a0 2024-09-10T08:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.211+0000 7ff98b7fe640 1 -- 172.21.15.172:0/3017867052 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9a404f050 con 0x56151b4f89a0 2024-09-10T08:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.211+0000 7ff9b527b740 10 client.25069 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.211+0000 7ff9b527b740 1 -- 172.21.15.172:0/3017867052 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56151b579270 con 0x56151b4f89a0 2024-09-10T08:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.211+0000 7ff98b7fe640 1 -- 172.21.15.172:0/3017867052 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9a4003e20 con 0x56151b4f89a0 2024-09-10T08:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.212+0000 7ff98b7fe640 1 -- 172.21.15.172:0/3017867052 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff9a4028750 con 0x56151b4f89a0 2024-09-10T08:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.212+0000 7ff9b527b740 10 client.25069 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.212+0000 7ff9b527b740 10 client.25069 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.212+0000 7ff9b527b740 20 client.25069 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.212+0000 7ff9b527b740 1 --2- 172.21.15.172:0/3017867052 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56151b579be0 0x56151b57bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.212+0000 7ff9b527b740 4 client.25069 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:47:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.212+0000 7ff9b527b740 1 -- 172.21.15.172:0/3017867052 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56151b463d70 con 0x56151b579be0 2024-09-10T08:47:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.212+0000 7ff9a37fe640 1 --2- 172.21.15.172:0/3017867052 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56151b579be0 0x56151b57bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.212+0000 7ff9a37fe640 1 --2- 172.21.15.172:0/3017867052 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56151b579be0 0x56151b57bfd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857cde0a7bb9e301 server_cookie=800a093e25a3b4cd in_seq=0 out_seq=0 2024-09-10T08:47:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.212+0000 7ff98b7fe640 10 client.25069 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.216+0000 7ff98b7fe640 1 -- 172.21.15.172:0/3017867052 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56151b463d70 con 0x56151b579be0 2024-09-10T08:47:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.216+0000 7ff98b7fe640 10 client.25069 handle_command_reply: tid=0 2024-09-10T08:47:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.216+0000 7ff9b527b740 1 client.25069 shutdown 2024-09-10T08:47:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.216+0000 7ff9b527b740 1 -- 172.21.15.172:0/3017867052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4f89a0 msgr2=0x56151b507940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.216+0000 7ff9b527b740 1 --2- 172.21.15.172:0/3017867052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4f89a0 0x56151b507940 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff9a402a040 tx=0x7ff9a40313e0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.216+0000 7ff9b527b740 1 -- 172.21.15.172:0/3017867052 shutdown_connections 2024-09-10T08:47:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.216+0000 7ff9b527b740 1 --2- 172.21.15.172:0/3017867052 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56151b579be0 0x56151b57bfd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.216+0000 7ff9b527b740 1 --2- 172.21.15.172:0/3017867052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56151b4fe370 0x56151b508e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.216+0000 7ff9b527b740 1 --2- 172.21.15.172:0/3017867052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56151b4f92e0 0x56151b507e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.216+0000 7ff9b527b740 1 --2- 172.21.15.172:0/3017867052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56151b4f89a0 0x56151b507940 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.216+0000 7ff98b7fe640 0 client.25069 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.216+0000 7ff9b527b740 1 -- 172.21.15.172:0/3017867052 >> 172.21.15.172:0/3017867052 conn(0x56151b4f4010 msgr2=0x56151b4f29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.216+0000 7ff9b527b740 1 -- 172.21.15.172:0/3017867052 shutdown_connections 2024-09-10T08:47:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.216+0000 7ff9b527b740 1 -- 172.21.15.172:0/3017867052 wait complete. 2024-09-10T08:47:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.216+0000 7ff9b527b740 20 client.25069 trim_cache size 0 max 16384 2024-09-10T08:47:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:47:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:47:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:47:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:47:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.217+0000 7ff9b23fd640 1 -- 172.21.15.172:0/2678052408 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff9980a2c30 msgr2=0x7ff9980a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.217+0000 7ff9b23fd640 1 --2- 172.21.15.172:0/2678052408 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff9980a2c30 0x7ff9980a50f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac1991d0 tx=0x7ff9a4009520 comp rx=0 tx=0).stop 2024-09-10T08:47:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.217+0000 7ff9b23fd640 1 -- 172.21.15.172:0/2678052408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac1025f0 msgr2=0x7ff9ac19f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.217+0000 7ff9b23fd640 1 --2- 172.21.15.172:0/2678052408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac1025f0 0x7ff9ac19f270 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff99c005940 tx=0x7ff99c031a30 comp rx=0 tx=0).stop 2024-09-10T08:47:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.218+0000 7ff9b23fd640 1 -- 172.21.15.172:0/2678052408 shutdown_connections 2024-09-10T08:47:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.218+0000 7ff9b23fd640 1 --2- 172.21.15.172:0/2678052408 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff9980a2c30 0x7ff9980a50f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.218+0000 7ff9b23fd640 1 --2- 172.21.15.172:0/2678052408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9ac1025f0 0x7ff9ac19f270 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.218+0000 7ff9b23fd640 1 --2- 172.21.15.172:0/2678052408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9ac101c30 0x7ff9ac198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.218+0000 7ff9b23fd640 1 --2- 172.21.15.172:0/2678052408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9ac100a40 0x7ff9ac197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.218+0000 7ff9b23fd640 1 -- 172.21.15.172:0/2678052408 >> 172.21.15.172:0/2678052408 conn(0x7ff9ac0fc1e0 msgr2=0x7ff9ac0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.218+0000 7ff9b23fd640 1 -- 172.21.15.172:0/2678052408 shutdown_connections 2024-09-10T08:47:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:24.218+0000 7ff9b23fd640 1 -- 172.21.15.172:0/2678052408 wait complete. 2024-09-10T08:47:24.235 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:24.381 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:24 smithi172 ceph-mon[30854]: pgmap v385: 129 pgs: 129 active+clean; 670 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T08:47:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:24 smithi191 ceph-mon[35314]: pgmap v385: 129 pgs: 129 active+clean; 670 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T08:47:24.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:24 smithi174 ceph-mon[34068]: pgmap v385: 129 pgs: 129 active+clean; 670 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T08:47:24.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.725+0000 7f5768ae2640 1 -- 172.21.15.172:0/2399130484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5764103740 msgr2=0x7f5764107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.725+0000 7f5768ae2640 1 --2- 172.21.15.172:0/2399130484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5764103740 0x7f5764107b80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f574c009930 tx=0x7f574c02a420 comp rx=0 tx=0).stop 2024-09-10T08:47:24.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.725+0000 7f5768ae2640 1 -- 172.21.15.172:0/2399130484 shutdown_connections 2024-09-10T08:47:24.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.725+0000 7f5768ae2640 1 --2- 172.21.15.172:0/2399130484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5764108290 0x7f576410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.725+0000 7f5768ae2640 1 --2- 172.21.15.172:0/2399130484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5764103740 0x7f5764107b80 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.725+0000 7f5768ae2640 1 --2- 172.21.15.172:0/2399130484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5764102d90 0x7f5764103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.725+0000 7f5768ae2640 1 -- 172.21.15.172:0/2399130484 >> 172.21.15.172:0/2399130484 conn(0x7f57640fc7d0 msgr2=0x7f57640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.725+0000 7f5768ae2640 1 -- 172.21.15.172:0/2399130484 shutdown_connections 2024-09-10T08:47:24.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.725+0000 7f5768ae2640 1 -- 172.21.15.172:0/2399130484 wait complete. 2024-09-10T08:47:24.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f5768ae2640 1 Processor -- start 2024-09-10T08:47:24.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f5768ae2640 1 -- start start 2024-09-10T08:47:24.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f5768ae2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5764102d90 0x7f5764198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f5768ae2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5764103740 0x7f5764198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f5768ae2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5764108290 0x7f576419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f5768ae2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5764111f60 con 0x7f5764102d90 2024-09-10T08:47:24.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f5768ae2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5764111de0 con 0x7f5764108290 2024-09-10T08:47:24.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f5768ae2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f57641120e0 con 0x7f5764103740 2024-09-10T08:47:24.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f576259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5764102d90 0x7f5764198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f576259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5764102d90 0x7f5764198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60336/0 (socket says 172.21.15.172:60336) 2024-09-10T08:47:24.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f576259c640 1 -- 172.21.15.172:0/1658440416 learned_addr learned my addr 172.21.15.172:0/1658440416 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:24.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f5761d9b640 1 --2- 172.21.15.172:0/1658440416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5764103740 0x7f5764198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f5762d9d640 1 --2- 172.21.15.172:0/1658440416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5764108290 0x7f576419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f576259c640 1 -- 172.21.15.172:0/1658440416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5764103740 msgr2=0x7f5764198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f576259c640 1 --2- 172.21.15.172:0/1658440416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5764103740 0x7f5764198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f576259c640 1 -- 172.21.15.172:0/1658440416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5764108290 msgr2=0x7f576419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f576259c640 1 --2- 172.21.15.172:0/1658440416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5764108290 0x7f576419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.726+0000 7f576259c640 1 -- 172.21.15.172:0/1658440416 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576419d530 con 0x7f5764102d90 2024-09-10T08:47:24.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.727+0000 7f576259c640 1 --2- 172.21.15.172:0/1658440416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5764102d90 0x7f5764198580 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f575400b4a0 tx=0x7f575400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.727+0000 7f575b7fe640 1 -- 172.21.15.172:0/1658440416 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754014070 con 0x7f5764102d90 2024-09-10T08:47:24.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.727+0000 7f575b7fe640 1 -- 172.21.15.172:0/1658440416 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5754010ac0 con 0x7f5764102d90 2024-09-10T08:47:24.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.727+0000 7f5768ae2640 1 -- 172.21.15.172:0/1658440416 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576419d820 con 0x7f5764102d90 2024-09-10T08:47:24.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.727+0000 7f575b7fe640 1 -- 172.21.15.172:0/1658440416 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754017c30 con 0x7f5764102d90 2024-09-10T08:47:24.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.727+0000 7f5768ae2640 1 -- 172.21.15.172:0/1658440416 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57641a5100 con 0x7f5764102d90 2024-09-10T08:47:24.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.729+0000 7f575b7fe640 1 -- 172.21.15.172:0/1658440416 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5754017070 con 0x7f5764102d90 2024-09-10T08:47:24.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.729+0000 7f576aebb740 1 Processor -- start 2024-09-10T08:47:24.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.729+0000 7f576aebb740 1 -- start start 2024-09-10T08:47:24.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f576aebb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abfb40 0x563a12abe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f576aebb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abe5d0 0x563a12abea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f576aebb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12ac9370 0x563a12acb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f576aebb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563a12a242f0 con 0x563a12abe5d0 2024-09-10T08:47:24.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f576aebb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563a129cbcf0 con 0x563a12abfb40 2024-09-10T08:47:24.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f576aebb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563a12a202b0 con 0x563a12ac9370 2024-09-10T08:47:24.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f575b7fe640 1 --2- 172.21.15.172:0/1658440416 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57500a2b80 0x7f57500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f576259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abe5d0 0x563a12abea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f576259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abe5d0 0x563a12abea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60344/0 (socket says 172.21.15.172:60344) 2024-09-10T08:47:24.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f5762d9d640 1 --2- 172.21.15.172:0/1658440416 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57500a2b80 0x7f57500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f576259c640 1 -- 172.21.15.172:0/2585821312 learned_addr learned my addr 172.21.15.172:0/2585821312 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:24.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f575b7fe640 1 -- 172.21.15.172:0/1658440416 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f57540cea80 con 0x7f5764102d90 2024-09-10T08:47:24.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f5762d9d640 1 --2- 172.21.15.172:0/2585821312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abfb40 0x563a12abe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f5761d9b640 1 --2- 172.21.15.172:0/2585821312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12ac9370 0x563a12acb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f576259c640 1 -- 172.21.15.172:0/2585821312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12ac9370 msgr2=0x563a12acb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f576259c640 1 --2- 172.21.15.172:0/2585821312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12ac9370 0x563a12acb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f576259c640 1 -- 172.21.15.172:0/2585821312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abfb40 msgr2=0x563a12abe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f576259c640 1 --2- 172.21.15.172:0/2585821312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abfb40 0x563a12abe090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f576259c640 1 -- 172.21.15.172:0/2585821312 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a129b03a0 con 0x563a12abe5d0 2024-09-10T08:47:24.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.730+0000 7f576259c640 1 --2- 172.21.15.172:0/2585821312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abe5d0 0x563a12abea10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f575406e010 tx=0x7f575406e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ee0345987c495e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.731+0000 7f5762d9d640 1 --2- 172.21.15.172:0/1658440416 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57500a2b80 0x7f57500a5040 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f575c005c40 tx=0x7f575c00a3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.731+0000 7f5758ff9640 1 -- 172.21.15.172:0/2585821312 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575401e070 con 0x563a12abe5d0 2024-09-10T08:47:24.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.731+0000 7f5758ff9640 1 -- 172.21.15.172:0/2585821312 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f575409b680 con 0x563a12abe5d0 2024-09-10T08:47:24.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.731+0000 7f5758ff9640 1 -- 172.21.15.172:0/2585821312 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57540a26b0 con 0x563a12abe5d0 2024-09-10T08:47:24.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.731+0000 7f576aebb740 1 -- 172.21.15.172:0/2585821312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abe5d0 msgr2=0x563a12abea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.731+0000 7f576aebb740 1 --2- 172.21.15.172:0/2585821312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abe5d0 0x563a12abea10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f575406e010 tx=0x7f575406e040 comp rx=0 tx=0).stop 2024-09-10T08:47:24.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.731+0000 7f576aebb740 1 -- 172.21.15.172:0/2585821312 shutdown_connections 2024-09-10T08:47:24.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.731+0000 7f576aebb740 1 --2- 172.21.15.172:0/2585821312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12ac9370 0x563a12acb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.731+0000 7f576aebb740 1 --2- 172.21.15.172:0/2585821312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abe5d0 0x563a12abea10 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.731+0000 7f576aebb740 1 --2- 172.21.15.172:0/2585821312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abfb40 0x563a12abe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.731+0000 7f576aebb740 1 -- 172.21.15.172:0/2585821312 >> 172.21.15.172:0/2585821312 conn(0x563a12a87310 msgr2=0x563a12abac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.732+0000 7f576aebb740 1 -- 172.21.15.172:0/2585821312 shutdown_connections 2024-09-10T08:47:24.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.732+0000 7f576aebb740 1 -- 172.21.15.172:0/2585821312 wait complete. 2024-09-10T08:47:24.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.732+0000 7f576aebb740 1 -- start start 2024-09-10T08:47:24.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.732+0000 7f576aebb740 1 Processor -- start 2024-09-10T08:47:24.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.732+0000 7f576aebb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abe5d0 0x563a12ac8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.732+0000 7f576aebb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12abfb40 0x563a12ac6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.732+0000 7f576aebb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12ac9370 0x563a12ac6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.732+0000 7f576aebb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563a12acc070 con 0x563a12ac9370 2024-09-10T08:47:24.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.732+0000 7f576aebb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563a129df060 con 0x563a12abe5d0 2024-09-10T08:47:24.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.732+0000 7f576aebb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563a12acc1f0 con 0x563a12abfb40 2024-09-10T08:47:24.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.732+0000 7f576aebb740 10 client.0 _ll_register_callbacks cb 0x563a127328c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:24.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.732+0000 7f5762d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12ac9370 0x563a12ac6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.732+0000 7f5762d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12ac9370 0x563a12ac6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60354/0 (socket says 172.21.15.172:60354) 2024-09-10T08:47:24.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.732+0000 7f5762d9d640 1 -- 172.21.15.172:0/126511712 learned_addr learned my addr 172.21.15.172:0/126511712 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:24.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.732+0000 7f576259c640 1 --2- 172.21.15.172:0/126511712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abe5d0 0x563a12ac8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.733+0000 7f5761d9b640 1 --2- 172.21.15.172:0/126511712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12abfb40 0x563a12ac6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.733+0000 7f5762d9d640 1 -- 172.21.15.172:0/126511712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12abfb40 msgr2=0x563a12ac6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.733+0000 7f5762d9d640 1 --2- 172.21.15.172:0/126511712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12abfb40 0x563a12ac6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.733+0000 7f5762d9d640 1 -- 172.21.15.172:0/126511712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abe5d0 msgr2=0x563a12ac8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.733+0000 7f5762d9d640 1 --2- 172.21.15.172:0/126511712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abe5d0 0x563a12ac8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.733+0000 7f5762d9d640 1 -- 172.21.15.172:0/126511712 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a127953c0 con 0x563a12ac9370 2024-09-10T08:47:24.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.733+0000 7f5761d9b640 1 --2- 172.21.15.172:0/126511712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12abfb40 0x563a12ac6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:24.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.733+0000 7f5762d9d640 1 --2- 172.21.15.172:0/126511712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12ac9370 0x563a12ac6810 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f575c013890 tx=0x7f575c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279bf76c42b27525 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.733+0000 7f5749ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:24.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.733+0000 7f5749ffb640 1 -- 172.21.15.172:0/126511712 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575c024030 con 0x563a12ac9370 2024-09-10T08:47:24.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.733+0000 7f5749ffb640 1 -- 172.21.15.172:0/126511712 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f575c0119c0 con 0x563a12ac9370 2024-09-10T08:47:24.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.733+0000 7f576aebb740 1 -- 172.21.15.172:0/126511712 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f575409b680 con 0x563a12ac9370 2024-09-10T08:47:24.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.733+0000 7f5749ffb640 1 -- 172.21.15.172:0/126511712 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f575c011bc0 con 0x563a12ac9370 2024-09-10T08:47:24.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.733+0000 7f576aebb740 10 client.15651 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:24.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.733+0000 7f576aebb740 1 -- 172.21.15.172:0/126511712 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a12ad7b90 con 0x563a12ac9370 2024-09-10T08:47:24.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.733+0000 7f5749ffb640 1 -- 172.21.15.172:0/126511712 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f575409b680 con 0x563a12ac9370 2024-09-10T08:47:24.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.734+0000 7f5749ffb640 1 -- 172.21.15.172:0/126511712 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f575c02b4e0 con 0x563a12ac9370 2024-09-10T08:47:24.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.734+0000 7f576aebb740 10 client.15651 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:24.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.734+0000 7f576aebb740 10 client.15651 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:47:24.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.734+0000 7f576aebb740 20 client.15651 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:24.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.734+0000 7f576aebb740 1 --2- 172.21.15.172:0/126511712 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x563a12ad8fa0 0x563a12adb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.734+0000 7f576aebb740 4 client.15651 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:24.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.734+0000 7f576aebb740 1 -- 172.21.15.172:0/126511712 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a12a31860 con 0x563a12ad8fa0 2024-09-10T08:47:24.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.734+0000 7f576259c640 1 --2- 172.21.15.172:0/126511712 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x563a12ad8fa0 0x563a12adb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.734+0000 7f576259c640 1 --2- 172.21.15.172:0/126511712 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x563a12ad8fa0 0x563a12adb440 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e15e98bc8d87692 server_cookie=31b0e88b64235118 in_seq=0 out_seq=0 2024-09-10T08:47:24.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.734+0000 7f5749ffb640 10 client.15651 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:47:24.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.735+0000 7f5749ffb640 1 -- 172.21.15.172:0/126511712 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a12a31860 con 0x563a12ad8fa0 2024-09-10T08:47:24.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.735+0000 7f5749ffb640 10 client.15651 handle_command_reply: tid=0 2024-09-10T08:47:24.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.735+0000 7f576aebb740 1 client.15651 shutdown 2024-09-10T08:47:24.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.735+0000 7f576aebb740 1 -- 172.21.15.172:0/126511712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12ac9370 msgr2=0x563a12ac6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.735+0000 7f576aebb740 1 --2- 172.21.15.172:0/126511712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12ac9370 0x563a12ac6810 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f575c013890 tx=0x7f575c004ab0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.735+0000 7f576aebb740 1 -- 172.21.15.172:0/126511712 shutdown_connections 2024-09-10T08:47:24.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.735+0000 7f576aebb740 1 --2- 172.21.15.172:0/126511712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12ac9370 0x563a12ac6810 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.735+0000 7f576aebb740 1 --2- 172.21.15.172:0/126511712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12abfb40 0x563a12ac6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.735+0000 7f576aebb740 1 --2- 172.21.15.172:0/126511712 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x563a12ad8fa0 0x563a12adb440 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.735+0000 7f576aebb740 1 --2- 172.21.15.172:0/126511712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abe5d0 0x563a12ac8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.735+0000 7f5749ffb640 0 client.15651 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:47:24.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.735+0000 7f576aebb740 1 -- 172.21.15.172:0/126511712 >> 172.21.15.172:0/126511712 conn(0x563a12a87310 msgr2=0x563a12ac0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.735+0000 7f576aebb740 1 -- 172.21.15.172:0/126511712 shutdown_connections 2024-09-10T08:47:24.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.735+0000 7f576aebb740 1 -- 172.21.15.172:0/126511712 wait complete. 2024-09-10T08:47:24.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.735+0000 7f576aebb740 20 client.15651 trim_cache size 0 max 16384 2024-09-10T08:47:24.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f576aebb740 1 Processor -- start 2024-09-10T08:47:24.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f576aebb740 1 -- start start 2024-09-10T08:47:24.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f576aebb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abe5d0 0x563a12abf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f576aebb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abfb40 0x563a12ac1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f576aebb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12ac9370 0x563a12ac16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f576aebb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f57641aecd0 con 0x563a12abfb40 2024-09-10T08:47:24.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f576aebb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f57641b0810 con 0x563a12abe5d0 2024-09-10T08:47:24.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f576aebb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f57641af3a0 con 0x563a12ac9370 2024-09-10T08:47:24.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f5761d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abfb40 0x563a12ac1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f5761d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abfb40 0x563a12ac1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60368/0 (socket says 172.21.15.172:60368) 2024-09-10T08:47:24.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f5761d9b640 1 -- 172.21.15.172:0/325558379 learned_addr learned my addr 172.21.15.172:0/325558379 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:24.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f576259c640 1 --2- 172.21.15.172:0/325558379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abe5d0 0x563a12abf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f5762d9d640 1 --2- 172.21.15.172:0/325558379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12ac9370 0x563a12ac16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f5761d9b640 1 -- 172.21.15.172:0/325558379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12ac9370 msgr2=0x563a12ac16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f5761d9b640 1 --2- 172.21.15.172:0/325558379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12ac9370 0x563a12ac16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f5761d9b640 1 -- 172.21.15.172:0/325558379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abe5d0 msgr2=0x563a12abf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f5761d9b640 1 --2- 172.21.15.172:0/325558379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abe5d0 0x563a12abf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f5761d9b640 1 -- 172.21.15.172:0/325558379 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a12ac45d0 con 0x563a12abfb40 2024-09-10T08:47:24.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f5762d9d640 1 --2- 172.21.15.172:0/325558379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12ac9370 0x563a12ac16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:24.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.749+0000 7f576259c640 1 --2- 172.21.15.172:0/325558379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abe5d0 0x563a12abf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:24.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.750+0000 7f5761d9b640 1 --2- 172.21.15.172:0/325558379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abfb40 0x563a12ac1180 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f57640697d0 tx=0x7f574c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849a2e54ba48f57c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.750+0000 7f574bfff640 1 -- 172.21.15.172:0/325558379 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c035040 con 0x563a12abfb40 2024-09-10T08:47:24.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.750+0000 7f574bfff640 1 -- 172.21.15.172:0/325558379 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f574c004030 con 0x563a12abfb40 2024-09-10T08:47:24.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.750+0000 7f574bfff640 1 -- 172.21.15.172:0/325558379 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c0047a0 con 0x563a12abfb40 2024-09-10T08:47:24.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.751+0000 7f576aebb740 1 -- 172.21.15.172:0/325558379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abfb40 msgr2=0x563a12ac1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.751+0000 7f576aebb740 1 --2- 172.21.15.172:0/325558379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abfb40 0x563a12ac1180 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f57640697d0 tx=0x7f574c005fd0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.751+0000 7f576aebb740 1 -- 172.21.15.172:0/325558379 shutdown_connections 2024-09-10T08:47:24.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.751+0000 7f576aebb740 1 --2- 172.21.15.172:0/325558379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12ac9370 0x563a12ac16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.751+0000 7f576aebb740 1 --2- 172.21.15.172:0/325558379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abfb40 0x563a12ac1180 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.751+0000 7f576aebb740 1 --2- 172.21.15.172:0/325558379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abe5d0 0x563a12abf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.751+0000 7f576aebb740 1 -- 172.21.15.172:0/325558379 >> 172.21.15.172:0/325558379 conn(0x563a12a87310 msgr2=0x563a12ab5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.751+0000 7f576aebb740 1 -- 172.21.15.172:0/325558379 shutdown_connections 2024-09-10T08:47:24.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.751+0000 7f576aebb740 1 -- 172.21.15.172:0/325558379 wait complete. 2024-09-10T08:47:24.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.751+0000 7f576aebb740 1 -- start start 2024-09-10T08:47:24.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.751+0000 7f576aebb740 1 Processor -- start 2024-09-10T08:47:24.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f576aebb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abe5d0 0x563a12ad19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f576aebb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abfb40 0x563a12acccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f576aebb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12ac9370 0x563a12acd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f576aebb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f57641b0140 con 0x563a12abfb40 2024-09-10T08:47:24.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f576aebb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f57641afa70 con 0x563a12abe5d0 2024-09-10T08:47:24.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f576aebb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f57641accb0 con 0x563a12ac9370 2024-09-10T08:47:24.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f576aebb740 10 client.0 _ll_register_callbacks cb 0x7f575c013180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:24.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f5761d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abfb40 0x563a12acccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f5761d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abfb40 0x563a12acccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60382/0 (socket says 172.21.15.172:60382) 2024-09-10T08:47:24.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f5761d9b640 1 -- 172.21.15.172:0/2795024099 learned_addr learned my addr 172.21.15.172:0/2795024099 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:24.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f576259c640 1 --2- 172.21.15.172:0/2795024099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abe5d0 0x563a12ad19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f5762d9d640 1 --2- 172.21.15.172:0/2795024099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12ac9370 0x563a12acd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f5761d9b640 1 -- 172.21.15.172:0/2795024099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12ac9370 msgr2=0x563a12acd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f5761d9b640 1 --2- 172.21.15.172:0/2795024099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12ac9370 0x563a12acd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f5761d9b640 1 -- 172.21.15.172:0/2795024099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abe5d0 msgr2=0x563a12ad19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:24.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f5761d9b640 1 --2- 172.21.15.172:0/2795024099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abe5d0 0x563a12ad19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f5761d9b640 1 -- 172.21.15.172:0/2795024099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a12acd7d0 con 0x563a12abfb40 2024-09-10T08:47:24.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f5761d9b640 1 --2- 172.21.15.172:0/2795024099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abfb40 0x563a12acccd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f574c031440 tx=0x7f574c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7703f8bfef71e9d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:24.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.752+0000 7f5749ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:24.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.753+0000 7f5749ffb640 1 -- 172.21.15.172:0/2795024099 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c035030 con 0x563a12abfb40 2024-09-10T08:47:24.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.753+0000 7f5749ffb640 1 -- 172.21.15.172:0/2795024099 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f574c0043f0 con 0x563a12abfb40 2024-09-10T08:47:24.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.753+0000 7f576aebb740 1 -- 172.21.15.172:0/2795024099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f574c004030 con 0x563a12abfb40 2024-09-10T08:47:24.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.753+0000 7f5749ffb640 1 -- 172.21.15.172:0/2795024099 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f574c057050 con 0x563a12abfb40 2024-09-10T08:47:24.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.753+0000 7f576aebb740 10 client.15657 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:24.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.753+0000 7f576aebb740 1 -- 172.21.15.172:0/2795024099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a12b43b00 con 0x563a12abfb40 2024-09-10T08:47:24.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.753+0000 7f5749ffb640 1 -- 172.21.15.172:0/2795024099 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f574c004030 con 0x563a12abfb40 2024-09-10T08:47:24.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.753+0000 7f5749ffb640 1 -- 172.21.15.172:0/2795024099 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f574c0035f0 con 0x563a12abfb40 2024-09-10T08:47:24.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.753+0000 7f576aebb740 10 client.15657 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:24.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.753+0000 7f576aebb740 10 client.15657 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:47:24.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.753+0000 7f576aebb740 20 client.15657 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:24.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.753+0000 7f576aebb740 1 --2- 172.21.15.172:0/2795024099 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x563a12b44250 0x563a12b46670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:24.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.753+0000 7f576aebb740 4 client.15657 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:47:24.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.753+0000 7f576aebb740 1 -- 172.21.15.172:0/2795024099 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a12a2bd70 con 0x563a12b44250 2024-09-10T08:47:24.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.754+0000 7f576259c640 1 --2- 172.21.15.172:0/2795024099 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x563a12b44250 0x563a12b46670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:24.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.754+0000 7f576259c640 1 --2- 172.21.15.172:0/2795024099 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x563a12b44250 0x563a12b46670 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a6321719ac4f7189 server_cookie=4d2f0185dd5f5881 in_seq=0 out_seq=0 2024-09-10T08:47:24.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.754+0000 7f5749ffb640 10 client.15657 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:47:24.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.754+0000 7f5749ffb640 1 -- 172.21.15.172:0/2795024099 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a12a2bd70 con 0x563a12b44250 2024-09-10T08:47:24.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.754+0000 7f5749ffb640 10 client.15657 handle_command_reply: tid=0 2024-09-10T08:47:24.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.754+0000 7f576aebb740 1 client.15657 shutdown 2024-09-10T08:47:24.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.754+0000 7f576aebb740 1 -- 172.21.15.172:0/2795024099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abfb40 msgr2=0x563a12acccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.754+0000 7f576aebb740 1 --2- 172.21.15.172:0/2795024099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abfb40 0x563a12acccd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f574c031440 tx=0x7f574c005780 comp rx=0 tx=0).stop 2024-09-10T08:47:24.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.754+0000 7f576aebb740 1 -- 172.21.15.172:0/2795024099 shutdown_connections 2024-09-10T08:47:24.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.754+0000 7f576aebb740 1 --2- 172.21.15.172:0/2795024099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a12ac9370 0x563a12acd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.754+0000 7f576aebb740 1 --2- 172.21.15.172:0/2795024099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a12abfb40 0x563a12acccd0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.754+0000 7f576aebb740 1 --2- 172.21.15.172:0/2795024099 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x563a12b44250 0x563a12b46670 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.754+0000 7f576aebb740 1 --2- 172.21.15.172:0/2795024099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a12abe5d0 0x563a12ad19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.754+0000 7f576aebb740 1 -- 172.21.15.172:0/2795024099 >> 172.21.15.172:0/2795024099 conn(0x563a12a87310 msgr2=0x563a12ac8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.755+0000 7f5749ffb640 0 client.15657 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:47:24.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.755+0000 7f576aebb740 1 -- 172.21.15.172:0/2795024099 shutdown_connections 2024-09-10T08:47:24.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.755+0000 7f576aebb740 1 -- 172.21.15.172:0/2795024099 wait complete. 2024-09-10T08:47:24.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.755+0000 7f576aebb740 20 client.15657 trim_cache size 0 max 16384 2024-09-10T08:47:24.767 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:47:24.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.755+0000 7f5768ae2640 1 -- 172.21.15.172:0/1658440416 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57500a2b80 msgr2=0x7f57500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.755+0000 7f5768ae2640 1 --2- 172.21.15.172:0/1658440416 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57500a2b80 0x7f57500a5040 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f575c005c40 tx=0x7f575c00a3c0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.755+0000 7f5768ae2640 1 -- 172.21.15.172:0/1658440416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5764102d90 msgr2=0x7f5764198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:24.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.755+0000 7f5768ae2640 1 --2- 172.21.15.172:0/1658440416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5764102d90 0x7f5764198580 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f575400b4a0 tx=0x7f575400b960 comp rx=0 tx=0).stop 2024-09-10T08:47:24.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.756+0000 7f5768ae2640 1 -- 172.21.15.172:0/1658440416 shutdown_connections 2024-09-10T08:47:24.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.756+0000 7f5768ae2640 1 --2- 172.21.15.172:0/1658440416 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57500a2b80 0x7f57500a5040 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.756+0000 7f5768ae2640 1 --2- 172.21.15.172:0/1658440416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5764108290 0x7f576419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.756+0000 7f5768ae2640 1 --2- 172.21.15.172:0/1658440416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5764103740 0x7f5764198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.756+0000 7f5768ae2640 1 --2- 172.21.15.172:0/1658440416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5764102d90 0x7f5764198580 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:24.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.756+0000 7f5768ae2640 1 -- 172.21.15.172:0/1658440416 >> 172.21.15.172:0/1658440416 conn(0x7f57640fc7d0 msgr2=0x7f5764105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:24.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.756+0000 7f5768ae2640 1 -- 172.21.15.172:0/1658440416 shutdown_connections 2024-09-10T08:47:24.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:24.756+0000 7f5768ae2640 1 -- 172.21.15.172:0/1658440416 wait complete. 2024-09-10T08:47:25.009 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:25.618 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T08:47:26.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.123+0000 7f62cfbb0640 1 -- 172.21.15.172:0/2202562414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62c8106a60 msgr2=0x7f62c8106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.123+0000 7f62cfbb0640 1 --2- 172.21.15.172:0/2202562414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62c8106a60 0x7f62c8106e40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009930 tx=0x7f62c402a420 comp rx=0 tx=0).stop 2024-09-10T08:47:26.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.123+0000 7f62cfbb0640 1 -- 172.21.15.172:0/2202562414 shutdown_connections 2024-09-10T08:47:26.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.123+0000 7f62cfbb0640 1 --2- 172.21.15.172:0/2202562414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62c8101400 0x7f62c810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.123+0000 7f62cfbb0640 1 --2- 172.21.15.172:0/2202562414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62c8100a60 0x7f62c8100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.123+0000 7f62cfbb0640 1 --2- 172.21.15.172:0/2202562414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62c8106a60 0x7f62c8106e40 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.123+0000 7f62cfbb0640 1 -- 172.21.15.172:0/2202562414 >> 172.21.15.172:0/2202562414 conn(0x7f62c80fc780 msgr2=0x7f62c80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:26.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.124+0000 7f62cfbb0640 1 -- 172.21.15.172:0/2202562414 shutdown_connections 2024-09-10T08:47:26.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.124+0000 7f62cfbb0640 1 -- 172.21.15.172:0/2202562414 wait complete. 2024-09-10T08:47:26.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.124+0000 7f62cfbb0640 1 Processor -- start 2024-09-10T08:47:26.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.124+0000 7f62cfbb0640 1 -- start start 2024-09-10T08:47:26.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.124+0000 7f62cfbb0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62c8100a60 0x7f62c81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.124+0000 7f62cfbb0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62c8101400 0x7f62c8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.124+0000 7f62cfbb0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62c8106a60 0x7f62c819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.124+0000 7f62cfbb0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f62c8110d10 con 0x7f62c8106a60 2024-09-10T08:47:26.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62cfbb0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62c8110b90 con 0x7f62c8101400 2024-09-10T08:47:26.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62cfbb0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f62c8110e90 con 0x7f62c8100a60 2024-09-10T08:47:26.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62ce14d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62c8106a60 0x7f62c819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62cd94c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62c8100a60 0x7f62c81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62ce14d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62c8106a60 0x7f62c819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60388/0 (socket says 172.21.15.172:60388) 2024-09-10T08:47:26.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62ce14d640 1 -- 172.21.15.172:0/2043348368 learned_addr learned my addr 172.21.15.172:0/2043348368 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:26.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62cd14b640 1 --2- 172.21.15.172:0/2043348368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62c8101400 0x7f62c8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62ce14d640 1 -- 172.21.15.172:0/2043348368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62c8100a60 msgr2=0x7f62c81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62ce14d640 1 --2- 172.21.15.172:0/2043348368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62c8100a60 0x7f62c81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62ce14d640 1 -- 172.21.15.172:0/2043348368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62c8101400 msgr2=0x7f62c8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62ce14d640 1 --2- 172.21.15.172:0/2043348368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62c8101400 0x7f62c8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62ce14d640 1 -- 172.21.15.172:0/2043348368 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c819d390 con 0x7f62c8106a60 2024-09-10T08:47:26.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62ce14d640 1 --2- 172.21.15.172:0/2043348368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62c8106a60 0x7f62c819ccb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f62c000cad0 tx=0x7f62c000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:26.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62beffd640 1 -- 172.21.15.172:0/2043348368 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c000ae20 con 0x7f62c8106a60 2024-09-10T08:47:26.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62beffd640 1 -- 172.21.15.172:0/2043348368 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62c0005020 con 0x7f62c8106a60 2024-09-10T08:47:26.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62cfbb0640 1 -- 172.21.15.172:0/2043348368 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c819d680 con 0x7f62c8106a60 2024-09-10T08:47:26.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62beffd640 1 -- 172.21.15.172:0/2043348368 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c000f6b0 con 0x7f62c8106a60 2024-09-10T08:47:26.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.125+0000 7f62cfbb0640 1 -- 172.21.15.172:0/2043348368 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c81a4f60 con 0x7f62c8106a60 2024-09-10T08:47:26.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.128+0000 7f62beffd640 1 -- 172.21.15.172:0/2043348368 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f62c00051d0 con 0x7f62c8106a60 2024-09-10T08:47:26.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.128+0000 7f62d1f89740 1 Processor -- start 2024-09-10T08:47:26.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.128+0000 7f62d1f89740 1 -- start start 2024-09-10T08:47:26.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.128+0000 7f62d1f89740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400ded660 0x561400deda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.128+0000 7f62d1f89740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400dedf80 0x561400dfc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.128+0000 7f62d1f89740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400dfcd50 0x561400dff140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.128+0000 7f62d1f89740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561400d532a0 con 0x561400ded660 2024-09-10T08:47:26.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.128+0000 7f62d1f89740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561400cfab10 con 0x561400dedf80 2024-09-10T08:47:26.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.128+0000 7f62d1f89740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561400d4f260 con 0x561400dfcd50 2024-09-10T08:47:26.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62cd14b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400ded660 0x561400deda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62beffd640 1 --2- 172.21.15.172:0/2043348368 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f62b40a2c00 0x7f62b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62cd14b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400ded660 0x561400deda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60400/0 (socket says 172.21.15.172:60400) 2024-09-10T08:47:26.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62cd14b640 1 -- 172.21.15.172:0/3586687163 learned_addr learned my addr 172.21.15.172:0/3586687163 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:26.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62cd14b640 1 --2- 172.21.15.172:0/3586687163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400dfcd50 0x561400dff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62cd94c640 1 --2- 172.21.15.172:0/3586687163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400dedf80 0x561400dfc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62ce14d640 1 --2- 172.21.15.172:0/2043348368 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f62b40a2c00 0x7f62b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62cd14b640 1 -- 172.21.15.172:0/3586687163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400dfcd50 msgr2=0x561400dff140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:26.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62cd14b640 1 --2- 172.21.15.172:0/3586687163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400dfcd50 0x561400dff140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62cd14b640 1 -- 172.21.15.172:0/3586687163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400dedf80 msgr2=0x561400dfc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:26.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62cd14b640 1 --2- 172.21.15.172:0/3586687163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400dedf80 0x561400dfc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62cd14b640 1 -- 172.21.15.172:0/3586687163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561400ac4b90 con 0x561400ded660 2024-09-10T08:47:26.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62beffd640 1 -- 172.21.15.172:0/2043348368 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f62c00cf2b0 con 0x7f62c8106a60 2024-09-10T08:47:26.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62cd14b640 1 --2- 172.21.15.172:0/3586687163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400ded660 0x561400deda40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f62b800be30 tx=0x7f62b8004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a32f3ed458e7eaf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:26.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62b3fff640 1 -- 172.21.15.172:0/3586687163 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8003e80 con 0x561400ded660 2024-09-10T08:47:26.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62ce14d640 1 --2- 172.21.15.172:0/2043348368 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f62b40a2c00 0x7f62b40a50c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f62c00ccdc0 tx=0x7f62c001f4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:26.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62b3fff640 1 -- 172.21.15.172:0/3586687163 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62b80054d0 con 0x561400ded660 2024-09-10T08:47:26.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.129+0000 7f62b3fff640 1 -- 172.21.15.172:0/3586687163 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8010460 con 0x561400ded660 2024-09-10T08:47:26.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.130+0000 7f62d1f89740 1 -- 172.21.15.172:0/3586687163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400ded660 msgr2=0x561400deda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.130+0000 7f62d1f89740 1 --2- 172.21.15.172:0/3586687163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400ded660 0x561400deda40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f62b800be30 tx=0x7f62b8004250 comp rx=0 tx=0).stop 2024-09-10T08:47:26.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.130+0000 7f62d1f89740 1 -- 172.21.15.172:0/3586687163 shutdown_connections 2024-09-10T08:47:26.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.130+0000 7f62d1f89740 1 --2- 172.21.15.172:0/3586687163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400dfcd50 0x561400dff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.130+0000 7f62d1f89740 1 --2- 172.21.15.172:0/3586687163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400dedf80 0x561400dfc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.130+0000 7f62d1f89740 1 --2- 172.21.15.172:0/3586687163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400ded660 0x561400deda40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.130+0000 7f62d1f89740 1 -- 172.21.15.172:0/3586687163 >> 172.21.15.172:0/3586687163 conn(0x561400de9cf0 msgr2=0x561400dea0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:26.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.130+0000 7f62d1f89740 1 -- 172.21.15.172:0/3586687163 shutdown_connections 2024-09-10T08:47:26.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.130+0000 7f62d1f89740 1 -- 172.21.15.172:0/3586687163 wait complete. 2024-09-10T08:47:26.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.130+0000 7f62d1f89740 1 -- start start 2024-09-10T08:47:26.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.131+0000 7f62d1f89740 1 Processor -- start 2024-09-10T08:47:26.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.131+0000 7f62d1f89740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400ded660 0x561400df9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.131+0000 7f62d1f89740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400dedf80 0x561400dfa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.131+0000 7f62d1f89740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400dfcd50 0x561400dfa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.131+0000 7f62d1f89740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561400dff920 con 0x561400dfcd50 2024-09-10T08:47:26.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.131+0000 7f62d1f89740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561400cdb6e0 con 0x561400dedf80 2024-09-10T08:47:26.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.131+0000 7f62d1f89740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561400dffaa0 con 0x561400ded660 2024-09-10T08:47:26.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.131+0000 7f62d1f89740 10 client.0 _ll_register_callbacks cb 0x561400cffa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:26.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.131+0000 7f62cd14b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400dfcd50 0x561400dfa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.131+0000 7f62cd14b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400dfcd50 0x561400dfa6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60406/0 (socket says 172.21.15.172:60406) 2024-09-10T08:47:26.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.131+0000 7f62cd14b640 1 -- 172.21.15.172:0/4080205692 learned_addr learned my addr 172.21.15.172:0/4080205692 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:26.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.131+0000 7f62cd94c640 1 --2- 172.21.15.172:0/4080205692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400dedf80 0x561400dfa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.131+0000 7f62cd14b640 1 --2- 172.21.15.172:0/4080205692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400ded660 0x561400df9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.132+0000 7f62cd94c640 1 -- 172.21.15.172:0/4080205692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400ded660 msgr2=0x561400df9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:26.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.132+0000 7f62cd94c640 1 --2- 172.21.15.172:0/4080205692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400ded660 0x561400df9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.132+0000 7f62cd94c640 1 -- 172.21.15.172:0/4080205692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400dfcd50 msgr2=0x561400dfa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:26.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.132+0000 7f62cd94c640 1 --2- 172.21.15.172:0/4080205692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400dfcd50 0x561400dfa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.132+0000 7f62cd94c640 1 -- 172.21.15.172:0/4080205692 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561400e08e50 con 0x561400dedf80 2024-09-10T08:47:26.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.132+0000 7f62cd14b640 1 --2- 172.21.15.172:0/4080205692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400dfcd50 0x561400dfa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:26.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.132+0000 7f62cd94c640 1 --2- 172.21.15.172:0/4080205692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400dedf80 0x561400dfa160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f62c4031440 tx=0x7f62c40316d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f773c07042f301fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:26.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.132+0000 7f62b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:47:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.132+0000 7f62b17fa640 1 -- 172.21.15.172:0/4080205692 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4035040 con 0x561400dedf80 2024-09-10T08:47:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.132+0000 7f62b17fa640 1 -- 172.21.15.172:0/4080205692 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f62c4004040 con 0x561400dedf80 2024-09-10T08:47:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.132+0000 7f62d1f89740 1 -- 172.21.15.172:0/4080205692 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62b80054d0 con 0x561400dedf80 2024-09-10T08:47:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.132+0000 7f62b17fa640 1 -- 172.21.15.172:0/4080205692 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f62c4004240 con 0x561400dedf80 2024-09-10T08:47:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.132+0000 7f62d1f89740 10 client.25142 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.132+0000 7f62d1f89740 1 -- 172.21.15.172:0/4080205692 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561400e0b7c0 con 0x561400dedf80 2024-09-10T08:47:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.132+0000 7f62b17fa640 1 -- 172.21.15.172:0/4080205692 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62b80054d0 con 0x561400dedf80 2024-09-10T08:47:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.133+0000 7f62b17fa640 1 -- 172.21.15.172:0/4080205692 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f62c4045470 con 0x561400dedf80 2024-09-10T08:47:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.133+0000 7f62d1f89740 10 client.25142 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.133+0000 7f62d1f89740 10 client.25142 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:47:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.133+0000 7f62d1f89740 20 client.25142 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.133+0000 7f62d1f89740 1 --2- 172.21.15.172:0/4080205692 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561400e0cc80 0x561400e0f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.133+0000 7f62d1f89740 4 client.25142 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.133+0000 7f62d1f89740 1 -- 172.21.15.172:0/4080205692 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561400d609d0 con 0x561400e0cc80 2024-09-10T08:47:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.133+0000 7f62cd14b640 1 --2- 172.21.15.172:0/4080205692 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561400e0cc80 0x561400e0f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.133+0000 7f62b17fa640 10 client.25142 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.133+0000 7f62cd14b640 1 --2- 172.21.15.172:0/4080205692 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561400e0cc80 0x561400e0f070 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a452862dba9a866 server_cookie=617716131cb9c6b6 in_seq=0 out_seq=0 2024-09-10T08:47:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.134+0000 7f62b17fa640 1 -- 172.21.15.172:0/4080205692 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561400d609d0 con 0x561400e0cc80 2024-09-10T08:47:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.134+0000 7f62b17fa640 10 client.25142 handle_command_reply: tid=0 2024-09-10T08:47:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.134+0000 7f62d1f89740 1 client.25142 shutdown 2024-09-10T08:47:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.134+0000 7f62d1f89740 1 -- 172.21.15.172:0/4080205692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400dedf80 msgr2=0x561400dfa160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.134+0000 7f62d1f89740 1 --2- 172.21.15.172:0/4080205692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400dedf80 0x561400dfa160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f62c4031440 tx=0x7f62c40316d0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.134+0000 7f62d1f89740 1 -- 172.21.15.172:0/4080205692 shutdown_connections 2024-09-10T08:47:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.134+0000 7f62d1f89740 1 --2- 172.21.15.172:0/4080205692 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561400e0cc80 0x561400e0f070 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.134+0000 7f62d1f89740 1 --2- 172.21.15.172:0/4080205692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400dfcd50 0x561400dfa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.134+0000 7f62d1f89740 1 --2- 172.21.15.172:0/4080205692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400dedf80 0x561400dfa160 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.134+0000 7f62d1f89740 1 --2- 172.21.15.172:0/4080205692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400ded660 0x561400df9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.134+0000 7f62b17fa640 0 client.25142 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:26.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.134+0000 7f62d1f89740 1 -- 172.21.15.172:0/4080205692 >> 172.21.15.172:0/4080205692 conn(0x561400de9cf0 msgr2=0x561400defa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:26.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.134+0000 7f62d1f89740 1 -- 172.21.15.172:0/4080205692 shutdown_connections 2024-09-10T08:47:26.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.134+0000 7f62d1f89740 1 -- 172.21.15.172:0/4080205692 wait complete. 2024-09-10T08:47:26.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.134+0000 7f62d1f89740 20 client.25142 trim_cache size 0 max 16384 2024-09-10T08:47:26.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.153+0000 7f62d1f89740 1 Processor -- start 2024-09-10T08:47:26.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.153+0000 7f62d1f89740 1 -- start start 2024-09-10T08:47:26.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.153+0000 7f62d1f89740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400de9cf0 0x561400e14940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.153+0000 7f62d1f89740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400ded660 0x561400e14e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.153+0000 7f62d1f89740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400dedf80 0x561400df6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.153+0000 7f62d1f89740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81ac990 con 0x561400dedf80 2024-09-10T08:47:26.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.153+0000 7f62d1f89740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81ae4d0 con 0x561400de9cf0 2024-09-10T08:47:26.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.153+0000 7f62d1f89740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81ad060 con 0x561400ded660 2024-09-10T08:47:26.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.153+0000 7f62cd14b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400dedf80 0x561400df6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.153+0000 7f62cd14b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400dedf80 0x561400df6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60420/0 (socket says 172.21.15.172:60420) 2024-09-10T08:47:26.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.153+0000 7f62cd14b640 1 -- 172.21.15.172:0/555881601 learned_addr learned my addr 172.21.15.172:0/555881601 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:26.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.153+0000 7f62cd94c640 1 --2- 172.21.15.172:0/555881601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400ded660 0x561400e14e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.153+0000 7f62cd14b640 1 --2- 172.21.15.172:0/555881601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400de9cf0 0x561400e14940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.154+0000 7f62cd14b640 1 -- 172.21.15.172:0/555881601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400ded660 msgr2=0x561400e14e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:26.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.154+0000 7f62cd14b640 1 --2- 172.21.15.172:0/555881601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400ded660 0x561400e14e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.154+0000 7f62cd14b640 1 -- 172.21.15.172:0/555881601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400de9cf0 msgr2=0x561400e14940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:26.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.154+0000 7f62cd14b640 1 --2- 172.21.15.172:0/555881601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400de9cf0 0x561400e14940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.154+0000 7f62cd14b640 1 -- 172.21.15.172:0/555881601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561400deddb0 con 0x561400dedf80 2024-09-10T08:47:26.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.154+0000 7f62cd14b640 1 --2- 172.21.15.172:0/555881601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400dedf80 0x561400df6e80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f62b8003e50 tx=0x7f62b800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1cdfdd9ae756720 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:26.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.154+0000 7f62b37fe640 1 -- 172.21.15.172:0/555881601 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8004830 con 0x561400dedf80 2024-09-10T08:47:26.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.154+0000 7f62b37fe640 1 -- 172.21.15.172:0/555881601 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62b8034020 con 0x561400dedf80 2024-09-10T08:47:26.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.154+0000 7f62b37fe640 1 -- 172.21.15.172:0/555881601 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8004c30 con 0x561400dedf80 2024-09-10T08:47:26.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.155+0000 7f62d1f89740 1 -- 172.21.15.172:0/555881601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400dedf80 msgr2=0x561400df6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.155+0000 7f62d1f89740 1 --2- 172.21.15.172:0/555881601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400dedf80 0x561400df6e80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f62b8003e50 tx=0x7f62b800b960 comp rx=0 tx=0).stop 2024-09-10T08:47:26.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.155+0000 7f62d1f89740 1 -- 172.21.15.172:0/555881601 shutdown_connections 2024-09-10T08:47:26.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.155+0000 7f62d1f89740 1 --2- 172.21.15.172:0/555881601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400dedf80 0x561400df6e80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.155+0000 7f62d1f89740 1 --2- 172.21.15.172:0/555881601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400ded660 0x561400e14e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.155+0000 7f62d1f89740 1 --2- 172.21.15.172:0/555881601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400de9cf0 0x561400e14940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.155+0000 7f62d1f89740 1 -- 172.21.15.172:0/555881601 >> 172.21.15.172:0/555881601 conn(0x561400df5610 msgr2=0x561400df84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:26.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.155+0000 7f62d1f89740 1 -- 172.21.15.172:0/555881601 shutdown_connections 2024-09-10T08:47:26.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.155+0000 7f62d1f89740 1 -- 172.21.15.172:0/555881601 wait complete. 2024-09-10T08:47:26.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.155+0000 7f62d1f89740 1 -- start start 2024-09-10T08:47:26.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.155+0000 7f62d1f89740 1 Processor -- start 2024-09-10T08:47:26.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.156+0000 7f62d1f89740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400de9cf0 0x561400e05500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.156+0000 7f62d1f89740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400ded660 0x561400e00830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.156+0000 7f62d1f89740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400dedf80 0x561400e00d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.156+0000 7f62d1f89740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81ade00 con 0x561400de9cf0 2024-09-10T08:47:26.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.156+0000 7f62d1f89740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81ad730 con 0x561400ded660 2024-09-10T08:47:26.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.156+0000 7f62d1f89740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f62c81aa970 con 0x561400dedf80 2024-09-10T08:47:26.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.156+0000 7f62d1f89740 10 client.0 _ll_register_callbacks cb 0x7f62c4031ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:26.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.156+0000 7f62cd14b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400de9cf0 0x561400e05500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.156+0000 7f62cd14b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400de9cf0 0x561400e05500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57120/0 (socket says 172.21.15.172:57120) 2024-09-10T08:47:26.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.156+0000 7f62cd14b640 1 -- 172.21.15.172:0/1534774923 learned_addr learned my addr 172.21.15.172:0/1534774923 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:26.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.156+0000 7f62cd94c640 1 --2- 172.21.15.172:0/1534774923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400ded660 0x561400e00830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.156+0000 7f62cd14b640 1 --2- 172.21.15.172:0/1534774923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400dedf80 0x561400e00d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.157+0000 7f62cd14b640 1 -- 172.21.15.172:0/1534774923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400dedf80 msgr2=0x561400e00d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:26.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.157+0000 7f62cd14b640 1 --2- 172.21.15.172:0/1534774923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400dedf80 0x561400e00d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.157+0000 7f62cd14b640 1 -- 172.21.15.172:0/1534774923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400ded660 msgr2=0x561400e00830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:26.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.157+0000 7f62cd14b640 1 --2- 172.21.15.172:0/1534774923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400ded660 0x561400e00830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.157+0000 7f62cd14b640 1 -- 172.21.15.172:0/1534774923 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561400e015c0 con 0x561400de9cf0 2024-09-10T08:47:26.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.157+0000 7f62cd94c640 1 --2- 172.21.15.172:0/1534774923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400ded660 0x561400e00830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:26.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.157+0000 7f62cd14b640 1 --2- 172.21.15.172:0/1534774923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400de9cf0 0x561400e05500 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f62b8030ba0 tx=0x7f62b8005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37cddc2ef92e6f20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:26.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.157+0000 7f62b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:26.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.157+0000 7f62b17fa640 1 -- 172.21.15.172:0/1534774923 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8046060 con 0x561400de9cf0 2024-09-10T08:47:26.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.157+0000 7f62d1f89740 1 -- 172.21.15.172:0/1534774923 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62b8034020 con 0x561400de9cf0 2024-09-10T08:47:26.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.157+0000 7f62b17fa640 1 -- 172.21.15.172:0/1534774923 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f62b8030430 con 0x561400de9cf0 2024-09-10T08:47:26.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.157+0000 7f62b17fa640 1 -- 172.21.15.172:0/1534774923 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f62b804d050 con 0x561400de9cf0 2024-09-10T08:47:26.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.157+0000 7f62d1f89740 10 client.15675 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:26.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.157+0000 7f62d1f89740 1 -- 172.21.15.172:0/1534774923 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561400e77980 con 0x561400de9cf0 2024-09-10T08:47:26.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.158+0000 7f62b17fa640 1 -- 172.21.15.172:0/1534774923 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62b8034020 con 0x561400de9cf0 2024-09-10T08:47:26.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.158+0000 7f62b17fa640 1 -- 172.21.15.172:0/1534774923 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f62b8005130 con 0x561400de9cf0 2024-09-10T08:47:26.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.158+0000 7f62d1f89740 10 client.15675 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:26.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.158+0000 7f62d1f89740 10 client.15675 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:47:26.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.158+0000 7f62d1f89740 20 client.15675 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:26.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.158+0000 7f62d1f89740 1 --2- 172.21.15.172:0/1534774923 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561400dfcd50 0x561400e79e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.158+0000 7f62d1f89740 4 client.15675 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T08:47:26.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.158+0000 7f62d1f89740 1 -- 172.21.15.172:0/1534774923 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x561400d5ad20 con 0x561400dfcd50 2024-09-10T08:47:26.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.158+0000 7f62cd94c640 1 --2- 172.21.15.172:0/1534774923 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561400dfcd50 0x561400e79e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.159+0000 7f62cd94c640 1 --2- 172.21.15.172:0/1534774923 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561400dfcd50 0x561400e79e80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fbaef4e147f1cdd server_cookie=43cbd6a6e7c48bc4 in_seq=0 out_seq=0 2024-09-10T08:47:26.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.159+0000 7f62b17fa640 10 client.15675 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:26.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.166+0000 7f62b17fa640 1 -- 172.21.15.172:0/1534774923 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x561400d5ad20 con 0x561400dfcd50 2024-09-10T08:47:26.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.166+0000 7f62b17fa640 10 client.15675 handle_command_reply: tid=0 2024-09-10T08:47:26.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.166+0000 7f62d1f89740 1 client.15675 shutdown 2024-09-10T08:47:26.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.167+0000 7f62d1f89740 1 -- 172.21.15.172:0/1534774923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400de9cf0 msgr2=0x561400e05500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.167+0000 7f62d1f89740 1 --2- 172.21.15.172:0/1534774923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400de9cf0 0x561400e05500 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f62b8030ba0 tx=0x7f62b8005f30 comp rx=0 tx=0).stop 2024-09-10T08:47:26.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.167+0000 7f62d1f89740 1 -- 172.21.15.172:0/1534774923 shutdown_connections 2024-09-10T08:47:26.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.167+0000 7f62d1f89740 1 --2- 172.21.15.172:0/1534774923 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561400dfcd50 0x561400e79e80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.167+0000 7f62d1f89740 1 --2- 172.21.15.172:0/1534774923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561400dedf80 0x561400e00d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.167+0000 7f62d1f89740 1 --2- 172.21.15.172:0/1534774923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561400ded660 0x561400e00830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.167+0000 7f62d1f89740 1 --2- 172.21.15.172:0/1534774923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561400de9cf0 0x561400e05500 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.167+0000 7f62d1f89740 1 -- 172.21.15.172:0/1534774923 >> 172.21.15.172:0/1534774923 conn(0x561400df5610 msgr2=0x561400def190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:26.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.167+0000 7f62b17fa640 0 client.15675 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:26.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.167+0000 7f62d1f89740 1 -- 172.21.15.172:0/1534774923 shutdown_connections 2024-09-10T08:47:26.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.167+0000 7f62d1f89740 1 -- 172.21.15.172:0/1534774923 wait complete. 2024-09-10T08:47:26.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.167+0000 7f62d1f89740 20 client.15675 trim_cache size 0 max 16384 2024-09-10T08:47:26.169 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:47:26.169 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 29, 2024-09-10T08:47:26.169 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:47:26.169 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 2, 2024-09-10T08:47:26.169 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:47:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "2d459bfa": { 2024-09-10T08:47:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 2, 2024-09-10T08:47:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:47:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:47:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:47:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:47:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:47:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:47:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 150.0, 2024-09-10T08:47:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:47:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:47:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:47:26.171 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:47:26.171 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:47:26.171 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:47:26.171 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:47:26.171 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:47:26.171 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:47:26.171 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:47:26.171 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:47:26.171 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:47:26.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.168+0000 7f62cfbb0640 1 -- 172.21.15.172:0/2043348368 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f62b40a2c00 msgr2=0x7f62b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.168+0000 7f62cfbb0640 1 --2- 172.21.15.172:0/2043348368 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f62b40a2c00 0x7f62b40a50c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f62c00ccdc0 tx=0x7f62c001f4a0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.168+0000 7f62cfbb0640 1 -- 172.21.15.172:0/2043348368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62c8106a60 msgr2=0x7f62c819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.168+0000 7f62cfbb0640 1 --2- 172.21.15.172:0/2043348368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62c8106a60 0x7f62c819ccb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f62c000cad0 tx=0x7f62c000cf90 comp rx=0 tx=0).stop 2024-09-10T08:47:26.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.168+0000 7f62cfbb0640 1 -- 172.21.15.172:0/2043348368 shutdown_connections 2024-09-10T08:47:26.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.168+0000 7f62cfbb0640 1 --2- 172.21.15.172:0/2043348368 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f62b40a2c00 0x7f62b40a50c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.168+0000 7f62cfbb0640 1 --2- 172.21.15.172:0/2043348368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62c8106a60 0x7f62c819ccb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.168+0000 7f62cfbb0640 1 --2- 172.21.15.172:0/2043348368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62c8101400 0x7f62c8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.168+0000 7f62cfbb0640 1 --2- 172.21.15.172:0/2043348368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62c8100a60 0x7f62c81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.168+0000 7f62cfbb0640 1 -- 172.21.15.172:0/2043348368 >> 172.21.15.172:0/2043348368 conn(0x7f62c80fc780 msgr2=0x7f62c8104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:26.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.168+0000 7f62cfbb0640 1 -- 172.21.15.172:0/2043348368 shutdown_connections 2024-09-10T08:47:26.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.168+0000 7f62cfbb0640 1 -- 172.21.15.172:0/2043348368 wait complete. 2024-09-10T08:47:26.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:26 smithi172 ceph-mon[30854]: pgmap v386: 129 pgs: 129 active+clean; 670 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-10T08:47:26.332 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:26.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:26 smithi191 ceph-mon[35314]: pgmap v386: 129 pgs: 129 active+clean; 670 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-10T08:47:26.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:26 smithi174 ceph-mon[34068]: pgmap v386: 129 pgs: 129 active+clean; 670 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-10T08:47:26.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.797+0000 7fa0d88ef640 1 -- 172.21.15.172:0/2471133560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0d4100a70 msgr2=0x7fa0d410ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.797+0000 7fa0d88ef640 1 --2- 172.21.15.172:0/2471133560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0d4100a70 0x7fa0d410ea60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009900 tx=0x7fa0cc02a410 comp rx=0 tx=0).stop 2024-09-10T08:47:26.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.798+0000 7fa0d88ef640 1 -- 172.21.15.172:0/2471133560 shutdown_connections 2024-09-10T08:47:26.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.798+0000 7fa0d88ef640 1 --2- 172.21.15.172:0/2471133560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0d4100a70 0x7fa0d410ea60 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.798+0000 7fa0d88ef640 1 --2- 172.21.15.172:0/2471133560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0d41000d0 0x7fa0d4100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.798+0000 7fa0d88ef640 1 --2- 172.21.15.172:0/2471133560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d4106ba0 0x7fa0d4106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.798+0000 7fa0d88ef640 1 -- 172.21.15.172:0/2471133560 >> 172.21.15.172:0/2471133560 conn(0x7fa0d4077210 msgr2=0x7fa0d40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:26.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.798+0000 7fa0d88ef640 1 -- 172.21.15.172:0/2471133560 shutdown_connections 2024-09-10T08:47:26.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.798+0000 7fa0d88ef640 1 -- 172.21.15.172:0/2471133560 wait complete. 2024-09-10T08:47:26.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.798+0000 7fa0d88ef640 1 Processor -- start 2024-09-10T08:47:26.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.798+0000 7fa0d88ef640 1 -- start start 2024-09-10T08:47:26.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d88ef640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0d41000d0 0x7fa0d4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d88ef640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d4100a70 0x7fa0d41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d88ef640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0d4106ba0 0x7fa0d419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d88ef640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d4111f40 con 0x7fa0d4106ba0 2024-09-10T08:47:26.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d88ef640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d4111dc0 con 0x7fa0d4100a70 2024-09-10T08:47:26.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d88ef640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d41120c0 con 0x7fa0d41000d0 2024-09-10T08:47:26.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0d4106ba0 0x7fa0d419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0d4106ba0 0x7fa0d419cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57140/0 (socket says 172.21.15.172:57140) 2024-09-10T08:47:26.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d2d9d640 1 -- 172.21.15.172:0/3272643766 learned_addr learned my addr 172.21.15.172:0/3272643766 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:26.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d259c640 1 --2- 172.21.15.172:0/3272643766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0d41000d0 0x7fa0d4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d1d9b640 1 --2- 172.21.15.172:0/3272643766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d4100a70 0x7fa0d41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d2d9d640 1 -- 172.21.15.172:0/3272643766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0d41000d0 msgr2=0x7fa0d4198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d2d9d640 1 --2- 172.21.15.172:0/3272643766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0d41000d0 0x7fa0d4198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d2d9d640 1 -- 172.21.15.172:0/3272643766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d4100a70 msgr2=0x7fa0d41989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d2d9d640 1 --2- 172.21.15.172:0/3272643766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d4100a70 0x7fa0d41989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d2d9d640 1 -- 172.21.15.172:0/3272643766 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d419d410 con 0x7fa0d4106ba0 2024-09-10T08:47:26.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d1d9b640 1 --2- 172.21.15.172:0/3272643766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d4100a70 0x7fa0d41989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:26.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0d2d9d640 1 --2- 172.21.15.172:0/3272643766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0d4106ba0 0x7fa0d419cd30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009230 tx=0x7fa0cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:26.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.799+0000 7fa0c37fe640 1 -- 172.21.15.172:0/3272643766 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc03f030 con 0x7fa0d4106ba0 2024-09-10T08:47:26.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.800+0000 7fa0d88ef640 1 -- 172.21.15.172:0/3272643766 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d419d6a0 con 0x7fa0d4106ba0 2024-09-10T08:47:26.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.800+0000 7fa0c37fe640 1 -- 172.21.15.172:0/3272643766 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0cc003800 con 0x7fa0d4106ba0 2024-09-10T08:47:26.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.800+0000 7fa0c37fe640 1 -- 172.21.15.172:0/3272643766 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc035070 con 0x7fa0d4106ba0 2024-09-10T08:47:26.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.800+0000 7fa0d88ef640 1 -- 172.21.15.172:0/3272643766 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d41a4fe0 con 0x7fa0d4106ba0 2024-09-10T08:47:26.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.802+0000 7fa0c37fe640 1 -- 172.21.15.172:0/3272643766 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa0cc0039b0 con 0x7fa0d4106ba0 2024-09-10T08:47:26.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.802+0000 7fa0dacc8740 1 Processor -- start 2024-09-10T08:47:26.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.802+0000 7fa0dacc8740 1 -- start start 2024-09-10T08:47:26.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.802+0000 7fa0dacc8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0dab40 0x5627db0d9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.802+0000 7fa0dacc8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0d95d0 0x5627db0d9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.802+0000 7fa0dacc8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0e4370 0x5627db0e6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.802+0000 7fa0dacc8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5627db03f2f0 con 0x5627db0e4370 2024-09-10T08:47:26.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.802+0000 7fa0dacc8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5627dafe6cf0 con 0x5627db0dab40 2024-09-10T08:47:26.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.802+0000 7fa0dacc8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5627db03b2b0 con 0x5627db0d95d0 2024-09-10T08:47:26.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.803+0000 7fa0c37fe640 1 --2- 172.21.15.172:0/3272643766 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0bc0a2b80 0x7fa0bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.803+0000 7fa0d1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0e4370 0x5627db0e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.803+0000 7fa0d259c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0d95d0 0x5627db0d9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.803+0000 7fa0d1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0e4370 0x5627db0e6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57150/0 (socket says 172.21.15.172:57150) 2024-09-10T08:47:26.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.803+0000 7fa0d1d9b640 1 -- 172.21.15.172:0/3138802514 learned_addr learned my addr 172.21.15.172:0/3138802514 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:26.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.803+0000 7fa0d1d9b640 1 --2- 172.21.15.172:0/3138802514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0dab40 0x5627db0d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.803+0000 7fa0d2d9d640 1 --2- 172.21.15.172:0/3272643766 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0bc0a2b80 0x7fa0bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.803+0000 7fa0c37fe640 1 -- 172.21.15.172:0/3272643766 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa0cc0efca0 con 0x7fa0d4106ba0 2024-09-10T08:47:26.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.803+0000 7fa0d259c640 1 -- 172.21.15.172:0/3138802514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0dab40 msgr2=0x5627db0d9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:26.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.803+0000 7fa0d259c640 1 --2- 172.21.15.172:0/3138802514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0dab40 0x5627db0d9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.803+0000 7fa0d259c640 1 -- 172.21.15.172:0/3138802514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0e4370 msgr2=0x5627db0e6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:26.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.803+0000 7fa0d259c640 1 --2- 172.21.15.172:0/3138802514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0e4370 0x5627db0e6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.803+0000 7fa0d259c640 1 -- 172.21.15.172:0/3138802514 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627dafcb3a0 con 0x5627db0d95d0 2024-09-10T08:47:26.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.803+0000 7fa0d1d9b640 1 --2- 172.21.15.172:0/3138802514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0e4370 0x5627db0e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:26.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.803+0000 7fa0d259c640 1 --2- 172.21.15.172:0/3138802514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0d95d0 0x5627db0d9a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4005c40 tx=0x7fa0c400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f0de91f98522b5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:26.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.803+0000 7fa0d2d9d640 1 --2- 172.21.15.172:0/3272643766 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0bc0a2b80 0x7fa0bc0a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0b8b70 tx=0x7fa0cc0b1680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:26.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.803+0000 7fa0c0ff9640 1 -- 172.21.15.172:0/3138802514 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4003b70 con 0x5627db0d95d0 2024-09-10T08:47:26.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.804+0000 7fa0c0ff9640 1 -- 172.21.15.172:0/3138802514 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0c4003d10 con 0x5627db0d95d0 2024-09-10T08:47:26.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.804+0000 7fa0c0ff9640 1 -- 172.21.15.172:0/3138802514 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4010b00 con 0x5627db0d95d0 2024-09-10T08:47:26.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.804+0000 7fa0dacc8740 1 -- 172.21.15.172:0/3138802514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0d95d0 msgr2=0x5627db0d9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.804+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/3138802514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0d95d0 0x5627db0d9a10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4005c40 tx=0x7fa0c400f790 comp rx=0 tx=0).stop 2024-09-10T08:47:26.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.804+0000 7fa0dacc8740 1 -- 172.21.15.172:0/3138802514 shutdown_connections 2024-09-10T08:47:26.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.804+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/3138802514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0e4370 0x5627db0e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.804+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/3138802514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0d95d0 0x5627db0d9a10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.804+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/3138802514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0dab40 0x5627db0d9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.804+0000 7fa0dacc8740 1 -- 172.21.15.172:0/3138802514 >> 172.21.15.172:0/3138802514 conn(0x5627db0a2310 msgr2=0x5627db0d5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:26.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.804+0000 7fa0dacc8740 1 -- 172.21.15.172:0/3138802514 shutdown_connections 2024-09-10T08:47:26.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.804+0000 7fa0dacc8740 1 -- 172.21.15.172:0/3138802514 wait complete. 2024-09-10T08:47:26.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0dacc8740 1 -- start start 2024-09-10T08:47:26.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0dacc8740 1 Processor -- start 2024-09-10T08:47:26.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0dacc8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0d95d0 0x5627db0e3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0dacc8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0dab40 0x5627db0e0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0dacc8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0e4370 0x5627db0e14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0dacc8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5627db0e7070 con 0x5627db0d95d0 2024-09-10T08:47:26.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0dacc8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5627daffa060 con 0x5627db0dab40 2024-09-10T08:47:26.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0dacc8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5627db0e71f0 con 0x5627db0e4370 2024-09-10T08:47:26.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0dacc8740 10 client.0 _ll_register_callbacks cb 0x5627dad4d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:26.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0d1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0d95d0 0x5627db0e3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0d1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0d95d0 0x5627db0e3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57152/0 (socket says 172.21.15.172:57152) 2024-09-10T08:47:26.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0d1d9b640 1 -- 172.21.15.172:0/1210043610 learned_addr learned my addr 172.21.15.172:0/1210043610 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:26.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0d1d9b640 1 --2- 172.21.15.172:0/1210043610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0e4370 0x5627db0e14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0d259c640 1 --2- 172.21.15.172:0/1210043610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0dab40 0x5627db0e0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0d1d9b640 1 -- 172.21.15.172:0/1210043610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0e4370 msgr2=0x5627db0e14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:26.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0d1d9b640 1 --2- 172.21.15.172:0/1210043610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0e4370 0x5627db0e14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0d1d9b640 1 -- 172.21.15.172:0/1210043610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0dab40 msgr2=0x5627db0e0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:26.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0d1d9b640 1 --2- 172.21.15.172:0/1210043610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0dab40 0x5627db0e0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0d1d9b640 1 -- 172.21.15.172:0/1210043610 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627dadb03c0 con 0x5627db0d95d0 2024-09-10T08:47:26.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.805+0000 7fa0d259c640 1 --2- 172.21.15.172:0/1210043610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0dab40 0x5627db0e0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:26.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.806+0000 7fa0d1d9b640 1 --2- 172.21.15.172:0/1210043610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0d95d0 0x5627db0e3ce0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8013ed0 tx=0x7fa0c8015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c331b9075e1169bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:26.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.806+0000 7fa0b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:26.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.806+0000 7fa0b9ffb640 1 -- 172.21.15.172:0/1210043610 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c80130d0 con 0x5627db0d95d0 2024-09-10T08:47:26.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.806+0000 7fa0b9ffb640 1 -- 172.21.15.172:0/1210043610 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa0c8004020 con 0x5627db0d95d0 2024-09-10T08:47:26.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.806+0000 7fa0dacc8740 1 -- 172.21.15.172:0/1210043610 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c4003d10 con 0x5627db0d95d0 2024-09-10T08:47:26.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.806+0000 7fa0b9ffb640 1 -- 172.21.15.172:0/1210043610 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa0c8026050 con 0x5627db0d95d0 2024-09-10T08:47:26.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.806+0000 7fa0dacc8740 10 client.15687 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:26.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.806+0000 7fa0dacc8740 1 -- 172.21.15.172:0/1210043610 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5627db0f2ae0 con 0x5627db0d95d0 2024-09-10T08:47:26.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.806+0000 7fa0b9ffb640 1 -- 172.21.15.172:0/1210043610 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0c4003d10 con 0x5627db0d95d0 2024-09-10T08:47:26.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.806+0000 7fa0b9ffb640 1 -- 172.21.15.172:0/1210043610 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa0c801f060 con 0x5627db0d95d0 2024-09-10T08:47:26.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.806+0000 7fa0dacc8740 10 client.15687 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:26.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.806+0000 7fa0dacc8740 10 client.15687 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:47:26.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.806+0000 7fa0dacc8740 20 client.15687 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:26.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.806+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/1210043610 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5627db0f3ff0 0x5627db0f6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.806+0000 7fa0dacc8740 4 client.15687 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:26.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.806+0000 7fa0dacc8740 1 -- 172.21.15.172:0/1210043610 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627db04c860 con 0x5627db0f3ff0 2024-09-10T08:47:26.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.807+0000 7fa0d259c640 1 --2- 172.21.15.172:0/1210043610 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5627db0f3ff0 0x5627db0f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.807+0000 7fa0b9ffb640 10 client.15687 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:47:26.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.807+0000 7fa0d259c640 1 --2- 172.21.15.172:0/1210043610 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5627db0f3ff0 0x5627db0f6450 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=da8e4ae47d780439 server_cookie=cab52eb24f47f5de in_seq=0 out_seq=0 2024-09-10T08:47:26.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.808+0000 7fa0b9ffb640 1 -- 172.21.15.172:0/1210043610 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627db04c860 con 0x5627db0f3ff0 2024-09-10T08:47:26.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.808+0000 7fa0b9ffb640 10 client.15687 handle_command_reply: tid=0 2024-09-10T08:47:26.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.808+0000 7fa0dacc8740 1 client.15687 shutdown 2024-09-10T08:47:26.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.808+0000 7fa0dacc8740 1 -- 172.21.15.172:0/1210043610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0d95d0 msgr2=0x5627db0e3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.808+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/1210043610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0d95d0 0x5627db0e3ce0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8013ed0 tx=0x7fa0c8015040 comp rx=0 tx=0).stop 2024-09-10T08:47:26.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.808+0000 7fa0dacc8740 1 -- 172.21.15.172:0/1210043610 shutdown_connections 2024-09-10T08:47:26.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.808+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/1210043610 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5627db0f3ff0 0x5627db0f6450 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.808+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/1210043610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0e4370 0x5627db0e14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.808+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/1210043610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0dab40 0x5627db0e0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.808+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/1210043610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0d95d0 0x5627db0e3ce0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.808+0000 7fa0b9ffb640 0 client.15687 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:47:26.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.808+0000 7fa0dacc8740 1 -- 172.21.15.172:0/1210043610 >> 172.21.15.172:0/1210043610 conn(0x5627db0a2310 msgr2=0x5627db0db960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:26.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.808+0000 7fa0dacc8740 1 -- 172.21.15.172:0/1210043610 shutdown_connections 2024-09-10T08:47:26.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.808+0000 7fa0dacc8740 1 -- 172.21.15.172:0/1210043610 wait complete. 2024-09-10T08:47:26.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.808+0000 7fa0dacc8740 20 client.15687 trim_cache size 0 max 16384 2024-09-10T08:47:26.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0dacc8740 1 Processor -- start 2024-09-10T08:47:26.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0dacc8740 1 -- start start 2024-09-10T08:47:26.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0dacc8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0d95d0 0x5627db0d4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0dacc8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0dab40 0x5627db0d4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0dacc8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0e4370 0x5627db0e5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0dacc8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d41aeb50 con 0x5627db0dab40 2024-09-10T08:47:26.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0dacc8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d41b0690 con 0x5627db0e4370 2024-09-10T08:47:26.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0dacc8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d41af220 con 0x5627db0d95d0 2024-09-10T08:47:26.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0d259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0dab40 0x5627db0d4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0d259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0dab40 0x5627db0d4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57162/0 (socket says 172.21.15.172:57162) 2024-09-10T08:47:26.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0d259c640 1 -- 172.21.15.172:0/2631991663 learned_addr learned my addr 172.21.15.172:0/2631991663 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:26.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0d1d9b640 1 --2- 172.21.15.172:0/2631991663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0d95d0 0x5627db0d4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0d259c640 1 -- 172.21.15.172:0/2631991663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0d95d0 msgr2=0x5627db0d4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:26.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0d259c640 1 --2- 172.21.15.172:0/2631991663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0d95d0 0x5627db0d4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0d259c640 1 -- 172.21.15.172:0/2631991663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0e4370 msgr2=0x5627db0e5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:26.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0d259c640 1 --2- 172.21.15.172:0/2631991663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0e4370 0x5627db0e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0d259c640 1 -- 172.21.15.172:0/2631991663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627db0e6ba0 con 0x5627db0dab40 2024-09-10T08:47:26.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0d1d9b640 1 --2- 172.21.15.172:0/2631991663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0d95d0 0x5627db0d4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:26.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.821+0000 7fa0d259c640 1 --2- 172.21.15.172:0/2631991663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0dab40 0x5627db0d4f60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4005c40 tx=0x7fa0c400ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2649320985c3fdd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:26.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.822+0000 7fa0bbfff640 1 -- 172.21.15.172:0/2631991663 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c40049e0 con 0x5627db0dab40 2024-09-10T08:47:26.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.822+0000 7fa0bbfff640 1 -- 172.21.15.172:0/2631991663 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0c4029070 con 0x5627db0dab40 2024-09-10T08:47:26.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.822+0000 7fa0bbfff640 1 -- 172.21.15.172:0/2631991663 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c40052a0 con 0x5627db0dab40 2024-09-10T08:47:26.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.822+0000 7fa0dacc8740 1 -- 172.21.15.172:0/2631991663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0dab40 msgr2=0x5627db0d4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.822+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/2631991663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0dab40 0x5627db0d4f60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4005c40 tx=0x7fa0c400ffa0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.822+0000 7fa0dacc8740 1 -- 172.21.15.172:0/2631991663 shutdown_connections 2024-09-10T08:47:26.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.822+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/2631991663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0e4370 0x5627db0e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.822+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/2631991663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0dab40 0x5627db0d4f60 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.822+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/2631991663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0d95d0 0x5627db0d4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.822+0000 7fa0dacc8740 1 -- 172.21.15.172:0/2631991663 >> 172.21.15.172:0/2631991663 conn(0x5627db0a2310 msgr2=0x5627db0d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:26.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.822+0000 7fa0dacc8740 1 -- 172.21.15.172:0/2631991663 shutdown_connections 2024-09-10T08:47:26.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.822+0000 7fa0dacc8740 1 -- 172.21.15.172:0/2631991663 wait complete. 2024-09-10T08:47:26.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.822+0000 7fa0dacc8740 1 -- start start 2024-09-10T08:47:26.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0dacc8740 1 Processor -- start 2024-09-10T08:47:26.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0dacc8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0d95d0 0x5627db0debc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0dacc8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0dab40 0x5627db0ec480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0dacc8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0e4370 0x5627db0ec9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0dacc8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d41affc0 con 0x5627db0e4370 2024-09-10T08:47:26.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0dacc8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d41af8f0 con 0x5627db0d95d0 2024-09-10T08:47:26.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0dacc8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d41acb30 con 0x5627db0dab40 2024-09-10T08:47:26.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0dacc8740 10 client.0 _ll_register_callbacks cb 0x7fa0c8013d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:26.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0d1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0e4370 0x5627db0ec9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0d1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0e4370 0x5627db0ec9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57170/0 (socket says 172.21.15.172:57170) 2024-09-10T08:47:26.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0d1d9b640 1 -- 172.21.15.172:0/2580580594 learned_addr learned my addr 172.21.15.172:0/2580580594 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:26.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0d259c640 1 --2- 172.21.15.172:0/2580580594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0dab40 0x5627db0ec480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0d1d9b640 1 -- 172.21.15.172:0/2580580594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0dab40 msgr2=0x5627db0ec480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:26.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0d1d9b640 1 --2- 172.21.15.172:0/2580580594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0dab40 0x5627db0ec480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0d1d9b640 1 -- 172.21.15.172:0/2580580594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0d95d0 msgr2=0x5627db0debc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:26.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0d1d9b640 1 --2- 172.21.15.172:0/2580580594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0d95d0 0x5627db0debc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0d1d9b640 1 -- 172.21.15.172:0/2580580594 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627db0fb830 con 0x5627db0e4370 2024-09-10T08:47:26.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0d1d9b640 1 --2- 172.21.15.172:0/2580580594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0e4370 0x5627db0ec9e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8003cc0 tx=0x7fa0c800e620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7721dca4c3eb9ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:26.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:26.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.823+0000 7fa0b9ffb640 1 -- 172.21.15.172:0/2580580594 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8005ea0 con 0x5627db0e4370 2024-09-10T08:47:26.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.824+0000 7fa0b9ffb640 1 -- 172.21.15.172:0/2580580594 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa0c80130d0 con 0x5627db0e4370 2024-09-10T08:47:26.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.824+0000 7fa0dacc8740 1 -- 172.21.15.172:0/2580580594 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c4029070 con 0x5627db0e4370 2024-09-10T08:47:26.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.824+0000 7fa0b9ffb640 1 -- 172.21.15.172:0/2580580594 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa0c803c050 con 0x5627db0e4370 2024-09-10T08:47:26.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.824+0000 7fa0dacc8740 10 client.15693 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:26.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.824+0000 7fa0dacc8740 1 -- 172.21.15.172:0/2580580594 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5627db15ee70 con 0x5627db0e4370 2024-09-10T08:47:26.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.824+0000 7fa0b9ffb640 1 -- 172.21.15.172:0/2580580594 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0c4029070 con 0x5627db0e4370 2024-09-10T08:47:26.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.824+0000 7fa0b9ffb640 1 -- 172.21.15.172:0/2580580594 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa0c800ebb0 con 0x5627db0e4370 2024-09-10T08:47:26.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.824+0000 7fa0dacc8740 10 client.15693 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:26.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.824+0000 7fa0dacc8740 10 client.15693 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:47:26.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.824+0000 7fa0dacc8740 20 client.15693 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:26.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.824+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/2580580594 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5627db15f4e0 0x5627db161900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:26.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.824+0000 7fa0dacc8740 4 client.15693 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:47:26.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.824+0000 7fa0dacc8740 1 -- 172.21.15.172:0/2580580594 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5627db046d70 con 0x5627db15f4e0 2024-09-10T08:47:26.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.824+0000 7fa0d259c640 1 --2- 172.21.15.172:0/2580580594 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5627db15f4e0 0x5627db161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.825+0000 7fa0d259c640 1 --2- 172.21.15.172:0/2580580594 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5627db15f4e0 0x5627db161900 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b419cec1c445d002 server_cookie=ad6c86b3aa4f8977 in_seq=0 out_seq=0 2024-09-10T08:47:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.825+0000 7fa0b9ffb640 10 client.15693 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:47:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.825+0000 7fa0b9ffb640 1 -- 172.21.15.172:0/2580580594 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5627db046d70 con 0x5627db15f4e0 2024-09-10T08:47:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.825+0000 7fa0b9ffb640 10 client.15693 handle_command_reply: tid=0 2024-09-10T08:47:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.825+0000 7fa0dacc8740 1 client.15693 shutdown 2024-09-10T08:47:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.825+0000 7fa0dacc8740 1 -- 172.21.15.172:0/2580580594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0e4370 msgr2=0x5627db0ec9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.825+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/2580580594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0e4370 0x5627db0ec9e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8003cc0 tx=0x7fa0c800e620 comp rx=0 tx=0).stop 2024-09-10T08:47:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.825+0000 7fa0dacc8740 1 -- 172.21.15.172:0/2580580594 shutdown_connections 2024-09-10T08:47:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.825+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/2580580594 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5627db15f4e0 0x5627db161900 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.825+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/2580580594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627db0e4370 0x5627db0ec9e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.825+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/2580580594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5627db0dab40 0x5627db0ec480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.825+0000 7fa0b9ffb640 0 client.15693 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:47:26.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.825+0000 7fa0dacc8740 1 --2- 172.21.15.172:0/2580580594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5627db0d95d0 0x5627db0debc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.825+0000 7fa0dacc8740 1 -- 172.21.15.172:0/2580580594 >> 172.21.15.172:0/2580580594 conn(0x5627db0a2310 msgr2=0x5627db0d4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:26.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.825+0000 7fa0dacc8740 1 -- 172.21.15.172:0/2580580594 shutdown_connections 2024-09-10T08:47:26.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.825+0000 7fa0dacc8740 1 -- 172.21.15.172:0/2580580594 wait complete. 2024-09-10T08:47:26.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.825+0000 7fa0dacc8740 20 client.15693 trim_cache size 0 max 16384 2024-09-10T08:47:26.843 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:47:26.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.826+0000 7fa0d88ef640 1 -- 172.21.15.172:0/3272643766 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0bc0a2b80 msgr2=0x7fa0bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.826+0000 7fa0d88ef640 1 --2- 172.21.15.172:0/3272643766 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0bc0a2b80 0x7fa0bc0a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0b8b70 tx=0x7fa0cc0b1680 comp rx=0 tx=0).stop 2024-09-10T08:47:26.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.826+0000 7fa0d88ef640 1 -- 172.21.15.172:0/3272643766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0d4106ba0 msgr2=0x7fa0d419cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:26.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.826+0000 7fa0d88ef640 1 --2- 172.21.15.172:0/3272643766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0d4106ba0 0x7fa0d419cd30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009230 tx=0x7fa0cc0042f0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.826+0000 7fa0d88ef640 1 -- 172.21.15.172:0/3272643766 shutdown_connections 2024-09-10T08:47:26.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.826+0000 7fa0d88ef640 1 --2- 172.21.15.172:0/3272643766 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa0bc0a2b80 0x7fa0bc0a5040 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.826+0000 7fa0d88ef640 1 --2- 172.21.15.172:0/3272643766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0d4106ba0 0x7fa0d419cd30 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.826+0000 7fa0d88ef640 1 --2- 172.21.15.172:0/3272643766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d4100a70 0x7fa0d41989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.826+0000 7fa0d88ef640 1 --2- 172.21.15.172:0/3272643766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0d41000d0 0x7fa0d4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.826+0000 7fa0d88ef640 1 -- 172.21.15.172:0/3272643766 >> 172.21.15.172:0/3272643766 conn(0x7fa0d4077210 msgr2=0x7fa0d4104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.826+0000 7fa0d88ef640 1 -- 172.21.15.172:0/3272643766 shutdown_connections 2024-09-10T08:47:26.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:26.826+0000 7fa0d88ef640 1 -- 172.21.15.172:0/3272643766 wait complete. 2024-09-10T08:47:27.659 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2d459bfa', quiesce took 5.8 seconds. Will release after: 24.2 2024-09-10T08:47:28.141 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:47:28.389 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:28.509 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:28 smithi172 ceph-mon[30854]: pgmap v387: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-10T08:47:28.544 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:28 smithi174 ceph-mon[34068]: pgmap v387: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-10T08:47:28.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:28 smithi191 ceph-mon[35314]: pgmap v387: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-10T08:47:28.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.863+0000 7f38ddb42640 1 -- 172.21.15.172:0/2299907057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38d8101420 msgr2=0x7f38d810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:28.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.863+0000 7f38ddb42640 1 --2- 172.21.15.172:0/2299907057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38d8101420 0x7f38d810d950 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009cd0 tx=0x7f38d002a8e0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.863+0000 7f38ddb42640 1 -- 172.21.15.172:0/2299907057 shutdown_connections 2024-09-10T08:47:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.863+0000 7f38ddb42640 1 --2- 172.21.15.172:0/2299907057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38d8101420 0x7f38d810d950 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.863+0000 7f38ddb42640 1 --2- 172.21.15.172:0/2299907057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38d8100a80 0x7f38d8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.863+0000 7f38ddb42640 1 --2- 172.21.15.172:0/2299907057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38d8106a80 0x7f38d8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.863+0000 7f38ddb42640 1 -- 172.21.15.172:0/2299907057 >> 172.21.15.172:0/2299907057 conn(0x7f38d80fc7a0 msgr2=0x7f38d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.863+0000 7f38ddb42640 1 -- 172.21.15.172:0/2299907057 shutdown_connections 2024-09-10T08:47:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.863+0000 7f38ddb42640 1 -- 172.21.15.172:0/2299907057 wait complete. 2024-09-10T08:47:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.864+0000 7f38ddb42640 1 Processor -- start 2024-09-10T08:47:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.864+0000 7f38ddb42640 1 -- start start 2024-09-10T08:47:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.864+0000 7f38ddb42640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38d8100a80 0x7f38d8197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.864+0000 7f38ddb42640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38d8101420 0x7f38d8197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.864+0000 7f38ddb42640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38d8106a80 0x7f38d819bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.864+0000 7f38ddb42640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38d8110d30 con 0x7f38d8100a80 2024-09-10T08:47:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.864+0000 7f38ddb42640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f38d8110bb0 con 0x7f38d8101420 2024-09-10T08:47:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.864+0000 7f38ddb42640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f38d8110eb0 con 0x7f38d8106a80 2024-09-10T08:47:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.864+0000 7f38d77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38d8100a80 0x7f38d8197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.864+0000 7f38d77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38d8100a80 0x7f38d8197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57190/0 (socket says 172.21.15.172:57190) 2024-09-10T08:47:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.864+0000 7f38d77fe640 1 -- 172.21.15.172:0/3293978783 learned_addr learned my addr 172.21.15.172:0/3293978783 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.865+0000 7f38d6ffd640 1 --2- 172.21.15.172:0/3293978783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38d8101420 0x7f38d8197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.865+0000 7f38d7fff640 1 --2- 172.21.15.172:0/3293978783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38d8106a80 0x7f38d819bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.865+0000 7f38d77fe640 1 -- 172.21.15.172:0/3293978783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38d8106a80 msgr2=0x7f38d819bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.865+0000 7f38d77fe640 1 --2- 172.21.15.172:0/3293978783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38d8106a80 0x7f38d819bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.865+0000 7f38d77fe640 1 -- 172.21.15.172:0/3293978783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38d8101420 msgr2=0x7f38d8197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.865+0000 7f38d77fe640 1 --2- 172.21.15.172:0/3293978783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38d8101420 0x7f38d8197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.865+0000 7f38d77fe640 1 -- 172.21.15.172:0/3293978783 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d819c200 con 0x7f38d8100a80 2024-09-10T08:47:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.865+0000 7f38d77fe640 1 --2- 172.21.15.172:0/3293978783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38d8100a80 0x7f38d8197250 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f38c800e9f0 tx=0x7f38c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.865+0000 7f38d4ff9640 1 -- 172.21.15.172:0/3293978783 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c800cd80 con 0x7f38d8100a80 2024-09-10T08:47:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.865+0000 7f38d4ff9640 1 -- 172.21.15.172:0/3293978783 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38c80054d0 con 0x7f38d8100a80 2024-09-10T08:47:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.865+0000 7f38ddb42640 1 -- 172.21.15.172:0/3293978783 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d819c4f0 con 0x7f38d8100a80 2024-09-10T08:47:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.865+0000 7f38d4ff9640 1 -- 172.21.15.172:0/3293978783 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8010490 con 0x7f38d8100a80 2024-09-10T08:47:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.865+0000 7f38ddb42640 1 -- 172.21.15.172:0/3293978783 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d81a3dd0 con 0x7f38d8100a80 2024-09-10T08:47:28.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.867+0000 7f38d4ff9640 1 -- 172.21.15.172:0/3293978783 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f38c8005020 con 0x7f38d8100a80 2024-09-10T08:47:28.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38dff1b740 1 Processor -- start 2024-09-10T08:47:28.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38dff1b740 1 -- start start 2024-09-10T08:47:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38dff1b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58da4b40 0x561d58da3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38dff1b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da35d0 0x561d58da3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38dff1b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58dae370 0x561d58db0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38dff1b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561d58d092f0 con 0x561d58da4b40 2024-09-10T08:47:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38dff1b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561d58cb0cf0 con 0x561d58dae370 2024-09-10T08:47:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38dff1b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561d58d052b0 con 0x561d58da35d0 2024-09-10T08:47:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58da4b40 0x561d58da3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38d4ff9640 1 --2- 172.21.15.172:0/3293978783 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38c00a2b80 0x7f38c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58da4b40 0x561d58da3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57196/0 (socket says 172.21.15.172:57196) 2024-09-10T08:47:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38d7fff640 1 -- 172.21.15.172:0/1643572072 learned_addr learned my addr 172.21.15.172:0/1643572072 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38d6ffd640 1 --2- 172.21.15.172:0/1643572072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58dae370 0x561d58db0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38d77fe640 1 --2- 172.21.15.172:0/1643572072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da35d0 0x561d58da3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38d4ff9640 1 -- 172.21.15.172:0/3293978783 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f38c80cf770 con 0x7f38d8100a80 2024-09-10T08:47:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38d7fff640 1 --2- 172.21.15.172:0/3293978783 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38c00a2b80 0x7f38c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38d6ffd640 1 -- 172.21.15.172:0/1643572072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da35d0 msgr2=0x561d58da3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38d6ffd640 1 --2- 172.21.15.172:0/1643572072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da35d0 0x561d58da3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38d6ffd640 1 -- 172.21.15.172:0/1643572072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58da4b40 msgr2=0x561d58da3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38d6ffd640 1 --2- 172.21.15.172:0/1643572072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58da4b40 0x561d58da3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.868+0000 7f38d6ffd640 1 -- 172.21.15.172:0/1643572072 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d58c953a0 con 0x561d58dae370 2024-09-10T08:47:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.869+0000 7f38d77fe640 1 --2- 172.21.15.172:0/1643572072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da35d0 0x561d58da3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.869+0000 7f38d6ffd640 1 --2- 172.21.15.172:0/1643572072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58dae370 0x561d58db0760 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009bf0 tx=0x7f38cc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6fc95a2859820f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.869+0000 7f38bdffb640 1 -- 172.21.15.172:0/1643572072 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc009da0 con 0x561d58dae370 2024-09-10T08:47:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.869+0000 7f38d7fff640 1 --2- 172.21.15.172:0/3293978783 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38c00a2b80 0x7f38c00a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f38d0005fe0 tx=0x7f38d002a8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.869+0000 7f38bdffb640 1 -- 172.21.15.172:0/1643572072 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38cc005020 con 0x561d58dae370 2024-09-10T08:47:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.869+0000 7f38bdffb640 1 -- 172.21.15.172:0/1643572072 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc004a30 con 0x561d58dae370 2024-09-10T08:47:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.870+0000 7f38dff1b740 1 -- 172.21.15.172:0/1643572072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58dae370 msgr2=0x561d58db0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.870+0000 7f38dff1b740 1 --2- 172.21.15.172:0/1643572072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58dae370 0x561d58db0760 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009bf0 tx=0x7f38cc011810 comp rx=0 tx=0).stop 2024-09-10T08:47:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.870+0000 7f38dff1b740 1 -- 172.21.15.172:0/1643572072 shutdown_connections 2024-09-10T08:47:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.870+0000 7f38dff1b740 1 --2- 172.21.15.172:0/1643572072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58dae370 0x561d58db0760 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.870+0000 7f38dff1b740 1 --2- 172.21.15.172:0/1643572072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da35d0 0x561d58da3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.870+0000 7f38dff1b740 1 --2- 172.21.15.172:0/1643572072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58da4b40 0x561d58da3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.870+0000 7f38dff1b740 1 -- 172.21.15.172:0/1643572072 >> 172.21.15.172:0/1643572072 conn(0x561d58d6c310 msgr2=0x561d58d9fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.870+0000 7f38dff1b740 1 -- 172.21.15.172:0/1643572072 shutdown_connections 2024-09-10T08:47:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.870+0000 7f38dff1b740 1 -- 172.21.15.172:0/1643572072 wait complete. 2024-09-10T08:47:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.870+0000 7f38dff1b740 1 -- start start 2024-09-10T08:47:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.870+0000 7f38dff1b740 1 Processor -- start 2024-09-10T08:47:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38dff1b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da35d0 0x561d58dadcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38dff1b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da4b40 0x561d58daaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38dff1b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 0x561d58dab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38dff1b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561d58db1070 con 0x561d58dae370 2024-09-10T08:47:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38dff1b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561d58cc4060 con 0x561d58da4b40 2024-09-10T08:47:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38dff1b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561d58db11f0 con 0x561d58da35d0 2024-09-10T08:47:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38dff1b740 10 client.0 _ll_register_callbacks cb 0x561d58a178c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 0x561d58dab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38d6ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da4b40 0x561d58daaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 0x561d58dab4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57200/0 (socket says 172.21.15.172:57200) 2024-09-10T08:47:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38d6ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da4b40 0x561d58daaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:34968/0 (socket says 172.21.15.172:34968) 2024-09-10T08:47:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38d7fff640 1 -- 172.21.15.172:0/1557099799 learned_addr learned my addr 172.21.15.172:0/1557099799 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38d77fe640 1 --2- 172.21.15.172:0/1557099799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da35d0 0x561d58dadcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38d6ffd640 1 -- 172.21.15.172:0/1557099799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da35d0 msgr2=0x561d58dadcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38d6ffd640 1 --2- 172.21.15.172:0/1557099799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da35d0 0x561d58dadcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38d6ffd640 1 -- 172.21.15.172:0/1557099799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 msgr2=0x561d58dab4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38d6ffd640 1 --2- 172.21.15.172:0/1557099799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 0x561d58dab4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38d6ffd640 1 -- 172.21.15.172:0/1557099799 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d58a7a3c0 con 0x561d58da4b40 2024-09-10T08:47:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38d6ffd640 1 --2- 172.21.15.172:0/1557099799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da4b40 0x561d58daaf60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f38cc005790 tx=0x7f38cc0056b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=693be50f9188d92b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:47:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38b6ffd640 1 -- 172.21.15.172:0/1557099799 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc029020 con 0x561d58da4b40 2024-09-10T08:47:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38dff1b740 1 -- 172.21.15.172:0/1557099799 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38cc005020 con 0x561d58da4b40 2024-09-10T08:47:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.871+0000 7f38b6ffd640 1 -- 172.21.15.172:0/1557099799 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f38cc004aa0 con 0x561d58da4b40 2024-09-10T08:47:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.872+0000 7f38b6ffd640 1 -- 172.21.15.172:0/1557099799 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38cc034050 con 0x561d58da4b40 2024-09-10T08:47:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.872+0000 7f38dff1b740 10 client.25166 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.872+0000 7f38dff1b740 1 -- 172.21.15.172:0/1557099799 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561d58dbca60 con 0x561d58da4b40 2024-09-10T08:47:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.872+0000 7f38b6ffd640 1 -- 172.21.15.172:0/1557099799 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38cc005020 con 0x561d58da4b40 2024-09-10T08:47:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.872+0000 7f38b6ffd640 1 -- 172.21.15.172:0/1557099799 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f38cc023420 con 0x561d58da4b40 2024-09-10T08:47:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.872+0000 7f38dff1b740 10 client.25166 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.872+0000 7f38dff1b740 10 client.25166 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:47:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.872+0000 7f38dff1b740 20 client.25166 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.872+0000 7f38dff1b740 1 --2- 172.21.15.172:0/1557099799 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x561d58dbde70 0x561d58dc0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.872+0000 7f38dff1b740 4 client.25166 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.872+0000 7f38dff1b740 1 -- 172.21.15.172:0/1557099799 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d58d16860 con 0x561d58dbde70 2024-09-10T08:47:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.872+0000 7f38d77fe640 1 --2- 172.21.15.172:0/1557099799 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x561d58dbde70 0x561d58dc0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.873+0000 7f38b6ffd640 10 client.25166 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:47:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.873+0000 7f38d77fe640 1 --2- 172.21.15.172:0/1557099799 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x561d58dbde70 0x561d58dc0260 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd56813c9f26b41a server_cookie=ec073547ea5d030e in_seq=0 out_seq=0 2024-09-10T08:47:28.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.873+0000 7f38b6ffd640 1 -- 172.21.15.172:0/1557099799 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d58d16860 con 0x561d58dbde70 2024-09-10T08:47:28.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.873+0000 7f38b6ffd640 10 client.25166 handle_command_reply: tid=0 2024-09-10T08:47:28.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.873+0000 7f38dff1b740 1 client.25166 shutdown 2024-09-10T08:47:28.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.874+0000 7f38dff1b740 1 -- 172.21.15.172:0/1557099799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da4b40 msgr2=0x561d58daaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.874+0000 7f38dff1b740 1 --2- 172.21.15.172:0/1557099799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da4b40 0x561d58daaf60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f38cc005790 tx=0x7f38cc0056b0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.874+0000 7f38dff1b740 1 -- 172.21.15.172:0/1557099799 shutdown_connections 2024-09-10T08:47:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.874+0000 7f38dff1b740 1 --2- 172.21.15.172:0/1557099799 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x561d58dbde70 0x561d58dc0260 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.874+0000 7f38dff1b740 1 --2- 172.21.15.172:0/1557099799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 0x561d58dab4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.874+0000 7f38dff1b740 1 --2- 172.21.15.172:0/1557099799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da4b40 0x561d58daaf60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.874+0000 7f38b6ffd640 0 client.25166 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:47:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.874+0000 7f38dff1b740 1 --2- 172.21.15.172:0/1557099799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da35d0 0x561d58dadcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.874+0000 7f38dff1b740 1 -- 172.21.15.172:0/1557099799 >> 172.21.15.172:0/1557099799 conn(0x561d58d6c310 msgr2=0x561d58da5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.874+0000 7f38dff1b740 1 -- 172.21.15.172:0/1557099799 shutdown_connections 2024-09-10T08:47:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.874+0000 7f38dff1b740 1 -- 172.21.15.172:0/1557099799 wait complete. 2024-09-10T08:47:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.874+0000 7f38dff1b740 20 client.25166 trim_cache size 0 max 16384 2024-09-10T08:47:28.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.886+0000 7f38dff1b740 1 Processor -- start 2024-09-10T08:47:28.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.886+0000 7f38dff1b740 1 -- start start 2024-09-10T08:47:28.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38dff1b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da35d0 0x561d58dba310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38dff1b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da4b40 0x561d58dc5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38dff1b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 0x561d58dc5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38dff1b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81ac910 con 0x561d58dae370 2024-09-10T08:47:28.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38dff1b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81ae450 con 0x561d58da35d0 2024-09-10T08:47:28.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38dff1b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81acfe0 con 0x561d58da4b40 2024-09-10T08:47:28.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38d77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da35d0 0x561d58dba310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 0x561d58dc5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38d77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da35d0 0x561d58dba310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:34980/0 (socket says 172.21.15.172:34980) 2024-09-10T08:47:28.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 0x561d58dc5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57210/0 (socket says 172.21.15.172:57210) 2024-09-10T08:47:28.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38d77fe640 1 -- 172.21.15.172:0/51494991 learned_addr learned my addr 172.21.15.172:0/51494991 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:28.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38d6ffd640 1 --2- 172.21.15.172:0/51494991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da4b40 0x561d58dc5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38d7fff640 1 -- 172.21.15.172:0/51494991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da4b40 msgr2=0x561d58dc5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:28.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38d7fff640 1 --2- 172.21.15.172:0/51494991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da4b40 0x561d58dc5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38d7fff640 1 -- 172.21.15.172:0/51494991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da35d0 msgr2=0x561d58dba310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:28.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38d7fff640 1 --2- 172.21.15.172:0/51494991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da35d0 0x561d58dba310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38d7fff640 1 -- 172.21.15.172:0/51494991 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d58d9f4f0 con 0x561d58dae370 2024-09-10T08:47:28.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38d77fe640 1 --2- 172.21.15.172:0/51494991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da35d0 0x561d58dba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:28.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.887+0000 7f38d7fff640 1 --2- 172.21.15.172:0/51494991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 0x561d58dc5fb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x561d58dae340 tx=0x7f38d0005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cafb107329c638 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:28.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.888+0000 7f38bd7fa640 1 -- 172.21.15.172:0/51494991 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d0031c90 con 0x561d58dae370 2024-09-10T08:47:28.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.888+0000 7f38bd7fa640 1 -- 172.21.15.172:0/51494991 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38d0036040 con 0x561d58dae370 2024-09-10T08:47:28.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.888+0000 7f38bd7fa640 1 -- 172.21.15.172:0/51494991 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d00305a0 con 0x561d58dae370 2024-09-10T08:47:28.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.888+0000 7f38dff1b740 1 -- 172.21.15.172:0/51494991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 msgr2=0x561d58dc5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:28.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.888+0000 7f38dff1b740 1 --2- 172.21.15.172:0/51494991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 0x561d58dc5fb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x561d58dae340 tx=0x7f38d0005d80 comp rx=0 tx=0).stop 2024-09-10T08:47:28.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.888+0000 7f38dff1b740 1 -- 172.21.15.172:0/51494991 shutdown_connections 2024-09-10T08:47:28.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.888+0000 7f38dff1b740 1 --2- 172.21.15.172:0/51494991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 0x561d58dc5fb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.888+0000 7f38dff1b740 1 --2- 172.21.15.172:0/51494991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da4b40 0x561d58dc5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.888+0000 7f38dff1b740 1 --2- 172.21.15.172:0/51494991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da35d0 0x561d58dba310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.888+0000 7f38dff1b740 1 -- 172.21.15.172:0/51494991 >> 172.21.15.172:0/51494991 conn(0x561d58d6c310 msgr2=0x561d58dacfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:28.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.888+0000 7f38dff1b740 1 -- 172.21.15.172:0/51494991 shutdown_connections 2024-09-10T08:47:28.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.888+0000 7f38dff1b740 1 -- 172.21.15.172:0/51494991 wait complete. 2024-09-10T08:47:28.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.888+0000 7f38dff1b740 1 -- start start 2024-09-10T08:47:28.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38dff1b740 1 Processor -- start 2024-09-10T08:47:28.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38dff1b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da35d0 0x561d58db6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38dff1b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da4b40 0x561d58db1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38dff1b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 0x561d58db24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38dff1b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81add80 con 0x561d58dae370 2024-09-10T08:47:28.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38dff1b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81ad6b0 con 0x561d58da35d0 2024-09-10T08:47:28.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38dff1b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81aa8f0 con 0x561d58da4b40 2024-09-10T08:47:28.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38dff1b740 10 client.0 _ll_register_callbacks cb 0x7f38cc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:28.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 0x561d58db24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 0x561d58db24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57218/0 (socket says 172.21.15.172:57218) 2024-09-10T08:47:28.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38d77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da35d0 0x561d58db6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38d7fff640 1 -- 172.21.15.172:0/3195367034 learned_addr learned my addr 172.21.15.172:0/3195367034 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:28.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38d6ffd640 1 --2- 172.21.15.172:0/3195367034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da4b40 0x561d58db1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38d7fff640 1 -- 172.21.15.172:0/3195367034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da4b40 msgr2=0x561d58db1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:28.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38d7fff640 1 --2- 172.21.15.172:0/3195367034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da4b40 0x561d58db1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38d7fff640 1 -- 172.21.15.172:0/3195367034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da35d0 msgr2=0x561d58db6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:28.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38d7fff640 1 --2- 172.21.15.172:0/3195367034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da35d0 0x561d58db6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38d7fff640 1 -- 172.21.15.172:0/3195367034 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d58db2d20 con 0x561d58dae370 2024-09-10T08:47:28.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.889+0000 7f38d77fe640 1 --2- 172.21.15.172:0/3195367034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da35d0 0x561d58db6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:28.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.890+0000 7f38d7fff640 1 --2- 172.21.15.172:0/3195367034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 0x561d58db24f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f38d00097b0 tx=0x7f38d0031c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779d9de9bb602695 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:28.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.890+0000 7f38b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:28.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.890+0000 7f38b6ffd640 1 -- 172.21.15.172:0/3195367034 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d003b590 con 0x561d58dae370 2024-09-10T08:47:28.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.890+0000 7f38b6ffd640 1 -- 172.21.15.172:0/3195367034 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f38d003b730 con 0x561d58dae370 2024-09-10T08:47:28.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.890+0000 7f38dff1b740 1 -- 172.21.15.172:0/3195367034 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38d0036040 con 0x561d58dae370 2024-09-10T08:47:28.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.890+0000 7f38b6ffd640 1 -- 172.21.15.172:0/3195367034 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38d003b970 con 0x561d58dae370 2024-09-10T08:47:28.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.890+0000 7f38dff1b740 10 client.15711 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:28.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.890+0000 7f38dff1b740 1 -- 172.21.15.172:0/3195367034 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561d58e28ad0 con 0x561d58dae370 2024-09-10T08:47:28.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.890+0000 7f38b6ffd640 1 -- 172.21.15.172:0/3195367034 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38d0036040 con 0x561d58dae370 2024-09-10T08:47:28.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.890+0000 7f38b6ffd640 1 -- 172.21.15.172:0/3195367034 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f38d0030070 con 0x561d58dae370 2024-09-10T08:47:28.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.890+0000 7f38dff1b740 10 client.15711 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:28.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.890+0000 7f38dff1b740 10 client.15711 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:47:28.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.890+0000 7f38dff1b740 20 client.15711 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:28.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.890+0000 7f38dff1b740 1 --2- 172.21.15.172:0/3195367034 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x561d58e29140 0x561d58e2b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:28.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.890+0000 7f38dff1b740 4 client.15711 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:47:28.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.890+0000 7f38dff1b740 1 -- 172.21.15.172:0/3195367034 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d58dab830 con 0x561d58e29140 2024-09-10T08:47:28.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.891+0000 7f38d77fe640 1 --2- 172.21.15.172:0/3195367034 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x561d58e29140 0x561d58e2b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.891+0000 7f38b6ffd640 10 client.15711 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:47:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.891+0000 7f38d77fe640 1 --2- 172.21.15.172:0/3195367034 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x561d58e29140 0x561d58e2b530 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd07978146bc8325 server_cookie=86aad9b0daca9432 in_seq=0 out_seq=0 2024-09-10T08:47:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.891+0000 7f38b6ffd640 1 -- 172.21.15.172:0/3195367034 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d58dab830 con 0x561d58e29140 2024-09-10T08:47:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.891+0000 7f38b6ffd640 10 client.15711 handle_command_reply: tid=0 2024-09-10T08:47:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.891+0000 7f38dff1b740 1 client.15711 shutdown 2024-09-10T08:47:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.891+0000 7f38dff1b740 1 -- 172.21.15.172:0/3195367034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 msgr2=0x561d58db24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.891+0000 7f38dff1b740 1 --2- 172.21.15.172:0/3195367034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 0x561d58db24f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f38d00097b0 tx=0x7f38d0031c90 comp rx=0 tx=0).stop 2024-09-10T08:47:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.891+0000 7f38dff1b740 1 -- 172.21.15.172:0/3195367034 shutdown_connections 2024-09-10T08:47:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.891+0000 7f38dff1b740 1 --2- 172.21.15.172:0/3195367034 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x561d58e29140 0x561d58e2b530 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.891+0000 7f38dff1b740 1 --2- 172.21.15.172:0/3195367034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d58dae370 0x561d58db24f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.891+0000 7f38dff1b740 1 --2- 172.21.15.172:0/3195367034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d58da4b40 0x561d58db1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.891+0000 7f38dff1b740 1 --2- 172.21.15.172:0/3195367034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d58da35d0 0x561d58db6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.891+0000 7f38dff1b740 1 -- 172.21.15.172:0/3195367034 >> 172.21.15.172:0/3195367034 conn(0x561d58d6c310 msgr2=0x561d58d9e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:28.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.891+0000 7f38b6ffd640 0 client.15711 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:47:28.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.891+0000 7f38dff1b740 1 -- 172.21.15.172:0/3195367034 shutdown_connections 2024-09-10T08:47:28.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.891+0000 7f38dff1b740 1 -- 172.21.15.172:0/3195367034 wait complete. 2024-09-10T08:47:28.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.892+0000 7f38dff1b740 20 client.15711 trim_cache size 0 max 16384 2024-09-10T08:47:28.899 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:47:28.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.892+0000 7f38ddb42640 1 -- 172.21.15.172:0/3293978783 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38c00a2b80 msgr2=0x7f38c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:28.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.892+0000 7f38ddb42640 1 --2- 172.21.15.172:0/3293978783 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38c00a2b80 0x7f38c00a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f38d0005fe0 tx=0x7f38d002a8e0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.892+0000 7f38ddb42640 1 -- 172.21.15.172:0/3293978783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38d8100a80 msgr2=0x7f38d8197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:28.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.892+0000 7f38ddb42640 1 --2- 172.21.15.172:0/3293978783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38d8100a80 0x7f38d8197250 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f38c800e9f0 tx=0x7f38c800edf0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.892+0000 7f38ddb42640 1 -- 172.21.15.172:0/3293978783 shutdown_connections 2024-09-10T08:47:28.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.892+0000 7f38ddb42640 1 --2- 172.21.15.172:0/3293978783 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38c00a2b80 0x7f38c00a5040 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.892+0000 7f38ddb42640 1 --2- 172.21.15.172:0/3293978783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38d8106a80 0x7f38d819bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.892+0000 7f38ddb42640 1 --2- 172.21.15.172:0/3293978783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38d8101420 0x7f38d8197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.892+0000 7f38ddb42640 1 --2- 172.21.15.172:0/3293978783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38d8100a80 0x7f38d8197250 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:28.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.892+0000 7f38ddb42640 1 -- 172.21.15.172:0/3293978783 >> 172.21.15.172:0/3293978783 conn(0x7f38d80fc7a0 msgr2=0x7f38d80fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:28.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.892+0000 7f38ddb42640 1 -- 172.21.15.172:0/3293978783 shutdown_connections 2024-09-10T08:47:28.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:28.892+0000 7f38ddb42640 1 -- 172.21.15.172:0/3293978783 wait complete. 2024-09-10T08:47:29.699 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:47:29.954 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:30.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:30 smithi172 ceph-mon[30854]: pgmap v388: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:47:30.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.459+0000 7fa7c23c8640 1 -- 172.21.15.172:0/207754291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7bc101de0 msgr2=0x7fa7bc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:30.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.459+0000 7fa7c23c8640 1 --2- 172.21.15.172:0/207754291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7bc101de0 0x7fa7bc1021c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009900 tx=0x7fa7ac02a410 comp rx=0 tx=0).stop 2024-09-10T08:47:30.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.459+0000 7fa7c23c8640 1 -- 172.21.15.172:0/207754291 shutdown_connections 2024-09-10T08:47:30.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.459+0000 7fa7c23c8640 1 --2- 172.21.15.172:0/207754291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7bc106f10 0x7fa7bc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.459+0000 7fa7c23c8640 1 --2- 172.21.15.172:0/207754291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7bc102790 0x7fa7bc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.459+0000 7fa7c23c8640 1 --2- 172.21.15.172:0/207754291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7bc101de0 0x7fa7bc1021c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.459+0000 7fa7c23c8640 1 -- 172.21.15.172:0/207754291 >> 172.21.15.172:0/207754291 conn(0x7fa7bc0fd880 msgr2=0x7fa7bc0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:30.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.459+0000 7fa7c23c8640 1 -- 172.21.15.172:0/207754291 shutdown_connections 2024-09-10T08:47:30.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.459+0000 7fa7c23c8640 1 -- 172.21.15.172:0/207754291 wait complete. 2024-09-10T08:47:30.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.459+0000 7fa7c23c8640 1 Processor -- start 2024-09-10T08:47:30.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.459+0000 7fa7c23c8640 1 -- start start 2024-09-10T08:47:30.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7c23c8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7bc101de0 0x7fa7bc1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7c23c8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7bc102790 0x7fa7bc198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7c23c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7bc106f10 0x7fa7bc19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7c23c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc111e20 con 0x7fa7bc106f10 2024-09-10T08:47:30.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7c23c8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc111ca0 con 0x7fa7bc101de0 2024-09-10T08:47:30.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7c23c8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc111fa0 con 0x7fa7bc102790 2024-09-10T08:47:30.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7c0965640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7bc106f10 0x7fa7bc19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7c0965640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7bc106f10 0x7fa7bc19cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57222/0 (socket says 172.21.15.172:57222) 2024-09-10T08:47:30.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7c0965640 1 -- 172.21.15.172:0/500042367 learned_addr learned my addr 172.21.15.172:0/500042367 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:30.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7bbfff640 1 --2- 172.21.15.172:0/500042367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7bc101de0 0x7fa7bc1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7b3fff640 1 --2- 172.21.15.172:0/500042367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7bc102790 0x7fa7bc198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7c0965640 1 -- 172.21.15.172:0/500042367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7bc102790 msgr2=0x7fa7bc198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:30.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7c0965640 1 --2- 172.21.15.172:0/500042367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7bc102790 0x7fa7bc198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7c0965640 1 -- 172.21.15.172:0/500042367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7bc101de0 msgr2=0x7fa7bc1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:30.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7c0965640 1 --2- 172.21.15.172:0/500042367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7bc101de0 0x7fa7bc1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7c0965640 1 -- 172.21.15.172:0/500042367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7bc19d290 con 0x7fa7bc106f10 2024-09-10T08:47:30.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7c0965640 1 --2- 172.21.15.172:0/500042367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7bc106f10 0x7fa7bc19cbb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400eaa0 tx=0x7fa7b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:30.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7b9ffb640 1 -- 172.21.15.172:0/500042367 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b400cdb0 con 0x7fa7bc106f10 2024-09-10T08:47:30.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7b9ffb640 1 -- 172.21.15.172:0/500042367 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7b4005020 con 0x7fa7bc106f10 2024-09-10T08:47:30.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7c23c8640 1 -- 172.21.15.172:0/500042367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7bc19d580 con 0x7fa7bc106f10 2024-09-10T08:47:30.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.460+0000 7fa7b9ffb640 1 -- 172.21.15.172:0/500042367 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b40116d0 con 0x7fa7bc106f10 2024-09-10T08:47:30.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.461+0000 7fa7c23c8640 1 -- 172.21.15.172:0/500042367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7bc1a4e60 con 0x7fa7bc106f10 2024-09-10T08:47:30.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.463+0000 7fa7b9ffb640 1 -- 172.21.15.172:0/500042367 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa7b40051d0 con 0x7fa7bc106f10 2024-09-10T08:47:30.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.463+0000 7fa7c47a1740 1 Processor -- start 2024-09-10T08:47:30.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.463+0000 7fa7c47a1740 1 -- start start 2024-09-10T08:47:30.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.463+0000 7fa7c47a1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352bb40 0x55e13352a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.463+0000 7fa7c47a1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e13352a5d0 0x55e13352aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.463+0000 7fa7c47a1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e133535370 0x55e133537760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.463+0000 7fa7c47a1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e1334902f0 con 0x55e13352bb40 2024-09-10T08:47:30.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.463+0000 7fa7c47a1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e133437cf0 con 0x55e133535370 2024-09-10T08:47:30.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.463+0000 7fa7c47a1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e13348c2b0 con 0x55e13352a5d0 2024-09-10T08:47:30.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.463+0000 7fa7b3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352bb40 0x55e13352a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.463+0000 7fa7b9ffb640 1 --2- 172.21.15.172:0/500042367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7a80a2b80 0x7fa7a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.463+0000 7fa7b3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352bb40 0x55e13352a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57226/0 (socket says 172.21.15.172:57226) 2024-09-10T08:47:30.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.463+0000 7fa7b3fff640 1 -- 172.21.15.172:0/2486049398 learned_addr learned my addr 172.21.15.172:0/2486049398 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:30.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.464+0000 7fa7bbfff640 1 --2- 172.21.15.172:0/2486049398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e13352a5d0 0x55e13352aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.464+0000 7fa7c0965640 1 --2- 172.21.15.172:0/500042367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7a80a2b80 0x7fa7a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.464+0000 7fa7b3fff640 1 --2- 172.21.15.172:0/2486049398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e133535370 0x55e133537760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.464+0000 7fa7b3fff640 1 -- 172.21.15.172:0/2486049398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e13352a5d0 msgr2=0x55e13352aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:30.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.464+0000 7fa7b9ffb640 1 -- 172.21.15.172:0/500042367 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa7b40cf480 con 0x7fa7bc106f10 2024-09-10T08:47:30.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.464+0000 7fa7b3fff640 1 --2- 172.21.15.172:0/2486049398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e13352a5d0 0x55e13352aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.464+0000 7fa7b3fff640 1 -- 172.21.15.172:0/2486049398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e133535370 msgr2=0x55e133537760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:30.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.464+0000 7fa7b3fff640 1 --2- 172.21.15.172:0/2486049398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e133535370 0x55e133537760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.464+0000 7fa7b3fff640 1 -- 172.21.15.172:0/2486049398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e13341c3a0 con 0x55e13352bb40 2024-09-10T08:47:30.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.464+0000 7fa7b3fff640 1 --2- 172.21.15.172:0/2486049398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352bb40 0x55e13352a090 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa7a40058a0 tx=0x7fa7a400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d87a236d152873 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:30.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.464+0000 7fa7c0965640 1 --2- 172.21.15.172:0/500042367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7a80a2b80 0x7fa7a80a5040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4097e90 tx=0x7fa7b406d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:30.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.464+0000 7fa7b27fc640 1 -- 172.21.15.172:0/2486049398 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4004030 con 0x55e13352bb40 2024-09-10T08:47:30.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.464+0000 7fa7b27fc640 1 -- 172.21.15.172:0/2486049398 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7a40035f0 con 0x55e13352bb40 2024-09-10T08:47:30.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.464+0000 7fa7b27fc640 1 -- 172.21.15.172:0/2486049398 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a40106c0 con 0x55e13352bb40 2024-09-10T08:47:30.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.465+0000 7fa7c47a1740 1 -- 172.21.15.172:0/2486049398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352bb40 msgr2=0x55e13352a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:30.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.465+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/2486049398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352bb40 0x55e13352a090 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa7a40058a0 tx=0x7fa7a400b840 comp rx=0 tx=0).stop 2024-09-10T08:47:30.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.465+0000 7fa7c47a1740 1 -- 172.21.15.172:0/2486049398 shutdown_connections 2024-09-10T08:47:30.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.465+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/2486049398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e133535370 0x55e133537760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.465+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/2486049398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e13352a5d0 0x55e13352aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.465+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/2486049398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352bb40 0x55e13352a090 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.465+0000 7fa7c47a1740 1 -- 172.21.15.172:0/2486049398 >> 172.21.15.172:0/2486049398 conn(0x55e1334f3310 msgr2=0x55e133526c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:30.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.465+0000 7fa7c47a1740 1 -- 172.21.15.172:0/2486049398 shutdown_connections 2024-09-10T08:47:30.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.465+0000 7fa7c47a1740 1 -- 172.21.15.172:0/2486049398 wait complete. 2024-09-10T08:47:30.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.466+0000 7fa7c47a1740 1 -- start start 2024-09-10T08:47:30.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.466+0000 7fa7c47a1740 1 Processor -- start 2024-09-10T08:47:30.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.466+0000 7fa7c47a1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352a5d0 0x55e133534d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.466+0000 7fa7c47a1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13352bb40 0x55e133532030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.466+0000 7fa7c47a1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e133535370 0x55e133532640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.466+0000 7fa7c47a1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e133538070 con 0x55e13352a5d0 2024-09-10T08:47:30.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.466+0000 7fa7c47a1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e13344b060 con 0x55e13352bb40 2024-09-10T08:47:30.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.466+0000 7fa7c47a1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e1335381f0 con 0x55e133535370 2024-09-10T08:47:30.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.466+0000 7fa7c47a1740 10 client.0 _ll_register_callbacks cb 0x55e13319e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:30.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.466+0000 7fa7b3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352a5d0 0x55e133534d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.466+0000 7fa7b3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352a5d0 0x55e133534d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57238/0 (socket says 172.21.15.172:57238) 2024-09-10T08:47:30.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.466+0000 7fa7b3fff640 1 -- 172.21.15.172:0/1841140284 learned_addr learned my addr 172.21.15.172:0/1841140284 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:30.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.466+0000 7fa7bbfff640 1 --2- 172.21.15.172:0/1841140284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13352bb40 0x55e133532030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.467+0000 7fa7b3fff640 1 --2- 172.21.15.172:0/1841140284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e133535370 0x55e133532640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.467+0000 7fa7b3fff640 1 -- 172.21.15.172:0/1841140284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e133535370 msgr2=0x55e133532640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:30.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.467+0000 7fa7b3fff640 1 --2- 172.21.15.172:0/1841140284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e133535370 0x55e133532640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.467+0000 7fa7b3fff640 1 -- 172.21.15.172:0/1841140284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13352bb40 msgr2=0x55e133532030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:30.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.467+0000 7fa7b3fff640 1 --2- 172.21.15.172:0/1841140284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13352bb40 0x55e133532030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.467+0000 7fa7b3fff640 1 -- 172.21.15.172:0/1841140284 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1332013c0 con 0x55e13352a5d0 2024-09-10T08:47:30.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.467+0000 7fa7b3fff640 1 --2- 172.21.15.172:0/1841140284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352a5d0 0x55e133534d80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4003d60 tx=0x7fa7a400bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1c8ae6fb5729eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:30.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.467+0000 7fa79b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:30.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.467+0000 7fa79b7fe640 1 -- 172.21.15.172:0/1841140284 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4005230 con 0x55e13352a5d0 2024-09-10T08:47:30.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.467+0000 7fa79b7fe640 1 -- 172.21.15.172:0/1841140284 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa7a40053d0 con 0x55e13352a5d0 2024-09-10T08:47:30.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.467+0000 7fa7c47a1740 1 -- 172.21.15.172:0/1841140284 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a40035f0 con 0x55e13352a5d0 2024-09-10T08:47:30.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.467+0000 7fa79b7fe640 1 -- 172.21.15.172:0/1841140284 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa7a4033050 con 0x55e13352a5d0 2024-09-10T08:47:30.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.467+0000 7fa7c47a1740 10 client.15723 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:30.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.467+0000 7fa7c47a1740 1 -- 172.21.15.172:0/1841140284 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e133543b40 con 0x55e13352a5d0 2024-09-10T08:47:30.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.467+0000 7fa79b7fe640 1 -- 172.21.15.172:0/1841140284 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7a40035f0 con 0x55e13352a5d0 2024-09-10T08:47:30.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.468+0000 7fa79b7fe640 1 -- 172.21.15.172:0/1841140284 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa7a4004850 con 0x55e13352a5d0 2024-09-10T08:47:30.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.468+0000 7fa7c47a1740 10 client.15723 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:30.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.468+0000 7fa7c47a1740 10 client.15723 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:47:30.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.468+0000 7fa7c47a1740 20 client.15723 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:30.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.468+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/1841140284 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55e133544e90 0x55e133547280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.468+0000 7fa7c47a1740 4 client.15723 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:30.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.468+0000 7fa7c47a1740 1 -- 172.21.15.172:0/1841140284 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e13349d860 con 0x55e133544e90 2024-09-10T08:47:30.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.468+0000 7fa7bbfff640 1 --2- 172.21.15.172:0/1841140284 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55e133544e90 0x55e133547280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.469+0000 7fa79b7fe640 10 client.15723 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:47:30.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.469+0000 7fa7bbfff640 1 --2- 172.21.15.172:0/1841140284 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55e133544e90 0x55e133547280 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f099f7425627fbb server_cookie=7af5967736fe6b6a in_seq=0 out_seq=0 2024-09-10T08:47:30.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.469+0000 7fa79b7fe640 1 -- 172.21.15.172:0/1841140284 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e13349d860 con 0x55e133544e90 2024-09-10T08:47:30.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.469+0000 7fa79b7fe640 10 client.15723 handle_command_reply: tid=0 2024-09-10T08:47:30.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.469+0000 7fa7c47a1740 1 client.15723 shutdown 2024-09-10T08:47:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.469+0000 7fa7c47a1740 1 -- 172.21.15.172:0/1841140284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352a5d0 msgr2=0x55e133534d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.469+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/1841140284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352a5d0 0x55e133534d80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4003d60 tx=0x7fa7a400bfa0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.469+0000 7fa7c47a1740 1 -- 172.21.15.172:0/1841140284 shutdown_connections 2024-09-10T08:47:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.469+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/1841140284 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55e133544e90 0x55e133547280 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.470+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/1841140284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e133535370 0x55e133532640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.470+0000 7fa79b7fe640 0 client.15723 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:47:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.470+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/1841140284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13352bb40 0x55e133532030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.470+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/1841140284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352a5d0 0x55e133534d80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.470+0000 7fa7c47a1740 1 -- 172.21.15.172:0/1841140284 >> 172.21.15.172:0/1841140284 conn(0x55e1334f3310 msgr2=0x55e13352c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.470+0000 7fa7c47a1740 1 -- 172.21.15.172:0/1841140284 shutdown_connections 2024-09-10T08:47:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.470+0000 7fa7c47a1740 1 -- 172.21.15.172:0/1841140284 wait complete. 2024-09-10T08:47:30.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.470+0000 7fa7c47a1740 20 client.15723 trim_cache size 0 max 16384 2024-09-10T08:47:30.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.484+0000 7fa7c47a1740 1 Processor -- start 2024-09-10T08:47:30.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.484+0000 7fa7c47a1740 1 -- start start 2024-09-10T08:47:30.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.484+0000 7fa7c47a1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352a5d0 0x55e133541310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.484+0000 7fa7c47a1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e13352bb40 0x55e13354c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.484+0000 7fa7c47a1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e133535370 0x55e13354ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.484+0000 7fa7c47a1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc1aeb50 con 0x55e13352a5d0 2024-09-10T08:47:30.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.484+0000 7fa7c47a1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc1b0690 con 0x55e133535370 2024-09-10T08:47:30.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.484+0000 7fa7c47a1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc1af220 con 0x55e13352bb40 2024-09-10T08:47:30.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.485+0000 7fa7b3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352a5d0 0x55e133541310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.485+0000 7fa7b3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352a5d0 0x55e133541310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57246/0 (socket says 172.21.15.172:57246) 2024-09-10T08:47:30.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.485+0000 7fa7b3fff640 1 -- 172.21.15.172:0/4122310619 learned_addr learned my addr 172.21.15.172:0/4122310619 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:30.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.485+0000 7fa7b3fff640 1 --2- 172.21.15.172:0/4122310619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e133535370 0x55e13354ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.485+0000 7fa7bbfff640 1 --2- 172.21.15.172:0/4122310619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e13352bb40 0x55e13354c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.485+0000 7fa7b3fff640 1 -- 172.21.15.172:0/4122310619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e13352bb40 msgr2=0x55e13354c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:30.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.485+0000 7fa7b3fff640 1 --2- 172.21.15.172:0/4122310619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e13352bb40 0x55e13354c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.485+0000 7fa7b3fff640 1 -- 172.21.15.172:0/4122310619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e133535370 msgr2=0x55e13354ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:30.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.485+0000 7fa7b3fff640 1 --2- 172.21.15.172:0/4122310619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e133535370 0x55e13354ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.485+0000 7fa7b3fff640 1 -- 172.21.15.172:0/4122310619 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e133532fd0 con 0x55e13352a5d0 2024-09-10T08:47:30.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.485+0000 7fa7bbfff640 1 --2- 172.21.15.172:0/4122310619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e13352bb40 0x55e13354c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:30.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.485+0000 7fa7b3fff640 1 --2- 172.21.15.172:0/4122310619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352a5d0 0x55e133541310 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400bc10 tx=0x7fa7a400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322108b9650428d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:30.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.485+0000 7fa7b1ffb640 1 -- 172.21.15.172:0/4122310619 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4010460 con 0x55e13352a5d0 2024-09-10T08:47:30.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.485+0000 7fa7b1ffb640 1 -- 172.21.15.172:0/4122310619 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7a402a040 con 0x55e13352a5d0 2024-09-10T08:47:30.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.485+0000 7fa7b1ffb640 1 -- 172.21.15.172:0/4122310619 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4004fe0 con 0x55e13352a5d0 2024-09-10T08:47:30.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.486+0000 7fa7c47a1740 1 -- 172.21.15.172:0/4122310619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352a5d0 msgr2=0x55e133541310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:30.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.486+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/4122310619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352a5d0 0x55e133541310 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400bc10 tx=0x7fa7a400bea0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.486+0000 7fa7c47a1740 1 -- 172.21.15.172:0/4122310619 shutdown_connections 2024-09-10T08:47:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.486+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/4122310619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e133535370 0x55e13354ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.486+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/4122310619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e13352bb40 0x55e13354c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.486+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/4122310619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e13352a5d0 0x55e133541310 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.486+0000 7fa7c47a1740 1 -- 172.21.15.172:0/4122310619 >> 172.21.15.172:0/4122310619 conn(0x55e1334f3310 msgr2=0x55e133531120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.486+0000 7fa7c47a1740 1 -- 172.21.15.172:0/4122310619 shutdown_connections 2024-09-10T08:47:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.486+0000 7fa7c47a1740 1 -- 172.21.15.172:0/4122310619 wait complete. 2024-09-10T08:47:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.487+0000 7fa7c47a1740 1 -- start start 2024-09-10T08:47:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.487+0000 7fa7c47a1740 1 Processor -- start 2024-09-10T08:47:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.487+0000 7fa7c47a1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e13352a5d0 0x55e1335385c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.487+0000 7fa7c47a1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13352bb40 0x55e133538b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.487+0000 7fa7c47a1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e133535370 0x55e1335ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.487+0000 7fa7c47a1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc1affc0 con 0x55e133535370 2024-09-10T08:47:30.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.487+0000 7fa7c47a1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc1af8f0 con 0x55e13352bb40 2024-09-10T08:47:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.487+0000 7fa7c47a1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa7bc1acb30 con 0x55e13352a5d0 2024-09-10T08:47:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.487+0000 7fa7c47a1740 10 client.0 _ll_register_callbacks cb 0x7fa7ac005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.487+0000 7fa7b3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e133535370 0x55e1335ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.487+0000 7fa7b3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e133535370 0x55e1335ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57260/0 (socket says 172.21.15.172:57260) 2024-09-10T08:47:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.487+0000 7fa7b3fff640 1 -- 172.21.15.172:0/934328422 learned_addr learned my addr 172.21.15.172:0/934328422 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.487+0000 7fa7b3fff640 1 --2- 172.21.15.172:0/934328422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e13352a5d0 0x55e1335385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.487+0000 7fa7bbfff640 1 --2- 172.21.15.172:0/934328422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13352bb40 0x55e133538b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.488+0000 7fa7b3fff640 1 -- 172.21.15.172:0/934328422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e13352a5d0 msgr2=0x55e1335385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.488+0000 7fa7b3fff640 1 --2- 172.21.15.172:0/934328422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e13352a5d0 0x55e1335385c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.488+0000 7fa7b3fff640 1 -- 172.21.15.172:0/934328422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13352bb40 msgr2=0x55e133538b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.488+0000 7fa7b3fff640 1 --2- 172.21.15.172:0/934328422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13352bb40 0x55e133538b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.488+0000 7fa7b3fff640 1 -- 172.21.15.172:0/934328422 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1335adde0 con 0x55e133535370 2024-09-10T08:47:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.488+0000 7fa7bbfff640 1 --2- 172.21.15.172:0/934328422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13352bb40 0x55e133538b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.488+0000 7fa7b3fff640 1 --2- 172.21.15.172:0/934328422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e133535370 0x55e1335ad8a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4003d00 tx=0x7fa7a400bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668c69807e94f214 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.488+0000 7fa79b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.488+0000 7fa79b7fe640 1 -- 172.21.15.172:0/934328422 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4010c90 con 0x55e133535370 2024-09-10T08:47:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.488+0000 7fa79b7fe640 1 -- 172.21.15.172:0/934328422 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa7a4004eb0 con 0x55e133535370 2024-09-10T08:47:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.488+0000 7fa7c47a1740 1 -- 172.21.15.172:0/934328422 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a402a040 con 0x55e133535370 2024-09-10T08:47:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.488+0000 7fa79b7fe640 1 -- 172.21.15.172:0/934328422 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa7a404f050 con 0x55e133535370 2024-09-10T08:47:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.488+0000 7fa7c47a1740 10 client.15729 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.488+0000 7fa7c47a1740 1 -- 172.21.15.172:0/934328422 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e1335aefe0 con 0x55e133535370 2024-09-10T08:47:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.488+0000 7fa79b7fe640 1 -- 172.21.15.172:0/934328422 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7a4010e30 con 0x55e133535370 2024-09-10T08:47:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.489+0000 7fa79b7fe640 1 -- 172.21.15.172:0/934328422 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa7a400e490 con 0x55e133535370 2024-09-10T08:47:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.489+0000 7fa7c47a1740 10 client.15729 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:30.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.489+0000 7fa7c47a1740 10 client.15729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:47:30.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.489+0000 7fa7c47a1740 20 client.15729 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:30.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.489+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/934328422 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55e133544e90 0x55e1335b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:30.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.489+0000 7fa7c47a1740 4 client.15729 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:47:30.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.489+0000 7fa7c47a1740 1 -- 172.21.15.172:0/934328422 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa7a402a040 con 0x55e133544e90 2024-09-10T08:47:30.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.489+0000 7fa7bbfff640 1 --2- 172.21.15.172:0/934328422 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55e133544e90 0x55e1335b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:30.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.489+0000 7fa7bbfff640 1 --2- 172.21.15.172:0/934328422 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55e133544e90 0x55e1335b1600 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=645aa2c4637f07d6 server_cookie=a9652768ba6a98a7 in_seq=0 out_seq=0 2024-09-10T08:47:30.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.489+0000 7fa79b7fe640 10 client.15729 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:47:30.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.490+0000 7fa79b7fe640 1 -- 172.21.15.172:0/934328422 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa7a402a040 con 0x55e133544e90 2024-09-10T08:47:30.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.490+0000 7fa79b7fe640 10 client.15729 handle_command_reply: tid=0 2024-09-10T08:47:30.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.490+0000 7fa7c47a1740 1 client.15729 shutdown 2024-09-10T08:47:30.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.490+0000 7fa7c47a1740 1 -- 172.21.15.172:0/934328422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e133535370 msgr2=0x55e1335ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:30.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.490+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/934328422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e133535370 0x55e1335ad8a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4003d00 tx=0x7fa7a400bba0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.490+0000 7fa7c47a1740 1 -- 172.21.15.172:0/934328422 shutdown_connections 2024-09-10T08:47:30.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.490+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/934328422 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55e133544e90 0x55e1335b1600 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.490+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/934328422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e133535370 0x55e1335ad8a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.490+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/934328422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13352bb40 0x55e133538b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.490+0000 7fa7c47a1740 1 --2- 172.21.15.172:0/934328422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e13352a5d0 0x55e1335385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.490+0000 7fa7c47a1740 1 -- 172.21.15.172:0/934328422 >> 172.21.15.172:0/934328422 conn(0x55e1334f3310 msgr2=0x55e13352c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:30.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.490+0000 7fa79b7fe640 0 client.15729 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:47:30.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.490+0000 7fa7c47a1740 1 -- 172.21.15.172:0/934328422 shutdown_connections 2024-09-10T08:47:30.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.490+0000 7fa7c47a1740 1 -- 172.21.15.172:0/934328422 wait complete. 2024-09-10T08:47:30.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.490+0000 7fa7c47a1740 20 client.15729 trim_cache size 0 max 16384 2024-09-10T08:47:30.508 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:47:30.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.491+0000 7fa7c23c8640 1 -- 172.21.15.172:0/500042367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7a80a2b80 msgr2=0x7fa7a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:30.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.491+0000 7fa7c23c8640 1 --2- 172.21.15.172:0/500042367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7a80a2b80 0x7fa7a80a5040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4097e90 tx=0x7fa7b406d040 comp rx=0 tx=0).stop 2024-09-10T08:47:30.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.491+0000 7fa7c23c8640 1 -- 172.21.15.172:0/500042367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7bc106f10 msgr2=0x7fa7bc19cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:30.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.491+0000 7fa7c23c8640 1 --2- 172.21.15.172:0/500042367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7bc106f10 0x7fa7bc19cbb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400eaa0 tx=0x7fa7b400ef60 comp rx=0 tx=0).stop 2024-09-10T08:47:30.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.491+0000 7fa7c23c8640 1 -- 172.21.15.172:0/500042367 shutdown_connections 2024-09-10T08:47:30.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.491+0000 7fa7c23c8640 1 --2- 172.21.15.172:0/500042367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7a80a2b80 0x7fa7a80a5040 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.491+0000 7fa7c23c8640 1 --2- 172.21.15.172:0/500042367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7bc106f10 0x7fa7bc19cbb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.491+0000 7fa7c23c8640 1 --2- 172.21.15.172:0/500042367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7bc102790 0x7fa7bc198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.491+0000 7fa7c23c8640 1 --2- 172.21.15.172:0/500042367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7bc101de0 0x7fa7bc1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:30.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.491+0000 7fa7c23c8640 1 -- 172.21.15.172:0/500042367 >> 172.21.15.172:0/500042367 conn(0x7fa7bc0fd880 msgr2=0x7fa7bc0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:30.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.491+0000 7fa7c23c8640 1 -- 172.21.15.172:0/500042367 shutdown_connections 2024-09-10T08:47:30.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:30.491+0000 7fa7c23c8640 1 -- 172.21.15.172:0/500042367 wait complete. 2024-09-10T08:47:30.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:30 smithi191 ceph-mon[35314]: pgmap v388: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:47:30.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:30 smithi174 ceph-mon[34068]: pgmap v388: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:47:31.178 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:47:31.433 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:31.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.928+0000 7f870c9aa640 1 -- 172.21.15.172:0/1077780458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8708102ef0 msgr2=0x7f870810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:31.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.928+0000 7f870c9aa640 1 --2- 172.21.15.172:0/1077780458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8708102ef0 0x7f870810da40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8700009ca0 tx=0x7f870002a950 comp rx=0 tx=0).stop 2024-09-10T08:47:31.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.928+0000 7f870c9aa640 1 -- 172.21.15.172:0/1077780458 shutdown_connections 2024-09-10T08:47:31.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.928+0000 7f870c9aa640 1 --2- 172.21.15.172:0/1077780458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8708102ef0 0x7f870810da40 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.928+0000 7f870c9aa640 1 --2- 172.21.15.172:0/1077780458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8708102570 0x7f87081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.928+0000 7f870c9aa640 1 --2- 172.21.15.172:0/1077780458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8708100e20 0x7f8708101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.928+0000 7f870c9aa640 1 -- 172.21.15.172:0/1077780458 >> 172.21.15.172:0/1077780458 conn(0x7f87080fc7d0 msgr2=0x7f87080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:31.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.928+0000 7f870c9aa640 1 -- 172.21.15.172:0/1077780458 shutdown_connections 2024-09-10T08:47:31.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.929+0000 7f870c9aa640 1 -- 172.21.15.172:0/1077780458 wait complete. 2024-09-10T08:47:31.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.929+0000 7f870c9aa640 1 Processor -- start 2024-09-10T08:47:31.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.929+0000 7f870c9aa640 1 -- start start 2024-09-10T08:47:31.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f870c9aa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8708100e20 0x7f8708198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f870c9aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8708102570 0x7f8708198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f870c9aa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8708102ef0 0x7f870819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f870c9aa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8708110e30 con 0x7f8708102570 2024-09-10T08:47:31.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f870c9aa640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8708110cb0 con 0x7f8708100e20 2024-09-10T08:47:31.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f870c9aa640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8708110fb0 con 0x7f8708102ef0 2024-09-10T08:47:31.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f86fffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8708102570 0x7f8708198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f86fffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8708102570 0x7f8708198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57280/0 (socket says 172.21.15.172:57280) 2024-09-10T08:47:31.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f86fffff640 1 -- 172.21.15.172:0/4200015268 learned_addr learned my addr 172.21.15.172:0/4200015268 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:31.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f8706d9d640 1 --2- 172.21.15.172:0/4200015268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8708102ef0 0x7f870819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f870659c640 1 --2- 172.21.15.172:0/4200015268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8708100e20 0x7f8708198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f86fffff640 1 -- 172.21.15.172:0/4200015268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8708102ef0 msgr2=0x7f870819cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:31.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f86fffff640 1 --2- 172.21.15.172:0/4200015268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8708102ef0 0x7f870819cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f86fffff640 1 -- 172.21.15.172:0/4200015268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8708100e20 msgr2=0x7f8708198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:31.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f86fffff640 1 --2- 172.21.15.172:0/4200015268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8708100e20 0x7f8708198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f86fffff640 1 -- 172.21.15.172:0/4200015268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870819d1f0 con 0x7f8708102570 2024-09-10T08:47:31.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f8706d9d640 1 --2- 172.21.15.172:0/4200015268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8708102ef0 0x7f870819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:31.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f86fffff640 1 --2- 172.21.15.172:0/4200015268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8708102570 0x7f8708198780 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f86f000c920 tx=0x7f86f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:31.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.930+0000 7f86ff7fe640 1 -- 172.21.15.172:0/4200015268 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0007c70 con 0x7f8708102570 2024-09-10T08:47:31.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.931+0000 7f870c9aa640 1 -- 172.21.15.172:0/4200015268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870819d4e0 con 0x7f8708102570 2024-09-10T08:47:31.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.931+0000 7f86ff7fe640 1 -- 172.21.15.172:0/4200015268 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86f0007e10 con 0x7f8708102570 2024-09-10T08:47:31.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.931+0000 7f86ff7fe640 1 -- 172.21.15.172:0/4200015268 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0018820 con 0x7f8708102570 2024-09-10T08:47:31.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.931+0000 7f870c9aa640 1 -- 172.21.15.172:0/4200015268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87081a4dc0 con 0x7f8708102570 2024-09-10T08:47:31.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.933+0000 7f86ff7fe640 1 -- 172.21.15.172:0/4200015268 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f86f00189c0 con 0x7f8708102570 2024-09-10T08:47:31.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.933+0000 7f870ed83740 1 Processor -- start 2024-09-10T08:47:31.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.933+0000 7f870ed83740 1 -- start start 2024-09-10T08:47:31.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.933+0000 7f870ed83740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afe0b40 0x564d7afdf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f870ed83740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afdf5d0 0x564d7afdfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f870ed83740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afee780 0x564d7aff0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f870ed83740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564d7af452f0 con 0x564d7afee780 2024-09-10T08:47:31.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f870ed83740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564d7aeeccf0 con 0x564d7afe0b40 2024-09-10T08:47:31.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f870ed83740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564d7af412b0 con 0x564d7afdf5d0 2024-09-10T08:47:31.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f86fffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afee780 0x564d7aff0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f8706d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afe0b40 0x564d7afdf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f870659c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afdf5d0 0x564d7afdfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f8706d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afe0b40 0x564d7afdf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:35084/0 (socket says 172.21.15.172:35084) 2024-09-10T08:47:31.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f8706d9d640 1 -- 172.21.15.172:0/986364218 learned_addr learned my addr 172.21.15.172:0/986364218 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:31.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f86ff7fe640 1 --2- 172.21.15.172:0/4200015268 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86f40a2b80 0x7f86f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.940 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f870659c640 1 -- 172.21.15.172:0/986364218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afe0b40 msgr2=0x564d7afdf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:31.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f870659c640 1 --2- 172.21.15.172:0/986364218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afe0b40 0x564d7afdf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f870659c640 1 -- 172.21.15.172:0/986364218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afee780 msgr2=0x564d7aff0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:31.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f870659c640 1 --2- 172.21.15.172:0/986364218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afee780 0x564d7aff0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f870659c640 1 -- 172.21.15.172:0/986364218 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d7aed13a0 con 0x564d7afdf5d0 2024-09-10T08:47:31.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f86fffff640 1 --2- 172.21.15.172:0/986364218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afee780 0x564d7aff0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:31.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.934+0000 7f8706d9d640 1 --2- 172.21.15.172:0/4200015268 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86f40a2b80 0x7f86f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.935+0000 7f870659c640 1 --2- 172.21.15.172:0/986364218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afdf5d0 0x564d7afdfa10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f86f800cba0 tx=0x7f86f800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddf1e3586f45753b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:31.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.935+0000 7f86ff7fe640 1 -- 172.21.15.172:0/4200015268 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f86f00cf5c0 con 0x7f8708102570 2024-09-10T08:47:31.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.935+0000 7f86fcff9640 1 -- 172.21.15.172:0/986364218 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8004230 con 0x564d7afdf5d0 2024-09-10T08:47:31.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.935+0000 7f86fcff9640 1 -- 172.21.15.172:0/986364218 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86f80043d0 con 0x564d7afdf5d0 2024-09-10T08:47:31.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.935+0000 7f86fcff9640 1 -- 172.21.15.172:0/986364218 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8004c60 con 0x564d7afdf5d0 2024-09-10T08:47:31.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.936+0000 7f870ed83740 1 -- 172.21.15.172:0/986364218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afdf5d0 msgr2=0x564d7afdfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:31.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.936+0000 7f870ed83740 1 --2- 172.21.15.172:0/986364218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afdf5d0 0x564d7afdfa10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f86f800cba0 tx=0x7f86f800e720 comp rx=0 tx=0).stop 2024-09-10T08:47:31.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.936+0000 7f8706d9d640 1 --2- 172.21.15.172:0/4200015268 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86f40a2b80 0x7f86f40a5040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8700009430 tx=0x7f87000095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:31.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.936+0000 7f870ed83740 1 -- 172.21.15.172:0/986364218 shutdown_connections 2024-09-10T08:47:31.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.936+0000 7f870ed83740 1 --2- 172.21.15.172:0/986364218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afee780 0x564d7aff0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.936+0000 7f870ed83740 1 --2- 172.21.15.172:0/986364218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afdf5d0 0x564d7afdfa10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.936+0000 7f870ed83740 1 --2- 172.21.15.172:0/986364218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afe0b40 0x564d7afdf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.936+0000 7f870ed83740 1 -- 172.21.15.172:0/986364218 >> 172.21.15.172:0/986364218 conn(0x564d7afa8310 msgr2=0x564d7afdbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:31.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.936+0000 7f870ed83740 1 -- 172.21.15.172:0/986364218 shutdown_connections 2024-09-10T08:47:31.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.936+0000 7f870ed83740 1 -- 172.21.15.172:0/986364218 wait complete. 2024-09-10T08:47:31.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.936+0000 7f870ed83740 1 -- start start 2024-09-10T08:47:31.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.936+0000 7f870ed83740 1 Processor -- start 2024-09-10T08:47:31.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.936+0000 7f870ed83740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afdf5d0 0x564d7afee060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.936+0000 7f870ed83740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afe0b40 0x564d7afeb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.936+0000 7f870ed83740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afee780 0x564d7afeb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f870ed83740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564d7aff1520 con 0x564d7afe0b40 2024-09-10T08:47:31.945 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f870ed83740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564d7af00060 con 0x564d7afdf5d0 2024-09-10T08:47:31.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f870ed83740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564d7aff16a0 con 0x564d7afee780 2024-09-10T08:47:31.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f870ed83740 10 client.0 _ll_register_callbacks cb 0x564d7ac538c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:31.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f86fffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afe0b40 0x564d7afeb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f86fffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afe0b40 0x564d7afeb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57292/0 (socket says 172.21.15.172:57292) 2024-09-10T08:47:31.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f86fffff640 1 -- 172.21.15.172:0/450778421 learned_addr learned my addr 172.21.15.172:0/450778421 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:31.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f870659c640 1 --2- 172.21.15.172:0/450778421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afdf5d0 0x564d7afee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f8706d9d640 1 --2- 172.21.15.172:0/450778421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afee780 0x564d7afeb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f86fffff640 1 -- 172.21.15.172:0/450778421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afee780 msgr2=0x564d7afeb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:31.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f86fffff640 1 --2- 172.21.15.172:0/450778421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afee780 0x564d7afeb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f86fffff640 1 -- 172.21.15.172:0/450778421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afdf5d0 msgr2=0x564d7afee060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:31.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f86fffff640 1 --2- 172.21.15.172:0/450778421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afdf5d0 0x564d7afee060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.946 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f86fffff640 1 -- 172.21.15.172:0/450778421 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d7acb63c0 con 0x564d7afe0b40 2024-09-10T08:47:31.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f86fffff640 1 --2- 172.21.15.172:0/450778421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afe0b40 0x564d7afeb360 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f86f0007590 tx=0x7f86f000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458f6e8e0109ecff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:31.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f86edffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:31.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f86edffb640 1 -- 172.21.15.172:0/450778421 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f009b600 con 0x564d7afe0b40 2024-09-10T08:47:31.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f86edffb640 1 -- 172.21.15.172:0/450778421 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f86f001fd80 con 0x564d7afe0b40 2024-09-10T08:47:31.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.937+0000 7f870ed83740 1 -- 172.21.15.172:0/450778421 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f80043d0 con 0x564d7afe0b40 2024-09-10T08:47:31.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.938+0000 7f86edffb640 1 -- 172.21.15.172:0/450778421 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f86f0018cb0 con 0x564d7afe0b40 2024-09-10T08:47:31.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.938+0000 7f870ed83740 10 client.15741 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:31.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.938+0000 7f870ed83740 1 -- 172.21.15.172:0/450778421 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564d7affcf70 con 0x564d7afe0b40 2024-09-10T08:47:31.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.938+0000 7f86edffb640 1 -- 172.21.15.172:0/450778421 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86f80043d0 con 0x564d7afe0b40 2024-09-10T08:47:31.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.938+0000 7f86edffb640 1 -- 172.21.15.172:0/450778421 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f86f00a42a0 con 0x564d7afe0b40 2024-09-10T08:47:31.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.938+0000 7f870ed83740 10 client.15741 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:31.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.938+0000 7f870ed83740 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.938+0000 7f870ed83740 20 client.15741 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.938+0000 7f870ed83740 1 --2- 172.21.15.172:0/450778421 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564d7affe430 0x564d7b0008c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.938+0000 7f870ed83740 4 client.15741 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.938+0000 7f870ed83740 1 -- 172.21.15.172:0/450778421 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d7af52860 con 0x564d7affe430 2024-09-10T08:47:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.938+0000 7f870659c640 1 --2- 172.21.15.172:0/450778421 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564d7affe430 0x564d7b0008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.939+0000 7f86edffb640 10 client.15741 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.939+0000 7f870659c640 1 --2- 172.21.15.172:0/450778421 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564d7affe430 0x564d7b0008c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2656f2501321a5c0 server_cookie=1afd4ccf6b86bfe in_seq=0 out_seq=0 2024-09-10T08:47:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.939+0000 7f86edffb640 1 -- 172.21.15.172:0/450778421 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d7af52860 con 0x564d7affe430 2024-09-10T08:47:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.939+0000 7f86edffb640 10 client.15741 handle_command_reply: tid=0 2024-09-10T08:47:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.939+0000 7f870ed83740 1 client.15741 shutdown 2024-09-10T08:47:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.940+0000 7f870ed83740 1 -- 172.21.15.172:0/450778421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afe0b40 msgr2=0x564d7afeb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:31.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.940+0000 7f870ed83740 1 --2- 172.21.15.172:0/450778421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afe0b40 0x564d7afeb360 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f86f0007590 tx=0x7f86f000cf60 comp rx=0 tx=0).stop 2024-09-10T08:47:31.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.940+0000 7f870ed83740 1 -- 172.21.15.172:0/450778421 shutdown_connections 2024-09-10T08:47:31.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.940+0000 7f870ed83740 1 --2- 172.21.15.172:0/450778421 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564d7affe430 0x564d7b0008c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.940+0000 7f870ed83740 1 --2- 172.21.15.172:0/450778421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afee780 0x564d7afeb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.940+0000 7f870ed83740 1 --2- 172.21.15.172:0/450778421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afe0b40 0x564d7afeb360 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.940+0000 7f870ed83740 1 --2- 172.21.15.172:0/450778421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afdf5d0 0x564d7afee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.940+0000 7f86edffb640 0 client.15741 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:31.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.940+0000 7f870ed83740 1 -- 172.21.15.172:0/450778421 >> 172.21.15.172:0/450778421 conn(0x564d7afa8310 msgr2=0x564d7afe1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:31.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.940+0000 7f870ed83740 1 -- 172.21.15.172:0/450778421 shutdown_connections 2024-09-10T08:47:31.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.940+0000 7f870ed83740 1 -- 172.21.15.172:0/450778421 wait complete. 2024-09-10T08:47:31.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.940+0000 7f870ed83740 20 client.15741 trim_cache size 0 max 16384 2024-09-10T08:47:31.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.953+0000 7f870ed83740 1 Processor -- start 2024-09-10T08:47:31.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.953+0000 7f870ed83740 1 -- start start 2024-09-10T08:47:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.953+0000 7f870ed83740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afdf5d0 0x564d7afdbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.953+0000 7f870ed83740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afe0b40 0x564d7affa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.953+0000 7f870ed83740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afee780 0x564d7b006190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.953+0000 7f870ed83740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f87081ac7f0 con 0x564d7afdf5d0 2024-09-10T08:47:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.953+0000 7f870ed83740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87081ae330 con 0x564d7afe0b40 2024-09-10T08:47:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.953+0000 7f870ed83740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f87081acec0 con 0x564d7afee780 2024-09-10T08:47:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.953+0000 7f870659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afdf5d0 0x564d7afdbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.953+0000 7f870659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afdf5d0 0x564d7afdbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57306/0 (socket says 172.21.15.172:57306) 2024-09-10T08:47:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.953+0000 7f870659c640 1 -- 172.21.15.172:0/2754508591 learned_addr learned my addr 172.21.15.172:0/2754508591 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.954+0000 7f86fffff640 1 --2- 172.21.15.172:0/2754508591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afe0b40 0x564d7affa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.954+0000 7f870659c640 1 -- 172.21.15.172:0/2754508591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afee780 msgr2=0x564d7b006190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.954+0000 7f8706d9d640 1 --2- 172.21.15.172:0/2754508591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afee780 0x564d7b006190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.954+0000 7f870659c640 1 --2- 172.21.15.172:0/2754508591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afee780 0x564d7b006190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.954+0000 7f870659c640 1 -- 172.21.15.172:0/2754508591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afe0b40 msgr2=0x564d7affa7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.954+0000 7f870659c640 1 --2- 172.21.15.172:0/2754508591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afe0b40 0x564d7affa7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.954+0000 7f870659c640 1 -- 172.21.15.172:0/2754508591 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d7b000f40 con 0x564d7afdf5d0 2024-09-10T08:47:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.954+0000 7f86fffff640 1 --2- 172.21.15.172:0/2754508591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afe0b40 0x564d7affa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.954+0000 7f8706d9d640 1 --2- 172.21.15.172:0/2754508591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afee780 0x564d7b006190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.954+0000 7f870659c640 1 --2- 172.21.15.172:0/2754508591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afdf5d0 0x564d7afdbe10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f86f8000c00 tx=0x7f86f800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50915be2ac0ad036 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.954+0000 7f86effff640 1 -- 172.21.15.172:0/2754508591 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f801a650 con 0x564d7afdf5d0 2024-09-10T08:47:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.954+0000 7f86effff640 1 -- 172.21.15.172:0/2754508591 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86f801aca0 con 0x564d7afdf5d0 2024-09-10T08:47:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.955+0000 7f86effff640 1 -- 172.21.15.172:0/2754508591 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8010bf0 con 0x564d7afdf5d0 2024-09-10T08:47:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.955+0000 7f870ed83740 1 -- 172.21.15.172:0/2754508591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afdf5d0 msgr2=0x564d7afdbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.955+0000 7f870ed83740 1 --2- 172.21.15.172:0/2754508591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afdf5d0 0x564d7afdbe10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f86f8000c00 tx=0x7f86f800cac0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.955+0000 7f870ed83740 1 -- 172.21.15.172:0/2754508591 shutdown_connections 2024-09-10T08:47:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.955+0000 7f870ed83740 1 --2- 172.21.15.172:0/2754508591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afee780 0x564d7b006190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.955+0000 7f870ed83740 1 --2- 172.21.15.172:0/2754508591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afe0b40 0x564d7affa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.955+0000 7f870ed83740 1 --2- 172.21.15.172:0/2754508591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afdf5d0 0x564d7afdbe10 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.955+0000 7f870ed83740 1 -- 172.21.15.172:0/2754508591 >> 172.21.15.172:0/2754508591 conn(0x564d7afa8310 msgr2=0x564d7afdb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.955+0000 7f870ed83740 1 -- 172.21.15.172:0/2754508591 shutdown_connections 2024-09-10T08:47:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.955+0000 7f870ed83740 1 -- 172.21.15.172:0/2754508591 wait complete. 2024-09-10T08:47:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.955+0000 7f870ed83740 1 -- start start 2024-09-10T08:47:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.955+0000 7f870ed83740 1 Processor -- start 2024-09-10T08:47:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f870ed83740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afdf5d0 0x564d7aff6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f870ed83740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afe0b40 0x564d7aff7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f870ed83740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afee780 0x564d7afe7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f870ed83740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f87081adc60 con 0x564d7afe0b40 2024-09-10T08:47:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f870ed83740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87081ad590 con 0x564d7afdf5d0 2024-09-10T08:47:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f870ed83740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f87081aa7d0 con 0x564d7afee780 2024-09-10T08:47:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f870ed83740 10 client.0 _ll_register_callbacks cb 0x7f86f00d5180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:31.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f86fffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afe0b40 0x564d7aff7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f870659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afdf5d0 0x564d7aff6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f86fffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afe0b40 0x564d7aff7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57320/0 (socket says 172.21.15.172:57320) 2024-09-10T08:47:31.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f86fffff640 1 -- 172.21.15.172:0/123399078 learned_addr learned my addr 172.21.15.172:0/123399078 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:31.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f8706d9d640 1 --2- 172.21.15.172:0/123399078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afee780 0x564d7afe7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f870659c640 1 -- 172.21.15.172:0/123399078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afee780 msgr2=0x564d7afe7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:31.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f870659c640 1 --2- 172.21.15.172:0/123399078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afee780 0x564d7afe7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f870659c640 1 -- 172.21.15.172:0/123399078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afe0b40 msgr2=0x564d7aff7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:31.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f870659c640 1 --2- 172.21.15.172:0/123399078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afe0b40 0x564d7aff7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f870659c640 1 -- 172.21.15.172:0/123399078 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d7afe87f0 con 0x564d7afdf5d0 2024-09-10T08:47:31.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f86fffff640 1 --2- 172.21.15.172:0/123399078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afe0b40 0x564d7aff7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:31.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f8706d9d640 1 --2- 172.21.15.172:0/123399078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afee780 0x564d7afe7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f870659c640 1 --2- 172.21.15.172:0/123399078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afdf5d0 0x564d7aff6d20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f86f800c4f0 tx=0x7f86f800ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30cf7c2248803ed7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.956+0000 7f86edffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:47:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.957+0000 7f86edffb640 1 -- 172.21.15.172:0/123399078 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8010870 con 0x564d7afdf5d0 2024-09-10T08:47:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.957+0000 7f86edffb640 1 -- 172.21.15.172:0/123399078 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f86f8010a10 con 0x564d7afdf5d0 2024-09-10T08:47:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.957+0000 7f870ed83740 1 -- 172.21.15.172:0/123399078 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f801aca0 con 0x564d7afdf5d0 2024-09-10T08:47:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.957+0000 7f86edffb640 1 -- 172.21.15.172:0/123399078 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f86f8010c10 con 0x564d7afdf5d0 2024-09-10T08:47:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.957+0000 7f870ed83740 10 client.25199 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.957+0000 7f870ed83740 1 -- 172.21.15.172:0/123399078 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564d7b068da0 con 0x564d7afdf5d0 2024-09-10T08:47:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.957+0000 7f86edffb640 1 -- 172.21.15.172:0/123399078 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86f801aca0 con 0x564d7afdf5d0 2024-09-10T08:47:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.957+0000 7f86edffb640 1 -- 172.21.15.172:0/123399078 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f86f801a730 con 0x564d7afdf5d0 2024-09-10T08:47:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.957+0000 7f870ed83740 10 client.25199 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:31.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.957+0000 7f870ed83740 10 client.25199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:31.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.957+0000 7f870ed83740 20 client.25199 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:31.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.957+0000 7f870ed83740 1 --2- 172.21.15.172:0/123399078 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564d7b069220 0x564d7b06b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:31.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.957+0000 7f870ed83740 4 client.25199 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:47:31.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.957+0000 7f870ed83740 1 -- 172.21.15.172:0/123399078 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d7af4cd70 con 0x564d7b069220 2024-09-10T08:47:31.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.958+0000 7f86fffff640 1 --2- 172.21.15.172:0/123399078 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564d7b069220 0x564d7b06b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:31.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.958+0000 7f86edffb640 10 client.25199 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:31.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.958+0000 7f86fffff640 1 --2- 172.21.15.172:0/123399078 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564d7b069220 0x564d7b06b610 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d01a468d9d66ab server_cookie=90d6690395b9859f in_seq=0 out_seq=0 2024-09-10T08:47:31.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.958+0000 7f86edffb640 1 -- 172.21.15.172:0/123399078 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d7af4cd70 con 0x564d7b069220 2024-09-10T08:47:31.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.958+0000 7f86edffb640 10 client.25199 handle_command_reply: tid=0 2024-09-10T08:47:31.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.958+0000 7f870ed83740 1 client.25199 shutdown 2024-09-10T08:47:31.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.958+0000 7f870ed83740 1 -- 172.21.15.172:0/123399078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afdf5d0 msgr2=0x564d7aff6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:31.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.958+0000 7f870ed83740 1 --2- 172.21.15.172:0/123399078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afdf5d0 0x564d7aff6d20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f86f800c4f0 tx=0x7f86f800ed50 comp rx=0 tx=0).stop 2024-09-10T08:47:31.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.958+0000 7f870ed83740 1 -- 172.21.15.172:0/123399078 shutdown_connections 2024-09-10T08:47:31.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.958+0000 7f870ed83740 1 --2- 172.21.15.172:0/123399078 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564d7b069220 0x564d7b06b610 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.958+0000 7f870ed83740 1 --2- 172.21.15.172:0/123399078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564d7afee780 0x564d7afe7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.958+0000 7f870ed83740 1 --2- 172.21.15.172:0/123399078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564d7afe0b40 0x564d7aff7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.958+0000 7f870ed83740 1 --2- 172.21.15.172:0/123399078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d7afdf5d0 0x564d7aff6d20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.958+0000 7f870ed83740 1 -- 172.21.15.172:0/123399078 >> 172.21.15.172:0/123399078 conn(0x564d7afa8310 msgr2=0x564d7afe97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:31.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.958+0000 7f86edffb640 0 client.25199 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:31.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.958+0000 7f870ed83740 1 -- 172.21.15.172:0/123399078 shutdown_connections 2024-09-10T08:47:31.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.958+0000 7f870ed83740 1 -- 172.21.15.172:0/123399078 wait complete. 2024-09-10T08:47:31.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.959+0000 7f870ed83740 20 client.25199 trim_cache size 0 max 16384 2024-09-10T08:47:31.966 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:47:31.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.959+0000 7f870c9aa640 1 -- 172.21.15.172:0/4200015268 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86f40a2b80 msgr2=0x7f86f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:31.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.959+0000 7f870c9aa640 1 --2- 172.21.15.172:0/4200015268 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86f40a2b80 0x7f86f40a5040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8700009430 tx=0x7f87000095c0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.959+0000 7f870c9aa640 1 -- 172.21.15.172:0/4200015268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8708102570 msgr2=0x7f8708198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:31.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.959+0000 7f870c9aa640 1 --2- 172.21.15.172:0/4200015268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8708102570 0x7f8708198780 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f86f000c920 tx=0x7f86f000cde0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.959+0000 7f870c9aa640 1 -- 172.21.15.172:0/4200015268 shutdown_connections 2024-09-10T08:47:31.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.959+0000 7f870c9aa640 1 --2- 172.21.15.172:0/4200015268 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86f40a2b80 0x7f86f40a5040 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.959+0000 7f870c9aa640 1 --2- 172.21.15.172:0/4200015268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8708102ef0 0x7f870819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.959+0000 7f870c9aa640 1 --2- 172.21.15.172:0/4200015268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8708102570 0x7f8708198780 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.959+0000 7f870c9aa640 1 --2- 172.21.15.172:0/4200015268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8708100e20 0x7f8708198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:31.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.959+0000 7f870c9aa640 1 -- 172.21.15.172:0/4200015268 >> 172.21.15.172:0/4200015268 conn(0x7f87080fc7d0 msgr2=0x7f870806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:31.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.959+0000 7f870c9aa640 1 -- 172.21.15.172:0/4200015268 shutdown_connections 2024-09-10T08:47:31.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:31.959+0000 7f870c9aa640 1 -- 172.21.15.172:0/4200015268 wait complete. 2024-09-10T08:47:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:32 smithi191 ceph-mon[35314]: pgmap v389: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:47:32.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:32 smithi172 ceph-mon[30854]: pgmap v389: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:47:32.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:32 smithi174 ceph-mon[34068]: pgmap v389: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:47:33.638 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:47:33.638 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:47:33.903 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:34.321 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:34 smithi172 ceph-mon[30854]: pgmap v390: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:47:34.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.408+0000 7f8d86994640 1 -- 172.21.15.172:0/585708008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d801082a0 msgr2=0x7f8d8010ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.408+0000 7f8d86994640 1 --2- 172.21.15.172:0/585708008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d801082a0 0x7f8d8010ec10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009cd0 tx=0x7f8d7802a8e0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.408+0000 7f8d86994640 1 -- 172.21.15.172:0/585708008 shutdown_connections 2024-09-10T08:47:34.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.408+0000 7f8d86994640 1 --2- 172.21.15.172:0/585708008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d801082a0 0x7f8d8010ec10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.408+0000 7f8d86994640 1 --2- 172.21.15.172:0/585708008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d80103720 0x7f8d80107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.408+0000 7f8d86994640 1 --2- 172.21.15.172:0/585708008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d80102d70 0x7f8d80103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.408+0000 7f8d86994640 1 -- 172.21.15.172:0/585708008 >> 172.21.15.172:0/585708008 conn(0x7f8d800fc7d0 msgr2=0x7f8d800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:34.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.408+0000 7f8d86994640 1 -- 172.21.15.172:0/585708008 shutdown_connections 2024-09-10T08:47:34.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.409+0000 7f8d86994640 1 -- 172.21.15.172:0/585708008 wait complete. 2024-09-10T08:47:34.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.409+0000 7f8d86994640 1 Processor -- start 2024-09-10T08:47:34.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.409+0000 7f8d86994640 1 -- start start 2024-09-10T08:47:34.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.410+0000 7f8d86994640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d80102d70 0x7f8d80198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.410+0000 7f8d86994640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d80103720 0x7f8d801989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.410+0000 7f8d86994640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d801082a0 0x7f8d8019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.410+0000 7f8d86994640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8d80111f70 con 0x7f8d80102d70 2024-09-10T08:47:34.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.410+0000 7f8d86994640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d80111df0 con 0x7f8d801082a0 2024-09-10T08:47:34.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.410+0000 7f8d86994640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8d801120f0 con 0x7f8d80103720 2024-09-10T08:47:34.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.410+0000 7f8d7ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d80102d70 0x7f8d80198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.410+0000 7f8d7ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d80102d70 0x7f8d80198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57340/0 (socket says 172.21.15.172:57340) 2024-09-10T08:47:34.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.410+0000 7f8d7ffff640 1 -- 172.21.15.172:0/636950530 learned_addr learned my addr 172.21.15.172:0/636950530 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:34.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.410+0000 7f8d77fff640 1 --2- 172.21.15.172:0/636950530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d80103720 0x7f8d801989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.410+0000 7f8d7ffff640 1 -- 172.21.15.172:0/636950530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d80103720 msgr2=0x7f8d801989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.410+0000 7f8d7ffff640 1 --2- 172.21.15.172:0/636950530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d80103720 0x7f8d801989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.410+0000 7f8d7ffff640 1 -- 172.21.15.172:0/636950530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d801082a0 msgr2=0x7f8d8019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.410+0000 7f8d7ffff640 1 --2- 172.21.15.172:0/636950530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d801082a0 0x7f8d8019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.410+0000 7f8d7ffff640 1 -- 172.21.15.172:0/636950530 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d8019d420 con 0x7f8d80102d70 2024-09-10T08:47:34.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.410+0000 7f8d7ffff640 1 --2- 172.21.15.172:0/636950530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d80102d70 0x7f8d80198470 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000d950 tx=0x7f8d7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.410+0000 7f8d7dffb640 1 -- 172.21.15.172:0/636950530 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70017070 con 0x7f8d80102d70 2024-09-10T08:47:34.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.411+0000 7f8d86994640 1 -- 172.21.15.172:0/636950530 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d8019d710 con 0x7f8d80102d70 2024-09-10T08:47:34.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.411+0000 7f8d7dffb640 1 -- 172.21.15.172:0/636950530 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d70009c30 con 0x7f8d80102d70 2024-09-10T08:47:34.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.411+0000 7f8d7dffb640 1 -- 172.21.15.172:0/636950530 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7000f620 con 0x7f8d80102d70 2024-09-10T08:47:34.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.411+0000 7f8d86994640 1 -- 172.21.15.172:0/636950530 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d801a4ff0 con 0x7f8d80102d70 2024-09-10T08:47:34.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.413+0000 7f8d7dffb640 1 -- 172.21.15.172:0/636950530 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8d700109a0 con 0x7f8d80102d70 2024-09-10T08:47:34.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.413+0000 7f8d88d6d740 1 Processor -- start 2024-09-10T08:47:34.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.414+0000 7f8d88d6d740 1 -- start start 2024-09-10T08:47:34.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.414+0000 7f8d88d6d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f80a60 0x5582e5f80e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.414+0000 7f8d88d6d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f81380 0x5582e5f8fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.414+0000 7f8d7dffb640 1 --2- 172.21.15.172:0/636950530 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8d6c0a2b80 0x7f8d6c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.414+0000 7f8d88d6d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f90150 0x5582e5f92540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.414+0000 7f8d88d6d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5582e5ee6280 con 0x5582e5f81380 2024-09-10T08:47:34.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.414+0000 7f8d88d6d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5582e5e8db10 con 0x5582e5f80a60 2024-09-10T08:47:34.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.414+0000 7f8d88d6d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5582e5ee2240 con 0x5582e5f90150 2024-09-10T08:47:34.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.414+0000 7f8d7dffb640 1 -- 172.21.15.172:0/636950530 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8d700cefe0 con 0x7f8d80102d70 2024-09-10T08:47:34.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.414+0000 7f8d77fff640 1 --2- 172.21.15.172:0/636950530 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8d6c0a2b80 0x7f8d6c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.414+0000 7f8d84f31640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f80a60 0x5582e5f80e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.414+0000 7f8d84f31640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f80a60 0x5582e5f80e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:35148/0 (socket says 172.21.15.172:35148) 2024-09-10T08:47:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.414+0000 7f8d84f31640 1 -- 172.21.15.172:0/3909142619 learned_addr learned my addr 172.21.15.172:0/3909142619 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.414+0000 7f8d84f31640 1 --2- 172.21.15.172:0/3909142619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f90150 0x5582e5f92540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.414+0000 7f8d7ffff640 1 --2- 172.21.15.172:0/3909142619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f81380 0x5582e5f8fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.415+0000 7f8d84f31640 1 -- 172.21.15.172:0/3909142619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f80a60 msgr2=0x5582e5f80e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.415+0000 7f8d84f31640 1 --2- 172.21.15.172:0/3909142619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f80a60 0x5582e5f80e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.415+0000 7f8d84f31640 1 -- 172.21.15.172:0/3909142619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f81380 msgr2=0x5582e5f8fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.415+0000 7f8d84f31640 1 --2- 172.21.15.172:0/3909142619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f81380 0x5582e5f8fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.415+0000 7f8d84f31640 1 -- 172.21.15.172:0/3909142619 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582e5e9a1f0 con 0x5582e5f90150 2024-09-10T08:47:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.415+0000 7f8d7ffff640 1 --2- 172.21.15.172:0/3909142619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f81380 0x5582e5f8fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.415+0000 7f8d77fff640 1 --2- 172.21.15.172:0/636950530 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8d6c0a2b80 0x7f8d6c0a5040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8d800fee90 tx=0x7f8d6800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.415+0000 7f8d84f31640 1 --2- 172.21.15.172:0/3909142619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f90150 0x5582e5f92540 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009ca0 tx=0x7f8d780095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae9fa97c2de43227 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.415+0000 7f8d767fc640 1 -- 172.21.15.172:0/3909142619 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78004d40 con 0x5582e5f90150 2024-09-10T08:47:34.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.415+0000 7f8d767fc640 1 -- 172.21.15.172:0/3909142619 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d78036040 con 0x5582e5f90150 2024-09-10T08:47:34.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.415+0000 7f8d767fc640 1 -- 172.21.15.172:0/3909142619 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78004230 con 0x5582e5f90150 2024-09-10T08:47:34.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.416+0000 7f8d88d6d740 1 -- 172.21.15.172:0/3909142619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f90150 msgr2=0x5582e5f92540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.416+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/3909142619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f90150 0x5582e5f92540 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009ca0 tx=0x7f8d780095c0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.416+0000 7f8d88d6d740 1 -- 172.21.15.172:0/3909142619 shutdown_connections 2024-09-10T08:47:34.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.416+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/3909142619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f90150 0x5582e5f92540 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.416+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/3909142619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f81380 0x5582e5f8fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.416+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/3909142619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f80a60 0x5582e5f80e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.416+0000 7f8d88d6d740 1 -- 172.21.15.172:0/3909142619 >> 172.21.15.172:0/3909142619 conn(0x5582e5f7d0f0 msgr2=0x5582e5f7d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:34.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.416+0000 7f8d88d6d740 1 -- 172.21.15.172:0/3909142619 shutdown_connections 2024-09-10T08:47:34.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.416+0000 7f8d88d6d740 1 -- 172.21.15.172:0/3909142619 wait complete. 2024-09-10T08:47:34.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.417+0000 7f8d88d6d740 1 -- start start 2024-09-10T08:47:34.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.417+0000 7f8d88d6d740 1 Processor -- start 2024-09-10T08:47:34.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.417+0000 7f8d88d6d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f80a60 0x5582e5f8d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.417+0000 7f8d88d6d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f81380 0x5582e5f8d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.417+0000 7f8d88d6d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f90150 0x5582e5f8dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.417+0000 7f8d88d6d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5582e5f92d20 con 0x5582e5f81380 2024-09-10T08:47:34.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.417+0000 7f8d88d6d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5582e5ea0f10 con 0x5582e5f80a60 2024-09-10T08:47:34.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.418+0000 7f8d77fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f81380 0x5582e5f8d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.417+0000 7f8d88d6d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5582e5f92ea0 con 0x5582e5f90150 2024-09-10T08:47:34.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.418+0000 7f8d77fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f81380 0x5582e5f8d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57368/0 (socket says 172.21.15.172:57368) 2024-09-10T08:47:34.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.418+0000 7f8d77fff640 1 -- 172.21.15.172:0/1100937709 learned_addr learned my addr 172.21.15.172:0/1100937709 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:34.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.418+0000 7f8d88d6d740 10 client.0 _ll_register_callbacks cb 0x5582e5e69a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:34.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.418+0000 7f8d84f31640 1 --2- 172.21.15.172:0/1100937709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f90150 0x5582e5f8dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.418+0000 7f8d7ffff640 1 --2- 172.21.15.172:0/1100937709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f80a60 0x5582e5f8d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.418+0000 7f8d77fff640 1 -- 172.21.15.172:0/1100937709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f90150 msgr2=0x5582e5f8dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.418+0000 7f8d77fff640 1 --2- 172.21.15.172:0/1100937709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f90150 0x5582e5f8dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.418+0000 7f8d77fff640 1 -- 172.21.15.172:0/1100937709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f80a60 msgr2=0x5582e5f8d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.418+0000 7f8d77fff640 1 --2- 172.21.15.172:0/1100937709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f80a60 0x5582e5f8d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.418+0000 7f8d77fff640 1 -- 172.21.15.172:0/1100937709 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582e5c576a0 con 0x5582e5f81380 2024-09-10T08:47:34.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.418+0000 7f8d77fff640 1 --2- 172.21.15.172:0/1100937709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f81380 0x5582e5f8d590 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8d680116f0 tx=0x7f8d68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc95d9d931e4e6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.418+0000 7f8d5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:34.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.418+0000 7f8d5f7fe640 1 -- 172.21.15.172:0/1100937709 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6801e830 con 0x5582e5f81380 2024-09-10T08:47:34.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.418+0000 7f8d5f7fe640 1 -- 172.21.15.172:0/1100937709 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8d6801e9d0 con 0x5582e5f81380 2024-09-10T08:47:34.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.418+0000 7f8d88d6d740 1 -- 172.21.15.172:0/1100937709 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d78036040 con 0x5582e5f81380 2024-09-10T08:47:34.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.419+0000 7f8d5f7fe640 1 -- 172.21.15.172:0/1100937709 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8d6801ebd0 con 0x5582e5f81380 2024-09-10T08:47:34.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.419+0000 7f8d88d6d740 10 client.15759 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:34.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.419+0000 7f8d88d6d740 1 -- 172.21.15.172:0/1100937709 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5582e5f9e9d0 con 0x5582e5f81380 2024-09-10T08:47:34.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.419+0000 7f8d5f7fe640 1 -- 172.21.15.172:0/1100937709 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d78036040 con 0x5582e5f81380 2024-09-10T08:47:34.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.419+0000 7f8d5f7fe640 1 -- 172.21.15.172:0/1100937709 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8d6801d3b0 con 0x5582e5f81380 2024-09-10T08:47:34.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.419+0000 7f8d88d6d740 10 client.15759 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:34.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.419+0000 7f8d88d6d740 10 client.15759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:34.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.419+0000 7f8d88d6d740 20 client.15759 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:34.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.419+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/1100937709 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5582e5f9fe90 0x5582e5fa2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.419+0000 7f8d88d6d740 4 client.15759 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:34.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.419+0000 7f8d88d6d740 1 -- 172.21.15.172:0/1100937709 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582e5ef37f0 con 0x5582e5f9fe90 2024-09-10T08:47:34.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.419+0000 7f8d7ffff640 1 --2- 172.21.15.172:0/1100937709 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5582e5f9fe90 0x5582e5fa2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.420+0000 7f8d7ffff640 1 --2- 172.21.15.172:0/1100937709 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5582e5f9fe90 0x5582e5fa2350 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e11ec3c3d114d6d server_cookie=d576800d57cb57da in_seq=0 out_seq=0 2024-09-10T08:47:34.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.420+0000 7f8d5f7fe640 10 client.15759 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:34.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.420+0000 7f8d5f7fe640 1 -- 172.21.15.172:0/1100937709 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5582e5ef37f0 con 0x5582e5f9fe90 2024-09-10T08:47:34.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.421+0000 7f8d5f7fe640 10 client.15759 handle_command_reply: tid=0 2024-09-10T08:47:34.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.421+0000 7f8d88d6d740 1 client.15759 shutdown 2024-09-10T08:47:34.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.421+0000 7f8d88d6d740 1 -- 172.21.15.172:0/1100937709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f81380 msgr2=0x5582e5f8d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.421+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/1100937709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f81380 0x5582e5f8d590 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8d680116f0 tx=0x7f8d68004730 comp rx=0 tx=0).stop 2024-09-10T08:47:34.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.421+0000 7f8d88d6d740 1 -- 172.21.15.172:0/1100937709 shutdown_connections 2024-09-10T08:47:34.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.421+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/1100937709 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5582e5f9fe90 0x5582e5fa2350 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.421+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/1100937709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f90150 0x5582e5f8dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.421+0000 7f8d5f7fe640 0 client.15759 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:34.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.421+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/1100937709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f81380 0x5582e5f8d590 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.421+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/1100937709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f80a60 0x5582e5f8d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.421+0000 7f8d88d6d740 1 -- 172.21.15.172:0/1100937709 >> 172.21.15.172:0/1100937709 conn(0x5582e5f7d0f0 msgr2=0x5582e5f82e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:34.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.421+0000 7f8d88d6d740 1 -- 172.21.15.172:0/1100937709 shutdown_connections 2024-09-10T08:47:34.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.421+0000 7f8d88d6d740 1 -- 172.21.15.172:0/1100937709 wait complete. 2024-09-10T08:47:34.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.421+0000 7f8d88d6d740 20 client.15759 trim_cache size 0 max 16384 2024-09-10T08:47:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.438+0000 7f8d88d6d740 1 Processor -- start 2024-09-10T08:47:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.438+0000 7f8d88d6d740 1 -- start start 2024-09-10T08:47:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.438+0000 7f8d88d6d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f80a60 0x5582e5fa7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.438+0000 7f8d88d6d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f81380 0x5582e5fa81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.438+0000 7f8d88d6d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f90150 0x5582e5f8a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.438+0000 7f8d88d6d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8d801aca50 con 0x5582e5f90150 2024-09-10T08:47:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.438+0000 7f8d88d6d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d801ae590 con 0x5582e5f80a60 2024-09-10T08:47:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.438+0000 7f8d88d6d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8d801ad120 con 0x5582e5f81380 2024-09-10T08:47:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.439+0000 7f8d84f31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f90150 0x5582e5f8a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.439+0000 7f8d84f31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f90150 0x5582e5f8a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57382/0 (socket says 172.21.15.172:57382) 2024-09-10T08:47:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.439+0000 7f8d84f31640 1 -- 172.21.15.172:0/1748902294 learned_addr learned my addr 172.21.15.172:0/1748902294 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.439+0000 7f8d7ffff640 1 --2- 172.21.15.172:0/1748902294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f80a60 0x5582e5fa7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.439+0000 7f8d77fff640 1 --2- 172.21.15.172:0/1748902294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f81380 0x5582e5fa81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.439+0000 7f8d84f31640 1 -- 172.21.15.172:0/1748902294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f81380 msgr2=0x5582e5fa81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.439+0000 7f8d84f31640 1 --2- 172.21.15.172:0/1748902294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f81380 0x5582e5fa81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.439+0000 7f8d84f31640 1 -- 172.21.15.172:0/1748902294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f80a60 msgr2=0x5582e5fa7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.439+0000 7f8d84f31640 1 --2- 172.21.15.172:0/1748902294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f80a60 0x5582e5fa7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.439+0000 7f8d84f31640 1 -- 172.21.15.172:0/1748902294 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582e5f8c000 con 0x5582e5f90150 2024-09-10T08:47:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.439+0000 7f8d7ffff640 1 --2- 172.21.15.172:0/1748902294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f80a60 0x5582e5fa7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.439+0000 7f8d84f31640 1 --2- 172.21.15.172:0/1748902294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f90150 0x5582e5f8a2b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5582e5f82600 tx=0x7f8d78031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd6e3a8969fb74d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.439+0000 7f8d75ffb640 1 -- 172.21.15.172:0/1748902294 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78004d40 con 0x5582e5f90150 2024-09-10T08:47:34.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.439+0000 7f8d75ffb640 1 -- 172.21.15.172:0/1748902294 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d7804c070 con 0x5582e5f90150 2024-09-10T08:47:34.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.439+0000 7f8d75ffb640 1 -- 172.21.15.172:0/1748902294 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78004330 con 0x5582e5f90150 2024-09-10T08:47:34.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.440+0000 7f8d88d6d740 1 -- 172.21.15.172:0/1748902294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f90150 msgr2=0x5582e5f8a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.440+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/1748902294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f90150 0x5582e5f8a2b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5582e5f82600 tx=0x7f8d78031670 comp rx=0 tx=0).stop 2024-09-10T08:47:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.440+0000 7f8d88d6d740 1 -- 172.21.15.172:0/1748902294 shutdown_connections 2024-09-10T08:47:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.440+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/1748902294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f90150 0x5582e5f8a2b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.440+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/1748902294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f81380 0x5582e5fa81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.440+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/1748902294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f80a60 0x5582e5fa7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.440+0000 7f8d88d6d740 1 -- 172.21.15.172:0/1748902294 >> 172.21.15.172:0/1748902294 conn(0x5582e5f7d0f0 msgr2=0x5582e5f82e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.440+0000 7f8d88d6d740 1 -- 172.21.15.172:0/1748902294 shutdown_connections 2024-09-10T08:47:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.440+0000 7f8d88d6d740 1 -- 172.21.15.172:0/1748902294 wait complete. 2024-09-10T08:47:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.441+0000 7f8d88d6d740 1 -- start start 2024-09-10T08:47:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.441+0000 7f8d88d6d740 1 Processor -- start 2024-09-10T08:47:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.441+0000 7f8d88d6d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f80a60 0x5582e5f984e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.441+0000 7f8d88d6d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f81380 0x5582e5f98a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.441+0000 7f8d88d6d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f90150 0x5582e5f94a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.441+0000 7f8d88d6d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8d801adec0 con 0x5582e5f90150 2024-09-10T08:47:34.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.441+0000 7f8d88d6d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d801ad7f0 con 0x5582e5f81380 2024-09-10T08:47:34.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.441+0000 7f8d88d6d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8d801aaa30 con 0x5582e5f80a60 2024-09-10T08:47:34.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.441+0000 7f8d88d6d740 10 client.0 _ll_register_callbacks cb 0x7f8d6800b870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:34.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.441+0000 7f8d84f31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f90150 0x5582e5f94a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.441+0000 7f8d84f31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f90150 0x5582e5f94a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57390/0 (socket says 172.21.15.172:57390) 2024-09-10T08:47:34.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.441+0000 7f8d84f31640 1 -- 172.21.15.172:0/3269830634 learned_addr learned my addr 172.21.15.172:0/3269830634 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:34.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.441+0000 7f8d7ffff640 1 --2- 172.21.15.172:0/3269830634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f80a60 0x5582e5f984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.442+0000 7f8d77fff640 1 --2- 172.21.15.172:0/3269830634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f81380 0x5582e5f98a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.442+0000 7f8d84f31640 1 -- 172.21.15.172:0/3269830634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f80a60 msgr2=0x5582e5f984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.442+0000 7f8d84f31640 1 --2- 172.21.15.172:0/3269830634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f80a60 0x5582e5f984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.442+0000 7f8d84f31640 1 -- 172.21.15.172:0/3269830634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f81380 msgr2=0x5582e5f98a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:34.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.442+0000 7f8d84f31640 1 --2- 172.21.15.172:0/3269830634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f81380 0x5582e5f98a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.442+0000 7f8d84f31640 1 -- 172.21.15.172:0/3269830634 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582e5f811b0 con 0x5582e5f90150 2024-09-10T08:47:34.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.442+0000 7f8d77fff640 1 --2- 172.21.15.172:0/3269830634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f81380 0x5582e5f98a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:34.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.442+0000 7f8d84f31640 1 --2- 172.21.15.172:0/3269830634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f90150 0x5582e5f94a70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8d78003c20 tx=0x7f8d78005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ac4560394ef029 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:34.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.442+0000 7f8d5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:34.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.442+0000 7f8d5f7fe640 1 -- 172.21.15.172:0/3269830634 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78057070 con 0x5582e5f90150 2024-09-10T08:47:34.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.442+0000 7f8d5f7fe640 1 -- 172.21.15.172:0/3269830634 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8d78004b40 con 0x5582e5f90150 2024-09-10T08:47:34.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.442+0000 7f8d88d6d740 1 -- 172.21.15.172:0/3269830634 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d7804c070 con 0x5582e5f90150 2024-09-10T08:47:34.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.442+0000 7f8d5f7fe640 1 -- 172.21.15.172:0/3269830634 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8d78004d40 con 0x5582e5f90150 2024-09-10T08:47:34.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.442+0000 7f8d88d6d740 10 client.15765 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:34.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.442+0000 7f8d88d6d740 1 -- 172.21.15.172:0/3269830634 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5582e600b720 con 0x5582e5f90150 2024-09-10T08:47:34.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.443+0000 7f8d5f7fe640 1 -- 172.21.15.172:0/3269830634 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d78036040 con 0x5582e5f90150 2024-09-10T08:47:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.443+0000 7f8d5f7fe640 1 -- 172.21.15.172:0/3269830634 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8d78030070 con 0x5582e5f90150 2024-09-10T08:47:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.443+0000 7f8d88d6d740 10 client.15765 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.443+0000 7f8d88d6d740 10 client.15765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.443+0000 7f8d88d6d740 20 client.15765 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.443+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/3269830634 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5582e5f9fe90 0x5582e600d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.443+0000 7f8d88d6d740 4 client.15765 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:47:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.443+0000 7f8d88d6d740 1 -- 172.21.15.172:0/3269830634 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f8d7804c070 con 0x5582e5f9fe90 2024-09-10T08:47:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.443+0000 7f8d7ffff640 1 --2- 172.21.15.172:0/3269830634 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5582e5f9fe90 0x5582e600d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.443+0000 7f8d7ffff640 1 --2- 172.21.15.172:0/3269830634 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5582e5f9fe90 0x5582e600d950 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=130c3545c0ca8d6d server_cookie=ae03ad64aecd4c3c in_seq=0 out_seq=0 2024-09-10T08:47:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.443+0000 7f8d5f7fe640 10 client.15765 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:34.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.444+0000 7f8d5f7fe640 1 -- 172.21.15.172:0/3269830634 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f8d7804c070 con 0x5582e5f9fe90 2024-09-10T08:47:34.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.444+0000 7f8d5f7fe640 10 client.15765 handle_command_reply: tid=0 2024-09-10T08:47:34.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.444+0000 7f8d88d6d740 1 client.15765 shutdown 2024-09-10T08:47:34.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.445+0000 7f8d88d6d740 1 -- 172.21.15.172:0/3269830634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f90150 msgr2=0x5582e5f94a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.445+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/3269830634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f90150 0x5582e5f94a70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8d78003c20 tx=0x7f8d78005df0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.445+0000 7f8d88d6d740 1 -- 172.21.15.172:0/3269830634 shutdown_connections 2024-09-10T08:47:34.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.445+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/3269830634 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5582e5f9fe90 0x5582e600d950 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.445+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/3269830634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5582e5f90150 0x5582e5f94a70 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.445+0000 7f8d5f7fe640 0 client.15765 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:34.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.445+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/3269830634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582e5f81380 0x5582e5f98a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.445+0000 7f8d88d6d740 1 --2- 172.21.15.172:0/3269830634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5582e5f80a60 0x5582e5f984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.445+0000 7f8d88d6d740 1 -- 172.21.15.172:0/3269830634 >> 172.21.15.172:0/3269830634 conn(0x5582e5f7d0f0 msgr2=0x5582e5f8cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:34.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.445+0000 7f8d88d6d740 1 -- 172.21.15.172:0/3269830634 shutdown_connections 2024-09-10T08:47:34.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.445+0000 7f8d88d6d740 1 -- 172.21.15.172:0/3269830634 wait complete. 2024-09-10T08:47:34.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.445+0000 7f8d88d6d740 20 client.15765 trim_cache size 0 max 16384 2024-09-10T08:47:34.451 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:47:34.451 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T08:47:34.451 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "671967c0-3316-4e0f-8099-ee1d5a03c3fa", 2024-09-10T08:47:34.451 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T08:47:34.451 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:47:34.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.446+0000 7f8d86994640 1 -- 172.21.15.172:0/636950530 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8d6c0a2b80 msgr2=0x7f8d6c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.446+0000 7f8d86994640 1 --2- 172.21.15.172:0/636950530 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8d6c0a2b80 0x7f8d6c0a5040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8d800fee90 tx=0x7f8d6800b390 comp rx=0 tx=0).stop 2024-09-10T08:47:34.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.446+0000 7f8d86994640 1 -- 172.21.15.172:0/636950530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d80102d70 msgr2=0x7f8d80198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:34.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.446+0000 7f8d86994640 1 --2- 172.21.15.172:0/636950530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d80102d70 0x7f8d80198470 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000d950 tx=0x7f8d7000de10 comp rx=0 tx=0).stop 2024-09-10T08:47:34.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.446+0000 7f8d86994640 1 -- 172.21.15.172:0/636950530 shutdown_connections 2024-09-10T08:47:34.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.446+0000 7f8d86994640 1 --2- 172.21.15.172:0/636950530 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8d6c0a2b80 0x7f8d6c0a5040 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.446+0000 7f8d86994640 1 --2- 172.21.15.172:0/636950530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d801082a0 0x7f8d8019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.446+0000 7f8d86994640 1 --2- 172.21.15.172:0/636950530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d80103720 0x7f8d801989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.446+0000 7f8d86994640 1 --2- 172.21.15.172:0/636950530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d80102d70 0x7f8d80198470 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:34.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.446+0000 7f8d86994640 1 -- 172.21.15.172:0/636950530 >> 172.21.15.172:0/636950530 conn(0x7f8d800fc7d0 msgr2=0x7f8d80105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:34.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.446+0000 7f8d86994640 1 -- 172.21.15.172:0/636950530 shutdown_connections 2024-09-10T08:47:34.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:34.446+0000 7f8d86994640 1 -- 172.21.15.172:0/636950530 wait complete. 2024-09-10T08:47:34.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:34 smithi191 ceph-mon[35314]: pgmap v390: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:47:34.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:34 smithi174 ceph-mon[34068]: pgmap v390: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:47:35.225 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:47:35.488 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:36.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.982+0000 7f1aedd23640 1 -- 172.21.15.172:0/777076409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ae8106a70 msgr2=0x7f1ae8106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:36.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.982+0000 7f1aedd23640 1 --2- 172.21.15.172:0/777076409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ae8106a70 0x7f1ae8106e50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009930 tx=0x7f1ad802a420 comp rx=0 tx=0).stop 2024-09-10T08:47:36.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.982+0000 7f1aedd23640 1 -- 172.21.15.172:0/777076409 shutdown_connections 2024-09-10T08:47:36.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.982+0000 7f1aedd23640 1 --2- 172.21.15.172:0/777076409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ae8101410 0x7f1ae810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.982+0000 7f1aedd23640 1 --2- 172.21.15.172:0/777076409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ae8100a70 0x7f1ae8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.982+0000 7f1aedd23640 1 --2- 172.21.15.172:0/777076409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ae8106a70 0x7f1ae8106e50 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.982+0000 7f1aedd23640 1 -- 172.21.15.172:0/777076409 >> 172.21.15.172:0/777076409 conn(0x7f1ae80fc790 msgr2=0x7f1ae80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:36.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.982+0000 7f1aedd23640 1 -- 172.21.15.172:0/777076409 shutdown_connections 2024-09-10T08:47:36.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.982+0000 7f1aedd23640 1 -- 172.21.15.172:0/777076409 wait complete. 2024-09-10T08:47:36.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.983+0000 7f1aedd23640 1 Processor -- start 2024-09-10T08:47:36.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.983+0000 7f1aedd23640 1 -- start start 2024-09-10T08:47:36.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.983+0000 7f1aedd23640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ae8100a70 0x7f1ae81972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.983+0000 7f1aedd23640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ae8101410 0x7f1ae81977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.983+0000 7f1aedd23640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ae8106a70 0x7f1ae819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.983+0000 7f1aedd23640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae8110ce0 con 0x7f1ae8100a70 2024-09-10T08:47:36.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.983+0000 7f1aedd23640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae8110b60 con 0x7f1ae8101410 2024-09-10T08:47:36.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.983+0000 7f1aedd23640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae8110e60 con 0x7f1ae8106a70 2024-09-10T08:47:36.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.984+0000 7f1ae77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ae8100a70 0x7f1ae81972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.984+0000 7f1ae77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ae8100a70 0x7f1ae81972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57408/0 (socket says 172.21.15.172:57408) 2024-09-10T08:47:36.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.984+0000 7f1ae77fe640 1 -- 172.21.15.172:0/4040262366 learned_addr learned my addr 172.21.15.172:0/4040262366 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:36.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.984+0000 7f1ae6ffd640 1 --2- 172.21.15.172:0/4040262366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ae8101410 0x7f1ae81977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.984+0000 7f1ae7fff640 1 --2- 172.21.15.172:0/4040262366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ae8106a70 0x7f1ae819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.984+0000 7f1ae77fe640 1 -- 172.21.15.172:0/4040262366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ae8106a70 msgr2=0x7f1ae819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:36.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.984+0000 7f1ae77fe640 1 --2- 172.21.15.172:0/4040262366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ae8106a70 0x7f1ae819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.984+0000 7f1ae77fe640 1 -- 172.21.15.172:0/4040262366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ae8101410 msgr2=0x7f1ae81977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.984+0000 7f1ae77fe640 1 --2- 172.21.15.172:0/4040262366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ae8101410 0x7f1ae81977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.984+0000 7f1ae77fe640 1 -- 172.21.15.172:0/4040262366 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae819c250 con 0x7f1ae8100a70 2024-09-10T08:47:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.984+0000 7f1ae7fff640 1 --2- 172.21.15.172:0/4040262366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ae8106a70 0x7f1ae819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.984+0000 7f1ae77fe640 1 --2- 172.21.15.172:0/4040262366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ae8100a70 0x7f1ae81972a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8004770 tx=0x7f1ad80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.984+0000 7f1ae4ff9640 1 -- 172.21.15.172:0/4040262366 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad803f070 con 0x7f1ae8100a70 2024-09-10T08:47:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.984+0000 7f1ae4ff9640 1 -- 172.21.15.172:0/4040262366 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ad803b460 con 0x7f1ae8100a70 2024-09-10T08:47:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.984+0000 7f1ae4ff9640 1 -- 172.21.15.172:0/4040262366 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8035070 con 0x7f1ae8100a70 2024-09-10T08:47:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.984+0000 7f1aedd23640 1 -- 172.21.15.172:0/4040262366 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae819c4e0 con 0x7f1ae8100a70 2024-09-10T08:47:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.984+0000 7f1aedd23640 1 -- 172.21.15.172:0/4040262366 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae819c900 con 0x7f1ae8100a70 2024-09-10T08:47:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.987+0000 7f1ae4ff9640 1 -- 172.21.15.172:0/4040262366 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1ad803b610 con 0x7f1ae8100a70 2024-09-10T08:47:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.987+0000 7f1ae4ff9640 1 --2- 172.21.15.172:0/4040262366 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1ad00a2b80 0x7f1ad00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.987+0000 7f1ae6ffd640 1 --2- 172.21.15.172:0/4040262366 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1ad00a2b80 0x7f1ad00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.987+0000 7f1ae4ff9640 1 -- 172.21.15.172:0/4040262366 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1ad80ef760 con 0x7f1ae8100a70 2024-09-10T08:47:36.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.988+0000 7f1ae6ffd640 1 --2- 172.21.15.172:0/4040262366 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1ad00a2b80 0x7f1ad00a5040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80fee40 tx=0x7f1adc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:36.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.988+0000 7f1af00fc740 1 Processor -- start 2024-09-10T08:47:36.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.988+0000 7f1af00fc740 1 -- start start 2024-09-10T08:47:36.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.988+0000 7f1af00fc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.988+0000 7f1af00fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34cfdc0 0x55cbc34da140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.988+0000 7f1af00fc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34da680 0x55cbc34dca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.988+0000 7f1af00fc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cbc34352e0 con 0x55cbc34da680 2024-09-10T08:47:36.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.988+0000 7f1af00fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cbc33dcce0 con 0x55cbc34cfdc0 2024-09-10T08:47:36.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.988+0000 7f1af00fc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55cbc34312a0 con 0x55cbc34cf4a0 2024-09-10T08:47:36.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.988+0000 7f1ae7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34da680 0x55cbc34dca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.989+0000 7f1ae7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34da680 0x55cbc34dca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57418/0 (socket says 172.21.15.172:57418) 2024-09-10T08:47:36.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.989+0000 7f1ae7fff640 1 -- 172.21.15.172:0/666380164 learned_addr learned my addr 172.21.15.172:0/666380164 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:36.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.989+0000 7f1ae77fe640 1 --2- 172.21.15.172:0/666380164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.989+0000 7f1ae6ffd640 1 --2- 172.21.15.172:0/666380164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34cfdc0 0x55cbc34da140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.989+0000 7f1ae7fff640 1 -- 172.21.15.172:0/666380164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 msgr2=0x55cbc34cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.989+0000 7f1ae7fff640 1 --2- 172.21.15.172:0/666380164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.989+0000 7f1ae7fff640 1 -- 172.21.15.172:0/666380164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34cfdc0 msgr2=0x55cbc34da140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.989+0000 7f1ae7fff640 1 --2- 172.21.15.172:0/666380164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34cfdc0 0x55cbc34da140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.989+0000 7f1ae7fff640 1 -- 172.21.15.172:0/666380164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbc33c1440 con 0x55cbc34da680 2024-09-10T08:47:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.989+0000 7f1ae6ffd640 1 --2- 172.21.15.172:0/666380164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34cfdc0 0x55cbc34da140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.989+0000 7f1ae7fff640 1 --2- 172.21.15.172:0/666380164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34da680 0x55cbc34dca70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009e40 tx=0x7f1ae000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91ae5cf25631c14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.989+0000 7f1acdffb640 1 -- 172.21.15.172:0/666380164 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0004d20 con 0x55cbc34da680 2024-09-10T08:47:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.989+0000 7f1acdffb640 1 -- 172.21.15.172:0/666380164 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ae00054d0 con 0x55cbc34da680 2024-09-10T08:47:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.989+0000 7f1acdffb640 1 -- 172.21.15.172:0/666380164 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0011800 con 0x55cbc34da680 2024-09-10T08:47:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.990+0000 7f1af00fc740 1 -- 172.21.15.172:0/666380164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34da680 msgr2=0x55cbc34dca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:36.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.990+0000 7f1af00fc740 1 --2- 172.21.15.172:0/666380164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34da680 0x55cbc34dca70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009e40 tx=0x7f1ae000f730 comp rx=0 tx=0).stop 2024-09-10T08:47:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.990+0000 7f1af00fc740 1 -- 172.21.15.172:0/666380164 shutdown_connections 2024-09-10T08:47:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.990+0000 7f1af00fc740 1 --2- 172.21.15.172:0/666380164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34da680 0x55cbc34dca70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.990+0000 7f1af00fc740 1 --2- 172.21.15.172:0/666380164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34cfdc0 0x55cbc34da140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.990+0000 7f1af00fc740 1 --2- 172.21.15.172:0/666380164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.990+0000 7f1af00fc740 1 -- 172.21.15.172:0/666380164 >> 172.21.15.172:0/666380164 conn(0x55cbc34cba30 msgr2=0x55cbc34cbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.990+0000 7f1af00fc740 1 -- 172.21.15.172:0/666380164 shutdown_connections 2024-09-10T08:47:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.990+0000 7f1af00fc740 1 -- 172.21.15.172:0/666380164 wait complete. 2024-09-10T08:47:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.991+0000 7f1af00fc740 1 -- start start 2024-09-10T08:47:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.991+0000 7f1af00fc740 1 Processor -- start 2024-09-10T08:47:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.991+0000 7f1af00fc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34d9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.991+0000 7f1af00fc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cfdc0 0x55cbc34d7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.991+0000 7f1af00fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34da680 0x55cbc34d77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.991+0000 7f1af00fc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cbc34df460 con 0x55cbc34cfdc0 2024-09-10T08:47:36.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.991+0000 7f1af00fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cbc33f0050 con 0x55cbc34da680 2024-09-10T08:47:36.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.991+0000 7f1af00fc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55cbc34df5e0 con 0x55cbc34cf4a0 2024-09-10T08:47:36.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.991+0000 7f1af00fc740 10 client.0 _ll_register_callbacks cb 0x55cbc3430160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:36.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.991+0000 7f1ae6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cfdc0 0x55cbc34d7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.991+0000 7f1ae77fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34d9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.991+0000 7f1ae77fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34d9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:35442/0 (socket says 172.21.15.172:35442) 2024-09-10T08:47:36.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.991+0000 7f1ae77fe640 1 -- 172.21.15.172:0/2174087625 learned_addr learned my addr 172.21.15.172:0/2174087625 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:36.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.991+0000 7f1ae6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cfdc0 0x55cbc34d7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57432/0 (socket says 172.21.15.172:57432) 2024-09-10T08:47:36.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.992+0000 7f1ae7fff640 1 --2- 172.21.15.172:0/2174087625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34da680 0x55cbc34d77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.992+0000 7f1ae77fe640 1 -- 172.21.15.172:0/2174087625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34da680 msgr2=0x55cbc34d77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:36.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.992+0000 7f1ae77fe640 1 --2- 172.21.15.172:0/2174087625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34da680 0x55cbc34d77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.992+0000 7f1ae77fe640 1 -- 172.21.15.172:0/2174087625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cfdc0 msgr2=0x55cbc34d7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:36.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.992+0000 7f1ae77fe640 1 --2- 172.21.15.172:0/2174087625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cfdc0 0x55cbc34d7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.992+0000 7f1ae77fe640 1 -- 172.21.15.172:0/2174087625 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbc31a63c0 con 0x55cbc34cf4a0 2024-09-10T08:47:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.992+0000 7f1ae6ffd640 1 --2- 172.21.15.172:0/2174087625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cfdc0 0x55cbc34d7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.992+0000 7f1ae77fe640 1 --2- 172.21.15.172:0/2174087625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34d9fb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80ecf00 tx=0x7f1ad80045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3b70138f8893a99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.992+0000 7f1ac6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:47:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.992+0000 7f1ac6ffd640 1 -- 172.21.15.172:0/2174087625 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad80d4e60 con 0x55cbc34cf4a0 2024-09-10T08:47:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.992+0000 7f1af00fc740 1 -- 172.21.15.172:0/2174087625 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ae00054d0 con 0x55cbc34cf4a0 2024-09-10T08:47:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.992+0000 7f1ac6ffd640 1 -- 172.21.15.172:0/2174087625 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1ad803b940 con 0x55cbc34cf4a0 2024-09-10T08:47:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.992+0000 7f1ac6ffd640 1 -- 172.21.15.172:0/2174087625 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1ad8048040 con 0x55cbc34cf4a0 2024-09-10T08:47:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.992+0000 7f1af00fc740 10 client.25183 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.992+0000 7f1af00fc740 1 -- 172.21.15.172:0/2174087625 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cbc34df510 con 0x55cbc34cf4a0 2024-09-10T08:47:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.993+0000 7f1ac6ffd640 1 -- 172.21.15.172:0/2174087625 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ae00054d0 con 0x55cbc34cf4a0 2024-09-10T08:47:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.993+0000 7f1ac6ffd640 1 -- 172.21.15.172:0/2174087625 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1ad804b060 con 0x55cbc34cf4a0 2024-09-10T08:47:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.993+0000 7f1af00fc740 10 client.25183 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.993+0000 7f1af00fc740 10 client.25183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:36.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.993+0000 7f1af00fc740 20 client.25183 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.993+0000 7f1af00fc740 1 --2- 172.21.15.172:0/2174087625 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cbc34ee7e0 0x55cbc34f0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.993+0000 7f1af00fc740 4 client.25183 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.993+0000 7f1af00fc740 1 -- 172.21.15.172:0/2174087625 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbc3442850 con 0x55cbc34ee7e0 2024-09-10T08:47:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.993+0000 7f1ae6ffd640 1 --2- 172.21.15.172:0/2174087625 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cbc34ee7e0 0x55cbc34f0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.993+0000 7f1ac6ffd640 10 client.25183 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.993+0000 7f1ae6ffd640 1 --2- 172.21.15.172:0/2174087625 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cbc34ee7e0 0x55cbc34f0bd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e0c70fc4d6aa16f server_cookie=4e89cd913807f612 in_seq=0 out_seq=0 2024-09-10T08:47:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.994+0000 7f1ac6ffd640 1 -- 172.21.15.172:0/2174087625 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbc3442850 con 0x55cbc34ee7e0 2024-09-10T08:47:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.994+0000 7f1ac6ffd640 10 client.25183 handle_command_reply: tid=0 2024-09-10T08:47:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.994+0000 7f1af00fc740 1 client.25183 shutdown 2024-09-10T08:47:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.994+0000 7f1af00fc740 1 -- 172.21.15.172:0/2174087625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 msgr2=0x55cbc34d9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.994+0000 7f1af00fc740 1 --2- 172.21.15.172:0/2174087625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34d9fb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80ecf00 tx=0x7f1ad80045a0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.994+0000 7f1af00fc740 1 -- 172.21.15.172:0/2174087625 shutdown_connections 2024-09-10T08:47:36.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.994+0000 7f1af00fc740 1 --2- 172.21.15.172:0/2174087625 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cbc34ee7e0 0x55cbc34f0bd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.994+0000 7f1af00fc740 1 --2- 172.21.15.172:0/2174087625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34da680 0x55cbc34d77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.994+0000 7f1af00fc740 1 --2- 172.21.15.172:0/2174087625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cfdc0 0x55cbc34d7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.994+0000 7f1af00fc740 1 --2- 172.21.15.172:0/2174087625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34d9fb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.994+0000 7f1ac6ffd640 0 client.25183 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:36.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.994+0000 7f1af00fc740 1 -- 172.21.15.172:0/2174087625 >> 172.21.15.172:0/2174087625 conn(0x55cbc34cba30 msgr2=0x55cbc34d1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:36.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.994+0000 7f1af00fc740 1 -- 172.21.15.172:0/2174087625 shutdown_connections 2024-09-10T08:47:36.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.995+0000 7f1af00fc740 1 -- 172.21.15.172:0/2174087625 wait complete. 2024-09-10T08:47:36.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:35.995+0000 7f1af00fc740 20 client.25183 trim_cache size 0 max 16384 2024-09-10T08:47:36.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1af00fc740 1 Processor -- start 2024-09-10T08:47:36.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1af00fc740 1 -- start start 2024-09-10T08:47:36.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1af00fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34cba30 0x55cbc34e86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1af00fc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34d7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1af00fc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cfdc0 0x55cbc34d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1af00fc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81ac940 con 0x55cbc34cfdc0 2024-09-10T08:47:36.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1af00fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81ae480 con 0x55cbc34cba30 2024-09-10T08:47:36.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1af00fc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81ad010 con 0x55cbc34cf4a0 2024-09-10T08:47:36.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1ae7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cfdc0 0x55cbc34d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1ae7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cfdc0 0x55cbc34d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57446/0 (socket says 172.21.15.172:57446) 2024-09-10T08:47:36.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1ae7fff640 1 -- 172.21.15.172:0/2269248187 learned_addr learned my addr 172.21.15.172:0/2269248187 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:36.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1ae6ffd640 1 --2- 172.21.15.172:0/2269248187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34d7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1ae77fe640 1 --2- 172.21.15.172:0/2269248187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34cba30 0x55cbc34e86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1ae7fff640 1 -- 172.21.15.172:0/2269248187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 msgr2=0x55cbc34d7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:36.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1ae7fff640 1 --2- 172.21.15.172:0/2269248187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34d7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1ae7fff640 1 -- 172.21.15.172:0/2269248187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34cba30 msgr2=0x55cbc34e86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:36.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1ae7fff640 1 --2- 172.21.15.172:0/2269248187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34cba30 0x55cbc34e86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1ae7fff640 1 -- 172.21.15.172:0/2269248187 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbc34da320 con 0x55cbc34cfdc0 2024-09-10T08:47:36.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1ae77fe640 1 --2- 172.21.15.172:0/2269248187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34cba30 0x55cbc34e86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:36.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.012+0000 7f1ae6ffd640 1 --2- 172.21.15.172:0/2269248187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34d7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:36.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.013+0000 7f1ae7fff640 1 --2- 172.21.15.172:0/2269248187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cfdc0 0x55cbc34d27f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009710 tx=0x7f1ae0009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18bfe0fad797d498 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:36.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.013+0000 7f1acd7fa640 1 -- 172.21.15.172:0/2269248187 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0028070 con 0x55cbc34cfdc0 2024-09-10T08:47:36.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.013+0000 7f1acd7fa640 1 -- 172.21.15.172:0/2269248187 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ae0005040 con 0x55cbc34cfdc0 2024-09-10T08:47:36.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.013+0000 7f1acd7fa640 1 -- 172.21.15.172:0/2269248187 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae00236d0 con 0x55cbc34cfdc0 2024-09-10T08:47:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.014+0000 7f1af00fc740 1 -- 172.21.15.172:0/2269248187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cfdc0 msgr2=0x55cbc34d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:36.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.014+0000 7f1af00fc740 1 --2- 172.21.15.172:0/2269248187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cfdc0 0x55cbc34d27f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009710 tx=0x7f1ae0009b10 comp rx=0 tx=0).stop 2024-09-10T08:47:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.014+0000 7f1af00fc740 1 -- 172.21.15.172:0/2269248187 shutdown_connections 2024-09-10T08:47:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.014+0000 7f1af00fc740 1 --2- 172.21.15.172:0/2269248187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cfdc0 0x55cbc34d27f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.014+0000 7f1af00fc740 1 --2- 172.21.15.172:0/2269248187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34d7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.014+0000 7f1af00fc740 1 --2- 172.21.15.172:0/2269248187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34cba30 0x55cbc34e86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.014+0000 7f1af00fc740 1 -- 172.21.15.172:0/2269248187 >> 172.21.15.172:0/2269248187 conn(0x55cbc34d7900 msgr2=0x55cbc34d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.014+0000 7f1af00fc740 1 -- 172.21.15.172:0/2269248187 shutdown_connections 2024-09-10T08:47:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.014+0000 7f1af00fc740 1 -- 172.21.15.172:0/2269248187 wait complete. 2024-09-10T08:47:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.014+0000 7f1af00fc740 1 -- start start 2024-09-10T08:47:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.014+0000 7f1af00fc740 1 Processor -- start 2024-09-10T08:47:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.014+0000 7f1af00fc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cba30 0x55cbc34e5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1af00fc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34e0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1af00fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34cfdc0 0x55cbc34e0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1af00fc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81addb0 con 0x55cbc34cba30 2024-09-10T08:47:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1af00fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81ad6e0 con 0x55cbc34cfdc0 2024-09-10T08:47:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1af00fc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae81aa920 con 0x55cbc34cf4a0 2024-09-10T08:47:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1af00fc740 10 client.0 _ll_register_callbacks cb 0x7f1ad8009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1ae77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cba30 0x55cbc34e5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1ae77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cba30 0x55cbc34e5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57454/0 (socket says 172.21.15.172:57454) 2024-09-10T08:47:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1ae77fe640 1 -- 172.21.15.172:0/4225000932 learned_addr learned my addr 172.21.15.172:0/4225000932 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1ae7fff640 1 --2- 172.21.15.172:0/4225000932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34cfdc0 0x55cbc34e0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1ae6ffd640 1 --2- 172.21.15.172:0/4225000932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34e0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1ae77fe640 1 -- 172.21.15.172:0/4225000932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 msgr2=0x55cbc34e0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1ae77fe640 1 --2- 172.21.15.172:0/4225000932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34e0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1ae77fe640 1 -- 172.21.15.172:0/4225000932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34cfdc0 msgr2=0x55cbc34e0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1ae77fe640 1 --2- 172.21.15.172:0/4225000932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34cfdc0 0x55cbc34e0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1ae77fe640 1 -- 172.21.15.172:0/4225000932 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbc3558200 con 0x55cbc34cba30 2024-09-10T08:47:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1ae77fe640 1 --2- 172.21.15.172:0/4225000932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cba30 0x55cbc34e5130 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8000c00 tx=0x7f1ad80b8500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75346867b2bef34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1ac6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1ac6ffd640 1 -- 172.21.15.172:0/4225000932 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad80d4e60 con 0x55cbc34cba30 2024-09-10T08:47:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1ac6ffd640 1 -- 172.21.15.172:0/4225000932 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1ad80bb2c0 con 0x55cbc34cba30 2024-09-10T08:47:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.015+0000 7f1af00fc740 1 -- 172.21.15.172:0/4225000932 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ae0005040 con 0x55cbc34cba30 2024-09-10T08:47:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.016+0000 7f1ac6ffd640 1 -- 172.21.15.172:0/4225000932 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1ad80c48d0 con 0x55cbc34cba30 2024-09-10T08:47:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.016+0000 7f1af00fc740 10 client.15783 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.016+0000 7f1af00fc740 1 -- 172.21.15.172:0/4225000932 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cbc35593d0 con 0x55cbc34cba30 2024-09-10T08:47:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.016+0000 7f1ac6ffd640 1 -- 172.21.15.172:0/4225000932 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ae0005040 con 0x55cbc34cba30 2024-09-10T08:47:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.016+0000 7f1ac6ffd640 1 -- 172.21.15.172:0/4225000932 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1ad809aad0 con 0x55cbc34cba30 2024-09-10T08:47:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.016+0000 7f1af00fc740 10 client.15783 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:36.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.016+0000 7f1af00fc740 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.016+0000 7f1af00fc740 20 client.15783 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.016+0000 7f1af00fc740 1 --2- 172.21.15.172:0/4225000932 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cbc34da680 0x55cbc355b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.016+0000 7f1af00fc740 4 client.15783 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:47:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.016+0000 7f1af00fc740 1 -- 172.21.15.172:0/4225000932 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cbc343cd60 con 0x55cbc34da680 2024-09-10T08:47:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.016+0000 7f1ae6ffd640 1 --2- 172.21.15.172:0/4225000932 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cbc34da680 0x55cbc355b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.017+0000 7f1ae6ffd640 1 --2- 172.21.15.172:0/4225000932 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cbc34da680 0x55cbc355b8c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e633e8029abe167d server_cookie=b2e930308b13843a in_seq=0 out_seq=0 2024-09-10T08:47:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.017+0000 7f1ac6ffd640 10 client.15783 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.017+0000 7f1ac6ffd640 1 -- 172.21.15.172:0/4225000932 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cbc343cd60 con 0x55cbc34da680 2024-09-10T08:47:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.017+0000 7f1ac6ffd640 10 client.15783 handle_command_reply: tid=0 2024-09-10T08:47:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.017+0000 7f1af00fc740 1 client.15783 shutdown 2024-09-10T08:47:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.017+0000 7f1af00fc740 1 -- 172.21.15.172:0/4225000932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cba30 msgr2=0x55cbc34e5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.017+0000 7f1af00fc740 1 --2- 172.21.15.172:0/4225000932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cba30 0x55cbc34e5130 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8000c00 tx=0x7f1ad80b8500 comp rx=0 tx=0).stop 2024-09-10T08:47:36.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.017+0000 7f1af00fc740 1 -- 172.21.15.172:0/4225000932 shutdown_connections 2024-09-10T08:47:36.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.017+0000 7f1af00fc740 1 --2- 172.21.15.172:0/4225000932 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cbc34da680 0x55cbc355b8c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.017+0000 7f1af00fc740 1 --2- 172.21.15.172:0/4225000932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbc34cfdc0 0x55cbc34e0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.017+0000 7f1ac6ffd640 0 client.15783 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:36.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.017+0000 7f1af00fc740 1 --2- 172.21.15.172:0/4225000932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cbc34cf4a0 0x55cbc34e0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.018+0000 7f1af00fc740 1 --2- 172.21.15.172:0/4225000932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cbc34cba30 0x55cbc34e5130 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.018+0000 7f1af00fc740 1 -- 172.21.15.172:0/4225000932 >> 172.21.15.172:0/4225000932 conn(0x55cbc34d7900 msgr2=0x55cbc34d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:36.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.018+0000 7f1af00fc740 1 -- 172.21.15.172:0/4225000932 shutdown_connections 2024-09-10T08:47:36.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.018+0000 7f1af00fc740 1 -- 172.21.15.172:0/4225000932 wait complete. 2024-09-10T08:47:36.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.018+0000 7f1af00fc740 20 client.15783 trim_cache size 0 max 16384 2024-09-10T08:47:36.026 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:47:36.027 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:47:36.027 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T08:47:36.027 INFO:teuthology.orchestra.run.smithi172.stdout: "671967c0-3316-4e0f-8099-ee1d5a03c3fa": { 2024-09-10T08:47:36.027 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T08:47:36.027 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "671967c0-3316-4e0f-8099-ee1d5a03c3fa", 2024-09-10T08:47:36.027 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T08:47:36.027 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:47:36.027 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:47:36.027 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:47:36.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.019+0000 7f1aedd23640 1 -- 172.21.15.172:0/4040262366 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1ad00a2b80 msgr2=0x7f1ad00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:36.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.019+0000 7f1aedd23640 1 --2- 172.21.15.172:0/4040262366 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1ad00a2b80 0x7f1ad00a5040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80fee40 tx=0x7f1adc00a2d0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.019+0000 7f1aedd23640 1 -- 172.21.15.172:0/4040262366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ae8100a70 msgr2=0x7f1ae81972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:36.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.019+0000 7f1aedd23640 1 --2- 172.21.15.172:0/4040262366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ae8100a70 0x7f1ae81972a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8004770 tx=0x7f1ad80042f0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.019+0000 7f1aedd23640 1 -- 172.21.15.172:0/4040262366 shutdown_connections 2024-09-10T08:47:36.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.019+0000 7f1aedd23640 1 --2- 172.21.15.172:0/4040262366 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1ad00a2b80 0x7f1ad00a5040 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.019+0000 7f1aedd23640 1 --2- 172.21.15.172:0/4040262366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ae8106a70 0x7f1ae819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.019+0000 7f1aedd23640 1 --2- 172.21.15.172:0/4040262366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ae8101410 0x7f1ae81977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.019+0000 7f1aedd23640 1 --2- 172.21.15.172:0/4040262366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ae8100a70 0x7f1ae81972a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:36.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.019+0000 7f1aedd23640 1 -- 172.21.15.172:0/4040262366 >> 172.21.15.172:0/4040262366 conn(0x7f1ae80fc790 msgr2=0x7f1ae80fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:36.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.019+0000 7f1aedd23640 1 -- 172.21.15.172:0/4040262366 shutdown_connections 2024-09-10T08:47:36.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:36.019+0000 7f1aedd23640 1 -- 172.21.15.172:0/4040262366 wait complete. 2024-09-10T08:47:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:36 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T08:47:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:36 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:47:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:36 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:47:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:36 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:47:36.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:36 smithi191 ceph-mon[35314]: pgmap v391: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T08:47:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:36 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T08:47:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:36 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:47:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:36 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:47:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:36 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:47:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:36 smithi172 ceph-mon[30854]: pgmap v391: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T08:47:36.728 INFO:tasks.cephfs.filesystem:scrub status for tag:671967c0-3316-4e0f-8099-ee1d5a03c3fa - {'path': '/', 'tag': '671967c0-3316-4e0f-8099-ee1d5a03c3fa', 'options': 'recursive,force'} 2024-09-10T08:47:36.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:36 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T08:47:36.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:36 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:47:36.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:36 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:47:36.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:36 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:47:36.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:36 smithi174 ceph-mon[34068]: pgmap v391: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T08:47:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.311+0000 7f64b28dd640 1 -- 172.21.15.172:0/402485276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64ac101c30 msgr2=0x7f64ac1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.311+0000 7f64b28dd640 1 --2- 172.21.15.172:0/402485276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64ac101c30 0x7f64ac1020b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009900 tx=0x7f64a002a410 comp rx=0 tx=0).stop 2024-09-10T08:47:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.311+0000 7f64b28dd640 1 -- 172.21.15.172:0/402485276 shutdown_connections 2024-09-10T08:47:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.311+0000 7f64b28dd640 1 --2- 172.21.15.172:0/402485276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64ac1025f0 0x7f64ac108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.311+0000 7f64b28dd640 1 --2- 172.21.15.172:0/402485276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64ac101c30 0x7f64ac1020b0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.311+0000 7f64b28dd640 1 --2- 172.21.15.172:0/402485276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64ac100a40 0x7f64ac100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.311+0000 7f64b28dd640 1 -- 172.21.15.172:0/402485276 >> 172.21.15.172:0/402485276 conn(0x7f64ac0fc1e0 msgr2=0x7f64ac0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.311+0000 7f64b28dd640 1 -- 172.21.15.172:0/402485276 shutdown_connections 2024-09-10T08:47:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.311+0000 7f64b28dd640 1 -- 172.21.15.172:0/402485276 wait complete. 2024-09-10T08:47:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.311+0000 7f64b28dd640 1 Processor -- start 2024-09-10T08:47:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.311+0000 7f64b28dd640 1 -- start start 2024-09-10T08:47:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64b28dd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64ac100a40 0x7f64ac196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64b28dd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64ac101c30 0x7f64ac1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64b28dd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64ac1025f0 0x7f64ac19e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64b28dd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac10c5c0 con 0x7f64ac100a40 2024-09-10T08:47:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64b28dd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac10c440 con 0x7f64ac101c30 2024-09-10T08:47:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64b28dd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac10c740 con 0x7f64ac1025f0 2024-09-10T08:47:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64abfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64ac100a40 0x7f64ac196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64abfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64ac100a40 0x7f64ac196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55600/0 (socket says 172.21.15.172:55600) 2024-09-10T08:47:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64abfff640 1 -- 172.21.15.172:0/2923798174 learned_addr learned my addr 172.21.15.172:0/2923798174 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64b0e7a640 1 --2- 172.21.15.172:0/2923798174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64ac1025f0 0x7f64ac19e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64ab7fe640 1 --2- 172.21.15.172:0/2923798174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64ac101c30 0x7f64ac1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64abfff640 1 -- 172.21.15.172:0/2923798174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64ac1025f0 msgr2=0x7f64ac19e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64abfff640 1 --2- 172.21.15.172:0/2923798174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64ac1025f0 0x7f64ac19e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64abfff640 1 -- 172.21.15.172:0/2923798174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64ac101c30 msgr2=0x7f64ac1971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64abfff640 1 --2- 172.21.15.172:0/2923798174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64ac101c30 0x7f64ac1971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64abfff640 1 -- 172.21.15.172:0/2923798174 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac19e870 con 0x7f64ac100a40 2024-09-10T08:47:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64ab7fe640 1 --2- 172.21.15.172:0/2923798174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64ac101c30 0x7f64ac1971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.312+0000 7f64abfff640 1 --2- 172.21.15.172:0/2923798174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64ac100a40 0x7f64ac196c60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f649c00e9c0 tx=0x7f649c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.313+0000 7f64a97fa640 1 -- 172.21.15.172:0/2923798174 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c00cd80 con 0x7f64ac100a40 2024-09-10T08:47:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.313+0000 7f64a97fa640 1 -- 172.21.15.172:0/2923798174 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f649c0035f0 con 0x7f64ac100a40 2024-09-10T08:47:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.313+0000 7f64a97fa640 1 -- 172.21.15.172:0/2923798174 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649c010690 con 0x7f64ac100a40 2024-09-10T08:47:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.313+0000 7f64b28dd640 1 -- 172.21.15.172:0/2923798174 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ac19eb60 con 0x7f64ac100a40 2024-09-10T08:47:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.313+0000 7f64b28dd640 1 -- 172.21.15.172:0/2923798174 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ac19f060 con 0x7f64ac100a40 2024-09-10T08:47:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.316+0000 7f64a97fa640 1 -- 172.21.15.172:0/2923798174 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f649c0037a0 con 0x7f64ac100a40 2024-09-10T08:47:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.316+0000 7f64b5675740 1 Processor -- start 2024-09-10T08:47:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.316+0000 7f64b5675740 1 -- start start 2024-09-10T08:47:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.316+0000 7f64b5675740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 0x55994f0b8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.316+0000 7f64b5675740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0b94f0 0x55994f0bdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.316+0000 7f64b5675740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0c0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.316+0000 7f64b5675740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55994f01a2f0 con 0x55994f0be580 2024-09-10T08:47:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.316+0000 7f64b5675740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55994efc2270 con 0x55994f0b94f0 2024-09-10T08:47:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.316+0000 7f64b5675740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55994f0162b0 con 0x55994f0b8bb0 2024-09-10T08:47:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.316+0000 7f64ab7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0c0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.316+0000 7f64ab7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0c0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55612/0 (socket says 172.21.15.172:55612) 2024-09-10T08:47:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.316+0000 7f64ab7fe640 1 -- 172.21.15.172:0/1254180357 learned_addr learned my addr 172.21.15.172:0/1254180357 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.316+0000 7f64b0e7a640 1 --2- 172.21.15.172:0/1254180357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 0x55994f0b8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.316+0000 7f64abfff640 1 --2- 172.21.15.172:0/1254180357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0b94f0 0x55994f0bdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.317+0000 7f64ab7fe640 1 -- 172.21.15.172:0/1254180357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 msgr2=0x55994f0b8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.317+0000 7f64a97fa640 1 --2- 172.21.15.172:0/2923798174 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f64940a2c30 0x7f64940a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.317+0000 7f64ab7fe640 1 --2- 172.21.15.172:0/1254180357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 0x55994f0b8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.317+0000 7f64ab7fe640 1 -- 172.21.15.172:0/1254180357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0b94f0 msgr2=0x55994f0bdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.317+0000 7f64ab7fe640 1 --2- 172.21.15.172:0/1254180357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0b94f0 0x55994f0bdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.317+0000 7f64ab7fe640 1 -- 172.21.15.172:0/1254180357 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55994efc1340 con 0x55994f0be580 2024-09-10T08:47:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.317+0000 7f64b0e7a640 1 --2- 172.21.15.172:0/2923798174 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f64940a2c30 0x7f64940a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.317+0000 7f64a97fa640 1 -- 172.21.15.172:0/2923798174 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f649c0cfaa0 con 0x7f64ac100a40 2024-09-10T08:47:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.317+0000 7f64ab7fe640 1 --2- 172.21.15.172:0/1254180357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0c0a40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f64a0005ab0 tx=0x7f64a00042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae8d43e4beb6df3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.317+0000 7f64927fc640 1 -- 172.21.15.172:0/1254180357 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a003f070 con 0x55994f0be580 2024-09-10T08:47:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.317+0000 7f64927fc640 1 -- 172.21.15.172:0/1254180357 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64a00044d0 con 0x55994f0be580 2024-09-10T08:47:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.317+0000 7f64927fc640 1 -- 172.21.15.172:0/1254180357 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0035030 con 0x55994f0be580 2024-09-10T08:47:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.318+0000 7f64b5675740 1 -- 172.21.15.172:0/1254180357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 msgr2=0x55994f0c0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.318+0000 7f64b5675740 1 --2- 172.21.15.172:0/1254180357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0c0a40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f64a0005ab0 tx=0x7f64a00042e0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.318+0000 7f64b0e7a640 1 --2- 172.21.15.172:0/2923798174 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f64940a2c30 0x7f64940a50f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f64a4005fd0 tx=0x7f64a4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.318+0000 7f64b5675740 1 -- 172.21.15.172:0/1254180357 shutdown_connections 2024-09-10T08:47:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.318+0000 7f64b5675740 1 --2- 172.21.15.172:0/1254180357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0c0a40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.318+0000 7f64b5675740 1 --2- 172.21.15.172:0/1254180357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0b94f0 0x55994f0bdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.318+0000 7f64b5675740 1 --2- 172.21.15.172:0/1254180357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 0x55994f0b8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.318+0000 7f64b5675740 1 -- 172.21.15.172:0/1254180357 >> 172.21.15.172:0/1254180357 conn(0x55994f0b2010 msgr2=0x55994f0b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.318+0000 7f64b5675740 1 -- 172.21.15.172:0/1254180357 shutdown_connections 2024-09-10T08:47:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.318+0000 7f64b5675740 1 -- 172.21.15.172:0/1254180357 wait complete. 2024-09-10T08:47:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.318+0000 7f64b5675740 1 -- start start 2024-09-10T08:47:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.318+0000 7f64b5675740 1 Processor -- start 2024-09-10T08:47:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64b5675740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 0x55994f0c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64b5675740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0b94f0 0x55994f0c3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64b5675740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0ca260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64b5675740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55994f0c14f0 con 0x55994f0be580 2024-09-10T08:47:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64b5675740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55994efa3290 con 0x55994f0b94f0 2024-09-10T08:47:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64b5675740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55994f0c1670 con 0x55994f0b8bb0 2024-09-10T08:47:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64b5675740 10 client.0 _ll_register_callbacks cb 0x55994efd4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64b0e7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64b0e7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0ca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55624/0 (socket says 172.21.15.172:55624) 2024-09-10T08:47:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64b0e7a640 1 -- 172.21.15.172:0/350452775 learned_addr learned my addr 172.21.15.172:0/350452775 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64ab7fe640 1 --2- 172.21.15.172:0/350452775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0b94f0 0x55994f0c3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64b0e7a640 1 -- 172.21.15.172:0/350452775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 msgr2=0x55994f0c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64abfff640 1 --2- 172.21.15.172:0/350452775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 0x55994f0c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64b0e7a640 1 --2- 172.21.15.172:0/350452775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 0x55994f0c2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64b0e7a640 1 -- 172.21.15.172:0/350452775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0b94f0 msgr2=0x55994f0c3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64b0e7a640 1 --2- 172.21.15.172:0/350452775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0b94f0 0x55994f0c3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64b0e7a640 1 -- 172.21.15.172:0/350452775 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55994ed2d210 con 0x55994f0be580 2024-09-10T08:47:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64abfff640 1 --2- 172.21.15.172:0/350452775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 0x55994f0c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f64b0e7a640 1 --2- 172.21.15.172:0/350452775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0ca260 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f64a40116e0 tx=0x7f64a4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02e2607cad07c58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f648b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.319+0000 7f648b7fe640 1 -- 172.21.15.172:0/350452775 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4013830 con 0x55994f0be580 2024-09-10T08:47:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.320+0000 7f64b5675740 1 -- 172.21.15.172:0/350452775 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64a00044d0 con 0x55994f0be580 2024-09-10T08:47:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.320+0000 7f648b7fe640 1 -- 172.21.15.172:0/350452775 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f64a40139d0 con 0x55994f0be580 2024-09-10T08:47:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.320+0000 7f648b7fe640 1 -- 172.21.15.172:0/350452775 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64a402f050 con 0x55994f0be580 2024-09-10T08:47:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.320+0000 7f64b5675740 10 client.15795 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.320+0000 7f64b5675740 1 -- 172.21.15.172:0/350452775 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55994edf9620 con 0x55994f0be580 2024-09-10T08:47:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.320+0000 7f648b7fe640 1 -- 172.21.15.172:0/350452775 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64a00044d0 con 0x55994f0be580 2024-09-10T08:47:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.320+0000 7f648b7fe640 1 -- 172.21.15.172:0/350452775 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f64a400f3f0 con 0x55994f0be580 2024-09-10T08:47:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.320+0000 7f64b5675740 10 client.15795 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.320+0000 7f64b5675740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.320+0000 7f64b5675740 20 client.15795 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.320+0000 7f64b5675740 1 --2- 172.21.15.172:0/350452775 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55994f0cea20 0x55994f0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.320+0000 7f64b5675740 4 client.15795 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.320+0000 7f64b5675740 1 -- 172.21.15.172:0/350452775 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55994f027a20 con 0x55994f0cea20 2024-09-10T08:47:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.320+0000 7f64abfff640 1 --2- 172.21.15.172:0/350452775 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55994f0cea20 0x55994f0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.321+0000 7f648b7fe640 10 client.15795 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.321+0000 7f64abfff640 1 --2- 172.21.15.172:0/350452775 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55994f0cea20 0x55994f0d0e10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62ec641a8f902ce0 server_cookie=c0c4419e265227e4 in_seq=0 out_seq=0 2024-09-10T08:47:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.322+0000 7f648b7fe640 1 -- 172.21.15.172:0/350452775 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55994f027a20 con 0x55994f0cea20 2024-09-10T08:47:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.322+0000 7f648b7fe640 10 client.15795 handle_command_reply: tid=0 2024-09-10T08:47:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.322+0000 7f64b5675740 1 client.15795 shutdown 2024-09-10T08:47:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.322+0000 7f64b5675740 1 -- 172.21.15.172:0/350452775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 msgr2=0x55994f0ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.322+0000 7f64b5675740 1 --2- 172.21.15.172:0/350452775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0ca260 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f64a40116e0 tx=0x7f64a4004ab0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.322+0000 7f64b5675740 1 -- 172.21.15.172:0/350452775 shutdown_connections 2024-09-10T08:47:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.322+0000 7f64b5675740 1 --2- 172.21.15.172:0/350452775 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55994f0cea20 0x55994f0d0e10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.322+0000 7f648b7fe640 0 client.15795 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.322+0000 7f64b5675740 1 --2- 172.21.15.172:0/350452775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0ca260 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.322+0000 7f64b5675740 1 --2- 172.21.15.172:0/350452775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0b94f0 0x55994f0c3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.322+0000 7f64b5675740 1 --2- 172.21.15.172:0/350452775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 0x55994f0c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.322+0000 7f64b5675740 1 -- 172.21.15.172:0/350452775 >> 172.21.15.172:0/350452775 conn(0x55994f0b2010 msgr2=0x55994f0b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.322+0000 7f64b5675740 1 -- 172.21.15.172:0/350452775 shutdown_connections 2024-09-10T08:47:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.322+0000 7f64b5675740 1 -- 172.21.15.172:0/350452775 wait complete. 2024-09-10T08:47:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.322+0000 7f64b5675740 20 client.15795 trim_cache size 0 max 16384 2024-09-10T08:47:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.337+0000 7f64b5675740 1 Processor -- start 2024-09-10T08:47:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.337+0000 7f64b5675740 1 -- start start 2024-09-10T08:47:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.337+0000 7f64b5675740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 0x55994f0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.337+0000 7f64b5675740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0b94f0 0x55994f0ca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.337+0000 7f64b5675740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0d5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.337+0000 7f64b5675740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac1a83a0 con 0x55994f0be580 2024-09-10T08:47:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.337+0000 7f64b5675740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac1a9ee0 con 0x55994f0b94f0 2024-09-10T08:47:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.337+0000 7f64b5675740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac1a8a70 con 0x55994f0b8bb0 2024-09-10T08:47:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.338+0000 7f64b0e7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0d5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.338+0000 7f64b0e7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0d5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55638/0 (socket says 172.21.15.172:55638) 2024-09-10T08:47:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.338+0000 7f64b0e7a640 1 -- 172.21.15.172:0/971749554 learned_addr learned my addr 172.21.15.172:0/971749554 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.338+0000 7f64abfff640 1 --2- 172.21.15.172:0/971749554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 0x55994f0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.338+0000 7f64ab7fe640 1 --2- 172.21.15.172:0/971749554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0b94f0 0x55994f0ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.338+0000 7f64b0e7a640 1 -- 172.21.15.172:0/971749554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 msgr2=0x55994f0b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.338+0000 7f64b0e7a640 1 --2- 172.21.15.172:0/971749554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 0x55994f0b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.338+0000 7f64b0e7a640 1 -- 172.21.15.172:0/971749554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0b94f0 msgr2=0x55994f0ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.338+0000 7f64b0e7a640 1 --2- 172.21.15.172:0/971749554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0b94f0 0x55994f0ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.338+0000 7f64b0e7a640 1 -- 172.21.15.172:0/971749554 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55994f0b2ca0 con 0x55994f0be580 2024-09-10T08:47:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.338+0000 7f64ab7fe640 1 --2- 172.21.15.172:0/971749554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0b94f0 0x55994f0ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.338+0000 7f64b0e7a640 1 --2- 172.21.15.172:0/971749554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0d5710 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f64a400f7e0 tx=0x7f64a4009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae580a7372cedfe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.338+0000 7f6491ffb640 1 -- 172.21.15.172:0/971749554 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4023070 con 0x55994f0be580 2024-09-10T08:47:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.338+0000 7f6491ffb640 1 -- 172.21.15.172:0/971749554 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64a4024020 con 0x55994f0be580 2024-09-10T08:47:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.338+0000 7f6491ffb640 1 -- 172.21.15.172:0/971749554 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a401f400 con 0x55994f0be580 2024-09-10T08:47:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.339+0000 7f64b5675740 1 -- 172.21.15.172:0/971749554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 msgr2=0x55994f0d5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.339+0000 7f64b5675740 1 --2- 172.21.15.172:0/971749554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0d5710 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f64a400f7e0 tx=0x7f64a4009d90 comp rx=0 tx=0).stop 2024-09-10T08:47:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.339+0000 7f64b5675740 1 -- 172.21.15.172:0/971749554 shutdown_connections 2024-09-10T08:47:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.339+0000 7f64b5675740 1 --2- 172.21.15.172:0/971749554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0be580 0x55994f0d5710 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.339+0000 7f64b5675740 1 --2- 172.21.15.172:0/971749554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0b94f0 0x55994f0ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.339+0000 7f64b5675740 1 --2- 172.21.15.172:0/971749554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 0x55994f0b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.339+0000 7f64b5675740 1 -- 172.21.15.172:0/971749554 >> 172.21.15.172:0/971749554 conn(0x55994f0b2010 msgr2=0x55994f0b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.339+0000 7f64b5675740 1 -- 172.21.15.172:0/971749554 shutdown_connections 2024-09-10T08:47:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.339+0000 7f64b5675740 1 -- 172.21.15.172:0/971749554 wait complete. 2024-09-10T08:47:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.339+0000 7f64b5675740 1 -- start start 2024-09-10T08:47:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64b5675740 1 Processor -- start 2024-09-10T08:47:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64b5675740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 0x55994f0c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64b5675740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0b94f0 0x55994f0c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64b5675740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0be580 0x55994f0c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64b5675740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac1a9810 con 0x55994f0b94f0 2024-09-10T08:47:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64b5675740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac1a9140 con 0x55994f0be580 2024-09-10T08:47:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64b5675740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac1a6380 con 0x55994f0b8bb0 2024-09-10T08:47:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64b5675740 10 client.0 _ll_register_callbacks cb 0x7f64a4009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64ab7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0b94f0 0x55994f0c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64ab7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0b94f0 0x55994f0c80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55642/0 (socket says 172.21.15.172:55642) 2024-09-10T08:47:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64ab7fe640 1 -- 172.21.15.172:0/187785681 learned_addr learned my addr 172.21.15.172:0/187785681 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64abfff640 1 --2- 172.21.15.172:0/187785681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 0x55994f0c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64b0e7a640 1 --2- 172.21.15.172:0/187785681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0be580 0x55994f0c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64ab7fe640 1 -- 172.21.15.172:0/187785681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 msgr2=0x55994f0c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64ab7fe640 1 --2- 172.21.15.172:0/187785681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 0x55994f0c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64ab7fe640 1 -- 172.21.15.172:0/187785681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0be580 msgr2=0x55994f0c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64ab7fe640 1 --2- 172.21.15.172:0/187785681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0be580 0x55994f0c8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64ab7fe640 1 -- 172.21.15.172:0/187785681 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55994f0c95f0 con 0x55994f0b94f0 2024-09-10T08:47:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64b0e7a640 1 --2- 172.21.15.172:0/187785681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0be580 0x55994f0c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f64ab7fe640 1 --2- 172.21.15.172:0/187785681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0b94f0 0x55994f0c80c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f64a0006010 tx=0x7f64a0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33706b8ce588f5d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.340+0000 7f648b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.341+0000 7f648b7fe640 1 -- 172.21.15.172:0/187785681 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a003f040 con 0x55994f0b94f0 2024-09-10T08:47:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.341+0000 7f648b7fe640 1 -- 172.21.15.172:0/187785681 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f64a00046c0 con 0x55994f0b94f0 2024-09-10T08:47:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.341+0000 7f64b5675740 1 -- 172.21.15.172:0/187785681 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64a4024020 con 0x55994f0b94f0 2024-09-10T08:47:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.341+0000 7f648b7fe640 1 -- 172.21.15.172:0/187785681 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64a0035040 con 0x55994f0b94f0 2024-09-10T08:47:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.341+0000 7f64b5675740 10 client.15801 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.341+0000 7f64b5675740 1 -- 172.21.15.172:0/187785681 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55994f139820 con 0x55994f0b94f0 2024-09-10T08:47:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.341+0000 7f648b7fe640 1 -- 172.21.15.172:0/187785681 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64a4024020 con 0x55994f0b94f0 2024-09-10T08:47:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.341+0000 7f648b7fe640 1 -- 172.21.15.172:0/187785681 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f64a003b6f0 con 0x55994f0b94f0 2024-09-10T08:47:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.341+0000 7f64b5675740 10 client.15801 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.341+0000 7f64b5675740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.341+0000 7f64b5675740 20 client.15801 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.341+0000 7f64b5675740 1 --2- 172.21.15.172:0/187785681 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55994f0cea20 0x55994f13bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.341+0000 7f64b5675740 4 client.15801 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:47:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.341+0000 7f64b5675740 1 -- 172.21.15.172:0/187785681 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55994f021d70 con 0x55994f0cea20 2024-09-10T08:47:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.342+0000 7f64abfff640 1 --2- 172.21.15.172:0/187785681 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55994f0cea20 0x55994f13bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.342+0000 7f64abfff640 1 --2- 172.21.15.172:0/187785681 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55994f0cea20 0x55994f13bde0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cb517fc97a0dc3d server_cookie=5fa1f54785562782 in_seq=0 out_seq=0 2024-09-10T08:47:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.342+0000 7f648b7fe640 10 client.15801 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.346+0000 7f648b7fe640 1 -- 172.21.15.172:0/187785681 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55994f021d70 con 0x55994f0cea20 2024-09-10T08:47:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.346+0000 7f648b7fe640 10 client.15801 handle_command_reply: tid=0 2024-09-10T08:47:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.346+0000 7f64b5675740 1 client.15801 shutdown 2024-09-10T08:47:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.346+0000 7f64b5675740 1 -- 172.21.15.172:0/187785681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0b94f0 msgr2=0x55994f0c80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.346+0000 7f64b5675740 1 --2- 172.21.15.172:0/187785681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0b94f0 0x55994f0c80c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f64a0006010 tx=0x7f64a0005270 comp rx=0 tx=0).stop 2024-09-10T08:47:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.346+0000 7f64b5675740 1 -- 172.21.15.172:0/187785681 shutdown_connections 2024-09-10T08:47:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.346+0000 7f64b5675740 1 --2- 172.21.15.172:0/187785681 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55994f0cea20 0x55994f13bde0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.346+0000 7f64b5675740 1 --2- 172.21.15.172:0/187785681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55994f0be580 0x55994f0c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.346+0000 7f64b5675740 1 --2- 172.21.15.172:0/187785681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55994f0b94f0 0x55994f0c80c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.346+0000 7f64b5675740 1 --2- 172.21.15.172:0/187785681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55994f0b8bb0 0x55994f0c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.346+0000 7f64b5675740 1 -- 172.21.15.172:0/187785681 >> 172.21.15.172:0/187785681 conn(0x55994f0b2010 msgr2=0x55994f0b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.346+0000 7f648b7fe640 0 client.15801 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.346+0000 7f64b5675740 1 -- 172.21.15.172:0/187785681 shutdown_connections 2024-09-10T08:47:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.346+0000 7f64b5675740 1 -- 172.21.15.172:0/187785681 wait complete. 2024-09-10T08:47:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.346+0000 7f64b5675740 20 client.15801 trim_cache size 0 max 16384 2024-09-10T08:47:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:47:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:47:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:47:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:47:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.347+0000 7f64b28dd640 1 -- 172.21.15.172:0/2923798174 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f64940a2c30 msgr2=0x7f64940a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.347+0000 7f64b28dd640 1 --2- 172.21.15.172:0/2923798174 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f64940a2c30 0x7f64940a50f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f64a4005fd0 tx=0x7f64a4004a40 comp rx=0 tx=0).stop 2024-09-10T08:47:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.347+0000 7f64b28dd640 1 -- 172.21.15.172:0/2923798174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64ac100a40 msgr2=0x7f64ac196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.347+0000 7f64b28dd640 1 --2- 172.21.15.172:0/2923798174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64ac100a40 0x7f64ac196c60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f649c00e9c0 tx=0x7f649c00ee80 comp rx=0 tx=0).stop 2024-09-10T08:47:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.347+0000 7f64b28dd640 1 -- 172.21.15.172:0/2923798174 shutdown_connections 2024-09-10T08:47:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.347+0000 7f64b28dd640 1 --2- 172.21.15.172:0/2923798174 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f64940a2c30 0x7f64940a50f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.347+0000 7f64b28dd640 1 --2- 172.21.15.172:0/2923798174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64ac1025f0 0x7f64ac19e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.347+0000 7f64b28dd640 1 --2- 172.21.15.172:0/2923798174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64ac101c30 0x7f64ac1971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.347+0000 7f64b28dd640 1 --2- 172.21.15.172:0/2923798174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64ac100a40 0x7f64ac196c60 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.347+0000 7f64b28dd640 1 -- 172.21.15.172:0/2923798174 >> 172.21.15.172:0/2923798174 conn(0x7f64ac0fc1e0 msgr2=0x7f64ac0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.347+0000 7f64b28dd640 1 -- 172.21.15.172:0/2923798174 shutdown_connections 2024-09-10T08:47:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:37.347+0000 7f64b28dd640 1 -- 172.21.15.172:0/2923798174 wait complete. 2024-09-10T08:47:37.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:37 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:47:37.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:37 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:47:37.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:37 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:47:38.565 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:38 smithi191 ceph-mon[35314]: pgmap v392: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T08:47:38.595 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:38 smithi172 ceph-mon[30854]: pgmap v392: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T08:47:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:38 smithi174 ceph-mon[34068]: pgmap v392: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T08:47:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:40 smithi191 ceph-mon[35314]: pgmap v393: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T08:47:40.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:40 smithi172 ceph-mon[30854]: pgmap v393: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T08:47:40.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:40 smithi174 ceph-mon[34068]: pgmap v393: 129 pgs: 129 active+clean; 671 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T08:47:42.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:42 smithi191 ceph-mon[35314]: pgmap v394: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T08:47:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:42 smithi172 ceph-mon[30854]: pgmap v394: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T08:47:42.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:42 smithi174 ceph-mon[34068]: pgmap v394: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T08:47:44.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:44 smithi191 ceph-mon[35314]: pgmap v395: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T08:47:44.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:44 smithi172 ceph-mon[30854]: pgmap v395: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T08:47:44.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:44 smithi174 ceph-mon[34068]: pgmap v395: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T08:47:46.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:46 smithi191 ceph-mon[35314]: pgmap v396: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:47:46.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:46 smithi172 ceph-mon[30854]: pgmap v396: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:47:46.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:46 smithi174 ceph-mon[34068]: pgmap v396: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:47:48.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:48 smithi191 ceph-mon[35314]: pgmap v397: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T08:47:48.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:48 smithi172 ceph-mon[30854]: pgmap v397: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T08:47:48.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:48 smithi174 ceph-mon[34068]: pgmap v397: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T08:47:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.480+0000 7ff8c130a640 1 -- 172.21.15.172:0/3826271254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8bc100a50 msgr2=0x7ff8bc100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.480+0000 7ff8c130a640 1 --2- 172.21.15.172:0/3826271254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8bc100a50 0x7ff8bc100e50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009900 tx=0x7ff8ac02a410 comp rx=0 tx=0).stop 2024-09-10T08:47:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.480+0000 7ff8c130a640 1 -- 172.21.15.172:0/3826271254 shutdown_connections 2024-09-10T08:47:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.480+0000 7ff8c130a640 1 --2- 172.21.15.172:0/3826271254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8bc102600 0x7ff8bc108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.480+0000 7ff8c130a640 1 --2- 172.21.15.172:0/3826271254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8bc101c40 0x7ff8bc1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.480+0000 7ff8c130a640 1 --2- 172.21.15.172:0/3826271254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8bc100a50 0x7ff8bc100e50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.480+0000 7ff8c130a640 1 -- 172.21.15.172:0/3826271254 >> 172.21.15.172:0/3826271254 conn(0x7ff8bc0fc1b0 msgr2=0x7ff8bc0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.480+0000 7ff8c130a640 1 -- 172.21.15.172:0/3826271254 shutdown_connections 2024-09-10T08:47:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.480+0000 7ff8c130a640 1 -- 172.21.15.172:0/3826271254 wait complete. 2024-09-10T08:47:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.481+0000 7ff8c130a640 1 Processor -- start 2024-09-10T08:47:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.481+0000 7ff8c130a640 1 -- start start 2024-09-10T08:47:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.481+0000 7ff8c130a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8bc100a50 0x7ff8bc06ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.481+0000 7ff8c130a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8bc101c40 0x7ff8bc19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.481+0000 7ff8c130a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8bc102600 0x7ff8bc19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.481+0000 7ff8c130a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc10c5b0 con 0x7ff8bc100a50 2024-09-10T08:47:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.481+0000 7ff8c130a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc10c430 con 0x7ff8bc101c40 2024-09-10T08:47:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.481+0000 7ff8c130a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc10c730 con 0x7ff8bc102600 2024-09-10T08:47:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.481+0000 7ff8baffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8bc100a50 0x7ff8bc06ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.481+0000 7ff8baffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8bc100a50 0x7ff8bc06ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55650/0 (socket says 172.21.15.172:55650) 2024-09-10T08:47:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.481+0000 7ff8baffd640 1 -- 172.21.15.172:0/3043946091 learned_addr learned my addr 172.21.15.172:0/3043946091 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.481+0000 7ff8ba7fc640 1 --2- 172.21.15.172:0/3043946091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8bc101c40 0x7ff8bc19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.482+0000 7ff8bb7fe640 1 --2- 172.21.15.172:0/3043946091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8bc102600 0x7ff8bc19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.482+0000 7ff8baffd640 1 -- 172.21.15.172:0/3043946091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8bc102600 msgr2=0x7ff8bc19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.482+0000 7ff8baffd640 1 --2- 172.21.15.172:0/3043946091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8bc102600 0x7ff8bc19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.482+0000 7ff8baffd640 1 -- 172.21.15.172:0/3043946091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8bc101c40 msgr2=0x7ff8bc19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.482+0000 7ff8baffd640 1 --2- 172.21.15.172:0/3043946091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8bc101c40 0x7ff8bc19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.482+0000 7ff8baffd640 1 -- 172.21.15.172:0/3043946091 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8bc19b700 con 0x7ff8bc100a50 2024-09-10T08:47:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.482+0000 7ff8bb7fe640 1 --2- 172.21.15.172:0/3043946091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8bc102600 0x7ff8bc19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:47:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.482+0000 7ff8ba7fc640 1 --2- 172.21.15.172:0/3043946091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8bc101c40 0x7ff8bc19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.482+0000 7ff8baffd640 1 --2- 172.21.15.172:0/3043946091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8bc100a50 0x7ff8bc06ac30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0314e0 tx=0x7ff8ac003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.482+0000 7ff8a3fff640 1 -- 172.21.15.172:0/3043946091 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac03f070 con 0x7ff8bc100a50 2024-09-10T08:47:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.482+0000 7ff8a3fff640 1 -- 172.21.15.172:0/3043946091 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8ac031e40 con 0x7ff8bc100a50 2024-09-10T08:47:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.482+0000 7ff8c130a640 1 -- 172.21.15.172:0/3043946091 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8bc19b930 con 0x7ff8bc100a50 2024-09-10T08:47:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.482+0000 7ff8a3fff640 1 -- 172.21.15.172:0/3043946091 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac035030 con 0x7ff8bc100a50 2024-09-10T08:47:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.482+0000 7ff8c130a640 1 -- 172.21.15.172:0/3043946091 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8bc19be70 con 0x7ff8bc100a50 2024-09-10T08:47:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.486+0000 7ff8c409e740 1 Processor -- start 2024-09-10T08:47:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.486+0000 7ff8c409e740 1 -- start start 2024-09-10T08:47:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.486+0000 7ff8c409e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d66990 0x5583b8d66d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.486+0000 7ff8c409e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d672d0 0x5583b8d6bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.486+0000 7ff8c409e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d6c360 0x5583b8d6e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.486+0000 7ff8c409e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5583b8cca2f0 con 0x5583b8d6c360 2024-09-10T08:47:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.486+0000 7ff8c409e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5583b8c72270 con 0x5583b8d66990 2024-09-10T08:47:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.486+0000 7ff8c409e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5583b8cc62b0 con 0x5583b8d672d0 2024-09-10T08:47:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.486+0000 7ff8a3fff640 1 -- 172.21.15.172:0/3043946091 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff8ac042070 con 0x7ff8bc100a50 2024-09-10T08:47:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.486+0000 7ff8ba7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d6c360 0x5583b8d6e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.486+0000 7ff8bb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d66990 0x5583b8d66d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.487+0000 7ff8bb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d66990 0x5583b8d66d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:56504/0 (socket says 172.21.15.172:56504) 2024-09-10T08:47:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.487+0000 7ff8bb7fe640 1 -- 172.21.15.172:0/3592401876 learned_addr learned my addr 172.21.15.172:0/3592401876 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.487+0000 7ff8baffd640 1 --2- 172.21.15.172:0/3592401876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d672d0 0x5583b8d6bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.487+0000 7ff8ba7fc640 1 -- 172.21.15.172:0/3592401876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d672d0 msgr2=0x5583b8d6bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.487+0000 7ff8ba7fc640 1 --2- 172.21.15.172:0/3592401876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d672d0 0x5583b8d6bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.487+0000 7ff8ba7fc640 1 -- 172.21.15.172:0/3592401876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d66990 msgr2=0x5583b8d66d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.487+0000 7ff8ba7fc640 1 --2- 172.21.15.172:0/3592401876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d66990 0x5583b8d66d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.487+0000 7ff8ba7fc640 1 -- 172.21.15.172:0/3592401876 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583b8c71340 con 0x5583b8d6c360 2024-09-10T08:47:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.487+0000 7ff8bb7fe640 1 --2- 172.21.15.172:0/3592401876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d66990 0x5583b8d66d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.487+0000 7ff8ba7fc640 1 --2- 172.21.15.172:0/3592401876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d6c360 0x5583b8d6e820 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc06a9a0 tx=0x7ff8b000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ddda3040c9a8c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.487+0000 7ff8a3fff640 1 --2- 172.21.15.172:0/3043946091 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff8a40a2cb0 0x7ff8a40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.488+0000 7ff8a3fff640 1 -- 172.21.15.172:0/3043946091 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff8ac0ef8b0 con 0x7ff8bc100a50 2024-09-10T08:47:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.488+0000 7ff8bb7fe640 1 --2- 172.21.15.172:0/3043946091 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff8a40a2cb0 0x7ff8a40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.487+0000 7ff8a17fa640 1 -- 172.21.15.172:0/3592401876 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0014070 con 0x5583b8d6c360 2024-09-10T08:47:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.488+0000 7ff8a17fa640 1 -- 172.21.15.172:0/3592401876 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8b0008e50 con 0x5583b8d6c360 2024-09-10T08:47:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.488+0000 7ff8a17fa640 1 -- 172.21.15.172:0/3592401876 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0010690 con 0x5583b8d6c360 2024-09-10T08:47:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.489+0000 7ff8c409e740 1 -- 172.21.15.172:0/3592401876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d6c360 msgr2=0x5583b8d6e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.489+0000 7ff8bb7fe640 1 --2- 172.21.15.172:0/3043946091 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff8a40a2cb0 0x7ff8a40a5170 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4011660 tx=0x7ff8b4009820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.489+0000 7ff8c409e740 1 --2- 172.21.15.172:0/3592401876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d6c360 0x5583b8d6e820 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc06a9a0 tx=0x7ff8b000e720 comp rx=0 tx=0).stop 2024-09-10T08:47:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.489+0000 7ff8c409e740 1 -- 172.21.15.172:0/3592401876 shutdown_connections 2024-09-10T08:47:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.489+0000 7ff8c409e740 1 --2- 172.21.15.172:0/3592401876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d6c360 0x5583b8d6e820 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.489+0000 7ff8c409e740 1 --2- 172.21.15.172:0/3592401876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d672d0 0x5583b8d6bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.489+0000 7ff8c409e740 1 --2- 172.21.15.172:0/3592401876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d66990 0x5583b8d66d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.489+0000 7ff8c409e740 1 -- 172.21.15.172:0/3592401876 >> 172.21.15.172:0/3592401876 conn(0x5583b8d62000 msgr2=0x5583b8d62400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.489+0000 7ff8c409e740 1 -- 172.21.15.172:0/3592401876 shutdown_connections 2024-09-10T08:47:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.489+0000 7ff8c409e740 1 -- 172.21.15.172:0/3592401876 wait complete. 2024-09-10T08:47:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.490+0000 7ff8c409e740 1 -- start start 2024-09-10T08:47:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.490+0000 7ff8c409e740 1 Processor -- start 2024-09-10T08:47:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.491+0000 7ff8c409e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d66990 0x5583b8d70bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.491+0000 7ff8c409e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d672d0 0x5583b8d71100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.491+0000 7ff8c409e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d6c360 0x5583b8d78140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.491+0000 7ff8c409e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5583b8d6f330 con 0x5583b8d6c360 2024-09-10T08:47:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.491+0000 7ff8bb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d6c360 0x5583b8d78140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.491+0000 7ff8ba7fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d672d0 0x5583b8d71100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.491+0000 7ff8ba7fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d672d0 0x5583b8d71100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:56404/0 (socket says 172.21.15.172:56404) 2024-09-10T08:47:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.491+0000 7ff8ba7fc640 1 -- 172.21.15.172:0/886843278 learned_addr learned my addr 172.21.15.172:0/886843278 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.491+0000 7ff8baffd640 1 --2- 172.21.15.172:0/886843278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d66990 0x5583b8d70bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.491+0000 7ff8c409e740 1 -- 172.21.15.172:0/886843278 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5583b8c53290 con 0x5583b8d66990 2024-09-10T08:47:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.491+0000 7ff8c409e740 1 -- 172.21.15.172:0/886843278 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5583b8d6f4b0 con 0x5583b8d672d0 2024-09-10T08:47:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.492+0000 7ff8ba7fc640 1 -- 172.21.15.172:0/886843278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d66990 msgr2=0x5583b8d70bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.492+0000 7ff8ba7fc640 1 --2- 172.21.15.172:0/886843278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d66990 0x5583b8d70bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.492+0000 7ff8ba7fc640 1 -- 172.21.15.172:0/886843278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d6c360 msgr2=0x5583b8d78140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.492+0000 7ff8ba7fc640 1 --2- 172.21.15.172:0/886843278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d6c360 0x5583b8d78140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.492+0000 7ff8ba7fc640 1 -- 172.21.15.172:0/886843278 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583b89dd210 con 0x5583b8d672d0 2024-09-10T08:47:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.492+0000 7ff8bb7fe640 1 --2- 172.21.15.172:0/886843278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d6c360 0x5583b8d78140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.492+0000 7ff8ba7fc640 1 --2- 172.21.15.172:0/886843278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d672d0 0x5583b8d71100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0005880 tx=0x7ff8b0004ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d02f44485da5ac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.492+0000 7ff89eff5640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:47:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.492+0000 7ff8c409e740 10 client.0 _ll_register_callbacks cb 0x5583b8c84140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.492+0000 7ff89eff5640 1 -- 172.21.15.172:0/886843278 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0014070 con 0x5583b8d672d0 2024-09-10T08:47:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.492+0000 7ff89eff5640 1 -- 172.21.15.172:0/886843278 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff8b0005210 con 0x5583b8d672d0 2024-09-10T08:47:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.492+0000 7ff8c409e740 1 -- 172.21.15.172:0/886843278 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8b0008e50 con 0x5583b8d672d0 2024-09-10T08:47:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.493+0000 7ff89eff5640 1 -- 172.21.15.172:0/886843278 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff8b0005450 con 0x5583b8d672d0 2024-09-10T08:47:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.493+0000 7ff8c409e740 10 client.25210 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.493+0000 7ff8c409e740 1 -- 172.21.15.172:0/886843278 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5583b8aa9620 con 0x5583b8d672d0 2024-09-10T08:47:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.494+0000 7ff89eff5640 1 -- 172.21.15.172:0/886843278 <== mon.1 v2:172.21.15.191:3300/0 4 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff8b0004030 con 0x5583b8d672d0 2024-09-10T08:47:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.494+0000 7ff8c409e740 10 client.25210 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.494+0000 7ff8c409e740 10 client.25210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.494+0000 7ff8c409e740 20 client.25210 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.494+0000 7ff8c409e740 1 --2- 172.21.15.172:0/886843278 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5583b8d7c3f0 0x5583b8d7e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.494+0000 7ff8c409e740 4 client.25210 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.494+0000 7ff8c409e740 1 -- 172.21.15.172:0/886843278 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583b8cd7a20 con 0x5583b8d7c3f0 2024-09-10T08:47:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.494+0000 7ff8baffd640 1 --2- 172.21.15.172:0/886843278 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5583b8d7c3f0 0x5583b8d7e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.495+0000 7ff89eff5640 10 client.25210 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.495+0000 7ff8baffd640 1 --2- 172.21.15.172:0/886843278 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5583b8d7c3f0 0x5583b8d7e8b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a0c52a6b5dd70c6 server_cookie=c13f565c4583614e in_seq=0 out_seq=0 2024-09-10T08:47:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.495+0000 7ff89eff5640 1 -- 172.21.15.172:0/886843278 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5583b8cd7a20 con 0x5583b8d7c3f0 2024-09-10T08:47:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.495+0000 7ff89eff5640 10 client.25210 handle_command_reply: tid=0 2024-09-10T08:47:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.495+0000 7ff8c409e740 1 client.25210 shutdown 2024-09-10T08:47:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.496+0000 7ff8c409e740 1 -- 172.21.15.172:0/886843278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d672d0 msgr2=0x5583b8d71100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.496+0000 7ff8c409e740 1 --2- 172.21.15.172:0/886843278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d672d0 0x5583b8d71100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0005880 tx=0x7ff8b0004ee0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.496+0000 7ff8c409e740 1 -- 172.21.15.172:0/886843278 shutdown_connections 2024-09-10T08:47:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.496+0000 7ff8c409e740 1 --2- 172.21.15.172:0/886843278 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5583b8d7c3f0 0x5583b8d7e8b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.496+0000 7ff89eff5640 0 client.25210 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.496+0000 7ff8c409e740 1 --2- 172.21.15.172:0/886843278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d6c360 0x5583b8d78140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.496+0000 7ff8c409e740 1 --2- 172.21.15.172:0/886843278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d672d0 0x5583b8d71100 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.496+0000 7ff8c409e740 1 --2- 172.21.15.172:0/886843278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d66990 0x5583b8d70bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.496+0000 7ff8c409e740 1 -- 172.21.15.172:0/886843278 >> 172.21.15.172:0/886843278 conn(0x5583b8d62000 msgr2=0x5583b8d63810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.496+0000 7ff8c409e740 1 -- 172.21.15.172:0/886843278 shutdown_connections 2024-09-10T08:47:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.496+0000 7ff8c409e740 1 -- 172.21.15.172:0/886843278 wait complete. 2024-09-10T08:47:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.496+0000 7ff8c409e740 20 client.25210 trim_cache size 0 max 16384 2024-09-10T08:47:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.524+0000 7ff8c409e740 1 Processor -- start 2024-09-10T08:47:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.524+0000 7ff8c409e740 1 -- start start 2024-09-10T08:47:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.524+0000 7ff8c409e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d66990 0x5583b8d623e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.524+0000 7ff8c409e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d672d0 0x5583b8d78100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.524+0000 7ff8c409e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d6c360 0x5583b8d62920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.524+0000 7ff8c409e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1b09f0 con 0x5583b8d66990 2024-09-10T08:47:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.524+0000 7ff8c409e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1b2530 con 0x5583b8d672d0 2024-09-10T08:47:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.524+0000 7ff8c409e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1b10c0 con 0x5583b8d6c360 2024-09-10T08:47:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.524+0000 7ff8baffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d66990 0x5583b8d623e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.524+0000 7ff8baffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d66990 0x5583b8d623e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55666/0 (socket says 172.21.15.172:55666) 2024-09-10T08:47:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.524+0000 7ff8baffd640 1 -- 172.21.15.172:0/1165811872 learned_addr learned my addr 172.21.15.172:0/1165811872 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.525+0000 7ff8ba7fc640 1 --2- 172.21.15.172:0/1165811872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d672d0 0x5583b8d78100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.525+0000 7ff8bb7fe640 1 --2- 172.21.15.172:0/1165811872 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d6c360 0x5583b8d62920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.525+0000 7ff8baffd640 1 -- 172.21.15.172:0/1165811872 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d6c360 msgr2=0x5583b8d62920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.525+0000 7ff8baffd640 1 --2- 172.21.15.172:0/1165811872 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d6c360 0x5583b8d62920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.525+0000 7ff8baffd640 1 -- 172.21.15.172:0/1165811872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d672d0 msgr2=0x5583b8d78100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.525+0000 7ff8baffd640 1 --2- 172.21.15.172:0/1165811872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d672d0 0x5583b8d78100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.525+0000 7ff8baffd640 1 -- 172.21.15.172:0/1165811872 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583b8d70700 con 0x5583b8d66990 2024-09-10T08:47:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.525+0000 7ff8ba7fc640 1 --2- 172.21.15.172:0/1165811872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d672d0 0x5583b8d78100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.525+0000 7ff8baffd640 1 --2- 172.21.15.172:0/1165811872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d66990 0x5583b8d623e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0b8720 tx=0x7ff8ac0f4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3b6f6667337e0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.525+0000 7ff8a0ff9640 1 -- 172.21.15.172:0/1165811872 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac049040 con 0x5583b8d66990 2024-09-10T08:47:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.525+0000 7ff8a0ff9640 1 -- 172.21.15.172:0/1165811872 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8ac03be50 con 0x5583b8d66990 2024-09-10T08:47:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.525+0000 7ff8a0ff9640 1 -- 172.21.15.172:0/1165811872 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac0c49c0 con 0x5583b8d66990 2024-09-10T08:47:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.526+0000 7ff8c409e740 1 -- 172.21.15.172:0/1165811872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d66990 msgr2=0x5583b8d623e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.526+0000 7ff8c409e740 1 --2- 172.21.15.172:0/1165811872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d66990 0x5583b8d623e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0b8720 tx=0x7ff8ac0f4040 comp rx=0 tx=0).stop 2024-09-10T08:47:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.526+0000 7ff8c409e740 1 -- 172.21.15.172:0/1165811872 shutdown_connections 2024-09-10T08:47:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.526+0000 7ff8c409e740 1 --2- 172.21.15.172:0/1165811872 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d6c360 0x5583b8d62920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.526+0000 7ff8c409e740 1 --2- 172.21.15.172:0/1165811872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d672d0 0x5583b8d78100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.526+0000 7ff8c409e740 1 --2- 172.21.15.172:0/1165811872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d66990 0x5583b8d623e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.526+0000 7ff8c409e740 1 -- 172.21.15.172:0/1165811872 >> 172.21.15.172:0/1165811872 conn(0x5583b8d62000 msgr2=0x5583b8d68470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.526+0000 7ff8c409e740 1 -- 172.21.15.172:0/1165811872 shutdown_connections 2024-09-10T08:47:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.526+0000 7ff8c409e740 1 -- 172.21.15.172:0/1165811872 wait complete. 2024-09-10T08:47:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.527+0000 7ff8c409e740 1 -- start start 2024-09-10T08:47:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.527+0000 7ff8c409e740 1 Processor -- start 2024-09-10T08:47:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.527+0000 7ff8c409e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d672d0 0x5583b8d75640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8c409e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d6c360 0x5583b8d75b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8c409e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d7c3f0 0x5583b8d76b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8c409e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1b1e60 con 0x5583b8d7c3f0 2024-09-10T08:47:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8c409e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1b1790 con 0x5583b8d672d0 2024-09-10T08:47:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8baffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d672d0 0x5583b8d75640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8baffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d672d0 0x5583b8d75640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:56530/0 (socket says 172.21.15.172:56530) 2024-09-10T08:47:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8baffd640 1 -- 172.21.15.172:0/4068491255 learned_addr learned my addr 172.21.15.172:0/4068491255 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8ba7fc640 1 --2- 172.21.15.172:0/4068491255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d6c360 0x5583b8d75b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8bb7fe640 1 --2- 172.21.15.172:0/4068491255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d7c3f0 0x5583b8d76b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8c409e740 1 -- 172.21.15.172:0/4068491255 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1ae9d0 con 0x5583b8d6c360 2024-09-10T08:47:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8c409e740 10 client.0 _ll_register_callbacks cb 0x7ff8b40096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8ba7fc640 1 -- 172.21.15.172:0/4068491255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d672d0 msgr2=0x5583b8d75640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8ba7fc640 1 --2- 172.21.15.172:0/4068491255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d672d0 0x5583b8d75640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8ba7fc640 1 -- 172.21.15.172:0/4068491255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d7c3f0 msgr2=0x5583b8d76b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8ba7fc640 1 --2- 172.21.15.172:0/4068491255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d7c3f0 0x5583b8d76b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8ba7fc640 1 -- 172.21.15.172:0/4068491255 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583b8d67f60 con 0x5583b8d6c360 2024-09-10T08:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8bb7fe640 1 --2- 172.21.15.172:0/4068491255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d7c3f0 0x5583b8d76b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.528+0000 7ff8baffd640 1 --2- 172.21.15.172:0/4068491255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d672d0 0x5583b8d75640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.529+0000 7ff8ba7fc640 1 --2- 172.21.15.172:0/4068491255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d6c360 0x5583b8d75b80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0005850 tx=0x7ff8b00084c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bce13cd5db42faa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.529+0000 7ff89eff5640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.529+0000 7ff89eff5640 1 -- 172.21.15.172:0/4068491255 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0023670 con 0x5583b8d6c360 2024-09-10T08:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.529+0000 7ff89eff5640 1 -- 172.21.15.172:0/4068491255 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff8b0023810 con 0x5583b8d6c360 2024-09-10T08:47:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.529+0000 7ff8c409e740 1 -- 172.21.15.172:0/4068491255 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8ac03be50 con 0x5583b8d6c360 2024-09-10T08:47:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.529+0000 7ff89eff5640 1 -- 172.21.15.172:0/4068491255 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff8b0023a70 con 0x5583b8d6c360 2024-09-10T08:47:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.529+0000 7ff8c409e740 10 client.25216 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.529+0000 7ff8c409e740 1 -- 172.21.15.172:0/4068491255 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5583b8d782a0 con 0x5583b8d6c360 2024-09-10T08:47:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.530+0000 7ff89eff5640 1 -- 172.21.15.172:0/4068491255 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff8ac03be50 con 0x5583b8d6c360 2024-09-10T08:47:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.531+0000 7ff89eff5640 1 -- 172.21.15.172:0/4068491255 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff8b0019450 con 0x5583b8d6c360 2024-09-10T08:47:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.531+0000 7ff8c409e740 10 client.25216 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.531+0000 7ff8c409e740 10 client.25216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:47:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.531+0000 7ff8c409e740 20 client.25216 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.531+0000 7ff8c409e740 1 --2- 172.21.15.172:0/4068491255 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5583b8de77a0 0x5583b8de9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.531+0000 7ff8c409e740 4 client.25216 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:47:50.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.531+0000 7ff8c409e740 1 -- 172.21.15.172:0/4068491255 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5583b8cd1d70 con 0x5583b8de77a0 2024-09-10T08:47:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.531+0000 7ff8baffd640 1 --2- 172.21.15.172:0/4068491255 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5583b8de77a0 0x5583b8de9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.532+0000 7ff89eff5640 10 client.25216 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.532+0000 7ff8baffd640 1 --2- 172.21.15.172:0/4068491255 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5583b8de77a0 0x5583b8de9b90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f83a20ef7fcf480 server_cookie=8e1532806afab7ee in_seq=0 out_seq=0 2024-09-10T08:47:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.535+0000 7ff89eff5640 1 -- 172.21.15.172:0/4068491255 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5583b8cd1d70 con 0x5583b8de77a0 2024-09-10T08:47:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.535+0000 7ff89eff5640 10 client.25216 handle_command_reply: tid=0 2024-09-10T08:47:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.535+0000 7ff8c409e740 1 client.25216 shutdown 2024-09-10T08:47:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.535+0000 7ff8c409e740 1 -- 172.21.15.172:0/4068491255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d6c360 msgr2=0x5583b8d75b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.535+0000 7ff8c409e740 1 --2- 172.21.15.172:0/4068491255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d6c360 0x5583b8d75b80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0005850 tx=0x7ff8b00084c0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.535+0000 7ff8c409e740 1 -- 172.21.15.172:0/4068491255 shutdown_connections 2024-09-10T08:47:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.535+0000 7ff8c409e740 1 --2- 172.21.15.172:0/4068491255 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5583b8de77a0 0x5583b8de9b90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.535+0000 7ff8c409e740 1 --2- 172.21.15.172:0/4068491255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583b8d7c3f0 0x5583b8d76b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.536+0000 7ff8c409e740 1 --2- 172.21.15.172:0/4068491255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583b8d6c360 0x5583b8d75b80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.536+0000 7ff8c409e740 1 --2- 172.21.15.172:0/4068491255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583b8d672d0 0x5583b8d75640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.536+0000 7ff8c409e740 1 -- 172.21.15.172:0/4068491255 >> 172.21.15.172:0/4068491255 conn(0x5583b8d62000 msgr2=0x5583b8d609f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.536+0000 7ff89eff5640 0 client.25216 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.536+0000 7ff8c409e740 1 -- 172.21.15.172:0/4068491255 shutdown_connections 2024-09-10T08:47:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.536+0000 7ff8c409e740 1 -- 172.21.15.172:0/4068491255 wait complete. 2024-09-10T08:47:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.536+0000 7ff8c409e740 20 client.25216 trim_cache size 0 max 16384 2024-09-10T08:47:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:47:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:47:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:47:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:47:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.536+0000 7ff8c130a640 1 -- 172.21.15.172:0/3043946091 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff8a40a2cb0 msgr2=0x7ff8a40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.536+0000 7ff8c130a640 1 --2- 172.21.15.172:0/3043946091 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff8a40a2cb0 0x7ff8a40a5170 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4011660 tx=0x7ff8b4009820 comp rx=0 tx=0).stop 2024-09-10T08:47:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.537+0000 7ff8c130a640 1 -- 172.21.15.172:0/3043946091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8bc100a50 msgr2=0x7ff8bc06ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.537+0000 7ff8c130a640 1 --2- 172.21.15.172:0/3043946091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8bc100a50 0x7ff8bc06ac30 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac0314e0 tx=0x7ff8ac003c80 comp rx=0 tx=0).stop 2024-09-10T08:47:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.537+0000 7ff8c130a640 1 -- 172.21.15.172:0/3043946091 shutdown_connections 2024-09-10T08:47:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.537+0000 7ff8c130a640 1 --2- 172.21.15.172:0/3043946091 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff8a40a2cb0 0x7ff8a40a5170 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.537+0000 7ff8c130a640 1 --2- 172.21.15.172:0/3043946091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8bc102600 0x7ff8bc19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.537+0000 7ff8c130a640 1 --2- 172.21.15.172:0/3043946091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8bc101c40 0x7ff8bc19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.537+0000 7ff8c130a640 1 --2- 172.21.15.172:0/3043946091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8bc100a50 0x7ff8bc06ac30 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.537+0000 7ff8c130a640 1 -- 172.21.15.172:0/3043946091 >> 172.21.15.172:0/3043946091 conn(0x7ff8bc0fc1b0 msgr2=0x7ff8bc104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.537+0000 7ff8c130a640 1 -- 172.21.15.172:0/3043946091 shutdown_connections 2024-09-10T08:47:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:47:50.537+0000 7ff8c130a640 1 -- 172.21.15.172:0/3043946091 wait complete. 2024-09-10T08:47:50.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:50 smithi191 ceph-mon[35314]: pgmap v398: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T08:47:50.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:50 smithi172 ceph-mon[30854]: pgmap v398: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T08:47:50.745 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:50 smithi174 ceph-mon[34068]: pgmap v398: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T08:47:51.861 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:47:52.262 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:52.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:52 smithi191 ceph-mon[35314]: pgmap v399: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:47:52.563 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:52 smithi172 ceph-mon[30854]: pgmap v399: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:47:52.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:52 smithi174 ceph-mon[34068]: pgmap v399: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:47:52.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.812+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3665853702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93e41082a0 msgr2=0x7f93e410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:52.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.812+0000 7f93e8a7b640 1 --2- 172.21.15.172:0/3665853702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93e41082a0 0x7f93e410ec40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00f0d0 tx=0x7f93dc02e710 comp rx=0 tx=0).stop 2024-09-10T08:47:52.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.812+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3665853702 shutdown_connections 2024-09-10T08:47:52.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.812+0000 7f93e8a7b640 1 --2- 172.21.15.172:0/3665853702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93e41082a0 0x7f93e410ec40 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:52.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.812+0000 7f93e8a7b640 1 --2- 172.21.15.172:0/3665853702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93e41036e0 0x7f93e4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:52.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.812+0000 7f93e8a7b640 1 --2- 172.21.15.172:0/3665853702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93e4102d30 0x7f93e4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:52.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.812+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3665853702 >> 172.21.15.172:0/3665853702 conn(0x7f93e40fc7d0 msgr2=0x7f93e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:52.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.812+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3665853702 shutdown_connections 2024-09-10T08:47:52.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.812+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3665853702 wait complete. 2024-09-10T08:47:52.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.813+0000 7f93e8a7b640 1 Processor -- start 2024-09-10T08:47:52.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.813+0000 7f93e8a7b640 1 -- start start 2024-09-10T08:47:52.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.813+0000 7f93e8a7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93e4102d30 0x7f93e4073d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:52.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.813+0000 7f93e8a7b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93e41036e0 0x7f93e406bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:52.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.813+0000 7f93e8a7b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93e41082a0 0x7f93e406c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:52.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.814+0000 7f93e259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93e4102d30 0x7f93e4073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:52.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.814+0000 7f93e259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93e4102d30 0x7f93e4073d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55698/0 (socket says 172.21.15.172:55698) 2024-09-10T08:47:52.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.814+0000 7f93e1d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93e41036e0 0x7f93e406bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:52.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.814+0000 7f93e2d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93e41082a0 0x7f93e406c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:52.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.814+0000 7f93e1d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93e41036e0 0x7f93e406bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:56450/0 (socket says 172.21.15.172:56450) 2024-09-10T08:47:52.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.814+0000 7f93e259c640 1 -- 172.21.15.172:0/3285392656 learned_addr learned my addr 172.21.15.172:0/3285392656 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:52.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.814+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3285392656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93e410fd60 con 0x7f93e4102d30 2024-09-10T08:47:52.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.814+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3285392656 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f93e410fbe0 con 0x7f93e41082a0 2024-09-10T08:47:52.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.814+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3285392656 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f93e410fee0 con 0x7f93e41036e0 2024-09-10T08:47:52.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.814+0000 7f93e259c640 1 -- 172.21.15.172:0/3285392656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93e41036e0 msgr2=0x7f93e406bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:52.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.814+0000 7f93e259c640 1 --2- 172.21.15.172:0/3285392656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93e41036e0 0x7f93e406bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:52.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.814+0000 7f93e259c640 1 -- 172.21.15.172:0/3285392656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93e41082a0 msgr2=0x7f93e406c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:52.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.814+0000 7f93e259c640 1 --2- 172.21.15.172:0/3285392656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93e41082a0 0x7f93e406c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:52.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.814+0000 7f93e259c640 1 -- 172.21.15.172:0/3285392656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e406ca40 con 0x7f93e4102d30 2024-09-10T08:47:52.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.815+0000 7f93e259c640 1 --2- 172.21.15.172:0/3285392656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93e4102d30 0x7f93e4073d10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f93d400d950 tx=0x7f93d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:52.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.815+0000 7f93d37fe640 1 -- 172.21.15.172:0/3285392656 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4017070 con 0x7f93e4102d30 2024-09-10T08:47:52.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.815+0000 7f93d37fe640 1 -- 172.21.15.172:0/3285392656 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93d4009c30 con 0x7f93e4102d30 2024-09-10T08:47:52.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.815+0000 7f93d37fe640 1 -- 172.21.15.172:0/3285392656 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d400f620 con 0x7f93e4102d30 2024-09-10T08:47:52.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.815+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3285392656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e41ab110 con 0x7f93e4102d30 2024-09-10T08:47:52.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.815+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3285392656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e41ab600 con 0x7f93e4102d30 2024-09-10T08:47:52.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.816+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3285392656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e406a850 con 0x7f93e4102d30 2024-09-10T08:47:52.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.818+0000 7f93d37fe640 1 -- 172.21.15.172:0/3285392656 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f93d40109a0 con 0x7f93e4102d30 2024-09-10T08:47:52.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.818+0000 7f93d37fe640 1 --2- 172.21.15.172:0/3285392656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f93cc0a2cd0 0x7f93cc0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:52.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.819+0000 7f93e1d9b640 1 --2- 172.21.15.172:0/3285392656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f93cc0a2cd0 0x7f93cc0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:52.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.819+0000 7f93d37fe640 1 -- 172.21.15.172:0/3285392656 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f93d40ceed0 con 0x7f93e4102d30 2024-09-10T08:47:52.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.819+0000 7f93e1d9b640 1 --2- 172.21.15.172:0/3285392656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f93cc0a2cd0 0x7f93cc0a5190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f93e406d4a0 tx=0x7f93d8013c20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:52.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:52.826+0000 7f93d37fe640 1 -- 172.21.15.172:0/3285392656 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93d4009de0 con 0x7f93e4102d30 2024-09-10T08:47:53.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:53.037+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3285392656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93e410a2b0 con 0x7f93e4102d30 2024-09-10T08:47:53.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:53.038+0000 7f93d37fe640 1 -- 172.21.15.172:0/3285392656 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7f93d409bbd0 con 0x7f93e4102d30 2024-09-10T08:47:53.040 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:47:53.040 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":29,"btime":"2024-09-10T08:46:37:242916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:46:36.267985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:47:53.040 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 29 2024-09-10T08:47:53.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:53.041+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3285392656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f93cc0a2cd0 msgr2=0x7f93cc0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:53.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:53.041+0000 7f93e8a7b640 1 --2- 172.21.15.172:0/3285392656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f93cc0a2cd0 0x7f93cc0a5190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f93e406d4a0 tx=0x7f93d8013c20 comp rx=0 tx=0).stop 2024-09-10T08:47:53.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:53.041+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3285392656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93e4102d30 msgr2=0x7f93e4073d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:53.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:53.041+0000 7f93e8a7b640 1 --2- 172.21.15.172:0/3285392656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93e4102d30 0x7f93e4073d10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f93d400d950 tx=0x7f93d400de10 comp rx=0 tx=0).stop 2024-09-10T08:47:53.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:53.041+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3285392656 shutdown_connections 2024-09-10T08:47:53.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:53.041+0000 7f93e8a7b640 1 --2- 172.21.15.172:0/3285392656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f93cc0a2cd0 0x7f93cc0a5190 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:53.042+0000 7f93e8a7b640 1 --2- 172.21.15.172:0/3285392656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93e41082a0 0x7f93e406c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:53.042+0000 7f93e8a7b640 1 --2- 172.21.15.172:0/3285392656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93e41036e0 0x7f93e406bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:53.042+0000 7f93e8a7b640 1 --2- 172.21.15.172:0/3285392656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93e4102d30 0x7f93e4073d10 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:53.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:53.042+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3285392656 >> 172.21.15.172:0/3285392656 conn(0x7f93e40fc7d0 msgr2=0x7f93e4105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:53.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:53.042+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3285392656 shutdown_connections 2024-09-10T08:47:53.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:53.042+0000 7f93e8a7b640 1 -- 172.21.15.172:0/3285392656 wait complete. 2024-09-10T08:47:53.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:53 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3285392656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:53.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:53 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3285392656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:53.787 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 2d459bfa --release --await' 2024-09-10T08:47:53.787 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 2d459bfa --release --await 2024-09-10T08:47:53.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:53 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3285392656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:54.198 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:54.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:54 smithi172 ceph-mon[30854]: pgmap v400: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T08:47:54.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:54 smithi174 ceph-mon[34068]: pgmap v400: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T08:47:54.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:54 smithi191 ceph-mon[35314]: pgmap v400: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T08:47:54.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.823+0000 7faa13e3b640 1 -- 172.21.15.172:0/3507789198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa0c101e20 msgr2=0x7faa0c1056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:54.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.823+0000 7faa13e3b640 1 --2- 172.21.15.172:0/3507789198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa0c101e20 0x7faa0c1056b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009860 tx=0x7fa9fc02a340 comp rx=0 tx=0).stop 2024-09-10T08:47:54.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.823+0000 7faa13e3b640 1 -- 172.21.15.172:0/3507789198 shutdown_connections 2024-09-10T08:47:54.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.823+0000 7faa13e3b640 1 --2- 172.21.15.172:0/3507789198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa0c105de0 0x7faa0c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.823+0000 7faa13e3b640 1 --2- 172.21.15.172:0/3507789198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa0c101e20 0x7faa0c1056b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.823+0000 7faa13e3b640 1 --2- 172.21.15.172:0/3507789198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa0c101470 0x7faa0c101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.823+0000 7faa13e3b640 1 -- 172.21.15.172:0/3507789198 >> 172.21.15.172:0/3507789198 conn(0x7faa0c077230 msgr2=0x7faa0c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:54.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.824+0000 7faa13e3b640 1 -- 172.21.15.172:0/3507789198 shutdown_connections 2024-09-10T08:47:54.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.824+0000 7faa13e3b640 1 -- 172.21.15.172:0/3507789198 wait complete. 2024-09-10T08:47:54.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.825+0000 7faa13e3b640 1 Processor -- start 2024-09-10T08:47:54.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.825+0000 7faa13e3b640 1 -- start start 2024-09-10T08:47:54.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.825+0000 7faa13e3b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa0c101470 0x7faa0c195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.826+0000 7faa13e3b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa0c101e20 0x7faa0c1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.826+0000 7faa13e3b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa0c105de0 0x7faa0c1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.826+0000 7faa13e3b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c10fbf0 con 0x7faa0c101470 2024-09-10T08:47:54.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.826+0000 7faa13e3b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c10fa70 con 0x7faa0c105de0 2024-09-10T08:47:54.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.826+0000 7faa13e3b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c10fd70 con 0x7faa0c101e20 2024-09-10T08:47:54.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.826+0000 7faa11bd7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa0c101470 0x7faa0c195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:54.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.826+0000 7faa11bd7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa0c101470 0x7faa0c195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55724/0 (socket says 172.21.15.172:55724) 2024-09-10T08:47:54.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.826+0000 7faa11bd7640 1 -- 172.21.15.172:0/2284362765 learned_addr learned my addr 172.21.15.172:0/2284362765 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:54.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.826+0000 7faa113d6640 1 --2- 172.21.15.172:0/2284362765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa0c101e20 0x7faa0c1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:54.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.826+0000 7faa123d8640 1 --2- 172.21.15.172:0/2284362765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa0c105de0 0x7faa0c1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:54.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.826+0000 7faa113d6640 1 -- 172.21.15.172:0/2284362765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa0c105de0 msgr2=0x7faa0c1969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:54.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.826+0000 7faa113d6640 1 --2- 172.21.15.172:0/2284362765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa0c105de0 0x7faa0c1969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.826+0000 7faa113d6640 1 -- 172.21.15.172:0/2284362765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa0c101470 msgr2=0x7faa0c195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:54.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.826+0000 7faa113d6640 1 --2- 172.21.15.172:0/2284362765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa0c101470 0x7faa0c195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.826+0000 7faa113d6640 1 -- 172.21.15.172:0/2284362765 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0c1aabf0 con 0x7faa0c101e20 2024-09-10T08:47:54.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.827+0000 7faa113d6640 1 --2- 172.21.15.172:0/2284362765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa0c101e20 0x7faa0c1964b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc02a860 tx=0x7fa9fc030bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:54.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.827+0000 7faa02ffd640 1 -- 172.21.15.172:0/2284362765 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc040020 con 0x7faa0c101e20 2024-09-10T08:47:54.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.827+0000 7faa02ffd640 1 -- 172.21.15.172:0/2284362765 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9fc038040 con 0x7faa0c101e20 2024-09-10T08:47:54.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.827+0000 7faa02ffd640 1 -- 172.21.15.172:0/2284362765 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc037070 con 0x7faa0c101e20 2024-09-10T08:47:54.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.828+0000 7faa13e3b640 1 -- 172.21.15.172:0/2284362765 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0c1aae80 con 0x7faa0c101e20 2024-09-10T08:47:54.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.828+0000 7faa13e3b640 1 -- 172.21.15.172:0/2284362765 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa0c1ab280 con 0x7faa0c101e20 2024-09-10T08:47:54.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.831+0000 7faa02ffd640 1 -- 172.21.15.172:0/2284362765 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa9fc043020 con 0x7faa0c101e20 2024-09-10T08:47:54.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.831+0000 7faa16214740 1 Processor -- start 2024-09-10T08:47:54.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.832+0000 7faa16214740 1 -- start start 2024-09-10T08:47:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.832+0000 7faa16214740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0060 0x558edeba0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.832+0000 7faa16214740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edeba0980 0x558edebaad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.832+0000 7faa16214740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edebab240 0x558edebad630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.832+0000 7faa16214740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558edeb05620 con 0x558edeba0980 2024-09-10T08:47:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.832+0000 7faa16214740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558edeaace50 con 0x558edeba0060 2024-09-10T08:47:54.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.832+0000 7faa16214740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558edeb015e0 con 0x558edebab240 2024-09-10T08:47:54.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.832+0000 7faa11bd7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edeba0980 0x558edebaad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:54.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.832+0000 7faa11bd7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edeba0980 0x558edebaad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55740/0 (socket says 172.21.15.172:55740) 2024-09-10T08:47:54.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.832+0000 7faa11bd7640 1 -- 172.21.15.172:0/1269241915 learned_addr learned my addr 172.21.15.172:0/1269241915 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:54.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.832+0000 7faa123d8640 1 --2- 172.21.15.172:0/1269241915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0060 0x558edeba0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:54.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.833+0000 7faa113d6640 1 --2- 172.21.15.172:0/1269241915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edebab240 0x558edebad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:54.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.833+0000 7faa123d8640 1 -- 172.21.15.172:0/1269241915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edebab240 msgr2=0x558edebad630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:54.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.833+0000 7faa123d8640 1 --2- 172.21.15.172:0/1269241915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edebab240 0x558edebad630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.833+0000 7faa123d8640 1 -- 172.21.15.172:0/1269241915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edeba0980 msgr2=0x558edebaad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:54.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.833+0000 7faa123d8640 1 --2- 172.21.15.172:0/1269241915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edeba0980 0x558edebaad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.833+0000 7faa123d8640 1 -- 172.21.15.172:0/1269241915 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558edea9dd40 con 0x558edeba0060 2024-09-10T08:47:54.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.833+0000 7faa11bd7640 1 --2- 172.21.15.172:0/1269241915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edeba0980 0x558edebaad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:54.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.833+0000 7faa123d8640 1 --2- 172.21.15.172:0/1269241915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0060 0x558edeba0440 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faa04009f60 tx=0x7faa04009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1487c59f6960156 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:54.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.833+0000 7fa9f7fff640 1 -- 172.21.15.172:0/1269241915 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0402d070 con 0x558edeba0060 2024-09-10T08:47:54.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.834+0000 7fa9f7fff640 1 -- 172.21.15.172:0/1269241915 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa04028910 con 0x558edeba0060 2024-09-10T08:47:54.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.834+0000 7fa9f7fff640 1 -- 172.21.15.172:0/1269241915 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04026800 con 0x558edeba0060 2024-09-10T08:47:54.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.834+0000 7faa02ffd640 1 --2- 172.21.15.172:0/2284362765 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa9f80a2c80 0x7fa9f80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.834+0000 7faa02ffd640 1 -- 172.21.15.172:0/2284362765 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa9fc0eef80 con 0x7faa0c101e20 2024-09-10T08:47:54.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.835+0000 7faa11bd7640 1 --2- 172.21.15.172:0/2284362765 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa9f80a2c80 0x7fa9f80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:54.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.835+0000 7faa11bd7640 1 --2- 172.21.15.172:0/2284362765 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa9f80a2c80 0x7fa9f80a5140 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x558edeb9d450 tx=0x7faa08009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:54.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.835+0000 7faa16214740 1 -- 172.21.15.172:0/1269241915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0060 msgr2=0x558edeba0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:54.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.835+0000 7faa16214740 1 --2- 172.21.15.172:0/1269241915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0060 0x558edeba0440 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faa04009f60 tx=0x7faa04009500 comp rx=0 tx=0).stop 2024-09-10T08:47:54.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.835+0000 7faa16214740 1 -- 172.21.15.172:0/1269241915 shutdown_connections 2024-09-10T08:47:54.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.835+0000 7faa16214740 1 --2- 172.21.15.172:0/1269241915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edebab240 0x558edebad630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.835+0000 7faa16214740 1 --2- 172.21.15.172:0/1269241915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edeba0980 0x558edebaad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.835+0000 7faa16214740 1 --2- 172.21.15.172:0/1269241915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0060 0x558edeba0440 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.835+0000 7faa16214740 1 -- 172.21.15.172:0/1269241915 >> 172.21.15.172:0/1269241915 conn(0x558edeb9c5f0 msgr2=0x558edeb9c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.835+0000 7faa16214740 1 -- 172.21.15.172:0/1269241915 shutdown_connections 2024-09-10T08:47:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.835+0000 7faa16214740 1 -- 172.21.15.172:0/1269241915 wait complete. 2024-09-10T08:47:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.836+0000 7faa16214740 1 -- start start 2024-09-10T08:47:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.836+0000 7faa16214740 1 Processor -- start 2024-09-10T08:47:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.836+0000 7faa16214740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edeba0060 0x558edeba80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.836+0000 7faa16214740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0980 0x558edeba8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.836+0000 7faa16214740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edebab240 0x558edeba8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.836+0000 7faa16214740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558edebade10 con 0x558edebab240 2024-09-10T08:47:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.836+0000 7faa16214740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558edea8df10 con 0x558edeba0980 2024-09-10T08:47:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.836+0000 7faa16214740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558edebadf90 con 0x558edeba0060 2024-09-10T08:47:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.837+0000 7faa11bd7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edeba0060 0x558edeba80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.837+0000 7faa11bd7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edeba0060 0x558edeba80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:56492/0 (socket says 172.21.15.172:56492) 2024-09-10T08:47:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.837+0000 7faa11bd7640 1 -- 172.21.15.172:0/754009172 learned_addr learned my addr 172.21.15.172:0/754009172 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.837+0000 7faa123d8640 1 --2- 172.21.15.172:0/754009172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edebab240 0x558edeba8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.837+0000 7faa113d6640 1 --2- 172.21.15.172:0/754009172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0980 0x558edeba8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.837+0000 7faa16214740 10 client.0 _ll_register_callbacks cb 0x558edeab1db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:54.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.837+0000 7faa11bd7640 1 -- 172.21.15.172:0/754009172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0980 msgr2=0x558edeba8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.837+0000 7faa11bd7640 1 --2- 172.21.15.172:0/754009172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0980 0x558edeba8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.837+0000 7faa11bd7640 1 -- 172.21.15.172:0/754009172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edebab240 msgr2=0x558edeba8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.837+0000 7faa11bd7640 1 --2- 172.21.15.172:0/754009172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edebab240 0x558edeba8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.837+0000 7faa11bd7640 1 -- 172.21.15.172:0/754009172 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ede814580 con 0x558edeba0060 2024-09-10T08:47:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.837+0000 7faa123d8640 1 --2- 172.21.15.172:0/754009172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edebab240 0x558edeba8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.837+0000 7faa16214740 1 -- 172.21.15.172:0/754009172 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa04028910 con 0x558edeba0060 2024-09-10T08:47:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.838+0000 7faa11bd7640 1 --2- 172.21.15.172:0/754009172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edeba0060 0x558edeba80f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faa0800fb20 tx=0x7faa08004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a92929faf67a5c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.838+0000 7fa9f57fa640 10 client.25234 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:47:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.838+0000 7fa9f57fa640 1 -- 172.21.15.172:0/754009172 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0802d060 con 0x558edeba0060 2024-09-10T08:47:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.838+0000 7fa9f57fa640 1 -- 172.21.15.172:0/754009172 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7faa08018030 con 0x558edeba0060 2024-09-10T08:47:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.838+0000 7fa9f57fa640 1 -- 172.21.15.172:0/754009172 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa08005020 con 0x558edeba0060 2024-09-10T08:47:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.838+0000 7faa16214740 10 client.25234 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:54.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.838+0000 7faa16214740 1 -- 172.21.15.172:0/754009172 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({fsmap=29}) -- 0x558ede8de960 con 0x558edeba0060 2024-09-10T08:47:54.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.839+0000 7fa9f57fa640 1 -- 172.21.15.172:0/754009172 <== mon.1 v2:172.21.15.191:3300/0 4 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7faa08030050 con 0x558edeba0060 2024-09-10T08:47:54.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.839+0000 7faa16214740 10 client.25234 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:54.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.839+0000 7faa16214740 10 client.25234 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:47:54.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.839+0000 7faa16214740 20 client.25234 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:54.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.839+0000 7faa16214740 1 --2- 172.21.15.172:0/754009172 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558edebedc20 0x558edebf0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.839+0000 7faa16214740 4 client.25234 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.839+0000 7faa16214740 1 -- 172.21.15.172:0/754009172 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558edeb12b90 con 0x558edebedc20 2024-09-10T08:47:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.839+0000 7faa113d6640 1 --2- 172.21.15.172:0/754009172 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558edebedc20 0x558edebf0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.840+0000 7faa113d6640 1 --2- 172.21.15.172:0/754009172 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558edebedc20 0x558edebf0090 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5ae278982958162 server_cookie=b5a049ee062cf04a in_seq=0 out_seq=0 2024-09-10T08:47:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.840+0000 7fa9f57fa640 10 client.25234 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.840+0000 7fa9f57fa640 1 -- 172.21.15.172:0/754009172 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558edeb12b90 con 0x558edebedc20 2024-09-10T08:47:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.840+0000 7fa9f57fa640 10 client.25234 handle_command_reply: tid=0 2024-09-10T08:47:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.840+0000 7faa16214740 1 client.25234 shutdown 2024-09-10T08:47:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.841+0000 7faa16214740 1 -- 172.21.15.172:0/754009172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edeba0060 msgr2=0x558edeba80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.841+0000 7faa16214740 1 --2- 172.21.15.172:0/754009172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edeba0060 0x558edeba80f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faa0800fb20 tx=0x7faa08004730 comp rx=0 tx=0).stop 2024-09-10T08:47:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.841+0000 7faa16214740 1 -- 172.21.15.172:0/754009172 shutdown_connections 2024-09-10T08:47:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.841+0000 7faa16214740 1 --2- 172.21.15.172:0/754009172 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558edebedc20 0x558edebf0090 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.841+0000 7fa9f57fa640 0 client.25234 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:54.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.841+0000 7faa16214740 1 --2- 172.21.15.172:0/754009172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edebab240 0x558edeba8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.841+0000 7faa16214740 1 --2- 172.21.15.172:0/754009172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0980 0x558edeba8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.841+0000 7faa16214740 1 --2- 172.21.15.172:0/754009172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edeba0060 0x558edeba80f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.841+0000 7faa16214740 1 -- 172.21.15.172:0/754009172 >> 172.21.15.172:0/754009172 conn(0x558edeb9c5f0 msgr2=0x558edeb9b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:54.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.841+0000 7faa16214740 1 -- 172.21.15.172:0/754009172 shutdown_connections 2024-09-10T08:47:54.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.841+0000 7faa16214740 1 -- 172.21.15.172:0/754009172 wait complete. 2024-09-10T08:47:54.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.841+0000 7faa16214740 20 client.25234 trim_cache size 0 max 16384 2024-09-10T08:47:54.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.874+0000 7faa16214740 1 Processor -- start 2024-09-10T08:47:54.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.874+0000 7faa16214740 1 -- start start 2024-09-10T08:47:54.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.875+0000 7faa16214740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edeba0060 0x558edeba2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.875+0000 7faa16214740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0980 0x558edeb9b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.875+0000 7faa16214740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edebab240 0x558edebf5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.875+0000 7faa113d6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0980 0x558edeb9b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:54.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.875+0000 7faa123d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edebab240 0x558edebf5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:54.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.875+0000 7faa123d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edebab240 0x558edebf5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55760/0 (socket says 172.21.15.172:55760) 2024-09-10T08:47:54.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.875+0000 7faa113d6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0980 0x558edeb9b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:56588/0 (socket says 172.21.15.172:56588) 2024-09-10T08:47:54.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.875+0000 7faa123d8640 1 -- 172.21.15.172:0/1650546741 learned_addr learned my addr 172.21.15.172:0/1650546741 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:54.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.875+0000 7faa16214740 1 -- 172.21.15.172:0/1650546741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c1ac820 con 0x558edebab240 2024-09-10T08:47:54.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.875+0000 7faa16214740 1 -- 172.21.15.172:0/1650546741 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c1ae360 con 0x558edeba0980 2024-09-10T08:47:54.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.875+0000 7faa16214740 1 -- 172.21.15.172:0/1650546741 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c1acef0 con 0x558edeba0060 2024-09-10T08:47:54.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.876+0000 7faa123d8640 1 -- 172.21.15.172:0/1650546741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edeba0060 msgr2=0x558edeba2b10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:47:54.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.876+0000 7faa123d8640 1 --2- 172.21.15.172:0/1650546741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edeba0060 0x558edeba2b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.876+0000 7faa123d8640 1 -- 172.21.15.172:0/1650546741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0980 msgr2=0x558edeb9b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:54.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.876+0000 7faa123d8640 1 --2- 172.21.15.172:0/1650546741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0980 0x558edeb9b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.876+0000 7faa123d8640 1 -- 172.21.15.172:0/1650546741 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558edeb9c090 con 0x558edebab240 2024-09-10T08:47:54.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.876+0000 7faa123d8640 1 --2- 172.21.15.172:0/1650546741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edebab240 0x558edebf5960 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7faa040263a0 tx=0x7faa040099d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37193f05d88d058d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:54.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.876+0000 7fa9f77fe640 1 -- 172.21.15.172:0/1650546741 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0402d070 con 0x558edebab240 2024-09-10T08:47:54.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.876+0000 7fa9f77fe640 1 -- 172.21.15.172:0/1650546741 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa04028cc0 con 0x558edebab240 2024-09-10T08:47:54.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.876+0000 7fa9f77fe640 1 -- 172.21.15.172:0/1650546741 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0401f070 con 0x558edebab240 2024-09-10T08:47:54.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.877+0000 7faa16214740 1 -- 172.21.15.172:0/1650546741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edebab240 msgr2=0x558edebf5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:54.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.877+0000 7faa16214740 1 --2- 172.21.15.172:0/1650546741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edebab240 0x558edebf5960 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7faa040263a0 tx=0x7faa040099d0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.878+0000 7faa16214740 1 -- 172.21.15.172:0/1650546741 shutdown_connections 2024-09-10T08:47:54.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.878+0000 7faa16214740 1 --2- 172.21.15.172:0/1650546741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edebab240 0x558edebf5960 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.878+0000 7faa16214740 1 --2- 172.21.15.172:0/1650546741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0980 0x558edeb9b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.878+0000 7faa16214740 1 --2- 172.21.15.172:0/1650546741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edeba0060 0x558edeba2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.878+0000 7faa16214740 1 -- 172.21.15.172:0/1650546741 >> 172.21.15.172:0/1650546741 conn(0x558edeb9c5f0 msgr2=0x558edeb9af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:54.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.878+0000 7faa16214740 1 -- 172.21.15.172:0/1650546741 shutdown_connections 2024-09-10T08:47:54.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.878+0000 7faa16214740 1 -- 172.21.15.172:0/1650546741 wait complete. 2024-09-10T08:47:54.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.879+0000 7faa16214740 1 -- start start 2024-09-10T08:47:54.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.879+0000 7faa16214740 1 Processor -- start 2024-09-10T08:47:54.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.880+0000 7faa16214740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0060 0x558edebaf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.880+0000 7faa16214740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edeba0980 0x558edebaf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.880+0000 7faa113d6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edeba0980 0x558edebaf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:54.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.880+0000 7faa113d6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edeba0980 0x558edebaf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55770/0 (socket says 172.21.15.172:55770) 2024-09-10T08:47:54.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.880+0000 7faa113d6640 1 -- 172.21.15.172:0/3353917566 learned_addr learned my addr 172.21.15.172:0/3353917566 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:54.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.880+0000 7faa16214740 1 --2- 172.21.15.172:0/3353917566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edebab240 0x558edec57930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.880+0000 7faa16214740 1 -- 172.21.15.172:0/3353917566 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c1adc90 con 0x558edeba0980 2024-09-10T08:47:54.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.880+0000 7faa16214740 1 -- 172.21.15.172:0/3353917566 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c1ad5c0 con 0x558edeba0060 2024-09-10T08:47:54.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.880+0000 7faa16214740 1 -- 172.21.15.172:0/3353917566 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7faa0c1a1e70 con 0x558edebab240 2024-09-10T08:47:54.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.881+0000 7faa123d8640 1 --2- 172.21.15.172:0/3353917566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edebab240 0x558edec57930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:54.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.881+0000 7faa113d6640 1 -- 172.21.15.172:0/3353917566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edebab240 msgr2=0x558edec57930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:54.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.881+0000 7faa113d6640 1 --2- 172.21.15.172:0/3353917566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edebab240 0x558edec57930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.881+0000 7faa113d6640 1 -- 172.21.15.172:0/3353917566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0060 msgr2=0x558edebaf390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:47:54.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.881+0000 7faa113d6640 1 --2- 172.21.15.172:0/3353917566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0060 0x558edebaf390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.881+0000 7faa113d6640 1 -- 172.21.15.172:0/3353917566 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558edec57e70 con 0x558edeba0980 2024-09-10T08:47:54.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.881+0000 7faa16214740 10 client.0 _ll_register_callbacks cb 0x7faa0800f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:54.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.881+0000 7faa16214740 1 -- 172.21.15.172:0/3353917566 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa04028cc0 con 0x558edeba0980 2024-09-10T08:47:54.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.881+0000 7faa113d6640 1 --2- 172.21.15.172:0/3353917566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edeba0980 0x558edebaf8d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0bb120 tx=0x7fa9fc0f2940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab39ce321362e7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:54.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.881+0000 7fa9f57fa640 10 client.15843 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:54.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.882+0000 7fa9f57fa640 1 -- 172.21.15.172:0/3353917566 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc0bb1f0 con 0x558edeba0980 2024-09-10T08:47:54.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.882+0000 7fa9f57fa640 1 -- 172.21.15.172:0/3353917566 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa9fc0b9720 con 0x558edeba0980 2024-09-10T08:47:54.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.882+0000 7fa9f57fa640 1 -- 172.21.15.172:0/3353917566 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa9fc0bad80 con 0x558edeba0980 2024-09-10T08:47:54.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.882+0000 7faa16214740 10 client.15843 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:54.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.882+0000 7faa16214740 1 -- 172.21.15.172:0/3353917566 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=29}) -- 0x558edec58fa0 con 0x558edeba0980 2024-09-10T08:47:54.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.882+0000 7fa9f57fa640 1 -- 172.21.15.172:0/3353917566 <== mon.0 v2:172.21.15.172:3300/0 4 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa9fc0b7a90 con 0x558edeba0980 2024-09-10T08:47:54.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.882+0000 7faa16214740 10 client.15843 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:54.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.882+0000 7faa16214740 10 client.15843 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:47:54.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.882+0000 7faa16214740 20 client.15843 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:54.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.882+0000 7faa16214740 1 --2- 172.21.15.172:0/3353917566 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558edebedc20 0x558edec5b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:54.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.882+0000 7faa16214740 4 client.15843 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2d459bfa", "release": true, "await": true}] 2024-09-10T08:47:54.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.882+0000 7faa16214740 1 -- 172.21.15.172:0/3353917566 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2d459bfa", "release": true, "await": true}) -- 0x558edeb0d0a0 con 0x558edebedc20 2024-09-10T08:47:54.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.883+0000 7faa11bd7640 1 --2- 172.21.15.172:0/3353917566 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558edebedc20 0x558edec5b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:54.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.883+0000 7fa9f57fa640 10 client.15843 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:54.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.883+0000 7faa11bd7640 1 --2- 172.21.15.172:0/3353917566 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558edebedc20 0x558edec5b5c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e83501f63f59825 server_cookie=4bc81e034f358a32 in_seq=0 out_seq=0 2024-09-10T08:47:54.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.886+0000 7fa9f57fa640 1 -- 172.21.15.172:0/3353917566 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x558edeb0d0a0 con 0x558edebedc20 2024-09-10T08:47:54.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.886+0000 7fa9f57fa640 10 client.15843 handle_command_reply: tid=0 2024-09-10T08:47:54.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.886+0000 7faa16214740 1 client.15843 shutdown 2024-09-10T08:47:54.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.886+0000 7faa16214740 1 -- 172.21.15.172:0/3353917566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edeba0980 msgr2=0x558edebaf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:54.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.886+0000 7faa16214740 1 --2- 172.21.15.172:0/3353917566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edeba0980 0x558edebaf8d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0bb120 tx=0x7fa9fc0f2940 comp rx=0 tx=0).stop 2024-09-10T08:47:54.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.886+0000 7faa16214740 1 -- 172.21.15.172:0/3353917566 shutdown_connections 2024-09-10T08:47:54.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.886+0000 7faa16214740 1 --2- 172.21.15.172:0/3353917566 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558edebedc20 0x558edec5b5c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.886+0000 7faa16214740 1 --2- 172.21.15.172:0/3353917566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558edebab240 0x558edec57930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.886+0000 7fa9f57fa640 0 client.15843 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:54.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.886+0000 7faa16214740 1 --2- 172.21.15.172:0/3353917566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558edeba0980 0x558edebaf8d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.886+0000 7faa16214740 1 --2- 172.21.15.172:0/3353917566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558edeba0060 0x558edebaf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.886+0000 7faa16214740 1 -- 172.21.15.172:0/3353917566 >> 172.21.15.172:0/3353917566 conn(0x558edeb9c5f0 msgr2=0x558edeba80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:54.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.886+0000 7faa16214740 1 -- 172.21.15.172:0/3353917566 shutdown_connections 2024-09-10T08:47:54.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.886+0000 7faa16214740 1 -- 172.21.15.172:0/3353917566 wait complete. 2024-09-10T08:47:54.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.886+0000 7faa16214740 20 client.15843 trim_cache size 0 max 16384 2024-09-10T08:47:54.890 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:47:54.890 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 29, 2024-09-10T08:47:54.890 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:47:54.890 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 4, 2024-09-10T08:47:54.890 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:47:54.890 INFO:teuthology.orchestra.run.smithi172.stdout: "2d459bfa": { 2024-09-10T08:47:54.890 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 4, 2024-09-10T08:47:54.891 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:47:54.891 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:47:54.891 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:47:54.891 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:47:54.891 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:47:54.891 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:47:54.891 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 150.0, 2024-09-10T08:47:54.891 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:47:54.891 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:47:54.891 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:47:54.891 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:47:54.891 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:47:54.891 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:47:54.892 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:47:54.892 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:47:54.892 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:47:54.892 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:47:54.892 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:47:54.892 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:47:54.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.888+0000 7faa13e3b640 1 -- 172.21.15.172:0/2284362765 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa9f80a2c80 msgr2=0x7fa9f80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:54.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.888+0000 7faa13e3b640 1 --2- 172.21.15.172:0/2284362765 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa9f80a2c80 0x7fa9f80a5140 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x558edeb9d450 tx=0x7faa08009260 comp rx=0 tx=0).stop 2024-09-10T08:47:54.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.888+0000 7faa13e3b640 1 -- 172.21.15.172:0/2284362765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa0c101e20 msgr2=0x7faa0c1964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:54.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.888+0000 7faa13e3b640 1 --2- 172.21.15.172:0/2284362765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa0c101e20 0x7faa0c1964b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc02a860 tx=0x7fa9fc030bd0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.888+0000 7faa13e3b640 1 -- 172.21.15.172:0/2284362765 shutdown_connections 2024-09-10T08:47:54.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.888+0000 7faa13e3b640 1 --2- 172.21.15.172:0/2284362765 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa9f80a2c80 0x7fa9f80a5140 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.888+0000 7faa13e3b640 1 --2- 172.21.15.172:0/2284362765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa0c105de0 0x7faa0c1969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.888+0000 7faa13e3b640 1 --2- 172.21.15.172:0/2284362765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa0c101e20 0x7faa0c1964b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.888+0000 7faa13e3b640 1 --2- 172.21.15.172:0/2284362765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa0c101470 0x7faa0c195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:54.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.888+0000 7faa13e3b640 1 -- 172.21.15.172:0/2284362765 >> 172.21.15.172:0/2284362765 conn(0x7faa0c077230 msgr2=0x7faa0c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:54.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.888+0000 7faa13e3b640 1 -- 172.21.15.172:0/2284362765 shutdown_connections 2024-09-10T08:47:54.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:54.888+0000 7faa13e3b640 1 -- 172.21.15.172:0/2284362765 wait complete. 2024-09-10T08:47:55.615 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2d459bfa', total seconds elapsed: 33.7 2024-09-10T08:47:55.615 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T08:47:55.615 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:47:55.988 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:56.423 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:56 smithi172 ceph-mon[30854]: pgmap v401: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T08:47:56.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.509+0000 7fb95da2b640 1 -- 172.21.15.172:0/771025151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb958100ca0 msgr2=0x7fb958106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:56.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.509+0000 7fb95da2b640 1 --2- 172.21.15.172:0/771025151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb958100ca0 0x7fb958106840 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb94c009930 tx=0x7fb94c02a420 comp rx=0 tx=0).stop 2024-09-10T08:47:56.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.509+0000 7fb95da2b640 1 -- 172.21.15.172:0/771025151 shutdown_connections 2024-09-10T08:47:56.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.509+0000 7fb95da2b640 1 --2- 172.21.15.172:0/771025151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb958106f50 0x7fb95810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:56.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.509+0000 7fb95da2b640 1 --2- 172.21.15.172:0/771025151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb958100ca0 0x7fb958106840 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:56.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.509+0000 7fb95da2b640 1 --2- 172.21.15.172:0/771025151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9581002f0 0x7fb9581006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:56.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.509+0000 7fb95da2b640 1 -- 172.21.15.172:0/771025151 >> 172.21.15.172:0/771025151 conn(0x7fb958077220 msgr2=0x7fb9580fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:56.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.509+0000 7fb95da2b640 1 -- 172.21.15.172:0/771025151 shutdown_connections 2024-09-10T08:47:56.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.509+0000 7fb95da2b640 1 -- 172.21.15.172:0/771025151 wait complete. 2024-09-10T08:47:56.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.510+0000 7fb95da2b640 1 Processor -- start 2024-09-10T08:47:56.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.510+0000 7fb95da2b640 1 -- start start 2024-09-10T08:47:56.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.510+0000 7fb95da2b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9581002f0 0x7fb958198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:56.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.510+0000 7fb95da2b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb958100ca0 0x7fb958198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:56.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.510+0000 7fb956ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9581002f0 0x7fb958198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:56.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.511+0000 7fb956ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9581002f0 0x7fb958198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39620/0 (socket says 172.21.15.172:39620) 2024-09-10T08:47:56.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.511+0000 7fb956ffd640 1 -- 172.21.15.172:0/2427984745 learned_addr learned my addr 172.21.15.172:0/2427984745 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:56.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.511+0000 7fb95da2b640 1 --2- 172.21.15.172:0/2427984745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb958106f50 0x7fb95819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:56.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.511+0000 7fb9567fc640 1 --2- 172.21.15.172:0/2427984745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb958100ca0 0x7fb958198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:56.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.511+0000 7fb95da2b640 1 -- 172.21.15.172:0/2427984745 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb958111e50 con 0x7fb9581002f0 2024-09-10T08:47:56.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.511+0000 7fb95da2b640 1 -- 172.21.15.172:0/2427984745 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb958111cd0 con 0x7fb958106f50 2024-09-10T08:47:56.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.511+0000 7fb95da2b640 1 -- 172.21.15.172:0/2427984745 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb958111fd0 con 0x7fb958100ca0 2024-09-10T08:47:56.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.511+0000 7fb9577fe640 1 --2- 172.21.15.172:0/2427984745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb958106f50 0x7fb95819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:56.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.511+0000 7fb956ffd640 1 -- 172.21.15.172:0/2427984745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb958100ca0 msgr2=0x7fb958198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:56.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.511+0000 7fb956ffd640 1 --2- 172.21.15.172:0/2427984745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb958100ca0 0x7fb958198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:56.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.511+0000 7fb956ffd640 1 -- 172.21.15.172:0/2427984745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb958106f50 msgr2=0x7fb95819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:56.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.511+0000 7fb956ffd640 1 --2- 172.21.15.172:0/2427984745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb958106f50 0x7fb95819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:56.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.511+0000 7fb956ffd640 1 -- 172.21.15.172:0/2427984745 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95819d340 con 0x7fb9581002f0 2024-09-10T08:47:56.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.511+0000 7fb9577fe640 1 --2- 172.21.15.172:0/2427984745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb958106f50 0x7fb95819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:47:56.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.511+0000 7fb956ffd640 1 --2- 172.21.15.172:0/2427984745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9581002f0 0x7fb958198430 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb94800d950 tx=0x7fb94800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:56.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.512+0000 7fb95ca29640 1 -- 172.21.15.172:0/2427984745 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948017030 con 0x7fb9581002f0 2024-09-10T08:47:56.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.512+0000 7fb95da2b640 1 -- 172.21.15.172:0/2427984745 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95819d630 con 0x7fb9581002f0 2024-09-10T08:47:56.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.512+0000 7fb95ca29640 1 -- 172.21.15.172:0/2427984745 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb948004020 con 0x7fb9581002f0 2024-09-10T08:47:56.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.512+0000 7fb95ca29640 1 -- 172.21.15.172:0/2427984745 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9480047a0 con 0x7fb9581002f0 2024-09-10T08:47:56.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.512+0000 7fb95da2b640 1 -- 172.21.15.172:0/2427984745 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9581a4f10 con 0x7fb9581002f0 2024-09-10T08:47:56.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.514+0000 7fb95da2b640 1 -- 172.21.15.172:0/2427984745 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb958199790 con 0x7fb9581002f0 2024-09-10T08:47:56.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.515+0000 7fb95ca29640 1 -- 172.21.15.172:0/2427984745 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb948009a30 con 0x7fb9581002f0 2024-09-10T08:47:56.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.515+0000 7fb95ca29640 1 --2- 172.21.15.172:0/2427984745 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb9400a2b80 0x7fb9400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:56.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.516+0000 7fb95ca29640 1 -- 172.21.15.172:0/2427984745 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb9480cee20 con 0x7fb9581002f0 2024-09-10T08:47:56.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.516+0000 7fb9567fc640 1 --2- 172.21.15.172:0/2427984745 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb9400a2b80 0x7fb9400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:56.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.517+0000 7fb9567fc640 1 --2- 172.21.15.172:0/2427984745 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb9400a2b80 0x7fb9400a5040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb94c004770 tx=0x7fb94c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:56.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.521+0000 7fb95ca29640 1 -- 172.21.15.172:0/2427984745 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb94809ba20 con 0x7fb9581002f0 2024-09-10T08:47:56.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.730+0000 7fb95da2b640 1 -- 172.21.15.172:0/2427984745 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb95802d860 con 0x7fb9581002f0 2024-09-10T08:47:56.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.732+0000 7fb95ca29640 1 -- 172.21.15.172:0/2427984745 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7fb94809b320 con 0x7fb9581002f0 2024-09-10T08:47:56.733 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:47:56.733 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":29,"btime":"2024-09-10T08:46:37:242916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:46:36.267985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:47:56.733 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 29 2024-09-10T08:47:56.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.735+0000 7fb95da2b640 1 -- 172.21.15.172:0/2427984745 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb9400a2b80 msgr2=0x7fb9400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:56.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.735+0000 7fb95da2b640 1 --2- 172.21.15.172:0/2427984745 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb9400a2b80 0x7fb9400a5040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb94c004770 tx=0x7fb94c000bc0 comp rx=0 tx=0).stop 2024-09-10T08:47:56.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.735+0000 7fb95da2b640 1 -- 172.21.15.172:0/2427984745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9581002f0 msgr2=0x7fb958198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:56.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.735+0000 7fb95da2b640 1 --2- 172.21.15.172:0/2427984745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9581002f0 0x7fb958198430 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb94800d950 tx=0x7fb94800de10 comp rx=0 tx=0).stop 2024-09-10T08:47:56.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.735+0000 7fb95da2b640 1 -- 172.21.15.172:0/2427984745 shutdown_connections 2024-09-10T08:47:56.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.735+0000 7fb95da2b640 1 --2- 172.21.15.172:0/2427984745 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb9400a2b80 0x7fb9400a5040 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:56.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.735+0000 7fb95da2b640 1 --2- 172.21.15.172:0/2427984745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb958106f50 0x7fb95819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:56.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.735+0000 7fb95da2b640 1 --2- 172.21.15.172:0/2427984745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb958100ca0 0x7fb958198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:56.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.735+0000 7fb95da2b640 1 --2- 172.21.15.172:0/2427984745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb9581002f0 0x7fb958198430 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:56.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.735+0000 7fb95da2b640 1 -- 172.21.15.172:0/2427984745 >> 172.21.15.172:0/2427984745 conn(0x7fb958077220 msgr2=0x7fb958105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:56.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.736+0000 7fb95da2b640 1 -- 172.21.15.172:0/2427984745 shutdown_connections 2024-09-10T08:47:56.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:56.736+0000 7fb95da2b640 1 -- 172.21.15.172:0/2427984745 wait complete. 2024-09-10T08:47:56.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:56 smithi174 ceph-mon[34068]: pgmap v401: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T08:47:56.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:56 smithi191 ceph-mon[35314]: pgmap v401: 129 pgs: 129 active+clean; 672 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T08:47:57.434 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:57 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2427984745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:57.472 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T08:47:57.473 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T08:47:57.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:57 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2427984745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:57.787 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:47:57.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:57 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2427984745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:47:58.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.367+0000 7f261579f640 1 -- 172.21.15.172:0/788653627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26101034b0 msgr2=0x7f261010b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:58.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.367+0000 7f261579f640 1 --2- 172.21.15.172:0/788653627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26101034b0 0x7f261010b9a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2604009900 tx=0x7f260402a480 comp rx=0 tx=0).stop 2024-09-10T08:47:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.367+0000 7f261579f640 1 -- 172.21.15.172:0/788653627 shutdown_connections 2024-09-10T08:47:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.367+0000 7f261579f640 1 --2- 172.21.15.172:0/788653627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f261010bfc0 0x7f261010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.367+0000 7f261579f640 1 --2- 172.21.15.172:0/788653627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26101034b0 0x7f261010b9a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.367+0000 7f261579f640 1 --2- 172.21.15.172:0/788653627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2610102b90 0x7f2610102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.367+0000 7f261579f640 1 -- 172.21.15.172:0/788653627 >> 172.21.15.172:0/788653627 conn(0x7f26100fc610 msgr2=0x7f26100fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.367+0000 7f261579f640 1 -- 172.21.15.172:0/788653627 shutdown_connections 2024-09-10T08:47:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.367+0000 7f261579f640 1 -- 172.21.15.172:0/788653627 wait complete. 2024-09-10T08:47:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.368+0000 7f261579f640 1 Processor -- start 2024-09-10T08:47:58.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.368+0000 7f261579f640 1 -- start start 2024-09-10T08:47:58.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.369+0000 7f261579f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2610102b90 0x7f261019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.369+0000 7f261579f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26101034b0 0x7f261019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.369+0000 7f260ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2610102b90 0x7f261019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.369+0000 7f260ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2610102b90 0x7f261019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39634/0 (socket says 172.21.15.172:39634) 2024-09-10T08:47:58.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.369+0000 7f260ffff640 1 -- 172.21.15.172:0/802221165 learned_addr learned my addr 172.21.15.172:0/802221165 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:58.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.369+0000 7f261579f640 1 --2- 172.21.15.172:0/802221165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261010bfc0 0x7f261019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.369+0000 7f261579f640 1 -- 172.21.15.172:0/802221165 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2610111c70 con 0x7f2610102b90 2024-09-10T08:47:58.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.369+0000 7f261579f640 1 -- 172.21.15.172:0/802221165 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2610111af0 con 0x7f26101034b0 2024-09-10T08:47:58.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.369+0000 7f261579f640 1 -- 172.21.15.172:0/802221165 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2610111df0 con 0x7f261010bfc0 2024-09-10T08:47:58.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.370+0000 7f260f7fe640 1 --2- 172.21.15.172:0/802221165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26101034b0 0x7f261019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.370+0000 7f2614f9e640 1 --2- 172.21.15.172:0/802221165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261010bfc0 0x7f261019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.370+0000 7f260ffff640 1 -- 172.21.15.172:0/802221165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261010bfc0 msgr2=0x7f261019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:58.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.370+0000 7f260ffff640 1 --2- 172.21.15.172:0/802221165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261010bfc0 0x7f261019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.370+0000 7f260ffff640 1 -- 172.21.15.172:0/802221165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26101034b0 msgr2=0x7f261019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:58.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.370+0000 7f260ffff640 1 --2- 172.21.15.172:0/802221165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26101034b0 0x7f261019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.370+0000 7f260ffff640 1 -- 172.21.15.172:0/802221165 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261019f3b0 con 0x7f2610102b90 2024-09-10T08:47:58.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.371+0000 7f260ffff640 1 --2- 172.21.15.172:0/802221165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2610102b90 0x7f261019a400 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f260000c970 tx=0x7f260000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:58.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.371+0000 7f260d7fa640 1 -- 172.21.15.172:0/802221165 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600007bb0 con 0x7f2610102b90 2024-09-10T08:47:58.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.371+0000 7f260d7fa640 1 -- 172.21.15.172:0/802221165 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2600007d50 con 0x7f2610102b90 2024-09-10T08:47:58.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.371+0000 7f260d7fa640 1 -- 172.21.15.172:0/802221165 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260000f450 con 0x7f2610102b90 2024-09-10T08:47:58.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.371+0000 7f261579f640 1 -- 172.21.15.172:0/802221165 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261019f6a0 con 0x7f2610102b90 2024-09-10T08:47:58.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.371+0000 7f261579f640 1 -- 172.21.15.172:0/802221165 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26101a6f80 con 0x7f2610102b90 2024-09-10T08:47:58.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.374+0000 7f260d7fa640 1 -- 172.21.15.172:0/802221165 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2600004f20 con 0x7f2610102b90 2024-09-10T08:47:58.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.375+0000 7f2618dda740 1 Processor -- start 2024-09-10T08:47:58.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.375+0000 7f2618dda740 1 -- start start 2024-09-10T08:47:58.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.375+0000 7f260d7fa640 1 --2- 172.21.15.172:0/802221165 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f25f80a2c00 0x7f25f80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.376+0000 7f2618dda740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbb900 0x556ed6cbbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.376+0000 7f2618dda740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cbc220 0x556ed6cc65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.376+0000 7f2618dda740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cc6ae0 0x556ed6cc8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.376+0000 7f260d7fa640 1 -- 172.21.15.172:0/802221165 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f26000ceff0 con 0x7f2610102b90 2024-09-10T08:47:58.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.376+0000 7f2618dda740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556ed6c212a0 con 0x556ed6cbc220 2024-09-10T08:47:58.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.376+0000 7f2618dda740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556ed6bc8b10 con 0x556ed6cbb900 2024-09-10T08:47:58.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.376+0000 7f2618dda740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556ed6c1d260 con 0x556ed6cc6ae0 2024-09-10T08:47:58.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.376+0000 7f2614f9e640 1 --2- 172.21.15.172:0/802221165 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f25f80a2c00 0x7f25f80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.376+0000 7f260ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbb900 0x556ed6cbbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.376+0000 7f260f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cbc220 0x556ed6cc65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.376+0000 7f260ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbb900 0x556ed6cbbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:60012/0 (socket says 172.21.15.172:60012) 2024-09-10T08:47:58.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.376+0000 7f260f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cbc220 0x556ed6cc65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39646/0 (socket says 172.21.15.172:39646) 2024-09-10T08:47:58.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.376+0000 7f2614f9e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cc6ae0 0x556ed6cc8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.376+0000 7f260f7fe640 1 -- 172.21.15.172:0/3856461289 learned_addr learned my addr 172.21.15.172:0/3856461289 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:58.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.377+0000 7f2614f9e640 1 --2- 172.21.15.172:0/802221165 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f25f80a2c00 0x7f25f80a50c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2608005fd0 tx=0x7f2608014b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:58.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.377+0000 7f2614f9e640 1 -- 172.21.15.172:0/3856461289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbb900 msgr2=0x556ed6cbbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:58.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.377+0000 7f2614f9e640 1 --2- 172.21.15.172:0/3856461289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbb900 0x556ed6cbbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.377+0000 7f2614f9e640 1 -- 172.21.15.172:0/3856461289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cbc220 msgr2=0x556ed6cc65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:58.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.377+0000 7f2614f9e640 1 --2- 172.21.15.172:0/3856461289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cbc220 0x556ed6cc65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.377+0000 7f2614f9e640 1 -- 172.21.15.172:0/3856461289 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ed6992b90 con 0x556ed6cc6ae0 2024-09-10T08:47:58.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.377+0000 7f260ffff640 1 --2- 172.21.15.172:0/3856461289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbb900 0x556ed6cbbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-10T08:47:58.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.377+0000 7f2614f9e640 1 --2- 172.21.15.172:0/3856461289 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cc6ae0 0x556ed6cc8ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2608012760 tx=0x7f260801e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a104549ac28e3683 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:58.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.377+0000 7f25f67fc640 1 -- 172.21.15.172:0/3856461289 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260801e9b0 con 0x556ed6cc6ae0 2024-09-10T08:47:58.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.377+0000 7f25f67fc640 1 -- 172.21.15.172:0/3856461289 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2608020440 con 0x556ed6cc6ae0 2024-09-10T08:47:58.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.378+0000 7f2618dda740 1 -- 172.21.15.172:0/3856461289 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cc6ae0 msgr2=0x556ed6cc8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:58.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.378+0000 7f2618dda740 1 --2- 172.21.15.172:0/3856461289 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cc6ae0 0x556ed6cc8ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2608012760 tx=0x7f260801e000 comp rx=0 tx=0).stop 2024-09-10T08:47:58.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.378+0000 7f2618dda740 1 -- 172.21.15.172:0/3856461289 shutdown_connections 2024-09-10T08:47:58.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.378+0000 7f2618dda740 1 --2- 172.21.15.172:0/3856461289 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cc6ae0 0x556ed6cc8ed0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.378+0000 7f2618dda740 1 --2- 172.21.15.172:0/3856461289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cbc220 0x556ed6cc65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.378+0000 7f2618dda740 1 --2- 172.21.15.172:0/3856461289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbb900 0x556ed6cbbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.378+0000 7f2618dda740 1 -- 172.21.15.172:0/3856461289 >> 172.21.15.172:0/3856461289 conn(0x556ed6cb7e90 msgr2=0x556ed6cb8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:58.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.379+0000 7f2618dda740 1 -- 172.21.15.172:0/3856461289 shutdown_connections 2024-09-10T08:47:58.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.379+0000 7f2618dda740 1 -- 172.21.15.172:0/3856461289 wait complete. 2024-09-10T08:47:58.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.379+0000 7f2618dda740 1 -- start start 2024-09-10T08:47:58.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.380+0000 7f2618dda740 1 Processor -- start 2024-09-10T08:47:58.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.380+0000 7f2618dda740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbb900 0x556ed6cc3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.380+0000 7f2618dda740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cbc220 0x556ed6cc3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.380+0000 7f2618dda740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cc6ae0 0x556ed6cc43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.380+0000 7f2618dda740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556ed6cc9750 con 0x556ed6cc6ae0 2024-09-10T08:47:58.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.380+0000 7f260ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbb900 0x556ed6cc3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.380+0000 7f2618dda740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556ed6ba96e0 con 0x556ed6cbb900 2024-09-10T08:47:58.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.380+0000 7f260ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbb900 0x556ed6cc3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:60022/0 (socket says 172.21.15.172:60022) 2024-09-10T08:47:58.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.381+0000 7f260ffff640 1 -- 172.21.15.172:0/1436155068 learned_addr learned my addr 172.21.15.172:0/1436155068 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:58.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.381+0000 7f260f7fe640 1 --2- 172.21.15.172:0/1436155068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cbc220 0x556ed6cc3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.381+0000 7f2614f9e640 1 --2- 172.21.15.172:0/1436155068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cc6ae0 0x556ed6cc43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.381+0000 7f2618dda740 1 -- 172.21.15.172:0/1436155068 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556ed6cc98d0 con 0x556ed6cbc220 2024-09-10T08:47:58.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.381+0000 7f2618dda740 10 client.0 _ll_register_callbacks cb 0x556ed6bcda80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:58.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.381+0000 7f2614f9e640 1 -- 172.21.15.172:0/1436155068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cbc220 msgr2=0x556ed6cc3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:58.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.381+0000 7f2614f9e640 1 --2- 172.21.15.172:0/1436155068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cbc220 0x556ed6cc3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.381+0000 7f2614f9e640 1 -- 172.21.15.172:0/1436155068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbb900 msgr2=0x556ed6cc3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:58.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.381+0000 7f2614f9e640 1 --2- 172.21.15.172:0/1436155068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbb900 0x556ed6cc3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.381+0000 7f2614f9e640 1 -- 172.21.15.172:0/1436155068 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ed6cd2c40 con 0x556ed6cc6ae0 2024-09-10T08:47:58.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.381+0000 7f2614f9e640 1 --2- 172.21.15.172:0/1436155068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cc6ae0 0x556ed6cc43a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2608009d80 tx=0x7f2608009fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75d0ee93f8ae756 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:58.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.381+0000 7f25ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:47:58.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.382+0000 7f25ef7fe640 1 -- 172.21.15.172:0/1436155068 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608004db0 con 0x556ed6cc6ae0 2024-09-10T08:47:58.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.382+0000 7f2618dda740 1 -- 172.21.15.172:0/1436155068 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2608020440 con 0x556ed6cc6ae0 2024-09-10T08:47:58.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.382+0000 7f25ef7fe640 1 -- 172.21.15.172:0/1436155068 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f260801f870 con 0x556ed6cc6ae0 2024-09-10T08:47:58.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.382+0000 7f25ef7fe640 1 -- 172.21.15.172:0/1436155068 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2608014020 con 0x556ed6cc6ae0 2024-09-10T08:47:58.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.382+0000 7f2618dda740 10 client.15861 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:58.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.382+0000 7f2618dda740 1 -- 172.21.15.172:0/1436155068 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556ed6cd5510 con 0x556ed6cc6ae0 2024-09-10T08:47:58.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.382+0000 7f25ef7fe640 1 -- 172.21.15.172:0/1436155068 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2608020440 con 0x556ed6cc6ae0 2024-09-10T08:47:58.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.383+0000 7f25ef7fe640 1 -- 172.21.15.172:0/1436155068 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f260800f040 con 0x556ed6cc6ae0 2024-09-10T08:47:58.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.383+0000 7f2618dda740 10 client.15861 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:58.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.383+0000 7f2618dda740 10 client.15861 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:47:58.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.383+0000 7f2618dda740 20 client.15861 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:58.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.383+0000 7f2618dda740 1 --2- 172.21.15.172:0/1436155068 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556ed6cd6920 0x556ed6cd8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.383+0000 7f2618dda740 4 client.15861 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:47:58.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.383+0000 7f2618dda740 1 -- 172.21.15.172:0/1436155068 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ed6c2e9d0 con 0x556ed6cd6920 2024-09-10T08:47:58.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.383+0000 7f260ffff640 1 --2- 172.21.15.172:0/1436155068 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556ed6cd6920 0x556ed6cd8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.384+0000 7f25ef7fe640 10 client.15861 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:58.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.384+0000 7f260ffff640 1 --2- 172.21.15.172:0/1436155068 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556ed6cd6920 0x556ed6cd8d10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aca79514a9bd53b7 server_cookie=bfa2a32b9e85807b in_seq=0 out_seq=0 2024-09-10T08:47:58.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.384+0000 7f25ef7fe640 1 -- 172.21.15.172:0/1436155068 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ed6c2e9d0 con 0x556ed6cd6920 2024-09-10T08:47:58.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.384+0000 7f25ef7fe640 10 client.15861 handle_command_reply: tid=0 2024-09-10T08:47:58.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.384+0000 7f2618dda740 1 client.15861 shutdown 2024-09-10T08:47:58.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.385+0000 7f2618dda740 1 -- 172.21.15.172:0/1436155068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cc6ae0 msgr2=0x556ed6cc43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:58.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.385+0000 7f2618dda740 1 --2- 172.21.15.172:0/1436155068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cc6ae0 0x556ed6cc43a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2608009d80 tx=0x7f2608009fc0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.385+0000 7f2618dda740 1 -- 172.21.15.172:0/1436155068 shutdown_connections 2024-09-10T08:47:58.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.385+0000 7f2618dda740 1 --2- 172.21.15.172:0/1436155068 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556ed6cd6920 0x556ed6cd8d10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.385+0000 7f2618dda740 1 --2- 172.21.15.172:0/1436155068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cc6ae0 0x556ed6cc43a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.385+0000 7f2618dda740 1 --2- 172.21.15.172:0/1436155068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cbc220 0x556ed6cc3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.385+0000 7f2618dda740 1 --2- 172.21.15.172:0/1436155068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbb900 0x556ed6cc3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.385+0000 7f2618dda740 1 -- 172.21.15.172:0/1436155068 >> 172.21.15.172:0/1436155068 conn(0x556ed6cb7e90 msgr2=0x556ed6cbdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:58.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.385+0000 7f25ef7fe640 0 client.15861 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:58.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.385+0000 7f2618dda740 1 -- 172.21.15.172:0/1436155068 shutdown_connections 2024-09-10T08:47:58.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.385+0000 7f2618dda740 1 -- 172.21.15.172:0/1436155068 wait complete. 2024-09-10T08:47:58.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.385+0000 7f2618dda740 20 client.15861 trim_cache size 0 max 16384 2024-09-10T08:47:58.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.419+0000 7f2618dda740 1 Processor -- start 2024-09-10T08:47:58.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.419+0000 7f2618dda740 1 -- start start 2024-09-10T08:47:58.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.419+0000 7f2618dda740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cbb900 0x556ed6cc5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.419+0000 7f2618dda740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbc220 0x556ed6cde4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.419+0000 7f2618dda740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cc6ae0 0x556ed6cdea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.419+0000 7f2618dda740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f26101b0b50 con 0x556ed6cc6ae0 2024-09-10T08:47:58.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.419+0000 7f2618dda740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26101b2690 con 0x556ed6cbc220 2024-09-10T08:47:58.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.419+0000 7f2618dda740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f26101b1220 con 0x556ed6cbb900 2024-09-10T08:47:58.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.419+0000 7f2614f9e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cc6ae0 0x556ed6cdea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.419+0000 7f2614f9e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cc6ae0 0x556ed6cdea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39656/0 (socket says 172.21.15.172:39656) 2024-09-10T08:47:58.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.419+0000 7f2614f9e640 1 -- 172.21.15.172:0/2605770458 learned_addr learned my addr 172.21.15.172:0/2605770458 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:58.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.419+0000 7f260f7fe640 1 --2- 172.21.15.172:0/2605770458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbc220 0x556ed6cde4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.420+0000 7f260ffff640 1 --2- 172.21.15.172:0/2605770458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cbb900 0x556ed6cc5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.420+0000 7f2614f9e640 1 -- 172.21.15.172:0/2605770458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cbb900 msgr2=0x556ed6cc5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:58.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.420+0000 7f2614f9e640 1 --2- 172.21.15.172:0/2605770458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cbb900 0x556ed6cc5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.420+0000 7f2614f9e640 1 -- 172.21.15.172:0/2605770458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbc220 msgr2=0x556ed6cde4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:58.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.420+0000 7f2614f9e640 1 --2- 172.21.15.172:0/2605770458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbc220 0x556ed6cde4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.420+0000 7f2614f9e640 1 -- 172.21.15.172:0/2605770458 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ed6cd9080 con 0x556ed6cc6ae0 2024-09-10T08:47:58.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.420+0000 7f260f7fe640 1 --2- 172.21.15.172:0/2605770458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbc220 0x556ed6cde4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:58.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.420+0000 7f2614f9e640 1 --2- 172.21.15.172:0/2605770458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cc6ae0 0x556ed6cdea30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2608012800 tx=0x7f2608005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8092f5fa5e93fd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:58.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.420+0000 7f25f5ffb640 1 -- 172.21.15.172:0/2605770458 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608020440 con 0x556ed6cc6ae0 2024-09-10T08:47:58.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.420+0000 7f25f5ffb640 1 -- 172.21.15.172:0/2605770458 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f260801e700 con 0x556ed6cc6ae0 2024-09-10T08:47:58.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.420+0000 7f25f5ffb640 1 -- 172.21.15.172:0/2605770458 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608041430 con 0x556ed6cc6ae0 2024-09-10T08:47:58.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.421+0000 7f2618dda740 1 -- 172.21.15.172:0/2605770458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cc6ae0 msgr2=0x556ed6cdea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:58.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.421+0000 7f2618dda740 1 --2- 172.21.15.172:0/2605770458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cc6ae0 0x556ed6cdea30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2608012800 tx=0x7f2608005c30 comp rx=0 tx=0).stop 2024-09-10T08:47:58.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.422+0000 7f2618dda740 1 -- 172.21.15.172:0/2605770458 shutdown_connections 2024-09-10T08:47:58.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.422+0000 7f2618dda740 1 --2- 172.21.15.172:0/2605770458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cc6ae0 0x556ed6cdea30 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.422+0000 7f2618dda740 1 --2- 172.21.15.172:0/2605770458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbc220 0x556ed6cde4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.422+0000 7f2618dda740 1 --2- 172.21.15.172:0/2605770458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cbb900 0x556ed6cc5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.422+0000 7f2618dda740 1 -- 172.21.15.172:0/2605770458 >> 172.21.15.172:0/2605770458 conn(0x556ed6cb7e90 msgr2=0x556ed6cb7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:58.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.422+0000 7f2618dda740 1 -- 172.21.15.172:0/2605770458 shutdown_connections 2024-09-10T08:47:58.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.422+0000 7f2618dda740 1 -- 172.21.15.172:0/2605770458 wait complete. 2024-09-10T08:47:58.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.423+0000 7f2618dda740 1 -- start start 2024-09-10T08:47:58.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.423+0000 7f2618dda740 1 Processor -- start 2024-09-10T08:47:58.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.424+0000 7f2618dda740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cbb900 0x556ed6ccf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.424+0000 7f260ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cbb900 0x556ed6ccf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.424+0000 7f260ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cbb900 0x556ed6ccf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39668/0 (socket says 172.21.15.172:39668) 2024-09-10T08:47:58.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.424+0000 7f260ffff640 1 -- 172.21.15.172:0/531275364 learned_addr learned my addr 172.21.15.172:0/531275364 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:47:58.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.424+0000 7f2618dda740 1 --2- 172.21.15.172:0/531275364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbc220 0x556ed6cc0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.424+0000 7f2618dda740 1 --2- 172.21.15.172:0/531275364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cc6ae0 0x556ed6cc0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.424+0000 7f2618dda740 1 -- 172.21.15.172:0/531275364 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f26101b1fc0 con 0x556ed6cbb900 2024-09-10T08:47:58.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.424+0000 7f2618dda740 1 -- 172.21.15.172:0/531275364 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26101b18f0 con 0x556ed6cbc220 2024-09-10T08:47:58.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.424+0000 7f2618dda740 1 -- 172.21.15.172:0/531275364 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f26101aeb30 con 0x556ed6cc6ae0 2024-09-10T08:47:58.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.424+0000 7f260f7fe640 1 --2- 172.21.15.172:0/531275364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbc220 0x556ed6cc0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.424+0000 7f2614f9e640 1 --2- 172.21.15.172:0/531275364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cc6ae0 0x556ed6cc0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.425+0000 7f2614f9e640 1 -- 172.21.15.172:0/531275364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbc220 msgr2=0x556ed6cc0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:58.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.425+0000 7f2614f9e640 1 --2- 172.21.15.172:0/531275364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbc220 0x556ed6cc0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.425+0000 7f2614f9e640 1 -- 172.21.15.172:0/531275364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cbb900 msgr2=0x556ed6ccf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:47:58.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.425+0000 7f2614f9e640 1 --2- 172.21.15.172:0/531275364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cbb900 0x556ed6ccf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.425+0000 7f2614f9e640 1 -- 172.21.15.172:0/531275364 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ed6cc1560 con 0x556ed6cc6ae0 2024-09-10T08:47:58.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.425+0000 7f260ffff640 1 --2- 172.21.15.172:0/531275364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cbb900 0x556ed6ccf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:47:58.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.425+0000 7f2618dda740 10 client.0 _ll_register_callbacks cb 0x7f2608020990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:47:58.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.425+0000 7f2618dda740 1 -- 172.21.15.172:0/531275364 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f260801e700 con 0x556ed6cc6ae0 2024-09-10T08:47:58.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.425+0000 7f2614f9e640 1 --2- 172.21.15.172:0/531275364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cc6ae0 0x556ed6cc0d60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f260801e9c0 tx=0x7f260801eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1349cf0a58731fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:47:58.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.425+0000 7f25ef7fe640 10 client.25255 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:47:58.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.426+0000 7f25ef7fe640 1 -- 172.21.15.172:0/531275364 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260801f7b0 con 0x556ed6cc6ae0 2024-09-10T08:47:58.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.426+0000 7f25ef7fe640 1 -- 172.21.15.172:0/531275364 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f260801f950 con 0x556ed6cc6ae0 2024-09-10T08:47:58.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.426+0000 7f25ef7fe640 1 -- 172.21.15.172:0/531275364 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f260801fbd0 con 0x556ed6cc6ae0 2024-09-10T08:47:58.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.426+0000 7f2618dda740 10 client.25255 fetch_fsmap learned FSMap version 29 2024-09-10T08:47:58.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.426+0000 7f2618dda740 1 -- 172.21.15.172:0/531275364 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({fsmap=29}) -- 0x556ed6d49a50 con 0x556ed6cc6ae0 2024-09-10T08:47:58.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.426+0000 7f25ef7fe640 1 -- 172.21.15.172:0/531275364 <== mon.1 v2:172.21.15.191:3300/0 4 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f260800f040 con 0x556ed6cc6ae0 2024-09-10T08:47:58.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.427+0000 7f2618dda740 10 client.25255 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:47:58.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.427+0000 7f2618dda740 10 client.25255 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:47:58.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.427+0000 7f2618dda740 20 client.25255 populate_metadata read hostname 'smithi172' 2024-09-10T08:47:58.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.427+0000 7f2618dda740 1 --2- 172.21.15.172:0/531275364 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556ed6d4a0c0 0x556ed6d4c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:47:58.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.427+0000 7f2618dda740 4 client.25255 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T08:47:58.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.427+0000 7f2618dda740 1 -- 172.21.15.172:0/531275364 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x556ed6c28d20 con 0x556ed6d4a0c0 2024-09-10T08:47:58.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.427+0000 7f260ffff640 1 --2- 172.21.15.172:0/531275364 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556ed6d4a0c0 0x556ed6d4c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:47:58.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.427+0000 7f25ef7fe640 10 client.25255 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:58.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.427+0000 7f260ffff640 1 --2- 172.21.15.172:0/531275364 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556ed6d4a0c0 0x556ed6d4c500 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa443fb2a4557cf0 server_cookie=2e9d8b6097dd5e82 in_seq=0 out_seq=0 2024-09-10T08:47:58.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.434+0000 7f25ef7fe640 1 -- 172.21.15.172:0/531275364 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x556ed6c28d20 con 0x556ed6d4a0c0 2024-09-10T08:47:58.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.434+0000 7f25ef7fe640 10 client.25255 handle_command_reply: tid=0 2024-09-10T08:47:58.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.434+0000 7f2618dda740 1 client.25255 shutdown 2024-09-10T08:47:58.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.434+0000 7f2618dda740 1 -- 172.21.15.172:0/531275364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cc6ae0 msgr2=0x556ed6cc0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:58.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.434+0000 7f2618dda740 1 --2- 172.21.15.172:0/531275364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cc6ae0 0x556ed6cc0d60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f260801e9c0 tx=0x7f260801eee0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.434+0000 7f2618dda740 1 -- 172.21.15.172:0/531275364 shutdown_connections 2024-09-10T08:47:58.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.435+0000 7f2618dda740 1 --2- 172.21.15.172:0/531275364 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556ed6d4a0c0 0x556ed6d4c500 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.435+0000 7f2618dda740 1 --2- 172.21.15.172:0/531275364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556ed6cc6ae0 0x556ed6cc0d60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.435+0000 7f25ef7fe640 0 client.25255 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:47:58.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.435+0000 7f2618dda740 1 --2- 172.21.15.172:0/531275364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ed6cbc220 0x556ed6cc0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.435+0000 7f2618dda740 1 --2- 172.21.15.172:0/531275364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ed6cbb900 0x556ed6ccf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.435+0000 7f2618dda740 1 -- 172.21.15.172:0/531275364 >> 172.21.15.172:0/531275364 conn(0x556ed6cb7e90 msgr2=0x556ed6cd1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:58.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.435+0000 7f2618dda740 1 -- 172.21.15.172:0/531275364 shutdown_connections 2024-09-10T08:47:58.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.435+0000 7f2618dda740 1 -- 172.21.15.172:0/531275364 wait complete. 2024-09-10T08:47:58.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.435+0000 7f2618dda740 20 client.25255 trim_cache size 0 max 16384 2024-09-10T08:47:58.437 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:47:58.437 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 29, 2024-09-10T08:47:58.438 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:47:58.438 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 6, 2024-09-10T08:47:58.438 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:47:58.438 INFO:teuthology.orchestra.run.smithi172.stdout: "428b29b7": { 2024-09-10T08:47:58.438 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 6, 2024-09-10T08:47:58.438 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:47:58.438 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:47:58.438 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:47:58.438 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:47:58.438 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:47:58.438 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:47:58.438 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 150.0, 2024-09-10T08:47:58.438 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:47:58.439 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:47:58.439 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:47:58.439 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:47:58.439 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:47:58.439 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:47:58.439 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:47:58.439 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:47:58.439 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:47:58.439 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:47:58.439 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:47:58.439 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:47:58.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.436+0000 7f261579f640 1 -- 172.21.15.172:0/802221165 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f25f80a2c00 msgr2=0x7f25f80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:58.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.436+0000 7f261579f640 1 --2- 172.21.15.172:0/802221165 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f25f80a2c00 0x7f25f80a50c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2608005fd0 tx=0x7f2608014b50 comp rx=0 tx=0).stop 2024-09-10T08:47:58.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.436+0000 7f261579f640 1 -- 172.21.15.172:0/802221165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2610102b90 msgr2=0x7f261019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:47:58.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.436+0000 7f261579f640 1 --2- 172.21.15.172:0/802221165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2610102b90 0x7f261019a400 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f260000c970 tx=0x7f260000ce30 comp rx=0 tx=0).stop 2024-09-10T08:47:58.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.437+0000 7f261579f640 1 -- 172.21.15.172:0/802221165 shutdown_connections 2024-09-10T08:47:58.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.437+0000 7f261579f640 1 --2- 172.21.15.172:0/802221165 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f25f80a2c00 0x7f25f80a50c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.437+0000 7f261579f640 1 --2- 172.21.15.172:0/802221165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261010bfc0 0x7f261019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.437+0000 7f261579f640 1 --2- 172.21.15.172:0/802221165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26101034b0 0x7f261019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.437+0000 7f261579f640 1 --2- 172.21.15.172:0/802221165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2610102b90 0x7f261019a400 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:47:58.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.437+0000 7f261579f640 1 -- 172.21.15.172:0/802221165 >> 172.21.15.172:0/802221165 conn(0x7f26100fc610 msgr2=0x7f26101017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:47:58.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.437+0000 7f261579f640 1 -- 172.21.15.172:0/802221165 shutdown_connections 2024-09-10T08:47:58.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:47:58.437+0000 7f261579f640 1 -- 172.21.15.172:0/802221165 wait complete. 2024-09-10T08:47:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:47:58 smithi172 ceph-mon[30854]: pgmap v402: 129 pgs: 129 active+clean; 673 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:47:58.643 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:47:58 smithi174 ceph-mon[34068]: pgmap v402: 129 pgs: 129 active+clean; 673 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:47:58.701 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:47:58 smithi191 ceph-mon[35314]: pgmap v402: 129 pgs: 129 active+clean; 673 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:47:59.248 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '428b29b7', quiesce took 3.6 seconds. Will release after: 26.4 2024-09-10T08:48:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:00 smithi172 ceph-mon[30854]: pgmap v403: 129 pgs: 129 active+clean; 673 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:00.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:00 smithi174 ceph-mon[34068]: pgmap v403: 129 pgs: 129 active+clean; 673 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:00.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:00 smithi191 ceph-mon[35314]: pgmap v403: 129 pgs: 129 active+clean; 673 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:01.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:01 smithi174 ceph-mon[34068]: pgmap v404: 129 pgs: 129 active+clean; 673 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:48:01.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:01 smithi191 ceph-mon[35314]: pgmap v404: 129 pgs: 129 active+clean; 673 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:48:01.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:01 smithi172 ceph-mon[30854]: pgmap v404: 129 pgs: 129 active+clean; 673 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:48:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.675+0000 7f63db5fd640 1 -- 172.21.15.172:0/1741555454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63d4100ce0 msgr2=0x7f63d4101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.675+0000 7f63db5fd640 1 --2- 172.21.15.172:0/1741555454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63d4100ce0 0x7f63d4101160 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009900 tx=0x7f63c402a410 comp rx=0 tx=0).stop 2024-09-10T08:48:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.675+0000 7f63db5fd640 1 -- 172.21.15.172:0/1741555454 shutdown_connections 2024-09-10T08:48:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.675+0000 7f63db5fd640 1 --2- 172.21.15.172:0/1741555454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63d41016a0 0x7f63d4108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.675+0000 7f63db5fd640 1 --2- 172.21.15.172:0/1741555454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63d4100ce0 0x7f63d4101160 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.675+0000 7f63db5fd640 1 --2- 172.21.15.172:0/1741555454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63d4102040 0x7f63d4102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.675+0000 7f63db5fd640 1 -- 172.21.15.172:0/1741555454 >> 172.21.15.172:0/1741555454 conn(0x7f63d40fb940 msgr2=0x7f63d40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:03.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.675+0000 7f63db5fd640 1 -- 172.21.15.172:0/1741555454 shutdown_connections 2024-09-10T08:48:03.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.675+0000 7f63db5fd640 1 -- 172.21.15.172:0/1741555454 wait complete. 2024-09-10T08:48:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.676+0000 7f63db5fd640 1 Processor -- start 2024-09-10T08:48:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.677+0000 7f63db5fd640 1 -- start start 2024-09-10T08:48:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.677+0000 7f63db5fd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63d4100ce0 0x7f63d4196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.677+0000 7f63db5fd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63d41016a0 0x7f63d4197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.677+0000 7f63db5fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63d4102040 0x7f63d419e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.677+0000 7f63db5fd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f63d410c5f0 con 0x7f63d4102040 2024-09-10T08:48:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.677+0000 7f63db5fd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f63d410c470 con 0x7f63d4100ce0 2024-09-10T08:48:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.677+0000 7f63db5fd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f63d410c770 con 0x7f63d41016a0 2024-09-10T08:48:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.677+0000 7f63d9b9a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63d4102040 0x7f63d419e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.677+0000 7f63d9b9a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63d4102040 0x7f63d419e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39678/0 (socket says 172.21.15.172:39678) 2024-09-10T08:48:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.677+0000 7f63d9b9a640 1 -- 172.21.15.172:0/1446054085 learned_addr learned my addr 172.21.15.172:0/1446054085 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.677+0000 7f63d9399640 1 --2- 172.21.15.172:0/1446054085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63d4100ce0 0x7f63d4196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.677+0000 7f63d8b98640 1 --2- 172.21.15.172:0/1446054085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63d41016a0 0x7f63d4197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.678+0000 7f63d9b9a640 1 -- 172.21.15.172:0/1446054085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63d41016a0 msgr2=0x7f63d4197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.678+0000 7f63d9b9a640 1 --2- 172.21.15.172:0/1446054085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63d41016a0 0x7f63d4197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.678+0000 7f63d9b9a640 1 -- 172.21.15.172:0/1446054085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63d4100ce0 msgr2=0x7f63d4196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.678+0000 7f63d9b9a640 1 --2- 172.21.15.172:0/1446054085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63d4100ce0 0x7f63d4196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.678+0000 7f63d9b9a640 1 -- 172.21.15.172:0/1446054085 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d419e7e0 con 0x7f63d4102040 2024-09-10T08:48:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.678+0000 7f63d8b98640 1 --2- 172.21.15.172:0/1446054085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63d41016a0 0x7f63d4197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.678+0000 7f63d9b9a640 1 --2- 172.21.15.172:0/1446054085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63d4102040 0x7f63d419e170 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00eb10 tx=0x7f63cc00d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.678+0000 7f63ca7fc640 1 -- 172.21.15.172:0/1446054085 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc004b20 con 0x7f63d4102040 2024-09-10T08:48:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.678+0000 7f63ca7fc640 1 -- 172.21.15.172:0/1446054085 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63cc017d00 con 0x7f63d4102040 2024-09-10T08:48:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.678+0000 7f63db5fd640 1 -- 172.21.15.172:0/1446054085 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d40ffc30 con 0x7f63d4102040 2024-09-10T08:48:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.678+0000 7f63ca7fc640 1 -- 172.21.15.172:0/1446054085 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc005660 con 0x7f63d4102040 2024-09-10T08:48:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.678+0000 7f63db5fd640 1 -- 172.21.15.172:0/1446054085 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d41001a0 con 0x7f63d4102040 2024-09-10T08:48:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.682+0000 7f63de47c740 1 Processor -- start 2024-09-10T08:48:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.682+0000 7f63ca7fc640 1 -- 172.21.15.172:0/1446054085 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f63cc005800 con 0x7f63d4102040 2024-09-10T08:48:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.682+0000 7f63de47c740 1 -- start start 2024-09-10T08:48:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.682+0000 7f63de47c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a1969a0 0x55b94a196da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.682+0000 7f63de47c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1972e0 0x55b94a19bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.682+0000 7f63de47c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a19c370 0x55b94a19e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.682+0000 7f63de47c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b94a0fa2f0 con 0x55b94a19c370 2024-09-10T08:48:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.682+0000 7f63de47c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b94a0a2270 con 0x55b94a1972e0 2024-09-10T08:48:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.682+0000 7f63de47c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b94a0f62b0 con 0x55b94a1969a0 2024-09-10T08:48:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.683+0000 7f63d8b98640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a1969a0 0x55b94a196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.683+0000 7f63d9399640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1972e0 0x55b94a19bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.683+0000 7f63d8b98640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a1969a0 0x55b94a196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:60390/0 (socket says 172.21.15.172:60390) 2024-09-10T08:48:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.683+0000 7f63d8b98640 1 -- 172.21.15.172:0/3403114577 learned_addr learned my addr 172.21.15.172:0/3403114577 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.683+0000 7f63d9399640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1972e0 0x55b94a19bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:60054/0 (socket says 172.21.15.172:60054) 2024-09-10T08:48:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.683+0000 7f63d8b98640 1 --2- 172.21.15.172:0/3403114577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a19c370 0x55b94a19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.683+0000 7f63d9399640 1 -- 172.21.15.172:0/3403114577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a1969a0 msgr2=0x55b94a196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.683+0000 7f63d9399640 1 --2- 172.21.15.172:0/3403114577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a1969a0 0x55b94a196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.683+0000 7f63d9399640 1 -- 172.21.15.172:0/3403114577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a19c370 msgr2=0x55b94a19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.683+0000 7f63d9399640 1 --2- 172.21.15.172:0/3403114577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a19c370 0x55b94a19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.683+0000 7f63d9399640 1 -- 172.21.15.172:0/3403114577 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b94a0a1340 con 0x55b94a1972e0 2024-09-10T08:48:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.683+0000 7f63d9399640 1 --2- 172.21.15.172:0/3403114577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1972e0 0x55b94a19bda0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009bf0 tx=0x7f63d0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac7989280c53dbab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.684+0000 7f63bf7fe640 1 -- 172.21.15.172:0/3403114577 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0004050 con 0x55b94a1972e0 2024-09-10T08:48:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.684+0000 7f63bf7fe640 1 -- 172.21.15.172:0/3403114577 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63d00054d0 con 0x55b94a1972e0 2024-09-10T08:48:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.684+0000 7f63bf7fe640 1 -- 172.21.15.172:0/3403114577 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0004c00 con 0x55b94a1972e0 2024-09-10T08:48:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.684+0000 7f63ca7fc640 1 --2- 172.21.15.172:0/1446054085 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f63c00a2cb0 0x7f63c00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.685+0000 7f63d8b98640 1 --2- 172.21.15.172:0/1446054085 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f63c00a2cb0 0x7f63c00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.685+0000 7f63de47c740 1 -- 172.21.15.172:0/3403114577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1972e0 msgr2=0x55b94a19bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.685+0000 7f63de47c740 1 --2- 172.21.15.172:0/3403114577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1972e0 0x55b94a19bda0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009bf0 tx=0x7f63d0011810 comp rx=0 tx=0).stop 2024-09-10T08:48:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.685+0000 7f63ca7fc640 1 -- 172.21.15.172:0/1446054085 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f63cc0d78a0 con 0x7f63d4102040 2024-09-10T08:48:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.685+0000 7f63de47c740 1 -- 172.21.15.172:0/3403114577 shutdown_connections 2024-09-10T08:48:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.685+0000 7f63de47c740 1 --2- 172.21.15.172:0/3403114577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a19c370 0x55b94a19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.685+0000 7f63de47c740 1 --2- 172.21.15.172:0/3403114577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1972e0 0x55b94a19bda0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.685+0000 7f63de47c740 1 --2- 172.21.15.172:0/3403114577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a1969a0 0x55b94a196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.685+0000 7f63de47c740 1 -- 172.21.15.172:0/3403114577 >> 172.21.15.172:0/3403114577 conn(0x55b94a192010 msgr2=0x55b94a192410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.685+0000 7f63d8b98640 1 --2- 172.21.15.172:0/1446054085 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f63c00a2cb0 0x7f63c00a5170 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f63c4000c00 tx=0x7f63c4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.685+0000 7f63de47c740 1 -- 172.21.15.172:0/3403114577 shutdown_connections 2024-09-10T08:48:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.685+0000 7f63de47c740 1 -- 172.21.15.172:0/3403114577 wait complete. 2024-09-10T08:48:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.686+0000 7f63de47c740 1 -- start start 2024-09-10T08:48:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.686+0000 7f63de47c740 1 Processor -- start 2024-09-10T08:48:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.687+0000 7f63de47c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a1969a0 0x55b94a1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.687+0000 7f63de47c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1972e0 0x55b94a1a1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.687+0000 7f63de47c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a19c370 0x55b94a1a8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.687+0000 7f63de47c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b94a19f340 con 0x55b94a19c370 2024-09-10T08:48:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.687+0000 7f63d8b98640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1972e0 0x55b94a1a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.687+0000 7f63d8b98640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1972e0 0x55b94a1a1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:60070/0 (socket says 172.21.15.172:60070) 2024-09-10T08:48:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.687+0000 7f63d8b98640 1 -- 172.21.15.172:0/758996632 learned_addr learned my addr 172.21.15.172:0/758996632 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.687+0000 7f63d9b9a640 1 --2- 172.21.15.172:0/758996632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a19c370 0x55b94a1a8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:03.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.687+0000 7f63d9399640 1 --2- 172.21.15.172:0/758996632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a1969a0 0x55b94a1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.687+0000 7f63de47c740 1 -- 172.21.15.172:0/758996632 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b94a083290 con 0x55b94a1972e0 2024-09-10T08:48:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.687+0000 7f63de47c740 1 -- 172.21.15.172:0/758996632 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b94a19f4c0 con 0x55b94a1969a0 2024-09-10T08:48:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.688+0000 7f63de47c740 10 client.0 _ll_register_callbacks cb 0x55b94a0b4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.688+0000 7f63d9b9a640 1 -- 172.21.15.172:0/758996632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a1969a0 msgr2=0x55b94a1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.688+0000 7f63d9b9a640 1 --2- 172.21.15.172:0/758996632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a1969a0 0x55b94a1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.688+0000 7f63d9b9a640 1 -- 172.21.15.172:0/758996632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1972e0 msgr2=0x55b94a1a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.688+0000 7f63d9b9a640 1 --2- 172.21.15.172:0/758996632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1972e0 0x55b94a1a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.688+0000 7f63d9b9a640 1 -- 172.21.15.172:0/758996632 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b949e0d210 con 0x55b94a19c370 2024-09-10T08:48:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.688+0000 7f63d9399640 1 --2- 172.21.15.172:0/758996632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a1969a0 0x55b94a1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.688+0000 7f63de47c740 1 -- 172.21.15.172:0/758996632 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63d00054d0 con 0x55b94a19c370 2024-09-10T08:48:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.688+0000 7f63d9b9a640 1 --2- 172.21.15.172:0/758996632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a19c370 0x55b94a1a8110 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0049d0 tx=0x7f63cc0a3870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4ba628afe71a9b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.688+0000 7f63bcff9640 10 client.15879 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.688+0000 7f63bcff9640 1 -- 172.21.15.172:0/758996632 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc0a7030 con 0x55b94a19c370 2024-09-10T08:48:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.688+0000 7f63bcff9640 1 -- 172.21.15.172:0/758996632 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f63cc0a04c0 con 0x55b94a19c370 2024-09-10T08:48:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.688+0000 7f63bcff9640 1 -- 172.21.15.172:0/758996632 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f63cc0aca10 con 0x55b94a19c370 2024-09-10T08:48:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.688+0000 7f63de47c740 10 client.15879 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.688+0000 7f63de47c740 1 -- 172.21.15.172:0/758996632 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55b949ed9620 con 0x55b94a19c370 2024-09-10T08:48:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.689+0000 7f63bcff9640 1 -- 172.21.15.172:0/758996632 <== mon.0 v2:172.21.15.172:3300/0 4 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f63cc0ac580 con 0x55b94a19c370 2024-09-10T08:48:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.689+0000 7f63de47c740 10 client.15879 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.689+0000 7f63de47c740 10 client.15879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.689+0000 7f63de47c740 20 client.15879 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.689+0000 7f63de47c740 1 --2- 172.21.15.172:0/758996632 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b94a1ac6a0 0x55b94a1aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.689+0000 7f63de47c740 4 client.15879 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.689+0000 7f63de47c740 1 -- 172.21.15.172:0/758996632 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b94a107a20 con 0x55b94a1ac6a0 2024-09-10T08:48:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.689+0000 7f63d9399640 1 --2- 172.21.15.172:0/758996632 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b94a1ac6a0 0x55b94a1aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.690+0000 7f63bcff9640 10 client.15879 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.690+0000 7f63d9399640 1 --2- 172.21.15.172:0/758996632 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b94a1ac6a0 0x55b94a1aeaf0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a8e74dfb2fcd60e server_cookie=3722c7313bfca4aa in_seq=0 out_seq=0 2024-09-10T08:48:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.690+0000 7f63bcff9640 1 -- 172.21.15.172:0/758996632 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b94a107a20 con 0x55b94a1ac6a0 2024-09-10T08:48:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.690+0000 7f63bcff9640 10 client.15879 handle_command_reply: tid=0 2024-09-10T08:48:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.690+0000 7f63de47c740 1 client.15879 shutdown 2024-09-10T08:48:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.690+0000 7f63de47c740 1 -- 172.21.15.172:0/758996632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a19c370 msgr2=0x55b94a1a8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.690+0000 7f63de47c740 1 --2- 172.21.15.172:0/758996632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a19c370 0x55b94a1a8110 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0049d0 tx=0x7f63cc0a3870 comp rx=0 tx=0).stop 2024-09-10T08:48:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.691+0000 7f63de47c740 1 -- 172.21.15.172:0/758996632 shutdown_connections 2024-09-10T08:48:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.691+0000 7f63de47c740 1 --2- 172.21.15.172:0/758996632 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b94a1ac6a0 0x55b94a1aeaf0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.691+0000 7f63de47c740 1 --2- 172.21.15.172:0/758996632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a19c370 0x55b94a1a8110 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.691+0000 7f63de47c740 1 --2- 172.21.15.172:0/758996632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1972e0 0x55b94a1a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.691+0000 7f63bcff9640 0 client.15879 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.691+0000 7f63de47c740 1 --2- 172.21.15.172:0/758996632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a1969a0 0x55b94a1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.691+0000 7f63de47c740 1 -- 172.21.15.172:0/758996632 >> 172.21.15.172:0/758996632 conn(0x55b94a192010 msgr2=0x55b94a193820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.691+0000 7f63de47c740 1 -- 172.21.15.172:0/758996632 shutdown_connections 2024-09-10T08:48:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.691+0000 7f63de47c740 1 -- 172.21.15.172:0/758996632 wait complete. 2024-09-10T08:48:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.691+0000 7f63de47c740 20 client.15879 trim_cache size 0 max 16384 2024-09-10T08:48:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.719+0000 7f63de47c740 1 Processor -- start 2024-09-10T08:48:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.719+0000 7f63de47c740 1 -- start start 2024-09-10T08:48:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.719+0000 7f63de47c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a1969a0 0x55b94a1909a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.719+0000 7f63de47c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a1972e0 0x55b94a1a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.719+0000 7f63de47c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a19c370 0x55b94a190ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.719+0000 7f63de47c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f63d41a8320 con 0x55b94a1969a0 2024-09-10T08:48:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.719+0000 7f63de47c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f63d41a9e60 con 0x55b94a19c370 2024-09-10T08:48:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.719+0000 7f63de47c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f63d41a89f0 con 0x55b94a1972e0 2024-09-10T08:48:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.719+0000 7f63d9399640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a1969a0 0x55b94a1909a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.719+0000 7f63d9399640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a1969a0 0x55b94a1909a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39716/0 (socket says 172.21.15.172:39716) 2024-09-10T08:48:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.719+0000 7f63d9399640 1 -- 172.21.15.172:0/2434544525 learned_addr learned my addr 172.21.15.172:0/2434544525 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.719+0000 7f63d8b98640 1 --2- 172.21.15.172:0/2434544525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a1972e0 0x55b94a1a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.719+0000 7f63d9b9a640 1 --2- 172.21.15.172:0/2434544525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a19c370 0x55b94a190ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.719+0000 7f63d9399640 1 -- 172.21.15.172:0/2434544525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a1972e0 msgr2=0x55b94a1a80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.720+0000 7f63d9399640 1 --2- 172.21.15.172:0/2434544525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a1972e0 0x55b94a1a80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.720+0000 7f63d9399640 1 -- 172.21.15.172:0/2434544525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a19c370 msgr2=0x55b94a190ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.720+0000 7f63d9399640 1 --2- 172.21.15.172:0/2434544525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a19c370 0x55b94a190ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.720+0000 7f63d9399640 1 -- 172.21.15.172:0/2434544525 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b94a1ac460 con 0x55b94a1969a0 2024-09-10T08:48:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.720+0000 7f63d9b9a640 1 --2- 172.21.15.172:0/2434544525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a19c370 0x55b94a190ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.720+0000 7f63d9399640 1 --2- 172.21.15.172:0/2434544525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a1969a0 0x55b94a1909a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f63d001b040 tx=0x7f63d0004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2366ed5ee738a06d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.720+0000 7f63beffd640 1 -- 172.21.15.172:0/2434544525 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0033070 con 0x55b94a1969a0 2024-09-10T08:48:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.720+0000 7f63beffd640 1 -- 172.21.15.172:0/2434544525 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63d0003aa0 con 0x55b94a1969a0 2024-09-10T08:48:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.720+0000 7f63beffd640 1 -- 172.21.15.172:0/2434544525 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d002e740 con 0x55b94a1969a0 2024-09-10T08:48:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.721+0000 7f63de47c740 1 -- 172.21.15.172:0/2434544525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a1969a0 msgr2=0x55b94a1909a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:03.726 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.721+0000 7f63de47c740 1 --2- 172.21.15.172:0/2434544525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a1969a0 0x55b94a1909a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f63d001b040 tx=0x7f63d0004700 comp rx=0 tx=0).stop 2024-09-10T08:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.721+0000 7f63de47c740 1 -- 172.21.15.172:0/2434544525 shutdown_connections 2024-09-10T08:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.721+0000 7f63de47c740 1 --2- 172.21.15.172:0/2434544525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a19c370 0x55b94a190ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.721+0000 7f63de47c740 1 --2- 172.21.15.172:0/2434544525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a1972e0 0x55b94a1a80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.721+0000 7f63de47c740 1 --2- 172.21.15.172:0/2434544525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a1969a0 0x55b94a1909a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.721+0000 7f63de47c740 1 -- 172.21.15.172:0/2434544525 >> 172.21.15.172:0/2434544525 conn(0x55b94a192010 msgr2=0x55b94a1a0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.721+0000 7f63de47c740 1 -- 172.21.15.172:0/2434544525 shutdown_connections 2024-09-10T08:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.721+0000 7f63de47c740 1 -- 172.21.15.172:0/2434544525 wait complete. 2024-09-10T08:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.721+0000 7f63de47c740 1 -- start start 2024-09-10T08:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.721+0000 7f63de47c740 1 Processor -- start 2024-09-10T08:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.722+0000 7f63de47c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1969a0 0x55b94a1a5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.722+0000 7f63de47c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a1972e0 0x55b94a1a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.722+0000 7f63de47c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a19c370 0x55b94a2153c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.722+0000 7f63d8b98640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a1972e0 0x55b94a1a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.722+0000 7f63d8b98640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a1972e0 0x55b94a1a56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39718/0 (socket says 172.21.15.172:39718) 2024-09-10T08:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.722+0000 7f63d8b98640 1 -- 172.21.15.172:0/3553496830 learned_addr learned my addr 172.21.15.172:0/3553496830 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.722+0000 7f63de47c740 1 -- 172.21.15.172:0/3553496830 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f63d41a9790 con 0x55b94a1972e0 2024-09-10T08:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.722+0000 7f63de47c740 1 -- 172.21.15.172:0/3553496830 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f63d41a90c0 con 0x55b94a1969a0 2024-09-10T08:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.722+0000 7f63de47c740 1 -- 172.21.15.172:0/3553496830 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f63d41a6300 con 0x55b94a19c370 2024-09-10T08:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.722+0000 7f63de47c740 10 client.0 _ll_register_callbacks cb 0x7f63cc005af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.723+0000 7f63d8b98640 1 -- 172.21.15.172:0/3553496830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a19c370 msgr2=0x55b94a2153c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.723+0000 7f63d9399640 1 --2- 172.21.15.172:0/3553496830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1969a0 0x55b94a1a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:03.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.723+0000 7f63d8b98640 1 --2- 172.21.15.172:0/3553496830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a19c370 0x55b94a2153c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.723+0000 7f63d8b98640 1 -- 172.21.15.172:0/3553496830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1969a0 msgr2=0x55b94a1a5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.723+0000 7f63d8b98640 1 --2- 172.21.15.172:0/3553496830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1969a0 0x55b94a1a5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.723+0000 7f63d8b98640 1 -- 172.21.15.172:0/3553496830 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b94a2159d0 con 0x55b94a1972e0 2024-09-10T08:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.723+0000 7f63d9399640 1 --2- 172.21.15.172:0/3553496830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1969a0 0x55b94a1a5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.723+0000 7f63d8b98640 1 --2- 172.21.15.172:0/3553496830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a1972e0 0x55b94a1a56b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f63c4005ab0 tx=0x7f63c402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabfb71ab6211f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.723+0000 7f63bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.723+0000 7f63bcff9640 1 -- 172.21.15.172:0/3553496830 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4035040 con 0x55b94a1972e0 2024-09-10T08:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.723+0000 7f63bcff9640 1 -- 172.21.15.172:0/3553496830 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f63c4043040 con 0x55b94a1972e0 2024-09-10T08:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.723+0000 7f63de47c740 1 -- 172.21.15.172:0/3553496830 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63d0003aa0 con 0x55b94a1972e0 2024-09-10T08:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.723+0000 7f63bcff9640 1 -- 172.21.15.172:0/3553496830 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f63c4064050 con 0x55b94a1972e0 2024-09-10T08:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.723+0000 7f63de47c740 10 client.15885 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:03.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.724+0000 7f63de47c740 1 -- 172.21.15.172:0/3553496830 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b94a216d30 con 0x55b94a1972e0 2024-09-10T08:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.724+0000 7f63bcff9640 1 -- 172.21.15.172:0/3553496830 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f63c4031e40 con 0x55b94a1972e0 2024-09-10T08:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.724+0000 7f63bcff9640 1 -- 172.21.15.172:0/3553496830 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f63c4048ca0 con 0x55b94a1972e0 2024-09-10T08:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.724+0000 7f63de47c740 10 client.15885 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.724+0000 7f63de47c740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.724+0000 7f63de47c740 20 client.15885 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.724+0000 7f63de47c740 1 --2- 172.21.15.172:0/3553496830 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b94a217700 0x55b94a219b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.724+0000 7f63de47c740 4 client.15885 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.724+0000 7f63de47c740 1 -- 172.21.15.172:0/3553496830 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f63d0003aa0 con 0x55b94a217700 2024-09-10T08:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.724+0000 7f63d9399640 1 --2- 172.21.15.172:0/3553496830 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b94a217700 0x55b94a219b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.725+0000 7f63bcff9640 10 client.15885 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:03.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.725+0000 7f63d9399640 1 --2- 172.21.15.172:0/3553496830 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b94a217700 0x55b94a219b20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43bfdeeec48bc888 server_cookie=4418a551cbb9ca81 in_seq=0 out_seq=0 2024-09-10T08:48:03.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.728+0000 7f63bcff9640 1 -- 172.21.15.172:0/3553496830 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f63d0003aa0 con 0x55b94a217700 2024-09-10T08:48:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.728+0000 7f63bcff9640 10 client.15885 handle_command_reply: tid=0 2024-09-10T08:48:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.728+0000 7f63de47c740 1 client.15885 shutdown 2024-09-10T08:48:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.728+0000 7f63de47c740 1 -- 172.21.15.172:0/3553496830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a1972e0 msgr2=0x55b94a1a56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.728+0000 7f63de47c740 1 --2- 172.21.15.172:0/3553496830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a1972e0 0x55b94a1a56b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f63c4005ab0 tx=0x7f63c402a410 comp rx=0 tx=0).stop 2024-09-10T08:48:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.728+0000 7f63de47c740 1 -- 172.21.15.172:0/3553496830 shutdown_connections 2024-09-10T08:48:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.728+0000 7f63de47c740 1 --2- 172.21.15.172:0/3553496830 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b94a217700 0x55b94a219b20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.728+0000 7f63bcff9640 0 client.15885 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.728+0000 7f63de47c740 1 --2- 172.21.15.172:0/3553496830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b94a19c370 0x55b94a2153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.729+0000 7f63de47c740 1 --2- 172.21.15.172:0/3553496830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b94a1972e0 0x55b94a1a56b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.729+0000 7f63de47c740 1 --2- 172.21.15.172:0/3553496830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b94a1969a0 0x55b94a1a5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.729+0000 7f63de47c740 1 -- 172.21.15.172:0/3553496830 >> 172.21.15.172:0/3553496830 conn(0x55b94a192010 msgr2=0x55b94a192ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.729+0000 7f63de47c740 1 -- 172.21.15.172:0/3553496830 shutdown_connections 2024-09-10T08:48:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.729+0000 7f63de47c740 1 -- 172.21.15.172:0/3553496830 wait complete. 2024-09-10T08:48:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.729+0000 7f63de47c740 20 client.15885 trim_cache size 0 max 16384 2024-09-10T08:48:03.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:48:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:48:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:48:03.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:48:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.729+0000 7f63db5fd640 1 -- 172.21.15.172:0/1446054085 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f63c00a2cb0 msgr2=0x7f63c00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.729+0000 7f63db5fd640 1 --2- 172.21.15.172:0/1446054085 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f63c00a2cb0 0x7f63c00a5170 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f63c4000c00 tx=0x7f63c4005a20 comp rx=0 tx=0).stop 2024-09-10T08:48:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.729+0000 7f63db5fd640 1 -- 172.21.15.172:0/1446054085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63d4102040 msgr2=0x7f63d419e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.729+0000 7f63db5fd640 1 --2- 172.21.15.172:0/1446054085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63d4102040 0x7f63d419e170 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00eb10 tx=0x7f63cc00d2e0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.730+0000 7f63db5fd640 1 -- 172.21.15.172:0/1446054085 shutdown_connections 2024-09-10T08:48:03.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.730+0000 7f63db5fd640 1 --2- 172.21.15.172:0/1446054085 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f63c00a2cb0 0x7f63c00a5170 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.730+0000 7f63db5fd640 1 --2- 172.21.15.172:0/1446054085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63d4102040 0x7f63d419e170 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.730+0000 7f63db5fd640 1 --2- 172.21.15.172:0/1446054085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63d41016a0 0x7f63d4197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.730+0000 7f63db5fd640 1 --2- 172.21.15.172:0/1446054085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63d4100ce0 0x7f63d4196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.730+0000 7f63db5fd640 1 -- 172.21.15.172:0/1446054085 >> 172.21.15.172:0/1446054085 conn(0x7f63d40fb940 msgr2=0x7f63d4105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.730+0000 7f63db5fd640 1 -- 172.21.15.172:0/1446054085 shutdown_connections 2024-09-10T08:48:03.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:03.730+0000 7f63db5fd640 1 -- 172.21.15.172:0/1446054085 wait complete. 2024-09-10T08:48:04.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:04 smithi191 ceph-mon[35314]: pgmap v405: 129 pgs: 129 active+clean; 673 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:04.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:04 smithi172 ceph-mon[30854]: pgmap v405: 129 pgs: 129 active+clean; 673 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:04.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:04 smithi174 ceph-mon[34068]: pgmap v405: 129 pgs: 129 active+clean; 673 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:06.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:06 smithi191 ceph-mon[35314]: pgmap v406: 129 pgs: 129 active+clean; 673 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:06 smithi172 ceph-mon[30854]: pgmap v406: 129 pgs: 129 active+clean; 673 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:06.729 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:48:06.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:06 smithi174 ceph-mon[34068]: pgmap v406: 129 pgs: 129 active+clean; 673 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:07.005 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:07.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.559+0000 7fc7563b5640 1 -- 172.21.15.172:0/3163476978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc750102d90 msgr2=0x7fc750103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:07.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.559+0000 7fc7563b5640 1 --2- 172.21.15.172:0/3163476978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc750102d90 0x7fc750103170 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc740009930 tx=0x7fc74002a420 comp rx=0 tx=0).stop 2024-09-10T08:48:07.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.559+0000 7fc7563b5640 1 -- 172.21.15.172:0/3163476978 shutdown_connections 2024-09-10T08:48:07.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.559+0000 7fc7563b5640 1 --2- 172.21.15.172:0/3163476978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc750108290 0x7fc75010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.560+0000 7fc7563b5640 1 --2- 172.21.15.172:0/3163476978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc750103740 0x7fc750107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.560+0000 7fc7563b5640 1 --2- 172.21.15.172:0/3163476978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc750102d90 0x7fc750103170 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.560+0000 7fc7563b5640 1 -- 172.21.15.172:0/3163476978 >> 172.21.15.172:0/3163476978 conn(0x7fc7500fc7d0 msgr2=0x7fc7500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:07.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.560+0000 7fc7563b5640 1 -- 172.21.15.172:0/3163476978 shutdown_connections 2024-09-10T08:48:07.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.560+0000 7fc7563b5640 1 -- 172.21.15.172:0/3163476978 wait complete. 2024-09-10T08:48:07.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.561+0000 7fc7563b5640 1 Processor -- start 2024-09-10T08:48:07.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.561+0000 7fc7563b5640 1 -- start start 2024-09-10T08:48:07.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.562+0000 7fc7563b5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc750102d90 0x7fc7501984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.562+0000 7fc7563b5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc750103740 0x7fc750198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.562+0000 7fc7563b5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc750108290 0x7fc75019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.562+0000 7fc7563b5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc750111f40 con 0x7fc750108290 2024-09-10T08:48:07.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.562+0000 7fc754952640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc750108290 0x7fc75019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.562+0000 7fc754952640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc750108290 0x7fc75019cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38644/0 (socket says 172.21.15.172:38644) 2024-09-10T08:48:07.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.562+0000 7fc754952640 1 -- 172.21.15.172:0/4094395549 learned_addr learned my addr 172.21.15.172:0/4094395549 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:07.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.562+0000 7fc7563b5640 1 -- 172.21.15.172:0/4094395549 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc750111dc0 con 0x7fc750102d90 2024-09-10T08:48:07.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.562+0000 7fc74f7fe640 1 --2- 172.21.15.172:0/4094395549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc750103740 0x7fc750198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.562+0000 7fc7563b5640 1 -- 172.21.15.172:0/4094395549 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501120c0 con 0x7fc750103740 2024-09-10T08:48:07.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.562+0000 7fc74ffff640 1 --2- 172.21.15.172:0/4094395549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc750102d90 0x7fc7501984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.563+0000 7fc754952640 1 -- 172.21.15.172:0/4094395549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc750103740 msgr2=0x7fc750198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:07.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.563+0000 7fc754952640 1 --2- 172.21.15.172:0/4094395549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc750103740 0x7fc750198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.563+0000 7fc754952640 1 -- 172.21.15.172:0/4094395549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc750102d90 msgr2=0x7fc7501984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:07.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.563+0000 7fc754952640 1 --2- 172.21.15.172:0/4094395549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc750102d90 0x7fc7501984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.563+0000 7fc754952640 1 -- 172.21.15.172:0/4094395549 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75019d470 con 0x7fc750108290 2024-09-10T08:48:07.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.563+0000 7fc754952640 1 --2- 172.21.15.172:0/4094395549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc750108290 0x7fc75019cd90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc74800cce0 tx=0x7fc74800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:07.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.563+0000 7fc74d7fa640 1 -- 172.21.15.172:0/4094395549 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74800f450 con 0x7fc750108290 2024-09-10T08:48:07.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.563+0000 7fc74d7fa640 1 -- 172.21.15.172:0/4094395549 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc74800faa0 con 0x7fc750108290 2024-09-10T08:48:07.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.564+0000 7fc74d7fa640 1 -- 172.21.15.172:0/4094395549 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748018b70 con 0x7fc750108290 2024-09-10T08:48:07.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.564+0000 7fc7563b5640 1 -- 172.21.15.172:0/4094395549 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75019d760 con 0x7fc750108290 2024-09-10T08:48:07.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.565+0000 7fc7563b5640 1 -- 172.21.15.172:0/4094395549 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75019dbc0 con 0x7fc750108290 2024-09-10T08:48:07.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.568+0000 7fc74d7fa640 1 -- 172.21.15.172:0/4094395549 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc74800fc50 con 0x7fc750108290 2024-09-10T08:48:07.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.569+0000 7fc75878e740 1 Processor -- start 2024-09-10T08:48:07.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.569+0000 7fc75878e740 1 -- start start 2024-09-10T08:48:07.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.569+0000 7fc75878e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97340 0x55f7eab97720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.569+0000 7fc75878e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97c60 0x55f7eaba6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.569+0000 7fc75878e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eaba6a90 0x55f7eaba8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.569+0000 7fc75878e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f7eaafd2e0 con 0x55f7eab97c60 2024-09-10T08:48:07.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.569+0000 7fc75878e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f7eaaa4ce0 con 0x55f7eaba6a90 2024-09-10T08:48:07.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.569+0000 7fc75878e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f7eaaf92a0 con 0x55f7eab97340 2024-09-10T08:48:07.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.569+0000 7fc74d7fa640 1 --2- 172.21.15.172:0/4094395549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc7380a2b50 0x7fc7380a5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.570+0000 7fc754952640 1 --2- 172.21.15.172:0/4094395549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc7380a2b50 0x7fc7380a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.570+0000 7fc74f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97340 0x55f7eab97720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.570+0000 7fc74f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eaba6a90 0x55f7eaba8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.570+0000 7fc74f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eaba6a90 0x55f7eaba8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:47118/0 (socket says 172.21.15.172:47118) 2024-09-10T08:48:07.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.570+0000 7fc74f7fe640 1 -- 172.21.15.172:0/1868991758 learned_addr learned my addr 172.21.15.172:0/1868991758 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:07.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.570+0000 7fc74ffff640 1 --2- 172.21.15.172:0/1868991758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97c60 0x55f7eaba6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.570+0000 7fc74f7fe640 1 -- 172.21.15.172:0/1868991758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97340 msgr2=0x55f7eab97720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:07.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.570+0000 7fc74f7fe640 1 --2- 172.21.15.172:0/1868991758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97340 0x55f7eab97720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.570+0000 7fc74f7fe640 1 -- 172.21.15.172:0/1868991758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97c60 msgr2=0x55f7eaba6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:07.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.570+0000 7fc74f7fe640 1 --2- 172.21.15.172:0/1868991758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97c60 0x55f7eaba6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.570+0000 7fc74f7fe640 1 -- 172.21.15.172:0/1868991758 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7eaa89440 con 0x55f7eaba6a90 2024-09-10T08:48:07.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.570+0000 7fc74ffff640 1 --2- 172.21.15.172:0/1868991758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97c60 0x55f7eaba6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:07.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.571+0000 7fc74d7fa640 1 -- 172.21.15.172:0/4094395549 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fc7480cfa20 con 0x7fc750108290 2024-09-10T08:48:07.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.571+0000 7fc754952640 1 --2- 172.21.15.172:0/4094395549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc7380a2b50 0x7fc7380a5010 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc748098320 tx=0x7fc748098250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:07.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.571+0000 7fc74f7fe640 1 --2- 172.21.15.172:0/1868991758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eaba6a90 0x55f7eaba8e80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc744014960 tx=0x7fc744004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c4da7638b1afe01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:07.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.571+0000 7fc7367fc640 1 -- 172.21.15.172:0/1868991758 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744005220 con 0x55f7eaba6a90 2024-09-10T08:48:07.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.571+0000 7fc7367fc640 1 -- 172.21.15.172:0/1868991758 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc744004c50 con 0x55f7eaba6a90 2024-09-10T08:48:07.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.571+0000 7fc7367fc640 1 -- 172.21.15.172:0/1868991758 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744011a20 con 0x55f7eaba6a90 2024-09-10T08:48:07.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.572+0000 7fc75878e740 1 -- 172.21.15.172:0/1868991758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eaba6a90 msgr2=0x55f7eaba8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:07.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.572+0000 7fc75878e740 1 --2- 172.21.15.172:0/1868991758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eaba6a90 0x55f7eaba8e80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc744014960 tx=0x7fc744004730 comp rx=0 tx=0).stop 2024-09-10T08:48:07.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.572+0000 7fc75878e740 1 -- 172.21.15.172:0/1868991758 shutdown_connections 2024-09-10T08:48:07.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.572+0000 7fc75878e740 1 --2- 172.21.15.172:0/1868991758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eaba6a90 0x55f7eaba8e80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.572+0000 7fc75878e740 1 --2- 172.21.15.172:0/1868991758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97c60 0x55f7eaba6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.572+0000 7fc75878e740 1 --2- 172.21.15.172:0/1868991758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97340 0x55f7eab97720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.572+0000 7fc75878e740 1 -- 172.21.15.172:0/1868991758 >> 172.21.15.172:0/1868991758 conn(0x55f7eab93a30 msgr2=0x55f7eab93e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:07.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.572+0000 7fc75878e740 1 -- 172.21.15.172:0/1868991758 shutdown_connections 2024-09-10T08:48:07.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.572+0000 7fc75878e740 1 -- 172.21.15.172:0/1868991758 wait complete. 2024-09-10T08:48:07.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.573+0000 7fc75878e740 1 -- start start 2024-09-10T08:48:07.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.573+0000 7fc75878e740 1 Processor -- start 2024-09-10T08:48:07.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.573+0000 7fc75878e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97340 0x55f7eaba6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.573+0000 7fc75878e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eab97c60 0x55f7eaba3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.573+0000 7fc75878e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eaba6a90 0x55f7eaba3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.574+0000 7fc74f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97340 0x55f7eaba6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.574+0000 7fc74f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97340 0x55f7eaba6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:34052/0 (socket says 172.21.15.172:34052) 2024-09-10T08:48:07.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.574+0000 7fc74f7fe640 1 -- 172.21.15.172:0/2801442033 learned_addr learned my addr 172.21.15.172:0/2801442033 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:07.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.574+0000 7fc75878e740 1 -- 172.21.15.172:0/2801442033 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f7eaba9660 con 0x55f7eaba6a90 2024-09-10T08:48:07.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.574+0000 7fc75878e740 1 -- 172.21.15.172:0/2801442033 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f7eaab8050 con 0x55f7eab97c60 2024-09-10T08:48:07.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.574+0000 7fc74ffff640 1 --2- 172.21.15.172:0/2801442033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eab97c60 0x55f7eaba3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.574+0000 7fc75878e740 1 -- 172.21.15.172:0/2801442033 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f7eaba97e0 con 0x55f7eab97340 2024-09-10T08:48:07.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.574+0000 7fc74f7fe640 1 --2- 172.21.15.172:0/2801442033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eaba6a90 0x55f7eaba3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.574+0000 7fc74ffff640 1 -- 172.21.15.172:0/2801442033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97340 msgr2=0x55f7eaba6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:07.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.574+0000 7fc74ffff640 1 --2- 172.21.15.172:0/2801442033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97340 0x55f7eaba6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.574+0000 7fc74ffff640 1 -- 172.21.15.172:0/2801442033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eaba6a90 msgr2=0x55f7eaba3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:07.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.574+0000 7fc74ffff640 1 --2- 172.21.15.172:0/2801442033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eaba6a90 0x55f7eaba3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.574+0000 7fc74ffff640 1 -- 172.21.15.172:0/2801442033 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7ea86e3c0 con 0x55f7eab97c60 2024-09-10T08:48:07.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.574+0000 7fc74f7fe640 1 --2- 172.21.15.172:0/2801442033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eaba6a90 0x55f7eaba3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:07.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.575+0000 7fc75878e740 10 client.0 _ll_register_callbacks cb 0x55f7eaaf8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:07.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.575+0000 7fc74ffff640 1 --2- 172.21.15.172:0/2801442033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eab97c60 0x55f7eaba3720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc740031440 tx=0x7fc7400316d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=674e4ebceeef2bd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:07.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.575+0000 7fc75878e740 1 -- 172.21.15.172:0/2801442033 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc744004c50 con 0x55f7eab97c60 2024-09-10T08:48:07.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.575+0000 7fc72f7fe640 10 client.25319 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:48:07.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.575+0000 7fc72f7fe640 1 -- 172.21.15.172:0/2801442033 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc740035040 con 0x55f7eab97c60 2024-09-10T08:48:07.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.575+0000 7fc72f7fe640 1 -- 172.21.15.172:0/2801442033 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fc740004450 con 0x55f7eab97c60 2024-09-10T08:48:07.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.575+0000 7fc72f7fe640 1 -- 172.21.15.172:0/2801442033 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc7400318c0 con 0x55f7eab97c60 2024-09-10T08:48:07.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.575+0000 7fc75878e740 10 client.25319 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:07.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.575+0000 7fc75878e740 1 -- 172.21.15.172:0/2801442033 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55f7eabb4f80 con 0x55f7eab97c60 2024-09-10T08:48:07.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.576+0000 7fc72f7fe640 1 -- 172.21.15.172:0/2801442033 <== mon.2 v2:172.21.15.174:3300/0 4 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc740005020 con 0x55f7eab97c60 2024-09-10T08:48:07.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.576+0000 7fc75878e740 10 client.25319 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:07.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.576+0000 7fc75878e740 10 client.25319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:07.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.576+0000 7fc75878e740 20 client.25319 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:07.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.576+0000 7fc75878e740 1 --2- 172.21.15.172:0/2801442033 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f7eabb6330 0x55f7eabb87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.576+0000 7fc75878e740 4 client.25319 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:07.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.576+0000 7fc75878e740 1 -- 172.21.15.172:0/2801442033 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7eab0a850 con 0x55f7eabb6330 2024-09-10T08:48:07.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.576+0000 7fc74f7fe640 1 --2- 172.21.15.172:0/2801442033 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f7eabb6330 0x55f7eabb87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.577+0000 7fc72f7fe640 10 client.25319 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:07.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.577+0000 7fc74f7fe640 1 --2- 172.21.15.172:0/2801442033 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f7eabb6330 0x55f7eabb87a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7144b77be7c33a46 server_cookie=ba1cf0f19bac480d in_seq=0 out_seq=0 2024-09-10T08:48:07.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.578+0000 7fc72f7fe640 1 -- 172.21.15.172:0/2801442033 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7eab0a850 con 0x55f7eabb6330 2024-09-10T08:48:07.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.578+0000 7fc72f7fe640 10 client.25319 handle_command_reply: tid=0 2024-09-10T08:48:07.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.578+0000 7fc75878e740 1 client.25319 shutdown 2024-09-10T08:48:07.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.578+0000 7fc75878e740 1 -- 172.21.15.172:0/2801442033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eab97c60 msgr2=0x55f7eaba3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:07.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.578+0000 7fc75878e740 1 --2- 172.21.15.172:0/2801442033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eab97c60 0x55f7eaba3720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc740031440 tx=0x7fc7400316d0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.578+0000 7fc75878e740 1 -- 172.21.15.172:0/2801442033 shutdown_connections 2024-09-10T08:48:07.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.578+0000 7fc75878e740 1 --2- 172.21.15.172:0/2801442033 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f7eabb6330 0x55f7eabb87a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.578+0000 7fc75878e740 1 --2- 172.21.15.172:0/2801442033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eaba6a90 0x55f7eaba3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.578+0000 7fc75878e740 1 --2- 172.21.15.172:0/2801442033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eab97c60 0x55f7eaba3720 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.578+0000 7fc75878e740 1 --2- 172.21.15.172:0/2801442033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97340 0x55f7eaba6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.578+0000 7fc72f7fe640 0 client.25319 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:07.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.578+0000 7fc75878e740 1 -- 172.21.15.172:0/2801442033 >> 172.21.15.172:0/2801442033 conn(0x55f7eab93a30 msgr2=0x55f7eab99790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:07.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.578+0000 7fc75878e740 1 -- 172.21.15.172:0/2801442033 shutdown_connections 2024-09-10T08:48:07.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.578+0000 7fc75878e740 1 -- 172.21.15.172:0/2801442033 wait complete. 2024-09-10T08:48:07.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.579+0000 7fc75878e740 20 client.25319 trim_cache size 0 max 16384 2024-09-10T08:48:07.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.612+0000 7fc75878e740 1 Processor -- start 2024-09-10T08:48:07.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.612+0000 7fc75878e740 1 -- start start 2024-09-10T08:48:07.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.612+0000 7fc75878e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97340 0x55f7eabb2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.612+0000 7fc75878e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97c60 0x55f7eabbe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.612+0000 7fc75878e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eaba6a90 0x55f7eabbe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.612+0000 7fc75878e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501ac680 con 0x55f7eab97c60 2024-09-10T08:48:07.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.612+0000 7fc75878e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501ae1c0 con 0x55f7eaba6a90 2024-09-10T08:48:07.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.612+0000 7fc75878e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501acd50 con 0x55f7eab97340 2024-09-10T08:48:07.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.613+0000 7fc74f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97340 0x55f7eabb2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.613+0000 7fc74f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eaba6a90 0x55f7eabbe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.613+0000 7fc74f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97340 0x55f7eabb2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:34066/0 (socket says 172.21.15.172:34066) 2024-09-10T08:48:07.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.613+0000 7fc74f7fe640 1 -- 172.21.15.172:0/1479311163 learned_addr learned my addr 172.21.15.172:0/1479311163 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:07.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.613+0000 7fc74ffff640 1 --2- 172.21.15.172:0/1479311163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97c60 0x55f7eabbe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.613+0000 7fc74f7fe640 1 -- 172.21.15.172:0/1479311163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97340 msgr2=0x55f7eabb2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:07.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.613+0000 7fc74f7fe640 1 --2- 172.21.15.172:0/1479311163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97340 0x55f7eabb2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.613+0000 7fc74f7fe640 1 -- 172.21.15.172:0/1479311163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97c60 msgr2=0x55f7eabbe070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.613+0000 7fc74f7fe640 1 --2- 172.21.15.172:0/1479311163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97c60 0x55f7eabbe070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.613+0000 7fc74f7fe640 1 -- 172.21.15.172:0/1479311163 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7eaba1c70 con 0x55f7eaba6a90 2024-09-10T08:48:07.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.613+0000 7fc74f7fe640 1 --2- 172.21.15.172:0/1479311163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eaba6a90 0x55f7eabbe5b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55f7eaba4350 tx=0x7fc744017040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f48f5c2b0d9e726c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:07.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.613+0000 7fc735ffb640 1 -- 172.21.15.172:0/1479311163 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744033070 con 0x55f7eaba6a90 2024-09-10T08:48:07.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.614+0000 7fc735ffb640 1 -- 172.21.15.172:0/1479311163 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7440145d0 con 0x55f7eaba6a90 2024-09-10T08:48:07.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.614+0000 7fc735ffb640 1 -- 172.21.15.172:0/1479311163 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744005170 con 0x55f7eaba6a90 2024-09-10T08:48:07.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.615+0000 7fc75878e740 1 -- 172.21.15.172:0/1479311163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eaba6a90 msgr2=0x55f7eabbe5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:07.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.615+0000 7fc75878e740 1 --2- 172.21.15.172:0/1479311163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eaba6a90 0x55f7eabbe5b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55f7eaba4350 tx=0x7fc744017040 comp rx=0 tx=0).stop 2024-09-10T08:48:07.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.615+0000 7fc75878e740 1 -- 172.21.15.172:0/1479311163 shutdown_connections 2024-09-10T08:48:07.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.615+0000 7fc75878e740 1 --2- 172.21.15.172:0/1479311163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eaba6a90 0x55f7eabbe5b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.615+0000 7fc75878e740 1 --2- 172.21.15.172:0/1479311163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97c60 0x55f7eabbe070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.615+0000 7fc75878e740 1 --2- 172.21.15.172:0/1479311163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97340 0x55f7eabb2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.615+0000 7fc75878e740 1 -- 172.21.15.172:0/1479311163 >> 172.21.15.172:0/1479311163 conn(0x55f7eab93a30 msgr2=0x55f7eab9ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:07.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.615+0000 7fc75878e740 1 -- 172.21.15.172:0/1479311163 shutdown_connections 2024-09-10T08:48:07.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.615+0000 7fc75878e740 1 -- 172.21.15.172:0/1479311163 wait complete. 2024-09-10T08:48:07.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.615+0000 7fc75878e740 1 -- start start 2024-09-10T08:48:07.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.616+0000 7fc75878e740 1 Processor -- start 2024-09-10T08:48:07.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.616+0000 7fc75878e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97340 0x55f7eabaf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.616+0000 7fc74f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97340 0x55f7eabaf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.617+0000 7fc74f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97340 0x55f7eabaf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38674/0 (socket says 172.21.15.172:38674) 2024-09-10T08:48:07.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.617+0000 7fc74f7fe640 1 -- 172.21.15.172:0/932252022 learned_addr learned my addr 172.21.15.172:0/932252022 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:07.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.617+0000 7fc75878e740 1 --2- 172.21.15.172:0/932252022 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97c60 0x55f7eabaa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.617+0000 7fc75878e740 1 --2- 172.21.15.172:0/932252022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eaba6a90 0x55f7eabaab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.617+0000 7fc75878e740 1 -- 172.21.15.172:0/932252022 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501adaf0 con 0x55f7eab97340 2024-09-10T08:48:07.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.617+0000 7fc75878e740 1 -- 172.21.15.172:0/932252022 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501ad420 con 0x55f7eaba6a90 2024-09-10T08:48:07.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.617+0000 7fc75878e740 1 -- 172.21.15.172:0/932252022 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501aa660 con 0x55f7eab97c60 2024-09-10T08:48:07.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.617+0000 7fc74ffff640 1 --2- 172.21.15.172:0/932252022 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97c60 0x55f7eabaa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.617+0000 7fc74f7fe640 1 -- 172.21.15.172:0/932252022 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97c60 msgr2=0x55f7eabaa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.617+0000 7fc74f7fe640 1 --2- 172.21.15.172:0/932252022 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97c60 0x55f7eabaa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.617+0000 7fc74f7fe640 1 -- 172.21.15.172:0/932252022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eaba6a90 msgr2=0x55f7eabaab60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:48:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.617+0000 7fc74f7fe640 1 --2- 172.21.15.172:0/932252022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eaba6a90 0x55f7eabaab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.617+0000 7fc74f7fe640 1 -- 172.21.15.172:0/932252022 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7eabab390 con 0x55f7eab97340 2024-09-10T08:48:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.618+0000 7fc75878e740 10 client.0 _ll_register_callbacks cb 0x7fc740005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.618+0000 7fc75878e740 1 -- 172.21.15.172:0/932252022 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7440145d0 con 0x55f7eab97340 2024-09-10T08:48:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.618+0000 7fc74f7fe640 1 --2- 172.21.15.172:0/932252022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97340 0x55f7eabaf290 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc744004690 tx=0x7fc74400af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e39a2029aa90ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.618+0000 7fc72f7fe640 10 client.15900 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.618+0000 7fc72f7fe640 1 -- 172.21.15.172:0/932252022 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74403b020 con 0x55f7eab97340 2024-09-10T08:48:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.618+0000 7fc72f7fe640 1 -- 172.21.15.172:0/932252022 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fc744004ff0 con 0x55f7eab97340 2024-09-10T08:48:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.618+0000 7fc72f7fe640 1 -- 172.21.15.172:0/932252022 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc744033040 con 0x55f7eab97340 2024-09-10T08:48:07.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.618+0000 7fc75878e740 10 client.15900 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:07.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.618+0000 7fc75878e740 1 -- 172.21.15.172:0/932252022 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55f7eac20e10 con 0x55f7eab97340 2024-09-10T08:48:07.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.619+0000 7fc72f7fe640 1 -- 172.21.15.172:0/932252022 <== mon.0 v2:172.21.15.172:3300/0 4 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc74402c070 con 0x55f7eab97340 2024-09-10T08:48:07.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.619+0000 7fc75878e740 10 client.15900 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:07.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.619+0000 7fc75878e740 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:07.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.619+0000 7fc75878e740 20 client.15900 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:07.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.619+0000 7fc75878e740 1 --2- 172.21.15.172:0/932252022 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f7eac211f0 0x55f7eac235e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:07.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.619+0000 7fc75878e740 4 client.15900 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:48:07.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.619+0000 7fc75878e740 1 -- 172.21.15.172:0/932252022 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f7eab04d60 con 0x55f7eac211f0 2024-09-10T08:48:07.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.619+0000 7fc74ffff640 1 --2- 172.21.15.172:0/932252022 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f7eac211f0 0x55f7eac235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:07.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.620+0000 7fc72f7fe640 10 client.15900 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:07.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.620+0000 7fc74ffff640 1 --2- 172.21.15.172:0/932252022 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f7eac211f0 0x55f7eac235e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4558421235539cee server_cookie=c0e84dec2309a095 in_seq=0 out_seq=0 2024-09-10T08:48:07.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.620+0000 7fc72f7fe640 1 -- 172.21.15.172:0/932252022 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f7eab04d60 con 0x55f7eac211f0 2024-09-10T08:48:07.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.620+0000 7fc72f7fe640 10 client.15900 handle_command_reply: tid=0 2024-09-10T08:48:07.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.620+0000 7fc75878e740 1 client.15900 shutdown 2024-09-10T08:48:07.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.620+0000 7fc75878e740 1 -- 172.21.15.172:0/932252022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97340 msgr2=0x55f7eabaf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:07.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.620+0000 7fc75878e740 1 --2- 172.21.15.172:0/932252022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97340 0x55f7eabaf290 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc744004690 tx=0x7fc74400af70 comp rx=0 tx=0).stop 2024-09-10T08:48:07.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.621+0000 7fc75878e740 1 -- 172.21.15.172:0/932252022 shutdown_connections 2024-09-10T08:48:07.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.621+0000 7fc75878e740 1 --2- 172.21.15.172:0/932252022 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f7eac211f0 0x55f7eac235e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.621+0000 7fc75878e740 1 --2- 172.21.15.172:0/932252022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7eaba6a90 0x55f7eabaab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.621+0000 7fc75878e740 1 --2- 172.21.15.172:0/932252022 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7eab97c60 0x55f7eabaa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.621+0000 7fc75878e740 1 --2- 172.21.15.172:0/932252022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f7eab97340 0x55f7eabaf290 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.621+0000 7fc75878e740 1 -- 172.21.15.172:0/932252022 >> 172.21.15.172:0/932252022 conn(0x55f7eab93a30 msgr2=0x55f7eaba4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:07.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.621+0000 7fc72f7fe640 0 client.15900 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:07.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.621+0000 7fc75878e740 1 -- 172.21.15.172:0/932252022 shutdown_connections 2024-09-10T08:48:07.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.621+0000 7fc75878e740 1 -- 172.21.15.172:0/932252022 wait complete. 2024-09-10T08:48:07.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.621+0000 7fc75878e740 20 client.15900 trim_cache size 0 max 16384 2024-09-10T08:48:07.625 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:48:07.625 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T08:48:07.625 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T08:48:07.625 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:48:07.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.622+0000 7fc7563b5640 1 -- 172.21.15.172:0/4094395549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc7380a2b50 msgr2=0x7fc7380a5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:07.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.622+0000 7fc7563b5640 1 --2- 172.21.15.172:0/4094395549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc7380a2b50 0x7fc7380a5010 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc748098320 tx=0x7fc748098250 comp rx=0 tx=0).stop 2024-09-10T08:48:07.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.622+0000 7fc7563b5640 1 -- 172.21.15.172:0/4094395549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc750108290 msgr2=0x7fc75019cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:07.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.622+0000 7fc7563b5640 1 --2- 172.21.15.172:0/4094395549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc750108290 0x7fc75019cd90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc74800cce0 tx=0x7fc74800a620 comp rx=0 tx=0).stop 2024-09-10T08:48:07.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.622+0000 7fc7563b5640 1 -- 172.21.15.172:0/4094395549 shutdown_connections 2024-09-10T08:48:07.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.622+0000 7fc7563b5640 1 --2- 172.21.15.172:0/4094395549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc7380a2b50 0x7fc7380a5010 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.622+0000 7fc7563b5640 1 --2- 172.21.15.172:0/4094395549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc750108290 0x7fc75019cd90 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.623+0000 7fc7563b5640 1 --2- 172.21.15.172:0/4094395549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc750103740 0x7fc750198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.623+0000 7fc7563b5640 1 --2- 172.21.15.172:0/4094395549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc750102d90 0x7fc7501984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:07.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.623+0000 7fc7563b5640 1 -- 172.21.15.172:0/4094395549 >> 172.21.15.172:0/4094395549 conn(0x7fc7500fc7d0 msgr2=0x7fc750105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:07.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.623+0000 7fc7563b5640 1 -- 172.21.15.172:0/4094395549 shutdown_connections 2024-09-10T08:48:07.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:07.623+0000 7fc7563b5640 1 -- 172.21.15.172:0/4094395549 wait complete. 2024-09-10T08:48:08.364 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:48:08.364 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:48:08.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:08 smithi191 ceph-mon[35314]: pgmap v407: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-10T08:48:08.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:08 smithi172 ceph-mon[30854]: pgmap v407: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-10T08:48:08.651 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:08.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:08 smithi174 ceph-mon[34068]: pgmap v407: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-10T08:48:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.162+0000 7f8685803640 1 -- 172.21.15.172:0/4117459676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86801008f0 msgr2=0x7f8680100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.162+0000 7f8685803640 1 --2- 172.21.15.172:0/4117459676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86801008f0 0x7f8680100d50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8674009930 tx=0x7f867402a420 comp rx=0 tx=0).stop 2024-09-10T08:48:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.163+0000 7f8685803640 1 -- 172.21.15.172:0/4117459676 shutdown_connections 2024-09-10T08:48:09.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.163+0000 7f8685803640 1 --2- 172.21.15.172:0/4117459676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8680101290 0x7f868010d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.163+0000 7f8685803640 1 --2- 172.21.15.172:0/4117459676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86801008f0 0x7f8680100d50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.163+0000 7f8685803640 1 --2- 172.21.15.172:0/4117459676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86801068f0 0x7f8680106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.163+0000 7f8685803640 1 -- 172.21.15.172:0/4117459676 >> 172.21.15.172:0/4117459676 conn(0x7f86800fc630 msgr2=0x7f86800fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.163+0000 7f8685803640 1 -- 172.21.15.172:0/4117459676 shutdown_connections 2024-09-10T08:48:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.163+0000 7f8685803640 1 -- 172.21.15.172:0/4117459676 wait complete. 2024-09-10T08:48:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.163+0000 7f8685803640 1 Processor -- start 2024-09-10T08:48:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.163+0000 7f8685803640 1 -- start start 2024-09-10T08:48:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.164+0000 7f8685803640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8680101290 0x7f868019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.164+0000 7f8685803640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86801068f0 0x7f868019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.164+0000 7f8685803640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f868019eee0 0x7f868019f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.164+0000 7f8685803640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8680110ba0 con 0x7f86801068f0 2024-09-10T08:48:09.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.164+0000 7f8685803640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8680110a20 con 0x7f868019eee0 2024-09-10T08:48:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.164+0000 7f8685803640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8680110d20 con 0x7f8680101290 2024-09-10T08:48:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.164+0000 7f867e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86801068f0 0x7f868019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.164+0000 7f867e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86801068f0 0x7f868019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38692/0 (socket says 172.21.15.172:38692) 2024-09-10T08:48:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.164+0000 7f867e7fc640 1 -- 172.21.15.172:0/3956139805 learned_addr learned my addr 172.21.15.172:0/3956139805 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.164+0000 7f867f7fe640 1 --2- 172.21.15.172:0/3956139805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f868019eee0 0x7f868019f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.164+0000 7f867effd640 1 --2- 172.21.15.172:0/3956139805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8680101290 0x7f868019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.164+0000 7f867e7fc640 1 -- 172.21.15.172:0/3956139805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8680101290 msgr2=0x7f868019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.164+0000 7f867e7fc640 1 --2- 172.21.15.172:0/3956139805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8680101290 0x7f868019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.164+0000 7f867e7fc640 1 -- 172.21.15.172:0/3956139805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f868019eee0 msgr2=0x7f868019f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.164+0000 7f867e7fc640 1 --2- 172.21.15.172:0/3956139805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f868019eee0 0x7f868019f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:09.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.164+0000 7f867e7fc640 1 -- 172.21.15.172:0/3956139805 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868019f9a0 con 0x7f86801068f0 2024-09-10T08:48:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.165+0000 7f867e7fc640 1 --2- 172.21.15.172:0/3956139805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86801068f0 0x7f868019ab50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8674009230 tx=0x7f86740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.165+0000 7f8684801640 1 -- 172.21.15.172:0/3956139805 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867403f030 con 0x7f86801068f0 2024-09-10T08:48:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.165+0000 7f8684801640 1 -- 172.21.15.172:0/3956139805 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8674003cb0 con 0x7f86801068f0 2024-09-10T08:48:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.165+0000 7f8684801640 1 -- 172.21.15.172:0/3956139805 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674035070 con 0x7f86801068f0 2024-09-10T08:48:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.165+0000 7f8685803640 1 -- 172.21.15.172:0/3956139805 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868019fc30 con 0x7f86801068f0 2024-09-10T08:48:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.165+0000 7f8685803640 1 -- 172.21.15.172:0/3956139805 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86801a7360 con 0x7f86801068f0 2024-09-10T08:48:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.167+0000 7f8685803640 1 -- 172.21.15.172:0/3956139805 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8680100d50 con 0x7f86801068f0 2024-09-10T08:48:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.168+0000 7f8684801640 1 -- 172.21.15.172:0/3956139805 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8674042030 con 0x7f86801068f0 2024-09-10T08:48:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.168+0000 7f8684801640 1 --2- 172.21.15.172:0/3956139805 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86680a2c80 0x7f86680a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.168+0000 7f8684801640 1 -- 172.21.15.172:0/3956139805 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f86740ef670 con 0x7f86801068f0 2024-09-10T08:48:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.169+0000 7f867effd640 1 --2- 172.21.15.172:0/3956139805 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86680a2c80 0x7f86680a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.169+0000 7f867effd640 1 --2- 172.21.15.172:0/3956139805 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86680a2c80 0x7f86680a5140 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f867000bbb0 tx=0x7f867000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:09.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.173+0000 7f8684801640 1 -- 172.21.15.172:0/3956139805 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86740bbbc0 con 0x7f86801068f0 2024-09-10T08:48:09.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.386+0000 7f8685803640 1 -- 172.21.15.172:0/3956139805 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86801a7750 con 0x7f86801068f0 2024-09-10T08:48:09.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.387+0000 7f8684801640 1 -- 172.21.15.172:0/3956139805 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7f86740bfec0 con 0x7f86801068f0 2024-09-10T08:48:09.392 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:48:09.392 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":29,"btime":"2024-09-10T08:46:37:242916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:46:36.267985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:48:09.393 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 29 2024-09-10T08:48:09.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.389+0000 7f8685803640 1 -- 172.21.15.172:0/3956139805 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86680a2c80 msgr2=0x7f86680a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:09.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.389+0000 7f8685803640 1 --2- 172.21.15.172:0/3956139805 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86680a2c80 0x7f86680a5140 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f867000bbb0 tx=0x7f867000b390 comp rx=0 tx=0).stop 2024-09-10T08:48:09.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.389+0000 7f8685803640 1 -- 172.21.15.172:0/3956139805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86801068f0 msgr2=0x7f868019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:09.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.389+0000 7f8685803640 1 --2- 172.21.15.172:0/3956139805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86801068f0 0x7f868019ab50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8674009230 tx=0x7f86740042f0 comp rx=0 tx=0).stop 2024-09-10T08:48:09.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.389+0000 7f8685803640 1 -- 172.21.15.172:0/3956139805 shutdown_connections 2024-09-10T08:48:09.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.389+0000 7f8685803640 1 --2- 172.21.15.172:0/3956139805 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86680a2c80 0x7f86680a5140 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:09.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.389+0000 7f8685803640 1 --2- 172.21.15.172:0/3956139805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f868019eee0 0x7f868019f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:09.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.389+0000 7f8685803640 1 --2- 172.21.15.172:0/3956139805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86801068f0 0x7f868019ab50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:09.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.389+0000 7f8685803640 1 --2- 172.21.15.172:0/3956139805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8680101290 0x7f868019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:09.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.389+0000 7f8685803640 1 -- 172.21.15.172:0/3956139805 >> 172.21.15.172:0/3956139805 conn(0x7f86800fc630 msgr2=0x7f868010c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:09.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.390+0000 7f8685803640 1 -- 172.21.15.172:0/3956139805 shutdown_connections 2024-09-10T08:48:09.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:09.390+0000 7f8685803640 1 -- 172.21.15.172:0/3956139805 wait complete. 2024-09-10T08:48:10.091 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T08:48:10.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:10 smithi172 ceph-mon[30854]: pgmap v408: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:10.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:10 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3956139805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:48:10.364 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:10.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:10 smithi191 ceph-mon[35314]: pgmap v408: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:10.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:10 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3956139805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:48:10.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:10 smithi174 ceph-mon[34068]: pgmap v408: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:10.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:10 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3956139805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:48:10.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.882+0000 7fa362993640 1 -- 172.21.15.172:0/1278474397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa35c102d30 msgr2=0x7fa35c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:10.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.882+0000 7fa362993640 1 --2- 172.21.15.172:0/1278474397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa35c102d30 0x7fa35c103110 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009930 tx=0x7fa34c02a440 comp rx=0 tx=0).stop 2024-09-10T08:48:10.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.882+0000 7fa362993640 1 -- 172.21.15.172:0/1278474397 shutdown_connections 2024-09-10T08:48:10.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.882+0000 7fa362993640 1 --2- 172.21.15.172:0/1278474397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35c1082a0 0x7fa35c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.882+0000 7fa362993640 1 --2- 172.21.15.172:0/1278474397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa35c1036e0 0x7fa35c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.882+0000 7fa362993640 1 --2- 172.21.15.172:0/1278474397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa35c102d30 0x7fa35c103110 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.882+0000 7fa362993640 1 -- 172.21.15.172:0/1278474397 >> 172.21.15.172:0/1278474397 conn(0x7fa35c0fc7d0 msgr2=0x7fa35c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:10.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.882+0000 7fa362993640 1 -- 172.21.15.172:0/1278474397 shutdown_connections 2024-09-10T08:48:10.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.882+0000 7fa362993640 1 -- 172.21.15.172:0/1278474397 wait complete. 2024-09-10T08:48:10.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.883+0000 7fa362993640 1 Processor -- start 2024-09-10T08:48:10.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.883+0000 7fa362993640 1 -- start start 2024-09-10T08:48:10.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.883+0000 7fa362993640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa35c102d30 0x7fa35c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.883+0000 7fa362993640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa35c1036e0 0x7fa35c198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.883+0000 7fa362993640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35c1082a0 0x7fa35c19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.883+0000 7fa362993640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c112010 con 0x7fa35c102d30 2024-09-10T08:48:10.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.883+0000 7fa362993640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c111e90 con 0x7fa35c1036e0 2024-09-10T08:48:10.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.883+0000 7fa362993640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c112190 con 0x7fa35c1082a0 2024-09-10T08:48:10.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.884+0000 7fa35bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa35c102d30 0x7fa35c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.884+0000 7fa35bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa35c102d30 0x7fa35c198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38714/0 (socket says 172.21.15.172:38714) 2024-09-10T08:48:10.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.884+0000 7fa35bfff640 1 -- 172.21.15.172:0/2329614553 learned_addr learned my addr 172.21.15.172:0/2329614553 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:10.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.884+0000 7fa360f30640 1 --2- 172.21.15.172:0/2329614553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35c1082a0 0x7fa35c19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.884+0000 7fa35b7fe640 1 --2- 172.21.15.172:0/2329614553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa35c1036e0 0x7fa35c198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.884+0000 7fa35bfff640 1 -- 172.21.15.172:0/2329614553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35c1082a0 msgr2=0x7fa35c19cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:10.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.884+0000 7fa35bfff640 1 --2- 172.21.15.172:0/2329614553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35c1082a0 0x7fa35c19cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.884+0000 7fa35bfff640 1 -- 172.21.15.172:0/2329614553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa35c1036e0 msgr2=0x7fa35c198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:10.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.884+0000 7fa35bfff640 1 --2- 172.21.15.172:0/2329614553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa35c1036e0 0x7fa35c198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.884+0000 7fa35bfff640 1 -- 172.21.15.172:0/2329614553 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c19d5e0 con 0x7fa35c102d30 2024-09-10T08:48:10.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.884+0000 7fa35bfff640 1 --2- 172.21.15.172:0/2329614553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa35c102d30 0x7fa35c198630 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa34c005940 tx=0x7fa34c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:10.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.884+0000 7fa3597fa640 1 -- 172.21.15.172:0/2329614553 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c03f030 con 0x7fa35c102d30 2024-09-10T08:48:10.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.884+0000 7fa362993640 1 -- 172.21.15.172:0/2329614553 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35c19d870 con 0x7fa35c102d30 2024-09-10T08:48:10.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.884+0000 7fa3597fa640 1 -- 172.21.15.172:0/2329614553 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa34c003cb0 con 0x7fa35c102d30 2024-09-10T08:48:10.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.884+0000 7fa3597fa640 1 -- 172.21.15.172:0/2329614553 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c035070 con 0x7fa35c102d30 2024-09-10T08:48:10.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.884+0000 7fa362993640 1 -- 172.21.15.172:0/2329614553 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35c1a51b0 con 0x7fa35c102d30 2024-09-10T08:48:10.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.887+0000 7fa3597fa640 1 -- 172.21.15.172:0/2329614553 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa34c042030 con 0x7fa35c102d30 2024-09-10T08:48:10.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.887+0000 7fa364d6c740 1 Processor -- start 2024-09-10T08:48:10.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.887+0000 7fa364d6c740 1 -- start start 2024-09-10T08:48:10.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.887+0000 7fa364d6c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73470cb40 0x55d73470b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.887+0000 7fa364d6c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d73470ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.887+0000 7fa364d6c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73471a780 0x55d73471cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.887+0000 7fa364d6c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d7346712f0 con 0x55d73470b5d0 2024-09-10T08:48:10.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.887+0000 7fa364d6c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d734618cf0 con 0x55d73470cb40 2024-09-10T08:48:10.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.887+0000 7fa364d6c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d73466d2b0 con 0x55d73471a780 2024-09-10T08:48:10.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.887+0000 7fa35bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d73470ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.887+0000 7fa35bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d73470ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38730/0 (socket says 172.21.15.172:38730) 2024-09-10T08:48:10.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.887+0000 7fa35bfff640 1 -- 172.21.15.172:0/2184277491 learned_addr learned my addr 172.21.15.172:0/2184277491 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:10.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.888+0000 7fa360f30640 1 --2- 172.21.15.172:0/2184277491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73470cb40 0x55d73470b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.888+0000 7fa35b7fe640 1 --2- 172.21.15.172:0/2184277491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73471a780 0x55d73471cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.888+0000 7fa3597fa640 1 --2- 172.21.15.172:0/2329614553 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3440a2b80 0x7fa3440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.888+0000 7fa35bfff640 1 -- 172.21.15.172:0/2184277491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73471a780 msgr2=0x55d73471cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:10.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.888+0000 7fa35bfff640 1 --2- 172.21.15.172:0/2184277491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73471a780 0x55d73471cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.888+0000 7fa35bfff640 1 -- 172.21.15.172:0/2184277491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73470cb40 msgr2=0x55d73470b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:10.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.888+0000 7fa35bfff640 1 --2- 172.21.15.172:0/2184277491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73470cb40 0x55d73470b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.888+0000 7fa35bfff640 1 -- 172.21.15.172:0/2184277491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7345fd3a0 con 0x55d73470b5d0 2024-09-10T08:48:10.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.888+0000 7fa360f30640 1 --2- 172.21.15.172:0/2329614553 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3440a2b80 0x7fa3440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.888+0000 7fa3597fa640 1 -- 172.21.15.172:0/2329614553 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa34c0ef830 con 0x7fa35c102d30 2024-09-10T08:48:10.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.888+0000 7fa35bfff640 1 --2- 172.21.15.172:0/2184277491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d73470ba10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa34c08e820 tx=0x7fa34c08dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3d8c4e1497dbb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:10.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.888+0000 7fa3427fc640 1 -- 172.21.15.172:0/2184277491 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c049070 con 0x55d73470b5d0 2024-09-10T08:48:10.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.888+0000 7fa360f30640 1 --2- 172.21.15.172:0/2329614553 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3440a2b80 0x7fa3440a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa354009fd0 tx=0x7fa354004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:10.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.888+0000 7fa3427fc640 1 -- 172.21.15.172:0/2184277491 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa34c08f2f0 con 0x55d73470b5d0 2024-09-10T08:48:10.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.888+0000 7fa3427fc640 1 -- 172.21.15.172:0/2184277491 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0bb870 con 0x55d73470b5d0 2024-09-10T08:48:10.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.889+0000 7fa364d6c740 1 -- 172.21.15.172:0/2184277491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 msgr2=0x55d73470ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:10.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.889+0000 7fa364d6c740 1 --2- 172.21.15.172:0/2184277491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d73470ba10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa34c08e820 tx=0x7fa34c08dd40 comp rx=0 tx=0).stop 2024-09-10T08:48:10.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.889+0000 7fa364d6c740 1 -- 172.21.15.172:0/2184277491 shutdown_connections 2024-09-10T08:48:10.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.889+0000 7fa364d6c740 1 --2- 172.21.15.172:0/2184277491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73471a780 0x55d73471cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.889+0000 7fa364d6c740 1 --2- 172.21.15.172:0/2184277491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d73470ba10 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.889+0000 7fa364d6c740 1 --2- 172.21.15.172:0/2184277491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73470cb40 0x55d73470b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.889+0000 7fa364d6c740 1 -- 172.21.15.172:0/2184277491 >> 172.21.15.172:0/2184277491 conn(0x55d7346d4310 msgr2=0x55d734707c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:10.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.889+0000 7fa364d6c740 1 -- 172.21.15.172:0/2184277491 shutdown_connections 2024-09-10T08:48:10.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.889+0000 7fa364d6c740 1 -- 172.21.15.172:0/2184277491 wait complete. 2024-09-10T08:48:10.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.890+0000 7fa364d6c740 1 -- start start 2024-09-10T08:48:10.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.890+0000 7fa364d6c740 1 Processor -- start 2024-09-10T08:48:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.890+0000 7fa364d6c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d73471a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.890+0000 7fa364d6c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73470cb40 0x55d734717420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.890+0000 7fa364d6c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73471a780 0x55d734717a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.890+0000 7fa364d6c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d73471d240 con 0x55d73470b5d0 2024-09-10T08:48:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.890+0000 7fa364d6c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d73462c060 con 0x55d73471a780 2024-09-10T08:48:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.890+0000 7fa364d6c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d73471d3c0 con 0x55d73470cb40 2024-09-10T08:48:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.890+0000 7fa364d6c740 10 client.0 _ll_register_callbacks cb 0x55d73437f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.890+0000 7fa35bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d73471a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.890+0000 7fa35bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d73471a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38736/0 (socket says 172.21.15.172:38736) 2024-09-10T08:48:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.890+0000 7fa35bfff640 1 -- 172.21.15.172:0/2278859269 learned_addr learned my addr 172.21.15.172:0/2278859269 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.890+0000 7fa35b7fe640 1 --2- 172.21.15.172:0/2278859269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73470cb40 0x55d734717420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.891+0000 7fa360f30640 1 --2- 172.21.15.172:0/2278859269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73471a780 0x55d734717a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.891+0000 7fa35bfff640 1 -- 172.21.15.172:0/2278859269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73470cb40 msgr2=0x55d734717420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.891+0000 7fa35bfff640 1 --2- 172.21.15.172:0/2278859269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73470cb40 0x55d734717420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.891+0000 7fa35bfff640 1 -- 172.21.15.172:0/2278859269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73471a780 msgr2=0x55d734717a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.891+0000 7fa35bfff640 1 --2- 172.21.15.172:0/2278859269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73471a780 0x55d734717a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.891+0000 7fa35bfff640 1 -- 172.21.15.172:0/2278859269 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7343e23c0 con 0x55d73470b5d0 2024-09-10T08:48:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.891+0000 7fa360f30640 1 --2- 172.21.15.172:0/2278859269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73471a780 0x55d734717a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.891+0000 7fa35bfff640 1 --2- 172.21.15.172:0/2278859269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d73471a170 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa34c08f4b0 tx=0x7fa34c004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d2dac820eaff94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.891+0000 7fa33b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.891+0000 7fa33b7fe640 1 -- 172.21.15.172:0/2278859269 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c049040 con 0x55d73470b5d0 2024-09-10T08:48:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.891+0000 7fa33b7fe640 1 -- 172.21.15.172:0/2278859269 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa34c0ba030 con 0x55d73470b5d0 2024-09-10T08:48:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.891+0000 7fa364d6c740 1 -- 172.21.15.172:0/2278859269 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d73467e860 con 0x55d73470b5d0 2024-09-10T08:48:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.891+0000 7fa33b7fe640 1 -- 172.21.15.172:0/2278859269 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa34c08ee60 con 0x55d73470b5d0 2024-09-10T08:48:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.891+0000 7fa364d6c740 10 client.15918 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.891+0000 7fa364d6c740 1 -- 172.21.15.172:0/2278859269 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d73471d240 con 0x55d73470b5d0 2024-09-10T08:48:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.891+0000 7fa33b7fe640 1 -- 172.21.15.172:0/2278859269 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55d73467e860 con 0x55d73470b5d0 2024-09-10T08:48:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.892+0000 7fa33b7fe640 1 -- 172.21.15.172:0/2278859269 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa34c0efce0 con 0x55d73470b5d0 2024-09-10T08:48:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.892+0000 7fa364d6c740 10 client.15918 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.892+0000 7fa364d6c740 10 client.15918 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:48:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.892+0000 7fa364d6c740 20 client.15918 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.892+0000 7fa364d6c740 1 --2- 172.21.15.172:0/2278859269 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d734729eb0 0x55d73472c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.892+0000 7fa364d6c740 4 client.15918 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.892+0000 7fa364d6c740 1 -- 172.21.15.172:0/2278859269 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d734678d70 con 0x55d734729eb0 2024-09-10T08:48:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.892+0000 7fa35b7fe640 1 --2- 172.21.15.172:0/2278859269 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d734729eb0 0x55d73472c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.892+0000 7fa35b7fe640 1 --2- 172.21.15.172:0/2278859269 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d734729eb0 0x55d73472c2a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=124a28113bf3f1d server_cookie=9469aed7fb077637 in_seq=0 out_seq=0 2024-09-10T08:48:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.892+0000 7fa33b7fe640 10 client.15918 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:48:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.893+0000 7fa33b7fe640 1 -- 172.21.15.172:0/2278859269 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d734678d70 con 0x55d734729eb0 2024-09-10T08:48:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.893+0000 7fa33b7fe640 10 client.15918 handle_command_reply: tid=0 2024-09-10T08:48:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.893+0000 7fa364d6c740 1 client.15918 shutdown 2024-09-10T08:48:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.893+0000 7fa364d6c740 1 -- 172.21.15.172:0/2278859269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 msgr2=0x55d73471a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.893+0000 7fa364d6c740 1 --2- 172.21.15.172:0/2278859269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d73471a170 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa34c08f4b0 tx=0x7fa34c004570 comp rx=0 tx=0).stop 2024-09-10T08:48:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.893+0000 7fa364d6c740 1 -- 172.21.15.172:0/2278859269 shutdown_connections 2024-09-10T08:48:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.893+0000 7fa364d6c740 1 --2- 172.21.15.172:0/2278859269 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d734729eb0 0x55d73472c2a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.893+0000 7fa364d6c740 1 --2- 172.21.15.172:0/2278859269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73471a780 0x55d734717a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.893+0000 7fa364d6c740 1 --2- 172.21.15.172:0/2278859269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73470cb40 0x55d734717420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.893+0000 7fa364d6c740 1 --2- 172.21.15.172:0/2278859269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d73471a170 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.893+0000 7fa364d6c740 1 -- 172.21.15.172:0/2278859269 >> 172.21.15.172:0/2278859269 conn(0x55d7346d4310 msgr2=0x55d73470d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.893+0000 7fa33b7fe640 0 client.15918 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:48:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.893+0000 7fa364d6c740 1 -- 172.21.15.172:0/2278859269 shutdown_connections 2024-09-10T08:48:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.893+0000 7fa364d6c740 1 -- 172.21.15.172:0/2278859269 wait complete. 2024-09-10T08:48:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.894+0000 7fa364d6c740 20 client.15918 trim_cache size 0 max 16384 2024-09-10T08:48:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa364d6c740 1 Processor -- start 2024-09-10T08:48:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa364d6c740 1 -- start start 2024-09-10T08:48:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa364d6c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73470b5d0 0x55d734726410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa364d6c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470cb40 0x55d734731b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa364d6c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73471a780 0x55d734732090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa364d6c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1ac9b0 con 0x55d73470cb40 2024-09-10T08:48:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa364d6c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1ae4f0 con 0x55d73470b5d0 2024-09-10T08:48:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa364d6c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1ad080 con 0x55d73471a780 2024-09-10T08:48:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa35b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470cb40 0x55d734731b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa35b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470cb40 0x55d734731b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38750/0 (socket says 172.21.15.172:38750) 2024-09-10T08:48:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa35b7fe640 1 -- 172.21.15.172:0/1583777953 learned_addr learned my addr 172.21.15.172:0/1583777953 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa35bfff640 1 --2- 172.21.15.172:0/1583777953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73470b5d0 0x55d734726410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa35b7fe640 1 -- 172.21.15.172:0/1583777953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73471a780 msgr2=0x55d734732090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa360f30640 1 --2- 172.21.15.172:0/1583777953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73471a780 0x55d734732090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa35b7fe640 1 --2- 172.21.15.172:0/1583777953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73471a780 0x55d734732090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa35b7fe640 1 -- 172.21.15.172:0/1583777953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73470b5d0 msgr2=0x55d734726410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa35b7fe640 1 --2- 172.21.15.172:0/1583777953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73470b5d0 0x55d734726410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa35b7fe640 1 -- 172.21.15.172:0/1583777953 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d734708960 con 0x55d73470cb40 2024-09-10T08:48:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.907+0000 7fa35bfff640 1 --2- 172.21.15.172:0/1583777953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73470b5d0 0x55d734726410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.908+0000 7fa35b7fe640 1 --2- 172.21.15.172:0/1583777953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470cb40 0x55d734731b50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0fef10 tx=0x7fa350009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104a4f63bc1b2f9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:10.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.908+0000 7fa341ffb640 1 -- 172.21.15.172:0/1583777953 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350025070 con 0x55d73470cb40 2024-09-10T08:48:10.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.908+0000 7fa341ffb640 1 -- 172.21.15.172:0/1583777953 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa350015e00 con 0x55d73470cb40 2024-09-10T08:48:10.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.908+0000 7fa341ffb640 1 -- 172.21.15.172:0/1583777953 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350014410 con 0x55d73470cb40 2024-09-10T08:48:10.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.908+0000 7fa364d6c740 1 -- 172.21.15.172:0/1583777953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470cb40 msgr2=0x55d734731b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:10.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.908+0000 7fa364d6c740 1 --2- 172.21.15.172:0/1583777953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470cb40 0x55d734731b50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0fef10 tx=0x7fa350009110 comp rx=0 tx=0).stop 2024-09-10T08:48:10.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.909+0000 7fa364d6c740 1 -- 172.21.15.172:0/1583777953 shutdown_connections 2024-09-10T08:48:10.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.909+0000 7fa364d6c740 1 --2- 172.21.15.172:0/1583777953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73471a780 0x55d734732090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.909+0000 7fa364d6c740 1 --2- 172.21.15.172:0/1583777953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470cb40 0x55d734731b50 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.909+0000 7fa364d6c740 1 --2- 172.21.15.172:0/1583777953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73470b5d0 0x55d734726410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.909+0000 7fa364d6c740 1 -- 172.21.15.172:0/1583777953 >> 172.21.15.172:0/1583777953 conn(0x55d7346d4310 msgr2=0x55d734715cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:10.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.909+0000 7fa364d6c740 1 -- 172.21.15.172:0/1583777953 shutdown_connections 2024-09-10T08:48:10.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.909+0000 7fa364d6c740 1 -- 172.21.15.172:0/1583777953 wait complete. 2024-09-10T08:48:10.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.909+0000 7fa364d6c740 1 -- start start 2024-09-10T08:48:10.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.909+0000 7fa364d6c740 1 Processor -- start 2024-09-10T08:48:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.909+0000 7fa364d6c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d7347229c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.909+0000 7fa364d6c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73470cb40 0x55d73471dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.909+0000 7fa364d6c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73471a780 0x55d73471e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.909+0000 7fa364d6c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1ade20 con 0x55d73470b5d0 2024-09-10T08:48:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.909+0000 7fa364d6c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1ad750 con 0x55d73470cb40 2024-09-10T08:48:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.909+0000 7fa364d6c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c1aa970 con 0x55d73471a780 2024-09-10T08:48:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.910+0000 7fa364d6c740 10 client.0 _ll_register_callbacks cb 0x7fa354009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.910+0000 7fa35bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d7347229c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.910+0000 7fa35bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d7347229c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38756/0 (socket says 172.21.15.172:38756) 2024-09-10T08:48:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.910+0000 7fa35bfff640 1 -- 172.21.15.172:0/4141042670 learned_addr learned my addr 172.21.15.172:0/4141042670 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.910+0000 7fa35b7fe640 1 --2- 172.21.15.172:0/4141042670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73470cb40 0x55d73471dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.910+0000 7fa360f30640 1 --2- 172.21.15.172:0/4141042670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73471a780 0x55d73471e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.910+0000 7fa35bfff640 1 -- 172.21.15.172:0/4141042670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73471a780 msgr2=0x55d73471e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.910+0000 7fa35bfff640 1 --2- 172.21.15.172:0/4141042670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73471a780 0x55d73471e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.910+0000 7fa35bfff640 1 -- 172.21.15.172:0/4141042670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73470cb40 msgr2=0x55d73471dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.910+0000 7fa35bfff640 1 --2- 172.21.15.172:0/4141042670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73470cb40 0x55d73471dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.910+0000 7fa35bfff640 1 -- 172.21.15.172:0/4141042670 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d73470e120 con 0x55d73470b5d0 2024-09-10T08:48:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.910+0000 7fa35b7fe640 1 --2- 172.21.15.172:0/4141042670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73470cb40 0x55d73471dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.910+0000 7fa35bfff640 1 --2- 172.21.15.172:0/4141042670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d7347229c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa34c032fd0 tx=0x7fa34c08d140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463cf9f30188e7e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.910+0000 7fa33b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.910+0000 7fa33b7fe640 1 -- 172.21.15.172:0/4141042670 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c049070 con 0x55d73470b5d0 2024-09-10T08:48:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.910+0000 7fa33b7fe640 1 -- 172.21.15.172:0/4141042670 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa34c08f000 con 0x55d73470b5d0 2024-09-10T08:48:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.910+0000 7fa364d6c740 1 -- 172.21.15.172:0/4141042670 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa350015e00 con 0x55d73470b5d0 2024-09-10T08:48:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.911+0000 7fa33b7fe640 1 -- 172.21.15.172:0/4141042670 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa34c0d4260 con 0x55d73470b5d0 2024-09-10T08:48:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.911+0000 7fa364d6c740 10 client.15924 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.911+0000 7fa364d6c740 1 -- 172.21.15.172:0/4141042670 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d73470e120 con 0x55d73470b5d0 2024-09-10T08:48:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.911+0000 7fa33b7fe640 1 -- 172.21.15.172:0/4141042670 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa350015e00 con 0x55d73470b5d0 2024-09-10T08:48:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.911+0000 7fa33b7fe640 1 -- 172.21.15.172:0/4141042670 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa34c08e290 con 0x55d73470b5d0 2024-09-10T08:48:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.911+0000 7fa364d6c740 10 client.15924 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.911+0000 7fa364d6c740 10 client.15924 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:48:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.911+0000 7fa364d6c740 20 client.15924 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.911+0000 7fa364d6c740 1 --2- 172.21.15.172:0/4141042670 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d7347948e0 0x55d734796cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.911+0000 7fa364d6c740 4 client.15924 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:48:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.911+0000 7fa364d6c740 1 -- 172.21.15.172:0/4141042670 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7345f5170 con 0x55d7347948e0 2024-09-10T08:48:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.911+0000 7fa35b7fe640 1 --2- 172.21.15.172:0/4141042670 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d7347948e0 0x55d734796cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.912+0000 7fa33b7fe640 10 client.15924 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:48:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.912+0000 7fa35b7fe640 1 --2- 172.21.15.172:0/4141042670 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d7347948e0 0x55d734796cd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d1924ecaf2a00f23 server_cookie=8e1d6cb22322dec5 in_seq=0 out_seq=0 2024-09-10T08:48:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.912+0000 7fa33b7fe640 1 -- 172.21.15.172:0/4141042670 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7345f5170 con 0x55d7347948e0 2024-09-10T08:48:10.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.912+0000 7fa33b7fe640 10 client.15924 handle_command_reply: tid=0 2024-09-10T08:48:10.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.912+0000 7fa364d6c740 1 client.15924 shutdown 2024-09-10T08:48:10.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.912+0000 7fa364d6c740 1 -- 172.21.15.172:0/4141042670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 msgr2=0x55d7347229c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:10.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.912+0000 7fa364d6c740 1 --2- 172.21.15.172:0/4141042670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d7347229c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa34c032fd0 tx=0x7fa34c08d140 comp rx=0 tx=0).stop 2024-09-10T08:48:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.912+0000 7fa364d6c740 1 -- 172.21.15.172:0/4141042670 shutdown_connections 2024-09-10T08:48:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.912+0000 7fa364d6c740 1 --2- 172.21.15.172:0/4141042670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d73471a780 0x55d73471e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.912+0000 7fa364d6c740 1 --2- 172.21.15.172:0/4141042670 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d7347948e0 0x55d734796cd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.912+0000 7fa364d6c740 1 --2- 172.21.15.172:0/4141042670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d73470cb40 0x55d73471dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.912+0000 7fa364d6c740 1 --2- 172.21.15.172:0/4141042670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d73470b5d0 0x55d7347229c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.912+0000 7fa33b7fe640 0 client.15924 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:48:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.912+0000 7fa364d6c740 1 -- 172.21.15.172:0/4141042670 >> 172.21.15.172:0/4141042670 conn(0x55d7346d4310 msgr2=0x55d73470d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:10.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.912+0000 7fa364d6c740 1 -- 172.21.15.172:0/4141042670 shutdown_connections 2024-09-10T08:48:10.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.912+0000 7fa364d6c740 1 -- 172.21.15.172:0/4141042670 wait complete. 2024-09-10T08:48:10.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.912+0000 7fa364d6c740 20 client.15924 trim_cache size 0 max 16384 2024-09-10T08:48:10.932 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:48:10.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.913+0000 7fa362993640 1 -- 172.21.15.172:0/2329614553 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3440a2b80 msgr2=0x7fa3440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:10.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.913+0000 7fa362993640 1 --2- 172.21.15.172:0/2329614553 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3440a2b80 0x7fa3440a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa354009fd0 tx=0x7fa354004b50 comp rx=0 tx=0).stop 2024-09-10T08:48:10.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.913+0000 7fa362993640 1 -- 172.21.15.172:0/2329614553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa35c102d30 msgr2=0x7fa35c198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:10.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.913+0000 7fa362993640 1 --2- 172.21.15.172:0/2329614553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa35c102d30 0x7fa35c198630 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa34c005940 tx=0x7fa34c0042f0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.913+0000 7fa362993640 1 -- 172.21.15.172:0/2329614553 shutdown_connections 2024-09-10T08:48:10.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.913+0000 7fa362993640 1 --2- 172.21.15.172:0/2329614553 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3440a2b80 0x7fa3440a5040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.913+0000 7fa362993640 1 --2- 172.21.15.172:0/2329614553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa35c1082a0 0x7fa35c19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.913+0000 7fa362993640 1 --2- 172.21.15.172:0/2329614553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa35c1036e0 0x7fa35c198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.913+0000 7fa362993640 1 --2- 172.21.15.172:0/2329614553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa35c102d30 0x7fa35c198630 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:10.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.913+0000 7fa362993640 1 -- 172.21.15.172:0/2329614553 >> 172.21.15.172:0/2329614553 conn(0x7fa35c0fc7d0 msgr2=0x7fa35c105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:10.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.913+0000 7fa362993640 1 -- 172.21.15.172:0/2329614553 shutdown_connections 2024-09-10T08:48:10.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:10.913+0000 7fa362993640 1 -- 172.21.15.172:0/2329614553 wait complete. 2024-09-10T08:48:11.907 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T08:48:12.170 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:12.519 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:12 smithi172 ceph-mon[30854]: pgmap v409: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:48:12.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:12 smithi191 ceph-mon[35314]: pgmap v409: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:48:12.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:12 smithi174 ceph-mon[34068]: pgmap v409: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:48:12.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.720+0000 7f4043539640 1 -- 172.21.15.172:0/3748087531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f403c106a90 msgr2=0x7f403c106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.720+0000 7f4043539640 1 --2- 172.21.15.172:0/3748087531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f403c106a90 0x7f403c106e70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4038009930 tx=0x7f403802a420 comp rx=0 tx=0).stop 2024-09-10T08:48:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.720+0000 7f4043539640 1 -- 172.21.15.172:0/3748087531 shutdown_connections 2024-09-10T08:48:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.720+0000 7f4043539640 1 --2- 172.21.15.172:0/3748087531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f403c101430 0x7f403c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.720+0000 7f4043539640 1 --2- 172.21.15.172:0/3748087531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f403c100a90 0x7f403c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.720+0000 7f4043539640 1 --2- 172.21.15.172:0/3748087531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f403c106a90 0x7f403c106e70 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.720+0000 7f4043539640 1 -- 172.21.15.172:0/3748087531 >> 172.21.15.172:0/3748087531 conn(0x7f403c0fc7d0 msgr2=0x7f403c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.721+0000 7f4043539640 1 -- 172.21.15.172:0/3748087531 shutdown_connections 2024-09-10T08:48:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.721+0000 7f4043539640 1 -- 172.21.15.172:0/3748087531 wait complete. 2024-09-10T08:48:12.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.721+0000 7f4043539640 1 Processor -- start 2024-09-10T08:48:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.721+0000 7f4043539640 1 -- start start 2024-09-10T08:48:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.721+0000 7f4043539640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f403c100a90 0x7f403c102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.721+0000 7f4043539640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f403c101430 0x7f403c102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.721+0000 7f4043539640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f403c106a90 0x7f403c1030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.721+0000 7f4043539640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f403c110d40 con 0x7f403c106a90 2024-09-10T08:48:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.721+0000 7f4043539640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f403c110bc0 con 0x7f403c101430 2024-09-10T08:48:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.721+0000 7f4043539640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f403c110ec0 con 0x7f403c100a90 2024-09-10T08:48:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.722+0000 7f4041ad6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f403c106a90 0x7f403c1030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.722+0000 7f4041ad6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f403c106a90 0x7f403c1030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38772/0 (socket says 172.21.15.172:38772) 2024-09-10T08:48:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.722+0000 7f4041ad6640 1 -- 172.21.15.172:0/2372719385 learned_addr learned my addr 172.21.15.172:0/2372719385 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.722+0000 7f40412d5640 1 --2- 172.21.15.172:0/2372719385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f403c100a90 0x7f403c102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.722+0000 7f4040ad4640 1 --2- 172.21.15.172:0/2372719385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f403c101430 0x7f403c102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.722+0000 7f4041ad6640 1 -- 172.21.15.172:0/2372719385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f403c100a90 msgr2=0x7f403c102660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.722+0000 7f4041ad6640 1 --2- 172.21.15.172:0/2372719385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f403c100a90 0x7f403c102660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.722+0000 7f4041ad6640 1 -- 172.21.15.172:0/2372719385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f403c101430 msgr2=0x7f403c102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.722+0000 7f4041ad6640 1 --2- 172.21.15.172:0/2372719385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f403c101430 0x7f403c102ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.722+0000 7f4041ad6640 1 -- 172.21.15.172:0/2372719385 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c1a6890 con 0x7f403c106a90 2024-09-10T08:48:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.722+0000 7f4041ad6640 1 --2- 172.21.15.172:0/2372719385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f403c106a90 0x7f403c1030e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f403400cad0 tx=0x7f403400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.722+0000 7f40327fc640 1 -- 172.21.15.172:0/2372719385 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403400ae20 con 0x7f403c106a90 2024-09-10T08:48:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.722+0000 7f4043539640 1 -- 172.21.15.172:0/2372719385 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c1a6b80 con 0x7f403c106a90 2024-09-10T08:48:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.722+0000 7f40327fc640 1 -- 172.21.15.172:0/2372719385 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4034005020 con 0x7f403c106a90 2024-09-10T08:48:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.722+0000 7f40327fc640 1 -- 172.21.15.172:0/2372719385 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403400f6b0 con 0x7f403c106a90 2024-09-10T08:48:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.722+0000 7f4043539640 1 -- 172.21.15.172:0/2372719385 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c1a7110 con 0x7f403c106a90 2024-09-10T08:48:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.725+0000 7f40327fc640 1 -- 172.21.15.172:0/2372719385 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f40340051d0 con 0x7f403c106a90 2024-09-10T08:48:12.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.725+0000 7f4045912740 1 Processor -- start 2024-09-10T08:48:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.725+0000 7f4045912740 1 -- start start 2024-09-10T08:48:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.725+0000 7f4045912740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563386448b40 0x563386447090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.725+0000 7f40327fc640 1 --2- 172.21.15.172:0/2372719385 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f40280a2c00 0x7f40280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.725+0000 7f4045912740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633864475d0 0x563386447a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.725+0000 7f4045912740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386452370 0x563386454760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.725+0000 7f4045912740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5633863ad2f0 con 0x563386452370 2024-09-10T08:48:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.725+0000 7f4045912740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563386354cf0 con 0x5633864475d0 2024-09-10T08:48:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.725+0000 7f4045912740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5633863a92b0 con 0x563386448b40 2024-09-10T08:48:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.726+0000 7f40412d5640 1 --2- 172.21.15.172:0/2372719385 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f40280a2c00 0x7f40280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.726+0000 7f4041ad6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386452370 0x563386454760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.726+0000 7f4040ad4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563386448b40 0x563386447090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.726+0000 7f4041ad6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386452370 0x563386454760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38782/0 (socket says 172.21.15.172:38782) 2024-09-10T08:48:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.726+0000 7f40327fc640 1 -- 172.21.15.172:0/2372719385 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f40340cf1f0 con 0x7f403c106a90 2024-09-10T08:48:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.726+0000 7f4041ad6640 1 -- 172.21.15.172:0/3610719296 learned_addr learned my addr 172.21.15.172:0/3610719296 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.726+0000 7f4040ad4640 1 --2- 172.21.15.172:0/3610719296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633864475d0 0x563386447a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.726+0000 7f4040ad4640 1 -- 172.21.15.172:0/3610719296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633864475d0 msgr2=0x563386447a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.726+0000 7f4040ad4640 1 --2- 172.21.15.172:0/3610719296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633864475d0 0x563386447a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.726+0000 7f4040ad4640 1 -- 172.21.15.172:0/3610719296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386452370 msgr2=0x563386454760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.726+0000 7f4040ad4640 1 --2- 172.21.15.172:0/3610719296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386452370 0x563386454760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.726+0000 7f4040ad4640 1 -- 172.21.15.172:0/3610719296 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633863393a0 con 0x563386448b40 2024-09-10T08:48:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.726+0000 7f4041ad6640 1 --2- 172.21.15.172:0/3610719296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386452370 0x563386454760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.726+0000 7f40412d5640 1 --2- 172.21.15.172:0/2372719385 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f40280a2c00 0x7f40280a50c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4038009900 tx=0x7f4038000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.726+0000 7f4040ad4640 1 --2- 172.21.15.172:0/3610719296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563386448b40 0x563386447090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x563386444ac0 tx=0x7f402c005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaf889859a39cda7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.727+0000 7f40277fe640 1 -- 172.21.15.172:0/3610719296 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c012d20 con 0x563386448b40 2024-09-10T08:48:12.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.727+0000 7f40277fe640 1 -- 172.21.15.172:0/3610719296 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f402c005020 con 0x563386448b40 2024-09-10T08:48:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.727+0000 7f40277fe640 1 -- 172.21.15.172:0/3610719296 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c004a00 con 0x563386448b40 2024-09-10T08:48:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.727+0000 7f4045912740 1 -- 172.21.15.172:0/3610719296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563386448b40 msgr2=0x563386447090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.727+0000 7f4045912740 1 --2- 172.21.15.172:0/3610719296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563386448b40 0x563386447090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x563386444ac0 tx=0x7f402c005c90 comp rx=0 tx=0).stop 2024-09-10T08:48:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.727+0000 7f4045912740 1 -- 172.21.15.172:0/3610719296 shutdown_connections 2024-09-10T08:48:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.727+0000 7f4045912740 1 --2- 172.21.15.172:0/3610719296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386452370 0x563386454760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.727+0000 7f4045912740 1 --2- 172.21.15.172:0/3610719296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633864475d0 0x563386447a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.727+0000 7f4045912740 1 --2- 172.21.15.172:0/3610719296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563386448b40 0x563386447090 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.727+0000 7f4045912740 1 -- 172.21.15.172:0/3610719296 >> 172.21.15.172:0/3610719296 conn(0x563386410310 msgr2=0x563386443c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.727+0000 7f4045912740 1 -- 172.21.15.172:0/3610719296 shutdown_connections 2024-09-10T08:48:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.727+0000 7f4045912740 1 -- 172.21.15.172:0/3610719296 wait complete. 2024-09-10T08:48:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.728+0000 7f4045912740 1 -- start start 2024-09-10T08:48:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.728+0000 7f4045912740 1 Processor -- start 2024-09-10T08:48:12.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.728+0000 7f4045912740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633864475d0 0x563386451d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.728+0000 7f4045912740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563386448b40 0x56338644f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.728+0000 7f4045912740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563386452370 0x56338644f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.728+0000 7f4045912740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563386455070 con 0x5633864475d0 2024-09-10T08:48:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.728+0000 7f4045912740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563386368060 con 0x563386452370 2024-09-10T08:48:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.728+0000 7f4045912740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5633864551f0 con 0x563386448b40 2024-09-10T08:48:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.728+0000 7f4045912740 10 client.0 _ll_register_callbacks cb 0x5633860bb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.728+0000 7f40412d5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633864475d0 0x563386451d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.728+0000 7f40412d5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633864475d0 0x563386451d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38794/0 (socket says 172.21.15.172:38794) 2024-09-10T08:48:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.728+0000 7f40412d5640 1 -- 172.21.15.172:0/1723207018 learned_addr learned my addr 172.21.15.172:0/1723207018 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.728+0000 7f4040ad4640 1 --2- 172.21.15.172:0/1723207018 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563386448b40 0x56338644f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.729+0000 7f4041ad6640 1 --2- 172.21.15.172:0/1723207018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563386452370 0x56338644f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.729+0000 7f40412d5640 1 -- 172.21.15.172:0/1723207018 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563386448b40 msgr2=0x56338644f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:12.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.729+0000 7f40412d5640 1 --2- 172.21.15.172:0/1723207018 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563386448b40 0x56338644f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.729+0000 7f40412d5640 1 -- 172.21.15.172:0/1723207018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563386452370 msgr2=0x56338644f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:12.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.729+0000 7f40412d5640 1 --2- 172.21.15.172:0/1723207018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563386452370 0x56338644f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.729+0000 7f40412d5640 1 -- 172.21.15.172:0/1723207018 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56338611e3c0 con 0x5633864475d0 2024-09-10T08:48:12.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.729+0000 7f4041ad6640 1 --2- 172.21.15.172:0/1723207018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563386452370 0x56338644f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:12.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.729+0000 7f40412d5640 1 --2- 172.21.15.172:0/1723207018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633864475d0 0x563386451d60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4038009930 tx=0x7f4038009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168caf20824899ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:12.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.729+0000 7f4024ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:12.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.729+0000 7f4024ff9640 1 -- 172.21.15.172:0/1723207018 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403804c070 con 0x5633864475d0 2024-09-10T08:48:12.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.729+0000 7f4024ff9640 1 -- 172.21.15.172:0/1723207018 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4038003800 con 0x5633864475d0 2024-09-10T08:48:12.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.729+0000 7f4045912740 1 -- 172.21.15.172:0/1723207018 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f402c005020 con 0x5633864475d0 2024-09-10T08:48:12.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.729+0000 7f4024ff9640 1 -- 172.21.15.172:0/1723207018 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4038058050 con 0x5633864475d0 2024-09-10T08:48:12.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.729+0000 7f4045912740 10 client.15936 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:12.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.729+0000 7f4045912740 1 -- 172.21.15.172:0/1723207018 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563386460b70 con 0x5633864475d0 2024-09-10T08:48:12.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.730+0000 7f4024ff9640 1 -- 172.21.15.172:0/1723207018 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f402c005020 con 0x5633864475d0 2024-09-10T08:48:12.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.730+0000 7f4024ff9640 1 -- 172.21.15.172:0/1723207018 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f403803c9a0 con 0x5633864475d0 2024-09-10T08:48:12.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.730+0000 7f4045912740 10 client.15936 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:12.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.730+0000 7f4045912740 10 client.15936 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:48:12.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.730+0000 7f4045912740 20 client.15936 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:12.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.730+0000 7f4045912740 1 --2- 172.21.15.172:0/1723207018 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x563386461fd0 0x563386464430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.730+0000 7f4045912740 4 client.15936 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:12.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.730+0000 7f4045912740 1 -- 172.21.15.172:0/1723207018 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633863ba860 con 0x563386461fd0 2024-09-10T08:48:12.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.730+0000 7f4040ad4640 1 --2- 172.21.15.172:0/1723207018 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x563386461fd0 0x563386464430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.731+0000 7f4024ff9640 10 client.15936 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:48:12.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.731+0000 7f4040ad4640 1 --2- 172.21.15.172:0/1723207018 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x563386461fd0 0x563386464430 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dff3105c1da58672 server_cookie=bb83bee29ec6a78 in_seq=0 out_seq=0 2024-09-10T08:48:12.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.731+0000 7f4024ff9640 1 -- 172.21.15.172:0/1723207018 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633863ba860 con 0x563386461fd0 2024-09-10T08:48:12.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.731+0000 7f4024ff9640 10 client.15936 handle_command_reply: tid=0 2024-09-10T08:48:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.731+0000 7f4045912740 1 client.15936 shutdown 2024-09-10T08:48:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.731+0000 7f4045912740 1 -- 172.21.15.172:0/1723207018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633864475d0 msgr2=0x563386451d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.731+0000 7f4045912740 1 --2- 172.21.15.172:0/1723207018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633864475d0 0x563386451d60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4038009930 tx=0x7f4038009260 comp rx=0 tx=0).stop 2024-09-10T08:48:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.732+0000 7f4045912740 1 -- 172.21.15.172:0/1723207018 shutdown_connections 2024-09-10T08:48:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.732+0000 7f4045912740 1 --2- 172.21.15.172:0/1723207018 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x563386461fd0 0x563386464430 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.732+0000 7f4045912740 1 --2- 172.21.15.172:0/1723207018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563386452370 0x56338644f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.732+0000 7f4045912740 1 --2- 172.21.15.172:0/1723207018 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563386448b40 0x56338644f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.732+0000 7f4045912740 1 --2- 172.21.15.172:0/1723207018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633864475d0 0x563386451d60 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.732+0000 7f4045912740 1 -- 172.21.15.172:0/1723207018 >> 172.21.15.172:0/1723207018 conn(0x563386410310 msgr2=0x563386449960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.732+0000 7f4024ff9640 0 client.15936 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:48:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.732+0000 7f4045912740 1 -- 172.21.15.172:0/1723207018 shutdown_connections 2024-09-10T08:48:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.732+0000 7f4045912740 1 -- 172.21.15.172:0/1723207018 wait complete. 2024-09-10T08:48:12.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.732+0000 7f4045912740 20 client.15936 trim_cache size 0 max 16384 2024-09-10T08:48:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f4045912740 1 Processor -- start 2024-09-10T08:48:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f4045912740 1 -- start start 2024-09-10T08:48:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f4045912740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633864475d0 0x56338645e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f4045912740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386448b40 0x56338644e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f4045912740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563386452370 0x563386469d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f4045912740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1b0dc0 con 0x563386448b40 2024-09-10T08:48:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f4045912740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1b2900 con 0x5633864475d0 2024-09-10T08:48:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f4045912740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1b1490 con 0x563386452370 2024-09-10T08:48:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f4040ad4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386448b40 0x56338644e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f4040ad4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386448b40 0x56338644e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38800/0 (socket says 172.21.15.172:38800) 2024-09-10T08:48:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f4040ad4640 1 -- 172.21.15.172:0/3014181603 learned_addr learned my addr 172.21.15.172:0/3014181603 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f40412d5640 1 --2- 172.21.15.172:0/3014181603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633864475d0 0x56338645e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f4041ad6640 1 --2- 172.21.15.172:0/3014181603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563386452370 0x563386469d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f4040ad4640 1 -- 172.21.15.172:0/3014181603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563386452370 msgr2=0x563386469d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f4040ad4640 1 --2- 172.21.15.172:0/3014181603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563386452370 0x563386469d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f4040ad4640 1 -- 172.21.15.172:0/3014181603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633864475d0 msgr2=0x56338645e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f4040ad4640 1 --2- 172.21.15.172:0/3014181603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633864475d0 0x56338645e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.745+0000 7f4040ad4640 1 -- 172.21.15.172:0/3014181603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56338644d560 con 0x563386448b40 2024-09-10T08:48:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.746+0000 7f4040ad4640 1 --2- 172.21.15.172:0/3014181603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386448b40 0x56338644e8d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x563386444ac0 tx=0x7f402c00bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefd378ec8a41407 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.746+0000 7f4026ffd640 1 -- 172.21.15.172:0/3014181603 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c02d070 con 0x563386448b40 2024-09-10T08:48:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.746+0000 7f4026ffd640 1 -- 172.21.15.172:0/3014181603 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f402c016040 con 0x563386448b40 2024-09-10T08:48:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.746+0000 7f4026ffd640 1 -- 172.21.15.172:0/3014181603 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c0286e0 con 0x563386448b40 2024-09-10T08:48:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.747+0000 7f4045912740 1 -- 172.21.15.172:0/3014181603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386448b40 msgr2=0x56338644e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.747+0000 7f4045912740 1 --2- 172.21.15.172:0/3014181603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386448b40 0x56338644e8d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x563386444ac0 tx=0x7f402c00bb50 comp rx=0 tx=0).stop 2024-09-10T08:48:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.747+0000 7f4045912740 1 -- 172.21.15.172:0/3014181603 shutdown_connections 2024-09-10T08:48:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.747+0000 7f4045912740 1 --2- 172.21.15.172:0/3014181603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563386452370 0x563386469d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.747+0000 7f4045912740 1 --2- 172.21.15.172:0/3014181603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386448b40 0x56338644e8d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.747+0000 7f4045912740 1 --2- 172.21.15.172:0/3014181603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633864475d0 0x56338645e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.747+0000 7f4045912740 1 -- 172.21.15.172:0/3014181603 >> 172.21.15.172:0/3014181603 conn(0x563386410310 msgr2=0x56338644d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.747+0000 7f4045912740 1 -- 172.21.15.172:0/3014181603 shutdown_connections 2024-09-10T08:48:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.747+0000 7f4045912740 1 -- 172.21.15.172:0/3014181603 wait complete. 2024-09-10T08:48:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.747+0000 7f4045912740 1 -- start start 2024-09-10T08:48:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.747+0000 7f4045912740 1 Processor -- start 2024-09-10T08:48:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4045912740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633864475d0 0x56338645a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4045912740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563386448b40 0x56338645ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4045912740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386452370 0x563386455bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4045912740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1b2230 con 0x563386452370 2024-09-10T08:48:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4045912740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1b1b60 con 0x563386448b40 2024-09-10T08:48:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4045912740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f403c1aeda0 con 0x5633864475d0 2024-09-10T08:48:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4045912740 10 client.0 _ll_register_callbacks cb 0x7f4038040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:12.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4040ad4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563386448b40 0x56338645ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f40412d5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633864475d0 0x56338645a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4041ad6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386452370 0x563386455bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f40412d5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633864475d0 0x56338645a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:34178/0 (socket says 172.21.15.172:34178) 2024-09-10T08:48:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4040ad4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563386448b40 0x56338645ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:47292/0 (socket says 172.21.15.172:47292) 2024-09-10T08:48:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f40412d5640 1 -- 172.21.15.172:0/2342028674 learned_addr learned my addr 172.21.15.172:0/2342028674 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4041ad6640 1 -- 172.21.15.172:0/2342028674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633864475d0 msgr2=0x56338645a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4041ad6640 1 --2- 172.21.15.172:0/2342028674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633864475d0 0x56338645a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4041ad6640 1 -- 172.21.15.172:0/2342028674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563386448b40 msgr2=0x56338645ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4041ad6640 1 --2- 172.21.15.172:0/2342028674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563386448b40 0x56338645ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4041ad6640 1 -- 172.21.15.172:0/2342028674 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563386456420 con 0x563386452370 2024-09-10T08:48:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4040ad4640 1 --2- 172.21.15.172:0/2342028674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563386448b40 0x56338645ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f40412d5640 1 --2- 172.21.15.172:0/2342028674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633864475d0 0x56338645a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:12.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4041ad6640 1 --2- 172.21.15.172:0/2342028674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386452370 0x563386455bf0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f40340058e0 tx=0x7f4034078b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be64cc1f1d99f018 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:12.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4024ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:12.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4024ff9640 1 -- 172.21.15.172:0/2342028674 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403401d040 con 0x563386452370 2024-09-10T08:48:12.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4024ff9640 1 -- 172.21.15.172:0/2342028674 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4034005560 con 0x563386452370 2024-09-10T08:48:12.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.748+0000 7f4045912740 1 -- 172.21.15.172:0/2342028674 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f402c016040 con 0x563386452370 2024-09-10T08:48:12.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.749+0000 7f4024ff9640 1 -- 172.21.15.172:0/2342028674 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4034052040 con 0x563386452370 2024-09-10T08:48:12.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.749+0000 7f4045912740 10 client.15942 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:12.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.749+0000 7f4045912740 1 -- 172.21.15.172:0/2342028674 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5633864cc910 con 0x563386452370 2024-09-10T08:48:12.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.749+0000 7f4024ff9640 1 -- 172.21.15.172:0/2342028674 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f402c016040 con 0x563386452370 2024-09-10T08:48:12.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.749+0000 7f4024ff9640 1 -- 172.21.15.172:0/2342028674 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f40340a3ed0 con 0x563386452370 2024-09-10T08:48:12.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.749+0000 7f4045912740 10 client.15942 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:12.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.749+0000 7f4045912740 10 client.15942 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:48:12.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.749+0000 7f4045912740 20 client.15942 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:12.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.749+0000 7f4045912740 1 --2- 172.21.15.172:0/2342028674 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5633864cd080 0x5633864cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.749+0000 7f4045912740 4 client.15942 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:48:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.749+0000 7f4045912740 1 -- 172.21.15.172:0/2342028674 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633863b4d70 con 0x5633864cd080 2024-09-10T08:48:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.749+0000 7f40412d5640 1 --2- 172.21.15.172:0/2342028674 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5633864cd080 0x5633864cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.750+0000 7f40412d5640 1 --2- 172.21.15.172:0/2342028674 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5633864cd080 0x5633864cf470 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9c9d41b03e14bab6 server_cookie=7c789565af4fe7d in_seq=0 out_seq=0 2024-09-10T08:48:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.750+0000 7f4024ff9640 10 client.15942 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:48:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.750+0000 7f4024ff9640 1 -- 172.21.15.172:0/2342028674 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633863b4d70 con 0x5633864cd080 2024-09-10T08:48:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.750+0000 7f4024ff9640 10 client.15942 handle_command_reply: tid=0 2024-09-10T08:48:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.750+0000 7f4045912740 1 client.15942 shutdown 2024-09-10T08:48:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.750+0000 7f4045912740 1 -- 172.21.15.172:0/2342028674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386452370 msgr2=0x563386455bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.750+0000 7f4045912740 1 --2- 172.21.15.172:0/2342028674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386452370 0x563386455bf0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f40340058e0 tx=0x7f4034078b30 comp rx=0 tx=0).stop 2024-09-10T08:48:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.750+0000 7f4045912740 1 -- 172.21.15.172:0/2342028674 shutdown_connections 2024-09-10T08:48:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.750+0000 7f4045912740 1 --2- 172.21.15.172:0/2342028674 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5633864cd080 0x5633864cf470 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.750+0000 7f4045912740 1 --2- 172.21.15.172:0/2342028674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563386452370 0x563386455bf0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.750+0000 7f4045912740 1 --2- 172.21.15.172:0/2342028674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563386448b40 0x56338645ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.750+0000 7f4045912740 1 --2- 172.21.15.172:0/2342028674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633864475d0 0x56338645a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.750+0000 7f4024ff9640 0 client.15942 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:48:12.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.750+0000 7f4045912740 1 -- 172.21.15.172:0/2342028674 >> 172.21.15.172:0/2342028674 conn(0x563386410310 msgr2=0x56338644d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:12.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.750+0000 7f4045912740 1 -- 172.21.15.172:0/2342028674 shutdown_connections 2024-09-10T08:48:12.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.750+0000 7f4045912740 1 -- 172.21.15.172:0/2342028674 wait complete. 2024-09-10T08:48:12.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.750+0000 7f4045912740 20 client.15942 trim_cache size 0 max 16384 2024-09-10T08:48:12.790 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:48:12.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.751+0000 7f4043539640 1 -- 172.21.15.172:0/2372719385 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f40280a2c00 msgr2=0x7f40280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:12.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.751+0000 7f4043539640 1 --2- 172.21.15.172:0/2372719385 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f40280a2c00 0x7f40280a50c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4038009900 tx=0x7f4038000bc0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.751+0000 7f4043539640 1 -- 172.21.15.172:0/2372719385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f403c106a90 msgr2=0x7f403c1030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:12.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.751+0000 7f4043539640 1 --2- 172.21.15.172:0/2372719385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f403c106a90 0x7f403c1030e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f403400cad0 tx=0x7f403400cf90 comp rx=0 tx=0).stop 2024-09-10T08:48:12.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.751+0000 7f4043539640 1 -- 172.21.15.172:0/2372719385 shutdown_connections 2024-09-10T08:48:12.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.751+0000 7f4043539640 1 --2- 172.21.15.172:0/2372719385 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f40280a2c00 0x7f40280a50c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.751+0000 7f4043539640 1 --2- 172.21.15.172:0/2372719385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f403c106a90 0x7f403c1030e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.751+0000 7f4043539640 1 --2- 172.21.15.172:0/2372719385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f403c101430 0x7f403c102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.751+0000 7f4043539640 1 --2- 172.21.15.172:0/2372719385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f403c100a90 0x7f403c102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:12.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.751+0000 7f4043539640 1 -- 172.21.15.172:0/2372719385 >> 172.21.15.172:0/2372719385 conn(0x7f403c0fc7d0 msgr2=0x7f403c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:12.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.751+0000 7f4043539640 1 -- 172.21.15.172:0/2372719385 shutdown_connections 2024-09-10T08:48:12.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:12.751+0000 7f4043539640 1 -- 172.21.15.172:0/2372719385 wait complete. 2024-09-10T08:48:13.512 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:48:13.556 INFO:tasks.workunit.client.0.smithi172.stderr:Updating files: 91% (11668/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-10T08:48:13.770 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:14.509 INFO:tasks.workunit.client.0.smithi172.stderr:Note: switching to '1cf819b7212a6048e92fc78c4eff963344f6d56b'. 2024-09-10T08:48:14.509 INFO:tasks.workunit.client.0.smithi172.stderr: 2024-09-10T08:48:14.509 INFO:tasks.workunit.client.0.smithi172.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T08:48:14.509 INFO:tasks.workunit.client.0.smithi172.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T08:48:14.509 INFO:tasks.workunit.client.0.smithi172.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T08:48:14.509 INFO:tasks.workunit.client.0.smithi172.stderr: 2024-09-10T08:48:14.509 INFO:tasks.workunit.client.0.smithi172.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T08:48:14.509 INFO:tasks.workunit.client.0.smithi172.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T08:48:14.509 INFO:tasks.workunit.client.0.smithi172.stderr: 2024-09-10T08:48:14.509 INFO:tasks.workunit.client.0.smithi172.stderr: git switch -c 2024-09-10T08:48:14.509 INFO:tasks.workunit.client.0.smithi172.stderr: 2024-09-10T08:48:14.509 INFO:tasks.workunit.client.0.smithi172.stderr:Or undo this operation with: 2024-09-10T08:48:14.510 INFO:tasks.workunit.client.0.smithi172.stderr: 2024-09-10T08:48:14.510 INFO:tasks.workunit.client.0.smithi172.stderr: git switch - 2024-09-10T08:48:14.510 INFO:tasks.workunit.client.0.smithi172.stderr: 2024-09-10T08:48:14.510 INFO:tasks.workunit.client.0.smithi172.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T08:48:14.510 INFO:tasks.workunit.client.0.smithi172.stderr: 2024-09-10T08:48:14.510 INFO:tasks.workunit.client.0.smithi172.stderr:HEAD is now at 1cf819b7212 Merge PR #59672 into wip-rishabh-testing-20240909.165402-squid 2024-09-10T08:48:14.510 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:14 smithi172 ceph-mon[30854]: pgmap v410: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:14.515 DEBUG:teuthology.orchestra.run.smithi172:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T08:48:14.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:14 smithi191 ceph-mon[35314]: pgmap v410: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:14.597 INFO:tasks.workunit.client.0.smithi172.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T08:48:14.599 INFO:tasks.workunit.client.0.smithi172.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T08:48:14.600 INFO:tasks.workunit.client.0.smithi172.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T08:48:14.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.591+0000 7f121866d640 1 -- 172.21.15.172:0/3279713442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1210100a60 msgr2=0x7f1210100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:14.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.591+0000 7f121866d640 1 --2- 172.21.15.172:0/3279713442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1210100a60 0x7f1210100ec0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1200009930 tx=0x7f120002a420 comp rx=0 tx=0).stop 2024-09-10T08:48:14.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.591+0000 7f121866d640 1 -- 172.21.15.172:0/3279713442 shutdown_connections 2024-09-10T08:48:14.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.591+0000 7f121866d640 1 --2- 172.21.15.172:0/3279713442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1210101400 0x7f121010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.591+0000 7f121866d640 1 --2- 172.21.15.172:0/3279713442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1210100a60 0x7f1210100ec0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.591+0000 7f121866d640 1 --2- 172.21.15.172:0/3279713442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1210106a60 0x7f1210106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.591+0000 7f121866d640 1 -- 172.21.15.172:0/3279713442 >> 172.21.15.172:0/3279713442 conn(0x7f12100fc780 msgr2=0x7f12100feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:14.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.591+0000 7f121866d640 1 -- 172.21.15.172:0/3279713442 shutdown_connections 2024-09-10T08:48:14.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.591+0000 7f121866d640 1 -- 172.21.15.172:0/3279713442 wait complete. 2024-09-10T08:48:14.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.591+0000 7f121866d640 1 Processor -- start 2024-09-10T08:48:14.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.591+0000 7f121866d640 1 -- start start 2024-09-10T08:48:14.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f121866d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1210100a60 0x7f1210198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f121866d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1210101400 0x7f1210198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f121866d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1210106a60 0x7f121019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f121866d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1210110db0 con 0x7f1210106a60 2024-09-10T08:48:14.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f121866d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1210110c30 con 0x7f1210101400 2024-09-10T08:48:14.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f121866d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1210110f30 con 0x7f1210100a60 2024-09-10T08:48:14.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f1216c0a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1210106a60 0x7f121019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:14.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f1216c0a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1210106a60 0x7f121019ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38814/0 (socket says 172.21.15.172:38814) 2024-09-10T08:48:14.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f1216c0a640 1 -- 172.21.15.172:0/3727513615 learned_addr learned my addr 172.21.15.172:0/3727513615 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:14.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f1215c08640 1 --2- 172.21.15.172:0/3727513615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1210101400 0x7f1210198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:14.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f1216c0a640 1 -- 172.21.15.172:0/3727513615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1210100a60 msgr2=0x7f1210198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:14.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f1216409640 1 --2- 172.21.15.172:0/3727513615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1210100a60 0x7f1210198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:14.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f1216c0a640 1 --2- 172.21.15.172:0/3727513615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1210100a60 0x7f1210198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f1216c0a640 1 -- 172.21.15.172:0/3727513615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1210101400 msgr2=0x7f1210198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:14.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f1216c0a640 1 --2- 172.21.15.172:0/3727513615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1210101400 0x7f1210198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f1216c0a640 1 -- 172.21.15.172:0/3727513615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121019d510 con 0x7f1210106a60 2024-09-10T08:48:14.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f1215c08640 1 --2- 172.21.15.172:0/3727513615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1210101400 0x7f1210198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:14.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f1216409640 1 --2- 172.21.15.172:0/3727513615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1210100a60 0x7f1210198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:14.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f1216c0a640 1 --2- 172.21.15.172:0/3727513615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1210106a60 0x7f121019ce30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f120800b970 tx=0x7f120800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:14.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f12077fe640 1 -- 172.21.15.172:0/3727513615 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208019070 con 0x7f1210106a60 2024-09-10T08:48:14.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f12077fe640 1 -- 172.21.15.172:0/3727513615 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1208014440 con 0x7f1210106a60 2024-09-10T08:48:14.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f121866d640 1 -- 172.21.15.172:0/3727513615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121019d800 con 0x7f1210106a60 2024-09-10T08:48:14.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f12077fe640 1 -- 172.21.15.172:0/3727513615 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12080075d0 con 0x7f1210106a60 2024-09-10T08:48:14.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.592+0000 7f121866d640 1 -- 172.21.15.172:0/3727513615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12101a50e0 con 0x7f1210106a60 2024-09-10T08:48:14.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.595+0000 7f12077fe640 1 -- 172.21.15.172:0/3727513615 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f120801c070 con 0x7f1210106a60 2024-09-10T08:48:14.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.595+0000 7f121aa46740 1 Processor -- start 2024-09-10T08:48:14.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.595+0000 7f121aa46740 1 -- start start 2024-09-10T08:48:14.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.595+0000 7f121aa46740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55636746db40 0x55636746c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.595+0000 7f121aa46740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 0x55636746ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.595+0000 7f121aa46740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556367477370 0x556367479760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.595+0000 7f121aa46740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5563673d22f0 con 0x55636746db40 2024-09-10T08:48:14.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.595+0000 7f121aa46740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556367379cf0 con 0x556367477370 2024-09-10T08:48:14.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.595+0000 7f121aa46740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5563673ce2b0 con 0x55636746c5d0 2024-09-10T08:48:14.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.595+0000 7f12077fe640 1 --2- 172.21.15.172:0/3727513615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11fc0a2b80 0x7f11fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.595+0000 7f1215c08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55636746db40 0x55636746c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:14.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.596+0000 7f1215c08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55636746db40 0x55636746c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38828/0 (socket says 172.21.15.172:38828) 2024-09-10T08:48:14.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.596+0000 7f1215c08640 1 -- 172.21.15.172:0/2417865414 learned_addr learned my addr 172.21.15.172:0/2417865414 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:14.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.596+0000 7f1216409640 1 --2- 172.21.15.172:0/2417865414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 0x55636746ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:14.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.596+0000 7f1215c08640 1 --2- 172.21.15.172:0/2417865414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556367477370 0x556367479760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:14.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.596+0000 7f1216c0a640 1 --2- 172.21.15.172:0/3727513615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11fc0a2b80 0x7f11fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:14.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.596+0000 7f1215c08640 1 -- 172.21.15.172:0/2417865414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 msgr2=0x55636746ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:14.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.596+0000 7f1215c08640 1 --2- 172.21.15.172:0/2417865414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 0x55636746ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.596+0000 7f1215c08640 1 -- 172.21.15.172:0/2417865414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556367477370 msgr2=0x556367479760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:14.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.596+0000 7f1215c08640 1 --2- 172.21.15.172:0/2417865414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556367477370 0x556367479760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.596+0000 7f1215c08640 1 -- 172.21.15.172:0/2417865414 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55636735e3a0 con 0x55636746db40 2024-09-10T08:48:14.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.596+0000 7f12077fe640 1 -- 172.21.15.172:0/3727513615 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f12080cea20 con 0x7f1210106a60 2024-09-10T08:48:14.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.596+0000 7f1215c08640 1 --2- 172.21.15.172:0/2417865414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55636746db40 0x55636746c090 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1200005ec0 tx=0x7f1200009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29433757d2c0736a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:14.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.596+0000 7f1216c0a640 1 --2- 172.21.15.172:0/3727513615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11fc0a2b80 0x7f11fc0a5040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f12080978f0 tx=0x7f12080c5ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:14.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.596+0000 7f1204ff9640 1 -- 172.21.15.172:0/2417865414 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200045070 con 0x55636746db40 2024-09-10T08:48:14.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.596+0000 7f1204ff9640 1 -- 172.21.15.172:0/2417865414 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1200005040 con 0x55636746db40 2024-09-10T08:48:14.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.596+0000 7f1204ff9640 1 -- 172.21.15.172:0/2417865414 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200035050 con 0x55636746db40 2024-09-10T08:48:14.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.597+0000 7f121aa46740 1 -- 172.21.15.172:0/2417865414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55636746db40 msgr2=0x55636746c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:14.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.597+0000 7f121aa46740 1 --2- 172.21.15.172:0/2417865414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55636746db40 0x55636746c090 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1200005ec0 tx=0x7f1200009560 comp rx=0 tx=0).stop 2024-09-10T08:48:14.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.597+0000 7f121aa46740 1 -- 172.21.15.172:0/2417865414 shutdown_connections 2024-09-10T08:48:14.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.597+0000 7f121aa46740 1 --2- 172.21.15.172:0/2417865414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556367477370 0x556367479760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.597+0000 7f121aa46740 1 --2- 172.21.15.172:0/2417865414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 0x55636746ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.597+0000 7f121aa46740 1 --2- 172.21.15.172:0/2417865414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55636746db40 0x55636746c090 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.597+0000 7f121aa46740 1 -- 172.21.15.172:0/2417865414 >> 172.21.15.172:0/2417865414 conn(0x556367435310 msgr2=0x556367468c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:14.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.597+0000 7f121aa46740 1 -- 172.21.15.172:0/2417865414 shutdown_connections 2024-09-10T08:48:14.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.597+0000 7f121aa46740 1 -- 172.21.15.172:0/2417865414 wait complete. 2024-09-10T08:48:14.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f121aa46740 1 -- start start 2024-09-10T08:48:14.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f121aa46740 1 Processor -- start 2024-09-10T08:48:14.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f121aa46740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 0x556367476de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f121aa46740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55636746db40 0x556367474090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f121aa46740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556367477370 0x5563674747d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f121aa46740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55636747a010 con 0x556367477370 2024-09-10T08:48:14.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f121aa46740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55636738d060 con 0x55636746db40 2024-09-10T08:48:14.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f121aa46740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55636747a190 con 0x55636746c5d0 2024-09-10T08:48:14.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f121aa46740 10 client.0 _ll_register_callbacks cb 0x5563670e08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:14.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f1215c08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556367477370 0x5563674747d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:14.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f1215c08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556367477370 0x5563674747d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38832/0 (socket says 172.21.15.172:38832) 2024-09-10T08:48:14.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f1215c08640 1 -- 172.21.15.172:0/316273052 learned_addr learned my addr 172.21.15.172:0/316273052 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:14.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f1215c08640 1 --2- 172.21.15.172:0/316273052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 0x556367476de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:14.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f1216409640 1 --2- 172.21.15.172:0/316273052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55636746db40 0x556367474090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:14.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f1215c08640 1 -- 172.21.15.172:0/316273052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 msgr2=0x556367476de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:14.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f1215c08640 1 --2- 172.21.15.172:0/316273052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 0x556367476de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f1215c08640 1 -- 172.21.15.172:0/316273052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55636746db40 msgr2=0x556367474090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:14.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f1215c08640 1 --2- 172.21.15.172:0/316273052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55636746db40 0x556367474090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.598+0000 7f1215c08640 1 -- 172.21.15.172:0/316273052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563671433c0 con 0x556367477370 2024-09-10T08:48:14.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.599+0000 7f1215c08640 1 --2- 172.21.15.172:0/316273052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556367477370 0x5563674747d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1200000c00 tx=0x7f1200004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc083e51328663d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:14.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.599+0000 7f11f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:14.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.599+0000 7f11f9ffb640 1 -- 172.21.15.172:0/316273052 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200035070 con 0x556367477370 2024-09-10T08:48:14.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.599+0000 7f11f9ffb640 1 -- 172.21.15.172:0/316273052 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f12000047c0 con 0x556367477370 2024-09-10T08:48:14.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.599+0000 7f121aa46740 1 -- 172.21.15.172:0/316273052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1200005040 con 0x556367477370 2024-09-10T08:48:14.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.599+0000 7f11f9ffb640 1 -- 172.21.15.172:0/316273052 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1200057050 con 0x556367477370 2024-09-10T08:48:14.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.599+0000 7f121aa46740 10 client.15954 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:14.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.599+0000 7f121aa46740 1 -- 172.21.15.172:0/316273052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556367485b40 con 0x556367477370 2024-09-10T08:48:14.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.599+0000 7f11f9ffb640 1 -- 172.21.15.172:0/316273052 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1200005040 con 0x556367477370 2024-09-10T08:48:14.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.599+0000 7f11f9ffb640 1 -- 172.21.15.172:0/316273052 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1200003ab0 con 0x556367477370 2024-09-10T08:48:14.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.600+0000 7f121aa46740 10 client.15954 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:14.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.600+0000 7f121aa46740 10 client.15954 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:48:14.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.600+0000 7f121aa46740 20 client.15954 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:14.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.600+0000 7f121aa46740 1 --2- 172.21.15.172:0/316273052 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x556367486e90 0x556367489280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.600+0000 7f121aa46740 4 client.15954 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:14.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.600+0000 7f121aa46740 1 -- 172.21.15.172:0/316273052 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563673df860 con 0x556367486e90 2024-09-10T08:48:14.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.600+0000 7f1216409640 1 --2- 172.21.15.172:0/316273052 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x556367486e90 0x556367489280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:14.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.600+0000 7f11f9ffb640 10 client.15954 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:48:14.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.600+0000 7f1216409640 1 --2- 172.21.15.172:0/316273052 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x556367486e90 0x556367489280 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c83422fc0cfc6eb1 server_cookie=3684daf73e74ad44 in_seq=0 out_seq=0 2024-09-10T08:48:14.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.601+0000 7f11f9ffb640 1 -- 172.21.15.172:0/316273052 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563673df860 con 0x556367486e90 2024-09-10T08:48:14.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.601+0000 7f11f9ffb640 10 client.15954 handle_command_reply: tid=0 2024-09-10T08:48:14.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.601+0000 7f121aa46740 1 client.15954 shutdown 2024-09-10T08:48:14.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.601+0000 7f121aa46740 1 -- 172.21.15.172:0/316273052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556367477370 msgr2=0x5563674747d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:14.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.601+0000 7f121aa46740 1 --2- 172.21.15.172:0/316273052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556367477370 0x5563674747d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1200000c00 tx=0x7f1200004580 comp rx=0 tx=0).stop 2024-09-10T08:48:14.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.601+0000 7f121aa46740 1 -- 172.21.15.172:0/316273052 shutdown_connections 2024-09-10T08:48:14.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.601+0000 7f121aa46740 1 --2- 172.21.15.172:0/316273052 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x556367486e90 0x556367489280 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.601+0000 7f11f9ffb640 0 client.15954 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:48:14.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.601+0000 7f121aa46740 1 --2- 172.21.15.172:0/316273052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556367477370 0x5563674747d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.601+0000 7f121aa46740 1 --2- 172.21.15.172:0/316273052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55636746db40 0x556367474090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.601+0000 7f121aa46740 1 --2- 172.21.15.172:0/316273052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 0x556367476de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.601+0000 7f121aa46740 1 -- 172.21.15.172:0/316273052 >> 172.21.15.172:0/316273052 conn(0x556367435310 msgr2=0x55636746e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:14.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.601+0000 7f121aa46740 1 -- 172.21.15.172:0/316273052 shutdown_connections 2024-09-10T08:48:14.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.601+0000 7f121aa46740 1 -- 172.21.15.172:0/316273052 wait complete. 2024-09-10T08:48:14.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.601+0000 7f121aa46740 20 client.15954 trim_cache size 0 max 16384 2024-09-10T08:48:14.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f121aa46740 1 Processor -- start 2024-09-10T08:48:14.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f121aa46740 1 -- start start 2024-09-10T08:48:14.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f121aa46740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 0x556367483430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f121aa46740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55636746db40 0x55636746f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f121aa46740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556367477370 0x55636746f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f121aa46740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f12101aeb30 con 0x55636746db40 2024-09-10T08:48:14.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f121aa46740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12101b0670 con 0x556367477370 2024-09-10T08:48:14.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f121aa46740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f12101af200 con 0x55636746c5d0 2024-09-10T08:48:14.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f1216409640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55636746db40 0x55636746f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:14.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f1216409640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55636746db40 0x55636746f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38836/0 (socket says 172.21.15.172:38836) 2024-09-10T08:48:14.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f1216409640 1 -- 172.21.15.172:0/4263510786 learned_addr learned my addr 172.21.15.172:0/4263510786 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:14.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f1215c08640 1 --2- 172.21.15.172:0/4263510786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 0x556367483430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:14.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f1216409640 1 -- 172.21.15.172:0/4263510786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 msgr2=0x556367483430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:14.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f1216409640 1 --2- 172.21.15.172:0/4263510786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 0x556367483430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f1216409640 1 -- 172.21.15.172:0/4263510786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556367477370 msgr2=0x55636746f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:14.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f1216409640 1 --2- 172.21.15.172:0/4263510786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556367477370 0x55636746f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f1216409640 1 -- 172.21.15.172:0/4263510786 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55636746fba0 con 0x55636746db40 2024-09-10T08:48:14.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.614+0000 7f1215c08640 1 --2- 172.21.15.172:0/4263510786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 0x556367483430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:14.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.615+0000 7f1216409640 1 --2- 172.21.15.172:0/4263510786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55636746db40 0x55636746f120 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x5563674756d0 tx=0x7f120c00a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c4086804f957b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:14.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.615+0000 7f11fbfff640 1 -- 172.21.15.172:0/4263510786 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c016770 con 0x55636746db40 2024-09-10T08:48:14.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.615+0000 7f11fbfff640 1 -- 172.21.15.172:0/4263510786 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f120c022070 con 0x55636746db40 2024-09-10T08:48:14.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.615+0000 7f11fbfff640 1 -- 172.21.15.172:0/4263510786 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c014d90 con 0x55636746db40 2024-09-10T08:48:14.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.615+0000 7f121aa46740 1 -- 172.21.15.172:0/4263510786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55636746db40 msgr2=0x55636746f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:14.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.615+0000 7f121aa46740 1 --2- 172.21.15.172:0/4263510786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55636746db40 0x55636746f120 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x5563674756d0 tx=0x7f120c00a7c0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.615+0000 7f121aa46740 1 -- 172.21.15.172:0/4263510786 shutdown_connections 2024-09-10T08:48:14.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.615+0000 7f121aa46740 1 --2- 172.21.15.172:0/4263510786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556367477370 0x55636746f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.615+0000 7f121aa46740 1 --2- 172.21.15.172:0/4263510786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55636746db40 0x55636746f120 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.615+0000 7f121aa46740 1 --2- 172.21.15.172:0/4263510786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 0x556367483430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.615+0000 7f121aa46740 1 -- 172.21.15.172:0/4263510786 >> 172.21.15.172:0/4263510786 conn(0x556367435310 msgr2=0x556367473180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:14.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.615+0000 7f121aa46740 1 -- 172.21.15.172:0/4263510786 shutdown_connections 2024-09-10T08:48:14.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.615+0000 7f121aa46740 1 -- 172.21.15.172:0/4263510786 wait complete. 2024-09-10T08:48:14.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f121aa46740 1 -- start start 2024-09-10T08:48:14.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f121aa46740 1 Processor -- start 2024-09-10T08:48:14.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f121aa46740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 0x55636748a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f121aa46740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55636746db40 0x55636748a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f121aa46740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556367477370 0x55636748aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.661 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:48:14.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f121aa46740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f12101affa0 con 0x556367477370 2024-09-10T08:48:14.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f121aa46740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12101af8d0 con 0x55636746db40 2024-09-10T08:48:14.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f121aa46740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f12101acb10 con 0x55636746c5d0 2024-09-10T08:48:14.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f121aa46740 10 client.0 _ll_register_callbacks cb 0x7f1200043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:14.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f1215c08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556367477370 0x55636748aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:14.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f1215c08640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556367477370 0x55636748aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38840/0 (socket says 172.21.15.172:38840) 2024-09-10T08:48:14.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f1215c08640 1 -- 172.21.15.172:0/1773544073 learned_addr learned my addr 172.21.15.172:0/1773544073 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:14.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f1216409640 1 --2- 172.21.15.172:0/1773544073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55636746db40 0x55636748a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:14.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f1215c08640 1 --2- 172.21.15.172:0/1773544073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 0x55636748a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:14.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f1215c08640 1 -- 172.21.15.172:0/1773544073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 msgr2=0x55636748a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:14.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f1215c08640 1 --2- 172.21.15.172:0/1773544073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 0x55636748a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f1215c08640 1 -- 172.21.15.172:0/1773544073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55636746db40 msgr2=0x55636748a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:14.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f1215c08640 1 --2- 172.21.15.172:0/1773544073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55636746db40 0x55636748a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.616+0000 7f1215c08640 1 -- 172.21.15.172:0/1773544073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563674f07d0 con 0x556367477370 2024-09-10T08:48:14.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.617+0000 7f1215c08640 1 --2- 172.21.15.172:0/1773544073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556367477370 0x55636748aab0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1200032c40 tx=0x7f1200009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45d36eb160f8b40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:14.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.617+0000 7f11f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:14.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.617+0000 7f11f9ffb640 1 -- 172.21.15.172:0/1773544073 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200035040 con 0x556367477370 2024-09-10T08:48:14.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.617+0000 7f11f9ffb640 1 -- 172.21.15.172:0/1773544073 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1200039a30 con 0x556367477370 2024-09-10T08:48:14.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.617+0000 7f121aa46740 1 -- 172.21.15.172:0/1773544073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120c022070 con 0x556367477370 2024-09-10T08:48:14.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.617+0000 7f11f9ffb640 1 -- 172.21.15.172:0/1773544073 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1200045070 con 0x556367477370 2024-09-10T08:48:14.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.617+0000 7f121aa46740 10 client.15960 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:14.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.617+0000 7f121aa46740 1 -- 172.21.15.172:0/1773544073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5563674f19d0 con 0x556367477370 2024-09-10T08:48:14.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.617+0000 7f11f9ffb640 1 -- 172.21.15.172:0/1773544073 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f120c022070 con 0x556367477370 2024-09-10T08:48:14.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.617+0000 7f11f9ffb640 1 -- 172.21.15.172:0/1773544073 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f120005a050 con 0x556367477370 2024-09-10T08:48:14.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.617+0000 7f121aa46740 10 client.15960 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:14.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.617+0000 7f121aa46740 10 client.15960 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:48:14.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.617+0000 7f121aa46740 20 client.15960 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:14.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.617+0000 7f121aa46740 1 --2- 172.21.15.172:0/1773544073 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x556367486e90 0x5563674f4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:14.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.617+0000 7f121aa46740 4 client.15960 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:48:14.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.617+0000 7f121aa46740 1 -- 172.21.15.172:0/1773544073 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563673d9d70 con 0x556367486e90 2024-09-10T08:48:14.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f1216409640 1 --2- 172.21.15.172:0/1773544073 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x556367486e90 0x5563674f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:14.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f11f9ffb640 10 client.15960 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:48:14.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f1216409640 1 --2- 172.21.15.172:0/1773544073 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x556367486e90 0x5563674f4070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e750a6db22b60bb6 server_cookie=1a91f26e5d3efc59 in_seq=0 out_seq=0 2024-09-10T08:48:14.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f11f9ffb640 1 -- 172.21.15.172:0/1773544073 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563673d9d70 con 0x556367486e90 2024-09-10T08:48:14.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f11f9ffb640 10 client.15960 handle_command_reply: tid=0 2024-09-10T08:48:14.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f121aa46740 1 client.15960 shutdown 2024-09-10T08:48:14.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f121aa46740 1 -- 172.21.15.172:0/1773544073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556367477370 msgr2=0x55636748aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:14.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f121aa46740 1 --2- 172.21.15.172:0/1773544073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556367477370 0x55636748aab0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1200032c40 tx=0x7f1200009560 comp rx=0 tx=0).stop 2024-09-10T08:48:14.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f121aa46740 1 -- 172.21.15.172:0/1773544073 shutdown_connections 2024-09-10T08:48:14.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f121aa46740 1 --2- 172.21.15.172:0/1773544073 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x556367486e90 0x5563674f4070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f121aa46740 1 --2- 172.21.15.172:0/1773544073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556367477370 0x55636748aab0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f11f9ffb640 0 client.15960 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:48:14.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f121aa46740 1 --2- 172.21.15.172:0/1773544073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55636746db40 0x55636748a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f121aa46740 1 --2- 172.21.15.172:0/1773544073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55636746c5d0 0x55636748a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f121aa46740 1 -- 172.21.15.172:0/1773544073 >> 172.21.15.172:0/1773544073 conn(0x556367435310 msgr2=0x5563674758f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:14.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f121aa46740 1 -- 172.21.15.172:0/1773544073 shutdown_connections 2024-09-10T08:48:14.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f121aa46740 1 -- 172.21.15.172:0/1773544073 wait complete. 2024-09-10T08:48:14.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.618+0000 7f121aa46740 20 client.15960 trim_cache size 0 max 16384 2024-09-10T08:48:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.619+0000 7f121866d640 1 -- 172.21.15.172:0/3727513615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11fc0a2b80 msgr2=0x7f11fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.619+0000 7f121866d640 1 --2- 172.21.15.172:0/3727513615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11fc0a2b80 0x7f11fc0a5040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f12080978f0 tx=0x7f12080c5ab0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.619+0000 7f121866d640 1 -- 172.21.15.172:0/3727513615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1210106a60 msgr2=0x7f121019ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.619+0000 7f121866d640 1 --2- 172.21.15.172:0/3727513615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1210106a60 0x7f121019ce30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f120800b970 tx=0x7f120800be30 comp rx=0 tx=0).stop 2024-09-10T08:48:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.619+0000 7f121866d640 1 -- 172.21.15.172:0/3727513615 shutdown_connections 2024-09-10T08:48:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.619+0000 7f121866d640 1 --2- 172.21.15.172:0/3727513615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11fc0a2b80 0x7f11fc0a5040 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.619+0000 7f121866d640 1 --2- 172.21.15.172:0/3727513615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1210106a60 0x7f121019ce30 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.619+0000 7f121866d640 1 --2- 172.21.15.172:0/3727513615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1210101400 0x7f1210198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.619+0000 7f121866d640 1 --2- 172.21.15.172:0/3727513615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1210100a60 0x7f1210198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:14.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.619+0000 7f121866d640 1 -- 172.21.15.172:0/3727513615 >> 172.21.15.172:0/3727513615 conn(0x7f12100fc780 msgr2=0x7f121006ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:14.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.619+0000 7f121866d640 1 -- 172.21.15.172:0/3727513615 shutdown_connections 2024-09-10T08:48:14.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:14.619+0000 7f121866d640 1 -- 172.21.15.172:0/3727513615 wait complete. 2024-09-10T08:48:14.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:14 smithi174 ceph-mon[34068]: pgmap v410: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:15.510 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:15 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:48:15.539 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:48:15.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:15 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:48:15.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:15 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:48:15.836 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:16.110 INFO:tasks.workunit.client.0.smithi172.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T08:48:16.212 INFO:tasks.workunit.client.0.smithi172.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T08:48:16.251 INFO:tasks.workunit.client.0.smithi172.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T08:48:16.253 INFO:tasks.workunit.client.0.smithi172.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T08:48:16.253 INFO:tasks.workunit.client.0.smithi172.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T08:48:16.294 INFO:tasks.workunit.client.0.smithi172.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T08:48:16.297 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T08:48:16.297 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T08:48:16.317 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-09-10T08:48:16.319 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-09-10T08:48:16.319 DEBUG:teuthology.orchestra.run.smithi172:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1cf819b7212a6048e92fc78c4eff963344f6d56b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-10T08:48:16.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.426+0000 7ff3f0b54640 1 -- 172.21.15.172:0/490571228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec100a80 msgr2=0x7ff3ec100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.426+0000 7ff3f0b54640 1 --2- 172.21.15.172:0/490571228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec100a80 0x7ff3ec100ee0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009930 tx=0x7ff3d402a420 comp rx=0 tx=0).stop 2024-09-10T08:48:16.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.426+0000 7ff3f0b54640 1 -- 172.21.15.172:0/490571228 shutdown_connections 2024-09-10T08:48:16.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.426+0000 7ff3f0b54640 1 --2- 172.21.15.172:0/490571228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3ec101420 0x7ff3ec10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.426+0000 7ff3f0b54640 1 --2- 172.21.15.172:0/490571228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec100a80 0x7ff3ec100ee0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.426+0000 7ff3f0b54640 1 --2- 172.21.15.172:0/490571228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3ec106a80 0x7ff3ec106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.426+0000 7ff3f0b54640 1 -- 172.21.15.172:0/490571228 >> 172.21.15.172:0/490571228 conn(0x7ff3ec0fc7a0 msgr2=0x7ff3ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:16.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.426+0000 7ff3f0b54640 1 -- 172.21.15.172:0/490571228 shutdown_connections 2024-09-10T08:48:16.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.426+0000 7ff3f0b54640 1 -- 172.21.15.172:0/490571228 wait complete. 2024-09-10T08:48:16.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.427+0000 7ff3f0b54640 1 Processor -- start 2024-09-10T08:48:16.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.427+0000 7ff3f0b54640 1 -- start start 2024-09-10T08:48:16.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.427+0000 7ff3f0b54640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec100a80 0x7ff3ec198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.427+0000 7ff3f0b54640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3ec101420 0x7ff3ec198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.427+0000 7ff3f0b54640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3ec106a80 0x7ff3ec19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.427+0000 7ff3f0b54640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec110d30 con 0x7ff3ec100a80 2024-09-10T08:48:16.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.427+0000 7ff3f0b54640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec110bb0 con 0x7ff3ec101420 2024-09-10T08:48:16.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.427+0000 7ff3f0b54640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec110eb0 con 0x7ff3ec106a80 2024-09-10T08:48:16.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.427+0000 7ff3ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec100a80 0x7ff3ec198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.427+0000 7ff3ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec100a80 0x7ff3ec198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55454/0 (socket says 172.21.15.172:55454) 2024-09-10T08:48:16.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.427+0000 7ff3ea59c640 1 -- 172.21.15.172:0/2703512641 learned_addr learned my addr 172.21.15.172:0/2703512641 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:16.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.427+0000 7ff3ead9d640 1 --2- 172.21.15.172:0/2703512641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3ec106a80 0x7ff3ec19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.427+0000 7ff3e9d9b640 1 --2- 172.21.15.172:0/2703512641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3ec101420 0x7ff3ec198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.428+0000 7ff3ea59c640 1 -- 172.21.15.172:0/2703512641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3ec106a80 msgr2=0x7ff3ec19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.428+0000 7ff3ea59c640 1 --2- 172.21.15.172:0/2703512641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3ec106a80 0x7ff3ec19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.428+0000 7ff3ea59c640 1 -- 172.21.15.172:0/2703512641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3ec101420 msgr2=0x7ff3ec198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.428+0000 7ff3ea59c640 1 --2- 172.21.15.172:0/2703512641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3ec101420 0x7ff3ec198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.428+0000 7ff3ea59c640 1 -- 172.21.15.172:0/2703512641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ec19d3c0 con 0x7ff3ec100a80 2024-09-10T08:48:16.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.428+0000 7ff3ead9d640 1 --2- 172.21.15.172:0/2703512641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3ec106a80 0x7ff3ec19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:16.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.428+0000 7ff3e9d9b640 1 --2- 172.21.15.172:0/2703512641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3ec101420 0x7ff3ec198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:16.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.428+0000 7ff3ea59c640 1 --2- 172.21.15.172:0/2703512641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec100a80 0x7ff3ec198410 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00e9f0 tx=0x7ff3dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:16.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.428+0000 7ff3e37fe640 1 -- 172.21.15.172:0/2703512641 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc00cd80 con 0x7ff3ec100a80 2024-09-10T08:48:16.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.428+0000 7ff3f0b54640 1 -- 172.21.15.172:0/2703512641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ec19d6b0 con 0x7ff3ec100a80 2024-09-10T08:48:16.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.428+0000 7ff3e37fe640 1 -- 172.21.15.172:0/2703512641 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3dc0054d0 con 0x7ff3ec100a80 2024-09-10T08:48:16.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.428+0000 7ff3e37fe640 1 -- 172.21.15.172:0/2703512641 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc010490 con 0x7ff3ec100a80 2024-09-10T08:48:16.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.428+0000 7ff3f0b54640 1 -- 172.21.15.172:0/2703512641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ec1a4f90 con 0x7ff3ec100a80 2024-09-10T08:48:16.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.430+0000 7ff3e37fe640 1 -- 172.21.15.172:0/2703512641 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff3dc005020 con 0x7ff3ec100a80 2024-09-10T08:48:16.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.430+0000 7ff3f2f2d740 1 Processor -- start 2024-09-10T08:48:16.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3f2f2d740 1 -- start start 2024-09-10T08:48:16.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3f2f2d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf08612b40 0x55bf08611090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3f2f2d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf086115d0 0x55bf08611a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3f2f2d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf0861c370 0x55bf0861e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3f2f2d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bf085772f0 con 0x55bf08612b40 2024-09-10T08:48:16.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3f2f2d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf0851ecf0 con 0x55bf0861c370 2024-09-10T08:48:16.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3f2f2d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bf085732b0 con 0x55bf086115d0 2024-09-10T08:48:16.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3ead9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf08612b40 0x55bf08611090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3ead9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf08612b40 0x55bf08611090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55460/0 (socket says 172.21.15.172:55460) 2024-09-10T08:48:16.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3ead9d640 1 -- 172.21.15.172:0/3917022102 learned_addr learned my addr 172.21.15.172:0/3917022102 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:16.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3e37fe640 1 --2- 172.21.15.172:0/2703512641 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff3d80a2b80 0x7ff3d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3ea59c640 1 --2- 172.21.15.172:0/3917022102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf086115d0 0x55bf08611a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3e9d9b640 1 --2- 172.21.15.172:0/3917022102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf0861c370 0x55bf0861e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3ead9d640 1 -- 172.21.15.172:0/3917022102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf086115d0 msgr2=0x55bf08611a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:16.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3ead9d640 1 --2- 172.21.15.172:0/3917022102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf086115d0 0x55bf08611a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3ead9d640 1 -- 172.21.15.172:0/3917022102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf0861c370 msgr2=0x55bf0861e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:16.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3ead9d640 1 --2- 172.21.15.172:0/3917022102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf0861c370 0x55bf0861e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3ead9d640 1 -- 172.21.15.172:0/3917022102 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf085033a0 con 0x55bf08612b40 2024-09-10T08:48:16.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3e9d9b640 1 --2- 172.21.15.172:0/3917022102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf0861c370 0x55bf0861e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:16.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3e37fe640 1 -- 172.21.15.172:0/2703512641 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff3dc0cf800 con 0x7ff3ec100a80 2024-09-10T08:48:16.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3ead9d640 1 --2- 172.21.15.172:0/2703512641 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff3d80a2b80 0x7ff3d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.431+0000 7ff3ead9d640 1 --2- 172.21.15.172:0/3917022102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf08612b40 0x55bf08611090 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400f530 tx=0x7ff3e400f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dca02ab5bc4e50f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:16.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.432+0000 7ff3e0ff9640 1 -- 172.21.15.172:0/3917022102 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4021070 con 0x55bf08612b40 2024-09-10T08:48:16.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.432+0000 7ff3e0ff9640 1 -- 172.21.15.172:0/3917022102 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3e401cdb0 con 0x55bf08612b40 2024-09-10T08:48:16.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.432+0000 7ff3e0ff9640 1 -- 172.21.15.172:0/3917022102 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e400dc90 con 0x55bf08612b40 2024-09-10T08:48:16.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.432+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3917022102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf08612b40 msgr2=0x55bf08611090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.432+0000 7ff3ead9d640 1 --2- 172.21.15.172:0/2703512641 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff3d80a2b80 0x7ff3d80a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff3e401e000 tx=0x7ff3e401c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:16.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.432+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/3917022102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf08612b40 0x55bf08611090 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400f530 tx=0x7ff3e400f9f0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3917022102 shutdown_connections 2024-09-10T08:48:16.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/3917022102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf0861c370 0x55bf0861e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/3917022102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf086115d0 0x55bf08611a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/3917022102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf08612b40 0x55bf08611090 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3917022102 >> 172.21.15.172:0/3917022102 conn(0x55bf085da310 msgr2=0x55bf0860dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:16.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3917022102 shutdown_connections 2024-09-10T08:48:16.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3917022102 wait complete. 2024-09-10T08:48:16.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3f2f2d740 1 -- start start 2024-09-10T08:48:16.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3f2f2d740 1 Processor -- start 2024-09-10T08:48:16.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3f2f2d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf086115d0 0x55bf0861bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3f2f2d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf08612b40 0x55bf08619020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3f2f2d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf0861c370 0x55bf08619600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3f2f2d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bf0861f070 con 0x55bf0861c370 2024-09-10T08:48:16.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3f2f2d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf08532060 con 0x55bf08612b40 2024-09-10T08:48:16.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3f2f2d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bf0861f1f0 con 0x55bf086115d0 2024-09-10T08:48:16.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3f2f2d740 10 client.0 _ll_register_callbacks cb 0x55bf082858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:16.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3ead9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf0861c370 0x55bf08619600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3ead9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf0861c370 0x55bf08619600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55466/0 (socket says 172.21.15.172:55466) 2024-09-10T08:48:16.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.433+0000 7ff3ead9d640 1 -- 172.21.15.172:0/3523316993 learned_addr learned my addr 172.21.15.172:0/3523316993 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:16.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.434+0000 7ff3e9d9b640 1 --2- 172.21.15.172:0/3523316993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf08612b40 0x55bf08619020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.434+0000 7ff3ea59c640 1 --2- 172.21.15.172:0/3523316993 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf086115d0 0x55bf0861bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.434+0000 7ff3ead9d640 1 -- 172.21.15.172:0/3523316993 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf086115d0 msgr2=0x55bf0861bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:16.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.434+0000 7ff3ead9d640 1 --2- 172.21.15.172:0/3523316993 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf086115d0 0x55bf0861bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.434+0000 7ff3ead9d640 1 -- 172.21.15.172:0/3523316993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf08612b40 msgr2=0x55bf08619020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:16.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.434+0000 7ff3ead9d640 1 --2- 172.21.15.172:0/3523316993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf08612b40 0x55bf08619020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.434+0000 7ff3ead9d640 1 -- 172.21.15.172:0/3523316993 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf082e83c0 con 0x55bf0861c370 2024-09-10T08:48:16.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.434+0000 7ff3ead9d640 1 --2- 172.21.15.172:0/3523316993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf0861c370 0x55bf08619600 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40124f0 tx=0x7ff3e40098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995340e5ec48dd4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:16.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.434+0000 7ff3d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:16.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.434+0000 7ff3d1ffb640 1 -- 172.21.15.172:0/3523316993 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4032060 con 0x55bf0861c370 2024-09-10T08:48:16.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.434+0000 7ff3d1ffb640 1 -- 172.21.15.172:0/3523316993 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff3e401c880 con 0x55bf0861c370 2024-09-10T08:48:16.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.434+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3523316993 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3e401cdb0 con 0x55bf0861c370 2024-09-10T08:48:16.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.434+0000 7ff3d1ffb640 1 -- 172.21.15.172:0/3523316993 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff3e4039050 con 0x55bf0861c370 2024-09-10T08:48:16.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.434+0000 7ff3f2f2d740 10 client.15972 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:16.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.434+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3523316993 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bf0862ab00 con 0x55bf0861c370 2024-09-10T08:48:16.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.434+0000 7ff3d1ffb640 1 -- 172.21.15.172:0/3523316993 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3e401cdb0 con 0x55bf0861c370 2024-09-10T08:48:16.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.435+0000 7ff3d1ffb640 1 -- 172.21.15.172:0/3523316993 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff3e4005050 con 0x55bf0861c370 2024-09-10T08:48:16.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.435+0000 7ff3f2f2d740 10 client.15972 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:16.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.435+0000 7ff3f2f2d740 10 client.15972 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:48:16.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.435+0000 7ff3f2f2d740 20 client.15972 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:16.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.435+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/3523316993 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55bf0862beb0 0x55bf0862e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.435+0000 7ff3f2f2d740 4 client.15972 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:16.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.435+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3523316993 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf08584860 con 0x55bf0862beb0 2024-09-10T08:48:16.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.435+0000 7ff3ea59c640 1 --2- 172.21.15.172:0/3523316993 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55bf0862beb0 0x55bf0862e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.435+0000 7ff3d1ffb640 10 client.15972 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:48:16.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.435+0000 7ff3ea59c640 1 --2- 172.21.15.172:0/3523316993 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55bf0862beb0 0x55bf0862e2a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d9f0b8572e11250 server_cookie=c909d1ea23454db in_seq=0 out_seq=0 2024-09-10T08:48:16.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.436+0000 7ff3d1ffb640 1 -- 172.21.15.172:0/3523316993 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf08584860 con 0x55bf0862beb0 2024-09-10T08:48:16.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.436+0000 7ff3d1ffb640 10 client.15972 handle_command_reply: tid=0 2024-09-10T08:48:16.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.436+0000 7ff3f2f2d740 1 client.15972 shutdown 2024-09-10T08:48:16.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.436+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3523316993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf0861c370 msgr2=0x55bf08619600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.436+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/3523316993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf0861c370 0x55bf08619600 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40124f0 tx=0x7ff3e40098c0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.436+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3523316993 shutdown_connections 2024-09-10T08:48:16.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.436+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/3523316993 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55bf0862beb0 0x55bf0862e2a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.436+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/3523316993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf0861c370 0x55bf08619600 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.436+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/3523316993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf08612b40 0x55bf08619020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.436+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/3523316993 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf086115d0 0x55bf0861bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.436+0000 7ff3d1ffb640 0 client.15972 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:48:16.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.436+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3523316993 >> 172.21.15.172:0/3523316993 conn(0x55bf085da310 msgr2=0x55bf08613960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:16.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.436+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3523316993 shutdown_connections 2024-09-10T08:48:16.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.436+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3523316993 wait complete. 2024-09-10T08:48:16.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.436+0000 7ff3f2f2d740 20 client.15972 trim_cache size 0 max 16384 2024-09-10T08:48:16.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.449+0000 7ff3f2f2d740 1 Processor -- start 2024-09-10T08:48:16.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.449+0000 7ff3f2f2d740 1 -- start start 2024-09-10T08:48:16.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3f2f2d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf086115d0 0x55bf08628310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3f2f2d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf08612b40 0x55bf086188e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3f2f2d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0861c370 0x55bf08633a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3f2f2d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec1aeb60 con 0x55bf086115d0 2024-09-10T08:48:16.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3f2f2d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec1b06a0 con 0x55bf08612b40 2024-09-10T08:48:16.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3f2f2d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec1af230 con 0x55bf0861c370 2024-09-10T08:48:16.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf086115d0 0x55bf08628310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf086115d0 0x55bf08628310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55468/0 (socket says 172.21.15.172:55468) 2024-09-10T08:48:16.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3ea59c640 1 -- 172.21.15.172:0/814630930 learned_addr learned my addr 172.21.15.172:0/814630930 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:16.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3e9d9b640 1 --2- 172.21.15.172:0/814630930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf08612b40 0x55bf086188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3ead9d640 1 --2- 172.21.15.172:0/814630930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0861c370 0x55bf08633a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3ea59c640 1 -- 172.21.15.172:0/814630930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0861c370 msgr2=0x55bf08633a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:16.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3ea59c640 1 --2- 172.21.15.172:0/814630930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0861c370 0x55bf08633a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3ea59c640 1 -- 172.21.15.172:0/814630930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf08612b40 msgr2=0x55bf086188e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:16.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3ea59c640 1 --2- 172.21.15.172:0/814630930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf08612b40 0x55bf086188e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3ea59c640 1 -- 172.21.15.172:0/814630930 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf0860d4f0 con 0x55bf086115d0 2024-09-10T08:48:16.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3e9d9b640 1 --2- 172.21.15.172:0/814630930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf08612b40 0x55bf086188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:16.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3ea59c640 1 --2- 172.21.15.172:0/814630930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf086115d0 0x55bf08628310 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00e9c0 tx=0x7ff3dc0980d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc3297fb8aa556f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:16.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3d3fff640 1 -- 172.21.15.172:0/814630930 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc09c400 con 0x55bf086115d0 2024-09-10T08:48:16.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3d3fff640 1 -- 172.21.15.172:0/814630930 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3dc09bd00 con 0x55bf086115d0 2024-09-10T08:48:16.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.450+0000 7ff3d3fff640 1 -- 172.21.15.172:0/814630930 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc016db0 con 0x55bf086115d0 2024-09-10T08:48:16.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.451+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/814630930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf086115d0 msgr2=0x55bf08628310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.451+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/814630930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf086115d0 0x55bf08628310 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00e9c0 tx=0x7ff3dc0980d0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.451+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/814630930 shutdown_connections 2024-09-10T08:48:16.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.451+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/814630930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf0861c370 0x55bf08633a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.451+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/814630930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf08612b40 0x55bf086188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.451+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/814630930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf086115d0 0x55bf08628310 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.451+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/814630930 >> 172.21.15.172:0/814630930 conn(0x55bf085da310 msgr2=0x55bf0862e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:16.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.451+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/814630930 shutdown_connections 2024-09-10T08:48:16.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.451+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/814630930 wait complete. 2024-09-10T08:48:16.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.451+0000 7ff3f2f2d740 1 -- start start 2024-09-10T08:48:16.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3f2f2d740 1 Processor -- start 2024-09-10T08:48:16.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3f2f2d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf086115d0 0x55bf0861fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3f2f2d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf08612b40 0x55bf08620120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3f2f2d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf0861c370 0x55bf08698de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3f2f2d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec1affd0 con 0x55bf0861c370 2024-09-10T08:48:16.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3f2f2d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec1af900 con 0x55bf08612b40 2024-09-10T08:48:16.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3f2f2d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec1acb40 con 0x55bf086115d0 2024-09-10T08:48:16.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3f2f2d740 10 client.0 _ll_register_callbacks cb 0x7ff3e401f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:16.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3ead9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf0861c370 0x55bf08698de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3ead9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf0861c370 0x55bf08698de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55474/0 (socket says 172.21.15.172:55474) 2024-09-10T08:48:16.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3ead9d640 1 -- 172.21.15.172:0/3338613265 learned_addr learned my addr 172.21.15.172:0/3338613265 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:16.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3ea59c640 1 --2- 172.21.15.172:0/3338613265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf086115d0 0x55bf0861fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3ead9d640 1 -- 172.21.15.172:0/3338613265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf086115d0 msgr2=0x55bf0861fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:16.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3ead9d640 1 --2- 172.21.15.172:0/3338613265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf086115d0 0x55bf0861fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3e9d9b640 1 --2- 172.21.15.172:0/3338613265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf08612b40 0x55bf08620120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3ead9d640 1 -- 172.21.15.172:0/3338613265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf08612b40 msgr2=0x55bf08620120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:16.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3ead9d640 1 --2- 172.21.15.172:0/3338613265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf08612b40 0x55bf08620120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3ead9d640 1 -- 172.21.15.172:0/3338613265 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf08699420 con 0x55bf0861c370 2024-09-10T08:48:16.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.452+0000 7ff3e9d9b640 1 --2- 172.21.15.172:0/3338613265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf08612b40 0x55bf08620120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:16.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.453+0000 7ff3ead9d640 1 --2- 172.21.15.172:0/3338613265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf0861c370 0x55bf08698de0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40055c0 tx=0x7ff3e4005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5cf992a498cc81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:16.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.453+0000 7ff3d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:16.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.453+0000 7ff3d1ffb640 1 -- 172.21.15.172:0/3338613265 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4032040 con 0x55bf0861c370 2024-09-10T08:48:16.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.453+0000 7ff3d1ffb640 1 -- 172.21.15.172:0/3338613265 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff3e400da60 con 0x55bf0861c370 2024-09-10T08:48:16.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.453+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3338613265 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3dc09bd00 con 0x55bf0861c370 2024-09-10T08:48:16.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.453+0000 7ff3d1ffb640 1 -- 172.21.15.172:0/3338613265 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff3e4021070 con 0x55bf0861c370 2024-09-10T08:48:16.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.453+0000 7ff3f2f2d740 10 client.15978 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:16.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.453+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3338613265 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bf0869a720 con 0x55bf0861c370 2024-09-10T08:48:16.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.453+0000 7ff3d1ffb640 1 -- 172.21.15.172:0/3338613265 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3dc09bd00 con 0x55bf0861c370 2024-09-10T08:48:16.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.453+0000 7ff3d1ffb640 1 -- 172.21.15.172:0/3338613265 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff3e403c070 con 0x55bf0861c370 2024-09-10T08:48:16.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.453+0000 7ff3f2f2d740 10 client.15978 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:16.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.453+0000 7ff3f2f2d740 10 client.15978 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:48:16.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.453+0000 7ff3f2f2d740 20 client.15978 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:16.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.453+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/3338613265 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55bf0862beb0 0x55bf0869cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.453+0000 7ff3f2f2d740 4 client.15978 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:48:16.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.453+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3338613265 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf0857ed70 con 0x55bf0862beb0 2024-09-10T08:48:16.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3ea59c640 1 --2- 172.21.15.172:0/3338613265 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55bf0862beb0 0x55bf0869cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3d1ffb640 10 client.15978 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:48:16.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3ea59c640 1 --2- 172.21.15.172:0/3338613265 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55bf0862beb0 0x55bf0869cde0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c856740c267e3c3 server_cookie=8acd80d248550c9f in_seq=0 out_seq=0 2024-09-10T08:48:16.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3d1ffb640 1 -- 172.21.15.172:0/3338613265 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf0857ed70 con 0x55bf0862beb0 2024-09-10T08:48:16.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3d1ffb640 10 client.15978 handle_command_reply: tid=0 2024-09-10T08:48:16.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3f2f2d740 1 client.15978 shutdown 2024-09-10T08:48:16.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3338613265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf0861c370 msgr2=0x55bf08698de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/3338613265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf0861c370 0x55bf08698de0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40055c0 tx=0x7ff3e4005aa0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3338613265 shutdown_connections 2024-09-10T08:48:16.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/3338613265 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55bf0862beb0 0x55bf0869cde0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/3338613265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf0861c370 0x55bf08698de0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/3338613265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf08612b40 0x55bf08620120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3f2f2d740 1 --2- 172.21.15.172:0/3338613265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf086115d0 0x55bf0861fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3338613265 >> 172.21.15.172:0/3338613265 conn(0x55bf085da310 msgr2=0x55bf0861a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:16.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3d1ffb640 0 client.15978 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:48:16.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3338613265 shutdown_connections 2024-09-10T08:48:16.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3f2f2d740 1 -- 172.21.15.172:0/3338613265 wait complete. 2024-09-10T08:48:16.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.454+0000 7ff3f2f2d740 20 client.15978 trim_cache size 0 max 16384 2024-09-10T08:48:16.466 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:48:16.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.455+0000 7ff3f0b54640 1 -- 172.21.15.172:0/2703512641 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff3d80a2b80 msgr2=0x7ff3d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.455+0000 7ff3f0b54640 1 --2- 172.21.15.172:0/2703512641 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff3d80a2b80 0x7ff3d80a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff3e401e000 tx=0x7ff3e401c000 comp rx=0 tx=0).stop 2024-09-10T08:48:16.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.455+0000 7ff3f0b54640 1 -- 172.21.15.172:0/2703512641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec100a80 msgr2=0x7ff3ec198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.455+0000 7ff3f0b54640 1 --2- 172.21.15.172:0/2703512641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec100a80 0x7ff3ec198410 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00e9f0 tx=0x7ff3dc00edf0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.455+0000 7ff3f0b54640 1 -- 172.21.15.172:0/2703512641 shutdown_connections 2024-09-10T08:48:16.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.455+0000 7ff3f0b54640 1 --2- 172.21.15.172:0/2703512641 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff3d80a2b80 0x7ff3d80a5040 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.455+0000 7ff3f0b54640 1 --2- 172.21.15.172:0/2703512641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3ec106a80 0x7ff3ec19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.455+0000 7ff3f0b54640 1 --2- 172.21.15.172:0/2703512641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3ec101420 0x7ff3ec198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.455+0000 7ff3f0b54640 1 --2- 172.21.15.172:0/2703512641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec100a80 0x7ff3ec198410 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.455+0000 7ff3f0b54640 1 -- 172.21.15.172:0/2703512641 >> 172.21.15.172:0/2703512641 conn(0x7ff3ec0fc7a0 msgr2=0x7ff3ec06ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:16.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.455+0000 7ff3f0b54640 1 -- 172.21.15.172:0/2703512641 shutdown_connections 2024-09-10T08:48:16.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:16.455+0000 7ff3f0b54640 1 -- 172.21.15.172:0/2703512641 wait complete. 2024-09-10T08:48:16.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:16 smithi191 ceph-mon[35314]: pgmap v411: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:16.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:16 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:48:16.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:16 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:48:16.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:16 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:48:16.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:16 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:48:16.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:16 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:48:16.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:16 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:48:16.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:16 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:48:16.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:16 smithi172 ceph-mon[30854]: pgmap v411: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:16.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:16 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:48:16.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:16 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:48:16.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:16 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:48:16.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:16 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:48:16.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:16 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:48:16.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:16 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:48:16.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:16 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:48:16.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:16 smithi174 ceph-mon[34068]: pgmap v411: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T08:48:16.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:16 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:48:16.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:16 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:48:16.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:16 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:48:16.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:16 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:48:16.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:16 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:48:16.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:16 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:48:16.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:16 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:48:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.822+0000 7fb4217a2640 1 -- 172.21.15.172:0/420344123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41c100870 msgr2=0x7fb41c100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.822+0000 7fb4217a2640 1 --2- 172.21.15.172:0/420344123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41c100870 0x7fb41c100c70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009900 tx=0x7fb40c02a410 comp rx=0 tx=0).stop 2024-09-10T08:48:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.822+0000 7fb4217a2640 1 -- 172.21.15.172:0/420344123 shutdown_connections 2024-09-10T08:48:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.822+0000 7fb4217a2640 1 --2- 172.21.15.172:0/420344123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb41c102420 0x7fb41c108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.822+0000 7fb4217a2640 1 --2- 172.21.15.172:0/420344123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb41c101a60 0x7fb41c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.822+0000 7fb4217a2640 1 --2- 172.21.15.172:0/420344123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41c100870 0x7fb41c100c70 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.822+0000 7fb4217a2640 1 -- 172.21.15.172:0/420344123 >> 172.21.15.172:0/420344123 conn(0x7fb41c0fbfd0 msgr2=0x7fb41c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.822+0000 7fb4217a2640 1 -- 172.21.15.172:0/420344123 shutdown_connections 2024-09-10T08:48:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.822+0000 7fb4217a2640 1 -- 172.21.15.172:0/420344123 wait complete. 2024-09-10T08:48:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb4217a2640 1 Processor -- start 2024-09-10T08:48:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb4217a2640 1 -- start start 2024-09-10T08:48:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb4217a2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb41c100870 0x7fb41c197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb4217a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb41c101a60 0x7fb41c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb4217a2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41c102420 0x7fb41c19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb4217a2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c10c3a0 con 0x7fb41c102420 2024-09-10T08:48:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb4217a2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c10c220 con 0x7fb41c101a60 2024-09-10T08:48:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb4217a2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c10c520 con 0x7fb41c100870 2024-09-10T08:48:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb420fa1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41c102420 0x7fb41c19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb420fa1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41c102420 0x7fb41c19f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55496/0 (socket says 172.21.15.172:55496) 2024-09-10T08:48:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb420fa1640 1 -- 172.21.15.172:0/671542732 learned_addr learned my addr 172.21.15.172:0/671542732 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb41b7fe640 1 --2- 172.21.15.172:0/671542732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb41c101a60 0x7fb41c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb41bfff640 1 --2- 172.21.15.172:0/671542732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb41c100870 0x7fb41c197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb420fa1640 1 -- 172.21.15.172:0/671542732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb41c100870 msgr2=0x7fb41c197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb420fa1640 1 --2- 172.21.15.172:0/671542732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb41c100870 0x7fb41c197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb420fa1640 1 -- 172.21.15.172:0/671542732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb41c101a60 msgr2=0x7fb41c198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb420fa1640 1 --2- 172.21.15.172:0/671542732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb41c101a60 0x7fb41c198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.823+0000 7fb420fa1640 1 -- 172.21.15.172:0/671542732 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41c19f800 con 0x7fb41c102420 2024-09-10T08:48:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.824+0000 7fb420fa1640 1 --2- 172.21.15.172:0/671542732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41c102420 0x7fb41c19f190 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb41400cad0 tx=0x7fb41400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.824+0000 7fb4197fa640 1 -- 172.21.15.172:0/671542732 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41400ae20 con 0x7fb41c102420 2024-09-10T08:48:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.824+0000 7fb4197fa640 1 -- 172.21.15.172:0/671542732 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb414005020 con 0x7fb41c102420 2024-09-10T08:48:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.824+0000 7fb4217a2640 1 -- 172.21.15.172:0/671542732 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41c19faf0 con 0x7fb41c102420 2024-09-10T08:48:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.824+0000 7fb4197fa640 1 -- 172.21.15.172:0/671542732 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41400f6b0 con 0x7fb41c102420 2024-09-10T08:48:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.824+0000 7fb4217a2640 1 -- 172.21.15.172:0/671542732 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41c1a0090 con 0x7fb41c102420 2024-09-10T08:48:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.826+0000 7fb4197fa640 1 -- 172.21.15.172:0/671542732 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb4140051d0 con 0x7fb41c102420 2024-09-10T08:48:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.826+0000 7fb425879740 1 Processor -- start 2024-09-10T08:48:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb425879740 1 -- start start 2024-09-10T08:48:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb425879740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151ba7b70 0x561151ba7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb425879740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba84b0 0x561151bacf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb425879740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151bad540 0x561151bafa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb425879740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561151b0b2f0 con 0x561151ba84b0 2024-09-10T08:48:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb425879740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561151ab3270 con 0x561151ba7b70 2024-09-10T08:48:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb425879740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561151b072b0 con 0x561151bad540 2024-09-10T08:48:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb41bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba84b0 0x561151bacf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb41bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba84b0 0x561151bacf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55508/0 (socket says 172.21.15.172:55508) 2024-09-10T08:48:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb41bfff640 1 -- 172.21.15.172:0/1502446710 learned_addr learned my addr 172.21.15.172:0/1502446710 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb41b7fe640 1 --2- 172.21.15.172:0/1502446710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151bad540 0x561151bafa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb4197fa640 1 --2- 172.21.15.172:0/671542732 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb4040a2c00 0x7fb4040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb41b7fe640 1 --2- 172.21.15.172:0/1502446710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151ba7b70 0x561151ba7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb41bfff640 1 -- 172.21.15.172:0/1502446710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151bad540 msgr2=0x561151bafa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb41bfff640 1 --2- 172.21.15.172:0/1502446710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151bad540 0x561151bafa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb41bfff640 1 -- 172.21.15.172:0/1502446710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151ba7b70 msgr2=0x561151ba7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb41bfff640 1 --2- 172.21.15.172:0/1502446710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151ba7b70 0x561151ba7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb41bfff640 1 -- 172.21.15.172:0/1502446710 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561151ab2340 con 0x561151ba84b0 2024-09-10T08:48:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb41b7fe640 1 --2- 172.21.15.172:0/1502446710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151ba7b70 0x561151ba7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb420fa1640 1 --2- 172.21.15.172:0/671542732 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb4040a2c00 0x7fb4040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb41bfff640 1 --2- 172.21.15.172:0/1502446710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba84b0 0x561151bacf70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb40c006010 tx=0x7fb40c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2dd0be730ff5a05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb4197fa640 1 -- 172.21.15.172:0/671542732 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb4140cf2a0 con 0x7fb41c102420 2024-09-10T08:48:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.827+0000 7fb4027fc640 1 -- 172.21.15.172:0/1502446710 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c03f070 con 0x561151ba84b0 2024-09-10T08:48:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.828+0000 7fb4027fc640 1 -- 172.21.15.172:0/1502446710 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb40c032e40 con 0x561151ba84b0 2024-09-10T08:48:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.828+0000 7fb4027fc640 1 -- 172.21.15.172:0/1502446710 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c035070 con 0x561151ba84b0 2024-09-10T08:48:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.828+0000 7fb425879740 1 -- 172.21.15.172:0/1502446710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba84b0 msgr2=0x561151bacf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.828+0000 7fb425879740 1 --2- 172.21.15.172:0/1502446710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba84b0 0x561151bacf70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb40c006010 tx=0x7fb40c009680 comp rx=0 tx=0).stop 2024-09-10T08:48:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.828+0000 7fb420fa1640 1 --2- 172.21.15.172:0/671542732 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb4040a2c00 0x7fb4040a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb414098170 tx=0x7fb414088040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.828+0000 7fb425879740 1 -- 172.21.15.172:0/1502446710 shutdown_connections 2024-09-10T08:48:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.828+0000 7fb425879740 1 --2- 172.21.15.172:0/1502446710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151bad540 0x561151bafa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.828+0000 7fb425879740 1 --2- 172.21.15.172:0/1502446710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba84b0 0x561151bacf70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.828+0000 7fb425879740 1 --2- 172.21.15.172:0/1502446710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151ba7b70 0x561151ba7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.828+0000 7fb425879740 1 -- 172.21.15.172:0/1502446710 >> 172.21.15.172:0/1502446710 conn(0x561151ba31e0 msgr2=0x561151ba35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.828+0000 7fb425879740 1 -- 172.21.15.172:0/1502446710 shutdown_connections 2024-09-10T08:48:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.828+0000 7fb425879740 1 -- 172.21.15.172:0/1502446710 wait complete. 2024-09-10T08:48:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.829+0000 7fb425879740 1 -- start start 2024-09-10T08:48:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.829+0000 7fb425879740 1 Processor -- start 2024-09-10T08:48:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.829+0000 7fb425879740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba7b70 0x561151bb1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.829+0000 7fb425879740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151ba84b0 0x561151bb2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.829+0000 7fb425879740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151bad540 0x561151bb9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.829+0000 7fb425879740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561151bb04b0 con 0x561151ba7b70 2024-09-10T08:48:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.829+0000 7fb425879740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561151a94290 con 0x561151bad540 2024-09-10T08:48:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.829+0000 7fb425879740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561151bb0630 con 0x561151ba84b0 2024-09-10T08:48:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.829+0000 7fb425879740 10 client.0 _ll_register_callbacks cb 0x561151ac5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.829+0000 7fb41b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba7b70 0x561151bb1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.829+0000 7fb41b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba7b70 0x561151bb1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55514/0 (socket says 172.21.15.172:55514) 2024-09-10T08:48:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.829+0000 7fb41b7fe640 1 -- 172.21.15.172:0/3366284703 learned_addr learned my addr 172.21.15.172:0/3366284703 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.829+0000 7fb41b7fe640 1 -- 172.21.15.172:0/3366284703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151ba84b0 msgr2=0x561151bb2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.829+0000 7fb41b7fe640 1 --2- 172.21.15.172:0/3366284703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151ba84b0 0x561151bb2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.829+0000 7fb41b7fe640 1 -- 172.21.15.172:0/3366284703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151bad540 msgr2=0x561151bb9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.829+0000 7fb41b7fe640 1 --2- 172.21.15.172:0/3366284703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151bad540 0x561151bb9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.829+0000 7fb41b7fe640 1 -- 172.21.15.172:0/3366284703 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56115181e210 con 0x561151ba7b70 2024-09-10T08:48:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.830+0000 7fb41b7fe640 1 --2- 172.21.15.172:0/3366284703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba7b70 0x561151bb1d00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb410012700 tx=0x7fb410012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24aaf468947cc89e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.830+0000 7fb3fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.830+0000 7fb3fb7fe640 1 -- 172.21.15.172:0/3366284703 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410004220 con 0x561151ba7b70 2024-09-10T08:48:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.830+0000 7fb3fb7fe640 1 -- 172.21.15.172:0/3366284703 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb4100043c0 con 0x561151ba7b70 2024-09-10T08:48:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.830+0000 7fb425879740 1 -- 172.21.15.172:0/3366284703 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561151b18a20 con 0x561151ba7b70 2024-09-10T08:48:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.830+0000 7fb3fb7fe640 1 -- 172.21.15.172:0/3366284703 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb410010930 con 0x561151ba7b70 2024-09-10T08:48:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.830+0000 7fb425879740 10 client.15990 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.830+0000 7fb425879740 1 -- 172.21.15.172:0/3366284703 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5611518ea620 con 0x561151ba7b70 2024-09-10T08:48:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.830+0000 7fb3fb7fe640 1 -- 172.21.15.172:0/3366284703 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561151b18a20 con 0x561151ba7b70 2024-09-10T08:48:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.830+0000 7fb3fb7fe640 1 -- 172.21.15.172:0/3366284703 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb41001c400 con 0x561151ba7b70 2024-09-10T08:48:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.830+0000 7fb425879740 10 client.15990 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.830+0000 7fb425879740 10 client.15990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.830+0000 7fb425879740 20 client.15990 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.830+0000 7fb425879740 1 --2- 172.21.15.172:0/3366284703 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561151bbd9b0 0x561151bbfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.830+0000 7fb425879740 4 client.15990 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.830+0000 7fb425879740 1 -- 172.21.15.172:0/3366284703 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561151b12d70 con 0x561151bbd9b0 2024-09-10T08:48:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.831+0000 7fb41bfff640 1 --2- 172.21.15.172:0/3366284703 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561151bbd9b0 0x561151bbfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.831+0000 7fb41bfff640 1 --2- 172.21.15.172:0/3366284703 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561151bbd9b0 0x561151bbfe00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b16d7b12cfa30fae server_cookie=4252085b25fe45ee in_seq=0 out_seq=0 2024-09-10T08:48:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.831+0000 7fb3fb7fe640 10 client.15990 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.832+0000 7fb3fb7fe640 1 -- 172.21.15.172:0/3366284703 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561151b12d70 con 0x561151bbd9b0 2024-09-10T08:48:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.832+0000 7fb3fb7fe640 10 client.15990 handle_command_reply: tid=0 2024-09-10T08:48:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.832+0000 7fb425879740 1 client.15990 shutdown 2024-09-10T08:48:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.832+0000 7fb425879740 1 -- 172.21.15.172:0/3366284703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba7b70 msgr2=0x561151bb1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.832+0000 7fb425879740 1 --2- 172.21.15.172:0/3366284703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba7b70 0x561151bb1d00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb410012700 tx=0x7fb410012bc0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.832+0000 7fb425879740 1 -- 172.21.15.172:0/3366284703 shutdown_connections 2024-09-10T08:48:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.832+0000 7fb425879740 1 --2- 172.21.15.172:0/3366284703 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561151bbd9b0 0x561151bbfe00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.832+0000 7fb425879740 1 --2- 172.21.15.172:0/3366284703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151bad540 0x561151bb9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.832+0000 7fb425879740 1 --2- 172.21.15.172:0/3366284703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151ba84b0 0x561151bb2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.832+0000 7fb425879740 1 --2- 172.21.15.172:0/3366284703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba7b70 0x561151bb1d00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.832+0000 7fb3fb7fe640 0 client.15990 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.832+0000 7fb425879740 1 -- 172.21.15.172:0/3366284703 >> 172.21.15.172:0/3366284703 conn(0x561151ba31e0 msgr2=0x561151ba49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.832+0000 7fb425879740 1 -- 172.21.15.172:0/3366284703 shutdown_connections 2024-09-10T08:48:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.832+0000 7fb425879740 1 -- 172.21.15.172:0/3366284703 wait complete. 2024-09-10T08:48:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.832+0000 7fb425879740 20 client.15990 trim_cache size 0 max 16384 2024-09-10T08:48:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.846+0000 7fb425879740 1 Processor -- start 2024-09-10T08:48:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.846+0000 7fb425879740 1 -- start start 2024-09-10T08:48:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.846+0000 7fb425879740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151ba7b70 0x561151bb21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.846+0000 7fb425879740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba84b0 0x561151bb9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.846+0000 7fb425879740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151bad540 0x561151ba1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.846+0000 7fb425879740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c1aa420 con 0x561151ba84b0 2024-09-10T08:48:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.846+0000 7fb425879740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c1abf60 con 0x561151ba7b70 2024-09-10T08:48:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.846+0000 7fb425879740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c1aaaf0 con 0x561151bad540 2024-09-10T08:48:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.846+0000 7fb41bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba84b0 0x561151bb9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.846+0000 7fb41bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba84b0 0x561151bb9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55524/0 (socket says 172.21.15.172:55524) 2024-09-10T08:48:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.846+0000 7fb41bfff640 1 -- 172.21.15.172:0/2741148864 learned_addr learned my addr 172.21.15.172:0/2741148864 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.846+0000 7fb41b7fe640 1 --2- 172.21.15.172:0/2741148864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151ba7b70 0x561151bb21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.846+0000 7fb41bfff640 1 -- 172.21.15.172:0/2741148864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151bad540 msgr2=0x561151ba1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.846+0000 7fb41bfff640 1 --2- 172.21.15.172:0/2741148864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151bad540 0x561151ba1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.846+0000 7fb41bfff640 1 -- 172.21.15.172:0/2741148864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151ba7b70 msgr2=0x561151bb21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.846+0000 7fb41bfff640 1 --2- 172.21.15.172:0/2741148864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151ba7b70 0x561151bb21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.846+0000 7fb41bfff640 1 -- 172.21.15.172:0/2741148864 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561151bb1840 con 0x561151ba84b0 2024-09-10T08:48:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.847+0000 7fb41bfff640 1 --2- 172.21.15.172:0/2741148864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba84b0 0x561151bb9240 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009230 tx=0x7fb40c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5bd3343b9f019cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:16.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.847+0000 7fb401ffb640 1 -- 172.21.15.172:0/2741148864 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c04b020 con 0x561151ba84b0 2024-09-10T08:48:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.847+0000 7fb401ffb640 1 -- 172.21.15.172:0/2741148864 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb40c031bd0 con 0x561151ba84b0 2024-09-10T08:48:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.847+0000 7fb401ffb640 1 -- 172.21.15.172:0/2741148864 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c03f070 con 0x561151ba84b0 2024-09-10T08:48:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.847+0000 7fb425879740 1 -- 172.21.15.172:0/2741148864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba84b0 msgr2=0x561151bb9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.847+0000 7fb425879740 1 --2- 172.21.15.172:0/2741148864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba84b0 0x561151bb9240 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009230 tx=0x7fb40c009260 comp rx=0 tx=0).stop 2024-09-10T08:48:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.847+0000 7fb425879740 1 -- 172.21.15.172:0/2741148864 shutdown_connections 2024-09-10T08:48:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.847+0000 7fb425879740 1 --2- 172.21.15.172:0/2741148864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151bad540 0x561151ba1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.847+0000 7fb425879740 1 --2- 172.21.15.172:0/2741148864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba84b0 0x561151bb9240 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.847+0000 7fb425879740 1 --2- 172.21.15.172:0/2741148864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151ba7b70 0x561151bb21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.847+0000 7fb425879740 1 -- 172.21.15.172:0/2741148864 >> 172.21.15.172:0/2741148864 conn(0x561151ba31e0 msgr2=0x561151bb1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.847+0000 7fb425879740 1 -- 172.21.15.172:0/2741148864 shutdown_connections 2024-09-10T08:48:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.847+0000 7fb425879740 1 -- 172.21.15.172:0/2741148864 wait complete. 2024-09-10T08:48:16.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb425879740 1 -- start start 2024-09-10T08:48:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb425879740 1 Processor -- start 2024-09-10T08:48:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb425879740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba7b70 0x561151bb6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb425879740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151ba84b0 0x561151bb6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb425879740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151bad540 0x561151bb7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb425879740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c1ab890 con 0x561151ba7b70 2024-09-10T08:48:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb425879740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c1ab1c0 con 0x561151bad540 2024-09-10T08:48:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb425879740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c1a8400 con 0x561151ba84b0 2024-09-10T08:48:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb425879740 10 client.0 _ll_register_callbacks cb 0x7fb41000b860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb41b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba7b70 0x561151bb6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb41b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba7b70 0x561151bb6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55532/0 (socket says 172.21.15.172:55532) 2024-09-10T08:48:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb41b7fe640 1 -- 172.21.15.172:0/500770614 learned_addr learned my addr 172.21.15.172:0/500770614 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb41bfff640 1 --2- 172.21.15.172:0/500770614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151ba84b0 0x561151bb6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb41b7fe640 1 --2- 172.21.15.172:0/500770614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151bad540 0x561151bb7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb41b7fe640 1 -- 172.21.15.172:0/500770614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151ba84b0 msgr2=0x561151bb6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb41b7fe640 1 --2- 172.21.15.172:0/500770614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151ba84b0 0x561151bb6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb41b7fe640 1 -- 172.21.15.172:0/500770614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151bad540 msgr2=0x561151bb7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb41b7fe640 1 --2- 172.21.15.172:0/500770614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151bad540 0x561151bb7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb41b7fe640 1 -- 172.21.15.172:0/500770614 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561151bb7f50 con 0x561151ba7b70 2024-09-10T08:48:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.848+0000 7fb41bfff640 1 --2- 172.21.15.172:0/500770614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151ba84b0 0x561151bb6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.849+0000 7fb41b7fe640 1 --2- 172.21.15.172:0/500770614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba7b70 0x561151bb6440 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb410005940 tx=0x7fb4100058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107a2f19496bfc6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.849+0000 7fb3fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.849+0000 7fb3fb7fe640 1 -- 172.21.15.172:0/500770614 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb410032020 con 0x561151ba7b70 2024-09-10T08:48:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.849+0000 7fb3fb7fe640 1 -- 172.21.15.172:0/500770614 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb410010cd0 con 0x561151ba7b70 2024-09-10T08:48:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.849+0000 7fb425879740 1 -- 172.21.15.172:0/500770614 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb40c031bd0 con 0x561151ba7b70 2024-09-10T08:48:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.849+0000 7fb3fb7fe640 1 -- 172.21.15.172:0/500770614 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb41003a050 con 0x561151ba7b70 2024-09-10T08:48:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.849+0000 7fb425879740 10 client.15996 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.849+0000 7fb425879740 1 -- 172.21.15.172:0/500770614 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561151bb9330 con 0x561151ba7b70 2024-09-10T08:48:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.849+0000 7fb3fb7fe640 1 -- 172.21.15.172:0/500770614 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb40c031bd0 con 0x561151ba7b70 2024-09-10T08:48:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.849+0000 7fb3fb7fe640 1 -- 172.21.15.172:0/500770614 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb4100047e0 con 0x561151ba7b70 2024-09-10T08:48:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.849+0000 7fb425879740 10 client.15996 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.849+0000 7fb425879740 10 client.15996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.849+0000 7fb425879740 20 client.15996 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.849+0000 7fb425879740 1 --2- 172.21.15.172:0/500770614 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561151c28b70 0x561151c2af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.849+0000 7fb425879740 4 client.15996 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:48:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.849+0000 7fb425879740 1 -- 172.21.15.172:0/500770614 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561151a8f900 con 0x561151c28b70 2024-09-10T08:48:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.849+0000 7fb41bfff640 1 --2- 172.21.15.172:0/500770614 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561151c28b70 0x561151c2af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.850+0000 7fb3fb7fe640 10 client.15996 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.850+0000 7fb41bfff640 1 --2- 172.21.15.172:0/500770614 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561151c28b70 0x561151c2af90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d29c160d406ab0ed server_cookie=a1bf43eca10e5af7 in_seq=0 out_seq=0 2024-09-10T08:48:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.854+0000 7fb3fb7fe640 1 -- 172.21.15.172:0/500770614 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561151a8f900 con 0x561151c28b70 2024-09-10T08:48:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.854+0000 7fb3fb7fe640 10 client.15996 handle_command_reply: tid=0 2024-09-10T08:48:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.854+0000 7fb425879740 1 client.15996 shutdown 2024-09-10T08:48:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.854+0000 7fb425879740 1 -- 172.21.15.172:0/500770614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba7b70 msgr2=0x561151bb6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.854+0000 7fb425879740 1 --2- 172.21.15.172:0/500770614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba7b70 0x561151bb6440 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb410005940 tx=0x7fb4100058d0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.854+0000 7fb425879740 1 -- 172.21.15.172:0/500770614 shutdown_connections 2024-09-10T08:48:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.854+0000 7fb425879740 1 --2- 172.21.15.172:0/500770614 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561151c28b70 0x561151c2af90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.854+0000 7fb425879740 1 --2- 172.21.15.172:0/500770614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561151bad540 0x561151bb7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.854+0000 7fb425879740 1 --2- 172.21.15.172:0/500770614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561151ba84b0 0x561151bb6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.854+0000 7fb3fb7fe640 0 client.15996 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.854+0000 7fb425879740 1 --2- 172.21.15.172:0/500770614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561151ba7b70 0x561151bb6440 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.854+0000 7fb425879740 1 -- 172.21.15.172:0/500770614 >> 172.21.15.172:0/500770614 conn(0x561151ba31e0 msgr2=0x561151ba4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.854+0000 7fb425879740 1 -- 172.21.15.172:0/500770614 shutdown_connections 2024-09-10T08:48:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.854+0000 7fb425879740 1 -- 172.21.15.172:0/500770614 wait complete. 2024-09-10T08:48:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.854+0000 7fb425879740 20 client.15996 trim_cache size 0 max 16384 2024-09-10T08:48:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:48:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:48:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:48:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:48:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.855+0000 7fb4217a2640 1 -- 172.21.15.172:0/671542732 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb4040a2c00 msgr2=0x7fb4040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.855+0000 7fb4217a2640 1 --2- 172.21.15.172:0/671542732 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb4040a2c00 0x7fb4040a50c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb414098170 tx=0x7fb414088040 comp rx=0 tx=0).stop 2024-09-10T08:48:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.855+0000 7fb4217a2640 1 -- 172.21.15.172:0/671542732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41c102420 msgr2=0x7fb41c19f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.855+0000 7fb4217a2640 1 --2- 172.21.15.172:0/671542732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41c102420 0x7fb41c19f190 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb41400cad0 tx=0x7fb41400cf90 comp rx=0 tx=0).stop 2024-09-10T08:48:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.855+0000 7fb4217a2640 1 -- 172.21.15.172:0/671542732 shutdown_connections 2024-09-10T08:48:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.855+0000 7fb4217a2640 1 --2- 172.21.15.172:0/671542732 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb4040a2c00 0x7fb4040a50c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.855+0000 7fb4217a2640 1 --2- 172.21.15.172:0/671542732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb41c102420 0x7fb41c19f190 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.855+0000 7fb4217a2640 1 --2- 172.21.15.172:0/671542732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb41c101a60 0x7fb41c198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.855+0000 7fb4217a2640 1 --2- 172.21.15.172:0/671542732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb41c100870 0x7fb41c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.855+0000 7fb4217a2640 1 -- 172.21.15.172:0/671542732 >> 172.21.15.172:0/671542732 conn(0x7fb41c0fbfd0 msgr2=0x7fb41c06a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.855+0000 7fb4217a2640 1 -- 172.21.15.172:0/671542732 shutdown_connections 2024-09-10T08:48:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:16.855+0000 7fb4217a2640 1 -- 172.21.15.172:0/671542732 wait complete. 2024-09-10T08:48:17.261 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:48:17.510 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:18.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.008+0000 7ffb16b2f640 1 -- 172.21.15.172:0/855004843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb10100a70 msgr2=0x7ffb1010ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:18.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.008+0000 7ffb16b2f640 1 --2- 172.21.15.172:0/855004843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb10100a70 0x7ffb1010ea60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009ca0 tx=0x7ffb0802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.008+0000 7ffb16b2f640 1 -- 172.21.15.172:0/855004843 shutdown_connections 2024-09-10T08:48:18.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.008+0000 7ffb16b2f640 1 --2- 172.21.15.172:0/855004843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb10100a70 0x7ffb1010ea60 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.008+0000 7ffb16b2f640 1 --2- 172.21.15.172:0/855004843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb101000d0 0x7ffb10100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.008+0000 7ffb16b2f640 1 --2- 172.21.15.172:0/855004843 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10106ba0 0x7ffb10106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.008+0000 7ffb16b2f640 1 -- 172.21.15.172:0/855004843 >> 172.21.15.172:0/855004843 conn(0x7ffb10077210 msgr2=0x7ffb100fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:18.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.008+0000 7ffb16b2f640 1 -- 172.21.15.172:0/855004843 shutdown_connections 2024-09-10T08:48:18.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.008+0000 7ffb16b2f640 1 -- 172.21.15.172:0/855004843 wait complete. 2024-09-10T08:48:18.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.008+0000 7ffb16b2f640 1 Processor -- start 2024-09-10T08:48:18.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb16b2f640 1 -- start start 2024-09-10T08:48:18.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb16b2f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb101000d0 0x7ffb101012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb16b2f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb10100a70 0x7ffb101017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb16b2f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10106ba0 0x7ffb10101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb16b2f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffb10111f40 con 0x7ffb101000d0 2024-09-10T08:48:18.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb16b2f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb10111dc0 con 0x7ffb10100a70 2024-09-10T08:48:18.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb16b2f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101120c0 con 0x7ffb10106ba0 2024-09-10T08:48:18.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb148cb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb101000d0 0x7ffb101012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:18.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb148cb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb101000d0 0x7ffb101012b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55548/0 (socket says 172.21.15.172:55548) 2024-09-10T08:48:18.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb148cb640 1 -- 172.21.15.172:0/3401810697 learned_addr learned my addr 172.21.15.172:0/3401810697 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:18.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb150cc640 1 --2- 172.21.15.172:0/3401810697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10106ba0 0x7ffb10101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:18.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb148cb640 1 -- 172.21.15.172:0/3401810697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10106ba0 msgr2=0x7ffb10101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:18.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb148cb640 1 --2- 172.21.15.172:0/3401810697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10106ba0 0x7ffb10101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb0ffff640 1 --2- 172.21.15.172:0/3401810697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb10100a70 0x7ffb101017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:18.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb148cb640 1 -- 172.21.15.172:0/3401810697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb10100a70 msgr2=0x7ffb101017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:18.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb148cb640 1 --2- 172.21.15.172:0/3401810697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb10100a70 0x7ffb101017f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb148cb640 1 -- 172.21.15.172:0/3401810697 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb101a6800 con 0x7ffb101000d0 2024-09-10T08:48:18.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb150cc640 1 --2- 172.21.15.172:0/3401810697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10106ba0 0x7ffb10101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:18.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb148cb640 1 --2- 172.21.15.172:0/3401810697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb101000d0 0x7ffb101012b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400c970 tx=0x7ffb0400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:18.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.009+0000 7ffb0dffb640 1 -- 172.21.15.172:0/3401810697 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04007bb0 con 0x7ffb101000d0 2024-09-10T08:48:18.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.010+0000 7ffb0dffb640 1 -- 172.21.15.172:0/3401810697 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb04007d50 con 0x7ffb101000d0 2024-09-10T08:48:18.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.010+0000 7ffb16b2f640 1 -- 172.21.15.172:0/3401810697 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb101a6af0 con 0x7ffb101000d0 2024-09-10T08:48:18.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.010+0000 7ffb0dffb640 1 -- 172.21.15.172:0/3401810697 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0400f450 con 0x7ffb101000d0 2024-09-10T08:48:18.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.010+0000 7ffb16b2f640 1 -- 172.21.15.172:0/3401810697 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb101a6fe0 con 0x7ffb101000d0 2024-09-10T08:48:18.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.012+0000 7ffb0dffb640 1 -- 172.21.15.172:0/3401810697 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ffb04004f20 con 0x7ffb101000d0 2024-09-10T08:48:18.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.012+0000 7ffb18f08740 1 Processor -- start 2024-09-10T08:48:18.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.012+0000 7ffb18f08740 1 -- start start 2024-09-10T08:48:18.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb0dffb640 1 --2- 172.21.15.172:0/3401810697 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffb000a2b80 0x7ffb000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb18f08740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775d1b40 0x5584775d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb18f08740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775d05d0 0x5584775d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb18f08740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775db370 0x5584775dd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb18f08740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5584775362f0 con 0x5584775d05d0 2024-09-10T08:48:18.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb18f08740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5584774ddcf0 con 0x5584775d1b40 2024-09-10T08:48:18.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb18f08740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5584775322b0 con 0x5584775db370 2024-09-10T08:48:18.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb0ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775d05d0 0x5584775d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:18.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb150cc640 1 --2- 172.21.15.172:0/3401810697 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffb000a2b80 0x7ffb000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:18.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb0ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775d05d0 0x5584775d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55560/0 (socket says 172.21.15.172:55560) 2024-09-10T08:48:18.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb0ffff640 1 -- 172.21.15.172:0/2448382285 learned_addr learned my addr 172.21.15.172:0/2448382285 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:18.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb0dffb640 1 -- 172.21.15.172:0/3401810697 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ffb040cf1a0 con 0x7ffb101000d0 2024-09-10T08:48:18.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb148cb640 1 --2- 172.21.15.172:0/2448382285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775d1b40 0x5584775d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:18.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb0ffff640 1 -- 172.21.15.172:0/2448382285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775db370 msgr2=0x5584775dd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:18.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb0ffff640 1 --2- 172.21.15.172:0/2448382285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775db370 0x5584775dd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb0ffff640 1 -- 172.21.15.172:0/2448382285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775d1b40 msgr2=0x5584775d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:18.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb0ffff640 1 --2- 172.21.15.172:0/2448382285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775d1b40 0x5584775d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb0ffff640 1 -- 172.21.15.172:0/2448382285 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584774c23a0 con 0x5584775d05d0 2024-09-10T08:48:18.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb0ffff640 1 --2- 172.21.15.172:0/2448382285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775d05d0 0x5584775d0a10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00bb10 tx=0x7ffafc00d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766ffeb50bb8010d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:18.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffb150cc640 1 --2- 172.21.15.172:0/3401810697 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffb000a2b80 0x7ffb000a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009a60 tx=0x7ffb08031450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:18.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffafaffd640 1 -- 172.21.15.172:0/2448382285 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc017420 con 0x5584775d05d0 2024-09-10T08:48:18.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffafaffd640 1 -- 172.21.15.172:0/2448382285 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffafc017a70 con 0x5584775d05d0 2024-09-10T08:48:18.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.013+0000 7ffafaffd640 1 -- 172.21.15.172:0/2448382285 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc00f980 con 0x5584775d05d0 2024-09-10T08:48:18.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.014+0000 7ffb18f08740 1 -- 172.21.15.172:0/2448382285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775d05d0 msgr2=0x5584775d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:18.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.014+0000 7ffb18f08740 1 --2- 172.21.15.172:0/2448382285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775d05d0 0x5584775d0a10 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00bb10 tx=0x7ffafc00d340 comp rx=0 tx=0).stop 2024-09-10T08:48:18.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.014+0000 7ffb18f08740 1 -- 172.21.15.172:0/2448382285 shutdown_connections 2024-09-10T08:48:18.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.014+0000 7ffb18f08740 1 --2- 172.21.15.172:0/2448382285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775db370 0x5584775dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.014+0000 7ffb18f08740 1 --2- 172.21.15.172:0/2448382285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775d05d0 0x5584775d0a10 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.014+0000 7ffb18f08740 1 --2- 172.21.15.172:0/2448382285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775d1b40 0x5584775d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.014+0000 7ffb18f08740 1 -- 172.21.15.172:0/2448382285 >> 172.21.15.172:0/2448382285 conn(0x558477599310 msgr2=0x5584775ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:18.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.014+0000 7ffb18f08740 1 -- 172.21.15.172:0/2448382285 shutdown_connections 2024-09-10T08:48:18.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.014+0000 7ffb18f08740 1 -- 172.21.15.172:0/2448382285 wait complete. 2024-09-10T08:48:18.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.014+0000 7ffb18f08740 1 -- start start 2024-09-10T08:48:18.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb18f08740 1 Processor -- start 2024-09-10T08:48:18.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb18f08740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775d05d0 0x5584775dacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb18f08740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775d1b40 0x5584775d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb18f08740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775db370 0x5584775d84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb18f08740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5584775de070 con 0x5584775db370 2024-09-10T08:48:18.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb18f08740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5584774f1060 con 0x5584775d1b40 2024-09-10T08:48:18.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb18f08740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5584775de1f0 con 0x5584775d05d0 2024-09-10T08:48:18.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb18f08740 10 client.0 _ll_register_callbacks cb 0x5584772448c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:18.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb150cc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775db370 0x5584775d84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:18.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb150cc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775db370 0x5584775d84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55562/0 (socket says 172.21.15.172:55562) 2024-09-10T08:48:18.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb150cc640 1 -- 172.21.15.172:0/1724055768 learned_addr learned my addr 172.21.15.172:0/1724055768 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:18.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb148cb640 1 --2- 172.21.15.172:0/1724055768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775d05d0 0x5584775dacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:18.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb0ffff640 1 --2- 172.21.15.172:0/1724055768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775d1b40 0x5584775d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:18.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb150cc640 1 -- 172.21.15.172:0/1724055768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775d05d0 msgr2=0x5584775dacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:18.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb150cc640 1 --2- 172.21.15.172:0/1724055768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775d05d0 0x5584775dacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb150cc640 1 -- 172.21.15.172:0/1724055768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775d1b40 msgr2=0x5584775d7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:18.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb150cc640 1 --2- 172.21.15.172:0/1724055768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775d1b40 0x5584775d7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb150cc640 1 -- 172.21.15.172:0/1724055768 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584772a73c0 con 0x5584775db370 2024-09-10T08:48:18.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.015+0000 7ffb0ffff640 1 --2- 172.21.15.172:0/1724055768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775d1b40 0x5584775d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:18.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.016+0000 7ffb150cc640 1 --2- 172.21.15.172:0/1724055768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775db370 0x5584775d84e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ffb08031e30 tx=0x7ffb0802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db331d1484afb2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:18.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.016+0000 7ffaeffff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:18.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.016+0000 7ffaeffff640 1 -- 172.21.15.172:0/1724055768 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb080047c0 con 0x5584775db370 2024-09-10T08:48:18.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.016+0000 7ffb18f08740 1 -- 172.21.15.172:0/1724055768 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffafc017a70 con 0x5584775db370 2024-09-10T08:48:18.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.016+0000 7ffaeffff640 1 -- 172.21.15.172:0/1724055768 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ffb080305c0 con 0x5584775db370 2024-09-10T08:48:18.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.016+0000 7ffaeffff640 1 -- 172.21.15.172:0/1724055768 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffb08057050 con 0x5584775db370 2024-09-10T08:48:18.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.016+0000 7ffb18f08740 10 client.16008 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:18.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.016+0000 7ffb18f08740 1 -- 172.21.15.172:0/1724055768 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5584775e9ab0 con 0x5584775db370 2024-09-10T08:48:18.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.016+0000 7ffaeffff640 1 -- 172.21.15.172:0/1724055768 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffafc017a70 con 0x5584775db370 2024-09-10T08:48:18.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.016+0000 7ffaeffff640 1 -- 172.21.15.172:0/1724055768 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ffb0802f860 con 0x5584775db370 2024-09-10T08:48:18.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.016+0000 7ffb18f08740 10 client.16008 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:18.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.016+0000 7ffb18f08740 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:18.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.016+0000 7ffb18f08740 20 client.16008 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:18.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.016+0000 7ffb18f08740 1 --2- 172.21.15.172:0/1724055768 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5584775eaf10 0x5584775ed3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.016+0000 7ffb18f08740 4 client.16008 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:18.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.016+0000 7ffb18f08740 1 -- 172.21.15.172:0/1724055768 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558477543860 con 0x5584775eaf10 2024-09-10T08:48:18.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.017+0000 7ffb148cb640 1 --2- 172.21.15.172:0/1724055768 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5584775eaf10 0x5584775ed3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:18.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.017+0000 7ffb148cb640 1 --2- 172.21.15.172:0/1724055768 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5584775eaf10 0x5584775ed3d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f84e0ed0b8290c40 server_cookie=17e939af41efd569 in_seq=0 out_seq=0 2024-09-10T08:48:18.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.017+0000 7ffaeffff640 10 client.16008 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:18.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.018+0000 7ffaeffff640 1 -- 172.21.15.172:0/1724055768 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558477543860 con 0x5584775eaf10 2024-09-10T08:48:18.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.018+0000 7ffaeffff640 10 client.16008 handle_command_reply: tid=0 2024-09-10T08:48:18.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.018+0000 7ffb18f08740 1 client.16008 shutdown 2024-09-10T08:48:18.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.018+0000 7ffb18f08740 1 -- 172.21.15.172:0/1724055768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775db370 msgr2=0x5584775d84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:18.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.018+0000 7ffb18f08740 1 --2- 172.21.15.172:0/1724055768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775db370 0x5584775d84e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ffb08031e30 tx=0x7ffb0802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.018+0000 7ffb18f08740 1 -- 172.21.15.172:0/1724055768 shutdown_connections 2024-09-10T08:48:18.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.018+0000 7ffb18f08740 1 --2- 172.21.15.172:0/1724055768 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5584775eaf10 0x5584775ed3d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.018+0000 7ffb18f08740 1 --2- 172.21.15.172:0/1724055768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775db370 0x5584775d84e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.018+0000 7ffb18f08740 1 --2- 172.21.15.172:0/1724055768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775d1b40 0x5584775d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.018+0000 7ffb18f08740 1 --2- 172.21.15.172:0/1724055768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775d05d0 0x5584775dacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.018+0000 7ffaeffff640 0 client.16008 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:18.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.018+0000 7ffb18f08740 1 -- 172.21.15.172:0/1724055768 >> 172.21.15.172:0/1724055768 conn(0x558477599310 msgr2=0x5584775d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:18.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.018+0000 7ffb18f08740 1 -- 172.21.15.172:0/1724055768 shutdown_connections 2024-09-10T08:48:18.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.018+0000 7ffb18f08740 1 -- 172.21.15.172:0/1724055768 wait complete. 2024-09-10T08:48:18.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.018+0000 7ffb18f08740 20 client.16008 trim_cache size 0 max 16384 2024-09-10T08:48:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb18f08740 1 Processor -- start 2024-09-10T08:48:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb18f08740 1 -- start start 2024-09-10T08:48:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb18f08740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775d05d0 0x5584775cce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb18f08740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775d1b40 0x5584775e7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb18f08740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775db370 0x5584775f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb18f08740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101b0cb0 con 0x5584775d1b40 2024-09-10T08:48:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb18f08740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101b27f0 con 0x5584775db370 2024-09-10T08:48:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb18f08740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101b1380 con 0x5584775d05d0 2024-09-10T08:48:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb0ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775d1b40 0x5584775e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb0ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775d1b40 0x5584775e7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55572/0 (socket says 172.21.15.172:55572) 2024-09-10T08:48:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb0ffff640 1 -- 172.21.15.172:0/1086180563 learned_addr learned my addr 172.21.15.172:0/1086180563 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:18.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb150cc640 1 --2- 172.21.15.172:0/1086180563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775db370 0x5584775f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:18.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb148cb640 1 --2- 172.21.15.172:0/1086180563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775d05d0 0x5584775cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:18.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb0ffff640 1 -- 172.21.15.172:0/1086180563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775d05d0 msgr2=0x5584775cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:18.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb0ffff640 1 --2- 172.21.15.172:0/1086180563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775d05d0 0x5584775cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb0ffff640 1 -- 172.21.15.172:0/1086180563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775db370 msgr2=0x5584775f2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:18.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb0ffff640 1 --2- 172.21.15.172:0/1086180563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775db370 0x5584775f2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb0ffff640 1 -- 172.21.15.172:0/1086180563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584775eda50 con 0x5584775d1b40 2024-09-10T08:48:18.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.032+0000 7ffb150cc640 1 --2- 172.21.15.172:0/1086180563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775db370 0x5584775f2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:18.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.033+0000 7ffb0ffff640 1 --2- 172.21.15.172:0/1086180563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775d1b40 0x5584775e7340 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00bfd0 tx=0x7ffafc00dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa319bd8954c99f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:18.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.033+0000 7ffafa7fc640 1 -- 172.21.15.172:0/1086180563 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc026020 con 0x5584775d1b40 2024-09-10T08:48:18.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.033+0000 7ffafa7fc640 1 -- 172.21.15.172:0/1086180563 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffafc00b700 con 0x5584775d1b40 2024-09-10T08:48:18.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.033+0000 7ffafa7fc640 1 -- 172.21.15.172:0/1086180563 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffafc004db0 con 0x5584775d1b40 2024-09-10T08:48:18.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.033+0000 7ffb18f08740 1 -- 172.21.15.172:0/1086180563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775d1b40 msgr2=0x5584775e7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:18.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.033+0000 7ffb18f08740 1 --2- 172.21.15.172:0/1086180563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775d1b40 0x5584775e7340 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00bfd0 tx=0x7ffafc00dee0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.033+0000 7ffb18f08740 1 -- 172.21.15.172:0/1086180563 shutdown_connections 2024-09-10T08:48:18.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.033+0000 7ffb18f08740 1 --2- 172.21.15.172:0/1086180563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775db370 0x5584775f2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.033+0000 7ffb18f08740 1 --2- 172.21.15.172:0/1086180563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775d1b40 0x5584775e7340 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.033+0000 7ffb18f08740 1 --2- 172.21.15.172:0/1086180563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775d05d0 0x5584775cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.033+0000 7ffb18f08740 1 -- 172.21.15.172:0/1086180563 >> 172.21.15.172:0/1086180563 conn(0x558477599310 msgr2=0x5584775cc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:18.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.034+0000 7ffb18f08740 1 -- 172.21.15.172:0/1086180563 shutdown_connections 2024-09-10T08:48:18.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.034+0000 7ffb18f08740 1 -- 172.21.15.172:0/1086180563 wait complete. 2024-09-10T08:48:18.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.034+0000 7ffb18f08740 1 -- start start 2024-09-10T08:48:18.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.034+0000 7ffb18f08740 1 Processor -- start 2024-09-10T08:48:18.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.034+0000 7ffb18f08740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775d05d0 0x5584775e3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.034+0000 7ffb18f08740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775d1b40 0x5584775df140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.034+0000 7ffb18f08740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775db370 0x5584775df8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.034+0000 7ffb18f08740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101b2120 con 0x5584775db370 2024-09-10T08:48:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.034+0000 7ffb18f08740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101b1a50 con 0x5584775d05d0 2024-09-10T08:48:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.034+0000 7ffb18f08740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101aec90 con 0x5584775d1b40 2024-09-10T08:48:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.034+0000 7ffb18f08740 10 client.0 _ll_register_callbacks cb 0x7ffb08009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.034+0000 7ffb150cc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775db370 0x5584775df8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffb150cc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775db370 0x5584775df8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55584/0 (socket says 172.21.15.172:55584) 2024-09-10T08:48:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffb150cc640 1 -- 172.21.15.172:0/2052863467 learned_addr learned my addr 172.21.15.172:0/2052863467 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffb0ffff640 1 --2- 172.21.15.172:0/2052863467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775d1b40 0x5584775df140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffb148cb640 1 --2- 172.21.15.172:0/2052863467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775d05d0 0x5584775e3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffb150cc640 1 -- 172.21.15.172:0/2052863467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775d1b40 msgr2=0x5584775df140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffb150cc640 1 --2- 172.21.15.172:0/2052863467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775d1b40 0x5584775df140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffb150cc640 1 -- 172.21.15.172:0/2052863467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775d05d0 msgr2=0x5584775e3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffb150cc640 1 --2- 172.21.15.172:0/2052863467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775d05d0 0x5584775e3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffb150cc640 1 -- 172.21.15.172:0/2052863467 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558477654c70 con 0x5584775db370 2024-09-10T08:48:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffb150cc640 1 --2- 172.21.15.172:0/2052863467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775db370 0x5584775df8d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009430 tx=0x7ffb08031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d88a1c5f4642dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffaeffff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffaeffff640 1 -- 172.21.15.172:0/2052863467 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0805f020 con 0x5584775db370 2024-09-10T08:48:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffaeffff640 1 -- 172.21.15.172:0/2052863467 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ffb08004b40 con 0x5584775db370 2024-09-10T08:48:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffb18f08740 1 -- 172.21.15.172:0/2052863467 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffafc00b700 con 0x5584775db370 2024-09-10T08:48:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffaeffff640 1 -- 172.21.15.172:0/2052863467 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffb0806a050 con 0x5584775db370 2024-09-10T08:48:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffb18f08740 10 client.16014 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffb18f08740 1 -- 172.21.15.172:0/2052863467 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558477655e00 con 0x5584775db370 2024-09-10T08:48:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.035+0000 7ffaeffff640 1 -- 172.21.15.172:0/2052863467 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffafc00b700 con 0x5584775db370 2024-09-10T08:48:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.036+0000 7ffaeffff640 1 -- 172.21.15.172:0/2052863467 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ffb08049770 con 0x5584775db370 2024-09-10T08:48:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.036+0000 7ffb18f08740 10 client.16014 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.036+0000 7ffb18f08740 10 client.16014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:18.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.036+0000 7ffb18f08740 20 client.16014 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:18.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.036+0000 7ffb18f08740 1 --2- 172.21.15.172:0/2052863467 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5584776565a0 0x5584776589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:18.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.036+0000 7ffb18f08740 4 client.16014 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:48:18.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.036+0000 7ffb18f08740 1 -- 172.21.15.172:0/2052863467 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55847753dd70 con 0x5584776565a0 2024-09-10T08:48:18.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.036+0000 7ffb148cb640 1 --2- 172.21.15.172:0/2052863467 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5584776565a0 0x5584776589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:18.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.036+0000 7ffb148cb640 1 --2- 172.21.15.172:0/2052863467 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5584776565a0 0x5584776589c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3170bd68fbaf465 server_cookie=4655612c0568e976 in_seq=0 out_seq=0 2024-09-10T08:48:18.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.036+0000 7ffaeffff640 10 client.16014 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:18.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.037+0000 7ffaeffff640 1 -- 172.21.15.172:0/2052863467 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55847753dd70 con 0x5584776565a0 2024-09-10T08:48:18.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.037+0000 7ffaeffff640 10 client.16014 handle_command_reply: tid=0 2024-09-10T08:48:18.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.037+0000 7ffb18f08740 1 client.16014 shutdown 2024-09-10T08:48:18.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.037+0000 7ffb18f08740 1 -- 172.21.15.172:0/2052863467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775db370 msgr2=0x5584775df8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:18.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.037+0000 7ffb18f08740 1 --2- 172.21.15.172:0/2052863467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775db370 0x5584775df8d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009430 tx=0x7ffb08031a30 comp rx=0 tx=0).stop 2024-09-10T08:48:18.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.037+0000 7ffb18f08740 1 -- 172.21.15.172:0/2052863467 shutdown_connections 2024-09-10T08:48:18.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.037+0000 7ffb18f08740 1 --2- 172.21.15.172:0/2052863467 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5584776565a0 0x5584776589c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.037+0000 7ffaeffff640 0 client.16014 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:18.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.037+0000 7ffb18f08740 1 --2- 172.21.15.172:0/2052863467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584775db370 0x5584775df8d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.037+0000 7ffb18f08740 1 --2- 172.21.15.172:0/2052863467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5584775d1b40 0x5584775df140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.037+0000 7ffb18f08740 1 --2- 172.21.15.172:0/2052863467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584775d05d0 0x5584775e3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.037+0000 7ffb18f08740 1 -- 172.21.15.172:0/2052863467 >> 172.21.15.172:0/2052863467 conn(0x558477599310 msgr2=0x5584775da000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:18.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.037+0000 7ffb18f08740 1 -- 172.21.15.172:0/2052863467 shutdown_connections 2024-09-10T08:48:18.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.037+0000 7ffb18f08740 1 -- 172.21.15.172:0/2052863467 wait complete. 2024-09-10T08:48:18.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.037+0000 7ffb18f08740 20 client.16014 trim_cache size 0 max 16384 2024-09-10T08:48:18.049 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:48:18.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.037+0000 7ffb16b2f640 1 -- 172.21.15.172:0/3401810697 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffb000a2b80 msgr2=0x7ffb000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:18.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.038+0000 7ffb16b2f640 1 --2- 172.21.15.172:0/3401810697 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffb000a2b80 0x7ffb000a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009a60 tx=0x7ffb08031450 comp rx=0 tx=0).stop 2024-09-10T08:48:18.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.038+0000 7ffb16b2f640 1 -- 172.21.15.172:0/3401810697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb101000d0 msgr2=0x7ffb101012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:18.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.038+0000 7ffb16b2f640 1 --2- 172.21.15.172:0/3401810697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb101000d0 0x7ffb101012b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400c970 tx=0x7ffb0400ce30 comp rx=0 tx=0).stop 2024-09-10T08:48:18.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.038+0000 7ffb16b2f640 1 -- 172.21.15.172:0/3401810697 shutdown_connections 2024-09-10T08:48:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.038+0000 7ffb16b2f640 1 --2- 172.21.15.172:0/3401810697 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffb000a2b80 0x7ffb000a5040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.038+0000 7ffb16b2f640 1 --2- 172.21.15.172:0/3401810697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10106ba0 0x7ffb10101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.038+0000 7ffb16b2f640 1 --2- 172.21.15.172:0/3401810697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb10100a70 0x7ffb101017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.038+0000 7ffb16b2f640 1 --2- 172.21.15.172:0/3401810697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb101000d0 0x7ffb101012b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.038+0000 7ffb16b2f640 1 -- 172.21.15.172:0/3401810697 >> 172.21.15.172:0/3401810697 conn(0x7ffb10077210 msgr2=0x7ffb10104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.038+0000 7ffb16b2f640 1 -- 172.21.15.172:0/3401810697 shutdown_connections 2024-09-10T08:48:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:18.038+0000 7ffb16b2f640 1 -- 172.21.15.172:0/3401810697 wait complete. 2024-09-10T08:48:18.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:18 smithi191 ceph-mon[35314]: pgmap v412: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:48:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:18 smithi172 ceph-mon[30854]: pgmap v412: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:48:18.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:18 smithi174 ceph-mon[34068]: pgmap v412: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T08:48:19.743 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:48:19.744 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:48:20.001 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:20.413 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:20 smithi172 ceph-mon[30854]: pgmap v413: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T08:48:20.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.498+0000 7fad44b2f640 1 -- 172.21.15.172:0/106450867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad40102ef0 msgr2=0x7fad4010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:20.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.498+0000 7fad44b2f640 1 --2- 172.21.15.172:0/106450867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad40102ef0 0x7fad4010da40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fad38009ca0 tx=0x7fad3802a950 comp rx=0 tx=0).stop 2024-09-10T08:48:20.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.498+0000 7fad44b2f640 1 -- 172.21.15.172:0/106450867 shutdown_connections 2024-09-10T08:48:20.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.498+0000 7fad44b2f640 1 --2- 172.21.15.172:0/106450867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad40102ef0 0x7fad4010da40 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.498+0000 7fad44b2f640 1 --2- 172.21.15.172:0/106450867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad40102570 0x7fad401029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.498+0000 7fad44b2f640 1 --2- 172.21.15.172:0/106450867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad40100e20 0x7fad40101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.498+0000 7fad44b2f640 1 -- 172.21.15.172:0/106450867 >> 172.21.15.172:0/106450867 conn(0x7fad400fc7d0 msgr2=0x7fad400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:20.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.498+0000 7fad44b2f640 1 -- 172.21.15.172:0/106450867 shutdown_connections 2024-09-10T08:48:20.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.498+0000 7fad44b2f640 1 -- 172.21.15.172:0/106450867 wait complete. 2024-09-10T08:48:20.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.498+0000 7fad44b2f640 1 Processor -- start 2024-09-10T08:48:20.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.498+0000 7fad44b2f640 1 -- start start 2024-09-10T08:48:20.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad44b2f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad40100e20 0x7fad401992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad44b2f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad40102570 0x7fad40199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad44b2f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad40102ef0 0x7fad401a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad44b2f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fad40110e70 con 0x7fad40100e20 2024-09-10T08:48:20.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad44b2f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fad40110cf0 con 0x7fad40102ef0 2024-09-10T08:48:20.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad44b2f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fad40110ff0 con 0x7fad40102570 2024-09-10T08:48:20.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad3e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad40100e20 0x7fad401992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad3e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad40100e20 0x7fad401992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55606/0 (socket says 172.21.15.172:55606) 2024-09-10T08:48:20.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad3e59c640 1 -- 172.21.15.172:0/2719113253 learned_addr learned my addr 172.21.15.172:0/2719113253 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:20.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad3dd9b640 1 --2- 172.21.15.172:0/2719113253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad40102570 0x7fad40199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad3ed9d640 1 --2- 172.21.15.172:0/2719113253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad40102ef0 0x7fad401a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad3e59c640 1 -- 172.21.15.172:0/2719113253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad40102570 msgr2=0x7fad40199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:20.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad3e59c640 1 --2- 172.21.15.172:0/2719113253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad40102570 0x7fad40199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad3e59c640 1 -- 172.21.15.172:0/2719113253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad40102ef0 msgr2=0x7fad401a4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:20.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad3e59c640 1 --2- 172.21.15.172:0/2719113253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad40102ef0 0x7fad401a4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad3e59c640 1 -- 172.21.15.172:0/2719113253 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad401a4880 con 0x7fad40100e20 2024-09-10T08:48:20.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad3ed9d640 1 --2- 172.21.15.172:0/2719113253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad40102ef0 0x7fad401a4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:20.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad3e59c640 1 --2- 172.21.15.172:0/2719113253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad40100e20 0x7fad401992c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fad3000c970 tx=0x7fad3000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:20.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad2f7fe640 1 -- 172.21.15.172:0/2719113253 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30007bb0 con 0x7fad40100e20 2024-09-10T08:48:20.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad2f7fe640 1 -- 172.21.15.172:0/2719113253 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad30007d50 con 0x7fad40100e20 2024-09-10T08:48:20.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad44b2f640 1 -- 172.21.15.172:0/2719113253 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad401a4a50 con 0x7fad40100e20 2024-09-10T08:48:20.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.499+0000 7fad2f7fe640 1 -- 172.21.15.172:0/2719113253 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30004e50 con 0x7fad40100e20 2024-09-10T08:48:20.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.500+0000 7fad44b2f640 1 -- 172.21.15.172:0/2719113253 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad401a4f10 con 0x7fad40100e20 2024-09-10T08:48:20.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.502+0000 7fad2f7fe640 1 -- 172.21.15.172:0/2719113253 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fad30005020 con 0x7fad40100e20 2024-09-10T08:48:20.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.502+0000 7fad46f08740 1 Processor -- start 2024-09-10T08:48:20.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.502+0000 7fad46f08740 1 -- start start 2024-09-10T08:48:20.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.502+0000 7fad46f08740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978db60 0x560eb978df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.502+0000 7fad46f08740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb978e480 0x560eb9798800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.502+0000 7fad2f7fe640 1 --2- 172.21.15.172:0/2719113253 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fad280a2d20 0x7fad280a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.502+0000 7fad46f08740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb9798d40 0x560eb979b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.502+0000 7fad46f08740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560eb96f3280 con 0x560eb9798d40 2024-09-10T08:48:20.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.502+0000 7fad46f08740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560eb969ab10 con 0x560eb978db60 2024-09-10T08:48:20.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.502+0000 7fad46f08740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560eb96ef240 con 0x560eb978e480 2024-09-10T08:48:20.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad3ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb9798d40 0x560eb979b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad3dd9b640 1 --2- 172.21.15.172:0/2719113253 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fad280a2d20 0x7fad280a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad3ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb9798d40 0x560eb979b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55618/0 (socket says 172.21.15.172:55618) 2024-09-10T08:48:20.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad3ed9d640 1 -- 172.21.15.172:0/634008919 learned_addr learned my addr 172.21.15.172:0/634008919 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:20.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad3e59c640 1 --2- 172.21.15.172:0/634008919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb978e480 0x560eb9798800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad3ed9d640 1 --2- 172.21.15.172:0/634008919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978db60 0x560eb978df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad2f7fe640 1 -- 172.21.15.172:0/2719113253 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fad300cf0d0 con 0x7fad40100e20 2024-09-10T08:48:20.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad3ed9d640 1 -- 172.21.15.172:0/634008919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb978e480 msgr2=0x560eb9798800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:20.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad3ed9d640 1 --2- 172.21.15.172:0/634008919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb978e480 0x560eb9798800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad3ed9d640 1 -- 172.21.15.172:0/634008919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978db60 msgr2=0x560eb978df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:20.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad3ed9d640 1 --2- 172.21.15.172:0/634008919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978db60 0x560eb978df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad3ed9d640 1 -- 172.21.15.172:0/634008919 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eb96a71f0 con 0x560eb9798d40 2024-09-10T08:48:20.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad3e59c640 1 --2- 172.21.15.172:0/634008919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb978e480 0x560eb9798800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:20.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad3dd9b640 1 --2- 172.21.15.172:0/2719113253 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fad280a2d20 0x7fad280a51e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fad4006aac0 tx=0x7fad34009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:20.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad3ed9d640 1 --2- 172.21.15.172:0/634008919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb9798d40 0x560eb979b130 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fad3802fcb0 tx=0x7fad38005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bab92534b7bb39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:20.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad2cff9640 1 -- 172.21.15.172:0/634008919 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad38034040 con 0x560eb9798d40 2024-09-10T08:48:20.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad2cff9640 1 -- 172.21.15.172:0/634008919 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad38003e20 con 0x560eb9798d40 2024-09-10T08:48:20.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.503+0000 7fad2cff9640 1 -- 172.21.15.172:0/634008919 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad38004330 con 0x560eb9798d40 2024-09-10T08:48:20.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.504+0000 7fad46f08740 1 -- 172.21.15.172:0/634008919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb9798d40 msgr2=0x560eb979b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:20.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.504+0000 7fad46f08740 1 --2- 172.21.15.172:0/634008919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb9798d40 0x560eb979b130 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fad3802fcb0 tx=0x7fad38005ef0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.504+0000 7fad46f08740 1 -- 172.21.15.172:0/634008919 shutdown_connections 2024-09-10T08:48:20.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.504+0000 7fad46f08740 1 --2- 172.21.15.172:0/634008919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb9798d40 0x560eb979b130 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.504+0000 7fad46f08740 1 --2- 172.21.15.172:0/634008919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb978e480 0x560eb9798800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.504+0000 7fad46f08740 1 --2- 172.21.15.172:0/634008919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978db60 0x560eb978df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.504+0000 7fad46f08740 1 -- 172.21.15.172:0/634008919 >> 172.21.15.172:0/634008919 conn(0x560eb978a0f0 msgr2=0x560eb978a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:20.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.504+0000 7fad46f08740 1 -- 172.21.15.172:0/634008919 shutdown_connections 2024-09-10T08:48:20.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.504+0000 7fad46f08740 1 -- 172.21.15.172:0/634008919 wait complete. 2024-09-10T08:48:20.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad46f08740 1 -- start start 2024-09-10T08:48:20.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad46f08740 1 Processor -- start 2024-09-10T08:48:20.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad46f08740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb978db60 0x560eb9795c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad46f08740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978e480 0x560eb9796150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad46f08740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb9798d40 0x560eb9796690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad46f08740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560eb979b910 con 0x560eb9798d40 2024-09-10T08:48:20.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad46f08740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560eb96adf10 con 0x560eb978e480 2024-09-10T08:48:20.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad46f08740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560eb979ba90 con 0x560eb978db60 2024-09-10T08:48:20.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad46f08740 10 client.0 _ll_register_callbacks cb 0x560eb9676a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:20.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad3ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb9798d40 0x560eb9796690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad3ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb9798d40 0x560eb9796690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55620/0 (socket says 172.21.15.172:55620) 2024-09-10T08:48:20.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad3ed9d640 1 -- 172.21.15.172:0/2012289273 learned_addr learned my addr 172.21.15.172:0/2012289273 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:20.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad3dd9b640 1 --2- 172.21.15.172:0/2012289273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978e480 0x560eb9796150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad3e59c640 1 --2- 172.21.15.172:0/2012289273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb978db60 0x560eb9795c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad3ed9d640 1 -- 172.21.15.172:0/2012289273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb978db60 msgr2=0x560eb9795c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:20.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad3ed9d640 1 --2- 172.21.15.172:0/2012289273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb978db60 0x560eb9795c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad3ed9d640 1 -- 172.21.15.172:0/2012289273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978e480 msgr2=0x560eb9796150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:20.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad3ed9d640 1 --2- 172.21.15.172:0/2012289273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978e480 0x560eb9796150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad3ed9d640 1 -- 172.21.15.172:0/2012289273 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eb94646a0 con 0x560eb9798d40 2024-09-10T08:48:20.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.505+0000 7fad3e59c640 1 --2- 172.21.15.172:0/2012289273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb978db60 0x560eb9795c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:20.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.506+0000 7fad3ed9d640 1 --2- 172.21.15.172:0/2012289273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb9798d40 0x560eb9796690 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fad38002230 tx=0x7fad38009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dacbb5697cc1ca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:20.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.506+0000 7fad25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:20.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.506+0000 7fad25ffb640 1 -- 172.21.15.172:0/2012289273 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad38034030 con 0x560eb9798d40 2024-09-10T08:48:20.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.506+0000 7fad25ffb640 1 -- 172.21.15.172:0/2012289273 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fad3804d850 con 0x560eb9798d40 2024-09-10T08:48:20.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.506+0000 7fad46f08740 1 -- 172.21.15.172:0/2012289273 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad38003e20 con 0x560eb9798d40 2024-09-10T08:48:20.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.506+0000 7fad25ffb640 1 -- 172.21.15.172:0/2012289273 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fad3805d050 con 0x560eb9798d40 2024-09-10T08:48:20.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.506+0000 7fad46f08740 10 client.16026 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:20.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.506+0000 7fad46f08740 1 -- 172.21.15.172:0/2012289273 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560eb979b910 con 0x560eb9798d40 2024-09-10T08:48:20.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.506+0000 7fad25ffb640 1 -- 172.21.15.172:0/2012289273 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad38003e20 con 0x560eb9798d40 2024-09-10T08:48:20.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.506+0000 7fad25ffb640 1 -- 172.21.15.172:0/2012289273 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fad380574c0 con 0x560eb9798d40 2024-09-10T08:48:20.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.506+0000 7fad46f08740 10 client.16026 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:20.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.506+0000 7fad46f08740 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:20.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.506+0000 7fad46f08740 20 client.16026 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:20.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.506+0000 7fad46f08740 1 --2- 172.21.15.172:0/2012289273 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560eb97a8690 0x560eb97aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.506+0000 7fad46f08740 4 client.16026 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:20.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.506+0000 7fad46f08740 1 -- 172.21.15.172:0/2012289273 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560eb97007f0 con 0x560eb97a8690 2024-09-10T08:48:20.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.507+0000 7fad3e59c640 1 --2- 172.21.15.172:0/2012289273 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560eb97a8690 0x560eb97aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.507+0000 7fad3e59c640 1 --2- 172.21.15.172:0/2012289273 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560eb97a8690 0x560eb97aaa80 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25a720a768058f4 server_cookie=3719e87faf7770d2 in_seq=0 out_seq=0 2024-09-10T08:48:20.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.507+0000 7fad25ffb640 10 client.16026 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:20.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.508+0000 7fad25ffb640 1 -- 172.21.15.172:0/2012289273 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560eb97007f0 con 0x560eb97a8690 2024-09-10T08:48:20.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.508+0000 7fad25ffb640 10 client.16026 handle_command_reply: tid=0 2024-09-10T08:48:20.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.508+0000 7fad46f08740 1 client.16026 shutdown 2024-09-10T08:48:20.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.508+0000 7fad46f08740 1 -- 172.21.15.172:0/2012289273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb9798d40 msgr2=0x560eb9796690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:20.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.508+0000 7fad46f08740 1 --2- 172.21.15.172:0/2012289273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb9798d40 0x560eb9796690 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fad38002230 tx=0x7fad38009990 comp rx=0 tx=0).stop 2024-09-10T08:48:20.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.508+0000 7fad46f08740 1 -- 172.21.15.172:0/2012289273 shutdown_connections 2024-09-10T08:48:20.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.508+0000 7fad46f08740 1 --2- 172.21.15.172:0/2012289273 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560eb97a8690 0x560eb97aaa80 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.508+0000 7fad46f08740 1 --2- 172.21.15.172:0/2012289273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb9798d40 0x560eb9796690 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.508+0000 7fad46f08740 1 --2- 172.21.15.172:0/2012289273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978e480 0x560eb9796150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.508+0000 7fad46f08740 1 --2- 172.21.15.172:0/2012289273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb978db60 0x560eb9795c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.508+0000 7fad25ffb640 0 client.16026 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:20.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.508+0000 7fad46f08740 1 -- 172.21.15.172:0/2012289273 >> 172.21.15.172:0/2012289273 conn(0x560eb978a0f0 msgr2=0x560eb978fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:20.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.508+0000 7fad46f08740 1 -- 172.21.15.172:0/2012289273 shutdown_connections 2024-09-10T08:48:20.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.508+0000 7fad46f08740 1 -- 172.21.15.172:0/2012289273 wait complete. 2024-09-10T08:48:20.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.508+0000 7fad46f08740 20 client.16026 trim_cache size 0 max 16384 2024-09-10T08:48:20.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.525+0000 7fad46f08740 1 Processor -- start 2024-09-10T08:48:20.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.525+0000 7fad46f08740 1 -- start start 2024-09-10T08:48:20.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.525+0000 7fad46f08740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978db60 0x560eb9797e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.525+0000 7fad46f08740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb978e480 0x560eb97964e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.525+0000 7fad46f08740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb9798d40 0x560eb97b0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.525+0000 7fad46f08740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fad401ac9d0 con 0x560eb978e480 2024-09-10T08:48:20.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.525+0000 7fad46f08740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fad401ae510 con 0x560eb978db60 2024-09-10T08:48:20.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.525+0000 7fad46f08740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fad401ad0a0 con 0x560eb9798d40 2024-09-10T08:48:20.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.525+0000 7fad3dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb978e480 0x560eb97964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.525+0000 7fad3dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb978e480 0x560eb97964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55626/0 (socket says 172.21.15.172:55626) 2024-09-10T08:48:20.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.525+0000 7fad3dd9b640 1 -- 172.21.15.172:0/2069236943 learned_addr learned my addr 172.21.15.172:0/2069236943 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:20.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.526+0000 7fad3e59c640 1 --2- 172.21.15.172:0/2069236943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978db60 0x560eb9797e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.526+0000 7fad3ed9d640 1 --2- 172.21.15.172:0/2069236943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb9798d40 0x560eb97b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.526+0000 7fad3dd9b640 1 -- 172.21.15.172:0/2069236943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb9798d40 msgr2=0x560eb97b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:20.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.526+0000 7fad3dd9b640 1 --2- 172.21.15.172:0/2069236943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb9798d40 0x560eb97b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.526+0000 7fad3dd9b640 1 -- 172.21.15.172:0/2069236943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978db60 msgr2=0x560eb9797e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:20.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.526+0000 7fad3dd9b640 1 --2- 172.21.15.172:0/2069236943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978db60 0x560eb9797e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.526+0000 7fad3dd9b640 1 -- 172.21.15.172:0/2069236943 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eb978aca0 con 0x560eb978e480 2024-09-10T08:48:20.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.526+0000 7fad3dd9b640 1 --2- 172.21.15.172:0/2069236943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb978e480 0x560eb97964e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fad3400ec60 tx=0x7fad34011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ce76bbdc7144f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:20.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.526+0000 7fad27fff640 1 -- 172.21.15.172:0/2069236943 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3401b040 con 0x560eb978e480 2024-09-10T08:48:20.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.526+0000 7fad27fff640 1 -- 172.21.15.172:0/2069236943 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad34026070 con 0x560eb978e480 2024-09-10T08:48:20.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.526+0000 7fad27fff640 1 -- 172.21.15.172:0/2069236943 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad340136b0 con 0x560eb978e480 2024-09-10T08:48:20.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 1 -- 172.21.15.172:0/2069236943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb978e480 msgr2=0x560eb97964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:20.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 1 --2- 172.21.15.172:0/2069236943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb978e480 0x560eb97964e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fad3400ec60 tx=0x7fad34011000 comp rx=0 tx=0).stop 2024-09-10T08:48:20.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 1 -- 172.21.15.172:0/2069236943 shutdown_connections 2024-09-10T08:48:20.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 1 --2- 172.21.15.172:0/2069236943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb9798d40 0x560eb97b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 1 --2- 172.21.15.172:0/2069236943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb978e480 0x560eb97964e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 1 --2- 172.21.15.172:0/2069236943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978db60 0x560eb9797e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 1 -- 172.21.15.172:0/2069236943 >> 172.21.15.172:0/2069236943 conn(0x560eb978a0f0 msgr2=0x560eb978fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:20.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 1 -- 172.21.15.172:0/2069236943 shutdown_connections 2024-09-10T08:48:20.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 1 -- 172.21.15.172:0/2069236943 wait complete. 2024-09-10T08:48:20.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 1 -- start start 2024-09-10T08:48:20.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 1 Processor -- start 2024-09-10T08:48:20.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978db60 0x560eb979cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb978e480 0x560eb979d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb9798d40 0x560eb979d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fad401ade40 con 0x560eb978e480 2024-09-10T08:48:20.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fad401ad770 con 0x560eb978db60 2024-09-10T08:48:20.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fad401aa9b0 con 0x560eb9798d40 2024-09-10T08:48:20.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.527+0000 7fad46f08740 10 client.0 _ll_register_callbacks cb 0x7fad3802f8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:20.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad3dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb978e480 0x560eb979d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad3e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978db60 0x560eb979cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad3ed9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb9798d40 0x560eb979d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad3dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb978e480 0x560eb979d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55634/0 (socket says 172.21.15.172:55634) 2024-09-10T08:48:20.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad3dd9b640 1 -- 172.21.15.172:0/3081437378 learned_addr learned my addr 172.21.15.172:0/3081437378 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:20.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad3ed9d640 1 -- 172.21.15.172:0/3081437378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978db60 msgr2=0x560eb979cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:20.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad3ed9d640 1 --2- 172.21.15.172:0/3081437378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978db60 0x560eb979cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad3ed9d640 1 -- 172.21.15.172:0/3081437378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb978e480 msgr2=0x560eb979d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:20.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad3ed9d640 1 --2- 172.21.15.172:0/3081437378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb978e480 0x560eb979d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad3ed9d640 1 -- 172.21.15.172:0/3081437378 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eb9812250 con 0x560eb9798d40 2024-09-10T08:48:20.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad3e59c640 1 --2- 172.21.15.172:0/3081437378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978db60 0x560eb979cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:20.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad3dd9b640 1 --2- 172.21.15.172:0/3081437378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb978e480 0x560eb979d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:20.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad3ed9d640 1 --2- 172.21.15.172:0/3081437378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb9798d40 0x560eb979d620 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fad38006010 tx=0x7fad3802fee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f605c1cc0fd8729 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:20.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:48:20.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad25ffb640 1 -- 172.21.15.172:0/3081437378 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad38034040 con 0x560eb9798d40 2024-09-10T08:48:20.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad46f08740 1 -- 172.21.15.172:0/3081437378 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad34026070 con 0x560eb9798d40 2024-09-10T08:48:20.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad25ffb640 1 -- 172.21.15.172:0/3081437378 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fad38057110 con 0x560eb9798d40 2024-09-10T08:48:20.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.528+0000 7fad25ffb640 1 -- 172.21.15.172:0/3081437378 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fad38057cc0 con 0x560eb9798d40 2024-09-10T08:48:20.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.529+0000 7fad46f08740 10 client.25363 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:20.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.529+0000 7fad46f08740 1 -- 172.21.15.172:0/3081437378 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560eb9813550 con 0x560eb9798d40 2024-09-10T08:48:20.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.529+0000 7fad25ffb640 1 -- 172.21.15.172:0/3081437378 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad34026070 con 0x560eb9798d40 2024-09-10T08:48:20.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.529+0000 7fad25ffb640 1 -- 172.21.15.172:0/3081437378 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fad38030440 con 0x560eb9798d40 2024-09-10T08:48:20.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.529+0000 7fad46f08740 10 client.25363 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:20.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.529+0000 7fad46f08740 10 client.25363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:20.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.529+0000 7fad46f08740 20 client.25363 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:20.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.529+0000 7fad46f08740 1 --2- 172.21.15.172:0/3081437378 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560eb9813ad0 0x560eb9815ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:20.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.529+0000 7fad46f08740 4 client.25363 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:48:20.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.529+0000 7fad46f08740 1 -- 172.21.15.172:0/3081437378 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560eb96fad00 con 0x560eb9813ad0 2024-09-10T08:48:20.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.529+0000 7fad3e59c640 1 --2- 172.21.15.172:0/3081437378 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560eb9813ad0 0x560eb9815ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:20.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.530+0000 7fad3e59c640 1 --2- 172.21.15.172:0/3081437378 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560eb9813ad0 0x560eb9815ec0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69b7fc3ab5428d18 server_cookie=db0641260328b823 in_seq=0 out_seq=0 2024-09-10T08:48:20.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.530+0000 7fad25ffb640 10 client.25363 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.531+0000 7fad25ffb640 1 -- 172.21.15.172:0/3081437378 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560eb96fad00 con 0x560eb9813ad0 2024-09-10T08:48:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.531+0000 7fad25ffb640 10 client.25363 handle_command_reply: tid=0 2024-09-10T08:48:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.531+0000 7fad46f08740 1 client.25363 shutdown 2024-09-10T08:48:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.531+0000 7fad46f08740 1 -- 172.21.15.172:0/3081437378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb9798d40 msgr2=0x560eb979d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.531+0000 7fad46f08740 1 --2- 172.21.15.172:0/3081437378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb9798d40 0x560eb979d620 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fad38006010 tx=0x7fad3802fee0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.531+0000 7fad46f08740 1 -- 172.21.15.172:0/3081437378 shutdown_connections 2024-09-10T08:48:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.531+0000 7fad46f08740 1 --2- 172.21.15.172:0/3081437378 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560eb9813ad0 0x560eb9815ec0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.531+0000 7fad46f08740 1 --2- 172.21.15.172:0/3081437378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560eb9798d40 0x560eb979d620 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.531+0000 7fad46f08740 1 --2- 172.21.15.172:0/3081437378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560eb978e480 0x560eb979d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.531+0000 7fad46f08740 1 --2- 172.21.15.172:0/3081437378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560eb978db60 0x560eb979cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.531+0000 7fad25ffb640 0 client.25363 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.531+0000 7fad46f08740 1 -- 172.21.15.172:0/3081437378 >> 172.21.15.172:0/3081437378 conn(0x560eb978a0f0 msgr2=0x560eb978fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:20.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.531+0000 7fad46f08740 1 -- 172.21.15.172:0/3081437378 shutdown_connections 2024-09-10T08:48:20.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.531+0000 7fad46f08740 1 -- 172.21.15.172:0/3081437378 wait complete. 2024-09-10T08:48:20.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.531+0000 7fad46f08740 20 client.25363 trim_cache size 0 max 16384 2024-09-10T08:48:20.537 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:48:20.537 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T08:48:20.537 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "4c22940a-7c0e-4cd3-9246-e330afa859f8", 2024-09-10T08:48:20.537 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T08:48:20.537 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:48:20.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.532+0000 7fad44b2f640 1 -- 172.21.15.172:0/2719113253 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fad280a2d20 msgr2=0x7fad280a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:20.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.532+0000 7fad44b2f640 1 --2- 172.21.15.172:0/2719113253 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fad280a2d20 0x7fad280a51e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fad4006aac0 tx=0x7fad34009110 comp rx=0 tx=0).stop 2024-09-10T08:48:20.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.532+0000 7fad44b2f640 1 -- 172.21.15.172:0/2719113253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad40100e20 msgr2=0x7fad401992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:20.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.532+0000 7fad44b2f640 1 --2- 172.21.15.172:0/2719113253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad40100e20 0x7fad401992c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fad3000c970 tx=0x7fad3000cd70 comp rx=0 tx=0).stop 2024-09-10T08:48:20.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.532+0000 7fad44b2f640 1 -- 172.21.15.172:0/2719113253 shutdown_connections 2024-09-10T08:48:20.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.532+0000 7fad44b2f640 1 --2- 172.21.15.172:0/2719113253 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fad280a2d20 0x7fad280a51e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.532+0000 7fad44b2f640 1 --2- 172.21.15.172:0/2719113253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad40102ef0 0x7fad401a4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.532+0000 7fad44b2f640 1 --2- 172.21.15.172:0/2719113253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad40102570 0x7fad40199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.532+0000 7fad44b2f640 1 --2- 172.21.15.172:0/2719113253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad40100e20 0x7fad401992c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:20.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.532+0000 7fad44b2f640 1 -- 172.21.15.172:0/2719113253 >> 172.21.15.172:0/2719113253 conn(0x7fad400fc7d0 msgr2=0x7fad400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:20.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.532+0000 7fad44b2f640 1 -- 172.21.15.172:0/2719113253 shutdown_connections 2024-09-10T08:48:20.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:20.532+0000 7fad44b2f640 1 -- 172.21.15.172:0/2719113253 wait complete. 2024-09-10T08:48:20.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:20 smithi191 ceph-mon[35314]: pgmap v413: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T08:48:20.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:20 smithi174 ceph-mon[34068]: pgmap v413: 129 pgs: 129 active+clean; 674 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T08:48:21.367 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:48:21.616 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:22.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.090+0000 7f32de650640 1 -- 172.21.15.172:0/1327898534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32d8106f10 msgr2=0x7f32d810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:22.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.090+0000 7f32de650640 1 --2- 172.21.15.172:0/1327898534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32d8106f10 0x7f32d810ea90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009ca0 tx=0x7f32d002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.090+0000 7f32de650640 1 -- 172.21.15.172:0/1327898534 shutdown_connections 2024-09-10T08:48:22.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.090+0000 7f32de650640 1 --2- 172.21.15.172:0/1327898534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32d8106f10 0x7f32d810ea90 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.090+0000 7f32de650640 1 --2- 172.21.15.172:0/1327898534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32d8102790 0x7f32d81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.090+0000 7f32de650640 1 --2- 172.21.15.172:0/1327898534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32d8101de0 0x7f32d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.090+0000 7f32de650640 1 -- 172.21.15.172:0/1327898534 >> 172.21.15.172:0/1327898534 conn(0x7f32d80fd8c0 msgr2=0x7f32d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:22.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.090+0000 7f32de650640 1 -- 172.21.15.172:0/1327898534 shutdown_connections 2024-09-10T08:48:22.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.090+0000 7f32de650640 1 -- 172.21.15.172:0/1327898534 wait complete. 2024-09-10T08:48:22.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.090+0000 7f32de650640 1 Processor -- start 2024-09-10T08:48:22.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.090+0000 7f32de650640 1 -- start start 2024-09-10T08:48:22.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32de650640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32d8101de0 0x7f32d8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32de650640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32d8102790 0x7f32d8198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32de650640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32d8106f10 0x7f32d819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32de650640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f32d8111e20 con 0x7f32d8102790 2024-09-10T08:48:22.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32de650640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f32d8111ca0 con 0x7f32d8101de0 2024-09-10T08:48:22.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32de650640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f32d8111fa0 con 0x7f32d8106f10 2024-09-10T08:48:22.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32d77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32d8102790 0x7f32d8198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:22.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32d77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32d8102790 0x7f32d8198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55652/0 (socket says 172.21.15.172:55652) 2024-09-10T08:48:22.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32d77fe640 1 -- 172.21.15.172:0/4015315457 learned_addr learned my addr 172.21.15.172:0/4015315457 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:22.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32d7fff640 1 --2- 172.21.15.172:0/4015315457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32d8101de0 0x7f32d8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:22.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32dcbed640 1 --2- 172.21.15.172:0/4015315457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32d8106f10 0x7f32d819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:22.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32d77fe640 1 -- 172.21.15.172:0/4015315457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32d8106f10 msgr2=0x7f32d819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:22.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32d77fe640 1 --2- 172.21.15.172:0/4015315457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32d8106f10 0x7f32d819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32d77fe640 1 -- 172.21.15.172:0/4015315457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32d8101de0 msgr2=0x7f32d8198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:22.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32d77fe640 1 --2- 172.21.15.172:0/4015315457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32d8101de0 0x7f32d8198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32d77fe640 1 -- 172.21.15.172:0/4015315457 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d819d2f0 con 0x7f32d8102790 2024-09-10T08:48:22.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32d7fff640 1 --2- 172.21.15.172:0/4015315457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32d8101de0 0x7f32d8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:22.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32dcbed640 1 --2- 172.21.15.172:0/4015315457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32d8106f10 0x7f32d819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:22.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.091+0000 7f32d77fe640 1 --2- 172.21.15.172:0/4015315457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32d8102790 0x7f32d8198880 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c970 tx=0x7f32c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:22.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.092+0000 7f32d57fa640 1 -- 172.21.15.172:0/4015315457 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0007bb0 con 0x7f32d8102790 2024-09-10T08:48:22.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.092+0000 7f32de650640 1 -- 172.21.15.172:0/4015315457 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d819d5e0 con 0x7f32d8102790 2024-09-10T08:48:22.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.092+0000 7f32d57fa640 1 -- 172.21.15.172:0/4015315457 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32c0007d50 con 0x7f32d8102790 2024-09-10T08:48:22.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.092+0000 7f32d57fa640 1 -- 172.21.15.172:0/4015315457 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c000f450 con 0x7f32d8102790 2024-09-10T08:48:22.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.092+0000 7f32de650640 1 -- 172.21.15.172:0/4015315457 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d81a4ec0 con 0x7f32d8102790 2024-09-10T08:48:22.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.094+0000 7f32d57fa640 1 -- 172.21.15.172:0/4015315457 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f32c0004f20 con 0x7f32d8102790 2024-09-10T08:48:22.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.094+0000 7f32e0a29740 1 Processor -- start 2024-09-10T08:48:22.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.094+0000 7f32e0a29740 1 -- start start 2024-09-10T08:48:22.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.094+0000 7f32e0a29740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d77374a0 0x5652d7737880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.094+0000 7f32e0a29740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7737dc0 0x5652d7742140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.094+0000 7f32e0a29740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d7742680 0x5652d7744a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.094+0000 7f32e0a29740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5652d769d2e0 con 0x5652d7737dc0 2024-09-10T08:48:22.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.094+0000 7f32e0a29740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5652d7644ce0 con 0x5652d77374a0 2024-09-10T08:48:22.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.094+0000 7f32e0a29740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5652d76992a0 con 0x5652d7742680 2024-09-10T08:48:22.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.094+0000 7f32d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7737dc0 0x5652d7742140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:22.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.095+0000 7f32d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7737dc0 0x5652d7742140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55660/0 (socket says 172.21.15.172:55660) 2024-09-10T08:48:22.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.095+0000 7f32d7fff640 1 -- 172.21.15.172:0/3481849957 learned_addr learned my addr 172.21.15.172:0/3481849957 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:22.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.095+0000 7f32d77fe640 1 --2- 172.21.15.172:0/3481849957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d7742680 0x5652d7744a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:22.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.095+0000 7f32dcbed640 1 --2- 172.21.15.172:0/3481849957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d77374a0 0x5652d7737880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:22.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.095+0000 7f32d57fa640 1 --2- 172.21.15.172:0/4015315457 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f32c40a2b80 0x7f32c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.095+0000 7f32d7fff640 1 -- 172.21.15.172:0/3481849957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d7742680 msgr2=0x5652d7744a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:22.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.095+0000 7f32d7fff640 1 --2- 172.21.15.172:0/3481849957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d7742680 0x5652d7744a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.095+0000 7f32d7fff640 1 -- 172.21.15.172:0/3481849957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d77374a0 msgr2=0x5652d7737880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:22.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.095+0000 7f32d7fff640 1 --2- 172.21.15.172:0/3481849957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d77374a0 0x5652d7737880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.095+0000 7f32d7fff640 1 -- 172.21.15.172:0/3481849957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652d7629440 con 0x5652d7737dc0 2024-09-10T08:48:22.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.095+0000 7f32d77fe640 1 --2- 172.21.15.172:0/3481849957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d7742680 0x5652d7744a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:22.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.095+0000 7f32dcbed640 1 --2- 172.21.15.172:0/4015315457 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f32c40a2b80 0x7f32c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:22.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.095+0000 7f32d57fa640 1 -- 172.21.15.172:0/4015315457 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f32c00cf140 con 0x7f32d8102790 2024-09-10T08:48:22.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.095+0000 7f32d7fff640 1 --2- 172.21.15.172:0/3481849957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7737dc0 0x5652d7742140 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009c50 tx=0x7f32c800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36af426f7de43af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:22.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.095+0000 7f32ce7fc640 1 -- 172.21.15.172:0/3481849957 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8016070 con 0x5652d7737dc0 2024-09-10T08:48:22.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.095+0000 7f32ce7fc640 1 -- 172.21.15.172:0/3481849957 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32c80054d0 con 0x5652d7737dc0 2024-09-10T08:48:22.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.095+0000 7f32ce7fc640 1 -- 172.21.15.172:0/3481849957 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8011470 con 0x5652d7737dc0 2024-09-10T08:48:22.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.096+0000 7f32e0a29740 1 -- 172.21.15.172:0/3481849957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7737dc0 msgr2=0x5652d7742140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:22.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.096+0000 7f32e0a29740 1 --2- 172.21.15.172:0/3481849957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7737dc0 0x5652d7742140 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009c50 tx=0x7f32c800e6c0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.096+0000 7f32dcbed640 1 --2- 172.21.15.172:0/4015315457 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f32c40a2b80 0x7f32c40a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009a60 tx=0x7f32d002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:22.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.096+0000 7f32e0a29740 1 -- 172.21.15.172:0/3481849957 shutdown_connections 2024-09-10T08:48:22.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.096+0000 7f32e0a29740 1 --2- 172.21.15.172:0/3481849957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d7742680 0x5652d7744a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.096+0000 7f32e0a29740 1 --2- 172.21.15.172:0/3481849957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7737dc0 0x5652d7742140 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.096+0000 7f32e0a29740 1 --2- 172.21.15.172:0/3481849957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d77374a0 0x5652d7737880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.096+0000 7f32e0a29740 1 -- 172.21.15.172:0/3481849957 >> 172.21.15.172:0/3481849957 conn(0x5652d7733a30 msgr2=0x5652d7733e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:22.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.096+0000 7f32e0a29740 1 -- 172.21.15.172:0/3481849957 shutdown_connections 2024-09-10T08:48:22.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.096+0000 7f32e0a29740 1 -- 172.21.15.172:0/3481849957 wait complete. 2024-09-10T08:48:22.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.096+0000 7f32e0a29740 1 -- start start 2024-09-10T08:48:22.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32e0a29740 1 Processor -- start 2024-09-10T08:48:22.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32e0a29740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d77374a0 0x5652d7741f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32e0a29740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7737dc0 0x5652d773f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32e0a29740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d7742680 0x5652d773f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32e0a29740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5652d7745250 con 0x5652d7737dc0 2024-09-10T08:48:22.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32e0a29740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5652d7658050 con 0x5652d7742680 2024-09-10T08:48:22.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32e0a29740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5652d77453d0 con 0x5652d77374a0 2024-09-10T08:48:22.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32e0a29740 10 client.0 _ll_register_callbacks cb 0x5652d7698160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:22.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32d77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7737dc0 0x5652d773f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:22.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32d77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7737dc0 0x5652d773f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55670/0 (socket says 172.21.15.172:55670) 2024-09-10T08:48:22.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32d77fe640 1 -- 172.21.15.172:0/2801414613 learned_addr learned my addr 172.21.15.172:0/2801414613 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:22.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32d7fff640 1 --2- 172.21.15.172:0/2801414613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d77374a0 0x5652d7741f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:22.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32dcbed640 1 --2- 172.21.15.172:0/2801414613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d7742680 0x5652d773f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:22.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32d77fe640 1 -- 172.21.15.172:0/2801414613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d77374a0 msgr2=0x5652d7741f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:22.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32d77fe640 1 --2- 172.21.15.172:0/2801414613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d77374a0 0x5652d7741f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32d77fe640 1 -- 172.21.15.172:0/2801414613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d7742680 msgr2=0x5652d773f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:22.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32d77fe640 1 --2- 172.21.15.172:0/2801414613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d7742680 0x5652d773f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32d77fe640 1 -- 172.21.15.172:0/2801414613 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652d740e3c0 con 0x5652d7737dc0 2024-09-10T08:48:22.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32dcbed640 1 --2- 172.21.15.172:0/2801414613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d7742680 0x5652d773f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:22.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32d77fe640 1 --2- 172.21.15.172:0/2801414613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7737dc0 0x5652d773f1b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f32c009b5d0 tx=0x7f32c0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f7d98b2df79d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:22.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.097+0000 7f32b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:22.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.098+0000 7f32b77fe640 1 -- 172.21.15.172:0/2801414613 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c001d040 con 0x5652d7737dc0 2024-09-10T08:48:22.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.098+0000 7f32b77fe640 1 -- 172.21.15.172:0/2801414613 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f32c0084bd0 con 0x5652d7737dc0 2024-09-10T08:48:22.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.098+0000 7f32e0a29740 1 -- 172.21.15.172:0/2801414613 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32c80054d0 con 0x5652d7737dc0 2024-09-10T08:48:22.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.098+0000 7f32b77fe640 1 -- 172.21.15.172:0/2801414613 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f32c009bd40 con 0x5652d7737dc0 2024-09-10T08:48:22.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.098+0000 7f32e0a29740 10 client.16044 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:22.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.098+0000 7f32e0a29740 1 -- 172.21.15.172:0/2801414613 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5652d7750b60 con 0x5652d7737dc0 2024-09-10T08:48:22.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.098+0000 7f32b77fe640 1 -- 172.21.15.172:0/2801414613 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32c80054d0 con 0x5652d7737dc0 2024-09-10T08:48:22.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.098+0000 7f32b77fe640 1 -- 172.21.15.172:0/2801414613 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f32c00a3e20 con 0x5652d7737dc0 2024-09-10T08:48:22.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.098+0000 7f32e0a29740 10 client.16044 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:22.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.098+0000 7f32e0a29740 10 client.16044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:22.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.098+0000 7f32e0a29740 20 client.16044 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:22.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.098+0000 7f32e0a29740 1 --2- 172.21.15.172:0/2801414613 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5652d7752020 0x5652d7754410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.098+0000 7f32e0a29740 4 client.16044 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.098+0000 7f32e0a29740 1 -- 172.21.15.172:0/2801414613 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652d76aa850 con 0x5652d7752020 2024-09-10T08:48:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.098+0000 7f32d7fff640 1 --2- 172.21.15.172:0/2801414613 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5652d7752020 0x5652d7754410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.099+0000 7f32d7fff640 1 --2- 172.21.15.172:0/2801414613 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5652d7752020 0x5652d7754410 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fe51fccf7828138 server_cookie=b7521d0402983863 in_seq=0 out_seq=0 2024-09-10T08:48:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.099+0000 7f32b77fe640 10 client.16044 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.099+0000 7f32b77fe640 1 -- 172.21.15.172:0/2801414613 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652d76aa850 con 0x5652d7752020 2024-09-10T08:48:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.099+0000 7f32b77fe640 10 client.16044 handle_command_reply: tid=0 2024-09-10T08:48:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.099+0000 7f32e0a29740 1 client.16044 shutdown 2024-09-10T08:48:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.100+0000 7f32e0a29740 1 -- 172.21.15.172:0/2801414613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7737dc0 msgr2=0x5652d773f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:22.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.100+0000 7f32e0a29740 1 --2- 172.21.15.172:0/2801414613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7737dc0 0x5652d773f1b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f32c009b5d0 tx=0x7f32c0005720 comp rx=0 tx=0).stop 2024-09-10T08:48:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.100+0000 7f32e0a29740 1 -- 172.21.15.172:0/2801414613 shutdown_connections 2024-09-10T08:48:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.100+0000 7f32e0a29740 1 --2- 172.21.15.172:0/2801414613 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5652d7752020 0x5652d7754410 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.100+0000 7f32e0a29740 1 --2- 172.21.15.172:0/2801414613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d7742680 0x5652d773f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.100+0000 7f32e0a29740 1 --2- 172.21.15.172:0/2801414613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7737dc0 0x5652d773f1b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.100+0000 7f32b77fe640 0 client.16044 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.100+0000 7f32e0a29740 1 --2- 172.21.15.172:0/2801414613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d77374a0 0x5652d7741f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.100+0000 7f32e0a29740 1 -- 172.21.15.172:0/2801414613 >> 172.21.15.172:0/2801414613 conn(0x5652d7733a30 msgr2=0x5652d7739760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.100+0000 7f32e0a29740 1 -- 172.21.15.172:0/2801414613 shutdown_connections 2024-09-10T08:48:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.100+0000 7f32e0a29740 1 -- 172.21.15.172:0/2801414613 wait complete. 2024-09-10T08:48:22.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.100+0000 7f32e0a29740 20 client.16044 trim_cache size 0 max 16384 2024-09-10T08:48:22.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.117+0000 7f32e0a29740 1 Processor -- start 2024-09-10T08:48:22.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.117+0000 7f32e0a29740 1 -- start start 2024-09-10T08:48:22.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.117+0000 7f32e0a29740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d77374a0 0x5652d774e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.117+0000 7f32e0a29740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d7737dc0 0x5652d77329c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.117+0000 7f32e0a29740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d7742680 0x5652d774a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.117+0000 7f32e0a29740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81aeb20 con 0x5652d77374a0 2024-09-10T08:48:22.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.117+0000 7f32e0a29740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81b0660 con 0x5652d7737dc0 2024-09-10T08:48:22.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.117+0000 7f32e0a29740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81af1f0 con 0x5652d7742680 2024-09-10T08:48:22.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.117+0000 7f32d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d77374a0 0x5652d774e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:22.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.117+0000 7f32d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d77374a0 0x5652d774e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55680/0 (socket says 172.21.15.172:55680) 2024-09-10T08:48:22.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.117+0000 7f32d7fff640 1 -- 172.21.15.172:0/2079842350 learned_addr learned my addr 172.21.15.172:0/2079842350 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:22.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.118+0000 7f32d77fe640 1 --2- 172.21.15.172:0/2079842350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d7737dc0 0x5652d77329c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:22.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.118+0000 7f32d7fff640 1 -- 172.21.15.172:0/2079842350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d7742680 msgr2=0x5652d774a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:22.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.118+0000 7f32d7fff640 1 --2- 172.21.15.172:0/2079842350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d7742680 0x5652d774a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.118+0000 7f32d7fff640 1 -- 172.21.15.172:0/2079842350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d7737dc0 msgr2=0x5652d77329c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:22.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.118+0000 7f32d7fff640 1 --2- 172.21.15.172:0/2079842350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d7737dc0 0x5652d77329c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.118+0000 7f32d7fff640 1 -- 172.21.15.172:0/2079842350 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652d7742270 con 0x5652d77374a0 2024-09-10T08:48:22.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.118+0000 7f32d77fe640 1 --2- 172.21.15.172:0/2079842350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d7737dc0 0x5652d77329c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:22.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.118+0000 7f32d7fff640 1 --2- 172.21.15.172:0/2079842350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d77374a0 0x5652d774e4b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f32c800f010 tx=0x7f32c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff061517d499af1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:22.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.118+0000 7f32cdffb640 1 -- 172.21.15.172:0/2079842350 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8016040 con 0x5652d77374a0 2024-09-10T08:48:22.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.118+0000 7f32cdffb640 1 -- 172.21.15.172:0/2079842350 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32c80054b0 con 0x5652d77374a0 2024-09-10T08:48:22.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.118+0000 7f32cdffb640 1 -- 172.21.15.172:0/2079842350 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8011470 con 0x5652d77374a0 2024-09-10T08:48:22.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.119+0000 7f32e0a29740 1 -- 172.21.15.172:0/2079842350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d77374a0 msgr2=0x5652d774e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:22.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.119+0000 7f32e0a29740 1 --2- 172.21.15.172:0/2079842350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d77374a0 0x5652d774e4b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f32c800f010 tx=0x7f32c8004ab0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.119+0000 7f32e0a29740 1 -- 172.21.15.172:0/2079842350 shutdown_connections 2024-09-10T08:48:22.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.119+0000 7f32e0a29740 1 --2- 172.21.15.172:0/2079842350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d7742680 0x5652d774a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.119+0000 7f32e0a29740 1 --2- 172.21.15.172:0/2079842350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d7737dc0 0x5652d77329c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.119+0000 7f32e0a29740 1 --2- 172.21.15.172:0/2079842350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d77374a0 0x5652d774e4b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.119+0000 7f32e0a29740 1 -- 172.21.15.172:0/2079842350 >> 172.21.15.172:0/2079842350 conn(0x5652d7733a30 msgr2=0x5652d7741230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:22.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.119+0000 7f32e0a29740 1 -- 172.21.15.172:0/2079842350 shutdown_connections 2024-09-10T08:48:22.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.119+0000 7f32e0a29740 1 -- 172.21.15.172:0/2079842350 wait complete. 2024-09-10T08:48:22.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.119+0000 7f32e0a29740 1 -- start start 2024-09-10T08:48:22.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.119+0000 7f32e0a29740 1 Processor -- start 2024-09-10T08:48:22.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.119+0000 7f32e0a29740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d77374a0 0x5652d774a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.119+0000 7f32e0a29740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d7737dc0 0x5652d774acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.119+0000 7f32e0a29740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7742680 0x5652d7748b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32e0a29740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81aff90 con 0x5652d7742680 2024-09-10T08:48:22.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32e0a29740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81af8c0 con 0x5652d7737dc0 2024-09-10T08:48:22.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32e0a29740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81acb00 con 0x5652d77374a0 2024-09-10T08:48:22.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32e0a29740 10 client.0 _ll_register_callbacks cb 0x7f32c00058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:22.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32dcbed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7742680 0x5652d7748b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:22.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32dcbed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7742680 0x5652d7748b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55690/0 (socket says 172.21.15.172:55690) 2024-09-10T08:48:22.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32d7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d77374a0 0x5652d774a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:22.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32dcbed640 1 -- 172.21.15.172:0/1497628920 learned_addr learned my addr 172.21.15.172:0/1497628920 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:22.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32d77fe640 1 --2- 172.21.15.172:0/1497628920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d7737dc0 0x5652d774acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:22.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32dcbed640 1 -- 172.21.15.172:0/1497628920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d77374a0 msgr2=0x5652d774a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:22.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32dcbed640 1 --2- 172.21.15.172:0/1497628920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d77374a0 0x5652d774a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32dcbed640 1 -- 172.21.15.172:0/1497628920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d7737dc0 msgr2=0x5652d774acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:22.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32dcbed640 1 --2- 172.21.15.172:0/1497628920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d7737dc0 0x5652d774acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32dcbed640 1 -- 172.21.15.172:0/1497628920 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652d77493a0 con 0x5652d7742680 2024-09-10T08:48:22.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32dcbed640 1 --2- 172.21.15.172:0/1497628920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7742680 0x5652d7748b70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009ca0 tx=0x7f32d00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5f8a74237b2413 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:22.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:22.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32b77fe640 1 -- 172.21.15.172:0/1497628920 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d004f070 con 0x5652d7742680 2024-09-10T08:48:22.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32e0a29740 1 -- 172.21.15.172:0/1497628920 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32c80054b0 con 0x5652d7742680 2024-09-10T08:48:22.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32b77fe640 1 -- 172.21.15.172:0/1497628920 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f32d0004720 con 0x5652d7742680 2024-09-10T08:48:22.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.120+0000 7f32b77fe640 1 -- 172.21.15.172:0/1497628920 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f32d005b050 con 0x5652d7742680 2024-09-10T08:48:22.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.121+0000 7f32e0a29740 10 client.16050 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:22.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.121+0000 7f32e0a29740 1 -- 172.21.15.172:0/1497628920 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5652d77bd630 con 0x5652d7742680 2024-09-10T08:48:22.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.121+0000 7f32b77fe640 1 -- 172.21.15.172:0/1497628920 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32c80054b0 con 0x5652d7742680 2024-09-10T08:48:22.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.121+0000 7f32b77fe640 1 -- 172.21.15.172:0/1497628920 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f32d004b4e0 con 0x5652d7742680 2024-09-10T08:48:22.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.121+0000 7f32e0a29740 10 client.16050 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:22.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.121+0000 7f32e0a29740 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:22.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.121+0000 7f32e0a29740 20 client.16050 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:22.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.121+0000 7f32e0a29740 1 --2- 172.21.15.172:0/1497628920 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5652d77bdd60 0x5652d77c0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:22.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.121+0000 7f32e0a29740 4 client.16050 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:48:22.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.121+0000 7f32e0a29740 1 -- 172.21.15.172:0/1497628920 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5652d76a4d60 con 0x5652d77bdd60 2024-09-10T08:48:22.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.121+0000 7f32d7fff640 1 --2- 172.21.15.172:0/1497628920 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5652d77bdd60 0x5652d77c0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:22.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.122+0000 7f32b77fe640 10 client.16050 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:22.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.122+0000 7f32d7fff640 1 --2- 172.21.15.172:0/1497628920 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5652d77bdd60 0x5652d77c0180 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9520e0c5ee88afbf server_cookie=1b58a080c1c0c5fe in_seq=0 out_seq=0 2024-09-10T08:48:22.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.122+0000 7f32b77fe640 1 -- 172.21.15.172:0/1497628920 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5652d76a4d60 con 0x5652d77bdd60 2024-09-10T08:48:22.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.122+0000 7f32b77fe640 10 client.16050 handle_command_reply: tid=0 2024-09-10T08:48:22.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.122+0000 7f32e0a29740 1 client.16050 shutdown 2024-09-10T08:48:22.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.122+0000 7f32e0a29740 1 -- 172.21.15.172:0/1497628920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7742680 msgr2=0x5652d7748b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:22.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.122+0000 7f32e0a29740 1 --2- 172.21.15.172:0/1497628920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7742680 0x5652d7748b70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009ca0 tx=0x7f32d00095c0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.122+0000 7f32e0a29740 1 -- 172.21.15.172:0/1497628920 shutdown_connections 2024-09-10T08:48:22.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.122+0000 7f32e0a29740 1 --2- 172.21.15.172:0/1497628920 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5652d77bdd60 0x5652d77c0180 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.122+0000 7f32e0a29740 1 --2- 172.21.15.172:0/1497628920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5652d7742680 0x5652d7748b70 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.122+0000 7f32e0a29740 1 --2- 172.21.15.172:0/1497628920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652d7737dc0 0x5652d774acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.122+0000 7f32e0a29740 1 --2- 172.21.15.172:0/1497628920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652d77374a0 0x5652d774a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.122+0000 7f32b77fe640 0 client.16050 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:22.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.122+0000 7f32e0a29740 1 -- 172.21.15.172:0/1497628920 >> 172.21.15.172:0/1497628920 conn(0x5652d7733a30 msgr2=0x5652d774d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:22.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.122+0000 7f32e0a29740 1 -- 172.21.15.172:0/1497628920 shutdown_connections 2024-09-10T08:48:22.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.122+0000 7f32e0a29740 1 -- 172.21.15.172:0/1497628920 wait complete. 2024-09-10T08:48:22.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.122+0000 7f32e0a29740 20 client.16050 trim_cache size 0 max 16384 2024-09-10T08:48:22.132 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:48:22.132 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T08:48:22.132 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T08:48:22.132 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:48:22.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.123+0000 7f32de650640 1 -- 172.21.15.172:0/4015315457 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f32c40a2b80 msgr2=0x7f32c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:22.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.123+0000 7f32de650640 1 --2- 172.21.15.172:0/4015315457 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f32c40a2b80 0x7f32c40a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009a60 tx=0x7f32d002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.123+0000 7f32de650640 1 -- 172.21.15.172:0/4015315457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32d8102790 msgr2=0x7f32d8198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:22.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.123+0000 7f32de650640 1 --2- 172.21.15.172:0/4015315457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32d8102790 0x7f32d8198880 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c970 tx=0x7f32c000ce30 comp rx=0 tx=0).stop 2024-09-10T08:48:22.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.123+0000 7f32de650640 1 -- 172.21.15.172:0/4015315457 shutdown_connections 2024-09-10T08:48:22.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.123+0000 7f32de650640 1 --2- 172.21.15.172:0/4015315457 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f32c40a2b80 0x7f32c40a5040 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.123+0000 7f32de650640 1 --2- 172.21.15.172:0/4015315457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32d8106f10 0x7f32d819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.123+0000 7f32de650640 1 --2- 172.21.15.172:0/4015315457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32d8102790 0x7f32d8198880 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.123+0000 7f32de650640 1 --2- 172.21.15.172:0/4015315457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32d8101de0 0x7f32d8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:22.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.123+0000 7f32de650640 1 -- 172.21.15.172:0/4015315457 >> 172.21.15.172:0/4015315457 conn(0x7f32d80fd8c0 msgr2=0x7f32d80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:22.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.123+0000 7f32de650640 1 -- 172.21.15.172:0/4015315457 shutdown_connections 2024-09-10T08:48:22.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:22.123+0000 7f32de650640 1 -- 172.21.15.172:0/4015315457 wait complete. 2024-09-10T08:48:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:22 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T08:48:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:22 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:48:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:22 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:48:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:22 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:48:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:22 smithi191 ceph-mon[35314]: pgmap v414: 129 pgs: 129 active+clean; 675 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-10T08:48:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:22 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:48:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:22 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T08:48:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:22 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:48:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:22 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:48:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:22 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:48:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:22 smithi172 ceph-mon[30854]: pgmap v414: 129 pgs: 129 active+clean; 675 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-10T08:48:22.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:22 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:48:22.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:22 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T08:48:22.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:22 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:48:22.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:22 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:48:22.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:22 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:48:22.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:22 smithi174 ceph-mon[34068]: pgmap v414: 129 pgs: 129 active+clean; 675 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-10T08:48:22.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:22 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:48:22.774 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:48:22.774 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:48:23.020 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:23.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.478+0000 7f5f99933640 1 -- 172.21.15.172:0/268350573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f94102570 msgr2=0x7f5f941029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:23.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.478+0000 7f5f99933640 1 --2- 172.21.15.172:0/268350573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f94102570 0x7f5f941029b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009900 tx=0x7f5f8802a480 comp rx=0 tx=0).stop 2024-09-10T08:48:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.478+0000 7f5f99933640 1 -- 172.21.15.172:0/268350573 shutdown_connections 2024-09-10T08:48:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.478+0000 7f5f99933640 1 --2- 172.21.15.172:0/268350573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f94102ef0 0x7f5f9410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.478+0000 7f5f99933640 1 --2- 172.21.15.172:0/268350573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f94102570 0x7f5f941029b0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.478+0000 7f5f99933640 1 --2- 172.21.15.172:0/268350573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f94100e20 0x7f5f94101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.478+0000 7f5f99933640 1 -- 172.21.15.172:0/268350573 >> 172.21.15.172:0/268350573 conn(0x7f5f940fc7d0 msgr2=0x7f5f940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.478+0000 7f5f99933640 1 -- 172.21.15.172:0/268350573 shutdown_connections 2024-09-10T08:48:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.478+0000 7f5f99933640 1 -- 172.21.15.172:0/268350573 wait complete. 2024-09-10T08:48:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.478+0000 7f5f99933640 1 Processor -- start 2024-09-10T08:48:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.478+0000 7f5f99933640 1 -- start start 2024-09-10T08:48:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f99933640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f94100e20 0x7f5f940fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f99933640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f94102570 0x7f5f940fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f99933640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f94102ef0 0x7f5f940fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f99933640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94110e50 con 0x7f5f94100e20 2024-09-10T08:48:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f99933640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94110cd0 con 0x7f5f94102570 2024-09-10T08:48:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f99933640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94110fd0 con 0x7f5f94102ef0 2024-09-10T08:48:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f92ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f94100e20 0x7f5f940fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f92ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f94100e20 0x7f5f940fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55710/0 (socket says 172.21.15.172:55710) 2024-09-10T08:48:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f92ffd640 1 -- 172.21.15.172:0/2899140717 learned_addr learned my addr 172.21.15.172:0/2899140717 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f937fe640 1 --2- 172.21.15.172:0/2899140717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f94102ef0 0x7f5f940fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f927fc640 1 --2- 172.21.15.172:0/2899140717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f94102570 0x7f5f940fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f92ffd640 1 -- 172.21.15.172:0/2899140717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f94102ef0 msgr2=0x7f5f940fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f92ffd640 1 --2- 172.21.15.172:0/2899140717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f94102ef0 0x7f5f940fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:23.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f92ffd640 1 -- 172.21.15.172:0/2899140717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f94102570 msgr2=0x7f5f940fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:23.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f92ffd640 1 --2- 172.21.15.172:0/2899140717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f94102570 0x7f5f940fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:23.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f92ffd640 1 -- 172.21.15.172:0/2899140717 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f941a6920 con 0x7f5f94100e20 2024-09-10T08:48:23.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f937fe640 1 --2- 172.21.15.172:0/2899140717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f94102ef0 0x7f5f940fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:23.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f92ffd640 1 --2- 172.21.15.172:0/2899140717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f94100e20 0x7f5f940fcc80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400c890 tx=0x7f5f8400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:23.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.479+0000 7f5f98931640 1 -- 172.21.15.172:0/2899140717 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84004220 con 0x7f5f94100e20 2024-09-10T08:48:23.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.480+0000 7f5f98931640 1 -- 172.21.15.172:0/2899140717 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f840043c0 con 0x7f5f94100e20 2024-09-10T08:48:23.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.480+0000 7f5f99933640 1 -- 172.21.15.172:0/2899140717 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f941a6c10 con 0x7f5f94100e20 2024-09-10T08:48:23.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.480+0000 7f5f98931640 1 -- 172.21.15.172:0/2899140717 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84004a00 con 0x7f5f94100e20 2024-09-10T08:48:23.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.480+0000 7f5f99933640 1 -- 172.21.15.172:0/2899140717 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f941a7150 con 0x7f5f94100e20 2024-09-10T08:48:23.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.481+0000 7f5f99933640 1 -- 172.21.15.172:0/2899140717 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f94101200 con 0x7f5f94100e20 2024-09-10T08:48:23.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.482+0000 7f5f98931640 1 -- 172.21.15.172:0/2899140717 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5f84004ba0 con 0x7f5f94100e20 2024-09-10T08:48:23.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.482+0000 7f5f98931640 1 --2- 172.21.15.172:0/2899140717 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f7c0a2b80 0x7f5f7c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:23.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.483+0000 7f5f927fc640 1 --2- 172.21.15.172:0/2899140717 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f7c0a2b80 0x7f5f7c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:23.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.483+0000 7f5f98931640 1 -- 172.21.15.172:0/2899140717 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5f840cf520 con 0x7f5f94100e20 2024-09-10T08:48:23.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.483+0000 7f5f927fc640 1 --2- 172.21.15.172:0/2899140717 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f7c0a2b80 0x7f5f7c0a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5f940ffee0 tx=0x7f5f880095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:23.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.486+0000 7f5f98931640 1 -- 172.21.15.172:0/2899140717 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f84014030 con 0x7f5f94100e20 2024-09-10T08:48:23.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.689+0000 7f5f99933640 1 -- 172.21.15.172:0/2899140717 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f940fe1b0 con 0x7f5f94100e20 2024-09-10T08:48:23.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.689+0000 7f5f98931640 1 -- 172.21.15.172:0/2899140717 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7f5f940fe1b0 con 0x7f5f94100e20 2024-09-10T08:48:23.691 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:48:23.691 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":29,"btime":"2024-09-10T08:46:37:242916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:46:36.267985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:48:23.691 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 29 2024-09-10T08:48:23.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.692+0000 7f5f99933640 1 -- 172.21.15.172:0/2899140717 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f7c0a2b80 msgr2=0x7f5f7c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:23.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.692+0000 7f5f99933640 1 --2- 172.21.15.172:0/2899140717 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f7c0a2b80 0x7f5f7c0a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5f940ffee0 tx=0x7f5f880095f0 comp rx=0 tx=0).stop 2024-09-10T08:48:23.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.692+0000 7f5f99933640 1 -- 172.21.15.172:0/2899140717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f94100e20 msgr2=0x7f5f940fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:23.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.692+0000 7f5f99933640 1 --2- 172.21.15.172:0/2899140717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f94100e20 0x7f5f940fcc80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400c890 tx=0x7f5f8400cd50 comp rx=0 tx=0).stop 2024-09-10T08:48:23.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.692+0000 7f5f99933640 1 -- 172.21.15.172:0/2899140717 shutdown_connections 2024-09-10T08:48:23.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.692+0000 7f5f99933640 1 --2- 172.21.15.172:0/2899140717 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f7c0a2b80 0x7f5f7c0a5040 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:23.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.692+0000 7f5f99933640 1 --2- 172.21.15.172:0/2899140717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f94102ef0 0x7f5f940fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:23.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.692+0000 7f5f99933640 1 --2- 172.21.15.172:0/2899140717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f94102570 0x7f5f940fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:23.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.692+0000 7f5f99933640 1 --2- 172.21.15.172:0/2899140717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f94100e20 0x7f5f940fcc80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:23.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.692+0000 7f5f99933640 1 -- 172.21.15.172:0/2899140717 >> 172.21.15.172:0/2899140717 conn(0x7f5f940fc7d0 msgr2=0x7f5f9406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:23.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.692+0000 7f5f99933640 1 -- 172.21.15.172:0/2899140717 shutdown_connections 2024-09-10T08:48:23.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:23.692+0000 7f5f99933640 1 -- 172.21.15.172:0/2899140717 wait complete. 2024-09-10T08:48:24.473 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:24 smithi172 ceph-mon[30854]: pgmap v415: 129 pgs: 129 active+clean; 675 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-10T08:48:24.473 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:24 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2899140717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:48:24.507 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T08:48:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:24 smithi191 ceph-mon[35314]: pgmap v415: 129 pgs: 129 active+clean; 675 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-10T08:48:24.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:24 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2899140717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:48:24.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:24 smithi174 ceph-mon[34068]: pgmap v415: 129 pgs: 129 active+clean; 675 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-10T08:48:24.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:24 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2899140717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:48:24.759 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:25.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.273+0000 7fd6f01cc640 1 -- 172.21.15.172:0/2107743625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e8105f40 msgr2=0x7fd6e81063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:25.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.273+0000 7fd6f01cc640 1 --2- 172.21.15.172:0/2107743625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e8105f40 0x7fd6e81063d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009ca0 tx=0x7fd6e002a940 comp rx=0 tx=0).stop 2024-09-10T08:48:25.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.273+0000 7fd6f01cc640 1 -- 172.21.15.172:0/2107743625 shutdown_connections 2024-09-10T08:48:25.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.273+0000 7fd6f01cc640 1 --2- 172.21.15.172:0/2107743625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e8105f40 0x7fd6e81063d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.273+0000 7fd6f01cc640 1 --2- 172.21.15.172:0/2107743625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6e8101450 0x7fd6e8105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.273+0000 7fd6f01cc640 1 --2- 172.21.15.172:0/2107743625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6e8100b30 0x7fd6e8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.273+0000 7fd6f01cc640 1 -- 172.21.15.172:0/2107743625 >> 172.21.15.172:0/2107743625 conn(0x7fd6e80fc7a0 msgr2=0x7fd6e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:25.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.273+0000 7fd6f01cc640 1 -- 172.21.15.172:0/2107743625 shutdown_connections 2024-09-10T08:48:25.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.273+0000 7fd6f01cc640 1 -- 172.21.15.172:0/2107743625 wait complete. 2024-09-10T08:48:25.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.274+0000 7fd6f01cc640 1 Processor -- start 2024-09-10T08:48:25.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.274+0000 7fd6f01cc640 1 -- start start 2024-09-10T08:48:25.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.274+0000 7fd6f01cc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e8100b30 0x7fd6e819ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.274+0000 7fd6f01cc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6e8101450 0x7fd6e819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.274+0000 7fd6f01cc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6e8105f40 0x7fd6e81a3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.274+0000 7fd6f01cc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e8118430 con 0x7fd6e8100b30 2024-09-10T08:48:25.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.274+0000 7fd6f01cc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81182b0 con 0x7fd6e8105f40 2024-09-10T08:48:25.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.274+0000 7fd6f01cc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81185b0 con 0x7fd6e8101450 2024-09-10T08:48:25.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.274+0000 7fd6edf68640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e8100b30 0x7fd6e819ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.274+0000 7fd6edf68640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e8100b30 0x7fd6e819ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55734/0 (socket says 172.21.15.172:55734) 2024-09-10T08:48:25.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.274+0000 7fd6edf68640 1 -- 172.21.15.172:0/1156575562 learned_addr learned my addr 172.21.15.172:0/1156575562 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:25.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.274+0000 7fd6ed767640 1 --2- 172.21.15.172:0/1156575562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6e8101450 0x7fd6e819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.274+0000 7fd6ee769640 1 --2- 172.21.15.172:0/1156575562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6e8105f40 0x7fd6e81a3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.275+0000 7fd6edf68640 1 -- 172.21.15.172:0/1156575562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6e8101450 msgr2=0x7fd6e819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:25.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.275+0000 7fd6edf68640 1 --2- 172.21.15.172:0/1156575562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6e8101450 0x7fd6e819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.275+0000 7fd6edf68640 1 -- 172.21.15.172:0/1156575562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6e8105f40 msgr2=0x7fd6e81a3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:25.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.275+0000 7fd6edf68640 1 --2- 172.21.15.172:0/1156575562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6e8105f40 0x7fd6e81a3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.275+0000 7fd6edf68640 1 -- 172.21.15.172:0/1156575562 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e81a3a60 con 0x7fd6e8100b30 2024-09-10T08:48:25.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.275+0000 7fd6edf68640 1 --2- 172.21.15.172:0/1156575562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e8100b30 0x7fd6e819ea30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400d950 tx=0x7fd6e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:25.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.276+0000 7fd6deffd640 1 -- 172.21.15.172:0/1156575562 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4017030 con 0x7fd6e8100b30 2024-09-10T08:48:25.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.276+0000 7fd6deffd640 1 -- 172.21.15.172:0/1156575562 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6e4004020 con 0x7fd6e8100b30 2024-09-10T08:48:25.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.276+0000 7fd6f01cc640 1 -- 172.21.15.172:0/1156575562 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e81a3d50 con 0x7fd6e8100b30 2024-09-10T08:48:25.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.276+0000 7fd6deffd640 1 -- 172.21.15.172:0/1156575562 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e40047a0 con 0x7fd6e8100b30 2024-09-10T08:48:25.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.276+0000 7fd6f01cc640 1 -- 172.21.15.172:0/1156575562 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e81ab5e0 con 0x7fd6e8100b30 2024-09-10T08:48:25.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.278+0000 7fd6deffd640 1 -- 172.21.15.172:0/1156575562 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd6e40041d0 con 0x7fd6e8100b30 2024-09-10T08:48:25.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.279+0000 7fd6deffd640 1 --2- 172.21.15.172:0/1156575562 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd6d40a2c00 0x7fd6d40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.279+0000 7fd6f25a5740 1 Processor -- start 2024-09-10T08:48:25.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.279+0000 7fd6deffd640 1 -- 172.21.15.172:0/1156575562 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd6e40ced00 con 0x7fd6e8100b30 2024-09-10T08:48:25.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.279+0000 7fd6ed767640 1 --2- 172.21.15.172:0/1156575562 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd6d40a2c00 0x7fd6d40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.279+0000 7fd6f25a5740 1 -- start start 2024-09-10T08:48:25.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.279+0000 7fd6f25a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b72b40 0x55aec3b71090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.279+0000 7fd6f25a5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b715d0 0x55aec3b71a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.279+0000 7fd6f25a5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b7c370 0x55aec3b7e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6f25a5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55aec3ad72f0 con 0x55aec3b7c370 2024-09-10T08:48:25.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6f25a5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aec3a7ecf0 con 0x55aec3b72b40 2024-09-10T08:48:25.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6f25a5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55aec3ad32b0 con 0x55aec3b715d0 2024-09-10T08:48:25.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6ed767640 1 --2- 172.21.15.172:0/1156575562 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd6d40a2c00 0x7fd6d40a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80ffcb0 tx=0x7fd6d8005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:25.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6ee769640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b7c370 0x55aec3b7e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6ee769640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b7c370 0x55aec3b7e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55738/0 (socket says 172.21.15.172:55738) 2024-09-10T08:48:25.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6ee769640 1 -- 172.21.15.172:0/2107882132 learned_addr learned my addr 172.21.15.172:0/2107882132 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:25.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6edf68640 1 --2- 172.21.15.172:0/2107882132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b72b40 0x55aec3b71090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6ed767640 1 --2- 172.21.15.172:0/2107882132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b715d0 0x55aec3b71a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6ee769640 1 -- 172.21.15.172:0/2107882132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b715d0 msgr2=0x55aec3b71a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:25.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6ee769640 1 --2- 172.21.15.172:0/2107882132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b715d0 0x55aec3b71a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6ee769640 1 -- 172.21.15.172:0/2107882132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b72b40 msgr2=0x55aec3b71090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:25.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6ee769640 1 --2- 172.21.15.172:0/2107882132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b72b40 0x55aec3b71090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6ee769640 1 -- 172.21.15.172:0/2107882132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aec3a633a0 con 0x55aec3b7c370 2024-09-10T08:48:25.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6edf68640 1 --2- 172.21.15.172:0/2107882132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b72b40 0x55aec3b71090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:25.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6ee769640 1 --2- 172.21.15.172:0/2107882132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b7c370 0x55aec3b7e760 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0035000 tx=0x7fd6e0009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d5f2bd53113a8f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:25.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6d3fff640 1 -- 172.21.15.172:0/2107882132 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0034040 con 0x55aec3b7c370 2024-09-10T08:48:25.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6d3fff640 1 -- 172.21.15.172:0/2107882132 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6e002f450 con 0x55aec3b7c370 2024-09-10T08:48:25.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.280+0000 7fd6d3fff640 1 -- 172.21.15.172:0/2107882132 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00308a0 con 0x55aec3b7c370 2024-09-10T08:48:25.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.281+0000 7fd6f25a5740 1 -- 172.21.15.172:0/2107882132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b7c370 msgr2=0x55aec3b7e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:25.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.281+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/2107882132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b7c370 0x55aec3b7e760 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0035000 tx=0x7fd6e0009820 comp rx=0 tx=0).stop 2024-09-10T08:48:25.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.281+0000 7fd6f25a5740 1 -- 172.21.15.172:0/2107882132 shutdown_connections 2024-09-10T08:48:25.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.281+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/2107882132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b7c370 0x55aec3b7e760 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.281+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/2107882132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b715d0 0x55aec3b71a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.281+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/2107882132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b72b40 0x55aec3b71090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.281+0000 7fd6f25a5740 1 -- 172.21.15.172:0/2107882132 >> 172.21.15.172:0/2107882132 conn(0x55aec3b3a310 msgr2=0x55aec3b6dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:25.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.281+0000 7fd6f25a5740 1 -- 172.21.15.172:0/2107882132 shutdown_connections 2024-09-10T08:48:25.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.281+0000 7fd6f25a5740 1 -- 172.21.15.172:0/2107882132 wait complete. 2024-09-10T08:48:25.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6f25a5740 1 -- start start 2024-09-10T08:48:25.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6f25a5740 1 Processor -- start 2024-09-10T08:48:25.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6f25a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b715d0 0x55aec3b7bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6f25a5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b72b40 0x55aec3b78f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6f25a5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b7c370 0x55aec3b79480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6f25a5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55aec3b7f010 con 0x55aec3b7c370 2024-09-10T08:48:25.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6f25a5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aec3a92060 con 0x55aec3b715d0 2024-09-10T08:48:25.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6f25a5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55aec3b7f190 con 0x55aec3b72b40 2024-09-10T08:48:25.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6f25a5740 10 client.0 _ll_register_callbacks cb 0x55aec37e58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:25.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6ee769640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b7c370 0x55aec3b79480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6ee769640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b7c370 0x55aec3b79480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55748/0 (socket says 172.21.15.172:55748) 2024-09-10T08:48:25.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6ee769640 1 -- 172.21.15.172:0/2597974838 learned_addr learned my addr 172.21.15.172:0/2597974838 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:25.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6edf68640 1 --2- 172.21.15.172:0/2597974838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b715d0 0x55aec3b7bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6ed767640 1 --2- 172.21.15.172:0/2597974838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b72b40 0x55aec3b78f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6ee769640 1 -- 172.21.15.172:0/2597974838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b72b40 msgr2=0x55aec3b78f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:25.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6ee769640 1 --2- 172.21.15.172:0/2597974838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b72b40 0x55aec3b78f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6ee769640 1 -- 172.21.15.172:0/2597974838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b715d0 msgr2=0x55aec3b7bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:25.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6ee769640 1 --2- 172.21.15.172:0/2597974838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b715d0 0x55aec3b7bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6ee769640 1 -- 172.21.15.172:0/2597974838 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aec38483c0 con 0x55aec3b7c370 2024-09-10T08:48:25.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.282+0000 7fd6edf68640 1 --2- 172.21.15.172:0/2597974838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b715d0 0x55aec3b7bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:25.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.283+0000 7fd6ee769640 1 --2- 172.21.15.172:0/2597974838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b7c370 0x55aec3b79480 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009bf0 tx=0x7fd6e0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8ec2198e993187 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:25.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.283+0000 7fd6d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:25.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.283+0000 7fd6d17fa640 1 -- 172.21.15.172:0/2597974838 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0051060 con 0x55aec3b7c370 2024-09-10T08:48:25.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.283+0000 7fd6f25a5740 1 -- 172.21.15.172:0/2597974838 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6e002f450 con 0x55aec3b7c370 2024-09-10T08:48:25.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.283+0000 7fd6d17fa640 1 -- 172.21.15.172:0/2597974838 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd6e0030640 con 0x55aec3b7c370 2024-09-10T08:48:25.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.283+0000 7fd6d17fa640 1 -- 172.21.15.172:0/2597974838 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd6e0058050 con 0x55aec3b7c370 2024-09-10T08:48:25.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.283+0000 7fd6f25a5740 10 client.16068 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:25.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.283+0000 7fd6f25a5740 1 -- 172.21.15.172:0/2597974838 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aec3b8aa30 con 0x55aec3b7c370 2024-09-10T08:48:25.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.283+0000 7fd6d17fa640 1 -- 172.21.15.172:0/2597974838 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6e002f450 con 0x55aec3b7c370 2024-09-10T08:48:25.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.283+0000 7fd6d17fa640 1 -- 172.21.15.172:0/2597974838 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd6e002f070 con 0x55aec3b7c370 2024-09-10T08:48:25.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.283+0000 7fd6f25a5740 10 client.16068 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:25.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.283+0000 7fd6f25a5740 10 client.16068 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:48:25.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.283+0000 7fd6f25a5740 20 client.16068 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:25.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.283+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/2597974838 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55aec3b8bd80 0x55aec3b8e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.283+0000 7fd6f25a5740 4 client.16068 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:25.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.283+0000 7fd6f25a5740 1 -- 172.21.15.172:0/2597974838 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aec3ae4860 con 0x55aec3b8bd80 2024-09-10T08:48:25.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.284+0000 7fd6edf68640 1 --2- 172.21.15.172:0/2597974838 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55aec3b8bd80 0x55aec3b8e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.284+0000 7fd6edf68640 1 --2- 172.21.15.172:0/2597974838 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55aec3b8bd80 0x55aec3b8e170 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b78e2db4d07f9ecb server_cookie=c41b4767a56ff410 in_seq=0 out_seq=0 2024-09-10T08:48:25.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.284+0000 7fd6d17fa640 10 client.16068 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:48:25.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.285+0000 7fd6d17fa640 1 -- 172.21.15.172:0/2597974838 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aec3ae4860 con 0x55aec3b8bd80 2024-09-10T08:48:25.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.285+0000 7fd6d17fa640 10 client.16068 handle_command_reply: tid=0 2024-09-10T08:48:25.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.285+0000 7fd6f25a5740 1 client.16068 shutdown 2024-09-10T08:48:25.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.285+0000 7fd6f25a5740 1 -- 172.21.15.172:0/2597974838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b7c370 msgr2=0x55aec3b79480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:25.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.285+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/2597974838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b7c370 0x55aec3b79480 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009bf0 tx=0x7fd6e0009580 comp rx=0 tx=0).stop 2024-09-10T08:48:25.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.285+0000 7fd6f25a5740 1 -- 172.21.15.172:0/2597974838 shutdown_connections 2024-09-10T08:48:25.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.285+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/2597974838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b7c370 0x55aec3b79480 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.285+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/2597974838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b72b40 0x55aec3b78f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.285+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/2597974838 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55aec3b8bd80 0x55aec3b8e170 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.285+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/2597974838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b715d0 0x55aec3b7bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.285+0000 7fd6d17fa640 0 client.16068 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:48:25.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.285+0000 7fd6f25a5740 1 -- 172.21.15.172:0/2597974838 >> 172.21.15.172:0/2597974838 conn(0x55aec3b3a310 msgr2=0x55aec3b73960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:25.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.285+0000 7fd6f25a5740 1 -- 172.21.15.172:0/2597974838 shutdown_connections 2024-09-10T08:48:25.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.285+0000 7fd6f25a5740 1 -- 172.21.15.172:0/2597974838 wait complete. 2024-09-10T08:48:25.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.285+0000 7fd6f25a5740 20 client.16068 trim_cache size 0 max 16384 2024-09-10T08:48:25.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6f25a5740 1 Processor -- start 2024-09-10T08:48:25.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6f25a5740 1 -- start start 2024-09-10T08:48:25.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6f25a5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b715d0 0x55aec3b88310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6f25a5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b72b40 0x55aec3b78800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6f25a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b7c370 0x55aec3b77810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6f25a5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81b5110 con 0x55aec3b72b40 2024-09-10T08:48:25.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6f25a5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81b6c50 con 0x55aec3b7c370 2024-09-10T08:48:25.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6f25a5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81b57e0 con 0x55aec3b715d0 2024-09-10T08:48:25.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6ed767640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b72b40 0x55aec3b78800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6ed767640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b72b40 0x55aec3b78800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55762/0 (socket says 172.21.15.172:55762) 2024-09-10T08:48:25.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6ed767640 1 -- 172.21.15.172:0/4220935020 learned_addr learned my addr 172.21.15.172:0/4220935020 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:25.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6ee769640 1 --2- 172.21.15.172:0/4220935020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b7c370 0x55aec3b77810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6edf68640 1 --2- 172.21.15.172:0/4220935020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b715d0 0x55aec3b88310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6ed767640 1 -- 172.21.15.172:0/4220935020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b715d0 msgr2=0x55aec3b88310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:25.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6ed767640 1 --2- 172.21.15.172:0/4220935020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b715d0 0x55aec3b88310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6ed767640 1 -- 172.21.15.172:0/4220935020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b7c370 msgr2=0x55aec3b77810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:25.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6ed767640 1 --2- 172.21.15.172:0/4220935020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b7c370 0x55aec3b77810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.298+0000 7fd6ed767640 1 -- 172.21.15.172:0/4220935020 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aec3b6d4f0 con 0x55aec3b72b40 2024-09-10T08:48:25.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.299+0000 7fd6ed767640 1 --2- 172.21.15.172:0/4220935020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b72b40 0x55aec3b78800 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd6e81a0010 tx=0x7fd6d8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5364a5389bdf3e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:25.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.299+0000 7fd6d37fe640 1 -- 172.21.15.172:0/4220935020 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8016070 con 0x55aec3b72b40 2024-09-10T08:48:25.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.299+0000 7fd6d37fe640 1 -- 172.21.15.172:0/4220935020 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6d80054d0 con 0x55aec3b72b40 2024-09-10T08:48:25.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.299+0000 7fd6d37fe640 1 -- 172.21.15.172:0/4220935020 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8004db0 con 0x55aec3b72b40 2024-09-10T08:48:25.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.299+0000 7fd6f25a5740 1 -- 172.21.15.172:0/4220935020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b72b40 msgr2=0x55aec3b78800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:25.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.299+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/4220935020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b72b40 0x55aec3b78800 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd6e81a0010 tx=0x7fd6d8012000 comp rx=0 tx=0).stop 2024-09-10T08:48:25.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.299+0000 7fd6f25a5740 1 -- 172.21.15.172:0/4220935020 shutdown_connections 2024-09-10T08:48:25.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.299+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/4220935020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b7c370 0x55aec3b77810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.299+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/4220935020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b72b40 0x55aec3b78800 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.299+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/4220935020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b715d0 0x55aec3b88310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.299+0000 7fd6f25a5740 1 -- 172.21.15.172:0/4220935020 >> 172.21.15.172:0/4220935020 conn(0x55aec3b3a310 msgr2=0x55aec3b78f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:25.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.299+0000 7fd6f25a5740 1 -- 172.21.15.172:0/4220935020 shutdown_connections 2024-09-10T08:48:25.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.299+0000 7fd6f25a5740 1 -- 172.21.15.172:0/4220935020 wait complete. 2024-09-10T08:48:25.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.300+0000 7fd6f25a5740 1 -- start start 2024-09-10T08:48:25.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.300+0000 7fd6f25a5740 1 Processor -- start 2024-09-10T08:48:25.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.300+0000 7fd6f25a5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b715d0 0x55aec3b7fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.300+0000 7fd6f25a5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b72b40 0x55aec3b7ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.300+0000 7fd6f25a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b7c370 0x55aec3bf46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.300+0000 7fd6f25a5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81b6580 con 0x55aec3b72b40 2024-09-10T08:48:25.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.300+0000 7fd6f25a5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81b5eb0 con 0x55aec3b7c370 2024-09-10T08:48:25.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.300+0000 7fd6f25a5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81b30f0 con 0x55aec3b715d0 2024-09-10T08:48:25.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.300+0000 7fd6f25a5740 10 client.0 _ll_register_callbacks cb 0x7fd6e40974a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:25.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.300+0000 7fd6ed767640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b72b40 0x55aec3b7ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.300+0000 7fd6ed767640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b72b40 0x55aec3b7ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55776/0 (socket says 172.21.15.172:55776) 2024-09-10T08:48:25.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.301+0000 7fd6ed767640 1 -- 172.21.15.172:0/88561427 learned_addr learned my addr 172.21.15.172:0/88561427 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:25.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.301+0000 7fd6ee769640 1 --2- 172.21.15.172:0/88561427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b7c370 0x55aec3bf46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.301+0000 7fd6edf68640 1 --2- 172.21.15.172:0/88561427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b715d0 0x55aec3b7fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.301+0000 7fd6ed767640 1 -- 172.21.15.172:0/88561427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b715d0 msgr2=0x55aec3b7fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:25.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.301+0000 7fd6ed767640 1 --2- 172.21.15.172:0/88561427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b715d0 0x55aec3b7fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.301+0000 7fd6ed767640 1 -- 172.21.15.172:0/88561427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b7c370 msgr2=0x55aec3bf46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:25.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.301+0000 7fd6ed767640 1 --2- 172.21.15.172:0/88561427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b7c370 0x55aec3bf46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.301+0000 7fd6ed767640 1 -- 172.21.15.172:0/88561427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aec3bf4c40 con 0x55aec3b72b40 2024-09-10T08:48:25.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.301+0000 7fd6ed767640 1 --2- 172.21.15.172:0/88561427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b72b40 0x55aec3b7ff70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8028040 tx=0x7fd6d8004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a00061a4723188c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:25.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.301+0000 7fd6d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:25.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.301+0000 7fd6d17fa640 1 -- 172.21.15.172:0/88561427 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8034020 con 0x55aec3b72b40 2024-09-10T08:48:25.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.301+0000 7fd6d17fa640 1 -- 172.21.15.172:0/88561427 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd6d8005260 con 0x55aec3b72b40 2024-09-10T08:48:25.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.301+0000 7fd6f25a5740 1 -- 172.21.15.172:0/88561427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d80054d0 con 0x55aec3b72b40 2024-09-10T08:48:25.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.301+0000 7fd6d17fa640 1 -- 172.21.15.172:0/88561427 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd6d8010430 con 0x55aec3b72b40 2024-09-10T08:48:25.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.301+0000 7fd6f25a5740 10 client.16074 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:25.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.301+0000 7fd6f25a5740 1 -- 172.21.15.172:0/88561427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aec3bf5f50 con 0x55aec3b72b40 2024-09-10T08:48:25.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.302+0000 7fd6d17fa640 1 -- 172.21.15.172:0/88561427 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6d80054d0 con 0x55aec3b72b40 2024-09-10T08:48:25.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.302+0000 7fd6d17fa640 1 -- 172.21.15.172:0/88561427 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd6d8025070 con 0x55aec3b72b40 2024-09-10T08:48:25.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.302+0000 7fd6f25a5740 10 client.16074 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:25.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.302+0000 7fd6f25a5740 10 client.16074 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:48:25.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.302+0000 7fd6f25a5740 20 client.16074 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:25.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.302+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/88561427 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55aec3bf6750 0x55aec3bf8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:25.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.302+0000 7fd6f25a5740 4 client.16074 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:48:25.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.302+0000 7fd6f25a5740 1 -- 172.21.15.172:0/88561427 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aec3aded70 con 0x55aec3bf6750 2024-09-10T08:48:25.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.302+0000 7fd6edf68640 1 --2- 172.21.15.172:0/88561427 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55aec3bf6750 0x55aec3bf8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:25.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.302+0000 7fd6d17fa640 10 client.16074 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:48:25.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.302+0000 7fd6edf68640 1 --2- 172.21.15.172:0/88561427 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55aec3bf6750 0x55aec3bf8b70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b08a30ea3efce590 server_cookie=a1d9ce573d8edd5b in_seq=0 out_seq=0 2024-09-10T08:48:25.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6d17fa640 1 -- 172.21.15.172:0/88561427 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aec3aded70 con 0x55aec3bf6750 2024-09-10T08:48:25.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6d17fa640 10 client.16074 handle_command_reply: tid=0 2024-09-10T08:48:25.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6f25a5740 1 client.16074 shutdown 2024-09-10T08:48:25.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6f25a5740 1 -- 172.21.15.172:0/88561427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b72b40 msgr2=0x55aec3b7ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:25.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/88561427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b72b40 0x55aec3b7ff70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8028040 tx=0x7fd6d8004c60 comp rx=0 tx=0).stop 2024-09-10T08:48:25.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6f25a5740 1 -- 172.21.15.172:0/88561427 shutdown_connections 2024-09-10T08:48:25.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/88561427 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55aec3bf6750 0x55aec3bf8b70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/88561427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aec3b7c370 0x55aec3bf46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/88561427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aec3b72b40 0x55aec3b7ff70 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6f25a5740 1 --2- 172.21.15.172:0/88561427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aec3b715d0 0x55aec3b7fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6d17fa640 0 client.16074 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:48:25.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6f25a5740 1 -- 172.21.15.172:0/88561427 >> 172.21.15.172:0/88561427 conn(0x55aec3b3a310 msgr2=0x55aec3b6cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:25.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6f25a5740 1 -- 172.21.15.172:0/88561427 shutdown_connections 2024-09-10T08:48:25.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6f25a5740 1 -- 172.21.15.172:0/88561427 wait complete. 2024-09-10T08:48:25.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6f25a5740 20 client.16074 trim_cache size 0 max 16384 2024-09-10T08:48:25.310 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:48:25.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6f01cc640 1 -- 172.21.15.172:0/1156575562 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd6d40a2c00 msgr2=0x7fd6d40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:25.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6f01cc640 1 --2- 172.21.15.172:0/1156575562 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd6d40a2c00 0x7fd6d40a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80ffcb0 tx=0x7fd6d8005f50 comp rx=0 tx=0).stop 2024-09-10T08:48:25.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6f01cc640 1 -- 172.21.15.172:0/1156575562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e8100b30 msgr2=0x7fd6e819ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:25.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.303+0000 7fd6f01cc640 1 --2- 172.21.15.172:0/1156575562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e8100b30 0x7fd6e819ea30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400d950 tx=0x7fd6e400de10 comp rx=0 tx=0).stop 2024-09-10T08:48:25.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.304+0000 7fd6f01cc640 1 -- 172.21.15.172:0/1156575562 shutdown_connections 2024-09-10T08:48:25.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.304+0000 7fd6f01cc640 1 --2- 172.21.15.172:0/1156575562 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd6d40a2c00 0x7fd6d40a50c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.304+0000 7fd6f01cc640 1 --2- 172.21.15.172:0/1156575562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6e8105f40 0x7fd6e81a3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.304+0000 7fd6f01cc640 1 --2- 172.21.15.172:0/1156575562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6e8101450 0x7fd6e819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.304+0000 7fd6f01cc640 1 --2- 172.21.15.172:0/1156575562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e8100b30 0x7fd6e819ea30 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:25.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.304+0000 7fd6f01cc640 1 -- 172.21.15.172:0/1156575562 >> 172.21.15.172:0/1156575562 conn(0x7fd6e80fc7a0 msgr2=0x7fd6e810f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:25.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.304+0000 7fd6f01cc640 1 -- 172.21.15.172:0/1156575562 shutdown_connections 2024-09-10T08:48:25.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:25.304+0000 7fd6f01cc640 1 -- 172.21.15.172:0/1156575562 wait complete. 2024-09-10T08:48:25.649 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:48:26.070 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T08:48:26.133 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:26.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:26 smithi172 ceph-mon[30854]: pgmap v416: 129 pgs: 129 active+clean; 675 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-10T08:48:26.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:26 smithi191 ceph-mon[35314]: pgmap v416: 129 pgs: 129 active+clean; 675 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-10T08:48:26.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.643+0000 7f6c76f4e640 1 -- 172.21.15.172:0/3155881877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c70101df0 msgr2=0x7f6c701021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:26.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.643+0000 7f6c76f4e640 1 --2- 172.21.15.172:0/3155881877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c70101df0 0x7f6c701021d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009900 tx=0x7f6c6402a410 comp rx=0 tx=0).stop 2024-09-10T08:48:26.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.643+0000 7f6c76f4e640 1 -- 172.21.15.172:0/3155881877 shutdown_connections 2024-09-10T08:48:26.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.643+0000 7f6c76f4e640 1 --2- 172.21.15.172:0/3155881877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c70106f20 0x7f6c7010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:26.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.643+0000 7f6c76f4e640 1 --2- 172.21.15.172:0/3155881877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c701027a0 0x7f6c701067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:26.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.643+0000 7f6c76f4e640 1 --2- 172.21.15.172:0/3155881877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c70101df0 0x7f6c701021d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:26.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.643+0000 7f6c76f4e640 1 -- 172.21.15.172:0/3155881877 >> 172.21.15.172:0/3155881877 conn(0x7f6c700fd8b0 msgr2=0x7f6c700ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:26.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.643+0000 7f6c76f4e640 1 -- 172.21.15.172:0/3155881877 shutdown_connections 2024-09-10T08:48:26.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.643+0000 7f6c76f4e640 1 -- 172.21.15.172:0/3155881877 wait complete. 2024-09-10T08:48:26.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c76f4e640 1 Processor -- start 2024-09-10T08:48:26.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c76f4e640 1 -- start start 2024-09-10T08:48:26.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c76f4e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c70101df0 0x7f6c70198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:26.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c76f4e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c701027a0 0x7f6c70198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:26.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c76f4e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c70106f20 0x7f6c7019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:26.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c76f4e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6c70111e80 con 0x7f6c70101df0 2024-09-10T08:48:26.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c76f4e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6c70111d00 con 0x7f6c701027a0 2024-09-10T08:48:26.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c76f4e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6c70112000 con 0x7f6c70106f20 2024-09-10T08:48:26.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c74cea640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c70101df0 0x7f6c70198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:26.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c74cea640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c70101df0 0x7f6c70198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34010/0 (socket says 172.21.15.172:34010) 2024-09-10T08:48:26.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c74cea640 1 -- 172.21.15.172:0/2557253525 learned_addr learned my addr 172.21.15.172:0/2557253525 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:26.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c6ffff640 1 --2- 172.21.15.172:0/2557253525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c701027a0 0x7f6c70198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:26.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c74cea640 1 -- 172.21.15.172:0/2557253525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c70106f20 msgr2=0x7f6c7019cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:26.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c74cea640 1 --2- 172.21.15.172:0/2557253525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c70106f20 0x7f6c7019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:26.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c74cea640 1 -- 172.21.15.172:0/2557253525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c701027a0 msgr2=0x7f6c70198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:26.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c74cea640 1 --2- 172.21.15.172:0/2557253525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c701027a0 0x7f6c70198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:26.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c74cea640 1 -- 172.21.15.172:0/2557253525 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7019d2b0 con 0x7f6c70101df0 2024-09-10T08:48:26.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.644+0000 7f6c6ffff640 1 --2- 172.21.15.172:0/2557253525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c701027a0 0x7f6c70198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:26.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.645+0000 7f6c74cea640 1 --2- 172.21.15.172:0/2557253525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c70101df0 0x7f6c70198300 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6c64005ab0 tx=0x7f6c64003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:26.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.645+0000 7f6c6dffb640 1 -- 172.21.15.172:0/2557253525 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6403f070 con 0x7f6c70101df0 2024-09-10T08:48:26.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.645+0000 7f6c6dffb640 1 -- 172.21.15.172:0/2557253525 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c64031d60 con 0x7f6c70101df0 2024-09-10T08:48:26.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.645+0000 7f6c76f4e640 1 -- 172.21.15.172:0/2557253525 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7019d540 con 0x7f6c70101df0 2024-09-10T08:48:26.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.645+0000 7f6c6dffb640 1 -- 172.21.15.172:0/2557253525 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64035030 con 0x7f6c70101df0 2024-09-10T08:48:26.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.645+0000 7f6c76f4e640 1 -- 172.21.15.172:0/2557253525 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c701a4e80 con 0x7f6c70101df0 2024-09-10T08:48:26.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.646+0000 7f6c76f4e640 1 -- 172.21.15.172:0/2557253525 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c701021d0 con 0x7f6c70101df0 2024-09-10T08:48:26.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.647+0000 7f6c6dffb640 1 -- 172.21.15.172:0/2557253525 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6c64042070 con 0x7f6c70101df0 2024-09-10T08:48:26.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.648+0000 7f6c6dffb640 1 --2- 172.21.15.172:0/2557253525 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6c600a2c00 0x7f6c600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:26.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.648+0000 7f6c6ffff640 1 --2- 172.21.15.172:0/2557253525 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6c600a2c00 0x7f6c600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:26.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.648+0000 7f6c6dffb640 1 -- 172.21.15.172:0/2557253525 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6c640efcb0 con 0x7f6c70101df0 2024-09-10T08:48:26.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.648+0000 7f6c6ffff640 1 --2- 172.21.15.172:0/2557253525 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6c600a2c00 0x7f6c600a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0058c0 tx=0x7f6c5c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:26.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.652+0000 7f6c6dffb640 1 -- 172.21.15.172:0/2557253525 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c640bc8b0 con 0x7f6c70101df0 2024-09-10T08:48:26.713 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:26.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:26 smithi174 ceph-mon[34068]: pgmap v416: 129 pgs: 129 active+clean; 675 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-10T08:48:26.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.868+0000 7f6c76f4e640 1 -- 172.21.15.172:0/2557253525 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c7010a100 con 0x7f6c70101df0 2024-09-10T08:48:26.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.869+0000 7f6c6dffb640 1 -- 172.21.15.172:0/2557253525 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7f6c640bc1b0 con 0x7f6c70101df0 2024-09-10T08:48:26.871 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:48:26.871 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":29,"btime":"2024-09-10T08:46:37:242916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:46:36.267985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:48:26.871 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 29 2024-09-10T08:48:26.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.872+0000 7f6c76f4e640 1 -- 172.21.15.172:0/2557253525 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6c600a2c00 msgr2=0x7f6c600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.872+0000 7f6c76f4e640 1 --2- 172.21.15.172:0/2557253525 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6c600a2c00 0x7f6c600a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0058c0 tx=0x7f6c5c005850 comp rx=0 tx=0).stop 2024-09-10T08:48:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.872+0000 7f6c76f4e640 1 -- 172.21.15.172:0/2557253525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c70101df0 msgr2=0x7f6c70198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.872+0000 7f6c76f4e640 1 --2- 172.21.15.172:0/2557253525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c70101df0 0x7f6c70198300 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6c64005ab0 tx=0x7f6c64003cd0 comp rx=0 tx=0).stop 2024-09-10T08:48:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.872+0000 7f6c76f4e640 1 -- 172.21.15.172:0/2557253525 shutdown_connections 2024-09-10T08:48:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.872+0000 7f6c76f4e640 1 --2- 172.21.15.172:0/2557253525 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6c600a2c00 0x7f6c600a50c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.872+0000 7f6c76f4e640 1 --2- 172.21.15.172:0/2557253525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c70106f20 0x7f6c7019cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.872+0000 7f6c76f4e640 1 --2- 172.21.15.172:0/2557253525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c701027a0 0x7f6c70198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.872+0000 7f6c76f4e640 1 --2- 172.21.15.172:0/2557253525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c70101df0 0x7f6c70198300 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.872+0000 7f6c76f4e640 1 -- 172.21.15.172:0/2557253525 >> 172.21.15.172:0/2557253525 conn(0x7f6c700fd8b0 msgr2=0x7f6c700fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.872+0000 7f6c76f4e640 1 -- 172.21.15.172:0/2557253525 shutdown_connections 2024-09-10T08:48:26.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:26.872+0000 7f6c76f4e640 1 -- 172.21.15.172:0/2557253525 wait complete. 2024-09-10T08:48:27.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:27 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2557253525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:48:27.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:27 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2557253525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:48:27.642 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 428b29b7 --release --await' 2024-09-10T08:48:27.642 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 428b29b7 --release --await 2024-09-10T08:48:27.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:27 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2557253525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:48:27.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.862+0000 7fe3e0d9f640 1 -- 172.21.15.172:0/3671748618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc1068d0 msgr2=0x7fe3dc106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:27.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.862+0000 7fe3e0d9f640 1 --2- 172.21.15.172:0/3671748618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc1068d0 0x7fe3dc106cb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009930 tx=0x7fe3cc02a420 comp rx=0 tx=0).stop 2024-09-10T08:48:27.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.862+0000 7fe3e0d9f640 1 -- 172.21.15.172:0/3671748618 shutdown_connections 2024-09-10T08:48:27.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.862+0000 7fe3e0d9f640 1 --2- 172.21.15.172:0/3671748618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3dc101270 0x7fe3dc10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.862+0000 7fe3e0d9f640 1 --2- 172.21.15.172:0/3671748618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3dc1008d0 0x7fe3dc100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.862+0000 7fe3e0d9f640 1 --2- 172.21.15.172:0/3671748618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc1068d0 0x7fe3dc106cb0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.862+0000 7fe3e0d9f640 1 -- 172.21.15.172:0/3671748618 >> 172.21.15.172:0/3671748618 conn(0x7fe3dc0fc610 msgr2=0x7fe3dc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:27.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.862+0000 7fe3e0d9f640 1 -- 172.21.15.172:0/3671748618 shutdown_connections 2024-09-10T08:48:27.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.862+0000 7fe3e0d9f640 1 -- 172.21.15.172:0/3671748618 wait complete. 2024-09-10T08:48:27.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.863+0000 7fe3e0d9f640 1 Processor -- start 2024-09-10T08:48:27.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.863+0000 7fe3e0d9f640 1 -- start start 2024-09-10T08:48:27.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.863+0000 7fe3e0d9f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3dc1008d0 0x7fe3dc10e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.863+0000 7fe3e0d9f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3dc101270 0x7fe3dc10e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.863+0000 7fe3e0d9f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc1068d0 0x7fe3dc10ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.863+0000 7fe3e0d9f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc111ca0 con 0x7fe3dc1068d0 2024-09-10T08:48:27.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.863+0000 7fe3e0d9f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc111b20 con 0x7fe3dc1008d0 2024-09-10T08:48:27.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.863+0000 7fe3e0d9f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc111e20 con 0x7fe3dc101270 2024-09-10T08:48:27.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.864+0000 7fe3dbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc1068d0 0x7fe3dc10ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.864+0000 7fe3dbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc1068d0 0x7fe3dc10ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34038/0 (socket says 172.21.15.172:34038) 2024-09-10T08:48:27.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.864+0000 7fe3dbfff640 1 -- 172.21.15.172:0/502126192 learned_addr learned my addr 172.21.15.172:0/502126192 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:27.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.864+0000 7fe3db7fe640 1 --2- 172.21.15.172:0/502126192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3dc1008d0 0x7fe3dc10e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.864+0000 7fe3daffd640 1 --2- 172.21.15.172:0/502126192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3dc101270 0x7fe3dc10e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.864+0000 7fe3dbfff640 1 -- 172.21.15.172:0/502126192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3dc101270 msgr2=0x7fe3dc10e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:27.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.864+0000 7fe3dbfff640 1 --2- 172.21.15.172:0/502126192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3dc101270 0x7fe3dc10e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.864+0000 7fe3dbfff640 1 -- 172.21.15.172:0/502126192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3dc1008d0 msgr2=0x7fe3dc10e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:27.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.864+0000 7fe3dbfff640 1 --2- 172.21.15.172:0/502126192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3dc1008d0 0x7fe3dc10e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.864+0000 7fe3dbfff640 1 -- 172.21.15.172:0/502126192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3dc1a6640 con 0x7fe3dc1068d0 2024-09-10T08:48:27.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.864+0000 7fe3daffd640 1 --2- 172.21.15.172:0/502126192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3dc101270 0x7fe3dc10e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:27.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.864+0000 7fe3dbfff640 1 --2- 172.21.15.172:0/502126192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc1068d0 0x7fe3dc10ecb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400eaa0 tx=0x7fe3d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:27.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.864+0000 7fe3d8ff9640 1 -- 172.21.15.172:0/502126192 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d400cdb0 con 0x7fe3dc1068d0 2024-09-10T08:48:27.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.864+0000 7fe3d8ff9640 1 -- 172.21.15.172:0/502126192 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3d4005020 con 0x7fe3dc1068d0 2024-09-10T08:48:27.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.864+0000 7fe3e0d9f640 1 -- 172.21.15.172:0/502126192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3dc1a6930 con 0x7fe3dc1068d0 2024-09-10T08:48:27.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.864+0000 7fe3d8ff9640 1 -- 172.21.15.172:0/502126192 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d40116d0 con 0x7fe3dc1068d0 2024-09-10T08:48:27.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.864+0000 7fe3e0d9f640 1 -- 172.21.15.172:0/502126192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3dc1a6e70 con 0x7fe3dc1068d0 2024-09-10T08:48:27.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.867+0000 7fe3d8ff9640 1 -- 172.21.15.172:0/502126192 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe3d40051d0 con 0x7fe3dc1068d0 2024-09-10T08:48:27.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.868+0000 7fe3e43da740 1 Processor -- start 2024-09-10T08:48:27.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.868+0000 7fe3e43da740 1 -- start start 2024-09-10T08:48:27.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.868+0000 7fe3e43da740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557380892d20 0x557380891270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.868+0000 7fe3e43da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573808917b0 0x557380891bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.868+0000 7fe3d8ff9640 1 --2- 172.21.15.172:0/502126192 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe3c40a2b80 0x7fe3c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.868+0000 7fe3e43da740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55738089c550 0x55738089e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.868+0000 7fe3e43da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5573807f72f0 con 0x5573808917b0 2024-09-10T08:48:27.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.868+0000 7fe3e43da740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55738079ecf0 con 0x557380892d20 2024-09-10T08:48:27.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.868+0000 7fe3e43da740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5573807f32b0 con 0x55738089c550 2024-09-10T08:48:27.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.868+0000 7fe3daffd640 1 --2- 172.21.15.172:0/502126192 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe3c40a2b80 0x7fe3c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.868+0000 7fe3db7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573808917b0 0x557380891bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.868+0000 7fe3d8ff9640 1 -- 172.21.15.172:0/502126192 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe3d40cf520 con 0x7fe3dc1068d0 2024-09-10T08:48:27.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.868+0000 7fe3db7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573808917b0 0x557380891bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34050/0 (socket says 172.21.15.172:34050) 2024-09-10T08:48:27.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.868+0000 7fe3db7fe640 1 -- 172.21.15.172:0/246167695 learned_addr learned my addr 172.21.15.172:0/246167695 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:27.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.868+0000 7fe3dbfff640 1 --2- 172.21.15.172:0/246167695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55738089c550 0x55738089e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.868+0000 7fe3daffd640 1 --2- 172.21.15.172:0/246167695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557380892d20 0x557380891270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.869+0000 7fe3db7fe640 1 -- 172.21.15.172:0/246167695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55738089c550 msgr2=0x55738089e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:27.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.869+0000 7fe3db7fe640 1 --2- 172.21.15.172:0/246167695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55738089c550 0x55738089e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.869+0000 7fe3db7fe640 1 -- 172.21.15.172:0/246167695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557380892d20 msgr2=0x557380891270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:27.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.869+0000 7fe3db7fe640 1 --2- 172.21.15.172:0/246167695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557380892d20 0x557380891270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.869+0000 7fe3db7fe640 1 -- 172.21.15.172:0/246167695 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573807833a0 con 0x5573808917b0 2024-09-10T08:48:27.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.869+0000 7fe3daffd640 1 --2- 172.21.15.172:0/502126192 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe3c40a2b80 0x7fe3c40a5040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0005fd0 tx=0x7fe3d000bb70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:27.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.869+0000 7fe3db7fe640 1 --2- 172.21.15.172:0/246167695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573808917b0 0x557380891bf0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009930 tx=0x7fe3cc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e1dcbb1f3fcfd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:27.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.869+0000 7fe3c1ffb640 1 -- 172.21.15.172:0/246167695 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc040070 con 0x5573808917b0 2024-09-10T08:48:27.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.869+0000 7fe3c1ffb640 1 -- 172.21.15.172:0/246167695 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3cc0316a0 con 0x5573808917b0 2024-09-10T08:48:27.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.869+0000 7fe3c1ffb640 1 -- 172.21.15.172:0/246167695 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc035040 con 0x5573808917b0 2024-09-10T08:48:27.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.870+0000 7fe3e43da740 1 -- 172.21.15.172:0/246167695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573808917b0 msgr2=0x557380891bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:27.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.870+0000 7fe3e43da740 1 --2- 172.21.15.172:0/246167695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573808917b0 0x557380891bf0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009930 tx=0x7fe3cc005fd0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.870+0000 7fe3e43da740 1 -- 172.21.15.172:0/246167695 shutdown_connections 2024-09-10T08:48:27.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.870+0000 7fe3e43da740 1 --2- 172.21.15.172:0/246167695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55738089c550 0x55738089e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.870+0000 7fe3e43da740 1 --2- 172.21.15.172:0/246167695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573808917b0 0x557380891bf0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.870+0000 7fe3e43da740 1 --2- 172.21.15.172:0/246167695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557380892d20 0x557380891270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.870+0000 7fe3e43da740 1 -- 172.21.15.172:0/246167695 >> 172.21.15.172:0/246167695 conn(0x55738085a310 msgr2=0x55738088de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:27.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.870+0000 7fe3e43da740 1 -- 172.21.15.172:0/246167695 shutdown_connections 2024-09-10T08:48:27.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.870+0000 7fe3e43da740 1 -- 172.21.15.172:0/246167695 wait complete. 2024-09-10T08:48:27.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.870+0000 7fe3e43da740 1 -- start start 2024-09-10T08:48:27.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.870+0000 7fe3e43da740 1 Processor -- start 2024-09-10T08:48:27.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3e43da740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573808917b0 0x55738089be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3e43da740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557380892d20 0x557380899130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3e43da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55738089c550 0x557380899670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3e43da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55738089f1f0 con 0x55738089c550 2024-09-10T08:48:27.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3e43da740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5573807b2060 con 0x557380892d20 2024-09-10T08:48:27.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3e43da740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55738089f370 con 0x5573808917b0 2024-09-10T08:48:27.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3e43da740 10 client.0 _ll_register_callbacks cb 0x5573805058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:27.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3dbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55738089c550 0x557380899670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3dbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55738089c550 0x557380899670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34058/0 (socket says 172.21.15.172:34058) 2024-09-10T08:48:27.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3dbfff640 1 -- 172.21.15.172:0/4114598353 learned_addr learned my addr 172.21.15.172:0/4114598353 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:27.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3db7fe640 1 --2- 172.21.15.172:0/4114598353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573808917b0 0x55738089be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3daffd640 1 --2- 172.21.15.172:0/4114598353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557380892d20 0x557380899130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3dbfff640 1 -- 172.21.15.172:0/4114598353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573808917b0 msgr2=0x55738089be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:27.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3dbfff640 1 --2- 172.21.15.172:0/4114598353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573808917b0 0x55738089be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3dbfff640 1 -- 172.21.15.172:0/4114598353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557380892d20 msgr2=0x557380899130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:27.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3dbfff640 1 --2- 172.21.15.172:0/4114598353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557380892d20 0x557380899130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3dbfff640 1 -- 172.21.15.172:0/4114598353 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573805683c0 con 0x55738089c550 2024-09-10T08:48:27.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3daffd640 1 --2- 172.21.15.172:0/4114598353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557380892d20 0x557380899130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:27.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3dbfff640 1 --2- 172.21.15.172:0/4114598353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55738089c550 0x557380899670 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe3d409b9b0 tx=0x7fe3d409b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c02b71c7c7eef4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:27.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3baffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:27.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.871+0000 7fe3baffd640 1 -- 172.21.15.172:0/4114598353 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d401f040 con 0x55738089c550 2024-09-10T08:48:27.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.872+0000 7fe3baffd640 1 -- 172.21.15.172:0/4114598353 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe3d4005560 con 0x55738089c550 2024-09-10T08:48:27.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.872+0000 7fe3e43da740 1 -- 172.21.15.172:0/4114598353 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3cc0316a0 con 0x55738089c550 2024-09-10T08:48:27.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.872+0000 7fe3baffd640 1 -- 172.21.15.172:0/4114598353 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe3d40b4050 con 0x55738089c550 2024-09-10T08:48:27.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.872+0000 7fe3e43da740 10 client.16092 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:27.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.872+0000 7fe3e43da740 1 -- 172.21.15.172:0/4114598353 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573808aabe0 con 0x55738089c550 2024-09-10T08:48:27.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.872+0000 7fe3baffd640 1 -- 172.21.15.172:0/4114598353 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3cc0316a0 con 0x55738089c550 2024-09-10T08:48:27.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.872+0000 7fe3baffd640 1 -- 172.21.15.172:0/4114598353 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe3d400f040 con 0x55738089c550 2024-09-10T08:48:27.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.872+0000 7fe3e43da740 10 client.16092 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:27.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.872+0000 7fe3e43da740 10 client.16092 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:48:27.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.872+0000 7fe3e43da740 20 client.16092 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:27.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.872+0000 7fe3e43da740 1 --2- 172.21.15.172:0/4114598353 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5573808ac040 0x5573808ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.872+0000 7fe3e43da740 4 client.16092 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:27.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.872+0000 7fe3e43da740 1 -- 172.21.15.172:0/4114598353 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557380804860 con 0x5573808ac040 2024-09-10T08:48:27.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.872+0000 7fe3db7fe640 1 --2- 172.21.15.172:0/4114598353 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5573808ac040 0x5573808ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.873+0000 7fe3baffd640 10 client.16092 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:48:27.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.873+0000 7fe3db7fe640 1 --2- 172.21.15.172:0/4114598353 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5573808ac040 0x5573808ae4e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f715614fdcfde0b5 server_cookie=5e3419e9e453a2de in_seq=0 out_seq=0 2024-09-10T08:48:27.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.874+0000 7fe3baffd640 1 -- 172.21.15.172:0/4114598353 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557380804860 con 0x5573808ac040 2024-09-10T08:48:27.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.874+0000 7fe3baffd640 10 client.16092 handle_command_reply: tid=0 2024-09-10T08:48:27.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.874+0000 7fe3e43da740 1 client.16092 shutdown 2024-09-10T08:48:27.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.874+0000 7fe3e43da740 1 -- 172.21.15.172:0/4114598353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55738089c550 msgr2=0x557380899670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:27.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.874+0000 7fe3e43da740 1 --2- 172.21.15.172:0/4114598353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55738089c550 0x557380899670 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe3d409b9b0 tx=0x7fe3d409b470 comp rx=0 tx=0).stop 2024-09-10T08:48:27.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.874+0000 7fe3e43da740 1 -- 172.21.15.172:0/4114598353 shutdown_connections 2024-09-10T08:48:27.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.874+0000 7fe3e43da740 1 --2- 172.21.15.172:0/4114598353 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5573808ac040 0x5573808ae4e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.874+0000 7fe3e43da740 1 --2- 172.21.15.172:0/4114598353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55738089c550 0x557380899670 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.874+0000 7fe3e43da740 1 --2- 172.21.15.172:0/4114598353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557380892d20 0x557380899130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.874+0000 7fe3e43da740 1 --2- 172.21.15.172:0/4114598353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5573808917b0 0x55738089be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.874+0000 7fe3e43da740 1 -- 172.21.15.172:0/4114598353 >> 172.21.15.172:0/4114598353 conn(0x55738085a310 msgr2=0x557380893b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:27.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.874+0000 7fe3baffd640 0 client.16092 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:48:27.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.874+0000 7fe3e43da740 1 -- 172.21.15.172:0/4114598353 shutdown_connections 2024-09-10T08:48:27.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.874+0000 7fe3e43da740 1 -- 172.21.15.172:0/4114598353 wait complete. 2024-09-10T08:48:27.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.874+0000 7fe3e43da740 20 client.16092 trim_cache size 0 max 16384 2024-09-10T08:48:27.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.889+0000 7fe3e43da740 1 Processor -- start 2024-09-10T08:48:27.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.889+0000 7fe3e43da740 1 -- start start 2024-09-10T08:48:27.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.889+0000 7fe3e43da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573808917b0 0x55738089b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.889+0000 7fe3e43da740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557380892d20 0x55738089d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.889+0000 7fe3e43da740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738089c550 0x55738089de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.889+0000 7fe3e43da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1b0bc0 con 0x5573808917b0 2024-09-10T08:48:27.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.889+0000 7fe3e43da740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1b2700 con 0x55738089c550 2024-09-10T08:48:27.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.889+0000 7fe3e43da740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1b1290 con 0x557380892d20 2024-09-10T08:48:27.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.890+0000 7fe3db7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573808917b0 0x55738089b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.890+0000 7fe3db7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573808917b0 0x55738089b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34068/0 (socket says 172.21.15.172:34068) 2024-09-10T08:48:27.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.890+0000 7fe3db7fe640 1 -- 172.21.15.172:0/283538521 learned_addr learned my addr 172.21.15.172:0/283538521 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:27.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.890+0000 7fe3dbfff640 1 --2- 172.21.15.172:0/283538521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738089c550 0x55738089de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.890+0000 7fe3daffd640 1 --2- 172.21.15.172:0/283538521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557380892d20 0x55738089d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.890+0000 7fe3db7fe640 1 -- 172.21.15.172:0/283538521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557380892d20 msgr2=0x55738089d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:27.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.890+0000 7fe3db7fe640 1 --2- 172.21.15.172:0/283538521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557380892d20 0x55738089d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.890+0000 7fe3db7fe640 1 -- 172.21.15.172:0/283538521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738089c550 msgr2=0x55738089de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:27.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.890+0000 7fe3db7fe640 1 --2- 172.21.15.172:0/283538521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738089c550 0x55738089de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.890+0000 7fe3db7fe640 1 -- 172.21.15.172:0/283538521 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55738089baa0 con 0x5573808917b0 2024-09-10T08:48:27.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.890+0000 7fe3dbfff640 1 --2- 172.21.15.172:0/283538521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738089c550 0x55738089de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:27.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.890+0000 7fe3db7fe640 1 --2- 172.21.15.172:0/283538521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573808917b0 0x55738089b0a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55738088eca0 tx=0x7fe3cc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b1c2653bde86ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:27.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.890+0000 7fe3c17fa640 1 -- 172.21.15.172:0/283538521 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc040040 con 0x5573808917b0 2024-09-10T08:48:27.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.890+0000 7fe3c17fa640 1 -- 172.21.15.172:0/283538521 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3cc031a50 con 0x5573808917b0 2024-09-10T08:48:27.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.890+0000 7fe3c17fa640 1 -- 172.21.15.172:0/283538521 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc035040 con 0x5573808917b0 2024-09-10T08:48:27.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.891+0000 7fe3e43da740 1 -- 172.21.15.172:0/283538521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573808917b0 msgr2=0x55738089b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:27.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.891+0000 7fe3e43da740 1 --2- 172.21.15.172:0/283538521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573808917b0 0x55738089b0a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55738088eca0 tx=0x7fe3cc000bc0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.891+0000 7fe3e43da740 1 -- 172.21.15.172:0/283538521 shutdown_connections 2024-09-10T08:48:27.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.891+0000 7fe3e43da740 1 --2- 172.21.15.172:0/283538521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738089c550 0x55738089de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.891+0000 7fe3e43da740 1 --2- 172.21.15.172:0/283538521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557380892d20 0x55738089d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.891+0000 7fe3e43da740 1 --2- 172.21.15.172:0/283538521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573808917b0 0x55738089b0a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.891+0000 7fe3e43da740 1 -- 172.21.15.172:0/283538521 >> 172.21.15.172:0/283538521 conn(0x55738085a310 msgr2=0x55738089a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:27.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.891+0000 7fe3e43da740 1 -- 172.21.15.172:0/283538521 shutdown_connections 2024-09-10T08:48:27.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.891+0000 7fe3e43da740 1 -- 172.21.15.172:0/283538521 wait complete. 2024-09-10T08:48:27.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.892+0000 7fe3e43da740 1 -- start start 2024-09-10T08:48:27.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.892+0000 7fe3e43da740 1 Processor -- start 2024-09-10T08:48:27.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.892+0000 7fe3e43da740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573808917b0 0x557380896d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.892+0000 7fe3e43da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557380892d20 0x5573808a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.892+0000 7fe3e43da740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55738089c550 0x5573808a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.892+0000 7fe3e43da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1b2030 con 0x557380892d20 2024-09-10T08:48:27.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.892+0000 7fe3e43da740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1b1960 con 0x5573808917b0 2024-09-10T08:48:27.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.892+0000 7fe3e43da740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1aeba0 con 0x55738089c550 2024-09-10T08:48:27.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.892+0000 7fe3e43da740 10 client.0 _ll_register_callbacks cb 0x7fe3d409b830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:27.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.892+0000 7fe3daffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557380892d20 0x5573808a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.893+0000 7fe3daffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557380892d20 0x5573808a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34074/0 (socket says 172.21.15.172:34074) 2024-09-10T08:48:27.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.893+0000 7fe3daffd640 1 -- 172.21.15.172:0/4158459673 learned_addr learned my addr 172.21.15.172:0/4158459673 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:27.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.893+0000 7fe3dbfff640 1 --2- 172.21.15.172:0/4158459673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55738089c550 0x5573808a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.893+0000 7fe3db7fe640 1 --2- 172.21.15.172:0/4158459673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573808917b0 0x557380896d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.893+0000 7fe3dbfff640 1 -- 172.21.15.172:0/4158459673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573808917b0 msgr2=0x557380896d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:27.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.893+0000 7fe3dbfff640 1 --2- 172.21.15.172:0/4158459673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573808917b0 0x557380896d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.893+0000 7fe3dbfff640 1 -- 172.21.15.172:0/4158459673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557380892d20 msgr2=0x5573808a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:27.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.893+0000 7fe3dbfff640 1 --2- 172.21.15.172:0/4158459673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557380892d20 0x5573808a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.893+0000 7fe3dbfff640 1 -- 172.21.15.172:0/4158459673 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573808b38c0 con 0x55738089c550 2024-09-10T08:48:27.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.893+0000 7fe3db7fe640 1 --2- 172.21.15.172:0/4158459673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573808917b0 0x557380896d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:27.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.893+0000 7fe3daffd640 1 --2- 172.21.15.172:0/4158459673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557380892d20 0x5573808a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:27.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.893+0000 7fe3dbfff640 1 --2- 172.21.15.172:0/4158459673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55738089c550 0x5573808a4b40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4000f80 tx=0x7fe3d4097cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91af07282d43ea8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:27.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.893+0000 7fe3baffd640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:48:27.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.893+0000 7fe3baffd640 1 -- 172.21.15.172:0/4158459673 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d401f070 con 0x55738089c550 2024-09-10T08:48:27.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.893+0000 7fe3e43da740 1 -- 172.21.15.172:0/4158459673 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3cc031a50 con 0x55738089c550 2024-09-10T08:48:27.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.893+0000 7fe3baffd640 1 -- 172.21.15.172:0/4158459673 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe3d409ab60 con 0x55738089c550 2024-09-10T08:48:27.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.893+0000 7fe3baffd640 1 -- 172.21.15.172:0/4158459673 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe3d407dbe0 con 0x55738089c550 2024-09-10T08:48:27.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.894+0000 7fe3e43da740 10 client.25408 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:27.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.894+0000 7fe3e43da740 1 -- 172.21.15.172:0/4158459673 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557380916f40 con 0x55738089c550 2024-09-10T08:48:27.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.894+0000 7fe3baffd640 1 -- 172.21.15.172:0/4158459673 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3cc031a50 con 0x55738089c550 2024-09-10T08:48:27.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.894+0000 7fe3baffd640 1 -- 172.21.15.172:0/4158459673 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe3d40a0f60 con 0x55738089c550 2024-09-10T08:48:27.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.894+0000 7fe3e43da740 10 client.25408 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:27.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.894+0000 7fe3e43da740 10 client.25408 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:48:27.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.894+0000 7fe3e43da740 20 client.25408 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:27.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.894+0000 7fe3e43da740 1 --2- 172.21.15.172:0/4158459673 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x557380917610 0x557380919a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:27.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.894+0000 7fe3e43da740 4 client.25408 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:48:27.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.894+0000 7fe3e43da740 1 -- 172.21.15.172:0/4158459673 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5573807fed70 con 0x557380917610 2024-09-10T08:48:27.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.894+0000 7fe3db7fe640 1 --2- 172.21.15.172:0/4158459673 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x557380917610 0x557380919a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.895+0000 7fe3baffd640 10 client.25408 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:48:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.895+0000 7fe3db7fe640 1 --2- 172.21.15.172:0/4158459673 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x557380917610 0x557380919a00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9c7777145c7a5577 server_cookie=ad592298b87ac895 in_seq=0 out_seq=0 2024-09-10T08:48:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.895+0000 7fe3baffd640 1 -- 172.21.15.172:0/4158459673 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5573807fed70 con 0x557380917610 2024-09-10T08:48:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.895+0000 7fe3baffd640 10 client.25408 handle_command_reply: tid=0 2024-09-10T08:48:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.895+0000 7fe3e43da740 1 client.25408 shutdown 2024-09-10T08:48:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.895+0000 7fe3e43da740 1 -- 172.21.15.172:0/4158459673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55738089c550 msgr2=0x5573808a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.895+0000 7fe3e43da740 1 --2- 172.21.15.172:0/4158459673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55738089c550 0x5573808a4b40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4000f80 tx=0x7fe3d4097cc0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.895+0000 7fe3e43da740 1 -- 172.21.15.172:0/4158459673 shutdown_connections 2024-09-10T08:48:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.895+0000 7fe3e43da740 1 --2- 172.21.15.172:0/4158459673 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x557380917610 0x557380919a00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.895+0000 7fe3e43da740 1 --2- 172.21.15.172:0/4158459673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55738089c550 0x5573808a4b40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.895+0000 7fe3baffd640 0 client.25408 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:48:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.895+0000 7fe3e43da740 1 --2- 172.21.15.172:0/4158459673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557380892d20 0x5573808a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.895+0000 7fe3e43da740 1 --2- 172.21.15.172:0/4158459673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573808917b0 0x557380896d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.895+0000 7fe3e43da740 1 -- 172.21.15.172:0/4158459673 >> 172.21.15.172:0/4158459673 conn(0x55738085a310 msgr2=0x557380897600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:27.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.895+0000 7fe3e43da740 1 -- 172.21.15.172:0/4158459673 shutdown_connections 2024-09-10T08:48:27.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.895+0000 7fe3e43da740 1 -- 172.21.15.172:0/4158459673 wait complete. 2024-09-10T08:48:27.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.895+0000 7fe3e43da740 20 client.25408 trim_cache size 0 max 16384 2024-09-10T08:48:27.903 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:48:27.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.896+0000 7fe3e0d9f640 1 -- 172.21.15.172:0/502126192 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe3c40a2b80 msgr2=0x7fe3c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:27.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.896+0000 7fe3e0d9f640 1 --2- 172.21.15.172:0/502126192 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe3c40a2b80 0x7fe3c40a5040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0005fd0 tx=0x7fe3d000bb70 comp rx=0 tx=0).stop 2024-09-10T08:48:27.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.896+0000 7fe3e0d9f640 1 -- 172.21.15.172:0/502126192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc1068d0 msgr2=0x7fe3dc10ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:27.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.896+0000 7fe3e0d9f640 1 --2- 172.21.15.172:0/502126192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc1068d0 0x7fe3dc10ecb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400eaa0 tx=0x7fe3d400ef60 comp rx=0 tx=0).stop 2024-09-10T08:48:27.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.896+0000 7fe3e0d9f640 1 -- 172.21.15.172:0/502126192 shutdown_connections 2024-09-10T08:48:27.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.896+0000 7fe3e0d9f640 1 --2- 172.21.15.172:0/502126192 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe3c40a2b80 0x7fe3c40a5040 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.896+0000 7fe3e0d9f640 1 --2- 172.21.15.172:0/502126192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc1068d0 0x7fe3dc10ecb0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.896+0000 7fe3e0d9f640 1 --2- 172.21.15.172:0/502126192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3dc101270 0x7fe3dc10e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.896+0000 7fe3e0d9f640 1 --2- 172.21.15.172:0/502126192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3dc1008d0 0x7fe3dc10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:27.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.896+0000 7fe3e0d9f640 1 -- 172.21.15.172:0/502126192 >> 172.21.15.172:0/502126192 conn(0x7fe3dc0fc610 msgr2=0x7fe3dc0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:27.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.897+0000 7fe3e0d9f640 1 -- 172.21.15.172:0/502126192 shutdown_connections 2024-09-10T08:48:27.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:27.897+0000 7fe3e0d9f640 1 -- 172.21.15.172:0/502126192 wait complete. 2024-09-10T08:48:28.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:28 smithi191 ceph-mon[35314]: pgmap v417: 129 pgs: 129 active+clean; 675 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-10T08:48:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:28 smithi172 ceph-mon[30854]: pgmap v417: 129 pgs: 129 active+clean; 675 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-10T08:48:28.643 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:28 smithi174 ceph-mon[34068]: pgmap v417: 129 pgs: 129 active+clean; 675 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-10T08:48:28.769 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:48:28.811 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:29.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.272+0000 7f2f3c3f6640 1 -- 172.21.15.172:0/738353411 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f34100a80 msgr2=0x7f2f34100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:29.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.272+0000 7f2f3c3f6640 1 --2- 172.21.15.172:0/738353411 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f34100a80 0x7f2f34100ee0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2f28009930 tx=0x7f2f2802a420 comp rx=0 tx=0).stop 2024-09-10T08:48:29.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.273+0000 7f2f3c3f6640 1 -- 172.21.15.172:0/738353411 shutdown_connections 2024-09-10T08:48:29.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.273+0000 7f2f3c3f6640 1 --2- 172.21.15.172:0/738353411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f34101420 0x7f2f3410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.273+0000 7f2f3c3f6640 1 --2- 172.21.15.172:0/738353411 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f34100a80 0x7f2f34100ee0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.273+0000 7f2f3c3f6640 1 --2- 172.21.15.172:0/738353411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f34106a80 0x7f2f34106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.273+0000 7f2f3c3f6640 1 -- 172.21.15.172:0/738353411 >> 172.21.15.172:0/738353411 conn(0x7f2f340fc7a0 msgr2=0x7f2f340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:29.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.273+0000 7f2f3c3f6640 1 -- 172.21.15.172:0/738353411 shutdown_connections 2024-09-10T08:48:29.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.273+0000 7f2f3c3f6640 1 -- 172.21.15.172:0/738353411 wait complete. 2024-09-10T08:48:29.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.273+0000 7f2f3c3f6640 1 Processor -- start 2024-09-10T08:48:29.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.273+0000 7f2f3c3f6640 1 -- start start 2024-09-10T08:48:29.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.273+0000 7f2f3c3f6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f34100a80 0x7f2f34199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.273+0000 7f2f3c3f6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f34101420 0x7f2f34199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.273+0000 7f2f3c3f6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f34106a80 0x7f2f341a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.273+0000 7f2f3c3f6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2f34110cf0 con 0x7f2f34106a80 2024-09-10T08:48:29.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.273+0000 7f2f3c3f6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f34110b70 con 0x7f2f34101420 2024-09-10T08:48:29.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.273+0000 7f2f3c3f6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2f34110e70 con 0x7f2f34100a80 2024-09-10T08:48:29.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f3a192640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f34100a80 0x7f2f34199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f3a993640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f34106a80 0x7f2f341a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f3a192640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f34100a80 0x7f2f34199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:42470/0 (socket says 172.21.15.172:42470) 2024-09-10T08:48:29.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f3a993640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f34106a80 0x7f2f341a42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34096/0 (socket says 172.21.15.172:34096) 2024-09-10T08:48:29.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f3a192640 1 -- 172.21.15.172:0/3175752212 learned_addr learned my addr 172.21.15.172:0/3175752212 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:29.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f39991640 1 --2- 172.21.15.172:0/3175752212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f34101420 0x7f2f34199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f3a993640 1 -- 172.21.15.172:0/3175752212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f34100a80 msgr2=0x7f2f34199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:29.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f3a993640 1 --2- 172.21.15.172:0/3175752212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f34100a80 0x7f2f34199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f3a993640 1 -- 172.21.15.172:0/3175752212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f34101420 msgr2=0x7f2f34199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:29.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f3a993640 1 --2- 172.21.15.172:0/3175752212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f34101420 0x7f2f34199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f3a993640 1 -- 172.21.15.172:0/3175752212 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f341a47e0 con 0x7f2f34106a80 2024-09-10T08:48:29.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f39991640 1 --2- 172.21.15.172:0/3175752212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f34101420 0x7f2f34199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:29.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f3a192640 1 --2- 172.21.15.172:0/3175752212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f34100a80 0x7f2f34199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:29.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f3a993640 1 --2- 172.21.15.172:0/3175752212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f34106a80 0x7f2f341a42a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c009fd0 tx=0x7f2f2c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:29.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f277fe640 1 -- 172.21.15.172:0/3175752212 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c00ce20 con 0x7f2f34106a80 2024-09-10T08:48:29.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f277fe640 1 -- 172.21.15.172:0/3175752212 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f2c005020 con 0x7f2f34106a80 2024-09-10T08:48:29.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f277fe640 1 -- 172.21.15.172:0/3175752212 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c010690 con 0x7f2f34106a80 2024-09-10T08:48:29.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f3c3f6640 1 -- 172.21.15.172:0/3175752212 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f341a49b0 con 0x7f2f34106a80 2024-09-10T08:48:29.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.274+0000 7f2f3c3f6640 1 -- 172.21.15.172:0/3175752212 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f341a4e70 con 0x7f2f34106a80 2024-09-10T08:48:29.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.277+0000 7f2f277fe640 1 -- 172.21.15.172:0/3175752212 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2f2c0051d0 con 0x7f2f34106a80 2024-09-10T08:48:29.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.277+0000 7f2f3e7cf740 1 Processor -- start 2024-09-10T08:48:29.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f3e7cf740 1 -- start start 2024-09-10T08:48:29.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f277fe640 1 --2- 172.21.15.172:0/3175752212 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2f200a2c00 0x7f2f200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f3e7cf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0def20 0x556b6d0df300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f3e7cf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0df840 0x556b6d0ee110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f3e7cf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0ee650 0x556b6d0f0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f3e7cf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556b6d044620 con 0x556b6d0def20 2024-09-10T08:48:29.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f3e7cf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556b6cfebe50 con 0x556b6d0ee650 2024-09-10T08:48:29.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f3e7cf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556b6d0405e0 con 0x556b6d0df840 2024-09-10T08:48:29.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f39991640 1 --2- 172.21.15.172:0/3175752212 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2f200a2c00 0x7f2f200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f277fe640 1 -- 172.21.15.172:0/3175752212 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2f2c0cfaa0 con 0x7f2f34106a80 2024-09-10T08:48:29.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f3a192640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0def20 0x556b6d0df300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f3a192640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0def20 0x556b6d0df300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34104/0 (socket says 172.21.15.172:34104) 2024-09-10T08:48:29.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f3a192640 1 -- 172.21.15.172:0/2732331399 learned_addr learned my addr 172.21.15.172:0/2732331399 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:29.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f3a993640 1 --2- 172.21.15.172:0/2732331399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0ee650 0x556b6d0f0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f39991640 1 --2- 172.21.15.172:0/2732331399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0df840 0x556b6d0ee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f3a993640 1 -- 172.21.15.172:0/2732331399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0df840 msgr2=0x556b6d0ee110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:29.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f3a993640 1 --2- 172.21.15.172:0/2732331399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0df840 0x556b6d0ee110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f3a993640 1 -- 172.21.15.172:0/2732331399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0def20 msgr2=0x556b6d0df300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:29.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f3a993640 1 --2- 172.21.15.172:0/2732331399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0def20 0x556b6d0df300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.278+0000 7f2f3a993640 1 -- 172.21.15.172:0/2732331399 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6cfdcd40 con 0x556b6d0ee650 2024-09-10T08:48:29.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.279+0000 7f2f3a993640 1 --2- 172.21.15.172:0/2732331399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0ee650 0x556b6d0f0a40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0cd5b0 tx=0x7f2f2c09c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5867bc8b2379441b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:29.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.279+0000 7f2f39991640 1 --2- 172.21.15.172:0/3175752212 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2f200a2c00 0x7f2f200a50c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2f28000c00 tx=0x7f2f28032b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:29.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.279+0000 7f2f24dea640 1 -- 172.21.15.172:0/2732331399 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c0a0350 con 0x556b6d0ee650 2024-09-10T08:48:29.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.279+0000 7f2f24dea640 1 -- 172.21.15.172:0/2732331399 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f2c0a4c10 con 0x556b6d0ee650 2024-09-10T08:48:29.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.279+0000 7f2f24dea640 1 -- 172.21.15.172:0/2732331399 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c098610 con 0x556b6d0ee650 2024-09-10T08:48:29.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.280+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/2732331399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0ee650 msgr2=0x556b6d0f0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:29.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.280+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/2732331399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0ee650 0x556b6d0f0a40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0cd5b0 tx=0x7f2f2c09c6a0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.280+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/2732331399 shutdown_connections 2024-09-10T08:48:29.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.280+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/2732331399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0ee650 0x556b6d0f0a40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.280+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/2732331399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0df840 0x556b6d0ee110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.280+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/2732331399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0def20 0x556b6d0df300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.280+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/2732331399 >> 172.21.15.172:0/2732331399 conn(0x556b6d0db5f0 msgr2=0x556b6d0db9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:29.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.280+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/2732331399 shutdown_connections 2024-09-10T08:48:29.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.280+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/2732331399 wait complete. 2024-09-10T08:48:29.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.280+0000 7f2f3e7cf740 1 -- start start 2024-09-10T08:48:29.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.280+0000 7f2f3e7cf740 1 Processor -- start 2024-09-10T08:48:29.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f3e7cf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0def20 0x556b6d0eb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f3e7cf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0df840 0x556b6d0ebac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f3e7cf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0ee650 0x556b6d0ec000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f3e7cf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556b6d0f1220 con 0x556b6d0ee650 2024-09-10T08:48:29.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f3e7cf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556b6cfccf10 con 0x556b6d0def20 2024-09-10T08:48:29.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f3e7cf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556b6d0f13a0 con 0x556b6d0df840 2024-09-10T08:48:29.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f3e7cf740 10 client.0 _ll_register_callbacks cb 0x556b6cff0db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:29.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f3a993640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0ee650 0x556b6d0ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f3a993640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0ee650 0x556b6d0ec000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34108/0 (socket says 172.21.15.172:34108) 2024-09-10T08:48:29.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f3a993640 1 -- 172.21.15.172:0/2025407883 learned_addr learned my addr 172.21.15.172:0/2025407883 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:29.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f39991640 1 --2- 172.21.15.172:0/2025407883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0df840 0x556b6d0ebac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f3a192640 1 --2- 172.21.15.172:0/2025407883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0def20 0x556b6d0eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f3a993640 1 -- 172.21.15.172:0/2025407883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0df840 msgr2=0x556b6d0ebac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:29.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f3a993640 1 --2- 172.21.15.172:0/2025407883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0df840 0x556b6d0ebac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f3a993640 1 -- 172.21.15.172:0/2025407883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0def20 msgr2=0x556b6d0eb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:29.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f3a993640 1 --2- 172.21.15.172:0/2025407883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0def20 0x556b6d0eb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f3a993640 1 -- 172.21.15.172:0/2025407883 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b6cd53580 con 0x556b6d0ee650 2024-09-10T08:48:29.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f3a993640 1 --2- 172.21.15.172:0/2025407883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0ee650 0x556b6d0ec000 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c098400 tx=0x7f2f2c0055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c25710e2baf3358 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:29.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:29.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.281+0000 7f2f1dffb640 1 -- 172.21.15.172:0/2025407883 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c085170 con 0x556b6d0ee650 2024-09-10T08:48:29.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.282+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/2025407883 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f2c0a4c10 con 0x556b6d0ee650 2024-09-10T08:48:29.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.282+0000 7f2f1dffb640 1 -- 172.21.15.172:0/2025407883 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2f2c0babe0 con 0x556b6d0ee650 2024-09-10T08:48:29.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.282+0000 7f2f1dffb640 1 -- 172.21.15.172:0/2025407883 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2f2c085310 con 0x556b6d0ee650 2024-09-10T08:48:29.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.282+0000 7f2f3e7cf740 10 client.16110 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:29.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.282+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/2025407883 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b6ce1d960 con 0x556b6d0ee650 2024-09-10T08:48:29.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.282+0000 7f2f1dffb640 1 -- 172.21.15.172:0/2025407883 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f2c0a4c10 con 0x556b6d0ee650 2024-09-10T08:48:29.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.282+0000 7f2f1dffb640 1 -- 172.21.15.172:0/2025407883 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2f2c0a4780 con 0x556b6d0ee650 2024-09-10T08:48:29.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.282+0000 7f2f3e7cf740 10 client.16110 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:29.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.282+0000 7f2f3e7cf740 10 client.16110 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:48:29.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.282+0000 7f2f3e7cf740 20 client.16110 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:29.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.282+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/2025407883 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556b6d0fe0e0 0x556b6d1004d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.282+0000 7f2f3e7cf740 4 client.16110 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:29.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.282+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/2025407883 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b6d051b90 con 0x556b6d0fe0e0 2024-09-10T08:48:29.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.283+0000 7f2f3a192640 1 --2- 172.21.15.172:0/2025407883 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556b6d0fe0e0 0x556b6d1004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.283+0000 7f2f3a192640 1 --2- 172.21.15.172:0/2025407883 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556b6d0fe0e0 0x556b6d1004d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9d6d007a4900cb9 server_cookie=9b3345c4998864b0 in_seq=0 out_seq=0 2024-09-10T08:48:29.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.283+0000 7f2f1dffb640 10 client.16110 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:29.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.284+0000 7f2f1dffb640 1 -- 172.21.15.172:0/2025407883 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b6d051b90 con 0x556b6d0fe0e0 2024-09-10T08:48:29.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.284+0000 7f2f1dffb640 10 client.16110 handle_command_reply: tid=0 2024-09-10T08:48:29.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.284+0000 7f2f3e7cf740 1 client.16110 shutdown 2024-09-10T08:48:29.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.284+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/2025407883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0ee650 msgr2=0x556b6d0ec000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:29.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.284+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/2025407883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0ee650 0x556b6d0ec000 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c098400 tx=0x7f2f2c0055b0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.284+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/2025407883 shutdown_connections 2024-09-10T08:48:29.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.284+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/2025407883 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556b6d0fe0e0 0x556b6d1004d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.284+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/2025407883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0ee650 0x556b6d0ec000 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.284+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/2025407883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0df840 0x556b6d0ebac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.284+0000 7f2f1dffb640 0 client.16110 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:29.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.284+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/2025407883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0def20 0x556b6d0eb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.284+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/2025407883 >> 172.21.15.172:0/2025407883 conn(0x556b6d0db5f0 msgr2=0x556b6d0e1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:29.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.284+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/2025407883 shutdown_connections 2024-09-10T08:48:29.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.284+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/2025407883 wait complete. 2024-09-10T08:48:29.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.284+0000 7f2f3e7cf740 20 client.16110 trim_cache size 0 max 16384 2024-09-10T08:48:29.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.302+0000 7f2f3e7cf740 1 Processor -- start 2024-09-10T08:48:29.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.302+0000 7f2f3e7cf740 1 -- start start 2024-09-10T08:48:29.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.303+0000 7f2f3e7cf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0db5f0 0x556b6d1034f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.303+0000 7f2f3e7cf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0def20 0x556b6d103a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.303+0000 7f2f3e7cf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0df840 0x556b6d0e87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.303+0000 7f2f3e7cf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2f341ac920 con 0x556b6d0df840 2024-09-10T08:48:29.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.303+0000 7f2f3e7cf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f341ae460 con 0x556b6d0db5f0 2024-09-10T08:48:29.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.303+0000 7f2f3e7cf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2f341acff0 con 0x556b6d0def20 2024-09-10T08:48:29.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.303+0000 7f2f3a993640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0df840 0x556b6d0e87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.303+0000 7f2f3a993640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0df840 0x556b6d0e87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34110/0 (socket says 172.21.15.172:34110) 2024-09-10T08:48:29.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.303+0000 7f2f3a993640 1 -- 172.21.15.172:0/1072377138 learned_addr learned my addr 172.21.15.172:0/1072377138 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:29.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.303+0000 7f2f39991640 1 --2- 172.21.15.172:0/1072377138 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0def20 0x556b6d103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.303+0000 7f2f3a192640 1 --2- 172.21.15.172:0/1072377138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0db5f0 0x556b6d1034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.303+0000 7f2f3a993640 1 -- 172.21.15.172:0/1072377138 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0def20 msgr2=0x556b6d103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:29.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.303+0000 7f2f3a993640 1 --2- 172.21.15.172:0/1072377138 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0def20 0x556b6d103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.303+0000 7f2f3a993640 1 -- 172.21.15.172:0/1072377138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0db5f0 msgr2=0x556b6d1034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:29.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.303+0000 7f2f3a993640 1 --2- 172.21.15.172:0/1072377138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0db5f0 0x556b6d1034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.303+0000 7f2f3a993640 1 -- 172.21.15.172:0/1072377138 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6d0df670 con 0x556b6d0df840 2024-09-10T08:48:29.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.303+0000 7f2f3a192640 1 --2- 172.21.15.172:0/1072377138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0db5f0 0x556b6d1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:29.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.304+0000 7f2f3a993640 1 --2- 172.21.15.172:0/1072377138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0df840 0x556b6d0e87e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c09bf30 tx=0x7f2f2c0cd290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1eeb397f52b0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:29.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.304+0000 7f2f1ffff640 1 -- 172.21.15.172:0/1072377138 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c0a4e60 con 0x556b6d0df840 2024-09-10T08:48:29.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.304+0000 7f2f1ffff640 1 -- 172.21.15.172:0/1072377138 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f2c096b10 con 0x556b6d0df840 2024-09-10T08:48:29.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.304+0000 7f2f1ffff640 1 -- 172.21.15.172:0/1072377138 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c0babe0 con 0x556b6d0df840 2024-09-10T08:48:29.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.304+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/1072377138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0df840 msgr2=0x556b6d0e87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:29.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.304+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/1072377138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0df840 0x556b6d0e87e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c09bf30 tx=0x7f2f2c0cd290 comp rx=0 tx=0).stop 2024-09-10T08:48:29.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.305+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/1072377138 shutdown_connections 2024-09-10T08:48:29.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.305+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/1072377138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0df840 0x556b6d0e87e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.305+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/1072377138 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0def20 0x556b6d103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.305+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/1072377138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0db5f0 0x556b6d1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.305+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/1072377138 >> 172.21.15.172:0/1072377138 conn(0x556b6d0e6f70 msgr2=0x556b6d0e9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:29.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.305+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/1072377138 shutdown_connections 2024-09-10T08:48:29.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.305+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/1072377138 wait complete. 2024-09-10T08:48:29.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.305+0000 7f2f3e7cf740 1 -- start start 2024-09-10T08:48:29.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.305+0000 7f2f3e7cf740 1 Processor -- start 2024-09-10T08:48:29.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.305+0000 7f2f3e7cf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0db5f0 0x556b6d1011e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.305+0000 7f2f3e7cf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0def20 0x556b6d101720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.305+0000 7f2f3e7cf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0df840 0x556b6d101c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.305+0000 7f2f3e7cf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2f341add90 con 0x556b6d0db5f0 2024-09-10T08:48:29.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.305+0000 7f2f3e7cf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f341ad6c0 con 0x556b6d0def20 2024-09-10T08:48:29.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.306+0000 7f2f3e7cf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2f341aa900 con 0x556b6d0df840 2024-09-10T08:48:29.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.306+0000 7f2f3a192640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0db5f0 0x556b6d1011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.306+0000 7f2f3a993640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0df840 0x556b6d101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.306+0000 7f2f3a192640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0db5f0 0x556b6d1011e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34118/0 (socket says 172.21.15.172:34118) 2024-09-10T08:48:29.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.306+0000 7f2f3a192640 1 -- 172.21.15.172:0/3449420618 learned_addr learned my addr 172.21.15.172:0/3449420618 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:29.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.306+0000 7f2f39991640 1 --2- 172.21.15.172:0/3449420618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0def20 0x556b6d101720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.306+0000 7f2f3e7cf740 10 client.0 _ll_register_callbacks cb 0x7f2f2c09b9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:29.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.306+0000 7f2f3a192640 1 -- 172.21.15.172:0/3449420618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0df840 msgr2=0x556b6d101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:29.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.306+0000 7f2f3a192640 1 --2- 172.21.15.172:0/3449420618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0df840 0x556b6d101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.306+0000 7f2f3a192640 1 -- 172.21.15.172:0/3449420618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0def20 msgr2=0x556b6d101720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:29.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.306+0000 7f2f3a192640 1 --2- 172.21.15.172:0/3449420618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0def20 0x556b6d101720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.306+0000 7f2f3a192640 1 -- 172.21.15.172:0/3449420618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b6d167a20 con 0x556b6d0db5f0 2024-09-10T08:48:29.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.306+0000 7f2f3a192640 1 --2- 172.21.15.172:0/3449420618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0db5f0 0x556b6d1011e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009db0 tx=0x7f2f30012c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0368240b494617 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:29.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.306+0000 7f2f1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:29.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.306+0000 7f2f1dffb640 1 -- 172.21.15.172:0/3449420618 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f30004460 con 0x556b6d0db5f0 2024-09-10T08:48:29.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.306+0000 7f2f1dffb640 1 -- 172.21.15.172:0/3449420618 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2f30012110 con 0x556b6d0db5f0 2024-09-10T08:48:29.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.306+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/3449420618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f2c096b10 con 0x556b6d0db5f0 2024-09-10T08:48:29.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.307+0000 7f2f1dffb640 1 -- 172.21.15.172:0/3449420618 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2f30012310 con 0x556b6d0db5f0 2024-09-10T08:48:29.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.307+0000 7f2f3e7cf740 10 client.16116 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:29.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.307+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/3449420618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b6d0e20a0 con 0x556b6d0db5f0 2024-09-10T08:48:29.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.307+0000 7f2f1dffb640 1 -- 172.21.15.172:0/3449420618 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f2c096b10 con 0x556b6d0db5f0 2024-09-10T08:48:29.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.307+0000 7f2f1dffb640 1 -- 172.21.15.172:0/3449420618 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2f30010a70 con 0x556b6d0db5f0 2024-09-10T08:48:29.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.307+0000 7f2f3e7cf740 10 client.16116 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:29.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.307+0000 7f2f3e7cf740 10 client.16116 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:48:29.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.307+0000 7f2f3e7cf740 20 client.16116 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:29.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.307+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/3449420618 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556b6d0ee650 0x556b6d0e4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.307+0000 7f2f3e7cf740 4 client.16116 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "428b29b7", "release": true, "await": true}] 2024-09-10T08:48:29.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.307+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/3449420618 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "set_id": "428b29b7", "release": true, "await": true}) -- 0x556b6d04c0a0 con 0x556b6d0ee650 2024-09-10T08:48:29.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.307+0000 7f2f39991640 1 --2- 172.21.15.172:0/3449420618 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556b6d0ee650 0x556b6d0e4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.308+0000 7f2f39991640 1 --2- 172.21.15.172:0/3449420618 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556b6d0ee650 0x556b6d0e4790 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1833c4a37ccb2c9 server_cookie=c8f186272d7a1ae1 in_seq=0 out_seq=0 2024-09-10T08:48:29.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.308+0000 7f2f1dffb640 10 client.16116 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:29.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.311+0000 7f2f1dffb640 1 -- 172.21.15.172:0/3449420618 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x556b6d04c0a0 con 0x556b6d0ee650 2024-09-10T08:48:29.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.311+0000 7f2f1dffb640 10 client.16116 handle_command_reply: tid=0 2024-09-10T08:48:29.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.311+0000 7f2f3e7cf740 1 client.16116 shutdown 2024-09-10T08:48:29.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.311+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/3449420618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0db5f0 msgr2=0x556b6d1011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:29.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.311+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/3449420618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0db5f0 0x556b6d1011e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009db0 tx=0x7f2f30012c20 comp rx=0 tx=0).stop 2024-09-10T08:48:29.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.311+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/3449420618 shutdown_connections 2024-09-10T08:48:29.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.311+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/3449420618 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556b6d0ee650 0x556b6d0e4790 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.311+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/3449420618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b6d0df840 0x556b6d101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.311+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/3449420618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b6d0def20 0x556b6d101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.311+0000 7f2f3e7cf740 1 --2- 172.21.15.172:0/3449420618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b6d0db5f0 0x556b6d1011e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.311+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/3449420618 >> 172.21.15.172:0/3449420618 conn(0x556b6d0e6f70 msgr2=0x556b6d0eb520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:29.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.312+0000 7f2f1dffb640 0 client.16116 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:29.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.312+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/3449420618 shutdown_connections 2024-09-10T08:48:29.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.312+0000 7f2f3e7cf740 1 -- 172.21.15.172:0/3449420618 wait complete. 2024-09-10T08:48:29.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.312+0000 7f2f3e7cf740 20 client.16116 trim_cache size 0 max 16384 2024-09-10T08:48:29.317 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:48:29.317 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 29, 2024-09-10T08:48:29.317 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:48:29.318 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 8, 2024-09-10T08:48:29.318 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:48:29.318 INFO:teuthology.orchestra.run.smithi172.stdout: "428b29b7": { 2024-09-10T08:48:29.318 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 8, 2024-09-10T08:48:29.318 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:48:29.318 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:48:29.318 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:48:29.318 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:48:29.318 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:48:29.318 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:48:29.318 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 150.0, 2024-09-10T08:48:29.319 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:48:29.319 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:48:29.319 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:48:29.319 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:48:29.319 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:48:29.319 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:48:29.319 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:48:29.319 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:48:29.319 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:48:29.319 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:48:29.319 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:48:29.319 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:48:29.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.312+0000 7f2f3c3f6640 1 -- 172.21.15.172:0/3175752212 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2f200a2c00 msgr2=0x7f2f200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:29.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.312+0000 7f2f3c3f6640 1 --2- 172.21.15.172:0/3175752212 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2f200a2c00 0x7f2f200a50c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2f28000c00 tx=0x7f2f28032b80 comp rx=0 tx=0).stop 2024-09-10T08:48:29.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.312+0000 7f2f3c3f6640 1 -- 172.21.15.172:0/3175752212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f34106a80 msgr2=0x7f2f341a42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:29.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.312+0000 7f2f3c3f6640 1 --2- 172.21.15.172:0/3175752212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f34106a80 0x7f2f341a42a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c009fd0 tx=0x7f2f2c00ef10 comp rx=0 tx=0).stop 2024-09-10T08:48:29.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.313+0000 7f2f3c3f6640 1 -- 172.21.15.172:0/3175752212 shutdown_connections 2024-09-10T08:48:29.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.313+0000 7f2f3c3f6640 1 --2- 172.21.15.172:0/3175752212 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2f200a2c00 0x7f2f200a50c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.313+0000 7f2f3c3f6640 1 --2- 172.21.15.172:0/3175752212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f34106a80 0x7f2f341a42a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.313+0000 7f2f3c3f6640 1 --2- 172.21.15.172:0/3175752212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f34101420 0x7f2f34199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.313+0000 7f2f3c3f6640 1 --2- 172.21.15.172:0/3175752212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f34100a80 0x7f2f34199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.313+0000 7f2f3c3f6640 1 -- 172.21.15.172:0/3175752212 >> 172.21.15.172:0/3175752212 conn(0x7f2f340fc7a0 msgr2=0x7f2f340fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:29.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.313+0000 7f2f3c3f6640 1 -- 172.21.15.172:0/3175752212 shutdown_connections 2024-09-10T08:48:29.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:29.313+0000 7f2f3c3f6640 1 -- 172.21.15.172:0/3175752212 wait complete. 2024-09-10T08:48:29.321 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:29.326 INFO:tasks.workunit.client.0.smithi172.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-10T08:48:29.327 INFO:tasks.workunit.client.0.smithi172.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-09-10T08:48:29.327 INFO:tasks.workunit.client.0.smithi172.stderr:+ trap -- retry EXIT 2024-09-10T08:48:29.327 INFO:tasks.workunit.client.0.smithi172.stderr:+ rm -rf ffsb 2024-09-10T08:48:29.328 INFO:tasks.workunit.client.0.smithi172.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-09-10T08:48:29.334 INFO:tasks.workunit.client.0.smithi172.stderr:Cloning into 'ffsb'... 2024-09-10T08:48:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.950+0000 7fde02430640 1 -- 172.21.15.172:0/3537877435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddfc107880 msgr2=0x7fddfc109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.950+0000 7fde02430640 1 --2- 172.21.15.172:0/3537877435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddfc107880 0x7fddfc109c90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009ca0 tx=0x7fddf402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.950+0000 7fde02430640 1 -- 172.21.15.172:0/3537877435 shutdown_connections 2024-09-10T08:48:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.950+0000 7fde02430640 1 --2- 172.21.15.172:0/3537877435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddfc107880 0x7fddfc109c90 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.950+0000 7fde02430640 1 --2- 172.21.15.172:0/3537877435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fddfc104f50 0x7fddfc107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.950+0000 7fde02430640 1 --2- 172.21.15.172:0/3537877435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddfc102620 0x7fddfc104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.950+0000 7fde02430640 1 -- 172.21.15.172:0/3537877435 >> 172.21.15.172:0/3537877435 conn(0x7fddfc0fc1e0 msgr2=0x7fddfc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.950+0000 7fde02430640 1 -- 172.21.15.172:0/3537877435 shutdown_connections 2024-09-10T08:48:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.950+0000 7fde02430640 1 -- 172.21.15.172:0/3537877435 wait complete. 2024-09-10T08:48:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.951+0000 7fde02430640 1 Processor -- start 2024-09-10T08:48:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.951+0000 7fde02430640 1 -- start start 2024-09-10T08:48:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.951+0000 7fde02430640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fddfc102620 0x7fddfc197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.951+0000 7fde02430640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddfc104f50 0x7fddfc198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.951+0000 7fde02430640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddfc107880 0x7fddfc19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.951+0000 7fde02430640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc10d6b0 con 0x7fddfc104f50 2024-09-10T08:48:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.951+0000 7fde02430640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc10d530 con 0x7fddfc102620 2024-09-10T08:48:29.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.951+0000 7fde02430640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc10d830 con 0x7fddfc107880 2024-09-10T08:48:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.951+0000 7fddfb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddfc104f50 0x7fddfc198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.952+0000 7fddfb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddfc104f50 0x7fddfc198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34136/0 (socket says 172.21.15.172:34136) 2024-09-10T08:48:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.952+0000 7fde009cd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddfc107880 0x7fddfc19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.952+0000 7fddfb7fe640 1 -- 172.21.15.172:0/1869331839 learned_addr learned my addr 172.21.15.172:0/1869331839 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.952+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fddfc102620 0x7fddfc197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.952+0000 7fddfb7fe640 1 -- 172.21.15.172:0/1869331839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddfc107880 msgr2=0x7fddfc19f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.952+0000 7fddfb7fe640 1 --2- 172.21.15.172:0/1869331839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddfc107880 0x7fddfc19f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.952+0000 7fddfb7fe640 1 -- 172.21.15.172:0/1869331839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fddfc102620 msgr2=0x7fddfc197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.952+0000 7fddfb7fe640 1 --2- 172.21.15.172:0/1869331839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fddfc102620 0x7fddfc197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.952+0000 7fddfb7fe640 1 -- 172.21.15.172:0/1869331839 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc19f960 con 0x7fddfc104f50 2024-09-10T08:48:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.952+0000 7fde009cd640 1 --2- 172.21.15.172:0/1869331839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddfc107880 0x7fddfc19f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.952+0000 7fddfbfff640 1 --2- 172.21.15.172:0/1869331839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fddfc102620 0x7fddfc197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.952+0000 7fddfb7fe640 1 --2- 172.21.15.172:0/1869331839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddfc104f50 0x7fddfc198280 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fddf000e9c0 tx=0x7fddf000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.952+0000 7fddf97fa640 1 -- 172.21.15.172:0/1869331839 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf000cd80 con 0x7fddfc104f50 2024-09-10T08:48:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.952+0000 7fddf97fa640 1 -- 172.21.15.172:0/1869331839 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddf00054d0 con 0x7fddfc104f50 2024-09-10T08:48:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.952+0000 7fde02430640 1 -- 172.21.15.172:0/1869331839 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddfc19fc50 con 0x7fddfc104f50 2024-09-10T08:48:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.952+0000 7fddf97fa640 1 -- 172.21.15.172:0/1869331839 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0010430 con 0x7fddfc104f50 2024-09-10T08:48:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.952+0000 7fde02430640 1 -- 172.21.15.172:0/1869331839 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddfc1a01a0 con 0x7fddfc104f50 2024-09-10T08:48:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fde051c8740 1 Processor -- start 2024-09-10T08:48:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fde051c8740 1 -- start start 2024-09-10T08:48:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fddf97fa640 1 -- 172.21.15.172:0/1869331839 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fddf0004fd0 con 0x7fddfc104f50 2024-09-10T08:48:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fde051c8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d2f9a0 0x563875d2fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fde051c8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d302e0 0x563875d34da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fde051c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d35370 0x563875d37830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fde051c8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563875c932f0 con 0x563875d2f9a0 2024-09-10T08:48:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fde051c8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563875c3b270 con 0x563875d302e0 2024-09-10T08:48:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fde051c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563875c8f2b0 con 0x563875d35370 2024-09-10T08:48:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fde009cd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d2f9a0 0x563875d2fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d302e0 0x563875d34da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fde009cd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d2f9a0 0x563875d2fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34144/0 (socket says 172.21.15.172:34144) 2024-09-10T08:48:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fde009cd640 1 -- 172.21.15.172:0/1295234937 learned_addr learned my addr 172.21.15.172:0/1295234937 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fddfbfff640 1 -- 172.21.15.172:0/1295234937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d35370 msgr2=0x563875d37830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fddfbfff640 1 --2- 172.21.15.172:0/1295234937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d35370 0x563875d37830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fddfbfff640 1 -- 172.21.15.172:0/1295234937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d2f9a0 msgr2=0x563875d2fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fddfbfff640 1 --2- 172.21.15.172:0/1295234937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d2f9a0 0x563875d2fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fddfbfff640 1 -- 172.21.15.172:0/1295234937 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563875c3a340 con 0x563875d302e0 2024-09-10T08:48:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fde009cd640 1 --2- 172.21.15.172:0/1295234937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d2f9a0 0x563875d2fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.956+0000 7fddf97fa640 1 --2- 172.21.15.172:0/1869331839 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdde40a2c30 0x7fdde40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.957+0000 7fde009cd640 1 --2- 172.21.15.172:0/1869331839 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdde40a2c30 0x7fdde40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.957+0000 7fddfbfff640 1 --2- 172.21.15.172:0/1295234937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d302e0 0x563875d34da0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fddec009fd0 tx=0x7fddec0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dbeb4f5cfbae355 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.957+0000 7fddf97fa640 1 -- 172.21.15.172:0/1869331839 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fddf00cf7f0 con 0x7fddfc104f50 2024-09-10T08:48:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.957+0000 7fdde27fc640 1 -- 172.21.15.172:0/1295234937 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec004310 con 0x563875d302e0 2024-09-10T08:48:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.957+0000 7fdde27fc640 1 -- 172.21.15.172:0/1295234937 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddec0044b0 con 0x563875d302e0 2024-09-10T08:48:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.957+0000 7fdde27fc640 1 -- 172.21.15.172:0/1295234937 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec0035f0 con 0x563875d302e0 2024-09-10T08:48:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.958+0000 7fde051c8740 1 -- 172.21.15.172:0/1295234937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d302e0 msgr2=0x563875d34da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.958+0000 7fde009cd640 1 --2- 172.21.15.172:0/1869331839 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdde40a2c30 0x7fdde40a50f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fddf4005e30 tx=0x7fddf4005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.958+0000 7fde051c8740 1 --2- 172.21.15.172:0/1295234937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d302e0 0x563875d34da0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fddec009fd0 tx=0x7fddec0118d0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.958+0000 7fde051c8740 1 -- 172.21.15.172:0/1295234937 shutdown_connections 2024-09-10T08:48:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.958+0000 7fde051c8740 1 --2- 172.21.15.172:0/1295234937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d35370 0x563875d37830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.958+0000 7fde051c8740 1 --2- 172.21.15.172:0/1295234937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d302e0 0x563875d34da0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.958+0000 7fde051c8740 1 --2- 172.21.15.172:0/1295234937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d2f9a0 0x563875d2fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.958+0000 7fde051c8740 1 -- 172.21.15.172:0/1295234937 >> 172.21.15.172:0/1295234937 conn(0x563875d2b010 msgr2=0x563875d2b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.958+0000 7fde051c8740 1 -- 172.21.15.172:0/1295234937 shutdown_connections 2024-09-10T08:48:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.958+0000 7fde051c8740 1 -- 172.21.15.172:0/1295234937 wait complete. 2024-09-10T08:48:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.958+0000 7fde051c8740 1 -- start start 2024-09-10T08:48:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.958+0000 7fde051c8740 1 Processor -- start 2024-09-10T08:48:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fde051c8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d2f9a0 0x563875d39c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fde051c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d302e0 0x563875d3a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fde051c8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d35370 0x563875d41160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fde051c8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563875d38340 con 0x563875d2f9a0 2024-09-10T08:48:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fde051c8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563875c1c290 con 0x563875d35370 2024-09-10T08:48:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fde051c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563875d384c0 con 0x563875d302e0 2024-09-10T08:48:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fde051c8740 10 client.0 _ll_register_callbacks cb 0x563875c4d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d2f9a0 0x563875d39c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.966 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d2f9a0 0x563875d39c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34154/0 (socket says 172.21.15.172:34154) 2024-09-10T08:48:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fde009cd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d35370 0x563875d41160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fddfbfff640 1 -- 172.21.15.172:0/2913097288 learned_addr learned my addr 172.21.15.172:0/2913097288 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fddfb7fe640 1 --2- 172.21.15.172:0/2913097288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d302e0 0x563875d3a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fddfb7fe640 1 -- 172.21.15.172:0/2913097288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d35370 msgr2=0x563875d41160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fddfb7fe640 1 --2- 172.21.15.172:0/2913097288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d35370 0x563875d41160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fddfb7fe640 1 -- 172.21.15.172:0/2913097288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d2f9a0 msgr2=0x563875d39c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fddfb7fe640 1 --2- 172.21.15.172:0/2913097288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d2f9a0 0x563875d39c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fddfb7fe640 1 -- 172.21.15.172:0/2913097288 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638759a6210 con 0x563875d302e0 2024-09-10T08:48:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fddfb7fe640 1 --2- 172.21.15.172:0/2913097288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d302e0 0x563875d3a190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fddf00980f0 tx=0x7fddf009b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64cfea3088f2332c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fdddb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:48:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fdddb7fe640 1 -- 172.21.15.172:0/2913097288 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf00a0040 con 0x563875d302e0 2024-09-10T08:48:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fdddb7fe640 1 -- 172.21.15.172:0/2913097288 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fddf00cfbe0 con 0x563875d302e0 2024-09-10T08:48:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.959+0000 7fde051c8740 1 -- 172.21.15.172:0/2913097288 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddec0044b0 con 0x563875d302e0 2024-09-10T08:48:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.960+0000 7fdddb7fe640 1 -- 172.21.15.172:0/2913097288 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fddf00a4900 con 0x563875d302e0 2024-09-10T08:48:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.960+0000 7fde051c8740 10 client.25435 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.960+0000 7fde051c8740 1 -- 172.21.15.172:0/2913097288 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563875a72620 con 0x563875d302e0 2024-09-10T08:48:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.960+0000 7fdddb7fe640 1 -- 172.21.15.172:0/2913097288 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddec0044b0 con 0x563875d302e0 2024-09-10T08:48:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.960+0000 7fdddb7fe640 1 -- 172.21.15.172:0/2913097288 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fddf009b0e0 con 0x563875d302e0 2024-09-10T08:48:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.960+0000 7fde051c8740 10 client.25435 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.960+0000 7fde051c8740 10 client.25435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.960+0000 7fde051c8740 20 client.25435 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.960+0000 7fde051c8740 1 --2- 172.21.15.172:0/2913097288 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563875d45820 0x563875d47c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.960+0000 7fde051c8740 4 client.25435 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.960+0000 7fde051c8740 1 -- 172.21.15.172:0/2913097288 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563875ca0a20 con 0x563875d45820 2024-09-10T08:48:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.961+0000 7fddfbfff640 1 --2- 172.21.15.172:0/2913097288 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563875d45820 0x563875d47c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.961+0000 7fdddb7fe640 10 client.25435 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.961+0000 7fddfbfff640 1 --2- 172.21.15.172:0/2913097288 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563875d45820 0x563875d47c10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee5e1a9b0efb816 server_cookie=dc29c88e85b85703 in_seq=0 out_seq=0 2024-09-10T08:48:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.962+0000 7fdddb7fe640 1 -- 172.21.15.172:0/2913097288 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563875ca0a20 con 0x563875d45820 2024-09-10T08:48:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.962+0000 7fdddb7fe640 10 client.25435 handle_command_reply: tid=0 2024-09-10T08:48:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.962+0000 7fde051c8740 1 client.25435 shutdown 2024-09-10T08:48:29.970 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.962+0000 7fde051c8740 1 -- 172.21.15.172:0/2913097288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d302e0 msgr2=0x563875d3a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.962+0000 7fde051c8740 1 --2- 172.21.15.172:0/2913097288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d302e0 0x563875d3a190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fddf00980f0 tx=0x7fddf009b890 comp rx=0 tx=0).stop 2024-09-10T08:48:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.962+0000 7fde051c8740 1 -- 172.21.15.172:0/2913097288 shutdown_connections 2024-09-10T08:48:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.962+0000 7fde051c8740 1 --2- 172.21.15.172:0/2913097288 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563875d45820 0x563875d47c10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.962+0000 7fde051c8740 1 --2- 172.21.15.172:0/2913097288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d35370 0x563875d41160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.962+0000 7fde051c8740 1 --2- 172.21.15.172:0/2913097288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d302e0 0x563875d3a190 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.962+0000 7fde051c8740 1 --2- 172.21.15.172:0/2913097288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d2f9a0 0x563875d39c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.971 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.962+0000 7fde051c8740 1 -- 172.21.15.172:0/2913097288 >> 172.21.15.172:0/2913097288 conn(0x563875d2b010 msgr2=0x563875d2c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.962+0000 7fdddb7fe640 0 client.25435 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.962+0000 7fde051c8740 1 -- 172.21.15.172:0/2913097288 shutdown_connections 2024-09-10T08:48:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.962+0000 7fde051c8740 1 -- 172.21.15.172:0/2913097288 wait complete. 2024-09-10T08:48:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.962+0000 7fde051c8740 20 client.25435 trim_cache size 0 max 16384 2024-09-10T08:48:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.979+0000 7fde051c8740 1 Processor -- start 2024-09-10T08:48:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.979+0000 7fde051c8740 1 -- start start 2024-09-10T08:48:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.979+0000 7fde051c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d2b010 0x563875d2b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.979+0000 7fde051c8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d2f9a0 0x563875d41120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.979+0000 7fde051c8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d302e0 0x563875d2b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.979+0000 7fde051c8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1a8420 con 0x563875d302e0 2024-09-10T08:48:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.979+0000 7fde051c8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1a9f60 con 0x563875d2f9a0 2024-09-10T08:48:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.979+0000 7fde051c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1a8af0 con 0x563875d2b010 2024-09-10T08:48:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.979+0000 7fddfb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d2f9a0 0x563875d41120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.979+0000 7fde009cd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d302e0 0x563875d2b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.979+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d2b010 0x563875d2b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.979+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d2b010 0x563875d2b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:42534/0 (socket says 172.21.15.172:42534) 2024-09-10T08:48:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.979+0000 7fddfbfff640 1 -- 172.21.15.172:0/3426267745 learned_addr learned my addr 172.21.15.172:0/3426267745 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.979+0000 7fddfb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d2f9a0 0x563875d41120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:42016/0 (socket says 172.21.15.172:42016) 2024-09-10T08:48:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.980+0000 7fddfbfff640 1 -- 172.21.15.172:0/3426267745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d2f9a0 msgr2=0x563875d41120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.980+0000 7fddfbfff640 1 --2- 172.21.15.172:0/3426267745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d2f9a0 0x563875d41120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.980+0000 7fddfbfff640 1 -- 172.21.15.172:0/3426267745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d302e0 msgr2=0x563875d2b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.980+0000 7fddfbfff640 1 --2- 172.21.15.172:0/3426267745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d302e0 0x563875d2b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.980+0000 7fddfbfff640 1 -- 172.21.15.172:0/3426267745 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563875d2ae30 con 0x563875d2b010 2024-09-10T08:48:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.980+0000 7fddfbfff640 1 --2- 172.21.15.172:0/3426267745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d2b010 0x563875d2b3f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fddec0058c0 tx=0x7fddec005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1022b5a0ecfa35be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.980+0000 7fdde1ffb640 1 -- 172.21.15.172:0/3426267745 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec038070 con 0x563875d2b010 2024-09-10T08:48:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.980+0000 7fdde1ffb640 1 -- 172.21.15.172:0/3426267745 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddec022af0 con 0x563875d2b010 2024-09-10T08:48:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.980+0000 7fdde1ffb640 1 -- 172.21.15.172:0/3426267745 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec003ae0 con 0x563875d2b010 2024-09-10T08:48:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.981+0000 7fde051c8740 1 -- 172.21.15.172:0/3426267745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d2b010 msgr2=0x563875d2b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.981+0000 7fde051c8740 1 --2- 172.21.15.172:0/3426267745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d2b010 0x563875d2b3f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fddec0058c0 tx=0x7fddec005d30 comp rx=0 tx=0).stop 2024-09-10T08:48:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.981+0000 7fde051c8740 1 -- 172.21.15.172:0/3426267745 shutdown_connections 2024-09-10T08:48:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.981+0000 7fde051c8740 1 --2- 172.21.15.172:0/3426267745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d302e0 0x563875d2b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.981+0000 7fde051c8740 1 --2- 172.21.15.172:0/3426267745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d2f9a0 0x563875d41120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.981+0000 7fde051c8740 1 --2- 172.21.15.172:0/3426267745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d2b010 0x563875d2b3f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.981+0000 7fde051c8740 1 -- 172.21.15.172:0/3426267745 >> 172.21.15.172:0/3426267745 conn(0x563875d34f00 msgr2=0x563875d31520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.981+0000 7fde051c8740 1 -- 172.21.15.172:0/3426267745 shutdown_connections 2024-09-10T08:48:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.981+0000 7fde051c8740 1 -- 172.21.15.172:0/3426267745 wait complete. 2024-09-10T08:48:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.981+0000 7fde051c8740 1 -- start start 2024-09-10T08:48:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.981+0000 7fde051c8740 1 Processor -- start 2024-09-10T08:48:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.982+0000 7fde051c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d2b010 0x563875d3df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.982+0000 7fde051c8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d2f9a0 0x563875d3e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.982+0000 7fde051c8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d302e0 0x563875dae450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.982+0000 7fde051c8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1a9890 con 0x563875d302e0 2024-09-10T08:48:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.982+0000 7fde051c8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1a91c0 con 0x563875d2f9a0 2024-09-10T08:48:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.982+0000 7fddfb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d2f9a0 0x563875d3e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.982+0000 7fddfb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d2f9a0 0x563875d3e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:42022/0 (socket says 172.21.15.172:42022) 2024-09-10T08:48:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.982+0000 7fddfbfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d2b010 0x563875d3df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.982+0000 7fddfb7fe640 1 -- 172.21.15.172:0/2882636816 learned_addr learned my addr 172.21.15.172:0/2882636816 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.982+0000 7fde051c8740 1 -- 172.21.15.172:0/2882636816 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc1a6400 con 0x563875d2b010 2024-09-10T08:48:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.982+0000 7fde009cd640 1 --2- 172.21.15.172:0/2882636816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d302e0 0x563875dae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.982+0000 7fde051c8740 10 client.0 _ll_register_callbacks cb 0x7fddf000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.982+0000 7fddfb7fe640 1 -- 172.21.15.172:0/2882636816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d2b010 msgr2=0x563875d3df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.982+0000 7fddfb7fe640 1 --2- 172.21.15.172:0/2882636816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d2b010 0x563875d3df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.982+0000 7fddfb7fe640 1 -- 172.21.15.172:0/2882636816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d302e0 msgr2=0x563875dae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.983+0000 7fddfb7fe640 1 --2- 172.21.15.172:0/2882636816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d302e0 0x563875dae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.983+0000 7fddfb7fe640 1 -- 172.21.15.172:0/2882636816 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563875d3eb40 con 0x563875d2f9a0 2024-09-10T08:48:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.983+0000 7fddfbfff640 1 --2- 172.21.15.172:0/2882636816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d2b010 0x563875d3df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.983+0000 7fde009cd640 1 --2- 172.21.15.172:0/2882636816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d302e0 0x563875dae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.983+0000 7fddfb7fe640 1 --2- 172.21.15.172:0/2882636816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d2f9a0 0x563875d3e4a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fddf009b990 tx=0x7fddf00cfb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d88cb9d41bc8a1e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.983+0000 7fdddb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:48:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.983+0000 7fdddb7fe640 1 -- 172.21.15.172:0/2882636816 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf00a0040 con 0x563875d2f9a0 2024-09-10T08:48:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.983+0000 7fdddb7fe640 1 -- 172.21.15.172:0/2882636816 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fddf00ba380 con 0x563875d2f9a0 2024-09-10T08:48:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.983+0000 7fde051c8740 1 -- 172.21.15.172:0/2882636816 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddec022af0 con 0x563875d2f9a0 2024-09-10T08:48:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.983+0000 7fdddb7fe640 1 -- 172.21.15.172:0/2882636816 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fddf0097f30 con 0x563875d2f9a0 2024-09-10T08:48:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.983+0000 7fde051c8740 10 client.25496 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.983+0000 7fde051c8740 1 -- 172.21.15.172:0/2882636816 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563875dafb00 con 0x563875d2f9a0 2024-09-10T08:48:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.983+0000 7fdddb7fe640 1 -- 172.21.15.172:0/2882636816 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddf00ba580 con 0x563875d2f9a0 2024-09-10T08:48:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.984+0000 7fdddb7fe640 1 -- 172.21.15.172:0/2882636816 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fddf00a4b50 con 0x563875d2f9a0 2024-09-10T08:48:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.984+0000 7fde051c8740 10 client.25496 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.984+0000 7fde051c8740 10 client.25496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.984+0000 7fde051c8740 20 client.25496 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.984+0000 7fde051c8740 1 --2- 172.21.15.172:0/2882636816 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563875d35370 0x563875db24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.984+0000 7fde051c8740 4 client.25496 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:48:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.984+0000 7fde051c8740 1 -- 172.21.15.172:0/2882636816 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fddec022af0 con 0x563875d35370 2024-09-10T08:48:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.984+0000 7fddfbfff640 1 --2- 172.21.15.172:0/2882636816 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563875d35370 0x563875db24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.985+0000 7fddfbfff640 1 --2- 172.21.15.172:0/2882636816 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563875d35370 0x563875db24e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=225219726ec15774 server_cookie=9ac0122a04d3286b in_seq=0 out_seq=0 2024-09-10T08:48:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:29.985+0000 7fdddb7fe640 10 client.25496 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:30.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.064+0000 7fdb3ea31640 1 -- 172.21.15.172:0/2681158079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb38102570 msgr2=0x7fdb381029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:30.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.064+0000 7fdb3ea31640 1 --2- 172.21.15.172:0/2681158079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb38102570 0x7fdb381029b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009900 tx=0x7fdb2c02a480 comp rx=0 tx=0).stop 2024-09-10T08:48:30.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.064+0000 7fdb3ea31640 1 -- 172.21.15.172:0/2681158079 shutdown_connections 2024-09-10T08:48:30.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.064+0000 7fdb3ea31640 1 --2- 172.21.15.172:0/2681158079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb38102ef0 0x7fdb3810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.064+0000 7fdb3ea31640 1 --2- 172.21.15.172:0/2681158079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb38102570 0x7fdb381029b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.064+0000 7fdb3ea31640 1 --2- 172.21.15.172:0/2681158079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb38100e20 0x7fdb38101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.064+0000 7fdb3ea31640 1 -- 172.21.15.172:0/2681158079 >> 172.21.15.172:0/2681158079 conn(0x7fdb380fc7d0 msgr2=0x7fdb380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:30.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.064+0000 7fdb3ea31640 1 -- 172.21.15.172:0/2681158079 shutdown_connections 2024-09-10T08:48:30.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.065+0000 7fdb3ea31640 1 -- 172.21.15.172:0/2681158079 wait complete. 2024-09-10T08:48:30.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.065+0000 7fdb3ea31640 1 Processor -- start 2024-09-10T08:48:30.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.065+0000 7fdb3ea31640 1 -- start start 2024-09-10T08:48:30.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.065+0000 7fdb3ea31640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb38100e20 0x7fdb38198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.065+0000 7fdb3ea31640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb38102570 0x7fdb38198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.065+0000 7fdb3ea31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb38102ef0 0x7fdb3819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.065+0000 7fdb3ea31640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdb38110e30 con 0x7fdb38102ef0 2024-09-10T08:48:30.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.065+0000 7fdb3ea31640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb38110cb0 con 0x7fdb38102570 2024-09-10T08:48:30.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.066+0000 7fdb3ea31640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdb38110fb0 con 0x7fdb38100e20 2024-09-10T08:48:30.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.066+0000 7fdb37fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb38100e20 0x7fdb38198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.066+0000 7fdb3cfce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb38102ef0 0x7fdb3819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.066+0000 7fdb377fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb38102570 0x7fdb38198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.066+0000 7fdb3cfce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb38102ef0 0x7fdb3819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34178/0 (socket says 172.21.15.172:34178) 2024-09-10T08:48:30.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.066+0000 7fdb37fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb38100e20 0x7fdb38198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:42560/0 (socket says 172.21.15.172:42560) 2024-09-10T08:48:30.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.066+0000 7fdb3cfce640 1 -- 172.21.15.172:0/1299183416 learned_addr learned my addr 172.21.15.172:0/1299183416 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:30.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.066+0000 7fdb377fe640 1 -- 172.21.15.172:0/1299183416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb38100e20 msgr2=0x7fdb38198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:30.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.066+0000 7fdb377fe640 1 --2- 172.21.15.172:0/1299183416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb38100e20 0x7fdb38198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.066+0000 7fdb377fe640 1 -- 172.21.15.172:0/1299183416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb38102ef0 msgr2=0x7fdb3819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:30.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.066+0000 7fdb377fe640 1 --2- 172.21.15.172:0/1299183416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb38102ef0 0x7fdb3819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.066+0000 7fdb377fe640 1 -- 172.21.15.172:0/1299183416 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3819d3b0 con 0x7fdb38102570 2024-09-10T08:48:30.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.066+0000 7fdb3cfce640 1 --2- 172.21.15.172:0/1299183416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb38102ef0 0x7fdb3819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:30.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.066+0000 7fdb377fe640 1 --2- 172.21.15.172:0/1299183416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb38102570 0x7fdb38198940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c005940 tx=0x7fdb2c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:30.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.066+0000 7fdb357fa640 1 -- 172.21.15.172:0/1299183416 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c03f030 con 0x7fdb38102570 2024-09-10T08:48:30.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.067+0000 7fdb3ea31640 1 -- 172.21.15.172:0/1299183416 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3819d640 con 0x7fdb38102570 2024-09-10T08:48:30.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.067+0000 7fdb357fa640 1 -- 172.21.15.172:0/1299183416 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb2c004440 con 0x7fdb38102570 2024-09-10T08:48:30.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.067+0000 7fdb357fa640 1 -- 172.21.15.172:0/1299183416 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c035070 con 0x7fdb38102570 2024-09-10T08:48:30.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.067+0000 7fdb3ea31640 1 -- 172.21.15.172:0/1299183416 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb381a4f80 con 0x7fdb38102570 2024-09-10T08:48:30.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.069+0000 7fdb357fa640 1 -- 172.21.15.172:0/1299183416 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fdb2c042030 con 0x7fdb38102570 2024-09-10T08:48:30.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb40e0a740 1 Processor -- start 2024-09-10T08:48:30.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb40e0a740 1 -- start start 2024-09-10T08:48:30.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb357fa640 1 --2- 172.21.15.172:0/1299183416 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb200a2b80 0x7fdb200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb40e0a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344eacab40 0x56344eac9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb40e0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eac95d0 0x56344eac9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb40e0a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344ead4370 0x56344ead6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb357fa640 1 -- 172.21.15.172:0/1299183416 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fdb2c0ef600 con 0x7fdb38102570 2024-09-10T08:48:30.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb37fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344eacab40 0x56344eac9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb3cfce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344ead4370 0x56344ead6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb3cfce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344ead4370 0x56344ead6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34194/0 (socket says 172.21.15.172:34194) 2024-09-10T08:48:30.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb37fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344eacab40 0x56344eac9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:42564/0 (socket says 172.21.15.172:42564) 2024-09-10T08:48:30.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb3cfce640 1 -- 172.21.15.172:0/2651718354 learned_addr learned my addr 172.21.15.172:0/2651718354 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:30.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb377fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eac95d0 0x56344eac9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb40e0a740 1 -- 172.21.15.172:0/2651718354 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56344ea2f2f0 con 0x56344ead4370 2024-09-10T08:48:30.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb40e0a740 1 -- 172.21.15.172:0/2651718354 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56344e9d6cf0 con 0x56344eac95d0 2024-09-10T08:48:30.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb40e0a740 1 -- 172.21.15.172:0/2651718354 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56344ea2b2b0 con 0x56344eacab40 2024-09-10T08:48:30.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb3cfce640 1 --2- 172.21.15.172:0/1299183416 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb200a2b80 0x7fdb200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb3cfce640 1 -- 172.21.15.172:0/2651718354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344eacab40 msgr2=0x56344eac9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:30.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb3cfce640 1 --2- 172.21.15.172:0/2651718354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344eacab40 0x56344eac9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb3cfce640 1 -- 172.21.15.172:0/2651718354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eac95d0 msgr2=0x56344eac9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:30.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb3cfce640 1 --2- 172.21.15.172:0/2651718354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eac95d0 0x56344eac9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.070+0000 7fdb3cfce640 1 -- 172.21.15.172:0/2651718354 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56344e9bb3a0 con 0x56344ead4370 2024-09-10T08:48:30.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.071+0000 7fdb37fff640 1 --2- 172.21.15.172:0/2651718354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344eacab40 0x56344eac9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:30.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.071+0000 7fdb3cfce640 1 --2- 172.21.15.172:0/2651718354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344ead4370 0x56344ead6760 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdb300136f0 tx=0x7fdb30013bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb55c7f78c224164 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:30.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.071+0000 7fdb3cfce640 1 --2- 172.21.15.172:0/1299183416 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb200a2b80 0x7fdb200a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000e720 tx=0x7fdb30018000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:30.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.071+0000 7fdb1e7fc640 1 -- 172.21.15.172:0/2651718354 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30025070 con 0x56344ead4370 2024-09-10T08:48:30.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.071+0000 7fdb1e7fc640 1 -- 172.21.15.172:0/2651718354 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb30011700 con 0x56344ead4370 2024-09-10T08:48:30.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.071+0000 7fdb1e7fc640 1 -- 172.21.15.172:0/2651718354 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3000cd80 con 0x56344ead4370 2024-09-10T08:48:30.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.072+0000 7fdb40e0a740 1 -- 172.21.15.172:0/2651718354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344ead4370 msgr2=0x56344ead6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:30.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.072+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/2651718354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344ead4370 0x56344ead6760 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdb300136f0 tx=0x7fdb30013bb0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.072+0000 7fdb40e0a740 1 -- 172.21.15.172:0/2651718354 shutdown_connections 2024-09-10T08:48:30.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.072+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/2651718354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344ead4370 0x56344ead6760 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.072+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/2651718354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eac95d0 0x56344eac9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.072+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/2651718354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344eacab40 0x56344eac9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.072+0000 7fdb40e0a740 1 -- 172.21.15.172:0/2651718354 >> 172.21.15.172:0/2651718354 conn(0x56344ea92310 msgr2=0x56344eac5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:30.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.072+0000 7fdb40e0a740 1 -- 172.21.15.172:0/2651718354 shutdown_connections 2024-09-10T08:48:30.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.072+0000 7fdb40e0a740 1 -- 172.21.15.172:0/2651718354 wait complete. 2024-09-10T08:48:30.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.072+0000 7fdb40e0a740 1 -- start start 2024-09-10T08:48:30.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.072+0000 7fdb40e0a740 1 Processor -- start 2024-09-10T08:48:30.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb40e0a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344eac95d0 0x56344ead3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb40e0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eacab40 0x56344ead1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb40e0a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344ead4370 0x56344ead1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb40e0a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56344eadb420 con 0x56344ead4370 2024-09-10T08:48:30.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb40e0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56344e9ea060 con 0x56344eacab40 2024-09-10T08:48:30.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb40e0a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56344eadb5a0 con 0x56344eac95d0 2024-09-10T08:48:30.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb40e0a740 10 client.0 _ll_register_callbacks cb 0x56344e73d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:30.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb3cfce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344ead4370 0x56344ead1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb377fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eacab40 0x56344ead1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb3cfce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344ead4370 0x56344ead1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34200/0 (socket says 172.21.15.172:34200) 2024-09-10T08:48:30.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb3cfce640 1 -- 172.21.15.172:0/4151074373 learned_addr learned my addr 172.21.15.172:0/4151074373 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:30.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb37fff640 1 --2- 172.21.15.172:0/4151074373 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344eac95d0 0x56344ead3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb3cfce640 1 -- 172.21.15.172:0/4151074373 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344eac95d0 msgr2=0x56344ead3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:30.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb3cfce640 1 --2- 172.21.15.172:0/4151074373 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344eac95d0 0x56344ead3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb3cfce640 1 -- 172.21.15.172:0/4151074373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eacab40 msgr2=0x56344ead1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:30.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb3cfce640 1 --2- 172.21.15.172:0/4151074373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eacab40 0x56344ead1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb3cfce640 1 -- 172.21.15.172:0/4151074373 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56344e7a03c0 con 0x56344ead4370 2024-09-10T08:48:30.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb37fff640 1 --2- 172.21.15.172:0/4151074373 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344eac95d0 0x56344ead3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:30.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb377fe640 1 --2- 172.21.15.172:0/4151074373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eacab40 0x56344ead1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:30.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb3cfce640 1 --2- 172.21.15.172:0/4151074373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344ead4370 0x56344ead1810 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdb30013fe0 tx=0x7fdb3000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23a336d6d26fc39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:30.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.073+0000 7fdb177fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:30.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.074+0000 7fdb177fe640 1 -- 172.21.15.172:0/4151074373 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3002e070 con 0x56344ead4370 2024-09-10T08:48:30.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.074+0000 7fdb177fe640 1 -- 172.21.15.172:0/4151074373 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fdb300049d0 con 0x56344ead4370 2024-09-10T08:48:30.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.074+0000 7fdb40e0a740 1 -- 172.21.15.172:0/4151074373 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb30011700 con 0x56344ead4370 2024-09-10T08:48:30.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.074+0000 7fdb177fe640 1 -- 172.21.15.172:0/4151074373 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb30025040 con 0x56344ead4370 2024-09-10T08:48:30.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.074+0000 7fdb40e0a740 10 client.16146 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:30.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.074+0000 7fdb40e0a740 1 -- 172.21.15.172:0/4151074373 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56344eae6f40 con 0x56344ead4370 2024-09-10T08:48:30.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.074+0000 7fdb177fe640 1 -- 172.21.15.172:0/4151074373 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb30011700 con 0x56344ead4370 2024-09-10T08:48:30.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.074+0000 7fdb177fe640 1 -- 172.21.15.172:0/4151074373 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdb30005680 con 0x56344ead4370 2024-09-10T08:48:30.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.074+0000 7fdb40e0a740 10 client.16146 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:30.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.074+0000 7fdb40e0a740 10 client.16146 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:48:30.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.074+0000 7fdb40e0a740 20 client.16146 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:30.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.074+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/4151074373 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x56344eae8350 0x56344eaea740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.074+0000 7fdb40e0a740 4 client.16146 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:30.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.074+0000 7fdb40e0a740 1 -- 172.21.15.172:0/4151074373 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56344ea3c860 con 0x56344eae8350 2024-09-10T08:48:30.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.075+0000 7fdb37fff640 1 --2- 172.21.15.172:0/4151074373 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x56344eae8350 0x56344eaea740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.075+0000 7fdb37fff640 1 --2- 172.21.15.172:0/4151074373 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x56344eae8350 0x56344eaea740 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fce0ee4032f4bc5 server_cookie=d605aa6d43dcb82d in_seq=0 out_seq=0 2024-09-10T08:48:30.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.075+0000 7fdb177fe640 10 client.16146 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:48:30.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.076+0000 7fdb177fe640 1 -- 172.21.15.172:0/4151074373 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56344ea3c860 con 0x56344eae8350 2024-09-10T08:48:30.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.076+0000 7fdb177fe640 10 client.16146 handle_command_reply: tid=0 2024-09-10T08:48:30.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.076+0000 7fdb40e0a740 1 client.16146 shutdown 2024-09-10T08:48:30.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.076+0000 7fdb40e0a740 1 -- 172.21.15.172:0/4151074373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344ead4370 msgr2=0x56344ead1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:30.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.076+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/4151074373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344ead4370 0x56344ead1810 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdb30013fe0 tx=0x7fdb3000ee40 comp rx=0 tx=0).stop 2024-09-10T08:48:30.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.076+0000 7fdb40e0a740 1 -- 172.21.15.172:0/4151074373 shutdown_connections 2024-09-10T08:48:30.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.076+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/4151074373 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x56344eae8350 0x56344eaea740 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.076+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/4151074373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344ead4370 0x56344ead1810 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.076+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/4151074373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eacab40 0x56344ead1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.076+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/4151074373 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344eac95d0 0x56344ead3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.076+0000 7fdb177fe640 0 client.16146 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:48:30.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.077+0000 7fdb40e0a740 1 -- 172.21.15.172:0/4151074373 >> 172.21.15.172:0/4151074373 conn(0x56344ea92310 msgr2=0x56344eacb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:30.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.077+0000 7fdb40e0a740 1 -- 172.21.15.172:0/4151074373 shutdown_connections 2024-09-10T08:48:30.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.077+0000 7fdb40e0a740 1 -- 172.21.15.172:0/4151074373 wait complete. 2024-09-10T08:48:30.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.077+0000 7fdb40e0a740 20 client.16146 trim_cache size 0 max 16384 2024-09-10T08:48:30.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.091+0000 7fdb40e0a740 1 Processor -- start 2024-09-10T08:48:30.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.091+0000 7fdb40e0a740 1 -- start start 2024-09-10T08:48:30.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.091+0000 7fdb40e0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eac95d0 0x56344eac5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.091+0000 7fdb40e0a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344eacab40 0x56344ead0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.091+0000 7fdb40e0a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344ead4370 0x56344eaefd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.091+0000 7fdb40e0a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381ac880 con 0x56344eacab40 2024-09-10T08:48:30.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.091+0000 7fdb40e0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381ae3c0 con 0x56344eac95d0 2024-09-10T08:48:30.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.091+0000 7fdb40e0a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381acf50 con 0x56344ead4370 2024-09-10T08:48:30.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.092+0000 7fdb377fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344eacab40 0x56344ead0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.092+0000 7fdb37fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eac95d0 0x56344eac5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.092+0000 7fdb377fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344eacab40 0x56344ead0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34202/0 (socket says 172.21.15.172:34202) 2024-09-10T08:48:30.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.092+0000 7fdb377fe640 1 -- 172.21.15.172:0/3767871724 learned_addr learned my addr 172.21.15.172:0/3767871724 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:30.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.092+0000 7fdb3cfce640 1 --2- 172.21.15.172:0/3767871724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344ead4370 0x56344eaefd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.092+0000 7fdb377fe640 1 -- 172.21.15.172:0/3767871724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344ead4370 msgr2=0x56344eaefd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:30.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.092+0000 7fdb377fe640 1 --2- 172.21.15.172:0/3767871724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344ead4370 0x56344eaefd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.092+0000 7fdb377fe640 1 -- 172.21.15.172:0/3767871724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eac95d0 msgr2=0x56344eac5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:30.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.092+0000 7fdb377fe640 1 --2- 172.21.15.172:0/3767871724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eac95d0 0x56344eac5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.092+0000 7fdb377fe640 1 -- 172.21.15.172:0/3767871724 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56344ead2010 con 0x56344eacab40 2024-09-10T08:48:30.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.092+0000 7fdb377fe640 1 --2- 172.21.15.172:0/3767871724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344eacab40 0x56344ead0060 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0bb850 tx=0x7fdb2c0bb750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd916fc1786fc7b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:30.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.092+0000 7fdb1dffb640 1 -- 172.21.15.172:0/3767871724 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c049040 con 0x56344eacab40 2024-09-10T08:48:30.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.092+0000 7fdb1dffb640 1 -- 172.21.15.172:0/3767871724 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb2c0bbb50 con 0x56344eacab40 2024-09-10T08:48:30.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.092+0000 7fdb1dffb640 1 -- 172.21.15.172:0/3767871724 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0b81c0 con 0x56344eacab40 2024-09-10T08:48:30.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.093+0000 7fdb40e0a740 1 -- 172.21.15.172:0/3767871724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344eacab40 msgr2=0x56344ead0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:30.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.093+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/3767871724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344eacab40 0x56344ead0060 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0bb850 tx=0x7fdb2c0bb750 comp rx=0 tx=0).stop 2024-09-10T08:48:30.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.093+0000 7fdb40e0a740 1 -- 172.21.15.172:0/3767871724 shutdown_connections 2024-09-10T08:48:30.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.093+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/3767871724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344ead4370 0x56344eaefd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.093+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/3767871724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344eacab40 0x56344ead0060 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.093+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/3767871724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eac95d0 0x56344eac5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.093+0000 7fdb40e0a740 1 -- 172.21.15.172:0/3767871724 >> 172.21.15.172:0/3767871724 conn(0x56344ea92310 msgr2=0x56344eacf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:30.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.093+0000 7fdb40e0a740 1 -- 172.21.15.172:0/3767871724 shutdown_connections 2024-09-10T08:48:30.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.093+0000 7fdb40e0a740 1 -- 172.21.15.172:0/3767871724 wait complete. 2024-09-10T08:48:30.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.093+0000 7fdb40e0a740 1 -- start start 2024-09-10T08:48:30.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.094+0000 7fdb40e0a740 1 Processor -- start 2024-09-10T08:48:30.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.094+0000 7fdb40e0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eac95d0 0x56344eae0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.094+0000 7fdb40e0a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344eacab40 0x56344eadc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.094+0000 7fdb40e0a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344ead4370 0x56344eadc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.094+0000 7fdb40e0a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381adcf0 con 0x56344eacab40 2024-09-10T08:48:30.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.094+0000 7fdb40e0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381ad620 con 0x56344eac95d0 2024-09-10T08:48:30.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.094+0000 7fdb40e0a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381aa860 con 0x56344ead4370 2024-09-10T08:48:30.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.094+0000 7fdb40e0a740 10 client.0 _ll_register_callbacks cb 0x7fdb2c03b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:30.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.094+0000 7fdb377fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344eacab40 0x56344eadc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.094+0000 7fdb37fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eac95d0 0x56344eae0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.094+0000 7fdb377fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344eacab40 0x56344eadc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34218/0 (socket says 172.21.15.172:34218) 2024-09-10T08:48:30.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.094+0000 7fdb377fe640 1 -- 172.21.15.172:0/1483732471 learned_addr learned my addr 172.21.15.172:0/1483732471 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:30.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.094+0000 7fdb3cfce640 1 --2- 172.21.15.172:0/1483732471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344ead4370 0x56344eadc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.095+0000 7fdb37fff640 1 -- 172.21.15.172:0/1483732471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344ead4370 msgr2=0x56344eadc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:30.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.095+0000 7fdb37fff640 1 --2- 172.21.15.172:0/1483732471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344ead4370 0x56344eadc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.095+0000 7fdb37fff640 1 -- 172.21.15.172:0/1483732471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344eacab40 msgr2=0x56344eadc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:30.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.095+0000 7fdb37fff640 1 --2- 172.21.15.172:0/1483732471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344eacab40 0x56344eadc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.095+0000 7fdb37fff640 1 -- 172.21.15.172:0/1483732471 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56344eadcb60 con 0x56344eac95d0 2024-09-10T08:48:30.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.095+0000 7fdb377fe640 1 --2- 172.21.15.172:0/1483732471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344eacab40 0x56344eadc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:30.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.095+0000 7fdb37fff640 1 --2- 172.21.15.172:0/1483732471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eac95d0 0x56344eae0d30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x56344eaeac80 tx=0x7fdb28010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=684bd251f28c74ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:30.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.095+0000 7fdb177fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:48:30.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.095+0000 7fdb177fe640 1 -- 172.21.15.172:0/1483732471 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28005220 con 0x56344eac95d0 2024-09-10T08:48:30.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.095+0000 7fdb177fe640 1 -- 172.21.15.172:0/1483732471 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fdb280053c0 con 0x56344eac95d0 2024-09-10T08:48:30.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.095+0000 7fdb40e0a740 1 -- 172.21.15.172:0/1483732471 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb2c0bbb50 con 0x56344eac95d0 2024-09-10T08:48:30.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.095+0000 7fdb177fe640 1 -- 172.21.15.172:0/1483732471 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb280055c0 con 0x56344eac95d0 2024-09-10T08:48:30.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.095+0000 7fdb40e0a740 10 client.25514 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:30.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.095+0000 7fdb40e0a740 1 -- 172.21.15.172:0/1483732471 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56344eb52b20 con 0x56344eac95d0 2024-09-10T08:48:30.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.096+0000 7fdb177fe640 1 -- 172.21.15.172:0/1483732471 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb2c0bbb50 con 0x56344eac95d0 2024-09-10T08:48:30.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.096+0000 7fdb177fe640 1 -- 172.21.15.172:0/1483732471 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdb28025030 con 0x56344eac95d0 2024-09-10T08:48:30.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.096+0000 7fdb40e0a740 10 client.25514 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:30.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.096+0000 7fdb40e0a740 10 client.25514 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:48:30.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.096+0000 7fdb40e0a740 20 client.25514 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:30.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.096+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/1483732471 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x56344eae8350 0x56344eb552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:30.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.096+0000 7fdb40e0a740 4 client.25514 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:48:30.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.096+0000 7fdb40e0a740 1 -- 172.21.15.172:0/1483732471 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56344ea36d70 con 0x56344eae8350 2024-09-10T08:48:30.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.096+0000 7fdb377fe640 1 --2- 172.21.15.172:0/1483732471 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x56344eae8350 0x56344eb552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:30.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.096+0000 7fdb177fe640 10 client.25514 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:48:30.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.096+0000 7fdb377fe640 1 --2- 172.21.15.172:0/1483732471 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x56344eae8350 0x56344eb552e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=424955d2b01ed0f0 server_cookie=22e34600e28524ba in_seq=0 out_seq=0 2024-09-10T08:48:30.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.097+0000 7fdb177fe640 1 -- 172.21.15.172:0/1483732471 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56344ea36d70 con 0x56344eae8350 2024-09-10T08:48:30.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.097+0000 7fdb177fe640 10 client.25514 handle_command_reply: tid=0 2024-09-10T08:48:30.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.097+0000 7fdb40e0a740 1 client.25514 shutdown 2024-09-10T08:48:30.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.097+0000 7fdb40e0a740 1 -- 172.21.15.172:0/1483732471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eac95d0 msgr2=0x56344eae0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:30.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.097+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/1483732471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eac95d0 0x56344eae0d30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x56344eaeac80 tx=0x7fdb28010800 comp rx=0 tx=0).stop 2024-09-10T08:48:30.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.097+0000 7fdb40e0a740 1 -- 172.21.15.172:0/1483732471 shutdown_connections 2024-09-10T08:48:30.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.097+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/1483732471 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x56344eae8350 0x56344eb552e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.097+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/1483732471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56344ead4370 0x56344eadc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.097+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/1483732471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56344eacab40 0x56344eadc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.097+0000 7fdb177fe640 0 client.25514 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:48:30.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.097+0000 7fdb40e0a740 1 --2- 172.21.15.172:0/1483732471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56344eac95d0 0x56344eae0d30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.097+0000 7fdb40e0a740 1 -- 172.21.15.172:0/1483732471 >> 172.21.15.172:0/1483732471 conn(0x56344ea92310 msgr2=0x56344ead0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:30.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.097+0000 7fdb40e0a740 1 -- 172.21.15.172:0/1483732471 shutdown_connections 2024-09-10T08:48:30.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.097+0000 7fdb40e0a740 1 -- 172.21.15.172:0/1483732471 wait complete. 2024-09-10T08:48:30.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.097+0000 7fdb40e0a740 20 client.25514 trim_cache size 0 max 16384 2024-09-10T08:48:30.106 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:48:30.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.098+0000 7fdb3ea31640 1 -- 172.21.15.172:0/1299183416 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb200a2b80 msgr2=0x7fdb200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:30.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.098+0000 7fdb3ea31640 1 --2- 172.21.15.172:0/1299183416 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb200a2b80 0x7fdb200a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000e720 tx=0x7fdb30018000 comp rx=0 tx=0).stop 2024-09-10T08:48:30.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.098+0000 7fdb3ea31640 1 -- 172.21.15.172:0/1299183416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb38102570 msgr2=0x7fdb38198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:30.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.098+0000 7fdb3ea31640 1 --2- 172.21.15.172:0/1299183416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb38102570 0x7fdb38198940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c005940 tx=0x7fdb2c004040 comp rx=0 tx=0).stop 2024-09-10T08:48:30.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.098+0000 7fdb3ea31640 1 -- 172.21.15.172:0/1299183416 shutdown_connections 2024-09-10T08:48:30.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.098+0000 7fdb3ea31640 1 --2- 172.21.15.172:0/1299183416 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb200a2b80 0x7fdb200a5040 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.098+0000 7fdb3ea31640 1 --2- 172.21.15.172:0/1299183416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdb38102ef0 0x7fdb3819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.098+0000 7fdb3ea31640 1 --2- 172.21.15.172:0/1299183416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb38102570 0x7fdb38198940 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.098+0000 7fdb3ea31640 1 --2- 172.21.15.172:0/1299183416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb38100e20 0x7fdb38198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.098+0000 7fdb3ea31640 1 -- 172.21.15.172:0/1299183416 >> 172.21.15.172:0/1299183416 conn(0x7fdb380fc7d0 msgr2=0x7fdb3806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:30.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.098+0000 7fdb3ea31640 1 -- 172.21.15.172:0/1299183416 shutdown_connections 2024-09-10T08:48:30.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:30.098+0000 7fdb3ea31640 1 -- 172.21.15.172:0/1299183416 wait complete. 2024-09-10T08:48:30.201 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.199+0000 7fdddb7fe640 1 -- 172.21.15.172:0/2882636816 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fddec022af0 con 0x563875d35370 2024-09-10T08:48:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.199+0000 7fdddb7fe640 10 client.25496 handle_command_reply: tid=0 2024-09-10T08:48:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.199+0000 7fde051c8740 1 client.25496 shutdown 2024-09-10T08:48:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.199+0000 7fde051c8740 1 -- 172.21.15.172:0/2882636816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d2f9a0 msgr2=0x563875d3e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.199+0000 7fde051c8740 1 --2- 172.21.15.172:0/2882636816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d2f9a0 0x563875d3e4a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fddf009b990 tx=0x7fddf00cfb20 comp rx=0 tx=0).stop 2024-09-10T08:48:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.199+0000 7fde051c8740 1 -- 172.21.15.172:0/2882636816 shutdown_connections 2024-09-10T08:48:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.199+0000 7fde051c8740 1 --2- 172.21.15.172:0/2882636816 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563875d35370 0x563875db24e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.199+0000 7fde051c8740 1 --2- 172.21.15.172:0/2882636816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563875d302e0 0x563875dae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.199+0000 7fde051c8740 1 --2- 172.21.15.172:0/2882636816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563875d2f9a0 0x563875d3e4a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.199+0000 7fdddb7fe640 0 client.25496 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.199+0000 7fde051c8740 1 --2- 172.21.15.172:0/2882636816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563875d2b010 0x563875d3df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.199+0000 7fde051c8740 1 -- 172.21.15.172:0/2882636816 >> 172.21.15.172:0/2882636816 conn(0x563875d34f00 msgr2=0x563875d299a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:30.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.199+0000 7fde051c8740 1 -- 172.21.15.172:0/2882636816 shutdown_connections 2024-09-10T08:48:30.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.199+0000 7fde051c8740 1 -- 172.21.15.172:0/2882636816 wait complete. 2024-09-10T08:48:30.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.199+0000 7fde051c8740 20 client.25496 trim_cache size 0 max 16384 2024-09-10T08:48:30.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:48:30.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:48:30.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:48:30.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:48:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.200+0000 7fde02430640 1 -- 172.21.15.172:0/1869331839 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdde40a2c30 msgr2=0x7fdde40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.200+0000 7fde02430640 1 --2- 172.21.15.172:0/1869331839 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdde40a2c30 0x7fdde40a50f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fddf4005e30 tx=0x7fddf4005d80 comp rx=0 tx=0).stop 2024-09-10T08:48:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.200+0000 7fde02430640 1 -- 172.21.15.172:0/1869331839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddfc104f50 msgr2=0x7fddfc198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.200+0000 7fde02430640 1 --2- 172.21.15.172:0/1869331839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddfc104f50 0x7fddfc198280 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fddf000e9c0 tx=0x7fddf000ee80 comp rx=0 tx=0).stop 2024-09-10T08:48:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.200+0000 7fde02430640 1 -- 172.21.15.172:0/1869331839 shutdown_connections 2024-09-10T08:48:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.200+0000 7fde02430640 1 --2- 172.21.15.172:0/1869331839 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdde40a2c30 0x7fdde40a50f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.200+0000 7fde02430640 1 --2- 172.21.15.172:0/1869331839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddfc107880 0x7fddfc19f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.200+0000 7fde02430640 1 --2- 172.21.15.172:0/1869331839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddfc104f50 0x7fddfc198280 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.200+0000 7fde02430640 1 --2- 172.21.15.172:0/1869331839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fddfc102620 0x7fddfc197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.200+0000 7fde02430640 1 -- 172.21.15.172:0/1869331839 >> 172.21.15.172:0/1869331839 conn(0x7fddfc0fc1e0 msgr2=0x7fddfc102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.200+0000 7fde02430640 1 -- 172.21.15.172:0/1869331839 shutdown_connections 2024-09-10T08:48:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:30.200+0000 7fde02430640 1 -- 172.21.15.172:0/1869331839 wait complete. 2024-09-10T08:48:30.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:30 smithi191 ceph-mon[35314]: pgmap v418: 129 pgs: 129 active+clean; 675 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-10T08:48:30.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:30 smithi172 ceph-mon[30854]: pgmap v418: 129 pgs: 129 active+clean; 675 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-10T08:48:30.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:30 smithi174 ceph-mon[34068]: pgmap v418: 129 pgs: 129 active+clean; 675 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-10T08:48:31.005 INFO:tasks.workunit.client.0.smithi172.stderr:+ trap - EXIT 2024-09-10T08:48:31.005 INFO:tasks.workunit.client.0.smithi172.stderr:+ cd ffsb 2024-09-10T08:48:31.006 INFO:tasks.workunit.client.0.smithi172.stderr:+ ./configure 2024-09-10T08:48:31.050 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:48:31.148 INFO:tasks.workunit.client.0.smithi172.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-10T08:48:31.216 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '428b29b7', total seconds elapsed: 35.6 2024-09-10T08:48:31.345 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.849+0000 7f71fa2c0640 1 -- 172.21.15.172:0/519143989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f410c170 msgr2=0x7f71f410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.849+0000 7f71fa2c0640 1 --2- 172.21.15.172:0/519143989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f410c170 0x7f71f410e610 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009ca0 tx=0x7f71ec02a950 comp rx=0 tx=0).stop 2024-09-10T08:48:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.849+0000 7f71fa2c0640 1 -- 172.21.15.172:0/519143989 shutdown_connections 2024-09-10T08:48:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.849+0000 7f71fa2c0640 1 --2- 172.21.15.172:0/519143989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f410c170 0x7f71f410e610 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.849+0000 7f71fa2c0640 1 --2- 172.21.15.172:0/519143989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71f4103660 0x7f71f410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.849+0000 7f71fa2c0640 1 --2- 172.21.15.172:0/519143989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71f4102d40 0x7f71f4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.849+0000 7f71fa2c0640 1 -- 172.21.15.172:0/519143989 >> 172.21.15.172:0/519143989 conn(0x7f71f40fc7a0 msgr2=0x7f71f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.850+0000 7f71fa2c0640 1 -- 172.21.15.172:0/519143989 shutdown_connections 2024-09-10T08:48:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.850+0000 7f71fa2c0640 1 -- 172.21.15.172:0/519143989 wait complete. 2024-09-10T08:48:31.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.850+0000 7f71fa2c0640 1 Processor -- start 2024-09-10T08:48:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.850+0000 7f71fa2c0640 1 -- start start 2024-09-10T08:48:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.850+0000 7f71fa2c0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71f4102d40 0x7f71f41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.850+0000 7f71fa2c0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f4103660 0x7f71f4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71fa2c0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71f410c170 0x7f71f4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71fa2c0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f71f4111de0 con 0x7f71f4103660 2024-09-10T08:48:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71fa2c0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f71f4111c60 con 0x7f71f4102d40 2024-09-10T08:48:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71fa2c0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f71f4111f60 con 0x7f71f410c170 2024-09-10T08:48:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71f37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f4103660 0x7f71f4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71f37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f4103660 0x7f71f4100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34234/0 (socket says 172.21.15.172:34234) 2024-09-10T08:48:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71f37fe640 1 -- 172.21.15.172:0/1160345229 learned_addr learned my addr 172.21.15.172:0/1160345229 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71f3fff640 1 --2- 172.21.15.172:0/1160345229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71f4102d40 0x7f71f41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71f885d640 1 --2- 172.21.15.172:0/1160345229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71f410c170 0x7f71f4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71f37fe640 1 -- 172.21.15.172:0/1160345229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71f410c170 msgr2=0x7f71f4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:31.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71f37fe640 1 --2- 172.21.15.172:0/1160345229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71f410c170 0x7f71f4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71f37fe640 1 -- 172.21.15.172:0/1160345229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71f4102d40 msgr2=0x7f71f41009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71f37fe640 1 --2- 172.21.15.172:0/1160345229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71f4102d40 0x7f71f41009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71f37fe640 1 -- 172.21.15.172:0/1160345229 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f41a6740 con 0x7f71f4103660 2024-09-10T08:48:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71f37fe640 1 --2- 172.21.15.172:0/1160345229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f4103660 0x7f71f4100ee0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f71e800e9f0 tx=0x7f71e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71f17fa640 1 -- 172.21.15.172:0/1160345229 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e800cab0 con 0x7f71f4103660 2024-09-10T08:48:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71f17fa640 1 -- 172.21.15.172:0/1160345229 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e800cc50 con 0x7f71f4103660 2024-09-10T08:48:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71fa2c0640 1 -- 172.21.15.172:0/1160345229 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f41a6a30 con 0x7f71f4103660 2024-09-10T08:48:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71f17fa640 1 -- 172.21.15.172:0/1160345229 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8018750 con 0x7f71f4103660 2024-09-10T08:48:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.851+0000 7f71fa2c0640 1 -- 172.21.15.172:0/1160345229 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f41a6fc0 con 0x7f71f4103660 2024-09-10T08:48:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.854+0000 7f71f17fa640 1 -- 172.21.15.172:0/1160345229 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f71e8018b00 con 0x7f71f4103660 2024-09-10T08:48:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.854+0000 7f71fc699740 1 Processor -- start 2024-09-10T08:48:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.854+0000 7f71fc699740 1 -- start start 2024-09-10T08:48:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.854+0000 7f71fc699740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a30b40 0x557b41a2f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.854+0000 7f71fc699740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a2fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.854+0000 7f71fc699740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 0x557b41a3c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.854+0000 7f71fc699740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557b419952f0 con 0x557b41a2f5d0 2024-09-10T08:48:31.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.854+0000 7f71fc699740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b4193ccf0 con 0x557b41a3a370 2024-09-10T08:48:31.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.854+0000 7f71fc699740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557b419912b0 con 0x557b41a30b40 2024-09-10T08:48:31.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.854+0000 7f71f17fa640 1 --2- 172.21.15.172:0/1160345229 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f71dc0a2c00 0x7f71dc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.854+0000 7f71f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a2fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.854+0000 7f71f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a2fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34240/0 (socket says 172.21.15.172:34240) 2024-09-10T08:48:31.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.854+0000 7f71f3fff640 1 -- 172.21.15.172:0/2609066448 learned_addr learned my addr 172.21.15.172:0/2609066448 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:31.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.855+0000 7f71f885d640 1 --2- 172.21.15.172:0/2609066448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a30b40 0x557b41a2f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.855+0000 7f71f37fe640 1 --2- 172.21.15.172:0/2609066448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 0x557b41a3c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.855+0000 7f71f885d640 1 --2- 172.21.15.172:0/1160345229 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f71dc0a2c00 0x7f71dc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.855+0000 7f71f17fa640 1 -- 172.21.15.172:0/1160345229 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f71e80cf710 con 0x7f71f4103660 2024-09-10T08:48:31.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.855+0000 7f71f3fff640 1 -- 172.21.15.172:0/2609066448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a30b40 msgr2=0x557b41a2f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:31.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.855+0000 7f71f3fff640 1 --2- 172.21.15.172:0/2609066448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a30b40 0x557b41a2f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.855+0000 7f71f3fff640 1 -- 172.21.15.172:0/2609066448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 msgr2=0x557b41a3c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:31.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.855+0000 7f71f3fff640 1 --2- 172.21.15.172:0/2609066448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 0x557b41a3c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.855+0000 7f71f3fff640 1 -- 172.21.15.172:0/2609066448 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b419213a0 con 0x557b41a2f5d0 2024-09-10T08:48:31.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.855+0000 7f71f3fff640 1 --2- 172.21.15.172:0/2609066448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a2fa10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f71e400ad00 tx=0x7f71e400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a248cdfaaa756d90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:31.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.855+0000 7f71da7fc640 1 -- 172.21.15.172:0/2609066448 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e401d070 con 0x557b41a2f5d0 2024-09-10T08:48:31.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.855+0000 7f71f885d640 1 --2- 172.21.15.172:0/1160345229 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f71dc0a2c00 0x7f71dc0a50c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f71ec005e30 tx=0x7f71ec005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:31.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.855+0000 7f71da7fc640 1 -- 172.21.15.172:0/2609066448 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e4018af0 con 0x557b41a2f5d0 2024-09-10T08:48:31.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.855+0000 7f71da7fc640 1 -- 172.21.15.172:0/2609066448 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4010b40 con 0x557b41a2f5d0 2024-09-10T08:48:31.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.856+0000 7f71fc699740 1 -- 172.21.15.172:0/2609066448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 msgr2=0x557b41a2fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:31.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.856+0000 7f71fc699740 1 --2- 172.21.15.172:0/2609066448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a2fa10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f71e400ad00 tx=0x7f71e400f790 comp rx=0 tx=0).stop 2024-09-10T08:48:31.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.856+0000 7f71fc699740 1 -- 172.21.15.172:0/2609066448 shutdown_connections 2024-09-10T08:48:31.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.856+0000 7f71fc699740 1 --2- 172.21.15.172:0/2609066448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 0x557b41a3c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.856+0000 7f71fc699740 1 --2- 172.21.15.172:0/2609066448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a2fa10 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.856+0000 7f71fc699740 1 --2- 172.21.15.172:0/2609066448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a30b40 0x557b41a2f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.856+0000 7f71fc699740 1 -- 172.21.15.172:0/2609066448 >> 172.21.15.172:0/2609066448 conn(0x557b419f8310 msgr2=0x557b41a2bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:31.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.856+0000 7f71fc699740 1 -- 172.21.15.172:0/2609066448 shutdown_connections 2024-09-10T08:48:31.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.856+0000 7f71fc699740 1 -- 172.21.15.172:0/2609066448 wait complete. 2024-09-10T08:48:31.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.856+0000 7f71fc699740 1 -- start start 2024-09-10T08:48:31.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71fc699740 1 Processor -- start 2024-09-10T08:48:31.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71fc699740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a39d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71fc699740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a30b40 0x557b41a37000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71fc699740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 0x557b41a375d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71fc699740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557b41a3d070 con 0x557b41a2f5d0 2024-09-10T08:48:31.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71fc699740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b41950060 con 0x557b41a3a370 2024-09-10T08:48:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71fc699740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557b41a3d1f0 con 0x557b41a30b40 2024-09-10T08:48:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71fc699740 10 client.0 _ll_register_callbacks cb 0x557b416a38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a39d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a39d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34254/0 (socket says 172.21.15.172:34254) 2024-09-10T08:48:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71f3fff640 1 -- 172.21.15.172:0/1627003116 learned_addr learned my addr 172.21.15.172:0/1627003116 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71f37fe640 1 --2- 172.21.15.172:0/1627003116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a30b40 0x557b41a37000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71f885d640 1 --2- 172.21.15.172:0/1627003116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 0x557b41a375d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71f3fff640 1 -- 172.21.15.172:0/1627003116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a30b40 msgr2=0x557b41a37000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71f3fff640 1 --2- 172.21.15.172:0/1627003116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a30b40 0x557b41a37000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71f3fff640 1 -- 172.21.15.172:0/1627003116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 msgr2=0x557b41a375d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71f3fff640 1 --2- 172.21.15.172:0/1627003116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 0x557b41a375d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.857+0000 7f71f3fff640 1 -- 172.21.15.172:0/1627003116 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b417063c0 con 0x557b41a2f5d0 2024-09-10T08:48:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.858+0000 7f71f3fff640 1 --2- 172.21.15.172:0/1627003116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a39d50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f71e401b040 tx=0x7f71e4018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454e2254042835f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:31.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.858+0000 7f71d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.858+0000 7f71d37fe640 1 -- 172.21.15.172:0/1627003116 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e401d040 con 0x557b41a2f5d0 2024-09-10T08:48:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.858+0000 7f71d37fe640 1 -- 172.21.15.172:0/1627003116 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f71e4010880 con 0x557b41a2f5d0 2024-09-10T08:48:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.858+0000 7f71fc699740 1 -- 172.21.15.172:0/1627003116 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71e4018af0 con 0x557b41a2f5d0 2024-09-10T08:48:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.858+0000 7f71d37fe640 1 -- 172.21.15.172:0/1627003116 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f71e4031050 con 0x557b41a2f5d0 2024-09-10T08:48:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.858+0000 7f71fc699740 10 client.16164 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.858+0000 7f71fc699740 1 -- 172.21.15.172:0/1627003116 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557b41a48af0 con 0x557b41a2f5d0 2024-09-10T08:48:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.858+0000 7f71d37fe640 1 -- 172.21.15.172:0/1627003116 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e4018af0 con 0x557b41a2f5d0 2024-09-10T08:48:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.858+0000 7f71d37fe640 1 -- 172.21.15.172:0/1627003116 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f71e4020030 con 0x557b41a2f5d0 2024-09-10T08:48:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.858+0000 7f71fc699740 10 client.16164 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.858+0000 7f71fc699740 10 client.16164 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:48:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.858+0000 7f71fc699740 20 client.16164 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.858+0000 7f71fc699740 1 --2- 172.21.15.172:0/1627003116 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x557b41a49ef0 0x557b41a4c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.858+0000 7f71fc699740 4 client.16164 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:31.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.858+0000 7f71fc699740 1 -- 172.21.15.172:0/1627003116 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b419a2860 con 0x557b41a49ef0 2024-09-10T08:48:31.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.858+0000 7f71f37fe640 1 --2- 172.21.15.172:0/1627003116 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x557b41a49ef0 0x557b41a4c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.859+0000 7f71d37fe640 10 client.16164 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:48:31.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.859+0000 7f71f37fe640 1 --2- 172.21.15.172:0/1627003116 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x557b41a49ef0 0x557b41a4c2e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b1d426d37734373c server_cookie=4c4174d589b65b08 in_seq=0 out_seq=0 2024-09-10T08:48:31.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.859+0000 7f71d37fe640 1 -- 172.21.15.172:0/1627003116 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b419a2860 con 0x557b41a49ef0 2024-09-10T08:48:31.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.859+0000 7f71d37fe640 10 client.16164 handle_command_reply: tid=0 2024-09-10T08:48:31.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.859+0000 7f71fc699740 1 client.16164 shutdown 2024-09-10T08:48:31.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.860+0000 7f71fc699740 1 -- 172.21.15.172:0/1627003116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 msgr2=0x557b41a39d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:31.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.860+0000 7f71fc699740 1 --2- 172.21.15.172:0/1627003116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a39d50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f71e401b040 tx=0x7f71e4018670 comp rx=0 tx=0).stop 2024-09-10T08:48:31.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.860+0000 7f71fc699740 1 -- 172.21.15.172:0/1627003116 shutdown_connections 2024-09-10T08:48:31.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.860+0000 7f71fc699740 1 --2- 172.21.15.172:0/1627003116 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x557b41a49ef0 0x557b41a4c2e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.860+0000 7f71fc699740 1 --2- 172.21.15.172:0/1627003116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 0x557b41a375d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.860+0000 7f71fc699740 1 --2- 172.21.15.172:0/1627003116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a30b40 0x557b41a37000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.860+0000 7f71fc699740 1 --2- 172.21.15.172:0/1627003116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a39d50 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.860+0000 7f71d37fe640 0 client.16164 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:48:31.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.860+0000 7f71fc699740 1 -- 172.21.15.172:0/1627003116 >> 172.21.15.172:0/1627003116 conn(0x557b419f8310 msgr2=0x557b41a31990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:31.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.860+0000 7f71fc699740 1 -- 172.21.15.172:0/1627003116 shutdown_connections 2024-09-10T08:48:31.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.860+0000 7f71fc699740 1 -- 172.21.15.172:0/1627003116 wait complete. 2024-09-10T08:48:31.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.860+0000 7f71fc699740 20 client.16164 trim_cache size 0 max 16384 2024-09-10T08:48:31.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71fc699740 1 Processor -- start 2024-09-10T08:48:31.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71fc699740 1 -- start start 2024-09-10T08:48:31.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71fc699740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a2bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71fc699740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a30b40 0x557b41a46310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71fc699740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 0x557b41a51a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71fc699740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41b0c60 con 0x557b41a2f5d0 2024-09-10T08:48:31.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71fc699740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41b27a0 con 0x557b41a3a370 2024-09-10T08:48:31.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71fc699740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41b1330 con 0x557b41a30b40 2024-09-10T08:48:31.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a2bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a2bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34264/0 (socket says 172.21.15.172:34264) 2024-09-10T08:48:31.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71f3fff640 1 -- 172.21.15.172:0/260811469 learned_addr learned my addr 172.21.15.172:0/260811469 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:31.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71f37fe640 1 --2- 172.21.15.172:0/260811469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a30b40 0x557b41a46310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71f885d640 1 --2- 172.21.15.172:0/260811469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 0x557b41a51a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71f3fff640 1 -- 172.21.15.172:0/260811469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a30b40 msgr2=0x557b41a46310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:31.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71f3fff640 1 --2- 172.21.15.172:0/260811469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a30b40 0x557b41a46310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71f3fff640 1 -- 172.21.15.172:0/260811469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 msgr2=0x557b41a51a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:31.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71f3fff640 1 --2- 172.21.15.172:0/260811469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 0x557b41a51a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71f3fff640 1 -- 172.21.15.172:0/260811469 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b41a37900 con 0x557b41a2f5d0 2024-09-10T08:48:31.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71f3fff640 1 --2- 172.21.15.172:0/260811469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a2bd70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f71e400ac50 tx=0x7f71e400a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b378dccc39ef3de6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:31.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71d9ffb640 1 -- 172.21.15.172:0/260811469 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e401d070 con 0x557b41a2f5d0 2024-09-10T08:48:31.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71d9ffb640 1 -- 172.21.15.172:0/260811469 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e4028070 con 0x557b41a2f5d0 2024-09-10T08:48:31.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.874+0000 7f71d9ffb640 1 -- 172.21.15.172:0/260811469 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e40106b0 con 0x557b41a2f5d0 2024-09-10T08:48:31.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.875+0000 7f71fc699740 1 -- 172.21.15.172:0/260811469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 msgr2=0x557b41a2bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:31.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.875+0000 7f71fc699740 1 --2- 172.21.15.172:0/260811469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a2bd70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f71e400ac50 tx=0x7f71e400a2d0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.875+0000 7f71fc699740 1 -- 172.21.15.172:0/260811469 shutdown_connections 2024-09-10T08:48:31.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.875+0000 7f71fc699740 1 --2- 172.21.15.172:0/260811469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 0x557b41a51a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.875+0000 7f71fc699740 1 --2- 172.21.15.172:0/260811469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a30b40 0x557b41a46310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.875+0000 7f71fc699740 1 --2- 172.21.15.172:0/260811469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a2f5d0 0x557b41a2bd70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.875+0000 7f71fc699740 1 -- 172.21.15.172:0/260811469 >> 172.21.15.172:0/260811469 conn(0x557b419f8310 msgr2=0x557b41a360f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:31.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.875+0000 7f71fc699740 1 -- 172.21.15.172:0/260811469 shutdown_connections 2024-09-10T08:48:31.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.875+0000 7f71fc699740 1 -- 172.21.15.172:0/260811469 wait complete. 2024-09-10T08:48:31.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.876+0000 7f71fc699740 1 -- start start 2024-09-10T08:48:31.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.876+0000 7f71fc699740 1 Processor -- start 2024-09-10T08:48:31.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.876+0000 7f71fc699740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a2f5d0 0x557b41a34080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.876+0000 7f71fc699740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a30b40 0x557b41a345c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.876+0000 7f71fc699740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 0x557b41a34b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.876+0000 7f71fc699740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41b20d0 con 0x557b41a30b40 2024-09-10T08:48:31.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.876+0000 7f71fc699740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41b1a00 con 0x557b41a3a370 2024-09-10T08:48:31.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.876+0000 7f71fc699740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41aec40 con 0x557b41a2f5d0 2024-09-10T08:48:31.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.876+0000 7f71fc699740 10 client.0 _ll_register_callbacks cb 0x7f71e401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:31.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.876+0000 7f71f37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a30b40 0x557b41a345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.876+0000 7f71f37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a30b40 0x557b41a345c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34278/0 (socket says 172.21.15.172:34278) 2024-09-10T08:48:31.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.876+0000 7f71f37fe640 1 -- 172.21.15.172:0/521915273 learned_addr learned my addr 172.21.15.172:0/521915273 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:31.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.876+0000 7f71f3fff640 1 --2- 172.21.15.172:0/521915273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a2f5d0 0x557b41a34080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.876+0000 7f71f885d640 1 --2- 172.21.15.172:0/521915273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 0x557b41a34b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71f37fe640 1 -- 172.21.15.172:0/521915273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a2f5d0 msgr2=0x557b41a34080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:31.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71f37fe640 1 --2- 172.21.15.172:0/521915273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a2f5d0 0x557b41a34080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71f37fe640 1 -- 172.21.15.172:0/521915273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 msgr2=0x557b41a34b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:31.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71f37fe640 1 --2- 172.21.15.172:0/521915273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 0x557b41a34b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71f37fe640 1 -- 172.21.15.172:0/521915273 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b41ab3a70 con 0x557b41a30b40 2024-09-10T08:48:31.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71f3fff640 1 --2- 172.21.15.172:0/521915273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a2f5d0 0x557b41a34080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:31.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71f37fe640 1 --2- 172.21.15.172:0/521915273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a30b40 0x557b41a345c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f71e8098070 tx=0x7f71e8014cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a029f691c55cfcd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:31.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:31.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71d37fe640 1 -- 172.21.15.172:0/521915273 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e809c310 con 0x557b41a30b40 2024-09-10T08:48:31.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71d37fe640 1 -- 172.21.15.172:0/521915273 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f71e8099f10 con 0x557b41a30b40 2024-09-10T08:48:31.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71fc699740 1 -- 172.21.15.172:0/521915273 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71e4028070 con 0x557b41a30b40 2024-09-10T08:48:31.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71d37fe640 1 -- 172.21.15.172:0/521915273 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f71e809bc10 con 0x557b41a30b40 2024-09-10T08:48:31.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71fc699740 10 client.16170 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:31.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71fc699740 1 -- 172.21.15.172:0/521915273 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557b41ab4d70 con 0x557b41a30b40 2024-09-10T08:48:31.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71d37fe640 1 -- 172.21.15.172:0/521915273 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e4028070 con 0x557b41a30b40 2024-09-10T08:48:31.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71d37fe640 1 -- 172.21.15.172:0/521915273 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f71e8098280 con 0x557b41a30b40 2024-09-10T08:48:31.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71fc699740 10 client.16170 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:31.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71fc699740 10 client.16170 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:48:31.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71fc699740 20 client.16170 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:31.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71fc699740 1 --2- 172.21.15.172:0/521915273 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x557b41ab5360 0x557b41ab7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:31.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71fc699740 4 client.16170 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:48:31.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.877+0000 7f71fc699740 1 -- 172.21.15.172:0/521915273 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b4199cd70 con 0x557b41ab5360 2024-09-10T08:48:31.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.878+0000 7f71f3fff640 1 --2- 172.21.15.172:0/521915273 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x557b41ab5360 0x557b41ab7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:31.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.878+0000 7f71f3fff640 1 --2- 172.21.15.172:0/521915273 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x557b41ab5360 0x557b41ab7780 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44a51c9c555f661a server_cookie=2da7d8da967d30ca in_seq=0 out_seq=0 2024-09-10T08:48:31.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.878+0000 7f71d37fe640 10 client.16170 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:48:31.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.878+0000 7f71d37fe640 1 -- 172.21.15.172:0/521915273 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b4199cd70 con 0x557b41ab5360 2024-09-10T08:48:31.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.878+0000 7f71d37fe640 10 client.16170 handle_command_reply: tid=0 2024-09-10T08:48:31.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.878+0000 7f71fc699740 1 client.16170 shutdown 2024-09-10T08:48:31.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.878+0000 7f71fc699740 1 -- 172.21.15.172:0/521915273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a30b40 msgr2=0x557b41a345c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:31.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.878+0000 7f71fc699740 1 --2- 172.21.15.172:0/521915273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a30b40 0x557b41a345c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f71e8098070 tx=0x7f71e8014cb0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.878+0000 7f71fc699740 1 -- 172.21.15.172:0/521915273 shutdown_connections 2024-09-10T08:48:31.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.878+0000 7f71fc699740 1 --2- 172.21.15.172:0/521915273 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x557b41ab5360 0x557b41ab7780 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.878+0000 7f71fc699740 1 --2- 172.21.15.172:0/521915273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b41a3a370 0x557b41a34b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.878+0000 7f71fc699740 1 --2- 172.21.15.172:0/521915273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b41a30b40 0x557b41a345c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.878+0000 7f71fc699740 1 --2- 172.21.15.172:0/521915273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b41a2f5d0 0x557b41a34080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.878+0000 7f71fc699740 1 -- 172.21.15.172:0/521915273 >> 172.21.15.172:0/521915273 conn(0x557b419f8310 msgr2=0x557b41a2a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:31.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.878+0000 7f71d37fe640 0 client.16170 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:48:31.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.878+0000 7f71fc699740 1 -- 172.21.15.172:0/521915273 shutdown_connections 2024-09-10T08:48:31.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.879+0000 7f71fc699740 1 -- 172.21.15.172:0/521915273 wait complete. 2024-09-10T08:48:31.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.879+0000 7f71fc699740 20 client.16170 trim_cache size 0 max 16384 2024-09-10T08:48:31.888 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:48:31.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.879+0000 7f71fa2c0640 1 -- 172.21.15.172:0/1160345229 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f71dc0a2c00 msgr2=0x7f71dc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:31.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.879+0000 7f71fa2c0640 1 --2- 172.21.15.172:0/1160345229 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f71dc0a2c00 0x7f71dc0a50c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f71ec005e30 tx=0x7f71ec005c50 comp rx=0 tx=0).stop 2024-09-10T08:48:31.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.879+0000 7f71fa2c0640 1 -- 172.21.15.172:0/1160345229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f4103660 msgr2=0x7f71f4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:31.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.879+0000 7f71fa2c0640 1 --2- 172.21.15.172:0/1160345229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f4103660 0x7f71f4100ee0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f71e800e9f0 tx=0x7f71e800edf0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.879+0000 7f71fa2c0640 1 -- 172.21.15.172:0/1160345229 shutdown_connections 2024-09-10T08:48:31.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.879+0000 7f71fa2c0640 1 --2- 172.21.15.172:0/1160345229 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f71dc0a2c00 0x7f71dc0a50c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.879+0000 7f71fa2c0640 1 --2- 172.21.15.172:0/1160345229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71f410c170 0x7f71f4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.879+0000 7f71fa2c0640 1 --2- 172.21.15.172:0/1160345229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f4103660 0x7f71f4100ee0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.879+0000 7f71fa2c0640 1 --2- 172.21.15.172:0/1160345229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71f4102d40 0x7f71f41009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:31.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.879+0000 7f71fa2c0640 1 -- 172.21.15.172:0/1160345229 >> 172.21.15.172:0/1160345229 conn(0x7f71f40fc7a0 msgr2=0x7f71f40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:31.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.879+0000 7f71fa2c0640 1 -- 172.21.15.172:0/1160345229 shutdown_connections 2024-09-10T08:48:31.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:31.879+0000 7f71fa2c0640 1 -- 172.21.15.172:0/1160345229 wait complete. 2024-09-10T08:48:32.157 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether build environment is sane... yes 2024-09-10T08:48:32.171 INFO:tasks.workunit.client.0.smithi172.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-10T08:48:32.171 INFO:tasks.workunit.client.0.smithi172.stdout:checking for gawk... gawk 2024-09-10T08:48:32.192 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether make sets $(MAKE)... yes 2024-09-10T08:48:32.234 INFO:tasks.workunit.client.0.smithi172.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-10T08:48:32.234 INFO:tasks.workunit.client.0.smithi172.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-10T08:48:32.251 INFO:tasks.workunit.client.0.smithi172.stdout:checking for style of include used by make... GNU 2024-09-10T08:48:32.256 INFO:tasks.workunit.client.0.smithi172.stdout:checking for gcc... gcc 2024-09-10T08:48:32.337 INFO:tasks.workunit.client.0.smithi172.stdout:checking for C compiler default output file name... a.out 2024-09-10T08:48:32.339 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether the C compiler works... yes 2024-09-10T08:48:32.343 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether we are cross compiling... no 2024-09-10T08:48:32.392 INFO:tasks.workunit.client.0.smithi172.stdout:checking for suffix of executables... 2024-09-10T08:48:32.434 INFO:tasks.workunit.client.0.smithi172.stdout:checking for suffix of object files... o 2024-09-10T08:48:32.475 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether we are using the GNU C compiler... yes 2024-09-10T08:48:32.517 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether gcc accepts -g... yes 2024-09-10T08:48:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:32 smithi191 ceph-mon[35314]: pgmap v419: 129 pgs: 129 active+clean; 1.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 85 KiB/s wr, 29 op/s 2024-09-10T08:48:32.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:32 smithi172 ceph-mon[30854]: pgmap v419: 129 pgs: 129 active+clean; 1.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 85 KiB/s wr, 29 op/s 2024-09-10T08:48:32.705 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:48:32.705 INFO:tasks.workunit.client.0.smithi172.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-10T08:48:32.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:32 smithi174 ceph-mon[34068]: pgmap v419: 129 pgs: 129 active+clean; 1.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 85 KiB/s wr, 29 op/s 2024-09-10T08:48:32.858 INFO:tasks.workunit.client.0.smithi172.stdout:checking dependency style of gcc... gcc3 2024-09-10T08:48:32.952 INFO:tasks.workunit.client.0.smithi172.stdout:checking how to run the C preprocessor... gcc -E 2024-09-10T08:48:32.979 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:33.018 INFO:tasks.workunit.client.0.smithi172.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-10T08:48:33.019 INFO:tasks.workunit.client.0.smithi172.stdout:checking for egrep... /usr/bin/grep -E 2024-09-10T08:48:33.032 INFO:tasks.workunit.client.0.smithi172.stdout:checking for AIX... no 2024-09-10T08:48:33.056 INFO:tasks.workunit.client.0.smithi172.stdout:checking for gcc... (cached) gcc 2024-09-10T08:48:33.068 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-10T08:48:33.069 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-10T08:48:33.070 INFO:tasks.workunit.client.0.smithi172.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-10T08:48:33.070 INFO:tasks.workunit.client.0.smithi172.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-10T08:48:33.190 INFO:tasks.workunit.client.0.smithi172.stdout:checking for main in -lm... yes 2024-09-10T08:48:33.331 INFO:tasks.workunit.client.0.smithi172.stdout:checking for main in -lpthread... yes 2024-09-10T08:48:33.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.519+0000 7ff315deb640 1 -- 172.21.15.172:0/3081061885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff310102570 msgr2=0x7ff3101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:33.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.519+0000 7ff315deb640 1 --2- 172.21.15.172:0/3081061885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff310102570 0x7ff3101029b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff304009900 tx=0x7ff30402a480 comp rx=0 tx=0).stop 2024-09-10T08:48:33.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.519+0000 7ff315deb640 1 -- 172.21.15.172:0/3081061885 shutdown_connections 2024-09-10T08:48:33.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.519+0000 7ff315deb640 1 --2- 172.21.15.172:0/3081061885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff310102ef0 0x7ff31010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.519+0000 7ff315deb640 1 --2- 172.21.15.172:0/3081061885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff310102570 0x7ff3101029b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.519+0000 7ff315deb640 1 --2- 172.21.15.172:0/3081061885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff310100e20 0x7ff310101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.519+0000 7ff315deb640 1 -- 172.21.15.172:0/3081061885 >> 172.21.15.172:0/3081061885 conn(0x7ff3100fc7d0 msgr2=0x7ff3100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:33.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.519+0000 7ff315deb640 1 -- 172.21.15.172:0/3081061885 shutdown_connections 2024-09-10T08:48:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.519+0000 7ff315deb640 1 -- 172.21.15.172:0/3081061885 wait complete. 2024-09-10T08:48:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.520+0000 7ff315deb640 1 Processor -- start 2024-09-10T08:48:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.520+0000 7ff315deb640 1 -- start start 2024-09-10T08:48:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.520+0000 7ff315deb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff310100e20 0x7ff310199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.520+0000 7ff315deb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff310102570 0x7ff3101997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.520+0000 7ff315deb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff310102ef0 0x7ff3101a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.520+0000 7ff315deb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff310110e30 con 0x7ff310100e20 2024-09-10T08:48:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.520+0000 7ff315deb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff310110cb0 con 0x7ff310102570 2024-09-10T08:48:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.520+0000 7ff315deb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff310110fb0 con 0x7ff310102ef0 2024-09-10T08:48:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.520+0000 7ff30f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff310100e20 0x7ff310199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.520+0000 7ff30f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff310100e20 0x7ff310199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34300/0 (socket says 172.21.15.172:34300) 2024-09-10T08:48:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.520+0000 7ff30f7fe640 1 -- 172.21.15.172:0/2161663201 learned_addr learned my addr 172.21.15.172:0/2161663201 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.521+0000 7ff30effd640 1 --2- 172.21.15.172:0/2161663201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff310102570 0x7ff3101997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.521+0000 7ff30ffff640 1 --2- 172.21.15.172:0/2161663201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff310102ef0 0x7ff3101a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.521+0000 7ff30f7fe640 1 -- 172.21.15.172:0/2161663201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff310102ef0 msgr2=0x7ff3101a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.521+0000 7ff30f7fe640 1 --2- 172.21.15.172:0/2161663201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff310102ef0 0x7ff3101a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.521+0000 7ff30f7fe640 1 -- 172.21.15.172:0/2161663201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff310102570 msgr2=0x7ff3101997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.521+0000 7ff30f7fe640 1 --2- 172.21.15.172:0/2161663201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff310102570 0x7ff3101997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.521+0000 7ff30f7fe640 1 -- 172.21.15.172:0/2161663201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3101a4820 con 0x7ff310100e20 2024-09-10T08:48:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.521+0000 7ff30ffff640 1 --2- 172.21.15.172:0/2161663201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff310102ef0 0x7ff3101a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.521+0000 7ff30f7fe640 1 --2- 172.21.15.172:0/2161663201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff310100e20 0x7ff310199260 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff30000d950 tx=0x7ff30000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.521+0000 7ff30cff9640 1 -- 172.21.15.172:0/2161663201 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300017030 con 0x7ff310100e20 2024-09-10T08:48:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.521+0000 7ff30cff9640 1 -- 172.21.15.172:0/2161663201 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff300004020 con 0x7ff310100e20 2024-09-10T08:48:33.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.521+0000 7ff315deb640 1 -- 172.21.15.172:0/2161663201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3101a49f0 con 0x7ff310100e20 2024-09-10T08:48:33.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.521+0000 7ff30cff9640 1 -- 172.21.15.172:0/2161663201 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3000047a0 con 0x7ff310100e20 2024-09-10T08:48:33.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.521+0000 7ff315deb640 1 -- 172.21.15.172:0/2161663201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3101a4f00 con 0x7ff310100e20 2024-09-10T08:48:33.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.524+0000 7ff30cff9640 1 -- 172.21.15.172:0/2161663201 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff3000041d0 con 0x7ff310100e20 2024-09-10T08:48:33.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.525+0000 7ff3181c4740 1 Processor -- start 2024-09-10T08:48:33.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.525+0000 7ff30cff9640 1 --2- 172.21.15.172:0/2161663201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2f80a2b80 0x7ff2f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.525+0000 7ff3181c4740 1 -- start start 2024-09-10T08:48:33.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.525+0000 7ff30ffff640 1 --2- 172.21.15.172:0/2161663201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2f80a2b80 0x7ff2f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.525+0000 7ff3181c4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a3657db40 0x562a3657c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.525+0000 7ff3181c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 0x562a3657ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.525+0000 7ff30cff9640 1 -- 172.21.15.172:0/2161663201 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff3000ceed0 con 0x7ff310100e20 2024-09-10T08:48:33.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.525+0000 7ff3181c4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a36587370 0x562a36589760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.525+0000 7ff3181c4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562a364e22f0 con 0x562a3657db40 2024-09-10T08:48:33.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.525+0000 7ff3181c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562a36489cf0 con 0x562a3657c5d0 2024-09-10T08:48:33.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.525+0000 7ff3181c4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562a364de2b0 con 0x562a36587370 2024-09-10T08:48:33.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.525+0000 7ff30f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a3657db40 0x562a3657c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.525+0000 7ff30f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a3657db40 0x562a3657c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34314/0 (socket says 172.21.15.172:34314) 2024-09-10T08:48:33.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.525+0000 7ff30f7fe640 1 -- 172.21.15.172:0/1932661899 learned_addr learned my addr 172.21.15.172:0/1932661899 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:33.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.525+0000 7ff30effd640 1 --2- 172.21.15.172:0/1932661899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 0x562a3657ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.526+0000 7ff30ffff640 1 --2- 172.21.15.172:0/1932661899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a36587370 0x562a36589760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.526+0000 7ff30f7fe640 1 -- 172.21.15.172:0/1932661899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a36587370 msgr2=0x562a36589760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:33.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.526+0000 7ff30f7fe640 1 --2- 172.21.15.172:0/1932661899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a36587370 0x562a36589760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.526+0000 7ff30f7fe640 1 -- 172.21.15.172:0/1932661899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 msgr2=0x562a3657ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:33.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.526+0000 7ff30f7fe640 1 --2- 172.21.15.172:0/1932661899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 0x562a3657ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.526+0000 7ff30f7fe640 1 -- 172.21.15.172:0/1932661899 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a3646e3a0 con 0x562a3657db40 2024-09-10T08:48:33.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.526+0000 7ff30ffff640 1 --2- 172.21.15.172:0/1932661899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a36587370 0x562a36589760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:33.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.526+0000 7ff30ffff640 1 --2- 172.21.15.172:0/2161663201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2f80a2b80 0x7ff2f80a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff308005fd0 tx=0x7ff30800f730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:33.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.526+0000 7ff30f7fe640 1 --2- 172.21.15.172:0/1932661899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a3657db40 0x562a3657c090 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff3000a3b10 tx=0x7ff30009adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55cf2693cea04166 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:33.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.526+0000 7ff2f5ffb640 1 -- 172.21.15.172:0/1932661899 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30009f780 con 0x562a3657db40 2024-09-10T08:48:33.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.526+0000 7ff2f5ffb640 1 -- 172.21.15.172:0/1932661899 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3000a4040 con 0x562a3657db40 2024-09-10T08:48:33.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.526+0000 7ff2f5ffb640 1 -- 172.21.15.172:0/1932661899 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30009a820 con 0x562a3657db40 2024-09-10T08:48:33.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.527+0000 7ff3181c4740 1 -- 172.21.15.172:0/1932661899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a3657db40 msgr2=0x562a3657c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:33.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.527+0000 7ff3181c4740 1 --2- 172.21.15.172:0/1932661899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a3657db40 0x562a3657c090 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff3000a3b10 tx=0x7ff30009adb0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.527+0000 7ff3181c4740 1 -- 172.21.15.172:0/1932661899 shutdown_connections 2024-09-10T08:48:33.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.527+0000 7ff3181c4740 1 --2- 172.21.15.172:0/1932661899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a36587370 0x562a36589760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.527+0000 7ff3181c4740 1 --2- 172.21.15.172:0/1932661899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 0x562a3657ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.527+0000 7ff3181c4740 1 --2- 172.21.15.172:0/1932661899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a3657db40 0x562a3657c090 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.527+0000 7ff3181c4740 1 -- 172.21.15.172:0/1932661899 >> 172.21.15.172:0/1932661899 conn(0x562a36545310 msgr2=0x562a36578c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:33.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.527+0000 7ff3181c4740 1 -- 172.21.15.172:0/1932661899 shutdown_connections 2024-09-10T08:48:33.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.527+0000 7ff3181c4740 1 -- 172.21.15.172:0/1932661899 wait complete. 2024-09-10T08:48:33.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.527+0000 7ff3181c4740 1 -- start start 2024-09-10T08:48:33.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff3181c4740 1 Processor -- start 2024-09-10T08:48:33.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff3181c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 0x562a36586d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff3181c4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a3657db40 0x562a36584000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff3181c4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a36587370 0x562a365845d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff3181c4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562a3658e470 con 0x562a3657db40 2024-09-10T08:48:33.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff3181c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562a3649d060 con 0x562a3657c5d0 2024-09-10T08:48:33.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff3181c4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562a3658e5f0 con 0x562a36587370 2024-09-10T08:48:33.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff3181c4740 10 client.0 _ll_register_callbacks cb 0x562a361f08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:33.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff30effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a3657db40 0x562a36584000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff30f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 0x562a36586d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff30effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a3657db40 0x562a36584000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34330/0 (socket says 172.21.15.172:34330) 2024-09-10T08:48:33.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff30effd640 1 -- 172.21.15.172:0/1538569683 learned_addr learned my addr 172.21.15.172:0/1538569683 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:33.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff30ffff640 1 --2- 172.21.15.172:0/1538569683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a36587370 0x562a365845d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff30f7fe640 1 -- 172.21.15.172:0/1538569683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a36587370 msgr2=0x562a365845d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:33.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff30f7fe640 1 --2- 172.21.15.172:0/1538569683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a36587370 0x562a365845d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff30f7fe640 1 -- 172.21.15.172:0/1538569683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a3657db40 msgr2=0x562a36584000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:33.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff30f7fe640 1 --2- 172.21.15.172:0/1538569683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a3657db40 0x562a36584000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.528+0000 7ff30f7fe640 1 -- 172.21.15.172:0/1538569683 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a362533c0 con 0x562a3657c5d0 2024-09-10T08:48:33.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.529+0000 7ff30f7fe640 1 --2- 172.21.15.172:0/1538569683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 0x562a36586d50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff3000a3b10 tx=0x7ff30009b550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52ffc9fee24f3c72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:33.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.529+0000 7ff2eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:48:33.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.529+0000 7ff2eeffd640 1 -- 172.21.15.172:0/1538569683 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300097a40 con 0x562a3657c5d0 2024-09-10T08:48:33.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.529+0000 7ff2eeffd640 1 -- 172.21.15.172:0/1538569683 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff3000a3c90 con 0x562a3657c5d0 2024-09-10T08:48:33.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.529+0000 7ff3181c4740 1 -- 172.21.15.172:0/1538569683 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3000a4040 con 0x562a3657c5d0 2024-09-10T08:48:33.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.530+0000 7ff2eeffd640 1 -- 172.21.15.172:0/1538569683 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff300097be0 con 0x562a3657c5d0 2024-09-10T08:48:33.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.530+0000 7ff3181c4740 10 client.25544 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:33.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.530+0000 7ff3181c4740 1 -- 172.21.15.172:0/1538569683 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562a36599e60 con 0x562a3657c5d0 2024-09-10T08:48:33.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.530+0000 7ff2eeffd640 1 -- 172.21.15.172:0/1538569683 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3000a4040 con 0x562a3657c5d0 2024-09-10T08:48:33.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.530+0000 7ff2eeffd640 1 -- 172.21.15.172:0/1538569683 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff300097120 con 0x562a3657c5d0 2024-09-10T08:48:33.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.530+0000 7ff3181c4740 10 client.25544 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:33.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.530+0000 7ff3181c4740 10 client.25544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:33.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.530+0000 7ff3181c4740 20 client.25544 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:33.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.530+0000 7ff3181c4740 1 --2- 172.21.15.172:0/1538569683 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562a3659b1b0 0x562a3659d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.530+0000 7ff3181c4740 4 client.25544 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:33.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.530+0000 7ff3181c4740 1 -- 172.21.15.172:0/1538569683 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a364ef860 con 0x562a3659b1b0 2024-09-10T08:48:33.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.531+0000 7ff30effd640 1 --2- 172.21.15.172:0/1538569683 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562a3659b1b0 0x562a3659d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.531+0000 7ff2eeffd640 10 client.25544 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:33.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.531+0000 7ff30effd640 1 --2- 172.21.15.172:0/1538569683 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562a3659b1b0 0x562a3659d5a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ec14568aa63e116 server_cookie=7c7a97c3371ab0f1 in_seq=0 out_seq=0 2024-09-10T08:48:33.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.532+0000 7ff2eeffd640 1 -- 172.21.15.172:0/1538569683 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a364ef860 con 0x562a3659b1b0 2024-09-10T08:48:33.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.532+0000 7ff2eeffd640 10 client.25544 handle_command_reply: tid=0 2024-09-10T08:48:33.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.532+0000 7ff3181c4740 1 client.25544 shutdown 2024-09-10T08:48:33.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.532+0000 7ff3181c4740 1 -- 172.21.15.172:0/1538569683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 msgr2=0x562a36586d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:33.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.532+0000 7ff3181c4740 1 --2- 172.21.15.172:0/1538569683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 0x562a36586d50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff3000a3b10 tx=0x7ff30009b550 comp rx=0 tx=0).stop 2024-09-10T08:48:33.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.532+0000 7ff3181c4740 1 -- 172.21.15.172:0/1538569683 shutdown_connections 2024-09-10T08:48:33.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.532+0000 7ff3181c4740 1 --2- 172.21.15.172:0/1538569683 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562a3659b1b0 0x562a3659d5a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.532+0000 7ff3181c4740 1 --2- 172.21.15.172:0/1538569683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a36587370 0x562a365845d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.532+0000 7ff3181c4740 1 --2- 172.21.15.172:0/1538569683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a3657db40 0x562a36584000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.532+0000 7ff3181c4740 1 --2- 172.21.15.172:0/1538569683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 0x562a36586d50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.532+0000 7ff2eeffd640 0 client.25544 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:33.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.532+0000 7ff3181c4740 1 -- 172.21.15.172:0/1538569683 >> 172.21.15.172:0/1538569683 conn(0x562a36545310 msgr2=0x562a3657e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:33.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.532+0000 7ff3181c4740 1 -- 172.21.15.172:0/1538569683 shutdown_connections 2024-09-10T08:48:33.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.532+0000 7ff3181c4740 1 -- 172.21.15.172:0/1538569683 wait complete. 2024-09-10T08:48:33.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.532+0000 7ff3181c4740 20 client.25544 trim_cache size 0 max 16384 2024-09-10T08:48:33.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.545+0000 7ff3181c4740 1 Processor -- start 2024-09-10T08:48:33.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.545+0000 7ff3181c4740 1 -- start start 2024-09-10T08:48:33.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.545+0000 7ff3181c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 0x562a365a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.545+0000 7ff3181c4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a3657db40 0x562a365a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.545+0000 7ff3181c4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a36587370 0x562a36581260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.545+0000 7ff3181c4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101aca10 con 0x562a36587370 2024-09-10T08:48:33.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.545+0000 7ff3181c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101ae550 con 0x562a3657c5d0 2024-09-10T08:48:33.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.545+0000 7ff3181c4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101ad0e0 con 0x562a3657db40 2024-09-10T08:48:33.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.546+0000 7ff30ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a36587370 0x562a36581260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.546+0000 7ff30f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 0x562a365a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.546+0000 7ff30ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a36587370 0x562a36581260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34344/0 (socket says 172.21.15.172:34344) 2024-09-10T08:48:33.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.546+0000 7ff30f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 0x562a365a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:42208/0 (socket says 172.21.15.172:42208) 2024-09-10T08:48:33.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.546+0000 7ff30ffff640 1 -- 172.21.15.172:0/1093269343 learned_addr learned my addr 172.21.15.172:0/1093269343 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:33.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.546+0000 7ff30effd640 1 --2- 172.21.15.172:0/1093269343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a3657db40 0x562a365a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.546+0000 7ff30f7fe640 1 -- 172.21.15.172:0/1093269343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a3657db40 msgr2=0x562a365a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:33.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.546+0000 7ff30f7fe640 1 --2- 172.21.15.172:0/1093269343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a3657db40 0x562a365a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.546+0000 7ff30f7fe640 1 -- 172.21.15.172:0/1093269343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a36587370 msgr2=0x562a36581260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:33.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.546+0000 7ff30f7fe640 1 --2- 172.21.15.172:0/1093269343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a36587370 0x562a36581260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.546+0000 7ff30f7fe640 1 -- 172.21.15.172:0/1093269343 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a3657f120 con 0x562a3657c5d0 2024-09-10T08:48:33.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.546+0000 7ff30effd640 1 --2- 172.21.15.172:0/1093269343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a3657db40 0x562a365a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:33.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.546+0000 7ff30f7fe640 1 --2- 172.21.15.172:0/1093269343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 0x562a365a05c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff30009b360 tx=0x7ff30009adb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=940d10ce7b62ea99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:33.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.546+0000 7ff2f57fa640 1 -- 172.21.15.172:0/1093269343 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300097a40 con 0x562a3657c5d0 2024-09-10T08:48:33.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.546+0000 7ff2f57fa640 1 -- 172.21.15.172:0/1093269343 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3000a4040 con 0x562a3657c5d0 2024-09-10T08:48:33.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.546+0000 7ff2f57fa640 1 -- 172.21.15.172:0/1093269343 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300009bf0 con 0x562a3657c5d0 2024-09-10T08:48:33.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.547+0000 7ff3181c4740 1 -- 172.21.15.172:0/1093269343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 msgr2=0x562a365a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.547+0000 7ff3181c4740 1 --2- 172.21.15.172:0/1093269343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 0x562a365a05c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff30009b360 tx=0x7ff30009adb0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.547+0000 7ff3181c4740 1 -- 172.21.15.172:0/1093269343 shutdown_connections 2024-09-10T08:48:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.547+0000 7ff3181c4740 1 --2- 172.21.15.172:0/1093269343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a36587370 0x562a36581260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.547+0000 7ff3181c4740 1 --2- 172.21.15.172:0/1093269343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a3657db40 0x562a365a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.547+0000 7ff3181c4740 1 --2- 172.21.15.172:0/1093269343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657c5d0 0x562a365a05c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.547+0000 7ff3181c4740 1 -- 172.21.15.172:0/1093269343 >> 172.21.15.172:0/1093269343 conn(0x562a36545310 msgr2=0x562a365830f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.547+0000 7ff3181c4740 1 -- 172.21.15.172:0/1093269343 shutdown_connections 2024-09-10T08:48:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.547+0000 7ff3181c4740 1 -- 172.21.15.172:0/1093269343 wait complete. 2024-09-10T08:48:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.548+0000 7ff3181c4740 1 -- start start 2024-09-10T08:48:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.548+0000 7ff3181c4740 1 Processor -- start 2024-09-10T08:48:33.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.548+0000 7ff3181c4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a3657c5d0 0x562a365a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.548+0000 7ff3181c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657db40 0x562a3659e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.548+0000 7ff3181c4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a36587370 0x562a3659e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.548+0000 7ff3181c4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101ade80 con 0x562a36587370 2024-09-10T08:48:33.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.548+0000 7ff3181c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101ad7b0 con 0x562a3657db40 2024-09-10T08:48:33.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.548+0000 7ff3181c4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101aa9f0 con 0x562a3657c5d0 2024-09-10T08:48:33.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.548+0000 7ff3181c4740 10 client.0 _ll_register_callbacks cb 0x7ff304009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:33.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.548+0000 7ff30ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a36587370 0x562a3659e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.548+0000 7ff30ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a36587370 0x562a3659e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34350/0 (socket says 172.21.15.172:34350) 2024-09-10T08:48:33.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.548+0000 7ff30ffff640 1 -- 172.21.15.172:0/3403746098 learned_addr learned my addr 172.21.15.172:0/3403746098 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:33.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.548+0000 7ff30f7fe640 1 --2- 172.21.15.172:0/3403746098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a3657c5d0 0x562a365a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.549+0000 7ff30effd640 1 --2- 172.21.15.172:0/3403746098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657db40 0x562a3659e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.549+0000 7ff30ffff640 1 -- 172.21.15.172:0/3403746098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a3657c5d0 msgr2=0x562a365a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:33.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.549+0000 7ff30ffff640 1 --2- 172.21.15.172:0/3403746098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a3657c5d0 0x562a365a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.549+0000 7ff30ffff640 1 -- 172.21.15.172:0/3403746098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657db40 msgr2=0x562a3659e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.549+0000 7ff30ffff640 1 --2- 172.21.15.172:0/3403746098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657db40 0x562a3659e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.549+0000 7ff30ffff640 1 -- 172.21.15.172:0/3403746098 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a3659ed30 con 0x562a36587370 2024-09-10T08:48:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.549+0000 7ff30f7fe640 1 --2- 172.21.15.172:0/3403746098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a3657c5d0 0x562a365a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.549+0000 7ff30ffff640 1 --2- 172.21.15.172:0/3403746098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a36587370 0x562a3659e550 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff308011950 tx=0x7ff308004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55590c45bd06bc61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.549+0000 7ff2eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.549+0000 7ff2eeffd640 1 -- 172.21.15.172:0/3403746098 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308021da0 con 0x562a36587370 2024-09-10T08:48:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.549+0000 7ff2eeffd640 1 -- 172.21.15.172:0/3403746098 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff3080293e0 con 0x562a36587370 2024-09-10T08:48:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.549+0000 7ff3181c4740 1 -- 172.21.15.172:0/3403746098 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3000a4040 con 0x562a36587370 2024-09-10T08:48:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.549+0000 7ff2eeffd640 1 -- 172.21.15.172:0/3403746098 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff3080295e0 con 0x562a36587370 2024-09-10T08:48:33.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.549+0000 7ff3181c4740 10 client.16188 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.549+0000 7ff3181c4740 1 -- 172.21.15.172:0/3403746098 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562a36605b00 con 0x562a36587370 2024-09-10T08:48:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.550+0000 7ff2eeffd640 1 -- 172.21.15.172:0/3403746098 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3000a4040 con 0x562a36587370 2024-09-10T08:48:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.550+0000 7ff2eeffd640 1 -- 172.21.15.172:0/3403746098 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff308028070 con 0x562a36587370 2024-09-10T08:48:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.550+0000 7ff3181c4740 10 client.16188 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.550+0000 7ff3181c4740 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.550+0000 7ff3181c4740 20 client.16188 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.550+0000 7ff3181c4740 1 --2- 172.21.15.172:0/3403746098 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562a3659b1b0 0x562a366082e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.550+0000 7ff3181c4740 4 client.16188 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:48:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.550+0000 7ff3181c4740 1 -- 172.21.15.172:0/3403746098 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a364e9d70 con 0x562a3659b1b0 2024-09-10T08:48:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.550+0000 7ff30f7fe640 1 --2- 172.21.15.172:0/3403746098 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562a3659b1b0 0x562a366082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.550+0000 7ff2eeffd640 10 client.16188 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.550+0000 7ff30f7fe640 1 --2- 172.21.15.172:0/3403746098 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562a3659b1b0 0x562a366082e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c5c9937552d785a server_cookie=3e7c400624ae98c6 in_seq=0 out_seq=0 2024-09-10T08:48:33.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.551+0000 7ff2eeffd640 1 -- 172.21.15.172:0/3403746098 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a364e9d70 con 0x562a3659b1b0 2024-09-10T08:48:33.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.551+0000 7ff2eeffd640 10 client.16188 handle_command_reply: tid=0 2024-09-10T08:48:33.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.551+0000 7ff3181c4740 1 client.16188 shutdown 2024-09-10T08:48:33.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.551+0000 7ff3181c4740 1 -- 172.21.15.172:0/3403746098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a36587370 msgr2=0x562a3659e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:33.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.551+0000 7ff3181c4740 1 --2- 172.21.15.172:0/3403746098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a36587370 0x562a3659e550 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff308011950 tx=0x7ff308004ab0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.551+0000 7ff3181c4740 1 -- 172.21.15.172:0/3403746098 shutdown_connections 2024-09-10T08:48:33.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.551+0000 7ff3181c4740 1 --2- 172.21.15.172:0/3403746098 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562a3659b1b0 0x562a366082e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.551+0000 7ff3181c4740 1 --2- 172.21.15.172:0/3403746098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a36587370 0x562a3659e550 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.551+0000 7ff3181c4740 1 --2- 172.21.15.172:0/3403746098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562a3657db40 0x562a3659e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.551+0000 7ff3181c4740 1 --2- 172.21.15.172:0/3403746098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562a3657c5d0 0x562a365a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.551+0000 7ff2eeffd640 0 client.16188 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:33.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.551+0000 7ff3181c4740 1 -- 172.21.15.172:0/3403746098 >> 172.21.15.172:0/3403746098 conn(0x562a36545310 msgr2=0x562a3657e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:33.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.551+0000 7ff3181c4740 1 -- 172.21.15.172:0/3403746098 shutdown_connections 2024-09-10T08:48:33.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.551+0000 7ff3181c4740 1 -- 172.21.15.172:0/3403746098 wait complete. 2024-09-10T08:48:33.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.551+0000 7ff3181c4740 20 client.16188 trim_cache size 0 max 16384 2024-09-10T08:48:33.560 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:48:33.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.552+0000 7ff315deb640 1 -- 172.21.15.172:0/2161663201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2f80a2b80 msgr2=0x7ff2f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:33.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.552+0000 7ff315deb640 1 --2- 172.21.15.172:0/2161663201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2f80a2b80 0x7ff2f80a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff308005fd0 tx=0x7ff30800f730 comp rx=0 tx=0).stop 2024-09-10T08:48:33.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.552+0000 7ff315deb640 1 -- 172.21.15.172:0/2161663201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff310100e20 msgr2=0x7ff310199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:33.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.552+0000 7ff315deb640 1 --2- 172.21.15.172:0/2161663201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff310100e20 0x7ff310199260 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff30000d950 tx=0x7ff30000de10 comp rx=0 tx=0).stop 2024-09-10T08:48:33.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.552+0000 7ff315deb640 1 -- 172.21.15.172:0/2161663201 shutdown_connections 2024-09-10T08:48:33.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.552+0000 7ff315deb640 1 --2- 172.21.15.172:0/2161663201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2f80a2b80 0x7ff2f80a5040 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.552+0000 7ff315deb640 1 --2- 172.21.15.172:0/2161663201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff310102ef0 0x7ff3101a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.552+0000 7ff315deb640 1 --2- 172.21.15.172:0/2161663201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff310102570 0x7ff3101997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.552+0000 7ff315deb640 1 --2- 172.21.15.172:0/2161663201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff310100e20 0x7ff310199260 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:33.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.552+0000 7ff315deb640 1 -- 172.21.15.172:0/2161663201 >> 172.21.15.172:0/2161663201 conn(0x7ff3100fc7d0 msgr2=0x7ff3100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:33.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.552+0000 7ff315deb640 1 -- 172.21.15.172:0/2161663201 shutdown_connections 2024-09-10T08:48:33.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:33.552+0000 7ff315deb640 1 -- 172.21.15.172:0/2161663201 wait complete. 2024-09-10T08:48:33.590 INFO:tasks.workunit.client.0.smithi172.stdout:checking for ANSI C header files... yes 2024-09-10T08:48:33.802 INFO:tasks.workunit.client.0.smithi172.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-10T08:48:33.871 INFO:tasks.workunit.client.0.smithi172.stdout:checking for sys/types.h... yes 2024-09-10T08:48:33.932 INFO:tasks.workunit.client.0.smithi172.stdout:checking for sys/stat.h... yes 2024-09-10T08:48:34.010 INFO:tasks.workunit.client.0.smithi172.stdout:checking for stdlib.h... yes 2024-09-10T08:48:34.076 INFO:tasks.workunit.client.0.smithi172.stdout:checking for string.h... yes 2024-09-10T08:48:34.146 INFO:tasks.workunit.client.0.smithi172.stdout:checking for memory.h... yes 2024-09-10T08:48:34.217 INFO:tasks.workunit.client.0.smithi172.stdout:checking for strings.h... yes 2024-09-10T08:48:34.280 INFO:tasks.workunit.client.0.smithi172.stdout:checking for inttypes.h... yes 2024-09-10T08:48:34.344 INFO:tasks.workunit.client.0.smithi172.stdout:checking for stdint.h... yes 2024-09-10T08:48:34.421 INFO:tasks.workunit.client.0.smithi172.stdout:checking for unistd.h... yes 2024-09-10T08:48:34.489 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:34 smithi172 ceph-mon[30854]: pgmap v420: 129 pgs: 129 active+clean; 1.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 94 KiB/s wr, 36 op/s 2024-09-10T08:48:34.524 INFO:tasks.workunit.client.0.smithi172.stdout:checking pthread.h usability... yes 2024-09-10T08:48:34.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:34 smithi191 ceph-mon[35314]: pgmap v420: 129 pgs: 129 active+clean; 1.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 94 KiB/s wr, 36 op/s 2024-09-10T08:48:34.553 INFO:tasks.workunit.client.0.smithi172.stdout:checking pthread.h presence... yes 2024-09-10T08:48:34.554 INFO:tasks.workunit.client.0.smithi172.stdout:checking for pthread.h... yes 2024-09-10T08:48:34.622 INFO:tasks.workunit.client.0.smithi172.stdout:checking fcntl.h usability... yes 2024-09-10T08:48:34.650 INFO:tasks.workunit.client.0.smithi172.stdout:checking fcntl.h presence... yes 2024-09-10T08:48:34.650 INFO:tasks.workunit.client.0.smithi172.stdout:checking for fcntl.h... yes 2024-09-10T08:48:34.721 INFO:tasks.workunit.client.0.smithi172.stdout:checking limits.h usability... yes 2024-09-10T08:48:34.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:34 smithi174 ceph-mon[34068]: pgmap v420: 129 pgs: 129 active+clean; 1.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 94 KiB/s wr, 36 op/s 2024-09-10T08:48:34.749 INFO:tasks.workunit.client.0.smithi172.stdout:checking limits.h presence... yes 2024-09-10T08:48:34.752 INFO:tasks.workunit.client.0.smithi172.stdout:checking for limits.h... yes 2024-09-10T08:48:34.772 INFO:tasks.workunit.client.0.smithi172.stdout:checking for stdint.h... (cached) yes 2024-09-10T08:48:34.844 INFO:tasks.workunit.client.0.smithi172.stdout:checking sys/time.h usability... yes 2024-09-10T08:48:34.873 INFO:tasks.workunit.client.0.smithi172.stdout:checking sys/time.h presence... yes 2024-09-10T08:48:34.873 INFO:tasks.workunit.client.0.smithi172.stdout:checking for sys/time.h... yes 2024-09-10T08:48:34.898 INFO:tasks.workunit.client.0.smithi172.stdout:checking for unistd.h... (cached) yes 2024-09-10T08:48:34.966 INFO:tasks.workunit.client.0.smithi172.stdout:checking sys/vfs.h usability... yes 2024-09-10T08:48:34.995 INFO:tasks.workunit.client.0.smithi172.stdout:checking sys/vfs.h presence... yes 2024-09-10T08:48:34.995 INFO:tasks.workunit.client.0.smithi172.stdout:checking for sys/vfs.h... yes 2024-09-10T08:48:35.058 INFO:tasks.workunit.client.0.smithi172.stdout:checking sys/limits.h usability... no 2024-09-10T08:48:35.088 INFO:tasks.workunit.client.0.smithi172.stdout:checking sys/limits.h presence... no 2024-09-10T08:48:35.088 INFO:tasks.workunit.client.0.smithi172.stdout:checking for sys/limits.h... no 2024-09-10T08:48:35.133 INFO:tasks.workunit.client.0.smithi172.stdout:checking for an ANSI C-conforming const... yes 2024-09-10T08:48:35.180 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-10T08:48:35.182 INFO:tasks.workunit.client.0.smithi172.stdout:checking for function prototypes... yes 2024-09-10T08:48:35.225 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether setvbuf arguments are reversed... no 2024-09-10T08:48:35.328 INFO:tasks.workunit.client.0.smithi172.stdout:checking for system... yes 2024-09-10T08:48:35.414 INFO:tasks.workunit.client.0.smithi172.stdout:checking for gettimeofday... yes 2024-09-10T08:48:35.498 INFO:tasks.workunit.client.0.smithi172.stdout:checking for mkdir... yes 2024-09-10T08:48:35.520 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:48:35.520 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:48:35.584 INFO:tasks.workunit.client.0.smithi172.stdout:checking for strerror... yes 2024-09-10T08:48:35.675 INFO:tasks.workunit.client.0.smithi172.stdout:checking for open64... yes 2024-09-10T08:48:35.771 INFO:tasks.workunit.client.0.smithi172.stdout:checking for stat64... yes 2024-09-10T08:48:35.797 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:35.877 INFO:tasks.workunit.client.0.smithi172.stdout:checking for fseeko64... yes 2024-09-10T08:48:35.964 INFO:tasks.workunit.client.0.smithi172.stdout:checking for lrand48_r... yes 2024-09-10T08:48:36.049 INFO:tasks.workunit.client.0.smithi172.stdout:checking for srand48_r... yes 2024-09-10T08:48:36.078 INFO:tasks.workunit.client.0.smithi172.stdout:configure: creating ./config.status 2024-09-10T08:48:36.249 INFO:tasks.workunit.client.0.smithi172.stdout:config.status: creating Makefile 2024-09-10T08:48:36.268 INFO:tasks.workunit.client.0.smithi172.stdout:config.status: creating config.h 2024-09-10T08:48:36.295 INFO:tasks.workunit.client.0.smithi172.stdout:config.status: executing depfiles commands 2024-09-10T08:48:36.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.300+0000 7fca657a3640 1 -- 172.21.15.172:0/1609032718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca60100a80 msgr2=0x7fca60100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:36.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.300+0000 7fca657a3640 1 --2- 172.21.15.172:0/1609032718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca60100a80 0x7fca60100ee0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fca54009890 tx=0x7fca5402a360 comp rx=0 tx=0).stop 2024-09-10T08:48:36.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.300+0000 7fca657a3640 1 -- 172.21.15.172:0/1609032718 shutdown_connections 2024-09-10T08:48:36.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.300+0000 7fca657a3640 1 --2- 172.21.15.172:0/1609032718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca60101420 0x7fca6010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.300+0000 7fca657a3640 1 --2- 172.21.15.172:0/1609032718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca60100a80 0x7fca60100ee0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.300+0000 7fca657a3640 1 --2- 172.21.15.172:0/1609032718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca60106a80 0x7fca60106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.300+0000 7fca657a3640 1 -- 172.21.15.172:0/1609032718 >> 172.21.15.172:0/1609032718 conn(0x7fca600fc7a0 msgr2=0x7fca600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:36.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.300+0000 7fca657a3640 1 -- 172.21.15.172:0/1609032718 shutdown_connections 2024-09-10T08:48:36.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.300+0000 7fca657a3640 1 -- 172.21.15.172:0/1609032718 wait complete. 2024-09-10T08:48:36.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.300+0000 7fca657a3640 1 Processor -- start 2024-09-10T08:48:36.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.301+0000 7fca657a3640 1 -- start start 2024-09-10T08:48:36.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.301+0000 7fca657a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca60100a80 0x7fca600fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.301+0000 7fca657a3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca60101420 0x7fca600fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.301+0000 7fca657a3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca60106a80 0x7fca600fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.301+0000 7fca657a3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fca60110cf0 con 0x7fca60100a80 2024-09-10T08:48:36.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.301+0000 7fca657a3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca60110b70 con 0x7fca60106a80 2024-09-10T08:48:36.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.301+0000 7fca657a3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fca60110e70 con 0x7fca60101420 2024-09-10T08:48:36.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.301+0000 7fca5effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca60100a80 0x7fca600fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.301+0000 7fca5effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca60100a80 0x7fca600fce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37476/0 (socket says 172.21.15.172:37476) 2024-09-10T08:48:36.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.301+0000 7fca5effd640 1 -- 172.21.15.172:0/3807720678 learned_addr learned my addr 172.21.15.172:0/3807720678 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:36.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.301+0000 7fca5e7fc640 1 --2- 172.21.15.172:0/3807720678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca60101420 0x7fca600fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.301+0000 7fca5f7fe640 1 --2- 172.21.15.172:0/3807720678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca60106a80 0x7fca600fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.302+0000 7fca5effd640 1 -- 172.21.15.172:0/3807720678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca60101420 msgr2=0x7fca600fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:36.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.302+0000 7fca5effd640 1 --2- 172.21.15.172:0/3807720678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca60101420 0x7fca600fd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.302+0000 7fca5effd640 1 -- 172.21.15.172:0/3807720678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca60106a80 msgr2=0x7fca600fd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:36.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.302+0000 7fca5effd640 1 --2- 172.21.15.172:0/3807720678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca60106a80 0x7fca600fd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.302+0000 7fca5effd640 1 -- 172.21.15.172:0/3807720678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca601a6820 con 0x7fca60100a80 2024-09-10T08:48:36.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.302+0000 7fca5e7fc640 1 --2- 172.21.15.172:0/3807720678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca60101420 0x7fca600fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:36.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.302+0000 7fca5effd640 1 --2- 172.21.15.172:0/3807720678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca60100a80 0x7fca600fce70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fca5000e9f0 tx=0x7fca5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:36.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.302+0000 7fca47fff640 1 -- 172.21.15.172:0/3807720678 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5000cd80 con 0x7fca60100a80 2024-09-10T08:48:36.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.302+0000 7fca657a3640 1 -- 172.21.15.172:0/3807720678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca601a6b10 con 0x7fca60100a80 2024-09-10T08:48:36.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.302+0000 7fca47fff640 1 -- 172.21.15.172:0/3807720678 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca500108e0 con 0x7fca60100a80 2024-09-10T08:48:36.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.302+0000 7fca47fff640 1 -- 172.21.15.172:0/3807720678 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca50018920 con 0x7fca60100a80 2024-09-10T08:48:36.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.302+0000 7fca657a3640 1 -- 172.21.15.172:0/3807720678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca601a7070 con 0x7fca60100a80 2024-09-10T08:48:36.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.305+0000 7fca47fff640 1 -- 172.21.15.172:0/3807720678 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fca50010a90 con 0x7fca60100a80 2024-09-10T08:48:36.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.305+0000 7fca67b7c740 1 Processor -- start 2024-09-10T08:48:36.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.305+0000 7fca67b7c740 1 -- start start 2024-09-10T08:48:36.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.305+0000 7fca67b7c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cadb60 0x55b6b9cadf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.305+0000 7fca67b7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cae480 0x55b6b9cb8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.305+0000 7fca67b7c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cbb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.305+0000 7fca67b7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b6b9c13280 con 0x55b6b9cae480 2024-09-10T08:48:36.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.305+0000 7fca47fff640 1 --2- 172.21.15.172:0/3807720678 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca480a2c00 0x7fca480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.305+0000 7fca67b7c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b6b9bbab10 con 0x55b6b9cadb60 2024-09-10T08:48:36.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.305+0000 7fca67b7c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b6b9c0f240 con 0x55b6b9cb8d40 2024-09-10T08:48:36.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.306+0000 7fca5effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cae480 0x55b6b9cb8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.306+0000 7fca5effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cae480 0x55b6b9cb8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37492/0 (socket says 172.21.15.172:37492) 2024-09-10T08:48:36.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.306+0000 7fca5f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cadb60 0x55b6b9cadf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.306+0000 7fca5effd640 1 -- 172.21.15.172:0/3053501492 learned_addr learned my addr 172.21.15.172:0/3053501492 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:36.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.306+0000 7fca5f7fe640 1 --2- 172.21.15.172:0/3807720678 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca480a2c00 0x7fca480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.306+0000 7fca5e7fc640 1 --2- 172.21.15.172:0/3053501492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cbb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.306+0000 7fca47fff640 1 -- 172.21.15.172:0/3807720678 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fca500cf580 con 0x7fca60100a80 2024-09-10T08:48:36.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.306+0000 7fca5effd640 1 -- 172.21.15.172:0/3053501492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cb8d40 msgr2=0x55b6b9cbb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:36.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.306+0000 7fca5effd640 1 --2- 172.21.15.172:0/3053501492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cbb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.306+0000 7fca5effd640 1 -- 172.21.15.172:0/3053501492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cadb60 msgr2=0x55b6b9cadf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:36.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.306+0000 7fca5effd640 1 --2- 172.21.15.172:0/3053501492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cadb60 0x55b6b9cadf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.306+0000 7fca5effd640 1 -- 172.21.15.172:0/3053501492 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b9bc71f0 con 0x55b6b9cae480 2024-09-10T08:48:36.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.306+0000 7fca5f7fe640 1 --2- 172.21.15.172:0/3807720678 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca480a2c00 0x7fca480a50c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fca58009ba0 tx=0x7fca58009b30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:36.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.306+0000 7fca5effd640 1 --2- 172.21.15.172:0/3053501492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cae480 0x55b6b9cb8800 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fca50098500 tx=0x7fca5001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a710e97df5be4e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:36.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.307+0000 7fca457fa640 1 -- 172.21.15.172:0/3053501492 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5009fe30 con 0x55b6b9cae480 2024-09-10T08:48:36.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.307+0000 7fca457fa640 1 -- 172.21.15.172:0/3053501492 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca500a46f0 con 0x55b6b9cae480 2024-09-10T08:48:36.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.307+0000 7fca457fa640 1 -- 172.21.15.172:0/3053501492 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca500981a0 con 0x55b6b9cae480 2024-09-10T08:48:36.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.307+0000 7fca67b7c740 1 -- 172.21.15.172:0/3053501492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cae480 msgr2=0x55b6b9cb8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:36.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.307+0000 7fca67b7c740 1 --2- 172.21.15.172:0/3053501492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cae480 0x55b6b9cb8800 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fca50098500 tx=0x7fca5001f000 comp rx=0 tx=0).stop 2024-09-10T08:48:36.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.308+0000 7fca67b7c740 1 -- 172.21.15.172:0/3053501492 shutdown_connections 2024-09-10T08:48:36.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.308+0000 7fca67b7c740 1 --2- 172.21.15.172:0/3053501492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cbb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.308+0000 7fca67b7c740 1 --2- 172.21.15.172:0/3053501492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cae480 0x55b6b9cb8800 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.308+0000 7fca67b7c740 1 --2- 172.21.15.172:0/3053501492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cadb60 0x55b6b9cadf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.308+0000 7fca67b7c740 1 -- 172.21.15.172:0/3053501492 >> 172.21.15.172:0/3053501492 conn(0x55b6b9caa0f0 msgr2=0x55b6b9caa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:36.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.308+0000 7fca67b7c740 1 -- 172.21.15.172:0/3053501492 shutdown_connections 2024-09-10T08:48:36.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.308+0000 7fca67b7c740 1 -- 172.21.15.172:0/3053501492 wait complete. 2024-09-10T08:48:36.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.308+0000 7fca67b7c740 1 -- start start 2024-09-10T08:48:36.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.308+0000 7fca67b7c740 1 Processor -- start 2024-09-10T08:48:36.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.308+0000 7fca67b7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cadb60 0x55b6b9cb5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.308+0000 7fca67b7c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cae480 0x55b6b9cb6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.308+0000 7fca67b7c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cb66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.308+0000 7fca67b7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b6b9cbb8b0 con 0x55b6b9cadb60 2024-09-10T08:48:36.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.308+0000 7fca67b7c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b6b9bcdf10 con 0x55b6b9cb8d40 2024-09-10T08:48:36.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.308+0000 7fca67b7c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b6b9cbba30 con 0x55b6b9cae480 2024-09-10T08:48:36.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.308+0000 7fca67b7c740 10 client.0 _ll_register_callbacks cb 0x55b6b9b96a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:36.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.309+0000 7fca5effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cadb60 0x55b6b9cb5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.309+0000 7fca5e7fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cae480 0x55b6b9cb6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.309+0000 7fca5e7fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cae480 0x55b6b9cb6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:43486/0 (socket says 172.21.15.172:43486) 2024-09-10T08:48:36.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.309+0000 7fca5e7fc640 1 -- 172.21.15.172:0/3523660098 learned_addr learned my addr 172.21.15.172:0/3523660098 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:36.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.309+0000 7fca5f7fe640 1 --2- 172.21.15.172:0/3523660098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cb66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.309+0000 7fca5e7fc640 1 -- 172.21.15.172:0/3523660098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cb8d40 msgr2=0x55b6b9cb66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:36.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.309+0000 7fca5e7fc640 1 --2- 172.21.15.172:0/3523660098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cb66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.309+0000 7fca5e7fc640 1 -- 172.21.15.172:0/3523660098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cadb60 msgr2=0x55b6b9cb5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:36.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.309+0000 7fca5e7fc640 1 --2- 172.21.15.172:0/3523660098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cadb60 0x55b6b9cb5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.309+0000 7fca5e7fc640 1 -- 172.21.15.172:0/3523660098 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6b99846a0 con 0x55b6b9cae480 2024-09-10T08:48:36.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.309+0000 7fca5effd640 1 --2- 172.21.15.172:0/3523660098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cadb60 0x55b6b9cb5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:36.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.309+0000 7fca5e7fc640 1 --2- 172.21.15.172:0/3523660098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cae480 0x55b6b9cb6160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fca54000c00 tx=0x7fca54030dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73548617970e87d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:36.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.309+0000 7fca42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:48:36.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.309+0000 7fca42ff5640 1 -- 172.21.15.172:0/3523660098 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54037040 con 0x55b6b9cae480 2024-09-10T08:48:36.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.309+0000 7fca67b7c740 1 -- 172.21.15.172:0/3523660098 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca500a46f0 con 0x55b6b9cae480 2024-09-10T08:48:36.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.309+0000 7fca42ff5640 1 -- 172.21.15.172:0/3523660098 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fca54041d30 con 0x55b6b9cae480 2024-09-10T08:48:36.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.309+0000 7fca42ff5640 1 -- 172.21.15.172:0/3523660098 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fca54004040 con 0x55b6b9cae480 2024-09-10T08:48:36.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.310+0000 7fca67b7c740 10 client.25498 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:36.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.310+0000 7fca67b7c740 1 -- 172.21.15.172:0/3523660098 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6b9cc7500 con 0x55b6b9cae480 2024-09-10T08:48:36.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.310+0000 7fca42ff5640 1 -- 172.21.15.172:0/3523660098 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca500a46f0 con 0x55b6b9cae480 2024-09-10T08:48:36.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.310+0000 7fca42ff5640 1 -- 172.21.15.172:0/3523660098 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fca540329b0 con 0x55b6b9cae480 2024-09-10T08:48:36.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.310+0000 7fca67b7c740 10 client.25498 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:36.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.310+0000 7fca67b7c740 10 client.25498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:36.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.310+0000 7fca67b7c740 20 client.25498 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:36.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.310+0000 7fca67b7c740 1 --2- 172.21.15.172:0/3523660098 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b6b9cc8910 0x55b6b9ccad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.310+0000 7fca67b7c740 4 client.25498 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:36.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.310+0000 7fca67b7c740 1 -- 172.21.15.172:0/3523660098 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6b9c207f0 con 0x55b6b9cc8910 2024-09-10T08:48:36.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.310+0000 7fca5effd640 1 --2- 172.21.15.172:0/3523660098 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b6b9cc8910 0x55b6b9ccad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.311+0000 7fca42ff5640 10 client.25498 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:36.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.311+0000 7fca5effd640 1 --2- 172.21.15.172:0/3523660098 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b6b9cc8910 0x55b6b9ccad20 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f71852a5df56a594 server_cookie=ed97a771c8894127 in_seq=0 out_seq=0 2024-09-10T08:48:36.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.311+0000 7fca42ff5640 1 -- 172.21.15.172:0/3523660098 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6b9c207f0 con 0x55b6b9cc8910 2024-09-10T08:48:36.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.311+0000 7fca42ff5640 10 client.25498 handle_command_reply: tid=0 2024-09-10T08:48:36.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.311+0000 7fca67b7c740 1 client.25498 shutdown 2024-09-10T08:48:36.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.312+0000 7fca67b7c740 1 -- 172.21.15.172:0/3523660098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cae480 msgr2=0x55b6b9cb6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:36.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.312+0000 7fca67b7c740 1 --2- 172.21.15.172:0/3523660098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cae480 0x55b6b9cb6160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fca54000c00 tx=0x7fca54030dc0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.312+0000 7fca67b7c740 1 -- 172.21.15.172:0/3523660098 shutdown_connections 2024-09-10T08:48:36.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.312+0000 7fca67b7c740 1 --2- 172.21.15.172:0/3523660098 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b6b9cc8910 0x55b6b9ccad20 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.312+0000 7fca42ff5640 0 client.25498 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:36.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.312+0000 7fca67b7c740 1 --2- 172.21.15.172:0/3523660098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cb66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.312+0000 7fca67b7c740 1 --2- 172.21.15.172:0/3523660098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cae480 0x55b6b9cb6160 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.312+0000 7fca67b7c740 1 --2- 172.21.15.172:0/3523660098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cadb60 0x55b6b9cb5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.312+0000 7fca67b7c740 1 -- 172.21.15.172:0/3523660098 >> 172.21.15.172:0/3523660098 conn(0x55b6b9caa0f0 msgr2=0x55b6b9ca9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:36.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.312+0000 7fca67b7c740 1 -- 172.21.15.172:0/3523660098 shutdown_connections 2024-09-10T08:48:36.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.312+0000 7fca67b7c740 1 -- 172.21.15.172:0/3523660098 wait complete. 2024-09-10T08:48:36.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.312+0000 7fca67b7c740 20 client.25498 trim_cache size 0 max 16384 2024-09-10T08:48:36.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.333+0000 7fca67b7c740 1 Processor -- start 2024-09-10T08:48:36.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.333+0000 7fca67b7c740 1 -- start start 2024-09-10T08:48:36.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.333+0000 7fca67b7c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cadb60 0x55b6b9cb05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.333+0000 7fca67b7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cae480 0x55b6b9cb7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.333+0000 7fca67b7c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cd05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.333+0000 7fca67b7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fca601ac8b0 con 0x55b6b9cae480 2024-09-10T08:48:36.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.333+0000 7fca67b7c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca601ae3f0 con 0x55b6b9cadb60 2024-09-10T08:48:36.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.333+0000 7fca67b7c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fca601acf80 con 0x55b6b9cb8d40 2024-09-10T08:48:36.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.333+0000 7fca5e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cae480 0x55b6b9cb7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.334+0000 7fca5e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cae480 0x55b6b9cb7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37504/0 (socket says 172.21.15.172:37504) 2024-09-10T08:48:36.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.334+0000 7fca5e7fc640 1 -- 172.21.15.172:0/2981257897 learned_addr learned my addr 172.21.15.172:0/2981257897 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:36.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.334+0000 7fca5effd640 1 --2- 172.21.15.172:0/2981257897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cadb60 0x55b6b9cb05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.334+0000 7fca5f7fe640 1 --2- 172.21.15.172:0/2981257897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cd05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.334+0000 7fca5e7fc640 1 -- 172.21.15.172:0/2981257897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cb8d40 msgr2=0x55b6b9cd05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:36.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.334+0000 7fca5e7fc640 1 --2- 172.21.15.172:0/2981257897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cd05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.334+0000 7fca5e7fc640 1 -- 172.21.15.172:0/2981257897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cadb60 msgr2=0x55b6b9cb05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:36.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.334+0000 7fca5e7fc640 1 --2- 172.21.15.172:0/2981257897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cadb60 0x55b6b9cb05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.334+0000 7fca5e7fc640 1 -- 172.21.15.172:0/2981257897 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b9cb84f0 con 0x55b6b9cae480 2024-09-10T08:48:36.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.334+0000 7fca5effd640 1 --2- 172.21.15.172:0/2981257897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cadb60 0x55b6b9cb05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:36.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.334+0000 7fca5f7fe640 1 --2- 172.21.15.172:0/2981257897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cd05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:36.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.334+0000 7fca5e7fc640 1 --2- 172.21.15.172:0/2981257897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cae480 0x55b6b9cb7a80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fca54030b60 tx=0x7fca54005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1792b93ae6c98b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:36.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.334+0000 7fca44ff9640 1 -- 172.21.15.172:0/2981257897 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54037070 con 0x55b6b9cae480 2024-09-10T08:48:36.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.334+0000 7fca44ff9640 1 -- 172.21.15.172:0/2981257897 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca540041e0 con 0x55b6b9cae480 2024-09-10T08:48:36.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.334+0000 7fca44ff9640 1 -- 172.21.15.172:0/2981257897 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54030340 con 0x55b6b9cae480 2024-09-10T08:48:36.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.335+0000 7fca67b7c740 1 -- 172.21.15.172:0/2981257897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cae480 msgr2=0x55b6b9cb7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:36.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.335+0000 7fca67b7c740 1 --2- 172.21.15.172:0/2981257897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cae480 0x55b6b9cb7a80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fca54030b60 tx=0x7fca54005c90 comp rx=0 tx=0).stop 2024-09-10T08:48:36.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.335+0000 7fca67b7c740 1 -- 172.21.15.172:0/2981257897 shutdown_connections 2024-09-10T08:48:36.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.335+0000 7fca67b7c740 1 --2- 172.21.15.172:0/2981257897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cd05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.335+0000 7fca67b7c740 1 --2- 172.21.15.172:0/2981257897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cae480 0x55b6b9cb7a80 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.335+0000 7fca67b7c740 1 --2- 172.21.15.172:0/2981257897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cadb60 0x55b6b9cb05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.335+0000 7fca67b7c740 1 -- 172.21.15.172:0/2981257897 >> 172.21.15.172:0/2981257897 conn(0x55b6b9caa0f0 msgr2=0x55b6b9cb7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:36.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.335+0000 7fca67b7c740 1 -- 172.21.15.172:0/2981257897 shutdown_connections 2024-09-10T08:48:36.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.335+0000 7fca67b7c740 1 -- 172.21.15.172:0/2981257897 wait complete. 2024-09-10T08:48:36.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.336+0000 7fca67b7c740 1 -- start start 2024-09-10T08:48:36.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.336+0000 7fca67b7c740 1 Processor -- start 2024-09-10T08:48:36.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.336+0000 7fca67b7c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cadb60 0x55b6b9cbcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.336+0000 7fca67b7c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cae480 0x55b6b9cbcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.336+0000 7fca67b7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cbd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.336+0000 7fca67b7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fca601add20 con 0x55b6b9cb8d40 2024-09-10T08:48:36.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.336+0000 7fca67b7c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca601ad650 con 0x55b6b9cae480 2024-09-10T08:48:36.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.336+0000 7fca67b7c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fca601aa890 con 0x55b6b9cadb60 2024-09-10T08:48:36.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.336+0000 7fca67b7c740 10 client.0 _ll_register_callbacks cb 0x7fca58009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:36.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.336+0000 7fca5f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cbd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.336+0000 7fca5effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cadb60 0x55b6b9cbcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.336+0000 7fca5e7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cae480 0x55b6b9cbcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.336+0000 7fca5f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cbd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37516/0 (socket says 172.21.15.172:37516) 2024-09-10T08:48:36.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.336+0000 7fca5f7fe640 1 -- 172.21.15.172:0/2787351186 learned_addr learned my addr 172.21.15.172:0/2787351186 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:36.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.336+0000 7fca5effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cadb60 0x55b6b9cbcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:43506/0 (socket says 172.21.15.172:43506) 2024-09-10T08:48:36.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.337+0000 7fca5e7fc640 1 -- 172.21.15.172:0/2787351186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cadb60 msgr2=0x55b6b9cbcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:36.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.337+0000 7fca5e7fc640 1 --2- 172.21.15.172:0/2787351186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cadb60 0x55b6b9cbcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.337+0000 7fca5e7fc640 1 -- 172.21.15.172:0/2787351186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cb8d40 msgr2=0x55b6b9cbd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:36.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.337+0000 7fca5e7fc640 1 --2- 172.21.15.172:0/2787351186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cbd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.337+0000 7fca5e7fc640 1 -- 172.21.15.172:0/2787351186 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6b9d32690 con 0x55b6b9cae480 2024-09-10T08:48:36.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.337+0000 7fca5e7fc640 1 --2- 172.21.15.172:0/2787351186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cae480 0x55b6b9cbcff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fca5402a880 tx=0x7fca5402a360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7efe240241e57abb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:36.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.337+0000 7fca42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:48:36.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.337+0000 7fca42ff5640 1 -- 172.21.15.172:0/2787351186 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54037040 con 0x55b6b9cae480 2024-09-10T08:48:36.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.337+0000 7fca42ff5640 1 -- 172.21.15.172:0/2787351186 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fca54030340 con 0x55b6b9cae480 2024-09-10T08:48:36.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.337+0000 7fca67b7c740 1 -- 172.21.15.172:0/2787351186 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca540041e0 con 0x55b6b9cae480 2024-09-10T08:48:36.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.337+0000 7fca42ff5640 1 -- 172.21.15.172:0/2787351186 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fca54032af0 con 0x55b6b9cae480 2024-09-10T08:48:36.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.337+0000 7fca67b7c740 10 client.25568 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:36.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.337+0000 7fca67b7c740 1 -- 172.21.15.172:0/2787351186 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6b9d339b0 con 0x55b6b9cae480 2024-09-10T08:48:36.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.338+0000 7fca42ff5640 1 -- 172.21.15.172:0/2787351186 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca54003870 con 0x55b6b9cae480 2024-09-10T08:48:36.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.338+0000 7fca42ff5640 1 -- 172.21.15.172:0/2787351186 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fca54057030 con 0x55b6b9cae480 2024-09-10T08:48:36.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.338+0000 7fca67b7c740 10 client.25568 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:36.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.338+0000 7fca67b7c740 10 client.25568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:36.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.338+0000 7fca67b7c740 20 client.25568 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:36.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.338+0000 7fca67b7c740 1 --2- 172.21.15.172:0/2787351186 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b6b9cc8910 0x55b6b9d35f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:36.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.338+0000 7fca67b7c740 4 client.25568 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:48:36.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.338+0000 7fca67b7c740 1 -- 172.21.15.172:0/2787351186 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fca540041e0 con 0x55b6b9cc8910 2024-09-10T08:48:36.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.338+0000 7fca5effd640 1 --2- 172.21.15.172:0/2787351186 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b6b9cc8910 0x55b6b9d35f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:36.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.338+0000 7fca5effd640 1 --2- 172.21.15.172:0/2787351186 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b6b9cc8910 0x55b6b9d35f40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3486d8392fa0a64d server_cookie=53a8951ef30b545f in_seq=0 out_seq=0 2024-09-10T08:48:36.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.338+0000 7fca42ff5640 10 client.25568 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:36.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.339+0000 7fca42ff5640 1 -- 172.21.15.172:0/2787351186 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fca540041e0 con 0x55b6b9cc8910 2024-09-10T08:48:36.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.339+0000 7fca42ff5640 10 client.25568 handle_command_reply: tid=0 2024-09-10T08:48:36.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.339+0000 7fca67b7c740 1 client.25568 shutdown 2024-09-10T08:48:36.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.339+0000 7fca67b7c740 1 -- 172.21.15.172:0/2787351186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cae480 msgr2=0x55b6b9cbcff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:36.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.339+0000 7fca67b7c740 1 --2- 172.21.15.172:0/2787351186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cae480 0x55b6b9cbcff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fca5402a880 tx=0x7fca5402a360 comp rx=0 tx=0).stop 2024-09-10T08:48:36.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.340+0000 7fca67b7c740 1 -- 172.21.15.172:0/2787351186 shutdown_connections 2024-09-10T08:48:36.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.340+0000 7fca67b7c740 1 --2- 172.21.15.172:0/2787351186 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b6b9cc8910 0x55b6b9d35f40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.340+0000 7fca42ff5640 0 client.25568 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:36.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.340+0000 7fca67b7c740 1 --2- 172.21.15.172:0/2787351186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6b9cb8d40 0x55b6b9cbd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.340+0000 7fca67b7c740 1 --2- 172.21.15.172:0/2787351186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6b9cae480 0x55b6b9cbcff0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.340+0000 7fca67b7c740 1 --2- 172.21.15.172:0/2787351186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6b9cadb60 0x55b6b9cbcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.340+0000 7fca67b7c740 1 -- 172.21.15.172:0/2787351186 >> 172.21.15.172:0/2787351186 conn(0x55b6b9caa0f0 msgr2=0x55b6b9ca9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:36.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.340+0000 7fca67b7c740 1 -- 172.21.15.172:0/2787351186 shutdown_connections 2024-09-10T08:48:36.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.340+0000 7fca67b7c740 1 -- 172.21.15.172:0/2787351186 wait complete. 2024-09-10T08:48:36.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.340+0000 7fca67b7c740 20 client.25568 trim_cache size 0 max 16384 2024-09-10T08:48:36.345 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:48:36.345 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T08:48:36.345 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "d6139d6e-8fbd-440e-ac9b-cb284f00390b", 2024-09-10T08:48:36.346 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T08:48:36.346 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:48:36.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.341+0000 7fca657a3640 1 -- 172.21.15.172:0/3807720678 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca480a2c00 msgr2=0x7fca480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:36.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.341+0000 7fca657a3640 1 --2- 172.21.15.172:0/3807720678 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca480a2c00 0x7fca480a50c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fca58009ba0 tx=0x7fca58009b30 comp rx=0 tx=0).stop 2024-09-10T08:48:36.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.341+0000 7fca657a3640 1 -- 172.21.15.172:0/3807720678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca60100a80 msgr2=0x7fca600fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:36.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.341+0000 7fca657a3640 1 --2- 172.21.15.172:0/3807720678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca60100a80 0x7fca600fce70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fca5000e9f0 tx=0x7fca5000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.341+0000 7fca657a3640 1 -- 172.21.15.172:0/3807720678 shutdown_connections 2024-09-10T08:48:36.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.341+0000 7fca657a3640 1 --2- 172.21.15.172:0/3807720678 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca480a2c00 0x7fca480a50c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.341+0000 7fca657a3640 1 --2- 172.21.15.172:0/3807720678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca60106a80 0x7fca600fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.341+0000 7fca657a3640 1 --2- 172.21.15.172:0/3807720678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca60101420 0x7fca600fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.341+0000 7fca657a3640 1 --2- 172.21.15.172:0/3807720678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca60100a80 0x7fca600fce70 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:36.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.341+0000 7fca657a3640 1 -- 172.21.15.172:0/3807720678 >> 172.21.15.172:0/3807720678 conn(0x7fca600fc7a0 msgr2=0x7fca6006aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:36.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.341+0000 7fca657a3640 1 -- 172.21.15.172:0/3807720678 shutdown_connections 2024-09-10T08:48:36.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:36.341+0000 7fca657a3640 1 -- 172.21.15.172:0/3807720678 wait complete. 2024-09-10T08:48:36.386 INFO:tasks.workunit.client.0.smithi172.stdout:config.status: executing default commands 2024-09-10T08:48:36.450 INFO:tasks.workunit.client.0.smithi172.stderr:+ make 2024-09-10T08:48:36.484 INFO:tasks.workunit.client.0.smithi172.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-09-10T08:48:36.490 INFO:tasks.workunit.client.0.smithi172.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-09-10T08:48:36.491 INFO:tasks.workunit.client.0.smithi172.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-09-10T08:48:36.491 INFO:tasks.workunit.client.0.smithi172.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-09-10T08:48:36.492 INFO:tasks.workunit.client.0.smithi172.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-09-10T08:48:36.492 INFO:tasks.workunit.client.0.smithi172.stderr: any GNU archive site. 2024-09-10T08:48:36.497 INFO:tasks.workunit.client.0.smithi172.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-09-10T08:48:36.501 INFO:tasks.workunit.client.0.smithi172.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-09-10T08:48:36.502 INFO:tasks.workunit.client.0.smithi172.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-09-10T08:48:36.502 INFO:tasks.workunit.client.0.smithi172.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-09-10T08:48:36.502 INFO:tasks.workunit.client.0.smithi172.stderr: You might want to install the `Automake' and `Perl' packages. 2024-09-10T08:48:36.502 INFO:tasks.workunit.client.0.smithi172.stderr: Grab them from any GNU archive site. 2024-09-10T08:48:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:36 smithi191 ceph-mon[35314]: pgmap v421: 129 pgs: 129 active+clean; 4.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 357 KiB/s wr, 84 op/s 2024-09-10T08:48:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:36 smithi172 ceph-mon[30854]: pgmap v421: 129 pgs: 129 active+clean; 4.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 357 KiB/s wr, 84 op/s 2024-09-10T08:48:36.720 INFO:tasks.workunit.client.0.smithi172.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-09-10T08:48:36.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:36 smithi174 ceph-mon[34068]: pgmap v421: 129 pgs: 129 active+clean; 4.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 357 KiB/s wr, 84 op/s 2024-09-10T08:48:37.305 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:48:37.540 INFO:tasks.workunit.client.0.smithi172.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-09-10T08:48:37.540 INFO:tasks.workunit.client.0.smithi172.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-09-10T08:48:37.540 INFO:tasks.workunit.client.0.smithi172.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-09-10T08:48:37.540 INFO:tasks.workunit.client.0.smithi172.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-09-10T08:48:37.558 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:48:37.692 INFO:tasks.workunit.client.0.smithi172.stdout:/bin/sh ./config.status --recheck 2024-09-10T08:48:37.752 INFO:tasks.workunit.client.0.smithi172.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-09-10T08:48:37.923 INFO:tasks.workunit.client.0.smithi172.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-10T08:48:38.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.094+0000 7f4affb3c640 1 -- 172.21.15.172:0/685013610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4af8074f40 msgr2=0x7f4af810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:38.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.094+0000 7f4affb3c640 1 --2- 172.21.15.172:0/685013610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4af8074f40 0x7f4af810de80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009900 tx=0x7f4ae802a480 comp rx=0 tx=0).stop 2024-09-10T08:48:38.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.094+0000 7f4affb3c640 1 -- 172.21.15.172:0/685013610 shutdown_connections 2024-09-10T08:48:38.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.094+0000 7f4affb3c640 1 --2- 172.21.15.172:0/685013610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4af810e3c0 0x7f4af8110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.094+0000 7f4affb3c640 1 --2- 172.21.15.172:0/685013610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4af8074f40 0x7f4af810de80 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.094+0000 7f4affb3c640 1 --2- 172.21.15.172:0/685013610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4af8074620 0x7f4af8074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.094+0000 7f4affb3c640 1 -- 172.21.15.172:0/685013610 >> 172.21.15.172:0/685013610 conn(0x7f4af80fd8c0 msgr2=0x7f4af80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:38.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.094+0000 7f4affb3c640 1 -- 172.21.15.172:0/685013610 shutdown_connections 2024-09-10T08:48:38.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.094+0000 7f4affb3c640 1 -- 172.21.15.172:0/685013610 wait complete. 2024-09-10T08:48:38.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.095+0000 7f4affb3c640 1 Processor -- start 2024-09-10T08:48:38.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.095+0000 7f4affb3c640 1 -- start start 2024-09-10T08:48:38.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.095+0000 7f4affb3c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4af8074620 0x7f4af819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.095+0000 7f4affb3c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4af8074f40 0x7f4af819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.095+0000 7f4affb3c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4af810e3c0 0x7f4af819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.095+0000 7f4affb3c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8114010 con 0x7f4af8074620 2024-09-10T08:48:38.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.095+0000 7f4affb3c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8113e90 con 0x7f4af810e3c0 2024-09-10T08:48:38.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.095+0000 7f4affb3c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8114190 con 0x7f4af8074f40 2024-09-10T08:48:38.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.095+0000 7f4afd8d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4af8074620 0x7f4af819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:38.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.095+0000 7f4afd8d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4af8074620 0x7f4af819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37536/0 (socket says 172.21.15.172:37536) 2024-09-10T08:48:38.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.095+0000 7f4afd8d8640 1 -- 172.21.15.172:0/843603895 learned_addr learned my addr 172.21.15.172:0/843603895 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:38.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.095+0000 7f4afd0d7640 1 --2- 172.21.15.172:0/843603895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4af8074f40 0x7f4af819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:38.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.095+0000 7f4afe0d9640 1 --2- 172.21.15.172:0/843603895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4af810e3c0 0x7f4af819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:38.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.096+0000 7f4afd8d8640 1 -- 172.21.15.172:0/843603895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4af8074f40 msgr2=0x7f4af819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:38.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.096+0000 7f4afd8d8640 1 --2- 172.21.15.172:0/843603895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4af8074f40 0x7f4af819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.096+0000 7f4afd8d8640 1 -- 172.21.15.172:0/843603895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4af810e3c0 msgr2=0x7f4af819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:38.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.096+0000 7f4afd8d8640 1 --2- 172.21.15.172:0/843603895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4af810e3c0 0x7f4af819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.096+0000 7f4afd8d8640 1 -- 172.21.15.172:0/843603895 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af819f5e0 con 0x7f4af8074620 2024-09-10T08:48:38.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.096+0000 7f4afd8d8640 1 --2- 172.21.15.172:0/843603895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4af8074620 0x7f4af819a630 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4af400e9f0 tx=0x7f4af400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:38.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.096+0000 7f4aeeffd640 1 -- 172.21.15.172:0/843603895 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af400cd80 con 0x7f4af8074620 2024-09-10T08:48:38.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.096+0000 7f4aeeffd640 1 -- 172.21.15.172:0/843603895 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4af40108e0 con 0x7f4af8074620 2024-09-10T08:48:38.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.096+0000 7f4affb3c640 1 -- 172.21.15.172:0/843603895 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af819f8d0 con 0x7f4af8074620 2024-09-10T08:48:38.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.096+0000 7f4aeeffd640 1 -- 172.21.15.172:0/843603895 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af4018920 con 0x7f4af8074620 2024-09-10T08:48:38.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.096+0000 7f4affb3c640 1 -- 172.21.15.172:0/843603895 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af81a71b0 con 0x7f4af8074620 2024-09-10T08:48:38.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.098+0000 7f4aeeffd640 1 -- 172.21.15.172:0/843603895 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4af4010a90 con 0x7f4af8074620 2024-09-10T08:48:38.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.099+0000 7f4b01f15740 1 Processor -- start 2024-09-10T08:48:38.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.099+0000 7f4b01f15740 1 -- start start 2024-09-10T08:48:38.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.099+0000 7f4b01f15740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486cb4a0 0x55a7486cb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.099+0000 7f4b01f15740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486cbdc0 0x55a7486d6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.099+0000 7f4b01f15740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486d6680 0x55a7486d8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.099+0000 7f4b01f15740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a7486312e0 con 0x55a7486d6680 2024-09-10T08:48:38.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.099+0000 7f4b01f15740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a7485d8ce0 con 0x55a7486cbdc0 2024-09-10T08:48:38.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.099+0000 7f4b01f15740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a74862d2a0 con 0x55a7486cb4a0 2024-09-10T08:48:38.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.099+0000 7f4aeeffd640 1 --2- 172.21.15.172:0/843603895 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4ae40a2c00 0x7f4ae40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.099+0000 7f4afd0d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486d6680 0x55a7486d8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:38.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.099+0000 7f4afd0d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486d6680 0x55a7486d8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37540/0 (socket says 172.21.15.172:37540) 2024-09-10T08:48:38.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.099+0000 7f4afd0d7640 1 -- 172.21.15.172:0/232717191 learned_addr learned my addr 172.21.15.172:0/232717191 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:38.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.099+0000 7f4afe0d9640 1 --2- 172.21.15.172:0/232717191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486cb4a0 0x55a7486cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:38.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.099+0000 7f4afd8d8640 1 --2- 172.21.15.172:0/232717191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486cbdc0 0x55a7486d6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:38.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.100+0000 7f4afe0d9640 1 --2- 172.21.15.172:0/843603895 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4ae40a2c00 0x7f4ae40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:38.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.100+0000 7f4aeeffd640 1 -- 172.21.15.172:0/843603895 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4af40cf620 con 0x7f4af8074620 2024-09-10T08:48:38.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.100+0000 7f4afd0d7640 1 -- 172.21.15.172:0/232717191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486cb4a0 msgr2=0x55a7486cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:38.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.100+0000 7f4afd0d7640 1 --2- 172.21.15.172:0/232717191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486cb4a0 0x55a7486cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.100+0000 7f4afd0d7640 1 -- 172.21.15.172:0/232717191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486cbdc0 msgr2=0x55a7486d6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:38.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.100+0000 7f4afd0d7640 1 --2- 172.21.15.172:0/232717191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486cbdc0 0x55a7486d6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.100+0000 7f4afd0d7640 1 -- 172.21.15.172:0/232717191 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7485bd440 con 0x55a7486d6680 2024-09-10T08:48:38.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.100+0000 7f4afd8d8640 1 --2- 172.21.15.172:0/232717191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486cbdc0 0x55a7486d6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:38.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.100+0000 7f4afe0d9640 1 --2- 172.21.15.172:0/843603895 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4ae40a2c00 0x7f4ae40a50c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4af0005fd0 tx=0x7f4af0009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:38.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.100+0000 7f4afd0d7640 1 --2- 172.21.15.172:0/232717191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486d6680 0x55a7486d8a70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009900 tx=0x7f4ae8032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e509529b79fb3085 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:38.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.100+0000 7f4ae3fff640 1 -- 172.21.15.172:0/232717191 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8040070 con 0x55a7486d6680 2024-09-10T08:48:38.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.100+0000 7f4ae3fff640 1 -- 172.21.15.172:0/232717191 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ae8004030 con 0x55a7486d6680 2024-09-10T08:48:38.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.100+0000 7f4ae3fff640 1 -- 172.21.15.172:0/232717191 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8035040 con 0x55a7486d6680 2024-09-10T08:48:38.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.101+0000 7f4b01f15740 1 -- 172.21.15.172:0/232717191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486d6680 msgr2=0x55a7486d8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:38.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.101+0000 7f4b01f15740 1 --2- 172.21.15.172:0/232717191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486d6680 0x55a7486d8a70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009900 tx=0x7f4ae8032bb0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.101+0000 7f4b01f15740 1 -- 172.21.15.172:0/232717191 shutdown_connections 2024-09-10T08:48:38.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.101+0000 7f4b01f15740 1 --2- 172.21.15.172:0/232717191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486d6680 0x55a7486d8a70 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.101+0000 7f4b01f15740 1 --2- 172.21.15.172:0/232717191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486cbdc0 0x55a7486d6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.101+0000 7f4b01f15740 1 --2- 172.21.15.172:0/232717191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486cb4a0 0x55a7486cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.101+0000 7f4b01f15740 1 -- 172.21.15.172:0/232717191 >> 172.21.15.172:0/232717191 conn(0x55a7486c7a30 msgr2=0x55a7486c7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:38.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.101+0000 7f4b01f15740 1 -- 172.21.15.172:0/232717191 shutdown_connections 2024-09-10T08:48:38.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.101+0000 7f4b01f15740 1 -- 172.21.15.172:0/232717191 wait complete. 2024-09-10T08:48:38.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.101+0000 7f4b01f15740 1 -- start start 2024-09-10T08:48:38.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.101+0000 7f4b01f15740 1 Processor -- start 2024-09-10T08:48:38.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.102+0000 7f4b01f15740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486cb4a0 0x55a7486d5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.102+0000 7f4b01f15740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486cbdc0 0x55a7486d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.102+0000 7f4b01f15740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486d6680 0x55a7486d37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.102+0000 7f4b01f15740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a7486d9250 con 0x55a7486d6680 2024-09-10T08:48:38.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.102+0000 7f4b01f15740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a7485ec050 con 0x55a7486cbdc0 2024-09-10T08:48:38.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.102+0000 7f4b01f15740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a7486d93d0 con 0x55a7486cb4a0 2024-09-10T08:48:38.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.102+0000 7f4b01f15740 10 client.0 _ll_register_callbacks cb 0x55a74862c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:38.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.102+0000 7f4afe0d9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486d6680 0x55a7486d37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:38.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.102+0000 7f4afe0d9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486d6680 0x55a7486d37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37556/0 (socket says 172.21.15.172:37556) 2024-09-10T08:48:38.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.102+0000 7f4afe0d9640 1 -- 172.21.15.172:0/661106019 learned_addr learned my addr 172.21.15.172:0/661106019 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:38.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.102+0000 7f4afd8d8640 1 --2- 172.21.15.172:0/661106019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486cb4a0 0x55a7486d5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:38.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.102+0000 7f4afd0d7640 1 --2- 172.21.15.172:0/661106019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486cbdc0 0x55a7486d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:38.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.102+0000 7f4afe0d9640 1 -- 172.21.15.172:0/661106019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486cb4a0 msgr2=0x55a7486d5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:38.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.102+0000 7f4afe0d9640 1 --2- 172.21.15.172:0/661106019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486cb4a0 0x55a7486d5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.102+0000 7f4afe0d9640 1 -- 172.21.15.172:0/661106019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486cbdc0 msgr2=0x55a7486d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:38.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.102+0000 7f4afe0d9640 1 --2- 172.21.15.172:0/661106019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486cbdc0 0x55a7486d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.102+0000 7f4afe0d9640 1 -- 172.21.15.172:0/661106019 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7483a23c0 con 0x55a7486d6680 2024-09-10T08:48:38.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.103+0000 7f4afe0d9640 1 --2- 172.21.15.172:0/661106019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486d6680 0x55a7486d37e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4af000aba0 tx=0x7f4af0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4677ad9b9caee5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:38.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.103+0000 7f4ae17fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:38.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.103+0000 7f4ae17fa640 1 -- 172.21.15.172:0/661106019 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af001eb30 con 0x55a7486d6680 2024-09-10T08:48:38.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.103+0000 7f4b01f15740 1 -- 172.21.15.172:0/661106019 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ae8004030 con 0x55a7486d6680 2024-09-10T08:48:38.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.103+0000 7f4ae17fa640 1 -- 172.21.15.172:0/661106019 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4af001ecd0 con 0x55a7486d6680 2024-09-10T08:48:38.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.103+0000 7f4ae17fa640 1 -- 172.21.15.172:0/661106019 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4af0033050 con 0x55a7486d6680 2024-09-10T08:48:38.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.103+0000 7f4b01f15740 10 client.16218 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:38.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.103+0000 7f4b01f15740 1 -- 172.21.15.172:0/661106019 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7486e4c70 con 0x55a7486d6680 2024-09-10T08:48:38.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.103+0000 7f4ae17fa640 1 -- 172.21.15.172:0/661106019 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ae8004030 con 0x55a7486d6680 2024-09-10T08:48:38.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.103+0000 7f4ae17fa640 1 -- 172.21.15.172:0/661106019 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4af000f040 con 0x55a7486d6680 2024-09-10T08:48:38.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.103+0000 7f4b01f15740 10 client.16218 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:38.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.103+0000 7f4b01f15740 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:38.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.103+0000 7f4b01f15740 20 client.16218 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:38.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.103+0000 7f4b01f15740 1 --2- 172.21.15.172:0/661106019 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a7486e6180 0x55a7486e8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.103+0000 7f4b01f15740 4 client.16218 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:38.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.103+0000 7f4b01f15740 1 -- 172.21.15.172:0/661106019 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a74863e850 con 0x55a7486e6180 2024-09-10T08:48:38.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.104+0000 7f4afd8d8640 1 --2- 172.21.15.172:0/661106019 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a7486e6180 0x55a7486e8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:38.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.104+0000 7f4ae17fa640 10 client.16218 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:38.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.104+0000 7f4afd8d8640 1 --2- 172.21.15.172:0/661106019 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a7486e6180 0x55a7486e8620 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48f3ef72ea0c61ac server_cookie=fa195f6d472e70d4 in_seq=0 out_seq=0 2024-09-10T08:48:38.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.104+0000 7f4ae17fa640 1 -- 172.21.15.172:0/661106019 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a74863e850 con 0x55a7486e6180 2024-09-10T08:48:38.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.104+0000 7f4ae17fa640 10 client.16218 handle_command_reply: tid=0 2024-09-10T08:48:38.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.104+0000 7f4b01f15740 1 client.16218 shutdown 2024-09-10T08:48:38.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.105+0000 7f4b01f15740 1 -- 172.21.15.172:0/661106019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486d6680 msgr2=0x55a7486d37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:38.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.105+0000 7f4b01f15740 1 --2- 172.21.15.172:0/661106019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486d6680 0x55a7486d37e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4af000aba0 tx=0x7f4af0004ab0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.105+0000 7f4b01f15740 1 -- 172.21.15.172:0/661106019 shutdown_connections 2024-09-10T08:48:38.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.105+0000 7f4b01f15740 1 --2- 172.21.15.172:0/661106019 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a7486e6180 0x55a7486e8620 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.105+0000 7f4b01f15740 1 --2- 172.21.15.172:0/661106019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486d6680 0x55a7486d37e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.105+0000 7f4b01f15740 1 --2- 172.21.15.172:0/661106019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486cbdc0 0x55a7486d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.105+0000 7f4b01f15740 1 --2- 172.21.15.172:0/661106019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486cb4a0 0x55a7486d5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.105+0000 7f4ae17fa640 0 client.16218 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:38.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.105+0000 7f4b01f15740 1 -- 172.21.15.172:0/661106019 >> 172.21.15.172:0/661106019 conn(0x55a7486c7a30 msgr2=0x55a7486cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:38.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.105+0000 7f4b01f15740 1 -- 172.21.15.172:0/661106019 shutdown_connections 2024-09-10T08:48:38.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.105+0000 7f4b01f15740 1 -- 172.21.15.172:0/661106019 wait complete. 2024-09-10T08:48:38.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.105+0000 7f4b01f15740 20 client.16218 trim_cache size 0 max 16384 2024-09-10T08:48:38.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4b01f15740 1 Processor -- start 2024-09-10T08:48:38.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4b01f15740 1 -- start start 2024-09-10T08:48:38.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4b01f15740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486cb4a0 0x55a7486d51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4b01f15740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486cbdc0 0x55a7486edef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4b01f15740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486d6680 0x55a7486ee430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4b01f15740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4af81b0d20 con 0x55a7486cbdc0 2024-09-10T08:48:38.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4b01f15740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4af81b2860 con 0x55a7486cb4a0 2024-09-10T08:48:38.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4b01f15740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4af81b13f0 con 0x55a7486d6680 2024-09-10T08:48:38.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4afd0d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486cbdc0 0x55a7486edef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:38.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4afd0d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486cbdc0 0x55a7486edef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37566/0 (socket says 172.21.15.172:37566) 2024-09-10T08:48:38.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4afd0d7640 1 -- 172.21.15.172:0/1484175052 learned_addr learned my addr 172.21.15.172:0/1484175052 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:38.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4afd8d8640 1 --2- 172.21.15.172:0/1484175052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486cb4a0 0x55a7486d51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:38.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4afd0d7640 1 -- 172.21.15.172:0/1484175052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486d6680 msgr2=0x55a7486ee430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:38.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4afd0d7640 1 --2- 172.21.15.172:0/1484175052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486d6680 0x55a7486ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4afd0d7640 1 -- 172.21.15.172:0/1484175052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486cb4a0 msgr2=0x55a7486d51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:38.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4afd0d7640 1 --2- 172.21.15.172:0/1484175052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486cb4a0 0x55a7486d51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4afd0d7640 1 -- 172.21.15.172:0/1484175052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7486cbbf0 con 0x55a7486cbdc0 2024-09-10T08:48:38.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.122+0000 7f4afd0d7640 1 --2- 172.21.15.172:0/1484175052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486cbdc0 0x55a7486edef0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8033000 tx=0x7f4ae80051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8944a7db4fbfeea6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:38.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.123+0000 7f4ae37fe640 1 -- 172.21.15.172:0/1484175052 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae804c070 con 0x55a7486cbdc0 2024-09-10T08:48:38.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.123+0000 7f4ae37fe640 1 -- 172.21.15.172:0/1484175052 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ae80314a0 con 0x55a7486cbdc0 2024-09-10T08:48:38.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.123+0000 7f4ae37fe640 1 -- 172.21.15.172:0/1484175052 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8040070 con 0x55a7486cbdc0 2024-09-10T08:48:38.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.123+0000 7f4b01f15740 1 -- 172.21.15.172:0/1484175052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486cbdc0 msgr2=0x55a7486edef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:38.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.123+0000 7f4b01f15740 1 --2- 172.21.15.172:0/1484175052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486cbdc0 0x55a7486edef0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8033000 tx=0x7f4ae80051c0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.123+0000 7f4b01f15740 1 -- 172.21.15.172:0/1484175052 shutdown_connections 2024-09-10T08:48:38.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.123+0000 7f4b01f15740 1 --2- 172.21.15.172:0/1484175052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486d6680 0x55a7486ee430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.123+0000 7f4b01f15740 1 --2- 172.21.15.172:0/1484175052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486cbdc0 0x55a7486edef0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.123+0000 7f4b01f15740 1 --2- 172.21.15.172:0/1484175052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486cb4a0 0x55a7486d51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.123+0000 7f4b01f15740 1 -- 172.21.15.172:0/1484175052 >> 172.21.15.172:0/1484175052 conn(0x55a7486c7a30 msgr2=0x55a7486d4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:38.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.123+0000 7f4b01f15740 1 -- 172.21.15.172:0/1484175052 shutdown_connections 2024-09-10T08:48:38.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.123+0000 7f4b01f15740 1 -- 172.21.15.172:0/1484175052 wait complete. 2024-09-10T08:48:38.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.124+0000 7f4b01f15740 1 -- start start 2024-09-10T08:48:38.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.124+0000 7f4b01f15740 1 Processor -- start 2024-09-10T08:48:38.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.124+0000 7f4b01f15740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486cb4a0 0x55a7486deed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.124+0000 7f4b01f15740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486cbdc0 0x55a7486dc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.124+0000 7f4b01f15740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486d6680 0x55a7486dc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.124+0000 7f4b01f15740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4af81b2190 con 0x55a7486cbdc0 2024-09-10T08:48:38.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.124+0000 7f4b01f15740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4af81b1ac0 con 0x55a7486d6680 2024-09-10T08:48:38.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.124+0000 7f4b01f15740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4af81aed00 con 0x55a7486cb4a0 2024-09-10T08:48:38.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.124+0000 7f4b01f15740 10 client.0 _ll_register_callbacks cb 0x7f4af00127a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:38.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.124+0000 7f4afd0d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486cbdc0 0x55a7486dc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:38.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.124+0000 7f4afd0d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486cbdc0 0x55a7486dc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37576/0 (socket says 172.21.15.172:37576) 2024-09-10T08:48:38.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.124+0000 7f4afd0d7640 1 -- 172.21.15.172:0/2788709470 learned_addr learned my addr 172.21.15.172:0/2788709470 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:38.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.124+0000 7f4afe0d9640 1 --2- 172.21.15.172:0/2788709470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486d6680 0x55a7486dc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:38.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.124+0000 7f4afd8d8640 1 --2- 172.21.15.172:0/2788709470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486cb4a0 0x55a7486deed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:38.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.125+0000 7f4afd0d7640 1 -- 172.21.15.172:0/2788709470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486cb4a0 msgr2=0x55a7486deed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:38.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.125+0000 7f4afd0d7640 1 --2- 172.21.15.172:0/2788709470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486cb4a0 0x55a7486deed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.125+0000 7f4afd0d7640 1 -- 172.21.15.172:0/2788709470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486d6680 msgr2=0x55a7486dc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:38.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.125+0000 7f4afd0d7640 1 --2- 172.21.15.172:0/2788709470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486d6680 0x55a7486dc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.125+0000 7f4afd0d7640 1 -- 172.21.15.172:0/2788709470 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a74874fed0 con 0x55a7486cbdc0 2024-09-10T08:48:38.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.125+0000 7f4afd0d7640 1 --2- 172.21.15.172:0/2788709470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486cbdc0 0x55a7486dc220 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8005230 tx=0x7f4ae8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57dbfd6178418367 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:38.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.125+0000 7f4ae17fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:38.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.125+0000 7f4ae17fa640 1 -- 172.21.15.172:0/2788709470 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae804c070 con 0x55a7486cbdc0 2024-09-10T08:48:38.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.125+0000 7f4ae17fa640 1 -- 172.21.15.172:0/2788709470 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4ae8005460 con 0x55a7486cbdc0 2024-09-10T08:48:38.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.125+0000 7f4b01f15740 1 -- 172.21.15.172:0/2788709470 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ae80314a0 con 0x55a7486cbdc0 2024-09-10T08:48:38.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.125+0000 7f4ae17fa640 1 -- 172.21.15.172:0/2788709470 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ae8059050 con 0x55a7486cbdc0 2024-09-10T08:48:38.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.125+0000 7f4b01f15740 10 client.16224 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:38.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.125+0000 7f4b01f15740 1 -- 172.21.15.172:0/2788709470 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a748751040 con 0x55a7486cbdc0 2024-09-10T08:48:38.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.125+0000 7f4ae17fa640 1 -- 172.21.15.172:0/2788709470 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ae80314a0 con 0x55a7486cbdc0 2024-09-10T08:48:38.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.125+0000 7f4ae17fa640 1 -- 172.21.15.172:0/2788709470 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4ae8003cb0 con 0x55a7486cbdc0 2024-09-10T08:48:38.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.126+0000 7f4b01f15740 10 client.16224 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:38.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.126+0000 7f4b01f15740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:38.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.126+0000 7f4b01f15740 20 client.16224 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:38.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.126+0000 7f4b01f15740 1 --2- 172.21.15.172:0/2788709470 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a7487512c0 0x55a7487536b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:38.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.126+0000 7f4b01f15740 4 client.16224 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:48:38.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.126+0000 7f4b01f15740 1 -- 172.21.15.172:0/2788709470 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a748638d60 con 0x55a7487512c0 2024-09-10T08:48:38.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.126+0000 7f4afd8d8640 1 --2- 172.21.15.172:0/2788709470 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a7487512c0 0x55a7487536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:38.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.126+0000 7f4afd8d8640 1 --2- 172.21.15.172:0/2788709470 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a7487512c0 0x55a7487536b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28a22214364ec2c2 server_cookie=23d216125c063f3e in_seq=0 out_seq=0 2024-09-10T08:48:38.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.126+0000 7f4ae17fa640 10 client.16224 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:38.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.126+0000 7f4ae17fa640 1 -- 172.21.15.172:0/2788709470 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a748638d60 con 0x55a7487512c0 2024-09-10T08:48:38.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.126+0000 7f4ae17fa640 10 client.16224 handle_command_reply: tid=0 2024-09-10T08:48:38.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.126+0000 7f4b01f15740 1 client.16224 shutdown 2024-09-10T08:48:38.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.126+0000 7f4b01f15740 1 -- 172.21.15.172:0/2788709470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486cbdc0 msgr2=0x55a7486dc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:38.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.126+0000 7f4b01f15740 1 --2- 172.21.15.172:0/2788709470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486cbdc0 0x55a7486dc220 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8005230 tx=0x7f4ae8031400 comp rx=0 tx=0).stop 2024-09-10T08:48:38.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4b01f15740 1 -- 172.21.15.172:0/2788709470 shutdown_connections 2024-09-10T08:48:38.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4b01f15740 1 --2- 172.21.15.172:0/2788709470 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a7487512c0 0x55a7487536b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4b01f15740 1 --2- 172.21.15.172:0/2788709470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7486d6680 0x55a7486dc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4b01f15740 1 --2- 172.21.15.172:0/2788709470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7486cbdc0 0x55a7486dc220 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4b01f15740 1 --2- 172.21.15.172:0/2788709470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7486cb4a0 0x55a7486deed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4ae17fa640 0 client.16224 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:38.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4b01f15740 1 -- 172.21.15.172:0/2788709470 >> 172.21.15.172:0/2788709470 conn(0x55a7486c7a30 msgr2=0x55a7486d5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:38.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4b01f15740 1 -- 172.21.15.172:0/2788709470 shutdown_connections 2024-09-10T08:48:38.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4b01f15740 1 -- 172.21.15.172:0/2788709470 wait complete. 2024-09-10T08:48:38.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4b01f15740 20 client.16224 trim_cache size 0 max 16384 2024-09-10T08:48:38.137 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:48:38.137 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:48:38.137 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T08:48:38.137 INFO:teuthology.orchestra.run.smithi172.stdout: "d6139d6e-8fbd-440e-ac9b-cb284f00390b": { 2024-09-10T08:48:38.138 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T08:48:38.138 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "d6139d6e-8fbd-440e-ac9b-cb284f00390b", 2024-09-10T08:48:38.138 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T08:48:38.138 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:48:38.138 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:48:38.138 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:48:38.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4affb3c640 1 -- 172.21.15.172:0/843603895 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4ae40a2c00 msgr2=0x7f4ae40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:38.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4affb3c640 1 --2- 172.21.15.172:0/843603895 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4ae40a2c00 0x7f4ae40a50c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4af0005fd0 tx=0x7f4af0009600 comp rx=0 tx=0).stop 2024-09-10T08:48:38.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4affb3c640 1 -- 172.21.15.172:0/843603895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4af8074620 msgr2=0x7f4af819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:38.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4affb3c640 1 --2- 172.21.15.172:0/843603895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4af8074620 0x7f4af819a630 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4af400e9f0 tx=0x7f4af400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4affb3c640 1 -- 172.21.15.172:0/843603895 shutdown_connections 2024-09-10T08:48:38.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4affb3c640 1 --2- 172.21.15.172:0/843603895 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4ae40a2c00 0x7f4ae40a50c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4affb3c640 1 --2- 172.21.15.172:0/843603895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4af810e3c0 0x7f4af819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4affb3c640 1 --2- 172.21.15.172:0/843603895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4af8074f40 0x7f4af819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4affb3c640 1 --2- 172.21.15.172:0/843603895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4af8074620 0x7f4af819a630 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:38.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4affb3c640 1 -- 172.21.15.172:0/843603895 >> 172.21.15.172:0/843603895 conn(0x7f4af80fd8c0 msgr2=0x7f4af80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:38.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4affb3c640 1 -- 172.21.15.172:0/843603895 shutdown_connections 2024-09-10T08:48:38.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:48:38.127+0000 7f4affb3c640 1 -- 172.21.15.172:0/843603895 wait complete. 2024-09-10T08:48:38.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:38 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T08:48:38.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:38 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:48:38.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:38 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:48:38.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:38 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:48:38.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:38 smithi172 ceph-mon[30854]: pgmap v422: 129 pgs: 129 active+clean; 4.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 388 KiB/s wr, 108 op/s 2024-09-10T08:48:38.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:38 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T08:48:38.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:38 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:48:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:38 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:48:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:38 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:48:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:38 smithi174 ceph-mon[34068]: pgmap v422: 129 pgs: 129 active+clean; 4.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 388 KiB/s wr, 108 op/s 2024-09-10T08:48:38.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:38 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T08:48:38.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:38 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:48:38.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:38 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:48:38.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:38 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:48:38.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:38 smithi191 ceph-mon[35314]: pgmap v422: 129 pgs: 129 active+clean; 4.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 388 KiB/s wr, 108 op/s 2024-09-10T08:48:38.933 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether build environment is sane... yes 2024-09-10T08:48:38.942 INFO:tasks.workunit.client.0.smithi172.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-10T08:48:38.943 INFO:tasks.workunit.client.0.smithi172.stdout:checking for gawk... gawk 2024-09-10T08:48:38.968 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether make sets $(MAKE)... yes 2024-09-10T08:48:39.010 INFO:tasks.workunit.client.0.smithi172.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-10T08:48:39.010 INFO:tasks.workunit.client.0.smithi172.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-10T08:48:39.032 INFO:tasks.workunit.client.0.smithi172.stdout:checking for style of include used by make... GNU 2024-09-10T08:48:39.037 INFO:tasks.workunit.client.0.smithi172.stdout:checking for gcc... gcc 2024-09-10T08:48:39.162 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether the C compiler works... yes 2024-09-10T08:48:39.162 INFO:tasks.workunit.client.0.smithi172.stdout:checking for C compiler default output file name... a.out 2024-09-10T08:48:39.213 INFO:tasks.workunit.client.0.smithi172.stdout:checking for suffix of executables... 2024-09-10T08:48:39.255 INFO:tasks.cephfs.filesystem:scrub status for tag:d6139d6e-8fbd-440e-ac9b-cb284f00390b - {'path': '/', 'tag': 'd6139d6e-8fbd-440e-ac9b-cb284f00390b', 'options': 'recursive,force'} 2024-09-10T08:48:39.274 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether we are cross compiling... no 2024-09-10T08:48:39.320 INFO:tasks.workunit.client.0.smithi172.stdout:checking for suffix of object files... o 2024-09-10T08:48:39.352 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether we are using the GNU C compiler... yes 2024-09-10T08:48:39.384 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether gcc accepts -g... yes 2024-09-10T08:48:39.431 INFO:tasks.workunit.client.0.smithi172.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-10T08:48:39.577 INFO:tasks.workunit.client.0.smithi172.stdout:checking dependency style of gcc... gcc3 2024-09-10T08:48:39.640 INFO:tasks.workunit.client.0.smithi172.stdout:checking how to run the C preprocessor... gcc -E 2024-09-10T08:48:39.703 INFO:tasks.workunit.client.0.smithi172.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-10T08:48:39.705 INFO:tasks.workunit.client.0.smithi172.stdout:checking for egrep... /usr/bin/grep -E 2024-09-10T08:48:39.870 INFO:tasks.workunit.client.0.smithi172.stdout:checking for ANSI C header files... yes 2024-09-10T08:48:39.914 INFO:tasks.workunit.client.0.smithi172.stdout:checking for sys/types.h... yes 2024-09-10T08:48:39.965 INFO:tasks.workunit.client.0.smithi172.stdout:checking for sys/stat.h... yes 2024-09-10T08:48:40.017 INFO:tasks.workunit.client.0.smithi172.stdout:checking for stdlib.h... yes 2024-09-10T08:48:40.070 INFO:tasks.workunit.client.0.smithi172.stdout:checking for string.h... yes 2024-09-10T08:48:40.122 INFO:tasks.workunit.client.0.smithi172.stdout:checking for memory.h... yes 2024-09-10T08:48:40.176 INFO:tasks.workunit.client.0.smithi172.stdout:checking for strings.h... yes 2024-09-10T08:48:40.237 INFO:tasks.workunit.client.0.smithi172.stdout:checking for inttypes.h... yes 2024-09-10T08:48:40.307 INFO:tasks.workunit.client.0.smithi172.stdout:checking for stdint.h... yes 2024-09-10T08:48:40.362 INFO:tasks.workunit.client.0.smithi172.stdout:checking for unistd.h... yes 2024-09-10T08:48:40.417 INFO:tasks.workunit.client.0.smithi172.stdout:checking minix/config.h usability... no 2024-09-10T08:48:40.455 INFO:tasks.workunit.client.0.smithi172.stdout:checking minix/config.h presence... no 2024-09-10T08:48:40.455 INFO:tasks.workunit.client.0.smithi172.stdout:checking for minix/config.h... no 2024-09-10T08:48:40.498 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-09-10T08:48:40.507 INFO:tasks.workunit.client.0.smithi172.stdout:checking for gcc... (cached) gcc 2024-09-10T08:48:40.557 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-10T08:48:40.557 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-10T08:48:40.558 INFO:tasks.workunit.client.0.smithi172.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-10T08:48:40.558 INFO:tasks.workunit.client.0.smithi172.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-10T08:48:40.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:40 smithi172 ceph-mon[30854]: pgmap v423: 129 pgs: 129 active+clean; 4.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 395 KiB/s wr, 115 op/s 2024-09-10T08:48:40.617 INFO:tasks.workunit.client.0.smithi172.stdout:checking for main in -lm... yes 2024-09-10T08:48:40.679 INFO:tasks.workunit.client.0.smithi172.stdout:checking for main in -lpthread... yes 2024-09-10T08:48:40.683 INFO:tasks.workunit.client.0.smithi172.stdout:checking for ANSI C header files... (cached) yes 2024-09-10T08:48:40.728 INFO:tasks.workunit.client.0.smithi172.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-10T08:48:40.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:40 smithi174 ceph-mon[34068]: pgmap v423: 129 pgs: 129 active+clean; 4.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 395 KiB/s wr, 115 op/s 2024-09-10T08:48:40.783 INFO:tasks.workunit.client.0.smithi172.stdout:checking pthread.h usability... yes 2024-09-10T08:48:40.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:40 smithi191 ceph-mon[35314]: pgmap v423: 129 pgs: 129 active+clean; 4.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 395 KiB/s wr, 115 op/s 2024-09-10T08:48:40.811 INFO:tasks.workunit.client.0.smithi172.stdout:checking pthread.h presence... yes 2024-09-10T08:48:40.811 INFO:tasks.workunit.client.0.smithi172.stdout:checking for pthread.h... yes 2024-09-10T08:48:40.877 INFO:tasks.workunit.client.0.smithi172.stdout:checking fcntl.h usability... yes 2024-09-10T08:48:40.899 INFO:tasks.workunit.client.0.smithi172.stdout:checking fcntl.h presence... yes 2024-09-10T08:48:40.899 INFO:tasks.workunit.client.0.smithi172.stdout:checking for fcntl.h... yes 2024-09-10T08:48:40.970 INFO:tasks.workunit.client.0.smithi172.stdout:checking limits.h usability... yes 2024-09-10T08:48:40.992 INFO:tasks.workunit.client.0.smithi172.stdout:checking limits.h presence... yes 2024-09-10T08:48:40.992 INFO:tasks.workunit.client.0.smithi172.stdout:checking for limits.h... yes 2024-09-10T08:48:41.012 INFO:tasks.workunit.client.0.smithi172.stdout:checking for stdint.h... (cached) yes 2024-09-10T08:48:41.063 INFO:tasks.workunit.client.0.smithi172.stdout:checking sys/time.h usability... yes 2024-09-10T08:48:41.085 INFO:tasks.workunit.client.0.smithi172.stdout:checking sys/time.h presence... yes 2024-09-10T08:48:41.085 INFO:tasks.workunit.client.0.smithi172.stdout:checking for sys/time.h... yes 2024-09-10T08:48:41.104 INFO:tasks.workunit.client.0.smithi172.stdout:checking for unistd.h... (cached) yes 2024-09-10T08:48:41.160 INFO:tasks.workunit.client.0.smithi172.stdout:checking sys/vfs.h usability... yes 2024-09-10T08:48:41.183 INFO:tasks.workunit.client.0.smithi172.stdout:checking sys/vfs.h presence... yes 2024-09-10T08:48:41.183 INFO:tasks.workunit.client.0.smithi172.stdout:checking for sys/vfs.h... yes 2024-09-10T08:48:41.299 INFO:tasks.workunit.client.0.smithi172.stdout:checking sys/limits.h usability... no 2024-09-10T08:48:41.342 INFO:tasks.workunit.client.0.smithi172.stdout:checking sys/limits.h presence... no 2024-09-10T08:48:41.342 INFO:tasks.workunit.client.0.smithi172.stdout:checking for sys/limits.h... no 2024-09-10T08:48:41.376 INFO:tasks.workunit.client.0.smithi172.stdout:checking for an ANSI C-conforming const... yes 2024-09-10T08:48:41.413 INFO:tasks.workunit.client.0.smithi172.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-10T08:48:41.494 INFO:tasks.workunit.client.0.smithi172.stdout:checking for system... yes 2024-09-10T08:48:41.570 INFO:tasks.workunit.client.0.smithi172.stdout:checking for gettimeofday... yes 2024-09-10T08:48:41.646 INFO:tasks.workunit.client.0.smithi172.stdout:checking for mkdir... yes 2024-09-10T08:48:41.722 INFO:tasks.workunit.client.0.smithi172.stdout:checking for strerror... yes 2024-09-10T08:48:41.801 INFO:tasks.workunit.client.0.smithi172.stdout:checking for open64... yes 2024-09-10T08:48:41.900 INFO:tasks.workunit.client.0.smithi172.stdout:checking for stat64... yes 2024-09-10T08:48:41.983 INFO:tasks.workunit.client.0.smithi172.stdout:checking for fseeko64... yes 2024-09-10T08:48:42.060 INFO:tasks.workunit.client.0.smithi172.stdout:checking for lrand48_r... yes 2024-09-10T08:48:42.137 INFO:tasks.workunit.client.0.smithi172.stdout:checking for srand48_r... yes 2024-09-10T08:48:42.165 INFO:tasks.workunit.client.0.smithi172.stdout:configure: creating ./config.status 2024-09-10T08:48:42.264 INFO:tasks.workunit.client.0.smithi172.stdout: /bin/sh ./config.status 2024-09-10T08:48:42.325 INFO:tasks.workunit.client.0.smithi172.stdout:config.status: creating Makefile 2024-09-10T08:48:42.344 INFO:tasks.workunit.client.0.smithi172.stdout:config.status: creating config.h 2024-09-10T08:48:42.367 INFO:tasks.workunit.client.0.smithi172.stdout:config.status: executing depfiles commands 2024-09-10T08:48:42.386 INFO:tasks.workunit.client.0.smithi172.stdout:config.status: executing default commands 2024-09-10T08:48:42.431 INFO:tasks.workunit.client.0.smithi172.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-09-10T08:48:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:42 smithi172 ceph-mon[30854]: pgmap v424: 129 pgs: 129 active+clean; 6.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 623 KiB/s wr, 151 op/s 2024-09-10T08:48:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:42 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:48:42.590 INFO:tasks.workunit.client.0.smithi172.stdout:rm -f stamp-h1 2024-09-10T08:48:42.593 INFO:tasks.workunit.client.0.smithi172.stdout:touch config.h.in 2024-09-10T08:48:42.595 INFO:tasks.workunit.client.0.smithi172.stdout:cd . && /bin/sh ./config.status config.h 2024-09-10T08:48:42.648 INFO:tasks.workunit.client.0.smithi172.stdout:config.status: creating config.h 2024-09-10T08:48:42.684 INFO:tasks.workunit.client.0.smithi172.stdout:make all-am 2024-09-10T08:48:42.708 INFO:tasks.workunit.client.0.smithi172.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-10T08:48:42.708 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-09-10T08:48:42.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:42 smithi174 ceph-mon[34068]: pgmap v424: 129 pgs: 129 active+clean; 6.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 623 KiB/s wr, 151 op/s 2024-09-10T08:48:42.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:42 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:48:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:42 smithi191 ceph-mon[35314]: pgmap v424: 129 pgs: 129 active+clean; 6.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 623 KiB/s wr, 151 op/s 2024-09-10T08:48:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:42 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:48:42.920 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-09-10T08:48:42.925 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-09-10T08:48:43.030 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-09-10T08:48:43.035 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-09-10T08:48:43.149 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-09-10T08:48:43.154 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-09-10T08:48:43.267 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-09-10T08:48:43.273 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-09-10T08:48:43.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.294+0000 7ffb23cc9640 1 -- 172.21.15.172:0/3237290274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb1c100a40 msgr2=0x7ffb1c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:43.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.294+0000 7ffb23cc9640 1 --2- 172.21.15.172:0/3237290274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb1c100a40 0x7ffb1c100e40 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009900 tx=0x7ffb1802a410 comp rx=0 tx=0).stop 2024-09-10T08:48:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.294+0000 7ffb23cc9640 1 -- 172.21.15.172:0/3237290274 shutdown_connections 2024-09-10T08:48:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.294+0000 7ffb23cc9640 1 --2- 172.21.15.172:0/3237290274 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb1c1025f0 0x7ffb1c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.294+0000 7ffb23cc9640 1 --2- 172.21.15.172:0/3237290274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb1c101c30 0x7ffb1c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.294+0000 7ffb23cc9640 1 --2- 172.21.15.172:0/3237290274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb1c100a40 0x7ffb1c100e40 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.294+0000 7ffb23cc9640 1 -- 172.21.15.172:0/3237290274 >> 172.21.15.172:0/3237290274 conn(0x7ffb1c0fc1e0 msgr2=0x7ffb1c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.295+0000 7ffb23cc9640 1 -- 172.21.15.172:0/3237290274 shutdown_connections 2024-09-10T08:48:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.295+0000 7ffb23cc9640 1 -- 172.21.15.172:0/3237290274 wait complete. 2024-09-10T08:48:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.295+0000 7ffb23cc9640 1 Processor -- start 2024-09-10T08:48:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.295+0000 7ffb23cc9640 1 -- start start 2024-09-10T08:48:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.295+0000 7ffb23cc9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb1c100a40 0x7ffb1c196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.295+0000 7ffb23cc9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb1c101c30 0x7ffb1c197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.295+0000 7ffb23cc9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb1c1025f0 0x7ffb1c19e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.295+0000 7ffb23cc9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c10c550 con 0x7ffb1c1025f0 2024-09-10T08:48:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.295+0000 7ffb23cc9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c10c3d0 con 0x7ffb1c101c30 2024-09-10T08:48:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.295+0000 7ffb23cc9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c10c6d0 con 0x7ffb1c100a40 2024-09-10T08:48:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.295+0000 7ffb22266640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb1c1025f0 0x7ffb1c19e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.295+0000 7ffb22266640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb1c1025f0 0x7ffb1c19e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37584/0 (socket says 172.21.15.172:37584) 2024-09-10T08:48:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.295+0000 7ffb22266640 1 -- 172.21.15.172:0/3689648462 learned_addr learned my addr 172.21.15.172:0/3689648462 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.295+0000 7ffb21264640 1 --2- 172.21.15.172:0/3689648462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb1c101c30 0x7ffb1c197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.295+0000 7ffb21a65640 1 --2- 172.21.15.172:0/3689648462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb1c100a40 0x7ffb1c196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.296+0000 7ffb22266640 1 -- 172.21.15.172:0/3689648462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb1c100a40 msgr2=0x7ffb1c196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.296+0000 7ffb22266640 1 --2- 172.21.15.172:0/3689648462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb1c100a40 0x7ffb1c196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.296+0000 7ffb22266640 1 -- 172.21.15.172:0/3689648462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb1c101c30 msgr2=0x7ffb1c197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.296+0000 7ffb22266640 1 --2- 172.21.15.172:0/3689648462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb1c101c30 0x7ffb1c197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.296+0000 7ffb22266640 1 -- 172.21.15.172:0/3689648462 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1c19e830 con 0x7ffb1c1025f0 2024-09-10T08:48:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.296+0000 7ffb22266640 1 --2- 172.21.15.172:0/3689648462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb1c1025f0 0x7ffb1c19e190 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400eaa0 tx=0x7ffb1400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.296+0000 7ffb12ffd640 1 -- 172.21.15.172:0/3689648462 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1400cdb0 con 0x7ffb1c1025f0 2024-09-10T08:48:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.296+0000 7ffb12ffd640 1 -- 172.21.15.172:0/3689648462 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb14005020 con 0x7ffb1c1025f0 2024-09-10T08:48:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.296+0000 7ffb23cc9640 1 -- 172.21.15.172:0/3689648462 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1c19eb20 con 0x7ffb1c1025f0 2024-09-10T08:48:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.296+0000 7ffb12ffd640 1 -- 172.21.15.172:0/3689648462 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb140116d0 con 0x7ffb1c1025f0 2024-09-10T08:48:43.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.296+0000 7ffb23cc9640 1 -- 172.21.15.172:0/3689648462 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1c19f070 con 0x7ffb1c1025f0 2024-09-10T08:48:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.298+0000 7ffb12ffd640 1 -- 172.21.15.172:0/3689648462 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ffb140051d0 con 0x7ffb1c1025f0 2024-09-10T08:48:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb26a61740 1 Processor -- start 2024-09-10T08:48:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb26a61740 1 -- start start 2024-09-10T08:48:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb26a61740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99ebb0 0x55f78c99efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb26a61740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c99f4f0 0x55f78c9a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb26a61740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c9a4580 0x55f78c9a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb26a61740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f78c9002f0 con 0x55f78c99f4f0 2024-09-10T08:48:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb26a61740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f78c8a8270 con 0x55f78c99ebb0 2024-09-10T08:48:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb26a61740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f78c8fc2b0 con 0x55f78c9a4580 2024-09-10T08:48:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb21a65640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c99f4f0 0x55f78c9a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb21a65640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c99f4f0 0x55f78c9a3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37590/0 (socket says 172.21.15.172:37590) 2024-09-10T08:48:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb21a65640 1 -- 172.21.15.172:0/2847574663 learned_addr learned my addr 172.21.15.172:0/2847574663 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb21264640 1 --2- 172.21.15.172:0/2847574663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99ebb0 0x55f78c99efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb12ffd640 1 --2- 172.21.15.172:0/3689648462 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffb080a2cb0 0x7ffb080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb21264640 1 --2- 172.21.15.172:0/2847574663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c9a4580 0x55f78c9a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb21a65640 1 -- 172.21.15.172:0/2847574663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c9a4580 msgr2=0x55f78c9a6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb21a65640 1 --2- 172.21.15.172:0/2847574663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c9a4580 0x55f78c9a6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb21a65640 1 -- 172.21.15.172:0/2847574663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99ebb0 msgr2=0x55f78c99efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb21a65640 1 --2- 172.21.15.172:0/2847574663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99ebb0 0x55f78c99efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.299+0000 7ffb21a65640 1 -- 172.21.15.172:0/2847574663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f78c8a7340 con 0x55f78c99f4f0 2024-09-10T08:48:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.300+0000 7ffb22266640 1 --2- 172.21.15.172:0/3689648462 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffb080a2cb0 0x7ffb080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.300+0000 7ffb21264640 1 --2- 172.21.15.172:0/2847574663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c9a4580 0x55f78c9a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:48:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.300+0000 7ffb12ffd640 1 -- 172.21.15.172:0/3689648462 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ffb140cf5d0 con 0x7ffb1c1025f0 2024-09-10T08:48:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.300+0000 7ffb21a65640 1 --2- 172.21.15.172:0/2847574663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c99f4f0 0x55f78c9a3fb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ffb180314e0 tx=0x7ffb18032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417d175df11f250 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.300+0000 7ffb07fff640 1 -- 172.21.15.172:0/2847574663 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1803f070 con 0x55f78c99f4f0 2024-09-10T08:48:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.300+0000 7ffb07fff640 1 -- 172.21.15.172:0/2847574663 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb18031780 con 0x55f78c99f4f0 2024-09-10T08:48:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.300+0000 7ffb07fff640 1 -- 172.21.15.172:0/2847574663 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb18035070 con 0x55f78c99f4f0 2024-09-10T08:48:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.301+0000 7ffb26a61740 1 -- 172.21.15.172:0/2847574663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c99f4f0 msgr2=0x55f78c9a3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.301+0000 7ffb22266640 1 --2- 172.21.15.172:0/3689648462 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffb080a2cb0 0x7ffb080a5170 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffb14084fd0 tx=0x7ffb140c6660 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.301+0000 7ffb26a61740 1 --2- 172.21.15.172:0/2847574663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c99f4f0 0x55f78c9a3fb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ffb180314e0 tx=0x7ffb18032cc0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.301+0000 7ffb26a61740 1 -- 172.21.15.172:0/2847574663 shutdown_connections 2024-09-10T08:48:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.301+0000 7ffb26a61740 1 --2- 172.21.15.172:0/2847574663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c9a4580 0x55f78c9a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.301+0000 7ffb26a61740 1 --2- 172.21.15.172:0/2847574663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c99f4f0 0x55f78c9a3fb0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.301+0000 7ffb26a61740 1 --2- 172.21.15.172:0/2847574663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99ebb0 0x55f78c99efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.301+0000 7ffb26a61740 1 -- 172.21.15.172:0/2847574663 >> 172.21.15.172:0/2847574663 conn(0x55f78c998010 msgr2=0x55f78c998410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:43.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.301+0000 7ffb26a61740 1 -- 172.21.15.172:0/2847574663 shutdown_connections 2024-09-10T08:48:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.301+0000 7ffb26a61740 1 -- 172.21.15.172:0/2847574663 wait complete. 2024-09-10T08:48:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.301+0000 7ffb26a61740 1 -- start start 2024-09-10T08:48:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.301+0000 7ffb26a61740 1 Processor -- start 2024-09-10T08:48:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.301+0000 7ffb26a61740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c99ebb0 0x55f78c9a8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb26a61740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99f4f0 0x55f78c9a92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb26a61740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c9b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb26a61740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f78c9a74f0 con 0x55f78c9a4580 2024-09-10T08:48:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb26a61740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f78c889290 con 0x55f78c99f4f0 2024-09-10T08:48:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb26a61740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f78c9a7670 con 0x55f78c99ebb0 2024-09-10T08:48:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb26a61740 10 client.0 _ll_register_callbacks cb 0x55f78c8ba140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb21264640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c9b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb21264640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c9b02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37602/0 (socket says 172.21.15.172:37602) 2024-09-10T08:48:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb21264640 1 -- 172.21.15.172:0/1333250211 learned_addr learned my addr 172.21.15.172:0/1333250211 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb21a65640 1 --2- 172.21.15.172:0/1333250211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99f4f0 0x55f78c9a92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb21264640 1 --2- 172.21.15.172:0/1333250211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c99ebb0 0x55f78c9a8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb21264640 1 -- 172.21.15.172:0/1333250211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c99ebb0 msgr2=0x55f78c9a8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb21264640 1 --2- 172.21.15.172:0/1333250211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c99ebb0 0x55f78c9a8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb21264640 1 -- 172.21.15.172:0/1333250211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99f4f0 msgr2=0x55f78c9a92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb21264640 1 --2- 172.21.15.172:0/1333250211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99f4f0 0x55f78c9a92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb21264640 1 -- 172.21.15.172:0/1333250211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f78c613210 con 0x55f78c9a4580 2024-09-10T08:48:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb21264640 1 --2- 172.21.15.172:0/1333250211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c9b02f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00a610 tx=0x7ffb0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b42cf3af194534 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb057fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb057fa640 1 -- 172.21.15.172:0/1333250211 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c010430 con 0x55f78c9a4580 2024-09-10T08:48:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb26a61740 1 -- 172.21.15.172:0/1333250211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb18031780 con 0x55f78c9a4580 2024-09-10T08:48:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb057fa640 1 -- 172.21.15.172:0/1333250211 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ffb0c0105d0 con 0x55f78c9a4580 2024-09-10T08:48:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb057fa640 1 -- 172.21.15.172:0/1333250211 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffb0c027050 con 0x55f78c9a4580 2024-09-10T08:48:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb26a61740 10 client.16236 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.302+0000 7ffb26a61740 1 -- 172.21.15.172:0/1333250211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f78c6df620 con 0x55f78c9a4580 2024-09-10T08:48:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.303+0000 7ffb057fa640 1 -- 172.21.15.172:0/1333250211 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb18031780 con 0x55f78c9a4580 2024-09-10T08:48:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.303+0000 7ffb057fa640 1 -- 172.21.15.172:0/1333250211 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ffb0c01f650 con 0x55f78c9a4580 2024-09-10T08:48:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.303+0000 7ffb26a61740 10 client.16236 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.303+0000 7ffb26a61740 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.303+0000 7ffb26a61740 20 client.16236 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.303+0000 7ffb26a61740 1 --2- 172.21.15.172:0/1333250211 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f78c9b49b0 0x55f78c9b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.303+0000 7ffb26a61740 4 client.16236 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.303+0000 7ffb26a61740 1 -- 172.21.15.172:0/1333250211 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f78c90da20 con 0x55f78c9b49b0 2024-09-10T08:48:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.303+0000 7ffb21a65640 1 --2- 172.21.15.172:0/1333250211 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f78c9b49b0 0x55f78c9b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.303+0000 7ffb057fa640 10 client.16236 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.303+0000 7ffb21a65640 1 --2- 172.21.15.172:0/1333250211 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f78c9b49b0 0x55f78c9b6e20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b544cbca9a353bb server_cookie=238f913e3ed35db2 in_seq=0 out_seq=0 2024-09-10T08:48:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.304+0000 7ffb057fa640 1 -- 172.21.15.172:0/1333250211 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f78c90da20 con 0x55f78c9b49b0 2024-09-10T08:48:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.304+0000 7ffb057fa640 10 client.16236 handle_command_reply: tid=0 2024-09-10T08:48:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.304+0000 7ffb26a61740 1 client.16236 shutdown 2024-09-10T08:48:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.304+0000 7ffb26a61740 1 -- 172.21.15.172:0/1333250211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 msgr2=0x55f78c9b02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.304+0000 7ffb26a61740 1 --2- 172.21.15.172:0/1333250211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c9b02f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00a610 tx=0x7ffb0c004730 comp rx=0 tx=0).stop 2024-09-10T08:48:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.304+0000 7ffb26a61740 1 -- 172.21.15.172:0/1333250211 shutdown_connections 2024-09-10T08:48:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.304+0000 7ffb26a61740 1 --2- 172.21.15.172:0/1333250211 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f78c9b49b0 0x55f78c9b6e20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.304+0000 7ffb26a61740 1 --2- 172.21.15.172:0/1333250211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c9b02f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.304+0000 7ffb057fa640 0 client.16236 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.304+0000 7ffb26a61740 1 --2- 172.21.15.172:0/1333250211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99f4f0 0x55f78c9a92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.304+0000 7ffb26a61740 1 --2- 172.21.15.172:0/1333250211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c99ebb0 0x55f78c9a8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.304+0000 7ffb26a61740 1 -- 172.21.15.172:0/1333250211 >> 172.21.15.172:0/1333250211 conn(0x55f78c998010 msgr2=0x55f78c999820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.304+0000 7ffb26a61740 1 -- 172.21.15.172:0/1333250211 shutdown_connections 2024-09-10T08:48:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.304+0000 7ffb26a61740 1 -- 172.21.15.172:0/1333250211 wait complete. 2024-09-10T08:48:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.305+0000 7ffb26a61740 20 client.16236 trim_cache size 0 max 16384 2024-09-10T08:48:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.320+0000 7ffb26a61740 1 Processor -- start 2024-09-10T08:48:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.320+0000 7ffb26a61740 1 -- start start 2024-09-10T08:48:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.320+0000 7ffb26a61740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99ebb0 0x55f78c9a8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.320+0000 7ffb26a61740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c99f4f0 0x55f78c9b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.320+0000 7ffb26a61740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c998ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.320+0000 7ffb26a61740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c1a8240 con 0x55f78c9a4580 2024-09-10T08:48:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.320+0000 7ffb26a61740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c1a9d80 con 0x55f78c99ebb0 2024-09-10T08:48:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.320+0000 7ffb26a61740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c1a8910 con 0x55f78c99f4f0 2024-09-10T08:48:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.321+0000 7ffb21264640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99ebb0 0x55f78c9a8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.321+0000 7ffb21264640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c998ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.321+0000 7ffb21264640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c998ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37614/0 (socket says 172.21.15.172:37614) 2024-09-10T08:48:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.321+0000 7ffb21264640 1 -- 172.21.15.172:0/3684263970 learned_addr learned my addr 172.21.15.172:0/3684263970 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.321+0000 7ffb21a65640 1 --2- 172.21.15.172:0/3684263970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c99f4f0 0x55f78c9b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.321+0000 7ffb21264640 1 -- 172.21.15.172:0/3684263970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c99f4f0 msgr2=0x55f78c9b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.321+0000 7ffb21264640 1 --2- 172.21.15.172:0/3684263970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c99f4f0 0x55f78c9b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.321+0000 7ffb21264640 1 -- 172.21.15.172:0/3684263970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99ebb0 msgr2=0x55f78c9a8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.321+0000 7ffb21264640 1 --2- 172.21.15.172:0/3684263970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99ebb0 0x55f78c9a8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.321+0000 7ffb21264640 1 -- 172.21.15.172:0/3684263970 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f78c9a7730 con 0x55f78c9a4580 2024-09-10T08:48:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.321+0000 7ffb21264640 1 --2- 172.21.15.172:0/3684263970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c998ec0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c01fb90 tx=0x7ffb0c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6641a6f7276c5988 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.321+0000 7ffb077fe640 1 -- 172.21.15.172:0/3684263970 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c00a380 con 0x55f78c9a4580 2024-09-10T08:48:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.321+0000 7ffb077fe640 1 -- 172.21.15.172:0/3684263970 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb0c01fc10 con 0x55f78c9a4580 2024-09-10T08:48:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.321+0000 7ffb077fe640 1 -- 172.21.15.172:0/3684263970 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0047a0 con 0x55f78c9a4580 2024-09-10T08:48:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.322+0000 7ffb26a61740 1 -- 172.21.15.172:0/3684263970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 msgr2=0x55f78c998ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.322+0000 7ffb26a61740 1 --2- 172.21.15.172:0/3684263970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c998ec0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c01fb90 tx=0x7ffb0c000bc0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.322+0000 7ffb26a61740 1 -- 172.21.15.172:0/3684263970 shutdown_connections 2024-09-10T08:48:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.322+0000 7ffb26a61740 1 --2- 172.21.15.172:0/3684263970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c998ec0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.322+0000 7ffb26a61740 1 --2- 172.21.15.172:0/3684263970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c99f4f0 0x55f78c9b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.322+0000 7ffb26a61740 1 --2- 172.21.15.172:0/3684263970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99ebb0 0x55f78c9a8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.322+0000 7ffb26a61740 1 -- 172.21.15.172:0/3684263970 >> 172.21.15.172:0/3684263970 conn(0x55f78c998010 msgr2=0x55f78c9a7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.322+0000 7ffb26a61740 1 -- 172.21.15.172:0/3684263970 shutdown_connections 2024-09-10T08:48:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.322+0000 7ffb26a61740 1 -- 172.21.15.172:0/3684263970 wait complete. 2024-09-10T08:48:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.322+0000 7ffb26a61740 1 -- start start 2024-09-10T08:48:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.322+0000 7ffb26a61740 1 Processor -- start 2024-09-10T08:48:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb26a61740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99ebb0 0x55f78c9ad940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb26a61740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c99f4f0 0x55f78c9ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb26a61740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c9aee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb26a61740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c1a96b0 con 0x55f78c9a4580 2024-09-10T08:48:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb26a61740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c1a8fe0 con 0x55f78c99ebb0 2024-09-10T08:48:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb26a61740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c1a6220 con 0x55f78c99f4f0 2024-09-10T08:48:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb26a61740 10 client.0 _ll_register_callbacks cb 0x7ffb0c00a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb21264640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c9aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb21264640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c9aee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37616/0 (socket says 172.21.15.172:37616) 2024-09-10T08:48:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb21264640 1 -- 172.21.15.172:0/125755902 learned_addr learned my addr 172.21.15.172:0/125755902 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb21a65640 1 --2- 172.21.15.172:0/125755902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c99f4f0 0x55f78c9ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb21264640 1 --2- 172.21.15.172:0/125755902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99ebb0 0x55f78c9ad940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb21a65640 1 -- 172.21.15.172:0/125755902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99ebb0 msgr2=0x55f78c9ad940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb21a65640 1 --2- 172.21.15.172:0/125755902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99ebb0 0x55f78c9ad940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb21a65640 1 -- 172.21.15.172:0/125755902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 msgr2=0x55f78c9aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb21a65640 1 --2- 172.21.15.172:0/125755902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c9aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb21a65640 1 -- 172.21.15.172:0/125755902 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f78c9a86c0 con 0x55f78c99f4f0 2024-09-10T08:48:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.323+0000 7ffb21264640 1 --2- 172.21.15.172:0/125755902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c9aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.324+0000 7ffb21a65640 1 --2- 172.21.15.172:0/125755902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c99f4f0 0x55f78c9ade80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffb1802a930 tx=0x7ffb18005690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af7655abc60e88e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.324+0000 7ffb057fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:48:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.324+0000 7ffb057fa640 1 -- 172.21.15.172:0/125755902 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1803f040 con 0x55f78c99f4f0 2024-09-10T08:48:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.324+0000 7ffb26a61740 1 -- 172.21.15.172:0/125755902 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb0c01fc10 con 0x55f78c99f4f0 2024-09-10T08:48:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.324+0000 7ffb057fa640 1 -- 172.21.15.172:0/125755902 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ffb18004230 con 0x55f78c99f4f0 2024-09-10T08:48:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.324+0000 7ffb057fa640 1 -- 172.21.15.172:0/125755902 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffb1805c050 con 0x55f78c99f4f0 2024-09-10T08:48:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.324+0000 7ffb26a61740 10 client.25534 fetch_fsmap learned FSMap version 29 2024-09-10T08:48:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.324+0000 7ffb26a61740 1 -- 172.21.15.172:0/125755902 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f78ca1f700 con 0x55f78c99f4f0 2024-09-10T08:48:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.324+0000 7ffb057fa640 1 -- 172.21.15.172:0/125755902 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb0c01fc10 con 0x55f78c99f4f0 2024-09-10T08:48:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.324+0000 7ffb057fa640 1 -- 172.21.15.172:0/125755902 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ffb1804c420 con 0x55f78c99f4f0 2024-09-10T08:48:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.324+0000 7ffb26a61740 10 client.25534 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T08:48:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.324+0000 7ffb26a61740 10 client.25534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.324+0000 7ffb26a61740 20 client.25534 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.324+0000 7ffb26a61740 1 --2- 172.21.15.172:0/125755902 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f78ca1fbd0 0x55f78ca21ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.324+0000 7ffb26a61740 4 client.25534 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:48:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.324+0000 7ffb26a61740 1 -- 172.21.15.172:0/125755902 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f78c907d70 con 0x55f78ca1fbd0 2024-09-10T08:48:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.324+0000 7ffb21264640 1 --2- 172.21.15.172:0/125755902 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f78ca1fbd0 0x55f78ca21ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.325+0000 7ffb057fa640 10 client.25534 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.325+0000 7ffb21264640 1 --2- 172.21.15.172:0/125755902 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f78ca1fbd0 0x55f78ca21ff0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b447773aaeae2c85 server_cookie=a93026980badaaac in_seq=0 out_seq=0 2024-09-10T08:48:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.435+0000 7ffb057fa640 1 -- 172.21.15.172:0/125755902 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f78c907d70 con 0x55f78ca1fbd0 2024-09-10T08:48:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.435+0000 7ffb057fa640 10 client.25534 handle_command_reply: tid=0 2024-09-10T08:48:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.435+0000 7ffb26a61740 1 client.25534 shutdown 2024-09-10T08:48:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.435+0000 7ffb26a61740 1 -- 172.21.15.172:0/125755902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c99f4f0 msgr2=0x55f78c9ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.435+0000 7ffb26a61740 1 --2- 172.21.15.172:0/125755902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c99f4f0 0x55f78c9ade80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffb1802a930 tx=0x7ffb18005690 comp rx=0 tx=0).stop 2024-09-10T08:48:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.435+0000 7ffb26a61740 1 -- 172.21.15.172:0/125755902 shutdown_connections 2024-09-10T08:48:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.435+0000 7ffb26a61740 1 --2- 172.21.15.172:0/125755902 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f78ca1fbd0 0x55f78ca21ff0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.435+0000 7ffb057fa640 0 client.25534 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.435+0000 7ffb26a61740 1 --2- 172.21.15.172:0/125755902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f78c9a4580 0x55f78c9aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.435+0000 7ffb26a61740 1 --2- 172.21.15.172:0/125755902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f78c99f4f0 0x55f78c9ade80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.435+0000 7ffb26a61740 1 --2- 172.21.15.172:0/125755902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78c99ebb0 0x55f78c9ad940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.435+0000 7ffb26a61740 1 -- 172.21.15.172:0/125755902 >> 172.21.15.172:0/125755902 conn(0x55f78c998010 msgr2=0x55f78c9969a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.435+0000 7ffb26a61740 1 -- 172.21.15.172:0/125755902 shutdown_connections 2024-09-10T08:48:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.435+0000 7ffb26a61740 1 -- 172.21.15.172:0/125755902 wait complete. 2024-09-10T08:48:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.436+0000 7ffb26a61740 20 client.25534 trim_cache size 0 max 16384 2024-09-10T08:48:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:48:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:48:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:48:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:48:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.436+0000 7ffb23cc9640 1 -- 172.21.15.172:0/3689648462 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffb080a2cb0 msgr2=0x7ffb080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.436+0000 7ffb23cc9640 1 --2- 172.21.15.172:0/3689648462 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffb080a2cb0 0x7ffb080a5170 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffb14084fd0 tx=0x7ffb140c6660 comp rx=0 tx=0).stop 2024-09-10T08:48:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.436+0000 7ffb23cc9640 1 -- 172.21.15.172:0/3689648462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb1c1025f0 msgr2=0x7ffb1c19e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.436+0000 7ffb23cc9640 1 --2- 172.21.15.172:0/3689648462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb1c1025f0 0x7ffb1c19e190 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400eaa0 tx=0x7ffb1400ef60 comp rx=0 tx=0).stop 2024-09-10T08:48:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.436+0000 7ffb23cc9640 1 -- 172.21.15.172:0/3689648462 shutdown_connections 2024-09-10T08:48:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.436+0000 7ffb23cc9640 1 --2- 172.21.15.172:0/3689648462 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffb080a2cb0 0x7ffb080a5170 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.436+0000 7ffb23cc9640 1 --2- 172.21.15.172:0/3689648462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb1c1025f0 0x7ffb1c19e190 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.436+0000 7ffb23cc9640 1 --2- 172.21.15.172:0/3689648462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb1c101c30 0x7ffb1c197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.436+0000 7ffb23cc9640 1 --2- 172.21.15.172:0/3689648462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb1c100a40 0x7ffb1c196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.436+0000 7ffb23cc9640 1 -- 172.21.15.172:0/3689648462 >> 172.21.15.172:0/3689648462 conn(0x7ffb1c0fc1e0 msgr2=0x7ffb1c105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.436+0000 7ffb23cc9640 1 -- 172.21.15.172:0/3689648462 shutdown_connections 2024-09-10T08:48:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:43.436+0000 7ffb23cc9640 1 -- 172.21.15.172:0/3689648462 wait complete. 2024-09-10T08:48:43.456 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-09-10T08:48:43.461 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-09-10T08:48:43.530 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-09-10T08:48:43.535 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-09-10T08:48:43.589 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-09-10T08:48:43.594 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-09-10T08:48:43.656 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-09-10T08:48:43.660 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-09-10T08:48:43.889 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-09-10T08:48:43.894 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-09-10T08:48:44.040 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-09-10T08:48:44.045 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-09-10T08:48:44.218 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-09-10T08:48:44.223 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-09-10T08:48:44.309 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-09-10T08:48:44.315 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-09-10T08:48:44.407 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-09-10T08:48:44.412 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-09-10T08:48:44.535 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-09-10T08:48:44.541 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-09-10T08:48:44.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:44 smithi172 ceph-mon[30854]: pgmap v425: 129 pgs: 129 active+clean; 6.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 551 KiB/s wr, 143 op/s 2024-09-10T08:48:44.583 INFO:tasks.workunit.client.0.smithi172.stderr:In file included from parser.h:22, 2024-09-10T08:48:44.583 INFO:tasks.workunit.client.0.smithi172.stderr: from parser.c:25: 2024-09-10T08:48:44.583 INFO:tasks.workunit.client.0.smithi172.stderr:parser.c: In function ‘init_filesys’: 2024-09-10T08:48:44.585 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T08:48:44.585 INFO:tasks.workunit.client.0.smithi172.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T08:48:44.585 INFO:tasks.workunit.client.0.smithi172.stderr: | ^ 2024-09-10T08:48:44.585 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-10T08:48:44.585 INFO:tasks.workunit.client.0.smithi172.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T08:48:44.585 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~ 2024-09-10T08:48:44.585 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-10T08:48:44.585 INFO:tasks.workunit.client.0.smithi172.stderr: 36 | container_of(ptr, type, member) 2024-09-10T08:48:44.586 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~ 2024-09-10T08:48:44.586 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-10T08:48:44.586 INFO:tasks.workunit.client.0.smithi172.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-10T08:48:44.586 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~ 2024-09-10T08:48:44.586 INFO:tasks.workunit.client.0.smithi172.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-10T08:48:44.586 INFO:tasks.workunit.client.0.smithi172.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-10T08:48:44.586 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T08:48:44.586 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T08:48:44.587 INFO:tasks.workunit.client.0.smithi172.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T08:48:44.587 INFO:tasks.workunit.client.0.smithi172.stderr: | ^ 2024-09-10T08:48:44.587 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-10T08:48:44.587 INFO:tasks.workunit.client.0.smithi172.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T08:48:44.587 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~ 2024-09-10T08:48:44.587 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-10T08:48:44.587 INFO:tasks.workunit.client.0.smithi172.stderr: 36 | container_of(ptr, type, member) 2024-09-10T08:48:44.587 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~ 2024-09-10T08:48:44.587 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-10T08:48:44.588 INFO:tasks.workunit.client.0.smithi172.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-10T08:48:44.588 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~ 2024-09-10T08:48:44.588 INFO:tasks.workunit.client.0.smithi172.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-10T08:48:44.588 INFO:tasks.workunit.client.0.smithi172.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-10T08:48:44.588 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T08:48:44.588 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T08:48:44.588 INFO:tasks.workunit.client.0.smithi172.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T08:48:44.589 INFO:tasks.workunit.client.0.smithi172.stderr: | ^ 2024-09-10T08:48:44.589 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-10T08:48:44.589 INFO:tasks.workunit.client.0.smithi172.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T08:48:44.589 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~ 2024-09-10T08:48:44.589 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-10T08:48:44.589 INFO:tasks.workunit.client.0.smithi172.stderr: 36 | container_of(ptr, type, member) 2024-09-10T08:48:44.589 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~ 2024-09-10T08:48:44.589 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-10T08:48:44.589 INFO:tasks.workunit.client.0.smithi172.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-10T08:48:44.590 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~ 2024-09-10T08:48:44.590 INFO:tasks.workunit.client.0.smithi172.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-10T08:48:44.590 INFO:tasks.workunit.client.0.smithi172.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-10T08:48:44.590 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T08:48:44.590 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T08:48:44.590 INFO:tasks.workunit.client.0.smithi172.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T08:48:44.590 INFO:tasks.workunit.client.0.smithi172.stderr: | ^ 2024-09-10T08:48:44.590 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-10T08:48:44.590 INFO:tasks.workunit.client.0.smithi172.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T08:48:44.590 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~ 2024-09-10T08:48:44.590 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-10T08:48:44.591 INFO:tasks.workunit.client.0.smithi172.stderr: 36 | container_of(ptr, type, member) 2024-09-10T08:48:44.591 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~ 2024-09-10T08:48:44.591 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-10T08:48:44.591 INFO:tasks.workunit.client.0.smithi172.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-10T08:48:44.591 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~ 2024-09-10T08:48:44.591 INFO:tasks.workunit.client.0.smithi172.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-09-10T08:48:44.591 INFO:tasks.workunit.client.0.smithi172.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-10T08:48:44.591 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T08:48:44.591 INFO:tasks.workunit.client.0.smithi172.stderr:parser.c: In function ‘init_tg_stats’: 2024-09-10T08:48:44.592 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T08:48:44.592 INFO:tasks.workunit.client.0.smithi172.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T08:48:44.592 INFO:tasks.workunit.client.0.smithi172.stderr: | ^ 2024-09-10T08:48:44.592 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-10T08:48:44.592 INFO:tasks.workunit.client.0.smithi172.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T08:48:44.592 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~ 2024-09-10T08:48:44.592 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-10T08:48:44.592 INFO:tasks.workunit.client.0.smithi172.stderr: 36 | container_of(ptr, type, member) 2024-09-10T08:48:44.592 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~ 2024-09-10T08:48:44.592 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-10T08:48:44.592 INFO:tasks.workunit.client.0.smithi172.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-10T08:48:44.593 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~ 2024-09-10T08:48:44.593 INFO:tasks.workunit.client.0.smithi172.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-10T08:48:44.593 INFO:tasks.workunit.client.0.smithi172.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-10T08:48:44.593 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T08:48:44.593 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T08:48:44.593 INFO:tasks.workunit.client.0.smithi172.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T08:48:44.593 INFO:tasks.workunit.client.0.smithi172.stderr: | ^ 2024-09-10T08:48:44.593 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-10T08:48:44.593 INFO:tasks.workunit.client.0.smithi172.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T08:48:44.593 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~ 2024-09-10T08:48:44.593 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-10T08:48:44.594 INFO:tasks.workunit.client.0.smithi172.stderr: 36 | container_of(ptr, type, member) 2024-09-10T08:48:44.594 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~ 2024-09-10T08:48:44.594 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-10T08:48:44.594 INFO:tasks.workunit.client.0.smithi172.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-10T08:48:44.594 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~ 2024-09-10T08:48:44.594 INFO:tasks.workunit.client.0.smithi172.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-10T08:48:44.594 INFO:tasks.workunit.client.0.smithi172.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-10T08:48:44.594 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T08:48:44.594 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T08:48:44.594 INFO:tasks.workunit.client.0.smithi172.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T08:48:44.595 INFO:tasks.workunit.client.0.smithi172.stderr: | ^ 2024-09-10T08:48:44.595 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-10T08:48:44.595 INFO:tasks.workunit.client.0.smithi172.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T08:48:44.595 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~ 2024-09-10T08:48:44.595 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-10T08:48:44.595 INFO:tasks.workunit.client.0.smithi172.stderr: 36 | container_of(ptr, type, member) 2024-09-10T08:48:44.595 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~ 2024-09-10T08:48:44.595 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-09-10T08:48:44.595 INFO:tasks.workunit.client.0.smithi172.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-10T08:48:44.595 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~ 2024-09-10T08:48:44.596 INFO:tasks.workunit.client.0.smithi172.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-10T08:48:44.596 INFO:tasks.workunit.client.0.smithi172.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-10T08:48:44.596 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T08:48:44.596 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T08:48:44.596 INFO:tasks.workunit.client.0.smithi172.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T08:48:44.596 INFO:tasks.workunit.client.0.smithi172.stderr: | ^ 2024-09-10T08:48:44.596 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-09-10T08:48:44.596 INFO:tasks.workunit.client.0.smithi172.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T08:48:44.597 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~ 2024-09-10T08:48:44.597 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-09-10T08:48:44.597 INFO:tasks.workunit.client.0.smithi172.stderr: 36 | container_of(ptr, type, member) 2024-09-10T08:48:44.597 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~ 2024-09-10T08:48:44.597 INFO:tasks.workunit.client.0.smithi172.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-09-10T08:48:44.597 INFO:tasks.workunit.client.0.smithi172.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-10T08:48:44.598 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~ 2024-09-10T08:48:44.598 INFO:tasks.workunit.client.0.smithi172.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-09-10T08:48:44.598 INFO:tasks.workunit.client.0.smithi172.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-10T08:48:44.598 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T08:48:44.631 INFO:tasks.workunit.client.0.smithi172.stderr:parser.c: In function ‘get_optstr’: 2024-09-10T08:48:44.631 INFO:tasks.workunit.client.0.smithi172.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-09-10T08:48:44.631 INFO:tasks.workunit.client.0.smithi172.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-09-10T08:48:44.631 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T08:48:44.631 INFO:tasks.workunit.client.0.smithi172.stderr:parser.c:200:14: note: source object declared here 2024-09-10T08:48:44.631 INFO:tasks.workunit.client.0.smithi172.stderr: 200 | char temp[BUFSIZE]; 2024-09-10T08:48:44.632 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~ 2024-09-10T08:48:44.637 INFO:tasks.workunit.client.0.smithi172.stderr:parser.c: In function ‘size64_convert’: 2024-09-10T08:48:44.637 INFO:tasks.workunit.client.0.smithi172.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-09-10T08:48:44.637 INFO:tasks.workunit.client.0.smithi172.stderr: 119 | memcpy(unit, "\0", 3); 2024-09-10T08:48:44.637 INFO:tasks.workunit.client.0.smithi172.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-09-10T08:48:44.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:44 smithi174 ceph-mon[34068]: pgmap v425: 129 pgs: 129 active+clean; 6.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 551 KiB/s wr, 143 op/s 2024-09-10T08:48:44.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:44 smithi191 ceph-mon[35314]: pgmap v425: 129 pgs: 129 active+clean; 6.8 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 551 KiB/s wr, 143 op/s 2024-09-10T08:48:44.941 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-09-10T08:48:44.946 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-09-10T08:48:45.020 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-09-10T08:48:45.025 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-09-10T08:48:45.131 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-09-10T08:48:45.136 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-09-10T08:48:45.180 INFO:tasks.workunit.client.0.smithi172.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-09-10T08:48:45.182 INFO:tasks.workunit.client.0.smithi172.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-09-10T08:48:45.282 INFO:tasks.workunit.client.0.smithi172.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-10T08:48:45.282 INFO:tasks.workunit.client.0.smithi172.stderr:+ cd .. 2024-09-10T08:48:45.284 INFO:tasks.workunit.client.0.smithi172.stderr:+ mkdir tmp 2024-09-10T08:48:45.286 INFO:tasks.workunit.client.0.smithi172.stderr:+ cd tmp 2024-09-10T08:48:45.287 INFO:tasks.workunit.client.0.smithi172.stderr:+ for f in $mydir/*.ffsb 2024-09-10T08:48:45.287 INFO:tasks.workunit.client.0.smithi172.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-09-10T08:48:45.287 INFO:tasks.workunit.client.0.smithi172.stdout:FFSB version 6.0-RC2 started 2024-09-10T08:48:45.287 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T08:48:45.291 INFO:tasks.workunit.client.0.smithi172.stdout:benchmark time = 300 2024-09-10T08:48:45.291 INFO:tasks.workunit.client.0.smithi172.stdout:ThreadGroup 0 2024-09-10T08:48:45.291 INFO:tasks.workunit.client.0.smithi172.stdout:================ 2024-09-10T08:48:45.291 INFO:tasks.workunit.client.0.smithi172.stdout: num_threads = 32 2024-09-10T08:48:45.291 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T08:48:45.291 INFO:tasks.workunit.client.0.smithi172.stdout: read_random = off 2024-09-10T08:48:45.291 INFO:tasks.workunit.client.0.smithi172.stdout: read_size = 0 (0B) 2024-09-10T08:48:45.291 INFO:tasks.workunit.client.0.smithi172.stdout: read_blocksize = 0 (0B) 2024-09-10T08:48:45.292 INFO:tasks.workunit.client.0.smithi172.stdout: read_skip = off 2024-09-10T08:48:45.292 INFO:tasks.workunit.client.0.smithi172.stdout: read_skipsize = 0 (0B) 2024-09-10T08:48:45.292 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T08:48:45.292 INFO:tasks.workunit.client.0.smithi172.stdout: write_random = on 2024-09-10T08:48:45.292 INFO:tasks.workunit.client.0.smithi172.stdout: write_size = 5242880 (5MB) 2024-09-10T08:48:45.292 INFO:tasks.workunit.client.0.smithi172.stdout: fsync_file = 0 2024-09-10T08:48:45.292 INFO:tasks.workunit.client.0.smithi172.stdout: write_blocksize = 4096 (4KB) 2024-09-10T08:48:45.292 INFO:tasks.workunit.client.0.smithi172.stdout: wait time = 0 2024-09-10T08:48:45.293 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T08:48:45.293 INFO:tasks.workunit.client.0.smithi172.stdout: op weights 2024-09-10T08:48:45.293 INFO:tasks.workunit.client.0.smithi172.stdout: read = 0 (0.00%) 2024-09-10T08:48:45.293 INFO:tasks.workunit.client.0.smithi172.stdout: readall = 0 (0.00%) 2024-09-10T08:48:45.293 INFO:tasks.workunit.client.0.smithi172.stdout: write = 1 (100.00%) 2024-09-10T08:48:45.293 INFO:tasks.workunit.client.0.smithi172.stdout: create = 0 (0.00%) 2024-09-10T08:48:45.293 INFO:tasks.workunit.client.0.smithi172.stdout: append = 0 (0.00%) 2024-09-10T08:48:45.293 INFO:tasks.workunit.client.0.smithi172.stdout: delete = 0 (0.00%) 2024-09-10T08:48:45.293 INFO:tasks.workunit.client.0.smithi172.stdout: metaop = 0 (0.00%) 2024-09-10T08:48:45.293 INFO:tasks.workunit.client.0.smithi172.stdout: createdir = 0 (0.00%) 2024-09-10T08:48:45.293 INFO:tasks.workunit.client.0.smithi172.stdout: stat = 0 (0.00%) 2024-09-10T08:48:45.293 INFO:tasks.workunit.client.0.smithi172.stdout: writeall = 0 (0.00%) 2024-09-10T08:48:45.293 INFO:tasks.workunit.client.0.smithi172.stdout: writeall_fsync = 0 (0.00%) 2024-09-10T08:48:45.294 INFO:tasks.workunit.client.0.smithi172.stdout: open_close = 0 (0.00%) 2024-09-10T08:48:45.294 INFO:tasks.workunit.client.0.smithi172.stdout: write_fsync = 0 (0.00%) 2024-09-10T08:48:45.294 INFO:tasks.workunit.client.0.smithi172.stdout: create_fsync = 0 (0.00%) 2024-09-10T08:48:45.294 INFO:tasks.workunit.client.0.smithi172.stdout: append_fsync = 0 (0.00%) 2024-09-10T08:48:45.294 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T08:48:45.294 INFO:tasks.workunit.client.0.smithi172.stdout:FileSystem . 2024-09-10T08:48:45.294 INFO:tasks.workunit.client.0.smithi172.stdout:========== 2024-09-10T08:48:45.294 INFO:tasks.workunit.client.0.smithi172.stdout: num_dirs = 0 2024-09-10T08:48:45.294 INFO:tasks.workunit.client.0.smithi172.stdout: starting files = 128 2024-09-10T08:48:45.294 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T08:48:45.294 INFO:tasks.workunit.client.0.smithi172.stdout: min file size = 104857600 (100MB) 2024-09-10T08:48:45.294 INFO:tasks.workunit.client.0.smithi172.stdout: max file size = 104857600 (100MB) 2024-09-10T08:48:45.295 INFO:tasks.workunit.client.0.smithi172.stdout: directio = off 2024-09-10T08:48:45.295 INFO:tasks.workunit.client.0.smithi172.stdout: alignedio = on 2024-09-10T08:48:45.295 INFO:tasks.workunit.client.0.smithi172.stdout: bufferedio = off 2024-09-10T08:48:45.295 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T08:48:45.295 INFO:tasks.workunit.client.0.smithi172.stdout: aging is off 2024-09-10T08:48:45.295 INFO:tasks.workunit.client.0.smithi172.stdout: current utilization = 0.04% 2024-09-10T08:48:45.295 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T08:48:45.295 INFO:tasks.workunit.client.0.smithi172.stdout:checking existing fs: . 2024-09-10T08:48:45.295 INFO:tasks.workunit.client.0.smithi172.stdout:recreating new fileset 2024-09-10T08:48:45.295 INFO:tasks.workunit.client.0.smithi172.stderr:opendir: No such file or directory 2024-09-10T08:48:46.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:46 smithi172 ceph-mon[30854]: pgmap v426: 129 pgs: 129 active+clean; 3.1 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 606 KiB/s wr, 201 op/s 2024-09-10T08:48:46.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:46 smithi174 ceph-mon[34068]: pgmap v426: 129 pgs: 129 active+clean; 3.1 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 606 KiB/s wr, 201 op/s 2024-09-10T08:48:46.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:46 smithi191 ceph-mon[35314]: pgmap v426: 129 pgs: 129 active+clean; 3.1 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 606 KiB/s wr, 201 op/s 2024-09-10T08:48:48.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:48 smithi172 ceph-mon[30854]: pgmap v427: 129 pgs: 129 active+clean; 21 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.9 MiB/s wr, 223 op/s 2024-09-10T08:48:48.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:48 smithi174 ceph-mon[34068]: pgmap v427: 129 pgs: 129 active+clean; 21 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.9 MiB/s wr, 223 op/s 2024-09-10T08:48:48.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:48 smithi191 ceph-mon[35314]: pgmap v427: 129 pgs: 129 active+clean; 21 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.9 MiB/s wr, 223 op/s 2024-09-10T08:48:50.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:50 smithi172 ceph-mon[30854]: pgmap v428: 129 pgs: 129 active+clean; 45 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 3.9 MiB/s wr, 217 op/s 2024-09-10T08:48:50.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:50 smithi174 ceph-mon[34068]: pgmap v428: 129 pgs: 129 active+clean; 45 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 3.9 MiB/s wr, 217 op/s 2024-09-10T08:48:50.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:50 smithi191 ceph-mon[35314]: pgmap v428: 129 pgs: 129 active+clean; 45 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 3.9 MiB/s wr, 217 op/s 2024-09-10T08:48:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:52 smithi172 ceph-mon[30854]: pgmap v429: 129 pgs: 129 active+clean; 114 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 9.6 MiB/s wr, 216 op/s 2024-09-10T08:48:52.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:52 smithi174 ceph-mon[34068]: pgmap v429: 129 pgs: 129 active+clean; 114 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 9.6 MiB/s wr, 216 op/s 2024-09-10T08:48:52.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:52 smithi191 ceph-mon[35314]: pgmap v429: 129 pgs: 129 active+clean; 114 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 9.6 MiB/s wr, 216 op/s 2024-09-10T08:48:54.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:54 smithi172 ceph-mon[30854]: pgmap v430: 129 pgs: 129 active+clean; 146 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 12 MiB/s wr, 181 op/s 2024-09-10T08:48:54.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:54 smithi174 ceph-mon[34068]: pgmap v430: 129 pgs: 129 active+clean; 146 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 12 MiB/s wr, 181 op/s 2024-09-10T08:48:54.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:54 smithi191 ceph-mon[35314]: pgmap v430: 129 pgs: 129 active+clean; 146 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 12 MiB/s wr, 181 op/s 2024-09-10T08:48:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.536+0000 7f72daff6640 1 -- 172.21.15.172:0/1674676126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72d4101b60 msgr2=0x7f72d4101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.536+0000 7f72daff6640 1 --2- 172.21.15.172:0/1674676126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72d4101b60 0x7f72d4101f60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f72c8009900 tx=0x7f72c802a410 comp rx=0 tx=0).stop 2024-09-10T08:48:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.536+0000 7f72daff6640 1 -- 172.21.15.172:0/1674676126 shutdown_connections 2024-09-10T08:48:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.536+0000 7f72daff6640 1 --2- 172.21.15.172:0/1674676126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72d4103710 0x7f72d4109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.536+0000 7f72daff6640 1 --2- 172.21.15.172:0/1674676126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72d4102d50 0x7f72d41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.536+0000 7f72daff6640 1 --2- 172.21.15.172:0/1674676126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72d4101b60 0x7f72d4101f60 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.536+0000 7f72daff6640 1 -- 172.21.15.172:0/1674676126 >> 172.21.15.172:0/1674676126 conn(0x7f72d40fd2e0 msgr2=0x7f72d40ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.536+0000 7f72daff6640 1 -- 172.21.15.172:0/1674676126 shutdown_connections 2024-09-10T08:48:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.536+0000 7f72daff6640 1 -- 172.21.15.172:0/1674676126 wait complete. 2024-09-10T08:48:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.537+0000 7f72daff6640 1 Processor -- start 2024-09-10T08:48:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.537+0000 7f72daff6640 1 -- start start 2024-09-10T08:48:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.537+0000 7f72daff6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72d4101b60 0x7f72d4197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.537+0000 7f72daff6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72d4102d50 0x7f72d4198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.537+0000 7f72daff6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72d4103710 0x7f72d419f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.537+0000 7f72daff6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f72d410d670 con 0x7f72d4102d50 2024-09-10T08:48:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.537+0000 7f72daff6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f72d410d4f0 con 0x7f72d4103710 2024-09-10T08:48:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.537+0000 7f72daff6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f72d410d7f0 con 0x7f72d4101b60 2024-09-10T08:48:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.537+0000 7f72d3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72d4102d50 0x7f72d4198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.537+0000 7f72d3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72d4102d50 0x7f72d4198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57286/0 (socket says 172.21.15.172:57286) 2024-09-10T08:48:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.537+0000 7f72d3fff640 1 -- 172.21.15.172:0/2534214424 learned_addr learned my addr 172.21.15.172:0/2534214424 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.537+0000 7f72d8d92640 1 --2- 172.21.15.172:0/2534214424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72d4101b60 0x7f72d4197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.537+0000 7f72d9593640 1 --2- 172.21.15.172:0/2534214424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72d4103710 0x7f72d419f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.538+0000 7f72d3fff640 1 -- 172.21.15.172:0/2534214424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72d4101b60 msgr2=0x7f72d4197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.538+0000 7f72d3fff640 1 --2- 172.21.15.172:0/2534214424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72d4101b60 0x7f72d4197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.538+0000 7f72d3fff640 1 -- 172.21.15.172:0/2534214424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72d4103710 msgr2=0x7f72d419f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.538+0000 7f72d3fff640 1 --2- 172.21.15.172:0/2534214424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72d4103710 0x7f72d419f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.538+0000 7f72d3fff640 1 -- 172.21.15.172:0/2534214424 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d419f950 con 0x7f72d4102d50 2024-09-10T08:48:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.538+0000 7f72d8d92640 1 --2- 172.21.15.172:0/2534214424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72d4101b60 0x7f72d4197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:48:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.538+0000 7f72d3fff640 1 --2- 172.21.15.172:0/2534214424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72d4102d50 0x7f72d4198280 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f72c000e9f0 tx=0x7f72c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.538+0000 7f72d1ffb640 1 -- 172.21.15.172:0/2534214424 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c000cd80 con 0x7f72d4102d50 2024-09-10T08:48:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.538+0000 7f72d1ffb640 1 -- 172.21.15.172:0/2534214424 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72c00108e0 con 0x7f72d4102d50 2024-09-10T08:48:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.538+0000 7f72d1ffb640 1 -- 172.21.15.172:0/2534214424 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0018920 con 0x7f72d4102d50 2024-09-10T08:48:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.538+0000 7f72daff6640 1 -- 172.21.15.172:0/2534214424 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d419fc40 con 0x7f72d4102d50 2024-09-10T08:48:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.538+0000 7f72daff6640 1 -- 172.21.15.172:0/2534214424 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d41a0140 con 0x7f72d4102d50 2024-09-10T08:48:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.540+0000 7f72d1ffb640 1 -- 172.21.15.172:0/2534214424 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f72c0010a90 con 0x7f72d4102d50 2024-09-10T08:48:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.541+0000 7f72dde6b740 1 Processor -- start 2024-09-10T08:48:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.541+0000 7f72dde6b740 1 -- start start 2024-09-10T08:48:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.541+0000 7f72d1ffb640 1 --2- 172.21.15.172:0/2534214424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f72c40a2cb0 0x7f72c40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.541+0000 7f72dde6b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a19a0 0x55637e5a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.541+0000 7f72dde6b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a22e0 0x55637e5a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.541+0000 7f72dde6b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a7370 0x55637e5a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.541+0000 7f72d1ffb640 1 -- 172.21.15.172:0/2534214424 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f72c00cf810 con 0x7f72d4102d50 2024-09-10T08:48:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72d9593640 1 --2- 172.21.15.172:0/2534214424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f72c40a2cb0 0x7f72c40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72d8d92640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a19a0 0x55637e5a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72d8d92640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a19a0 0x55637e5a1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57298/0 (socket says 172.21.15.172:57298) 2024-09-10T08:48:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72d8d92640 1 -- 172.21.15.172:0/2233614867 learned_addr learned my addr 172.21.15.172:0/2233614867 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72d3fff640 1 --2- 172.21.15.172:0/2233614867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a22e0 0x55637e5a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72d9593640 1 --2- 172.21.15.172:0/2233614867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a7370 0x55637e5a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72dde6b740 1 -- 172.21.15.172:0/2233614867 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55637e5052f0 con 0x55637e5a19a0 2024-09-10T08:48:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72dde6b740 1 -- 172.21.15.172:0/2233614867 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55637e4ad270 con 0x55637e5a7370 2024-09-10T08:48:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72dde6b740 1 -- 172.21.15.172:0/2233614867 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55637e5012b0 con 0x55637e5a22e0 2024-09-10T08:48:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72d8d92640 1 -- 172.21.15.172:0/2233614867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a22e0 msgr2=0x55637e5a6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72d8d92640 1 --2- 172.21.15.172:0/2233614867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a22e0 0x55637e5a6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72d8d92640 1 -- 172.21.15.172:0/2233614867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a7370 msgr2=0x55637e5a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72d8d92640 1 --2- 172.21.15.172:0/2233614867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a7370 0x55637e5a9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72d8d92640 1 -- 172.21.15.172:0/2233614867 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637e4ac340 con 0x55637e5a19a0 2024-09-10T08:48:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72d9593640 1 --2- 172.21.15.172:0/2233614867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a7370 0x55637e5a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72d9593640 1 --2- 172.21.15.172:0/2534214424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f72c40a2cb0 0x7f72c40a5170 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f72cc005c40 tx=0x7f72cc00acd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72d8d92640 1 --2- 172.21.15.172:0/2233614867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a19a0 0x55637e5a1da0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f72c80314e0 tx=0x7f72c8032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d2d6b2fcd001ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.542+0000 7f72beffd640 1 -- 172.21.15.172:0/2233614867 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c803f070 con 0x55637e5a19a0 2024-09-10T08:48:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.543+0000 7f72beffd640 1 -- 172.21.15.172:0/2233614867 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72c8031780 con 0x55637e5a19a0 2024-09-10T08:48:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.543+0000 7f72beffd640 1 -- 172.21.15.172:0/2233614867 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c8035070 con 0x55637e5a19a0 2024-09-10T08:48:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.543+0000 7f72dde6b740 1 -- 172.21.15.172:0/2233614867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a19a0 msgr2=0x55637e5a1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.543+0000 7f72dde6b740 1 --2- 172.21.15.172:0/2233614867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a19a0 0x55637e5a1da0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f72c80314e0 tx=0x7f72c8032dc0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.543+0000 7f72dde6b740 1 -- 172.21.15.172:0/2233614867 shutdown_connections 2024-09-10T08:48:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.543+0000 7f72dde6b740 1 --2- 172.21.15.172:0/2233614867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a7370 0x55637e5a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.543+0000 7f72dde6b740 1 --2- 172.21.15.172:0/2233614867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a22e0 0x55637e5a6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.543+0000 7f72dde6b740 1 --2- 172.21.15.172:0/2233614867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a19a0 0x55637e5a1da0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.543+0000 7f72dde6b740 1 -- 172.21.15.172:0/2233614867 >> 172.21.15.172:0/2233614867 conn(0x55637e59d010 msgr2=0x55637e59d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.543+0000 7f72dde6b740 1 -- 172.21.15.172:0/2233614867 shutdown_connections 2024-09-10T08:48:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.543+0000 7f72dde6b740 1 -- 172.21.15.172:0/2233614867 wait complete. 2024-09-10T08:48:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.544+0000 7f72dde6b740 1 -- start start 2024-09-10T08:48:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.544+0000 7f72dde6b740 1 Processor -- start 2024-09-10T08:48:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.544+0000 7f72dde6b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a19a0 0x55637e5abb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.544+0000 7f72dde6b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.544+0000 7f72dde6b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a7370 0x55637e5b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.544+0000 7f72dde6b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55637e5aa2e0 con 0x55637e5a22e0 2024-09-10T08:48:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.544+0000 7f72dde6b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55637e48e290 con 0x55637e5a19a0 2024-09-10T08:48:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.544+0000 7f72dde6b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55637e5aa460 con 0x55637e5a7370 2024-09-10T08:48:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.544+0000 7f72dde6b740 10 client.0 _ll_register_callbacks cb 0x55637e4bf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.544+0000 7f72d3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.544+0000 7f72d3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57308/0 (socket says 172.21.15.172:57308) 2024-09-10T08:48:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.544+0000 7f72d3fff640 1 -- 172.21.15.172:0/1846134051 learned_addr learned my addr 172.21.15.172:0/1846134051 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.545+0000 7f72d9593640 1 --2- 172.21.15.172:0/1846134051 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a7370 0x55637e5b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.545+0000 7f72d8d92640 1 --2- 172.21.15.172:0/1846134051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a19a0 0x55637e5abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.545+0000 7f72d3fff640 1 -- 172.21.15.172:0/1846134051 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a7370 msgr2=0x55637e5b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.545+0000 7f72d3fff640 1 --2- 172.21.15.172:0/1846134051 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a7370 0x55637e5b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.545+0000 7f72d3fff640 1 -- 172.21.15.172:0/1846134051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a19a0 msgr2=0x55637e5abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.545+0000 7f72d3fff640 1 --2- 172.21.15.172:0/1846134051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a19a0 0x55637e5abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.545+0000 7f72d3fff640 1 -- 172.21.15.172:0/1846134051 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637e218210 con 0x55637e5a22e0 2024-09-10T08:48:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.545+0000 7f72d3fff640 1 --2- 172.21.15.172:0/1846134051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5ac040 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f72c009bca0 tx=0x7f72c009b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e736bb5f57b785c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.545+0000 7f72b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.545+0000 7f72b3fff640 1 -- 172.21.15.172:0/1846134051 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c009bd10 con 0x55637e5a22e0 2024-09-10T08:48:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.545+0000 7f72dde6b740 1 -- 172.21.15.172:0/1846134051 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72c8031780 con 0x55637e5a22e0 2024-09-10T08:48:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.545+0000 7f72b3fff640 1 -- 172.21.15.172:0/1846134051 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f72c009b160 con 0x55637e5a22e0 2024-09-10T08:48:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.545+0000 7f72b3fff640 1 -- 172.21.15.172:0/1846134051 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72c00a00c0 con 0x55637e5a22e0 2024-09-10T08:48:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.545+0000 7f72dde6b740 10 client.16254 fetch_fsmap learned FSMap version 30 2024-09-10T08:48:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.545+0000 7f72dde6b740 1 -- 172.21.15.172:0/1846134051 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55637e2e4620 con 0x55637e5a22e0 2024-09-10T08:48:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.546+0000 7f72b3fff640 1 -- 172.21.15.172:0/1846134051 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72c8031780 con 0x55637e5a22e0 2024-09-10T08:48:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.546+0000 7f72b3fff640 1 -- 172.21.15.172:0/1846134051 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f72c0018ac0 con 0x55637e5a22e0 2024-09-10T08:48:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.546+0000 7f72dde6b740 10 client.16254 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T08:48:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.546+0000 7f72dde6b740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.546+0000 7f72dde6b740 20 client.16254 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.546+0000 7f72dde6b740 1 --2- 172.21.15.172:0/1846134051 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55637e5bbb40 0x55637e5bdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.546+0000 7f72dde6b740 4 client.16254 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:48:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.546+0000 7f72dde6b740 1 -- 172.21.15.172:0/1846134051 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55637e512a20 con 0x55637e5bbb40 2024-09-10T08:48:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.546+0000 7f72d8d92640 1 --2- 172.21.15.172:0/1846134051 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55637e5bbb40 0x55637e5bdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.547+0000 7f72b3fff640 10 client.16254 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.547+0000 7f72d8d92640 1 --2- 172.21.15.172:0/1846134051 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55637e5bbb40 0x55637e5bdfe0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9cd54d047c4018c server_cookie=1d1c6e7ff6c3d083 in_seq=0 out_seq=0 2024-09-10T08:48:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.547+0000 7f72b3fff640 1 -- 172.21.15.172:0/1846134051 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55637e512a20 con 0x55637e5bbb40 2024-09-10T08:48:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.547+0000 7f72b3fff640 10 client.16254 handle_command_reply: tid=0 2024-09-10T08:48:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.547+0000 7f72dde6b740 1 client.16254 shutdown 2024-09-10T08:48:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.547+0000 7f72dde6b740 1 -- 172.21.15.172:0/1846134051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 msgr2=0x55637e5ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.547+0000 7f72dde6b740 1 --2- 172.21.15.172:0/1846134051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5ac040 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f72c009bca0 tx=0x7f72c009b780 comp rx=0 tx=0).stop 2024-09-10T08:48:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.547+0000 7f72dde6b740 1 -- 172.21.15.172:0/1846134051 shutdown_connections 2024-09-10T08:48:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.547+0000 7f72dde6b740 1 --2- 172.21.15.172:0/1846134051 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55637e5bbb40 0x55637e5bdfe0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.547+0000 7f72dde6b740 1 --2- 172.21.15.172:0/1846134051 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a7370 0x55637e5b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.547+0000 7f72dde6b740 1 --2- 172.21.15.172:0/1846134051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5ac040 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.547+0000 7f72dde6b740 1 --2- 172.21.15.172:0/1846134051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a19a0 0x55637e5abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.547+0000 7f72b3fff640 0 client.16254 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.547+0000 7f72dde6b740 1 -- 172.21.15.172:0/1846134051 >> 172.21.15.172:0/1846134051 conn(0x55637e59d010 msgr2=0x55637e59e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.548+0000 7f72dde6b740 1 -- 172.21.15.172:0/1846134051 shutdown_connections 2024-09-10T08:48:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.548+0000 7f72dde6b740 1 -- 172.21.15.172:0/1846134051 wait complete. 2024-09-10T08:48:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.548+0000 7f72dde6b740 20 client.16254 trim_cache size 0 max 16384 2024-09-10T08:48:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.564+0000 7f72dde6b740 1 Processor -- start 2024-09-10T08:48:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.564+0000 7f72dde6b740 1 -- start start 2024-09-10T08:48:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72dde6b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a19a0 0x55637e59c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72dde6b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72dde6b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a7370 0x55637e5c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72dde6b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41a82d0 con 0x55637e5a22e0 2024-09-10T08:48:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72dde6b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41a9e40 con 0x55637e5a19a0 2024-09-10T08:48:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72dde6b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41a89a0 con 0x55637e5a7370 2024-09-10T08:48:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72d3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72d3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5b3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57320/0 (socket says 172.21.15.172:57320) 2024-09-10T08:48:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72d3fff640 1 -- 172.21.15.172:0/4101406564 learned_addr learned my addr 172.21.15.172:0/4101406564 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72d8d92640 1 --2- 172.21.15.172:0/4101406564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a19a0 0x55637e59c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72d9593640 1 --2- 172.21.15.172:0/4101406564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a7370 0x55637e5c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72d3fff640 1 -- 172.21.15.172:0/4101406564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a7370 msgr2=0x55637e5c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72d3fff640 1 --2- 172.21.15.172:0/4101406564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a7370 0x55637e5c28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72d3fff640 1 -- 172.21.15.172:0/4101406564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a19a0 msgr2=0x55637e59c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72d3fff640 1 --2- 172.21.15.172:0/4101406564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a19a0 0x55637e59c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72d3fff640 1 -- 172.21.15.172:0/4101406564 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637e5b4520 con 0x55637e5a22e0 2024-09-10T08:48:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72d3fff640 1 --2- 172.21.15.172:0/4101406564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5b3040 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f72c000e9c0 tx=0x7f72c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53515642b4a8946e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.565+0000 7f72be7fc640 1 -- 172.21.15.172:0/4101406564 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c009bd10 con 0x55637e5a22e0 2024-09-10T08:48:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.566+0000 7f72be7fc640 1 -- 172.21.15.172:0/4101406564 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72c00a4980 con 0x55637e5a22e0 2024-09-10T08:48:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.566+0000 7f72be7fc640 1 -- 172.21.15.172:0/4101406564 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c00979d0 con 0x55637e5a22e0 2024-09-10T08:48:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.566+0000 7f72dde6b740 1 -- 172.21.15.172:0/4101406564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 msgr2=0x55637e5b3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.566+0000 7f72dde6b740 1 --2- 172.21.15.172:0/4101406564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5b3040 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f72c000e9c0 tx=0x7f72c0000bc0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.566+0000 7f72dde6b740 1 -- 172.21.15.172:0/4101406564 shutdown_connections 2024-09-10T08:48:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.566+0000 7f72dde6b740 1 --2- 172.21.15.172:0/4101406564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a7370 0x55637e5c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.566+0000 7f72dde6b740 1 --2- 172.21.15.172:0/4101406564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5b3040 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.566+0000 7f72dde6b740 1 --2- 172.21.15.172:0/4101406564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a19a0 0x55637e59c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.567+0000 7f72dde6b740 1 -- 172.21.15.172:0/4101406564 >> 172.21.15.172:0/4101406564 conn(0x55637e59d010 msgr2=0x55637e59b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.567+0000 7f72dde6b740 1 -- 172.21.15.172:0/4101406564 shutdown_connections 2024-09-10T08:48:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.567+0000 7f72dde6b740 1 -- 172.21.15.172:0/4101406564 wait complete. 2024-09-10T08:48:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.567+0000 7f72dde6b740 1 -- start start 2024-09-10T08:48:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.567+0000 7f72dde6b740 1 Processor -- start 2024-09-10T08:48:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.567+0000 7f72dde6b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a19a0 0x55637e5b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.567+0000 7f72dde6b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.567+0000 7f72dde6b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a7370 0x55637e5b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.567+0000 7f72dde6b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41a9740 con 0x55637e5a22e0 2024-09-10T08:48:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.567+0000 7f72dde6b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41a9070 con 0x55637e5a19a0 2024-09-10T08:48:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.567+0000 7f72dde6b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f72d41a62b0 con 0x55637e5a7370 2024-09-10T08:48:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.567+0000 7f72dde6b740 10 client.0 _ll_register_callbacks cb 0x7f72c00978b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:48:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72d3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72d3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5b0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57334/0 (socket says 172.21.15.172:57334) 2024-09-10T08:48:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72d3fff640 1 -- 172.21.15.172:0/2623415162 learned_addr learned my addr 172.21.15.172:0/2623415162 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:48:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72d8d92640 1 --2- 172.21.15.172:0/2623415162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a19a0 0x55637e5b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72d9593640 1 --2- 172.21.15.172:0/2623415162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a7370 0x55637e5b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72d3fff640 1 -- 172.21.15.172:0/2623415162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a7370 msgr2=0x55637e5b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72d3fff640 1 --2- 172.21.15.172:0/2623415162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a7370 0x55637e5b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72d3fff640 1 -- 172.21.15.172:0/2623415162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a19a0 msgr2=0x55637e5b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:48:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72d3fff640 1 --2- 172.21.15.172:0/2623415162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a19a0 0x55637e5b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72d3fff640 1 -- 172.21.15.172:0/2623415162 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637e5b25a0 con 0x55637e5a22e0 2024-09-10T08:48:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72d8d92640 1 --2- 172.21.15.172:0/2623415162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a19a0 0x55637e5b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:48:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:48:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72d3fff640 1 --2- 172.21.15.172:0/2623415162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5b0f80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f72c000c6a0 tx=0x7f72c009b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e467f71d3a72ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:48:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72b3fff640 1 -- 172.21.15.172:0/2623415162 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c009b670 con 0x55637e5a22e0 2024-09-10T08:48:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72dde6b740 1 -- 172.21.15.172:0/2623415162 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72c00a4980 con 0x55637e5a22e0 2024-09-10T08:48:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72b3fff640 1 -- 172.21.15.172:0/2623415162 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f72c0097bd0 con 0x55637e5a22e0 2024-09-10T08:48:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72b3fff640 1 -- 172.21.15.172:0/2623415162 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72c009bd10 con 0x55637e5a22e0 2024-09-10T08:48:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72dde6b740 10 client.16260 fetch_fsmap learned FSMap version 30 2024-09-10T08:48:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.568+0000 7f72dde6b740 1 -- 172.21.15.172:0/2623415162 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55637e5b25a0 con 0x55637e5a22e0 2024-09-10T08:48:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.569+0000 7f72b3fff640 1 -- 172.21.15.172:0/2623415162 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72c00a4980 con 0x55637e5a22e0 2024-09-10T08:48:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.569+0000 7f72b3fff640 1 -- 172.21.15.172:0/2623415162 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8374+0+0 (secure 0 0 0) 0x7f72c00ab050 con 0x55637e5a22e0 2024-09-10T08:48:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.569+0000 7f72dde6b740 10 client.16260 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T08:48:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.569+0000 7f72dde6b740 10 client.16260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:48:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.569+0000 7f72dde6b740 20 client.16260 populate_metadata read hostname 'smithi172' 2024-09-10T08:48:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.569+0000 7f72dde6b740 1 --2- 172.21.15.172:0/2623415162 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55637e627140 0x55637e629560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:48:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.569+0000 7f72dde6b740 4 client.16260 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:48:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.569+0000 7f72dde6b740 1 -- 172.21.15.172:0/2623415162 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55637e50cd70 con 0x55637e627140 2024-09-10T08:48:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.569+0000 7f72d8d92640 1 --2- 172.21.15.172:0/2623415162 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55637e627140 0x55637e629560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:48:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.569+0000 7f72b3fff640 10 client.16260 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.569+0000 7f72d8d92640 1 --2- 172.21.15.172:0/2623415162 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55637e627140 0x55637e629560 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba4cf16b39ca97dd server_cookie=b3299c2214f0537 in_seq=0 out_seq=0 2024-09-10T08:48:56.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:56 smithi172 ceph-mon[30854]: pgmap v431: 129 pgs: 129 active+clean; 234 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 19 MiB/s wr, 174 op/s 2024-09-10T08:48:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.591+0000 7f72b3fff640 1 -- 172.21.15.172:0/2623415162 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55637e50cd70 con 0x55637e627140 2024-09-10T08:48:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.591+0000 7f72b3fff640 10 client.16260 handle_command_reply: tid=0 2024-09-10T08:48:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.591+0000 7f72dde6b740 1 client.16260 shutdown 2024-09-10T08:48:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.591+0000 7f72dde6b740 1 -- 172.21.15.172:0/2623415162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 msgr2=0x55637e5b0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.591+0000 7f72dde6b740 1 --2- 172.21.15.172:0/2623415162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5b0f80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f72c000c6a0 tx=0x7f72c009b5b0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.591+0000 7f72dde6b740 1 -- 172.21.15.172:0/2623415162 shutdown_connections 2024-09-10T08:48:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.591+0000 7f72dde6b740 1 --2- 172.21.15.172:0/2623415162 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55637e627140 0x55637e629560 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.591+0000 7f72dde6b740 1 --2- 172.21.15.172:0/2623415162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55637e5a7370 0x55637e5b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.591+0000 7f72dde6b740 1 --2- 172.21.15.172:0/2623415162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55637e5a22e0 0x55637e5b0f80 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.591+0000 7f72b3fff640 0 client.16260 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:48:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.591+0000 7f72dde6b740 1 --2- 172.21.15.172:0/2623415162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637e5a19a0 0x55637e5b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.591+0000 7f72dde6b740 1 -- 172.21.15.172:0/2623415162 >> 172.21.15.172:0/2623415162 conn(0x55637e59d010 msgr2=0x55637e59b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.591+0000 7f72dde6b740 1 -- 172.21.15.172:0/2623415162 shutdown_connections 2024-09-10T08:48:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.591+0000 7f72dde6b740 1 -- 172.21.15.172:0/2623415162 wait complete. 2024-09-10T08:48:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.592+0000 7f72dde6b740 20 client.16260 trim_cache size 0 max 16384 2024-09-10T08:48:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:48:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:48:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:48:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:48:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.592+0000 7f72daff6640 1 -- 172.21.15.172:0/2534214424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f72c40a2cb0 msgr2=0x7f72c40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.592+0000 7f72daff6640 1 --2- 172.21.15.172:0/2534214424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f72c40a2cb0 0x7f72c40a5170 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f72cc005c40 tx=0x7f72cc00acd0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.592+0000 7f72daff6640 1 -- 172.21.15.172:0/2534214424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72d4102d50 msgr2=0x7f72d4198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:48:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.592+0000 7f72daff6640 1 --2- 172.21.15.172:0/2534214424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72d4102d50 0x7f72d4198280 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f72c000e9f0 tx=0x7f72c000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.592+0000 7f72daff6640 1 -- 172.21.15.172:0/2534214424 shutdown_connections 2024-09-10T08:48:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.592+0000 7f72daff6640 1 --2- 172.21.15.172:0/2534214424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f72c40a2cb0 0x7f72c40a5170 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.592+0000 7f72daff6640 1 --2- 172.21.15.172:0/2534214424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72d4103710 0x7f72d419f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.592+0000 7f72daff6640 1 --2- 172.21.15.172:0/2534214424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72d4102d50 0x7f72d4198280 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.592+0000 7f72daff6640 1 --2- 172.21.15.172:0/2534214424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72d4101b60 0x7f72d4197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:48:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.592+0000 7f72daff6640 1 -- 172.21.15.172:0/2534214424 >> 172.21.15.172:0/2534214424 conn(0x7f72d40fd2e0 msgr2=0x7f72d4106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:48:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.592+0000 7f72daff6640 1 -- 172.21.15.172:0/2534214424 shutdown_connections 2024-09-10T08:48:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:48:56.592+0000 7f72daff6640 1 -- 172.21.15.172:0/2534214424 wait complete. 2024-09-10T08:48:56.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:56 smithi174 ceph-mon[34068]: pgmap v431: 129 pgs: 129 active+clean; 234 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 19 MiB/s wr, 174 op/s 2024-09-10T08:48:56.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:56 smithi191 ceph-mon[35314]: pgmap v431: 129 pgs: 129 active+clean; 234 MiB data, 827 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 19 MiB/s wr, 174 op/s 2024-09-10T08:48:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:57 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:48:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:48:57 smithi172 ceph-mon[30854]: pgmap v432: 129 pgs: 129 active+clean; 298 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 25 MiB/s wr, 113 op/s 2024-09-10T08:48:57.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:57 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:48:57.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:48:57 smithi174 ceph-mon[34068]: pgmap v432: 129 pgs: 129 active+clean; 298 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 25 MiB/s wr, 113 op/s 2024-09-10T08:48:57.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:57 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:48:57.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:48:57 smithi191 ceph-mon[35314]: pgmap v432: 129 pgs: 129 active+clean; 298 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 25 MiB/s wr, 113 op/s 2024-09-10T08:49:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:00 smithi191 ceph-mon[35314]: pgmap v433: 129 pgs: 129 active+clean; 314 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 MiB/s wr, 47 op/s 2024-09-10T08:49:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:00 smithi172 ceph-mon[30854]: pgmap v433: 129 pgs: 129 active+clean; 314 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 MiB/s wr, 47 op/s 2024-09-10T08:49:00.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:00 smithi174 ceph-mon[34068]: pgmap v433: 129 pgs: 129 active+clean; 314 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 24 MiB/s wr, 47 op/s 2024-09-10T08:49:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:02 smithi191 ceph-mon[35314]: pgmap v434: 129 pgs: 129 active+clean; 418 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 31 MiB/s wr, 40 op/s 2024-09-10T08:49:02.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:02 smithi172 ceph-mon[30854]: pgmap v434: 129 pgs: 129 active+clean; 418 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 31 MiB/s wr, 40 op/s 2024-09-10T08:49:02.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:02 smithi174 ceph-mon[34068]: pgmap v434: 129 pgs: 129 active+clean; 418 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 31 MiB/s wr, 40 op/s 2024-09-10T08:49:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:03 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:49:03.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:03 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:49:03.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:03 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:49:04.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:04 smithi191 ceph-mon[35314]: pgmap v435: 129 pgs: 129 active+clean; 430 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 31 op/s 2024-09-10T08:49:04.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:04 smithi172 ceph-mon[30854]: pgmap v435: 129 pgs: 129 active+clean; 430 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 31 op/s 2024-09-10T08:49:04.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:04 smithi174 ceph-mon[34068]: pgmap v435: 129 pgs: 129 active+clean; 430 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 31 op/s 2024-09-10T08:49:06.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:06 smithi191 ceph-mon[35314]: pgmap v436: 129 pgs: 129 active+clean; 514 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 35 op/s 2024-09-10T08:49:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:06 smithi172 ceph-mon[30854]: pgmap v436: 129 pgs: 129 active+clean; 514 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 35 op/s 2024-09-10T08:49:06.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:06 smithi174 ceph-mon[34068]: pgmap v436: 129 pgs: 129 active+clean; 514 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 35 op/s 2024-09-10T08:49:08.553 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:08 smithi191 ceph-mon[35314]: pgmap v437: 129 pgs: 129 active+clean; 598 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 MiB/s wr, 35 op/s 2024-09-10T08:49:08.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:08 smithi172 ceph-mon[30854]: pgmap v437: 129 pgs: 129 active+clean; 598 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 MiB/s wr, 35 op/s 2024-09-10T08:49:08.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:08 smithi174 ceph-mon[34068]: pgmap v437: 129 pgs: 129 active+clean; 598 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 30 MiB/s wr, 35 op/s 2024-09-10T08:49:09.257 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:49:09.547 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.690+0000 7f5f3bd6d640 1 -- 172.21.15.172:0/4246575603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f34100a40 msgr2=0x7f5f34100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.690+0000 7f5f3bd6d640 1 --2- 172.21.15.172:0/4246575603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f34100a40 0x7f5f34100e40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009900 tx=0x7f5f3002a410 comp rx=0 tx=0).stop 2024-09-10T08:49:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.691+0000 7f5f3bd6d640 1 -- 172.21.15.172:0/4246575603 shutdown_connections 2024-09-10T08:49:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.691+0000 7f5f3bd6d640 1 --2- 172.21.15.172:0/4246575603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f341025f0 0x7f5f34108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.691+0000 7f5f3bd6d640 1 --2- 172.21.15.172:0/4246575603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f34101c30 0x7f5f341020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.691+0000 7f5f3bd6d640 1 --2- 172.21.15.172:0/4246575603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f34100a40 0x7f5f34100e40 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.691+0000 7f5f3bd6d640 1 -- 172.21.15.172:0/4246575603 >> 172.21.15.172:0/4246575603 conn(0x7f5f340fc1e0 msgr2=0x7f5f340fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.691+0000 7f5f3bd6d640 1 -- 172.21.15.172:0/4246575603 shutdown_connections 2024-09-10T08:49:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.691+0000 7f5f3bd6d640 1 -- 172.21.15.172:0/4246575603 wait complete. 2024-09-10T08:49:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.691+0000 7f5f3bd6d640 1 Processor -- start 2024-09-10T08:49:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.691+0000 7f5f3bd6d640 1 -- start start 2024-09-10T08:49:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.691+0000 7f5f3bd6d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f34100a40 0x7f5f3406a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.691+0000 7f5f3bd6d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f34101c30 0x7f5f3419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f3bd6d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f341025f0 0x7f5f3419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f3bd6d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3410c640 con 0x7f5f341025f0 2024-09-10T08:49:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f3bd6d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3410c4c0 con 0x7f5f34101c30 2024-09-10T08:49:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f3bd6d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3410c7c0 con 0x7f5f34100a40 2024-09-10T08:49:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f3a30a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f341025f0 0x7f5f3419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f39b09640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f34100a40 0x7f5f3406a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f3a30a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f341025f0 0x7f5f3419f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48482/0 (socket says 172.21.15.172:48482) 2024-09-10T08:49:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f3a30a640 1 -- 172.21.15.172:0/3523370747 learned_addr learned my addr 172.21.15.172:0/3523370747 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f39308640 1 --2- 172.21.15.172:0/3523370747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f34101c30 0x7f5f3419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f3a30a640 1 -- 172.21.15.172:0/3523370747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f34100a40 msgr2=0x7f5f3406a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f3a30a640 1 --2- 172.21.15.172:0/3523370747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f34100a40 0x7f5f3406a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f3a30a640 1 -- 172.21.15.172:0/3523370747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f34101c30 msgr2=0x7f5f3419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f3a30a640 1 --2- 172.21.15.172:0/3523370747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f34101c30 0x7f5f3419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f3a30a640 1 -- 172.21.15.172:0/3523370747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3419f980 con 0x7f5f341025f0 2024-09-10T08:49:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f39b09640 1 --2- 172.21.15.172:0/3523370747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f34100a40 0x7f5f3406a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f3a30a640 1 --2- 172.21.15.172:0/3523370747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f341025f0 0x7f5f3419f0c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00eaa0 tx=0x7f5f2c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f2affd640 1 -- 172.21.15.172:0/3523370747 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c004d20 con 0x7f5f341025f0 2024-09-10T08:49:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f3bd6d640 1 -- 172.21.15.172:0/3523370747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f3419fc70 con 0x7f5f341025f0 2024-09-10T08:49:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f2affd640 1 -- 172.21.15.172:0/3523370747 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f2c005020 con 0x7f5f341025f0 2024-09-10T08:49:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.692+0000 7f5f2affd640 1 -- 172.21.15.172:0/3523370747 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c0116d0 con 0x7f5f341025f0 2024-09-10T08:49:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.693+0000 7f5f3bd6d640 1 -- 172.21.15.172:0/3523370747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f341a0240 con 0x7f5f341025f0 2024-09-10T08:49:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.695+0000 7f5f2affd640 1 -- 172.21.15.172:0/3523370747 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5f2c00caf0 con 0x7f5f341025f0 2024-09-10T08:49:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.696+0000 7f5f3ebec740 1 Processor -- start 2024-09-10T08:49:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.696+0000 7f5f3ebec740 1 -- start start 2024-09-10T08:49:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.696+0000 7f5f3ebec740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5659a0 0x55933f565da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.696+0000 7f5f3ebec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f5662e0 0x55933f56ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.696+0000 7f5f3ebec740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f56b370 0x55933f56d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.696+0000 7f5f3ebec740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55933f4c92f0 con 0x55933f5659a0 2024-09-10T08:49:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.696+0000 7f5f3ebec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55933f471270 con 0x55933f5662e0 2024-09-10T08:49:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.696+0000 7f5f3ebec740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55933f4c52b0 con 0x55933f56b370 2024-09-10T08:49:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.696+0000 7f5f2affd640 1 --2- 172.21.15.172:0/3523370747 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f200a2c30 0x7f5f200a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.696+0000 7f5f39308640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5659a0 0x55933f565da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.696+0000 7f5f39308640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5659a0 0x55933f565da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48496/0 (socket says 172.21.15.172:48496) 2024-09-10T08:49:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.696+0000 7f5f39308640 1 -- 172.21.15.172:0/3964772199 learned_addr learned my addr 172.21.15.172:0/3964772199 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.696+0000 7f5f3a30a640 1 --2- 172.21.15.172:0/3523370747 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f200a2c30 0x7f5f200a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.697+0000 7f5f39b09640 1 --2- 172.21.15.172:0/3964772199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f5662e0 0x55933f56ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.697+0000 7f5f39308640 1 --2- 172.21.15.172:0/3964772199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f56b370 0x55933f56d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.697+0000 7f5f2affd640 1 -- 172.21.15.172:0/3523370747 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5f2c0cf450 con 0x7f5f341025f0 2024-09-10T08:49:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.697+0000 7f5f39308640 1 -- 172.21.15.172:0/3964772199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f56b370 msgr2=0x55933f56d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.697+0000 7f5f39308640 1 --2- 172.21.15.172:0/3964772199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f56b370 0x55933f56d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.697+0000 7f5f39308640 1 -- 172.21.15.172:0/3964772199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f5662e0 msgr2=0x55933f56ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.697+0000 7f5f39308640 1 --2- 172.21.15.172:0/3964772199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f5662e0 0x55933f56ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.697+0000 7f5f39308640 1 -- 172.21.15.172:0/3964772199 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55933f470340 con 0x55933f5659a0 2024-09-10T08:49:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.697+0000 7f5f3a30a640 1 --2- 172.21.15.172:0/3523370747 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f200a2c30 0x7f5f200a50f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0ccf60 tx=0x7f5f2c000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.697+0000 7f5f39308640 1 --2- 172.21.15.172:0/3964772199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5659a0 0x55933f565da0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009f60 tx=0x7f5f240095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f008f2ee76ea0ccb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.697+0000 7f5f1ffff640 1 -- 172.21.15.172:0/3964772199 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24017610 con 0x55933f5659a0 2024-09-10T08:49:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.697+0000 7f5f1ffff640 1 -- 172.21.15.172:0/3964772199 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f2401c070 con 0x55933f5659a0 2024-09-10T08:49:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.697+0000 7f5f1ffff640 1 -- 172.21.15.172:0/3964772199 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24015900 con 0x55933f5659a0 2024-09-10T08:49:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.698+0000 7f5f3ebec740 1 -- 172.21.15.172:0/3964772199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5659a0 msgr2=0x55933f565da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.698+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/3964772199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5659a0 0x55933f565da0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009f60 tx=0x7f5f240095d0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.698+0000 7f5f3ebec740 1 -- 172.21.15.172:0/3964772199 shutdown_connections 2024-09-10T08:49:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.698+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/3964772199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f56b370 0x55933f56d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.698+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/3964772199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f5662e0 0x55933f56ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.698+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/3964772199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5659a0 0x55933f565da0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.698+0000 7f5f3ebec740 1 -- 172.21.15.172:0/3964772199 >> 172.21.15.172:0/3964772199 conn(0x55933f561010 msgr2=0x55933f561410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.698+0000 7f5f3ebec740 1 -- 172.21.15.172:0/3964772199 shutdown_connections 2024-09-10T08:49:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.698+0000 7f5f3ebec740 1 -- 172.21.15.172:0/3964772199 wait complete. 2024-09-10T08:49:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.699+0000 7f5f3ebec740 1 -- start start 2024-09-10T08:49:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.699+0000 7f5f3ebec740 1 Processor -- start 2024-09-10T08:49:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.699+0000 7f5f3ebec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f5659a0 0x55933f56fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.699+0000 7f5f3ebec740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5662e0 0x55933f570010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.699+0000 7f5f3ebec740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f56b370 0x55933f577050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.699+0000 7f5f3ebec740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55933f56e2e0 con 0x55933f5662e0 2024-09-10T08:49:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.699+0000 7f5f3ebec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55933f452290 con 0x55933f5659a0 2024-09-10T08:49:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.699+0000 7f5f3ebec740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55933f56e460 con 0x55933f56b370 2024-09-10T08:49:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.699+0000 7f5f3ebec740 10 client.0 _ll_register_callbacks cb 0x55933f483140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.699+0000 7f5f39b09640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5662e0 0x55933f570010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.699+0000 7f5f39b09640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5662e0 0x55933f570010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48502/0 (socket says 172.21.15.172:48502) 2024-09-10T08:49:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.699+0000 7f5f39b09640 1 -- 172.21.15.172:0/1215622078 learned_addr learned my addr 172.21.15.172:0/1215622078 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.700+0000 7f5f39308640 1 --2- 172.21.15.172:0/1215622078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f56b370 0x55933f577050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.700+0000 7f5f39308640 1 --2- 172.21.15.172:0/1215622078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f5659a0 0x55933f56fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.700+0000 7f5f39b09640 1 -- 172.21.15.172:0/1215622078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f56b370 msgr2=0x55933f577050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.700+0000 7f5f39b09640 1 --2- 172.21.15.172:0/1215622078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f56b370 0x55933f577050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.700+0000 7f5f39b09640 1 -- 172.21.15.172:0/1215622078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f5659a0 msgr2=0x55933f56fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.700+0000 7f5f39b09640 1 --2- 172.21.15.172:0/1215622078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f5659a0 0x55933f56fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.700+0000 7f5f39b09640 1 -- 172.21.15.172:0/1215622078 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55933f1dc210 con 0x55933f5662e0 2024-09-10T08:49:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.700+0000 7f5f39308640 1 --2- 172.21.15.172:0/1215622078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f56b370 0x55933f577050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:49:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.700+0000 7f5f39b09640 1 --2- 172.21.15.172:0/1215622078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5662e0 0x55933f570010 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5f30005c40 tx=0x7f5f30003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b713534dd2b83e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.700+0000 7f5f1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.700+0000 7f5f1d7fa640 1 -- 172.21.15.172:0/1215622078 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30035040 con 0x55933f5662e0 2024-09-10T08:49:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.700+0000 7f5f1d7fa640 1 -- 172.21.15.172:0/1215622078 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5f30031580 con 0x55933f5662e0 2024-09-10T08:49:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.700+0000 7f5f3ebec740 1 -- 172.21.15.172:0/1215622078 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f2401c070 con 0x55933f5662e0 2024-09-10T08:49:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.700+0000 7f5f1d7fa640 1 -- 172.21.15.172:0/1215622078 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5f30031780 con 0x55933f5662e0 2024-09-10T08:49:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.700+0000 7f5f3ebec740 10 client.16272 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.700+0000 7f5f3ebec740 1 -- 172.21.15.172:0/1215622078 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55933f2a8620 con 0x55933f5662e0 2024-09-10T08:49:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.701+0000 7f5f1d7fa640 1 -- 172.21.15.172:0/1215622078 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f2401c070 con 0x55933f5662e0 2024-09-10T08:49:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.701+0000 7f5f1d7fa640 1 -- 172.21.15.172:0/1215622078 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f5f3003cac0 con 0x55933f5662e0 2024-09-10T08:49:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.701+0000 7f5f3ebec740 10 client.16272 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.701+0000 7f5f3ebec740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:49:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.701+0000 7f5f3ebec740 20 client.16272 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.701+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/1215622078 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55933f57b740 0x55933f57db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.701+0000 7f5f3ebec740 4 client.16272 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:49:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.701+0000 7f5f3ebec740 1 -- 172.21.15.172:0/1215622078 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55933f4d6a20 con 0x55933f57b740 2024-09-10T08:49:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.701+0000 7f5f39308640 1 --2- 172.21.15.172:0/1215622078 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55933f57b740 0x55933f57db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.702+0000 7f5f1d7fa640 10 client.16272 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.702+0000 7f5f39308640 1 --2- 172.21.15.172:0/1215622078 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55933f57b740 0x55933f57db70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1acdc8878d19aad0 server_cookie=39cdd95f8bbd72b2 in_seq=0 out_seq=0 2024-09-10T08:49:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.702+0000 7f5f1d7fa640 1 -- 172.21.15.172:0/1215622078 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55933f4d6a20 con 0x55933f57b740 2024-09-10T08:49:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.702+0000 7f5f1d7fa640 10 client.16272 handle_command_reply: tid=0 2024-09-10T08:49:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.702+0000 7f5f3ebec740 1 client.16272 shutdown 2024-09-10T08:49:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.702+0000 7f5f3ebec740 1 -- 172.21.15.172:0/1215622078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5662e0 msgr2=0x55933f570010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.702+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/1215622078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5662e0 0x55933f570010 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5f30005c40 tx=0x7f5f30003960 comp rx=0 tx=0).stop 2024-09-10T08:49:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.703+0000 7f5f3ebec740 1 -- 172.21.15.172:0/1215622078 shutdown_connections 2024-09-10T08:49:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.703+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/1215622078 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55933f57b740 0x55933f57db70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.703+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/1215622078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f56b370 0x55933f577050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.703+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/1215622078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5662e0 0x55933f570010 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.703+0000 7f5f1d7fa640 0 client.16272 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.703+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/1215622078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f5659a0 0x55933f56fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.703+0000 7f5f3ebec740 1 -- 172.21.15.172:0/1215622078 >> 172.21.15.172:0/1215622078 conn(0x55933f561010 msgr2=0x55933f562820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.703+0000 7f5f3ebec740 1 -- 172.21.15.172:0/1215622078 shutdown_connections 2024-09-10T08:49:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.703+0000 7f5f3ebec740 1 -- 172.21.15.172:0/1215622078 wait complete. 2024-09-10T08:49:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.703+0000 7f5f3ebec740 20 client.16272 trim_cache size 0 max 16384 2024-09-10T08:49:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.720+0000 7f5f3ebec740 1 Processor -- start 2024-09-10T08:49:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.720+0000 7f5f3ebec740 1 -- start start 2024-09-10T08:49:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.720+0000 7f5f3ebec740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5659a0 0x55933f5600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.720+0000 7f5f3ebec740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f5662e0 0x55933f577010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.720+0000 7f5f3ebec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f56b370 0x55933f582470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.720+0000 7f5f3ebec740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341aa530 con 0x55933f5659a0 2024-09-10T08:49:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.720+0000 7f5f3ebec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341ac0d0 con 0x55933f56b370 2024-09-10T08:49:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.720+0000 7f5f3ebec740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341aac00 con 0x55933f5662e0 2024-09-10T08:49:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.721+0000 7f5f39308640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5659a0 0x55933f5600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.721+0000 7f5f39308640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5659a0 0x55933f5600b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48508/0 (socket says 172.21.15.172:48508) 2024-09-10T08:49:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.721+0000 7f5f39308640 1 -- 172.21.15.172:0/293258663 learned_addr learned my addr 172.21.15.172:0/293258663 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.721+0000 7f5f39b09640 1 --2- 172.21.15.172:0/293258663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f5662e0 0x55933f577010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.721+0000 7f5f39308640 1 --2- 172.21.15.172:0/293258663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f56b370 0x55933f582470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.721+0000 7f5f39308640 1 -- 172.21.15.172:0/293258663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f5662e0 msgr2=0x55933f577010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.721+0000 7f5f39308640 1 --2- 172.21.15.172:0/293258663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f5662e0 0x55933f577010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.721+0000 7f5f39308640 1 -- 172.21.15.172:0/293258663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f56b370 msgr2=0x55933f582470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.721+0000 7f5f39308640 1 --2- 172.21.15.172:0/293258663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f56b370 0x55933f582470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.721+0000 7f5f39308640 1 -- 172.21.15.172:0/293258663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55933f5667c0 con 0x55933f5659a0 2024-09-10T08:49:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.721+0000 7f5f39b09640 1 --2- 172.21.15.172:0/293258663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f5662e0 0x55933f577010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:49:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.721+0000 7f5f39308640 1 --2- 172.21.15.172:0/293258663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5659a0 0x55933f5600b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5f24011010 tx=0x7f5f24009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696ed9323332de7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.721+0000 7f5f1f7fe640 1 -- 172.21.15.172:0/293258663 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2402b040 con 0x55933f5659a0 2024-09-10T08:49:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.721+0000 7f5f1f7fe640 1 -- 172.21.15.172:0/293258663 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f2401c040 con 0x55933f5659a0 2024-09-10T08:49:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.721+0000 7f5f1f7fe640 1 -- 172.21.15.172:0/293258663 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24021400 con 0x55933f5659a0 2024-09-10T08:49:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.722+0000 7f5f3ebec740 1 -- 172.21.15.172:0/293258663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5659a0 msgr2=0x55933f5600b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.722+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/293258663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5659a0 0x55933f5600b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5f24011010 tx=0x7f5f24009f90 comp rx=0 tx=0).stop 2024-09-10T08:49:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.722+0000 7f5f3ebec740 1 -- 172.21.15.172:0/293258663 shutdown_connections 2024-09-10T08:49:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.722+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/293258663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f56b370 0x55933f582470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.722+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/293258663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f5662e0 0x55933f577010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.722+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/293258663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5659a0 0x55933f5600b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.722+0000 7f5f3ebec740 1 -- 172.21.15.172:0/293258663 >> 172.21.15.172:0/293258663 conn(0x55933f561010 msgr2=0x55933f55f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.722+0000 7f5f3ebec740 1 -- 172.21.15.172:0/293258663 shutdown_connections 2024-09-10T08:49:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.722+0000 7f5f3ebec740 1 -- 172.21.15.172:0/293258663 wait complete. 2024-09-10T08:49:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.723+0000 7f5f3ebec740 1 -- start start 2024-09-10T08:49:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.723+0000 7f5f3ebec740 1 Processor -- start 2024-09-10T08:49:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.723+0000 7f5f3ebec740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f5659a0 0x55933f574420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.723+0000 7f5f3ebec740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5662e0 0x55933f574960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.723+0000 7f5f3ebec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f56b370 0x55933f575920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.723+0000 7f5f3ebec740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341ab9a0 con 0x55933f5662e0 2024-09-10T08:49:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.723+0000 7f5f3ebec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341ab2d0 con 0x55933f56b370 2024-09-10T08:49:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.723+0000 7f5f3ebec740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5f341a8510 con 0x55933f5659a0 2024-09-10T08:49:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.723+0000 7f5f3ebec740 10 client.0 _ll_register_callbacks cb 0x7f5f300043f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.723+0000 7f5f39b09640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5662e0 0x55933f574960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.723+0000 7f5f39b09640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5662e0 0x55933f574960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48520/0 (socket says 172.21.15.172:48520) 2024-09-10T08:49:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.723+0000 7f5f39b09640 1 -- 172.21.15.172:0/2118592366 learned_addr learned my addr 172.21.15.172:0/2118592366 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.723+0000 7f5f39308640 1 --2- 172.21.15.172:0/2118592366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f5659a0 0x55933f574420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.723+0000 7f5f39308640 1 --2- 172.21.15.172:0/2118592366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f56b370 0x55933f575920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f39b09640 1 -- 172.21.15.172:0/2118592366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f5659a0 msgr2=0x55933f574420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f39b09640 1 --2- 172.21.15.172:0/2118592366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f5659a0 0x55933f574420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f39b09640 1 -- 172.21.15.172:0/2118592366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f56b370 msgr2=0x55933f575920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f39b09640 1 --2- 172.21.15.172:0/2118592366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f56b370 0x55933f575920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f39b09640 1 -- 172.21.15.172:0/2118592366 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55933f575f30 con 0x55933f5662e0 2024-09-10T08:49:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f39308640 1 --2- 172.21.15.172:0/2118592366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f56b370 0x55933f575920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:49:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f39b09640 1 --2- 172.21.15.172:0/2118592366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5662e0 0x55933f574960 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5f3002a930 tx=0x7f5f30009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc952133531bc204 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f1d7fa640 1 -- 172.21.15.172:0/2118592366 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30035040 con 0x55933f5662e0 2024-09-10T08:49:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f1d7fa640 1 -- 172.21.15.172:0/2118592366 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5f30031b30 con 0x55933f5662e0 2024-09-10T08:49:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f3ebec740 1 -- 172.21.15.172:0/2118592366 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f2401c040 con 0x55933f5662e0 2024-09-10T08:49:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f1d7fa640 1 -- 172.21.15.172:0/2118592366 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5f3005a050 con 0x55933f5662e0 2024-09-10T08:49:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f3ebec740 10 client.16278 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f3ebec740 1 -- 172.21.15.172:0/2118592366 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55933f577240 con 0x55933f5662e0 2024-09-10T08:49:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f1d7fa640 1 -- 172.21.15.172:0/2118592366 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f2401c040 con 0x55933f5662e0 2024-09-10T08:49:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f1d7fa640 1 -- 172.21.15.172:0/2118592366 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f5f300052c0 con 0x55933f5662e0 2024-09-10T08:49:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f3ebec740 10 client.16278 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f3ebec740 10 client.16278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:49:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.724+0000 7f5f3ebec740 20 client.16278 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.725+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/2118592366 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55933f5e6af0 0x55933f5e8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.725+0000 7f5f3ebec740 4 client.16278 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:49:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.725+0000 7f5f3ebec740 1 -- 172.21.15.172:0/2118592366 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55933f4d0d70 con 0x55933f5e6af0 2024-09-10T08:49:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.725+0000 7f5f39308640 1 --2- 172.21.15.172:0/2118592366 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55933f5e6af0 0x55933f5e8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.725+0000 7f5f1d7fa640 10 client.16278 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.725+0000 7f5f39308640 1 --2- 172.21.15.172:0/2118592366 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55933f5e6af0 0x55933f5e8f10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=673752d3a561ddc1 server_cookie=4d32ee2de09bb915 in_seq=0 out_seq=0 2024-09-10T08:49:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.739+0000 7f5f1d7fa640 1 -- 172.21.15.172:0/2118592366 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55933f4d0d70 con 0x55933f5e6af0 2024-09-10T08:49:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.739+0000 7f5f1d7fa640 10 client.16278 handle_command_reply: tid=0 2024-09-10T08:49:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.739+0000 7f5f3ebec740 1 client.16278 shutdown 2024-09-10T08:49:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.739+0000 7f5f3ebec740 1 -- 172.21.15.172:0/2118592366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5662e0 msgr2=0x55933f574960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.739+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/2118592366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5662e0 0x55933f574960 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5f3002a930 tx=0x7f5f30009230 comp rx=0 tx=0).stop 2024-09-10T08:49:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.739+0000 7f5f3ebec740 1 -- 172.21.15.172:0/2118592366 shutdown_connections 2024-09-10T08:49:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.739+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/2118592366 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55933f5e6af0 0x55933f5e8f10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.739+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/2118592366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55933f56b370 0x55933f575920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.739+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/2118592366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55933f5662e0 0x55933f574960 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.739+0000 7f5f1d7fa640 0 client.16278 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.739+0000 7f5f3ebec740 1 --2- 172.21.15.172:0/2118592366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55933f5659a0 0x55933f574420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.739+0000 7f5f3ebec740 1 -- 172.21.15.172:0/2118592366 >> 172.21.15.172:0/2118592366 conn(0x55933f561010 msgr2=0x55933f561f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.739+0000 7f5f3ebec740 1 -- 172.21.15.172:0/2118592366 shutdown_connections 2024-09-10T08:49:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.739+0000 7f5f3ebec740 1 -- 172.21.15.172:0/2118592366 wait complete. 2024-09-10T08:49:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.739+0000 7f5f3ebec740 20 client.16278 trim_cache size 0 max 16384 2024-09-10T08:49:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:49:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:49:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:49:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:49:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.740+0000 7f5f3bd6d640 1 -- 172.21.15.172:0/3523370747 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f200a2c30 msgr2=0x7f5f200a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.740+0000 7f5f3bd6d640 1 --2- 172.21.15.172:0/3523370747 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f200a2c30 0x7f5f200a50f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0ccf60 tx=0x7f5f2c000f40 comp rx=0 tx=0).stop 2024-09-10T08:49:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.740+0000 7f5f3bd6d640 1 -- 172.21.15.172:0/3523370747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f341025f0 msgr2=0x7f5f3419f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.740+0000 7f5f3bd6d640 1 --2- 172.21.15.172:0/3523370747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f341025f0 0x7f5f3419f0c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00eaa0 tx=0x7f5f2c00eea0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.740+0000 7f5f3bd6d640 1 -- 172.21.15.172:0/3523370747 shutdown_connections 2024-09-10T08:49:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.740+0000 7f5f3bd6d640 1 --2- 172.21.15.172:0/3523370747 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5f200a2c30 0x7f5f200a50f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.740+0000 7f5f3bd6d640 1 --2- 172.21.15.172:0/3523370747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f341025f0 0x7f5f3419f0c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.740+0000 7f5f3bd6d640 1 --2- 172.21.15.172:0/3523370747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f34101c30 0x7f5f3419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.740+0000 7f5f3bd6d640 1 --2- 172.21.15.172:0/3523370747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f34100a40 0x7f5f3406a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.740+0000 7f5f3bd6d640 1 -- 172.21.15.172:0/3523370747 >> 172.21.15.172:0/3523370747 conn(0x7f5f340fc1e0 msgr2=0x7f5f34104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.740+0000 7f5f3bd6d640 1 -- 172.21.15.172:0/3523370747 shutdown_connections 2024-09-10T08:49:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:09.740+0000 7f5f3bd6d640 1 -- 172.21.15.172:0/3523370747 wait complete. 2024-09-10T08:49:10.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.079+0000 7fb18c835640 1 -- 172.21.15.172:0/3359904663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb188102780 msgr2=0x7fb1881067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:10.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.079+0000 7fb18c835640 1 --2- 172.21.15.172:0/3359904663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb188102780 0x7fb1881067d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009900 tx=0x7fb17c02a410 comp rx=0 tx=0).stop 2024-09-10T08:49:10.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.079+0000 7fb18c835640 1 -- 172.21.15.172:0/3359904663 shutdown_connections 2024-09-10T08:49:10.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.079+0000 7fb18c835640 1 --2- 172.21.15.172:0/3359904663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb188106f00 0x7fb18810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.079+0000 7fb18c835640 1 --2- 172.21.15.172:0/3359904663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb188102780 0x7fb1881067d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.079+0000 7fb18c835640 1 --2- 172.21.15.172:0/3359904663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb188101dd0 0x7fb1881021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.079+0000 7fb18c835640 1 -- 172.21.15.172:0/3359904663 >> 172.21.15.172:0/3359904663 conn(0x7fb1880fd890 msgr2=0x7fb1880ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:10.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.079+0000 7fb18c835640 1 -- 172.21.15.172:0/3359904663 shutdown_connections 2024-09-10T08:49:10.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.079+0000 7fb18c835640 1 -- 172.21.15.172:0/3359904663 wait complete. 2024-09-10T08:49:10.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.080+0000 7fb18c835640 1 Processor -- start 2024-09-10T08:49:10.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.080+0000 7fb18c835640 1 -- start start 2024-09-10T08:49:10.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.080+0000 7fb18c835640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb188101dd0 0x7fb18819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.080+0000 7fb18c835640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb188102780 0x7fb18819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.080+0000 7fb18c835640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb188106f00 0x7fb18819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.080+0000 7fb18c835640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb188111e10 con 0x7fb188106f00 2024-09-10T08:49:10.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.080+0000 7fb18c835640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb188111c90 con 0x7fb188102780 2024-09-10T08:49:10.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.080+0000 7fb18c835640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb188111f90 con 0x7fb188101dd0 2024-09-10T08:49:10.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.080+0000 7fb186d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb188106f00 0x7fb18819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:10.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.080+0000 7fb186d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb188106f00 0x7fb18819ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48544/0 (socket says 172.21.15.172:48544) 2024-09-10T08:49:10.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.080+0000 7fb186d9d640 1 -- 172.21.15.172:0/828642992 learned_addr learned my addr 172.21.15.172:0/828642992 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:10.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.081+0000 7fb18659c640 1 --2- 172.21.15.172:0/828642992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb188101dd0 0x7fb18819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:10.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.081+0000 7fb185d9b640 1 --2- 172.21.15.172:0/828642992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb188102780 0x7fb18819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:10.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.081+0000 7fb186d9d640 1 -- 172.21.15.172:0/828642992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb188101dd0 msgr2=0x7fb18819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:10.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.081+0000 7fb186d9d640 1 --2- 172.21.15.172:0/828642992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb188101dd0 0x7fb18819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.081+0000 7fb186d9d640 1 -- 172.21.15.172:0/828642992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb188102780 msgr2=0x7fb18819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:10.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.081+0000 7fb186d9d640 1 --2- 172.21.15.172:0/828642992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb188102780 0x7fb18819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.081+0000 7fb186d9d640 1 -- 172.21.15.172:0/828642992 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18819f540 con 0x7fb188106f00 2024-09-10T08:49:10.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.081+0000 7fb186d9d640 1 --2- 172.21.15.172:0/828642992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb188106f00 0x7fb18819ee60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb18000eaa0 tx=0x7fb18000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:10.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.081+0000 7fb1777fe640 1 -- 172.21.15.172:0/828642992 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18000cdb0 con 0x7fb188106f00 2024-09-10T08:49:10.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.081+0000 7fb18c835640 1 -- 172.21.15.172:0/828642992 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18819f830 con 0x7fb188106f00 2024-09-10T08:49:10.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.081+0000 7fb1777fe640 1 -- 172.21.15.172:0/828642992 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb180005020 con 0x7fb188106f00 2024-09-10T08:49:10.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.081+0000 7fb1777fe640 1 -- 172.21.15.172:0/828642992 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1800116d0 con 0x7fb188106f00 2024-09-10T08:49:10.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.082+0000 7fb18c835640 1 -- 172.21.15.172:0/828642992 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1881a7110 con 0x7fb188106f00 2024-09-10T08:49:10.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.084+0000 7fb1777fe640 1 -- 172.21.15.172:0/828642992 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb1800051d0 con 0x7fb188106f00 2024-09-10T08:49:10.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.085+0000 7fb1777fe640 1 --2- 172.21.15.172:0/828642992 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb1700a2b80 0x7fb1700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.085+0000 7fb18659c640 1 --2- 172.21.15.172:0/828642992 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb1700a2b80 0x7fb1700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:10.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.085+0000 7fb1777fe640 1 -- 172.21.15.172:0/828642992 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb1800cf430 con 0x7fb188106f00 2024-09-10T08:49:10.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.085+0000 7fb18ec0e740 1 Processor -- start 2024-09-10T08:49:10.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.085+0000 7fb18ec0e740 1 -- start start 2024-09-10T08:49:10.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.085+0000 7fb18659c640 1 --2- 172.21.15.172:0/828642992 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb1700a2b80 0x7fb1700a5040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb178009bd0 tx=0x7fb178005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:10.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.085+0000 7fb18ec0e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb6404a0 0x55bacb640880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.085+0000 7fb18ec0e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb640dc0 0x55bacb64b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.086+0000 7fb18ec0e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb64b680 0x55bacb64da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.086+0000 7fb18ec0e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bacb5a62e0 con 0x55bacb64b680 2024-09-10T08:49:10.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.086+0000 7fb18ec0e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bacb54dce0 con 0x55bacb640dc0 2024-09-10T08:49:10.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.086+0000 7fb18ec0e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bacb5a22a0 con 0x55bacb6404a0 2024-09-10T08:49:10.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.086+0000 7fb186d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb64b680 0x55bacb64da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:10.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.086+0000 7fb186d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb64b680 0x55bacb64da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48548/0 (socket says 172.21.15.172:48548) 2024-09-10T08:49:10.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.086+0000 7fb186d9d640 1 -- 172.21.15.172:0/328335692 learned_addr learned my addr 172.21.15.172:0/328335692 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:10.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.086+0000 7fb18659c640 1 --2- 172.21.15.172:0/328335692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb6404a0 0x55bacb640880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:10.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.086+0000 7fb186d9d640 1 -- 172.21.15.172:0/328335692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb6404a0 msgr2=0x55bacb640880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:10.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.086+0000 7fb186d9d640 1 --2- 172.21.15.172:0/328335692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb6404a0 0x55bacb640880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.086+0000 7fb186d9d640 1 -- 172.21.15.172:0/328335692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb640dc0 msgr2=0x55bacb64b140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:49:10.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.087+0000 7fb185d9b640 1 --2- 172.21.15.172:0/328335692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb640dc0 0x55bacb64b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:10.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.087+0000 7fb186d9d640 1 --2- 172.21.15.172:0/328335692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb640dc0 0x55bacb64b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.087+0000 7fb186d9d640 1 -- 172.21.15.172:0/328335692 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bacb532440 con 0x55bacb64b680 2024-09-10T08:49:10.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.087+0000 7fb186d9d640 1 --2- 172.21.15.172:0/328335692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb64b680 0x55bacb64da70 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb18009b970 tx=0x7fb18009baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f080d0156fc4ae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:10.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.087+0000 7fb174ff9640 1 -- 172.21.15.172:0/328335692 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18001f040 con 0x55bacb64b680 2024-09-10T08:49:10.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.087+0000 7fb174ff9640 1 -- 172.21.15.172:0/328335692 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1800afe50 con 0x55bacb64b680 2024-09-10T08:49:10.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.087+0000 7fb174ff9640 1 -- 172.21.15.172:0/328335692 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1800860e0 con 0x55bacb64b680 2024-09-10T08:49:10.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.088+0000 7fb18ec0e740 1 -- 172.21.15.172:0/328335692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb64b680 msgr2=0x55bacb64da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:10.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.088+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/328335692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb64b680 0x55bacb64da70 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb18009b970 tx=0x7fb18009baa0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.088+0000 7fb18ec0e740 1 -- 172.21.15.172:0/328335692 shutdown_connections 2024-09-10T08:49:10.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.088+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/328335692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb64b680 0x55bacb64da70 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.088+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/328335692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb640dc0 0x55bacb64b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.088+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/328335692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb6404a0 0x55bacb640880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.088+0000 7fb18ec0e740 1 -- 172.21.15.172:0/328335692 >> 172.21.15.172:0/328335692 conn(0x55bacb63ca30 msgr2=0x55bacb63ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:10.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.088+0000 7fb18ec0e740 1 -- 172.21.15.172:0/328335692 shutdown_connections 2024-09-10T08:49:10.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.088+0000 7fb18ec0e740 1 -- 172.21.15.172:0/328335692 wait complete. 2024-09-10T08:49:10.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.089+0000 7fb18ec0e740 1 -- start start 2024-09-10T08:49:10.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.089+0000 7fb18ec0e740 1 Processor -- start 2024-09-10T08:49:10.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.089+0000 7fb18ec0e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb6404a0 0x55bacb64b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.089+0000 7fb18ec0e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb640dc0 0x55bacb6482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.089+0000 7fb18ec0e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb64b680 0x55bacb648830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.089+0000 7fb18659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb6404a0 0x55bacb64b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:10.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.089+0000 7fb18659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb6404a0 0x55bacb64b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48552/0 (socket says 172.21.15.172:48552) 2024-09-10T08:49:10.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.089+0000 7fb18659c640 1 -- 172.21.15.172:0/1639205810 learned_addr learned my addr 172.21.15.172:0/1639205810 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:10.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.089+0000 7fb18ec0e740 1 -- 172.21.15.172:0/1639205810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bacb64e250 con 0x55bacb6404a0 2024-09-10T08:49:10.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.089+0000 7fb18ec0e740 1 -- 172.21.15.172:0/1639205810 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bacb561050 con 0x55bacb64b680 2024-09-10T08:49:10.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.089+0000 7fb185d9b640 1 --2- 172.21.15.172:0/1639205810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb640dc0 0x55bacb6482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:10.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.090+0000 7fb186d9d640 1 --2- 172.21.15.172:0/1639205810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb64b680 0x55bacb648830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:10.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.090+0000 7fb18ec0e740 1 -- 172.21.15.172:0/1639205810 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bacb64e3d0 con 0x55bacb640dc0 2024-09-10T08:49:10.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.090+0000 7fb18ec0e740 10 client.0 _ll_register_callbacks cb 0x55bacb5a1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:10.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.090+0000 7fb18659c640 1 -- 172.21.15.172:0/1639205810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb640dc0 msgr2=0x55bacb6482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:10.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.090+0000 7fb18659c640 1 --2- 172.21.15.172:0/1639205810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb640dc0 0x55bacb6482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.090+0000 7fb18659c640 1 -- 172.21.15.172:0/1639205810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb64b680 msgr2=0x55bacb648830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:10.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.090+0000 7fb18659c640 1 --2- 172.21.15.172:0/1639205810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb64b680 0x55bacb648830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.090+0000 7fb18659c640 1 -- 172.21.15.172:0/1639205810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bacb3173c0 con 0x55bacb6404a0 2024-09-10T08:49:10.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.090+0000 7fb185d9b640 1 --2- 172.21.15.172:0/1639205810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb640dc0 0x55bacb6482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:10.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.090+0000 7fb18659c640 1 --2- 172.21.15.172:0/1639205810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb6404a0 0x55bacb64b000 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb178003950 tx=0x7fb178004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8caf7a2a77cc64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:10.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.090+0000 7fb16dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:10.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.090+0000 7fb16dffb640 1 -- 172.21.15.172:0/1639205810 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17800e6a0 con 0x55bacb6404a0 2024-09-10T08:49:10.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.090+0000 7fb16dffb640 1 -- 172.21.15.172:0/1639205810 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb17800e840 con 0x55bacb6404a0 2024-09-10T08:49:10.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.090+0000 7fb18ec0e740 1 -- 172.21.15.172:0/1639205810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1800afe50 con 0x55bacb6404a0 2024-09-10T08:49:10.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.091+0000 7fb16dffb640 1 -- 172.21.15.172:0/1639205810 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb17800ea40 con 0x55bacb6404a0 2024-09-10T08:49:10.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.091+0000 7fb18ec0e740 10 client.16290 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:10.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.091+0000 7fb18ec0e740 1 -- 172.21.15.172:0/1639205810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bacb659c60 con 0x55bacb6404a0 2024-09-10T08:49:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.091+0000 7fb16dffb640 1 -- 172.21.15.172:0/1639205810 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1800afe50 con 0x55bacb6404a0 2024-09-10T08:49:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.091+0000 7fb16dffb640 1 -- 172.21.15.172:0/1639205810 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fb178027420 con 0x55bacb6404a0 2024-09-10T08:49:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.091+0000 7fb18ec0e740 10 client.16290 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.091+0000 7fb18ec0e740 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:49:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.091+0000 7fb18ec0e740 20 client.16290 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.091+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/1639205810 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bacb65b0c0 0x55bacb65d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.091+0000 7fb18ec0e740 4 client.16290 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:49:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.091+0000 7fb18ec0e740 1 -- 172.21.15.172:0/1639205810 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bacb5b3850 con 0x55bacb65b0c0 2024-09-10T08:49:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.091+0000 7fb185d9b640 1 --2- 172.21.15.172:0/1639205810 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bacb65b0c0 0x55bacb65d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.092+0000 7fb16dffb640 10 client.16290 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.092+0000 7fb185d9b640 1 --2- 172.21.15.172:0/1639205810 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bacb65b0c0 0x55bacb65d530 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1e13c764d11c40 server_cookie=a6d9205f5656b0cb in_seq=0 out_seq=0 2024-09-10T08:49:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.093+0000 7fb16dffb640 1 -- 172.21.15.172:0/1639205810 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bacb5b3850 con 0x55bacb65b0c0 2024-09-10T08:49:10.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.093+0000 7fb16dffb640 10 client.16290 handle_command_reply: tid=0 2024-09-10T08:49:10.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.093+0000 7fb18ec0e740 1 client.16290 shutdown 2024-09-10T08:49:10.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.093+0000 7fb18ec0e740 1 -- 172.21.15.172:0/1639205810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb6404a0 msgr2=0x55bacb64b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:10.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.093+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/1639205810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb6404a0 0x55bacb64b000 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb178003950 tx=0x7fb178004730 comp rx=0 tx=0).stop 2024-09-10T08:49:10.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.093+0000 7fb18ec0e740 1 -- 172.21.15.172:0/1639205810 shutdown_connections 2024-09-10T08:49:10.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.093+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/1639205810 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bacb65b0c0 0x55bacb65d530 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.093+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/1639205810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb64b680 0x55bacb648830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.093+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/1639205810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb640dc0 0x55bacb6482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.093+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/1639205810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb6404a0 0x55bacb64b000 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.093+0000 7fb16dffb640 0 client.16290 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:10.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.093+0000 7fb18ec0e740 1 -- 172.21.15.172:0/1639205810 >> 172.21.15.172:0/1639205810 conn(0x55bacb63ca30 msgr2=0x55bacb642710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:10.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.093+0000 7fb18ec0e740 1 -- 172.21.15.172:0/1639205810 shutdown_connections 2024-09-10T08:49:10.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.093+0000 7fb18ec0e740 1 -- 172.21.15.172:0/1639205810 wait complete. 2024-09-10T08:49:10.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.093+0000 7fb18ec0e740 20 client.16290 trim_cache size 0 max 16384 2024-09-10T08:49:10.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb18ec0e740 1 Processor -- start 2024-09-10T08:49:10.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb18ec0e740 1 -- start start 2024-09-10T08:49:10.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb18ec0e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb6404a0 0x55bacb6574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb18ec0e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb640dc0 0x55bacb662e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb18ec0e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb64b680 0x55bacb663340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb18ec0e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb1881b0dd0 con 0x55bacb640dc0 2024-09-10T08:49:10.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb18ec0e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb1881b2970 con 0x55bacb6404a0 2024-09-10T08:49:10.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb18ec0e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb1881b14a0 con 0x55bacb64b680 2024-09-10T08:49:10.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb185d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb640dc0 0x55bacb662e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:10.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb185d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb640dc0 0x55bacb662e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48568/0 (socket says 172.21.15.172:48568) 2024-09-10T08:49:10.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb186d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb64b680 0x55bacb663340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:10.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb185d9b640 1 -- 172.21.15.172:0/1927069954 learned_addr learned my addr 172.21.15.172:0/1927069954 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:10.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb18659c640 1 --2- 172.21.15.172:0/1927069954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb6404a0 0x55bacb6574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:10.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb185d9b640 1 -- 172.21.15.172:0/1927069954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb64b680 msgr2=0x55bacb663340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:10.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb185d9b640 1 --2- 172.21.15.172:0/1927069954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb64b680 0x55bacb663340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb185d9b640 1 -- 172.21.15.172:0/1927069954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb6404a0 msgr2=0x55bacb6574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:10.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb185d9b640 1 --2- 172.21.15.172:0/1927069954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb6404a0 0x55bacb6574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb185d9b640 1 -- 172.21.15.172:0/1927069954 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bacb648ba0 con 0x55bacb640dc0 2024-09-10T08:49:10.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.111+0000 7fb186d9d640 1 --2- 172.21.15.172:0/1927069954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb64b680 0x55bacb663340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:10.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.112+0000 7fb185d9b640 1 --2- 172.21.15.172:0/1927069954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb640dc0 0x55bacb662e00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009900 tx=0x7fb17c005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6daefeafc0eb25fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:10.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.112+0000 7fb16ffff640 1 -- 172.21.15.172:0/1927069954 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c035040 con 0x55bacb640dc0 2024-09-10T08:49:10.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.112+0000 7fb16ffff640 1 -- 172.21.15.172:0/1927069954 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb17c0054f0 con 0x55bacb640dc0 2024-09-10T08:49:10.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.112+0000 7fb16ffff640 1 -- 172.21.15.172:0/1927069954 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c004c80 con 0x55bacb640dc0 2024-09-10T08:49:10.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.113+0000 7fb18ec0e740 1 -- 172.21.15.172:0/1927069954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb640dc0 msgr2=0x55bacb662e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:10.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.113+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/1927069954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb640dc0 0x55bacb662e00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009900 tx=0x7fb17c005890 comp rx=0 tx=0).stop 2024-09-10T08:49:10.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.113+0000 7fb18ec0e740 1 -- 172.21.15.172:0/1927069954 shutdown_connections 2024-09-10T08:49:10.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.113+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/1927069954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb64b680 0x55bacb663340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.113+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/1927069954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb640dc0 0x55bacb662e00 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.113+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/1927069954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb6404a0 0x55bacb6574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.113+0000 7fb18ec0e740 1 -- 172.21.15.172:0/1927069954 >> 172.21.15.172:0/1927069954 conn(0x55bacb63ca30 msgr2=0x55bacb647a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:10.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.113+0000 7fb18ec0e740 1 -- 172.21.15.172:0/1927069954 shutdown_connections 2024-09-10T08:49:10.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.113+0000 7fb18ec0e740 1 -- 172.21.15.172:0/1927069954 wait complete. 2024-09-10T08:49:10.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.113+0000 7fb18ec0e740 1 -- start start 2024-09-10T08:49:10.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.113+0000 7fb18ec0e740 1 Processor -- start 2024-09-10T08:49:10.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.113+0000 7fb18ec0e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb6404a0 0x55bacb64eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb18ec0e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb640dc0 0x55bacb64f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb18ec0e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb64b680 0x55bacb6bdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb18ec0e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb1881b2240 con 0x55bacb6404a0 2024-09-10T08:49:10.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb18ec0e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb1881b1b70 con 0x55bacb640dc0 2024-09-10T08:49:10.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb18ec0e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb1881aedb0 con 0x55bacb64b680 2024-09-10T08:49:10.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb18659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb6404a0 0x55bacb64eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:10.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb18659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb6404a0 0x55bacb64eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48572/0 (socket says 172.21.15.172:48572) 2024-09-10T08:49:10.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb18659c640 1 -- 172.21.15.172:0/4237687942 learned_addr learned my addr 172.21.15.172:0/4237687942 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:10.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb185d9b640 1 --2- 172.21.15.172:0/4237687942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb640dc0 0x55bacb64f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:10.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb18ec0e740 10 client.0 _ll_register_callbacks cb 0x7fb178004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:10.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb18659c640 1 -- 172.21.15.172:0/4237687942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb64b680 msgr2=0x55bacb6bdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:10.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb18659c640 1 --2- 172.21.15.172:0/4237687942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb64b680 0x55bacb6bdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb18659c640 1 -- 172.21.15.172:0/4237687942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb640dc0 msgr2=0x55bacb64f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:10.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb18659c640 1 --2- 172.21.15.172:0/4237687942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb640dc0 0x55bacb64f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb18659c640 1 -- 172.21.15.172:0/4237687942 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bacb6be5a0 con 0x55bacb6404a0 2024-09-10T08:49:10.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb18659c640 1 --2- 172.21.15.172:0/4237687942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb6404a0 0x55bacb64eef0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb1780119c0 tx=0x7fb1780276d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab9d77e72ffbbc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:10.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb16dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:10.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb16dffb640 1 -- 172.21.15.172:0/4237687942 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17800edf0 con 0x55bacb6404a0 2024-09-10T08:49:10.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb18ec0e740 1 -- 172.21.15.172:0/4237687942 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb17c0054f0 con 0x55bacb6404a0 2024-09-10T08:49:10.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.114+0000 7fb16dffb640 1 -- 172.21.15.172:0/4237687942 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb178009040 con 0x55bacb6404a0 2024-09-10T08:49:10.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.115+0000 7fb16dffb640 1 -- 172.21.15.172:0/4237687942 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb178044050 con 0x55bacb6404a0 2024-09-10T08:49:10.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.115+0000 7fb18ec0e740 10 client.16296 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:10.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.115+0000 7fb18ec0e740 1 -- 172.21.15.172:0/4237687942 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bacb6c5820 con 0x55bacb6404a0 2024-09-10T08:49:10.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.115+0000 7fb16dffb640 1 -- 172.21.15.172:0/4237687942 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb17c0054f0 con 0x55bacb6404a0 2024-09-10T08:49:10.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.115+0000 7fb16dffb640 1 -- 172.21.15.172:0/4237687942 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fb17803ba50 con 0x55bacb6404a0 2024-09-10T08:49:10.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.115+0000 7fb18ec0e740 10 client.16296 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:10.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.115+0000 7fb18ec0e740 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:49:10.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.115+0000 7fb18ec0e740 20 client.16296 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:10.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.115+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/4237687942 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bacb65b0c0 0x55bacb6c8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:10.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.115+0000 7fb18ec0e740 4 client.16296 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:49:10.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.115+0000 7fb18ec0e740 1 -- 172.21.15.172:0/4237687942 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bacb5add60 con 0x55bacb65b0c0 2024-09-10T08:49:10.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.115+0000 7fb185d9b640 1 --2- 172.21.15.172:0/4237687942 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bacb65b0c0 0x55bacb6c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:10.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.116+0000 7fb185d9b640 1 --2- 172.21.15.172:0/4237687942 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bacb65b0c0 0x55bacb6c8070 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2644cd5ec4fb635e server_cookie=94a63bec88b9bbe2 in_seq=0 out_seq=0 2024-09-10T08:49:10.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.116+0000 7fb16dffb640 10 client.16296 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:10.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.116+0000 7fb16dffb640 1 -- 172.21.15.172:0/4237687942 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bacb5add60 con 0x55bacb65b0c0 2024-09-10T08:49:10.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.116+0000 7fb16dffb640 10 client.16296 handle_command_reply: tid=0 2024-09-10T08:49:10.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.116+0000 7fb18ec0e740 1 client.16296 shutdown 2024-09-10T08:49:10.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.116+0000 7fb18ec0e740 1 -- 172.21.15.172:0/4237687942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb6404a0 msgr2=0x55bacb64eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:10.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.116+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/4237687942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb6404a0 0x55bacb64eef0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb1780119c0 tx=0x7fb1780276d0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.116+0000 7fb18ec0e740 1 -- 172.21.15.172:0/4237687942 shutdown_connections 2024-09-10T08:49:10.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.116+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/4237687942 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bacb65b0c0 0x55bacb6c8070 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.116+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/4237687942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bacb64b680 0x55bacb6bdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.116+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/4237687942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bacb640dc0 0x55bacb64f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.116+0000 7fb18ec0e740 1 --2- 172.21.15.172:0/4237687942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bacb6404a0 0x55bacb64eef0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.116+0000 7fb18ec0e740 1 -- 172.21.15.172:0/4237687942 >> 172.21.15.172:0/4237687942 conn(0x55bacb63ca30 msgr2=0x55bacb643050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:10.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.116+0000 7fb16dffb640 0 client.16296 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:10.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.116+0000 7fb18ec0e740 1 -- 172.21.15.172:0/4237687942 shutdown_connections 2024-09-10T08:49:10.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.116+0000 7fb18ec0e740 1 -- 172.21.15.172:0/4237687942 wait complete. 2024-09-10T08:49:10.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.116+0000 7fb18ec0e740 20 client.16296 trim_cache size 0 max 16384 2024-09-10T08:49:10.125 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:49:10.125 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T08:49:10.125 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T08:49:10.125 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:49:10.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.117+0000 7fb18c835640 1 -- 172.21.15.172:0/828642992 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb1700a2b80 msgr2=0x7fb1700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:10.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.117+0000 7fb18c835640 1 --2- 172.21.15.172:0/828642992 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb1700a2b80 0x7fb1700a5040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb178009bd0 tx=0x7fb178005f90 comp rx=0 tx=0).stop 2024-09-10T08:49:10.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.117+0000 7fb18c835640 1 -- 172.21.15.172:0/828642992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb188106f00 msgr2=0x7fb18819ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:10.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.117+0000 7fb18c835640 1 --2- 172.21.15.172:0/828642992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb188106f00 0x7fb18819ee60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb18000eaa0 tx=0x7fb18000ef60 comp rx=0 tx=0).stop 2024-09-10T08:49:10.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.117+0000 7fb18c835640 1 -- 172.21.15.172:0/828642992 shutdown_connections 2024-09-10T08:49:10.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.117+0000 7fb18c835640 1 --2- 172.21.15.172:0/828642992 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb1700a2b80 0x7fb1700a5040 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.117+0000 7fb18c835640 1 --2- 172.21.15.172:0/828642992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb188106f00 0x7fb18819ee60 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.117+0000 7fb18c835640 1 --2- 172.21.15.172:0/828642992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb188102780 0x7fb18819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.117+0000 7fb18c835640 1 --2- 172.21.15.172:0/828642992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb188101dd0 0x7fb18819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:10.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.117+0000 7fb18c835640 1 -- 172.21.15.172:0/828642992 >> 172.21.15.172:0/828642992 conn(0x7fb1880fd890 msgr2=0x7fb1880fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:10.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.117+0000 7fb18c835640 1 -- 172.21.15.172:0/828642992 shutdown_connections 2024-09-10T08:49:10.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:10.117+0000 7fb18c835640 1 -- 172.21.15.172:0/828642992 wait complete. 2024-09-10T08:49:10.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:10 smithi191 ceph-mon[35314]: pgmap v438: 129 pgs: 129 active+clean; 622 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 MiB/s wr, 29 op/s 2024-09-10T08:49:10.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:10 smithi172 ceph-mon[30854]: pgmap v438: 129 pgs: 129 active+clean; 622 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 MiB/s wr, 29 op/s 2024-09-10T08:49:10.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:10 smithi174 ceph-mon[34068]: pgmap v438: 129 pgs: 129 active+clean; 622 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 MiB/s wr, 29 op/s 2024-09-10T08:49:10.937 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:49:10.937 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:49:11.217 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:11.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.736+0000 7f375a9d5640 1 -- 172.21.15.172:0/1935643217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3754106b80 msgr2=0x7f3754106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:11.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.736+0000 7f375a9d5640 1 --2- 172.21.15.172:0/1935643217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3754106b80 0x7f3754106f60 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3744009900 tx=0x7f374402a410 comp rx=0 tx=0).stop 2024-09-10T08:49:11.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.736+0000 7f375a9d5640 1 -- 172.21.15.172:0/1935643217 shutdown_connections 2024-09-10T08:49:11.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.736+0000 7f375a9d5640 1 --2- 172.21.15.172:0/1935643217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3754100aa0 0x7f375410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:11.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.736+0000 7f375a9d5640 1 --2- 172.21.15.172:0/1935643217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3754100100 0x7f3754100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:11.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.736+0000 7f375a9d5640 1 --2- 172.21.15.172:0/1935643217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3754106b80 0x7f3754106f60 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:11.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.736+0000 7f375a9d5640 1 -- 172.21.15.172:0/1935643217 >> 172.21.15.172:0/1935643217 conn(0x7f3754077230 msgr2=0x7f37540fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:11.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.736+0000 7f375a9d5640 1 -- 172.21.15.172:0/1935643217 shutdown_connections 2024-09-10T08:49:11.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.737+0000 7f375a9d5640 1 -- 172.21.15.172:0/1935643217 wait complete. 2024-09-10T08:49:11.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.737+0000 7f375a9d5640 1 Processor -- start 2024-09-10T08:49:11.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.737+0000 7f375a9d5640 1 -- start start 2024-09-10T08:49:11.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f375a9d5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3754100100 0x7f37541012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:11.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f375a9d5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3754100aa0 0x7f3754101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:11.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f375a9d5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3754106b80 0x7f3754101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:11.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f375a9d5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3754111f00 con 0x7f3754106b80 2024-09-10T08:49:11.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f375a9d5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3754111d80 con 0x7f3754100100 2024-09-10T08:49:11.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f375a9d5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3754112080 con 0x7f3754100aa0 2024-09-10T08:49:11.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f3758f72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3754106b80 0x7f3754101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:11.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f3758f72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3754106b80 0x7f3754101d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48592/0 (socket says 172.21.15.172:48592) 2024-09-10T08:49:11.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f3758f72640 1 -- 172.21.15.172:0/267801899 learned_addr learned my addr 172.21.15.172:0/267801899 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:11.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f3753fff640 1 --2- 172.21.15.172:0/267801899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3754100100 0x7f37541012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:11.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f37537fe640 1 --2- 172.21.15.172:0/267801899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3754100aa0 0x7f3754101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:11.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f3758f72640 1 -- 172.21.15.172:0/267801899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3754100aa0 msgr2=0x7f3754101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:11.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f3758f72640 1 --2- 172.21.15.172:0/267801899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3754100aa0 0x7f3754101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:11.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f3758f72640 1 -- 172.21.15.172:0/267801899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3754100100 msgr2=0x7f37541012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:11.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f3758f72640 1 --2- 172.21.15.172:0/267801899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3754100100 0x7f37541012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:11.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f3758f72640 1 -- 172.21.15.172:0/267801899 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37541a68e0 con 0x7f3754106b80 2024-09-10T08:49:11.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f3753fff640 1 --2- 172.21.15.172:0/267801899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3754100100 0x7f37541012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:11.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.738+0000 7f37537fe640 1 --2- 172.21.15.172:0/267801899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3754100aa0 0x7f3754101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:11.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.739+0000 7f3758f72640 1 --2- 172.21.15.172:0/267801899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3754106b80 0x7f3754101d60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f374c00eb40 tx=0x7f374c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:11.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.739+0000 7f37517fa640 1 -- 172.21.15.172:0/267801899 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c004b20 con 0x7f3754106b80 2024-09-10T08:49:11.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.739+0000 7f375a9d5640 1 -- 172.21.15.172:0/267801899 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37541a6bd0 con 0x7f3754106b80 2024-09-10T08:49:11.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.739+0000 7f37517fa640 1 -- 172.21.15.172:0/267801899 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f374c004cc0 con 0x7f3754106b80 2024-09-10T08:49:11.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.739+0000 7f37517fa640 1 -- 172.21.15.172:0/267801899 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c0107b0 con 0x7f3754106b80 2024-09-10T08:49:11.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.739+0000 7f375a9d5640 1 -- 172.21.15.172:0/267801899 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37541a7110 con 0x7f3754106b80 2024-09-10T08:49:11.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.740+0000 7f375a9d5640 1 -- 172.21.15.172:0/267801899 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3754100560 con 0x7f3754106b80 2024-09-10T08:49:11.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.741+0000 7f37517fa640 1 -- 172.21.15.172:0/267801899 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f374c005020 con 0x7f3754106b80 2024-09-10T08:49:11.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.742+0000 7f37517fa640 1 --2- 172.21.15.172:0/267801899 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f373c0a2b80 0x7f373c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:11.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.742+0000 7f3753fff640 1 --2- 172.21.15.172:0/267801899 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f373c0a2b80 0x7f373c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:11.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.742+0000 7f37517fa640 1 -- 172.21.15.172:0/267801899 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f374c0cfa90 con 0x7f3754106b80 2024-09-10T08:49:11.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.743+0000 7f3753fff640 1 --2- 172.21.15.172:0/267801899 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f373c0a2b80 0x7f373c0a5040 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3744031440 tx=0x7f3744005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:11.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.747+0000 7f37517fa640 1 -- 172.21.15.172:0/267801899 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f374c09c690 con 0x7f3754106b80 2024-09-10T08:49:11.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.991+0000 7f375a9d5640 1 -- 172.21.15.172:0/267801899 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f375410a0c0 con 0x7f3754106b80 2024-09-10T08:49:11.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.992+0000 7f37517fa640 1 -- 172.21.15.172:0/267801899 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+12004 (secure 0 0 0) 0x7f374c09bf90 con 0x7f3754106b80 2024-09-10T08:49:11.994 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:49:11.994 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":31,"btime":"2024-09-10T08:49:02:257905+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:49:01.296584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:49:11.994 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 31 2024-09-10T08:49:11.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.996+0000 7f375a9d5640 1 -- 172.21.15.172:0/267801899 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f373c0a2b80 msgr2=0x7f373c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:11.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.996+0000 7f375a9d5640 1 --2- 172.21.15.172:0/267801899 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f373c0a2b80 0x7f373c0a5040 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3744031440 tx=0x7f3744005b50 comp rx=0 tx=0).stop 2024-09-10T08:49:11.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.996+0000 7f375a9d5640 1 -- 172.21.15.172:0/267801899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3754106b80 msgr2=0x7f3754101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:11.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.996+0000 7f375a9d5640 1 --2- 172.21.15.172:0/267801899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3754106b80 0x7f3754101d60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f374c00eb40 tx=0x7f374c00c2d0 comp rx=0 tx=0).stop 2024-09-10T08:49:11.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.996+0000 7f375a9d5640 1 -- 172.21.15.172:0/267801899 shutdown_connections 2024-09-10T08:49:11.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.996+0000 7f375a9d5640 1 --2- 172.21.15.172:0/267801899 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f373c0a2b80 0x7f373c0a5040 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:11.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.996+0000 7f375a9d5640 1 --2- 172.21.15.172:0/267801899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3754106b80 0x7f3754101d60 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:11.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.996+0000 7f375a9d5640 1 --2- 172.21.15.172:0/267801899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3754100aa0 0x7f3754101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:11.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.996+0000 7f375a9d5640 1 --2- 172.21.15.172:0/267801899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3754100100 0x7f37541012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:11.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.996+0000 7f375a9d5640 1 -- 172.21.15.172:0/267801899 >> 172.21.15.172:0/267801899 conn(0x7f3754077230 msgr2=0x7f3754104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:11.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.996+0000 7f375a9d5640 1 -- 172.21.15.172:0/267801899 shutdown_connections 2024-09-10T08:49:11.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:11.996+0000 7f375a9d5640 1 -- 172.21.15.172:0/267801899 wait complete. 2024-09-10T08:49:12.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:12 smithi191 ceph-mon[35314]: pgmap v439: 129 pgs: 129 active+clean; 698 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 MiB/s wr, 34 op/s 2024-09-10T08:49:12.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:12 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/267801899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:49:12.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:12 smithi172 ceph-mon[30854]: pgmap v439: 129 pgs: 129 active+clean; 698 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 MiB/s wr, 34 op/s 2024-09-10T08:49:12.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:12 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/267801899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:49:12.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:12 smithi174 ceph-mon[34068]: pgmap v439: 129 pgs: 129 active+clean; 698 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 MiB/s wr, 34 op/s 2024-09-10T08:49:12.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:12 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/267801899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:49:12.746 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T08:49:13.012 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:13.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.497+0000 7fb029151640 1 -- 172.21.15.172:0/2319060044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0241002f0 msgr2=0x7fb0241006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:13.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.497+0000 7fb029151640 1 --2- 172.21.15.172:0/2319060044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0241002f0 0x7fb0241006d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb014009930 tx=0x7fb01402a420 comp rx=0 tx=0).stop 2024-09-10T08:49:13.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.498+0000 7fb029151640 1 -- 172.21.15.172:0/2319060044 shutdown_connections 2024-09-10T08:49:13.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.498+0000 7fb029151640 1 --2- 172.21.15.172:0/2319060044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb024106f50 0x7fb02410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.498+0000 7fb029151640 1 --2- 172.21.15.172:0/2319060044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb024100ca0 0x7fb024106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.498+0000 7fb029151640 1 --2- 172.21.15.172:0/2319060044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0241002f0 0x7fb0241006d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.498+0000 7fb029151640 1 -- 172.21.15.172:0/2319060044 >> 172.21.15.172:0/2319060044 conn(0x7fb024077220 msgr2=0x7fb0240fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:13.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.498+0000 7fb029151640 1 -- 172.21.15.172:0/2319060044 shutdown_connections 2024-09-10T08:49:13.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.498+0000 7fb029151640 1 -- 172.21.15.172:0/2319060044 wait complete. 2024-09-10T08:49:13.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.498+0000 7fb029151640 1 Processor -- start 2024-09-10T08:49:13.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb029151640 1 -- start start 2024-09-10T08:49:13.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb029151640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0241002f0 0x7fb0241983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb029151640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb024100ca0 0x7fb024198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb029151640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb024106f50 0x7fb02419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb029151640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb024111ea0 con 0x7fb024106f50 2024-09-10T08:49:13.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb029151640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb024111d20 con 0x7fb0241002f0 2024-09-10T08:49:13.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb029151640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb024112020 con 0x7fb024100ca0 2024-09-10T08:49:13.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb02359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb024106f50 0x7fb02419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb022d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0241002f0 0x7fb0241983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb022d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0241002f0 0x7fb0241983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:56886/0 (socket says 172.21.15.172:56886) 2024-09-10T08:49:13.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb02359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb024106f50 0x7fb02419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48610/0 (socket says 172.21.15.172:48610) 2024-09-10T08:49:13.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb022d9d640 1 -- 172.21.15.172:0/435114554 learned_addr learned my addr 172.21.15.172:0/435114554 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:13.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb02259c640 1 --2- 172.21.15.172:0/435114554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb024100ca0 0x7fb024198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb022d9d640 1 -- 172.21.15.172:0/435114554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb024100ca0 msgr2=0x7fb024198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:13.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb022d9d640 1 --2- 172.21.15.172:0/435114554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb024100ca0 0x7fb024198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb022d9d640 1 -- 172.21.15.172:0/435114554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb024106f50 msgr2=0x7fb02419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:13.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb022d9d640 1 --2- 172.21.15.172:0/435114554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb024106f50 0x7fb02419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb022d9d640 1 -- 172.21.15.172:0/435114554 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02419d390 con 0x7fb0241002f0 2024-09-10T08:49:13.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.499+0000 7fb02359e640 1 --2- 172.21.15.172:0/435114554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb024106f50 0x7fb02419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:13.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.500+0000 7fb022d9d640 1 --2- 172.21.15.172:0/435114554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0241002f0 0x7fb0241983e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb014005ab0 tx=0x7fb014003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:13.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.500+0000 7fb00bfff640 1 -- 172.21.15.172:0/435114554 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01403f070 con 0x7fb0241002f0 2024-09-10T08:49:13.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.500+0000 7fb029151640 1 -- 172.21.15.172:0/435114554 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02419d620 con 0x7fb0241002f0 2024-09-10T08:49:13.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.500+0000 7fb00bfff640 1 -- 172.21.15.172:0/435114554 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb014031b50 con 0x7fb0241002f0 2024-09-10T08:49:13.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.500+0000 7fb00bfff640 1 -- 172.21.15.172:0/435114554 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014035030 con 0x7fb0241002f0 2024-09-10T08:49:13.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.500+0000 7fb029151640 1 -- 172.21.15.172:0/435114554 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0241011a0 con 0x7fb0241002f0 2024-09-10T08:49:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.502+0000 7fb00bfff640 1 -- 172.21.15.172:0/435114554 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb014042070 con 0x7fb0241002f0 2024-09-10T08:49:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.503+0000 7fb02b52a740 1 Processor -- start 2024-09-10T08:49:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.503+0000 7fb00bfff640 1 --2- 172.21.15.172:0/435114554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb00c0a2b80 0x7fb00c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.503+0000 7fb02b52a740 1 -- start start 2024-09-10T08:49:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.503+0000 7fb02359e640 1 --2- 172.21.15.172:0/435114554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb00c0a2b80 0x7fb00c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.503+0000 7fb00bfff640 1 -- 172.21.15.172:0/435114554 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb0140efa80 con 0x7fb0241002f0 2024-09-10T08:49:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.503+0000 7fb02b52a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817bf0b40 0x557817bef090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.503+0000 7fb02b52a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817befa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.503+0000 7fb02b52a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817c02b80 0x557817c04f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.503+0000 7fb02b52a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557817b552f0 con 0x557817bef5d0 2024-09-10T08:49:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.503+0000 7fb02b52a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557817afccf0 con 0x557817bf0b40 2024-09-10T08:49:13.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.503+0000 7fb02b52a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557817b512b0 con 0x557817c02b80 2024-09-10T08:49:13.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.503+0000 7fb02259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817befa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.503+0000 7fb02259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817befa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48614/0 (socket says 172.21.15.172:48614) 2024-09-10T08:49:13.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.503+0000 7fb02259c640 1 -- 172.21.15.172:0/3175439785 learned_addr learned my addr 172.21.15.172:0/3175439785 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:13.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.503+0000 7fb022d9d640 1 --2- 172.21.15.172:0/3175439785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817bf0b40 0x557817bef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.504+0000 7fb02359e640 1 --2- 172.21.15.172:0/3175439785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817c02b80 0x557817c04f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.504+0000 7fb02259c640 1 -- 172.21.15.172:0/3175439785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817c02b80 msgr2=0x557817c04f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:13.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.504+0000 7fb02259c640 1 --2- 172.21.15.172:0/3175439785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817c02b80 0x557817c04f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.504+0000 7fb02259c640 1 -- 172.21.15.172:0/3175439785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817bf0b40 msgr2=0x557817bef090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:13.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.504+0000 7fb02259c640 1 --2- 172.21.15.172:0/3175439785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817bf0b40 0x557817bef090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.504+0000 7fb02259c640 1 -- 172.21.15.172:0/3175439785 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557817ae13a0 con 0x557817bef5d0 2024-09-10T08:49:13.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.504+0000 7fb022d9d640 1 --2- 172.21.15.172:0/3175439785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817bf0b40 0x557817bef090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:49:13.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.504+0000 7fb02359e640 1 --2- 172.21.15.172:0/435114554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb00c0a2b80 0x7fb00c0a5040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb01c005fd0 tx=0x7fb01c010ac0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:13.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.504+0000 7fb02259c640 1 --2- 172.21.15.172:0/3175439785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817befa10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb018009c60 tx=0x7fb018011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e33a2bbc76bd87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:13.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.504+0000 7fb0097fa640 1 -- 172.21.15.172:0/3175439785 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018004230 con 0x557817bef5d0 2024-09-10T08:49:13.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.504+0000 7fb0097fa640 1 -- 172.21.15.172:0/3175439785 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0180043d0 con 0x557817bef5d0 2024-09-10T08:49:13.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.504+0000 7fb0097fa640 1 -- 172.21.15.172:0/3175439785 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018004d60 con 0x557817bef5d0 2024-09-10T08:49:13.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.505+0000 7fb02b52a740 1 -- 172.21.15.172:0/3175439785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 msgr2=0x557817befa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:13.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.505+0000 7fb02b52a740 1 --2- 172.21.15.172:0/3175439785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817befa10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb018009c60 tx=0x7fb018011810 comp rx=0 tx=0).stop 2024-09-10T08:49:13.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.505+0000 7fb02b52a740 1 -- 172.21.15.172:0/3175439785 shutdown_connections 2024-09-10T08:49:13.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.505+0000 7fb02b52a740 1 --2- 172.21.15.172:0/3175439785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817c02b80 0x557817c04f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.505+0000 7fb02b52a740 1 --2- 172.21.15.172:0/3175439785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817befa10 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.505+0000 7fb02b52a740 1 --2- 172.21.15.172:0/3175439785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817bf0b40 0x557817bef090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.505+0000 7fb02b52a740 1 -- 172.21.15.172:0/3175439785 >> 172.21.15.172:0/3175439785 conn(0x557817bb8310 msgr2=0x557817bebc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:13.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.505+0000 7fb02b52a740 1 -- 172.21.15.172:0/3175439785 shutdown_connections 2024-09-10T08:49:13.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.505+0000 7fb02b52a740 1 -- 172.21.15.172:0/3175439785 wait complete. 2024-09-10T08:49:13.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.505+0000 7fb02b52a740 1 -- start start 2024-09-10T08:49:13.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.505+0000 7fb02b52a740 1 Processor -- start 2024-09-10T08:49:13.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb02b52a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817c02520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb02b52a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817bf0b40 0x557817bff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb02b52a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817c02b80 0x557817bffd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb02b52a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557817c05880 con 0x557817bef5d0 2024-09-10T08:49:13.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb02b52a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557817b10060 con 0x557817c02b80 2024-09-10T08:49:13.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb02b52a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557817c05a00 con 0x557817bf0b40 2024-09-10T08:49:13.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb02b52a740 10 client.0 _ll_register_callbacks cb 0x5578178638c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:13.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb022d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817c02520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb022d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817c02520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48618/0 (socket says 172.21.15.172:48618) 2024-09-10T08:49:13.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb022d9d640 1 -- 172.21.15.172:0/2791296720 learned_addr learned my addr 172.21.15.172:0/2791296720 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:13.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb02259c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817bf0b40 0x557817bff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb02359e640 1 --2- 172.21.15.172:0/2791296720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817c02b80 0x557817bffd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb022d9d640 1 -- 172.21.15.172:0/2791296720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817bf0b40 msgr2=0x557817bff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:13.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb022d9d640 1 --2- 172.21.15.172:0/2791296720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817bf0b40 0x557817bff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb022d9d640 1 -- 172.21.15.172:0/2791296720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817c02b80 msgr2=0x557817bffd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:13.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb022d9d640 1 --2- 172.21.15.172:0/2791296720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817c02b80 0x557817bffd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb022d9d640 1 -- 172.21.15.172:0/2791296720 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578178c63c0 con 0x557817bef5d0 2024-09-10T08:49:13.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb022d9d640 1 --2- 172.21.15.172:0/2791296720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817c02520 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb014006010 tx=0x7fb01402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13713b0c0ff28b10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:13.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb006ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:13.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.506+0000 7fb006ff5640 1 -- 172.21.15.172:0/2791296720 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140d7060 con 0x557817bef5d0 2024-09-10T08:49:13.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.507+0000 7fb006ff5640 1 -- 172.21.15.172:0/2791296720 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb0140b8640 con 0x557817bef5d0 2024-09-10T08:49:13.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.507+0000 7fb02b52a740 1 -- 172.21.15.172:0/2791296720 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0180043d0 con 0x557817bef5d0 2024-09-10T08:49:13.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.507+0000 7fb006ff5640 1 -- 172.21.15.172:0/2791296720 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb014049040 con 0x557817bef5d0 2024-09-10T08:49:13.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.507+0000 7fb02b52a740 10 client.16314 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:13.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.507+0000 7fb02b52a740 1 -- 172.21.15.172:0/2791296720 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557817c112d0 con 0x557817bef5d0 2024-09-10T08:49:13.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.507+0000 7fb006ff5640 1 -- 172.21.15.172:0/2791296720 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0180043d0 con 0x557817bef5d0 2024-09-10T08:49:13.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.507+0000 7fb006ff5640 1 -- 172.21.15.172:0/2791296720 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fb0140bf030 con 0x557817bef5d0 2024-09-10T08:49:13.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.507+0000 7fb02b52a740 10 client.16314 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:13.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.507+0000 7fb02b52a740 10 client.16314 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:49:13.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.507+0000 7fb02b52a740 20 client.16314 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:13.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.507+0000 7fb02b52a740 1 --2- 172.21.15.172:0/2791296720 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x557817c12780 0x557817c14bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.508+0000 7fb02b52a740 4 client.16314 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:49:13.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.508+0000 7fb02b52a740 1 -- 172.21.15.172:0/2791296720 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557817b62860 con 0x557817c12780 2024-09-10T08:49:13.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.508+0000 7fb02259c640 1 --2- 172.21.15.172:0/2791296720 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x557817c12780 0x557817c14bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.509+0000 7fb006ff5640 10 client.16314 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:49:13.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.509+0000 7fb02259c640 1 --2- 172.21.15.172:0/2791296720 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x557817c12780 0x557817c14bb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a8663018f8296c27 server_cookie=64c36a3e9fdc612b in_seq=0 out_seq=0 2024-09-10T08:49:13.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.509+0000 7fb006ff5640 1 -- 172.21.15.172:0/2791296720 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557817b62860 con 0x557817c12780 2024-09-10T08:49:13.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.509+0000 7fb006ff5640 10 client.16314 handle_command_reply: tid=0 2024-09-10T08:49:13.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.510+0000 7fb02b52a740 1 client.16314 shutdown 2024-09-10T08:49:13.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.510+0000 7fb02b52a740 1 -- 172.21.15.172:0/2791296720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 msgr2=0x557817c02520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:13.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.510+0000 7fb02b52a740 1 --2- 172.21.15.172:0/2791296720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817c02520 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb014006010 tx=0x7fb01402a420 comp rx=0 tx=0).stop 2024-09-10T08:49:13.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.510+0000 7fb02b52a740 1 -- 172.21.15.172:0/2791296720 shutdown_connections 2024-09-10T08:49:13.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.510+0000 7fb02b52a740 1 --2- 172.21.15.172:0/2791296720 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x557817c12780 0x557817c14bb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.510+0000 7fb02b52a740 1 --2- 172.21.15.172:0/2791296720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817c02b80 0x557817bffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.510+0000 7fb02b52a740 1 --2- 172.21.15.172:0/2791296720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817bf0b40 0x557817bff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.510+0000 7fb006ff5640 0 client.16314 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:49:13.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.510+0000 7fb02b52a740 1 --2- 172.21.15.172:0/2791296720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817c02520 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.510+0000 7fb02b52a740 1 -- 172.21.15.172:0/2791296720 >> 172.21.15.172:0/2791296720 conn(0x557817bb8310 msgr2=0x557817bf1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:13.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.510+0000 7fb02b52a740 1 -- 172.21.15.172:0/2791296720 shutdown_connections 2024-09-10T08:49:13.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.510+0000 7fb02b52a740 1 -- 172.21.15.172:0/2791296720 wait complete. 2024-09-10T08:49:13.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.510+0000 7fb02b52a740 20 client.16314 trim_cache size 0 max 16384 2024-09-10T08:49:13.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.530+0000 7fb02b52a740 1 Processor -- start 2024-09-10T08:49:13.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.531+0000 7fb02b52a740 1 -- start start 2024-09-10T08:49:13.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.531+0000 7fb02b52a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817c0eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.531+0000 7fb02b52a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817bf0b40 0x557817bf1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.531+0000 7fb02b52a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817c02b80 0x557817c1a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.531+0000 7fb02b52a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241ac910 con 0x557817bef5d0 2024-09-10T08:49:13.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.531+0000 7fb02b52a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241ae4b0 con 0x557817c02b80 2024-09-10T08:49:13.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.531+0000 7fb02b52a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241acfe0 con 0x557817bf0b40 2024-09-10T08:49:13.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.531+0000 7fb022d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817c0eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.531+0000 7fb022d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817c0eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48620/0 (socket says 172.21.15.172:48620) 2024-09-10T08:49:13.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.531+0000 7fb022d9d640 1 -- 172.21.15.172:0/789961079 learned_addr learned my addr 172.21.15.172:0/789961079 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:13.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.531+0000 7fb02359e640 1 --2- 172.21.15.172:0/789961079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817c02b80 0x557817c1a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.531+0000 7fb02259c640 1 --2- 172.21.15.172:0/789961079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817bf0b40 0x557817bf1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.531+0000 7fb022d9d640 1 -- 172.21.15.172:0/789961079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817bf0b40 msgr2=0x557817bf1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:13.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.531+0000 7fb022d9d640 1 --2- 172.21.15.172:0/789961079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817bf0b40 0x557817bf1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.531+0000 7fb022d9d640 1 -- 172.21.15.172:0/789961079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817c02b80 msgr2=0x557817c1a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:13.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.531+0000 7fb022d9d640 1 --2- 172.21.15.172:0/789961079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817c02b80 0x557817c1a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.531+0000 7fb022d9d640 1 -- 172.21.15.172:0/789961079 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557817beb510 con 0x557817bef5d0 2024-09-10T08:49:13.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.532+0000 7fb022d9d640 1 --2- 172.21.15.172:0/789961079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817c0eb20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb01409c040 tx=0x7fb014009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cd64a4fbe2f48b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:13.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.532+0000 7fb008ff9640 1 -- 172.21.15.172:0/789961079 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140bb930 con 0x557817bef5d0 2024-09-10T08:49:13.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.532+0000 7fb008ff9640 1 -- 172.21.15.172:0/789961079 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0140c02d0 con 0x557817bef5d0 2024-09-10T08:49:13.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.532+0000 7fb008ff9640 1 -- 172.21.15.172:0/789961079 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140a50c0 con 0x557817bef5d0 2024-09-10T08:49:13.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.532+0000 7fb02b52a740 1 -- 172.21.15.172:0/789961079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 msgr2=0x557817c0eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:13.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.532+0000 7fb02b52a740 1 --2- 172.21.15.172:0/789961079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817c0eb20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb01409c040 tx=0x7fb014009260 comp rx=0 tx=0).stop 2024-09-10T08:49:13.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.533+0000 7fb02b52a740 1 -- 172.21.15.172:0/789961079 shutdown_connections 2024-09-10T08:49:13.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.533+0000 7fb02b52a740 1 --2- 172.21.15.172:0/789961079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817c02b80 0x557817c1a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.533+0000 7fb02b52a740 1 --2- 172.21.15.172:0/789961079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817bf0b40 0x557817bf1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.533+0000 7fb02b52a740 1 --2- 172.21.15.172:0/789961079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bef5d0 0x557817c0eb20 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.533+0000 7fb02b52a740 1 -- 172.21.15.172:0/789961079 >> 172.21.15.172:0/789961079 conn(0x557817bb8310 msgr2=0x557817bf1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:13.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.533+0000 7fb02b52a740 1 -- 172.21.15.172:0/789961079 shutdown_connections 2024-09-10T08:49:13.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.533+0000 7fb02b52a740 1 -- 172.21.15.172:0/789961079 wait complete. 2024-09-10T08:49:13.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.533+0000 7fb02b52a740 1 -- start start 2024-09-10T08:49:13.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.533+0000 7fb02b52a740 1 Processor -- start 2024-09-10T08:49:13.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.533+0000 7fb02b52a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817bef5d0 0x557817c0ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.533+0000 7fb02b52a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bf0b40 0x557817c0b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.533+0000 7fb02b52a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817c02b80 0x557817c07c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.533+0000 7fb02b52a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241add80 con 0x557817bf0b40 2024-09-10T08:49:13.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.533+0000 7fb02b52a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241ad6b0 con 0x557817bef5d0 2024-09-10T08:49:13.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.533+0000 7fb02b52a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb0241aa8f0 con 0x557817c02b80 2024-09-10T08:49:13.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.534+0000 7fb02b52a740 10 client.0 _ll_register_callbacks cb 0x7fb014009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:13.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.534+0000 7fb02259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bf0b40 0x557817c0b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.534+0000 7fb02259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bf0b40 0x557817c0b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48624/0 (socket says 172.21.15.172:48624) 2024-09-10T08:49:13.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.534+0000 7fb02259c640 1 -- 172.21.15.172:0/3557160405 learned_addr learned my addr 172.21.15.172:0/3557160405 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:13.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.534+0000 7fb02359e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817c02b80 0x557817c07c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.534+0000 7fb022d9d640 1 --2- 172.21.15.172:0/3557160405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817bef5d0 0x557817c0ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.534+0000 7fb02259c640 1 -- 172.21.15.172:0/3557160405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817c02b80 msgr2=0x557817c07c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:13.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.534+0000 7fb02259c640 1 --2- 172.21.15.172:0/3557160405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817c02b80 0x557817c07c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.534+0000 7fb02259c640 1 -- 172.21.15.172:0/3557160405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817bef5d0 msgr2=0x557817c0ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:13.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.534+0000 7fb02259c640 1 --2- 172.21.15.172:0/3557160405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817bef5d0 0x557817c0ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.534+0000 7fb02259c640 1 -- 172.21.15.172:0/3557160405 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557817c081a0 con 0x557817bf0b40 2024-09-10T08:49:13.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.534+0000 7fb02259c640 1 --2- 172.21.15.172:0/3557160405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bf0b40 0x557817c0b1d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb018009c60 tx=0x7fb018009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465ba7cbbd5cf217 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:13.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.534+0000 7fb006ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:13.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.534+0000 7fb006ff5640 1 -- 172.21.15.172:0/3557160405 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01801e040 con 0x557817bf0b40 2024-09-10T08:49:13.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.534+0000 7fb006ff5640 1 -- 172.21.15.172:0/3557160405 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb018004250 con 0x557817bf0b40 2024-09-10T08:49:13.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.534+0000 7fb02b52a740 1 -- 172.21.15.172:0/3557160405 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0140c02d0 con 0x557817bf0b40 2024-09-10T08:49:13.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.535+0000 7fb006ff5640 1 -- 172.21.15.172:0/3557160405 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb018038050 con 0x557817bf0b40 2024-09-10T08:49:13.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.535+0000 7fb02b52a740 10 client.16320 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:13.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.535+0000 7fb02b52a740 1 -- 172.21.15.172:0/3557160405 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557817c7dc30 con 0x557817bf0b40 2024-09-10T08:49:13.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.535+0000 7fb006ff5640 1 -- 172.21.15.172:0/3557160405 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0140c02d0 con 0x557817bf0b40 2024-09-10T08:49:13.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.535+0000 7fb006ff5640 1 -- 172.21.15.172:0/3557160405 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fb01802e030 con 0x557817bf0b40 2024-09-10T08:49:13.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.535+0000 7fb02b52a740 10 client.16320 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:13.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.535+0000 7fb02b52a740 10 client.16320 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:49:13.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.535+0000 7fb02b52a740 20 client.16320 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:13.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.535+0000 7fb02b52a740 1 --2- 172.21.15.172:0/3557160405 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x557817c7e1f0 0x557817c80610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:13.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.535+0000 7fb02b52a740 4 client.16320 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:49:13.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.535+0000 7fb02b52a740 1 -- 172.21.15.172:0/3557160405 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557817b5cd70 con 0x557817c7e1f0 2024-09-10T08:49:13.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.535+0000 7fb022d9d640 1 --2- 172.21.15.172:0/3557160405 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x557817c7e1f0 0x557817c80610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.536+0000 7fb022d9d640 1 --2- 172.21.15.172:0/3557160405 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x557817c7e1f0 0x557817c80610 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=236be54ff388e0c2 server_cookie=55a2e43b1be1887e in_seq=0 out_seq=0 2024-09-10T08:49:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.536+0000 7fb006ff5640 10 client.16320 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:49:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.536+0000 7fb006ff5640 1 -- 172.21.15.172:0/3557160405 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557817b5cd70 con 0x557817c7e1f0 2024-09-10T08:49:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.536+0000 7fb006ff5640 10 client.16320 handle_command_reply: tid=0 2024-09-10T08:49:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.536+0000 7fb02b52a740 1 client.16320 shutdown 2024-09-10T08:49:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.536+0000 7fb02b52a740 1 -- 172.21.15.172:0/3557160405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bf0b40 msgr2=0x557817c0b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.536+0000 7fb02b52a740 1 --2- 172.21.15.172:0/3557160405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bf0b40 0x557817c0b1d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb018009c60 tx=0x7fb018009640 comp rx=0 tx=0).stop 2024-09-10T08:49:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.536+0000 7fb02b52a740 1 -- 172.21.15.172:0/3557160405 shutdown_connections 2024-09-10T08:49:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.536+0000 7fb02b52a740 1 --2- 172.21.15.172:0/3557160405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557817c02b80 0x557817c07c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.536+0000 7fb02b52a740 1 --2- 172.21.15.172:0/3557160405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557817bf0b40 0x557817c0b1d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.536+0000 7fb02b52a740 1 --2- 172.21.15.172:0/3557160405 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x557817c7e1f0 0x557817c80610 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.536+0000 7fb006ff5640 0 client.16320 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:49:13.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.536+0000 7fb02b52a740 1 --2- 172.21.15.172:0/3557160405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557817bef5d0 0x557817c0ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.536+0000 7fb02b52a740 1 -- 172.21.15.172:0/3557160405 >> 172.21.15.172:0/3557160405 conn(0x557817bb8310 msgr2=0x557817c01030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:13.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.536+0000 7fb02b52a740 1 -- 172.21.15.172:0/3557160405 shutdown_connections 2024-09-10T08:49:13.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.536+0000 7fb02b52a740 1 -- 172.21.15.172:0/3557160405 wait complete. 2024-09-10T08:49:13.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.536+0000 7fb02b52a740 20 client.16320 trim_cache size 0 max 16384 2024-09-10T08:49:13.544 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:49:13.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.537+0000 7fb029151640 1 -- 172.21.15.172:0/435114554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb00c0a2b80 msgr2=0x7fb00c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:13.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.537+0000 7fb029151640 1 --2- 172.21.15.172:0/435114554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb00c0a2b80 0x7fb00c0a5040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb01c005fd0 tx=0x7fb01c010ac0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.537+0000 7fb029151640 1 -- 172.21.15.172:0/435114554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0241002f0 msgr2=0x7fb0241983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:13.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.537+0000 7fb029151640 1 --2- 172.21.15.172:0/435114554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0241002f0 0x7fb0241983e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb014005ab0 tx=0x7fb014003b90 comp rx=0 tx=0).stop 2024-09-10T08:49:13.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.537+0000 7fb029151640 1 -- 172.21.15.172:0/435114554 shutdown_connections 2024-09-10T08:49:13.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.537+0000 7fb029151640 1 --2- 172.21.15.172:0/435114554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb00c0a2b80 0x7fb00c0a5040 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.537+0000 7fb029151640 1 --2- 172.21.15.172:0/435114554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb024106f50 0x7fb02419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.537+0000 7fb029151640 1 --2- 172.21.15.172:0/435114554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb024100ca0 0x7fb024198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.537+0000 7fb029151640 1 --2- 172.21.15.172:0/435114554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0241002f0 0x7fb0241983e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:13.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.537+0000 7fb029151640 1 -- 172.21.15.172:0/435114554 >> 172.21.15.172:0/435114554 conn(0x7fb024077220 msgr2=0x7fb024105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:13.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.537+0000 7fb029151640 1 -- 172.21.15.172:0/435114554 shutdown_connections 2024-09-10T08:49:13.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:13.537+0000 7fb029151640 1 -- 172.21.15.172:0/435114554 wait complete. 2024-09-10T08:49:14.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:14 smithi191 ceph-mon[35314]: pgmap v440: 129 pgs: 129 active+clean; 758 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-09-10T08:49:14.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:14 smithi172 ceph-mon[30854]: pgmap v440: 129 pgs: 129 active+clean; 758 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-09-10T08:49:14.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:14 smithi174 ceph-mon[34068]: pgmap v440: 129 pgs: 129 active+clean; 758 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-09-10T08:49:14.781 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T08:49:15.074 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:15.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.589+0000 7feee9be7640 1 -- 172.21.15.172:0/1768110413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee4101de0 msgr2=0x7feee41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:15.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.589+0000 7feee9be7640 1 --2- 172.21.15.172:0/1768110413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee4101de0 0x7feee41021c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7feed4009900 tx=0x7feed402a410 comp rx=0 tx=0).stop 2024-09-10T08:49:15.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.589+0000 7feee9be7640 1 -- 172.21.15.172:0/1768110413 shutdown_connections 2024-09-10T08:49:15.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.589+0000 7feee9be7640 1 --2- 172.21.15.172:0/1768110413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feee4106f10 0x7feee410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.589+0000 7feee9be7640 1 --2- 172.21.15.172:0/1768110413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feee4102790 0x7feee41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.589+0000 7feee9be7640 1 --2- 172.21.15.172:0/1768110413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee4101de0 0x7feee41021c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.589+0000 7feee9be7640 1 -- 172.21.15.172:0/1768110413 >> 172.21.15.172:0/1768110413 conn(0x7feee40fd8e0 msgr2=0x7feee40ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:15.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.589+0000 7feee9be7640 1 -- 172.21.15.172:0/1768110413 shutdown_connections 2024-09-10T08:49:15.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.589+0000 7feee9be7640 1 -- 172.21.15.172:0/1768110413 wait complete. 2024-09-10T08:49:15.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.590+0000 7feee9be7640 1 Processor -- start 2024-09-10T08:49:15.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.590+0000 7feee9be7640 1 -- start start 2024-09-10T08:49:15.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.590+0000 7feee9be7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee4101de0 0x7feee4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.590+0000 7feee9be7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feee4102790 0x7feee41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.590+0000 7feee9be7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feee4106f10 0x7feee419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.590+0000 7feee9be7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feee4111e20 con 0x7feee4101de0 2024-09-10T08:49:15.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.590+0000 7feee9be7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feee4111ca0 con 0x7feee4102790 2024-09-10T08:49:15.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.590+0000 7feee9be7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7feee4111fa0 con 0x7feee4106f10 2024-09-10T08:49:15.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.590+0000 7feee37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee4101de0 0x7feee4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.590+0000 7feee37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee4101de0 0x7feee4198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48648/0 (socket says 172.21.15.172:48648) 2024-09-10T08:49:15.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.590+0000 7feee37fe640 1 -- 172.21.15.172:0/1626990170 learned_addr learned my addr 172.21.15.172:0/1626990170 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:15.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.591+0000 7feee2ffd640 1 --2- 172.21.15.172:0/1626990170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feee4102790 0x7feee41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.591+0000 7feee3fff640 1 --2- 172.21.15.172:0/1626990170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feee4106f10 0x7feee419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.591+0000 7feee37fe640 1 -- 172.21.15.172:0/1626990170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feee4106f10 msgr2=0x7feee419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:15.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.591+0000 7feee37fe640 1 --2- 172.21.15.172:0/1626990170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feee4106f10 0x7feee419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.591+0000 7feee37fe640 1 -- 172.21.15.172:0/1626990170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feee4102790 msgr2=0x7feee41988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:15.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.591+0000 7feee37fe640 1 --2- 172.21.15.172:0/1626990170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feee4102790 0x7feee41988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.591+0000 7feee37fe640 1 -- 172.21.15.172:0/1626990170 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee419d310 con 0x7feee4101de0 2024-09-10T08:49:15.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.591+0000 7feee37fe640 1 --2- 172.21.15.172:0/1626990170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee4101de0 0x7feee4198360 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7feed4005940 tx=0x7feed4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:15.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.591+0000 7feee0ff9640 1 -- 172.21.15.172:0/1626990170 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed403f030 con 0x7feee4101de0 2024-09-10T08:49:15.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.591+0000 7feee0ff9640 1 -- 172.21.15.172:0/1626990170 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feed4003aa0 con 0x7feee4101de0 2024-09-10T08:49:15.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.591+0000 7feee9be7640 1 -- 172.21.15.172:0/1626990170 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee419d5a0 con 0x7feee4101de0 2024-09-10T08:49:15.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.591+0000 7feee0ff9640 1 -- 172.21.15.172:0/1626990170 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4035070 con 0x7feee4101de0 2024-09-10T08:49:15.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.591+0000 7feee9be7640 1 -- 172.21.15.172:0/1626990170 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee41a4ee0 con 0x7feee4101de0 2024-09-10T08:49:15.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.594+0000 7feee0ff9640 1 -- 172.21.15.172:0/1626990170 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7feed4042030 con 0x7feee4101de0 2024-09-10T08:49:15.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.594+0000 7feeebfc0740 1 Processor -- start 2024-09-10T08:49:15.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.594+0000 7feeebfc0740 1 -- start start 2024-09-10T08:49:15.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.594+0000 7feeebfc0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a822b40 0x55c03a821090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.594+0000 7feeebfc0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a8215d0 0x55c03a821a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.594+0000 7feeebfc0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a82c370 0x55c03a82e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.594+0000 7feeebfc0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c03a7872f0 con 0x55c03a8215d0 2024-09-10T08:49:15.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.594+0000 7feeebfc0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c03a72ecf0 con 0x55c03a822b40 2024-09-10T08:49:15.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.594+0000 7feeebfc0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c03a7832b0 con 0x55c03a82c370 2024-09-10T08:49:15.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.594+0000 7feee37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a8215d0 0x55c03a821a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.594+0000 7feee3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a822b40 0x55c03a821090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.595+0000 7feee3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a822b40 0x55c03a821090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:56950/0 (socket says 172.21.15.172:56950) 2024-09-10T08:49:15.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.595+0000 7feee3fff640 1 -- 172.21.15.172:0/135063695 learned_addr learned my addr 172.21.15.172:0/135063695 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:15.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.595+0000 7feee2ffd640 1 --2- 172.21.15.172:0/135063695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a82c370 0x55c03a82e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.595+0000 7feee3fff640 1 -- 172.21.15.172:0/135063695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a82c370 msgr2=0x55c03a82e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:15.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.595+0000 7feee3fff640 1 --2- 172.21.15.172:0/135063695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a82c370 0x55c03a82e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.595+0000 7feee3fff640 1 -- 172.21.15.172:0/135063695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a8215d0 msgr2=0x55c03a821a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:15.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.595+0000 7feee3fff640 1 --2- 172.21.15.172:0/135063695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a8215d0 0x55c03a821a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.595+0000 7feee3fff640 1 -- 172.21.15.172:0/135063695 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c03a7133a0 con 0x55c03a822b40 2024-09-10T08:49:15.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.595+0000 7feee0ff9640 1 --2- 172.21.15.172:0/1626990170 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feecc0a2c00 0x7feecc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.595+0000 7feee3fff640 1 --2- 172.21.15.172:0/135063695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a822b40 0x55c03a821090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feedc009e60 tx=0x7feedc010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bda00ac6b5c9c239 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:15.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.595+0000 7feee37fe640 1 --2- 172.21.15.172:0/1626990170 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feecc0a2c00 0x7feecc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.595+0000 7feee0ff9640 1 -- 172.21.15.172:0/1626990170 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7feed40ef950 con 0x7feee4101de0 2024-09-10T08:49:15.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.595+0000 7feec9ffb640 1 -- 172.21.15.172:0/135063695 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc016070 con 0x55c03a822b40 2024-09-10T08:49:15.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.595+0000 7feec9ffb640 1 -- 172.21.15.172:0/135063695 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feedc004d20 con 0x55c03a822b40 2024-09-10T08:49:15.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.595+0000 7feec9ffb640 1 -- 172.21.15.172:0/135063695 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc012740 con 0x55c03a822b40 2024-09-10T08:49:15.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.596+0000 7feeebfc0740 1 -- 172.21.15.172:0/135063695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a822b40 msgr2=0x55c03a821090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:15.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.596+0000 7feeebfc0740 1 --2- 172.21.15.172:0/135063695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a822b40 0x55c03a821090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feedc009e60 tx=0x7feedc010740 comp rx=0 tx=0).stop 2024-09-10T08:49:15.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.596+0000 7feee37fe640 1 --2- 172.21.15.172:0/1626990170 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feecc0a2c00 0x7feecc0a50c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feed4046000 tx=0x7feed40f35a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:15.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.596+0000 7feeebfc0740 1 -- 172.21.15.172:0/135063695 shutdown_connections 2024-09-10T08:49:15.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.596+0000 7feeebfc0740 1 --2- 172.21.15.172:0/135063695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a82c370 0x55c03a82e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.596+0000 7feeebfc0740 1 --2- 172.21.15.172:0/135063695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a8215d0 0x55c03a821a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.596+0000 7feeebfc0740 1 --2- 172.21.15.172:0/135063695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a822b40 0x55c03a821090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.596+0000 7feeebfc0740 1 -- 172.21.15.172:0/135063695 >> 172.21.15.172:0/135063695 conn(0x55c03a7ea310 msgr2=0x55c03a81dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:15.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.596+0000 7feeebfc0740 1 -- 172.21.15.172:0/135063695 shutdown_connections 2024-09-10T08:49:15.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.596+0000 7feeebfc0740 1 -- 172.21.15.172:0/135063695 wait complete. 2024-09-10T08:49:15.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.597+0000 7feeebfc0740 1 -- start start 2024-09-10T08:49:15.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.597+0000 7feeebfc0740 1 Processor -- start 2024-09-10T08:49:15.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.597+0000 7feeebfc0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a8215d0 0x55c03a82bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.597+0000 7feeebfc0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a822b40 0x55c03a829070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.597+0000 7feeebfc0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a82c370 0x55c03a8296d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.597+0000 7feeebfc0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c03a82f070 con 0x55c03a82c370 2024-09-10T08:49:15.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.597+0000 7feeebfc0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c03a742060 con 0x55c03a8215d0 2024-09-10T08:49:15.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.597+0000 7feeebfc0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c03a82f1f0 con 0x55c03a822b40 2024-09-10T08:49:15.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.597+0000 7feeebfc0740 10 client.0 _ll_register_callbacks cb 0x55c03a4958c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:15.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.598+0000 7feee3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a8215d0 0x55c03a82bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.598+0000 7feee3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a8215d0 0x55c03a82bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:56952/0 (socket says 172.21.15.172:56952) 2024-09-10T08:49:15.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.598+0000 7feee3fff640 1 -- 172.21.15.172:0/427031832 learned_addr learned my addr 172.21.15.172:0/427031832 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:15.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.598+0000 7feee3fff640 1 --2- 172.21.15.172:0/427031832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a82c370 0x55c03a8296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.598+0000 7feee2ffd640 1 --2- 172.21.15.172:0/427031832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a822b40 0x55c03a829070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.598+0000 7feee3fff640 1 -- 172.21.15.172:0/427031832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a822b40 msgr2=0x55c03a829070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:15.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.598+0000 7feee3fff640 1 --2- 172.21.15.172:0/427031832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a822b40 0x55c03a829070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.598+0000 7feee3fff640 1 -- 172.21.15.172:0/427031832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a82c370 msgr2=0x55c03a8296d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:15.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.598+0000 7feee3fff640 1 --2- 172.21.15.172:0/427031832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a82c370 0x55c03a8296d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.598+0000 7feee3fff640 1 -- 172.21.15.172:0/427031832 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c03a4f83c0 con 0x55c03a8215d0 2024-09-10T08:49:15.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.598+0000 7feee2ffd640 1 --2- 172.21.15.172:0/427031832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a822b40 0x55c03a829070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:49:15.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.598+0000 7feee3fff640 1 --2- 172.21.15.172:0/427031832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a8215d0 0x55c03a82bd70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feedc005a00 tx=0x7feedc005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b45ee33531b4d42c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:15.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.598+0000 7feec2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:49:15.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.598+0000 7feec2ffd640 1 -- 172.21.15.172:0/427031832 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc016040 con 0x55c03a8215d0 2024-09-10T08:49:15.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.598+0000 7feeebfc0740 1 -- 172.21.15.172:0/427031832 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feedc004d20 con 0x55c03a8215d0 2024-09-10T08:49:15.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.598+0000 7feec2ffd640 1 -- 172.21.15.172:0/427031832 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7feedc012830 con 0x55c03a8215d0 2024-09-10T08:49:15.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.598+0000 7feec2ffd640 1 -- 172.21.15.172:0/427031832 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7feedc02f050 con 0x55c03a8215d0 2024-09-10T08:49:15.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.599+0000 7feeebfc0740 10 client.25682 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:15.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.599+0000 7feeebfc0740 1 -- 172.21.15.172:0/427031832 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c03a83ab50 con 0x55c03a8215d0 2024-09-10T08:49:15.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.599+0000 7feec2ffd640 1 -- 172.21.15.172:0/427031832 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feedc004d20 con 0x55c03a8215d0 2024-09-10T08:49:15.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.599+0000 7feec2ffd640 1 -- 172.21.15.172:0/427031832 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7feedc00f040 con 0x55c03a8215d0 2024-09-10T08:49:15.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.599+0000 7feeebfc0740 10 client.25682 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:15.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.599+0000 7feeebfc0740 10 client.25682 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:49:15.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.599+0000 7feeebfc0740 20 client.25682 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:15.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.599+0000 7feeebfc0740 1 --2- 172.21.15.172:0/427031832 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55c03a83bf00 0x55c03a83e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.599+0000 7feeebfc0740 4 client.25682 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:49:15.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.599+0000 7feeebfc0740 1 -- 172.21.15.172:0/427031832 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c03a794860 con 0x55c03a83bf00 2024-09-10T08:49:15.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.599+0000 7feee2ffd640 1 --2- 172.21.15.172:0/427031832 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55c03a83bf00 0x55c03a83e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.600+0000 7feee2ffd640 1 --2- 172.21.15.172:0/427031832 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55c03a83bf00 0x55c03a83e2f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=adf08c6b5f66592f server_cookie=c4aa961da423d73f in_seq=0 out_seq=0 2024-09-10T08:49:15.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.600+0000 7feec2ffd640 10 client.25682 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:49:15.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.601+0000 7feec2ffd640 1 -- 172.21.15.172:0/427031832 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c03a794860 con 0x55c03a83bf00 2024-09-10T08:49:15.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.601+0000 7feec2ffd640 10 client.25682 handle_command_reply: tid=0 2024-09-10T08:49:15.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.601+0000 7feeebfc0740 1 client.25682 shutdown 2024-09-10T08:49:15.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.601+0000 7feeebfc0740 1 -- 172.21.15.172:0/427031832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a8215d0 msgr2=0x55c03a82bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:15.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.601+0000 7feeebfc0740 1 --2- 172.21.15.172:0/427031832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a8215d0 0x55c03a82bd70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feedc005a00 tx=0x7feedc005b00 comp rx=0 tx=0).stop 2024-09-10T08:49:15.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.601+0000 7feeebfc0740 1 -- 172.21.15.172:0/427031832 shutdown_connections 2024-09-10T08:49:15.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.601+0000 7feeebfc0740 1 --2- 172.21.15.172:0/427031832 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55c03a83bf00 0x55c03a83e2f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.601+0000 7feeebfc0740 1 --2- 172.21.15.172:0/427031832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a82c370 0x55c03a8296d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.601+0000 7feeebfc0740 1 --2- 172.21.15.172:0/427031832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a822b40 0x55c03a829070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.601+0000 7feeebfc0740 1 --2- 172.21.15.172:0/427031832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a8215d0 0x55c03a82bd70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.601+0000 7feec2ffd640 0 client.25682 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:49:15.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.601+0000 7feeebfc0740 1 -- 172.21.15.172:0/427031832 >> 172.21.15.172:0/427031832 conn(0x55c03a7ea310 msgr2=0x55c03a823930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:15.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.601+0000 7feeebfc0740 1 -- 172.21.15.172:0/427031832 shutdown_connections 2024-09-10T08:49:15.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.601+0000 7feeebfc0740 1 -- 172.21.15.172:0/427031832 wait complete. 2024-09-10T08:49:15.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.601+0000 7feeebfc0740 20 client.25682 trim_cache size 0 max 16384 2024-09-10T08:49:15.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.618+0000 7feeebfc0740 1 Processor -- start 2024-09-10T08:49:15.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.618+0000 7feeebfc0740 1 -- start start 2024-09-10T08:49:15.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.618+0000 7feeebfc0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a8215d0 0x55c03a8282e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.618+0000 7feeebfc0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a822b40 0x55c03a824120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.618+0000 7feeebfc0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a82c370 0x55c03a824660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.618+0000 7feeebfc0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feee41aeb50 con 0x55c03a822b40 2024-09-10T08:49:15.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.618+0000 7feeebfc0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feee41b06f0 con 0x55c03a82c370 2024-09-10T08:49:15.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.618+0000 7feeebfc0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7feee41af220 con 0x55c03a8215d0 2024-09-10T08:49:15.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.618+0000 7feee2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a822b40 0x55c03a824120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.619+0000 7feee2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a822b40 0x55c03a824120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48676/0 (socket says 172.21.15.172:48676) 2024-09-10T08:49:15.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.619+0000 7feee2ffd640 1 -- 172.21.15.172:0/1490862087 learned_addr learned my addr 172.21.15.172:0/1490862087 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:15.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.619+0000 7feee3fff640 1 --2- 172.21.15.172:0/1490862087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a8215d0 0x55c03a8282e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.619+0000 7feee3fff640 1 --2- 172.21.15.172:0/1490862087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a82c370 0x55c03a824660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.619+0000 7feee2ffd640 1 -- 172.21.15.172:0/1490862087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a8215d0 msgr2=0x55c03a8282e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:15.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.619+0000 7feee2ffd640 1 --2- 172.21.15.172:0/1490862087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a8215d0 0x55c03a8282e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.619+0000 7feee2ffd640 1 -- 172.21.15.172:0/1490862087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a82c370 msgr2=0x55c03a824660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:15.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.619+0000 7feee2ffd640 1 --2- 172.21.15.172:0/1490862087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a82c370 0x55c03a824660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.619+0000 7feee2ffd640 1 -- 172.21.15.172:0/1490862087 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c03a8275c0 con 0x55c03a822b40 2024-09-10T08:49:15.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.619+0000 7feee2ffd640 1 --2- 172.21.15.172:0/1490862087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a822b40 0x55c03a824120 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x55c03a822070 tx=0x7feed800bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546d9e697fc12432 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:15.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.619+0000 7feec97fa640 1 -- 172.21.15.172:0/1490862087 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8003de0 con 0x55c03a822b40 2024-09-10T08:49:15.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.619+0000 7feec97fa640 1 -- 172.21.15.172:0/1490862087 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feed80054d0 con 0x55c03a822b40 2024-09-10T08:49:15.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.619+0000 7feec97fa640 1 -- 172.21.15.172:0/1490862087 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8011440 con 0x55c03a822b40 2024-09-10T08:49:15.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.620+0000 7feeebfc0740 1 -- 172.21.15.172:0/1490862087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a822b40 msgr2=0x55c03a824120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:15.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.620+0000 7feeebfc0740 1 --2- 172.21.15.172:0/1490862087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a822b40 0x55c03a824120 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x55c03a822070 tx=0x7feed800bd20 comp rx=0 tx=0).stop 2024-09-10T08:49:15.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.620+0000 7feeebfc0740 1 -- 172.21.15.172:0/1490862087 shutdown_connections 2024-09-10T08:49:15.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.620+0000 7feeebfc0740 1 --2- 172.21.15.172:0/1490862087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a82c370 0x55c03a824660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.620+0000 7feeebfc0740 1 --2- 172.21.15.172:0/1490862087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a822b40 0x55c03a824120 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.620+0000 7feeebfc0740 1 --2- 172.21.15.172:0/1490862087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a8215d0 0x55c03a8282e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.620+0000 7feeebfc0740 1 -- 172.21.15.172:0/1490862087 >> 172.21.15.172:0/1490862087 conn(0x55c03a7ea310 msgr2=0x55c03a81cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:15.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.620+0000 7feeebfc0740 1 -- 172.21.15.172:0/1490862087 shutdown_connections 2024-09-10T08:49:15.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.620+0000 7feeebfc0740 1 -- 172.21.15.172:0/1490862087 wait complete. 2024-09-10T08:49:15.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.620+0000 7feeebfc0740 1 -- start start 2024-09-10T08:49:15.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.621+0000 7feeebfc0740 1 Processor -- start 2024-09-10T08:49:15.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.621+0000 7feeebfc0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a8215d0 0x55c03a834480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.621+0000 7feeebfc0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a822b40 0x55c03a8349c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.621+0000 7feeebfc0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a82c370 0x55c03a89fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.621+0000 7feeebfc0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feee41affc0 con 0x55c03a822b40 2024-09-10T08:49:15.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.621+0000 7feeebfc0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feee41af8f0 con 0x55c03a82c370 2024-09-10T08:49:15.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.621+0000 7feeebfc0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7feee41acb30 con 0x55c03a8215d0 2024-09-10T08:49:15.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.621+0000 7feeebfc0740 10 client.0 _ll_register_callbacks cb 0x7feed40f3470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:15.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.621+0000 7feee2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a822b40 0x55c03a8349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.621+0000 7feee2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a822b40 0x55c03a8349c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48682/0 (socket says 172.21.15.172:48682) 2024-09-10T08:49:15.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.621+0000 7feee2ffd640 1 -- 172.21.15.172:0/922076075 learned_addr learned my addr 172.21.15.172:0/922076075 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:15.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.621+0000 7feee3fff640 1 --2- 172.21.15.172:0/922076075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a82c370 0x55c03a89fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.621+0000 7feee3fff640 1 --2- 172.21.15.172:0/922076075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a8215d0 0x55c03a834480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.621+0000 7feee2ffd640 1 -- 172.21.15.172:0/922076075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a8215d0 msgr2=0x55c03a834480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:15.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.621+0000 7feee2ffd640 1 --2- 172.21.15.172:0/922076075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a8215d0 0x55c03a834480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.621+0000 7feee2ffd640 1 -- 172.21.15.172:0/922076075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a82c370 msgr2=0x55c03a89fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:15.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.622+0000 7feee2ffd640 1 --2- 172.21.15.172:0/922076075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a82c370 0x55c03a89fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.622+0000 7feee2ffd640 1 -- 172.21.15.172:0/922076075 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c03a835030 con 0x55c03a822b40 2024-09-10T08:49:15.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.622+0000 7feee2ffd640 1 --2- 172.21.15.172:0/922076075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a822b40 0x55c03a8349c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7feed8021000 tx=0x7feed8004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198ccf14f0abf6ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:15.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.622+0000 7feec2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:15.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.622+0000 7feec2ffd640 1 -- 172.21.15.172:0/922076075 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8005200 con 0x55c03a822b40 2024-09-10T08:49:15.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.622+0000 7feec2ffd640 1 -- 172.21.15.172:0/922076075 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7feed8004700 con 0x55c03a822b40 2024-09-10T08:49:15.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.622+0000 7feeebfc0740 1 -- 172.21.15.172:0/922076075 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed80054d0 con 0x55c03a822b40 2024-09-10T08:49:15.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.622+0000 7feec2ffd640 1 -- 172.21.15.172:0/922076075 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7feed8015020 con 0x55c03a822b40 2024-09-10T08:49:15.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.622+0000 7feeebfc0740 10 client.16338 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:15.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.622+0000 7feeebfc0740 1 -- 172.21.15.172:0/922076075 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c03a8ab370 con 0x55c03a822b40 2024-09-10T08:49:15.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.622+0000 7feec2ffd640 1 -- 172.21.15.172:0/922076075 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feed80054d0 con 0x55c03a822b40 2024-09-10T08:49:15.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.623+0000 7feec2ffd640 1 -- 172.21.15.172:0/922076075 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7feed8011af0 con 0x55c03a822b40 2024-09-10T08:49:15.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.623+0000 7feeebfc0740 10 client.16338 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:15.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.623+0000 7feeebfc0740 10 client.16338 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:49:15.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.623+0000 7feeebfc0740 20 client.16338 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:15.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.623+0000 7feeebfc0740 1 --2- 172.21.15.172:0/922076075 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55c03a8aba60 0x55c03a8ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:15.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.623+0000 7feeebfc0740 4 client.16338 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:49:15.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.623+0000 7feeebfc0740 1 -- 172.21.15.172:0/922076075 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c03a78ed70 con 0x55c03a8aba60 2024-09-10T08:49:15.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.623+0000 7feee3fff640 1 --2- 172.21.15.172:0/922076075 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55c03a8aba60 0x55c03a8ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:15.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.623+0000 7feec2ffd640 10 client.16338 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:49:15.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.623+0000 7feee3fff640 1 --2- 172.21.15.172:0/922076075 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55c03a8aba60 0x55c03a8ade80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7c77e65b558b30de server_cookie=68bc174b864c1200 in_seq=0 out_seq=0 2024-09-10T08:49:15.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.624+0000 7feec2ffd640 1 -- 172.21.15.172:0/922076075 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c03a78ed70 con 0x55c03a8aba60 2024-09-10T08:49:15.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.624+0000 7feec2ffd640 10 client.16338 handle_command_reply: tid=0 2024-09-10T08:49:15.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.624+0000 7feeebfc0740 1 client.16338 shutdown 2024-09-10T08:49:15.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.624+0000 7feeebfc0740 1 -- 172.21.15.172:0/922076075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a822b40 msgr2=0x55c03a8349c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:15.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.624+0000 7feeebfc0740 1 --2- 172.21.15.172:0/922076075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a822b40 0x55c03a8349c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7feed8021000 tx=0x7feed8004d90 comp rx=0 tx=0).stop 2024-09-10T08:49:15.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.624+0000 7feeebfc0740 1 -- 172.21.15.172:0/922076075 shutdown_connections 2024-09-10T08:49:15.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.624+0000 7feeebfc0740 1 --2- 172.21.15.172:0/922076075 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55c03a8aba60 0x55c03a8ade80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.624+0000 7feeebfc0740 1 --2- 172.21.15.172:0/922076075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c03a82c370 0x55c03a89fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.624+0000 7feec2ffd640 0 client.16338 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:49:15.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.624+0000 7feeebfc0740 1 --2- 172.21.15.172:0/922076075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c03a822b40 0x55c03a8349c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.624+0000 7feeebfc0740 1 --2- 172.21.15.172:0/922076075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c03a8215d0 0x55c03a834480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.624+0000 7feeebfc0740 1 -- 172.21.15.172:0/922076075 >> 172.21.15.172:0/922076075 conn(0x55c03a7ea310 msgr2=0x55c03a82a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:15.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.624+0000 7feeebfc0740 1 -- 172.21.15.172:0/922076075 shutdown_connections 2024-09-10T08:49:15.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.624+0000 7feeebfc0740 1 -- 172.21.15.172:0/922076075 wait complete. 2024-09-10T08:49:15.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.624+0000 7feeebfc0740 20 client.16338 trim_cache size 0 max 16384 2024-09-10T08:49:15.631 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:49:15.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.625+0000 7feee9be7640 1 -- 172.21.15.172:0/1626990170 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feecc0a2c00 msgr2=0x7feecc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:15.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.625+0000 7feee9be7640 1 --2- 172.21.15.172:0/1626990170 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feecc0a2c00 0x7feecc0a50c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feed4046000 tx=0x7feed40f35a0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.625+0000 7feee9be7640 1 -- 172.21.15.172:0/1626990170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee4101de0 msgr2=0x7feee4198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:15.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.625+0000 7feee9be7640 1 --2- 172.21.15.172:0/1626990170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee4101de0 0x7feee4198360 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7feed4005940 tx=0x7feed4004270 comp rx=0 tx=0).stop 2024-09-10T08:49:15.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.625+0000 7feee9be7640 1 -- 172.21.15.172:0/1626990170 shutdown_connections 2024-09-10T08:49:15.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.625+0000 7feee9be7640 1 --2- 172.21.15.172:0/1626990170 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feecc0a2c00 0x7feecc0a50c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.625+0000 7feee9be7640 1 --2- 172.21.15.172:0/1626990170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feee4106f10 0x7feee419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.625+0000 7feee9be7640 1 --2- 172.21.15.172:0/1626990170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feee4102790 0x7feee41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.625+0000 7feee9be7640 1 --2- 172.21.15.172:0/1626990170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee4101de0 0x7feee4198360 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:15.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.625+0000 7feee9be7640 1 -- 172.21.15.172:0/1626990170 >> 172.21.15.172:0/1626990170 conn(0x7feee40fd8e0 msgr2=0x7feee40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:15.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.625+0000 7feee9be7640 1 -- 172.21.15.172:0/1626990170 shutdown_connections 2024-09-10T08:49:15.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:15.625+0000 7feee9be7640 1 -- 172.21.15.172:0/1626990170 wait complete. 2024-09-10T08:49:16.420 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:49:16.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:16 smithi191 ceph-mon[35314]: pgmap v441: 129 pgs: 129 active+clean; 830 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 28 op/s 2024-09-10T08:49:16.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:16 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:49:16.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:16 smithi172 ceph-mon[30854]: pgmap v441: 129 pgs: 129 active+clean; 830 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 28 op/s 2024-09-10T08:49:16.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:16 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:49:16.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:16 smithi174 ceph-mon[34068]: pgmap v441: 129 pgs: 129 active+clean; 830 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 33 MiB/s wr, 28 op/s 2024-09-10T08:49:16.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:16 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:49:16.747 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:17.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.256+0000 7fa3bf436640 1 -- 172.21.15.172:0/3859612321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3b8102ef0 msgr2=0x7fa3b810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:17.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.256+0000 7fa3bf436640 1 --2- 172.21.15.172:0/3859612321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3b8102ef0 0x7fa3b810da40 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009900 tx=0x7fa3b002a480 comp rx=0 tx=0).stop 2024-09-10T08:49:17.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.256+0000 7fa3bf436640 1 -- 172.21.15.172:0/3859612321 shutdown_connections 2024-09-10T08:49:17.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.256+0000 7fa3bf436640 1 --2- 172.21.15.172:0/3859612321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3b8102ef0 0x7fa3b810da40 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.256+0000 7fa3bf436640 1 --2- 172.21.15.172:0/3859612321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3b8102570 0x7fa3b81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.256+0000 7fa3bf436640 1 --2- 172.21.15.172:0/3859612321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3b8100e20 0x7fa3b8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.256+0000 7fa3bf436640 1 -- 172.21.15.172:0/3859612321 >> 172.21.15.172:0/3859612321 conn(0x7fa3b80fc7d0 msgr2=0x7fa3b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:17.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.256+0000 7fa3bf436640 1 -- 172.21.15.172:0/3859612321 shutdown_connections 2024-09-10T08:49:17.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.256+0000 7fa3bf436640 1 -- 172.21.15.172:0/3859612321 wait complete. 2024-09-10T08:49:17.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.256+0000 7fa3bf436640 1 Processor -- start 2024-09-10T08:49:17.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bf436640 1 -- start start 2024-09-10T08:49:17.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bf436640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3b8100e20 0x7fa3b81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bf436640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3b8102570 0x7fa3b8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bf436640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3b8102ef0 0x7fa3b819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bf436640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b8110e30 con 0x7fa3b8100e20 2024-09-10T08:49:17.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bf436640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b8110cb0 con 0x7fa3b8102570 2024-09-10T08:49:17.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bf436640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b8110fb0 con 0x7fa3b8102ef0 2024-09-10T08:49:17.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bd1d2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3b8100e20 0x7fa3b81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:17.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bd1d2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3b8100e20 0x7fa3b81983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53612/0 (socket says 172.21.15.172:53612) 2024-09-10T08:49:17.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bd1d2640 1 -- 172.21.15.172:0/3915921575 learned_addr learned my addr 172.21.15.172:0/3915921575 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:17.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bc9d1640 1 --2- 172.21.15.172:0/3915921575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3b8102570 0x7fa3b8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:17.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bd9d3640 1 --2- 172.21.15.172:0/3915921575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3b8102ef0 0x7fa3b819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:17.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bd1d2640 1 -- 172.21.15.172:0/3915921575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3b8102ef0 msgr2=0x7fa3b819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:17.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bd1d2640 1 --2- 172.21.15.172:0/3915921575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3b8102ef0 0x7fa3b819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bd1d2640 1 -- 172.21.15.172:0/3915921575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3b8102570 msgr2=0x7fa3b8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:17.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bd1d2640 1 --2- 172.21.15.172:0/3915921575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3b8102570 0x7fa3b8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bd1d2640 1 -- 172.21.15.172:0/3915921575 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b819d3a0 con 0x7fa3b8100e20 2024-09-10T08:49:17.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.257+0000 7fa3bd1d2640 1 --2- 172.21.15.172:0/3915921575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3b8100e20 0x7fa3b81983f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400ebc0 tx=0x7fa3b400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:17.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.258+0000 7fa3ae7fc640 1 -- 172.21.15.172:0/3915921575 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4010630 con 0x7fa3b8100e20 2024-09-10T08:49:17.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.258+0000 7fa3ae7fc640 1 -- 172.21.15.172:0/3915921575 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3b4010c80 con 0x7fa3b8100e20 2024-09-10T08:49:17.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.258+0000 7fa3bf436640 1 -- 172.21.15.172:0/3915921575 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b819d690 con 0x7fa3b8100e20 2024-09-10T08:49:17.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.258+0000 7fa3ae7fc640 1 -- 172.21.15.172:0/3915921575 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4018b30 con 0x7fa3b8100e20 2024-09-10T08:49:17.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.258+0000 7fa3bf436640 1 -- 172.21.15.172:0/3915921575 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b81a4f70 con 0x7fa3b8100e20 2024-09-10T08:49:17.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.260+0000 7fa3ae7fc640 1 -- 172.21.15.172:0/3915921575 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa3b4018cd0 con 0x7fa3b8100e20 2024-09-10T08:49:17.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.261+0000 7fa3c180f740 1 Processor -- start 2024-09-10T08:49:17.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.261+0000 7fa3c180f740 1 -- start start 2024-09-10T08:49:17.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.261+0000 7fa3c180f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb76cb40 0x560ceb76b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.261+0000 7fa3c180f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76b5d0 0x560ceb76ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.261+0000 7fa3c180f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb776370 0x560ceb778760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.261+0000 7fa3c180f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560ceb6d12f0 con 0x560ceb76b5d0 2024-09-10T08:49:17.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.261+0000 7fa3c180f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560ceb678cf0 con 0x560ceb76cb40 2024-09-10T08:49:17.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.261+0000 7fa3c180f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560ceb6cd2b0 con 0x560ceb776370 2024-09-10T08:49:17.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.261+0000 7fa3bd1d2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76b5d0 0x560ceb76ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:17.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.261+0000 7fa3bd1d2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76b5d0 0x560ceb76ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53616/0 (socket says 172.21.15.172:53616) 2024-09-10T08:49:17.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.261+0000 7fa3bd1d2640 1 -- 172.21.15.172:0/53489879 learned_addr learned my addr 172.21.15.172:0/53489879 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:17.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.261+0000 7fa3bd9d3640 1 --2- 172.21.15.172:0/53489879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb76cb40 0x560ceb76b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:17.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.261+0000 7fa3ae7fc640 1 --2- 172.21.15.172:0/3915921575 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3a40a2c00 0x7fa3a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.261+0000 7fa3bc9d1640 1 --2- 172.21.15.172:0/53489879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb776370 0x560ceb778760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:17.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.262+0000 7fa3bd9d3640 1 --2- 172.21.15.172:0/3915921575 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3a40a2c00 0x7fa3a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:17.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.262+0000 7fa3ae7fc640 1 -- 172.21.15.172:0/3915921575 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa3b40cf970 con 0x7fa3b8100e20 2024-09-10T08:49:17.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.262+0000 7fa3bd1d2640 1 -- 172.21.15.172:0/53489879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb776370 msgr2=0x560ceb778760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:17.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.262+0000 7fa3bd1d2640 1 --2- 172.21.15.172:0/53489879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb776370 0x560ceb778760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.262+0000 7fa3bd1d2640 1 -- 172.21.15.172:0/53489879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb76cb40 msgr2=0x560ceb76b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:17.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.262+0000 7fa3bd1d2640 1 --2- 172.21.15.172:0/53489879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb76cb40 0x560ceb76b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.262+0000 7fa3bd1d2640 1 -- 172.21.15.172:0/53489879 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ceb65d3a0 con 0x560ceb76b5d0 2024-09-10T08:49:17.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.262+0000 7fa3bd1d2640 1 --2- 172.21.15.172:0/53489879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76b5d0 0x560ceb76ba10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa3b409bb70 tx=0x7fa3b400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5faec098154e5b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:17.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.262+0000 7fa3bd9d3640 1 --2- 172.21.15.172:0/3915921575 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3a40a2c00 0x7fa3a40a50c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0005c40 tx=0x7fa3b002a480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:17.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.262+0000 7fa3a37fe640 1 -- 172.21.15.172:0/53489879 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b409b7d0 con 0x560ceb76b5d0 2024-09-10T08:49:17.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.262+0000 7fa3a37fe640 1 -- 172.21.15.172:0/53489879 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3b409c570 con 0x560ceb76b5d0 2024-09-10T08:49:17.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.262+0000 7fa3a37fe640 1 -- 172.21.15.172:0/53489879 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b40984e0 con 0x560ceb76b5d0 2024-09-10T08:49:17.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.263+0000 7fa3c180f740 1 -- 172.21.15.172:0/53489879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76b5d0 msgr2=0x560ceb76ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:17.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.263+0000 7fa3c180f740 1 --2- 172.21.15.172:0/53489879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76b5d0 0x560ceb76ba10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa3b409bb70 tx=0x7fa3b400cf90 comp rx=0 tx=0).stop 2024-09-10T08:49:17.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.263+0000 7fa3c180f740 1 -- 172.21.15.172:0/53489879 shutdown_connections 2024-09-10T08:49:17.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.263+0000 7fa3c180f740 1 --2- 172.21.15.172:0/53489879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb776370 0x560ceb778760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.263+0000 7fa3c180f740 1 --2- 172.21.15.172:0/53489879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76b5d0 0x560ceb76ba10 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.263+0000 7fa3c180f740 1 --2- 172.21.15.172:0/53489879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb76cb40 0x560ceb76b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.263+0000 7fa3c180f740 1 -- 172.21.15.172:0/53489879 >> 172.21.15.172:0/53489879 conn(0x560ceb734310 msgr2=0x560ceb767c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:17.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.263+0000 7fa3c180f740 1 -- 172.21.15.172:0/53489879 shutdown_connections 2024-09-10T08:49:17.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.263+0000 7fa3c180f740 1 -- 172.21.15.172:0/53489879 wait complete. 2024-09-10T08:49:17.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.264+0000 7fa3c180f740 1 -- start start 2024-09-10T08:49:17.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.264+0000 7fa3c180f740 1 Processor -- start 2024-09-10T08:49:17.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.264+0000 7fa3c180f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb76b5d0 0x560ceb775d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.264+0000 7fa3c180f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb76cb40 0x560ceb773030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.264+0000 7fa3c180f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb776370 0x560ceb773640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.264+0000 7fa3c180f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560ceb779010 con 0x560ceb776370 2024-09-10T08:49:17.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.264+0000 7fa3c180f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560ceb68c060 con 0x560ceb76b5d0 2024-09-10T08:49:17.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.264+0000 7fa3c180f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560ceb779190 con 0x560ceb76cb40 2024-09-10T08:49:17.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.264+0000 7fa3c180f740 10 client.0 _ll_register_callbacks cb 0x560ceb3df8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:17.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3bd1d2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb76b5d0 0x560ceb775d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:17.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3bd9d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb776370 0x560ceb773640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:17.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3bd1d2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb76b5d0 0x560ceb775d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:36938/0 (socket says 172.21.15.172:36938) 2024-09-10T08:49:17.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3bd1d2640 1 -- 172.21.15.172:0/1581984277 learned_addr learned my addr 172.21.15.172:0/1581984277 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:17.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3bc9d1640 1 --2- 172.21.15.172:0/1581984277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb76cb40 0x560ceb773030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:17.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3bd1d2640 1 -- 172.21.15.172:0/1581984277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb76cb40 msgr2=0x560ceb773030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:17.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3bd1d2640 1 --2- 172.21.15.172:0/1581984277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb76cb40 0x560ceb773030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3bd1d2640 1 -- 172.21.15.172:0/1581984277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb776370 msgr2=0x560ceb773640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:17.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3bd1d2640 1 --2- 172.21.15.172:0/1581984277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb776370 0x560ceb773640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3bd1d2640 1 -- 172.21.15.172:0/1581984277 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ceb4423c0 con 0x560ceb76b5d0 2024-09-10T08:49:17.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3bd1d2640 1 --2- 172.21.15.172:0/1581984277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb76b5d0 0x560ceb775d80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa3b40982d0 tx=0x7fa3b409bae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70ac05b618a7846d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:17.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:49:17.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3a0ff9640 1 -- 172.21.15.172:0/1581984277 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b409b7d0 con 0x560ceb76b5d0 2024-09-10T08:49:17.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3a0ff9640 1 -- 172.21.15.172:0/1581984277 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa3b40109e0 con 0x560ceb76b5d0 2024-09-10T08:49:17.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3c180f740 1 -- 172.21.15.172:0/1581984277 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3b409c570 con 0x560ceb76b5d0 2024-09-10T08:49:17.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3a0ff9640 1 -- 172.21.15.172:0/1581984277 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa3b409be70 con 0x560ceb76b5d0 2024-09-10T08:49:17.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3c180f740 10 client.25700 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:17.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.265+0000 7fa3c180f740 1 -- 172.21.15.172:0/1581984277 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560ceb784ae0 con 0x560ceb76b5d0 2024-09-10T08:49:17.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.266+0000 7fa3a0ff9640 1 -- 172.21.15.172:0/1581984277 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3b409c570 con 0x560ceb76b5d0 2024-09-10T08:49:17.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.266+0000 7fa3a0ff9640 1 -- 172.21.15.172:0/1581984277 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fa3b407ace0 con 0x560ceb76b5d0 2024-09-10T08:49:17.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.266+0000 7fa3c180f740 10 client.25700 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:17.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.266+0000 7fa3c180f740 10 client.25700 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:49:17.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.266+0000 7fa3c180f740 20 client.25700 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:17.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.266+0000 7fa3c180f740 1 --2- 172.21.15.172:0/1581984277 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560ceb785e30 0x560ceb788220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.266+0000 7fa3c180f740 4 client.25700 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:49:17.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.266+0000 7fa3c180f740 1 -- 172.21.15.172:0/1581984277 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ceb6de860 con 0x560ceb785e30 2024-09-10T08:49:17.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.266+0000 7fa3bc9d1640 1 --2- 172.21.15.172:0/1581984277 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560ceb785e30 0x560ceb788220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:17.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.267+0000 7fa3bc9d1640 1 --2- 172.21.15.172:0/1581984277 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560ceb785e30 0x560ceb788220 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=490a0c22411e3958 server_cookie=d5469788c5f75a8 in_seq=0 out_seq=0 2024-09-10T08:49:17.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.267+0000 7fa3a0ff9640 10 client.25700 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:49:17.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.267+0000 7fa3a0ff9640 1 -- 172.21.15.172:0/1581984277 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ceb6de860 con 0x560ceb785e30 2024-09-10T08:49:17.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.267+0000 7fa3a0ff9640 10 client.25700 handle_command_reply: tid=0 2024-09-10T08:49:17.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.267+0000 7fa3c180f740 1 client.25700 shutdown 2024-09-10T08:49:17.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.267+0000 7fa3c180f740 1 -- 172.21.15.172:0/1581984277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb76b5d0 msgr2=0x560ceb775d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:17.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.267+0000 7fa3c180f740 1 --2- 172.21.15.172:0/1581984277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb76b5d0 0x560ceb775d80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa3b40982d0 tx=0x7fa3b409bae0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.267+0000 7fa3c180f740 1 -- 172.21.15.172:0/1581984277 shutdown_connections 2024-09-10T08:49:17.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.267+0000 7fa3c180f740 1 --2- 172.21.15.172:0/1581984277 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560ceb785e30 0x560ceb788220 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.267+0000 7fa3c180f740 1 --2- 172.21.15.172:0/1581984277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb776370 0x560ceb773640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.267+0000 7fa3c180f740 1 --2- 172.21.15.172:0/1581984277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb76cb40 0x560ceb773030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.267+0000 7fa3c180f740 1 --2- 172.21.15.172:0/1581984277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb76b5d0 0x560ceb775d80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.267+0000 7fa3a0ff9640 0 client.25700 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:49:17.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.267+0000 7fa3c180f740 1 -- 172.21.15.172:0/1581984277 >> 172.21.15.172:0/1581984277 conn(0x560ceb734310 msgr2=0x560ceb76d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:17.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.268+0000 7fa3c180f740 1 -- 172.21.15.172:0/1581984277 shutdown_connections 2024-09-10T08:49:17.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.268+0000 7fa3c180f740 1 -- 172.21.15.172:0/1581984277 wait complete. 2024-09-10T08:49:17.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.268+0000 7fa3c180f740 20 client.25700 trim_cache size 0 max 16384 2024-09-10T08:49:17.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.284+0000 7fa3c180f740 1 Processor -- start 2024-09-10T08:49:17.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.284+0000 7fa3c180f740 1 -- start start 2024-09-10T08:49:17.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.284+0000 7fa3c180f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76b5d0 0x560ceb7822b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.284+0000 7fa3c180f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb76cb40 0x560ceb766bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.284+0000 7fa3c180f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb776370 0x560ceb78d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.284+0000 7fa3c180f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b81ac830 con 0x560ceb76b5d0 2024-09-10T08:49:17.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.284+0000 7fa3c180f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b81ae3d0 con 0x560ceb76cb40 2024-09-10T08:49:17.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.284+0000 7fa3c180f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b81acf00 con 0x560ceb776370 2024-09-10T08:49:17.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.285+0000 7fa3bd1d2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76b5d0 0x560ceb7822b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:17.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.285+0000 7fa3bd1d2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76b5d0 0x560ceb7822b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53638/0 (socket says 172.21.15.172:53638) 2024-09-10T08:49:17.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.285+0000 7fa3bd1d2640 1 -- 172.21.15.172:0/3809654192 learned_addr learned my addr 172.21.15.172:0/3809654192 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:17.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.285+0000 7fa3bc9d1640 1 --2- 172.21.15.172:0/3809654192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb76cb40 0x560ceb766bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:17.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.285+0000 7fa3bd1d2640 1 -- 172.21.15.172:0/3809654192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb776370 msgr2=0x560ceb78d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:17.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.285+0000 7fa3bd1d2640 1 --2- 172.21.15.172:0/3809654192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb776370 0x560ceb78d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.285+0000 7fa3bd1d2640 1 -- 172.21.15.172:0/3809654192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb76cb40 msgr2=0x560ceb766bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:17.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.285+0000 7fa3bd1d2640 1 --2- 172.21.15.172:0/3809654192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb76cb40 0x560ceb766bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.285+0000 7fa3bd1d2640 1 -- 172.21.15.172:0/3809654192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ceb771fe0 con 0x560ceb76b5d0 2024-09-10T08:49:17.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.285+0000 7fa3bd1d2640 1 --2- 172.21.15.172:0/3809654192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76b5d0 0x560ceb7822b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4098840 tx=0x7fa3b40bc090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c423f08b16842a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:17.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.285+0000 7fa3a2ffd640 1 -- 172.21.15.172:0/3809654192 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b409b7d0 con 0x560ceb76b5d0 2024-09-10T08:49:17.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.285+0000 7fa3a2ffd640 1 -- 172.21.15.172:0/3809654192 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3b409c570 con 0x560ceb76b5d0 2024-09-10T08:49:17.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.285+0000 7fa3a2ffd640 1 -- 172.21.15.172:0/3809654192 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4098110 con 0x560ceb76b5d0 2024-09-10T08:49:17.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.286+0000 7fa3c180f740 1 -- 172.21.15.172:0/3809654192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76b5d0 msgr2=0x560ceb7822b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:17.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.286+0000 7fa3c180f740 1 --2- 172.21.15.172:0/3809654192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76b5d0 0x560ceb7822b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4098840 tx=0x7fa3b40bc090 comp rx=0 tx=0).stop 2024-09-10T08:49:17.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.286+0000 7fa3c180f740 1 -- 172.21.15.172:0/3809654192 shutdown_connections 2024-09-10T08:49:17.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.286+0000 7fa3c180f740 1 --2- 172.21.15.172:0/3809654192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb776370 0x560ceb78d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.286+0000 7fa3c180f740 1 --2- 172.21.15.172:0/3809654192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb76cb40 0x560ceb766bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.286+0000 7fa3c180f740 1 --2- 172.21.15.172:0/3809654192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76b5d0 0x560ceb7822b0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.286+0000 7fa3c180f740 1 -- 172.21.15.172:0/3809654192 >> 172.21.15.172:0/3809654192 conn(0x560ceb734310 msgr2=0x560ceb7664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:17.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.286+0000 7fa3c180f740 1 -- 172.21.15.172:0/3809654192 shutdown_connections 2024-09-10T08:49:17.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.286+0000 7fa3c180f740 1 -- 172.21.15.172:0/3809654192 wait complete. 2024-09-10T08:49:17.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.287+0000 7fa3c180f740 1 -- start start 2024-09-10T08:49:17.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.287+0000 7fa3c180f740 1 Processor -- start 2024-09-10T08:49:17.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.287+0000 7fa3c180f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb76b5d0 0x560ceb77a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.287+0000 7fa3c180f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76cb40 0x560ceb77a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.287+0000 7fa3c180f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb776370 0x560ceb77abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.287+0000 7fa3c180f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b81adca0 con 0x560ceb76cb40 2024-09-10T08:49:17.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.287+0000 7fa3c180f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b81ad5d0 con 0x560ceb776370 2024-09-10T08:49:17.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.287+0000 7fa3c180f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b81aa810 con 0x560ceb76b5d0 2024-09-10T08:49:17.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.287+0000 7fa3c180f740 10 client.0 _ll_register_callbacks cb 0x7fa3b40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:17.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.287+0000 7fa3bc9d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76cb40 0x560ceb77a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:17.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.287+0000 7fa3bc9d1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76cb40 0x560ceb77a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53640/0 (socket says 172.21.15.172:53640) 2024-09-10T08:49:17.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.287+0000 7fa3bc9d1640 1 -- 172.21.15.172:0/1903015038 learned_addr learned my addr 172.21.15.172:0/1903015038 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:17.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.287+0000 7fa3bd1d2640 1 --2- 172.21.15.172:0/1903015038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb76b5d0 0x560ceb77a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:17.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.288+0000 7fa3bd9d3640 1 --2- 172.21.15.172:0/1903015038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb776370 0x560ceb77abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:17.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.288+0000 7fa3bc9d1640 1 -- 172.21.15.172:0/1903015038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb76b5d0 msgr2=0x560ceb77a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:17.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.288+0000 7fa3bc9d1640 1 --2- 172.21.15.172:0/1903015038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb76b5d0 0x560ceb77a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.288+0000 7fa3bc9d1640 1 -- 172.21.15.172:0/1903015038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb776370 msgr2=0x560ceb77abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:17.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.288+0000 7fa3bc9d1640 1 --2- 172.21.15.172:0/1903015038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb776370 0x560ceb77abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.288+0000 7fa3bc9d1640 1 -- 172.21.15.172:0/1903015038 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ceb7ef790 con 0x560ceb76cb40 2024-09-10T08:49:17.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.288+0000 7fa3bc9d1640 1 --2- 172.21.15.172:0/1903015038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76cb40 0x560ceb77a690 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x560ceb76c070 tx=0x7fa3a80109b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04752cbd3ffc983 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:17.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.288+0000 7fa3a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:17.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.288+0000 7fa3a0ff9640 1 -- 172.21.15.172:0/1903015038 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8022070 con 0x560ceb76cb40 2024-09-10T08:49:17.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.288+0000 7fa3a0ff9640 1 -- 172.21.15.172:0/1903015038 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa3a801f4c0 con 0x560ceb76cb40 2024-09-10T08:49:17.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.288+0000 7fa3c180f740 1 -- 172.21.15.172:0/1903015038 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3b409c570 con 0x560ceb76cb40 2024-09-10T08:49:17.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.288+0000 7fa3a0ff9640 1 -- 172.21.15.172:0/1903015038 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa3a801f6c0 con 0x560ceb76cb40 2024-09-10T08:49:17.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.288+0000 7fa3c180f740 10 client.16356 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:17.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.288+0000 7fa3c180f740 1 -- 172.21.15.172:0/1903015038 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560ceb7f0ad0 con 0x560ceb76cb40 2024-09-10T08:49:17.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.288+0000 7fa3a0ff9640 1 -- 172.21.15.172:0/1903015038 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3b409c570 con 0x560ceb76cb40 2024-09-10T08:49:17.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.289+0000 7fa3a0ff9640 1 -- 172.21.15.172:0/1903015038 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fa3a8003c50 con 0x560ceb76cb40 2024-09-10T08:49:17.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.289+0000 7fa3c180f740 10 client.16356 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:17.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.289+0000 7fa3c180f740 10 client.16356 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:49:17.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.289+0000 7fa3c180f740 20 client.16356 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:17.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.289+0000 7fa3c180f740 1 --2- 172.21.15.172:0/1903015038 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560ceb7f1280 0x560ceb7f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:17.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.289+0000 7fa3c180f740 4 client.16356 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:49:17.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.289+0000 7fa3c180f740 1 -- 172.21.15.172:0/1903015038 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ceb6d8d70 con 0x560ceb7f1280 2024-09-10T08:49:17.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.289+0000 7fa3bd1d2640 1 --2- 172.21.15.172:0/1903015038 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560ceb7f1280 0x560ceb7f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:17.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.289+0000 7fa3bd1d2640 1 --2- 172.21.15.172:0/1903015038 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560ceb7f1280 0x560ceb7f36a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=707d153d860857e8 server_cookie=d2fc235402f99b30 in_seq=0 out_seq=0 2024-09-10T08:49:17.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.289+0000 7fa3a0ff9640 10 client.16356 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:49:17.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.290+0000 7fa3a0ff9640 1 -- 172.21.15.172:0/1903015038 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ceb6d8d70 con 0x560ceb7f1280 2024-09-10T08:49:17.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.290+0000 7fa3a0ff9640 10 client.16356 handle_command_reply: tid=0 2024-09-10T08:49:17.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.290+0000 7fa3c180f740 1 client.16356 shutdown 2024-09-10T08:49:17.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.290+0000 7fa3c180f740 1 -- 172.21.15.172:0/1903015038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76cb40 msgr2=0x560ceb77a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:17.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.290+0000 7fa3c180f740 1 --2- 172.21.15.172:0/1903015038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76cb40 0x560ceb77a690 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x560ceb76c070 tx=0x7fa3a80109b0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.290+0000 7fa3c180f740 1 -- 172.21.15.172:0/1903015038 shutdown_connections 2024-09-10T08:49:17.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.290+0000 7fa3c180f740 1 --2- 172.21.15.172:0/1903015038 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x560ceb7f1280 0x560ceb7f36a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.290+0000 7fa3c180f740 1 --2- 172.21.15.172:0/1903015038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ceb776370 0x560ceb77abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.290+0000 7fa3c180f740 1 --2- 172.21.15.172:0/1903015038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ceb76cb40 0x560ceb77a690 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.290+0000 7fa3c180f740 1 --2- 172.21.15.172:0/1903015038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560ceb76b5d0 0x560ceb77a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.290+0000 7fa3c180f740 1 -- 172.21.15.172:0/1903015038 >> 172.21.15.172:0/1903015038 conn(0x560ceb734310 msgr2=0x560ceb766d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:17.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.290+0000 7fa3a0ff9640 0 client.16356 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:49:17.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.290+0000 7fa3c180f740 1 -- 172.21.15.172:0/1903015038 shutdown_connections 2024-09-10T08:49:17.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.290+0000 7fa3c180f740 1 -- 172.21.15.172:0/1903015038 wait complete. 2024-09-10T08:49:17.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.290+0000 7fa3c180f740 20 client.16356 trim_cache size 0 max 16384 2024-09-10T08:49:17.297 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:49:17.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.291+0000 7fa3bf436640 1 -- 172.21.15.172:0/3915921575 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3a40a2c00 msgr2=0x7fa3a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:17.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.291+0000 7fa3bf436640 1 --2- 172.21.15.172:0/3915921575 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3a40a2c00 0x7fa3a40a50c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0005c40 tx=0x7fa3b002a480 comp rx=0 tx=0).stop 2024-09-10T08:49:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.291+0000 7fa3bf436640 1 -- 172.21.15.172:0/3915921575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3b8100e20 msgr2=0x7fa3b81983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.291+0000 7fa3bf436640 1 --2- 172.21.15.172:0/3915921575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3b8100e20 0x7fa3b81983f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400ebc0 tx=0x7fa3b400c6a0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.291+0000 7fa3bf436640 1 -- 172.21.15.172:0/3915921575 shutdown_connections 2024-09-10T08:49:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.291+0000 7fa3bf436640 1 --2- 172.21.15.172:0/3915921575 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3a40a2c00 0x7fa3a40a50c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.291+0000 7fa3bf436640 1 --2- 172.21.15.172:0/3915921575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3b8102ef0 0x7fa3b819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.291+0000 7fa3bf436640 1 --2- 172.21.15.172:0/3915921575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3b8102570 0x7fa3b8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.291+0000 7fa3bf436640 1 --2- 172.21.15.172:0/3915921575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3b8100e20 0x7fa3b81983f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.291+0000 7fa3bf436640 1 -- 172.21.15.172:0/3915921575 >> 172.21.15.172:0/3915921575 conn(0x7fa3b80fc7d0 msgr2=0x7fa3b8069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.291+0000 7fa3bf436640 1 -- 172.21.15.172:0/3915921575 shutdown_connections 2024-09-10T08:49:17.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:17.291+0000 7fa3bf436640 1 -- 172.21.15.172:0/3915921575 wait complete. 2024-09-10T08:49:17.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:49:17.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:49:17.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:49:17.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:49:17.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:49:17.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:49:17.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:49:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:49:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:49:18.134 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:49:18.418 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:18.542 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:18 smithi172 ceph-mon[30854]: pgmap v442: 129 pgs: 129 active+clean; 906 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 27 op/s 2024-09-10T08:49:18.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:18 smithi191 ceph-mon[35314]: pgmap v442: 129 pgs: 129 active+clean; 906 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 27 op/s 2024-09-10T08:49:18.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:18 smithi174 ceph-mon[34068]: pgmap v442: 129 pgs: 129 active+clean; 906 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 MiB/s wr, 27 op/s 2024-09-10T08:49:18.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.967+0000 7f6e839a2640 1 -- 172.21.15.172:0/3698982005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e7c102ef0 msgr2=0x7f6e7c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:18.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.967+0000 7f6e839a2640 1 --2- 172.21.15.172:0/3698982005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e7c102ef0 0x7f6e7c10da40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009ca0 tx=0x7f6e7402a950 comp rx=0 tx=0).stop 2024-09-10T08:49:18.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.967+0000 7f6e839a2640 1 -- 172.21.15.172:0/3698982005 shutdown_connections 2024-09-10T08:49:18.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.967+0000 7f6e839a2640 1 --2- 172.21.15.172:0/3698982005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e7c102ef0 0x7f6e7c10da40 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:18.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.967+0000 7f6e839a2640 1 --2- 172.21.15.172:0/3698982005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e7c102570 0x7f6e7c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:18.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.967+0000 7f6e839a2640 1 --2- 172.21.15.172:0/3698982005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e7c100e20 0x7f6e7c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:18.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.967+0000 7f6e839a2640 1 -- 172.21.15.172:0/3698982005 >> 172.21.15.172:0/3698982005 conn(0x7f6e7c0fc7d0 msgr2=0x7f6e7c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:18.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.968+0000 7f6e839a2640 1 -- 172.21.15.172:0/3698982005 shutdown_connections 2024-09-10T08:49:18.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.968+0000 7f6e839a2640 1 -- 172.21.15.172:0/3698982005 wait complete. 2024-09-10T08:49:18.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.968+0000 7f6e839a2640 1 Processor -- start 2024-09-10T08:49:18.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.968+0000 7f6e839a2640 1 -- start start 2024-09-10T08:49:18.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.968+0000 7f6e839a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e7c100e20 0x7f6e7c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:18.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.968+0000 7f6e839a2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e7c102570 0x7f6e7c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:18.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.968+0000 7f6e839a2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e7c102ef0 0x7f6e7c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:18.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e839a2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c110e80 con 0x7f6e7c102570 2024-09-10T08:49:18.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e839a2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c110d00 con 0x7f6e7c100e20 2024-09-10T08:49:18.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e839a2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c111000 con 0x7f6e7c102ef0 2024-09-10T08:49:18.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e80f3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e7c102570 0x7f6e7c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:18.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e80f3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e7c102570 0x7f6e7c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53654/0 (socket says 172.21.15.172:53654) 2024-09-10T08:49:18.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e80f3d640 1 -- 172.21.15.172:0/890860068 learned_addr learned my addr 172.21.15.172:0/890860068 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:18.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e8173e640 1 --2- 172.21.15.172:0/890860068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e7c100e20 0x7f6e7c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:18.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e81f3f640 1 --2- 172.21.15.172:0/890860068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e7c102ef0 0x7f6e7c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:18.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e80f3d640 1 -- 172.21.15.172:0/890860068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e7c102ef0 msgr2=0x7f6e7c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:18.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e80f3d640 1 --2- 172.21.15.172:0/890860068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e7c102ef0 0x7f6e7c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:18.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e80f3d640 1 -- 172.21.15.172:0/890860068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e7c100e20 msgr2=0x7f6e7c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:18.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e80f3d640 1 --2- 172.21.15.172:0/890860068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e7c100e20 0x7f6e7c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:18.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e80f3d640 1 -- 172.21.15.172:0/890860068 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7c19d380 con 0x7f6e7c102570 2024-09-10T08:49:18.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e80f3d640 1 --2- 172.21.15.172:0/890860068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e7c102570 0x7f6e7c198910 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00c970 tx=0x7f6e6c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:18.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e727fc640 1 -- 172.21.15.172:0/890860068 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c007bb0 con 0x7f6e7c102570 2024-09-10T08:49:18.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e839a2640 1 -- 172.21.15.172:0/890860068 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7c19d670 con 0x7f6e7c102570 2024-09-10T08:49:18.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e727fc640 1 -- 172.21.15.172:0/890860068 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e6c007d50 con 0x7f6e7c102570 2024-09-10T08:49:18.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e727fc640 1 -- 172.21.15.172:0/890860068 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c00f450 con 0x7f6e7c102570 2024-09-10T08:49:18.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.969+0000 7f6e839a2640 1 -- 172.21.15.172:0/890860068 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e7c1a4f50 con 0x7f6e7c102570 2024-09-10T08:49:18.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.972+0000 7f6e727fc640 1 -- 172.21.15.172:0/890860068 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6e6c004f60 con 0x7f6e7c102570 2024-09-10T08:49:18.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.973+0000 7f6e85d7b740 1 Processor -- start 2024-09-10T08:49:18.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.973+0000 7f6e85d7b740 1 -- start start 2024-09-10T08:49:18.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.973+0000 7f6e727fc640 1 --2- 172.21.15.172:0/890860068 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e680a2c00 0x7f6e680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:18.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.973+0000 7f6e85d7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17272b40 0x564c17271090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:18.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.973+0000 7f6e85d7b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c172715d0 0x564c17271a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:18.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.973+0000 7f6e85d7b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17280780 0x564c17282b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:18.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.973+0000 7f6e727fc640 1 -- 172.21.15.172:0/890860068 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6e6c0cef70 con 0x7f6e7c102570 2024-09-10T08:49:18.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.973+0000 7f6e85d7b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564c171d72f0 con 0x564c17280780 2024-09-10T08:49:18.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.973+0000 7f6e85d7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564c1717ecf0 con 0x564c17272b40 2024-09-10T08:49:18.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.973+0000 7f6e85d7b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564c171d32b0 con 0x564c172715d0 2024-09-10T08:49:18.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.973+0000 7f6e81f3f640 1 --2- 172.21.15.172:0/890860068 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e680a2c00 0x7f6e680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:18.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.973+0000 7f6e80f3d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c172715d0 0x564c17271a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:18.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.973+0000 7f6e8173e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17272b40 0x564c17271090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:18.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.973+0000 7f6e80f3d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c172715d0 0x564c17271a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:54374/0 (socket says 172.21.15.172:54374) 2024-09-10T08:49:18.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.973+0000 7f6e80f3d640 1 -- 172.21.15.172:0/1143680060 learned_addr learned my addr 172.21.15.172:0/1143680060 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:18.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.974+0000 7f6e81f3f640 1 --2- 172.21.15.172:0/1143680060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17280780 0x564c17282b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:18.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.974+0000 7f6e8173e640 1 -- 172.21.15.172:0/1143680060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c172715d0 msgr2=0x564c17271a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:18.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.974+0000 7f6e8173e640 1 --2- 172.21.15.172:0/1143680060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c172715d0 0x564c17271a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:18.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.974+0000 7f6e8173e640 1 -- 172.21.15.172:0/1143680060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17280780 msgr2=0x564c17282b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:18.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.974+0000 7f6e8173e640 1 --2- 172.21.15.172:0/1143680060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17280780 0x564c17282b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:18.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.974+0000 7f6e8173e640 1 -- 172.21.15.172:0/1143680060 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c171633a0 con 0x564c17272b40 2024-09-10T08:49:18.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.974+0000 7f6e80f3d640 1 --2- 172.21.15.172:0/1143680060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c172715d0 0x564c17271a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:18.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.974+0000 7f6e81f3f640 1 --2- 172.21.15.172:0/890860068 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e680a2c00 0x7f6e680a50c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6e74006010 tx=0x7f6e74003970 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:18.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.974+0000 7f6e8173e640 1 --2- 172.21.15.172:0/1143680060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17272b40 0x564c17271090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6e78005fd0 tx=0x7f6e780107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7871df3316612ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:18.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.974+0000 7f6e677fe640 1 -- 172.21.15.172:0/1143680060 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7801c680 con 0x564c17272b40 2024-09-10T08:49:18.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.974+0000 7f6e677fe640 1 -- 172.21.15.172:0/1143680060 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e7800be20 con 0x564c17272b40 2024-09-10T08:49:18.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.974+0000 7f6e677fe640 1 -- 172.21.15.172:0/1143680060 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e78011b30 con 0x564c17272b40 2024-09-10T08:49:18.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.975+0000 7f6e85d7b740 1 -- 172.21.15.172:0/1143680060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17272b40 msgr2=0x564c17271090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:18.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.975+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/1143680060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17272b40 0x564c17271090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6e78005fd0 tx=0x7f6e780107a0 comp rx=0 tx=0).stop 2024-09-10T08:49:18.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.975+0000 7f6e85d7b740 1 -- 172.21.15.172:0/1143680060 shutdown_connections 2024-09-10T08:49:18.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.975+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/1143680060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17280780 0x564c17282b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:18.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.975+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/1143680060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c172715d0 0x564c17271a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:18.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.975+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/1143680060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17272b40 0x564c17271090 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:18.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.975+0000 7f6e85d7b740 1 -- 172.21.15.172:0/1143680060 >> 172.21.15.172:0/1143680060 conn(0x564c1723a310 msgr2=0x564c1726dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:18.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.975+0000 7f6e85d7b740 1 -- 172.21.15.172:0/1143680060 shutdown_connections 2024-09-10T08:49:18.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.975+0000 7f6e85d7b740 1 -- 172.21.15.172:0/1143680060 wait complete. 2024-09-10T08:49:18.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.976+0000 7f6e85d7b740 1 -- start start 2024-09-10T08:49:18.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.976+0000 7f6e85d7b740 1 Processor -- start 2024-09-10T08:49:18.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.976+0000 7f6e85d7b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c172715d0 0x564c17280110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:18.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.976+0000 7f6e85d7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17272b40 0x564c1727d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:18.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.976+0000 7f6e85d7b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17280780 0x564c1727d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:18.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.976+0000 7f6e85d7b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564c17283480 con 0x564c17280780 2024-09-10T08:49:18.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.976+0000 7f6e85d7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564c17192060 con 0x564c17272b40 2024-09-10T08:49:18.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.976+0000 7f6e85d7b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564c17283600 con 0x564c172715d0 2024-09-10T08:49:18.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.976+0000 7f6e85d7b740 10 client.0 _ll_register_callbacks cb 0x564c16ee58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:18.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.976+0000 7f6e81f3f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17280780 0x564c1727d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:18.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.976+0000 7f6e81f3f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17280780 0x564c1727d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53666/0 (socket says 172.21.15.172:53666) 2024-09-10T08:49:18.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.976+0000 7f6e81f3f640 1 -- 172.21.15.172:0/2008106981 learned_addr learned my addr 172.21.15.172:0/2008106981 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:18.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.977+0000 7f6e80f3d640 1 --2- 172.21.15.172:0/2008106981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17272b40 0x564c1727d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:18.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.977+0000 7f6e8173e640 1 --2- 172.21.15.172:0/2008106981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c172715d0 0x564c17280110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:18.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.977+0000 7f6e81f3f640 1 -- 172.21.15.172:0/2008106981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c172715d0 msgr2=0x564c17280110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:18.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.977+0000 7f6e81f3f640 1 --2- 172.21.15.172:0/2008106981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c172715d0 0x564c17280110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:18.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.977+0000 7f6e81f3f640 1 -- 172.21.15.172:0/2008106981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17272b40 msgr2=0x564c1727d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:18.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.977+0000 7f6e81f3f640 1 --2- 172.21.15.172:0/2008106981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17272b40 0x564c1727d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:18.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.977+0000 7f6e81f3f640 1 -- 172.21.15.172:0/2008106981 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c16f483c0 con 0x564c17280780 2024-09-10T08:49:18.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.977+0000 7f6e80f3d640 1 --2- 172.21.15.172:0/2008106981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17272b40 0x564c1727d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:49:18.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.977+0000 7f6e81f3f640 1 --2- 172.21.15.172:0/2008106981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17280780 0x564c1727d940 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6e74035000 tx=0x7f6e7402a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81eed3f4c5b8108 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:18.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.977+0000 7f6e64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:18.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.977+0000 7f6e64ff9640 1 -- 172.21.15.172:0/2008106981 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74043040 con 0x564c17280780 2024-09-10T08:49:18.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.977+0000 7f6e64ff9640 1 -- 172.21.15.172:0/2008106981 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6e740307c0 con 0x564c17280780 2024-09-10T08:49:18.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.977+0000 7f6e85d7b740 1 -- 172.21.15.172:0/2008106981 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e7800be20 con 0x564c17280780 2024-09-10T08:49:18.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.977+0000 7f6e64ff9640 1 -- 172.21.15.172:0/2008106981 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6e74034040 con 0x564c17280780 2024-09-10T08:49:18.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.977+0000 7f6e85d7b740 10 client.16368 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:18.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.977+0000 7f6e85d7b740 1 -- 172.21.15.172:0/2008106981 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564c1728eeb0 con 0x564c17280780 2024-09-10T08:49:18.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.978+0000 7f6e64ff9640 1 -- 172.21.15.172:0/2008106981 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e7800be20 con 0x564c17280780 2024-09-10T08:49:18.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.978+0000 7f6e64ff9640 1 -- 172.21.15.172:0/2008106981 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f6e7404bac0 con 0x564c17280780 2024-09-10T08:49:18.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.978+0000 7f6e85d7b740 10 client.16368 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:18.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.978+0000 7f6e85d7b740 10 client.16368 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:49:18.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.978+0000 7f6e85d7b740 20 client.16368 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:18.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.978+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/2008106981 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x564c17290370 0x564c172927e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:18.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.978+0000 7f6e85d7b740 4 client.16368 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:49:18.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.978+0000 7f6e85d7b740 1 -- 172.21.15.172:0/2008106981 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c171e4860 con 0x564c17290370 2024-09-10T08:49:18.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.978+0000 7f6e8173e640 1 --2- 172.21.15.172:0/2008106981 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x564c17290370 0x564c172927e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:18.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.979+0000 7f6e8173e640 1 --2- 172.21.15.172:0/2008106981 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x564c17290370 0x564c172927e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a1b33bf67580ea29 server_cookie=306232523c2336a1 in_seq=0 out_seq=0 2024-09-10T08:49:18.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.979+0000 7f6e64ff9640 10 client.16368 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:49:18.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.979+0000 7f6e64ff9640 1 -- 172.21.15.172:0/2008106981 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c171e4860 con 0x564c17290370 2024-09-10T08:49:18.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.979+0000 7f6e64ff9640 10 client.16368 handle_command_reply: tid=0 2024-09-10T08:49:18.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.979+0000 7f6e85d7b740 1 client.16368 shutdown 2024-09-10T08:49:18.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.980+0000 7f6e85d7b740 1 -- 172.21.15.172:0/2008106981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17280780 msgr2=0x564c1727d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:18.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.980+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/2008106981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17280780 0x564c1727d940 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6e74035000 tx=0x7f6e7402a950 comp rx=0 tx=0).stop 2024-09-10T08:49:18.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.980+0000 7f6e85d7b740 1 -- 172.21.15.172:0/2008106981 shutdown_connections 2024-09-10T08:49:18.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.980+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/2008106981 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x564c17290370 0x564c172927e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:18.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.980+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/2008106981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17280780 0x564c1727d940 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:18.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.980+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/2008106981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17272b40 0x564c1727d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:18.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.980+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/2008106981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c172715d0 0x564c17280110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:18.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.980+0000 7f6e85d7b740 1 -- 172.21.15.172:0/2008106981 >> 172.21.15.172:0/2008106981 conn(0x564c1723a310 msgr2=0x564c17273960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:18.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.980+0000 7f6e64ff9640 0 client.16368 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:49:18.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.980+0000 7f6e85d7b740 1 -- 172.21.15.172:0/2008106981 shutdown_connections 2024-09-10T08:49:18.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.980+0000 7f6e85d7b740 1 -- 172.21.15.172:0/2008106981 wait complete. 2024-09-10T08:49:18.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.980+0000 7f6e85d7b740 20 client.16368 trim_cache size 0 max 16384 2024-09-10T08:49:19.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.993+0000 7f6e85d7b740 1 Processor -- start 2024-09-10T08:49:19.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.993+0000 7f6e85d7b740 1 -- start start 2024-09-10T08:49:19.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e85d7b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c172715d0 0x564c1728c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:19.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e85d7b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17272b40 0x564c17298100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:19.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e85d7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17280780 0x564c17298640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:19.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e85d7b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c1ac8d0 con 0x564c17272b40 2024-09-10T08:49:19.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e85d7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c1ae470 con 0x564c17280780 2024-09-10T08:49:19.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e85d7b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c1acfa0 con 0x564c172715d0 2024-09-10T08:49:19.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e80f3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17272b40 0x564c17298100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:19.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e81f3f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17280780 0x564c17298640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:19.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e81f3f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17280780 0x564c17298640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:37014/0 (socket says 172.21.15.172:37014) 2024-09-10T08:49:19.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e81f3f640 1 -- 172.21.15.172:0/1770701727 learned_addr learned my addr 172.21.15.172:0/1770701727 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:19.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e80f3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17272b40 0x564c17298100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53676/0 (socket says 172.21.15.172:53676) 2024-09-10T08:49:19.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e8173e640 1 --2- 172.21.15.172:0/1770701727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c172715d0 0x564c1728c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:19.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e81f3f640 1 -- 172.21.15.172:0/1770701727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c172715d0 msgr2=0x564c1728c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:19.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e81f3f640 1 --2- 172.21.15.172:0/1770701727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c172715d0 0x564c1728c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:19.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e81f3f640 1 -- 172.21.15.172:0/1770701727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17272b40 msgr2=0x564c17298100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:19.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e81f3f640 1 --2- 172.21.15.172:0/1770701727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17272b40 0x564c17298100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:19.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e81f3f640 1 -- 172.21.15.172:0/1770701727 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c17274120 con 0x564c17280780 2024-09-10T08:49:19.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.994+0000 7f6e81f3f640 1 --2- 172.21.15.172:0/1770701727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17280780 0x564c17298640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6e7402ae70 tx=0x7f6e74009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb23093d65dee79b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:19.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.995+0000 7f6e66ffd640 1 -- 172.21.15.172:0/1770701727 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74066070 con 0x564c17280780 2024-09-10T08:49:19.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.995+0000 7f6e66ffd640 1 -- 172.21.15.172:0/1770701727 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e74044040 con 0x564c17280780 2024-09-10T08:49:19.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.995+0000 7f6e66ffd640 1 -- 172.21.15.172:0/1770701727 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e740305c0 con 0x564c17280780 2024-09-10T08:49:19.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.995+0000 7f6e85d7b740 1 -- 172.21.15.172:0/1770701727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17280780 msgr2=0x564c17298640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:19.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.995+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/1770701727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17280780 0x564c17298640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6e7402ae70 tx=0x7f6e74009560 comp rx=0 tx=0).stop 2024-09-10T08:49:19.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.995+0000 7f6e85d7b740 1 -- 172.21.15.172:0/1770701727 shutdown_connections 2024-09-10T08:49:19.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.995+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/1770701727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17280780 0x564c17298640 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:19.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.995+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/1770701727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c17272b40 0x564c17298100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:19.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.995+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/1770701727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c172715d0 0x564c1728c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:19.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.995+0000 7f6e85d7b740 1 -- 172.21.15.172:0/1770701727 >> 172.21.15.172:0/1770701727 conn(0x564c1723a310 msgr2=0x564c1727d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:19.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.995+0000 7f6e85d7b740 1 -- 172.21.15.172:0/1770701727 shutdown_connections 2024-09-10T08:49:19.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.995+0000 7f6e85d7b740 1 -- 172.21.15.172:0/1770701727 wait complete. 2024-09-10T08:49:19.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.996+0000 7f6e85d7b740 1 -- start start 2024-09-10T08:49:19.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.996+0000 7f6e85d7b740 1 Processor -- start 2024-09-10T08:49:19.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.996+0000 7f6e85d7b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c172715d0 0x564c17284730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:19.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.996+0000 7f6e85d7b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c17272b40 0x564c17284c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:19.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.996+0000 7f6e85d7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17280780 0x564c172851b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:19.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.996+0000 7f6e85d7b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c1add40 con 0x564c172715d0 2024-09-10T08:49:19.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.996+0000 7f6e85d7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c1ad670 con 0x564c17280780 2024-09-10T08:49:19.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.996+0000 7f6e85d7b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c1aa8b0 con 0x564c17272b40 2024-09-10T08:49:19.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.996+0000 7f6e85d7b740 10 client.0 _ll_register_callbacks cb 0x7f6e7402fca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:19.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.997+0000 7f6e8173e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c172715d0 0x564c17284730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:19.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.997+0000 7f6e8173e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c172715d0 0x564c17284730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53678/0 (socket says 172.21.15.172:53678) 2024-09-10T08:49:19.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.997+0000 7f6e8173e640 1 -- 172.21.15.172:0/923259323 learned_addr learned my addr 172.21.15.172:0/923259323 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:19.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.997+0000 7f6e81f3f640 1 --2- 172.21.15.172:0/923259323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17280780 0x564c172851b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:19.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.997+0000 7f6e80f3d640 1 --2- 172.21.15.172:0/923259323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c17272b40 0x564c17284c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:19.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.997+0000 7f6e81f3f640 1 -- 172.21.15.172:0/923259323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c17272b40 msgr2=0x564c17284c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:19.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.997+0000 7f6e81f3f640 1 --2- 172.21.15.172:0/923259323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c17272b40 0x564c17284c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:19.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.997+0000 7f6e81f3f640 1 -- 172.21.15.172:0/923259323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c172715d0 msgr2=0x564c17284730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:19.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.997+0000 7f6e81f3f640 1 --2- 172.21.15.172:0/923259323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c172715d0 0x564c17284730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:19.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.997+0000 7f6e81f3f640 1 -- 172.21.15.172:0/923259323 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c172fa230 con 0x564c17280780 2024-09-10T08:49:19.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.997+0000 7f6e8173e640 1 --2- 172.21.15.172:0/923259323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c172715d0 0x564c17284730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:19.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.997+0000 7f6e81f3f640 1 --2- 172.21.15.172:0/923259323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17280780 0x564c172851b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6e7404bf30 tx=0x7f6e7404bd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8508bea1ee179d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:19.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.997+0000 7f6e64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:49:19.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.997+0000 7f6e64ff9640 1 -- 172.21.15.172:0/923259323 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74066070 con 0x564c17280780 2024-09-10T08:49:19.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.997+0000 7f6e85d7b740 1 -- 172.21.15.172:0/923259323 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e74044040 con 0x564c17280780 2024-09-10T08:49:19.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.997+0000 7f6e64ff9640 1 -- 172.21.15.172:0/923259323 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6e7404b640 con 0x564c17280780 2024-09-10T08:49:19.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.998+0000 7f6e64ff9640 1 -- 172.21.15.172:0/923259323 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6e7404a9a0 con 0x564c17280780 2024-09-10T08:49:19.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.998+0000 7f6e85d7b740 10 client.25724 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:19.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.998+0000 7f6e85d7b740 1 -- 172.21.15.172:0/923259323 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564c172fb4f0 con 0x564c17280780 2024-09-10T08:49:19.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.998+0000 7f6e64ff9640 1 -- 172.21.15.172:0/923259323 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e74044040 con 0x564c17280780 2024-09-10T08:49:19.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.998+0000 7f6e64ff9640 1 -- 172.21.15.172:0/923259323 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f6e74072030 con 0x564c17280780 2024-09-10T08:49:19.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.998+0000 7f6e85d7b740 10 client.25724 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:19.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.998+0000 7f6e85d7b740 10 client.25724 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:49:19.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.998+0000 7f6e85d7b740 20 client.25724 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:19.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.998+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/923259323 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x564c172fb970 0x564c172fdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:19.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.998+0000 7f6e85d7b740 4 client.25724 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:49:19.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.998+0000 7f6e85d7b740 1 -- 172.21.15.172:0/923259323 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c171ded70 con 0x564c172fb970 2024-09-10T08:49:19.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.999+0000 7f6e8173e640 1 --2- 172.21.15.172:0/923259323 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x564c172fb970 0x564c172fdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:19.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.999+0000 7f6e64ff9640 10 client.25724 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:49:19.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.999+0000 7f6e8173e640 1 --2- 172.21.15.172:0/923259323 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x564c172fb970 0x564c172fdd60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed8734f1d930e9bc server_cookie=7af07fcf9aee287a in_seq=0 out_seq=0 2024-09-10T08:49:19.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.999+0000 7f6e64ff9640 1 -- 172.21.15.172:0/923259323 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c171ded70 con 0x564c172fb970 2024-09-10T08:49:19.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.999+0000 7f6e64ff9640 10 client.25724 handle_command_reply: tid=0 2024-09-10T08:49:19.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.999+0000 7f6e85d7b740 1 client.25724 shutdown 2024-09-10T08:49:19.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.999+0000 7f6e85d7b740 1 -- 172.21.15.172:0/923259323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17280780 msgr2=0x564c172851b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:19.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.999+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/923259323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17280780 0x564c172851b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6e7404bf30 tx=0x7f6e7404bd20 comp rx=0 tx=0).stop 2024-09-10T08:49:19.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.999+0000 7f6e85d7b740 1 -- 172.21.15.172:0/923259323 shutdown_connections 2024-09-10T08:49:19.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.999+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/923259323 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x564c172fb970 0x564c172fdd60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:19.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.999+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/923259323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c17280780 0x564c172851b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:19.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.999+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/923259323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c17272b40 0x564c17284c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:19.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.999+0000 7f6e85d7b740 1 --2- 172.21.15.172:0/923259323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c172715d0 0x564c17284730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:19.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.999+0000 7f6e85d7b740 1 -- 172.21.15.172:0/923259323 >> 172.21.15.172:0/923259323 conn(0x564c1723a310 msgr2=0x564c1727ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:19.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:18.999+0000 7f6e64ff9640 0 client.25724 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:49:19.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:19.000+0000 7f6e85d7b740 1 -- 172.21.15.172:0/923259323 shutdown_connections 2024-09-10T08:49:19.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:19.000+0000 7f6e85d7b740 1 -- 172.21.15.172:0/923259323 wait complete. 2024-09-10T08:49:19.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:19.000+0000 7f6e85d7b740 20 client.25724 trim_cache size 0 max 16384 2024-09-10T08:49:19.014 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:49:19.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:19.000+0000 7f6e839a2640 1 -- 172.21.15.172:0/890860068 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e680a2c00 msgr2=0x7f6e680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:19.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:19.000+0000 7f6e839a2640 1 --2- 172.21.15.172:0/890860068 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e680a2c00 0x7f6e680a50c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6e74006010 tx=0x7f6e74003970 comp rx=0 tx=0).stop 2024-09-10T08:49:19.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:19.000+0000 7f6e839a2640 1 -- 172.21.15.172:0/890860068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e7c102570 msgr2=0x7f6e7c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:19.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:19.000+0000 7f6e839a2640 1 --2- 172.21.15.172:0/890860068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e7c102570 0x7f6e7c198910 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00c970 tx=0x7f6e6c00ce30 comp rx=0 tx=0).stop 2024-09-10T08:49:19.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:19.000+0000 7f6e839a2640 1 -- 172.21.15.172:0/890860068 shutdown_connections 2024-09-10T08:49:19.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:19.000+0000 7f6e839a2640 1 --2- 172.21.15.172:0/890860068 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e680a2c00 0x7f6e680a50c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:19.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:19.000+0000 7f6e839a2640 1 --2- 172.21.15.172:0/890860068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e7c102ef0 0x7f6e7c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:19.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:19.000+0000 7f6e839a2640 1 --2- 172.21.15.172:0/890860068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e7c102570 0x7f6e7c198910 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:19.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:19.000+0000 7f6e839a2640 1 --2- 172.21.15.172:0/890860068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e7c100e20 0x7f6e7c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:19.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:19.000+0000 7f6e839a2640 1 -- 172.21.15.172:0/890860068 >> 172.21.15.172:0/890860068 conn(0x7f6e7c0fc7d0 msgr2=0x7f6e7c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:19.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:19.001+0000 7f6e839a2640 1 -- 172.21.15.172:0/890860068 shutdown_connections 2024-09-10T08:49:19.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:19.001+0000 7f6e839a2640 1 -- 172.21.15.172:0/890860068 wait complete. 2024-09-10T08:49:20.562 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:20 smithi191 ceph-mon[35314]: pgmap v443: 129 pgs: 129 active+clean; 934 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-09-10T08:49:20.671 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:20 smithi172 ceph-mon[30854]: pgmap v443: 129 pgs: 129 active+clean; 934 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-09-10T08:49:20.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:20 smithi174 ceph-mon[34068]: pgmap v443: 129 pgs: 129 active+clean; 934 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-09-10T08:49:21.322 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:49:21.612 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:22.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:22 smithi191 ceph-mon[35314]: pgmap v444: 129 pgs: 129 active+clean; 998 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 MiB/s wr, 28 op/s 2024-09-10T08:49:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:22 smithi172 ceph-mon[30854]: pgmap v444: 129 pgs: 129 active+clean; 998 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 MiB/s wr, 28 op/s 2024-09-10T08:49:22.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:22 smithi174 ceph-mon[34068]: pgmap v444: 129 pgs: 129 active+clean; 998 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 31 MiB/s wr, 28 op/s 2024-09-10T08:49:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.840+0000 7ff2412e4640 1 -- 172.21.15.172:0/3746300821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c107820 msgr2=0x7ff23c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.840+0000 7ff2412e4640 1 --2- 172.21.15.172:0/3746300821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c107820 0x7ff23c109c90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff234009ca0 tx=0x7ff23402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.840+0000 7ff2412e4640 1 -- 172.21.15.172:0/3746300821 shutdown_connections 2024-09-10T08:49:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.840+0000 7ff2412e4640 1 --2- 172.21.15.172:0/3746300821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c107820 0x7ff23c109c90 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.840+0000 7ff2412e4640 1 --2- 172.21.15.172:0/3746300821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff23c104ef0 0x7ff23c1072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.840+0000 7ff2412e4640 1 --2- 172.21.15.172:0/3746300821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff23c1025c0 0x7ff23c1049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.840+0000 7ff2412e4640 1 -- 172.21.15.172:0/3746300821 >> 172.21.15.172:0/3746300821 conn(0x7ff23c0fc1e0 msgr2=0x7ff23c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.840+0000 7ff2412e4640 1 -- 172.21.15.172:0/3746300821 shutdown_connections 2024-09-10T08:49:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.840+0000 7ff2412e4640 1 -- 172.21.15.172:0/3746300821 wait complete. 2024-09-10T08:49:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.841+0000 7ff2412e4640 1 Processor -- start 2024-09-10T08:49:22.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.841+0000 7ff2412e4640 1 -- start start 2024-09-10T08:49:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.841+0000 7ff2412e4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff23c1025c0 0x7ff23c197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.841+0000 7ff2412e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff23c104ef0 0x7ff23c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.841+0000 7ff2412e4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c107820 0x7ff23c19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.841+0000 7ff2412e4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c10d690 con 0x7ff23c107820 2024-09-10T08:49:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.841+0000 7ff2412e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c10d510 con 0x7ff23c104ef0 2024-09-10T08:49:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.841+0000 7ff2412e4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c10d810 con 0x7ff23c1025c0 2024-09-10T08:49:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.842+0000 7ff23b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c107820 0x7ff23c19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.842+0000 7ff23b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c107820 0x7ff23c19f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53694/0 (socket says 172.21.15.172:53694) 2024-09-10T08:49:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.842+0000 7ff23b7fe640 1 -- 172.21.15.172:0/2836417363 learned_addr learned my addr 172.21.15.172:0/2836417363 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.842+0000 7ff23a7fc640 1 --2- 172.21.15.172:0/2836417363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff23c104ef0 0x7ff23c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.842+0000 7ff23affd640 1 --2- 172.21.15.172:0/2836417363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff23c1025c0 0x7ff23c197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.842+0000 7ff23b7fe640 1 -- 172.21.15.172:0/2836417363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff23c1025c0 msgr2=0x7ff23c197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.842+0000 7ff23b7fe640 1 --2- 172.21.15.172:0/2836417363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff23c1025c0 0x7ff23c197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.842+0000 7ff23b7fe640 1 -- 172.21.15.172:0/2836417363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff23c104ef0 msgr2=0x7ff23c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.842+0000 7ff23b7fe640 1 --2- 172.21.15.172:0/2836417363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff23c104ef0 0x7ff23c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.842+0000 7ff23b7fe640 1 -- 172.21.15.172:0/2836417363 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c19f950 con 0x7ff23c107820 2024-09-10T08:49:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.842+0000 7ff23b7fe640 1 --2- 172.21.15.172:0/2836417363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c107820 0x7ff23c19f2b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff234031970 tx=0x7ff234003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.842+0000 7ff223fff640 1 -- 172.21.15.172:0/2836417363 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234032ec0 con 0x7ff23c107820 2024-09-10T08:49:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.842+0000 7ff2412e4640 1 -- 172.21.15.172:0/2836417363 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23c19fbe0 con 0x7ff23c107820 2024-09-10T08:49:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.842+0000 7ff223fff640 1 -- 172.21.15.172:0/2836417363 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2340044f0 con 0x7ff23c107820 2024-09-10T08:49:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.842+0000 7ff223fff640 1 -- 172.21.15.172:0/2836417363 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234030d10 con 0x7ff23c107820 2024-09-10T08:49:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.842+0000 7ff2412e4640 1 -- 172.21.15.172:0/2836417363 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23c1a0130 con 0x7ff23c107820 2024-09-10T08:49:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.845+0000 7ff223fff640 1 -- 172.21.15.172:0/2836417363 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff234004040 con 0x7ff23c107820 2024-09-10T08:49:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.845+0000 7ff244084740 1 Processor -- start 2024-09-10T08:49:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff244084740 1 -- start start 2024-09-10T08:49:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff244084740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa64db0 0x5633dfa651b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff244084740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa656f0 0x5633dfa6a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff244084740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa6a780 0x5633dfa6cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff244084740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5633df9c42f0 con 0x5633dfa64db0 2024-09-10T08:49:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff244084740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5633df96c270 con 0x5633dfa656f0 2024-09-10T08:49:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff244084740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5633df9c02b0 con 0x5633dfa6a780 2024-09-10T08:49:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff23a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa64db0 0x5633dfa651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff23a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa64db0 0x5633dfa651b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53710/0 (socket says 172.21.15.172:53710) 2024-09-10T08:49:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff23a7fc640 1 -- 172.21.15.172:0/2428736326 learned_addr learned my addr 172.21.15.172:0/2428736326 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff223fff640 1 --2- 172.21.15.172:0/2836417363 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2240a2dd0 0x7ff2240a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff23a7fc640 1 --2- 172.21.15.172:0/2428736326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa6a780 0x5633dfa6cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff23affd640 1 --2- 172.21.15.172:0/2428736326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa656f0 0x5633dfa6a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff23a7fc640 1 -- 172.21.15.172:0/2428736326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa6a780 msgr2=0x5633dfa6cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff23a7fc640 1 --2- 172.21.15.172:0/2428736326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa6a780 0x5633dfa6cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff23a7fc640 1 -- 172.21.15.172:0/2428736326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa656f0 msgr2=0x5633dfa6a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff23a7fc640 1 --2- 172.21.15.172:0/2428736326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa656f0 0x5633dfa6a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff23a7fc640 1 -- 172.21.15.172:0/2428736326 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633df96b340 con 0x5633dfa64db0 2024-09-10T08:49:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff23affd640 1 --2- 172.21.15.172:0/2428736326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa656f0 0x5633dfa6a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:49:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff223fff640 1 -- 172.21.15.172:0/2836417363 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff2340f09c0 con 0x7ff23c107820 2024-09-10T08:49:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff23b7fe640 1 --2- 172.21.15.172:0/2836417363 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2240a2dd0 0x7ff2240a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.846+0000 7ff23a7fc640 1 --2- 172.21.15.172:0/2428736326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa64db0 0x5633dfa651b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff230012530 tx=0x7ff23000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbdd7a3d2396a2c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.847+0000 7ff2217fa640 1 -- 172.21.15.172:0/2428736326 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230005020 con 0x5633dfa64db0 2024-09-10T08:49:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.847+0000 7ff2217fa640 1 -- 172.21.15.172:0/2428736326 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff230005670 con 0x5633dfa64db0 2024-09-10T08:49:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.847+0000 7ff23b7fe640 1 --2- 172.21.15.172:0/2836417363 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2240a2dd0 0x7ff2240a5290 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff23404a040 tx=0x7ff2340095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.847+0000 7ff2217fa640 1 -- 172.21.15.172:0/2428736326 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2300106c0 con 0x5633dfa64db0 2024-09-10T08:49:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 1 -- 172.21.15.172:0/2428736326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa64db0 msgr2=0x5633dfa651b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 1 --2- 172.21.15.172:0/2428736326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa64db0 0x5633dfa651b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff230012530 tx=0x7ff23000b650 comp rx=0 tx=0).stop 2024-09-10T08:49:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 1 -- 172.21.15.172:0/2428736326 shutdown_connections 2024-09-10T08:49:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 1 --2- 172.21.15.172:0/2428736326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa6a780 0x5633dfa6cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 1 --2- 172.21.15.172:0/2428736326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa656f0 0x5633dfa6a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 1 --2- 172.21.15.172:0/2428736326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa64db0 0x5633dfa651b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 1 -- 172.21.15.172:0/2428736326 >> 172.21.15.172:0/2428736326 conn(0x5633dfa5c010 msgr2=0x5633dfa5c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 1 -- 172.21.15.172:0/2428736326 shutdown_connections 2024-09-10T08:49:22.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 1 -- 172.21.15.172:0/2428736326 wait complete. 2024-09-10T08:49:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 1 -- start start 2024-09-10T08:49:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 1 Processor -- start 2024-09-10T08:49:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa64db0 0x5633dfa6f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa656f0 0x5633dfa6f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa6a780 0x5633dfa765b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5633dfa6d750 con 0x5633dfa6a780 2024-09-10T08:49:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5633df94d290 con 0x5633dfa64db0 2024-09-10T08:49:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5633dfa6d8d0 con 0x5633dfa656f0 2024-09-10T08:49:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff244084740 10 client.0 _ll_register_callbacks cb 0x5633df97e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff23a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa6a780 0x5633dfa765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff23a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa6a780 0x5633dfa765b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53720/0 (socket says 172.21.15.172:53720) 2024-09-10T08:49:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff23a7fc640 1 -- 172.21.15.172:0/3736765746 learned_addr learned my addr 172.21.15.172:0/3736765746 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.848+0000 7ff23affd640 1 --2- 172.21.15.172:0/3736765746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa656f0 0x5633dfa6f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.849+0000 7ff23a7fc640 1 --2- 172.21.15.172:0/3736765746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa64db0 0x5633dfa6f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.849+0000 7ff23a7fc640 1 -- 172.21.15.172:0/3736765746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa656f0 msgr2=0x5633dfa6f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.849+0000 7ff23a7fc640 1 --2- 172.21.15.172:0/3736765746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa656f0 0x5633dfa6f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.849+0000 7ff23a7fc640 1 -- 172.21.15.172:0/3736765746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa64db0 msgr2=0x5633dfa6f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.849+0000 7ff23a7fc640 1 --2- 172.21.15.172:0/3736765746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa64db0 0x5633dfa6f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.849+0000 7ff23a7fc640 1 -- 172.21.15.172:0/3736765746 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633df6d7210 con 0x5633dfa6a780 2024-09-10T08:49:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.849+0000 7ff23a7fc640 1 --2- 172.21.15.172:0/3736765746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa6a780 0x5633dfa765b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff2300123a0 tx=0x7ff230004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81719ee50f296d84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.849+0000 7ff21eff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.849+0000 7ff21eff5640 1 -- 172.21.15.172:0/3736765746 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230027040 con 0x5633dfa6a780 2024-09-10T08:49:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.849+0000 7ff21eff5640 1 -- 172.21.15.172:0/3736765746 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff2300052a0 con 0x5633dfa6a780 2024-09-10T08:49:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.849+0000 7ff244084740 1 -- 172.21.15.172:0/3736765746 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff230005670 con 0x5633dfa6a780 2024-09-10T08:49:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.849+0000 7ff21eff5640 1 -- 172.21.15.172:0/3736765746 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff2300054a0 con 0x5633dfa6a780 2024-09-10T08:49:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.849+0000 7ff244084740 10 client.16386 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.849+0000 7ff244084740 1 -- 172.21.15.172:0/3736765746 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5633df7a3620 con 0x5633dfa6a780 2024-09-10T08:49:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.850+0000 7ff21eff5640 1 -- 172.21.15.172:0/3736765746 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff230005670 con 0x5633dfa6a780 2024-09-10T08:49:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.850+0000 7ff21eff5640 1 -- 172.21.15.172:0/3736765746 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7ff230010460 con 0x5633dfa6a780 2024-09-10T08:49:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.850+0000 7ff244084740 10 client.16386 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.850+0000 7ff244084740 10 client.16386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:49:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.850+0000 7ff244084740 20 client.16386 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.850+0000 7ff244084740 1 --2- 172.21.15.172:0/3736765746 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5633dfa7a950 0x5633dfa7cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.850+0000 7ff244084740 4 client.16386 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:49:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.850+0000 7ff244084740 1 -- 172.21.15.172:0/3736765746 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633df9d1a20 con 0x5633dfa7a950 2024-09-10T08:49:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.850+0000 7ff23affd640 1 --2- 172.21.15.172:0/3736765746 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5633dfa7a950 0x5633dfa7cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.850+0000 7ff21eff5640 10 client.16386 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.850+0000 7ff23affd640 1 --2- 172.21.15.172:0/3736765746 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5633dfa7a950 0x5633dfa7cd40 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57f1712e78b6e3bf server_cookie=3a1009c87da52c50 in_seq=0 out_seq=0 2024-09-10T08:49:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.851+0000 7ff21eff5640 1 -- 172.21.15.172:0/3736765746 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633df9d1a20 con 0x5633dfa7a950 2024-09-10T08:49:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.851+0000 7ff21eff5640 10 client.16386 handle_command_reply: tid=0 2024-09-10T08:49:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.851+0000 7ff244084740 1 client.16386 shutdown 2024-09-10T08:49:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.851+0000 7ff244084740 1 -- 172.21.15.172:0/3736765746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa6a780 msgr2=0x5633dfa765b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.851+0000 7ff244084740 1 --2- 172.21.15.172:0/3736765746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa6a780 0x5633dfa765b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff2300123a0 tx=0x7ff230004a20 comp rx=0 tx=0).stop 2024-09-10T08:49:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.851+0000 7ff244084740 1 -- 172.21.15.172:0/3736765746 shutdown_connections 2024-09-10T08:49:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.851+0000 7ff244084740 1 --2- 172.21.15.172:0/3736765746 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5633dfa7a950 0x5633dfa7cd40 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.851+0000 7ff244084740 1 --2- 172.21.15.172:0/3736765746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa6a780 0x5633dfa765b0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.851+0000 7ff244084740 1 --2- 172.21.15.172:0/3736765746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa656f0 0x5633dfa6f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.851+0000 7ff21eff5640 0 client.16386 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.851+0000 7ff244084740 1 --2- 172.21.15.172:0/3736765746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa64db0 0x5633dfa6f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.851+0000 7ff244084740 1 -- 172.21.15.172:0/3736765746 >> 172.21.15.172:0/3736765746 conn(0x5633dfa5c010 msgr2=0x5633dfa5d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.851+0000 7ff244084740 1 -- 172.21.15.172:0/3736765746 shutdown_connections 2024-09-10T08:49:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.851+0000 7ff244084740 1 -- 172.21.15.172:0/3736765746 wait complete. 2024-09-10T08:49:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.851+0000 7ff244084740 20 client.16386 trim_cache size 0 max 16384 2024-09-10T08:49:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.866+0000 7ff244084740 1 Processor -- start 2024-09-10T08:49:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.866+0000 7ff244084740 1 -- start start 2024-09-10T08:49:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff244084740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa64db0 0x5633dfa5a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff244084740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa656f0 0x5633dfa764d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff244084740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa6a780 0x5633dfa5aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff244084740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c1a83a0 con 0x5633dfa656f0 2024-09-10T08:49:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff244084740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c1a9f40 con 0x5633dfa6a780 2024-09-10T08:49:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff244084740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c1a8a70 con 0x5633dfa64db0 2024-09-10T08:49:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff23affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa656f0 0x5633dfa764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff23affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa656f0 0x5633dfa764d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53722/0 (socket says 172.21.15.172:53722) 2024-09-10T08:49:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff23affd640 1 -- 172.21.15.172:0/4166250915 learned_addr learned my addr 172.21.15.172:0/4166250915 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff23a7fc640 1 --2- 172.21.15.172:0/4166250915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa6a780 0x5633dfa5aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff23affd640 1 -- 172.21.15.172:0/4166250915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa64db0 msgr2=0x5633dfa5a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff23affd640 1 --2- 172.21.15.172:0/4166250915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa64db0 0x5633dfa5a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff23affd640 1 -- 172.21.15.172:0/4166250915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa6a780 msgr2=0x5633dfa5aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff23affd640 1 --2- 172.21.15.172:0/4166250915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa6a780 0x5633dfa5aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff23affd640 1 -- 172.21.15.172:0/4166250915 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633dfa5cca0 con 0x5633dfa656f0 2024-09-10T08:49:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff23affd640 1 --2- 172.21.15.172:0/4166250915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa656f0 0x5633dfa764d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00ab20 tx=0x7ff22c014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a28f51d05461476 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff220ff9640 1 -- 172.21.15.172:0/4166250915 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0140d0 con 0x5633dfa656f0 2024-09-10T08:49:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff220ff9640 1 -- 172.21.15.172:0/4166250915 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff22c014270 con 0x5633dfa656f0 2024-09-10T08:49:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.867+0000 7ff220ff9640 1 -- 172.21.15.172:0/4166250915 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c026710 con 0x5633dfa656f0 2024-09-10T08:49:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.868+0000 7ff244084740 1 -- 172.21.15.172:0/4166250915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa656f0 msgr2=0x5633dfa764d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.868+0000 7ff244084740 1 --2- 172.21.15.172:0/4166250915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa656f0 0x5633dfa764d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00ab20 tx=0x7ff22c014840 comp rx=0 tx=0).stop 2024-09-10T08:49:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.868+0000 7ff244084740 1 -- 172.21.15.172:0/4166250915 shutdown_connections 2024-09-10T08:49:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.868+0000 7ff244084740 1 --2- 172.21.15.172:0/4166250915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa6a780 0x5633dfa5aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.868+0000 7ff244084740 1 --2- 172.21.15.172:0/4166250915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa656f0 0x5633dfa764d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.874 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.868+0000 7ff244084740 1 --2- 172.21.15.172:0/4166250915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa64db0 0x5633dfa5a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.868+0000 7ff244084740 1 -- 172.21.15.172:0/4166250915 >> 172.21.15.172:0/4166250915 conn(0x5633dfa5c010 msgr2=0x5633dfa6f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.868+0000 7ff244084740 1 -- 172.21.15.172:0/4166250915 shutdown_connections 2024-09-10T08:49:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.868+0000 7ff244084740 1 -- 172.21.15.172:0/4166250915 wait complete. 2024-09-10T08:49:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.868+0000 7ff244084740 1 -- start start 2024-09-10T08:49:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.868+0000 7ff244084740 1 Processor -- start 2024-09-10T08:49:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.868+0000 7ff244084740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa64db0 0x5633dfa73790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.868+0000 7ff244084740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa656f0 0x5633dfa73cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.868+0000 7ff244084740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa6a780 0x5633dfa74ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff244084740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c1a9810 con 0x5633dfa6a780 2024-09-10T08:49:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff244084740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c1a9140 con 0x5633dfa64db0 2024-09-10T08:49:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff244084740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c1a6380 con 0x5633dfa656f0 2024-09-10T08:49:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff244084740 10 client.0 _ll_register_callbacks cb 0x7ff230012410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:22.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff23a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa6a780 0x5633dfa74ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff23a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa6a780 0x5633dfa74ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53728/0 (socket says 172.21.15.172:53728) 2024-09-10T08:49:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff23a7fc640 1 -- 172.21.15.172:0/189696581 learned_addr learned my addr 172.21.15.172:0/189696581 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff23a7fc640 1 --2- 172.21.15.172:0/189696581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa64db0 0x5633dfa73790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff23affd640 1 --2- 172.21.15.172:0/189696581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa656f0 0x5633dfa73cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff23a7fc640 1 -- 172.21.15.172:0/189696581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa656f0 msgr2=0x5633dfa73cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff23a7fc640 1 --2- 172.21.15.172:0/189696581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa656f0 0x5633dfa73cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff23a7fc640 1 -- 172.21.15.172:0/189696581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa64db0 msgr2=0x5633dfa73790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff23a7fc640 1 --2- 172.21.15.172:0/189696581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa64db0 0x5633dfa73790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff23a7fc640 1 -- 172.21.15.172:0/189696581 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633dfa752f0 con 0x5633dfa6a780 2024-09-10T08:49:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff23a7fc640 1 --2- 172.21.15.172:0/189696581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa6a780 0x5633dfa74ce0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff23000b930 tx=0x7ff23000b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6890a9d040bb0bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:22.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff21eff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff21eff5640 1 -- 172.21.15.172:0/189696581 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230005280 con 0x5633dfa6a780 2024-09-10T08:49:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff21eff5640 1 -- 172.21.15.172:0/189696581 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff230010a20 con 0x5633dfa6a780 2024-09-10T08:49:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff244084740 1 -- 172.21.15.172:0/189696581 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff22c014270 con 0x5633dfa6a780 2024-09-10T08:49:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff21eff5640 1 -- 172.21.15.172:0/189696581 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff230010c20 con 0x5633dfa6a780 2024-09-10T08:49:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff244084740 10 client.16392 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.869+0000 7ff244084740 1 -- 172.21.15.172:0/189696581 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5633dfa76600 con 0x5633dfa6a780 2024-09-10T08:49:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.870+0000 7ff21eff5640 1 -- 172.21.15.172:0/189696581 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff22c014270 con 0x5633dfa6a780 2024-09-10T08:49:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.870+0000 7ff21eff5640 1 -- 172.21.15.172:0/189696581 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7ff230036030 con 0x5633dfa6a780 2024-09-10T08:49:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.870+0000 7ff244084740 10 client.16392 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.870+0000 7ff244084740 10 client.16392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:49:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.870+0000 7ff244084740 20 client.16392 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.870+0000 7ff244084740 1 --2- 172.21.15.172:0/189696581 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5633dfa7a950 0x5633dfae7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.870+0000 7ff244084740 4 client.16392 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:49:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.870+0000 7ff244084740 1 -- 172.21.15.172:0/189696581 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5633df9cbd70 con 0x5633dfa7a950 2024-09-10T08:49:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.870+0000 7ff23affd640 1 --2- 172.21.15.172:0/189696581 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5633dfa7a950 0x5633dfae7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.870+0000 7ff21eff5640 10 client.16392 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.870+0000 7ff23affd640 1 --2- 172.21.15.172:0/189696581 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5633dfa7a950 0x5633dfae7bc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb1b36cae2eb396 server_cookie=791586778409c97d in_seq=0 out_seq=0 2024-09-10T08:49:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.881+0000 7ff21eff5640 1 -- 172.21.15.172:0/189696581 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5633df9cbd70 con 0x5633dfa7a950 2024-09-10T08:49:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.881+0000 7ff21eff5640 10 client.16392 handle_command_reply: tid=0 2024-09-10T08:49:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.881+0000 7ff244084740 1 client.16392 shutdown 2024-09-10T08:49:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.881+0000 7ff244084740 1 -- 172.21.15.172:0/189696581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa6a780 msgr2=0x5633dfa74ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.881+0000 7ff244084740 1 --2- 172.21.15.172:0/189696581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa6a780 0x5633dfa74ce0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff23000b930 tx=0x7ff23000b520 comp rx=0 tx=0).stop 2024-09-10T08:49:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.881+0000 7ff244084740 1 -- 172.21.15.172:0/189696581 shutdown_connections 2024-09-10T08:49:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.881+0000 7ff244084740 1 --2- 172.21.15.172:0/189696581 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5633dfa7a950 0x5633dfae7bc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.881+0000 7ff244084740 1 --2- 172.21.15.172:0/189696581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633dfa6a780 0x5633dfa74ce0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.881+0000 7ff21eff5640 0 client.16392 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.881+0000 7ff244084740 1 --2- 172.21.15.172:0/189696581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5633dfa656f0 0x5633dfa73cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.881+0000 7ff244084740 1 --2- 172.21.15.172:0/189696581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633dfa64db0 0x5633dfa73790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.881+0000 7ff244084740 1 -- 172.21.15.172:0/189696581 >> 172.21.15.172:0/189696581 conn(0x5633dfa5c010 msgr2=0x5633dfa6eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.881+0000 7ff244084740 1 -- 172.21.15.172:0/189696581 shutdown_connections 2024-09-10T08:49:22.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.881+0000 7ff244084740 1 -- 172.21.15.172:0/189696581 wait complete. 2024-09-10T08:49:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.882+0000 7ff244084740 20 client.16392 trim_cache size 0 max 16384 2024-09-10T08:49:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:49:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:49:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:49:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:49:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.882+0000 7ff2412e4640 1 -- 172.21.15.172:0/2836417363 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2240a2dd0 msgr2=0x7ff2240a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.882+0000 7ff2412e4640 1 --2- 172.21.15.172:0/2836417363 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2240a2dd0 0x7ff2240a5290 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff23404a040 tx=0x7ff2340095c0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.882+0000 7ff2412e4640 1 -- 172.21.15.172:0/2836417363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c107820 msgr2=0x7ff23c19f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.882+0000 7ff2412e4640 1 --2- 172.21.15.172:0/2836417363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c107820 0x7ff23c19f2b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff234031970 tx=0x7ff234003ba0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.882+0000 7ff2412e4640 1 -- 172.21.15.172:0/2836417363 shutdown_connections 2024-09-10T08:49:22.885 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.882+0000 7ff2412e4640 1 --2- 172.21.15.172:0/2836417363 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff2240a2dd0 0x7ff2240a5290 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.882+0000 7ff2412e4640 1 --2- 172.21.15.172:0/2836417363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff23c107820 0x7ff23c19f2b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.882+0000 7ff2412e4640 1 --2- 172.21.15.172:0/2836417363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff23c104ef0 0x7ff23c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.882+0000 7ff2412e4640 1 --2- 172.21.15.172:0/2836417363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff23c1025c0 0x7ff23c197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.882+0000 7ff2412e4640 1 -- 172.21.15.172:0/2836417363 >> 172.21.15.172:0/2836417363 conn(0x7ff23c0fc1e0 msgr2=0x7ff23c102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.882+0000 7ff2412e4640 1 -- 172.21.15.172:0/2836417363 shutdown_connections 2024-09-10T08:49:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:22.882+0000 7ff2412e4640 1 -- 172.21.15.172:0/2836417363 wait complete. 2024-09-10T08:49:24.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:24 smithi172 ceph-mon[30854]: pgmap v445: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-09-10T08:49:24.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:24 smithi174 ceph-mon[34068]: pgmap v445: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-09-10T08:49:24.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:24 smithi191 ceph-mon[35314]: pgmap v445: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-09-10T08:49:25.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.856+0000 7f3c16382640 1 -- 172.21.15.172:0/339403625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c101082a0 msgr2=0x7f3c1010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:25.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.856+0000 7f3c16382640 1 --2- 172.21.15.172:0/339403625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c101082a0 0x7f3c1010ec40 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009cd0 tx=0x7f3c0802a8e0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.856+0000 7f3c16382640 1 -- 172.21.15.172:0/339403625 shutdown_connections 2024-09-10T08:49:25.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.856+0000 7f3c16382640 1 --2- 172.21.15.172:0/339403625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c101082a0 0x7f3c1010ec40 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.856+0000 7f3c16382640 1 --2- 172.21.15.172:0/339403625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c101036e0 0x7f3c10107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.856+0000 7f3c16382640 1 --2- 172.21.15.172:0/339403625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c10102d30 0x7f3c10103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.856+0000 7f3c16382640 1 -- 172.21.15.172:0/339403625 >> 172.21.15.172:0/339403625 conn(0x7f3c100fc7d0 msgr2=0x7f3c100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:25.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.856+0000 7f3c16382640 1 -- 172.21.15.172:0/339403625 shutdown_connections 2024-09-10T08:49:25.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.856+0000 7f3c16382640 1 -- 172.21.15.172:0/339403625 wait complete. 2024-09-10T08:49:25.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.857+0000 7f3c16382640 1 Processor -- start 2024-09-10T08:49:25.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.857+0000 7f3c16382640 1 -- start start 2024-09-10T08:49:25.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.857+0000 7f3c16382640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c10102d30 0x7f3c10100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.857+0000 7f3c16382640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c101036e0 0x7f3c10100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.857+0000 7f3c16382640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c101082a0 0x7f3c10101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.857+0000 7f3c16382640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10111f50 con 0x7f3c101082a0 2024-09-10T08:49:25.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.857+0000 7f3c16382640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10111dd0 con 0x7f3c101036e0 2024-09-10T08:49:25.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.857+0000 7f3c16382640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3c101120d0 con 0x7f3c10102d30 2024-09-10T08:49:25.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.857+0000 7f3c1491f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c101082a0 0x7f3c10101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.857+0000 7f3c1491f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c101082a0 0x7f3c10101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53752/0 (socket says 172.21.15.172:53752) 2024-09-10T08:49:25.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.857+0000 7f3c1491f640 1 -- 172.21.15.172:0/3434724106 learned_addr learned my addr 172.21.15.172:0/3434724106 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:25.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.857+0000 7f3c0ffff640 1 --2- 172.21.15.172:0/3434724106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c10102d30 0x7f3c10100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.858+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/3434724106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c101036e0 0x7f3c10100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.858+0000 7f3c1491f640 1 -- 172.21.15.172:0/3434724106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c10102d30 msgr2=0x7f3c10100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:25.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.858+0000 7f3c1491f640 1 --2- 172.21.15.172:0/3434724106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c10102d30 0x7f3c10100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.858+0000 7f3c1491f640 1 -- 172.21.15.172:0/3434724106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c101036e0 msgr2=0x7f3c10100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:25.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.858+0000 7f3c1491f640 1 --2- 172.21.15.172:0/3434724106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c101036e0 0x7f3c10100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.858+0000 7f3c1491f640 1 -- 172.21.15.172:0/3434724106 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c101a68e0 con 0x7f3c101082a0 2024-09-10T08:49:25.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.858+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/3434724106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c101036e0 0x7f3c10100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:49:25.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.858+0000 7f3c1491f640 1 --2- 172.21.15.172:0/3434724106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c101082a0 0x7f3c10101410 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3c08005cc0 tx=0x7f3c08003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:25.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.858+0000 7f3c0d7fa640 1 -- 172.21.15.172:0/3434724106 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08004450 con 0x7f3c101082a0 2024-09-10T08:49:25.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.858+0000 7f3c16382640 1 -- 172.21.15.172:0/3434724106 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c101a6b70 con 0x7f3c101082a0 2024-09-10T08:49:25.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.858+0000 7f3c0d7fa640 1 -- 172.21.15.172:0/3434724106 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c08031c90 con 0x7f3c101082a0 2024-09-10T08:49:25.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.858+0000 7f3c0d7fa640 1 -- 172.21.15.172:0/3434724106 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08030e70 con 0x7f3c101082a0 2024-09-10T08:49:25.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.858+0000 7f3c16382640 1 -- 172.21.15.172:0/3434724106 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c101a7100 con 0x7f3c101082a0 2024-09-10T08:49:25.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.861+0000 7f3c1875b740 1 Processor -- start 2024-09-10T08:49:25.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.861+0000 7f3c0d7fa640 1 -- 172.21.15.172:0/3434724106 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3c080045f0 con 0x7f3c101082a0 2024-09-10T08:49:25.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.861+0000 7f3c1875b740 1 -- start start 2024-09-10T08:49:25.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.861+0000 7f3c1875b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a42ab40 0x563e5a429090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.861+0000 7f3c1875b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a4295d0 0x563e5a429a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.861+0000 7f3c1875b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a436760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.861+0000 7f3c1875b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563e5a38f2f0 con 0x563e5a434370 2024-09-10T08:49:25.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.861+0000 7f3c1875b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563e5a336cf0 con 0x563e5a42ab40 2024-09-10T08:49:25.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.861+0000 7f3c1875b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563e5a38b2b0 con 0x563e5a4295d0 2024-09-10T08:49:25.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.861+0000 7f3c0f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a436760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.862+0000 7f3c0f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a436760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53768/0 (socket says 172.21.15.172:53768) 2024-09-10T08:49:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.862+0000 7f3c0f7fe640 1 -- 172.21.15.172:0/3717878223 learned_addr learned my addr 172.21.15.172:0/3717878223 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.862+0000 7f3c0ffff640 1 --2- 172.21.15.172:0/3717878223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a4295d0 0x563e5a429a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.862+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/3717878223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a42ab40 0x563e5a429090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.862+0000 7f3c0d7fa640 1 --2- 172.21.15.172:0/3434724106 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3bf80a2b80 0x7f3bf80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.862+0000 7f3c0f7fe640 1 -- 172.21.15.172:0/3717878223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a4295d0 msgr2=0x563e5a429a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.862+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/3717878223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a4295d0 0x563e5a429a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.862+0000 7f3c0f7fe640 1 -- 172.21.15.172:0/3717878223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a42ab40 msgr2=0x563e5a429090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:25.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.862+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/3717878223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a42ab40 0x563e5a429090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.862+0000 7f3c0f7fe640 1 -- 172.21.15.172:0/3717878223 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5a31b3a0 con 0x563e5a434370 2024-09-10T08:49:25.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.862+0000 7f3c0d7fa640 1 -- 172.21.15.172:0/3434724106 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3c0802e030 con 0x7f3c101082a0 2024-09-10T08:49:25.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.862+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/3717878223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a436760 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3c04013950 tx=0x7f3c0400c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1525303e24ad4d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:25.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.862+0000 7f3c1491f640 1 --2- 172.21.15.172:0/3434724106 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3bf80a2b80 0x7f3bf80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.862+0000 7f3bf67fc640 1 -- 172.21.15.172:0/3717878223 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04005220 con 0x563e5a434370 2024-09-10T08:49:25.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.862+0000 7f3bf67fc640 1 -- 172.21.15.172:0/3717878223 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c04004c50 con 0x563e5a434370 2024-09-10T08:49:25.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.862+0000 7f3bf67fc640 1 -- 172.21.15.172:0/3717878223 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04011a40 con 0x563e5a434370 2024-09-10T08:49:25.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.863+0000 7f3c1491f640 1 --2- 172.21.15.172:0/3434724106 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3bf80a2b80 0x7f3bf80a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3c080bccc0 tx=0x7f3c080048e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:25.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.863+0000 7f3c1875b740 1 -- 172.21.15.172:0/3717878223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 msgr2=0x563e5a436760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:25.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.863+0000 7f3c1875b740 1 --2- 172.21.15.172:0/3717878223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a436760 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3c04013950 tx=0x7f3c0400c670 comp rx=0 tx=0).stop 2024-09-10T08:49:25.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.863+0000 7f3c1875b740 1 -- 172.21.15.172:0/3717878223 shutdown_connections 2024-09-10T08:49:25.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.863+0000 7f3c1875b740 1 --2- 172.21.15.172:0/3717878223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a436760 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.863+0000 7f3c1875b740 1 --2- 172.21.15.172:0/3717878223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a4295d0 0x563e5a429a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.863+0000 7f3c1875b740 1 --2- 172.21.15.172:0/3717878223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a42ab40 0x563e5a429090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.863+0000 7f3c1875b740 1 -- 172.21.15.172:0/3717878223 >> 172.21.15.172:0/3717878223 conn(0x563e5a3f2310 msgr2=0x563e5a425c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:25.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.863+0000 7f3c1875b740 1 -- 172.21.15.172:0/3717878223 shutdown_connections 2024-09-10T08:49:25.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.863+0000 7f3c1875b740 1 -- 172.21.15.172:0/3717878223 wait complete. 2024-09-10T08:49:25.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.864+0000 7f3c1875b740 1 -- start start 2024-09-10T08:49:25.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.864+0000 7f3c1875b740 1 Processor -- start 2024-09-10T08:49:25.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.864+0000 7f3c1875b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a4295d0 0x563e5a433d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.864+0000 7f3c1875b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a42ab40 0x563e5a431010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.864+0000 7f3c1875b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a434370 0x563e5a431600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.864+0000 7f3c1875b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563e5a437070 con 0x563e5a4295d0 2024-09-10T08:49:25.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.864+0000 7f3c1875b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563e5a34a060 con 0x563e5a434370 2024-09-10T08:49:25.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.864+0000 7f3c1875b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563e5a4371f0 con 0x563e5a42ab40 2024-09-10T08:49:25.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.864+0000 7f3c1875b740 10 client.0 _ll_register_callbacks cb 0x563e5a09d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:25.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.864+0000 7f3c0f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a4295d0 0x563e5a433d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.864+0000 7f3c0f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a4295d0 0x563e5a433d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53778/0 (socket says 172.21.15.172:53778) 2024-09-10T08:49:25.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.864+0000 7f3c0f7fe640 1 -- 172.21.15.172:0/654975280 learned_addr learned my addr 172.21.15.172:0/654975280 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:25.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.865+0000 7f3c0ffff640 1 --2- 172.21.15.172:0/654975280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a42ab40 0x563e5a431010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.865+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/654975280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a434370 0x563e5a431600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.865+0000 7f3c0f7fe640 1 -- 172.21.15.172:0/654975280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a42ab40 msgr2=0x563e5a431010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:25.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.865+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/654975280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a42ab40 0x563e5a431010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.865+0000 7f3c0f7fe640 1 -- 172.21.15.172:0/654975280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a434370 msgr2=0x563e5a431600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:25.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.865+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/654975280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a434370 0x563e5a431600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.865+0000 7f3c0f7fe640 1 -- 172.21.15.172:0/654975280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5a1003c0 con 0x563e5a4295d0 2024-09-10T08:49:25.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.865+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/654975280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a4295d0 0x563e5a433d60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3c04013920 tx=0x7f3c04005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d739bc48c39406 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:25.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.865+0000 7f3bef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:25.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.865+0000 7f3bef7fe640 1 -- 172.21.15.172:0/654975280 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c040048c0 con 0x563e5a4295d0 2024-09-10T08:49:25.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.865+0000 7f3c1875b740 1 -- 172.21.15.172:0/654975280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c04004c50 con 0x563e5a4295d0 2024-09-10T08:49:25.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.865+0000 7f3bef7fe640 1 -- 172.21.15.172:0/654975280 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3c040117e0 con 0x563e5a4295d0 2024-09-10T08:49:25.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.865+0000 7f3bef7fe640 1 -- 172.21.15.172:0/654975280 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3c04037050 con 0x563e5a4295d0 2024-09-10T08:49:25.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.865+0000 7f3c1875b740 10 client.16404 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:25.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.865+0000 7f3c1875b740 1 -- 172.21.15.172:0/654975280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563e5a442b70 con 0x563e5a4295d0 2024-09-10T08:49:25.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.866+0000 7f3bef7fe640 1 -- 172.21.15.172:0/654975280 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c04004c50 con 0x563e5a4295d0 2024-09-10T08:49:25.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.866+0000 7f3bef7fe640 1 -- 172.21.15.172:0/654975280 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f3c0401f6d0 con 0x563e5a4295d0 2024-09-10T08:49:25.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.866+0000 7f3c1875b740 10 client.16404 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:25.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.866+0000 7f3c1875b740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:49:25.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.866+0000 7f3c1875b740 20 client.16404 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:25.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.866+0000 7f3c1875b740 1 --2- 172.21.15.172:0/654975280 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563e5a4442e0 0x563e5a446700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.866+0000 7f3c1875b740 4 client.16404 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:49:25.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.866+0000 7f3c1875b740 1 -- 172.21.15.172:0/654975280 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e5a39c860 con 0x563e5a4442e0 2024-09-10T08:49:25.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.866+0000 7f3c0ffff640 1 --2- 172.21.15.172:0/654975280 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563e5a4442e0 0x563e5a446700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.867+0000 7f3bef7fe640 10 client.16404 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:25.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.867+0000 7f3c0ffff640 1 --2- 172.21.15.172:0/654975280 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563e5a4442e0 0x563e5a446700 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bdaa69a50764aeb server_cookie=45ebf737f5b86c82 in_seq=0 out_seq=0 2024-09-10T08:49:25.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.867+0000 7f3bef7fe640 1 -- 172.21.15.172:0/654975280 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e5a39c860 con 0x563e5a4442e0 2024-09-10T08:49:25.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.867+0000 7f3bef7fe640 10 client.16404 handle_command_reply: tid=0 2024-09-10T08:49:25.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.867+0000 7f3c1875b740 1 client.16404 shutdown 2024-09-10T08:49:25.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.868+0000 7f3c1875b740 1 -- 172.21.15.172:0/654975280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a4295d0 msgr2=0x563e5a433d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:25.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.868+0000 7f3c1875b740 1 --2- 172.21.15.172:0/654975280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a4295d0 0x563e5a433d60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3c04013920 tx=0x7f3c04005700 comp rx=0 tx=0).stop 2024-09-10T08:49:25.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.868+0000 7f3c1875b740 1 -- 172.21.15.172:0/654975280 shutdown_connections 2024-09-10T08:49:25.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.868+0000 7f3c1875b740 1 --2- 172.21.15.172:0/654975280 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563e5a4442e0 0x563e5a446700 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.868+0000 7f3c1875b740 1 --2- 172.21.15.172:0/654975280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a434370 0x563e5a431600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.868+0000 7f3c1875b740 1 --2- 172.21.15.172:0/654975280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a42ab40 0x563e5a431010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.868+0000 7f3c1875b740 1 --2- 172.21.15.172:0/654975280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a4295d0 0x563e5a433d60 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.868+0000 7f3bef7fe640 0 client.16404 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:25.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.868+0000 7f3c1875b740 1 -- 172.21.15.172:0/654975280 >> 172.21.15.172:0/654975280 conn(0x563e5a3f2310 msgr2=0x563e5a42b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:25.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.868+0000 7f3c1875b740 1 -- 172.21.15.172:0/654975280 shutdown_connections 2024-09-10T08:49:25.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.868+0000 7f3c1875b740 1 -- 172.21.15.172:0/654975280 wait complete. 2024-09-10T08:49:25.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.868+0000 7f3c1875b740 20 client.16404 trim_cache size 0 max 16384 2024-09-10T08:49:25.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.890+0000 7f3c1875b740 1 Processor -- start 2024-09-10T08:49:25.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.890+0000 7f3c1875b740 1 -- start start 2024-09-10T08:49:25.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.890+0000 7f3c1875b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a4295d0 0x563e5a425d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.890+0000 7f3c1875b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a42ab40 0x563e5a440370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.890+0000 7f3c1875b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a44bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.890+0000 7f3c1875b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3c101ac830 con 0x563e5a434370 2024-09-10T08:49:25.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.890+0000 7f3c1875b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c101ae3d0 con 0x563e5a42ab40 2024-09-10T08:49:25.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.890+0000 7f3c1875b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3c101acf00 con 0x563e5a4295d0 2024-09-10T08:49:25.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.890+0000 7f3c0f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a44bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.890+0000 7f3c0f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a44bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53790/0 (socket says 172.21.15.172:53790) 2024-09-10T08:49:25.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.890+0000 7f3c0f7fe640 1 -- 172.21.15.172:0/1971666759 learned_addr learned my addr 172.21.15.172:0/1971666759 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:25.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.890+0000 7f3c0ffff640 1 --2- 172.21.15.172:0/1971666759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a42ab40 0x563e5a440370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.890+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/1971666759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a4295d0 0x563e5a425d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.890+0000 7f3c0f7fe640 1 -- 172.21.15.172:0/1971666759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a4295d0 msgr2=0x563e5a425d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:25.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.890+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/1971666759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a4295d0 0x563e5a425d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.891+0000 7f3c0f7fe640 1 -- 172.21.15.172:0/1971666759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a42ab40 msgr2=0x563e5a440370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:25.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.891+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/1971666759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a42ab40 0x563e5a440370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.891+0000 7f3c0f7fe640 1 -- 172.21.15.172:0/1971666759 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5a433880 con 0x563e5a434370 2024-09-10T08:49:25.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.891+0000 7f3c0ffff640 1 --2- 172.21.15.172:0/1971666759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a42ab40 0x563e5a440370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:25.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.891+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/1971666759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a44bfd0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400c730 tx=0x7f3c04013f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b41c6f9c9d63259 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:25.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.891+0000 7f3bf5ffb640 1 -- 172.21.15.172:0/1971666759 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0400ca80 con 0x563e5a434370 2024-09-10T08:49:25.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.891+0000 7f3bf5ffb640 1 -- 172.21.15.172:0/1971666759 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c04029040 con 0x563e5a434370 2024-09-10T08:49:25.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.891+0000 7f3bf5ffb640 1 -- 172.21.15.172:0/1971666759 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04011a10 con 0x563e5a434370 2024-09-10T08:49:25.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.892+0000 7f3c1875b740 1 -- 172.21.15.172:0/1971666759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 msgr2=0x563e5a44bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:25.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.892+0000 7f3c1875b740 1 --2- 172.21.15.172:0/1971666759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a44bfd0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400c730 tx=0x7f3c04013f90 comp rx=0 tx=0).stop 2024-09-10T08:49:25.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.892+0000 7f3c1875b740 1 -- 172.21.15.172:0/1971666759 shutdown_connections 2024-09-10T08:49:25.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.892+0000 7f3c1875b740 1 --2- 172.21.15.172:0/1971666759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a44bfd0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.892+0000 7f3c1875b740 1 --2- 172.21.15.172:0/1971666759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a42ab40 0x563e5a440370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.892+0000 7f3c1875b740 1 --2- 172.21.15.172:0/1971666759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a4295d0 0x563e5a425d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.892+0000 7f3c1875b740 1 -- 172.21.15.172:0/1971666759 >> 172.21.15.172:0/1971666759 conn(0x563e5a3f2310 msgr2=0x563e5a425660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:25.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.892+0000 7f3c1875b740 1 -- 172.21.15.172:0/1971666759 shutdown_connections 2024-09-10T08:49:25.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.892+0000 7f3c1875b740 1 -- 172.21.15.172:0/1971666759 wait complete. 2024-09-10T08:49:25.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.892+0000 7f3c1875b740 1 -- start start 2024-09-10T08:49:25.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c1875b740 1 Processor -- start 2024-09-10T08:49:25.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c1875b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a4295d0 0x563e5a43c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c1875b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a42ab40 0x563e5a43cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c1875b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a42d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c1875b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3c101adca0 con 0x563e5a434370 2024-09-10T08:49:25.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c1875b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c101ad5d0 con 0x563e5a42ab40 2024-09-10T08:49:25.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c1875b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3c101aa830 con 0x563e5a4295d0 2024-09-10T08:49:25.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c1875b740 10 client.0 _ll_register_callbacks cb 0x7f3c0400cde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:25.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c0f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a42d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c0f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a42d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53798/0 (socket says 172.21.15.172:53798) 2024-09-10T08:49:25.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c0f7fe640 1 -- 172.21.15.172:0/2543095846 learned_addr learned my addr 172.21.15.172:0/2543095846 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:25.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c0ffff640 1 --2- 172.21.15.172:0/2543095846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a42ab40 0x563e5a43cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/2543095846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a4295d0 0x563e5a43c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c0f7fe640 1 -- 172.21.15.172:0/2543095846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a4295d0 msgr2=0x563e5a43c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:25.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/2543095846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a4295d0 0x563e5a43c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c0f7fe640 1 -- 172.21.15.172:0/2543095846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a42ab40 msgr2=0x563e5a43cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:25.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/2543095846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a42ab40 0x563e5a43cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.893+0000 7f3c0f7fe640 1 -- 172.21.15.172:0/2543095846 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5a42e110 con 0x563e5a434370 2024-09-10T08:49:25.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.894+0000 7f3c0f7fe640 1 --2- 172.21.15.172:0/2543095846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a42d8e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3c04035040 tx=0x7f3c04031170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53ad6117e90a33d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:25.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.894+0000 7f3bef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:25.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.894+0000 7f3bef7fe640 1 -- 172.21.15.172:0/2543095846 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0400ca80 con 0x563e5a434370 2024-09-10T08:49:25.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.894+0000 7f3bef7fe640 1 -- 172.21.15.172:0/2543095846 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3c0401f440 con 0x563e5a434370 2024-09-10T08:49:25.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.894+0000 7f3c1875b740 1 -- 172.21.15.172:0/2543095846 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c04029040 con 0x563e5a434370 2024-09-10T08:49:25.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.894+0000 7f3bef7fe640 1 -- 172.21.15.172:0/2543095846 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3c0400cc20 con 0x563e5a434370 2024-09-10T08:49:25.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.894+0000 7f3c1875b740 10 client.16410 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:25.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.894+0000 7f3c1875b740 1 -- 172.21.15.172:0/2543095846 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563e5a4aeb00 con 0x563e5a434370 2024-09-10T08:49:25.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.894+0000 7f3bef7fe640 1 -- 172.21.15.172:0/2543095846 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c04029040 con 0x563e5a434370 2024-09-10T08:49:25.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.894+0000 7f3bef7fe640 1 -- 172.21.15.172:0/2543095846 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f3c04004680 con 0x563e5a434370 2024-09-10T08:49:25.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.894+0000 7f3c1875b740 10 client.16410 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:25.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.894+0000 7f3c1875b740 10 client.16410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:49:25.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.895+0000 7f3c1875b740 20 client.16410 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:25.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.895+0000 7f3c1875b740 1 --2- 172.21.15.172:0/2543095846 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563e5a4af170 0x563e5a4b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:25.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.895+0000 7f3c1875b740 4 client.16410 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:49:25.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.895+0000 7f3c1875b740 1 -- 172.21.15.172:0/2543095846 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e5a396d70 con 0x563e5a4af170 2024-09-10T08:49:25.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.895+0000 7f3c0ffff640 1 --2- 172.21.15.172:0/2543095846 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563e5a4af170 0x563e5a4b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:25.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.895+0000 7f3bef7fe640 10 client.16410 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:25.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.895+0000 7f3c0ffff640 1 --2- 172.21.15.172:0/2543095846 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563e5a4af170 0x563e5a4b1590 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=406b43dd02bcfd8f server_cookie=be6ad7573716d64e in_seq=0 out_seq=0 2024-09-10T08:49:25.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.896+0000 7f3bef7fe640 1 -- 172.21.15.172:0/2543095846 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e5a396d70 con 0x563e5a4af170 2024-09-10T08:49:25.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.896+0000 7f3bef7fe640 10 client.16410 handle_command_reply: tid=0 2024-09-10T08:49:25.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.896+0000 7f3c1875b740 1 client.16410 shutdown 2024-09-10T08:49:25.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.896+0000 7f3c1875b740 1 -- 172.21.15.172:0/2543095846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 msgr2=0x563e5a42d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:25.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.896+0000 7f3c1875b740 1 --2- 172.21.15.172:0/2543095846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a42d8e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3c04035040 tx=0x7f3c04031170 comp rx=0 tx=0).stop 2024-09-10T08:49:25.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.896+0000 7f3c1875b740 1 -- 172.21.15.172:0/2543095846 shutdown_connections 2024-09-10T08:49:25.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.896+0000 7f3c1875b740 1 --2- 172.21.15.172:0/2543095846 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563e5a4af170 0x563e5a4b1590 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.896+0000 7f3c1875b740 1 --2- 172.21.15.172:0/2543095846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5a434370 0x563e5a42d8e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.896+0000 7f3c1875b740 1 --2- 172.21.15.172:0/2543095846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a42ab40 0x563e5a43cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.896+0000 7f3c1875b740 1 --2- 172.21.15.172:0/2543095846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563e5a4295d0 0x563e5a43c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.896+0000 7f3c1875b740 1 -- 172.21.15.172:0/2543095846 >> 172.21.15.172:0/2543095846 conn(0x563e5a3f2310 msgr2=0x563e5a42b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:25.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.896+0000 7f3bef7fe640 0 client.16410 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:25.758 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:49:25.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.896+0000 7f3c1875b740 1 -- 172.21.15.172:0/2543095846 shutdown_connections 2024-09-10T08:49:25.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.896+0000 7f3c1875b740 1 -- 172.21.15.172:0/2543095846 wait complete. 2024-09-10T08:49:25.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.896+0000 7f3c1875b740 20 client.16410 trim_cache size 0 max 16384 2024-09-10T08:49:25.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.897+0000 7f3c16382640 1 -- 172.21.15.172:0/3434724106 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3bf80a2b80 msgr2=0x7f3bf80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:25.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.897+0000 7f3c16382640 1 --2- 172.21.15.172:0/3434724106 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3bf80a2b80 0x7f3bf80a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3c080bccc0 tx=0x7f3c080048e0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.897+0000 7f3c16382640 1 -- 172.21.15.172:0/3434724106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c101082a0 msgr2=0x7f3c10101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:25.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.897+0000 7f3c16382640 1 --2- 172.21.15.172:0/3434724106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c101082a0 0x7f3c10101410 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3c08005cc0 tx=0x7f3c08003c20 comp rx=0 tx=0).stop 2024-09-10T08:49:25.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.897+0000 7f3c16382640 1 -- 172.21.15.172:0/3434724106 shutdown_connections 2024-09-10T08:49:25.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.897+0000 7f3c16382640 1 --2- 172.21.15.172:0/3434724106 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3bf80a2b80 0x7f3bf80a5040 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.897+0000 7f3c16382640 1 --2- 172.21.15.172:0/3434724106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c101082a0 0x7f3c10101410 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.897+0000 7f3c16382640 1 --2- 172.21.15.172:0/3434724106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c101036e0 0x7f3c10100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.897+0000 7f3c16382640 1 --2- 172.21.15.172:0/3434724106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c10102d30 0x7f3c10100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:25.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.897+0000 7f3c16382640 1 -- 172.21.15.172:0/3434724106 >> 172.21.15.172:0/3434724106 conn(0x7f3c100fc7d0 msgr2=0x7f3c10105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:25.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.897+0000 7f3c16382640 1 -- 172.21.15.172:0/3434724106 shutdown_connections 2024-09-10T08:49:25.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:24.897+0000 7f3c16382640 1 -- 172.21.15.172:0/3434724106 wait complete. 2024-09-10T08:49:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:26 smithi172 ceph-mon[30854]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-09-10T08:49:26.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:26 smithi174 ceph-mon[34068]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-09-10T08:49:26.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:26 smithi191 ceph-mon[35314]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-09-10T08:49:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:28 smithi172 ceph-mon[30854]: pgmap v447: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-09-10T08:49:28.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:28 smithi174 ceph-mon[34068]: pgmap v447: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-09-10T08:49:28.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:28 smithi191 ceph-mon[35314]: pgmap v447: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-09-10T08:49:30.573 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:30 smithi172 ceph-mon[30854]: pgmap v448: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 22 op/s 2024-09-10T08:49:30.602 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:30 smithi191 ceph-mon[35314]: pgmap v448: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 22 op/s 2024-09-10T08:49:30.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:30 smithi174 ceph-mon[34068]: pgmap v448: 129 pgs: 129 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 25 MiB/s wr, 22 op/s 2024-09-10T08:49:31.602 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:49:31.602 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:49:31.882 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:32.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.427+0000 7f4abc489640 1 -- 172.21.15.172:0/3557690971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ab4100de0 msgr2=0x7f4ab4104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:32.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.427+0000 7f4abc489640 1 --2- 172.21.15.172:0/3557690971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ab4100de0 0x7f4ab4104e30 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400aa10 tx=0x7f4aa402b720 comp rx=0 tx=0).stop 2024-09-10T08:49:32.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.427+0000 7f4abc489640 1 -- 172.21.15.172:0/3557690971 shutdown_connections 2024-09-10T08:49:32.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.427+0000 7f4abc489640 1 --2- 172.21.15.172:0/3557690971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ab4105560 0x7f4ab410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.427+0000 7f4abc489640 1 --2- 172.21.15.172:0/3557690971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ab4100de0 0x7f4ab4104e30 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.427+0000 7f4abc489640 1 --2- 172.21.15.172:0/3557690971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ab4100430 0x7f4ab4100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.427+0000 7f4abc489640 1 -- 172.21.15.172:0/3557690971 >> 172.21.15.172:0/3557690971 conn(0x7f4ab4077230 msgr2=0x7f4ab40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.427+0000 7f4abc489640 1 -- 172.21.15.172:0/3557690971 shutdown_connections 2024-09-10T08:49:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.427+0000 7f4abc489640 1 -- 172.21.15.172:0/3557690971 wait complete. 2024-09-10T08:49:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.427+0000 7f4abc489640 1 Processor -- start 2024-09-10T08:49:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.427+0000 7f4abc489640 1 -- start start 2024-09-10T08:49:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4abc489640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ab4100430 0x7f4ab419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4abc489640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ab4100de0 0x7f4ab419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4abc489640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ab4105560 0x7f4ab419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4abc489640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab410fc10 con 0x7f4ab4105560 2024-09-10T08:49:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4abc489640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab410fa90 con 0x7f4ab4100430 2024-09-10T08:49:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4abc489640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab410fd90 con 0x7f4ab4100de0 2024-09-10T08:49:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4abaa26640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ab4105560 0x7f4ab419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4aba225640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ab4100430 0x7f4ab419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4abaa26640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ab4105560 0x7f4ab419ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46788/0 (socket says 172.21.15.172:46788) 2024-09-10T08:49:32.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4abaa26640 1 -- 172.21.15.172:0/417426331 learned_addr learned my addr 172.21.15.172:0/417426331 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:32.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4ab9a24640 1 --2- 172.21.15.172:0/417426331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ab4100de0 0x7f4ab419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4abaa26640 1 -- 172.21.15.172:0/417426331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ab4100de0 msgr2=0x7f4ab419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:32.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4abaa26640 1 --2- 172.21.15.172:0/417426331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ab4100de0 0x7f4ab419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4abaa26640 1 -- 172.21.15.172:0/417426331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ab4100430 msgr2=0x7f4ab419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:32.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4abaa26640 1 --2- 172.21.15.172:0/417426331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ab4100430 0x7f4ab419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4abaa26640 1 -- 172.21.15.172:0/417426331 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab419f570 con 0x7f4ab4105560 2024-09-10T08:49:32.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4aba225640 1 --2- 172.21.15.172:0/417426331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ab4100430 0x7f4ab419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:32.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.428+0000 7f4abaa26640 1 --2- 172.21.15.172:0/417426331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ab4105560 0x7f4ab419ee90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00eaa0 tx=0x7f4aac00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:32.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.429+0000 7f4aab7fe640 1 -- 172.21.15.172:0/417426331 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac004d20 con 0x7f4ab4105560 2024-09-10T08:49:32.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.429+0000 7f4aab7fe640 1 -- 172.21.15.172:0/417426331 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4aac005020 con 0x7f4ab4105560 2024-09-10T08:49:32.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.429+0000 7f4abc489640 1 -- 172.21.15.172:0/417426331 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab419f860 con 0x7f4ab4105560 2024-09-10T08:49:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.429+0000 7f4aab7fe640 1 -- 172.21.15.172:0/417426331 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac0116b0 con 0x7f4ab4105560 2024-09-10T08:49:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.429+0000 7f4abc489640 1 -- 172.21.15.172:0/417426331 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab41a7140 con 0x7f4ab4105560 2024-09-10T08:49:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.431+0000 7f4aab7fe640 1 -- 172.21.15.172:0/417426331 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4aac0051d0 con 0x7f4ab4105560 2024-09-10T08:49:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.432+0000 7f4abe862740 1 Processor -- start 2024-09-10T08:49:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.432+0000 7f4abe862740 1 -- start start 2024-09-10T08:49:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.432+0000 7f4aab7fe640 1 --2- 172.21.15.172:0/417426331 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4aa00a2b80 0x7f4aa00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.432+0000 7f4abe862740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d3b60 0x55850c1d3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.432+0000 7f4abe862740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d4480 0x55850c1de800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.432+0000 7f4abe862740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1ded40 0x55850c1e1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.432+0000 7f4aab7fe640 1 -- 172.21.15.172:0/417426331 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4aac0cf560 con 0x7f4ab4105560 2024-09-10T08:49:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.432+0000 7f4abe862740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55850c139280 con 0x55850c1d3b60 2024-09-10T08:49:32.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.432+0000 7f4abe862740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55850c0e0b10 con 0x55850c1d4480 2024-09-10T08:49:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.432+0000 7f4abe862740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55850c135240 con 0x55850c1ded40 2024-09-10T08:49:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.432+0000 7f4ab9a24640 1 --2- 172.21.15.172:0/417426331 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4aa00a2b80 0x7f4aa00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.432+0000 7f4aba225640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d3b60 0x55850c1d3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.432+0000 7f4aba225640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d3b60 0x55850c1d3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46802/0 (socket says 172.21.15.172:46802) 2024-09-10T08:49:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.432+0000 7f4aba225640 1 -- 172.21.15.172:0/3176485474 learned_addr learned my addr 172.21.15.172:0/3176485474 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.433+0000 7f4ab9a24640 1 --2- 172.21.15.172:0/3176485474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d4480 0x55850c1de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.433+0000 7f4abaa26640 1 --2- 172.21.15.172:0/3176485474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1ded40 0x55850c1e1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.433+0000 7f4aba225640 1 -- 172.21.15.172:0/3176485474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1ded40 msgr2=0x55850c1e1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.433+0000 7f4aba225640 1 --2- 172.21.15.172:0/3176485474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1ded40 0x55850c1e1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.433+0000 7f4aba225640 1 -- 172.21.15.172:0/3176485474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d4480 msgr2=0x55850c1de800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.433+0000 7f4aba225640 1 --2- 172.21.15.172:0/3176485474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d4480 0x55850c1de800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.433+0000 7f4aba225640 1 -- 172.21.15.172:0/3176485474 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55850c0ed1f0 con 0x55850c1d3b60 2024-09-10T08:49:32.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.433+0000 7f4ab9a24640 1 --2- 172.21.15.172:0/417426331 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4aa00a2b80 0x7f4aa00a5040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4030550 tx=0x7f4aa4030690 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:32.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.433+0000 7f4aba225640 1 --2- 172.21.15.172:0/3176485474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d3b60 0x55850c1d3f40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0009e70 tx=0x7f4ab000e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee869ca855174903 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:32.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.433+0000 7f4aa8ff9640 1 -- 172.21.15.172:0/3176485474 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0024070 con 0x55850c1d3b60 2024-09-10T08:49:32.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.433+0000 7f4aa8ff9640 1 -- 172.21.15.172:0/3176485474 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ab001f440 con 0x55850c1d3b60 2024-09-10T08:49:32.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.433+0000 7f4aa8ff9640 1 -- 172.21.15.172:0/3176485474 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab001d540 con 0x55850c1d3b60 2024-09-10T08:49:32.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.434+0000 7f4abe862740 1 -- 172.21.15.172:0/3176485474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d3b60 msgr2=0x55850c1d3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:32.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.434+0000 7f4abe862740 1 --2- 172.21.15.172:0/3176485474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d3b60 0x55850c1d3f40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0009e70 tx=0x7f4ab000e5a0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.434+0000 7f4abe862740 1 -- 172.21.15.172:0/3176485474 shutdown_connections 2024-09-10T08:49:32.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.434+0000 7f4abe862740 1 --2- 172.21.15.172:0/3176485474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1ded40 0x55850c1e1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.434+0000 7f4abe862740 1 --2- 172.21.15.172:0/3176485474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d4480 0x55850c1de800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.434+0000 7f4abe862740 1 --2- 172.21.15.172:0/3176485474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d3b60 0x55850c1d3f40 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.434+0000 7f4abe862740 1 -- 172.21.15.172:0/3176485474 >> 172.21.15.172:0/3176485474 conn(0x55850c1d00f0 msgr2=0x55850c1d04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:32.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.434+0000 7f4abe862740 1 -- 172.21.15.172:0/3176485474 shutdown_connections 2024-09-10T08:49:32.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.434+0000 7f4abe862740 1 -- 172.21.15.172:0/3176485474 wait complete. 2024-09-10T08:49:32.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.435+0000 7f4abe862740 1 -- start start 2024-09-10T08:49:32.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.435+0000 7f4abe862740 1 Processor -- start 2024-09-10T08:49:32.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.435+0000 7f4abe862740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d3b60 0x55850c1dbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.435+0000 7f4abe862740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1d4480 0x55850c1dc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.435+0000 7f4abe862740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1ded40 0x55850c1dc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.435+0000 7f4abe862740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55850c1e1910 con 0x55850c1ded40 2024-09-10T08:49:32.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.435+0000 7f4abe862740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55850c0f3f10 con 0x55850c1d3b60 2024-09-10T08:49:32.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.435+0000 7f4abe862740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55850c1e1a90 con 0x55850c1d4480 2024-09-10T08:49:32.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.435+0000 7f4abe862740 10 client.0 _ll_register_callbacks cb 0x55850c0bca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:32.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.435+0000 7f4abaa26640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1ded40 0x55850c1dc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.435+0000 7f4abaa26640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1ded40 0x55850c1dc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46810/0 (socket says 172.21.15.172:46810) 2024-09-10T08:49:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.435+0000 7f4abaa26640 1 -- 172.21.15.172:0/1589944470 learned_addr learned my addr 172.21.15.172:0/1589944470 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.435+0000 7f4ab9a24640 1 --2- 172.21.15.172:0/1589944470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1d4480 0x55850c1dc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.435+0000 7f4aba225640 1 --2- 172.21.15.172:0/1589944470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d3b60 0x55850c1dbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.436+0000 7f4ab9a24640 1 -- 172.21.15.172:0/1589944470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d3b60 msgr2=0x55850c1dbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.436+0000 7f4ab9a24640 1 --2- 172.21.15.172:0/1589944470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d3b60 0x55850c1dbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.436+0000 7f4ab9a24640 1 -- 172.21.15.172:0/1589944470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1ded40 msgr2=0x55850c1dc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.436+0000 7f4ab9a24640 1 --2- 172.21.15.172:0/1589944470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1ded40 0x55850c1dc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.436+0000 7f4ab9a24640 1 -- 172.21.15.172:0/1589944470 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55850beaa6a0 con 0x55850c1d4480 2024-09-10T08:49:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.436+0000 7f4abaa26640 1 --2- 172.21.15.172:0/1589944470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1ded40 0x55850c1dc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.436+0000 7f4ab9a24640 1 --2- 172.21.15.172:0/1589944470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1d4480 0x55850c1dc1b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400a340 tx=0x7f4aa402b720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6f19adae4174b17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.436+0000 7f4a9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:49:32.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.436+0000 7f4a9dffb640 1 -- 172.21.15.172:0/1589944470 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4050060 con 0x55850c1d4480 2024-09-10T08:49:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.436+0000 7f4abe862740 1 -- 172.21.15.172:0/1589944470 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ab001f440 con 0x55850c1d4480 2024-09-10T08:49:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.436+0000 7f4a9dffb640 1 -- 172.21.15.172:0/1589944470 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4aa4004950 con 0x55850c1d4480 2024-09-10T08:49:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.436+0000 7f4a9dffb640 1 -- 172.21.15.172:0/1589944470 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4aa4057050 con 0x55850c1d4480 2024-09-10T08:49:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.436+0000 7f4abe862740 10 client.25690 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.436+0000 7f4abe862740 1 -- 172.21.15.172:0/1589944470 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55850c1ed550 con 0x55850c1d4480 2024-09-10T08:49:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.436+0000 7f4a9dffb640 1 -- 172.21.15.172:0/1589944470 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ab001f440 con 0x55850c1d4480 2024-09-10T08:49:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.437+0000 7f4a9dffb640 1 -- 172.21.15.172:0/1589944470 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f4aa403b040 con 0x55850c1d4480 2024-09-10T08:49:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.437+0000 7f4abe862740 10 client.25690 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.437+0000 7f4abe862740 10 client.25690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:49:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.437+0000 7f4abe862740 20 client.25690 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.437+0000 7f4abe862740 1 --2- 172.21.15.172:0/1589944470 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55850c1eea60 0x55850c1f0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.437+0000 7f4abe862740 4 client.25690 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:49:32.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.437+0000 7f4abe862740 1 -- 172.21.15.172:0/1589944470 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55850c1467f0 con 0x55850c1eea60 2024-09-10T08:49:32.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.437+0000 7f4aba225640 1 --2- 172.21.15.172:0/1589944470 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55850c1eea60 0x55850c1f0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.437+0000 7f4a9dffb640 10 client.25690 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:32.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.437+0000 7f4aba225640 1 --2- 172.21.15.172:0/1589944470 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55850c1eea60 0x55850c1f0e80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62e4cc34aaa3925c server_cookie=438be5f51d1e23b4 in_seq=0 out_seq=0 2024-09-10T08:49:32.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.438+0000 7f4a9dffb640 1 -- 172.21.15.172:0/1589944470 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55850c1467f0 con 0x55850c1eea60 2024-09-10T08:49:32.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.438+0000 7f4a9dffb640 10 client.25690 handle_command_reply: tid=0 2024-09-10T08:49:32.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.438+0000 7f4abe862740 1 client.25690 shutdown 2024-09-10T08:49:32.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.438+0000 7f4abe862740 1 -- 172.21.15.172:0/1589944470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1d4480 msgr2=0x55850c1dc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:32.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.438+0000 7f4abe862740 1 --2- 172.21.15.172:0/1589944470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1d4480 0x55850c1dc1b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400a340 tx=0x7f4aa402b720 comp rx=0 tx=0).stop 2024-09-10T08:49:32.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.438+0000 7f4abe862740 1 -- 172.21.15.172:0/1589944470 shutdown_connections 2024-09-10T08:49:32.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.438+0000 7f4abe862740 1 --2- 172.21.15.172:0/1589944470 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55850c1eea60 0x55850c1f0e80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.438+0000 7f4abe862740 1 --2- 172.21.15.172:0/1589944470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1ded40 0x55850c1dc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.438+0000 7f4abe862740 1 --2- 172.21.15.172:0/1589944470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1d4480 0x55850c1dc1b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.438+0000 7f4a9dffb640 0 client.25690 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:32.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.438+0000 7f4abe862740 1 --2- 172.21.15.172:0/1589944470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d3b60 0x55850c1dbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.438+0000 7f4abe862740 1 -- 172.21.15.172:0/1589944470 >> 172.21.15.172:0/1589944470 conn(0x55850c1d00f0 msgr2=0x55850c1cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:32.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.438+0000 7f4abe862740 1 -- 172.21.15.172:0/1589944470 shutdown_connections 2024-09-10T08:49:32.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.438+0000 7f4abe862740 1 -- 172.21.15.172:0/1589944470 wait complete. 2024-09-10T08:49:32.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.438+0000 7f4abe862740 20 client.25690 trim_cache size 0 max 16384 2024-09-10T08:49:32.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.457+0000 7f4abe862740 1 Processor -- start 2024-09-10T08:49:32.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.457+0000 7f4abe862740 1 -- start start 2024-09-10T08:49:32.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.457+0000 7f4abe862740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1d00f0 0x55850c1ddad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.457+0000 7f4abe862740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d3b60 0x55850c1f6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.457+0000 7f4abe862740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d4480 0x55850c1f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.457+0000 7f4abe862740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41ac950 con 0x55850c1d4480 2024-09-10T08:49:32.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.457+0000 7f4abe862740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41ae4f0 con 0x55850c1d3b60 2024-09-10T08:49:32.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.458+0000 7f4abe862740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41ad020 con 0x55850c1d00f0 2024-09-10T08:49:32.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.458+0000 7f4abaa26640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d4480 0x55850c1f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.458+0000 7f4abaa26640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d4480 0x55850c1f6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46820/0 (socket says 172.21.15.172:46820) 2024-09-10T08:49:32.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.458+0000 7f4abaa26640 1 -- 172.21.15.172:0/2657133247 learned_addr learned my addr 172.21.15.172:0/2657133247 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:32.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.458+0000 7f4aba225640 1 --2- 172.21.15.172:0/2657133247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1d00f0 0x55850c1ddad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.458+0000 7f4ab9a24640 1 --2- 172.21.15.172:0/2657133247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d3b60 0x55850c1f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.458+0000 7f4abaa26640 1 -- 172.21.15.172:0/2657133247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1d00f0 msgr2=0x55850c1ddad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:32.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.458+0000 7f4abaa26640 1 --2- 172.21.15.172:0/2657133247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1d00f0 0x55850c1ddad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.458+0000 7f4abaa26640 1 -- 172.21.15.172:0/2657133247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d3b60 msgr2=0x55850c1f6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:32.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.458+0000 7f4abaa26640 1 --2- 172.21.15.172:0/2657133247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d3b60 0x55850c1f6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.458+0000 7f4abaa26640 1 -- 172.21.15.172:0/2657133247 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55850c1cfb90 con 0x55850c1d4480 2024-09-10T08:49:32.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.458+0000 7f4ab9a24640 1 --2- 172.21.15.172:0/2657133247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d3b60 0x55850c1f6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:49:32.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.458+0000 7f4abaa26640 1 --2- 172.21.15.172:0/2657133247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d4480 0x55850c1f6c90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4aac098430 tx=0x7f4aac009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1abb5dcb9cfd62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:32.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.458+0000 7f4a9ffff640 1 -- 172.21.15.172:0/2657133247 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac0b3070 con 0x55850c1d4480 2024-09-10T08:49:32.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.458+0000 7f4a9ffff640 1 -- 172.21.15.172:0/2657133247 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4aac09fe10 con 0x55850c1d4480 2024-09-10T08:49:32.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.458+0000 7f4a9ffff640 1 -- 172.21.15.172:0/2657133247 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac01f040 con 0x55850c1d4480 2024-09-10T08:49:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.459+0000 7f4abe862740 1 -- 172.21.15.172:0/2657133247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d4480 msgr2=0x55850c1f6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.459+0000 7f4abe862740 1 --2- 172.21.15.172:0/2657133247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d4480 0x55850c1f6c90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4aac098430 tx=0x7f4aac009730 comp rx=0 tx=0).stop 2024-09-10T08:49:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.459+0000 7f4abe862740 1 -- 172.21.15.172:0/2657133247 shutdown_connections 2024-09-10T08:49:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.459+0000 7f4abe862740 1 --2- 172.21.15.172:0/2657133247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d4480 0x55850c1f6c90 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.459+0000 7f4abe862740 1 --2- 172.21.15.172:0/2657133247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d3b60 0x55850c1f6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.459+0000 7f4abe862740 1 --2- 172.21.15.172:0/2657133247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1d00f0 0x55850c1ddad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.459+0000 7f4abe862740 1 -- 172.21.15.172:0/2657133247 >> 172.21.15.172:0/2657133247 conn(0x55850c1d7660 msgr2=0x55850c1dd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:32.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.459+0000 7f4abe862740 1 -- 172.21.15.172:0/2657133247 shutdown_connections 2024-09-10T08:49:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.459+0000 7f4abe862740 1 -- 172.21.15.172:0/2657133247 wait complete. 2024-09-10T08:49:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.460+0000 7f4abe862740 1 -- start start 2024-09-10T08:49:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.460+0000 7f4abe862740 1 Processor -- start 2024-09-10T08:49:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.460+0000 7f4abe862740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d00f0 0x55850c1e3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.460+0000 7f4abe862740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d3b60 0x55850c1e3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.460+0000 7f4abe862740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1d4480 0x55850c258720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.460+0000 7f4abe862740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41addc0 con 0x55850c1d00f0 2024-09-10T08:49:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.460+0000 7f4abe862740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41ad6f0 con 0x55850c1d3b60 2024-09-10T08:49:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.460+0000 7f4abe862740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41aa930 con 0x55850c1d4480 2024-09-10T08:49:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.460+0000 7f4abe862740 10 client.0 _ll_register_callbacks cb 0x7f4aa4030fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.460+0000 7f4aba225640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d00f0 0x55850c1e3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.460+0000 7f4aba225640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d00f0 0x55850c1e3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46834/0 (socket says 172.21.15.172:46834) 2024-09-10T08:49:32.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.460+0000 7f4aba225640 1 -- 172.21.15.172:0/3235300207 learned_addr learned my addr 172.21.15.172:0/3235300207 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:32.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.461+0000 7f4ab9a24640 1 --2- 172.21.15.172:0/3235300207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d3b60 0x55850c1e3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.461+0000 7f4abaa26640 1 --2- 172.21.15.172:0/3235300207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1d4480 0x55850c258720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.461+0000 7f4aba225640 1 -- 172.21.15.172:0/3235300207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1d4480 msgr2=0x55850c258720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:32.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.461+0000 7f4aba225640 1 --2- 172.21.15.172:0/3235300207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1d4480 0x55850c258720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.461+0000 7f4aba225640 1 -- 172.21.15.172:0/3235300207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d3b60 msgr2=0x55850c1e3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:32.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.461+0000 7f4aba225640 1 --2- 172.21.15.172:0/3235300207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d3b60 0x55850c1e3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.461+0000 7f4aba225640 1 -- 172.21.15.172:0/3235300207 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55850c258ce0 con 0x55850c1d00f0 2024-09-10T08:49:32.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.461+0000 7f4abaa26640 1 --2- 172.21.15.172:0/3235300207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1d4480 0x55850c258720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:49:32.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.461+0000 7f4aba225640 1 --2- 172.21.15.172:0/3235300207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d00f0 0x55850c1e3210 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0008010 tx=0x7f4ab0009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c227764252cfde server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:32.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.461+0000 7f4a9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:32.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.461+0000 7f4a9dffb640 1 -- 172.21.15.172:0/3235300207 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0039060 con 0x55850c1d00f0 2024-09-10T08:49:32.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.461+0000 7f4a9dffb640 1 -- 172.21.15.172:0/3235300207 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4ab0017c50 con 0x55850c1d00f0 2024-09-10T08:49:32.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.461+0000 7f4abe862740 1 -- 172.21.15.172:0/3235300207 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aac09fe10 con 0x55850c1d00f0 2024-09-10T08:49:32.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.461+0000 7f4a9dffb640 1 -- 172.21.15.172:0/3235300207 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4ab0040050 con 0x55850c1d00f0 2024-09-10T08:49:32.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.461+0000 7f4abe862740 10 client.16428 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:32.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.461+0000 7f4abe862740 1 -- 172.21.15.172:0/3235300207 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55850c25a040 con 0x55850c1d00f0 2024-09-10T08:49:32.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.462+0000 7f4a9dffb640 1 -- 172.21.15.172:0/3235300207 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4aac09fe10 con 0x55850c1d00f0 2024-09-10T08:49:32.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.462+0000 7f4a9dffb640 1 -- 172.21.15.172:0/3235300207 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f4ab0033070 con 0x55850c1d00f0 2024-09-10T08:49:32.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.462+0000 7f4abe862740 10 client.16428 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:32.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.462+0000 7f4abe862740 10 client.16428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:49:32.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.462+0000 7f4abe862740 20 client.16428 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:32.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.462+0000 7f4abe862740 1 --2- 172.21.15.172:0/3235300207 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55850c1ded40 0x55850c25c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:32.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.462+0000 7f4abe862740 4 client.16428 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:49:32.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.462+0000 7f4abe862740 1 -- 172.21.15.172:0/3235300207 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55850c140d00 con 0x55850c1ded40 2024-09-10T08:49:32.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.462+0000 7f4ab9a24640 1 --2- 172.21.15.172:0/3235300207 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55850c1ded40 0x55850c25c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:32.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.463+0000 7f4a9dffb640 10 client.16428 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:32.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.463+0000 7f4ab9a24640 1 --2- 172.21.15.172:0/3235300207 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55850c1ded40 0x55850c25c4f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba0c0797ef5fc5f7 server_cookie=52b1e6fb70a0d29f in_seq=0 out_seq=0 2024-09-10T08:49:32.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.463+0000 7f4a9dffb640 1 -- 172.21.15.172:0/3235300207 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55850c140d00 con 0x55850c1ded40 2024-09-10T08:49:32.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.463+0000 7f4a9dffb640 10 client.16428 handle_command_reply: tid=0 2024-09-10T08:49:32.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.463+0000 7f4abe862740 1 client.16428 shutdown 2024-09-10T08:49:32.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.463+0000 7f4abe862740 1 -- 172.21.15.172:0/3235300207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d00f0 msgr2=0x55850c1e3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:32.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.463+0000 7f4abe862740 1 --2- 172.21.15.172:0/3235300207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d00f0 0x55850c1e3210 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0008010 tx=0x7f4ab0009890 comp rx=0 tx=0).stop 2024-09-10T08:49:32.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.464+0000 7f4abe862740 1 -- 172.21.15.172:0/3235300207 shutdown_connections 2024-09-10T08:49:32.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.464+0000 7f4abe862740 1 --2- 172.21.15.172:0/3235300207 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55850c1ded40 0x55850c25c4f0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.464+0000 7f4abe862740 1 --2- 172.21.15.172:0/3235300207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55850c1d4480 0x55850c258720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.464+0000 7f4abe862740 1 --2- 172.21.15.172:0/3235300207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55850c1d3b60 0x55850c1e3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.464+0000 7f4a9dffb640 0 client.16428 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:32.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.464+0000 7f4abe862740 1 --2- 172.21.15.172:0/3235300207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55850c1d00f0 0x55850c1e3210 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.464+0000 7f4abe862740 1 -- 172.21.15.172:0/3235300207 >> 172.21.15.172:0/3235300207 conn(0x55850c1d7660 msgr2=0x55850c1dc450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:32.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.464+0000 7f4abe862740 1 -- 172.21.15.172:0/3235300207 shutdown_connections 2024-09-10T08:49:32.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.464+0000 7f4abe862740 1 -- 172.21.15.172:0/3235300207 wait complete. 2024-09-10T08:49:32.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.464+0000 7f4abe862740 20 client.16428 trim_cache size 0 max 16384 2024-09-10T08:49:32.469 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:49:32.469 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T08:49:32.470 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "942597b4-6365-4a2d-a3f8-f093da8586e6", 2024-09-10T08:49:32.470 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T08:49:32.470 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:49:32.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.464+0000 7f4abc489640 1 -- 172.21.15.172:0/417426331 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4aa00a2b80 msgr2=0x7f4aa00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:32.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.464+0000 7f4abc489640 1 --2- 172.21.15.172:0/417426331 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4aa00a2b80 0x7f4aa00a5040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4030550 tx=0x7f4aa4030690 comp rx=0 tx=0).stop 2024-09-10T08:49:32.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.464+0000 7f4abc489640 1 -- 172.21.15.172:0/417426331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ab4105560 msgr2=0x7f4ab419ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:32.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.464+0000 7f4abc489640 1 --2- 172.21.15.172:0/417426331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ab4105560 0x7f4ab419ee90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00eaa0 tx=0x7f4aac00eea0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.465+0000 7f4abc489640 1 -- 172.21.15.172:0/417426331 shutdown_connections 2024-09-10T08:49:32.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.465+0000 7f4abc489640 1 --2- 172.21.15.172:0/417426331 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4aa00a2b80 0x7f4aa00a5040 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.465+0000 7f4abc489640 1 --2- 172.21.15.172:0/417426331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ab4105560 0x7f4ab419ee90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.465+0000 7f4abc489640 1 --2- 172.21.15.172:0/417426331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ab4100de0 0x7f4ab419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.465+0000 7f4abc489640 1 --2- 172.21.15.172:0/417426331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ab4100430 0x7f4ab419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:32.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.465+0000 7f4abc489640 1 -- 172.21.15.172:0/417426331 >> 172.21.15.172:0/417426331 conn(0x7f4ab4077230 msgr2=0x7f4ab4103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:32.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.465+0000 7f4abc489640 1 -- 172.21.15.172:0/417426331 shutdown_connections 2024-09-10T08:49:32.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:32.465+0000 7f4abc489640 1 -- 172.21.15.172:0/417426331 wait complete. 2024-09-10T08:49:32.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:32 smithi172 ceph-mon[30854]: pgmap v449: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-09-10T08:49:32.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:32 smithi174 ceph-mon[34068]: pgmap v449: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-09-10T08:49:32.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:32 smithi191 ceph-mon[35314]: pgmap v449: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-09-10T08:49:33.439 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:49:33.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:33 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:49:33.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:33 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:49:33.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:33 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:33.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:33 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:49:33.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:33 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:33.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:33 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:33.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:33 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:49:33.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:33 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:33.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:33 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:49:33.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:33 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:33.724 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:49:33.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:33 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:49:33.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:33 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:49:33.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:33 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:33.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:33 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:49:33.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:33 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:33.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:33 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:33.744 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:33 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:49:33.744 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:33 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:33.744 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:33 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:49:33.744 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:33 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:33.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:33 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:49:33.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:33 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:49:33.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:33 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:33.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:33 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:49:33.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:33 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:33.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:33 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:33.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:33 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:49:33.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:33 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:33.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:33 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:49:33.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:33 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:34.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.238+0000 7ff771db9640 1 -- 172.21.15.172:0/3119369088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76c101de0 msgr2=0x7ff76c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:34.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.238+0000 7ff771db9640 1 --2- 172.21.15.172:0/3119369088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76c101de0 0x7ff76c1021c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009900 tx=0x7ff75c02a410 comp rx=0 tx=0).stop 2024-09-10T08:49:34.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.238+0000 7ff771db9640 1 -- 172.21.15.172:0/3119369088 shutdown_connections 2024-09-10T08:49:34.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.238+0000 7ff771db9640 1 --2- 172.21.15.172:0/3119369088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c106f10 0x7ff76c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.238+0000 7ff771db9640 1 --2- 172.21.15.172:0/3119369088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c102790 0x7ff76c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.238+0000 7ff771db9640 1 --2- 172.21.15.172:0/3119369088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76c101de0 0x7ff76c1021c0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.238+0000 7ff771db9640 1 -- 172.21.15.172:0/3119369088 >> 172.21.15.172:0/3119369088 conn(0x7ff76c0fd8e0 msgr2=0x7ff76c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:34.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.238+0000 7ff771db9640 1 -- 172.21.15.172:0/3119369088 shutdown_connections 2024-09-10T08:49:34.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.238+0000 7ff771db9640 1 -- 172.21.15.172:0/3119369088 wait complete. 2024-09-10T08:49:34.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.239+0000 7ff771db9640 1 Processor -- start 2024-09-10T08:49:34.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.239+0000 7ff771db9640 1 -- start start 2024-09-10T08:49:34.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.239+0000 7ff771db9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c101de0 0x7ff76c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.239+0000 7ff771db9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c102790 0x7ff76c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.239+0000 7ff771db9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76c106f10 0x7ff76c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.239+0000 7ff771db9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c111e20 con 0x7ff76c106f10 2024-09-10T08:49:34.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.239+0000 7ff771db9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c111ca0 con 0x7ff76c102790 2024-09-10T08:49:34.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.239+0000 7ff771db9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c111fa0 con 0x7ff76c101de0 2024-09-10T08:49:34.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.239+0000 7ff76bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76c106f10 0x7ff76c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.239+0000 7ff76bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76c106f10 0x7ff76c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46860/0 (socket says 172.21.15.172:46860) 2024-09-10T08:49:34.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.239+0000 7ff76bfff640 1 -- 172.21.15.172:0/4046666798 learned_addr learned my addr 172.21.15.172:0/4046666798 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:34.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.239+0000 7ff76affd640 1 --2- 172.21.15.172:0/4046666798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c102790 0x7ff76c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.240+0000 7ff76b7fe640 1 --2- 172.21.15.172:0/4046666798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c101de0 0x7ff76c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.240+0000 7ff76bfff640 1 -- 172.21.15.172:0/4046666798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c101de0 msgr2=0x7ff76c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:34.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.240+0000 7ff76bfff640 1 --2- 172.21.15.172:0/4046666798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c101de0 0x7ff76c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.240+0000 7ff76bfff640 1 -- 172.21.15.172:0/4046666798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c102790 msgr2=0x7ff76c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:34.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.240+0000 7ff76bfff640 1 --2- 172.21.15.172:0/4046666798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c102790 0x7ff76c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.240+0000 7ff76bfff640 1 -- 172.21.15.172:0/4046666798 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76c19d310 con 0x7ff76c106f10 2024-09-10T08:49:34.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.240+0000 7ff76affd640 1 --2- 172.21.15.172:0/4046666798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c102790 0x7ff76c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:34.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.240+0000 7ff76bfff640 1 --2- 172.21.15.172:0/4046666798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76c106f10 0x7ff76c19cc30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff76400eaa0 tx=0x7ff76400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:34.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.240+0000 7ff768ff9640 1 -- 172.21.15.172:0/4046666798 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76400cdb0 con 0x7ff76c106f10 2024-09-10T08:49:34.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.240+0000 7ff771db9640 1 -- 172.21.15.172:0/4046666798 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76c19d600 con 0x7ff76c106f10 2024-09-10T08:49:34.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.240+0000 7ff768ff9640 1 -- 172.21.15.172:0/4046666798 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff764005020 con 0x7ff76c106f10 2024-09-10T08:49:34.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.240+0000 7ff768ff9640 1 -- 172.21.15.172:0/4046666798 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7640116d0 con 0x7ff76c106f10 2024-09-10T08:49:34.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.240+0000 7ff771db9640 1 -- 172.21.15.172:0/4046666798 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff76c1a4ee0 con 0x7ff76c106f10 2024-09-10T08:49:34.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.243+0000 7ff768ff9640 1 -- 172.21.15.172:0/4046666798 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff7640051d0 con 0x7ff76c106f10 2024-09-10T08:49:34.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.244+0000 7ff774192740 1 Processor -- start 2024-09-10T08:49:34.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.244+0000 7ff774192740 1 -- start start 2024-09-10T08:49:34.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.244+0000 7ff774192740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606eff340 0x55a606eff720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.244+0000 7ff774192740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606effc60 0x55a606f0e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.244+0000 7ff774192740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606f0ea90 0x55a606f10e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.244+0000 7ff774192740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a606e652e0 con 0x55a606effc60 2024-09-10T08:49:34.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.244+0000 7ff774192740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a606e0cce0 con 0x55a606f0ea90 2024-09-10T08:49:34.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.244+0000 7ff774192740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a606e612a0 con 0x55a606eff340 2024-09-10T08:49:34.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.244+0000 7ff768ff9640 1 --2- 172.21.15.172:0/4046666798 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7540a2b80 0x7ff7540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.244+0000 7ff76b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606effc60 0x55a606f0e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.244+0000 7ff76b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606effc60 0x55a606f0e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46872/0 (socket says 172.21.15.172:46872) 2024-09-10T08:49:34.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.244+0000 7ff76b7fe640 1 -- 172.21.15.172:0/1414093667 learned_addr learned my addr 172.21.15.172:0/1414093667 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:34.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.244+0000 7ff76affd640 1 --2- 172.21.15.172:0/1414093667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606eff340 0x55a606eff720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.244+0000 7ff768ff9640 1 -- 172.21.15.172:0/4046666798 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff7640cf4a0 con 0x7ff76c106f10 2024-09-10T08:49:34.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.244+0000 7ff76affd640 1 --2- 172.21.15.172:0/1414093667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606f0ea90 0x55a606f10e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.244+0000 7ff76bfff640 1 --2- 172.21.15.172:0/4046666798 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7540a2b80 0x7ff7540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.245+0000 7ff76b7fe640 1 -- 172.21.15.172:0/1414093667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606eff340 msgr2=0x55a606eff720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:34.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.245+0000 7ff76b7fe640 1 --2- 172.21.15.172:0/1414093667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606eff340 0x55a606eff720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.245+0000 7ff76b7fe640 1 -- 172.21.15.172:0/1414093667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606f0ea90 msgr2=0x55a606f10e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:34.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.245+0000 7ff76b7fe640 1 --2- 172.21.15.172:0/1414093667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606f0ea90 0x55a606f10e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.245+0000 7ff76b7fe640 1 -- 172.21.15.172:0/1414093667 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a606df1440 con 0x55a606effc60 2024-09-10T08:49:34.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.245+0000 7ff76b7fe640 1 --2- 172.21.15.172:0/1414093667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606effc60 0x55a606f0e550 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009900 tx=0x7ff75c032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28349bce0af02016 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:34.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.245+0000 7ff76bfff640 1 --2- 172.21.15.172:0/4046666798 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7540a2b80 0x7ff7540a5040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff764097eb0 tx=0x7ff764096040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:34.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.245+0000 7ff751ffb640 1 -- 172.21.15.172:0/1414093667 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c040070 con 0x55a606effc60 2024-09-10T08:49:34.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.245+0000 7ff751ffb640 1 -- 172.21.15.172:0/1414093667 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff75c004030 con 0x55a606effc60 2024-09-10T08:49:34.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.245+0000 7ff751ffb640 1 -- 172.21.15.172:0/1414093667 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c035040 con 0x55a606effc60 2024-09-10T08:49:34.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.246+0000 7ff774192740 1 -- 172.21.15.172:0/1414093667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606effc60 msgr2=0x55a606f0e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:34.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.246+0000 7ff774192740 1 --2- 172.21.15.172:0/1414093667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606effc60 0x55a606f0e550 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009900 tx=0x7ff75c032bb0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.246+0000 7ff774192740 1 -- 172.21.15.172:0/1414093667 shutdown_connections 2024-09-10T08:49:34.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.246+0000 7ff774192740 1 --2- 172.21.15.172:0/1414093667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606f0ea90 0x55a606f10e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.246+0000 7ff774192740 1 --2- 172.21.15.172:0/1414093667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606effc60 0x55a606f0e550 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.246+0000 7ff774192740 1 --2- 172.21.15.172:0/1414093667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606eff340 0x55a606eff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.246+0000 7ff774192740 1 -- 172.21.15.172:0/1414093667 >> 172.21.15.172:0/1414093667 conn(0x55a606efba30 msgr2=0x55a606efbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:34.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.246+0000 7ff774192740 1 -- 172.21.15.172:0/1414093667 shutdown_connections 2024-09-10T08:49:34.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.246+0000 7ff774192740 1 -- 172.21.15.172:0/1414093667 wait complete. 2024-09-10T08:49:34.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.246+0000 7ff774192740 1 -- start start 2024-09-10T08:49:34.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.247+0000 7ff774192740 1 Processor -- start 2024-09-10T08:49:34.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.247+0000 7ff774192740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606eff340 0x55a606f0e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.247+0000 7ff774192740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606effc60 0x55a606f0b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.247+0000 7ff774192740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606f0ea90 0x55a606f0bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.247+0000 7ff774192740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a606f11660 con 0x55a606effc60 2024-09-10T08:49:34.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.247+0000 7ff774192740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a606e20050 con 0x55a606f0ea90 2024-09-10T08:49:34.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.247+0000 7ff774192740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a606f117e0 con 0x55a606eff340 2024-09-10T08:49:34.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.247+0000 7ff774192740 10 client.0 _ll_register_callbacks cb 0x55a606e60160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:34.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.247+0000 7ff76b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606effc60 0x55a606f0b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.247+0000 7ff76b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606effc60 0x55a606f0b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46888/0 (socket says 172.21.15.172:46888) 2024-09-10T08:49:34.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.247+0000 7ff76b7fe640 1 -- 172.21.15.172:0/1193641170 learned_addr learned my addr 172.21.15.172:0/1193641170 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:34.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.247+0000 7ff76affd640 1 --2- 172.21.15.172:0/1193641170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606eff340 0x55a606f0e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.247+0000 7ff76b7fe640 1 -- 172.21.15.172:0/1193641170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606eff340 msgr2=0x55a606f0e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:34.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.247+0000 7ff76b7fe640 1 --2- 172.21.15.172:0/1193641170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606eff340 0x55a606f0e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.247+0000 7ff76b7fe640 1 -- 172.21.15.172:0/1193641170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606f0ea90 msgr2=0x55a606f0bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:34.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.247+0000 7ff76b7fe640 1 --2- 172.21.15.172:0/1193641170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606f0ea90 0x55a606f0bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.247+0000 7ff76b7fe640 1 -- 172.21.15.172:0/1193641170 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a606bd63c0 con 0x55a606effc60 2024-09-10T08:49:34.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.248+0000 7ff76b7fe640 1 --2- 172.21.15.172:0/1193641170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606effc60 0x55a606f0b6f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff75c032fd0 tx=0x7ff75c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323285365cc65dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:34.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.248+0000 7ff74affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:34.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.248+0000 7ff74affd640 1 -- 172.21.15.172:0/1193641170 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c040070 con 0x55a606effc60 2024-09-10T08:49:34.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.248+0000 7ff774192740 1 -- 172.21.15.172:0/1193641170 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff75c004030 con 0x55a606effc60 2024-09-10T08:49:34.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.248+0000 7ff74affd640 1 -- 172.21.15.172:0/1193641170 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff75c005460 con 0x55a606effc60 2024-09-10T08:49:34.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.248+0000 7ff74affd640 1 -- 172.21.15.172:0/1193641170 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff75c056050 con 0x55a606effc60 2024-09-10T08:49:34.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.248+0000 7ff774192740 10 client.16440 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:34.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.248+0000 7ff774192740 1 -- 172.21.15.172:0/1193641170 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a606f1d0e0 con 0x55a606effc60 2024-09-10T08:49:34.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.248+0000 7ff74affd640 1 -- 172.21.15.172:0/1193641170 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff75c004030 con 0x55a606effc60 2024-09-10T08:49:34.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.249+0000 7ff74affd640 1 -- 172.21.15.172:0/1193641170 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7ff75c004ce0 con 0x55a606effc60 2024-09-10T08:49:34.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.249+0000 7ff774192740 10 client.16440 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:34.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.249+0000 7ff774192740 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:49:34.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.249+0000 7ff774192740 20 client.16440 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:34.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.249+0000 7ff774192740 1 --2- 172.21.15.172:0/1193641170 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a606f1e430 0x55a606f20820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.249+0000 7ff774192740 4 client.16440 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:49:34.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.249+0000 7ff774192740 1 -- 172.21.15.172:0/1193641170 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a606e72850 con 0x55a606f1e430 2024-09-10T08:49:34.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.249+0000 7ff76affd640 1 --2- 172.21.15.172:0/1193641170 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a606f1e430 0x55a606f20820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.249+0000 7ff76affd640 1 --2- 172.21.15.172:0/1193641170 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a606f1e430 0x55a606f20820 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42b164470d989c88 server_cookie=8db1b61ee592ded9 in_seq=0 out_seq=0 2024-09-10T08:49:34.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.249+0000 7ff74affd640 10 client.16440 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:34.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.250+0000 7ff74affd640 1 -- 172.21.15.172:0/1193641170 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a606e72850 con 0x55a606f1e430 2024-09-10T08:49:34.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.250+0000 7ff74affd640 10 client.16440 handle_command_reply: tid=0 2024-09-10T08:49:34.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.250+0000 7ff774192740 1 client.16440 shutdown 2024-09-10T08:49:34.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.250+0000 7ff774192740 1 -- 172.21.15.172:0/1193641170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606effc60 msgr2=0x55a606f0b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:34.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.250+0000 7ff774192740 1 --2- 172.21.15.172:0/1193641170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606effc60 0x55a606f0b6f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff75c032fd0 tx=0x7ff75c005db0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.250+0000 7ff774192740 1 -- 172.21.15.172:0/1193641170 shutdown_connections 2024-09-10T08:49:34.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.250+0000 7ff774192740 1 --2- 172.21.15.172:0/1193641170 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a606f1e430 0x55a606f20820 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.250+0000 7ff774192740 1 --2- 172.21.15.172:0/1193641170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606f0ea90 0x55a606f0bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.250+0000 7ff774192740 1 --2- 172.21.15.172:0/1193641170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606effc60 0x55a606f0b6f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.250+0000 7ff774192740 1 --2- 172.21.15.172:0/1193641170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606eff340 0x55a606f0e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.250+0000 7ff774192740 1 -- 172.21.15.172:0/1193641170 >> 172.21.15.172:0/1193641170 conn(0x55a606efba30 msgr2=0x55a606f01790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:34.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.250+0000 7ff74affd640 0 client.16440 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:34.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.251+0000 7ff774192740 1 -- 172.21.15.172:0/1193641170 shutdown_connections 2024-09-10T08:49:34.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.251+0000 7ff774192740 1 -- 172.21.15.172:0/1193641170 wait complete. 2024-09-10T08:49:34.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.251+0000 7ff774192740 20 client.16440 trim_cache size 0 max 16384 2024-09-10T08:49:34.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.276+0000 7ff774192740 1 Processor -- start 2024-09-10T08:49:34.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.276+0000 7ff774192740 1 -- start start 2024-09-10T08:49:34.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.276+0000 7ff774192740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606eff340 0x55a606f1a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.276+0000 7ff774192740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606effc60 0x55a606f09fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.276+0000 7ff774192740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606f0ea90 0x55a606efaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.276+0000 7ff774192740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c1ac8b0 con 0x55a606eff340 2024-09-10T08:49:34.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.276+0000 7ff774192740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c1ae450 con 0x55a606f0ea90 2024-09-10T08:49:34.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.276+0000 7ff774192740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c1acf80 con 0x55a606effc60 2024-09-10T08:49:34.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.276+0000 7ff76affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606eff340 0x55a606f1a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.277+0000 7ff76affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606eff340 0x55a606f1a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46896/0 (socket says 172.21.15.172:46896) 2024-09-10T08:49:34.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.277+0000 7ff76affd640 1 -- 172.21.15.172:0/3018773638 learned_addr learned my addr 172.21.15.172:0/3018773638 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:34.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.277+0000 7ff76b7fe640 1 --2- 172.21.15.172:0/3018773638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606effc60 0x55a606f09fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.277+0000 7ff76affd640 1 -- 172.21.15.172:0/3018773638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606effc60 msgr2=0x55a606f09fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:34.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.277+0000 7ff76affd640 1 --2- 172.21.15.172:0/3018773638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606effc60 0x55a606f09fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.277+0000 7ff76affd640 1 -- 172.21.15.172:0/3018773638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606f0ea90 msgr2=0x55a606efaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:34.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.277+0000 7ff76affd640 1 --2- 172.21.15.172:0/3018773638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606f0ea90 0x55a606efaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.277+0000 7ff76affd640 1 -- 172.21.15.172:0/3018773638 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a606efcbb0 con 0x55a606eff340 2024-09-10T08:49:34.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.277+0000 7ff76b7fe640 1 --2- 172.21.15.172:0/3018773638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606effc60 0x55a606f09fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:49:34.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.277+0000 7ff76affd640 1 --2- 172.21.15.172:0/3018773638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606eff340 0x55a606f1a930 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55a606f20e70 tx=0x7ff7600039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4ce3427a87163b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:34.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.277+0000 7ff7517fa640 1 -- 172.21.15.172:0/3018773638 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760012730 con 0x55a606eff340 2024-09-10T08:49:34.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.277+0000 7ff7517fa640 1 -- 172.21.15.172:0/3018773638 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff760023070 con 0x55a606eff340 2024-09-10T08:49:34.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.277+0000 7ff7517fa640 1 -- 172.21.15.172:0/3018773638 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76001e8e0 con 0x55a606eff340 2024-09-10T08:49:34.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.278+0000 7ff774192740 1 -- 172.21.15.172:0/3018773638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606eff340 msgr2=0x55a606f1a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:34.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.278+0000 7ff774192740 1 --2- 172.21.15.172:0/3018773638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606eff340 0x55a606f1a930 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55a606f20e70 tx=0x7ff7600039c0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.278+0000 7ff774192740 1 -- 172.21.15.172:0/3018773638 shutdown_connections 2024-09-10T08:49:34.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.278+0000 7ff774192740 1 --2- 172.21.15.172:0/3018773638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606f0ea90 0x55a606efaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.278+0000 7ff774192740 1 --2- 172.21.15.172:0/3018773638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606effc60 0x55a606f09fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.278+0000 7ff774192740 1 --2- 172.21.15.172:0/3018773638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606eff340 0x55a606f1a930 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.278+0000 7ff774192740 1 -- 172.21.15.172:0/3018773638 >> 172.21.15.172:0/3018773638 conn(0x55a606efba30 msgr2=0x55a606efa350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:34.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.278+0000 7ff774192740 1 -- 172.21.15.172:0/3018773638 shutdown_connections 2024-09-10T08:49:34.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.278+0000 7ff774192740 1 -- 172.21.15.172:0/3018773638 wait complete. 2024-09-10T08:49:34.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.278+0000 7ff774192740 1 -- start start 2024-09-10T08:49:34.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.279+0000 7ff774192740 1 Processor -- start 2024-09-10T08:49:34.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.279+0000 7ff774192740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606eff340 0x55a606f12b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.279+0000 7ff774192740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606effc60 0x55a606f13060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.279+0000 7ff774192740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606f0ea90 0x55a606f87ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.279+0000 7ff774192740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c1add20 con 0x55a606f0ea90 2024-09-10T08:49:34.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.279+0000 7ff774192740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c1ad650 con 0x55a606effc60 2024-09-10T08:49:34.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.279+0000 7ff774192740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c1aa890 con 0x55a606eff340 2024-09-10T08:49:34.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.279+0000 7ff774192740 10 client.0 _ll_register_callbacks cb 0x7ff76000bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:34.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.279+0000 7ff76affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606f0ea90 0x55a606f87ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.279+0000 7ff76affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606f0ea90 0x55a606f87ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46906/0 (socket says 172.21.15.172:46906) 2024-09-10T08:49:34.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.279+0000 7ff76affd640 1 -- 172.21.15.172:0/1806529107 learned_addr learned my addr 172.21.15.172:0/1806529107 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:34.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.279+0000 7ff76affd640 1 --2- 172.21.15.172:0/1806529107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606eff340 0x55a606f12b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.279+0000 7ff76b7fe640 1 --2- 172.21.15.172:0/1806529107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606effc60 0x55a606f13060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.280+0000 7ff76affd640 1 -- 172.21.15.172:0/1806529107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606eff340 msgr2=0x55a606f12b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:34.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.280+0000 7ff76affd640 1 --2- 172.21.15.172:0/1806529107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606eff340 0x55a606f12b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.280+0000 7ff76affd640 1 -- 172.21.15.172:0/1806529107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606effc60 msgr2=0x55a606f13060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:34.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.280+0000 7ff76affd640 1 --2- 172.21.15.172:0/1806529107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606effc60 0x55a606f13060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.280+0000 7ff76affd640 1 -- 172.21.15.172:0/1806529107 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a606f138f0 con 0x55a606f0ea90 2024-09-10T08:49:34.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.280+0000 7ff76affd640 1 --2- 172.21.15.172:0/1806529107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606f0ea90 0x55a606f87ea0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff7600042c0 tx=0x7ff760016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf5588461392b32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:34.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.280+0000 7ff74affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:34.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.280+0000 7ff74affd640 1 -- 172.21.15.172:0/1806529107 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760005020 con 0x55a606f0ea90 2024-09-10T08:49:34.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.280+0000 7ff74affd640 1 -- 172.21.15.172:0/1806529107 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff7600051c0 con 0x55a606f0ea90 2024-09-10T08:49:34.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.280+0000 7ff774192740 1 -- 172.21.15.172:0/1806529107 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff760023070 con 0x55a606f0ea90 2024-09-10T08:49:34.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.280+0000 7ff74affd640 1 -- 172.21.15.172:0/1806529107 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff760038050 con 0x55a606f0ea90 2024-09-10T08:49:34.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.280+0000 7ff774192740 10 client.16446 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:34.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.280+0000 7ff774192740 1 -- 172.21.15.172:0/1806529107 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a606f894c0 con 0x55a606f0ea90 2024-09-10T08:49:34.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.280+0000 7ff74affd640 1 -- 172.21.15.172:0/1806529107 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff76001c040 con 0x55a606f0ea90 2024-09-10T08:49:34.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.281+0000 7ff74affd640 1 -- 172.21.15.172:0/1806529107 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7ff76001e310 con 0x55a606f0ea90 2024-09-10T08:49:34.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.281+0000 7ff774192740 10 client.16446 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:34.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.281+0000 7ff774192740 10 client.16446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:49:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.281+0000 7ff774192740 20 client.16446 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.281+0000 7ff774192740 1 --2- 172.21.15.172:0/1806529107 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a606f89b80 0x55a606f8bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.281+0000 7ff774192740 4 client.16446 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:49:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.281+0000 7ff774192740 1 -- 172.21.15.172:0/1806529107 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ff760023070 con 0x55a606f89b80 2024-09-10T08:49:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.281+0000 7ff76b7fe640 1 --2- 172.21.15.172:0/1806529107 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a606f89b80 0x55a606f8bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.281+0000 7ff76b7fe640 1 --2- 172.21.15.172:0/1806529107 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a606f89b80 0x55a606f8bfa0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b873780b67d16037 server_cookie=8fbcad3e19332ce7 in_seq=0 out_seq=0 2024-09-10T08:49:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.281+0000 7ff74affd640 10 client.16446 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.282+0000 7ff74affd640 1 -- 172.21.15.172:0/1806529107 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7ff760023070 con 0x55a606f89b80 2024-09-10T08:49:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.282+0000 7ff74affd640 10 client.16446 handle_command_reply: tid=0 2024-09-10T08:49:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.282+0000 7ff774192740 1 client.16446 shutdown 2024-09-10T08:49:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.282+0000 7ff774192740 1 -- 172.21.15.172:0/1806529107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606f0ea90 msgr2=0x55a606f87ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.282+0000 7ff774192740 1 --2- 172.21.15.172:0/1806529107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606f0ea90 0x55a606f87ea0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff7600042c0 tx=0x7ff760016040 comp rx=0 tx=0).stop 2024-09-10T08:49:34.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.282+0000 7ff774192740 1 -- 172.21.15.172:0/1806529107 shutdown_connections 2024-09-10T08:49:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.282+0000 7ff774192740 1 --2- 172.21.15.172:0/1806529107 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a606f89b80 0x55a606f8bfa0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.282+0000 7ff74affd640 0 client.16446 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.282+0000 7ff774192740 1 --2- 172.21.15.172:0/1806529107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a606f0ea90 0x55a606f87ea0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.282+0000 7ff774192740 1 --2- 172.21.15.172:0/1806529107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a606effc60 0x55a606f13060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.282+0000 7ff774192740 1 --2- 172.21.15.172:0/1806529107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a606eff340 0x55a606f12b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.282+0000 7ff774192740 1 -- 172.21.15.172:0/1806529107 >> 172.21.15.172:0/1806529107 conn(0x55a606efba30 msgr2=0x55a606efab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.282+0000 7ff774192740 1 -- 172.21.15.172:0/1806529107 shutdown_connections 2024-09-10T08:49:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.282+0000 7ff774192740 1 -- 172.21.15.172:0/1806529107 wait complete. 2024-09-10T08:49:34.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.282+0000 7ff774192740 20 client.16446 trim_cache size 0 max 16384 2024-09-10T08:49:34.288 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:49:34.289 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:49:34.289 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T08:49:34.289 INFO:teuthology.orchestra.run.smithi172.stdout: "942597b4-6365-4a2d-a3f8-f093da8586e6": { 2024-09-10T08:49:34.289 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T08:49:34.289 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "942597b4-6365-4a2d-a3f8-f093da8586e6", 2024-09-10T08:49:34.289 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T08:49:34.289 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:49:34.289 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:49:34.289 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:49:34.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.283+0000 7ff771db9640 1 -- 172.21.15.172:0/4046666798 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7540a2b80 msgr2=0x7ff7540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:34.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.283+0000 7ff771db9640 1 --2- 172.21.15.172:0/4046666798 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7540a2b80 0x7ff7540a5040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff764097eb0 tx=0x7ff764096040 comp rx=0 tx=0).stop 2024-09-10T08:49:34.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.283+0000 7ff771db9640 1 -- 172.21.15.172:0/4046666798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76c106f10 msgr2=0x7ff76c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:34.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.283+0000 7ff771db9640 1 --2- 172.21.15.172:0/4046666798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76c106f10 0x7ff76c19cc30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff76400eaa0 tx=0x7ff76400ef60 comp rx=0 tx=0).stop 2024-09-10T08:49:34.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.283+0000 7ff771db9640 1 -- 172.21.15.172:0/4046666798 shutdown_connections 2024-09-10T08:49:34.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.283+0000 7ff771db9640 1 --2- 172.21.15.172:0/4046666798 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7540a2b80 0x7ff7540a5040 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.283+0000 7ff771db9640 1 --2- 172.21.15.172:0/4046666798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff76c106f10 0x7ff76c19cc30 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.283+0000 7ff771db9640 1 --2- 172.21.15.172:0/4046666798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c102790 0x7ff76c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.283+0000 7ff771db9640 1 --2- 172.21.15.172:0/4046666798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c101de0 0x7ff76c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:34.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.283+0000 7ff771db9640 1 -- 172.21.15.172:0/4046666798 >> 172.21.15.172:0/4046666798 conn(0x7ff76c0fd8e0 msgr2=0x7ff76c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:34.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.283+0000 7ff771db9640 1 -- 172.21.15.172:0/4046666798 shutdown_connections 2024-09-10T08:49:34.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:49:34.283+0000 7ff771db9640 1 -- 172.21.15.172:0/4046666798 wait complete. 2024-09-10T08:49:34.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:34 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T08:49:34.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:34 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:34.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:34 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:49:34.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:34 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:34.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:34 smithi172 ceph-mon[30854]: pgmap v450: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-09-10T08:49:34.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:34 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T08:49:34.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:34 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:34.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:34 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:49:34.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:34 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:34.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:34 smithi174 ceph-mon[34068]: pgmap v450: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-09-10T08:49:34.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:34 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T08:49:34.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:34 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:34.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:34 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:49:34.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:34 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:49:34.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:34 smithi191 ceph-mon[35314]: pgmap v450: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-09-10T08:49:35.004 INFO:tasks.cephfs.filesystem:scrub status for tag:942597b4-6365-4a2d-a3f8-f093da8586e6 - {'path': '/', 'tag': '942597b4-6365-4a2d-a3f8-f093da8586e6', 'options': 'recursive,force'} 2024-09-10T08:49:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.977+0000 7f8d0c417640 1 -- 172.21.15.172:0/3771220966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d04100a40 msgr2=0x7f8d04100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.977+0000 7f8d0c417640 1 --2- 172.21.15.172:0/3771220966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d04100a40 0x7f8d04100e40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009900 tx=0x7f8d0002a410 comp rx=0 tx=0).stop 2024-09-10T08:49:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.977+0000 7f8d0c417640 1 -- 172.21.15.172:0/3771220966 shutdown_connections 2024-09-10T08:49:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.977+0000 7f8d0c417640 1 --2- 172.21.15.172:0/3771220966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d041025f0 0x7f8d04108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.977+0000 7f8d0c417640 1 --2- 172.21.15.172:0/3771220966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d04101c30 0x7f8d041020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.977+0000 7f8d0c417640 1 --2- 172.21.15.172:0/3771220966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d04100a40 0x7f8d04100e40 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.977+0000 7f8d0c417640 1 -- 172.21.15.172:0/3771220966 >> 172.21.15.172:0/3771220966 conn(0x7f8d040fc1e0 msgr2=0x7f8d040fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.977+0000 7f8d0c417640 1 -- 172.21.15.172:0/3771220966 shutdown_connections 2024-09-10T08:49:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.977+0000 7f8d0c417640 1 -- 172.21.15.172:0/3771220966 wait complete. 2024-09-10T08:49:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.978+0000 7f8d0c417640 1 Processor -- start 2024-09-10T08:49:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.978+0000 7f8d0c417640 1 -- start start 2024-09-10T08:49:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.978+0000 7f8d0c417640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d04100a40 0x7f8d041049d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.978+0000 7f8d0c417640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d04101c30 0x7f8d04103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.978+0000 7f8d0c417640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d041025f0 0x7f8d04103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.978+0000 7f8d0c417640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0410c570 con 0x7f8d04100a40 2024-09-10T08:49:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.978+0000 7f8d0c417640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0410c3f0 con 0x7f8d04101c30 2024-09-10T08:49:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.978+0000 7f8d0c417640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0410c6f0 con 0x7f8d041025f0 2024-09-10T08:49:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.978+0000 7f8d0a1b3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d04100a40 0x7f8d041049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.978+0000 7f8d0a1b3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d04100a40 0x7f8d041049d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46922/0 (socket says 172.21.15.172:46922) 2024-09-10T08:49:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.978+0000 7f8d0a1b3640 1 -- 172.21.15.172:0/2181479572 learned_addr learned my addr 172.21.15.172:0/2181479572 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.978+0000 7f8d0a9b4640 1 --2- 172.21.15.172:0/2181479572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d041025f0 0x7f8d04103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.978+0000 7f8d099b2640 1 --2- 172.21.15.172:0/2181479572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d04101c30 0x7f8d04103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.979+0000 7f8d0a1b3640 1 -- 172.21.15.172:0/2181479572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d041025f0 msgr2=0x7f8d04103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.979+0000 7f8d0a1b3640 1 --2- 172.21.15.172:0/2181479572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d041025f0 0x7f8d04103550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.979+0000 7f8d0a1b3640 1 -- 172.21.15.172:0/2181479572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d04101c30 msgr2=0x7f8d04103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.979+0000 7f8d0a1b3640 1 --2- 172.21.15.172:0/2181479572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d04101c30 0x7f8d04103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.979+0000 7f8d0a1b3640 1 -- 172.21.15.172:0/2181479572 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d04103de0 con 0x7f8d04100a40 2024-09-10T08:49:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.979+0000 7f8d099b2640 1 --2- 172.21.15.172:0/2181479572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d04101c30 0x7f8d04103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:49:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.979+0000 7f8d0a1b3640 1 --2- 172.21.15.172:0/2181479572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d04100a40 0x7f8d041049d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8d000314e0 tx=0x7f8d000042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.979+0000 7f8cfb7fe640 1 -- 172.21.15.172:0/2181479572 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0003f030 con 0x7f8d04100a40 2024-09-10T08:49:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.979+0000 7f8d0c417640 1 -- 172.21.15.172:0/2181479572 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d041a1dd0 con 0x7f8d04100a40 2024-09-10T08:49:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.979+0000 7f8cfb7fe640 1 -- 172.21.15.172:0/2181479572 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d00003cb0 con 0x7f8d04100a40 2024-09-10T08:49:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.979+0000 7f8cfb7fe640 1 -- 172.21.15.172:0/2181479572 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d00035070 con 0x7f8d04100a40 2024-09-10T08:49:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.979+0000 7f8d0c417640 1 -- 172.21.15.172:0/2181479572 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d041a2230 con 0x7f8d04100a40 2024-09-10T08:49:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.981+0000 7f8cfb7fe640 1 -- 172.21.15.172:0/2181479572 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8d00042030 con 0x7f8d04100a40 2024-09-10T08:49:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.982+0000 7f8d0f289740 1 Processor -- start 2024-09-10T08:49:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.982+0000 7f8d0f289740 1 -- start start 2024-09-10T08:49:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.982+0000 7f8cfb7fe640 1 --2- 172.21.15.172:0/2181479572 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8cf00a2c30 0x7f8cf00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.982+0000 7f8d0f289740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 0x55bf920f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.982+0000 7f8d0f289740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 0x55bf920fcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.982+0000 7f8d0f289740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920fd370 0x55bf920ff830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.982+0000 7f8d0f289740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bf9205b2f0 con 0x55bf920f82e0 2024-09-10T08:49:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.982+0000 7f8d0f289740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf92003270 con 0x55bf920f79a0 2024-09-10T08:49:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.982+0000 7f8d0f289740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bf920572b0 con 0x55bf920fd370 2024-09-10T08:49:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.983+0000 7f8cfb7fe640 1 -- 172.21.15.172:0/2181479572 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8d000ef9d0 con 0x7f8d04100a40 2024-09-10T08:49:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.983+0000 7f8d0a1b3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 0x55bf920f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.983+0000 7f8d0a9b4640 1 --2- 172.21.15.172:0/2181479572 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8cf00a2c30 0x7f8cf00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.983+0000 7f8d0a1b3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 0x55bf920f7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:50732/0 (socket says 172.21.15.172:50732) 2024-09-10T08:49:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.983+0000 7f8d0a1b3640 1 -- 172.21.15.172:0/599743261 learned_addr learned my addr 172.21.15.172:0/599743261 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.983+0000 7f8d0a9b4640 1 --2- 172.21.15.172:0/599743261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920fd370 0x55bf920ff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.983+0000 7f8d0a1b3640 1 -- 172.21.15.172:0/599743261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920fd370 msgr2=0x55bf920ff830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.983+0000 7f8d0a1b3640 1 --2- 172.21.15.172:0/599743261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920fd370 0x55bf920ff830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.983+0000 7f8d0a1b3640 1 -- 172.21.15.172:0/599743261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 msgr2=0x55bf920fcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.983+0000 7f8d0a1b3640 1 --2- 172.21.15.172:0/599743261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 0x55bf920fcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.983+0000 7f8d0a1b3640 1 -- 172.21.15.172:0/599743261 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf92002340 con 0x55bf920f79a0 2024-09-10T08:49:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.983+0000 7f8d0a9b4640 1 --2- 172.21.15.172:0/599743261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920fd370 0x55bf920ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:49:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.983+0000 7f8d0a1b3640 1 --2- 172.21.15.172:0/599743261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 0x55bf920f7da0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8d000c4610 tx=0x7f8d0002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b10da31328726b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.983+0000 7f8d0a9b4640 1 --2- 172.21.15.172:0/2181479572 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8cf00a2c30 0x7f8cf00a50f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005fd0 tx=0x7f8cfc009c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.983+0000 7f8cf8bfe640 1 -- 172.21.15.172:0/599743261 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d00049040 con 0x55bf920f79a0 2024-09-10T08:49:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.983+0000 7f8cf8bfe640 1 -- 172.21.15.172:0/599743261 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d000bc5d0 con 0x55bf920f79a0 2024-09-10T08:49:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.983+0000 7f8cf8bfe640 1 -- 172.21.15.172:0/599743261 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d000dd300 con 0x55bf920f79a0 2024-09-10T08:49:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.984+0000 7f8d0f289740 1 -- 172.21.15.172:0/599743261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 msgr2=0x55bf920f7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.984+0000 7f8d0f289740 1 --2- 172.21.15.172:0/599743261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 0x55bf920f7da0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8d000c4610 tx=0x7f8d0002a410 comp rx=0 tx=0).stop 2024-09-10T08:49:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.984+0000 7f8d0f289740 1 -- 172.21.15.172:0/599743261 shutdown_connections 2024-09-10T08:49:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.984+0000 7f8d0f289740 1 --2- 172.21.15.172:0/599743261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920fd370 0x55bf920ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.984+0000 7f8d0f289740 1 --2- 172.21.15.172:0/599743261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 0x55bf920fcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.984+0000 7f8d0f289740 1 --2- 172.21.15.172:0/599743261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 0x55bf920f7da0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.984+0000 7f8d0f289740 1 -- 172.21.15.172:0/599743261 >> 172.21.15.172:0/599743261 conn(0x55bf920f3010 msgr2=0x55bf920f3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.984+0000 7f8d0f289740 1 -- 172.21.15.172:0/599743261 shutdown_connections 2024-09-10T08:49:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.984+0000 7f8d0f289740 1 -- 172.21.15.172:0/599743261 wait complete. 2024-09-10T08:49:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.985+0000 7f8d0f289740 1 -- start start 2024-09-10T08:49:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.985+0000 7f8d0f289740 1 Processor -- start 2024-09-10T08:49:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.985+0000 7f8d0f289740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 0x55bf92101890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.985+0000 7f8d0f289740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 0x55bf92101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.985+0000 7f8d0f289740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920fd370 0x55bf92108da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.985+0000 7f8d0f289740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bf92003270 con 0x55bf920f82e0 2024-09-10T08:49:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.985+0000 7f8d0f289740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf91fe4290 con 0x55bf920f79a0 2024-09-10T08:49:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.985+0000 7f8d0f289740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bf92100100 con 0x55bf920fd370 2024-09-10T08:49:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.985+0000 7f8d0f289740 10 client.0 _ll_register_callbacks cb 0x55bf92015140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.985+0000 7f8d099b2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 0x55bf92101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.985+0000 7f8d099b2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 0x55bf92101dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46952/0 (socket says 172.21.15.172:46952) 2024-09-10T08:49:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.985+0000 7f8d099b2640 1 -- 172.21.15.172:0/3067837112 learned_addr learned my addr 172.21.15.172:0/3067837112 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.985+0000 7f8d0a1b3640 1 --2- 172.21.15.172:0/3067837112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 0x55bf92101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.985+0000 7f8d0a9b4640 1 --2- 172.21.15.172:0/3067837112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920fd370 0x55bf92108da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.986+0000 7f8d0a1b3640 1 -- 172.21.15.172:0/3067837112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920fd370 msgr2=0x55bf92108da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.986+0000 7f8d0a1b3640 1 --2- 172.21.15.172:0/3067837112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920fd370 0x55bf92108da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.986+0000 7f8d0a1b3640 1 -- 172.21.15.172:0/3067837112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 msgr2=0x55bf92101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.986+0000 7f8d0a1b3640 1 --2- 172.21.15.172:0/3067837112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 0x55bf92101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:35.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.986+0000 7f8d0a1b3640 1 -- 172.21.15.172:0/3067837112 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf91d6e210 con 0x55bf920f79a0 2024-09-10T08:49:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.986+0000 7f8d099b2640 1 --2- 172.21.15.172:0/3067837112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 0x55bf92101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.986+0000 7f8d0a1b3640 1 --2- 172.21.15.172:0/3067837112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 0x55bf92101890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8d000c0440 tx=0x7f8d000c02f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f1751eb0baaa920 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.986+0000 7f8cedffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:49:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.986+0000 7f8cedffb640 1 -- 172.21.15.172:0/3067837112 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d00049070 con 0x55bf920f79a0 2024-09-10T08:49:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.986+0000 7f8cedffb640 1 -- 172.21.15.172:0/3067837112 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8d000b8540 con 0x55bf920f79a0 2024-09-10T08:49:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.986+0000 7f8d0f289740 1 -- 172.21.15.172:0/3067837112 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d000bc5d0 con 0x55bf920f79a0 2024-09-10T08:49:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.986+0000 7f8cedffb640 1 -- 172.21.15.172:0/3067837112 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8d000c4b40 con 0x55bf920f79a0 2024-09-10T08:49:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.986+0000 7f8d0f289740 10 client.25787 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.986+0000 7f8d0f289740 1 -- 172.21.15.172:0/3067837112 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bf91e3a620 con 0x55bf920f79a0 2024-09-10T08:49:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.986+0000 7f8cedffb640 1 -- 172.21.15.172:0/3067837112 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d000bc5d0 con 0x55bf920f79a0 2024-09-10T08:49:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.986+0000 7f8cedffb640 1 -- 172.21.15.172:0/3067837112 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f8d000bc130 con 0x55bf920f79a0 2024-09-10T08:49:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.987+0000 7f8d0f289740 10 client.25787 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:35.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.987+0000 7f8d0f289740 10 client.25787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:49:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.987+0000 7f8d0f289740 20 client.25787 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.987+0000 7f8d0f289740 1 --2- 172.21.15.172:0/3067837112 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bf9210d0b0 0x55bf9210f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.987+0000 7f8d0f289740 4 client.25787 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:49:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.987+0000 7f8d0f289740 1 -- 172.21.15.172:0/3067837112 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf92068a20 con 0x55bf9210d0b0 2024-09-10T08:49:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.987+0000 7f8d099b2640 1 --2- 172.21.15.172:0/3067837112 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bf9210d0b0 0x55bf9210f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.987+0000 7f8d099b2640 1 --2- 172.21.15.172:0/3067837112 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bf9210d0b0 0x55bf9210f4a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7dce5fdef8ee355 server_cookie=71c5224cc38b0e34 in_seq=0 out_seq=0 2024-09-10T08:49:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.987+0000 7f8cedffb640 10 client.25787 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.988+0000 7f8cedffb640 1 -- 172.21.15.172:0/3067837112 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf92068a20 con 0x55bf9210d0b0 2024-09-10T08:49:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.988+0000 7f8cedffb640 10 client.25787 handle_command_reply: tid=0 2024-09-10T08:49:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.988+0000 7f8d0f289740 1 client.25787 shutdown 2024-09-10T08:49:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.988+0000 7f8d0f289740 1 -- 172.21.15.172:0/3067837112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 msgr2=0x55bf92101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.988+0000 7f8d0f289740 1 --2- 172.21.15.172:0/3067837112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 0x55bf92101890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8d000c0440 tx=0x7f8d000c02f0 comp rx=0 tx=0).stop 2024-09-10T08:49:35.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.988+0000 7f8d0f289740 1 -- 172.21.15.172:0/3067837112 shutdown_connections 2024-09-10T08:49:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.988+0000 7f8d0f289740 1 --2- 172.21.15.172:0/3067837112 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bf9210d0b0 0x55bf9210f4a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.988+0000 7f8d0f289740 1 --2- 172.21.15.172:0/3067837112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920fd370 0x55bf92108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.988+0000 7f8d0f289740 1 --2- 172.21.15.172:0/3067837112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 0x55bf92101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.988+0000 7f8d0f289740 1 --2- 172.21.15.172:0/3067837112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 0x55bf92101890 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.988+0000 7f8cedffb640 0 client.25787 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.988+0000 7f8d0f289740 1 -- 172.21.15.172:0/3067837112 >> 172.21.15.172:0/3067837112 conn(0x55bf920f3010 msgr2=0x55bf920f4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.988+0000 7f8d0f289740 1 -- 172.21.15.172:0/3067837112 shutdown_connections 2024-09-10T08:49:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.988+0000 7f8d0f289740 1 -- 172.21.15.172:0/3067837112 wait complete. 2024-09-10T08:49:35.999 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:35.988+0000 7f8d0f289740 20 client.25787 trim_cache size 0 max 16384 2024-09-10T08:49:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.005+0000 7f8d0f289740 1 Processor -- start 2024-09-10T08:49:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d0f289740 1 -- start start 2024-09-10T08:49:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d0f289740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920f79a0 0x55bf920f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d0f289740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f82e0 0x55bf92108d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d0f289740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920fd370 0x55bf920f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d0f289740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041ac7e0 con 0x55bf920fd370 2024-09-10T08:49:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d0f289740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041ae380 con 0x55bf920f82e0 2024-09-10T08:49:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d0f289740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041aceb0 con 0x55bf920f79a0 2024-09-10T08:49:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d0a9b4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920fd370 0x55bf920f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d0a9b4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920fd370 0x55bf920f3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46960/0 (socket says 172.21.15.172:46960) 2024-09-10T08:49:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d0a9b4640 1 -- 172.21.15.172:0/2406385153 learned_addr learned my addr 172.21.15.172:0/2406385153 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d099b2640 1 --2- 172.21.15.172:0/2406385153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f82e0 0x55bf92108d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d0a1b3640 1 --2- 172.21.15.172:0/2406385153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920f79a0 0x55bf920f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d0a9b4640 1 -- 172.21.15.172:0/2406385153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920f79a0 msgr2=0x55bf920f7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d0a9b4640 1 --2- 172.21.15.172:0/2406385153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920f79a0 0x55bf920f7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d0a9b4640 1 -- 172.21.15.172:0/2406385153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f82e0 msgr2=0x55bf92108d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d0a9b4640 1 --2- 172.21.15.172:0/2406385153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f82e0 0x55bf92108d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d0a9b4640 1 -- 172.21.15.172:0/2406385153 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf9210fb00 con 0x55bf920fd370 2024-09-10T08:49:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.006+0000 7f8d0a9b4640 1 --2- 172.21.15.172:0/2406385153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920fd370 0x55bf920f3e90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc012930 tx=0x7f8cfc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93503dbaa50835b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.007+0000 7f8ceffff640 1 -- 172.21.15.172:0/2406385153 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0118e0 con 0x55bf920fd370 2024-09-10T08:49:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.007+0000 7f8ceffff640 1 -- 172.21.15.172:0/2406385153 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cfc021480 con 0x55bf920fd370 2024-09-10T08:49:36.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.007+0000 7f8ceffff640 1 -- 172.21.15.172:0/2406385153 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc020620 con 0x55bf920fd370 2024-09-10T08:49:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.007+0000 7f8d0f289740 1 -- 172.21.15.172:0/2406385153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920fd370 msgr2=0x55bf920f3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.007+0000 7f8d0f289740 1 --2- 172.21.15.172:0/2406385153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920fd370 0x55bf920f3e90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc012930 tx=0x7f8cfc013000 comp rx=0 tx=0).stop 2024-09-10T08:49:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d0f289740 1 -- 172.21.15.172:0/2406385153 shutdown_connections 2024-09-10T08:49:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d0f289740 1 --2- 172.21.15.172:0/2406385153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920fd370 0x55bf920f3e90 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d0f289740 1 --2- 172.21.15.172:0/2406385153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f82e0 0x55bf92108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.012 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d0f289740 1 --2- 172.21.15.172:0/2406385153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920f79a0 0x55bf920f7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d0f289740 1 -- 172.21.15.172:0/2406385153 >> 172.21.15.172:0/2406385153 conn(0x55bf920f3010 msgr2=0x55bf92101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d0f289740 1 -- 172.21.15.172:0/2406385153 shutdown_connections 2024-09-10T08:49:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d0f289740 1 -- 172.21.15.172:0/2406385153 wait complete. 2024-09-10T08:49:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d0f289740 1 -- start start 2024-09-10T08:49:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d0f289740 1 Processor -- start 2024-09-10T08:49:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d0f289740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 0x55bf92105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d0f289740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 0x55bf92106260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d0f289740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920fd370 0x55bf92175d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d0f289740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041adc50 con 0x55bf920f82e0 2024-09-10T08:49:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d0f289740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041ad580 con 0x55bf920f79a0 2024-09-10T08:49:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d0f289740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8d041aa7c0 con 0x55bf920fd370 2024-09-10T08:49:36.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d0f289740 10 client.0 _ll_register_callbacks cb 0x7f8d000b8200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d099b2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 0x55bf92106260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d099b2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 0x55bf92106260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46976/0 (socket says 172.21.15.172:46976) 2024-09-10T08:49:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d099b2640 1 -- 172.21.15.172:0/3320548280 learned_addr learned my addr 172.21.15.172:0/3320548280 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.008+0000 7f8d0a9b4640 1 --2- 172.21.15.172:0/3320548280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920fd370 0x55bf92175d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.009+0000 7f8d0a1b3640 1 --2- 172.21.15.172:0/3320548280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 0x55bf92105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.009+0000 7f8d099b2640 1 -- 172.21.15.172:0/3320548280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920fd370 msgr2=0x55bf92175d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.009+0000 7f8d099b2640 1 --2- 172.21.15.172:0/3320548280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920fd370 0x55bf92175d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.009+0000 7f8d099b2640 1 -- 172.21.15.172:0/3320548280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 msgr2=0x55bf92105d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.009+0000 7f8d099b2640 1 --2- 172.21.15.172:0/3320548280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 0x55bf92105d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.009+0000 7f8d099b2640 1 -- 172.21.15.172:0/3320548280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf91e3a620 con 0x55bf920f82e0 2024-09-10T08:49:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.009+0000 7f8d099b2640 1 --2- 172.21.15.172:0/3320548280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 0x55bf92106260 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x55bf9210fcd0 tx=0x7f8cf4005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368647637959487d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.009+0000 7f8cedffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.009+0000 7f8cedffb640 1 -- 172.21.15.172:0/3320548280 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4010920 con 0x55bf920f82e0 2024-09-10T08:49:36.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.009+0000 7f8cedffb640 1 -- 172.21.15.172:0/3320548280 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8cf4010ac0 con 0x55bf920f82e0 2024-09-10T08:49:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.009+0000 7f8d0f289740 1 -- 172.21.15.172:0/3320548280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cfc021480 con 0x55bf920f82e0 2024-09-10T08:49:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.009+0000 7f8cedffb640 1 -- 172.21.15.172:0/3320548280 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8cf4010cc0 con 0x55bf920f82e0 2024-09-10T08:49:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.009+0000 7f8d0f289740 10 client.16461 fetch_fsmap learned FSMap version 31 2024-09-10T08:49:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.009+0000 7f8d0f289740 1 -- 172.21.15.172:0/3320548280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bf92177450 con 0x55bf920f82e0 2024-09-10T08:49:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.009+0000 7f8cedffb640 1 -- 172.21.15.172:0/3320548280 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8cfc021480 con 0x55bf920f82e0 2024-09-10T08:49:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.010+0000 7f8cedffb640 1 -- 172.21.15.172:0/3320548280 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f8cf4020070 con 0x55bf920f82e0 2024-09-10T08:49:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.010+0000 7f8d0f289740 10 client.16461 fetch_fsmap finished waiting for FSMap version 31 2024-09-10T08:49:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.010+0000 7f8d0f289740 10 client.16461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:49:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.010+0000 7f8d0f289740 20 client.16461 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.010+0000 7f8d0f289740 1 --2- 172.21.15.172:0/3320548280 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bf92177ee0 0x55bf9217a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.010+0000 7f8d0f289740 4 client.16461 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:49:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.010+0000 7f8d0f289740 1 -- 172.21.15.172:0/3320548280 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf92062d70 con 0x55bf92177ee0 2024-09-10T08:49:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.010+0000 7f8d0a1b3640 1 --2- 172.21.15.172:0/3320548280 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bf92177ee0 0x55bf9217a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.010+0000 7f8cedffb640 10 client.16461 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.010+0000 7f8d0a1b3640 1 --2- 172.21.15.172:0/3320548280 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bf92177ee0 0x55bf9217a300 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e04d6e26775a7d server_cookie=9d76aba73e26c1fb in_seq=0 out_seq=0 2024-09-10T08:49:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.019+0000 7f8cedffb640 1 -- 172.21.15.172:0/3320548280 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf92062d70 con 0x55bf92177ee0 2024-09-10T08:49:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.019+0000 7f8cedffb640 10 client.16461 handle_command_reply: tid=0 2024-09-10T08:49:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.019+0000 7f8d0f289740 1 client.16461 shutdown 2024-09-10T08:49:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.020+0000 7f8d0f289740 1 -- 172.21.15.172:0/3320548280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 msgr2=0x55bf92106260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.020+0000 7f8d0f289740 1 --2- 172.21.15.172:0/3320548280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 0x55bf92106260 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x55bf9210fcd0 tx=0x7f8cf4005750 comp rx=0 tx=0).stop 2024-09-10T08:49:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.020+0000 7f8d0f289740 1 -- 172.21.15.172:0/3320548280 shutdown_connections 2024-09-10T08:49:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.020+0000 7f8d0f289740 1 --2- 172.21.15.172:0/3320548280 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bf92177ee0 0x55bf9217a300 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.020+0000 7f8d0f289740 1 --2- 172.21.15.172:0/3320548280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf920fd370 0x55bf92175d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.020+0000 7f8cedffb640 0 client.16461 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.020+0000 7f8d0f289740 1 --2- 172.21.15.172:0/3320548280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf920f82e0 0x55bf92106260 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.020+0000 7f8d0f289740 1 --2- 172.21.15.172:0/3320548280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf920f79a0 0x55bf92105d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.020+0000 7f8d0f289740 1 -- 172.21.15.172:0/3320548280 >> 172.21.15.172:0/3320548280 conn(0x55bf920f3010 msgr2=0x55bf920f19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.020+0000 7f8d0f289740 1 -- 172.21.15.172:0/3320548280 shutdown_connections 2024-09-10T08:49:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.020+0000 7f8d0f289740 1 -- 172.21.15.172:0/3320548280 wait complete. 2024-09-10T08:49:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.020+0000 7f8d0f289740 20 client.16461 trim_cache size 0 max 16384 2024-09-10T08:49:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:49:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:49:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:49:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:49:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.020+0000 7f8d0c417640 1 -- 172.21.15.172:0/2181479572 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8cf00a2c30 msgr2=0x7f8cf00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.020+0000 7f8d0c417640 1 --2- 172.21.15.172:0/2181479572 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8cf00a2c30 0x7f8cf00a50f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005fd0 tx=0x7f8cfc009c30 comp rx=0 tx=0).stop 2024-09-10T08:49:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.020+0000 7f8d0c417640 1 -- 172.21.15.172:0/2181479572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d04100a40 msgr2=0x7f8d041049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.020+0000 7f8d0c417640 1 --2- 172.21.15.172:0/2181479572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d04100a40 0x7f8d041049d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8d000314e0 tx=0x7f8d000042e0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.021+0000 7f8d0c417640 1 -- 172.21.15.172:0/2181479572 shutdown_connections 2024-09-10T08:49:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.021+0000 7f8d0c417640 1 --2- 172.21.15.172:0/2181479572 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8cf00a2c30 0x7f8cf00a50f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.021+0000 7f8d0c417640 1 --2- 172.21.15.172:0/2181479572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d041025f0 0x7f8d04103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.021+0000 7f8d0c417640 1 --2- 172.21.15.172:0/2181479572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d04101c30 0x7f8d04103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.021+0000 7f8d0c417640 1 --2- 172.21.15.172:0/2181479572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d04100a40 0x7f8d041049d0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.021+0000 7f8d0c417640 1 -- 172.21.15.172:0/2181479572 >> 172.21.15.172:0/2181479572 conn(0x7f8d040fc1e0 msgr2=0x7f8d040698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.021+0000 7f8d0c417640 1 -- 172.21.15.172:0/2181479572 shutdown_connections 2024-09-10T08:49:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:36.021+0000 7f8d0c417640 1 -- 172.21.15.172:0/2181479572 wait complete. 2024-09-10T08:49:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:36 smithi172 ceph-mon[30854]: pgmap v451: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 MiB/s wr, 40 op/s 2024-09-10T08:49:36.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:36 smithi174 ceph-mon[34068]: pgmap v451: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 MiB/s wr, 40 op/s 2024-09-10T08:49:36.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:36 smithi191 ceph-mon[35314]: pgmap v451: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 MiB/s wr, 40 op/s 2024-09-10T08:49:38.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:38 smithi172 ceph-mon[30854]: pgmap v452: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 MiB/s wr, 49 op/s 2024-09-10T08:49:38.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:38 smithi174 ceph-mon[34068]: pgmap v452: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 MiB/s wr, 49 op/s 2024-09-10T08:49:38.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:38 smithi191 ceph-mon[35314]: pgmap v452: 129 pgs: 129 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 MiB/s wr, 49 op/s 2024-09-10T08:49:40.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:40 smithi172 ceph-mon[30854]: pgmap v453: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 46 op/s 2024-09-10T08:49:40.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:40 smithi174 ceph-mon[34068]: pgmap v453: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 46 op/s 2024-09-10T08:49:40.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:40 smithi191 ceph-mon[35314]: pgmap v453: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 46 op/s 2024-09-10T08:49:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:42 smithi172 ceph-mon[30854]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 30 MiB/s wr, 52 op/s 2024-09-10T08:49:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:42 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:49:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:42 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:49:42.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:42 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:49:42.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:42 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:49:42.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:42 smithi174 ceph-mon[34068]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 30 MiB/s wr, 52 op/s 2024-09-10T08:49:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:42 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:49:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:42 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:49:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:42 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:49:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:42 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:49:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:42 smithi191 ceph-mon[35314]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 30 MiB/s wr, 52 op/s 2024-09-10T08:49:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:42 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:49:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:42 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:49:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:42 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:49:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:42 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:49:44.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:44 smithi172 ceph-mon[30854]: pgmap v455: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 MiB/s wr, 47 op/s 2024-09-10T08:49:44.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:44 smithi174 ceph-mon[34068]: pgmap v455: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 MiB/s wr, 47 op/s 2024-09-10T08:49:44.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:44 smithi191 ceph-mon[35314]: pgmap v455: 129 pgs: 129 active+clean; 1.6 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 MiB/s wr, 47 op/s 2024-09-10T08:49:45.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:45 smithi174 ceph-mon[34068]: pgmap v456: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 31 MiB/s wr, 53 op/s 2024-09-10T08:49:45.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:45 smithi191 ceph-mon[35314]: pgmap v456: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 31 MiB/s wr, 53 op/s 2024-09-10T08:49:46.173 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:45 smithi172 ceph-mon[30854]: pgmap v456: 129 pgs: 129 active+clean; 1.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 31 MiB/s wr, 53 op/s 2024-09-10T08:49:48.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:48 smithi191 ceph-mon[35314]: pgmap v457: 129 pgs: 129 active+clean; 1.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 MiB/s wr, 37 op/s 2024-09-10T08:49:48.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:48 smithi172 ceph-mon[30854]: pgmap v457: 129 pgs: 129 active+clean; 1.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 MiB/s wr, 37 op/s 2024-09-10T08:49:48.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:48 smithi174 ceph-mon[34068]: pgmap v457: 129 pgs: 129 active+clean; 1.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 MiB/s wr, 37 op/s 2024-09-10T08:49:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.113+0000 7f3e9978d640 1 -- 172.21.15.172:0/4282356002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e94101c30 msgr2=0x7f3e941020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.113+0000 7f3e9978d640 1 --2- 172.21.15.172:0/4282356002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e94101c30 0x7f3e941020b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009900 tx=0x7f3e8802a410 comp rx=0 tx=0).stop 2024-09-10T08:49:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.113+0000 7f3e9978d640 1 -- 172.21.15.172:0/4282356002 shutdown_connections 2024-09-10T08:49:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.113+0000 7f3e9978d640 1 --2- 172.21.15.172:0/4282356002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e941025f0 0x7f3e94108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.113+0000 7f3e9978d640 1 --2- 172.21.15.172:0/4282356002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e94101c30 0x7f3e941020b0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.113+0000 7f3e9978d640 1 --2- 172.21.15.172:0/4282356002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e94100a40 0x7f3e94100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.113+0000 7f3e9978d640 1 -- 172.21.15.172:0/4282356002 >> 172.21.15.172:0/4282356002 conn(0x7f3e940fc1e0 msgr2=0x7f3e940fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.113+0000 7f3e9978d640 1 -- 172.21.15.172:0/4282356002 shutdown_connections 2024-09-10T08:49:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.113+0000 7f3e9978d640 1 -- 172.21.15.172:0/4282356002 wait complete. 2024-09-10T08:49:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.114+0000 7f3e9978d640 1 Processor -- start 2024-09-10T08:49:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.114+0000 7f3e9978d640 1 -- start start 2024-09-10T08:49:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.114+0000 7f3e9978d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e94100a40 0x7f3e94196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.114+0000 7f3e9978d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e94101c30 0x7f3e941971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.114+0000 7f3e9978d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e941025f0 0x7f3e9419e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.114+0000 7f3e9978d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9410c570 con 0x7f3e94100a40 2024-09-10T08:49:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.114+0000 7f3e9978d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9410c3f0 con 0x7f3e941025f0 2024-09-10T08:49:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.114+0000 7f3e9978d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9410c6f0 con 0x7f3e94101c30 2024-09-10T08:49:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.114+0000 7f3e92ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e94100a40 0x7f3e94196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.114+0000 7f3e92ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e94100a40 0x7f3e94196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54460/0 (socket says 172.21.15.172:54460) 2024-09-10T08:49:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.115+0000 7f3e92ffd640 1 -- 172.21.15.172:0/2911573586 learned_addr learned my addr 172.21.15.172:0/2911573586 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.115+0000 7f3e927fc640 1 --2- 172.21.15.172:0/2911573586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e94101c30 0x7f3e941971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.115+0000 7f3e937fe640 1 --2- 172.21.15.172:0/2911573586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e941025f0 0x7f3e9419e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.115+0000 7f3e92ffd640 1 -- 172.21.15.172:0/2911573586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e94101c30 msgr2=0x7f3e941971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.115+0000 7f3e92ffd640 1 --2- 172.21.15.172:0/2911573586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e94101c30 0x7f3e941971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.115+0000 7f3e92ffd640 1 -- 172.21.15.172:0/2911573586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e941025f0 msgr2=0x7f3e9419e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.115+0000 7f3e92ffd640 1 --2- 172.21.15.172:0/2911573586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e941025f0 0x7f3e9419e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.115+0000 7f3e92ffd640 1 -- 172.21.15.172:0/2911573586 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9419e880 con 0x7f3e94100a40 2024-09-10T08:49:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.115+0000 7f3e92ffd640 1 --2- 172.21.15.172:0/2911573586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e94100a40 0x7f3e94196c70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400d950 tx=0x7f3e8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.115+0000 7f3e7bfff640 1 -- 172.21.15.172:0/2911573586 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e84017070 con 0x7f3e94100a40 2024-09-10T08:49:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.115+0000 7f3e7bfff640 1 -- 172.21.15.172:0/2911573586 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e84009c30 con 0x7f3e94100a40 2024-09-10T08:49:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.115+0000 7f3e9978d640 1 -- 172.21.15.172:0/2911573586 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e9419eb70 con 0x7f3e94100a40 2024-09-10T08:49:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.115+0000 7f3e7bfff640 1 -- 172.21.15.172:0/2911573586 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8400f620 con 0x7f3e94100a40 2024-09-10T08:49:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.115+0000 7f3e9978d640 1 -- 172.21.15.172:0/2911573586 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e9419f070 con 0x7f3e94100a40 2024-09-10T08:49:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.118+0000 7f3e7bfff640 1 -- 172.21.15.172:0/2911573586 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3e8401a070 con 0x7f3e94100a40 2024-09-10T08:49:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.118+0000 7f3e9c601740 1 Processor -- start 2024-09-10T08:49:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.118+0000 7f3e9c601740 1 -- start start 2024-09-10T08:49:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.118+0000 7f3e9c601740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec9595bb0 0x55cec9595fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.118+0000 7f3e9c601740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec95964f0 0x55cec959afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.118+0000 7f3e9c601740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec959b580 0x55cec959da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.118+0000 7f3e9c601740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cec94f72f0 con 0x55cec95964f0 2024-09-10T08:49:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.118+0000 7f3e9c601740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cec949f270 con 0x55cec9595bb0 2024-09-10T08:49:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.118+0000 7f3e9c601740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55cec94f32b0 con 0x55cec959b580 2024-09-10T08:49:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.119+0000 7f3e92ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec95964f0 0x55cec959afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.119+0000 7f3e92ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec95964f0 0x55cec959afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54462/0 (socket says 172.21.15.172:54462) 2024-09-10T08:49:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.119+0000 7f3e92ffd640 1 -- 172.21.15.172:0/2867651758 learned_addr learned my addr 172.21.15.172:0/2867651758 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.119+0000 7f3e937fe640 1 --2- 172.21.15.172:0/2867651758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec9595bb0 0x55cec9595fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.119+0000 7f3e7bfff640 1 --2- 172.21.15.172:0/2911573586 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3e7c0a2c30 0x7f3e7c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.119+0000 7f3e927fc640 1 --2- 172.21.15.172:0/2867651758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec959b580 0x55cec959da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.119+0000 7f3e937fe640 1 --2- 172.21.15.172:0/2911573586 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3e7c0a2c30 0x7f3e7c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.119+0000 7f3e7bfff640 1 -- 172.21.15.172:0/2911573586 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3e840cedd0 con 0x7f3e94100a40 2024-09-10T08:49:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.119+0000 7f3e937fe640 1 -- 172.21.15.172:0/2867651758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec959b580 msgr2=0x55cec959da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.119+0000 7f3e937fe640 1 --2- 172.21.15.172:0/2867651758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec959b580 0x55cec959da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.119+0000 7f3e937fe640 1 -- 172.21.15.172:0/2867651758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec95964f0 msgr2=0x55cec959afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.119+0000 7f3e937fe640 1 --2- 172.21.15.172:0/2867651758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec95964f0 0x55cec959afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.119+0000 7f3e937fe640 1 -- 172.21.15.172:0/2867651758 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec949e340 con 0x55cec9595bb0 2024-09-10T08:49:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.119+0000 7f3e927fc640 1 --2- 172.21.15.172:0/2867651758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec959b580 0x55cec959da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.119+0000 7f3e92ffd640 1 --2- 172.21.15.172:0/2867651758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec95964f0 0x55cec959afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.119+0000 7f3e937fe640 1 --2- 172.21.15.172:0/2867651758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec9595bb0 0x55cec9595fb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00a2c0 tx=0x7f3e8c013da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a71bd48d8bd6ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.120+0000 7f3e937fe640 1 --2- 172.21.15.172:0/2911573586 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3e7c0a2c30 0x7f3e7c0a50f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0117b0 tx=0x7f3e8c01e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.120+0000 7f3e797fa640 1 -- 172.21.15.172:0/2867651758 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c013230 con 0x55cec9595bb0 2024-09-10T08:49:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.120+0000 7f3e797fa640 1 -- 172.21.15.172:0/2867651758 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e8c021070 con 0x55cec9595bb0 2024-09-10T08:49:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.120+0000 7f3e797fa640 1 -- 172.21.15.172:0/2867651758 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c012b80 con 0x55cec9595bb0 2024-09-10T08:49:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.120+0000 7f3e9c601740 1 -- 172.21.15.172:0/2867651758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec9595bb0 msgr2=0x55cec9595fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.120+0000 7f3e9c601740 1 --2- 172.21.15.172:0/2867651758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec9595bb0 0x55cec9595fb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00a2c0 tx=0x7f3e8c013da0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e9c601740 1 -- 172.21.15.172:0/2867651758 shutdown_connections 2024-09-10T08:49:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e9c601740 1 --2- 172.21.15.172:0/2867651758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec959b580 0x55cec959da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e9c601740 1 --2- 172.21.15.172:0/2867651758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec95964f0 0x55cec959afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e9c601740 1 --2- 172.21.15.172:0/2867651758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec9595bb0 0x55cec9595fb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e9c601740 1 -- 172.21.15.172:0/2867651758 >> 172.21.15.172:0/2867651758 conn(0x55cec958f010 msgr2=0x55cec958f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e9c601740 1 -- 172.21.15.172:0/2867651758 shutdown_connections 2024-09-10T08:49:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e9c601740 1 -- 172.21.15.172:0/2867651758 wait complete. 2024-09-10T08:49:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e9c601740 1 -- start start 2024-09-10T08:49:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e9c601740 1 Processor -- start 2024-09-10T08:49:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e9c601740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec9595bb0 0x55cec959fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e9c601740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec95964f0 0x55cec95a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e9c601740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec959b580 0x55cec95a7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e9c601740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cec959e4f0 con 0x55cec959b580 2024-09-10T08:49:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e9c601740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cec9480290 con 0x55cec9595bb0 2024-09-10T08:49:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e9c601740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55cec959e670 con 0x55cec95964f0 2024-09-10T08:49:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e9c601740 10 client.0 _ll_register_callbacks cb 0x55cec94b1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e937fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec959b580 0x55cec95a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e937fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec959b580 0x55cec95a7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54468/0 (socket says 172.21.15.172:54468) 2024-09-10T08:49:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e937fe640 1 -- 172.21.15.172:0/2748623361 learned_addr learned my addr 172.21.15.172:0/2748623361 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e92ffd640 1 --2- 172.21.15.172:0/2748623361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec9595bb0 0x55cec959fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.121+0000 7f3e927fc640 1 --2- 172.21.15.172:0/2748623361 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec95964f0 0x55cec95a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.122+0000 7f3e937fe640 1 -- 172.21.15.172:0/2748623361 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec95964f0 msgr2=0x55cec95a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.122+0000 7f3e937fe640 1 --2- 172.21.15.172:0/2748623361 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec95964f0 0x55cec95a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.122+0000 7f3e937fe640 1 -- 172.21.15.172:0/2748623361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec9595bb0 msgr2=0x55cec959fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.122+0000 7f3e937fe640 1 --2- 172.21.15.172:0/2748623361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec9595bb0 0x55cec959fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.122+0000 7f3e937fe640 1 -- 172.21.15.172:0/2748623361 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec920a210 con 0x55cec959b580 2024-09-10T08:49:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.122+0000 7f3e92ffd640 1 --2- 172.21.15.172:0/2748623361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec9595bb0 0x55cec959fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.122+0000 7f3e937fe640 1 --2- 172.21.15.172:0/2748623361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec959b580 0x55cec95a7300 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00ae80 tx=0x7f3e8c01dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fc45b9909da41 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.122+0000 7f3e76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.122+0000 7f3e76ff5640 1 -- 172.21.15.172:0/2748623361 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c029030 con 0x55cec959b580 2024-09-10T08:49:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.122+0000 7f3e76ff5640 1 -- 172.21.15.172:0/2748623361 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3e8c004a00 con 0x55cec959b580 2024-09-10T08:49:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.122+0000 7f3e9c601740 1 -- 172.21.15.172:0/2748623361 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e8c021070 con 0x55cec959b580 2024-09-10T08:49:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.122+0000 7f3e76ff5640 1 -- 172.21.15.172:0/2748623361 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3e8c03e050 con 0x55cec959b580 2024-09-10T08:49:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.122+0000 7f3e9c601740 10 client.16473 fetch_fsmap learned FSMap version 32 2024-09-10T08:49:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.122+0000 7f3e9c601740 1 -- 172.21.15.172:0/2748623361 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cec92d6620 con 0x55cec959b580 2024-09-10T08:49:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.123+0000 7f3e76ff5640 1 -- 172.21.15.172:0/2748623361 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e8c021070 con 0x55cec959b580 2024-09-10T08:49:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.123+0000 7f3e76ff5640 1 -- 172.21.15.172:0/2748623361 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f3e8c02f070 con 0x55cec959b580 2024-09-10T08:49:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.123+0000 7f3e9c601740 10 client.16473 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T08:49:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.123+0000 7f3e9c601740 10 client.16473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:49:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.123+0000 7f3e9c601740 20 client.16473 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.123+0000 7f3e9c601740 1 --2- 172.21.15.172:0/2748623361 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cec95ab780 0x55cec95adb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.123+0000 7f3e9c601740 4 client.16473 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:49:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.123+0000 7f3e9c601740 1 -- 172.21.15.172:0/2748623361 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cec9504a20 con 0x55cec95ab780 2024-09-10T08:49:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.123+0000 7f3e92ffd640 1 --2- 172.21.15.172:0/2748623361 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cec95ab780 0x55cec95adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.123+0000 7f3e92ffd640 1 --2- 172.21.15.172:0/2748623361 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cec95ab780 0x55cec95adb70 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5fa43c16b070669 server_cookie=83265cfb8aade545 in_seq=0 out_seq=0 2024-09-10T08:49:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.123+0000 7f3e76ff5640 10 client.16473 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.124+0000 7f3e76ff5640 1 -- 172.21.15.172:0/2748623361 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cec9504a20 con 0x55cec95ab780 2024-09-10T08:49:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.124+0000 7f3e76ff5640 10 client.16473 handle_command_reply: tid=0 2024-09-10T08:49:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.124+0000 7f3e9c601740 1 client.16473 shutdown 2024-09-10T08:49:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.124+0000 7f3e9c601740 1 -- 172.21.15.172:0/2748623361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec959b580 msgr2=0x55cec95a7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.124+0000 7f3e9c601740 1 --2- 172.21.15.172:0/2748623361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec959b580 0x55cec95a7300 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00ae80 tx=0x7f3e8c01dbb0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.124+0000 7f3e9c601740 1 -- 172.21.15.172:0/2748623361 shutdown_connections 2024-09-10T08:49:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.124+0000 7f3e9c601740 1 --2- 172.21.15.172:0/2748623361 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cec95ab780 0x55cec95adb70 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.124+0000 7f3e9c601740 1 --2- 172.21.15.172:0/2748623361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec959b580 0x55cec95a7300 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.124+0000 7f3e9c601740 1 --2- 172.21.15.172:0/2748623361 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec95964f0 0x55cec95a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.124+0000 7f3e9c601740 1 --2- 172.21.15.172:0/2748623361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec9595bb0 0x55cec959fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.124+0000 7f3e9c601740 1 -- 172.21.15.172:0/2748623361 >> 172.21.15.172:0/2748623361 conn(0x55cec958f010 msgr2=0x55cec9590820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.124+0000 7f3e76ff5640 0 client.16473 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.124+0000 7f3e9c601740 1 -- 172.21.15.172:0/2748623361 shutdown_connections 2024-09-10T08:49:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.124+0000 7f3e9c601740 1 -- 172.21.15.172:0/2748623361 wait complete. 2024-09-10T08:49:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.124+0000 7f3e9c601740 20 client.16473 trim_cache size 0 max 16384 2024-09-10T08:49:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e9c601740 1 Processor -- start 2024-09-10T08:49:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e9c601740 1 -- start start 2024-09-10T08:49:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e9c601740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec9595bb0 0x55cec958ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e9c601740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec95964f0 0x55cec9590460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e9c601740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec959b580 0x55cec95909a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e9c601740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3e941a82c0 con 0x55cec95964f0 2024-09-10T08:49:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e9c601740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3e941a9e90 con 0x55cec959b580 2024-09-10T08:49:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e9c601740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3e941a8990 con 0x55cec9595bb0 2024-09-10T08:49:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e927fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec95964f0 0x55cec9590460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e927fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec95964f0 0x55cec9590460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54476/0 (socket says 172.21.15.172:54476) 2024-09-10T08:49:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e927fc640 1 -- 172.21.15.172:0/3725911933 learned_addr learned my addr 172.21.15.172:0/3725911933 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e937fe640 1 --2- 172.21.15.172:0/3725911933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec959b580 0x55cec95909a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e92ffd640 1 --2- 172.21.15.172:0/3725911933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec9595bb0 0x55cec958ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e927fc640 1 -- 172.21.15.172:0/3725911933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec9595bb0 msgr2=0x55cec958ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e927fc640 1 --2- 172.21.15.172:0/3725911933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec9595bb0 0x55cec958ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e927fc640 1 -- 172.21.15.172:0/3725911933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec959b580 msgr2=0x55cec95909a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e927fc640 1 --2- 172.21.15.172:0/3725911933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec959b580 0x55cec95909a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e927fc640 1 -- 172.21.15.172:0/3725911933 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec959f8c0 con 0x55cec95964f0 2024-09-10T08:49:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e937fe640 1 --2- 172.21.15.172:0/3725911933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec959b580 0x55cec95909a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:49:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.141+0000 7f3e92ffd640 1 --2- 172.21.15.172:0/3725911933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec9595bb0 0x55cec958ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:49:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.142+0000 7f3e927fc640 1 --2- 172.21.15.172:0/3725911933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec95964f0 0x55cec9590460 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009900 tx=0x7f3e88031f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acfb407292b54d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.142+0000 7f3e78ff9640 1 -- 172.21.15.172:0/3725911933 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e88035040 con 0x55cec95964f0 2024-09-10T08:49:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.142+0000 7f3e78ff9640 1 -- 172.21.15.172:0/3725911933 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e880316c0 con 0x55cec95964f0 2024-09-10T08:49:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.142+0000 7f3e78ff9640 1 -- 172.21.15.172:0/3725911933 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e88003e10 con 0x55cec95964f0 2024-09-10T08:49:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.143+0000 7f3e9c601740 1 -- 172.21.15.172:0/3725911933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec95964f0 msgr2=0x55cec9590460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.143+0000 7f3e9c601740 1 --2- 172.21.15.172:0/3725911933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec95964f0 0x55cec9590460 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3e88009900 tx=0x7f3e88031f40 comp rx=0 tx=0).stop 2024-09-10T08:49:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.143+0000 7f3e9c601740 1 -- 172.21.15.172:0/3725911933 shutdown_connections 2024-09-10T08:49:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.143+0000 7f3e9c601740 1 --2- 172.21.15.172:0/3725911933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec959b580 0x55cec95909a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.143+0000 7f3e9c601740 1 --2- 172.21.15.172:0/3725911933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec95964f0 0x55cec9590460 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.143+0000 7f3e9c601740 1 --2- 172.21.15.172:0/3725911933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec9595bb0 0x55cec958ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.143+0000 7f3e9c601740 1 -- 172.21.15.172:0/3725911933 >> 172.21.15.172:0/3725911933 conn(0x55cec958f010 msgr2=0x55cec959fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.143+0000 7f3e9c601740 1 -- 172.21.15.172:0/3725911933 shutdown_connections 2024-09-10T08:49:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.143+0000 7f3e9c601740 1 -- 172.21.15.172:0/3725911933 wait complete. 2024-09-10T08:49:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.143+0000 7f3e9c601740 1 -- start start 2024-09-10T08:49:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.143+0000 7f3e9c601740 1 Processor -- start 2024-09-10T08:49:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.143+0000 7f3e9c601740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec9595bb0 0x55cec9593f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.143+0000 7f3e9c601740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec95964f0 0x55cec95944d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.143+0000 7f3e9c601740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec959b580 0x55cec9616450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.143+0000 7f3e9c601740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3e941a9730 con 0x55cec959b580 2024-09-10T08:49:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.143+0000 7f3e9c601740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3e941a9060 con 0x55cec95964f0 2024-09-10T08:49:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.143+0000 7f3e9c601740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3e941a62a0 con 0x55cec9595bb0 2024-09-10T08:49:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e9c601740 10 client.0 _ll_register_callbacks cb 0x7f3e84097610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:49:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e937fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec959b580 0x55cec9616450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e937fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec959b580 0x55cec9616450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54490/0 (socket says 172.21.15.172:54490) 2024-09-10T08:49:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e937fe640 1 -- 172.21.15.172:0/3349948823 learned_addr learned my addr 172.21.15.172:0/3349948823 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:49:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e927fc640 1 --2- 172.21.15.172:0/3349948823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec95964f0 0x55cec95944d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e92ffd640 1 --2- 172.21.15.172:0/3349948823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec9595bb0 0x55cec9593f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e937fe640 1 -- 172.21.15.172:0/3349948823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec9595bb0 msgr2=0x55cec9593f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e937fe640 1 --2- 172.21.15.172:0/3349948823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec9595bb0 0x55cec9593f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e937fe640 1 -- 172.21.15.172:0/3349948823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec95964f0 msgr2=0x55cec95944d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:49:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e937fe640 1 --2- 172.21.15.172:0/3349948823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec95964f0 0x55cec95944d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e937fe640 1 -- 172.21.15.172:0/3349948823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec9616990 con 0x55cec959b580 2024-09-10T08:49:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e927fc640 1 --2- 172.21.15.172:0/3349948823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec95964f0 0x55cec95944d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:49:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e937fe640 1 --2- 172.21.15.172:0/3349948823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec959b580 0x55cec9616450 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c01df50 tx=0x7f3e8c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07763a2fa81a642 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:49:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:49:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e76ff5640 1 -- 172.21.15.172:0/3349948823 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c046020 con 0x55cec959b580 2024-09-10T08:49:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e76ff5640 1 -- 172.21.15.172:0/3349948823 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3e8c004c00 con 0x55cec959b580 2024-09-10T08:49:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e9c601740 1 -- 172.21.15.172:0/3349948823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e880316c0 con 0x55cec959b580 2024-09-10T08:49:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e76ff5640 1 -- 172.21.15.172:0/3349948823 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3e8c029070 con 0x55cec959b580 2024-09-10T08:49:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e9c601740 10 client.16479 fetch_fsmap learned FSMap version 32 2024-09-10T08:49:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.144+0000 7f3e9c601740 1 -- 172.21.15.172:0/3349948823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cec9617b50 con 0x55cec959b580 2024-09-10T08:49:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.145+0000 7f3e76ff5640 1 -- 172.21.15.172:0/3349948823 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e880316c0 con 0x55cec959b580 2024-09-10T08:49:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.145+0000 7f3e76ff5640 1 -- 172.21.15.172:0/3349948823 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f3e8c02f040 con 0x55cec959b580 2024-09-10T08:49:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.145+0000 7f3e9c601740 10 client.16479 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T08:49:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.145+0000 7f3e9c601740 10 client.16479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:49:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.145+0000 7f3e9c601740 20 client.16479 populate_metadata read hostname 'smithi172' 2024-09-10T08:49:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.145+0000 7f3e9c601740 1 --2- 172.21.15.172:0/3349948823 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cec96185e0 0x55cec961aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:49:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.145+0000 7f3e9c601740 4 client.16479 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:49:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.145+0000 7f3e9c601740 1 -- 172.21.15.172:0/3349948823 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cec94fed70 con 0x55cec96185e0 2024-09-10T08:49:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.145+0000 7f3e92ffd640 1 --2- 172.21.15.172:0/3349948823 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cec96185e0 0x55cec961aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:49:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.145+0000 7f3e76ff5640 10 client.16479 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.145+0000 7f3e92ffd640 1 --2- 172.21.15.172:0/3349948823 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cec96185e0 0x55cec961aa00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb84c169235bb289 server_cookie=ef621fd268a9b208 in_seq=0 out_seq=0 2024-09-10T08:49:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.153+0000 7f3e76ff5640 1 -- 172.21.15.172:0/3349948823 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cec94fed70 con 0x55cec96185e0 2024-09-10T08:49:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.153+0000 7f3e76ff5640 10 client.16479 handle_command_reply: tid=0 2024-09-10T08:49:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.153+0000 7f3e9c601740 1 client.16479 shutdown 2024-09-10T08:49:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.153+0000 7f3e9c601740 1 -- 172.21.15.172:0/3349948823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec959b580 msgr2=0x55cec9616450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.153+0000 7f3e9c601740 1 --2- 172.21.15.172:0/3349948823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec959b580 0x55cec9616450 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c01df50 tx=0x7f3e8c00a5c0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.153+0000 7f3e9c601740 1 -- 172.21.15.172:0/3349948823 shutdown_connections 2024-09-10T08:49:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.153+0000 7f3e9c601740 1 --2- 172.21.15.172:0/3349948823 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cec96185e0 0x55cec961aa00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.153+0000 7f3e9c601740 1 --2- 172.21.15.172:0/3349948823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cec959b580 0x55cec9616450 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.153+0000 7f3e9c601740 1 --2- 172.21.15.172:0/3349948823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec95964f0 0x55cec95944d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.153+0000 7f3e9c601740 1 --2- 172.21.15.172:0/3349948823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cec9595bb0 0x55cec9593f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.153+0000 7f3e9c601740 1 -- 172.21.15.172:0/3349948823 >> 172.21.15.172:0/3349948823 conn(0x55cec958f010 msgr2=0x55cec959fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.153+0000 7f3e76ff5640 0 client.16479 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:49:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.153+0000 7f3e9c601740 1 -- 172.21.15.172:0/3349948823 shutdown_connections 2024-09-10T08:49:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.153+0000 7f3e9c601740 1 -- 172.21.15.172:0/3349948823 wait complete. 2024-09-10T08:49:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.153+0000 7f3e9c601740 20 client.16479 trim_cache size 0 max 16384 2024-09-10T08:49:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:49:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:49:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:49:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:49:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.154+0000 7f3e9978d640 1 -- 172.21.15.172:0/2911573586 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3e7c0a2c30 msgr2=0x7f3e7c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.154+0000 7f3e9978d640 1 --2- 172.21.15.172:0/2911573586 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3e7c0a2c30 0x7f3e7c0a50f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0117b0 tx=0x7f3e8c01e000 comp rx=0 tx=0).stop 2024-09-10T08:49:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.154+0000 7f3e9978d640 1 -- 172.21.15.172:0/2911573586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e94100a40 msgr2=0x7f3e94196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:49:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.154+0000 7f3e9978d640 1 --2- 172.21.15.172:0/2911573586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e94100a40 0x7f3e94196c70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400d950 tx=0x7f3e8400de10 comp rx=0 tx=0).stop 2024-09-10T08:49:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.154+0000 7f3e9978d640 1 -- 172.21.15.172:0/2911573586 shutdown_connections 2024-09-10T08:49:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.154+0000 7f3e9978d640 1 --2- 172.21.15.172:0/2911573586 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3e7c0a2c30 0x7f3e7c0a50f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.154+0000 7f3e9978d640 1 --2- 172.21.15.172:0/2911573586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e941025f0 0x7f3e9419e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.154+0000 7f3e9978d640 1 --2- 172.21.15.172:0/2911573586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e94101c30 0x7f3e941971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.154+0000 7f3e9978d640 1 --2- 172.21.15.172:0/2911573586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e94100a40 0x7f3e94196c70 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:49:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.154+0000 7f3e9978d640 1 -- 172.21.15.172:0/2911573586 >> 172.21.15.172:0/2911573586 conn(0x7f3e940fc1e0 msgr2=0x7f3e94104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:49:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.154+0000 7f3e9978d640 1 -- 172.21.15.172:0/2911573586 shutdown_connections 2024-09-10T08:49:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:49:49.154+0000 7f3e9978d640 1 -- 172.21.15.172:0/2911573586 wait complete. 2024-09-10T08:49:50.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:50 smithi191 ceph-mon[35314]: pgmap v458: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-09-10T08:49:50.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:50 smithi172 ceph-mon[30854]: pgmap v458: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-09-10T08:49:50.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:50 smithi174 ceph-mon[34068]: pgmap v458: 129 pgs: 129 active+clean; 1.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-09-10T08:49:52.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:52 smithi191 ceph-mon[35314]: pgmap v459: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-09-10T08:49:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:52 smithi172 ceph-mon[30854]: pgmap v459: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-09-10T08:49:52.745 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:52 smithi174 ceph-mon[34068]: pgmap v459: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-09-10T08:49:54.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:54 smithi191 ceph-mon[35314]: pgmap v460: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:49:54.585 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:54 smithi172 ceph-mon[30854]: pgmap v460: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:49:54.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:54 smithi174 ceph-mon[34068]: pgmap v460: 129 pgs: 129 active+clean; 1.8 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:49:56.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:56 smithi191 ceph-mon[35314]: pgmap v461: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 27 op/s 2024-09-10T08:49:56.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:56 smithi172 ceph-mon[30854]: pgmap v461: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 27 op/s 2024-09-10T08:49:56.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:56 smithi174 ceph-mon[34068]: pgmap v461: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 31 MiB/s wr, 27 op/s 2024-09-10T08:49:58.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:49:58 smithi191 ceph-mon[35314]: pgmap v462: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 24 op/s 2024-09-10T08:49:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:49:58 smithi172 ceph-mon[30854]: pgmap v462: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 24 op/s 2024-09-10T08:49:58.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:49:58 smithi174 ceph-mon[34068]: pgmap v462: 129 pgs: 129 active+clean; 2.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 24 op/s 2024-09-10T08:50:00.544 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:00 smithi174 ceph-mon[34068]: pgmap v463: 129 pgs: 129 active+clean; 2.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 22 op/s 2024-09-10T08:50:00.544 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:00 smithi174 ceph-mon[34068]: overall HEALTH_OK 2024-09-10T08:50:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:00 smithi191 ceph-mon[35314]: pgmap v463: 129 pgs: 129 active+clean; 2.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 22 op/s 2024-09-10T08:50:00.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:00 smithi191 ceph-mon[35314]: overall HEALTH_OK 2024-09-10T08:50:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:00 smithi172 ceph-mon[30854]: pgmap v463: 129 pgs: 129 active+clean; 2.0 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 22 op/s 2024-09-10T08:50:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:00 smithi172 ceph-mon[30854]: overall HEALTH_OK 2024-09-10T08:50:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.252+0000 7f6d77be8640 1 -- 172.21.15.172:0/2200078085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d70100a40 msgr2=0x7f6d70100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.252+0000 7f6d77be8640 1 --2- 172.21.15.172:0/2200078085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d70100a40 0x7f6d70100e40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009900 tx=0x7f6d6c02a410 comp rx=0 tx=0).stop 2024-09-10T08:50:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.252+0000 7f6d77be8640 1 -- 172.21.15.172:0/2200078085 shutdown_connections 2024-09-10T08:50:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.252+0000 7f6d77be8640 1 --2- 172.21.15.172:0/2200078085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d701025f0 0x7f6d70108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.252+0000 7f6d77be8640 1 --2- 172.21.15.172:0/2200078085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d70101c30 0x7f6d701020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.252+0000 7f6d77be8640 1 --2- 172.21.15.172:0/2200078085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d70100a40 0x7f6d70100e40 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.252+0000 7f6d77be8640 1 -- 172.21.15.172:0/2200078085 >> 172.21.15.172:0/2200078085 conn(0x7f6d700fc1e0 msgr2=0x7f6d700fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.252+0000 7f6d77be8640 1 -- 172.21.15.172:0/2200078085 shutdown_connections 2024-09-10T08:50:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.252+0000 7f6d77be8640 1 -- 172.21.15.172:0/2200078085 wait complete. 2024-09-10T08:50:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.252+0000 7f6d77be8640 1 Processor -- start 2024-09-10T08:50:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.252+0000 7f6d77be8640 1 -- start start 2024-09-10T08:50:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.252+0000 7f6d77be8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d70100a40 0x7f6d70196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d77be8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d70101c30 0x7f6d70197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d77be8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d701025f0 0x7f6d7019e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d77be8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7010c570 con 0x7f6d70100a40 2024-09-10T08:50:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d77be8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7010c3f0 con 0x7f6d70101c30 2024-09-10T08:50:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d77be8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7010c6f0 con 0x7f6d701025f0 2024-09-10T08:50:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d75984640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d70100a40 0x7f6d70196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d75984640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d70100a40 0x7f6d70196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40900/0 (socket says 172.21.15.172:40900) 2024-09-10T08:50:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d75984640 1 -- 172.21.15.172:0/4158767154 learned_addr learned my addr 172.21.15.172:0/4158767154 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d75183640 1 --2- 172.21.15.172:0/4158767154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d70101c30 0x7f6d70197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d76185640 1 --2- 172.21.15.172:0/4158767154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d701025f0 0x7f6d7019e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d75984640 1 -- 172.21.15.172:0/4158767154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d701025f0 msgr2=0x7f6d7019e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d75984640 1 --2- 172.21.15.172:0/4158767154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d701025f0 0x7f6d7019e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d75984640 1 -- 172.21.15.172:0/4158767154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d70101c30 msgr2=0x7f6d70197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d75984640 1 --2- 172.21.15.172:0/4158767154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d70101c30 0x7f6d70197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d75984640 1 -- 172.21.15.172:0/4158767154 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7019e830 con 0x7f6d70100a40 2024-09-10T08:50:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d75984640 1 --2- 172.21.15.172:0/4158767154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d70100a40 0x7f6d70196c20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0314e0 tx=0x7f6d6c031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d66ffd640 1 -- 172.21.15.172:0/4158767154 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c03f030 con 0x7f6d70100a40 2024-09-10T08:50:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d77be8640 1 -- 172.21.15.172:0/4158767154 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7019eac0 con 0x7f6d70100a40 2024-09-10T08:50:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d66ffd640 1 -- 172.21.15.172:0/4158767154 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d6c0044b0 con 0x7f6d70100a40 2024-09-10T08:50:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d66ffd640 1 -- 172.21.15.172:0/4158767154 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c035070 con 0x7f6d70100a40 2024-09-10T08:50:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.253+0000 7f6d77be8640 1 -- 172.21.15.172:0/4158767154 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d7019f000 con 0x7f6d70100a40 2024-09-10T08:50:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.256+0000 7f6d7aa62740 1 Processor -- start 2024-09-10T08:50:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.256+0000 7f6d66ffd640 1 -- 172.21.15.172:0/4158767154 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6d6c042030 con 0x7f6d70100a40 2024-09-10T08:50:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.256+0000 7f6d7aa62740 1 -- start start 2024-09-10T08:50:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d7aa62740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d47f9a0 0x558e4d47fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d7aa62740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 0x558e4d484da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d7aa62740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d485370 0x558e4d487830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d7aa62740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558e4d3e32f0 con 0x558e4d485370 2024-09-10T08:50:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d7aa62740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e4d38b270 con 0x558e4d4802e0 2024-09-10T08:50:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d7aa62740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558e4d3df2b0 con 0x558e4d47f9a0 2024-09-10T08:50:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d75183640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d485370 0x558e4d487830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d75183640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d485370 0x558e4d487830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40916/0 (socket says 172.21.15.172:40916) 2024-09-10T08:50:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d75183640 1 -- 172.21.15.172:0/3710329957 learned_addr learned my addr 172.21.15.172:0/3710329957 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:02.265 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d75984640 1 --2- 172.21.15.172:0/3710329957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 0x558e4d484da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d76185640 1 --2- 172.21.15.172:0/3710329957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d47f9a0 0x558e4d47fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d75183640 1 -- 172.21.15.172:0/3710329957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d47f9a0 msgr2=0x558e4d47fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d75183640 1 --2- 172.21.15.172:0/3710329957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d47f9a0 0x558e4d47fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d75183640 1 -- 172.21.15.172:0/3710329957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 msgr2=0x558e4d484da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d75183640 1 --2- 172.21.15.172:0/3710329957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 0x558e4d484da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d75183640 1 -- 172.21.15.172:0/3710329957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e4d38a340 con 0x558e4d485370 2024-09-10T08:50:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d66ffd640 1 --2- 172.21.15.172:0/4158767154 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6d5c0a2c30 0x7f6d5c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d75183640 1 --2- 172.21.15.172:0/3710329957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d485370 0x558e4d487830 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000bd10 tx=0x7f6d600107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1a7aba8277a61c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.257+0000 7f6d76185640 1 --2- 172.21.15.172:0/4158767154 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6d5c0a2c30 0x7f6d5c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.258+0000 7f6d66ffd640 1 -- 172.21.15.172:0/4158767154 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6d6c0ef980 con 0x7f6d70100a40 2024-09-10T08:50:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.258+0000 7f6d5bfff640 1 -- 172.21.15.172:0/3710329957 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60004030 con 0x558e4d485370 2024-09-10T08:50:02.266 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.258+0000 7f6d5bfff640 1 -- 172.21.15.172:0/3710329957 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d600035f0 con 0x558e4d485370 2024-09-10T08:50:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.258+0000 7f6d5bfff640 1 -- 172.21.15.172:0/3710329957 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d600117b0 con 0x558e4d485370 2024-09-10T08:50:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.259+0000 7f6d7aa62740 1 -- 172.21.15.172:0/3710329957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d485370 msgr2=0x558e4d487830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.259+0000 7f6d76185640 1 --2- 172.21.15.172:0/4158767154 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6d5c0a2c30 0x7f6d5c0a50f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6d68005fd0 tx=0x7f6d68009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.259+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/3710329957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d485370 0x558e4d487830 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000bd10 tx=0x7f6d600107a0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.259+0000 7f6d7aa62740 1 -- 172.21.15.172:0/3710329957 shutdown_connections 2024-09-10T08:50:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.259+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/3710329957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d485370 0x558e4d487830 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.259+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/3710329957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 0x558e4d484da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.259+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/3710329957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d47f9a0 0x558e4d47fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.259+0000 7f6d7aa62740 1 -- 172.21.15.172:0/3710329957 >> 172.21.15.172:0/3710329957 conn(0x558e4d47b010 msgr2=0x558e4d47b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.259+0000 7f6d7aa62740 1 -- 172.21.15.172:0/3710329957 shutdown_connections 2024-09-10T08:50:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.259+0000 7f6d7aa62740 1 -- 172.21.15.172:0/3710329957 wait complete. 2024-09-10T08:50:02.267 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.259+0000 7f6d7aa62740 1 -- start start 2024-09-10T08:50:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.259+0000 7f6d7aa62740 1 Processor -- start 2024-09-10T08:50:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d7aa62740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d489bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d7aa62740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 0x558e4d48a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d7aa62740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d485370 0x558e4d491140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d7aa62740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558e4d488340 con 0x558e4d47f9a0 2024-09-10T08:50:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d7aa62740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e4d36c290 con 0x558e4d4802e0 2024-09-10T08:50:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d7aa62740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558e4d4884c0 con 0x558e4d485370 2024-09-10T08:50:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d7aa62740 10 client.0 _ll_register_callbacks cb 0x558e4d39d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d75984640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d489bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d75984640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d489bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40928/0 (socket says 172.21.15.172:40928) 2024-09-10T08:50:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d75984640 1 -- 172.21.15.172:0/2785737778 learned_addr learned my addr 172.21.15.172:0/2785737778 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d76185640 1 --2- 172.21.15.172:0/2785737778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d485370 0x558e4d491140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d75183640 1 --2- 172.21.15.172:0/2785737778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 0x558e4d48a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d75984640 1 -- 172.21.15.172:0/2785737778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d485370 msgr2=0x558e4d491140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d75984640 1 --2- 172.21.15.172:0/2785737778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d485370 0x558e4d491140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d75984640 1 -- 172.21.15.172:0/2785737778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 msgr2=0x558e4d48a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d75984640 1 --2- 172.21.15.172:0/2785737778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 0x558e4d48a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d75984640 1 -- 172.21.15.172:0/2785737778 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e4d0f6210 con 0x558e4d47f9a0 2024-09-10T08:50:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d75984640 1 --2- 172.21.15.172:0/2785737778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d489bc0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c042000 tx=0x7f6d6c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf445c0244dab19b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d597fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d597fa640 1 -- 172.21.15.172:0/2785737778 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c049050 con 0x558e4d47f9a0 2024-09-10T08:50:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d597fa640 1 -- 172.21.15.172:0/2785737778 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6d6c0bc2e0 con 0x558e4d47f9a0 2024-09-10T08:50:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.260+0000 7f6d7aa62740 1 -- 172.21.15.172:0/2785737778 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d600035f0 con 0x558e4d47f9a0 2024-09-10T08:50:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.261+0000 7f6d597fa640 1 -- 172.21.15.172:0/2785737778 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6d6c0d8050 con 0x558e4d47f9a0 2024-09-10T08:50:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.261+0000 7f6d7aa62740 10 client.16491 fetch_fsmap learned FSMap version 32 2024-09-10T08:50:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.261+0000 7f6d7aa62740 1 -- 172.21.15.172:0/2785737778 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558e4d1c2620 con 0x558e4d47f9a0 2024-09-10T08:50:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.261+0000 7f6d597fa640 1 -- 172.21.15.172:0/2785737778 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d600035f0 con 0x558e4d47f9a0 2024-09-10T08:50:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.261+0000 7f6d597fa640 1 -- 172.21.15.172:0/2785737778 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f6d6c0c4660 con 0x558e4d47f9a0 2024-09-10T08:50:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.261+0000 7f6d7aa62740 10 client.16491 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T08:50:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.261+0000 7f6d7aa62740 10 client.16491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.261+0000 7f6d7aa62740 20 client.16491 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.261+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/2785737778 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558e4d495920 0x558e4d497dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.261+0000 7f6d7aa62740 4 client.16491 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:50:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.261+0000 7f6d7aa62740 1 -- 172.21.15.172:0/2785737778 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e4d3f0a20 con 0x558e4d495920 2024-09-10T08:50:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.261+0000 7f6d75183640 1 --2- 172.21.15.172:0/2785737778 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558e4d495920 0x558e4d497dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.262+0000 7f6d597fa640 10 client.16491 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.262+0000 7f6d75183640 1 --2- 172.21.15.172:0/2785737778 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558e4d495920 0x558e4d497dc0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1524125f91d48571 server_cookie=4fdb3783c2c7f0ab in_seq=0 out_seq=0 2024-09-10T08:50:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.263+0000 7f6d597fa640 1 -- 172.21.15.172:0/2785737778 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e4d3f0a20 con 0x558e4d495920 2024-09-10T08:50:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.263+0000 7f6d597fa640 10 client.16491 handle_command_reply: tid=0 2024-09-10T08:50:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.263+0000 7f6d7aa62740 1 client.16491 shutdown 2024-09-10T08:50:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.263+0000 7f6d7aa62740 1 -- 172.21.15.172:0/2785737778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 msgr2=0x558e4d489bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.263+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/2785737778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d489bc0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c042000 tx=0x7f6d6c009620 comp rx=0 tx=0).stop 2024-09-10T08:50:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.263+0000 7f6d7aa62740 1 -- 172.21.15.172:0/2785737778 shutdown_connections 2024-09-10T08:50:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.263+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/2785737778 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558e4d495920 0x558e4d497dc0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.263+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/2785737778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d485370 0x558e4d491140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.263+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/2785737778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 0x558e4d48a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.263+0000 7f6d597fa640 0 client.16491 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.263+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/2785737778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d489bc0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.263+0000 7f6d7aa62740 1 -- 172.21.15.172:0/2785737778 >> 172.21.15.172:0/2785737778 conn(0x558e4d47b010 msgr2=0x558e4d47c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.263+0000 7f6d7aa62740 1 -- 172.21.15.172:0/2785737778 shutdown_connections 2024-09-10T08:50:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.263+0000 7f6d7aa62740 1 -- 172.21.15.172:0/2785737778 wait complete. 2024-09-10T08:50:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.263+0000 7f6d7aa62740 20 client.16491 trim_cache size 0 max 16384 2024-09-10T08:50:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d7aa62740 1 Processor -- start 2024-09-10T08:50:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d7aa62740 1 -- start start 2024-09-10T08:50:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d7aa62740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d48a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d7aa62740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 0x558e4d491100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d7aa62740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d485370 0x558e4d47a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d7aa62740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701a81f0 con 0x558e4d47f9a0 2024-09-10T08:50:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d7aa62740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701a9dc0 con 0x558e4d4802e0 2024-09-10T08:50:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d7aa62740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701a88c0 con 0x558e4d485370 2024-09-10T08:50:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d75984640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d48a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d75984640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d48a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40930/0 (socket says 172.21.15.172:40930) 2024-09-10T08:50:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d75984640 1 -- 172.21.15.172:0/2115072547 learned_addr learned my addr 172.21.15.172:0/2115072547 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d76185640 1 --2- 172.21.15.172:0/2115072547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d485370 0x558e4d47a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d75183640 1 --2- 172.21.15.172:0/2115072547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 0x558e4d491100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d75984640 1 -- 172.21.15.172:0/2115072547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d485370 msgr2=0x558e4d47a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d75984640 1 --2- 172.21.15.172:0/2115072547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d485370 0x558e4d47a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d75984640 1 -- 172.21.15.172:0/2115072547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 msgr2=0x558e4d491100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:02.285 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d75984640 1 --2- 172.21.15.172:0/2115072547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 0x558e4d491100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d75984640 1 -- 172.21.15.172:0/2115072547 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e4d4807c0 con 0x558e4d47f9a0 2024-09-10T08:50:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.283+0000 7f6d75183640 1 --2- 172.21.15.172:0/2115072547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 0x558e4d491100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:02.286 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.284+0000 7f6d75984640 1 --2- 172.21.15.172:0/2115072547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d48a190 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0f51b0 tx=0x7f6d6c0c4a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5706b8099852fa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.287+0000 7f6d5b7fe640 1 -- 172.21.15.172:0/2115072547 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c0a0070 con 0x558e4d47f9a0 2024-09-10T08:50:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.287+0000 7f6d5b7fe640 1 -- 172.21.15.172:0/2115072547 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d6c003e20 con 0x558e4d47f9a0 2024-09-10T08:50:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.287+0000 7f6d5b7fe640 1 -- 172.21.15.172:0/2115072547 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c049050 con 0x558e4d47f9a0 2024-09-10T08:50:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.288+0000 7f6d7aa62740 1 -- 172.21.15.172:0/2115072547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 msgr2=0x558e4d48a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.288+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/2115072547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d48a190 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0f51b0 tx=0x7f6d6c0c4a20 comp rx=0 tx=0).stop 2024-09-10T08:50:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.288+0000 7f6d7aa62740 1 -- 172.21.15.172:0/2115072547 shutdown_connections 2024-09-10T08:50:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.288+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/2115072547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d485370 0x558e4d47a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.288+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/2115072547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 0x558e4d491100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.288+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/2115072547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d48a190 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.288+0000 7f6d7aa62740 1 -- 172.21.15.172:0/2115072547 >> 172.21.15.172:0/2115072547 conn(0x558e4d47b010 msgr2=0x558e4d4799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.288+0000 7f6d7aa62740 1 -- 172.21.15.172:0/2115072547 shutdown_connections 2024-09-10T08:50:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.288+0000 7f6d7aa62740 1 -- 172.21.15.172:0/2115072547 wait complete. 2024-09-10T08:50:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.288+0000 7f6d7aa62740 1 -- start start 2024-09-10T08:50:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d7aa62740 1 Processor -- start 2024-09-10T08:50:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d7aa62740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d48e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d7aa62740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 0x558e4d48eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d7aa62740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d485370 0x558e4d48fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d7aa62740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701a9660 con 0x558e4d47f9a0 2024-09-10T08:50:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d7aa62740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701a8f90 con 0x558e4d4802e0 2024-09-10T08:50:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d7aa62740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6d701a61d0 con 0x558e4d485370 2024-09-10T08:50:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d7aa62740 10 client.0 _ll_register_callbacks cb 0x7f6d6c0f5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d75984640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d48e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d75984640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d48e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40936/0 (socket says 172.21.15.172:40936) 2024-09-10T08:50:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d75984640 1 -- 172.21.15.172:0/3143568328 learned_addr learned my addr 172.21.15.172:0/3143568328 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d75183640 1 --2- 172.21.15.172:0/3143568328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 0x558e4d48eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d76185640 1 --2- 172.21.15.172:0/3143568328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d485370 0x558e4d48fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d75984640 1 -- 172.21.15.172:0/3143568328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d485370 msgr2=0x558e4d48fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d75984640 1 --2- 172.21.15.172:0/3143568328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d485370 0x558e4d48fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d75984640 1 -- 172.21.15.172:0/3143568328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 msgr2=0x558e4d48eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d75984640 1 --2- 172.21.15.172:0/3143568328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 0x558e4d48eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.289+0000 7f6d75984640 1 -- 172.21.15.172:0/3143568328 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e4d490170 con 0x558e4d47f9a0 2024-09-10T08:50:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.290+0000 7f6d75984640 1 --2- 172.21.15.172:0/3143568328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d48e610 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c03d040 tx=0x7f6d6c0c4890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bfd98c55c8d49c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.290+0000 7f6d597fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.290+0000 7f6d597fa640 1 -- 172.21.15.172:0/3143568328 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c049070 con 0x558e4d47f9a0 2024-09-10T08:50:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.290+0000 7f6d597fa640 1 -- 172.21.15.172:0/3143568328 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6d6c0b7b40 con 0x558e4d47f9a0 2024-09-10T08:50:02.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.290+0000 7f6d7aa62740 1 -- 172.21.15.172:0/3143568328 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558e4d3ead70 con 0x558e4d47f9a0 2024-09-10T08:50:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.290+0000 7f6d597fa640 1 -- 172.21.15.172:0/3143568328 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6d6c0a0070 con 0x558e4d47f9a0 2024-09-10T08:50:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.290+0000 7f6d7aa62740 10 client.16497 fetch_fsmap learned FSMap version 32 2024-09-10T08:50:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.290+0000 7f6d7aa62740 1 -- 172.21.15.172:0/3143568328 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558e4d490170 con 0x558e4d47f9a0 2024-09-10T08:50:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.290+0000 7f6d597fa640 1 -- 172.21.15.172:0/3143568328 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x558e4d3ead70 con 0x558e4d47f9a0 2024-09-10T08:50:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.290+0000 7f6d597fa640 1 -- 172.21.15.172:0/3143568328 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 32) ==== 8382+0+0 (secure 0 0 0) 0x7f6d6c0daac0 con 0x558e4d47f9a0 2024-09-10T08:50:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.290+0000 7f6d7aa62740 10 client.16497 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T08:50:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.290+0000 7f6d7aa62740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.290+0000 7f6d7aa62740 20 client.16497 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.290+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/3143568328 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558e4d500b90 0x558e4d502fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.290+0000 7f6d7aa62740 4 client.16497 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:50:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.290+0000 7f6d7aa62740 1 -- 172.21.15.172:0/3143568328 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e4d367900 con 0x558e4d500b90 2024-09-10T08:50:02.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.291+0000 7f6d75183640 1 --2- 172.21.15.172:0/3143568328 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558e4d500b90 0x558e4d502fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.291+0000 7f6d75183640 1 --2- 172.21.15.172:0/3143568328 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558e4d500b90 0x558e4d502fb0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eb520d5302a365f server_cookie=e741197328daf396 in_seq=0 out_seq=0 2024-09-10T08:50:02.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.291+0000 7f6d597fa640 10 client.16497 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.297+0000 7f6d597fa640 1 -- 172.21.15.172:0/3143568328 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558e4d367900 con 0x558e4d500b90 2024-09-10T08:50:02.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.297+0000 7f6d597fa640 10 client.16497 handle_command_reply: tid=0 2024-09-10T08:50:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.297+0000 7f6d7aa62740 1 client.16497 shutdown 2024-09-10T08:50:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.297+0000 7f6d7aa62740 1 -- 172.21.15.172:0/3143568328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 msgr2=0x558e4d48e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.297+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/3143568328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d48e610 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c03d040 tx=0x7f6d6c0c4890 comp rx=0 tx=0).stop 2024-09-10T08:50:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d7aa62740 1 -- 172.21.15.172:0/3143568328 shutdown_connections 2024-09-10T08:50:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/3143568328 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558e4d500b90 0x558e4d502fb0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/3143568328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e4d485370 0x558e4d48fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/3143568328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e4d4802e0 0x558e4d48eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d7aa62740 1 --2- 172.21.15.172:0/3143568328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e4d47f9a0 0x558e4d48e610 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d597fa640 0 client.16497 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d7aa62740 1 -- 172.21.15.172:0/3143568328 >> 172.21.15.172:0/3143568328 conn(0x558e4d47b010 msgr2=0x558e4d4799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d7aa62740 1 -- 172.21.15.172:0/3143568328 shutdown_connections 2024-09-10T08:50:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d7aa62740 1 -- 172.21.15.172:0/3143568328 wait complete. 2024-09-10T08:50:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d7aa62740 20 client.16497 trim_cache size 0 max 16384 2024-09-10T08:50:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:50:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:50:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:50:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:50:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d77be8640 1 -- 172.21.15.172:0/4158767154 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6d5c0a2c30 msgr2=0x7f6d5c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d77be8640 1 --2- 172.21.15.172:0/4158767154 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6d5c0a2c30 0x7f6d5c0a50f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6d68005fd0 tx=0x7f6d68009500 comp rx=0 tx=0).stop 2024-09-10T08:50:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d77be8640 1 -- 172.21.15.172:0/4158767154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d70100a40 msgr2=0x7f6d70196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d77be8640 1 --2- 172.21.15.172:0/4158767154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d70100a40 0x7f6d70196c20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0314e0 tx=0x7f6d6c031a30 comp rx=0 tx=0).stop 2024-09-10T08:50:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d77be8640 1 -- 172.21.15.172:0/4158767154 shutdown_connections 2024-09-10T08:50:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d77be8640 1 --2- 172.21.15.172:0/4158767154 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6d5c0a2c30 0x7f6d5c0a50f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d77be8640 1 --2- 172.21.15.172:0/4158767154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d701025f0 0x7f6d7019e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d77be8640 1 --2- 172.21.15.172:0/4158767154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d70101c30 0x7f6d70197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d77be8640 1 --2- 172.21.15.172:0/4158767154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d70100a40 0x7f6d70196c20 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d77be8640 1 -- 172.21.15.172:0/4158767154 >> 172.21.15.172:0/4158767154 conn(0x7f6d700fc1e0 msgr2=0x7f6d70104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.298+0000 7f6d77be8640 1 -- 172.21.15.172:0/4158767154 shutdown_connections 2024-09-10T08:50:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:02.299+0000 7f6d77be8640 1 -- 172.21.15.172:0/4158767154 wait complete. 2024-09-10T08:50:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:02 smithi191 ceph-mon[35314]: pgmap v464: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-09-10T08:50:02.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:02 smithi172 ceph-mon[30854]: pgmap v464: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-09-10T08:50:02.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:02 smithi174 ceph-mon[34068]: pgmap v464: 129 pgs: 129 active+clean; 2.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-09-10T08:50:04.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:04 smithi191 ceph-mon[35314]: pgmap v465: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-09-10T08:50:04.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:04 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:50:04.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:04 smithi172 ceph-mon[30854]: pgmap v465: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-09-10T08:50:04.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:04 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:50:04.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:04 smithi174 ceph-mon[34068]: pgmap v465: 129 pgs: 129 active+clean; 2.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-09-10T08:50:04.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:04 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:50:05.006 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:50:05.293 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:50:06.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:06 smithi191 ceph-mon[35314]: pgmap v466: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-09-10T08:50:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:06 smithi172 ceph-mon[30854]: pgmap v466: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-09-10T08:50:06.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:06 smithi174 ceph-mon[34068]: pgmap v466: 129 pgs: 129 active+clean; 2.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-09-10T08:50:08.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:08 smithi191 ceph-mon[35314]: pgmap v467: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 31 MiB/s wr, 24 op/s 2024-09-10T08:50:08.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:08 smithi172 ceph-mon[30854]: pgmap v467: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 31 MiB/s wr, 24 op/s 2024-09-10T08:50:08.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:08 smithi174 ceph-mon[34068]: pgmap v467: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 31 MiB/s wr, 24 op/s 2024-09-10T08:50:09.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.689+0000 7f2688033640 1 -- 172.21.15.172:0/1381167419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2680101470 msgr2=0x7f2680101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:09.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.689+0000 7f2688033640 1 --2- 172.21.15.172:0/1381167419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2680101470 0x7f2680101850 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f267c009900 tx=0x7f267c02a410 comp rx=0 tx=0).stop 2024-09-10T08:50:09.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.689+0000 7f2688033640 1 -- 172.21.15.172:0/1381167419 shutdown_connections 2024-09-10T08:50:09.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.689+0000 7f2688033640 1 --2- 172.21.15.172:0/1381167419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2680105de0 0x7f268010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.689+0000 7f2688033640 1 --2- 172.21.15.172:0/1381167419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2680101e20 0x7f26801056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.689+0000 7f2688033640 1 --2- 172.21.15.172:0/1381167419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2680101470 0x7f2680101850 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.689+0000 7f2688033640 1 -- 172.21.15.172:0/1381167419 >> 172.21.15.172:0/1381167419 conn(0x7f2680077230 msgr2=0x7f26800fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:09.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.689+0000 7f2688033640 1 -- 172.21.15.172:0/1381167419 shutdown_connections 2024-09-10T08:50:09.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.689+0000 7f2688033640 1 -- 172.21.15.172:0/1381167419 wait complete. 2024-09-10T08:50:09.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.690+0000 7f2688033640 1 Processor -- start 2024-09-10T08:50:09.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.690+0000 7f2688033640 1 -- start start 2024-09-10T08:50:09.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.690+0000 7f2688033640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2680101470 0x7f268019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.690+0000 7f2688033640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2680101e20 0x7f268019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.690+0000 7f2688033640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2680105de0 0x7f268019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.691+0000 7f2688033640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2680110cf0 con 0x7f2680101470 2024-09-10T08:50:09.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.691+0000 7f2688033640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2680110b70 con 0x7f2680101e20 2024-09-10T08:50:09.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.691+0000 7f2688033640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2680110e70 con 0x7f2680105de0 2024-09-10T08:50:09.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.691+0000 7f2685dcf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2680101470 0x7f268019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.691+0000 7f2685dcf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2680101470 0x7f268019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38528/0 (socket says 172.21.15.172:38528) 2024-09-10T08:50:09.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.691+0000 7f2685dcf640 1 -- 172.21.15.172:0/3899633313 learned_addr learned my addr 172.21.15.172:0/3899633313 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:09.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.691+0000 7f26865d0640 1 --2- 172.21.15.172:0/3899633313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2680105de0 0x7f268019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.691+0000 7f26855ce640 1 --2- 172.21.15.172:0/3899633313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2680101e20 0x7f268019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.691+0000 7f2685dcf640 1 -- 172.21.15.172:0/3899633313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2680105de0 msgr2=0x7f268019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:09.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.691+0000 7f2685dcf640 1 --2- 172.21.15.172:0/3899633313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2680105de0 0x7f268019ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.691+0000 7f2685dcf640 1 -- 172.21.15.172:0/3899633313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2680101e20 msgr2=0x7f268019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:09.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.691+0000 7f2685dcf640 1 --2- 172.21.15.172:0/3899633313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2680101e20 0x7f268019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.691+0000 7f2685dcf640 1 -- 172.21.15.172:0/3899633313 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268019f4e0 con 0x7f2680101470 2024-09-10T08:50:09.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.691+0000 7f2685dcf640 1 --2- 172.21.15.172:0/3899633313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2680101470 0x7f268019a530 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f267c005940 tx=0x7f267c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:09.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.692+0000 7f2676ffd640 1 -- 172.21.15.172:0/3899633313 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c03f030 con 0x7f2680101470 2024-09-10T08:50:09.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.692+0000 7f2676ffd640 1 -- 172.21.15.172:0/3899633313 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f267c003cb0 con 0x7f2680101470 2024-09-10T08:50:09.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.692+0000 7f2676ffd640 1 -- 172.21.15.172:0/3899633313 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c035070 con 0x7f2680101470 2024-09-10T08:50:09.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.692+0000 7f2688033640 1 -- 172.21.15.172:0/3899633313 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268019f770 con 0x7f2680101470 2024-09-10T08:50:09.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.692+0000 7f2688033640 1 -- 172.21.15.172:0/3899633313 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26801a70b0 con 0x7f2680101470 2024-09-10T08:50:09.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.696+0000 7f268a40c740 1 Processor -- start 2024-09-10T08:50:09.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.696+0000 7f2676ffd640 1 -- 172.21.15.172:0/3899633313 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f267c042030 con 0x7f2680101470 2024-09-10T08:50:09.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.696+0000 7f268a40c740 1 -- start start 2024-09-10T08:50:09.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.696+0000 7f268a40c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa840cb0 0x55a4aa841090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.696+0000 7f268a40c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa84b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.696+0000 7f268a40c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa84be90 0x55a4aa84e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.696+0000 7f268a40c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a4aa79e2e0 con 0x55a4aa8415d0 2024-09-10T08:50:09.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.696+0000 7f268a40c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a4aa745ce0 con 0x55a4aa840cb0 2024-09-10T08:50:09.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.696+0000 7f268a40c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a4aa79a2a0 con 0x55a4aa84be90 2024-09-10T08:50:09.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.696+0000 7f2685dcf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa84b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.696+0000 7f2685dcf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa84b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38532/0 (socket says 172.21.15.172:38532) 2024-09-10T08:50:09.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.697+0000 7f2685dcf640 1 -- 172.21.15.172:0/198311182 learned_addr learned my addr 172.21.15.172:0/198311182 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:09.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.697+0000 7f26855ce640 1 --2- 172.21.15.172:0/198311182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa84be90 0x55a4aa84e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.697+0000 7f26865d0640 1 --2- 172.21.15.172:0/198311182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa840cb0 0x55a4aa841090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.697+0000 7f2685dcf640 1 -- 172.21.15.172:0/198311182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa84be90 msgr2=0x55a4aa84e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:09.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.697+0000 7f2685dcf640 1 --2- 172.21.15.172:0/198311182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa84be90 0x55a4aa84e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.697+0000 7f2685dcf640 1 -- 172.21.15.172:0/198311182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa840cb0 msgr2=0x55a4aa841090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:09.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.697+0000 7f2685dcf640 1 --2- 172.21.15.172:0/198311182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa840cb0 0x55a4aa841090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.697+0000 7f2685dcf640 1 -- 172.21.15.172:0/198311182 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4aa72a440 con 0x55a4aa8415d0 2024-09-10T08:50:09.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.697+0000 7f2676ffd640 1 --2- 172.21.15.172:0/3899633313 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f266c0a2b80 0x7f266c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.697+0000 7f26865d0640 1 --2- 172.21.15.172:0/3899633313 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f266c0a2b80 0x7f266c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.698+0000 7f2676ffd640 1 -- 172.21.15.172:0/3899633313 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f267c0efb50 con 0x7f2680101470 2024-09-10T08:50:09.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.698+0000 7f2685dcf640 1 --2- 172.21.15.172:0/198311182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa84b950 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f267c0f3cf0 tx=0x7f267c0f3de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f37cee6f51dc57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:09.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.698+0000 7f26865d0640 1 --2- 172.21.15.172:0/3899633313 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f266c0a2b80 0x7f266c0a5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2678005fd0 tx=0x7f26780096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:09.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.698+0000 7f266bfff640 1 -- 172.21.15.172:0/198311182 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c0bbe20 con 0x55a4aa8415d0 2024-09-10T08:50:09.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.698+0000 7f266bfff640 1 -- 172.21.15.172:0/198311182 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f267c03b460 con 0x55a4aa8415d0 2024-09-10T08:50:09.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.698+0000 7f266bfff640 1 -- 172.21.15.172:0/198311182 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c005d50 con 0x55a4aa8415d0 2024-09-10T08:50:09.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.699+0000 7f268a40c740 1 -- 172.21.15.172:0/198311182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 msgr2=0x55a4aa84b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:09.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.699+0000 7f268a40c740 1 --2- 172.21.15.172:0/198311182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa84b950 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f267c0f3cf0 tx=0x7f267c0f3de0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.699+0000 7f268a40c740 1 -- 172.21.15.172:0/198311182 shutdown_connections 2024-09-10T08:50:09.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.699+0000 7f268a40c740 1 --2- 172.21.15.172:0/198311182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa84be90 0x55a4aa84e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.699+0000 7f268a40c740 1 --2- 172.21.15.172:0/198311182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa84b950 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.699+0000 7f268a40c740 1 --2- 172.21.15.172:0/198311182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa840cb0 0x55a4aa841090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.699+0000 7f268a40c740 1 -- 172.21.15.172:0/198311182 >> 172.21.15.172:0/198311182 conn(0x55a4aa83d240 msgr2=0x55a4aa83d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:09.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.699+0000 7f268a40c740 1 -- 172.21.15.172:0/198311182 shutdown_connections 2024-09-10T08:50:09.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.699+0000 7f268a40c740 1 -- 172.21.15.172:0/198311182 wait complete. 2024-09-10T08:50:09.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.700+0000 7f268a40c740 1 -- start start 2024-09-10T08:50:09.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.700+0000 7f268a40c740 1 Processor -- start 2024-09-10T08:50:09.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.701+0000 7f268a40c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa840cb0 0x55a4aa84b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.701+0000 7f268a40c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa848a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.701+0000 7f2685dcf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa840cb0 0x55a4aa84b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.701+0000 7f2685dcf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa840cb0 0x55a4aa84b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:36624/0 (socket says 172.21.15.172:36624) 2024-09-10T08:50:09.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.701+0000 7f2685dcf640 1 -- 172.21.15.172:0/126223149 learned_addr learned my addr 172.21.15.172:0/126223149 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:09.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.701+0000 7f26855ce640 1 --2- 172.21.15.172:0/126223149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa848a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.701+0000 7f268a40c740 1 --2- 172.21.15.172:0/126223149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa84be90 0x55a4aa848ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.701+0000 7f268a40c740 1 -- 172.21.15.172:0/126223149 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a4aa84e820 con 0x55a4aa8415d0 2024-09-10T08:50:09.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.702+0000 7f268a40c740 1 -- 172.21.15.172:0/126223149 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a4aa759050 con 0x55a4aa84be90 2024-09-10T08:50:09.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.702+0000 7f268a40c740 1 -- 172.21.15.172:0/126223149 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a4aa84e9a0 con 0x55a4aa840cb0 2024-09-10T08:50:09.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.702+0000 7f26865d0640 1 --2- 172.21.15.172:0/126223149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa84be90 0x55a4aa848ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.702+0000 7f26855ce640 1 -- 172.21.15.172:0/126223149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa840cb0 msgr2=0x55a4aa84b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:09.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.702+0000 7f26855ce640 1 --2- 172.21.15.172:0/126223149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa840cb0 0x55a4aa84b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.702+0000 7f26855ce640 1 -- 172.21.15.172:0/126223149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa84be90 msgr2=0x55a4aa848ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:09.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.702+0000 7f26855ce640 1 --2- 172.21.15.172:0/126223149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa84be90 0x55a4aa848ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.702+0000 7f26855ce640 1 -- 172.21.15.172:0/126223149 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4aa50f3c0 con 0x55a4aa8415d0 2024-09-10T08:50:09.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.702+0000 7f268a40c740 10 client.0 _ll_register_callbacks cb 0x55a4aa799160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:09.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.702+0000 7f268a40c740 1 -- 172.21.15.172:0/126223149 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a4aa7ab850 con 0x55a4aa8415d0 2024-09-10T08:50:09.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.702+0000 7f26855ce640 1 --2- 172.21.15.172:0/126223149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa848a90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f26801040d0 tx=0x7f2670012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c391e61a30ef2a9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:09.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.702+0000 7f26697fa640 10 client.16509 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:09.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.703+0000 7f26697fa640 1 -- 172.21.15.172:0/126223149 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670004320 con 0x55a4aa8415d0 2024-09-10T08:50:09.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.703+0000 7f26697fa640 1 -- 172.21.15.172:0/126223149 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2670011920 con 0x55a4aa8415d0 2024-09-10T08:50:09.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.703+0000 7f26697fa640 1 -- 172.21.15.172:0/126223149 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2670011b20 con 0x55a4aa8415d0 2024-09-10T08:50:09.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.703+0000 7f268a40c740 10 client.16509 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:09.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.703+0000 7f268a40c740 1 -- 172.21.15.172:0/126223149 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=33}) -- 0x55a4aa85a030 con 0x55a4aa8415d0 2024-09-10T08:50:09.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.704+0000 7f26697fa640 1 -- 172.21.15.172:0/126223149 <== mon.0 v2:172.21.15.172:3300/0 4 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f2670011cc0 con 0x55a4aa8415d0 2024-09-10T08:50:09.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.704+0000 7f268a40c740 10 client.16509 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:09.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.704+0000 7f268a40c740 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:09.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.704+0000 7f268a40c740 20 client.16509 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:09.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.704+0000 7f268a40c740 1 --2- 172.21.15.172:0/126223149 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a4aa85b3e0 0x55a4aa85d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.704+0000 7f268a40c740 4 client.16509 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:50:09.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.704+0000 7f268a40c740 1 -- 172.21.15.172:0/126223149 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4aa7a5d60 con 0x55a4aa85b3e0 2024-09-10T08:50:09.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.704+0000 7f2685dcf640 1 --2- 172.21.15.172:0/126223149 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a4aa85b3e0 0x55a4aa85d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.706+0000 7f26697fa640 10 client.16509 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:09.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.706+0000 7f2685dcf640 1 --2- 172.21.15.172:0/126223149 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a4aa85b3e0 0x55a4aa85d860 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=373fb86813c94208 server_cookie=972bb7959196f852 in_seq=0 out_seq=0 2024-09-10T08:50:09.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.706+0000 7f26697fa640 1 -- 172.21.15.172:0/126223149 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4aa7a5d60 con 0x55a4aa85b3e0 2024-09-10T08:50:09.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.706+0000 7f26697fa640 10 client.16509 handle_command_reply: tid=0 2024-09-10T08:50:09.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.706+0000 7f268a40c740 1 client.16509 shutdown 2024-09-10T08:50:09.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.707+0000 7f268a40c740 1 -- 172.21.15.172:0/126223149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 msgr2=0x55a4aa848a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:09.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.707+0000 7f268a40c740 1 --2- 172.21.15.172:0/126223149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa848a90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f26801040d0 tx=0x7f2670012b60 comp rx=0 tx=0).stop 2024-09-10T08:50:09.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.707+0000 7f268a40c740 1 -- 172.21.15.172:0/126223149 shutdown_connections 2024-09-10T08:50:09.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.707+0000 7f268a40c740 1 --2- 172.21.15.172:0/126223149 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a4aa85b3e0 0x55a4aa85d860 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.707+0000 7f268a40c740 1 --2- 172.21.15.172:0/126223149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa84be90 0x55a4aa848ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.707+0000 7f26697fa640 0 client.16509 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:09.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.707+0000 7f268a40c740 1 --2- 172.21.15.172:0/126223149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa848a90 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.707+0000 7f268a40c740 1 --2- 172.21.15.172:0/126223149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa840cb0 0x55a4aa84b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.707+0000 7f268a40c740 1 -- 172.21.15.172:0/126223149 >> 172.21.15.172:0/126223149 conn(0x55a4aa83d240 msgr2=0x55a4aa842f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:09.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.707+0000 7f268a40c740 1 -- 172.21.15.172:0/126223149 shutdown_connections 2024-09-10T08:50:09.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.707+0000 7f268a40c740 1 -- 172.21.15.172:0/126223149 wait complete. 2024-09-10T08:50:09.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.707+0000 7f268a40c740 20 client.16509 trim_cache size 0 max 16384 2024-09-10T08:50:09.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.732+0000 7f268a40c740 1 Processor -- start 2024-09-10T08:50:09.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.732+0000 7f268a40c740 1 -- start start 2024-09-10T08:50:09.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.732+0000 7f268a40c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa840cb0 0x55a4aa863130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.732+0000 7f268a40c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa863670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.732+0000 7f268a40c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa84be90 0x55a4aa853b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.732+0000 7f268a40c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f26801ac8e0 con 0x55a4aa8415d0 2024-09-10T08:50:09.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.732+0000 7f268a40c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26801ae4e0 con 0x55a4aa84be90 2024-09-10T08:50:09.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.732+0000 7f268a40c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f26801acfb0 con 0x55a4aa840cb0 2024-09-10T08:50:09.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.732+0000 7f26855ce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa863670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.732+0000 7f26855ce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa863670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38544/0 (socket says 172.21.15.172:38544) 2024-09-10T08:50:09.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.732+0000 7f26855ce640 1 -- 172.21.15.172:0/1315470025 learned_addr learned my addr 172.21.15.172:0/1315470025 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:09.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.733+0000 7f26865d0640 1 --2- 172.21.15.172:0/1315470025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa84be90 0x55a4aa853b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.733+0000 7f26855ce640 1 -- 172.21.15.172:0/1315470025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa840cb0 msgr2=0x55a4aa863130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:09.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.733+0000 7f2685dcf640 1 --2- 172.21.15.172:0/1315470025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa840cb0 0x55a4aa863130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.733+0000 7f26855ce640 1 --2- 172.21.15.172:0/1315470025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa840cb0 0x55a4aa863130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.733+0000 7f26855ce640 1 -- 172.21.15.172:0/1315470025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa84be90 msgr2=0x55a4aa853b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:09.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.733+0000 7f26855ce640 1 --2- 172.21.15.172:0/1315470025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa84be90 0x55a4aa853b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.733+0000 7f26855ce640 1 -- 172.21.15.172:0/1315470025 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4aa841400 con 0x55a4aa8415d0 2024-09-10T08:50:09.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.733+0000 7f2685dcf640 1 --2- 172.21.15.172:0/1315470025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa840cb0 0x55a4aa863130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:50:09.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.733+0000 7f26865d0640 1 --2- 172.21.15.172:0/1315470025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa84be90 0x55a4aa853b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:09.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.733+0000 7f26855ce640 1 --2- 172.21.15.172:0/1315470025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa863670 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2670011870 tx=0x7f267000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7e91aa77211b1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:09.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.733+0000 7f266b7fe640 1 -- 172.21.15.172:0/1315470025 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670004320 con 0x55a4aa8415d0 2024-09-10T08:50:09.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.733+0000 7f266b7fe640 1 -- 172.21.15.172:0/1315470025 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a4aa7ab850 con 0x55a4aa8415d0 2024-09-10T08:50:09.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.733+0000 7f266b7fe640 1 -- 172.21.15.172:0/1315470025 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267001c420 con 0x55a4aa8415d0 2024-09-10T08:50:09.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.734+0000 7f268a40c740 1 -- 172.21.15.172:0/1315470025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 msgr2=0x55a4aa863670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:09.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.734+0000 7f268a40c740 1 --2- 172.21.15.172:0/1315470025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa863670 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2670011870 tx=0x7f267000e000 comp rx=0 tx=0).stop 2024-09-10T08:50:09.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.734+0000 7f268a40c740 1 -- 172.21.15.172:0/1315470025 shutdown_connections 2024-09-10T08:50:09.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.734+0000 7f268a40c740 1 --2- 172.21.15.172:0/1315470025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa84be90 0x55a4aa853b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.734+0000 7f268a40c740 1 --2- 172.21.15.172:0/1315470025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa863670 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.734+0000 7f268a40c740 1 --2- 172.21.15.172:0/1315470025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa840cb0 0x55a4aa863130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.734+0000 7f268a40c740 1 -- 172.21.15.172:0/1315470025 >> 172.21.15.172:0/1315470025 conn(0x55a4aa83d240 msgr2=0x55a4aa84ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:09.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.734+0000 7f268a40c740 1 -- 172.21.15.172:0/1315470025 shutdown_connections 2024-09-10T08:50:09.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.734+0000 7f268a40c740 1 -- 172.21.15.172:0/1315470025 wait complete. 2024-09-10T08:50:09.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.735+0000 7f268a40c740 1 -- start start 2024-09-10T08:50:09.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.735+0000 7f268a40c740 1 Processor -- start 2024-09-10T08:50:09.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.735+0000 7f268a40c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa840cb0 0x55a4aa854200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.735+0000 7f268a40c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa8515a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.735+0000 7f268a40c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa84be90 0x55a4aa851ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.735+0000 7f268a40c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f26801add80 con 0x55a4aa8415d0 2024-09-10T08:50:09.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.735+0000 7f268a40c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26801ad6b0 con 0x55a4aa84be90 2024-09-10T08:50:09.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.735+0000 7f268a40c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f26801aa900 con 0x55a4aa840cb0 2024-09-10T08:50:09.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.735+0000 7f268a40c740 10 client.0 _ll_register_callbacks cb 0x7f26700097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:09.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.735+0000 7f26855ce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa8515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.735+0000 7f26855ce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa8515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38546/0 (socket says 172.21.15.172:38546) 2024-09-10T08:50:09.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.735+0000 7f26855ce640 1 -- 172.21.15.172:0/1742076204 learned_addr learned my addr 172.21.15.172:0/1742076204 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:09.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.736+0000 7f26865d0640 1 --2- 172.21.15.172:0/1742076204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa84be90 0x55a4aa851ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.736+0000 7f2685dcf640 1 --2- 172.21.15.172:0/1742076204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa840cb0 0x55a4aa854200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.736+0000 7f26855ce640 1 -- 172.21.15.172:0/1742076204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa840cb0 msgr2=0x55a4aa854200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:09.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.736+0000 7f26855ce640 1 --2- 172.21.15.172:0/1742076204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa840cb0 0x55a4aa854200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.736+0000 7f26855ce640 1 -- 172.21.15.172:0/1742076204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa84be90 msgr2=0x55a4aa851ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:09.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.736+0000 7f26855ce640 1 --2- 172.21.15.172:0/1742076204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa84be90 0x55a4aa851ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.736+0000 7f26855ce640 1 -- 172.21.15.172:0/1742076204 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4aa852310 con 0x55a4aa8415d0 2024-09-10T08:50:09.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.736+0000 7f26855ce640 1 --2- 172.21.15.172:0/1742076204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa8515a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2670003610 tx=0x7f2670011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e808e9d7a24228e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:09.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.736+0000 7f26697fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:09.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.736+0000 7f26697fa640 1 -- 172.21.15.172:0/1742076204 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670011920 con 0x55a4aa8415d0 2024-09-10T08:50:09.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.736+0000 7f268a40c740 1 -- 172.21.15.172:0/1742076204 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a4aa7ab850 con 0x55a4aa8415d0 2024-09-10T08:50:09.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.736+0000 7f26697fa640 1 -- 172.21.15.172:0/1742076204 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f26700039c0 con 0x55a4aa8415d0 2024-09-10T08:50:09.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.737+0000 7f26697fa640 1 -- 172.21.15.172:0/1742076204 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2670046050 con 0x55a4aa8415d0 2024-09-10T08:50:09.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.737+0000 7f268a40c740 10 client.16515 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:09.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.737+0000 7f268a40c740 1 -- 172.21.15.172:0/1742076204 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a4aa8c6000 con 0x55a4aa8415d0 2024-09-10T08:50:09.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.737+0000 7f26697fa640 1 -- 172.21.15.172:0/1742076204 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a4aa7ab850 con 0x55a4aa8415d0 2024-09-10T08:50:09.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.737+0000 7f26697fa640 1 -- 172.21.15.172:0/1742076204 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f267001c420 con 0x55a4aa8415d0 2024-09-10T08:50:09.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.737+0000 7f268a40c740 10 client.16515 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:09.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.737+0000 7f268a40c740 10 client.16515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:09.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.737+0000 7f268a40c740 20 client.16515 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:09.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.737+0000 7f268a40c740 1 --2- 172.21.15.172:0/1742076204 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a4aa85b3e0 0x55a4aa8c8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:09.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.737+0000 7f268a40c740 4 client.16515 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:50:09.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.737+0000 7f268a40c740 1 -- 172.21.15.172:0/1742076204 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a4aa722210 con 0x55a4aa85b3e0 2024-09-10T08:50:09.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.738+0000 7f2685dcf640 1 --2- 172.21.15.172:0/1742076204 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a4aa85b3e0 0x55a4aa8c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:09.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.738+0000 7f2685dcf640 1 --2- 172.21.15.172:0/1742076204 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a4aa85b3e0 0x55a4aa8c8840 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c76061c3f3e6eba8 server_cookie=54022305608b4cdc in_seq=0 out_seq=0 2024-09-10T08:50:09.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.738+0000 7f26697fa640 10 client.16515 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:09.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.738+0000 7f26697fa640 1 -- 172.21.15.172:0/1742076204 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a4aa722210 con 0x55a4aa85b3e0 2024-09-10T08:50:09.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.738+0000 7f26697fa640 10 client.16515 handle_command_reply: tid=0 2024-09-10T08:50:09.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.738+0000 7f268a40c740 1 client.16515 shutdown 2024-09-10T08:50:09.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.738+0000 7f268a40c740 1 -- 172.21.15.172:0/1742076204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 msgr2=0x55a4aa8515a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:09.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.738+0000 7f268a40c740 1 --2- 172.21.15.172:0/1742076204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa8515a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2670003610 tx=0x7f2670011ef0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.739+0000 7f268a40c740 1 -- 172.21.15.172:0/1742076204 shutdown_connections 2024-09-10T08:50:09.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.739+0000 7f268a40c740 1 --2- 172.21.15.172:0/1742076204 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a4aa85b3e0 0x55a4aa8c8840 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.739+0000 7f268a40c740 1 --2- 172.21.15.172:0/1742076204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a4aa84be90 0x55a4aa851ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.739+0000 7f268a40c740 1 --2- 172.21.15.172:0/1742076204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a4aa8415d0 0x55a4aa8515a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.739+0000 7f268a40c740 1 --2- 172.21.15.172:0/1742076204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a4aa840cb0 0x55a4aa854200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.739+0000 7f26697fa640 0 client.16515 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:09.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.739+0000 7f268a40c740 1 -- 172.21.15.172:0/1742076204 >> 172.21.15.172:0/1742076204 conn(0x55a4aa83d240 msgr2=0x55a4aa833350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:09.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.739+0000 7f268a40c740 1 -- 172.21.15.172:0/1742076204 shutdown_connections 2024-09-10T08:50:09.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.739+0000 7f268a40c740 1 -- 172.21.15.172:0/1742076204 wait complete. 2024-09-10T08:50:09.704 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:50:09.704 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T08:50:09.704 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T08:50:09.704 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:50:09.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.739+0000 7f268a40c740 20 client.16515 trim_cache size 0 max 16384 2024-09-10T08:50:09.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.739+0000 7f2688033640 1 -- 172.21.15.172:0/3899633313 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f266c0a2b80 msgr2=0x7f266c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:09.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.739+0000 7f2688033640 1 --2- 172.21.15.172:0/3899633313 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f266c0a2b80 0x7f266c0a5040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2678005fd0 tx=0x7f26780096a0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.739+0000 7f2688033640 1 -- 172.21.15.172:0/3899633313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2680101470 msgr2=0x7f268019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:09.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.739+0000 7f2688033640 1 --2- 172.21.15.172:0/3899633313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2680101470 0x7f268019a530 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f267c005940 tx=0x7f267c0042f0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.740+0000 7f2688033640 1 -- 172.21.15.172:0/3899633313 shutdown_connections 2024-09-10T08:50:09.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.740+0000 7f2688033640 1 --2- 172.21.15.172:0/3899633313 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f266c0a2b80 0x7f266c0a5040 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.740+0000 7f2688033640 1 --2- 172.21.15.172:0/3899633313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2680105de0 0x7f268019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.740+0000 7f2688033640 1 --2- 172.21.15.172:0/3899633313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2680101e20 0x7f268019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.740+0000 7f2688033640 1 --2- 172.21.15.172:0/3899633313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2680101470 0x7f268019a530 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:09.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.740+0000 7f2688033640 1 -- 172.21.15.172:0/3899633313 >> 172.21.15.172:0/3899633313 conn(0x7f2680077230 msgr2=0x7f2680104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:09.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.740+0000 7f2688033640 1 -- 172.21.15.172:0/3899633313 shutdown_connections 2024-09-10T08:50:09.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:08.740+0000 7f2688033640 1 -- 172.21.15.172:0/3899633313 wait complete. 2024-09-10T08:50:10.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:10 smithi172 ceph-mon[30854]: pgmap v468: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-09-10T08:50:10.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:10 smithi174 ceph-mon[34068]: pgmap v468: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-09-10T08:50:10.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:10 smithi191 ceph-mon[35314]: pgmap v468: 129 pgs: 129 active+clean; 2.3 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-09-10T08:50:11.494 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:50:11.494 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:50:11.790 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:50:12.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.299+0000 7f6afc6e7640 1 -- 172.21.15.172:0/252671306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af4102d30 msgr2=0x7f6af4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:12.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.299+0000 7f6afc6e7640 1 --2- 172.21.15.172:0/252671306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af4102d30 0x7f6af4103110 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009900 tx=0x7f6af002a480 comp rx=0 tx=0).stop 2024-09-10T08:50:12.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.299+0000 7f6afc6e7640 1 -- 172.21.15.172:0/252671306 shutdown_connections 2024-09-10T08:50:12.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.299+0000 7f6afc6e7640 1 --2- 172.21.15.172:0/252671306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af410c160 0x7f6af410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:12.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.299+0000 7f6afc6e7640 1 --2- 172.21.15.172:0/252671306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6af4103650 0x7f6af410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:12.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.299+0000 7f6afc6e7640 1 --2- 172.21.15.172:0/252671306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af4102d30 0x7f6af4103110 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:12.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.299+0000 7f6afc6e7640 1 -- 172.21.15.172:0/252671306 >> 172.21.15.172:0/252671306 conn(0x7f6af40fc790 msgr2=0x7f6af40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:12.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.299+0000 7f6afc6e7640 1 -- 172.21.15.172:0/252671306 shutdown_connections 2024-09-10T08:50:12.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.299+0000 7f6afc6e7640 1 -- 172.21.15.172:0/252671306 wait complete. 2024-09-10T08:50:12.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.300+0000 7f6afc6e7640 1 Processor -- start 2024-09-10T08:50:12.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.300+0000 7f6afc6e7640 1 -- start start 2024-09-10T08:50:12.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.300+0000 7f6afc6e7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6af4102d30 0x7f6af4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:12.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.300+0000 7f6afc6e7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af4103650 0x7f6af41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:12.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.300+0000 7f6afc6e7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af410c160 0x7f6af419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:12.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.300+0000 7f6afc6e7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6af4111e10 con 0x7f6af410c160 2024-09-10T08:50:12.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.300+0000 7f6afc6e7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6af4111c90 con 0x7f6af4103650 2024-09-10T08:50:12.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.300+0000 7f6afc6e7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6af4111f90 con 0x7f6af4102d30 2024-09-10T08:50:12.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.300+0000 7f6af9c82640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af4103650 0x7f6af41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:12.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.300+0000 7f6afac84640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af410c160 0x7f6af419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:12.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.300+0000 7f6af9c82640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af4103650 0x7f6af41988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:53268/0 (socket says 172.21.15.172:53268) 2024-09-10T08:50:12.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.300+0000 7f6af9c82640 1 -- 172.21.15.172:0/3820635263 learned_addr learned my addr 172.21.15.172:0/3820635263 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:12.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.301+0000 7f6afa483640 1 --2- 172.21.15.172:0/3820635263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6af4102d30 0x7f6af4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:12.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.301+0000 7f6af9c82640 1 -- 172.21.15.172:0/3820635263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6af4102d30 msgr2=0x7f6af4198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:12.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.301+0000 7f6af9c82640 1 --2- 172.21.15.172:0/3820635263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6af4102d30 0x7f6af4198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:12.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.301+0000 7f6af9c82640 1 -- 172.21.15.172:0/3820635263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af410c160 msgr2=0x7f6af419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:12.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.301+0000 7f6af9c82640 1 --2- 172.21.15.172:0/3820635263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af410c160 0x7f6af419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:12.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.301+0000 7f6af9c82640 1 -- 172.21.15.172:0/3820635263 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af419d310 con 0x7f6af4103650 2024-09-10T08:50:12.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.301+0000 7f6af9c82640 1 --2- 172.21.15.172:0/3820635263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af4103650 0x7f6af41988a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400e9f0 tx=0x7f6ae400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:12.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.301+0000 7f6aeb7fe640 1 -- 172.21.15.172:0/3820635263 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae400ccc0 con 0x7f6af4103650 2024-09-10T08:50:12.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.301+0000 7f6afc6e7640 1 -- 172.21.15.172:0/3820635263 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af419d600 con 0x7f6af4103650 2024-09-10T08:50:12.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.301+0000 7f6aeb7fe640 1 -- 172.21.15.172:0/3820635263 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ae400ce60 con 0x7f6af4103650 2024-09-10T08:50:12.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.301+0000 7f6aeb7fe640 1 -- 172.21.15.172:0/3820635263 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4010430 con 0x7f6af4103650 2024-09-10T08:50:12.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.301+0000 7f6afc6e7640 1 -- 172.21.15.172:0/3820635263 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af41a4ee0 con 0x7f6af4103650 2024-09-10T08:50:12.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.303+0000 7f6afc6e7640 1 -- 172.21.15.172:0/3820635263 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad8005290 con 0x7f6af4103650 2024-09-10T08:50:12.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.304+0000 7f6aeb7fe640 1 -- 172.21.15.172:0/3820635263 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6ae40046c0 con 0x7f6af4103650 2024-09-10T08:50:12.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.305+0000 7f6aeb7fe640 1 --2- 172.21.15.172:0/3820635263 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6ae00a2b80 0x7f6ae00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:12.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.305+0000 7f6afa483640 1 --2- 172.21.15.172:0/3820635263 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6ae00a2b80 0x7f6ae00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:12.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.305+0000 7f6afa483640 1 --2- 172.21.15.172:0/3820635263 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6ae00a2b80 0x7f6ae00a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6af0031440 tx=0x7f6af00095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:12.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.308+0000 7f6aeb7fe640 1 -- 172.21.15.172:0/3820635263 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6ae4099b20 con 0x7f6af4103650 2024-09-10T08:50:12.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.312+0000 7f6aeb7fe640 1 -- 172.21.15.172:0/3820635263 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ae409c180 con 0x7f6af4103650 2024-09-10T08:50:12.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.558+0000 7f6afc6e7640 1 -- 172.21.15.172:0/3820635263 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ad8005580 con 0x7f6af4103650 2024-09-10T08:50:12.559 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:12 smithi172 ceph-mon[30854]: pgmap v469: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-09-10T08:50:12.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.559+0000 7f6aeb7fe640 1 -- 172.21.15.172:0/3820635263 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+12007 (secure 0 0 0) 0x7f6ae409ba80 con 0x7f6af4103650 2024-09-10T08:50:12.560 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:50:12.560 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":33,"btime":"2024-09-10T08:50:03:285217+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:50:02.654978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:50:12.560 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 33 2024-09-10T08:50:12.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.562+0000 7f6afc6e7640 1 -- 172.21.15.172:0/3820635263 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6ae00a2b80 msgr2=0x7f6ae00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:12.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.562+0000 7f6afc6e7640 1 --2- 172.21.15.172:0/3820635263 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6ae00a2b80 0x7f6ae00a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6af0031440 tx=0x7f6af00095f0 comp rx=0 tx=0).stop 2024-09-10T08:50:12.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.562+0000 7f6afc6e7640 1 -- 172.21.15.172:0/3820635263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af4103650 msgr2=0x7f6af41988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:12.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.562+0000 7f6afc6e7640 1 --2- 172.21.15.172:0/3820635263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af4103650 0x7f6af41988a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400e9f0 tx=0x7f6ae400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:50:12.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.562+0000 7f6afc6e7640 1 -- 172.21.15.172:0/3820635263 shutdown_connections 2024-09-10T08:50:12.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.562+0000 7f6afc6e7640 1 --2- 172.21.15.172:0/3820635263 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6ae00a2b80 0x7f6ae00a5040 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:12.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.562+0000 7f6afc6e7640 1 --2- 172.21.15.172:0/3820635263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af410c160 0x7f6af419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:12.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.562+0000 7f6afc6e7640 1 --2- 172.21.15.172:0/3820635263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af4103650 0x7f6af41988a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:12.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.562+0000 7f6afc6e7640 1 --2- 172.21.15.172:0/3820635263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6af4102d30 0x7f6af4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:12.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.562+0000 7f6afc6e7640 1 -- 172.21.15.172:0/3820635263 >> 172.21.15.172:0/3820635263 conn(0x7f6af40fc790 msgr2=0x7f6af40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:12.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.562+0000 7f6afc6e7640 1 -- 172.21.15.172:0/3820635263 shutdown_connections 2024-09-10T08:50:12.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:12.562+0000 7f6afc6e7640 1 -- 172.21.15.172:0/3820635263 wait complete. 2024-09-10T08:50:12.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:12 smithi174 ceph-mon[34068]: pgmap v469: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-09-10T08:50:12.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:12 smithi191 ceph-mon[35314]: pgmap v469: 129 pgs: 129 active+clean; 2.4 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-09-10T08:50:13.297 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T08:50:13.326 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:13 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3820635263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:50:13.589 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:50:13.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:13 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3820635263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:50:13.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:13 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3820635263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:50:14.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.129+0000 7f35635a0640 1 -- 172.21.15.172:0/454285442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c102790 msgr2=0x7f355c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:14.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.129+0000 7f35635a0640 1 --2- 172.21.15.172:0/454285442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c102790 0x7f355c1067e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f354c009900 tx=0x7f354c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.129+0000 7f35635a0640 1 -- 172.21.15.172:0/454285442 shutdown_connections 2024-09-10T08:50:14.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.129+0000 7f35635a0640 1 --2- 172.21.15.172:0/454285442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f355c106f10 0x7f355c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.129+0000 7f35635a0640 1 --2- 172.21.15.172:0/454285442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c102790 0x7f355c1067e0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.129+0000 7f35635a0640 1 --2- 172.21.15.172:0/454285442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f355c101de0 0x7f355c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.129+0000 7f35635a0640 1 -- 172.21.15.172:0/454285442 >> 172.21.15.172:0/454285442 conn(0x7f355c0fd8e0 msgr2=0x7f355c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:14.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.129+0000 7f35635a0640 1 -- 172.21.15.172:0/454285442 shutdown_connections 2024-09-10T08:50:14.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.129+0000 7f35635a0640 1 -- 172.21.15.172:0/454285442 wait complete. 2024-09-10T08:50:14.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.130+0000 7f35635a0640 1 Processor -- start 2024-09-10T08:50:14.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.130+0000 7f35635a0640 1 -- start start 2024-09-10T08:50:14.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.130+0000 7f35635a0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f355c101de0 0x7f355c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.130+0000 7f35635a0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f355c102790 0x7f355c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.130+0000 7f35635a0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c106f10 0x7f355c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.130+0000 7f35635a0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f355c111de0 con 0x7f355c106f10 2024-09-10T08:50:14.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.130+0000 7f35635a0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f355c111c60 con 0x7f355c101de0 2024-09-10T08:50:14.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.130+0000 7f35635a0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f355c111f60 con 0x7f355c102790 2024-09-10T08:50:14.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.130+0000 7f3561b3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c106f10 0x7f355c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.130+0000 7f3561b3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c106f10 0x7f355c19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38578/0 (socket says 172.21.15.172:38578) 2024-09-10T08:50:14.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.131+0000 7f3561b3d640 1 -- 172.21.15.172:0/2084079585 learned_addr learned my addr 172.21.15.172:0/2084079585 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:14.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.131+0000 7f3560b3b640 1 --2- 172.21.15.172:0/2084079585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f355c102790 0x7f355c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.131+0000 7f356133c640 1 --2- 172.21.15.172:0/2084079585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f355c101de0 0x7f355c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.131+0000 7f3561b3d640 1 -- 172.21.15.172:0/2084079585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f355c102790 msgr2=0x7f355c1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:14.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.131+0000 7f3561b3d640 1 --2- 172.21.15.172:0/2084079585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f355c102790 0x7f355c1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.131+0000 7f3561b3d640 1 -- 172.21.15.172:0/2084079585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f355c101de0 msgr2=0x7f355c1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:14.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.131+0000 7f3561b3d640 1 --2- 172.21.15.172:0/2084079585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f355c101de0 0x7f355c1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.131+0000 7f3561b3d640 1 -- 172.21.15.172:0/2084079585 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355c19d250 con 0x7f355c106f10 2024-09-10T08:50:14.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.131+0000 7f3561b3d640 1 --2- 172.21.15.172:0/2084079585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c106f10 0x7f355c19cb70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f355400cce0 tx=0x7f355400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:14.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.131+0000 7f35527fc640 1 -- 172.21.15.172:0/2084079585 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355400f450 con 0x7f355c106f10 2024-09-10T08:50:14.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.131+0000 7f35527fc640 1 -- 172.21.15.172:0/2084079585 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f355400faa0 con 0x7f355c106f10 2024-09-10T08:50:14.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.131+0000 7f35635a0640 1 -- 172.21.15.172:0/2084079585 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355c19d540 con 0x7f355c106f10 2024-09-10T08:50:14.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.131+0000 7f35527fc640 1 -- 172.21.15.172:0/2084079585 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554018a80 con 0x7f355c106f10 2024-09-10T08:50:14.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.131+0000 7f35635a0640 1 -- 172.21.15.172:0/2084079585 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355c1a4e20 con 0x7f355c106f10 2024-09-10T08:50:14.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.134+0000 7f35527fc640 1 -- 172.21.15.172:0/2084079585 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f355400fc50 con 0x7f355c106f10 2024-09-10T08:50:14.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.134+0000 7f35527fc640 1 --2- 172.21.15.172:0/2084079585 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f35480a2b80 0x7f35480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.134+0000 7f356133c640 1 --2- 172.21.15.172:0/2084079585 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f35480a2b80 0x7f35480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.134+0000 7f35527fc640 1 -- 172.21.15.172:0/2084079585 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f35540cf760 con 0x7f355c106f10 2024-09-10T08:50:14.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.135+0000 7f3565979740 1 Processor -- start 2024-09-10T08:50:14.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.135+0000 7f3565979740 1 -- start start 2024-09-10T08:50:14.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.135+0000 7f356133c640 1 --2- 172.21.15.172:0/2084079585 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f35480a2b80 0x7f35480a5040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f355800abe0 tx=0x7f355800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:14.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.135+0000 7f3565979740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3eeb40 0x5617ed3ed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.135+0000 7f3565979740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3ed5d0 0x5617ed3eda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.135+0000 7f3565979740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3f8370 0x5617ed3fa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.135+0000 7f3565979740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5617ed3532f0 con 0x5617ed3eeb40 2024-09-10T08:50:14.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.135+0000 7f3565979740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5617ed2facf0 con 0x5617ed3ed5d0 2024-09-10T08:50:14.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.135+0000 7f3565979740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5617ed34f2b0 con 0x5617ed3f8370 2024-09-10T08:50:14.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.135+0000 7f356133c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3eeb40 0x5617ed3ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.135+0000 7f356133c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3eeb40 0x5617ed3ed090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38580/0 (socket says 172.21.15.172:38580) 2024-09-10T08:50:14.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.135+0000 7f356133c640 1 -- 172.21.15.172:0/1577701770 learned_addr learned my addr 172.21.15.172:0/1577701770 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:14.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.135+0000 7f3560b3b640 1 --2- 172.21.15.172:0/1577701770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3ed5d0 0x5617ed3eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.135+0000 7f3561b3d640 1 --2- 172.21.15.172:0/1577701770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3f8370 0x5617ed3fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.136+0000 7f356133c640 1 -- 172.21.15.172:0/1577701770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3f8370 msgr2=0x5617ed3fa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:14.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.136+0000 7f356133c640 1 --2- 172.21.15.172:0/1577701770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3f8370 0x5617ed3fa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.136+0000 7f356133c640 1 -- 172.21.15.172:0/1577701770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3ed5d0 msgr2=0x5617ed3eda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:14.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.136+0000 7f356133c640 1 --2- 172.21.15.172:0/1577701770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3ed5d0 0x5617ed3eda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.136+0000 7f356133c640 1 -- 172.21.15.172:0/1577701770 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617ed2df3a0 con 0x5617ed3eeb40 2024-09-10T08:50:14.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.136+0000 7f356133c640 1 --2- 172.21.15.172:0/1577701770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3eeb40 0x5617ed3ed090 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f355800ffd0 tx=0x7f3558012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36418a8338d40f77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:14.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.136+0000 7f35477fe640 1 -- 172.21.15.172:0/1577701770 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3558016070 con 0x5617ed3eeb40 2024-09-10T08:50:14.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.136+0000 7f35477fe640 1 -- 172.21.15.172:0/1577701770 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f355801b8d0 con 0x5617ed3eeb40 2024-09-10T08:50:14.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.136+0000 7f35477fe640 1 -- 172.21.15.172:0/1577701770 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355801c8c0 con 0x5617ed3eeb40 2024-09-10T08:50:14.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.137+0000 7f3565979740 1 -- 172.21.15.172:0/1577701770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3eeb40 msgr2=0x5617ed3ed090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:14.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.137+0000 7f3565979740 1 --2- 172.21.15.172:0/1577701770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3eeb40 0x5617ed3ed090 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f355800ffd0 tx=0x7f3558012000 comp rx=0 tx=0).stop 2024-09-10T08:50:14.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.137+0000 7f3565979740 1 -- 172.21.15.172:0/1577701770 shutdown_connections 2024-09-10T08:50:14.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.137+0000 7f3565979740 1 --2- 172.21.15.172:0/1577701770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3f8370 0x5617ed3fa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.137+0000 7f3565979740 1 --2- 172.21.15.172:0/1577701770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3ed5d0 0x5617ed3eda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.137+0000 7f3565979740 1 --2- 172.21.15.172:0/1577701770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3eeb40 0x5617ed3ed090 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.137+0000 7f3565979740 1 -- 172.21.15.172:0/1577701770 >> 172.21.15.172:0/1577701770 conn(0x5617ed3b6310 msgr2=0x5617ed3e9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:14.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.137+0000 7f3565979740 1 -- 172.21.15.172:0/1577701770 shutdown_connections 2024-09-10T08:50:14.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.137+0000 7f3565979740 1 -- 172.21.15.172:0/1577701770 wait complete. 2024-09-10T08:50:14.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.137+0000 7f3565979740 1 -- start start 2024-09-10T08:50:14.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.137+0000 7f3565979740 1 Processor -- start 2024-09-10T08:50:14.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.138+0000 7f3565979740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3ed5d0 0x5617ed3f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.138+0000 7f3565979740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3eeb40 0x5617ed3f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.138+0000 7f3565979740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3f8370 0x5617ed3f5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.138+0000 7f3565979740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5617ed3ff480 con 0x5617ed3f8370 2024-09-10T08:50:14.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.138+0000 7f3565979740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5617ed30e060 con 0x5617ed3ed5d0 2024-09-10T08:50:14.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.138+0000 7f3565979740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5617ed3ff600 con 0x5617ed3eeb40 2024-09-10T08:50:14.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.138+0000 7f3565979740 10 client.0 _ll_register_callbacks cb 0x5617ed0618c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:14.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.138+0000 7f3561b3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3f8370 0x5617ed3f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.138+0000 7f3560b3b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3eeb40 0x5617ed3f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.138+0000 7f3560b3b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3eeb40 0x5617ed3f4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:36704/0 (socket says 172.21.15.172:36704) 2024-09-10T08:50:14.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.138+0000 7f3560b3b640 1 -- 172.21.15.172:0/1487070121 learned_addr learned my addr 172.21.15.172:0/1487070121 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:14.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.138+0000 7f356133c640 1 --2- 172.21.15.172:0/1487070121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3ed5d0 0x5617ed3f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.138+0000 7f3560b3b640 1 -- 172.21.15.172:0/1487070121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3ed5d0 msgr2=0x5617ed3f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:14.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.138+0000 7f3560b3b640 1 --2- 172.21.15.172:0/1487070121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3ed5d0 0x5617ed3f7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.138+0000 7f3560b3b640 1 -- 172.21.15.172:0/1487070121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3f8370 msgr2=0x5617ed3f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:14.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.138+0000 7f3560b3b640 1 --2- 172.21.15.172:0/1487070121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3f8370 0x5617ed3f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.138+0000 7f3560b3b640 1 -- 172.21.15.172:0/1487070121 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617ed0c43c0 con 0x5617ed3eeb40 2024-09-10T08:50:14.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.139+0000 7f356133c640 1 --2- 172.21.15.172:0/1487070121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3ed5d0 0x5617ed3f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:14.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.139+0000 7f3561b3d640 1 --2- 172.21.15.172:0/1487070121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3f8370 0x5617ed3f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:50:14.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.139+0000 7f3560b3b640 1 --2- 172.21.15.172:0/1487070121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3eeb40 0x5617ed3f4fd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f354c009900 tx=0x7f354c0316d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1483fb5277c2340b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:14.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.139+0000 7f3544ff9640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:50:14.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.139+0000 7f3544ff9640 1 -- 172.21.15.172:0/1487070121 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c035040 con 0x5617ed3eeb40 2024-09-10T08:50:14.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.139+0000 7f3544ff9640 1 -- 172.21.15.172:0/1487070121 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f354c004040 con 0x5617ed3eeb40 2024-09-10T08:50:14.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.139+0000 7f3565979740 1 -- 172.21.15.172:0/1487070121 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f355801b8d0 con 0x5617ed3eeb40 2024-09-10T08:50:14.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.139+0000 7f3544ff9640 1 -- 172.21.15.172:0/1487070121 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f354c004240 con 0x5617ed3eeb40 2024-09-10T08:50:14.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.139+0000 7f3565979740 10 client.25789 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:14.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.139+0000 7f3565979740 1 -- 172.21.15.172:0/1487070121 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5617ed40aed0 con 0x5617ed3eeb40 2024-09-10T08:50:14.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.140+0000 7f3544ff9640 1 -- 172.21.15.172:0/1487070121 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f355801b8d0 con 0x5617ed3eeb40 2024-09-10T08:50:14.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.140+0000 7f3544ff9640 1 -- 172.21.15.172:0/1487070121 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f354c046590 con 0x5617ed3eeb40 2024-09-10T08:50:14.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.140+0000 7f3565979740 10 client.25789 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:14.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.140+0000 7f3565979740 10 client.25789 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:50:14.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.140+0000 7f3565979740 20 client.25789 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:14.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.140+0000 7f3565979740 1 --2- 172.21.15.172:0/1487070121 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5617ed40c3e0 0x5617ed40e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.140+0000 7f3565979740 4 client.25789 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:50:14.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.140+0000 7f3565979740 1 -- 172.21.15.172:0/1487070121 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617ed360860 con 0x5617ed40c3e0 2024-09-10T08:50:14.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.140+0000 7f356133c640 1 --2- 172.21.15.172:0/1487070121 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5617ed40c3e0 0x5617ed40e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.141+0000 7f3544ff9640 10 client.25789 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:50:14.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.141+0000 7f356133c640 1 --2- 172.21.15.172:0/1487070121 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5617ed40c3e0 0x5617ed40e820 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2455f32afc47f400 server_cookie=d0bc8162ccb67638 in_seq=0 out_seq=0 2024-09-10T08:50:14.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.141+0000 7f3544ff9640 1 -- 172.21.15.172:0/1487070121 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617ed360860 con 0x5617ed40c3e0 2024-09-10T08:50:14.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.141+0000 7f3544ff9640 10 client.25789 handle_command_reply: tid=0 2024-09-10T08:50:14.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.141+0000 7f3565979740 1 client.25789 shutdown 2024-09-10T08:50:14.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.142+0000 7f3565979740 1 -- 172.21.15.172:0/1487070121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3eeb40 msgr2=0x5617ed3f4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:14.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.142+0000 7f3565979740 1 --2- 172.21.15.172:0/1487070121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3eeb40 0x5617ed3f4fd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f354c009900 tx=0x7f354c0316d0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.142+0000 7f3565979740 1 -- 172.21.15.172:0/1487070121 shutdown_connections 2024-09-10T08:50:14.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.142+0000 7f3565979740 1 --2- 172.21.15.172:0/1487070121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3f8370 0x5617ed3f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.142+0000 7f3565979740 1 --2- 172.21.15.172:0/1487070121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3eeb40 0x5617ed3f4fd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.142+0000 7f3565979740 1 --2- 172.21.15.172:0/1487070121 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5617ed40c3e0 0x5617ed40e820 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.142+0000 7f3565979740 1 --2- 172.21.15.172:0/1487070121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3ed5d0 0x5617ed3f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.142+0000 7f3565979740 1 -- 172.21.15.172:0/1487070121 >> 172.21.15.172:0/1487070121 conn(0x5617ed3b6310 msgr2=0x5617ed3ef960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:14.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.142+0000 7f3544ff9640 0 client.25789 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:50:14.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.142+0000 7f3565979740 1 -- 172.21.15.172:0/1487070121 shutdown_connections 2024-09-10T08:50:14.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.142+0000 7f3565979740 1 -- 172.21.15.172:0/1487070121 wait complete. 2024-09-10T08:50:14.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.142+0000 7f3565979740 20 client.25789 trim_cache size 0 max 16384 2024-09-10T08:50:14.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3565979740 1 Processor -- start 2024-09-10T08:50:14.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3565979740 1 -- start start 2024-09-10T08:50:14.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3565979740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3ed5d0 0x5617ed408720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3565979740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3eeb40 0x5617ed3f4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3565979740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3f8370 0x5617ed3f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3565979740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f355c1ac830 con 0x5617ed3eeb40 2024-09-10T08:50:14.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3565979740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f355c1ae430 con 0x5617ed3f8370 2024-09-10T08:50:14.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3565979740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f355c1acf00 con 0x5617ed3ed5d0 2024-09-10T08:50:14.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3560b3b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3eeb40 0x5617ed3f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3560b3b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3eeb40 0x5617ed3f4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38598/0 (socket says 172.21.15.172:38598) 2024-09-10T08:50:14.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3560b3b640 1 -- 172.21.15.172:0/3517569804 learned_addr learned my addr 172.21.15.172:0/3517569804 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:14.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3561b3d640 1 --2- 172.21.15.172:0/3517569804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3f8370 0x5617ed3f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f356133c640 1 --2- 172.21.15.172:0/3517569804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3ed5d0 0x5617ed408720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3560b3b640 1 -- 172.21.15.172:0/3517569804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3ed5d0 msgr2=0x5617ed408720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:14.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3560b3b640 1 --2- 172.21.15.172:0/3517569804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3ed5d0 0x5617ed408720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3560b3b640 1 -- 172.21.15.172:0/3517569804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3f8370 msgr2=0x5617ed3f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:14.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3560b3b640 1 --2- 172.21.15.172:0/3517569804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3f8370 0x5617ed3f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3560b3b640 1 -- 172.21.15.172:0/3517569804 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617ed3ea730 con 0x5617ed3eeb40 2024-09-10T08:50:14.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.156+0000 7f3561b3d640 1 --2- 172.21.15.172:0/3517569804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3f8370 0x5617ed3f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:14.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.157+0000 7f3560b3b640 1 --2- 172.21.15.172:0/3517569804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3eeb40 0x5617ed3f4890 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f354c02a910 tx=0x7f354c02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff5a9e9b2f8adbc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:14.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.157+0000 7f3546ffd640 1 -- 172.21.15.172:0/3517569804 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c035030 con 0x5617ed3eeb40 2024-09-10T08:50:14.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.157+0000 7f3546ffd640 1 -- 172.21.15.172:0/3517569804 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f354c031940 con 0x5617ed3eeb40 2024-09-10T08:50:14.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.157+0000 7f3546ffd640 1 -- 172.21.15.172:0/3517569804 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c0466d0 con 0x5617ed3eeb40 2024-09-10T08:50:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.158+0000 7f3565979740 1 -- 172.21.15.172:0/3517569804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3eeb40 msgr2=0x5617ed3f4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.158+0000 7f3565979740 1 --2- 172.21.15.172:0/3517569804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3eeb40 0x5617ed3f4890 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f354c02a910 tx=0x7f354c02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.158+0000 7f3565979740 1 -- 172.21.15.172:0/3517569804 shutdown_connections 2024-09-10T08:50:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.158+0000 7f3565979740 1 --2- 172.21.15.172:0/3517569804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3f8370 0x5617ed3f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.158+0000 7f3565979740 1 --2- 172.21.15.172:0/3517569804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3eeb40 0x5617ed3f4890 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.158+0000 7f3565979740 1 --2- 172.21.15.172:0/3517569804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3ed5d0 0x5617ed408720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.158+0000 7f3565979740 1 -- 172.21.15.172:0/3517569804 >> 172.21.15.172:0/3517569804 conn(0x5617ed3b6310 msgr2=0x5617ed3f38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.158+0000 7f3565979740 1 -- 172.21.15.172:0/3517569804 shutdown_connections 2024-09-10T08:50:14.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.158+0000 7f3565979740 1 -- 172.21.15.172:0/3517569804 wait complete. 2024-09-10T08:50:14.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.158+0000 7f3565979740 1 -- start start 2024-09-10T08:50:14.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.158+0000 7f3565979740 1 Processor -- start 2024-09-10T08:50:14.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3565979740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3ed5d0 0x5617ed46f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3565979740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3eeb40 0x5617ed46f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3565979740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3f8370 0x5617ed3ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3565979740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f355c1adcd0 con 0x5617ed3f8370 2024-09-10T08:50:14.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3565979740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f355c1ad600 con 0x5617ed3eeb40 2024-09-10T08:50:14.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3565979740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f355c1aa810 con 0x5617ed3ed5d0 2024-09-10T08:50:14.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3565979740 10 client.0 _ll_register_callbacks cb 0x7f354c031d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:14.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3561b3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3f8370 0x5617ed3ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3561b3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3f8370 0x5617ed3ffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38608/0 (socket says 172.21.15.172:38608) 2024-09-10T08:50:14.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3561b3d640 1 -- 172.21.15.172:0/2255066793 learned_addr learned my addr 172.21.15.172:0/2255066793 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:14.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3560b3b640 1 --2- 172.21.15.172:0/2255066793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3eeb40 0x5617ed46f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f356133c640 1 --2- 172.21.15.172:0/2255066793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3ed5d0 0x5617ed46f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3561b3d640 1 -- 172.21.15.172:0/2255066793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3ed5d0 msgr2=0x5617ed46f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:14.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3561b3d640 1 --2- 172.21.15.172:0/2255066793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3ed5d0 0x5617ed46f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3561b3d640 1 -- 172.21.15.172:0/2255066793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3eeb40 msgr2=0x5617ed46f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:14.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3561b3d640 1 --2- 172.21.15.172:0/2255066793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3eeb40 0x5617ed46f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3561b3d640 1 -- 172.21.15.172:0/2255066793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617ed400500 con 0x5617ed3f8370 2024-09-10T08:50:14.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3561b3d640 1 --2- 172.21.15.172:0/2255066793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3f8370 0x5617ed3ffe10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3554018550 tx=0x7f355409c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c0aaf00b2b888c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:14.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.159+0000 7f3544ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:14.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.160+0000 7f3544ff9640 1 -- 172.21.15.172:0/2255066793 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35540b4e60 con 0x5617ed3f8370 2024-09-10T08:50:14.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.160+0000 7f3544ff9640 1 -- 172.21.15.172:0/2255066793 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f355409b2b0 con 0x5617ed3f8370 2024-09-10T08:50:14.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.160+0000 7f3565979740 1 -- 172.21.15.172:0/2255066793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f354c031940 con 0x5617ed3f8370 2024-09-10T08:50:14.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.160+0000 7f3544ff9640 1 -- 172.21.15.172:0/2255066793 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f35540a48d0 con 0x5617ed3f8370 2024-09-10T08:50:14.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.160+0000 7f3565979740 10 client.16539 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:14.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.160+0000 7f3565979740 1 -- 172.21.15.172:0/2255066793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5617ed476be0 con 0x5617ed3f8370 2024-09-10T08:50:14.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.160+0000 7f3544ff9640 1 -- 172.21.15.172:0/2255066793 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35540b1070 con 0x5617ed3f8370 2024-09-10T08:50:14.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.160+0000 7f3544ff9640 1 -- 172.21.15.172:0/2255066793 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f3554065070 con 0x5617ed3f8370 2024-09-10T08:50:14.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.160+0000 7f3565979740 10 client.16539 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:14.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.160+0000 7f3565979740 10 client.16539 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:50:14.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.161+0000 7f3565979740 20 client.16539 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:14.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.161+0000 7f3565979740 1 --2- 172.21.15.172:0/2255066793 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5617ed477300 0x5617ed479720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:14.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.161+0000 7f3565979740 4 client.16539 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:50:14.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.161+0000 7f3565979740 1 -- 172.21.15.172:0/2255066793 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f354c031940 con 0x5617ed477300 2024-09-10T08:50:14.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.161+0000 7f356133c640 1 --2- 172.21.15.172:0/2255066793 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5617ed477300 0x5617ed479720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:14.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.161+0000 7f3544ff9640 10 client.16539 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:50:14.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.161+0000 7f356133c640 1 --2- 172.21.15.172:0/2255066793 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5617ed477300 0x5617ed479720 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=62b8e75ea7d9ea53 server_cookie=145846fa8dd70a6e in_seq=0 out_seq=0 2024-09-10T08:50:14.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.161+0000 7f3544ff9640 1 -- 172.21.15.172:0/2255066793 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f354c031940 con 0x5617ed477300 2024-09-10T08:50:14.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.161+0000 7f3544ff9640 10 client.16539 handle_command_reply: tid=0 2024-09-10T08:50:14.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.161+0000 7f3565979740 1 client.16539 shutdown 2024-09-10T08:50:14.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.162+0000 7f3565979740 1 -- 172.21.15.172:0/2255066793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3f8370 msgr2=0x5617ed3ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:14.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.162+0000 7f3565979740 1 --2- 172.21.15.172:0/2255066793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3f8370 0x5617ed3ffe10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3554018550 tx=0x7f355409c360 comp rx=0 tx=0).stop 2024-09-10T08:50:14.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.162+0000 7f3565979740 1 -- 172.21.15.172:0/2255066793 shutdown_connections 2024-09-10T08:50:14.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.162+0000 7f3565979740 1 --2- 172.21.15.172:0/2255066793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617ed3f8370 0x5617ed3ffe10 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.162+0000 7f3565979740 1 --2- 172.21.15.172:0/2255066793 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5617ed477300 0x5617ed479720 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.162+0000 7f3565979740 1 --2- 172.21.15.172:0/2255066793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617ed3eeb40 0x5617ed46f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.162+0000 7f3565979740 1 --2- 172.21.15.172:0/2255066793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5617ed3ed5d0 0x5617ed46f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.162+0000 7f3565979740 1 -- 172.21.15.172:0/2255066793 >> 172.21.15.172:0/2255066793 conn(0x5617ed3b6310 msgr2=0x5617ed3f7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:14.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.162+0000 7f3544ff9640 0 client.16539 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:50:14.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.162+0000 7f3565979740 1 -- 172.21.15.172:0/2255066793 shutdown_connections 2024-09-10T08:50:14.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.162+0000 7f3565979740 1 -- 172.21.15.172:0/2255066793 wait complete. 2024-09-10T08:50:14.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.162+0000 7f3565979740 20 client.16539 trim_cache size 0 max 16384 2024-09-10T08:50:14.168 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:50:14.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.163+0000 7f35635a0640 1 -- 172.21.15.172:0/2084079585 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f35480a2b80 msgr2=0x7f35480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:14.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.163+0000 7f35635a0640 1 --2- 172.21.15.172:0/2084079585 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f35480a2b80 0x7f35480a5040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f355800abe0 tx=0x7f355800a2d0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.163+0000 7f35635a0640 1 -- 172.21.15.172:0/2084079585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c106f10 msgr2=0x7f355c19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:14.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.163+0000 7f35635a0640 1 --2- 172.21.15.172:0/2084079585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c106f10 0x7f355c19cb70 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f355400cce0 tx=0x7f355400a620 comp rx=0 tx=0).stop 2024-09-10T08:50:14.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.163+0000 7f35635a0640 1 -- 172.21.15.172:0/2084079585 shutdown_connections 2024-09-10T08:50:14.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.163+0000 7f35635a0640 1 --2- 172.21.15.172:0/2084079585 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f35480a2b80 0x7f35480a5040 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.163+0000 7f35635a0640 1 --2- 172.21.15.172:0/2084079585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f355c106f10 0x7f355c19cb70 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.163+0000 7f35635a0640 1 --2- 172.21.15.172:0/2084079585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f355c102790 0x7f355c1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.163+0000 7f35635a0640 1 --2- 172.21.15.172:0/2084079585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f355c101de0 0x7f355c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:14.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.163+0000 7f35635a0640 1 -- 172.21.15.172:0/2084079585 >> 172.21.15.172:0/2084079585 conn(0x7f355c0fd8e0 msgr2=0x7f355c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:14.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.163+0000 7f35635a0640 1 -- 172.21.15.172:0/2084079585 shutdown_connections 2024-09-10T08:50:14.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:14.163+0000 7f35635a0640 1 -- 172.21.15.172:0/2084079585 wait complete. 2024-09-10T08:50:14.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:14 smithi172 ceph-mon[30854]: pgmap v470: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 21 op/s 2024-09-10T08:50:14.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:14 smithi174 ceph-mon[34068]: pgmap v470: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 21 op/s 2024-09-10T08:50:14.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:14 smithi191 ceph-mon[35314]: pgmap v470: 129 pgs: 129 active+clean; 2.4 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 21 op/s 2024-09-10T08:50:14.984 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T08:50:15.313 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:50:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.406+0000 7f1e26978640 1 -- 172.21.15.172:0/1233514539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e200ffbb0 msgr2=0x7f1e20100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.406+0000 7f1e26978640 1 --2- 172.21.15.172:0/1233514539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e200ffbb0 0x7f1e20100030 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009930 tx=0x7f1e1402a420 comp rx=0 tx=0).stop 2024-09-10T08:50:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.406+0000 7f1e26978640 1 -- 172.21.15.172:0/1233514539 shutdown_connections 2024-09-10T08:50:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.406+0000 7f1e26978640 1 --2- 172.21.15.172:0/1233514539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e20100570 0x7f1e2010a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.406+0000 7f1e26978640 1 --2- 172.21.15.172:0/1233514539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e200ffbb0 0x7f1e20100030 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.406+0000 7f1e26978640 1 --2- 172.21.15.172:0/1233514539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e20103140 0x7f1e20103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.406+0000 7f1e26978640 1 -- 172.21.15.172:0/1233514539 >> 172.21.15.172:0/1233514539 conn(0x7f1e200fb940 msgr2=0x7f1e200fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.406+0000 7f1e26978640 1 -- 172.21.15.172:0/1233514539 shutdown_connections 2024-09-10T08:50:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.406+0000 7f1e26978640 1 -- 172.21.15.172:0/1233514539 wait complete. 2024-09-10T08:50:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.406+0000 7f1e26978640 1 Processor -- start 2024-09-10T08:50:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.407+0000 7f1e26978640 1 -- start start 2024-09-10T08:50:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.407+0000 7f1e26978640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e200ffbb0 0x7f1e20197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.407+0000 7f1e26978640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e20100570 0x7f1e20198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.407+0000 7f1e26978640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e20103140 0x7f1e2019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.407+0000 7f1e26978640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2010d690 con 0x7f1e200ffbb0 2024-09-10T08:50:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.407+0000 7f1e26978640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2010d510 con 0x7f1e20100570 2024-09-10T08:50:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.407+0000 7f1e26978640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2010d810 con 0x7f1e20103140 2024-09-10T08:50:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.407+0000 7f1e1ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e200ffbb0 0x7f1e20197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.407+0000 7f1e1ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e200ffbb0 0x7f1e20197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38622/0 (socket says 172.21.15.172:38622) 2024-09-10T08:50:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.407+0000 7f1e1ffff640 1 -- 172.21.15.172:0/2704255102 learned_addr learned my addr 172.21.15.172:0/2704255102 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.407+0000 7f1e1f7fe640 1 --2- 172.21.15.172:0/2704255102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e20100570 0x7f1e20198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.407+0000 7f1e24f15640 1 --2- 172.21.15.172:0/2704255102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e20103140 0x7f1e2019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.407+0000 7f1e1ffff640 1 -- 172.21.15.172:0/2704255102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e20103140 msgr2=0x7f1e2019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.407+0000 7f1e1ffff640 1 --2- 172.21.15.172:0/2704255102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e20103140 0x7f1e2019f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.407+0000 7f1e1ffff640 1 -- 172.21.15.172:0/2704255102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e20100570 msgr2=0x7f1e20198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.407+0000 7f1e1ffff640 1 --2- 172.21.15.172:0/2704255102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e20100570 0x7f1e20198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.407+0000 7f1e1ffff640 1 -- 172.21.15.172:0/2704255102 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2019f900 con 0x7f1e200ffbb0 2024-09-10T08:50:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.408+0000 7f1e1ffff640 1 --2- 172.21.15.172:0/2704255102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e200ffbb0 0x7f1e20197cc0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000c970 tx=0x7f1e1000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.408+0000 7f1e1d7fa640 1 -- 172.21.15.172:0/2704255102 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10007c10 con 0x7f1e200ffbb0 2024-09-10T08:50:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.408+0000 7f1e1d7fa640 1 -- 172.21.15.172:0/2704255102 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e10007db0 con 0x7f1e200ffbb0 2024-09-10T08:50:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.408+0000 7f1e26978640 1 -- 172.21.15.172:0/2704255102 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2019fbf0 con 0x7f1e200ffbb0 2024-09-10T08:50:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.408+0000 7f1e1d7fa640 1 -- 172.21.15.172:0/2704255102 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1000f870 con 0x7f1e200ffbb0 2024-09-10T08:50:15.411 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.408+0000 7f1e26978640 1 -- 172.21.15.172:0/2704255102 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e201a0110 con 0x7f1e200ffbb0 2024-09-10T08:50:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.410+0000 7f1e1d7fa640 1 -- 172.21.15.172:0/2704255102 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1e1000fa10 con 0x7f1e200ffbb0 2024-09-10T08:50:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.410+0000 7f1e297f1740 1 Processor -- start 2024-09-10T08:50:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e297f1740 1 -- start start 2024-09-10T08:50:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e297f1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb304f9a0 0x55feb304fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e297f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb3054da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e297f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb3055370 0x55feb3057830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e297f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55feb2fb32f0 con 0x55feb30502e0 2024-09-10T08:50:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e297f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55feb2f5b270 con 0x55feb3055370 2024-09-10T08:50:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e297f1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55feb2faf2b0 con 0x55feb304f9a0 2024-09-10T08:50:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e1ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb3054da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e1ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb3054da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38630/0 (socket says 172.21.15.172:38630) 2024-09-10T08:50:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e1ffff640 1 -- 172.21.15.172:0/2673940689 learned_addr learned my addr 172.21.15.172:0/2673940689 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e1d7fa640 1 --2- 172.21.15.172:0/2704255102 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1e080a2c30 0x7f1e080a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e1f7fe640 1 --2- 172.21.15.172:0/2673940689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb3055370 0x55feb3057830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e24f15640 1 --2- 172.21.15.172:0/2673940689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb304f9a0 0x55feb304fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e24f15640 1 --2- 172.21.15.172:0/2704255102 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1e080a2c30 0x7f1e080a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e1ffff640 1 -- 172.21.15.172:0/2673940689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb304f9a0 msgr2=0x55feb304fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e1ffff640 1 --2- 172.21.15.172:0/2673940689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb304f9a0 0x55feb304fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e1ffff640 1 -- 172.21.15.172:0/2673940689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb3055370 msgr2=0x55feb3057830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e1ffff640 1 --2- 172.21.15.172:0/2673940689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb3055370 0x55feb3057830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e1ffff640 1 -- 172.21.15.172:0/2673940689 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feb2f5a340 con 0x55feb30502e0 2024-09-10T08:50:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.411+0000 7f1e1d7fa640 1 -- 172.21.15.172:0/2704255102 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1e100cf690 con 0x7f1e200ffbb0 2024-09-10T08:50:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.412+0000 7f1e1ffff640 1 --2- 172.21.15.172:0/2673940689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb3054da0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1e10098080 tx=0x7f1e1009b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12665195ad536a3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.412+0000 7f1e24f15640 1 --2- 172.21.15.172:0/2704255102 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1e080a2c30 0x7f1e080a50f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1e18005fd0 tx=0x7f1e18009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.412+0000 7f1e067fc640 1 -- 172.21.15.172:0/2673940689 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1009b580 con 0x55feb30502e0 2024-09-10T08:50:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.412+0000 7f1e067fc640 1 -- 172.21.15.172:0/2673940689 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e10018b50 con 0x55feb30502e0 2024-09-10T08:50:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.412+0000 7f1e067fc640 1 -- 172.21.15.172:0/2673940689 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e100110f0 con 0x55feb30502e0 2024-09-10T08:50:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.413+0000 7f1e297f1740 1 -- 172.21.15.172:0/2673940689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 msgr2=0x55feb3054da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.413+0000 7f1e297f1740 1 --2- 172.21.15.172:0/2673940689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb3054da0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1e10098080 tx=0x7f1e1009b890 comp rx=0 tx=0).stop 2024-09-10T08:50:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.413+0000 7f1e297f1740 1 -- 172.21.15.172:0/2673940689 shutdown_connections 2024-09-10T08:50:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.413+0000 7f1e297f1740 1 --2- 172.21.15.172:0/2673940689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb3055370 0x55feb3057830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.413+0000 7f1e297f1740 1 --2- 172.21.15.172:0/2673940689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb3054da0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.413+0000 7f1e297f1740 1 --2- 172.21.15.172:0/2673940689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb304f9a0 0x55feb304fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.413+0000 7f1e297f1740 1 -- 172.21.15.172:0/2673940689 >> 172.21.15.172:0/2673940689 conn(0x55feb304b010 msgr2=0x55feb304b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.413+0000 7f1e297f1740 1 -- 172.21.15.172:0/2673940689 shutdown_connections 2024-09-10T08:50:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.413+0000 7f1e297f1740 1 -- 172.21.15.172:0/2673940689 wait complete. 2024-09-10T08:50:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.413+0000 7f1e297f1740 1 -- start start 2024-09-10T08:50:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.413+0000 7f1e297f1740 1 Processor -- start 2024-09-10T08:50:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e297f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb304f9a0 0x55feb3059aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e297f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb30502e0 0x55feb3059fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e297f1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb3055370 0x55feb3061020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e297f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55feb3058280 con 0x55feb304f9a0 2024-09-10T08:50:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e297f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55feb2f3c290 con 0x55feb30502e0 2024-09-10T08:50:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e297f1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55feb3058400 con 0x55feb3055370 2024-09-10T08:50:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e297f1740 10 client.0 _ll_register_callbacks cb 0x55feb2f6d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e1ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb304f9a0 0x55feb3059aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e1ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb304f9a0 0x55feb3059aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38632/0 (socket says 172.21.15.172:38632) 2024-09-10T08:50:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e1ffff640 1 -- 172.21.15.172:0/662761148 learned_addr learned my addr 172.21.15.172:0/662761148 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e1f7fe640 1 --2- 172.21.15.172:0/662761148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb30502e0 0x55feb3059fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e1ffff640 1 -- 172.21.15.172:0/662761148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb3055370 msgr2=0x55feb3061020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e1ffff640 1 --2- 172.21.15.172:0/662761148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb3055370 0x55feb3061020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e1ffff640 1 -- 172.21.15.172:0/662761148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb30502e0 msgr2=0x55feb3059fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e1ffff640 1 --2- 172.21.15.172:0/662761148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb30502e0 0x55feb3059fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e1ffff640 1 -- 172.21.15.172:0/662761148 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feb2cc6210 con 0x55feb304f9a0 2024-09-10T08:50:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e1f7fe640 1 --2- 172.21.15.172:0/662761148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb30502e0 0x55feb3059fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:50:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1e1ffff640 1 --2- 172.21.15.172:0/662761148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb304f9a0 0x55feb3059aa0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1e100a42d0 tx=0x7f1e100d53a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860ee41e5f0b9fb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1dff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:15.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1dff7fe640 1 -- 172.21.15.172:0/662761148 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1009b580 con 0x55feb304f9a0 2024-09-10T08:50:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.414+0000 7f1dff7fe640 1 -- 172.21.15.172:0/662761148 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1e10098310 con 0x55feb304f9a0 2024-09-10T08:50:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.415+0000 7f1e297f1740 1 -- 172.21.15.172:0/662761148 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55feb2fc0a20 con 0x55feb304f9a0 2024-09-10T08:50:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.415+0000 7f1dff7fe640 1 -- 172.21.15.172:0/662761148 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1e1009c320 con 0x55feb304f9a0 2024-09-10T08:50:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.415+0000 7f1e297f1740 10 client.16551 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.415+0000 7f1e297f1740 1 -- 172.21.15.172:0/662761148 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55feb2d92620 con 0x55feb304f9a0 2024-09-10T08:50:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.415+0000 7f1dff7fe640 1 -- 172.21.15.172:0/662761148 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55feb2fc0a20 con 0x55feb304f9a0 2024-09-10T08:50:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.415+0000 7f1dff7fe640 1 -- 172.21.15.172:0/662761148 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f1e100ce020 con 0x55feb304f9a0 2024-09-10T08:50:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.415+0000 7f1e297f1740 10 client.16551 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.415+0000 7f1e297f1740 10 client.16551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.415+0000 7f1e297f1740 20 client.16551 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.415+0000 7f1e297f1740 1 --2- 172.21.15.172:0/662761148 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55feb3065320 0x55feb3067730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.415+0000 7f1e297f1740 4 client.16551 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:50:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.415+0000 7f1e297f1740 1 -- 172.21.15.172:0/662761148 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55feb2fbad70 con 0x55feb3065320 2024-09-10T08:50:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.415+0000 7f1e1f7fe640 1 --2- 172.21.15.172:0/662761148 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55feb3065320 0x55feb3067730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.416+0000 7f1e1f7fe640 1 --2- 172.21.15.172:0/662761148 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55feb3065320 0x55feb3067730 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=869c7cb9fd119163 server_cookie=bf2f621b82e93fbf in_seq=0 out_seq=0 2024-09-10T08:50:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.416+0000 7f1dff7fe640 10 client.16551 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.416+0000 7f1dff7fe640 1 -- 172.21.15.172:0/662761148 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55feb2fbad70 con 0x55feb3065320 2024-09-10T08:50:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.416+0000 7f1dff7fe640 10 client.16551 handle_command_reply: tid=0 2024-09-10T08:50:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.416+0000 7f1e297f1740 1 client.16551 shutdown 2024-09-10T08:50:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.417+0000 7f1e297f1740 1 -- 172.21.15.172:0/662761148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb304f9a0 msgr2=0x55feb3059aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.417+0000 7f1e297f1740 1 --2- 172.21.15.172:0/662761148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb304f9a0 0x55feb3059aa0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1e100a42d0 tx=0x7f1e100d53a0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.417+0000 7f1e297f1740 1 -- 172.21.15.172:0/662761148 shutdown_connections 2024-09-10T08:50:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.417+0000 7f1e297f1740 1 --2- 172.21.15.172:0/662761148 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55feb3065320 0x55feb3067730 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.417+0000 7f1e297f1740 1 --2- 172.21.15.172:0/662761148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb3055370 0x55feb3061020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.417+0000 7f1e297f1740 1 --2- 172.21.15.172:0/662761148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb30502e0 0x55feb3059fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.417+0000 7f1e297f1740 1 --2- 172.21.15.172:0/662761148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb304f9a0 0x55feb3059aa0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.417+0000 7f1e297f1740 1 -- 172.21.15.172:0/662761148 >> 172.21.15.172:0/662761148 conn(0x55feb304b010 msgr2=0x55feb304c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.417+0000 7f1dff7fe640 0 client.16551 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.417+0000 7f1e297f1740 1 -- 172.21.15.172:0/662761148 shutdown_connections 2024-09-10T08:50:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.417+0000 7f1e297f1740 1 -- 172.21.15.172:0/662761148 wait complete. 2024-09-10T08:50:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.417+0000 7f1e297f1740 20 client.16551 trim_cache size 0 max 16384 2024-09-10T08:50:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.431+0000 7f1e297f1740 1 Processor -- start 2024-09-10T08:50:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.431+0000 7f1e297f1740 1 -- start start 2024-09-10T08:50:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.431+0000 7f1e297f1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb304f9a0 0x55feb304c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.431+0000 7f1e297f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb3060fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.431+0000 7f1e297f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb3055370 0x55feb306c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.431+0000 7f1e297f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1e201aa560 con 0x55feb30502e0 2024-09-10T08:50:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.431+0000 7f1e297f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e201ac160 con 0x55feb3055370 2024-09-10T08:50:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.431+0000 7f1e297f1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1e201aac30 con 0x55feb304f9a0 2024-09-10T08:50:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.431+0000 7f1e1f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb3060fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.431+0000 7f1e1f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb3060fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38638/0 (socket says 172.21.15.172:38638) 2024-09-10T08:50:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.431+0000 7f1e1f7fe640 1 -- 172.21.15.172:0/2661286662 learned_addr learned my addr 172.21.15.172:0/2661286662 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.431+0000 7f1e24f15640 1 --2- 172.21.15.172:0/2661286662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb3055370 0x55feb306c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.431+0000 7f1e1ffff640 1 --2- 172.21.15.172:0/2661286662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb304f9a0 0x55feb304c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.432+0000 7f1e1f7fe640 1 -- 172.21.15.172:0/2661286662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb304f9a0 msgr2=0x55feb304c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.432+0000 7f1e1f7fe640 1 --2- 172.21.15.172:0/2661286662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb304f9a0 0x55feb304c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.432+0000 7f1e1f7fe640 1 -- 172.21.15.172:0/2661286662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb3055370 msgr2=0x55feb306c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.432+0000 7f1e1f7fe640 1 --2- 172.21.15.172:0/2661286662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb3055370 0x55feb306c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.432+0000 7f1e1f7fe640 1 -- 172.21.15.172:0/2661286662 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feb3050720 con 0x55feb30502e0 2024-09-10T08:50:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.432+0000 7f1e1f7fe640 1 --2- 172.21.15.172:0/2661286662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb3060fe0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009230 tx=0x7f1e14000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298befcd1e29e62c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.432+0000 7f1e05ffb640 1 -- 172.21.15.172:0/2661286662 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1404a070 con 0x55feb30502e0 2024-09-10T08:50:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.432+0000 7f1e05ffb640 1 -- 172.21.15.172:0/2661286662 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e140043f0 con 0x55feb30502e0 2024-09-10T08:50:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.432+0000 7f1e05ffb640 1 -- 172.21.15.172:0/2661286662 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14035040 con 0x55feb30502e0 2024-09-10T08:50:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.433+0000 7f1e297f1740 1 -- 172.21.15.172:0/2661286662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 msgr2=0x55feb3060fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.433+0000 7f1e297f1740 1 --2- 172.21.15.172:0/2661286662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb3060fe0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009230 tx=0x7f1e14000bc0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.433+0000 7f1e297f1740 1 -- 172.21.15.172:0/2661286662 shutdown_connections 2024-09-10T08:50:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.433+0000 7f1e297f1740 1 --2- 172.21.15.172:0/2661286662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb3055370 0x55feb306c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.433+0000 7f1e297f1740 1 --2- 172.21.15.172:0/2661286662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb3060fe0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.433+0000 7f1e297f1740 1 --2- 172.21.15.172:0/2661286662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb304f9a0 0x55feb304c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.433+0000 7f1e297f1740 1 -- 172.21.15.172:0/2661286662 >> 172.21.15.172:0/2661286662 conn(0x55feb304b010 msgr2=0x55feb304bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.433+0000 7f1e297f1740 1 -- 172.21.15.172:0/2661286662 shutdown_connections 2024-09-10T08:50:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.433+0000 7f1e297f1740 1 -- 172.21.15.172:0/2661286662 wait complete. 2024-09-10T08:50:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.433+0000 7f1e297f1740 1 -- start start 2024-09-10T08:50:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.433+0000 7f1e297f1740 1 Processor -- start 2024-09-10T08:50:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e297f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb304f9a0 0x55feb305e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e297f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb305e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e297f1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb3055370 0x55feb305f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e297f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1e201aba00 con 0x55feb30502e0 2024-09-10T08:50:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e297f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e201ab330 con 0x55feb304f9a0 2024-09-10T08:50:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e297f1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1e201a8540 con 0x55feb3055370 2024-09-10T08:50:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e297f1740 10 client.0 _ll_register_callbacks cb 0x7f1e14009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e1f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb305e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e1f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb305e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38646/0 (socket says 172.21.15.172:38646) 2024-09-10T08:50:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e1f7fe640 1 -- 172.21.15.172:0/820759555 learned_addr learned my addr 172.21.15.172:0/820759555 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e1ffff640 1 --2- 172.21.15.172:0/820759555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb304f9a0 0x55feb305e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e24f15640 1 --2- 172.21.15.172:0/820759555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb3055370 0x55feb305f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e1f7fe640 1 -- 172.21.15.172:0/820759555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb3055370 msgr2=0x55feb305f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e1f7fe640 1 --2- 172.21.15.172:0/820759555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb3055370 0x55feb305f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e1f7fe640 1 -- 172.21.15.172:0/820759555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb304f9a0 msgr2=0x55feb305e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e1f7fe640 1 --2- 172.21.15.172:0/820759555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb304f9a0 0x55feb305e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e1f7fe640 1 -- 172.21.15.172:0/820759555 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feb305fce0 con 0x55feb30502e0 2024-09-10T08:50:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1e1f7fe640 1 --2- 172.21.15.172:0/820759555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb305e6c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1e1402a460 tx=0x7f1e140317f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f759af5264b81ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.434+0000 7f1dff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.435+0000 7f1dff7fe640 1 -- 172.21.15.172:0/820759555 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1404a040 con 0x55feb30502e0 2024-09-10T08:50:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.435+0000 7f1e297f1740 1 -- 172.21.15.172:0/820759555 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e140043f0 con 0x55feb30502e0 2024-09-10T08:50:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.435+0000 7f1dff7fe640 1 -- 172.21.15.172:0/820759555 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1e14003b70 con 0x55feb30502e0 2024-09-10T08:50:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.435+0000 7f1dff7fe640 1 -- 172.21.15.172:0/820759555 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1e1405c050 con 0x55feb30502e0 2024-09-10T08:50:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.435+0000 7f1e297f1740 10 client.16557 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.435+0000 7f1e297f1740 1 -- 172.21.15.172:0/820759555 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55feb3060ff0 con 0x55feb30502e0 2024-09-10T08:50:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.435+0000 7f1dff7fe640 1 -- 172.21.15.172:0/820759555 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e140043f0 con 0x55feb30502e0 2024-09-10T08:50:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.435+0000 7f1dff7fe640 1 -- 172.21.15.172:0/820759555 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f1e1404d070 con 0x55feb30502e0 2024-09-10T08:50:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.435+0000 7f1e297f1740 10 client.16557 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.435+0000 7f1e297f1740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.435+0000 7f1e297f1740 20 client.16557 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.435+0000 7f1e297f1740 1 --2- 172.21.15.172:0/820759555 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55feb30d04f0 0x55feb30d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.435+0000 7f1e297f1740 4 client.16557 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:50:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.435+0000 7f1e297f1740 1 -- 172.21.15.172:0/820759555 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55feb2f37900 con 0x55feb30d04f0 2024-09-10T08:50:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.435+0000 7f1e1ffff640 1 --2- 172.21.15.172:0/820759555 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55feb30d04f0 0x55feb30d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.436+0000 7f1e1ffff640 1 --2- 172.21.15.172:0/820759555 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55feb30d04f0 0x55feb30d2910 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2815b5983233f25a server_cookie=6a11b8333a630cc9 in_seq=0 out_seq=0 2024-09-10T08:50:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.436+0000 7f1dff7fe640 10 client.16557 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.442+0000 7f1dff7fe640 1 -- 172.21.15.172:0/820759555 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55feb2f37900 con 0x55feb30d04f0 2024-09-10T08:50:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.442+0000 7f1dff7fe640 10 client.16557 handle_command_reply: tid=0 2024-09-10T08:50:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.442+0000 7f1e297f1740 1 client.16557 shutdown 2024-09-10T08:50:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.442+0000 7f1e297f1740 1 -- 172.21.15.172:0/820759555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 msgr2=0x55feb305e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.442+0000 7f1e297f1740 1 --2- 172.21.15.172:0/820759555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb305e6c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1e1402a460 tx=0x7f1e140317f0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.442+0000 7f1e297f1740 1 -- 172.21.15.172:0/820759555 shutdown_connections 2024-09-10T08:50:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.442+0000 7f1e297f1740 1 --2- 172.21.15.172:0/820759555 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55feb30d04f0 0x55feb30d2910 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1dff7fe640 0 client.16557 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.442+0000 7f1e297f1740 1 --2- 172.21.15.172:0/820759555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55feb3055370 0x55feb305f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1e297f1740 1 --2- 172.21.15.172:0/820759555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55feb30502e0 0x55feb305e6c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1e297f1740 1 --2- 172.21.15.172:0/820759555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55feb304f9a0 0x55feb305e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1e297f1740 1 -- 172.21.15.172:0/820759555 >> 172.21.15.172:0/820759555 conn(0x55feb304b010 msgr2=0x55feb304bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1e297f1740 1 -- 172.21.15.172:0/820759555 shutdown_connections 2024-09-10T08:50:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1e297f1740 1 -- 172.21.15.172:0/820759555 wait complete. 2024-09-10T08:50:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1e297f1740 20 client.16557 trim_cache size 0 max 16384 2024-09-10T08:50:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:50:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:50:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:50:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:50:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1e26978640 1 -- 172.21.15.172:0/2704255102 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1e080a2c30 msgr2=0x7f1e080a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1e26978640 1 --2- 172.21.15.172:0/2704255102 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1e080a2c30 0x7f1e080a50f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1e18005fd0 tx=0x7f1e18009600 comp rx=0 tx=0).stop 2024-09-10T08:50:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1e26978640 1 -- 172.21.15.172:0/2704255102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e200ffbb0 msgr2=0x7f1e20197cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1e26978640 1 --2- 172.21.15.172:0/2704255102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e200ffbb0 0x7f1e20197cc0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000c970 tx=0x7f1e1000ce30 comp rx=0 tx=0).stop 2024-09-10T08:50:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1e26978640 1 -- 172.21.15.172:0/2704255102 shutdown_connections 2024-09-10T08:50:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1e26978640 1 --2- 172.21.15.172:0/2704255102 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1e080a2c30 0x7f1e080a50f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1e26978640 1 --2- 172.21.15.172:0/2704255102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e20103140 0x7f1e2019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1e26978640 1 --2- 172.21.15.172:0/2704255102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e20100570 0x7f1e20198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1e26978640 1 --2- 172.21.15.172:0/2704255102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e200ffbb0 0x7f1e20197cc0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1e26978640 1 -- 172.21.15.172:0/2704255102 >> 172.21.15.172:0/2704255102 conn(0x7f1e200fb940 msgr2=0x7f1e20106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.443+0000 7f1e26978640 1 -- 172.21.15.172:0/2704255102 shutdown_connections 2024-09-10T08:50:15.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:15.444+0000 7f1e26978640 1 -- 172.21.15.172:0/2704255102 wait complete. 2024-09-10T08:50:15.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.887+0000 7f0c5d45a640 1 -- 172.21.15.172:0/674624413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c58100430 msgr2=0x7f0c58100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.887+0000 7f0c5d45a640 1 --2- 172.21.15.172:0/674624413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c58100430 0x7f0c58100810 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009900 tx=0x7f0c4802a410 comp rx=0 tx=0).stop 2024-09-10T08:50:15.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.887+0000 7f0c5d45a640 1 -- 172.21.15.172:0/674624413 shutdown_connections 2024-09-10T08:50:15.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.887+0000 7f0c5d45a640 1 --2- 172.21.15.172:0/674624413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c58105560 0x7f0c5810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.887+0000 7f0c5d45a640 1 --2- 172.21.15.172:0/674624413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c58100de0 0x7f0c58104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.887+0000 7f0c5d45a640 1 --2- 172.21.15.172:0/674624413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c58100430 0x7f0c58100810 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.887+0000 7f0c5d45a640 1 -- 172.21.15.172:0/674624413 >> 172.21.15.172:0/674624413 conn(0x7f0c58077230 msgr2=0x7f0c580fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:15.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.887+0000 7f0c5d45a640 1 -- 172.21.15.172:0/674624413 shutdown_connections 2024-09-10T08:50:15.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.887+0000 7f0c5d45a640 1 -- 172.21.15.172:0/674624413 wait complete. 2024-09-10T08:50:15.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.888+0000 7f0c5d45a640 1 Processor -- start 2024-09-10T08:50:15.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.888+0000 7f0c5d45a640 1 -- start start 2024-09-10T08:50:15.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.888+0000 7f0c5d45a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c58100430 0x7f0c58196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.888+0000 7f0c5d45a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c58100de0 0x7f0c58196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.888+0000 7f0c5d45a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c58105560 0x7f0c5819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.888+0000 7f0c5d45a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5810fbf0 con 0x7f0c58105560 2024-09-10T08:50:15.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.888+0000 7f0c5d45a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5810fa70 con 0x7f0c58100de0 2024-09-10T08:50:15.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.888+0000 7f0c5d45a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5810fd70 con 0x7f0c58100430 2024-09-10T08:50:15.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.888+0000 7f0c577fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c58105560 0x7f0c5819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.889+0000 7f0c56ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c58100430 0x7f0c58196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.889+0000 7f0c577fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c58105560 0x7f0c5819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38664/0 (socket says 172.21.15.172:38664) 2024-09-10T08:50:15.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.889+0000 7f0c577fe640 1 -- 172.21.15.172:0/2983814597 learned_addr learned my addr 172.21.15.172:0/2983814597 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:15.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.889+0000 7f0c567fc640 1 --2- 172.21.15.172:0/2983814597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c58100de0 0x7f0c58196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.889+0000 7f0c577fe640 1 -- 172.21.15.172:0/2983814597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c58100430 msgr2=0x7f0c58196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.889+0000 7f0c577fe640 1 --2- 172.21.15.172:0/2983814597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c58100430 0x7f0c58196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.889+0000 7f0c577fe640 1 -- 172.21.15.172:0/2983814597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c58100de0 msgr2=0x7f0c58196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.889+0000 7f0c577fe640 1 --2- 172.21.15.172:0/2983814597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c58100de0 0x7f0c58196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.889+0000 7f0c577fe640 1 -- 172.21.15.172:0/2983814597 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5819b0e0 con 0x7f0c58105560 2024-09-10T08:50:15.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.889+0000 7f0c567fc640 1 --2- 172.21.15.172:0/2983814597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c58100de0 0x7f0c58196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:15.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.889+0000 7f0c577fe640 1 --2- 172.21.15.172:0/2983814597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c58105560 0x7f0c5819aa00 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000de10 tx=0x7f0c5000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:15.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.889+0000 7f0c3ffff640 1 -- 172.21.15.172:0/2983814597 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50003b70 con 0x7f0c58105560 2024-09-10T08:50:15.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.889+0000 7f0c5d45a640 1 -- 172.21.15.172:0/2983814597 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c5819b3d0 con 0x7f0c58105560 2024-09-10T08:50:15.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.889+0000 7f0c3ffff640 1 -- 172.21.15.172:0/2983814597 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c50003d10 con 0x7f0c58105560 2024-09-10T08:50:15.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.889+0000 7f0c3ffff640 1 -- 172.21.15.172:0/2983814597 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5000ea60 con 0x7f0c58105560 2024-09-10T08:50:15.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.889+0000 7f0c5d45a640 1 -- 172.21.15.172:0/2983814597 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c581a2cb0 con 0x7f0c58105560 2024-09-10T08:50:15.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.892+0000 7f0c3ffff640 1 -- 172.21.15.172:0/2983814597 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0c5000ec00 con 0x7f0c58105560 2024-09-10T08:50:15.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.892+0000 7f0c5f833740 1 Processor -- start 2024-09-10T08:50:15.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.892+0000 7f0c5f833740 1 -- start start 2024-09-10T08:50:15.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c3ffff640 1 --2- 172.21.15.172:0/2983814597 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c400a2b80 0x7f0c400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c5f833740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635776b40 0x556635775090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c5f833740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5566357755d0 0x556635775a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c5f833740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635780370 0x556635782760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c5f833740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5566356db2f0 con 0x556635776b40 2024-09-10T08:50:15.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c5f833740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556635682cf0 con 0x556635780370 2024-09-10T08:50:15.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c5f833740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5566356d72b0 con 0x5566357755d0 2024-09-10T08:50:15.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c3ffff640 1 -- 172.21.15.172:0/2983814597 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0c500cf6b0 con 0x7f0c58105560 2024-09-10T08:50:15.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c56ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635776b40 0x556635775090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c567fc640 1 --2- 172.21.15.172:0/2983814597 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c400a2b80 0x7f0c400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c56ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635776b40 0x556635775090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38676/0 (socket says 172.21.15.172:38676) 2024-09-10T08:50:15.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c56ffd640 1 -- 172.21.15.172:0/320670265 learned_addr learned my addr 172.21.15.172:0/320670265 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:15.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c577fe640 1 --2- 172.21.15.172:0/320670265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635780370 0x556635782760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c56ffd640 1 -- 172.21.15.172:0/320670265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5566357755d0 msgr2=0x556635775a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:50:15.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c56ffd640 1 --2- 172.21.15.172:0/320670265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5566357755d0 0x556635775a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c56ffd640 1 -- 172.21.15.172:0/320670265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635780370 msgr2=0x556635782760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:15.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c56ffd640 1 --2- 172.21.15.172:0/320670265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635780370 0x556635782760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.893+0000 7f0c56ffd640 1 -- 172.21.15.172:0/320670265 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566356673a0 con 0x556635776b40 2024-09-10T08:50:15.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.894+0000 7f0c56ffd640 1 --2- 172.21.15.172:0/320670265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635776b40 0x556635775090 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009900 tx=0x7f0c48032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d354f87191b54e92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:15.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.894+0000 7f0c567fc640 1 --2- 172.21.15.172:0/2983814597 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c400a2b80 0x7f0c400a5040 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0c580ff930 tx=0x7f0c4c00b690 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:15.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.894+0000 7f0c3d7fa640 1 -- 172.21.15.172:0/320670265 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c48040070 con 0x556635776b40 2024-09-10T08:50:15.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.894+0000 7f0c3d7fa640 1 -- 172.21.15.172:0/320670265 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c48004030 con 0x556635776b40 2024-09-10T08:50:15.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.894+0000 7f0c3d7fa640 1 -- 172.21.15.172:0/320670265 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c48035040 con 0x556635776b40 2024-09-10T08:50:15.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.895+0000 7f0c5f833740 1 -- 172.21.15.172:0/320670265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635776b40 msgr2=0x556635775090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.895+0000 7f0c5f833740 1 --2- 172.21.15.172:0/320670265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635776b40 0x556635775090 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009900 tx=0x7f0c48032bb0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.895+0000 7f0c5f833740 1 -- 172.21.15.172:0/320670265 shutdown_connections 2024-09-10T08:50:15.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.895+0000 7f0c5f833740 1 --2- 172.21.15.172:0/320670265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635780370 0x556635782760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.895+0000 7f0c5f833740 1 --2- 172.21.15.172:0/320670265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5566357755d0 0x556635775a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.895+0000 7f0c5f833740 1 --2- 172.21.15.172:0/320670265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635776b40 0x556635775090 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.895+0000 7f0c5f833740 1 -- 172.21.15.172:0/320670265 >> 172.21.15.172:0/320670265 conn(0x55663573e310 msgr2=0x556635771c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:15.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.895+0000 7f0c5f833740 1 -- 172.21.15.172:0/320670265 shutdown_connections 2024-09-10T08:50:15.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.895+0000 7f0c5f833740 1 -- 172.21.15.172:0/320670265 wait complete. 2024-09-10T08:50:15.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.895+0000 7f0c5f833740 1 -- start start 2024-09-10T08:50:15.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.895+0000 7f0c5f833740 1 Processor -- start 2024-09-10T08:50:15.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.895+0000 7f0c5f833740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5566357755d0 0x55663577fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c5f833740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635776b40 0x55663577d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c5f833740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635780370 0x55663577d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c5f833740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556635783010 con 0x556635780370 2024-09-10T08:50:15.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c5f833740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556635696060 con 0x556635776b40 2024-09-10T08:50:15.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c5f833740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556635783190 con 0x5566357755d0 2024-09-10T08:50:15.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c5f833740 10 client.0 _ll_register_callbacks cb 0x5566353e98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:15.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c577fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635780370 0x55663577d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c577fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635780370 0x55663577d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38682/0 (socket says 172.21.15.172:38682) 2024-09-10T08:50:15.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c577fe640 1 -- 172.21.15.172:0/794697272 learned_addr learned my addr 172.21.15.172:0/794697272 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:15.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c567fc640 1 --2- 172.21.15.172:0/794697272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635776b40 0x55663577d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c56ffd640 1 --2- 172.21.15.172:0/794697272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5566357755d0 0x55663577fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c577fe640 1 -- 172.21.15.172:0/794697272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5566357755d0 msgr2=0x55663577fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:15.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c577fe640 1 --2- 172.21.15.172:0/794697272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5566357755d0 0x55663577fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c577fe640 1 -- 172.21.15.172:0/794697272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635776b40 msgr2=0x55663577d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:15.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c577fe640 1 --2- 172.21.15.172:0/794697272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635776b40 0x55663577d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c577fe640 1 -- 172.21.15.172:0/794697272 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55663544c3c0 con 0x556635780370 2024-09-10T08:50:15.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c577fe640 1 --2- 172.21.15.172:0/794697272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635780370 0x55663577d650 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0c500cce50 tx=0x7f0c5009b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6ccb2c816b44b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:15.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.896+0000 7f0c3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:15.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.897+0000 7f0c3aff5640 1 -- 172.21.15.172:0/794697272 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5009ff60 con 0x556635780370 2024-09-10T08:50:15.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.897+0000 7f0c3aff5640 1 -- 172.21.15.172:0/794697272 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0c50098220 con 0x556635780370 2024-09-10T08:50:15.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.897+0000 7f0c5f833740 1 -- 172.21.15.172:0/794697272 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c48004030 con 0x556635780370 2024-09-10T08:50:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.897+0000 7f0c3aff5640 1 -- 172.21.15.172:0/794697272 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0c500a4820 con 0x556635780370 2024-09-10T08:50:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.897+0000 7f0c5f833740 10 client.16569 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.897+0000 7f0c5f833740 1 -- 172.21.15.172:0/794697272 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55663578eb40 con 0x556635780370 2024-09-10T08:50:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.897+0000 7f0c3aff5640 1 -- 172.21.15.172:0/794697272 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c48004030 con 0x556635780370 2024-09-10T08:50:15.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.897+0000 7f0c3aff5640 1 -- 172.21.15.172:0/794697272 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f0c5000b040 con 0x556635780370 2024-09-10T08:50:15.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.897+0000 7f0c5f833740 10 client.16569 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:15.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.897+0000 7f0c5f833740 10 client.16569 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:50:15.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.897+0000 7f0c5f833740 20 client.16569 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:15.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.897+0000 7f0c5f833740 1 --2- 172.21.15.172:0/794697272 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55663578ffa0 0x556635792420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.897+0000 7f0c5f833740 4 client.16569 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:50:15.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.897+0000 7f0c5f833740 1 -- 172.21.15.172:0/794697272 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566356e8860 con 0x55663578ffa0 2024-09-10T08:50:15.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.898+0000 7f0c56ffd640 1 --2- 172.21.15.172:0/794697272 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55663578ffa0 0x556635792420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.898+0000 7f0c56ffd640 1 --2- 172.21.15.172:0/794697272 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55663578ffa0 0x556635792420 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e65ab5a1012da1f6 server_cookie=9efd84ebb04f9ed9 in_seq=0 out_seq=0 2024-09-10T08:50:15.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.898+0000 7f0c3aff5640 10 client.16569 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:50:15.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.899+0000 7f0c3aff5640 1 -- 172.21.15.172:0/794697272 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566356e8860 con 0x55663578ffa0 2024-09-10T08:50:15.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.899+0000 7f0c3aff5640 10 client.16569 handle_command_reply: tid=0 2024-09-10T08:50:15.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.899+0000 7f0c5f833740 1 client.16569 shutdown 2024-09-10T08:50:15.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.899+0000 7f0c5f833740 1 -- 172.21.15.172:0/794697272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635780370 msgr2=0x55663577d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.899+0000 7f0c5f833740 1 --2- 172.21.15.172:0/794697272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635780370 0x55663577d650 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0c500cce50 tx=0x7f0c5009b770 comp rx=0 tx=0).stop 2024-09-10T08:50:15.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.899+0000 7f0c5f833740 1 -- 172.21.15.172:0/794697272 shutdown_connections 2024-09-10T08:50:15.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.899+0000 7f0c5f833740 1 --2- 172.21.15.172:0/794697272 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55663578ffa0 0x556635792420 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.899+0000 7f0c5f833740 1 --2- 172.21.15.172:0/794697272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635780370 0x55663577d650 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.899+0000 7f0c5f833740 1 --2- 172.21.15.172:0/794697272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635776b40 0x55663577d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.899+0000 7f0c3aff5640 0 client.16569 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:50:15.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.899+0000 7f0c5f833740 1 --2- 172.21.15.172:0/794697272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5566357755d0 0x55663577fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.899+0000 7f0c5f833740 1 -- 172.21.15.172:0/794697272 >> 172.21.15.172:0/794697272 conn(0x55663573e310 msgr2=0x556635777960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:15.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.900+0000 7f0c5f833740 1 -- 172.21.15.172:0/794697272 shutdown_connections 2024-09-10T08:50:15.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.900+0000 7f0c5f833740 1 -- 172.21.15.172:0/794697272 wait complete. 2024-09-10T08:50:15.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.900+0000 7f0c5f833740 20 client.16569 trim_cache size 0 max 16384 2024-09-10T08:50:15.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.919+0000 7f0c5f833740 1 Processor -- start 2024-09-10T08:50:15.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.919+0000 7f0c5f833740 1 -- start start 2024-09-10T08:50:15.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.919+0000 7f0c5f833740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566357755d0 0x55663578c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.919+0000 7f0c5f833740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635776b40 0x556635778120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.919+0000 7f0c5f833740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556635780370 0x556635778660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.919+0000 7f0c5f833740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581b0b60 con 0x5566357755d0 2024-09-10T08:50:15.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.919+0000 7f0c5f833740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581b2760 con 0x556635776b40 2024-09-10T08:50:15.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.919+0000 7f0c5f833740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581b1230 con 0x556635780370 2024-09-10T08:50:15.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.919+0000 7f0c56ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566357755d0 0x55663578c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.919+0000 7f0c56ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566357755d0 0x55663578c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38690/0 (socket says 172.21.15.172:38690) 2024-09-10T08:50:15.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.919+0000 7f0c56ffd640 1 -- 172.21.15.172:0/2126562857 learned_addr learned my addr 172.21.15.172:0/2126562857 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:15.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.919+0000 7f0c567fc640 1 --2- 172.21.15.172:0/2126562857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635776b40 0x556635778120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.920+0000 7f0c56ffd640 1 -- 172.21.15.172:0/2126562857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556635780370 msgr2=0x556635778660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:15.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.920+0000 7f0c577fe640 1 --2- 172.21.15.172:0/2126562857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556635780370 0x556635778660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.920+0000 7f0c56ffd640 1 --2- 172.21.15.172:0/2126562857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556635780370 0x556635778660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.920+0000 7f0c56ffd640 1 -- 172.21.15.172:0/2126562857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635776b40 msgr2=0x556635778120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:15.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.920+0000 7f0c56ffd640 1 --2- 172.21.15.172:0/2126562857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635776b40 0x556635778120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.920+0000 7f0c56ffd640 1 -- 172.21.15.172:0/2126562857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556635771ed0 con 0x5566357755d0 2024-09-10T08:50:15.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.920+0000 7f0c567fc640 1 --2- 172.21.15.172:0/2126562857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635776b40 0x556635778120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:50:15.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.920+0000 7f0c56ffd640 1 --2- 172.21.15.172:0/2126562857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566357755d0 0x55663578c2b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009900 tx=0x7f0c480051e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe4e0cad7313e0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:15.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.920+0000 7f0c3cff9640 1 -- 172.21.15.172:0/2126562857 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c48040070 con 0x5566357755d0 2024-09-10T08:50:15.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.920+0000 7f0c3cff9640 1 -- 172.21.15.172:0/2126562857 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c480054c0 con 0x5566357755d0 2024-09-10T08:50:15.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.920+0000 7f0c3cff9640 1 -- 172.21.15.172:0/2126562857 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c48035040 con 0x5566357755d0 2024-09-10T08:50:15.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.921+0000 7f0c5f833740 1 -- 172.21.15.172:0/2126562857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566357755d0 msgr2=0x55663578c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.921+0000 7f0c5f833740 1 --2- 172.21.15.172:0/2126562857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566357755d0 0x55663578c2b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009900 tx=0x7f0c480051e0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.921+0000 7f0c5f833740 1 -- 172.21.15.172:0/2126562857 shutdown_connections 2024-09-10T08:50:15.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.921+0000 7f0c5f833740 1 --2- 172.21.15.172:0/2126562857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556635780370 0x556635778660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.921+0000 7f0c5f833740 1 --2- 172.21.15.172:0/2126562857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556635776b40 0x556635778120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.921+0000 7f0c5f833740 1 --2- 172.21.15.172:0/2126562857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566357755d0 0x55663578c2b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.921+0000 7f0c5f833740 1 -- 172.21.15.172:0/2126562857 >> 172.21.15.172:0/2126562857 conn(0x55663573e310 msgr2=0x556635770d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:15.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.921+0000 7f0c5f833740 1 -- 172.21.15.172:0/2126562857 shutdown_connections 2024-09-10T08:50:15.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.921+0000 7f0c5f833740 1 -- 172.21.15.172:0/2126562857 wait complete. 2024-09-10T08:50:15.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.922+0000 7f0c5f833740 1 -- start start 2024-09-10T08:50:15.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.922+0000 7f0c5f833740 1 Processor -- start 2024-09-10T08:50:15.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.922+0000 7f0c5f833740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566357755d0 0x556635788960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.922+0000 7f0c5f833740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556635776b40 0x556635783c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.922+0000 7f0c5f833740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635780370 0x5566357841d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.922+0000 7f0c5f833740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581b2000 con 0x556635780370 2024-09-10T08:50:15.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.922+0000 7f0c5f833740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581b1930 con 0x5566357755d0 2024-09-10T08:50:15.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.922+0000 7f0c5f833740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581aeb40 con 0x556635776b40 2024-09-10T08:50:15.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.922+0000 7f0c577fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635780370 0x5566357841d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.922+0000 7f0c5f833740 10 client.0 _ll_register_callbacks cb 0x7f0c5000bf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:15.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.922+0000 7f0c577fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635780370 0x5566357841d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38698/0 (socket says 172.21.15.172:38698) 2024-09-10T08:50:15.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.922+0000 7f0c577fe640 1 -- 172.21.15.172:0/3000884077 learned_addr learned my addr 172.21.15.172:0/3000884077 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:15.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.922+0000 7f0c567fc640 1 --2- 172.21.15.172:0/3000884077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556635776b40 0x556635783c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.922+0000 7f0c56ffd640 1 --2- 172.21.15.172:0/3000884077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566357755d0 0x556635788960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.923+0000 7f0c577fe640 1 -- 172.21.15.172:0/3000884077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556635776b40 msgr2=0x556635783c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:15.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.923+0000 7f0c577fe640 1 --2- 172.21.15.172:0/3000884077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556635776b40 0x556635783c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.923+0000 7f0c577fe640 1 -- 172.21.15.172:0/3000884077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566357755d0 msgr2=0x556635788960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:15.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.923+0000 7f0c577fe640 1 --2- 172.21.15.172:0/3000884077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566357755d0 0x556635788960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.923+0000 7f0c577fe640 1 -- 172.21.15.172:0/3000884077 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566357847c0 con 0x556635780370 2024-09-10T08:50:15.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.923+0000 7f0c56ffd640 1 --2- 172.21.15.172:0/3000884077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566357755d0 0x556635788960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:50:15.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.923+0000 7f0c577fe640 1 --2- 172.21.15.172:0/3000884077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635780370 0x5566357841d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0c50098580 tx=0x7f0c5000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf7ac7c5cdd59eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:15.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.923+0000 7f0c3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:15.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.923+0000 7f0c3aff5640 1 -- 172.21.15.172:0/3000884077 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c500b8070 con 0x556635780370 2024-09-10T08:50:15.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.923+0000 7f0c3aff5640 1 -- 172.21.15.172:0/3000884077 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0c50097900 con 0x556635780370 2024-09-10T08:50:15.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.923+0000 7f0c5f833740 1 -- 172.21.15.172:0/3000884077 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c480054c0 con 0x556635780370 2024-09-10T08:50:15.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.923+0000 7f0c3aff5640 1 -- 172.21.15.172:0/3000884077 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0c50092050 con 0x556635780370 2024-09-10T08:50:15.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.923+0000 7f0c5f833740 10 client.16575 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:15.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.923+0000 7f0c5f833740 1 -- 172.21.15.172:0/3000884077 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5566357faac0 con 0x556635780370 2024-09-10T08:50:15.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.923+0000 7f0c3aff5640 1 -- 172.21.15.172:0/3000884077 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c480054c0 con 0x556635780370 2024-09-10T08:50:15.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.924+0000 7f0c3aff5640 1 -- 172.21.15.172:0/3000884077 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f0c50086400 con 0x556635780370 2024-09-10T08:50:15.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.924+0000 7f0c5f833740 10 client.16575 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:15.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.924+0000 7f0c5f833740 10 client.16575 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:50:15.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.924+0000 7f0c5f833740 20 client.16575 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:15.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.924+0000 7f0c5f833740 1 --2- 172.21.15.172:0/3000884077 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5566357fb180 0x5566357fd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:15.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.924+0000 7f0c5f833740 4 client.16575 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:50:15.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.924+0000 7f0c5f833740 1 -- 172.21.15.172:0/3000884077 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566356e2d70 con 0x5566357fb180 2024-09-10T08:50:15.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.924+0000 7f0c56ffd640 1 --2- 172.21.15.172:0/3000884077 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5566357fb180 0x5566357fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:15.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.924+0000 7f0c3aff5640 10 client.16575 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:50:15.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.924+0000 7f0c56ffd640 1 --2- 172.21.15.172:0/3000884077 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5566357fb180 0x5566357fd570 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=39ae8a67fd47f887 server_cookie=298d61dcdbf30ece in_seq=0 out_seq=0 2024-09-10T08:50:15.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.924+0000 7f0c3aff5640 1 -- 172.21.15.172:0/3000884077 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566356e2d70 con 0x5566357fb180 2024-09-10T08:50:15.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.924+0000 7f0c3aff5640 10 client.16575 handle_command_reply: tid=0 2024-09-10T08:50:15.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.924+0000 7f0c5f833740 1 client.16575 shutdown 2024-09-10T08:50:15.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.925+0000 7f0c5f833740 1 -- 172.21.15.172:0/3000884077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635780370 msgr2=0x5566357841d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.925+0000 7f0c5f833740 1 --2- 172.21.15.172:0/3000884077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635780370 0x5566357841d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0c50098580 tx=0x7f0c5000eef0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.925+0000 7f0c5f833740 1 -- 172.21.15.172:0/3000884077 shutdown_connections 2024-09-10T08:50:15.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.925+0000 7f0c5f833740 1 --2- 172.21.15.172:0/3000884077 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5566357fb180 0x5566357fd570 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.925+0000 7f0c5f833740 1 --2- 172.21.15.172:0/3000884077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556635780370 0x5566357841d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.925+0000 7f0c5f833740 1 --2- 172.21.15.172:0/3000884077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556635776b40 0x556635783c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.925+0000 7f0c5f833740 1 --2- 172.21.15.172:0/3000884077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566357755d0 0x556635788960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.925+0000 7f0c3aff5640 0 client.16575 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:50:15.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.925+0000 7f0c5f833740 1 -- 172.21.15.172:0/3000884077 >> 172.21.15.172:0/3000884077 conn(0x55663573e310 msgr2=0x55663577e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:15.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.925+0000 7f0c5f833740 1 -- 172.21.15.172:0/3000884077 shutdown_connections 2024-09-10T08:50:15.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.925+0000 7f0c5f833740 1 -- 172.21.15.172:0/3000884077 wait complete. 2024-09-10T08:50:15.932 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.925+0000 7f0c5f833740 20 client.16575 trim_cache size 0 max 16384 2024-09-10T08:50:15.932 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:50:15.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.926+0000 7f0c5d45a640 1 -- 172.21.15.172:0/2983814597 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c400a2b80 msgr2=0x7f0c400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.926+0000 7f0c5d45a640 1 --2- 172.21.15.172:0/2983814597 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c400a2b80 0x7f0c400a5040 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0c580ff930 tx=0x7f0c4c00b690 comp rx=0 tx=0).stop 2024-09-10T08:50:15.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.926+0000 7f0c5d45a640 1 -- 172.21.15.172:0/2983814597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c58105560 msgr2=0x7f0c5819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:15.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.926+0000 7f0c5d45a640 1 --2- 172.21.15.172:0/2983814597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c58105560 0x7f0c5819aa00 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000de10 tx=0x7f0c5000b730 comp rx=0 tx=0).stop 2024-09-10T08:50:15.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.926+0000 7f0c5d45a640 1 -- 172.21.15.172:0/2983814597 shutdown_connections 2024-09-10T08:50:15.933 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.926+0000 7f0c5d45a640 1 --2- 172.21.15.172:0/2983814597 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c400a2b80 0x7f0c400a5040 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.926+0000 7f0c5d45a640 1 --2- 172.21.15.172:0/2983814597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c58105560 0x7f0c5819aa00 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.926+0000 7f0c5d45a640 1 --2- 172.21.15.172:0/2983814597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c58100de0 0x7f0c58196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.926+0000 7f0c5d45a640 1 --2- 172.21.15.172:0/2983814597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c58100430 0x7f0c58196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:15.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.926+0000 7f0c5d45a640 1 -- 172.21.15.172:0/2983814597 >> 172.21.15.172:0/2983814597 conn(0x7f0c58077230 msgr2=0x7f0c58103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:15.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.926+0000 7f0c5d45a640 1 -- 172.21.15.172:0/2983814597 shutdown_connections 2024-09-10T08:50:15.934 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:15.926+0000 7f0c5d45a640 1 -- 172.21.15.172:0/2983814597 wait complete. 2024-09-10T08:50:16.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:16 smithi172 ceph-mon[30854]: pgmap v471: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 26 op/s 2024-09-10T08:50:16.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:16 smithi174 ceph-mon[34068]: pgmap v471: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 26 op/s 2024-09-10T08:50:16.750 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:50:16.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:16 smithi191 ceph-mon[35314]: pgmap v471: 129 pgs: 129 active+clean; 2.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 26 op/s 2024-09-10T08:50:17.045 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:50:17.489 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:50:17.489 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:50:17.489 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:50:17.489 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:50:17.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.588+0000 7f9464e23640 1 -- 172.21.15.172:0/3122154081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9460103640 msgr2=0x7f946010bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:17.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.588+0000 7f9464e23640 1 --2- 172.21.15.172:0/3122154081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9460103640 0x7f946010bb30 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9454009900 tx=0x7f945402a480 comp rx=0 tx=0).stop 2024-09-10T08:50:17.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.588+0000 7f9464e23640 1 -- 172.21.15.172:0/3122154081 shutdown_connections 2024-09-10T08:50:17.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.588+0000 7f9464e23640 1 --2- 172.21.15.172:0/3122154081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f946010c150 0x7f946010e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.588+0000 7f9464e23640 1 --2- 172.21.15.172:0/3122154081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9460103640 0x7f946010bb30 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.588+0000 7f9464e23640 1 --2- 172.21.15.172:0/3122154081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9460102d20 0x7f9460103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.588+0000 7f9464e23640 1 -- 172.21.15.172:0/3122154081 >> 172.21.15.172:0/3122154081 conn(0x7f94600fc780 msgr2=0x7f94600feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:17.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.588+0000 7f9464e23640 1 -- 172.21.15.172:0/3122154081 shutdown_connections 2024-09-10T08:50:17.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.589+0000 7f9464e23640 1 -- 172.21.15.172:0/3122154081 wait complete. 2024-09-10T08:50:17.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.589+0000 7f9464e23640 1 Processor -- start 2024-09-10T08:50:17.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.589+0000 7f9464e23640 1 -- start start 2024-09-10T08:50:17.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f9464e23640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9460102d20 0x7f94601983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f9464e23640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9460103640 0x7f94601988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f9464e23640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f946010c150 0x7f946019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f9464e23640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9460111e50 con 0x7f946010c150 2024-09-10T08:50:17.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f9464e23640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9460111cd0 con 0x7f9460102d20 2024-09-10T08:50:17.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f9464e23640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9460111fd0 con 0x7f9460103640 2024-09-10T08:50:17.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f945ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f946010c150 0x7f946019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f945ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f946010c150 0x7f946019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46412/0 (socket says 172.21.15.172:46412) 2024-09-10T08:50:17.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f945ed9d640 1 -- 172.21.15.172:0/2109823304 learned_addr learned my addr 172.21.15.172:0/2109823304 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:17.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f945e59c640 1 --2- 172.21.15.172:0/2109823304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9460102d20 0x7f94601983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f945dd9b640 1 --2- 172.21.15.172:0/2109823304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9460103640 0x7f94601988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f945ed9d640 1 -- 172.21.15.172:0/2109823304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9460103640 msgr2=0x7f94601988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:17.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f945ed9d640 1 --2- 172.21.15.172:0/2109823304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9460103640 0x7f94601988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f945ed9d640 1 -- 172.21.15.172:0/2109823304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9460102d20 msgr2=0x7f94601983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:17.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f945ed9d640 1 --2- 172.21.15.172:0/2109823304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9460102d20 0x7f94601983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f945ed9d640 1 -- 172.21.15.172:0/2109823304 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946019d350 con 0x7f946010c150 2024-09-10T08:50:17.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f945dd9b640 1 --2- 172.21.15.172:0/2109823304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9460103640 0x7f94601988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:17.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f945e59c640 1 --2- 172.21.15.172:0/2109823304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9460102d20 0x7f94601983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:50:17.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f945ed9d640 1 --2- 172.21.15.172:0/2109823304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f946010c150 0x7f946019cc70 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f945800eaa0 tx=0x7f945800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:17.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.590+0000 7f944f7fe640 1 -- 172.21.15.172:0/2109823304 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945800cdb0 con 0x7f946010c150 2024-09-10T08:50:17.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.591+0000 7f944f7fe640 1 -- 172.21.15.172:0/2109823304 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9458005020 con 0x7f946010c150 2024-09-10T08:50:17.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.591+0000 7f9464e23640 1 -- 172.21.15.172:0/2109823304 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946019d640 con 0x7f946010c150 2024-09-10T08:50:17.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.591+0000 7f944f7fe640 1 -- 172.21.15.172:0/2109823304 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94580116d0 con 0x7f946010c150 2024-09-10T08:50:17.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.591+0000 7f9464e23640 1 -- 172.21.15.172:0/2109823304 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94601a4f20 con 0x7f946010c150 2024-09-10T08:50:17.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.593+0000 7f944f7fe640 1 -- 172.21.15.172:0/2109823304 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f94580051d0 con 0x7f946010c150 2024-09-10T08:50:17.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.593+0000 7f94671fc740 1 Processor -- start 2024-09-10T08:50:17.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f94671fc740 1 -- start start 2024-09-10T08:50:17.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f94671fc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b4bb40 0x561722b4a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f94671fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b4a5d0 0x561722b4aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f94671fc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b55370 0x561722b57760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f94671fc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561722ab02f0 con 0x561722b55370 2024-09-10T08:50:17.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f94671fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561722a57cf0 con 0x561722b4a5d0 2024-09-10T08:50:17.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f94671fc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561722aac2b0 con 0x561722b4bb40 2024-09-10T08:50:17.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f945e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b4a5d0 0x561722b4aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f945e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b4a5d0 0x561722b4aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:33388/0 (socket says 172.21.15.172:33388) 2024-09-10T08:50:17.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f945e59c640 1 -- 172.21.15.172:0/2409035487 learned_addr learned my addr 172.21.15.172:0/2409035487 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:17.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f945dd9b640 1 --2- 172.21.15.172:0/2409035487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b55370 0x561722b57760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f944f7fe640 1 --2- 172.21.15.172:0/2109823304 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94480a2b80 0x7f94480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f945dd9b640 1 --2- 172.21.15.172:0/2409035487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b4bb40 0x561722b4a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f945e59c640 1 -- 172.21.15.172:0/2409035487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b4bb40 msgr2=0x561722b4a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:17.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f945e59c640 1 --2- 172.21.15.172:0/2409035487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b4bb40 0x561722b4a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f945e59c640 1 -- 172.21.15.172:0/2409035487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b55370 msgr2=0x561722b57760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:17.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f945e59c640 1 --2- 172.21.15.172:0/2409035487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b55370 0x561722b57760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f945e59c640 1 -- 172.21.15.172:0/2409035487 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561722a3c3a0 con 0x561722b4a5d0 2024-09-10T08:50:17.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f945dd9b640 1 --2- 172.21.15.172:0/2409035487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b4bb40 0x561722b4a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:17.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.594+0000 7f944f7fe640 1 -- 172.21.15.172:0/2109823304 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f94580cf4b0 con 0x7f946010c150 2024-09-10T08:50:17.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.595+0000 7f945ed9d640 1 --2- 172.21.15.172:0/2109823304 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94480a2b80 0x7f94480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.595+0000 7f945e59c640 1 --2- 172.21.15.172:0/2409035487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b4a5d0 0x561722b4aa10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f94500115f0 tx=0x7f9450011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8314256632f8537b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:17.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.595+0000 7f945ed9d640 1 --2- 172.21.15.172:0/2109823304 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94480a2b80 0x7f94480a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f945800efd0 tx=0x7f9458086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:17.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.595+0000 7f944cff9640 1 -- 172.21.15.172:0/2409035487 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945001e070 con 0x561722b4a5d0 2024-09-10T08:50:17.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.595+0000 7f944cff9640 1 -- 172.21.15.172:0/2409035487 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f945000faf0 con 0x561722b4a5d0 2024-09-10T08:50:17.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.595+0000 7f944cff9640 1 -- 172.21.15.172:0/2409035487 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450019ac0 con 0x561722b4a5d0 2024-09-10T08:50:17.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.596+0000 7f94671fc740 1 -- 172.21.15.172:0/2409035487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b4a5d0 msgr2=0x561722b4aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:17.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.596+0000 7f94671fc740 1 --2- 172.21.15.172:0/2409035487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b4a5d0 0x561722b4aa10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f94500115f0 tx=0x7f9450011ab0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.596+0000 7f94671fc740 1 -- 172.21.15.172:0/2409035487 shutdown_connections 2024-09-10T08:50:17.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.596+0000 7f94671fc740 1 --2- 172.21.15.172:0/2409035487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b55370 0x561722b57760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.596+0000 7f94671fc740 1 --2- 172.21.15.172:0/2409035487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b4a5d0 0x561722b4aa10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.596+0000 7f94671fc740 1 --2- 172.21.15.172:0/2409035487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b4bb40 0x561722b4a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.596+0000 7f94671fc740 1 -- 172.21.15.172:0/2409035487 >> 172.21.15.172:0/2409035487 conn(0x561722b13310 msgr2=0x561722b46c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:17.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.596+0000 7f94671fc740 1 -- 172.21.15.172:0/2409035487 shutdown_connections 2024-09-10T08:50:17.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.596+0000 7f94671fc740 1 -- 172.21.15.172:0/2409035487 wait complete. 2024-09-10T08:50:17.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.597+0000 7f94671fc740 1 -- start start 2024-09-10T08:50:17.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.597+0000 7f94671fc740 1 Processor -- start 2024-09-10T08:50:17.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.597+0000 7f94671fc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4a5d0 0x561722b54d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.597+0000 7f94671fc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b4bb40 0x561722b52020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.597+0000 7f94671fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b55370 0x561722b52610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.597+0000 7f94671fc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561722b5c480 con 0x561722b4a5d0 2024-09-10T08:50:17.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.597+0000 7f94671fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561722a6b060 con 0x561722b55370 2024-09-10T08:50:17.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.597+0000 7f94671fc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561722b5c600 con 0x561722b4bb40 2024-09-10T08:50:17.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.597+0000 7f945dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4a5d0 0x561722b54d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.597+0000 7f945dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4a5d0 0x561722b54d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46422/0 (socket says 172.21.15.172:46422) 2024-09-10T08:50:17.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.597+0000 7f945dd9b640 1 -- 172.21.15.172:0/3769755652 learned_addr learned my addr 172.21.15.172:0/3769755652 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:17.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.597+0000 7f94671fc740 10 client.0 _ll_register_callbacks cb 0x5617227be8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:17.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.597+0000 7f945dd9b640 1 --2- 172.21.15.172:0/3769755652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b55370 0x561722b52610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.597+0000 7f945e59c640 1 --2- 172.21.15.172:0/3769755652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b4bb40 0x561722b52020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.598+0000 7f945dd9b640 1 -- 172.21.15.172:0/3769755652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b4bb40 msgr2=0x561722b52020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:17.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.598+0000 7f945dd9b640 1 --2- 172.21.15.172:0/3769755652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b4bb40 0x561722b52020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.598+0000 7f945dd9b640 1 -- 172.21.15.172:0/3769755652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b55370 msgr2=0x561722b52610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:17.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.598+0000 7f945dd9b640 1 --2- 172.21.15.172:0/3769755652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b55370 0x561722b52610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.598+0000 7f945dd9b640 1 -- 172.21.15.172:0/3769755652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617228213c0 con 0x561722b4a5d0 2024-09-10T08:50:17.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.598+0000 7f945dd9b640 1 --2- 172.21.15.172:0/3769755652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4a5d0 0x561722b54d70 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9454009900 tx=0x7f9454005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647279df1d7606bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:17.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.598+0000 7f9445ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:17.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.598+0000 7f9445ffb640 1 -- 172.21.15.172:0/3769755652 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454035030 con 0x561722b4a5d0 2024-09-10T08:50:17.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.598+0000 7f9445ffb640 1 -- 172.21.15.172:0/3769755652 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9454047040 con 0x561722b4a5d0 2024-09-10T08:50:17.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.598+0000 7f94671fc740 1 -- 172.21.15.172:0/3769755652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f945000faf0 con 0x561722b4a5d0 2024-09-10T08:50:17.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.598+0000 7f9445ffb640 1 -- 172.21.15.172:0/3769755652 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f94540055f0 con 0x561722b4a5d0 2024-09-10T08:50:17.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.598+0000 7f94671fc740 10 client.16587 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:17.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.598+0000 7f94671fc740 1 -- 172.21.15.172:0/3769755652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561722b67f00 con 0x561722b4a5d0 2024-09-10T08:50:17.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.598+0000 7f9445ffb640 1 -- 172.21.15.172:0/3769755652 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f945000faf0 con 0x561722b4a5d0 2024-09-10T08:50:17.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.599+0000 7f9445ffb640 1 -- 172.21.15.172:0/3769755652 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f9454003bf0 con 0x561722b4a5d0 2024-09-10T08:50:17.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.599+0000 7f94671fc740 10 client.16587 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:17.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.599+0000 7f94671fc740 10 client.16587 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:50:17.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.599+0000 7f94671fc740 20 client.16587 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:17.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.599+0000 7f94671fc740 1 --2- 172.21.15.172:0/3769755652 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x561722b693c0 0x561722b6b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.599+0000 7f94671fc740 4 client.16587 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:50:17.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.599+0000 7f94671fc740 1 -- 172.21.15.172:0/3769755652 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561722abd860 con 0x561722b693c0 2024-09-10T08:50:17.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.599+0000 7f945e59c640 1 --2- 172.21.15.172:0/3769755652 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x561722b693c0 0x561722b6b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.600+0000 7f945e59c640 1 --2- 172.21.15.172:0/3769755652 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x561722b693c0 0x561722b6b7b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96ac4c4738d90944 server_cookie=1ea8a87caaf90c3c in_seq=0 out_seq=0 2024-09-10T08:50:17.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.600+0000 7f9445ffb640 10 client.16587 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:50:17.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.600+0000 7f9445ffb640 1 -- 172.21.15.172:0/3769755652 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561722abd860 con 0x561722b693c0 2024-09-10T08:50:17.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.600+0000 7f9445ffb640 10 client.16587 handle_command_reply: tid=0 2024-09-10T08:50:17.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.600+0000 7f94671fc740 1 client.16587 shutdown 2024-09-10T08:50:17.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.600+0000 7f94671fc740 1 -- 172.21.15.172:0/3769755652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4a5d0 msgr2=0x561722b54d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:17.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.600+0000 7f94671fc740 1 --2- 172.21.15.172:0/3769755652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4a5d0 0x561722b54d70 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9454009900 tx=0x7f9454005bc0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.600+0000 7f94671fc740 1 -- 172.21.15.172:0/3769755652 shutdown_connections 2024-09-10T08:50:17.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.600+0000 7f94671fc740 1 --2- 172.21.15.172:0/3769755652 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x561722b693c0 0x561722b6b7b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.600+0000 7f94671fc740 1 --2- 172.21.15.172:0/3769755652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b55370 0x561722b52610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.600+0000 7f94671fc740 1 --2- 172.21.15.172:0/3769755652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b4bb40 0x561722b52020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.600+0000 7f9445ffb640 0 client.16587 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:50:17.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.600+0000 7f94671fc740 1 --2- 172.21.15.172:0/3769755652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4a5d0 0x561722b54d70 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.600+0000 7f94671fc740 1 -- 172.21.15.172:0/3769755652 >> 172.21.15.172:0/3769755652 conn(0x561722b13310 msgr2=0x561722b4c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:17.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.600+0000 7f94671fc740 1 -- 172.21.15.172:0/3769755652 shutdown_connections 2024-09-10T08:50:17.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.601+0000 7f94671fc740 1 -- 172.21.15.172:0/3769755652 wait complete. 2024-09-10T08:50:17.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.601+0000 7f94671fc740 20 client.16587 trim_cache size 0 max 16384 2024-09-10T08:50:17.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.613+0000 7f94671fc740 1 Processor -- start 2024-09-10T08:50:17.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.613+0000 7f94671fc740 1 -- start start 2024-09-10T08:50:17.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f94671fc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b4a5d0 0x561722b45bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f94671fc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4bb40 0x561722b4d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f94671fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b55370 0x561722b4d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f94671fc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f94601aca50 con 0x561722b4bb40 2024-09-10T08:50:17.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f94671fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94601ae650 con 0x561722b55370 2024-09-10T08:50:17.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f94671fc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f94601ad120 con 0x561722b4a5d0 2024-09-10T08:50:17.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f945e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4bb40 0x561722b4d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f945e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4bb40 0x561722b4d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46436/0 (socket says 172.21.15.172:46436) 2024-09-10T08:50:17.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f945e59c640 1 -- 172.21.15.172:0/7701453 learned_addr learned my addr 172.21.15.172:0/7701453 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:17.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f945dd9b640 1 --2- 172.21.15.172:0/7701453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b4a5d0 0x561722b45bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f945dd9b640 1 --2- 172.21.15.172:0/7701453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b55370 0x561722b4d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f945e59c640 1 -- 172.21.15.172:0/7701453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b4a5d0 msgr2=0x561722b45bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:17.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f945e59c640 1 --2- 172.21.15.172:0/7701453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b4a5d0 0x561722b45bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f945e59c640 1 -- 172.21.15.172:0/7701453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b55370 msgr2=0x561722b4d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:17.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f945e59c640 1 --2- 172.21.15.172:0/7701453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b55370 0x561722b4d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f945e59c640 1 -- 172.21.15.172:0/7701453 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561722b50ff0 con 0x561722b4bb40 2024-09-10T08:50:17.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f945dd9b640 1 --2- 172.21.15.172:0/7701453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b4a5d0 0x561722b45bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:17.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.614+0000 7f945e59c640 1 --2- 172.21.15.172:0/7701453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4bb40 0x561722b4d120 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f94500194f0 tx=0x7f94500196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e533ae4bb3243e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:17.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.615+0000 7f9447fff640 1 -- 172.21.15.172:0/7701453 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450028030 con 0x561722b4bb40 2024-09-10T08:50:17.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.615+0000 7f9447fff640 1 -- 172.21.15.172:0/7701453 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f945000faf0 con 0x561722b4bb40 2024-09-10T08:50:17.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.615+0000 7f9447fff640 1 -- 172.21.15.172:0/7701453 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945001e040 con 0x561722b4bb40 2024-09-10T08:50:17.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.615+0000 7f94671fc740 1 -- 172.21.15.172:0/7701453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4bb40 msgr2=0x561722b4d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:17.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.615+0000 7f94671fc740 1 --2- 172.21.15.172:0/7701453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4bb40 0x561722b4d120 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f94500194f0 tx=0x7f94500196a0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.616+0000 7f94671fc740 1 -- 172.21.15.172:0/7701453 shutdown_connections 2024-09-10T08:50:17.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.616+0000 7f94671fc740 1 --2- 172.21.15.172:0/7701453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b55370 0x561722b4d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.616+0000 7f94671fc740 1 --2- 172.21.15.172:0/7701453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4bb40 0x561722b4d120 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.616+0000 7f94671fc740 1 --2- 172.21.15.172:0/7701453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b4a5d0 0x561722b45bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.616+0000 7f94671fc740 1 -- 172.21.15.172:0/7701453 >> 172.21.15.172:0/7701453 conn(0x561722b13310 msgr2=0x561722b454e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:17.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.616+0000 7f94671fc740 1 -- 172.21.15.172:0/7701453 shutdown_connections 2024-09-10T08:50:17.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.616+0000 7f94671fc740 1 -- 172.21.15.172:0/7701453 wait complete. 2024-09-10T08:50:17.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.616+0000 7f94671fc740 1 -- start start 2024-09-10T08:50:17.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.616+0000 7f94671fc740 1 Processor -- start 2024-09-10T08:50:17.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.616+0000 7f94671fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b4a5d0 0x561722b61890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.616+0000 7f94671fc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4bb40 0x561722b61dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.616+0000 7f94671fc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b55370 0x561722b5e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.616+0000 7f94671fc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f94601adef0 con 0x561722b4bb40 2024-09-10T08:50:17.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.616+0000 7f94671fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94601ad820 con 0x561722b4a5d0 2024-09-10T08:50:17.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.616+0000 7f94671fc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f94601aaa30 con 0x561722b55370 2024-09-10T08:50:17.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.616+0000 7f94671fc740 10 client.0 _ll_register_callbacks cb 0x7f9450005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:17.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.617+0000 7f945e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4bb40 0x561722b61dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.617+0000 7f945e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4bb40 0x561722b61dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46442/0 (socket says 172.21.15.172:46442) 2024-09-10T08:50:17.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.617+0000 7f945dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b4a5d0 0x561722b61890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.617+0000 7f945e59c640 1 -- 172.21.15.172:0/2881499948 learned_addr learned my addr 172.21.15.172:0/2881499948 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:17.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.617+0000 7f945dd9b640 1 --2- 172.21.15.172:0/2881499948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b55370 0x561722b5e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.617+0000 7f945e59c640 1 -- 172.21.15.172:0/2881499948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b55370 msgr2=0x561722b5e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:17.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.617+0000 7f945e59c640 1 --2- 172.21.15.172:0/2881499948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b55370 0x561722b5e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.617+0000 7f945e59c640 1 -- 172.21.15.172:0/2881499948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b4a5d0 msgr2=0x561722b61890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:17.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.617+0000 7f945e59c640 1 --2- 172.21.15.172:0/2881499948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b4a5d0 0x561722b61890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.617+0000 7f945e59c640 1 -- 172.21.15.172:0/2881499948 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561722b52980 con 0x561722b4bb40 2024-09-10T08:50:17.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.617+0000 7f945dd9b640 1 --2- 172.21.15.172:0/2881499948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b4a5d0 0x561722b61890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:50:17.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.617+0000 7f945e59c640 1 --2- 172.21.15.172:0/2881499948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4bb40 0x561722b61dd0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f945001c040 tx=0x7f9450005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1adfe26855c81c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:17.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.617+0000 7f9445ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:17.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.617+0000 7f9445ffb640 1 -- 172.21.15.172:0/2881499948 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450030020 con 0x561722b4bb40 2024-09-10T08:50:17.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.617+0000 7f9445ffb640 1 -- 172.21.15.172:0/2881499948 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9450016070 con 0x561722b4bb40 2024-09-10T08:50:17.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.617+0000 7f94671fc740 1 -- 172.21.15.172:0/2881499948 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f945000faf0 con 0x561722b4bb40 2024-09-10T08:50:17.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.618+0000 7f9445ffb640 1 -- 172.21.15.172:0/2881499948 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9450028030 con 0x561722b4bb40 2024-09-10T08:50:17.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.618+0000 7f94671fc740 10 client.16593 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:17.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.618+0000 7f94671fc740 1 -- 172.21.15.172:0/2881499948 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561722bd4330 con 0x561722b4bb40 2024-09-10T08:50:17.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.618+0000 7f9445ffb640 1 -- 172.21.15.172:0/2881499948 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f945000faf0 con 0x561722b4bb40 2024-09-10T08:50:17.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.618+0000 7f9445ffb640 1 -- 172.21.15.172:0/2881499948 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f9450019710 con 0x561722b4bb40 2024-09-10T08:50:17.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.618+0000 7f94671fc740 10 client.16593 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:17.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.618+0000 7f94671fc740 10 client.16593 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:50:17.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.618+0000 7f94671fc740 20 client.16593 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:17.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.618+0000 7f94671fc740 1 --2- 172.21.15.172:0/2881499948 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x561722bd48a0 0x561722bd6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:17.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.618+0000 7f94671fc740 4 client.16593 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:50:17.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.618+0000 7f94671fc740 1 -- 172.21.15.172:0/2881499948 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561722ab7d70 con 0x561722bd48a0 2024-09-10T08:50:17.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.618+0000 7f945dd9b640 1 --2- 172.21.15.172:0/2881499948 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x561722bd48a0 0x561722bd6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:17.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.619+0000 7f945dd9b640 1 --2- 172.21.15.172:0/2881499948 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x561722bd48a0 0x561722bd6cc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25bccff787006e0d server_cookie=270c2145fe891b96 in_seq=0 out_seq=0 2024-09-10T08:50:17.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.619+0000 7f9445ffb640 10 client.16593 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:50:17.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.619+0000 7f9445ffb640 1 -- 172.21.15.172:0/2881499948 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561722ab7d70 con 0x561722bd48a0 2024-09-10T08:50:17.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.619+0000 7f9445ffb640 10 client.16593 handle_command_reply: tid=0 2024-09-10T08:50:17.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.619+0000 7f94671fc740 1 client.16593 shutdown 2024-09-10T08:50:17.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.619+0000 7f94671fc740 1 -- 172.21.15.172:0/2881499948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4bb40 msgr2=0x561722b61dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:17.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.619+0000 7f94671fc740 1 --2- 172.21.15.172:0/2881499948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4bb40 0x561722b61dd0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f945001c040 tx=0x7f9450005dc0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.619+0000 7f94671fc740 1 -- 172.21.15.172:0/2881499948 shutdown_connections 2024-09-10T08:50:17.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.619+0000 7f94671fc740 1 --2- 172.21.15.172:0/2881499948 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x561722bd48a0 0x561722bd6cc0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.619+0000 7f94671fc740 1 --2- 172.21.15.172:0/2881499948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561722b55370 0x561722b5e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.619+0000 7f94671fc740 1 --2- 172.21.15.172:0/2881499948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561722b4bb40 0x561722b61dd0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.619+0000 7f94671fc740 1 --2- 172.21.15.172:0/2881499948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561722b4a5d0 0x561722b61890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.619+0000 7f94671fc740 1 -- 172.21.15.172:0/2881499948 >> 172.21.15.172:0/2881499948 conn(0x561722b13310 msgr2=0x561722b45d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:17.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.619+0000 7f9445ffb640 0 client.16593 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:50:17.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.619+0000 7f94671fc740 1 -- 172.21.15.172:0/2881499948 shutdown_connections 2024-09-10T08:50:17.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.619+0000 7f94671fc740 1 -- 172.21.15.172:0/2881499948 wait complete. 2024-09-10T08:50:17.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.619+0000 7f94671fc740 20 client.16593 trim_cache size 0 max 16384 2024-09-10T08:50:17.625 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:50:17.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.620+0000 7f9464e23640 1 -- 172.21.15.172:0/2109823304 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94480a2b80 msgr2=0x7f94480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:17.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.620+0000 7f9464e23640 1 --2- 172.21.15.172:0/2109823304 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94480a2b80 0x7f94480a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f945800efd0 tx=0x7f9458086040 comp rx=0 tx=0).stop 2024-09-10T08:50:17.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.620+0000 7f9464e23640 1 -- 172.21.15.172:0/2109823304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f946010c150 msgr2=0x7f946019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:17.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.620+0000 7f9464e23640 1 --2- 172.21.15.172:0/2109823304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f946010c150 0x7f946019cc70 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f945800eaa0 tx=0x7f945800ef60 comp rx=0 tx=0).stop 2024-09-10T08:50:17.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.620+0000 7f9464e23640 1 -- 172.21.15.172:0/2109823304 shutdown_connections 2024-09-10T08:50:17.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.620+0000 7f9464e23640 1 --2- 172.21.15.172:0/2109823304 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94480a2b80 0x7f94480a5040 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.620+0000 7f9464e23640 1 --2- 172.21.15.172:0/2109823304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f946010c150 0x7f946019cc70 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.620+0000 7f9464e23640 1 --2- 172.21.15.172:0/2109823304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9460103640 0x7f94601988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.620+0000 7f9464e23640 1 --2- 172.21.15.172:0/2109823304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9460102d20 0x7f94601983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:17.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.620+0000 7f9464e23640 1 -- 172.21.15.172:0/2109823304 >> 172.21.15.172:0/2109823304 conn(0x7f94600fc780 msgr2=0x7f94600fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:17.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.620+0000 7f9464e23640 1 -- 172.21.15.172:0/2109823304 shutdown_connections 2024-09-10T08:50:17.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:17.620+0000 7f9464e23640 1 -- 172.21.15.172:0/2109823304 wait complete. 2024-09-10T08:50:17.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:50:17.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:50:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:50:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:50:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:50:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:50:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:50:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:50:18.409 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:50:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:18 smithi172 ceph-mon[30854]: pgmap v472: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 22 op/s 2024-09-10T08:50:18.694 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:50:18.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:18 smithi174 ceph-mon[34068]: pgmap v472: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 22 op/s 2024-09-10T08:50:18.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:18 smithi191 ceph-mon[35314]: pgmap v472: 129 pgs: 129 active+clean; 2.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 22 op/s 2024-09-10T08:50:19.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.203+0000 7f9666d9d640 1 -- 172.21.15.172:0/4219410448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96601008d0 msgr2=0x7f9660100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:19.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.203+0000 7f9666d9d640 1 --2- 172.21.15.172:0/4219410448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96601008d0 0x7f9660100d30 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9650009930 tx=0x7f965002a420 comp rx=0 tx=0).stop 2024-09-10T08:50:19.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.203+0000 7f9666d9d640 1 -- 172.21.15.172:0/4219410448 shutdown_connections 2024-09-10T08:50:19.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.203+0000 7f9666d9d640 1 --2- 172.21.15.172:0/4219410448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9660101270 0x7f966010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.203+0000 7f9666d9d640 1 --2- 172.21.15.172:0/4219410448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96601008d0 0x7f9660100d30 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.203+0000 7f9666d9d640 1 --2- 172.21.15.172:0/4219410448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96601068d0 0x7f9660106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.203+0000 7f9666d9d640 1 -- 172.21.15.172:0/4219410448 >> 172.21.15.172:0/4219410448 conn(0x7f96600fc610 msgr2=0x7f96600fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:19.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.203+0000 7f9666d9d640 1 -- 172.21.15.172:0/4219410448 shutdown_connections 2024-09-10T08:50:19.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.203+0000 7f9666d9d640 1 -- 172.21.15.172:0/4219410448 wait complete. 2024-09-10T08:50:19.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.203+0000 7f9666d9d640 1 Processor -- start 2024-09-10T08:50:19.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.203+0000 7f9666d9d640 1 -- start start 2024-09-10T08:50:19.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.204+0000 7f9666d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96601008d0 0x7f96601981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.204+0000 7f9666d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9660101270 0x7f9660198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.204+0000 7f9666d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96601068d0 0x7f966019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.204+0000 7f9666d9d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9660110b80 con 0x7f96601068d0 2024-09-10T08:50:19.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.204+0000 7f9666d9d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9660110a00 con 0x7f96601008d0 2024-09-10T08:50:19.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.204+0000 7f9666d9d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9660110d00 con 0x7f9660101270 2024-09-10T08:50:19.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.204+0000 7f966659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96601068d0 0x7f966019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.204+0000 7f966659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96601068d0 0x7f966019caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46462/0 (socket says 172.21.15.172:46462) 2024-09-10T08:50:19.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.204+0000 7f966659c640 1 -- 172.21.15.172:0/2930729977 learned_addr learned my addr 172.21.15.172:0/2930729977 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:19.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.204+0000 7f966559a640 1 --2- 172.21.15.172:0/2930729977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9660101270 0x7f9660198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.204+0000 7f9665d9b640 1 --2- 172.21.15.172:0/2930729977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96601008d0 0x7f96601981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.204+0000 7f966659c640 1 -- 172.21.15.172:0/2930729977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9660101270 msgr2=0x7f9660198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:19.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.204+0000 7f966659c640 1 --2- 172.21.15.172:0/2930729977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9660101270 0x7f9660198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.204+0000 7f966659c640 1 -- 172.21.15.172:0/2930729977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96601008d0 msgr2=0x7f96601981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:19.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.204+0000 7f966659c640 1 --2- 172.21.15.172:0/2930729977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96601008d0 0x7f96601981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.204+0000 7f966659c640 1 -- 172.21.15.172:0/2930729977 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966019d180 con 0x7f96601068d0 2024-09-10T08:50:19.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.204+0000 7f966559a640 1 --2- 172.21.15.172:0/2930729977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9660101270 0x7f9660198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:50:19.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.205+0000 7f966659c640 1 --2- 172.21.15.172:0/2930729977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96601068d0 0x7f966019caa0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f965800cce0 tx=0x7f965800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:19.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.205+0000 7f9656ffd640 1 -- 172.21.15.172:0/2930729977 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965800f450 con 0x7f96601068d0 2024-09-10T08:50:19.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.205+0000 7f9666d9d640 1 -- 172.21.15.172:0/2930729977 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966019d470 con 0x7f96601068d0 2024-09-10T08:50:19.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.205+0000 7f9656ffd640 1 -- 172.21.15.172:0/2930729977 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f965800faa0 con 0x7f96601068d0 2024-09-10T08:50:19.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.205+0000 7f9656ffd640 1 -- 172.21.15.172:0/2930729977 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658018b70 con 0x7f96601068d0 2024-09-10T08:50:19.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.205+0000 7f9666d9d640 1 -- 172.21.15.172:0/2930729977 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96601a4d50 con 0x7f96601068d0 2024-09-10T08:50:19.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.207+0000 7f9656ffd640 1 -- 172.21.15.172:0/2930729977 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9658018d10 con 0x7f96601068d0 2024-09-10T08:50:19.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.208+0000 7f966a3d8740 1 Processor -- start 2024-09-10T08:50:19.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.208+0000 7f966a3d8740 1 -- start start 2024-09-10T08:50:19.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.208+0000 7f966a3d8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2cad20 0x55d4ef2c9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.208+0000 7f966a3d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.208+0000 7f966a3d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2d8960 0x55d4ef2dad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.208+0000 7f966a3d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d4ef22f2f0 con 0x55d4ef2d8960 2024-09-10T08:50:19.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.208+0000 7f966a3d8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d4ef1d6cf0 con 0x55d4ef2cad20 2024-09-10T08:50:19.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.208+0000 7f966a3d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d4ef22b2b0 con 0x55d4ef2c97b0 2024-09-10T08:50:19.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.208+0000 7f966559a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2d8960 0x55d4ef2dad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.209+0000 7f966559a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2d8960 0x55d4ef2dad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46466/0 (socket says 172.21.15.172:46466) 2024-09-10T08:50:19.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.209+0000 7f966559a640 1 -- 172.21.15.172:0/1335565546 learned_addr learned my addr 172.21.15.172:0/1335565546 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:19.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.209+0000 7f9665d9b640 1 --2- 172.21.15.172:0/1335565546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.209+0000 7f9656ffd640 1 --2- 172.21.15.172:0/2930729977 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f964c0a2b80 0x7f964c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.209+0000 7f966559a640 1 --2- 172.21.15.172:0/1335565546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2cad20 0x55d4ef2c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.209+0000 7f966659c640 1 --2- 172.21.15.172:0/2930729977 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f964c0a2b80 0x7f964c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.209+0000 7f9665d9b640 1 -- 172.21.15.172:0/1335565546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2cad20 msgr2=0x55d4ef2c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:19.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.209+0000 7f9665d9b640 1 --2- 172.21.15.172:0/1335565546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2cad20 0x55d4ef2c9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.209+0000 7f9665d9b640 1 -- 172.21.15.172:0/1335565546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2d8960 msgr2=0x55d4ef2dad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:19.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.209+0000 7f9665d9b640 1 --2- 172.21.15.172:0/1335565546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2d8960 0x55d4ef2dad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.209+0000 7f9665d9b640 1 -- 172.21.15.172:0/1335565546 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4ef1bb3a0 con 0x55d4ef2c97b0 2024-09-10T08:50:19.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.209+0000 7f9656ffd640 1 -- 172.21.15.172:0/2930729977 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f96580cf780 con 0x7f96601068d0 2024-09-10T08:50:19.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.209+0000 7f966559a640 1 --2- 172.21.15.172:0/1335565546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2cad20 0x55d4ef2c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:50:19.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.209+0000 7f9665d9b640 1 --2- 172.21.15.172:0/1335565546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2c9bf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f965c009c00 tx=0x7f965c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=367339f0b3f5cdd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:19.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.209+0000 7f966659c640 1 --2- 172.21.15.172:0/2930729977 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f964c0a2b80 0x7f964c0a5040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f96580a43c0 tx=0x7f96580c6810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:19.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.209+0000 7f964bfff640 1 -- 172.21.15.172:0/1335565546 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c004050 con 0x55d4ef2c97b0 2024-09-10T08:50:19.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.210+0000 7f964bfff640 1 -- 172.21.15.172:0/1335565546 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f965c0054d0 con 0x55d4ef2c97b0 2024-09-10T08:50:19.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.210+0000 7f964bfff640 1 -- 172.21.15.172:0/1335565546 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c004c00 con 0x55d4ef2c97b0 2024-09-10T08:50:19.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.210+0000 7f966a3d8740 1 -- 172.21.15.172:0/1335565546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2c97b0 msgr2=0x55d4ef2c9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:19.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.210+0000 7f966a3d8740 1 --2- 172.21.15.172:0/1335565546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2c9bf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f965c009c00 tx=0x7f965c011810 comp rx=0 tx=0).stop 2024-09-10T08:50:19.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.210+0000 7f966a3d8740 1 -- 172.21.15.172:0/1335565546 shutdown_connections 2024-09-10T08:50:19.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.210+0000 7f966a3d8740 1 --2- 172.21.15.172:0/1335565546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2d8960 0x55d4ef2dad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.210+0000 7f966a3d8740 1 --2- 172.21.15.172:0/1335565546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2c9bf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.210+0000 7f966a3d8740 1 --2- 172.21.15.172:0/1335565546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2cad20 0x55d4ef2c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.210+0000 7f966a3d8740 1 -- 172.21.15.172:0/1335565546 >> 172.21.15.172:0/1335565546 conn(0x55d4ef292310 msgr2=0x55d4ef2c5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:19.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.211+0000 7f966a3d8740 1 -- 172.21.15.172:0/1335565546 shutdown_connections 2024-09-10T08:50:19.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.211+0000 7f966a3d8740 1 -- 172.21.15.172:0/1335565546 wait complete. 2024-09-10T08:50:19.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.211+0000 7f966a3d8740 1 -- start start 2024-09-10T08:50:19.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.211+0000 7f966a3d8740 1 Processor -- start 2024-09-10T08:50:19.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.211+0000 7f966a3d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2d82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.211+0000 7f966a3d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2cad20 0x55d4ef2d5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.211+0000 7f966a3d8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2d8960 0x55d4ef2d5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.211+0000 7f966a3d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d4ef2db660 con 0x55d4ef2cad20 2024-09-10T08:50:19.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.211+0000 7f966a3d8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d4ef1ea060 con 0x55d4ef2d8960 2024-09-10T08:50:19.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.211+0000 7f966a3d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d4ef2db7e0 con 0x55d4ef2c97b0 2024-09-10T08:50:19.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.212+0000 7f966a3d8740 10 client.0 _ll_register_callbacks cb 0x55d4eef3d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:19.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.212+0000 7f966559a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2d82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.212+0000 7f966559a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2d82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:42344/0 (socket says 172.21.15.172:42344) 2024-09-10T08:50:19.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.212+0000 7f9665d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2cad20 0x55d4ef2d5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.212+0000 7f966559a640 1 -- 172.21.15.172:0/2705316022 learned_addr learned my addr 172.21.15.172:0/2705316022 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:19.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.212+0000 7f966559a640 1 --2- 172.21.15.172:0/2705316022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2d8960 0x55d4ef2d5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.212+0000 7f966559a640 1 -- 172.21.15.172:0/2705316022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2d8960 msgr2=0x55d4ef2d5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:19.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.212+0000 7f966559a640 1 --2- 172.21.15.172:0/2705316022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2d8960 0x55d4ef2d5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.212+0000 7f966559a640 1 -- 172.21.15.172:0/2705316022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2cad20 msgr2=0x55d4ef2d5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:19.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.212+0000 7f966559a640 1 --2- 172.21.15.172:0/2705316022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2cad20 0x55d4ef2d5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.212+0000 7f966559a640 1 -- 172.21.15.172:0/2705316022 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4eefa03c0 con 0x55d4ef2c97b0 2024-09-10T08:50:19.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.212+0000 7f966559a640 1 --2- 172.21.15.172:0/2705316022 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2d82b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9650009230 tx=0x7f965002a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64267cf4a5136e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:19.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.212+0000 7f96497fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:50:19.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.212+0000 7f96497fa640 1 -- 172.21.15.172:0/2705316022 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650035040 con 0x55d4ef2c97b0 2024-09-10T08:50:19.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.212+0000 7f966a3d8740 1 -- 172.21.15.172:0/2705316022 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f965c0054d0 con 0x55d4ef2c97b0 2024-09-10T08:50:19.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.212+0000 7f96497fa640 1 -- 172.21.15.172:0/2705316022 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9650003800 con 0x55d4ef2c97b0 2024-09-10T08:50:19.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.213+0000 7f96497fa640 1 -- 172.21.15.172:0/2705316022 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9650055050 con 0x55d4ef2c97b0 2024-09-10T08:50:19.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.213+0000 7f966a3d8740 10 client.25852 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:19.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.213+0000 7f966a3d8740 1 -- 172.21.15.172:0/2705316022 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d4ef2e7070 con 0x55d4ef2c97b0 2024-09-10T08:50:19.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.213+0000 7f96497fa640 1 -- 172.21.15.172:0/2705316022 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f965c0054d0 con 0x55d4ef2c97b0 2024-09-10T08:50:19.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.213+0000 7f96497fa640 1 -- 172.21.15.172:0/2705316022 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f965003d960 con 0x55d4ef2c97b0 2024-09-10T08:50:19.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.213+0000 7f966a3d8740 10 client.25852 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:19.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.213+0000 7f966a3d8740 10 client.25852 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:50:19.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.213+0000 7f966a3d8740 20 client.25852 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:19.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.213+0000 7f966a3d8740 1 --2- 172.21.15.172:0/2705316022 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55d4ef2e8910 0x55d4ef2ead60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.213+0000 7f966a3d8740 4 client.25852 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:50:19.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.213+0000 7f966a3d8740 1 -- 172.21.15.172:0/2705316022 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4ef23c860 con 0x55d4ef2e8910 2024-09-10T08:50:19.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.213+0000 7f9665d9b640 1 --2- 172.21.15.172:0/2705316022 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55d4ef2e8910 0x55d4ef2ead60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.214+0000 7f9665d9b640 1 --2- 172.21.15.172:0/2705316022 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55d4ef2e8910 0x55d4ef2ead60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d7d1b1c4b0b34b4 server_cookie=381c112614e423d6 in_seq=0 out_seq=0 2024-09-10T08:50:19.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.214+0000 7f96497fa640 10 client.25852 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:50:19.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.214+0000 7f96497fa640 1 -- 172.21.15.172:0/2705316022 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d4ef23c860 con 0x55d4ef2e8910 2024-09-10T08:50:19.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.214+0000 7f96497fa640 10 client.25852 handle_command_reply: tid=0 2024-09-10T08:50:19.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.214+0000 7f966a3d8740 1 client.25852 shutdown 2024-09-10T08:50:19.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.214+0000 7f966a3d8740 1 -- 172.21.15.172:0/2705316022 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2c97b0 msgr2=0x55d4ef2d82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.214+0000 7f966a3d8740 1 --2- 172.21.15.172:0/2705316022 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2d82b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9650009230 tx=0x7f965002a420 comp rx=0 tx=0).stop 2024-09-10T08:50:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.215+0000 7f966a3d8740 1 -- 172.21.15.172:0/2705316022 shutdown_connections 2024-09-10T08:50:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.215+0000 7f966a3d8740 1 --2- 172.21.15.172:0/2705316022 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55d4ef2e8910 0x55d4ef2ead60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.215+0000 7f966a3d8740 1 --2- 172.21.15.172:0/2705316022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2d8960 0x55d4ef2d5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.215+0000 7f966a3d8740 1 --2- 172.21.15.172:0/2705316022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2cad20 0x55d4ef2d5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.215+0000 7f966a3d8740 1 --2- 172.21.15.172:0/2705316022 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2d82b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.215+0000 7f966a3d8740 1 -- 172.21.15.172:0/2705316022 >> 172.21.15.172:0/2705316022 conn(0x55d4ef292310 msgr2=0x55d4ef2cbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.215+0000 7f96497fa640 0 client.25852 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:50:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.215+0000 7f966a3d8740 1 -- 172.21.15.172:0/2705316022 shutdown_connections 2024-09-10T08:50:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.215+0000 7f966a3d8740 1 -- 172.21.15.172:0/2705316022 wait complete. 2024-09-10T08:50:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.215+0000 7f966a3d8740 20 client.25852 trim_cache size 0 max 16384 2024-09-10T08:50:19.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.228+0000 7f966a3d8740 1 Processor -- start 2024-09-10T08:50:19.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.228+0000 7f966a3d8740 1 -- start start 2024-09-10T08:50:19.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.228+0000 7f966a3d8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2d9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.228+0000 7f966a3d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2cad20 0x55d4ef2da290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.228+0000 7f966a3d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2d8960 0x55d4ef2d3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.228+0000 7f966a3d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f96601ac730 con 0x55d4ef2d8960 2024-09-10T08:50:19.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.228+0000 7f966a3d8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96601ae330 con 0x55d4ef2c97b0 2024-09-10T08:50:19.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.228+0000 7f966a3d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f96601ace00 con 0x55d4ef2cad20 2024-09-10T08:50:19.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.228+0000 7f966559a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2d8960 0x55d4ef2d3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.228+0000 7f966559a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2d8960 0x55d4ef2d3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46476/0 (socket says 172.21.15.172:46476) 2024-09-10T08:50:19.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.228+0000 7f966559a640 1 -- 172.21.15.172:0/2233698327 learned_addr learned my addr 172.21.15.172:0/2233698327 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:19.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.228+0000 7f966559a640 1 --2- 172.21.15.172:0/2233698327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2d9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.229+0000 7f9665d9b640 1 --2- 172.21.15.172:0/2233698327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2cad20 0x55d4ef2da290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.229+0000 7f966559a640 1 -- 172.21.15.172:0/2233698327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2cad20 msgr2=0x55d4ef2da290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:19.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.229+0000 7f966559a640 1 --2- 172.21.15.172:0/2233698327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2cad20 0x55d4ef2da290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.229+0000 7f966559a640 1 -- 172.21.15.172:0/2233698327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2c97b0 msgr2=0x55d4ef2d9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:19.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.229+0000 7f966559a640 1 --2- 172.21.15.172:0/2233698327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2d9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.229+0000 7f966559a640 1 -- 172.21.15.172:0/2233698327 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4ef2d3ad0 con 0x55d4ef2d8960 2024-09-10T08:50:19.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.229+0000 7f966559a640 1 --2- 172.21.15.172:0/2233698327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2d8960 0x55d4ef2d3e50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9650033000 tx=0x7f9650000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a745e9ef235abe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:19.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.229+0000 7f964b7fe640 1 -- 172.21.15.172:0/2233698327 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650035040 con 0x55d4ef2d8960 2024-09-10T08:50:19.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.229+0000 7f964b7fe640 1 -- 172.21.15.172:0/2233698327 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9650042030 con 0x55d4ef2d8960 2024-09-10T08:50:19.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.229+0000 7f964b7fe640 1 -- 172.21.15.172:0/2233698327 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965003db40 con 0x55d4ef2d8960 2024-09-10T08:50:19.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.230+0000 7f966a3d8740 1 -- 172.21.15.172:0/2233698327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2d8960 msgr2=0x55d4ef2d3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:19.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.230+0000 7f966a3d8740 1 --2- 172.21.15.172:0/2233698327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2d8960 0x55d4ef2d3e50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9650033000 tx=0x7f9650000bc0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.230+0000 7f966a3d8740 1 -- 172.21.15.172:0/2233698327 shutdown_connections 2024-09-10T08:50:19.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.230+0000 7f966a3d8740 1 --2- 172.21.15.172:0/2233698327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2d8960 0x55d4ef2d3e50 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.230+0000 7f966a3d8740 1 --2- 172.21.15.172:0/2233698327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2cad20 0x55d4ef2da290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.230+0000 7f966a3d8740 1 --2- 172.21.15.172:0/2233698327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2d9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.230+0000 7f966a3d8740 1 -- 172.21.15.172:0/2233698327 >> 172.21.15.172:0/2233698327 conn(0x55d4ef292310 msgr2=0x55d4ef2d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:19.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.230+0000 7f966a3d8740 1 -- 172.21.15.172:0/2233698327 shutdown_connections 2024-09-10T08:50:19.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.230+0000 7f966a3d8740 1 -- 172.21.15.172:0/2233698327 wait complete. 2024-09-10T08:50:19.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.231+0000 7f966a3d8740 1 -- start start 2024-09-10T08:50:19.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.231+0000 7f966a3d8740 1 Processor -- start 2024-09-10T08:50:19.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.231+0000 7f966a3d8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2e8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.231+0000 7f966a3d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2cad20 0x55d4ef2e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.231+0000 7f966a3d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2d8960 0x55d4ef2e9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.232+0000 7f966a3d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f96601adbd0 con 0x55d4ef2cad20 2024-09-10T08:50:19.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.232+0000 7f966a3d8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96601ad500 con 0x55d4ef2c97b0 2024-09-10T08:50:19.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.232+0000 7f966a3d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f96601aa710 con 0x55d4ef2d8960 2024-09-10T08:50:19.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.232+0000 7f966a3d8740 10 client.0 _ll_register_callbacks cb 0x7f9650005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:19.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.232+0000 7f966559a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2d8960 0x55d4ef2e9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.232+0000 7f966559a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2d8960 0x55d4ef2e9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:42362/0 (socket says 172.21.15.172:42362) 2024-09-10T08:50:19.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.232+0000 7f966559a640 1 -- 172.21.15.172:0/1484707439 learned_addr learned my addr 172.21.15.172:0/1484707439 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:19.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.232+0000 7f966559a640 1 --2- 172.21.15.172:0/1484707439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2e8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.232+0000 7f9665d9b640 1 --2- 172.21.15.172:0/1484707439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2cad20 0x55d4ef2e8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.232+0000 7f966559a640 1 -- 172.21.15.172:0/1484707439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2c97b0 msgr2=0x55d4ef2e8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:19.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.232+0000 7f966559a640 1 --2- 172.21.15.172:0/1484707439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2e8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.232+0000 7f966559a640 1 -- 172.21.15.172:0/1484707439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2cad20 msgr2=0x55d4ef2e8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:19.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.232+0000 7f966559a640 1 --2- 172.21.15.172:0/1484707439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2cad20 0x55d4ef2e8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.232+0000 7f966559a640 1 -- 172.21.15.172:0/1484707439 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4ef351440 con 0x55d4ef2d8960 2024-09-10T08:50:19.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.232+0000 7f9665d9b640 1 --2- 172.21.15.172:0/1484707439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2cad20 0x55d4ef2e8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:50:19.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.232+0000 7f966559a640 1 --2- 172.21.15.172:0/1484707439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2d8960 0x55d4ef2e9390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9650041040 tx=0x7f9650009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab0096e03b62f958 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:19.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.232+0000 7f96497fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:50:19.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.233+0000 7f96497fa640 1 -- 172.21.15.172:0/1484707439 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650035070 con 0x55d4ef2d8960 2024-09-10T08:50:19.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.233+0000 7f96497fa640 1 -- 172.21.15.172:0/1484707439 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f965003d450 con 0x55d4ef2d8960 2024-09-10T08:50:19.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.233+0000 7f966a3d8740 1 -- 172.21.15.172:0/1484707439 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9650042030 con 0x55d4ef2d8960 2024-09-10T08:50:19.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.233+0000 7f96497fa640 1 -- 172.21.15.172:0/1484707439 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f965003d650 con 0x55d4ef2d8960 2024-09-10T08:50:19.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.233+0000 7f966a3d8740 10 client.25858 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:19.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.233+0000 7f966a3d8740 1 -- 172.21.15.172:0/1484707439 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d4ef3526e0 con 0x55d4ef2d8960 2024-09-10T08:50:19.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.233+0000 7f96497fa640 1 -- 172.21.15.172:0/1484707439 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9650042030 con 0x55d4ef2d8960 2024-09-10T08:50:19.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.233+0000 7f96497fa640 1 -- 172.21.15.172:0/1484707439 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f965004c8e0 con 0x55d4ef2d8960 2024-09-10T08:50:19.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.233+0000 7f966a3d8740 10 client.25858 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:19.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.233+0000 7f966a3d8740 10 client.25858 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:50:19.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.233+0000 7f966a3d8740 20 client.25858 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:19.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.233+0000 7f966a3d8740 1 --2- 172.21.15.172:0/1484707439 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55d4ef352e20 0x55d4ef355240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:19.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.234+0000 7f966a3d8740 4 client.25858 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:50:19.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.234+0000 7f966a3d8740 1 -- 172.21.15.172:0/1484707439 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4ef236d70 con 0x55d4ef352e20 2024-09-10T08:50:19.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.234+0000 7f9665d9b640 1 --2- 172.21.15.172:0/1484707439 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55d4ef352e20 0x55d4ef355240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:19.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.234+0000 7f96497fa640 10 client.25858 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:50:19.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.234+0000 7f9665d9b640 1 --2- 172.21.15.172:0/1484707439 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55d4ef352e20 0x55d4ef355240 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7509ac7dc81216b8 server_cookie=50df4b1823fa567d in_seq=0 out_seq=0 2024-09-10T08:50:19.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.234+0000 7f96497fa640 1 -- 172.21.15.172:0/1484707439 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4ef236d70 con 0x55d4ef352e20 2024-09-10T08:50:19.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.234+0000 7f96497fa640 10 client.25858 handle_command_reply: tid=0 2024-09-10T08:50:19.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.235+0000 7f966a3d8740 1 client.25858 shutdown 2024-09-10T08:50:19.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.235+0000 7f966a3d8740 1 -- 172.21.15.172:0/1484707439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2d8960 msgr2=0x55d4ef2e9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:19.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.235+0000 7f966a3d8740 1 --2- 172.21.15.172:0/1484707439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2d8960 0x55d4ef2e9390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9650041040 tx=0x7f9650009520 comp rx=0 tx=0).stop 2024-09-10T08:50:19.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.235+0000 7f966a3d8740 1 -- 172.21.15.172:0/1484707439 shutdown_connections 2024-09-10T08:50:19.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.235+0000 7f966a3d8740 1 --2- 172.21.15.172:0/1484707439 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55d4ef352e20 0x55d4ef355240 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.235+0000 7f966a3d8740 1 --2- 172.21.15.172:0/1484707439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d4ef2d8960 0x55d4ef2e9390 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.235+0000 7f966a3d8740 1 --2- 172.21.15.172:0/1484707439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef2cad20 0x55d4ef2e8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.235+0000 7f96497fa640 0 client.25858 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:50:19.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.235+0000 7f966a3d8740 1 --2- 172.21.15.172:0/1484707439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4ef2c97b0 0x55d4ef2e8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.235+0000 7f966a3d8740 1 -- 172.21.15.172:0/1484707439 >> 172.21.15.172:0/1484707439 conn(0x55d4ef292310 msgr2=0x55d4ef2c4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:19.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.235+0000 7f966a3d8740 1 -- 172.21.15.172:0/1484707439 shutdown_connections 2024-09-10T08:50:19.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.235+0000 7f966a3d8740 1 -- 172.21.15.172:0/1484707439 wait complete. 2024-09-10T08:50:19.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.236+0000 7f966a3d8740 20 client.25858 trim_cache size 0 max 16384 2024-09-10T08:50:19.245 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:50:19.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.236+0000 7f9666d9d640 1 -- 172.21.15.172:0/2930729977 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f964c0a2b80 msgr2=0x7f964c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:19.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.236+0000 7f9666d9d640 1 --2- 172.21.15.172:0/2930729977 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f964c0a2b80 0x7f964c0a5040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f96580a43c0 tx=0x7f96580c6810 comp rx=0 tx=0).stop 2024-09-10T08:50:19.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.236+0000 7f9666d9d640 1 -- 172.21.15.172:0/2930729977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96601068d0 msgr2=0x7f966019caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:19.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.236+0000 7f9666d9d640 1 --2- 172.21.15.172:0/2930729977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96601068d0 0x7f966019caa0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f965800cce0 tx=0x7f965800a620 comp rx=0 tx=0).stop 2024-09-10T08:50:19.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.236+0000 7f9666d9d640 1 -- 172.21.15.172:0/2930729977 shutdown_connections 2024-09-10T08:50:19.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.236+0000 7f9666d9d640 1 --2- 172.21.15.172:0/2930729977 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f964c0a2b80 0x7f964c0a5040 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.236+0000 7f9666d9d640 1 --2- 172.21.15.172:0/2930729977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96601068d0 0x7f966019caa0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.237+0000 7f9666d9d640 1 --2- 172.21.15.172:0/2930729977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9660101270 0x7f9660198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.237+0000 7f9666d9d640 1 --2- 172.21.15.172:0/2930729977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96601008d0 0x7f96601981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:19.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.237+0000 7f9666d9d640 1 -- 172.21.15.172:0/2930729977 >> 172.21.15.172:0/2930729977 conn(0x7f96600fc610 msgr2=0x7f9660104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:19.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.237+0000 7f9666d9d640 1 -- 172.21.15.172:0/2930729977 shutdown_connections 2024-09-10T08:50:19.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:19.237+0000 7f9666d9d640 1 -- 172.21.15.172:0/2930729977 wait complete. 2024-09-10T08:50:20.053 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:50:20.336 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:50:20.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:20 smithi172 ceph-mon[30854]: pgmap v473: 129 pgs: 129 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-09-10T08:50:20.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:20 smithi174 ceph-mon[34068]: pgmap v473: 129 pgs: 129 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-09-10T08:50:20.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:20 smithi191 ceph-mon[35314]: pgmap v473: 129 pgs: 129 active+clean; 2.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-09-10T08:50:20.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.861+0000 7f8401a87640 1 -- 172.21.15.172:0/721495488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83fc1036e0 msgr2=0x7f83fc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:20.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.861+0000 7f8401a87640 1 --2- 172.21.15.172:0/721495488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83fc1036e0 0x7f83fc107b90 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009930 tx=0x7f83f002a420 comp rx=0 tx=0).stop 2024-09-10T08:50:20.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.861+0000 7f8401a87640 1 -- 172.21.15.172:0/721495488 shutdown_connections 2024-09-10T08:50:20.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.861+0000 7f8401a87640 1 --2- 172.21.15.172:0/721495488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83fc1082a0 0x7f83fc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.861+0000 7f8401a87640 1 --2- 172.21.15.172:0/721495488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83fc1036e0 0x7f83fc107b90 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.861+0000 7f8401a87640 1 --2- 172.21.15.172:0/721495488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83fc102d30 0x7f83fc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.861+0000 7f8401a87640 1 -- 172.21.15.172:0/721495488 >> 172.21.15.172:0/721495488 conn(0x7f83fc0fc7d0 msgr2=0x7f83fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.861+0000 7f8401a87640 1 -- 172.21.15.172:0/721495488 shutdown_connections 2024-09-10T08:50:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.861+0000 7f8401a87640 1 -- 172.21.15.172:0/721495488 wait complete. 2024-09-10T08:50:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.861+0000 7f8401a87640 1 Processor -- start 2024-09-10T08:50:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f8401a87640 1 -- start start 2024-09-10T08:50:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f8401a87640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83fc102d30 0x7f83fc198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f8401a87640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83fc1036e0 0x7f83fc198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f8401a87640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83fc1082a0 0x7f83fc19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f8401a87640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc111f50 con 0x7f83fc102d30 2024-09-10T08:50:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f8401a87640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc111dd0 con 0x7f83fc1036e0 2024-09-10T08:50:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f8401a87640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc1120d0 con 0x7f83fc1082a0 2024-09-10T08:50:20.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f83fb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83fc102d30 0x7f83fc198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f83fb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83fc102d30 0x7f83fc198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46504/0 (socket says 172.21.15.172:46504) 2024-09-10T08:50:20.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f83fb7fe640 1 -- 172.21.15.172:0/4020207144 learned_addr learned my addr 172.21.15.172:0/4020207144 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:20.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f83faffd640 1 --2- 172.21.15.172:0/4020207144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83fc1036e0 0x7f83fc198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f83fbfff640 1 --2- 172.21.15.172:0/4020207144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83fc1082a0 0x7f83fc19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f83fb7fe640 1 -- 172.21.15.172:0/4020207144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83fc1082a0 msgr2=0x7f83fc19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:20.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f83fb7fe640 1 --2- 172.21.15.172:0/4020207144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83fc1082a0 0x7f83fc19ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f83fb7fe640 1 -- 172.21.15.172:0/4020207144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83fc1036e0 msgr2=0x7f83fc198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:20.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f83fb7fe640 1 --2- 172.21.15.172:0/4020207144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83fc1036e0 0x7f83fc198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f83fb7fe640 1 -- 172.21.15.172:0/4020207144 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83fc19d540 con 0x7f83fc102d30 2024-09-10T08:50:20.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.862+0000 7f83fbfff640 1 --2- 172.21.15.172:0/4020207144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83fc1082a0 0x7f83fc19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:50:20.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.863+0000 7f83fb7fe640 1 --2- 172.21.15.172:0/4020207144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83fc102d30 0x7f83fc198590 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00e9f0 tx=0x7f83ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:20.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.863+0000 7f83f8ff9640 1 -- 172.21.15.172:0/4020207144 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00cd80 con 0x7f83fc102d30 2024-09-10T08:50:20.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.863+0000 7f8401a87640 1 -- 172.21.15.172:0/4020207144 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83fc19d830 con 0x7f83fc102d30 2024-09-10T08:50:20.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.863+0000 7f83f8ff9640 1 -- 172.21.15.172:0/4020207144 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83ec0054d0 con 0x7f83fc102d30 2024-09-10T08:50:20.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.863+0000 7f83f8ff9640 1 -- 172.21.15.172:0/4020207144 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec010490 con 0x7f83fc102d30 2024-09-10T08:50:20.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.863+0000 7f8401a87640 1 -- 172.21.15.172:0/4020207144 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83fc1a5110 con 0x7f83fc102d30 2024-09-10T08:50:20.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.865+0000 7f83f8ff9640 1 -- 172.21.15.172:0/4020207144 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f83ec005020 con 0x7f83fc102d30 2024-09-10T08:50:20.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.865+0000 7f8403e60740 1 Processor -- start 2024-09-10T08:50:20.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.865+0000 7f8403e60740 1 -- start start 2024-09-10T08:50:20.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f8403e60740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a6f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f8403e60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a6f65d0 0x56055a6f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f8403e60740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a705780 0x56055a707b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f8403e60740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56055a65c2f0 con 0x56055a6f7b40 2024-09-10T08:50:20.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f8403e60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56055a603cf0 con 0x56055a6f65d0 2024-09-10T08:50:20.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f8403e60740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56055a6582b0 con 0x56055a705780 2024-09-10T08:50:20.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f83fbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a6f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f83fbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a6f6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46516/0 (socket says 172.21.15.172:46516) 2024-09-10T08:50:20.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f83fbfff640 1 -- 172.21.15.172:0/3536631402 learned_addr learned my addr 172.21.15.172:0/3536631402 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:20.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f83fb7fe640 1 --2- 172.21.15.172:0/3536631402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a6f65d0 0x56055a6f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f83faffd640 1 --2- 172.21.15.172:0/3536631402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a705780 0x56055a707b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f83fbfff640 1 -- 172.21.15.172:0/3536631402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a705780 msgr2=0x56055a707b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:20.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f83fbfff640 1 --2- 172.21.15.172:0/3536631402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a705780 0x56055a707b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f83fbfff640 1 -- 172.21.15.172:0/3536631402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a6f65d0 msgr2=0x56055a6f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:20.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f83fbfff640 1 --2- 172.21.15.172:0/3536631402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a6f65d0 0x56055a6f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f83fbfff640 1 -- 172.21.15.172:0/3536631402 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56055a5e83a0 con 0x56055a6f7b40 2024-09-10T08:50:20.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f83faffd640 1 --2- 172.21.15.172:0/3536631402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a705780 0x56055a707b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:50:20.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.866+0000 7f83f8ff9640 1 --2- 172.21.15.172:0/4020207144 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83e40a2b80 0x7f83e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.867+0000 7f83fbfff640 1 --2- 172.21.15.172:0/3536631402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a6f6090 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009f10 tx=0x7f83f400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594198a9de16814b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:20.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.867+0000 7f83fb7fe640 1 --2- 172.21.15.172:0/4020207144 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83e40a2b80 0x7f83e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.867+0000 7f83f8ff9640 1 -- 172.21.15.172:0/4020207144 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f83ec0cf880 con 0x7f83fc102d30 2024-09-10T08:50:20.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.867+0000 7f83e1ffb640 1 -- 172.21.15.172:0/3536631402 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4003b70 con 0x56055a6f7b40 2024-09-10T08:50:20.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.867+0000 7f83e1ffb640 1 -- 172.21.15.172:0/3536631402 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83f4003d10 con 0x56055a6f7b40 2024-09-10T08:50:20.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.867+0000 7f83e1ffb640 1 -- 172.21.15.172:0/3536631402 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4011a00 con 0x56055a6f7b40 2024-09-10T08:50:20.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.868+0000 7f8403e60740 1 -- 172.21.15.172:0/3536631402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 msgr2=0x56055a6f6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:20.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.868+0000 7f83fb7fe640 1 --2- 172.21.15.172:0/4020207144 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83e40a2b80 0x7f83e40a5040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0cd390 tx=0x7f83ec0d5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:20.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.868+0000 7f8403e60740 1 --2- 172.21.15.172:0/3536631402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a6f6090 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009f10 tx=0x7f83f400f730 comp rx=0 tx=0).stop 2024-09-10T08:50:20.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.868+0000 7f8403e60740 1 -- 172.21.15.172:0/3536631402 shutdown_connections 2024-09-10T08:50:20.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.868+0000 7f8403e60740 1 --2- 172.21.15.172:0/3536631402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a705780 0x56055a707b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.868+0000 7f8403e60740 1 --2- 172.21.15.172:0/3536631402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a6f65d0 0x56055a6f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.868+0000 7f8403e60740 1 --2- 172.21.15.172:0/3536631402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a6f6090 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.868+0000 7f8403e60740 1 -- 172.21.15.172:0/3536631402 >> 172.21.15.172:0/3536631402 conn(0x56055a6bf310 msgr2=0x56055a6f2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:20.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.868+0000 7f8403e60740 1 -- 172.21.15.172:0/3536631402 shutdown_connections 2024-09-10T08:50:20.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.868+0000 7f8403e60740 1 -- 172.21.15.172:0/3536631402 wait complete. 2024-09-10T08:50:20.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.868+0000 7f8403e60740 1 -- start start 2024-09-10T08:50:20.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.868+0000 7f8403e60740 1 Processor -- start 2024-09-10T08:50:20.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f8403e60740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a6f65d0 0x56055a7051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f8403e60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a6f7b40 0x56055a702480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f8403e60740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a705780 0x56055a702c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f8403e60740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56055a708480 con 0x56055a705780 2024-09-10T08:50:20.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f8403e60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56055a617060 con 0x56055a6f7b40 2024-09-10T08:50:20.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f8403e60740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56055a708600 con 0x56055a6f65d0 2024-09-10T08:50:20.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f8403e60740 10 client.0 _ll_register_callbacks cb 0x56055a36a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:20.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f83fbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a705780 0x56055a702c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f83fbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a705780 0x56055a702c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46530/0 (socket says 172.21.15.172:46530) 2024-09-10T08:50:20.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f83fbfff640 1 -- 172.21.15.172:0/1988608985 learned_addr learned my addr 172.21.15.172:0/1988608985 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:20.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f83fbfff640 1 --2- 172.21.15.172:0/1988608985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a6f65d0 0x56055a7051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f83faffd640 1 --2- 172.21.15.172:0/1988608985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a6f7b40 0x56055a702480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f83fbfff640 1 -- 172.21.15.172:0/1988608985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a6f65d0 msgr2=0x56055a7051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:20.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f83fbfff640 1 --2- 172.21.15.172:0/1988608985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a6f65d0 0x56055a7051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f83fbfff640 1 -- 172.21.15.172:0/1988608985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a6f7b40 msgr2=0x56055a702480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:20.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f83fbfff640 1 --2- 172.21.15.172:0/1988608985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a6f7b40 0x56055a702480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f83fbfff640 1 -- 172.21.15.172:0/1988608985 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56055a3cd3c0 con 0x56055a705780 2024-09-10T08:50:20.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.869+0000 7f83faffd640 1 --2- 172.21.15.172:0/1988608985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a6f7b40 0x56055a702480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:20.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.870+0000 7f83fbfff640 1 --2- 172.21.15.172:0/1988608985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a705780 0x56055a702c10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f83f4011580 tx=0x7f83f4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c3eb99ec66c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:20.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.870+0000 7f83daffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:20.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.870+0000 7f83daffd640 1 -- 172.21.15.172:0/1988608985 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4024040 con 0x56055a705780 2024-09-10T08:50:20.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.870+0000 7f8403e60740 1 -- 172.21.15.172:0/1988608985 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83f4003d10 con 0x56055a705780 2024-09-10T08:50:20.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.870+0000 7f83daffd640 1 -- 172.21.15.172:0/1988608985 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f83f40129c0 con 0x56055a705780 2024-09-10T08:50:20.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.870+0000 7f83daffd640 1 -- 172.21.15.172:0/1988608985 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f83f4032050 con 0x56055a705780 2024-09-10T08:50:20.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.870+0000 7f8403e60740 10 client.16620 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:20.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.870+0000 7f8403e60740 1 -- 172.21.15.172:0/1988608985 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56055a713ee0 con 0x56055a705780 2024-09-10T08:50:20.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.870+0000 7f83daffd640 1 -- 172.21.15.172:0/1988608985 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83f4003d10 con 0x56055a705780 2024-09-10T08:50:20.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.870+0000 7f83daffd640 1 -- 172.21.15.172:0/1988608985 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f83f4004400 con 0x56055a705780 2024-09-10T08:50:20.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.871+0000 7f8403e60740 10 client.16620 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:20.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.871+0000 7f8403e60740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:20.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.871+0000 7f8403e60740 20 client.16620 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:20.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.871+0000 7f8403e60740 1 --2- 172.21.15.172:0/1988608985 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56055a7152f0 0x56055a7176e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.871+0000 7f8403e60740 4 client.16620 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:50:20.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.871+0000 7f8403e60740 1 -- 172.21.15.172:0/1988608985 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56055a669860 con 0x56055a7152f0 2024-09-10T08:50:20.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.871+0000 7f83faffd640 1 --2- 172.21.15.172:0/1988608985 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56055a7152f0 0x56055a7176e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.871+0000 7f83daffd640 10 client.16620 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:20.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.872+0000 7f83faffd640 1 --2- 172.21.15.172:0/1988608985 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56055a7152f0 0x56055a7176e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e58fe1197bcadb7b server_cookie=1f2d161215543a7b in_seq=0 out_seq=0 2024-09-10T08:50:20.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.872+0000 7f83daffd640 1 -- 172.21.15.172:0/1988608985 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56055a669860 con 0x56055a7152f0 2024-09-10T08:50:20.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.872+0000 7f83daffd640 10 client.16620 handle_command_reply: tid=0 2024-09-10T08:50:20.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.872+0000 7f8403e60740 1 client.16620 shutdown 2024-09-10T08:50:20.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.872+0000 7f8403e60740 1 -- 172.21.15.172:0/1988608985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a705780 msgr2=0x56055a702c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:20.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.872+0000 7f8403e60740 1 --2- 172.21.15.172:0/1988608985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a705780 0x56055a702c10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f83f4011580 tx=0x7f83f4009500 comp rx=0 tx=0).stop 2024-09-10T08:50:20.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.873+0000 7f8403e60740 1 -- 172.21.15.172:0/1988608985 shutdown_connections 2024-09-10T08:50:20.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.873+0000 7f8403e60740 1 --2- 172.21.15.172:0/1988608985 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56055a7152f0 0x56055a7176e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.873+0000 7f8403e60740 1 --2- 172.21.15.172:0/1988608985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a705780 0x56055a702c10 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.873+0000 7f8403e60740 1 --2- 172.21.15.172:0/1988608985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a6f7b40 0x56055a702480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.873+0000 7f8403e60740 1 --2- 172.21.15.172:0/1988608985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a6f65d0 0x56055a7051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.873+0000 7f83daffd640 0 client.16620 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:20.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.873+0000 7f8403e60740 1 -- 172.21.15.172:0/1988608985 >> 172.21.15.172:0/1988608985 conn(0x56055a6bf310 msgr2=0x56055a6f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:20.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.873+0000 7f8403e60740 1 -- 172.21.15.172:0/1988608985 shutdown_connections 2024-09-10T08:50:20.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.873+0000 7f8403e60740 1 -- 172.21.15.172:0/1988608985 wait complete. 2024-09-10T08:50:20.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.873+0000 7f8403e60740 20 client.16620 trim_cache size 0 max 16384 2024-09-10T08:50:20.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.890+0000 7f8403e60740 1 Processor -- start 2024-09-10T08:50:20.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.890+0000 7f8403e60740 1 -- start start 2024-09-10T08:50:20.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.890+0000 7f8403e60740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a6f65d0 0x56055a701d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.890+0000 7f8403e60740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a6f2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.890+0000 7f8403e60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a705780 0x56055a6f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.890+0000 7f8403e60740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc1ac9f0 con 0x56055a6f7b40 2024-09-10T08:50:20.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.890+0000 7f8403e60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc1ae5f0 con 0x56055a705780 2024-09-10T08:50:20.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.890+0000 7f8403e60740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc1ad0c0 con 0x56055a6f65d0 2024-09-10T08:50:20.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.890+0000 7f83faffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a6f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.890+0000 7f83faffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a6f2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46540/0 (socket says 172.21.15.172:46540) 2024-09-10T08:50:20.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.890+0000 7f83faffd640 1 -- 172.21.15.172:0/515308673 learned_addr learned my addr 172.21.15.172:0/515308673 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:20.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.890+0000 7f83fbfff640 1 --2- 172.21.15.172:0/515308673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a6f65d0 0x56055a701d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.890+0000 7f83fbfff640 1 --2- 172.21.15.172:0/515308673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a705780 0x56055a6f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.891+0000 7f83faffd640 1 -- 172.21.15.172:0/515308673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a6f65d0 msgr2=0x56055a701d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:20.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.891+0000 7f83faffd640 1 --2- 172.21.15.172:0/515308673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a6f65d0 0x56055a701d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.891+0000 7f83faffd640 1 -- 172.21.15.172:0/515308673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a705780 msgr2=0x56055a6f2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:20.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.891+0000 7f83faffd640 1 --2- 172.21.15.172:0/515308673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a705780 0x56055a6f2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.891+0000 7f83faffd640 1 -- 172.21.15.172:0/515308673 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56055a6f3180 con 0x56055a6f7b40 2024-09-10T08:50:20.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.891+0000 7f83fbfff640 1 --2- 172.21.15.172:0/515308673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a705780 0x56055a6f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:20.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.891+0000 7f83faffd640 1 --2- 172.21.15.172:0/515308673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a6f2700 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f83f0005ab0 tx=0x7f83f0009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3de45101bd453c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.891+0000 7f83e17fa640 1 -- 172.21.15.172:0/515308673 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f0047070 con 0x56055a6f7b40 2024-09-10T08:50:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.891+0000 7f83e17fa640 1 -- 172.21.15.172:0/515308673 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83f0005040 con 0x56055a6f7b40 2024-09-10T08:50:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.891+0000 7f83e17fa640 1 -- 172.21.15.172:0/515308673 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f0035040 con 0x56055a6f7b40 2024-09-10T08:50:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.892+0000 7f8403e60740 1 -- 172.21.15.172:0/515308673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 msgr2=0x56055a6f2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.892+0000 7f8403e60740 1 --2- 172.21.15.172:0/515308673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a6f2700 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f83f0005ab0 tx=0x7f83f0009650 comp rx=0 tx=0).stop 2024-09-10T08:50:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.892+0000 7f8403e60740 1 -- 172.21.15.172:0/515308673 shutdown_connections 2024-09-10T08:50:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.892+0000 7f8403e60740 1 --2- 172.21.15.172:0/515308673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a705780 0x56055a6f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.892+0000 7f8403e60740 1 --2- 172.21.15.172:0/515308673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a6f2700 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.892+0000 7f8403e60740 1 --2- 172.21.15.172:0/515308673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a6f65d0 0x56055a701d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.892+0000 7f8403e60740 1 -- 172.21.15.172:0/515308673 >> 172.21.15.172:0/515308673 conn(0x56055a6bf310 msgr2=0x56055a6eda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.892+0000 7f8403e60740 1 -- 172.21.15.172:0/515308673 shutdown_connections 2024-09-10T08:50:20.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.892+0000 7f8403e60740 1 -- 172.21.15.172:0/515308673 wait complete. 2024-09-10T08:50:20.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.892+0000 7f8403e60740 1 -- start start 2024-09-10T08:50:20.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.893+0000 7f8403e60740 1 Processor -- start 2024-09-10T08:50:20.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.893+0000 7f8403e60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a6f65d0 0x56055a70dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.893+0000 7f8403e60740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a709160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.893+0000 7f8403e60740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a705780 0x56055a7096a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.893+0000 7f8403e60740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc1ade90 con 0x56055a6f7b40 2024-09-10T08:50:20.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.893+0000 7f8403e60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc1ad7c0 con 0x56055a6f65d0 2024-09-10T08:50:20.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.893+0000 7f8403e60740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc1aa9d0 con 0x56055a705780 2024-09-10T08:50:20.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.893+0000 7f8403e60740 10 client.0 _ll_register_callbacks cb 0x7f83f4015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:20.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.893+0000 7f83faffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a709160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.893+0000 7f83faffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a709160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46556/0 (socket says 172.21.15.172:46556) 2024-09-10T08:50:20.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.893+0000 7f83faffd640 1 -- 172.21.15.172:0/817867811 learned_addr learned my addr 172.21.15.172:0/817867811 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:20.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.893+0000 7f83fbfff640 1 --2- 172.21.15.172:0/817867811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a6f65d0 0x56055a70dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.893+0000 7f83fbfff640 1 --2- 172.21.15.172:0/817867811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a705780 0x56055a7096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.894+0000 7f83faffd640 1 -- 172.21.15.172:0/817867811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a705780 msgr2=0x56055a7096a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:20.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.894+0000 7f83faffd640 1 --2- 172.21.15.172:0/817867811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a705780 0x56055a7096a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.894+0000 7f83faffd640 1 -- 172.21.15.172:0/817867811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a6f65d0 msgr2=0x56055a70dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:20.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.894+0000 7f83faffd640 1 --2- 172.21.15.172:0/817867811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a6f65d0 0x56055a70dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.894+0000 7f83faffd640 1 -- 172.21.15.172:0/817867811 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56055a709c90 con 0x56055a6f7b40 2024-09-10T08:50:20.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.894+0000 7f83fbfff640 1 --2- 172.21.15.172:0/817867811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a705780 0x56055a7096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:20.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.894+0000 7f83faffd640 1 --2- 172.21.15.172:0/817867811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a709160 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f83f003d040 tx=0x7f83f0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9dfd249435dc26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:20.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.894+0000 7f83daffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:20.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.894+0000 7f83daffd640 1 -- 172.21.15.172:0/817867811 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f0035070 con 0x56055a6f7b40 2024-09-10T08:50:20.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.894+0000 7f83daffd640 1 -- 172.21.15.172:0/817867811 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f83f00052b0 con 0x56055a6f7b40 2024-09-10T08:50:20.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.894+0000 7f8403e60740 1 -- 172.21.15.172:0/817867811 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83f0005040 con 0x56055a6f7b40 2024-09-10T08:50:20.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.894+0000 7f83daffd640 1 -- 172.21.15.172:0/817867811 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f83f0047040 con 0x56055a6f7b40 2024-09-10T08:50:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.894+0000 7f8403e60740 10 client.16626 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.894+0000 7f8403e60740 1 -- 172.21.15.172:0/817867811 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56055a77fbf0 con 0x56055a6f7b40 2024-09-10T08:50:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.895+0000 7f83daffd640 1 -- 172.21.15.172:0/817867811 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83f0005040 con 0x56055a6f7b40 2024-09-10T08:50:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.895+0000 7f83daffd640 1 -- 172.21.15.172:0/817867811 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f83f0053030 con 0x56055a6f7b40 2024-09-10T08:50:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.895+0000 7f8403e60740 10 client.16626 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.895+0000 7f8403e60740 10 client.16626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.895+0000 7f8403e60740 20 client.16626 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.895+0000 7f8403e60740 1 --2- 172.21.15.172:0/817867811 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56055a7803f0 0x56055a782810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.895+0000 7f8403e60740 4 client.16626 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:50:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.895+0000 7f8403e60740 1 -- 172.21.15.172:0/817867811 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56055a663d70 con 0x56055a7803f0 2024-09-10T08:50:20.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.895+0000 7f83fbfff640 1 --2- 172.21.15.172:0/817867811 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56055a7803f0 0x56055a782810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:20.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.896+0000 7f83daffd640 10 client.16626 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:20.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.896+0000 7f83fbfff640 1 --2- 172.21.15.172:0/817867811 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56055a7803f0 0x56055a782810 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78fd52621a63baad server_cookie=e5ebda4da6e680e1 in_seq=0 out_seq=0 2024-09-10T08:50:20.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.896+0000 7f83daffd640 1 -- 172.21.15.172:0/817867811 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56055a663d70 con 0x56055a7803f0 2024-09-10T08:50:20.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.896+0000 7f83daffd640 10 client.16626 handle_command_reply: tid=0 2024-09-10T08:50:20.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.896+0000 7f8403e60740 1 client.16626 shutdown 2024-09-10T08:50:20.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.896+0000 7f8403e60740 1 -- 172.21.15.172:0/817867811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 msgr2=0x56055a709160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:20.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.896+0000 7f8403e60740 1 --2- 172.21.15.172:0/817867811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a709160 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f83f003d040 tx=0x7f83f0031400 comp rx=0 tx=0).stop 2024-09-10T08:50:20.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.896+0000 7f8403e60740 1 -- 172.21.15.172:0/817867811 shutdown_connections 2024-09-10T08:50:20.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.896+0000 7f8403e60740 1 --2- 172.21.15.172:0/817867811 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56055a7803f0 0x56055a782810 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.896+0000 7f8403e60740 1 --2- 172.21.15.172:0/817867811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56055a705780 0x56055a7096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.896+0000 7f83daffd640 0 client.16626 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:20.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.896+0000 7f8403e60740 1 --2- 172.21.15.172:0/817867811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56055a6f7b40 0x56055a709160 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.896+0000 7f8403e60740 1 --2- 172.21.15.172:0/817867811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56055a6f65d0 0x56055a70dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.896+0000 7f8403e60740 1 -- 172.21.15.172:0/817867811 >> 172.21.15.172:0/817867811 conn(0x56055a6bf310 msgr2=0x56055a701cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:20.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.896+0000 7f8403e60740 1 -- 172.21.15.172:0/817867811 shutdown_connections 2024-09-10T08:50:20.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.897+0000 7f8403e60740 1 -- 172.21.15.172:0/817867811 wait complete. 2024-09-10T08:50:20.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.897+0000 7f8403e60740 20 client.16626 trim_cache size 0 max 16384 2024-09-10T08:50:20.902 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:50:20.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.897+0000 7f8401a87640 1 -- 172.21.15.172:0/4020207144 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83e40a2b80 msgr2=0x7f83e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:20.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.897+0000 7f8401a87640 1 --2- 172.21.15.172:0/4020207144 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83e40a2b80 0x7f83e40a5040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0cd390 tx=0x7f83ec0d5550 comp rx=0 tx=0).stop 2024-09-10T08:50:20.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.897+0000 7f8401a87640 1 -- 172.21.15.172:0/4020207144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83fc102d30 msgr2=0x7f83fc198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:20.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.897+0000 7f8401a87640 1 --2- 172.21.15.172:0/4020207144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83fc102d30 0x7f83fc198590 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00e9f0 tx=0x7f83ec00edf0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.897+0000 7f8401a87640 1 -- 172.21.15.172:0/4020207144 shutdown_connections 2024-09-10T08:50:20.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.897+0000 7f8401a87640 1 --2- 172.21.15.172:0/4020207144 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83e40a2b80 0x7f83e40a5040 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.898+0000 7f8401a87640 1 --2- 172.21.15.172:0/4020207144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83fc1082a0 0x7f83fc19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.898+0000 7f8401a87640 1 --2- 172.21.15.172:0/4020207144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83fc1036e0 0x7f83fc198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.898+0000 7f8401a87640 1 --2- 172.21.15.172:0/4020207144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83fc102d30 0x7f83fc198590 secure :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00e9f0 tx=0x7f83ec00edf0 comp rx=0 tx=0).stop 2024-09-10T08:50:20.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.898+0000 7f8401a87640 1 -- 172.21.15.172:0/4020207144 >> 172.21.15.172:0/4020207144 conn(0x7f83fc0fc7d0 msgr2=0x7f83fc106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:20.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.898+0000 7f8401a87640 1 -- 172.21.15.172:0/4020207144 shutdown_connections 2024-09-10T08:50:20.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:20.898+0000 7f8401a87640 1 -- 172.21.15.172:0/4020207144 wait complete. 2024-09-10T08:50:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:22 smithi172 ceph-mon[30854]: pgmap v474: 129 pgs: 129 active+clean; 2.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-09-10T08:50:22.624 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:50:22.624 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:50:22.646 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T08:50:22.646 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T08:50:22.647 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:50:22.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:22 smithi174 ceph-mon[34068]: pgmap v474: 129 pgs: 129 active+clean; 2.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-09-10T08:50:22.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:22 smithi191 ceph-mon[35314]: pgmap v474: 129 pgs: 129 active+clean; 2.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-09-10T08:50:22.923 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:50:23.472 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:50:23.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.478+0000 7f7441100640 1 -- 172.21.15.172:0/588506943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f743c102780 msgr2=0x7f743c1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:23.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.478+0000 7f7441100640 1 --2- 172.21.15.172:0/588506943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f743c102780 0x7f743c1067d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7430009900 tx=0x7f743002a410 comp rx=0 tx=0).stop 2024-09-10T08:50:23.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.478+0000 7f7441100640 1 -- 172.21.15.172:0/588506943 shutdown_connections 2024-09-10T08:50:23.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.478+0000 7f7441100640 1 --2- 172.21.15.172:0/588506943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f743c106f00 0x7f743c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.478+0000 7f7441100640 1 --2- 172.21.15.172:0/588506943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f743c102780 0x7f743c1067d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.478+0000 7f7441100640 1 --2- 172.21.15.172:0/588506943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f743c101dd0 0x7f743c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.478+0000 7f7441100640 1 -- 172.21.15.172:0/588506943 >> 172.21.15.172:0/588506943 conn(0x7f743c0fd8b0 msgr2=0x7f743c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:23.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.478+0000 7f7441100640 1 -- 172.21.15.172:0/588506943 shutdown_connections 2024-09-10T08:50:23.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.479+0000 7f7441100640 1 -- 172.21.15.172:0/588506943 wait complete. 2024-09-10T08:50:23.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.479+0000 7f7441100640 1 Processor -- start 2024-09-10T08:50:23.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.480+0000 7f7441100640 1 -- start start 2024-09-10T08:50:23.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.480+0000 7f7441100640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f743c101dd0 0x7f743c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.480+0000 7f7441100640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f743c102780 0x7f743c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.480+0000 7f7441100640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f743c106f00 0x7f743c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.480+0000 7f7441100640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f743c111e10 con 0x7f743c101dd0 2024-09-10T08:50:23.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.480+0000 7f7441100640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f743c111c90 con 0x7f743c106f00 2024-09-10T08:50:23.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.480+0000 7f7441100640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f743c111f90 con 0x7f743c102780 2024-09-10T08:50:23.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.480+0000 7f743ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f743c101dd0 0x7f743c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:23.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.480+0000 7f743ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f743c101dd0 0x7f743c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46578/0 (socket says 172.21.15.172:46578) 2024-09-10T08:50:23.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.480+0000 7f743ad9d640 1 -- 172.21.15.172:0/1076968307 learned_addr learned my addr 172.21.15.172:0/1076968307 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:23.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.480+0000 7f743a59c640 1 --2- 172.21.15.172:0/1076968307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f743c102780 0x7f743c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:23.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.480+0000 7f743b59e640 1 --2- 172.21.15.172:0/1076968307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f743c106f00 0x7f743c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:23.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.481+0000 7f743ad9d640 1 -- 172.21.15.172:0/1076968307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f743c102780 msgr2=0x7f743c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:23.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.481+0000 7f743ad9d640 1 --2- 172.21.15.172:0/1076968307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f743c102780 0x7f743c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.481+0000 7f743ad9d640 1 -- 172.21.15.172:0/1076968307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f743c106f00 msgr2=0x7f743c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:23.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.481+0000 7f743ad9d640 1 --2- 172.21.15.172:0/1076968307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f743c106f00 0x7f743c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.481+0000 7f743ad9d640 1 -- 172.21.15.172:0/1076968307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743c19d320 con 0x7f743c101dd0 2024-09-10T08:50:23.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.481+0000 7f743a59c640 1 --2- 172.21.15.172:0/1076968307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f743c102780 0x7f743c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:50:23.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.481+0000 7f743ad9d640 1 --2- 172.21.15.172:0/1076968307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f743c101dd0 0x7f743c198370 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f742c00d950 tx=0x7f742c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:23.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.481+0000 7f7423fff640 1 -- 172.21.15.172:0/1076968307 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c017030 con 0x7f743c101dd0 2024-09-10T08:50:23.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.481+0000 7f7423fff640 1 -- 172.21.15.172:0/1076968307 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f742c004020 con 0x7f743c101dd0 2024-09-10T08:50:23.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.481+0000 7f7423fff640 1 -- 172.21.15.172:0/1076968307 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c0047a0 con 0x7f743c101dd0 2024-09-10T08:50:23.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.481+0000 7f7441100640 1 -- 172.21.15.172:0/1076968307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743c19d610 con 0x7f743c101dd0 2024-09-10T08:50:23.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.481+0000 7f7441100640 1 -- 172.21.15.172:0/1076968307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743c1a4ef0 con 0x7f743c101dd0 2024-09-10T08:50:23.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.484+0000 7f7423fff640 1 -- 172.21.15.172:0/1076968307 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f742c0041d0 con 0x7f743c101dd0 2024-09-10T08:50:23.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.484+0000 7f74434d9740 1 Processor -- start 2024-09-10T08:50:23.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.484+0000 7f74434d9740 1 -- start start 2024-09-10T08:50:23.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.485+0000 7f74434d9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e78b60 0x5583c7e78f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.485+0000 7f74434d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e79480 0x5583c7e83800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.485+0000 7f74434d9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e83d40 0x5583c7e86130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.485+0000 7f74434d9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5583c7dde280 con 0x5583c7e78b60 2024-09-10T08:50:23.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.485+0000 7f74434d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5583c7d85b10 con 0x5583c7e79480 2024-09-10T08:50:23.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.485+0000 7f74434d9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5583c7dda240 con 0x5583c7e83d40 2024-09-10T08:50:23.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.485+0000 7f743b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e78b60 0x5583c7e78f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:23.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.485+0000 7f743b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e78b60 0x5583c7e78f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46590/0 (socket says 172.21.15.172:46590) 2024-09-10T08:50:23.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.485+0000 7f743b59e640 1 -- 172.21.15.172:0/3162318340 learned_addr learned my addr 172.21.15.172:0/3162318340 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:23.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.485+0000 7f743ad9d640 1 --2- 172.21.15.172:0/3162318340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e79480 0x5583c7e83800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:23.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.485+0000 7f743a59c640 1 --2- 172.21.15.172:0/3162318340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e83d40 0x5583c7e86130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:23.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.485+0000 7f743b59e640 1 -- 172.21.15.172:0/3162318340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e83d40 msgr2=0x5583c7e86130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:23.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.485+0000 7f743b59e640 1 --2- 172.21.15.172:0/3162318340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e83d40 0x5583c7e86130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.485+0000 7f743b59e640 1 -- 172.21.15.172:0/3162318340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e79480 msgr2=0x5583c7e83800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:23.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.485+0000 7f743b59e640 1 --2- 172.21.15.172:0/3162318340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e79480 0x5583c7e83800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.485+0000 7f743b59e640 1 -- 172.21.15.172:0/3162318340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583c7d921f0 con 0x5583c7e78b60 2024-09-10T08:50:23.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.486+0000 7f743ad9d640 1 --2- 172.21.15.172:0/3162318340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e79480 0x5583c7e83800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:50:23.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.486+0000 7f743b59e640 1 --2- 172.21.15.172:0/3162318340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e78b60 0x5583c7e78f40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7434009f70 tx=0x7f7434010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974b9802eaa5e09a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:23.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.486+0000 7f7423fff640 1 --2- 172.21.15.172:0/1076968307 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f74240a2b80 0x7f74240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.486+0000 7f74217fa640 1 -- 172.21.15.172:0/3162318340 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434004d20 con 0x5583c7e78b60 2024-09-10T08:50:23.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.486+0000 7f74217fa640 1 -- 172.21.15.172:0/3162318340 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74340054d0 con 0x5583c7e78b60 2024-09-10T08:50:23.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.486+0000 7f74217fa640 1 -- 172.21.15.172:0/3162318340 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434022910 con 0x5583c7e78b60 2024-09-10T08:50:23.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.486+0000 7f743ad9d640 1 --2- 172.21.15.172:0/1076968307 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f74240a2b80 0x7f74240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:23.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.487+0000 7f74434d9740 1 -- 172.21.15.172:0/3162318340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e78b60 msgr2=0x5583c7e78f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:23.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.487+0000 7f74434d9740 1 --2- 172.21.15.172:0/3162318340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e78b60 0x5583c7e78f40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7434009f70 tx=0x7f7434010800 comp rx=0 tx=0).stop 2024-09-10T08:50:23.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.487+0000 7f743ad9d640 1 --2- 172.21.15.172:0/1076968307 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f74240a2b80 0x7f74240a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f742c0cc850 tx=0x7f742c0d9ae0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:23.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.487+0000 7f7423fff640 1 -- 172.21.15.172:0/1076968307 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f742c0ced40 con 0x7f743c101dd0 2024-09-10T08:50:23.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.487+0000 7f74434d9740 1 -- 172.21.15.172:0/3162318340 shutdown_connections 2024-09-10T08:50:23.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.487+0000 7f74434d9740 1 --2- 172.21.15.172:0/3162318340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e83d40 0x5583c7e86130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.487+0000 7f74434d9740 1 --2- 172.21.15.172:0/3162318340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e79480 0x5583c7e83800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.487+0000 7f74434d9740 1 --2- 172.21.15.172:0/3162318340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e78b60 0x5583c7e78f40 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.487+0000 7f74434d9740 1 -- 172.21.15.172:0/3162318340 >> 172.21.15.172:0/3162318340 conn(0x5583c7e750f0 msgr2=0x5583c7e754d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:23.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.487+0000 7f74434d9740 1 -- 172.21.15.172:0/3162318340 shutdown_connections 2024-09-10T08:50:23.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.487+0000 7f74434d9740 1 -- 172.21.15.172:0/3162318340 wait complete. 2024-09-10T08:50:23.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.488+0000 7f74434d9740 1 -- start start 2024-09-10T08:50:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.488+0000 7f74434d9740 1 Processor -- start 2024-09-10T08:50:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.488+0000 7f74434d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e78b60 0x5583c7e80ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.488+0000 7f74434d9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e79480 0x5583c7e811e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.488+0000 7f74434d9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e83d40 0x5583c7e81720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.488+0000 7f74434d9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5583c7e86910 con 0x5583c7e83d40 2024-09-10T08:50:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.488+0000 7f74434d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5583c7d98f10 con 0x5583c7e78b60 2024-09-10T08:50:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.488+0000 7f74434d9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5583c7e86a90 con 0x5583c7e79480 2024-09-10T08:50:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.489+0000 7f74434d9740 10 client.0 _ll_register_callbacks cb 0x5583c7d61a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.489+0000 7f743b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e83d40 0x5583c7e81720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.489+0000 7f743b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e83d40 0x5583c7e81720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46592/0 (socket says 172.21.15.172:46592) 2024-09-10T08:50:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.489+0000 7f743b59e640 1 -- 172.21.15.172:0/239871280 learned_addr learned my addr 172.21.15.172:0/239871280 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:23.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.489+0000 7f743a59c640 1 --2- 172.21.15.172:0/239871280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e79480 0x5583c7e811e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.489+0000 7f743b59e640 1 --2- 172.21.15.172:0/239871280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e78b60 0x5583c7e80ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.489+0000 7f743b59e640 1 -- 172.21.15.172:0/239871280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e79480 msgr2=0x5583c7e811e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.489+0000 7f743b59e640 1 --2- 172.21.15.172:0/239871280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e79480 0x5583c7e811e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.489+0000 7f743b59e640 1 -- 172.21.15.172:0/239871280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e78b60 msgr2=0x5583c7e80ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.489+0000 7f743b59e640 1 --2- 172.21.15.172:0/239871280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e78b60 0x5583c7e80ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.489+0000 7f743b59e640 1 -- 172.21.15.172:0/239871280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583c7b4f6a0 con 0x5583c7e83d40 2024-09-10T08:50:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.489+0000 7f743b59e640 1 --2- 172.21.15.172:0/239871280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e83d40 0x5583c7e81720 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7434022a70 tx=0x7f7434004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da3b6b557d3179d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.489+0000 7f741eff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.490+0000 7f741eff5640 1 -- 172.21.15.172:0/239871280 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434035040 con 0x5583c7e83d40 2024-09-10T08:50:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.490+0000 7f74434d9740 1 -- 172.21.15.172:0/239871280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74340054d0 con 0x5583c7e83d40 2024-09-10T08:50:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.490+0000 7f741eff5640 1 -- 172.21.15.172:0/239871280 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f7434005940 con 0x5583c7e83d40 2024-09-10T08:50:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.490+0000 7f741eff5640 1 -- 172.21.15.172:0/239871280 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7434005000 con 0x5583c7e83d40 2024-09-10T08:50:23.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.490+0000 7f74434d9740 10 client.16638 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:23.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.490+0000 7f74434d9740 1 -- 172.21.15.172:0/239871280 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5583c7e92590 con 0x5583c7e83d40 2024-09-10T08:50:23.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.490+0000 7f741eff5640 1 -- 172.21.15.172:0/239871280 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74340054d0 con 0x5583c7e83d40 2024-09-10T08:50:23.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.490+0000 7f741eff5640 1 -- 172.21.15.172:0/239871280 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f743400f040 con 0x5583c7e83d40 2024-09-10T08:50:23.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.491+0000 7f74434d9740 10 client.16638 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:23.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.491+0000 7f74434d9740 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:23.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.491+0000 7f74434d9740 20 client.16638 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:23.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.491+0000 7f74434d9740 1 --2- 172.21.15.172:0/239871280 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5583c7e93940 0x5583c7e95d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.491+0000 7f74434d9740 4 client.16638 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:50:23.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.491+0000 7f74434d9740 1 -- 172.21.15.172:0/239871280 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583c7deb7f0 con 0x5583c7e93940 2024-09-10T08:50:23.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.491+0000 7f743a59c640 1 --2- 172.21.15.172:0/239871280 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5583c7e93940 0x5583c7e95d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:23.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.492+0000 7f741eff5640 10 client.16638 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:23.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.492+0000 7f743a59c640 1 --2- 172.21.15.172:0/239871280 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5583c7e93940 0x5583c7e95d30 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2e9d2eabe88b22d server_cookie=faaba5bfcb72fcc3 in_seq=0 out_seq=0 2024-09-10T08:50:23.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.492+0000 7f741eff5640 1 -- 172.21.15.172:0/239871280 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5583c7deb7f0 con 0x5583c7e93940 2024-09-10T08:50:23.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.492+0000 7f741eff5640 10 client.16638 handle_command_reply: tid=0 2024-09-10T08:50:23.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.492+0000 7f74434d9740 1 client.16638 shutdown 2024-09-10T08:50:23.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.492+0000 7f74434d9740 1 -- 172.21.15.172:0/239871280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e83d40 msgr2=0x5583c7e81720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:23.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.492+0000 7f74434d9740 1 --2- 172.21.15.172:0/239871280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e83d40 0x5583c7e81720 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7434022a70 tx=0x7f7434004f60 comp rx=0 tx=0).stop 2024-09-10T08:50:23.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.492+0000 7f74434d9740 1 -- 172.21.15.172:0/239871280 shutdown_connections 2024-09-10T08:50:23.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.493+0000 7f74434d9740 1 --2- 172.21.15.172:0/239871280 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5583c7e93940 0x5583c7e95d30 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.493+0000 7f74434d9740 1 --2- 172.21.15.172:0/239871280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e83d40 0x5583c7e81720 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.493+0000 7f74434d9740 1 --2- 172.21.15.172:0/239871280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e79480 0x5583c7e811e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.493+0000 7f74434d9740 1 --2- 172.21.15.172:0/239871280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e78b60 0x5583c7e80ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.493+0000 7f741eff5640 0 client.16638 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:23.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.493+0000 7f74434d9740 1 -- 172.21.15.172:0/239871280 >> 172.21.15.172:0/239871280 conn(0x5583c7e750f0 msgr2=0x5583c7e74270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:23.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.493+0000 7f74434d9740 1 -- 172.21.15.172:0/239871280 shutdown_connections 2024-09-10T08:50:23.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.493+0000 7f74434d9740 1 -- 172.21.15.172:0/239871280 wait complete. 2024-09-10T08:50:23.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.493+0000 7f74434d9740 20 client.16638 trim_cache size 0 max 16384 2024-09-10T08:50:23.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.519+0000 7f74434d9740 1 Processor -- start 2024-09-10T08:50:23.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.520+0000 7f74434d9740 1 -- start start 2024-09-10T08:50:23.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.520+0000 7f74434d9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e78b60 0x5583c7e81570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.520+0000 7f74434d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e79480 0x5583c7e9b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.520+0000 7f74434d9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e83d40 0x5583c7e9b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.520+0000 7f74434d9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1aca30 con 0x5583c7e83d40 2024-09-10T08:50:23.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.520+0000 7f74434d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1ae630 con 0x5583c7e79480 2024-09-10T08:50:23.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.520+0000 7f74434d9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1ad100 con 0x5583c7e78b60 2024-09-10T08:50:23.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.520+0000 7f743b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e83d40 0x5583c7e9b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:23.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.520+0000 7f743b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e83d40 0x5583c7e9b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46604/0 (socket says 172.21.15.172:46604) 2024-09-10T08:50:23.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.520+0000 7f743b59e640 1 -- 172.21.15.172:0/2167427610 learned_addr learned my addr 172.21.15.172:0/2167427610 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:23.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.520+0000 7f743a59c640 1 --2- 172.21.15.172:0/2167427610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e79480 0x5583c7e9b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:23.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.520+0000 7f743b59e640 1 --2- 172.21.15.172:0/2167427610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e78b60 0x5583c7e81570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:23.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.520+0000 7f743b59e640 1 -- 172.21.15.172:0/2167427610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e78b60 msgr2=0x5583c7e81570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:23.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.520+0000 7f743b59e640 1 --2- 172.21.15.172:0/2167427610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e78b60 0x5583c7e81570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.520+0000 7f743b59e640 1 -- 172.21.15.172:0/2167427610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e79480 msgr2=0x5583c7e9b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:23.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.520+0000 7f743b59e640 1 --2- 172.21.15.172:0/2167427610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e79480 0x5583c7e9b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.520+0000 7f743b59e640 1 -- 172.21.15.172:0/2167427610 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583c7e792b0 con 0x5583c7e83d40 2024-09-10T08:50:23.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.521+0000 7f743b59e640 1 --2- 172.21.15.172:0/2167427610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e83d40 0x5583c7e9b990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7434003b90 tx=0x7f7434010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaad03f2df0b3981 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:23.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.521+0000 7f7420ff9640 1 -- 172.21.15.172:0/2167427610 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434022cb0 con 0x5583c7e83d40 2024-09-10T08:50:23.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.521+0000 7f7420ff9640 1 -- 172.21.15.172:0/2167427610 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74340056f0 con 0x5583c7e83d40 2024-09-10T08:50:23.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.521+0000 7f7420ff9640 1 -- 172.21.15.172:0/2167427610 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743402b6e0 con 0x5583c7e83d40 2024-09-10T08:50:23.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.522+0000 7f74434d9740 1 -- 172.21.15.172:0/2167427610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e83d40 msgr2=0x5583c7e9b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:23.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.522+0000 7f74434d9740 1 --2- 172.21.15.172:0/2167427610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e83d40 0x5583c7e9b990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7434003b90 tx=0x7f7434010800 comp rx=0 tx=0).stop 2024-09-10T08:50:23.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.522+0000 7f74434d9740 1 -- 172.21.15.172:0/2167427610 shutdown_connections 2024-09-10T08:50:23.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.522+0000 7f74434d9740 1 --2- 172.21.15.172:0/2167427610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e83d40 0x5583c7e9b990 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.522+0000 7f74434d9740 1 --2- 172.21.15.172:0/2167427610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e79480 0x5583c7e9b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.522+0000 7f74434d9740 1 --2- 172.21.15.172:0/2167427610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e78b60 0x5583c7e81570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.522+0000 7f74434d9740 1 -- 172.21.15.172:0/2167427610 >> 172.21.15.172:0/2167427610 conn(0x5583c7e750f0 msgr2=0x5583c7e82c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:23.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.522+0000 7f74434d9740 1 -- 172.21.15.172:0/2167427610 shutdown_connections 2024-09-10T08:50:23.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.522+0000 7f74434d9740 1 -- 172.21.15.172:0/2167427610 wait complete. 2024-09-10T08:50:23.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.522+0000 7f74434d9740 1 -- start start 2024-09-10T08:50:23.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.522+0000 7f74434d9740 1 Processor -- start 2024-09-10T08:50:23.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f74434d9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e78b60 0x5583c7e8c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f74434d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e79480 0x5583c7e87450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f74434d9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e83d40 0x5583c7e87990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f74434d9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1aded0 con 0x5583c7e78b60 2024-09-10T08:50:23.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f74434d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1ad800 con 0x5583c7e79480 2024-09-10T08:50:23.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f74434d9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1aaa10 con 0x5583c7e83d40 2024-09-10T08:50:23.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f743b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e78b60 0x5583c7e8c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:23.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f74434d9740 10 client.0 _ll_register_callbacks cb 0x7f742c0d99b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:23.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f743b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e78b60 0x5583c7e8c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46616/0 (socket says 172.21.15.172:46616) 2024-09-10T08:50:23.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f743b59e640 1 -- 172.21.15.172:0/690727071 learned_addr learned my addr 172.21.15.172:0/690727071 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:23.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f743a59c640 1 --2- 172.21.15.172:0/690727071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e79480 0x5583c7e87450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:23.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f743b59e640 1 -- 172.21.15.172:0/690727071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e83d40 msgr2=0x5583c7e87990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:23.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f743b59e640 1 --2- 172.21.15.172:0/690727071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e83d40 0x5583c7e87990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f743b59e640 1 -- 172.21.15.172:0/690727071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e79480 msgr2=0x5583c7e87450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:23.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f743b59e640 1 --2- 172.21.15.172:0/690727071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e79480 0x5583c7e87450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f743b59e640 1 -- 172.21.15.172:0/690727071 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583c7e87ed0 con 0x5583c7e78b60 2024-09-10T08:50:23.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f743a59c640 1 --2- 172.21.15.172:0/690727071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e79480 0x5583c7e87450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:50:23.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f743b59e640 1 --2- 172.21.15.172:0/690727071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e78b60 0x5583c7e8c120 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7434000f80 tx=0x7f74340227b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69e4393a656bb3c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:23.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.523+0000 7f741eff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:23.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.524+0000 7f741eff5640 1 -- 172.21.15.172:0/690727071 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434022c70 con 0x5583c7e78b60 2024-09-10T08:50:23.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.524+0000 7f741eff5640 1 -- 172.21.15.172:0/690727071 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f74340051a0 con 0x5583c7e78b60 2024-09-10T08:50:23.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.524+0000 7f74434d9740 1 -- 172.21.15.172:0/690727071 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74340056f0 con 0x5583c7e78b60 2024-09-10T08:50:23.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.524+0000 7f741eff5640 1 -- 172.21.15.172:0/690727071 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f74340053a0 con 0x5583c7e78b60 2024-09-10T08:50:23.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.524+0000 7f74434d9740 10 client.16644 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:23.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.524+0000 7f74434d9740 1 -- 172.21.15.172:0/690727071 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5583c7efe150 con 0x5583c7e78b60 2024-09-10T08:50:23.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.524+0000 7f741eff5640 1 -- 172.21.15.172:0/690727071 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74340056f0 con 0x5583c7e78b60 2024-09-10T08:50:23.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.524+0000 7f741eff5640 1 -- 172.21.15.172:0/690727071 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f743402ba30 con 0x5583c7e78b60 2024-09-10T08:50:23.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.524+0000 7f74434d9740 10 client.16644 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:23.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.524+0000 7f74434d9740 10 client.16644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:23.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.524+0000 7f74434d9740 20 client.16644 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:23.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.524+0000 7f74434d9740 1 --2- 172.21.15.172:0/690727071 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5583c7efe830 0x5583c7f00c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:23.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.525+0000 7f74434d9740 4 client.16644 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:50:23.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.525+0000 7f74434d9740 1 -- 172.21.15.172:0/690727071 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5583c7de5d00 con 0x5583c7efe830 2024-09-10T08:50:23.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.525+0000 7f743a59c640 1 --2- 172.21.15.172:0/690727071 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5583c7efe830 0x5583c7f00c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:23.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.525+0000 7f741eff5640 10 client.16644 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:23.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.525+0000 7f743a59c640 1 --2- 172.21.15.172:0/690727071 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5583c7efe830 0x5583c7f00c50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c724dbdded2d84ef server_cookie=dc77269eb337f998 in_seq=0 out_seq=0 2024-09-10T08:50:23.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.526+0000 7f741eff5640 1 -- 172.21.15.172:0/690727071 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5583c7de5d00 con 0x5583c7efe830 2024-09-10T08:50:23.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.526+0000 7f741eff5640 10 client.16644 handle_command_reply: tid=0 2024-09-10T08:50:23.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.526+0000 7f74434d9740 1 client.16644 shutdown 2024-09-10T08:50:23.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.526+0000 7f74434d9740 1 -- 172.21.15.172:0/690727071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e78b60 msgr2=0x5583c7e8c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:23.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.526+0000 7f74434d9740 1 --2- 172.21.15.172:0/690727071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e78b60 0x5583c7e8c120 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7434000f80 tx=0x7f74340227b0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.526+0000 7f74434d9740 1 -- 172.21.15.172:0/690727071 shutdown_connections 2024-09-10T08:50:23.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.526+0000 7f74434d9740 1 --2- 172.21.15.172:0/690727071 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5583c7efe830 0x5583c7f00c50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.526+0000 7f74434d9740 1 --2- 172.21.15.172:0/690727071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5583c7e83d40 0x5583c7e87990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.526+0000 7f741eff5640 0 client.16644 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:23.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.526+0000 7f74434d9740 1 --2- 172.21.15.172:0/690727071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5583c7e79480 0x5583c7e87450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.526+0000 7f74434d9740 1 --2- 172.21.15.172:0/690727071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5583c7e78b60 0x5583c7e8c120 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.526+0000 7f74434d9740 1 -- 172.21.15.172:0/690727071 >> 172.21.15.172:0/690727071 conn(0x5583c7e750f0 msgr2=0x5583c7e7a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:23.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.526+0000 7f74434d9740 1 -- 172.21.15.172:0/690727071 shutdown_connections 2024-09-10T08:50:23.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.527+0000 7f74434d9740 1 -- 172.21.15.172:0/690727071 wait complete. 2024-09-10T08:50:23.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.527+0000 7f74434d9740 20 client.16644 trim_cache size 0 max 16384 2024-09-10T08:50:23.533 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:50:23.533 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T08:50:23.533 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "ad70bd8d-8317-43da-85d0-0b5cd613f1dd", 2024-09-10T08:50:23.533 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T08:50:23.534 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:50:23.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.527+0000 7f7441100640 1 -- 172.21.15.172:0/1076968307 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f74240a2b80 msgr2=0x7f74240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:23.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.527+0000 7f7441100640 1 --2- 172.21.15.172:0/1076968307 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f74240a2b80 0x7f74240a5040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f742c0cc850 tx=0x7f742c0d9ae0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.527+0000 7f7441100640 1 -- 172.21.15.172:0/1076968307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f743c101dd0 msgr2=0x7f743c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:23.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.527+0000 7f7441100640 1 --2- 172.21.15.172:0/1076968307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f743c101dd0 0x7f743c198370 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f742c00d950 tx=0x7f742c00de10 comp rx=0 tx=0).stop 2024-09-10T08:50:23.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.527+0000 7f7441100640 1 -- 172.21.15.172:0/1076968307 shutdown_connections 2024-09-10T08:50:23.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.527+0000 7f7441100640 1 --2- 172.21.15.172:0/1076968307 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f74240a2b80 0x7f74240a5040 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.527+0000 7f7441100640 1 --2- 172.21.15.172:0/1076968307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f743c106f00 0x7f743c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.527+0000 7f7441100640 1 --2- 172.21.15.172:0/1076968307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f743c102780 0x7f743c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.527+0000 7f7441100640 1 --2- 172.21.15.172:0/1076968307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f743c101dd0 0x7f743c198370 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:23.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.527+0000 7f7441100640 1 -- 172.21.15.172:0/1076968307 >> 172.21.15.172:0/1076968307 conn(0x7f743c0fd8b0 msgr2=0x7f743c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:23.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.528+0000 7f7441100640 1 -- 172.21.15.172:0/1076968307 shutdown_connections 2024-09-10T08:50:23.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:23.528+0000 7f7441100640 1 -- 172.21.15.172:0/1076968307 wait complete. 2024-09-10T08:50:24.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.290+0000 7f75d924f640 1 -- 172.21.15.172:0/2245548795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75d4102ef0 msgr2=0x7f75d410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:24.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.290+0000 7f75d924f640 1 --2- 172.21.15.172:0/2245548795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75d4102ef0 0x7f75d410da40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009ca0 tx=0x7f75cc02a950 comp rx=0 tx=0).stop 2024-09-10T08:50:24.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.290+0000 7f75d924f640 1 -- 172.21.15.172:0/2245548795 shutdown_connections 2024-09-10T08:50:24.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.290+0000 7f75d924f640 1 --2- 172.21.15.172:0/2245548795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75d4102ef0 0x7f75d410da40 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:24.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.290+0000 7f75d924f640 1 --2- 172.21.15.172:0/2245548795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75d4102570 0x7f75d41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:24.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.290+0000 7f75d924f640 1 --2- 172.21.15.172:0/2245548795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d4100e20 0x7f75d4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:24.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.290+0000 7f75d924f640 1 -- 172.21.15.172:0/2245548795 >> 172.21.15.172:0/2245548795 conn(0x7f75d40fc7d0 msgr2=0x7f75d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:24.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.291+0000 7f75d924f640 1 -- 172.21.15.172:0/2245548795 shutdown_connections 2024-09-10T08:50:24.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.291+0000 7f75d924f640 1 -- 172.21.15.172:0/2245548795 wait complete. 2024-09-10T08:50:24.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.291+0000 7f75d924f640 1 Processor -- start 2024-09-10T08:50:24.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.292+0000 7f75d924f640 1 -- start start 2024-09-10T08:50:24.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.292+0000 7f75d924f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75d4100e20 0x7f75d41991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:24.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.292+0000 7f75d924f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75d4102570 0x7f75d4199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:24.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.292+0000 7f75d924f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d4102ef0 0x7f75d41a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:24.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.292+0000 7f75d924f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f75d4110e50 con 0x7f75d4100e20 2024-09-10T08:50:24.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.292+0000 7f75d924f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f75d4110cd0 con 0x7f75d4102570 2024-09-10T08:50:24.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.292+0000 7f75d924f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f75d4110fd0 con 0x7f75d4102ef0 2024-09-10T08:50:24.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.292+0000 7f75d2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75d4100e20 0x7f75d41991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:24.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.292+0000 7f75d2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75d4100e20 0x7f75d41991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46646/0 (socket says 172.21.15.172:46646) 2024-09-10T08:50:24.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.292+0000 7f75d2d9d640 1 -- 172.21.15.172:0/514178401 learned_addr learned my addr 172.21.15.172:0/514178401 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:24.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.292+0000 7f75d259c640 1 --2- 172.21.15.172:0/514178401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75d4102570 0x7f75d4199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:24.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.292+0000 7f75d359e640 1 --2- 172.21.15.172:0/514178401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d4102ef0 0x7f75d41a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:24.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.292+0000 7f75d2d9d640 1 -- 172.21.15.172:0/514178401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d4102ef0 msgr2=0x7f75d41a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:24.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.292+0000 7f75d2d9d640 1 --2- 172.21.15.172:0/514178401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d4102ef0 0x7f75d41a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:24.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.292+0000 7f75d2d9d640 1 -- 172.21.15.172:0/514178401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75d4102570 msgr2=0x7f75d4199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:24.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.292+0000 7f75d2d9d640 1 --2- 172.21.15.172:0/514178401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75d4102570 0x7f75d4199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:24.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.292+0000 7f75d2d9d640 1 -- 172.21.15.172:0/514178401 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d41a4780 con 0x7f75d4100e20 2024-09-10T08:50:24.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.293+0000 7f75d2d9d640 1 --2- 172.21.15.172:0/514178401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75d4100e20 0x7f75d41991c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f75c400c920 tx=0x7f75c400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:24.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.293+0000 7f75bbfff640 1 -- 172.21.15.172:0/514178401 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4007bb0 con 0x7f75d4100e20 2024-09-10T08:50:24.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.293+0000 7f75bbfff640 1 -- 172.21.15.172:0/514178401 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75c4007d50 con 0x7f75d4100e20 2024-09-10T08:50:24.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.293+0000 7f75bbfff640 1 -- 172.21.15.172:0/514178401 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4018730 con 0x7f75d4100e20 2024-09-10T08:50:24.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.293+0000 7f75d924f640 1 -- 172.21.15.172:0/514178401 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d4196fb0 con 0x7f75d4100e20 2024-09-10T08:50:24.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.293+0000 7f75d924f640 1 -- 172.21.15.172:0/514178401 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d41974a0 con 0x7f75d4100e20 2024-09-10T08:50:24.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.294+0000 7f75d924f640 1 -- 172.21.15.172:0/514178401 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d4101200 con 0x7f75d4100e20 2024-09-10T08:50:24.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.295+0000 7f75bbfff640 1 -- 172.21.15.172:0/514178401 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f75c400f450 con 0x7f75d4100e20 2024-09-10T08:50:24.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.296+0000 7f75bbfff640 1 --2- 172.21.15.172:0/514178401 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f75bc0a2b80 0x7f75bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:24.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.296+0000 7f75d259c640 1 --2- 172.21.15.172:0/514178401 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f75bc0a2b80 0x7f75bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:24.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.296+0000 7f75bbfff640 1 -- 172.21.15.172:0/514178401 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f75c40cf4f0 con 0x7f75d4100e20 2024-09-10T08:50:24.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.297+0000 7f75d259c640 1 --2- 172.21.15.172:0/514178401 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f75bc0a2b80 0x7f75bc0a5040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f75d406ab00 tx=0x7f75c8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:24.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.301+0000 7f75bbfff640 1 -- 172.21.15.172:0/514178401 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f75c4014030 con 0x7f75d4100e20 2024-09-10T08:50:24.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.553+0000 7f75d924f640 1 -- 172.21.15.172:0/514178401 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75d4197760 con 0x7f75d4100e20 2024-09-10T08:50:24.556 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:24 smithi172 ceph-mon[30854]: pgmap v475: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-10T08:50:24.556 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:24 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:50:24.556 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:24 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:50:24.556 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:24 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:50:24.556 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:24 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:24.557 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:24 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:24.557 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:24 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:50:24.557 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:24 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:24.557 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:24 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:50:24.557 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:24 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:24.557 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:24 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:24.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.554+0000 7f75bbfff640 1 -- 172.21.15.172:0/514178401 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+12007 (secure 0 0 0) 0x7f75c400de20 con 0x7f75d4100e20 2024-09-10T08:50:24.558 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:50:24.558 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":33,"btime":"2024-09-10T08:50:03:285217+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:50:02.654978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:50:24.559 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 33 2024-09-10T08:50:24.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.557+0000 7f75d924f640 1 -- 172.21.15.172:0/514178401 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f75bc0a2b80 msgr2=0x7f75bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:24.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.557+0000 7f75d924f640 1 --2- 172.21.15.172:0/514178401 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f75bc0a2b80 0x7f75bc0a5040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f75d406ab00 tx=0x7f75c8009110 comp rx=0 tx=0).stop 2024-09-10T08:50:24.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.557+0000 7f75d924f640 1 -- 172.21.15.172:0/514178401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75d4100e20 msgr2=0x7f75d41991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:24.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.557+0000 7f75d924f640 1 --2- 172.21.15.172:0/514178401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75d4100e20 0x7f75d41991c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f75c400c920 tx=0x7f75c400cd20 comp rx=0 tx=0).stop 2024-09-10T08:50:24.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.557+0000 7f75d924f640 1 -- 172.21.15.172:0/514178401 shutdown_connections 2024-09-10T08:50:24.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.557+0000 7f75d924f640 1 --2- 172.21.15.172:0/514178401 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f75bc0a2b80 0x7f75bc0a5040 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:24.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.557+0000 7f75d924f640 1 --2- 172.21.15.172:0/514178401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d4102ef0 0x7f75d41a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:24.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.557+0000 7f75d924f640 1 --2- 172.21.15.172:0/514178401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75d4102570 0x7f75d4199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:24.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.557+0000 7f75d924f640 1 --2- 172.21.15.172:0/514178401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75d4100e20 0x7f75d41991c0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:24.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.557+0000 7f75d924f640 1 -- 172.21.15.172:0/514178401 >> 172.21.15.172:0/514178401 conn(0x7f75d40fc7d0 msgr2=0x7f75d4106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:24.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.557+0000 7f75d924f640 1 -- 172.21.15.172:0/514178401 shutdown_connections 2024-09-10T08:50:24.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:24.557+0000 7f75d924f640 1 -- 172.21.15.172:0/514178401 wait complete. 2024-09-10T08:50:24.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:24 smithi174 ceph-mon[34068]: pgmap v475: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-10T08:50:24.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:24 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:50:24.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:24 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:50:24.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:24 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:50:24.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:24 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:24.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:24 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:24.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:24 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:50:24.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:24 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:24.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:24 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:50:24.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:24 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:24.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:24 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:24.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:24 smithi191 ceph-mon[35314]: pgmap v475: 129 pgs: 129 active+clean; 2.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-10T08:50:24.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:24 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:50:24.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:24 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:50:24.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:24 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:50:24.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:24 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:24.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:24 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:24.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:24 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:50:24.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:24 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:24.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:24 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:50:24.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:24 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:24.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:24 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:24.987 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:50:25.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:25 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T08:50:25.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:25 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:25.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:25 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:50:25.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:25 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:25.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:25 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/514178401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:50:25.672 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T08:50:25.672 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T08:50:25.712 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:50:25.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:25 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T08:50:25.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:25 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:25.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:25 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:50:25.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:25 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:25.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:25 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/514178401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:50:25.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:25 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T08:50:25.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:25 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:25.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:25 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:50:25.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:25 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:50:25.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:25 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/514178401' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:50:26.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.233+0000 7f7b25872640 1 -- 172.21.15.172:0/3123240113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b20103740 msgr2=0x7f7b20107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:26.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.233+0000 7f7b25872640 1 --2- 172.21.15.172:0/3123240113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b20103740 0x7f7b20107b80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009930 tx=0x7f7b1402a420 comp rx=0 tx=0).stop 2024-09-10T08:50:26.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.233+0000 7f7b25872640 1 -- 172.21.15.172:0/3123240113 shutdown_connections 2024-09-10T08:50:26.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.233+0000 7f7b25872640 1 --2- 172.21.15.172:0/3123240113 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b20108290 0x7f7b2010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.233+0000 7f7b25872640 1 --2- 172.21.15.172:0/3123240113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b20103740 0x7f7b20107b80 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.233+0000 7f7b25872640 1 --2- 172.21.15.172:0/3123240113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b20102d90 0x7f7b20103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.233+0000 7f7b25872640 1 -- 172.21.15.172:0/3123240113 >> 172.21.15.172:0/3123240113 conn(0x7f7b200fc7d0 msgr2=0x7f7b200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:26.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.233+0000 7f7b25872640 1 -- 172.21.15.172:0/3123240113 shutdown_connections 2024-09-10T08:50:26.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.233+0000 7f7b25872640 1 -- 172.21.15.172:0/3123240113 wait complete. 2024-09-10T08:50:26.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.234+0000 7f7b25872640 1 Processor -- start 2024-09-10T08:50:26.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.234+0000 7f7b25872640 1 -- start start 2024-09-10T08:50:26.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.234+0000 7f7b25872640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b20102d90 0x7f7b201984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.234+0000 7f7b25872640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b20103740 0x7f7b20198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.234+0000 7f7b25872640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b20108290 0x7f7b2019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.234+0000 7f7b25872640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7b20111f40 con 0x7f7b20103740 2024-09-10T08:50:26.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.234+0000 7f7b25872640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b20111dc0 con 0x7f7b20102d90 2024-09-10T08:50:26.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.234+0000 7f7b25872640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7b201120c0 con 0x7f7b20108290 2024-09-10T08:50:26.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.234+0000 7f7b1e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b20103740 0x7f7b20198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.234+0000 7f7b1e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b20103740 0x7f7b20198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38312/0 (socket says 172.21.15.172:38312) 2024-09-10T08:50:26.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.234+0000 7f7b1e7fc640 1 -- 172.21.15.172:0/1832653315 learned_addr learned my addr 172.21.15.172:0/1832653315 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:26.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.235+0000 7f7b1effd640 1 --2- 172.21.15.172:0/1832653315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b20102d90 0x7f7b201984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.235+0000 7f7b1f7fe640 1 --2- 172.21.15.172:0/1832653315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b20108290 0x7f7b2019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.235+0000 7f7b1e7fc640 1 -- 172.21.15.172:0/1832653315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b20108290 msgr2=0x7f7b2019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:26.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.235+0000 7f7b1e7fc640 1 --2- 172.21.15.172:0/1832653315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b20108290 0x7f7b2019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.235+0000 7f7b1e7fc640 1 -- 172.21.15.172:0/1832653315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b20102d90 msgr2=0x7f7b201984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:26.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.235+0000 7f7b1e7fc640 1 --2- 172.21.15.172:0/1832653315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b20102d90 0x7f7b201984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.235+0000 7f7b1e7fc640 1 -- 172.21.15.172:0/1832653315 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2019d490 con 0x7f7b20103740 2024-09-10T08:50:26.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.235+0000 7f7b1f7fe640 1 --2- 172.21.15.172:0/1832653315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b20108290 0x7f7b2019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:50:26.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.235+0000 7f7b1e7fc640 1 --2- 172.21.15.172:0/1832653315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b20103740 0x7f7b20198a20 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7b14031440 tx=0x7f7b14004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:26.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.235+0000 7f7b24870640 1 -- 172.21.15.172:0/1832653315 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1403f030 con 0x7f7b20103740 2024-09-10T08:50:26.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.235+0000 7f7b24870640 1 -- 172.21.15.172:0/1832653315 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b14003aa0 con 0x7f7b20103740 2024-09-10T08:50:26.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.235+0000 7f7b24870640 1 -- 172.21.15.172:0/1832653315 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14035070 con 0x7f7b20103740 2024-09-10T08:50:26.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.235+0000 7f7b25872640 1 -- 172.21.15.172:0/1832653315 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2019d720 con 0x7f7b20103740 2024-09-10T08:50:26.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.235+0000 7f7b25872640 1 -- 172.21.15.172:0/1832653315 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b2019dbb0 con 0x7f7b20103740 2024-09-10T08:50:26.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.238+0000 7f7b24870640 1 -- 172.21.15.172:0/1832653315 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7b14042030 con 0x7f7b20103740 2024-09-10T08:50:26.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.238+0000 7f7b27c4b740 1 Processor -- start 2024-09-10T08:50:26.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.238+0000 7f7b27c4b740 1 -- start start 2024-09-10T08:50:26.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.238+0000 7f7b27c4b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b6433380 0x55f2b6433760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.238+0000 7f7b27c4b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 0x55f2b6442550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.238+0000 7f7b27c4b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6442a90 0x55f2b6444e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.238+0000 7f7b27c4b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f2b63992e0 con 0x55f2b6442a90 2024-09-10T08:50:26.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.238+0000 7f7b27c4b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f2b6340ce0 con 0x55f2b6433ca0 2024-09-10T08:50:26.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.238+0000 7f7b27c4b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f2b63952a0 con 0x55f2b6433380 2024-09-10T08:50:26.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.239+0000 7f7b1e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6442a90 0x55f2b6444e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.239+0000 7f7b1e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6442a90 0x55f2b6444e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38318/0 (socket says 172.21.15.172:38318) 2024-09-10T08:50:26.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.239+0000 7f7b1e7fc640 1 -- 172.21.15.172:0/18507926 learned_addr learned my addr 172.21.15.172:0/18507926 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:26.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.239+0000 7f7b24870640 1 --2- 172.21.15.172:0/1832653315 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7b080a2c00 0x7f7b080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.239+0000 7f7b1effd640 1 --2- 172.21.15.172:0/18507926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 0x55f2b6442550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.239+0000 7f7b1f7fe640 1 --2- 172.21.15.172:0/18507926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b6433380 0x55f2b6433760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.239+0000 7f7b1f7fe640 1 --2- 172.21.15.172:0/1832653315 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7b080a2c00 0x7f7b080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.239+0000 7f7b24870640 1 -- 172.21.15.172:0/1832653315 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f7b140ef910 con 0x7f7b20103740 2024-09-10T08:50:26.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.239+0000 7f7b1effd640 1 -- 172.21.15.172:0/18507926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b6433380 msgr2=0x55f2b6433760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:26.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.239+0000 7f7b1effd640 1 --2- 172.21.15.172:0/18507926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b6433380 0x55f2b6433760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.239+0000 7f7b1effd640 1 -- 172.21.15.172:0/18507926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6442a90 msgr2=0x55f2b6444e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:26.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.239+0000 7f7b1effd640 1 --2- 172.21.15.172:0/18507926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6442a90 0x55f2b6444e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.239+0000 7f7b1effd640 1 -- 172.21.15.172:0/18507926 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b6325440 con 0x55f2b6433ca0 2024-09-10T08:50:26.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.239+0000 7f7b1e7fc640 1 --2- 172.21.15.172:0/18507926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6442a90 0x55f2b6444e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:26.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.239+0000 7f7b1f7fe640 1 --2- 172.21.15.172:0/18507926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b6433380 0x55f2b6433760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:26.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.239+0000 7f7b1effd640 1 --2- 172.21.15.172:0/18507926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 0x55f2b6442550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009c20 tx=0x7f7b10011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79adb7df91504a29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:26.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.239+0000 7f7b1f7fe640 1 --2- 172.21.15.172:0/1832653315 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7b080a2c00 0x7f7b080a50c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7b18005fd0 tx=0x7f7b18005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:26.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.240+0000 7f7b05ffb640 1 -- 172.21.15.172:0/18507926 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b10004050 con 0x55f2b6433ca0 2024-09-10T08:50:26.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.240+0000 7f7b05ffb640 1 -- 172.21.15.172:0/18507926 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b100054d0 con 0x55f2b6433ca0 2024-09-10T08:50:26.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.240+0000 7f7b05ffb640 1 -- 172.21.15.172:0/18507926 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b10004d30 con 0x55f2b6433ca0 2024-09-10T08:50:26.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.240+0000 7f7b27c4b740 1 -- 172.21.15.172:0/18507926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 msgr2=0x55f2b6442550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:26.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.240+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/18507926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 0x55f2b6442550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009c20 tx=0x7f7b10011810 comp rx=0 tx=0).stop 2024-09-10T08:50:26.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.241+0000 7f7b27c4b740 1 -- 172.21.15.172:0/18507926 shutdown_connections 2024-09-10T08:50:26.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.241+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/18507926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6442a90 0x55f2b6444e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.241+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/18507926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 0x55f2b6442550 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.241+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/18507926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b6433380 0x55f2b6433760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.241+0000 7f7b27c4b740 1 -- 172.21.15.172:0/18507926 >> 172.21.15.172:0/18507926 conn(0x55f2b642fa30 msgr2=0x55f2b642fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:26.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.241+0000 7f7b27c4b740 1 -- 172.21.15.172:0/18507926 shutdown_connections 2024-09-10T08:50:26.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.241+0000 7f7b27c4b740 1 -- 172.21.15.172:0/18507926 wait complete. 2024-09-10T08:50:26.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.241+0000 7f7b27c4b740 1 -- start start 2024-09-10T08:50:26.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.241+0000 7f7b27c4b740 1 Processor -- start 2024-09-10T08:50:26.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.241+0000 7f7b27c4b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433380 0x55f2b6442380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.467 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:50:26.467 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:50:26.467 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T08:50:26.467 INFO:teuthology.orchestra.run.smithi172.stdout: "ad70bd8d-8317-43da-85d0-0b5cd613f1dd": { 2024-09-10T08:50:26.467 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T08:50:26.467 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "ad70bd8d-8317-43da-85d0-0b5cd613f1dd", 2024-09-10T08:50:26.467 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T08:50:26.467 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:50:26.467 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:50:26.467 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:50:26.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.241+0000 7f7b27c4b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 0x55f2b643f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.241+0000 7f7b27c4b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b6442a90 0x55f2b643fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7b27c4b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f2b6445660 con 0x55f2b6433380 2024-09-10T08:50:26.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7b27c4b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f2b6354050 con 0x55f2b6433ca0 2024-09-10T08:50:26.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7b27c4b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f2b64457e0 con 0x55f2b6442a90 2024-09-10T08:50:26.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7b27c4b740 10 client.0 _ll_register_callbacks cb 0x55f2b6394160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:26.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7b1e7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 0x55f2b643f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7b1effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433380 0x55f2b6442380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7b1e7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 0x55f2b643f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:38068/0 (socket says 172.21.15.172:38068) 2024-09-10T08:50:26.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7b1e7fc640 1 -- 172.21.15.172:0/1262599987 learned_addr learned my addr 172.21.15.172:0/1262599987 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:26.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7b1f7fe640 1 --2- 172.21.15.172:0/1262599987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b6442a90 0x55f2b643fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7b1effd640 1 -- 172.21.15.172:0/1262599987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b6442a90 msgr2=0x55f2b643fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:26.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7b1effd640 1 --2- 172.21.15.172:0/1262599987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b6442a90 0x55f2b643fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7b1effd640 1 -- 172.21.15.172:0/1262599987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 msgr2=0x55f2b643f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:26.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7b1effd640 1 --2- 172.21.15.172:0/1262599987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 0x55f2b643f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7b1effd640 1 -- 172.21.15.172:0/1262599987 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b610a3c0 con 0x55f2b6433380 2024-09-10T08:50:26.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7b1e7fc640 1 --2- 172.21.15.172:0/1262599987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 0x55f2b643f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:50:26.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7b1effd640 1 --2- 172.21.15.172:0/1262599987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433380 0x55f2b6442380 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7b10000c00 tx=0x7f7b10005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d197d60c486fac98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:26.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7afeffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:26.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7afeffd640 1 -- 172.21.15.172:0/1262599987 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b10003a00 con 0x55f2b6433380 2024-09-10T08:50:26.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7afeffd640 1 -- 172.21.15.172:0/1262599987 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f7b10003ba0 con 0x55f2b6433380 2024-09-10T08:50:26.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.242+0000 7f7b27c4b740 1 -- 172.21.15.172:0/1262599987 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b100054d0 con 0x55f2b6433380 2024-09-10T08:50:26.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.243+0000 7f7afeffd640 1 -- 172.21.15.172:0/1262599987 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7b10033050 con 0x55f2b6433380 2024-09-10T08:50:26.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.243+0000 7f7b27c4b740 10 client.16662 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:26.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.243+0000 7f7b27c4b740 1 -- 172.21.15.172:0/1262599987 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f2b6451010 con 0x55f2b6433380 2024-09-10T08:50:26.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.243+0000 7f7afeffd640 1 -- 172.21.15.172:0/1262599987 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b100054d0 con 0x55f2b6433380 2024-09-10T08:50:26.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.243+0000 7f7afeffd640 1 -- 172.21.15.172:0/1262599987 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f7b1002e440 con 0x55f2b6433380 2024-09-10T08:50:26.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.243+0000 7f7b27c4b740 10 client.16662 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:26.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.243+0000 7f7b27c4b740 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:26.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.243+0000 7f7b27c4b740 20 client.16662 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:26.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.243+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/1262599987 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f2b6452410 0x55f2b6454800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.243+0000 7f7b27c4b740 4 client.16662 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:50:26.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.243+0000 7f7b27c4b740 1 -- 172.21.15.172:0/1262599987 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2b63a6850 con 0x55f2b6452410 2024-09-10T08:50:26.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.243+0000 7f7b1e7fc640 1 --2- 172.21.15.172:0/1262599987 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f2b6452410 0x55f2b6454800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.244+0000 7f7afeffd640 10 client.16662 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:26.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.244+0000 7f7b1e7fc640 1 --2- 172.21.15.172:0/1262599987 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f2b6452410 0x55f2b6454800 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82c41cf6a784d46e server_cookie=8bf9c97ac8504808 in_seq=0 out_seq=0 2024-09-10T08:50:26.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.244+0000 7f7afeffd640 1 -- 172.21.15.172:0/1262599987 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f2b63a6850 con 0x55f2b6452410 2024-09-10T08:50:26.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.244+0000 7f7afeffd640 10 client.16662 handle_command_reply: tid=0 2024-09-10T08:50:26.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.244+0000 7f7b27c4b740 1 client.16662 shutdown 2024-09-10T08:50:26.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.245+0000 7f7b27c4b740 1 -- 172.21.15.172:0/1262599987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433380 msgr2=0x55f2b6442380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:26.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.245+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/1262599987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433380 0x55f2b6442380 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f7b10000c00 tx=0x7f7b10005720 comp rx=0 tx=0).stop 2024-09-10T08:50:26.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.245+0000 7f7b27c4b740 1 -- 172.21.15.172:0/1262599987 shutdown_connections 2024-09-10T08:50:26.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.245+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/1262599987 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f2b6452410 0x55f2b6454800 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.245+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/1262599987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b6442a90 0x55f2b643fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.245+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/1262599987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 0x55f2b643f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.245+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/1262599987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433380 0x55f2b6442380 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.245+0000 7f7b27c4b740 1 -- 172.21.15.172:0/1262599987 >> 172.21.15.172:0/1262599987 conn(0x55f2b642fa30 msgr2=0x55f2b6435790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:26.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.245+0000 7f7afeffd640 0 client.16662 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:26.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.245+0000 7f7b27c4b740 1 -- 172.21.15.172:0/1262599987 shutdown_connections 2024-09-10T08:50:26.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.245+0000 7f7b27c4b740 1 -- 172.21.15.172:0/1262599987 wait complete. 2024-09-10T08:50:26.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.245+0000 7f7b27c4b740 20 client.16662 trim_cache size 0 max 16384 2024-09-10T08:50:26.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.264+0000 7f7b27c4b740 1 Processor -- start 2024-09-10T08:50:26.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.264+0000 7f7b27c4b740 1 -- start start 2024-09-10T08:50:26.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.264+0000 7f7b27c4b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b642fa30 0x55f2b64403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.264+0000 7f7b27c4b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433380 0x55f2b644e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.264+0000 7f7b27c4b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 0x55f2b643ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.264+0000 7f7b27c4b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7b201ac970 con 0x55f2b6433380 2024-09-10T08:50:26.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.264+0000 7f7b27c4b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b201ae570 con 0x55f2b6433ca0 2024-09-10T08:50:26.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.264+0000 7f7b27c4b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7b201ad040 con 0x55f2b642fa30 2024-09-10T08:50:26.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.264+0000 7f7b1e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433380 0x55f2b644e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.264+0000 7f7b1e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433380 0x55f2b644e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38336/0 (socket says 172.21.15.172:38336) 2024-09-10T08:50:26.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.264+0000 7f7b1e7fc640 1 -- 172.21.15.172:0/959813171 learned_addr learned my addr 172.21.15.172:0/959813171 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:26.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.265+0000 7f7b1effd640 1 --2- 172.21.15.172:0/959813171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b642fa30 0x55f2b64403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.265+0000 7f7b1f7fe640 1 --2- 172.21.15.172:0/959813171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 0x55f2b643ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.265+0000 7f7b1e7fc640 1 -- 172.21.15.172:0/959813171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b642fa30 msgr2=0x55f2b64403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:26.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.265+0000 7f7b1e7fc640 1 --2- 172.21.15.172:0/959813171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b642fa30 0x55f2b64403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.265+0000 7f7b1e7fc640 1 -- 172.21.15.172:0/959813171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 msgr2=0x55f2b643ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:26.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.265+0000 7f7b1e7fc640 1 --2- 172.21.15.172:0/959813171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 0x55f2b643ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.265+0000 7f7b1e7fc640 1 -- 172.21.15.172:0/959813171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b6433ad0 con 0x55f2b6433380 2024-09-10T08:50:26.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.265+0000 7f7b1effd640 1 --2- 172.21.15.172:0/959813171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b642fa30 0x55f2b64403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:26.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.265+0000 7f7b1e7fc640 1 --2- 172.21.15.172:0/959813171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433380 0x55f2b644e900 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f7b140b8270 tx=0x7f7b140dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64014a92918a34ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:26.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.265+0000 7f7b057fa640 1 -- 172.21.15.172:0/959813171 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14049040 con 0x55f2b6433380 2024-09-10T08:50:26.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.265+0000 7f7b057fa640 1 -- 172.21.15.172:0/959813171 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b140c01c0 con 0x55f2b6433380 2024-09-10T08:50:26.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.265+0000 7f7b057fa640 1 -- 172.21.15.172:0/959813171 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14003c50 con 0x55f2b6433380 2024-09-10T08:50:26.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.266+0000 7f7b27c4b740 1 -- 172.21.15.172:0/959813171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433380 msgr2=0x55f2b644e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:26.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.266+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/959813171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433380 0x55f2b644e900 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f7b140b8270 tx=0x7f7b140dc040 comp rx=0 tx=0).stop 2024-09-10T08:50:26.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.266+0000 7f7b27c4b740 1 -- 172.21.15.172:0/959813171 shutdown_connections 2024-09-10T08:50:26.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.266+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/959813171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433ca0 0x55f2b643ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.266+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/959813171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433380 0x55f2b644e900 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.266+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/959813171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b642fa30 0x55f2b64403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.266+0000 7f7b27c4b740 1 -- 172.21.15.172:0/959813171 >> 172.21.15.172:0/959813171 conn(0x55f2b6454bf0 msgr2=0x55f2b643df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:26.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.266+0000 7f7b27c4b740 1 -- 172.21.15.172:0/959813171 shutdown_connections 2024-09-10T08:50:26.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.266+0000 7f7b27c4b740 1 -- 172.21.15.172:0/959813171 wait complete. 2024-09-10T08:50:26.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.266+0000 7f7b27c4b740 1 -- start start 2024-09-10T08:50:26.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b27c4b740 1 Processor -- start 2024-09-10T08:50:26.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b27c4b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b642fa30 0x55f2b6445b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b27c4b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433380 0x55f2b64460d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b27c4b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433ca0 0x55f2b64baff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b27c4b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7b201ade10 con 0x55f2b6433ca0 2024-09-10T08:50:26.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b27c4b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b201ad740 con 0x55f2b6433380 2024-09-10T08:50:26.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b27c4b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7b201aa950 con 0x55f2b642fa30 2024-09-10T08:50:26.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b27c4b740 10 client.0 _ll_register_callbacks cb 0x7f7b10009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:26.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b1f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433ca0 0x55f2b64baff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b1f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433ca0 0x55f2b64baff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38350/0 (socket says 172.21.15.172:38350) 2024-09-10T08:50:26.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b1f7fe640 1 -- 172.21.15.172:0/1068635837 learned_addr learned my addr 172.21.15.172:0/1068635837 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:26.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b1effd640 1 --2- 172.21.15.172:0/1068635837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b642fa30 0x55f2b6445b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b1e7fc640 1 --2- 172.21.15.172:0/1068635837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433380 0x55f2b64460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b1f7fe640 1 -- 172.21.15.172:0/1068635837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b642fa30 msgr2=0x55f2b6445b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:26.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b1f7fe640 1 --2- 172.21.15.172:0/1068635837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b642fa30 0x55f2b6445b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b1f7fe640 1 -- 172.21.15.172:0/1068635837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433380 msgr2=0x55f2b64460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:26.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b1f7fe640 1 --2- 172.21.15.172:0/1068635837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433380 0x55f2b64460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.267+0000 7f7b1f7fe640 1 -- 172.21.15.172:0/1068635837 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b64bb530 con 0x55f2b6433ca0 2024-09-10T08:50:26.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.268+0000 7f7b1f7fe640 1 --2- 172.21.15.172:0/1068635837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433ca0 0x55f2b64baff0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f7b1801db50 tx=0x7f7b18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75756b0c89ba17d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:26.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.268+0000 7f7afeffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:26.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.268+0000 7f7afeffd640 1 -- 172.21.15.172:0/1068635837 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1802b060 con 0x55f2b6433ca0 2024-09-10T08:50:26.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.268+0000 7f7afeffd640 1 -- 172.21.15.172:0/1068635837 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f7b1801bcb0 con 0x55f2b6433ca0 2024-09-10T08:50:26.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.268+0000 7f7b27c4b740 1 -- 172.21.15.172:0/1068635837 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b140c01c0 con 0x55f2b6433ca0 2024-09-10T08:50:26.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.268+0000 7f7afeffd640 1 -- 172.21.15.172:0/1068635837 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7b18032050 con 0x55f2b6433ca0 2024-09-10T08:50:26.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.268+0000 7f7b27c4b740 10 client.16668 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:26.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.268+0000 7f7b27c4b740 1 -- 172.21.15.172:0/1068635837 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f2b64bc700 con 0x55f2b6433ca0 2024-09-10T08:50:26.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.268+0000 7f7afeffd640 1 -- 172.21.15.172:0/1068635837 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b140c01c0 con 0x55f2b6433ca0 2024-09-10T08:50:26.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.269+0000 7f7afeffd640 1 -- 172.21.15.172:0/1068635837 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f7b18025070 con 0x55f2b6433ca0 2024-09-10T08:50:26.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.269+0000 7f7b27c4b740 10 client.16668 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:26.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.269+0000 7f7b27c4b740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:26.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.269+0000 7f7b27c4b740 20 client.16668 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:26.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.269+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/1068635837 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f2b6442a90 0x55f2b64bef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:26.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.269+0000 7f7b27c4b740 4 client.16668 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:50:26.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.269+0000 7f7b27c4b740 1 -- 172.21.15.172:0/1068635837 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f2b643ff00 con 0x55f2b6442a90 2024-09-10T08:50:26.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.269+0000 7f7b1effd640 1 --2- 172.21.15.172:0/1068635837 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f2b6442a90 0x55f2b64bef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:26.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.269+0000 7f7b1effd640 1 --2- 172.21.15.172:0/1068635837 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f2b6442a90 0x55f2b64bef50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=273794db473dc3d4 server_cookie=f792754c825fddb2 in_seq=0 out_seq=0 2024-09-10T08:50:26.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.269+0000 7f7afeffd640 10 client.16668 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:26.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.269+0000 7f7afeffd640 1 -- 172.21.15.172:0/1068635837 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f2b643ff00 con 0x55f2b6442a90 2024-09-10T08:50:26.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.269+0000 7f7afeffd640 10 client.16668 handle_command_reply: tid=0 2024-09-10T08:50:26.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.269+0000 7f7b27c4b740 1 client.16668 shutdown 2024-09-10T08:50:26.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.269+0000 7f7b27c4b740 1 -- 172.21.15.172:0/1068635837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433ca0 msgr2=0x55f2b64baff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:26.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.269+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/1068635837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433ca0 0x55f2b64baff0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f7b1801db50 tx=0x7f7b18004a40 comp rx=0 tx=0).stop 2024-09-10T08:50:26.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.270+0000 7f7b27c4b740 1 -- 172.21.15.172:0/1068635837 shutdown_connections 2024-09-10T08:50:26.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.270+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/1068635837 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f2b6442a90 0x55f2b64bef50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.270+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/1068635837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f2b6433ca0 0x55f2b64baff0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.270+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/1068635837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2b6433380 0x55f2b64460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.270+0000 7f7b27c4b740 1 --2- 172.21.15.172:0/1068635837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2b642fa30 0x55f2b6445b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.270+0000 7f7b27c4b740 1 -- 172.21.15.172:0/1068635837 >> 172.21.15.172:0/1068635837 conn(0x55f2b6454bf0 msgr2=0x55f2b643e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:26.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.270+0000 7f7afeffd640 0 client.16668 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:26.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.270+0000 7f7b27c4b740 1 -- 172.21.15.172:0/1068635837 shutdown_connections 2024-09-10T08:50:26.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.270+0000 7f7b27c4b740 1 -- 172.21.15.172:0/1068635837 wait complete. 2024-09-10T08:50:26.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.270+0000 7f7b27c4b740 20 client.16668 trim_cache size 0 max 16384 2024-09-10T08:50:26.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.270+0000 7f7b25872640 1 -- 172.21.15.172:0/1832653315 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7b080a2c00 msgr2=0x7f7b080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:26.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.270+0000 7f7b25872640 1 --2- 172.21.15.172:0/1832653315 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7b080a2c00 0x7f7b080a50c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7b18005fd0 tx=0x7f7b18005c80 comp rx=0 tx=0).stop 2024-09-10T08:50:26.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.270+0000 7f7b25872640 1 -- 172.21.15.172:0/1832653315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b20103740 msgr2=0x7f7b20198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:26.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.270+0000 7f7b25872640 1 --2- 172.21.15.172:0/1832653315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b20103740 0x7f7b20198a20 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f7b14031440 tx=0x7f7b14004270 comp rx=0 tx=0).stop 2024-09-10T08:50:26.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.270+0000 7f7b25872640 1 -- 172.21.15.172:0/1832653315 shutdown_connections 2024-09-10T08:50:26.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.271+0000 7f7b25872640 1 --2- 172.21.15.172:0/1832653315 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7b080a2c00 0x7f7b080a50c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.271+0000 7f7b25872640 1 --2- 172.21.15.172:0/1832653315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b20108290 0x7f7b2019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.271+0000 7f7b25872640 1 --2- 172.21.15.172:0/1832653315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b20103740 0x7f7b20198a20 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.271+0000 7f7b25872640 1 --2- 172.21.15.172:0/1832653315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b20102d90 0x7f7b201984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:26.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.271+0000 7f7b25872640 1 -- 172.21.15.172:0/1832653315 >> 172.21.15.172:0/1832653315 conn(0x7f7b200fc7d0 msgr2=0x7f7b20105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:26.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.271+0000 7f7b25872640 1 -- 172.21.15.172:0/1832653315 shutdown_connections 2024-09-10T08:50:26.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:26.271+0000 7f7b25872640 1 -- 172.21.15.172:0/1832653315 wait complete. 2024-09-10T08:50:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:26 smithi172 ceph-mon[30854]: pgmap v476: 129 pgs: 129 active+clean; 2.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 30 MiB/s wr, 36 op/s 2024-09-10T08:50:26.718 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:50:26.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:26 smithi174 ceph-mon[34068]: pgmap v476: 129 pgs: 129 active+clean; 2.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 30 MiB/s wr, 36 op/s 2024-09-10T08:50:26.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:26 smithi191 ceph-mon[35314]: pgmap v476: 129 pgs: 129 active+clean; 2.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 30 MiB/s wr, 36 op/s 2024-09-10T08:50:27.292 INFO:tasks.cephfs.filesystem:scrub status for tag:ad70bd8d-8317-43da-85d0-0b5cd613f1dd - {'path': '/', 'tag': 'ad70bd8d-8317-43da-85d0-0b5cd613f1dd', 'options': 'recursive,force'} 2024-09-10T08:50:27.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.757+0000 7f4fbc30b640 1 -- 172.21.15.172:0/1129603596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fb4100100 msgr2=0x7f4fb4100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:27.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.757+0000 7f4fbc30b640 1 --2- 172.21.15.172:0/1129603596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fb4100100 0x7f4fb4100560 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009900 tx=0x7f4fa402a410 comp rx=0 tx=0).stop 2024-09-10T08:50:27.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.757+0000 7f4fbc30b640 1 -- 172.21.15.172:0/1129603596 shutdown_connections 2024-09-10T08:50:27.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.757+0000 7f4fbc30b640 1 --2- 172.21.15.172:0/1129603596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fb4100aa0 0x7f4fb410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.757+0000 7f4fbc30b640 1 --2- 172.21.15.172:0/1129603596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fb4100100 0x7f4fb4100560 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.757+0000 7f4fbc30b640 1 --2- 172.21.15.172:0/1129603596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fb4106b80 0x7f4fb4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.757+0000 7f4fbc30b640 1 -- 172.21.15.172:0/1129603596 >> 172.21.15.172:0/1129603596 conn(0x7f4fb4077230 msgr2=0x7f4fb40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:27.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.757+0000 7f4fbc30b640 1 -- 172.21.15.172:0/1129603596 shutdown_connections 2024-09-10T08:50:27.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.757+0000 7f4fbc30b640 1 -- 172.21.15.172:0/1129603596 wait complete. 2024-09-10T08:50:27.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.758+0000 7f4fbc30b640 1 Processor -- start 2024-09-10T08:50:27.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.758+0000 7f4fbc30b640 1 -- start start 2024-09-10T08:50:27.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fbc30b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fb4100100 0x7f4fb4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fbc30b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fb4100aa0 0x7f4fb41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fbc30b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fb4106b80 0x7f4fb419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fbc30b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb4111f00 con 0x7f4fb4100100 2024-09-10T08:50:27.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fbc30b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb4111d80 con 0x7f4fb4100aa0 2024-09-10T08:50:27.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fbc30b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb4112080 con 0x7f4fb4106b80 2024-09-10T08:50:27.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fba0a7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fb4100100 0x7f4fb4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fba0a7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fb4100100 0x7f4fb4198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38368/0 (socket says 172.21.15.172:38368) 2024-09-10T08:50:27.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fba0a7640 1 -- 172.21.15.172:0/2808479659 learned_addr learned my addr 172.21.15.172:0/2808479659 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:27.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fba8a8640 1 --2- 172.21.15.172:0/2808479659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fb4106b80 0x7f4fb419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fb98a6640 1 --2- 172.21.15.172:0/2808479659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fb4100aa0 0x7f4fb41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fba0a7640 1 -- 172.21.15.172:0/2808479659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fb4106b80 msgr2=0x7f4fb419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:27.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fba0a7640 1 --2- 172.21.15.172:0/2808479659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fb4106b80 0x7f4fb419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fba0a7640 1 -- 172.21.15.172:0/2808479659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fb4100aa0 msgr2=0x7f4fb41989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:27.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fba0a7640 1 --2- 172.21.15.172:0/2808479659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fb4100aa0 0x7f4fb41989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fba0a7640 1 -- 172.21.15.172:0/2808479659 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb419d430 con 0x7f4fb4100100 2024-09-10T08:50:27.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fb98a6640 1 --2- 172.21.15.172:0/2808479659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fb4100aa0 0x7f4fb41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:27.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.759+0000 7f4fba0a7640 1 --2- 172.21.15.172:0/2808479659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fb4100100 0x7f4fb4198480 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000d950 tx=0x7f4fb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:27.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.760+0000 7f4fab7fe640 1 -- 172.21.15.172:0/2808479659 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0017030 con 0x7f4fb4100100 2024-09-10T08:50:27.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.760+0000 7f4fbc30b640 1 -- 172.21.15.172:0/2808479659 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb419d720 con 0x7f4fb4100100 2024-09-10T08:50:27.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.760+0000 7f4fab7fe640 1 -- 172.21.15.172:0/2808479659 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fb0004020 con 0x7f4fb4100100 2024-09-10T08:50:27.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.760+0000 7f4fab7fe640 1 -- 172.21.15.172:0/2808479659 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb00047a0 con 0x7f4fb4100100 2024-09-10T08:50:27.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.760+0000 7f4fbc30b640 1 -- 172.21.15.172:0/2808479659 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb41a5000 con 0x7f4fb4100100 2024-09-10T08:50:27.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.762+0000 7f4fab7fe640 1 -- 172.21.15.172:0/2808479659 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4fb00041d0 con 0x7f4fb4100100 2024-09-10T08:50:27.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.763+0000 7f4fbe6e4740 1 Processor -- start 2024-09-10T08:50:27.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.763+0000 7f4fab7fe640 1 --2- 172.21.15.172:0/2808479659 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4fa00a2b80 0x7f4fa00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.763+0000 7f4fbe6e4740 1 -- start start 2024-09-10T08:50:27.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.763+0000 7f4fba8a8640 1 --2- 172.21.15.172:0/2808479659 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4fa00a2b80 0x7f4fa00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.763+0000 7f4fab7fe640 1 -- 172.21.15.172:0/2808479659 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4fb00cedc0 con 0x7f4fb4100100 2024-09-10T08:50:27.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.763+0000 7f4fbe6e4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 0x562c4050ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.763+0000 7f4fbe6e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c4050f170 0x562c405194f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.763+0000 7f4fbe6e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c40519a30 0x562c4051be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.763+0000 7f4fbe6e4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562c40473160 con 0x562c4050e850 2024-09-10T08:50:27.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.763+0000 7f4fbe6e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562c4041a720 con 0x562c40519a30 2024-09-10T08:50:27.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.763+0000 7f4fbe6e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562c4046f120 con 0x562c4050f170 2024-09-10T08:50:27.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.763+0000 7f4fba0a7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 0x562c4050ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.764+0000 7f4fba0a7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 0x562c4050ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38374/0 (socket says 172.21.15.172:38374) 2024-09-10T08:50:27.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.764+0000 7f4fba0a7640 1 -- 172.21.15.172:0/3769242578 learned_addr learned my addr 172.21.15.172:0/3769242578 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:27.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.764+0000 7f4fb98a6640 1 --2- 172.21.15.172:0/3769242578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c4050f170 0x562c405194f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.764+0000 7f4fba8a8640 1 --2- 172.21.15.172:0/3769242578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c40519a30 0x562c4051be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.764+0000 7f4fb98a6640 1 -- 172.21.15.172:0/3769242578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c40519a30 msgr2=0x562c4051be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:27.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.764+0000 7f4fb98a6640 1 --2- 172.21.15.172:0/3769242578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c40519a30 0x562c4051be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.764+0000 7f4fb98a6640 1 -- 172.21.15.172:0/3769242578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 msgr2=0x562c4050ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:27.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.764+0000 7f4fb98a6640 1 --2- 172.21.15.172:0/3769242578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 0x562c4050ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.764+0000 7f4fb98a6640 1 -- 172.21.15.172:0/3769242578 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c401e4ba0 con 0x562c4050f170 2024-09-10T08:50:27.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.764+0000 7f4fba8a8640 1 --2- 172.21.15.172:0/2808479659 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4fa00a2b80 0x7f4fa00a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4fb41077c0 tx=0x7f4fac009f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:27.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.764+0000 7f4fb98a6640 1 --2- 172.21.15.172:0/3769242578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c4050f170 0x562c405194f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009230 tx=0x7f4fa4032cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95bb380b15ce394f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:27.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.764+0000 7f4fa8dea640 1 -- 172.21.15.172:0/3769242578 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa403f070 con 0x562c4050f170 2024-09-10T08:50:27.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.764+0000 7f4fa8dea640 1 -- 172.21.15.172:0/3769242578 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fa40316a0 con 0x562c4050f170 2024-09-10T08:50:27.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.764+0000 7f4fa8dea640 1 -- 172.21.15.172:0/3769242578 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4035070 con 0x562c4050f170 2024-09-10T08:50:27.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.765+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/3769242578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c4050f170 msgr2=0x562c405194f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:27.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.765+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/3769242578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c4050f170 0x562c405194f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009230 tx=0x7f4fa4032cb0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.765+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/3769242578 shutdown_connections 2024-09-10T08:50:27.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.765+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/3769242578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c40519a30 0x562c4051be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.765+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/3769242578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c4050f170 0x562c405194f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.765+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/3769242578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 0x562c4050ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.765+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/3769242578 >> 172.21.15.172:0/3769242578 conn(0x562c4050ade0 msgr2=0x562c4050b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:27.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.765+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/3769242578 shutdown_connections 2024-09-10T08:50:27.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.765+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/3769242578 wait complete. 2024-09-10T08:50:27.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.766+0000 7f4fbe6e4740 1 -- start start 2024-09-10T08:50:27.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.766+0000 7f4fbe6e4740 1 Processor -- start 2024-09-10T08:50:27.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.766+0000 7f4fbe6e4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 0x562c40516960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.766+0000 7f4fbe6e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c4050f170 0x562c40516ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.766+0000 7f4fbe6e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c40519a30 0x562c405173e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.766+0000 7f4fbe6e4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562c4051c600 con 0x562c4050e850 2024-09-10T08:50:27.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.766+0000 7f4fbe6e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562c403fb900 con 0x562c4050f170 2024-09-10T08:50:27.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.766+0000 7f4fba0a7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 0x562c40516960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.766+0000 7f4fbe6e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562c4051c780 con 0x562c40519a30 2024-09-10T08:50:27.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.766+0000 7f4fba0a7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 0x562c40516960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38380/0 (socket says 172.21.15.172:38380) 2024-09-10T08:50:27.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.766+0000 7f4fba0a7640 1 -- 172.21.15.172:0/674549734 learned_addr learned my addr 172.21.15.172:0/674549734 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:27.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.766+0000 7f4fbe6e4740 10 client.0 _ll_register_callbacks cb 0x562c403f9780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:27.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.766+0000 7f4fba8a8640 1 --2- 172.21.15.172:0/674549734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c40519a30 0x562c405173e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.767+0000 7f4fb98a6640 1 --2- 172.21.15.172:0/674549734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c4050f170 0x562c40516ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.767+0000 7f4fba0a7640 1 -- 172.21.15.172:0/674549734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c40519a30 msgr2=0x562c405173e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:27.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.767+0000 7f4fba0a7640 1 --2- 172.21.15.172:0/674549734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c40519a30 0x562c405173e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.767+0000 7f4fba0a7640 1 -- 172.21.15.172:0/674549734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c4050f170 msgr2=0x562c40516ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:27.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.767+0000 7f4fba0a7640 1 --2- 172.21.15.172:0/674549734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c4050f170 0x562c40516ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.767+0000 7f4fba0a7640 1 -- 172.21.15.172:0/674549734 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c4043bc50 con 0x562c4050e850 2024-09-10T08:50:27.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.767+0000 7f4fba0a7640 1 --2- 172.21.15.172:0/674549734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 0x562c40516960 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009610 tx=0x7f4fb009ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9e29a8350cb6ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:27.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.767+0000 7f4f9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:27.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.767+0000 7f4f9dffb640 1 -- 172.21.15.172:0/674549734 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb00a3f30 con 0x562c4050e850 2024-09-10T08:50:27.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.767+0000 7f4f9dffb640 1 -- 172.21.15.172:0/674549734 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4fb00bb0c0 con 0x562c4050e850 2024-09-10T08:50:27.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.767+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/674549734 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fa40316a0 con 0x562c4050e850 2024-09-10T08:50:27.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.767+0000 7f4f9dffb640 1 -- 172.21.15.172:0/674549734 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4fb00b4240 con 0x562c4050e850 2024-09-10T08:50:27.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.767+0000 7f4fbe6e4740 10 client.16680 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:27.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.767+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/674549734 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562c405282e0 con 0x562c4050e850 2024-09-10T08:50:27.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.767+0000 7f4f9dffb640 1 -- 172.21.15.172:0/674549734 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fb00bb2c0 con 0x562c4050e850 2024-09-10T08:50:27.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.768+0000 7f4f9dffb640 1 -- 172.21.15.172:0/674549734 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f4fb0097010 con 0x562c4050e850 2024-09-10T08:50:27.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.768+0000 7f4fbe6e4740 10 client.16680 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:27.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.768+0000 7f4fbe6e4740 10 client.16680 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:50:27.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.768+0000 7f4fbe6e4740 20 client.16680 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:27.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.768+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/674549734 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562c40529680 0x562c4052bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.768+0000 7f4fbe6e4740 4 client.16680 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:50:27.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.768+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/674549734 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4fa40316a0 con 0x562c40529680 2024-09-10T08:50:27.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.768+0000 7f4fb98a6640 1 --2- 172.21.15.172:0/674549734 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562c40529680 0x562c4052bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.768+0000 7f4f9dffb640 10 client.16680 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:27.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.768+0000 7f4fb98a6640 1 --2- 172.21.15.172:0/674549734 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562c40529680 0x562c4052bb20 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b0f2fa537c6ead server_cookie=5aa7a4327d7cb1af in_seq=0 out_seq=0 2024-09-10T08:50:27.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.769+0000 7f4f9dffb640 1 -- 172.21.15.172:0/674549734 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4fa40316a0 con 0x562c40529680 2024-09-10T08:50:27.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.769+0000 7f4f9dffb640 10 client.16680 handle_command_reply: tid=0 2024-09-10T08:50:27.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.769+0000 7f4fbe6e4740 1 client.16680 shutdown 2024-09-10T08:50:27.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.769+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/674549734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 msgr2=0x562c40516960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:27.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.769+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/674549734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 0x562c40516960 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009610 tx=0x7f4fb009ba80 comp rx=0 tx=0).stop 2024-09-10T08:50:27.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.770+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/674549734 shutdown_connections 2024-09-10T08:50:27.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.770+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/674549734 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562c40529680 0x562c4052bb20 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.770+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/674549734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c40519a30 0x562c405173e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.770+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/674549734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c4050f170 0x562c40516ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.770+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/674549734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 0x562c40516960 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.770+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/674549734 >> 172.21.15.172:0/674549734 conn(0x562c4050ade0 msgr2=0x562c40510b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:27.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.770+0000 7f4f9dffb640 0 client.16680 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:27.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.770+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/674549734 shutdown_connections 2024-09-10T08:50:27.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.770+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/674549734 wait complete. 2024-09-10T08:50:27.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.770+0000 7f4fbe6e4740 20 client.16680 trim_cache size 0 max 16384 2024-09-10T08:50:27.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fbe6e4740 1 Processor -- start 2024-09-10T08:50:27.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fbe6e4740 1 -- start start 2024-09-10T08:50:27.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fbe6e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c4050ade0 0x562c40510960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fbe6e4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 0x562c40515230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fbe6e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c4050f170 0x562c405313f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fbe6e4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb41aea60 con 0x562c4050e850 2024-09-10T08:50:27.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fbe6e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb41b0660 con 0x562c4050ade0 2024-09-10T08:50:27.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fbe6e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb41af130 con 0x562c4050f170 2024-09-10T08:50:27.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fb98a6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 0x562c40515230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fb98a6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 0x562c40515230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38388/0 (socket says 172.21.15.172:38388) 2024-09-10T08:50:27.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fb98a6640 1 -- 172.21.15.172:0/633928721 learned_addr learned my addr 172.21.15.172:0/633928721 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:27.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fba0a7640 1 --2- 172.21.15.172:0/633928721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c4050ade0 0x562c40510960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fba8a8640 1 --2- 172.21.15.172:0/633928721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c4050f170 0x562c405313f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fb98a6640 1 -- 172.21.15.172:0/633928721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c4050f170 msgr2=0x562c405313f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:27.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fb98a6640 1 --2- 172.21.15.172:0/633928721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c4050f170 0x562c405313f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fb98a6640 1 -- 172.21.15.172:0/633928721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c4050ade0 msgr2=0x562c40510960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:27.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fb98a6640 1 --2- 172.21.15.172:0/633928721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c4050ade0 0x562c40510960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.789+0000 7f4fb98a6640 1 -- 172.21.15.172:0/633928721 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c405063f0 con 0x562c4050e850 2024-09-10T08:50:27.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.790+0000 7f4fb98a6640 1 --2- 172.21.15.172:0/633928721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 0x562c40515230 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009230 tx=0x7f4fa4031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19cfeb88dbdc464 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:27.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.790+0000 7f4f9ffff640 1 -- 172.21.15.172:0/633928721 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa404b070 con 0x562c4050e850 2024-09-10T08:50:27.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.790+0000 7f4f9ffff640 1 -- 172.21.15.172:0/633928721 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fa40054f0 con 0x562c4050e850 2024-09-10T08:50:27.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.790+0000 7f4f9ffff640 1 -- 172.21.15.172:0/633928721 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa403f040 con 0x562c4050e850 2024-09-10T08:50:27.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.791+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/633928721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 msgr2=0x562c40515230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:27.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.791+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/633928721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 0x562c40515230 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009230 tx=0x7f4fa4031750 comp rx=0 tx=0).stop 2024-09-10T08:50:27.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.791+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/633928721 shutdown_connections 2024-09-10T08:50:27.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.791+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/633928721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c4050f170 0x562c405313f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.791+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/633928721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050e850 0x562c40515230 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.791+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/633928721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c4050ade0 0x562c40510960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.791+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/633928721 >> 172.21.15.172:0/633928721 conn(0x562c40512350 msgr2=0x562c40517140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:27.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.791+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/633928721 shutdown_connections 2024-09-10T08:50:27.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.791+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/633928721 wait complete. 2024-09-10T08:50:27.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.791+0000 7f4fbe6e4740 1 -- start start 2024-09-10T08:50:27.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.791+0000 7f4fbe6e4740 1 Processor -- start 2024-09-10T08:50:27.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fbe6e4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050ade0 0x562c40521d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fbe6e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c4050e850 0x562c40522260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fbe6e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c4050f170 0x562c4051e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fbe6e4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb41aff00 con 0x562c4050ade0 2024-09-10T08:50:27.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fbe6e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb41af830 con 0x562c4050f170 2024-09-10T08:50:27.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fbe6e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb41aca40 con 0x562c4050e850 2024-09-10T08:50:27.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fbe6e4740 10 client.0 _ll_register_callbacks cb 0x7f4fb009ac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:27.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fba0a7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050ade0 0x562c40521d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fba0a7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050ade0 0x562c40521d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38392/0 (socket says 172.21.15.172:38392) 2024-09-10T08:50:27.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fba0a7640 1 -- 172.21.15.172:0/4131408737 learned_addr learned my addr 172.21.15.172:0/4131408737 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:27.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fb98a6640 1 --2- 172.21.15.172:0/4131408737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c4050e850 0x562c40522260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fba8a8640 1 --2- 172.21.15.172:0/4131408737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c4050f170 0x562c4051e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fba0a7640 1 -- 172.21.15.172:0/4131408737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c4050e850 msgr2=0x562c40522260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:27.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fba0a7640 1 --2- 172.21.15.172:0/4131408737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c4050e850 0x562c40522260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fba0a7640 1 -- 172.21.15.172:0/4131408737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c4050f170 msgr2=0x562c4051e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:27.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fba0a7640 1 --2- 172.21.15.172:0/4131408737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c4050f170 0x562c4051e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fba0a7640 1 -- 172.21.15.172:0/4131408737 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c4051e960 con 0x562c4050ade0 2024-09-10T08:50:27.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fba8a8640 1 --2- 172.21.15.172:0/4131408737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c4050f170 0x562c4051e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:27.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4fba0a7640 1 --2- 172.21.15.172:0/4131408737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050ade0 0x562c40521d20 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0097480 tx=0x7f4fb00bb2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b599ac9da1c3457e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:27.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.792+0000 7f4f9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:27.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.793+0000 7f4f9dffb640 1 -- 172.21.15.172:0/4131408737 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0097560 con 0x562c4050ade0 2024-09-10T08:50:27.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.793+0000 7f4f9dffb640 1 -- 172.21.15.172:0/4131408737 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4fb009f120 con 0x562c4050ade0 2024-09-10T08:50:27.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.793+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/4131408737 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fa40054f0 con 0x562c4050ade0 2024-09-10T08:50:27.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.793+0000 7f4f9dffb640 1 -- 172.21.15.172:0/4131408737 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4fb0065050 con 0x562c4050ade0 2024-09-10T08:50:27.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.793+0000 7f4fbe6e4740 10 client.16686 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:27.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.793+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/4131408737 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562c40594fb0 con 0x562c4050ade0 2024-09-10T08:50:27.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.793+0000 7f4f9dffb640 1 -- 172.21.15.172:0/4131408737 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fa40054f0 con 0x562c4050ade0 2024-09-10T08:50:27.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.793+0000 7f4f9dffb640 1 -- 172.21.15.172:0/4131408737 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f4fb0066240 con 0x562c4050ade0 2024-09-10T08:50:27.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.793+0000 7f4fbe6e4740 10 client.16686 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:27.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.793+0000 7f4fbe6e4740 10 client.16686 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:50:27.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.794+0000 7f4fbe6e4740 20 client.16686 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:27.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.794+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/4131408737 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562c40519a30 0x562c405976a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:27.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.794+0000 7f4fbe6e4740 4 client.16686 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T08:50:27.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.794+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/4131408737 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x562c40480890 con 0x562c40519a30 2024-09-10T08:50:27.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.794+0000 7f4fb98a6640 1 --2- 172.21.15.172:0/4131408737 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562c40519a30 0x562c405976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:27.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.794+0000 7f4f9dffb640 10 client.16686 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:27.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.794+0000 7f4fb98a6640 1 --2- 172.21.15.172:0/4131408737 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562c40519a30 0x562c405976a0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebd89e22058cdfe2 server_cookie=d074c7b983f90e6f in_seq=0 out_seq=0 2024-09-10T08:50:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.899+0000 7f4f9dffb640 1 -- 172.21.15.172:0/4131408737 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562c40480890 con 0x562c40519a30 2024-09-10T08:50:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.899+0000 7f4f9dffb640 10 client.16686 handle_command_reply: tid=0 2024-09-10T08:50:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.899+0000 7f4fbe6e4740 1 client.16686 shutdown 2024-09-10T08:50:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.900+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/4131408737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050ade0 msgr2=0x562c40521d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.900+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/4131408737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050ade0 0x562c40521d20 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0097480 tx=0x7f4fb00bb2e0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.900+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/4131408737 shutdown_connections 2024-09-10T08:50:27.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.900+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/4131408737 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562c40519a30 0x562c405976a0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.900+0000 7f4f9dffb640 0 client.16686 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:27.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.900+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/4131408737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c4050f170 0x562c4051e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.900+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/4131408737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562c4050e850 0x562c40522260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.900+0000 7f4fbe6e4740 1 --2- 172.21.15.172:0/4131408737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562c4050ade0 0x562c40521d20 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.900+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/4131408737 >> 172.21.15.172:0/4131408737 conn(0x562c40512350 msgr2=0x562c40517140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:27.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.900+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/4131408737 shutdown_connections 2024-09-10T08:50:27.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.900+0000 7f4fbe6e4740 1 -- 172.21.15.172:0/4131408737 wait complete. 2024-09-10T08:50:27.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.900+0000 7f4fbe6e4740 20 client.16686 trim_cache size 0 max 16384 2024-09-10T08:50:27.904 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:50:27.904 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 33, 2024-09-10T08:50:27.904 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:50:27.904 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 10, 2024-09-10T08:50:27.905 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:50:27.905 INFO:teuthology.orchestra.run.smithi172.stdout: "5e6021f1": { 2024-09-10T08:50:27.905 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 10, 2024-09-10T08:50:27.905 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:50:27.905 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:50:27.905 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:50:27.905 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:50:27.905 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:50:27.905 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:50:27.905 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 142.8, 2024-09-10T08:50:27.906 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:50:27.906 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:50:27.906 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:50:27.906 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:50:27.906 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:50:27.906 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:50:27.906 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:50:27.906 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:50:27.906 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:50:27.906 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:50:27.906 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:50:27.906 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:50:27.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.901+0000 7f4fbc30b640 1 -- 172.21.15.172:0/2808479659 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4fa00a2b80 msgr2=0x7f4fa00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:27.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.901+0000 7f4fbc30b640 1 --2- 172.21.15.172:0/2808479659 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4fa00a2b80 0x7f4fa00a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4fb41077c0 tx=0x7f4fac009f90 comp rx=0 tx=0).stop 2024-09-10T08:50:27.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.901+0000 7f4fbc30b640 1 -- 172.21.15.172:0/2808479659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fb4100100 msgr2=0x7f4fb4198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:27.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.901+0000 7f4fbc30b640 1 --2- 172.21.15.172:0/2808479659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fb4100100 0x7f4fb4198480 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000d950 tx=0x7f4fb000de10 comp rx=0 tx=0).stop 2024-09-10T08:50:27.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.901+0000 7f4fbc30b640 1 -- 172.21.15.172:0/2808479659 shutdown_connections 2024-09-10T08:50:27.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.901+0000 7f4fbc30b640 1 --2- 172.21.15.172:0/2808479659 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4fa00a2b80 0x7f4fa00a5040 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.901+0000 7f4fbc30b640 1 --2- 172.21.15.172:0/2808479659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fb4106b80 0x7f4fb419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.901+0000 7f4fbc30b640 1 --2- 172.21.15.172:0/2808479659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fb4100aa0 0x7f4fb41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.901+0000 7f4fbc30b640 1 --2- 172.21.15.172:0/2808479659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fb4100100 0x7f4fb4198480 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:27.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.901+0000 7f4fbc30b640 1 -- 172.21.15.172:0/2808479659 >> 172.21.15.172:0/2808479659 conn(0x7f4fb4077230 msgr2=0x7f4fb4104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:27.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.901+0000 7f4fbc30b640 1 -- 172.21.15.172:0/2808479659 shutdown_connections 2024-09-10T08:50:27.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:27.901+0000 7f4fbc30b640 1 -- 172.21.15.172:0/2808479659 wait complete. 2024-09-10T08:50:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.530+0000 7fa42c352640 1 -- 172.21.15.172:0/2534831224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4241013c0 msgr2=0x7fa424101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.530+0000 7fa42c352640 1 --2- 172.21.15.172:0/2534831224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4241013c0 0x7fa424101840 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa418009900 tx=0x7fa41802a410 comp rx=0 tx=0).stop 2024-09-10T08:50:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.530+0000 7fa42c352640 1 -- 172.21.15.172:0/2534831224 shutdown_connections 2024-09-10T08:50:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.530+0000 7fa42c352640 1 --2- 172.21.15.172:0/2534831224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa424101d80 0x7fa424108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.530+0000 7fa42c352640 1 --2- 172.21.15.172:0/2534831224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4241013c0 0x7fa424101840 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.530+0000 7fa42c352640 1 --2- 172.21.15.172:0/2534831224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4241001d0 0x7fa4241005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.530+0000 7fa42c352640 1 -- 172.21.15.172:0/2534831224 >> 172.21.15.172:0/2534831224 conn(0x7fa4240fb930 msgr2=0x7fa4240fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.530+0000 7fa42c352640 1 -- 172.21.15.172:0/2534831224 shutdown_connections 2024-09-10T08:50:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.530+0000 7fa42c352640 1 -- 172.21.15.172:0/2534831224 wait complete. 2024-09-10T08:50:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.530+0000 7fa42c352640 1 Processor -- start 2024-09-10T08:50:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.530+0000 7fa42c352640 1 -- start start 2024-09-10T08:50:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.530+0000 7fa42c352640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4241001d0 0x7fa424196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa42c352640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4241013c0 0x7fa424197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa42c352640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa424101d80 0x7fa42419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa42c352640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa42410bd40 con 0x7fa4241001d0 2024-09-10T08:50:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa42c352640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa42410bbc0 con 0x7fa424101d80 2024-09-10T08:50:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa42c352640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa42410bec0 con 0x7fa4241013c0 2024-09-10T08:50:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa42a0ee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4241001d0 0x7fa424196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa42a0ee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4241001d0 0x7fa424196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38412/0 (socket says 172.21.15.172:38412) 2024-09-10T08:50:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa42a0ee640 1 -- 172.21.15.172:0/3025572313 learned_addr learned my addr 172.21.15.172:0/3025572313 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa42a8ef640 1 --2- 172.21.15.172:0/3025572313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa424101d80 0x7fa42419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa4298ed640 1 --2- 172.21.15.172:0/3025572313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4241013c0 0x7fa424197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa42a0ee640 1 -- 172.21.15.172:0/3025572313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4241013c0 msgr2=0x7fa424197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa42a0ee640 1 --2- 172.21.15.172:0/3025572313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4241013c0 0x7fa424197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa42a0ee640 1 -- 172.21.15.172:0/3025572313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa424101d80 msgr2=0x7fa42419e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa42a0ee640 1 --2- 172.21.15.172:0/3025572313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa424101d80 0x7fa42419e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa42a0ee640 1 -- 172.21.15.172:0/3025572313 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42419e830 con 0x7fa4241001d0 2024-09-10T08:50:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa42a0ee640 1 --2- 172.21.15.172:0/3025572313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4241001d0 0x7fa424196c20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa42000d950 tx=0x7fa42000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa4177fe640 1 -- 172.21.15.172:0/3025572313 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420017070 con 0x7fa4241001d0 2024-09-10T08:50:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa4177fe640 1 -- 172.21.15.172:0/3025572313 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa420009c30 con 0x7fa4241001d0 2024-09-10T08:50:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa42c352640 1 -- 172.21.15.172:0/3025572313 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42419eb20 con 0x7fa4241001d0 2024-09-10T08:50:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa4177fe640 1 -- 172.21.15.172:0/3025572313 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42000f620 con 0x7fa4241001d0 2024-09-10T08:50:28.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.531+0000 7fa42c352640 1 -- 172.21.15.172:0/3025572313 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa42419f020 con 0x7fa4241001d0 2024-09-10T08:50:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.534+0000 7fa4177fe640 1 -- 172.21.15.172:0/3025572313 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa42001a070 con 0x7fa4241001d0 2024-09-10T08:50:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.534+0000 7fa42f1ca740 1 Processor -- start 2024-09-10T08:50:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.534+0000 7fa42f1ca740 1 -- start start 2024-09-10T08:50:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.534+0000 7fa42f1ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860c5bb0 0x555c860c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.534+0000 7fa42f1ca740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c64f0 0x555c860cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.534+0000 7fa42f1ca740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860cb580 0x555c860cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.534+0000 7fa42f1ca740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555c860272f0 con 0x555c860c64f0 2024-09-10T08:50:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.534+0000 7fa42f1ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555c85fcf270 con 0x555c860c5bb0 2024-09-10T08:50:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.534+0000 7fa42f1ca740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555c860232b0 con 0x555c860cb580 2024-09-10T08:50:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.534+0000 7fa42a0ee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c64f0 0x555c860cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.534+0000 7fa42a0ee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c64f0 0x555c860cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38422/0 (socket says 172.21.15.172:38422) 2024-09-10T08:50:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.534+0000 7fa42a0ee640 1 -- 172.21.15.172:0/3980558157 learned_addr learned my addr 172.21.15.172:0/3980558157 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.534+0000 7fa4298ed640 1 --2- 172.21.15.172:0/3980558157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860cb580 0x555c860cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.535+0000 7fa42a8ef640 1 --2- 172.21.15.172:0/3980558157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860c5bb0 0x555c860c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.535+0000 7fa4177fe640 1 --2- 172.21.15.172:0/3025572313 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4100a2c30 0x7fa4100a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.535+0000 7fa42a0ee640 1 -- 172.21.15.172:0/3980558157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860cb580 msgr2=0x555c860cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.535+0000 7fa42a0ee640 1 --2- 172.21.15.172:0/3980558157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860cb580 0x555c860cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.535+0000 7fa42a0ee640 1 -- 172.21.15.172:0/3980558157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860c5bb0 msgr2=0x555c860c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.535+0000 7fa42a0ee640 1 --2- 172.21.15.172:0/3980558157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860c5bb0 0x555c860c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.535+0000 7fa42a0ee640 1 -- 172.21.15.172:0/3980558157 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c85fce340 con 0x555c860c64f0 2024-09-10T08:50:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.535+0000 7fa42a8ef640 1 --2- 172.21.15.172:0/3980558157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860c5bb0 0x555c860c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.535+0000 7fa42a8ef640 1 --2- 172.21.15.172:0/3025572313 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4100a2c30 0x7fa4100a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.535+0000 7fa4177fe640 1 -- 172.21.15.172:0/3025572313 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa4200cee80 con 0x7fa4241001d0 2024-09-10T08:50:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.535+0000 7fa42a0ee640 1 --2- 172.21.15.172:0/3980558157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c64f0 0x555c860cafb0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa42004b430 tx=0x7fa420023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f6025330508297 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.535+0000 7fa414bfe640 1 -- 172.21.15.172:0/3980558157 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420021070 con 0x555c860c64f0 2024-09-10T08:50:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.535+0000 7fa414bfe640 1 -- 172.21.15.172:0/3980558157 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa420009de0 con 0x555c860c64f0 2024-09-10T08:50:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.535+0000 7fa414bfe640 1 -- 172.21.15.172:0/3980558157 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420095e60 con 0x555c860c64f0 2024-09-10T08:50:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.536+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3980558157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c64f0 msgr2=0x555c860cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.536+0000 7fa42a8ef640 1 --2- 172.21.15.172:0/3025572313 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4100a2c30 0x7fa4100a50f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0098b0 tx=0x7fa41c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.536+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3980558157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c64f0 0x555c860cafb0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa42004b430 tx=0x7fa420023000 comp rx=0 tx=0).stop 2024-09-10T08:50:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.536+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3980558157 shutdown_connections 2024-09-10T08:50:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.536+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3980558157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860cb580 0x555c860cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.536+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3980558157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c64f0 0x555c860cafb0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.536+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3980558157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860c5bb0 0x555c860c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.536+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3980558157 >> 172.21.15.172:0/3980558157 conn(0x555c860bf010 msgr2=0x555c860bf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.536+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3980558157 shutdown_connections 2024-09-10T08:50:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.536+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3980558157 wait complete. 2024-09-10T08:50:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.536+0000 7fa42f1ca740 1 -- start start 2024-09-10T08:50:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.536+0000 7fa42f1ca740 1 Processor -- start 2024-09-10T08:50:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42f1ca740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c5bb0 0x555c860cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42f1ca740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860c64f0 0x555c860d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42f1ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860cb580 0x555c860d7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42f1ca740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555c860ce4f0 con 0x555c860c5bb0 2024-09-10T08:50:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42f1ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555c85fb0290 con 0x555c860cb580 2024-09-10T08:50:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42f1ca740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555c860ce670 con 0x555c860c64f0 2024-09-10T08:50:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42f1ca740 10 client.0 _ll_register_callbacks cb 0x555c85fe1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42a0ee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c5bb0 0x555c860cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42a0ee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c5bb0 0x555c860cfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38432/0 (socket says 172.21.15.172:38432) 2024-09-10T08:50:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42a0ee640 1 -- 172.21.15.172:0/3631845993 learned_addr learned my addr 172.21.15.172:0/3631845993 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa4298ed640 1 --2- 172.21.15.172:0/3631845993 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860c64f0 0x555c860d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42a0ee640 1 -- 172.21.15.172:0/3631845993 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860c64f0 msgr2=0x555c860d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42a0ee640 1 --2- 172.21.15.172:0/3631845993 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860c64f0 0x555c860d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42a0ee640 1 -- 172.21.15.172:0/3631845993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860cb580 msgr2=0x555c860d7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42a0ee640 1 --2- 172.21.15.172:0/3631845993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860cb580 0x555c860d7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42a0ee640 1 -- 172.21.15.172:0/3631845993 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c85d3a210 con 0x555c860c5bb0 2024-09-10T08:50:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42a0ee640 1 --2- 172.21.15.172:0/3631845993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c5bb0 0x555c860cfd10 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa4200a3ac0 tx=0x7fa4200d2410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90094a8e63d8d7fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa40dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa40dffb640 1 -- 172.21.15.172:0/3631845993 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420021070 con 0x555c860c5bb0 2024-09-10T08:50:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa40dffb640 1 -- 172.21.15.172:0/3631845993 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa4200af560 con 0x555c860c5bb0 2024-09-10T08:50:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3631845993 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa420009de0 con 0x555c860c5bb0 2024-09-10T08:50:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa40dffb640 1 -- 172.21.15.172:0/3631845993 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa420095e60 con 0x555c860c5bb0 2024-09-10T08:50:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42f1ca740 10 client.16698 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.537+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3631845993 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555c85e06620 con 0x555c860c5bb0 2024-09-10T08:50:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.538+0000 7fa40dffb640 1 -- 172.21.15.172:0/3631845993 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa420009de0 con 0x555c860c5bb0 2024-09-10T08:50:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.538+0000 7fa40dffb640 1 -- 172.21.15.172:0/3631845993 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fa420097960 con 0x555c860c5bb0 2024-09-10T08:50:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.538+0000 7fa42f1ca740 10 client.16698 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.538+0000 7fa42f1ca740 10 client.16698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.538+0000 7fa42f1ca740 20 client.16698 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.538+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3631845993 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555c860db5e0 0x555c860dda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.538+0000 7fa42f1ca740 4 client.16698 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:50:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.538+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3631845993 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c86034a20 con 0x555c860db5e0 2024-09-10T08:50:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.538+0000 7fa4298ed640 1 --2- 172.21.15.172:0/3631845993 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555c860db5e0 0x555c860dda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.539+0000 7fa40dffb640 10 client.16698 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.539+0000 7fa4298ed640 1 --2- 172.21.15.172:0/3631845993 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555c860db5e0 0x555c860dda30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ba5347fe7a00131 server_cookie=ffe184b6de1dda81 in_seq=0 out_seq=0 2024-09-10T08:50:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.539+0000 7fa40dffb640 1 -- 172.21.15.172:0/3631845993 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c86034a20 con 0x555c860db5e0 2024-09-10T08:50:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.539+0000 7fa40dffb640 10 client.16698 handle_command_reply: tid=0 2024-09-10T08:50:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.539+0000 7fa42f1ca740 1 client.16698 shutdown 2024-09-10T08:50:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.539+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3631845993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c5bb0 msgr2=0x555c860cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.539+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3631845993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c5bb0 0x555c860cfd10 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa4200a3ac0 tx=0x7fa4200d2410 comp rx=0 tx=0).stop 2024-09-10T08:50:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.539+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3631845993 shutdown_connections 2024-09-10T08:50:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.539+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3631845993 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555c860db5e0 0x555c860dda30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.539+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3631845993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860cb580 0x555c860d7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.539+0000 7fa40dffb640 0 client.16698 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.539+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3631845993 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860c64f0 0x555c860d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.539+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3631845993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c5bb0 0x555c860cfd10 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.539+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3631845993 >> 172.21.15.172:0/3631845993 conn(0x555c860bf010 msgr2=0x555c860c0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.540+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3631845993 shutdown_connections 2024-09-10T08:50:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.540+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3631845993 wait complete. 2024-09-10T08:50:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.540+0000 7fa42f1ca740 20 client.16698 trim_cache size 0 max 16384 2024-09-10T08:50:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.554+0000 7fa42f1ca740 1 Processor -- start 2024-09-10T08:50:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.554+0000 7fa42f1ca740 1 -- start start 2024-09-10T08:50:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.554+0000 7fa42f1ca740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860c5bb0 0x555c860d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.554+0000 7fa42f1ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860c64f0 0x555c860d7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.554+0000 7fa42f1ca740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860cb580 0x555c860e22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.554+0000 7fa42f1ca740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241a8340 con 0x555c860cb580 2024-09-10T08:50:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.554+0000 7fa42f1ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241a9f40 con 0x555c860c64f0 2024-09-10T08:50:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.554+0000 7fa42f1ca740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241a8a10 con 0x555c860c5bb0 2024-09-10T08:50:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.554+0000 7fa42a8ef640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860cb580 0x555c860e22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.555+0000 7fa42a8ef640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860cb580 0x555c860e22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38440/0 (socket says 172.21.15.172:38440) 2024-09-10T08:50:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.555+0000 7fa42a8ef640 1 -- 172.21.15.172:0/3862817917 learned_addr learned my addr 172.21.15.172:0/3862817917 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.555+0000 7fa4298ed640 1 --2- 172.21.15.172:0/3862817917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860c64f0 0x555c860d7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.555+0000 7fa42a0ee640 1 --2- 172.21.15.172:0/3862817917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860c5bb0 0x555c860d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.555+0000 7fa42a8ef640 1 -- 172.21.15.172:0/3862817917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860c5bb0 msgr2=0x555c860d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.555+0000 7fa42a8ef640 1 --2- 172.21.15.172:0/3862817917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860c5bb0 0x555c860d0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.555+0000 7fa42a8ef640 1 -- 172.21.15.172:0/3862817917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860c64f0 msgr2=0x555c860d7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.555+0000 7fa42a8ef640 1 --2- 172.21.15.172:0/3862817917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860c64f0 0x555c860d7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.555+0000 7fa42a8ef640 1 -- 172.21.15.172:0/3862817917 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c860c0e90 con 0x555c860cb580 2024-09-10T08:50:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.555+0000 7fa42a0ee640 1 --2- 172.21.15.172:0/3862817917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860c5bb0 0x555c860d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:50:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.555+0000 7fa42a8ef640 1 --2- 172.21.15.172:0/3862817917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860cb580 0x555c860e22a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa41c009830 tx=0x7fa41c015070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c659fd01629fdae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.555+0000 7fa40ffff640 1 -- 172.21.15.172:0/3862817917 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c01e8e0 con 0x555c860cb580 2024-09-10T08:50:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.555+0000 7fa40ffff640 1 -- 172.21.15.172:0/3862817917 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa41c023070 con 0x555c860cb580 2024-09-10T08:50:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.555+0000 7fa40ffff640 1 -- 172.21.15.172:0/3862817917 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c028e60 con 0x555c860cb580 2024-09-10T08:50:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.556+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3862817917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860cb580 msgr2=0x555c860e22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.556+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3862817917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860cb580 0x555c860e22a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa41c009830 tx=0x7fa41c015070 comp rx=0 tx=0).stop 2024-09-10T08:50:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.556+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3862817917 shutdown_connections 2024-09-10T08:50:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.556+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3862817917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860cb580 0x555c860e22a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.556+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3862817917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860c64f0 0x555c860d7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.556+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3862817917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860c5bb0 0x555c860d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.556+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3862817917 >> 172.21.15.172:0/3862817917 conn(0x555c860bf010 msgr2=0x555c860c0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.556+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3862817917 shutdown_connections 2024-09-10T08:50:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.556+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3862817917 wait complete. 2024-09-10T08:50:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.556+0000 7fa42f1ca740 1 -- start start 2024-09-10T08:50:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.556+0000 7fa42f1ca740 1 Processor -- start 2024-09-10T08:50:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42f1ca740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c5bb0 0x555c860d50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42f1ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860c64f0 0x555c860d5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42f1ca740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860cb580 0x555c860d6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42f1ca740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241a97e0 con 0x555c860c5bb0 2024-09-10T08:50:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42f1ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241a9110 con 0x555c860c64f0 2024-09-10T08:50:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42f1ca740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241a6320 con 0x555c860cb580 2024-09-10T08:50:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42f1ca740 10 client.0 _ll_register_callbacks cb 0x7fa418009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42a0ee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c5bb0 0x555c860d50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42a0ee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c5bb0 0x555c860d50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38446/0 (socket says 172.21.15.172:38446) 2024-09-10T08:50:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42a0ee640 1 -- 172.21.15.172:0/3673677818 learned_addr learned my addr 172.21.15.172:0/3673677818 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa4298ed640 1 --2- 172.21.15.172:0/3673677818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860c64f0 0x555c860d5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42a8ef640 1 --2- 172.21.15.172:0/3673677818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860cb580 0x555c860d6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42a0ee640 1 -- 172.21.15.172:0/3673677818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860cb580 msgr2=0x555c860d6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42a0ee640 1 --2- 172.21.15.172:0/3673677818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860cb580 0x555c860d6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42a0ee640 1 -- 172.21.15.172:0/3673677818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860c64f0 msgr2=0x555c860d5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42a0ee640 1 --2- 172.21.15.172:0/3673677818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860c64f0 0x555c860d5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42a0ee640 1 -- 172.21.15.172:0/3673677818 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c860d6c40 con 0x555c860c5bb0 2024-09-10T08:50:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42a8ef640 1 --2- 172.21.15.172:0/3673677818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860cb580 0x555c860d6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:50:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42a0ee640 1 --2- 172.21.15.172:0/3673677818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c5bb0 0x555c860d50e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa420097390 tx=0x7fa4200b43c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3918d1be360afa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa40dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa40dffb640 1 -- 172.21.15.172:0/3673677818 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420021070 con 0x555c860c5bb0 2024-09-10T08:50:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa40dffb640 1 -- 172.21.15.172:0/3673677818 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa4200d2800 con 0x555c860c5bb0 2024-09-10T08:50:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3673677818 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa41c023070 con 0x555c860c5bb0 2024-09-10T08:50:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa40dffb640 1 -- 172.21.15.172:0/3673677818 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa42009aec0 con 0x555c860c5bb0 2024-09-10T08:50:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42f1ca740 10 client.16704 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3673677818 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555c86146980 con 0x555c860c5bb0 2024-09-10T08:50:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.557+0000 7fa40dffb640 1 -- 172.21.15.172:0/3673677818 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa41c023070 con 0x555c860c5bb0 2024-09-10T08:50:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.558+0000 7fa40dffb640 1 -- 172.21.15.172:0/3673677818 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fa4200a3b60 con 0x555c860c5bb0 2024-09-10T08:50:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.558+0000 7fa42f1ca740 10 client.16704 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.558+0000 7fa42f1ca740 10 client.16704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.558+0000 7fa42f1ca740 20 client.16704 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.558+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3673677818 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555c861471e0 0x555c861495d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.558+0000 7fa42f1ca740 4 client.16704 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:50:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.558+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3673677818 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c8602ed70 con 0x555c861471e0 2024-09-10T08:50:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.558+0000 7fa4298ed640 1 --2- 172.21.15.172:0/3673677818 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555c861471e0 0x555c861495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.558+0000 7fa40dffb640 10 client.16704 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.558+0000 7fa4298ed640 1 --2- 172.21.15.172:0/3673677818 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555c861471e0 0x555c861495d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c372f5d6d30b2c79 server_cookie=bf6ffb4623f68770 in_seq=0 out_seq=0 2024-09-10T08:50:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.564+0000 7fa40dffb640 1 -- 172.21.15.172:0/3673677818 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c8602ed70 con 0x555c861471e0 2024-09-10T08:50:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.564+0000 7fa40dffb640 10 client.16704 handle_command_reply: tid=0 2024-09-10T08:50:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.564+0000 7fa42f1ca740 1 client.16704 shutdown 2024-09-10T08:50:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.564+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3673677818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c5bb0 msgr2=0x555c860d50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.564+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3673677818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c5bb0 0x555c860d50e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa420097390 tx=0x7fa4200b43c0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.564+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3673677818 shutdown_connections 2024-09-10T08:50:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.564+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3673677818 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555c861471e0 0x555c861495d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.564+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3673677818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555c860cb580 0x555c860d6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.564+0000 7fa40dffb640 0 client.16704 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.564+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3673677818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c860c64f0 0x555c860d5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.564+0000 7fa42f1ca740 1 --2- 172.21.15.172:0/3673677818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c860c5bb0 0x555c860d50e0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.564+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3673677818 >> 172.21.15.172:0/3673677818 conn(0x555c860bf010 msgr2=0x555c860c0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.564+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3673677818 shutdown_connections 2024-09-10T08:50:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.564+0000 7fa42f1ca740 1 -- 172.21.15.172:0/3673677818 wait complete. 2024-09-10T08:50:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.565+0000 7fa42f1ca740 20 client.16704 trim_cache size 0 max 16384 2024-09-10T08:50:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:50:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:50:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:50:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:50:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.565+0000 7fa42c352640 1 -- 172.21.15.172:0/3025572313 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4100a2c30 msgr2=0x7fa4100a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.565+0000 7fa42c352640 1 --2- 172.21.15.172:0/3025572313 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4100a2c30 0x7fa4100a50f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0098b0 tx=0x7fa41c004a40 comp rx=0 tx=0).stop 2024-09-10T08:50:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.565+0000 7fa42c352640 1 -- 172.21.15.172:0/3025572313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4241001d0 msgr2=0x7fa424196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.565+0000 7fa42c352640 1 --2- 172.21.15.172:0/3025572313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4241001d0 0x7fa424196c20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa42000d950 tx=0x7fa42000de10 comp rx=0 tx=0).stop 2024-09-10T08:50:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.565+0000 7fa42c352640 1 -- 172.21.15.172:0/3025572313 shutdown_connections 2024-09-10T08:50:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.565+0000 7fa42c352640 1 --2- 172.21.15.172:0/3025572313 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4100a2c30 0x7fa4100a50f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.565+0000 7fa42c352640 1 --2- 172.21.15.172:0/3025572313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa424101d80 0x7fa42419e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.565+0000 7fa42c352640 1 --2- 172.21.15.172:0/3025572313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4241013c0 0x7fa424197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.565+0000 7fa42c352640 1 --2- 172.21.15.172:0/3025572313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4241001d0 0x7fa424196c20 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.565+0000 7fa42c352640 1 -- 172.21.15.172:0/3025572313 >> 172.21.15.172:0/3025572313 conn(0x7fa4240fb930 msgr2=0x7fa4241049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.565+0000 7fa42c352640 1 -- 172.21.15.172:0/3025572313 shutdown_connections 2024-09-10T08:50:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:28.565+0000 7fa42c352640 1 -- 172.21.15.172:0/3025572313 wait complete. 2024-09-10T08:50:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:28 smithi172 ceph-mon[30854]: pgmap v477: 129 pgs: 129 active+clean; 2.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 45 op/s 2024-09-10T08:50:28.697 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5e6021f1', quiesce took 6.1 seconds. Will release after: 16.65 2024-09-10T08:50:28.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:28 smithi174 ceph-mon[34068]: pgmap v477: 129 pgs: 129 active+clean; 2.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 45 op/s 2024-09-10T08:50:28.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:28 smithi191 ceph-mon[35314]: pgmap v477: 129 pgs: 129 active+clean; 2.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 45 op/s 2024-09-10T08:50:29.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:29 smithi174 ceph-mon[34068]: pgmap v478: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 26 MiB/s wr, 46 op/s 2024-09-10T08:50:29.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:29 smithi191 ceph-mon[35314]: pgmap v478: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 26 MiB/s wr, 46 op/s 2024-09-10T08:50:29.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:29 smithi172 ceph-mon[30854]: pgmap v478: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 26 MiB/s wr, 46 op/s 2024-09-10T08:50:32.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:32 smithi191 ceph-mon[35314]: pgmap v479: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 29 MiB/s wr, 52 op/s 2024-09-10T08:50:32.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:32 smithi172 ceph-mon[30854]: pgmap v479: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 29 MiB/s wr, 52 op/s 2024-09-10T08:50:32.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:32 smithi174 ceph-mon[34068]: pgmap v479: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 29 MiB/s wr, 52 op/s 2024-09-10T08:50:34.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:34 smithi191 ceph-mon[35314]: pgmap v480: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 21 MiB/s wr, 46 op/s 2024-09-10T08:50:34.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:34 smithi172 ceph-mon[30854]: pgmap v480: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 21 MiB/s wr, 46 op/s 2024-09-10T08:50:34.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:34 smithi174 ceph-mon[34068]: pgmap v480: 129 pgs: 129 active+clean; 2.9 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 21 MiB/s wr, 46 op/s 2024-09-10T08:50:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:36 smithi191 ceph-mon[35314]: pgmap v481: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 20 MiB/s wr, 47 op/s 2024-09-10T08:50:36.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:36 smithi172 ceph-mon[30854]: pgmap v481: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 20 MiB/s wr, 47 op/s 2024-09-10T08:50:36.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:36 smithi174 ceph-mon[34068]: pgmap v481: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 20 MiB/s wr, 47 op/s 2024-09-10T08:50:37.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:37 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:50:37.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:37 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:50:37.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:37 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:50:37.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:37 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:50:37.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:37 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:50:37.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:37 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:50:37.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:37 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:50:37.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:37 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:50:37.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:37 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:50:37.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:37 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:50:37.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:37 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:50:37.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:37 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:50:38.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:38 smithi191 ceph-mon[35314]: pgmap v482: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 33 op/s 2024-09-10T08:50:38.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:38 smithi172 ceph-mon[30854]: pgmap v482: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 33 op/s 2024-09-10T08:50:38.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:38 smithi174 ceph-mon[34068]: pgmap v482: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 13 MiB/s wr, 33 op/s 2024-09-10T08:50:40.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:40 smithi191 ceph-mon[35314]: pgmap v483: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 MiB/s wr, 20 op/s 2024-09-10T08:50:40.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:40 smithi172 ceph-mon[30854]: pgmap v483: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 MiB/s wr, 20 op/s 2024-09-10T08:50:40.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:40 smithi174 ceph-mon[34068]: pgmap v483: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.3 MiB/s wr, 20 op/s 2024-09-10T08:50:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.663+0000 7fa803386640 1 -- 172.21.15.172:0/965281440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7fc0ffbc0 msgr2=0x7fa7fc100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.663+0000 7fa803386640 1 --2- 172.21.15.172:0/965281440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7fc0ffbc0 0x7fa7fc100040 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0009930 tx=0x7fa7f002a420 comp rx=0 tx=0).stop 2024-09-10T08:50:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.663+0000 7fa803386640 1 -- 172.21.15.172:0/965281440 shutdown_connections 2024-09-10T08:50:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.663+0000 7fa803386640 1 --2- 172.21.15.172:0/965281440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7fc100580 0x7fa7fc10a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.663+0000 7fa803386640 1 --2- 172.21.15.172:0/965281440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7fc0ffbc0 0x7fa7fc100040 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.663+0000 7fa803386640 1 --2- 172.21.15.172:0/965281440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7fc103150 0x7fa7fc103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.663+0000 7fa803386640 1 -- 172.21.15.172:0/965281440 >> 172.21.15.172:0/965281440 conn(0x7fa7fc0fb970 msgr2=0x7fa7fc0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.663+0000 7fa803386640 1 -- 172.21.15.172:0/965281440 shutdown_connections 2024-09-10T08:50:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.663+0000 7fa803386640 1 -- 172.21.15.172:0/965281440 wait complete. 2024-09-10T08:50:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.664+0000 7fa803386640 1 Processor -- start 2024-09-10T08:50:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.664+0000 7fa803386640 1 -- start start 2024-09-10T08:50:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.664+0000 7fa803386640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7fc0ffbc0 0x7fa7fc197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.664+0000 7fa803386640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7fc100580 0x7fa7fc198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.664+0000 7fa803386640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7fc103150 0x7fa7fc19f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.664+0000 7fa803386640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc10d6c0 con 0x7fa7fc100580 2024-09-10T08:50:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.664+0000 7fa803386640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc10d540 con 0x7fa7fc103150 2024-09-10T08:50:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.664+0000 7fa803386640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc10d840 con 0x7fa7fc0ffbc0 2024-09-10T08:50:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.664+0000 7fa800921640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7fc100580 0x7fa7fc198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.664+0000 7fa800921640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7fc100580 0x7fa7fc198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48410/0 (socket says 172.21.15.172:48410) 2024-09-10T08:50:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.664+0000 7fa800921640 1 -- 172.21.15.172:0/1936195150 learned_addr learned my addr 172.21.15.172:0/1936195150 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.664+0000 7fa801122640 1 --2- 172.21.15.172:0/1936195150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7fc0ffbc0 0x7fa7fc197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.664+0000 7fa801923640 1 --2- 172.21.15.172:0/1936195150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7fc103150 0x7fa7fc19f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.664+0000 7fa800921640 1 -- 172.21.15.172:0/1936195150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7fc0ffbc0 msgr2=0x7fa7fc197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.664+0000 7fa800921640 1 --2- 172.21.15.172:0/1936195150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7fc0ffbc0 0x7fa7fc197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.664+0000 7fa800921640 1 -- 172.21.15.172:0/1936195150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7fc103150 msgr2=0x7fa7fc19f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.664+0000 7fa800921640 1 --2- 172.21.15.172:0/1936195150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7fc103150 0x7fa7fc19f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.665+0000 7fa800921640 1 -- 172.21.15.172:0/1936195150 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc19f9d0 con 0x7fa7fc100580 2024-09-10T08:50:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.665+0000 7fa800921640 1 --2- 172.21.15.172:0/1936195150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7fc100580 0x7fa7fc198300 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0004770 tx=0x7fa7f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.665+0000 7fa7fa7fc640 1 -- 172.21.15.172:0/1936195150 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f003f070 con 0x7fa7fc100580 2024-09-10T08:50:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.665+0000 7fa7fa7fc640 1 -- 172.21.15.172:0/1936195150 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7f003b460 con 0x7fa7fc100580 2024-09-10T08:50:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.665+0000 7fa803386640 1 -- 172.21.15.172:0/1936195150 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7fc19fc60 con 0x7fa7fc100580 2024-09-10T08:50:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.665+0000 7fa7fa7fc640 1 -- 172.21.15.172:0/1936195150 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0035070 con 0x7fa7fc100580 2024-09-10T08:50:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.665+0000 7fa803386640 1 -- 172.21.15.172:0/1936195150 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7fc1a0150 con 0x7fa7fc100580 2024-09-10T08:50:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.667+0000 7fa7fa7fc640 1 -- 172.21.15.172:0/1936195150 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa7f003b610 con 0x7fa7fc100580 2024-09-10T08:50:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.668+0000 7fa8061fe740 1 Processor -- start 2024-09-10T08:50:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.668+0000 7fa8061fe740 1 -- start start 2024-09-10T08:50:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.668+0000 7fa8061fe740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 0x5594795901b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.668+0000 7fa8061fe740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x5594795951b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.668+0000 7fa8061fe740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559479595780 0x559479597c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.668+0000 7fa8061fe740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5594794ef2f0 con 0x5594795906f0 2024-09-10T08:50:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.668+0000 7fa8061fe740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559479497270 con 0x55947958fdb0 2024-09-10T08:50:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.668+0000 7fa8061fe740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5594794eb2b0 con 0x559479595780 2024-09-10T08:50:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.668+0000 7fa801122640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x5594795951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.668+0000 7fa801122640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x5594795951b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48418/0 (socket says 172.21.15.172:48418) 2024-09-10T08:50:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.668+0000 7fa801122640 1 -- 172.21.15.172:0/1049018605 learned_addr learned my addr 172.21.15.172:0/1049018605 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.668+0000 7fa7fa7fc640 1 --2- 172.21.15.172:0/1936195150 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7e40a2c30 0x7fa7e40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.669+0000 7fa800921640 1 --2- 172.21.15.172:0/1049018605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559479595780 0x559479597c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.669+0000 7fa801923640 1 --2- 172.21.15.172:0/1049018605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 0x5594795901b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.669+0000 7fa801923640 1 --2- 172.21.15.172:0/1936195150 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7e40a2c30 0x7fa7e40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.669+0000 7fa801122640 1 -- 172.21.15.172:0/1049018605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559479595780 msgr2=0x559479597c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.669+0000 7fa801122640 1 --2- 172.21.15.172:0/1049018605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559479595780 0x559479597c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.669+0000 7fa801122640 1 -- 172.21.15.172:0/1049018605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 msgr2=0x5594795901b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.669+0000 7fa801122640 1 --2- 172.21.15.172:0/1049018605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 0x5594795901b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.669+0000 7fa801122640 1 -- 172.21.15.172:0/1049018605 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559479496340 con 0x5594795906f0 2024-09-10T08:50:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.669+0000 7fa800921640 1 --2- 172.21.15.172:0/1049018605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559479595780 0x559479597c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:50:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.669+0000 7fa7fa7fc640 1 -- 172.21.15.172:0/1936195150 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa7f00ef870 con 0x7fa7fc100580 2024-09-10T08:50:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.669+0000 7fa801122640 1 --2- 172.21.15.172:0/1049018605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x5594795951b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00bc70 tx=0x7fa7ec0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc6edd92739dd3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.669+0000 7fa7e37fe640 1 -- 172.21.15.172:0/1049018605 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec004020 con 0x5594795906f0 2024-09-10T08:50:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.669+0000 7fa7e37fe640 1 -- 172.21.15.172:0/1049018605 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7ec0041c0 con 0x5594795906f0 2024-09-10T08:50:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.669+0000 7fa801923640 1 --2- 172.21.15.172:0/1936195150 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7e40a2c30 0x7fa7e40a50f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4005c40 tx=0x7fa7f4005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.669+0000 7fa7e37fe640 1 -- 172.21.15.172:0/1049018605 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec011660 con 0x5594795906f0 2024-09-10T08:50:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.670+0000 7fa8061fe740 1 -- 172.21.15.172:0/1049018605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 msgr2=0x5594795951b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.670+0000 7fa8061fe740 1 --2- 172.21.15.172:0/1049018605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x5594795951b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00bc70 tx=0x7fa7ec0107a0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.670+0000 7fa8061fe740 1 -- 172.21.15.172:0/1049018605 shutdown_connections 2024-09-10T08:50:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.670+0000 7fa8061fe740 1 --2- 172.21.15.172:0/1049018605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559479595780 0x559479597c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.670+0000 7fa8061fe740 1 --2- 172.21.15.172:0/1049018605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x5594795951b0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.670+0000 7fa8061fe740 1 --2- 172.21.15.172:0/1049018605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 0x5594795901b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.670+0000 7fa8061fe740 1 -- 172.21.15.172:0/1049018605 >> 172.21.15.172:0/1049018605 conn(0x559479587010 msgr2=0x559479587410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.670+0000 7fa8061fe740 1 -- 172.21.15.172:0/1049018605 shutdown_connections 2024-09-10T08:50:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.670+0000 7fa8061fe740 1 -- 172.21.15.172:0/1049018605 wait complete. 2024-09-10T08:50:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.670+0000 7fa8061fe740 1 -- start start 2024-09-10T08:50:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.670+0000 7fa8061fe740 1 Processor -- start 2024-09-10T08:50:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa8061fe740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 0x559479599fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa8061fe740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x55947959a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa8061fe740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559479595780 0x5594795a1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa8061fe740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559479598750 con 0x5594795906f0 2024-09-10T08:50:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa8061fe740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559479478290 con 0x55947958fdb0 2024-09-10T08:50:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa8061fe740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5594795988d0 con 0x559479595780 2024-09-10T08:50:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa8061fe740 10 client.0 _ll_register_callbacks cb 0x5594794a9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa800921640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x55947959a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa800921640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x55947959a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48420/0 (socket says 172.21.15.172:48420) 2024-09-10T08:50:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa800921640 1 -- 172.21.15.172:0/1091777796 learned_addr learned my addr 172.21.15.172:0/1091777796 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa801923640 1 --2- 172.21.15.172:0/1091777796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559479595780 0x5594795a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa801122640 1 --2- 172.21.15.172:0/1091777796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 0x559479599fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa800921640 1 -- 172.21.15.172:0/1091777796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559479595780 msgr2=0x5594795a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa800921640 1 --2- 172.21.15.172:0/1091777796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559479595780 0x5594795a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa800921640 1 -- 172.21.15.172:0/1091777796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 msgr2=0x559479599fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa800921640 1 --2- 172.21.15.172:0/1091777796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 0x559479599fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa800921640 1 -- 172.21.15.172:0/1091777796 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559479202210 con 0x5594795906f0 2024-09-10T08:50:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa801122640 1 --2- 172.21.15.172:0/1091777796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 0x559479599fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:50:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa800921640 1 --2- 172.21.15.172:0/1091777796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x55947959a4e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0000c00 tx=0x7fa7f0086e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01c86c7a13a61b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.671+0000 7fa7e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.672+0000 7fa7e0ff9640 1 -- 172.21.15.172:0/1091777796 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0048070 con 0x5594795906f0 2024-09-10T08:50:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.672+0000 7fa7e0ff9640 1 -- 172.21.15.172:0/1091777796 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa7f003b940 con 0x5594795906f0 2024-09-10T08:50:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.672+0000 7fa8061fe740 1 -- 172.21.15.172:0/1091777796 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7ec0041c0 con 0x5594795906f0 2024-09-10T08:50:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.672+0000 7fa7e0ff9640 1 -- 172.21.15.172:0/1091777796 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa7f00044e0 con 0x5594795906f0 2024-09-10T08:50:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.672+0000 7fa8061fe740 10 client.16716 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.672+0000 7fa8061fe740 1 -- 172.21.15.172:0/1091777796 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5594792ce620 con 0x5594795906f0 2024-09-10T08:50:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.672+0000 7fa7e0ff9640 1 -- 172.21.15.172:0/1091777796 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7ec0041c0 con 0x5594795906f0 2024-09-10T08:50:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.672+0000 7fa7e0ff9640 1 -- 172.21.15.172:0/1091777796 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fa7f004b060 con 0x5594795906f0 2024-09-10T08:50:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.672+0000 7fa8061fe740 10 client.16716 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.672+0000 7fa8061fe740 10 client.16716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.672+0000 7fa8061fe740 20 client.16716 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.672+0000 7fa8061fe740 1 --2- 172.21.15.172:0/1091777796 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5594795a5ad0 0x5594795a7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.672+0000 7fa8061fe740 4 client.16716 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:50:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.672+0000 7fa8061fe740 1 -- 172.21.15.172:0/1091777796 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594794fca20 con 0x5594795a5ad0 2024-09-10T08:50:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.673+0000 7fa801122640 1 --2- 172.21.15.172:0/1091777796 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5594795a5ad0 0x5594795a7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.673+0000 7fa801122640 1 --2- 172.21.15.172:0/1091777796 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5594795a5ad0 0x5594795a7f90 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e2c77d49a2d1396 server_cookie=e5aa413482987464 in_seq=0 out_seq=0 2024-09-10T08:50:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.673+0000 7fa7e0ff9640 10 client.16716 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.674+0000 7fa7e0ff9640 1 -- 172.21.15.172:0/1091777796 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594794fca20 con 0x5594795a5ad0 2024-09-10T08:50:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.674+0000 7fa7e0ff9640 10 client.16716 handle_command_reply: tid=0 2024-09-10T08:50:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.674+0000 7fa8061fe740 1 client.16716 shutdown 2024-09-10T08:50:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.674+0000 7fa8061fe740 1 -- 172.21.15.172:0/1091777796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 msgr2=0x55947959a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.674+0000 7fa8061fe740 1 --2- 172.21.15.172:0/1091777796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x55947959a4e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0000c00 tx=0x7fa7f0086e00 comp rx=0 tx=0).stop 2024-09-10T08:50:41.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.674+0000 7fa8061fe740 1 -- 172.21.15.172:0/1091777796 shutdown_connections 2024-09-10T08:50:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.674+0000 7fa8061fe740 1 --2- 172.21.15.172:0/1091777796 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5594795a5ad0 0x5594795a7f90 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.674+0000 7fa8061fe740 1 --2- 172.21.15.172:0/1091777796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559479595780 0x5594795a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.674+0000 7fa8061fe740 1 --2- 172.21.15.172:0/1091777796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x55947959a4e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.674+0000 7fa7e0ff9640 0 client.16716 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.674+0000 7fa8061fe740 1 --2- 172.21.15.172:0/1091777796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 0x559479599fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.674+0000 7fa8061fe740 1 -- 172.21.15.172:0/1091777796 >> 172.21.15.172:0/1091777796 conn(0x559479587010 msgr2=0x559479588820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.674+0000 7fa8061fe740 1 -- 172.21.15.172:0/1091777796 shutdown_connections 2024-09-10T08:50:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.674+0000 7fa8061fe740 1 -- 172.21.15.172:0/1091777796 wait complete. 2024-09-10T08:50:41.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.674+0000 7fa8061fe740 20 client.16716 trim_cache size 0 max 16384 2024-09-10T08:50:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa8061fe740 1 Processor -- start 2024-09-10T08:50:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa8061fe740 1 -- start start 2024-09-10T08:50:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa8061fe740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 0x5594795860b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa8061fe740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594795906f0 0x5594795a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa8061fe740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559479595780 0x5594795ac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa8061fe740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1a8370 con 0x559479595780 2024-09-10T08:50:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa8061fe740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1a9f70 con 0x55947958fdb0 2024-09-10T08:50:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa8061fe740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1a8a40 con 0x5594795906f0 2024-09-10T08:50:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa801923640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559479595780 0x5594795ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa801923640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559479595780 0x5594795ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48432/0 (socket says 172.21.15.172:48432) 2024-09-10T08:50:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa801923640 1 -- 172.21.15.172:0/1623304450 learned_addr learned my addr 172.21.15.172:0/1623304450 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa800921640 1 --2- 172.21.15.172:0/1623304450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594795906f0 0x5594795a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa801122640 1 --2- 172.21.15.172:0/1623304450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 0x5594795860b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa801923640 1 -- 172.21.15.172:0/1623304450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594795906f0 msgr2=0x5594795a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa801923640 1 --2- 172.21.15.172:0/1623304450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594795906f0 0x5594795a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.695 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa801923640 1 -- 172.21.15.172:0/1623304450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 msgr2=0x5594795860b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa801923640 1 --2- 172.21.15.172:0/1623304450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 0x5594795860b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa801923640 1 -- 172.21.15.172:0/1623304450 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559479599ae0 con 0x559479595780 2024-09-10T08:50:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa801122640 1 --2- 172.21.15.172:0/1623304450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 0x5594795860b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.689+0000 7fa801923640 1 --2- 172.21.15.172:0/1623304450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559479595780 0x5594795ac890 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa7f401c4f0 tx=0x7fa7f4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9329f78903d0d6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.690+0000 7fa7e2ffd640 1 -- 172.21.15.172:0/1623304450 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f400f6e0 con 0x559479595780 2024-09-10T08:50:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.690+0000 7fa7e2ffd640 1 -- 172.21.15.172:0/1623304450 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7f402b070 con 0x559479595780 2024-09-10T08:50:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.690+0000 7fa7e2ffd640 1 -- 172.21.15.172:0/1623304450 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4026c70 con 0x559479595780 2024-09-10T08:50:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.690+0000 7fa8061fe740 1 -- 172.21.15.172:0/1623304450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559479595780 msgr2=0x5594795ac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.690+0000 7fa8061fe740 1 --2- 172.21.15.172:0/1623304450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559479595780 0x5594795ac890 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa7f401c4f0 tx=0x7fa7f4014040 comp rx=0 tx=0).stop 2024-09-10T08:50:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.690+0000 7fa8061fe740 1 -- 172.21.15.172:0/1623304450 shutdown_connections 2024-09-10T08:50:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.690+0000 7fa8061fe740 1 --2- 172.21.15.172:0/1623304450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559479595780 0x5594795ac890 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.690+0000 7fa8061fe740 1 --2- 172.21.15.172:0/1623304450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594795906f0 0x5594795a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.690+0000 7fa8061fe740 1 --2- 172.21.15.172:0/1623304450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 0x5594795860b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.690+0000 7fa8061fe740 1 -- 172.21.15.172:0/1623304450 >> 172.21.15.172:0/1623304450 conn(0x559479587010 msgr2=0x5594795859a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.690+0000 7fa8061fe740 1 -- 172.21.15.172:0/1623304450 shutdown_connections 2024-09-10T08:50:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.690+0000 7fa8061fe740 1 -- 172.21.15.172:0/1623304450 wait complete. 2024-09-10T08:50:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.690+0000 7fa8061fe740 1 -- start start 2024-09-10T08:50:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa8061fe740 1 Processor -- start 2024-09-10T08:50:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa8061fe740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 0x55947959edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa8061fe740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x55947959f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.697 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa8061fe740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559479595780 0x5594795a0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa8061fe740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1a9810 con 0x5594795906f0 2024-09-10T08:50:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa8061fe740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1a9140 con 0x55947958fdb0 2024-09-10T08:50:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa8061fe740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1a6350 con 0x559479595780 2024-09-10T08:50:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa8061fe740 10 client.0 _ll_register_callbacks cb 0x7fa7f0009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa800921640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x55947959f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa800921640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x55947959f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48436/0 (socket says 172.21.15.172:48436) 2024-09-10T08:50:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa800921640 1 -- 172.21.15.172:0/3594123028 learned_addr learned my addr 172.21.15.172:0/3594123028 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa801923640 1 --2- 172.21.15.172:0/3594123028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559479595780 0x5594795a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:41.698 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa800921640 1 -- 172.21.15.172:0/3594123028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559479595780 msgr2=0x5594795a0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa800921640 1 --2- 172.21.15.172:0/3594123028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559479595780 0x5594795a0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa800921640 1 -- 172.21.15.172:0/3594123028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 msgr2=0x55947959edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa800921640 1 --2- 172.21.15.172:0/3594123028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 0x55947959edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa800921640 1 -- 172.21.15.172:0/3594123028 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559479591380 con 0x5594795906f0 2024-09-10T08:50:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa800921640 1 --2- 172.21.15.172:0/3594123028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x55947959f330 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0005940 tx=0x7fa7f003bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b94af1aeea8b66 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.691+0000 7fa7e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.692+0000 7fa7e0ff9640 1 -- 172.21.15.172:0/3594123028 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0048070 con 0x5594795906f0 2024-09-10T08:50:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.692+0000 7fa7e0ff9640 1 -- 172.21.15.172:0/3594123028 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa7f00c4750 con 0x5594795906f0 2024-09-10T08:50:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.692+0000 7fa8061fe740 1 -- 172.21.15.172:0/3594123028 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7f402b070 con 0x5594795906f0 2024-09-10T08:50:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.692+0000 7fa7e0ff9640 1 -- 172.21.15.172:0/3594123028 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa7f00bc470 con 0x5594795906f0 2024-09-10T08:50:41.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.692+0000 7fa8061fe740 10 client.16722 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.692+0000 7fa8061fe740 1 -- 172.21.15.172:0/3594123028 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5594796109a0 con 0x5594795906f0 2024-09-10T08:50:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.692+0000 7fa7e0ff9640 1 -- 172.21.15.172:0/3594123028 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7f402b070 con 0x5594795906f0 2024-09-10T08:50:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.692+0000 7fa7e0ff9640 1 -- 172.21.15.172:0/3594123028 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fa7f00d2070 con 0x5594795906f0 2024-09-10T08:50:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.692+0000 7fa8061fe740 10 client.16722 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.692+0000 7fa8061fe740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.692+0000 7fa8061fe740 20 client.16722 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.692+0000 7fa8061fe740 1 --2- 172.21.15.172:0/3594123028 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5594796111f0 0x559479613610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.692+0000 7fa8061fe740 4 client.16722 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:50:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.692+0000 7fa8061fe740 1 -- 172.21.15.172:0/3594123028 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5594794f6d70 con 0x5594796111f0 2024-09-10T08:50:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.693+0000 7fa801122640 1 --2- 172.21.15.172:0/3594123028 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5594796111f0 0x559479613610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:41.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.693+0000 7fa801122640 1 --2- 172.21.15.172:0/3594123028 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5594796111f0 0x559479613610 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60b721c42b38ba1b server_cookie=1f2382a12c1cc8c7 in_seq=0 out_seq=0 2024-09-10T08:50:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.693+0000 7fa7e0ff9640 10 client.16722 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.699+0000 7fa7e0ff9640 1 -- 172.21.15.172:0/3594123028 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5594794f6d70 con 0x5594796111f0 2024-09-10T08:50:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.700+0000 7fa7e0ff9640 10 client.16722 handle_command_reply: tid=0 2024-09-10T08:50:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.700+0000 7fa8061fe740 1 client.16722 shutdown 2024-09-10T08:50:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.700+0000 7fa8061fe740 1 -- 172.21.15.172:0/3594123028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 msgr2=0x55947959f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.700+0000 7fa8061fe740 1 --2- 172.21.15.172:0/3594123028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x55947959f330 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0005940 tx=0x7fa7f003bb70 comp rx=0 tx=0).stop 2024-09-10T08:50:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.700+0000 7fa8061fe740 1 -- 172.21.15.172:0/3594123028 shutdown_connections 2024-09-10T08:50:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.700+0000 7fa8061fe740 1 --2- 172.21.15.172:0/3594123028 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5594796111f0 0x559479613610 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.700+0000 7fa8061fe740 1 --2- 172.21.15.172:0/3594123028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559479595780 0x5594795a0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.700+0000 7fa7e0ff9640 0 client.16722 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.700+0000 7fa8061fe740 1 --2- 172.21.15.172:0/3594123028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594795906f0 0x55947959f330 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.700+0000 7fa8061fe740 1 --2- 172.21.15.172:0/3594123028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55947958fdb0 0x55947959edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.700+0000 7fa8061fe740 1 -- 172.21.15.172:0/3594123028 >> 172.21.15.172:0/3594123028 conn(0x559479587010 msgr2=0x5594795859a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.700+0000 7fa8061fe740 1 -- 172.21.15.172:0/3594123028 shutdown_connections 2024-09-10T08:50:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.700+0000 7fa8061fe740 1 -- 172.21.15.172:0/3594123028 wait complete. 2024-09-10T08:50:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.700+0000 7fa8061fe740 20 client.16722 trim_cache size 0 max 16384 2024-09-10T08:50:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:50:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:50:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:50:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:50:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.701+0000 7fa803386640 1 -- 172.21.15.172:0/1936195150 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7e40a2c30 msgr2=0x7fa7e40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.701+0000 7fa803386640 1 --2- 172.21.15.172:0/1936195150 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7e40a2c30 0x7fa7e40a50f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4005c40 tx=0x7fa7f4005bd0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.701+0000 7fa803386640 1 -- 172.21.15.172:0/1936195150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7fc100580 msgr2=0x7fa7fc198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.701+0000 7fa803386640 1 --2- 172.21.15.172:0/1936195150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7fc100580 0x7fa7fc198300 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0004770 tx=0x7fa7f00042f0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.701+0000 7fa803386640 1 -- 172.21.15.172:0/1936195150 shutdown_connections 2024-09-10T08:50:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.701+0000 7fa803386640 1 --2- 172.21.15.172:0/1936195150 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa7e40a2c30 0x7fa7e40a50f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.701+0000 7fa803386640 1 --2- 172.21.15.172:0/1936195150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7fc103150 0x7fa7fc19f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.701+0000 7fa803386640 1 --2- 172.21.15.172:0/1936195150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7fc100580 0x7fa7fc198300 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.701+0000 7fa803386640 1 --2- 172.21.15.172:0/1936195150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7fc0ffbc0 0x7fa7fc197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.701+0000 7fa803386640 1 -- 172.21.15.172:0/1936195150 >> 172.21.15.172:0/1936195150 conn(0x7fa7fc0fb970 msgr2=0x7fa7fc1062f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.701+0000 7fa803386640 1 -- 172.21.15.172:0/1936195150 shutdown_connections 2024-09-10T08:50:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:41.701+0000 7fa803386640 1 -- 172.21.15.172:0/1936195150 wait complete. 2024-09-10T08:50:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:42 smithi172 ceph-mon[30854]: pgmap v484: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 18 op/s 2024-09-10T08:50:42.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:42 smithi174 ceph-mon[34068]: pgmap v484: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 18 op/s 2024-09-10T08:50:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:42 smithi191 ceph-mon[35314]: pgmap v484: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.3 MiB/s wr, 18 op/s 2024-09-10T08:50:44.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:44 smithi172 ceph-mon[30854]: pgmap v485: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 656 KiB/s wr, 10 op/s 2024-09-10T08:50:44.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:44 smithi174 ceph-mon[34068]: pgmap v485: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 656 KiB/s wr, 10 op/s 2024-09-10T08:50:44.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:44 smithi191 ceph-mon[35314]: pgmap v485: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 656 KiB/s wr, 10 op/s 2024-09-10T08:50:45.348 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:50:45.612 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:50:46.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.079+0000 7effd48fd640 1 -- 172.21.15.172:0/220189738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd010c2f0 msgr2=0x7effd010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:46.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.079+0000 7effd48fd640 1 --2- 172.21.15.172:0/220189738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd010c2f0 0x7effd010e790 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7effc8009900 tx=0x7effc802a410 comp rx=0 tx=0).stop 2024-09-10T08:50:46.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.079+0000 7effd48fd640 1 -- 172.21.15.172:0/220189738 shutdown_connections 2024-09-10T08:50:46.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.079+0000 7effd48fd640 1 --2- 172.21.15.172:0/220189738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd010c2f0 0x7effd010e790 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:46.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.079+0000 7effd48fd640 1 --2- 172.21.15.172:0/220189738 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effd01037a0 0x7effd010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:46.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.079+0000 7effd48fd640 1 --2- 172.21.15.172:0/220189738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7effd0102df0 0x7effd01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:46.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.079+0000 7effd48fd640 1 -- 172.21.15.172:0/220189738 >> 172.21.15.172:0/220189738 conn(0x7effd00fc7d0 msgr2=0x7effd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:46.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.079+0000 7effd48fd640 1 -- 172.21.15.172:0/220189738 shutdown_connections 2024-09-10T08:50:46.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.079+0000 7effd48fd640 1 -- 172.21.15.172:0/220189738 wait complete. 2024-09-10T08:50:46.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.080+0000 7effd48fd640 1 Processor -- start 2024-09-10T08:50:46.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.080+0000 7effd48fd640 1 -- start start 2024-09-10T08:50:46.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.080+0000 7effd48fd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effd0102df0 0x7effd01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:46.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.080+0000 7effd48fd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7effd01037a0 0x7effd0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:46.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.080+0000 7effd48fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd010c2f0 0x7effd019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:46.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.080+0000 7effd48fd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7effd0112080 con 0x7effd010c2f0 2024-09-10T08:50:46.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.080+0000 7effd48fd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7effd0111f00 con 0x7effd01037a0 2024-09-10T08:50:46.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.080+0000 7effd48fd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7effd0112200 con 0x7effd0102df0 2024-09-10T08:50:46.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.080+0000 7effced9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd010c2f0 0x7effd019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:46.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.080+0000 7effced9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd010c2f0 0x7effd019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48452/0 (socket says 172.21.15.172:48452) 2024-09-10T08:50:46.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.080+0000 7effced9d640 1 -- 172.21.15.172:0/1442107511 learned_addr learned my addr 172.21.15.172:0/1442107511 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:46.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.081+0000 7effcdd9b640 1 --2- 172.21.15.172:0/1442107511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7effd01037a0 0x7effd0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:46.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.081+0000 7effce59c640 1 --2- 172.21.15.172:0/1442107511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effd0102df0 0x7effd01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:46.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.081+0000 7effced9d640 1 -- 172.21.15.172:0/1442107511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effd0102df0 msgr2=0x7effd01985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:46.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.081+0000 7effced9d640 1 --2- 172.21.15.172:0/1442107511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effd0102df0 0x7effd01985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:46.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.081+0000 7effced9d640 1 -- 172.21.15.172:0/1442107511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7effd01037a0 msgr2=0x7effd0198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:46.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.081+0000 7effced9d640 1 --2- 172.21.15.172:0/1442107511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7effd01037a0 0x7effd0198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:46.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.081+0000 7effced9d640 1 -- 172.21.15.172:0/1442107511 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd019d570 con 0x7effd010c2f0 2024-09-10T08:50:46.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.081+0000 7effcdd9b640 1 --2- 172.21.15.172:0/1442107511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7effd01037a0 0x7effd0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:46.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.081+0000 7effced9d640 1 --2- 172.21.15.172:0/1442107511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd010c2f0 0x7effd019ce90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7effc8009a20 tx=0x7effc80042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:46.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.081+0000 7effc77fe640 1 -- 172.21.15.172:0/1442107511 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc803f030 con 0x7effd010c2f0 2024-09-10T08:50:46.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.081+0000 7effc77fe640 1 -- 172.21.15.172:0/1442107511 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effc80044d0 con 0x7effd010c2f0 2024-09-10T08:50:46.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.081+0000 7effd48fd640 1 -- 172.21.15.172:0/1442107511 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd019d800 con 0x7effd010c2f0 2024-09-10T08:50:46.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.081+0000 7effc77fe640 1 -- 172.21.15.172:0/1442107511 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8035070 con 0x7effd010c2f0 2024-09-10T08:50:46.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.081+0000 7effd48fd640 1 -- 172.21.15.172:0/1442107511 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd01a5140 con 0x7effd010c2f0 2024-09-10T08:50:46.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.082+0000 7effd48fd640 1 -- 172.21.15.172:0/1442107511 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd01031d0 con 0x7effd010c2f0 2024-09-10T08:50:46.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.083+0000 7effc77fe640 1 -- 172.21.15.172:0/1442107511 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7effc8042030 con 0x7effd010c2f0 2024-09-10T08:50:46.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.084+0000 7effc77fe640 1 --2- 172.21.15.172:0/1442107511 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7effbc0a2c00 0x7effbc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:46.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.084+0000 7effce59c640 1 --2- 172.21.15.172:0/1442107511 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7effbc0a2c00 0x7effbc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:46.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.084+0000 7effc77fe640 1 -- 172.21.15.172:0/1442107511 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7effc80efd20 con 0x7effd010c2f0 2024-09-10T08:50:46.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.084+0000 7effce59c640 1 --2- 172.21.15.172:0/1442107511 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7effbc0a2c00 0x7effbc0a50c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7effc0009f20 tx=0x7effc00094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:46.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.088+0000 7effc77fe640 1 -- 172.21.15.172:0/1442107511 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7effc80bc920 con 0x7effd010c2f0 2024-09-10T08:50:46.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.291+0000 7effd48fd640 1 -- 172.21.15.172:0/1442107511 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effd01072c0 con 0x7effd010c2f0 2024-09-10T08:50:46.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.292+0000 7effc77fe640 1 -- 172.21.15.172:0/1442107511 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+12007 (secure 0 0 0) 0x7effc80bc220 con 0x7effd010c2f0 2024-09-10T08:50:46.294 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:50:46.294 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":33,"btime":"2024-09-10T08:50:03:285217+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:50:02.654978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:50:46.294 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 33 2024-09-10T08:50:46.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.294+0000 7effd48fd640 1 -- 172.21.15.172:0/1442107511 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7effbc0a2c00 msgr2=0x7effbc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:46.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.294+0000 7effd48fd640 1 --2- 172.21.15.172:0/1442107511 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7effbc0a2c00 0x7effbc0a50c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7effc0009f20 tx=0x7effc00094f0 comp rx=0 tx=0).stop 2024-09-10T08:50:46.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.294+0000 7effd48fd640 1 -- 172.21.15.172:0/1442107511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd010c2f0 msgr2=0x7effd019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:46.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.294+0000 7effd48fd640 1 --2- 172.21.15.172:0/1442107511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd010c2f0 0x7effd019ce90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7effc8009a20 tx=0x7effc80042e0 comp rx=0 tx=0).stop 2024-09-10T08:50:46.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.294+0000 7effd48fd640 1 -- 172.21.15.172:0/1442107511 shutdown_connections 2024-09-10T08:50:46.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.294+0000 7effd48fd640 1 --2- 172.21.15.172:0/1442107511 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7effbc0a2c00 0x7effbc0a50c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:46.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.294+0000 7effd48fd640 1 --2- 172.21.15.172:0/1442107511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7effd010c2f0 0x7effd019ce90 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:46.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.294+0000 7effd48fd640 1 --2- 172.21.15.172:0/1442107511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7effd01037a0 0x7effd0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:46.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.294+0000 7effd48fd640 1 --2- 172.21.15.172:0/1442107511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effd0102df0 0x7effd01985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:46.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.294+0000 7effd48fd640 1 -- 172.21.15.172:0/1442107511 >> 172.21.15.172:0/1442107511 conn(0x7effd00fc7d0 msgr2=0x7effd00fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:46.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.294+0000 7effd48fd640 1 -- 172.21.15.172:0/1442107511 shutdown_connections 2024-09-10T08:50:46.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:46.294+0000 7effd48fd640 1 -- 172.21.15.172:0/1442107511 wait complete. 2024-09-10T08:50:46.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:46 smithi172 ceph-mon[30854]: pgmap v486: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 656 KiB/s wr, 13 op/s 2024-09-10T08:50:46.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:46 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1442107511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:50:46.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:46 smithi174 ceph-mon[34068]: pgmap v486: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 656 KiB/s wr, 13 op/s 2024-09-10T08:50:46.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:46 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1442107511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:50:46.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:46 smithi191 ceph-mon[35314]: pgmap v486: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 656 KiB/s wr, 13 op/s 2024-09-10T08:50:46.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:46 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1442107511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:50:47.013 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 5e6021f1 --release --await' 2024-09-10T08:50:47.014 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 5e6021f1 --release --await 2024-09-10T08:50:47.260 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:50:47.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.736+0000 7fa846646640 1 -- 172.21.15.172:0/833944131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa840103660 msgr2=0x7fa84010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:47.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.736+0000 7fa846646640 1 --2- 172.21.15.172:0/833944131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa840103660 0x7fa84010bb50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa828009900 tx=0x7fa82802a480 comp rx=0 tx=0).stop 2024-09-10T08:50:47.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.736+0000 7fa846646640 1 -- 172.21.15.172:0/833944131 shutdown_connections 2024-09-10T08:50:47.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.736+0000 7fa846646640 1 --2- 172.21.15.172:0/833944131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa84010c170 0x7fa84010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.736+0000 7fa846646640 1 --2- 172.21.15.172:0/833944131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa840103660 0x7fa84010bb50 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.736+0000 7fa846646640 1 --2- 172.21.15.172:0/833944131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa840102d40 0x7fa840103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.736+0000 7fa846646640 1 -- 172.21.15.172:0/833944131 >> 172.21.15.172:0/833944131 conn(0x7fa8400fc7a0 msgr2=0x7fa8400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:47.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.736+0000 7fa846646640 1 -- 172.21.15.172:0/833944131 shutdown_connections 2024-09-10T08:50:47.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.736+0000 7fa846646640 1 -- 172.21.15.172:0/833944131 wait complete. 2024-09-10T08:50:47.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa846646640 1 Processor -- start 2024-09-10T08:50:47.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa846646640 1 -- start start 2024-09-10T08:50:47.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa846646640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa840102d40 0x7fa840198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa846646640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa840103660 0x7fa840198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa846646640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa84010c170 0x7fa84019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa846646640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa840111e20 con 0x7fa840102d40 2024-09-10T08:50:47.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa846646640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa840111ca0 con 0x7fa840103660 2024-09-10T08:50:47.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa846646640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa840111fa0 con 0x7fa84010c170 2024-09-10T08:50:47.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa840102d40 0x7fa840198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:47.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa840102d40 0x7fa840198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35746/0 (socket says 172.21.15.172:35746) 2024-09-10T08:50:47.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa83ffff640 1 -- 172.21.15.172:0/2091741014 learned_addr learned my addr 172.21.15.172:0/2091741014 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:47.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa837fff640 1 --2- 172.21.15.172:0/2091741014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa840103660 0x7fa840198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:47.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa83ffff640 1 -- 172.21.15.172:0/2091741014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa84010c170 msgr2=0x7fa84019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:47.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa844be3640 1 --2- 172.21.15.172:0/2091741014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa84010c170 0x7fa84019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:47.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa83ffff640 1 --2- 172.21.15.172:0/2091741014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa84010c170 0x7fa84019cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa83ffff640 1 -- 172.21.15.172:0/2091741014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa840103660 msgr2=0x7fa840198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa83ffff640 1 --2- 172.21.15.172:0/2091741014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa840103660 0x7fa840198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa83ffff640 1 -- 172.21.15.172:0/2091741014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84019d2d0 con 0x7fa840102d40 2024-09-10T08:50:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.737+0000 7fa837fff640 1 --2- 172.21.15.172:0/2091741014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa840103660 0x7fa840198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.738+0000 7fa83ffff640 1 --2- 172.21.15.172:0/2091741014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa840102d40 0x7fa840198320 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa83000c890 tx=0x7fa83000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.738+0000 7fa83dffb640 1 -- 172.21.15.172:0/2091741014 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830004220 con 0x7fa840102d40 2024-09-10T08:50:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.738+0000 7fa83dffb640 1 -- 172.21.15.172:0/2091741014 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8300043c0 con 0x7fa840102d40 2024-09-10T08:50:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.738+0000 7fa846646640 1 -- 172.21.15.172:0/2091741014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84019d5c0 con 0x7fa840102d40 2024-09-10T08:50:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.738+0000 7fa83dffb640 1 -- 172.21.15.172:0/2091741014 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830004a00 con 0x7fa840102d40 2024-09-10T08:50:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.738+0000 7fa846646640 1 -- 172.21.15.172:0/2091741014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8401a4ea0 con 0x7fa840102d40 2024-09-10T08:50:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.740+0000 7fa83dffb640 1 -- 172.21.15.172:0/2091741014 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa830004ba0 con 0x7fa840102d40 2024-09-10T08:50:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.740+0000 7fa848a1f740 1 Processor -- start 2024-09-10T08:50:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.740+0000 7fa848a1f740 1 -- start start 2024-09-10T08:50:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa848a1f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea63f20 0x55d6dea64300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa848a1f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea64840 0x55d6dea73110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa848a1f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 0x55d6dea75a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa848a1f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d6de9c9620 con 0x55d6dea64840 2024-09-10T08:50:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa848a1f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d6de970e50 con 0x55d6dea73650 2024-09-10T08:50:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa848a1f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d6de9c55e0 con 0x55d6dea63f20 2024-09-10T08:50:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea64840 0x55d6dea73110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea64840 0x55d6dea73110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35752/0 (socket says 172.21.15.172:35752) 2024-09-10T08:50:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa83ffff640 1 -- 172.21.15.172:0/694630403 learned_addr learned my addr 172.21.15.172:0/694630403 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa844be3640 1 --2- 172.21.15.172:0/694630403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea63f20 0x55d6dea64300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa83dffb640 1 --2- 172.21.15.172:0/2091741014 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa82c0a2b80 0x7fa82c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa837fff640 1 --2- 172.21.15.172:0/694630403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 0x55d6dea75a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa844be3640 1 --2- 172.21.15.172:0/2091741014 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa82c0a2b80 0x7fa82c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa83ffff640 1 -- 172.21.15.172:0/694630403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea63f20 msgr2=0x55d6dea64300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa83ffff640 1 --2- 172.21.15.172:0/694630403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea63f20 0x55d6dea64300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa83ffff640 1 -- 172.21.15.172:0/694630403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 msgr2=0x55d6dea75a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa83ffff640 1 --2- 172.21.15.172:0/694630403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 0x55d6dea75a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa83dffb640 1 -- 172.21.15.172:0/2091741014 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa8300cf110 con 0x7fa840102d40 2024-09-10T08:50:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.741+0000 7fa83ffff640 1 -- 172.21.15.172:0/694630403 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6de961d40 con 0x55d6dea64840 2024-09-10T08:50:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.742+0000 7fa83ffff640 1 --2- 172.21.15.172:0/694630403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea64840 0x55d6dea73110 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa83009b310 tx=0x7fa830000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0e58e389a36fd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.742+0000 7fa8367fc640 1 -- 172.21.15.172:0/694630403 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83001d070 con 0x55d6dea64840 2024-09-10T08:50:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.742+0000 7fa844be3640 1 --2- 172.21.15.172:0/2091741014 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa82c0a2b80 0x7fa82c0a5040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa838005cc0 tx=0x7fa838004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.742+0000 7fa8367fc640 1 -- 172.21.15.172:0/694630403 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa83009bd10 con 0x55d6dea64840 2024-09-10T08:50:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.742+0000 7fa8367fc640 1 -- 172.21.15.172:0/694630403 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83009b610 con 0x55d6dea64840 2024-09-10T08:50:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.742+0000 7fa848a1f740 1 -- 172.21.15.172:0/694630403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea64840 msgr2=0x55d6dea73110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.742+0000 7fa848a1f740 1 --2- 172.21.15.172:0/694630403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea64840 0x55d6dea73110 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa83009b310 tx=0x7fa830000bc0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.742+0000 7fa848a1f740 1 -- 172.21.15.172:0/694630403 shutdown_connections 2024-09-10T08:50:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.742+0000 7fa848a1f740 1 --2- 172.21.15.172:0/694630403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 0x55d6dea75a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.742+0000 7fa848a1f740 1 --2- 172.21.15.172:0/694630403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea64840 0x55d6dea73110 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.742+0000 7fa848a1f740 1 --2- 172.21.15.172:0/694630403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea63f20 0x55d6dea64300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.742+0000 7fa848a1f740 1 -- 172.21.15.172:0/694630403 >> 172.21.15.172:0/694630403 conn(0x55d6dea605f0 msgr2=0x55d6dea609d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.743+0000 7fa848a1f740 1 -- 172.21.15.172:0/694630403 shutdown_connections 2024-09-10T08:50:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.743+0000 7fa848a1f740 1 -- 172.21.15.172:0/694630403 wait complete. 2024-09-10T08:50:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.743+0000 7fa848a1f740 1 -- start start 2024-09-10T08:50:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.743+0000 7fa848a1f740 1 Processor -- start 2024-09-10T08:50:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.743+0000 7fa848a1f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea63f20 0x55d6dea70580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.743+0000 7fa848a1f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea64840 0x55d6dea70ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.743+0000 7fa848a1f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 0x55d6dea71000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.743+0000 7fa848a1f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d6dea761c0 con 0x55d6dea63f20 2024-09-10T08:50:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.743+0000 7fa848a1f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d6de951f10 con 0x55d6dea73650 2024-09-10T08:50:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.743+0000 7fa848a1f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d6dea76340 con 0x55d6dea64840 2024-09-10T08:50:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.743+0000 7fa848a1f740 10 client.0 _ll_register_callbacks cb 0x55d6de975db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.743+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea63f20 0x55d6dea70580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.743+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea63f20 0x55d6dea70580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35764/0 (socket says 172.21.15.172:35764) 2024-09-10T08:50:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.743+0000 7fa83ffff640 1 -- 172.21.15.172:0/794720269 learned_addr learned my addr 172.21.15.172:0/794720269 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa837fff640 1 --2- 172.21.15.172:0/794720269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea64840 0x55d6dea70ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:47.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa844be3640 1 --2- 172.21.15.172:0/794720269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 0x55d6dea71000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:47.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa83ffff640 1 -- 172.21.15.172:0/794720269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea64840 msgr2=0x55d6dea70ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:47.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa83ffff640 1 --2- 172.21.15.172:0/794720269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea64840 0x55d6dea70ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa83ffff640 1 -- 172.21.15.172:0/794720269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 msgr2=0x55d6dea71000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:47.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa83ffff640 1 --2- 172.21.15.172:0/794720269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 0x55d6dea71000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa83ffff640 1 -- 172.21.15.172:0/794720269 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6de6d8580 con 0x55d6dea63f20 2024-09-10T08:50:47.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa837fff640 1 --2- 172.21.15.172:0/794720269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea64840 0x55d6dea70ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:47.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa83ffff640 1 --2- 172.21.15.172:0/794720269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea63f20 0x55d6dea70580 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa830097fe0 tx=0x7fa83009afe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab90ab369f74f73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:47.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa81f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:47.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa81f7fe640 1 -- 172.21.15.172:0/794720269 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83001d040 con 0x55d6dea63f20 2024-09-10T08:50:47.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa81f7fe640 1 -- 172.21.15.172:0/794720269 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa8300050b0 con 0x55d6dea63f20 2024-09-10T08:50:47.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa848a1f740 1 -- 172.21.15.172:0/794720269 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa83009bd10 con 0x55d6dea63f20 2024-09-10T08:50:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa81f7fe640 1 -- 172.21.15.172:0/794720269 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa83009b610 con 0x55d6dea63f20 2024-09-10T08:50:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa848a1f740 10 client.16740 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa848a1f740 1 -- 172.21.15.172:0/794720269 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d6de7a2960 con 0x55d6dea63f20 2024-09-10T08:50:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa81f7fe640 1 -- 172.21.15.172:0/794720269 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa83009bd10 con 0x55d6dea63f20 2024-09-10T08:50:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.744+0000 7fa81f7fe640 1 -- 172.21.15.172:0/794720269 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fa830097360 con 0x55d6dea63f20 2024-09-10T08:50:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.745+0000 7fa848a1f740 10 client.16740 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.745+0000 7fa848a1f740 10 client.16740 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:50:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.745+0000 7fa848a1f740 20 client.16740 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.745+0000 7fa848a1f740 1 --2- 172.21.15.172:0/794720269 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d6dea83080 0x55d6dea85470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.745+0000 7fa848a1f740 4 client.16740 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:50:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.745+0000 7fa848a1f740 1 -- 172.21.15.172:0/794720269 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6de9d6b90 con 0x55d6dea83080 2024-09-10T08:50:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.745+0000 7fa837fff640 1 --2- 172.21.15.172:0/794720269 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d6dea83080 0x55d6dea85470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.745+0000 7fa81f7fe640 10 client.16740 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.745+0000 7fa837fff640 1 --2- 172.21.15.172:0/794720269 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d6dea83080 0x55d6dea85470 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fedb8063fe2a2076 server_cookie=8c0319e20f267c7d in_seq=0 out_seq=0 2024-09-10T08:50:47.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.746+0000 7fa81f7fe640 1 -- 172.21.15.172:0/794720269 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d6de9d6b90 con 0x55d6dea83080 2024-09-10T08:50:47.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.746+0000 7fa81f7fe640 10 client.16740 handle_command_reply: tid=0 2024-09-10T08:50:47.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.746+0000 7fa848a1f740 1 client.16740 shutdown 2024-09-10T08:50:47.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.746+0000 7fa848a1f740 1 -- 172.21.15.172:0/794720269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea63f20 msgr2=0x55d6dea70580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:47.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.746+0000 7fa848a1f740 1 --2- 172.21.15.172:0/794720269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea63f20 0x55d6dea70580 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa830097fe0 tx=0x7fa83009afe0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.746+0000 7fa848a1f740 1 -- 172.21.15.172:0/794720269 shutdown_connections 2024-09-10T08:50:47.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.746+0000 7fa848a1f740 1 --2- 172.21.15.172:0/794720269 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d6dea83080 0x55d6dea85470 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.746+0000 7fa848a1f740 1 --2- 172.21.15.172:0/794720269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 0x55d6dea71000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.746+0000 7fa81f7fe640 0 client.16740 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:47.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.746+0000 7fa848a1f740 1 --2- 172.21.15.172:0/794720269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea64840 0x55d6dea70ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.746+0000 7fa848a1f740 1 --2- 172.21.15.172:0/794720269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea63f20 0x55d6dea70580 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.746+0000 7fa848a1f740 1 -- 172.21.15.172:0/794720269 >> 172.21.15.172:0/794720269 conn(0x55d6dea605f0 msgr2=0x55d6dea66350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:47.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.746+0000 7fa848a1f740 1 -- 172.21.15.172:0/794720269 shutdown_connections 2024-09-10T08:50:47.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.746+0000 7fa848a1f740 1 -- 172.21.15.172:0/794720269 wait complete. 2024-09-10T08:50:47.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.746+0000 7fa848a1f740 20 client.16740 trim_cache size 0 max 16384 2024-09-10T08:50:47.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.763+0000 7fa848a1f740 1 Processor -- start 2024-09-10T08:50:47.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.763+0000 7fa848a1f740 1 -- start start 2024-09-10T08:50:47.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.763+0000 7fa848a1f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea63f20 0x55d6dea660d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.763+0000 7fa848a1f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea64840 0x55d6dea66610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.763+0000 7fa848a1f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 0x55d6dea8ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.763+0000 7fa848a1f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401ac8f0 con 0x55d6dea63f20 2024-09-10T08:50:47.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.763+0000 7fa848a1f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401ae4f0 con 0x55d6dea73650 2024-09-10T08:50:47.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.763+0000 7fa848a1f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401acfc0 con 0x55d6dea64840 2024-09-10T08:50:47.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.763+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea63f20 0x55d6dea660d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:47.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.763+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea63f20 0x55d6dea660d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35776/0 (socket says 172.21.15.172:35776) 2024-09-10T08:50:47.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.764+0000 7fa83ffff640 1 -- 172.21.15.172:0/765825434 learned_addr learned my addr 172.21.15.172:0/765825434 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:47.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.764+0000 7fa837fff640 1 --2- 172.21.15.172:0/765825434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea64840 0x55d6dea66610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:47.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.764+0000 7fa844be3640 1 --2- 172.21.15.172:0/765825434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 0x55d6dea8ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:47.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.764+0000 7fa83ffff640 1 -- 172.21.15.172:0/765825434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea64840 msgr2=0x55d6dea66610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:47.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.764+0000 7fa83ffff640 1 --2- 172.21.15.172:0/765825434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea64840 0x55d6dea66610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.764+0000 7fa83ffff640 1 -- 172.21.15.172:0/765825434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 msgr2=0x55d6dea8ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:47.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.764+0000 7fa83ffff640 1 --2- 172.21.15.172:0/765825434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 0x55d6dea8ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.764+0000 7fa83ffff640 1 -- 172.21.15.172:0/765825434 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6dea64670 con 0x55d6dea63f20 2024-09-10T08:50:47.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.764+0000 7fa83ffff640 1 --2- 172.21.15.172:0/765825434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea63f20 0x55d6dea660d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa830097a70 tx=0x7fa830099b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bf8d4f96af359a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:47.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.764+0000 7fa835ffb640 1 -- 172.21.15.172:0/765825434 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83001d070 con 0x55d6dea63f20 2024-09-10T08:50:47.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.764+0000 7fa835ffb640 1 -- 172.21.15.172:0/765825434 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa83009bd10 con 0x55d6dea63f20 2024-09-10T08:50:47.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.764+0000 7fa835ffb640 1 -- 172.21.15.172:0/765825434 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830084ba0 con 0x55d6dea63f20 2024-09-10T08:50:47.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.765+0000 7fa848a1f740 1 -- 172.21.15.172:0/765825434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea63f20 msgr2=0x55d6dea660d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:47.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.765+0000 7fa848a1f740 1 --2- 172.21.15.172:0/765825434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea63f20 0x55d6dea660d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa830097a70 tx=0x7fa830099b20 comp rx=0 tx=0).stop 2024-09-10T08:50:47.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.765+0000 7fa848a1f740 1 -- 172.21.15.172:0/765825434 shutdown_connections 2024-09-10T08:50:47.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.765+0000 7fa848a1f740 1 --2- 172.21.15.172:0/765825434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 0x55d6dea8ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.765+0000 7fa848a1f740 1 --2- 172.21.15.172:0/765825434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea64840 0x55d6dea66610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.765+0000 7fa848a1f740 1 --2- 172.21.15.172:0/765825434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea63f20 0x55d6dea660d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.765+0000 7fa848a1f740 1 -- 172.21.15.172:0/765825434 >> 172.21.15.172:0/765825434 conn(0x55d6dea605f0 msgr2=0x55d6dea659c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:47.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.765+0000 7fa848a1f740 1 -- 172.21.15.172:0/765825434 shutdown_connections 2024-09-10T08:50:47.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.765+0000 7fa848a1f740 1 -- 172.21.15.172:0/765825434 wait complete. 2024-09-10T08:50:47.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.765+0000 7fa848a1f740 1 -- start start 2024-09-10T08:50:47.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.765+0000 7fa848a1f740 1 Processor -- start 2024-09-10T08:50:47.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa848a1f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea63f20 0x55d6dea77f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa848a1f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea64840 0x55d6dea78460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa848a1f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 0x55d6deae5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa848a1f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401add90 con 0x55d6dea64840 2024-09-10T08:50:47.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa848a1f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401ad6c0 con 0x55d6dea73650 2024-09-10T08:50:47.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa848a1f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401aa8b0 con 0x55d6dea63f20 2024-09-10T08:50:47.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa848a1f740 10 client.0 _ll_register_callbacks cb 0x7fa83009b4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:47.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa837fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea64840 0x55d6dea78460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:47.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa837fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea64840 0x55d6dea78460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35780/0 (socket says 172.21.15.172:35780) 2024-09-10T08:50:47.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa837fff640 1 -- 172.21.15.172:0/2725348355 learned_addr learned my addr 172.21.15.172:0/2725348355 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:47.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa83ffff640 1 --2- 172.21.15.172:0/2725348355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea63f20 0x55d6dea77f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:47.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa837fff640 1 -- 172.21.15.172:0/2725348355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea63f20 msgr2=0x55d6dea77f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:47.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa837fff640 1 --2- 172.21.15.172:0/2725348355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea63f20 0x55d6dea77f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa837fff640 1 -- 172.21.15.172:0/2725348355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 msgr2=0x55d6deae5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:47.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa837fff640 1 --2- 172.21.15.172:0/2725348355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 0x55d6deae5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa837fff640 1 -- 172.21.15.172:0/2725348355 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6deae6290 con 0x55d6dea64840 2024-09-10T08:50:47.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa837fff640 1 --2- 172.21.15.172:0/2725348355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea64840 0x55d6dea78460 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa82802a9a0 tx=0x7fa828031610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e0e7586b6632a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:47.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa81f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:47.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa81f7fe640 1 -- 172.21.15.172:0/2725348355 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828035040 con 0x55d6dea64840 2024-09-10T08:50:47.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa81f7fe640 1 -- 172.21.15.172:0/2725348355 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa828004230 con 0x55d6dea64840 2024-09-10T08:50:47.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.766+0000 7fa848a1f740 1 -- 172.21.15.172:0/2725348355 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa83009bd10 con 0x55d6dea64840 2024-09-10T08:50:47.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.767+0000 7fa81f7fe640 1 -- 172.21.15.172:0/2725348355 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa828055050 con 0x55d6dea64840 2024-09-10T08:50:47.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.767+0000 7fa848a1f740 10 client.16746 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:47.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.767+0000 7fa848a1f740 1 -- 172.21.15.172:0/2725348355 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d6deaee7c0 con 0x55d6dea64840 2024-09-10T08:50:47.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.767+0000 7fa81f7fe640 1 -- 172.21.15.172:0/2725348355 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa83009bd10 con 0x55d6dea64840 2024-09-10T08:50:47.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.767+0000 7fa81f7fe640 1 -- 172.21.15.172:0/2725348355 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fa828004a60 con 0x55d6dea64840 2024-09-10T08:50:47.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.767+0000 7fa848a1f740 10 client.16746 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:47.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.767+0000 7fa848a1f740 10 client.16746 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:50:47.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.767+0000 7fa848a1f740 20 client.16746 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:47.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.767+0000 7fa848a1f740 1 --2- 172.21.15.172:0/2725348355 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d6deaeefb0 0x55d6deaf13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:47.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.767+0000 7fa848a1f740 4 client.16746 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5e6021f1", "release": true, "await": true}] 2024-09-10T08:50:47.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.767+0000 7fa848a1f740 1 -- 172.21.15.172:0/2725348355 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5e6021f1", "release": true, "await": true}) -- 0x55d6de9d10a0 con 0x55d6deaeefb0 2024-09-10T08:50:47.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.767+0000 7fa83ffff640 1 --2- 172.21.15.172:0/2725348355 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d6deaeefb0 0x55d6deaf13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:47.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.768+0000 7fa83ffff640 1 --2- 172.21.15.172:0/2725348355 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d6deaeefb0 0x55d6deaf13d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a304171eb705feb8 server_cookie=40e0e274d4e45c52 in_seq=0 out_seq=0 2024-09-10T08:50:47.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.768+0000 7fa81f7fe640 10 client.16746 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:47.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.781+0000 7fa81f7fe640 1 -- 172.21.15.172:0/2725348355 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d6de9d10a0 con 0x55d6deaeefb0 2024-09-10T08:50:47.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.781+0000 7fa81f7fe640 10 client.16746 handle_command_reply: tid=0 2024-09-10T08:50:47.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.781+0000 7fa848a1f740 1 client.16746 shutdown 2024-09-10T08:50:47.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.781+0000 7fa848a1f740 1 -- 172.21.15.172:0/2725348355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea64840 msgr2=0x55d6dea78460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:47.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.781+0000 7fa848a1f740 1 --2- 172.21.15.172:0/2725348355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea64840 0x55d6dea78460 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa82802a9a0 tx=0x7fa828031610 comp rx=0 tx=0).stop 2024-09-10T08:50:47.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.781+0000 7fa848a1f740 1 -- 172.21.15.172:0/2725348355 shutdown_connections 2024-09-10T08:50:47.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.781+0000 7fa848a1f740 1 --2- 172.21.15.172:0/2725348355 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d6deaeefb0 0x55d6deaf13d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.781+0000 7fa848a1f740 1 --2- 172.21.15.172:0/2725348355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d6dea73650 0x55d6deae5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.781+0000 7fa848a1f740 1 --2- 172.21.15.172:0/2725348355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d6dea64840 0x55d6dea78460 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.781+0000 7fa848a1f740 1 --2- 172.21.15.172:0/2725348355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6dea63f20 0x55d6dea77f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.781+0000 7fa848a1f740 1 -- 172.21.15.172:0/2725348355 >> 172.21.15.172:0/2725348355 conn(0x55d6dea605f0 msgr2=0x55d6dea70c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:47.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.782+0000 7fa81f7fe640 0 client.16746 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:47.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.782+0000 7fa848a1f740 1 -- 172.21.15.172:0/2725348355 shutdown_connections 2024-09-10T08:50:47.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.782+0000 7fa848a1f740 1 -- 172.21.15.172:0/2725348355 wait complete. 2024-09-10T08:50:47.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.782+0000 7fa848a1f740 20 client.16746 trim_cache size 0 max 16384 2024-09-10T08:50:47.784 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:50:47.784 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 33, 2024-09-10T08:50:47.784 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:50:47.784 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 12, 2024-09-10T08:50:47.784 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:50:47.784 INFO:teuthology.orchestra.run.smithi172.stdout: "5e6021f1": { 2024-09-10T08:50:47.784 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 12, 2024-09-10T08:50:47.784 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:50:47.784 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:50:47.785 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:50:47.785 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:50:47.785 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:50:47.785 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:50:47.785 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 142.8, 2024-09-10T08:50:47.785 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:50:47.785 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:50:47.785 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:50:47.785 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:50:47.785 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:50:47.785 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:50:47.785 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:50:47.785 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:50:47.786 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:50:47.786 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:50:47.786 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:50:47.786 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:50:47.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.782+0000 7fa846646640 1 -- 172.21.15.172:0/2091741014 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa82c0a2b80 msgr2=0x7fa82c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:47.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.782+0000 7fa846646640 1 --2- 172.21.15.172:0/2091741014 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa82c0a2b80 0x7fa82c0a5040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa838005cc0 tx=0x7fa838004ab0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.782+0000 7fa846646640 1 -- 172.21.15.172:0/2091741014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa840102d40 msgr2=0x7fa840198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:47.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.782+0000 7fa846646640 1 --2- 172.21.15.172:0/2091741014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa840102d40 0x7fa840198320 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa83000c890 tx=0x7fa83000cd50 comp rx=0 tx=0).stop 2024-09-10T08:50:47.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.782+0000 7fa846646640 1 -- 172.21.15.172:0/2091741014 shutdown_connections 2024-09-10T08:50:47.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.782+0000 7fa846646640 1 --2- 172.21.15.172:0/2091741014 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa82c0a2b80 0x7fa82c0a5040 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.782+0000 7fa846646640 1 --2- 172.21.15.172:0/2091741014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa84010c170 0x7fa84019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.783+0000 7fa846646640 1 --2- 172.21.15.172:0/2091741014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa840103660 0x7fa840198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.783+0000 7fa846646640 1 --2- 172.21.15.172:0/2091741014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa840102d40 0x7fa840198320 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:47.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.783+0000 7fa846646640 1 -- 172.21.15.172:0/2091741014 >> 172.21.15.172:0/2091741014 conn(0x7fa8400fc7a0 msgr2=0x7fa8400fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:47.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.783+0000 7fa846646640 1 -- 172.21.15.172:0/2091741014 shutdown_connections 2024-09-10T08:50:47.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:47.783+0000 7fa846646640 1 -- 172.21.15.172:0/2091741014 wait complete. 2024-09-10T08:50:48.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:48 smithi172 ceph-mon[30854]: pgmap v487: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 316 KiB/s wr, 12 op/s 2024-09-10T08:50:48.660 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5e6021f1', total seconds elapsed: 26.0 2024-09-10T08:50:48.661 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T08:50:48.661 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:50:48.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:48 smithi174 ceph-mon[34068]: pgmap v487: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 316 KiB/s wr, 12 op/s 2024-09-10T08:50:48.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:48 smithi191 ceph-mon[35314]: pgmap v487: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 316 KiB/s wr, 12 op/s 2024-09-10T08:50:48.950 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:50:49.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.436+0000 7f96109e0640 1 -- 172.21.15.172:0/4225851229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f960c108290 msgr2=0x7f960c10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:49.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.436+0000 7f96109e0640 1 --2- 172.21.15.172:0/4225851229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f960c108290 0x7f960c10ec30 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9604009cd0 tx=0x7f960402a8e0 comp rx=0 tx=0).stop 2024-09-10T08:50:49.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.436+0000 7f96109e0640 1 -- 172.21.15.172:0/4225851229 shutdown_connections 2024-09-10T08:50:49.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.436+0000 7f96109e0640 1 --2- 172.21.15.172:0/4225851229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f960c108290 0x7f960c10ec30 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:49.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.436+0000 7f96109e0640 1 --2- 172.21.15.172:0/4225851229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f960c103740 0x7f960c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:49.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.436+0000 7f96109e0640 1 --2- 172.21.15.172:0/4225851229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f960c102d90 0x7f960c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:49.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.436+0000 7f96109e0640 1 -- 172.21.15.172:0/4225851229 >> 172.21.15.172:0/4225851229 conn(0x7f960c0fc7d0 msgr2=0x7f960c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:49.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.436+0000 7f96109e0640 1 -- 172.21.15.172:0/4225851229 shutdown_connections 2024-09-10T08:50:49.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.436+0000 7f96109e0640 1 -- 172.21.15.172:0/4225851229 wait complete. 2024-09-10T08:50:49.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.437+0000 7f96109e0640 1 Processor -- start 2024-09-10T08:50:49.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.437+0000 7f96109e0640 1 -- start start 2024-09-10T08:50:49.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.437+0000 7f96109e0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f960c102d90 0x7f960c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:49.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.437+0000 7f96109e0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f960c103740 0x7f960c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:49.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f96109e0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f960c108290 0x7f960c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:49.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f96109e0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f960c111f40 con 0x7f960c102d90 2024-09-10T08:50:49.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f96109e0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f960c111dc0 con 0x7f960c108290 2024-09-10T08:50:49.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f96109e0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1120c0 con 0x7f960c103740 2024-09-10T08:50:49.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f960a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f960c102d90 0x7f960c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:49.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f960a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f960c102d90 0x7f960c198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35808/0 (socket says 172.21.15.172:35808) 2024-09-10T08:50:49.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f960a59c640 1 -- 172.21.15.172:0/745471005 learned_addr learned my addr 172.21.15.172:0/745471005 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:49.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f9609d9b640 1 --2- 172.21.15.172:0/745471005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f960c103740 0x7f960c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:49.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f960ad9d640 1 --2- 172.21.15.172:0/745471005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f960c108290 0x7f960c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:49.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f960a59c640 1 -- 172.21.15.172:0/745471005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f960c103740 msgr2=0x7f960c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:49.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f960a59c640 1 --2- 172.21.15.172:0/745471005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f960c103740 0x7f960c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:49.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f960a59c640 1 -- 172.21.15.172:0/745471005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f960c108290 msgr2=0x7f960c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:49.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f960a59c640 1 --2- 172.21.15.172:0/745471005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f960c108290 0x7f960c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:49.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f960a59c640 1 -- 172.21.15.172:0/745471005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c19d410 con 0x7f960c102d90 2024-09-10T08:50:49.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f960a59c640 1 --2- 172.21.15.172:0/745471005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f960c102d90 0x7f960c198460 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00d770 tx=0x7f95fc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:49.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f95fb7fe640 1 -- 172.21.15.172:0/745471005 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc017070 con 0x7f960c102d90 2024-09-10T08:50:49.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f95fb7fe640 1 -- 172.21.15.172:0/745471005 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95fc0054d0 con 0x7f960c102d90 2024-09-10T08:50:49.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f96109e0640 1 -- 172.21.15.172:0/745471005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960c19d700 con 0x7f960c102d90 2024-09-10T08:50:49.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.438+0000 7f95fb7fe640 1 -- 172.21.15.172:0/745471005 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc004cf0 con 0x7f960c102d90 2024-09-10T08:50:49.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.439+0000 7f96109e0640 1 -- 172.21.15.172:0/745471005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960c1a4fe0 con 0x7f960c102d90 2024-09-10T08:50:49.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.440+0000 7f96109e0640 1 -- 172.21.15.172:0/745471005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f960c103170 con 0x7f960c102d90 2024-09-10T08:50:49.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.442+0000 7f95fb7fe640 1 -- 172.21.15.172:0/745471005 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f95fc004e90 con 0x7f960c102d90 2024-09-10T08:50:49.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.442+0000 7f95fb7fe640 1 --2- 172.21.15.172:0/745471005 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f95f40a2c00 0x7f95f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:49.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.443+0000 7f95fb7fe640 1 -- 172.21.15.172:0/745471005 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f95fc0cf080 con 0x7f960c102d90 2024-09-10T08:50:49.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.443+0000 7f9609d9b640 1 --2- 172.21.15.172:0/745471005 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f95f40a2c00 0x7f95f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:49.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.443+0000 7f9609d9b640 1 --2- 172.21.15.172:0/745471005 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f95f40a2c00 0x7f95f40a50c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f960c0fee30 tx=0x7f960000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:49.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.446+0000 7f95fb7fe640 1 -- 172.21.15.172:0/745471005 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95fc09b6b0 con 0x7f960c102d90 2024-09-10T08:50:49.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.692+0000 7f96109e0640 1 -- 172.21.15.172:0/745471005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f960c10a2a0 con 0x7f960c102d90 2024-09-10T08:50:49.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.693+0000 7f95fb7fe640 1 -- 172.21.15.172:0/745471005 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+12007 (secure 0 0 0) 0x7f95fc09f9b0 con 0x7f960c102d90 2024-09-10T08:50:49.694 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:50:49.694 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":33,"btime":"2024-09-10T08:50:03:285217+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:50:02.654978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:50:49.694 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 33 2024-09-10T08:50:49.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.695+0000 7f96109e0640 1 -- 172.21.15.172:0/745471005 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f95f40a2c00 msgr2=0x7f95f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:49.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.695+0000 7f96109e0640 1 --2- 172.21.15.172:0/745471005 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f95f40a2c00 0x7f95f40a50c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f960c0fee30 tx=0x7f960000a2d0 comp rx=0 tx=0).stop 2024-09-10T08:50:49.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.695+0000 7f96109e0640 1 -- 172.21.15.172:0/745471005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f960c102d90 msgr2=0x7f960c198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:49.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.695+0000 7f96109e0640 1 --2- 172.21.15.172:0/745471005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f960c102d90 0x7f960c198460 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00d770 tx=0x7f95fc00dc30 comp rx=0 tx=0).stop 2024-09-10T08:50:49.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.696+0000 7f96109e0640 1 -- 172.21.15.172:0/745471005 shutdown_connections 2024-09-10T08:50:49.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.696+0000 7f96109e0640 1 --2- 172.21.15.172:0/745471005 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f95f40a2c00 0x7f95f40a50c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:49.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.696+0000 7f96109e0640 1 --2- 172.21.15.172:0/745471005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f960c108290 0x7f960c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:49.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.696+0000 7f96109e0640 1 --2- 172.21.15.172:0/745471005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f960c103740 0x7f960c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:49.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.696+0000 7f96109e0640 1 --2- 172.21.15.172:0/745471005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f960c102d90 0x7f960c198460 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:49.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.696+0000 7f96109e0640 1 -- 172.21.15.172:0/745471005 >> 172.21.15.172:0/745471005 conn(0x7f960c0fc7d0 msgr2=0x7f960c105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:49.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.696+0000 7f96109e0640 1 -- 172.21.15.172:0/745471005 shutdown_connections 2024-09-10T08:50:49.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:49.696+0000 7f96109e0640 1 -- 172.21.15.172:0/745471005 wait complete. 2024-09-10T08:50:50.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:50 smithi172 ceph-mon[30854]: pgmap v488: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 343 KiB/s wr, 11 op/s 2024-09-10T08:50:50.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:50 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/745471005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:50:50.497 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T08:50:50.497 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T08:50:50.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:50 smithi174 ceph-mon[34068]: pgmap v488: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 343 KiB/s wr, 11 op/s 2024-09-10T08:50:50.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:50 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/745471005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:50:50.784 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:50:50.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:50 smithi191 ceph-mon[35314]: pgmap v488: 129 pgs: 129 active+clean; 2.9 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 343 KiB/s wr, 11 op/s 2024-09-10T08:50:50.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:50 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/745471005' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:50:51.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.273+0000 7fb67fb19640 1 -- 172.21.15.172:0/4090114756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb678102d30 msgr2=0x7fb678103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:51.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.273+0000 7fb67fb19640 1 --2- 172.21.15.172:0/4090114756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb678102d30 0x7fb678103110 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb674009930 tx=0x7fb67402a420 comp rx=0 tx=0).stop 2024-09-10T08:50:51.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.273+0000 7fb67fb19640 1 -- 172.21.15.172:0/4090114756 shutdown_connections 2024-09-10T08:50:51.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.273+0000 7fb67fb19640 1 --2- 172.21.15.172:0/4090114756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6781082a0 0x7fb67810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.273+0000 7fb67fb19640 1 --2- 172.21.15.172:0/4090114756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6781036e0 0x7fb678107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.273+0000 7fb67fb19640 1 --2- 172.21.15.172:0/4090114756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb678102d30 0x7fb678103110 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.273+0000 7fb67fb19640 1 -- 172.21.15.172:0/4090114756 >> 172.21.15.172:0/4090114756 conn(0x7fb6780fc7d0 msgr2=0x7fb6780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:51.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.273+0000 7fb67fb19640 1 -- 172.21.15.172:0/4090114756 shutdown_connections 2024-09-10T08:50:51.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.273+0000 7fb67fb19640 1 -- 172.21.15.172:0/4090114756 wait complete. 2024-09-10T08:50:51.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.274+0000 7fb67fb19640 1 Processor -- start 2024-09-10T08:50:51.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.274+0000 7fb67fb19640 1 -- start start 2024-09-10T08:50:51.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.274+0000 7fb67fb19640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb678102d30 0x7fb6781983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.274+0000 7fb67fb19640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6781036e0 0x7fb6781988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.274+0000 7fb67fb19640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6781082a0 0x7fb67819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.274+0000 7fb67fb19640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb678111f50 con 0x7fb678102d30 2024-09-10T08:50:51.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.274+0000 7fb67fb19640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb678111dd0 con 0x7fb6781036e0 2024-09-10T08:50:51.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.274+0000 7fb67fb19640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb6781120d0 con 0x7fb6781082a0 2024-09-10T08:50:51.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.274+0000 7fb67d8b5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb678102d30 0x7fb6781983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.274+0000 7fb67d8b5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb678102d30 0x7fb6781983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35832/0 (socket says 172.21.15.172:35832) 2024-09-10T08:50:51.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.274+0000 7fb67d8b5640 1 -- 172.21.15.172:0/3769314298 learned_addr learned my addr 172.21.15.172:0/3769314298 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:51.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.274+0000 7fb67d0b4640 1 --2- 172.21.15.172:0/3769314298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6781036e0 0x7fb6781988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.274+0000 7fb67e0b6640 1 --2- 172.21.15.172:0/3769314298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6781082a0 0x7fb67819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.275+0000 7fb67d8b5640 1 -- 172.21.15.172:0/3769314298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6781082a0 msgr2=0x7fb67819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:51.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.275+0000 7fb67d8b5640 1 --2- 172.21.15.172:0/3769314298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6781082a0 0x7fb67819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.275+0000 7fb67d8b5640 1 -- 172.21.15.172:0/3769314298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6781036e0 msgr2=0x7fb6781988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:51.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.275+0000 7fb67d8b5640 1 --2- 172.21.15.172:0/3769314298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6781036e0 0x7fb6781988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.275+0000 7fb67d8b5640 1 -- 172.21.15.172:0/3769314298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67819d350 con 0x7fb678102d30 2024-09-10T08:50:51.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.275+0000 7fb67d8b5640 1 --2- 172.21.15.172:0/3769314298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb678102d30 0x7fb6781983a0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb674005940 tx=0x7fb6740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:51.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.275+0000 7fb66effd640 1 -- 172.21.15.172:0/3769314298 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67403f030 con 0x7fb678102d30 2024-09-10T08:50:51.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.275+0000 7fb66effd640 1 -- 172.21.15.172:0/3769314298 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb674003cb0 con 0x7fb678102d30 2024-09-10T08:50:51.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.275+0000 7fb66effd640 1 -- 172.21.15.172:0/3769314298 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674035070 con 0x7fb678102d30 2024-09-10T08:50:51.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.275+0000 7fb67fb19640 1 -- 172.21.15.172:0/3769314298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb67819d5e0 con 0x7fb678102d30 2024-09-10T08:50:51.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.275+0000 7fb67fb19640 1 -- 172.21.15.172:0/3769314298 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6781a4f20 con 0x7fb678102d30 2024-09-10T08:50:51.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.277+0000 7fb66effd640 1 -- 172.21.15.172:0/3769314298 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb674042030 con 0x7fb678102d30 2024-09-10T08:50:51.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.278+0000 7fb681ef2740 1 Processor -- start 2024-09-10T08:50:51.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.278+0000 7fb681ef2740 1 -- start start 2024-09-10T08:50:51.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.278+0000 7fb681ef2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a028c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.278+0000 7fb681ef2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a0334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.278+0000 7fb681ef2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a035e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.278+0000 7fb681ef2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559179f8d160 con 0x55917a029170 2024-09-10T08:50:51.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.278+0000 7fb681ef2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559179f34720 con 0x55917a033a30 2024-09-10T08:50:51.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.278+0000 7fb681ef2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559179f89120 con 0x55917a028850 2024-09-10T08:50:51.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.278+0000 7fb67d8b5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a0334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.278+0000 7fb67d8b5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a0334f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35846/0 (socket says 172.21.15.172:35846) 2024-09-10T08:50:51.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.278+0000 7fb67d8b5640 1 -- 172.21.15.172:0/4287984927 learned_addr learned my addr 172.21.15.172:0/4287984927 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:51.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.279+0000 7fb67e0b6640 1 --2- 172.21.15.172:0/4287984927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a028c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.279+0000 7fb66effd640 1 --2- 172.21.15.172:0/3769314298 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb6640a2b80 0x7fb6640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.279+0000 7fb67d0b4640 1 --2- 172.21.15.172:0/4287984927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a035e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.279+0000 7fb67d8b5640 1 -- 172.21.15.172:0/4287984927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 msgr2=0x55917a028c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:51.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.279+0000 7fb67d8b5640 1 --2- 172.21.15.172:0/4287984927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a028c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.279+0000 7fb67d8b5640 1 -- 172.21.15.172:0/4287984927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 msgr2=0x55917a035e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:51.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.279+0000 7fb67d8b5640 1 --2- 172.21.15.172:0/4287984927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a035e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.279+0000 7fb67e0b6640 1 --2- 172.21.15.172:0/3769314298 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb6640a2b80 0x7fb6640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.279+0000 7fb67d8b5640 1 -- 172.21.15.172:0/4287984927 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559179cfeba0 con 0x55917a029170 2024-09-10T08:50:51.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.279+0000 7fb67d0b4640 1 --2- 172.21.15.172:0/4287984927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a035e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:51.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.279+0000 7fb66effd640 1 -- 172.21.15.172:0/3769314298 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb6740ef5c0 con 0x7fb678102d30 2024-09-10T08:50:51.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.279+0000 7fb67d8b5640 1 --2- 172.21.15.172:0/4287984927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a0334f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb67407b210 tx=0x7fb67407bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2066829e52794034 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:51.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.279+0000 7fb663fff640 1 -- 172.21.15.172:0/4287984927 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6740bb600 con 0x55917a029170 2024-09-10T08:50:51.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.279+0000 7fb67e0b6640 1 --2- 172.21.15.172:0/3769314298 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb6640a2b80 0x7fb6640a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb670007f00 tx=0x7fb670004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:51.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.279+0000 7fb663fff640 1 -- 172.21.15.172:0/4287984927 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6740bc1c0 con 0x55917a029170 2024-09-10T08:50:51.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.279+0000 7fb663fff640 1 -- 172.21.15.172:0/4287984927 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674003e60 con 0x55917a029170 2024-09-10T08:50:51.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.280+0000 7fb681ef2740 1 -- 172.21.15.172:0/4287984927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 msgr2=0x55917a0334f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:51.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.280+0000 7fb681ef2740 1 --2- 172.21.15.172:0/4287984927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a0334f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb67407b210 tx=0x7fb67407bb30 comp rx=0 tx=0).stop 2024-09-10T08:50:51.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.280+0000 7fb681ef2740 1 -- 172.21.15.172:0/4287984927 shutdown_connections 2024-09-10T08:50:51.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.280+0000 7fb681ef2740 1 --2- 172.21.15.172:0/4287984927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a035e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.280+0000 7fb681ef2740 1 --2- 172.21.15.172:0/4287984927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a0334f0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.280+0000 7fb681ef2740 1 --2- 172.21.15.172:0/4287984927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a028c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.280+0000 7fb681ef2740 1 -- 172.21.15.172:0/4287984927 >> 172.21.15.172:0/4287984927 conn(0x55917a024de0 msgr2=0x55917a0251c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:51.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.280+0000 7fb681ef2740 1 -- 172.21.15.172:0/4287984927 shutdown_connections 2024-09-10T08:50:51.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.280+0000 7fb681ef2740 1 -- 172.21.15.172:0/4287984927 wait complete. 2024-09-10T08:50:51.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.281+0000 7fb681ef2740 1 -- start start 2024-09-10T08:50:51.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.281+0000 7fb681ef2740 1 Processor -- start 2024-09-10T08:50:51.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.281+0000 7fb681ef2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a030890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.281+0000 7fb681ef2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a030dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.281+0000 7fb681ef2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a031310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.282+0000 7fb681ef2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55917a0363c0 con 0x55917a029170 2024-09-10T08:50:51.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.282+0000 7fb67d8b5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a030890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.282+0000 7fb681ef2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559179f15900 con 0x55917a033a30 2024-09-10T08:50:51.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.282+0000 7fb681ef2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55917a036540 con 0x55917a028850 2024-09-10T08:50:51.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.282+0000 7fb67d0b4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a030dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.282+0000 7fb67d8b5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a030890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:49118/0 (socket says 172.21.15.172:49118) 2024-09-10T08:50:51.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.282+0000 7fb67d8b5640 1 -- 172.21.15.172:0/2145768250 learned_addr learned my addr 172.21.15.172:0/2145768250 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:51.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.282+0000 7fb67e0b6640 1 --2- 172.21.15.172:0/2145768250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a031310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.282+0000 7fb681ef2740 10 client.0 _ll_register_callbacks cb 0x559179f13780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:51.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.282+0000 7fb67e0b6640 1 -- 172.21.15.172:0/2145768250 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 msgr2=0x55917a030890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:51.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.282+0000 7fb67e0b6640 1 --2- 172.21.15.172:0/2145768250 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a030890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.282+0000 7fb67e0b6640 1 -- 172.21.15.172:0/2145768250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 msgr2=0x55917a030dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:51.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.282+0000 7fb67e0b6640 1 --2- 172.21.15.172:0/2145768250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a030dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.282+0000 7fb67e0b6640 1 -- 172.21.15.172:0/2145768250 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559179f55c50 con 0x55917a033a30 2024-09-10T08:50:51.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.282+0000 7fb67d0b4640 1 --2- 172.21.15.172:0/2145768250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a030dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:50:51.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.282+0000 7fb67e0b6640 1 --2- 172.21.15.172:0/2145768250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a031310 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb670012e50 tx=0x7fb670007470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ea3244a56bc04b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:51.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.282+0000 7fb6617fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:50:51.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.283+0000 7fb6617fa640 1 -- 172.21.15.172:0/2145768250 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670022070 con 0x55917a033a30 2024-09-10T08:50:51.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.283+0000 7fb681ef2740 1 -- 172.21.15.172:0/2145768250 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6740bc1c0 con 0x55917a033a30 2024-09-10T08:50:51.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.283+0000 7fb6617fa640 1 -- 172.21.15.172:0/2145768250 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb67000fdd0 con 0x55917a033a30 2024-09-10T08:50:51.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.283+0000 7fb6617fa640 1 -- 172.21.15.172:0/2145768250 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb67002e050 con 0x55917a033a30 2024-09-10T08:50:51.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.283+0000 7fb681ef2740 10 client.26048 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:51.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.283+0000 7fb681ef2740 1 -- 172.21.15.172:0/2145768250 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55917a041f70 con 0x55917a033a30 2024-09-10T08:50:51.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.283+0000 7fb6617fa640 1 -- 172.21.15.172:0/2145768250 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6740bc1c0 con 0x55917a033a30 2024-09-10T08:50:51.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.283+0000 7fb6617fa640 1 -- 172.21.15.172:0/2145768250 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fb6700103d0 con 0x55917a033a30 2024-09-10T08:50:51.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.283+0000 7fb681ef2740 10 client.26048 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:51.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.283+0000 7fb681ef2740 10 client.26048 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:50:51.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.283+0000 7fb681ef2740 20 client.26048 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:51.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.283+0000 7fb681ef2740 1 --2- 172.21.15.172:0/2145768250 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55917a043440 0x55917a0458c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.283+0000 7fb681ef2740 4 client.26048 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:50:51.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.283+0000 7fb681ef2740 1 -- 172.21.15.172:0/2145768250 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559179f9a890 con 0x55917a043440 2024-09-10T08:50:51.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.284+0000 7fb67d8b5640 1 --2- 172.21.15.172:0/2145768250 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55917a043440 0x55917a0458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.284+0000 7fb6617fa640 10 client.26048 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:51.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.284+0000 7fb67d8b5640 1 --2- 172.21.15.172:0/2145768250 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55917a043440 0x55917a0458c0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16239af99e98d0fb server_cookie=1a82d76a9e0e62c7 in_seq=0 out_seq=0 2024-09-10T08:50:51.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.285+0000 7fb6617fa640 1 -- 172.21.15.172:0/2145768250 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559179f9a890 con 0x55917a043440 2024-09-10T08:50:51.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.285+0000 7fb6617fa640 10 client.26048 handle_command_reply: tid=0 2024-09-10T08:50:51.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.285+0000 7fb681ef2740 1 client.26048 shutdown 2024-09-10T08:50:51.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.285+0000 7fb681ef2740 1 -- 172.21.15.172:0/2145768250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 msgr2=0x55917a031310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:51.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.285+0000 7fb681ef2740 1 --2- 172.21.15.172:0/2145768250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a031310 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb670012e50 tx=0x7fb670007470 comp rx=0 tx=0).stop 2024-09-10T08:50:51.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.285+0000 7fb681ef2740 1 -- 172.21.15.172:0/2145768250 shutdown_connections 2024-09-10T08:50:51.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.285+0000 7fb681ef2740 1 --2- 172.21.15.172:0/2145768250 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55917a043440 0x55917a0458c0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.285+0000 7fb681ef2740 1 --2- 172.21.15.172:0/2145768250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a031310 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.285+0000 7fb6617fa640 0 client.26048 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:51.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.285+0000 7fb681ef2740 1 --2- 172.21.15.172:0/2145768250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a030dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.285+0000 7fb681ef2740 1 --2- 172.21.15.172:0/2145768250 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a030890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.285+0000 7fb681ef2740 1 -- 172.21.15.172:0/2145768250 >> 172.21.15.172:0/2145768250 conn(0x55917a024de0 msgr2=0x55917a02ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:51.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.285+0000 7fb681ef2740 1 -- 172.21.15.172:0/2145768250 shutdown_connections 2024-09-10T08:50:51.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.285+0000 7fb681ef2740 1 -- 172.21.15.172:0/2145768250 wait complete. 2024-09-10T08:50:51.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.285+0000 7fb681ef2740 20 client.26048 trim_cache size 0 max 16384 2024-09-10T08:50:51.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.305+0000 7fb681ef2740 1 Processor -- start 2024-09-10T08:50:51.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.305+0000 7fb681ef2740 1 -- start start 2024-09-10T08:50:51.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.305+0000 7fb681ef2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a02a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.305+0000 7fb681ef2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a04b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.305+0000 7fb681ef2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a04b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.305+0000 7fb681ef2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb6781ac870 con 0x55917a029170 2024-09-10T08:50:51.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.305+0000 7fb681ef2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6781ae470 con 0x55917a033a30 2024-09-10T08:50:51.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.305+0000 7fb681ef2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb6781acf40 con 0x55917a028850 2024-09-10T08:50:51.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.305+0000 7fb67d0b4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a04b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.305+0000 7fb67d0b4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a04b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35870/0 (socket says 172.21.15.172:35870) 2024-09-10T08:50:51.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.305+0000 7fb67d0b4640 1 -- 172.21.15.172:0/2831775228 learned_addr learned my addr 172.21.15.172:0/2831775228 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:51.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.305+0000 7fb67e0b6640 1 --2- 172.21.15.172:0/2831775228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a04b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.305+0000 7fb67d8b5640 1 --2- 172.21.15.172:0/2831775228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a02a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.306+0000 7fb67d0b4640 1 -- 172.21.15.172:0/2831775228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 msgr2=0x55917a02a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:51.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.306+0000 7fb67d0b4640 1 --2- 172.21.15.172:0/2831775228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a02a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.306+0000 7fb67d0b4640 1 -- 172.21.15.172:0/2831775228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 msgr2=0x55917a04b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:51.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.306+0000 7fb67d0b4640 1 --2- 172.21.15.172:0/2831775228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a04b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.306+0000 7fb67d0b4640 1 -- 172.21.15.172:0/2831775228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917a024c10 con 0x55917a029170 2024-09-10T08:50:51.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.306+0000 7fb67d8b5640 1 --2- 172.21.15.172:0/2831775228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a02a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:51.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.306+0000 7fb67d0b4640 1 --2- 172.21.15.172:0/2831775228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a04b1d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb67806a370 tx=0x7fb66800a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f14792b35ed0d14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:51.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.306+0000 7fb6637fe640 1 -- 172.21.15.172:0/2831775228 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6680142e0 con 0x55917a029170 2024-09-10T08:50:51.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.306+0000 7fb6637fe640 1 -- 172.21.15.172:0/2831775228 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6680054d0 con 0x55917a029170 2024-09-10T08:50:51.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.306+0000 7fb6637fe640 1 -- 172.21.15.172:0/2831775228 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb668012e20 con 0x55917a029170 2024-09-10T08:50:51.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.307+0000 7fb681ef2740 1 -- 172.21.15.172:0/2831775228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 msgr2=0x55917a04b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:51.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.307+0000 7fb681ef2740 1 --2- 172.21.15.172:0/2831775228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a04b1d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb67806a370 tx=0x7fb66800a2d0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.307+0000 7fb681ef2740 1 -- 172.21.15.172:0/2831775228 shutdown_connections 2024-09-10T08:50:51.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.307+0000 7fb681ef2740 1 --2- 172.21.15.172:0/2831775228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a04b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.307+0000 7fb681ef2740 1 --2- 172.21.15.172:0/2831775228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a04b1d0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.307+0000 7fb681ef2740 1 --2- 172.21.15.172:0/2831775228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a02a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.307+0000 7fb681ef2740 1 -- 172.21.15.172:0/2831775228 >> 172.21.15.172:0/2831775228 conn(0x55917a024de0 msgr2=0x55917a02a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:51.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.307+0000 7fb681ef2740 1 -- 172.21.15.172:0/2831775228 shutdown_connections 2024-09-10T08:50:51.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.307+0000 7fb681ef2740 1 -- 172.21.15.172:0/2831775228 wait complete. 2024-09-10T08:50:51.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.307+0000 7fb681ef2740 1 -- start start 2024-09-10T08:50:51.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.307+0000 7fb681ef2740 1 Processor -- start 2024-09-10T08:50:51.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.308+0000 7fb681ef2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a0384b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.308+0000 7fb681ef2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a0389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.308+0000 7fb681ef2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a0ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.308+0000 7fb681ef2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb6781add10 con 0x55917a029170 2024-09-10T08:50:51.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.308+0000 7fb681ef2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6781ad640 con 0x55917a033a30 2024-09-10T08:50:51.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.308+0000 7fb681ef2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb6781aa870 con 0x55917a028850 2024-09-10T08:50:51.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.308+0000 7fb681ef2740 10 client.0 _ll_register_callbacks cb 0x7fb670007f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:51.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.308+0000 7fb67d8b5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a0384b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.308+0000 7fb67d8b5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a0384b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:49132/0 (socket says 172.21.15.172:49132) 2024-09-10T08:50:51.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.308+0000 7fb67d8b5640 1 -- 172.21.15.172:0/2154402297 learned_addr learned my addr 172.21.15.172:0/2154402297 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:51.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.308+0000 7fb67d0b4640 1 --2- 172.21.15.172:0/2154402297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a0389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.308+0000 7fb67e0b6640 1 --2- 172.21.15.172:0/2154402297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a0ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.308+0000 7fb67d8b5640 1 -- 172.21.15.172:0/2154402297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 msgr2=0x55917a0ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:51.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.308+0000 7fb67d8b5640 1 --2- 172.21.15.172:0/2154402297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a0ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.308+0000 7fb67d8b5640 1 -- 172.21.15.172:0/2154402297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 msgr2=0x55917a0389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:51.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.308+0000 7fb67d8b5640 1 --2- 172.21.15.172:0/2154402297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a0389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.308+0000 7fb67d8b5640 1 -- 172.21.15.172:0/2154402297 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917a0ad6e0 con 0x55917a028850 2024-09-10T08:50:51.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.309+0000 7fb67d0b4640 1 --2- 172.21.15.172:0/2154402297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a0389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:51.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.309+0000 7fb67d8b5640 1 --2- 172.21.15.172:0/2154402297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a0384b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb674009900 tx=0x7fb674004460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c7ef0b631e05dbb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:51.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.309+0000 7fb6617fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:50:51.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.309+0000 7fb6617fa640 1 -- 172.21.15.172:0/2154402297 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674049040 con 0x55917a028850 2024-09-10T08:50:51.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.309+0000 7fb681ef2740 1 -- 172.21.15.172:0/2154402297 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6680054d0 con 0x55917a028850 2024-09-10T08:50:51.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.309+0000 7fb6617fa640 1 -- 172.21.15.172:0/2154402297 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb6740b9b60 con 0x55917a028850 2024-09-10T08:50:51.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.309+0000 7fb6617fa640 1 -- 172.21.15.172:0/2154402297 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb6740bb420 con 0x55917a028850 2024-09-10T08:50:51.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.309+0000 7fb681ef2740 10 client.25981 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:51.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.309+0000 7fb681ef2740 1 -- 172.21.15.172:0/2154402297 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55917a0ae9f0 con 0x55917a028850 2024-09-10T08:50:51.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.309+0000 7fb6617fa640 1 -- 172.21.15.172:0/2154402297 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6680054d0 con 0x55917a028850 2024-09-10T08:50:51.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.309+0000 7fb6617fa640 1 -- 172.21.15.172:0/2154402297 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fb67404c060 con 0x55917a028850 2024-09-10T08:50:51.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.309+0000 7fb681ef2740 10 client.25981 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:51.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.309+0000 7fb681ef2740 10 client.25981 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:50:51.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.309+0000 7fb681ef2740 20 client.25981 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:51.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.309+0000 7fb681ef2740 1 --2- 172.21.15.172:0/2154402297 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55917a043440 0x55917a0b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:51.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.309+0000 7fb681ef2740 4 client.25981 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T08:50:51.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.310+0000 7fb681ef2740 1 -- 172.21.15.172:0/2154402297 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x55917a020410 con 0x55917a043440 2024-09-10T08:50:51.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.310+0000 7fb67d0b4640 1 --2- 172.21.15.172:0/2154402297 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55917a043440 0x55917a0b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:51.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.310+0000 7fb67d0b4640 1 --2- 172.21.15.172:0/2154402297 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55917a043440 0x55917a0b0f40 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b83dfc0879e5fe22 server_cookie=c3858086d55f536c in_seq=0 out_seq=0 2024-09-10T08:50:51.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.310+0000 7fb6617fa640 10 client.25981 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:51.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.383+0000 7fb6617fa640 1 -- 172.21.15.172:0/2154402297 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55917a020410 con 0x55917a043440 2024-09-10T08:50:51.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.383+0000 7fb6617fa640 10 client.25981 handle_command_reply: tid=0 2024-09-10T08:50:51.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.383+0000 7fb681ef2740 1 client.25981 shutdown 2024-09-10T08:50:51.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.383+0000 7fb681ef2740 1 -- 172.21.15.172:0/2154402297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 msgr2=0x55917a0384b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:51.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.383+0000 7fb681ef2740 1 --2- 172.21.15.172:0/2154402297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a0384b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb674009900 tx=0x7fb674004460 comp rx=0 tx=0).stop 2024-09-10T08:50:51.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb681ef2740 1 -- 172.21.15.172:0/2154402297 shutdown_connections 2024-09-10T08:50:51.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb681ef2740 1 --2- 172.21.15.172:0/2154402297 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55917a043440 0x55917a0b0f40 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb681ef2740 1 --2- 172.21.15.172:0/2154402297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a033a30 0x55917a0ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb6617fa640 0 client.25981 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:51.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb681ef2740 1 --2- 172.21.15.172:0/2154402297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55917a029170 0x55917a0389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb681ef2740 1 --2- 172.21.15.172:0/2154402297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55917a028850 0x55917a0384b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb681ef2740 1 -- 172.21.15.172:0/2154402297 >> 172.21.15.172:0/2154402297 conn(0x55917a024de0 msgr2=0x55917a032910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:51.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb681ef2740 1 -- 172.21.15.172:0/2154402297 shutdown_connections 2024-09-10T08:50:51.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb681ef2740 1 -- 172.21.15.172:0/2154402297 wait complete. 2024-09-10T08:50:51.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb681ef2740 20 client.25981 trim_cache size 0 max 16384 2024-09-10T08:50:51.386 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:50:51.386 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 33, 2024-09-10T08:50:51.386 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:50:51.386 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 14, 2024-09-10T08:50:51.387 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:50:51.387 INFO:teuthology.orchestra.run.smithi172.stdout: "8cccb995": { 2024-09-10T08:50:51.387 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 14, 2024-09-10T08:50:51.387 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:50:51.387 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:50:51.387 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:50:51.387 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:50:51.387 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:50:51.387 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:50:51.387 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 142.8, 2024-09-10T08:50:51.387 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:50:51.387 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:50:51.387 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:50:51.388 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:50:51.388 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:50:51.388 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:50:51.388 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:50:51.388 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:50:51.388 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:50:51.388 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:50:51.388 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:50:51.388 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:50:51.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb67fb19640 1 -- 172.21.15.172:0/3769314298 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb6640a2b80 msgr2=0x7fb6640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:51.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb67fb19640 1 --2- 172.21.15.172:0/3769314298 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb6640a2b80 0x7fb6640a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb670007f00 tx=0x7fb670004a40 comp rx=0 tx=0).stop 2024-09-10T08:50:51.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb67fb19640 1 -- 172.21.15.172:0/3769314298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb678102d30 msgr2=0x7fb6781983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:51.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb67fb19640 1 --2- 172.21.15.172:0/3769314298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb678102d30 0x7fb6781983a0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb674005940 tx=0x7fb6740042f0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb67fb19640 1 -- 172.21.15.172:0/3769314298 shutdown_connections 2024-09-10T08:50:51.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb67fb19640 1 --2- 172.21.15.172:0/3769314298 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb6640a2b80 0x7fb6640a5040 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb67fb19640 1 --2- 172.21.15.172:0/3769314298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6781082a0 0x7fb67819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb67fb19640 1 --2- 172.21.15.172:0/3769314298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6781036e0 0x7fb6781988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb67fb19640 1 --2- 172.21.15.172:0/3769314298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb678102d30 0x7fb6781983a0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:51.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.384+0000 7fb67fb19640 1 -- 172.21.15.172:0/3769314298 >> 172.21.15.172:0/3769314298 conn(0x7fb6780fc7d0 msgr2=0x7fb678105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:51.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.385+0000 7fb67fb19640 1 -- 172.21.15.172:0/3769314298 shutdown_connections 2024-09-10T08:50:51.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:51.385+0000 7fb67fb19640 1 -- 172.21.15.172:0/3769314298 wait complete. 2024-09-10T08:50:52.062 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8cccb995', quiesce took 3.4 seconds. Will release after: 19.35 2024-09-10T08:50:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:52 smithi172 ceph-mon[30854]: pgmap v489: 129 pgs: 129 active+clean; 3.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.0 MiB/s wr, 14 op/s 2024-09-10T08:50:52.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:52 smithi174 ceph-mon[34068]: pgmap v489: 129 pgs: 129 active+clean; 3.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.0 MiB/s wr, 14 op/s 2024-09-10T08:50:52.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:52 smithi191 ceph-mon[35314]: pgmap v489: 129 pgs: 129 active+clean; 3.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.0 MiB/s wr, 14 op/s 2024-09-10T08:50:54.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:54 smithi172 ceph-mon[30854]: pgmap v490: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.4 MiB/s wr, 12 op/s 2024-09-10T08:50:54.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:54 smithi174 ceph-mon[34068]: pgmap v490: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.4 MiB/s wr, 12 op/s 2024-09-10T08:50:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.796+0000 7fb59485e640 1 -- 172.21.15.172:0/2365317600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb590103710 msgr2=0x7fb590109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.796+0000 7fb59485e640 1 --2- 172.21.15.172:0/2365317600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb590103710 0x7fb590109fa0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb588009ca0 tx=0x7fb58802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.796+0000 7fb59485e640 1 -- 172.21.15.172:0/2365317600 shutdown_connections 2024-09-10T08:50:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.796+0000 7fb59485e640 1 --2- 172.21.15.172:0/2365317600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb590103710 0x7fb590109fa0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.796+0000 7fb59485e640 1 --2- 172.21.15.172:0/2365317600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590102d50 0x7fb5901031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.796+0000 7fb59485e640 1 --2- 172.21.15.172:0/2365317600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb590101b60 0x7fb590101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.796+0000 7fb59485e640 1 -- 172.21.15.172:0/2365317600 >> 172.21.15.172:0/2365317600 conn(0x7fb5900fd2e0 msgr2=0x7fb5900ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.796+0000 7fb59485e640 1 -- 172.21.15.172:0/2365317600 shutdown_connections 2024-09-10T08:50:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.796+0000 7fb59485e640 1 -- 172.21.15.172:0/2365317600 wait complete. 2024-09-10T08:50:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.797+0000 7fb59485e640 1 Processor -- start 2024-09-10T08:50:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.797+0000 7fb59485e640 1 -- start start 2024-09-10T08:50:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.797+0000 7fb59485e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb590101b60 0x7fb590197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.797+0000 7fb59485e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590102d50 0x7fb5901981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.797+0000 7fb59485e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb590103710 0x7fb59019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.797+0000 7fb59485e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb59010d670 con 0x7fb590101b60 2024-09-10T08:50:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.797+0000 7fb59485e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb59010d4f0 con 0x7fb590102d50 2024-09-10T08:50:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.797+0000 7fb59485e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb59010d7f0 con 0x7fb590103710 2024-09-10T08:50:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.798+0000 7fb58e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb590101b60 0x7fb590197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.798+0000 7fb58e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb590101b60 0x7fb590197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35894/0 (socket says 172.21.15.172:35894) 2024-09-10T08:50:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.798+0000 7fb58e59c640 1 -- 172.21.15.172:0/4240779335 learned_addr learned my addr 172.21.15.172:0/4240779335 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.798+0000 7fb58ed9d640 1 --2- 172.21.15.172:0/4240779335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb590103710 0x7fb59019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.798+0000 7fb587fff640 1 --2- 172.21.15.172:0/4240779335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590102d50 0x7fb5901981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.798+0000 7fb58e59c640 1 -- 172.21.15.172:0/4240779335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb590103710 msgr2=0x7fb59019f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.798+0000 7fb58e59c640 1 --2- 172.21.15.172:0/4240779335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb590103710 0x7fb59019f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.798+0000 7fb58e59c640 1 -- 172.21.15.172:0/4240779335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590102d50 msgr2=0x7fb5901981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.798+0000 7fb58e59c640 1 --2- 172.21.15.172:0/4240779335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590102d50 0x7fb5901981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.798+0000 7fb58e59c640 1 -- 172.21.15.172:0/4240779335 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59019f8b0 con 0x7fb590101b60 2024-09-10T08:50:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.798+0000 7fb58e59c640 1 --2- 172.21.15.172:0/4240779335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb590101b60 0x7fb590197c70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb58000c970 tx=0x7fb58000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.798+0000 7fb5877fe640 1 -- 172.21.15.172:0/4240779335 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580004220 con 0x7fb590101b60 2024-09-10T08:50:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.798+0000 7fb5877fe640 1 -- 172.21.15.172:0/4240779335 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5800043c0 con 0x7fb590101b60 2024-09-10T08:50:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.798+0000 7fb5877fe640 1 -- 172.21.15.172:0/4240779335 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58000f450 con 0x7fb590101b60 2024-09-10T08:50:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.798+0000 7fb59485e640 1 -- 172.21.15.172:0/4240779335 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59019fba0 con 0x7fb590101b60 2024-09-10T08:50:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.798+0000 7fb59485e640 1 -- 172.21.15.172:0/4240779335 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5901a00a0 con 0x7fb590101b60 2024-09-10T08:50:54.804 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:54 smithi191 ceph-mon[35314]: pgmap v490: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.4 MiB/s wr, 12 op/s 2024-09-10T08:50:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.801+0000 7fb5877fe640 1 -- 172.21.15.172:0/4240779335 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb5800079b0 con 0x7fb590101b60 2024-09-10T08:50:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.801+0000 7fb5975fd740 1 Processor -- start 2024-09-10T08:50:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.801+0000 7fb5975fd740 1 -- start start 2024-09-10T08:50:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.801+0000 7fb5975fd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f634db0 0x55843f6351b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.801+0000 7fb5975fd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f6356f0 0x55843f63a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.801+0000 7fb5975fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f63a780 0x55843f63cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.801+0000 7fb5975fd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55843f5942f0 con 0x55843f6356f0 2024-09-10T08:50:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.801+0000 7fb5975fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55843f53c270 con 0x55843f63a780 2024-09-10T08:50:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.801+0000 7fb5975fd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55843f5902b0 con 0x55843f634db0 2024-09-10T08:50:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.801+0000 7fb58e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f6356f0 0x55843f63a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.801+0000 7fb58e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f6356f0 0x55843f63a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35910/0 (socket says 172.21.15.172:35910) 2024-09-10T08:50:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.801+0000 7fb58e59c640 1 -- 172.21.15.172:0/2254135008 learned_addr learned my addr 172.21.15.172:0/2254135008 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.802+0000 7fb5877fe640 1 --2- 172.21.15.172:0/4240779335 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb57c0a2cb0 0x7fb57c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.801+0000 7fb587fff640 1 --2- 172.21.15.172:0/2254135008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f63a780 0x55843f63cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.802+0000 7fb58ed9d640 1 --2- 172.21.15.172:0/2254135008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f634db0 0x55843f6351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.802+0000 7fb58ed9d640 1 --2- 172.21.15.172:0/4240779335 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb57c0a2cb0 0x7fb57c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.802+0000 7fb5877fe640 1 -- 172.21.15.172:0/4240779335 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb5800cf0e0 con 0x7fb590101b60 2024-09-10T08:50:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.802+0000 7fb58e59c640 1 -- 172.21.15.172:0/2254135008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f634db0 msgr2=0x55843f6351b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.802+0000 7fb58e59c640 1 --2- 172.21.15.172:0/2254135008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f634db0 0x55843f6351b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.802+0000 7fb58e59c640 1 -- 172.21.15.172:0/2254135008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f63a780 msgr2=0x55843f63cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.802+0000 7fb58e59c640 1 --2- 172.21.15.172:0/2254135008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f63a780 0x55843f63cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.802+0000 7fb58e59c640 1 -- 172.21.15.172:0/2254135008 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55843f53b340 con 0x55843f6356f0 2024-09-10T08:50:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.802+0000 7fb58e59c640 1 --2- 172.21.15.172:0/2254135008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f6356f0 0x55843f63a1b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb580097a40 tx=0x7fb580004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1ca664a96fe1d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.802+0000 7fb584ff9640 1 -- 172.21.15.172:0/2254135008 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58001d070 con 0x55843f6356f0 2024-09-10T08:50:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.802+0000 7fb58ed9d640 1 --2- 172.21.15.172:0/4240779335 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb57c0a2cb0 0x7fb57c0a5170 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb588009430 tx=0x7fb588005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.802+0000 7fb584ff9640 1 -- 172.21.15.172:0/2254135008 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5800d27d0 con 0x55843f6356f0 2024-09-10T08:50:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.802+0000 7fb584ff9640 1 -- 172.21.15.172:0/2254135008 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580003610 con 0x55843f6356f0 2024-09-10T08:50:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.803+0000 7fb5975fd740 1 -- 172.21.15.172:0/2254135008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f6356f0 msgr2=0x55843f63a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.803+0000 7fb5975fd740 1 --2- 172.21.15.172:0/2254135008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f6356f0 0x55843f63a1b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb580097a40 tx=0x7fb580004570 comp rx=0 tx=0).stop 2024-09-10T08:50:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.803+0000 7fb5975fd740 1 -- 172.21.15.172:0/2254135008 shutdown_connections 2024-09-10T08:50:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.803+0000 7fb5975fd740 1 --2- 172.21.15.172:0/2254135008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f63a780 0x55843f63cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.803+0000 7fb5975fd740 1 --2- 172.21.15.172:0/2254135008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f6356f0 0x55843f63a1b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.803+0000 7fb5975fd740 1 --2- 172.21.15.172:0/2254135008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f634db0 0x55843f6351b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.803+0000 7fb5975fd740 1 -- 172.21.15.172:0/2254135008 >> 172.21.15.172:0/2254135008 conn(0x55843f630420 msgr2=0x55843f630820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.803+0000 7fb5975fd740 1 -- 172.21.15.172:0/2254135008 shutdown_connections 2024-09-10T08:50:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.803+0000 7fb5975fd740 1 -- 172.21.15.172:0/2254135008 wait complete. 2024-09-10T08:50:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.803+0000 7fb5975fd740 1 -- start start 2024-09-10T08:50:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb5975fd740 1 Processor -- start 2024-09-10T08:50:54.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb5975fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f634db0 0x55843f63f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb5975fd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f6356f0 0x55843f63f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb5975fd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f63a780 0x55843f646510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb5975fd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55843f63d6f0 con 0x55843f6356f0 2024-09-10T08:50:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb5975fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55843f51d290 con 0x55843f634db0 2024-09-10T08:50:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb5975fd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55843f63d870 con 0x55843f63a780 2024-09-10T08:50:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb5975fd740 10 client.0 _ll_register_callbacks cb 0x55843f54e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb587fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f6356f0 0x55843f63f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb587fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f6356f0 0x55843f63f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35916/0 (socket says 172.21.15.172:35916) 2024-09-10T08:50:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb587fff640 1 -- 172.21.15.172:0/2374640948 learned_addr learned my addr 172.21.15.172:0/2374640948 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb58e59c640 1 --2- 172.21.15.172:0/2374640948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f634db0 0x55843f63f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb58ed9d640 1 --2- 172.21.15.172:0/2374640948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f63a780 0x55843f646510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb587fff640 1 -- 172.21.15.172:0/2374640948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f63a780 msgr2=0x55843f646510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb587fff640 1 --2- 172.21.15.172:0/2374640948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f63a780 0x55843f646510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb587fff640 1 -- 172.21.15.172:0/2374640948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f634db0 msgr2=0x55843f63f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb587fff640 1 --2- 172.21.15.172:0/2374640948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f634db0 0x55843f63f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb587fff640 1 -- 172.21.15.172:0/2374640948 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55843f2a7210 con 0x55843f6356f0 2024-09-10T08:50:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb587fff640 1 --2- 172.21.15.172:0/2374640948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f6356f0 0x55843f63f540 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb57800beb0 tx=0x7fb578012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c54aff70686a11c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.804+0000 7fb575ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.805+0000 7fb575ffb640 1 -- 172.21.15.172:0/2374640948 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578004230 con 0x55843f6356f0 2024-09-10T08:50:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.805+0000 7fb575ffb640 1 -- 172.21.15.172:0/2374640948 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb5780043d0 con 0x55843f6356f0 2024-09-10T08:50:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.805+0000 7fb5975fd740 1 -- 172.21.15.172:0/2374640948 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55843f5a1a20 con 0x55843f6356f0 2024-09-10T08:50:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.805+0000 7fb575ffb640 1 -- 172.21.15.172:0/2374640948 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb5780106a0 con 0x55843f6356f0 2024-09-10T08:50:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.805+0000 7fb5975fd740 10 client.16782 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.805+0000 7fb5975fd740 1 -- 172.21.15.172:0/2374640948 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55843f373620 con 0x55843f6356f0 2024-09-10T08:50:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.805+0000 7fb575ffb640 1 -- 172.21.15.172:0/2374640948 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55843f5a1a20 con 0x55843f6356f0 2024-09-10T08:50:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.805+0000 7fb575ffb640 1 -- 172.21.15.172:0/2374640948 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fb578019070 con 0x55843f6356f0 2024-09-10T08:50:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.805+0000 7fb5975fd740 10 client.16782 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.805+0000 7fb5975fd740 10 client.16782 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.805+0000 7fb5975fd740 20 client.16782 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.805+0000 7fb5975fd740 1 --2- 172.21.15.172:0/2374640948 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55843f64acd0 0x55843f64d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.805+0000 7fb5975fd740 4 client.16782 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:50:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.805+0000 7fb5975fd740 1 -- 172.21.15.172:0/2374640948 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55843f59bd70 con 0x55843f64acd0 2024-09-10T08:50:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.806+0000 7fb58e59c640 1 --2- 172.21.15.172:0/2374640948 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55843f64acd0 0x55843f64d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.806+0000 7fb58e59c640 1 --2- 172.21.15.172:0/2374640948 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55843f64acd0 0x55843f64d150 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=624d791d8f60c81b server_cookie=75fce18a8284d2ce in_seq=0 out_seq=0 2024-09-10T08:50:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.806+0000 7fb575ffb640 10 client.16782 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.807+0000 7fb575ffb640 1 -- 172.21.15.172:0/2374640948 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55843f59bd70 con 0x55843f64acd0 2024-09-10T08:50:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.807+0000 7fb575ffb640 10 client.16782 handle_command_reply: tid=0 2024-09-10T08:50:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.807+0000 7fb5975fd740 1 client.16782 shutdown 2024-09-10T08:50:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.807+0000 7fb5975fd740 1 -- 172.21.15.172:0/2374640948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f6356f0 msgr2=0x55843f63f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.807+0000 7fb5975fd740 1 --2- 172.21.15.172:0/2374640948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f6356f0 0x55843f63f540 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb57800beb0 tx=0x7fb578012820 comp rx=0 tx=0).stop 2024-09-10T08:50:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.807+0000 7fb5975fd740 1 -- 172.21.15.172:0/2374640948 shutdown_connections 2024-09-10T08:50:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.807+0000 7fb5975fd740 1 --2- 172.21.15.172:0/2374640948 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55843f64acd0 0x55843f64d150 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.807+0000 7fb5975fd740 1 --2- 172.21.15.172:0/2374640948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f63a780 0x55843f646510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.807+0000 7fb575ffb640 0 client.16782 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.807+0000 7fb5975fd740 1 --2- 172.21.15.172:0/2374640948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f6356f0 0x55843f63f540 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.807+0000 7fb5975fd740 1 --2- 172.21.15.172:0/2374640948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f634db0 0x55843f63f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.807+0000 7fb5975fd740 1 -- 172.21.15.172:0/2374640948 >> 172.21.15.172:0/2374640948 conn(0x55843f630420 msgr2=0x55843f62b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.807+0000 7fb5975fd740 1 -- 172.21.15.172:0/2374640948 shutdown_connections 2024-09-10T08:50:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.807+0000 7fb5975fd740 1 -- 172.21.15.172:0/2374640948 wait complete. 2024-09-10T08:50:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.807+0000 7fb5975fd740 20 client.16782 trim_cache size 0 max 16384 2024-09-10T08:50:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.821+0000 7fb5975fd740 1 Processor -- start 2024-09-10T08:50:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.821+0000 7fb5975fd740 1 -- start start 2024-09-10T08:50:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.821+0000 7fb5975fd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f634db0 0x55843f63f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.821+0000 7fb5975fd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f6356f0 0x55843f6464d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.821+0000 7fb5975fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f63a780 0x55843f63f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.821+0000 7fb5975fd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901a8380 con 0x55843f634db0 2024-09-10T08:50:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.821+0000 7fb5975fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901a9f80 con 0x55843f63a780 2024-09-10T08:50:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.821+0000 7fb5975fd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901a8a50 con 0x55843f6356f0 2024-09-10T08:50:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.822+0000 7fb58e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f634db0 0x55843f63f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.822+0000 7fb58e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f634db0 0x55843f63f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35926/0 (socket says 172.21.15.172:35926) 2024-09-10T08:50:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.822+0000 7fb58e59c640 1 -- 172.21.15.172:0/2550134235 learned_addr learned my addr 172.21.15.172:0/2550134235 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.822+0000 7fb587fff640 1 --2- 172.21.15.172:0/2550134235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f6356f0 0x55843f6464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.822+0000 7fb58ed9d640 1 --2- 172.21.15.172:0/2550134235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f63a780 0x55843f63f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.822+0000 7fb58e59c640 1 -- 172.21.15.172:0/2550134235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f6356f0 msgr2=0x55843f6464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.822+0000 7fb58e59c640 1 --2- 172.21.15.172:0/2550134235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f6356f0 0x55843f6464d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.822+0000 7fb58e59c640 1 -- 172.21.15.172:0/2550134235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f63a780 msgr2=0x55843f63f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.822+0000 7fb58e59c640 1 --2- 172.21.15.172:0/2550134235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f63a780 0x55843f63f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.822+0000 7fb58e59c640 1 -- 172.21.15.172:0/2550134235 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55843f63eb40 con 0x55843f634db0 2024-09-10T08:50:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.822+0000 7fb587fff640 1 --2- 172.21.15.172:0/2550134235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f6356f0 0x55843f6464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:50:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.822+0000 7fb58e59c640 1 --2- 172.21.15.172:0/2550134235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f634db0 0x55843f63f000 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb5800ccbf0 tx=0x7fb5800979b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f2171ed1d35646 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.822+0000 7fb577fff640 1 -- 172.21.15.172:0/2550134235 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58001d070 con 0x55843f634db0 2024-09-10T08:50:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.822+0000 7fb577fff640 1 -- 172.21.15.172:0/2550134235 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb58009bce0 con 0x55843f634db0 2024-09-10T08:50:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.822+0000 7fb577fff640 1 -- 172.21.15.172:0/2550134235 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580097c50 con 0x55843f634db0 2024-09-10T08:50:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 1 -- 172.21.15.172:0/2550134235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f634db0 msgr2=0x55843f63f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 1 --2- 172.21.15.172:0/2550134235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f634db0 0x55843f63f000 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb5800ccbf0 tx=0x7fb5800979b0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 1 -- 172.21.15.172:0/2550134235 shutdown_connections 2024-09-10T08:50:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 1 --2- 172.21.15.172:0/2550134235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f63a780 0x55843f63f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 1 --2- 172.21.15.172:0/2550134235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f6356f0 0x55843f6464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 1 --2- 172.21.15.172:0/2550134235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f634db0 0x55843f63f000 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 1 -- 172.21.15.172:0/2550134235 >> 172.21.15.172:0/2550134235 conn(0x55843f630420 msgr2=0x55843f639fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 1 -- 172.21.15.172:0/2550134235 shutdown_connections 2024-09-10T08:50:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 1 -- 172.21.15.172:0/2550134235 wait complete. 2024-09-10T08:50:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 1 -- start start 2024-09-10T08:50:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 1 Processor -- start 2024-09-10T08:50:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f634db0 0x55843f643090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f6356f0 0x55843f6435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f63a780 0x55843f6b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901a9820 con 0x55843f63a780 2024-09-10T08:50:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901a9150 con 0x55843f6356f0 2024-09-10T08:50:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901a6360 con 0x55843f634db0 2024-09-10T08:50:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb5975fd740 10 client.0 _ll_register_callbacks cb 0x7fb57800b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.823+0000 7fb58ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f63a780 0x55843f6b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb58ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f63a780 0x55843f6b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35934/0 (socket says 172.21.15.172:35934) 2024-09-10T08:50:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb58ed9d640 1 -- 172.21.15.172:0/1640973990 learned_addr learned my addr 172.21.15.172:0/1640973990 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb587fff640 1 --2- 172.21.15.172:0/1640973990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f6356f0 0x55843f6435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb58e59c640 1 --2- 172.21.15.172:0/1640973990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f634db0 0x55843f643090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb58ed9d640 1 -- 172.21.15.172:0/1640973990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f634db0 msgr2=0x55843f643090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb58ed9d640 1 --2- 172.21.15.172:0/1640973990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f634db0 0x55843f643090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb58ed9d640 1 -- 172.21.15.172:0/1640973990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f6356f0 msgr2=0x55843f6435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb58ed9d640 1 --2- 172.21.15.172:0/1640973990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f6356f0 0x55843f6435d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb58ed9d640 1 -- 172.21.15.172:0/1640973990 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55843f6466c0 con 0x55843f63a780 2024-09-10T08:50:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb58ed9d640 1 --2- 172.21.15.172:0/1640973990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f63a780 0x55843f6b3a20 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb58802ade0 tx=0x7fb5880095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47abd003a31d3ba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb575ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:54.833 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb575ffb640 1 -- 172.21.15.172:0/1640973990 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588003e60 con 0x55843f63a780 2024-09-10T08:50:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb575ffb640 1 -- 172.21.15.172:0/1640973990 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb588004410 con 0x55843f63a780 2024-09-10T08:50:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb5975fd740 1 -- 172.21.15.172:0/1640973990 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb58009bce0 con 0x55843f63a780 2024-09-10T08:50:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb575ffb640 1 -- 172.21.15.172:0/1640973990 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb588004610 con 0x55843f63a780 2024-09-10T08:50:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb5975fd740 10 client.16788 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb5975fd740 1 -- 172.21.15.172:0/1640973990 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55843f6b4ef0 con 0x55843f63a780 2024-09-10T08:50:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.824+0000 7fb575ffb640 1 -- 172.21.15.172:0/1640973990 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb58009bce0 con 0x55843f63a780 2024-09-10T08:50:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.825+0000 7fb575ffb640 1 -- 172.21.15.172:0/1640973990 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fb58802f500 con 0x55843f63a780 2024-09-10T08:50:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.825+0000 7fb5975fd740 10 client.16788 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.825+0000 7fb5975fd740 10 client.16788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.825+0000 7fb5975fd740 20 client.16788 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.825+0000 7fb5975fd740 1 --2- 172.21.15.172:0/1640973990 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55843f6b5930 0x55843f6b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.825+0000 7fb5975fd740 4 client.16788 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:50:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.825+0000 7fb5975fd740 1 -- 172.21.15.172:0/1640973990 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55843f518900 con 0x55843f6b5930 2024-09-10T08:50:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.825+0000 7fb58e59c640 1 --2- 172.21.15.172:0/1640973990 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55843f6b5930 0x55843f6b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.825+0000 7fb575ffb640 10 client.16788 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:54.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.825+0000 7fb58e59c640 1 --2- 172.21.15.172:0/1640973990 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55843f6b5930 0x55843f6b7d50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aac137663b3f2d3 server_cookie=1fd7e48cae5c9a13 in_seq=0 out_seq=0 2024-09-10T08:50:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.832+0000 7fb575ffb640 1 -- 172.21.15.172:0/1640973990 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55843f518900 con 0x55843f6b5930 2024-09-10T08:50:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.832+0000 7fb575ffb640 10 client.16788 handle_command_reply: tid=0 2024-09-10T08:50:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.832+0000 7fb5975fd740 1 client.16788 shutdown 2024-09-10T08:50:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.832+0000 7fb5975fd740 1 -- 172.21.15.172:0/1640973990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f63a780 msgr2=0x55843f6b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.832+0000 7fb5975fd740 1 --2- 172.21.15.172:0/1640973990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f63a780 0x55843f6b3a20 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb58802ade0 tx=0x7fb5880095c0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.832+0000 7fb5975fd740 1 -- 172.21.15.172:0/1640973990 shutdown_connections 2024-09-10T08:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.832+0000 7fb5975fd740 1 --2- 172.21.15.172:0/1640973990 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55843f6b5930 0x55843f6b7d50 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.832+0000 7fb5975fd740 1 --2- 172.21.15.172:0/1640973990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55843f63a780 0x55843f6b3a20 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.832+0000 7fb5975fd740 1 --2- 172.21.15.172:0/1640973990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55843f6356f0 0x55843f6435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.832+0000 7fb5975fd740 1 --2- 172.21.15.172:0/1640973990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55843f634db0 0x55843f643090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.832+0000 7fb5975fd740 1 -- 172.21.15.172:0/1640973990 >> 172.21.15.172:0/1640973990 conn(0x55843f630420 msgr2=0x55843f639b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.832+0000 7fb575ffb640 0 client.16788 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.832+0000 7fb5975fd740 1 -- 172.21.15.172:0/1640973990 shutdown_connections 2024-09-10T08:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.832+0000 7fb5975fd740 1 -- 172.21.15.172:0/1640973990 wait complete. 2024-09-10T08:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.832+0000 7fb5975fd740 20 client.16788 trim_cache size 0 max 16384 2024-09-10T08:50:54.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:50:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:50:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:50:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:50:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.833+0000 7fb59485e640 1 -- 172.21.15.172:0/4240779335 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb57c0a2cb0 msgr2=0x7fb57c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.833+0000 7fb59485e640 1 --2- 172.21.15.172:0/4240779335 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb57c0a2cb0 0x7fb57c0a5170 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb588009430 tx=0x7fb588005d80 comp rx=0 tx=0).stop 2024-09-10T08:50:54.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.833+0000 7fb59485e640 1 -- 172.21.15.172:0/4240779335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb590101b60 msgr2=0x7fb590197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.833+0000 7fb59485e640 1 --2- 172.21.15.172:0/4240779335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb590101b60 0x7fb590197c70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb58000c970 tx=0x7fb58000ce30 comp rx=0 tx=0).stop 2024-09-10T08:50:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.833+0000 7fb59485e640 1 -- 172.21.15.172:0/4240779335 shutdown_connections 2024-09-10T08:50:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.833+0000 7fb59485e640 1 --2- 172.21.15.172:0/4240779335 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb57c0a2cb0 0x7fb57c0a5170 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.833+0000 7fb59485e640 1 --2- 172.21.15.172:0/4240779335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb590103710 0x7fb59019f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.833+0000 7fb59485e640 1 --2- 172.21.15.172:0/4240779335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590102d50 0x7fb5901981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.833+0000 7fb59485e640 1 --2- 172.21.15.172:0/4240779335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb590101b60 0x7fb590197c70 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.833+0000 7fb59485e640 1 -- 172.21.15.172:0/4240779335 >> 172.21.15.172:0/4240779335 conn(0x7fb5900fd2e0 msgr2=0x7fb5901060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.833+0000 7fb59485e640 1 -- 172.21.15.172:0/4240779335 shutdown_connections 2024-09-10T08:50:54.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:50:54.833+0000 7fb59485e640 1 -- 172.21.15.172:0/4240779335 wait complete. 2024-09-10T08:50:56.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:56 smithi172 ceph-mon[30854]: pgmap v491: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 MiB/s wr, 16 op/s 2024-09-10T08:50:56.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:56 smithi174 ceph-mon[34068]: pgmap v491: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 MiB/s wr, 16 op/s 2024-09-10T08:50:56.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:56 smithi191 ceph-mon[35314]: pgmap v491: 129 pgs: 129 active+clean; 3.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.7 MiB/s wr, 16 op/s 2024-09-10T08:50:57.294 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:50:57.550 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:50:58.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.020+0000 7f4ff4614640 1 -- 172.21.15.172:0/1207779069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fec102790 msgr2=0x7f4fec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:58.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.020+0000 7f4ff4614640 1 --2- 172.21.15.172:0/1207779069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fec102790 0x7f4fec1067e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009900 tx=0x7f4fdc02a3f0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.020+0000 7f4ff4614640 1 -- 172.21.15.172:0/1207779069 shutdown_connections 2024-09-10T08:50:58.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.020+0000 7f4ff4614640 1 --2- 172.21.15.172:0/1207779069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fec106f10 0x7f4fec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.020+0000 7f4ff4614640 1 --2- 172.21.15.172:0/1207779069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fec102790 0x7f4fec1067e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.020+0000 7f4ff4614640 1 --2- 172.21.15.172:0/1207779069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fec101de0 0x7f4fec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.020+0000 7f4ff4614640 1 -- 172.21.15.172:0/1207779069 >> 172.21.15.172:0/1207779069 conn(0x7f4fec0fd8c0 msgr2=0x7f4fec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:58.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.020+0000 7f4ff4614640 1 -- 172.21.15.172:0/1207779069 shutdown_connections 2024-09-10T08:50:58.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.020+0000 7f4ff4614640 1 -- 172.21.15.172:0/1207779069 wait complete. 2024-09-10T08:50:58.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.021+0000 7f4ff4614640 1 Processor -- start 2024-09-10T08:50:58.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.021+0000 7f4ff4614640 1 -- start start 2024-09-10T08:50:58.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.021+0000 7f4ff4614640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fec101de0 0x7f4fec198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.021+0000 7f4ff4614640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fec102790 0x7f4fec1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.021+0000 7f4ff4614640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fec106f10 0x7f4fec19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.021+0000 7f4ff4614640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec111e60 con 0x7f4fec102790 2024-09-10T08:50:58.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.021+0000 7f4ff4614640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec111ce0 con 0x7f4fec106f10 2024-09-10T08:50:58.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.021+0000 7f4ff4614640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec111fe0 con 0x7f4fec101de0 2024-09-10T08:50:58.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.021+0000 7f4ff1baf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fec102790 0x7f4fec1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.021+0000 7f4ff1baf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fec102790 0x7f4fec1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55412/0 (socket says 172.21.15.172:55412) 2024-09-10T08:50:58.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.021+0000 7f4ff1baf640 1 -- 172.21.15.172:0/8296059 learned_addr learned my addr 172.21.15.172:0/8296059 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:58.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.021+0000 7f4ff2bb1640 1 --2- 172.21.15.172:0/8296059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fec106f10 0x7f4fec19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.022+0000 7f4ff23b0640 1 --2- 172.21.15.172:0/8296059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fec101de0 0x7f4fec198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.022+0000 7f4ff1baf640 1 -- 172.21.15.172:0/8296059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fec101de0 msgr2=0x7f4fec198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:58.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.022+0000 7f4ff1baf640 1 --2- 172.21.15.172:0/8296059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fec101de0 0x7f4fec198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.022+0000 7f4ff1baf640 1 -- 172.21.15.172:0/8296059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fec106f10 msgr2=0x7f4fec19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:58.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.022+0000 7f4ff1baf640 1 --2- 172.21.15.172:0/8296059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fec106f10 0x7f4fec19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.022+0000 7f4ff1baf640 1 -- 172.21.15.172:0/8296059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fec19d420 con 0x7f4fec102790 2024-09-10T08:50:58.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.022+0000 7f4ff23b0640 1 --2- 172.21.15.172:0/8296059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fec101de0 0x7f4fec198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:50:58.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.022+0000 7f4ff1baf640 1 --2- 172.21.15.172:0/8296059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fec102790 0x7f4fec1989b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc005940 tx=0x7f4fdc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:58.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.022+0000 7f4fe37fe640 1 -- 172.21.15.172:0/8296059 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc03f030 con 0x7f4fec102790 2024-09-10T08:50:58.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.022+0000 7f4fe37fe640 1 -- 172.21.15.172:0/8296059 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fdc003aa0 con 0x7f4fec102790 2024-09-10T08:50:58.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.022+0000 7f4ff4614640 1 -- 172.21.15.172:0/8296059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fec19d6b0 con 0x7f4fec102790 2024-09-10T08:50:58.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.022+0000 7f4fe37fe640 1 -- 172.21.15.172:0/8296059 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc035070 con 0x7f4fec102790 2024-09-10T08:50:58.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.022+0000 7f4ff4614640 1 -- 172.21.15.172:0/8296059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fec1a4ff0 con 0x7f4fec102790 2024-09-10T08:50:58.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.024+0000 7f4fe37fe640 1 -- 172.21.15.172:0/8296059 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4fdc042030 con 0x7f4fec102790 2024-09-10T08:50:58.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.025+0000 7f4ff69ed740 1 Processor -- start 2024-09-10T08:50:58.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.025+0000 7f4ff69ed740 1 -- start start 2024-09-10T08:50:58.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.025+0000 7f4ff69ed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1c4a0 0x563f3fa1c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.025+0000 7f4ff69ed740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa27140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.025+0000 7f4ff69ed740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa27680 0x563f3fa29a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.025+0000 7f4ff69ed740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563f3f9822e0 con 0x563f3fa1cdc0 2024-09-10T08:50:58.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.025+0000 7f4ff69ed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563f3f929ce0 con 0x563f3fa1c4a0 2024-09-10T08:50:58.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.025+0000 7f4ff69ed740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563f3f97e2a0 con 0x563f3fa27680 2024-09-10T08:50:58.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.025+0000 7f4fe37fe640 1 --2- 172.21.15.172:0/8296059 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4fd80a2b80 0x7f4fd80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.025+0000 7f4ff23b0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa27140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.025+0000 7f4ff23b0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa27140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55428/0 (socket says 172.21.15.172:55428) 2024-09-10T08:50:58.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.025+0000 7f4ff23b0640 1 -- 172.21.15.172:0/3077126966 learned_addr learned my addr 172.21.15.172:0/3077126966 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:58.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.025+0000 7f4ff2bb1640 1 --2- 172.21.15.172:0/8296059 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4fd80a2b80 0x7f4fd80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.025+0000 7f4fe37fe640 1 -- 172.21.15.172:0/8296059 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4fdc0ef880 con 0x7f4fec102790 2024-09-10T08:50:58.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.025+0000 7f4ff2bb1640 1 --2- 172.21.15.172:0/3077126966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1c4a0 0x563f3fa1c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.025+0000 7f4ff1baf640 1 --2- 172.21.15.172:0/3077126966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa27680 0x563f3fa29a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.026+0000 7f4ff23b0640 1 -- 172.21.15.172:0/3077126966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa27680 msgr2=0x563f3fa29a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:58.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.026+0000 7f4ff23b0640 1 --2- 172.21.15.172:0/3077126966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa27680 0x563f3fa29a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.026+0000 7f4ff23b0640 1 -- 172.21.15.172:0/3077126966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1c4a0 msgr2=0x563f3fa1c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:58.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.026+0000 7f4ff23b0640 1 --2- 172.21.15.172:0/3077126966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1c4a0 0x563f3fa1c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.026+0000 7f4ff23b0640 1 -- 172.21.15.172:0/3077126966 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f3f90e440 con 0x563f3fa1cdc0 2024-09-10T08:50:58.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.026+0000 7f4ff1baf640 1 --2- 172.21.15.172:0/3077126966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa27680 0x563f3fa29a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:50:58.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.026+0000 7f4ff23b0640 1 --2- 172.21.15.172:0/3077126966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa27140 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009c50 tx=0x7f4fe800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516b6ac500b1060f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:58.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.026+0000 7f4ff2bb1640 1 --2- 172.21.15.172:0/8296059 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4fd80a2b80 0x7f4fd80a5040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400a700 tx=0x7f4fe4009540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:58.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.026+0000 7f4fe0ff9640 1 -- 172.21.15.172:0/3077126966 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8013770 con 0x563f3fa1cdc0 2024-09-10T08:50:58.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.026+0000 7f4fe0ff9640 1 -- 172.21.15.172:0/3077126966 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fe8013dc0 con 0x563f3fa1cdc0 2024-09-10T08:50:58.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.026+0000 7f4fe0ff9640 1 -- 172.21.15.172:0/3077126966 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe801bcb0 con 0x563f3fa1cdc0 2024-09-10T08:50:58.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.027+0000 7f4ff69ed740 1 -- 172.21.15.172:0/3077126966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 msgr2=0x563f3fa27140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:58.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.027+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/3077126966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa27140 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009c50 tx=0x7f4fe800e6c0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.027+0000 7f4ff69ed740 1 -- 172.21.15.172:0/3077126966 shutdown_connections 2024-09-10T08:50:58.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.027+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/3077126966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa27680 0x563f3fa29a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.027+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/3077126966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa27140 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.027+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/3077126966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1c4a0 0x563f3fa1c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.027+0000 7f4ff69ed740 1 -- 172.21.15.172:0/3077126966 >> 172.21.15.172:0/3077126966 conn(0x563f3fa18a30 msgr2=0x563f3fa18e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:58.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.027+0000 7f4ff69ed740 1 -- 172.21.15.172:0/3077126966 shutdown_connections 2024-09-10T08:50:58.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.027+0000 7f4ff69ed740 1 -- 172.21.15.172:0/3077126966 wait complete. 2024-09-10T08:50:58.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.027+0000 7f4ff69ed740 1 -- start start 2024-09-10T08:50:58.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.027+0000 7f4ff69ed740 1 Processor -- start 2024-09-10T08:50:58.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.027+0000 7f4ff69ed740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa1c4a0 0x563f3fa26f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.027+0000 7f4ff69ed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1cdc0 0x563f3fa24210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff69ed740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa27680 0x563f3fa24750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff69ed740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563f3fa2a250 con 0x563f3fa27680 2024-09-10T08:50:58.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff69ed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563f3f93d050 con 0x563f3fa1cdc0 2024-09-10T08:50:58.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff69ed740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563f3fa2a3d0 con 0x563f3fa1c4a0 2024-09-10T08:50:58.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff69ed740 10 client.0 _ll_register_callbacks cb 0x563f3f97d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:58.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff2bb1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa27680 0x563f3fa24750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff2bb1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa27680 0x563f3fa24750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55444/0 (socket says 172.21.15.172:55444) 2024-09-10T08:50:58.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff2bb1640 1 -- 172.21.15.172:0/575088642 learned_addr learned my addr 172.21.15.172:0/575088642 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:58.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff1baf640 1 --2- 172.21.15.172:0/575088642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1cdc0 0x563f3fa24210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff23b0640 1 --2- 172.21.15.172:0/575088642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa1c4a0 0x563f3fa26f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff2bb1640 1 -- 172.21.15.172:0/575088642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa1c4a0 msgr2=0x563f3fa26f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:58.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff2bb1640 1 --2- 172.21.15.172:0/575088642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa1c4a0 0x563f3fa26f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff2bb1640 1 -- 172.21.15.172:0/575088642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1cdc0 msgr2=0x563f3fa24210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:58.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff2bb1640 1 --2- 172.21.15.172:0/575088642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1cdc0 0x563f3fa24210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff2bb1640 1 -- 172.21.15.172:0/575088642 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f3f6f33c0 con 0x563f3fa27680 2024-09-10T08:50:58.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff2bb1640 1 --2- 172.21.15.172:0/575088642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa27680 0x563f3fa24750 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4019d80 tx=0x7f4fe4004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3ab2d7d27df1b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:58.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4fd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:58.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4fd5ffb640 1 -- 172.21.15.172:0/575088642 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4029060 con 0x563f3fa27680 2024-09-10T08:50:58.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4fd5ffb640 1 -- 172.21.15.172:0/575088642 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4fe4013070 con 0x563f3fa27680 2024-09-10T08:50:58.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff69ed740 1 -- 172.21.15.172:0/575088642 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fe8013dc0 con 0x563f3fa27680 2024-09-10T08:50:58.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4fd5ffb640 1 -- 172.21.15.172:0/575088642 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4fe4030050 con 0x563f3fa27680 2024-09-10T08:50:58.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff69ed740 10 client.16800 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:58.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.028+0000 7f4ff69ed740 1 -- 172.21.15.172:0/575088642 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563f3fa35b90 con 0x563f3fa27680 2024-09-10T08:50:58.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.029+0000 7f4fd5ffb640 1 -- 172.21.15.172:0/575088642 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fe8013dc0 con 0x563f3fa27680 2024-09-10T08:50:58.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.029+0000 7f4fd5ffb640 1 -- 172.21.15.172:0/575088642 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f4fe4024a40 con 0x563f3fa27680 2024-09-10T08:50:58.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.029+0000 7f4ff69ed740 10 client.16800 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:58.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.029+0000 7f4ff69ed740 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:58.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.029+0000 7f4ff69ed740 20 client.16800 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:58.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.029+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/575088642 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563f3fa37050 0x563f3fa394f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.029+0000 7f4ff69ed740 4 client.16800 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:50:58.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.029+0000 7f4ff69ed740 1 -- 172.21.15.172:0/575088642 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f3f98f850 con 0x563f3fa37050 2024-09-10T08:50:58.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.029+0000 7f4ff23b0640 1 --2- 172.21.15.172:0/575088642 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563f3fa37050 0x563f3fa394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.030+0000 7f4ff23b0640 1 --2- 172.21.15.172:0/575088642 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563f3fa37050 0x563f3fa394f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=249dccb7555f7ac1 server_cookie=54c5eb01c8795222 in_seq=0 out_seq=0 2024-09-10T08:50:58.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.030+0000 7f4fd5ffb640 10 client.16800 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:58.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.030+0000 7f4fd5ffb640 1 -- 172.21.15.172:0/575088642 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563f3f98f850 con 0x563f3fa37050 2024-09-10T08:50:58.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.030+0000 7f4fd5ffb640 10 client.16800 handle_command_reply: tid=0 2024-09-10T08:50:58.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.030+0000 7f4ff69ed740 1 client.16800 shutdown 2024-09-10T08:50:58.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.030+0000 7f4ff69ed740 1 -- 172.21.15.172:0/575088642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa27680 msgr2=0x563f3fa24750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:58.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.030+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/575088642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa27680 0x563f3fa24750 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4019d80 tx=0x7f4fe4004b50 comp rx=0 tx=0).stop 2024-09-10T08:50:58.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.030+0000 7f4ff69ed740 1 -- 172.21.15.172:0/575088642 shutdown_connections 2024-09-10T08:50:58.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.030+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/575088642 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563f3fa37050 0x563f3fa394f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.031+0000 7f4fd5ffb640 0 client.16800 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:58.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.031+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/575088642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa27680 0x563f3fa24750 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.031+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/575088642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1cdc0 0x563f3fa24210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.031+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/575088642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa1c4a0 0x563f3fa26f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.031+0000 7f4ff69ed740 1 -- 172.21.15.172:0/575088642 >> 172.21.15.172:0/575088642 conn(0x563f3fa18a30 msgr2=0x563f3fa1e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:58.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.031+0000 7f4ff69ed740 1 -- 172.21.15.172:0/575088642 shutdown_connections 2024-09-10T08:50:58.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.031+0000 7f4ff69ed740 1 -- 172.21.15.172:0/575088642 wait complete. 2024-09-10T08:50:58.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.031+0000 7f4ff69ed740 20 client.16800 trim_cache size 0 max 16384 2024-09-10T08:50:58.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff69ed740 1 Processor -- start 2024-09-10T08:50:58.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff69ed740 1 -- start start 2024-09-10T08:50:58.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff69ed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1c4a0 0x563f3fa26180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff69ed740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa3edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff69ed740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa27680 0x563f3fa3f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff69ed740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1ac960 con 0x563f3fa1cdc0 2024-09-10T08:50:58.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff69ed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1ae560 con 0x563f3fa1c4a0 2024-09-10T08:50:58.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff69ed740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1ad030 con 0x563f3fa27680 2024-09-10T08:50:58.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff1baf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa3edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff1baf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa3edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55452/0 (socket says 172.21.15.172:55452) 2024-09-10T08:50:58.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff1baf640 1 -- 172.21.15.172:0/2306284558 learned_addr learned my addr 172.21.15.172:0/2306284558 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:58.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff2bb1640 1 --2- 172.21.15.172:0/2306284558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa27680 0x563f3fa3f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff23b0640 1 --2- 172.21.15.172:0/2306284558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1c4a0 0x563f3fa26180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff1baf640 1 -- 172.21.15.172:0/2306284558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa27680 msgr2=0x563f3fa3f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:58.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff1baf640 1 --2- 172.21.15.172:0/2306284558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa27680 0x563f3fa3f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff1baf640 1 -- 172.21.15.172:0/2306284558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1c4a0 msgr2=0x563f3fa26180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:58.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff1baf640 1 --2- 172.21.15.172:0/2306284558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1c4a0 0x563f3fa26180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff1baf640 1 -- 172.21.15.172:0/2306284558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f3fa1cbf0 con 0x563f3fa1cdc0 2024-09-10T08:50:58.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff23b0640 1 --2- 172.21.15.172:0/2306284558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1c4a0 0x563f3fa26180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:50:58.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4ff1baf640 1 --2- 172.21.15.172:0/2306284558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa3edc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0b8750 tx=0x7f4fdc03b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c32dfc92f3b3883 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:58.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.048+0000 7f4fd7fff640 1 -- 172.21.15.172:0/2306284558 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc049040 con 0x563f3fa1cdc0 2024-09-10T08:50:58.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.049+0000 7f4fd7fff640 1 -- 172.21.15.172:0/2306284558 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fdc0c0130 con 0x563f3fa1cdc0 2024-09-10T08:50:58.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.049+0000 7f4fd7fff640 1 -- 172.21.15.172:0/2306284558 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc0b83f0 con 0x563f3fa1cdc0 2024-09-10T08:50:58.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.049+0000 7f4ff69ed740 1 -- 172.21.15.172:0/2306284558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 msgr2=0x563f3fa3edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:58.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.049+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/2306284558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa3edc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0b8750 tx=0x7f4fdc03b520 comp rx=0 tx=0).stop 2024-09-10T08:50:58.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.049+0000 7f4ff69ed740 1 -- 172.21.15.172:0/2306284558 shutdown_connections 2024-09-10T08:50:58.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.049+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/2306284558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa27680 0x563f3fa3f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.049+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/2306284558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa3edc0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.049+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/2306284558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1c4a0 0x563f3fa26180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.049+0000 7f4ff69ed740 1 -- 172.21.15.172:0/2306284558 >> 172.21.15.172:0/2306284558 conn(0x563f3fa18a30 msgr2=0x563f3fa25a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:58.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.049+0000 7f4ff69ed740 1 -- 172.21.15.172:0/2306284558 shutdown_connections 2024-09-10T08:50:58.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.049+0000 7f4ff69ed740 1 -- 172.21.15.172:0/2306284558 wait complete. 2024-09-10T08:50:58.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff69ed740 1 -- start start 2024-09-10T08:50:58.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff69ed740 1 Processor -- start 2024-09-10T08:50:58.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff69ed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1c4a0 0x563f3fa2fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff69ed740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa2cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff69ed740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa27680 0x563f3fa2d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff69ed740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1ade00 con 0x563f3fa1cdc0 2024-09-10T08:50:58.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff69ed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1ad730 con 0x563f3fa1c4a0 2024-09-10T08:50:58.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff69ed740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1aa940 con 0x563f3fa27680 2024-09-10T08:50:58.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff69ed740 10 client.0 _ll_register_callbacks cb 0x7f4fe4009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:50:58.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff1baf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa2cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff1baf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa2cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55462/0 (socket says 172.21.15.172:55462) 2024-09-10T08:50:58.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff1baf640 1 -- 172.21.15.172:0/4171584249 learned_addr learned my addr 172.21.15.172:0/4171584249 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:58.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff23b0640 1 --2- 172.21.15.172:0/4171584249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1c4a0 0x563f3fa2fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff2bb1640 1 --2- 172.21.15.172:0/4171584249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa27680 0x563f3fa2d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff1baf640 1 -- 172.21.15.172:0/4171584249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa27680 msgr2=0x563f3fa2d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:58.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff1baf640 1 --2- 172.21.15.172:0/4171584249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa27680 0x563f3fa2d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff1baf640 1 -- 172.21.15.172:0/4171584249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1c4a0 msgr2=0x563f3fa2fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:50:58.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff1baf640 1 --2- 172.21.15.172:0/4171584249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1c4a0 0x563f3fa2fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff1baf640 1 -- 172.21.15.172:0/4171584249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f3fa2d9f0 con 0x563f3fa1cdc0 2024-09-10T08:50:58.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.050+0000 7f4ff23b0640 1 --2- 172.21.15.172:0/4171584249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1c4a0 0x563f3fa2fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:50:58.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.051+0000 7f4ff1baf640 1 --2- 172.21.15.172:0/4171584249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa2cec0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0ed070 tx=0x7f4fdc0c4b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6754e8775181c9a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:58.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.051+0000 7f4fd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:50:58.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.051+0000 7f4fd5ffb640 1 -- 172.21.15.172:0/4171584249 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc049040 con 0x563f3fa1cdc0 2024-09-10T08:50:58.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.051+0000 7f4fd5ffb640 1 -- 172.21.15.172:0/4171584249 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4fdc0b83f0 con 0x563f3fa1cdc0 2024-09-10T08:50:58.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.051+0000 7f4ff69ed740 1 -- 172.21.15.172:0/4171584249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fdc0c0130 con 0x563f3fa1cdc0 2024-09-10T08:50:58.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.051+0000 7f4fd5ffb640 1 -- 172.21.15.172:0/4171584249 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4fdc0c4560 con 0x563f3fa1cdc0 2024-09-10T08:50:58.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.051+0000 7f4ff69ed740 10 client.16806 fetch_fsmap learned FSMap version 33 2024-09-10T08:50:58.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.051+0000 7f4ff69ed740 1 -- 172.21.15.172:0/4171584249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563f3faa1b50 con 0x563f3fa1cdc0 2024-09-10T08:50:58.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.051+0000 7f4fd5ffb640 1 -- 172.21.15.172:0/4171584249 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fdc0c0130 con 0x563f3fa1cdc0 2024-09-10T08:50:58.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.051+0000 7f4fd5ffb640 1 -- 172.21.15.172:0/4171584249 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f4fdc0b7ad0 con 0x563f3fa1cdc0 2024-09-10T08:50:58.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.051+0000 7f4ff69ed740 10 client.16806 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:50:58.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.051+0000 7f4ff69ed740 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:50:58.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.051+0000 7f4ff69ed740 20 client.16806 populate_metadata read hostname 'smithi172' 2024-09-10T08:50:58.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.051+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/4171584249 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563f3faa2390 0x563f3faa47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:58.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.051+0000 7f4ff69ed740 4 client.16806 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:50:58.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.051+0000 7f4ff69ed740 1 -- 172.21.15.172:0/4171584249 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563f3f989d60 con 0x563f3faa2390 2024-09-10T08:50:58.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.051+0000 7f4ff23b0640 1 --2- 172.21.15.172:0/4171584249 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563f3faa2390 0x563f3faa47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:58.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.052+0000 7f4fd5ffb640 10 client.16806 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:58.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.052+0000 7f4ff23b0640 1 --2- 172.21.15.172:0/4171584249 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563f3faa2390 0x563f3faa47b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82810c8ac2bad0f1 server_cookie=9594fc127a10b9d3 in_seq=0 out_seq=0 2024-09-10T08:50:58.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.052+0000 7f4fd5ffb640 1 -- 172.21.15.172:0/4171584249 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563f3f989d60 con 0x563f3faa2390 2024-09-10T08:50:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.052+0000 7f4fd5ffb640 10 client.16806 handle_command_reply: tid=0 2024-09-10T08:50:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.052+0000 7f4ff69ed740 1 client.16806 shutdown 2024-09-10T08:50:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.052+0000 7f4ff69ed740 1 -- 172.21.15.172:0/4171584249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 msgr2=0x563f3fa2cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.052+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/4171584249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa2cec0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0ed070 tx=0x7f4fdc0c4b40 comp rx=0 tx=0).stop 2024-09-10T08:50:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.052+0000 7f4ff69ed740 1 -- 172.21.15.172:0/4171584249 shutdown_connections 2024-09-10T08:50:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.052+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/4171584249 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563f3faa2390 0x563f3faa47b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.052+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/4171584249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f3fa27680 0x563f3fa2d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.052+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/4171584249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563f3fa1cdc0 0x563f3fa2cec0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.052+0000 7f4ff69ed740 1 --2- 172.21.15.172:0/4171584249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f3fa1c4a0 0x563f3fa2fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.052+0000 7f4ff69ed740 1 -- 172.21.15.172:0/4171584249 >> 172.21.15.172:0/4171584249 conn(0x563f3fa18a30 msgr2=0x563f3fa26290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.052+0000 7f4fd5ffb640 0 client.16806 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:50:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.052+0000 7f4ff69ed740 1 -- 172.21.15.172:0/4171584249 shutdown_connections 2024-09-10T08:50:58.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.052+0000 7f4ff69ed740 1 -- 172.21.15.172:0/4171584249 wait complete. 2024-09-10T08:50:58.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.052+0000 7f4ff69ed740 20 client.16806 trim_cache size 0 max 16384 2024-09-10T08:50:58.063 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:50:58.063 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T08:50:58.063 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T08:50:58.063 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:50:58.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.053+0000 7f4ff4614640 1 -- 172.21.15.172:0/8296059 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4fd80a2b80 msgr2=0x7f4fd80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:58.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.053+0000 7f4ff4614640 1 --2- 172.21.15.172:0/8296059 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4fd80a2b80 0x7f4fd80a5040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400a700 tx=0x7f4fe4009540 comp rx=0 tx=0).stop 2024-09-10T08:50:58.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.053+0000 7f4ff4614640 1 -- 172.21.15.172:0/8296059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fec102790 msgr2=0x7f4fec1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:58.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.053+0000 7f4ff4614640 1 --2- 172.21.15.172:0/8296059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fec102790 0x7f4fec1989b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc005940 tx=0x7f4fdc004270 comp rx=0 tx=0).stop 2024-09-10T08:50:58.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.053+0000 7f4ff4614640 1 -- 172.21.15.172:0/8296059 shutdown_connections 2024-09-10T08:50:58.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.053+0000 7f4ff4614640 1 --2- 172.21.15.172:0/8296059 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4fd80a2b80 0x7f4fd80a5040 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.053+0000 7f4ff4614640 1 --2- 172.21.15.172:0/8296059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fec106f10 0x7f4fec19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.053+0000 7f4ff4614640 1 --2- 172.21.15.172:0/8296059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fec102790 0x7f4fec1989b0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.053+0000 7f4ff4614640 1 --2- 172.21.15.172:0/8296059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fec101de0 0x7f4fec198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:58.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.053+0000 7f4ff4614640 1 -- 172.21.15.172:0/8296059 >> 172.21.15.172:0/8296059 conn(0x7f4fec0fd8c0 msgr2=0x7f4fec0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:58.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.053+0000 7f4ff4614640 1 -- 172.21.15.172:0/8296059 shutdown_connections 2024-09-10T08:50:58.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:58.053+0000 7f4ff4614640 1 -- 172.21.15.172:0/8296059 wait complete. 2024-09-10T08:50:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:50:58 smithi172 ceph-mon[30854]: pgmap v492: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.0 MiB/s wr, 15 op/s 2024-09-10T08:50:58.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:50:58 smithi174 ceph-mon[34068]: pgmap v492: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.0 MiB/s wr, 15 op/s 2024-09-10T08:50:58.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:50:58 smithi191 ceph-mon[35314]: pgmap v492: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.0 MiB/s wr, 15 op/s 2024-09-10T08:50:58.858 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:50:58.858 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:50:59.107 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:50:59.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.597+0000 7f5eb3f9b640 1 -- 172.21.15.172:0/483105894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eac102de0 msgr2=0x7f5eac1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:59.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.597+0000 7f5eb3f9b640 1 --2- 172.21.15.172:0/483105894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eac102de0 0x7f5eac1031c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009900 tx=0x7f5ea402a480 comp rx=0 tx=0).stop 2024-09-10T08:50:59.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.597+0000 7f5eb3f9b640 1 -- 172.21.15.172:0/483105894 shutdown_connections 2024-09-10T08:50:59.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.597+0000 7f5eb3f9b640 1 --2- 172.21.15.172:0/483105894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5eac10c180 0x7f5eac10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:59.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.597+0000 7f5eb3f9b640 1 --2- 172.21.15.172:0/483105894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eac103700 0x7f5eac10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:59.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.597+0000 7f5eb3f9b640 1 --2- 172.21.15.172:0/483105894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eac102de0 0x7f5eac1031c0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:59.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.597+0000 7f5eb3f9b640 1 -- 172.21.15.172:0/483105894 >> 172.21.15.172:0/483105894 conn(0x7f5eac0fc7a0 msgr2=0x7f5eac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:59.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.597+0000 7f5eb3f9b640 1 -- 172.21.15.172:0/483105894 shutdown_connections 2024-09-10T08:50:59.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.597+0000 7f5eb3f9b640 1 -- 172.21.15.172:0/483105894 wait complete. 2024-09-10T08:50:59.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.597+0000 7f5eb3f9b640 1 Processor -- start 2024-09-10T08:50:59.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eb3f9b640 1 -- start start 2024-09-10T08:50:59.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eb3f9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eac102de0 0x7f5eac198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:59.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eb3f9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eac103700 0x7f5eac1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:59.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eb3f9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5eac10c180 0x7f5eac19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:59.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eb3f9b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac111e60 con 0x7f5eac102de0 2024-09-10T08:50:59.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eb3f9b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac111ce0 con 0x7f5eac103700 2024-09-10T08:50:59.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eb3f9b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac111fe0 con 0x7f5eac10c180 2024-09-10T08:50:59.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eb1d37640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eac102de0 0x7f5eac198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:59.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eb1d37640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eac102de0 0x7f5eac198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55474/0 (socket says 172.21.15.172:55474) 2024-09-10T08:50:59.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eb1d37640 1 -- 172.21.15.172:0/1565539467 learned_addr learned my addr 172.21.15.172:0/1565539467 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:50:59.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eb1d37640 1 -- 172.21.15.172:0/1565539467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5eac10c180 msgr2=0x7f5eac19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:59.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eb1d37640 1 --2- 172.21.15.172:0/1565539467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5eac10c180 0x7f5eac19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:59.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eb1d37640 1 -- 172.21.15.172:0/1565539467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eac103700 msgr2=0x7f5eac1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:59.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eb1d37640 1 --2- 172.21.15.172:0/1565539467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eac103700 0x7f5eac1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:59.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eb1d37640 1 -- 172.21.15.172:0/1565539467 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac19d310 con 0x7f5eac102de0 2024-09-10T08:50:59.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eb1d37640 1 --2- 172.21.15.172:0/1565539467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eac102de0 0x7f5eac198360 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4006010 tx=0x7f5ea4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:59.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eaaffd640 1 -- 172.21.15.172:0/1565539467 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea403f030 con 0x7f5eac102de0 2024-09-10T08:50:59.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eaaffd640 1 -- 172.21.15.172:0/1565539467 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ea40035f0 con 0x7f5eac102de0 2024-09-10T08:50:59.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.598+0000 7f5eb3f9b640 1 -- 172.21.15.172:0/1565539467 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eac19d5a0 con 0x7f5eac102de0 2024-09-10T08:50:59.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.599+0000 7f5eaaffd640 1 -- 172.21.15.172:0/1565539467 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4035070 con 0x7f5eac102de0 2024-09-10T08:50:59.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.599+0000 7f5eb3f9b640 1 -- 172.21.15.172:0/1565539467 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eac1a4ee0 con 0x7f5eac102de0 2024-09-10T08:50:59.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.600+0000 7f5eb3f9b640 1 -- 172.21.15.172:0/1565539467 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eac06afd0 con 0x7f5eac102de0 2024-09-10T08:50:59.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.601+0000 7f5eaaffd640 1 -- 172.21.15.172:0/1565539467 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5ea4003b10 con 0x7f5eac102de0 2024-09-10T08:50:59.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.601+0000 7f5eaaffd640 1 --2- 172.21.15.172:0/1565539467 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5e940a2b80 0x7f5e940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:50:59.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.601+0000 7f5eb1536640 1 --2- 172.21.15.172:0/1565539467 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5e940a2b80 0x7f5e940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:50:59.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.601+0000 7f5eaaffd640 1 -- 172.21.15.172:0/1565539467 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5ea40effd0 con 0x7f5eac102de0 2024-09-10T08:50:59.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.602+0000 7f5eb1536640 1 --2- 172.21.15.172:0/1565539467 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5e940a2b80 0x7f5e940a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0feda0 tx=0x7f5ea0009250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:50:59.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.605+0000 7f5eaaffd640 1 -- 172.21.15.172:0/1565539467 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ea40bcbd0 con 0x7f5eac102de0 2024-09-10T08:50:59.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.806+0000 7f5eb3f9b640 1 -- 172.21.15.172:0/1565539467 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5eac107240 con 0x7f5eac102de0 2024-09-10T08:50:59.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.807+0000 7f5eaaffd640 1 -- 172.21.15.172:0/1565539467 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+12007 (secure 0 0 0) 0x7f5ea40bc4d0 con 0x7f5eac102de0 2024-09-10T08:50:59.808 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:50:59.809 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":33,"btime":"2024-09-10T08:50:03:285217+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:50:02.654978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:50:59.809 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 33 2024-09-10T08:50:59.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.809+0000 7f5eb3f9b640 1 -- 172.21.15.172:0/1565539467 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5e940a2b80 msgr2=0x7f5e940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:59.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.809+0000 7f5eb3f9b640 1 --2- 172.21.15.172:0/1565539467 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5e940a2b80 0x7f5e940a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0feda0 tx=0x7f5ea0009250 comp rx=0 tx=0).stop 2024-09-10T08:50:59.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.809+0000 7f5eb3f9b640 1 -- 172.21.15.172:0/1565539467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eac102de0 msgr2=0x7f5eac198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:50:59.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.809+0000 7f5eb3f9b640 1 --2- 172.21.15.172:0/1565539467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eac102de0 0x7f5eac198360 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4006010 tx=0x7f5ea4004270 comp rx=0 tx=0).stop 2024-09-10T08:50:59.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.809+0000 7f5eb3f9b640 1 -- 172.21.15.172:0/1565539467 shutdown_connections 2024-09-10T08:50:59.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.809+0000 7f5eb3f9b640 1 --2- 172.21.15.172:0/1565539467 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5e940a2b80 0x7f5e940a5040 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:59.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.809+0000 7f5eb3f9b640 1 --2- 172.21.15.172:0/1565539467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5eac10c180 0x7f5eac19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:59.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.809+0000 7f5eb3f9b640 1 --2- 172.21.15.172:0/1565539467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eac103700 0x7f5eac1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:59.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.809+0000 7f5eb3f9b640 1 --2- 172.21.15.172:0/1565539467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eac102de0 0x7f5eac198360 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:50:59.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.809+0000 7f5eb3f9b640 1 -- 172.21.15.172:0/1565539467 >> 172.21.15.172:0/1565539467 conn(0x7f5eac0fc7a0 msgr2=0x7f5eac0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:50:59.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.809+0000 7f5eb3f9b640 1 -- 172.21.15.172:0/1565539467 shutdown_connections 2024-09-10T08:50:59.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:50:59.810+0000 7f5eb3f9b640 1 -- 172.21.15.172:0/1565539467 wait complete. 2024-09-10T08:51:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:00 smithi172 ceph-mon[30854]: pgmap v493: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-09-10T08:51:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:00 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1565539467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:51:00.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:00 smithi174 ceph-mon[34068]: pgmap v493: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-09-10T08:51:00.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:00 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1565539467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:51:00.776 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T08:51:00.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:00 smithi191 ceph-mon[35314]: pgmap v493: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.0 MiB/s wr, 14 op/s 2024-09-10T08:51:00.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:00 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1565539467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:51:01.024 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:51:01.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.532+0000 7f33c5490640 1 -- 172.21.15.172:0/794241323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c0100e20 msgr2=0x7f33c0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:01.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.532+0000 7f33c5490640 1 --2- 172.21.15.172:0/794241323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c0100e20 0x7f33c0101200 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009900 tx=0x7f33b002a480 comp rx=0 tx=0).stop 2024-09-10T08:51:01.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.532+0000 7f33c5490640 1 -- 172.21.15.172:0/794241323 shutdown_connections 2024-09-10T08:51:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.532+0000 7f33c5490640 1 --2- 172.21.15.172:0/794241323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33c0102ef0 0x7f33c010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.532+0000 7f33c5490640 1 --2- 172.21.15.172:0/794241323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33c0102570 0x7f33c01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.532+0000 7f33c5490640 1 --2- 172.21.15.172:0/794241323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c0100e20 0x7f33c0101200 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.532+0000 7f33c5490640 1 -- 172.21.15.172:0/794241323 >> 172.21.15.172:0/794241323 conn(0x7f33c00fc7d0 msgr2=0x7f33c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.532+0000 7f33c5490640 1 -- 172.21.15.172:0/794241323 shutdown_connections 2024-09-10T08:51:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.532+0000 7f33c5490640 1 -- 172.21.15.172:0/794241323 wait complete. 2024-09-10T08:51:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.532+0000 7f33c5490640 1 Processor -- start 2024-09-10T08:51:01.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.533+0000 7f33c5490640 1 -- start start 2024-09-10T08:51:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.533+0000 7f33c5490640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33c0100e20 0x7f33c0199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.533+0000 7f33c5490640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c0102570 0x7f33c01997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.533+0000 7f33c5490640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33c0102ef0 0x7f33c01a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.533+0000 7f33c5490640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f33c0110e00 con 0x7f33c0102570 2024-09-10T08:51:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.533+0000 7f33c5490640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33c0110c80 con 0x7f33c0102ef0 2024-09-10T08:51:01.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.533+0000 7f33c5490640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f33c0110f80 con 0x7f33c0100e20 2024-09-10T08:51:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.533+0000 7f33b67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c0102570 0x7f33c01997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.533+0000 7f33b67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c0102570 0x7f33c01997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55484/0 (socket says 172.21.15.172:55484) 2024-09-10T08:51:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.533+0000 7f33b67fc640 1 -- 172.21.15.172:0/402760450 learned_addr learned my addr 172.21.15.172:0/402760450 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.533+0000 7f33beffd640 1 --2- 172.21.15.172:0/402760450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33c0100e20 0x7f33c0199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.533+0000 7f33b67fc640 1 -- 172.21.15.172:0/402760450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33c0100e20 msgr2=0x7f33c0199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.533+0000 7f33b67fc640 1 --2- 172.21.15.172:0/402760450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33c0100e20 0x7f33c0199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.533+0000 7f33b67fc640 1 -- 172.21.15.172:0/402760450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33c0102ef0 msgr2=0x7f33c01a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:01.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.533+0000 7f33b67fc640 1 --2- 172.21.15.172:0/402760450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33c0102ef0 0x7f33c01a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.533+0000 7f33b67fc640 1 -- 172.21.15.172:0/402760450 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c01a4830 con 0x7f33c0102570 2024-09-10T08:51:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.534+0000 7f33b67fc640 1 --2- 172.21.15.172:0/402760450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c0102570 0x7f33c01997b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f33a800c920 tx=0x7f33a800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.534+0000 7f33bcff9640 1 -- 172.21.15.172:0/402760450 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8007c70 con 0x7f33c0102570 2024-09-10T08:51:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.534+0000 7f33c5490640 1 -- 172.21.15.172:0/402760450 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c01a4a00 con 0x7f33c0102570 2024-09-10T08:51:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.534+0000 7f33bcff9640 1 -- 172.21.15.172:0/402760450 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33a8007e10 con 0x7f33c0102570 2024-09-10T08:51:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.534+0000 7f33bcff9640 1 -- 172.21.15.172:0/402760450 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8018820 con 0x7f33c0102570 2024-09-10T08:51:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.534+0000 7f33c5490640 1 -- 172.21.15.172:0/402760450 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c01a4f40 con 0x7f33c0102570 2024-09-10T08:51:01.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.536+0000 7f33bcff9640 1 -- 172.21.15.172:0/402760450 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f33a80189c0 con 0x7f33c0102570 2024-09-10T08:51:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.536+0000 7f33c7869740 1 Processor -- start 2024-09-10T08:51:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33c7869740 1 -- start start 2024-09-10T08:51:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33c7869740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd6b40 0x565316dd5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33c7869740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316dd55d0 0x565316dd5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33c7869740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316de0370 0x565316de2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33c7869740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x565316d3b2f0 con 0x565316de0370 2024-09-10T08:51:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33c7869740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565316ce2cf0 con 0x565316dd6b40 2024-09-10T08:51:01.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33c7869740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x565316d372b0 con 0x565316dd55d0 2024-09-10T08:51:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33b67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316de0370 0x565316de2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33bcff9640 1 --2- 172.21.15.172:0/402760450 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f33ac0a2b80 0x7f33ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33b67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316de0370 0x565316de2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55488/0 (socket says 172.21.15.172:55488) 2024-09-10T08:51:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33b67fc640 1 -- 172.21.15.172:0/1890777925 learned_addr learned my addr 172.21.15.172:0/1890777925 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33bf7fe640 1 --2- 172.21.15.172:0/1890777925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd6b40 0x565316dd5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33beffd640 1 --2- 172.21.15.172:0/1890777925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316dd55d0 0x565316dd5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:01.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33bf7fe640 1 --2- 172.21.15.172:0/402760450 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f33ac0a2b80 0x7f33ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:01.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33bcff9640 1 -- 172.21.15.172:0/402760450 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f33a80cf5c0 con 0x7f33c0102570 2024-09-10T08:51:01.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33b67fc640 1 -- 172.21.15.172:0/1890777925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316dd55d0 msgr2=0x565316dd5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:01.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33b67fc640 1 --2- 172.21.15.172:0/1890777925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316dd55d0 0x565316dd5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33b67fc640 1 -- 172.21.15.172:0/1890777925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd6b40 msgr2=0x565316dd5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:01.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33b67fc640 1 --2- 172.21.15.172:0/1890777925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd6b40 0x565316dd5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.537+0000 7f33b67fc640 1 -- 172.21.15.172:0/1890777925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565316cc73a0 con 0x565316de0370 2024-09-10T08:51:01.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.538+0000 7f33b67fc640 1 --2- 172.21.15.172:0/1890777925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316de0370 0x565316de2760 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f33a8098490 tx=0x7f33a8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a981e421f74c12e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:01.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.538+0000 7f33bf7fe640 1 --2- 172.21.15.172:0/402760450 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f33ac0a2b80 0x7f33ac0a5040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f33b8005c40 tx=0x7f33b8009200 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:01.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.538+0000 7f33b57fa640 1 -- 172.21.15.172:0/1890777925 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8018cb0 con 0x565316de0370 2024-09-10T08:51:01.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.538+0000 7f33b57fa640 1 -- 172.21.15.172:0/1890777925 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33a809cdf0 con 0x565316de0370 2024-09-10T08:51:01.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.538+0000 7f33b57fa640 1 -- 172.21.15.172:0/1890777925 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a80cf360 con 0x565316de0370 2024-09-10T08:51:01.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 1 -- 172.21.15.172:0/1890777925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316de0370 msgr2=0x565316de2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:01.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 1 --2- 172.21.15.172:0/1890777925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316de0370 0x565316de2760 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f33a8098490 tx=0x7f33a8000bc0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 1 -- 172.21.15.172:0/1890777925 shutdown_connections 2024-09-10T08:51:01.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 1 --2- 172.21.15.172:0/1890777925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316de0370 0x565316de2760 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 1 --2- 172.21.15.172:0/1890777925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316dd55d0 0x565316dd5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 1 --2- 172.21.15.172:0/1890777925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd6b40 0x565316dd5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 1 -- 172.21.15.172:0/1890777925 >> 172.21.15.172:0/1890777925 conn(0x565316d9e310 msgr2=0x565316dd1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:01.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 1 -- 172.21.15.172:0/1890777925 shutdown_connections 2024-09-10T08:51:01.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 1 -- 172.21.15.172:0/1890777925 wait complete. 2024-09-10T08:51:01.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 1 -- start start 2024-09-10T08:51:01.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 1 Processor -- start 2024-09-10T08:51:01.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd55d0 0x565316ddfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316dd6b40 0x565316ddd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316de0370 0x565316ddd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x565316de3010 con 0x565316dd6b40 2024-09-10T08:51:01.591 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565316cf6060 con 0x565316dd55d0 2024-09-10T08:51:01.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x565316de3190 con 0x565316de0370 2024-09-10T08:51:01.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33c7869740 10 client.0 _ll_register_callbacks cb 0x565316a498c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:01.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33b67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316dd6b40 0x565316ddd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:01.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33b67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316dd6b40 0x565316ddd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55494/0 (socket says 172.21.15.172:55494) 2024-09-10T08:51:01.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.539+0000 7f33b67fc640 1 -- 172.21.15.172:0/1996395549 learned_addr learned my addr 172.21.15.172:0/1996395549 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:01.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.540+0000 7f33beffd640 1 --2- 172.21.15.172:0/1996395549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd55d0 0x565316ddfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:01.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.540+0000 7f33bf7fe640 1 --2- 172.21.15.172:0/1996395549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316de0370 0x565316ddd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:01.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.540+0000 7f33b67fc640 1 -- 172.21.15.172:0/1996395549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316de0370 msgr2=0x565316ddd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:01.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.540+0000 7f33b67fc640 1 --2- 172.21.15.172:0/1996395549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316de0370 0x565316ddd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.540+0000 7f33b67fc640 1 -- 172.21.15.172:0/1996395549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd55d0 msgr2=0x565316ddfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:01.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.540+0000 7f33b67fc640 1 --2- 172.21.15.172:0/1996395549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd55d0 0x565316ddfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.540+0000 7f33b67fc640 1 -- 172.21.15.172:0/1996395549 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565316aac3c0 con 0x565316dd6b40 2024-09-10T08:51:01.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.540+0000 7f33b67fc640 1 --2- 172.21.15.172:0/1996395549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316dd6b40 0x565316ddd050 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f33a8097f20 tx=0x7f33a8097de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc4176826a9b05b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:01.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.540+0000 7f33a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:01.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.540+0000 7f33a67fc640 1 -- 172.21.15.172:0/1996395549 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a80b9060 con 0x565316dd6b40 2024-09-10T08:51:01.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.540+0000 7f33a67fc640 1 -- 172.21.15.172:0/1996395549 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f33a8099dc0 con 0x565316dd6b40 2024-09-10T08:51:01.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.540+0000 7f33c7869740 1 -- 172.21.15.172:0/1996395549 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565316d48860 con 0x565316dd6b40 2024-09-10T08:51:01.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.540+0000 7f33a67fc640 1 -- 172.21.15.172:0/1996395549 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f33a80b4070 con 0x565316dd6b40 2024-09-10T08:51:01.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.540+0000 7f33c7869740 10 client.16824 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:01.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.540+0000 7f33c7869740 1 -- 172.21.15.172:0/1996395549 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x565316deeb20 con 0x565316dd6b40 2024-09-10T08:51:01.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.540+0000 7f33a67fc640 1 -- 172.21.15.172:0/1996395549 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x565316d48860 con 0x565316dd6b40 2024-09-10T08:51:01.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.541+0000 7f33a67fc640 1 -- 172.21.15.172:0/1996395549 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f33a809bde0 con 0x565316dd6b40 2024-09-10T08:51:01.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.541+0000 7f33c7869740 10 client.16824 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:01.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.541+0000 7f33c7869740 10 client.16824 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:51:01.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.541+0000 7f33c7869740 20 client.16824 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:01.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.541+0000 7f33c7869740 1 --2- 172.21.15.172:0/1996395549 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x565316defe70 0x565316df2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.541+0000 7f33c7869740 4 client.16824 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:51:01.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.541+0000 7f33c7869740 1 -- 172.21.15.172:0/1996395549 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565316d42d70 con 0x565316defe70 2024-09-10T08:51:01.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.541+0000 7f33beffd640 1 --2- 172.21.15.172:0/1996395549 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x565316defe70 0x565316df2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:01.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.541+0000 7f33a67fc640 10 client.16824 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:51:01.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.541+0000 7f33beffd640 1 --2- 172.21.15.172:0/1996395549 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x565316defe70 0x565316df2260 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7736fd251970de39 server_cookie=fb5135a633f1898e in_seq=0 out_seq=0 2024-09-10T08:51:01.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.542+0000 7f33a67fc640 1 -- 172.21.15.172:0/1996395549 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565316d42d70 con 0x565316defe70 2024-09-10T08:51:01.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.542+0000 7f33a67fc640 10 client.16824 handle_command_reply: tid=0 2024-09-10T08:51:01.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.542+0000 7f33c7869740 1 client.16824 shutdown 2024-09-10T08:51:01.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.542+0000 7f33c7869740 1 -- 172.21.15.172:0/1996395549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316dd6b40 msgr2=0x565316ddd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:01.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.542+0000 7f33c7869740 1 --2- 172.21.15.172:0/1996395549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316dd6b40 0x565316ddd050 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f33a8097f20 tx=0x7f33a8097de0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.542+0000 7f33c7869740 1 -- 172.21.15.172:0/1996395549 shutdown_connections 2024-09-10T08:51:01.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.542+0000 7f33c7869740 1 --2- 172.21.15.172:0/1996395549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316de0370 0x565316ddd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.542+0000 7f33c7869740 1 --2- 172.21.15.172:0/1996395549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316dd6b40 0x565316ddd050 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.542+0000 7f33c7869740 1 --2- 172.21.15.172:0/1996395549 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x565316defe70 0x565316df2260 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.542+0000 7f33c7869740 1 --2- 172.21.15.172:0/1996395549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd55d0 0x565316ddfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.542+0000 7f33a67fc640 0 client.16824 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:51:01.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.542+0000 7f33c7869740 1 -- 172.21.15.172:0/1996395549 >> 172.21.15.172:0/1996395549 conn(0x565316d9e310 msgr2=0x565316dd7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:01.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.542+0000 7f33c7869740 1 -- 172.21.15.172:0/1996395549 shutdown_connections 2024-09-10T08:51:01.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.542+0000 7f33c7869740 1 -- 172.21.15.172:0/1996395549 wait complete. 2024-09-10T08:51:01.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.543+0000 7f33c7869740 20 client.16824 trim_cache size 0 max 16384 2024-09-10T08:51:01.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.555+0000 7f33c7869740 1 Processor -- start 2024-09-10T08:51:01.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.555+0000 7f33c7869740 1 -- start start 2024-09-10T08:51:01.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.555+0000 7f33c7869740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316dd55d0 0x565316ddfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.555+0000 7f33c7869740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd6b40 0x565316dd8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.555+0000 7f33c7869740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316de0370 0x565316dd8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.555+0000 7f33c7869740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f33c01aeab0 con 0x565316de0370 2024-09-10T08:51:01.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.556+0000 7f33c7869740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33c01b06b0 con 0x565316dd6b40 2024-09-10T08:51:01.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.556+0000 7f33c7869740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f33c01af180 con 0x565316dd55d0 2024-09-10T08:51:01.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.556+0000 7f33bf7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316de0370 0x565316dd8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:01.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.556+0000 7f33bf7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316de0370 0x565316dd8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55504/0 (socket says 172.21.15.172:55504) 2024-09-10T08:51:01.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.556+0000 7f33bf7fe640 1 -- 172.21.15.172:0/3119006837 learned_addr learned my addr 172.21.15.172:0/3119006837 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:01.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.556+0000 7f33b67fc640 1 --2- 172.21.15.172:0/3119006837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd6b40 0x565316dd8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:01.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.556+0000 7f33beffd640 1 --2- 172.21.15.172:0/3119006837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316dd55d0 0x565316ddfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:01.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.556+0000 7f33bf7fe640 1 -- 172.21.15.172:0/3119006837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316dd55d0 msgr2=0x565316ddfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:01.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.556+0000 7f33bf7fe640 1 --2- 172.21.15.172:0/3119006837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316dd55d0 0x565316ddfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.556+0000 7f33bf7fe640 1 -- 172.21.15.172:0/3119006837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd6b40 msgr2=0x565316dd8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:01.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.556+0000 7f33bf7fe640 1 --2- 172.21.15.172:0/3119006837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd6b40 0x565316dd8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.556+0000 7f33bf7fe640 1 -- 172.21.15.172:0/3119006837 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565316ddb5a0 con 0x565316de0370 2024-09-10T08:51:01.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.556+0000 7f33bf7fe640 1 --2- 172.21.15.172:0/3119006837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316de0370 0x565316dd8660 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f33b8009470 tx=0x7f33b800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2f438b2b3b5d5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:01.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.556+0000 7f33b4ff9640 1 -- 172.21.15.172:0/3119006837 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8004b20 con 0x565316de0370 2024-09-10T08:51:01.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.556+0000 7f33b4ff9640 1 -- 172.21.15.172:0/3119006837 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33b8004cc0 con 0x565316de0370 2024-09-10T08:51:01.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.556+0000 7f33b4ff9640 1 -- 172.21.15.172:0/3119006837 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8027700 con 0x565316de0370 2024-09-10T08:51:01.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.557+0000 7f33c7869740 1 -- 172.21.15.172:0/3119006837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316de0370 msgr2=0x565316dd8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:01.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.557+0000 7f33c7869740 1 --2- 172.21.15.172:0/3119006837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316de0370 0x565316dd8660 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f33b8009470 tx=0x7f33b800c000 comp rx=0 tx=0).stop 2024-09-10T08:51:01.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.557+0000 7f33c7869740 1 -- 172.21.15.172:0/3119006837 shutdown_connections 2024-09-10T08:51:01.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.557+0000 7f33c7869740 1 --2- 172.21.15.172:0/3119006837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316de0370 0x565316dd8660 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.557+0000 7f33c7869740 1 --2- 172.21.15.172:0/3119006837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd6b40 0x565316dd8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.557+0000 7f33c7869740 1 --2- 172.21.15.172:0/3119006837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316dd55d0 0x565316ddfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.557+0000 7f33c7869740 1 -- 172.21.15.172:0/3119006837 >> 172.21.15.172:0/3119006837 conn(0x565316d9e310 msgr2=0x565316dcca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:01.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.557+0000 7f33c7869740 1 -- 172.21.15.172:0/3119006837 shutdown_connections 2024-09-10T08:51:01.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.557+0000 7f33c7869740 1 -- 172.21.15.172:0/3119006837 wait complete. 2024-09-10T08:51:01.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.557+0000 7f33c7869740 1 -- start start 2024-09-10T08:51:01.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.557+0000 7f33c7869740 1 Processor -- start 2024-09-10T08:51:01.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33c7869740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd55d0 0x565316df56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33c7869740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316dd6b40 0x565316df2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33c7869740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316de0370 0x565316df2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33c7869740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f33c01aff50 con 0x565316dd6b40 2024-09-10T08:51:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33c7869740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33c01af880 con 0x565316dd55d0 2024-09-10T08:51:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33c7869740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f33c01aca90 con 0x565316de0370 2024-09-10T08:51:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33c7869740 10 client.0 _ll_register_callbacks cb 0x7f33b8009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33b67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316dd6b40 0x565316df2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33b67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316dd6b40 0x565316df2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55510/0 (socket says 172.21.15.172:55510) 2024-09-10T08:51:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33b67fc640 1 -- 172.21.15.172:0/1342072353 learned_addr learned my addr 172.21.15.172:0/1342072353 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33beffd640 1 --2- 172.21.15.172:0/1342072353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd55d0 0x565316df56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33b67fc640 1 -- 172.21.15.172:0/1342072353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316de0370 msgr2=0x565316df2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33b67fc640 1 --2- 172.21.15.172:0/1342072353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316de0370 0x565316df2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33b67fc640 1 -- 172.21.15.172:0/1342072353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd55d0 msgr2=0x565316df56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:01.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33b67fc640 1 --2- 172.21.15.172:0/1342072353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd55d0 0x565316df56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33b67fc640 1 -- 172.21.15.172:0/1342072353 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565316df34c0 con 0x565316dd6b40 2024-09-10T08:51:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33b67fc640 1 --2- 172.21.15.172:0/1342072353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316dd6b40 0x565316df2a40 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f33a8098440 tx=0x7f33a800fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbbe7185394c5f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33a67fc640 1 -- 172.21.15.172:0/1342072353 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a809b890 con 0x565316dd6b40 2024-09-10T08:51:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33a67fc640 1 -- 172.21.15.172:0/1342072353 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f33a8099dc0 con 0x565316dd6b40 2024-09-10T08:51:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.558+0000 7f33c7869740 1 -- 172.21.15.172:0/1342072353 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33b8004cc0 con 0x565316dd6b40 2024-09-10T08:51:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.559+0000 7f33a67fc640 1 -- 172.21.15.172:0/1342072353 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f33a807e050 con 0x565316dd6b40 2024-09-10T08:51:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.559+0000 7f33c7869740 10 client.16830 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.559+0000 7f33c7869740 1 -- 172.21.15.172:0/1342072353 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x565316e5a4c0 con 0x565316dd6b40 2024-09-10T08:51:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.559+0000 7f33a67fc640 1 -- 172.21.15.172:0/1342072353 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33b8004cc0 con 0x565316dd6b40 2024-09-10T08:51:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.559+0000 7f33a67fc640 1 -- 172.21.15.172:0/1342072353 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f33a809a980 con 0x565316dd6b40 2024-09-10T08:51:01.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.559+0000 7f33c7869740 10 client.16830 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:01.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.559+0000 7f33c7869740 10 client.16830 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:51:01.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.559+0000 7f33c7869740 20 client.16830 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:01.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.559+0000 7f33c7869740 1 --2- 172.21.15.172:0/1342072353 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x565316e5aa20 0x565316e5ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:01.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.559+0000 7f33c7869740 4 client.16830 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:51:01.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.559+0000 7f33c7869740 1 -- 172.21.15.172:0/1342072353 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565316cbf170 con 0x565316e5aa20 2024-09-10T08:51:01.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.559+0000 7f33beffd640 1 --2- 172.21.15.172:0/1342072353 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x565316e5aa20 0x565316e5ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:01.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.560+0000 7f33beffd640 1 --2- 172.21.15.172:0/1342072353 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x565316e5aa20 0x565316e5ce40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6103eff08eed18c7 server_cookie=e86bf738eebb6205 in_seq=0 out_seq=0 2024-09-10T08:51:01.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.560+0000 7f33a67fc640 10 client.16830 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:51:01.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.560+0000 7f33a67fc640 1 -- 172.21.15.172:0/1342072353 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565316cbf170 con 0x565316e5aa20 2024-09-10T08:51:01.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.560+0000 7f33a67fc640 10 client.16830 handle_command_reply: tid=0 2024-09-10T08:51:01.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.560+0000 7f33c7869740 1 client.16830 shutdown 2024-09-10T08:51:01.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.560+0000 7f33c7869740 1 -- 172.21.15.172:0/1342072353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316dd6b40 msgr2=0x565316df2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:01.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.560+0000 7f33c7869740 1 --2- 172.21.15.172:0/1342072353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316dd6b40 0x565316df2a40 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f33a8098440 tx=0x7f33a800fe00 comp rx=0 tx=0).stop 2024-09-10T08:51:01.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.560+0000 7f33c7869740 1 -- 172.21.15.172:0/1342072353 shutdown_connections 2024-09-10T08:51:01.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.560+0000 7f33c7869740 1 --2- 172.21.15.172:0/1342072353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565316de0370 0x565316df2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.560+0000 7f33c7869740 1 --2- 172.21.15.172:0/1342072353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565316dd6b40 0x565316df2a40 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.560+0000 7f33c7869740 1 --2- 172.21.15.172:0/1342072353 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x565316e5aa20 0x565316e5ce40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.560+0000 7f33c7869740 1 --2- 172.21.15.172:0/1342072353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565316dd55d0 0x565316df56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.560+0000 7f33a67fc640 0 client.16830 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:51:01.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.560+0000 7f33c7869740 1 -- 172.21.15.172:0/1342072353 >> 172.21.15.172:0/1342072353 conn(0x565316d9e310 msgr2=0x565316ddf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:01.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.560+0000 7f33c7869740 1 -- 172.21.15.172:0/1342072353 shutdown_connections 2024-09-10T08:51:01.602 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:51:01.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.560+0000 7f33c7869740 1 -- 172.21.15.172:0/1342072353 wait complete. 2024-09-10T08:51:01.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.560+0000 7f33c7869740 20 client.16830 trim_cache size 0 max 16384 2024-09-10T08:51:01.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.561+0000 7f33c5490640 1 -- 172.21.15.172:0/402760450 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f33ac0a2b80 msgr2=0x7f33ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:01.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.561+0000 7f33c5490640 1 --2- 172.21.15.172:0/402760450 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f33ac0a2b80 0x7f33ac0a5040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f33b8005c40 tx=0x7f33b8009200 comp rx=0 tx=0).stop 2024-09-10T08:51:01.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.561+0000 7f33c5490640 1 -- 172.21.15.172:0/402760450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c0102570 msgr2=0x7f33c01997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:01.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.561+0000 7f33c5490640 1 --2- 172.21.15.172:0/402760450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c0102570 0x7f33c01997b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f33a800c920 tx=0x7f33a800cde0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.561+0000 7f33c5490640 1 -- 172.21.15.172:0/402760450 shutdown_connections 2024-09-10T08:51:01.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.561+0000 7f33c5490640 1 --2- 172.21.15.172:0/402760450 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f33ac0a2b80 0x7f33ac0a5040 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.561+0000 7f33c5490640 1 --2- 172.21.15.172:0/402760450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33c0102ef0 0x7f33c01a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.561+0000 7f33c5490640 1 --2- 172.21.15.172:0/402760450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c0102570 0x7f33c01997b0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.561+0000 7f33c5490640 1 --2- 172.21.15.172:0/402760450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33c0100e20 0x7f33c0199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:01.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.561+0000 7f33c5490640 1 -- 172.21.15.172:0/402760450 >> 172.21.15.172:0/402760450 conn(0x7f33c00fc7d0 msgr2=0x7f33c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:01.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.561+0000 7f33c5490640 1 -- 172.21.15.172:0/402760450 shutdown_connections 2024-09-10T08:51:01.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:01.561+0000 7f33c5490640 1 -- 172.21.15.172:0/402760450 wait complete. 2024-09-10T08:51:02.358 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T08:51:02.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:02 smithi172 ceph-mon[30854]: pgmap v494: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.2 MiB/s wr, 16 op/s 2024-09-10T08:51:02.602 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:02 smithi191 ceph-mon[35314]: pgmap v494: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.2 MiB/s wr, 16 op/s 2024-09-10T08:51:02.607 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:51:02.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:02 smithi174 ceph-mon[34068]: pgmap v494: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.2 MiB/s wr, 16 op/s 2024-09-10T08:51:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.075+0000 7f39af93c640 1 -- 172.21.15.172:0/3593328816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a81036e0 msgr2=0x7f39a8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.075+0000 7f39af93c640 1 --2- 172.21.15.172:0/3593328816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a81036e0 0x7f39a8107b90 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f399c009930 tx=0x7f399c02a420 comp rx=0 tx=0).stop 2024-09-10T08:51:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.075+0000 7f39af93c640 1 -- 172.21.15.172:0/3593328816 shutdown_connections 2024-09-10T08:51:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.075+0000 7f39af93c640 1 --2- 172.21.15.172:0/3593328816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39a81082a0 0x7f39a810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.075+0000 7f39af93c640 1 --2- 172.21.15.172:0/3593328816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a81036e0 0x7f39a8107b90 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.075+0000 7f39af93c640 1 --2- 172.21.15.172:0/3593328816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a8102d30 0x7f39a8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.076+0000 7f39af93c640 1 -- 172.21.15.172:0/3593328816 >> 172.21.15.172:0/3593328816 conn(0x7f39a80fc7d0 msgr2=0x7f39a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.076+0000 7f39af93c640 1 -- 172.21.15.172:0/3593328816 shutdown_connections 2024-09-10T08:51:03.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.076+0000 7f39af93c640 1 -- 172.21.15.172:0/3593328816 wait complete. 2024-09-10T08:51:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.076+0000 7f39af93c640 1 Processor -- start 2024-09-10T08:51:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.076+0000 7f39af93c640 1 -- start start 2024-09-10T08:51:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.076+0000 7f39af93c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a8102d30 0x7f39a81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.076+0000 7f39af93c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a81036e0 0x7f39a8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.076+0000 7f39af93c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39a81082a0 0x7f39a819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.076+0000 7f39af93c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f39a8111f70 con 0x7f39a8102d30 2024-09-10T08:51:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.076+0000 7f39af93c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f39a8111df0 con 0x7f39a81036e0 2024-09-10T08:51:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.076+0000 7f39af93c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f39a81120f0 con 0x7f39a81082a0 2024-09-10T08:51:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.077+0000 7f39ad6d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a8102d30 0x7f39a81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.077+0000 7f39ad6d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a8102d30 0x7f39a81985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55524/0 (socket says 172.21.15.172:55524) 2024-09-10T08:51:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.077+0000 7f39ad6d8640 1 -- 172.21.15.172:0/3123984671 learned_addr learned my addr 172.21.15.172:0/3123984671 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:03.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.077+0000 7f39aced7640 1 --2- 172.21.15.172:0/3123984671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a81036e0 0x7f39a8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:03.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.077+0000 7f39aded9640 1 --2- 172.21.15.172:0/3123984671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39a81082a0 0x7f39a819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:03.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.077+0000 7f39ad6d8640 1 -- 172.21.15.172:0/3123984671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39a81082a0 msgr2=0x7f39a819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:03.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.077+0000 7f39ad6d8640 1 --2- 172.21.15.172:0/3123984671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39a81082a0 0x7f39a819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.077+0000 7f39ad6d8640 1 -- 172.21.15.172:0/3123984671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a81036e0 msgr2=0x7f39a8198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:03.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.077+0000 7f39ad6d8640 1 --2- 172.21.15.172:0/3123984671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a81036e0 0x7f39a8198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.077+0000 7f39ad6d8640 1 -- 172.21.15.172:0/3123984671 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a819d550 con 0x7f39a8102d30 2024-09-10T08:51:03.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.077+0000 7f39aded9640 1 --2- 172.21.15.172:0/3123984671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39a81082a0 0x7f39a819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:03.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.077+0000 7f39ad6d8640 1 --2- 172.21.15.172:0/3123984671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a8102d30 0x7f39a81985a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f399800cce0 tx=0x7f399800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:03.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.077+0000 7f39a67fc640 1 -- 172.21.15.172:0/3123984671 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998004b20 con 0x7f39a8102d30 2024-09-10T08:51:03.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.077+0000 7f39af93c640 1 -- 172.21.15.172:0/3123984671 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a819d840 con 0x7f39a8102d30 2024-09-10T08:51:03.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.077+0000 7f39a67fc640 1 -- 172.21.15.172:0/3123984671 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3998004cc0 con 0x7f39a8102d30 2024-09-10T08:51:03.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.077+0000 7f39a67fc640 1 -- 172.21.15.172:0/3123984671 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399800f6b0 con 0x7f39a8102d30 2024-09-10T08:51:03.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.077+0000 7f39af93c640 1 -- 172.21.15.172:0/3123984671 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a81a5120 con 0x7f39a8102d30 2024-09-10T08:51:03.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.079+0000 7f39a67fc640 1 -- 172.21.15.172:0/3123984671 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f399800f910 con 0x7f39a8102d30 2024-09-10T08:51:03.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.080+0000 7f39b1d15740 1 Processor -- start 2024-09-10T08:51:03.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.080+0000 7f39b1d15740 1 -- start start 2024-09-10T08:51:03.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.080+0000 7f39b1d15740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65febb40 0x561c65fea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.080+0000 7f39b1d15740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c65feaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.080+0000 7f39b1d15740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65ff5370 0x561c65ff7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.080+0000 7f39b1d15740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561c65f502f0 con 0x561c65fea5d0 2024-09-10T08:51:03.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.080+0000 7f39b1d15740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c65ef7cf0 con 0x561c65ff5370 2024-09-10T08:51:03.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.080+0000 7f39b1d15740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561c65f4c2b0 con 0x561c65febb40 2024-09-10T08:51:03.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.080+0000 7f39a67fc640 1 --2- 172.21.15.172:0/3123984671 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f39900a2b80 0x7f39900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.080+0000 7f39ad6d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c65feaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:03.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.080+0000 7f39ad6d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c65feaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55530/0 (socket says 172.21.15.172:55530) 2024-09-10T08:51:03.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.080+0000 7f39ad6d8640 1 -- 172.21.15.172:0/363420188 learned_addr learned my addr 172.21.15.172:0/363420188 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:03.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.080+0000 7f39aded9640 1 --2- 172.21.15.172:0/3123984671 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f39900a2b80 0x7f39900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:03.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.080+0000 7f39aded9640 1 --2- 172.21.15.172:0/363420188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65febb40 0x561c65fea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:03.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.081+0000 7f39aced7640 1 --2- 172.21.15.172:0/363420188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65ff5370 0x561c65ff7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:03.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.081+0000 7f39a67fc640 1 -- 172.21.15.172:0/3123984671 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f39980cf560 con 0x7f39a8102d30 2024-09-10T08:51:03.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.081+0000 7f39ad6d8640 1 -- 172.21.15.172:0/363420188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65febb40 msgr2=0x561c65fea090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:03.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.081+0000 7f39ad6d8640 1 --2- 172.21.15.172:0/363420188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65febb40 0x561c65fea090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.081+0000 7f39ad6d8640 1 -- 172.21.15.172:0/363420188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65ff5370 msgr2=0x561c65ff7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:03.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.081+0000 7f39ad6d8640 1 --2- 172.21.15.172:0/363420188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65ff5370 0x561c65ff7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.081+0000 7f39ad6d8640 1 -- 172.21.15.172:0/363420188 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c65edc3a0 con 0x561c65fea5d0 2024-09-10T08:51:03.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.081+0000 7f39aded9640 1 --2- 172.21.15.172:0/3123984671 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f39900a2b80 0x7f39900a5040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f39a0006fd0 tx=0x7f39a00097c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:03.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.081+0000 7f39ad6d8640 1 --2- 172.21.15.172:0/363420188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c65feaa10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3998004af0 tx=0x7f3998005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3356ce4242ceaeb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:03.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.081+0000 7f398f7fe640 1 -- 172.21.15.172:0/363420188 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399801d070 con 0x561c65fea5d0 2024-09-10T08:51:03.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.081+0000 7f398f7fe640 1 -- 172.21.15.172:0/363420188 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f399809ba60 con 0x561c65fea5d0 2024-09-10T08:51:03.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.081+0000 7f398f7fe640 1 -- 172.21.15.172:0/363420188 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980980d0 con 0x561c65fea5d0 2024-09-10T08:51:03.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.082+0000 7f39b1d15740 1 -- 172.21.15.172:0/363420188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 msgr2=0x561c65feaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:03.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.082+0000 7f39b1d15740 1 --2- 172.21.15.172:0/363420188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c65feaa10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3998004af0 tx=0x7f3998005880 comp rx=0 tx=0).stop 2024-09-10T08:51:03.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.082+0000 7f39b1d15740 1 -- 172.21.15.172:0/363420188 shutdown_connections 2024-09-10T08:51:03.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.082+0000 7f39b1d15740 1 --2- 172.21.15.172:0/363420188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65ff5370 0x561c65ff7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.082+0000 7f39b1d15740 1 --2- 172.21.15.172:0/363420188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c65feaa10 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.082+0000 7f39b1d15740 1 --2- 172.21.15.172:0/363420188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65febb40 0x561c65fea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.082+0000 7f39b1d15740 1 -- 172.21.15.172:0/363420188 >> 172.21.15.172:0/363420188 conn(0x561c65fb3310 msgr2=0x561c65fe6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:03.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.082+0000 7f39b1d15740 1 -- 172.21.15.172:0/363420188 shutdown_connections 2024-09-10T08:51:03.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.082+0000 7f39b1d15740 1 -- 172.21.15.172:0/363420188 wait complete. 2024-09-10T08:51:03.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.082+0000 7f39b1d15740 1 -- start start 2024-09-10T08:51:03.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.082+0000 7f39b1d15740 1 Processor -- start 2024-09-10T08:51:03.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39b1d15740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65fea5d0 0x561c65ff4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39b1d15740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65febb40 0x561c65ff2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39b1d15740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65ff5370 0x561c65ff2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39b1d15740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561c65ff8070 con 0x561c65ff5370 2024-09-10T08:51:03.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39b1d15740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c65f0b060 con 0x561c65febb40 2024-09-10T08:51:03.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39b1d15740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561c65ff81f0 con 0x561c65fea5d0 2024-09-10T08:51:03.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39b1d15740 10 client.0 _ll_register_callbacks cb 0x561c65c5e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:03.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39aded9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65ff5370 0x561c65ff2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:03.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39aded9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65ff5370 0x561c65ff2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55532/0 (socket says 172.21.15.172:55532) 2024-09-10T08:51:03.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39aded9640 1 -- 172.21.15.172:0/2250141825 learned_addr learned my addr 172.21.15.172:0/2250141825 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:03.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39ad6d8640 1 --2- 172.21.15.172:0/2250141825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65fea5d0 0x561c65ff4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:03.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39aced7640 1 --2- 172.21.15.172:0/2250141825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65febb40 0x561c65ff2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:03.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39aded9640 1 -- 172.21.15.172:0/2250141825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65fea5d0 msgr2=0x561c65ff4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:03.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39aded9640 1 --2- 172.21.15.172:0/2250141825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65fea5d0 0x561c65ff4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39aded9640 1 -- 172.21.15.172:0/2250141825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65febb40 msgr2=0x561c65ff2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:03.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39aded9640 1 --2- 172.21.15.172:0/2250141825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65febb40 0x561c65ff2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39aded9640 1 -- 172.21.15.172:0/2250141825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c65cc13c0 con 0x561c65ff5370 2024-09-10T08:51:03.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39ad6d8640 1 --2- 172.21.15.172:0/2250141825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65fea5d0 0x561c65ff4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:03.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39aced7640 1 --2- 172.21.15.172:0/2250141825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65febb40 0x561c65ff2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:03.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f39aded9640 1 --2- 172.21.15.172:0/2250141825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65ff5370 0x561c65ff2810 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f39a0018da0 tx=0x7f39a0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c29f1581adba87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:03.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.083+0000 7f398cff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:03.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.084+0000 7f398cff9640 1 -- 172.21.15.172:0/2250141825 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a000f040 con 0x561c65ff5370 2024-09-10T08:51:03.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.084+0000 7f398cff9640 1 -- 172.21.15.172:0/2250141825 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f39a0017df0 con 0x561c65ff5370 2024-09-10T08:51:03.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.084+0000 7f39b1d15740 1 -- 172.21.15.172:0/2250141825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f399809ba60 con 0x561c65ff5370 2024-09-10T08:51:03.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.084+0000 7f398cff9640 1 -- 172.21.15.172:0/2250141825 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f39a0018dd0 con 0x561c65ff5370 2024-09-10T08:51:03.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.084+0000 7f39b1d15740 10 client.16842 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:03.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.084+0000 7f39b1d15740 1 -- 172.21.15.172:0/2250141825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561c66003b90 con 0x561c65ff5370 2024-09-10T08:51:03.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.084+0000 7f398cff9640 1 -- 172.21.15.172:0/2250141825 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f399809ba60 con 0x561c65ff5370 2024-09-10T08:51:03.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.084+0000 7f398cff9640 1 -- 172.21.15.172:0/2250141825 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f39a00065e0 con 0x561c65ff5370 2024-09-10T08:51:03.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.084+0000 7f39b1d15740 10 client.16842 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:03.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.084+0000 7f39b1d15740 10 client.16842 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:51:03.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.084+0000 7f39b1d15740 20 client.16842 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:03.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.084+0000 7f39b1d15740 1 --2- 172.21.15.172:0/2250141825 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x561c66005050 0x561c660074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.084+0000 7f39b1d15740 4 client.16842 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:51:03.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.084+0000 7f39b1d15740 1 -- 172.21.15.172:0/2250141825 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c65f5d860 con 0x561c66005050 2024-09-10T08:51:03.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.085+0000 7f39ad6d8640 1 --2- 172.21.15.172:0/2250141825 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x561c66005050 0x561c660074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:03.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.085+0000 7f398cff9640 10 client.16842 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:51:03.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.085+0000 7f39ad6d8640 1 --2- 172.21.15.172:0/2250141825 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x561c66005050 0x561c660074f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eeb66ab0d9573129 server_cookie=7fd0dd80d3bad5a4 in_seq=0 out_seq=0 2024-09-10T08:51:03.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.086+0000 7f398cff9640 1 -- 172.21.15.172:0/2250141825 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c65f5d860 con 0x561c66005050 2024-09-10T08:51:03.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.086+0000 7f398cff9640 10 client.16842 handle_command_reply: tid=0 2024-09-10T08:51:03.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.086+0000 7f39b1d15740 1 client.16842 shutdown 2024-09-10T08:51:03.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.086+0000 7f39b1d15740 1 -- 172.21.15.172:0/2250141825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65ff5370 msgr2=0x561c65ff2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:03.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.086+0000 7f39b1d15740 1 --2- 172.21.15.172:0/2250141825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65ff5370 0x561c65ff2810 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f39a0018da0 tx=0x7f39a0018000 comp rx=0 tx=0).stop 2024-09-10T08:51:03.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.086+0000 7f39b1d15740 1 -- 172.21.15.172:0/2250141825 shutdown_connections 2024-09-10T08:51:03.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.086+0000 7f39b1d15740 1 --2- 172.21.15.172:0/2250141825 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x561c66005050 0x561c660074f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.086+0000 7f39b1d15740 1 --2- 172.21.15.172:0/2250141825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65ff5370 0x561c65ff2810 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.086+0000 7f39b1d15740 1 --2- 172.21.15.172:0/2250141825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65febb40 0x561c65ff2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.086+0000 7f398cff9640 0 client.16842 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:51:03.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.086+0000 7f39b1d15740 1 --2- 172.21.15.172:0/2250141825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65fea5d0 0x561c65ff4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.086+0000 7f39b1d15740 1 -- 172.21.15.172:0/2250141825 >> 172.21.15.172:0/2250141825 conn(0x561c65fb3310 msgr2=0x561c65fec990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:03.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.086+0000 7f39b1d15740 1 -- 172.21.15.172:0/2250141825 shutdown_connections 2024-09-10T08:51:03.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.086+0000 7f39b1d15740 1 -- 172.21.15.172:0/2250141825 wait complete. 2024-09-10T08:51:03.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.086+0000 7f39b1d15740 20 client.16842 trim_cache size 0 max 16384 2024-09-10T08:51:03.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.099+0000 7f39b1d15740 1 Processor -- start 2024-09-10T08:51:03.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.099+0000 7f39b1d15740 1 -- start start 2024-09-10T08:51:03.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.099+0000 7f39b1d15740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c66007440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.099+0000 7f39b1d15740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65febb40 0x561c65fed180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.099+0000 7f39b1d15740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65ff5370 0x561c65fed6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.099+0000 7f39b1d15740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f39a81aebe0 con 0x561c65fea5d0 2024-09-10T08:51:03.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.099+0000 7f39b1d15740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f39a81b07e0 con 0x561c65ff5370 2024-09-10T08:51:03.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.099+0000 7f39b1d15740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f39a81af2b0 con 0x561c65febb40 2024-09-10T08:51:03.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.099+0000 7f39ad6d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c66007440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:03.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.099+0000 7f39ad6d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c66007440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55534/0 (socket says 172.21.15.172:55534) 2024-09-10T08:51:03.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.099+0000 7f39ad6d8640 1 -- 172.21.15.172:0/755885313 learned_addr learned my addr 172.21.15.172:0/755885313 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:03.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.099+0000 7f39aced7640 1 --2- 172.21.15.172:0/755885313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65febb40 0x561c65fed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:03.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.099+0000 7f39ad6d8640 1 -- 172.21.15.172:0/755885313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65febb40 msgr2=0x561c65fed180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:03.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.099+0000 7f39ad6d8640 1 --2- 172.21.15.172:0/755885313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65febb40 0x561c65fed180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.099+0000 7f39ad6d8640 1 -- 172.21.15.172:0/755885313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65ff5370 msgr2=0x561c65fed6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:03.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.099+0000 7f39ad6d8640 1 --2- 172.21.15.172:0/755885313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65ff5370 0x561c65fed6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.099+0000 7f39ad6d8640 1 -- 172.21.15.172:0/755885313 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c65ff05d0 con 0x561c65fea5d0 2024-09-10T08:51:03.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.100+0000 7f39ad6d8640 1 --2- 172.21.15.172:0/755885313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c66007440 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f399809b9f0 tx=0x7f3998005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fee777b02138114 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:03.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.100+0000 7f398effd640 1 -- 172.21.15.172:0/755885313 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399801d040 con 0x561c65fea5d0 2024-09-10T08:51:03.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.100+0000 7f398effd640 1 -- 172.21.15.172:0/755885313 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39980a8070 con 0x561c65fea5d0 2024-09-10T08:51:03.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.100+0000 7f398effd640 1 -- 172.21.15.172:0/755885313 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980a4240 con 0x561c65fea5d0 2024-09-10T08:51:03.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.100+0000 7f39b1d15740 1 -- 172.21.15.172:0/755885313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 msgr2=0x561c66007440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:03.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.100+0000 7f39b1d15740 1 --2- 172.21.15.172:0/755885313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c66007440 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f399809b9f0 tx=0x7f3998005810 comp rx=0 tx=0).stop 2024-09-10T08:51:03.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.101+0000 7f39b1d15740 1 -- 172.21.15.172:0/755885313 shutdown_connections 2024-09-10T08:51:03.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.101+0000 7f39b1d15740 1 --2- 172.21.15.172:0/755885313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65ff5370 0x561c65fed6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.101+0000 7f39b1d15740 1 --2- 172.21.15.172:0/755885313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65febb40 0x561c65fed180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.101+0000 7f39b1d15740 1 --2- 172.21.15.172:0/755885313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c66007440 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.101+0000 7f39b1d15740 1 -- 172.21.15.172:0/755885313 >> 172.21.15.172:0/755885313 conn(0x561c65fb3310 msgr2=0x561c65fe1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:03.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.101+0000 7f39b1d15740 1 -- 172.21.15.172:0/755885313 shutdown_connections 2024-09-10T08:51:03.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.101+0000 7f39b1d15740 1 -- 172.21.15.172:0/755885313 wait complete. 2024-09-10T08:51:03.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.101+0000 7f39b1d15740 1 -- start start 2024-09-10T08:51:03.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.101+0000 7f39b1d15740 1 Processor -- start 2024-09-10T08:51:03.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.101+0000 7f39b1d15740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c65feef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.101+0000 7f39b1d15740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65febb40 0x561c65fef4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.101+0000 7f39b1d15740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65ff5370 0x561c65fefa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.101+0000 7f39b1d15740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f39a81b0080 con 0x561c65fea5d0 2024-09-10T08:51:03.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.101+0000 7f39b1d15740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f39a81af9b0 con 0x561c65ff5370 2024-09-10T08:51:03.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.101+0000 7f39b1d15740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f39a81acbc0 con 0x561c65febb40 2024-09-10T08:51:03.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.101+0000 7f39b1d15740 10 client.0 _ll_register_callbacks cb 0x7f39a0018a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:03.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f39ad6d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c65feef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:03.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f39aded9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65ff5370 0x561c65fefa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:03.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f39ad6d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c65feef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55538/0 (socket says 172.21.15.172:55538) 2024-09-10T08:51:03.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f39ad6d8640 1 -- 172.21.15.172:0/3685625726 learned_addr learned my addr 172.21.15.172:0/3685625726 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:03.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f39aced7640 1 --2- 172.21.15.172:0/3685625726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65febb40 0x561c65fef4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:03.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f39ad6d8640 1 -- 172.21.15.172:0/3685625726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65febb40 msgr2=0x561c65fef4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:03.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f39ad6d8640 1 --2- 172.21.15.172:0/3685625726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65febb40 0x561c65fef4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f39ad6d8640 1 -- 172.21.15.172:0/3685625726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65ff5370 msgr2=0x561c65fefa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:03.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f39ad6d8640 1 --2- 172.21.15.172:0/3685625726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65ff5370 0x561c65fefa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f39ad6d8640 1 -- 172.21.15.172:0/3685625726 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c6606ede0 con 0x561c65fea5d0 2024-09-10T08:51:03.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f39ad6d8640 1 --2- 172.21.15.172:0/3685625726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c65feef80 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3998097ec0 tx=0x7f3998007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf7f4d981f8d535 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:03.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f398cff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:03.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f398cff9640 1 -- 172.21.15.172:0/3685625726 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980b3020 con 0x561c65fea5d0 2024-09-10T08:51:03.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f398cff9640 1 -- 172.21.15.172:0/3685625726 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f39980980d0 con 0x561c65fea5d0 2024-09-10T08:51:03.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f39b1d15740 1 -- 172.21.15.172:0/3685625726 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39980a8070 con 0x561c65fea5d0 2024-09-10T08:51:03.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f398cff9640 1 -- 172.21.15.172:0/3685625726 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f399801d070 con 0x561c65fea5d0 2024-09-10T08:51:03.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f39b1d15740 10 client.16848 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:03.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.102+0000 7f39b1d15740 1 -- 172.21.15.172:0/3685625726 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561c6606ffc0 con 0x561c65fea5d0 2024-09-10T08:51:03.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.103+0000 7f398cff9640 1 -- 172.21.15.172:0/3685625726 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39980a8070 con 0x561c65fea5d0 2024-09-10T08:51:03.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.103+0000 7f398cff9640 1 -- 172.21.15.172:0/3685625726 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f39980977b0 con 0x561c65fea5d0 2024-09-10T08:51:03.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.103+0000 7f39b1d15740 10 client.16848 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:03.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.103+0000 7f39b1d15740 10 client.16848 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:51:03.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.103+0000 7f39b1d15740 20 client.16848 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:03.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.103+0000 7f39b1d15740 1 --2- 172.21.15.172:0/3685625726 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x561c660701e0 0x561c660725d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:03.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.103+0000 7f39b1d15740 4 client.16848 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:51:03.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.103+0000 7f39b1d15740 1 -- 172.21.15.172:0/3685625726 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c65f57d70 con 0x561c660701e0 2024-09-10T08:51:03.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.103+0000 7f39aced7640 1 --2- 172.21.15.172:0/3685625726 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x561c660701e0 0x561c660725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:03.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.103+0000 7f39aced7640 1 --2- 172.21.15.172:0/3685625726 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x561c660701e0 0x561c660725d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd9253fbad4a1115 server_cookie=b674adcb72ac27ab in_seq=0 out_seq=0 2024-09-10T08:51:03.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.103+0000 7f398cff9640 10 client.16848 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:51:03.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.104+0000 7f398cff9640 1 -- 172.21.15.172:0/3685625726 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c65f57d70 con 0x561c660701e0 2024-09-10T08:51:03.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.104+0000 7f398cff9640 10 client.16848 handle_command_reply: tid=0 2024-09-10T08:51:03.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.104+0000 7f39b1d15740 1 client.16848 shutdown 2024-09-10T08:51:03.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.104+0000 7f39b1d15740 1 -- 172.21.15.172:0/3685625726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 msgr2=0x561c65feef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:03.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.104+0000 7f39b1d15740 1 --2- 172.21.15.172:0/3685625726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c65feef80 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3998097ec0 tx=0x7f3998007870 comp rx=0 tx=0).stop 2024-09-10T08:51:03.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.104+0000 7f39b1d15740 1 -- 172.21.15.172:0/3685625726 shutdown_connections 2024-09-10T08:51:03.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.104+0000 7f39b1d15740 1 --2- 172.21.15.172:0/3685625726 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x561c660701e0 0x561c660725d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.104+0000 7f39b1d15740 1 --2- 172.21.15.172:0/3685625726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c65ff5370 0x561c65fefa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.104+0000 7f39b1d15740 1 --2- 172.21.15.172:0/3685625726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561c65febb40 0x561c65fef4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.104+0000 7f39b1d15740 1 --2- 172.21.15.172:0/3685625726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561c65fea5d0 0x561c65feef80 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.104+0000 7f398cff9640 0 client.16848 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:51:03.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.104+0000 7f39b1d15740 1 -- 172.21.15.172:0/3685625726 >> 172.21.15.172:0/3685625726 conn(0x561c65fb3310 msgr2=0x561c65ff38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:03.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.104+0000 7f39b1d15740 1 -- 172.21.15.172:0/3685625726 shutdown_connections 2024-09-10T08:51:03.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.104+0000 7f39b1d15740 1 -- 172.21.15.172:0/3685625726 wait complete. 2024-09-10T08:51:03.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.104+0000 7f39b1d15740 20 client.16848 trim_cache size 0 max 16384 2024-09-10T08:51:03.120 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:51:03.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.105+0000 7f39af93c640 1 -- 172.21.15.172:0/3123984671 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f39900a2b80 msgr2=0x7f39900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:03.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.105+0000 7f39af93c640 1 --2- 172.21.15.172:0/3123984671 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f39900a2b80 0x7f39900a5040 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f39a0006fd0 tx=0x7f39a00097c0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.105+0000 7f39af93c640 1 -- 172.21.15.172:0/3123984671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a8102d30 msgr2=0x7f39a81985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:03.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.105+0000 7f39af93c640 1 --2- 172.21.15.172:0/3123984671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a8102d30 0x7f39a81985a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f399800cce0 tx=0x7f399800a620 comp rx=0 tx=0).stop 2024-09-10T08:51:03.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.105+0000 7f39af93c640 1 -- 172.21.15.172:0/3123984671 shutdown_connections 2024-09-10T08:51:03.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.105+0000 7f39af93c640 1 --2- 172.21.15.172:0/3123984671 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f39900a2b80 0x7f39900a5040 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.105+0000 7f39af93c640 1 --2- 172.21.15.172:0/3123984671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39a81082a0 0x7f39a819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.105+0000 7f39af93c640 1 --2- 172.21.15.172:0/3123984671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a81036e0 0x7f39a8198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.105+0000 7f39af93c640 1 --2- 172.21.15.172:0/3123984671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39a8102d30 0x7f39a81985a0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:03.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.105+0000 7f39af93c640 1 -- 172.21.15.172:0/3123984671 >> 172.21.15.172:0/3123984671 conn(0x7f39a80fc7d0 msgr2=0x7f39a8105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:03.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.105+0000 7f39af93c640 1 -- 172.21.15.172:0/3123984671 shutdown_connections 2024-09-10T08:51:03.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:03.105+0000 7f39af93c640 1 -- 172.21.15.172:0/3123984671 wait complete. 2024-09-10T08:51:03.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:03 smithi174 ceph-mon[34068]: pgmap v495: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.6 MiB/s wr, 13 op/s 2024-09-10T08:51:03.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:03 smithi191 ceph-mon[35314]: pgmap v495: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.6 MiB/s wr, 13 op/s 2024-09-10T08:51:03.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:03 smithi172 ceph-mon[30854]: pgmap v495: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.6 MiB/s wr, 13 op/s 2024-09-10T08:51:03.938 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:51:04.190 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:51:04.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.662+0000 7f8b9825b640 1 -- 172.21.15.172:0/1672793405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b90101df0 msgr2=0x7f8b901021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:04.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.662+0000 7f8b9825b640 1 --2- 172.21.15.172:0/1672793405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b90101df0 0x7f8b901021d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8b84009900 tx=0x7f8b8402a410 comp rx=0 tx=0).stop 2024-09-10T08:51:04.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.662+0000 7f8b9825b640 1 -- 172.21.15.172:0/1672793405 shutdown_connections 2024-09-10T08:51:04.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.662+0000 7f8b9825b640 1 --2- 172.21.15.172:0/1672793405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b90106f20 0x7f8b9010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.662+0000 7f8b9825b640 1 --2- 172.21.15.172:0/1672793405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b901027a0 0x7f8b901067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.662+0000 7f8b9825b640 1 --2- 172.21.15.172:0/1672793405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b90101df0 0x7f8b901021d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.662+0000 7f8b9825b640 1 -- 172.21.15.172:0/1672793405 >> 172.21.15.172:0/1672793405 conn(0x7f8b900fd8b0 msgr2=0x7f8b900ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:04.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.662+0000 7f8b9825b640 1 -- 172.21.15.172:0/1672793405 shutdown_connections 2024-09-10T08:51:04.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.662+0000 7f8b9825b640 1 -- 172.21.15.172:0/1672793405 wait complete. 2024-09-10T08:51:04.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b9825b640 1 Processor -- start 2024-09-10T08:51:04.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b9825b640 1 -- start start 2024-09-10T08:51:04.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b9825b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b90101df0 0x7f8b901983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b9825b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b901027a0 0x7f8b90198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b9825b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b90106f20 0x7f8b9019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b9825b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8b90111e70 con 0x7f8b901027a0 2024-09-10T08:51:04.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b9825b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8b90111cf0 con 0x7f8b90106f20 2024-09-10T08:51:04.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b9825b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8b90111ff0 con 0x7f8b90101df0 2024-09-10T08:51:04.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b957f6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b901027a0 0x7f8b90198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b957f6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b901027a0 0x7f8b90198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55568/0 (socket says 172.21.15.172:55568) 2024-09-10T08:51:04.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b957f6640 1 -- 172.21.15.172:0/466261177 learned_addr learned my addr 172.21.15.172:0/466261177 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:04.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b95ff7640 1 --2- 172.21.15.172:0/466261177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b90101df0 0x7f8b901983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b957f6640 1 -- 172.21.15.172:0/466261177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b90101df0 msgr2=0x7f8b901983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:04.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b957f6640 1 --2- 172.21.15.172:0/466261177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b90101df0 0x7f8b901983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b967f8640 1 --2- 172.21.15.172:0/466261177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b90106f20 0x7f8b9019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b957f6640 1 -- 172.21.15.172:0/466261177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b90106f20 msgr2=0x7f8b9019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:04.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b957f6640 1 --2- 172.21.15.172:0/466261177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b90106f20 0x7f8b9019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.663+0000 7f8b957f6640 1 -- 172.21.15.172:0/466261177 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9019d3a0 con 0x7f8b901027a0 2024-09-10T08:51:04.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.664+0000 7f8b957f6640 1 --2- 172.21.15.172:0/466261177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b901027a0 0x7f8b90198930 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00b4a0 tx=0x7f8b8c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:04.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.664+0000 7f8b82ffd640 1 -- 172.21.15.172:0/466261177 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c014070 con 0x7f8b901027a0 2024-09-10T08:51:04.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.664+0000 7f8b82ffd640 1 -- 172.21.15.172:0/466261177 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b8c010ac0 con 0x7f8b901027a0 2024-09-10T08:51:04.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.664+0000 7f8b9825b640 1 -- 172.21.15.172:0/466261177 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9019d690 con 0x7f8b901027a0 2024-09-10T08:51:04.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.664+0000 7f8b82ffd640 1 -- 172.21.15.172:0/466261177 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c017c30 con 0x7f8b901027a0 2024-09-10T08:51:04.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.664+0000 7f8b9825b640 1 -- 172.21.15.172:0/466261177 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b901a4f70 con 0x7f8b901027a0 2024-09-10T08:51:04.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.666+0000 7f8b82ffd640 1 -- 172.21.15.172:0/466261177 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8b8c017070 con 0x7f8b901027a0 2024-09-10T08:51:04.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.666+0000 7f8b9a634740 1 Processor -- start 2024-09-10T08:51:04.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b9a634740 1 -- start start 2024-09-10T08:51:04.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b9a634740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327e5b40 0x55b9327e4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b9a634740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327e45d0 0x55b9327e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b9a634740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327ef370 0x55b9327f1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b82ffd640 1 --2- 172.21.15.172:0/466261177 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8b7c0a2b80 0x7f8b7c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b9a634740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b93274a2f0 con 0x55b9327ef370 2024-09-10T08:51:04.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b9a634740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b9326f1cf0 con 0x55b9327e5b40 2024-09-10T08:51:04.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b9a634740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b9327462b0 con 0x55b9327e45d0 2024-09-10T08:51:04.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b957f6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327ef370 0x55b9327f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b967f8640 1 --2- 172.21.15.172:0/466261177 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8b7c0a2b80 0x7f8b7c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b957f6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327ef370 0x55b9327f1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55584/0 (socket says 172.21.15.172:55584) 2024-09-10T08:51:04.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b957f6640 1 -- 172.21.15.172:0/3153361733 learned_addr learned my addr 172.21.15.172:0/3153361733 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:04.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b82ffd640 1 -- 172.21.15.172:0/466261177 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8b8c0ceb20 con 0x7f8b901027a0 2024-09-10T08:51:04.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b967f8640 1 --2- 172.21.15.172:0/3153361733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327e5b40 0x55b9327e4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b95ff7640 1 --2- 172.21.15.172:0/3153361733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327e45d0 0x55b9327e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b957f6640 1 -- 172.21.15.172:0/3153361733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327e45d0 msgr2=0x55b9327e4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:04.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b957f6640 1 --2- 172.21.15.172:0/3153361733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327e45d0 0x55b9327e4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b957f6640 1 -- 172.21.15.172:0/3153361733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327e5b40 msgr2=0x55b9327e4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:04.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b957f6640 1 --2- 172.21.15.172:0/3153361733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327e5b40 0x55b9327e4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.667+0000 7f8b957f6640 1 -- 172.21.15.172:0/3153361733 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9326d63a0 con 0x55b9327ef370 2024-09-10T08:51:04.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.668+0000 7f8b957f6640 1 --2- 172.21.15.172:0/3153361733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327ef370 0x55b9327f1760 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c017590 tx=0x7f8b8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d0b00d566adc0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:04.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.668+0000 7f8b967f8640 1 --2- 172.21.15.172:0/466261177 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8b7c0a2b80 0x7f8b7c0a5040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8b88014580 tx=0x7f8b88014980 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:04.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.668+0000 7f8b7bfff640 1 -- 172.21.15.172:0/3153361733 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0b4400 con 0x55b9327ef370 2024-09-10T08:51:04.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.668+0000 7f8b7bfff640 1 -- 172.21.15.172:0/3153361733 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b8c09b720 con 0x55b9327ef370 2024-09-10T08:51:04.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.668+0000 7f8b7bfff640 1 -- 172.21.15.172:0/3153361733 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0b4220 con 0x55b9327ef370 2024-09-10T08:51:04.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.669+0000 7f8b9a634740 1 -- 172.21.15.172:0/3153361733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327ef370 msgr2=0x55b9327f1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:04.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.669+0000 7f8b9a634740 1 --2- 172.21.15.172:0/3153361733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327ef370 0x55b9327f1760 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c017590 tx=0x7f8b8c0046c0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.669+0000 7f8b9a634740 1 -- 172.21.15.172:0/3153361733 shutdown_connections 2024-09-10T08:51:04.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.669+0000 7f8b9a634740 1 --2- 172.21.15.172:0/3153361733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327ef370 0x55b9327f1760 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.669+0000 7f8b9a634740 1 --2- 172.21.15.172:0/3153361733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327e45d0 0x55b9327e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.669+0000 7f8b9a634740 1 --2- 172.21.15.172:0/3153361733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327e5b40 0x55b9327e4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.669+0000 7f8b9a634740 1 -- 172.21.15.172:0/3153361733 >> 172.21.15.172:0/3153361733 conn(0x55b9327ad310 msgr2=0x55b9327e0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:04.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.669+0000 7f8b9a634740 1 -- 172.21.15.172:0/3153361733 shutdown_connections 2024-09-10T08:51:04.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.669+0000 7f8b9a634740 1 -- 172.21.15.172:0/3153361733 wait complete. 2024-09-10T08:51:04.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.669+0000 7f8b9a634740 1 -- start start 2024-09-10T08:51:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.669+0000 7f8b9a634740 1 Processor -- start 2024-09-10T08:51:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.669+0000 7f8b9a634740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e45d0 0x55b9327eed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.669+0000 7f8b9a634740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327e5b40 0x55b9327ec020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.669+0000 7f8b9a634740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327ef370 0x55b9327ec610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b9a634740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b9327f2070 con 0x55b9327e45d0 2024-09-10T08:51:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b9a634740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b932705060 con 0x55b9327e5b40 2024-09-10T08:51:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b9a634740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b9327f21f0 con 0x55b9327ef370 2024-09-10T08:51:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b9a634740 10 client.0 _ll_register_callbacks cb 0x55b9324588c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b95ff7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e45d0 0x55b9327eed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b95ff7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e45d0 0x55b9327eed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55600/0 (socket says 172.21.15.172:55600) 2024-09-10T08:51:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b95ff7640 1 -- 172.21.15.172:0/4186078708 learned_addr learned my addr 172.21.15.172:0/4186078708 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:04.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b957f6640 1 --2- 172.21.15.172:0/4186078708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327e5b40 0x55b9327ec020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b967f8640 1 --2- 172.21.15.172:0/4186078708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327ef370 0x55b9327ec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b95ff7640 1 -- 172.21.15.172:0/4186078708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327ef370 msgr2=0x55b9327ec610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:04.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b95ff7640 1 --2- 172.21.15.172:0/4186078708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327ef370 0x55b9327ec610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b95ff7640 1 -- 172.21.15.172:0/4186078708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327e5b40 msgr2=0x55b9327ec020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:04.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b95ff7640 1 --2- 172.21.15.172:0/4186078708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327e5b40 0x55b9327ec020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b95ff7640 1 -- 172.21.15.172:0/4186078708 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9324bb3c0 con 0x55b9327e45d0 2024-09-10T08:51:04.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b967f8640 1 --2- 172.21.15.172:0/4186078708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327ef370 0x55b9327ec610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:51:04.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b95ff7640 1 --2- 172.21.15.172:0/4186078708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e45d0 0x55b9327eed70 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8b8402a930 tx=0x7f8b84031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2ed4be5067ad3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:04.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b797fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:04.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b797fa640 1 -- 172.21.15.172:0/4186078708 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b84035070 con 0x55b9327e45d0 2024-09-10T08:51:04.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b797fa640 1 -- 172.21.15.172:0/4186078708 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8b84004240 con 0x55b9327e45d0 2024-09-10T08:51:04.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b9a634740 1 -- 172.21.15.172:0/4186078708 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b8c09b720 con 0x55b9327e45d0 2024-09-10T08:51:04.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b797fa640 1 -- 172.21.15.172:0/4186078708 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8b84004440 con 0x55b9327e45d0 2024-09-10T08:51:04.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b9a634740 10 client.16860 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:04.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.670+0000 7f8b9a634740 1 -- 172.21.15.172:0/4186078708 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b9327fdb10 con 0x55b9327e45d0 2024-09-10T08:51:04.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.671+0000 7f8b797fa640 1 -- 172.21.15.172:0/4186078708 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b8c09b720 con 0x55b9327e45d0 2024-09-10T08:51:04.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.671+0000 7f8b797fa640 1 -- 172.21.15.172:0/4186078708 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f8b84004c20 con 0x55b9327e45d0 2024-09-10T08:51:04.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.671+0000 7f8b9a634740 10 client.16860 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:04.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.671+0000 7f8b9a634740 10 client.16860 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:51:04.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.671+0000 7f8b9a634740 20 client.16860 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:04.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.671+0000 7f8b9a634740 1 --2- 172.21.15.172:0/4186078708 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55b9327fefd0 0x55b9328013e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.671+0000 7f8b9a634740 4 client.16860 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:51:04.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.671+0000 7f8b9a634740 1 -- 172.21.15.172:0/4186078708 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b932757860 con 0x55b9327fefd0 2024-09-10T08:51:04.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.671+0000 7f8b957f6640 1 --2- 172.21.15.172:0/4186078708 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55b9327fefd0 0x55b9328013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.672+0000 7f8b957f6640 1 --2- 172.21.15.172:0/4186078708 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55b9327fefd0 0x55b9328013e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21a4403ec65b335f server_cookie=6e60a3635b8de56 in_seq=0 out_seq=0 2024-09-10T08:51:04.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.672+0000 7f8b797fa640 10 client.16860 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:51:04.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.672+0000 7f8b797fa640 1 -- 172.21.15.172:0/4186078708 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b932757860 con 0x55b9327fefd0 2024-09-10T08:51:04.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.672+0000 7f8b797fa640 10 client.16860 handle_command_reply: tid=0 2024-09-10T08:51:04.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.672+0000 7f8b9a634740 1 client.16860 shutdown 2024-09-10T08:51:04.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.672+0000 7f8b9a634740 1 -- 172.21.15.172:0/4186078708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e45d0 msgr2=0x55b9327eed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:04.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.672+0000 7f8b9a634740 1 --2- 172.21.15.172:0/4186078708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e45d0 0x55b9327eed70 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8b8402a930 tx=0x7f8b84031750 comp rx=0 tx=0).stop 2024-09-10T08:51:04.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.673+0000 7f8b9a634740 1 -- 172.21.15.172:0/4186078708 shutdown_connections 2024-09-10T08:51:04.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.673+0000 7f8b9a634740 1 --2- 172.21.15.172:0/4186078708 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55b9327fefd0 0x55b9328013e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.673+0000 7f8b9a634740 1 --2- 172.21.15.172:0/4186078708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327ef370 0x55b9327ec610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.673+0000 7f8b9a634740 1 --2- 172.21.15.172:0/4186078708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327e5b40 0x55b9327ec020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.673+0000 7f8b797fa640 0 client.16860 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:51:04.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.673+0000 7f8b9a634740 1 --2- 172.21.15.172:0/4186078708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e45d0 0x55b9327eed70 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.673+0000 7f8b9a634740 1 -- 172.21.15.172:0/4186078708 >> 172.21.15.172:0/4186078708 conn(0x55b9327ad310 msgr2=0x55b9327e6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:04.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.673+0000 7f8b9a634740 1 -- 172.21.15.172:0/4186078708 shutdown_connections 2024-09-10T08:51:04.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.673+0000 7f8b9a634740 1 -- 172.21.15.172:0/4186078708 wait complete. 2024-09-10T08:51:04.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.673+0000 7f8b9a634740 20 client.16860 trim_cache size 0 max 16384 2024-09-10T08:51:04.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b9a634740 1 Processor -- start 2024-09-10T08:51:04.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b9a634740 1 -- start start 2024-09-10T08:51:04.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b9a634740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327e45d0 0x55b9327edf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b9a634740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e5b40 0x55b9327e7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b9a634740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327ef370 0x55b9327e7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b9a634740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8b901aeb80 con 0x55b9327e5b40 2024-09-10T08:51:04.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b9a634740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8b901b0780 con 0x55b9327e45d0 2024-09-10T08:51:04.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b9a634740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8b901af250 con 0x55b9327ef370 2024-09-10T08:51:04.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b957f6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e5b40 0x55b9327e7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b957f6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e5b40 0x55b9327e7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55610/0 (socket says 172.21.15.172:55610) 2024-09-10T08:51:04.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b957f6640 1 -- 172.21.15.172:0/1819489539 learned_addr learned my addr 172.21.15.172:0/1819489539 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:04.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b95ff7640 1 --2- 172.21.15.172:0/1819489539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327e45d0 0x55b9327edf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b967f8640 1 --2- 172.21.15.172:0/1819489539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327ef370 0x55b9327e7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b957f6640 1 -- 172.21.15.172:0/1819489539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327ef370 msgr2=0x55b9327e7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:04.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b957f6640 1 --2- 172.21.15.172:0/1819489539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327ef370 0x55b9327e7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b957f6640 1 -- 172.21.15.172:0/1819489539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327e45d0 msgr2=0x55b9327edf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:04.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b957f6640 1 --2- 172.21.15.172:0/1819489539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327e45d0 0x55b9327edf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b957f6640 1 -- 172.21.15.172:0/1819489539 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9327f20c0 con 0x55b9327e5b40 2024-09-10T08:51:04.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b967f8640 1 --2- 172.21.15.172:0/1819489539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327ef370 0x55b9327e7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:04.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.686+0000 7f8b957f6640 1 --2- 172.21.15.172:0/1819489539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e5b40 0x55b9327e7120 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0cc630 tx=0x7f8b8c09ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc08206302eed3ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:04.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.687+0000 7f8b7b7fe640 1 -- 172.21.15.172:0/1819489539 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0b3070 con 0x55b9327e5b40 2024-09-10T08:51:04.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.687+0000 7f8b7b7fe640 1 -- 172.21.15.172:0/1819489539 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b8c09b720 con 0x55b9327e5b40 2024-09-10T08:51:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.687+0000 7f8b7b7fe640 1 -- 172.21.15.172:0/1819489539 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c01e040 con 0x55b9327e5b40 2024-09-10T08:51:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.687+0000 7f8b9a634740 1 -- 172.21.15.172:0/1819489539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e5b40 msgr2=0x55b9327e7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.687+0000 7f8b9a634740 1 --2- 172.21.15.172:0/1819489539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e5b40 0x55b9327e7120 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0cc630 tx=0x7f8b8c09ac60 comp rx=0 tx=0).stop 2024-09-10T08:51:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.687+0000 7f8b9a634740 1 -- 172.21.15.172:0/1819489539 shutdown_connections 2024-09-10T08:51:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.687+0000 7f8b9a634740 1 --2- 172.21.15.172:0/1819489539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327ef370 0x55b9327e7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.687+0000 7f8b9a634740 1 --2- 172.21.15.172:0/1819489539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e5b40 0x55b9327e7120 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.687+0000 7f8b9a634740 1 --2- 172.21.15.172:0/1819489539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327e45d0 0x55b9327edf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.687+0000 7f8b9a634740 1 -- 172.21.15.172:0/1819489539 >> 172.21.15.172:0/1819489539 conn(0x55b9327ad310 msgr2=0x55b9327ed880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.687+0000 7f8b9a634740 1 -- 172.21.15.172:0/1819489539 shutdown_connections 2024-09-10T08:51:04.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.687+0000 7f8b9a634740 1 -- 172.21.15.172:0/1819489539 wait complete. 2024-09-10T08:51:04.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.688+0000 7f8b9a634740 1 -- start start 2024-09-10T08:51:04.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.688+0000 7f8b9a634740 1 Processor -- start 2024-09-10T08:51:04.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.688+0000 7f8b9a634740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327e45d0 0x55b9327f3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.688+0000 7f8b9a634740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e5b40 0x55b9327f3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.688+0000 7f8b9a634740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327ef370 0x55b9327f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.688+0000 7f8b9a634740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8b901b0020 con 0x55b9327e5b40 2024-09-10T08:51:04.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.688+0000 7f8b9a634740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8b901af950 con 0x55b9327ef370 2024-09-10T08:51:04.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.688+0000 7f8b9a634740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8b901acb60 con 0x55b9327e45d0 2024-09-10T08:51:04.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.688+0000 7f8b957f6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e5b40 0x55b9327f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.688+0000 7f8b9a634740 10 client.0 _ll_register_callbacks cb 0x7f8b84009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:04.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.688+0000 7f8b957f6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e5b40 0x55b9327f3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55612/0 (socket says 172.21.15.172:55612) 2024-09-10T08:51:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.688+0000 7f8b957f6640 1 -- 172.21.15.172:0/123158663 learned_addr learned my addr 172.21.15.172:0/123158663 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.688+0000 7f8b95ff7640 1 --2- 172.21.15.172:0/123158663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327e45d0 0x55b9327f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b967f8640 1 --2- 172.21.15.172:0/123158663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327ef370 0x55b9327f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b957f6640 1 -- 172.21.15.172:0/123158663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327e45d0 msgr2=0x55b9327f3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b957f6640 1 --2- 172.21.15.172:0/123158663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327e45d0 0x55b9327f3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b957f6640 1 -- 172.21.15.172:0/123158663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327ef370 msgr2=0x55b9327f3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b957f6640 1 --2- 172.21.15.172:0/123158663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327ef370 0x55b9327f3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b957f6640 1 -- 172.21.15.172:0/123158663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b93286d0e0 con 0x55b9327e5b40 2024-09-10T08:51:04.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b95ff7640 1 --2- 172.21.15.172:0/123158663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327e45d0 0x55b9327f3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:51:04.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b967f8640 1 --2- 172.21.15.172:0/123158663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327ef370 0x55b9327f3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:04.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b957f6640 1 --2- 172.21.15.172:0/123158663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e5b40 0x55b9327f3750 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c017590 tx=0x7f8b8c0a3da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41afd9d4794dfee7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:04.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b797fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:04.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b797fa640 1 -- 172.21.15.172:0/123158663 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c064020 con 0x55b9327e5b40 2024-09-10T08:51:04.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b797fa640 1 -- 172.21.15.172:0/123158663 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8b8c0b9e80 con 0x55b9327e5b40 2024-09-10T08:51:04.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b9a634740 1 -- 172.21.15.172:0/123158663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b8c09b720 con 0x55b9327e5b40 2024-09-10T08:51:04.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b797fa640 1 -- 172.21.15.172:0/123158663 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8b8c082050 con 0x55b9327e5b40 2024-09-10T08:51:04.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b9a634740 10 client.16866 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:04.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b9a634740 1 -- 172.21.15.172:0/123158663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b93286e310 con 0x55b9327e5b40 2024-09-10T08:51:04.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b797fa640 1 -- 172.21.15.172:0/123158663 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b8c09b720 con 0x55b9327e5b40 2024-09-10T08:51:04.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b797fa640 1 -- 172.21.15.172:0/123158663 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f8b8c0a3800 con 0x55b9327e5b40 2024-09-10T08:51:04.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b9a634740 10 client.16866 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:04.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b9a634740 10 client.16866 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:51:04.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b9a634740 20 client.16866 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:04.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.689+0000 7f8b9a634740 1 --2- 172.21.15.172:0/123158663 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55b93286e580 0x55b932870970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:04.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.690+0000 7f8b9a634740 4 client.16866 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:51:04.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.690+0000 7f8b9a634740 1 -- 172.21.15.172:0/123158663 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b932751d70 con 0x55b93286e580 2024-09-10T08:51:04.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.690+0000 7f8b95ff7640 1 --2- 172.21.15.172:0/123158663 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55b93286e580 0x55b932870970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.690+0000 7f8b95ff7640 1 --2- 172.21.15.172:0/123158663 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55b93286e580 0x55b932870970 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e46ce52dc0c44418 server_cookie=6a07340bd5b8a2d3 in_seq=0 out_seq=0 2024-09-10T08:51:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.690+0000 7f8b797fa640 10 client.16866 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:51:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.690+0000 7f8b797fa640 1 -- 172.21.15.172:0/123158663 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b932751d70 con 0x55b93286e580 2024-09-10T08:51:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.690+0000 7f8b797fa640 10 client.16866 handle_command_reply: tid=0 2024-09-10T08:51:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.690+0000 7f8b9a634740 1 client.16866 shutdown 2024-09-10T08:51:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.690+0000 7f8b9a634740 1 -- 172.21.15.172:0/123158663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e5b40 msgr2=0x55b9327f3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.690+0000 7f8b9a634740 1 --2- 172.21.15.172:0/123158663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e5b40 0x55b9327f3750 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c017590 tx=0x7f8b8c0a3da0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9a634740 1 -- 172.21.15.172:0/123158663 shutdown_connections 2024-09-10T08:51:04.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9a634740 1 --2- 172.21.15.172:0/123158663 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55b93286e580 0x55b932870970 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9a634740 1 --2- 172.21.15.172:0/123158663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9327ef370 0x55b9327f3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9a634740 1 --2- 172.21.15.172:0/123158663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9327e5b40 0x55b9327f3750 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9a634740 1 --2- 172.21.15.172:0/123158663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9327e45d0 0x55b9327f3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b797fa640 0 client.16866 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:51:04.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9a634740 1 -- 172.21.15.172:0/123158663 >> 172.21.15.172:0/123158663 conn(0x55b9327ad310 msgr2=0x55b9327ee0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:04.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9a634740 1 -- 172.21.15.172:0/123158663 shutdown_connections 2024-09-10T08:51:04.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9a634740 1 -- 172.21.15.172:0/123158663 wait complete. 2024-09-10T08:51:04.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9a634740 20 client.16866 trim_cache size 0 max 16384 2024-09-10T08:51:04.709 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:51:04.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9825b640 1 -- 172.21.15.172:0/466261177 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8b7c0a2b80 msgr2=0x7f8b7c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:04.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9825b640 1 --2- 172.21.15.172:0/466261177 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8b7c0a2b80 0x7f8b7c0a5040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8b88014580 tx=0x7f8b88014980 comp rx=0 tx=0).stop 2024-09-10T08:51:04.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9825b640 1 -- 172.21.15.172:0/466261177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b901027a0 msgr2=0x7f8b90198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:04.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9825b640 1 --2- 172.21.15.172:0/466261177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b901027a0 0x7f8b90198930 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00b4a0 tx=0x7f8b8c00b960 comp rx=0 tx=0).stop 2024-09-10T08:51:04.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9825b640 1 -- 172.21.15.172:0/466261177 shutdown_connections 2024-09-10T08:51:04.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9825b640 1 --2- 172.21.15.172:0/466261177 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8b7c0a2b80 0x7f8b7c0a5040 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9825b640 1 --2- 172.21.15.172:0/466261177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b90106f20 0x7f8b9019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9825b640 1 --2- 172.21.15.172:0/466261177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b901027a0 0x7f8b90198930 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9825b640 1 --2- 172.21.15.172:0/466261177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b90101df0 0x7f8b901983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:04.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9825b640 1 -- 172.21.15.172:0/466261177 >> 172.21.15.172:0/466261177 conn(0x7f8b900fd8b0 msgr2=0x7f8b900fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:04.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9825b640 1 -- 172.21.15.172:0/466261177 shutdown_connections 2024-09-10T08:51:04.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:04.691+0000 7f8b9825b640 1 -- 172.21.15.172:0/466261177 wait complete. 2024-09-10T08:51:05.584 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:51:05.832 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:51:06.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.318+0000 7f16c152c640 1 -- 172.21.15.172:0/4066834332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16bc1082a0 msgr2=0x7f16bc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:06.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.318+0000 7f16c152c640 1 --2- 172.21.15.172:0/4066834332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16bc1082a0 0x7f16bc10ec40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009cd0 tx=0x7f16ac02a8e0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.318+0000 7f16c152c640 1 -- 172.21.15.172:0/4066834332 shutdown_connections 2024-09-10T08:51:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.318+0000 7f16c152c640 1 --2- 172.21.15.172:0/4066834332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16bc1082a0 0x7f16bc10ec40 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.318+0000 7f16c152c640 1 --2- 172.21.15.172:0/4066834332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16bc1036e0 0x7f16bc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.318+0000 7f16c152c640 1 --2- 172.21.15.172:0/4066834332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16bc102d30 0x7f16bc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.318+0000 7f16c152c640 1 -- 172.21.15.172:0/4066834332 >> 172.21.15.172:0/4066834332 conn(0x7f16bc0fc7d0 msgr2=0x7f16bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.319+0000 7f16c152c640 1 -- 172.21.15.172:0/4066834332 shutdown_connections 2024-09-10T08:51:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.319+0000 7f16c152c640 1 -- 172.21.15.172:0/4066834332 wait complete. 2024-09-10T08:51:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.319+0000 7f16c152c640 1 Processor -- start 2024-09-10T08:51:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.319+0000 7f16c152c640 1 -- start start 2024-09-10T08:51:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.320+0000 7f16c152c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16bc102d30 0x7f16bc198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.320+0000 7f16c152c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16bc1036e0 0x7f16bc198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.320+0000 7f16c152c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16bc1082a0 0x7f16bc19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.320+0000 7f16c152c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc111fb0 con 0x7f16bc1036e0 2024-09-10T08:51:06.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.320+0000 7f16c152c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc111e30 con 0x7f16bc1082a0 2024-09-10T08:51:06.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.320+0000 7f16c152c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc112130 con 0x7f16bc102d30 2024-09-10T08:51:06.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.320+0000 7f16ba7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16bc1036e0 0x7f16bc198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.320+0000 7f16ba7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16bc1036e0 0x7f16bc198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51796/0 (socket says 172.21.15.172:51796) 2024-09-10T08:51:06.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.320+0000 7f16ba7fc640 1 -- 172.21.15.172:0/929912356 learned_addr learned my addr 172.21.15.172:0/929912356 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:06.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.320+0000 7f16baffd640 1 --2- 172.21.15.172:0/929912356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16bc102d30 0x7f16bc198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.320+0000 7f16ba7fc640 1 -- 172.21.15.172:0/929912356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16bc102d30 msgr2=0x7f16bc198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:06.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.320+0000 7f16ba7fc640 1 --2- 172.21.15.172:0/929912356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16bc102d30 0x7f16bc198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.320+0000 7f16ba7fc640 1 -- 172.21.15.172:0/929912356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16bc1082a0 msgr2=0x7f16bc19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:06.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.320+0000 7f16bb7fe640 1 --2- 172.21.15.172:0/929912356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16bc1082a0 0x7f16bc19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.320+0000 7f16ba7fc640 1 --2- 172.21.15.172:0/929912356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16bc1082a0 0x7f16bc19cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.320+0000 7f16ba7fc640 1 -- 172.21.15.172:0/929912356 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc19d4b0 con 0x7f16bc1036e0 2024-09-10T08:51:06.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.320+0000 7f16ba7fc640 1 --2- 172.21.15.172:0/929912356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16bc1036e0 0x7f16bc198a40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f16a400d740 tx=0x7f16a400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:06.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.321+0000 7f16b3fff640 1 -- 172.21.15.172:0/929912356 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4017070 con 0x7f16bc1036e0 2024-09-10T08:51:06.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.321+0000 7f16c152c640 1 -- 172.21.15.172:0/929912356 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16bc19d7a0 con 0x7f16bc1036e0 2024-09-10T08:51:06.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.321+0000 7f16b3fff640 1 -- 172.21.15.172:0/929912356 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16a4010460 con 0x7f16bc1036e0 2024-09-10T08:51:06.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.321+0000 7f16b3fff640 1 -- 172.21.15.172:0/929912356 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a400f600 con 0x7f16bc1036e0 2024-09-10T08:51:06.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.321+0000 7f16c152c640 1 -- 172.21.15.172:0/929912356 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16bc1a5080 con 0x7f16bc1036e0 2024-09-10T08:51:06.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.324+0000 7f16b3fff640 1 -- 172.21.15.172:0/929912356 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f16a401a070 con 0x7f16bc1036e0 2024-09-10T08:51:06.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.324+0000 7f16c3905740 1 Processor -- start 2024-09-10T08:51:06.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.324+0000 7f16c3905740 1 -- start start 2024-09-10T08:51:06.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.324+0000 7f16c3905740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255068db40 0x56255068c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.324+0000 7f16c3905740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068c5d0 0x56255068ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.324+0000 7f16c3905740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562550697370 0x562550699760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.324+0000 7f16c3905740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5625505f22f0 con 0x56255068db40 2024-09-10T08:51:06.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.324+0000 7f16c3905740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562550599cf0 con 0x56255068c5d0 2024-09-10T08:51:06.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.324+0000 7f16c3905740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5625505ee2b0 con 0x562550697370 2024-09-10T08:51:06.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.324+0000 7f16bb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255068db40 0x56255068c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.324+0000 7f16bb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255068db40 0x56255068c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51802/0 (socket says 172.21.15.172:51802) 2024-09-10T08:51:06.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.324+0000 7f16bb7fe640 1 -- 172.21.15.172:0/1732034851 learned_addr learned my addr 172.21.15.172:0/1732034851 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:06.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.325+0000 7f16baffd640 1 --2- 172.21.15.172:0/1732034851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068c5d0 0x56255068ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.325+0000 7f16ba7fc640 1 --2- 172.21.15.172:0/1732034851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562550697370 0x562550699760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.325+0000 7f16bb7fe640 1 -- 172.21.15.172:0/1732034851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562550697370 msgr2=0x562550699760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:06.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.325+0000 7f16bb7fe640 1 --2- 172.21.15.172:0/1732034851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562550697370 0x562550699760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.325+0000 7f16bb7fe640 1 -- 172.21.15.172:0/1732034851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068c5d0 msgr2=0x56255068ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:06.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.325+0000 7f16bb7fe640 1 --2- 172.21.15.172:0/1732034851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068c5d0 0x56255068ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.325+0000 7f16bb7fe640 1 -- 172.21.15.172:0/1732034851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56255057e3a0 con 0x56255068db40 2024-09-10T08:51:06.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.325+0000 7f16b3fff640 1 --2- 172.21.15.172:0/929912356 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16a80a2b80 0x7f16a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.325+0000 7f16baffd640 1 --2- 172.21.15.172:0/929912356 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16a80a2b80 0x7f16a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.325+0000 7f16b3fff640 1 -- 172.21.15.172:0/929912356 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f16a40cedb0 con 0x7f16bc1036e0 2024-09-10T08:51:06.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.325+0000 7f16bb7fe640 1 --2- 172.21.15.172:0/1732034851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255068db40 0x56255068c090 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f16ac005cc0 tx=0x7f16ac005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc38f1f3cc493eea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:06.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.325+0000 7f16b17fa640 1 -- 172.21.15.172:0/1732034851 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac031850 con 0x56255068db40 2024-09-10T08:51:06.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.325+0000 7f16b17fa640 1 -- 172.21.15.172:0/1732034851 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16ac036040 con 0x56255068db40 2024-09-10T08:51:06.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.325+0000 7f16b17fa640 1 -- 172.21.15.172:0/1732034851 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac0043c0 con 0x56255068db40 2024-09-10T08:51:06.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.326+0000 7f16c3905740 1 -- 172.21.15.172:0/1732034851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255068db40 msgr2=0x56255068c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:06.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.326+0000 7f16c3905740 1 --2- 172.21.15.172:0/1732034851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255068db40 0x56255068c090 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f16ac005cc0 tx=0x7f16ac005f70 comp rx=0 tx=0).stop 2024-09-10T08:51:06.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.326+0000 7f16baffd640 1 --2- 172.21.15.172:0/929912356 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16a80a2b80 0x7f16a80a5040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x562550689ac0 tx=0x7f16b4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:06.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.326+0000 7f16c3905740 1 -- 172.21.15.172:0/1732034851 shutdown_connections 2024-09-10T08:51:06.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.326+0000 7f16c3905740 1 --2- 172.21.15.172:0/1732034851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562550697370 0x562550699760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.326+0000 7f16c3905740 1 --2- 172.21.15.172:0/1732034851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068c5d0 0x56255068ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.326+0000 7f16c3905740 1 --2- 172.21.15.172:0/1732034851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255068db40 0x56255068c090 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.326+0000 7f16c3905740 1 -- 172.21.15.172:0/1732034851 >> 172.21.15.172:0/1732034851 conn(0x562550655310 msgr2=0x562550688c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:06.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.326+0000 7f16c3905740 1 -- 172.21.15.172:0/1732034851 shutdown_connections 2024-09-10T08:51:06.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.326+0000 7f16c3905740 1 -- 172.21.15.172:0/1732034851 wait complete. 2024-09-10T08:51:06.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.326+0000 7f16c3905740 1 -- start start 2024-09-10T08:51:06.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.326+0000 7f16c3905740 1 Processor -- start 2024-09-10T08:51:06.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16c3905740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56255068c5d0 0x562550696d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16c3905740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068db40 0x562550693ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16c3905740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562550697370 0x5625506945e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16c3905740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56255069a010 con 0x562550697370 2024-09-10T08:51:06.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16c3905740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5625505ad060 con 0x56255068db40 2024-09-10T08:51:06.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16c3905740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56255069a190 con 0x56255068c5d0 2024-09-10T08:51:06.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16c3905740 10 client.0 _ll_register_callbacks cb 0x5625503008c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:06.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16bb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562550697370 0x5625506945e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16bb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562550697370 0x5625506945e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51812/0 (socket says 172.21.15.172:51812) 2024-09-10T08:51:06.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16bb7fe640 1 -- 172.21.15.172:0/186462026 learned_addr learned my addr 172.21.15.172:0/186462026 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:06.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16baffd640 1 --2- 172.21.15.172:0/186462026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56255068c5d0 0x562550696d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16ba7fc640 1 --2- 172.21.15.172:0/186462026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068db40 0x562550693ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16bb7fe640 1 -- 172.21.15.172:0/186462026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56255068c5d0 msgr2=0x562550696d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:06.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16bb7fe640 1 --2- 172.21.15.172:0/186462026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56255068c5d0 0x562550696d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16bb7fe640 1 -- 172.21.15.172:0/186462026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068db40 msgr2=0x562550693ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:06.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16bb7fe640 1 --2- 172.21.15.172:0/186462026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068db40 0x562550693ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16bb7fe640 1 -- 172.21.15.172:0/186462026 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625503633c0 con 0x562550697370 2024-09-10T08:51:06.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16baffd640 1 --2- 172.21.15.172:0/186462026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56255068c5d0 0x562550696d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:51:06.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16ba7fc640 1 --2- 172.21.15.172:0/186462026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068db40 0x562550693ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:51:06.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16bb7fe640 1 --2- 172.21.15.172:0/186462026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562550697370 0x5625506945e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f16ac02a920 tx=0x7f16ac009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303f27b2c392c788 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:06.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:06.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.327+0000 7f16a27fc640 1 -- 172.21.15.172:0/186462026 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac003e80 con 0x562550697370 2024-09-10T08:51:06.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.328+0000 7f16a27fc640 1 -- 172.21.15.172:0/186462026 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f16ac004140 con 0x562550697370 2024-09-10T08:51:06.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.328+0000 7f16c3905740 1 -- 172.21.15.172:0/186462026 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16ac036040 con 0x562550697370 2024-09-10T08:51:06.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.328+0000 7f16a27fc640 1 -- 172.21.15.172:0/186462026 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f16ac05a050 con 0x562550697370 2024-09-10T08:51:06.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.328+0000 7f16c3905740 10 client.16878 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:06.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.328+0000 7f16c3905740 1 -- 172.21.15.172:0/186462026 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5625506a5a80 con 0x562550697370 2024-09-10T08:51:06.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.328+0000 7f16a27fc640 1 -- 172.21.15.172:0/186462026 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16ac036040 con 0x562550697370 2024-09-10T08:51:06.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.328+0000 7f16a27fc640 1 -- 172.21.15.172:0/186462026 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f16ac030070 con 0x562550697370 2024-09-10T08:51:06.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.328+0000 7f16c3905740 10 client.16878 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:06.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.328+0000 7f16c3905740 10 client.16878 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:51:06.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.328+0000 7f16c3905740 20 client.16878 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:06.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.328+0000 7f16c3905740 1 --2- 172.21.15.172:0/186462026 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5625506a6dd0 0x5625506a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.328+0000 7f16c3905740 4 client.16878 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:51:06.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.328+0000 7f16c3905740 1 -- 172.21.15.172:0/186462026 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625505ff860 con 0x5625506a6dd0 2024-09-10T08:51:06.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.329+0000 7f16baffd640 1 --2- 172.21.15.172:0/186462026 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5625506a6dd0 0x5625506a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.329+0000 7f16a27fc640 10 client.16878 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:51:06.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.329+0000 7f16baffd640 1 --2- 172.21.15.172:0/186462026 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5625506a6dd0 0x5625506a91c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=86248c5c7a663d61 server_cookie=c06da5ca3b8ba043 in_seq=0 out_seq=0 2024-09-10T08:51:06.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.330+0000 7f16a27fc640 1 -- 172.21.15.172:0/186462026 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625505ff860 con 0x5625506a6dd0 2024-09-10T08:51:06.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.330+0000 7f16a27fc640 10 client.16878 handle_command_reply: tid=0 2024-09-10T08:51:06.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.330+0000 7f16c3905740 1 client.16878 shutdown 2024-09-10T08:51:06.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.330+0000 7f16c3905740 1 -- 172.21.15.172:0/186462026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562550697370 msgr2=0x5625506945e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:06.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.330+0000 7f16c3905740 1 --2- 172.21.15.172:0/186462026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562550697370 0x5625506945e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f16ac02a920 tx=0x7f16ac009880 comp rx=0 tx=0).stop 2024-09-10T08:51:06.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.330+0000 7f16c3905740 1 -- 172.21.15.172:0/186462026 shutdown_connections 2024-09-10T08:51:06.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.330+0000 7f16c3905740 1 --2- 172.21.15.172:0/186462026 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5625506a6dd0 0x5625506a91c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.330+0000 7f16c3905740 1 --2- 172.21.15.172:0/186462026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562550697370 0x5625506945e0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.330+0000 7f16c3905740 1 --2- 172.21.15.172:0/186462026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068db40 0x562550693ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.330+0000 7f16a27fc640 0 client.16878 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:51:06.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.330+0000 7f16c3905740 1 --2- 172.21.15.172:0/186462026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56255068c5d0 0x562550696d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.330+0000 7f16c3905740 1 -- 172.21.15.172:0/186462026 >> 172.21.15.172:0/186462026 conn(0x562550655310 msgr2=0x56255068e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:06.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.330+0000 7f16c3905740 1 -- 172.21.15.172:0/186462026 shutdown_connections 2024-09-10T08:51:06.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.330+0000 7f16c3905740 1 -- 172.21.15.172:0/186462026 wait complete. 2024-09-10T08:51:06.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.330+0000 7f16c3905740 20 client.16878 trim_cache size 0 max 16384 2024-09-10T08:51:06.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.343+0000 7f16c3905740 1 Processor -- start 2024-09-10T08:51:06.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.343+0000 7f16c3905740 1 -- start start 2024-09-10T08:51:06.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.343+0000 7f16c3905740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068c5d0 0x5625506ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.343+0000 7f16c3905740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56255068db40 0x5625506acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.343+0000 7f16c3905740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562550697370 0x56255068f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.343+0000 7f16c3905740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1aeaa0 con 0x562550697370 2024-09-10T08:51:06.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.343+0000 7f16c3905740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1b06a0 con 0x56255068c5d0 2024-09-10T08:51:06.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.343+0000 7f16c3905740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1af170 con 0x56255068db40 2024-09-10T08:51:06.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.343+0000 7f16bb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562550697370 0x56255068f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.343+0000 7f16bb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562550697370 0x56255068f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51822/0 (socket says 172.21.15.172:51822) 2024-09-10T08:51:06.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.343+0000 7f16bb7fe640 1 -- 172.21.15.172:0/3172398869 learned_addr learned my addr 172.21.15.172:0/3172398869 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:06.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.344+0000 7f16baffd640 1 --2- 172.21.15.172:0/3172398869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068c5d0 0x5625506ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.344+0000 7f16ba7fc640 1 --2- 172.21.15.172:0/3172398869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56255068db40 0x5625506acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.344+0000 7f16bb7fe640 1 -- 172.21.15.172:0/3172398869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56255068db40 msgr2=0x5625506acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:06.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.344+0000 7f16bb7fe640 1 --2- 172.21.15.172:0/3172398869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56255068db40 0x5625506acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.344+0000 7f16bb7fe640 1 -- 172.21.15.172:0/3172398869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068c5d0 msgr2=0x5625506ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:06.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.344+0000 7f16bb7fe640 1 --2- 172.21.15.172:0/3172398869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068c5d0 0x5625506ac5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.344+0000 7f16bb7fe640 1 -- 172.21.15.172:0/3172398869 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625506884f0 con 0x562550697370 2024-09-10T08:51:06.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.344+0000 7f16baffd640 1 --2- 172.21.15.172:0/3172398869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068c5d0 0x5625506ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:06.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.344+0000 7f16ba7fc640 1 --2- 172.21.15.172:0/3172398869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56255068db40 0x5625506acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:06.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.344+0000 7f16bb7fe640 1 --2- 172.21.15.172:0/3172398869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562550697370 0x56255068f120 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f16ac000f80 tx=0x7f16ac005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1eeb2b574760134 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:06.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.344+0000 7f16b0ff9640 1 -- 172.21.15.172:0/3172398869 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac005bf0 con 0x562550697370 2024-09-10T08:51:06.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.344+0000 7f16b0ff9640 1 -- 172.21.15.172:0/3172398869 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16ac036040 con 0x562550697370 2024-09-10T08:51:06.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.344+0000 7f16b0ff9640 1 -- 172.21.15.172:0/3172398869 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac051c30 con 0x562550697370 2024-09-10T08:51:06.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.345+0000 7f16c3905740 1 -- 172.21.15.172:0/3172398869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562550697370 msgr2=0x56255068f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:06.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.345+0000 7f16c3905740 1 --2- 172.21.15.172:0/3172398869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562550697370 0x56255068f120 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f16ac000f80 tx=0x7f16ac005fd0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.345+0000 7f16c3905740 1 -- 172.21.15.172:0/3172398869 shutdown_connections 2024-09-10T08:51:06.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.345+0000 7f16c3905740 1 --2- 172.21.15.172:0/3172398869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562550697370 0x56255068f120 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.345+0000 7f16c3905740 1 --2- 172.21.15.172:0/3172398869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56255068db40 0x5625506acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.345+0000 7f16c3905740 1 --2- 172.21.15.172:0/3172398869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068c5d0 0x5625506ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.345+0000 7f16c3905740 1 -- 172.21.15.172:0/3172398869 >> 172.21.15.172:0/3172398869 conn(0x562550655310 msgr2=0x562550696070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:06.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.345+0000 7f16c3905740 1 -- 172.21.15.172:0/3172398869 shutdown_connections 2024-09-10T08:51:06.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.345+0000 7f16c3905740 1 -- 172.21.15.172:0/3172398869 wait complete. 2024-09-10T08:51:06.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.345+0000 7f16c3905740 1 -- start start 2024-09-10T08:51:06.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.345+0000 7f16c3905740 1 Processor -- start 2024-09-10T08:51:06.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16c3905740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068c5d0 0x562550691090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16c3905740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255068db40 0x5625506915d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16c3905740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562550697370 0x56255070cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16c3905740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1aff40 con 0x56255068db40 2024-09-10T08:51:06.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16c3905740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1af870 con 0x56255068c5d0 2024-09-10T08:51:06.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16c3905740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1aca80 con 0x562550697370 2024-09-10T08:51:06.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16c3905740 10 client.0 _ll_register_callbacks cb 0x7f16ac03f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:06.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16ba7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255068db40 0x5625506915d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16ba7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255068db40 0x5625506915d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51826/0 (socket says 172.21.15.172:51826) 2024-09-10T08:51:06.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16ba7fc640 1 -- 172.21.15.172:0/2738230718 learned_addr learned my addr 172.21.15.172:0/2738230718 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:06.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16bb7fe640 1 --2- 172.21.15.172:0/2738230718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562550697370 0x56255070cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16baffd640 1 --2- 172.21.15.172:0/2738230718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068c5d0 0x562550691090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16ba7fc640 1 -- 172.21.15.172:0/2738230718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562550697370 msgr2=0x56255070cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:06.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16ba7fc640 1 --2- 172.21.15.172:0/2738230718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562550697370 0x56255070cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16ba7fc640 1 -- 172.21.15.172:0/2738230718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068c5d0 msgr2=0x562550691090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:06.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16ba7fc640 1 --2- 172.21.15.172:0/2738230718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068c5d0 0x562550691090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16ba7fc640 1 -- 172.21.15.172:0/2738230718 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56255070d1a0 con 0x56255068db40 2024-09-10T08:51:06.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16ba7fc640 1 --2- 172.21.15.172:0/2738230718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255068db40 0x5625506915d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f16a40cc550 tx=0x7f16a4010610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6305df5120faae3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:06.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:06.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16a27fc640 1 -- 172.21.15.172:0/2738230718 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4021040 con 0x56255068db40 2024-09-10T08:51:06.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16a27fc640 1 -- 172.21.15.172:0/2738230718 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f16a40a3a90 con 0x56255068db40 2024-09-10T08:51:06.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.346+0000 7f16c3905740 1 -- 172.21.15.172:0/2738230718 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16ac036040 con 0x56255068db40 2024-09-10T08:51:06.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.347+0000 7f16a27fc640 1 -- 172.21.15.172:0/2738230718 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f16a409b2b0 con 0x56255068db40 2024-09-10T08:51:06.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.347+0000 7f16c3905740 10 client.16884 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:06.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.347+0000 7f16c3905740 1 -- 172.21.15.172:0/2738230718 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5625507156a0 con 0x56255068db40 2024-09-10T08:51:06.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.347+0000 7f16a27fc640 1 -- 172.21.15.172:0/2738230718 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16ac036040 con 0x56255068db40 2024-09-10T08:51:06.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.347+0000 7f16a27fc640 1 -- 172.21.15.172:0/2738230718 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f16a40b40d0 con 0x56255068db40 2024-09-10T08:51:06.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.347+0000 7f16c3905740 10 client.16884 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:06.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.347+0000 7f16c3905740 10 client.16884 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:51:06.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.347+0000 7f16c3905740 20 client.16884 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:06.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.347+0000 7f16c3905740 1 --2- 172.21.15.172:0/2738230718 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x562550715e70 0x562550718290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:06.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.347+0000 7f16c3905740 4 client.16884 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:51:06.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.347+0000 7f16c3905740 1 -- 172.21.15.172:0/2738230718 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625505f9d70 con 0x562550715e70 2024-09-10T08:51:06.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.347+0000 7f16baffd640 1 --2- 172.21.15.172:0/2738230718 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x562550715e70 0x562550718290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:06.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.348+0000 7f16a27fc640 10 client.16884 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:51:06.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.348+0000 7f16baffd640 1 --2- 172.21.15.172:0/2738230718 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x562550715e70 0x562550718290 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f61196b0a135af0f server_cookie=80b7fd6bdbdf9326 in_seq=0 out_seq=0 2024-09-10T08:51:06.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.348+0000 7f16a27fc640 1 -- 172.21.15.172:0/2738230718 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625505f9d70 con 0x562550715e70 2024-09-10T08:51:06.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.348+0000 7f16a27fc640 10 client.16884 handle_command_reply: tid=0 2024-09-10T08:51:06.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.348+0000 7f16c3905740 1 client.16884 shutdown 2024-09-10T08:51:06.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.348+0000 7f16c3905740 1 -- 172.21.15.172:0/2738230718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255068db40 msgr2=0x5625506915d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:06.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.348+0000 7f16c3905740 1 --2- 172.21.15.172:0/2738230718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255068db40 0x5625506915d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f16a40cc550 tx=0x7f16a4010610 comp rx=0 tx=0).stop 2024-09-10T08:51:06.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.348+0000 7f16c3905740 1 -- 172.21.15.172:0/2738230718 shutdown_connections 2024-09-10T08:51:06.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.348+0000 7f16c3905740 1 --2- 172.21.15.172:0/2738230718 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x562550715e70 0x562550718290 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.348+0000 7f16c3905740 1 --2- 172.21.15.172:0/2738230718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562550697370 0x56255070cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.348+0000 7f16c3905740 1 --2- 172.21.15.172:0/2738230718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56255068db40 0x5625506915d0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.348+0000 7f16c3905740 1 --2- 172.21.15.172:0/2738230718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56255068c5d0 0x562550691090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.348+0000 7f16c3905740 1 -- 172.21.15.172:0/2738230718 >> 172.21.15.172:0/2738230718 conn(0x562550655310 msgr2=0x5625506874e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:06.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.348+0000 7f16a27fc640 0 client.16884 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:51:06.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.348+0000 7f16c3905740 1 -- 172.21.15.172:0/2738230718 shutdown_connections 2024-09-10T08:51:06.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.348+0000 7f16c3905740 1 -- 172.21.15.172:0/2738230718 wait complete. 2024-09-10T08:51:06.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.348+0000 7f16c3905740 20 client.16884 trim_cache size 0 max 16384 2024-09-10T08:51:06.358 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:51:06.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.349+0000 7f16c152c640 1 -- 172.21.15.172:0/929912356 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16a80a2b80 msgr2=0x7f16a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:06.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.349+0000 7f16c152c640 1 --2- 172.21.15.172:0/929912356 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16a80a2b80 0x7f16a80a5040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x562550689ac0 tx=0x7f16b4008040 comp rx=0 tx=0).stop 2024-09-10T08:51:06.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.349+0000 7f16c152c640 1 -- 172.21.15.172:0/929912356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16bc1036e0 msgr2=0x7f16bc198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:06.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.349+0000 7f16c152c640 1 --2- 172.21.15.172:0/929912356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16bc1036e0 0x7f16bc198a40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f16a400d740 tx=0x7f16a400db40 comp rx=0 tx=0).stop 2024-09-10T08:51:06.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.349+0000 7f16c152c640 1 -- 172.21.15.172:0/929912356 shutdown_connections 2024-09-10T08:51:06.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.349+0000 7f16c152c640 1 --2- 172.21.15.172:0/929912356 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16a80a2b80 0x7f16a80a5040 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.349+0000 7f16c152c640 1 --2- 172.21.15.172:0/929912356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16bc1082a0 0x7f16bc19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.349+0000 7f16c152c640 1 --2- 172.21.15.172:0/929912356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16bc1036e0 0x7f16bc198a40 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.349+0000 7f16c152c640 1 --2- 172.21.15.172:0/929912356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16bc102d30 0x7f16bc198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:06.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.349+0000 7f16c152c640 1 -- 172.21.15.172:0/929912356 >> 172.21.15.172:0/929912356 conn(0x7f16bc0fc7d0 msgr2=0x7f16bc105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:06.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.349+0000 7f16c152c640 1 -- 172.21.15.172:0/929912356 shutdown_connections 2024-09-10T08:51:06.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:06.349+0000 7f16c152c640 1 -- 172.21.15.172:0/929912356 wait complete. 2024-09-10T08:51:06.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:06 smithi191 ceph-mon[35314]: pgmap v496: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.2 MiB/s wr, 14 op/s 2024-09-10T08:51:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:06 smithi172 ceph-mon[30854]: pgmap v496: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.2 MiB/s wr, 14 op/s 2024-09-10T08:51:06.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:06 smithi174 ceph-mon[34068]: pgmap v496: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.2 MiB/s wr, 14 op/s 2024-09-10T08:51:07.060 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:51:07.309 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:51:07.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.801+0000 7f583917e640 1 -- 172.21.15.172:0/83763019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5834101420 msgr2=0x7f583410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.801+0000 7f583917e640 1 --2- 172.21.15.172:0/83763019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5834101420 0x7f583410d950 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f582c009cd0 tx=0x7f582c02a8e0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.801+0000 7f583917e640 1 -- 172.21.15.172:0/83763019 shutdown_connections 2024-09-10T08:51:07.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.801+0000 7f583917e640 1 --2- 172.21.15.172:0/83763019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5834101420 0x7f583410d950 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.801+0000 7f583917e640 1 --2- 172.21.15.172:0/83763019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5834100a80 0x7f5834100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.801+0000 7f583917e640 1 --2- 172.21.15.172:0/83763019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5834106a80 0x7f5834106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.801+0000 7f583917e640 1 -- 172.21.15.172:0/83763019 >> 172.21.15.172:0/83763019 conn(0x7f58340fc7a0 msgr2=0x7f58340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:07.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.801+0000 7f583917e640 1 -- 172.21.15.172:0/83763019 shutdown_connections 2024-09-10T08:51:07.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.801+0000 7f583917e640 1 -- 172.21.15.172:0/83763019 wait complete. 2024-09-10T08:51:07.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f583917e640 1 Processor -- start 2024-09-10T08:51:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f583917e640 1 -- start start 2024-09-10T08:51:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f583917e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5834100a80 0x7f5834199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f583917e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5834101420 0x7f58341998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f583917e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5834106a80 0x7f58341a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f583917e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5834110dd0 con 0x7f5834106a80 2024-09-10T08:51:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f583917e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5834110c50 con 0x7f5834101420 2024-09-10T08:51:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f583917e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5834110f50 con 0x7f5834100a80 2024-09-10T08:51:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f583359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5834106a80 0x7f58341a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f583359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5834106a80 0x7f58341a43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51852/0 (socket says 172.21.15.172:51852) 2024-09-10T08:51:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f583359e640 1 -- 172.21.15.172:0/3110228591 learned_addr learned my addr 172.21.15.172:0/3110228591 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f5832d9d640 1 --2- 172.21.15.172:0/3110228591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5834100a80 0x7f5834199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f582bfff640 1 --2- 172.21.15.172:0/3110228591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5834101420 0x7f58341998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f583359e640 1 -- 172.21.15.172:0/3110228591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5834100a80 msgr2=0x7f5834199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f583359e640 1 --2- 172.21.15.172:0/3110228591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5834100a80 0x7f5834199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f583359e640 1 -- 172.21.15.172:0/3110228591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5834101420 msgr2=0x7f58341998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f583359e640 1 --2- 172.21.15.172:0/3110228591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5834101420 0x7f58341998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f583359e640 1 -- 172.21.15.172:0/3110228591 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58341a4920 con 0x7f5834106a80 2024-09-10T08:51:07.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f5832d9d640 1 --2- 172.21.15.172:0/3110228591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5834100a80 0x7f5834199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:51:07.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f582bfff640 1 --2- 172.21.15.172:0/3110228591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5834101420 0x7f58341998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:51:07.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.802+0000 7f583359e640 1 --2- 172.21.15.172:0/3110228591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5834106a80 0x7f58341a43e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f582c005cc0 tx=0x7f582c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:07.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.803+0000 7f5830d99640 1 -- 172.21.15.172:0/3110228591 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c004240 con 0x7f5834106a80 2024-09-10T08:51:07.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.803+0000 7f5830d99640 1 -- 172.21.15.172:0/3110228591 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f582c0305a0 con 0x7f5834106a80 2024-09-10T08:51:07.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.803+0000 7f583917e640 1 -- 172.21.15.172:0/3110228591 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58341a4af0 con 0x7f5834106a80 2024-09-10T08:51:07.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.803+0000 7f5830d99640 1 -- 172.21.15.172:0/3110228591 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c0466c0 con 0x7f5834106a80 2024-09-10T08:51:07.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.803+0000 7f583917e640 1 -- 172.21.15.172:0/3110228591 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58341a4fa0 con 0x7f5834106a80 2024-09-10T08:51:07.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.806+0000 7f5830d99640 1 -- 172.21.15.172:0/3110228591 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f582c030ac0 con 0x7f5834106a80 2024-09-10T08:51:07.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.806+0000 7f583b557740 1 Processor -- start 2024-09-10T08:51:07.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.806+0000 7f583b557740 1 -- start start 2024-09-10T08:51:07.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.806+0000 7f583b557740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a6eb40 0x55ee63a6d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.806+0000 7f583b557740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6d5d0 0x55ee63a6da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.806+0000 7f583b557740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a78370 0x55ee63a7a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.806+0000 7f583b557740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ee639d32f0 con 0x55ee63a6d5d0 2024-09-10T08:51:07.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.806+0000 7f583b557740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ee6397acf0 con 0x55ee63a6eb40 2024-09-10T08:51:07.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.806+0000 7f583b557740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ee639cf2b0 con 0x55ee63a78370 2024-09-10T08:51:07.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.806+0000 7f5832d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6d5d0 0x55ee63a6da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.806+0000 7f5832d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6d5d0 0x55ee63a6da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51858/0 (socket says 172.21.15.172:51858) 2024-09-10T08:51:07.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.806+0000 7f5832d9d640 1 -- 172.21.15.172:0/3229802660 learned_addr learned my addr 172.21.15.172:0/3229802660 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:07.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.807+0000 7f582bfff640 1 --2- 172.21.15.172:0/3229802660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a78370 0x55ee63a7a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.807+0000 7f5830d99640 1 --2- 172.21.15.172:0/3110228591 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f58200a2b80 0x7f58200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.807+0000 7f582bfff640 1 --2- 172.21.15.172:0/3229802660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a6eb40 0x55ee63a6d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.807+0000 7f5832d9d640 1 -- 172.21.15.172:0/3229802660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a78370 msgr2=0x55ee63a7a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:07.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.807+0000 7f5832d9d640 1 --2- 172.21.15.172:0/3229802660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a78370 0x55ee63a7a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.807+0000 7f5832d9d640 1 -- 172.21.15.172:0/3229802660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a6eb40 msgr2=0x55ee63a6d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:07.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.807+0000 7f5832d9d640 1 --2- 172.21.15.172:0/3229802660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a6eb40 0x55ee63a6d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.807+0000 7f5832d9d640 1 -- 172.21.15.172:0/3229802660 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee6395f3a0 con 0x55ee63a6d5d0 2024-09-10T08:51:07.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.807+0000 7f582bfff640 1 --2- 172.21.15.172:0/3229802660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a6eb40 0x55ee63a6d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:07.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.807+0000 7f583359e640 1 --2- 172.21.15.172:0/3110228591 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f58200a2b80 0x7f58200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.807+0000 7f5830d99640 1 -- 172.21.15.172:0/3110228591 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f582c02e030 con 0x7f5834106a80 2024-09-10T08:51:07.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.807+0000 7f5832d9d640 1 --2- 172.21.15.172:0/3229802660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6d5d0 0x55ee63a6da10 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5824007850 tx=0x7f5824007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11afd7b998389bea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:07.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.807+0000 7f58297fa640 1 -- 172.21.15.172:0/3229802660 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582400d950 con 0x55ee63a6d5d0 2024-09-10T08:51:07.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.807+0000 7f58297fa640 1 -- 172.21.15.172:0/3229802660 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5824018070 con 0x55ee63a6d5d0 2024-09-10T08:51:07.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.807+0000 7f58297fa640 1 -- 172.21.15.172:0/3229802660 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824013d90 con 0x55ee63a6d5d0 2024-09-10T08:51:07.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.808+0000 7f583b557740 1 -- 172.21.15.172:0/3229802660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6d5d0 msgr2=0x55ee63a6da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.808+0000 7f583359e640 1 --2- 172.21.15.172:0/3110228591 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f58200a2b80 0x7f58200a5040 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f582c0bcc70 tx=0x7f582c03d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:07.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.808+0000 7f583b557740 1 --2- 172.21.15.172:0/3229802660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6d5d0 0x55ee63a6da10 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f5824007850 tx=0x7f5824007d10 comp rx=0 tx=0).stop 2024-09-10T08:51:07.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.808+0000 7f583b557740 1 -- 172.21.15.172:0/3229802660 shutdown_connections 2024-09-10T08:51:07.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.808+0000 7f583b557740 1 --2- 172.21.15.172:0/3229802660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a78370 0x55ee63a7a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.808+0000 7f583b557740 1 --2- 172.21.15.172:0/3229802660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6d5d0 0x55ee63a6da10 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.808+0000 7f583b557740 1 --2- 172.21.15.172:0/3229802660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a6eb40 0x55ee63a6d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.808+0000 7f583b557740 1 -- 172.21.15.172:0/3229802660 >> 172.21.15.172:0/3229802660 conn(0x55ee63a36310 msgr2=0x55ee63a69c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:07.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.808+0000 7f583b557740 1 -- 172.21.15.172:0/3229802660 shutdown_connections 2024-09-10T08:51:07.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.808+0000 7f583b557740 1 -- 172.21.15.172:0/3229802660 wait complete. 2024-09-10T08:51:07.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.808+0000 7f583b557740 1 -- start start 2024-09-10T08:51:07.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.808+0000 7f583b557740 1 Processor -- start 2024-09-10T08:51:07.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f583b557740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a6d5d0 0x55ee63a77c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f583b557740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6eb40 0x55ee63a74f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f583b557740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a78370 0x55ee63a75480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f583b557740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ee63a7b070 con 0x55ee63a6eb40 2024-09-10T08:51:07.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f583b557740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ee6398e060 con 0x55ee63a6d5d0 2024-09-10T08:51:07.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f583b557740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ee63a7b1f0 con 0x55ee63a78370 2024-09-10T08:51:07.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f583b557740 10 client.0 _ll_register_callbacks cb 0x55ee636e18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:07.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f5832d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6eb40 0x55ee63a74f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f5832d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6eb40 0x55ee63a74f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51866/0 (socket says 172.21.15.172:51866) 2024-09-10T08:51:07.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f5832d9d640 1 -- 172.21.15.172:0/2628556090 learned_addr learned my addr 172.21.15.172:0/2628556090 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:07.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f582bfff640 1 --2- 172.21.15.172:0/2628556090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a78370 0x55ee63a75480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f5832d9d640 1 -- 172.21.15.172:0/2628556090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a78370 msgr2=0x55ee63a75480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:07.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f5832d9d640 1 --2- 172.21.15.172:0/2628556090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a78370 0x55ee63a75480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f582bfff640 1 --2- 172.21.15.172:0/2628556090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a6d5d0 0x55ee63a77c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f5832d9d640 1 -- 172.21.15.172:0/2628556090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a6d5d0 msgr2=0x55ee63a77c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:07.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f5832d9d640 1 --2- 172.21.15.172:0/2628556090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a6d5d0 0x55ee63a77c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f5832d9d640 1 -- 172.21.15.172:0/2628556090 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee637443c0 con 0x55ee63a6eb40 2024-09-10T08:51:07.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f582bfff640 1 --2- 172.21.15.172:0/2628556090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a6d5d0 0x55ee63a77c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:07.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f5832d9d640 1 --2- 172.21.15.172:0/2628556090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6eb40 0x55ee63a74f40 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5824004700 tx=0x7f5824013f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c715f74fc9f381e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:07.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f581a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:07.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f581a7fc640 1 -- 172.21.15.172:0/2628556090 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582400d950 con 0x55ee63a6eb40 2024-09-10T08:51:07.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f581a7fc640 1 -- 172.21.15.172:0/2628556090 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f582400daf0 con 0x55ee63a6eb40 2024-09-10T08:51:07.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.809+0000 7f583b557740 1 -- 172.21.15.172:0/2628556090 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5824018070 con 0x55ee63a6eb40 2024-09-10T08:51:07.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.810+0000 7f581a7fc640 1 -- 172.21.15.172:0/2628556090 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f582400dcf0 con 0x55ee63a6eb40 2024-09-10T08:51:07.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.810+0000 7f583b557740 10 client.16896 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:07.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.810+0000 7f583b557740 1 -- 172.21.15.172:0/2628556090 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ee63a86a00 con 0x55ee63a6eb40 2024-09-10T08:51:07.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.810+0000 7f581a7fc640 1 -- 172.21.15.172:0/2628556090 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5824018070 con 0x55ee63a6eb40 2024-09-10T08:51:07.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.810+0000 7f581a7fc640 1 -- 172.21.15.172:0/2628556090 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f582401d400 con 0x55ee63a6eb40 2024-09-10T08:51:07.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.810+0000 7f583b557740 10 client.16896 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:07.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.810+0000 7f583b557740 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:51:07.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.810+0000 7f583b557740 20 client.16896 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:07.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.810+0000 7f583b557740 1 --2- 172.21.15.172:0/2628556090 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ee63a87ec0 0x55ee63a8a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.810+0000 7f583b557740 4 client.16896 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:51:07.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.810+0000 7f583b557740 1 -- 172.21.15.172:0/2628556090 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee639e0860 con 0x55ee63a87ec0 2024-09-10T08:51:07.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.810+0000 7f582bfff640 1 --2- 172.21.15.172:0/2628556090 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ee63a87ec0 0x55ee63a8a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.811+0000 7f581a7fc640 10 client.16896 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:07.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.811+0000 7f582bfff640 1 --2- 172.21.15.172:0/2628556090 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ee63a87ec0 0x55ee63a8a2b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=342e00691f82922c server_cookie=968d8b4df338ea0 in_seq=0 out_seq=0 2024-09-10T08:51:07.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.812+0000 7f581a7fc640 1 -- 172.21.15.172:0/2628556090 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee639e0860 con 0x55ee63a87ec0 2024-09-10T08:51:07.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.812+0000 7f581a7fc640 10 client.16896 handle_command_reply: tid=0 2024-09-10T08:51:07.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.812+0000 7f583b557740 1 client.16896 shutdown 2024-09-10T08:51:07.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.812+0000 7f583b557740 1 -- 172.21.15.172:0/2628556090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6eb40 msgr2=0x55ee63a74f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.812+0000 7f583b557740 1 --2- 172.21.15.172:0/2628556090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6eb40 0x55ee63a74f40 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5824004700 tx=0x7f5824013f00 comp rx=0 tx=0).stop 2024-09-10T08:51:07.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.812+0000 7f583b557740 1 -- 172.21.15.172:0/2628556090 shutdown_connections 2024-09-10T08:51:07.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.812+0000 7f583b557740 1 --2- 172.21.15.172:0/2628556090 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ee63a87ec0 0x55ee63a8a2b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.812+0000 7f583b557740 1 --2- 172.21.15.172:0/2628556090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a78370 0x55ee63a75480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.812+0000 7f581a7fc640 0 client.16896 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:07.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.812+0000 7f583b557740 1 --2- 172.21.15.172:0/2628556090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6eb40 0x55ee63a74f40 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.812+0000 7f583b557740 1 --2- 172.21.15.172:0/2628556090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a6d5d0 0x55ee63a77c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.812+0000 7f583b557740 1 -- 172.21.15.172:0/2628556090 >> 172.21.15.172:0/2628556090 conn(0x55ee63a36310 msgr2=0x55ee63a6f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:07.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.812+0000 7f583b557740 1 -- 172.21.15.172:0/2628556090 shutdown_connections 2024-09-10T08:51:07.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.812+0000 7f583b557740 1 -- 172.21.15.172:0/2628556090 wait complete. 2024-09-10T08:51:07.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.812+0000 7f583b557740 20 client.16896 trim_cache size 0 max 16384 2024-09-10T08:51:07.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.825+0000 7f583b557740 1 Processor -- start 2024-09-10T08:51:07.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.825+0000 7f583b557740 1 -- start start 2024-09-10T08:51:07.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.825+0000 7f583b557740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6d5d0 0x55ee63a68c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f583b557740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a6eb40 0x55ee63a70120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f583b557740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a78370 0x55ee63a70660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f583b557740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f58341aeaf0 con 0x55ee63a6d5d0 2024-09-10T08:51:07.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f583b557740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f58341b06f0 con 0x55ee63a78370 2024-09-10T08:51:07.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f583b557740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f58341af1c0 con 0x55ee63a6eb40 2024-09-10T08:51:07.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f582bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6d5d0 0x55ee63a68c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f582bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6d5d0 0x55ee63a68c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51872/0 (socket says 172.21.15.172:51872) 2024-09-10T08:51:07.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f582bfff640 1 -- 172.21.15.172:0/54440410 learned_addr learned my addr 172.21.15.172:0/54440410 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:07.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f5832d9d640 1 --2- 172.21.15.172:0/54440410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a6eb40 0x55ee63a70120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f582bfff640 1 --2- 172.21.15.172:0/54440410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a78370 0x55ee63a70660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f582bfff640 1 -- 172.21.15.172:0/54440410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a6eb40 msgr2=0x55ee63a70120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:07.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f582bfff640 1 --2- 172.21.15.172:0/54440410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a6eb40 0x55ee63a70120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f582bfff640 1 -- 172.21.15.172:0/54440410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a78370 msgr2=0x55ee63a70660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:07.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f582bfff640 1 --2- 172.21.15.172:0/54440410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a78370 0x55ee63a70660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f582bfff640 1 -- 172.21.15.172:0/54440410 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee63a73ef0 con 0x55ee63a6d5d0 2024-09-10T08:51:07.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f582bfff640 1 --2- 172.21.15.172:0/54440410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6d5d0 0x55ee63a68c10 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f581c004700 tx=0x7f581c00a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e669d2f4c191f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:07.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f5828ff9640 1 -- 172.21.15.172:0/54440410 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c022020 con 0x55ee63a6d5d0 2024-09-10T08:51:07.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f5828ff9640 1 -- 172.21.15.172:0/54440410 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f581c0149e0 con 0x55ee63a6d5d0 2024-09-10T08:51:07.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.826+0000 7f5828ff9640 1 -- 172.21.15.172:0/54440410 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c01c690 con 0x55ee63a6d5d0 2024-09-10T08:51:07.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.827+0000 7f583b557740 1 -- 172.21.15.172:0/54440410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6d5d0 msgr2=0x55ee63a68c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.827+0000 7f583b557740 1 --2- 172.21.15.172:0/54440410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6d5d0 0x55ee63a68c10 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f581c004700 tx=0x7f581c00a2b0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.827+0000 7f583b557740 1 -- 172.21.15.172:0/54440410 shutdown_connections 2024-09-10T08:51:07.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.827+0000 7f583b557740 1 --2- 172.21.15.172:0/54440410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a78370 0x55ee63a70660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.827+0000 7f583b557740 1 --2- 172.21.15.172:0/54440410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a6eb40 0x55ee63a70120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.827+0000 7f583b557740 1 --2- 172.21.15.172:0/54440410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6d5d0 0x55ee63a68c10 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.827+0000 7f583b557740 1 -- 172.21.15.172:0/54440410 >> 172.21.15.172:0/54440410 conn(0x55ee63a36310 msgr2=0x55ee63a68500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:07.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.827+0000 7f583b557740 1 -- 172.21.15.172:0/54440410 shutdown_connections 2024-09-10T08:51:07.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.827+0000 7f583b557740 1 -- 172.21.15.172:0/54440410 wait complete. 2024-09-10T08:51:07.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f583b557740 1 -- start start 2024-09-10T08:51:07.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f583b557740 1 Processor -- start 2024-09-10T08:51:07.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f583b557740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a6d5d0 0x55ee63a80af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f583b557740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6eb40 0x55ee63a71bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f583b557740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a78370 0x55ee63a720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f583b557740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f58341aff90 con 0x55ee63a6eb40 2024-09-10T08:51:07.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f583b557740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f58341af8c0 con 0x55ee63a6d5d0 2024-09-10T08:51:07.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f583b557740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f58341acad0 con 0x55ee63a78370 2024-09-10T08:51:07.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f583b557740 10 client.0 _ll_register_callbacks cb 0x7f58240076d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:07.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f5832d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6eb40 0x55ee63a71bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f5832d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6eb40 0x55ee63a71bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51876/0 (socket says 172.21.15.172:51876) 2024-09-10T08:51:07.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f5832d9d640 1 -- 172.21.15.172:0/1661618400 learned_addr learned my addr 172.21.15.172:0/1661618400 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:07.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f582bfff640 1 --2- 172.21.15.172:0/1661618400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a6d5d0 0x55ee63a80af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f582bfff640 1 --2- 172.21.15.172:0/1661618400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a78370 0x55ee63a720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f5832d9d640 1 -- 172.21.15.172:0/1661618400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a78370 msgr2=0x55ee63a720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:07.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f5832d9d640 1 --2- 172.21.15.172:0/1661618400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a78370 0x55ee63a720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f5832d9d640 1 -- 172.21.15.172:0/1661618400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a6d5d0 msgr2=0x55ee63a80af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:07.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f5832d9d640 1 --2- 172.21.15.172:0/1661618400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a6d5d0 0x55ee63a80af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f5832d9d640 1 -- 172.21.15.172:0/1661618400 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee63a72800 con 0x55ee63a6eb40 2024-09-10T08:51:07.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.828+0000 7f5832d9d640 1 --2- 172.21.15.172:0/1661618400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6eb40 0x55ee63a71bb0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f582401d820 tx=0x7f582401d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266e30c2fece8c16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:07.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.829+0000 7f581a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:07.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.829+0000 7f581a7fc640 1 -- 172.21.15.172:0/1661618400 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58240074a0 con 0x55ee63a6eb40 2024-09-10T08:51:07.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.829+0000 7f581a7fc640 1 -- 172.21.15.172:0/1661618400 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5824013750 con 0x55ee63a6eb40 2024-09-10T08:51:07.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.829+0000 7f583b557740 1 -- 172.21.15.172:0/1661618400 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f581c0149e0 con 0x55ee63a6eb40 2024-09-10T08:51:07.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.829+0000 7f581a7fc640 1 -- 172.21.15.172:0/1661618400 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f582400d530 con 0x55ee63a6eb40 2024-09-10T08:51:07.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.829+0000 7f583b557740 10 client.16902 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:07.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.829+0000 7f583b557740 1 -- 172.21.15.172:0/1661618400 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ee63af6cc0 con 0x55ee63a6eb40 2024-09-10T08:51:07.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.829+0000 7f581a7fc640 1 -- 172.21.15.172:0/1661618400 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f581c0149e0 con 0x55ee63a6eb40 2024-09-10T08:51:07.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.829+0000 7f581a7fc640 1 -- 172.21.15.172:0/1661618400 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f5824034430 con 0x55ee63a6eb40 2024-09-10T08:51:07.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.829+0000 7f583b557740 10 client.16902 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:07.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.829+0000 7f583b557740 10 client.16902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:51:07.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.829+0000 7f583b557740 20 client.16902 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:07.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.829+0000 7f583b557740 1 --2- 172.21.15.172:0/1661618400 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ee63af7490 0x55ee63af98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.829+0000 7f583b557740 4 client.16902 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:51:07.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.829+0000 7f583b557740 1 -- 172.21.15.172:0/1661618400 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee639dad70 con 0x55ee63af7490 2024-09-10T08:51:07.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.830+0000 7f582bfff640 1 --2- 172.21.15.172:0/1661618400 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ee63af7490 0x55ee63af98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.830+0000 7f582bfff640 1 --2- 172.21.15.172:0/1661618400 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ee63af7490 0x55ee63af98b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b80a7a568fbdcdb server_cookie=ac5446f6f0afa65b in_seq=0 out_seq=0 2024-09-10T08:51:07.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.830+0000 7f581a7fc640 10 client.16902 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:07.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.830+0000 7f581a7fc640 1 -- 172.21.15.172:0/1661618400 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee639dad70 con 0x55ee63af7490 2024-09-10T08:51:07.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.830+0000 7f581a7fc640 10 client.16902 handle_command_reply: tid=0 2024-09-10T08:51:07.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.830+0000 7f583b557740 1 client.16902 shutdown 2024-09-10T08:51:07.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.830+0000 7f583b557740 1 -- 172.21.15.172:0/1661618400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6eb40 msgr2=0x55ee63a71bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.830+0000 7f583b557740 1 --2- 172.21.15.172:0/1661618400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6eb40 0x55ee63a71bb0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f582401d820 tx=0x7f582401d6b0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.830+0000 7f583b557740 1 -- 172.21.15.172:0/1661618400 shutdown_connections 2024-09-10T08:51:07.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.830+0000 7f583b557740 1 --2- 172.21.15.172:0/1661618400 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ee63af7490 0x55ee63af98b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.830+0000 7f583b557740 1 --2- 172.21.15.172:0/1661618400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee63a78370 0x55ee63a720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.830+0000 7f583b557740 1 --2- 172.21.15.172:0/1661618400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee63a6eb40 0x55ee63a71bb0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.830+0000 7f583b557740 1 --2- 172.21.15.172:0/1661618400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee63a6d5d0 0x55ee63a80af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.830+0000 7f583b557740 1 -- 172.21.15.172:0/1661618400 >> 172.21.15.172:0/1661618400 conn(0x55ee63a36310 msgr2=0x55ee63a76fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:07.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.830+0000 7f581a7fc640 0 client.16902 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:07.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.830+0000 7f583b557740 1 -- 172.21.15.172:0/1661618400 shutdown_connections 2024-09-10T08:51:07.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.830+0000 7f583b557740 1 -- 172.21.15.172:0/1661618400 wait complete. 2024-09-10T08:51:07.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.831+0000 7f583b557740 20 client.16902 trim_cache size 0 max 16384 2024-09-10T08:51:07.843 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:51:07.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.831+0000 7f583917e640 1 -- 172.21.15.172:0/3110228591 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f58200a2b80 msgr2=0x7f58200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.831+0000 7f583917e640 1 --2- 172.21.15.172:0/3110228591 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f58200a2b80 0x7f58200a5040 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f582c0bcc70 tx=0x7f582c03d040 comp rx=0 tx=0).stop 2024-09-10T08:51:07.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.831+0000 7f583917e640 1 -- 172.21.15.172:0/3110228591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5834106a80 msgr2=0x7f58341a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.831+0000 7f583917e640 1 --2- 172.21.15.172:0/3110228591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5834106a80 0x7f58341a43e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f582c005cc0 tx=0x7f582c003ba0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.831+0000 7f583917e640 1 -- 172.21.15.172:0/3110228591 shutdown_connections 2024-09-10T08:51:07.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.831+0000 7f583917e640 1 --2- 172.21.15.172:0/3110228591 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f58200a2b80 0x7f58200a5040 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.831+0000 7f583917e640 1 --2- 172.21.15.172:0/3110228591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5834106a80 0x7f58341a43e0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.831+0000 7f583917e640 1 --2- 172.21.15.172:0/3110228591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5834101420 0x7f58341998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.831+0000 7f583917e640 1 --2- 172.21.15.172:0/3110228591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5834100a80 0x7f5834199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.831+0000 7f583917e640 1 -- 172.21.15.172:0/3110228591 >> 172.21.15.172:0/3110228591 conn(0x7f58340fc7a0 msgr2=0x7f58340fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:07.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.831+0000 7f583917e640 1 -- 172.21.15.172:0/3110228591 shutdown_connections 2024-09-10T08:51:07.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:07.831+0000 7f583917e640 1 -- 172.21.15.172:0/3110228591 wait complete. 2024-09-10T08:51:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.921+0000 7f08b787b640 1 -- 172.21.15.172:0/1734337430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08b0102d50 msgr2=0x7f08b01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.921+0000 7f08b787b640 1 --2- 172.21.15.172:0/1734337430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08b0102d50 0x7f08b01031d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009900 tx=0x7f08a002a410 comp rx=0 tx=0).stop 2024-09-10T08:51:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.921+0000 7f08b787b640 1 -- 172.21.15.172:0/1734337430 shutdown_connections 2024-09-10T08:51:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.921+0000 7f08b787b640 1 --2- 172.21.15.172:0/1734337430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08b0103710 0x7f08b0109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.921+0000 7f08b787b640 1 --2- 172.21.15.172:0/1734337430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08b0102d50 0x7f08b01031d0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.921+0000 7f08b787b640 1 --2- 172.21.15.172:0/1734337430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08b0101b60 0x7f08b0101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.921+0000 7f08b787b640 1 -- 172.21.15.172:0/1734337430 >> 172.21.15.172:0/1734337430 conn(0x7f08b00fd2e0 msgr2=0x7f08b00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.921+0000 7f08b787b640 1 -- 172.21.15.172:0/1734337430 shutdown_connections 2024-09-10T08:51:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.921+0000 7f08b787b640 1 -- 172.21.15.172:0/1734337430 wait complete. 2024-09-10T08:51:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.921+0000 7f08b787b640 1 Processor -- start 2024-09-10T08:51:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.921+0000 7f08b787b640 1 -- start start 2024-09-10T08:51:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.921+0000 7f08b787b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08b0101b60 0x7f08b0075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.921+0000 7f08b787b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08b0102d50 0x7f08b00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.921+0000 7f08b787b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08b0103710 0x7f08b0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08b787b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f08b010d6c0 con 0x7f08b0101b60 2024-09-10T08:51:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08b787b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08b010d540 con 0x7f08b0102d50 2024-09-10T08:51:07.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08b787b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08b010d840 con 0x7f08b0103710 2024-09-10T08:51:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08b5617640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08b0101b60 0x7f08b0075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08b5617640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08b0101b60 0x7f08b0075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51904/0 (socket says 172.21.15.172:51904) 2024-09-10T08:51:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08b5617640 1 -- 172.21.15.172:0/2745337396 learned_addr learned my addr 172.21.15.172:0/2745337396 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08b5e18640 1 --2- 172.21.15.172:0/2745337396 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08b0103710 0x7f08b0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08b4e16640 1 --2- 172.21.15.172:0/2745337396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08b0102d50 0x7f08b00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08b5617640 1 -- 172.21.15.172:0/2745337396 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08b0103710 msgr2=0x7f08b0074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08b5617640 1 --2- 172.21.15.172:0/2745337396 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08b0103710 0x7f08b0074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08b5617640 1 -- 172.21.15.172:0/2745337396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08b0102d50 msgr2=0x7f08b00745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08b5617640 1 --2- 172.21.15.172:0/2745337396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08b0102d50 0x7f08b00745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08b5617640 1 -- 172.21.15.172:0/2745337396 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b00753b0 con 0x7f08b0101b60 2024-09-10T08:51:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08b5617640 1 --2- 172.21.15.172:0/2745337396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08b0101b60 0x7f08b0075fa0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00d950 tx=0x7f08ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08a67fc640 1 -- 172.21.15.172:0/2745337396 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac017020 con 0x7f08b0101b60 2024-09-10T08:51:07.927 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08a67fc640 1 -- 172.21.15.172:0/2745337396 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08ac009c30 con 0x7f08b0101b60 2024-09-10T08:51:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08b787b640 1 -- 172.21.15.172:0/2745337396 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b01a1dd0 con 0x7f08b0101b60 2024-09-10T08:51:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08a67fc640 1 -- 172.21.15.172:0/2745337396 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac00f6b0 con 0x7f08b0101b60 2024-09-10T08:51:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.922+0000 7f08b787b640 1 -- 172.21.15.172:0/2745337396 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b01a21d0 con 0x7f08b0101b60 2024-09-10T08:51:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.924+0000 7f08a67fc640 1 -- 172.21.15.172:0/2745337396 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f08ac01a020 con 0x7f08b0101b60 2024-09-10T08:51:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.925+0000 7f08ba61c740 1 Processor -- start 2024-09-10T08:51:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.925+0000 7f08ba61c740 1 -- start start 2024-09-10T08:51:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.925+0000 7f08ba61c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4ac99a0 0x564ee4ac9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.925+0000 7f08ba61c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4aca2e0 0x564ee4aceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.925+0000 7f08ba61c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4acf370 0x564ee4ad1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.925+0000 7f08ba61c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564ee4a2d2f0 con 0x564ee4acf370 2024-09-10T08:51:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.925+0000 7f08ba61c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564ee49d5270 con 0x564ee4aca2e0 2024-09-10T08:51:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.925+0000 7f08ba61c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564ee4a292b0 con 0x564ee4ac99a0 2024-09-10T08:51:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.925+0000 7f08a67fc640 1 --2- 172.21.15.172:0/2745337396 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f089c0a2c30 0x7f089c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.925+0000 7f08b4e16640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4acf370 0x564ee4ad1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.925+0000 7f08b4e16640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4acf370 0x564ee4ad1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51918/0 (socket says 172.21.15.172:51918) 2024-09-10T08:51:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.925+0000 7f08b4e16640 1 -- 172.21.15.172:0/1101151323 learned_addr learned my addr 172.21.15.172:0/1101151323 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.925+0000 7f08b5e18640 1 --2- 172.21.15.172:0/1101151323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4ac99a0 0x564ee4ac9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.925+0000 7f08b5e18640 1 --2- 172.21.15.172:0/2745337396 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f089c0a2c30 0x7f089c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.925+0000 7f08a67fc640 1 -- 172.21.15.172:0/2745337396 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f08ac0cf170 con 0x7f08b0101b60 2024-09-10T08:51:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.925+0000 7f08b5617640 1 --2- 172.21.15.172:0/1101151323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4aca2e0 0x564ee4aceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.926+0000 7f08b4e16640 1 -- 172.21.15.172:0/1101151323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4ac99a0 msgr2=0x564ee4ac9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.926+0000 7f08b4e16640 1 --2- 172.21.15.172:0/1101151323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4ac99a0 0x564ee4ac9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.926+0000 7f08b4e16640 1 -- 172.21.15.172:0/1101151323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4aca2e0 msgr2=0x564ee4aceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.926+0000 7f08b4e16640 1 --2- 172.21.15.172:0/1101151323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4aca2e0 0x564ee4aceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.926+0000 7f08b4e16640 1 -- 172.21.15.172:0/1101151323 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ee49d4340 con 0x564ee4acf370 2024-09-10T08:51:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.926+0000 7f08b4e16640 1 --2- 172.21.15.172:0/1101151323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4acf370 0x564ee4ad1830 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f08a002aa00 tx=0x7f08a0004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460c23f675ead734 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.926+0000 7f08b5e18640 1 --2- 172.21.15.172:0/2745337396 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f089c0a2c30 0x7f089c0a50f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f08a800bfd0 tx=0x7f08a8004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.926+0000 7f089b7fe640 1 -- 172.21.15.172:0/1101151323 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0035030 con 0x564ee4acf370 2024-09-10T08:51:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.926+0000 7f089b7fe640 1 -- 172.21.15.172:0/1101151323 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08a00046c0 con 0x564ee4acf370 2024-09-10T08:51:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.926+0000 7f089b7fe640 1 -- 172.21.15.172:0/1101151323 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0003850 con 0x564ee4acf370 2024-09-10T08:51:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.927+0000 7f08ba61c740 1 -- 172.21.15.172:0/1101151323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4acf370 msgr2=0x564ee4ad1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.927+0000 7f08ba61c740 1 --2- 172.21.15.172:0/1101151323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4acf370 0x564ee4ad1830 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f08a002aa00 tx=0x7f08a0004390 comp rx=0 tx=0).stop 2024-09-10T08:51:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.927+0000 7f08ba61c740 1 -- 172.21.15.172:0/1101151323 shutdown_connections 2024-09-10T08:51:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.927+0000 7f08ba61c740 1 --2- 172.21.15.172:0/1101151323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4acf370 0x564ee4ad1830 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.927+0000 7f08ba61c740 1 --2- 172.21.15.172:0/1101151323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4aca2e0 0x564ee4aceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.927+0000 7f08ba61c740 1 --2- 172.21.15.172:0/1101151323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4ac99a0 0x564ee4ac9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.927+0000 7f08ba61c740 1 -- 172.21.15.172:0/1101151323 >> 172.21.15.172:0/1101151323 conn(0x564ee4ac5010 msgr2=0x564ee4ac5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.927+0000 7f08ba61c740 1 -- 172.21.15.172:0/1101151323 shutdown_connections 2024-09-10T08:51:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.927+0000 7f08ba61c740 1 -- 172.21.15.172:0/1101151323 wait complete. 2024-09-10T08:51:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.927+0000 7f08ba61c740 1 -- start start 2024-09-10T08:51:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.927+0000 7f08ba61c740 1 Processor -- start 2024-09-10T08:51:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.927+0000 7f08ba61c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4ac99a0 0x564ee4ad3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.927+0000 7f08ba61c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4aca2e0 0x564ee4ad4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.927+0000 7f08ba61c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4acf370 0x564ee4adb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08ba61c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564ee4ad22e0 con 0x564ee4aca2e0 2024-09-10T08:51:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08ba61c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564ee49b6290 con 0x564ee4ac99a0 2024-09-10T08:51:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08ba61c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564ee4ad2460 con 0x564ee4acf370 2024-09-10T08:51:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08ba61c740 10 client.0 _ll_register_callbacks cb 0x564ee49e7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08b4e16640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4aca2e0 0x564ee4ad4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08b4e16640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4aca2e0 0x564ee4ad4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51926/0 (socket says 172.21.15.172:51926) 2024-09-10T08:51:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08b4e16640 1 -- 172.21.15.172:0/4153418351 learned_addr learned my addr 172.21.15.172:0/4153418351 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08b5617640 1 --2- 172.21.15.172:0/4153418351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4ac99a0 0x564ee4ad3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08b5e18640 1 --2- 172.21.15.172:0/4153418351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4acf370 0x564ee4adb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08b4e16640 1 -- 172.21.15.172:0/4153418351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4acf370 msgr2=0x564ee4adb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08b4e16640 1 --2- 172.21.15.172:0/4153418351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4acf370 0x564ee4adb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08b4e16640 1 -- 172.21.15.172:0/4153418351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4ac99a0 msgr2=0x564ee4ad3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08b4e16640 1 --2- 172.21.15.172:0/4153418351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4ac99a0 0x564ee4ad3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08b4e16640 1 -- 172.21.15.172:0/4153418351 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ee4740210 con 0x564ee4aca2e0 2024-09-10T08:51:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08b5617640 1 --2- 172.21.15.172:0/4153418351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4ac99a0 0x564ee4ad3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:51:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08b4e16640 1 --2- 172.21.15.172:0/4153418351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4aca2e0 0x564ee4ad4070 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f08a0005ab0 tx=0x7f08a0005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25471f4d2b941ccb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f0898ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:07.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f0898ff9640 1 -- 172.21.15.172:0/4153418351 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0035030 con 0x564ee4aca2e0 2024-09-10T08:51:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f0898ff9640 1 -- 172.21.15.172:0/4153418351 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f08a0031580 con 0x564ee4aca2e0 2024-09-10T08:51:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08ba61c740 1 -- 172.21.15.172:0/4153418351 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08a00046c0 con 0x564ee4aca2e0 2024-09-10T08:51:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f0898ff9640 1 -- 172.21.15.172:0/4153418351 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f08a0056050 con 0x564ee4aca2e0 2024-09-10T08:51:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08ba61c740 10 client.16914 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.928+0000 7f08ba61c740 1 -- 172.21.15.172:0/4153418351 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564ee480c620 con 0x564ee4aca2e0 2024-09-10T08:51:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.929+0000 7f0898ff9640 1 -- 172.21.15.172:0/4153418351 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08a00046c0 con 0x564ee4aca2e0 2024-09-10T08:51:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.929+0000 7f0898ff9640 1 -- 172.21.15.172:0/4153418351 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f08a003b630 con 0x564ee4aca2e0 2024-09-10T08:51:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.929+0000 7f08ba61c740 10 client.16914 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.929+0000 7f08ba61c740 10 client.16914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:51:07.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.929+0000 7f08ba61c740 20 client.16914 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.929+0000 7f08ba61c740 1 --2- 172.21.15.172:0/4153418351 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564ee4adf3b0 0x564ee4ae17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.929+0000 7f08ba61c740 4 client.16914 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:51:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.929+0000 7f08ba61c740 1 -- 172.21.15.172:0/4153418351 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ee4a3aa20 con 0x564ee4adf3b0 2024-09-10T08:51:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.929+0000 7f08b5617640 1 --2- 172.21.15.172:0/4153418351 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564ee4adf3b0 0x564ee4ae17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.930+0000 7f0898ff9640 10 client.16914 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.930+0000 7f08b5617640 1 --2- 172.21.15.172:0/4153418351 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564ee4adf3b0 0x564ee4ae17a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8636e0e0e0f2379b server_cookie=d2c44d2c603f7d0c in_seq=0 out_seq=0 2024-09-10T08:51:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.930+0000 7f0898ff9640 1 -- 172.21.15.172:0/4153418351 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ee4a3aa20 con 0x564ee4adf3b0 2024-09-10T08:51:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.930+0000 7f0898ff9640 10 client.16914 handle_command_reply: tid=0 2024-09-10T08:51:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.930+0000 7f08ba61c740 1 client.16914 shutdown 2024-09-10T08:51:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.930+0000 7f08ba61c740 1 -- 172.21.15.172:0/4153418351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4aca2e0 msgr2=0x564ee4ad4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.930+0000 7f08ba61c740 1 --2- 172.21.15.172:0/4153418351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4aca2e0 0x564ee4ad4070 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f08a0005ab0 tx=0x7f08a0005210 comp rx=0 tx=0).stop 2024-09-10T08:51:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.930+0000 7f08ba61c740 1 -- 172.21.15.172:0/4153418351 shutdown_connections 2024-09-10T08:51:07.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.930+0000 7f08ba61c740 1 --2- 172.21.15.172:0/4153418351 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564ee4adf3b0 0x564ee4ae17a0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.930+0000 7f08ba61c740 1 --2- 172.21.15.172:0/4153418351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4acf370 0x564ee4adb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.930+0000 7f08ba61c740 1 --2- 172.21.15.172:0/4153418351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4aca2e0 0x564ee4ad4070 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.930+0000 7f0898ff9640 0 client.16914 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.930+0000 7f08ba61c740 1 --2- 172.21.15.172:0/4153418351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4ac99a0 0x564ee4ad3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.930+0000 7f08ba61c740 1 -- 172.21.15.172:0/4153418351 >> 172.21.15.172:0/4153418351 conn(0x564ee4ac5010 msgr2=0x564ee4ac6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.931+0000 7f08ba61c740 1 -- 172.21.15.172:0/4153418351 shutdown_connections 2024-09-10T08:51:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.931+0000 7f08ba61c740 1 -- 172.21.15.172:0/4153418351 wait complete. 2024-09-10T08:51:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.931+0000 7f08ba61c740 20 client.16914 trim_cache size 0 max 16384 2024-09-10T08:51:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f08ba61c740 1 Processor -- start 2024-09-10T08:51:07.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f08ba61c740 1 -- start start 2024-09-10T08:51:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f08ba61c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4ac99a0 0x564ee4ac6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f08ba61c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4aca2e0 0x564ee4adb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f08ba61c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4acf370 0x564ee4ae6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f08ba61c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f08b01ac6b0 con 0x564ee4ac99a0 2024-09-10T08:51:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f08ba61c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08b01ae2b0 con 0x564ee4acf370 2024-09-10T08:51:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f08ba61c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08b01acd80 con 0x564ee4aca2e0 2024-09-10T08:51:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f08b5617640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4ac99a0 0x564ee4ac6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f08b5617640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4ac99a0 0x564ee4ac6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51940/0 (socket says 172.21.15.172:51940) 2024-09-10T08:51:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f08b5617640 1 -- 172.21.15.172:0/2321377601 learned_addr learned my addr 172.21.15.172:0/2321377601 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f08b5617640 1 -- 172.21.15.172:0/2321377601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4aca2e0 msgr2=0x564ee4adb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f08b5617640 1 --2- 172.21.15.172:0/2321377601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4aca2e0 0x564ee4adb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f08b5617640 1 -- 172.21.15.172:0/2321377601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4acf370 msgr2=0x564ee4ae6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f08b5617640 1 --2- 172.21.15.172:0/2321377601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4acf370 0x564ee4ae6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f08b5617640 1 -- 172.21.15.172:0/2321377601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ee4ad3670 con 0x564ee4ac99a0 2024-09-10T08:51:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f08b5617640 1 --2- 172.21.15.172:0/2321377601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4ac99a0 0x564ee4ac6b10 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f08ac09b310 tx=0x7f08ac086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291d0b4abf2d1081 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f089affd640 1 -- 172.21.15.172:0/2321377601 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac09f9c0 con 0x564ee4ac99a0 2024-09-10T08:51:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f089affd640 1 -- 172.21.15.172:0/2321377601 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08ac097040 con 0x564ee4ac99a0 2024-09-10T08:51:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.945+0000 7f089affd640 1 -- 172.21.15.172:0/2321377601 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac010a10 con 0x564ee4ac99a0 2024-09-10T08:51:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.946+0000 7f08ba61c740 1 -- 172.21.15.172:0/2321377601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4ac99a0 msgr2=0x564ee4ac6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.946+0000 7f08ba61c740 1 --2- 172.21.15.172:0/2321377601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4ac99a0 0x564ee4ac6b10 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f08ac09b310 tx=0x7f08ac086040 comp rx=0 tx=0).stop 2024-09-10T08:51:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.946+0000 7f08ba61c740 1 -- 172.21.15.172:0/2321377601 shutdown_connections 2024-09-10T08:51:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.946+0000 7f08ba61c740 1 --2- 172.21.15.172:0/2321377601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4acf370 0x564ee4ae6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.946+0000 7f08ba61c740 1 --2- 172.21.15.172:0/2321377601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4aca2e0 0x564ee4adb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.946+0000 7f08ba61c740 1 --2- 172.21.15.172:0/2321377601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4ac99a0 0x564ee4ac6b10 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.946+0000 7f08ba61c740 1 -- 172.21.15.172:0/2321377601 >> 172.21.15.172:0/2321377601 conn(0x564ee4ac5010 msgr2=0x564ee4ac53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.946+0000 7f08ba61c740 1 -- 172.21.15.172:0/2321377601 shutdown_connections 2024-09-10T08:51:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.946+0000 7f08ba61c740 1 -- 172.21.15.172:0/2321377601 wait complete. 2024-09-10T08:51:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.946+0000 7f08ba61c740 1 -- start start 2024-09-10T08:51:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.946+0000 7f08ba61c740 1 Processor -- start 2024-09-10T08:51:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08ba61c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4ac99a0 0x564ee4adaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08ba61c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4aca2e0 0x564ee4adaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08ba61c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4acf370 0x564ee4ad9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08ba61c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f08b01adb50 con 0x564ee4aca2e0 2024-09-10T08:51:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08ba61c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08b01ad480 con 0x564ee4ac99a0 2024-09-10T08:51:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08ba61c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08b01aa690 con 0x564ee4acf370 2024-09-10T08:51:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08ba61c740 10 client.0 _ll_register_callbacks cb 0x7f08a800bd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08b4e16640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4aca2e0 0x564ee4adaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08b4e16640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4aca2e0 0x564ee4adaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51944/0 (socket says 172.21.15.172:51944) 2024-09-10T08:51:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08b4e16640 1 -- 172.21.15.172:0/3974350216 learned_addr learned my addr 172.21.15.172:0/3974350216 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08b5e18640 1 --2- 172.21.15.172:0/3974350216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4acf370 0x564ee4ad9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08b4e16640 1 -- 172.21.15.172:0/3974350216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4acf370 msgr2=0x564ee4ad9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08b4e16640 1 --2- 172.21.15.172:0/3974350216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4acf370 0x564ee4ad9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08b4e16640 1 -- 172.21.15.172:0/3974350216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4ac99a0 msgr2=0x564ee4adaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08b5617640 1 --2- 172.21.15.172:0/3974350216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4ac99a0 0x564ee4adaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08b4e16640 1 --2- 172.21.15.172:0/3974350216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4ac99a0 0x564ee4adaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08b4e16640 1 -- 172.21.15.172:0/3974350216 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ee4ad9840 con 0x564ee4aca2e0 2024-09-10T08:51:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f08b4e16640 1 --2- 172.21.15.172:0/3974350216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4aca2e0 0x564ee4adaf40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f08a003ba30 tx=0x7f08a0004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c366ed242738801 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.947+0000 7f0898ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.948+0000 7f0898ff9640 1 -- 172.21.15.172:0/3974350216 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0035040 con 0x564ee4aca2e0 2024-09-10T08:51:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.948+0000 7f08ba61c740 1 -- 172.21.15.172:0/3974350216 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08ac097040 con 0x564ee4aca2e0 2024-09-10T08:51:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.948+0000 7f0898ff9640 1 -- 172.21.15.172:0/3974350216 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f08a003b430 con 0x564ee4aca2e0 2024-09-10T08:51:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.948+0000 7f0898ff9640 1 -- 172.21.15.172:0/3974350216 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f08a00046c0 con 0x564ee4aca2e0 2024-09-10T08:51:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.948+0000 7f08ba61c740 10 client.16920 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.948+0000 7f08ba61c740 1 -- 172.21.15.172:0/3974350216 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564ee4b4ab20 con 0x564ee4aca2e0 2024-09-10T08:51:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.948+0000 7f0898ff9640 1 -- 172.21.15.172:0/3974350216 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08ac097040 con 0x564ee4aca2e0 2024-09-10T08:51:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.948+0000 7f0898ff9640 1 -- 172.21.15.172:0/3974350216 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f08a005b440 con 0x564ee4aca2e0 2024-09-10T08:51:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.948+0000 7f08ba61c740 10 client.16920 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.948+0000 7f08ba61c740 10 client.16920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:51:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.948+0000 7f08ba61c740 20 client.16920 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:07.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.948+0000 7f08ba61c740 1 --2- 172.21.15.172:0/3974350216 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564ee4b4b410 0x564ee4b4d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.948+0000 7f08ba61c740 4 client.16920 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:51:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.948+0000 7f08ba61c740 1 -- 172.21.15.172:0/3974350216 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ee4a34d70 con 0x564ee4b4b410 2024-09-10T08:51:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.948+0000 7f08b5617640 1 --2- 172.21.15.172:0/3974350216 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564ee4b4b410 0x564ee4b4d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.949+0000 7f08b5617640 1 --2- 172.21.15.172:0/3974350216 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564ee4b4b410 0x564ee4b4d830 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=312d9ae58d72ec95 server_cookie=a9e6dc1a3f5fa8fb in_seq=0 out_seq=0 2024-09-10T08:51:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.949+0000 7f0898ff9640 10 client.16920 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.956+0000 7f0898ff9640 1 -- 172.21.15.172:0/3974350216 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ee4a34d70 con 0x564ee4b4b410 2024-09-10T08:51:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.956+0000 7f0898ff9640 10 client.16920 handle_command_reply: tid=0 2024-09-10T08:51:07.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.956+0000 7f08ba61c740 1 client.16920 shutdown 2024-09-10T08:51:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.956+0000 7f08ba61c740 1 -- 172.21.15.172:0/3974350216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4aca2e0 msgr2=0x564ee4adaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.956+0000 7f08ba61c740 1 --2- 172.21.15.172:0/3974350216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4aca2e0 0x564ee4adaf40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f08a003ba30 tx=0x7f08a0004c00 comp rx=0 tx=0).stop 2024-09-10T08:51:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.956+0000 7f08ba61c740 1 -- 172.21.15.172:0/3974350216 shutdown_connections 2024-09-10T08:51:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.956+0000 7f08ba61c740 1 --2- 172.21.15.172:0/3974350216 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564ee4b4b410 0x564ee4b4d830 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.956+0000 7f08ba61c740 1 --2- 172.21.15.172:0/3974350216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ee4acf370 0x564ee4ad9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.956+0000 7f08ba61c740 1 --2- 172.21.15.172:0/3974350216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ee4aca2e0 0x564ee4adaf40 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.956+0000 7f08ba61c740 1 --2- 172.21.15.172:0/3974350216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ee4ac99a0 0x564ee4adaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.956+0000 7f0898ff9640 0 client.16920 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.956+0000 7f08ba61c740 1 -- 172.21.15.172:0/3974350216 >> 172.21.15.172:0/3974350216 conn(0x564ee4ac5010 msgr2=0x564ee4ac53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.956+0000 7f08ba61c740 1 -- 172.21.15.172:0/3974350216 shutdown_connections 2024-09-10T08:51:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.956+0000 7f08ba61c740 1 -- 172.21.15.172:0/3974350216 wait complete. 2024-09-10T08:51:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.956+0000 7f08ba61c740 20 client.16920 trim_cache size 0 max 16384 2024-09-10T08:51:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:51:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:51:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:51:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:51:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.957+0000 7f08b787b640 1 -- 172.21.15.172:0/2745337396 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f089c0a2c30 msgr2=0x7f089c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.957+0000 7f08b787b640 1 --2- 172.21.15.172:0/2745337396 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f089c0a2c30 0x7f089c0a50f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f08a800bfd0 tx=0x7f08a8004a40 comp rx=0 tx=0).stop 2024-09-10T08:51:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.957+0000 7f08b787b640 1 -- 172.21.15.172:0/2745337396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08b0101b60 msgr2=0x7f08b0075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.957+0000 7f08b787b640 1 --2- 172.21.15.172:0/2745337396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08b0101b60 0x7f08b0075fa0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00d950 tx=0x7f08ac00de10 comp rx=0 tx=0).stop 2024-09-10T08:51:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.957+0000 7f08b787b640 1 -- 172.21.15.172:0/2745337396 shutdown_connections 2024-09-10T08:51:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.957+0000 7f08b787b640 1 --2- 172.21.15.172:0/2745337396 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f089c0a2c30 0x7f089c0a50f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.957+0000 7f08b787b640 1 --2- 172.21.15.172:0/2745337396 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08b0103710 0x7f08b0074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.957+0000 7f08b787b640 1 --2- 172.21.15.172:0/2745337396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08b0102d50 0x7f08b00745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.957+0000 7f08b787b640 1 --2- 172.21.15.172:0/2745337396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08b0101b60 0x7f08b0075fa0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.957+0000 7f08b787b640 1 -- 172.21.15.172:0/2745337396 >> 172.21.15.172:0/2745337396 conn(0x7f08b00fd2e0 msgr2=0x7f08b01060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.957+0000 7f08b787b640 1 -- 172.21.15.172:0/2745337396 shutdown_connections 2024-09-10T08:51:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:07.957+0000 7f08b787b640 1 -- 172.21.15.172:0/2745337396 wait complete. 2024-09-10T08:51:08.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:08 smithi191 ceph-mon[35314]: pgmap v497: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 892 KiB/s wr, 12 op/s 2024-09-10T08:51:08.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:08 smithi172 ceph-mon[30854]: pgmap v497: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 892 KiB/s wr, 12 op/s 2024-09-10T08:51:08.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:08 smithi174 ceph-mon[34068]: pgmap v497: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 892 KiB/s wr, 12 op/s 2024-09-10T08:51:09.680 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:51:09.680 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:51:09.950 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:51:10.478 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:10 smithi172 ceph-mon[30854]: pgmap v498: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 550 KiB/s wr, 10 op/s 2024-09-10T08:51:10.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.476+0000 7fc580347640 1 -- 172.21.15.172:0/1052137545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc578102ef0 msgr2=0x7fc57810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:10.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.476+0000 7fc580347640 1 --2- 172.21.15.172:0/1052137545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc578102ef0 0x7fc57810da40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc568009ca0 tx=0x7fc56802a950 comp rx=0 tx=0).stop 2024-09-10T08:51:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.476+0000 7fc580347640 1 -- 172.21.15.172:0/1052137545 shutdown_connections 2024-09-10T08:51:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.476+0000 7fc580347640 1 --2- 172.21.15.172:0/1052137545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc578102ef0 0x7fc57810da40 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.476+0000 7fc580347640 1 --2- 172.21.15.172:0/1052137545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578102570 0x7fc5781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.476+0000 7fc580347640 1 --2- 172.21.15.172:0/1052137545 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc578100e20 0x7fc578101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.476+0000 7fc580347640 1 -- 172.21.15.172:0/1052137545 >> 172.21.15.172:0/1052137545 conn(0x7fc5780fc7d0 msgr2=0x7fc5780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.476+0000 7fc580347640 1 -- 172.21.15.172:0/1052137545 shutdown_connections 2024-09-10T08:51:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.476+0000 7fc580347640 1 -- 172.21.15.172:0/1052137545 wait complete. 2024-09-10T08:51:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.476+0000 7fc580347640 1 Processor -- start 2024-09-10T08:51:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc580347640 1 -- start start 2024-09-10T08:51:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc580347640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc578100e20 0x7fc578199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc580347640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc578102570 0x7fc578199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc580347640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578102ef0 0x7fc5781a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc580347640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc578110e50 con 0x7fc578100e20 2024-09-10T08:51:10.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc580347640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc578110cd0 con 0x7fc578102ef0 2024-09-10T08:51:10.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc580347640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc578110fd0 con 0x7fc578102570 2024-09-10T08:51:10.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc57e0e3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc578100e20 0x7fc578199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc57e0e3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc578100e20 0x7fc578199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51966/0 (socket says 172.21.15.172:51966) 2024-09-10T08:51:10.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc57e0e3640 1 -- 172.21.15.172:0/224931219 learned_addr learned my addr 172.21.15.172:0/224931219 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:10.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc57e8e4640 1 --2- 172.21.15.172:0/224931219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578102ef0 0x7fc5781a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc57d8e2640 1 --2- 172.21.15.172:0/224931219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc578102570 0x7fc578199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc57e0e3640 1 -- 172.21.15.172:0/224931219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc578102570 msgr2=0x7fc578199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:10.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc57e0e3640 1 --2- 172.21.15.172:0/224931219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc578102570 0x7fc578199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc57e0e3640 1 -- 172.21.15.172:0/224931219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578102ef0 msgr2=0x7fc5781a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:10.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc57e0e3640 1 --2- 172.21.15.172:0/224931219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578102ef0 0x7fc5781a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc57e0e3640 1 -- 172.21.15.172:0/224931219 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5781a47c0 con 0x7fc578100e20 2024-09-10T08:51:10.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc57e8e4640 1 --2- 172.21.15.172:0/224931219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578102ef0 0x7fc5781a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:51:10.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.477+0000 7fc57e0e3640 1 --2- 172.21.15.172:0/224931219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc578100e20 0x7fc578199200 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc57000b4a0 tx=0x7fc57000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:10.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.478+0000 7fc5777fe640 1 -- 172.21.15.172:0/224931219 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570014070 con 0x7fc578100e20 2024-09-10T08:51:10.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.478+0000 7fc5777fe640 1 -- 172.21.15.172:0/224931219 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc570010ac0 con 0x7fc578100e20 2024-09-10T08:51:10.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.478+0000 7fc5777fe640 1 -- 172.21.15.172:0/224931219 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570017c30 con 0x7fc578100e20 2024-09-10T08:51:10.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.478+0000 7fc580347640 1 -- 172.21.15.172:0/224931219 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5781a4990 con 0x7fc578100e20 2024-09-10T08:51:10.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.478+0000 7fc580347640 1 -- 172.21.15.172:0/224931219 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5781a4ea0 con 0x7fc578100e20 2024-09-10T08:51:10.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.480+0000 7fc582720740 1 Processor -- start 2024-09-10T08:51:10.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.480+0000 7fc582720740 1 -- start start 2024-09-10T08:51:10.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.481+0000 7fc582720740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe61cb60 0x5595fe61cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.481+0000 7fc582720740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe61d480 0x5595fe627800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.481+0000 7fc582720740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe627d40 0x5595fe62a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.481+0000 7fc582720740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5595fe582280 con 0x5595fe627d40 2024-09-10T08:51:10.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.481+0000 7fc582720740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5595fe529b10 con 0x5595fe61cb60 2024-09-10T08:51:10.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.481+0000 7fc582720740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5595fe57e240 con 0x5595fe61d480 2024-09-10T08:51:10.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.481+0000 7fc57d8e2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe627d40 0x5595fe62a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.481+0000 7fc57d8e2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe627d40 0x5595fe62a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51974/0 (socket says 172.21.15.172:51974) 2024-09-10T08:51:10.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.481+0000 7fc57d8e2640 1 -- 172.21.15.172:0/1361863759 learned_addr learned my addr 172.21.15.172:0/1361863759 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:10.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.481+0000 7fc5777fe640 1 -- 172.21.15.172:0/224931219 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc570017070 con 0x7fc578100e20 2024-09-10T08:51:10.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.481+0000 7fc57e8e4640 1 --2- 172.21.15.172:0/1361863759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe61cb60 0x5595fe61cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.481+0000 7fc57e0e3640 1 --2- 172.21.15.172:0/1361863759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe61d480 0x5595fe627800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.481+0000 7fc57d8e2640 1 -- 172.21.15.172:0/1361863759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe61d480 msgr2=0x5595fe627800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:10.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.482+0000 7fc57d8e2640 1 --2- 172.21.15.172:0/1361863759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe61d480 0x5595fe627800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.482+0000 7fc57d8e2640 1 -- 172.21.15.172:0/1361863759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe61cb60 msgr2=0x5595fe61cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:10.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.482+0000 7fc57d8e2640 1 --2- 172.21.15.172:0/1361863759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe61cb60 0x5595fe61cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.482+0000 7fc57d8e2640 1 -- 172.21.15.172:0/1361863759 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595fe5361f0 con 0x5595fe627d40 2024-09-10T08:51:10.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.482+0000 7fc57e0e3640 1 --2- 172.21.15.172:0/1361863759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe61d480 0x5595fe627800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:10.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.482+0000 7fc57d8e2640 1 --2- 172.21.15.172:0/1361863759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe627d40 0x5595fe62a130 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00bb10 tx=0x7fc56c00d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66998464b7216409 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:10.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.482+0000 7fc574dea640 1 -- 172.21.15.172:0/1361863759 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c017420 con 0x5595fe627d40 2024-09-10T08:51:10.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.482+0000 7fc574dea640 1 -- 172.21.15.172:0/1361863759 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc56c017a70 con 0x5595fe627d40 2024-09-10T08:51:10.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.482+0000 7fc5777fe640 1 --2- 172.21.15.172:0/224931219 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5600a2c00 0x7fc5600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.482+0000 7fc574dea640 1 -- 172.21.15.172:0/1361863759 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c00fa80 con 0x5595fe627d40 2024-09-10T08:51:10.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.483+0000 7fc57e8e4640 1 --2- 172.21.15.172:0/224931219 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5600a2c00 0x7fc5600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.483+0000 7fc5777fe640 1 -- 172.21.15.172:0/224931219 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fc5700ce790 con 0x7fc578100e20 2024-09-10T08:51:10.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.483+0000 7fc57e8e4640 1 --2- 172.21.15.172:0/224931219 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5600a2c00 0x7fc5600a50c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc568009430 tx=0x7fc568009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:10.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.483+0000 7fc582720740 1 -- 172.21.15.172:0/1361863759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe627d40 msgr2=0x5595fe62a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:10.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.483+0000 7fc582720740 1 --2- 172.21.15.172:0/1361863759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe627d40 0x5595fe62a130 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00bb10 tx=0x7fc56c00d340 comp rx=0 tx=0).stop 2024-09-10T08:51:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.483+0000 7fc582720740 1 -- 172.21.15.172:0/1361863759 shutdown_connections 2024-09-10T08:51:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.483+0000 7fc582720740 1 --2- 172.21.15.172:0/1361863759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe627d40 0x5595fe62a130 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.483+0000 7fc582720740 1 --2- 172.21.15.172:0/1361863759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe61d480 0x5595fe627800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.483+0000 7fc582720740 1 --2- 172.21.15.172:0/1361863759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe61cb60 0x5595fe61cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.483+0000 7fc582720740 1 -- 172.21.15.172:0/1361863759 >> 172.21.15.172:0/1361863759 conn(0x5595fe6190f0 msgr2=0x5595fe6194d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.484+0000 7fc582720740 1 -- 172.21.15.172:0/1361863759 shutdown_connections 2024-09-10T08:51:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.484+0000 7fc582720740 1 -- 172.21.15.172:0/1361863759 wait complete. 2024-09-10T08:51:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.484+0000 7fc582720740 1 -- start start 2024-09-10T08:51:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.484+0000 7fc582720740 1 Processor -- start 2024-09-10T08:51:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.484+0000 7fc582720740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe61cb60 0x5595fe624b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.484+0000 7fc582720740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61d480 0x5595fe6250a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.484+0000 7fc582720740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe627d40 0x5595fe6255e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.485+0000 7fc582720740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5595fe62a910 con 0x5595fe61d480 2024-09-10T08:51:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.485+0000 7fc57d8e2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61d480 0x5595fe6250a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.485+0000 7fc57e0e3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe61cb60 0x5595fe624b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.485+0000 7fc57e0e3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe61cb60 0x5595fe624b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:34016/0 (socket says 172.21.15.172:34016) 2024-09-10T08:51:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.485+0000 7fc57d8e2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61d480 0x5595fe6250a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51982/0 (socket says 172.21.15.172:51982) 2024-09-10T08:51:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.485+0000 7fc57e0e3640 1 -- 172.21.15.172:0/3124955679 learned_addr learned my addr 172.21.15.172:0/3124955679 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.485+0000 7fc582720740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5595fe53cf10 con 0x5595fe61cb60 2024-09-10T08:51:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.485+0000 7fc582720740 1 -- 172.21.15.172:0/3124955679 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5595fe62aa90 con 0x5595fe627d40 2024-09-10T08:51:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.485+0000 7fc57e8e4640 1 --2- 172.21.15.172:0/3124955679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe627d40 0x5595fe6255e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.485+0000 7fc582720740 10 client.0 _ll_register_callbacks cb 0x5595fe505a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.485+0000 7fc57e0e3640 1 -- 172.21.15.172:0/3124955679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe627d40 msgr2=0x5595fe6255e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.485+0000 7fc57e0e3640 1 --2- 172.21.15.172:0/3124955679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe627d40 0x5595fe6255e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.485+0000 7fc57e0e3640 1 -- 172.21.15.172:0/3124955679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61d480 msgr2=0x5595fe6250a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:10.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.485+0000 7fc57e0e3640 1 --2- 172.21.15.172:0/3124955679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61d480 0x5595fe6250a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.485+0000 7fc57e0e3640 1 -- 172.21.15.172:0/3124955679 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595fe2f36a0 con 0x5595fe61cb60 2024-09-10T08:51:10.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.485+0000 7fc57e0e3640 1 --2- 172.21.15.172:0/3124955679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe61cb60 0x5595fe624b60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc57000b470 tx=0x7fc57009a6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d4cba295b71be43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:10.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.485+0000 7fc55dffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:51:10.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.486+0000 7fc55dffb640 1 -- 172.21.15.172:0/3124955679 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57001e050 con 0x5595fe61cb60 2024-09-10T08:51:10.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.486+0000 7fc55dffb640 1 -- 172.21.15.172:0/3124955679 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fc570097300 con 0x5595fe61cb60 2024-09-10T08:51:10.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.486+0000 7fc582720740 1 -- 172.21.15.172:0/3124955679 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc56c017a70 con 0x5595fe61cb60 2024-09-10T08:51:10.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.486+0000 7fc55dffb640 1 -- 172.21.15.172:0/3124955679 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc5700a3900 con 0x5595fe61cb60 2024-09-10T08:51:10.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.486+0000 7fc582720740 10 client.26147 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:10.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.486+0000 7fc582720740 1 -- 172.21.15.172:0/3124955679 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5595fe63a870 con 0x5595fe61cb60 2024-09-10T08:51:10.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.486+0000 7fc55dffb640 1 -- 172.21.15.172:0/3124955679 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc570010c70 con 0x5595fe61cb60 2024-09-10T08:51:10.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.486+0000 7fc55dffb640 1 -- 172.21.15.172:0/3124955679 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fc5700969e0 con 0x5595fe61cb60 2024-09-10T08:51:10.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.486+0000 7fc582720740 10 client.26147 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:10.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.486+0000 7fc582720740 10 client.26147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:51:10.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.486+0000 7fc582720740 20 client.26147 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:10.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.486+0000 7fc582720740 1 --2- 172.21.15.172:0/3124955679 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5595fe63bc80 0x5595fe63e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.486+0000 7fc582720740 4 client.26147 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:51:10.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.486+0000 7fc582720740 1 -- 172.21.15.172:0/3124955679 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc56c017a70 con 0x5595fe63bc80 2024-09-10T08:51:10.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.487+0000 7fc57d8e2640 1 --2- 172.21.15.172:0/3124955679 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5595fe63bc80 0x5595fe63e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.487+0000 7fc57d8e2640 1 --2- 172.21.15.172:0/3124955679 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5595fe63bc80 0x5595fe63e100 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=179039fe1d5e173e server_cookie=c2d7dc1b480ac927 in_seq=0 out_seq=0 2024-09-10T08:51:10.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.487+0000 7fc55dffb640 10 client.26147 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:10.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.488+0000 7fc55dffb640 1 -- 172.21.15.172:0/3124955679 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fc56c017a70 con 0x5595fe63bc80 2024-09-10T08:51:10.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.488+0000 7fc55dffb640 10 client.26147 handle_command_reply: tid=0 2024-09-10T08:51:10.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.488+0000 7fc582720740 1 client.26147 shutdown 2024-09-10T08:51:10.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.488+0000 7fc582720740 1 -- 172.21.15.172:0/3124955679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe61cb60 msgr2=0x5595fe624b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:10.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.488+0000 7fc582720740 1 --2- 172.21.15.172:0/3124955679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe61cb60 0x5595fe624b60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc57000b470 tx=0x7fc57009a6b0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.488+0000 7fc582720740 1 -- 172.21.15.172:0/3124955679 shutdown_connections 2024-09-10T08:51:10.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.488+0000 7fc582720740 1 --2- 172.21.15.172:0/3124955679 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5595fe63bc80 0x5595fe63e100 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.488+0000 7fc582720740 1 --2- 172.21.15.172:0/3124955679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe627d40 0x5595fe6255e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.488+0000 7fc582720740 1 --2- 172.21.15.172:0/3124955679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61d480 0x5595fe6250a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.488+0000 7fc582720740 1 --2- 172.21.15.172:0/3124955679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe61cb60 0x5595fe624b60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.488+0000 7fc582720740 1 -- 172.21.15.172:0/3124955679 >> 172.21.15.172:0/3124955679 conn(0x5595fe6190f0 msgr2=0x5595fe618270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:10.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.488+0000 7fc55dffb640 0 client.26147 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:10.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.488+0000 7fc582720740 1 -- 172.21.15.172:0/3124955679 shutdown_connections 2024-09-10T08:51:10.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.488+0000 7fc582720740 1 -- 172.21.15.172:0/3124955679 wait complete. 2024-09-10T08:51:10.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.488+0000 7fc582720740 20 client.26147 trim_cache size 0 max 16384 2024-09-10T08:51:10.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.519+0000 7fc582720740 1 Processor -- start 2024-09-10T08:51:10.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.519+0000 7fc582720740 1 -- start start 2024-09-10T08:51:10.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc582720740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe61cb60 0x5595fe61ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc582720740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61d480 0x5595fe6439d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc582720740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe627d40 0x5595fe643f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.522 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc582720740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781ac8f0 con 0x5595fe61d480 2024-09-10T08:51:10.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc582720740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781ae4f0 con 0x5595fe61cb60 2024-09-10T08:51:10.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc582720740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781acfc0 con 0x5595fe627d40 2024-09-10T08:51:10.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc57d8e2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61d480 0x5595fe6439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc57d8e2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61d480 0x5595fe6439d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51988/0 (socket says 172.21.15.172:51988) 2024-09-10T08:51:10.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc57d8e2640 1 -- 172.21.15.172:0/2929887724 learned_addr learned my addr 172.21.15.172:0/2929887724 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:10.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc57e0e3640 1 --2- 172.21.15.172:0/2929887724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe61cb60 0x5595fe61ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc57e8e4640 1 --2- 172.21.15.172:0/2929887724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe627d40 0x5595fe643f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc57d8e2640 1 -- 172.21.15.172:0/2929887724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe627d40 msgr2=0x5595fe643f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:10.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc57d8e2640 1 --2- 172.21.15.172:0/2929887724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe627d40 0x5595fe643f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc57d8e2640 1 -- 172.21.15.172:0/2929887724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe61cb60 msgr2=0x5595fe61ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:10.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc57d8e2640 1 --2- 172.21.15.172:0/2929887724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe61cb60 0x5595fe61ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc57d8e2640 1 -- 172.21.15.172:0/2929887724 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595fe6230b0 con 0x5595fe61d480 2024-09-10T08:51:10.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc57e8e4640 1 --2- 172.21.15.172:0/2929887724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe627d40 0x5595fe643f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:10.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc57d8e2640 1 --2- 172.21.15.172:0/2929887724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61d480 0x5595fe6439d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00f6d0 tx=0x7fc56c00d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712fd789b3216697 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:10.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.520+0000 7fc55ffff640 1 -- 172.21.15.172:0/2929887724 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c00b620 con 0x5595fe61d480 2024-09-10T08:51:10.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.521+0000 7fc55ffff640 1 -- 172.21.15.172:0/2929887724 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc56c00ddd0 con 0x5595fe61d480 2024-09-10T08:51:10.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.521+0000 7fc55ffff640 1 -- 172.21.15.172:0/2929887724 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c00fb00 con 0x5595fe61d480 2024-09-10T08:51:10.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.521+0000 7fc582720740 1 -- 172.21.15.172:0/2929887724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61d480 msgr2=0x5595fe6439d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:10.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.521+0000 7fc582720740 1 --2- 172.21.15.172:0/2929887724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61d480 0x5595fe6439d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00f6d0 tx=0x7fc56c00d340 comp rx=0 tx=0).stop 2024-09-10T08:51:10.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.521+0000 7fc582720740 1 -- 172.21.15.172:0/2929887724 shutdown_connections 2024-09-10T08:51:10.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.521+0000 7fc582720740 1 --2- 172.21.15.172:0/2929887724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe627d40 0x5595fe643f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.521+0000 7fc582720740 1 --2- 172.21.15.172:0/2929887724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61d480 0x5595fe6439d0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.521+0000 7fc582720740 1 --2- 172.21.15.172:0/2929887724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe61cb60 0x5595fe61ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.521+0000 7fc582720740 1 -- 172.21.15.172:0/2929887724 >> 172.21.15.172:0/2929887724 conn(0x5595fe6190f0 msgr2=0x5595fe61e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:10.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.521+0000 7fc582720740 1 -- 172.21.15.172:0/2929887724 shutdown_connections 2024-09-10T08:51:10.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.521+0000 7fc582720740 1 -- 172.21.15.172:0/2929887724 wait complete. 2024-09-10T08:51:10.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc582720740 1 -- start start 2024-09-10T08:51:10.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc582720740 1 Processor -- start 2024-09-10T08:51:10.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc582720740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61cb60 0x5595fe62fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc582720740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe61d480 0x5595fe630430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc582720740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe627d40 0x5595fe62cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc582720740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781add90 con 0x5595fe61cb60 2024-09-10T08:51:10.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc582720740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781ad6c0 con 0x5595fe627d40 2024-09-10T08:51:10.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc582720740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc5781aa8d0 con 0x5595fe61d480 2024-09-10T08:51:10.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc582720740 10 client.0 _ll_register_callbacks cb 0x7fc5700174e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:10.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc57e0e3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61cb60 0x5595fe62fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc57e0e3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61cb60 0x5595fe62fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52002/0 (socket says 172.21.15.172:52002) 2024-09-10T08:51:10.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc57e0e3640 1 -- 172.21.15.172:0/2956299101 learned_addr learned my addr 172.21.15.172:0/2956299101 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:10.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc57d8e2640 1 --2- 172.21.15.172:0/2956299101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe61d480 0x5595fe630430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc57e8e4640 1 --2- 172.21.15.172:0/2956299101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe627d40 0x5595fe62cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc57e0e3640 1 -- 172.21.15.172:0/2956299101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe61d480 msgr2=0x5595fe630430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:10.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc57e0e3640 1 --2- 172.21.15.172:0/2956299101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe61d480 0x5595fe630430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc57e0e3640 1 -- 172.21.15.172:0/2956299101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe627d40 msgr2=0x5595fe62cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:10.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc57e0e3640 1 --2- 172.21.15.172:0/2956299101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe627d40 0x5595fe62cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc57e0e3640 1 -- 172.21.15.172:0/2956299101 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595fe62d4c0 con 0x5595fe61cb60 2024-09-10T08:51:10.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc57d8e2640 1 --2- 172.21.15.172:0/2956299101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe61d480 0x5595fe630430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:51:10.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.522+0000 7fc57e8e4640 1 --2- 172.21.15.172:0/2956299101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe627d40 0x5595fe62cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:10.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.523+0000 7fc57e0e3640 1 --2- 172.21.15.172:0/2956299101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61cb60 0x5595fe62fef0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc570098f40 tx=0x7fc5700047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5fa414ce9d8309 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:10.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.523+0000 7fc55dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:10.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.523+0000 7fc55dffb640 1 -- 172.21.15.172:0/2956299101 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57001e070 con 0x5595fe61cb60 2024-09-10T08:51:10.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.523+0000 7fc55dffb640 1 -- 172.21.15.172:0/2956299101 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fc570099c90 con 0x5595fe61cb60 2024-09-10T08:51:10.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.523+0000 7fc582720740 1 -- 172.21.15.172:0/2956299101 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc56c00ddd0 con 0x5595fe61cb60 2024-09-10T08:51:10.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.523+0000 7fc55dffb640 1 -- 172.21.15.172:0/2956299101 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc570081050 con 0x5595fe61cb60 2024-09-10T08:51:10.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.523+0000 7fc582720740 10 client.16938 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:10.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.523+0000 7fc582720740 1 -- 172.21.15.172:0/2956299101 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5595fe6a7440 con 0x5595fe61cb60 2024-09-10T08:51:10.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.523+0000 7fc55dffb640 1 -- 172.21.15.172:0/2956299101 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc56c00ddd0 con 0x5595fe61cb60 2024-09-10T08:51:10.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.523+0000 7fc55dffb640 1 -- 172.21.15.172:0/2956299101 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fc5700106a0 con 0x5595fe61cb60 2024-09-10T08:51:10.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.523+0000 7fc582720740 10 client.16938 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:10.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.523+0000 7fc582720740 10 client.16938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:51:10.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.523+0000 7fc582720740 20 client.16938 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:10.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.523+0000 7fc582720740 1 --2- 172.21.15.172:0/2956299101 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5595fe6a7bf0 0x5595fe6aa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:10.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.523+0000 7fc582720740 4 client.16938 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:51:10.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.523+0000 7fc582720740 1 -- 172.21.15.172:0/2956299101 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5595fe58f7f0 con 0x5595fe6a7bf0 2024-09-10T08:51:10.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.524+0000 7fc57d8e2640 1 --2- 172.21.15.172:0/2956299101 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5595fe6a7bf0 0x5595fe6aa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:10.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.524+0000 7fc55dffb640 10 client.16938 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:10.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.524+0000 7fc57d8e2640 1 --2- 172.21.15.172:0/2956299101 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5595fe6a7bf0 0x5595fe6aa010 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80fa128fbfcb1bd2 server_cookie=def893eb278fb9c7 in_seq=0 out_seq=0 2024-09-10T08:51:10.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.525+0000 7fc55dffb640 1 -- 172.21.15.172:0/2956299101 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5595fe58f7f0 con 0x5595fe6a7bf0 2024-09-10T08:51:10.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.525+0000 7fc55dffb640 10 client.16938 handle_command_reply: tid=0 2024-09-10T08:51:10.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.525+0000 7fc582720740 1 client.16938 shutdown 2024-09-10T08:51:10.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.525+0000 7fc582720740 1 -- 172.21.15.172:0/2956299101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61cb60 msgr2=0x5595fe62fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:10.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.525+0000 7fc582720740 1 --2- 172.21.15.172:0/2956299101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61cb60 0x5595fe62fef0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc570098f40 tx=0x7fc5700047d0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.526+0000 7fc582720740 1 -- 172.21.15.172:0/2956299101 shutdown_connections 2024-09-10T08:51:10.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.526+0000 7fc582720740 1 --2- 172.21.15.172:0/2956299101 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5595fe6a7bf0 0x5595fe6aa010 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.526+0000 7fc582720740 1 --2- 172.21.15.172:0/2956299101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595fe627d40 0x5595fe62cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.526+0000 7fc582720740 1 --2- 172.21.15.172:0/2956299101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595fe61d480 0x5595fe630430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.526+0000 7fc582720740 1 --2- 172.21.15.172:0/2956299101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595fe61cb60 0x5595fe62fef0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.526+0000 7fc582720740 1 -- 172.21.15.172:0/2956299101 >> 172.21.15.172:0/2956299101 conn(0x5595fe6190f0 msgr2=0x5595fe618270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:10.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.526+0000 7fc55dffb640 0 client.16938 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:10.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.526+0000 7fc582720740 1 -- 172.21.15.172:0/2956299101 shutdown_connections 2024-09-10T08:51:10.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.526+0000 7fc582720740 1 -- 172.21.15.172:0/2956299101 wait complete. 2024-09-10T08:51:10.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.526+0000 7fc582720740 20 client.16938 trim_cache size 0 max 16384 2024-09-10T08:51:10.533 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:51:10.534 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T08:51:10.534 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "2e66925c-42fe-42e8-a37a-67785caf02f8", 2024-09-10T08:51:10.534 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T08:51:10.534 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:51:10.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.526+0000 7fc580347640 1 -- 172.21.15.172:0/224931219 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5600a2c00 msgr2=0x7fc5600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:10.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.526+0000 7fc580347640 1 --2- 172.21.15.172:0/224931219 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5600a2c00 0x7fc5600a50c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc568009430 tx=0x7fc568009360 comp rx=0 tx=0).stop 2024-09-10T08:51:10.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.526+0000 7fc580347640 1 -- 172.21.15.172:0/224931219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc578100e20 msgr2=0x7fc578199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:10.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.526+0000 7fc580347640 1 --2- 172.21.15.172:0/224931219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc578100e20 0x7fc578199200 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc57000b4a0 tx=0x7fc57000b960 comp rx=0 tx=0).stop 2024-09-10T08:51:10.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.527+0000 7fc580347640 1 -- 172.21.15.172:0/224931219 shutdown_connections 2024-09-10T08:51:10.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.527+0000 7fc580347640 1 --2- 172.21.15.172:0/224931219 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc5600a2c00 0x7fc5600a50c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.527+0000 7fc580347640 1 --2- 172.21.15.172:0/224931219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578102ef0 0x7fc5781a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.527+0000 7fc580347640 1 --2- 172.21.15.172:0/224931219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc578102570 0x7fc578199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.527+0000 7fc580347640 1 --2- 172.21.15.172:0/224931219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc578100e20 0x7fc578199200 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:10.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.527+0000 7fc580347640 1 -- 172.21.15.172:0/224931219 >> 172.21.15.172:0/224931219 conn(0x7fc5780fc7d0 msgr2=0x7fc578106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:10.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.527+0000 7fc580347640 1 -- 172.21.15.172:0/224931219 shutdown_connections 2024-09-10T08:51:10.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:10.527+0000 7fc580347640 1 -- 172.21.15.172:0/224931219 wait complete. 2024-09-10T08:51:10.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:10 smithi191 ceph-mon[35314]: pgmap v498: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 550 KiB/s wr, 10 op/s 2024-09-10T08:51:10.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:10 smithi174 ceph-mon[34068]: pgmap v498: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 550 KiB/s wr, 10 op/s 2024-09-10T08:51:11.376 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:51:11.413 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:51:11.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:11 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:51:11.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:11 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:51:11.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:11 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:51:11.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:11 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:11.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:11 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:11.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:11 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:51:11.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:11 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:11.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:11 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:51:11.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:11 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:11.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:11 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:11.635 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:51:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:11 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:51:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:11 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:51:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:11 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:51:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:11 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:11 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:11 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:51:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:11 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:11 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:51:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:11 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:11 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:11.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:11 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:51:11.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:11 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:51:11.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:11 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:51:11.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:11 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:11.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:11 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:11.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:11 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:51:11.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:11 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:11.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:11 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:51:11.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:11 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:11.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:11 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:11.907 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:51:12.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.115+0000 7fcacd3b0640 1 -- 172.21.15.172:0/2765964859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcac810c170 msgr2=0x7fcac810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.115+0000 7fcacd3b0640 1 --2- 172.21.15.172:0/2765964859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcac810c170 0x7fcac810e610 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009ca0 tx=0x7fcac002a950 comp rx=0 tx=0).stop 2024-09-10T08:51:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.115+0000 7fcacd3b0640 1 -- 172.21.15.172:0/2765964859 shutdown_connections 2024-09-10T08:51:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.115+0000 7fcacd3b0640 1 --2- 172.21.15.172:0/2765964859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcac810c170 0x7fcac810e610 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.115+0000 7fcacd3b0640 1 --2- 172.21.15.172:0/2765964859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcac8103660 0x7fcac810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.115+0000 7fcacd3b0640 1 --2- 172.21.15.172:0/2765964859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcac8102d40 0x7fcac8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.115+0000 7fcacd3b0640 1 -- 172.21.15.172:0/2765964859 >> 172.21.15.172:0/2765964859 conn(0x7fcac80fc7a0 msgr2=0x7fcac80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.115+0000 7fcacd3b0640 1 -- 172.21.15.172:0/2765964859 shutdown_connections 2024-09-10T08:51:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.115+0000 7fcacd3b0640 1 -- 172.21.15.172:0/2765964859 wait complete. 2024-09-10T08:51:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.116+0000 7fcacd3b0640 1 Processor -- start 2024-09-10T08:51:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.116+0000 7fcacd3b0640 1 -- start start 2024-09-10T08:51:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.116+0000 7fcacd3b0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcac8102d40 0x7fcac8198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.116+0000 7fcacd3b0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcac8103660 0x7fcac8198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.116+0000 7fcacd3b0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcac810c170 0x7fcac819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.116+0000 7fcacd3b0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8111e20 con 0x7fcac8103660 2024-09-10T08:51:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.116+0000 7fcacd3b0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8111ca0 con 0x7fcac810c170 2024-09-10T08:51:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.116+0000 7fcacd3b0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8111fa0 con 0x7fcac8102d40 2024-09-10T08:51:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.116+0000 7fcac67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcac8103660 0x7fcac8198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.117+0000 7fcac67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcac8103660 0x7fcac8198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52018/0 (socket says 172.21.15.172:52018) 2024-09-10T08:51:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.117+0000 7fcac67fc640 1 -- 172.21.15.172:0/4099076907 learned_addr learned my addr 172.21.15.172:0/4099076907 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.117+0000 7fcac6ffd640 1 --2- 172.21.15.172:0/4099076907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcac8102d40 0x7fcac8198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.117+0000 7fcac77fe640 1 --2- 172.21.15.172:0/4099076907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcac810c170 0x7fcac819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.117+0000 7fcac67fc640 1 -- 172.21.15.172:0/4099076907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcac8102d40 msgr2=0x7fcac8198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.117+0000 7fcac67fc640 1 --2- 172.21.15.172:0/4099076907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcac8102d40 0x7fcac8198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.117+0000 7fcac67fc640 1 -- 172.21.15.172:0/4099076907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcac810c170 msgr2=0x7fcac819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.117+0000 7fcac67fc640 1 --2- 172.21.15.172:0/4099076907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcac810c170 0x7fcac819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.117+0000 7fcac67fc640 1 -- 172.21.15.172:0/4099076907 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac819d2b0 con 0x7fcac8103660 2024-09-10T08:51:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.117+0000 7fcac6ffd640 1 --2- 172.21.15.172:0/4099076907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcac8102d40 0x7fcac8198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.117+0000 7fcac67fc640 1 --2- 172.21.15.172:0/4099076907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcac8103660 0x7fcac8198840 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fcab800c940 tx=0x7fcab800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.117+0000 7fcabffff640 1 -- 172.21.15.172:0/4099076907 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8004220 con 0x7fcac8103660 2024-09-10T08:51:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.117+0000 7fcabffff640 1 -- 172.21.15.172:0/4099076907 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcab80043c0 con 0x7fcac8103660 2024-09-10T08:51:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.117+0000 7fcabffff640 1 -- 172.21.15.172:0/4099076907 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8004a00 con 0x7fcac8103660 2024-09-10T08:51:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.117+0000 7fcacd3b0640 1 -- 172.21.15.172:0/4099076907 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac819d5a0 con 0x7fcac8103660 2024-09-10T08:51:12.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.117+0000 7fcacd3b0640 1 -- 172.21.15.172:0/4099076907 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac81a4e80 con 0x7fcac8103660 2024-09-10T08:51:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.119+0000 7fcabffff640 1 -- 172.21.15.172:0/4099076907 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fcab80079b0 con 0x7fcac8103660 2024-09-10T08:51:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.120+0000 7fcacf789740 1 Processor -- start 2024-09-10T08:51:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.120+0000 7fcacf789740 1 -- start start 2024-09-10T08:51:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.120+0000 7fcabffff640 1 --2- 172.21.15.172:0/4099076907 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcab40a2b80 0x7fcab40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.120+0000 7fcacf789740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784cf4a0 0x558e784cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.120+0000 7fcacf789740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784cfdc0 0x558e784da140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.120+0000 7fcacf789740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784da680 0x558e784dca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.120+0000 7fcacf789740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558e784352e0 con 0x558e784cf4a0 2024-09-10T08:51:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.120+0000 7fcacf789740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e783dcce0 con 0x558e784cfdc0 2024-09-10T08:51:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.120+0000 7fcacf789740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558e784312a0 con 0x558e784da680 2024-09-10T08:51:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.120+0000 7fcac77fe640 1 --2- 172.21.15.172:0/4099076907 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcab40a2b80 0x7fcab40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.120+0000 7fcac6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784cf4a0 0x558e784cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.120+0000 7fcac6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784cf4a0 0x558e784cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52026/0 (socket says 172.21.15.172:52026) 2024-09-10T08:51:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.120+0000 7fcac6ffd640 1 -- 172.21.15.172:0/278276097 learned_addr learned my addr 172.21.15.172:0/278276097 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.121+0000 7fcabffff640 1 -- 172.21.15.172:0/4099076907 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcab80cf320 con 0x7fcac8103660 2024-09-10T08:51:12.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.121+0000 7fcac67fc640 1 --2- 172.21.15.172:0/278276097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784cfdc0 0x558e784da140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.121+0000 7fcac77fe640 1 --2- 172.21.15.172:0/278276097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784da680 0x558e784dca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.121+0000 7fcac6ffd640 1 -- 172.21.15.172:0/278276097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784da680 msgr2=0x558e784dca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.121+0000 7fcac6ffd640 1 --2- 172.21.15.172:0/278276097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784da680 0x558e784dca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.121+0000 7fcac6ffd640 1 -- 172.21.15.172:0/278276097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784cfdc0 msgr2=0x558e784da140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.121+0000 7fcac6ffd640 1 --2- 172.21.15.172:0/278276097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784cfdc0 0x558e784da140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.121+0000 7fcac6ffd640 1 -- 172.21.15.172:0/278276097 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e783c1440 con 0x558e784cf4a0 2024-09-10T08:51:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.121+0000 7fcac67fc640 1 --2- 172.21.15.172:0/278276097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784cfdc0 0x558e784da140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:51:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.121+0000 7fcac6ffd640 1 --2- 172.21.15.172:0/278276097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784cf4a0 0x558e784cf880 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009c50 tx=0x7fcab000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14865e30310a623a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:12.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.121+0000 7fcac77fe640 1 --2- 172.21.15.172:0/4099076907 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcab40a2b80 0x7fcab40a5040 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcac0005fa0 tx=0x7fcac002f950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.121+0000 7fcabd7fa640 1 -- 172.21.15.172:0/278276097 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0016070 con 0x558e784cf4a0 2024-09-10T08:51:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.121+0000 7fcabd7fa640 1 -- 172.21.15.172:0/278276097 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcab0011920 con 0x558e784cf4a0 2024-09-10T08:51:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.121+0000 7fcabd7fa640 1 -- 172.21.15.172:0/278276097 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0010770 con 0x558e784cf4a0 2024-09-10T08:51:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.122+0000 7fcacf789740 1 -- 172.21.15.172:0/278276097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784cf4a0 msgr2=0x558e784cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.122+0000 7fcacf789740 1 --2- 172.21.15.172:0/278276097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784cf4a0 0x558e784cf880 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009c50 tx=0x7fcab000e6c0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.122+0000 7fcacf789740 1 -- 172.21.15.172:0/278276097 shutdown_connections 2024-09-10T08:51:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.122+0000 7fcacf789740 1 --2- 172.21.15.172:0/278276097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784da680 0x558e784dca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.122+0000 7fcacf789740 1 --2- 172.21.15.172:0/278276097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784cfdc0 0x558e784da140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.122+0000 7fcacf789740 1 --2- 172.21.15.172:0/278276097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784cf4a0 0x558e784cf880 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.122+0000 7fcacf789740 1 -- 172.21.15.172:0/278276097 >> 172.21.15.172:0/278276097 conn(0x558e784cba30 msgr2=0x558e784cbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.122+0000 7fcacf789740 1 -- 172.21.15.172:0/278276097 shutdown_connections 2024-09-10T08:51:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.122+0000 7fcacf789740 1 -- 172.21.15.172:0/278276097 wait complete. 2024-09-10T08:51:12.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.122+0000 7fcacf789740 1 -- start start 2024-09-10T08:51:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.122+0000 7fcacf789740 1 Processor -- start 2024-09-10T08:51:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcacf789740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784cf4a0 0x558e784d9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcacf789740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784cfdc0 0x558e784d7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcacf789740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784da680 0x558e784d77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcacf789740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558e784dd250 con 0x558e784cf4a0 2024-09-10T08:51:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcacf789740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e783f0050 con 0x558e784da680 2024-09-10T08:51:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcacf789740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558e784dd3d0 con 0x558e784cfdc0 2024-09-10T08:51:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcacf789740 10 client.0 _ll_register_callbacks cb 0x558e78430160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcac6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784cf4a0 0x558e784d9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcac6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784cf4a0 0x558e784d9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52042/0 (socket says 172.21.15.172:52042) 2024-09-10T08:51:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcac6ffd640 1 -- 172.21.15.172:0/332505014 learned_addr learned my addr 172.21.15.172:0/332505014 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcac77fe640 1 --2- 172.21.15.172:0/332505014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784da680 0x558e784d77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcac67fc640 1 --2- 172.21.15.172:0/332505014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784cfdc0 0x558e784d7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcac6ffd640 1 -- 172.21.15.172:0/332505014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784cfdc0 msgr2=0x558e784d7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:12.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcac6ffd640 1 --2- 172.21.15.172:0/332505014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784cfdc0 0x558e784d7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcac6ffd640 1 -- 172.21.15.172:0/332505014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784da680 msgr2=0x558e784d77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:12.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcac6ffd640 1 --2- 172.21.15.172:0/332505014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784da680 0x558e784d77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcac6ffd640 1 -- 172.21.15.172:0/332505014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e781a63c0 con 0x558e784cf4a0 2024-09-10T08:51:12.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcac67fc640 1 --2- 172.21.15.172:0/332505014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784cfdc0 0x558e784d7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:12.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcac77fe640 1 --2- 172.21.15.172:0/332505014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784da680 0x558e784d77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:51:12.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcac6ffd640 1 --2- 172.21.15.172:0/332505014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784cf4a0 0x558e784d9fd0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fcab0005e90 tx=0x7fcab0009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad3ce4183c676c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:12.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcaae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:12.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.123+0000 7fcaae7fc640 1 -- 172.21.15.172:0/332505014 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0016050 con 0x558e784cf4a0 2024-09-10T08:51:12.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.124+0000 7fcacf789740 1 -- 172.21.15.172:0/332505014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab0011920 con 0x558e784cf4a0 2024-09-10T08:51:12.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.124+0000 7fcaae7fc640 1 -- 172.21.15.172:0/332505014 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcab0005040 con 0x558e784cf4a0 2024-09-10T08:51:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.124+0000 7fcaae7fc640 1 -- 172.21.15.172:0/332505014 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcab0005240 con 0x558e784cf4a0 2024-09-10T08:51:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.124+0000 7fcacf789740 10 client.16950 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.124+0000 7fcacf789740 1 -- 172.21.15.172:0/332505014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558e784e8c80 con 0x558e784cf4a0 2024-09-10T08:51:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.124+0000 7fcaae7fc640 1 -- 172.21.15.172:0/332505014 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcab0011920 con 0x558e784cf4a0 2024-09-10T08:51:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.124+0000 7fcaae7fc640 1 -- 172.21.15.172:0/332505014 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fcab002b030 con 0x558e784cf4a0 2024-09-10T08:51:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.124+0000 7fcacf789740 10 client.16950 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.124+0000 7fcacf789740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:51:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.124+0000 7fcacf789740 20 client.16950 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.124+0000 7fcacf789740 1 --2- 172.21.15.172:0/332505014 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558e784ea140 0x558e784ec530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.124+0000 7fcacf789740 4 client.16950 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:51:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.124+0000 7fcacf789740 1 -- 172.21.15.172:0/332505014 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e78442850 con 0x558e784ea140 2024-09-10T08:51:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.124+0000 7fcac67fc640 1 --2- 172.21.15.172:0/332505014 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558e784ea140 0x558e784ec530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.125+0000 7fcaae7fc640 10 client.16950 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.125+0000 7fcac67fc640 1 --2- 172.21.15.172:0/332505014 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558e784ea140 0x558e784ec530 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0e398ea990cf5eb server_cookie=1d80e04cfa089d2c in_seq=0 out_seq=0 2024-09-10T08:51:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.125+0000 7fcaae7fc640 1 -- 172.21.15.172:0/332505014 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e78442850 con 0x558e784ea140 2024-09-10T08:51:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.125+0000 7fcaae7fc640 10 client.16950 handle_command_reply: tid=0 2024-09-10T08:51:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.126+0000 7fcacf789740 1 client.16950 shutdown 2024-09-10T08:51:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.126+0000 7fcacf789740 1 -- 172.21.15.172:0/332505014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784cf4a0 msgr2=0x558e784d9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.126+0000 7fcacf789740 1 --2- 172.21.15.172:0/332505014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784cf4a0 0x558e784d9fd0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fcab0005e90 tx=0x7fcab0009f70 comp rx=0 tx=0).stop 2024-09-10T08:51:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.126+0000 7fcacf789740 1 -- 172.21.15.172:0/332505014 shutdown_connections 2024-09-10T08:51:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.126+0000 7fcacf789740 1 --2- 172.21.15.172:0/332505014 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558e784ea140 0x558e784ec530 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.126+0000 7fcacf789740 1 --2- 172.21.15.172:0/332505014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784da680 0x558e784d77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.126+0000 7fcacf789740 1 --2- 172.21.15.172:0/332505014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784cfdc0 0x558e784d7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.126+0000 7fcacf789740 1 --2- 172.21.15.172:0/332505014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784cf4a0 0x558e784d9fd0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.126+0000 7fcaae7fc640 0 client.16950 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:12.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.126+0000 7fcacf789740 1 -- 172.21.15.172:0/332505014 >> 172.21.15.172:0/332505014 conn(0x558e784cba30 msgr2=0x558e784d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:12.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.126+0000 7fcacf789740 1 -- 172.21.15.172:0/332505014 shutdown_connections 2024-09-10T08:51:12.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.126+0000 7fcacf789740 1 -- 172.21.15.172:0/332505014 wait complete. 2024-09-10T08:51:12.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.126+0000 7fcacf789740 20 client.16950 trim_cache size 0 max 16384 2024-09-10T08:51:12.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcacf789740 1 Processor -- start 2024-09-10T08:51:12.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcacf789740 1 -- start start 2024-09-10T08:51:12.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcacf789740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784cf4a0 0x558e784e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcacf789740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784cfdc0 0x558e784d5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcacf789740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784da680 0x558e784f1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcacf789740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81aeb50 con 0x558e784da680 2024-09-10T08:51:12.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcacf789740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81b0750 con 0x558e784cf4a0 2024-09-10T08:51:12.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcacf789740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81af220 con 0x558e784cfdc0 2024-09-10T08:51:12.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcac77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784da680 0x558e784f1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcac77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784da680 0x558e784f1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52050/0 (socket says 172.21.15.172:52050) 2024-09-10T08:51:12.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcac77fe640 1 -- 172.21.15.172:0/3605369096 learned_addr learned my addr 172.21.15.172:0/3605369096 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:12.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcac6ffd640 1 --2- 172.21.15.172:0/3605369096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784cf4a0 0x558e784e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcac67fc640 1 --2- 172.21.15.172:0/3605369096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784cfdc0 0x558e784d5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcac77fe640 1 -- 172.21.15.172:0/3605369096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784cfdc0 msgr2=0x558e784d5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:12.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcac77fe640 1 --2- 172.21.15.172:0/3605369096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784cfdc0 0x558e784d5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcac77fe640 1 -- 172.21.15.172:0/3605369096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784cf4a0 msgr2=0x558e784e64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:12.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcac77fe640 1 --2- 172.21.15.172:0/3605369096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784cf4a0 0x558e784e64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcac77fe640 1 -- 172.21.15.172:0/3605369096 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e784d6230 con 0x558e784da680 2024-09-10T08:51:12.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.143+0000 7fcac67fc640 1 --2- 172.21.15.172:0/3605369096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784cfdc0 0x558e784d5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:12.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.144+0000 7fcac77fe640 1 --2- 172.21.15.172:0/3605369096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784da680 0x558e784f1c20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fcac0006010 tx=0x7fcac0048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616b48668f110d4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:12.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.144+0000 7fcabcff9640 1 -- 172.21.15.172:0/3605369096 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0034040 con 0x558e784da680 2024-09-10T08:51:12.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.144+0000 7fcabcff9640 1 -- 172.21.15.172:0/3605369096 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcac0003b70 con 0x558e784da680 2024-09-10T08:51:12.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.144+0000 7fcabcff9640 1 -- 172.21.15.172:0/3605369096 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac003d790 con 0x558e784da680 2024-09-10T08:51:12.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 1 -- 172.21.15.172:0/3605369096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784da680 msgr2=0x558e784f1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:12.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 1 --2- 172.21.15.172:0/3605369096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784da680 0x558e784f1c20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fcac0006010 tx=0x7fcac0048000 comp rx=0 tx=0).stop 2024-09-10T08:51:12.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 1 -- 172.21.15.172:0/3605369096 shutdown_connections 2024-09-10T08:51:12.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 1 --2- 172.21.15.172:0/3605369096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784da680 0x558e784f1c20 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 1 --2- 172.21.15.172:0/3605369096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784cfdc0 0x558e784d5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 1 --2- 172.21.15.172:0/3605369096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784cf4a0 0x558e784e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 1 -- 172.21.15.172:0/3605369096 >> 172.21.15.172:0/3605369096 conn(0x558e784cba30 msgr2=0x558e784d2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:12.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 1 -- 172.21.15.172:0/3605369096 shutdown_connections 2024-09-10T08:51:12.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 1 -- 172.21.15.172:0/3605369096 wait complete. 2024-09-10T08:51:12.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 1 -- start start 2024-09-10T08:51:12.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 1 Processor -- start 2024-09-10T08:51:12.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784cf4a0 0x558e784e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784cfdc0 0x558e784dde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784da680 0x558e784de360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81afff0 con 0x558e784da680 2024-09-10T08:51:12.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81af920 con 0x558e784cfdc0 2024-09-10T08:51:12.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81acb30 con 0x558e784cf4a0 2024-09-10T08:51:12.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcacf789740 10 client.0 _ll_register_callbacks cb 0x7fcab0009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:12.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.145+0000 7fcac77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784da680 0x558e784de360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.146+0000 7fcac77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784da680 0x558e784de360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52052/0 (socket says 172.21.15.172:52052) 2024-09-10T08:51:12.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.146+0000 7fcac77fe640 1 -- 172.21.15.172:0/2417762575 learned_addr learned my addr 172.21.15.172:0/2417762575 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:12.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.146+0000 7fcac6ffd640 1 --2- 172.21.15.172:0/2417762575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784cf4a0 0x558e784e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.146+0000 7fcac67fc640 1 --2- 172.21.15.172:0/2417762575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784cfdc0 0x558e784dde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.146+0000 7fcac77fe640 1 -- 172.21.15.172:0/2417762575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784cf4a0 msgr2=0x558e784e2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:12.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.146+0000 7fcac77fe640 1 --2- 172.21.15.172:0/2417762575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784cf4a0 0x558e784e2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.146+0000 7fcac77fe640 1 -- 172.21.15.172:0/2417762575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784cfdc0 msgr2=0x558e784dde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:12.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.146+0000 7fcac77fe640 1 --2- 172.21.15.172:0/2417762575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784cfdc0 0x558e784dde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.146+0000 7fcac77fe640 1 -- 172.21.15.172:0/2417762575 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e784de920 con 0x558e784da680 2024-09-10T08:51:12.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.146+0000 7fcac77fe640 1 --2- 172.21.15.172:0/2417762575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784da680 0x558e784de360 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fcac002ae70 tx=0x7fcac0004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb44ab63c5908430 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:12.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.146+0000 7fcaae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:12.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.146+0000 7fcaae7fc640 1 -- 172.21.15.172:0/2417762575 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0056020 con 0x558e784da680 2024-09-10T08:51:12.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.146+0000 7fcaae7fc640 1 -- 172.21.15.172:0/2417762575 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcac003d4d0 con 0x558e784da680 2024-09-10T08:51:12.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.146+0000 7fcacf789740 1 -- 172.21.15.172:0/2417762575 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcac0003b70 con 0x558e784da680 2024-09-10T08:51:12.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.146+0000 7fcaae7fc640 1 -- 172.21.15.172:0/2417762575 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcac0061050 con 0x558e784da680 2024-09-10T08:51:12.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.146+0000 7fcacf789740 10 client.16956 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:12.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.146+0000 7fcacf789740 1 -- 172.21.15.172:0/2417762575 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558e78558cf0 con 0x558e784da680 2024-09-10T08:51:12.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.147+0000 7fcaae7fc640 1 -- 172.21.15.172:0/2417762575 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcac0003b70 con 0x558e784da680 2024-09-10T08:51:12.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.147+0000 7fcaae7fc640 1 -- 172.21.15.172:0/2417762575 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7fcac00306d0 con 0x558e784da680 2024-09-10T08:51:12.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.147+0000 7fcacf789740 10 client.16956 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:12.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.147+0000 7fcacf789740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:51:12.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.147+0000 7fcacf789740 20 client.16956 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:12.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.147+0000 7fcacf789740 1 --2- 172.21.15.172:0/2417762575 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558e785593e0 0x558e7855b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:12.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.147+0000 7fcacf789740 4 client.16956 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:51:12.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.147+0000 7fcacf789740 1 -- 172.21.15.172:0/2417762575 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558e7843cd60 con 0x558e785593e0 2024-09-10T08:51:12.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.147+0000 7fcac6ffd640 1 --2- 172.21.15.172:0/2417762575 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558e785593e0 0x558e7855b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:12.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.147+0000 7fcaae7fc640 10 client.16956 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:12.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.147+0000 7fcac6ffd640 1 --2- 172.21.15.172:0/2417762575 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558e785593e0 0x558e7855b800 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6e9ebc19344f33f server_cookie=56a3b200cf6956c2 in_seq=0 out_seq=0 2024-09-10T08:51:12.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcaae7fc640 1 -- 172.21.15.172:0/2417762575 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558e7843cd60 con 0x558e785593e0 2024-09-10T08:51:12.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcaae7fc640 10 client.16956 handle_command_reply: tid=0 2024-09-10T08:51:12.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcacf789740 1 client.16956 shutdown 2024-09-10T08:51:12.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcacf789740 1 -- 172.21.15.172:0/2417762575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784da680 msgr2=0x558e784de360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:12.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcacf789740 1 --2- 172.21.15.172:0/2417762575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784da680 0x558e784de360 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fcac002ae70 tx=0x7fcac0004790 comp rx=0 tx=0).stop 2024-09-10T08:51:12.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcacf789740 1 -- 172.21.15.172:0/2417762575 shutdown_connections 2024-09-10T08:51:12.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcacf789740 1 --2- 172.21.15.172:0/2417762575 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558e785593e0 0x558e7855b800 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcacf789740 1 --2- 172.21.15.172:0/2417762575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e784da680 0x558e784de360 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcacf789740 1 --2- 172.21.15.172:0/2417762575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e784cfdc0 0x558e784dde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcacf789740 1 --2- 172.21.15.172:0/2417762575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558e784cf4a0 0x558e784e2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcacf789740 1 -- 172.21.15.172:0/2417762575 >> 172.21.15.172:0/2417762575 conn(0x558e784cba30 msgr2=0x558e784caa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:12.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcaae7fc640 0 client.16956 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:12.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcacf789740 1 -- 172.21.15.172:0/2417762575 shutdown_connections 2024-09-10T08:51:12.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcacf789740 1 -- 172.21.15.172:0/2417762575 wait complete. 2024-09-10T08:51:12.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcacf789740 20 client.16956 trim_cache size 0 max 16384 2024-09-10T08:51:12.158 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:51:12.158 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:51:12.158 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T08:51:12.158 INFO:teuthology.orchestra.run.smithi172.stdout: "2e66925c-42fe-42e8-a37a-67785caf02f8": { 2024-09-10T08:51:12.158 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T08:51:12.158 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "2e66925c-42fe-42e8-a37a-67785caf02f8", 2024-09-10T08:51:12.158 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T08:51:12.158 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:51:12.158 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:51:12.159 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:51:12.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcacd3b0640 1 -- 172.21.15.172:0/4099076907 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcab40a2b80 msgr2=0x7fcab40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:12.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcacd3b0640 1 --2- 172.21.15.172:0/4099076907 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcab40a2b80 0x7fcab40a5040 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcac0005fa0 tx=0x7fcac002f950 comp rx=0 tx=0).stop 2024-09-10T08:51:12.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcacd3b0640 1 -- 172.21.15.172:0/4099076907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcac8103660 msgr2=0x7fcac8198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:12.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.148+0000 7fcacd3b0640 1 --2- 172.21.15.172:0/4099076907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcac8103660 0x7fcac8198840 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fcab800c940 tx=0x7fcab800ce00 comp rx=0 tx=0).stop 2024-09-10T08:51:12.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.149+0000 7fcacd3b0640 1 -- 172.21.15.172:0/4099076907 shutdown_connections 2024-09-10T08:51:12.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.149+0000 7fcacd3b0640 1 --2- 172.21.15.172:0/4099076907 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcab40a2b80 0x7fcab40a5040 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.149+0000 7fcacd3b0640 1 --2- 172.21.15.172:0/4099076907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcac810c170 0x7fcac819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.149+0000 7fcacd3b0640 1 --2- 172.21.15.172:0/4099076907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcac8103660 0x7fcac8198840 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.149+0000 7fcacd3b0640 1 --2- 172.21.15.172:0/4099076907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcac8102d40 0x7fcac8198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:12.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.149+0000 7fcacd3b0640 1 -- 172.21.15.172:0/4099076907 >> 172.21.15.172:0/4099076907 conn(0x7fcac80fc7a0 msgr2=0x7fcac80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:12.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.149+0000 7fcacd3b0640 1 -- 172.21.15.172:0/4099076907 shutdown_connections 2024-09-10T08:51:12.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.149+0000 7fcacd3b0640 1 -- 172.21.15.172:0/4099076907 wait complete. 2024-09-10T08:51:12.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:12 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T08:51:12.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:12 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:12.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:12 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:51:12.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:12 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:12.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:12 smithi172 ceph-mon[30854]: pgmap v499: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 551 KiB/s wr, 16 op/s 2024-09-10T08:51:12.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:12 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T08:51:12.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:12 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:12.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:12 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:51:12.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:12 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:12.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:12 smithi174 ceph-mon[34068]: pgmap v499: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 551 KiB/s wr, 16 op/s 2024-09-10T08:51:12.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:12 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T08:51:12.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:12 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:12.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:12 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:51:12.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:12 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:51:12.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:12 smithi191 ceph-mon[35314]: pgmap v499: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 551 KiB/s wr, 16 op/s 2024-09-10T08:51:13.358 INFO:tasks.cephfs.filesystem:scrub status for tag:2e66925c-42fe-42e8-a37a-67785caf02f8 - {'path': '/', 'tag': '2e66925c-42fe-42e8-a37a-67785caf02f8', 'options': 'recursive,force'} 2024-09-10T08:51:13.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.886+0000 7f95180fa640 1 -- 172.21.15.172:0/2579216743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9510101de0 msgr2=0x7f95101021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:13.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.886+0000 7f95180fa640 1 --2- 172.21.15.172:0/2579216743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9510101de0 0x7f95101021c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9500009900 tx=0x7f950002a410 comp rx=0 tx=0).stop 2024-09-10T08:51:13.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.886+0000 7f95180fa640 1 -- 172.21.15.172:0/2579216743 shutdown_connections 2024-09-10T08:51:13.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.886+0000 7f95180fa640 1 --2- 172.21.15.172:0/2579216743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9510106f10 0x7f951010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:13.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.886+0000 7f95180fa640 1 --2- 172.21.15.172:0/2579216743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9510102790 0x7f95101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:13.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.886+0000 7f95180fa640 1 --2- 172.21.15.172:0/2579216743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9510101de0 0x7f95101021c0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:13.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.886+0000 7f95180fa640 1 -- 172.21.15.172:0/2579216743 >> 172.21.15.172:0/2579216743 conn(0x7f95100fd8c0 msgr2=0x7f95100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:13.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.886+0000 7f95180fa640 1 -- 172.21.15.172:0/2579216743 shutdown_connections 2024-09-10T08:51:13.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.886+0000 7f95180fa640 1 -- 172.21.15.172:0/2579216743 wait complete. 2024-09-10T08:51:13.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.887+0000 7f95180fa640 1 Processor -- start 2024-09-10T08:51:13.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.887+0000 7f95180fa640 1 -- start start 2024-09-10T08:51:13.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.887+0000 7f95180fa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9510101de0 0x7f9510198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:13.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.887+0000 7f95180fa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9510102790 0x7f95101988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:13.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.887+0000 7f95180fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9510106f10 0x7f951019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:13.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.887+0000 7f95180fa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9510111e40 con 0x7f9510106f10 2024-09-10T08:51:13.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.887+0000 7f95180fa640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9510111cc0 con 0x7f9510101de0 2024-09-10T08:51:13.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.887+0000 7f95180fa640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9510111fc0 con 0x7f9510102790 2024-09-10T08:51:13.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.887+0000 7f9516697640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9510106f10 0x7f951019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:13.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.887+0000 7f9516697640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9510106f10 0x7f951019cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52062/0 (socket says 172.21.15.172:52062) 2024-09-10T08:51:13.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.887+0000 7f9516697640 1 -- 172.21.15.172:0/1656266912 learned_addr learned my addr 172.21.15.172:0/1656266912 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:13.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.887+0000 7f9515e96640 1 --2- 172.21.15.172:0/1656266912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9510101de0 0x7f9510198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:13.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.887+0000 7f9515695640 1 --2- 172.21.15.172:0/1656266912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9510102790 0x7f95101988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:13.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.887+0000 7f9516697640 1 -- 172.21.15.172:0/1656266912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9510102790 msgr2=0x7f95101988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:13.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.887+0000 7f9516697640 1 --2- 172.21.15.172:0/1656266912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9510102790 0x7f95101988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:13.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.888+0000 7f9516697640 1 -- 172.21.15.172:0/1656266912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9510101de0 msgr2=0x7f9510198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:13.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.888+0000 7f9516697640 1 --2- 172.21.15.172:0/1656266912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9510101de0 0x7f9510198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:13.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.888+0000 7f9516697640 1 -- 172.21.15.172:0/1656266912 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951019d330 con 0x7f9510106f10 2024-09-10T08:51:13.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.888+0000 7f9515e96640 1 --2- 172.21.15.172:0/1656266912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9510101de0 0x7f9510198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:13.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.888+0000 7f9515695640 1 --2- 172.21.15.172:0/1656266912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9510102790 0x7f95101988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:13.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.888+0000 7f9516697640 1 --2- 172.21.15.172:0/1656266912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9510106f10 0x7f951019cc50 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f950800cce0 tx=0x7f950800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:13.456 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:51:13.457 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":33,"btime":"2024-09-10T08:50:03:285217+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:50:02.654978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:51:13.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.888+0000 7f9506ffd640 1 -- 172.21.15.172:0/1656266912 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950800f450 con 0x7f9510106f10 2024-09-10T08:51:13.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.888+0000 7f9506ffd640 1 -- 172.21.15.172:0/1656266912 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f950800faa0 con 0x7f9510106f10 2024-09-10T08:51:13.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.888+0000 7f95180fa640 1 -- 172.21.15.172:0/1656266912 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951019d620 con 0x7f9510106f10 2024-09-10T08:51:13.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.888+0000 7f9506ffd640 1 -- 172.21.15.172:0/1656266912 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508018b70 con 0x7f9510106f10 2024-09-10T08:51:13.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.888+0000 7f95180fa640 1 -- 172.21.15.172:0/1656266912 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95101a4f00 con 0x7f9510106f10 2024-09-10T08:51:13.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.889+0000 7f95180fa640 1 -- 172.21.15.172:0/1656266912 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95101021c0 con 0x7f9510106f10 2024-09-10T08:51:13.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.890+0000 7f9506ffd640 1 -- 172.21.15.172:0/1656266912 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9508018d10 con 0x7f9510106f10 2024-09-10T08:51:13.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.890+0000 7f9506ffd640 1 --2- 172.21.15.172:0/1656266912 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94fc0a2b80 0x7f94fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:13.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.891+0000 7f9515e96640 1 --2- 172.21.15.172:0/1656266912 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94fc0a2b80 0x7f94fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:13.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.891+0000 7f9506ffd640 1 -- 172.21.15.172:0/1656266912 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f95080cf8f0 con 0x7f9510106f10 2024-09-10T08:51:13.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.891+0000 7f9515e96640 1 --2- 172.21.15.172:0/1656266912 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94fc0a2b80 0x7f94fc0a5040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9500000c00 tx=0x7f9500031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:13.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:12.895+0000 7f9506ffd640 1 -- 172.21.15.172:0/1656266912 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9508014030 con 0x7f9510106f10 2024-09-10T08:51:13.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:13.098+0000 7f95180fa640 1 -- 172.21.15.172:0/1656266912 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9510109b60 con 0x7f9510106f10 2024-09-10T08:51:13.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:13.098+0000 7f9506ffd640 1 -- 172.21.15.172:0/1656266912 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+12007 (secure 0 0 0) 0x7f950809c490 con 0x7f9510106f10 2024-09-10T08:51:13.458 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 33 2024-09-10T08:51:13.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:13.101+0000 7f95180fa640 1 -- 172.21.15.172:0/1656266912 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94fc0a2b80 msgr2=0x7f94fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:13.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:13.101+0000 7f95180fa640 1 --2- 172.21.15.172:0/1656266912 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94fc0a2b80 0x7f94fc0a5040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9500000c00 tx=0x7f9500031400 comp rx=0 tx=0).stop 2024-09-10T08:51:13.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:13.101+0000 7f95180fa640 1 -- 172.21.15.172:0/1656266912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9510106f10 msgr2=0x7f951019cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:13.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:13.101+0000 7f95180fa640 1 --2- 172.21.15.172:0/1656266912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9510106f10 0x7f951019cc50 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f950800cce0 tx=0x7f950800a620 comp rx=0 tx=0).stop 2024-09-10T08:51:13.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:13.101+0000 7f95180fa640 1 -- 172.21.15.172:0/1656266912 shutdown_connections 2024-09-10T08:51:13.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:13.101+0000 7f95180fa640 1 --2- 172.21.15.172:0/1656266912 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94fc0a2b80 0x7f94fc0a5040 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:13.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:13.101+0000 7f95180fa640 1 --2- 172.21.15.172:0/1656266912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9510106f10 0x7f951019cc50 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:13.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:13.101+0000 7f95180fa640 1 --2- 172.21.15.172:0/1656266912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9510102790 0x7f95101988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:13.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:13.101+0000 7f95180fa640 1 --2- 172.21.15.172:0/1656266912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9510101de0 0x7f9510198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:13.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:13.101+0000 7f95180fa640 1 -- 172.21.15.172:0/1656266912 >> 172.21.15.172:0/1656266912 conn(0x7f95100fd8c0 msgr2=0x7f95100fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:13.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:13.101+0000 7f95180fa640 1 -- 172.21.15.172:0/1656266912 shutdown_connections 2024-09-10T08:51:13.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:13.101+0000 7f95180fa640 1 -- 172.21.15.172:0/1656266912 wait complete. 2024-09-10T08:51:13.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:13 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1656266912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:51:13.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:13 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1656266912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:51:13.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:13 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1656266912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:51:14.069 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 8cccb995 --release --await' 2024-09-10T08:51:14.069 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 8cccb995 --release --await 2024-09-10T08:51:14.318 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:51:14.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:14 smithi172 ceph-mon[30854]: pgmap v500: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.9 KiB/s wr, 17 op/s 2024-09-10T08:51:14.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:14 smithi174 ceph-mon[34068]: pgmap v500: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.9 KiB/s wr, 17 op/s 2024-09-10T08:51:14.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:14 smithi191 ceph-mon[35314]: pgmap v500: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.9 KiB/s wr, 17 op/s 2024-09-10T08:51:14.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.814+0000 7f228f871640 1 -- 172.21.15.172:0/3701601219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2288101df0 msgr2=0x7f22881021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:14.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.814+0000 7f228f871640 1 --2- 172.21.15.172:0/3701601219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2288101df0 0x7f22881021d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2284009900 tx=0x7f228402a410 comp rx=0 tx=0).stop 2024-09-10T08:51:14.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.814+0000 7f228f871640 1 -- 172.21.15.172:0/3701601219 shutdown_connections 2024-09-10T08:51:14.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.814+0000 7f228f871640 1 --2- 172.21.15.172:0/3701601219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2288106f20 0x7f228810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.814+0000 7f228f871640 1 --2- 172.21.15.172:0/3701601219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22881027a0 0x7f22881067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.814+0000 7f228f871640 1 --2- 172.21.15.172:0/3701601219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2288101df0 0x7f22881021d0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.814+0000 7f228f871640 1 -- 172.21.15.172:0/3701601219 >> 172.21.15.172:0/3701601219 conn(0x7f22880fd8f0 msgr2=0x7f22880ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:14.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.814+0000 7f228f871640 1 -- 172.21.15.172:0/3701601219 shutdown_connections 2024-09-10T08:51:14.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.814+0000 7f228f871640 1 -- 172.21.15.172:0/3701601219 wait complete. 2024-09-10T08:51:14.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.815+0000 7f228f871640 1 Processor -- start 2024-09-10T08:51:14.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.815+0000 7f228f871640 1 -- start start 2024-09-10T08:51:14.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.815+0000 7f228f871640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2288101df0 0x7f2288198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.815+0000 7f228f871640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22881027a0 0x7f2288198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.815+0000 7f228f871640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2288106f20 0x7f228819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.815+0000 7f228f871640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2288111e30 con 0x7f22881027a0 2024-09-10T08:51:14.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.815+0000 7f228f871640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2288111cb0 con 0x7f2288101df0 2024-09-10T08:51:14.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.815+0000 7f228f871640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2288111fb0 con 0x7f2288106f20 2024-09-10T08:51:14.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.815+0000 7f228ce0c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22881027a0 0x7f2288198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.815+0000 7f228ce0c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22881027a0 0x7f2288198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52074/0 (socket says 172.21.15.172:52074) 2024-09-10T08:51:14.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.815+0000 7f228ce0c640 1 -- 172.21.15.172:0/828806103 learned_addr learned my addr 172.21.15.172:0/828806103 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:14.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.815+0000 7f228de0e640 1 --2- 172.21.15.172:0/828806103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2288106f20 0x7f228819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.815+0000 7f228d60d640 1 --2- 172.21.15.172:0/828806103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2288101df0 0x7f2288198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.815+0000 7f228ce0c640 1 -- 172.21.15.172:0/828806103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2288106f20 msgr2=0x7f228819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:14.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.815+0000 7f228ce0c640 1 --2- 172.21.15.172:0/828806103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2288106f20 0x7f228819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.816+0000 7f228ce0c640 1 -- 172.21.15.172:0/828806103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2288101df0 msgr2=0x7f2288198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:14.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.816+0000 7f228ce0c640 1 --2- 172.21.15.172:0/828806103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2288101df0 0x7f2288198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.816+0000 7f228ce0c640 1 -- 172.21.15.172:0/828806103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228819d3c0 con 0x7f22881027a0 2024-09-10T08:51:14.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.816+0000 7f228ce0c640 1 --2- 172.21.15.172:0/828806103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22881027a0 0x7f2288198950 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f227800d950 tx=0x7f227800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:14.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.816+0000 7f227e7fc640 1 -- 172.21.15.172:0/828806103 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278017070 con 0x7f22881027a0 2024-09-10T08:51:14.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.816+0000 7f228f871640 1 -- 172.21.15.172:0/828806103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228819d6b0 con 0x7f22881027a0 2024-09-10T08:51:14.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.816+0000 7f227e7fc640 1 -- 172.21.15.172:0/828806103 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2278009c30 con 0x7f22881027a0 2024-09-10T08:51:14.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.816+0000 7f227e7fc640 1 -- 172.21.15.172:0/828806103 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227800f620 con 0x7f22881027a0 2024-09-10T08:51:14.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.816+0000 7f228f871640 1 -- 172.21.15.172:0/828806103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22881a4f90 con 0x7f22881027a0 2024-09-10T08:51:14.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.818+0000 7f227e7fc640 1 -- 172.21.15.172:0/828806103 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f22780109a0 con 0x7f22881027a0 2024-09-10T08:51:14.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.819+0000 7f2291c4a740 1 Processor -- start 2024-09-10T08:51:14.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.819+0000 7f2291c4a740 1 -- start start 2024-09-10T08:51:14.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.819+0000 7f2291c4a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefbf40 0x5651acefc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.819+0000 7f2291c4a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acefc860 0x5651acf0b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.819+0000 7f2291c4a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acf0b650 0x5651acf0da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.819+0000 7f2291c4a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5651ace61620 con 0x5651acefbf40 2024-09-10T08:51:14.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.819+0000 7f2291c4a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5651ace08e50 con 0x5651acf0b650 2024-09-10T08:51:14.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.819+0000 7f2291c4a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5651ace5d5e0 con 0x5651acefc860 2024-09-10T08:51:14.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.819+0000 7f227e7fc640 1 --2- 172.21.15.172:0/828806103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22740a2b80 0x7f22740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.819+0000 7f228de0e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefbf40 0x5651acefc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.819+0000 7f228de0e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefbf40 0x5651acefc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52078/0 (socket says 172.21.15.172:52078) 2024-09-10T08:51:14.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.819+0000 7f228de0e640 1 -- 172.21.15.172:0/7660453 learned_addr learned my addr 172.21.15.172:0/7660453 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:14.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.819+0000 7f228de0e640 1 --2- 172.21.15.172:0/828806103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22740a2b80 0x7f22740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.819+0000 7f228d60d640 1 --2- 172.21.15.172:0/7660453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acefc860 0x5651acf0b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.819+0000 7f227e7fc640 1 -- 172.21.15.172:0/828806103 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f22780cef40 con 0x7f22881027a0 2024-09-10T08:51:14.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.819+0000 7f228ce0c640 1 --2- 172.21.15.172:0/7660453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acf0b650 0x5651acf0da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.820+0000 7f228de0e640 1 -- 172.21.15.172:0/7660453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acefc860 msgr2=0x5651acf0b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:14.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.820+0000 7f228de0e640 1 --2- 172.21.15.172:0/7660453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acefc860 0x5651acf0b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.820+0000 7f228de0e640 1 -- 172.21.15.172:0/7660453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acf0b650 msgr2=0x5651acf0da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:14.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.820+0000 7f228de0e640 1 --2- 172.21.15.172:0/7660453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acf0b650 0x5651acf0da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.820+0000 7f228de0e640 1 -- 172.21.15.172:0/7660453 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651acdf9d40 con 0x5651acefbf40 2024-09-10T08:51:14.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.820+0000 7f228de0e640 1 --2- 172.21.15.172:0/7660453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefbf40 0x5651acefc320 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f2280012d00 tx=0x7f2280010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e84bc37fee4a74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:14.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.820+0000 7f228de0e640 1 --2- 172.21.15.172:0/828806103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22740a2b80 0x7f22740a5040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2280010c40 tx=0x7f228000c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:14.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.820+0000 7f22737fe640 1 -- 172.21.15.172:0/7660453 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280024070 con 0x5651acefbf40 2024-09-10T08:51:14.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.820+0000 7f22737fe640 1 -- 172.21.15.172:0/7660453 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2280009d20 con 0x5651acefbf40 2024-09-10T08:51:14.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.820+0000 7f22737fe640 1 -- 172.21.15.172:0/7660453 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280005280 con 0x5651acefbf40 2024-09-10T08:51:14.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.821+0000 7f2291c4a740 1 -- 172.21.15.172:0/7660453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefbf40 msgr2=0x5651acefc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:14.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.821+0000 7f2291c4a740 1 --2- 172.21.15.172:0/7660453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefbf40 0x5651acefc320 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f2280012d00 tx=0x7f2280010740 comp rx=0 tx=0).stop 2024-09-10T08:51:14.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.821+0000 7f2291c4a740 1 -- 172.21.15.172:0/7660453 shutdown_connections 2024-09-10T08:51:14.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.821+0000 7f2291c4a740 1 --2- 172.21.15.172:0/7660453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acf0b650 0x5651acf0da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.821+0000 7f2291c4a740 1 --2- 172.21.15.172:0/7660453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acefc860 0x5651acf0b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.821+0000 7f2291c4a740 1 --2- 172.21.15.172:0/7660453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefbf40 0x5651acefc320 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.821+0000 7f2291c4a740 1 -- 172.21.15.172:0/7660453 >> 172.21.15.172:0/7660453 conn(0x5651acef85f0 msgr2=0x5651acef89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:14.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.821+0000 7f2291c4a740 1 -- 172.21.15.172:0/7660453 shutdown_connections 2024-09-10T08:51:14.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.821+0000 7f2291c4a740 1 -- 172.21.15.172:0/7660453 wait complete. 2024-09-10T08:51:14.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.821+0000 7f2291c4a740 1 -- start start 2024-09-10T08:51:14.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.821+0000 7f2291c4a740 1 Processor -- start 2024-09-10T08:51:14.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f2291c4a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acefbf40 0x5651acf08610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f2291c4a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefc860 0x5651acf08b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f2291c4a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acf0b650 0x5651acf09090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f2291c4a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5651acf0e220 con 0x5651acefc860 2024-09-10T08:51:14.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f2291c4a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5651acde9f10 con 0x5651acf0b650 2024-09-10T08:51:14.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f2291c4a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5651acf0e3a0 con 0x5651acefbf40 2024-09-10T08:51:14.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f2291c4a740 10 client.0 _ll_register_callbacks cb 0x5651ace0ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:14.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f228ce0c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefc860 0x5651acf08b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f228ce0c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefc860 0x5651acf08b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52082/0 (socket says 172.21.15.172:52082) 2024-09-10T08:51:14.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f228ce0c640 1 -- 172.21.15.172:0/2650758727 learned_addr learned my addr 172.21.15.172:0/2650758727 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:14.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f228d60d640 1 --2- 172.21.15.172:0/2650758727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acefbf40 0x5651acf08610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f228de0e640 1 --2- 172.21.15.172:0/2650758727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acf0b650 0x5651acf09090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f228ce0c640 1 -- 172.21.15.172:0/2650758727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acefbf40 msgr2=0x5651acf08610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:14.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f228ce0c640 1 --2- 172.21.15.172:0/2650758727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acefbf40 0x5651acf08610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f228ce0c640 1 -- 172.21.15.172:0/2650758727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acf0b650 msgr2=0x5651acf09090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:14.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f228ce0c640 1 --2- 172.21.15.172:0/2650758727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acf0b650 0x5651acf09090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f228ce0c640 1 -- 172.21.15.172:0/2650758727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651acb70580 con 0x5651acefc860 2024-09-10T08:51:14.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f228ce0c640 1 --2- 172.21.15.172:0/2650758727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefc860 0x5651acf08b50 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f227809b1f0 tx=0x7f227809b030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceed2be1ae06786d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:14.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.822+0000 7f2270ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:14.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.823+0000 7f2270ff9640 1 -- 172.21.15.172:0/2650758727 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278020070 con 0x5651acefc860 2024-09-10T08:51:14.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.823+0000 7f2270ff9640 1 -- 172.21.15.172:0/2650758727 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f22780ccba0 con 0x5651acefc860 2024-09-10T08:51:14.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.823+0000 7f2291c4a740 1 -- 172.21.15.172:0/2650758727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5651ace6eb90 con 0x5651acefc860 2024-09-10T08:51:14.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.823+0000 7f2270ff9640 1 -- 172.21.15.172:0/2650758727 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f227809b4f0 con 0x5651acefc860 2024-09-10T08:51:14.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.823+0000 7f2291c4a740 10 client.16974 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:14.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.823+0000 7f2291c4a740 1 -- 172.21.15.172:0/2650758727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5651acc3a960 con 0x5651acefc860 2024-09-10T08:51:14.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.823+0000 7f2270ff9640 1 -- 172.21.15.172:0/2650758727 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5651ace6eb90 con 0x5651acefc860 2024-09-10T08:51:14.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.823+0000 7f2270ff9640 1 -- 172.21.15.172:0/2650758727 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f2278010460 con 0x5651acefc860 2024-09-10T08:51:14.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.823+0000 7f2291c4a740 10 client.16974 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:14.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.823+0000 7f2291c4a740 10 client.16974 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:51:14.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.823+0000 7f2291c4a740 20 client.16974 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:14.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.823+0000 7f2291c4a740 1 --2- 172.21.15.172:0/2650758727 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5651acf1b370 0x5651acf1d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.823+0000 7f2291c4a740 4 client.16974 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:51:14.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.823+0000 7f2291c4a740 1 -- 172.21.15.172:0/2650758727 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651ace690a0 con 0x5651acf1b370 2024-09-10T08:51:14.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.824+0000 7f228d60d640 1 --2- 172.21.15.172:0/2650758727 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5651acf1b370 0x5651acf1d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.824+0000 7f2270ff9640 10 client.16974 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:14.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.824+0000 7f228d60d640 1 --2- 172.21.15.172:0/2650758727 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5651acf1b370 0x5651acf1d760 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e6f4ec5a17d2ca4 server_cookie=cdf8010e723e1d74 in_seq=0 out_seq=0 2024-09-10T08:51:14.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.825+0000 7f2270ff9640 1 -- 172.21.15.172:0/2650758727 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651ace690a0 con 0x5651acf1b370 2024-09-10T08:51:14.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.825+0000 7f2270ff9640 10 client.16974 handle_command_reply: tid=0 2024-09-10T08:51:14.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.825+0000 7f2291c4a740 1 client.16974 shutdown 2024-09-10T08:51:14.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.825+0000 7f2291c4a740 1 -- 172.21.15.172:0/2650758727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefc860 msgr2=0x5651acf08b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:14.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.825+0000 7f2291c4a740 1 --2- 172.21.15.172:0/2650758727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefc860 0x5651acf08b50 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f227809b1f0 tx=0x7f227809b030 comp rx=0 tx=0).stop 2024-09-10T08:51:14.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.825+0000 7f2291c4a740 1 -- 172.21.15.172:0/2650758727 shutdown_connections 2024-09-10T08:51:14.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.825+0000 7f2291c4a740 1 --2- 172.21.15.172:0/2650758727 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5651acf1b370 0x5651acf1d760 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.825+0000 7f2291c4a740 1 --2- 172.21.15.172:0/2650758727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acf0b650 0x5651acf09090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.825+0000 7f2291c4a740 1 --2- 172.21.15.172:0/2650758727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefc860 0x5651acf08b50 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.825+0000 7f2270ff9640 0 client.16974 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:14.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.825+0000 7f2291c4a740 1 --2- 172.21.15.172:0/2650758727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acefbf40 0x5651acf08610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.825+0000 7f2291c4a740 1 -- 172.21.15.172:0/2650758727 >> 172.21.15.172:0/2650758727 conn(0x5651acef85f0 msgr2=0x5651acefe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:14.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.825+0000 7f2291c4a740 1 -- 172.21.15.172:0/2650758727 shutdown_connections 2024-09-10T08:51:14.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.825+0000 7f2291c4a740 1 -- 172.21.15.172:0/2650758727 wait complete. 2024-09-10T08:51:14.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.825+0000 7f2291c4a740 20 client.16974 trim_cache size 0 max 16384 2024-09-10T08:51:14.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.843+0000 7f2291c4a740 1 Processor -- start 2024-09-10T08:51:14.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.843+0000 7f2291c4a740 1 -- start start 2024-09-10T08:51:14.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.843+0000 7f2291c4a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acefbf40 0x5651acf0a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.843+0000 7f2291c4a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acefc860 0x5651acf22fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.843+0000 7f2291c4a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acf0b650 0x5651acf23500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.843+0000 7f2291c4a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f22881ac8e0 con 0x5651acf0b650 2024-09-10T08:51:14.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.843+0000 7f2291c4a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f22881ae4e0 con 0x5651acefc860 2024-09-10T08:51:14.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.843+0000 7f2291c4a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f22881acfb0 con 0x5651acefbf40 2024-09-10T08:51:14.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.843+0000 7f228de0e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acf0b650 0x5651acf23500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.843+0000 7f228de0e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acf0b650 0x5651acf23500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52098/0 (socket says 172.21.15.172:52098) 2024-09-10T08:51:14.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.843+0000 7f228de0e640 1 -- 172.21.15.172:0/3670726349 learned_addr learned my addr 172.21.15.172:0/3670726349 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:14.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.843+0000 7f228ce0c640 1 --2- 172.21.15.172:0/3670726349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acefc860 0x5651acf22fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.843+0000 7f228d60d640 1 --2- 172.21.15.172:0/3670726349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acefbf40 0x5651acf0a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.844+0000 7f228de0e640 1 -- 172.21.15.172:0/3670726349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acefbf40 msgr2=0x5651acf0a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:14.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.844+0000 7f228de0e640 1 --2- 172.21.15.172:0/3670726349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acefbf40 0x5651acf0a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.844+0000 7f228de0e640 1 -- 172.21.15.172:0/3670726349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acefc860 msgr2=0x5651acf22fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:14.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.844+0000 7f228de0e640 1 --2- 172.21.15.172:0/3670726349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acefc860 0x5651acf22fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.844+0000 7f228de0e640 1 -- 172.21.15.172:0/3670726349 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651acf1dd90 con 0x5651acf0b650 2024-09-10T08:51:14.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.844+0000 7f228d60d640 1 --2- 172.21.15.172:0/3670726349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acefbf40 0x5651acf0a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:14.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.844+0000 7f228de0e640 1 --2- 172.21.15.172:0/3670726349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acf0b650 0x5651acf23500 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f22800096e0 tx=0x7f2280010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc88d875c65f1ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:14.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.844+0000 7f2272ffd640 1 -- 172.21.15.172:0/3670726349 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280024040 con 0x5651acf0b650 2024-09-10T08:51:14.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.844+0000 7f2272ffd640 1 -- 172.21.15.172:0/3670726349 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2280012810 con 0x5651acf0b650 2024-09-10T08:51:14.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.844+0000 7f2272ffd640 1 -- 172.21.15.172:0/3670726349 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280004e90 con 0x5651acf0b650 2024-09-10T08:51:14.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.845+0000 7f2291c4a740 1 -- 172.21.15.172:0/3670726349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acf0b650 msgr2=0x5651acf23500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:14.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.845+0000 7f2291c4a740 1 --2- 172.21.15.172:0/3670726349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acf0b650 0x5651acf23500 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f22800096e0 tx=0x7f2280010f70 comp rx=0 tx=0).stop 2024-09-10T08:51:14.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.845+0000 7f2291c4a740 1 -- 172.21.15.172:0/3670726349 shutdown_connections 2024-09-10T08:51:14.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.845+0000 7f2291c4a740 1 --2- 172.21.15.172:0/3670726349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acf0b650 0x5651acf23500 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.845+0000 7f2291c4a740 1 --2- 172.21.15.172:0/3670726349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acefc860 0x5651acf22fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.845+0000 7f2291c4a740 1 --2- 172.21.15.172:0/3670726349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acefbf40 0x5651acf0a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.845+0000 7f2291c4a740 1 -- 172.21.15.172:0/3670726349 >> 172.21.15.172:0/3670726349 conn(0x5651acef85f0 msgr2=0x5651acf09e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:14.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.845+0000 7f2291c4a740 1 -- 172.21.15.172:0/3670726349 shutdown_connections 2024-09-10T08:51:14.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.845+0000 7f2291c4a740 1 -- 172.21.15.172:0/3670726349 wait complete. 2024-09-10T08:51:14.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.845+0000 7f2291c4a740 1 -- start start 2024-09-10T08:51:14.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.845+0000 7f2291c4a740 1 Processor -- start 2024-09-10T08:51:14.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f2291c4a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acefbf40 0x5651acf13ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f2291c4a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefc860 0x5651acf0fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f2291c4a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acf0b650 0x5651acf103d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f2291c4a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f22881add80 con 0x5651acefc860 2024-09-10T08:51:14.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f2291c4a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f22881ad6b0 con 0x5651acefbf40 2024-09-10T08:51:14.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f2291c4a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f22881aa8c0 con 0x5651acf0b650 2024-09-10T08:51:14.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f2291c4a740 10 client.0 _ll_register_callbacks cb 0x7f2278097790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:14.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f228ce0c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefc860 0x5651acf0fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f228ce0c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefc860 0x5651acf0fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52108/0 (socket says 172.21.15.172:52108) 2024-09-10T08:51:14.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f228ce0c640 1 -- 172.21.15.172:0/2918703052 learned_addr learned my addr 172.21.15.172:0/2918703052 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:14.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f228d60d640 1 --2- 172.21.15.172:0/2918703052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acefbf40 0x5651acf13ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f228de0e640 1 --2- 172.21.15.172:0/2918703052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acf0b650 0x5651acf103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f228ce0c640 1 -- 172.21.15.172:0/2918703052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acf0b650 msgr2=0x5651acf103d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:14.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f228ce0c640 1 --2- 172.21.15.172:0/2918703052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acf0b650 0x5651acf103d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f228ce0c640 1 -- 172.21.15.172:0/2918703052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acefbf40 msgr2=0x5651acf13ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:14.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f228ce0c640 1 --2- 172.21.15.172:0/2918703052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acefbf40 0x5651acf13ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f228ce0c640 1 -- 172.21.15.172:0/2918703052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651acf10910 con 0x5651acefc860 2024-09-10T08:51:14.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.846+0000 7f228de0e640 1 --2- 172.21.15.172:0/2918703052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acf0b650 0x5651acf103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:14.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.847+0000 7f228ce0c640 1 --2- 172.21.15.172:0/2918703052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefc860 0x5651acf0fe90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f22780cc730 tx=0x7f227809b160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67995b0d6c25510 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:14.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.847+0000 7f2270ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:14.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.847+0000 7f2270ff9640 1 -- 172.21.15.172:0/2918703052 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278020070 con 0x5651acefc860 2024-09-10T08:51:14.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.847+0000 7f2270ff9640 1 -- 172.21.15.172:0/2918703052 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f22780b4220 con 0x5651acefc860 2024-09-10T08:51:14.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.847+0000 7f2291c4a740 1 -- 172.21.15.172:0/2918703052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2280012810 con 0x5651acefc860 2024-09-10T08:51:14.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.847+0000 7f2270ff9640 1 -- 172.21.15.172:0/2918703052 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2278010670 con 0x5651acefc860 2024-09-10T08:51:14.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.847+0000 7f2291c4a740 10 client.16980 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:14.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.847+0000 7f2291c4a740 1 -- 172.21.15.172:0/2918703052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5651acf86c80 con 0x5651acefc860 2024-09-10T08:51:14.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.847+0000 7f2270ff9640 1 -- 172.21.15.172:0/2918703052 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2280012810 con 0x5651acefc860 2024-09-10T08:51:14.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.847+0000 7f2270ff9640 1 -- 172.21.15.172:0/2918703052 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f227809a3b0 con 0x5651acefc860 2024-09-10T08:51:14.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.847+0000 7f2291c4a740 10 client.16980 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:14.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.847+0000 7f2291c4a740 10 client.16980 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:51:14.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.847+0000 7f2291c4a740 20 client.16980 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:14.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.847+0000 7f2291c4a740 1 --2- 172.21.15.172:0/2918703052 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5651acf873f0 0x5651acf89810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:14.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.848+0000 7f2291c4a740 4 client.16980 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8cccb995", "release": true, "await": true}] 2024-09-10T08:51:14.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.848+0000 7f2291c4a740 1 -- 172.21.15.172:0/2918703052 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8cccb995", "release": true, "await": true}) -- 0x5651acde55a0 con 0x5651acf873f0 2024-09-10T08:51:14.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.848+0000 7f228d60d640 1 --2- 172.21.15.172:0/2918703052 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5651acf873f0 0x5651acf89810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:14.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.848+0000 7f228d60d640 1 --2- 172.21.15.172:0/2918703052 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5651acf873f0 0x5651acf89810 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8be4fb5b1cd29458 server_cookie=688413084bec4bc3 in_seq=0 out_seq=0 2024-09-10T08:51:14.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.848+0000 7f2270ff9640 10 client.16980 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:14.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.870+0000 7f2270ff9640 1 -- 172.21.15.172:0/2918703052 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5651acde55a0 con 0x5651acf873f0 2024-09-10T08:51:14.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.870+0000 7f2270ff9640 10 client.16980 handle_command_reply: tid=0 2024-09-10T08:51:14.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.870+0000 7f2291c4a740 1 client.16980 shutdown 2024-09-10T08:51:14.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.870+0000 7f2291c4a740 1 -- 172.21.15.172:0/2918703052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefc860 msgr2=0x5651acf0fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:14.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.870+0000 7f2291c4a740 1 --2- 172.21.15.172:0/2918703052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefc860 0x5651acf0fe90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f22780cc730 tx=0x7f227809b160 comp rx=0 tx=0).stop 2024-09-10T08:51:14.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.870+0000 7f2291c4a740 1 -- 172.21.15.172:0/2918703052 shutdown_connections 2024-09-10T08:51:14.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.870+0000 7f2291c4a740 1 --2- 172.21.15.172:0/2918703052 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5651acf873f0 0x5651acf89810 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.870+0000 7f2291c4a740 1 --2- 172.21.15.172:0/2918703052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651acf0b650 0x5651acf103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.870+0000 7f2291c4a740 1 --2- 172.21.15.172:0/2918703052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5651acefc860 0x5651acf0fe90 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.870+0000 7f2291c4a740 1 --2- 172.21.15.172:0/2918703052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651acefbf40 0x5651acf13ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.870+0000 7f2291c4a740 1 -- 172.21.15.172:0/2918703052 >> 172.21.15.172:0/2918703052 conn(0x5651acef85f0 msgr2=0x5651acf0a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:14.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.870+0000 7f2270ff9640 0 client.16980 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:14.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.870+0000 7f2291c4a740 1 -- 172.21.15.172:0/2918703052 shutdown_connections 2024-09-10T08:51:14.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.870+0000 7f2291c4a740 1 -- 172.21.15.172:0/2918703052 wait complete. 2024-09-10T08:51:14.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.870+0000 7f2291c4a740 20 client.16980 trim_cache size 0 max 16384 2024-09-10T08:51:14.873 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:51:14.873 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 33, 2024-09-10T08:51:14.873 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:51:14.874 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 16, 2024-09-10T08:51:14.874 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:51:14.874 INFO:teuthology.orchestra.run.smithi172.stdout: "8cccb995": { 2024-09-10T08:51:14.874 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 16, 2024-09-10T08:51:14.874 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:51:14.874 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:51:14.874 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:51:14.874 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:51:14.874 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:51:14.874 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:51:14.875 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 142.8, 2024-09-10T08:51:14.875 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:51:14.875 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:51:14.875 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:51:14.875 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:51:14.875 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:51:14.875 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:51:14.875 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:51:14.875 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:51:14.875 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:51:14.875 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:51:14.876 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:51:14.876 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:51:14.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.871+0000 7f228f871640 1 -- 172.21.15.172:0/828806103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22740a2b80 msgr2=0x7f22740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:14.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.871+0000 7f228f871640 1 --2- 172.21.15.172:0/828806103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22740a2b80 0x7f22740a5040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2280010c40 tx=0x7f228000c000 comp rx=0 tx=0).stop 2024-09-10T08:51:14.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.871+0000 7f228f871640 1 -- 172.21.15.172:0/828806103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22881027a0 msgr2=0x7f2288198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:14.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.871+0000 7f228f871640 1 --2- 172.21.15.172:0/828806103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22881027a0 0x7f2288198950 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f227800d950 tx=0x7f227800de10 comp rx=0 tx=0).stop 2024-09-10T08:51:14.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.871+0000 7f228f871640 1 -- 172.21.15.172:0/828806103 shutdown_connections 2024-09-10T08:51:14.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.871+0000 7f228f871640 1 --2- 172.21.15.172:0/828806103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22740a2b80 0x7f22740a5040 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.871+0000 7f228f871640 1 --2- 172.21.15.172:0/828806103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2288106f20 0x7f228819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.871+0000 7f228f871640 1 --2- 172.21.15.172:0/828806103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22881027a0 0x7f2288198950 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.871+0000 7f228f871640 1 --2- 172.21.15.172:0/828806103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2288101df0 0x7f2288198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:14.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.871+0000 7f228f871640 1 -- 172.21.15.172:0/828806103 >> 172.21.15.172:0/828806103 conn(0x7f22880fd8f0 msgr2=0x7f22880fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:14.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.871+0000 7f228f871640 1 -- 172.21.15.172:0/828806103 shutdown_connections 2024-09-10T08:51:14.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:14.871+0000 7f228f871640 1 -- 172.21.15.172:0/828806103 wait complete. 2024-09-10T08:51:15.645 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8cccb995', total seconds elapsed: 27.0 2024-09-10T08:51:16.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:16 smithi172 ceph-mon[30854]: pgmap v501: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.5 KiB/s wr, 33 op/s 2024-09-10T08:51:16.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:16 smithi174 ceph-mon[34068]: pgmap v501: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.5 KiB/s wr, 33 op/s 2024-09-10T08:51:16.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:16 smithi191 ceph-mon[35314]: pgmap v501: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.5 KiB/s wr, 33 op/s 2024-09-10T08:51:17.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:17 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:51:17.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:17 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:51:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:17 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:51:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:18 smithi172 ceph-mon[30854]: pgmap v502: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.3 MiB/s wr, 35 op/s 2024-09-10T08:51:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:51:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:51:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:51:18.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:18 smithi174 ceph-mon[34068]: pgmap v502: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.3 MiB/s wr, 35 op/s 2024-09-10T08:51:18.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:51:18.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:51:18.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:51:18.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:18 smithi191 ceph-mon[35314]: pgmap v502: 129 pgs: 129 active+clean; 3.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 2.3 MiB/s wr, 35 op/s 2024-09-10T08:51:18.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:51:18.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:51:18.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:51:20.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:20 smithi172 ceph-mon[30854]: pgmap v503: 129 pgs: 129 active+clean; 3.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.0 MiB/s wr, 35 op/s 2024-09-10T08:51:20.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:20 smithi174 ceph-mon[34068]: pgmap v503: 129 pgs: 129 active+clean; 3.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.0 MiB/s wr, 35 op/s 2024-09-10T08:51:20.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:20 smithi191 ceph-mon[35314]: pgmap v503: 129 pgs: 129 active+clean; 3.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.0 MiB/s wr, 35 op/s 2024-09-10T08:51:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.062+0000 7f5c828f1640 1 -- 172.21.15.172:0/2815996108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c7c1025f0 msgr2=0x7f5c7c108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.062+0000 7f5c828f1640 1 --2- 172.21.15.172:0/2815996108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c7c1025f0 0x7f5c7c108e80 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009ca0 tx=0x7f5c7402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.062+0000 7f5c828f1640 1 -- 172.21.15.172:0/2815996108 shutdown_connections 2024-09-10T08:51:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.062+0000 7f5c828f1640 1 --2- 172.21.15.172:0/2815996108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c7c1025f0 0x7f5c7c108e80 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.062+0000 7f5c828f1640 1 --2- 172.21.15.172:0/2815996108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c7c101c30 0x7f5c7c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.062+0000 7f5c828f1640 1 --2- 172.21.15.172:0/2815996108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c7c100a40 0x7f5c7c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.062+0000 7f5c828f1640 1 -- 172.21.15.172:0/2815996108 >> 172.21.15.172:0/2815996108 conn(0x7f5c7c0fc1e0 msgr2=0x7f5c7c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.062+0000 7f5c828f1640 1 -- 172.21.15.172:0/2815996108 shutdown_connections 2024-09-10T08:51:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.062+0000 7f5c828f1640 1 -- 172.21.15.172:0/2815996108 wait complete. 2024-09-10T08:51:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.062+0000 7f5c828f1640 1 Processor -- start 2024-09-10T08:51:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.062+0000 7f5c828f1640 1 -- start start 2024-09-10T08:51:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c828f1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c7c100a40 0x7f5c7c1049d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c828f1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c7c101c30 0x7f5c7c103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c828f1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c7c1025f0 0x7f5c7c103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c828f1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c10c550 con 0x7f5c7c100a40 2024-09-10T08:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c828f1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c10c3d0 con 0x7f5c7c1025f0 2024-09-10T08:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c828f1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c10c6d0 con 0x7f5c7c101c30 2024-09-10T08:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c7bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c7c100a40 0x7f5c7c1049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c7bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c7c100a40 0x7f5c7c1049d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58232/0 (socket says 172.21.15.172:58232) 2024-09-10T08:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c7bfff640 1 -- 172.21.15.172:0/2616673656 learned_addr learned my addr 172.21.15.172:0/2616673656 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c7b7fe640 1 --2- 172.21.15.172:0/2616673656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c7c101c30 0x7f5c7c103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c7bfff640 1 -- 172.21.15.172:0/2616673656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c7c101c30 msgr2=0x7f5c7c103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c80e8e640 1 --2- 172.21.15.172:0/2616673656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c7c1025f0 0x7f5c7c103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c7bfff640 1 --2- 172.21.15.172:0/2616673656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c7c101c30 0x7f5c7c103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c7bfff640 1 -- 172.21.15.172:0/2616673656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c7c1025f0 msgr2=0x7f5c7c103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c7bfff640 1 --2- 172.21.15.172:0/2616673656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c7c1025f0 0x7f5c7c103550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c7bfff640 1 -- 172.21.15.172:0/2616673656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7c103de0 con 0x7f5c7c100a40 2024-09-10T08:51:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c7bfff640 1 --2- 172.21.15.172:0/2616673656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c7c100a40 0x7f5c7c1049d0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00e9c0 tx=0x7f5c6c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.063+0000 7f5c797fa640 1 -- 172.21.15.172:0/2616673656 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c00cd80 con 0x7f5c7c100a40 2024-09-10T08:51:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.064+0000 7f5c797fa640 1 -- 172.21.15.172:0/2616673656 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c6c0035f0 con 0x7f5c7c100a40 2024-09-10T08:51:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.064+0000 7f5c828f1640 1 -- 172.21.15.172:0/2616673656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7c1a1d90 con 0x7f5c7c100a40 2024-09-10T08:51:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.064+0000 7f5c797fa640 1 -- 172.21.15.172:0/2616673656 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c010690 con 0x7f5c7c100a40 2024-09-10T08:51:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.064+0000 7f5c828f1640 1 -- 172.21.15.172:0/2616673656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7c1a21d0 con 0x7f5c7c100a40 2024-09-10T08:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c797fa640 1 -- 172.21.15.172:0/2616673656 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5c6c0037a0 con 0x7f5c7c100a40 2024-09-10T08:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c85691740 1 Processor -- start 2024-09-10T08:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c85691740 1 -- start start 2024-09-10T08:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c85691740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bbfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c85691740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc02e0 0x558fa7bc4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c85691740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bc5370 0x558fa7bc7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c85691740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558fa7b232f0 con 0x558fa7bbf9a0 2024-09-10T08:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c85691740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558fa7acb270 con 0x558fa7bc5370 2024-09-10T08:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c85691740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558fa7b1f2b0 con 0x558fa7bc02e0 2024-09-10T08:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c80e8e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bbfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c80e8e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bbfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58240/0 (socket says 172.21.15.172:58240) 2024-09-10T08:51:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c80e8e640 1 -- 172.21.15.172:0/2938634949 learned_addr learned my addr 172.21.15.172:0/2938634949 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c7bfff640 1 --2- 172.21.15.172:0/2938634949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc02e0 0x558fa7bc4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c7b7fe640 1 --2- 172.21.15.172:0/2938634949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bc5370 0x558fa7bc7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c80e8e640 1 -- 172.21.15.172:0/2938634949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc02e0 msgr2=0x558fa7bc4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c80e8e640 1 --2- 172.21.15.172:0/2938634949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc02e0 0x558fa7bc4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c80e8e640 1 -- 172.21.15.172:0/2938634949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bc5370 msgr2=0x558fa7bc7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c80e8e640 1 --2- 172.21.15.172:0/2938634949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bc5370 0x558fa7bc7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c797fa640 1 --2- 172.21.15.172:0/2616673656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5c640a2c30 0x7f5c640a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.067+0000 7f5c80e8e640 1 -- 172.21.15.172:0/2938634949 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fa7aca340 con 0x558fa7bbf9a0 2024-09-10T08:51:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.068+0000 7f5c797fa640 1 -- 172.21.15.172:0/2616673656 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5c6c0cf8f0 con 0x7f5c7c100a40 2024-09-10T08:51:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.068+0000 7f5c80e8e640 1 --2- 172.21.15.172:0/2616673656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5c640a2c30 0x7f5c640a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.068+0000 7f5c80e8e640 1 --2- 172.21.15.172:0/2938634949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bbfda0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5c7402ade0 tx=0x7f5c74009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5f430db2db53de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.068+0000 7f5c627fc640 1 -- 172.21.15.172:0/2938634949 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7404c070 con 0x558fa7bbf9a0 2024-09-10T08:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.068+0000 7f5c627fc640 1 -- 172.21.15.172:0/2938634949 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c74036070 con 0x558fa7bbf9a0 2024-09-10T08:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.068+0000 7f5c627fc640 1 -- 172.21.15.172:0/2938634949 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74030bd0 con 0x558fa7bbf9a0 2024-09-10T08:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.069+0000 7f5c85691740 1 -- 172.21.15.172:0/2938634949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 msgr2=0x558fa7bbfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.069+0000 7f5c85691740 1 --2- 172.21.15.172:0/2938634949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bbfda0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5c7402ade0 tx=0x7f5c74009360 comp rx=0 tx=0).stop 2024-09-10T08:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.069+0000 7f5c80e8e640 1 --2- 172.21.15.172:0/2616673656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5c640a2c30 0x7f5c640a50f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5c74049000 tx=0x7f5c7404a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.069+0000 7f5c85691740 1 -- 172.21.15.172:0/2938634949 shutdown_connections 2024-09-10T08:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.069+0000 7f5c85691740 1 --2- 172.21.15.172:0/2938634949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bc5370 0x558fa7bc7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.069+0000 7f5c85691740 1 --2- 172.21.15.172:0/2938634949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc02e0 0x558fa7bc4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.069+0000 7f5c85691740 1 --2- 172.21.15.172:0/2938634949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bbfda0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.069+0000 7f5c85691740 1 -- 172.21.15.172:0/2938634949 >> 172.21.15.172:0/2938634949 conn(0x558fa7bbb010 msgr2=0x558fa7bbb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.069+0000 7f5c85691740 1 -- 172.21.15.172:0/2938634949 shutdown_connections 2024-09-10T08:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.069+0000 7f5c85691740 1 -- 172.21.15.172:0/2938634949 wait complete. 2024-09-10T08:51:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.069+0000 7f5c85691740 1 -- start start 2024-09-10T08:51:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c85691740 1 Processor -- start 2024-09-10T08:51:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c85691740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bbf9a0 0x558fa7bc9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c85691740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bc02e0 0x558fa7bca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c85691740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc5370 0x558fa7bd1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c85691740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558fa7bc8340 con 0x558fa7bc02e0 2024-09-10T08:51:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c85691740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558fa7aac290 con 0x558fa7bbf9a0 2024-09-10T08:51:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c85691740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558fa7bc84c0 con 0x558fa7bc5370 2024-09-10T08:51:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c85691740 10 client.0 _ll_register_callbacks cb 0x558fa7add140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c7b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bc02e0 0x558fa7bca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c7b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bc02e0 0x558fa7bca110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58246/0 (socket says 172.21.15.172:58246) 2024-09-10T08:51:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c7b7fe640 1 -- 172.21.15.172:0/4024933097 learned_addr learned my addr 172.21.15.172:0/4024933097 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c7bfff640 1 --2- 172.21.15.172:0/4024933097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bbf9a0 0x558fa7bc9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c80e8e640 1 --2- 172.21.15.172:0/4024933097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc5370 0x558fa7bd1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c7b7fe640 1 -- 172.21.15.172:0/4024933097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc5370 msgr2=0x558fa7bd1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c7b7fe640 1 --2- 172.21.15.172:0/4024933097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc5370 0x558fa7bd1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c7b7fe640 1 -- 172.21.15.172:0/4024933097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bbf9a0 msgr2=0x558fa7bc9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c7b7fe640 1 --2- 172.21.15.172:0/4024933097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bbf9a0 0x558fa7bc9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.070+0000 7f5c7b7fe640 1 -- 172.21.15.172:0/4024933097 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fa7836210 con 0x558fa7bc02e0 2024-09-10T08:51:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.071+0000 7f5c7b7fe640 1 --2- 172.21.15.172:0/4024933097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bc02e0 0x558fa7bca110 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000bea0 tx=0x7f5c7000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0f920faa40812a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.071+0000 7f5c5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.071+0000 7f5c5b7fe640 1 -- 172.21.15.172:0/4024933097 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70004230 con 0x558fa7bc02e0 2024-09-10T08:51:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.071+0000 7f5c85691740 1 -- 172.21.15.172:0/4024933097 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c74036070 con 0x558fa7bc02e0 2024-09-10T08:51:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.071+0000 7f5c5b7fe640 1 -- 172.21.15.172:0/4024933097 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5c700043d0 con 0x558fa7bc02e0 2024-09-10T08:51:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.071+0000 7f5c5b7fe640 1 -- 172.21.15.172:0/4024933097 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5c70024050 con 0x558fa7bc02e0 2024-09-10T08:51:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.071+0000 7f5c85691740 10 client.16992 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.071+0000 7f5c85691740 1 -- 172.21.15.172:0/4024933097 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558fa7902620 con 0x558fa7bc02e0 2024-09-10T08:51:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.071+0000 7f5c5b7fe640 1 -- 172.21.15.172:0/4024933097 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c74036070 con 0x558fa7bc02e0 2024-09-10T08:51:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.071+0000 7f5c5b7fe640 1 -- 172.21.15.172:0/4024933097 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f5c7001b820 con 0x558fa7bc02e0 2024-09-10T08:51:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.071+0000 7f5c85691740 10 client.16992 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.071+0000 7f5c85691740 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:51:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.071+0000 7f5c85691740 20 client.16992 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.071+0000 7f5c85691740 1 --2- 172.21.15.172:0/4024933097 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558fa7bd58a0 0x558fa7bd7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.071+0000 7f5c85691740 4 client.16992 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:51:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.071+0000 7f5c85691740 1 -- 172.21.15.172:0/4024933097 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fa7b30a20 con 0x558fa7bd58a0 2024-09-10T08:51:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.072+0000 7f5c7bfff640 1 --2- 172.21.15.172:0/4024933097 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558fa7bd58a0 0x558fa7bd7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.072+0000 7f5c5b7fe640 10 client.16992 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.072+0000 7f5c7bfff640 1 --2- 172.21.15.172:0/4024933097 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558fa7bd58a0 0x558fa7bd7d10 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c214fb60d6b662f6 server_cookie=f039ba1d7af0ad93 in_seq=0 out_seq=0 2024-09-10T08:51:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.073+0000 7f5c5b7fe640 1 -- 172.21.15.172:0/4024933097 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558fa7b30a20 con 0x558fa7bd58a0 2024-09-10T08:51:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.073+0000 7f5c5b7fe640 10 client.16992 handle_command_reply: tid=0 2024-09-10T08:51:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.073+0000 7f5c85691740 1 client.16992 shutdown 2024-09-10T08:51:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.073+0000 7f5c85691740 1 -- 172.21.15.172:0/4024933097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bc02e0 msgr2=0x558fa7bca110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.073+0000 7f5c85691740 1 --2- 172.21.15.172:0/4024933097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bc02e0 0x558fa7bca110 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000bea0 tx=0x7f5c7000bf90 comp rx=0 tx=0).stop 2024-09-10T08:51:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.073+0000 7f5c85691740 1 -- 172.21.15.172:0/4024933097 shutdown_connections 2024-09-10T08:51:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.073+0000 7f5c85691740 1 --2- 172.21.15.172:0/4024933097 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558fa7bd58a0 0x558fa7bd7d10 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.073+0000 7f5c85691740 1 --2- 172.21.15.172:0/4024933097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc5370 0x558fa7bd1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.073+0000 7f5c5b7fe640 0 client.16992 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.073+0000 7f5c85691740 1 --2- 172.21.15.172:0/4024933097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bc02e0 0x558fa7bca110 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.073+0000 7f5c85691740 1 --2- 172.21.15.172:0/4024933097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bbf9a0 0x558fa7bc9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.073+0000 7f5c85691740 1 -- 172.21.15.172:0/4024933097 >> 172.21.15.172:0/4024933097 conn(0x558fa7bbb010 msgr2=0x558fa7bbc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.073+0000 7f5c85691740 1 -- 172.21.15.172:0/4024933097 shutdown_connections 2024-09-10T08:51:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.073+0000 7f5c85691740 1 -- 172.21.15.172:0/4024933097 wait complete. 2024-09-10T08:51:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.073+0000 7f5c85691740 20 client.16992 trim_cache size 0 max 16384 2024-09-10T08:51:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c85691740 1 Processor -- start 2024-09-10T08:51:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c85691740 1 -- start start 2024-09-10T08:51:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c85691740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bc4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c85691740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bc02e0 0x558fa7bd1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c85691740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc5370 0x558fa7bba0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c85691740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c1ac670 con 0x558fa7bbf9a0 2024-09-10T08:51:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c85691740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c1ae270 con 0x558fa7bc02e0 2024-09-10T08:51:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c85691740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c1acd40 con 0x558fa7bc5370 2024-09-10T08:51:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c7bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bc4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c7bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bc4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58262/0 (socket says 172.21.15.172:58262) 2024-09-10T08:51:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c7bfff640 1 -- 172.21.15.172:0/2755008224 learned_addr learned my addr 172.21.15.172:0/2755008224 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c80e8e640 1 --2- 172.21.15.172:0/2755008224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc5370 0x558fa7bba0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c7bfff640 1 -- 172.21.15.172:0/2755008224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc5370 msgr2=0x558fa7bba0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c7b7fe640 1 --2- 172.21.15.172:0/2755008224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bc02e0 0x558fa7bd1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c7bfff640 1 --2- 172.21.15.172:0/2755008224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc5370 0x558fa7bba0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c7bfff640 1 -- 172.21.15.172:0/2755008224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bc02e0 msgr2=0x558fa7bd1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c7bfff640 1 --2- 172.21.15.172:0/2755008224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bc02e0 0x558fa7bd1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c7bfff640 1 -- 172.21.15.172:0/2755008224 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fa7bc9710 con 0x558fa7bbf9a0 2024-09-10T08:51:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.092+0000 7f5c7b7fe640 1 --2- 172.21.15.172:0/2755008224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bc02e0 0x558fa7bd1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.093+0000 7f5c7bfff640 1 --2- 172.21.15.172:0/2755008224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bc4ae0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c6a0 tx=0x7f5c6c003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2758d20053235fa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.093+0000 7f5c61ffb640 1 -- 172.21.15.172:0/2755008224 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c09b930 con 0x558fa7bbf9a0 2024-09-10T08:51:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.093+0000 7f5c61ffb640 1 -- 172.21.15.172:0/2755008224 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c6c003b30 con 0x558fa7bbf9a0 2024-09-10T08:51:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.093+0000 7f5c61ffb640 1 -- 172.21.15.172:0/2755008224 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c09b750 con 0x558fa7bbf9a0 2024-09-10T08:51:21.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.093+0000 7f5c85691740 1 -- 172.21.15.172:0/2755008224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 msgr2=0x558fa7bc4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.093+0000 7f5c85691740 1 --2- 172.21.15.172:0/2755008224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bc4ae0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c6a0 tx=0x7f5c6c003e30 comp rx=0 tx=0).stop 2024-09-10T08:51:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.093+0000 7f5c85691740 1 -- 172.21.15.172:0/2755008224 shutdown_connections 2024-09-10T08:51:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.093+0000 7f5c85691740 1 --2- 172.21.15.172:0/2755008224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc5370 0x558fa7bba0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.093+0000 7f5c85691740 1 --2- 172.21.15.172:0/2755008224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bc02e0 0x558fa7bd1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.093+0000 7f5c85691740 1 --2- 172.21.15.172:0/2755008224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bc4ae0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.093+0000 7f5c85691740 1 -- 172.21.15.172:0/2755008224 >> 172.21.15.172:0/2755008224 conn(0x558fa7bbb010 msgr2=0x558fa7bb99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.094+0000 7f5c85691740 1 -- 172.21.15.172:0/2755008224 shutdown_connections 2024-09-10T08:51:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.094+0000 7f5c85691740 1 -- 172.21.15.172:0/2755008224 wait complete. 2024-09-10T08:51:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.094+0000 7f5c85691740 1 -- start start 2024-09-10T08:51:21.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.094+0000 7f5c85691740 1 Processor -- start 2024-09-10T08:51:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.094+0000 7f5c85691740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bd0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.094+0000 7f5c85691740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc02e0 0x558fa7bd0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.094+0000 7f5c85691740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bc5370 0x558fa7bcf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.094+0000 7f5c85691740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c1adb10 con 0x558fa7bbf9a0 2024-09-10T08:51:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.094+0000 7f5c85691740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c1ad440 con 0x558fa7bc5370 2024-09-10T08:51:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.094+0000 7f5c85691740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c1aa650 con 0x558fa7bc02e0 2024-09-10T08:51:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.094+0000 7f5c85691740 10 client.0 _ll_register_callbacks cb 0x7f5c7000b3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.094+0000 7f5c7bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bd0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c7bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bd0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58266/0 (socket says 172.21.15.172:58266) 2024-09-10T08:51:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c7bfff640 1 -- 172.21.15.172:0/3343155564 learned_addr learned my addr 172.21.15.172:0/3343155564 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c80e8e640 1 --2- 172.21.15.172:0/3343155564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bc5370 0x558fa7bcf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c7b7fe640 1 --2- 172.21.15.172:0/3343155564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc02e0 0x558fa7bd0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c7bfff640 1 -- 172.21.15.172:0/3343155564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc02e0 msgr2=0x558fa7bd0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c7bfff640 1 --2- 172.21.15.172:0/3343155564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc02e0 0x558fa7bd0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c7bfff640 1 -- 172.21.15.172:0/3343155564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bc5370 msgr2=0x558fa7bcf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c7bfff640 1 --2- 172.21.15.172:0/3343155564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bc5370 0x558fa7bcf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c7bfff640 1 -- 172.21.15.172:0/3343155564 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fa7bbbca0 con 0x558fa7bbf9a0 2024-09-10T08:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c80e8e640 1 --2- 172.21.15.172:0/3343155564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bc5370 0x558fa7bcf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c7bfff640 1 --2- 172.21.15.172:0/3343155564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bd0a00 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c018000 tx=0x7f5c6c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30dc74216f276b9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c5b7fe640 1 -- 172.21.15.172:0/3343155564 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c09b930 con 0x558fa7bbf9a0 2024-09-10T08:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c5b7fe640 1 -- 172.21.15.172:0/3343155564 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5c6c098460 con 0x558fa7bbf9a0 2024-09-10T08:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c85691740 1 -- 172.21.15.172:0/3343155564 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558fa7b2ad70 con 0x558fa7bbf9a0 2024-09-10T08:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c5b7fe640 1 -- 172.21.15.172:0/3343155564 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5c6c09bdf0 con 0x558fa7bbf9a0 2024-09-10T08:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c85691740 10 client.16998 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.095+0000 7f5c85691740 1 -- 172.21.15.172:0/3343155564 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558fa7c40df0 con 0x558fa7bbf9a0 2024-09-10T08:51:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.096+0000 7f5c5b7fe640 1 -- 172.21.15.172:0/3343155564 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c6c0b9020 con 0x558fa7bbf9a0 2024-09-10T08:51:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.096+0000 7f5c5b7fe640 1 -- 172.21.15.172:0/3343155564 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f5c6c097bd0 con 0x558fa7bbf9a0 2024-09-10T08:51:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.096+0000 7f5c85691740 10 client.16998 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.096+0000 7f5c85691740 10 client.16998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:51:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.096+0000 7f5c85691740 20 client.16998 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.096+0000 7f5c85691740 1 --2- 172.21.15.172:0/3343155564 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558fa7c41880 0x558fa7c43ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.096+0000 7f5c85691740 4 client.16998 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:51:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.096+0000 7f5c85691740 1 -- 172.21.15.172:0/3343155564 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558fa7aa7900 con 0x558fa7c41880 2024-09-10T08:51:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.096+0000 7f5c7b7fe640 1 --2- 172.21.15.172:0/3343155564 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558fa7c41880 0x558fa7c43ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.096+0000 7f5c5b7fe640 10 client.16998 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.096+0000 7f5c7b7fe640 1 --2- 172.21.15.172:0/3343155564 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558fa7c41880 0x558fa7c43ca0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5732736805a807f server_cookie=8e874e2ef0210454 in_seq=0 out_seq=0 2024-09-10T08:51:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.100+0000 7f5c5b7fe640 1 -- 172.21.15.172:0/3343155564 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558fa7aa7900 con 0x558fa7c41880 2024-09-10T08:51:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.100+0000 7f5c5b7fe640 10 client.16998 handle_command_reply: tid=0 2024-09-10T08:51:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.100+0000 7f5c85691740 1 client.16998 shutdown 2024-09-10T08:51:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.100+0000 7f5c85691740 1 -- 172.21.15.172:0/3343155564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 msgr2=0x558fa7bd0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.100+0000 7f5c85691740 1 --2- 172.21.15.172:0/3343155564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bd0a00 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c018000 tx=0x7f5c6c00c7d0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.100+0000 7f5c85691740 1 -- 172.21.15.172:0/3343155564 shutdown_connections 2024-09-10T08:51:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.100+0000 7f5c85691740 1 --2- 172.21.15.172:0/3343155564 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558fa7c41880 0x558fa7c43ca0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.100+0000 7f5c85691740 1 --2- 172.21.15.172:0/3343155564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa7bc5370 0x558fa7bcf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.100+0000 7f5c85691740 1 --2- 172.21.15.172:0/3343155564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa7bc02e0 0x558fa7bd0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.100+0000 7f5c5b7fe640 0 client.16998 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.100+0000 7f5c85691740 1 --2- 172.21.15.172:0/3343155564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa7bbf9a0 0x558fa7bd0a00 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.100+0000 7f5c85691740 1 -- 172.21.15.172:0/3343155564 >> 172.21.15.172:0/3343155564 conn(0x558fa7bbb010 msgr2=0x558fa7bb99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.101+0000 7f5c85691740 1 -- 172.21.15.172:0/3343155564 shutdown_connections 2024-09-10T08:51:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.101+0000 7f5c85691740 1 -- 172.21.15.172:0/3343155564 wait complete. 2024-09-10T08:51:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.101+0000 7f5c85691740 20 client.16998 trim_cache size 0 max 16384 2024-09-10T08:51:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:51:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:51:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:51:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:51:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.101+0000 7f5c828f1640 1 -- 172.21.15.172:0/2616673656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5c640a2c30 msgr2=0x7f5c640a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.101+0000 7f5c828f1640 1 --2- 172.21.15.172:0/2616673656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5c640a2c30 0x7f5c640a50f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5c74049000 tx=0x7f5c7404a000 comp rx=0 tx=0).stop 2024-09-10T08:51:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.101+0000 7f5c828f1640 1 -- 172.21.15.172:0/2616673656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c7c100a40 msgr2=0x7f5c7c1049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.101+0000 7f5c828f1640 1 --2- 172.21.15.172:0/2616673656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c7c100a40 0x7f5c7c1049d0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00e9c0 tx=0x7f5c6c00ee80 comp rx=0 tx=0).stop 2024-09-10T08:51:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.101+0000 7f5c828f1640 1 -- 172.21.15.172:0/2616673656 shutdown_connections 2024-09-10T08:51:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.101+0000 7f5c828f1640 1 --2- 172.21.15.172:0/2616673656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5c640a2c30 0x7f5c640a50f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.101+0000 7f5c828f1640 1 --2- 172.21.15.172:0/2616673656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c7c1025f0 0x7f5c7c103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.101+0000 7f5c828f1640 1 --2- 172.21.15.172:0/2616673656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c7c101c30 0x7f5c7c103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.101+0000 7f5c828f1640 1 --2- 172.21.15.172:0/2616673656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c7c100a40 0x7f5c7c1049d0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.101+0000 7f5c828f1640 1 -- 172.21.15.172:0/2616673656 >> 172.21.15.172:0/2616673656 conn(0x7f5c7c0fc1e0 msgr2=0x7f5c7c0fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.101+0000 7f5c828f1640 1 -- 172.21.15.172:0/2616673656 shutdown_connections 2024-09-10T08:51:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:21.101+0000 7f5c828f1640 1 -- 172.21.15.172:0/2616673656 wait complete. 2024-09-10T08:51:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:22 smithi172 ceph-mon[30854]: pgmap v504: 129 pgs: 129 active+clean; 3.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.8 MiB/s wr, 39 op/s 2024-09-10T08:51:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:22 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:51:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:22 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:51:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:22 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:51:22.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:22 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:51:22.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:22 smithi174 ceph-mon[34068]: pgmap v504: 129 pgs: 129 active+clean; 3.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.8 MiB/s wr, 39 op/s 2024-09-10T08:51:22.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:22 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:51:22.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:22 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:51:22.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:22 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:51:22.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:22 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:51:22.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:22 smithi191 ceph-mon[35314]: pgmap v504: 129 pgs: 129 active+clean; 3.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.8 MiB/s wr, 39 op/s 2024-09-10T08:51:22.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:22 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:51:22.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:22 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:51:22.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:22 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:51:22.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:22 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:51:24.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:24 smithi174 ceph-mon[34068]: pgmap v505: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 34 op/s 2024-09-10T08:51:24.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:24 smithi191 ceph-mon[35314]: pgmap v505: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 34 op/s 2024-09-10T08:51:24.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:24 smithi172 ceph-mon[30854]: pgmap v505: 129 pgs: 129 active+clean; 3.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 34 op/s 2024-09-10T08:51:26.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:26 smithi174 ceph-mon[34068]: pgmap v506: 129 pgs: 129 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 MiB/s wr, 35 op/s 2024-09-10T08:51:26.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:26 smithi191 ceph-mon[35314]: pgmap v506: 129 pgs: 129 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 MiB/s wr, 35 op/s 2024-09-10T08:51:26.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:26 smithi172 ceph-mon[30854]: pgmap v506: 129 pgs: 129 active+clean; 3.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 MiB/s wr, 35 op/s 2024-09-10T08:51:28.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:28 smithi174 ceph-mon[34068]: pgmap v507: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 23 op/s 2024-09-10T08:51:28.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:28 smithi191 ceph-mon[35314]: pgmap v507: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 23 op/s 2024-09-10T08:51:28.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:28 smithi172 ceph-mon[30854]: pgmap v507: 129 pgs: 129 active+clean; 3.3 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 23 op/s 2024-09-10T08:51:30.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:30 smithi174 ceph-mon[34068]: pgmap v508: 129 pgs: 129 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-09-10T08:51:30.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:30 smithi191 ceph-mon[35314]: pgmap v508: 129 pgs: 129 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-09-10T08:51:30.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:30 smithi172 ceph-mon[30854]: pgmap v508: 129 pgs: 129 active+clean; 3.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-09-10T08:51:31.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:31 smithi174 ceph-mon[34068]: pgmap v509: 129 pgs: 129 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 24 op/s 2024-09-10T08:51:31.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:31 smithi191 ceph-mon[35314]: pgmap v509: 129 pgs: 129 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 24 op/s 2024-09-10T08:51:31.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:31 smithi172 ceph-mon[30854]: pgmap v509: 129 pgs: 129 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 24 op/s 2024-09-10T08:51:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.195+0000 7f5b4deac640 1 -- 172.21.15.172:0/1403616673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b480761b0 msgr2=0x7f5b48074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.195+0000 7f5b4deac640 1 --2- 172.21.15.172:0/1403616673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b480761b0 0x7f5b48074600 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009900 tx=0x7f5b3802a480 comp rx=0 tx=0).stop 2024-09-10T08:51:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.195+0000 7f5b4deac640 1 -- 172.21.15.172:0/1403616673 shutdown_connections 2024-09-10T08:51:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.195+0000 7f5b4deac640 1 --2- 172.21.15.172:0/1403616673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b48109a10 0x7f5b4810bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.195+0000 7f5b4deac640 1 --2- 172.21.15.172:0/1403616673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b48074bd0 0x7f5b48075030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.195+0000 7f5b4deac640 1 --2- 172.21.15.172:0/1403616673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b480761b0 0x7f5b48074600 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.195+0000 7f5b4deac640 1 -- 172.21.15.172:0/1403616673 >> 172.21.15.172:0/1403616673 conn(0x7f5b480fd2e0 msgr2=0x7f5b480ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.196+0000 7f5b4deac640 1 -- 172.21.15.172:0/1403616673 shutdown_connections 2024-09-10T08:51:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.196+0000 7f5b4deac640 1 -- 172.21.15.172:0/1403616673 wait complete. 2024-09-10T08:51:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.196+0000 7f5b4deac640 1 Processor -- start 2024-09-10T08:51:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.196+0000 7f5b4deac640 1 -- start start 2024-09-10T08:51:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b4deac640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b48074bd0 0x7f5b48199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b4deac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b480761b0 0x7f5b4819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b4deac640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b48109a10 0x7f5b481a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b4deac640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4810f8f0 con 0x7f5b480761b0 2024-09-10T08:51:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b4deac640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4810f770 con 0x7f5b48074bd0 2024-09-10T08:51:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b4deac640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4810fa70 con 0x7f5b48109a10 2024-09-10T08:51:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b46ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b480761b0 0x7f5b4819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b46ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b480761b0 0x7f5b4819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41968/0 (socket says 172.21.15.172:41968) 2024-09-10T08:51:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b46ffd640 1 -- 172.21.15.172:0/4143086424 learned_addr learned my addr 172.21.15.172:0/4143086424 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b47fff640 1 --2- 172.21.15.172:0/4143086424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b48109a10 0x7f5b481a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b477fe640 1 --2- 172.21.15.172:0/4143086424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b48074bd0 0x7f5b48199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b46ffd640 1 -- 172.21.15.172:0/4143086424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b48109a10 msgr2=0x7f5b481a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b46ffd640 1 --2- 172.21.15.172:0/4143086424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b48109a10 0x7f5b481a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b46ffd640 1 -- 172.21.15.172:0/4143086424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b48074bd0 msgr2=0x7f5b48199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b46ffd640 1 --2- 172.21.15.172:0/4143086424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b48074bd0 0x7f5b48199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b46ffd640 1 -- 172.21.15.172:0/4143086424 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b481a1b60 con 0x7f5b480761b0 2024-09-10T08:51:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b46ffd640 1 --2- 172.21.15.172:0/4143086424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b480761b0 0x7f5b4819a480 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00e9f0 tx=0x7f5b3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b44ff9640 1 -- 172.21.15.172:0/4143086424 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c00cd80 con 0x7f5b480761b0 2024-09-10T08:51:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b4deac640 1 -- 172.21.15.172:0/4143086424 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b481a1e50 con 0x7f5b480761b0 2024-09-10T08:51:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b44ff9640 1 -- 172.21.15.172:0/4143086424 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b3c0108e0 con 0x7f5b480761b0 2024-09-10T08:51:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.197+0000 7f5b44ff9640 1 -- 172.21.15.172:0/4143086424 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c018920 con 0x7f5b480761b0 2024-09-10T08:51:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.198+0000 7f5b4deac640 1 -- 172.21.15.172:0/4143086424 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b481a23e0 con 0x7f5b480761b0 2024-09-10T08:51:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.200+0000 7f5b44ff9640 1 -- 172.21.15.172:0/4143086424 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5b3c010a90 con 0x7f5b480761b0 2024-09-10T08:51:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.200+0000 7f5b50c4c740 1 Processor -- start 2024-09-10T08:51:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.200+0000 7f5b50c4c740 1 -- start start 2024-09-10T08:51:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.200+0000 7f5b44ff9640 1 --2- 172.21.15.172:0/4143086424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b300a2c30 0x7f5b300a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.200+0000 7f5b50c4c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561008819a0 0x556100881da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.200+0000 7f5b50c4c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008822e0 0x556100886da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b50c4c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556100887370 0x556100889830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b50c4c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5561007e52f0 con 0x556100887370 2024-09-10T08:51:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b50c4c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55610078d270 con 0x5561008822e0 2024-09-10T08:51:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b50c4c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5561007e12b0 con 0x5561008819a0 2024-09-10T08:51:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b44ff9640 1 -- 172.21.15.172:0/4143086424 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5b3c0cf660 con 0x7f5b480761b0 2024-09-10T08:51:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b47fff640 1 --2- 172.21.15.172:0/4143086424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b300a2c30 0x7f5b300a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b477fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561008819a0 0x556100881da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b477fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561008819a0 0x556100881da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:35612/0 (socket says 172.21.15.172:35612) 2024-09-10T08:51:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b477fe640 1 -- 172.21.15.172:0/4270968084 learned_addr learned my addr 172.21.15.172:0/4270968084 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b47fff640 1 --2- 172.21.15.172:0/4270968084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556100887370 0x556100889830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b46ffd640 1 --2- 172.21.15.172:0/4270968084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008822e0 0x556100886da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b477fe640 1 -- 172.21.15.172:0/4270968084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008822e0 msgr2=0x556100886da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b477fe640 1 --2- 172.21.15.172:0/4270968084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008822e0 0x556100886da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b477fe640 1 -- 172.21.15.172:0/4270968084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556100887370 msgr2=0x556100889830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b477fe640 1 --2- 172.21.15.172:0/4270968084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556100887370 0x556100889830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b477fe640 1 -- 172.21.15.172:0/4270968084 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55610078c340 con 0x5561008819a0 2024-09-10T08:51:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b47fff640 1 --2- 172.21.15.172:0/4143086424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b300a2c30 0x7f5b300a50f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009fd0 tx=0x7f5b40009ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b477fe640 1 --2- 172.21.15.172:0/4270968084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561008819a0 0x556100881da0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009900 tx=0x7f5b38031c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb1fad1dacc1e657 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.201+0000 7f5b2dffb640 1 -- 172.21.15.172:0/4270968084 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38035030 con 0x5561008819a0 2024-09-10T08:51:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.202+0000 7f5b2dffb640 1 -- 172.21.15.172:0/4270968084 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b38040070 con 0x5561008819a0 2024-09-10T08:51:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.202+0000 7f5b2dffb640 1 -- 172.21.15.172:0/4270968084 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38003b20 con 0x5561008819a0 2024-09-10T08:51:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.202+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4270968084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561008819a0 msgr2=0x556100881da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.202+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/4270968084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561008819a0 0x556100881da0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009900 tx=0x7f5b38031c30 comp rx=0 tx=0).stop 2024-09-10T08:51:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.202+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4270968084 shutdown_connections 2024-09-10T08:51:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.202+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/4270968084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556100887370 0x556100889830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.202+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/4270968084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008822e0 0x556100886da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.202+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/4270968084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561008819a0 0x556100881da0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.202+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4270968084 >> 172.21.15.172:0/4270968084 conn(0x55610087d010 msgr2=0x55610087d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.202+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4270968084 shutdown_connections 2024-09-10T08:51:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.202+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4270968084 wait complete. 2024-09-10T08:51:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.203+0000 7f5b50c4c740 1 -- start start 2024-09-10T08:51:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.203+0000 7f5b50c4c740 1 Processor -- start 2024-09-10T08:51:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.203+0000 7f5b50c4c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008819a0 0x55610088bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.203+0000 7f5b50c4c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561008822e0 0x55610088c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.203+0000 7f5b50c4c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556100887370 0x556100893050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.203+0000 7f5b50c4c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55610088a2e0 con 0x556100887370 2024-09-10T08:51:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.203+0000 7f5b50c4c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55610076e290 con 0x5561008819a0 2024-09-10T08:51:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.203+0000 7f5b50c4c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55610088a460 con 0x5561008822e0 2024-09-10T08:51:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.203+0000 7f5b50c4c740 10 client.0 _ll_register_callbacks cb 0x55610079f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.203+0000 7f5b47fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556100887370 0x556100893050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.203+0000 7f5b47fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556100887370 0x556100893050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41976/0 (socket says 172.21.15.172:41976) 2024-09-10T08:51:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.203+0000 7f5b47fff640 1 -- 172.21.15.172:0/4093719052 learned_addr learned my addr 172.21.15.172:0/4093719052 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.203+0000 7f5b46ffd640 1 --2- 172.21.15.172:0/4093719052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561008822e0 0x55610088c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.203+0000 7f5b477fe640 1 --2- 172.21.15.172:0/4093719052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008819a0 0x55610088bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.203+0000 7f5b47fff640 1 -- 172.21.15.172:0/4093719052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561008822e0 msgr2=0x55610088c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.203+0000 7f5b47fff640 1 --2- 172.21.15.172:0/4093719052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561008822e0 0x55610088c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.204+0000 7f5b47fff640 1 -- 172.21.15.172:0/4093719052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008819a0 msgr2=0x55610088bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.204+0000 7f5b47fff640 1 --2- 172.21.15.172:0/4093719052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008819a0 0x55610088bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.204+0000 7f5b47fff640 1 -- 172.21.15.172:0/4093719052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561004f8210 con 0x556100887370 2024-09-10T08:51:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.204+0000 7f5b477fe640 1 --2- 172.21.15.172:0/4093719052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008819a0 0x55610088bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:51:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.204+0000 7f5b47fff640 1 --2- 172.21.15.172:0/4093719052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556100887370 0x556100893050 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5b4001a620 tx=0x7f5b4001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca674b2edd5828b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.204+0000 7f5b26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.204+0000 7f5b26ffd640 1 -- 172.21.15.172:0/4093719052 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b40014040 con 0x556100887370 2024-09-10T08:51:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.204+0000 7f5b26ffd640 1 -- 172.21.15.172:0/4093719052 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5b40026620 con 0x556100887370 2024-09-10T08:51:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.204+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4093719052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b38040070 con 0x556100887370 2024-09-10T08:51:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.204+0000 7f5b26ffd640 1 -- 172.21.15.172:0/4093719052 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5b40031050 con 0x556100887370 2024-09-10T08:51:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.204+0000 7f5b50c4c740 10 client.17010 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.204+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4093719052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5561005c4620 con 0x556100887370 2024-09-10T08:51:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.204+0000 7f5b26ffd640 1 -- 172.21.15.172:0/4093719052 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b38040070 con 0x556100887370 2024-09-10T08:51:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.204+0000 7f5b26ffd640 1 -- 172.21.15.172:0/4093719052 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f5b40025630 con 0x556100887370 2024-09-10T08:51:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.205+0000 7f5b50c4c740 10 client.17010 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.205+0000 7f5b50c4c740 10 client.17010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:51:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.205+0000 7f5b50c4c740 20 client.17010 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.205+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/4093719052 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5561008978a0 0x556100899d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.205+0000 7f5b50c4c740 4 client.17010 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:51:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.205+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4093719052 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561007f2a20 con 0x5561008978a0 2024-09-10T08:51:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.205+0000 7f5b477fe640 1 --2- 172.21.15.172:0/4093719052 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5561008978a0 0x556100899d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.205+0000 7f5b26ffd640 10 client.17010 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.205+0000 7f5b477fe640 1 --2- 172.21.15.172:0/4093719052 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5561008978a0 0x556100899d40 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bdb6d4df65e396d server_cookie=fb06b756ffc657c6 in_seq=0 out_seq=0 2024-09-10T08:51:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.206+0000 7f5b26ffd640 1 -- 172.21.15.172:0/4093719052 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5561007f2a20 con 0x5561008978a0 2024-09-10T08:51:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.206+0000 7f5b26ffd640 10 client.17010 handle_command_reply: tid=0 2024-09-10T08:51:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.206+0000 7f5b50c4c740 1 client.17010 shutdown 2024-09-10T08:51:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.206+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4093719052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556100887370 msgr2=0x556100893050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.206+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/4093719052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556100887370 0x556100893050 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5b4001a620 tx=0x7f5b4001a000 comp rx=0 tx=0).stop 2024-09-10T08:51:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.206+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4093719052 shutdown_connections 2024-09-10T08:51:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.206+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/4093719052 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5561008978a0 0x556100899d40 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.206+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/4093719052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556100887370 0x556100893050 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.206+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/4093719052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5561008822e0 0x55610088c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.206+0000 7f5b26ffd640 0 client.17010 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.206+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/4093719052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008819a0 0x55610088bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.206+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4093719052 >> 172.21.15.172:0/4093719052 conn(0x55610087d010 msgr2=0x55610087e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.206+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4093719052 shutdown_connections 2024-09-10T08:51:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.206+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4093719052 wait complete. 2024-09-10T08:51:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.206+0000 7f5b50c4c740 20 client.17010 trim_cache size 0 max 16384 2024-09-10T08:51:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.225+0000 7f5b50c4c740 1 Processor -- start 2024-09-10T08:51:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.225+0000 7f5b50c4c740 1 -- start start 2024-09-10T08:51:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.225+0000 7f5b50c4c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561008819a0 0x55610087c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.225+0000 7f5b50c4c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008822e0 0x556100892f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.225+0000 7f5b50c4c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556100887370 0x55610089e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.225+0000 7f5b50c4c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5b481ac840 con 0x5561008819a0 2024-09-10T08:51:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.225+0000 7f5b50c4c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b481ae440 con 0x5561008822e0 2024-09-10T08:51:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.225+0000 7f5b50c4c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5b481acf10 con 0x556100887370 2024-09-10T08:51:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.225+0000 7f5b477fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561008819a0 0x55610087c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.225+0000 7f5b477fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561008819a0 0x55610087c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41982/0 (socket says 172.21.15.172:41982) 2024-09-10T08:51:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.225+0000 7f5b477fe640 1 -- 172.21.15.172:0/4237347589 learned_addr learned my addr 172.21.15.172:0/4237347589 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.226+0000 7f5b46ffd640 1 --2- 172.21.15.172:0/4237347589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008822e0 0x556100892f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.226+0000 7f5b47fff640 1 --2- 172.21.15.172:0/4237347589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556100887370 0x55610089e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.226+0000 7f5b477fe640 1 -- 172.21.15.172:0/4237347589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556100887370 msgr2=0x55610089e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.226+0000 7f5b477fe640 1 --2- 172.21.15.172:0/4237347589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556100887370 0x55610089e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.226+0000 7f5b477fe640 1 -- 172.21.15.172:0/4237347589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008822e0 msgr2=0x556100892f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.226+0000 7f5b477fe640 1 --2- 172.21.15.172:0/4237347589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008822e0 0x556100892f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.226+0000 7f5b477fe640 1 -- 172.21.15.172:0/4237347589 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55610087ce30 con 0x5561008819a0 2024-09-10T08:51:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.226+0000 7f5b477fe640 1 --2- 172.21.15.172:0/4237347589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561008819a0 0x55610087c0b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x55610087de90 tx=0x7f5b380035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b697579639941d7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.226+0000 7f5b2d7fa640 1 -- 172.21.15.172:0/4237347589 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38035040 con 0x5561008819a0 2024-09-10T08:51:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.226+0000 7f5b2d7fa640 1 -- 172.21.15.172:0/4237347589 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b38040040 con 0x5561008819a0 2024-09-10T08:51:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.226+0000 7f5b2d7fa640 1 -- 172.21.15.172:0/4237347589 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3803c440 con 0x5561008819a0 2024-09-10T08:51:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.227+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4237347589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561008819a0 msgr2=0x55610087c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.227+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/4237347589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561008819a0 0x55610087c0b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x55610087de90 tx=0x7f5b380035f0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.227+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4237347589 shutdown_connections 2024-09-10T08:51:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.227+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/4237347589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556100887370 0x55610089e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.227+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/4237347589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008822e0 0x556100892f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.227+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/4237347589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561008819a0 0x55610087c0b0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.227+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4237347589 >> 172.21.15.172:0/4237347589 conn(0x55610087d010 msgr2=0x55610087b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.227+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4237347589 shutdown_connections 2024-09-10T08:51:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.227+0000 7f5b50c4c740 1 -- 172.21.15.172:0/4237347589 wait complete. 2024-09-10T08:51:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.227+0000 7f5b50c4c740 1 -- start start 2024-09-10T08:51:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.227+0000 7f5b50c4c740 1 Processor -- start 2024-09-10T08:51:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b50c4c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008819a0 0x556100890370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b50c4c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561008822e0 0x5561008908b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b50c4c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556100887370 0x556100891850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b50c4c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5b481adce0 con 0x5561008822e0 2024-09-10T08:51:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b50c4c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b481ad610 con 0x5561008819a0 2024-09-10T08:51:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b50c4c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5b481aa820 con 0x556100887370 2024-09-10T08:51:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b50c4c740 10 client.0 _ll_register_callbacks cb 0x7f5b400099d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b46ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561008822e0 0x5561008908b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b46ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561008822e0 0x5561008908b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41984/0 (socket says 172.21.15.172:41984) 2024-09-10T08:51:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b46ffd640 1 -- 172.21.15.172:0/2467473921 learned_addr learned my addr 172.21.15.172:0/2467473921 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b477fe640 1 --2- 172.21.15.172:0/2467473921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008819a0 0x556100890370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b47fff640 1 --2- 172.21.15.172:0/2467473921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556100887370 0x556100891850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b46ffd640 1 -- 172.21.15.172:0/2467473921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556100887370 msgr2=0x556100891850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b46ffd640 1 --2- 172.21.15.172:0/2467473921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556100887370 0x556100891850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b46ffd640 1 -- 172.21.15.172:0/2467473921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008819a0 msgr2=0x556100890370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b46ffd640 1 --2- 172.21.15.172:0/2467473921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008819a0 0x556100890370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b46ffd640 1 -- 172.21.15.172:0/2467473921 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556100891e60 con 0x5561008822e0 2024-09-10T08:51:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b46ffd640 1 --2- 172.21.15.172:0/2467473921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561008822e0 0x5561008908b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c0cd170 tx=0x7f5b3c09bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71b5a62ed2b92a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b26ffd640 1 -- 172.21.15.172:0/2467473921 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c010430 con 0x5561008822e0 2024-09-10T08:51:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b26ffd640 1 -- 172.21.15.172:0/2467473921 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5b3c0105d0 con 0x5561008822e0 2024-09-10T08:51:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.228+0000 7f5b50c4c740 1 -- 172.21.15.172:0/2467473921 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b38040040 con 0x5561008822e0 2024-09-10T08:51:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.229+0000 7f5b26ffd640 1 -- 172.21.15.172:0/2467473921 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5b3c09b060 con 0x5561008822e0 2024-09-10T08:51:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.229+0000 7f5b50c4c740 10 client.17016 fetch_fsmap learned FSMap version 33 2024-09-10T08:51:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.229+0000 7f5b50c4c740 1 -- 172.21.15.172:0/2467473921 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5561008931e0 con 0x5561008822e0 2024-09-10T08:51:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.229+0000 7f5b26ffd640 1 -- 172.21.15.172:0/2467473921 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b38040040 con 0x5561008822e0 2024-09-10T08:51:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.229+0000 7f5b26ffd640 1 -- 172.21.15.172:0/2467473921 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8386+0+0 (secure 0 0 0) 0x7f5b3c018cc0 con 0x5561008822e0 2024-09-10T08:51:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.229+0000 7f5b50c4c740 10 client.17016 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T08:51:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.229+0000 7f5b50c4c740 10 client.17016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:51:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.229+0000 7f5b50c4c740 20 client.17016 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.229+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/2467473921 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556100902cd0 0x5561009050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.229+0000 7f5b50c4c740 4 client.17016 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:51:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.229+0000 7f5b50c4c740 1 -- 172.21.15.172:0/2467473921 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561007ecd70 con 0x556100902cd0 2024-09-10T08:51:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.229+0000 7f5b477fe640 1 --2- 172.21.15.172:0/2467473921 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556100902cd0 0x5561009050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.230+0000 7f5b26ffd640 10 client.17016 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.230+0000 7f5b477fe640 1 --2- 172.21.15.172:0/2467473921 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556100902cd0 0x5561009050f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=626fe40a360ab61b server_cookie=cb0538365b4bd2e in_seq=0 out_seq=0 2024-09-10T08:51:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.235+0000 7f5b26ffd640 1 -- 172.21.15.172:0/2467473921 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5561007ecd70 con 0x556100902cd0 2024-09-10T08:51:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.235+0000 7f5b26ffd640 10 client.17016 handle_command_reply: tid=0 2024-09-10T08:51:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.235+0000 7f5b50c4c740 1 client.17016 shutdown 2024-09-10T08:51:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.236+0000 7f5b50c4c740 1 -- 172.21.15.172:0/2467473921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561008822e0 msgr2=0x5561008908b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.236+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/2467473921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561008822e0 0x5561008908b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c0cd170 tx=0x7f5b3c09bcd0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.236+0000 7f5b50c4c740 1 -- 172.21.15.172:0/2467473921 shutdown_connections 2024-09-10T08:51:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.236+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/2467473921 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556100902cd0 0x5561009050f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.236+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/2467473921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556100887370 0x556100891850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.236+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/2467473921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561008822e0 0x5561008908b0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.236+0000 7f5b26ffd640 0 client.17016 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.236+0000 7f5b50c4c740 1 --2- 172.21.15.172:0/2467473921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561008819a0 0x556100890370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.236+0000 7f5b50c4c740 1 -- 172.21.15.172:0/2467473921 >> 172.21.15.172:0/2467473921 conn(0x55610087d010 msgr2=0x55610087b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.236+0000 7f5b50c4c740 1 -- 172.21.15.172:0/2467473921 shutdown_connections 2024-09-10T08:51:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.236+0000 7f5b50c4c740 1 -- 172.21.15.172:0/2467473921 wait complete. 2024-09-10T08:51:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.236+0000 7f5b50c4c740 20 client.17016 trim_cache size 0 max 16384 2024-09-10T08:51:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:51:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:51:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:51:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:51:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.236+0000 7f5b4deac640 1 -- 172.21.15.172:0/4143086424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b300a2c30 msgr2=0x7f5b300a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.236+0000 7f5b4deac640 1 --2- 172.21.15.172:0/4143086424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b300a2c30 0x7f5b300a50f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009fd0 tx=0x7f5b40009ec0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.236+0000 7f5b4deac640 1 -- 172.21.15.172:0/4143086424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b480761b0 msgr2=0x7f5b4819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.236+0000 7f5b4deac640 1 --2- 172.21.15.172:0/4143086424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b480761b0 0x7f5b4819a480 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00e9f0 tx=0x7f5b3c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.236+0000 7f5b4deac640 1 -- 172.21.15.172:0/4143086424 shutdown_connections 2024-09-10T08:51:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.237+0000 7f5b4deac640 1 --2- 172.21.15.172:0/4143086424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5b300a2c30 0x7f5b300a50f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.237+0000 7f5b4deac640 1 --2- 172.21.15.172:0/4143086424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b48109a10 0x7f5b481a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.237+0000 7f5b4deac640 1 --2- 172.21.15.172:0/4143086424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b480761b0 0x7f5b4819a480 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.237+0000 7f5b4deac640 1 --2- 172.21.15.172:0/4143086424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b48074bd0 0x7f5b48199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.237+0000 7f5b4deac640 1 -- 172.21.15.172:0/4143086424 >> 172.21.15.172:0/4143086424 conn(0x7f5b480fd2e0 msgr2=0x7f5b481059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.237+0000 7f5b4deac640 1 -- 172.21.15.172:0/4143086424 shutdown_connections 2024-09-10T08:51:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:34.237+0000 7f5b4deac640 1 -- 172.21.15.172:0/4143086424 wait complete. 2024-09-10T08:51:34.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:34 smithi191 ceph-mon[35314]: pgmap v510: 129 pgs: 129 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-10T08:51:34.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:34 smithi172 ceph-mon[30854]: pgmap v510: 129 pgs: 129 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-10T08:51:34.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:34 smithi174 ceph-mon[34068]: pgmap v510: 129 pgs: 129 active+clean; 3.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-10T08:51:36.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:36 smithi191 ceph-mon[35314]: pgmap v511: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-09-10T08:51:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:36 smithi172 ceph-mon[30854]: pgmap v511: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-09-10T08:51:36.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:36 smithi174 ceph-mon[34068]: pgmap v511: 129 pgs: 129 active+clean; 3.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-09-10T08:51:38.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:38 smithi172 ceph-mon[30854]: pgmap v512: 129 pgs: 129 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-09-10T08:51:38.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:38 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:51:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:38 smithi174 ceph-mon[34068]: pgmap v512: 129 pgs: 129 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-09-10T08:51:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:38 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:51:38.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:38 smithi191 ceph-mon[35314]: pgmap v512: 129 pgs: 129 active+clean; 3.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 23 op/s 2024-09-10T08:51:38.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:38 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:51:40.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:40 smithi172 ceph-mon[30854]: pgmap v513: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:51:40.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:40 smithi174 ceph-mon[34068]: pgmap v513: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:51:40.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:40 smithi191 ceph-mon[35314]: pgmap v513: 129 pgs: 129 active+clean; 3.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:51:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:42 smithi172 ceph-mon[30854]: pgmap v514: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-09-10T08:51:42.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:42 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:51:42.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:42 smithi174 ceph-mon[34068]: pgmap v514: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-09-10T08:51:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:42 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:51:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:42 smithi191 ceph-mon[35314]: pgmap v514: 129 pgs: 129 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-09-10T08:51:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:42 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:51:43.359 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:51:43.646 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:51:44.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.205+0000 7fc154c96640 1 -- 172.21.15.172:0/1350628609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc150102d30 msgr2=0x7fc150103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:44.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.205+0000 7fc154c96640 1 --2- 172.21.15.172:0/1350628609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc150102d30 0x7fc150103110 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc140009930 tx=0x7fc14002a420 comp rx=0 tx=0).stop 2024-09-10T08:51:44.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.205+0000 7fc154c96640 1 -- 172.21.15.172:0/1350628609 shutdown_connections 2024-09-10T08:51:44.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.205+0000 7fc154c96640 1 --2- 172.21.15.172:0/1350628609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1501082a0 0x7fc15010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.205+0000 7fc154c96640 1 --2- 172.21.15.172:0/1350628609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1501036e0 0x7fc150107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.205+0000 7fc154c96640 1 --2- 172.21.15.172:0/1350628609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc150102d30 0x7fc150103110 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.205+0000 7fc154c96640 1 -- 172.21.15.172:0/1350628609 >> 172.21.15.172:0/1350628609 conn(0x7fc1500fc7d0 msgr2=0x7fc1500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.205+0000 7fc154c96640 1 -- 172.21.15.172:0/1350628609 shutdown_connections 2024-09-10T08:51:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.205+0000 7fc154c96640 1 -- 172.21.15.172:0/1350628609 wait complete. 2024-09-10T08:51:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.206+0000 7fc154c96640 1 Processor -- start 2024-09-10T08:51:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.206+0000 7fc154c96640 1 -- start start 2024-09-10T08:51:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.206+0000 7fc154c96640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc150102d30 0x7fc150198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.206+0000 7fc154c96640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1501036e0 0x7fc150198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.206+0000 7fc154c96640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1501082a0 0x7fc15019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.206+0000 7fc154c96640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc150111f50 con 0x7fc1501036e0 2024-09-10T08:51:44.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.207+0000 7fc14dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1501036e0 0x7fc150198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.207+0000 7fc154c96640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc150111dd0 con 0x7fc150102d30 2024-09-10T08:51:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.207+0000 7fc154c96640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc1501120d0 con 0x7fc1501082a0 2024-09-10T08:51:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.207+0000 7fc14dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1501036e0 0x7fc150198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58016/0 (socket says 172.21.15.172:58016) 2024-09-10T08:51:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.207+0000 7fc14dd9b640 1 -- 172.21.15.172:0/3696327034 learned_addr learned my addr 172.21.15.172:0/3696327034 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.207+0000 7fc14ed9d640 1 --2- 172.21.15.172:0/3696327034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1501082a0 0x7fc15019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.207+0000 7fc14e59c640 1 --2- 172.21.15.172:0/3696327034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc150102d30 0x7fc150198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.207+0000 7fc14dd9b640 1 -- 172.21.15.172:0/3696327034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1501082a0 msgr2=0x7fc15019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.207+0000 7fc14dd9b640 1 --2- 172.21.15.172:0/3696327034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1501082a0 0x7fc15019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.207+0000 7fc14dd9b640 1 -- 172.21.15.172:0/3696327034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc150102d30 msgr2=0x7fc150198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.207+0000 7fc14dd9b640 1 --2- 172.21.15.172:0/3696327034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc150102d30 0x7fc150198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.207+0000 7fc14dd9b640 1 -- 172.21.15.172:0/3696327034 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15019d4e0 con 0x7fc1501036e0 2024-09-10T08:51:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.207+0000 7fc14dd9b640 1 --2- 172.21.15.172:0/3696327034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1501036e0 0x7fc150198a70 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc14400d950 tx=0x7fc14400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:44.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.207+0000 7fc13f7fe640 1 -- 172.21.15.172:0/3696327034 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc144017070 con 0x7fc1501036e0 2024-09-10T08:51:44.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.208+0000 7fc154c96640 1 -- 172.21.15.172:0/3696327034 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15019d7d0 con 0x7fc1501036e0 2024-09-10T08:51:44.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.208+0000 7fc13f7fe640 1 -- 172.21.15.172:0/3696327034 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc144009c30 con 0x7fc1501036e0 2024-09-10T08:51:44.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.208+0000 7fc13f7fe640 1 -- 172.21.15.172:0/3696327034 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1440047a0 con 0x7fc1501036e0 2024-09-10T08:51:44.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.208+0000 7fc154c96640 1 -- 172.21.15.172:0/3696327034 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1501a50b0 con 0x7fc1501036e0 2024-09-10T08:51:44.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.210+0000 7fc13f7fe640 1 -- 172.21.15.172:0/3696327034 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc144004020 con 0x7fc1501036e0 2024-09-10T08:51:44.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.210+0000 7fc15706f740 1 Processor -- start 2024-09-10T08:51:44.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.211+0000 7fc15706f740 1 -- start start 2024-09-10T08:51:44.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.211+0000 7fc15706f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb34a0 0x55ff30fb3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.211+0000 7fc15706f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fbe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.211+0000 7fc15706f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fbe680 0x55ff30fc0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.211+0000 7fc15706f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff30f192e0 con 0x55ff30fb3dc0 2024-09-10T08:51:44.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.211+0000 7fc15706f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ff30ec0ce0 con 0x55ff30fb34a0 2024-09-10T08:51:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.211+0000 7fc15706f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ff30f152a0 con 0x55ff30fbe680 2024-09-10T08:51:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.211+0000 7fc14e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fbe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.211+0000 7fc14e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fbe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58020/0 (socket says 172.21.15.172:58020) 2024-09-10T08:51:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.211+0000 7fc14e59c640 1 -- 172.21.15.172:0/1957934014 learned_addr learned my addr 172.21.15.172:0/1957934014 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.211+0000 7fc13f7fe640 1 --2- 172.21.15.172:0/3696327034 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc1380a2c00 0x7fc1380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.211+0000 7fc14ed9d640 1 --2- 172.21.15.172:0/1957934014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb34a0 0x55ff30fb3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.211+0000 7fc14ed9d640 1 --2- 172.21.15.172:0/3696327034 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc1380a2c00 0x7fc1380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.211+0000 7fc14dd9b640 1 --2- 172.21.15.172:0/1957934014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fbe680 0x55ff30fc0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.212+0000 7fc14e59c640 1 -- 172.21.15.172:0/1957934014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fbe680 msgr2=0x55ff30fc0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.212+0000 7fc14e59c640 1 --2- 172.21.15.172:0/1957934014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fbe680 0x55ff30fc0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.212+0000 7fc14e59c640 1 -- 172.21.15.172:0/1957934014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb34a0 msgr2=0x55ff30fb3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.212+0000 7fc14e59c640 1 --2- 172.21.15.172:0/1957934014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb34a0 0x55ff30fb3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.212+0000 7fc14e59c640 1 -- 172.21.15.172:0/1957934014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff30ea5440 con 0x55ff30fb3dc0 2024-09-10T08:51:44.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.212+0000 7fc14dd9b640 1 --2- 172.21.15.172:0/1957934014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fbe680 0x55ff30fc0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:44.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.212+0000 7fc13f7fe640 1 -- 172.21.15.172:0/3696327034 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fc1440ceef0 con 0x7fc1501036e0 2024-09-10T08:51:44.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.212+0000 7fc14e59c640 1 --2- 172.21.15.172:0/1957934014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fbe140 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc140009930 tx=0x7fc140005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efd703365ad8d4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:44.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.212+0000 7fc14ed9d640 1 --2- 172.21.15.172:0/3696327034 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc1380a2c00 0x7fc1380a50c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc14800afd0 tx=0x7fc14800a4d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:44.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.212+0000 7fc13cff9640 1 -- 172.21.15.172:0/1957934014 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140040070 con 0x55ff30fb3dc0 2024-09-10T08:51:44.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.212+0000 7fc13cff9640 1 -- 172.21.15.172:0/1957934014 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1400316a0 con 0x55ff30fb3dc0 2024-09-10T08:51:44.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.212+0000 7fc13cff9640 1 -- 172.21.15.172:0/1957934014 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140035040 con 0x55ff30fb3dc0 2024-09-10T08:51:44.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.213+0000 7fc15706f740 1 -- 172.21.15.172:0/1957934014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 msgr2=0x55ff30fbe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:44.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.213+0000 7fc15706f740 1 --2- 172.21.15.172:0/1957934014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fbe140 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc140009930 tx=0x7fc140005fd0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.213+0000 7fc15706f740 1 -- 172.21.15.172:0/1957934014 shutdown_connections 2024-09-10T08:51:44.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.213+0000 7fc15706f740 1 --2- 172.21.15.172:0/1957934014 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fbe680 0x55ff30fc0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.213+0000 7fc15706f740 1 --2- 172.21.15.172:0/1957934014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fbe140 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.213+0000 7fc15706f740 1 --2- 172.21.15.172:0/1957934014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb34a0 0x55ff30fb3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.213+0000 7fc15706f740 1 -- 172.21.15.172:0/1957934014 >> 172.21.15.172:0/1957934014 conn(0x55ff30fafa30 msgr2=0x55ff30fafe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:44.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.213+0000 7fc15706f740 1 -- 172.21.15.172:0/1957934014 shutdown_connections 2024-09-10T08:51:44.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.213+0000 7fc15706f740 1 -- 172.21.15.172:0/1957934014 wait complete. 2024-09-10T08:51:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.213+0000 7fc15706f740 1 -- start start 2024-09-10T08:51:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc15706f740 1 Processor -- start 2024-09-10T08:51:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc15706f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fb34a0 0x55ff30fbdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc15706f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb3dc0 0x55ff30fbb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc15706f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fbe680 0x55ff30fbb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc15706f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff30fc5660 con 0x55ff30fbe680 2024-09-10T08:51:44.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc15706f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ff30ed4050 con 0x55ff30fb3dc0 2024-09-10T08:51:44.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc15706f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ff30fc57e0 con 0x55ff30fb34a0 2024-09-10T08:51:44.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc15706f740 10 client.0 _ll_register_callbacks cb 0x55ff30f14160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:44.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc14ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fbe680 0x55ff30fbb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc14ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fbe680 0x55ff30fbb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58030/0 (socket says 172.21.15.172:58030) 2024-09-10T08:51:44.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc14ed9d640 1 -- 172.21.15.172:0/2043229263 learned_addr learned my addr 172.21.15.172:0/2043229263 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:44.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc14e59c640 1 --2- 172.21.15.172:0/2043229263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fb34a0 0x55ff30fbdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc14dd9b640 1 --2- 172.21.15.172:0/2043229263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb3dc0 0x55ff30fbb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc14ed9d640 1 -- 172.21.15.172:0/2043229263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fb34a0 msgr2=0x55ff30fbdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:44.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc14ed9d640 1 --2- 172.21.15.172:0/2043229263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fb34a0 0x55ff30fbdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc14ed9d640 1 -- 172.21.15.172:0/2043229263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb3dc0 msgr2=0x55ff30fbb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:44.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc14ed9d640 1 --2- 172.21.15.172:0/2043229263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb3dc0 0x55ff30fbb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc14ed9d640 1 -- 172.21.15.172:0/2043229263 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff30c8a3c0 con 0x55ff30fbe680 2024-09-10T08:51:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.214+0000 7fc14e59c640 1 --2- 172.21.15.172:0/2043229263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fb34a0 0x55ff30fbdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:51:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.215+0000 7fc14ed9d640 1 --2- 172.21.15.172:0/2043229263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fbe680 0x55ff30fbb6e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc148011f00 tx=0x7fc148004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4542420eed2bed3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.215+0000 7fc135ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.215+0000 7fc135ffb640 1 -- 172.21.15.172:0/2043229263 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148004b20 con 0x55ff30fbe680 2024-09-10T08:51:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.215+0000 7fc135ffb640 1 -- 172.21.15.172:0/2043229263 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fc14801d040 con 0x55ff30fbe680 2024-09-10T08:51:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.215+0000 7fc15706f740 1 -- 172.21.15.172:0/2043229263 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1400316a0 con 0x55ff30fbe680 2024-09-10T08:51:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.215+0000 7fc135ffb640 1 -- 172.21.15.172:0/2043229263 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc14802f050 con 0x55ff30fbe680 2024-09-10T08:51:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.215+0000 7fc15706f740 10 client.17028 fetch_fsmap learned FSMap version 35 2024-09-10T08:51:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.215+0000 7fc15706f740 1 -- 172.21.15.172:0/2043229263 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ff30fd0f70 con 0x55ff30fbe680 2024-09-10T08:51:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.215+0000 7fc135ffb640 1 -- 172.21.15.172:0/2043229263 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1400316a0 con 0x55ff30fbe680 2024-09-10T08:51:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.216+0000 7fc135ffb640 1 -- 172.21.15.172:0/2043229263 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fc148021070 con 0x55ff30fbe680 2024-09-10T08:51:44.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.216+0000 7fc15706f740 10 client.17028 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T08:51:44.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.216+0000 7fc15706f740 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:51:44.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.216+0000 7fc15706f740 20 client.17028 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:44.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.216+0000 7fc15706f740 1 --2- 172.21.15.172:0/2043229263 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ff30fd2480 0x55ff30fd4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.216+0000 7fc15706f740 4 client.17028 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:51:44.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.216+0000 7fc15706f740 1 -- 172.21.15.172:0/2043229263 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff30f26850 con 0x55ff30fd2480 2024-09-10T08:51:44.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.216+0000 7fc14e59c640 1 --2- 172.21.15.172:0/2043229263 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ff30fd2480 0x55ff30fd4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.217+0000 7fc135ffb640 10 client.17028 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:44.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.217+0000 7fc14e59c640 1 --2- 172.21.15.172:0/2043229263 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ff30fd2480 0x55ff30fd4940 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41575a0b96553372 server_cookie=50445f14b06c525b in_seq=0 out_seq=0 2024-09-10T08:51:44.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.217+0000 7fc135ffb640 1 -- 172.21.15.172:0/2043229263 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff30f26850 con 0x55ff30fd2480 2024-09-10T08:51:44.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.217+0000 7fc135ffb640 10 client.17028 handle_command_reply: tid=0 2024-09-10T08:51:44.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.217+0000 7fc15706f740 1 client.17028 shutdown 2024-09-10T08:51:44.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.217+0000 7fc15706f740 1 -- 172.21.15.172:0/2043229263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fbe680 msgr2=0x55ff30fbb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:44.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.218+0000 7fc15706f740 1 --2- 172.21.15.172:0/2043229263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fbe680 0x55ff30fbb6e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc148011f00 tx=0x7fc148004a40 comp rx=0 tx=0).stop 2024-09-10T08:51:44.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.218+0000 7fc15706f740 1 -- 172.21.15.172:0/2043229263 shutdown_connections 2024-09-10T08:51:44.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.218+0000 7fc15706f740 1 --2- 172.21.15.172:0/2043229263 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ff30fd2480 0x55ff30fd4940 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.218+0000 7fc135ffb640 0 client.17028 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:44.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.218+0000 7fc15706f740 1 --2- 172.21.15.172:0/2043229263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fbe680 0x55ff30fbb6e0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.218+0000 7fc15706f740 1 --2- 172.21.15.172:0/2043229263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb3dc0 0x55ff30fbb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.218+0000 7fc15706f740 1 --2- 172.21.15.172:0/2043229263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fb34a0 0x55ff30fbdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.218+0000 7fc15706f740 1 -- 172.21.15.172:0/2043229263 >> 172.21.15.172:0/2043229263 conn(0x55ff30fafa30 msgr2=0x55ff30fb5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:44.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.218+0000 7fc15706f740 1 -- 172.21.15.172:0/2043229263 shutdown_connections 2024-09-10T08:51:44.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.218+0000 7fc15706f740 1 -- 172.21.15.172:0/2043229263 wait complete. 2024-09-10T08:51:44.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.218+0000 7fc15706f740 20 client.17028 trim_cache size 0 max 16384 2024-09-10T08:51:44.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.241+0000 7fc15706f740 1 Processor -- start 2024-09-10T08:51:44.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.241+0000 7fc15706f740 1 -- start start 2024-09-10T08:51:44.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.241+0000 7fc15706f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fafa30 0x55ff30fbd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.241+0000 7fc15706f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb34a0 0x55ff30fda210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.241+0000 7fc15706f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fda750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.241+0000 7fc15706f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc1501ac9a0 con 0x55ff30fb3dc0 2024-09-10T08:51:44.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.241+0000 7fc15706f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1501ae540 con 0x55ff30fb34a0 2024-09-10T08:51:44.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.241+0000 7fc15706f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc1501ad070 con 0x55ff30fafa30 2024-09-10T08:51:44.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.241+0000 7fc14ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fda750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.241+0000 7fc14ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fda750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58038/0 (socket says 172.21.15.172:58038) 2024-09-10T08:51:44.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.241+0000 7fc14ed9d640 1 -- 172.21.15.172:0/592226052 learned_addr learned my addr 172.21.15.172:0/592226052 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:44.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.241+0000 7fc14e59c640 1 --2- 172.21.15.172:0/592226052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fafa30 0x55ff30fbd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.242+0000 7fc14dd9b640 1 --2- 172.21.15.172:0/592226052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb34a0 0x55ff30fda210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.242+0000 7fc14ed9d640 1 -- 172.21.15.172:0/592226052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fafa30 msgr2=0x55ff30fbd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:44.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.242+0000 7fc14ed9d640 1 --2- 172.21.15.172:0/592226052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fafa30 0x55ff30fbd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.242+0000 7fc14ed9d640 1 -- 172.21.15.172:0/592226052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb34a0 msgr2=0x55ff30fda210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:44.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.242+0000 7fc14ed9d640 1 --2- 172.21.15.172:0/592226052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb34a0 0x55ff30fda210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.242+0000 7fc14ed9d640 1 -- 172.21.15.172:0/592226052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff30faf860 con 0x55ff30fb3dc0 2024-09-10T08:51:44.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.242+0000 7fc14e59c640 1 --2- 172.21.15.172:0/592226052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fafa30 0x55ff30fbd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:51:44.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.242+0000 7fc14ed9d640 1 --2- 172.21.15.172:0/592226052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fda750 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fc148012010 tx=0x7fc148014470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e6606e0bf3fa37 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:44.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.242+0000 7fc137fff640 1 -- 172.21.15.172:0/592226052 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148018070 con 0x55ff30fb3dc0 2024-09-10T08:51:44.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.242+0000 7fc137fff640 1 -- 172.21.15.172:0/592226052 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc148004be0 con 0x55ff30fb3dc0 2024-09-10T08:51:44.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.242+0000 7fc137fff640 1 -- 172.21.15.172:0/592226052 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1480051c0 con 0x55ff30fb3dc0 2024-09-10T08:51:44.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.243+0000 7fc15706f740 1 -- 172.21.15.172:0/592226052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 msgr2=0x55ff30fda750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:44.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.243+0000 7fc15706f740 1 --2- 172.21.15.172:0/592226052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fda750 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fc148012010 tx=0x7fc148014470 comp rx=0 tx=0).stop 2024-09-10T08:51:44.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.243+0000 7fc15706f740 1 -- 172.21.15.172:0/592226052 shutdown_connections 2024-09-10T08:51:44.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.243+0000 7fc15706f740 1 --2- 172.21.15.172:0/592226052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fda750 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.243+0000 7fc15706f740 1 --2- 172.21.15.172:0/592226052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb34a0 0x55ff30fda210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.243+0000 7fc15706f740 1 --2- 172.21.15.172:0/592226052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fafa30 0x55ff30fbd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.243+0000 7fc15706f740 1 -- 172.21.15.172:0/592226052 >> 172.21.15.172:0/592226052 conn(0x55ff30fbb840 msgr2=0x55ff30fbca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.243+0000 7fc15706f740 1 -- 172.21.15.172:0/592226052 shutdown_connections 2024-09-10T08:51:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.243+0000 7fc15706f740 1 -- 172.21.15.172:0/592226052 wait complete. 2024-09-10T08:51:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.244+0000 7fc15706f740 1 -- start start 2024-09-10T08:51:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.244+0000 7fc15706f740 1 Processor -- start 2024-09-10T08:51:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.244+0000 7fc15706f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fafa30 0x55ff30fc8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.244+0000 7fc15706f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb34a0 0x55ff30fc8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.244+0000 7fc15706f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fc91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.244+0000 7fc15706f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc1501ade40 con 0x55ff30fb3dc0 2024-09-10T08:51:44.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.244+0000 7fc15706f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1501ad770 con 0x55ff30fb34a0 2024-09-10T08:51:44.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.244+0000 7fc15706f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc1501aa980 con 0x55ff30fafa30 2024-09-10T08:51:44.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.244+0000 7fc15706f740 10 client.0 _ll_register_callbacks cb 0x7fc14800aab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:44.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.244+0000 7fc14ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fc91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.244+0000 7fc14ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fc91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58054/0 (socket says 172.21.15.172:58054) 2024-09-10T08:51:44.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.244+0000 7fc14ed9d640 1 -- 172.21.15.172:0/1073038888 learned_addr learned my addr 172.21.15.172:0/1073038888 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:44.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.244+0000 7fc14e59c640 1 --2- 172.21.15.172:0/1073038888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fafa30 0x55ff30fc8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.244+0000 7fc14dd9b640 1 --2- 172.21.15.172:0/1073038888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb34a0 0x55ff30fc8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.245+0000 7fc14ed9d640 1 -- 172.21.15.172:0/1073038888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fafa30 msgr2=0x55ff30fc8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:44.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.245+0000 7fc14ed9d640 1 --2- 172.21.15.172:0/1073038888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fafa30 0x55ff30fc8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.245+0000 7fc14ed9d640 1 -- 172.21.15.172:0/1073038888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb34a0 msgr2=0x55ff30fc8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:44.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.245+0000 7fc14ed9d640 1 --2- 172.21.15.172:0/1073038888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb34a0 0x55ff30fc8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.245+0000 7fc14ed9d640 1 -- 172.21.15.172:0/1073038888 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff3103c1f0 con 0x55ff30fb3dc0 2024-09-10T08:51:44.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.245+0000 7fc14ed9d640 1 --2- 172.21.15.172:0/1073038888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fc91a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc148011c20 tx=0x7fc148004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c93a0682dc2eb35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:44.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.245+0000 7fc135ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:44.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.245+0000 7fc135ffb640 1 -- 172.21.15.172:0/1073038888 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148018070 con 0x55ff30fb3dc0 2024-09-10T08:51:44.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.245+0000 7fc135ffb640 1 -- 172.21.15.172:0/1073038888 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fc148020880 con 0x55ff30fb3dc0 2024-09-10T08:51:44.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.245+0000 7fc15706f740 1 -- 172.21.15.172:0/1073038888 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc148004be0 con 0x55ff30fb3dc0 2024-09-10T08:51:44.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.245+0000 7fc135ffb640 1 -- 172.21.15.172:0/1073038888 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc148020a80 con 0x55ff30fb3dc0 2024-09-10T08:51:44.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.245+0000 7fc15706f740 10 client.17034 fetch_fsmap learned FSMap version 35 2024-09-10T08:51:44.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.245+0000 7fc15706f740 1 -- 172.21.15.172:0/1073038888 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ff3103d460 con 0x55ff30fb3dc0 2024-09-10T08:51:44.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.246+0000 7fc135ffb640 1 -- 172.21.15.172:0/1073038888 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc148004be0 con 0x55ff30fb3dc0 2024-09-10T08:51:44.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.246+0000 7fc135ffb640 1 -- 172.21.15.172:0/1073038888 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fc1480224c0 con 0x55ff30fb3dc0 2024-09-10T08:51:44.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.246+0000 7fc15706f740 10 client.17034 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T08:51:44.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.246+0000 7fc15706f740 10 client.17034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:51:44.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.246+0000 7fc15706f740 20 client.17034 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:44.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.246+0000 7fc15706f740 1 --2- 172.21.15.172:0/1073038888 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ff30fbe680 0x55ff3103faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:44.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.246+0000 7fc15706f740 4 client.17034 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:51:44.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.246+0000 7fc15706f740 1 -- 172.21.15.172:0/1073038888 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ff30f20d60 con 0x55ff30fbe680 2024-09-10T08:51:44.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.246+0000 7fc14e59c640 1 --2- 172.21.15.172:0/1073038888 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ff30fbe680 0x55ff3103faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:44.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.246+0000 7fc14e59c640 1 --2- 172.21.15.172:0/1073038888 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ff30fbe680 0x55ff3103faa0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e44d8bddad0082d8 server_cookie=7ac35ead75edecd2 in_seq=0 out_seq=0 2024-09-10T08:51:44.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.246+0000 7fc135ffb640 10 client.17034 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:44.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.247+0000 7fc135ffb640 1 -- 172.21.15.172:0/1073038888 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ff30f20d60 con 0x55ff30fbe680 2024-09-10T08:51:44.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.247+0000 7fc135ffb640 10 client.17034 handle_command_reply: tid=0 2024-09-10T08:51:44.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.247+0000 7fc15706f740 1 client.17034 shutdown 2024-09-10T08:51:44.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.247+0000 7fc15706f740 1 -- 172.21.15.172:0/1073038888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 msgr2=0x55ff30fc91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:44.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.247+0000 7fc15706f740 1 --2- 172.21.15.172:0/1073038888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fc91a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc148011c20 tx=0x7fc148004ab0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.247+0000 7fc15706f740 1 -- 172.21.15.172:0/1073038888 shutdown_connections 2024-09-10T08:51:44.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.247+0000 7fc15706f740 1 --2- 172.21.15.172:0/1073038888 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ff30fbe680 0x55ff3103faa0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.247+0000 7fc15706f740 1 --2- 172.21.15.172:0/1073038888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff30fb3dc0 0x55ff30fc91a0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.247+0000 7fc15706f740 1 --2- 172.21.15.172:0/1073038888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff30fb34a0 0x55ff30fc8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.247+0000 7fc15706f740 1 --2- 172.21.15.172:0/1073038888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff30fafa30 0x55ff30fc8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.247+0000 7fc15706f740 1 -- 172.21.15.172:0/1073038888 >> 172.21.15.172:0/1073038888 conn(0x55ff30fbb840 msgr2=0x55ff30fbd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:44.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.247+0000 7fc135ffb640 0 client.17034 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:44.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.247+0000 7fc15706f740 1 -- 172.21.15.172:0/1073038888 shutdown_connections 2024-09-10T08:51:44.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.247+0000 7fc15706f740 1 -- 172.21.15.172:0/1073038888 wait complete. 2024-09-10T08:51:44.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.247+0000 7fc15706f740 20 client.17034 trim_cache size 0 max 16384 2024-09-10T08:51:44.259 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:51:44.259 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T08:51:44.259 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T08:51:44.259 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:51:44.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.248+0000 7fc154c96640 1 -- 172.21.15.172:0/3696327034 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc1380a2c00 msgr2=0x7fc1380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:44.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.248+0000 7fc154c96640 1 --2- 172.21.15.172:0/3696327034 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc1380a2c00 0x7fc1380a50c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc14800afd0 tx=0x7fc14800a4d0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.248+0000 7fc154c96640 1 -- 172.21.15.172:0/3696327034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1501036e0 msgr2=0x7fc150198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:44.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.248+0000 7fc154c96640 1 --2- 172.21.15.172:0/3696327034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1501036e0 0x7fc150198a70 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc14400d950 tx=0x7fc14400de10 comp rx=0 tx=0).stop 2024-09-10T08:51:44.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.248+0000 7fc154c96640 1 -- 172.21.15.172:0/3696327034 shutdown_connections 2024-09-10T08:51:44.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.248+0000 7fc154c96640 1 --2- 172.21.15.172:0/3696327034 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc1380a2c00 0x7fc1380a50c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.248+0000 7fc154c96640 1 --2- 172.21.15.172:0/3696327034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1501082a0 0x7fc15019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.248+0000 7fc154c96640 1 --2- 172.21.15.172:0/3696327034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1501036e0 0x7fc150198a70 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.248+0000 7fc154c96640 1 --2- 172.21.15.172:0/3696327034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc150102d30 0x7fc150198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:44.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.248+0000 7fc154c96640 1 -- 172.21.15.172:0/3696327034 >> 172.21.15.172:0/3696327034 conn(0x7fc1500fc7d0 msgr2=0x7fc150105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:44.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.248+0000 7fc154c96640 1 -- 172.21.15.172:0/3696327034 shutdown_connections 2024-09-10T08:51:44.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:44.248+0000 7fc154c96640 1 -- 172.21.15.172:0/3696327034 wait complete. 2024-09-10T08:51:44.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:44 smithi172 ceph-mon[30854]: pgmap v515: 129 pgs: 129 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-09-10T08:51:44.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:44 smithi174 ceph-mon[34068]: pgmap v515: 129 pgs: 129 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-09-10T08:51:44.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:44 smithi191 ceph-mon[35314]: pgmap v515: 129 pgs: 129 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-09-10T08:51:45.136 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:51:45.140 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:51:45.446 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:51:45.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.970+0000 7f047bffd640 1 -- 172.21.15.172:0/350971961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0474100e20 msgr2=0x7f0474101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:45.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.970+0000 7f047bffd640 1 --2- 172.21.15.172:0/350971961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0474100e20 0x7f0474101200 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0470009900 tx=0x7f047002a480 comp rx=0 tx=0).stop 2024-09-10T08:51:45.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.970+0000 7f047bffd640 1 -- 172.21.15.172:0/350971961 shutdown_connections 2024-09-10T08:51:45.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.970+0000 7f047bffd640 1 --2- 172.21.15.172:0/350971961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0474102ef0 0x7f047410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:45.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.970+0000 7f047bffd640 1 --2- 172.21.15.172:0/350971961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0474102570 0x7f04741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:45.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.970+0000 7f047bffd640 1 --2- 172.21.15.172:0/350971961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0474100e20 0x7f0474101200 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:45.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.970+0000 7f047bffd640 1 -- 172.21.15.172:0/350971961 >> 172.21.15.172:0/350971961 conn(0x7f04740fc7d0 msgr2=0x7f04740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:45.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.970+0000 7f047bffd640 1 -- 172.21.15.172:0/350971961 shutdown_connections 2024-09-10T08:51:45.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.970+0000 7f047bffd640 1 -- 172.21.15.172:0/350971961 wait complete. 2024-09-10T08:51:45.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.970+0000 7f047bffd640 1 Processor -- start 2024-09-10T08:51:45.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.970+0000 7f047bffd640 1 -- start start 2024-09-10T08:51:45.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.971+0000 7f047bffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0474100e20 0x7f0474199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:45.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.971+0000 7f047bffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0474102570 0x7f04741997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:45.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.971+0000 7f047bffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0474102ef0 0x7f04741a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:45.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.971+0000 7f047bffd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0474110e30 con 0x7f0474102ef0 2024-09-10T08:51:45.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.971+0000 7f047bffd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0474110cb0 con 0x7f0474102570 2024-09-10T08:51:45.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.971+0000 7f047bffd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0474110fb0 con 0x7f0474100e20 2024-09-10T08:51:45.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.971+0000 7f047a59a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0474102ef0 0x7f04741a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:45.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.971+0000 7f047a59a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0474102ef0 0x7f04741a42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58064/0 (socket says 172.21.15.172:58064) 2024-09-10T08:51:45.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.971+0000 7f047a59a640 1 -- 172.21.15.172:0/2350105452 learned_addr learned my addr 172.21.15.172:0/2350105452 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:45.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.971+0000 7f0479d99640 1 --2- 172.21.15.172:0/2350105452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0474100e20 0x7f0474199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:45.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.971+0000 7f0479598640 1 --2- 172.21.15.172:0/2350105452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0474102570 0x7f04741997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:45.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.971+0000 7f047a59a640 1 -- 172.21.15.172:0/2350105452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0474100e20 msgr2=0x7f0474199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:45.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.971+0000 7f047a59a640 1 --2- 172.21.15.172:0/2350105452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0474100e20 0x7f0474199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:45.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.971+0000 7f047a59a640 1 -- 172.21.15.172:0/2350105452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0474102570 msgr2=0x7f04741997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:45.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.971+0000 7f047a59a640 1 --2- 172.21.15.172:0/2350105452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0474102570 0x7f04741997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:45.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.971+0000 7f047a59a640 1 -- 172.21.15.172:0/2350105452 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04741a4820 con 0x7f0474102ef0 2024-09-10T08:51:45.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.972+0000 7f047a59a640 1 --2- 172.21.15.172:0/2350105452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0474102ef0 0x7f04741a42e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f046c00eaa0 tx=0x7f046c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:45.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.972+0000 7f046affd640 1 -- 172.21.15.172:0/2350105452 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c00cdb0 con 0x7f0474102ef0 2024-09-10T08:51:45.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.972+0000 7f047bffd640 1 -- 172.21.15.172:0/2350105452 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04741a49f0 con 0x7f0474102ef0 2024-09-10T08:51:45.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.972+0000 7f046affd640 1 -- 172.21.15.172:0/2350105452 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f046c005020 con 0x7f0474102ef0 2024-09-10T08:51:45.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.972+0000 7f046affd640 1 -- 172.21.15.172:0/2350105452 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c0116d0 con 0x7f0474102ef0 2024-09-10T08:51:45.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.972+0000 7f047bffd640 1 -- 172.21.15.172:0/2350105452 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04741a4f30 con 0x7f0474102ef0 2024-09-10T08:51:45.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.973+0000 7f047bffd640 1 -- 172.21.15.172:0/2350105452 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0474101200 con 0x7f0474102ef0 2024-09-10T08:51:45.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.974+0000 7f046affd640 1 -- 172.21.15.172:0/2350105452 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f046c0051d0 con 0x7f0474102ef0 2024-09-10T08:51:45.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.975+0000 7f046affd640 1 --2- 172.21.15.172:0/2350105452 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04600a2c00 0x7f04600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:45.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.975+0000 7f0479d99640 1 --2- 172.21.15.172:0/2350105452 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04600a2c00 0x7f04600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:45.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.975+0000 7f046affd640 1 -- 172.21.15.172:0/2350105452 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f046c0cf330 con 0x7f0474102ef0 2024-09-10T08:51:45.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.975+0000 7f0479d99640 1 --2- 172.21.15.172:0/2350105452 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04600a2c00 0x7f04600a50c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0470009230 tx=0x7f04700058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:45.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:45.980+0000 7f046affd640 1 -- 172.21.15.172:0/2350105452 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f046c016030 con 0x7f0474102ef0 2024-09-10T08:51:46.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:46.224+0000 7f047bffd640 1 -- 172.21.15.172:0/2350105452 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0474109090 con 0x7f0474102ef0 2024-09-10T08:51:46.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:46.225+0000 7f046affd640 1 -- 172.21.15.172:0/2350105452 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+12003 (secure 0 0 0) 0x7f046c09b980 con 0x7f0474102ef0 2024-09-10T08:51:46.227 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:51:46.227 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":35,"btime":"2024-09-10T08:51:41:307821+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:51:41.299894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:51:46.227 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 35 2024-09-10T08:51:46.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:46.228+0000 7f047bffd640 1 -- 172.21.15.172:0/2350105452 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04600a2c00 msgr2=0x7f04600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:46.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:46.228+0000 7f047bffd640 1 --2- 172.21.15.172:0/2350105452 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04600a2c00 0x7f04600a50c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0470009230 tx=0x7f04700058d0 comp rx=0 tx=0).stop 2024-09-10T08:51:46.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:46.228+0000 7f047bffd640 1 -- 172.21.15.172:0/2350105452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0474102ef0 msgr2=0x7f04741a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:46.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:46.228+0000 7f047bffd640 1 --2- 172.21.15.172:0/2350105452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0474102ef0 0x7f04741a42e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f046c00eaa0 tx=0x7f046c00ef60 comp rx=0 tx=0).stop 2024-09-10T08:51:46.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:46.228+0000 7f047bffd640 1 -- 172.21.15.172:0/2350105452 shutdown_connections 2024-09-10T08:51:46.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:46.228+0000 7f047bffd640 1 --2- 172.21.15.172:0/2350105452 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04600a2c00 0x7f04600a50c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:46.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:46.228+0000 7f047bffd640 1 --2- 172.21.15.172:0/2350105452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0474102ef0 0x7f04741a42e0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:46.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:46.228+0000 7f047bffd640 1 --2- 172.21.15.172:0/2350105452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0474102570 0x7f04741997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:46.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:46.228+0000 7f047bffd640 1 --2- 172.21.15.172:0/2350105452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0474100e20 0x7f0474199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:46.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:46.228+0000 7f047bffd640 1 -- 172.21.15.172:0/2350105452 >> 172.21.15.172:0/2350105452 conn(0x7f04740fc7d0 msgr2=0x7f04740fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:46.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:46.228+0000 7f047bffd640 1 -- 172.21.15.172:0/2350105452 shutdown_connections 2024-09-10T08:51:46.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:46.228+0000 7f047bffd640 1 -- 172.21.15.172:0/2350105452 wait complete. 2024-09-10T08:51:46.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:46 smithi172 ceph-mon[30854]: pgmap v516: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-10T08:51:46.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:46 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2350105452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:51:46.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:46 smithi174 ceph-mon[34068]: pgmap v516: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-10T08:51:46.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:46 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2350105452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:51:46.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:46 smithi191 ceph-mon[35314]: pgmap v516: 129 pgs: 129 active+clean; 3.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-10T08:51:46.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:46 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2350105452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:51:47.066 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T08:51:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.345+0000 7f66ceaa6640 1 -- 172.21.15.172:0/1392582714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66c81013d0 msgr2=0x7f66c8101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.345+0000 7f66ceaa6640 1 --2- 172.21.15.172:0/1392582714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66c81013d0 0x7f66c8101850 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009900 tx=0x7f66b402a3f0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.346+0000 7f66ceaa6640 1 -- 172.21.15.172:0/1392582714 shutdown_connections 2024-09-10T08:51:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.346+0000 7f66ceaa6640 1 --2- 172.21.15.172:0/1392582714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66c8101d90 0x7f66c8108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.346+0000 7f66ceaa6640 1 --2- 172.21.15.172:0/1392582714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66c81013d0 0x7f66c8101850 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.346+0000 7f66ceaa6640 1 --2- 172.21.15.172:0/1392582714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c81001e0 0x7f66c81005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.346+0000 7f66ceaa6640 1 -- 172.21.15.172:0/1392582714 >> 172.21.15.172:0/1392582714 conn(0x7f66c80fb940 msgr2=0x7f66c80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.346+0000 7f66ceaa6640 1 -- 172.21.15.172:0/1392582714 shutdown_connections 2024-09-10T08:51:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.346+0000 7f66ceaa6640 1 -- 172.21.15.172:0/1392582714 wait complete. 2024-09-10T08:51:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.346+0000 7f66ceaa6640 1 Processor -- start 2024-09-10T08:51:47.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.347+0000 7f66ceaa6640 1 -- start start 2024-09-10T08:51:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.347+0000 7f66ceaa6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c81001e0 0x7f66c80fd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.347+0000 7f66ceaa6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66c81013d0 0x7f66c819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.347+0000 7f66ceaa6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66c8101d90 0x7f66c819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.347+0000 7f66ceaa6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f66c810bda0 con 0x7f66c8101d90 2024-09-10T08:51:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.347+0000 7f66ceaa6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66c810bc20 con 0x7f66c81013d0 2024-09-10T08:51:47.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.347+0000 7f66ceaa6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f66c810bf20 con 0x7f66c81001e0 2024-09-10T08:51:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.347+0000 7f66cd043640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66c8101d90 0x7f66c819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.347+0000 7f66cd043640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66c8101d90 0x7f66c819ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52066/0 (socket says 172.21.15.172:52066) 2024-09-10T08:51:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.347+0000 7f66cd043640 1 -- 172.21.15.172:0/2112147375 learned_addr learned my addr 172.21.15.172:0/2112147375 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.347+0000 7f66cc842640 1 --2- 172.21.15.172:0/2112147375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c81001e0 0x7f66c80fd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.347+0000 7f66c7fff640 1 --2- 172.21.15.172:0/2112147375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66c81013d0 0x7f66c819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.347+0000 7f66cd043640 1 -- 172.21.15.172:0/2112147375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c81001e0 msgr2=0x7f66c80fd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.347+0000 7f66cd043640 1 --2- 172.21.15.172:0/2112147375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c81001e0 0x7f66c80fd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.347+0000 7f66cd043640 1 -- 172.21.15.172:0/2112147375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66c81013d0 msgr2=0x7f66c819c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.347+0000 7f66cd043640 1 --2- 172.21.15.172:0/2112147375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66c81013d0 0x7f66c819c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.347+0000 7f66cd043640 1 -- 172.21.15.172:0/2112147375 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c819f860 con 0x7f66c8101d90 2024-09-10T08:51:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.348+0000 7f66cd043640 1 --2- 172.21.15.172:0/2112147375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66c8101d90 0x7f66c819ef50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f66c000cce0 tx=0x7f66c000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:47.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.348+0000 7f66c5ffb640 1 -- 172.21.15.172:0/2112147375 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c000f450 con 0x7f66c8101d90 2024-09-10T08:51:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.348+0000 7f66c5ffb640 1 -- 172.21.15.172:0/2112147375 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66c000faa0 con 0x7f66c8101d90 2024-09-10T08:51:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.348+0000 7f66ceaa6640 1 -- 172.21.15.172:0/2112147375 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c819faf0 con 0x7f66c8101d90 2024-09-10T08:51:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.348+0000 7f66c5ffb640 1 -- 172.21.15.172:0/2112147375 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0018b70 con 0x7f66c8101d90 2024-09-10T08:51:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.348+0000 7f66ceaa6640 1 -- 172.21.15.172:0/2112147375 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c819ffc0 con 0x7f66c8101d90 2024-09-10T08:51:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.351+0000 7f66c5ffb640 1 -- 172.21.15.172:0/2112147375 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f66c0018d10 con 0x7f66c8101d90 2024-09-10T08:51:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.351+0000 7f66c5ffb640 1 --2- 172.21.15.172:0/2112147375 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66b80a2c30 0x7f66b80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.351+0000 7f66c5ffb640 1 -- 172.21.15.172:0/2112147375 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f66c00cf740 con 0x7f66c8101d90 2024-09-10T08:51:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.351+0000 7f66cc842640 1 --2- 172.21.15.172:0/2112147375 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66b80a2c30 0x7f66b80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.352+0000 7f66d183b740 1 Processor -- start 2024-09-10T08:51:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.352+0000 7f66d183b740 1 -- start start 2024-09-10T08:51:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.352+0000 7f66cc842640 1 --2- 172.21.15.172:0/2112147375 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66b80a2c30 0x7f66b80a50f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f66c8101230 tx=0x7f66bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.352+0000 7f66d183b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e95f9a0 0x555e0e95fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.352+0000 7f66d183b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e9602e0 0x555e0e964da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.352+0000 7f66d183b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e965370 0x555e0e967830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.352+0000 7f66d183b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555e0e8c32f0 con 0x555e0e95f9a0 2024-09-10T08:51:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.352+0000 7f66d183b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555e0e86b270 con 0x555e0e965370 2024-09-10T08:51:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.352+0000 7f66d183b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555e0e8bf2b0 con 0x555e0e9602e0 2024-09-10T08:51:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.352+0000 7f66cc842640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e95f9a0 0x555e0e95fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.352+0000 7f66cc842640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e95f9a0 0x555e0e95fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52084/0 (socket says 172.21.15.172:52084) 2024-09-10T08:51:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.352+0000 7f66cc842640 1 -- 172.21.15.172:0/1844510895 learned_addr learned my addr 172.21.15.172:0/1844510895 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.352+0000 7f66c7fff640 1 --2- 172.21.15.172:0/1844510895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e9602e0 0x555e0e964da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.353+0000 7f66cd043640 1 --2- 172.21.15.172:0/1844510895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e965370 0x555e0e967830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.353+0000 7f66cc842640 1 -- 172.21.15.172:0/1844510895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e9602e0 msgr2=0x555e0e964da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.353+0000 7f66cc842640 1 --2- 172.21.15.172:0/1844510895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e9602e0 0x555e0e964da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.353+0000 7f66cc842640 1 -- 172.21.15.172:0/1844510895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e965370 msgr2=0x555e0e967830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.353+0000 7f66cc842640 1 --2- 172.21.15.172:0/1844510895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e965370 0x555e0e967830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.353+0000 7f66cc842640 1 -- 172.21.15.172:0/1844510895 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e0e86a340 con 0x555e0e95f9a0 2024-09-10T08:51:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.353+0000 7f66c7fff640 1 --2- 172.21.15.172:0/1844510895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e9602e0 0x555e0e964da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:51:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.353+0000 7f66cd043640 1 --2- 172.21.15.172:0/1844510895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e965370 0x555e0e967830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:51:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.353+0000 7f66cc842640 1 --2- 172.21.15.172:0/1844510895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e95f9a0 0x555e0e95fda0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00ffa0 tx=0x7f66bc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2fc6a38276ce01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.353+0000 7f66b2ffd640 1 -- 172.21.15.172:0/1844510895 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc016070 con 0x555e0e95f9a0 2024-09-10T08:51:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.353+0000 7f66b2ffd640 1 -- 172.21.15.172:0/1844510895 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66bc01b8d0 con 0x555e0e95f9a0 2024-09-10T08:51:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.353+0000 7f66b2ffd640 1 -- 172.21.15.172:0/1844510895 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc01c8e0 con 0x555e0e95f9a0 2024-09-10T08:51:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.354+0000 7f66d183b740 1 -- 172.21.15.172:0/1844510895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e95f9a0 msgr2=0x555e0e95fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.354+0000 7f66d183b740 1 --2- 172.21.15.172:0/1844510895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e95f9a0 0x555e0e95fda0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00ffa0 tx=0x7f66bc012000 comp rx=0 tx=0).stop 2024-09-10T08:51:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.354+0000 7f66d183b740 1 -- 172.21.15.172:0/1844510895 shutdown_connections 2024-09-10T08:51:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.354+0000 7f66d183b740 1 --2- 172.21.15.172:0/1844510895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e965370 0x555e0e967830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.354+0000 7f66d183b740 1 --2- 172.21.15.172:0/1844510895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e9602e0 0x555e0e964da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.354+0000 7f66d183b740 1 --2- 172.21.15.172:0/1844510895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e95f9a0 0x555e0e95fda0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.354+0000 7f66d183b740 1 -- 172.21.15.172:0/1844510895 >> 172.21.15.172:0/1844510895 conn(0x555e0e95b010 msgr2=0x555e0e95b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:47.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.354+0000 7f66d183b740 1 -- 172.21.15.172:0/1844510895 shutdown_connections 2024-09-10T08:51:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.354+0000 7f66d183b740 1 -- 172.21.15.172:0/1844510895 wait complete. 2024-09-10T08:51:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.355+0000 7f66d183b740 1 -- start start 2024-09-10T08:51:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.355+0000 7f66d183b740 1 Processor -- start 2024-09-10T08:51:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.355+0000 7f66d183b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e95f9a0 0x555e0e969ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.355+0000 7f66d183b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e9602e0 0x555e0e96a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.355+0000 7f66d183b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e965370 0x555e0e971120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.355+0000 7f66d183b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555e0e968340 con 0x555e0e9602e0 2024-09-10T08:51:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.355+0000 7f66d183b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555e0e84c290 con 0x555e0e95f9a0 2024-09-10T08:51:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.355+0000 7f66d183b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555e0e9684c0 con 0x555e0e965370 2024-09-10T08:51:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.355+0000 7f66d183b740 10 client.0 _ll_register_callbacks cb 0x555e0e87d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:47.364 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.355+0000 7f66c7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e9602e0 0x555e0e96a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.355+0000 7f66c7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e9602e0 0x555e0e96a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52096/0 (socket says 172.21.15.172:52096) 2024-09-10T08:51:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.355+0000 7f66c7fff640 1 -- 172.21.15.172:0/2807826325 learned_addr learned my addr 172.21.15.172:0/2807826325 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.356+0000 7f66cd043640 1 --2- 172.21.15.172:0/2807826325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e965370 0x555e0e971120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.356+0000 7f66cc842640 1 --2- 172.21.15.172:0/2807826325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e95f9a0 0x555e0e969ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.356+0000 7f66c7fff640 1 -- 172.21.15.172:0/2807826325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e965370 msgr2=0x555e0e971120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.356+0000 7f66c7fff640 1 --2- 172.21.15.172:0/2807826325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e965370 0x555e0e971120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.356+0000 7f66c7fff640 1 -- 172.21.15.172:0/2807826325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e95f9a0 msgr2=0x555e0e969ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.356+0000 7f66c7fff640 1 --2- 172.21.15.172:0/2807826325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e95f9a0 0x555e0e969ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.356+0000 7f66c7fff640 1 -- 172.21.15.172:0/2807826325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e0e5d6210 con 0x555e0e9602e0 2024-09-10T08:51:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.356+0000 7f66cc842640 1 --2- 172.21.15.172:0/2807826325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e95f9a0 0x555e0e969ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:51:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.356+0000 7f66c7fff640 1 --2- 172.21.15.172:0/2807826325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e9602e0 0x555e0e96a0e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009900 tx=0x7f66b4031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794672c21b1a025 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.356+0000 7f66a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.356+0000 7f66a7fff640 1 -- 172.21.15.172:0/2807826325 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4035070 con 0x555e0e9602e0 2024-09-10T08:51:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.356+0000 7f66a7fff640 1 -- 172.21.15.172:0/2807826325 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f66b4004250 con 0x555e0e9602e0 2024-09-10T08:51:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.356+0000 7f66d183b740 1 -- 172.21.15.172:0/2807826325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66bc01b8d0 con 0x555e0e9602e0 2024-09-10T08:51:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.356+0000 7f66a7fff640 1 -- 172.21.15.172:0/2807826325 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f66b4004450 con 0x555e0e9602e0 2024-09-10T08:51:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.356+0000 7f66d183b740 10 client.17052 fetch_fsmap learned FSMap version 35 2024-09-10T08:51:47.366 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.356+0000 7f66d183b740 1 -- 172.21.15.172:0/2807826325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555e0e6a2620 con 0x555e0e9602e0 2024-09-10T08:51:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.357+0000 7f66a7fff640 1 -- 172.21.15.172:0/2807826325 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66bc01b8d0 con 0x555e0e9602e0 2024-09-10T08:51:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.357+0000 7f66a7fff640 1 -- 172.21.15.172:0/2807826325 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f66b4045880 con 0x555e0e9602e0 2024-09-10T08:51:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.357+0000 7f66d183b740 10 client.17052 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T08:51:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.357+0000 7f66d183b740 10 client.17052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:51:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.357+0000 7f66d183b740 20 client.17052 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.358+0000 7f66d183b740 1 --2- 172.21.15.172:0/2807826325 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555e0e979c70 0x555e0e97c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.358+0000 7f66d183b740 4 client.17052 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:51:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.358+0000 7f66d183b740 1 -- 172.21.15.172:0/2807826325 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e0e8d0a20 con 0x555e0e979c70 2024-09-10T08:51:47.367 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.358+0000 7f66cc842640 1 --2- 172.21.15.172:0/2807826325 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555e0e979c70 0x555e0e97c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.358+0000 7f66a7fff640 10 client.17052 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.358+0000 7f66cc842640 1 --2- 172.21.15.172:0/2807826325 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555e0e979c70 0x555e0e97c0a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba216d6ce8ab349f server_cookie=1fcc334baf471a3d in_seq=0 out_seq=0 2024-09-10T08:51:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.359+0000 7f66a7fff640 1 -- 172.21.15.172:0/2807826325 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e0e8d0a20 con 0x555e0e979c70 2024-09-10T08:51:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.359+0000 7f66a7fff640 10 client.17052 handle_command_reply: tid=0 2024-09-10T08:51:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.359+0000 7f66d183b740 1 client.17052 shutdown 2024-09-10T08:51:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.359+0000 7f66d183b740 1 -- 172.21.15.172:0/2807826325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e9602e0 msgr2=0x555e0e96a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.359+0000 7f66d183b740 1 --2- 172.21.15.172:0/2807826325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e9602e0 0x555e0e96a0e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009900 tx=0x7f66b4031a90 comp rx=0 tx=0).stop 2024-09-10T08:51:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.359+0000 7f66d183b740 1 -- 172.21.15.172:0/2807826325 shutdown_connections 2024-09-10T08:51:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.359+0000 7f66d183b740 1 --2- 172.21.15.172:0/2807826325 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555e0e979c70 0x555e0e97c0a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.359+0000 7f66d183b740 1 --2- 172.21.15.172:0/2807826325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e965370 0x555e0e971120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.359+0000 7f66d183b740 1 --2- 172.21.15.172:0/2807826325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e9602e0 0x555e0e96a0e0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.359+0000 7f66d183b740 1 --2- 172.21.15.172:0/2807826325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e95f9a0 0x555e0e969ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.359+0000 7f66a7fff640 0 client.17052 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.359+0000 7f66d183b740 1 -- 172.21.15.172:0/2807826325 >> 172.21.15.172:0/2807826325 conn(0x555e0e95b010 msgr2=0x555e0e95c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.359+0000 7f66d183b740 1 -- 172.21.15.172:0/2807826325 shutdown_connections 2024-09-10T08:51:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.359+0000 7f66d183b740 1 -- 172.21.15.172:0/2807826325 wait complete. 2024-09-10T08:51:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.360+0000 7f66d183b740 20 client.17052 trim_cache size 0 max 16384 2024-09-10T08:51:47.369 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:51:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.376+0000 7f66d183b740 1 Processor -- start 2024-09-10T08:51:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.376+0000 7f66d183b740 1 -- start start 2024-09-10T08:51:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.376+0000 7f66d183b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e95f9a0 0x555e0e964ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.376+0000 7f66d183b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e9602e0 0x555e0e9710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.376+0000 7f66d183b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e965370 0x555e0e95a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.376+0000 7f66d183b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f66c81a8080 con 0x555e0e9602e0 2024-09-10T08:51:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.376+0000 7f66d183b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66c81a9c20 con 0x555e0e965370 2024-09-10T08:51:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.376+0000 7f66d183b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f66c81a8750 con 0x555e0e95f9a0 2024-09-10T08:51:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.376+0000 7f66c7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e9602e0 0x555e0e9710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.376+0000 7f66c7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e9602e0 0x555e0e9710e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52110/0 (socket says 172.21.15.172:52110) 2024-09-10T08:51:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.376+0000 7f66c7fff640 1 -- 172.21.15.172:0/3131268318 learned_addr learned my addr 172.21.15.172:0/3131268318 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.377+0000 7f66cc842640 1 --2- 172.21.15.172:0/3131268318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e95f9a0 0x555e0e964ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.377+0000 7f66cd043640 1 --2- 172.21.15.172:0/3131268318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e965370 0x555e0e95a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.377+0000 7f66cc842640 1 -- 172.21.15.172:0/3131268318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e965370 msgr2=0x555e0e95a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.377+0000 7f66cc842640 1 --2- 172.21.15.172:0/3131268318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e965370 0x555e0e95a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.377+0000 7f66cc842640 1 -- 172.21.15.172:0/3131268318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e9602e0 msgr2=0x555e0e9710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.377+0000 7f66cc842640 1 --2- 172.21.15.172:0/3131268318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e9602e0 0x555e0e9710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.377+0000 7f66cc842640 1 -- 172.21.15.172:0/3131268318 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e0e9552f0 con 0x555e0e95f9a0 2024-09-10T08:51:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.377+0000 7f66c7fff640 1 --2- 172.21.15.172:0/3131268318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e9602e0 0x555e0e9710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:51:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.377+0000 7f66cc842640 1 --2- 172.21.15.172:0/3131268318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e95f9a0 0x555e0e964ae0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x555e0e9690d0 tx=0x7f66bc00af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36b701ec76e34f49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.377+0000 7f66b27fc640 1 -- 172.21.15.172:0/3131268318 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc034070 con 0x555e0e95f9a0 2024-09-10T08:51:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.377+0000 7f66b27fc640 1 -- 172.21.15.172:0/3131268318 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66bc005500 con 0x555e0e95f9a0 2024-09-10T08:51:47.381 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.377+0000 7f66b27fc640 1 -- 172.21.15.172:0/3131268318 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc016030 con 0x555e0e95f9a0 2024-09-10T08:51:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.378+0000 7f66d183b740 1 -- 172.21.15.172:0/3131268318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e95f9a0 msgr2=0x555e0e964ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.378+0000 7f66d183b740 1 --2- 172.21.15.172:0/3131268318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e95f9a0 0x555e0e964ae0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x555e0e9690d0 tx=0x7f66bc00af90 comp rx=0 tx=0).stop 2024-09-10T08:51:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.378+0000 7f66d183b740 1 -- 172.21.15.172:0/3131268318 shutdown_connections 2024-09-10T08:51:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.378+0000 7f66d183b740 1 --2- 172.21.15.172:0/3131268318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e965370 0x555e0e95a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.378+0000 7f66d183b740 1 --2- 172.21.15.172:0/3131268318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e9602e0 0x555e0e9710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.378+0000 7f66d183b740 1 --2- 172.21.15.172:0/3131268318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e95f9a0 0x555e0e964ae0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.378+0000 7f66d183b740 1 -- 172.21.15.172:0/3131268318 >> 172.21.15.172:0/3131268318 conn(0x555e0e95b010 msgr2=0x555e0e9599a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.378+0000 7f66d183b740 1 -- 172.21.15.172:0/3131268318 shutdown_connections 2024-09-10T08:51:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.378+0000 7f66d183b740 1 -- 172.21.15.172:0/3131268318 wait complete. 2024-09-10T08:51:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.379+0000 7f66d183b740 1 -- start start 2024-09-10T08:51:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.379+0000 7f66d183b740 1 Processor -- start 2024-09-10T08:51:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.379+0000 7f66d183b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e95f9a0 0x555e0e96ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.379+0000 7f66d183b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e9602e0 0x555e0e96f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.379+0000 7f66d183b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e965370 0x555e0e9701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.379+0000 7f66d183b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f66c81a9520 con 0x555e0e95f9a0 2024-09-10T08:51:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.379+0000 7f66d183b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66c81a8e50 con 0x555e0e965370 2024-09-10T08:51:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.379+0000 7f66d183b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f66c81a6060 con 0x555e0e9602e0 2024-09-10T08:51:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.379+0000 7f66d183b740 10 client.0 _ll_register_callbacks cb 0x7f66b4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.379+0000 7f66cc842640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e95f9a0 0x555e0e96ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.379+0000 7f66c7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e9602e0 0x555e0e96f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.379+0000 7f66cc842640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e95f9a0 0x555e0e96ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52118/0 (socket says 172.21.15.172:52118) 2024-09-10T08:51:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.379+0000 7f66cc842640 1 -- 172.21.15.172:0/74461421 learned_addr learned my addr 172.21.15.172:0/74461421 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.379+0000 7f66cd043640 1 --2- 172.21.15.172:0/74461421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e965370 0x555e0e9701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.380+0000 7f66c7fff640 1 -- 172.21.15.172:0/74461421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e965370 msgr2=0x555e0e9701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.380+0000 7f66c7fff640 1 --2- 172.21.15.172:0/74461421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e965370 0x555e0e9701b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.380+0000 7f66c7fff640 1 -- 172.21.15.172:0/74461421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e95f9a0 msgr2=0x555e0e96ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.380+0000 7f66c7fff640 1 --2- 172.21.15.172:0/74461421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e95f9a0 0x555e0e96ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.380+0000 7f66c7fff640 1 -- 172.21.15.172:0/74461421 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e0e9707c0 con 0x555e0e9602e0 2024-09-10T08:51:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.380+0000 7f66cc842640 1 --2- 172.21.15.172:0/74461421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e95f9a0 0x555e0e96ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:51:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.380+0000 7f66c7fff640 1 --2- 172.21.15.172:0/74461421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e9602e0 0x555e0e96f1a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f66b4005db0 tx=0x7f66b4045940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c29ad46f7b79ced6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.380+0000 7f66a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:51:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.380+0000 7f66a7fff640 1 -- 172.21.15.172:0/74461421 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4056020 con 0x555e0e9602e0 2024-09-10T08:51:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.380+0000 7f66d183b740 1 -- 172.21.15.172:0/74461421 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66bc005500 con 0x555e0e9602e0 2024-09-10T08:51:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.380+0000 7f66a7fff640 1 -- 172.21.15.172:0/74461421 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f66b403c9b0 con 0x555e0e9602e0 2024-09-10T08:51:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.380+0000 7f66a7fff640 1 -- 172.21.15.172:0/74461421 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f66b4035050 con 0x555e0e9602e0 2024-09-10T08:51:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.380+0000 7f66d183b740 10 client.26170 fetch_fsmap learned FSMap version 35 2024-09-10T08:51:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.380+0000 7f66d183b740 1 -- 172.21.15.172:0/74461421 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555e0e9e4ea0 con 0x555e0e9602e0 2024-09-10T08:51:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.380+0000 7f66a7fff640 1 -- 172.21.15.172:0/74461421 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66bc005500 con 0x555e0e9602e0 2024-09-10T08:51:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.381+0000 7f66a7fff640 1 -- 172.21.15.172:0/74461421 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f66b40528a0 con 0x555e0e9602e0 2024-09-10T08:51:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.381+0000 7f66d183b740 10 client.26170 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T08:51:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.381+0000 7f66d183b740 10 client.26170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:51:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.381+0000 7f66d183b740 20 client.26170 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.381+0000 7f66d183b740 1 --2- 172.21.15.172:0/74461421 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555e0e9e5900 0x555e0e9e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.381+0000 7f66d183b740 4 client.26170 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:51:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.381+0000 7f66d183b740 1 -- 172.21.15.172:0/74461421 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e0e8cad70 con 0x555e0e9e5900 2024-09-10T08:51:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.381+0000 7f66cc842640 1 --2- 172.21.15.172:0/74461421 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555e0e9e5900 0x555e0e9e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.381+0000 7f66cc842640 1 --2- 172.21.15.172:0/74461421 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555e0e9e5900 0x555e0e9e7d20 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=203da01edac088af server_cookie=9a45292efc135e5f in_seq=0 out_seq=0 2024-09-10T08:51:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.381+0000 7f66a7fff640 10 client.26170 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.387+0000 7f66a7fff640 1 -- 172.21.15.172:0/74461421 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e0e8cad70 con 0x555e0e9e5900 2024-09-10T08:51:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.387+0000 7f66a7fff640 10 client.26170 handle_command_reply: tid=0 2024-09-10T08:51:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.387+0000 7f66d183b740 1 client.26170 shutdown 2024-09-10T08:51:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.387+0000 7f66d183b740 1 -- 172.21.15.172:0/74461421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e9602e0 msgr2=0x555e0e96f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.387+0000 7f66d183b740 1 --2- 172.21.15.172:0/74461421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e9602e0 0x555e0e96f1a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f66b4005db0 tx=0x7f66b4045940 comp rx=0 tx=0).stop 2024-09-10T08:51:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.387+0000 7f66d183b740 1 -- 172.21.15.172:0/74461421 shutdown_connections 2024-09-10T08:51:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.387+0000 7f66d183b740 1 --2- 172.21.15.172:0/74461421 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555e0e9e5900 0x555e0e9e7d20 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.387+0000 7f66d183b740 1 --2- 172.21.15.172:0/74461421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e0e965370 0x555e0e9701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.387+0000 7f66a7fff640 0 client.26170 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:51:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.387+0000 7f66d183b740 1 --2- 172.21.15.172:0/74461421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e0e9602e0 0x555e0e96f1a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.387+0000 7f66d183b740 1 --2- 172.21.15.172:0/74461421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e0e95f9a0 0x555e0e96ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.387+0000 7f66d183b740 1 -- 172.21.15.172:0/74461421 >> 172.21.15.172:0/74461421 conn(0x555e0e95b010 msgr2=0x555e0e95bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.387+0000 7f66d183b740 1 -- 172.21.15.172:0/74461421 shutdown_connections 2024-09-10T08:51:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.387+0000 7f66d183b740 1 -- 172.21.15.172:0/74461421 wait complete. 2024-09-10T08:51:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.388+0000 7f66d183b740 20 client.26170 trim_cache size 0 max 16384 2024-09-10T08:51:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:51:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:51:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:51:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:51:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.388+0000 7f66ceaa6640 1 -- 172.21.15.172:0/2112147375 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66b80a2c30 msgr2=0x7f66b80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.388+0000 7f66ceaa6640 1 --2- 172.21.15.172:0/2112147375 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66b80a2c30 0x7f66b80a50f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f66c8101230 tx=0x7f66bc005f90 comp rx=0 tx=0).stop 2024-09-10T08:51:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.388+0000 7f66ceaa6640 1 -- 172.21.15.172:0/2112147375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66c8101d90 msgr2=0x7f66c819ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.388+0000 7f66ceaa6640 1 --2- 172.21.15.172:0/2112147375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66c8101d90 0x7f66c819ef50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f66c000cce0 tx=0x7f66c000a620 comp rx=0 tx=0).stop 2024-09-10T08:51:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.388+0000 7f66ceaa6640 1 -- 172.21.15.172:0/2112147375 shutdown_connections 2024-09-10T08:51:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.388+0000 7f66ceaa6640 1 --2- 172.21.15.172:0/2112147375 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66b80a2c30 0x7f66b80a50f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.388+0000 7f66ceaa6640 1 --2- 172.21.15.172:0/2112147375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66c8101d90 0x7f66c819ef50 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.388+0000 7f66ceaa6640 1 --2- 172.21.15.172:0/2112147375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66c81013d0 0x7f66c819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.388+0000 7f66ceaa6640 1 --2- 172.21.15.172:0/2112147375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66c81001e0 0x7f66c80fd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.388+0000 7f66ceaa6640 1 -- 172.21.15.172:0/2112147375 >> 172.21.15.172:0/2112147375 conn(0x7f66c80fb940 msgr2=0x7f66c8104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:47.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.388+0000 7f66ceaa6640 1 -- 172.21.15.172:0/2112147375 shutdown_connections 2024-09-10T08:51:47.394 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:51:47.388+0000 7f66ceaa6640 1 -- 172.21.15.172:0/2112147375 wait complete. 2024-09-10T08:51:47.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.964+0000 7fcea9685640 1 -- 172.21.15.172:0/3798460168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcea4102ef0 msgr2=0x7fcea410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:47.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.964+0000 7fcea9685640 1 --2- 172.21.15.172:0/3798460168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcea4102ef0 0x7fcea410da40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009ca0 tx=0x7fce9c02a950 comp rx=0 tx=0).stop 2024-09-10T08:51:47.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.964+0000 7fcea9685640 1 -- 172.21.15.172:0/3798460168 shutdown_connections 2024-09-10T08:51:47.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.964+0000 7fcea9685640 1 --2- 172.21.15.172:0/3798460168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcea4102ef0 0x7fcea410da40 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.964+0000 7fcea9685640 1 --2- 172.21.15.172:0/3798460168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcea4102570 0x7fcea41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.965+0000 7fcea9685640 1 --2- 172.21.15.172:0/3798460168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcea4100e20 0x7fcea4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.965+0000 7fcea9685640 1 -- 172.21.15.172:0/3798460168 >> 172.21.15.172:0/3798460168 conn(0x7fcea40fc7d0 msgr2=0x7fcea40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:47.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.965+0000 7fcea9685640 1 -- 172.21.15.172:0/3798460168 shutdown_connections 2024-09-10T08:51:47.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.965+0000 7fcea9685640 1 -- 172.21.15.172:0/3798460168 wait complete. 2024-09-10T08:51:47.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.965+0000 7fcea9685640 1 Processor -- start 2024-09-10T08:51:47.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.965+0000 7fcea9685640 1 -- start start 2024-09-10T08:51:47.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fcea9685640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcea4100e20 0x7fcea41992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fcea9685640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcea4102570 0x7fcea41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fcea9685640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcea4102ef0 0x7fcea41a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fcea9685640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcea4110e30 con 0x7fcea4102ef0 2024-09-10T08:51:47.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fcea9685640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcea4110cb0 con 0x7fcea4102570 2024-09-10T08:51:47.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fcea9685640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcea4110fb0 con 0x7fcea4100e20 2024-09-10T08:51:47.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcea4102ef0 0x7fcea41a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fcea37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcea4102ef0 0x7fcea41a4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52134/0 (socket says 172.21.15.172:52134) 2024-09-10T08:51:47.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fcea37fe640 1 -- 172.21.15.172:0/128826919 learned_addr learned my addr 172.21.15.172:0/128826919 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:47.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fcea2ffd640 1 --2- 172.21.15.172:0/128826919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcea4100e20 0x7fcea41992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fcea27fc640 1 --2- 172.21.15.172:0/128826919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcea4102570 0x7fcea41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fcea37fe640 1 -- 172.21.15.172:0/128826919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcea4100e20 msgr2=0x7fcea41992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:47.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fcea37fe640 1 --2- 172.21.15.172:0/128826919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcea4100e20 0x7fcea41992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fcea37fe640 1 -- 172.21.15.172:0/128826919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcea4102570 msgr2=0x7fcea41997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:47.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fcea37fe640 1 --2- 172.21.15.172:0/128826919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcea4102570 0x7fcea41997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fcea37fe640 1 -- 172.21.15.172:0/128826919 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea41a4860 con 0x7fcea4102ef0 2024-09-10T08:51:47.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fcea37fe640 1 --2- 172.21.15.172:0/128826919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcea4102ef0 0x7fcea41a4320 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fce9c02ae70 tx=0x7fce9c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:47.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.966+0000 7fce8bfff640 1 -- 172.21.15.172:0/128826919 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c034040 con 0x7fcea4102ef0 2024-09-10T08:51:47.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.967+0000 7fcea9685640 1 -- 172.21.15.172:0/128826919 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea4197090 con 0x7fcea4102ef0 2024-09-10T08:51:47.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.967+0000 7fce8bfff640 1 -- 172.21.15.172:0/128826919 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce9c0308f0 con 0x7fcea4102ef0 2024-09-10T08:51:47.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.967+0000 7fce8bfff640 1 -- 172.21.15.172:0/128826919 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c004780 con 0x7fcea4102ef0 2024-09-10T08:51:47.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.967+0000 7fcea9685640 1 -- 172.21.15.172:0/128826919 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea41975c0 con 0x7fcea4102ef0 2024-09-10T08:51:47.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.969+0000 7fce8bfff640 1 -- 172.21.15.172:0/128826919 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fce9c037020 con 0x7fcea4102ef0 2024-09-10T08:51:47.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.969+0000 7fceaba5e740 1 Processor -- start 2024-09-10T08:51:47.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fceaba5e740 1 -- start start 2024-09-10T08:51:47.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fceaba5e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a728962b40 0x55a728961090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fceaba5e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7289615d0 0x55a728961a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fceaba5e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a72896c370 0x55a72896e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fceaba5e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a7288c72f0 con 0x55a72896c370 2024-09-10T08:51:47.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fceaba5e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a72886ecf0 con 0x55a728962b40 2024-09-10T08:51:47.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fceaba5e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a7288c32b0 con 0x55a7289615d0 2024-09-10T08:51:47.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fcea27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a72896c370 0x55a72896e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fcea27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a72896c370 0x55a72896e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52144/0 (socket says 172.21.15.172:52144) 2024-09-10T08:51:47.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fcea2ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7289615d0 0x55a728961a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fcea27fc640 1 -- 172.21.15.172:0/2725641551 learned_addr learned my addr 172.21.15.172:0/2725641551 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:47.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fce8bfff640 1 --2- 172.21.15.172:0/128826919 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fce8c0a2b80 0x7fce8c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fcea27fc640 1 --2- 172.21.15.172:0/2725641551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a728962b40 0x55a728961090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fcea37fe640 1 --2- 172.21.15.172:0/128826919 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fce8c0a2b80 0x7fce8c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fcea27fc640 1 -- 172.21.15.172:0/2725641551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7289615d0 msgr2=0x55a728961a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:47.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fcea27fc640 1 --2- 172.21.15.172:0/2725641551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7289615d0 0x55a728961a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fcea27fc640 1 -- 172.21.15.172:0/2725641551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a728962b40 msgr2=0x55a728961090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:47.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fcea27fc640 1 --2- 172.21.15.172:0/2725641551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a728962b40 0x55a728961090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.970+0000 7fcea27fc640 1 -- 172.21.15.172:0/2725641551 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7288533a0 con 0x55a72896c370 2024-09-10T08:51:47.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.971+0000 7fce8bfff640 1 -- 172.21.15.172:0/128826919 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fce9c0f0440 con 0x7fcea4102ef0 2024-09-10T08:51:47.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.971+0000 7fcea27fc640 1 --2- 172.21.15.172:0/2725641551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a72896c370 0x55a72896e760 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fce9801b5b0 tx=0x7fce98011040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debdad3b727cfaaa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:47.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.971+0000 7fce897fa640 1 -- 172.21.15.172:0/2725641551 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9801e420 con 0x55a72896c370 2024-09-10T08:51:47.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.971+0000 7fcea37fe640 1 --2- 172.21.15.172:0/128826919 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fce8c0a2b80 0x7fce8c0a5040 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0bc8d0 tx=0x7fce9c08fa00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:47.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.971+0000 7fce897fa640 1 -- 172.21.15.172:0/2725641551 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce98023070 con 0x55a72896c370 2024-09-10T08:51:47.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.971+0000 7fce897fa640 1 -- 172.21.15.172:0/2725641551 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9801d740 con 0x55a72896c370 2024-09-10T08:51:47.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.972+0000 7fceaba5e740 1 -- 172.21.15.172:0/2725641551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a72896c370 msgr2=0x55a72896e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:47.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.972+0000 7fceaba5e740 1 --2- 172.21.15.172:0/2725641551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a72896c370 0x55a72896e760 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fce9801b5b0 tx=0x7fce98011040 comp rx=0 tx=0).stop 2024-09-10T08:51:47.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.972+0000 7fceaba5e740 1 -- 172.21.15.172:0/2725641551 shutdown_connections 2024-09-10T08:51:47.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.972+0000 7fceaba5e740 1 --2- 172.21.15.172:0/2725641551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a72896c370 0x55a72896e760 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.972+0000 7fceaba5e740 1 --2- 172.21.15.172:0/2725641551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7289615d0 0x55a728961a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.972+0000 7fceaba5e740 1 --2- 172.21.15.172:0/2725641551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a728962b40 0x55a728961090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.972+0000 7fceaba5e740 1 -- 172.21.15.172:0/2725641551 >> 172.21.15.172:0/2725641551 conn(0x55a72892a310 msgr2=0x55a72895dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:47.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.972+0000 7fceaba5e740 1 -- 172.21.15.172:0/2725641551 shutdown_connections 2024-09-10T08:51:47.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.972+0000 7fceaba5e740 1 -- 172.21.15.172:0/2725641551 wait complete. 2024-09-10T08:51:47.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.972+0000 7fceaba5e740 1 -- start start 2024-09-10T08:51:47.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fceaba5e740 1 Processor -- start 2024-09-10T08:51:47.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fceaba5e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7289615d0 0x55a72896bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fceaba5e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a728962b40 0x55a728969030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fceaba5e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72896c370 0x55a728969640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fceaba5e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a72896f070 con 0x55a728962b40 2024-09-10T08:51:47.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fceaba5e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a728882060 con 0x55a72896c370 2024-09-10T08:51:47.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fceaba5e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a72896f1f0 con 0x55a7289615d0 2024-09-10T08:51:47.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fceaba5e740 10 client.0 _ll_register_callbacks cb 0x55a7285d58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:47.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fcea2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a728962b40 0x55a728969030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fcea2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a728962b40 0x55a728969030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52148/0 (socket says 172.21.15.172:52148) 2024-09-10T08:51:47.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fcea27fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7289615d0 0x55a72896bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fcea2ffd640 1 -- 172.21.15.172:0/3812796806 learned_addr learned my addr 172.21.15.172:0/3812796806 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:47.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fcea27fc640 1 --2- 172.21.15.172:0/3812796806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72896c370 0x55a728969640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fcea2ffd640 1 -- 172.21.15.172:0/3812796806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7289615d0 msgr2=0x55a72896bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:47.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fcea2ffd640 1 --2- 172.21.15.172:0/3812796806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7289615d0 0x55a72896bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fcea2ffd640 1 -- 172.21.15.172:0/3812796806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72896c370 msgr2=0x55a728969640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:47.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fcea2ffd640 1 --2- 172.21.15.172:0/3812796806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72896c370 0x55a728969640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fcea2ffd640 1 -- 172.21.15.172:0/3812796806 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7286383c0 con 0x55a728962b40 2024-09-10T08:51:47.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.973+0000 7fcea27fc640 1 --2- 172.21.15.172:0/3812796806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7289615d0 0x55a72896bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:51:47.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.974+0000 7fcea2ffd640 1 --2- 172.21.15.172:0/3812796806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a728962b40 0x55a728969030 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fce9400bad0 tx=0x7fce9400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589a3e508a7a9640 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:47.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.974+0000 7fce86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:47.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.974+0000 7fce86ff5640 1 -- 172.21.15.172:0/3812796806 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94005020 con 0x55a728962b40 2024-09-10T08:51:47.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.974+0000 7fce86ff5640 1 -- 172.21.15.172:0/3812796806 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fce940051c0 con 0x55a728962b40 2024-09-10T08:51:47.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.974+0000 7fceaba5e740 1 -- 172.21.15.172:0/3812796806 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce98023070 con 0x55a728962b40 2024-09-10T08:51:47.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.974+0000 7fce86ff5640 1 -- 172.21.15.172:0/3812796806 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fce94005400 con 0x55a728962b40 2024-09-10T08:51:47.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.974+0000 7fceaba5e740 10 client.17070 fetch_fsmap learned FSMap version 35 2024-09-10T08:51:47.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.974+0000 7fceaba5e740 1 -- 172.21.15.172:0/3812796806 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a72897abd0 con 0x55a728962b40 2024-09-10T08:51:47.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.974+0000 7fce86ff5640 1 -- 172.21.15.172:0/3812796806 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce98023070 con 0x55a728962b40 2024-09-10T08:51:47.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.975+0000 7fce86ff5640 1 -- 172.21.15.172:0/3812796806 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fce9401b6a0 con 0x55a728962b40 2024-09-10T08:51:47.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.975+0000 7fceaba5e740 10 client.17070 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T08:51:47.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.975+0000 7fceaba5e740 10 client.17070 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:51:47.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.975+0000 7fceaba5e740 20 client.17070 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:47.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.975+0000 7fceaba5e740 1 --2- 172.21.15.172:0/3812796806 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55a72897c390 0x55a72897e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.975+0000 7fceaba5e740 4 client.17070 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:51:47.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.975+0000 7fceaba5e740 1 -- 172.21.15.172:0/3812796806 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7288d4860 con 0x55a72897c390 2024-09-10T08:51:47.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.975+0000 7fcea27fc640 1 --2- 172.21.15.172:0/3812796806 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55a72897c390 0x55a72897e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.986 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.975+0000 7fce86ff5640 10 client.17070 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:51:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.975+0000 7fcea27fc640 1 --2- 172.21.15.172:0/3812796806 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55a72897c390 0x55a72897e7b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c46ed1d07d69b1c0 server_cookie=5d5b5d1f3d116ff7 in_seq=0 out_seq=0 2024-09-10T08:51:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.976+0000 7fce86ff5640 1 -- 172.21.15.172:0/3812796806 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7288d4860 con 0x55a72897c390 2024-09-10T08:51:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.976+0000 7fce86ff5640 10 client.17070 handle_command_reply: tid=0 2024-09-10T08:51:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.976+0000 7fceaba5e740 1 client.17070 shutdown 2024-09-10T08:51:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.976+0000 7fceaba5e740 1 -- 172.21.15.172:0/3812796806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a728962b40 msgr2=0x55a728969030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.976+0000 7fceaba5e740 1 --2- 172.21.15.172:0/3812796806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a728962b40 0x55a728969030 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fce9400bad0 tx=0x7fce9400bed0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.976+0000 7fceaba5e740 1 -- 172.21.15.172:0/3812796806 shutdown_connections 2024-09-10T08:51:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.976+0000 7fceaba5e740 1 --2- 172.21.15.172:0/3812796806 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55a72897c390 0x55a72897e7b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.976+0000 7fceaba5e740 1 --2- 172.21.15.172:0/3812796806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72896c370 0x55a728969640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.976+0000 7fceaba5e740 1 --2- 172.21.15.172:0/3812796806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a728962b40 0x55a728969030 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.976+0000 7fce86ff5640 0 client.17070 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:51:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.976+0000 7fceaba5e740 1 --2- 172.21.15.172:0/3812796806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7289615d0 0x55a72896bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.987 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.976+0000 7fceaba5e740 1 -- 172.21.15.172:0/3812796806 >> 172.21.15.172:0/3812796806 conn(0x55a72892a310 msgr2=0x55a728963960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:47.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.976+0000 7fceaba5e740 1 -- 172.21.15.172:0/3812796806 shutdown_connections 2024-09-10T08:51:47.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.976+0000 7fceaba5e740 1 -- 172.21.15.172:0/3812796806 wait complete. 2024-09-10T08:51:47.988 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.977+0000 7fceaba5e740 20 client.17070 trim_cache size 0 max 16384 2024-09-10T08:51:47.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.989+0000 7fceaba5e740 1 Processor -- start 2024-09-10T08:51:47.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.989+0000 7fceaba5e740 1 -- start start 2024-09-10T08:51:47.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.989+0000 7fceaba5e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7289615d0 0x55a728969000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.989+0000 7fceaba5e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72896c370 0x55a728978340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.993 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.990+0000 7fceaba5e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72895df20 0x55a728958ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.990+0000 7fceaba5e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41ac930 con 0x55a7289615d0 2024-09-10T08:51:47.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.990+0000 7fceaba5e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41ae4d0 con 0x55a72896c370 2024-09-10T08:51:47.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.990+0000 7fceaba5e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41ad000 con 0x55a72895df20 2024-09-10T08:51:47.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.990+0000 7fcea27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7289615d0 0x55a728969000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.990+0000 7fcea27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7289615d0 0x55a728969000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52150/0 (socket says 172.21.15.172:52150) 2024-09-10T08:51:47.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.990+0000 7fcea27fc640 1 -- 172.21.15.172:0/84615827 learned_addr learned my addr 172.21.15.172:0/84615827 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:47.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.990+0000 7fcea2ffd640 1 --2- 172.21.15.172:0/84615827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72896c370 0x55a728978340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.990+0000 7fcea27fc640 1 --2- 172.21.15.172:0/84615827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72895df20 0x55a728958ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:47.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.990+0000 7fcea2ffd640 1 -- 172.21.15.172:0/84615827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72895df20 msgr2=0x55a728958ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:47.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.990+0000 7fcea2ffd640 1 --2- 172.21.15.172:0/84615827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72895df20 0x55a728958ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.990+0000 7fcea2ffd640 1 -- 172.21.15.172:0/84615827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7289615d0 msgr2=0x55a728969000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:47.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.990+0000 7fcea2ffd640 1 --2- 172.21.15.172:0/84615827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7289615d0 0x55a728969000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.990+0000 7fcea2ffd640 1 -- 172.21.15.172:0/84615827 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a728968020 con 0x55a72896c370 2024-09-10T08:51:47.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.990+0000 7fcea27fc640 1 --2- 172.21.15.172:0/84615827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7289615d0 0x55a728969000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:51:47.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.990+0000 7fcea2ffd640 1 --2- 172.21.15.172:0/84615827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72896c370 0x55a728978340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55a72896a5e0 tx=0x7fce940044e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20e86dc1735a21d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:47.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.991+0000 7fce88ff9640 1 -- 172.21.15.172:0/84615827 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9402c070 con 0x55a72896c370 2024-09-10T08:51:47.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.991+0000 7fce88ff9640 1 -- 172.21.15.172:0/84615827 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce9400b400 con 0x55a72896c370 2024-09-10T08:51:47.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.991+0000 7fce88ff9640 1 -- 172.21.15.172:0/84615827 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94011a80 con 0x55a72896c370 2024-09-10T08:51:47.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.991+0000 7fceaba5e740 1 -- 172.21.15.172:0/84615827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72896c370 msgr2=0x55a728978340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:47.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.991+0000 7fceaba5e740 1 --2- 172.21.15.172:0/84615827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72896c370 0x55a728978340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55a72896a5e0 tx=0x7fce940044e0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.992+0000 7fceaba5e740 1 -- 172.21.15.172:0/84615827 shutdown_connections 2024-09-10T08:51:47.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.992+0000 7fceaba5e740 1 --2- 172.21.15.172:0/84615827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72895df20 0x55a728958ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.992+0000 7fceaba5e740 1 --2- 172.21.15.172:0/84615827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72896c370 0x55a728978340 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.992+0000 7fceaba5e740 1 --2- 172.21.15.172:0/84615827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7289615d0 0x55a728969000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:47.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.992+0000 7fceaba5e740 1 -- 172.21.15.172:0/84615827 >> 172.21.15.172:0/84615827 conn(0x55a72892a310 msgr2=0x55a7289688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:47.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.992+0000 7fceaba5e740 1 -- 172.21.15.172:0/84615827 shutdown_connections 2024-09-10T08:51:47.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.992+0000 7fceaba5e740 1 -- 172.21.15.172:0/84615827 wait complete. 2024-09-10T08:51:47.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.992+0000 7fceaba5e740 1 -- start start 2024-09-10T08:51:47.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.992+0000 7fceaba5e740 1 Processor -- start 2024-09-10T08:51:47.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.992+0000 7fceaba5e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72895df20 0x55a728974c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.992+0000 7fceaba5e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7289615d0 0x55a728965e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.992+0000 7fceaba5e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72896c370 0x55a728966400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:47.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.992+0000 7fceaba5e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41addd0 con 0x55a7289615d0 2024-09-10T08:51:48.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.992+0000 7fceaba5e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41ad700 con 0x55a72895df20 2024-09-10T08:51:48.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.992+0000 7fceaba5e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41aa910 con 0x55a72896c370 2024-09-10T08:51:48.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fceaba5e740 10 client.0 _ll_register_callbacks cb 0x7fce940040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:48.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fcea27fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72895df20 0x55a728974c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:48.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fcea2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7289615d0 0x55a728965e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:48.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fcea27fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72895df20 0x55a728974c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:42402/0 (socket says 172.21.15.172:42402) 2024-09-10T08:51:48.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fcea27fc640 1 -- 172.21.15.172:0/1063662276 learned_addr learned my addr 172.21.15.172:0/1063662276 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:48.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fcea27fc640 1 --2- 172.21.15.172:0/1063662276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72896c370 0x55a728966400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:48.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fcea27fc640 1 -- 172.21.15.172:0/1063662276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72896c370 msgr2=0x55a728966400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:48.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fcea27fc640 1 --2- 172.21.15.172:0/1063662276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72896c370 0x55a728966400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:48.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fcea27fc640 1 -- 172.21.15.172:0/1063662276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7289615d0 msgr2=0x55a728965e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:48.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fcea27fc640 1 --2- 172.21.15.172:0/1063662276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7289615d0 0x55a728965e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:48.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fcea27fc640 1 -- 172.21.15.172:0/1063662276 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7289699b0 con 0x55a72895df20 2024-09-10T08:51:48.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fcea27fc640 1 --2- 172.21.15.172:0/1063662276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72895df20 0x55a728974c40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fce9801ebe0 tx=0x7fce9800f7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a5580d8dda8b394 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:48.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fce86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:51:48.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fce86ff5640 1 -- 172.21.15.172:0/1063662276 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9801ec40 con 0x55a72895df20 2024-09-10T08:51:48.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fceaba5e740 1 -- 172.21.15.172:0/1063662276 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce9400b400 con 0x55a72895df20 2024-09-10T08:51:48.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fce86ff5640 1 -- 172.21.15.172:0/1063662276 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fce98005040 con 0x55a72895df20 2024-09-10T08:51:48.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fce86ff5640 1 -- 172.21.15.172:0/1063662276 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fce98049050 con 0x55a72895df20 2024-09-10T08:51:48.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fceaba5e740 10 client.26264 fetch_fsmap learned FSMap version 35 2024-09-10T08:51:48.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.993+0000 7fceaba5e740 1 -- 172.21.15.172:0/1063662276 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a7289e6ca0 con 0x55a72895df20 2024-09-10T08:51:48.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.994+0000 7fce86ff5640 1 -- 172.21.15.172:0/1063662276 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce9400b400 con 0x55a72895df20 2024-09-10T08:51:48.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.994+0000 7fce86ff5640 1 -- 172.21.15.172:0/1063662276 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fce98040070 con 0x55a72895df20 2024-09-10T08:51:48.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.994+0000 7fceaba5e740 10 client.26264 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T08:51:48.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.994+0000 7fceaba5e740 10 client.26264 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:51:48.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.994+0000 7fceaba5e740 20 client.26264 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:48.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.994+0000 7fceaba5e740 1 --2- 172.21.15.172:0/1063662276 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55a7289e7180 0x55a7289e9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:48.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.994+0000 7fceaba5e740 4 client.26264 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:51:48.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.994+0000 7fceaba5e740 1 -- 172.21.15.172:0/1063662276 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7288ced70 con 0x55a7289e7180 2024-09-10T08:51:48.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.994+0000 7fcea2ffd640 1 --2- 172.21.15.172:0/1063662276 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55a7289e7180 0x55a7289e9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:48.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.995+0000 7fcea2ffd640 1 --2- 172.21.15.172:0/1063662276 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55a7289e7180 0x55a7289e9570 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8359c49c6790fb5e server_cookie=6e816fec2db1d7d0 in_seq=0 out_seq=0 2024-09-10T08:51:48.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.995+0000 7fce86ff5640 10 client.26264 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:51:48.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.995+0000 7fce86ff5640 1 -- 172.21.15.172:0/1063662276 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7288ced70 con 0x55a7289e7180 2024-09-10T08:51:48.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.995+0000 7fce86ff5640 10 client.26264 handle_command_reply: tid=0 2024-09-10T08:51:48.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.995+0000 7fceaba5e740 1 client.26264 shutdown 2024-09-10T08:51:48.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.995+0000 7fceaba5e740 1 -- 172.21.15.172:0/1063662276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72895df20 msgr2=0x55a728974c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.995+0000 7fceaba5e740 1 --2- 172.21.15.172:0/1063662276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72895df20 0x55a728974c40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fce9801ebe0 tx=0x7fce9800f7a0 comp rx=0 tx=0).stop 2024-09-10T08:51:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.995+0000 7fceaba5e740 1 -- 172.21.15.172:0/1063662276 shutdown_connections 2024-09-10T08:51:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.995+0000 7fceaba5e740 1 --2- 172.21.15.172:0/1063662276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72896c370 0x55a728966400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.995+0000 7fceaba5e740 1 --2- 172.21.15.172:0/1063662276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7289615d0 0x55a728965e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.995+0000 7fceaba5e740 1 --2- 172.21.15.172:0/1063662276 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55a7289e7180 0x55a7289e9570 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.995+0000 7fceaba5e740 1 --2- 172.21.15.172:0/1063662276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a72895df20 0x55a728974c40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.995+0000 7fceaba5e740 1 -- 172.21.15.172:0/1063662276 >> 172.21.15.172:0/1063662276 conn(0x55a72892a310 msgr2=0x55a7289688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.995+0000 7fce86ff5640 0 client.26264 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:51:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.995+0000 7fceaba5e740 1 -- 172.21.15.172:0/1063662276 shutdown_connections 2024-09-10T08:51:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.995+0000 7fceaba5e740 1 -- 172.21.15.172:0/1063662276 wait complete. 2024-09-10T08:51:48.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.995+0000 7fceaba5e740 20 client.26264 trim_cache size 0 max 16384 2024-09-10T08:51:48.004 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:51:48.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.996+0000 7fcea9685640 1 -- 172.21.15.172:0/128826919 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fce8c0a2b80 msgr2=0x7fce8c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:48.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.996+0000 7fcea9685640 1 --2- 172.21.15.172:0/128826919 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fce8c0a2b80 0x7fce8c0a5040 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0bc8d0 tx=0x7fce9c08fa00 comp rx=0 tx=0).stop 2024-09-10T08:51:48.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.996+0000 7fcea9685640 1 -- 172.21.15.172:0/128826919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcea4102ef0 msgr2=0x7fcea41a4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:48.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.996+0000 7fcea9685640 1 --2- 172.21.15.172:0/128826919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcea4102ef0 0x7fcea41a4320 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fce9c02ae70 tx=0x7fce9c02f5d0 comp rx=0 tx=0).stop 2024-09-10T08:51:48.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.996+0000 7fcea9685640 1 -- 172.21.15.172:0/128826919 shutdown_connections 2024-09-10T08:51:48.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.996+0000 7fcea9685640 1 --2- 172.21.15.172:0/128826919 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fce8c0a2b80 0x7fce8c0a5040 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:48.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.996+0000 7fcea9685640 1 --2- 172.21.15.172:0/128826919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcea4102ef0 0x7fcea41a4320 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:48.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.996+0000 7fcea9685640 1 --2- 172.21.15.172:0/128826919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcea4102570 0x7fcea41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:48.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.996+0000 7fcea9685640 1 --2- 172.21.15.172:0/128826919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcea4100e20 0x7fcea41992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:48.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.996+0000 7fcea9685640 1 -- 172.21.15.172:0/128826919 >> 172.21.15.172:0/128826919 conn(0x7fcea40fc7d0 msgr2=0x7fcea40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:48.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.996+0000 7fcea9685640 1 -- 172.21.15.172:0/128826919 shutdown_connections 2024-09-10T08:51:48.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:47.996+0000 7fcea9685640 1 -- 172.21.15.172:0/128826919 wait complete. 2024-09-10T08:51:48.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:48 smithi172 ceph-mon[30854]: pgmap v517: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 MiB/s wr, 23 op/s 2024-09-10T08:51:48.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:48 smithi174 ceph-mon[34068]: pgmap v517: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 MiB/s wr, 23 op/s 2024-09-10T08:51:48.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:48 smithi191 ceph-mon[35314]: pgmap v517: 129 pgs: 129 active+clean; 3.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 MiB/s wr, 23 op/s 2024-09-10T08:51:50.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:50 smithi172 ceph-mon[30854]: pgmap v518: 129 pgs: 129 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 21 op/s 2024-09-10T08:51:50.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:50 smithi174 ceph-mon[34068]: pgmap v518: 129 pgs: 129 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 21 op/s 2024-09-10T08:51:50.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:50 smithi191 ceph-mon[35314]: pgmap v518: 129 pgs: 129 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 21 op/s 2024-09-10T08:51:52.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:52 smithi174 ceph-mon[34068]: pgmap v519: 129 pgs: 129 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-09-10T08:51:52.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:52 smithi191 ceph-mon[35314]: pgmap v519: 129 pgs: 129 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-09-10T08:51:52.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:52 smithi172 ceph-mon[30854]: pgmap v519: 129 pgs: 129 active+clean; 4.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-09-10T08:51:54.409 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:54 smithi172 ceph-mon[30854]: pgmap v520: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-09-10T08:51:54.439 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T08:51:54.739 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:51:54.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:54 smithi174 ceph-mon[34068]: pgmap v520: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-09-10T08:51:54.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:54 smithi191 ceph-mon[35314]: pgmap v520: 129 pgs: 129 active+clean; 4.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-09-10T08:51:56.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:56 smithi174 ceph-mon[34068]: pgmap v521: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 26 op/s 2024-09-10T08:51:56.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:56 smithi191 ceph-mon[35314]: pgmap v521: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 26 op/s 2024-09-10T08:51:56.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:56 smithi172 ceph-mon[30854]: pgmap v521: 129 pgs: 129 active+clean; 4.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 26 op/s 2024-09-10T08:51:58.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.907+0000 7f6885e55640 1 -- 172.21.15.172:0/1442866537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6880102790 msgr2=0x7f68801067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:58.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.907+0000 7f6885e55640 1 --2- 172.21.15.172:0/1442866537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6880102790 0x7f68801067e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6874009900 tx=0x7f687402a410 comp rx=0 tx=0).stop 2024-09-10T08:51:58.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.907+0000 7f6885e55640 1 -- 172.21.15.172:0/1442866537 shutdown_connections 2024-09-10T08:51:58.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.907+0000 7f6885e55640 1 --2- 172.21.15.172:0/1442866537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6880106f10 0x7f688010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.907+0000 7f6885e55640 1 --2- 172.21.15.172:0/1442866537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6880102790 0x7f68801067e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.907+0000 7f6885e55640 1 --2- 172.21.15.172:0/1442866537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6880101de0 0x7f68801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.907+0000 7f6885e55640 1 -- 172.21.15.172:0/1442866537 >> 172.21.15.172:0/1442866537 conn(0x7f68800fd8e0 msgr2=0x7f68800ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:58.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.907+0000 7f6885e55640 1 -- 172.21.15.172:0/1442866537 shutdown_connections 2024-09-10T08:51:58.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.907+0000 7f6885e55640 1 -- 172.21.15.172:0/1442866537 wait complete. 2024-09-10T08:51:58.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.907+0000 7f6885e55640 1 Processor -- start 2024-09-10T08:51:58.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f6885e55640 1 -- start start 2024-09-10T08:51:58.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f6885e55640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6880101de0 0x7f68800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f6885e55640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6880102790 0x7f6880074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f6885e55640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6880106f10 0x7f6880074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f6885e55640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6880111de0 con 0x7f6880102790 2024-09-10T08:51:58.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f6885e55640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6880111c60 con 0x7f6880101de0 2024-09-10T08:51:58.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f6885e55640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6880111f60 con 0x7f6880106f10 2024-09-10T08:51:58.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f687effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6880102790 0x7f6880074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f687effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6880102790 0x7f6880074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50722/0 (socket says 172.21.15.172:50722) 2024-09-10T08:51:58.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f687effd640 1 -- 172.21.15.172:0/2663388683 learned_addr learned my addr 172.21.15.172:0/2663388683 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:58.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f687f7fe640 1 --2- 172.21.15.172:0/2663388683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6880101de0 0x7f68800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f687ffff640 1 --2- 172.21.15.172:0/2663388683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6880106f10 0x7f6880074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f687effd640 1 -- 172.21.15.172:0/2663388683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6880106f10 msgr2=0x7f6880074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:58.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f687effd640 1 --2- 172.21.15.172:0/2663388683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6880106f10 0x7f6880074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f687effd640 1 -- 172.21.15.172:0/2663388683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6880101de0 msgr2=0x7f68800744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:58.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f687effd640 1 --2- 172.21.15.172:0/2663388683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6880101de0 0x7f68800744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f687effd640 1 -- 172.21.15.172:0/2663388683 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68801a6820 con 0x7f6880102790 2024-09-10T08:51:58.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.908+0000 7f687effd640 1 --2- 172.21.15.172:0/2663388683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6880102790 0x7f6880074a30 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f6874005940 tx=0x7f68740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:58.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.909+0000 7f687cff9640 1 -- 172.21.15.172:0/2663388683 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687403f030 con 0x7f6880102790 2024-09-10T08:51:58.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.909+0000 7f6885e55640 1 -- 172.21.15.172:0/2663388683 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68801a6ab0 con 0x7f6880102790 2024-09-10T08:51:58.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.909+0000 7f687cff9640 1 -- 172.21.15.172:0/2663388683 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6874003cb0 con 0x7f6880102790 2024-09-10T08:51:58.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.909+0000 7f687cff9640 1 -- 172.21.15.172:0/2663388683 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874035070 con 0x7f6880102790 2024-09-10T08:51:58.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.909+0000 7f6885e55640 1 -- 172.21.15.172:0/2663388683 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68801a6fe0 con 0x7f6880102790 2024-09-10T08:51:58.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.911+0000 7f687cff9640 1 -- 172.21.15.172:0/2663388683 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6874042030 con 0x7f6880102790 2024-09-10T08:51:58.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.912+0000 7f688822e740 1 Processor -- start 2024-09-10T08:51:58.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.912+0000 7f688822e740 1 -- start start 2024-09-10T08:51:58.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.912+0000 7f688822e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 0x56106173c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.912+0000 7f688822e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56106173c5d0 0x56106173ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.912+0000 7f688822e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561061747370 0x561061749760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.912+0000 7f688822e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5610616a22f0 con 0x561061747370 2024-09-10T08:51:58.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.912+0000 7f688822e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561061649cf0 con 0x56106173db40 2024-09-10T08:51:58.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.912+0000 7f688822e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56106169e2b0 con 0x56106173c5d0 2024-09-10T08:51:58.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.912+0000 7f687cff9640 1 --2- 172.21.15.172:0/2663388683 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68680a2c00 0x7f68680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.912+0000 7f687f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56106173c5d0 0x56106173ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.912+0000 7f687f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56106173c5d0 0x56106173ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:59680/0 (socket says 172.21.15.172:59680) 2024-09-10T08:51:58.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.912+0000 7f687f7fe640 1 -- 172.21.15.172:0/3652443393 learned_addr learned my addr 172.21.15.172:0/3652443393 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:58.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.912+0000 7f687effd640 1 --2- 172.21.15.172:0/3652443393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561061747370 0x561061749760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.912+0000 7f687ffff640 1 --2- 172.21.15.172:0/3652443393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 0x56106173c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.913+0000 7f687cff9640 1 -- 172.21.15.172:0/2663388683 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f68740efb50 con 0x7f6880102790 2024-09-10T08:51:58.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.913+0000 7f687f7fe640 1 -- 172.21.15.172:0/3652443393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 msgr2=0x56106173c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:58.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.913+0000 7f687f7fe640 1 --2- 172.21.15.172:0/3652443393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 0x56106173c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.913+0000 7f687f7fe640 1 -- 172.21.15.172:0/3652443393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561061747370 msgr2=0x561061749760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:58.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.913+0000 7f687f7fe640 1 --2- 172.21.15.172:0/3652443393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561061747370 0x561061749760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.913+0000 7f687f7fe640 1 -- 172.21.15.172:0/3652443393 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56106162e3a0 con 0x56106173c5d0 2024-09-10T08:51:58.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.913+0000 7f687effd640 1 --2- 172.21.15.172:0/3652443393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561061747370 0x561061749760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:51:58.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.913+0000 7f687ffff640 1 --2- 172.21.15.172:0/3652443393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 0x56106173c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:51:58.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.913+0000 7f687ffff640 1 --2- 172.21.15.172:0/2663388683 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68680a2c00 0x7f68680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.913+0000 7f687f7fe640 1 --2- 172.21.15.172:0/3652443393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56106173c5d0 0x56106173ca10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f687000bcd0 tx=0x7f68700107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab09bb22eb95bdc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:58.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.913+0000 7f6865ffb640 1 -- 172.21.15.172:0/3652443393 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870005220 con 0x56106173c5d0 2024-09-10T08:51:58.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.913+0000 7f687ffff640 1 --2- 172.21.15.172:0/2663388683 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68680a2c00 0x7f68680a50c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f68780094d0 tx=0x7f6878004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:58.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.913+0000 7f6865ffb640 1 -- 172.21.15.172:0/3652443393 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6870004c50 con 0x56106173c5d0 2024-09-10T08:51:58.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.913+0000 7f6865ffb640 1 -- 172.21.15.172:0/3652443393 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68700116a0 con 0x56106173c5d0 2024-09-10T08:51:58.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.914+0000 7f688822e740 1 -- 172.21.15.172:0/3652443393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56106173c5d0 msgr2=0x56106173ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:58.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.914+0000 7f688822e740 1 --2- 172.21.15.172:0/3652443393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56106173c5d0 0x56106173ca10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f687000bcd0 tx=0x7f68700107a0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.914+0000 7f688822e740 1 -- 172.21.15.172:0/3652443393 shutdown_connections 2024-09-10T08:51:58.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.914+0000 7f688822e740 1 --2- 172.21.15.172:0/3652443393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561061747370 0x561061749760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.914+0000 7f688822e740 1 --2- 172.21.15.172:0/3652443393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56106173c5d0 0x56106173ca10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.914+0000 7f688822e740 1 --2- 172.21.15.172:0/3652443393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 0x56106173c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.914+0000 7f688822e740 1 -- 172.21.15.172:0/3652443393 >> 172.21.15.172:0/3652443393 conn(0x561061705310 msgr2=0x561061738c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:58.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.914+0000 7f688822e740 1 -- 172.21.15.172:0/3652443393 shutdown_connections 2024-09-10T08:51:58.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.914+0000 7f688822e740 1 -- 172.21.15.172:0/3652443393 wait complete. 2024-09-10T08:51:58.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.915+0000 7f688822e740 1 -- start start 2024-09-10T08:51:58.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.915+0000 7f688822e740 1 Processor -- start 2024-09-10T08:51:58.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.915+0000 7f688822e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56106173c5d0 0x561061746d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.915+0000 7f688822e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 0x561061744010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.915+0000 7f688822e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561061747370 0x561061744600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.915+0000 7f688822e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56106174a070 con 0x56106173c5d0 2024-09-10T08:51:58.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.915+0000 7f688822e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56106165d060 con 0x56106173db40 2024-09-10T08:51:58.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.915+0000 7f688822e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56106174a1f0 con 0x561061747370 2024-09-10T08:51:58.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.915+0000 7f688822e740 10 client.0 _ll_register_callbacks cb 0x5610613b08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:58.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.915+0000 7f687f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56106173c5d0 0x561061746d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.915+0000 7f687f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56106173c5d0 0x561061746d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50742/0 (socket says 172.21.15.172:50742) 2024-09-10T08:51:58.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.915+0000 7f687f7fe640 1 -- 172.21.15.172:0/3173225173 learned_addr learned my addr 172.21.15.172:0/3173225173 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:58.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.915+0000 7f687effd640 1 --2- 172.21.15.172:0/3173225173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 0x561061744010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.916+0000 7f687ffff640 1 --2- 172.21.15.172:0/3173225173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561061747370 0x561061744600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.916+0000 7f687f7fe640 1 -- 172.21.15.172:0/3173225173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561061747370 msgr2=0x561061744600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:58.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.916+0000 7f687f7fe640 1 --2- 172.21.15.172:0/3173225173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561061747370 0x561061744600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.916+0000 7f687f7fe640 1 -- 172.21.15.172:0/3173225173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 msgr2=0x561061744010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:58.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.916+0000 7f687f7fe640 1 --2- 172.21.15.172:0/3173225173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 0x561061744010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.916+0000 7f687f7fe640 1 -- 172.21.15.172:0/3173225173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610614133c0 con 0x56106173c5d0 2024-09-10T08:51:58.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.916+0000 7f687effd640 1 --2- 172.21.15.172:0/3173225173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 0x561061744010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:51:58.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.916+0000 7f687ffff640 1 --2- 172.21.15.172:0/3173225173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561061747370 0x561061744600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:51:58.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.916+0000 7f687f7fe640 1 --2- 172.21.15.172:0/3173225173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56106173c5d0 0x561061746d60 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f68700048f0 tx=0x7f6870004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7556b631ca4e010f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:58.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.916+0000 7f685effd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:58.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.916+0000 7f685effd640 1 -- 172.21.15.172:0/3173225173 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870005010 con 0x56106173c5d0 2024-09-10T08:51:58.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.916+0000 7f685effd640 1 -- 172.21.15.172:0/3173225173 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f68700051b0 con 0x56106173c5d0 2024-09-10T08:51:58.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.916+0000 7f688822e740 1 -- 172.21.15.172:0/3173225173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6870004c50 con 0x56106173c5d0 2024-09-10T08:51:58.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.916+0000 7f685effd640 1 -- 172.21.15.172:0/3173225173 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6870004020 con 0x56106173c5d0 2024-09-10T08:51:58.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.916+0000 7f688822e740 10 client.17088 fetch_fsmap learned FSMap version 35 2024-09-10T08:51:58.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.916+0000 7f688822e740 1 -- 172.21.15.172:0/3173225173 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561061755b10 con 0x56106173c5d0 2024-09-10T08:51:58.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.917+0000 7f685effd640 1 -- 172.21.15.172:0/3173225173 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6870004c50 con 0x56106173c5d0 2024-09-10T08:51:58.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.917+0000 7f685effd640 1 -- 172.21.15.172:0/3173225173 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f6870003aa0 con 0x56106173c5d0 2024-09-10T08:51:58.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.917+0000 7f688822e740 10 client.17088 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T08:51:58.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.917+0000 7f688822e740 10 client.17088 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:51:58.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.917+0000 7f688822e740 20 client.17088 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:58.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.917+0000 7f688822e740 1 --2- 172.21.15.172:0/3173225173 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x561061756e60 0x561061759250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.917+0000 7f688822e740 4 client.17088 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:51:58.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.917+0000 7f688822e740 1 -- 172.21.15.172:0/3173225173 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610616af860 con 0x561061756e60 2024-09-10T08:51:58.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.917+0000 7f687effd640 1 --2- 172.21.15.172:0/3173225173 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x561061756e60 0x561061759250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.917+0000 7f687effd640 1 --2- 172.21.15.172:0/3173225173 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x561061756e60 0x561061759250 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2494758c0b11474a server_cookie=66e0926fa7db1480 in_seq=0 out_seq=0 2024-09-10T08:51:58.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.917+0000 7f685effd640 10 client.17088 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:51:58.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.918+0000 7f685effd640 1 -- 172.21.15.172:0/3173225173 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610616af860 con 0x561061756e60 2024-09-10T08:51:58.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.918+0000 7f685effd640 10 client.17088 handle_command_reply: tid=0 2024-09-10T08:51:58.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.918+0000 7f688822e740 1 client.17088 shutdown 2024-09-10T08:51:58.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.919+0000 7f688822e740 1 -- 172.21.15.172:0/3173225173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56106173c5d0 msgr2=0x561061746d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:58.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.919+0000 7f688822e740 1 --2- 172.21.15.172:0/3173225173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56106173c5d0 0x561061746d60 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f68700048f0 tx=0x7f6870004920 comp rx=0 tx=0).stop 2024-09-10T08:51:58.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.919+0000 7f688822e740 1 -- 172.21.15.172:0/3173225173 shutdown_connections 2024-09-10T08:51:58.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.919+0000 7f688822e740 1 --2- 172.21.15.172:0/3173225173 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x561061756e60 0x561061759250 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.919+0000 7f688822e740 1 --2- 172.21.15.172:0/3173225173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561061747370 0x561061744600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.919+0000 7f688822e740 1 --2- 172.21.15.172:0/3173225173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 0x561061744010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.919+0000 7f688822e740 1 --2- 172.21.15.172:0/3173225173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56106173c5d0 0x561061746d60 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.919+0000 7f685effd640 0 client.17088 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:51:58.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.919+0000 7f688822e740 1 -- 172.21.15.172:0/3173225173 >> 172.21.15.172:0/3173225173 conn(0x561061705310 msgr2=0x56106173e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:58.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.919+0000 7f688822e740 1 -- 172.21.15.172:0/3173225173 shutdown_connections 2024-09-10T08:51:58.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.919+0000 7f688822e740 1 -- 172.21.15.172:0/3173225173 wait complete. 2024-09-10T08:51:58.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.919+0000 7f688822e740 20 client.17088 trim_cache size 0 max 16384 2024-09-10T08:51:58.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.933+0000 7f688822e740 1 Processor -- start 2024-09-10T08:51:58.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.933+0000 7f688822e740 1 -- start start 2024-09-10T08:51:58.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.933+0000 7f688822e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56106173c5d0 0x561061742ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.933+0000 7f688822e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56106173db40 0x56106175be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.933+0000 7f688822e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561061747370 0x56106175c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.933+0000 7f688822e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f68801b0d40 con 0x56106173c5d0 2024-09-10T08:51:58.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.933+0000 7f688822e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68801b28e0 con 0x561061747370 2024-09-10T08:51:58.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.933+0000 7f688822e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f68801b1410 con 0x56106173db40 2024-09-10T08:51:58.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.934+0000 7f687f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56106173c5d0 0x561061742ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.934+0000 7f687f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56106173c5d0 0x561061742ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50750/0 (socket says 172.21.15.172:50750) 2024-09-10T08:51:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.934+0000 7f687f7fe640 1 -- 172.21.15.172:0/2393272147 learned_addr learned my addr 172.21.15.172:0/2393272147 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.934+0000 7f687ffff640 1 --2- 172.21.15.172:0/2393272147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561061747370 0x56106175c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.934+0000 7f687effd640 1 --2- 172.21.15.172:0/2393272147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56106173db40 0x56106175be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.934+0000 7f687f7fe640 1 -- 172.21.15.172:0/2393272147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56106173db40 msgr2=0x56106175be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.934+0000 7f687f7fe640 1 --2- 172.21.15.172:0/2393272147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56106173db40 0x56106175be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.934+0000 7f687f7fe640 1 -- 172.21.15.172:0/2393272147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561061747370 msgr2=0x56106175c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.934+0000 7f687f7fe640 1 --2- 172.21.15.172:0/2393272147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561061747370 0x56106175c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.934+0000 7f687f7fe640 1 -- 172.21.15.172:0/2393272147 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56106173f120 con 0x56106173c5d0 2024-09-10T08:51:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.934+0000 7f687f7fe640 1 --2- 172.21.15.172:0/2393272147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56106173c5d0 0x561061742ff0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f6870011010 tx=0x7f6870001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd34e7a543fce65 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.934+0000 7f68657fa640 1 -- 172.21.15.172:0/2393272147 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687000b810 con 0x56106173c5d0 2024-09-10T08:51:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.934+0000 7f68657fa640 1 -- 172.21.15.172:0/2393272147 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6870004990 con 0x56106173c5d0 2024-09-10T08:51:58.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.934+0000 7f68657fa640 1 -- 172.21.15.172:0/2393272147 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870004420 con 0x56106173c5d0 2024-09-10T08:51:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.935+0000 7f688822e740 1 -- 172.21.15.172:0/2393272147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56106173c5d0 msgr2=0x561061742ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.935+0000 7f688822e740 1 --2- 172.21.15.172:0/2393272147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56106173c5d0 0x561061742ff0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f6870011010 tx=0x7f6870001e70 comp rx=0 tx=0).stop 2024-09-10T08:51:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.935+0000 7f688822e740 1 -- 172.21.15.172:0/2393272147 shutdown_connections 2024-09-10T08:51:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.935+0000 7f688822e740 1 --2- 172.21.15.172:0/2393272147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561061747370 0x56106175c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.935+0000 7f688822e740 1 --2- 172.21.15.172:0/2393272147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56106173db40 0x56106175be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.935+0000 7f688822e740 1 --2- 172.21.15.172:0/2393272147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56106173c5d0 0x561061742ff0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.935+0000 7f688822e740 1 -- 172.21.15.172:0/2393272147 >> 172.21.15.172:0/2393272147 conn(0x561061705310 msgr2=0x5610617428e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.935+0000 7f688822e740 1 -- 172.21.15.172:0/2393272147 shutdown_connections 2024-09-10T08:51:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.935+0000 7f688822e740 1 -- 172.21.15.172:0/2393272147 wait complete. 2024-09-10T08:51:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.936+0000 7f688822e740 1 -- start start 2024-09-10T08:51:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.936+0000 7f688822e740 1 Processor -- start 2024-09-10T08:51:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.936+0000 7f688822e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56106173c5d0 0x5610617595f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.936+0000 7f688822e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 0x561061759b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.936+0000 7f688822e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561061747370 0x5610617b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.936+0000 7f687ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561061747370 0x5610617b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.936+0000 7f687ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561061747370 0x5610617b9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50760/0 (socket says 172.21.15.172:50760) 2024-09-10T08:51:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.936+0000 7f687ffff640 1 -- 172.21.15.172:0/955837828 learned_addr learned my addr 172.21.15.172:0/955837828 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:51:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.936+0000 7f688822e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f68801b21e0 con 0x561061747370 2024-09-10T08:51:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.936+0000 7f688822e740 1 -- 172.21.15.172:0/955837828 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68801b1b10 con 0x56106173db40 2024-09-10T08:51:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.936+0000 7f688822e740 1 -- 172.21.15.172:0/955837828 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f68801aed20 con 0x56106173c5d0 2024-09-10T08:51:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.936+0000 7f687effd640 1 --2- 172.21.15.172:0/955837828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 0x561061759b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.936+0000 7f688822e740 10 client.0 _ll_register_callbacks cb 0x7f6874005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:51:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.936+0000 7f687f7fe640 1 --2- 172.21.15.172:0/955837828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56106173c5d0 0x5610617595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.937+0000 7f687ffff640 1 -- 172.21.15.172:0/955837828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56106173c5d0 msgr2=0x5610617595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.937+0000 7f687ffff640 1 --2- 172.21.15.172:0/955837828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56106173c5d0 0x5610617595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.937+0000 7f687ffff640 1 -- 172.21.15.172:0/955837828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 msgr2=0x561061759b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:51:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.937+0000 7f687ffff640 1 --2- 172.21.15.172:0/955837828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 0x561061759b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.937+0000 7f687ffff640 1 -- 172.21.15.172:0/955837828 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610617b9e00 con 0x561061747370 2024-09-10T08:51:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.937+0000 7f687effd640 1 --2- 172.21.15.172:0/955837828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 0x561061759b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:51:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.937+0000 7f687ffff640 1 --2- 172.21.15.172:0/955837828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561061747370 0x5610617b9760 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f68780043d0 tx=0x7f6878004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74e9c5f96c0cae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:51:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.937+0000 7f685effd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:51:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.937+0000 7f685effd640 1 -- 172.21.15.172:0/955837828 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687802d060 con 0x561061747370 2024-09-10T08:51:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.937+0000 7f685effd640 1 -- 172.21.15.172:0/955837828 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f687801d440 con 0x561061747370 2024-09-10T08:51:58.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.937+0000 7f688822e740 1 -- 172.21.15.172:0/955837828 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6870004990 con 0x561061747370 2024-09-10T08:51:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.937+0000 7f685effd640 1 -- 172.21.15.172:0/955837828 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6878034050 con 0x561061747370 2024-09-10T08:51:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.938+0000 7f688822e740 10 client.17094 fetch_fsmap learned FSMap version 35 2024-09-10T08:51:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.938+0000 7f688822e740 1 -- 172.21.15.172:0/955837828 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5610617c11d0 con 0x561061747370 2024-09-10T08:51:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.938+0000 7f685effd640 1 -- 172.21.15.172:0/955837828 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6870004990 con 0x561061747370 2024-09-10T08:51:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.938+0000 7f685effd640 1 -- 172.21.15.172:0/955837828 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f6878027030 con 0x561061747370 2024-09-10T08:51:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.938+0000 7f688822e740 10 client.17094 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T08:51:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.938+0000 7f688822e740 10 client.17094 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:51:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.938+0000 7f688822e740 20 client.17094 populate_metadata read hostname 'smithi172' 2024-09-10T08:51:58.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.938+0000 7f688822e740 1 --2- 172.21.15.172:0/955837828 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5610617c1980 0x5610617c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:51:58.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.938+0000 7f688822e740 4 client.17094 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:51:58.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.938+0000 7f688822e740 1 -- 172.21.15.172:0/955837828 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610616a9d70 con 0x5610617c1980 2024-09-10T08:51:58.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.938+0000 7f687f7fe640 1 --2- 172.21.15.172:0/955837828 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5610617c1980 0x5610617c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:51:58.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.939+0000 7f687f7fe640 1 --2- 172.21.15.172:0/955837828 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5610617c1980 0x5610617c3da0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fdf66b068205274 server_cookie=53fac8731b1226fb in_seq=0 out_seq=0 2024-09-10T08:51:58.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.939+0000 7f685effd640 10 client.17094 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:51:58.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.939+0000 7f685effd640 1 -- 172.21.15.172:0/955837828 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610616a9d70 con 0x5610617c1980 2024-09-10T08:51:58.457 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:51:58.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.939+0000 7f685effd640 10 client.17094 handle_command_reply: tid=0 2024-09-10T08:51:58.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.939+0000 7f688822e740 1 client.17094 shutdown 2024-09-10T08:51:58.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.939+0000 7f688822e740 1 -- 172.21.15.172:0/955837828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561061747370 msgr2=0x5610617b9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:58.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.939+0000 7f688822e740 1 --2- 172.21.15.172:0/955837828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561061747370 0x5610617b9760 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f68780043d0 tx=0x7f6878004ab0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.939+0000 7f688822e740 1 -- 172.21.15.172:0/955837828 shutdown_connections 2024-09-10T08:51:58.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.939+0000 7f688822e740 1 --2- 172.21.15.172:0/955837828 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5610617c1980 0x5610617c3da0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.939+0000 7f688822e740 1 --2- 172.21.15.172:0/955837828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561061747370 0x5610617b9760 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.939+0000 7f685effd640 0 client.17094 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:51:58.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.939+0000 7f688822e740 1 --2- 172.21.15.172:0/955837828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56106173db40 0x561061759b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.939+0000 7f688822e740 1 --2- 172.21.15.172:0/955837828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56106173c5d0 0x5610617595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.939+0000 7f688822e740 1 -- 172.21.15.172:0/955837828 >> 172.21.15.172:0/955837828 conn(0x561061705310 msgr2=0x561061743100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.939+0000 7f688822e740 1 -- 172.21.15.172:0/955837828 shutdown_connections 2024-09-10T08:51:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.939+0000 7f688822e740 1 -- 172.21.15.172:0/955837828 wait complete. 2024-09-10T08:51:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.940+0000 7f688822e740 20 client.17094 trim_cache size 0 max 16384 2024-09-10T08:51:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.940+0000 7f6885e55640 1 -- 172.21.15.172:0/2663388683 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68680a2c00 msgr2=0x7f68680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.940+0000 7f6885e55640 1 --2- 172.21.15.172:0/2663388683 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68680a2c00 0x7f68680a50c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f68780094d0 tx=0x7f6878004a40 comp rx=0 tx=0).stop 2024-09-10T08:51:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.940+0000 7f6885e55640 1 -- 172.21.15.172:0/2663388683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6880102790 msgr2=0x7f6880074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:51:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.940+0000 7f6885e55640 1 --2- 172.21.15.172:0/2663388683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6880102790 0x7f6880074a30 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f6874005940 tx=0x7f68740042f0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.940+0000 7f6885e55640 1 -- 172.21.15.172:0/2663388683 shutdown_connections 2024-09-10T08:51:58.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.940+0000 7f6885e55640 1 --2- 172.21.15.172:0/2663388683 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68680a2c00 0x7f68680a50c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.940+0000 7f6885e55640 1 --2- 172.21.15.172:0/2663388683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6880106f10 0x7f6880074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.940+0000 7f6885e55640 1 --2- 172.21.15.172:0/2663388683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6880102790 0x7f6880074a30 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.940+0000 7f6885e55640 1 --2- 172.21.15.172:0/2663388683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6880101de0 0x7f68800744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:51:58.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.940+0000 7f6885e55640 1 -- 172.21.15.172:0/2663388683 >> 172.21.15.172:0/2663388683 conn(0x7f68800fd8e0 msgr2=0x7f68800fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:51:58.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.940+0000 7f6885e55640 1 -- 172.21.15.172:0/2663388683 shutdown_connections 2024-09-10T08:51:58.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:51:57.940+0000 7f6885e55640 1 -- 172.21.15.172:0/2663388683 wait complete. 2024-09-10T08:51:58.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:58 smithi174 ceph-mon[34068]: pgmap v522: 129 pgs: 129 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-09-10T08:51:58.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:58 smithi191 ceph-mon[35314]: pgmap v522: 129 pgs: 129 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-09-10T08:51:58.812 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:58 smithi172 ceph-mon[30854]: pgmap v522: 129 pgs: 129 active+clean; 4.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 22 op/s 2024-09-10T08:51:59.255 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:51:59.553 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:51:59.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:51:59 smithi172 ceph-mon[30854]: pgmap v523: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-09-10T08:51:59.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:51:59 smithi174 ceph-mon[34068]: pgmap v523: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-09-10T08:51:59.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:51:59 smithi191 ceph-mon[35314]: pgmap v523: 129 pgs: 129 active+clean; 4.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 MiB/s wr, 20 op/s 2024-09-10T08:52:00.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.085+0000 7f309683f640 1 -- 172.21.15.172:0/650689749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3090102ef0 msgr2=0x7f309010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.085+0000 7f309683f640 1 --2- 172.21.15.172:0/650689749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3090102ef0 0x7f309010da40 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3088009ca0 tx=0x7f308802a950 comp rx=0 tx=0).stop 2024-09-10T08:52:00.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.085+0000 7f309683f640 1 -- 172.21.15.172:0/650689749 shutdown_connections 2024-09-10T08:52:00.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.085+0000 7f309683f640 1 --2- 172.21.15.172:0/650689749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3090102ef0 0x7f309010da40 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.085+0000 7f309683f640 1 --2- 172.21.15.172:0/650689749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3090102570 0x7f30901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.085+0000 7f309683f640 1 --2- 172.21.15.172:0/650689749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3090100e20 0x7f3090101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.085+0000 7f309683f640 1 -- 172.21.15.172:0/650689749 >> 172.21.15.172:0/650689749 conn(0x7f30900fc7d0 msgr2=0x7f30900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:00.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.085+0000 7f309683f640 1 -- 172.21.15.172:0/650689749 shutdown_connections 2024-09-10T08:52:00.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.085+0000 7f309683f640 1 -- 172.21.15.172:0/650689749 wait complete. 2024-09-10T08:52:00.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.085+0000 7f309683f640 1 Processor -- start 2024-09-10T08:52:00.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.085+0000 7f309683f640 1 -- start start 2024-09-10T08:52:00.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.086+0000 7f309683f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3090100e20 0x7f30901983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.086+0000 7f309683f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3090102570 0x7f3090198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.086+0000 7f309683f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3090102ef0 0x7f309019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.086+0000 7f309683f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f309006a540 con 0x7f3090102ef0 2024-09-10T08:52:00.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.086+0000 7f309683f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f309006a3c0 con 0x7f3090102570 2024-09-10T08:52:00.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.086+0000 7f309683f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f309006a6c0 con 0x7f3090100e20 2024-09-10T08:52:00.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.086+0000 7f3094ddc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3090102ef0 0x7f309019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.086+0000 7f3094ddc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3090102ef0 0x7f309019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50784/0 (socket says 172.21.15.172:50784) 2024-09-10T08:52:00.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.086+0000 7f3094ddc640 1 -- 172.21.15.172:0/2524011275 learned_addr learned my addr 172.21.15.172:0/2524011275 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:00.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.086+0000 7f308f7fe640 1 --2- 172.21.15.172:0/2524011275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3090102570 0x7f3090198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.086+0000 7f308ffff640 1 --2- 172.21.15.172:0/2524011275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3090100e20 0x7f30901983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.086+0000 7f3094ddc640 1 -- 172.21.15.172:0/2524011275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3090100e20 msgr2=0x7f30901983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.086+0000 7f3094ddc640 1 --2- 172.21.15.172:0/2524011275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3090100e20 0x7f30901983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.086+0000 7f3094ddc640 1 -- 172.21.15.172:0/2524011275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3090102570 msgr2=0x7f3090198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.086+0000 7f3094ddc640 1 --2- 172.21.15.172:0/2524011275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3090102570 0x7f3090198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.087+0000 7f3094ddc640 1 -- 172.21.15.172:0/2524011275 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309019d370 con 0x7f3090102ef0 2024-09-10T08:52:00.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.087+0000 7f3094ddc640 1 --2- 172.21.15.172:0/2524011275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3090102ef0 0x7f309019cc90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3088009dc0 tx=0x7f3088003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:00.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.087+0000 7f308d7fa640 1 -- 172.21.15.172:0/2524011275 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088034040 con 0x7f3090102ef0 2024-09-10T08:52:00.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.087+0000 7f308d7fa640 1 -- 172.21.15.172:0/2524011275 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3088003e50 con 0x7f3090102ef0 2024-09-10T08:52:00.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.087+0000 7f308d7fa640 1 -- 172.21.15.172:0/2524011275 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088004670 con 0x7f3090102ef0 2024-09-10T08:52:00.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.087+0000 7f309683f640 1 -- 172.21.15.172:0/2524011275 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309019d600 con 0x7f3090102ef0 2024-09-10T08:52:00.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.087+0000 7f309683f640 1 -- 172.21.15.172:0/2524011275 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30901a4f40 con 0x7f3090102ef0 2024-09-10T08:52:00.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.090+0000 7f308d7fa640 1 -- 172.21.15.172:0/2524011275 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3088037020 con 0x7f3090102ef0 2024-09-10T08:52:00.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.090+0000 7f3098c18740 1 Processor -- start 2024-09-10T08:52:00.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.090+0000 7f3098c18740 1 -- start start 2024-09-10T08:52:00.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f3098c18740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3887b40 0x55e7d3886090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f3098c18740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d38865d0 0x55e7d3886a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f3098c18740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d3895780 0x55e7d3897b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f3098c18740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e7d37ec2f0 con 0x55e7d3887b40 2024-09-10T08:52:00.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f3098c18740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e7d3793cf0 con 0x55e7d38865d0 2024-09-10T08:52:00.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f3098c18740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e7d37e82b0 con 0x55e7d3895780 2024-09-10T08:52:00.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f308f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3887b40 0x55e7d3886090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f308f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3887b40 0x55e7d3886090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50788/0 (socket says 172.21.15.172:50788) 2024-09-10T08:52:00.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f308f7fe640 1 -- 172.21.15.172:0/2372087119 learned_addr learned my addr 172.21.15.172:0/2372087119 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:00.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f308d7fa640 1 --2- 172.21.15.172:0/2524011275 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30780a2c00 0x7f30780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f308ffff640 1 --2- 172.21.15.172:0/2372087119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d38865d0 0x55e7d3886a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f308f7fe640 1 --2- 172.21.15.172:0/2372087119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d3895780 0x55e7d3897b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f3094ddc640 1 --2- 172.21.15.172:0/2524011275 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30780a2c00 0x7f30780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f308d7fa640 1 -- 172.21.15.172:0/2524011275 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f30880f0190 con 0x7f3090102ef0 2024-09-10T08:52:00.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f308f7fe640 1 -- 172.21.15.172:0/2372087119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d3895780 msgr2=0x55e7d3897b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:00.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f308f7fe640 1 --2- 172.21.15.172:0/2372087119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d3895780 0x55e7d3897b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f308f7fe640 1 -- 172.21.15.172:0/2372087119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d38865d0 msgr2=0x55e7d3886a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:00.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f308f7fe640 1 --2- 172.21.15.172:0/2372087119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d38865d0 0x55e7d3886a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.091+0000 7f308f7fe640 1 -- 172.21.15.172:0/2372087119 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7d37783a0 con 0x55e7d3887b40 2024-09-10T08:52:00.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.092+0000 7f308f7fe640 1 --2- 172.21.15.172:0/2372087119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3887b40 0x55e7d3886090 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x55e7d3887070 tx=0x7f3084009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d036c645374e0502 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:00.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.092+0000 7f3094ddc640 1 --2- 172.21.15.172:0/2524011275 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30780a2c00 0x7f30780a50c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f30880091b0 tx=0x7f3088087310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:00.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.092+0000 7f30767fc640 1 -- 172.21.15.172:0/2372087119 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084013130 con 0x55e7d3887b40 2024-09-10T08:52:00.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.092+0000 7f30767fc640 1 -- 172.21.15.172:0/2372087119 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3084013e50 con 0x55e7d3887b40 2024-09-10T08:52:00.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.092+0000 7f30767fc640 1 -- 172.21.15.172:0/2372087119 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30840047a0 con 0x55e7d3887b40 2024-09-10T08:52:00.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.093+0000 7f3098c18740 1 -- 172.21.15.172:0/2372087119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3887b40 msgr2=0x55e7d3886090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.093+0000 7f3098c18740 1 --2- 172.21.15.172:0/2372087119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3887b40 0x55e7d3886090 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x55e7d3887070 tx=0x7f3084009830 comp rx=0 tx=0).stop 2024-09-10T08:52:00.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.093+0000 7f3098c18740 1 -- 172.21.15.172:0/2372087119 shutdown_connections 2024-09-10T08:52:00.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.093+0000 7f3098c18740 1 --2- 172.21.15.172:0/2372087119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d3895780 0x55e7d3897b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.093+0000 7f3098c18740 1 --2- 172.21.15.172:0/2372087119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d38865d0 0x55e7d3886a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.093+0000 7f3098c18740 1 --2- 172.21.15.172:0/2372087119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3887b40 0x55e7d3886090 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.093+0000 7f3098c18740 1 -- 172.21.15.172:0/2372087119 >> 172.21.15.172:0/2372087119 conn(0x55e7d384f310 msgr2=0x55e7d3882c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:00.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.093+0000 7f3098c18740 1 -- 172.21.15.172:0/2372087119 shutdown_connections 2024-09-10T08:52:00.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.093+0000 7f3098c18740 1 -- 172.21.15.172:0/2372087119 wait complete. 2024-09-10T08:52:00.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.093+0000 7f3098c18740 1 -- start start 2024-09-10T08:52:00.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.093+0000 7f3098c18740 1 Processor -- start 2024-09-10T08:52:00.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f3098c18740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d38865d0 0x55e7d3894fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f3098c18740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3887b40 0x55e7d38924c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f3098c18740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d3895780 0x55e7d3892a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f3098c18740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e7d3898420 con 0x55e7d3887b40 2024-09-10T08:52:00.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f3098c18740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e7d37a7060 con 0x55e7d3895780 2024-09-10T08:52:00.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f3098c18740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e7d38985a0 con 0x55e7d38865d0 2024-09-10T08:52:00.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f3098c18740 10 client.0 _ll_register_callbacks cb 0x55e7d34fa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:00.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f308f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d38865d0 0x55e7d3894fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f308f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d38865d0 0x55e7d3894fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:59758/0 (socket says 172.21.15.172:59758) 2024-09-10T08:52:00.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f308f7fe640 1 -- 172.21.15.172:0/2728656950 learned_addr learned my addr 172.21.15.172:0/2728656950 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:00.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f308f7fe640 1 --2- 172.21.15.172:0/2728656950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d3895780 0x55e7d3892a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f308ffff640 1 --2- 172.21.15.172:0/2728656950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3887b40 0x55e7d38924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f308f7fe640 1 -- 172.21.15.172:0/2728656950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d3895780 msgr2=0x55e7d3892a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:00.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f308f7fe640 1 --2- 172.21.15.172:0/2728656950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d3895780 0x55e7d3892a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f308f7fe640 1 -- 172.21.15.172:0/2728656950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3887b40 msgr2=0x55e7d38924c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:00.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f308f7fe640 1 --2- 172.21.15.172:0/2728656950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3887b40 0x55e7d38924c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f308f7fe640 1 -- 172.21.15.172:0/2728656950 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7d355d3c0 con 0x55e7d38865d0 2024-09-10T08:52:00.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.094+0000 7f308ffff640 1 --2- 172.21.15.172:0/2728656950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3887b40 0x55e7d38924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:00.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.095+0000 7f308f7fe640 1 --2- 172.21.15.172:0/2728656950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d38865d0 0x55e7d3894fa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f30840052c0 tx=0x7f3084005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faf90f9cfeffadf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:00.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.095+0000 7f306f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:52:00.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.095+0000 7f306f7fe640 1 -- 172.21.15.172:0/2728656950 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308400fc60 con 0x55e7d38865d0 2024-09-10T08:52:00.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.095+0000 7f306f7fe640 1 -- 172.21.15.172:0/2728656950 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f308400fe00 con 0x55e7d38865d0 2024-09-10T08:52:00.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.095+0000 7f3098c18740 1 -- 172.21.15.172:0/2728656950 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3084013e50 con 0x55e7d38865d0 2024-09-10T08:52:00.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.096+0000 7f306f7fe640 1 -- 172.21.15.172:0/2728656950 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3084004680 con 0x55e7d38865d0 2024-09-10T08:52:00.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.096+0000 7f3098c18740 10 client.26218 fetch_fsmap learned FSMap version 35 2024-09-10T08:52:00.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.096+0000 7f3098c18740 1 -- 172.21.15.172:0/2728656950 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e7d3899b30 con 0x55e7d38865d0 2024-09-10T08:52:00.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.096+0000 7f306f7fe640 1 -- 172.21.15.172:0/2728656950 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3084004cd0 con 0x55e7d38865d0 2024-09-10T08:52:00.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.096+0000 7f306f7fe640 1 -- 172.21.15.172:0/2728656950 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f3084021840 con 0x55e7d38865d0 2024-09-10T08:52:00.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.096+0000 7f3098c18740 10 client.26218 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T08:52:00.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.096+0000 7f3098c18740 10 client.26218 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:52:00.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.096+0000 7f3098c18740 20 client.26218 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:00.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.096+0000 7f3098c18740 1 --2- 172.21.15.172:0/2728656950 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e7d389aff0 0x55e7d389d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.096+0000 7f3098c18740 4 client.26218 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:00.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.096+0000 7f3098c18740 1 -- 172.21.15.172:0/2728656950 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3084013e50 con 0x55e7d389aff0 2024-09-10T08:52:00.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.097+0000 7f308ffff640 1 --2- 172.21.15.172:0/2728656950 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e7d389aff0 0x55e7d389d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.097+0000 7f308ffff640 1 --2- 172.21.15.172:0/2728656950 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e7d389aff0 0x55e7d389d410 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9322b02fc831e058 server_cookie=bd34145d577c8acb in_seq=0 out_seq=0 2024-09-10T08:52:00.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.097+0000 7f306f7fe640 10 client.26218 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:52:00.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.098+0000 7f306f7fe640 1 -- 172.21.15.172:0/2728656950 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f3084013e50 con 0x55e7d389aff0 2024-09-10T08:52:00.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.098+0000 7f306f7fe640 10 client.26218 handle_command_reply: tid=0 2024-09-10T08:52:00.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.098+0000 7f3098c18740 1 client.26218 shutdown 2024-09-10T08:52:00.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.098+0000 7f3098c18740 1 -- 172.21.15.172:0/2728656950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d38865d0 msgr2=0x55e7d3894fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.098+0000 7f3098c18740 1 --2- 172.21.15.172:0/2728656950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d38865d0 0x55e7d3894fa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f30840052c0 tx=0x7f3084005f90 comp rx=0 tx=0).stop 2024-09-10T08:52:00.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.098+0000 7f3098c18740 1 -- 172.21.15.172:0/2728656950 shutdown_connections 2024-09-10T08:52:00.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.098+0000 7f3098c18740 1 --2- 172.21.15.172:0/2728656950 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e7d389aff0 0x55e7d389d410 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.098+0000 7f3098c18740 1 --2- 172.21.15.172:0/2728656950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d3895780 0x55e7d3892a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.098+0000 7f3098c18740 1 --2- 172.21.15.172:0/2728656950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3887b40 0x55e7d38924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.098+0000 7f306f7fe640 0 client.26218 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:52:00.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.098+0000 7f3098c18740 1 --2- 172.21.15.172:0/2728656950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d38865d0 0x55e7d3894fa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.098+0000 7f3098c18740 1 -- 172.21.15.172:0/2728656950 >> 172.21.15.172:0/2728656950 conn(0x55e7d384f310 msgr2=0x55e7d3888990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:00.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.098+0000 7f3098c18740 1 -- 172.21.15.172:0/2728656950 shutdown_connections 2024-09-10T08:52:00.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.098+0000 7f3098c18740 1 -- 172.21.15.172:0/2728656950 wait complete. 2024-09-10T08:52:00.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.098+0000 7f3098c18740 20 client.26218 trim_cache size 0 max 16384 2024-09-10T08:52:00.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.112+0000 7f3098c18740 1 Processor -- start 2024-09-10T08:52:00.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.112+0000 7f3098c18740 1 -- start start 2024-09-10T08:52:00.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.112+0000 7f3098c18740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d38865d0 0x55e7d38814e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.112+0000 7f3098c18740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d3887b40 0x55e7d3894d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.112+0000 7f3098c18740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3895780 0x55e7d38bd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.112+0000 7f3098c18740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30901ac790 con 0x55e7d3895780 2024-09-10T08:52:00.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.112+0000 7f3098c18740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30901ae330 con 0x55e7d3887b40 2024-09-10T08:52:00.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.112+0000 7f3098c18740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f30901ace60 con 0x55e7d38865d0 2024-09-10T08:52:00.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.112+0000 7f308f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3895780 0x55e7d38bd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.112+0000 7f308f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3895780 0x55e7d38bd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50800/0 (socket says 172.21.15.172:50800) 2024-09-10T08:52:00.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.112+0000 7f308f7fe640 1 -- 172.21.15.172:0/1370683491 learned_addr learned my addr 172.21.15.172:0/1370683491 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:00.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.112+0000 7f308f7fe640 1 -- 172.21.15.172:0/1370683491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d38865d0 msgr2=0x55e7d38814e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:00.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.112+0000 7f308f7fe640 1 --2- 172.21.15.172:0/1370683491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d38865d0 0x55e7d38814e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.112+0000 7f308f7fe640 1 -- 172.21.15.172:0/1370683491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d3887b40 msgr2=0x55e7d3894d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:00.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.112+0000 7f308f7fe640 1 --2- 172.21.15.172:0/1370683491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d3887b40 0x55e7d3894d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.112+0000 7f308f7fe640 1 -- 172.21.15.172:0/1370683491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7d3883730 con 0x55e7d3895780 2024-09-10T08:52:00.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.113+0000 7f308f7fe640 1 --2- 172.21.15.172:0/1370683491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3895780 0x55e7d38bd0f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3084009fd0 tx=0x7f30840092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a783fe90fabd46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:00.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.113+0000 7f3075ffb640 1 -- 172.21.15.172:0/1370683491 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084009780 con 0x55e7d3895780 2024-09-10T08:52:00.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.113+0000 7f3075ffb640 1 -- 172.21.15.172:0/1370683491 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3084013e50 con 0x55e7d3895780 2024-09-10T08:52:00.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.113+0000 7f3075ffb640 1 -- 172.21.15.172:0/1370683491 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30840105f0 con 0x55e7d3895780 2024-09-10T08:52:00.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.114+0000 7f3098c18740 1 -- 172.21.15.172:0/1370683491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3895780 msgr2=0x55e7d38bd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.114+0000 7f3098c18740 1 --2- 172.21.15.172:0/1370683491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3895780 0x55e7d38bd0f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3084009fd0 tx=0x7f30840092e0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.114+0000 7f3098c18740 1 -- 172.21.15.172:0/1370683491 shutdown_connections 2024-09-10T08:52:00.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.114+0000 7f3098c18740 1 --2- 172.21.15.172:0/1370683491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3895780 0x55e7d38bd0f0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.114+0000 7f3098c18740 1 --2- 172.21.15.172:0/1370683491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d3887b40 0x55e7d3894d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.114+0000 7f3098c18740 1 --2- 172.21.15.172:0/1370683491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d38865d0 0x55e7d38814e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.114+0000 7f3098c18740 1 -- 172.21.15.172:0/1370683491 >> 172.21.15.172:0/1370683491 conn(0x55e7d384f310 msgr2=0x55e7d3891480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:00.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.114+0000 7f3098c18740 1 -- 172.21.15.172:0/1370683491 shutdown_connections 2024-09-10T08:52:00.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.114+0000 7f3098c18740 1 -- 172.21.15.172:0/1370683491 wait complete. 2024-09-10T08:52:00.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.114+0000 7f3098c18740 1 -- start start 2024-09-10T08:52:00.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.115+0000 7f3098c18740 1 Processor -- start 2024-09-10T08:52:00.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.115+0000 7f3098c18740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d38865d0 0x55e7d38a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.115+0000 7f3098c18740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d3887b40 0x55e7d38a33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.115+0000 7f3098c18740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3895780 0x55e7d389fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.115+0000 7f3098c18740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30901adc30 con 0x55e7d3895780 2024-09-10T08:52:00.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.115+0000 7f3098c18740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30901ad560 con 0x55e7d3887b40 2024-09-10T08:52:00.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.115+0000 7f3098c18740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f30901aa770 con 0x55e7d38865d0 2024-09-10T08:52:00.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.115+0000 7f3098c18740 10 client.0 _ll_register_callbacks cb 0x7f308000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:00.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.116+0000 7f308f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3895780 0x55e7d389fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.116+0000 7f308f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3895780 0x55e7d389fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50810/0 (socket says 172.21.15.172:50810) 2024-09-10T08:52:00.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.116+0000 7f308f7fe640 1 -- 172.21.15.172:0/619365864 learned_addr learned my addr 172.21.15.172:0/619365864 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:00.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.116+0000 7f308f7fe640 1 --2- 172.21.15.172:0/619365864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d38865d0 0x55e7d38a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.116+0000 7f308ffff640 1 --2- 172.21.15.172:0/619365864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d3887b40 0x55e7d38a33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.116+0000 7f308f7fe640 1 -- 172.21.15.172:0/619365864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d38865d0 msgr2=0x55e7d38a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:00.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.116+0000 7f308f7fe640 1 --2- 172.21.15.172:0/619365864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d38865d0 0x55e7d38a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.116+0000 7f308f7fe640 1 -- 172.21.15.172:0/619365864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d3887b40 msgr2=0x55e7d38a33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:00.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.116+0000 7f308f7fe640 1 --2- 172.21.15.172:0/619365864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d3887b40 0x55e7d38a33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.116+0000 7f308f7fe640 1 -- 172.21.15.172:0/619365864 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7d38a03c0 con 0x55e7d3895780 2024-09-10T08:52:00.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.116+0000 7f308f7fe640 1 --2- 172.21.15.172:0/619365864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3895780 0x55e7d389fb90 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f30840053d0 tx=0x7f3084005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9104dd2768c9514 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:00.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.116+0000 7f306f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:00.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.116+0000 7f306f7fe640 1 -- 172.21.15.172:0/619365864 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308400fe70 con 0x55e7d3895780 2024-09-10T08:52:00.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.117+0000 7f306f7fe640 1 -- 172.21.15.172:0/619365864 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f30840105f0 con 0x55e7d3895780 2024-09-10T08:52:00.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.117+0000 7f3098c18740 1 -- 172.21.15.172:0/619365864 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3084013e50 con 0x55e7d3895780 2024-09-10T08:52:00.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.117+0000 7f306f7fe640 1 -- 172.21.15.172:0/619365864 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f30840107f0 con 0x55e7d3895780 2024-09-10T08:52:00.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.117+0000 7f3098c18740 10 client.17112 fetch_fsmap learned FSMap version 35 2024-09-10T08:52:00.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.117+0000 7f3098c18740 1 -- 172.21.15.172:0/619365864 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e7d391cc30 con 0x55e7d3895780 2024-09-10T08:52:00.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.117+0000 7f306f7fe640 1 -- 172.21.15.172:0/619365864 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3084013e50 con 0x55e7d3895780 2024-09-10T08:52:00.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.117+0000 7f306f7fe640 1 -- 172.21.15.172:0/619365864 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f3084034070 con 0x55e7d3895780 2024-09-10T08:52:00.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.117+0000 7f3098c18740 10 client.17112 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T08:52:00.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.117+0000 7f3098c18740 10 client.17112 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:52:00.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.117+0000 7f3098c18740 20 client.17112 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:00.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.117+0000 7f3098c18740 1 --2- 172.21.15.172:0/619365864 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e7d391d3a0 0x55e7d391f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.117+0000 7f3098c18740 4 client.17112 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:52:00.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.117+0000 7f3098c18740 1 -- 172.21.15.172:0/619365864 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7d37f9860 con 0x55e7d391d3a0 2024-09-10T08:52:00.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.118+0000 7f308ffff640 1 --2- 172.21.15.172:0/619365864 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e7d391d3a0 0x55e7d391f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.118+0000 7f308ffff640 1 --2- 172.21.15.172:0/619365864 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e7d391d3a0 0x55e7d391f7c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4390b8859642f2fe server_cookie=8f7fd7dee66de233 in_seq=0 out_seq=0 2024-09-10T08:52:00.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.118+0000 7f306f7fe640 10 client.17112 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:52:00.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.118+0000 7f306f7fe640 1 -- 172.21.15.172:0/619365864 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7d37f9860 con 0x55e7d391d3a0 2024-09-10T08:52:00.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.118+0000 7f306f7fe640 10 client.17112 handle_command_reply: tid=0 2024-09-10T08:52:00.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.118+0000 7f3098c18740 1 client.17112 shutdown 2024-09-10T08:52:00.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.119+0000 7f3098c18740 1 -- 172.21.15.172:0/619365864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3895780 msgr2=0x55e7d389fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.119+0000 7f3098c18740 1 --2- 172.21.15.172:0/619365864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3895780 0x55e7d389fb90 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f30840053d0 tx=0x7f3084005710 comp rx=0 tx=0).stop 2024-09-10T08:52:00.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.119+0000 7f3098c18740 1 -- 172.21.15.172:0/619365864 shutdown_connections 2024-09-10T08:52:00.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.119+0000 7f3098c18740 1 --2- 172.21.15.172:0/619365864 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e7d391d3a0 0x55e7d391f7c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.119+0000 7f3098c18740 1 --2- 172.21.15.172:0/619365864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e7d3895780 0x55e7d389fb90 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.119+0000 7f3098c18740 1 --2- 172.21.15.172:0/619365864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7d3887b40 0x55e7d38a33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.119+0000 7f3098c18740 1 --2- 172.21.15.172:0/619365864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e7d38865d0 0x55e7d38a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.119+0000 7f3098c18740 1 -- 172.21.15.172:0/619365864 >> 172.21.15.172:0/619365864 conn(0x55e7d384f310 msgr2=0x55e7d3891d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:00.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.119+0000 7f306f7fe640 0 client.17112 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:52:00.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.119+0000 7f3098c18740 1 -- 172.21.15.172:0/619365864 shutdown_connections 2024-09-10T08:52:00.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.119+0000 7f3098c18740 1 -- 172.21.15.172:0/619365864 wait complete. 2024-09-10T08:52:00.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.119+0000 7f3098c18740 20 client.17112 trim_cache size 0 max 16384 2024-09-10T08:52:00.126 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:52:00.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.120+0000 7f309683f640 1 -- 172.21.15.172:0/2524011275 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30780a2c00 msgr2=0x7f30780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.120+0000 7f309683f640 1 --2- 172.21.15.172:0/2524011275 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30780a2c00 0x7f30780a50c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f30880091b0 tx=0x7f3088087310 comp rx=0 tx=0).stop 2024-09-10T08:52:00.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.120+0000 7f309683f640 1 -- 172.21.15.172:0/2524011275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3090102ef0 msgr2=0x7f309019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.120+0000 7f309683f640 1 --2- 172.21.15.172:0/2524011275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3090102ef0 0x7f309019cc90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3088009dc0 tx=0x7f3088003ce0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.120+0000 7f309683f640 1 -- 172.21.15.172:0/2524011275 shutdown_connections 2024-09-10T08:52:00.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.120+0000 7f309683f640 1 --2- 172.21.15.172:0/2524011275 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30780a2c00 0x7f30780a50c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.120+0000 7f309683f640 1 --2- 172.21.15.172:0/2524011275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3090102ef0 0x7f309019cc90 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.120+0000 7f309683f640 1 --2- 172.21.15.172:0/2524011275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3090102570 0x7f3090198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.120+0000 7f309683f640 1 --2- 172.21.15.172:0/2524011275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3090100e20 0x7f30901983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.120+0000 7f309683f640 1 -- 172.21.15.172:0/2524011275 >> 172.21.15.172:0/2524011275 conn(0x7f30900fc7d0 msgr2=0x7f30901045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:00.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.120+0000 7f309683f640 1 -- 172.21.15.172:0/2524011275 shutdown_connections 2024-09-10T08:52:00.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:00.120+0000 7f309683f640 1 -- 172.21.15.172:0/2524011275 wait complete. 2024-09-10T08:52:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.483+0000 7f4bb60d7640 1 -- 172.21.15.172:0/1952978437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bb0101c30 msgr2=0x7f4bb01020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.483+0000 7f4bb60d7640 1 --2- 172.21.15.172:0/1952978437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bb0101c30 0x7f4bb01020b0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009900 tx=0x7f4ba402a410 comp rx=0 tx=0).stop 2024-09-10T08:52:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.483+0000 7f4bb60d7640 1 -- 172.21.15.172:0/1952978437 shutdown_connections 2024-09-10T08:52:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.483+0000 7f4bb60d7640 1 --2- 172.21.15.172:0/1952978437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bb01025f0 0x7f4bb0108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.483+0000 7f4bb60d7640 1 --2- 172.21.15.172:0/1952978437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bb0101c30 0x7f4bb01020b0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.483+0000 7f4bb60d7640 1 --2- 172.21.15.172:0/1952978437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bb0100a40 0x7f4bb0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.483+0000 7f4bb60d7640 1 -- 172.21.15.172:0/1952978437 >> 172.21.15.172:0/1952978437 conn(0x7f4bb00fc1e0 msgr2=0x7f4bb00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.483+0000 7f4bb60d7640 1 -- 172.21.15.172:0/1952978437 shutdown_connections 2024-09-10T08:52:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.483+0000 7f4bb60d7640 1 -- 172.21.15.172:0/1952978437 wait complete. 2024-09-10T08:52:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.484+0000 7f4bb60d7640 1 Processor -- start 2024-09-10T08:52:00.486 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.484+0000 7f4bb60d7640 1 -- start start 2024-09-10T08:52:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.484+0000 7f4bb60d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bb0100a40 0x7f4bb0196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.484+0000 7f4bb60d7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bb0101c30 0x7f4bb0197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.484+0000 7f4bb60d7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bb01025f0 0x7f4bb019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.484+0000 7f4bb60d7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb010c550 con 0x7f4bb0100a40 2024-09-10T08:52:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.484+0000 7f4bb60d7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb010c3d0 con 0x7f4bb01025f0 2024-09-10T08:52:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.484+0000 7f4bb60d7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb010c6d0 con 0x7f4bb0101c30 2024-09-10T08:52:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.485+0000 7f4baf7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bb0100a40 0x7f4bb0196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.485+0000 7f4baf7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bb0100a40 0x7f4bb0196ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50838/0 (socket says 172.21.15.172:50838) 2024-09-10T08:52:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.485+0000 7f4baf7fe640 1 -- 172.21.15.172:0/3999734329 learned_addr learned my addr 172.21.15.172:0/3999734329 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.485+0000 7f4baeffd640 1 --2- 172.21.15.172:0/3999734329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bb0101c30 0x7f4bb0197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.485+0000 7f4baffff640 1 --2- 172.21.15.172:0/3999734329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bb01025f0 0x7f4bb019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.485+0000 7f4baf7fe640 1 -- 172.21.15.172:0/3999734329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bb0101c30 msgr2=0x7f4bb0197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.485+0000 7f4baf7fe640 1 --2- 172.21.15.172:0/3999734329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bb0101c30 0x7f4bb0197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.485+0000 7f4baf7fe640 1 -- 172.21.15.172:0/3999734329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bb01025f0 msgr2=0x7f4bb019e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.485+0000 7f4baf7fe640 1 --2- 172.21.15.172:0/3999734329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bb01025f0 0x7f4bb019e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.485+0000 7f4baf7fe640 1 -- 172.21.15.172:0/3999734329 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb00698a0 con 0x7f4bb0100a40 2024-09-10T08:52:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.485+0000 7f4baf7fe640 1 --2- 172.21.15.172:0/3999734329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bb0100a40 0x7f4bb0196ae0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000c970 tx=0x7f4ba000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.485+0000 7f4bacff9640 1 -- 172.21.15.172:0/3999734329 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0007c10 con 0x7f4bb0100a40 2024-09-10T08:52:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.485+0000 7f4bb60d7640 1 -- 172.21.15.172:0/3999734329 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb0069b90 con 0x7f4bb0100a40 2024-09-10T08:52:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.485+0000 7f4bacff9640 1 -- 172.21.15.172:0/3999734329 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ba0007db0 con 0x7f4bb0100a40 2024-09-10T08:52:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.485+0000 7f4bacff9640 1 -- 172.21.15.172:0/3999734329 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba000f7d0 con 0x7f4bb0100a40 2024-09-10T08:52:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.485+0000 7f4bb60d7640 1 -- 172.21.15.172:0/3999734329 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb006a100 con 0x7f4bb0100a40 2024-09-10T08:52:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.488+0000 7f4bacff9640 1 -- 172.21.15.172:0/3999734329 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4ba000fa30 con 0x7f4bb0100a40 2024-09-10T08:52:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.488+0000 7f4bb8e72740 1 Processor -- start 2024-09-10T08:52:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.488+0000 7f4bb8e72740 1 -- start start 2024-09-10T08:52:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.488+0000 7f4bb8e72740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991bea9a0 0x564991beada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.488+0000 7f4bb8e72740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 0x564991befda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.488+0000 7f4bb8e72740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bf0370 0x564991bf2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.488+0000 7f4bb8e72740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564991b4e2f0 con 0x564991beb2e0 2024-09-10T08:52:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.488+0000 7f4bb8e72740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564991af6270 con 0x564991bf0370 2024-09-10T08:52:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.488+0000 7f4bb8e72740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564991b4a2b0 con 0x564991bea9a0 2024-09-10T08:52:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.488+0000 7f4baf7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 0x564991befda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.488+0000 7f4baf7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 0x564991befda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50840/0 (socket says 172.21.15.172:50840) 2024-09-10T08:52:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.488+0000 7f4baf7fe640 1 -- 172.21.15.172:0/1912298568 learned_addr learned my addr 172.21.15.172:0/1912298568 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.488+0000 7f4baffff640 1 --2- 172.21.15.172:0/1912298568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991bea9a0 0x564991beada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.489+0000 7f4baeffd640 1 --2- 172.21.15.172:0/1912298568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bf0370 0x564991bf2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.489+0000 7f4baf7fe640 1 -- 172.21.15.172:0/1912298568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991bea9a0 msgr2=0x564991beada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.489+0000 7f4baf7fe640 1 --2- 172.21.15.172:0/1912298568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991bea9a0 0x564991beada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.489+0000 7f4baf7fe640 1 -- 172.21.15.172:0/1912298568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bf0370 msgr2=0x564991bf2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.489+0000 7f4baf7fe640 1 --2- 172.21.15.172:0/1912298568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bf0370 0x564991bf2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.489+0000 7f4baf7fe640 1 -- 172.21.15.172:0/1912298568 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564991af5340 con 0x564991beb2e0 2024-09-10T08:52:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.489+0000 7f4bacff9640 1 --2- 172.21.15.172:0/3999734329 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4b980a2c30 0x7f4b980a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.489+0000 7f4baf7fe640 1 --2- 172.21.15.172:0/1912298568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 0x564991befda0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00d5c10 tx=0x7f4ba00d5d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e669f30a52021d5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.489+0000 7f4baffff640 1 --2- 172.21.15.172:0/3999734329 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4b980a2c30 0x7f4b980a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.489+0000 7f4bacff9640 1 -- 172.21.15.172:0/3999734329 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4ba00cf6f0 con 0x7f4bb0100a40 2024-09-10T08:52:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.489+0000 7f4b95ffb640 1 -- 172.21.15.172:0/1912298568 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0097f20 con 0x564991beb2e0 2024-09-10T08:52:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.489+0000 7f4b95ffb640 1 -- 172.21.15.172:0/1912298568 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ba0018ac0 con 0x564991beb2e0 2024-09-10T08:52:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.489+0000 7f4b95ffb640 1 -- 172.21.15.172:0/1912298568 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba009b5e0 con 0x564991beb2e0 2024-09-10T08:52:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.490+0000 7f4bb8e72740 1 -- 172.21.15.172:0/1912298568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 msgr2=0x564991befda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.490+0000 7f4baffff640 1 --2- 172.21.15.172:0/3999734329 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4b980a2c30 0x7f4b980a50f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8007ed0 tx=0x7f4ba8007680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.490+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/1912298568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 0x564991befda0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00d5c10 tx=0x7f4ba00d5d00 comp rx=0 tx=0).stop 2024-09-10T08:52:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.490+0000 7f4bb8e72740 1 -- 172.21.15.172:0/1912298568 shutdown_connections 2024-09-10T08:52:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.490+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/1912298568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bf0370 0x564991bf2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.490+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/1912298568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 0x564991befda0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.490+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/1912298568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991bea9a0 0x564991beada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.490+0000 7f4bb8e72740 1 -- 172.21.15.172:0/1912298568 >> 172.21.15.172:0/1912298568 conn(0x564991be6010 msgr2=0x564991be6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.490+0000 7f4bb8e72740 1 -- 172.21.15.172:0/1912298568 shutdown_connections 2024-09-10T08:52:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.490+0000 7f4bb8e72740 1 -- 172.21.15.172:0/1912298568 wait complete. 2024-09-10T08:52:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.491+0000 7f4bb8e72740 1 -- start start 2024-09-10T08:52:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.491+0000 7f4bb8e72740 1 Processor -- start 2024-09-10T08:52:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.491+0000 7f4bb8e72740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991bea9a0 0x564991bf4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.491+0000 7f4bb8e72740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 0x564991bf50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.491+0000 7f4bb8e72740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bf0370 0x564991bfc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.491+0000 7f4bb8e72740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564991bf3340 con 0x564991beb2e0 2024-09-10T08:52:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.491+0000 7f4bb8e72740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564991ad7290 con 0x564991bf0370 2024-09-10T08:52:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.491+0000 7f4bb8e72740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564991bf34c0 con 0x564991bea9a0 2024-09-10T08:52:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.491+0000 7f4bb8e72740 10 client.0 _ll_register_callbacks cb 0x564991b08140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.491+0000 7f4baeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 0x564991bf50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.491+0000 7f4baeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 0x564991bf50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50848/0 (socket says 172.21.15.172:50848) 2024-09-10T08:52:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.491+0000 7f4baeffd640 1 -- 172.21.15.172:0/167369629 learned_addr learned my addr 172.21.15.172:0/167369629 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.491+0000 7f4baffff640 1 --2- 172.21.15.172:0/167369629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bf0370 0x564991bfc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.491+0000 7f4baf7fe640 1 --2- 172.21.15.172:0/167369629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991bea9a0 0x564991bf4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.492+0000 7f4baeffd640 1 -- 172.21.15.172:0/167369629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991bea9a0 msgr2=0x564991bf4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.492+0000 7f4baeffd640 1 --2- 172.21.15.172:0/167369629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991bea9a0 0x564991bf4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.492+0000 7f4baeffd640 1 -- 172.21.15.172:0/167369629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bf0370 msgr2=0x564991bfc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.492+0000 7f4baeffd640 1 --2- 172.21.15.172:0/167369629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bf0370 0x564991bfc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.492+0000 7f4baeffd640 1 -- 172.21.15.172:0/167369629 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564991861210 con 0x564991beb2e0 2024-09-10T08:52:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.492+0000 7f4baeffd640 1 --2- 172.21.15.172:0/167369629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 0x564991bf50d0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4005ec0 tx=0x7f4ba40317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21fc919811b95e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.492+0000 7f4b8effd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.492+0000 7f4b8effd640 1 -- 172.21.15.172:0/167369629 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4035050 con 0x564991beb2e0 2024-09-10T08:52:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.492+0000 7f4bb8e72740 1 -- 172.21.15.172:0/167369629 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564991b5ba20 con 0x564991beb2e0 2024-09-10T08:52:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.492+0000 7f4b8effd640 1 -- 172.21.15.172:0/167369629 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4ba4004450 con 0x564991beb2e0 2024-09-10T08:52:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.492+0000 7f4b8effd640 1 -- 172.21.15.172:0/167369629 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4ba40035f0 con 0x564991beb2e0 2024-09-10T08:52:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.492+0000 7f4bb8e72740 10 client.17124 fetch_fsmap learned FSMap version 35 2024-09-10T08:52:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.492+0000 7f4bb8e72740 1 -- 172.21.15.172:0/167369629 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56499192d620 con 0x564991beb2e0 2024-09-10T08:52:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.492+0000 7f4b8effd640 1 -- 172.21.15.172:0/167369629 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564991b5ba20 con 0x564991beb2e0 2024-09-10T08:52:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.492+0000 7f4b8effd640 1 -- 172.21.15.172:0/167369629 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f4ba4043070 con 0x564991beb2e0 2024-09-10T08:52:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.493+0000 7f4bb8e72740 10 client.17124 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T08:52:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.493+0000 7f4bb8e72740 10 client.17124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.493+0000 7f4bb8e72740 20 client.17124 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.493+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/167369629 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564991c00820 0x564991c02c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.493+0000 7f4bb8e72740 4 client.17124 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.493+0000 7f4bb8e72740 1 -- 172.21.15.172:0/167369629 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564991b55d70 con 0x564991c00820 2024-09-10T08:52:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.493+0000 7f4baf7fe640 1 --2- 172.21.15.172:0/167369629 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564991c00820 0x564991c02c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.493+0000 7f4b8effd640 10 client.17124 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.493+0000 7f4baf7fe640 1 --2- 172.21.15.172:0/167369629 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564991c00820 0x564991c02c50 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92919b0ee071e139 server_cookie=53f897e5fb97f737 in_seq=0 out_seq=0 2024-09-10T08:52:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.494+0000 7f4b8effd640 1 -- 172.21.15.172:0/167369629 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564991b55d70 con 0x564991c00820 2024-09-10T08:52:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.494+0000 7f4b8effd640 10 client.17124 handle_command_reply: tid=0 2024-09-10T08:52:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.494+0000 7f4bb8e72740 1 client.17124 shutdown 2024-09-10T08:52:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.494+0000 7f4bb8e72740 1 -- 172.21.15.172:0/167369629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 msgr2=0x564991bf50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.494+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/167369629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 0x564991bf50d0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4005ec0 tx=0x7f4ba40317b0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.494+0000 7f4bb8e72740 1 -- 172.21.15.172:0/167369629 shutdown_connections 2024-09-10T08:52:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.494+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/167369629 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564991c00820 0x564991c02c50 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.494+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/167369629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bf0370 0x564991bfc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.494+0000 7f4b8effd640 0 client.17124 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.494+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/167369629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 0x564991bf50d0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.494+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/167369629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991bea9a0 0x564991bf4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.494+0000 7f4bb8e72740 1 -- 172.21.15.172:0/167369629 >> 172.21.15.172:0/167369629 conn(0x564991be6010 msgr2=0x564991be7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.494+0000 7f4bb8e72740 1 -- 172.21.15.172:0/167369629 shutdown_connections 2024-09-10T08:52:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.494+0000 7f4bb8e72740 1 -- 172.21.15.172:0/167369629 wait complete. 2024-09-10T08:52:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.495+0000 7f4bb8e72740 20 client.17124 trim_cache size 0 max 16384 2024-09-10T08:52:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.512+0000 7f4bb8e72740 1 Processor -- start 2024-09-10T08:52:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.512+0000 7f4bb8e72740 1 -- start start 2024-09-10T08:52:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.512+0000 7f4bb8e72740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bea9a0 0x564991be7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.512+0000 7f4bb8e72740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 0x564991bfc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.512+0000 7f4bb8e72740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991bf0370 0x564991c07550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.512+0000 7f4bb8e72740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb01aa3f0 con 0x564991beb2e0 2024-09-10T08:52:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.512+0000 7f4bb8e72740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb01abf90 con 0x564991bea9a0 2024-09-10T08:52:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.512+0000 7f4bb8e72740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb01aaac0 con 0x564991bf0370 2024-09-10T08:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.512+0000 7f4baeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 0x564991bfc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.512+0000 7f4baeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 0x564991bfc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50864/0 (socket says 172.21.15.172:50864) 2024-09-10T08:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.513+0000 7f4baeffd640 1 -- 172.21.15.172:0/1338610291 learned_addr learned my addr 172.21.15.172:0/1338610291 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.513+0000 7f4baffff640 1 --2- 172.21.15.172:0/1338610291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991bf0370 0x564991c07550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.513+0000 7f4baf7fe640 1 --2- 172.21.15.172:0/1338610291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bea9a0 0x564991be7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.513+0000 7f4baffff640 1 -- 172.21.15.172:0/1338610291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bea9a0 msgr2=0x564991be7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.513+0000 7f4baffff640 1 --2- 172.21.15.172:0/1338610291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bea9a0 0x564991be7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.513+0000 7f4baffff640 1 -- 172.21.15.172:0/1338610291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 msgr2=0x564991bfc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.513+0000 7f4baffff640 1 --2- 172.21.15.172:0/1338610291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 0x564991bfc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.513+0000 7f4baffff640 1 -- 172.21.15.172:0/1338610291 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564991be5e30 con 0x564991bf0370 2024-09-10T08:52:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.513+0000 7f4baffff640 1 --2- 172.21.15.172:0/1338610291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991bf0370 0x564991c07550 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8011ca0 tx=0x7f4ba8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4b4b3d13443fc34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.513+0000 7f4b957fa640 1 -- 172.21.15.172:0/1338610291 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba801c6d0 con 0x564991bf0370 2024-09-10T08:52:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.513+0000 7f4b957fa640 1 -- 172.21.15.172:0/1338610291 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ba801cd20 con 0x564991bf0370 2024-09-10T08:52:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.513+0000 7f4b957fa640 1 -- 172.21.15.172:0/1338610291 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8026e40 con 0x564991bf0370 2024-09-10T08:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.514+0000 7f4bb8e72740 1 -- 172.21.15.172:0/1338610291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991bf0370 msgr2=0x564991c07550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.514+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/1338610291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991bf0370 0x564991c07550 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8011ca0 tx=0x7f4ba8014000 comp rx=0 tx=0).stop 2024-09-10T08:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.514+0000 7f4bb8e72740 1 -- 172.21.15.172:0/1338610291 shutdown_connections 2024-09-10T08:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.514+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/1338610291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991bf0370 0x564991c07550 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.514+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/1338610291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991beb2e0 0x564991bfc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.514+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/1338610291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bea9a0 0x564991be7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.514+0000 7f4bb8e72740 1 -- 172.21.15.172:0/1338610291 >> 172.21.15.172:0/1338610291 conn(0x564991be6010 msgr2=0x564991be63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.514+0000 7f4bb8e72740 1 -- 172.21.15.172:0/1338610291 shutdown_connections 2024-09-10T08:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.514+0000 7f4bb8e72740 1 -- 172.21.15.172:0/1338610291 wait complete. 2024-09-10T08:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.514+0000 7f4bb8e72740 1 -- start start 2024-09-10T08:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4bb8e72740 1 Processor -- start 2024-09-10T08:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4bb8e72740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991bea9a0 0x564991bfb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4bb8e72740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991beb2e0 0x564991bfbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4bb8e72740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bf0370 0x564991bf9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4bb8e72740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb01ab890 con 0x564991bea9a0 2024-09-10T08:52:00.521 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4bb8e72740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb01ab1c0 con 0x564991bf0370 2024-09-10T08:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4bb8e72740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb01a83d0 con 0x564991beb2e0 2024-09-10T08:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4bb8e72740 10 client.0 _ll_register_callbacks cb 0x7f4ba4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4baf7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991bea9a0 0x564991bfb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4baf7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991bea9a0 0x564991bfb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50876/0 (socket says 172.21.15.172:50876) 2024-09-10T08:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4baf7fe640 1 -- 172.21.15.172:0/4156250456 learned_addr learned my addr 172.21.15.172:0/4156250456 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4baeffd640 1 --2- 172.21.15.172:0/4156250456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991beb2e0 0x564991bfbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4baffff640 1 --2- 172.21.15.172:0/4156250456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bf0370 0x564991bf9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4baf7fe640 1 -- 172.21.15.172:0/4156250456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991beb2e0 msgr2=0x564991bfbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4baf7fe640 1 --2- 172.21.15.172:0/4156250456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991beb2e0 0x564991bfbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4baf7fe640 1 -- 172.21.15.172:0/4156250456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bf0370 msgr2=0x564991bf9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4baf7fe640 1 --2- 172.21.15.172:0/4156250456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bf0370 0x564991bf9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.522 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4baf7fe640 1 -- 172.21.15.172:0/4156250456 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564991bfa730 con 0x564991bea9a0 2024-09-10T08:52:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.515+0000 7f4baeffd640 1 --2- 172.21.15.172:0/4156250456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991beb2e0 0x564991bfbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.516+0000 7f4baf7fe640 1 --2- 172.21.15.172:0/4156250456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991bea9a0 0x564991bfb970 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0007590 tx=0x7f4ba0007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b3c60e1a53202e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.516+0000 7f4b8effd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.516+0000 7f4b8effd640 1 -- 172.21.15.172:0/4156250456 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba009b7c0 con 0x564991bea9a0 2024-09-10T08:52:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.516+0000 7f4b8effd640 1 -- 172.21.15.172:0/4156250456 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4ba00cf460 con 0x564991bea9a0 2024-09-10T08:52:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.516+0000 7f4bb8e72740 1 -- 172.21.15.172:0/4156250456 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ba801cd20 con 0x564991bea9a0 2024-09-10T08:52:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.516+0000 7f4b8effd640 1 -- 172.21.15.172:0/4156250456 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4ba009ba50 con 0x564991bea9a0 2024-09-10T08:52:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.516+0000 7f4bb8e72740 10 client.17130 fetch_fsmap learned FSMap version 35 2024-09-10T08:52:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.516+0000 7f4bb8e72740 1 -- 172.21.15.172:0/4156250456 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564991c6bfb0 con 0x564991bea9a0 2024-09-10T08:52:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.516+0000 7f4b8effd640 1 -- 172.21.15.172:0/4156250456 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ba801cd20 con 0x564991bea9a0 2024-09-10T08:52:00.523 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.516+0000 7f4b8effd640 1 -- 172.21.15.172:0/4156250456 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f4ba00d2ce0 con 0x564991bea9a0 2024-09-10T08:52:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.516+0000 7f4bb8e72740 10 client.17130 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T08:52:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.516+0000 7f4bb8e72740 10 client.17130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.516+0000 7f4bb8e72740 20 client.17130 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.516+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/4156250456 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564991c6c510 0x564991c6e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.516+0000 7f4bb8e72740 4 client.17130 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:52:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.516+0000 7f4bb8e72740 1 -- 172.21.15.172:0/4156250456 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564991ad2900 con 0x564991c6c510 2024-09-10T08:52:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.517+0000 7f4baeffd640 1 --2- 172.21.15.172:0/4156250456 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564991c6c510 0x564991c6e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.517+0000 7f4baeffd640 1 --2- 172.21.15.172:0/4156250456 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564991c6c510 0x564991c6e900 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f897724d5b6be79 server_cookie=ca13945d6e981b4c in_seq=0 out_seq=0 2024-09-10T08:52:00.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.517+0000 7f4b8effd640 10 client.17130 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:00.525 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.523+0000 7f4b8effd640 1 -- 172.21.15.172:0/4156250456 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564991ad2900 con 0x564991c6c510 2024-09-10T08:52:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.523+0000 7f4b8effd640 10 client.17130 handle_command_reply: tid=0 2024-09-10T08:52:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.523+0000 7f4bb8e72740 1 client.17130 shutdown 2024-09-10T08:52:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.523+0000 7f4bb8e72740 1 -- 172.21.15.172:0/4156250456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991bea9a0 msgr2=0x564991bfb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.523+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/4156250456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991bea9a0 0x564991bfb970 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0007590 tx=0x7f4ba0007f60 comp rx=0 tx=0).stop 2024-09-10T08:52:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.523+0000 7f4bb8e72740 1 -- 172.21.15.172:0/4156250456 shutdown_connections 2024-09-10T08:52:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.523+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/4156250456 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564991c6c510 0x564991c6e900 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.523+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/4156250456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564991bf0370 0x564991bf9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.523+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/4156250456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564991beb2e0 0x564991bfbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.523+0000 7f4bb8e72740 1 --2- 172.21.15.172:0/4156250456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564991bea9a0 0x564991bfb970 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.523+0000 7f4b8effd640 0 client.17130 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.523+0000 7f4bb8e72740 1 -- 172.21.15.172:0/4156250456 >> 172.21.15.172:0/4156250456 conn(0x564991be6010 msgr2=0x564991be63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.523+0000 7f4bb8e72740 1 -- 172.21.15.172:0/4156250456 shutdown_connections 2024-09-10T08:52:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.524+0000 7f4bb8e72740 1 -- 172.21.15.172:0/4156250456 wait complete. 2024-09-10T08:52:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.524+0000 7f4bb8e72740 20 client.17130 trim_cache size 0 max 16384 2024-09-10T08:52:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:52:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:52:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:52:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:52:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.524+0000 7f4bb60d7640 1 -- 172.21.15.172:0/3999734329 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4b980a2c30 msgr2=0x7f4b980a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.524+0000 7f4bb60d7640 1 --2- 172.21.15.172:0/3999734329 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4b980a2c30 0x7f4b980a50f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8007ed0 tx=0x7f4ba8007680 comp rx=0 tx=0).stop 2024-09-10T08:52:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.524+0000 7f4bb60d7640 1 -- 172.21.15.172:0/3999734329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bb0100a40 msgr2=0x7f4bb0196ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.524+0000 7f4bb60d7640 1 --2- 172.21.15.172:0/3999734329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bb0100a40 0x7f4bb0196ae0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000c970 tx=0x7f4ba000ce30 comp rx=0 tx=0).stop 2024-09-10T08:52:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.524+0000 7f4bb60d7640 1 -- 172.21.15.172:0/3999734329 shutdown_connections 2024-09-10T08:52:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.524+0000 7f4bb60d7640 1 --2- 172.21.15.172:0/3999734329 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4b980a2c30 0x7f4b980a50f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.524+0000 7f4bb60d7640 1 --2- 172.21.15.172:0/3999734329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bb01025f0 0x7f4bb019e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.524+0000 7f4bb60d7640 1 --2- 172.21.15.172:0/3999734329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bb0101c30 0x7f4bb0197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.524+0000 7f4bb60d7640 1 --2- 172.21.15.172:0/3999734329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bb0100a40 0x7f4bb0196ae0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.524+0000 7f4bb60d7640 1 -- 172.21.15.172:0/3999734329 >> 172.21.15.172:0/3999734329 conn(0x7f4bb00fc1e0 msgr2=0x7f4bb00fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.524+0000 7f4bb60d7640 1 -- 172.21.15.172:0/3999734329 shutdown_connections 2024-09-10T08:52:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:00.524+0000 7f4bb60d7640 1 -- 172.21.15.172:0/3999734329 wait complete. 2024-09-10T08:52:00.897 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:52:01.204 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:52:01.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.727+0000 7f6d48738640 1 -- 172.21.15.172:0/4277327666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4010e4f0 msgr2=0x7f6d401108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:01.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.727+0000 7f6d48738640 1 --2- 172.21.15.172:0/4277327666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4010e4f0 0x7f6d401108e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009ca0 tx=0x7f6d3802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.727+0000 7f6d48738640 1 -- 172.21.15.172:0/4277327666 shutdown_connections 2024-09-10T08:52:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.727+0000 7f6d48738640 1 --2- 172.21.15.172:0/4277327666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4010e4f0 0x7f6d401108e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.727+0000 7f6d48738640 1 --2- 172.21.15.172:0/4277327666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d401012f0 0x7f6d4010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.727+0000 7f6d48738640 1 --2- 172.21.15.172:0/4277327666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d40100940 0x7f6d40100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.727+0000 7f6d48738640 1 -- 172.21.15.172:0/4277327666 >> 172.21.15.172:0/4277327666 conn(0x7f6d400fc7d0 msgr2=0x7f6d400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.727+0000 7f6d48738640 1 -- 172.21.15.172:0/4277327666 shutdown_connections 2024-09-10T08:52:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.727+0000 7f6d48738640 1 -- 172.21.15.172:0/4277327666 wait complete. 2024-09-10T08:52:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.728+0000 7f6d48738640 1 Processor -- start 2024-09-10T08:52:01.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.728+0000 7f6d48738640 1 -- start start 2024-09-10T08:52:01.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.728+0000 7f6d48738640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d40100940 0x7f6d4019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.728+0000 7f6d48738640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d401012f0 0x7f6d4019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.728+0000 7f6d48738640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4010e4f0 0x7f6d4019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.728+0000 7f6d48738640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d40114020 con 0x7f6d4010e4f0 2024-09-10T08:52:01.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.728+0000 7f6d48738640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d40113ea0 con 0x7f6d401012f0 2024-09-10T08:52:01.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.728+0000 7f6d48738640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401141a0 con 0x7f6d40100940 2024-09-10T08:52:01.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.728+0000 7f6d46cd5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4010e4f0 0x7f6d4019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.728+0000 7f6d46cd5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4010e4f0 0x7f6d4019ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50888/0 (socket says 172.21.15.172:50888) 2024-09-10T08:52:01.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.728+0000 7f6d46cd5640 1 -- 172.21.15.172:0/3429930512 learned_addr learned my addr 172.21.15.172:0/3429930512 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:01.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.728+0000 7f6d464d4640 1 --2- 172.21.15.172:0/3429930512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d40100940 0x7f6d4019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.728+0000 7f6d45cd3640 1 --2- 172.21.15.172:0/3429930512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d401012f0 0x7f6d4019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.729+0000 7f6d46cd5640 1 -- 172.21.15.172:0/3429930512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d40100940 msgr2=0x7f6d4019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:01.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.729+0000 7f6d46cd5640 1 --2- 172.21.15.172:0/3429930512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d40100940 0x7f6d4019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.729+0000 7f6d46cd5640 1 -- 172.21.15.172:0/3429930512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d401012f0 msgr2=0x7f6d4019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:01.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.729+0000 7f6d46cd5640 1 --2- 172.21.15.172:0/3429930512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d401012f0 0x7f6d4019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.729+0000 7f6d46cd5640 1 -- 172.21.15.172:0/3429930512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4019f500 con 0x7f6d4010e4f0 2024-09-10T08:52:01.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.729+0000 7f6d46cd5640 1 --2- 172.21.15.172:0/3429930512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4010e4f0 0x7f6d4019ee20 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f6d380098d0 tx=0x7f6d38032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:01.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.729+0000 7f6d377fe640 1 -- 172.21.15.172:0/3429930512 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38003d30 con 0x7f6d4010e4f0 2024-09-10T08:52:01.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.729+0000 7f6d377fe640 1 -- 172.21.15.172:0/3429930512 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d380044f0 con 0x7f6d4010e4f0 2024-09-10T08:52:01.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.729+0000 7f6d48738640 1 -- 172.21.15.172:0/3429930512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4019f790 con 0x7f6d4010e4f0 2024-09-10T08:52:01.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.729+0000 7f6d377fe640 1 -- 172.21.15.172:0/3429930512 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38030c30 con 0x7f6d4010e4f0 2024-09-10T08:52:01.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.729+0000 7f6d48738640 1 -- 172.21.15.172:0/3429930512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d401a70d0 con 0x7f6d4010e4f0 2024-09-10T08:52:01.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.732+0000 7f6d377fe640 1 -- 172.21.15.172:0/3429930512 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6d380046a0 con 0x7f6d4010e4f0 2024-09-10T08:52:01.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.732+0000 7f6d4ab11740 1 Processor -- start 2024-09-10T08:52:01.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.732+0000 7f6d4ab11740 1 -- start start 2024-09-10T08:52:01.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d4ab11740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeeeab40 0x5589aeee9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d4ab11740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeee95d0 0x5589aeee9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d4ab11740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeef4370 0x5589aeef6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d4ab11740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5589aee4f2f0 con 0x5589aeeeab40 2024-09-10T08:52:01.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d4ab11740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5589aedf6cf0 con 0x5589aeee95d0 2024-09-10T08:52:01.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d4ab11740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5589aee4b2b0 con 0x5589aeef4370 2024-09-10T08:52:01.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d45cd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeeeab40 0x5589aeee9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d45cd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeeeab40 0x5589aeee9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50902/0 (socket says 172.21.15.172:50902) 2024-09-10T08:52:01.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d45cd3640 1 -- 172.21.15.172:0/1701228439 learned_addr learned my addr 172.21.15.172:0/1701228439 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:01.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d464d4640 1 --2- 172.21.15.172:0/1701228439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeee95d0 0x5589aeee9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d377fe640 1 --2- 172.21.15.172:0/3429930512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6d2c0a2c00 0x7f6d2c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d45cd3640 1 --2- 172.21.15.172:0/1701228439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeef4370 0x5589aeef6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d45cd3640 1 -- 172.21.15.172:0/1701228439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeef4370 msgr2=0x5589aeef6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:01.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d45cd3640 1 --2- 172.21.15.172:0/1701228439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeef4370 0x5589aeef6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d45cd3640 1 -- 172.21.15.172:0/1701228439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeee95d0 msgr2=0x5589aeee9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:01.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d45cd3640 1 --2- 172.21.15.172:0/1701228439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeee95d0 0x5589aeee9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d45cd3640 1 -- 172.21.15.172:0/1701228439 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589aeddb3a0 con 0x5589aeeeab40 2024-09-10T08:52:01.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d377fe640 1 -- 172.21.15.172:0/3429930512 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6d380f0960 con 0x7f6d4010e4f0 2024-09-10T08:52:01.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.733+0000 7f6d46cd5640 1 --2- 172.21.15.172:0/3429930512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6d2c0a2c00 0x7f6d2c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.734+0000 7f6d45cd3640 1 --2- 172.21.15.172:0/1701228439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeeeab40 0x5589aeee9090 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f6d300058a0 tx=0x7f6d3000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1821a584319584b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:01.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.734+0000 7f6d34ff9640 1 -- 172.21.15.172:0/1701228439 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30004030 con 0x5589aeeeab40 2024-09-10T08:52:01.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.734+0000 7f6d46cd5640 1 --2- 172.21.15.172:0/3429930512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6d2c0a2c00 0x7f6d2c0a50c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6d380c56c0 tx=0x7f6d3802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:01.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.734+0000 7f6d34ff9640 1 -- 172.21.15.172:0/1701228439 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d300041d0 con 0x5589aeeeab40 2024-09-10T08:52:01.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.734+0000 7f6d34ff9640 1 -- 172.21.15.172:0/1701228439 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3000f6e0 con 0x5589aeeeab40 2024-09-10T08:52:01.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.735+0000 7f6d4ab11740 1 -- 172.21.15.172:0/1701228439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeeeab40 msgr2=0x5589aeee9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:01.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.735+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/1701228439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeeeab40 0x5589aeee9090 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f6d300058a0 tx=0x7f6d3000b7c0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.735+0000 7f6d4ab11740 1 -- 172.21.15.172:0/1701228439 shutdown_connections 2024-09-10T08:52:01.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.735+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/1701228439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeef4370 0x5589aeef6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.735+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/1701228439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeee95d0 0x5589aeee9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.735+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/1701228439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeeeab40 0x5589aeee9090 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.735+0000 7f6d4ab11740 1 -- 172.21.15.172:0/1701228439 >> 172.21.15.172:0/1701228439 conn(0x5589aeeb2310 msgr2=0x5589aeee5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:01.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.735+0000 7f6d4ab11740 1 -- 172.21.15.172:0/1701228439 shutdown_connections 2024-09-10T08:52:01.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.735+0000 7f6d4ab11740 1 -- 172.21.15.172:0/1701228439 wait complete. 2024-09-10T08:52:01.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.735+0000 7f6d4ab11740 1 -- start start 2024-09-10T08:52:01.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.736+0000 7f6d4ab11740 1 Processor -- start 2024-09-10T08:52:01.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.736+0000 7f6d4ab11740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeee95d0 0x5589aeef3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.736+0000 7f6d4ab11740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeeeab40 0x5589aeef1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.736+0000 7f6d4ab11740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeef4370 0x5589aeef1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.736+0000 7f6d4ab11740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5589aeef7070 con 0x5589aeef4370 2024-09-10T08:52:01.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.736+0000 7f6d4ab11740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5589aee0a060 con 0x5589aeee95d0 2024-09-10T08:52:01.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.736+0000 7f6d4ab11740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5589aeef71f0 con 0x5589aeeeab40 2024-09-10T08:52:01.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.736+0000 7f6d4ab11740 10 client.0 _ll_register_callbacks cb 0x5589aeb5d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:01.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.736+0000 7f6d45cd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeef4370 0x5589aeef1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.736+0000 7f6d45cd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeef4370 0x5589aeef1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50914/0 (socket says 172.21.15.172:50914) 2024-09-10T08:52:01.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.736+0000 7f6d45cd3640 1 -- 172.21.15.172:0/476516345 learned_addr learned my addr 172.21.15.172:0/476516345 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:01.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.736+0000 7f6d464d4640 1 --2- 172.21.15.172:0/476516345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeeeab40 0x5589aeef1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.736+0000 7f6d45cd3640 1 --2- 172.21.15.172:0/476516345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeee95d0 0x5589aeef3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.737+0000 7f6d464d4640 1 -- 172.21.15.172:0/476516345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeee95d0 msgr2=0x5589aeef3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:01.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.737+0000 7f6d464d4640 1 --2- 172.21.15.172:0/476516345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeee95d0 0x5589aeef3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.737+0000 7f6d464d4640 1 -- 172.21.15.172:0/476516345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeef4370 msgr2=0x5589aeef1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:01.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.737+0000 7f6d464d4640 1 --2- 172.21.15.172:0/476516345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeef4370 0x5589aeef1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.737+0000 7f6d464d4640 1 -- 172.21.15.172:0/476516345 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589aebc03c0 con 0x5589aeeeab40 2024-09-10T08:52:01.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.737+0000 7f6d45cd3640 1 --2- 172.21.15.172:0/476516345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeef4370 0x5589aeef1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:01.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.737+0000 7f6d464d4640 1 --2- 172.21.15.172:0/476516345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeeeab40 0x5589aeef1040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c005fd0 tx=0x7f6d3c00bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fc58d3d266369a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:01.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.737+0000 7f6d29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:52:01.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.737+0000 7f6d29ffb640 1 -- 172.21.15.172:0/476516345 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c01b440 con 0x5589aeeeab40 2024-09-10T08:52:01.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.737+0000 7f6d4ab11740 1 -- 172.21.15.172:0/476516345 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d300041d0 con 0x5589aeeeab40 2024-09-10T08:52:01.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.737+0000 7f6d29ffb640 1 -- 172.21.15.172:0/476516345 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6d3c01b5e0 con 0x5589aeeeab40 2024-09-10T08:52:01.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.738+0000 7f6d29ffb640 1 -- 172.21.15.172:0/476516345 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6d3c024050 con 0x5589aeeeab40 2024-09-10T08:52:01.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.738+0000 7f6d4ab11740 10 client.26245 fetch_fsmap learned FSMap version 36 2024-09-10T08:52:01.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.738+0000 7f6d4ab11740 1 -- 172.21.15.172:0/476516345 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5589aef02bf0 con 0x5589aeeeab40 2024-09-10T08:52:01.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.738+0000 7f6d29ffb640 1 -- 172.21.15.172:0/476516345 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d300041d0 con 0x5589aeeeab40 2024-09-10T08:52:01.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.739+0000 7f6d29ffb640 1 -- 172.21.15.172:0/476516345 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f6d3c019020 con 0x5589aeeeab40 2024-09-10T08:52:01.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.739+0000 7f6d4ab11740 10 client.26245 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T08:52:01.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.739+0000 7f6d4ab11740 10 client.26245 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:52:01.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.739+0000 7f6d4ab11740 20 client.26245 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:01.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.739+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/476516345 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5589aef04050 0x5589aef06470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.739+0000 7f6d4ab11740 4 client.26245 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:01.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.739+0000 7f6d4ab11740 1 -- 172.21.15.172:0/476516345 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589aee5c860 con 0x5589aef04050 2024-09-10T08:52:01.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.739+0000 7f6d45cd3640 1 --2- 172.21.15.172:0/476516345 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5589aef04050 0x5589aef06470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.740+0000 7f6d45cd3640 1 --2- 172.21.15.172:0/476516345 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5589aef04050 0x5589aef06470 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82be979a52adf22b server_cookie=666bc8f743002db3 in_seq=0 out_seq=0 2024-09-10T08:52:01.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.740+0000 7f6d29ffb640 10 client.26245 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:52:01.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.740+0000 7f6d29ffb640 1 -- 172.21.15.172:0/476516345 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589aee5c860 con 0x5589aef04050 2024-09-10T08:52:01.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.740+0000 7f6d29ffb640 10 client.26245 handle_command_reply: tid=0 2024-09-10T08:52:01.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.740+0000 7f6d4ab11740 1 client.26245 shutdown 2024-09-10T08:52:01.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.741+0000 7f6d4ab11740 1 -- 172.21.15.172:0/476516345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeeeab40 msgr2=0x5589aeef1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:01.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.741+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/476516345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeeeab40 0x5589aeef1040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c005fd0 tx=0x7f6d3c00bd80 comp rx=0 tx=0).stop 2024-09-10T08:52:01.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.741+0000 7f6d4ab11740 1 -- 172.21.15.172:0/476516345 shutdown_connections 2024-09-10T08:52:01.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.741+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/476516345 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5589aef04050 0x5589aef06470 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.741+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/476516345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeef4370 0x5589aeef1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.741+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/476516345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeeeab40 0x5589aeef1040 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.741+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/476516345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeee95d0 0x5589aeef3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.741+0000 7f6d4ab11740 1 -- 172.21.15.172:0/476516345 >> 172.21.15.172:0/476516345 conn(0x5589aeeb2310 msgr2=0x5589aeeeb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:01.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.741+0000 7f6d29ffb640 0 client.26245 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:52:01.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.741+0000 7f6d4ab11740 1 -- 172.21.15.172:0/476516345 shutdown_connections 2024-09-10T08:52:01.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.741+0000 7f6d4ab11740 1 -- 172.21.15.172:0/476516345 wait complete. 2024-09-10T08:52:01.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.741+0000 7f6d4ab11740 20 client.26245 trim_cache size 0 max 16384 2024-09-10T08:52:01.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.761+0000 7f6d4ab11740 1 Processor -- start 2024-09-10T08:52:01.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.761+0000 7f6d4ab11740 1 -- start start 2024-09-10T08:52:01.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.761+0000 7f6d4ab11740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeee95d0 0x5589aeef1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.761+0000 7f6d4ab11740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeeeab40 0x5589aef00310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.761+0000 7f6d4ab11740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeef4370 0x5589aef0bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.761+0000 7f6d4ab11740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401b0d30 con 0x5589aeee95d0 2024-09-10T08:52:01.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.761+0000 7f6d4ab11740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401b28a0 con 0x5589aeef4370 2024-09-10T08:52:01.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.761+0000 7f6d4ab11740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401b1400 con 0x5589aeeeab40 2024-09-10T08:52:01.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.761+0000 7f6d45cd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeee95d0 0x5589aeef1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.761+0000 7f6d45cd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeee95d0 0x5589aeef1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50928/0 (socket says 172.21.15.172:50928) 2024-09-10T08:52:01.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.761+0000 7f6d45cd3640 1 -- 172.21.15.172:0/3375963319 learned_addr learned my addr 172.21.15.172:0/3375963319 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:01.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.761+0000 7f6d464d4640 1 --2- 172.21.15.172:0/3375963319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeeeab40 0x5589aef00310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.761+0000 7f6d45cd3640 1 --2- 172.21.15.172:0/3375963319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeef4370 0x5589aef0bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.762+0000 7f6d45cd3640 1 -- 172.21.15.172:0/3375963319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeeeab40 msgr2=0x5589aef00310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:01.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.762+0000 7f6d45cd3640 1 --2- 172.21.15.172:0/3375963319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeeeab40 0x5589aef00310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.762+0000 7f6d45cd3640 1 -- 172.21.15.172:0/3375963319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeef4370 msgr2=0x5589aef0bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:01.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.762+0000 7f6d45cd3640 1 --2- 172.21.15.172:0/3375963319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeef4370 0x5589aef0bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.762+0000 7f6d45cd3640 1 -- 172.21.15.172:0/3375963319 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589aeee5ed0 con 0x5589aeee95d0 2024-09-10T08:52:01.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.762+0000 7f6d45cd3640 1 --2- 172.21.15.172:0/3375963319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeee95d0 0x5589aeef1010 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f6d30001eb0 tx=0x7f6d3000bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62088cae36776119 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:01.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.762+0000 7f6d2bfff640 1 -- 172.21.15.172:0/3375963319 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30004150 con 0x5589aeee95d0 2024-09-10T08:52:01.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.762+0000 7f6d2bfff640 1 -- 172.21.15.172:0/3375963319 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d300054f0 con 0x5589aeee95d0 2024-09-10T08:52:01.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.762+0000 7f6d2bfff640 1 -- 172.21.15.172:0/3375963319 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30030350 con 0x5589aeee95d0 2024-09-10T08:52:01.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.763+0000 7f6d4ab11740 1 -- 172.21.15.172:0/3375963319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeee95d0 msgr2=0x5589aeef1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:01.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.763+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/3375963319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeee95d0 0x5589aeef1010 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f6d30001eb0 tx=0x7f6d3000bf20 comp rx=0 tx=0).stop 2024-09-10T08:52:01.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.763+0000 7f6d4ab11740 1 -- 172.21.15.172:0/3375963319 shutdown_connections 2024-09-10T08:52:01.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.763+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/3375963319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeef4370 0x5589aef0bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.768 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.763+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/3375963319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeeeab40 0x5589aef00310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.763+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/3375963319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeee95d0 0x5589aeef1010 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.763+0000 7f6d4ab11740 1 -- 172.21.15.172:0/3375963319 >> 172.21.15.172:0/3375963319 conn(0x5589aeeb2310 msgr2=0x5589aeef0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:01.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.763+0000 7f6d4ab11740 1 -- 172.21.15.172:0/3375963319 shutdown_connections 2024-09-10T08:52:01.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.763+0000 7f6d4ab11740 1 -- 172.21.15.172:0/3375963319 wait complete. 2024-09-10T08:52:01.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.763+0000 7f6d4ab11740 1 -- start start 2024-09-10T08:52:01.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.763+0000 7f6d4ab11740 1 Processor -- start 2024-09-10T08:52:01.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.764+0000 7f6d4ab11740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeee95d0 0x5589aeefcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.764+0000 7f6d4ab11740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeeeab40 0x5589aeef7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.764+0000 7f6d4ab11740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeef4370 0x5589aeef7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.764+0000 7f6d4ab11740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401b21d0 con 0x5589aeef4370 2024-09-10T08:52:01.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.764+0000 7f6d4ab11740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401b1b00 con 0x5589aeee95d0 2024-09-10T08:52:01.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.764+0000 7f6d45cd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeef4370 0x5589aeef7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.764+0000 7f6d4ab11740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6d401aed10 con 0x5589aeeeab40 2024-09-10T08:52:01.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.764+0000 7f6d45cd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeef4370 0x5589aeef7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50932/0 (socket says 172.21.15.172:50932) 2024-09-10T08:52:01.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.764+0000 7f6d45cd3640 1 -- 172.21.15.172:0/701725001 learned_addr learned my addr 172.21.15.172:0/701725001 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:01.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.764+0000 7f6d4ab11740 10 client.0 _ll_register_callbacks cb 0x7f6d3c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:01.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.764+0000 7f6d45cd3640 1 --2- 172.21.15.172:0/701725001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeee95d0 0x5589aeefcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.764+0000 7f6d464d4640 1 --2- 172.21.15.172:0/701725001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeeeab40 0x5589aeef7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.764+0000 7f6d464d4640 1 -- 172.21.15.172:0/701725001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeee95d0 msgr2=0x5589aeefcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:01.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.764+0000 7f6d464d4640 1 --2- 172.21.15.172:0/701725001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeee95d0 0x5589aeefcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.764+0000 7f6d464d4640 1 -- 172.21.15.172:0/701725001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeef4370 msgr2=0x5589aeef7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:01.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.764+0000 7f6d464d4640 1 --2- 172.21.15.172:0/701725001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeef4370 0x5589aeef7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.764+0000 7f6d464d4640 1 -- 172.21.15.172:0/701725001 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589aeef8230 con 0x5589aeeeab40 2024-09-10T08:52:01.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.765+0000 7f6d464d4640 1 --2- 172.21.15.172:0/701725001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeeeab40 0x5589aeef7440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0106d0 tx=0x7f6d3c010b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c582bb5cfdc29ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:01.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.765+0000 7f6d29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:52:01.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.765+0000 7f6d29ffb640 1 -- 172.21.15.172:0/701725001 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c0039c0 con 0x5589aeeeab40 2024-09-10T08:52:01.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.765+0000 7f6d29ffb640 1 -- 172.21.15.172:0/701725001 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6d3c010de0 con 0x5589aeeeab40 2024-09-10T08:52:01.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.765+0000 7f6d4ab11740 1 -- 172.21.15.172:0/701725001 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d300054f0 con 0x5589aeeeab40 2024-09-10T08:52:01.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.765+0000 7f6d29ffb640 1 -- 172.21.15.172:0/701725001 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6d3c03a050 con 0x5589aeeeab40 2024-09-10T08:52:01.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.765+0000 7f6d4ab11740 10 client.26251 fetch_fsmap learned FSMap version 36 2024-09-10T08:52:01.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.765+0000 7f6d4ab11740 1 -- 172.21.15.172:0/701725001 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5589aef6dd20 con 0x5589aeeeab40 2024-09-10T08:52:01.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.765+0000 7f6d29ffb640 1 -- 172.21.15.172:0/701725001 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d300054f0 con 0x5589aeeeab40 2024-09-10T08:52:01.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.765+0000 7f6d29ffb640 1 -- 172.21.15.172:0/701725001 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f6d3c019040 con 0x5589aeeeab40 2024-09-10T08:52:01.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.766+0000 7f6d4ab11740 10 client.26251 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T08:52:01.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.766+0000 7f6d4ab11740 10 client.26251 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:52:01.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.766+0000 7f6d4ab11740 20 client.26251 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:01.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.766+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/701725001 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5589aef6e520 0x5589aef70940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:01.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.766+0000 7f6d4ab11740 4 client.26251 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:52:01.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.766+0000 7f6d4ab11740 1 -- 172.21.15.172:0/701725001 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589aee56d70 con 0x5589aef6e520 2024-09-10T08:52:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.766+0000 7f6d45cd3640 1 --2- 172.21.15.172:0/701725001 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5589aef6e520 0x5589aef70940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.766+0000 7f6d29ffb640 10 client.26251 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:52:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.766+0000 7f6d45cd3640 1 --2- 172.21.15.172:0/701725001 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5589aef6e520 0x5589aef70940 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cddfe8f816f3a35a server_cookie=c3b8f40ab85b43c9 in_seq=0 out_seq=0 2024-09-10T08:52:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.766+0000 7f6d29ffb640 1 -- 172.21.15.172:0/701725001 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589aee56d70 con 0x5589aef6e520 2024-09-10T08:52:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.766+0000 7f6d29ffb640 10 client.26251 handle_command_reply: tid=0 2024-09-10T08:52:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.766+0000 7f6d4ab11740 1 client.26251 shutdown 2024-09-10T08:52:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.767+0000 7f6d4ab11740 1 -- 172.21.15.172:0/701725001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeeeab40 msgr2=0x5589aeef7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.767+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/701725001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeeeab40 0x5589aeef7440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0106d0 tx=0x7f6d3c010b00 comp rx=0 tx=0).stop 2024-09-10T08:52:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.767+0000 7f6d4ab11740 1 -- 172.21.15.172:0/701725001 shutdown_connections 2024-09-10T08:52:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.767+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/701725001 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5589aef6e520 0x5589aef70940 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.767+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/701725001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589aeef4370 0x5589aeef7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.767+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/701725001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5589aeeeab40 0x5589aeef7440 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.767+0000 7f6d4ab11740 1 --2- 172.21.15.172:0/701725001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589aeee95d0 0x5589aeefcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.767+0000 7f6d4ab11740 1 -- 172.21.15.172:0/701725001 >> 172.21.15.172:0/701725001 conn(0x5589aeeb2310 msgr2=0x5589aeeef910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:01.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.767+0000 7f6d29ffb640 0 client.26251 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:52:01.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.767+0000 7f6d4ab11740 1 -- 172.21.15.172:0/701725001 shutdown_connections 2024-09-10T08:52:01.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.767+0000 7f6d4ab11740 1 -- 172.21.15.172:0/701725001 wait complete. 2024-09-10T08:52:01.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.767+0000 7f6d4ab11740 20 client.26251 trim_cache size 0 max 16384 2024-09-10T08:52:01.773 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:52:01.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.767+0000 7f6d48738640 1 -- 172.21.15.172:0/3429930512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6d2c0a2c00 msgr2=0x7f6d2c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:01.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.768+0000 7f6d48738640 1 --2- 172.21.15.172:0/3429930512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6d2c0a2c00 0x7f6d2c0a50c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6d380c56c0 tx=0x7f6d3802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.768+0000 7f6d48738640 1 -- 172.21.15.172:0/3429930512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4010e4f0 msgr2=0x7f6d4019ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:01.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.768+0000 7f6d48738640 1 --2- 172.21.15.172:0/3429930512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4010e4f0 0x7f6d4019ee20 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f6d380098d0 tx=0x7f6d38032ff0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.768+0000 7f6d48738640 1 -- 172.21.15.172:0/3429930512 shutdown_connections 2024-09-10T08:52:01.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.768+0000 7f6d48738640 1 --2- 172.21.15.172:0/3429930512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6d2c0a2c00 0x7f6d2c0a50c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.768+0000 7f6d48738640 1 --2- 172.21.15.172:0/3429930512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4010e4f0 0x7f6d4019ee20 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.768+0000 7f6d48738640 1 --2- 172.21.15.172:0/3429930512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d401012f0 0x7f6d4019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.768+0000 7f6d48738640 1 --2- 172.21.15.172:0/3429930512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d40100940 0x7f6d4019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:01.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.768+0000 7f6d48738640 1 -- 172.21.15.172:0/3429930512 >> 172.21.15.172:0/3429930512 conn(0x7f6d400fc7d0 msgr2=0x7f6d400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:01.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.768+0000 7f6d48738640 1 -- 172.21.15.172:0/3429930512 shutdown_connections 2024-09-10T08:52:01.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:01.768+0000 7f6d48738640 1 -- 172.21.15.172:0/3429930512 wait complete. 2024-09-10T08:52:02.580 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:02 smithi172 ceph-mon[30854]: pgmap v524: 129 pgs: 129 active+clean; 4.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 24 op/s 2024-09-10T08:52:02.580 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:02 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:52:02.608 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:52:02.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:02 smithi174 ceph-mon[34068]: pgmap v524: 129 pgs: 129 active+clean; 4.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 24 op/s 2024-09-10T08:52:02.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:02 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:52:02.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:02 smithi191 ceph-mon[35314]: pgmap v524: 129 pgs: 129 active+clean; 4.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 24 op/s 2024-09-10T08:52:02.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:02 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:52:02.909 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:52:03.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.405+0000 7f8780c50640 1 -- 172.21.15.172:0/424622568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f877c1036c0 msgr2=0x7f877c10bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:03.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.405+0000 7f8780c50640 1 --2- 172.21.15.172:0/424622568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f877c1036c0 0x7f877c10bbb0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8764009900 tx=0x7f876402a480 comp rx=0 tx=0).stop 2024-09-10T08:52:03.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.405+0000 7f8780c50640 1 -- 172.21.15.172:0/424622568 shutdown_connections 2024-09-10T08:52:03.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.405+0000 7f8780c50640 1 --2- 172.21.15.172:0/424622568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f877c10c180 0x7f877c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.405+0000 7f8780c50640 1 --2- 172.21.15.172:0/424622568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f877c1036c0 0x7f877c10bbb0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.405+0000 7f8780c50640 1 --2- 172.21.15.172:0/424622568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f877c102da0 0x7f877c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.405+0000 7f8780c50640 1 -- 172.21.15.172:0/424622568 >> 172.21.15.172:0/424622568 conn(0x7f877c0fc7a0 msgr2=0x7f877c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:03.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.405+0000 7f8780c50640 1 -- 172.21.15.172:0/424622568 shutdown_connections 2024-09-10T08:52:03.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.405+0000 7f8780c50640 1 -- 172.21.15.172:0/424622568 wait complete. 2024-09-10T08:52:03.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.406+0000 7f8780c50640 1 Processor -- start 2024-09-10T08:52:03.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.406+0000 7f8780c50640 1 -- start start 2024-09-10T08:52:03.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.406+0000 7f8780c50640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f877c102da0 0x7f877c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.406+0000 7f8780c50640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f877c1036c0 0x7f877c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.406+0000 7f8780c50640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f877c10c180 0x7f877c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.406+0000 7f8780c50640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f877c111e10 con 0x7f877c102da0 2024-09-10T08:52:03.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.406+0000 7f8780c50640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f877c111c90 con 0x7f877c1036c0 2024-09-10T08:52:03.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.406+0000 7f8780c50640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f877c111f90 con 0x7f877c10c180 2024-09-10T08:52:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.407+0000 7f877a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f877c102da0 0x7f877c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.407+0000 7f877a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f877c102da0 0x7f877c198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50946/0 (socket says 172.21.15.172:50946) 2024-09-10T08:52:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.407+0000 7f877a59c640 1 -- 172.21.15.172:0/3851966678 learned_addr learned my addr 172.21.15.172:0/3851966678 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.407+0000 7f8773fff640 1 --2- 172.21.15.172:0/3851966678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f877c1036c0 0x7f877c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.407+0000 7f877ad9d640 1 --2- 172.21.15.172:0/3851966678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f877c10c180 0x7f877c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.407+0000 7f877a59c640 1 -- 172.21.15.172:0/3851966678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f877c10c180 msgr2=0x7f877c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.407+0000 7f877a59c640 1 --2- 172.21.15.172:0/3851966678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f877c10c180 0x7f877c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.407+0000 7f877a59c640 1 -- 172.21.15.172:0/3851966678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f877c1036c0 msgr2=0x7f877c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.407+0000 7f877a59c640 1 --2- 172.21.15.172:0/3851966678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f877c1036c0 0x7f877c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.407+0000 7f877a59c640 1 -- 172.21.15.172:0/3851966678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877c19d340 con 0x7f877c102da0 2024-09-10T08:52:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.407+0000 7f877a59c640 1 --2- 172.21.15.172:0/3851966678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f877c102da0 0x7f877c198390 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f876c00e9f0 tx=0x7f876c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.407+0000 7f87737fe640 1 -- 172.21.15.172:0/3851966678 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876c00cd80 con 0x7f877c102da0 2024-09-10T08:52:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.407+0000 7f87737fe640 1 -- 172.21.15.172:0/3851966678 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f876c0108e0 con 0x7f877c102da0 2024-09-10T08:52:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.407+0000 7f8780c50640 1 -- 172.21.15.172:0/3851966678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f877c19d630 con 0x7f877c102da0 2024-09-10T08:52:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.407+0000 7f87737fe640 1 -- 172.21.15.172:0/3851966678 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876c018920 con 0x7f877c102da0 2024-09-10T08:52:03.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.407+0000 7f8780c50640 1 -- 172.21.15.172:0/3851966678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f877c1a4f10 con 0x7f877c102da0 2024-09-10T08:52:03.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.410+0000 7f87737fe640 1 -- 172.21.15.172:0/3851966678 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f876c010a90 con 0x7f877c102da0 2024-09-10T08:52:03.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.411+0000 7f8783029740 1 Processor -- start 2024-09-10T08:52:03.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.411+0000 7f8783029740 1 -- start start 2024-09-10T08:52:03.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.411+0000 7f8783029740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c06b40 0x558ff3c05090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.411+0000 7f8783029740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c055d0 0x558ff3c05a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.411+0000 7f8783029740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c10370 0x558ff3c12760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.411+0000 7f8783029740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558ff3b6b2f0 con 0x558ff3c10370 2024-09-10T08:52:03.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.411+0000 7f8783029740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558ff3b12cf0 con 0x558ff3c055d0 2024-09-10T08:52:03.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.411+0000 7f8783029740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558ff3b672b0 con 0x558ff3c06b40 2024-09-10T08:52:03.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.412+0000 7f87737fe640 1 --2- 172.21.15.172:0/3851966678 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f87680a2c00 0x7f87680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.412+0000 7f8773fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c10370 0x558ff3c12760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.412+0000 7f8773fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c10370 0x558ff3c12760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50956/0 (socket says 172.21.15.172:50956) 2024-09-10T08:52:03.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.412+0000 7f8773fff640 1 -- 172.21.15.172:0/2523223118 learned_addr learned my addr 172.21.15.172:0/2523223118 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:03.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.412+0000 7f877a59c640 1 --2- 172.21.15.172:0/2523223118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c055d0 0x558ff3c05a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.412+0000 7f877ad9d640 1 --2- 172.21.15.172:0/3851966678 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f87680a2c00 0x7f87680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.412+0000 7f8773fff640 1 -- 172.21.15.172:0/2523223118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c06b40 msgr2=0x558ff3c05090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:03.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.412+0000 7f877ad9d640 1 --2- 172.21.15.172:0/2523223118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c06b40 0x558ff3c05090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.412+0000 7f8773fff640 1 --2- 172.21.15.172:0/2523223118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c06b40 0x558ff3c05090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.412+0000 7f8773fff640 1 -- 172.21.15.172:0/2523223118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c055d0 msgr2=0x558ff3c05a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:03.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.412+0000 7f8773fff640 1 --2- 172.21.15.172:0/2523223118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c055d0 0x558ff3c05a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.412+0000 7f8773fff640 1 -- 172.21.15.172:0/2523223118 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff3af73a0 con 0x558ff3c10370 2024-09-10T08:52:03.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.412+0000 7f87737fe640 1 -- 172.21.15.172:0/3851966678 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f876c0cf610 con 0x7f877c102da0 2024-09-10T08:52:03.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.412+0000 7f877ad9d640 1 --2- 172.21.15.172:0/2523223118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c06b40 0x558ff3c05090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:03.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.412+0000 7f877a59c640 1 --2- 172.21.15.172:0/2523223118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c055d0 0x558ff3c05a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:03.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.412+0000 7f8773fff640 1 --2- 172.21.15.172:0/2523223118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c10370 0x558ff3c12760 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8764009900 tx=0x7f8764032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f955e9c90991907 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:03.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.412+0000 7f8770ff9640 1 -- 172.21.15.172:0/2523223118 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8764040070 con 0x558ff3c10370 2024-09-10T08:52:03.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.413+0000 7f877ad9d640 1 --2- 172.21.15.172:0/3851966678 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f87680a2c00 0x7f87680a50c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8774009990 tx=0x7f8774009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:03.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.413+0000 7f8770ff9640 1 -- 172.21.15.172:0/2523223118 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8764004030 con 0x558ff3c10370 2024-09-10T08:52:03.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.413+0000 7f8770ff9640 1 -- 172.21.15.172:0/2523223118 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8764035040 con 0x558ff3c10370 2024-09-10T08:52:03.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.413+0000 7f8783029740 1 -- 172.21.15.172:0/2523223118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c10370 msgr2=0x558ff3c12760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:03.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.413+0000 7f8783029740 1 --2- 172.21.15.172:0/2523223118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c10370 0x558ff3c12760 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8764009900 tx=0x7f8764032bb0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.413+0000 7f8783029740 1 -- 172.21.15.172:0/2523223118 shutdown_connections 2024-09-10T08:52:03.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.413+0000 7f8783029740 1 --2- 172.21.15.172:0/2523223118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c10370 0x558ff3c12760 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.413+0000 7f8783029740 1 --2- 172.21.15.172:0/2523223118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c055d0 0x558ff3c05a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.413+0000 7f8783029740 1 --2- 172.21.15.172:0/2523223118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c06b40 0x558ff3c05090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.413+0000 7f8783029740 1 -- 172.21.15.172:0/2523223118 >> 172.21.15.172:0/2523223118 conn(0x558ff3bce310 msgr2=0x558ff3c01c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:03.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.414+0000 7f8783029740 1 -- 172.21.15.172:0/2523223118 shutdown_connections 2024-09-10T08:52:03.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.414+0000 7f8783029740 1 -- 172.21.15.172:0/2523223118 wait complete. 2024-09-10T08:52:03.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.414+0000 7f8783029740 1 -- start start 2024-09-10T08:52:03.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.414+0000 7f8783029740 1 Processor -- start 2024-09-10T08:52:03.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.414+0000 7f8783029740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c055d0 0x558ff3c0fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.414+0000 7f8783029740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c06b40 0x558ff3c0cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.414+0000 7f8783029740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c10370 0x558ff3c0d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.414+0000 7f8783029740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558ff3c17420 con 0x558ff3c055d0 2024-09-10T08:52:03.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.414+0000 7f8783029740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558ff3b26060 con 0x558ff3c06b40 2024-09-10T08:52:03.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.414+0000 7f8783029740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558ff3c175a0 con 0x558ff3c10370 2024-09-10T08:52:03.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.414+0000 7f8783029740 10 client.0 _ll_register_callbacks cb 0x558ff38798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:03.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.415+0000 7f877a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c055d0 0x558ff3c0fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.415+0000 7f877a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c055d0 0x558ff3c0fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50972/0 (socket says 172.21.15.172:50972) 2024-09-10T08:52:03.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.415+0000 7f877a59c640 1 -- 172.21.15.172:0/3677064335 learned_addr learned my addr 172.21.15.172:0/3677064335 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:03.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.415+0000 7f8773fff640 1 --2- 172.21.15.172:0/3677064335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c06b40 0x558ff3c0cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.415+0000 7f877ad9d640 1 --2- 172.21.15.172:0/3677064335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c10370 0x558ff3c0d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.415+0000 7f877a59c640 1 -- 172.21.15.172:0/3677064335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c10370 msgr2=0x558ff3c0d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:03.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.415+0000 7f877a59c640 1 --2- 172.21.15.172:0/3677064335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c10370 0x558ff3c0d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.415+0000 7f877a59c640 1 -- 172.21.15.172:0/3677064335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c06b40 msgr2=0x558ff3c0cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:03.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.415+0000 7f877a59c640 1 --2- 172.21.15.172:0/3677064335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c06b40 0x558ff3c0cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.415+0000 7f877a59c640 1 -- 172.21.15.172:0/3677064335 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff38dc3c0 con 0x558ff3c055d0 2024-09-10T08:52:03.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.415+0000 7f877ad9d640 1 --2- 172.21.15.172:0/3677064335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c10370 0x558ff3c0d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:03.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.415+0000 7f877a59c640 1 --2- 172.21.15.172:0/3677064335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c055d0 0x558ff3c0fd20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f876c09b8c0 tx=0x7f876c06ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd3664d6c76f3cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:03.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.415+0000 7f8761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:03.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.415+0000 7f8761ffb640 1 -- 172.21.15.172:0/3677064335 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876c09bbc0 con 0x558ff3c055d0 2024-09-10T08:52:03.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.415+0000 7f8761ffb640 1 -- 172.21.15.172:0/3677064335 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f876c0cf370 con 0x558ff3c055d0 2024-09-10T08:52:03.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.415+0000 7f8783029740 1 -- 172.21.15.172:0/3677064335 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8764004030 con 0x558ff3c055d0 2024-09-10T08:52:03.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.416+0000 7f8761ffb640 1 -- 172.21.15.172:0/3677064335 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f876c09fec0 con 0x558ff3c055d0 2024-09-10T08:52:03.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.416+0000 7f8783029740 10 client.17160 fetch_fsmap learned FSMap version 36 2024-09-10T08:52:03.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.416+0000 7f8783029740 1 -- 172.21.15.172:0/3677064335 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558ff3c22de0 con 0x558ff3c055d0 2024-09-10T08:52:03.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.416+0000 7f8761ffb640 1 -- 172.21.15.172:0/3677064335 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8764004030 con 0x558ff3c055d0 2024-09-10T08:52:03.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.416+0000 7f8761ffb640 1 -- 172.21.15.172:0/3677064335 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f876c09b010 con 0x558ff3c055d0 2024-09-10T08:52:03.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.416+0000 7f8783029740 10 client.17160 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T08:52:03.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.416+0000 7f8783029740 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:03.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.416+0000 7f8783029740 20 client.17160 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:03.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.416+0000 7f8783029740 1 --2- 172.21.15.172:0/3677064335 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558ff3c242a0 0x558ff3c26710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.416+0000 7f8783029740 4 client.17160 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:03.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.416+0000 7f8783029740 1 -- 172.21.15.172:0/3677064335 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ff3b78860 con 0x558ff3c242a0 2024-09-10T08:52:03.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.416+0000 7f8773fff640 1 --2- 172.21.15.172:0/3677064335 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558ff3c242a0 0x558ff3c26710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.417+0000 7f8761ffb640 10 client.17160 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:03.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.417+0000 7f8773fff640 1 --2- 172.21.15.172:0/3677064335 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558ff3c242a0 0x558ff3c26710 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fed1d467087e90e server_cookie=d98727878a4f3349 in_seq=0 out_seq=0 2024-09-10T08:52:03.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.418+0000 7f8761ffb640 1 -- 172.21.15.172:0/3677064335 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ff3b78860 con 0x558ff3c242a0 2024-09-10T08:52:03.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.418+0000 7f8761ffb640 10 client.17160 handle_command_reply: tid=0 2024-09-10T08:52:03.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.418+0000 7f8783029740 1 client.17160 shutdown 2024-09-10T08:52:03.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.418+0000 7f8783029740 1 -- 172.21.15.172:0/3677064335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c055d0 msgr2=0x558ff3c0fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:03.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.418+0000 7f8783029740 1 --2- 172.21.15.172:0/3677064335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c055d0 0x558ff3c0fd20 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f876c09b8c0 tx=0x7f876c06ac00 comp rx=0 tx=0).stop 2024-09-10T08:52:03.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.418+0000 7f8783029740 1 -- 172.21.15.172:0/3677064335 shutdown_connections 2024-09-10T08:52:03.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.418+0000 7f8783029740 1 --2- 172.21.15.172:0/3677064335 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558ff3c242a0 0x558ff3c26710 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.418+0000 7f8783029740 1 --2- 172.21.15.172:0/3677064335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c10370 0x558ff3c0d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.418+0000 7f8783029740 1 --2- 172.21.15.172:0/3677064335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c06b40 0x558ff3c0cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.418+0000 7f8783029740 1 --2- 172.21.15.172:0/3677064335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c055d0 0x558ff3c0fd20 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.418+0000 7f8783029740 1 -- 172.21.15.172:0/3677064335 >> 172.21.15.172:0/3677064335 conn(0x558ff3bce310 msgr2=0x558ff3c07960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:03.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.418+0000 7f8761ffb640 0 client.17160 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:03.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.418+0000 7f8783029740 1 -- 172.21.15.172:0/3677064335 shutdown_connections 2024-09-10T08:52:03.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.418+0000 7f8783029740 1 -- 172.21.15.172:0/3677064335 wait complete. 2024-09-10T08:52:03.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.418+0000 7f8783029740 20 client.17160 trim_cache size 0 max 16384 2024-09-10T08:52:03.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.433+0000 7f8783029740 1 Processor -- start 2024-09-10T08:52:03.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.433+0000 7f8783029740 1 -- start start 2024-09-10T08:52:03.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.433+0000 7f8783029740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c055d0 0x558ff3c08120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.433+0000 7f8783029740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c06b40 0x558ff3c08660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.433+0000 7f8783029740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c10370 0x558ff3c2c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.433+0000 7f8783029740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1ac9f0 con 0x558ff3c06b40 2024-09-10T08:52:03.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.433+0000 7f8783029740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1ae560 con 0x558ff3c10370 2024-09-10T08:52:03.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.433+0000 7f8783029740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1ad0c0 con 0x558ff3c055d0 2024-09-10T08:52:03.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.433+0000 7f8773fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c06b40 0x558ff3c08660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.433+0000 7f8773fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c06b40 0x558ff3c08660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50980/0 (socket says 172.21.15.172:50980) 2024-09-10T08:52:03.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.433+0000 7f8773fff640 1 -- 172.21.15.172:0/3343888256 learned_addr learned my addr 172.21.15.172:0/3343888256 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:03.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.433+0000 7f877a59c640 1 --2- 172.21.15.172:0/3343888256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c055d0 0x558ff3c08120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.434+0000 7f877ad9d640 1 --2- 172.21.15.172:0/3343888256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c10370 0x558ff3c2c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.434+0000 7f8773fff640 1 -- 172.21.15.172:0/3343888256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c055d0 msgr2=0x558ff3c08120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:03.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.434+0000 7f8773fff640 1 --2- 172.21.15.172:0/3343888256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c055d0 0x558ff3c08120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.434+0000 7f8773fff640 1 -- 172.21.15.172:0/3343888256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c10370 msgr2=0x558ff3c2c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:03.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.434+0000 7f8773fff640 1 --2- 172.21.15.172:0/3343888256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c10370 0x558ff3c2c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.434+0000 7f8773fff640 1 -- 172.21.15.172:0/3343888256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ff3c06530 con 0x558ff3c06b40 2024-09-10T08:52:03.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.434+0000 7f8773fff640 1 --2- 172.21.15.172:0/3343888256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c06b40 0x558ff3c08660 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8764005230 tx=0x7f876402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836f795209505a11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:03.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.434+0000 7f8763fff640 1 -- 172.21.15.172:0/3343888256 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8764040040 con 0x558ff3c06b40 2024-09-10T08:52:03.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.434+0000 7f8763fff640 1 -- 172.21.15.172:0/3343888256 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8764004260 con 0x558ff3c06b40 2024-09-10T08:52:03.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.434+0000 7f8763fff640 1 -- 172.21.15.172:0/3343888256 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8764035040 con 0x558ff3c06b40 2024-09-10T08:52:03.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.435+0000 7f8783029740 1 -- 172.21.15.172:0/3343888256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c06b40 msgr2=0x558ff3c08660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:03.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.435+0000 7f8783029740 1 --2- 172.21.15.172:0/3343888256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c06b40 0x558ff3c08660 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8764005230 tx=0x7f876402a480 comp rx=0 tx=0).stop 2024-09-10T08:52:03.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.435+0000 7f8783029740 1 -- 172.21.15.172:0/3343888256 shutdown_connections 2024-09-10T08:52:03.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.435+0000 7f8783029740 1 --2- 172.21.15.172:0/3343888256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c10370 0x558ff3c2c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.435+0000 7f8783029740 1 --2- 172.21.15.172:0/3343888256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c06b40 0x558ff3c08660 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.435+0000 7f8783029740 1 --2- 172.21.15.172:0/3343888256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c055d0 0x558ff3c08120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.435+0000 7f8783029740 1 -- 172.21.15.172:0/3343888256 >> 172.21.15.172:0/3343888256 conn(0x558ff3bce310 msgr2=0x558ff3c0f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:03.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.435+0000 7f8783029740 1 -- 172.21.15.172:0/3343888256 shutdown_connections 2024-09-10T08:52:03.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.435+0000 7f8783029740 1 -- 172.21.15.172:0/3343888256 wait complete. 2024-09-10T08:52:03.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.435+0000 7f8783029740 1 -- start start 2024-09-10T08:52:03.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f8783029740 1 Processor -- start 2024-09-10T08:52:03.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f8783029740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c055d0 0x558ff3c18660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f8783029740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c06b40 0x558ff3c18ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f8783029740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c10370 0x558ff3c190e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f8783029740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1ade90 con 0x558ff3c10370 2024-09-10T08:52:03.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f8783029740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1ad7c0 con 0x558ff3c06b40 2024-09-10T08:52:03.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f8783029740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1aa9d0 con 0x558ff3c055d0 2024-09-10T08:52:03.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f8783029740 10 client.0 _ll_register_callbacks cb 0x7f876c0965e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:03.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f877ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c10370 0x558ff3c190e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f877a59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c055d0 0x558ff3c18660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f877ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c10370 0x558ff3c190e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50982/0 (socket says 172.21.15.172:50982) 2024-09-10T08:52:03.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f877ad9d640 1 -- 172.21.15.172:0/2129882627 learned_addr learned my addr 172.21.15.172:0/2129882627 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:03.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f8773fff640 1 --2- 172.21.15.172:0/2129882627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c06b40 0x558ff3c18ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f877ad9d640 1 -- 172.21.15.172:0/2129882627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c055d0 msgr2=0x558ff3c18660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:03.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f877ad9d640 1 --2- 172.21.15.172:0/2129882627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c055d0 0x558ff3c18660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f877ad9d640 1 -- 172.21.15.172:0/2129882627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c06b40 msgr2=0x558ff3c18ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:03.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f877ad9d640 1 --2- 172.21.15.172:0/2129882627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c06b40 0x558ff3c18ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f877ad9d640 1 -- 172.21.15.172:0/2129882627 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ff3c8e0d0 con 0x558ff3c10370 2024-09-10T08:52:03.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.436+0000 7f8773fff640 1 --2- 172.21.15.172:0/2129882627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c06b40 0x558ff3c18ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:03.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.437+0000 7f877ad9d640 1 --2- 172.21.15.172:0/2129882627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c10370 0x558ff3c190e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f8774009fd0 tx=0x7f8774004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c290c86257d2e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:03.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.437+0000 7f8761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:03.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.437+0000 7f8761ffb640 1 -- 172.21.15.172:0/2129882627 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877401d6e0 con 0x558ff3c10370 2024-09-10T08:52:03.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.437+0000 7f8761ffb640 1 -- 172.21.15.172:0/2129882627 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f877401d880 con 0x558ff3c10370 2024-09-10T08:52:03.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.437+0000 7f8783029740 1 -- 172.21.15.172:0/2129882627 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8764004260 con 0x558ff3c10370 2024-09-10T08:52:03.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.437+0000 7f8761ffb640 1 -- 172.21.15.172:0/2129882627 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f877401da80 con 0x558ff3c10370 2024-09-10T08:52:03.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.437+0000 7f8783029740 10 client.17166 fetch_fsmap learned FSMap version 36 2024-09-10T08:52:03.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.437+0000 7f8783029740 1 -- 172.21.15.172:0/2129882627 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558ff3c8f410 con 0x558ff3c10370 2024-09-10T08:52:03.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.437+0000 7f8761ffb640 1 -- 172.21.15.172:0/2129882627 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8764004260 con 0x558ff3c10370 2024-09-10T08:52:03.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.437+0000 7f8761ffb640 1 -- 172.21.15.172:0/2129882627 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f877401b420 con 0x558ff3c10370 2024-09-10T08:52:03.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.437+0000 7f8783029740 10 client.17166 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T08:52:03.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.437+0000 7f8783029740 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:03.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.437+0000 7f8783029740 20 client.17166 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:03.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.438+0000 7f8783029740 1 --2- 172.21.15.172:0/2129882627 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558ff3c8fc10 0x558ff3c92030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:03.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.438+0000 7f8783029740 4 client.17166 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:52:03.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.438+0000 7f8783029740 1 -- 172.21.15.172:0/2129882627 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ff3b72d70 con 0x558ff3c8fc10 2024-09-10T08:52:03.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.438+0000 7f877a59c640 1 --2- 172.21.15.172:0/2129882627 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558ff3c8fc10 0x558ff3c92030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:03.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.438+0000 7f8761ffb640 10 client.17166 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:03.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.438+0000 7f877a59c640 1 --2- 172.21.15.172:0/2129882627 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558ff3c8fc10 0x558ff3c92030 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3207ff6a8e389ce server_cookie=8befeeb7ba8542a0 in_seq=0 out_seq=0 2024-09-10T08:52:03.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.438+0000 7f8761ffb640 1 -- 172.21.15.172:0/2129882627 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ff3b72d70 con 0x558ff3c8fc10 2024-09-10T08:52:03.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.438+0000 7f8761ffb640 10 client.17166 handle_command_reply: tid=0 2024-09-10T08:52:03.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.438+0000 7f8783029740 1 client.17166 shutdown 2024-09-10T08:52:03.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8783029740 1 -- 172.21.15.172:0/2129882627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c10370 msgr2=0x558ff3c190e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:03.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8783029740 1 --2- 172.21.15.172:0/2129882627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c10370 0x558ff3c190e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f8774009fd0 tx=0x7f8774004ab0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8783029740 1 -- 172.21.15.172:0/2129882627 shutdown_connections 2024-09-10T08:52:03.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8783029740 1 --2- 172.21.15.172:0/2129882627 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558ff3c8fc10 0x558ff3c92030 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8783029740 1 --2- 172.21.15.172:0/2129882627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ff3c10370 0x558ff3c190e0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8783029740 1 --2- 172.21.15.172:0/2129882627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ff3c06b40 0x558ff3c18ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8783029740 1 --2- 172.21.15.172:0/2129882627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ff3c055d0 0x558ff3c18660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8783029740 1 -- 172.21.15.172:0/2129882627 >> 172.21.15.172:0/2129882627 conn(0x558ff3bce310 msgr2=0x558ff3c0f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:03.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8761ffb640 0 client.17166 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:03.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8783029740 1 -- 172.21.15.172:0/2129882627 shutdown_connections 2024-09-10T08:52:03.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8783029740 1 -- 172.21.15.172:0/2129882627 wait complete. 2024-09-10T08:52:03.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8783029740 20 client.17166 trim_cache size 0 max 16384 2024-09-10T08:52:03.447 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:52:03.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8780c50640 1 -- 172.21.15.172:0/3851966678 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f87680a2c00 msgr2=0x7f87680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:03.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8780c50640 1 --2- 172.21.15.172:0/3851966678 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f87680a2c00 0x7f87680a50c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8774009990 tx=0x7f8774009920 comp rx=0 tx=0).stop 2024-09-10T08:52:03.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8780c50640 1 -- 172.21.15.172:0/3851966678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f877c102da0 msgr2=0x7f877c198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:03.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8780c50640 1 --2- 172.21.15.172:0/3851966678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f877c102da0 0x7f877c198390 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f876c00e9f0 tx=0x7f876c00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8780c50640 1 -- 172.21.15.172:0/3851966678 shutdown_connections 2024-09-10T08:52:03.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8780c50640 1 --2- 172.21.15.172:0/3851966678 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f87680a2c00 0x7f87680a50c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8780c50640 1 --2- 172.21.15.172:0/3851966678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f877c10c180 0x7f877c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8780c50640 1 --2- 172.21.15.172:0/3851966678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f877c1036c0 0x7f877c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8780c50640 1 --2- 172.21.15.172:0/3851966678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f877c102da0 0x7f877c198390 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:03.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.439+0000 7f8780c50640 1 -- 172.21.15.172:0/3851966678 >> 172.21.15.172:0/3851966678 conn(0x7f877c0fc7a0 msgr2=0x7f877c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:03.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.440+0000 7f8780c50640 1 -- 172.21.15.172:0/3851966678 shutdown_connections 2024-09-10T08:52:03.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:03.440+0000 7f8780c50640 1 -- 172.21.15.172:0/3851966678 wait complete. 2024-09-10T08:52:04.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:04 smithi172 ceph-mon[30854]: pgmap v525: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-09-10T08:52:04.602 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:04 smithi191 ceph-mon[35314]: pgmap v525: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-09-10T08:52:04.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:04 smithi174 ceph-mon[34068]: pgmap v525: 129 pgs: 129 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 24 MiB/s wr, 19 op/s 2024-09-10T08:52:05.361 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:52:05.361 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:52:05.633 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:52:06.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.150+0000 7f59f589b640 1 -- 172.21.15.172:0/2702533929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59f0103740 msgr2=0x7f59f0107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:06.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.150+0000 7f59f589b640 1 --2- 172.21.15.172:0/2702533929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59f0103740 0x7f59f0107b80 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009930 tx=0x7f59e402a420 comp rx=0 tx=0).stop 2024-09-10T08:52:06.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.150+0000 7f59f589b640 1 -- 172.21.15.172:0/2702533929 shutdown_connections 2024-09-10T08:52:06.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.150+0000 7f59f589b640 1 --2- 172.21.15.172:0/2702533929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59f0108290 0x7f59f010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.150+0000 7f59f589b640 1 --2- 172.21.15.172:0/2702533929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59f0103740 0x7f59f0107b80 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.150+0000 7f59f589b640 1 --2- 172.21.15.172:0/2702533929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f0102d90 0x7f59f0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.150+0000 7f59f589b640 1 -- 172.21.15.172:0/2702533929 >> 172.21.15.172:0/2702533929 conn(0x7f59f00fc7d0 msgr2=0x7f59f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:06.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.150+0000 7f59f589b640 1 -- 172.21.15.172:0/2702533929 shutdown_connections 2024-09-10T08:52:06.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.150+0000 7f59f589b640 1 -- 172.21.15.172:0/2702533929 wait complete. 2024-09-10T08:52:06.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.151+0000 7f59f589b640 1 Processor -- start 2024-09-10T08:52:06.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.151+0000 7f59f589b640 1 -- start start 2024-09-10T08:52:06.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.151+0000 7f59f589b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59f0102d90 0x7f59f01009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.151+0000 7f59f589b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f0103740 0x7f59f0100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.151+0000 7f59f589b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59f0108290 0x7f59f0101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.151+0000 7f59f589b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f59f0111f40 con 0x7f59f0108290 2024-09-10T08:52:06.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.151+0000 7f59f589b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59f0111dc0 con 0x7f59f0103740 2024-09-10T08:52:06.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.151+0000 7f59f589b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01120c0 con 0x7f59f0102d90 2024-09-10T08:52:06.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.151+0000 7f59ef7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59f0108290 0x7f59f0101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:06.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.151+0000 7f59ef7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59f0108290 0x7f59f0101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50996/0 (socket says 172.21.15.172:50996) 2024-09-10T08:52:06.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.151+0000 7f59ef7fe640 1 -- 172.21.15.172:0/4028285909 learned_addr learned my addr 172.21.15.172:0/4028285909 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:06.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.151+0000 7f59ee7fc640 1 --2- 172.21.15.172:0/4028285909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f0103740 0x7f59f0100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:06.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.151+0000 7f59eeffd640 1 --2- 172.21.15.172:0/4028285909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59f0102d90 0x7f59f01009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:06.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.152+0000 7f59ef7fe640 1 -- 172.21.15.172:0/4028285909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59f0102d90 msgr2=0x7f59f01009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:06.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.152+0000 7f59ef7fe640 1 --2- 172.21.15.172:0/4028285909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59f0102d90 0x7f59f01009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.152+0000 7f59ef7fe640 1 -- 172.21.15.172:0/4028285909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f0103740 msgr2=0x7f59f0100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:06.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.152+0000 7f59ef7fe640 1 --2- 172.21.15.172:0/4028285909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f0103740 0x7f59f0100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.152+0000 7f59ef7fe640 1 -- 172.21.15.172:0/4028285909 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f01a69a0 con 0x7f59f0108290 2024-09-10T08:52:06.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.152+0000 7f59eeffd640 1 --2- 172.21.15.172:0/4028285909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59f0102d90 0x7f59f01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:06.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.152+0000 7f59ee7fc640 1 --2- 172.21.15.172:0/4028285909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f0103740 0x7f59f0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:06.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.152+0000 7f59ef7fe640 1 --2- 172.21.15.172:0/4028285909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59f0108290 0x7f59f0101470 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f59e800eb10 tx=0x7f59e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:06.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.152+0000 7f59f4899640 1 -- 172.21.15.172:0/4028285909 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e8004b20 con 0x7f59f0108290 2024-09-10T08:52:06.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.152+0000 7f59f4899640 1 -- 172.21.15.172:0/4028285909 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59e8004cc0 con 0x7f59f0108290 2024-09-10T08:52:06.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.152+0000 7f59f589b640 1 -- 172.21.15.172:0/4028285909 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f01a6c90 con 0x7f59f0108290 2024-09-10T08:52:06.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.152+0000 7f59f4899640 1 -- 172.21.15.172:0/4028285909 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e8010690 con 0x7f59f0108290 2024-09-10T08:52:06.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.152+0000 7f59f589b640 1 -- 172.21.15.172:0/4028285909 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f01a71f0 con 0x7f59f0108290 2024-09-10T08:52:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.155+0000 7f59f4899640 1 -- 172.21.15.172:0/4028285909 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f59e8005020 con 0x7f59f0108290 2024-09-10T08:52:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.155+0000 7f59f7c74740 1 Processor -- start 2024-09-10T08:52:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59f7c74740 1 -- start start 2024-09-10T08:52:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59f7c74740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c04799b60 0x564c04799f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59f7c74740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c0479a480 0x564c047a4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59f7c74740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c047a4d40 0x564c047a7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59f7c74740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564c046ff280 con 0x564c04799b60 2024-09-10T08:52:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59f7c74740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564c046a6b10 con 0x564c0479a480 2024-09-10T08:52:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59f7c74740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564c046fb240 con 0x564c047a4d40 2024-09-10T08:52:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59ee7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c04799b60 0x564c04799f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59f4899640 1 --2- 172.21.15.172:0/4028285909 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f59d80a2c00 0x7f59d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59ee7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c04799b60 0x564c04799f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:32968/0 (socket says 172.21.15.172:32968) 2024-09-10T08:52:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59ee7fc640 1 -- 172.21.15.172:0/2770164839 learned_addr learned my addr 172.21.15.172:0/2770164839 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59ef7fe640 1 --2- 172.21.15.172:0/4028285909 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f59d80a2c00 0x7f59d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59ee7fc640 1 --2- 172.21.15.172:0/2770164839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c047a4d40 0x564c047a7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59eeffd640 1 --2- 172.21.15.172:0/2770164839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c0479a480 0x564c047a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59ee7fc640 1 -- 172.21.15.172:0/2770164839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c047a4d40 msgr2=0x564c047a7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59ee7fc640 1 --2- 172.21.15.172:0/2770164839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c047a4d40 0x564c047a7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59f4899640 1 -- 172.21.15.172:0/4028285909 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f59e80cfa20 con 0x7f59f0108290 2024-09-10T08:52:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59ee7fc640 1 -- 172.21.15.172:0/2770164839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c0479a480 msgr2=0x564c047a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59ee7fc640 1 --2- 172.21.15.172:0/2770164839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c0479a480 0x564c047a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59ee7fc640 1 -- 172.21.15.172:0/2770164839 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c046b31f0 con 0x564c04799b60 2024-09-10T08:52:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.156+0000 7f59eeffd640 1 --2- 172.21.15.172:0/2770164839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c0479a480 0x564c047a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.157+0000 7f59ee7fc640 1 --2- 172.21.15.172:0/2770164839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c04799b60 0x564c04799f40 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f59e4005e00 tx=0x7f59e4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1072ac916faa000 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.157+0000 7f59d5ffb640 1 -- 172.21.15.172:0/2770164839 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4044070 con 0x564c04799b60 2024-09-10T08:52:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.157+0000 7f59d5ffb640 1 -- 172.21.15.172:0/2770164839 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59e40054d0 con 0x564c04799b60 2024-09-10T08:52:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.157+0000 7f59d5ffb640 1 -- 172.21.15.172:0/2770164839 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4035040 con 0x564c04799b60 2024-09-10T08:52:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.158+0000 7f59f7c74740 1 -- 172.21.15.172:0/2770164839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c04799b60 msgr2=0x564c04799f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.158+0000 7f59ef7fe640 1 --2- 172.21.15.172:0/4028285909 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f59d80a2c00 0x7f59d80a50c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f59e8021000 tx=0x7f59e800efd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.158+0000 7f59f7c74740 1 --2- 172.21.15.172:0/2770164839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c04799b60 0x564c04799f40 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f59e4005e00 tx=0x7f59e4005d50 comp rx=0 tx=0).stop 2024-09-10T08:52:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.158+0000 7f59f7c74740 1 -- 172.21.15.172:0/2770164839 shutdown_connections 2024-09-10T08:52:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.158+0000 7f59f7c74740 1 --2- 172.21.15.172:0/2770164839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c047a4d40 0x564c047a7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.158+0000 7f59f7c74740 1 --2- 172.21.15.172:0/2770164839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c0479a480 0x564c047a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.158+0000 7f59f7c74740 1 --2- 172.21.15.172:0/2770164839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c04799b60 0x564c04799f40 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.158+0000 7f59f7c74740 1 -- 172.21.15.172:0/2770164839 >> 172.21.15.172:0/2770164839 conn(0x564c047960f0 msgr2=0x564c047964d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.158+0000 7f59f7c74740 1 -- 172.21.15.172:0/2770164839 shutdown_connections 2024-09-10T08:52:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.158+0000 7f59f7c74740 1 -- 172.21.15.172:0/2770164839 wait complete. 2024-09-10T08:52:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.158+0000 7f59f7c74740 1 -- start start 2024-09-10T08:52:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.159+0000 7f59f7c74740 1 Processor -- start 2024-09-10T08:52:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.159+0000 7f59f7c74740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c04799b60 0x564c047a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.159+0000 7f59f7c74740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c0479a480 0x564c047a2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.159+0000 7f59f7c74740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c047a4d40 0x564c047a2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.159+0000 7f59f7c74740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564c047a7910 con 0x564c04799b60 2024-09-10T08:52:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.159+0000 7f59f7c74740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564c046b9f10 con 0x564c047a4d40 2024-09-10T08:52:06.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.159+0000 7f59f7c74740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564c047a7a90 con 0x564c0479a480 2024-09-10T08:52:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.159+0000 7f59f7c74740 10 client.0 _ll_register_callbacks cb 0x564c04682a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.159+0000 7f59ee7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c04799b60 0x564c047a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.159+0000 7f59ee7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c04799b60 0x564c047a1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:32984/0 (socket says 172.21.15.172:32984) 2024-09-10T08:52:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.159+0000 7f59ee7fc640 1 -- 172.21.15.172:0/3567195342 learned_addr learned my addr 172.21.15.172:0/3567195342 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.159+0000 7f59eeffd640 1 --2- 172.21.15.172:0/3567195342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c0479a480 0x564c047a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.159+0000 7f59ee7fc640 1 -- 172.21.15.172:0/3567195342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c0479a480 msgr2=0x564c047a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.159+0000 7f59ee7fc640 1 --2- 172.21.15.172:0/3567195342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c0479a480 0x564c047a2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.159+0000 7f59ee7fc640 1 -- 172.21.15.172:0/3567195342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c047a4d40 msgr2=0x564c047a2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.159+0000 7f59ee7fc640 1 --2- 172.21.15.172:0/3567195342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c047a4d40 0x564c047a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.159+0000 7f59ee7fc640 1 -- 172.21.15.172:0/3567195342 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c044706a0 con 0x564c04799b60 2024-09-10T08:52:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.160+0000 7f59ee7fc640 1 --2- 172.21.15.172:0/3567195342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c04799b60 0x564c047a1be0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f59e402a940 tx=0x7f59e4004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea59fe51cb5171e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.160+0000 7f59ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:06.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.160+0000 7f59ceffd640 1 -- 172.21.15.172:0/3567195342 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e404d060 con 0x564c04799b60 2024-09-10T08:52:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.160+0000 7f59ceffd640 1 -- 172.21.15.172:0/3567195342 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f59e4004970 con 0x564c04799b60 2024-09-10T08:52:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.160+0000 7f59f7c74740 1 -- 172.21.15.172:0/3567195342 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59e40054d0 con 0x564c04799b60 2024-09-10T08:52:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.160+0000 7f59ceffd640 1 -- 172.21.15.172:0/3567195342 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f59e4054050 con 0x564c04799b60 2024-09-10T08:52:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.160+0000 7f59f7c74740 10 client.17178 fetch_fsmap learned FSMap version 36 2024-09-10T08:52:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.160+0000 7f59f7c74740 1 -- 172.21.15.172:0/3567195342 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564c047b3450 con 0x564c04799b60 2024-09-10T08:52:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.160+0000 7f59ceffd640 1 -- 172.21.15.172:0/3567195342 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59e40054d0 con 0x564c04799b60 2024-09-10T08:52:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.160+0000 7f59ceffd640 1 -- 172.21.15.172:0/3567195342 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f59e40039f0 con 0x564c04799b60 2024-09-10T08:52:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.161+0000 7f59f7c74740 10 client.17178 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T08:52:06.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.161+0000 7f59f7c74740 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.161+0000 7f59f7c74740 20 client.17178 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.161+0000 7f59f7c74740 1 --2- 172.21.15.172:0/3567195342 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564c047b47a0 0x564c047b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.161+0000 7f59f7c74740 4 client.17178 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.161+0000 7f59f7c74740 1 -- 172.21.15.172:0/3567195342 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c0470c7f0 con 0x564c047b47a0 2024-09-10T08:52:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.161+0000 7f59eeffd640 1 --2- 172.21.15.172:0/3567195342 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564c047b47a0 0x564c047b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.161+0000 7f59ceffd640 10 client.17178 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.161+0000 7f59eeffd640 1 --2- 172.21.15.172:0/3567195342 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564c047b47a0 0x564c047b6b90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b499798c3d92ce8e server_cookie=c430c188a92424b0 in_seq=0 out_seq=0 2024-09-10T08:52:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.162+0000 7f59ceffd640 1 -- 172.21.15.172:0/3567195342 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c0470c7f0 con 0x564c047b47a0 2024-09-10T08:52:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.162+0000 7f59ceffd640 10 client.17178 handle_command_reply: tid=0 2024-09-10T08:52:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.162+0000 7f59f7c74740 1 client.17178 shutdown 2024-09-10T08:52:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.162+0000 7f59f7c74740 1 -- 172.21.15.172:0/3567195342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c04799b60 msgr2=0x564c047a1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.162+0000 7f59f7c74740 1 --2- 172.21.15.172:0/3567195342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c04799b60 0x564c047a1be0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f59e402a940 tx=0x7f59e4004e60 comp rx=0 tx=0).stop 2024-09-10T08:52:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.162+0000 7f59f7c74740 1 -- 172.21.15.172:0/3567195342 shutdown_connections 2024-09-10T08:52:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.162+0000 7f59f7c74740 1 --2- 172.21.15.172:0/3567195342 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564c047b47a0 0x564c047b6b90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.162+0000 7f59f7c74740 1 --2- 172.21.15.172:0/3567195342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c047a4d40 0x564c047a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.162+0000 7f59f7c74740 1 --2- 172.21.15.172:0/3567195342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c0479a480 0x564c047a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.162+0000 7f59f7c74740 1 --2- 172.21.15.172:0/3567195342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c04799b60 0x564c047a1be0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.162+0000 7f59f7c74740 1 -- 172.21.15.172:0/3567195342 >> 172.21.15.172:0/3567195342 conn(0x564c047960f0 msgr2=0x564c04795270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.162+0000 7f59ceffd640 0 client.17178 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.163+0000 7f59f7c74740 1 -- 172.21.15.172:0/3567195342 shutdown_connections 2024-09-10T08:52:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.163+0000 7f59f7c74740 1 -- 172.21.15.172:0/3567195342 wait complete. 2024-09-10T08:52:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.163+0000 7f59f7c74740 20 client.17178 trim_cache size 0 max 16384 2024-09-10T08:52:06.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.184+0000 7f59f7c74740 1 Processor -- start 2024-09-10T08:52:06.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.184+0000 7f59f7c74740 1 -- start start 2024-09-10T08:52:06.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.185+0000 7f59f7c74740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c04799b60 0x564c047a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.185+0000 7f59f7c74740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c0479a480 0x564c047bc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.185+0000 7f59f7c74740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c047a4d40 0x564c047bc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.185+0000 7f59f7c74740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01b0c50 con 0x564c0479a480 2024-09-10T08:52:06.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.185+0000 7f59f7c74740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01b27c0 con 0x564c04799b60 2024-09-10T08:52:06.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.185+0000 7f59f7c74740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01b1320 con 0x564c047a4d40 2024-09-10T08:52:06.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.185+0000 7f59eeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c0479a480 0x564c047bc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:06.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.185+0000 7f59eeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c0479a480 0x564c047bc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:32988/0 (socket says 172.21.15.172:32988) 2024-09-10T08:52:06.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.185+0000 7f59eeffd640 1 -- 172.21.15.172:0/838489095 learned_addr learned my addr 172.21.15.172:0/838489095 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:06.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.185+0000 7f59ee7fc640 1 --2- 172.21.15.172:0/838489095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c047a4d40 0x564c047bc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:06.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.185+0000 7f59ee7fc640 1 --2- 172.21.15.172:0/838489095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c04799b60 0x564c047a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:06.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.185+0000 7f59eeffd640 1 -- 172.21.15.172:0/838489095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c047a4d40 msgr2=0x564c047bc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:06.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.185+0000 7f59eeffd640 1 --2- 172.21.15.172:0/838489095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c047a4d40 0x564c047bc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.185+0000 7f59eeffd640 1 -- 172.21.15.172:0/838489095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c04799b60 msgr2=0x564c047a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:06.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.185+0000 7f59eeffd640 1 --2- 172.21.15.172:0/838489095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c04799b60 0x564c047a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.185+0000 7f59eeffd640 1 -- 172.21.15.172:0/838489095 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c0479a2b0 con 0x564c0479a480 2024-09-10T08:52:06.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.185+0000 7f59eeffd640 1 --2- 172.21.15.172:0/838489095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c0479a480 0x564c047bc130 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009b10 tx=0x7f59e0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85a44563b1372eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:06.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.186+0000 7f59d57fa640 1 -- 172.21.15.172:0/838489095 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e001c600 con 0x564c0479a480 2024-09-10T08:52:06.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.186+0000 7f59d57fa640 1 -- 172.21.15.172:0/838489095 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59e0021070 con 0x564c0479a480 2024-09-10T08:52:06.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.186+0000 7f59d57fa640 1 -- 172.21.15.172:0/838489095 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e000fc80 con 0x564c0479a480 2024-09-10T08:52:06.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.186+0000 7f59f7c74740 1 -- 172.21.15.172:0/838489095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c0479a480 msgr2=0x564c047bc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:06.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.186+0000 7f59f7c74740 1 --2- 172.21.15.172:0/838489095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c0479a480 0x564c047bc130 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009b10 tx=0x7f59e0011810 comp rx=0 tx=0).stop 2024-09-10T08:52:06.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.187+0000 7f59f7c74740 1 -- 172.21.15.172:0/838489095 shutdown_connections 2024-09-10T08:52:06.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.187+0000 7f59f7c74740 1 --2- 172.21.15.172:0/838489095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c047a4d40 0x564c047bc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.187+0000 7f59f7c74740 1 --2- 172.21.15.172:0/838489095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c0479a480 0x564c047bc130 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.187+0000 7f59f7c74740 1 --2- 172.21.15.172:0/838489095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c04799b60 0x564c047a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.187+0000 7f59f7c74740 1 -- 172.21.15.172:0/838489095 >> 172.21.15.172:0/838489095 conn(0x564c047960f0 msgr2=0x564c047a0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:06.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.187+0000 7f59f7c74740 1 -- 172.21.15.172:0/838489095 shutdown_connections 2024-09-10T08:52:06.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.187+0000 7f59f7c74740 1 -- 172.21.15.172:0/838489095 wait complete. 2024-09-10T08:52:06.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.187+0000 7f59f7c74740 1 -- start start 2024-09-10T08:52:06.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.187+0000 7f59f7c74740 1 Processor -- start 2024-09-10T08:52:06.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.188+0000 7f59f7c74740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c04799b60 0x564c047ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.188+0000 7f59f7c74740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c0479a480 0x564c047a86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.188+0000 7f59f7c74740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c047a4d40 0x564c047a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.188+0000 7f59f7c74740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01b20f0 con 0x564c0479a480 2024-09-10T08:52:06.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.188+0000 7f59f7c74740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01b1a20 con 0x564c04799b60 2024-09-10T08:52:06.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.188+0000 7f59f7c74740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01aec30 con 0x564c047a4d40 2024-09-10T08:52:06.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.188+0000 7f59f7c74740 10 client.0 _ll_register_callbacks cb 0x7f59e4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:06.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.188+0000 7f59eeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c0479a480 0x564c047a86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:06.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.188+0000 7f59eeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c0479a480 0x564c047a86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33004/0 (socket says 172.21.15.172:33004) 2024-09-10T08:52:06.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.188+0000 7f59eeffd640 1 -- 172.21.15.172:0/586877608 learned_addr learned my addr 172.21.15.172:0/586877608 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:06.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.188+0000 7f59ee7fc640 1 --2- 172.21.15.172:0/586877608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c04799b60 0x564c047ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:06.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.188+0000 7f59ee7fc640 1 --2- 172.21.15.172:0/586877608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c047a4d40 0x564c047a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:06.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.188+0000 7f59eeffd640 1 -- 172.21.15.172:0/586877608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c047a4d40 msgr2=0x564c047a8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:06.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.188+0000 7f59eeffd640 1 --2- 172.21.15.172:0/586877608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c047a4d40 0x564c047a8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.188+0000 7f59eeffd640 1 -- 172.21.15.172:0/586877608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c04799b60 msgr2=0x564c047ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:06.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.189+0000 7f59eeffd640 1 --2- 172.21.15.172:0/586877608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c04799b60 0x564c047ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.189+0000 7f59eeffd640 1 -- 172.21.15.172:0/586877608 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c047a9440 con 0x564c0479a480 2024-09-10T08:52:06.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.189+0000 7f59ee7fc640 1 --2- 172.21.15.172:0/586877608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c047a4d40 0x564c047a8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:06.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.189+0000 7f59eeffd640 1 --2- 172.21.15.172:0/586877608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c0479a480 0x564c047a86d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f59e00097d0 tx=0x7f59e0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb052951b3c78e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:06.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.189+0000 7f59ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:06.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.189+0000 7f59ceffd640 1 -- 172.21.15.172:0/586877608 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e002a070 con 0x564c0479a480 2024-09-10T08:52:06.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.189+0000 7f59ceffd640 1 -- 172.21.15.172:0/586877608 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f59e0004250 con 0x564c0479a480 2024-09-10T08:52:06.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.189+0000 7f59f7c74740 1 -- 172.21.15.172:0/586877608 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59e0021070 con 0x564c0479a480 2024-09-10T08:52:06.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.190+0000 7f59ceffd640 1 -- 172.21.15.172:0/586877608 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f59e0036050 con 0x564c0479a480 2024-09-10T08:52:06.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.190+0000 7f59f7c74740 10 client.17184 fetch_fsmap learned FSMap version 36 2024-09-10T08:52:06.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.190+0000 7f59f7c74740 1 -- 172.21.15.172:0/586877608 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564c0481f070 con 0x564c0479a480 2024-09-10T08:52:06.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.190+0000 7f59ceffd640 1 -- 172.21.15.172:0/586877608 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59e0021070 con 0x564c0479a480 2024-09-10T08:52:06.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.190+0000 7f59ceffd640 1 -- 172.21.15.172:0/586877608 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f59e00053a0 con 0x564c0479a480 2024-09-10T08:52:06.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.191+0000 7f59f7c74740 10 client.17184 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T08:52:06.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.191+0000 7f59f7c74740 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:06.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.191+0000 7f59f7c74740 20 client.17184 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:06.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.191+0000 7f59f7c74740 1 --2- 172.21.15.172:0/586877608 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564c0481f750 0x564c04821b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:06.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.191+0000 7f59f7c74740 4 client.17184 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:52:06.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.191+0000 7f59f7c74740 1 -- 172.21.15.172:0/586877608 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564c04706d00 con 0x564c0481f750 2024-09-10T08:52:06.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.191+0000 7f59ee7fc640 1 --2- 172.21.15.172:0/586877608 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564c0481f750 0x564c04821b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:06.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.192+0000 7f59ee7fc640 1 --2- 172.21.15.172:0/586877608 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564c0481f750 0x564c04821b70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4453916083eddf server_cookie=53e1a191b4e7c6d8 in_seq=0 out_seq=0 2024-09-10T08:52:06.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.192+0000 7f59ceffd640 10 client.17184 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:06.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.193+0000 7f59ceffd640 1 -- 172.21.15.172:0/586877608 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564c04706d00 con 0x564c0481f750 2024-09-10T08:52:06.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.193+0000 7f59ceffd640 10 client.17184 handle_command_reply: tid=0 2024-09-10T08:52:06.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.193+0000 7f59f7c74740 1 client.17184 shutdown 2024-09-10T08:52:06.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.194+0000 7f59f7c74740 1 -- 172.21.15.172:0/586877608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c0479a480 msgr2=0x564c047a86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:06.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.194+0000 7f59f7c74740 1 --2- 172.21.15.172:0/586877608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c0479a480 0x564c047a86d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f59e00097d0 tx=0x7f59e0009800 comp rx=0 tx=0).stop 2024-09-10T08:52:06.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.194+0000 7f59f7c74740 1 -- 172.21.15.172:0/586877608 shutdown_connections 2024-09-10T08:52:06.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.194+0000 7f59f7c74740 1 --2- 172.21.15.172:0/586877608 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564c0481f750 0x564c04821b70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.194+0000 7f59f7c74740 1 --2- 172.21.15.172:0/586877608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c047a4d40 0x564c047a8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.194+0000 7f59f7c74740 1 --2- 172.21.15.172:0/586877608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564c0479a480 0x564c047a86d0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.194+0000 7f59f7c74740 1 --2- 172.21.15.172:0/586877608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c04799b60 0x564c047ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.194+0000 7f59f7c74740 1 -- 172.21.15.172:0/586877608 >> 172.21.15.172:0/586877608 conn(0x564c047960f0 msgr2=0x564c047a0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:06.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.194+0000 7f59ceffd640 0 client.17184 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:06.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.194+0000 7f59f7c74740 1 -- 172.21.15.172:0/586877608 shutdown_connections 2024-09-10T08:52:06.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.194+0000 7f59f7c74740 1 -- 172.21.15.172:0/586877608 wait complete. 2024-09-10T08:52:06.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.194+0000 7f59f7c74740 20 client.17184 trim_cache size 0 max 16384 2024-09-10T08:52:06.203 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:52:06.203 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T08:52:06.203 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "10646d78-1fa3-44e4-9a01-a8ee703f14c1", 2024-09-10T08:52:06.204 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T08:52:06.204 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:52:06.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.195+0000 7f59f589b640 1 -- 172.21.15.172:0/4028285909 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f59d80a2c00 msgr2=0x7f59d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:06.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.195+0000 7f59f589b640 1 --2- 172.21.15.172:0/4028285909 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f59d80a2c00 0x7f59d80a50c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f59e8021000 tx=0x7f59e800efd0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.195+0000 7f59f589b640 1 -- 172.21.15.172:0/4028285909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59f0108290 msgr2=0x7f59f0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:06.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.195+0000 7f59f589b640 1 --2- 172.21.15.172:0/4028285909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59f0108290 0x7f59f0101470 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f59e800eb10 tx=0x7f59e800c2d0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.195+0000 7f59f589b640 1 -- 172.21.15.172:0/4028285909 shutdown_connections 2024-09-10T08:52:06.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.195+0000 7f59f589b640 1 --2- 172.21.15.172:0/4028285909 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f59d80a2c00 0x7f59d80a50c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.195+0000 7f59f589b640 1 --2- 172.21.15.172:0/4028285909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59f0108290 0x7f59f0101470 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.195+0000 7f59f589b640 1 --2- 172.21.15.172:0/4028285909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59f0103740 0x7f59f0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.195+0000 7f59f589b640 1 --2- 172.21.15.172:0/4028285909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59f0102d90 0x7f59f01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:06.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.195+0000 7f59f589b640 1 -- 172.21.15.172:0/4028285909 >> 172.21.15.172:0/4028285909 conn(0x7f59f00fc7d0 msgr2=0x7f59f0105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:06.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.195+0000 7f59f589b640 1 -- 172.21.15.172:0/4028285909 shutdown_connections 2024-09-10T08:52:06.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:06.195+0000 7f59f589b640 1 -- 172.21.15.172:0/4028285909 wait complete. 2024-09-10T08:52:06.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:06 smithi172 ceph-mon[30854]: pgmap v526: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-09-10T08:52:06.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:06 smithi174 ceph-mon[34068]: pgmap v526: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-09-10T08:52:06.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:06 smithi191 ceph-mon[35314]: pgmap v526: 129 pgs: 129 active+clean; 4.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-09-10T08:52:07.099 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:52:07.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:07 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T08:52:07.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:07 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:07 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:52:07.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:07 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:52:07.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:07 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:52:07.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:07 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:52:07.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:07 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:07 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:07 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:07 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:52:07.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:07 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:07 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:52:07.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:07 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:07 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.371 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:52:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:07 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T08:52:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:07 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:07 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:52:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:07 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:52:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:07 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:52:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:07 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:52:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:07 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:07 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:07 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:07 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:52:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:07 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:07 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:52:07.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:07 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:07 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:07 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T08:52:07.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:07 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:07 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:52:07.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:07 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:52:07.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:07 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:52:07.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:07 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:52:07.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:07 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:07 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:07 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:07 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:52:07.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:07 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:07 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:52:07.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:07 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:07 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:07.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.873+0000 7fe01557b640 1 -- 172.21.15.172:0/990305672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe010106a90 msgr2=0x7fe010106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:07.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.873+0000 7fe01557b640 1 --2- 172.21.15.172:0/990305672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe010106a90 0x7fe010106e70 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe000009930 tx=0x7fe00002a420 comp rx=0 tx=0).stop 2024-09-10T08:52:07.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.873+0000 7fe01557b640 1 -- 172.21.15.172:0/990305672 shutdown_connections 2024-09-10T08:52:07.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.873+0000 7fe01557b640 1 --2- 172.21.15.172:0/990305672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe010101430 0x7fe01010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.873+0000 7fe01557b640 1 --2- 172.21.15.172:0/990305672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe010100a90 0x7fe010100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.873+0000 7fe01557b640 1 --2- 172.21.15.172:0/990305672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe010106a90 0x7fe010106e70 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.873+0000 7fe01557b640 1 -- 172.21.15.172:0/990305672 >> 172.21.15.172:0/990305672 conn(0x7fe0100fc7d0 msgr2=0x7fe0100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:07.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.873+0000 7fe01557b640 1 -- 172.21.15.172:0/990305672 shutdown_connections 2024-09-10T08:52:07.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.873+0000 7fe01557b640 1 -- 172.21.15.172:0/990305672 wait complete. 2024-09-10T08:52:07.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.874+0000 7fe01557b640 1 Processor -- start 2024-09-10T08:52:07.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.874+0000 7fe01557b640 1 -- start start 2024-09-10T08:52:07.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.874+0000 7fe01557b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe010100a90 0x7fe0101a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.874+0000 7fe01557b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe010101430 0x7fe0101a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.875+0000 7fe01557b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe010106a90 0x7fe01019e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.875+0000 7fe01557b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe010110d40 con 0x7fe010101430 2024-09-10T08:52:07.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.875+0000 7fe01557b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe010110bc0 con 0x7fe010100a90 2024-09-10T08:52:07.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.875+0000 7fe01557b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe010110ec0 con 0x7fe010106a90 2024-09-10T08:52:07.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.875+0000 7fe007fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe010101430 0x7fe0101a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:07.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.875+0000 7fe007fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe010101430 0x7fe0101a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33028/0 (socket says 172.21.15.172:33028) 2024-09-10T08:52:07.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.875+0000 7fe007fff640 1 -- 172.21.15.172:0/3452130596 learned_addr learned my addr 172.21.15.172:0/3452130596 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:07.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.875+0000 7fe00f7fe640 1 --2- 172.21.15.172:0/3452130596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe010106a90 0x7fe01019e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:07.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.875+0000 7fe00effd640 1 --2- 172.21.15.172:0/3452130596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe010100a90 0x7fe0101a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:07.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.875+0000 7fe00f7fe640 1 -- 172.21.15.172:0/3452130596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe010100a90 msgr2=0x7fe0101a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:07.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.875+0000 7fe00f7fe640 1 --2- 172.21.15.172:0/3452130596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe010100a90 0x7fe0101a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.875+0000 7fe00f7fe640 1 -- 172.21.15.172:0/3452130596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe010101430 msgr2=0x7fe0101a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:07.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.875+0000 7fe00f7fe640 1 --2- 172.21.15.172:0/3452130596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe010101430 0x7fe0101a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.875+0000 7fe00f7fe640 1 -- 172.21.15.172:0/3452130596 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01019f230 con 0x7fe010106a90 2024-09-10T08:52:07.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.876+0000 7fe00f7fe640 1 --2- 172.21.15.172:0/3452130596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe010106a90 0x7fe01019e9c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe00800e940 tx=0x7fe00800ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:07.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.876+0000 7fe00cff9640 1 -- 172.21.15.172:0/3452130596 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008004d20 con 0x7fe010106a90 2024-09-10T08:52:07.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.876+0000 7fe01557b640 1 -- 172.21.15.172:0/3452130596 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01019f520 con 0x7fe010106a90 2024-09-10T08:52:07.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.876+0000 7fe00cff9640 1 -- 172.21.15.172:0/3452130596 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe008005020 con 0x7fe010106a90 2024-09-10T08:52:07.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.876+0000 7fe00cff9640 1 -- 172.21.15.172:0/3452130596 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0080116b0 con 0x7fe010106a90 2024-09-10T08:52:07.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.876+0000 7fe01557b640 1 -- 172.21.15.172:0/3452130596 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0101ab620 con 0x7fe010106a90 2024-09-10T08:52:07.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.878+0000 7fe00cff9640 1 -- 172.21.15.172:0/3452130596 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe00800cb20 con 0x7fe010106a90 2024-09-10T08:52:07.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.878+0000 7fe017954740 1 Processor -- start 2024-09-10T08:52:07.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.878+0000 7fe017954740 1 -- start start 2024-09-10T08:52:07.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.879+0000 7fe017954740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bf4a0 0x55bb208bf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.879+0000 7fe017954740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bfdc0 0x55bb208ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.879+0000 7fe017954740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208ca680 0x55bb208cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.879+0000 7fe017954740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bb208252e0 con 0x55bb208bfdc0 2024-09-10T08:52:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.879+0000 7fe017954740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bb207ccce0 con 0x55bb208bf4a0 2024-09-10T08:52:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.879+0000 7fe017954740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bb208212a0 con 0x55bb208ca680 2024-09-10T08:52:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.879+0000 7fe00effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bfdc0 0x55bb208ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.879+0000 7fe00effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bfdc0 0x55bb208ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33044/0 (socket says 172.21.15.172:33044) 2024-09-10T08:52:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.879+0000 7fe00effd640 1 -- 172.21.15.172:0/3050821919 learned_addr learned my addr 172.21.15.172:0/3050821919 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.879+0000 7fe007fff640 1 --2- 172.21.15.172:0/3050821919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bf4a0 0x55bb208bf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.879+0000 7fe00effd640 1 -- 172.21.15.172:0/3050821919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208ca680 msgr2=0x55bb208cca70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:52:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.879+0000 7fe00effd640 1 --2- 172.21.15.172:0/3050821919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208ca680 0x55bb208cca70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.879+0000 7fe00effd640 1 -- 172.21.15.172:0/3050821919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bf4a0 msgr2=0x55bb208bf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:07.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.880+0000 7fe00effd640 1 --2- 172.21.15.172:0/3050821919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bf4a0 0x55bb208bf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.880+0000 7fe00effd640 1 -- 172.21.15.172:0/3050821919 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb207b1440 con 0x55bb208bfdc0 2024-09-10T08:52:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.880+0000 7fe00cff9640 1 --2- 172.21.15.172:0/3452130596 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdffc0a2b80 0x7fdffc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.880+0000 7fe00effd640 1 --2- 172.21.15.172:0/3050821919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bfdc0 0x55bb208ca140 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe00002a940 tx=0x7fe000005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdb4ed6e6bd7f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.880+0000 7fe007fff640 1 --2- 172.21.15.172:0/3452130596 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdffc0a2b80 0x7fdffc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.880+0000 7fe00cff9640 1 -- 172.21.15.172:0/3452130596 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe0080cf400 con 0x7fe010106a90 2024-09-10T08:52:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.880+0000 7fe0057fa640 1 -- 172.21.15.172:0/3050821919 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000040070 con 0x55bb208bfdc0 2024-09-10T08:52:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.880+0000 7fe0057fa640 1 -- 172.21.15.172:0/3050821919 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0000316a0 con 0x55bb208bfdc0 2024-09-10T08:52:07.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.880+0000 7fe0057fa640 1 -- 172.21.15.172:0/3050821919 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000035040 con 0x55bb208bfdc0 2024-09-10T08:52:07.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.881+0000 7fe017954740 1 -- 172.21.15.172:0/3050821919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bfdc0 msgr2=0x55bb208ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:07.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.881+0000 7fe007fff640 1 --2- 172.21.15.172:0/3452130596 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdffc0a2b80 0x7fdffc0a5040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55bb208bcdb0 tx=0x7fdff800b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:07.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.881+0000 7fe017954740 1 --2- 172.21.15.172:0/3050821919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bfdc0 0x55bb208ca140 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe00002a940 tx=0x7fe000005c00 comp rx=0 tx=0).stop 2024-09-10T08:52:07.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.881+0000 7fe017954740 1 -- 172.21.15.172:0/3050821919 shutdown_connections 2024-09-10T08:52:07.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.881+0000 7fe017954740 1 --2- 172.21.15.172:0/3050821919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208ca680 0x55bb208cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.881+0000 7fe017954740 1 --2- 172.21.15.172:0/3050821919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bfdc0 0x55bb208ca140 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.881+0000 7fe017954740 1 --2- 172.21.15.172:0/3050821919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bf4a0 0x55bb208bf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.881+0000 7fe017954740 1 -- 172.21.15.172:0/3050821919 >> 172.21.15.172:0/3050821919 conn(0x55bb208bba30 msgr2=0x55bb208bbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:07.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.881+0000 7fe017954740 1 -- 172.21.15.172:0/3050821919 shutdown_connections 2024-09-10T08:52:07.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.881+0000 7fe017954740 1 -- 172.21.15.172:0/3050821919 wait complete. 2024-09-10T08:52:07.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.882+0000 7fe017954740 1 -- start start 2024-09-10T08:52:07.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.882+0000 7fe017954740 1 Processor -- start 2024-09-10T08:52:07.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.882+0000 7fe017954740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bf4a0 0x55bb208c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.882+0000 7fe017954740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bfdc0 0x55bb208c7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.882+0000 7fe017954740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208ca680 0x55bb208c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.882+0000 7fe017954740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bb208cd2f0 con 0x55bb208bfdc0 2024-09-10T08:52:07.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.882+0000 7fe017954740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bb207e0050 con 0x55bb208bf4a0 2024-09-10T08:52:07.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.882+0000 7fe017954740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bb208cd470 con 0x55bb208ca680 2024-09-10T08:52:07.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.882+0000 7fe017954740 10 client.0 _ll_register_callbacks cb 0x55bb20820160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:07.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.882+0000 7fe007fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bfdc0 0x55bb208c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:07.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.882+0000 7fe007fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bfdc0 0x55bb208c7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33054/0 (socket says 172.21.15.172:33054) 2024-09-10T08:52:07.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.882+0000 7fe007fff640 1 -- 172.21.15.172:0/44714873 learned_addr learned my addr 172.21.15.172:0/44714873 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:07.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.882+0000 7fe00f7fe640 1 --2- 172.21.15.172:0/44714873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208ca680 0x55bb208c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:07.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.883+0000 7fe007fff640 1 -- 172.21.15.172:0/44714873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208ca680 msgr2=0x55bb208c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:07.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.883+0000 7fe007fff640 1 --2- 172.21.15.172:0/44714873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208ca680 0x55bb208c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.883+0000 7fe007fff640 1 -- 172.21.15.172:0/44714873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bf4a0 msgr2=0x55bb208c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:07.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.883+0000 7fe007fff640 1 --2- 172.21.15.172:0/44714873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bf4a0 0x55bb208c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.883+0000 7fe007fff640 1 -- 172.21.15.172:0/44714873 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb205963c0 con 0x55bb208bfdc0 2024-09-10T08:52:07.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.883+0000 7fe007fff640 1 --2- 172.21.15.172:0/44714873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bfdc0 0x55bb208c7220 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fdff8003f40 tx=0x7fdff80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af57a68a099182f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:07.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.883+0000 7fdff67fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:07.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.883+0000 7fdff67fc640 1 -- 172.21.15.172:0/44714873 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff801b870 con 0x55bb208bfdc0 2024-09-10T08:52:07.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.883+0000 7fe017954740 1 -- 172.21.15.172:0/44714873 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0000316a0 con 0x55bb208bfdc0 2024-09-10T08:52:07.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.883+0000 7fdff67fc640 1 -- 172.21.15.172:0/44714873 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fdff801ba10 con 0x55bb208bfdc0 2024-09-10T08:52:07.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.883+0000 7fdff67fc640 1 -- 172.21.15.172:0/44714873 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdff801bc10 con 0x55bb208bfdc0 2024-09-10T08:52:07.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.883+0000 7fe017954740 10 client.17196 fetch_fsmap learned FSMap version 36 2024-09-10T08:52:07.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.883+0000 7fe017954740 1 -- 172.21.15.172:0/44714873 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55bb208d8d10 con 0x55bb208bfdc0 2024-09-10T08:52:07.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.883+0000 7fdff67fc640 1 -- 172.21.15.172:0/44714873 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0000316a0 con 0x55bb208bfdc0 2024-09-10T08:52:07.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.884+0000 7fdff67fc640 1 -- 172.21.15.172:0/44714873 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fdff8024070 con 0x55bb208bfdc0 2024-09-10T08:52:07.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.884+0000 7fe017954740 10 client.17196 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T08:52:07.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.884+0000 7fe017954740 10 client.17196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:07.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.884+0000 7fe017954740 20 client.17196 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:07.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.884+0000 7fe017954740 1 --2- 172.21.15.172:0/44714873 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bb208da5b0 0x55bb208dca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.884+0000 7fe017954740 4 client.17196 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:07.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.884+0000 7fe017954740 1 -- 172.21.15.172:0/44714873 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb20832850 con 0x55bb208da5b0 2024-09-10T08:52:07.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.884+0000 7fe00effd640 1 --2- 172.21.15.172:0/44714873 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bb208da5b0 0x55bb208dca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:07.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.884+0000 7fdff67fc640 10 client.17196 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:07.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.884+0000 7fe00effd640 1 --2- 172.21.15.172:0/44714873 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bb208da5b0 0x55bb208dca70 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c36a5ef7eb5b72fa server_cookie=f2fefd76396cb325 in_seq=0 out_seq=0 2024-09-10T08:52:07.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.885+0000 7fdff67fc640 1 -- 172.21.15.172:0/44714873 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb20832850 con 0x55bb208da5b0 2024-09-10T08:52:07.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.885+0000 7fdff67fc640 10 client.17196 handle_command_reply: tid=0 2024-09-10T08:52:07.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.885+0000 7fe017954740 1 client.17196 shutdown 2024-09-10T08:52:07.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.885+0000 7fe017954740 1 -- 172.21.15.172:0/44714873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bfdc0 msgr2=0x55bb208c7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:07.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.885+0000 7fe017954740 1 --2- 172.21.15.172:0/44714873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bfdc0 0x55bb208c7220 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fdff8003f40 tx=0x7fdff80046c0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.885+0000 7fe017954740 1 -- 172.21.15.172:0/44714873 shutdown_connections 2024-09-10T08:52:07.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.885+0000 7fe017954740 1 --2- 172.21.15.172:0/44714873 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bb208da5b0 0x55bb208dca70 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.885+0000 7fdff67fc640 0 client.17196 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:07.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.885+0000 7fe017954740 1 --2- 172.21.15.172:0/44714873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208ca680 0x55bb208c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.885+0000 7fe017954740 1 --2- 172.21.15.172:0/44714873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bfdc0 0x55bb208c7220 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.885+0000 7fe017954740 1 --2- 172.21.15.172:0/44714873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bf4a0 0x55bb208c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.885+0000 7fe017954740 1 -- 172.21.15.172:0/44714873 >> 172.21.15.172:0/44714873 conn(0x55bb208bba30 msgr2=0x55bb208c1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:07.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.885+0000 7fe017954740 1 -- 172.21.15.172:0/44714873 shutdown_connections 2024-09-10T08:52:07.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.885+0000 7fe017954740 1 -- 172.21.15.172:0/44714873 wait complete. 2024-09-10T08:52:07.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.885+0000 7fe017954740 20 client.17196 trim_cache size 0 max 16384 2024-09-10T08:52:07.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.907+0000 7fe017954740 1 Processor -- start 2024-09-10T08:52:07.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.907+0000 7fe017954740 1 -- start start 2024-09-10T08:52:07.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe017954740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bba30 0x55bb208baa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe017954740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208bf4a0 0x55bb208e2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe017954740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bfdc0 0x55bb208e28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe017954740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe0101b5080 con 0x55bb208bba30 2024-09-10T08:52:07.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe017954740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe0101b6bf0 con 0x55bb208bfdc0 2024-09-10T08:52:07.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe017954740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe0101b5750 con 0x55bb208bf4a0 2024-09-10T08:52:07.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe00effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bba30 0x55bb208baa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:07.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe00effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bba30 0x55bb208baa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33070/0 (socket says 172.21.15.172:33070) 2024-09-10T08:52:07.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe00effd640 1 -- 172.21.15.172:0/567944305 learned_addr learned my addr 172.21.15.172:0/567944305 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:07.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe00f7fe640 1 --2- 172.21.15.172:0/567944305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bfdc0 0x55bb208e28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:07.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe007fff640 1 --2- 172.21.15.172:0/567944305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208bf4a0 0x55bb208e2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:07.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe00effd640 1 -- 172.21.15.172:0/567944305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208bf4a0 msgr2=0x55bb208e2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:07.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe00effd640 1 --2- 172.21.15.172:0/567944305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208bf4a0 0x55bb208e2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe00effd640 1 -- 172.21.15.172:0/567944305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bfdc0 msgr2=0x55bb208e28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:07.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe00effd640 1 --2- 172.21.15.172:0/567944305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bfdc0 0x55bb208e28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe00effd640 1 -- 172.21.15.172:0/567944305 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb208bb860 con 0x55bb208bba30 2024-09-10T08:52:07.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe00f7fe640 1 --2- 172.21.15.172:0/567944305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bfdc0 0x55bb208e28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:07.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.908+0000 7fe007fff640 1 --2- 172.21.15.172:0/567944305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208bf4a0 0x55bb208e2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:07.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.909+0000 7fe00effd640 1 --2- 172.21.15.172:0/567944305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bba30 0x55bb208baa60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x55bb208bc890 tx=0x7fe000005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b1f6045858815a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:07.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.909+0000 7fe004ff9640 1 -- 172.21.15.172:0/567944305 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000051070 con 0x55bb208bba30 2024-09-10T08:52:07.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.909+0000 7fe004ff9640 1 -- 172.21.15.172:0/567944305 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0000054d0 con 0x55bb208bba30 2024-09-10T08:52:07.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.909+0000 7fe004ff9640 1 -- 172.21.15.172:0/567944305 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000035070 con 0x55bb208bba30 2024-09-10T08:52:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.910+0000 7fe017954740 1 -- 172.21.15.172:0/567944305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bba30 msgr2=0x55bb208baa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.910+0000 7fe017954740 1 --2- 172.21.15.172:0/567944305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bba30 0x55bb208baa60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x55bb208bc890 tx=0x7fe000005fd0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.910+0000 7fe017954740 1 -- 172.21.15.172:0/567944305 shutdown_connections 2024-09-10T08:52:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.910+0000 7fe017954740 1 --2- 172.21.15.172:0/567944305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bfdc0 0x55bb208e28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.910+0000 7fe017954740 1 --2- 172.21.15.172:0/567944305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208bf4a0 0x55bb208e2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.910+0000 7fe017954740 1 --2- 172.21.15.172:0/567944305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bba30 0x55bb208baa60 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.910+0000 7fe017954740 1 -- 172.21.15.172:0/567944305 >> 172.21.15.172:0/567944305 conn(0x55bb208c78c0 msgr2=0x55bb208ba350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.910+0000 7fe017954740 1 -- 172.21.15.172:0/567944305 shutdown_connections 2024-09-10T08:52:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.910+0000 7fe017954740 1 -- 172.21.15.172:0/567944305 wait complete. 2024-09-10T08:52:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.910+0000 7fe017954740 1 -- start start 2024-09-10T08:52:07.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.910+0000 7fe017954740 1 Processor -- start 2024-09-10T08:52:07.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe017954740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208bba30 0x55bb208cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe017954740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bf4a0 0x55bb208d04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe017954740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bfdc0 0x55bb2093c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe017954740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe0101b6520 con 0x55bb208bf4a0 2024-09-10T08:52:07.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe017954740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe0101b5e50 con 0x55bb208bfdc0 2024-09-10T08:52:07.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe017954740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe0101b3060 con 0x55bb208bba30 2024-09-10T08:52:07.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe017954740 10 client.0 _ll_register_callbacks cb 0x7fdff8004280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:07.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe007fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bf4a0 0x55bb208d04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:07.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe007fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bf4a0 0x55bb208d04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33080/0 (socket says 172.21.15.172:33080) 2024-09-10T08:52:07.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe007fff640 1 -- 172.21.15.172:0/2125761241 learned_addr learned my addr 172.21.15.172:0/2125761241 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:07.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe00f7fe640 1 --2- 172.21.15.172:0/2125761241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bfdc0 0x55bb2093c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:07.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe00effd640 1 --2- 172.21.15.172:0/2125761241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208bba30 0x55bb208cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:07.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe007fff640 1 -- 172.21.15.172:0/2125761241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208bba30 msgr2=0x55bb208cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:07.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe007fff640 1 --2- 172.21.15.172:0/2125761241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208bba30 0x55bb208cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe007fff640 1 -- 172.21.15.172:0/2125761241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bfdc0 msgr2=0x55bb2093c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:07.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe007fff640 1 --2- 172.21.15.172:0/2125761241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bfdc0 0x55bb2093c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe007fff640 1 -- 172.21.15.172:0/2125761241 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb208bc6c0 con 0x55bb208bf4a0 2024-09-10T08:52:07.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.911+0000 7fe00f7fe640 1 --2- 172.21.15.172:0/2125761241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bfdc0 0x55bb2093c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:07.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.912+0000 7fe007fff640 1 --2- 172.21.15.172:0/2125761241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bf4a0 0x55bb208d04c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fdff80058c0 tx=0x7fdff800b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6079b27029b9cc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:07.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.912+0000 7fdff67fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:07.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.912+0000 7fdff67fc640 1 -- 172.21.15.172:0/2125761241 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff801b870 con 0x55bb208bf4a0 2024-09-10T08:52:07.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.912+0000 7fdff67fc640 1 -- 172.21.15.172:0/2125761241 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fdff801ba10 con 0x55bb208bf4a0 2024-09-10T08:52:07.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.912+0000 7fe017954740 1 -- 172.21.15.172:0/2125761241 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0000054d0 con 0x55bb208bf4a0 2024-09-10T08:52:07.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.912+0000 7fdff67fc640 1 -- 172.21.15.172:0/2125761241 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdff803d050 con 0x55bb208bf4a0 2024-09-10T08:52:07.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.912+0000 7fe017954740 10 client.17202 fetch_fsmap learned FSMap version 36 2024-09-10T08:52:07.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.912+0000 7fe017954740 1 -- 172.21.15.172:0/2125761241 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55bb20944d40 con 0x55bb208bf4a0 2024-09-10T08:52:07.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.912+0000 7fdff67fc640 1 -- 172.21.15.172:0/2125761241 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0000054d0 con 0x55bb208bf4a0 2024-09-10T08:52:07.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.912+0000 7fdff67fc640 1 -- 172.21.15.172:0/2125761241 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7fdff8024070 con 0x55bb208bf4a0 2024-09-10T08:52:07.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.912+0000 7fe017954740 10 client.17202 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T08:52:07.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.912+0000 7fe017954740 10 client.17202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:07.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.912+0000 7fe017954740 20 client.17202 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:07.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.912+0000 7fe017954740 1 --2- 172.21.15.172:0/2125761241 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bb208ca680 0x55bb20947500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:07.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.912+0000 7fe017954740 4 client.17202 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:52:07.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.912+0000 7fe017954740 1 -- 172.21.15.172:0/2125761241 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bb2082cd60 con 0x55bb208ca680 2024-09-10T08:52:07.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.913+0000 7fe00effd640 1 --2- 172.21.15.172:0/2125761241 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bb208ca680 0x55bb20947500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.913+0000 7fe00effd640 1 --2- 172.21.15.172:0/2125761241 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bb208ca680 0x55bb20947500 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edb3737889f9d04a server_cookie=77b41ce7c4dcd618 in_seq=0 out_seq=0 2024-09-10T08:52:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.913+0000 7fdff67fc640 10 client.17202 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.913+0000 7fdff67fc640 1 -- 172.21.15.172:0/2125761241 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bb2082cd60 con 0x55bb208ca680 2024-09-10T08:52:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.913+0000 7fdff67fc640 10 client.17202 handle_command_reply: tid=0 2024-09-10T08:52:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.913+0000 7fe017954740 1 client.17202 shutdown 2024-09-10T08:52:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.913+0000 7fe017954740 1 -- 172.21.15.172:0/2125761241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bf4a0 msgr2=0x55bb208d04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.913+0000 7fe017954740 1 --2- 172.21.15.172:0/2125761241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bf4a0 0x55bb208d04c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fdff80058c0 tx=0x7fdff800b530 comp rx=0 tx=0).stop 2024-09-10T08:52:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.913+0000 7fe017954740 1 -- 172.21.15.172:0/2125761241 shutdown_connections 2024-09-10T08:52:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.913+0000 7fe017954740 1 --2- 172.21.15.172:0/2125761241 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bb208ca680 0x55bb20947500 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.913+0000 7fe017954740 1 --2- 172.21.15.172:0/2125761241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb208bfdc0 0x55bb2093c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.913+0000 7fe017954740 1 --2- 172.21.15.172:0/2125761241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb208bf4a0 0x55bb208d04c0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.913+0000 7fdff67fc640 0 client.17202 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:07.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.913+0000 7fe017954740 1 --2- 172.21.15.172:0/2125761241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bb208bba30 0x55bb208cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.913+0000 7fe017954740 1 -- 172.21.15.172:0/2125761241 >> 172.21.15.172:0/2125761241 conn(0x55bb208c78c0 msgr2=0x55bb208c56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.914+0000 7fe017954740 1 -- 172.21.15.172:0/2125761241 shutdown_connections 2024-09-10T08:52:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.914+0000 7fe017954740 1 -- 172.21.15.172:0/2125761241 wait complete. 2024-09-10T08:52:07.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.914+0000 7fe017954740 20 client.17202 trim_cache size 0 max 16384 2024-09-10T08:52:07.921 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:52:07.921 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:52:07.921 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T08:52:07.921 INFO:teuthology.orchestra.run.smithi172.stdout: "10646d78-1fa3-44e4-9a01-a8ee703f14c1": { 2024-09-10T08:52:07.921 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T08:52:07.922 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "10646d78-1fa3-44e4-9a01-a8ee703f14c1", 2024-09-10T08:52:07.922 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T08:52:07.922 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:52:07.922 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:52:07.922 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:52:07.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.914+0000 7fe01557b640 1 -- 172.21.15.172:0/3452130596 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdffc0a2b80 msgr2=0x7fdffc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:07.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.914+0000 7fe01557b640 1 --2- 172.21.15.172:0/3452130596 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdffc0a2b80 0x7fdffc0a5040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55bb208bcdb0 tx=0x7fdff800b480 comp rx=0 tx=0).stop 2024-09-10T08:52:07.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.914+0000 7fe01557b640 1 -- 172.21.15.172:0/3452130596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe010106a90 msgr2=0x7fe01019e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:07.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.914+0000 7fe01557b640 1 --2- 172.21.15.172:0/3452130596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe010106a90 0x7fe01019e9c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe00800e940 tx=0x7fe00800ee00 comp rx=0 tx=0).stop 2024-09-10T08:52:07.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.914+0000 7fe01557b640 1 -- 172.21.15.172:0/3452130596 shutdown_connections 2024-09-10T08:52:07.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.914+0000 7fe01557b640 1 --2- 172.21.15.172:0/3452130596 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdffc0a2b80 0x7fdffc0a5040 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.914+0000 7fe01557b640 1 --2- 172.21.15.172:0/3452130596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe010106a90 0x7fe01019e9c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.914+0000 7fe01557b640 1 --2- 172.21.15.172:0/3452130596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe010101430 0x7fe0101a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.914+0000 7fe01557b640 1 --2- 172.21.15.172:0/3452130596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe010100a90 0x7fe0101a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:07.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.914+0000 7fe01557b640 1 -- 172.21.15.172:0/3452130596 >> 172.21.15.172:0/3452130596 conn(0x7fe0100fc7d0 msgr2=0x7fe01006a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:07.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.915+0000 7fe01557b640 1 -- 172.21.15.172:0/3452130596 shutdown_connections 2024-09-10T08:52:07.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:07.915+0000 7fe01557b640 1 -- 172.21.15.172:0/3452130596 wait complete. 2024-09-10T08:52:08.355 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:08 smithi172 ceph-mon[30854]: pgmap v527: 129 pgs: 129 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-09-10T08:52:08.682 INFO:tasks.cephfs.filesystem:scrub status for tag:10646d78-1fa3-44e4-9a01-a8ee703f14c1 - {'path': '/', 'tag': '10646d78-1fa3-44e4-9a01-a8ee703f14c1', 'options': 'recursive,force'} 2024-09-10T08:52:08.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:08 smithi174 ceph-mon[34068]: pgmap v527: 129 pgs: 129 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-09-10T08:52:08.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:08 smithi191 ceph-mon[35314]: pgmap v527: 129 pgs: 129 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 26 op/s 2024-09-10T08:52:09.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:09 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:52:09.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:09 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:52:09.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:09 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:52:10.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:10 smithi172 ceph-mon[30854]: pgmap v528: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 32 op/s 2024-09-10T08:52:10.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:10 smithi174 ceph-mon[34068]: pgmap v528: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 32 op/s 2024-09-10T08:52:10.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:10 smithi191 ceph-mon[35314]: pgmap v528: 129 pgs: 129 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 32 op/s 2024-09-10T08:52:12.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:12 smithi172 ceph-mon[30854]: pgmap v529: 129 pgs: 129 active+clean; 4.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 49 op/s 2024-09-10T08:52:12.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:12 smithi174 ceph-mon[34068]: pgmap v529: 129 pgs: 129 active+clean; 4.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 49 op/s 2024-09-10T08:52:12.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:12 smithi191 ceph-mon[35314]: pgmap v529: 129 pgs: 129 active+clean; 4.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 MiB/s wr, 49 op/s 2024-09-10T08:52:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.625+0000 7f52d7453640 1 -- 172.21.15.172:0/4082333480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52d01025c0 msgr2=0x7f52d01049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.625+0000 7f52d7453640 1 --2- 172.21.15.172:0/4082333480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52d01025c0 0x7f52d01049b0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009900 tx=0x7f52cc02a410 comp rx=0 tx=0).stop 2024-09-10T08:52:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.625+0000 7f52d7453640 1 -- 172.21.15.172:0/4082333480 shutdown_connections 2024-09-10T08:52:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.625+0000 7f52d7453640 1 --2- 172.21.15.172:0/4082333480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0107820 0x7f52d0109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.625+0000 7f52d7453640 1 --2- 172.21.15.172:0/4082333480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52d0104ef0 0x7f52d01072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.625+0000 7f52d7453640 1 --2- 172.21.15.172:0/4082333480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52d01025c0 0x7f52d01049b0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.625+0000 7f52d7453640 1 -- 172.21.15.172:0/4082333480 >> 172.21.15.172:0/4082333480 conn(0x7f52d00fc1e0 msgr2=0x7f52d00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.625+0000 7f52d7453640 1 -- 172.21.15.172:0/4082333480 shutdown_connections 2024-09-10T08:52:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.625+0000 7f52d7453640 1 -- 172.21.15.172:0/4082333480 wait complete. 2024-09-10T08:52:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.626+0000 7f52d7453640 1 Processor -- start 2024-09-10T08:52:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.626+0000 7f52d7453640 1 -- start start 2024-09-10T08:52:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.626+0000 7f52d7453640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52d01025c0 0x7f52d0101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.626+0000 7f52d7453640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0104ef0 0x7f52d0100570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.626+0000 7f52d7453640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52d0107820 0x7f52d0100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.626+0000 7f52d7453640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52d010d6b0 con 0x7f52d0107820 2024-09-10T08:52:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.626+0000 7f52d7453640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f52d010d530 con 0x7f52d0104ef0 2024-09-10T08:52:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.626+0000 7f52d7453640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f52d010d830 con 0x7f52d01025c0 2024-09-10T08:52:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.626+0000 7f52d59f0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52d0107820 0x7f52d0100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.626+0000 7f52d49ee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0104ef0 0x7f52d0100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.626+0000 7f52d49ee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0104ef0 0x7f52d0100570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:54802/0 (socket says 172.21.15.172:54802) 2024-09-10T08:52:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.626+0000 7f52d51ef640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52d01025c0 0x7f52d0101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.626+0000 7f52d49ee640 1 -- 172.21.15.172:0/3045408103 learned_addr learned my addr 172.21.15.172:0/3045408103 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.626+0000 7f52d59f0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52d0107820 0x7f52d0100ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33090/0 (socket says 172.21.15.172:33090) 2024-09-10T08:52:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.627+0000 7f52d59f0640 1 -- 172.21.15.172:0/3045408103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52d01025c0 msgr2=0x7f52d0101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.627+0000 7f52d59f0640 1 --2- 172.21.15.172:0/3045408103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52d01025c0 0x7f52d0101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.627+0000 7f52d59f0640 1 -- 172.21.15.172:0/3045408103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0104ef0 msgr2=0x7f52d0100570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.627+0000 7f52d59f0640 1 --2- 172.21.15.172:0/3045408103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0104ef0 0x7f52d0100570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.627+0000 7f52d59f0640 1 -- 172.21.15.172:0/3045408103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d0101340 con 0x7f52d0107820 2024-09-10T08:52:13.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.627+0000 7f52d49ee640 1 --2- 172.21.15.172:0/3045408103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0104ef0 0x7f52d0100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:13.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.627+0000 7f52d59f0640 1 --2- 172.21.15.172:0/3045408103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52d0107820 0x7f52d0100ab0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f52c8005fd0 tx=0x7f52c800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:13.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.627+0000 7f52c67fc640 1 -- 172.21.15.172:0/3045408103 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c800cdf0 con 0x7f52d0107820 2024-09-10T08:52:13.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.627+0000 7f52c67fc640 1 -- 172.21.15.172:0/3045408103 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52c8005020 con 0x7f52d0107820 2024-09-10T08:52:13.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.627+0000 7f52d7453640 1 -- 172.21.15.172:0/3045408103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d01a1e90 con 0x7f52d0107820 2024-09-10T08:52:13.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.627+0000 7f52c67fc640 1 -- 172.21.15.172:0/3045408103 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8010690 con 0x7f52d0107820 2024-09-10T08:52:13.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.627+0000 7f52d7453640 1 -- 172.21.15.172:0/3045408103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d01a22a0 con 0x7f52d0107820 2024-09-10T08:52:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.630+0000 7f52c67fc640 1 -- 172.21.15.172:0/3045408103 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f52c8004a40 con 0x7f52d0107820 2024-09-10T08:52:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.630+0000 7f52da1e9740 1 Processor -- start 2024-09-10T08:52:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.630+0000 7f52da1e9740 1 -- start start 2024-09-10T08:52:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.630+0000 7f52da1e9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb9099a0 0x55eabb909da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.636 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.630+0000 7f52da1e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90a2e0 0x55eabb90eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.630+0000 7f52da1e9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90f370 0x55eabb911830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.630+0000 7f52da1e9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55eabb86d2f0 con 0x55eabb90f370 2024-09-10T08:52:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.630+0000 7f52da1e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eabb815270 con 0x55eabb90a2e0 2024-09-10T08:52:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.630+0000 7f52da1e9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55eabb8692b0 con 0x55eabb9099a0 2024-09-10T08:52:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.631+0000 7f52d49ee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90f370 0x55eabb911830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.631+0000 7f52d51ef640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90a2e0 0x55eabb90eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.631+0000 7f52d51ef640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90a2e0 0x55eabb90eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:54818/0 (socket says 172.21.15.172:54818) 2024-09-10T08:52:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.631+0000 7f52d51ef640 1 -- 172.21.15.172:0/1133833001 learned_addr learned my addr 172.21.15.172:0/1133833001 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.631+0000 7f52d49ee640 1 --2- 172.21.15.172:0/1133833001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90f370 0x55eabb911830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33098/0 (socket says 172.21.15.172:33098) 2024-09-10T08:52:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.631+0000 7f52c67fc640 1 --2- 172.21.15.172:0/3045408103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f52bc0a2c30 0x7f52bc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.631+0000 7f52d49ee640 1 --2- 172.21.15.172:0/1133833001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb9099a0 0x55eabb909da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.631+0000 7f52d59f0640 1 --2- 172.21.15.172:0/3045408103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f52bc0a2c30 0x7f52bc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.631+0000 7f52d51ef640 1 -- 172.21.15.172:0/1133833001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb9099a0 msgr2=0x55eabb909da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.631+0000 7f52d51ef640 1 --2- 172.21.15.172:0/1133833001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb9099a0 0x55eabb909da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.631+0000 7f52d51ef640 1 -- 172.21.15.172:0/1133833001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90f370 msgr2=0x55eabb911830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.631+0000 7f52d51ef640 1 --2- 172.21.15.172:0/1133833001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90f370 0x55eabb911830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.631+0000 7f52d51ef640 1 -- 172.21.15.172:0/1133833001 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eabb814340 con 0x55eabb90a2e0 2024-09-10T08:52:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.631+0000 7f52d49ee640 1 --2- 172.21.15.172:0/1133833001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90f370 0x55eabb911830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.631+0000 7f52c67fc640 1 -- 172.21.15.172:0/3045408103 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f52c80cfa00 con 0x7f52d0107820 2024-09-10T08:52:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.631+0000 7f52d59f0640 1 --2- 172.21.15.172:0/3045408103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f52bc0a2c30 0x7f52bc0a50f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f52c80a4640 tx=0x7f52c80913e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.631+0000 7f52d51ef640 1 --2- 172.21.15.172:0/1133833001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90a2e0 0x55eabb90eda0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009900 tx=0x7f52cc005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85027fb2ba139513 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.632+0000 7f52bb7fe640 1 -- 172.21.15.172:0/1133833001 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc035030 con 0x55eabb90a2e0 2024-09-10T08:52:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.632+0000 7f52bb7fe640 1 -- 172.21.15.172:0/1133833001 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52cc040070 con 0x55eabb90a2e0 2024-09-10T08:52:13.638 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.632+0000 7f52bb7fe640 1 -- 172.21.15.172:0/1133833001 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0039d0 con 0x55eabb90a2e0 2024-09-10T08:52:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.632+0000 7f52da1e9740 1 -- 172.21.15.172:0/1133833001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90a2e0 msgr2=0x55eabb90eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.632+0000 7f52da1e9740 1 --2- 172.21.15.172:0/1133833001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90a2e0 0x55eabb90eda0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009900 tx=0x7f52cc005c00 comp rx=0 tx=0).stop 2024-09-10T08:52:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.632+0000 7f52da1e9740 1 -- 172.21.15.172:0/1133833001 shutdown_connections 2024-09-10T08:52:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.632+0000 7f52da1e9740 1 --2- 172.21.15.172:0/1133833001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90f370 0x55eabb911830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.632+0000 7f52da1e9740 1 --2- 172.21.15.172:0/1133833001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90a2e0 0x55eabb90eda0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.632+0000 7f52da1e9740 1 --2- 172.21.15.172:0/1133833001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb9099a0 0x55eabb909da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.633+0000 7f52da1e9740 1 -- 172.21.15.172:0/1133833001 >> 172.21.15.172:0/1133833001 conn(0x55eabb905010 msgr2=0x55eabb905410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.633+0000 7f52da1e9740 1 -- 172.21.15.172:0/1133833001 shutdown_connections 2024-09-10T08:52:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.633+0000 7f52da1e9740 1 -- 172.21.15.172:0/1133833001 wait complete. 2024-09-10T08:52:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.633+0000 7f52da1e9740 1 -- start start 2024-09-10T08:52:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.633+0000 7f52da1e9740 1 Processor -- start 2024-09-10T08:52:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.633+0000 7f52da1e9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb9099a0 0x55eabb913b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.633+0000 7f52da1e9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90a2e0 0x55eabb9140a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.633+0000 7f52da1e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90f370 0x55eabb91b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.633+0000 7f52da1e9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55eabb9122e0 con 0x55eabb90a2e0 2024-09-10T08:52:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.633+0000 7f52da1e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eabb7f6290 con 0x55eabb90f370 2024-09-10T08:52:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.633+0000 7f52da1e9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55eabb912460 con 0x55eabb9099a0 2024-09-10T08:52:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.633+0000 7f52da1e9740 10 client.0 _ll_register_callbacks cb 0x55eabb827140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52d51ef640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90a2e0 0x55eabb9140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52d51ef640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90a2e0 0x55eabb9140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33110/0 (socket says 172.21.15.172:33110) 2024-09-10T08:52:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52d51ef640 1 -- 172.21.15.172:0/3129940073 learned_addr learned my addr 172.21.15.172:0/3129940073 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52d49ee640 1 --2- 172.21.15.172:0/3129940073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90f370 0x55eabb91b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52d49ee640 1 --2- 172.21.15.172:0/3129940073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb9099a0 0x55eabb913b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52d51ef640 1 -- 172.21.15.172:0/3129940073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb9099a0 msgr2=0x55eabb913b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:13.640 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52d51ef640 1 --2- 172.21.15.172:0/3129940073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb9099a0 0x55eabb913b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52d51ef640 1 -- 172.21.15.172:0/3129940073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90f370 msgr2=0x55eabb91b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52d51ef640 1 --2- 172.21.15.172:0/3129940073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90f370 0x55eabb91b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52d51ef640 1 -- 172.21.15.172:0/3129940073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eabb580210 con 0x55eabb90a2e0 2024-09-10T08:52:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52d49ee640 1 --2- 172.21.15.172:0/3129940073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb9099a0 0x55eabb913b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52d51ef640 1 --2- 172.21.15.172:0/3129940073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90a2e0 0x55eabb9140a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f52cc02a930 tx=0x7f52cc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd6a33e0b9645ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52b8ff9640 1 -- 172.21.15.172:0/3129940073 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc051060 con 0x55eabb90a2e0 2024-09-10T08:52:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52b8ff9640 1 -- 172.21.15.172:0/3129940073 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f52cc03c850 con 0x55eabb90a2e0 2024-09-10T08:52:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52da1e9740 1 -- 172.21.15.172:0/3129940073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52cc040070 con 0x55eabb90a2e0 2024-09-10T08:52:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52b8ff9640 1 -- 172.21.15.172:0/3129940073 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f52cc058050 con 0x55eabb90a2e0 2024-09-10T08:52:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52da1e9740 10 client.17214 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.634+0000 7f52da1e9740 1 -- 172.21.15.172:0/3129940073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55eabb64c620 con 0x55eabb90a2e0 2024-09-10T08:52:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.635+0000 7f52b8ff9640 1 -- 172.21.15.172:0/3129940073 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52cc040070 con 0x55eabb90a2e0 2024-09-10T08:52:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.635+0000 7f52b8ff9640 1 -- 172.21.15.172:0/3129940073 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f52cc044890 con 0x55eabb90a2e0 2024-09-10T08:52:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.635+0000 7f52da1e9740 10 client.17214 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.635+0000 7f52da1e9740 10 client.17214 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.635+0000 7f52da1e9740 20 client.17214 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.635+0000 7f52da1e9740 1 --2- 172.21.15.172:0/3129940073 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55eabb91f3e0 0x55eabb9217d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.635+0000 7f52da1e9740 4 client.17214 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.635+0000 7f52da1e9740 1 -- 172.21.15.172:0/3129940073 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eabb87aa20 con 0x55eabb91f3e0 2024-09-10T08:52:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.635+0000 7f52d49ee640 1 --2- 172.21.15.172:0/3129940073 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55eabb91f3e0 0x55eabb9217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.636+0000 7f52b8ff9640 10 client.17214 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.636+0000 7f52d49ee640 1 --2- 172.21.15.172:0/3129940073 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55eabb91f3e0 0x55eabb9217d0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8b9a420c4a9d3bf server_cookie=b12e6b97f87ca4df in_seq=0 out_seq=0 2024-09-10T08:52:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.636+0000 7f52b8ff9640 1 -- 172.21.15.172:0/3129940073 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eabb87aa20 con 0x55eabb91f3e0 2024-09-10T08:52:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.636+0000 7f52b8ff9640 10 client.17214 handle_command_reply: tid=0 2024-09-10T08:52:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.636+0000 7f52da1e9740 1 client.17214 shutdown 2024-09-10T08:52:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.636+0000 7f52da1e9740 1 -- 172.21.15.172:0/3129940073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90a2e0 msgr2=0x55eabb9140a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.636+0000 7f52da1e9740 1 --2- 172.21.15.172:0/3129940073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90a2e0 0x55eabb9140a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f52cc02a930 tx=0x7f52cc02a410 comp rx=0 tx=0).stop 2024-09-10T08:52:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.636+0000 7f52da1e9740 1 -- 172.21.15.172:0/3129940073 shutdown_connections 2024-09-10T08:52:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.636+0000 7f52da1e9740 1 --2- 172.21.15.172:0/3129940073 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55eabb91f3e0 0x55eabb9217d0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.636+0000 7f52da1e9740 1 --2- 172.21.15.172:0/3129940073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90f370 0x55eabb91b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.636+0000 7f52b8ff9640 0 client.17214 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.636+0000 7f52da1e9740 1 --2- 172.21.15.172:0/3129940073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90a2e0 0x55eabb9140a0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.636+0000 7f52da1e9740 1 --2- 172.21.15.172:0/3129940073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb9099a0 0x55eabb913b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.636+0000 7f52da1e9740 1 -- 172.21.15.172:0/3129940073 >> 172.21.15.172:0/3129940073 conn(0x55eabb905010 msgr2=0x55eabb906820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.637+0000 7f52da1e9740 1 -- 172.21.15.172:0/3129940073 shutdown_connections 2024-09-10T08:52:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.637+0000 7f52da1e9740 1 -- 172.21.15.172:0/3129940073 wait complete. 2024-09-10T08:52:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.637+0000 7f52da1e9740 20 client.17214 trim_cache size 0 max 16384 2024-09-10T08:52:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.655+0000 7f52da1e9740 1 Processor -- start 2024-09-10T08:52:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.655+0000 7f52da1e9740 1 -- start start 2024-09-10T08:52:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.655+0000 7f52da1e9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb9099a0 0x55eabb9053f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.655+0000 7f52da1e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90a2e0 0x55eabb91b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.655+0000 7f52da1e9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90f370 0x55eabb905930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.655+0000 7f52da1e9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52d01ac6b0 con 0x55eabb90f370 2024-09-10T08:52:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.655+0000 7f52da1e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f52d01ae1f0 con 0x55eabb90a2e0 2024-09-10T08:52:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.655+0000 7f52da1e9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f52d01acd80 con 0x55eabb9099a0 2024-09-10T08:52:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.655+0000 7f52d49ee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90f370 0x55eabb905930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.655+0000 7f52d49ee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90f370 0x55eabb905930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33124/0 (socket says 172.21.15.172:33124) 2024-09-10T08:52:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.655+0000 7f52d49ee640 1 -- 172.21.15.172:0/570088717 learned_addr learned my addr 172.21.15.172:0/570088717 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.656+0000 7f52d51ef640 1 --2- 172.21.15.172:0/570088717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90a2e0 0x55eabb91b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.656+0000 7f52d49ee640 1 --2- 172.21.15.172:0/570088717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb9099a0 0x55eabb9053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.656+0000 7f52d51ef640 1 -- 172.21.15.172:0/570088717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb9099a0 msgr2=0x55eabb9053f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.656+0000 7f52d51ef640 1 --2- 172.21.15.172:0/570088717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb9099a0 0x55eabb9053f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.656+0000 7f52d51ef640 1 -- 172.21.15.172:0/570088717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90f370 msgr2=0x55eabb905930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.656+0000 7f52d51ef640 1 --2- 172.21.15.172:0/570088717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90f370 0x55eabb905930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.656+0000 7f52d51ef640 1 -- 172.21.15.172:0/570088717 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eabb912b30 con 0x55eabb90a2e0 2024-09-10T08:52:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.656+0000 7f52d49ee640 1 --2- 172.21.15.172:0/570088717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90f370 0x55eabb905930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.656+0000 7f52d51ef640 1 --2- 172.21.15.172:0/570088717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90a2e0 0x55eabb91b0a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0092a0 tx=0x7f52cc02ab10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab14d0a69acc404e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.656+0000 7f52baffd640 1 -- 172.21.15.172:0/570088717 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc035070 con 0x55eabb90a2e0 2024-09-10T08:52:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.656+0000 7f52baffd640 1 -- 172.21.15.172:0/570088717 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52cc040040 con 0x55eabb90a2e0 2024-09-10T08:52:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.656+0000 7f52baffd640 1 -- 172.21.15.172:0/570088717 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc03cd70 con 0x55eabb90a2e0 2024-09-10T08:52:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.657+0000 7f52da1e9740 1 -- 172.21.15.172:0/570088717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90a2e0 msgr2=0x55eabb91b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.657+0000 7f52da1e9740 1 --2- 172.21.15.172:0/570088717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90a2e0 0x55eabb91b0a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0092a0 tx=0x7f52cc02ab10 comp rx=0 tx=0).stop 2024-09-10T08:52:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.657+0000 7f52da1e9740 1 -- 172.21.15.172:0/570088717 shutdown_connections 2024-09-10T08:52:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.657+0000 7f52da1e9740 1 --2- 172.21.15.172:0/570088717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb90f370 0x55eabb905930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.657+0000 7f52da1e9740 1 --2- 172.21.15.172:0/570088717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90a2e0 0x55eabb91b0a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.657+0000 7f52da1e9740 1 --2- 172.21.15.172:0/570088717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb9099a0 0x55eabb9053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.657+0000 7f52da1e9740 1 -- 172.21.15.172:0/570088717 >> 172.21.15.172:0/570088717 conn(0x55eabb905010 msgr2=0x55eabb913b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.657+0000 7f52da1e9740 1 -- 172.21.15.172:0/570088717 shutdown_connections 2024-09-10T08:52:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.657+0000 7f52da1e9740 1 -- 172.21.15.172:0/570088717 wait complete. 2024-09-10T08:52:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.657+0000 7f52da1e9740 1 -- start start 2024-09-10T08:52:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52da1e9740 1 Processor -- start 2024-09-10T08:52:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52da1e9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb9099a0 0x55eabb917f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52da1e9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb90a2e0 0x55eabb918480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52da1e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90f370 0x55eabb987ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52da1e9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52d01adb20 con 0x55eabb9099a0 2024-09-10T08:52:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52da1e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f52d01ad450 con 0x55eabb90f370 2024-09-10T08:52:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52da1e9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f52d01aa690 con 0x55eabb90a2e0 2024-09-10T08:52:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52da1e9740 10 client.0 _ll_register_callbacks cb 0x7f52c000bcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52d49ee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb9099a0 0x55eabb917f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52d49ee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb9099a0 0x55eabb917f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33132/0 (socket says 172.21.15.172:33132) 2024-09-10T08:52:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52d49ee640 1 -- 172.21.15.172:0/4054642876 learned_addr learned my addr 172.21.15.172:0/4054642876 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52d51ef640 1 --2- 172.21.15.172:0/4054642876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb90a2e0 0x55eabb918480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52d49ee640 1 --2- 172.21.15.172:0/4054642876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90f370 0x55eabb987ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52d49ee640 1 -- 172.21.15.172:0/4054642876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb90a2e0 msgr2=0x55eabb918480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52d49ee640 1 --2- 172.21.15.172:0/4054642876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb90a2e0 0x55eabb918480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52d49ee640 1 -- 172.21.15.172:0/4054642876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90f370 msgr2=0x55eabb987ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52d49ee640 1 --2- 172.21.15.172:0/4054642876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90f370 0x55eabb987ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52d49ee640 1 -- 172.21.15.172:0/4054642876 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eabb988530 con 0x55eabb9099a0 2024-09-10T08:52:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52d49ee640 1 --2- 172.21.15.172:0/4054642876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb9099a0 0x55eabb917f40 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f52d0101ca0 tx=0x7f52c0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6602c6a6eabf83 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.658+0000 7f52b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.659+0000 7f52b8ff9640 1 -- 172.21.15.172:0/4054642876 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0034060 con 0x55eabb9099a0 2024-09-10T08:52:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.659+0000 7f52b8ff9640 1 -- 172.21.15.172:0/4054642876 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f52c0023860 con 0x55eabb9099a0 2024-09-10T08:52:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.659+0000 7f52da1e9740 1 -- 172.21.15.172:0/4054642876 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52cc040040 con 0x55eabb9099a0 2024-09-10T08:52:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.659+0000 7f52b8ff9640 1 -- 172.21.15.172:0/4054642876 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f52c003b050 con 0x55eabb9099a0 2024-09-10T08:52:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.659+0000 7f52da1e9740 10 client.17220 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.659+0000 7f52da1e9740 1 -- 172.21.15.172:0/4054642876 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55eabb989810 con 0x55eabb9099a0 2024-09-10T08:52:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.659+0000 7f52b8ff9640 1 -- 172.21.15.172:0/4054642876 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52cc040040 con 0x55eabb9099a0 2024-09-10T08:52:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.659+0000 7f52b8ff9640 1 -- 172.21.15.172:0/4054642876 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f52c002b070 con 0x55eabb9099a0 2024-09-10T08:52:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.659+0000 7f52da1e9740 10 client.17220 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.659+0000 7f52da1e9740 10 client.17220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.659+0000 7f52da1e9740 20 client.17220 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.659+0000 7f52da1e9740 1 --2- 172.21.15.172:0/4054642876 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55eabb91f3e0 0x55eabb98c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.659+0000 7f52da1e9740 4 client.17220 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:52:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.659+0000 7f52da1e9740 1 -- 172.21.15.172:0/4054642876 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eabb874d70 con 0x55eabb91f3e0 2024-09-10T08:52:13.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.660+0000 7f52d51ef640 1 --2- 172.21.15.172:0/4054642876 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55eabb91f3e0 0x55eabb98c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:13.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.660+0000 7f52d51ef640 1 --2- 172.21.15.172:0/4054642876 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55eabb91f3e0 0x55eabb98c1e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5168ed3bb9bb5fcd server_cookie=930b178e2e44db6b in_seq=0 out_seq=0 2024-09-10T08:52:13.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.660+0000 7f52b8ff9640 10 client.17220 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.666+0000 7f52b8ff9640 1 -- 172.21.15.172:0/4054642876 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eabb874d70 con 0x55eabb91f3e0 2024-09-10T08:52:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.666+0000 7f52b8ff9640 10 client.17220 handle_command_reply: tid=0 2024-09-10T08:52:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.666+0000 7f52da1e9740 1 client.17220 shutdown 2024-09-10T08:52:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.666+0000 7f52da1e9740 1 -- 172.21.15.172:0/4054642876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb9099a0 msgr2=0x55eabb917f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.666+0000 7f52da1e9740 1 --2- 172.21.15.172:0/4054642876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb9099a0 0x55eabb917f40 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f52d0101ca0 tx=0x7f52c0014040 comp rx=0 tx=0).stop 2024-09-10T08:52:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.666+0000 7f52da1e9740 1 -- 172.21.15.172:0/4054642876 shutdown_connections 2024-09-10T08:52:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.666+0000 7f52da1e9740 1 --2- 172.21.15.172:0/4054642876 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55eabb91f3e0 0x55eabb98c1e0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.666+0000 7f52da1e9740 1 --2- 172.21.15.172:0/4054642876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eabb90f370 0x55eabb987ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.666+0000 7f52da1e9740 1 --2- 172.21.15.172:0/4054642876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eabb90a2e0 0x55eabb918480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.666+0000 7f52da1e9740 1 --2- 172.21.15.172:0/4054642876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eabb9099a0 0x55eabb917f40 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.666+0000 7f52b8ff9640 0 client.17220 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.666+0000 7f52da1e9740 1 -- 172.21.15.172:0/4054642876 >> 172.21.15.172:0/4054642876 conn(0x55eabb905010 msgr2=0x55eabb9039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.666+0000 7f52da1e9740 1 -- 172.21.15.172:0/4054642876 shutdown_connections 2024-09-10T08:52:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.666+0000 7f52da1e9740 1 -- 172.21.15.172:0/4054642876 wait complete. 2024-09-10T08:52:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.666+0000 7f52da1e9740 20 client.17220 trim_cache size 0 max 16384 2024-09-10T08:52:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:52:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:52:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:52:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:52:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.667+0000 7f52d7453640 1 -- 172.21.15.172:0/3045408103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f52bc0a2c30 msgr2=0x7f52bc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.667+0000 7f52d7453640 1 --2- 172.21.15.172:0/3045408103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f52bc0a2c30 0x7f52bc0a50f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f52c80a4640 tx=0x7f52c80913e0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.667+0000 7f52d7453640 1 -- 172.21.15.172:0/3045408103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52d0107820 msgr2=0x7f52d0100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.667+0000 7f52d7453640 1 --2- 172.21.15.172:0/3045408103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52d0107820 0x7f52d0100ab0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f52c8005fd0 tx=0x7f52c800ef10 comp rx=0 tx=0).stop 2024-09-10T08:52:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.667+0000 7f52d7453640 1 -- 172.21.15.172:0/3045408103 shutdown_connections 2024-09-10T08:52:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.667+0000 7f52d7453640 1 --2- 172.21.15.172:0/3045408103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f52bc0a2c30 0x7f52bc0a50f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.667+0000 7f52d7453640 1 --2- 172.21.15.172:0/3045408103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52d0107820 0x7f52d0100ab0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.667+0000 7f52d7453640 1 --2- 172.21.15.172:0/3045408103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0104ef0 0x7f52d0100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.667+0000 7f52d7453640 1 --2- 172.21.15.172:0/3045408103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52d01025c0 0x7f52d0101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.667+0000 7f52d7453640 1 -- 172.21.15.172:0/3045408103 >> 172.21.15.172:0/3045408103 conn(0x7f52d00fc1e0 msgr2=0x7f52d0103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.667+0000 7f52d7453640 1 -- 172.21.15.172:0/3045408103 shutdown_connections 2024-09-10T08:52:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:13.667+0000 7f52d7453640 1 -- 172.21.15.172:0/3045408103 wait complete. 2024-09-10T08:52:14.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:14 smithi172 ceph-mon[30854]: pgmap v530: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 43 op/s 2024-09-10T08:52:14.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:14 smithi174 ceph-mon[34068]: pgmap v530: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 43 op/s 2024-09-10T08:52:14.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:14 smithi191 ceph-mon[35314]: pgmap v530: 129 pgs: 129 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 43 op/s 2024-09-10T08:52:16.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:16 smithi174 ceph-mon[34068]: pgmap v531: 129 pgs: 129 active+clean; 4.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 MiB/s wr, 51 op/s 2024-09-10T08:52:16.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:16 smithi191 ceph-mon[35314]: pgmap v531: 129 pgs: 129 active+clean; 4.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 MiB/s wr, 51 op/s 2024-09-10T08:52:16.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:16 smithi172 ceph-mon[30854]: pgmap v531: 129 pgs: 129 active+clean; 4.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 MiB/s wr, 51 op/s 2024-09-10T08:52:17.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:17 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:52:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:17 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:52:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:17 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:52:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:17 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:52:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:17 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:52:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:17 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:52:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:17 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:52:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:17 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:52:17.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:17 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:52:17.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:17 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:52:17.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:17 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:52:17.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:17 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:52:18.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:18 smithi174 ceph-mon[34068]: pgmap v532: 129 pgs: 129 active+clean; 4.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 MiB/s wr, 45 op/s 2024-09-10T08:52:18.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:18 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:52:18.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:18 smithi191 ceph-mon[35314]: pgmap v532: 129 pgs: 129 active+clean; 4.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 MiB/s wr, 45 op/s 2024-09-10T08:52:18.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:18 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:52:18.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:18 smithi172 ceph-mon[30854]: pgmap v532: 129 pgs: 129 active+clean; 4.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 MiB/s wr, 45 op/s 2024-09-10T08:52:18.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:18 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:52:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:52:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:52:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:52:19.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:52:19.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:52:19.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:52:19.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:52:19.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:52:19.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:52:20.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:20 smithi174 ceph-mon[34068]: pgmap v533: 129 pgs: 129 active+clean; 4.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 MiB/s wr, 40 op/s 2024-09-10T08:52:20.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:20 smithi191 ceph-mon[35314]: pgmap v533: 129 pgs: 129 active+clean; 4.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 MiB/s wr, 40 op/s 2024-09-10T08:52:20.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:20 smithi172 ceph-mon[30854]: pgmap v533: 129 pgs: 129 active+clean; 4.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 MiB/s wr, 40 op/s 2024-09-10T08:52:22.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:22 smithi174 ceph-mon[34068]: pgmap v534: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 38 op/s 2024-09-10T08:52:22.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:22 smithi191 ceph-mon[35314]: pgmap v534: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 38 op/s 2024-09-10T08:52:22.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:22 smithi172 ceph-mon[30854]: pgmap v534: 129 pgs: 129 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 38 op/s 2024-09-10T08:52:24.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:24 smithi174 ceph-mon[34068]: pgmap v535: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:52:24.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:24 smithi191 ceph-mon[35314]: pgmap v535: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:52:24.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:24 smithi172 ceph-mon[30854]: pgmap v535: 129 pgs: 129 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:52:25.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:25 smithi174 ceph-mon[34068]: pgmap v536: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 27 op/s 2024-09-10T08:52:25.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:25 smithi191 ceph-mon[35314]: pgmap v536: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 27 op/s 2024-09-10T08:52:25.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:25 smithi172 ceph-mon[30854]: pgmap v536: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 27 op/s 2024-09-10T08:52:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.764+0000 7f3cdbe77640 1 -- 172.21.15.172:0/80050047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd4102600 msgr2=0x7f3cd4108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.764+0000 7f3cdbe77640 1 --2- 172.21.15.172:0/80050047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd4102600 0x7f3cd4108e90 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009ca0 tx=0x7f3ccc02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.764+0000 7f3cdbe77640 1 -- 172.21.15.172:0/80050047 shutdown_connections 2024-09-10T08:52:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.764+0000 7f3cdbe77640 1 --2- 172.21.15.172:0/80050047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd4102600 0x7f3cd4108e90 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.764+0000 7f3cdbe77640 1 --2- 172.21.15.172:0/80050047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd4101c40 0x7f3cd41020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.764+0000 7f3cdbe77640 1 --2- 172.21.15.172:0/80050047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd4100a50 0x7f3cd4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.764+0000 7f3cdbe77640 1 -- 172.21.15.172:0/80050047 >> 172.21.15.172:0/80050047 conn(0x7f3cd40fc1b0 msgr2=0x7f3cd40fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.764+0000 7f3cdbe77640 1 -- 172.21.15.172:0/80050047 shutdown_connections 2024-09-10T08:52:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.764+0000 7f3cdbe77640 1 -- 172.21.15.172:0/80050047 wait complete. 2024-09-10T08:52:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.765+0000 7f3cdbe77640 1 Processor -- start 2024-09-10T08:52:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.765+0000 7f3cdbe77640 1 -- start start 2024-09-10T08:52:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.765+0000 7f3cdbe77640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd4100a50 0x7f3cd4197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.765+0000 7f3cdbe77640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd4101c40 0x7f3cd4198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.765+0000 7f3cdbe77640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd4102600 0x7f3cd419f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.765+0000 7f3cdbe77640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd410c5b0 con 0x7f3cd4100a50 2024-09-10T08:52:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.765+0000 7f3cdbe77640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd410c430 con 0x7f3cd4101c40 2024-09-10T08:52:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.765+0000 7f3cdbe77640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd410c730 con 0x7f3cd4102600 2024-09-10T08:52:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.765+0000 7f3cd9c13640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd4100a50 0x7f3cd4197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.765+0000 7f3cd9c13640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd4100a50 0x7f3cd4197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34600/0 (socket says 172.21.15.172:34600) 2024-09-10T08:52:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.765+0000 7f3cd9c13640 1 -- 172.21.15.172:0/4244905249 learned_addr learned my addr 172.21.15.172:0/4244905249 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.765+0000 7f3cda414640 1 --2- 172.21.15.172:0/4244905249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd4102600 0x7f3cd419f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.765+0000 7f3cd9412640 1 --2- 172.21.15.172:0/4244905249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd4101c40 0x7f3cd4198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.766+0000 7f3cd9c13640 1 -- 172.21.15.172:0/4244905249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd4102600 msgr2=0x7f3cd419f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.766+0000 7f3cd9c13640 1 --2- 172.21.15.172:0/4244905249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd4102600 0x7f3cd419f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.766+0000 7f3cd9c13640 1 -- 172.21.15.172:0/4244905249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd4101c40 msgr2=0x7f3cd4198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.766+0000 7f3cd9c13640 1 --2- 172.21.15.172:0/4244905249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd4101c40 0x7f3cd4198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.766+0000 7f3cd9c13640 1 -- 172.21.15.172:0/4244905249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd419f900 con 0x7f3cd4100a50 2024-09-10T08:52:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.766+0000 7f3cd9c13640 1 --2- 172.21.15.172:0/4244905249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd4100a50 0x7f3cd4197cf0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000e9c0 tx=0x7f3cd000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.766+0000 7f3ccaffd640 1 -- 172.21.15.172:0/4244905249 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd000cd80 con 0x7f3cd4100a50 2024-09-10T08:52:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.766+0000 7f3ccaffd640 1 -- 172.21.15.172:0/4244905249 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cd00035f0 con 0x7f3cd4100a50 2024-09-10T08:52:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.766+0000 7f3cdbe77640 1 -- 172.21.15.172:0/4244905249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd4102f90 con 0x7f3cd4100a50 2024-09-10T08:52:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.766+0000 7f3ccaffd640 1 -- 172.21.15.172:0/4244905249 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0010690 con 0x7f3cd4100a50 2024-09-10T08:52:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.766+0000 7f3cdbe77640 1 -- 172.21.15.172:0/4244905249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd41034e0 con 0x7f3cd4100a50 2024-09-10T08:52:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.769+0000 7f3ccaffd640 1 -- 172.21.15.172:0/4244905249 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3cd00037a0 con 0x7f3cd4100a50 2024-09-10T08:52:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.770+0000 7f3cdec0f740 1 Processor -- start 2024-09-10T08:52:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.770+0000 7f3cdec0f740 1 -- start start 2024-09-10T08:52:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.770+0000 7f3cdec0f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef297815b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.771+0000 7f3cda414640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef297815b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.771+0000 7f3cda414640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef297815b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34604/0 (socket says 172.21.15.172:34604) 2024-09-10T08:52:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.771+0000 7f3cda414640 1 -- 172.21.15.172:0/408318228 learned_addr learned my addr 172.21.15.172:0/408318228 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.771+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/408318228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef29781af0 0x55ef297865b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.771+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/408318228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29786b80 0x55ef29789040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.771+0000 7f3cdec0f740 1 -- 172.21.15.172:0/408318228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ef296dc2f0 con 0x55ef297811b0 2024-09-10T08:52:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.771+0000 7f3cdec0f740 1 -- 172.21.15.172:0/408318228 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef29684270 con 0x55ef29786b80 2024-09-10T08:52:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.771+0000 7f3cdec0f740 1 -- 172.21.15.172:0/408318228 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ef296d82b0 con 0x55ef29781af0 2024-09-10T08:52:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.771+0000 7f3cda414640 1 -- 172.21.15.172:0/408318228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef29781af0 msgr2=0x55ef297865b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.771+0000 7f3cda414640 1 --2- 172.21.15.172:0/408318228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef29781af0 0x55ef297865b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.771+0000 7f3cda414640 1 -- 172.21.15.172:0/408318228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29786b80 msgr2=0x55ef29789040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:52:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.771+0000 7f3cda414640 1 --2- 172.21.15.172:0/408318228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29786b80 0x55ef29789040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.771+0000 7f3cda414640 1 -- 172.21.15.172:0/408318228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef29683340 con 0x55ef297811b0 2024-09-10T08:52:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.772+0000 7f3ccaffd640 1 --2- 172.21.15.172:0/4244905249 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cc00a2c30 0x7f3cc00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.772+0000 7f3cda414640 1 --2- 172.21.15.172:0/408318228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef297815b0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc02aeb0 tx=0x7f3ccc005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a059a2ca38391e0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.772+0000 7f3cd9c13640 1 --2- 172.21.15.172:0/4244905249 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cc00a2c30 0x7f3cc00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.772+0000 7f3cbffff640 1 -- 172.21.15.172:0/408318228 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc031650 con 0x55ef297811b0 2024-09-10T08:52:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.772+0000 7f3cbffff640 1 -- 172.21.15.172:0/408318228 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ccc036030 con 0x55ef297811b0 2024-09-10T08:52:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.772+0000 7f3cbffff640 1 -- 172.21.15.172:0/408318228 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc0043c0 con 0x55ef297811b0 2024-09-10T08:52:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.772+0000 7f3ccaffd640 1 -- 172.21.15.172:0/4244905249 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3cd00cf7d0 con 0x7f3cd4100a50 2024-09-10T08:52:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.773+0000 7f3cdec0f740 1 -- 172.21.15.172:0/408318228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 msgr2=0x55ef297815b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.773+0000 7f3cd9c13640 1 --2- 172.21.15.172:0/4244905249 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cc00a2c30 0x7f3cc00a50f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00980a0 tx=0x7f3cd0009700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.773+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/408318228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef297815b0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc02aeb0 tx=0x7f3ccc005ef0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.773+0000 7f3cdec0f740 1 -- 172.21.15.172:0/408318228 shutdown_connections 2024-09-10T08:52:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.773+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/408318228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29786b80 0x55ef29789040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.773+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/408318228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef29781af0 0x55ef297865b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.773+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/408318228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef297815b0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.773+0000 7f3cdec0f740 1 -- 172.21.15.172:0/408318228 >> 172.21.15.172:0/408318228 conn(0x55ef29774010 msgr2=0x55ef29774410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.773+0000 7f3cdec0f740 1 -- 172.21.15.172:0/408318228 shutdown_connections 2024-09-10T08:52:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.773+0000 7f3cdec0f740 1 -- 172.21.15.172:0/408318228 wait complete. 2024-09-10T08:52:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.774+0000 7f3cdec0f740 1 -- start start 2024-09-10T08:52:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.774+0000 7f3cdec0f740 1 Processor -- start 2024-09-10T08:52:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.774+0000 7f3cdec0f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef2978b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.774+0000 7f3cdec0f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef29781af0 0x55ef2978b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.774+0000 7f3cdec0f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29786b80 0x55ef29792790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.774+0000 7f3cdec0f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ef29789970 con 0x55ef297811b0 2024-09-10T08:52:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.774+0000 7f3cdec0f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef29665290 con 0x55ef29786b80 2024-09-10T08:52:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.774+0000 7f3cdec0f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ef29789af0 con 0x55ef29781af0 2024-09-10T08:52:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.774+0000 7f3cdec0f740 10 client.0 _ll_register_callbacks cb 0x55ef29696140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.774+0000 7f3cda414640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef2978b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.774+0000 7f3cda414640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef2978b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34608/0 (socket says 172.21.15.172:34608) 2024-09-10T08:52:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.774+0000 7f3cda414640 1 -- 172.21.15.172:0/1535095179 learned_addr learned my addr 172.21.15.172:0/1535095179 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.774+0000 7f3cd9412640 1 --2- 172.21.15.172:0/1535095179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef29781af0 0x55ef2978b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.775+0000 7f3cda414640 1 --2- 172.21.15.172:0/1535095179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29786b80 0x55ef29792790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.775+0000 7f3cda414640 1 -- 172.21.15.172:0/1535095179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef29781af0 msgr2=0x55ef2978b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.775+0000 7f3cda414640 1 --2- 172.21.15.172:0/1535095179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef29781af0 0x55ef2978b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.775+0000 7f3cda414640 1 -- 172.21.15.172:0/1535095179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29786b80 msgr2=0x55ef29792790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.775+0000 7f3cda414640 1 --2- 172.21.15.172:0/1535095179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29786b80 0x55ef29792790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.775+0000 7f3cda414640 1 -- 172.21.15.172:0/1535095179 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef293ef210 con 0x55ef297811b0 2024-09-10T08:52:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.775+0000 7f3cda414640 1 --2- 172.21.15.172:0/1535095179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef2978b280 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc004af0 tx=0x7f3ccc009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c9eca6a278e6bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.775+0000 7f3cbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.775+0000 7f3cbd7fa640 1 -- 172.21.15.172:0/1535095179 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc058060 con 0x55ef297811b0 2024-09-10T08:52:26.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.775+0000 7f3cdec0f740 1 -- 172.21.15.172:0/1535095179 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ccc036030 con 0x55ef297811b0 2024-09-10T08:52:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.775+0000 7f3cbd7fa640 1 -- 172.21.15.172:0/1535095179 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3ccc004340 con 0x55ef297811b0 2024-09-10T08:52:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.775+0000 7f3cbd7fa640 1 -- 172.21.15.172:0/1535095179 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3ccc05f050 con 0x55ef297811b0 2024-09-10T08:52:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.776+0000 7f3cdec0f740 10 client.17232 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.776+0000 7f3cdec0f740 1 -- 172.21.15.172:0/1535095179 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ef294bb620 con 0x55ef297811b0 2024-09-10T08:52:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.776+0000 7f3cbd7fa640 1 -- 172.21.15.172:0/1535095179 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ccc036030 con 0x55ef297811b0 2024-09-10T08:52:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.776+0000 7f3cbd7fa640 1 -- 172.21.15.172:0/1535095179 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f3ccc043070 con 0x55ef297811b0 2024-09-10T08:52:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.776+0000 7f3cdec0f740 10 client.17232 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.776+0000 7f3cdec0f740 10 client.17232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.776+0000 7f3cdec0f740 20 client.17232 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.776+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/1535095179 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ef29796b30 0x55ef29798f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.776+0000 7f3cdec0f740 4 client.17232 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.776+0000 7f3cdec0f740 1 -- 172.21.15.172:0/1535095179 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef296e9a20 con 0x55ef29796b30 2024-09-10T08:52:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.776+0000 7f3cd9412640 1 --2- 172.21.15.172:0/1535095179 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ef29796b30 0x55ef29798f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.777+0000 7f3cbd7fa640 10 client.17232 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.777+0000 7f3cd9412640 1 --2- 172.21.15.172:0/1535095179 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ef29796b30 0x55ef29798f20 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21ddd4c14dfee8d3 server_cookie=705a8e6bd1102ab6 in_seq=0 out_seq=0 2024-09-10T08:52:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.778+0000 7f3cbd7fa640 1 -- 172.21.15.172:0/1535095179 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef296e9a20 con 0x55ef29796b30 2024-09-10T08:52:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.778+0000 7f3cbd7fa640 10 client.17232 handle_command_reply: tid=0 2024-09-10T08:52:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.778+0000 7f3cdec0f740 1 client.17232 shutdown 2024-09-10T08:52:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.778+0000 7f3cdec0f740 1 -- 172.21.15.172:0/1535095179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 msgr2=0x55ef2978b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.778+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/1535095179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef2978b280 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc004af0 tx=0x7f3ccc009800 comp rx=0 tx=0).stop 2024-09-10T08:52:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.778+0000 7f3cdec0f740 1 -- 172.21.15.172:0/1535095179 shutdown_connections 2024-09-10T08:52:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.778+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/1535095179 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ef29796b30 0x55ef29798f20 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.778+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/1535095179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29786b80 0x55ef29792790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.778+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/1535095179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef29781af0 0x55ef2978b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.778+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/1535095179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef2978b280 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.778+0000 7f3cbd7fa640 0 client.17232 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.778+0000 7f3cdec0f740 1 -- 172.21.15.172:0/1535095179 >> 172.21.15.172:0/1535095179 conn(0x55ef29774010 msgr2=0x55ef29775820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.778+0000 7f3cdec0f740 1 -- 172.21.15.172:0/1535095179 shutdown_connections 2024-09-10T08:52:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.778+0000 7f3cdec0f740 1 -- 172.21.15.172:0/1535095179 wait complete. 2024-09-10T08:52:26.785 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.778+0000 7f3cdec0f740 20 client.17232 trim_cache size 0 max 16384 2024-09-10T08:52:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.800+0000 7f3cdec0f740 1 Processor -- start 2024-09-10T08:52:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cdec0f740 1 -- start start 2024-09-10T08:52:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cdec0f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29774010 0x55ef29781590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cdec0f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef297926b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cdec0f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef29781af0 0x55ef297730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cdec0f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41a82b0 con 0x55ef297811b0 2024-09-10T08:52:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cdec0f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41a9df0 con 0x55ef29774010 2024-09-10T08:52:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cdec0f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41a8980 con 0x55ef29781af0 2024-09-10T08:52:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cd9412640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef297926b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cd9412640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef297926b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34610/0 (socket says 172.21.15.172:34610) 2024-09-10T08:52:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cd9412640 1 -- 172.21.15.172:0/3281117300 learned_addr learned my addr 172.21.15.172:0/3281117300 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cda414640 1 --2- 172.21.15.172:0/3281117300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef29781af0 0x55ef297730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cda414640 1 --2- 172.21.15.172:0/3281117300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29774010 0x55ef29781590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cd9412640 1 -- 172.21.15.172:0/3281117300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef29781af0 msgr2=0x55ef297730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cd9412640 1 --2- 172.21.15.172:0/3281117300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef29781af0 0x55ef297730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cd9412640 1 -- 172.21.15.172:0/3281117300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29774010 msgr2=0x55ef29781590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cd9412640 1 --2- 172.21.15.172:0/3281117300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29774010 0x55ef29781590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cd9412640 1 -- 172.21.15.172:0/3281117300 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef29773e30 con 0x55ef297811b0 2024-09-10T08:52:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cda414640 1 --2- 172.21.15.172:0/3281117300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29774010 0x55ef29781590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.801+0000 7f3cd9412640 1 --2- 172.21.15.172:0/3281117300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef297926b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4005d00 tx=0x7f3cc40054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5871c260475af2b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.802+0000 7f3cbf7fe640 1 -- 172.21.15.172:0/3281117300 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4004b80 con 0x55ef297811b0 2024-09-10T08:52:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.802+0000 7f3cbf7fe640 1 -- 172.21.15.172:0/3281117300 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cc4004d20 con 0x55ef297811b0 2024-09-10T08:52:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.802+0000 7f3cbf7fe640 1 -- 172.21.15.172:0/3281117300 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4003e20 con 0x55ef297811b0 2024-09-10T08:52:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.802+0000 7f3cdec0f740 1 -- 172.21.15.172:0/3281117300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 msgr2=0x55ef297926b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.802+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/3281117300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef297926b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4005d00 tx=0x7f3cc40054d0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.802+0000 7f3cdec0f740 1 -- 172.21.15.172:0/3281117300 shutdown_connections 2024-09-10T08:52:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.802+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/3281117300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef29781af0 0x55ef297730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.802+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/3281117300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef297811b0 0x55ef297926b0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.802+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/3281117300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29774010 0x55ef29781590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.802+0000 7f3cdec0f740 1 -- 172.21.15.172:0/3281117300 >> 172.21.15.172:0/3281117300 conn(0x55ef29786710 msgr2=0x55ef297729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.803+0000 7f3cdec0f740 1 -- 172.21.15.172:0/3281117300 shutdown_connections 2024-09-10T08:52:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.803+0000 7f3cdec0f740 1 -- 172.21.15.172:0/3281117300 wait complete. 2024-09-10T08:52:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.803+0000 7f3cdec0f740 1 -- start start 2024-09-10T08:52:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.803+0000 7f3cdec0f740 1 Processor -- start 2024-09-10T08:52:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.803+0000 7f3cdec0f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29774010 0x55ef2978fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.803+0000 7f3cdec0f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef297811b0 0x55ef29790170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.803+0000 7f3cdec0f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef29781af0 0x55ef29791180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.803+0000 7f3cdec0f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41a9720 con 0x55ef29781af0 2024-09-10T08:52:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.803+0000 7f3cdec0f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41a9050 con 0x55ef29774010 2024-09-10T08:52:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.803+0000 7f3cdec0f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41a6290 con 0x55ef297811b0 2024-09-10T08:52:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.803+0000 7f3cdec0f740 10 client.0 _ll_register_callbacks cb 0x7f3ccc004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.803+0000 7f3cda414640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef29781af0 0x55ef29791180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.803+0000 7f3cd9412640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef297811b0 0x55ef29790170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.803+0000 7f3cda414640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef29781af0 0x55ef29791180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34622/0 (socket says 172.21.15.172:34622) 2024-09-10T08:52:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.803+0000 7f3cd9412640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef297811b0 0x55ef29790170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:38610/0 (socket says 172.21.15.172:38610) 2024-09-10T08:52:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.803+0000 7f3cda414640 1 -- 172.21.15.172:0/2919249611 learned_addr learned my addr 172.21.15.172:0/2919249611 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.803+0000 7f3cda414640 1 --2- 172.21.15.172:0/2919249611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29774010 0x55ef2978fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.804+0000 7f3cd9412640 1 -- 172.21.15.172:0/2919249611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29774010 msgr2=0x55ef2978fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.804+0000 7f3cd9412640 1 --2- 172.21.15.172:0/2919249611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29774010 0x55ef2978fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.804+0000 7f3cd9412640 1 -- 172.21.15.172:0/2919249611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef29781af0 msgr2=0x55ef29791180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:26.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.804+0000 7f3cd9412640 1 --2- 172.21.15.172:0/2919249611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef29781af0 0x55ef29791180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.804+0000 7f3cd9412640 1 -- 172.21.15.172:0/2919249611 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef29791790 con 0x55ef297811b0 2024-09-10T08:52:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.804+0000 7f3cda414640 1 --2- 172.21.15.172:0/2919249611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef29781af0 0x55ef29791180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.804+0000 7f3cd9412640 1 --2- 172.21.15.172:0/2919249611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef297811b0 0x55ef29790170 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4004700 tx=0x7f3cc400bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75db853f66a5c3fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.804+0000 7f3cbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:52:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.804+0000 7f3cbd7fa640 1 -- 172.21.15.172:0/2919249611 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4003b20 con 0x55ef297811b0 2024-09-10T08:52:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.804+0000 7f3cdec0f740 1 -- 172.21.15.172:0/2919249611 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc4004d20 con 0x55ef297811b0 2024-09-10T08:52:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.804+0000 7f3cbd7fa640 1 -- 172.21.15.172:0/2919249611 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3cc4021040 con 0x55ef297811b0 2024-09-10T08:52:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.804+0000 7f3cbd7fa640 1 -- 172.21.15.172:0/2919249611 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3cc4003cc0 con 0x55ef297811b0 2024-09-10T08:52:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.804+0000 7f3cdec0f740 10 client.26326 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.804+0000 7f3cdec0f740 1 -- 172.21.15.172:0/2919249611 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ef29792a10 con 0x55ef297811b0 2024-09-10T08:52:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.805+0000 7f3cbd7fa640 1 -- 172.21.15.172:0/2919249611 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cc4004d20 con 0x55ef297811b0 2024-09-10T08:52:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.805+0000 7f3cbd7fa640 1 -- 172.21.15.172:0/2919249611 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f3cc40119a0 con 0x55ef297811b0 2024-09-10T08:52:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.805+0000 7f3cdec0f740 10 client.26326 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.805+0000 7f3cdec0f740 10 client.26326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.805+0000 7f3cdec0f740 20 client.26326 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.805+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/2919249611 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ef29786b80 0x55ef29803f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.805+0000 7f3cdec0f740 4 client.26326 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:52:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.805+0000 7f3cdec0f740 1 -- 172.21.15.172:0/2919249611 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef296e3d70 con 0x55ef29786b80 2024-09-10T08:52:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.805+0000 7f3cda414640 1 --2- 172.21.15.172:0/2919249611 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ef29786b80 0x55ef29803f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.805+0000 7f3cbd7fa640 10 client.26326 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:26.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.805+0000 7f3cda414640 1 --2- 172.21.15.172:0/2919249611 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ef29786b80 0x55ef29803f50 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef1c2eba0a12772b server_cookie=6a579daa9303881c in_seq=0 out_seq=0 2024-09-10T08:52:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cbd7fa640 1 -- 172.21.15.172:0/2919249611 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef296e3d70 con 0x55ef29786b80 2024-09-10T08:52:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cbd7fa640 10 client.26326 handle_command_reply: tid=0 2024-09-10T08:52:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cdec0f740 1 client.26326 shutdown 2024-09-10T08:52:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cdec0f740 1 -- 172.21.15.172:0/2919249611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef297811b0 msgr2=0x55ef29790170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/2919249611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef297811b0 0x55ef29790170 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4004700 tx=0x7f3cc400bcd0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cdec0f740 1 -- 172.21.15.172:0/2919249611 shutdown_connections 2024-09-10T08:52:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/2919249611 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ef29786b80 0x55ef29803f50 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/2919249611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ef29781af0 0x55ef29791180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cbd7fa640 0 client.26326 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/2919249611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ef297811b0 0x55ef29790170 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cdec0f740 1 --2- 172.21.15.172:0/2919249611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef29774010 0x55ef2978fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cdec0f740 1 -- 172.21.15.172:0/2919249611 >> 172.21.15.172:0/2919249611 conn(0x55ef29786710 msgr2=0x55ef297756b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cdec0f740 1 -- 172.21.15.172:0/2919249611 shutdown_connections 2024-09-10T08:52:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cdec0f740 1 -- 172.21.15.172:0/2919249611 wait complete. 2024-09-10T08:52:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cdec0f740 20 client.26326 trim_cache size 0 max 16384 2024-09-10T08:52:26.814 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:52:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:52:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:52:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:52:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cdbe77640 1 -- 172.21.15.172:0/4244905249 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cc00a2c30 msgr2=0x7f3cc00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cdbe77640 1 --2- 172.21.15.172:0/4244905249 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cc00a2c30 0x7f3cc00a50f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00980a0 tx=0x7f3cd0009700 comp rx=0 tx=0).stop 2024-09-10T08:52:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cdbe77640 1 -- 172.21.15.172:0/4244905249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd4100a50 msgr2=0x7f3cd4197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.810+0000 7f3cdbe77640 1 --2- 172.21.15.172:0/4244905249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd4100a50 0x7f3cd4197cf0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000e9c0 tx=0x7f3cd000ee80 comp rx=0 tx=0).stop 2024-09-10T08:52:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.811+0000 7f3cdbe77640 1 -- 172.21.15.172:0/4244905249 shutdown_connections 2024-09-10T08:52:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.811+0000 7f3cdbe77640 1 --2- 172.21.15.172:0/4244905249 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cc00a2c30 0x7f3cc00a50f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.811+0000 7f3cdbe77640 1 --2- 172.21.15.172:0/4244905249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd4102600 0x7f3cd419f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.811+0000 7f3cdbe77640 1 --2- 172.21.15.172:0/4244905249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd4101c40 0x7f3cd4198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.811+0000 7f3cdbe77640 1 --2- 172.21.15.172:0/4244905249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd4100a50 0x7f3cd4197cf0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.811+0000 7f3cdbe77640 1 -- 172.21.15.172:0/4244905249 >> 172.21.15.172:0/4244905249 conn(0x7f3cd40fc1b0 msgr2=0x7f3cd406a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.811+0000 7f3cdbe77640 1 -- 172.21.15.172:0/4244905249 shutdown_connections 2024-09-10T08:52:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:26.811+0000 7f3cdbe77640 1 -- 172.21.15.172:0/4244905249 wait complete. 2024-09-10T08:52:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:28 smithi172 ceph-mon[30854]: pgmap v537: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 21 op/s 2024-09-10T08:52:28.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:28 smithi174 ceph-mon[34068]: pgmap v537: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 21 op/s 2024-09-10T08:52:28.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:28 smithi191 ceph-mon[35314]: pgmap v537: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 21 op/s 2024-09-10T08:52:30.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:30 smithi172 ceph-mon[30854]: pgmap v538: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-09-10T08:52:30.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:30 smithi174 ceph-mon[34068]: pgmap v538: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-09-10T08:52:30.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:30 smithi191 ceph-mon[35314]: pgmap v538: 129 pgs: 129 active+clean; 5.0 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-09-10T08:52:32.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:32 smithi172 ceph-mon[30854]: pgmap v539: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-09-10T08:52:32.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:32 smithi174 ceph-mon[34068]: pgmap v539: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-09-10T08:52:32.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:32 smithi191 ceph-mon[35314]: pgmap v539: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-09-10T08:52:34.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:34 smithi172 ceph-mon[30854]: pgmap v540: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:52:34.602 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:34 smithi191 ceph-mon[35314]: pgmap v540: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:52:34.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:34 smithi174 ceph-mon[34068]: pgmap v540: 129 pgs: 129 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:52:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:36 smithi172 ceph-mon[30854]: pgmap v541: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 26 op/s 2024-09-10T08:52:36.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:36 smithi174 ceph-mon[34068]: pgmap v541: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 26 op/s 2024-09-10T08:52:36.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:36 smithi191 ceph-mon[35314]: pgmap v541: 129 pgs: 129 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 26 op/s 2024-09-10T08:52:38.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:38 smithi172 ceph-mon[30854]: pgmap v542: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 21 op/s 2024-09-10T08:52:38.683 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:52:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:38 smithi174 ceph-mon[34068]: pgmap v542: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 21 op/s 2024-09-10T08:52:38.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:38 smithi191 ceph-mon[35314]: pgmap v542: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 29 MiB/s wr, 21 op/s 2024-09-10T08:52:38.966 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:52:39.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.561+0000 7f3bacd74640 1 -- 172.21.15.172:0/3015864738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ba8101420 msgr2=0x7f3ba810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.561+0000 7f3bacd74640 1 --2- 172.21.15.172:0/3015864738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ba8101420 0x7f3ba810d950 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009cd0 tx=0x7f3ba002a8e0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.561+0000 7f3bacd74640 1 -- 172.21.15.172:0/3015864738 shutdown_connections 2024-09-10T08:52:39.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.561+0000 7f3bacd74640 1 --2- 172.21.15.172:0/3015864738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ba8101420 0x7f3ba810d950 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.561+0000 7f3bacd74640 1 --2- 172.21.15.172:0/3015864738 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ba8100a80 0x7f3ba8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.561+0000 7f3bacd74640 1 --2- 172.21.15.172:0/3015864738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ba8106a80 0x7f3ba8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.561+0000 7f3bacd74640 1 -- 172.21.15.172:0/3015864738 >> 172.21.15.172:0/3015864738 conn(0x7f3ba80fc7a0 msgr2=0x7f3ba80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:39.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.561+0000 7f3bacd74640 1 -- 172.21.15.172:0/3015864738 shutdown_connections 2024-09-10T08:52:39.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.561+0000 7f3bacd74640 1 -- 172.21.15.172:0/3015864738 wait complete. 2024-09-10T08:52:39.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.562+0000 7f3bacd74640 1 Processor -- start 2024-09-10T08:52:39.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.562+0000 7f3bacd74640 1 -- start start 2024-09-10T08:52:39.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.562+0000 7f3bacd74640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ba8100a80 0x7f3ba81982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.562+0000 7f3bacd74640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ba8101420 0x7f3ba8198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.562+0000 7f3bacd74640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ba8106a80 0x7f3ba819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.562+0000 7f3bacd74640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba8110d30 con 0x7f3ba8101420 2024-09-10T08:52:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.562+0000 7f3bacd74640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba8110bb0 con 0x7f3ba8106a80 2024-09-10T08:52:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.562+0000 7f3bacd74640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba8110eb0 con 0x7f3ba8100a80 2024-09-10T08:52:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.562+0000 7f3ba5d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ba8101420 0x7f3ba8198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.562+0000 7f3ba5d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ba8101420 0x7f3ba8198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41480/0 (socket says 172.21.15.172:41480) 2024-09-10T08:52:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.562+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/1640446046 learned_addr learned my addr 172.21.15.172:0/1640446046 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.563+0000 7f3ba659c640 1 --2- 172.21.15.172:0/1640446046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ba8100a80 0x7f3ba81982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.563+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/1640446046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ba8100a80 msgr2=0x7f3ba81982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.563+0000 7f3ba5d9b640 1 --2- 172.21.15.172:0/1640446046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ba8100a80 0x7f3ba81982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.563+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/1640446046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ba8106a80 msgr2=0x7f3ba819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.563+0000 7f3ba6d9d640 1 --2- 172.21.15.172:0/1640446046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ba8106a80 0x7f3ba819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.563+0000 7f3ba5d9b640 1 --2- 172.21.15.172:0/1640446046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ba8106a80 0x7f3ba819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.563+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/1640446046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba819d2a0 con 0x7f3ba8101420 2024-09-10T08:52:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.563+0000 7f3ba659c640 1 --2- 172.21.15.172:0/1640446046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ba8100a80 0x7f3ba81982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.563+0000 7f3ba6d9d640 1 --2- 172.21.15.172:0/1640446046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ba8106a80 0x7f3ba819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.563+0000 7f3ba5d9b640 1 --2- 172.21.15.172:0/1640446046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ba8101420 0x7f3ba8198830 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00d950 tx=0x7f3b9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.563+0000 7f3b977fe640 1 -- 172.21.15.172:0/1640446046 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c017070 con 0x7f3ba8101420 2024-09-10T08:52:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.563+0000 7f3bacd74640 1 -- 172.21.15.172:0/1640446046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba819d590 con 0x7f3ba8101420 2024-09-10T08:52:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.563+0000 7f3bacd74640 1 -- 172.21.15.172:0/1640446046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba81a4e70 con 0x7f3ba8101420 2024-09-10T08:52:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.563+0000 7f3b977fe640 1 -- 172.21.15.172:0/1640446046 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b9c009c30 con 0x7f3ba8101420 2024-09-10T08:52:39.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.564+0000 7f3b977fe640 1 -- 172.21.15.172:0/1640446046 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c00f620 con 0x7f3ba8101420 2024-09-10T08:52:39.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.566+0000 7f3b977fe640 1 -- 172.21.15.172:0/1640446046 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3b9c00f7c0 con 0x7f3ba8101420 2024-09-10T08:52:39.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.567+0000 7f3b977fe640 1 --2- 172.21.15.172:0/1640446046 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3b900a2c00 0x7f3b900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.567+0000 7f3baf14d740 1 Processor -- start 2024-09-10T08:52:39.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.567+0000 7f3b977fe640 1 -- 172.21.15.172:0/1640446046 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3b9c0cee90 con 0x7f3ba8101420 2024-09-10T08:52:39.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.567+0000 7f3ba659c640 1 --2- 172.21.15.172:0/1640446046 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3b900a2c00 0x7f3b900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.567+0000 7f3baf14d740 1 -- start start 2024-09-10T08:52:39.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.568+0000 7f3baf14d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a027ff380 0x559a027ff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.568+0000 7f3baf14d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a0280e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.568+0000 7f3baf14d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 0x559a02810e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.568+0000 7f3baf14d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559a027652e0 con 0x559a0280ea90 2024-09-10T08:52:39.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.568+0000 7f3baf14d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559a0270cce0 con 0x559a027ff380 2024-09-10T08:52:39.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.568+0000 7f3baf14d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559a027612a0 con 0x559a027ffca0 2024-09-10T08:52:39.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.568+0000 7f3ba659c640 1 --2- 172.21.15.172:0/1640446046 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3b900a2c00 0x7f3b900a50c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3b98005d90 tx=0x7f3b98005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:39.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.568+0000 7f3ba5d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a0280e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.568+0000 7f3ba5d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a0280e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:48880/0 (socket says 172.21.15.172:48880) 2024-09-10T08:52:39.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.568+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/2290364941 learned_addr learned my addr 172.21.15.172:0/2290364941 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:39.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.568+0000 7f3ba6d9d640 1 --2- 172.21.15.172:0/2290364941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 0x559a02810e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.568+0000 7f3ba659c640 1 --2- 172.21.15.172:0/2290364941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a027ff380 0x559a027ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.568+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/2290364941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a027ff380 msgr2=0x559a027ff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:39.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.568+0000 7f3ba5d9b640 1 --2- 172.21.15.172:0/2290364941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a027ff380 0x559a027ff760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.568+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/2290364941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 msgr2=0x559a02810e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:39.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.568+0000 7f3ba5d9b640 1 --2- 172.21.15.172:0/2290364941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 0x559a02810e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.568+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/2290364941 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a026f1440 con 0x559a027ffca0 2024-09-10T08:52:39.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.569+0000 7f3ba5d9b640 1 --2- 172.21.15.172:0/2290364941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a0280e550 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c09b090 tx=0x7f3b9c009de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bef3ad0163fa97fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:39.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.569+0000 7f3b94ff9640 1 -- 172.21.15.172:0/2290364941 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c020050 con 0x559a027ffca0 2024-09-10T08:52:39.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.569+0000 7f3b94ff9640 1 -- 172.21.15.172:0/2290364941 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b9c0a4000 con 0x559a027ffca0 2024-09-10T08:52:39.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.569+0000 7f3b94ff9640 1 -- 172.21.15.172:0/2290364941 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0970e0 con 0x559a027ffca0 2024-09-10T08:52:39.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.570+0000 7f3baf14d740 1 -- 172.21.15.172:0/2290364941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 msgr2=0x559a0280e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.570+0000 7f3baf14d740 1 --2- 172.21.15.172:0/2290364941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a0280e550 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c09b090 tx=0x7f3b9c009de0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.570+0000 7f3baf14d740 1 -- 172.21.15.172:0/2290364941 shutdown_connections 2024-09-10T08:52:39.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.570+0000 7f3baf14d740 1 --2- 172.21.15.172:0/2290364941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 0x559a02810e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.570+0000 7f3baf14d740 1 --2- 172.21.15.172:0/2290364941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a0280e550 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.570+0000 7f3baf14d740 1 --2- 172.21.15.172:0/2290364941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a027ff380 0x559a027ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.570+0000 7f3baf14d740 1 -- 172.21.15.172:0/2290364941 >> 172.21.15.172:0/2290364941 conn(0x559a027fba30 msgr2=0x559a027fbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:39.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.570+0000 7f3baf14d740 1 -- 172.21.15.172:0/2290364941 shutdown_connections 2024-09-10T08:52:39.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.570+0000 7f3baf14d740 1 -- 172.21.15.172:0/2290364941 wait complete. 2024-09-10T08:52:39.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.570+0000 7f3baf14d740 1 -- start start 2024-09-10T08:52:39.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.571+0000 7f3baf14d740 1 Processor -- start 2024-09-10T08:52:39.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.571+0000 7f3baf14d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a027ff380 0x559a0280e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.571+0000 7f3baf14d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a0280b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.571+0000 7f3baf14d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a0280ea90 0x559a0280bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.571+0000 7f3baf14d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559a02811600 con 0x559a027ff380 2024-09-10T08:52:39.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.571+0000 7f3baf14d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559a02720050 con 0x559a0280ea90 2024-09-10T08:52:39.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.571+0000 7f3baf14d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559a02811780 con 0x559a027ffca0 2024-09-10T08:52:39.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.571+0000 7f3baf14d740 10 client.0 _ll_register_callbacks cb 0x559a02760160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:39.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.571+0000 7f3ba5d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a0280b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.571+0000 7f3ba5d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a0280b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:48894/0 (socket says 172.21.15.172:48894) 2024-09-10T08:52:39.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.571+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/2771858912 learned_addr learned my addr 172.21.15.172:0/2771858912 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:39.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.571+0000 7f3ba659c640 1 --2- 172.21.15.172:0/2771858912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a027ff380 0x559a0280e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.571+0000 7f3ba6d9d640 1 --2- 172.21.15.172:0/2771858912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a0280ea90 0x559a0280bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.572+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/2771858912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a0280ea90 msgr2=0x559a0280bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:39.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.572+0000 7f3ba5d9b640 1 --2- 172.21.15.172:0/2771858912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a0280ea90 0x559a0280bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.572+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/2771858912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a027ff380 msgr2=0x559a0280e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:39.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.572+0000 7f3ba5d9b640 1 --2- 172.21.15.172:0/2771858912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a027ff380 0x559a0280e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.572+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/2771858912 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a024d63c0 con 0x559a027ffca0 2024-09-10T08:52:39.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.572+0000 7f3ba659c640 1 --2- 172.21.15.172:0/2771858912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a027ff380 0x559a0280e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:39.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.572+0000 7f3ba5d9b640 1 --2- 172.21.15.172:0/2771858912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a0280b710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0cc680 tx=0x7f3b9c09f870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81a662ee9ebaf4f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:39.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.572+0000 7f3b8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:52:39.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.572+0000 7f3b8dffb640 1 -- 172.21.15.172:0/2771858912 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c014030 con 0x559a027ffca0 2024-09-10T08:52:39.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.572+0000 7f3baf14d740 1 -- 172.21.15.172:0/2771858912 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b9c0a4000 con 0x559a027ffca0 2024-09-10T08:52:39.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.572+0000 7f3b8dffb640 1 -- 172.21.15.172:0/2771858912 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3b9c09a9e0 con 0x559a027ffca0 2024-09-10T08:52:39.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.572+0000 7f3b8dffb640 1 -- 172.21.15.172:0/2771858912 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3b9c020070 con 0x559a027ffca0 2024-09-10T08:52:39.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.572+0000 7f3baf14d740 10 client.26338 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:39.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.572+0000 7f3baf14d740 1 -- 172.21.15.172:0/2771858912 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559a02811600 con 0x559a027ffca0 2024-09-10T08:52:39.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.573+0000 7f3b8dffb640 1 -- 172.21.15.172:0/2771858912 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b9c0a4000 con 0x559a027ffca0 2024-09-10T08:52:39.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.573+0000 7f3b8dffb640 1 -- 172.21.15.172:0/2771858912 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f3b9c0bb270 con 0x559a027ffca0 2024-09-10T08:52:39.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.573+0000 7f3baf14d740 10 client.26338 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:39.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.573+0000 7f3baf14d740 10 client.26338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:39.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.573+0000 7f3baf14d740 20 client.26338 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:39.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.573+0000 7f3baf14d740 1 --2- 172.21.15.172:0/2771858912 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559a0281e270 0x559a02820660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.573+0000 7f3baf14d740 4 client.26338 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:39.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.573+0000 7f3baf14d740 1 -- 172.21.15.172:0/2771858912 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a02772850 con 0x559a0281e270 2024-09-10T08:52:39.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.573+0000 7f3ba659c640 1 --2- 172.21.15.172:0/2771858912 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559a0281e270 0x559a02820660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.574+0000 7f3ba659c640 1 --2- 172.21.15.172:0/2771858912 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559a0281e270 0x559a02820660 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8111dde5f3b8e4c6 server_cookie=a93a32bfb2cd64a1 in_seq=0 out_seq=0 2024-09-10T08:52:39.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.574+0000 7f3b8dffb640 10 client.26338 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:39.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.574+0000 7f3b8dffb640 1 -- 172.21.15.172:0/2771858912 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a02772850 con 0x559a0281e270 2024-09-10T08:52:39.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.574+0000 7f3b8dffb640 10 client.26338 handle_command_reply: tid=0 2024-09-10T08:52:39.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.574+0000 7f3baf14d740 1 client.26338 shutdown 2024-09-10T08:52:39.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.574+0000 7f3baf14d740 1 -- 172.21.15.172:0/2771858912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 msgr2=0x559a0280b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.574+0000 7f3baf14d740 1 --2- 172.21.15.172:0/2771858912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a0280b710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0cc680 tx=0x7f3b9c09f870 comp rx=0 tx=0).stop 2024-09-10T08:52:39.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.575+0000 7f3baf14d740 1 -- 172.21.15.172:0/2771858912 shutdown_connections 2024-09-10T08:52:39.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.575+0000 7f3baf14d740 1 --2- 172.21.15.172:0/2771858912 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559a0281e270 0x559a02820660 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.575+0000 7f3baf14d740 1 --2- 172.21.15.172:0/2771858912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a0280ea90 0x559a0280bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.575+0000 7f3baf14d740 1 --2- 172.21.15.172:0/2771858912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a0280b710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.575+0000 7f3baf14d740 1 --2- 172.21.15.172:0/2771858912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a027ff380 0x559a0280e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.575+0000 7f3b8dffb640 0 client.26338 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:39.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.575+0000 7f3baf14d740 1 -- 172.21.15.172:0/2771858912 >> 172.21.15.172:0/2771858912 conn(0x559a027fba30 msgr2=0x559a02801790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:39.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.575+0000 7f3baf14d740 1 -- 172.21.15.172:0/2771858912 shutdown_connections 2024-09-10T08:52:39.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.575+0000 7f3baf14d740 1 -- 172.21.15.172:0/2771858912 wait complete. 2024-09-10T08:52:39.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.575+0000 7f3baf14d740 20 client.26338 trim_cache size 0 max 16384 2024-09-10T08:52:39.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.606+0000 7f3baf14d740 1 Processor -- start 2024-09-10T08:52:39.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.606+0000 7f3baf14d740 1 -- start start 2024-09-10T08:52:39.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.606+0000 7f3baf14d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a027ff380 0x559a0281a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.606+0000 7f3baf14d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a02825dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.606+0000 7f3baf14d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 0x559a02826310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.607+0000 7f3baf14d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8800fa70 con 0x559a0280ea90 2024-09-10T08:52:39.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.607+0000 7f3baf14d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b880115b0 con 0x559a027ff380 2024-09-10T08:52:39.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.607+0000 7f3baf14d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3b88010140 con 0x559a027ffca0 2024-09-10T08:52:39.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.607+0000 7f3ba6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 0x559a02826310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.607+0000 7f3ba6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 0x559a02826310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41518/0 (socket says 172.21.15.172:41518) 2024-09-10T08:52:39.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.607+0000 7f3ba6d9d640 1 -- 172.21.15.172:0/30318790 learned_addr learned my addr 172.21.15.172:0/30318790 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:39.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.607+0000 7f3ba5d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a02825dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.607+0000 7f3ba659c640 1 --2- 172.21.15.172:0/30318790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a027ff380 0x559a0281a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.607+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/30318790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a027ff380 msgr2=0x559a0281a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:39.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.607+0000 7f3ba5d9b640 1 --2- 172.21.15.172:0/30318790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a027ff380 0x559a0281a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.607+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/30318790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 msgr2=0x559a02826310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:39.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.607+0000 7f3ba5d9b640 1 --2- 172.21.15.172:0/30318790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 0x559a02826310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.607+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/30318790 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a0280c070 con 0x559a027ffca0 2024-09-10T08:52:39.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.607+0000 7f3ba6d9d640 1 --2- 172.21.15.172:0/30318790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 0x559a02826310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:39.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.607+0000 7f3ba659c640 1 --2- 172.21.15.172:0/30318790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a027ff380 0x559a0281a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:39.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.607+0000 7f3ba5d9b640 1 --2- 172.21.15.172:0/30318790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a02825dd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0977f0 tx=0x7f3b9c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=204eb81a9ef76b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:39.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.607+0000 7f3b8ffff640 1 -- 172.21.15.172:0/30318790 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c014070 con 0x559a027ffca0 2024-09-10T08:52:39.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.608+0000 7f3b8ffff640 1 -- 172.21.15.172:0/30318790 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b9c09f500 con 0x559a027ffca0 2024-09-10T08:52:39.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.608+0000 7f3b8ffff640 1 -- 172.21.15.172:0/30318790 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0b4330 con 0x559a027ffca0 2024-09-10T08:52:39.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.608+0000 7f3baf14d740 1 -- 172.21.15.172:0/30318790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 msgr2=0x559a02825dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.608+0000 7f3baf14d740 1 --2- 172.21.15.172:0/30318790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a02825dd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0977f0 tx=0x7f3b9c000bc0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.608+0000 7f3baf14d740 1 -- 172.21.15.172:0/30318790 shutdown_connections 2024-09-10T08:52:39.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.608+0000 7f3baf14d740 1 --2- 172.21.15.172:0/30318790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 0x559a02826310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.608+0000 7f3baf14d740 1 --2- 172.21.15.172:0/30318790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a02825dd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.608+0000 7f3baf14d740 1 --2- 172.21.15.172:0/30318790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a027ff380 0x559a0281a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.608+0000 7f3baf14d740 1 -- 172.21.15.172:0/30318790 >> 172.21.15.172:0/30318790 conn(0x559a027fba30 msgr2=0x559a02806d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:39.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.608+0000 7f3baf14d740 1 -- 172.21.15.172:0/30318790 shutdown_connections 2024-09-10T08:52:39.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.609+0000 7f3baf14d740 1 -- 172.21.15.172:0/30318790 wait complete. 2024-09-10T08:52:39.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.609+0000 7f3baf14d740 1 -- start start 2024-09-10T08:52:39.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.609+0000 7f3baf14d740 1 Processor -- start 2024-09-10T08:52:39.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.609+0000 7f3baf14d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a027ff380 0x559a028126c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.609+0000 7f3baf14d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a02812c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.609+0000 7f3baf14d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 0x559a02813140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.609+0000 7f3baf14d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3b88010ee0 con 0x559a0280ea90 2024-09-10T08:52:39.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.609+0000 7f3baf14d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b88010810 con 0x559a027ff380 2024-09-10T08:52:39.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.609+0000 7f3baf14d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8800da50 con 0x559a027ffca0 2024-09-10T08:52:39.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.609+0000 7f3baf14d740 10 client.0 _ll_register_callbacks cb 0x7f3ba0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:39.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.609+0000 7f3ba6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 0x559a02813140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.609+0000 7f3ba5d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a02812c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.609+0000 7f3ba6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 0x559a02813140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41534/0 (socket says 172.21.15.172:41534) 2024-09-10T08:52:39.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.610+0000 7f3ba6d9d640 1 -- 172.21.15.172:0/710620024 learned_addr learned my addr 172.21.15.172:0/710620024 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:39.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.610+0000 7f3ba659c640 1 --2- 172.21.15.172:0/710620024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a027ff380 0x559a028126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.610+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/710620024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a027ff380 msgr2=0x559a028126c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:39.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.610+0000 7f3ba5d9b640 1 --2- 172.21.15.172:0/710620024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a027ff380 0x559a028126c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.610+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/710620024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 msgr2=0x559a02813140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:39.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.610+0000 7f3ba5d9b640 1 --2- 172.21.15.172:0/710620024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 0x559a02813140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.610+0000 7f3ba5d9b640 1 -- 172.21.15.172:0/710620024 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a02887da0 con 0x559a027ffca0 2024-09-10T08:52:39.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.610+0000 7f3ba659c640 1 --2- 172.21.15.172:0/710620024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a027ff380 0x559a028126c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:39.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.610+0000 7f3ba6d9d640 1 --2- 172.21.15.172:0/710620024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 0x559a02813140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:39.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.610+0000 7f3ba5d9b640 1 --2- 172.21.15.172:0/710620024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a02812c00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c09b090 tx=0x7f3b9c0097b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fff33aa57359142 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:39.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.610+0000 7f3b8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:52:39.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.610+0000 7f3b8dffb640 1 -- 172.21.15.172:0/710620024 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c014070 con 0x559a027ffca0 2024-09-10T08:52:39.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.610+0000 7f3b8dffb640 1 -- 172.21.15.172:0/710620024 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3b9c09a9f0 con 0x559a027ffca0 2024-09-10T08:52:39.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.610+0000 7f3baf14d740 1 -- 172.21.15.172:0/710620024 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b9c09f500 con 0x559a027ffca0 2024-09-10T08:52:39.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.611+0000 7f3b8dffb640 1 -- 172.21.15.172:0/710620024 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3b9c09b160 con 0x559a027ffca0 2024-09-10T08:52:39.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.611+0000 7f3baf14d740 10 client.26344 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:39.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.611+0000 7f3baf14d740 1 -- 172.21.15.172:0/710620024 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559a02887da0 con 0x559a027ffca0 2024-09-10T08:52:39.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.611+0000 7f3b8dffb640 1 -- 172.21.15.172:0/710620024 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b9c09f500 con 0x559a027ffca0 2024-09-10T08:52:39.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.611+0000 7f3b8dffb640 1 -- 172.21.15.172:0/710620024 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f3b9c097270 con 0x559a027ffca0 2024-09-10T08:52:39.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.611+0000 7f3baf14d740 10 client.26344 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:39.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.611+0000 7f3baf14d740 10 client.26344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:39.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.611+0000 7f3baf14d740 20 client.26344 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:39.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.611+0000 7f3baf14d740 1 --2- 172.21.15.172:0/710620024 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559a02889300 0x559a0288b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.611+0000 7f3baf14d740 4 client.26344 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:52:39.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.611+0000 7f3baf14d740 1 -- 172.21.15.172:0/710620024 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a0276cd60 con 0x559a02889300 2024-09-10T08:52:39.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.612+0000 7f3ba659c640 1 --2- 172.21.15.172:0/710620024 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559a02889300 0x559a0288b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.612+0000 7f3b8dffb640 10 client.26344 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:39.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.612+0000 7f3ba659c640 1 --2- 172.21.15.172:0/710620024 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559a02889300 0x559a0288b6f0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc5342c02b3ced93 server_cookie=51e55f3f8c0c6175 in_seq=0 out_seq=0 2024-09-10T08:52:39.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.612+0000 7f3b8dffb640 1 -- 172.21.15.172:0/710620024 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559a0276cd60 con 0x559a02889300 2024-09-10T08:52:39.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.612+0000 7f3b8dffb640 10 client.26344 handle_command_reply: tid=0 2024-09-10T08:52:39.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.612+0000 7f3baf14d740 1 client.26344 shutdown 2024-09-10T08:52:39.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.612+0000 7f3baf14d740 1 -- 172.21.15.172:0/710620024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 msgr2=0x559a02812c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.612+0000 7f3baf14d740 1 --2- 172.21.15.172:0/710620024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a02812c00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c09b090 tx=0x7f3b9c0097b0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.612+0000 7f3baf14d740 1 -- 172.21.15.172:0/710620024 shutdown_connections 2024-09-10T08:52:39.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.612+0000 7f3baf14d740 1 --2- 172.21.15.172:0/710620024 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559a02889300 0x559a0288b6f0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.612+0000 7f3baf14d740 1 --2- 172.21.15.172:0/710620024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a0280ea90 0x559a02813140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.612+0000 7f3baf14d740 1 --2- 172.21.15.172:0/710620024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a027ffca0 0x559a02812c00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.612+0000 7f3baf14d740 1 --2- 172.21.15.172:0/710620024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a027ff380 0x559a028126c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.612+0000 7f3baf14d740 1 -- 172.21.15.172:0/710620024 >> 172.21.15.172:0/710620024 conn(0x559a027fba30 msgr2=0x559a0280cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:39.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.612+0000 7f3b8dffb640 0 client.26344 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:39.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.613+0000 7f3baf14d740 1 -- 172.21.15.172:0/710620024 shutdown_connections 2024-09-10T08:52:39.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.613+0000 7f3baf14d740 1 -- 172.21.15.172:0/710620024 wait complete. 2024-09-10T08:52:39.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.613+0000 7f3baf14d740 20 client.26344 trim_cache size 0 max 16384 2024-09-10T08:52:39.619 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:52:39.619 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T08:52:39.619 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T08:52:39.619 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:52:39.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.613+0000 7f3bacd74640 1 -- 172.21.15.172:0/1640446046 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3b900a2c00 msgr2=0x7f3b900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.613+0000 7f3bacd74640 1 --2- 172.21.15.172:0/1640446046 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3b900a2c00 0x7f3b900a50c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3b98005d90 tx=0x7f3b98005d20 comp rx=0 tx=0).stop 2024-09-10T08:52:39.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.613+0000 7f3bacd74640 1 -- 172.21.15.172:0/1640446046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ba8101420 msgr2=0x7f3ba8198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.613+0000 7f3bacd74640 1 --2- 172.21.15.172:0/1640446046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ba8101420 0x7f3ba8198830 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00d950 tx=0x7f3b9c00de10 comp rx=0 tx=0).stop 2024-09-10T08:52:39.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.613+0000 7f3bacd74640 1 -- 172.21.15.172:0/1640446046 shutdown_connections 2024-09-10T08:52:39.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.614+0000 7f3bacd74640 1 --2- 172.21.15.172:0/1640446046 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3b900a2c00 0x7f3b900a50c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.614+0000 7f3bacd74640 1 --2- 172.21.15.172:0/1640446046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ba8106a80 0x7f3ba819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.614+0000 7f3bacd74640 1 --2- 172.21.15.172:0/1640446046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ba8101420 0x7f3ba8198830 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.614+0000 7f3bacd74640 1 --2- 172.21.15.172:0/1640446046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ba8100a80 0x7f3ba81982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.614+0000 7f3bacd74640 1 -- 172.21.15.172:0/1640446046 >> 172.21.15.172:0/1640446046 conn(0x7f3ba80fc7a0 msgr2=0x7f3ba806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:39.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.614+0000 7f3bacd74640 1 -- 172.21.15.172:0/1640446046 shutdown_connections 2024-09-10T08:52:39.621 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:39.614+0000 7f3bacd74640 1 -- 172.21.15.172:0/1640446046 wait complete. 2024-09-10T08:52:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.917+0000 7fb51081e640 1 -- 172.21.15.172:0/58162839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb50c100a40 msgr2=0x7fb50c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.917+0000 7fb51081e640 1 --2- 172.21.15.172:0/58162839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb50c100a40 0x7fb50c100e40 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009900 tx=0x7fb4fc02a410 comp rx=0 tx=0).stop 2024-09-10T08:52:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.917+0000 7fb51081e640 1 -- 172.21.15.172:0/58162839 shutdown_connections 2024-09-10T08:52:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.917+0000 7fb51081e640 1 --2- 172.21.15.172:0/58162839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb50c1025f0 0x7fb50c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.917+0000 7fb51081e640 1 --2- 172.21.15.172:0/58162839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb50c101c30 0x7fb50c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.917+0000 7fb51081e640 1 --2- 172.21.15.172:0/58162839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb50c100a40 0x7fb50c100e40 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.917+0000 7fb51081e640 1 -- 172.21.15.172:0/58162839 >> 172.21.15.172:0/58162839 conn(0x7fb50c0fc1e0 msgr2=0x7fb50c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.917+0000 7fb51081e640 1 -- 172.21.15.172:0/58162839 shutdown_connections 2024-09-10T08:52:39.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.917+0000 7fb51081e640 1 -- 172.21.15.172:0/58162839 wait complete. 2024-09-10T08:52:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.917+0000 7fb51081e640 1 Processor -- start 2024-09-10T08:52:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb51081e640 1 -- start start 2024-09-10T08:52:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb51081e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb50c100a40 0x7fb50c197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb51081e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb50c101c30 0x7fb50c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb51081e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb50c1025f0 0x7fb50c19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb51081e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c10c570 con 0x7fb50c100a40 2024-09-10T08:52:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb51081e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c10c3f0 con 0x7fb50c1025f0 2024-09-10T08:52:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb51081e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c10c6f0 con 0x7fb50c101c30 2024-09-10T08:52:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb50a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb50c100a40 0x7fb50c197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb50a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb50c100a40 0x7fb50c197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41552/0 (socket says 172.21.15.172:41552) 2024-09-10T08:52:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb50a59c640 1 -- 172.21.15.172:0/1888719721 learned_addr learned my addr 172.21.15.172:0/1888719721 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb509d9b640 1 --2- 172.21.15.172:0/1888719721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb50c101c30 0x7fb50c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb50ad9d640 1 --2- 172.21.15.172:0/1888719721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb50c1025f0 0x7fb50c19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb50a59c640 1 -- 172.21.15.172:0/1888719721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb50c101c30 msgr2=0x7fb50c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb50a59c640 1 --2- 172.21.15.172:0/1888719721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb50c101c30 0x7fb50c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb50a59c640 1 -- 172.21.15.172:0/1888719721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb50c1025f0 msgr2=0x7fb50c19f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb50a59c640 1 --2- 172.21.15.172:0/1888719721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb50c1025f0 0x7fb50c19f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb50a59c640 1 -- 172.21.15.172:0/1888719721 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50c19f960 con 0x7fb50c100a40 2024-09-10T08:52:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.918+0000 7fb50ad9d640 1 --2- 172.21.15.172:0/1888719721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb50c1025f0 0x7fb50c19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.919+0000 7fb50a59c640 1 --2- 172.21.15.172:0/1888719721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb50c100a40 0x7fb50c197d50 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0314e0 tx=0x7fb4fc0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.919+0000 7fb4fb7fe640 1 -- 172.21.15.172:0/1888719721 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc03f030 con 0x7fb50c100a40 2024-09-10T08:52:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.919+0000 7fb51081e640 1 -- 172.21.15.172:0/1888719721 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50c19fbf0 con 0x7fb50c100a40 2024-09-10T08:52:39.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.919+0000 7fb4fb7fe640 1 -- 172.21.15.172:0/1888719721 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4fc003cb0 con 0x7fb50c100a40 2024-09-10T08:52:39.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.919+0000 7fb4fb7fe640 1 -- 172.21.15.172:0/1888719721 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc035070 con 0x7fb50c100a40 2024-09-10T08:52:39.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.919+0000 7fb51081e640 1 -- 172.21.15.172:0/1888719721 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb50c1a0130 con 0x7fb50c100a40 2024-09-10T08:52:39.930 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.922+0000 7fb4fb7fe640 1 -- 172.21.15.172:0/1888719721 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb4fc042030 con 0x7fb50c100a40 2024-09-10T08:52:39.930 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.922+0000 7fb51368c740 1 Processor -- start 2024-09-10T08:52:39.930 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.922+0000 7fb51368c740 1 -- start start 2024-09-10T08:52:39.930 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.922+0000 7fb51368c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f2299a0 0x555d2f229da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.930 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.922+0000 7fb51368c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22a2e0 0x555d2f22eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.930 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.922+0000 7fb51368c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f22f370 0x555d2f231830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.930 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.922+0000 7fb51368c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555d2f18d2f0 con 0x555d2f22a2e0 2024-09-10T08:52:39.930 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.922+0000 7fb51368c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555d2f135270 con 0x555d2f22f370 2024-09-10T08:52:39.930 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.922+0000 7fb51368c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555d2f1892b0 con 0x555d2f2299a0 2024-09-10T08:52:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.923+0000 7fb4fb7fe640 1 --2- 172.21.15.172:0/1888719721 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb4f40a2c30 0x7fb4f40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.923+0000 7fb50a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22a2e0 0x555d2f22eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.923+0000 7fb50a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22a2e0 0x555d2f22eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41554/0 (socket says 172.21.15.172:41554) 2024-09-10T08:52:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.923+0000 7fb50ad9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f2299a0 0x555d2f229da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.923+0000 7fb50a59c640 1 -- 172.21.15.172:0/1690073110 learned_addr learned my addr 172.21.15.172:0/1690073110 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.923+0000 7fb50ad9d640 1 --2- 172.21.15.172:0/1888719721 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb4f40a2c30 0x7fb4f40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.923+0000 7fb509d9b640 1 --2- 172.21.15.172:0/1690073110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f22f370 0x555d2f231830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.923+0000 7fb4fb7fe640 1 -- 172.21.15.172:0/1888719721 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb4fc0efa70 con 0x7fb50c100a40 2024-09-10T08:52:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.923+0000 7fb50a59c640 1 -- 172.21.15.172:0/1690073110 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f2299a0 msgr2=0x555d2f229da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.923+0000 7fb50a59c640 1 --2- 172.21.15.172:0/1690073110 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f2299a0 0x555d2f229da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.923+0000 7fb50a59c640 1 -- 172.21.15.172:0/1690073110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f22f370 msgr2=0x555d2f231830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.923+0000 7fb50a59c640 1 --2- 172.21.15.172:0/1690073110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f22f370 0x555d2f231830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.923+0000 7fb50a59c640 1 -- 172.21.15.172:0/1690073110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d2f134340 con 0x555d2f22a2e0 2024-09-10T08:52:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.923+0000 7fb50a59c640 1 --2- 172.21.15.172:0/1690073110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22a2e0 0x555d2f22eda0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0dabb0 tx=0x7fb4fc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d3ba5cc791de56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.923+0000 7fb50ad9d640 1 --2- 172.21.15.172:0/1888719721 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb4f40a2c30 0x7fb4f40a50f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb504009fd0 tx=0x7fb504009a70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.923+0000 7fb4f8ff9640 1 -- 172.21.15.172:0/1690073110 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc049070 con 0x555d2f22a2e0 2024-09-10T08:52:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.924+0000 7fb4f8ff9640 1 -- 172.21.15.172:0/1690073110 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4fc0bbf70 con 0x555d2f22a2e0 2024-09-10T08:52:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.924+0000 7fb4f8ff9640 1 -- 172.21.15.172:0/1690073110 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc03bc90 con 0x555d2f22a2e0 2024-09-10T08:52:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.924+0000 7fb51368c740 1 -- 172.21.15.172:0/1690073110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22a2e0 msgr2=0x555d2f22eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.924+0000 7fb51368c740 1 --2- 172.21.15.172:0/1690073110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22a2e0 0x555d2f22eda0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0dabb0 tx=0x7fb4fc02a410 comp rx=0 tx=0).stop 2024-09-10T08:52:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.924+0000 7fb51368c740 1 -- 172.21.15.172:0/1690073110 shutdown_connections 2024-09-10T08:52:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.924+0000 7fb51368c740 1 --2- 172.21.15.172:0/1690073110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f22f370 0x555d2f231830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.924+0000 7fb51368c740 1 --2- 172.21.15.172:0/1690073110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22a2e0 0x555d2f22eda0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.924+0000 7fb51368c740 1 --2- 172.21.15.172:0/1690073110 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f2299a0 0x555d2f229da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.924+0000 7fb51368c740 1 -- 172.21.15.172:0/1690073110 >> 172.21.15.172:0/1690073110 conn(0x555d2f225010 msgr2=0x555d2f225410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.925+0000 7fb51368c740 1 -- 172.21.15.172:0/1690073110 shutdown_connections 2024-09-10T08:52:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.925+0000 7fb51368c740 1 -- 172.21.15.172:0/1690073110 wait complete. 2024-09-10T08:52:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.925+0000 7fb51368c740 1 -- start start 2024-09-10T08:52:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.925+0000 7fb51368c740 1 Processor -- start 2024-09-10T08:52:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.925+0000 7fb51368c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f2299a0 0x555d2f2337e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.925+0000 7fb51368c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f22a2e0 0x555d2f233d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.925+0000 7fb51368c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22f370 0x555d2f23ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.925+0000 7fb51368c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555d2f18d2f0 con 0x555d2f22f370 2024-09-10T08:52:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.925+0000 7fb51368c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555d2f116290 con 0x555d2f2299a0 2024-09-10T08:52:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.925+0000 7fb51368c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555d2f232100 con 0x555d2f22a2e0 2024-09-10T08:52:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.925+0000 7fb50ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22f370 0x555d2f23ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.925+0000 7fb50ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22f370 0x555d2f23ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41562/0 (socket says 172.21.15.172:41562) 2024-09-10T08:52:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.925+0000 7fb50ad9d640 1 -- 172.21.15.172:0/1032149366 learned_addr learned my addr 172.21.15.172:0/1032149366 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.925+0000 7fb509d9b640 1 --2- 172.21.15.172:0/1032149366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f22a2e0 0x555d2f233d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.926+0000 7fb51368c740 10 client.0 _ll_register_callbacks cb 0x555d2f147140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.926+0000 7fb50a59c640 1 --2- 172.21.15.172:0/1032149366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f2299a0 0x555d2f2337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.926+0000 7fb509d9b640 1 -- 172.21.15.172:0/1032149366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f2299a0 msgr2=0x555d2f2337e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.926+0000 7fb509d9b640 1 --2- 172.21.15.172:0/1032149366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f2299a0 0x555d2f2337e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.926+0000 7fb509d9b640 1 -- 172.21.15.172:0/1032149366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22f370 msgr2=0x555d2f23ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.926+0000 7fb509d9b640 1 --2- 172.21.15.172:0/1032149366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22f370 0x555d2f23ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.926+0000 7fb509d9b640 1 -- 172.21.15.172:0/1032149366 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d2eea0210 con 0x555d2f22a2e0 2024-09-10T08:52:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.926+0000 7fb50ad9d640 1 --2- 172.21.15.172:0/1032149366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22f370 0x555d2f23ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.926+0000 7fb509d9b640 1 --2- 172.21.15.172:0/1032149366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f22a2e0 0x555d2f233d20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb500005fd0 tx=0x7fb50000bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f4077d805f8dcfb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:39.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.926+0000 7fb4f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:52:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.926+0000 7fb4f1ffb640 1 -- 172.21.15.172:0/1032149366 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500003e20 con 0x555d2f22a2e0 2024-09-10T08:52:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.926+0000 7fb51368c740 1 -- 172.21.15.172:0/1032149366 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4fc0bbf70 con 0x555d2f22a2e0 2024-09-10T08:52:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.926+0000 7fb4f1ffb640 1 -- 172.21.15.172:0/1032149366 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb500005020 con 0x555d2f22a2e0 2024-09-10T08:52:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.926+0000 7fb4f1ffb640 1 -- 172.21.15.172:0/1032149366 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb500024050 con 0x555d2f22a2e0 2024-09-10T08:52:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.926+0000 7fb51368c740 10 client.26356 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.926+0000 7fb51368c740 1 -- 172.21.15.172:0/1032149366 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555d2ef6c620 con 0x555d2f22a2e0 2024-09-10T08:52:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.927+0000 7fb4f1ffb640 1 -- 172.21.15.172:0/1032149366 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4fc0bbf70 con 0x555d2f22a2e0 2024-09-10T08:52:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.927+0000 7fb4f1ffb640 1 -- 172.21.15.172:0/1032149366 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fb50001b470 con 0x555d2f22a2e0 2024-09-10T08:52:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.927+0000 7fb51368c740 10 client.26356 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.927+0000 7fb51368c740 10 client.26356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.927+0000 7fb51368c740 20 client.26356 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.927+0000 7fb51368c740 1 --2- 172.21.15.172:0/1032149366 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555d2f23f560 0x555d2f2419c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.927+0000 7fb51368c740 4 client.26356 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.927+0000 7fb51368c740 1 -- 172.21.15.172:0/1032149366 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d2f19aa20 con 0x555d2f23f560 2024-09-10T08:52:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.927+0000 7fb50a59c640 1 --2- 172.21.15.172:0/1032149366 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555d2f23f560 0x555d2f2419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.928+0000 7fb4f1ffb640 10 client.26356 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.928+0000 7fb50a59c640 1 --2- 172.21.15.172:0/1032149366 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555d2f23f560 0x555d2f2419c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1f1a33416609357 server_cookie=6d21cc78d768be2a in_seq=0 out_seq=0 2024-09-10T08:52:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.928+0000 7fb4f1ffb640 1 -- 172.21.15.172:0/1032149366 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d2f19aa20 con 0x555d2f23f560 2024-09-10T08:52:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.928+0000 7fb4f1ffb640 10 client.26356 handle_command_reply: tid=0 2024-09-10T08:52:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.929+0000 7fb51368c740 1 client.26356 shutdown 2024-09-10T08:52:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.929+0000 7fb51368c740 1 -- 172.21.15.172:0/1032149366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f22a2e0 msgr2=0x555d2f233d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.929+0000 7fb51368c740 1 --2- 172.21.15.172:0/1032149366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f22a2e0 0x555d2f233d20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb500005fd0 tx=0x7fb50000bd80 comp rx=0 tx=0).stop 2024-09-10T08:52:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.929+0000 7fb51368c740 1 -- 172.21.15.172:0/1032149366 shutdown_connections 2024-09-10T08:52:39.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.929+0000 7fb51368c740 1 --2- 172.21.15.172:0/1032149366 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555d2f23f560 0x555d2f2419c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.929+0000 7fb51368c740 1 --2- 172.21.15.172:0/1032149366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22f370 0x555d2f23ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.929+0000 7fb4f1ffb640 0 client.26356 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.929+0000 7fb51368c740 1 --2- 172.21.15.172:0/1032149366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f22a2e0 0x555d2f233d20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.929+0000 7fb51368c740 1 --2- 172.21.15.172:0/1032149366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f2299a0 0x555d2f2337e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.929+0000 7fb51368c740 1 -- 172.21.15.172:0/1032149366 >> 172.21.15.172:0/1032149366 conn(0x555d2f225010 msgr2=0x555d2f226820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.929+0000 7fb51368c740 1 -- 172.21.15.172:0/1032149366 shutdown_connections 2024-09-10T08:52:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.929+0000 7fb51368c740 1 -- 172.21.15.172:0/1032149366 wait complete. 2024-09-10T08:52:39.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.929+0000 7fb51368c740 20 client.26356 trim_cache size 0 max 16384 2024-09-10T08:52:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb51368c740 1 Processor -- start 2024-09-10T08:52:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb51368c740 1 -- start start 2024-09-10T08:52:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb51368c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f2299a0 0x555d2f229d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb51368c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f22a2e0 0x555d2f23ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb51368c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f22f370 0x555d2f2240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb51368c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1aa580 con 0x555d2f2299a0 2024-09-10T08:52:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb51368c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1ac0c0 con 0x555d2f22f370 2024-09-10T08:52:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb51368c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1aac50 con 0x555d2f22a2e0 2024-09-10T08:52:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb50a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f2299a0 0x555d2f229d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb50a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f2299a0 0x555d2f229d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41566/0 (socket says 172.21.15.172:41566) 2024-09-10T08:52:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb50a59c640 1 -- 172.21.15.172:0/1538022083 learned_addr learned my addr 172.21.15.172:0/1538022083 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb509d9b640 1 --2- 172.21.15.172:0/1538022083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f22a2e0 0x555d2f23ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb50ad9d640 1 --2- 172.21.15.172:0/1538022083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f22f370 0x555d2f2240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb50a59c640 1 -- 172.21.15.172:0/1538022083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f22a2e0 msgr2=0x555d2f23ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb50a59c640 1 --2- 172.21.15.172:0/1538022083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f22a2e0 0x555d2f23ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb50a59c640 1 -- 172.21.15.172:0/1538022083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f22f370 msgr2=0x555d2f2240b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb50a59c640 1 --2- 172.21.15.172:0/1538022083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f22f370 0x555d2f2240b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb50a59c640 1 -- 172.21.15.172:0/1538022083 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d2f224e30 con 0x555d2f2299a0 2024-09-10T08:52:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.944+0000 7fb509d9b640 1 --2- 172.21.15.172:0/1538022083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f22a2e0 0x555d2f23ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.945+0000 7fb50a59c640 1 --2- 172.21.15.172:0/1538022083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f2299a0 0x555d2f229d80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0ed580 tx=0x7fb4fc0bbe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c194c791174b349 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.945+0000 7fb4f3fff640 1 -- 172.21.15.172:0/1538022083 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc0d6070 con 0x555d2f2299a0 2024-09-10T08:52:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.945+0000 7fb4f3fff640 1 -- 172.21.15.172:0/1538022083 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4fc0b8550 con 0x555d2f2299a0 2024-09-10T08:52:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.945+0000 7fb4f3fff640 1 -- 172.21.15.172:0/1538022083 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc0d4030 con 0x555d2f2299a0 2024-09-10T08:52:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.946+0000 7fb51368c740 1 -- 172.21.15.172:0/1538022083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f2299a0 msgr2=0x555d2f229d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.946+0000 7fb51368c740 1 --2- 172.21.15.172:0/1538022083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f2299a0 0x555d2f229d80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0ed580 tx=0x7fb4fc0bbe70 comp rx=0 tx=0).stop 2024-09-10T08:52:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.946+0000 7fb51368c740 1 -- 172.21.15.172:0/1538022083 shutdown_connections 2024-09-10T08:52:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.946+0000 7fb51368c740 1 --2- 172.21.15.172:0/1538022083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f22f370 0x555d2f2240b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.946+0000 7fb51368c740 1 --2- 172.21.15.172:0/1538022083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f22a2e0 0x555d2f23ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.946+0000 7fb51368c740 1 --2- 172.21.15.172:0/1538022083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f2299a0 0x555d2f229d80 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.946+0000 7fb51368c740 1 -- 172.21.15.172:0/1538022083 >> 172.21.15.172:0/1538022083 conn(0x555d2f225010 msgr2=0x555d2f2239a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.946+0000 7fb51368c740 1 -- 172.21.15.172:0/1538022083 shutdown_connections 2024-09-10T08:52:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.946+0000 7fb51368c740 1 -- 172.21.15.172:0/1538022083 wait complete. 2024-09-10T08:52:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.946+0000 7fb51368c740 1 -- start start 2024-09-10T08:52:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.946+0000 7fb51368c740 1 Processor -- start 2024-09-10T08:52:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb51368c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f2299a0 0x555d2f238050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb51368c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22a2e0 0x555d2f238590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb51368c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f22f370 0x555d2f2395a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb51368c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1ab9f0 con 0x555d2f22a2e0 2024-09-10T08:52:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb51368c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1ab320 con 0x555d2f22f370 2024-09-10T08:52:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb51368c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c1a8560 con 0x555d2f2299a0 2024-09-10T08:52:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb51368c740 10 client.0 _ll_register_callbacks cb 0x7fb5000042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb509d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22a2e0 0x555d2f238590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb509d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22a2e0 0x555d2f238590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41582/0 (socket says 172.21.15.172:41582) 2024-09-10T08:52:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb509d9b640 1 -- 172.21.15.172:0/2238143141 learned_addr learned my addr 172.21.15.172:0/2238143141 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb50a59c640 1 --2- 172.21.15.172:0/2238143141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f2299a0 0x555d2f238050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb50ad9d640 1 --2- 172.21.15.172:0/2238143141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f22f370 0x555d2f2395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb509d9b640 1 -- 172.21.15.172:0/2238143141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f2299a0 msgr2=0x555d2f238050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb509d9b640 1 --2- 172.21.15.172:0/2238143141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f2299a0 0x555d2f238050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb509d9b640 1 -- 172.21.15.172:0/2238143141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f22f370 msgr2=0x555d2f2395a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb509d9b640 1 --2- 172.21.15.172:0/2238143141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f22f370 0x555d2f2395a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb509d9b640 1 -- 172.21.15.172:0/2238143141 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d2f239bb0 con 0x555d2f22a2e0 2024-09-10T08:52:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb50a59c640 1 --2- 172.21.15.172:0/2238143141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f2299a0 0x555d2f238050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb50ad9d640 1 --2- 172.21.15.172:0/2238143141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f22f370 0x555d2f2395a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb509d9b640 1 --2- 172.21.15.172:0/2238143141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22a2e0 0x555d2f238590 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x555d2f234cc0 tx=0x7fb500003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2759d4a057fbdc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb4f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb4f1ffb640 1 -- 172.21.15.172:0/2238143141 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50002c070 con 0x555d2f22a2e0 2024-09-10T08:52:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb4f1ffb640 1 -- 172.21.15.172:0/2238143141 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb50001bab0 con 0x555d2f22a2e0 2024-09-10T08:52:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.947+0000 7fb51368c740 1 -- 172.21.15.172:0/2238143141 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4fc0b8550 con 0x555d2f22a2e0 2024-09-10T08:52:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.948+0000 7fb4f1ffb640 1 -- 172.21.15.172:0/2238143141 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb500038050 con 0x555d2f22a2e0 2024-09-10T08:52:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.948+0000 7fb51368c740 10 client.17274 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.948+0000 7fb51368c740 1 -- 172.21.15.172:0/2238143141 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555d2f23af20 con 0x555d2f22a2e0 2024-09-10T08:52:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.948+0000 7fb4f1ffb640 1 -- 172.21.15.172:0/2238143141 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4fc0b8550 con 0x555d2f22a2e0 2024-09-10T08:52:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.948+0000 7fb4f1ffb640 1 -- 172.21.15.172:0/2238143141 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fb500011670 con 0x555d2f22a2e0 2024-09-10T08:52:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.948+0000 7fb51368c740 10 client.17274 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.948+0000 7fb51368c740 10 client.17274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.948+0000 7fb51368c740 20 client.17274 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.948+0000 7fb51368c740 1 --2- 172.21.15.172:0/2238143141 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555d2f23f560 0x555d2f2ac990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.948+0000 7fb51368c740 4 client.17274 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:52:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.948+0000 7fb51368c740 1 -- 172.21.15.172:0/2238143141 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d2f194d70 con 0x555d2f23f560 2024-09-10T08:52:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.948+0000 7fb50a59c640 1 --2- 172.21.15.172:0/2238143141 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555d2f23f560 0x555d2f2ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.949+0000 7fb50a59c640 1 --2- 172.21.15.172:0/2238143141 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555d2f23f560 0x555d2f2ac990 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1423ca4922dc2a69 server_cookie=85c7c1b963e1ff3e in_seq=0 out_seq=0 2024-09-10T08:52:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.949+0000 7fb4f1ffb640 10 client.17274 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.955+0000 7fb4f1ffb640 1 -- 172.21.15.172:0/2238143141 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555d2f194d70 con 0x555d2f23f560 2024-09-10T08:52:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.955+0000 7fb4f1ffb640 10 client.17274 handle_command_reply: tid=0 2024-09-10T08:52:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.955+0000 7fb51368c740 1 client.17274 shutdown 2024-09-10T08:52:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.955+0000 7fb51368c740 1 -- 172.21.15.172:0/2238143141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22a2e0 msgr2=0x555d2f238590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.955+0000 7fb51368c740 1 --2- 172.21.15.172:0/2238143141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22a2e0 0x555d2f238590 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x555d2f234cc0 tx=0x7fb500003ea0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.955+0000 7fb51368c740 1 -- 172.21.15.172:0/2238143141 shutdown_connections 2024-09-10T08:52:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.955+0000 7fb51368c740 1 --2- 172.21.15.172:0/2238143141 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555d2f23f560 0x555d2f2ac990 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.955+0000 7fb51368c740 1 --2- 172.21.15.172:0/2238143141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d2f22f370 0x555d2f2395a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.955+0000 7fb51368c740 1 --2- 172.21.15.172:0/2238143141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d2f22a2e0 0x555d2f238590 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.955+0000 7fb4f1ffb640 0 client.17274 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.955+0000 7fb51368c740 1 --2- 172.21.15.172:0/2238143141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d2f2299a0 0x555d2f238050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.955+0000 7fb51368c740 1 -- 172.21.15.172:0/2238143141 >> 172.21.15.172:0/2238143141 conn(0x555d2f225010 msgr2=0x555d2f2241c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.955+0000 7fb51368c740 1 -- 172.21.15.172:0/2238143141 shutdown_connections 2024-09-10T08:52:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.955+0000 7fb51368c740 1 -- 172.21.15.172:0/2238143141 wait complete. 2024-09-10T08:52:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.955+0000 7fb51368c740 20 client.17274 trim_cache size 0 max 16384 2024-09-10T08:52:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:52:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:52:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:52:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:52:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.956+0000 7fb51081e640 1 -- 172.21.15.172:0/1888719721 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb4f40a2c30 msgr2=0x7fb4f40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.956+0000 7fb51081e640 1 --2- 172.21.15.172:0/1888719721 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb4f40a2c30 0x7fb4f40a50f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb504009fd0 tx=0x7fb504009a70 comp rx=0 tx=0).stop 2024-09-10T08:52:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.956+0000 7fb51081e640 1 -- 172.21.15.172:0/1888719721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb50c100a40 msgr2=0x7fb50c197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.956+0000 7fb51081e640 1 --2- 172.21.15.172:0/1888719721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb50c100a40 0x7fb50c197d50 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc0314e0 tx=0x7fb4fc0042e0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.956+0000 7fb51081e640 1 -- 172.21.15.172:0/1888719721 shutdown_connections 2024-09-10T08:52:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.956+0000 7fb51081e640 1 --2- 172.21.15.172:0/1888719721 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb4f40a2c30 0x7fb4f40a50f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.956+0000 7fb51081e640 1 --2- 172.21.15.172:0/1888719721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb50c1025f0 0x7fb50c19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.956+0000 7fb51081e640 1 --2- 172.21.15.172:0/1888719721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb50c101c30 0x7fb50c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.956+0000 7fb51081e640 1 --2- 172.21.15.172:0/1888719721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb50c100a40 0x7fb50c197d50 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.956+0000 7fb51081e640 1 -- 172.21.15.172:0/1888719721 >> 172.21.15.172:0/1888719721 conn(0x7fb50c0fc1e0 msgr2=0x7fb50c069950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.956+0000 7fb51081e640 1 -- 172.21.15.172:0/1888719721 shutdown_connections 2024-09-10T08:52:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:39.956+0000 7fb51081e640 1 -- 172.21.15.172:0/1888719721 wait complete. 2024-09-10T08:52:40.147 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T08:52:40.147 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-10T08:52:40.147 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:52:40.476 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:52:40.476 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:52:40.514 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:52:40.640 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:40 smithi172 ceph-mon[30854]: pgmap v543: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:52:40.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:40 smithi174 ceph-mon[34068]: pgmap v543: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:52:40.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:40 smithi191 ceph-mon[35314]: pgmap v543: 129 pgs: 129 active+clean; 5.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:52:41.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.086+0000 7ff999053640 1 -- 172.21.15.172:0/4086552246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff994102ef0 msgr2=0x7ff99410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:41.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.086+0000 7ff999053640 1 --2- 172.21.15.172:0/4086552246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff994102ef0 0x7ff99410da40 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009ca0 tx=0x7ff98c02a950 comp rx=0 tx=0).stop 2024-09-10T08:52:41.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.086+0000 7ff999053640 1 -- 172.21.15.172:0/4086552246 shutdown_connections 2024-09-10T08:52:41.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.086+0000 7ff999053640 1 --2- 172.21.15.172:0/4086552246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff994102ef0 0x7ff99410da40 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:41.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.086+0000 7ff999053640 1 --2- 172.21.15.172:0/4086552246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff994102570 0x7ff9941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:41.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.086+0000 7ff999053640 1 --2- 172.21.15.172:0/4086552246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff994100e20 0x7ff994101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:41.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.086+0000 7ff999053640 1 -- 172.21.15.172:0/4086552246 >> 172.21.15.172:0/4086552246 conn(0x7ff9940fc7d0 msgr2=0x7ff9940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:41.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.086+0000 7ff999053640 1 -- 172.21.15.172:0/4086552246 shutdown_connections 2024-09-10T08:52:41.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.086+0000 7ff999053640 1 -- 172.21.15.172:0/4086552246 wait complete. 2024-09-10T08:52:41.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.087+0000 7ff999053640 1 Processor -- start 2024-09-10T08:52:41.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.087+0000 7ff999053640 1 -- start start 2024-09-10T08:52:41.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.087+0000 7ff999053640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff994100e20 0x7ff9941992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:41.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.087+0000 7ff999053640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff994102570 0x7ff9941997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:41.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.087+0000 7ff999053640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff994102ef0 0x7ff9941a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:41.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.088+0000 7ff999053640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff994110e80 con 0x7ff994102ef0 2024-09-10T08:52:41.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.088+0000 7ff999053640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff994110d00 con 0x7ff994102570 2024-09-10T08:52:41.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.088+0000 7ff999053640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff994111000 con 0x7ff994100e20 2024-09-10T08:52:41.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.088+0000 7ff99359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff994102ef0 0x7ff9941a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:41.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.088+0000 7ff99359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff994102ef0 0x7ff9941a4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41596/0 (socket says 172.21.15.172:41596) 2024-09-10T08:52:41.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.088+0000 7ff99259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff994102570 0x7ff9941997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:41.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.088+0000 7ff99359e640 1 -- 172.21.15.172:0/416629310 learned_addr learned my addr 172.21.15.172:0/416629310 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:41.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.088+0000 7ff992d9d640 1 --2- 172.21.15.172:0/416629310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff994100e20 0x7ff9941992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:41.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.088+0000 7ff99259c640 1 -- 172.21.15.172:0/416629310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff994100e20 msgr2=0x7ff9941992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:41.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.088+0000 7ff99259c640 1 --2- 172.21.15.172:0/416629310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff994100e20 0x7ff9941992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:41.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.088+0000 7ff99259c640 1 -- 172.21.15.172:0/416629310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff994102ef0 msgr2=0x7ff9941a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:41.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.088+0000 7ff99259c640 1 --2- 172.21.15.172:0/416629310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff994102ef0 0x7ff9941a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:41.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.088+0000 7ff99259c640 1 -- 172.21.15.172:0/416629310 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9941970a0 con 0x7ff994102570 2024-09-10T08:52:41.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.088+0000 7ff99359e640 1 --2- 172.21.15.172:0/416629310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff994102ef0 0x7ff9941a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:41.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.088+0000 7ff99259c640 1 --2- 172.21.15.172:0/416629310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff994102570 0x7ff9941997f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff988005fd0 tx=0x7ff98800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:41.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.088+0000 7ff97bfff640 1 -- 172.21.15.172:0/416629310 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988009c30 con 0x7ff994102570 2024-09-10T08:52:41.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.089+0000 7ff999053640 1 -- 172.21.15.172:0/416629310 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff994197390 con 0x7ff994102570 2024-09-10T08:52:41.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.089+0000 7ff97bfff640 1 -- 172.21.15.172:0/416629310 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff988009dd0 con 0x7ff994102570 2024-09-10T08:52:41.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.089+0000 7ff97bfff640 1 -- 172.21.15.172:0/416629310 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988003e40 con 0x7ff994102570 2024-09-10T08:52:41.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.089+0000 7ff999053640 1 -- 172.21.15.172:0/416629310 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff994197920 con 0x7ff994102570 2024-09-10T08:52:41.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.090+0000 7ff999053640 1 -- 172.21.15.172:0/416629310 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9941021b0 con 0x7ff994102570 2024-09-10T08:52:41.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.091+0000 7ff97bfff640 1 -- 172.21.15.172:0/416629310 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff988003fe0 con 0x7ff994102570 2024-09-10T08:52:41.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.092+0000 7ff97bfff640 1 --2- 172.21.15.172:0/416629310 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff97c0a2b80 0x7ff97c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:41.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.092+0000 7ff992d9d640 1 --2- 172.21.15.172:0/416629310 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff97c0a2b80 0x7ff97c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:41.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.092+0000 7ff97bfff640 1 -- 172.21.15.172:0/416629310 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff9880cef90 con 0x7ff994102570 2024-09-10T08:52:41.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.092+0000 7ff992d9d640 1 --2- 172.21.15.172:0/416629310 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff97c0a2b80 0x7ff97c0a5040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff984009bd0 tx=0x7ff984005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:41.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.095+0000 7ff97bfff640 1 -- 172.21.15.172:0/416629310 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff988017020 con 0x7ff994102570 2024-09-10T08:52:41.139 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:52:41.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.365+0000 7ff999053640 1 -- 172.21.15.172:0/416629310 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff99406a560 con 0x7ff994102570 2024-09-10T08:52:41.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.366+0000 7ff97bfff640 1 -- 172.21.15.172:0/416629310 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+12001 (secure 0 0 0) 0x7ff98809b4e0 con 0x7ff994102570 2024-09-10T08:52:41.367 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:52:41.367 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":37,"btime":"2024-09-10T08:52:08:313493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:52:07.658341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:52:41.367 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 37 2024-09-10T08:52:41.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.368+0000 7ff999053640 1 -- 172.21.15.172:0/416629310 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff97c0a2b80 msgr2=0x7ff97c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:41.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.368+0000 7ff999053640 1 --2- 172.21.15.172:0/416629310 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff97c0a2b80 0x7ff97c0a5040 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff984009bd0 tx=0x7ff984005f90 comp rx=0 tx=0).stop 2024-09-10T08:52:41.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.368+0000 7ff999053640 1 -- 172.21.15.172:0/416629310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff994102570 msgr2=0x7ff9941997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:41.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.368+0000 7ff999053640 1 --2- 172.21.15.172:0/416629310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff994102570 0x7ff9941997f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff988005fd0 tx=0x7ff98800dde0 comp rx=0 tx=0).stop 2024-09-10T08:52:41.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.369+0000 7ff999053640 1 -- 172.21.15.172:0/416629310 shutdown_connections 2024-09-10T08:52:41.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.369+0000 7ff999053640 1 --2- 172.21.15.172:0/416629310 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff97c0a2b80 0x7ff97c0a5040 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:41.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.369+0000 7ff999053640 1 --2- 172.21.15.172:0/416629310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff994102ef0 0x7ff9941a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:41.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.369+0000 7ff999053640 1 --2- 172.21.15.172:0/416629310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff994102570 0x7ff9941997f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:41.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.369+0000 7ff999053640 1 --2- 172.21.15.172:0/416629310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff994100e20 0x7ff9941992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:41.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.369+0000 7ff999053640 1 -- 172.21.15.172:0/416629310 >> 172.21.15.172:0/416629310 conn(0x7ff9940fc7d0 msgr2=0x7ff9940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:41.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.369+0000 7ff999053640 1 -- 172.21.15.172:0/416629310 shutdown_connections 2024-09-10T08:52:41.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.369+0000 7ff999053640 1 -- 172.21.15.172:0/416629310 wait complete. 2024-09-10T08:52:41.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.820+0000 7f8916e99640 1 -- 172.21.15.172:0/1385333464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8910102d30 msgr2=0x7f8910103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:41.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.820+0000 7f8916e99640 1 --2- 172.21.15.172:0/1385333464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8910102d30 0x7f8910103110 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f8904009930 tx=0x7f890402a420 comp rx=0 tx=0).stop 2024-09-10T08:52:41.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.820+0000 7f8916e99640 1 -- 172.21.15.172:0/1385333464 shutdown_connections 2024-09-10T08:52:41.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.820+0000 7f8916e99640 1 --2- 172.21.15.172:0/1385333464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89101082a0 0x7f891010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:41.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.820+0000 7f8916e99640 1 --2- 172.21.15.172:0/1385333464 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89101036e0 0x7f8910107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:41.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.820+0000 7f8916e99640 1 --2- 172.21.15.172:0/1385333464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8910102d30 0x7f8910103110 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:41.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.820+0000 7f8916e99640 1 -- 172.21.15.172:0/1385333464 >> 172.21.15.172:0/1385333464 conn(0x7f89100fc7d0 msgr2=0x7f89100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:41.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.820+0000 7f8916e99640 1 -- 172.21.15.172:0/1385333464 shutdown_connections 2024-09-10T08:52:41.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.820+0000 7f8916e99640 1 -- 172.21.15.172:0/1385333464 wait complete. 2024-09-10T08:52:41.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.820+0000 7f8916e99640 1 Processor -- start 2024-09-10T08:52:41.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.820+0000 7f8916e99640 1 -- start start 2024-09-10T08:52:41.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.821+0000 7f8916e99640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8910102d30 0x7f8910198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:41.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.821+0000 7f8916e99640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89101036e0 0x7f8910198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:41.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.821+0000 7f8916e99640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89101082a0 0x7f891019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:41.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.821+0000 7f8916e99640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8910111f50 con 0x7f8910102d30 2024-09-10T08:52:41.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.821+0000 7f8916e99640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8910111dd0 con 0x7f89101082a0 2024-09-10T08:52:41.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.821+0000 7f8916e99640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f89101120d0 con 0x7f89101036e0 2024-09-10T08:52:41.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.821+0000 7f8914c35640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8910102d30 0x7f8910198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:41.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.821+0000 7f8914c35640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8910102d30 0x7f8910198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41618/0 (socket says 172.21.15.172:41618) 2024-09-10T08:52:41.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.821+0000 7f8914c35640 1 -- 172.21.15.172:0/1937182344 learned_addr learned my addr 172.21.15.172:0/1937182344 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:41.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.821+0000 7f8915436640 1 --2- 172.21.15.172:0/1937182344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89101082a0 0x7f891019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:41.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.821+0000 7f890ffff640 1 --2- 172.21.15.172:0/1937182344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89101036e0 0x7f8910198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:41.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.821+0000 7f8914c35640 1 -- 172.21.15.172:0/1937182344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89101036e0 msgr2=0x7f8910198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:41.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.821+0000 7f8914c35640 1 --2- 172.21.15.172:0/1937182344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89101036e0 0x7f8910198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:41.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.821+0000 7f8914c35640 1 -- 172.21.15.172:0/1937182344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89101082a0 msgr2=0x7f891019cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:41.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.821+0000 7f8914c35640 1 --2- 172.21.15.172:0/1937182344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89101082a0 0x7f891019cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:41.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.821+0000 7f8914c35640 1 -- 172.21.15.172:0/1937182344 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891019d4c0 con 0x7f8910102d30 2024-09-10T08:52:41.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.821+0000 7f890ffff640 1 --2- 172.21.15.172:0/1937182344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89101036e0 0x7f8910198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:41.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.822+0000 7f8914c35640 1 --2- 172.21.15.172:0/1937182344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8910102d30 0x7f8910198510 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f8904005940 tx=0x7f89040040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:41.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.822+0000 7f890dffb640 1 -- 172.21.15.172:0/1937182344 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890403f030 con 0x7f8910102d30 2024-09-10T08:52:41.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.822+0000 7f890dffb640 1 -- 172.21.15.172:0/1937182344 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89040044c0 con 0x7f8910102d30 2024-09-10T08:52:41.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.822+0000 7f890dffb640 1 -- 172.21.15.172:0/1937182344 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904035070 con 0x7f8910102d30 2024-09-10T08:52:41.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.822+0000 7f8916e99640 1 -- 172.21.15.172:0/1937182344 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891019d750 con 0x7f8910102d30 2024-09-10T08:52:41.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.822+0000 7f8916e99640 1 -- 172.21.15.172:0/1937182344 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89101008c0 con 0x7f8910102d30 2024-09-10T08:52:41.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.823+0000 7f8916e99640 1 -- 172.21.15.172:0/1937182344 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f891006a850 con 0x7f8910102d30 2024-09-10T08:52:41.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.824+0000 7f890dffb640 1 -- 172.21.15.172:0/1937182344 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8904042030 con 0x7f8910102d30 2024-09-10T08:52:41.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.825+0000 7f890dffb640 1 --2- 172.21.15.172:0/1937182344 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f89000a2c00 0x7f89000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:41.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.825+0000 7f890ffff640 1 --2- 172.21.15.172:0/1937182344 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f89000a2c00 0x7f89000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:41.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.825+0000 7f890dffb640 1 -- 172.21.15.172:0/1937182344 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f89040ef940 con 0x7f8910102d30 2024-09-10T08:52:41.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.826+0000 7f890ffff640 1 --2- 172.21.15.172:0/1937182344 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f89000a2c00 0x7f89000a50c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f89100fee80 tx=0x7f88fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:41.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:41.829+0000 7f890dffb640 1 -- 172.21.15.172:0/1937182344 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f89040bc4e0 con 0x7f8910102d30 2024-09-10T08:52:42.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:42.076+0000 7f8916e99640 1 -- 172.21.15.172:0/1937182344 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8910103110 con 0x7f8910102d30 2024-09-10T08:52:42.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:42.077+0000 7f890dffb640 1 -- 172.21.15.172:0/1937182344 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+12001 (secure 0 0 0) 0x7f89040bbde0 con 0x7f8910102d30 2024-09-10T08:52:42.078 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:52:42.078 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":37,"btime":"2024-09-10T08:52:08:313493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:52:07.658341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:52:42.079 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 37 2024-09-10T08:52:42.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:42.080+0000 7f8916e99640 1 -- 172.21.15.172:0/1937182344 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f89000a2c00 msgr2=0x7f89000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:42.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:42.080+0000 7f8916e99640 1 --2- 172.21.15.172:0/1937182344 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f89000a2c00 0x7f89000a50c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f89100fee80 tx=0x7f88fc005f90 comp rx=0 tx=0).stop 2024-09-10T08:52:42.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:42.080+0000 7f8916e99640 1 -- 172.21.15.172:0/1937182344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8910102d30 msgr2=0x7f8910198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:42.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:42.080+0000 7f8916e99640 1 --2- 172.21.15.172:0/1937182344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8910102d30 0x7f8910198510 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f8904005940 tx=0x7f89040040c0 comp rx=0 tx=0).stop 2024-09-10T08:52:42.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:42.080+0000 7f8916e99640 1 -- 172.21.15.172:0/1937182344 shutdown_connections 2024-09-10T08:52:42.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:42.080+0000 7f8916e99640 1 --2- 172.21.15.172:0/1937182344 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f89000a2c00 0x7f89000a50c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:42.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:42.080+0000 7f8916e99640 1 --2- 172.21.15.172:0/1937182344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89101082a0 0x7f891019cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:42.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:42.080+0000 7f8916e99640 1 --2- 172.21.15.172:0/1937182344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89101036e0 0x7f8910198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:42.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:42.080+0000 7f8916e99640 1 --2- 172.21.15.172:0/1937182344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8910102d30 0x7f8910198510 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:42.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:42.080+0000 7f8916e99640 1 -- 172.21.15.172:0/1937182344 >> 172.21.15.172:0/1937182344 conn(0x7f89100fc7d0 msgr2=0x7f8910105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:42.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:42.080+0000 7f8916e99640 1 -- 172.21.15.172:0/1937182344 shutdown_connections 2024-09-10T08:52:42.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:42.080+0000 7f8916e99640 1 -- 172.21.15.172:0/1937182344 wait complete. 2024-09-10T08:52:42.323 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-10T08:52:42.323 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-10T08:52:42.647 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:42 smithi172 ceph-mon[30854]: pgmap v544: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-09-10T08:52:42.647 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:42 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/416629310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:52:42.647 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:42 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1937182344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:52:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:42 smithi174 ceph-mon[34068]: pgmap v544: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-09-10T08:52:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:42 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/416629310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:52:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:42 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1937182344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:52:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:42 smithi191 ceph-mon[35314]: pgmap v544: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 26 op/s 2024-09-10T08:52:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:42 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/416629310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:52:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:42 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1937182344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:52:42.969 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T08:52:43.096 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:52:43.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.635+0000 7fe592ca2640 1 -- 172.21.15.172:0/2507074145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe58c100e20 msgr2=0x7fe58c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:43.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.635+0000 7fe592ca2640 1 --2- 172.21.15.172:0/2507074145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe58c100e20 0x7fe58c101200 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe580009900 tx=0x7fe58002a480 comp rx=0 tx=0).stop 2024-09-10T08:52:43.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.635+0000 7fe592ca2640 1 -- 172.21.15.172:0/2507074145 shutdown_connections 2024-09-10T08:52:43.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.635+0000 7fe592ca2640 1 --2- 172.21.15.172:0/2507074145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe58c102ef0 0x7fe58c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.635+0000 7fe592ca2640 1 --2- 172.21.15.172:0/2507074145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe58c102570 0x7fe58c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.635+0000 7fe592ca2640 1 --2- 172.21.15.172:0/2507074145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe58c100e20 0x7fe58c101200 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.635+0000 7fe592ca2640 1 -- 172.21.15.172:0/2507074145 >> 172.21.15.172:0/2507074145 conn(0x7fe58c0fc7d0 msgr2=0x7fe58c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:43.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.635+0000 7fe592ca2640 1 -- 172.21.15.172:0/2507074145 shutdown_connections 2024-09-10T08:52:43.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.635+0000 7fe592ca2640 1 -- 172.21.15.172:0/2507074145 wait complete. 2024-09-10T08:52:43.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.635+0000 7fe592ca2640 1 Processor -- start 2024-09-10T08:52:43.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.635+0000 7fe592ca2640 1 -- start start 2024-09-10T08:52:43.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe592ca2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe58c100e20 0x7fe58c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe592ca2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe58c102570 0x7fe58c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe592ca2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe58c102ef0 0x7fe58c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe592ca2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c110e30 con 0x7fe58c102570 2024-09-10T08:52:43.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe592ca2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c110cb0 con 0x7fe58c102ef0 2024-09-10T08:52:43.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe592ca2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c110fb0 con 0x7fe58c100e20 2024-09-10T08:52:43.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe58bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe58c102570 0x7fe58c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:43.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe590a3e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe58c100e20 0x7fe58c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:43.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe58bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe58c102570 0x7fe58c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41650/0 (socket says 172.21.15.172:41650) 2024-09-10T08:52:43.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe58bfff640 1 -- 172.21.15.172:0/48367211 learned_addr learned my addr 172.21.15.172:0/48367211 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:43.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe59123f640 1 --2- 172.21.15.172:0/48367211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe58c102ef0 0x7fe58c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:43.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe58bfff640 1 -- 172.21.15.172:0/48367211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe58c100e20 msgr2=0x7fe58c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:43.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe58bfff640 1 --2- 172.21.15.172:0/48367211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe58c100e20 0x7fe58c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe58bfff640 1 -- 172.21.15.172:0/48367211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe58c102ef0 msgr2=0x7fe58c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:43.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe58bfff640 1 --2- 172.21.15.172:0/48367211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe58c102ef0 0x7fe58c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe58bfff640 1 -- 172.21.15.172:0/48367211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c1a68e0 con 0x7fe58c102570 2024-09-10T08:52:43.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe590a3e640 1 --2- 172.21.15.172:0/48367211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe58c100e20 0x7fe58c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:43.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe59123f640 1 --2- 172.21.15.172:0/48367211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe58c102ef0 0x7fe58c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:43.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe58bfff640 1 --2- 172.21.15.172:0/48367211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe58c102570 0x7fe58c0fd1c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe57800d950 tx=0x7fe57800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:43.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe589ffb640 1 -- 172.21.15.172:0/48367211 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578017070 con 0x7fe58c102570 2024-09-10T08:52:43.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.636+0000 7fe592ca2640 1 -- 172.21.15.172:0/48367211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe58c1a6bd0 con 0x7fe58c102570 2024-09-10T08:52:43.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.637+0000 7fe589ffb640 1 -- 172.21.15.172:0/48367211 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe578009c30 con 0x7fe58c102570 2024-09-10T08:52:43.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.637+0000 7fe589ffb640 1 -- 172.21.15.172:0/48367211 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57800f620 con 0x7fe58c102570 2024-09-10T08:52:43.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.637+0000 7fe592ca2640 1 -- 172.21.15.172:0/48367211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe58c1a7160 con 0x7fe58c102570 2024-09-10T08:52:43.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.639+0000 7fe589ffb640 1 -- 172.21.15.172:0/48367211 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe5780109a0 con 0x7fe58c102570 2024-09-10T08:52:43.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.640+0000 7fe589ffb640 1 --2- 172.21.15.172:0/48367211 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe57c0a2c00 0x7fe57c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.640+0000 7fe59507b740 1 Processor -- start 2024-09-10T08:52:43.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.640+0000 7fe59507b740 1 -- start start 2024-09-10T08:52:43.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.640+0000 7fe590a3e640 1 --2- 172.21.15.172:0/48367211 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe57c0a2c00 0x7fe57c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:43.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.640+0000 7fe589ffb640 1 -- 172.21.15.172:0/48367211 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe5780cef60 con 0x7fe58c102570 2024-09-10T08:52:43.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.640+0000 7fe59507b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9d9760 0x555f7d9d9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.640+0000 7fe59507b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9da080 0x555f7d9e4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.640+0000 7fe59507b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9e4940 0x555f7d9e6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.640+0000 7fe59507b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555f7d93f2a0 con 0x555f7d9d9760 2024-09-10T08:52:43.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.640+0000 7fe59507b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555f7d8e6b10 con 0x555f7d9e4940 2024-09-10T08:52:43.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.640+0000 7fe59507b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555f7d93b260 con 0x555f7d9da080 2024-09-10T08:52:43.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.640+0000 7fe590a3e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9d9760 0x555f7d9d9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:43.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.640+0000 7fe590a3e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9d9760 0x555f7d9d9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41660/0 (socket says 172.21.15.172:41660) 2024-09-10T08:52:43.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.640+0000 7fe590a3e640 1 -- 172.21.15.172:0/1050643837 learned_addr learned my addr 172.21.15.172:0/1050643837 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:43.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.641+0000 7fe58bfff640 1 --2- 172.21.15.172:0/1050643837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9da080 0x555f7d9e4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:43.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.641+0000 7fe59123f640 1 --2- 172.21.15.172:0/1050643837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9e4940 0x555f7d9e6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:43.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.641+0000 7fe590a3e640 1 --2- 172.21.15.172:0/48367211 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe57c0a2c00 0x7fe57c0a50c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe580009230 tx=0x7fe580032ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:43.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.641+0000 7fe58bfff640 1 -- 172.21.15.172:0/1050643837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9e4940 msgr2=0x555f7d9e6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:43.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.641+0000 7fe58bfff640 1 --2- 172.21.15.172:0/1050643837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9e4940 0x555f7d9e6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.641+0000 7fe58bfff640 1 -- 172.21.15.172:0/1050643837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9d9760 msgr2=0x555f7d9d9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:43.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.641+0000 7fe58bfff640 1 --2- 172.21.15.172:0/1050643837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9d9760 0x555f7d9d9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.641+0000 7fe58bfff640 1 -- 172.21.15.172:0/1050643837 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f7d6b0b90 con 0x555f7d9da080 2024-09-10T08:52:43.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.641+0000 7fe58bfff640 1 --2- 172.21.15.172:0/1050643837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9da080 0x555f7d9e4400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe57809b160 tx=0x7fe57809aef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c01c5e000c73e51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:43.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.641+0000 7fe576ffd640 1 -- 172.21.15.172:0/1050643837 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578020040 con 0x555f7d9da080 2024-09-10T08:52:43.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.641+0000 7fe576ffd640 1 -- 172.21.15.172:0/1050643837 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe57809f810 con 0x555f7d9da080 2024-09-10T08:52:43.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.641+0000 7fe576ffd640 1 -- 172.21.15.172:0/1050643837 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5780cce20 con 0x555f7d9da080 2024-09-10T08:52:43.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.642+0000 7fe59507b740 1 -- 172.21.15.172:0/1050643837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9da080 msgr2=0x555f7d9e4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:43.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.642+0000 7fe59507b740 1 --2- 172.21.15.172:0/1050643837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9da080 0x555f7d9e4400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe57809b160 tx=0x7fe57809aef0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.642+0000 7fe59507b740 1 -- 172.21.15.172:0/1050643837 shutdown_connections 2024-09-10T08:52:43.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.642+0000 7fe59507b740 1 --2- 172.21.15.172:0/1050643837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9e4940 0x555f7d9e6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.642+0000 7fe59507b740 1 --2- 172.21.15.172:0/1050643837 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9da080 0x555f7d9e4400 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.642+0000 7fe59507b740 1 --2- 172.21.15.172:0/1050643837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9d9760 0x555f7d9d9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.642+0000 7fe59507b740 1 -- 172.21.15.172:0/1050643837 >> 172.21.15.172:0/1050643837 conn(0x555f7d9d5cf0 msgr2=0x555f7d9d60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:43.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.642+0000 7fe59507b740 1 -- 172.21.15.172:0/1050643837 shutdown_connections 2024-09-10T08:52:43.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.642+0000 7fe59507b740 1 -- 172.21.15.172:0/1050643837 wait complete. 2024-09-10T08:52:43.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.642+0000 7fe59507b740 1 -- start start 2024-09-10T08:52:43.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.643+0000 7fe59507b740 1 Processor -- start 2024-09-10T08:52:43.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.643+0000 7fe59507b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9d9760 0x555f7d9e1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.643+0000 7fe59507b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9da080 0x555f7d9e1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.643+0000 7fe59507b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9e4940 0x555f7d9e2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.643+0000 7fe59507b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555f7d9e7510 con 0x555f7d9e4940 2024-09-10T08:52:43.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.643+0000 7fe59507b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555f7d8c76e0 con 0x555f7d9da080 2024-09-10T08:52:43.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.643+0000 7fe59507b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555f7d9e7690 con 0x555f7d9d9760 2024-09-10T08:52:43.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.643+0000 7fe59507b740 10 client.0 _ll_register_callbacks cb 0x555f7d8eba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:43.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.643+0000 7fe59123f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9e4940 0x555f7d9e2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:43.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.643+0000 7fe59123f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9e4940 0x555f7d9e2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41674/0 (socket says 172.21.15.172:41674) 2024-09-10T08:52:43.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.643+0000 7fe59123f640 1 -- 172.21.15.172:0/3033444684 learned_addr learned my addr 172.21.15.172:0/3033444684 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:43.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.643+0000 7fe590a3e640 1 --2- 172.21.15.172:0/3033444684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9d9760 0x555f7d9e1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:43.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.644+0000 7fe58bfff640 1 --2- 172.21.15.172:0/3033444684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9da080 0x555f7d9e1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:43.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.644+0000 7fe59123f640 1 -- 172.21.15.172:0/3033444684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9d9760 msgr2=0x555f7d9e1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:43.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.644+0000 7fe59123f640 1 --2- 172.21.15.172:0/3033444684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9d9760 0x555f7d9e1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.644+0000 7fe59123f640 1 -- 172.21.15.172:0/3033444684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9da080 msgr2=0x555f7d9e1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:43.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.644+0000 7fe59123f640 1 --2- 172.21.15.172:0/3033444684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9da080 0x555f7d9e1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.644+0000 7fe59123f640 1 -- 172.21.15.172:0/3033444684 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f7d9f0a00 con 0x555f7d9e4940 2024-09-10T08:52:43.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.644+0000 7fe59123f640 1 --2- 172.21.15.172:0/3033444684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9e4940 0x555f7d9e2290 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fe584013990 tx=0x7fe584013e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4d0d70da291433 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:43.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.644+0000 7fe56bfff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:43.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.645+0000 7fe56bfff640 1 -- 172.21.15.172:0/3033444684 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584011e70 con 0x555f7d9e4940 2024-09-10T08:52:43.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.645+0000 7fe56bfff640 1 -- 172.21.15.172:0/3033444684 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe584004b20 con 0x555f7d9e4940 2024-09-10T08:52:43.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.645+0000 7fe59507b740 1 -- 172.21.15.172:0/3033444684 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe57809f810 con 0x555f7d9e4940 2024-09-10T08:52:43.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.645+0000 7fe56bfff640 1 -- 172.21.15.172:0/3033444684 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe584004d20 con 0x555f7d9e4940 2024-09-10T08:52:43.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.645+0000 7fe59507b740 10 client.17298 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:43.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.645+0000 7fe59507b740 1 -- 172.21.15.172:0/3033444684 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555f7d9f3420 con 0x555f7d9e4940 2024-09-10T08:52:43.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.645+0000 7fe56bfff640 1 -- 172.21.15.172:0/3033444684 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe57809f810 con 0x555f7d9e4940 2024-09-10T08:52:43.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.646+0000 7fe56bfff640 1 -- 172.21.15.172:0/3033444684 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fe58401a030 con 0x555f7d9e4940 2024-09-10T08:52:43.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.646+0000 7fe59507b740 10 client.17298 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:43.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.646+0000 7fe59507b740 10 client.17298 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:52:43.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.646+0000 7fe59507b740 20 client.17298 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:43.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.646+0000 7fe59507b740 1 --2- 172.21.15.172:0/3033444684 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555f7d9f48e0 0x555f7d9f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.646+0000 7fe59507b740 4 client.17298 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:43.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.646+0000 7fe59507b740 1 -- 172.21.15.172:0/3033444684 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f7d94c9d0 con 0x555f7d9f48e0 2024-09-10T08:52:43.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.646+0000 7fe590a3e640 1 --2- 172.21.15.172:0/3033444684 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555f7d9f48e0 0x555f7d9f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:43.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.646+0000 7fe56bfff640 10 client.17298 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:43.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.646+0000 7fe590a3e640 1 --2- 172.21.15.172:0/3033444684 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555f7d9f48e0 0x555f7d9f6d40 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=649289280e8cccf8 server_cookie=2f532853bbad167a in_seq=0 out_seq=0 2024-09-10T08:52:43.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.647+0000 7fe56bfff640 1 -- 172.21.15.172:0/3033444684 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f7d94c9d0 con 0x555f7d9f48e0 2024-09-10T08:52:43.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.647+0000 7fe56bfff640 10 client.17298 handle_command_reply: tid=0 2024-09-10T08:52:43.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.647+0000 7fe59507b740 1 client.17298 shutdown 2024-09-10T08:52:43.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.647+0000 7fe59507b740 1 -- 172.21.15.172:0/3033444684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9e4940 msgr2=0x555f7d9e2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:43.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.647+0000 7fe59507b740 1 --2- 172.21.15.172:0/3033444684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9e4940 0x555f7d9e2290 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fe584013990 tx=0x7fe584013e50 comp rx=0 tx=0).stop 2024-09-10T08:52:43.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.647+0000 7fe59507b740 1 -- 172.21.15.172:0/3033444684 shutdown_connections 2024-09-10T08:52:43.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.647+0000 7fe59507b740 1 --2- 172.21.15.172:0/3033444684 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555f7d9f48e0 0x555f7d9f6d40 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.648+0000 7fe59507b740 1 --2- 172.21.15.172:0/3033444684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9e4940 0x555f7d9e2290 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.648+0000 7fe59507b740 1 --2- 172.21.15.172:0/3033444684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9da080 0x555f7d9e1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.648+0000 7fe56bfff640 0 client.17298 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:43.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.648+0000 7fe59507b740 1 --2- 172.21.15.172:0/3033444684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9d9760 0x555f7d9e1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.648+0000 7fe59507b740 1 -- 172.21.15.172:0/3033444684 >> 172.21.15.172:0/3033444684 conn(0x555f7d9d5cf0 msgr2=0x555f7d9d4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:43.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.648+0000 7fe59507b740 1 -- 172.21.15.172:0/3033444684 shutdown_connections 2024-09-10T08:52:43.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.648+0000 7fe59507b740 1 -- 172.21.15.172:0/3033444684 wait complete. 2024-09-10T08:52:43.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.648+0000 7fe59507b740 20 client.17298 trim_cache size 0 max 16384 2024-09-10T08:52:43.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.674+0000 7fe59507b740 1 Processor -- start 2024-09-10T08:52:43.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.675+0000 7fe59507b740 1 -- start start 2024-09-10T08:52:43.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.675+0000 7fe59507b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9d9760 0x555f7d9dc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.675+0000 7fe59507b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9da080 0x555f7d9fc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.675+0000 7fe59507b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9e4940 0x555f7d9fcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.675+0000 7fe59507b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c1b0d90 con 0x555f7d9e4940 2024-09-10T08:52:43.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.675+0000 7fe59507b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c1b28d0 con 0x555f7d9d9760 2024-09-10T08:52:43.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.675+0000 7fe59507b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c1b1460 con 0x555f7d9da080 2024-09-10T08:52:43.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.675+0000 7fe59123f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9e4940 0x555f7d9fcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:43.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.675+0000 7fe590a3e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9d9760 0x555f7d9dc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:43.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.675+0000 7fe59123f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9e4940 0x555f7d9fcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41682/0 (socket says 172.21.15.172:41682) 2024-09-10T08:52:43.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.675+0000 7fe59123f640 1 -- 172.21.15.172:0/3179458902 learned_addr learned my addr 172.21.15.172:0/3179458902 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:43.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.675+0000 7fe58bfff640 1 --2- 172.21.15.172:0/3179458902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9da080 0x555f7d9fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:43.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.676+0000 7fe590a3e640 1 -- 172.21.15.172:0/3179458902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9da080 msgr2=0x555f7d9fc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:43.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.676+0000 7fe590a3e640 1 --2- 172.21.15.172:0/3179458902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9da080 0x555f7d9fc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.676+0000 7fe590a3e640 1 -- 172.21.15.172:0/3179458902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9e4940 msgr2=0x555f7d9fcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:43.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.676+0000 7fe590a3e640 1 --2- 172.21.15.172:0/3179458902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9e4940 0x555f7d9fcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.676+0000 7fe590a3e640 1 -- 172.21.15.172:0/3179458902 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f7d9d1260 con 0x555f7d9d9760 2024-09-10T08:52:43.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.676+0000 7fe590a3e640 1 --2- 172.21.15.172:0/3179458902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9d9760 0x555f7d9dc1c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe5800096c0 tx=0x7fe58003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e8180d4365d72e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:43.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.676+0000 7fe5767fc640 1 -- 172.21.15.172:0/3179458902 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580043070 con 0x555f7d9d9760 2024-09-10T08:52:43.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.676+0000 7fe5767fc640 1 -- 172.21.15.172:0/3179458902 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe580004c00 con 0x555f7d9d9760 2024-09-10T08:52:43.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.676+0000 7fe5767fc640 1 -- 172.21.15.172:0/3179458902 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580035040 con 0x555f7d9d9760 2024-09-10T08:52:43.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.677+0000 7fe59507b740 1 -- 172.21.15.172:0/3179458902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9d9760 msgr2=0x555f7d9dc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:43.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.677+0000 7fe59507b740 1 --2- 172.21.15.172:0/3179458902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9d9760 0x555f7d9dc1c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe5800096c0 tx=0x7fe58003f000 comp rx=0 tx=0).stop 2024-09-10T08:52:43.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.677+0000 7fe59507b740 1 -- 172.21.15.172:0/3179458902 shutdown_connections 2024-09-10T08:52:43.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.677+0000 7fe59507b740 1 --2- 172.21.15.172:0/3179458902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9e4940 0x555f7d9fcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.677+0000 7fe59507b740 1 --2- 172.21.15.172:0/3179458902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9da080 0x555f7d9fc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.677+0000 7fe59507b740 1 --2- 172.21.15.172:0/3179458902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9d9760 0x555f7d9dc1c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.677+0000 7fe59507b740 1 -- 172.21.15.172:0/3179458902 >> 172.21.15.172:0/3179458902 conn(0x555f7d9d5cf0 msgr2=0x555f7d9e3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:43.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.677+0000 7fe59507b740 1 -- 172.21.15.172:0/3179458902 shutdown_connections 2024-09-10T08:52:43.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.677+0000 7fe59507b740 1 -- 172.21.15.172:0/3179458902 wait complete. 2024-09-10T08:52:43.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.678+0000 7fe59507b740 1 -- start start 2024-09-10T08:52:43.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.678+0000 7fe59507b740 1 Processor -- start 2024-09-10T08:52:43.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.678+0000 7fe59507b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9d9760 0x555f7d9e8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.678+0000 7fe59507b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9da080 0x555f7d9e8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.679+0000 7fe590a3e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9d9760 0x555f7d9e8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:43.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.679+0000 7fe590a3e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9d9760 0x555f7d9e8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41686/0 (socket says 172.21.15.172:41686) 2024-09-10T08:52:43.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.679+0000 7fe590a3e640 1 -- 172.21.15.172:0/3211611945 learned_addr learned my addr 172.21.15.172:0/3211611945 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:43.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.679+0000 7fe59507b740 1 --2- 172.21.15.172:0/3211611945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9e4940 0x555f7da5e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.679+0000 7fe58bfff640 1 --2- 172.21.15.172:0/3211611945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9da080 0x555f7d9e8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:43.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.679+0000 7fe59507b740 1 -- 172.21.15.172:0/3211611945 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c1b2200 con 0x555f7d9d9760 2024-09-10T08:52:43.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.679+0000 7fe59507b740 1 -- 172.21.15.172:0/3211611945 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c1b1b30 con 0x555f7d9e4940 2024-09-10T08:52:43.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.679+0000 7fe59507b740 1 -- 172.21.15.172:0/3211611945 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c1aed70 con 0x555f7d9da080 2024-09-10T08:52:43.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.679+0000 7fe590a3e640 1 -- 172.21.15.172:0/3211611945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9da080 msgr2=0x555f7d9e8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:43.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.679+0000 7fe590a3e640 1 --2- 172.21.15.172:0/3211611945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9da080 0x555f7d9e8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.679+0000 7fe590a3e640 1 -- 172.21.15.172:0/3211611945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9e4940 msgr2=0x555f7da5e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:43.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.679+0000 7fe590a3e640 1 --2- 172.21.15.172:0/3211611945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9e4940 0x555f7da5e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.679+0000 7fe590a3e640 1 -- 172.21.15.172:0/3211611945 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f7da5eb70 con 0x555f7d9d9760 2024-09-10T08:52:43.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.679+0000 7fe59507b740 10 client.0 _ll_register_callbacks cb 0x7fe5840135a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:43.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.679+0000 7fe590a3e640 1 --2- 172.21.15.172:0/3211611945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9d9760 0x555f7d9e8a60 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe580000c00 tx=0x7fe580009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701c8fb305cbddfc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:43.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.679+0000 7fe59507b740 1 -- 172.21.15.172:0/3211611945 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe580004c00 con 0x555f7d9d9760 2024-09-10T08:52:43.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.679+0000 7fe56bfff640 10 client.17304 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:43.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.680+0000 7fe56bfff640 1 -- 172.21.15.172:0/3211611945 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580067050 con 0x555f7d9d9760 2024-09-10T08:52:43.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.680+0000 7fe56bfff640 1 -- 172.21.15.172:0/3211611945 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe580004750 con 0x555f7d9d9760 2024-09-10T08:52:43.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.680+0000 7fe56bfff640 1 -- 172.21.15.172:0/3211611945 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe58005d070 con 0x555f7d9d9760 2024-09-10T08:52:43.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.680+0000 7fe59507b740 10 client.17304 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.680+0000 7fe59507b740 1 -- 172.21.15.172:0/3211611945 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=37}) -- 0x555f7da5faa0 con 0x555f7d9d9760 2024-09-10T08:52:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.680+0000 7fe56bfff640 1 -- 172.21.15.172:0/3211611945 <== mon.0 v2:172.21.15.172:3300/0 4 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fe580003790 con 0x555f7d9d9760 2024-09-10T08:52:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.680+0000 7fe59507b740 10 client.17304 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.680+0000 7fe59507b740 10 client.17304 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:52:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.680+0000 7fe59507b740 20 client.17304 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.680+0000 7fe59507b740 1 --2- 172.21.15.172:0/3211611945 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555f7da60130 0x555f7da625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.680+0000 7fe59507b740 4 client.17304 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-10T08:52:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.680+0000 7fe59507b740 1 -- 172.21.15.172:0/3211611945 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x555f7d946d20 con 0x555f7da60130 2024-09-10T08:52:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.681+0000 7fe58bfff640 1 --2- 172.21.15.172:0/3211611945 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555f7da60130 0x555f7da625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.682+0000 7fe56bfff640 10 client.17304 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:43.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.682+0000 7fe58bfff640 1 --2- 172.21.15.172:0/3211611945 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555f7da60130 0x555f7da625d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75212f84d6ea8bd6 server_cookie=e4bfbca4bcf871d3 in_seq=0 out_seq=0 2024-09-10T08:52:43.687 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:52:43.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.753+0000 7fe56bfff640 1 -- 172.21.15.172:0/3211611945 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555f7d946d20 con 0x555f7da60130 2024-09-10T08:52:43.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.753+0000 7fe56bfff640 10 client.17304 handle_command_reply: tid=0 2024-09-10T08:52:43.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.754+0000 7fe59507b740 1 client.17304 shutdown 2024-09-10T08:52:43.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.754+0000 7fe59507b740 1 -- 172.21.15.172:0/3211611945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9d9760 msgr2=0x555f7d9e8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:43.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.754+0000 7fe59507b740 1 --2- 172.21.15.172:0/3211611945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9d9760 0x555f7d9e8a60 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe580000c00 tx=0x7fe580009260 comp rx=0 tx=0).stop 2024-09-10T08:52:43.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.754+0000 7fe59507b740 1 -- 172.21.15.172:0/3211611945 shutdown_connections 2024-09-10T08:52:43.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.754+0000 7fe59507b740 1 --2- 172.21.15.172:0/3211611945 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555f7da60130 0x555f7da625d0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.754+0000 7fe59507b740 1 --2- 172.21.15.172:0/3211611945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f7d9e4940 0x555f7da5e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.754+0000 7fe59507b740 1 --2- 172.21.15.172:0/3211611945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f7d9da080 0x555f7d9e8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.754+0000 7fe56bfff640 0 client.17304 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:43.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.754+0000 7fe59507b740 1 --2- 172.21.15.172:0/3211611945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f7d9d9760 0x555f7d9e8a60 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.754+0000 7fe59507b740 1 -- 172.21.15.172:0/3211611945 >> 172.21.15.172:0/3211611945 conn(0x555f7d9d5cf0 msgr2=0x555f7d9d4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:43.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.754+0000 7fe59507b740 1 -- 172.21.15.172:0/3211611945 shutdown_connections 2024-09-10T08:52:43.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.754+0000 7fe59507b740 1 -- 172.21.15.172:0/3211611945 wait complete. 2024-09-10T08:52:43.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.754+0000 7fe59507b740 20 client.17304 trim_cache size 0 max 16384 2024-09-10T08:52:43.757 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:52:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 37, 2024-09-10T08:52:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:52:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 18, 2024-09-10T08:52:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:52:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "bd6b2e81": { 2024-09-10T08:52:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 18, 2024-09-10T08:52:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:52:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:52:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:52:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:52:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:52:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:52:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 140.1, 2024-09-10T08:52:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:52:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:52:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:52:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:52:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:52:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:52:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:52:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:52:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:52:43.759 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:52:43.759 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:52:43.759 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:52:43.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.755+0000 7fe592ca2640 1 -- 172.21.15.172:0/48367211 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe57c0a2c00 msgr2=0x7fe57c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:43.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.755+0000 7fe592ca2640 1 --2- 172.21.15.172:0/48367211 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe57c0a2c00 0x7fe57c0a50c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe580009230 tx=0x7fe580032ec0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.755+0000 7fe592ca2640 1 -- 172.21.15.172:0/48367211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe58c102570 msgr2=0x7fe58c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:43.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.755+0000 7fe592ca2640 1 --2- 172.21.15.172:0/48367211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe58c102570 0x7fe58c0fd1c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe57800d950 tx=0x7fe57800de10 comp rx=0 tx=0).stop 2024-09-10T08:52:43.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.755+0000 7fe592ca2640 1 -- 172.21.15.172:0/48367211 shutdown_connections 2024-09-10T08:52:43.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.755+0000 7fe592ca2640 1 --2- 172.21.15.172:0/48367211 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe57c0a2c00 0x7fe57c0a50c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.755+0000 7fe592ca2640 1 --2- 172.21.15.172:0/48367211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe58c102ef0 0x7fe58c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.755+0000 7fe592ca2640 1 --2- 172.21.15.172:0/48367211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe58c102570 0x7fe58c0fd1c0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.755+0000 7fe592ca2640 1 --2- 172.21.15.172:0/48367211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe58c100e20 0x7fe58c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:43.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.755+0000 7fe592ca2640 1 -- 172.21.15.172:0/48367211 >> 172.21.15.172:0/48367211 conn(0x7fe58c0fc7d0 msgr2=0x7fe58c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:43.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.755+0000 7fe592ca2640 1 -- 172.21.15.172:0/48367211 shutdown_connections 2024-09-10T08:52:43.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:43.755+0000 7fe592ca2640 1 -- 172.21.15.172:0/48367211 wait complete. 2024-09-10T08:52:44.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.383+0000 7f6834d32640 1 -- 172.21.15.172:0/772166800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6830101df0 msgr2=0x7f68301021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:44.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.383+0000 7f6834d32640 1 --2- 172.21.15.172:0/772166800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6830101df0 0x7f68301021d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f6820009900 tx=0x7f682002a410 comp rx=0 tx=0).stop 2024-09-10T08:52:44.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.383+0000 7f6834d32640 1 -- 172.21.15.172:0/772166800 shutdown_connections 2024-09-10T08:52:44.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.383+0000 7f6834d32640 1 --2- 172.21.15.172:0/772166800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6830106f20 0x7f683010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.383+0000 7f6834d32640 1 --2- 172.21.15.172:0/772166800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68301027a0 0x7f68301067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.383+0000 7f6834d32640 1 --2- 172.21.15.172:0/772166800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6830101df0 0x7f68301021d0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.383+0000 7f6834d32640 1 -- 172.21.15.172:0/772166800 >> 172.21.15.172:0/772166800 conn(0x7f68300fd8b0 msgr2=0x7f68300ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:44.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.384+0000 7f6834d32640 1 -- 172.21.15.172:0/772166800 shutdown_connections 2024-09-10T08:52:44.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.384+0000 7f6834d32640 1 -- 172.21.15.172:0/772166800 wait complete. 2024-09-10T08:52:44.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.384+0000 7f6834d32640 1 Processor -- start 2024-09-10T08:52:44.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.384+0000 7f6834d32640 1 -- start start 2024-09-10T08:52:44.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.384+0000 7f6834d32640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6830101df0 0x7f68301983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.384+0000 7f6834d32640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68301027a0 0x7f6830198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.384+0000 7f6834d32640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6830106f20 0x7f683019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.384+0000 7f6834d32640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6830111e30 con 0x7f68301027a0 2024-09-10T08:52:44.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.384+0000 7f6834d32640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6830111cb0 con 0x7f6830106f20 2024-09-10T08:52:44.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.384+0000 7f6834d32640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6830111fb0 con 0x7f6830101df0 2024-09-10T08:52:44.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.385+0000 7f682dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68301027a0 0x7f6830198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.385+0000 7f682dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68301027a0 0x7f6830198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41704/0 (socket says 172.21.15.172:41704) 2024-09-10T08:52:44.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.385+0000 7f682dd9b640 1 -- 172.21.15.172:0/2033028059 learned_addr learned my addr 172.21.15.172:0/2033028059 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:44.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.385+0000 7f682e59c640 1 --2- 172.21.15.172:0/2033028059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6830101df0 0x7f68301983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.385+0000 7f682ed9d640 1 --2- 172.21.15.172:0/2033028059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6830106f20 0x7f683019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.385+0000 7f682dd9b640 1 -- 172.21.15.172:0/2033028059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6830101df0 msgr2=0x7f68301983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:44.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.385+0000 7f682dd9b640 1 --2- 172.21.15.172:0/2033028059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6830101df0 0x7f68301983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.385+0000 7f682dd9b640 1 -- 172.21.15.172:0/2033028059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6830106f20 msgr2=0x7f683019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:44.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.385+0000 7f682dd9b640 1 --2- 172.21.15.172:0/2033028059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6830106f20 0x7f683019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.385+0000 7f682dd9b640 1 -- 172.21.15.172:0/2033028059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683019d380 con 0x7f68301027a0 2024-09-10T08:52:44.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.385+0000 7f682dd9b640 1 --2- 172.21.15.172:0/2033028059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68301027a0 0x7f6830198910 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f682400d770 tx=0x7f682400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:44.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.385+0000 7f681f7fe640 1 -- 172.21.15.172:0/2033028059 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824017070 con 0x7f68301027a0 2024-09-10T08:52:44.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.385+0000 7f681f7fe640 1 -- 172.21.15.172:0/2033028059 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68240054d0 con 0x7f68301027a0 2024-09-10T08:52:44.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.385+0000 7f6834d32640 1 -- 172.21.15.172:0/2033028059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683019d670 con 0x7f68301027a0 2024-09-10T08:52:44.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.385+0000 7f681f7fe640 1 -- 172.21.15.172:0/2033028059 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824004cf0 con 0x7f68301027a0 2024-09-10T08:52:44.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.385+0000 7f6834d32640 1 -- 172.21.15.172:0/2033028059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68301a4f50 con 0x7f68301027a0 2024-09-10T08:52:44.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.387+0000 7f681f7fe640 1 -- 172.21.15.172:0/2033028059 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6824004e90 con 0x7f68301027a0 2024-09-10T08:52:44.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.388+0000 7f683710b740 1 Processor -- start 2024-09-10T08:52:44.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.388+0000 7f683710b740 1 -- start start 2024-09-10T08:52:44.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.388+0000 7f683710b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391db40 0x55d12391c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.388+0000 7f683710b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d12391c5d0 0x55d12391ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.388+0000 7f683710b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d123927370 0x55d123929760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.388+0000 7f683710b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d1238822f0 con 0x55d12391db40 2024-09-10T08:52:44.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.388+0000 7f683710b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d123829cf0 con 0x55d123927370 2024-09-10T08:52:44.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.388+0000 7f683710b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d12387e2b0 con 0x55d12391c5d0 2024-09-10T08:52:44.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.388+0000 7f681f7fe640 1 --2- 172.21.15.172:0/2033028059 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68180a2b80 0x7f68180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.388+0000 7f682ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391db40 0x55d12391c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.388+0000 7f682ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391db40 0x55d12391c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41706/0 (socket says 172.21.15.172:41706) 2024-09-10T08:52:44.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.388+0000 7f682ed9d640 1 -- 172.21.15.172:0/3623560254 learned_addr learned my addr 172.21.15.172:0/3623560254 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:44.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.389+0000 7f682e59c640 1 --2- 172.21.15.172:0/3623560254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d12391c5d0 0x55d12391ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.389+0000 7f681f7fe640 1 -- 172.21.15.172:0/2033028059 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f68240ceec0 con 0x7f68301027a0 2024-09-10T08:52:44.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.389+0000 7f682ed9d640 1 --2- 172.21.15.172:0/2033028059 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68180a2b80 0x7f68180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.389+0000 7f682dd9b640 1 --2- 172.21.15.172:0/3623560254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d123927370 0x55d123929760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.389+0000 7f682ed9d640 1 -- 172.21.15.172:0/3623560254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d12391c5d0 msgr2=0x55d12391ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:44.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.389+0000 7f682ed9d640 1 --2- 172.21.15.172:0/3623560254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d12391c5d0 0x55d12391ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.389+0000 7f682ed9d640 1 -- 172.21.15.172:0/3623560254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d123927370 msgr2=0x55d123929760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:44.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.389+0000 7f682ed9d640 1 --2- 172.21.15.172:0/3623560254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d123927370 0x55d123929760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.389+0000 7f682ed9d640 1 -- 172.21.15.172:0/3623560254 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d12380e3a0 con 0x55d12391db40 2024-09-10T08:52:44.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.389+0000 7f682ed9d640 1 --2- 172.21.15.172:0/3623560254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391db40 0x55d12391c090 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6828012810 tx=0x7f6828012cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada89e267bae29f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:44.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.389+0000 7f681cff9640 1 -- 172.21.15.172:0/3623560254 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828003d40 con 0x55d12391db40 2024-09-10T08:52:44.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.389+0000 7f681cff9640 1 -- 172.21.15.172:0/3623560254 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6828012030 con 0x55d12391db40 2024-09-10T08:52:44.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.389+0000 7f681cff9640 1 -- 172.21.15.172:0/3623560254 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68280043d0 con 0x55d12391db40 2024-09-10T08:52:44.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.390+0000 7f683710b740 1 -- 172.21.15.172:0/3623560254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391db40 msgr2=0x55d12391c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:44.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.390+0000 7f682ed9d640 1 --2- 172.21.15.172:0/2033028059 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68180a2b80 0x7f68180a5040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f682801f000 tx=0x7f682801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:44.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.390+0000 7f683710b740 1 --2- 172.21.15.172:0/3623560254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391db40 0x55d12391c090 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6828012810 tx=0x7f6828012cd0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.390+0000 7f683710b740 1 -- 172.21.15.172:0/3623560254 shutdown_connections 2024-09-10T08:52:44.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.390+0000 7f683710b740 1 --2- 172.21.15.172:0/3623560254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d123927370 0x55d123929760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.390+0000 7f683710b740 1 --2- 172.21.15.172:0/3623560254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d12391c5d0 0x55d12391ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.390+0000 7f683710b740 1 --2- 172.21.15.172:0/3623560254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391db40 0x55d12391c090 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.390+0000 7f683710b740 1 -- 172.21.15.172:0/3623560254 >> 172.21.15.172:0/3623560254 conn(0x55d1238e5310 msgr2=0x55d123918c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:44.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.390+0000 7f683710b740 1 -- 172.21.15.172:0/3623560254 shutdown_connections 2024-09-10T08:52:44.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.390+0000 7f683710b740 1 -- 172.21.15.172:0/3623560254 wait complete. 2024-09-10T08:52:44.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.390+0000 7f683710b740 1 -- start start 2024-09-10T08:52:44.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f683710b740 1 Processor -- start 2024-09-10T08:52:44.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f683710b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391c5d0 0x55d123926d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f683710b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d12391db40 0x55d123924010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f683710b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d123927370 0x55d123924600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f683710b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d12392a070 con 0x55d12391c5d0 2024-09-10T08:52:44.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f683710b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d12383d060 con 0x55d123927370 2024-09-10T08:52:44.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f683710b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d12392a1f0 con 0x55d12391db40 2024-09-10T08:52:44.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f683710b740 10 client.0 _ll_register_callbacks cb 0x55d1235908c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:44.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f682e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391c5d0 0x55d123926d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f682e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391c5d0 0x55d123926d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41720/0 (socket says 172.21.15.172:41720) 2024-09-10T08:52:44.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f682e59c640 1 -- 172.21.15.172:0/1895598002 learned_addr learned my addr 172.21.15.172:0/1895598002 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:44.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f682e59c640 1 -- 172.21.15.172:0/1895598002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d12391db40 msgr2=0x55d123924010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:44.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f682dd9b640 1 --2- 172.21.15.172:0/1895598002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d12391db40 0x55d123924010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f682ed9d640 1 --2- 172.21.15.172:0/1895598002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d123927370 0x55d123924600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f682e59c640 1 --2- 172.21.15.172:0/1895598002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d12391db40 0x55d123924010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f682e59c640 1 -- 172.21.15.172:0/1895598002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d123927370 msgr2=0x55d123924600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:44.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f682e59c640 1 --2- 172.21.15.172:0/1895598002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d123927370 0x55d123924600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f682e59c640 1 -- 172.21.15.172:0/1895598002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1235f33c0 con 0x55d12391c5d0 2024-09-10T08:52:44.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f682dd9b640 1 --2- 172.21.15.172:0/1895598002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d12391db40 0x55d123924010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:44.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f682e59c640 1 --2- 172.21.15.172:0/1895598002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391c5d0 0x55d123926d60 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f682002a930 tx=0x7f68200317f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9201957903f64ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:44.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.391+0000 7f6815ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:44.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.392+0000 7f6815ffb640 1 -- 172.21.15.172:0/1895598002 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820035040 con 0x55d12391c5d0 2024-09-10T08:52:44.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.392+0000 7f683710b740 1 -- 172.21.15.172:0/1895598002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6828012030 con 0x55d12391c5d0 2024-09-10T08:52:44.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.392+0000 7f6815ffb640 1 -- 172.21.15.172:0/1895598002 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6820004240 con 0x55d12391c5d0 2024-09-10T08:52:44.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.392+0000 7f6815ffb640 1 -- 172.21.15.172:0/1895598002 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f682004e050 con 0x55d12391c5d0 2024-09-10T08:52:44.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.392+0000 7f683710b740 10 client.17316 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:44.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.392+0000 7f683710b740 1 -- 172.21.15.172:0/1895598002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d123935b60 con 0x55d12391c5d0 2024-09-10T08:52:44.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.392+0000 7f6815ffb640 1 -- 172.21.15.172:0/1895598002 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6828012030 con 0x55d12391c5d0 2024-09-10T08:52:44.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.392+0000 7f6815ffb640 1 -- 172.21.15.172:0/1895598002 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f68200049b0 con 0x55d12391c5d0 2024-09-10T08:52:44.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.392+0000 7f683710b740 10 client.17316 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:44.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.392+0000 7f683710b740 10 client.17316 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:52:44.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.392+0000 7f683710b740 20 client.17316 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:44.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.392+0000 7f683710b740 1 --2- 172.21.15.172:0/1895598002 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d123936f70 0x55d123939360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.392+0000 7f683710b740 4 client.17316 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:44.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.392+0000 7f683710b740 1 -- 172.21.15.172:0/1895598002 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d12388f860 con 0x55d123936f70 2024-09-10T08:52:44.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.392+0000 7f682dd9b640 1 --2- 172.21.15.172:0/1895598002 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d123936f70 0x55d123939360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.393+0000 7f6815ffb640 10 client.17316 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:52:44.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.393+0000 7f682dd9b640 1 --2- 172.21.15.172:0/1895598002 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d123936f70 0x55d123939360 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c8ca51e49266f800 server_cookie=a9e9455ffbf0e93a in_seq=0 out_seq=0 2024-09-10T08:52:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.394+0000 7f6815ffb640 1 -- 172.21.15.172:0/1895598002 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d12388f860 con 0x55d123936f70 2024-09-10T08:52:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.394+0000 7f6815ffb640 10 client.17316 handle_command_reply: tid=0 2024-09-10T08:52:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.394+0000 7f683710b740 1 client.17316 shutdown 2024-09-10T08:52:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.394+0000 7f683710b740 1 -- 172.21.15.172:0/1895598002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391c5d0 msgr2=0x55d123926d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.394+0000 7f683710b740 1 --2- 172.21.15.172:0/1895598002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391c5d0 0x55d123926d60 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f682002a930 tx=0x7f68200317f0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.394+0000 7f683710b740 1 -- 172.21.15.172:0/1895598002 shutdown_connections 2024-09-10T08:52:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.394+0000 7f683710b740 1 --2- 172.21.15.172:0/1895598002 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d123936f70 0x55d123939360 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.394+0000 7f6815ffb640 0 client.17316 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:52:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.394+0000 7f683710b740 1 --2- 172.21.15.172:0/1895598002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d123927370 0x55d123924600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.394+0000 7f683710b740 1 --2- 172.21.15.172:0/1895598002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d12391db40 0x55d123924010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.394+0000 7f683710b740 1 --2- 172.21.15.172:0/1895598002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391c5d0 0x55d123926d60 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.394+0000 7f683710b740 1 -- 172.21.15.172:0/1895598002 >> 172.21.15.172:0/1895598002 conn(0x55d1238e5310 msgr2=0x55d12391e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:44.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.394+0000 7f683710b740 1 -- 172.21.15.172:0/1895598002 shutdown_connections 2024-09-10T08:52:44.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.394+0000 7f683710b740 1 -- 172.21.15.172:0/1895598002 wait complete. 2024-09-10T08:52:44.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.394+0000 7f683710b740 20 client.17316 trim_cache size 0 max 16384 2024-09-10T08:52:44.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.406+0000 7f683710b740 1 Processor -- start 2024-09-10T08:52:44.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.406+0000 7f683710b740 1 -- start start 2024-09-10T08:52:44.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.407+0000 7f683710b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d12391c5d0 0x55d123918d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.407+0000 7f683710b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d12391db40 0x55d12391f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.407+0000 7f683710b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d123927370 0x55d12391f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.407+0000 7f683710b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f68301aeb40 con 0x55d123927370 2024-09-10T08:52:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.407+0000 7f683710b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68301b0680 con 0x55d12391c5d0 2024-09-10T08:52:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.407+0000 7f683710b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f68301af210 con 0x55d12391db40 2024-09-10T08:52:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.407+0000 7f682e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d12391c5d0 0x55d123918d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.407+0000 7f682e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d12391c5d0 0x55d123918d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:59740/0 (socket says 172.21.15.172:59740) 2024-09-10T08:52:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.407+0000 7f682e59c640 1 -- 172.21.15.172:0/1164495767 learned_addr learned my addr 172.21.15.172:0/1164495767 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.407+0000 7f682ed9d640 1 --2- 172.21.15.172:0/1164495767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d123927370 0x55d12391f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.407+0000 7f682dd9b640 1 --2- 172.21.15.172:0/1164495767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d12391db40 0x55d12391f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.407+0000 7f682e59c640 1 -- 172.21.15.172:0/1164495767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d12391db40 msgr2=0x55d12391f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:44.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.407+0000 7f682e59c640 1 --2- 172.21.15.172:0/1164495767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d12391db40 0x55d12391f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.407+0000 7f682e59c640 1 -- 172.21.15.172:0/1164495767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d123927370 msgr2=0x55d12391f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.407+0000 7f682e59c640 1 --2- 172.21.15.172:0/1164495767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d123927370 0x55d12391f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.407+0000 7f682e59c640 1 -- 172.21.15.172:0/1164495767 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1239396d0 con 0x55d12391c5d0 2024-09-10T08:52:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.407+0000 7f682ed9d640 1 --2- 172.21.15.172:0/1164495767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d123927370 0x55d12391f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.408+0000 7f682e59c640 1 --2- 172.21.15.172:0/1164495767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d12391c5d0 0x55d123918d70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f682004c040 tx=0x7f682002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70b83f61ddb33432 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.408+0000 7f6817fff640 1 -- 172.21.15.172:0/1164495767 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820057070 con 0x55d12391c5d0 2024-09-10T08:52:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.408+0000 7f6817fff640 1 -- 172.21.15.172:0/1164495767 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6820004b70 con 0x55d12391c5d0 2024-09-10T08:52:44.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.408+0000 7f6817fff640 1 -- 172.21.15.172:0/1164495767 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820035070 con 0x55d12391c5d0 2024-09-10T08:52:44.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.408+0000 7f683710b740 1 -- 172.21.15.172:0/1164495767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d12391c5d0 msgr2=0x55d123918d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:44.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.408+0000 7f683710b740 1 --2- 172.21.15.172:0/1164495767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d12391c5d0 0x55d123918d70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f682004c040 tx=0x7f682002a410 comp rx=0 tx=0).stop 2024-09-10T08:52:44.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.408+0000 7f683710b740 1 -- 172.21.15.172:0/1164495767 shutdown_connections 2024-09-10T08:52:44.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.408+0000 7f683710b740 1 --2- 172.21.15.172:0/1164495767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d123927370 0x55d12391f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.408+0000 7f683710b740 1 --2- 172.21.15.172:0/1164495767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d12391db40 0x55d12391f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.408+0000 7f683710b740 1 --2- 172.21.15.172:0/1164495767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d12391c5d0 0x55d123918d70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.408+0000 7f683710b740 1 -- 172.21.15.172:0/1164495767 >> 172.21.15.172:0/1164495767 conn(0x55d1238e5310 msgr2=0x55d123917d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:44.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.408+0000 7f683710b740 1 -- 172.21.15.172:0/1164495767 shutdown_connections 2024-09-10T08:52:44.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.408+0000 7f683710b740 1 -- 172.21.15.172:0/1164495767 wait complete. 2024-09-10T08:52:44.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.409+0000 7f683710b740 1 -- start start 2024-09-10T08:52:44.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.409+0000 7f683710b740 1 Processor -- start 2024-09-10T08:52:44.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.409+0000 7f683710b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d12391c5d0 0x55d12392f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.409+0000 7f683710b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391db40 0x55d12392f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.409+0000 7f683710b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d123927370 0x55d12392c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.409+0000 7f683710b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f68301affb0 con 0x55d12391db40 2024-09-10T08:52:44.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.409+0000 7f683710b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68301af8e0 con 0x55d12391c5d0 2024-09-10T08:52:44.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.409+0000 7f683710b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f68301acb20 con 0x55d123927370 2024-09-10T08:52:44.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.409+0000 7f683710b740 10 client.0 _ll_register_callbacks cb 0x7f68200314a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:44.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.409+0000 7f682dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391db40 0x55d12392f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.409+0000 7f682dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391db40 0x55d12392f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41740/0 (socket says 172.21.15.172:41740) 2024-09-10T08:52:44.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.409+0000 7f682dd9b640 1 -- 172.21.15.172:0/3618843698 learned_addr learned my addr 172.21.15.172:0/3618843698 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:44.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.409+0000 7f682e59c640 1 --2- 172.21.15.172:0/3618843698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d12391c5d0 0x55d12392f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.409+0000 7f682ed9d640 1 --2- 172.21.15.172:0/3618843698 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d123927370 0x55d12392c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.410+0000 7f682e59c640 1 -- 172.21.15.172:0/3618843698 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d123927370 msgr2=0x55d12392c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:44.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.410+0000 7f682e59c640 1 --2- 172.21.15.172:0/3618843698 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d123927370 0x55d12392c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.410+0000 7f682e59c640 1 -- 172.21.15.172:0/3618843698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391db40 msgr2=0x55d12392f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:44.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.410+0000 7f682e59c640 1 --2- 172.21.15.172:0/3618843698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391db40 0x55d12392f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.410+0000 7f682e59c640 1 -- 172.21.15.172:0/3618843698 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d12392c990 con 0x55d12391c5d0 2024-09-10T08:52:44.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.410+0000 7f682dd9b640 1 --2- 172.21.15.172:0/3618843698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391db40 0x55d12392f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:44.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.410+0000 7f682e59c640 1 --2- 172.21.15.172:0/3618843698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d12391c5d0 0x55d12392f480 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f68200059d0 tx=0x7f68200318e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f672aeb3ecbcc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:44.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.410+0000 7f6815ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:52:44.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.410+0000 7f6815ffb640 1 -- 172.21.15.172:0/3618843698 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820035070 con 0x55d12391c5d0 2024-09-10T08:52:44.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.410+0000 7f6815ffb640 1 -- 172.21.15.172:0/3618843698 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f68200048c0 con 0x55d12391c5d0 2024-09-10T08:52:44.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.410+0000 7f683710b740 1 -- 172.21.15.172:0/3618843698 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6820004b70 con 0x55d12391c5d0 2024-09-10T08:52:44.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.410+0000 7f6815ffb640 1 -- 172.21.15.172:0/3618843698 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6820057070 con 0x55d12391c5d0 2024-09-10T08:52:44.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.410+0000 7f683710b740 10 client.26471 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:44.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.410+0000 7f683710b740 1 -- 172.21.15.172:0/3618843698 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d1239a2360 con 0x55d12391c5d0 2024-09-10T08:52:44.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.410+0000 7f6815ffb640 1 -- 172.21.15.172:0/3618843698 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6820004b70 con 0x55d12391c5d0 2024-09-10T08:52:44.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.410+0000 7f6815ffb640 1 -- 172.21.15.172:0/3618843698 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f682005a020 con 0x55d12391c5d0 2024-09-10T08:52:44.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.411+0000 7f683710b740 10 client.26471 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:44.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.411+0000 7f683710b740 10 client.26471 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:52:44.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.411+0000 7f683710b740 20 client.26471 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:44.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.411+0000 7f683710b740 1 --2- 172.21.15.172:0/3618843698 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d123936f70 0x55d1239a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:44.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.411+0000 7f683710b740 4 client.26471 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:52:44.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.411+0000 7f683710b740 1 -- 172.21.15.172:0/3618843698 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d123889d70 con 0x55d123936f70 2024-09-10T08:52:44.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.411+0000 7f682dd9b640 1 --2- 172.21.15.172:0/3618843698 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d123936f70 0x55d1239a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:44.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.411+0000 7f682dd9b640 1 --2- 172.21.15.172:0/3618843698 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d123936f70 0x55d1239a49d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=61552c21b8359244 server_cookie=9c39c6bdb96918b7 in_seq=0 out_seq=0 2024-09-10T08:52:44.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.411+0000 7f6815ffb640 10 client.26471 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:52:44.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.411+0000 7f6815ffb640 1 -- 172.21.15.172:0/3618843698 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d123889d70 con 0x55d123936f70 2024-09-10T08:52:44.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.411+0000 7f6815ffb640 10 client.26471 handle_command_reply: tid=0 2024-09-10T08:52:44.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f683710b740 1 client.26471 shutdown 2024-09-10T08:52:44.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f683710b740 1 -- 172.21.15.172:0/3618843698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d12391c5d0 msgr2=0x55d12392f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:44.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f683710b740 1 --2- 172.21.15.172:0/3618843698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d12391c5d0 0x55d12392f480 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f68200059d0 tx=0x7f68200318e0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f683710b740 1 -- 172.21.15.172:0/3618843698 shutdown_connections 2024-09-10T08:52:44.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f683710b740 1 --2- 172.21.15.172:0/3618843698 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d123927370 0x55d12392c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f683710b740 1 --2- 172.21.15.172:0/3618843698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d12391db40 0x55d12392f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f683710b740 1 --2- 172.21.15.172:0/3618843698 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d123936f70 0x55d1239a49d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f683710b740 1 --2- 172.21.15.172:0/3618843698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d12391c5d0 0x55d12392f480 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f683710b740 1 -- 172.21.15.172:0/3618843698 >> 172.21.15.172:0/3618843698 conn(0x55d1238e5310 msgr2=0x55d123925870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:44.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f6815ffb640 0 client.26471 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:52:44.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f683710b740 1 -- 172.21.15.172:0/3618843698 shutdown_connections 2024-09-10T08:52:44.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f683710b740 1 -- 172.21.15.172:0/3618843698 wait complete. 2024-09-10T08:52:44.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f683710b740 20 client.26471 trim_cache size 0 max 16384 2024-09-10T08:52:44.419 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:52:44.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f6834d32640 1 -- 172.21.15.172:0/2033028059 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68180a2b80 msgr2=0x7f68180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:44.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f6834d32640 1 --2- 172.21.15.172:0/2033028059 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68180a2b80 0x7f68180a5040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f682801f000 tx=0x7f682801e000 comp rx=0 tx=0).stop 2024-09-10T08:52:44.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f6834d32640 1 -- 172.21.15.172:0/2033028059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68301027a0 msgr2=0x7f6830198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:44.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f6834d32640 1 --2- 172.21.15.172:0/2033028059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68301027a0 0x7f6830198910 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f682400d770 tx=0x7f682400dc30 comp rx=0 tx=0).stop 2024-09-10T08:52:44.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f6834d32640 1 -- 172.21.15.172:0/2033028059 shutdown_connections 2024-09-10T08:52:44.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f6834d32640 1 --2- 172.21.15.172:0/2033028059 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68180a2b80 0x7f68180a5040 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f6834d32640 1 --2- 172.21.15.172:0/2033028059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6830106f20 0x7f683019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f6834d32640 1 --2- 172.21.15.172:0/2033028059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68301027a0 0x7f6830198910 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f6834d32640 1 --2- 172.21.15.172:0/2033028059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6830101df0 0x7f68301983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:44.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.412+0000 7f6834d32640 1 -- 172.21.15.172:0/2033028059 >> 172.21.15.172:0/2033028059 conn(0x7f68300fd8b0 msgr2=0x7f68300fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:44.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.413+0000 7f6834d32640 1 -- 172.21.15.172:0/2033028059 shutdown_connections 2024-09-10T08:52:44.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:44.413+0000 7f6834d32640 1 -- 172.21.15.172:0/2033028059 wait complete. 2024-09-10T08:52:44.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:44 smithi172 ceph-mon[30854]: pgmap v545: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:52:44.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:44 smithi174 ceph-mon[34068]: pgmap v545: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:52:44.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:44 smithi191 ceph-mon[35314]: pgmap v545: 129 pgs: 129 active+clean; 5.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:52:45.109 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bd6b2e81', quiesce took 5.0 seconds. Will release after: 15.100000000000001 2024-09-10T08:52:45.597 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T08:52:45.845 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:52:46.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.364+0000 7f83d0e3c640 1 -- 172.21.15.172:0/453282273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83cc108290 msgr2=0x7f83cc10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:46.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.364+0000 7f83d0e3c640 1 --2- 172.21.15.172:0/453282273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83cc108290 0x7f83cc10ec30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009cd0 tx=0x7f83c402a8e0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.364+0000 7f83d0e3c640 1 -- 172.21.15.172:0/453282273 shutdown_connections 2024-09-10T08:52:46.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.364+0000 7f83d0e3c640 1 --2- 172.21.15.172:0/453282273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83cc108290 0x7f83cc10ec30 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.364+0000 7f83d0e3c640 1 --2- 172.21.15.172:0/453282273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83cc103740 0x7f83cc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.364+0000 7f83d0e3c640 1 --2- 172.21.15.172:0/453282273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83cc102d90 0x7f83cc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.364+0000 7f83d0e3c640 1 -- 172.21.15.172:0/453282273 >> 172.21.15.172:0/453282273 conn(0x7f83cc0fc7d0 msgr2=0x7f83cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:46.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.364+0000 7f83d0e3c640 1 -- 172.21.15.172:0/453282273 shutdown_connections 2024-09-10T08:52:46.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.364+0000 7f83d0e3c640 1 -- 172.21.15.172:0/453282273 wait complete. 2024-09-10T08:52:46.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.364+0000 7f83d0e3c640 1 Processor -- start 2024-09-10T08:52:46.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83d0e3c640 1 -- start start 2024-09-10T08:52:46.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83d0e3c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83cc102d90 0x7f83cc1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83d0e3c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83cc103740 0x7f83cc100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83d0e3c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83cc108290 0x7f83cc101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83d0e3c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc111f80 con 0x7f83cc103740 2024-09-10T08:52:46.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83d0e3c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc111e00 con 0x7f83cc108290 2024-09-10T08:52:46.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83d0e3c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc112100 con 0x7f83cc102d90 2024-09-10T08:52:46.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83c9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83cc103740 0x7f83cc100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:46.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83c9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83cc103740 0x7f83cc100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44568/0 (socket says 172.21.15.172:44568) 2024-09-10T08:52:46.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83c9d9b640 1 -- 172.21.15.172:0/241791257 learned_addr learned my addr 172.21.15.172:0/241791257 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:46.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83cad9d640 1 --2- 172.21.15.172:0/241791257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83cc108290 0x7f83cc101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:46.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83c9d9b640 1 -- 172.21.15.172:0/241791257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83cc102d90 msgr2=0x7f83cc1009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:46.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83c9d9b640 1 --2- 172.21.15.172:0/241791257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83cc102d90 0x7f83cc1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83c9d9b640 1 -- 172.21.15.172:0/241791257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83cc108290 msgr2=0x7f83cc101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:46.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83c9d9b640 1 --2- 172.21.15.172:0/241791257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83cc108290 0x7f83cc101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83c9d9b640 1 -- 172.21.15.172:0/241791257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83cc1a69c0 con 0x7f83cc103740 2024-09-10T08:52:46.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83cad9d640 1 --2- 172.21.15.172:0/241791257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83cc108290 0x7f83cc101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:46.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83c9d9b640 1 --2- 172.21.15.172:0/241791257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83cc103740 0x7f83cc100f30 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f83c000d770 tx=0x7f83c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:46.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83bb7fe640 1 -- 172.21.15.172:0/241791257 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0017070 con 0x7f83cc103740 2024-09-10T08:52:46.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83bb7fe640 1 -- 172.21.15.172:0/241791257 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83c0009c30 con 0x7f83cc103740 2024-09-10T08:52:46.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83d0e3c640 1 -- 172.21.15.172:0/241791257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83cc1a6cb0 con 0x7f83cc103740 2024-09-10T08:52:46.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.365+0000 7f83bb7fe640 1 -- 172.21.15.172:0/241791257 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c000f620 con 0x7f83cc103740 2024-09-10T08:52:46.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.366+0000 7f83d0e3c640 1 -- 172.21.15.172:0/241791257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83cc1a71a0 con 0x7f83cc103740 2024-09-10T08:52:46.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.368+0000 7f83bb7fe640 1 -- 172.21.15.172:0/241791257 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f83c001a070 con 0x7f83cc103740 2024-09-10T08:52:46.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.368+0000 7f83d3215740 1 Processor -- start 2024-09-10T08:52:46.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.368+0000 7f83d3215740 1 -- start start 2024-09-10T08:52:46.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.368+0000 7f83bb7fe640 1 --2- 172.21.15.172:0/241791257 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83b40a2b80 0x7f83b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.368+0000 7f83d3215740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a7b40 0x558c883a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.368+0000 7f83d3215740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883a65d0 0x558c883a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.368+0000 7f83d3215740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883b1370 0x558c883b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.368+0000 7f83d3215740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558c8830c2f0 con 0x558c883b1370 2024-09-10T08:52:46.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.368+0000 7f83d3215740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558c882b3cf0 con 0x558c883a65d0 2024-09-10T08:52:46.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.369+0000 7f83d3215740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558c883082b0 con 0x558c883a7b40 2024-09-10T08:52:46.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.369+0000 7f83bb7fe640 1 -- 172.21.15.172:0/241791257 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f83c00cef80 con 0x7f83cc103740 2024-09-10T08:52:46.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.369+0000 7f83cad9d640 1 --2- 172.21.15.172:0/241791257 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83b40a2b80 0x7f83b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:46.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.369+0000 7f83ca59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a7b40 0x558c883a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:46.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.369+0000 7f83cad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883b1370 0x558c883b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:46.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.369+0000 7f83c9d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883a65d0 0x558c883a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:46.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.369+0000 7f83ca59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a7b40 0x558c883a6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:44940/0 (socket says 172.21.15.172:44940) 2024-09-10T08:52:46.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.369+0000 7f83ca59c640 1 -- 172.21.15.172:0/1933734076 learned_addr learned my addr 172.21.15.172:0/1933734076 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:46.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.369+0000 7f83ca59c640 1 -- 172.21.15.172:0/1933734076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883a65d0 msgr2=0x558c883a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:46.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.369+0000 7f83ca59c640 1 --2- 172.21.15.172:0/1933734076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883a65d0 0x558c883a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.369+0000 7f83ca59c640 1 -- 172.21.15.172:0/1933734076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883b1370 msgr2=0x558c883b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:46.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.369+0000 7f83ca59c640 1 --2- 172.21.15.172:0/1933734076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883b1370 0x558c883b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.369+0000 7f83ca59c640 1 -- 172.21.15.172:0/1933734076 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c882983a0 con 0x558c883a7b40 2024-09-10T08:52:46.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.369+0000 7f83cad9d640 1 --2- 172.21.15.172:0/1933734076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883b1370 0x558c883b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:46.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.369+0000 7f83cad9d640 1 --2- 172.21.15.172:0/241791257 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83b40a2b80 0x7f83b40a5040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f83c4033040 tx=0x7f83c4009910 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:46.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.369+0000 7f83ca59c640 1 --2- 172.21.15.172:0/1933734076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a7b40 0x558c883a6090 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009820 tx=0x7f83bc009ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8bc933725dd2ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:46.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.370+0000 7f83b8ff9640 1 -- 172.21.15.172:0/1933734076 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc015020 con 0x558c883a7b40 2024-09-10T08:52:46.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.370+0000 7f83b8ff9640 1 -- 172.21.15.172:0/1933734076 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83bc011420 con 0x558c883a7b40 2024-09-10T08:52:46.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.370+0000 7f83b8ff9640 1 -- 172.21.15.172:0/1933734076 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc0105c0 con 0x558c883a7b40 2024-09-10T08:52:46.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.370+0000 7f83d3215740 1 -- 172.21.15.172:0/1933734076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a7b40 msgr2=0x558c883a6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:46.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.370+0000 7f83d3215740 1 --2- 172.21.15.172:0/1933734076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a7b40 0x558c883a6090 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009820 tx=0x7f83bc009ce0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.370+0000 7f83d3215740 1 -- 172.21.15.172:0/1933734076 shutdown_connections 2024-09-10T08:52:46.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.370+0000 7f83d3215740 1 --2- 172.21.15.172:0/1933734076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883b1370 0x558c883b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.370+0000 7f83d3215740 1 --2- 172.21.15.172:0/1933734076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883a65d0 0x558c883a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.370+0000 7f83d3215740 1 --2- 172.21.15.172:0/1933734076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a7b40 0x558c883a6090 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.370+0000 7f83d3215740 1 -- 172.21.15.172:0/1933734076 >> 172.21.15.172:0/1933734076 conn(0x558c8836f310 msgr2=0x558c883a2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:46.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.371+0000 7f83d3215740 1 -- 172.21.15.172:0/1933734076 shutdown_connections 2024-09-10T08:52:46.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.371+0000 7f83d3215740 1 -- 172.21.15.172:0/1933734076 wait complete. 2024-09-10T08:52:46.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.371+0000 7f83d3215740 1 -- start start 2024-09-10T08:52:46.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.371+0000 7f83d3215740 1 Processor -- start 2024-09-10T08:52:46.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.371+0000 7f83d3215740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883a65d0 0x558c883b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.371+0000 7f83d3215740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883a7b40 0x558c883adf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.371+0000 7f83d3215740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883b1370 0x558c883ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.371+0000 7f83d3215740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558c883b4070 con 0x558c883a7b40 2024-09-10T08:52:46.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.371+0000 7f83d3215740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558c882c7060 con 0x558c883a65d0 2024-09-10T08:52:46.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.371+0000 7f83d3215740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558c883b41f0 con 0x558c883b1370 2024-09-10T08:52:46.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.371+0000 7f83d3215740 10 client.0 _ll_register_callbacks cb 0x558c8801a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:46.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.371+0000 7f83c9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883a7b40 0x558c883adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:46.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.371+0000 7f83c9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883a7b40 0x558c883adf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44586/0 (socket says 172.21.15.172:44586) 2024-09-10T08:52:46.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.371+0000 7f83c9d9b640 1 -- 172.21.15.172:0/2055185573 learned_addr learned my addr 172.21.15.172:0/2055185573 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:46.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.371+0000 7f83cad9d640 1 --2- 172.21.15.172:0/2055185573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883b1370 0x558c883ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:46.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83ca59c640 1 --2- 172.21.15.172:0/2055185573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883a65d0 0x558c883b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:46.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83c9d9b640 1 -- 172.21.15.172:0/2055185573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883b1370 msgr2=0x558c883ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:46.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83c9d9b640 1 --2- 172.21.15.172:0/2055185573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883b1370 0x558c883ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83c9d9b640 1 -- 172.21.15.172:0/2055185573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883a65d0 msgr2=0x558c883b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:46.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83c9d9b640 1 --2- 172.21.15.172:0/2055185573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883a65d0 0x558c883b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83c9d9b640 1 -- 172.21.15.172:0/2055185573 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c8807d3c0 con 0x558c883a7b40 2024-09-10T08:52:46.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83ca59c640 1 --2- 172.21.15.172:0/2055185573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883a65d0 0x558c883b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:46.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83c9d9b640 1 --2- 172.21.15.172:0/2055185573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883a7b40 0x558c883adf40 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f83c00cc720 tx=0x7f83c0009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edc78507d4cbc8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:46.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:46.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83b1ffb640 1 -- 172.21.15.172:0/2055185573 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0021070 con 0x558c883a7b40 2024-09-10T08:52:46.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83b1ffb640 1 -- 172.21.15.172:0/2055185573 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f83c00ccbe0 con 0x558c883a7b40 2024-09-10T08:52:46.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83d3215740 1 -- 172.21.15.172:0/2055185573 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83bc011420 con 0x558c883a7b40 2024-09-10T08:52:46.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83b1ffb640 1 -- 172.21.15.172:0/2055185573 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f83c009f830 con 0x558c883a7b40 2024-09-10T08:52:46.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83d3215740 10 client.17334 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:46.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83d3215740 1 -- 172.21.15.172:0/2055185573 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558c883bfa60 con 0x558c883a7b40 2024-09-10T08:52:46.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83b1ffb640 1 -- 172.21.15.172:0/2055185573 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83bc011420 con 0x558c883a7b40 2024-09-10T08:52:46.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83b1ffb640 1 -- 172.21.15.172:0/2055185573 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f83c00b4120 con 0x558c883a7b40 2024-09-10T08:52:46.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83d3215740 10 client.17334 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:46.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83d3215740 10 client.17334 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:52:46.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83d3215740 20 client.17334 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:46.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83d3215740 1 --2- 172.21.15.172:0/2055185573 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x558c883c0e70 0x558c883c3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83d3215740 4 client.17334 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:46.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.372+0000 7f83d3215740 1 -- 172.21.15.172:0/2055185573 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c88319860 con 0x558c883c0e70 2024-09-10T08:52:46.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.373+0000 7f83ca59c640 1 --2- 172.21.15.172:0/2055185573 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x558c883c0e70 0x558c883c3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:46.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.373+0000 7f83ca59c640 1 --2- 172.21.15.172:0/2055185573 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x558c883c0e70 0x558c883c3260 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8514c41b185cd4db server_cookie=595a1f747227176d in_seq=0 out_seq=0 2024-09-10T08:52:46.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.373+0000 7f83b1ffb640 10 client.17334 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:52:46.391 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.374+0000 7f83b1ffb640 1 -- 172.21.15.172:0/2055185573 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c88319860 con 0x558c883c0e70 2024-09-10T08:52:46.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.374+0000 7f83b1ffb640 10 client.17334 handle_command_reply: tid=0 2024-09-10T08:52:46.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.374+0000 7f83d3215740 1 client.17334 shutdown 2024-09-10T08:52:46.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.374+0000 7f83d3215740 1 -- 172.21.15.172:0/2055185573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883a7b40 msgr2=0x558c883adf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:46.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.374+0000 7f83d3215740 1 --2- 172.21.15.172:0/2055185573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883a7b40 0x558c883adf40 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f83c00cc720 tx=0x7f83c0009f20 comp rx=0 tx=0).stop 2024-09-10T08:52:46.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.374+0000 7f83d3215740 1 -- 172.21.15.172:0/2055185573 shutdown_connections 2024-09-10T08:52:46.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.374+0000 7f83d3215740 1 --2- 172.21.15.172:0/2055185573 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x558c883c0e70 0x558c883c3260 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.374+0000 7f83d3215740 1 --2- 172.21.15.172:0/2055185573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883b1370 0x558c883ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.374+0000 7f83b1ffb640 0 client.17334 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:52:46.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.374+0000 7f83d3215740 1 --2- 172.21.15.172:0/2055185573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883a7b40 0x558c883adf40 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.374+0000 7f83d3215740 1 --2- 172.21.15.172:0/2055185573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883a65d0 0x558c883b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.374+0000 7f83d3215740 1 -- 172.21.15.172:0/2055185573 >> 172.21.15.172:0/2055185573 conn(0x558c8836f310 msgr2=0x558c883a8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:46.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.374+0000 7f83d3215740 1 -- 172.21.15.172:0/2055185573 shutdown_connections 2024-09-10T08:52:46.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.374+0000 7f83d3215740 1 -- 172.21.15.172:0/2055185573 wait complete. 2024-09-10T08:52:46.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.374+0000 7f83d3215740 20 client.17334 trim_cache size 0 max 16384 2024-09-10T08:52:46.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.387+0000 7f83d3215740 1 Processor -- start 2024-09-10T08:52:46.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.387+0000 7f83d3215740 1 -- start start 2024-09-10T08:52:46.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.387+0000 7f83d3215740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a65d0 0x558c883bd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.387+0000 7f83d3215740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883a7b40 0x558c883a9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.387+0000 7f83d3215740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883b1370 0x558c883a9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.387+0000 7f83d3215740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1b0db0 con 0x558c883b1370 2024-09-10T08:52:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.387+0000 7f83d3215740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1b28f0 con 0x558c883a7b40 2024-09-10T08:52:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.387+0000 7f83d3215740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1b1480 con 0x558c883a65d0 2024-09-10T08:52:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.387+0000 7f83cad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883b1370 0x558c883a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.387+0000 7f83cad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883b1370 0x558c883a9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44594/0 (socket says 172.21.15.172:44594) 2024-09-10T08:52:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.387+0000 7f83cad9d640 1 -- 172.21.15.172:0/1697441974 learned_addr learned my addr 172.21.15.172:0/1697441974 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.387+0000 7f83ca59c640 1 --2- 172.21.15.172:0/1697441974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a65d0 0x558c883bd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.387+0000 7f83c9d9b640 1 --2- 172.21.15.172:0/1697441974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883a7b40 0x558c883a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.388+0000 7f83ca59c640 1 -- 172.21.15.172:0/1697441974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883a7b40 msgr2=0x558c883a9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.388+0000 7f83ca59c640 1 --2- 172.21.15.172:0/1697441974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883a7b40 0x558c883a9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.388+0000 7f83ca59c640 1 -- 172.21.15.172:0/1697441974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883b1370 msgr2=0x558c883a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.388+0000 7f83ca59c640 1 --2- 172.21.15.172:0/1697441974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883b1370 0x558c883a9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.388+0000 7f83ca59c640 1 -- 172.21.15.172:0/1697441974 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c883b1000 con 0x558c883a65d0 2024-09-10T08:52:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.388+0000 7f83cad9d640 1 --2- 172.21.15.172:0/1697441974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883b1370 0x558c883a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.388+0000 7f83ca59c640 1 --2- 172.21.15.172:0/1697441974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a65d0 0x558c883bd340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f83bc006cc0 tx=0x7f83bc0091c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7315dc27f3fbd9db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.388+0000 7f83b3fff640 1 -- 172.21.15.172:0/1697441974 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc02a070 con 0x558c883a65d0 2024-09-10T08:52:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.388+0000 7f83b3fff640 1 -- 172.21.15.172:0/1697441974 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83bc011420 con 0x558c883a65d0 2024-09-10T08:52:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.388+0000 7f83b3fff640 1 -- 172.21.15.172:0/1697441974 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc015050 con 0x558c883a65d0 2024-09-10T08:52:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 1 -- 172.21.15.172:0/1697441974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a65d0 msgr2=0x558c883bd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 1 --2- 172.21.15.172:0/1697441974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a65d0 0x558c883bd340 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f83bc006cc0 tx=0x7f83bc0091c0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 1 -- 172.21.15.172:0/1697441974 shutdown_connections 2024-09-10T08:52:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 1 --2- 172.21.15.172:0/1697441974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883b1370 0x558c883a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 1 --2- 172.21.15.172:0/1697441974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883a7b40 0x558c883a9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 1 --2- 172.21.15.172:0/1697441974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a65d0 0x558c883bd340 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 1 -- 172.21.15.172:0/1697441974 >> 172.21.15.172:0/1697441974 conn(0x558c8836f310 msgr2=0x558c883ad030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:46.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 1 -- 172.21.15.172:0/1697441974 shutdown_connections 2024-09-10T08:52:46.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 1 -- 172.21.15.172:0/1697441974 wait complete. 2024-09-10T08:52:46.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 1 -- start start 2024-09-10T08:52:46.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 1 Processor -- start 2024-09-10T08:52:46.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883a65d0 0x558c883b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a7b40 0x558c883b5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883b1370 0x558c883b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1b2220 con 0x558c883a65d0 2024-09-10T08:52:46.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1b1b50 con 0x558c883b1370 2024-09-10T08:52:46.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1aed90 con 0x558c883a7b40 2024-09-10T08:52:46.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83d3215740 10 client.0 _ll_register_callbacks cb 0x7f83c009afc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:46.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83ca59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883a65d0 0x558c883b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:46.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83ca59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883a65d0 0x558c883b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44600/0 (socket says 172.21.15.172:44600) 2024-09-10T08:52:46.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.389+0000 7f83ca59c640 1 -- 172.21.15.172:0/193068879 learned_addr learned my addr 172.21.15.172:0/193068879 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:46.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.390+0000 7f83c9d9b640 1 --2- 172.21.15.172:0/193068879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a7b40 0x558c883b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:46.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.390+0000 7f83cad9d640 1 --2- 172.21.15.172:0/193068879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883b1370 0x558c883b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:46.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.390+0000 7f83ca59c640 1 -- 172.21.15.172:0/193068879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a7b40 msgr2=0x558c883b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:46.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.390+0000 7f83ca59c640 1 --2- 172.21.15.172:0/193068879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a7b40 0x558c883b5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.390+0000 7f83ca59c640 1 -- 172.21.15.172:0/193068879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883b1370 msgr2=0x558c883b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:46.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.390+0000 7f83ca59c640 1 --2- 172.21.15.172:0/193068879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883b1370 0x558c883b5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.390+0000 7f83ca59c640 1 -- 172.21.15.172:0/193068879 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c8842aab0 con 0x558c883a65d0 2024-09-10T08:52:46.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.390+0000 7f83c9d9b640 1 --2- 172.21.15.172:0/193068879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a7b40 0x558c883b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:52:46.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.390+0000 7f83ca59c640 1 --2- 172.21.15.172:0/193068879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883a65d0 0x558c883b5150 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f83bc006bd0 tx=0x7f83bc006c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b930eae79064a59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:46.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.390+0000 7f83b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:46.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.390+0000 7f83b1ffb640 1 -- 172.21.15.172:0/193068879 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc034060 con 0x558c883a65d0 2024-09-10T08:52:46.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.390+0000 7f83b1ffb640 1 -- 172.21.15.172:0/193068879 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f83bc00bce0 con 0x558c883a65d0 2024-09-10T08:52:46.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.390+0000 7f83d3215740 1 -- 172.21.15.172:0/193068879 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83bc011420 con 0x558c883a65d0 2024-09-10T08:52:46.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.390+0000 7f83b1ffb640 1 -- 172.21.15.172:0/193068879 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f83bc015030 con 0x558c883a65d0 2024-09-10T08:52:46.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.390+0000 7f83d3215740 10 client.17340 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:46.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.390+0000 7f83d3215740 1 -- 172.21.15.172:0/193068879 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558c8842bcf0 con 0x558c883a65d0 2024-09-10T08:52:46.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.391+0000 7f83b1ffb640 1 -- 172.21.15.172:0/193068879 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83bc011420 con 0x558c883a65d0 2024-09-10T08:52:46.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.391+0000 7f83b1ffb640 1 -- 172.21.15.172:0/193068879 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f83bc025440 con 0x558c883a65d0 2024-09-10T08:52:46.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.391+0000 7f83d3215740 10 client.17340 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:46.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.391+0000 7f83d3215740 10 client.17340 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:52:46.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.391+0000 7f83d3215740 20 client.17340 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:46.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.391+0000 7f83d3215740 1 --2- 172.21.15.172:0/193068879 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x558c883c0e70 0x558c8842e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.391+0000 7f83d3215740 4 client.17340 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:52:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.391+0000 7f83d3215740 1 -- 172.21.15.172:0/193068879 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c88313d70 con 0x558c883c0e70 2024-09-10T08:52:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.391+0000 7f83c9d9b640 1 --2- 172.21.15.172:0/193068879 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x558c883c0e70 0x558c8842e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.391+0000 7f83b1ffb640 10 client.17340 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:52:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.391+0000 7f83c9d9b640 1 --2- 172.21.15.172:0/193068879 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x558c883c0e70 0x558c8842e220 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2eda826c5be7ec39 server_cookie=3bc1988bd7067d10 in_seq=0 out_seq=0 2024-09-10T08:52:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83b1ffb640 1 -- 172.21.15.172:0/193068879 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c88313d70 con 0x558c883c0e70 2024-09-10T08:52:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83b1ffb640 10 client.17340 handle_command_reply: tid=0 2024-09-10T08:52:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d3215740 1 client.17340 shutdown 2024-09-10T08:52:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d3215740 1 -- 172.21.15.172:0/193068879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883a65d0 msgr2=0x558c883b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d3215740 1 --2- 172.21.15.172:0/193068879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883a65d0 0x558c883b5150 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f83bc006bd0 tx=0x7f83bc006c00 comp rx=0 tx=0).stop 2024-09-10T08:52:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d3215740 1 -- 172.21.15.172:0/193068879 shutdown_connections 2024-09-10T08:52:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d3215740 1 --2- 172.21.15.172:0/193068879 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x558c883c0e70 0x558c8842e220 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83b1ffb640 0 client.17340 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:52:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d3215740 1 --2- 172.21.15.172:0/193068879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c883b1370 0x558c883b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d3215740 1 --2- 172.21.15.172:0/193068879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c883a7b40 0x558c883b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d3215740 1 --2- 172.21.15.172:0/193068879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c883a65d0 0x558c883b5150 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d3215740 1 -- 172.21.15.172:0/193068879 >> 172.21.15.172:0/193068879 conn(0x558c8836f310 msgr2=0x558c883af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d3215740 1 -- 172.21.15.172:0/193068879 shutdown_connections 2024-09-10T08:52:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d3215740 1 -- 172.21.15.172:0/193068879 wait complete. 2024-09-10T08:52:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d3215740 20 client.17340 trim_cache size 0 max 16384 2024-09-10T08:52:46.405 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:52:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d0e3c640 1 -- 172.21.15.172:0/241791257 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83b40a2b80 msgr2=0x7f83b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d0e3c640 1 --2- 172.21.15.172:0/241791257 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83b40a2b80 0x7f83b40a5040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f83c4033040 tx=0x7f83c4009910 comp rx=0 tx=0).stop 2024-09-10T08:52:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d0e3c640 1 -- 172.21.15.172:0/241791257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83cc103740 msgr2=0x7f83cc100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d0e3c640 1 --2- 172.21.15.172:0/241791257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83cc103740 0x7f83cc100f30 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f83c000d770 tx=0x7f83c000dc30 comp rx=0 tx=0).stop 2024-09-10T08:52:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d0e3c640 1 -- 172.21.15.172:0/241791257 shutdown_connections 2024-09-10T08:52:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d0e3c640 1 --2- 172.21.15.172:0/241791257 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83b40a2b80 0x7f83b40a5040 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d0e3c640 1 --2- 172.21.15.172:0/241791257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83cc108290 0x7f83cc101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d0e3c640 1 --2- 172.21.15.172:0/241791257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83cc103740 0x7f83cc100f30 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.392+0000 7f83d0e3c640 1 --2- 172.21.15.172:0/241791257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83cc102d90 0x7f83cc1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.393+0000 7f83d0e3c640 1 -- 172.21.15.172:0/241791257 >> 172.21.15.172:0/241791257 conn(0x7f83cc0fc7d0 msgr2=0x7f83cc105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.393+0000 7f83d0e3c640 1 -- 172.21.15.172:0/241791257 shutdown_connections 2024-09-10T08:52:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:46.393+0000 7f83d0e3c640 1 -- 172.21.15.172:0/241791257 wait complete. 2024-09-10T08:52:46.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:46 smithi172 ceph-mon[30854]: pgmap v546: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-09-10T08:52:46.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:46 smithi174 ceph-mon[34068]: pgmap v546: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-09-10T08:52:46.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:46 smithi191 ceph-mon[35314]: pgmap v546: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-09-10T08:52:47.014 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:52:47.262 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:52:47.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.721+0000 7efecfe9c640 1 -- 172.21.15.172:0/1315872133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec810c170 msgr2=0x7efec810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:47.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.721+0000 7efecfe9c640 1 --2- 172.21.15.172:0/1315872133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec810c170 0x7efec810e610 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7efec0009ca0 tx=0x7efec002a950 comp rx=0 tx=0).stop 2024-09-10T08:52:47.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.721+0000 7efecfe9c640 1 -- 172.21.15.172:0/1315872133 shutdown_connections 2024-09-10T08:52:47.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.721+0000 7efecfe9c640 1 --2- 172.21.15.172:0/1315872133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec810c170 0x7efec810e610 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.721+0000 7efecfe9c640 1 --2- 172.21.15.172:0/1315872133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efec8103660 0x7efec810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.721+0000 7efecfe9c640 1 --2- 172.21.15.172:0/1315872133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efec8102d40 0x7efec8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.721+0000 7efecfe9c640 1 -- 172.21.15.172:0/1315872133 >> 172.21.15.172:0/1315872133 conn(0x7efec80fc7a0 msgr2=0x7efec80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:47.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.721+0000 7efecfe9c640 1 -- 172.21.15.172:0/1315872133 shutdown_connections 2024-09-10T08:52:47.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.721+0000 7efecfe9c640 1 -- 172.21.15.172:0/1315872133 wait complete. 2024-09-10T08:52:47.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efecfe9c640 1 Processor -- start 2024-09-10T08:52:47.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efecfe9c640 1 -- start start 2024-09-10T08:52:47.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efecfe9c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec8102d40 0x7efec8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efecfe9c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efec8103660 0x7efec8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efecfe9c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efec810c170 0x7efec819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efecfe9c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efec8111ee0 con 0x7efec8102d40 2024-09-10T08:52:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efecfe9c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efec8111d60 con 0x7efec810c170 2024-09-10T08:52:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efecfe9c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efec8112060 con 0x7efec8103660 2024-09-10T08:52:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efecdc38640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec8102d40 0x7efec8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efecdc38640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec8102d40 0x7efec8198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44622/0 (socket says 172.21.15.172:44622) 2024-09-10T08:52:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efecdc38640 1 -- 172.21.15.172:0/1575594505 learned_addr learned my addr 172.21.15.172:0/1575594505 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efece439640 1 --2- 172.21.15.172:0/1575594505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efec810c170 0x7efec819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efecdc38640 1 -- 172.21.15.172:0/1575594505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efec8103660 msgr2=0x7efec8198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efecdc38640 1 --2- 172.21.15.172:0/1575594505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efec8103660 0x7efec8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efecdc38640 1 -- 172.21.15.172:0/1575594505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efec810c170 msgr2=0x7efec819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efecdc38640 1 --2- 172.21.15.172:0/1575594505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efec810c170 0x7efec819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efecdc38640 1 -- 172.21.15.172:0/1575594505 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec819d3f0 con 0x7efec8102d40 2024-09-10T08:52:47.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efece439640 1 --2- 172.21.15.172:0/1575594505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efec810c170 0x7efec819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.722+0000 7efecdc38640 1 --2- 172.21.15.172:0/1575594505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec8102d40 0x7efec8198440 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7efec400b4a0 tx=0x7efec400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.723+0000 7efebeffd640 1 -- 172.21.15.172:0/1575594505 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4014070 con 0x7efec8102d40 2024-09-10T08:52:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.723+0000 7efebeffd640 1 -- 172.21.15.172:0/1575594505 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efec4010ac0 con 0x7efec8102d40 2024-09-10T08:52:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.723+0000 7efecfe9c640 1 -- 172.21.15.172:0/1575594505 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec819d6e0 con 0x7efec8102d40 2024-09-10T08:52:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.723+0000 7efebeffd640 1 -- 172.21.15.172:0/1575594505 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4017c30 con 0x7efec8102d40 2024-09-10T08:52:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.723+0000 7efecfe9c640 1 -- 172.21.15.172:0/1575594505 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec81a4fc0 con 0x7efec8102d40 2024-09-10T08:52:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.725+0000 7efebeffd640 1 -- 172.21.15.172:0/1575594505 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efec4017070 con 0x7efec8102d40 2024-09-10T08:52:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.726+0000 7efed2275740 1 Processor -- start 2024-09-10T08:52:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.726+0000 7efed2275740 1 -- start start 2024-09-10T08:52:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.726+0000 7efed2275740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e56090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.726+0000 7efed2275740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e565d0 0x55ec43e56a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.726+0000 7efed2275740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e61370 0x55ec43e63760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.726+0000 7efed2275740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ec43dbc2f0 con 0x55ec43e57b40 2024-09-10T08:52:47.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.726+0000 7efed2275740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ec43d63cf0 con 0x55ec43e565d0 2024-09-10T08:52:47.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.726+0000 7efed2275740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ec43db82b0 con 0x55ec43e61370 2024-09-10T08:52:47.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.726+0000 7efece439640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e56090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:47.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.726+0000 7efece439640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e56090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44624/0 (socket says 172.21.15.172:44624) 2024-09-10T08:52:47.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.726+0000 7efece439640 1 -- 172.21.15.172:0/3952525546 learned_addr learned my addr 172.21.15.172:0/3952525546 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:47.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.726+0000 7efecdc38640 1 --2- 172.21.15.172:0/3952525546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e565d0 0x55ec43e56a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:47.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.726+0000 7efecd437640 1 --2- 172.21.15.172:0/3952525546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e61370 0x55ec43e63760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:47.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.726+0000 7efece439640 1 -- 172.21.15.172:0/3952525546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e61370 msgr2=0x55ec43e63760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:47.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.727+0000 7efebeffd640 1 --2- 172.21.15.172:0/1575594505 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efeb40a2b80 0x7efeb40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.727+0000 7efece439640 1 --2- 172.21.15.172:0/3952525546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e61370 0x55ec43e63760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.727+0000 7efece439640 1 -- 172.21.15.172:0/3952525546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e565d0 msgr2=0x55ec43e56a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:47.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.727+0000 7efece439640 1 --2- 172.21.15.172:0/3952525546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e565d0 0x55ec43e56a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.727+0000 7efece439640 1 -- 172.21.15.172:0/3952525546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec43d483a0 con 0x55ec43e57b40 2024-09-10T08:52:47.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.727+0000 7efecdc38640 1 --2- 172.21.15.172:0/3952525546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e565d0 0x55ec43e56a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:47.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.727+0000 7efecd437640 1 --2- 172.21.15.172:0/3952525546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e61370 0x55ec43e63760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:47.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.727+0000 7efebeffd640 1 -- 172.21.15.172:0/1575594505 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7efec40ceb20 con 0x7efec8102d40 2024-09-10T08:52:47.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.727+0000 7efece439640 1 --2- 172.21.15.172:0/1575594505 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efeb40a2b80 0x7efeb40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:47.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.727+0000 7efece439640 1 --2- 172.21.15.172:0/3952525546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e56090 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7efec002ff00 tx=0x7efec002ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0f59136107fc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:47.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.727+0000 7efeb3fff640 1 -- 172.21.15.172:0/3952525546 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0034050 con 0x55ec43e57b40 2024-09-10T08:52:47.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.727+0000 7efeb3fff640 1 -- 172.21.15.172:0/3952525546 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efec002f5e0 con 0x55ec43e57b40 2024-09-10T08:52:47.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.727+0000 7efeb3fff640 1 -- 172.21.15.172:0/3952525546 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0030720 con 0x55ec43e57b40 2024-09-10T08:52:47.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.728+0000 7efed2275740 1 -- 172.21.15.172:0/3952525546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 msgr2=0x55ec43e56090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:47.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.728+0000 7efece439640 1 --2- 172.21.15.172:0/1575594505 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efeb40a2b80 0x7efeb40a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efec0048000 tx=0x7efec0049000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:47.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.728+0000 7efed2275740 1 --2- 172.21.15.172:0/3952525546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e56090 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7efec002ff00 tx=0x7efec002ff30 comp rx=0 tx=0).stop 2024-09-10T08:52:47.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.728+0000 7efed2275740 1 -- 172.21.15.172:0/3952525546 shutdown_connections 2024-09-10T08:52:47.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.728+0000 7efed2275740 1 --2- 172.21.15.172:0/3952525546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e61370 0x55ec43e63760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.728+0000 7efed2275740 1 --2- 172.21.15.172:0/3952525546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e565d0 0x55ec43e56a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.728+0000 7efed2275740 1 --2- 172.21.15.172:0/3952525546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e56090 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.728+0000 7efed2275740 1 -- 172.21.15.172:0/3952525546 >> 172.21.15.172:0/3952525546 conn(0x55ec43e1f310 msgr2=0x55ec43e52c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:47.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.728+0000 7efed2275740 1 -- 172.21.15.172:0/3952525546 shutdown_connections 2024-09-10T08:52:47.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.728+0000 7efed2275740 1 -- 172.21.15.172:0/3952525546 wait complete. 2024-09-10T08:52:47.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.728+0000 7efed2275740 1 -- start start 2024-09-10T08:52:47.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efed2275740 1 Processor -- start 2024-09-10T08:52:47.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efed2275740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e565d0 0x55ec43e60cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efed2275740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e5dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efed2275740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e61370 0x55ec43e5e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efed2275740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ec43e64070 con 0x55ec43e57b40 2024-09-10T08:52:47.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efed2275740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ec43d77060 con 0x55ec43e565d0 2024-09-10T08:52:47.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efed2275740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ec43e641f0 con 0x55ec43e61370 2024-09-10T08:52:47.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efed2275740 10 client.0 _ll_register_callbacks cb 0x55ec43aca8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:47.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efecd437640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e5dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:47.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efecd437640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e5dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44630/0 (socket says 172.21.15.172:44630) 2024-09-10T08:52:47.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efecd437640 1 -- 172.21.15.172:0/1293141128 learned_addr learned my addr 172.21.15.172:0/1293141128 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:47.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efecdc38640 1 --2- 172.21.15.172:0/1293141128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e565d0 0x55ec43e60cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:47.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efecd437640 1 -- 172.21.15.172:0/1293141128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e61370 msgr2=0x55ec43e5e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:47.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efece439640 1 --2- 172.21.15.172:0/1293141128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e61370 0x55ec43e5e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:47.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efecd437640 1 --2- 172.21.15.172:0/1293141128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e61370 0x55ec43e5e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efecd437640 1 -- 172.21.15.172:0/1293141128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e565d0 msgr2=0x55ec43e60cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:47.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efecd437640 1 --2- 172.21.15.172:0/1293141128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e565d0 0x55ec43e60cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efecd437640 1 -- 172.21.15.172:0/1293141128 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec43b2d3c0 con 0x55ec43e57b40 2024-09-10T08:52:47.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.729+0000 7efece439640 1 --2- 172.21.15.172:0/1293141128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e61370 0x55ec43e5e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:47.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.730+0000 7efecd437640 1 --2- 172.21.15.172:0/1293141128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e5dfd0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7efeb80103f0 tx=0x7efeb80108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f817fb87fcb08dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:47.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.730+0000 7efeb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:47.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.730+0000 7efeb17fa640 1 -- 172.21.15.172:0/1293141128 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb800e2e0 con 0x55ec43e57b40 2024-09-10T08:52:47.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.730+0000 7efeb17fa640 1 -- 172.21.15.172:0/1293141128 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7efeb800c4c0 con 0x55ec43e57b40 2024-09-10T08:52:47.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.730+0000 7efed2275740 1 -- 172.21.15.172:0/1293141128 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efec002f5e0 con 0x55ec43e57b40 2024-09-10T08:52:47.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.730+0000 7efeb17fa640 1 -- 172.21.15.172:0/1293141128 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efeb8024050 con 0x55ec43e57b40 2024-09-10T08:52:47.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.730+0000 7efed2275740 10 client.17352 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:47.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.730+0000 7efed2275740 1 -- 172.21.15.172:0/1293141128 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ec43e6fa90 con 0x55ec43e57b40 2024-09-10T08:52:47.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.730+0000 7efeb17fa640 1 -- 172.21.15.172:0/1293141128 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efec002f5e0 con 0x55ec43e57b40 2024-09-10T08:52:47.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.730+0000 7efeb17fa640 1 -- 172.21.15.172:0/1293141128 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7efeb801f030 con 0x55ec43e57b40 2024-09-10T08:52:47.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.730+0000 7efed2275740 10 client.17352 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:47.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.730+0000 7efed2275740 10 client.17352 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:52:47.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.730+0000 7efed2275740 20 client.17352 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:47.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.730+0000 7efed2275740 1 --2- 172.21.15.172:0/1293141128 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55ec43e70ff0 0x55ec43e73410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.730+0000 7efed2275740 4 client.17352 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:47.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.730+0000 7efed2275740 1 -- 172.21.15.172:0/1293141128 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec43dc9860 con 0x55ec43e70ff0 2024-09-10T08:52:47.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.731+0000 7efecdc38640 1 --2- 172.21.15.172:0/1293141128 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55ec43e70ff0 0x55ec43e73410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:47.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.731+0000 7efeb17fa640 10 client.17352 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:52:47.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.731+0000 7efecdc38640 1 --2- 172.21.15.172:0/1293141128 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55ec43e70ff0 0x55ec43e73410 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a993d6642b73e00a server_cookie=37c179759f14f8f4 in_seq=0 out_seq=0 2024-09-10T08:52:47.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.732+0000 7efeb17fa640 1 -- 172.21.15.172:0/1293141128 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec43dc9860 con 0x55ec43e70ff0 2024-09-10T08:52:47.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.732+0000 7efeb17fa640 10 client.17352 handle_command_reply: tid=0 2024-09-10T08:52:47.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.732+0000 7efed2275740 1 client.17352 shutdown 2024-09-10T08:52:47.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.732+0000 7efed2275740 1 -- 172.21.15.172:0/1293141128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 msgr2=0x55ec43e5dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:47.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.732+0000 7efed2275740 1 --2- 172.21.15.172:0/1293141128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e5dfd0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7efeb80103f0 tx=0x7efeb80108b0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.732+0000 7efed2275740 1 -- 172.21.15.172:0/1293141128 shutdown_connections 2024-09-10T08:52:47.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.732+0000 7efed2275740 1 --2- 172.21.15.172:0/1293141128 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55ec43e70ff0 0x55ec43e73410 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.732+0000 7efed2275740 1 --2- 172.21.15.172:0/1293141128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e61370 0x55ec43e5e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.732+0000 7efed2275740 1 --2- 172.21.15.172:0/1293141128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e5dfd0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.732+0000 7efed2275740 1 --2- 172.21.15.172:0/1293141128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e565d0 0x55ec43e60cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.732+0000 7efeb17fa640 0 client.17352 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:52:47.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.732+0000 7efed2275740 1 -- 172.21.15.172:0/1293141128 >> 172.21.15.172:0/1293141128 conn(0x55ec43e1f310 msgr2=0x55ec43e58930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:47.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.732+0000 7efed2275740 1 -- 172.21.15.172:0/1293141128 shutdown_connections 2024-09-10T08:52:47.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.732+0000 7efed2275740 1 -- 172.21.15.172:0/1293141128 wait complete. 2024-09-10T08:52:47.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.732+0000 7efed2275740 20 client.17352 trim_cache size 0 max 16384 2024-09-10T08:52:47.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.745+0000 7efed2275740 1 Processor -- start 2024-09-10T08:52:47.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.745+0000 7efed2275740 1 -- start start 2024-09-10T08:52:47.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.745+0000 7efed2275740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e565d0 0x55ec43e52d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.745+0000 7efed2275740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e57b40 0x55ec43e6d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.745+0000 7efed2275740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e61370 0x55ec43e5ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.745+0000 7efed2275740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efec81ac9a0 con 0x55ec43e61370 2024-09-10T08:52:47.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.745+0000 7efed2275740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efec81ae4e0 con 0x55ec43e57b40 2024-09-10T08:52:47.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.745+0000 7efed2275740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efec81ad070 con 0x55ec43e565d0 2024-09-10T08:52:47.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.745+0000 7efece439640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e61370 0x55ec43e5ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:47.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.745+0000 7efece439640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e61370 0x55ec43e5ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44640/0 (socket says 172.21.15.172:44640) 2024-09-10T08:52:47.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.745+0000 7efece439640 1 -- 172.21.15.172:0/1858949476 learned_addr learned my addr 172.21.15.172:0/1858949476 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:47.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.746+0000 7efecdc38640 1 --2- 172.21.15.172:0/1858949476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e565d0 0x55ec43e52d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:47.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.746+0000 7efecd437640 1 --2- 172.21.15.172:0/1858949476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e57b40 0x55ec43e6d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:47.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.746+0000 7efece439640 1 -- 172.21.15.172:0/1858949476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e565d0 msgr2=0x55ec43e52d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:47.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.746+0000 7efece439640 1 --2- 172.21.15.172:0/1858949476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e565d0 0x55ec43e52d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.746+0000 7efece439640 1 -- 172.21.15.172:0/1858949476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e57b40 msgr2=0x55ec43e6d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:47.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.746+0000 7efece439640 1 --2- 172.21.15.172:0/1858949476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e57b40 0x55ec43e6d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.746+0000 7efece439640 1 -- 172.21.15.172:0/1858949476 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec43e59120 con 0x55ec43e61370 2024-09-10T08:52:47.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.746+0000 7efecd437640 1 --2- 172.21.15.172:0/1858949476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e57b40 0x55ec43e6d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:47.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.746+0000 7efece439640 1 --2- 172.21.15.172:0/1858949476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e61370 0x55ec43e5ff40 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7efec0009ca0 tx=0x7efec004f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7305417ef00aa9f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:47.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.746+0000 7efeb37fe640 1 -- 172.21.15.172:0/1858949476 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0034050 con 0x55ec43e61370 2024-09-10T08:52:47.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.746+0000 7efeb37fe640 1 -- 172.21.15.172:0/1858949476 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efec0054070 con 0x55ec43e61370 2024-09-10T08:52:47.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.746+0000 7efeb37fe640 1 -- 172.21.15.172:0/1858949476 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0003f00 con 0x55ec43e61370 2024-09-10T08:52:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.747+0000 7efed2275740 1 -- 172.21.15.172:0/1858949476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e61370 msgr2=0x55ec43e5ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.747+0000 7efed2275740 1 --2- 172.21.15.172:0/1858949476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e61370 0x55ec43e5ff40 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7efec0009ca0 tx=0x7efec004f000 comp rx=0 tx=0).stop 2024-09-10T08:52:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.747+0000 7efed2275740 1 -- 172.21.15.172:0/1858949476 shutdown_connections 2024-09-10T08:52:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.747+0000 7efed2275740 1 --2- 172.21.15.172:0/1858949476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e61370 0x55ec43e5ff40 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.747+0000 7efed2275740 1 --2- 172.21.15.172:0/1858949476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e57b40 0x55ec43e6d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.747+0000 7efed2275740 1 --2- 172.21.15.172:0/1858949476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e565d0 0x55ec43e52d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.747+0000 7efed2275740 1 -- 172.21.15.172:0/1858949476 >> 172.21.15.172:0/1858949476 conn(0x55ec43e1f310 msgr2=0x55ec43e5f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.747+0000 7efed2275740 1 -- 172.21.15.172:0/1858949476 shutdown_connections 2024-09-10T08:52:47.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.747+0000 7efed2275740 1 -- 172.21.15.172:0/1858949476 wait complete. 2024-09-10T08:52:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.747+0000 7efed2275740 1 -- start start 2024-09-10T08:52:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.747+0000 7efed2275740 1 Processor -- start 2024-09-10T08:52:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efed2275740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e565d0 0x55ec43e5b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efed2275740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e5b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efed2275740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e61370 0x55ec43e5bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efed2275740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efec81ade10 con 0x55ec43e57b40 2024-09-10T08:52:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efed2275740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efec81ad740 con 0x55ec43e61370 2024-09-10T08:52:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efed2275740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efec81aa980 con 0x55ec43e565d0 2024-09-10T08:52:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efed2275740 10 client.0 _ll_register_callbacks cb 0x7efec806b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efecd437640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e5b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:47.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efecd437640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e5b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44648/0 (socket says 172.21.15.172:44648) 2024-09-10T08:52:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efecd437640 1 -- 172.21.15.172:0/480255213 learned_addr learned my addr 172.21.15.172:0/480255213 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efecdc38640 1 --2- 172.21.15.172:0/480255213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e565d0 0x55ec43e5b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efece439640 1 --2- 172.21.15.172:0/480255213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e61370 0x55ec43e5bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efecd437640 1 -- 172.21.15.172:0/480255213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e565d0 msgr2=0x55ec43e5b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efecd437640 1 --2- 172.21.15.172:0/480255213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e565d0 0x55ec43e5b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efecd437640 1 -- 172.21.15.172:0/480255213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e61370 msgr2=0x55ec43e5bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efecd437640 1 --2- 172.21.15.172:0/480255213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e61370 0x55ec43e5bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efecd437640 1 -- 172.21.15.172:0/480255213 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec43edade0 con 0x55ec43e57b40 2024-09-10T08:52:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efecdc38640 1 --2- 172.21.15.172:0/480255213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e565d0 0x55ec43e5b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efecd437640 1 --2- 172.21.15.172:0/480255213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e5b6c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7efeb8000c00 tx=0x7efeb8010ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36af9f1fad321aa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efeb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efeb17fa640 1 -- 172.21.15.172:0/480255213 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb802c070 con 0x55ec43e57b40 2024-09-10T08:52:47.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efeb17fa640 1 -- 172.21.15.172:0/480255213 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7efeb80173f0 con 0x55ec43e57b40 2024-09-10T08:52:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.748+0000 7efed2275740 1 -- 172.21.15.172:0/480255213 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efec0054070 con 0x55ec43e57b40 2024-09-10T08:52:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.749+0000 7efeb17fa640 1 -- 172.21.15.172:0/480255213 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efeb8037050 con 0x55ec43e57b40 2024-09-10T08:52:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.749+0000 7efed2275740 10 client.17358 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.749+0000 7efed2275740 1 -- 172.21.15.172:0/480255213 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ec43edc0c0 con 0x55ec43e57b40 2024-09-10T08:52:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.749+0000 7efeb17fa640 1 -- 172.21.15.172:0/480255213 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efec0054070 con 0x55ec43e57b40 2024-09-10T08:52:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.749+0000 7efeb17fa640 1 -- 172.21.15.172:0/480255213 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7efeb801f040 con 0x55ec43e57b40 2024-09-10T08:52:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.749+0000 7efed2275740 10 client.17358 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.749+0000 7efed2275740 10 client.17358 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:52:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.749+0000 7efed2275740 20 client.17358 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.749+0000 7efed2275740 1 --2- 172.21.15.172:0/480255213 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55ec43edc870 0x55ec43edec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.749+0000 7efed2275740 4 client.17358 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:52:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.749+0000 7efed2275740 1 -- 172.21.15.172:0/480255213 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec43dc3d70 con 0x55ec43edc870 2024-09-10T08:52:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.749+0000 7efecdc38640 1 --2- 172.21.15.172:0/480255213 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55ec43edc870 0x55ec43edec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:47.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.750+0000 7efeb17fa640 10 client.17358 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:52:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.750+0000 7efecdc38640 1 --2- 172.21.15.172:0/480255213 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55ec43edc870 0x55ec43edec90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fdace2c9cc0ac2bc server_cookie=2e7230705bcc44f6 in_seq=0 out_seq=0 2024-09-10T08:52:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.750+0000 7efeb17fa640 1 -- 172.21.15.172:0/480255213 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec43dc3d70 con 0x55ec43edc870 2024-09-10T08:52:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.750+0000 7efeb17fa640 10 client.17358 handle_command_reply: tid=0 2024-09-10T08:52:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.750+0000 7efed2275740 1 client.17358 shutdown 2024-09-10T08:52:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.750+0000 7efed2275740 1 -- 172.21.15.172:0/480255213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 msgr2=0x55ec43e5b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.750+0000 7efed2275740 1 --2- 172.21.15.172:0/480255213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e5b6c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7efeb8000c00 tx=0x7efeb8010ea0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.750+0000 7efed2275740 1 -- 172.21.15.172:0/480255213 shutdown_connections 2024-09-10T08:52:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.750+0000 7efed2275740 1 --2- 172.21.15.172:0/480255213 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55ec43edc870 0x55ec43edec90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.750+0000 7efed2275740 1 --2- 172.21.15.172:0/480255213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec43e61370 0x55ec43e5bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.750+0000 7efeb17fa640 0 client.17358 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:52:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.750+0000 7efed2275740 1 --2- 172.21.15.172:0/480255213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ec43e57b40 0x55ec43e5b6c0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.750+0000 7efed2275740 1 --2- 172.21.15.172:0/480255213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ec43e565d0 0x55ec43e5b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.750+0000 7efed2275740 1 -- 172.21.15.172:0/480255213 >> 172.21.15.172:0/480255213 conn(0x55ec43e1f310 msgr2=0x55ec43e60050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:47.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.750+0000 7efed2275740 1 -- 172.21.15.172:0/480255213 shutdown_connections 2024-09-10T08:52:47.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.750+0000 7efed2275740 1 -- 172.21.15.172:0/480255213 wait complete. 2024-09-10T08:52:47.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.750+0000 7efed2275740 20 client.17358 trim_cache size 0 max 16384 2024-09-10T08:52:47.757 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:52:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.751+0000 7efecfe9c640 1 -- 172.21.15.172:0/1575594505 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efeb40a2b80 msgr2=0x7efeb40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.751+0000 7efecfe9c640 1 --2- 172.21.15.172:0/1575594505 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efeb40a2b80 0x7efeb40a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efec0048000 tx=0x7efec0049000 comp rx=0 tx=0).stop 2024-09-10T08:52:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.751+0000 7efecfe9c640 1 -- 172.21.15.172:0/1575594505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec8102d40 msgr2=0x7efec8198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.751+0000 7efecfe9c640 1 --2- 172.21.15.172:0/1575594505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec8102d40 0x7efec8198440 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7efec400b4a0 tx=0x7efec400b960 comp rx=0 tx=0).stop 2024-09-10T08:52:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.751+0000 7efecfe9c640 1 -- 172.21.15.172:0/1575594505 shutdown_connections 2024-09-10T08:52:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.751+0000 7efecfe9c640 1 --2- 172.21.15.172:0/1575594505 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efeb40a2b80 0x7efeb40a5040 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.751+0000 7efecfe9c640 1 --2- 172.21.15.172:0/1575594505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efec810c170 0x7efec819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.751+0000 7efecfe9c640 1 --2- 172.21.15.172:0/1575594505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efec8103660 0x7efec8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.751+0000 7efecfe9c640 1 --2- 172.21.15.172:0/1575594505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec8102d40 0x7efec8198440 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.751+0000 7efecfe9c640 1 -- 172.21.15.172:0/1575594505 >> 172.21.15.172:0/1575594505 conn(0x7efec80fc7a0 msgr2=0x7efec80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:47.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.751+0000 7efecfe9c640 1 -- 172.21.15.172:0/1575594505 shutdown_connections 2024-09-10T08:52:47.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:47.751+0000 7efecfe9c640 1 -- 172.21.15.172:0/1575594505 wait complete. 2024-09-10T08:52:48.509 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:52:48.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:48 smithi174 ceph-mon[34068]: pgmap v547: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-09-10T08:52:48.758 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:52:48.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:48 smithi191 ceph-mon[35314]: pgmap v547: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-09-10T08:52:48.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:48 smithi172 ceph-mon[30854]: pgmap v547: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-09-10T08:52:49.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.275+0000 7f23b48fe640 1 -- 172.21.15.172:0/2457380571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b0102d40 msgr2=0x7f23b0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:49.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.275+0000 7f23b48fe640 1 --2- 172.21.15.172:0/2457380571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b0102d40 0x7f23b0103120 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009900 tx=0x7f23a002a480 comp rx=0 tx=0).stop 2024-09-10T08:52:49.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.275+0000 7f23b48fe640 1 -- 172.21.15.172:0/2457380571 shutdown_connections 2024-09-10T08:52:49.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.275+0000 7f23b48fe640 1 --2- 172.21.15.172:0/2457380571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23b010c170 0x7f23b010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.275+0000 7f23b48fe640 1 --2- 172.21.15.172:0/2457380571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23b0103660 0x7f23b010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.275+0000 7f23b48fe640 1 --2- 172.21.15.172:0/2457380571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b0102d40 0x7f23b0103120 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.275+0000 7f23b48fe640 1 -- 172.21.15.172:0/2457380571 >> 172.21.15.172:0/2457380571 conn(0x7f23b00fc7a0 msgr2=0x7f23b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:49.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.275+0000 7f23b48fe640 1 -- 172.21.15.172:0/2457380571 shutdown_connections 2024-09-10T08:52:49.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.275+0000 7f23b48fe640 1 -- 172.21.15.172:0/2457380571 wait complete. 2024-09-10T08:52:49.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.276+0000 7f23b48fe640 1 Processor -- start 2024-09-10T08:52:49.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.276+0000 7f23b48fe640 1 -- start start 2024-09-10T08:52:49.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.276+0000 7f23b48fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23b0102d40 0x7f23b01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.276+0000 7f23b48fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b0103660 0x7f23b0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.276+0000 7f23b48fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23b010c170 0x7f23b019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.276+0000 7f23b48fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23b0111ee0 con 0x7f23b0103660 2024-09-10T08:52:49.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.276+0000 7f23b48fe640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23b0111d60 con 0x7f23b010c170 2024-09-10T08:52:49.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.276+0000 7f23b48fe640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f23b0112060 con 0x7f23b0102d40 2024-09-10T08:52:49.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.276+0000 7f23a7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b0103660 0x7f23b0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:49.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.276+0000 7f23a7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b0103660 0x7f23b0198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44664/0 (socket says 172.21.15.172:44664) 2024-09-10T08:52:49.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.277+0000 7f23a7fff640 1 -- 172.21.15.172:0/83764453 learned_addr learned my addr 172.21.15.172:0/83764453 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:49.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.277+0000 7f23ae59c640 1 --2- 172.21.15.172:0/83764453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23b0102d40 0x7f23b01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:49.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.277+0000 7f23aed9d640 1 --2- 172.21.15.172:0/83764453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23b010c170 0x7f23b019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:49.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.277+0000 7f23a7fff640 1 -- 172.21.15.172:0/83764453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23b0102d40 msgr2=0x7f23b01984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:49.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.277+0000 7f23a7fff640 1 --2- 172.21.15.172:0/83764453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23b0102d40 0x7f23b01984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.277+0000 7f23a7fff640 1 -- 172.21.15.172:0/83764453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23b010c170 msgr2=0x7f23b019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:49.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.277+0000 7f23a7fff640 1 --2- 172.21.15.172:0/83764453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23b010c170 0x7f23b019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.277+0000 7f23a7fff640 1 -- 172.21.15.172:0/83764453 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b019d490 con 0x7f23b0103660 2024-09-10T08:52:49.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.277+0000 7f23a7fff640 1 --2- 172.21.15.172:0/83764453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b0103660 0x7f23b0198a20 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f239800b3f0 tx=0x7f239800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:49.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.277+0000 7f23a77fe640 1 -- 172.21.15.172:0/83764453 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398015020 con 0x7f23b0103660 2024-09-10T08:52:49.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.277+0000 7f23a77fe640 1 -- 172.21.15.172:0/83764453 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2398011ce0 con 0x7f23b0103660 2024-09-10T08:52:49.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.277+0000 7f23b48fe640 1 -- 172.21.15.172:0/83764453 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b019d780 con 0x7f23b0103660 2024-09-10T08:52:49.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.277+0000 7f23a77fe640 1 -- 172.21.15.172:0/83764453 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398010d80 con 0x7f23b0103660 2024-09-10T08:52:49.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.277+0000 7f23b48fe640 1 -- 172.21.15.172:0/83764453 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b01a5060 con 0x7f23b0103660 2024-09-10T08:52:49.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.279+0000 7f23a77fe640 1 -- 172.21.15.172:0/83764453 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2398018020 con 0x7f23b0103660 2024-09-10T08:52:49.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.280+0000 7f23b6cd7740 1 Processor -- start 2024-09-10T08:52:49.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.280+0000 7f23b6cd7740 1 -- start start 2024-09-10T08:52:49.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.280+0000 7f23b6cd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c5bb40 0x55a281c5a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.280+0000 7f23b6cd7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c5a5d0 0x55a281c5aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.280+0000 7f23b6cd7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c65370 0x55a281c67760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.280+0000 7f23b6cd7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a281bc02f0 con 0x55a281c5a5d0 2024-09-10T08:52:49.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.280+0000 7f23b6cd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a281b67cf0 con 0x55a281c5bb40 2024-09-10T08:52:49.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.280+0000 7f23b6cd7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a281bbc2b0 con 0x55a281c65370 2024-09-10T08:52:49.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.280+0000 7f23a77fe640 1 --2- 172.21.15.172:0/83764453 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f239c0a2b80 0x7f239c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.280+0000 7f23ae59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c5a5d0 0x55a281c5aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:49.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.280+0000 7f23ae59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c5a5d0 0x55a281c5aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44672/0 (socket says 172.21.15.172:44672) 2024-09-10T08:52:49.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.280+0000 7f23ae59c640 1 -- 172.21.15.172:0/2817224985 learned_addr learned my addr 172.21.15.172:0/2817224985 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:49.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.281+0000 7f23aed9d640 1 --2- 172.21.15.172:0/2817224985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c5bb40 0x55a281c5a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:49.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.281+0000 7f23a7fff640 1 --2- 172.21.15.172:0/2817224985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c65370 0x55a281c67760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:49.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.281+0000 7f23aed9d640 1 --2- 172.21.15.172:0/83764453 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f239c0a2b80 0x7f239c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:49.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.281+0000 7f23a77fe640 1 -- 172.21.15.172:0/83764453 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f23980cebb0 con 0x7f23b0103660 2024-09-10T08:52:49.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.281+0000 7f23ae59c640 1 -- 172.21.15.172:0/2817224985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c65370 msgr2=0x55a281c67760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:49.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.281+0000 7f23ae59c640 1 --2- 172.21.15.172:0/2817224985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c65370 0x55a281c67760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.281+0000 7f23ae59c640 1 -- 172.21.15.172:0/2817224985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c5bb40 msgr2=0x55a281c5a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:49.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.281+0000 7f23ae59c640 1 --2- 172.21.15.172:0/2817224985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c5bb40 0x55a281c5a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.281+0000 7f23ae59c640 1 -- 172.21.15.172:0/2817224985 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a281b4c3a0 con 0x55a281c5a5d0 2024-09-10T08:52:49.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.281+0000 7f23a7fff640 1 --2- 172.21.15.172:0/2817224985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c65370 0x55a281c67760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:49.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.281+0000 7f23aed9d640 1 --2- 172.21.15.172:0/83764453 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f239c0a2b80 0x7f239c0a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f23a800bd40 tx=0x7f23a8009be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:49.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.281+0000 7f23ae59c640 1 --2- 172.21.15.172:0/2817224985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c5a5d0 0x55a281c5aa10 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009900 tx=0x7f23a0031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca2370bc91179ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:49.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.282+0000 7f23a4ff9640 1 -- 172.21.15.172:0/2817224985 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a0040070 con 0x55a281c5a5d0 2024-09-10T08:52:49.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.282+0000 7f23a4ff9640 1 -- 172.21.15.172:0/2817224985 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23a00316a0 con 0x55a281c5a5d0 2024-09-10T08:52:49.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.282+0000 7f23a4ff9640 1 -- 172.21.15.172:0/2817224985 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a0035040 con 0x55a281c5a5d0 2024-09-10T08:52:49.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.282+0000 7f23b6cd7740 1 -- 172.21.15.172:0/2817224985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c5a5d0 msgr2=0x55a281c5aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:49.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.282+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/2817224985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c5a5d0 0x55a281c5aa10 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009900 tx=0x7f23a0031dc0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.282+0000 7f23b6cd7740 1 -- 172.21.15.172:0/2817224985 shutdown_connections 2024-09-10T08:52:49.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.282+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/2817224985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c65370 0x55a281c67760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.282+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/2817224985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c5a5d0 0x55a281c5aa10 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.282+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/2817224985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c5bb40 0x55a281c5a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.282+0000 7f23b6cd7740 1 -- 172.21.15.172:0/2817224985 >> 172.21.15.172:0/2817224985 conn(0x55a281c23310 msgr2=0x55a281c56c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:49.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.282+0000 7f23b6cd7740 1 -- 172.21.15.172:0/2817224985 shutdown_connections 2024-09-10T08:52:49.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.282+0000 7f23b6cd7740 1 -- 172.21.15.172:0/2817224985 wait complete. 2024-09-10T08:52:49.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23b6cd7740 1 -- start start 2024-09-10T08:52:49.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23b6cd7740 1 Processor -- start 2024-09-10T08:52:49.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23b6cd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c5a5d0 0x55a281c64d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23b6cd7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c5bb40 0x55a281c62000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23b6cd7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c65370 0x55a281c625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23b6cd7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a281c68010 con 0x55a281c65370 2024-09-10T08:52:49.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23b6cd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a281b7b060 con 0x55a281c5a5d0 2024-09-10T08:52:49.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23b6cd7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a281c68190 con 0x55a281c5bb40 2024-09-10T08:52:49.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23b6cd7740 10 client.0 _ll_register_callbacks cb 0x55a2818ce8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:49.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23aed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c65370 0x55a281c625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:49.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23aed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c65370 0x55a281c625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44684/0 (socket says 172.21.15.172:44684) 2024-09-10T08:52:49.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23aed9d640 1 -- 172.21.15.172:0/2619944407 learned_addr learned my addr 172.21.15.172:0/2619944407 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:49.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23a7fff640 1 --2- 172.21.15.172:0/2619944407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c5bb40 0x55a281c62000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:49.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23ae59c640 1 --2- 172.21.15.172:0/2619944407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c5a5d0 0x55a281c64d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:49.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23aed9d640 1 -- 172.21.15.172:0/2619944407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c5bb40 msgr2=0x55a281c62000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:49.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23aed9d640 1 --2- 172.21.15.172:0/2619944407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c5bb40 0x55a281c62000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23aed9d640 1 -- 172.21.15.172:0/2619944407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c5a5d0 msgr2=0x55a281c64d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:49.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23aed9d640 1 --2- 172.21.15.172:0/2619944407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c5a5d0 0x55a281c64d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.283+0000 7f23aed9d640 1 -- 172.21.15.172:0/2619944407 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2819313c0 con 0x55a281c65370 2024-09-10T08:52:49.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.284+0000 7f23aed9d640 1 --2- 172.21.15.172:0/2619944407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c65370 0x55a281c625d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f23a800bfa0 tx=0x7f23a801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7379c4feb102ebc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:49.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.284+0000 7f2395ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:49.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.284+0000 7f2395ffb640 1 -- 172.21.15.172:0/2619944407 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8014040 con 0x55a281c65370 2024-09-10T08:52:49.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.284+0000 7f2395ffb640 1 -- 172.21.15.172:0/2619944407 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f23a8024600 con 0x55a281c65370 2024-09-10T08:52:49.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.284+0000 7f23b6cd7740 1 -- 172.21.15.172:0/2619944407 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23a00316a0 con 0x55a281c65370 2024-09-10T08:52:49.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.284+0000 7f2395ffb640 1 -- 172.21.15.172:0/2619944407 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f23a8032050 con 0x55a281c65370 2024-09-10T08:52:49.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.284+0000 7f23b6cd7740 10 client.17370 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:49.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.284+0000 7f23b6cd7740 1 -- 172.21.15.172:0/2619944407 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a281c73a00 con 0x55a281c65370 2024-09-10T08:52:49.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.284+0000 7f2395ffb640 1 -- 172.21.15.172:0/2619944407 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23a00316a0 con 0x55a281c65370 2024-09-10T08:52:49.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.284+0000 7f2395ffb640 1 -- 172.21.15.172:0/2619944407 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f23a802d5d0 con 0x55a281c65370 2024-09-10T08:52:49.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.284+0000 7f23b6cd7740 10 client.17370 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:49.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.284+0000 7f23b6cd7740 10 client.17370 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:52:49.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.284+0000 7f23b6cd7740 20 client.17370 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:49.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.284+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/2619944407 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a281c74ec0 0x55a281c77360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.284+0000 7f23b6cd7740 4 client.17370 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:49.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.284+0000 7f23b6cd7740 1 -- 172.21.15.172:0/2619944407 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a281bcd860 con 0x55a281c74ec0 2024-09-10T08:52:49.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.285+0000 7f23ae59c640 1 --2- 172.21.15.172:0/2619944407 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a281c74ec0 0x55a281c77360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:49.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.285+0000 7f2395ffb640 10 client.17370 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:52:49.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.285+0000 7f23ae59c640 1 --2- 172.21.15.172:0/2619944407 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a281c74ec0 0x55a281c77360 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=24ee02dcf8449410 server_cookie=29e77c0591621d6 in_seq=0 out_seq=0 2024-09-10T08:52:49.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.286+0000 7f2395ffb640 1 -- 172.21.15.172:0/2619944407 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a281bcd860 con 0x55a281c74ec0 2024-09-10T08:52:49.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.286+0000 7f2395ffb640 10 client.17370 handle_command_reply: tid=0 2024-09-10T08:52:49.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.286+0000 7f23b6cd7740 1 client.17370 shutdown 2024-09-10T08:52:49.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.286+0000 7f23b6cd7740 1 -- 172.21.15.172:0/2619944407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c65370 msgr2=0x55a281c625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:49.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.286+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/2619944407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c65370 0x55a281c625d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f23a800bfa0 tx=0x7f23a801a000 comp rx=0 tx=0).stop 2024-09-10T08:52:49.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.286+0000 7f23b6cd7740 1 -- 172.21.15.172:0/2619944407 shutdown_connections 2024-09-10T08:52:49.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.286+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/2619944407 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a281c74ec0 0x55a281c77360 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.286+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/2619944407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c65370 0x55a281c625d0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.286+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/2619944407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c5bb40 0x55a281c62000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.286+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/2619944407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c5a5d0 0x55a281c64d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.286+0000 7f2395ffb640 0 client.17370 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:52:49.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.286+0000 7f23b6cd7740 1 -- 172.21.15.172:0/2619944407 >> 172.21.15.172:0/2619944407 conn(0x55a281c23310 msgr2=0x55a281c5c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:49.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.286+0000 7f23b6cd7740 1 -- 172.21.15.172:0/2619944407 shutdown_connections 2024-09-10T08:52:49.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.286+0000 7f23b6cd7740 1 -- 172.21.15.172:0/2619944407 wait complete. 2024-09-10T08:52:49.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.286+0000 7f23b6cd7740 20 client.17370 trim_cache size 0 max 16384 2024-09-10T08:52:49.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.299+0000 7f23b6cd7740 1 Processor -- start 2024-09-10T08:52:49.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.299+0000 7f23b6cd7740 1 -- start start 2024-09-10T08:52:49.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.299+0000 7f23b6cd7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c5a5d0 0x55a281c56e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.299+0000 7f23b6cd7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c5bb40 0x55a281c712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.299+0000 7f23b6cd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c65370 0x55a281c5d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.299+0000 7f23b6cd7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23b01aebe0 con 0x55a281c5a5d0 2024-09-10T08:52:49.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.299+0000 7f23b6cd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23b01b0720 con 0x55a281c65370 2024-09-10T08:52:49.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.299+0000 7f23b6cd7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f23b01af2b0 con 0x55a281c5bb40 2024-09-10T08:52:49.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.299+0000 7f23ae59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c5a5d0 0x55a281c56e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:49.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.299+0000 7f23ae59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c5a5d0 0x55a281c56e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44694/0 (socket says 172.21.15.172:44694) 2024-09-10T08:52:49.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.299+0000 7f23ae59c640 1 -- 172.21.15.172:0/1631625769 learned_addr learned my addr 172.21.15.172:0/1631625769 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:49.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.299+0000 7f23ae59c640 1 -- 172.21.15.172:0/1631625769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c5bb40 msgr2=0x55a281c712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:49.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.299+0000 7f23ae59c640 1 --2- 172.21.15.172:0/1631625769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c5bb40 0x55a281c712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.300+0000 7f23ae59c640 1 -- 172.21.15.172:0/1631625769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c65370 msgr2=0x55a281c5d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:49.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.300+0000 7f23ae59c640 1 --2- 172.21.15.172:0/1631625769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c65370 0x55a281c5d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.300+0000 7f23ae59c640 1 -- 172.21.15.172:0/1631625769 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a281c60550 con 0x55a281c5a5d0 2024-09-10T08:52:49.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.300+0000 7f23ae59c640 1 --2- 172.21.15.172:0/1631625769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c5a5d0 0x55a281c56e10 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x55a281c57ac0 tx=0x7f23a0003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe8a4dc85a09785 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:49.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.300+0000 7f2397fff640 1 -- 172.21.15.172:0/1631625769 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a0035040 con 0x55a281c5a5d0 2024-09-10T08:52:49.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.300+0000 7f2397fff640 1 -- 172.21.15.172:0/1631625769 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23a00054d0 con 0x55a281c5a5d0 2024-09-10T08:52:49.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.300+0000 7f2397fff640 1 -- 172.21.15.172:0/1631625769 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a0040040 con 0x55a281c5a5d0 2024-09-10T08:52:49.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.300+0000 7f23b6cd7740 1 -- 172.21.15.172:0/1631625769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c5a5d0 msgr2=0x55a281c56e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:49.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.300+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/1631625769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c5a5d0 0x55a281c56e10 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x55a281c57ac0 tx=0x7f23a0003880 comp rx=0 tx=0).stop 2024-09-10T08:52:49.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23b6cd7740 1 -- 172.21.15.172:0/1631625769 shutdown_connections 2024-09-10T08:52:49.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/1631625769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c65370 0x55a281c5d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/1631625769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c5bb40 0x55a281c712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/1631625769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c5a5d0 0x55a281c56e10 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23b6cd7740 1 -- 172.21.15.172:0/1631625769 >> 172.21.15.172:0/1631625769 conn(0x55a281c23310 msgr2=0x55a281c56700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:49.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23b6cd7740 1 -- 172.21.15.172:0/1631625769 shutdown_connections 2024-09-10T08:52:49.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23b6cd7740 1 -- 172.21.15.172:0/1631625769 wait complete. 2024-09-10T08:52:49.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23b6cd7740 1 -- start start 2024-09-10T08:52:49.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23b6cd7740 1 Processor -- start 2024-09-10T08:52:49.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23b6cd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c5a5d0 0x55a281c6d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23b6cd7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c5bb40 0x55a281c6dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23b6cd7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c65370 0x55a281c68b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23b6cd7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23b01b0050 con 0x55a281c65370 2024-09-10T08:52:49.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23b6cd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23b01af980 con 0x55a281c5a5d0 2024-09-10T08:52:49.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23b6cd7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f23b01acbc0 con 0x55a281c5bb40 2024-09-10T08:52:49.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23b6cd7740 10 client.0 _ll_register_callbacks cb 0x7f23a80099d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:49.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23aed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c65370 0x55a281c68b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:49.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23aed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c65370 0x55a281c68b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44706/0 (socket says 172.21.15.172:44706) 2024-09-10T08:52:49.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.301+0000 7f23aed9d640 1 -- 172.21.15.172:0/758774540 learned_addr learned my addr 172.21.15.172:0/758774540 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:49.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.302+0000 7f23ae59c640 1 --2- 172.21.15.172:0/758774540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c5a5d0 0x55a281c6d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:49.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.302+0000 7f23a7fff640 1 --2- 172.21.15.172:0/758774540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c5bb40 0x55a281c6dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:49.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.302+0000 7f23aed9d640 1 -- 172.21.15.172:0/758774540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c5bb40 msgr2=0x55a281c6dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:49.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.302+0000 7f23aed9d640 1 --2- 172.21.15.172:0/758774540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c5bb40 0x55a281c6dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.302+0000 7f23aed9d640 1 -- 172.21.15.172:0/758774540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c5a5d0 msgr2=0x55a281c6d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:49.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.302+0000 7f23aed9d640 1 --2- 172.21.15.172:0/758774540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c5a5d0 0x55a281c6d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.302+0000 7f23aed9d640 1 -- 172.21.15.172:0/758774540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a281c69350 con 0x55a281c65370 2024-09-10T08:52:49.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.302+0000 7f23aed9d640 1 --2- 172.21.15.172:0/758774540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c65370 0x55a281c68b50 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f23a802d5c0 tx=0x7f23a802d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3394efcc22ab9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:49.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.302+0000 7f2395ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:49.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.302+0000 7f2395ffb640 1 -- 172.21.15.172:0/758774540 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a803d060 con 0x55a281c65370 2024-09-10T08:52:49.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.302+0000 7f2395ffb640 1 -- 172.21.15.172:0/758774540 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f23a8036840 con 0x55a281c65370 2024-09-10T08:52:49.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.302+0000 7f23b6cd7740 1 -- 172.21.15.172:0/758774540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23a00054d0 con 0x55a281c65370 2024-09-10T08:52:49.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.302+0000 7f2395ffb640 1 -- 172.21.15.172:0/758774540 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f23a8008e90 con 0x55a281c65370 2024-09-10T08:52:49.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.302+0000 7f23b6cd7740 10 client.17376 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:49.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.302+0000 7f23b6cd7740 1 -- 172.21.15.172:0/758774540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a281ce8000 con 0x55a281c65370 2024-09-10T08:52:49.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.302+0000 7f2395ffb640 1 -- 172.21.15.172:0/758774540 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23a00054d0 con 0x55a281c65370 2024-09-10T08:52:49.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.303+0000 7f2395ffb640 1 -- 172.21.15.172:0/758774540 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f23a80385d0 con 0x55a281c65370 2024-09-10T08:52:49.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.303+0000 7f23b6cd7740 10 client.17376 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:49.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.303+0000 7f23b6cd7740 10 client.17376 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:52:49.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.303+0000 7f23b6cd7740 20 client.17376 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:49.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.303+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/758774540 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a281ce8720 0x55a281ceab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:49.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.303+0000 7f23b6cd7740 4 client.17376 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:52:49.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.303+0000 7f23b6cd7740 1 -- 172.21.15.172:0/758774540 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a281bc7d70 con 0x55a281ce8720 2024-09-10T08:52:49.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.303+0000 7f23ae59c640 1 --2- 172.21.15.172:0/758774540 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a281ce8720 0x55a281ceab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:49.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.303+0000 7f2395ffb640 10 client.17376 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:52:49.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.303+0000 7f23ae59c640 1 --2- 172.21.15.172:0/758774540 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a281ce8720 0x55a281ceab40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8b37467e870f7669 server_cookie=3d5ef413d4598c08 in_seq=0 out_seq=0 2024-09-10T08:52:49.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.303+0000 7f2395ffb640 1 -- 172.21.15.172:0/758774540 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a281bc7d70 con 0x55a281ce8720 2024-09-10T08:52:49.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.303+0000 7f2395ffb640 10 client.17376 handle_command_reply: tid=0 2024-09-10T08:52:49.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.303+0000 7f23b6cd7740 1 client.17376 shutdown 2024-09-10T08:52:49.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.303+0000 7f23b6cd7740 1 -- 172.21.15.172:0/758774540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c65370 msgr2=0x55a281c68b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:49.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.303+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/758774540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c65370 0x55a281c68b50 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f23a802d5c0 tx=0x7f23a802d5f0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b6cd7740 1 -- 172.21.15.172:0/758774540 shutdown_connections 2024-09-10T08:52:49.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/758774540 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a281ce8720 0x55a281ceab40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/758774540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a281c65370 0x55a281c68b50 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/758774540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a281c5bb40 0x55a281c6dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b6cd7740 1 --2- 172.21.15.172:0/758774540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a281c5a5d0 0x55a281c6d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b6cd7740 1 -- 172.21.15.172:0/758774540 >> 172.21.15.172:0/758774540 conn(0x55a281c23310 msgr2=0x55a281c554e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:49.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f2395ffb640 0 client.17376 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:52:49.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b6cd7740 1 -- 172.21.15.172:0/758774540 shutdown_connections 2024-09-10T08:52:49.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b6cd7740 1 -- 172.21.15.172:0/758774540 wait complete. 2024-09-10T08:52:49.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b6cd7740 20 client.17376 trim_cache size 0 max 16384 2024-09-10T08:52:49.319 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:52:49.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b48fe640 1 -- 172.21.15.172:0/83764453 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f239c0a2b80 msgr2=0x7f239c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:49.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b48fe640 1 --2- 172.21.15.172:0/83764453 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f239c0a2b80 0x7f239c0a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f23a800bd40 tx=0x7f23a8009be0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b48fe640 1 -- 172.21.15.172:0/83764453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b0103660 msgr2=0x7f23b0198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:49.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b48fe640 1 --2- 172.21.15.172:0/83764453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b0103660 0x7f23b0198a20 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f239800b3f0 tx=0x7f239800b8b0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b48fe640 1 -- 172.21.15.172:0/83764453 shutdown_connections 2024-09-10T08:52:49.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b48fe640 1 --2- 172.21.15.172:0/83764453 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f239c0a2b80 0x7f239c0a5040 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b48fe640 1 --2- 172.21.15.172:0/83764453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23b010c170 0x7f23b019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b48fe640 1 --2- 172.21.15.172:0/83764453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23b0103660 0x7f23b0198a20 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b48fe640 1 --2- 172.21.15.172:0/83764453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23b0102d40 0x7f23b01984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:49.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b48fe640 1 -- 172.21.15.172:0/83764453 >> 172.21.15.172:0/83764453 conn(0x7f23b00fc7a0 msgr2=0x7f23b00fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:49.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b48fe640 1 -- 172.21.15.172:0/83764453 shutdown_connections 2024-09-10T08:52:49.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:49.304+0000 7f23b48fe640 1 -- 172.21.15.172:0/83764453 wait complete. 2024-09-10T08:52:49.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:49 smithi172 ceph-mon[30854]: pgmap v548: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 20 op/s 2024-09-10T08:52:49.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:49 smithi174 ceph-mon[34068]: pgmap v548: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 20 op/s 2024-09-10T08:52:49.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:49 smithi191 ceph-mon[35314]: pgmap v548: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 20 op/s 2024-09-10T08:52:50.063 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:52:50.310 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:52:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.777+0000 7fccefcef640 1 -- 172.21.15.172:0/3062845508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcce8106f50 msgr2=0x7fcce810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.777+0000 7fccefcef640 1 --2- 172.21.15.172:0/3062845508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcce8106f50 0x7fcce810eaf0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009cd0 tx=0x7fcce002a8e0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.777+0000 7fccefcef640 1 -- 172.21.15.172:0/3062845508 shutdown_connections 2024-09-10T08:52:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.777+0000 7fccefcef640 1 --2- 172.21.15.172:0/3062845508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcce8106f50 0x7fcce810eaf0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.777+0000 7fccefcef640 1 --2- 172.21.15.172:0/3062845508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcce8100ca0 0x7fcce8106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.777+0000 7fccefcef640 1 --2- 172.21.15.172:0/3062845508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce81002f0 0x7fcce81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.777+0000 7fccefcef640 1 -- 172.21.15.172:0/3062845508 >> 172.21.15.172:0/3062845508 conn(0x7fcce8077210 msgr2=0x7fcce80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.777+0000 7fccefcef640 1 -- 172.21.15.172:0/3062845508 shutdown_connections 2024-09-10T08:52:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.777+0000 7fccefcef640 1 -- 172.21.15.172:0/3062845508 wait complete. 2024-09-10T08:52:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.777+0000 7fccefcef640 1 Processor -- start 2024-09-10T08:52:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.777+0000 7fccefcef640 1 -- start start 2024-09-10T08:52:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fccefcef640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcce81002f0 0x7fcce8101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fccefcef640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcce8100ca0 0x7fcce81017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fccefcef640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce8106f50 0x7fcce8101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fccefcef640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8111f30 con 0x7fcce81002f0 2024-09-10T08:52:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fccefcef640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8111db0 con 0x7fcce8106f50 2024-09-10T08:52:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fccefcef640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81120b0 con 0x7fcce8100ca0 2024-09-10T08:52:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fcceda8b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcce81002f0 0x7fcce8101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fcceda8b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcce81002f0 0x7fcce8101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44730/0 (socket says 172.21.15.172:44730) 2024-09-10T08:52:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fcceda8b640 1 -- 172.21.15.172:0/1752915888 learned_addr learned my addr 172.21.15.172:0/1752915888 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fcced28a640 1 --2- 172.21.15.172:0/1752915888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcce8100ca0 0x7fcce81017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fccee28c640 1 --2- 172.21.15.172:0/1752915888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce8106f50 0x7fcce8101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fcceda8b640 1 -- 172.21.15.172:0/1752915888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcce8100ca0 msgr2=0x7fcce81017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fcceda8b640 1 --2- 172.21.15.172:0/1752915888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcce8100ca0 0x7fcce81017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fcceda8b640 1 -- 172.21.15.172:0/1752915888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce8106f50 msgr2=0x7fcce8101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fcceda8b640 1 --2- 172.21.15.172:0/1752915888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce8106f50 0x7fcce8101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fcceda8b640 1 -- 172.21.15.172:0/1752915888 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce81a6a50 con 0x7fcce81002f0 2024-09-10T08:52:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fccee28c640 1 --2- 172.21.15.172:0/1752915888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce8106f50 0x7fcce8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fcceda8b640 1 --2- 172.21.15.172:0/1752915888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcce81002f0 0x7fcce8101270 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00b4a0 tx=0x7fccdc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fccdaffd640 1 -- 172.21.15.172:0/1752915888 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc014070 con 0x7fcce81002f0 2024-09-10T08:52:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fccdaffd640 1 -- 172.21.15.172:0/1752915888 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fccdc010ac0 con 0x7fcce81002f0 2024-09-10T08:52:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fccdaffd640 1 -- 172.21.15.172:0/1752915888 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc017c30 con 0x7fcce81002f0 2024-09-10T08:52:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fccefcef640 1 -- 172.21.15.172:0/1752915888 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce81a6d40 con 0x7fcce81002f0 2024-09-10T08:52:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.778+0000 7fccefcef640 1 -- 172.21.15.172:0/1752915888 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce81a7270 con 0x7fcce81002f0 2024-09-10T08:52:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.781+0000 7fccdaffd640 1 -- 172.21.15.172:0/1752915888 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fccdc017070 con 0x7fcce81002f0 2024-09-10T08:52:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.781+0000 7fccf20c8740 1 Processor -- start 2024-09-10T08:52:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.781+0000 7fccf20c8740 1 -- start start 2024-09-10T08:52:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.781+0000 7fccf20c8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56211746db40 0x56211746c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.781+0000 7fccf20c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746c5d0 0x56211746ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.781+0000 7fccf20c8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x562117479760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.781+0000 7fccf20c8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5621173d22f0 con 0x562117477370 2024-09-10T08:52:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.781+0000 7fccf20c8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562117379cf0 con 0x56211746db40 2024-09-10T08:52:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.781+0000 7fccf20c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5621173ce2b0 con 0x56211746c5d0 2024-09-10T08:52:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fcced28a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x562117479760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fcceda8b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746c5d0 0x56211746ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fcced28a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x562117479760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44738/0 (socket says 172.21.15.172:44738) 2024-09-10T08:52:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fcced28a640 1 -- 172.21.15.172:0/2681164768 learned_addr learned my addr 172.21.15.172:0/2681164768 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fccee28c640 1 --2- 172.21.15.172:0/2681164768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56211746db40 0x56211746c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fccdaffd640 1 --2- 172.21.15.172:0/1752915888 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccd40a2c00 0x7fccd40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fccee28c640 1 --2- 172.21.15.172:0/1752915888 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccd40a2c00 0x7fccd40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fcced28a640 1 -- 172.21.15.172:0/2681164768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746c5d0 msgr2=0x56211746ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fcced28a640 1 --2- 172.21.15.172:0/2681164768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746c5d0 0x56211746ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fcced28a640 1 -- 172.21.15.172:0/2681164768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56211746db40 msgr2=0x56211746c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fcced28a640 1 --2- 172.21.15.172:0/2681164768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56211746db40 0x56211746c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fcced28a640 1 -- 172.21.15.172:0/2681164768 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56211735e3a0 con 0x562117477370 2024-09-10T08:52:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fccdaffd640 1 -- 172.21.15.172:0/1752915888 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fccdc0cea80 con 0x7fcce81002f0 2024-09-10T08:52:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fcced28a640 1 --2- 172.21.15.172:0/2681164768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x562117479760 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fcce400b7c0 tx=0x7fcce400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a462010faae5484 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fccd3fff640 1 -- 172.21.15.172:0/2681164768 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce400d950 con 0x562117477370 2024-09-10T08:52:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fccd3fff640 1 -- 172.21.15.172:0/2681164768 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcce4018070 con 0x562117477370 2024-09-10T08:52:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fccd3fff640 1 -- 172.21.15.172:0/2681164768 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce40123e0 con 0x562117477370 2024-09-10T08:52:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.782+0000 7fccee28c640 1 --2- 172.21.15.172:0/1752915888 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccd40a2c00 0x7fccd40a50c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009a60 tx=0x7fcce002a8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.783+0000 7fccf20c8740 1 -- 172.21.15.172:0/2681164768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 msgr2=0x562117479760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.783+0000 7fccf20c8740 1 --2- 172.21.15.172:0/2681164768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x562117479760 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fcce400b7c0 tx=0x7fcce400bc80 comp rx=0 tx=0).stop 2024-09-10T08:52:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.783+0000 7fccf20c8740 1 -- 172.21.15.172:0/2681164768 shutdown_connections 2024-09-10T08:52:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.783+0000 7fccf20c8740 1 --2- 172.21.15.172:0/2681164768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x562117479760 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.783+0000 7fccf20c8740 1 --2- 172.21.15.172:0/2681164768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746c5d0 0x56211746ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.783+0000 7fccf20c8740 1 --2- 172.21.15.172:0/2681164768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56211746db40 0x56211746c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.783+0000 7fccf20c8740 1 -- 172.21.15.172:0/2681164768 >> 172.21.15.172:0/2681164768 conn(0x562117435310 msgr2=0x562117468c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.783+0000 7fccf20c8740 1 -- 172.21.15.172:0/2681164768 shutdown_connections 2024-09-10T08:52:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.783+0000 7fccf20c8740 1 -- 172.21.15.172:0/2681164768 wait complete. 2024-09-10T08:52:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.784+0000 7fccf20c8740 1 -- start start 2024-09-10T08:52:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.784+0000 7fccf20c8740 1 Processor -- start 2024-09-10T08:52:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.784+0000 7fccf20c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746c5d0 0x562117476cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.784+0000 7fccf20c8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56211746db40 0x562117473f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.784+0000 7fccf20c8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x5621174744d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.784+0000 7fccf20c8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56211747a070 con 0x562117477370 2024-09-10T08:52:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.784+0000 7fccf20c8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56211738d060 con 0x56211746db40 2024-09-10T08:52:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.784+0000 7fccf20c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56211747a1f0 con 0x56211746c5d0 2024-09-10T08:52:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.784+0000 7fccf20c8740 10 client.0 _ll_register_callbacks cb 0x5621170e08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.784+0000 7fccee28c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x5621174744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.784+0000 7fccee28c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x5621174744d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44744/0 (socket says 172.21.15.172:44744) 2024-09-10T08:52:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.784+0000 7fccee28c640 1 -- 172.21.15.172:0/2824687568 learned_addr learned my addr 172.21.15.172:0/2824687568 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.784+0000 7fcceda8b640 1 --2- 172.21.15.172:0/2824687568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746c5d0 0x562117476cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.784+0000 7fccee28c640 1 -- 172.21.15.172:0/2824687568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746c5d0 msgr2=0x562117476cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.784+0000 7fccee28c640 1 --2- 172.21.15.172:0/2824687568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746c5d0 0x562117476cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.784+0000 7fccee28c640 1 -- 172.21.15.172:0/2824687568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56211746db40 msgr2=0x562117473f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccee28c640 1 --2- 172.21.15.172:0/2824687568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56211746db40 0x562117473f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccee28c640 1 -- 172.21.15.172:0/2824687568 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621171433c0 con 0x562117477370 2024-09-10T08:52:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccee28c640 1 --2- 172.21.15.172:0/2824687568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x5621174744d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fcce0003e50 tx=0x7fcce00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1037d0093fcba1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccd17fa640 1 -- 172.21.15.172:0/2824687568 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0004810 con 0x562117477370 2024-09-10T08:52:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccd17fa640 1 -- 172.21.15.172:0/2824687568 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcce00307b0 con 0x562117477370 2024-09-10T08:52:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccf20c8740 1 -- 172.21.15.172:0/2824687568 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce4018070 con 0x562117477370 2024-09-10T08:52:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccd17fa640 1 -- 172.21.15.172:0/2824687568 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcce0058050 con 0x562117477370 2024-09-10T08:52:50.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccf20c8740 10 client.17388 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:50.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccf20c8740 1 -- 172.21.15.172:0/2824687568 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562117485a80 con 0x562117477370 2024-09-10T08:52:50.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccd17fa640 1 -- 172.21.15.172:0/2824687568 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcce4018070 con 0x562117477370 2024-09-10T08:52:50.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccd17fa640 1 -- 172.21.15.172:0/2824687568 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fcce0030070 con 0x562117477370 2024-09-10T08:52:50.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccf20c8740 10 client.17388 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:50.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccf20c8740 10 client.17388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:50.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccf20c8740 20 client.17388 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:50.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccf20c8740 1 --2- 172.21.15.172:0/2824687568 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562117486fe0 0x5621174894a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccf20c8740 4 client.17388 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:50.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.785+0000 7fccf20c8740 1 -- 172.21.15.172:0/2824687568 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621173df860 con 0x562117486fe0 2024-09-10T08:52:50.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.786+0000 7fcceda8b640 1 --2- 172.21.15.172:0/2824687568 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562117486fe0 0x5621174894a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:50.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.786+0000 7fcceda8b640 1 --2- 172.21.15.172:0/2824687568 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562117486fe0 0x5621174894a0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9465771ad17ba5 server_cookie=77a409e4867c8c84 in_seq=0 out_seq=0 2024-09-10T08:52:50.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.786+0000 7fccd17fa640 10 client.17388 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:50.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.787+0000 7fccd17fa640 1 -- 172.21.15.172:0/2824687568 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621173df860 con 0x562117486fe0 2024-09-10T08:52:50.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.787+0000 7fccd17fa640 10 client.17388 handle_command_reply: tid=0 2024-09-10T08:52:50.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.787+0000 7fccf20c8740 1 client.17388 shutdown 2024-09-10T08:52:50.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.787+0000 7fccf20c8740 1 -- 172.21.15.172:0/2824687568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 msgr2=0x5621174744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:50.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.787+0000 7fccf20c8740 1 --2- 172.21.15.172:0/2824687568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x5621174744d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fcce0003e50 tx=0x7fcce00095c0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.787+0000 7fccf20c8740 1 -- 172.21.15.172:0/2824687568 shutdown_connections 2024-09-10T08:52:50.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.787+0000 7fccf20c8740 1 --2- 172.21.15.172:0/2824687568 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562117486fe0 0x5621174894a0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.787+0000 7fccf20c8740 1 --2- 172.21.15.172:0/2824687568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x5621174744d0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.787+0000 7fccf20c8740 1 --2- 172.21.15.172:0/2824687568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56211746db40 0x562117473f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.787+0000 7fccf20c8740 1 --2- 172.21.15.172:0/2824687568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746c5d0 0x562117476cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.787+0000 7fccf20c8740 1 -- 172.21.15.172:0/2824687568 >> 172.21.15.172:0/2824687568 conn(0x562117435310 msgr2=0x56211746e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:50.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.787+0000 7fccd17fa640 0 client.17388 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:50.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.787+0000 7fccf20c8740 1 -- 172.21.15.172:0/2824687568 shutdown_connections 2024-09-10T08:52:50.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.787+0000 7fccf20c8740 1 -- 172.21.15.172:0/2824687568 wait complete. 2024-09-10T08:52:50.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.787+0000 7fccf20c8740 20 client.17388 trim_cache size 0 max 16384 2024-09-10T08:52:50.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fccf20c8740 1 Processor -- start 2024-09-10T08:52:50.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fccf20c8740 1 -- start start 2024-09-10T08:52:50.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fccf20c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746c5d0 0x562117483310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fccf20c8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56211746db40 0x562117478760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fccf20c8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562117477370 0x562117478ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fccf20c8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81b0e40 con 0x56211746db40 2024-09-10T08:52:50.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fccf20c8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81b2980 con 0x562117477370 2024-09-10T08:52:50.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fccf20c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81b1510 con 0x56211746c5d0 2024-09-10T08:52:50.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fcced28a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56211746db40 0x562117478760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:50.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fcced28a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56211746db40 0x562117478760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44760/0 (socket says 172.21.15.172:44760) 2024-09-10T08:52:50.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fcced28a640 1 -- 172.21.15.172:0/827453147 learned_addr learned my addr 172.21.15.172:0/827453147 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:50.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fcceda8b640 1 --2- 172.21.15.172:0/827453147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746c5d0 0x562117483310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:50.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fccee28c640 1 --2- 172.21.15.172:0/827453147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562117477370 0x562117478ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:50.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fcced28a640 1 -- 172.21.15.172:0/827453147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746c5d0 msgr2=0x562117483310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:50.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fcced28a640 1 --2- 172.21.15.172:0/827453147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746c5d0 0x562117483310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fcced28a640 1 -- 172.21.15.172:0/827453147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562117477370 msgr2=0x562117478ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:50.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fcced28a640 1 --2- 172.21.15.172:0/827453147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562117477370 0x562117478ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fcced28a640 1 -- 172.21.15.172:0/827453147 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562117468ed0 con 0x56211746db40 2024-09-10T08:52:50.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.800+0000 7fccee28c640 1 --2- 172.21.15.172:0/827453147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562117477370 0x562117478ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:50.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.801+0000 7fcced28a640 1 --2- 172.21.15.172:0/827453147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56211746db40 0x562117478760 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x56211746d070 tx=0x7fcce400eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e41b3de7e0a9328 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:50.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.801+0000 7fccd37fe640 1 -- 172.21.15.172:0/827453147 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce4022070 con 0x56211746db40 2024-09-10T08:52:50.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.801+0000 7fccd37fe640 1 -- 172.21.15.172:0/827453147 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcce4018040 con 0x56211746db40 2024-09-10T08:52:50.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.801+0000 7fccd37fe640 1 -- 172.21.15.172:0/827453147 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce401c680 con 0x56211746db40 2024-09-10T08:52:50.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.801+0000 7fccf20c8740 1 -- 172.21.15.172:0/827453147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56211746db40 msgr2=0x562117478760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:50.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.801+0000 7fccf20c8740 1 --2- 172.21.15.172:0/827453147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56211746db40 0x562117478760 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x56211746d070 tx=0x7fcce400eaf0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.801+0000 7fccf20c8740 1 -- 172.21.15.172:0/827453147 shutdown_connections 2024-09-10T08:52:50.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.801+0000 7fccf20c8740 1 --2- 172.21.15.172:0/827453147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562117477370 0x562117478ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.801+0000 7fccf20c8740 1 --2- 172.21.15.172:0/827453147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56211746db40 0x562117478760 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.801+0000 7fccf20c8740 1 --2- 172.21.15.172:0/827453147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746c5d0 0x562117483310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.801+0000 7fccf20c8740 1 -- 172.21.15.172:0/827453147 >> 172.21.15.172:0/827453147 conn(0x562117435310 msgr2=0x562117473f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:50.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.802+0000 7fccf20c8740 1 -- 172.21.15.172:0/827453147 shutdown_connections 2024-09-10T08:52:50.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.802+0000 7fccf20c8740 1 -- 172.21.15.172:0/827453147 wait complete. 2024-09-10T08:52:50.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.802+0000 7fccf20c8740 1 -- start start 2024-09-10T08:52:50.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.802+0000 7fccf20c8740 1 Processor -- start 2024-09-10T08:52:50.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.802+0000 7fccf20c8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56211746c5d0 0x56211748ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.802+0000 7fccf20c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746db40 0x56211748ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.802+0000 7fccf20c8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x56211748f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.802+0000 7fccf20c8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81b22b0 con 0x562117477370 2024-09-10T08:52:50.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.802+0000 7fccf20c8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81b1be0 con 0x56211746c5d0 2024-09-10T08:52:50.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.802+0000 7fccf20c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81aee20 con 0x56211746db40 2024-09-10T08:52:50.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.802+0000 7fccf20c8740 10 client.0 _ll_register_callbacks cb 0x7fcce0031a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:50.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fccee28c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x56211748f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:50.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fccee28c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x56211748f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44772/0 (socket says 172.21.15.172:44772) 2024-09-10T08:52:50.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fcceda8b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56211746c5d0 0x56211748ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:50.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fccee28c640 1 -- 172.21.15.172:0/974562601 learned_addr learned my addr 172.21.15.172:0/974562601 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:50.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fcced28a640 1 --2- 172.21.15.172:0/974562601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746db40 0x56211748ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:50.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fccee28c640 1 -- 172.21.15.172:0/974562601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746db40 msgr2=0x56211748ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:50.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fccee28c640 1 --2- 172.21.15.172:0/974562601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746db40 0x56211748ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fccee28c640 1 -- 172.21.15.172:0/974562601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56211746c5d0 msgr2=0x56211748ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:50.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fccee28c640 1 --2- 172.21.15.172:0/974562601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56211746c5d0 0x56211748ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fccee28c640 1 -- 172.21.15.172:0/974562601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56211747d990 con 0x562117477370 2024-09-10T08:52:50.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fccee28c640 1 --2- 172.21.15.172:0/974562601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x56211748f490 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fcce00318c0 tx=0x7fcce0003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759474b374a3573d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:50.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fccd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:50.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fccd17fa640 1 -- 172.21.15.172:0/974562601 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0004b40 con 0x562117477370 2024-09-10T08:52:50.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fccd17fa640 1 -- 172.21.15.172:0/974562601 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcce0004ce0 con 0x562117477370 2024-09-10T08:52:50.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fccf20c8740 1 -- 172.21.15.172:0/974562601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce4018040 con 0x562117477370 2024-09-10T08:52:50.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fccd17fa640 1 -- 172.21.15.172:0/974562601 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcce0069050 con 0x562117477370 2024-09-10T08:52:50.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fccf20c8740 10 client.17394 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:50.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.803+0000 7fccf20c8740 1 -- 172.21.15.172:0/974562601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5621174f1fc0 con 0x562117477370 2024-09-10T08:52:50.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.804+0000 7fccd17fa640 1 -- 172.21.15.172:0/974562601 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcce4018040 con 0x562117477370 2024-09-10T08:52:50.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.804+0000 7fccd17fa640 1 -- 172.21.15.172:0/974562601 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fcce0030040 con 0x562117477370 2024-09-10T08:52:50.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.804+0000 7fccf20c8740 10 client.17394 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:50.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.804+0000 7fccf20c8740 10 client.17394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:50.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.804+0000 7fccf20c8740 20 client.17394 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:50.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.804+0000 7fccf20c8740 1 --2- 172.21.15.172:0/974562601 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5621174f2790 0x5621174f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:50.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.804+0000 7fccf20c8740 4 client.17394 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:52:50.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.804+0000 7fccf20c8740 1 -- 172.21.15.172:0/974562601 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621173d9d70 con 0x5621174f2790 2024-09-10T08:52:50.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.804+0000 7fcceda8b640 1 --2- 172.21.15.172:0/974562601 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5621174f2790 0x5621174f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:50.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.804+0000 7fccd17fa640 10 client.17394 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:50.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.804+0000 7fcceda8b640 1 --2- 172.21.15.172:0/974562601 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5621174f2790 0x5621174f4bb0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74181dc2e63522b2 server_cookie=6f3b4e4f0038ca2d in_seq=0 out_seq=0 2024-09-10T08:52:50.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccd17fa640 1 -- 172.21.15.172:0/974562601 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621173d9d70 con 0x5621174f2790 2024-09-10T08:52:50.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccd17fa640 10 client.17394 handle_command_reply: tid=0 2024-09-10T08:52:50.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccf20c8740 1 client.17394 shutdown 2024-09-10T08:52:50.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccf20c8740 1 -- 172.21.15.172:0/974562601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 msgr2=0x56211748f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:50.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccf20c8740 1 --2- 172.21.15.172:0/974562601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x56211748f490 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fcce00318c0 tx=0x7fcce0003b30 comp rx=0 tx=0).stop 2024-09-10T08:52:50.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccf20c8740 1 -- 172.21.15.172:0/974562601 shutdown_connections 2024-09-10T08:52:50.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccf20c8740 1 --2- 172.21.15.172:0/974562601 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5621174f2790 0x5621174f4bb0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccf20c8740 1 --2- 172.21.15.172:0/974562601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562117477370 0x56211748f490 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccf20c8740 1 --2- 172.21.15.172:0/974562601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56211746db40 0x56211748ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccd17fa640 0 client.17394 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:50.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccf20c8740 1 --2- 172.21.15.172:0/974562601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56211746c5d0 0x56211748ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccf20c8740 1 -- 172.21.15.172:0/974562601 >> 172.21.15.172:0/974562601 conn(0x562117435310 msgr2=0x5621174757d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:50.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccf20c8740 1 -- 172.21.15.172:0/974562601 shutdown_connections 2024-09-10T08:52:50.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccf20c8740 1 -- 172.21.15.172:0/974562601 wait complete. 2024-09-10T08:52:50.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccf20c8740 20 client.17394 trim_cache size 0 max 16384 2024-09-10T08:52:50.818 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:52:50.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccefcef640 1 -- 172.21.15.172:0/1752915888 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccd40a2c00 msgr2=0x7fccd40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:50.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccefcef640 1 --2- 172.21.15.172:0/1752915888 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccd40a2c00 0x7fccd40a50c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009a60 tx=0x7fcce002a8e0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccefcef640 1 -- 172.21.15.172:0/1752915888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcce81002f0 msgr2=0x7fcce8101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:50.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.805+0000 7fccefcef640 1 --2- 172.21.15.172:0/1752915888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcce81002f0 0x7fcce8101270 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00b4a0 tx=0x7fccdc00b960 comp rx=0 tx=0).stop 2024-09-10T08:52:50.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.806+0000 7fccefcef640 1 -- 172.21.15.172:0/1752915888 shutdown_connections 2024-09-10T08:52:50.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.806+0000 7fccefcef640 1 --2- 172.21.15.172:0/1752915888 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccd40a2c00 0x7fccd40a50c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.806+0000 7fccefcef640 1 --2- 172.21.15.172:0/1752915888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce8106f50 0x7fcce8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.806+0000 7fccefcef640 1 --2- 172.21.15.172:0/1752915888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcce8100ca0 0x7fcce81017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.806+0000 7fccefcef640 1 --2- 172.21.15.172:0/1752915888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcce81002f0 0x7fcce8101270 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:50.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.806+0000 7fccefcef640 1 -- 172.21.15.172:0/1752915888 >> 172.21.15.172:0/1752915888 conn(0x7fcce8077210 msgr2=0x7fcce8105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:50.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.806+0000 7fccefcef640 1 -- 172.21.15.172:0/1752915888 shutdown_connections 2024-09-10T08:52:50.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:50.806+0000 7fccefcef640 1 -- 172.21.15.172:0/1752915888 wait complete. 2024-09-10T08:52:52.522 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:52:52.523 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:52:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:52 smithi172 ceph-mon[30854]: pgmap v549: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 21 op/s 2024-09-10T08:52:52.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:52 smithi174 ceph-mon[34068]: pgmap v549: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 21 op/s 2024-09-10T08:52:52.786 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:52:52.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:52 smithi191 ceph-mon[35314]: pgmap v549: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 21 op/s 2024-09-10T08:52:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.047+0000 7efec4a17640 1 -- 172.21.15.172:0/2994008691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec01025f0 msgr2=0x7efec0108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.047+0000 7efec4a17640 1 --2- 172.21.15.172:0/2994008691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec01025f0 0x7efec0108e80 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009ca0 tx=0x7efeb802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.047+0000 7efec4a17640 1 -- 172.21.15.172:0/2994008691 shutdown_connections 2024-09-10T08:52:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.047+0000 7efec4a17640 1 --2- 172.21.15.172:0/2994008691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec01025f0 0x7efec0108e80 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.047+0000 7efec4a17640 1 --2- 172.21.15.172:0/2994008691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efec0101c30 0x7efec01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.047+0000 7efec4a17640 1 --2- 172.21.15.172:0/2994008691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efec0100a40 0x7efec0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.047+0000 7efec4a17640 1 -- 172.21.15.172:0/2994008691 >> 172.21.15.172:0/2994008691 conn(0x7efec00fc1e0 msgr2=0x7efec00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.047+0000 7efec4a17640 1 -- 172.21.15.172:0/2994008691 shutdown_connections 2024-09-10T08:52:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.047+0000 7efec4a17640 1 -- 172.21.15.172:0/2994008691 wait complete. 2024-09-10T08:52:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.047+0000 7efec4a17640 1 Processor -- start 2024-09-10T08:52:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.047+0000 7efec4a17640 1 -- start start 2024-09-10T08:52:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efec4a17640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efec0100a40 0x7efec0197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efec4a17640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec0101c30 0x7efec01982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efec4a17640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efec01025f0 0x7efec019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efec4a17640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efec010c5f0 con 0x7efec0101c30 2024-09-10T08:52:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efec4a17640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efec010c470 con 0x7efec0100a40 2024-09-10T08:52:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efec4a17640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efec010c770 con 0x7efec01025f0 2024-09-10T08:52:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efebdd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec0101c30 0x7efec01982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efebdd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec0101c30 0x7efec01982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44800/0 (socket says 172.21.15.172:44800) 2024-09-10T08:52:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efebdd9b640 1 -- 172.21.15.172:0/2280050957 learned_addr learned my addr 172.21.15.172:0/2280050957 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efebe59c640 1 --2- 172.21.15.172:0/2280050957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efec0100a40 0x7efec0197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efebed9d640 1 --2- 172.21.15.172:0/2280050957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efec01025f0 0x7efec019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efebdd9b640 1 -- 172.21.15.172:0/2280050957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efec01025f0 msgr2=0x7efec019f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efebdd9b640 1 --2- 172.21.15.172:0/2280050957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efec01025f0 0x7efec019f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efebdd9b640 1 -- 172.21.15.172:0/2280050957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efec0100a40 msgr2=0x7efec0197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efebdd9b640 1 --2- 172.21.15.172:0/2280050957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efec0100a40 0x7efec0197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efebdd9b640 1 -- 172.21.15.172:0/2280050957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec019f980 con 0x7efec0101c30 2024-09-10T08:52:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efebdd9b640 1 --2- 172.21.15.172:0/2280050957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec0101c30 0x7efec01982a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7efeb000d950 tx=0x7efeb000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efeaf7fe640 1 -- 172.21.15.172:0/2280050957 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0017030 con 0x7efec0101c30 2024-09-10T08:52:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efeaf7fe640 1 -- 172.21.15.172:0/2280050957 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efeb0004020 con 0x7efec0101c30 2024-09-10T08:52:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efeaf7fe640 1 -- 172.21.15.172:0/2280050957 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0005570 con 0x7efec0101c30 2024-09-10T08:52:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efec4a17640 1 -- 172.21.15.172:0/2280050957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec019fc70 con 0x7efec0101c30 2024-09-10T08:52:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.048+0000 7efec4a17640 1 -- 172.21.15.172:0/2280050957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec01a01c0 con 0x7efec0101c30 2024-09-10T08:52:53.058 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.051+0000 7efec7889740 1 Processor -- start 2024-09-10T08:52:53.058 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.051+0000 7efec7889740 1 -- start start 2024-09-10T08:52:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efec7889740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba25da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efec7889740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba262e0 0x563daba2ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efec7889740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba2b370 0x563daba2d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efec7889740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563dab9892f0 con 0x563daba259a0 2024-09-10T08:52:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efec7889740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563dab931270 con 0x563daba262e0 2024-09-10T08:52:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efec7889740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563dab9852b0 con 0x563daba2b370 2024-09-10T08:52:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efeaf7fe640 1 -- 172.21.15.172:0/2280050957 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efeb00041d0 con 0x7efec0101c30 2024-09-10T08:52:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efebed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba25da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efebed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba25da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44806/0 (socket says 172.21.15.172:44806) 2024-09-10T08:52:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efebed9d640 1 -- 172.21.15.172:0/4226086754 learned_addr learned my addr 172.21.15.172:0/4226086754 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efebe59c640 1 --2- 172.21.15.172:0/4226086754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba262e0 0x563daba2ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efebed9d640 1 -- 172.21.15.172:0/4226086754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba2b370 msgr2=0x563daba2d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efebed9d640 1 --2- 172.21.15.172:0/4226086754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba2b370 0x563daba2d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efebed9d640 1 -- 172.21.15.172:0/4226086754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba262e0 msgr2=0x563daba2ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efebed9d640 1 --2- 172.21.15.172:0/4226086754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba262e0 0x563daba2ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efebed9d640 1 -- 172.21.15.172:0/4226086754 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dab930340 con 0x563daba259a0 2024-09-10T08:52:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efebed9d640 1 --2- 172.21.15.172:0/4226086754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba25da0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7efeb8035000 tx=0x7efeb8005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c437667c11786d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.052+0000 7efeacff9640 1 -- 172.21.15.172:0/4226086754 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb8031b40 con 0x563daba259a0 2024-09-10T08:52:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.053+0000 7efeacff9640 1 -- 172.21.15.172:0/4226086754 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efeb8036040 con 0x563daba259a0 2024-09-10T08:52:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.053+0000 7efeacff9640 1 -- 172.21.15.172:0/4226086754 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb80046b0 con 0x563daba259a0 2024-09-10T08:52:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.053+0000 7efeaf7fe640 1 --2- 172.21.15.172:0/2280050957 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efea80a2c30 0x7efea80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.053+0000 7efebe59c640 1 --2- 172.21.15.172:0/2280050957 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efea80a2c30 0x7efea80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.053+0000 7efeaf7fe640 1 -- 172.21.15.172:0/2280050957 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7efeb00ceda0 con 0x7efec0101c30 2024-09-10T08:52:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.053+0000 7efec7889740 1 -- 172.21.15.172:0/4226086754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 msgr2=0x563daba25da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.053+0000 7efebe59c640 1 --2- 172.21.15.172:0/2280050957 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efea80a2c30 0x7efea80a50f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x563daba26d10 tx=0x7efeb4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.053+0000 7efec7889740 1 --2- 172.21.15.172:0/4226086754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba25da0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7efeb8035000 tx=0x7efeb8005fa0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efec7889740 1 -- 172.21.15.172:0/4226086754 shutdown_connections 2024-09-10T08:52:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efec7889740 1 --2- 172.21.15.172:0/4226086754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba2b370 0x563daba2d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efec7889740 1 --2- 172.21.15.172:0/4226086754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba262e0 0x563daba2ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efec7889740 1 --2- 172.21.15.172:0/4226086754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba25da0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efec7889740 1 -- 172.21.15.172:0/4226086754 >> 172.21.15.172:0/4226086754 conn(0x563daba21010 msgr2=0x563daba21410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efec7889740 1 -- 172.21.15.172:0/4226086754 shutdown_connections 2024-09-10T08:52:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efec7889740 1 -- 172.21.15.172:0/4226086754 wait complete. 2024-09-10T08:52:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efec7889740 1 -- start start 2024-09-10T08:52:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efec7889740 1 Processor -- start 2024-09-10T08:52:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efec7889740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba259a0 0x563daba2fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efec7889740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba262e0 0x563daba30200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efec7889740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba2b370 0x563daba371d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efec7889740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563daba2e2e0 con 0x563daba2b370 2024-09-10T08:52:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efec7889740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563dab912290 con 0x563daba259a0 2024-09-10T08:52:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efec7889740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563daba2e460 con 0x563daba262e0 2024-09-10T08:52:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efec7889740 10 client.0 _ll_register_callbacks cb 0x563dab943140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efebed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba2b370 0x563daba371d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efebed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba2b370 0x563daba371d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44818/0 (socket says 172.21.15.172:44818) 2024-09-10T08:52:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efebed9d640 1 -- 172.21.15.172:0/2391257754 learned_addr learned my addr 172.21.15.172:0/2391257754 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.054+0000 7efebe59c640 1 --2- 172.21.15.172:0/2391257754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba259a0 0x563daba2fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.055+0000 7efebdd9b640 1 --2- 172.21.15.172:0/2391257754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba262e0 0x563daba30200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.055+0000 7efebed9d640 1 -- 172.21.15.172:0/2391257754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba262e0 msgr2=0x563daba30200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.055+0000 7efebed9d640 1 --2- 172.21.15.172:0/2391257754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba262e0 0x563daba30200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.055+0000 7efebed9d640 1 -- 172.21.15.172:0/2391257754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba259a0 msgr2=0x563daba2fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.055+0000 7efebed9d640 1 --2- 172.21.15.172:0/2391257754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba259a0 0x563daba2fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.055+0000 7efebed9d640 1 -- 172.21.15.172:0/2391257754 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dab69c210 con 0x563daba2b370 2024-09-10T08:52:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.055+0000 7efebed9d640 1 --2- 172.21.15.172:0/2391257754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba2b370 0x563daba371d0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009ca0 tx=0x7efeb80317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268e8f2d187a14db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.055+0000 7efea5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.055+0000 7efea5ffb640 1 -- 172.21.15.172:0/2391257754 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb804a040 con 0x563daba2b370 2024-09-10T08:52:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.055+0000 7efea5ffb640 1 -- 172.21.15.172:0/2391257754 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7efeb80043f0 con 0x563daba2b370 2024-09-10T08:52:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.055+0000 7efec7889740 1 -- 172.21.15.172:0/2391257754 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeb8036040 con 0x563daba2b370 2024-09-10T08:52:53.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.055+0000 7efea5ffb640 1 -- 172.21.15.172:0/2391257754 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efeb805c050 con 0x563daba2b370 2024-09-10T08:52:53.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.055+0000 7efec7889740 10 client.17406 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:53.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.055+0000 7efec7889740 1 -- 172.21.15.172:0/2391257754 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563dab768620 con 0x563daba2b370 2024-09-10T08:52:53.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.055+0000 7efea5ffb640 1 -- 172.21.15.172:0/2391257754 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efeb8036040 con 0x563daba2b370 2024-09-10T08:52:53.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.055+0000 7efea5ffb640 1 -- 172.21.15.172:0/2391257754 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7efeb804d070 con 0x563daba2b370 2024-09-10T08:52:53.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.056+0000 7efec7889740 10 client.17406 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:53.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.056+0000 7efec7889740 10 client.17406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:53.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.056+0000 7efec7889740 20 client.17406 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:53.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.056+0000 7efec7889740 1 --2- 172.21.15.172:0/2391257754 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563daba3b4d0 0x563daba3d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.056+0000 7efec7889740 4 client.17406 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:53.064 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.056+0000 7efec7889740 1 -- 172.21.15.172:0/2391257754 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563dab996a20 con 0x563daba3b4d0 2024-09-10T08:52:53.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.056+0000 7efebe59c640 1 --2- 172.21.15.172:0/2391257754 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563daba3b4d0 0x563daba3d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.056+0000 7efea5ffb640 10 client.17406 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:53.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.056+0000 7efebe59c640 1 --2- 172.21.15.172:0/2391257754 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563daba3b4d0 0x563daba3d8c0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bee8764223dfc0b server_cookie=232bcd11949cb52 in_seq=0 out_seq=0 2024-09-10T08:52:53.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.057+0000 7efea5ffb640 1 -- 172.21.15.172:0/2391257754 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563dab996a20 con 0x563daba3b4d0 2024-09-10T08:52:53.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.057+0000 7efea5ffb640 10 client.17406 handle_command_reply: tid=0 2024-09-10T08:52:53.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.057+0000 7efec7889740 1 client.17406 shutdown 2024-09-10T08:52:53.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.057+0000 7efec7889740 1 -- 172.21.15.172:0/2391257754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba2b370 msgr2=0x563daba371d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.057+0000 7efec7889740 1 --2- 172.21.15.172:0/2391257754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba2b370 0x563daba371d0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009ca0 tx=0x7efeb80317a0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.057+0000 7efec7889740 1 -- 172.21.15.172:0/2391257754 shutdown_connections 2024-09-10T08:52:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.057+0000 7efec7889740 1 --2- 172.21.15.172:0/2391257754 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563daba3b4d0 0x563daba3d8c0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.057+0000 7efec7889740 1 --2- 172.21.15.172:0/2391257754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba2b370 0x563daba371d0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.057+0000 7efec7889740 1 --2- 172.21.15.172:0/2391257754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba262e0 0x563daba30200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.057+0000 7efec7889740 1 --2- 172.21.15.172:0/2391257754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba259a0 0x563daba2fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.057+0000 7efec7889740 1 -- 172.21.15.172:0/2391257754 >> 172.21.15.172:0/2391257754 conn(0x563daba21010 msgr2=0x563daba22820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.057+0000 7efea5ffb640 0 client.17406 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.057+0000 7efec7889740 1 -- 172.21.15.172:0/2391257754 shutdown_connections 2024-09-10T08:52:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.057+0000 7efec7889740 1 -- 172.21.15.172:0/2391257754 wait complete. 2024-09-10T08:52:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.057+0000 7efec7889740 20 client.17406 trim_cache size 0 max 16384 2024-09-10T08:52:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.071+0000 7efec7889740 1 Processor -- start 2024-09-10T08:52:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.071+0000 7efec7889740 1 -- start start 2024-09-10T08:52:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.071+0000 7efec7889740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba2fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.071+0000 7efec7889740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba262e0 0x563daba37190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.071+0000 7efec7889740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba2b370 0x563daba30200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.071+0000 7efec7889740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efec01aa5d0 con 0x563daba259a0 2024-09-10T08:52:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.071+0000 7efec7889740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efec01ac110 con 0x563daba262e0 2024-09-10T08:52:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.071+0000 7efec7889740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efec01aaca0 con 0x563daba2b370 2024-09-10T08:52:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.072+0000 7efebe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba2fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.072+0000 7efebe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba2fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44822/0 (socket says 172.21.15.172:44822) 2024-09-10T08:52:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.072+0000 7efebe59c640 1 -- 172.21.15.172:0/3604699346 learned_addr learned my addr 172.21.15.172:0/3604699346 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.072+0000 7efebdd9b640 1 --2- 172.21.15.172:0/3604699346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba262e0 0x563daba37190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.072+0000 7efebed9d640 1 --2- 172.21.15.172:0/3604699346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba2b370 0x563daba30200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.072+0000 7efebe59c640 1 -- 172.21.15.172:0/3604699346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba2b370 msgr2=0x563daba30200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.072+0000 7efebe59c640 1 --2- 172.21.15.172:0/3604699346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba2b370 0x563daba30200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.072+0000 7efebe59c640 1 -- 172.21.15.172:0/3604699346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba262e0 msgr2=0x563daba37190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.072+0000 7efebe59c640 1 --2- 172.21.15.172:0/3604699346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba262e0 0x563daba37190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.072+0000 7efebe59c640 1 -- 172.21.15.172:0/3604699346 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563daba2d7d0 con 0x563daba259a0 2024-09-10T08:52:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.072+0000 7efebdd9b640 1 --2- 172.21.15.172:0/3604699346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba262e0 0x563daba37190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.072+0000 7efebe59c640 1 --2- 172.21.15.172:0/3604699346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba2fcc0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7efeb4016040 tx=0x7efeb4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603ce09078844495 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.072+0000 7efea7fff640 1 -- 172.21.15.172:0/3604699346 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb40239a0 con 0x563daba259a0 2024-09-10T08:52:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.072+0000 7efea7fff640 1 -- 172.21.15.172:0/3604699346 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efeb402f070 con 0x563daba259a0 2024-09-10T08:52:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.072+0000 7efea7fff640 1 -- 172.21.15.172:0/3604699346 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb402ab20 con 0x563daba259a0 2024-09-10T08:52:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 1 -- 172.21.15.172:0/3604699346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 msgr2=0x563daba2fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 1 --2- 172.21.15.172:0/3604699346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba2fcc0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7efeb4016040 tx=0x7efeb4009f90 comp rx=0 tx=0).stop 2024-09-10T08:52:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 1 -- 172.21.15.172:0/3604699346 shutdown_connections 2024-09-10T08:52:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 1 --2- 172.21.15.172:0/3604699346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba2b370 0x563daba30200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 1 --2- 172.21.15.172:0/3604699346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba262e0 0x563daba37190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 1 --2- 172.21.15.172:0/3604699346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba2fcc0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 1 -- 172.21.15.172:0/3604699346 >> 172.21.15.172:0/3604699346 conn(0x563daba21010 msgr2=0x563daba275c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 1 -- 172.21.15.172:0/3604699346 shutdown_connections 2024-09-10T08:52:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 1 -- 172.21.15.172:0/3604699346 wait complete. 2024-09-10T08:52:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 1 -- start start 2024-09-10T08:52:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 1 Processor -- start 2024-09-10T08:52:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba34120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba262e0 0x563daba34660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba2b370 0x563dabaa4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efec01aba40 con 0x563daba259a0 2024-09-10T08:52:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efec01ab370 con 0x563daba2b370 2024-09-10T08:52:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efec01a85b0 con 0x563daba262e0 2024-09-10T08:52:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efec7889740 10 client.0 _ll_register_callbacks cb 0x7efeb40093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efebe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba34120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efebe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba34120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44828/0 (socket says 172.21.15.172:44828) 2024-09-10T08:52:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.073+0000 7efebe59c640 1 -- 172.21.15.172:0/1419208398 learned_addr learned my addr 172.21.15.172:0/1419208398 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.074+0000 7efebdd9b640 1 --2- 172.21.15.172:0/1419208398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba262e0 0x563daba34660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.074+0000 7efebed9d640 1 --2- 172.21.15.172:0/1419208398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba2b370 0x563dabaa4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.074+0000 7efebe59c640 1 -- 172.21.15.172:0/1419208398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba262e0 msgr2=0x563daba34660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.074+0000 7efebe59c640 1 --2- 172.21.15.172:0/1419208398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba262e0 0x563daba34660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.074+0000 7efebe59c640 1 -- 172.21.15.172:0/1419208398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba2b370 msgr2=0x563dabaa4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.074+0000 7efebe59c640 1 --2- 172.21.15.172:0/1419208398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba2b370 0x563dabaa4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.074+0000 7efebe59c640 1 -- 172.21.15.172:0/1419208398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dabaa45f0 con 0x563daba259a0 2024-09-10T08:52:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.074+0000 7efebe59c640 1 --2- 172.21.15.172:0/1419208398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba34120 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7efeb4006e10 tx=0x7efeb402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bad9327127b8707 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.074+0000 7efea5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.074+0000 7efea5ffb640 1 -- 172.21.15.172:0/1419208398 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb4039020 con 0x563daba259a0 2024-09-10T08:52:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.074+0000 7efec7889740 1 -- 172.21.15.172:0/1419208398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeb402f070 con 0x563daba259a0 2024-09-10T08:52:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.074+0000 7efea5ffb640 1 -- 172.21.15.172:0/1419208398 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7efeb40059c0 con 0x563daba259a0 2024-09-10T08:52:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.074+0000 7efea5ffb640 1 -- 172.21.15.172:0/1419208398 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efeb402ad30 con 0x563daba259a0 2024-09-10T08:52:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.074+0000 7efec7889740 10 client.17412 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.074+0000 7efec7889740 1 -- 172.21.15.172:0/1419208398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563dabaa5850 con 0x563daba259a0 2024-09-10T08:52:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.074+0000 7efea5ffb640 1 -- 172.21.15.172:0/1419208398 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efeb402f070 con 0x563daba259a0 2024-09-10T08:52:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.075+0000 7efea5ffb640 1 -- 172.21.15.172:0/1419208398 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7efeb4020040 con 0x563daba259a0 2024-09-10T08:52:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.075+0000 7efec7889740 10 client.17412 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.075+0000 7efec7889740 10 client.17412 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.075+0000 7efec7889740 20 client.17412 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.075+0000 7efec7889740 1 --2- 172.21.15.172:0/1419208398 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563dabaa6200 0x563dabaa8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.075+0000 7efec7889740 4 client.17412 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:52:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.075+0000 7efec7889740 1 -- 172.21.15.172:0/1419208398 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563dab990d70 con 0x563dabaa6200 2024-09-10T08:52:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.075+0000 7efebdd9b640 1 --2- 172.21.15.172:0/1419208398 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563dabaa6200 0x563dabaa8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.075+0000 7efea5ffb640 10 client.17412 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.075+0000 7efebdd9b640 1 --2- 172.21.15.172:0/1419208398 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563dabaa6200 0x563dabaa8620 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cbfd1a7b7e8f4 server_cookie=18bd7f0c1eca7a25 in_seq=0 out_seq=0 2024-09-10T08:52:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.082+0000 7efea5ffb640 1 -- 172.21.15.172:0/1419208398 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563dab990d70 con 0x563dabaa6200 2024-09-10T08:52:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.082+0000 7efea5ffb640 10 client.17412 handle_command_reply: tid=0 2024-09-10T08:52:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.082+0000 7efec7889740 1 client.17412 shutdown 2024-09-10T08:52:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.082+0000 7efec7889740 1 -- 172.21.15.172:0/1419208398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 msgr2=0x563daba34120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.082+0000 7efec7889740 1 --2- 172.21.15.172:0/1419208398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba34120 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7efeb4006e10 tx=0x7efeb402a560 comp rx=0 tx=0).stop 2024-09-10T08:52:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.082+0000 7efec7889740 1 -- 172.21.15.172:0/1419208398 shutdown_connections 2024-09-10T08:52:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.082+0000 7efec7889740 1 --2- 172.21.15.172:0/1419208398 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563dabaa6200 0x563dabaa8620 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.082+0000 7efec7889740 1 --2- 172.21.15.172:0/1419208398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563daba2b370 0x563dabaa4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec7889740 1 --2- 172.21.15.172:0/1419208398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563daba262e0 0x563daba34660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec7889740 1 --2- 172.21.15.172:0/1419208398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563daba259a0 0x563daba34120 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec7889740 1 -- 172.21.15.172:0/1419208398 >> 172.21.15.172:0/1419208398 conn(0x563daba21010 msgr2=0x563daba3d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efea5ffb640 0 client.17412 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec7889740 1 -- 172.21.15.172:0/1419208398 shutdown_connections 2024-09-10T08:52:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec7889740 1 -- 172.21.15.172:0/1419208398 wait complete. 2024-09-10T08:52:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec7889740 20 client.17412 trim_cache size 0 max 16384 2024-09-10T08:52:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:52:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:52:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:52:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:52:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec4a17640 1 -- 172.21.15.172:0/2280050957 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efea80a2c30 msgr2=0x7efea80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec4a17640 1 --2- 172.21.15.172:0/2280050957 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efea80a2c30 0x7efea80a50f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x563daba26d10 tx=0x7efeb4009110 comp rx=0 tx=0).stop 2024-09-10T08:52:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec4a17640 1 -- 172.21.15.172:0/2280050957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec0101c30 msgr2=0x7efec01982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec4a17640 1 --2- 172.21.15.172:0/2280050957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec0101c30 0x7efec01982a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7efeb000d950 tx=0x7efeb000dd50 comp rx=0 tx=0).stop 2024-09-10T08:52:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec4a17640 1 -- 172.21.15.172:0/2280050957 shutdown_connections 2024-09-10T08:52:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec4a17640 1 --2- 172.21.15.172:0/2280050957 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efea80a2c30 0x7efea80a50f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec4a17640 1 --2- 172.21.15.172:0/2280050957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efec01025f0 0x7efec019f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec4a17640 1 --2- 172.21.15.172:0/2280050957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efec0101c30 0x7efec01982a0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec4a17640 1 --2- 172.21.15.172:0/2280050957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efec0100a40 0x7efec0197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec4a17640 1 -- 172.21.15.172:0/2280050957 >> 172.21.15.172:0/2280050957 conn(0x7efec00fc1e0 msgr2=0x7efec00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec4a17640 1 -- 172.21.15.172:0/2280050957 shutdown_connections 2024-09-10T08:52:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:52:53.083+0000 7efec4a17640 1 -- 172.21.15.172:0/2280050957 wait complete. 2024-09-10T08:52:53.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.278+0000 7f5668a59640 1 -- 172.21.15.172:0/4112668225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5664101420 msgr2=0x7f566410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.278+0000 7f5668a59640 1 --2- 172.21.15.172:0/4112668225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5664101420 0x7f566410d950 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f565c009cd0 tx=0x7f565c02a8e0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.278+0000 7f5668a59640 1 -- 172.21.15.172:0/4112668225 shutdown_connections 2024-09-10T08:52:53.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.278+0000 7f5668a59640 1 --2- 172.21.15.172:0/4112668225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5664101420 0x7f566410d950 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.278+0000 7f5668a59640 1 --2- 172.21.15.172:0/4112668225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5664100a80 0x7f5664100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.278+0000 7f5668a59640 1 --2- 172.21.15.172:0/4112668225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5664106a80 0x7f5664106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.278+0000 7f5668a59640 1 -- 172.21.15.172:0/4112668225 >> 172.21.15.172:0/4112668225 conn(0x7f56640fc7a0 msgr2=0x7f56640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:53.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.278+0000 7f5668a59640 1 -- 172.21.15.172:0/4112668225 shutdown_connections 2024-09-10T08:52:53.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.278+0000 7f5668a59640 1 -- 172.21.15.172:0/4112668225 wait complete. 2024-09-10T08:52:53.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.279+0000 7f5668a59640 1 Processor -- start 2024-09-10T08:52:53.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.279+0000 7f5668a59640 1 -- start start 2024-09-10T08:52:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f5668a59640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5664100a80 0x7f5664197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f5668a59640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5664101420 0x7f5664197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f5668a59640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5664106a80 0x7f566419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f5668a59640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5664110d30 con 0x7f5664100a80 2024-09-10T08:52:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f5668a59640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5664110bb0 con 0x7f5664106a80 2024-09-10T08:52:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f5668a59640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5664110eb0 con 0x7f5664101420 2024-09-10T08:52:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f566259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5664100a80 0x7f5664197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f566259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5664100a80 0x7f5664197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44848/0 (socket says 172.21.15.172:44848) 2024-09-10T08:52:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f566259c640 1 -- 172.21.15.172:0/2025427454 learned_addr learned my addr 172.21.15.172:0/2025427454 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f565bfff640 1 --2- 172.21.15.172:0/2025427454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5664101420 0x7f5664197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f5662d9d640 1 --2- 172.21.15.172:0/2025427454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5664106a80 0x7f566419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f566259c640 1 -- 172.21.15.172:0/2025427454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5664101420 msgr2=0x7f5664197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f566259c640 1 --2- 172.21.15.172:0/2025427454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5664101420 0x7f5664197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f566259c640 1 -- 172.21.15.172:0/2025427454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5664106a80 msgr2=0x7f566419bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f566259c640 1 --2- 172.21.15.172:0/2025427454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5664106a80 0x7f566419bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f566259c640 1 -- 172.21.15.172:0/2025427454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566419c200 con 0x7f5664100a80 2024-09-10T08:52:53.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f5662d9d640 1 --2- 172.21.15.172:0/2025427454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5664106a80 0x7f566419bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:53.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f566259c640 1 --2- 172.21.15.172:0/2025427454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5664100a80 0x7f5664197250 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f565400c890 tx=0x7f565400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:53.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f565b7fe640 1 -- 172.21.15.172:0/2025427454 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654004220 con 0x7f5664100a80 2024-09-10T08:52:53.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f565b7fe640 1 -- 172.21.15.172:0/2025427454 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56540043c0 con 0x7f5664100a80 2024-09-10T08:52:53.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f565b7fe640 1 -- 172.21.15.172:0/2025427454 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654004a00 con 0x7f5664100a80 2024-09-10T08:52:53.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f5668a59640 1 -- 172.21.15.172:0/2025427454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566419c4f0 con 0x7f5664100a80 2024-09-10T08:52:53.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.280+0000 7f5668a59640 1 -- 172.21.15.172:0/2025427454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56641a3dd0 con 0x7f5664100a80 2024-09-10T08:52:53.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.283+0000 7f565b7fe640 1 -- 172.21.15.172:0/2025427454 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5654004ba0 con 0x7f5664100a80 2024-09-10T08:52:53.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.283+0000 7f566ae32740 1 Processor -- start 2024-09-10T08:52:53.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.283+0000 7f566ae32740 1 -- start start 2024-09-10T08:52:53.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.283+0000 7f565b7fe640 1 --2- 172.21.15.172:0/2025427454 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56500a2b80 0x7f56500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.283+0000 7f566ae32740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176b9e0 0x56493176bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.283+0000 7f566ae32740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56493176c300 0x564931778a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.283+0000 7f566ae32740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564931778f50 0x56493177b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.283+0000 7f566ae32740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5649316d1280 con 0x56493176b9e0 2024-09-10T08:52:53.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.283+0000 7f566ae32740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564931678b10 con 0x564931778f50 2024-09-10T08:52:53.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.283+0000 7f566ae32740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5649316cd240 con 0x56493176c300 2024-09-10T08:52:53.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f5662d9d640 1 --2- 172.21.15.172:0/2025427454 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56500a2b80 0x7f56500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f565b7fe640 1 -- 172.21.15.172:0/2025427454 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f56540cf030 con 0x7f5664100a80 2024-09-10T08:52:53.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f566259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176b9e0 0x56493176bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f566259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176b9e0 0x56493176bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44864/0 (socket says 172.21.15.172:44864) 2024-09-10T08:52:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f566259c640 1 -- 172.21.15.172:0/3651340627 learned_addr learned my addr 172.21.15.172:0/3651340627 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f5662d9d640 1 --2- 172.21.15.172:0/3651340627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564931778f50 0x56493177b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f565bfff640 1 --2- 172.21.15.172:0/3651340627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56493176c300 0x564931778a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f566259c640 1 -- 172.21.15.172:0/3651340627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56493176c300 msgr2=0x564931778a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f566259c640 1 --2- 172.21.15.172:0/3651340627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56493176c300 0x564931778a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f566259c640 1 -- 172.21.15.172:0/3651340627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564931778f50 msgr2=0x56493177b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f566259c640 1 --2- 172.21.15.172:0/3651340627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564931778f50 0x56493177b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f566259c640 1 -- 172.21.15.172:0/3651340627 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649316851f0 con 0x56493176b9e0 2024-09-10T08:52:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f565bfff640 1 --2- 172.21.15.172:0/3651340627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56493176c300 0x564931778a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f566259c640 1 --2- 172.21.15.172:0/3651340627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176b9e0 0x56493176bdc0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5654097f00 tx=0x7f565409ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fe0bad3d6cf40a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f5662d9d640 1 --2- 172.21.15.172:0/2025427454 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56500a2b80 0x7f56500a5040 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f565c009ca0 tx=0x7f565c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:53.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f5658ff9640 1 -- 172.21.15.172:0/3651340627 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56540bb070 con 0x56493176b9e0 2024-09-10T08:52:53.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f5658ff9640 1 -- 172.21.15.172:0/3651340627 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5654083070 con 0x56493176b9e0 2024-09-10T08:52:53.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.284+0000 7f5658ff9640 1 -- 172.21.15.172:0/3651340627 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565401d040 con 0x56493176b9e0 2024-09-10T08:52:53.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.285+0000 7f566ae32740 1 -- 172.21.15.172:0/3651340627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176b9e0 msgr2=0x56493176bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.285+0000 7f566ae32740 1 --2- 172.21.15.172:0/3651340627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176b9e0 0x56493176bdc0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5654097f00 tx=0x7f565409ae90 comp rx=0 tx=0).stop 2024-09-10T08:52:53.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.285+0000 7f566ae32740 1 -- 172.21.15.172:0/3651340627 shutdown_connections 2024-09-10T08:52:53.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.285+0000 7f566ae32740 1 --2- 172.21.15.172:0/3651340627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564931778f50 0x56493177b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.285+0000 7f566ae32740 1 --2- 172.21.15.172:0/3651340627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56493176c300 0x564931778a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.285+0000 7f566ae32740 1 --2- 172.21.15.172:0/3651340627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176b9e0 0x56493176bdc0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.285+0000 7f566ae32740 1 -- 172.21.15.172:0/3651340627 >> 172.21.15.172:0/3651340627 conn(0x5649317680f0 msgr2=0x5649317684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:53.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.285+0000 7f566ae32740 1 -- 172.21.15.172:0/3651340627 shutdown_connections 2024-09-10T08:52:53.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.285+0000 7f566ae32740 1 -- 172.21.15.172:0/3651340627 wait complete. 2024-09-10T08:52:53.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.285+0000 7f566ae32740 1 -- start start 2024-09-10T08:52:53.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f566ae32740 1 Processor -- start 2024-09-10T08:52:53.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f566ae32740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176b9e0 0x5649317742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f566ae32740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56493176c300 0x564931774800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f566ae32740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564931778f50 0x564931782980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f566ae32740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56493176e980 con 0x56493176b9e0 2024-09-10T08:52:53.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f566ae32740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56493168bf10 con 0x56493176c300 2024-09-10T08:52:53.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f566ae32740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56493176eb00 con 0x564931778f50 2024-09-10T08:52:53.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f566ae32740 10 client.0 _ll_register_callbacks cb 0x564931654a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:53.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f566259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176b9e0 0x5649317742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f566259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176b9e0 0x5649317742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44872/0 (socket says 172.21.15.172:44872) 2024-09-10T08:52:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f566259c640 1 -- 172.21.15.172:0/2360741895 learned_addr learned my addr 172.21.15.172:0/2360741895 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f565bfff640 1 --2- 172.21.15.172:0/2360741895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56493176c300 0x564931774800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f5662d9d640 1 --2- 172.21.15.172:0/2360741895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564931778f50 0x564931782980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f566259c640 1 -- 172.21.15.172:0/2360741895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564931778f50 msgr2=0x564931782980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f566259c640 1 --2- 172.21.15.172:0/2360741895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564931778f50 0x564931782980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f566259c640 1 -- 172.21.15.172:0/2360741895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56493176c300 msgr2=0x564931774800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f566259c640 1 --2- 172.21.15.172:0/2360741895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56493176c300 0x564931774800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f566259c640 1 -- 172.21.15.172:0/2360741895 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649314426a0 con 0x56493176b9e0 2024-09-10T08:52:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f565bfff640 1 --2- 172.21.15.172:0/2360741895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56493176c300 0x564931774800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f566259c640 1 --2- 172.21.15.172:0/2360741895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176b9e0 0x5649317742c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f56540cc820 tx=0x7f565400a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd627a2e2309ecb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.286+0000 7f5649ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:53.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.287+0000 7f5649ffb640 1 -- 172.21.15.172:0/2360741895 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565401d070 con 0x56493176b9e0 2024-09-10T08:52:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.287+0000 7f5649ffb640 1 -- 172.21.15.172:0/2360741895 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f56540050b0 con 0x56493176b9e0 2024-09-10T08:52:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.287+0000 7f566ae32740 1 -- 172.21.15.172:0/2360741895 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5654083070 con 0x56493176b9e0 2024-09-10T08:52:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.287+0000 7f5649ffb640 1 -- 172.21.15.172:0/2360741895 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f56540bb040 con 0x56493176b9e0 2024-09-10T08:52:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.287+0000 7f566ae32740 10 client.17424 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.287+0000 7f566ae32740 1 -- 172.21.15.172:0/2360741895 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564931785530 con 0x56493176b9e0 2024-09-10T08:52:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.287+0000 7f5649ffb640 1 -- 172.21.15.172:0/2360741895 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5654083070 con 0x56493176b9e0 2024-09-10T08:52:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.287+0000 7f5649ffb640 1 -- 172.21.15.172:0/2360741895 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f5654081cb0 con 0x56493176b9e0 2024-09-10T08:52:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.287+0000 7f566ae32740 10 client.17424 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.287+0000 7f566ae32740 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.287+0000 7f566ae32740 20 client.17424 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.287+0000 7f566ae32740 1 --2- 172.21.15.172:0/2360741895 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564931786880 0x564931788c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.287+0000 7f566ae32740 4 client.17424 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:53.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.287+0000 7f566ae32740 1 -- 172.21.15.172:0/2360741895 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649316de7f0 con 0x564931786880 2024-09-10T08:52:53.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.287+0000 7f565bfff640 1 --2- 172.21.15.172:0/2360741895 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564931786880 0x564931788c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.288+0000 7f565bfff640 1 --2- 172.21.15.172:0/2360741895 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564931786880 0x564931788c70 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c49e3be9ce69282a server_cookie=efa0da24942697d in_seq=0 out_seq=0 2024-09-10T08:52:53.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.288+0000 7f5649ffb640 10 client.17424 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:53.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.288+0000 7f5649ffb640 1 -- 172.21.15.172:0/2360741895 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649316de7f0 con 0x564931786880 2024-09-10T08:52:53.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.288+0000 7f5649ffb640 10 client.17424 handle_command_reply: tid=0 2024-09-10T08:52:53.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.288+0000 7f566ae32740 1 client.17424 shutdown 2024-09-10T08:52:53.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.289+0000 7f566ae32740 1 -- 172.21.15.172:0/2360741895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176b9e0 msgr2=0x5649317742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.289+0000 7f566ae32740 1 --2- 172.21.15.172:0/2360741895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176b9e0 0x5649317742c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f56540cc820 tx=0x7f565400a590 comp rx=0 tx=0).stop 2024-09-10T08:52:53.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.289+0000 7f566ae32740 1 -- 172.21.15.172:0/2360741895 shutdown_connections 2024-09-10T08:52:53.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.289+0000 7f566ae32740 1 --2- 172.21.15.172:0/2360741895 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564931786880 0x564931788c70 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.289+0000 7f566ae32740 1 --2- 172.21.15.172:0/2360741895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564931778f50 0x564931782980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.289+0000 7f566ae32740 1 --2- 172.21.15.172:0/2360741895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56493176c300 0x564931774800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.289+0000 7f5649ffb640 0 client.17424 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:53.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.289+0000 7f566ae32740 1 --2- 172.21.15.172:0/2360741895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176b9e0 0x5649317742c0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.289+0000 7f566ae32740 1 -- 172.21.15.172:0/2360741895 >> 172.21.15.172:0/2360741895 conn(0x5649317680f0 msgr2=0x564931767270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:53.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.289+0000 7f566ae32740 1 -- 172.21.15.172:0/2360741895 shutdown_connections 2024-09-10T08:52:53.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.289+0000 7f566ae32740 1 -- 172.21.15.172:0/2360741895 wait complete. 2024-09-10T08:52:53.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.289+0000 7f566ae32740 20 client.17424 trim_cache size 0 max 16384 2024-09-10T08:52:53.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.305+0000 7f566ae32740 1 Processor -- start 2024-09-10T08:52:53.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f566ae32740 1 -- start start 2024-09-10T08:52:53.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f566ae32740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56493176b9e0 0x564931778130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f566ae32740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176c300 0x5649317824a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f566ae32740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564931778f50 0x564931771520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f566ae32740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f56641aca00 con 0x56493176c300 2024-09-10T08:52:53.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f566ae32740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56641ae540 con 0x56493176b9e0 2024-09-10T08:52:53.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f566ae32740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f56641ad0d0 con 0x564931778f50 2024-09-10T08:52:53.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f565bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176c300 0x5649317824a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f565bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176c300 0x5649317824a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44884/0 (socket says 172.21.15.172:44884) 2024-09-10T08:52:53.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f565bfff640 1 -- 172.21.15.172:0/2367431914 learned_addr learned my addr 172.21.15.172:0/2367431914 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:53.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f566259c640 1 --2- 172.21.15.172:0/2367431914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56493176b9e0 0x564931778130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f565bfff640 1 -- 172.21.15.172:0/2367431914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564931778f50 msgr2=0x564931771520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:53.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f565bfff640 1 --2- 172.21.15.172:0/2367431914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564931778f50 0x564931771520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f565bfff640 1 -- 172.21.15.172:0/2367431914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56493176b9e0 msgr2=0x564931778130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:53.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f565bfff640 1 --2- 172.21.15.172:0/2367431914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56493176b9e0 0x564931778130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f565bfff640 1 -- 172.21.15.172:0/2367431914 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564931772810 con 0x56493176c300 2024-09-10T08:52:53.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f565bfff640 1 --2- 172.21.15.172:0/2367431914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176c300 0x5649317824a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f564c00b780 tx=0x7f564c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34c0fe5e02261a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:53.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.306+0000 7f564bfff640 1 -- 172.21.15.172:0/2367431914 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c01f070 con 0x56493176c300 2024-09-10T08:52:53.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.307+0000 7f564bfff640 1 -- 172.21.15.172:0/2367431914 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f564c003d20 con 0x56493176c300 2024-09-10T08:52:53.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.307+0000 7f564bfff640 1 -- 172.21.15.172:0/2367431914 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c01b520 con 0x56493176c300 2024-09-10T08:52:53.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.307+0000 7f566ae32740 1 -- 172.21.15.172:0/2367431914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176c300 msgr2=0x5649317824a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.307+0000 7f566ae32740 1 --2- 172.21.15.172:0/2367431914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176c300 0x5649317824a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f564c00b780 tx=0x7f564c00b390 comp rx=0 tx=0).stop 2024-09-10T08:52:53.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.307+0000 7f566ae32740 1 -- 172.21.15.172:0/2367431914 shutdown_connections 2024-09-10T08:52:53.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.307+0000 7f566ae32740 1 --2- 172.21.15.172:0/2367431914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564931778f50 0x564931771520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.307+0000 7f566ae32740 1 --2- 172.21.15.172:0/2367431914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176c300 0x5649317824a0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.307+0000 7f566ae32740 1 --2- 172.21.15.172:0/2367431914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56493176b9e0 0x564931778130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.307+0000 7f566ae32740 1 -- 172.21.15.172:0/2367431914 >> 172.21.15.172:0/2367431914 conn(0x5649317680f0 msgr2=0x564931777a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:53.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.307+0000 7f566ae32740 1 -- 172.21.15.172:0/2367431914 shutdown_connections 2024-09-10T08:52:53.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.307+0000 7f566ae32740 1 -- 172.21.15.172:0/2367431914 wait complete. 2024-09-10T08:52:53.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.307+0000 7f566ae32740 1 -- start start 2024-09-10T08:52:53.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f566ae32740 1 Processor -- start 2024-09-10T08:52:53.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f566ae32740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56493176b9e0 0x564931780420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f566ae32740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176c300 0x564931780960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f566ae32740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564931778f50 0x5649317f01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f566ae32740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f56641ade70 con 0x56493176c300 2024-09-10T08:52:53.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f566ae32740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56641ad7a0 con 0x56493176b9e0 2024-09-10T08:52:53.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f566ae32740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f56641aa9e0 con 0x564931778f50 2024-09-10T08:52:53.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f566ae32740 10 client.0 _ll_register_callbacks cb 0x7f565c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:53.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f565bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176c300 0x564931780960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f565bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176c300 0x564931780960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44886/0 (socket says 172.21.15.172:44886) 2024-09-10T08:52:53.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f565bfff640 1 -- 172.21.15.172:0/1854431465 learned_addr learned my addr 172.21.15.172:0/1854431465 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:53.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f5662d9d640 1 --2- 172.21.15.172:0/1854431465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564931778f50 0x5649317f01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f566259c640 1 --2- 172.21.15.172:0/1854431465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56493176b9e0 0x564931780420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f565bfff640 1 -- 172.21.15.172:0/1854431465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564931778f50 msgr2=0x5649317f01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:53.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f565bfff640 1 --2- 172.21.15.172:0/1854431465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564931778f50 0x5649317f01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f565bfff640 1 -- 172.21.15.172:0/1854431465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56493176b9e0 msgr2=0x564931780420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:53.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f565bfff640 1 --2- 172.21.15.172:0/1854431465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56493176b9e0 0x564931780420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f565bfff640 1 -- 172.21.15.172:0/1854431465 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649317f0700 con 0x56493176c300 2024-09-10T08:52:53.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f5662d9d640 1 --2- 172.21.15.172:0/1854431465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564931778f50 0x5649317f01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:53.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.308+0000 7f566259c640 1 --2- 172.21.15.172:0/1854431465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56493176b9e0 0x564931780420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:53.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.309+0000 7f565bfff640 1 --2- 172.21.15.172:0/1854431465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176c300 0x564931780960 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f564c005b90 tx=0x7f564c003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c390e7c9b34d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:53.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.309+0000 7f5649ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:53.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.309+0000 7f5649ffb640 1 -- 172.21.15.172:0/1854431465 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c031060 con 0x56493176c300 2024-09-10T08:52:53.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.309+0000 7f566ae32740 1 -- 172.21.15.172:0/1854431465 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f564c003d20 con 0x56493176c300 2024-09-10T08:52:53.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.309+0000 7f5649ffb640 1 -- 172.21.15.172:0/1854431465 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f564c012860 con 0x56493176c300 2024-09-10T08:52:53.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.309+0000 7f5649ffb640 1 -- 172.21.15.172:0/1854431465 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f564c038050 con 0x56493176c300 2024-09-10T08:52:53.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.309+0000 7f566ae32740 10 client.17430 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:53.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.309+0000 7f566ae32740 1 -- 172.21.15.172:0/1854431465 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5649317f1950 con 0x56493176c300 2024-09-10T08:52:53.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.309+0000 7f5649ffb640 1 -- 172.21.15.172:0/1854431465 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f564c003d20 con 0x56493176c300 2024-09-10T08:52:53.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.309+0000 7f5649ffb640 1 -- 172.21.15.172:0/1854431465 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f564c02b030 con 0x56493176c300 2024-09-10T08:52:53.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.309+0000 7f566ae32740 10 client.17430 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:53.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.309+0000 7f566ae32740 10 client.17430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:53.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.309+0000 7f566ae32740 20 client.17430 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:53.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.309+0000 7f566ae32740 1 --2- 172.21.15.172:0/1854431465 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5649317f1f60 0x5649317f4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:53.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.309+0000 7f566ae32740 4 client.17430 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:52:53.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.309+0000 7f566ae32740 1 -- 172.21.15.172:0/1854431465 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5649316d8d00 con 0x5649317f1f60 2024-09-10T08:52:53.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.310+0000 7f566259c640 1 --2- 172.21.15.172:0/1854431465 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5649317f1f60 0x5649317f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:53.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.310+0000 7f566259c640 1 --2- 172.21.15.172:0/1854431465 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5649317f1f60 0x5649317f4380 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e9c44b85056a7e2 server_cookie=5df51b1b15ba4f6d in_seq=0 out_seq=0 2024-09-10T08:52:53.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.310+0000 7f5649ffb640 10 client.17430 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:53.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.311+0000 7f5649ffb640 1 -- 172.21.15.172:0/1854431465 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5649316d8d00 con 0x5649317f1f60 2024-09-10T08:52:53.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.311+0000 7f5649ffb640 10 client.17430 handle_command_reply: tid=0 2024-09-10T08:52:53.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.311+0000 7f566ae32740 1 client.17430 shutdown 2024-09-10T08:52:53.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.311+0000 7f566ae32740 1 -- 172.21.15.172:0/1854431465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176c300 msgr2=0x564931780960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.311+0000 7f566ae32740 1 --2- 172.21.15.172:0/1854431465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176c300 0x564931780960 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f564c005b90 tx=0x7f564c003c30 comp rx=0 tx=0).stop 2024-09-10T08:52:53.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.311+0000 7f566ae32740 1 -- 172.21.15.172:0/1854431465 shutdown_connections 2024-09-10T08:52:53.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.311+0000 7f566ae32740 1 --2- 172.21.15.172:0/1854431465 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5649317f1f60 0x5649317f4380 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.311+0000 7f566ae32740 1 --2- 172.21.15.172:0/1854431465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564931778f50 0x5649317f01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.311+0000 7f566ae32740 1 --2- 172.21.15.172:0/1854431465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56493176c300 0x564931780960 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.311+0000 7f566ae32740 1 --2- 172.21.15.172:0/1854431465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56493176b9e0 0x564931780420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.311+0000 7f566ae32740 1 -- 172.21.15.172:0/1854431465 >> 172.21.15.172:0/1854431465 conn(0x5649317680f0 msgr2=0x564931767270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:53.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.311+0000 7f5649ffb640 0 client.17430 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:53.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.311+0000 7f566ae32740 1 -- 172.21.15.172:0/1854431465 shutdown_connections 2024-09-10T08:52:53.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.311+0000 7f566ae32740 1 -- 172.21.15.172:0/1854431465 wait complete. 2024-09-10T08:52:53.330 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.311+0000 7f566ae32740 20 client.17430 trim_cache size 0 max 16384 2024-09-10T08:52:53.330 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:52:53.330 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T08:52:53.330 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "9724cbd5-b97e-4410-8c69-bbfd75a9823b", 2024-09-10T08:52:53.330 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T08:52:53.330 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:52:53.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.312+0000 7f5668a59640 1 -- 172.21.15.172:0/2025427454 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56500a2b80 msgr2=0x7f56500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.312+0000 7f5668a59640 1 --2- 172.21.15.172:0/2025427454 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56500a2b80 0x7f56500a5040 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f565c009ca0 tx=0x7f565c005fd0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.312+0000 7f5668a59640 1 -- 172.21.15.172:0/2025427454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5664100a80 msgr2=0x7f5664197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:53.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.312+0000 7f5668a59640 1 --2- 172.21.15.172:0/2025427454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5664100a80 0x7f5664197250 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f565400c890 tx=0x7f565400cd50 comp rx=0 tx=0).stop 2024-09-10T08:52:53.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.312+0000 7f5668a59640 1 -- 172.21.15.172:0/2025427454 shutdown_connections 2024-09-10T08:52:53.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.312+0000 7f5668a59640 1 --2- 172.21.15.172:0/2025427454 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56500a2b80 0x7f56500a5040 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.331 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.312+0000 7f5668a59640 1 --2- 172.21.15.172:0/2025427454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5664106a80 0x7f566419bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.312+0000 7f5668a59640 1 --2- 172.21.15.172:0/2025427454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5664101420 0x7f5664197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.312+0000 7f5668a59640 1 --2- 172.21.15.172:0/2025427454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5664100a80 0x7f5664197250 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:53.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.312+0000 7f5668a59640 1 -- 172.21.15.172:0/2025427454 >> 172.21.15.172:0/2025427454 conn(0x7f56640fc7a0 msgr2=0x7f56640fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:53.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.312+0000 7f5668a59640 1 -- 172.21.15.172:0/2025427454 shutdown_connections 2024-09-10T08:52:53.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:53.312+0000 7f5668a59640 1 -- 172.21.15.172:0/2025427454 wait complete. 2024-09-10T08:52:54.134 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:52:54.384 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:52:54.505 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:54 smithi172 ceph-mon[30854]: pgmap v550: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 MiB/s wr, 14 op/s 2024-09-10T08:52:54.505 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:54 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T08:52:54.505 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:54 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.505 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:54 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:52:54.505 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:54 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:52:54.505 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:54 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:52:54.505 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:54 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:52:54.506 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:54 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.506 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:54 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.506 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:54 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:52:54.506 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:54 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.506 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:54 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.506 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:54 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:52:54.506 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:54 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.506 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:54 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:54 smithi174 ceph-mon[34068]: pgmap v550: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 MiB/s wr, 14 op/s 2024-09-10T08:52:54.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:54 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T08:52:54.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:54 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:54 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:52:54.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:54 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:52:54.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:54 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:52:54.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:54 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:52:54.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:54 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:54 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:54 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:52:54.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:54 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:54 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:54 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:52:54.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:54 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:54 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:54 smithi191 ceph-mon[35314]: pgmap v550: 129 pgs: 129 active+clean; 5.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 MiB/s wr, 14 op/s 2024-09-10T08:52:54.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:54 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T08:52:54.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:54 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:54 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:52:54.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:54 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:52:54.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:54 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:52:54.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:54 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:52:54.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:54 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:54 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:54 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:52:54.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:54 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:54 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:54 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:52:54.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:54 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:54 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:52:54.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.823+0000 7fa12e1a1640 1 -- 172.21.15.172:0/1128423249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa128100980 msgr2=0x7fa128100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:54.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.823+0000 7fa12e1a1640 1 --2- 172.21.15.172:0/1128423249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa128100980 0x7fa128100d60 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fa124009900 tx=0x7fa12402a470 comp rx=0 tx=0).stop 2024-09-10T08:52:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.823+0000 7fa12e1a1640 1 -- 172.21.15.172:0/1128423249 shutdown_connections 2024-09-10T08:52:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.823+0000 7fa12e1a1640 1 --2- 172.21.15.172:0/1128423249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa128105d90 0x7fa128106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.823+0000 7fa12e1a1640 1 --2- 172.21.15.172:0/1128423249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1281012a0 0x7fa128105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.823+0000 7fa12e1a1640 1 --2- 172.21.15.172:0/1128423249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa128100980 0x7fa128100d60 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.823+0000 7fa12e1a1640 1 -- 172.21.15.172:0/1128423249 >> 172.21.15.172:0/1128423249 conn(0x7fa1280fc630 msgr2=0x7fa1280fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.823+0000 7fa12e1a1640 1 -- 172.21.15.172:0/1128423249 shutdown_connections 2024-09-10T08:52:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.823+0000 7fa12e1a1640 1 -- 172.21.15.172:0/1128423249 wait complete. 2024-09-10T08:52:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.824+0000 7fa12e1a1640 1 Processor -- start 2024-09-10T08:52:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.824+0000 7fa12e1a1640 1 -- start start 2024-09-10T08:52:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.824+0000 7fa12e1a1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa128100980 0x7fa1281a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.824+0000 7fa12e1a1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1281012a0 0x7fa1281a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.824+0000 7fa12e1a1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa128105d90 0x7fa12819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa12e1a1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa128118270 con 0x7fa128105d90 2024-09-10T08:52:54.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa12e1a1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281180f0 con 0x7fa128100980 2024-09-10T08:52:54.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa12e1a1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281183f0 con 0x7fa1281012a0 2024-09-10T08:52:54.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa12d9a0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa128105d90 0x7fa12819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:54.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa12c99e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1281012a0 0x7fa1281a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:54.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa12d9a0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa128105d90 0x7fa12819e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44900/0 (socket says 172.21.15.172:44900) 2024-09-10T08:52:54.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa12d9a0640 1 -- 172.21.15.172:0/1787916099 learned_addr learned my addr 172.21.15.172:0/1787916099 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:54.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa12d19f640 1 --2- 172.21.15.172:0/1787916099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa128100980 0x7fa1281a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:54.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa12d9a0640 1 -- 172.21.15.172:0/1787916099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1281012a0 msgr2=0x7fa1281a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:54.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa12d9a0640 1 --2- 172.21.15.172:0/1787916099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1281012a0 0x7fa1281a4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa12d9a0640 1 -- 172.21.15.172:0/1787916099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa128100980 msgr2=0x7fa1281a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa12d9a0640 1 --2- 172.21.15.172:0/1787916099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa128100980 0x7fa1281a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa12d9a0640 1 -- 172.21.15.172:0/1787916099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12819eed0 con 0x7fa128105d90 2024-09-10T08:52:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa12d9a0640 1 --2- 172.21.15.172:0/1787916099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa128105d90 0x7fa12819e660 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fa12000cad0 tx=0x7fa12000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa11e7fc640 1 -- 172.21.15.172:0/1787916099 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12000ae20 con 0x7fa128105d90 2024-09-10T08:52:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa11e7fc640 1 -- 172.21.15.172:0/1787916099 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa120005020 con 0x7fa128105d90 2024-09-10T08:52:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa12e1a1640 1 -- 172.21.15.172:0/1787916099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12819f1c0 con 0x7fa128105d90 2024-09-10T08:52:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa11e7fc640 1 -- 172.21.15.172:0/1787916099 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12000f6b0 con 0x7fa128105d90 2024-09-10T08:52:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.825+0000 7fa12e1a1640 1 -- 172.21.15.172:0/1787916099 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1281ab510 con 0x7fa128105d90 2024-09-10T08:52:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.828+0000 7fa1317dc740 1 Processor -- start 2024-09-10T08:52:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.828+0000 7fa11e7fc640 1 -- 172.21.15.172:0/1787916099 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa1200051d0 con 0x7fa128105d90 2024-09-10T08:52:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.828+0000 7fa1317dc740 1 -- start start 2024-09-10T08:52:54.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa1317dc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11670 0x55ee95c11a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa1317dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c11f90 0x55ee95c1c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa1317dc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c1c850 0x55ee95c1ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa1317dc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ee95b772e0 con 0x55ee95c11670 2024-09-10T08:52:54.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa1317dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ee95b1ece0 con 0x55ee95c11f90 2024-09-10T08:52:54.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa1317dc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ee95b732a0 con 0x55ee95c1c850 2024-09-10T08:52:54.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa12c99e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11670 0x55ee95c11a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:54.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa12c99e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11670 0x55ee95c11a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44916/0 (socket says 172.21.15.172:44916) 2024-09-10T08:52:54.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa12c99e640 1 -- 172.21.15.172:0/15803197 learned_addr learned my addr 172.21.15.172:0/15803197 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:54.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa11e7fc640 1 --2- 172.21.15.172:0/1787916099 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa1140a2c00 0x7fa1140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa12c99e640 1 --2- 172.21.15.172:0/15803197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c1c850 0x55ee95c1ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:54.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa12c99e640 1 -- 172.21.15.172:0/15803197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c1c850 msgr2=0x55ee95c1ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:54.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa12c99e640 1 --2- 172.21.15.172:0/15803197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c1c850 0x55ee95c1ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa12c99e640 1 -- 172.21.15.172:0/15803197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c11f90 msgr2=0x55ee95c1c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:54.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa12c99e640 1 --2- 172.21.15.172:0/15803197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c11f90 0x55ee95c1c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa12c99e640 1 -- 172.21.15.172:0/15803197 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee95b03440 con 0x55ee95c11670 2024-09-10T08:52:54.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa12d9a0640 1 --2- 172.21.15.172:0/1787916099 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa1140a2c00 0x7fa1140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:54.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa11e7fc640 1 -- 172.21.15.172:0/1787916099 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa1200cf360 con 0x7fa128105d90 2024-09-10T08:52:54.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.829+0000 7fa12c99e640 1 --2- 172.21.15.172:0/15803197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11670 0x55ee95c11a50 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fa11800a450 tx=0x7fa11800a770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cce332737c913a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:54.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.830+0000 7fa1137fe640 1 -- 172.21.15.172:0/15803197 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118019030 con 0x55ee95c11670 2024-09-10T08:52:54.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.830+0000 7fa1137fe640 1 -- 172.21.15.172:0/15803197 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1180110d0 con 0x55ee95c11670 2024-09-10T08:52:54.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.830+0000 7fa1137fe640 1 -- 172.21.15.172:0/15803197 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118004de0 con 0x55ee95c11670 2024-09-10T08:52:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.830+0000 7fa1317dc740 1 -- 172.21.15.172:0/15803197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11670 msgr2=0x55ee95c11a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.830+0000 7fa12d9a0640 1 --2- 172.21.15.172:0/1787916099 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa1140a2c00 0x7fa1140a50c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa12009b7f0 tx=0x7fa12001baf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.830+0000 7fa1317dc740 1 --2- 172.21.15.172:0/15803197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11670 0x55ee95c11a50 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fa11800a450 tx=0x7fa11800a770 comp rx=0 tx=0).stop 2024-09-10T08:52:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.830+0000 7fa1317dc740 1 -- 172.21.15.172:0/15803197 shutdown_connections 2024-09-10T08:52:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.830+0000 7fa1317dc740 1 --2- 172.21.15.172:0/15803197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c1c850 0x55ee95c1ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.830+0000 7fa1317dc740 1 --2- 172.21.15.172:0/15803197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c11f90 0x55ee95c1c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.830+0000 7fa1317dc740 1 --2- 172.21.15.172:0/15803197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11670 0x55ee95c11a50 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.830+0000 7fa1317dc740 1 -- 172.21.15.172:0/15803197 >> 172.21.15.172:0/15803197 conn(0x55ee95c0dc00 msgr2=0x55ee95c0dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.831+0000 7fa1317dc740 1 -- 172.21.15.172:0/15803197 shutdown_connections 2024-09-10T08:52:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.831+0000 7fa1317dc740 1 -- 172.21.15.172:0/15803197 wait complete. 2024-09-10T08:52:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.831+0000 7fa1317dc740 1 -- start start 2024-09-10T08:52:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.831+0000 7fa1317dc740 1 Processor -- start 2024-09-10T08:52:54.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.831+0000 7fa1317dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c11670 0x55ee95c1c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.831+0000 7fa1317dc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11f90 0x55ee95c19440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.831+0000 7fa1317dc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c1c850 0x55ee95c19980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.831+0000 7fa1317dc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ee95c1f420 con 0x55ee95c11f90 2024-09-10T08:52:54.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.831+0000 7fa1317dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ee95b32050 con 0x55ee95c11670 2024-09-10T08:52:54.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.831+0000 7fa1317dc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ee95c1f5a0 con 0x55ee95c1c850 2024-09-10T08:52:54.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa1317dc740 10 client.0 _ll_register_callbacks cb 0x55ee95b72160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:54.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa12d19f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11f90 0x55ee95c19440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:54.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa12d19f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11f90 0x55ee95c19440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44918/0 (socket says 172.21.15.172:44918) 2024-09-10T08:52:54.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa12d19f640 1 -- 172.21.15.172:0/1464575676 learned_addr learned my addr 172.21.15.172:0/1464575676 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:54.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa12c99e640 1 --2- 172.21.15.172:0/1464575676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c1c850 0x55ee95c19980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:54.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa12c99e640 1 --2- 172.21.15.172:0/1464575676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c11670 0x55ee95c1c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:54.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa12d19f640 1 -- 172.21.15.172:0/1464575676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c1c850 msgr2=0x55ee95c19980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:54.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa12d19f640 1 --2- 172.21.15.172:0/1464575676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c1c850 0x55ee95c19980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa12d19f640 1 -- 172.21.15.172:0/1464575676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c11670 msgr2=0x55ee95c1c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:54.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa12d19f640 1 --2- 172.21.15.172:0/1464575676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c11670 0x55ee95c1c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa12d19f640 1 -- 172.21.15.172:0/1464575676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee958e83c0 con 0x55ee95c11f90 2024-09-10T08:52:54.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa12c99e640 1 --2- 172.21.15.172:0/1464575676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c1c850 0x55ee95c19980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:52:54.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa12d19f640 1 --2- 172.21.15.172:0/1464575676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11f90 0x55ee95c19440 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fa124009340 tx=0x7fa1240316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e54fca40211dfb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:54.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa110ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:54.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa110ff9640 1 -- 172.21.15.172:0/1464575676 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12403f070 con 0x55ee95c11f90 2024-09-10T08:52:54.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa110ff9640 1 -- 172.21.15.172:0/1464575676 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa124004450 con 0x55ee95c11f90 2024-09-10T08:52:54.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa1317dc740 1 -- 172.21.15.172:0/1464575676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1180110d0 con 0x55ee95c11f90 2024-09-10T08:52:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa110ff9640 1 -- 172.21.15.172:0/1464575676 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa12404c050 con 0x55ee95c11f90 2024-09-10T08:52:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa1317dc740 10 client.17442 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.832+0000 7fa1317dc740 1 -- 172.21.15.172:0/1464575676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ee95c2add0 con 0x55ee95c11f90 2024-09-10T08:52:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.833+0000 7fa110ff9640 1 -- 172.21.15.172:0/1464575676 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1180110d0 con 0x55ee95c11f90 2024-09-10T08:52:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.833+0000 7fa110ff9640 1 -- 172.21.15.172:0/1464575676 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fa124004b10 con 0x55ee95c11f90 2024-09-10T08:52:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.833+0000 7fa1317dc740 10 client.17442 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.833+0000 7fa1317dc740 10 client.17442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.833+0000 7fa1317dc740 20 client.17442 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.833+0000 7fa1317dc740 1 --2- 172.21.15.172:0/1464575676 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ee95c2c290 0x55ee95c2e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.833+0000 7fa1317dc740 4 client.17442 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:52:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.833+0000 7fa1317dc740 1 -- 172.21.15.172:0/1464575676 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee95b84850 con 0x55ee95c2c290 2024-09-10T08:52:54.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.833+0000 7fa12c99e640 1 --2- 172.21.15.172:0/1464575676 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ee95c2c290 0x55ee95c2e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:54.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.833+0000 7fa110ff9640 10 client.17442 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:54.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.833+0000 7fa12c99e640 1 --2- 172.21.15.172:0/1464575676 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ee95c2c290 0x55ee95c2e6b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9180c69ff4dfab0b server_cookie=c10f5a52a9abeb9 in_seq=0 out_seq=0 2024-09-10T08:52:54.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.834+0000 7fa110ff9640 1 -- 172.21.15.172:0/1464575676 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee95b84850 con 0x55ee95c2c290 2024-09-10T08:52:54.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.834+0000 7fa110ff9640 10 client.17442 handle_command_reply: tid=0 2024-09-10T08:52:54.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.834+0000 7fa1317dc740 1 client.17442 shutdown 2024-09-10T08:52:54.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.834+0000 7fa1317dc740 1 -- 172.21.15.172:0/1464575676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11f90 msgr2=0x55ee95c19440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:54.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.834+0000 7fa1317dc740 1 --2- 172.21.15.172:0/1464575676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11f90 0x55ee95c19440 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fa124009340 tx=0x7fa1240316d0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.834+0000 7fa1317dc740 1 -- 172.21.15.172:0/1464575676 shutdown_connections 2024-09-10T08:52:54.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.834+0000 7fa1317dc740 1 --2- 172.21.15.172:0/1464575676 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ee95c2c290 0x55ee95c2e6b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.834+0000 7fa1317dc740 1 --2- 172.21.15.172:0/1464575676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c1c850 0x55ee95c19980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.834+0000 7fa1317dc740 1 --2- 172.21.15.172:0/1464575676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11f90 0x55ee95c19440 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.834+0000 7fa1317dc740 1 --2- 172.21.15.172:0/1464575676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c11670 0x55ee95c1c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.834+0000 7fa1317dc740 1 -- 172.21.15.172:0/1464575676 >> 172.21.15.172:0/1464575676 conn(0x55ee95c0dc00 msgr2=0x55ee95c13930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:54.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.834+0000 7fa110ff9640 0 client.17442 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:54.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.834+0000 7fa1317dc740 1 -- 172.21.15.172:0/1464575676 shutdown_connections 2024-09-10T08:52:54.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.835+0000 7fa1317dc740 1 -- 172.21.15.172:0/1464575676 wait complete. 2024-09-10T08:52:54.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.835+0000 7fa1317dc740 20 client.17442 trim_cache size 0 max 16384 2024-09-10T08:52:54.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.851+0000 7fa1317dc740 1 Processor -- start 2024-09-10T08:52:54.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.851+0000 7fa1317dc740 1 -- start start 2024-09-10T08:52:54.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa1317dc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11670 0x55ee95c28680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa1317dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c11f90 0x55ee95c33f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa1317dc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c1c850 0x55ee95c344c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa1317dc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281b4f80 con 0x55ee95c11670 2024-09-10T08:52:54.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa1317dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281b6ac0 con 0x55ee95c11f90 2024-09-10T08:52:54.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa1317dc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281b5650 con 0x55ee95c1c850 2024-09-10T08:52:54.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa12c99e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11670 0x55ee95c28680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:54.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa12c99e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11670 0x55ee95c28680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44930/0 (socket says 172.21.15.172:44930) 2024-09-10T08:52:54.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa12c99e640 1 -- 172.21.15.172:0/68242629 learned_addr learned my addr 172.21.15.172:0/68242629 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:54.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa12d19f640 1 --2- 172.21.15.172:0/68242629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c11f90 0x55ee95c33f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:54.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa12c99e640 1 -- 172.21.15.172:0/68242629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c1c850 msgr2=0x55ee95c344c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:54.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa12c99e640 1 --2- 172.21.15.172:0/68242629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c1c850 0x55ee95c344c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa12c99e640 1 -- 172.21.15.172:0/68242629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c11f90 msgr2=0x55ee95c33f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:54.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa12c99e640 1 --2- 172.21.15.172:0/68242629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c11f90 0x55ee95c33f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa12c99e640 1 -- 172.21.15.172:0/68242629 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee95c0da30 con 0x55ee95c11670 2024-09-10T08:52:54.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa12c99e640 1 --2- 172.21.15.172:0/68242629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11670 0x55ee95c28680 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fa118004990 tx=0x7fa1180049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c936e79946123b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:54.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa112ffd640 1 -- 172.21.15.172:0/68242629 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa118019040 con 0x55ee95c11670 2024-09-10T08:52:54.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa112ffd640 1 -- 172.21.15.172:0/68242629 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1180110b0 con 0x55ee95c11670 2024-09-10T08:52:54.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.852+0000 7fa112ffd640 1 -- 172.21.15.172:0/68242629 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11802b980 con 0x55ee95c11670 2024-09-10T08:52:54.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.853+0000 7fa1317dc740 1 -- 172.21.15.172:0/68242629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11670 msgr2=0x55ee95c28680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:54.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.853+0000 7fa1317dc740 1 --2- 172.21.15.172:0/68242629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11670 0x55ee95c28680 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fa118004990 tx=0x7fa1180049c0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.853+0000 7fa1317dc740 1 -- 172.21.15.172:0/68242629 shutdown_connections 2024-09-10T08:52:54.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.853+0000 7fa1317dc740 1 --2- 172.21.15.172:0/68242629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c1c850 0x55ee95c344c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.853+0000 7fa1317dc740 1 --2- 172.21.15.172:0/68242629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c11f90 0x55ee95c33f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.853+0000 7fa1317dc740 1 --2- 172.21.15.172:0/68242629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11670 0x55ee95c28680 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.853+0000 7fa1317dc740 1 -- 172.21.15.172:0/68242629 >> 172.21.15.172:0/68242629 conn(0x55ee95c0dc00 msgr2=0x55ee95c1b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.853+0000 7fa1317dc740 1 -- 172.21.15.172:0/68242629 shutdown_connections 2024-09-10T08:52:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.853+0000 7fa1317dc740 1 -- 172.21.15.172:0/68242629 wait complete. 2024-09-10T08:52:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.853+0000 7fa1317dc740 1 -- start start 2024-09-10T08:52:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa1317dc740 1 Processor -- start 2024-09-10T08:52:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa1317dc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c11670 0x55ee95c24d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa1317dc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11f90 0x55ee95c220c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa1317dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c1c850 0x55ee95c22600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa1317dc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281b63f0 con 0x55ee95c11f90 2024-09-10T08:52:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa1317dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281b5d20 con 0x55ee95c1c850 2024-09-10T08:52:54.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa1317dc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281b2f60 con 0x55ee95c11670 2024-09-10T08:52:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa1317dc740 10 client.0 _ll_register_callbacks cb 0x7fa124009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:52:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa12d19f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11f90 0x55ee95c220c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa12d19f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11f90 0x55ee95c220c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44934/0 (socket says 172.21.15.172:44934) 2024-09-10T08:52:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa12d19f640 1 -- 172.21.15.172:0/1921900171 learned_addr learned my addr 172.21.15.172:0/1921900171 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:52:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa12c99e640 1 --2- 172.21.15.172:0/1921900171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c11670 0x55ee95c24d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa12c99e640 1 --2- 172.21.15.172:0/1921900171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c1c850 0x55ee95c22600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa12d19f640 1 -- 172.21.15.172:0/1921900171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c11670 msgr2=0x55ee95c24d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa12d19f640 1 --2- 172.21.15.172:0/1921900171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c11670 0x55ee95c24d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa12d19f640 1 -- 172.21.15.172:0/1921900171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c1c850 msgr2=0x55ee95c22600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:52:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa12d19f640 1 --2- 172.21.15.172:0/1921900171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c1c850 0x55ee95c22600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa12d19f640 1 -- 172.21.15.172:0/1921900171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee95c22bf0 con 0x55ee95c11f90 2024-09-10T08:52:54.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.854+0000 7fa12c99e640 1 --2- 172.21.15.172:0/1921900171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c11670 0x55ee95c24d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:52:54.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.855+0000 7fa12d19f640 1 --2- 172.21.15.172:0/1921900171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11f90 0x55ee95c220c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fa124001eb0 tx=0x7fa124031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc9e77323d0334 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:52:54.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.855+0000 7fa110ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:52:54.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.855+0000 7fa110ff9640 1 -- 172.21.15.172:0/1921900171 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12403f070 con 0x55ee95c11f90 2024-09-10T08:52:54.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.855+0000 7fa110ff9640 1 -- 172.21.15.172:0/1921900171 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa124003850 con 0x55ee95c11f90 2024-09-10T08:52:54.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.855+0000 7fa1317dc740 1 -- 172.21.15.172:0/1921900171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1180110b0 con 0x55ee95c11f90 2024-09-10T08:52:54.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.855+0000 7fa110ff9640 1 -- 172.21.15.172:0/1921900171 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa124060050 con 0x55ee95c11f90 2024-09-10T08:52:54.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.855+0000 7fa1317dc740 10 client.17448 fetch_fsmap learned FSMap version 37 2024-09-10T08:52:54.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.855+0000 7fa1317dc740 1 -- 172.21.15.172:0/1921900171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ee95c96d50 con 0x55ee95c11f90 2024-09-10T08:52:54.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.855+0000 7fa110ff9640 1 -- 172.21.15.172:0/1921900171 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1180110b0 con 0x55ee95c11f90 2024-09-10T08:52:54.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.855+0000 7fa110ff9640 1 -- 172.21.15.172:0/1921900171 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fa124004a70 con 0x55ee95c11f90 2024-09-10T08:52:54.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.855+0000 7fa1317dc740 10 client.17448 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:52:54.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.855+0000 7fa1317dc740 10 client.17448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:52:54.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.855+0000 7fa1317dc740 20 client.17448 populate_metadata read hostname 'smithi172' 2024-09-10T08:52:54.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.855+0000 7fa1317dc740 1 --2- 172.21.15.172:0/1921900171 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ee95c97500 0x55ee95c99920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:52:54.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.855+0000 7fa1317dc740 4 client.17448 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:52:54.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.855+0000 7fa1317dc740 1 -- 172.21.15.172:0/1921900171 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee95b7ed60 con 0x55ee95c97500 2024-09-10T08:52:54.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.856+0000 7fa12c99e640 1 --2- 172.21.15.172:0/1921900171 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ee95c97500 0x55ee95c99920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:52:54.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.856+0000 7fa110ff9640 10 client.17448 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:54.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.856+0000 7fa12c99e640 1 --2- 172.21.15.172:0/1921900171 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ee95c97500 0x55ee95c99920 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fc100b39127c943 server_cookie=91d7f40baa50cef7 in_seq=0 out_seq=0 2024-09-10T08:52:54.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.856+0000 7fa110ff9640 1 -- 172.21.15.172:0/1921900171 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ee95b7ed60 con 0x55ee95c97500 2024-09-10T08:52:54.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.856+0000 7fa110ff9640 10 client.17448 handle_command_reply: tid=0 2024-09-10T08:52:54.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.856+0000 7fa1317dc740 1 client.17448 shutdown 2024-09-10T08:52:54.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.856+0000 7fa1317dc740 1 -- 172.21.15.172:0/1921900171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11f90 msgr2=0x55ee95c220c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:54.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.856+0000 7fa1317dc740 1 --2- 172.21.15.172:0/1921900171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11f90 0x55ee95c220c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fa124001eb0 tx=0x7fa124031970 comp rx=0 tx=0).stop 2024-09-10T08:52:54.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa1317dc740 1 -- 172.21.15.172:0/1921900171 shutdown_connections 2024-09-10T08:52:54.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa1317dc740 1 --2- 172.21.15.172:0/1921900171 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ee95c97500 0x55ee95c99920 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa1317dc740 1 --2- 172.21.15.172:0/1921900171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee95c1c850 0x55ee95c22600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa1317dc740 1 --2- 172.21.15.172:0/1921900171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee95c11f90 0x55ee95c220c0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa1317dc740 1 --2- 172.21.15.172:0/1921900171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee95c11670 0x55ee95c24d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa1317dc740 1 -- 172.21.15.172:0/1921900171 >> 172.21.15.172:0/1921900171 conn(0x55ee95c0dc00 msgr2=0x55ee95c0c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:54.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa110ff9640 0 client.17448 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:52:54.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa1317dc740 1 -- 172.21.15.172:0/1921900171 shutdown_connections 2024-09-10T08:52:54.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa1317dc740 1 -- 172.21.15.172:0/1921900171 wait complete. 2024-09-10T08:52:54.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa1317dc740 20 client.17448 trim_cache size 0 max 16384 2024-09-10T08:52:54.863 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:52:54.864 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:52:54.864 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T08:52:54.864 INFO:teuthology.orchestra.run.smithi172.stdout: "9724cbd5-b97e-4410-8c69-bbfd75a9823b": { 2024-09-10T08:52:54.864 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T08:52:54.864 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "9724cbd5-b97e-4410-8c69-bbfd75a9823b", 2024-09-10T08:52:54.864 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T08:52:54.864 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:52:54.864 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:52:54.864 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:52:54.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa12e1a1640 1 -- 172.21.15.172:0/1787916099 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa1140a2c00 msgr2=0x7fa1140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:54.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa12e1a1640 1 --2- 172.21.15.172:0/1787916099 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa1140a2c00 0x7fa1140a50c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa12009b7f0 tx=0x7fa12001baf0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa12e1a1640 1 -- 172.21.15.172:0/1787916099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa128105d90 msgr2=0x7fa12819e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:52:54.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa12e1a1640 1 --2- 172.21.15.172:0/1787916099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa128105d90 0x7fa12819e660 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fa12000cad0 tx=0x7fa12000cf90 comp rx=0 tx=0).stop 2024-09-10T08:52:54.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa12e1a1640 1 -- 172.21.15.172:0/1787916099 shutdown_connections 2024-09-10T08:52:54.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa12e1a1640 1 --2- 172.21.15.172:0/1787916099 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa1140a2c00 0x7fa1140a50c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa12e1a1640 1 --2- 172.21.15.172:0/1787916099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa128105d90 0x7fa12819e660 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa12e1a1640 1 --2- 172.21.15.172:0/1787916099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1281012a0 0x7fa1281a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa12e1a1640 1 --2- 172.21.15.172:0/1787916099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa128100980 0x7fa1281a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:52:54.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa12e1a1640 1 -- 172.21.15.172:0/1787916099 >> 172.21.15.172:0/1787916099 conn(0x7fa1280fc630 msgr2=0x7fa12810f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:52:54.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.857+0000 7fa12e1a1640 1 -- 172.21.15.172:0/1787916099 shutdown_connections 2024-09-10T08:52:54.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:52:54.858+0000 7fa12e1a1640 1 -- 172.21.15.172:0/1787916099 wait complete. 2024-09-10T08:52:55.687 INFO:tasks.cephfs.filesystem:scrub status for tag:9724cbd5-b97e-4410-8c69-bbfd75a9823b - {'path': '/', 'tag': '9724cbd5-b97e-4410-8c69-bbfd75a9823b', 'options': 'recursive,force'} 2024-09-10T08:52:56.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:56 smithi172 ceph-mon[30854]: pgmap v551: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.9 MiB/s wr, 33 op/s 2024-09-10T08:52:56.744 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:56 smithi174 ceph-mon[34068]: pgmap v551: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.9 MiB/s wr, 33 op/s 2024-09-10T08:52:56.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:56 smithi191 ceph-mon[35314]: pgmap v551: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.9 MiB/s wr, 33 op/s 2024-09-10T08:52:58.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:52:58 smithi172 ceph-mon[30854]: pgmap v552: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.9 MiB/s wr, 35 op/s 2024-09-10T08:52:58.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:52:58 smithi174 ceph-mon[34068]: pgmap v552: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.9 MiB/s wr, 35 op/s 2024-09-10T08:52:58.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:52:58 smithi191 ceph-mon[35314]: pgmap v552: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1.9 MiB/s wr, 35 op/s 2024-09-10T08:53:00.210 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:53:00.474 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:53:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:00 smithi172 ceph-mon[30854]: pgmap v553: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 585 KiB/s wr, 34 op/s 2024-09-10T08:53:00.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:00 smithi174 ceph-mon[34068]: pgmap v553: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 585 KiB/s wr, 34 op/s 2024-09-10T08:53:00.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:00 smithi191 ceph-mon[35314]: pgmap v553: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 585 KiB/s wr, 34 op/s 2024-09-10T08:53:00.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.959+0000 7f4460470640 1 -- 172.21.15.172:0/968591606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4458102da0 msgr2=0x7f4458103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:00.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.959+0000 7f4460470640 1 --2- 172.21.15.172:0/968591606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4458102da0 0x7f4458103180 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f4450009900 tx=0x7f445002a480 comp rx=0 tx=0).stop 2024-09-10T08:53:00.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.959+0000 7f4460470640 1 -- 172.21.15.172:0/968591606 shutdown_connections 2024-09-10T08:53:00.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.959+0000 7f4460470640 1 --2- 172.21.15.172:0/968591606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f445810c180 0x7f445810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:00.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.959+0000 7f4460470640 1 --2- 172.21.15.172:0/968591606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44581036c0 0x7f445810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:00.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.959+0000 7f4460470640 1 --2- 172.21.15.172:0/968591606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4458102da0 0x7f4458103180 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:00.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.959+0000 7f4460470640 1 -- 172.21.15.172:0/968591606 >> 172.21.15.172:0/968591606 conn(0x7f44580fc7a0 msgr2=0x7f44580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:00.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.959+0000 7f4460470640 1 -- 172.21.15.172:0/968591606 shutdown_connections 2024-09-10T08:53:00.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.959+0000 7f4460470640 1 -- 172.21.15.172:0/968591606 wait complete. 2024-09-10T08:53:00.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f4460470640 1 Processor -- start 2024-09-10T08:53:00.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f4460470640 1 -- start start 2024-09-10T08:53:00.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f4460470640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4458102da0 0x7f4458100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:00.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f4460470640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44581036c0 0x7f4458100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:00.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f4460470640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f445810c180 0x7f4458101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:00.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f4460470640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4458111e10 con 0x7f445810c180 2024-09-10T08:53:00.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f4460470640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4458111c90 con 0x7f44581036c0 2024-09-10T08:53:00.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f4460470640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4458111f90 con 0x7f4458102da0 2024-09-10T08:53:00.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f445ea0d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f445810c180 0x7f4458101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:00.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f445ea0d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f445810c180 0x7f4458101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37522/0 (socket says 172.21.15.172:37522) 2024-09-10T08:53:00.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f445ea0d640 1 -- 172.21.15.172:0/3659667529 learned_addr learned my addr 172.21.15.172:0/3659667529 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:00.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f445e20c640 1 --2- 172.21.15.172:0/3659667529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4458102da0 0x7f4458100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:00.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f445da0b640 1 --2- 172.21.15.172:0/3659667529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44581036c0 0x7f4458100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:00.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f445ea0d640 1 -- 172.21.15.172:0/3659667529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4458102da0 msgr2=0x7f4458100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:00.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f445ea0d640 1 --2- 172.21.15.172:0/3659667529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4458102da0 0x7f4458100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:00.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f445ea0d640 1 -- 172.21.15.172:0/3659667529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44581036c0 msgr2=0x7f4458100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:00.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f445ea0d640 1 --2- 172.21.15.172:0/3659667529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44581036c0 0x7f4458100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:00.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f445ea0d640 1 -- 172.21.15.172:0/3659667529 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44581a6870 con 0x7f445810c180 2024-09-10T08:53:00.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.960+0000 7f445ea0d640 1 --2- 172.21.15.172:0/3659667529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f445810c180 0x7f4458101480 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f444800d950 tx=0x7f444800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:00.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.961+0000 7f44577fe640 1 -- 172.21.15.172:0/3659667529 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448017030 con 0x7f445810c180 2024-09-10T08:53:00.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.961+0000 7f44577fe640 1 -- 172.21.15.172:0/3659667529 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4448004020 con 0x7f445810c180 2024-09-10T08:53:00.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.961+0000 7f4460470640 1 -- 172.21.15.172:0/3659667529 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44581a6b60 con 0x7f445810c180 2024-09-10T08:53:00.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.961+0000 7f44577fe640 1 -- 172.21.15.172:0/3659667529 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448003b20 con 0x7f445810c180 2024-09-10T08:53:00.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.961+0000 7f4460470640 1 -- 172.21.15.172:0/3659667529 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44581a70a0 con 0x7f445810c180 2024-09-10T08:53:00.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.962+0000 7f4460470640 1 -- 172.21.15.172:0/3659667529 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4458103180 con 0x7f445810c180 2024-09-10T08:53:00.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.963+0000 7f44577fe640 1 -- 172.21.15.172:0/3659667529 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f444801a070 con 0x7f445810c180 2024-09-10T08:53:00.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.964+0000 7f44577fe640 1 --2- 172.21.15.172:0/3659667529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44400a2c00 0x7f44400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:00.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.964+0000 7f445e20c640 1 --2- 172.21.15.172:0/3659667529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44400a2c00 0x7f44400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:00.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.964+0000 7f44577fe640 1 -- 172.21.15.172:0/3659667529 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f44480ceae0 con 0x7f445810c180 2024-09-10T08:53:00.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.964+0000 7f445e20c640 1 --2- 172.21.15.172:0/3659667529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44400a2c00 0x7f44400a50c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4450031440 tx=0x7f4450005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:00.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:00.968+0000 7f44577fe640 1 -- 172.21.15.172:0/3659667529 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4448021070 con 0x7f445810c180 2024-09-10T08:53:01.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:01.172+0000 7f4460470640 1 -- 172.21.15.172:0/3659667529 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4458107220 con 0x7f445810c180 2024-09-10T08:53:01.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:01.173+0000 7f44577fe640 1 -- 172.21.15.172:0/3659667529 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+12001 (secure 0 0 0) 0x7f444809b6e0 con 0x7f445810c180 2024-09-10T08:53:01.174 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:53:01.174 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":37,"btime":"2024-09-10T08:52:08:313493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:52:07.658341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:53:01.174 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 37 2024-09-10T08:53:01.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:01.175+0000 7f4460470640 1 -- 172.21.15.172:0/3659667529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44400a2c00 msgr2=0x7f44400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:01.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:01.175+0000 7f4460470640 1 --2- 172.21.15.172:0/3659667529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44400a2c00 0x7f44400a50c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4450031440 tx=0x7f4450005b50 comp rx=0 tx=0).stop 2024-09-10T08:53:01.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:01.175+0000 7f4460470640 1 -- 172.21.15.172:0/3659667529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f445810c180 msgr2=0x7f4458101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:01.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:01.175+0000 7f4460470640 1 --2- 172.21.15.172:0/3659667529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f445810c180 0x7f4458101480 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f444800d950 tx=0x7f444800de10 comp rx=0 tx=0).stop 2024-09-10T08:53:01.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:01.175+0000 7f4460470640 1 -- 172.21.15.172:0/3659667529 shutdown_connections 2024-09-10T08:53:01.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:01.175+0000 7f4460470640 1 --2- 172.21.15.172:0/3659667529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44400a2c00 0x7f44400a50c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:01.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:01.175+0000 7f4460470640 1 --2- 172.21.15.172:0/3659667529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f445810c180 0x7f4458101480 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:01.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:01.175+0000 7f4460470640 1 --2- 172.21.15.172:0/3659667529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44581036c0 0x7f4458100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:01.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:01.175+0000 7f4460470640 1 --2- 172.21.15.172:0/3659667529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4458102da0 0x7f4458100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:01.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:01.175+0000 7f4460470640 1 -- 172.21.15.172:0/3659667529 >> 172.21.15.172:0/3659667529 conn(0x7f44580fc7a0 msgr2=0x7f44580fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:01.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:01.175+0000 7f4460470640 1 -- 172.21.15.172:0/3659667529 shutdown_connections 2024-09-10T08:53:01.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:01.175+0000 7f4460470640 1 -- 172.21.15.172:0/3659667529 wait complete. 2024-09-10T08:53:01.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:01 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3659667529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:53:01.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:01 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3659667529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:53:01.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:01 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3659667529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:53:01.966 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id bd6b2e81 --release --await' 2024-09-10T08:53:01.967 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id bd6b2e81 --release --await 2024-09-10T08:53:02.217 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:53:02.648 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:02 smithi172 ceph-mon[30854]: pgmap v554: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 244 KiB/s wr, 36 op/s 2024-09-10T08:53:02.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:02 smithi174 ceph-mon[34068]: pgmap v554: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 244 KiB/s wr, 36 op/s 2024-09-10T08:53:02.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.732+0000 7f85aee9a640 1 -- 172.21.15.172:0/222157947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85a81027a0 msgr2=0x7f85a81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:02.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.732+0000 7f85aee9a640 1 --2- 172.21.15.172:0/222157947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85a81027a0 0x7f85a81067f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8590009900 tx=0x7f859002a410 comp rx=0 tx=0).stop 2024-09-10T08:53:02.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.733+0000 7f85aee9a640 1 -- 172.21.15.172:0/222157947 shutdown_connections 2024-09-10T08:53:02.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.733+0000 7f85aee9a640 1 --2- 172.21.15.172:0/222157947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85a8106f20 0x7f85a810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.733+0000 7f85aee9a640 1 --2- 172.21.15.172:0/222157947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85a81027a0 0x7f85a81067f0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.733+0000 7f85aee9a640 1 --2- 172.21.15.172:0/222157947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85a8101df0 0x7f85a81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.733+0000 7f85aee9a640 1 -- 172.21.15.172:0/222157947 >> 172.21.15.172:0/222157947 conn(0x7f85a80fd8f0 msgr2=0x7f85a80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:02.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.733+0000 7f85aee9a640 1 -- 172.21.15.172:0/222157947 shutdown_connections 2024-09-10T08:53:02.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.733+0000 7f85aee9a640 1 -- 172.21.15.172:0/222157947 wait complete. 2024-09-10T08:53:02.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.733+0000 7f85aee9a640 1 Processor -- start 2024-09-10T08:53:02.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.733+0000 7f85aee9a640 1 -- start start 2024-09-10T08:53:02.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.733+0000 7f85aee9a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85a8101df0 0x7f85a819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.733+0000 7f85aee9a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85a81027a0 0x7f85a819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.733+0000 7f85aee9a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85a8106f20 0x7f85a819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.733+0000 7f85aee9a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8111e30 con 0x7f85a81027a0 2024-09-10T08:53:02.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.733+0000 7f85aee9a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8111cb0 con 0x7f85a8101df0 2024-09-10T08:53:02.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.733+0000 7f85aee9a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8111fb0 con 0x7f85a8106f20 2024-09-10T08:53:02.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.734+0000 7f85a7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85a81027a0 0x7f85a819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:02.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.734+0000 7f85a7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85a81027a0 0x7f85a819aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37548/0 (socket says 172.21.15.172:37548) 2024-09-10T08:53:02.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.734+0000 7f85a7fff640 1 -- 172.21.15.172:0/3793261486 learned_addr learned my addr 172.21.15.172:0/3793261486 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:02.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.734+0000 7f85ad437640 1 --2- 172.21.15.172:0/3793261486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85a8106f20 0x7f85a819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:02.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.734+0000 7f85acc36640 1 --2- 172.21.15.172:0/3793261486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85a8101df0 0x7f85a819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:02.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.734+0000 7f85a7fff640 1 -- 172.21.15.172:0/3793261486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85a8106f20 msgr2=0x7f85a819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:02.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.734+0000 7f85a7fff640 1 --2- 172.21.15.172:0/3793261486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85a8106f20 0x7f85a819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.734+0000 7f85a7fff640 1 -- 172.21.15.172:0/3793261486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85a8101df0 msgr2=0x7f85a819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:02.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.734+0000 7f85a7fff640 1 --2- 172.21.15.172:0/3793261486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85a8101df0 0x7f85a819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.734+0000 7f85a7fff640 1 -- 172.21.15.172:0/3793261486 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a819f540 con 0x7f85a81027a0 2024-09-10T08:53:02.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.734+0000 7f85acc36640 1 --2- 172.21.15.172:0/3793261486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85a8101df0 0x7f85a819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:02.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.734+0000 7f85a7fff640 1 --2- 172.21.15.172:0/3793261486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85a81027a0 0x7f85a819aad0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8590005940 tx=0x7f8590004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:02.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.734+0000 7f85a5ffb640 1 -- 172.21.15.172:0/3793261486 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859003f030 con 0x7f85a81027a0 2024-09-10T08:53:02.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.735+0000 7f85aee9a640 1 -- 172.21.15.172:0/3793261486 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a819f7d0 con 0x7f85a81027a0 2024-09-10T08:53:02.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.735+0000 7f85a5ffb640 1 -- 172.21.15.172:0/3793261486 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8590003aa0 con 0x7f85a81027a0 2024-09-10T08:53:02.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.735+0000 7f85a5ffb640 1 -- 172.21.15.172:0/3793261486 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859003f030 con 0x7f85a81027a0 2024-09-10T08:53:02.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.735+0000 7f85aee9a640 1 -- 172.21.15.172:0/3793261486 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a81a7110 con 0x7f85a81027a0 2024-09-10T08:53:02.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.737+0000 7f85a5ffb640 1 -- 172.21.15.172:0/3793261486 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8590042030 con 0x7f85a81027a0 2024-09-10T08:53:02.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.737+0000 7f85b1273740 1 Processor -- start 2024-09-10T08:53:02.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.737+0000 7f85b1273740 1 -- start start 2024-09-10T08:53:02.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85b1273740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed060 0x55b0b15ed440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85b1273740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15ed980 0x55b0b15f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85b1273740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15f8240 0x55b0b15fa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85b1273740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b0b1552620 con 0x55b0b15ed060 2024-09-10T08:53:02.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85b1273740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b0b14f9e50 con 0x55b0b15f8240 2024-09-10T08:53:02.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85b1273740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b0b154e5e0 con 0x55b0b15ed980 2024-09-10T08:53:02.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85a5ffb640 1 --2- 172.21.15.172:0/3793261486 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f85940a2e00 0x7f85940a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85ad437640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed060 0x55b0b15ed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:02.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85acc36640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15ed980 0x55b0b15f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:02.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85ad437640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed060 0x55b0b15ed440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37558/0 (socket says 172.21.15.172:37558) 2024-09-10T08:53:02.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85ad437640 1 -- 172.21.15.172:0/1460127648 learned_addr learned my addr 172.21.15.172:0/1460127648 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:02.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85ad437640 1 --2- 172.21.15.172:0/3793261486 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f85940a2e00 0x7f85940a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:02.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85a5ffb640 1 -- 172.21.15.172:0/3793261486 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f85900ef460 con 0x7f85a81027a0 2024-09-10T08:53:02.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85a7fff640 1 --2- 172.21.15.172:0/1460127648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15f8240 0x55b0b15fa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:02.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85acc36640 1 -- 172.21.15.172:0/1460127648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15f8240 msgr2=0x55b0b15fa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:02.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85acc36640 1 --2- 172.21.15.172:0/1460127648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15f8240 0x55b0b15fa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85acc36640 1 -- 172.21.15.172:0/1460127648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed060 msgr2=0x55b0b15ed440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:02.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85acc36640 1 --2- 172.21.15.172:0/1460127648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed060 0x55b0b15ed440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85acc36640 1 -- 172.21.15.172:0/1460127648 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0b14ead40 con 0x55b0b15ed980 2024-09-10T08:53:02.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85a7fff640 1 --2- 172.21.15.172:0/1460127648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15f8240 0x55b0b15fa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:02.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.738+0000 7f85acc36640 1 --2- 172.21.15.172:0/1460127648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15ed980 0x55b0b15f7d00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f85a000cbd0 tx=0x7f85a000e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7047dadbb9c5515d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:02.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.739+0000 7f85ad437640 1 --2- 172.21.15.172:0/3793261486 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f85940a2e00 0x7f85940a52c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f85980098b0 tx=0x7f8598009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:02.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.739+0000 7f859effd640 1 -- 172.21.15.172:0/1460127648 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0005220 con 0x55b0b15ed980 2024-09-10T08:53:02.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.739+0000 7f859effd640 1 -- 172.21.15.172:0/1460127648 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85a0004c50 con 0x55b0b15ed980 2024-09-10T08:53:02.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.739+0000 7f859effd640 1 -- 172.21.15.172:0/1460127648 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0010720 con 0x55b0b15ed980 2024-09-10T08:53:02.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.739+0000 7f85b1273740 1 -- 172.21.15.172:0/1460127648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15ed980 msgr2=0x55b0b15f7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:02.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.739+0000 7f85b1273740 1 --2- 172.21.15.172:0/1460127648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15ed980 0x55b0b15f7d00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f85a000cbd0 tx=0x7f85a000e720 comp rx=0 tx=0).stop 2024-09-10T08:53:02.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.739+0000 7f85b1273740 1 -- 172.21.15.172:0/1460127648 shutdown_connections 2024-09-10T08:53:02.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.739+0000 7f85b1273740 1 --2- 172.21.15.172:0/1460127648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15f8240 0x55b0b15fa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.739+0000 7f85b1273740 1 --2- 172.21.15.172:0/1460127648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15ed980 0x55b0b15f7d00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.739+0000 7f85b1273740 1 --2- 172.21.15.172:0/1460127648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed060 0x55b0b15ed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.739+0000 7f85b1273740 1 -- 172.21.15.172:0/1460127648 >> 172.21.15.172:0/1460127648 conn(0x55b0b15e95f0 msgr2=0x55b0b15e99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:02.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.740+0000 7f85b1273740 1 -- 172.21.15.172:0/1460127648 shutdown_connections 2024-09-10T08:53:02.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.740+0000 7f85b1273740 1 -- 172.21.15.172:0/1460127648 wait complete. 2024-09-10T08:53:02.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.740+0000 7f85b1273740 1 -- start start 2024-09-10T08:53:02.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.740+0000 7f85b1273740 1 Processor -- start 2024-09-10T08:53:02.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.740+0000 7f85b1273740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15ed060 0x55b0b15f5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.740+0000 7f85b1273740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15ed980 0x55b0b15f5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.740+0000 7f85b1273740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15f8240 0x55b0b15f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.740+0000 7f85b1273740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b0b15fae10 con 0x55b0b15f8240 2024-09-10T08:53:02.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.740+0000 7f85b1273740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b0b14daf10 con 0x55b0b15ed060 2024-09-10T08:53:02.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.740+0000 7f85b1273740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b0b15faf90 con 0x55b0b15ed980 2024-09-10T08:53:02.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.740+0000 7f85b1273740 10 client.0 _ll_register_callbacks cb 0x55b0b14fedb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:02.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.740+0000 7f85ad437640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15f8240 0x55b0b15f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:02.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.740+0000 7f85a7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15ed980 0x55b0b15f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:02.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.740+0000 7f85ad437640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15f8240 0x55b0b15f5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37560/0 (socket says 172.21.15.172:37560) 2024-09-10T08:53:02.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.741+0000 7f85ad437640 1 -- 172.21.15.172:0/2913840765 learned_addr learned my addr 172.21.15.172:0/2913840765 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:02.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.741+0000 7f85acc36640 1 --2- 172.21.15.172:0/2913840765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15ed060 0x55b0b15f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:02.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.741+0000 7f85ad437640 1 -- 172.21.15.172:0/2913840765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15ed980 msgr2=0x55b0b15f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:02.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.741+0000 7f85ad437640 1 --2- 172.21.15.172:0/2913840765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15ed980 0x55b0b15f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.741+0000 7f85ad437640 1 -- 172.21.15.172:0/2913840765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15ed060 msgr2=0x55b0b15f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:02.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.741+0000 7f85ad437640 1 --2- 172.21.15.172:0/2913840765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15ed060 0x55b0b15f5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.741+0000 7f85ad437640 1 -- 172.21.15.172:0/2913840765 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0b1261580 con 0x55b0b15f8240 2024-09-10T08:53:02.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.741+0000 7f85ad437640 1 --2- 172.21.15.172:0/2913840765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15f8240 0x55b0b15f5ba0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8598012a00 tx=0x7f8598004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3a353b9213d54 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:02.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.741+0000 7f8587fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:02.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.741+0000 7f8587fff640 1 -- 172.21.15.172:0/2913840765 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859801ed00 con 0x55b0b15f8240 2024-09-10T08:53:02.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.741+0000 7f8587fff640 1 -- 172.21.15.172:0/2913840765 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8598004b20 con 0x55b0b15f8240 2024-09-10T08:53:02.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.741+0000 7f85b1273740 1 -- 172.21.15.172:0/2913840765 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a0004c50 con 0x55b0b15f8240 2024-09-10T08:53:02.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.741+0000 7f8587fff640 1 -- 172.21.15.172:0/2913840765 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8598004d20 con 0x55b0b15f8240 2024-09-10T08:53:02.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.741+0000 7f85b1273740 10 client.17466 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:02.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.741+0000 7f85b1273740 1 -- 172.21.15.172:0/2913840765 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b0b132b960 con 0x55b0b15f8240 2024-09-10T08:53:02.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.742+0000 7f8587fff640 1 -- 172.21.15.172:0/2913840765 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85a0004c50 con 0x55b0b15f8240 2024-09-10T08:53:02.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.742+0000 7f8587fff640 1 -- 172.21.15.172:0/2913840765 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f859800f040 con 0x55b0b15f8240 2024-09-10T08:53:02.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.742+0000 7f85b1273740 10 client.17466 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:02.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.742+0000 7f85b1273740 10 client.17466 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:53:02.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.742+0000 7f85b1273740 20 client.17466 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:02.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.742+0000 7f85b1273740 1 --2- 172.21.15.172:0/2913840765 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b0b1607dd0 0x55b0b160a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.742+0000 7f85b1273740 4 client.17466 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:53:02.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.742+0000 7f85b1273740 1 -- 172.21.15.172:0/2913840765 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0b155fb90 con 0x55b0b1607dd0 2024-09-10T08:53:02.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.742+0000 7f85acc36640 1 --2- 172.21.15.172:0/2913840765 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b0b1607dd0 0x55b0b160a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:02.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.743+0000 7f8587fff640 10 client.17466 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:02.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.743+0000 7f85acc36640 1 --2- 172.21.15.172:0/2913840765 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b0b1607dd0 0x55b0b160a240 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56cc26a08ec8ab86 server_cookie=3cf4337194eaf6c7 in_seq=0 out_seq=0 2024-09-10T08:53:02.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.743+0000 7f8587fff640 1 -- 172.21.15.172:0/2913840765 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0b155fb90 con 0x55b0b1607dd0 2024-09-10T08:53:02.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.743+0000 7f8587fff640 10 client.17466 handle_command_reply: tid=0 2024-09-10T08:53:02.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.743+0000 7f85b1273740 1 client.17466 shutdown 2024-09-10T08:53:02.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.743+0000 7f85b1273740 1 -- 172.21.15.172:0/2913840765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15f8240 msgr2=0x55b0b15f5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:02.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.743+0000 7f85b1273740 1 --2- 172.21.15.172:0/2913840765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15f8240 0x55b0b15f5ba0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8598012a00 tx=0x7f8598004a40 comp rx=0 tx=0).stop 2024-09-10T08:53:02.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.743+0000 7f85b1273740 1 -- 172.21.15.172:0/2913840765 shutdown_connections 2024-09-10T08:53:02.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.743+0000 7f85b1273740 1 --2- 172.21.15.172:0/2913840765 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b0b1607dd0 0x55b0b160a240 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.743+0000 7f85b1273740 1 --2- 172.21.15.172:0/2913840765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15f8240 0x55b0b15f5ba0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.743+0000 7f85b1273740 1 --2- 172.21.15.172:0/2913840765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15ed980 0x55b0b15f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.743+0000 7f85b1273740 1 --2- 172.21.15.172:0/2913840765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15ed060 0x55b0b15f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.743+0000 7f8587fff640 0 client.17466 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:02.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.743+0000 7f85b1273740 1 -- 172.21.15.172:0/2913840765 >> 172.21.15.172:0/2913840765 conn(0x55b0b15e95f0 msgr2=0x55b0b15ef320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:02.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.743+0000 7f85b1273740 1 -- 172.21.15.172:0/2913840765 shutdown_connections 2024-09-10T08:53:02.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.744+0000 7f85b1273740 1 -- 172.21.15.172:0/2913840765 wait complete. 2024-09-10T08:53:02.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.744+0000 7f85b1273740 20 client.17466 trim_cache size 0 max 16384 2024-09-10T08:53:02.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.760+0000 7f85b1273740 1 Processor -- start 2024-09-10T08:53:02.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.760+0000 7f85b1273740 1 -- start start 2024-09-10T08:53:02.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f85b1273740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed060 0x55b0b15e8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f85b1273740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15ed980 0x55b0b160fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f85b1273740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15f8240 0x55b0b1610050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.764 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f85b1273740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81b0de0 con 0x55b0b15ed060 2024-09-10T08:53:02.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f85b1273740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81b2920 con 0x55b0b15f8240 2024-09-10T08:53:02.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f85b1273740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81b14b0 con 0x55b0b15ed980 2024-09-10T08:53:02.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f85acc36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed060 0x55b0b15e8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:02.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f85acc36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed060 0x55b0b15e8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37574/0 (socket says 172.21.15.172:37574) 2024-09-10T08:53:02.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f85acc36640 1 -- 172.21.15.172:0/3332113925 learned_addr learned my addr 172.21.15.172:0/3332113925 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:02.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f85ad437640 1 --2- 172.21.15.172:0/3332113925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15f8240 0x55b0b1610050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:02.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f85acc36640 1 -- 172.21.15.172:0/3332113925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15ed980 msgr2=0x55b0b160fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:02.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f85acc36640 1 --2- 172.21.15.172:0/3332113925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15ed980 0x55b0b160fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f85acc36640 1 -- 172.21.15.172:0/3332113925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15f8240 msgr2=0x55b0b1610050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:02.765 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f85acc36640 1 --2- 172.21.15.172:0/3332113925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15f8240 0x55b0b1610050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f85acc36640 1 -- 172.21.15.172:0/3332113925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0b160a8c0 con 0x55b0b15ed060 2024-09-10T08:53:02.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f85ad437640 1 --2- 172.21.15.172:0/3332113925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15f8240 0x55b0b1610050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:53:02.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f85acc36640 1 --2- 172.21.15.172:0/3332113925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed060 0x55b0b15e8640 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f85a000f010 tx=0x7f85a001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3305b12caaf06b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:02.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f859e7fc640 1 -- 172.21.15.172:0/3332113925 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0004220 con 0x55b0b15ed060 2024-09-10T08:53:02.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f859e7fc640 1 -- 172.21.15.172:0/3332113925 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85a00043c0 con 0x55b0b15ed060 2024-09-10T08:53:02.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.761+0000 7f859e7fc640 1 -- 172.21.15.172:0/3332113925 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a001a6c0 con 0x55b0b15ed060 2024-09-10T08:53:02.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.762+0000 7f85b1273740 1 -- 172.21.15.172:0/3332113925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed060 msgr2=0x55b0b15e8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:02.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.762+0000 7f85b1273740 1 --2- 172.21.15.172:0/3332113925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed060 0x55b0b15e8640 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f85a000f010 tx=0x7f85a001e040 comp rx=0 tx=0).stop 2024-09-10T08:53:02.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.762+0000 7f85b1273740 1 -- 172.21.15.172:0/3332113925 shutdown_connections 2024-09-10T08:53:02.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.762+0000 7f85b1273740 1 --2- 172.21.15.172:0/3332113925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15f8240 0x55b0b1610050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.762+0000 7f85b1273740 1 --2- 172.21.15.172:0/3332113925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15ed980 0x55b0b160fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.766 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.762+0000 7f85b1273740 1 --2- 172.21.15.172:0/3332113925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed060 0x55b0b15e8640 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.762+0000 7f85b1273740 1 -- 172.21.15.172:0/3332113925 >> 172.21.15.172:0/3332113925 conn(0x55b0b15e95f0 msgr2=0x55b0b15e7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:02.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.762+0000 7f85b1273740 1 -- 172.21.15.172:0/3332113925 shutdown_connections 2024-09-10T08:53:02.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.762+0000 7f85b1273740 1 -- 172.21.15.172:0/3332113925 wait complete. 2024-09-10T08:53:02.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.762+0000 7f85b1273740 1 -- start start 2024-09-10T08:53:02.767 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85b1273740 1 Processor -- start 2024-09-10T08:53:02.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85b1273740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15ed060 0x55b0b16006b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85b1273740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed980 0x55b0b15fca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85b1273740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15f8240 0x55b0b15fcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85b1273740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81b2250 con 0x55b0b15ed980 2024-09-10T08:53:02.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85b1273740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81b1b80 con 0x55b0b15ed060 2024-09-10T08:53:02.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85b1273740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81aedc0 con 0x55b0b15f8240 2024-09-10T08:53:02.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85b1273740 10 client.0 _ll_register_callbacks cb 0x7f8598009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:02.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85a7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed980 0x55b0b15fca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:02.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85a7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed980 0x55b0b15fca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37576/0 (socket says 172.21.15.172:37576) 2024-09-10T08:53:02.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85a7fff640 1 -- 172.21.15.172:0/3341703495 learned_addr learned my addr 172.21.15.172:0/3341703495 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:02.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85acc36640 1 --2- 172.21.15.172:0/3341703495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15ed060 0x55b0b16006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:02.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85a7fff640 1 -- 172.21.15.172:0/3341703495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15f8240 msgr2=0x55b0b15fcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:02.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85ad437640 1 --2- 172.21.15.172:0/3341703495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15f8240 0x55b0b15fcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:02.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85a7fff640 1 --2- 172.21.15.172:0/3341703495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15f8240 0x55b0b15fcf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85a7fff640 1 -- 172.21.15.172:0/3341703495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15ed060 msgr2=0x55b0b16006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:02.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85a7fff640 1 --2- 172.21.15.172:0/3341703495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15ed060 0x55b0b16006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85a7fff640 1 -- 172.21.15.172:0/3341703495 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0b15fd540 con 0x55b0b15ed980 2024-09-10T08:53:02.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85acc36640 1 --2- 172.21.15.172:0/3341703495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15ed060 0x55b0b16006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:53:02.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.763+0000 7f85ad437640 1 --2- 172.21.15.172:0/3341703495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15f8240 0x55b0b15fcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:53:02.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.764+0000 7f85a7fff640 1 --2- 172.21.15.172:0/3341703495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed980 0x55b0b15fca40 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8590005e00 tx=0x7f859003b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eddb46d3f6a3f68 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:02.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.764+0000 7f8587fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:02.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.764+0000 7f8587fff640 1 -- 172.21.15.172:0/3341703495 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85900d4060 con 0x55b0b15ed980 2024-09-10T08:53:02.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.764+0000 7f8587fff640 1 -- 172.21.15.172:0/3341703495 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f85900b7fd0 con 0x55b0b15ed980 2024-09-10T08:53:02.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.764+0000 7f85b1273740 1 -- 172.21.15.172:0/3341703495 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a00043c0 con 0x55b0b15ed980 2024-09-10T08:53:02.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.764+0000 7f8587fff640 1 -- 172.21.15.172:0/3341703495 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8590035040 con 0x55b0b15ed980 2024-09-10T08:53:02.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.764+0000 7f85b1273740 10 client.17472 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:02.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.764+0000 7f85b1273740 1 -- 172.21.15.172:0/3341703495 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b0b1673870 con 0x55b0b15ed980 2024-09-10T08:53:02.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.764+0000 7f8587fff640 1 -- 172.21.15.172:0/3341703495 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85a00043c0 con 0x55b0b15ed980 2024-09-10T08:53:02.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.764+0000 7f8587fff640 1 -- 172.21.15.172:0/3341703495 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f85900da170 con 0x55b0b15ed980 2024-09-10T08:53:02.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.764+0000 7f85b1273740 10 client.17472 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:02.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.764+0000 7f85b1273740 10 client.17472 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:53:02.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.764+0000 7f85b1273740 20 client.17472 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:02.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.764+0000 7f85b1273740 1 --2- 172.21.15.172:0/3341703495 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b0b1673f40 0x55b0b1676330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:02.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.764+0000 7f85b1273740 4 client.17472 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bd6b2e81", "release": true, "await": true}] 2024-09-10T08:53:02.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.764+0000 7f85b1273740 1 -- 172.21.15.172:0/3341703495 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bd6b2e81", "release": true, "await": true}) -- 0x55b0b155a0a0 con 0x55b0b1673f40 2024-09-10T08:53:02.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.765+0000 7f85acc36640 1 --2- 172.21.15.172:0/3341703495 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b0b1673f40 0x55b0b1676330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:02.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.765+0000 7f85acc36640 1 --2- 172.21.15.172:0/3341703495 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b0b1673f40 0x55b0b1676330 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cde7e5d02fa541d2 server_cookie=d2d47e7c2e296c97 in_seq=0 out_seq=0 2024-09-10T08:53:02.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.765+0000 7f8587fff640 10 client.17472 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:02.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.783+0000 7f8587fff640 1 -- 172.21.15.172:0/3341703495 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b0b155a0a0 con 0x55b0b1673f40 2024-09-10T08:53:02.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.783+0000 7f8587fff640 10 client.17472 handle_command_reply: tid=0 2024-09-10T08:53:02.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.783+0000 7f85b1273740 1 client.17472 shutdown 2024-09-10T08:53:02.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.783+0000 7f85b1273740 1 -- 172.21.15.172:0/3341703495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed980 msgr2=0x55b0b15fca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:02.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.783+0000 7f85b1273740 1 --2- 172.21.15.172:0/3341703495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed980 0x55b0b15fca40 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8590005e00 tx=0x7f859003b590 comp rx=0 tx=0).stop 2024-09-10T08:53:02.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.784+0000 7f85b1273740 1 -- 172.21.15.172:0/3341703495 shutdown_connections 2024-09-10T08:53:02.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.784+0000 7f85b1273740 1 --2- 172.21.15.172:0/3341703495 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b0b1673f40 0x55b0b1676330 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.784+0000 7f85b1273740 1 --2- 172.21.15.172:0/3341703495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0b15f8240 0x55b0b15fcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.784+0000 7f8587fff640 0 client.17472 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:02.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.784+0000 7f85b1273740 1 --2- 172.21.15.172:0/3341703495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0b15ed980 0x55b0b15fca40 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.784+0000 7f85b1273740 1 --2- 172.21.15.172:0/3341703495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b15ed060 0x55b0b16006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.784+0000 7f85b1273740 1 -- 172.21.15.172:0/3341703495 >> 172.21.15.172:0/3341703495 conn(0x55b0b15e95f0 msgr2=0x55b0b15ef2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:02.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.784+0000 7f85b1273740 1 -- 172.21.15.172:0/3341703495 shutdown_connections 2024-09-10T08:53:02.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.784+0000 7f85b1273740 1 -- 172.21.15.172:0/3341703495 wait complete. 2024-09-10T08:53:02.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.784+0000 7f85b1273740 20 client.17472 trim_cache size 0 max 16384 2024-09-10T08:53:02.787 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:53:02.787 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 37, 2024-09-10T08:53:02.788 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:53:02.788 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 20, 2024-09-10T08:53:02.788 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:53:02.788 INFO:teuthology.orchestra.run.smithi172.stdout: "bd6b2e81": { 2024-09-10T08:53:02.788 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 20, 2024-09-10T08:53:02.788 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:53:02.788 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:53:02.788 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:53:02.788 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:53:02.788 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:53:02.788 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:53:02.788 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 140.1, 2024-09-10T08:53:02.789 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:53:02.789 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:53:02.789 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:53:02.789 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:53:02.789 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:53:02.789 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:53:02.789 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:53:02.789 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:53:02.789 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:53:02.789 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:53:02.789 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:53:02.790 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:53:02.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.785+0000 7f85aee9a640 1 -- 172.21.15.172:0/3793261486 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f85940a2e00 msgr2=0x7f85940a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:02.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.785+0000 7f85aee9a640 1 --2- 172.21.15.172:0/3793261486 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f85940a2e00 0x7f85940a52c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f85980098b0 tx=0x7f8598009600 comp rx=0 tx=0).stop 2024-09-10T08:53:02.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.785+0000 7f85aee9a640 1 -- 172.21.15.172:0/3793261486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85a81027a0 msgr2=0x7f85a819aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:02.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.785+0000 7f85aee9a640 1 --2- 172.21.15.172:0/3793261486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85a81027a0 0x7f85a819aad0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8590005940 tx=0x7f8590004270 comp rx=0 tx=0).stop 2024-09-10T08:53:02.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.785+0000 7f85aee9a640 1 -- 172.21.15.172:0/3793261486 shutdown_connections 2024-09-10T08:53:02.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.785+0000 7f85aee9a640 1 --2- 172.21.15.172:0/3793261486 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f85940a2e00 0x7f85940a52c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.785+0000 7f85aee9a640 1 --2- 172.21.15.172:0/3793261486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85a8106f20 0x7f85a819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.785+0000 7f85aee9a640 1 --2- 172.21.15.172:0/3793261486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85a81027a0 0x7f85a819aad0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.785+0000 7f85aee9a640 1 --2- 172.21.15.172:0/3793261486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85a8101df0 0x7f85a819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:02.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.785+0000 7f85aee9a640 1 -- 172.21.15.172:0/3793261486 >> 172.21.15.172:0/3793261486 conn(0x7f85a80fd8f0 msgr2=0x7f85a80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:02.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.785+0000 7f85aee9a640 1 -- 172.21.15.172:0/3793261486 shutdown_connections 2024-09-10T08:53:02.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:02.785+0000 7f85aee9a640 1 -- 172.21.15.172:0/3793261486 wait complete. 2024-09-10T08:53:02.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:02 smithi191 ceph-mon[35314]: pgmap v554: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 244 KiB/s wr, 36 op/s 2024-09-10T08:53:03.508 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bd6b2e81', total seconds elapsed: 23.4 2024-09-10T08:53:03.509 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-10T08:53:03.509 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:53:03.808 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:53:04.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.317+0000 7fca178f4640 1 -- 172.21.15.172:0/2022127189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca10102790 msgr2=0x7fca101067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:04.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.317+0000 7fca178f4640 1 --2- 172.21.15.172:0/2022127189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca10102790 0x7fca101067e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fca00009900 tx=0x7fca0002a410 comp rx=0 tx=0).stop 2024-09-10T08:53:04.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.317+0000 7fca178f4640 1 -- 172.21.15.172:0/2022127189 shutdown_connections 2024-09-10T08:53:04.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.317+0000 7fca178f4640 1 --2- 172.21.15.172:0/2022127189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca10106f10 0x7fca1010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:04.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.317+0000 7fca178f4640 1 --2- 172.21.15.172:0/2022127189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca10102790 0x7fca101067e0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:04.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.317+0000 7fca178f4640 1 --2- 172.21.15.172:0/2022127189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca10101de0 0x7fca101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:04.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.317+0000 7fca178f4640 1 -- 172.21.15.172:0/2022127189 >> 172.21.15.172:0/2022127189 conn(0x7fca100fd8c0 msgr2=0x7fca100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:04.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.317+0000 7fca178f4640 1 -- 172.21.15.172:0/2022127189 shutdown_connections 2024-09-10T08:53:04.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.317+0000 7fca178f4640 1 -- 172.21.15.172:0/2022127189 wait complete. 2024-09-10T08:53:04.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.318+0000 7fca178f4640 1 Processor -- start 2024-09-10T08:53:04.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.318+0000 7fca178f4640 1 -- start start 2024-09-10T08:53:04.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.318+0000 7fca178f4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca10101de0 0x7fca1019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:04.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.318+0000 7fca178f4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca10102790 0x7fca1019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:04.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.318+0000 7fca178f4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca10106f10 0x7fca1019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:04.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.318+0000 7fca178f4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fca10111e20 con 0x7fca10106f10 2024-09-10T08:53:04.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.318+0000 7fca178f4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca10111ca0 con 0x7fca10102790 2024-09-10T08:53:04.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.318+0000 7fca178f4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fca10111fa0 con 0x7fca10101de0 2024-09-10T08:53:04.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.319+0000 7fca15e91640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca10106f10 0x7fca1019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:04.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.319+0000 7fca15690640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca10101de0 0x7fca1019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:04.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.319+0000 7fca15690640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca10101de0 0x7fca1019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:47592/0 (socket says 172.21.15.172:47592) 2024-09-10T08:53:04.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.319+0000 7fca15e91640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca10106f10 0x7fca1019ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37590/0 (socket says 172.21.15.172:37590) 2024-09-10T08:53:04.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.319+0000 7fca15690640 1 -- 172.21.15.172:0/3977189770 learned_addr learned my addr 172.21.15.172:0/3977189770 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:04.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.319+0000 7fca14e8f640 1 --2- 172.21.15.172:0/3977189770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca10102790 0x7fca1019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:04.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.319+0000 7fca15e91640 1 -- 172.21.15.172:0/3977189770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca10101de0 msgr2=0x7fca1019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:04.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.319+0000 7fca15e91640 1 --2- 172.21.15.172:0/3977189770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca10101de0 0x7fca1019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:04.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.319+0000 7fca15e91640 1 -- 172.21.15.172:0/3977189770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca10102790 msgr2=0x7fca1019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:04.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.319+0000 7fca15e91640 1 --2- 172.21.15.172:0/3977189770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca10102790 0x7fca1019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:04.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.319+0000 7fca15e91640 1 -- 172.21.15.172:0/3977189770 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1019f550 con 0x7fca10106f10 2024-09-10T08:53:04.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.319+0000 7fca15690640 1 --2- 172.21.15.172:0/3977189770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca10101de0 0x7fca1019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:04.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.319+0000 7fca15e91640 1 --2- 172.21.15.172:0/3977189770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca10106f10 0x7fca1019ee70 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fca0800e940 tx=0x7fca0800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:04.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.319+0000 7fca067fc640 1 -- 172.21.15.172:0/3977189770 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08004d20 con 0x7fca10106f10 2024-09-10T08:53:04.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.319+0000 7fca178f4640 1 -- 172.21.15.172:0/3977189770 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca1019f840 con 0x7fca10106f10 2024-09-10T08:53:04.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.319+0000 7fca067fc640 1 -- 172.21.15.172:0/3977189770 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca08005020 con 0x7fca10106f10 2024-09-10T08:53:04.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.319+0000 7fca067fc640 1 -- 172.21.15.172:0/3977189770 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca080116d0 con 0x7fca10106f10 2024-09-10T08:53:04.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.320+0000 7fca178f4640 1 -- 172.21.15.172:0/3977189770 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca101a7120 con 0x7fca10106f10 2024-09-10T08:53:04.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.321+0000 7fca178f4640 1 -- 172.21.15.172:0/3977189770 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca101021c0 con 0x7fca10106f10 2024-09-10T08:53:04.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.322+0000 7fca067fc640 1 -- 172.21.15.172:0/3977189770 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fca0800cb20 con 0x7fca10106f10 2024-09-10T08:53:04.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.323+0000 7fca067fc640 1 --2- 172.21.15.172:0/3977189770 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc9fc0a2b80 0x7fc9fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:04.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.323+0000 7fca067fc640 1 -- 172.21.15.172:0/3977189770 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fca080cf340 con 0x7fca10106f10 2024-09-10T08:53:04.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.323+0000 7fca15690640 1 --2- 172.21.15.172:0/3977189770 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc9fc0a2b80 0x7fc9fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:04.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.323+0000 7fca15690640 1 --2- 172.21.15.172:0/3977189770 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc9fc0a2b80 0x7fc9fc0a5040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0104e0 tx=0x7fca0c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:04.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.327+0000 7fca067fc640 1 -- 172.21.15.172:0/3977189770 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca08016030 con 0x7fca10106f10 2024-09-10T08:53:04.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.568+0000 7fca178f4640 1 -- 172.21.15.172:0/3977189770 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca101a72f0 con 0x7fca10106f10 2024-09-10T08:53:04.570 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:04 smithi172 ceph-mon[30854]: pgmap v555: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 244 KiB/s wr, 34 op/s 2024-09-10T08:53:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.569+0000 7fca067fc640 1 -- 172.21.15.172:0/3977189770 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+12001 (secure 0 0 0) 0x7fca0809b990 con 0x7fca10106f10 2024-09-10T08:53:04.571 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:53:04.571 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":37,"btime":"2024-09-10T08:52:08:313493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:52:07.658341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:53:04.572 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 37 2024-09-10T08:53:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.571+0000 7fca178f4640 1 -- 172.21.15.172:0/3977189770 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc9fc0a2b80 msgr2=0x7fc9fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.571+0000 7fca178f4640 1 --2- 172.21.15.172:0/3977189770 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc9fc0a2b80 0x7fc9fc0a5040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0104e0 tx=0x7fca0c00b410 comp rx=0 tx=0).stop 2024-09-10T08:53:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.571+0000 7fca178f4640 1 -- 172.21.15.172:0/3977189770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca10106f10 msgr2=0x7fca1019ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.571+0000 7fca178f4640 1 --2- 172.21.15.172:0/3977189770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca10106f10 0x7fca1019ee70 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fca0800e940 tx=0x7fca0800ee00 comp rx=0 tx=0).stop 2024-09-10T08:53:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.571+0000 7fca178f4640 1 -- 172.21.15.172:0/3977189770 shutdown_connections 2024-09-10T08:53:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.571+0000 7fca178f4640 1 --2- 172.21.15.172:0/3977189770 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc9fc0a2b80 0x7fc9fc0a5040 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.571+0000 7fca178f4640 1 --2- 172.21.15.172:0/3977189770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca10106f10 0x7fca1019ee70 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:04.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.571+0000 7fca178f4640 1 --2- 172.21.15.172:0/3977189770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca10102790 0x7fca1019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:04.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.571+0000 7fca178f4640 1 --2- 172.21.15.172:0/3977189770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca10101de0 0x7fca1019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:04.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.571+0000 7fca178f4640 1 -- 172.21.15.172:0/3977189770 >> 172.21.15.172:0/3977189770 conn(0x7fca100fd8c0 msgr2=0x7fca100fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:04.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.572+0000 7fca178f4640 1 -- 172.21.15.172:0/3977189770 shutdown_connections 2024-09-10T08:53:04.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:04.572+0000 7fca178f4640 1 -- 172.21.15.172:0/3977189770 wait complete. 2024-09-10T08:53:04.601 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:04 smithi191 ceph-mon[35314]: pgmap v555: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 244 KiB/s wr, 34 op/s 2024-09-10T08:53:04.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:04 smithi174 ceph-mon[34068]: pgmap v555: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 244 KiB/s wr, 34 op/s 2024-09-10T08:53:05.290 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-10T08:53:05.290 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-10T08:53:05.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:05 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3977189770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:53:05.597 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:53:05.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:05 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3977189770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:53:05.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:05 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3977189770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:53:06.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.118+0000 7f7f8e6d3640 1 -- 172.21.15.172:0/2639996323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f88103670 msgr2=0x7f7f8810bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.118+0000 7f7f8e6d3640 1 --2- 172.21.15.172:0/2639996323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f88103670 0x7f7f8810bb60 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009900 tx=0x7f7f7c02a490 comp rx=0 tx=0).stop 2024-09-10T08:53:06.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.118+0000 7f7f8e6d3640 1 -- 172.21.15.172:0/2639996323 shutdown_connections 2024-09-10T08:53:06.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.118+0000 7f7f8e6d3640 1 --2- 172.21.15.172:0/2639996323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f8810c180 0x7f7f8810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.118+0000 7f7f8e6d3640 1 --2- 172.21.15.172:0/2639996323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f88103670 0x7f7f8810bb60 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.118+0000 7f7f8e6d3640 1 --2- 172.21.15.172:0/2639996323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f88102d50 0x7f7f88103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.118+0000 7f7f8e6d3640 1 -- 172.21.15.172:0/2639996323 >> 172.21.15.172:0/2639996323 conn(0x7f7f880fc7d0 msgr2=0x7f7f880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:06.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.118+0000 7f7f8e6d3640 1 -- 172.21.15.172:0/2639996323 shutdown_connections 2024-09-10T08:53:06.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.118+0000 7f7f8e6d3640 1 -- 172.21.15.172:0/2639996323 wait complete. 2024-09-10T08:53:06.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.119+0000 7f7f8e6d3640 1 Processor -- start 2024-09-10T08:53:06.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.119+0000 7f7f8e6d3640 1 -- start start 2024-09-10T08:53:06.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.119+0000 7f7f8e6d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f88102d50 0x7f7f881009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.119+0000 7f7f8e6d3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f88103670 0x7f7f88100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.119+0000 7f7f8e6d3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f8810c180 0x7f7f88101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.119+0000 7f7f8e6d3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88111ef0 con 0x7f7f88102d50 2024-09-10T08:53:06.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.119+0000 7f7f8e6d3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88111d70 con 0x7f7f88103670 2024-09-10T08:53:06.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.119+0000 7f7f8e6d3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7f88112070 con 0x7f7f8810c180 2024-09-10T08:53:06.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.120+0000 7f7f87fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f88102d50 0x7f7f881009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.120+0000 7f7f877fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f88103670 0x7f7f88100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.120+0000 7f7f87fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f88102d50 0x7f7f881009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37612/0 (socket says 172.21.15.172:37612) 2024-09-10T08:53:06.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.120+0000 7f7f87fff640 1 -- 172.21.15.172:0/1407983699 learned_addr learned my addr 172.21.15.172:0/1407983699 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:06.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.120+0000 7f7f8cc70640 1 --2- 172.21.15.172:0/1407983699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f8810c180 0x7f7f88101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.120+0000 7f7f87fff640 1 -- 172.21.15.172:0/1407983699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f8810c180 msgr2=0x7f7f88101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.120+0000 7f7f87fff640 1 --2- 172.21.15.172:0/1407983699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f8810c180 0x7f7f88101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.120+0000 7f7f87fff640 1 -- 172.21.15.172:0/1407983699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f88103670 msgr2=0x7f7f88100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.120+0000 7f7f87fff640 1 --2- 172.21.15.172:0/1407983699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f88103670 0x7f7f88100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.120+0000 7f7f87fff640 1 -- 172.21.15.172:0/1407983699 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f881a6990 con 0x7f7f88102d50 2024-09-10T08:53:06.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.120+0000 7f7f877fe640 1 --2- 172.21.15.172:0/1407983699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f88103670 0x7f7f88100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:06.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.120+0000 7f7f87fff640 1 --2- 172.21.15.172:0/1407983699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f88102d50 0x7f7f881009b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800b4a0 tx=0x7f7f7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:06.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.120+0000 7f7f857fa640 1 -- 172.21.15.172:0/1407983699 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78014070 con 0x7f7f88102d50 2024-09-10T08:53:06.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.120+0000 7f7f857fa640 1 -- 172.21.15.172:0/1407983699 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f78010ac0 con 0x7f7f88102d50 2024-09-10T08:53:06.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.120+0000 7f7f8e6d3640 1 -- 172.21.15.172:0/1407983699 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f881a6c80 con 0x7f7f88102d50 2024-09-10T08:53:06.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.121+0000 7f7f857fa640 1 -- 172.21.15.172:0/1407983699 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78017c30 con 0x7f7f88102d50 2024-09-10T08:53:06.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.121+0000 7f7f8e6d3640 1 -- 172.21.15.172:0/1407983699 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f881a7190 con 0x7f7f88102d50 2024-09-10T08:53:06.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.123+0000 7f7f857fa640 1 -- 172.21.15.172:0/1407983699 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7f78017070 con 0x7f7f88102d50 2024-09-10T08:53:06.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.124+0000 7f7f857fa640 1 --2- 172.21.15.172:0/1407983699 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7f700a2c00 0x7f7f700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.124+0000 7f7f877fe640 1 --2- 172.21.15.172:0/1407983699 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7f700a2c00 0x7f7f700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.124+0000 7f7f857fa640 1 -- 172.21.15.172:0/1407983699 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f7f780cea20 con 0x7f7f88102d50 2024-09-10T08:53:06.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.125+0000 7f7f90aac740 1 Processor -- start 2024-09-10T08:53:06.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.125+0000 7f7f877fe640 1 --2- 172.21.15.172:0/1407983699 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7f700a2c00 0x7f7f700a50c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c031440 tx=0x7f7f7c005e60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:06.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.125+0000 7f7f90aac740 1 -- start start 2024-09-10T08:53:06.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.125+0000 7f7f90aac740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe16fc760 0x561fe16fcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.125+0000 7f7f90aac740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe16fd080 0x561fe1707400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.125+0000 7f7f90aac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe1707940 0x561fe1709d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.125+0000 7f7f90aac740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561fe16622a0 con 0x561fe16fd080 2024-09-10T08:53:06.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.125+0000 7f7f90aac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561fe1609b10 con 0x561fe1707940 2024-09-10T08:53:06.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.125+0000 7f7f90aac740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561fe165e260 con 0x561fe16fc760 2024-09-10T08:53:06.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.125+0000 7f7f87fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe16fc760 0x561fe16fcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.125+0000 7f7f877fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe16fd080 0x561fe1707400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.125+0000 7f7f87fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe16fc760 0x561fe16fcb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:47634/0 (socket says 172.21.15.172:47634) 2024-09-10T08:53:06.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.125+0000 7f7f87fff640 1 -- 172.21.15.172:0/22664111 learned_addr learned my addr 172.21.15.172:0/22664111 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:06.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.126+0000 7f7f87fff640 1 -- 172.21.15.172:0/22664111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe1707940 msgr2=0x561fe1709d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:06.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.126+0000 7f7f87fff640 1 --2- 172.21.15.172:0/22664111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe1707940 0x561fe1709d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.126+0000 7f7f87fff640 1 -- 172.21.15.172:0/22664111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe16fd080 msgr2=0x561fe1707400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:06.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.126+0000 7f7f87fff640 1 --2- 172.21.15.172:0/22664111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe16fd080 0x561fe1707400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.126+0000 7f7f87fff640 1 -- 172.21.15.172:0/22664111 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe13d3b90 con 0x561fe16fc760 2024-09-10T08:53:06.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.126+0000 7f7f877fe640 1 --2- 172.21.15.172:0/22664111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe16fd080 0x561fe1707400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:06.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.126+0000 7f7f87fff640 1 --2- 172.21.15.172:0/22664111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe16fc760 0x561fe16fcb40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7f780973d0 tx=0x7f7f7809a990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48c22a65e3d3c4b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:06.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.126+0000 7f7f6e7fc640 1 -- 172.21.15.172:0/22664111 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7801e050 con 0x561fe16fc760 2024-09-10T08:53:06.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.126+0000 7f7f6e7fc640 1 -- 172.21.15.172:0/22664111 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f780a3b30 con 0x561fe16fc760 2024-09-10T08:53:06.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.126+0000 7f7f6e7fc640 1 -- 172.21.15.172:0/22664111 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f780a36a0 con 0x561fe16fc760 2024-09-10T08:53:06.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.127+0000 7f7f90aac740 1 -- 172.21.15.172:0/22664111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe16fc760 msgr2=0x561fe16fcb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.127+0000 7f7f90aac740 1 --2- 172.21.15.172:0/22664111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe16fc760 0x561fe16fcb40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7f780973d0 tx=0x7f7f7809a990 comp rx=0 tx=0).stop 2024-09-10T08:53:06.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.127+0000 7f7f90aac740 1 -- 172.21.15.172:0/22664111 shutdown_connections 2024-09-10T08:53:06.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.127+0000 7f7f90aac740 1 --2- 172.21.15.172:0/22664111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe1707940 0x561fe1709d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.128+0000 7f7f90aac740 1 --2- 172.21.15.172:0/22664111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe16fd080 0x561fe1707400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.128+0000 7f7f90aac740 1 --2- 172.21.15.172:0/22664111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe16fc760 0x561fe16fcb40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.128+0000 7f7f90aac740 1 -- 172.21.15.172:0/22664111 >> 172.21.15.172:0/22664111 conn(0x561fe16f8cf0 msgr2=0x561fe16f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:06.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.128+0000 7f7f90aac740 1 -- 172.21.15.172:0/22664111 shutdown_connections 2024-09-10T08:53:06.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.128+0000 7f7f90aac740 1 -- 172.21.15.172:0/22664111 wait complete. 2024-09-10T08:53:06.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.128+0000 7f7f90aac740 1 -- start start 2024-09-10T08:53:06.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.128+0000 7f7f90aac740 1 Processor -- start 2024-09-10T08:53:06.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.129+0000 7f7f90aac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 0x561fe1704870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.129+0000 7f7f90aac740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe16fd080 0x561fe1704db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.129+0000 7f7f90aac740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe1707940 0x561fe17052f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.129+0000 7f7f90aac740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561fe170a510 con 0x561fe16fd080 2024-09-10T08:53:06.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.129+0000 7f7f90aac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561fe15ea6e0 con 0x561fe16fc760 2024-09-10T08:53:06.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.129+0000 7f7f90aac740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561fe170a690 con 0x561fe1707940 2024-09-10T08:53:06.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.129+0000 7f7f90aac740 10 client.0 _ll_register_callbacks cb 0x561fe160ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:06.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.129+0000 7f7f87fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 0x561fe1704870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.129+0000 7f7f87fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 0x561fe1704870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:58416/0 (socket says 172.21.15.172:58416) 2024-09-10T08:53:06.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.129+0000 7f7f87fff640 1 -- 172.21.15.172:0/726634973 learned_addr learned my addr 172.21.15.172:0/726634973 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:06.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.129+0000 7f7f8cc70640 1 --2- 172.21.15.172:0/726634973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe1707940 0x561fe17052f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.129+0000 7f7f87fff640 1 -- 172.21.15.172:0/726634973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe1707940 msgr2=0x561fe17052f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:06.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.129+0000 7f7f87fff640 1 --2- 172.21.15.172:0/726634973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe1707940 0x561fe17052f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.129+0000 7f7f87fff640 1 -- 172.21.15.172:0/726634973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe16fd080 msgr2=0x561fe1704db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:06.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.129+0000 7f7f87fff640 1 --2- 172.21.15.172:0/726634973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe16fd080 0x561fe1704db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.130+0000 7f7f87fff640 1 -- 172.21.15.172:0/726634973 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe1713a50 con 0x561fe16fc760 2024-09-10T08:53:06.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.130+0000 7f7f87fff640 1 --2- 172.21.15.172:0/726634973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 0x561fe1704870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7f78004700 tx=0x7f7f78097660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71b21fea03d3b482 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:06.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.130+0000 7f7f677fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:53:06.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.130+0000 7f7f677fe640 1 -- 172.21.15.172:0/726634973 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7807e060 con 0x561fe16fc760 2024-09-10T08:53:06.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.130+0000 7f7f677fe640 1 -- 172.21.15.172:0/726634973 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f7f780a36a0 con 0x561fe16fc760 2024-09-10T08:53:06.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.130+0000 7f7f90aac740 1 -- 172.21.15.172:0/726634973 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f780a3b30 con 0x561fe16fc760 2024-09-10T08:53:06.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.130+0000 7f7f677fe640 1 -- 172.21.15.172:0/726634973 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7f7801e020 con 0x561fe16fc760 2024-09-10T08:53:06.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.131+0000 7f7f90aac740 10 client.26576 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:06.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.131+0000 7f7f90aac740 1 -- 172.21.15.172:0/726634973 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561fe17163e0 con 0x561fe16fc760 2024-09-10T08:53:06.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.131+0000 7f7f677fe640 1 -- 172.21.15.172:0/726634973 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f780a3b30 con 0x561fe16fc760 2024-09-10T08:53:06.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.131+0000 7f7f677fe640 1 -- 172.21.15.172:0/726634973 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f7f780b9b00 con 0x561fe16fc760 2024-09-10T08:53:06.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.131+0000 7f7f90aac740 10 client.26576 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:06.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.131+0000 7f7f90aac740 10 client.26576 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:53:06.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.131+0000 7f7f90aac740 20 client.26576 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:06.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.131+0000 7f7f90aac740 1 --2- 172.21.15.172:0/726634973 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561fe1717730 0x561fe1719b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.131+0000 7f7f90aac740 4 client.26576 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:53:06.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.131+0000 7f7f90aac740 1 -- 172.21.15.172:0/726634973 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fe166f9d0 con 0x561fe1717730 2024-09-10T08:53:06.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.131+0000 7f7f877fe640 1 --2- 172.21.15.172:0/726634973 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561fe1717730 0x561fe1719b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.132+0000 7f7f677fe640 10 client.26576 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:06.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.132+0000 7f7f877fe640 1 --2- 172.21.15.172:0/726634973 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561fe1717730 0x561fe1719b20 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbff469b18ef1b8b server_cookie=28a2ceb98ed95b1a in_seq=0 out_seq=0 2024-09-10T08:53:06.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.132+0000 7f7f677fe640 1 -- 172.21.15.172:0/726634973 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fe166f9d0 con 0x561fe1717730 2024-09-10T08:53:06.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.132+0000 7f7f677fe640 10 client.26576 handle_command_reply: tid=0 2024-09-10T08:53:06.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.133+0000 7f7f90aac740 1 client.26576 shutdown 2024-09-10T08:53:06.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.133+0000 7f7f90aac740 1 -- 172.21.15.172:0/726634973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 msgr2=0x561fe1704870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.133+0000 7f7f90aac740 1 --2- 172.21.15.172:0/726634973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 0x561fe1704870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7f78004700 tx=0x7f7f78097660 comp rx=0 tx=0).stop 2024-09-10T08:53:06.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.133+0000 7f7f90aac740 1 -- 172.21.15.172:0/726634973 shutdown_connections 2024-09-10T08:53:06.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.133+0000 7f7f90aac740 1 --2- 172.21.15.172:0/726634973 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561fe1717730 0x561fe1719b20 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.133+0000 7f7f90aac740 1 --2- 172.21.15.172:0/726634973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe1707940 0x561fe17052f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.133+0000 7f7f677fe640 0 client.26576 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:06.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.133+0000 7f7f90aac740 1 --2- 172.21.15.172:0/726634973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe16fd080 0x561fe1704db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.133+0000 7f7f90aac740 1 --2- 172.21.15.172:0/726634973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 0x561fe1704870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.133+0000 7f7f90aac740 1 -- 172.21.15.172:0/726634973 >> 172.21.15.172:0/726634973 conn(0x561fe16f8cf0 msgr2=0x561fe16fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:06.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.133+0000 7f7f90aac740 1 -- 172.21.15.172:0/726634973 shutdown_connections 2024-09-10T08:53:06.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.133+0000 7f7f90aac740 1 -- 172.21.15.172:0/726634973 wait complete. 2024-09-10T08:53:06.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.133+0000 7f7f90aac740 20 client.26576 trim_cache size 0 max 16384 2024-09-10T08:53:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.159+0000 7f7f90aac740 1 Processor -- start 2024-09-10T08:53:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.159+0000 7f7f90aac740 1 -- start start 2024-09-10T08:53:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.160+0000 7f7f90aac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 0x561fe1712f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.160+0000 7f7f90aac740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe16fd080 0x561fe17031a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.160+0000 7f7f90aac740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe1707940 0x561fe1701b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.160+0000 7f7f90aac740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881b0ef0 con 0x561fe16fd080 2024-09-10T08:53:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.160+0000 7f7f90aac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881b2a30 con 0x561fe16fc760 2024-09-10T08:53:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.160+0000 7f7f90aac740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881b15c0 con 0x561fe1707940 2024-09-10T08:53:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.160+0000 7f7f877fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe16fd080 0x561fe17031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.160+0000 7f7f877fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe16fd080 0x561fe17031a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44508/0 (socket says 172.21.15.172:44508) 2024-09-10T08:53:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.160+0000 7f7f877fe640 1 -- 172.21.15.172:0/1091866565 learned_addr learned my addr 172.21.15.172:0/1091866565 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.160+0000 7f7f8cc70640 1 --2- 172.21.15.172:0/1091866565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe1707940 0x561fe1701b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.160+0000 7f7f87fff640 1 --2- 172.21.15.172:0/1091866565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 0x561fe1712f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.160+0000 7f7f877fe640 1 -- 172.21.15.172:0/1091866565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe1707940 msgr2=0x561fe1701b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.160+0000 7f7f877fe640 1 --2- 172.21.15.172:0/1091866565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe1707940 0x561fe1701b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.160+0000 7f7f877fe640 1 -- 172.21.15.172:0/1091866565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 msgr2=0x561fe1712f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.160+0000 7f7f877fe640 1 --2- 172.21.15.172:0/1091866565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 0x561fe1712f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.160+0000 7f7f877fe640 1 -- 172.21.15.172:0/1091866565 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fe1702e20 con 0x561fe16fd080 2024-09-10T08:53:06.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.161+0000 7f7f877fe640 1 --2- 172.21.15.172:0/1091866565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe16fd080 0x561fe17031a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c052fd0 tx=0x7f7f7c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fe31176e998f81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.161+0000 7f7f6dffb640 1 -- 172.21.15.172:0/1091866565 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c035040 con 0x561fe16fd080 2024-09-10T08:53:06.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.161+0000 7f7f6dffb640 1 -- 172.21.15.172:0/1091866565 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f7c0054d0 con 0x561fe16fd080 2024-09-10T08:53:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.161+0000 7f7f6dffb640 1 -- 172.21.15.172:0/1091866565 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c004a00 con 0x561fe16fd080 2024-09-10T08:53:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.162+0000 7f7f90aac740 1 -- 172.21.15.172:0/1091866565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe16fd080 msgr2=0x561fe17031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.162+0000 7f7f90aac740 1 --2- 172.21.15.172:0/1091866565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe16fd080 0x561fe17031a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c052fd0 tx=0x7f7f7c03d000 comp rx=0 tx=0).stop 2024-09-10T08:53:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.162+0000 7f7f90aac740 1 -- 172.21.15.172:0/1091866565 shutdown_connections 2024-09-10T08:53:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.162+0000 7f7f90aac740 1 --2- 172.21.15.172:0/1091866565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe1707940 0x561fe1701b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.162+0000 7f7f90aac740 1 --2- 172.21.15.172:0/1091866565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe16fd080 0x561fe17031a0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.162+0000 7f7f90aac740 1 --2- 172.21.15.172:0/1091866565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 0x561fe1712f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.162+0000 7f7f90aac740 1 -- 172.21.15.172:0/1091866565 >> 172.21.15.172:0/1091866565 conn(0x561fe16f8cf0 msgr2=0x561fe17068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.162+0000 7f7f90aac740 1 -- 172.21.15.172:0/1091866565 shutdown_connections 2024-09-10T08:53:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.162+0000 7f7f90aac740 1 -- 172.21.15.172:0/1091866565 wait complete. 2024-09-10T08:53:06.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.162+0000 7f7f90aac740 1 -- start start 2024-09-10T08:53:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.162+0000 7f7f90aac740 1 Processor -- start 2024-09-10T08:53:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f90aac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 0x561fe171d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f90aac740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe16fd080 0x561fe171b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f90aac740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe1707940 0x561fe171ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f90aac740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881b2360 con 0x561fe1707940 2024-09-10T08:53:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f90aac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881b1c90 con 0x561fe16fc760 2024-09-10T08:53:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f90aac740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7f881aeed0 con 0x561fe16fd080 2024-09-10T08:53:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f90aac740 10 client.0 _ll_register_callbacks cb 0x7f7f780175f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f877fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe16fd080 0x561fe171b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f877fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe16fd080 0x561fe171b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:48644/0 (socket says 172.21.15.172:48644) 2024-09-10T08:53:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f877fe640 1 -- 172.21.15.172:0/3903996427 learned_addr learned my addr 172.21.15.172:0/3903996427 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f87fff640 1 --2- 172.21.15.172:0/3903996427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 0x561fe171d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f8cc70640 1 --2- 172.21.15.172:0/3903996427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe1707940 0x561fe171ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f87fff640 1 -- 172.21.15.172:0/3903996427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe16fd080 msgr2=0x561fe171b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f87fff640 1 --2- 172.21.15.172:0/3903996427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe16fd080 0x561fe171b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f87fff640 1 -- 172.21.15.172:0/3903996427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe1707940 msgr2=0x561fe171ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f87fff640 1 --2- 172.21.15.172:0/3903996427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe1707940 0x561fe171ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f87fff640 1 -- 172.21.15.172:0/3903996427 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fe171c290 con 0x561fe16fc760 2024-09-10T08:53:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f8cc70640 1 --2- 172.21.15.172:0/3903996427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe1707940 0x561fe171ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f877fe640 1 --2- 172.21.15.172:0/3903996427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe16fd080 0x561fe171b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f87fff640 1 --2- 172.21.15.172:0/3903996427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 0x561fe171d1a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7f78099220 tx=0x7f7f78010e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3324a2cb87e5e915 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:06.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.163+0000 7f7f677fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:53:06.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.164+0000 7f7f677fe640 1 -- 172.21.15.172:0/3903996427 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f780a3b30 con 0x561fe16fc760 2024-09-10T08:53:06.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.164+0000 7f7f677fe640 1 -- 172.21.15.172:0/3903996427 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f7f78099270 con 0x561fe16fc760 2024-09-10T08:53:06.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.164+0000 7f7f90aac740 1 -- 172.21.15.172:0/3903996427 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f7c0054d0 con 0x561fe16fc760 2024-09-10T08:53:06.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.164+0000 7f7f677fe640 1 -- 172.21.15.172:0/3903996427 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7f780af170 con 0x561fe16fc760 2024-09-10T08:53:06.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.164+0000 7f7f90aac740 10 client.26582 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:06.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.164+0000 7f7f90aac740 1 -- 172.21.15.172:0/3903996427 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561fe1783000 con 0x561fe16fc760 2024-09-10T08:53:06.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.164+0000 7f7f677fe640 1 -- 172.21.15.172:0/3903996427 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f7c0054d0 con 0x561fe16fc760 2024-09-10T08:53:06.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.164+0000 7f7f677fe640 1 -- 172.21.15.172:0/3903996427 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f7f780a36a0 con 0x561fe16fc760 2024-09-10T08:53:06.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.164+0000 7f7f90aac740 10 client.26582 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:06.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.164+0000 7f7f90aac740 10 client.26582 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:53:06.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.164+0000 7f7f90aac740 20 client.26582 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:06.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.164+0000 7f7f90aac740 1 --2- 172.21.15.172:0/3903996427 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561fe1717730 0x561fe1785600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.164+0000 7f7f90aac740 4 client.26582 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-10T08:53:06.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.164+0000 7f7f90aac740 1 -- 172.21.15.172:0/3903996427 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x561fe1669d20 con 0x561fe1717730 2024-09-10T08:53:06.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.164+0000 7f7f877fe640 1 --2- 172.21.15.172:0/3903996427 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561fe1717730 0x561fe1785600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.165+0000 7f7f677fe640 10 client.26582 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:06.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.165+0000 7f7f877fe640 1 --2- 172.21.15.172:0/3903996427 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561fe1717730 0x561fe1785600 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efc592f76954ea20 server_cookie=aaec74c0681afda9 in_seq=0 out_seq=0 2024-09-10T08:53:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.189+0000 7fc96a718640 1 -- 172.21.15.172:0/2333225806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc964104ef0 msgr2=0x7fc9641072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.189+0000 7fc96a718640 1 --2- 172.21.15.172:0/2333225806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc964104ef0 0x7fc9641072e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fc958009900 tx=0x7fc95802a410 comp rx=0 tx=0).stop 2024-09-10T08:53:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.189+0000 7fc96a718640 1 -- 172.21.15.172:0/2333225806 shutdown_connections 2024-09-10T08:53:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.189+0000 7fc96a718640 1 --2- 172.21.15.172:0/2333225806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc964107820 0x7fc964109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.189+0000 7fc96a718640 1 --2- 172.21.15.172:0/2333225806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc964104ef0 0x7fc9641072e0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.189+0000 7fc96a718640 1 --2- 172.21.15.172:0/2333225806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9641025c0 0x7fc9641049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.189+0000 7fc96a718640 1 -- 172.21.15.172:0/2333225806 >> 172.21.15.172:0/2333225806 conn(0x7fc9640fc1e0 msgr2=0x7fc9640fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.189+0000 7fc96a718640 1 -- 172.21.15.172:0/2333225806 shutdown_connections 2024-09-10T08:53:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.189+0000 7fc96a718640 1 -- 172.21.15.172:0/2333225806 wait complete. 2024-09-10T08:53:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.189+0000 7fc96a718640 1 Processor -- start 2024-09-10T08:53:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.189+0000 7fc96a718640 1 -- start start 2024-09-10T08:53:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc96a718640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9641025c0 0x7fc964197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc96a718640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc964104ef0 0x7fc9641982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc96a718640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc964107820 0x7fc96419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc96a718640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc96410d6a0 con 0x7fc964104ef0 2024-09-10T08:53:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc96a718640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc96410d520 con 0x7fc9641025c0 2024-09-10T08:53:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc96a718640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc96410d820 con 0x7fc964107820 2024-09-10T08:53:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc9637fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc964104ef0 0x7fc9641982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc9637fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc964104ef0 0x7fc9641982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44530/0 (socket says 172.21.15.172:44530) 2024-09-10T08:53:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc9637fe640 1 -- 172.21.15.172:0/2830215053 learned_addr learned my addr 172.21.15.172:0/2830215053 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc963fff640 1 --2- 172.21.15.172:0/2830215053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9641025c0 0x7fc964197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc968cb5640 1 --2- 172.21.15.172:0/2830215053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc964107820 0x7fc96419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc9637fe640 1 -- 172.21.15.172:0/2830215053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc964107820 msgr2=0x7fc96419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc9637fe640 1 --2- 172.21.15.172:0/2830215053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc964107820 0x7fc96419f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc9637fe640 1 -- 172.21.15.172:0/2830215053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9641025c0 msgr2=0x7fc964197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc9637fe640 1 --2- 172.21.15.172:0/2830215053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9641025c0 0x7fc964197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc9637fe640 1 -- 172.21.15.172:0/2830215053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96419f9b0 con 0x7fc964104ef0 2024-09-10T08:53:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc9637fe640 1 --2- 172.21.15.172:0/2830215053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc964104ef0 0x7fc9641982e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fc9580314e0 tx=0x7fc958004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc9617fa640 1 -- 172.21.15.172:0/2830215053 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95803f030 con 0x7fc964104ef0 2024-09-10T08:53:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc9617fa640 1 -- 172.21.15.172:0/2830215053 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc958003aa0 con 0x7fc964104ef0 2024-09-10T08:53:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc96a718640 1 -- 172.21.15.172:0/2830215053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96419fc40 con 0x7fc964104ef0 2024-09-10T08:53:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc9617fa640 1 -- 172.21.15.172:0/2830215053 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958035070 con 0x7fc964104ef0 2024-09-10T08:53:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.190+0000 7fc96a718640 1 -- 172.21.15.172:0/2830215053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9641a0180 con 0x7fc964104ef0 2024-09-10T08:53:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.193+0000 7fc9617fa640 1 -- 172.21.15.172:0/2830215053 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc958042030 con 0x7fc964104ef0 2024-09-10T08:53:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.193+0000 7fc96d4b1740 1 Processor -- start 2024-09-10T08:53:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.193+0000 7fc96d4b1740 1 -- start start 2024-09-10T08:53:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.193+0000 7fc96d4b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562508410db0 0x5625084111b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.193+0000 7fc96d4b1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625084116f0 0x5625084161b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.193+0000 7fc96d4b1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562508416780 0x562508418c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.193+0000 7fc96d4b1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5625083702f0 con 0x5625084116f0 2024-09-10T08:53:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.193+0000 7fc96d4b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562508318270 con 0x562508410db0 2024-09-10T08:53:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.193+0000 7fc96d4b1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56250836c2b0 con 0x562508416780 2024-09-10T08:53:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.193+0000 7fc963fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625084116f0 0x5625084161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.193+0000 7fc963fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625084116f0 0x5625084161b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44534/0 (socket says 172.21.15.172:44534) 2024-09-10T08:53:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.193+0000 7fc963fff640 1 -- 172.21.15.172:0/135326260 learned_addr learned my addr 172.21.15.172:0/135326260 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.193+0000 7fc9617fa640 1 --2- 172.21.15.172:0/2830215053 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc94c0a2cb0 0x7fc94c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.193+0000 7fc968cb5640 1 --2- 172.21.15.172:0/135326260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562508410db0 0x5625084111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.193+0000 7fc9637fe640 1 --2- 172.21.15.172:0/135326260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562508416780 0x562508418c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.194+0000 7fc968cb5640 1 --2- 172.21.15.172:0/2830215053 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc94c0a2cb0 0x7fc94c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.194+0000 7fc963fff640 1 -- 172.21.15.172:0/135326260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562508416780 msgr2=0x562508418c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.194+0000 7fc963fff640 1 --2- 172.21.15.172:0/135326260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562508416780 0x562508418c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.194+0000 7fc963fff640 1 -- 172.21.15.172:0/135326260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562508410db0 msgr2=0x5625084111b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.194+0000 7fc963fff640 1 --2- 172.21.15.172:0/135326260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562508410db0 0x5625084111b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.194+0000 7fc963fff640 1 -- 172.21.15.172:0/135326260 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562508317340 con 0x5625084116f0 2024-09-10T08:53:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.194+0000 7fc9617fa640 1 -- 172.21.15.172:0/2830215053 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fc9580efaa0 con 0x7fc964104ef0 2024-09-10T08:53:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.194+0000 7fc963fff640 1 --2- 172.21.15.172:0/135326260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625084116f0 0x5625084161b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fc954009ca0 tx=0x7fc954011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbad68b5bcb8d86f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.194+0000 7fc94a7fc640 1 -- 172.21.15.172:0/135326260 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc954005220 con 0x5625084116f0 2024-09-10T08:53:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.194+0000 7fc968cb5640 1 --2- 172.21.15.172:0/2830215053 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc94c0a2cb0 0x7fc94c0a5170 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc95c005c40 tx=0x7fc95c004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.194+0000 7fc94a7fc640 1 -- 172.21.15.172:0/135326260 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc954004c50 con 0x5625084116f0 2024-09-10T08:53:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.194+0000 7fc94a7fc640 1 -- 172.21.15.172:0/135326260 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9540196d0 con 0x5625084116f0 2024-09-10T08:53:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.195+0000 7fc96d4b1740 1 -- 172.21.15.172:0/135326260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625084116f0 msgr2=0x5625084161b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.195+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/135326260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625084116f0 0x5625084161b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fc954009ca0 tx=0x7fc954011810 comp rx=0 tx=0).stop 2024-09-10T08:53:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.195+0000 7fc96d4b1740 1 -- 172.21.15.172:0/135326260 shutdown_connections 2024-09-10T08:53:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.195+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/135326260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562508416780 0x562508418c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.195+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/135326260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625084116f0 0x5625084161b0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.195+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/135326260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562508410db0 0x5625084111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.195+0000 7fc96d4b1740 1 -- 172.21.15.172:0/135326260 >> 172.21.15.172:0/135326260 conn(0x562508408010 msgr2=0x562508408410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.195+0000 7fc96d4b1740 1 -- 172.21.15.172:0/135326260 shutdown_connections 2024-09-10T08:53:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.195+0000 7fc96d4b1740 1 -- 172.21.15.172:0/135326260 wait complete. 2024-09-10T08:53:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.195+0000 7fc96d4b1740 1 -- start start 2024-09-10T08:53:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.195+0000 7fc96d4b1740 1 Processor -- start 2024-09-10T08:53:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc96d4b1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562508410db0 0x56250841afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc96d4b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625084116f0 0x56250841b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc96d4b1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562508416780 0x562508422520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc96d4b1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562508419750 con 0x562508416780 2024-09-10T08:53:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc96d4b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5625082f9290 con 0x5625084116f0 2024-09-10T08:53:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc96d4b1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5625084198d0 con 0x562508410db0 2024-09-10T08:53:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc96d4b1740 10 client.0 _ll_register_callbacks cb 0x56250832a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc968cb5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562508416780 0x562508422520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc968cb5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562508416780 0x562508422520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44546/0 (socket says 172.21.15.172:44546) 2024-09-10T08:53:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc9637fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625084116f0 0x56250841b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc968cb5640 1 -- 172.21.15.172:0/1669750565 learned_addr learned my addr 172.21.15.172:0/1669750565 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc963fff640 1 --2- 172.21.15.172:0/1669750565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562508410db0 0x56250841afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc968cb5640 1 -- 172.21.15.172:0/1669750565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562508410db0 msgr2=0x56250841afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc968cb5640 1 --2- 172.21.15.172:0/1669750565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562508410db0 0x56250841afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc968cb5640 1 -- 172.21.15.172:0/1669750565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625084116f0 msgr2=0x56250841b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc968cb5640 1 --2- 172.21.15.172:0/1669750565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625084116f0 0x56250841b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc968cb5640 1 -- 172.21.15.172:0/1669750565 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562508083210 con 0x562508416780 2024-09-10T08:53:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc9637fe640 1 --2- 172.21.15.172:0/1669750565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625084116f0 0x56250841b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc968cb5640 1 --2- 172.21.15.172:0/1669750565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562508416780 0x562508422520 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc95c003dc0 tx=0x7fc95c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea439f605225640 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc9437fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc9437fe640 1 -- 172.21.15.172:0/1669750565 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c025030 con 0x562508416780 2024-09-10T08:53:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc9437fe640 1 -- 172.21.15.172:0/1669750565 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fc95c004550 con 0x562508416780 2024-09-10T08:53:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc96d4b1740 1 -- 172.21.15.172:0/1669750565 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc954004c50 con 0x562508416780 2024-09-10T08:53:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc9437fe640 1 -- 172.21.15.172:0/1669750565 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc95c004790 con 0x562508416780 2024-09-10T08:53:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc96d4b1740 10 client.17505 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.196+0000 7fc96d4b1740 1 -- 172.21.15.172:0/1669750565 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56250814f620 con 0x562508416780 2024-09-10T08:53:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.197+0000 7fc9437fe640 1 -- 172.21.15.172:0/1669750565 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc954004c50 con 0x562508416780 2024-09-10T08:53:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.197+0000 7fc9437fe640 1 -- 172.21.15.172:0/1669750565 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fc95c01f040 con 0x562508416780 2024-09-10T08:53:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.197+0000 7fc96d4b1740 10 client.17505 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.197+0000 7fc96d4b1740 10 client.17505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.197+0000 7fc96d4b1740 20 client.17505 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.197+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/1669750565 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562508426c90 0x562508429080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.197+0000 7fc96d4b1740 4 client.17505 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:53:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.197+0000 7fc96d4b1740 1 -- 172.21.15.172:0/1669750565 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56250837da20 con 0x562508426c90 2024-09-10T08:53:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.197+0000 7fc963fff640 1 --2- 172.21.15.172:0/1669750565 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562508426c90 0x562508429080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.197+0000 7fc9437fe640 10 client.17505 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.197+0000 7fc963fff640 1 --2- 172.21.15.172:0/1669750565 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562508426c90 0x562508429080 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e324c29578367ad8 server_cookie=ce391a1654de7207 in_seq=0 out_seq=0 2024-09-10T08:53:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.198+0000 7fc9437fe640 1 -- 172.21.15.172:0/1669750565 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56250837da20 con 0x562508426c90 2024-09-10T08:53:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.198+0000 7fc9437fe640 10 client.17505 handle_command_reply: tid=0 2024-09-10T08:53:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.198+0000 7fc96d4b1740 1 client.17505 shutdown 2024-09-10T08:53:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.199+0000 7fc96d4b1740 1 -- 172.21.15.172:0/1669750565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562508416780 msgr2=0x562508422520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.199+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/1669750565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562508416780 0x562508422520 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc95c003dc0 tx=0x7fc95c004a40 comp rx=0 tx=0).stop 2024-09-10T08:53:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.199+0000 7fc96d4b1740 1 -- 172.21.15.172:0/1669750565 shutdown_connections 2024-09-10T08:53:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.199+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/1669750565 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562508426c90 0x562508429080 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.199+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/1669750565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562508416780 0x562508422520 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.199+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/1669750565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625084116f0 0x56250841b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.199+0000 7fc9437fe640 0 client.17505 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.199+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/1669750565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562508410db0 0x56250841afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.199+0000 7fc96d4b1740 1 -- 172.21.15.172:0/1669750565 >> 172.21.15.172:0/1669750565 conn(0x562508408010 msgr2=0x562508409820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.199+0000 7fc96d4b1740 1 -- 172.21.15.172:0/1669750565 shutdown_connections 2024-09-10T08:53:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.199+0000 7fc96d4b1740 1 -- 172.21.15.172:0/1669750565 wait complete. 2024-09-10T08:53:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.199+0000 7fc96d4b1740 20 client.17505 trim_cache size 0 max 16384 2024-09-10T08:53:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc96d4b1740 1 Processor -- start 2024-09-10T08:53:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc96d4b1740 1 -- start start 2024-09-10T08:53:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc96d4b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562508410db0 0x562508408ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc96d4b1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625084116f0 0x5625084224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc96d4b1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562508416780 0x562508409400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc96d4b1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc9641a82d0 con 0x5625084116f0 2024-09-10T08:53:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc96d4b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9641a9e10 con 0x562508410db0 2024-09-10T08:53:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc96d4b1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc9641a89a0 con 0x562508416780 2024-09-10T08:53:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc9637fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625084116f0 0x5625084224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc9637fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625084116f0 0x5625084224e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44558/0 (socket says 172.21.15.172:44558) 2024-09-10T08:53:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc9637fe640 1 -- 172.21.15.172:0/2878713068 learned_addr learned my addr 172.21.15.172:0/2878713068 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc963fff640 1 --2- 172.21.15.172:0/2878713068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562508410db0 0x562508408ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc968cb5640 1 --2- 172.21.15.172:0/2878713068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562508416780 0x562508409400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc9637fe640 1 -- 172.21.15.172:0/2878713068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562508416780 msgr2=0x562508409400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc9637fe640 1 --2- 172.21.15.172:0/2878713068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562508416780 0x562508409400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc9637fe640 1 -- 172.21.15.172:0/2878713068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562508410db0 msgr2=0x562508408ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc9637fe640 1 --2- 172.21.15.172:0/2878713068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562508410db0 0x562508408ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc9637fe640 1 -- 172.21.15.172:0/2878713068 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562508411bd0 con 0x5625084116f0 2024-09-10T08:53:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.213+0000 7fc9637fe640 1 --2- 172.21.15.172:0/2878713068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625084116f0 0x5625084224e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc9580b8420 tx=0x7fc9580a8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf73eeb44abc679 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.214+0000 7fc949ffb640 1 -- 172.21.15.172:0/2878713068 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958049040 con 0x5625084116f0 2024-09-10T08:53:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.214+0000 7fc949ffb640 1 -- 172.21.15.172:0/2878713068 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9580bc050 con 0x5625084116f0 2024-09-10T08:53:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.214+0000 7fc949ffb640 1 -- 172.21.15.172:0/2878713068 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958003c50 con 0x5625084116f0 2024-09-10T08:53:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.214+0000 7fc96d4b1740 1 -- 172.21.15.172:0/2878713068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625084116f0 msgr2=0x5625084224e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.214+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/2878713068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625084116f0 0x5625084224e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc9580b8420 tx=0x7fc9580a8040 comp rx=0 tx=0).stop 2024-09-10T08:53:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.214+0000 7fc96d4b1740 1 -- 172.21.15.172:0/2878713068 shutdown_connections 2024-09-10T08:53:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.214+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/2878713068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562508416780 0x562508409400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.214+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/2878713068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5625084116f0 0x5625084224e0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.214+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/2878713068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562508410db0 0x562508408ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.214+0000 7fc96d4b1740 1 -- 172.21.15.172:0/2878713068 >> 172.21.15.172:0/2878713068 conn(0x562508408010 msgr2=0x56250841afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.214+0000 7fc96d4b1740 1 -- 172.21.15.172:0/2878713068 shutdown_connections 2024-09-10T08:53:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.214+0000 7fc96d4b1740 1 -- 172.21.15.172:0/2878713068 wait complete. 2024-09-10T08:53:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.214+0000 7fc96d4b1740 1 -- start start 2024-09-10T08:53:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc96d4b1740 1 Processor -- start 2024-09-10T08:53:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc96d4b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562508410db0 0x56250841f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc96d4b1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5625084116f0 0x56250841fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc96d4b1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562508416780 0x562508420ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc96d4b1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc9641a9740 con 0x562508416780 2024-09-10T08:53:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc96d4b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9641a9070 con 0x562508410db0 2024-09-10T08:53:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc96d4b1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc9641a62b0 con 0x5625084116f0 2024-09-10T08:53:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc96d4b1740 10 client.0 _ll_register_callbacks cb 0x7fc95c003a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc968cb5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562508416780 0x562508420ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc968cb5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562508416780 0x562508420ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44572/0 (socket says 172.21.15.172:44572) 2024-09-10T08:53:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc968cb5640 1 -- 172.21.15.172:0/1448946060 learned_addr learned my addr 172.21.15.172:0/1448946060 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc963fff640 1 --2- 172.21.15.172:0/1448946060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562508410db0 0x56250841f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc9637fe640 1 --2- 172.21.15.172:0/1448946060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5625084116f0 0x56250841fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc968cb5640 1 -- 172.21.15.172:0/1448946060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5625084116f0 msgr2=0x56250841fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc968cb5640 1 --2- 172.21.15.172:0/1448946060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5625084116f0 0x56250841fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc968cb5640 1 -- 172.21.15.172:0/1448946060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562508410db0 msgr2=0x56250841f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc968cb5640 1 --2- 172.21.15.172:0/1448946060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562508410db0 0x56250841f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc968cb5640 1 -- 172.21.15.172:0/1448946060 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625084210e0 con 0x562508416780 2024-09-10T08:53:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc963fff640 1 --2- 172.21.15.172:0/1448946060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562508410db0 0x56250841f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.215+0000 7fc9637fe640 1 --2- 172.21.15.172:0/1448946060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5625084116f0 0x56250841fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.216+0000 7fc968cb5640 1 --2- 172.21.15.172:0/1448946060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562508416780 0x562508420ad0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fc95c003a10 tx=0x7fc95c009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac3789483359800 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.216+0000 7fc9437fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.216+0000 7fc9437fe640 1 -- 172.21.15.172:0/1448946060 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c039020 con 0x562508416780 2024-09-10T08:53:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.216+0000 7fc9437fe640 1 -- 172.21.15.172:0/1448946060 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fc95c004420 con 0x562508416780 2024-09-10T08:53:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.216+0000 7fc96d4b1740 1 -- 172.21.15.172:0/1448946060 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9580bc050 con 0x562508416780 2024-09-10T08:53:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.216+0000 7fc9437fe640 1 -- 172.21.15.172:0/1448946060 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc95c044050 con 0x562508416780 2024-09-10T08:53:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.216+0000 7fc96d4b1740 10 client.17511 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.216+0000 7fc96d4b1740 1 -- 172.21.15.172:0/1448946060 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5625084223f0 con 0x562508416780 2024-09-10T08:53:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.216+0000 7fc9437fe640 1 -- 172.21.15.172:0/1448946060 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9580bc050 con 0x562508416780 2024-09-10T08:53:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.216+0000 7fc9437fe640 1 -- 172.21.15.172:0/1448946060 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fc95c0404c0 con 0x562508416780 2024-09-10T08:53:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.216+0000 7fc96d4b1740 10 client.17511 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.216+0000 7fc96d4b1740 10 client.17511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.216+0000 7fc96d4b1740 20 client.17511 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.216+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/1448946060 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562508491d30 0x562508494150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.216+0000 7fc96d4b1740 4 client.17511 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:53:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.216+0000 7fc96d4b1740 1 -- 172.21.15.172:0/1448946060 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562508377d70 con 0x562508491d30 2024-09-10T08:53:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.216+0000 7fc963fff640 1 --2- 172.21.15.172:0/1448946060 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562508491d30 0x562508494150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.217+0000 7fc9437fe640 10 client.17511 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.217+0000 7fc963fff640 1 --2- 172.21.15.172:0/1448946060 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562508491d30 0x562508494150 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7c6a1f3844dd8b5 server_cookie=45cad9d6fa1d4b56 in_seq=0 out_seq=0 2024-09-10T08:53:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.223+0000 7fc9437fe640 1 -- 172.21.15.172:0/1448946060 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562508377d70 con 0x562508491d30 2024-09-10T08:53:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.223+0000 7fc9437fe640 10 client.17511 handle_command_reply: tid=0 2024-09-10T08:53:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.223+0000 7fc96d4b1740 1 client.17511 shutdown 2024-09-10T08:53:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.223+0000 7fc96d4b1740 1 -- 172.21.15.172:0/1448946060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562508416780 msgr2=0x562508420ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.223+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/1448946060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562508416780 0x562508420ad0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fc95c003a10 tx=0x7fc95c009f20 comp rx=0 tx=0).stop 2024-09-10T08:53:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.223+0000 7fc96d4b1740 1 -- 172.21.15.172:0/1448946060 shutdown_connections 2024-09-10T08:53:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.223+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/1448946060 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562508491d30 0x562508494150 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.223+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/1448946060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562508416780 0x562508420ad0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.223+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/1448946060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5625084116f0 0x56250841fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.223+0000 7fc9437fe640 0 client.17511 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.223+0000 7fc96d4b1740 1 --2- 172.21.15.172:0/1448946060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562508410db0 0x56250841f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.223+0000 7fc96d4b1740 1 -- 172.21.15.172:0/1448946060 >> 172.21.15.172:0/1448946060 conn(0x562508408010 msgr2=0x5625084069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.223+0000 7fc96d4b1740 1 -- 172.21.15.172:0/1448946060 shutdown_connections 2024-09-10T08:53:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.223+0000 7fc96d4b1740 1 -- 172.21.15.172:0/1448946060 wait complete. 2024-09-10T08:53:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.223+0000 7fc96d4b1740 20 client.17511 trim_cache size 0 max 16384 2024-09-10T08:53:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:53:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:53:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:53:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:53:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.224+0000 7fc96a718640 1 -- 172.21.15.172:0/2830215053 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc94c0a2cb0 msgr2=0x7fc94c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.224+0000 7fc96a718640 1 --2- 172.21.15.172:0/2830215053 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc94c0a2cb0 0x7fc94c0a5170 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc95c005c40 tx=0x7fc95c004ab0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.224+0000 7fc96a718640 1 -- 172.21.15.172:0/2830215053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc964104ef0 msgr2=0x7fc9641982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.224+0000 7fc96a718640 1 --2- 172.21.15.172:0/2830215053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc964104ef0 0x7fc9641982e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fc9580314e0 tx=0x7fc958004270 comp rx=0 tx=0).stop 2024-09-10T08:53:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.224+0000 7fc96a718640 1 -- 172.21.15.172:0/2830215053 shutdown_connections 2024-09-10T08:53:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.224+0000 7fc96a718640 1 --2- 172.21.15.172:0/2830215053 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fc94c0a2cb0 0x7fc94c0a5170 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.224+0000 7fc96a718640 1 --2- 172.21.15.172:0/2830215053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc964107820 0x7fc96419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.224+0000 7fc96a718640 1 --2- 172.21.15.172:0/2830215053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc964104ef0 0x7fc9641982e0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.224+0000 7fc96a718640 1 --2- 172.21.15.172:0/2830215053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9641025c0 0x7fc964197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.224+0000 7fc96a718640 1 -- 172.21.15.172:0/2830215053 >> 172.21.15.172:0/2830215053 conn(0x7fc9640fc1e0 msgr2=0x7fc964102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.224+0000 7fc96a718640 1 -- 172.21.15.172:0/2830215053 shutdown_connections 2024-09-10T08:53:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:06.224+0000 7fc96a718640 1 -- 172.21.15.172:0/2830215053 wait complete. 2024-09-10T08:53:06.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.239+0000 7f7f677fe640 1 -- 172.21.15.172:0/3903996427 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561fe1669d20 con 0x561fe1717730 2024-09-10T08:53:06.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.239+0000 7f7f677fe640 10 client.26582 handle_command_reply: tid=0 2024-09-10T08:53:06.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.240+0000 7f7f90aac740 1 client.26582 shutdown 2024-09-10T08:53:06.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.240+0000 7f7f90aac740 1 -- 172.21.15.172:0/3903996427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 msgr2=0x561fe171d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.240+0000 7f7f90aac740 1 --2- 172.21.15.172:0/3903996427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 0x561fe171d1a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7f78099220 tx=0x7f7f78010e80 comp rx=0 tx=0).stop 2024-09-10T08:53:06.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.240+0000 7f7f90aac740 1 -- 172.21.15.172:0/3903996427 shutdown_connections 2024-09-10T08:53:06.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.240+0000 7f7f90aac740 1 --2- 172.21.15.172:0/3903996427 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561fe1717730 0x561fe1785600 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.240+0000 7f7f90aac740 1 --2- 172.21.15.172:0/3903996427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561fe1707940 0x561fe171ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.240+0000 7f7f90aac740 1 --2- 172.21.15.172:0/3903996427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561fe16fd080 0x561fe171b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.240+0000 7f7f677fe640 0 client.26582 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:06.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.240+0000 7f7f90aac740 1 --2- 172.21.15.172:0/3903996427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fe16fc760 0x561fe171d1a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.240+0000 7f7f90aac740 1 -- 172.21.15.172:0/3903996427 >> 172.21.15.172:0/3903996427 conn(0x561fe16f8cf0 msgr2=0x561fe16f7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:06.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.240+0000 7f7f90aac740 1 -- 172.21.15.172:0/3903996427 shutdown_connections 2024-09-10T08:53:06.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.240+0000 7f7f90aac740 1 -- 172.21.15.172:0/3903996427 wait complete. 2024-09-10T08:53:06.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.240+0000 7f7f90aac740 20 client.26582 trim_cache size 0 max 16384 2024-09-10T08:53:06.243 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:53:06.243 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 37, 2024-09-10T08:53:06.243 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:53:06.244 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 22, 2024-09-10T08:53:06.244 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:53:06.244 INFO:teuthology.orchestra.run.smithi172.stdout: "c0f1fdaf": { 2024-09-10T08:53:06.244 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 22, 2024-09-10T08:53:06.244 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:53:06.244 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:53:06.244 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:53:06.244 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:53:06.244 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:53:06.244 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:53:06.244 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 140.1, 2024-09-10T08:53:06.244 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:53:06.244 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:53:06.245 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:53:06.245 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:53:06.245 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:53:06.245 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:53:06.245 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:53:06.245 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:53:06.245 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:53:06.245 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:53:06.245 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:53:06.245 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:53:06.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.241+0000 7f7f8e6d3640 1 -- 172.21.15.172:0/1407983699 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7f700a2c00 msgr2=0x7f7f700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.241+0000 7f7f8e6d3640 1 --2- 172.21.15.172:0/1407983699 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7f700a2c00 0x7f7f700a50c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c031440 tx=0x7f7f7c005e60 comp rx=0 tx=0).stop 2024-09-10T08:53:06.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.241+0000 7f7f8e6d3640 1 -- 172.21.15.172:0/1407983699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f88102d50 msgr2=0x7f7f881009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:06.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.241+0000 7f7f8e6d3640 1 --2- 172.21.15.172:0/1407983699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f88102d50 0x7f7f881009b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800b4a0 tx=0x7f7f7800b960 comp rx=0 tx=0).stop 2024-09-10T08:53:06.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.241+0000 7f7f8e6d3640 1 -- 172.21.15.172:0/1407983699 shutdown_connections 2024-09-10T08:53:06.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.241+0000 7f7f8e6d3640 1 --2- 172.21.15.172:0/1407983699 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7f700a2c00 0x7f7f700a50c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.241+0000 7f7f8e6d3640 1 --2- 172.21.15.172:0/1407983699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f8810c180 0x7f7f88101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.241+0000 7f7f8e6d3640 1 --2- 172.21.15.172:0/1407983699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f88103670 0x7f7f88100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.241+0000 7f7f8e6d3640 1 --2- 172.21.15.172:0/1407983699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f88102d50 0x7f7f881009b0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:06.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.241+0000 7f7f8e6d3640 1 -- 172.21.15.172:0/1407983699 >> 172.21.15.172:0/1407983699 conn(0x7f7f880fc7d0 msgr2=0x7f7f880fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:06.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.241+0000 7f7f8e6d3640 1 -- 172.21.15.172:0/1407983699 shutdown_connections 2024-09-10T08:53:06.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:06.241+0000 7f7f8e6d3640 1 -- 172.21.15.172:0/1407983699 wait complete. 2024-09-10T08:53:06.718 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:06 smithi172 ceph-mon[30854]: pgmap v556: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 685 KiB/s wr, 37 op/s 2024-09-10T08:53:06.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:06 smithi174 ceph-mon[34068]: pgmap v556: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 685 KiB/s wr, 37 op/s 2024-09-10T08:53:06.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:06 smithi191 ceph-mon[35314]: pgmap v556: 129 pgs: 129 active+clean; 5.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 685 KiB/s wr, 37 op/s 2024-09-10T08:53:07.066 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c0f1fdaf', quiesce took 3.6 seconds. Will release after: 16.5 2024-09-10T08:53:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:07 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:53:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:07 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:53:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:07 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:53:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:07 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:53:07.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:07 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:53:07.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:07 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:53:07.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:07 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:53:07.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:07 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:53:07.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:07 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:53:07.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:07 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:53:07.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:07 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:53:07.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:07 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:53:08.734 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:08 smithi172 ceph-mon[30854]: pgmap v557: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 19 op/s 2024-09-10T08:53:08.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:08 smithi174 ceph-mon[34068]: pgmap v557: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 19 op/s 2024-09-10T08:53:08.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:08 smithi191 ceph-mon[35314]: pgmap v557: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 19 op/s 2024-09-10T08:53:09.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:09 smithi174 ceph-mon[34068]: pgmap v558: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.4 MiB/s wr, 12 op/s 2024-09-10T08:53:09.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:09 smithi191 ceph-mon[35314]: pgmap v558: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.4 MiB/s wr, 12 op/s 2024-09-10T08:53:09.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:09 smithi172 ceph-mon[30854]: pgmap v558: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.4 MiB/s wr, 12 op/s 2024-09-10T08:53:12.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:12 smithi172 ceph-mon[30854]: pgmap v559: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.1 MiB/s wr, 16 op/s 2024-09-10T08:53:12.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:12 smithi174 ceph-mon[34068]: pgmap v559: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.1 MiB/s wr, 16 op/s 2024-09-10T08:53:12.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:12 smithi191 ceph-mon[35314]: pgmap v559: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.1 MiB/s wr, 16 op/s 2024-09-10T08:53:14.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:14 smithi172 ceph-mon[30854]: pgmap v560: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.1 MiB/s wr, 13 op/s 2024-09-10T08:53:14.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:14 smithi174 ceph-mon[34068]: pgmap v560: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.1 MiB/s wr, 13 op/s 2024-09-10T08:53:14.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:14 smithi191 ceph-mon[35314]: pgmap v560: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 8.1 MiB/s wr, 13 op/s 2024-09-10T08:53:16.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:16 smithi172 ceph-mon[30854]: pgmap v561: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.2 MiB/s wr, 16 op/s 2024-09-10T08:53:16.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:16 smithi174 ceph-mon[34068]: pgmap v561: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.2 MiB/s wr, 16 op/s 2024-09-10T08:53:16.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:16 smithi191 ceph-mon[35314]: pgmap v561: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.2 MiB/s wr, 16 op/s 2024-09-10T08:53:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:18 smithi172 ceph-mon[30854]: pgmap v562: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.6 MiB/s wr, 13 op/s 2024-09-10T08:53:18.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:18 smithi174 ceph-mon[34068]: pgmap v562: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.6 MiB/s wr, 13 op/s 2024-09-10T08:53:18.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:18 smithi191 ceph-mon[35314]: pgmap v562: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.6 MiB/s wr, 13 op/s 2024-09-10T08:53:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.312+0000 7fb869713640 1 -- 172.21.15.172:0/2830981847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb864100d10 msgr2=0x7fb864101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.312+0000 7fb869713640 1 --2- 172.21.15.172:0/2830981847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb864100d10 0x7fb864101190 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb858009900 tx=0x7fb85802a410 comp rx=0 tx=0).stop 2024-09-10T08:53:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.312+0000 7fb869713640 1 -- 172.21.15.172:0/2830981847 shutdown_connections 2024-09-10T08:53:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.312+0000 7fb869713640 1 --2- 172.21.15.172:0/2830981847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8641016d0 0x7fb864108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.312+0000 7fb869713640 1 --2- 172.21.15.172:0/2830981847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb864100d10 0x7fb864101190 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.312+0000 7fb869713640 1 --2- 172.21.15.172:0/2830981847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb864101fe0 0x7fb8641023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.312+0000 7fb869713640 1 -- 172.21.15.172:0/2830981847 >> 172.21.15.172:0/2830981847 conn(0x7fb8640fb930 msgr2=0x7fb8640fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.312+0000 7fb869713640 1 -- 172.21.15.172:0/2830981847 shutdown_connections 2024-09-10T08:53:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.312+0000 7fb869713640 1 -- 172.21.15.172:0/2830981847 wait complete. 2024-09-10T08:53:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.312+0000 7fb869713640 1 Processor -- start 2024-09-10T08:53:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb869713640 1 -- start start 2024-09-10T08:53:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb869713640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb864100d10 0x7fb864104cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb869713640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8641016d0 0x7fb8641032f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb869713640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb864101fe0 0x7fb864103830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb869713640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb86410c5f0 con 0x7fb864100d10 2024-09-10T08:53:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb869713640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb86410c470 con 0x7fb864101fe0 2024-09-10T08:53:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb869713640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb86410c770 con 0x7fb8641016d0 2024-09-10T08:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb862ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb864100d10 0x7fb864104cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb862ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb864100d10 0x7fb864104cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54764/0 (socket says 172.21.15.172:54764) 2024-09-10T08:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb862ffd640 1 -- 172.21.15.172:0/1133983948 learned_addr learned my addr 172.21.15.172:0/1133983948 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb8627fc640 1 --2- 172.21.15.172:0/1133983948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8641016d0 0x7fb8641032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb8637fe640 1 --2- 172.21.15.172:0/1133983948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb864101fe0 0x7fb864103830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb862ffd640 1 -- 172.21.15.172:0/1133983948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8641016d0 msgr2=0x7fb8641032f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb862ffd640 1 --2- 172.21.15.172:0/1133983948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8641016d0 0x7fb8641032f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb862ffd640 1 -- 172.21.15.172:0/1133983948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb864101fe0 msgr2=0x7fb864103830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb862ffd640 1 --2- 172.21.15.172:0/1133983948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb864101fe0 0x7fb864103830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb862ffd640 1 -- 172.21.15.172:0/1133983948 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8641040c0 con 0x7fb864100d10 2024-09-10T08:53:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb8627fc640 1 --2- 172.21.15.172:0/1133983948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8641016d0 0x7fb8641032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb862ffd640 1 --2- 172.21.15.172:0/1133983948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb864100d10 0x7fb864104cb0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb85400d770 tx=0x7fb85400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.313+0000 7fb84bfff640 1 -- 172.21.15.172:0/1133983948 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854017070 con 0x7fb864100d10 2024-09-10T08:53:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.314+0000 7fb869713640 1 -- 172.21.15.172:0/1133983948 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8641a1ea0 con 0x7fb864100d10 2024-09-10T08:53:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.314+0000 7fb84bfff640 1 -- 172.21.15.172:0/1133983948 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb854009c30 con 0x7fb864100d10 2024-09-10T08:53:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.314+0000 7fb84bfff640 1 -- 172.21.15.172:0/1133983948 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854003d00 con 0x7fb864100d10 2024-09-10T08:53:19.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.314+0000 7fb869713640 1 -- 172.21.15.172:0/1133983948 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8641a2320 con 0x7fb864100d10 2024-09-10T08:53:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.316+0000 7fb84bfff640 1 -- 172.21.15.172:0/1133983948 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb854003ea0 con 0x7fb864100d10 2024-09-10T08:53:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.316+0000 7fb86c4a3740 1 Processor -- start 2024-09-10T08:53:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.316+0000 7fb86c4a3740 1 -- start start 2024-09-10T08:53:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.316+0000 7fb86c4a3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3129a0 0x555e5a312da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.316+0000 7fb86c4a3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a3132e0 0x555e5a317da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.316+0000 7fb86c4a3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a31a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.316+0000 7fb86c4a3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555e5a2762f0 con 0x555e5a318370 2024-09-10T08:53:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.316+0000 7fb86c4a3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555e5a21e270 con 0x555e5a3129a0 2024-09-10T08:53:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.316+0000 7fb86c4a3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555e5a2722b0 con 0x555e5a3132e0 2024-09-10T08:53:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.316+0000 7fb84bfff640 1 --2- 172.21.15.172:0/1133983948 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb84c0a2c30 0x7fb84c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.316+0000 7fb8627fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a31a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.317+0000 7fb8627fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a31a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54774/0 (socket says 172.21.15.172:54774) 2024-09-10T08:53:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.317+0000 7fb8627fc640 1 -- 172.21.15.172:0/4025378320 learned_addr learned my addr 172.21.15.172:0/4025378320 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.317+0000 7fb8637fe640 1 --2- 172.21.15.172:0/4025378320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3129a0 0x555e5a312da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.317+0000 7fb8637fe640 1 --2- 172.21.15.172:0/1133983948 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb84c0a2c30 0x7fb84c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.317+0000 7fb84bfff640 1 -- 172.21.15.172:0/1133983948 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb8540cf0e0 con 0x7fb864100d10 2024-09-10T08:53:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.317+0000 7fb862ffd640 1 --2- 172.21.15.172:0/4025378320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a3132e0 0x555e5a317da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.317+0000 7fb8627fc640 1 -- 172.21.15.172:0/4025378320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a3132e0 msgr2=0x555e5a317da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.317+0000 7fb8627fc640 1 --2- 172.21.15.172:0/4025378320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a3132e0 0x555e5a317da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.317+0000 7fb8627fc640 1 -- 172.21.15.172:0/4025378320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3129a0 msgr2=0x555e5a312da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.317+0000 7fb8627fc640 1 --2- 172.21.15.172:0/4025378320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3129a0 0x555e5a312da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.317+0000 7fb8627fc640 1 -- 172.21.15.172:0/4025378320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e5a21d340 con 0x555e5a318370 2024-09-10T08:53:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.317+0000 7fb862ffd640 1 --2- 172.21.15.172:0/4025378320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a3132e0 0x555e5a317da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.317+0000 7fb8627fc640 1 --2- 172.21.15.172:0/4025378320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a31a830 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb85802a930 tx=0x7fb8580318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9361af079704a50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.317+0000 7fb8637fe640 1 --2- 172.21.15.172:0/1133983948 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb84c0a2c30 0x7fb84c0a50f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb85c014960 tx=0x7fb85c001300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.317+0000 7fb8497fa640 1 -- 172.21.15.172:0/4025378320 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858035030 con 0x555e5a318370 2024-09-10T08:53:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.317+0000 7fb8497fa640 1 -- 172.21.15.172:0/4025378320 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb858031cb0 con 0x555e5a318370 2024-09-10T08:53:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.317+0000 7fb8497fa640 1 -- 172.21.15.172:0/4025378320 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858004920 con 0x555e5a318370 2024-09-10T08:53:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.318+0000 7fb86c4a3740 1 -- 172.21.15.172:0/4025378320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 msgr2=0x555e5a31a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.318+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/4025378320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a31a830 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb85802a930 tx=0x7fb8580318d0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.318+0000 7fb86c4a3740 1 -- 172.21.15.172:0/4025378320 shutdown_connections 2024-09-10T08:53:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.318+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/4025378320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a31a830 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.318+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/4025378320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a3132e0 0x555e5a317da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.318+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/4025378320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3129a0 0x555e5a312da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.318+0000 7fb86c4a3740 1 -- 172.21.15.172:0/4025378320 >> 172.21.15.172:0/4025378320 conn(0x555e5a30e010 msgr2=0x555e5a30e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.318+0000 7fb86c4a3740 1 -- 172.21.15.172:0/4025378320 shutdown_connections 2024-09-10T08:53:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.318+0000 7fb86c4a3740 1 -- 172.21.15.172:0/4025378320 wait complete. 2024-09-10T08:53:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.318+0000 7fb86c4a3740 1 -- start start 2024-09-10T08:53:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.318+0000 7fb86c4a3740 1 Processor -- start 2024-09-10T08:53:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.318+0000 7fb86c4a3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3129a0 0x555e5a31caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.318+0000 7fb86c4a3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a3132e0 0x555e5a31cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.318+0000 7fb86c4a3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a318370 0x555e5a324020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.318+0000 7fb86c4a3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555e5a31b2e0 con 0x555e5a3132e0 2024-09-10T08:53:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.318+0000 7fb86c4a3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555e5a1ff290 con 0x555e5a3129a0 2024-09-10T08:53:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.318+0000 7fb86c4a3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555e5a31b460 con 0x555e5a318370 2024-09-10T08:53:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb86c4a3740 10 client.0 _ll_register_callbacks cb 0x555e5a230140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb8627fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a3132e0 0x555e5a31cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb862ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3129a0 0x555e5a31caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb8627fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a3132e0 0x555e5a31cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54780/0 (socket says 172.21.15.172:54780) 2024-09-10T08:53:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb8627fc640 1 -- 172.21.15.172:0/2997154857 learned_addr learned my addr 172.21.15.172:0/2997154857 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb862ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3129a0 0x555e5a31caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:55334/0 (socket says 172.21.15.172:55334) 2024-09-10T08:53:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb8637fe640 1 --2- 172.21.15.172:0/2997154857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a318370 0x555e5a324020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb8627fc640 1 -- 172.21.15.172:0/2997154857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a318370 msgr2=0x555e5a324020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb8627fc640 1 --2- 172.21.15.172:0/2997154857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a318370 0x555e5a324020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb8627fc640 1 -- 172.21.15.172:0/2997154857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3129a0 msgr2=0x555e5a31caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb8627fc640 1 --2- 172.21.15.172:0/2997154857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3129a0 0x555e5a31caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb8627fc640 1 -- 172.21.15.172:0/2997154857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e59f89210 con 0x555e5a3132e0 2024-09-10T08:53:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb862ffd640 1 --2- 172.21.15.172:0/2997154857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3129a0 0x555e5a31caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb8627fc640 1 --2- 172.21.15.172:0/2997154857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a3132e0 0x555e5a31cfe0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb858043a90 tx=0x7fb858043ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7e938182155a98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb846ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb846ff5640 1 -- 172.21.15.172:0/2997154857 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858035030 con 0x555e5a3132e0 2024-09-10T08:53:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb846ff5640 1 -- 172.21.15.172:0/2997154857 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb858031ab0 con 0x555e5a3132e0 2024-09-10T08:53:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb86c4a3740 1 -- 172.21.15.172:0/2997154857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb858031cb0 con 0x555e5a3132e0 2024-09-10T08:53:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb846ff5640 1 -- 172.21.15.172:0/2997154857 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb858031e60 con 0x555e5a3132e0 2024-09-10T08:53:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb86c4a3740 10 client.17523 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.319+0000 7fb86c4a3740 1 -- 172.21.15.172:0/2997154857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555e5a055620 con 0x555e5a3132e0 2024-09-10T08:53:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.320+0000 7fb846ff5640 1 -- 172.21.15.172:0/2997154857 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb858031cb0 con 0x555e5a3132e0 2024-09-10T08:53:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.320+0000 7fb846ff5640 1 -- 172.21.15.172:0/2997154857 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fb858043ce0 con 0x555e5a3132e0 2024-09-10T08:53:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.320+0000 7fb86c4a3740 10 client.17523 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.320+0000 7fb86c4a3740 10 client.17523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.320+0000 7fb86c4a3740 20 client.17523 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.320+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/2997154857 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555e5a328320 0x555e5a32a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.320+0000 7fb86c4a3740 4 client.17523 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:53:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.320+0000 7fb86c4a3740 1 -- 172.21.15.172:0/2997154857 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e5a283a20 con 0x555e5a328320 2024-09-10T08:53:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.320+0000 7fb862ffd640 1 --2- 172.21.15.172:0/2997154857 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555e5a328320 0x555e5a32a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.320+0000 7fb846ff5640 10 client.17523 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.320+0000 7fb862ffd640 1 --2- 172.21.15.172:0/2997154857 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555e5a328320 0x555e5a32a710 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3947b4eec0515824 server_cookie=3a3844e09f83cb0f in_seq=0 out_seq=0 2024-09-10T08:53:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.321+0000 7fb846ff5640 1 -- 172.21.15.172:0/2997154857 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e5a283a20 con 0x555e5a328320 2024-09-10T08:53:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.321+0000 7fb846ff5640 10 client.17523 handle_command_reply: tid=0 2024-09-10T08:53:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.321+0000 7fb86c4a3740 1 client.17523 shutdown 2024-09-10T08:53:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.321+0000 7fb86c4a3740 1 -- 172.21.15.172:0/2997154857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a3132e0 msgr2=0x555e5a31cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.321+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/2997154857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a3132e0 0x555e5a31cfe0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb858043a90 tx=0x7fb858043ac0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.321+0000 7fb86c4a3740 1 -- 172.21.15.172:0/2997154857 shutdown_connections 2024-09-10T08:53:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.321+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/2997154857 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555e5a328320 0x555e5a32a710 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.321+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/2997154857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a318370 0x555e5a324020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.321+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/2997154857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a3132e0 0x555e5a31cfe0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.321+0000 7fb846ff5640 0 client.17523 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.321+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/2997154857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3129a0 0x555e5a31caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.321+0000 7fb86c4a3740 1 -- 172.21.15.172:0/2997154857 >> 172.21.15.172:0/2997154857 conn(0x555e5a30e010 msgr2=0x555e5a30f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.321+0000 7fb86c4a3740 1 -- 172.21.15.172:0/2997154857 shutdown_connections 2024-09-10T08:53:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.321+0000 7fb86c4a3740 1 -- 172.21.15.172:0/2997154857 wait complete. 2024-09-10T08:53:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.321+0000 7fb86c4a3740 20 client.17523 trim_cache size 0 max 16384 2024-09-10T08:53:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.338+0000 7fb86c4a3740 1 Processor -- start 2024-09-10T08:53:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.338+0000 7fb86c4a3740 1 -- start start 2024-09-10T08:53:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.338+0000 7fb86c4a3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a3129a0 0x555e5a30f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.338+0000 7fb86c4a3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3132e0 0x555e5a323fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.338+0000 7fb86c4a3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a30f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.338+0000 7fb86c4a3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641ac7e0 con 0x555e5a318370 2024-09-10T08:53:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.338+0000 7fb86c4a3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641ae320 con 0x555e5a3132e0 2024-09-10T08:53:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.338+0000 7fb86c4a3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641aceb0 con 0x555e5a3129a0 2024-09-10T08:53:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.338+0000 7fb8637fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a30f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.338+0000 7fb8637fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a30f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54790/0 (socket says 172.21.15.172:54790) 2024-09-10T08:53:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.338+0000 7fb8637fe640 1 -- 172.21.15.172:0/1843103205 learned_addr learned my addr 172.21.15.172:0/1843103205 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.339+0000 7fb862ffd640 1 --2- 172.21.15.172:0/1843103205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a3129a0 0x555e5a30f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.339+0000 7fb8627fc640 1 --2- 172.21.15.172:0/1843103205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3132e0 0x555e5a323fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.339+0000 7fb8637fe640 1 -- 172.21.15.172:0/1843103205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a3129a0 msgr2=0x555e5a30f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.339+0000 7fb8637fe640 1 --2- 172.21.15.172:0/1843103205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a3129a0 0x555e5a30f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.339+0000 7fb8637fe640 1 -- 172.21.15.172:0/1843103205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3132e0 msgr2=0x555e5a323fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.339+0000 7fb8637fe640 1 --2- 172.21.15.172:0/1843103205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3132e0 0x555e5a323fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.339+0000 7fb8637fe640 1 -- 172.21.15.172:0/1843103205 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e5a31aba0 con 0x555e5a318370 2024-09-10T08:53:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.339+0000 7fb8637fe640 1 --2- 172.21.15.172:0/1843103205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a30f940 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fb85c014e50 tx=0x7fb85c001450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc93656f6f862f5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.339+0000 7fb848ff9640 1 -- 172.21.15.172:0/1843103205 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c029070 con 0x555e5a318370 2024-09-10T08:53:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.339+0000 7fb848ff9640 1 -- 172.21.15.172:0/1843103205 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb85c00a070 con 0x555e5a318370 2024-09-10T08:53:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.339+0000 7fb848ff9640 1 -- 172.21.15.172:0/1843103205 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c024c40 con 0x555e5a318370 2024-09-10T08:53:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 1 -- 172.21.15.172:0/1843103205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 msgr2=0x555e5a30f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/1843103205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a30f940 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fb85c014e50 tx=0x7fb85c001450 comp rx=0 tx=0).stop 2024-09-10T08:53:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 1 -- 172.21.15.172:0/1843103205 shutdown_connections 2024-09-10T08:53:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/1843103205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a30f940 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/1843103205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3132e0 0x555e5a323fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/1843103205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a3129a0 0x555e5a30f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 1 -- 172.21.15.172:0/1843103205 >> 172.21.15.172:0/1843103205 conn(0x555e5a30e010 msgr2=0x555e5a30d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 1 -- 172.21.15.172:0/1843103205 shutdown_connections 2024-09-10T08:53:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 1 -- 172.21.15.172:0/1843103205 wait complete. 2024-09-10T08:53:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 1 -- start start 2024-09-10T08:53:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 1 Processor -- start 2024-09-10T08:53:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3129a0 0x555e5a321860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a3132e0 0x555e5a321da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a322d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641adc50 con 0x555e5a318370 2024-09-10T08:53:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641ad580 con 0x555e5a3129a0 2024-09-10T08:53:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb8641aa7c0 con 0x555e5a3132e0 2024-09-10T08:53:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.340+0000 7fb86c4a3740 10 client.0 _ll_register_callbacks cb 0x7fb85c001e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb8637fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a322d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb8637fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a322d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54798/0 (socket says 172.21.15.172:54798) 2024-09-10T08:53:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb8637fe640 1 -- 172.21.15.172:0/1018441692 learned_addr learned my addr 172.21.15.172:0/1018441692 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb862ffd640 1 --2- 172.21.15.172:0/1018441692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3129a0 0x555e5a321860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb8627fc640 1 --2- 172.21.15.172:0/1018441692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a3132e0 0x555e5a321da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb8637fe640 1 -- 172.21.15.172:0/1018441692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a3132e0 msgr2=0x555e5a321da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb8637fe640 1 --2- 172.21.15.172:0/1018441692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a3132e0 0x555e5a321da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb8637fe640 1 -- 172.21.15.172:0/1018441692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3129a0 msgr2=0x555e5a321860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb8637fe640 1 --2- 172.21.15.172:0/1018441692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3129a0 0x555e5a321860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb8637fe640 1 -- 172.21.15.172:0/1018441692 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e5a323350 con 0x555e5a318370 2024-09-10T08:53:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb8637fe640 1 --2- 172.21.15.172:0/1018441692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a322d40 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fb85c004b90 tx=0x7fb85c018930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838af4a3a15cb43c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb846ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb846ff5640 1 -- 172.21.15.172:0/1018441692 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c029070 con 0x555e5a318370 2024-09-10T08:53:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb846ff5640 1 -- 172.21.15.172:0/1018441692 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb85c02e460 con 0x555e5a318370 2024-09-10T08:53:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb86c4a3740 1 -- 172.21.15.172:0/1018441692 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb85c00a070 con 0x555e5a318370 2024-09-10T08:53:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb846ff5640 1 -- 172.21.15.172:0/1018441692 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb85c037050 con 0x555e5a318370 2024-09-10T08:53:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb86c4a3740 10 client.17529 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb86c4a3740 1 -- 172.21.15.172:0/1018441692 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555e5a392f60 con 0x555e5a318370 2024-09-10T08:53:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.341+0000 7fb846ff5640 1 -- 172.21.15.172:0/1018441692 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb85c00a070 con 0x555e5a318370 2024-09-10T08:53:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.342+0000 7fb846ff5640 1 -- 172.21.15.172:0/1018441692 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fb85c023070 con 0x555e5a318370 2024-09-10T08:53:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.342+0000 7fb86c4a3740 10 client.17529 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.342+0000 7fb86c4a3740 10 client.17529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.342+0000 7fb86c4a3740 20 client.17529 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.342+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/1018441692 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555e5a393830 0x555e5a395c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.342+0000 7fb86c4a3740 4 client.17529 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:53:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.342+0000 7fb86c4a3740 1 -- 172.21.15.172:0/1018441692 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e5a27dd70 con 0x555e5a393830 2024-09-10T08:53:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.342+0000 7fb862ffd640 1 --2- 172.21.15.172:0/1018441692 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555e5a393830 0x555e5a395c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.342+0000 7fb846ff5640 10 client.17529 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.342+0000 7fb862ffd640 1 --2- 172.21.15.172:0/1018441692 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555e5a393830 0x555e5a395c50 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28997e5f4809d167 server_cookie=1185fa4c1fcf9198 in_seq=0 out_seq=0 2024-09-10T08:53:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb846ff5640 1 -- 172.21.15.172:0/1018441692 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e5a27dd70 con 0x555e5a393830 2024-09-10T08:53:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb846ff5640 10 client.17529 handle_command_reply: tid=0 2024-09-10T08:53:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb86c4a3740 1 client.17529 shutdown 2024-09-10T08:53:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb86c4a3740 1 -- 172.21.15.172:0/1018441692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 msgr2=0x555e5a322d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/1018441692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a322d40 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fb85c004b90 tx=0x7fb85c018930 comp rx=0 tx=0).stop 2024-09-10T08:53:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb86c4a3740 1 -- 172.21.15.172:0/1018441692 shutdown_connections 2024-09-10T08:53:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/1018441692 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555e5a393830 0x555e5a395c50 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/1018441692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e5a318370 0x555e5a322d40 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb846ff5640 0 client.17529 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/1018441692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555e5a3132e0 0x555e5a321da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb86c4a3740 1 --2- 172.21.15.172:0/1018441692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e5a3129a0 0x555e5a321860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb86c4a3740 1 -- 172.21.15.172:0/1018441692 >> 172.21.15.172:0/1018441692 conn(0x555e5a30e010 msgr2=0x555e5a30d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb86c4a3740 1 -- 172.21.15.172:0/1018441692 shutdown_connections 2024-09-10T08:53:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb86c4a3740 1 -- 172.21.15.172:0/1018441692 wait complete. 2024-09-10T08:53:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb86c4a3740 20 client.17529 trim_cache size 0 max 16384 2024-09-10T08:53:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:53:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:53:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:53:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:53:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb869713640 1 -- 172.21.15.172:0/1133983948 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb84c0a2c30 msgr2=0x7fb84c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb869713640 1 --2- 172.21.15.172:0/1133983948 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb84c0a2c30 0x7fb84c0a50f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb85c014960 tx=0x7fb85c001300 comp rx=0 tx=0).stop 2024-09-10T08:53:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb869713640 1 -- 172.21.15.172:0/1133983948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb864100d10 msgr2=0x7fb864104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb869713640 1 --2- 172.21.15.172:0/1133983948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb864100d10 0x7fb864104cb0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb85400d770 tx=0x7fb85400dc30 comp rx=0 tx=0).stop 2024-09-10T08:53:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb869713640 1 -- 172.21.15.172:0/1133983948 shutdown_connections 2024-09-10T08:53:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb869713640 1 --2- 172.21.15.172:0/1133983948 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb84c0a2c30 0x7fb84c0a50f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb869713640 1 --2- 172.21.15.172:0/1133983948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb864101fe0 0x7fb864103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb869713640 1 --2- 172.21.15.172:0/1133983948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8641016d0 0x7fb8641032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb869713640 1 --2- 172.21.15.172:0/1133983948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb864100d10 0x7fb864104cb0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.348+0000 7fb869713640 1 -- 172.21.15.172:0/1133983948 >> 172.21.15.172:0/1133983948 conn(0x7fb8640fb930 msgr2=0x7fb8640fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.349+0000 7fb869713640 1 -- 172.21.15.172:0/1133983948 shutdown_connections 2024-09-10T08:53:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:19.349+0000 7fb869713640 1 -- 172.21.15.172:0/1133983948 wait complete. 2024-09-10T08:53:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:53:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:53:19.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:53:19.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:53:19.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:53:19.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:53:19.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:53:19.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:53:19.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:53:19.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T08:53:19.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:53:19.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:53:19.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:53:20.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:20 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi191 to 176.0M 2024-09-10T08:53:20.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:20 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi172 to 176.0M 2024-09-10T08:53:20.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:20 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-10T08:53:20.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:20 smithi174 ceph-mon[34068]: Unable to set osd_memory_target on smithi191 to 184585318: error parsing value: Value '184585318' is below minimum 939524096 2024-09-10T08:53:20.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:20 smithi174 ceph-mon[34068]: Unable to set osd_memory_target on smithi174 to 184564531: error parsing value: Value '184564531' is below minimum 939524096 2024-09-10T08:53:20.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:20 smithi174 ceph-mon[34068]: Unable to set osd_memory_target on smithi172 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-10T08:53:20.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:20 smithi174 ceph-mon[34068]: pgmap v563: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 MiB/s wr, 12 op/s 2024-09-10T08:53:20.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:20 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi191 to 176.0M 2024-09-10T08:53:20.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:20 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi172 to 176.0M 2024-09-10T08:53:20.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:20 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-10T08:53:20.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:20 smithi191 ceph-mon[35314]: Unable to set osd_memory_target on smithi191 to 184585318: error parsing value: Value '184585318' is below minimum 939524096 2024-09-10T08:53:20.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:20 smithi191 ceph-mon[35314]: Unable to set osd_memory_target on smithi174 to 184564531: error parsing value: Value '184564531' is below minimum 939524096 2024-09-10T08:53:20.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:20 smithi191 ceph-mon[35314]: Unable to set osd_memory_target on smithi172 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-10T08:53:20.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:20 smithi191 ceph-mon[35314]: pgmap v563: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 MiB/s wr, 12 op/s 2024-09-10T08:53:20.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:20 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi191 to 176.0M 2024-09-10T08:53:20.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:20 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi172 to 176.0M 2024-09-10T08:53:20.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:20 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-10T08:53:20.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:20 smithi172 ceph-mon[30854]: Unable to set osd_memory_target on smithi191 to 184585318: error parsing value: Value '184585318' is below minimum 939524096 2024-09-10T08:53:20.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:20 smithi172 ceph-mon[30854]: Unable to set osd_memory_target on smithi174 to 184564531: error parsing value: Value '184564531' is below minimum 939524096 2024-09-10T08:53:20.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:20 smithi172 ceph-mon[30854]: Unable to set osd_memory_target on smithi172 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-10T08:53:20.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:20 smithi172 ceph-mon[30854]: pgmap v563: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 MiB/s wr, 12 op/s 2024-09-10T08:53:22.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:22 smithi174 ceph-mon[34068]: pgmap v564: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.8 MiB/s wr, 14 op/s 2024-09-10T08:53:22.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:22 smithi191 ceph-mon[35314]: pgmap v564: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.8 MiB/s wr, 14 op/s 2024-09-10T08:53:22.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:22 smithi172 ceph-mon[30854]: pgmap v564: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.8 MiB/s wr, 14 op/s 2024-09-10T08:53:23.567 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:53:23.831 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:53:24.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.340+0000 7f27f7a0b640 1 -- 172.21.15.172:0/763385595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27f0108290 msgr2=0x7f27f010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:24.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.340+0000 7f27f7a0b640 1 --2- 172.21.15.172:0/763385595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27f0108290 0x7f27f010ec30 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009c10 tx=0x7f27e802a6e0 comp rx=0 tx=0).stop 2024-09-10T08:53:24.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.340+0000 7f27f7a0b640 1 -- 172.21.15.172:0/763385595 shutdown_connections 2024-09-10T08:53:24.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.340+0000 7f27f7a0b640 1 --2- 172.21.15.172:0/763385595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27f0108290 0x7f27f010ec30 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:24.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.340+0000 7f27f7a0b640 1 --2- 172.21.15.172:0/763385595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27f0103740 0x7f27f0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:24.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.340+0000 7f27f7a0b640 1 --2- 172.21.15.172:0/763385595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27f0102d90 0x7f27f0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:24.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.340+0000 7f27f7a0b640 1 -- 172.21.15.172:0/763385595 >> 172.21.15.172:0/763385595 conn(0x7f27f00fc7d0 msgr2=0x7f27f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:24.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.340+0000 7f27f7a0b640 1 -- 172.21.15.172:0/763385595 shutdown_connections 2024-09-10T08:53:24.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.340+0000 7f27f7a0b640 1 -- 172.21.15.172:0/763385595 wait complete. 2024-09-10T08:53:24.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.341+0000 7f27f7a0b640 1 Processor -- start 2024-09-10T08:53:24.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.341+0000 7f27f7a0b640 1 -- start start 2024-09-10T08:53:24.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.341+0000 7f27f7a0b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27f0102d90 0x7f27f01009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:24.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.341+0000 7f27f7a0b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27f0103740 0x7f27f0100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:24.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.341+0000 7f27f7a0b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27f0108290 0x7f27f0101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:24.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.341+0000 7f27f7a0b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f27f0111f40 con 0x7f27f0108290 2024-09-10T08:53:24.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.341+0000 7f27f7a0b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27f0111dc0 con 0x7f27f0103740 2024-09-10T08:53:24.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.341+0000 7f27f7a0b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01120c0 con 0x7f27f0102d90 2024-09-10T08:53:24.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.342+0000 7f27f5fa8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27f0108290 0x7f27f0101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:24.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.342+0000 7f27f5fa8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27f0108290 0x7f27f0101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54806/0 (socket says 172.21.15.172:54806) 2024-09-10T08:53:24.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.342+0000 7f27f5fa8640 1 -- 172.21.15.172:0/4015785791 learned_addr learned my addr 172.21.15.172:0/4015785791 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:24.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.342+0000 7f27f57a7640 1 --2- 172.21.15.172:0/4015785791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27f0102d90 0x7f27f01009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:24.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.342+0000 7f27f4fa6640 1 --2- 172.21.15.172:0/4015785791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27f0103740 0x7f27f0100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:24.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.342+0000 7f27f5fa8640 1 -- 172.21.15.172:0/4015785791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27f0102d90 msgr2=0x7f27f01009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:24.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.342+0000 7f27f5fa8640 1 --2- 172.21.15.172:0/4015785791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27f0102d90 0x7f27f01009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:24.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.342+0000 7f27f5fa8640 1 -- 172.21.15.172:0/4015785791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27f0103740 msgr2=0x7f27f0100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:24.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.342+0000 7f27f5fa8640 1 --2- 172.21.15.172:0/4015785791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27f0103740 0x7f27f0100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:24.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.342+0000 7f27f5fa8640 1 -- 172.21.15.172:0/4015785791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f01a6950 con 0x7f27f0108290 2024-09-10T08:53:24.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.342+0000 7f27f5fa8640 1 --2- 172.21.15.172:0/4015785791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27f0108290 0x7f27f0101470 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f27e8004af0 tx=0x7f27e8003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:24.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.344+0000 7f27e27fc640 1 -- 172.21.15.172:0/4015785791 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8041020 con 0x7f27f0108290 2024-09-10T08:53:24.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.344+0000 7f27e27fc640 1 -- 172.21.15.172:0/4015785791 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27e8037040 con 0x7f27f0108290 2024-09-10T08:53:24.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.344+0000 7f27f7a0b640 1 -- 172.21.15.172:0/4015785791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f01a6be0 con 0x7f27f0108290 2024-09-10T08:53:24.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.344+0000 7f27e27fc640 1 -- 172.21.15.172:0/4015785791 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e8036070 con 0x7f27f0108290 2024-09-10T08:53:24.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.344+0000 7f27f7a0b640 1 -- 172.21.15.172:0/4015785791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f01a70c0 con 0x7f27f0108290 2024-09-10T08:53:24.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.345+0000 7f27f7a0b640 1 -- 172.21.15.172:0/4015785791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f006a7f0 con 0x7f27f0108290 2024-09-10T08:53:24.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.346+0000 7f27e27fc640 1 -- 172.21.15.172:0/4015785791 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f27e8044020 con 0x7f27f0108290 2024-09-10T08:53:24.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.347+0000 7f27e27fc640 1 --2- 172.21.15.172:0/4015785791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f27dc0a2b80 0x7f27dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:24.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.347+0000 7f27f57a7640 1 --2- 172.21.15.172:0/4015785791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f27dc0a2b80 0x7f27dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:24.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.347+0000 7f27e27fc640 1 -- 172.21.15.172:0/4015785791 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f27e80efc60 con 0x7f27f0108290 2024-09-10T08:53:24.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.347+0000 7f27f57a7640 1 --2- 172.21.15.172:0/4015785791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f27dc0a2b80 0x7f27dc0a5040 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0058c0 tx=0x7f27ec00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:24.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.351+0000 7f27e27fc640 1 -- 172.21.15.172:0/4015785791 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27e804b030 con 0x7f27f0108290 2024-09-10T08:53:24.543 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:24 smithi172 ceph-mon[30854]: pgmap v565: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 132 KiB/s wr, 10 op/s 2024-09-10T08:53:24.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.576+0000 7f27f7a0b640 1 -- 172.21.15.172:0/4015785791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27f010a2a0 con 0x7f27f0108290 2024-09-10T08:53:24.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.576+0000 7f27e27fc640 1 -- 172.21.15.172:0/4015785791 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+12001 (secure 0 0 0) 0x7f27e80bc800 con 0x7f27f0108290 2024-09-10T08:53:24.577 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:53:24.578 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":37,"btime":"2024-09-10T08:52:08:313493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:52:07.658341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:53:24.578 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 37 2024-09-10T08:53:24.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.579+0000 7f27f7a0b640 1 -- 172.21.15.172:0/4015785791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f27dc0a2b80 msgr2=0x7f27dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:24.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.579+0000 7f27f7a0b640 1 --2- 172.21.15.172:0/4015785791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f27dc0a2b80 0x7f27dc0a5040 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0058c0 tx=0x7f27ec00a250 comp rx=0 tx=0).stop 2024-09-10T08:53:24.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.579+0000 7f27f7a0b640 1 -- 172.21.15.172:0/4015785791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27f0108290 msgr2=0x7f27f0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:24.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.579+0000 7f27f7a0b640 1 --2- 172.21.15.172:0/4015785791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27f0108290 0x7f27f0101470 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f27e8004af0 tx=0x7f27e8003d60 comp rx=0 tx=0).stop 2024-09-10T08:53:24.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.579+0000 7f27f7a0b640 1 -- 172.21.15.172:0/4015785791 shutdown_connections 2024-09-10T08:53:24.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.579+0000 7f27f7a0b640 1 --2- 172.21.15.172:0/4015785791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f27dc0a2b80 0x7f27dc0a5040 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:24.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.579+0000 7f27f7a0b640 1 --2- 172.21.15.172:0/4015785791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27f0108290 0x7f27f0101470 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:24.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.579+0000 7f27f7a0b640 1 --2- 172.21.15.172:0/4015785791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27f0103740 0x7f27f0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:24.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.579+0000 7f27f7a0b640 1 --2- 172.21.15.172:0/4015785791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27f0102d90 0x7f27f01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:24.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.579+0000 7f27f7a0b640 1 -- 172.21.15.172:0/4015785791 >> 172.21.15.172:0/4015785791 conn(0x7f27f00fc7d0 msgr2=0x7f27f0105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:24.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.579+0000 7f27f7a0b640 1 -- 172.21.15.172:0/4015785791 shutdown_connections 2024-09-10T08:53:24.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:24.579+0000 7f27f7a0b640 1 -- 172.21.15.172:0/4015785791 wait complete. 2024-09-10T08:53:24.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:24 smithi174 ceph-mon[34068]: pgmap v565: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 132 KiB/s wr, 10 op/s 2024-09-10T08:53:24.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:24 smithi191 ceph-mon[35314]: pgmap v565: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 132 KiB/s wr, 10 op/s 2024-09-10T08:53:25.362 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id c0f1fdaf --release --await' 2024-09-10T08:53:25.362 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id c0f1fdaf --release --await 2024-09-10T08:53:25.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:25 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/4015785791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:53:25.620 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:53:25.688 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:53:25.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:25 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/4015785791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:53:25.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:25 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/4015785791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:53:26.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.112+0000 7fb0d6a71640 1 -- 172.21.15.172:0/4216290939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d0101420 msgr2=0x7fb0d010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:26.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.112+0000 7fb0d6a71640 1 --2- 172.21.15.172:0/4216290939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d0101420 0x7fb0d010d950 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009cd0 tx=0x7fb0c802a8e0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.112+0000 7fb0d6a71640 1 -- 172.21.15.172:0/4216290939 shutdown_connections 2024-09-10T08:53:26.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.112+0000 7fb0d6a71640 1 --2- 172.21.15.172:0/4216290939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d0101420 0x7fb0d010d950 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.112+0000 7fb0d6a71640 1 --2- 172.21.15.172:0/4216290939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d0100a80 0x7fb0d0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.112+0000 7fb0d6a71640 1 --2- 172.21.15.172:0/4216290939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0d0106a80 0x7fb0d0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.112+0000 7fb0d6a71640 1 -- 172.21.15.172:0/4216290939 >> 172.21.15.172:0/4216290939 conn(0x7fb0d00fc7a0 msgr2=0x7fb0d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:26.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.112+0000 7fb0d6a71640 1 -- 172.21.15.172:0/4216290939 shutdown_connections 2024-09-10T08:53:26.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.112+0000 7fb0d6a71640 1 -- 172.21.15.172:0/4216290939 wait complete. 2024-09-10T08:53:26.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0d6a71640 1 Processor -- start 2024-09-10T08:53:26.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0d6a71640 1 -- start start 2024-09-10T08:53:26.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0d6a71640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d0100a80 0x7fb0d01983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0d6a71640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d0101420 0x7fb0d01988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0d6a71640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0d0106a80 0x7fb0d019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0d6a71640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d0110d50 con 0x7fb0d0101420 2024-09-10T08:53:26.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0d6a71640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d0110bd0 con 0x7fb0d0100a80 2024-09-10T08:53:26.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0d6a71640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d0110ed0 con 0x7fb0d0106a80 2024-09-10T08:53:26.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0cffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d0101420 0x7fb0d01988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0cffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d0101420 0x7fb0d01988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54822/0 (socket says 172.21.15.172:54822) 2024-09-10T08:53:26.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0cffff640 1 -- 172.21.15.172:0/781319669 learned_addr learned my addr 172.21.15.172:0/781319669 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:26.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0d500e640 1 --2- 172.21.15.172:0/781319669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0d0106a80 0x7fb0d019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0d480d640 1 --2- 172.21.15.172:0/781319669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d0100a80 0x7fb0d01983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0cffff640 1 -- 172.21.15.172:0/781319669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0d0106a80 msgr2=0x7fb0d019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:26.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0cffff640 1 --2- 172.21.15.172:0/781319669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0d0106a80 0x7fb0d019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0cffff640 1 -- 172.21.15.172:0/781319669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d0100a80 msgr2=0x7fb0d01983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:26.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0cffff640 1 --2- 172.21.15.172:0/781319669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d0100a80 0x7fb0d01983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.113+0000 7fb0cffff640 1 -- 172.21.15.172:0/781319669 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d019d350 con 0x7fb0d0101420 2024-09-10T08:53:26.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.114+0000 7fb0cffff640 1 --2- 172.21.15.172:0/781319669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d0101420 0x7fb0d01988e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00b4a0 tx=0x7fb0bc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:26.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.114+0000 7fb0cdffb640 1 -- 172.21.15.172:0/781319669 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc014070 con 0x7fb0d0101420 2024-09-10T08:53:26.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.114+0000 7fb0cdffb640 1 -- 172.21.15.172:0/781319669 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0bc010ac0 con 0x7fb0d0101420 2024-09-10T08:53:26.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.114+0000 7fb0cdffb640 1 -- 172.21.15.172:0/781319669 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc017c30 con 0x7fb0d0101420 2024-09-10T08:53:26.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.114+0000 7fb0d6a71640 1 -- 172.21.15.172:0/781319669 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d019d640 con 0x7fb0d0101420 2024-09-10T08:53:26.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.114+0000 7fb0d6a71640 1 -- 172.21.15.172:0/781319669 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d01a4f20 con 0x7fb0d0101420 2024-09-10T08:53:26.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.116+0000 7fb0cdffb640 1 -- 172.21.15.172:0/781319669 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb0bc017070 con 0x7fb0d0101420 2024-09-10T08:53:26.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.117+0000 7fb0d8e4a740 1 Processor -- start 2024-09-10T08:53:26.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.117+0000 7fb0cdffb640 1 --2- 172.21.15.172:0/781319669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0c00a2b80 0x7fb0c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.117+0000 7fb0cdffb640 1 -- 172.21.15.172:0/781319669 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb0bc0ceb20 con 0x7fb0d0101420 2024-09-10T08:53:26.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.117+0000 7fb0d8e4a740 1 -- start start 2024-09-10T08:53:26.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.117+0000 7fb0d500e640 1 --2- 172.21.15.172:0/781319669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0c00a2b80 0x7fb0c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0d8e4a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 0x557b147e0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0d8e4a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147e0bc0 0x557b147dabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0d8e4a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147db260 0x557b147db6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0d8e4a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557b14736620 con 0x557b147db260 2024-09-10T08:53:26.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0d8e4a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b146dde50 con 0x557b147e0bc0 2024-09-10T08:53:26.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0d8e4a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557b147325e0 con 0x557b147e02a0 2024-09-10T08:53:26.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0d480d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 0x557b147e0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0d480d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 0x557b147e0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:45454/0 (socket says 172.21.15.172:45454) 2024-09-10T08:53:26.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0d480d640 1 -- 172.21.15.172:0/3076047449 learned_addr learned my addr 172.21.15.172:0/3076047449 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:26.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0d500e640 1 --2- 172.21.15.172:0/3076047449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147db260 0x557b147db6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0cffff640 1 --2- 172.21.15.172:0/3076047449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147e0bc0 0x557b147dabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0d500e640 1 --2- 172.21.15.172:0/781319669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0c00a2b80 0x7fb0c00a5040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8006010 tx=0x7fb0c8009a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:26.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0d480d640 1 -- 172.21.15.172:0/3076047449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147e0bc0 msgr2=0x557b147dabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:26.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0d480d640 1 --2- 172.21.15.172:0/3076047449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147e0bc0 0x557b147dabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0d480d640 1 -- 172.21.15.172:0/3076047449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147db260 msgr2=0x557b147db6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:26.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0d480d640 1 --2- 172.21.15.172:0/3076047449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147db260 0x557b147db6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0d480d640 1 -- 172.21.15.172:0/3076047449 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b146ced40 con 0x557b147e02a0 2024-09-10T08:53:26.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0cffff640 1 --2- 172.21.15.172:0/3076047449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147e0bc0 0x557b147dabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:26.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.118+0000 7fb0d500e640 1 --2- 172.21.15.172:0/3076047449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147db260 0x557b147db6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:26.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.119+0000 7fb0d480d640 1 --2- 172.21.15.172:0/3076047449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 0x557b147e0680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4009b60 tx=0x7fb0c400e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3490e29955d6c617 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:26.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.119+0000 7fb0baffd640 1 -- 172.21.15.172:0/3076047449 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4016070 con 0x557b147e02a0 2024-09-10T08:53:26.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.119+0000 7fb0baffd640 1 -- 172.21.15.172:0/3076047449 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0c4009e50 con 0x557b147e02a0 2024-09-10T08:53:26.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.119+0000 7fb0baffd640 1 -- 172.21.15.172:0/3076047449 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4004dd0 con 0x557b147e02a0 2024-09-10T08:53:26.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.120+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/3076047449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 msgr2=0x557b147e0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:26.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.120+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/3076047449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 0x557b147e0680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4009b60 tx=0x7fb0c400e6c0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.120+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/3076047449 shutdown_connections 2024-09-10T08:53:26.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.120+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/3076047449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147db260 0x557b147db6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.120+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/3076047449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147e0bc0 0x557b147dabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.120+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/3076047449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 0x557b147e0680 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.120+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/3076047449 >> 172.21.15.172:0/3076047449 conn(0x557b147cd0a0 msgr2=0x557b147cd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:26.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.120+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/3076047449 shutdown_connections 2024-09-10T08:53:26.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.120+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/3076047449 wait complete. 2024-09-10T08:53:26.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.120+0000 7fb0d8e4a740 1 -- start start 2024-09-10T08:53:26.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.120+0000 7fb0d8e4a740 1 Processor -- start 2024-09-10T08:53:26.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0d8e4a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147db260 0x557b147e4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0d8e4a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 0x557b147e4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0d8e4a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 0x557b147e4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0d8e4a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557b147e75e0 con 0x557b147e0bc0 2024-09-10T08:53:26.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0d8e4a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b146bef10 con 0x557b147db260 2024-09-10T08:53:26.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0d8e4a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557b147e7760 con 0x557b147e02a0 2024-09-10T08:53:26.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0d8e4a740 10 client.0 _ll_register_callbacks cb 0x557b146e2db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:26.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0d500e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 0x557b147e4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0cffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 0x557b147e4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0cffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 0x557b147e4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:45462/0 (socket says 172.21.15.172:45462) 2024-09-10T08:53:26.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0cffff640 1 -- 172.21.15.172:0/1072921031 learned_addr learned my addr 172.21.15.172:0/1072921031 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:26.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0d480d640 1 --2- 172.21.15.172:0/1072921031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147db260 0x557b147e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0cffff640 1 -- 172.21.15.172:0/1072921031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147db260 msgr2=0x557b147e4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:26.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0cffff640 1 --2- 172.21.15.172:0/1072921031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147db260 0x557b147e4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0cffff640 1 -- 172.21.15.172:0/1072921031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 msgr2=0x557b147e4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:26.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0cffff640 1 --2- 172.21.15.172:0/1072921031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 0x557b147e4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0cffff640 1 -- 172.21.15.172:0/1072921031 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b14445580 con 0x557b147e02a0 2024-09-10T08:53:26.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0cffff640 1 --2- 172.21.15.172:0/1072921031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 0x557b147e4640 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc09afb0 tx=0x7fb0bc09b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71f5f7924ecb44fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:26.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.121+0000 7fb0affff640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:53:26.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.122+0000 7fb0affff640 1 -- 172.21.15.172:0/1072921031 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc01e070 con 0x557b147e02a0 2024-09-10T08:53:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.122+0000 7fb0affff640 1 -- 172.21.15.172:0/1072921031 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb0bc099320 con 0x557b147e02a0 2024-09-10T08:53:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.122+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/1072921031 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0c4009e50 con 0x557b147e02a0 2024-09-10T08:53:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.122+0000 7fb0affff640 1 -- 172.21.15.172:0/1072921031 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb0bc09f0d0 con 0x557b147e02a0 2024-09-10T08:53:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.122+0000 7fb0d8e4a740 10 client.26560 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.122+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/1072921031 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557b1450f960 con 0x557b147e02a0 2024-09-10T08:53:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.122+0000 7fb0affff640 1 -- 172.21.15.172:0/1072921031 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0c4009e50 con 0x557b147e02a0 2024-09-10T08:53:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.122+0000 7fb0affff640 1 -- 172.21.15.172:0/1072921031 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fb0bc0a3800 con 0x557b147e02a0 2024-09-10T08:53:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.122+0000 7fb0d8e4a740 10 client.26560 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.122+0000 7fb0d8e4a740 10 client.26560 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:53:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.122+0000 7fb0d8e4a740 20 client.26560 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.122+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/1072921031 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557b147f4670 0x557b147f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.122+0000 7fb0d8e4a740 4 client.26560 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:53:26.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.122+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/1072921031 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b14743b90 con 0x557b147f4670 2024-09-10T08:53:26.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.123+0000 7fb0d480d640 1 --2- 172.21.15.172:0/1072921031 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557b147f4670 0x557b147f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.123+0000 7fb0d480d640 1 --2- 172.21.15.172:0/1072921031 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557b147f4670 0x557b147f6a60 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=334a1be7bfb480f9 server_cookie=88a9b1c357068317 in_seq=0 out_seq=0 2024-09-10T08:53:26.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.123+0000 7fb0affff640 10 client.26560 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:26.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.124+0000 7fb0affff640 1 -- 172.21.15.172:0/1072921031 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b14743b90 con 0x557b147f4670 2024-09-10T08:53:26.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.124+0000 7fb0affff640 10 client.26560 handle_command_reply: tid=0 2024-09-10T08:53:26.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.124+0000 7fb0d8e4a740 1 client.26560 shutdown 2024-09-10T08:53:26.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.124+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/1072921031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 msgr2=0x557b147e4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:26.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.124+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/1072921031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 0x557b147e4640 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc09afb0 tx=0x7fb0bc09b860 comp rx=0 tx=0).stop 2024-09-10T08:53:26.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.124+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/1072921031 shutdown_connections 2024-09-10T08:53:26.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.124+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/1072921031 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557b147f4670 0x557b147f6a60 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.124+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/1072921031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 0x557b147e4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.124+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/1072921031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 0x557b147e4640 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.124+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/1072921031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147db260 0x557b147e4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.124+0000 7fb0affff640 0 client.26560 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:26.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.124+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/1072921031 >> 172.21.15.172:0/1072921031 conn(0x557b147cd0a0 msgr2=0x557b147cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:26.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.124+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/1072921031 shutdown_connections 2024-09-10T08:53:26.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.124+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/1072921031 wait complete. 2024-09-10T08:53:26.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.124+0000 7fb0d8e4a740 20 client.26560 trim_cache size 0 max 16384 2024-09-10T08:53:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.142+0000 7fb0d8e4a740 1 Processor -- start 2024-09-10T08:53:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.142+0000 7fb0d8e4a740 1 -- start start 2024-09-10T08:53:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.142+0000 7fb0d8e4a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147db260 0x557b147dabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.142+0000 7fb0d8e4a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147e02a0 0x557b147dfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.142+0000 7fb0d8e4a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 0x557b147fc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.142+0000 7fb0d8e4a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d01ac880 con 0x557b147e0bc0 2024-09-10T08:53:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.142+0000 7fb0d8e4a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d01ae3c0 con 0x557b147e02a0 2024-09-10T08:53:26.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.142+0000 7fb0d8e4a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d01acf50 con 0x557b147db260 2024-09-10T08:53:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.143+0000 7fb0d500e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 0x557b147fc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.143+0000 7fb0d500e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 0x557b147fc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54856/0 (socket says 172.21.15.172:54856) 2024-09-10T08:53:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.143+0000 7fb0d500e640 1 -- 172.21.15.172:0/2162352162 learned_addr learned my addr 172.21.15.172:0/2162352162 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.143+0000 7fb0d480d640 1 --2- 172.21.15.172:0/2162352162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147db260 0x557b147dabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.143+0000 7fb0cffff640 1 --2- 172.21.15.172:0/2162352162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147e02a0 0x557b147dfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.143+0000 7fb0d500e640 1 -- 172.21.15.172:0/2162352162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147db260 msgr2=0x557b147dabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.143+0000 7fb0d500e640 1 --2- 172.21.15.172:0/2162352162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147db260 0x557b147dabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.143+0000 7fb0d500e640 1 -- 172.21.15.172:0/2162352162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147e02a0 msgr2=0x557b147dfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.143+0000 7fb0d500e640 1 --2- 172.21.15.172:0/2162352162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147e02a0 0x557b147dfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.143+0000 7fb0d500e640 1 -- 172.21.15.172:0/2162352162 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b147dddd0 con 0x557b147e0bc0 2024-09-10T08:53:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.143+0000 7fb0cffff640 1 --2- 172.21.15.172:0/2162352162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147e02a0 0x557b147dfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:53:26.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.143+0000 7fb0d500e640 1 --2- 172.21.15.172:0/2162352162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 0x557b147fc310 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8035000 tx=0x7fb0c803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798405212ecb6513 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:26.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.143+0000 7fb0ba7fc640 1 -- 172.21.15.172:0/2162352162 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8043070 con 0x557b147e0bc0 2024-09-10T08:53:26.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.143+0000 7fb0ba7fc640 1 -- 172.21.15.172:0/2162352162 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0c8036040 con 0x557b147e0bc0 2024-09-10T08:53:26.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.143+0000 7fb0ba7fc640 1 -- 172.21.15.172:0/2162352162 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8030db0 con 0x557b147e0bc0 2024-09-10T08:53:26.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.144+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/2162352162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 msgr2=0x557b147fc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:26.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.144+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/2162352162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 0x557b147fc310 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8035000 tx=0x7fb0c803f000 comp rx=0 tx=0).stop 2024-09-10T08:53:26.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.144+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/2162352162 shutdown_connections 2024-09-10T08:53:26.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.144+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/2162352162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 0x557b147fc310 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.144+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/2162352162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147e02a0 0x557b147dfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.144+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/2162352162 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147db260 0x557b147dabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.144+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/2162352162 >> 172.21.15.172:0/2162352162 conn(0x557b147cd0a0 msgr2=0x557b147e48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:26.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.144+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/2162352162 shutdown_connections 2024-09-10T08:53:26.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.144+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/2162352162 wait complete. 2024-09-10T08:53:26.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.145+0000 7fb0d8e4a740 1 -- start start 2024-09-10T08:53:26.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.145+0000 7fb0d8e4a740 1 Processor -- start 2024-09-10T08:53:26.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.145+0000 7fb0d8e4a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147db260 0x557b147ec9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.145+0000 7fb0d8e4a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 0x557b147ecf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.145+0000 7fb0d8e4a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 0x557b147e99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.145+0000 7fb0d8e4a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d01adcf0 con 0x557b147e0bc0 2024-09-10T08:53:26.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.145+0000 7fb0d8e4a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d01ad620 con 0x557b147db260 2024-09-10T08:53:26.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.145+0000 7fb0d8e4a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d01aa860 con 0x557b147e02a0 2024-09-10T08:53:26.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.145+0000 7fb0d8e4a740 10 client.0 _ll_register_callbacks cb 0x7fb0bc09aa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:26.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.146+0000 7fb0d500e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 0x557b147e99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.146+0000 7fb0cffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 0x557b147ecf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.146+0000 7fb0d500e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 0x557b147e99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54860/0 (socket says 172.21.15.172:54860) 2024-09-10T08:53:26.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.146+0000 7fb0cffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 0x557b147ecf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:45490/0 (socket says 172.21.15.172:45490) 2024-09-10T08:53:26.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.146+0000 7fb0d500e640 1 -- 172.21.15.172:0/1145692739 learned_addr learned my addr 172.21.15.172:0/1145692739 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:26.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.146+0000 7fb0d480d640 1 --2- 172.21.15.172:0/1145692739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147db260 0x557b147ec9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.146+0000 7fb0d500e640 1 -- 172.21.15.172:0/1145692739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 msgr2=0x557b147ecf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:26.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.146+0000 7fb0d500e640 1 --2- 172.21.15.172:0/1145692739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 0x557b147ecf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.146+0000 7fb0d500e640 1 -- 172.21.15.172:0/1145692739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147db260 msgr2=0x557b147ec9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:26.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.146+0000 7fb0d500e640 1 --2- 172.21.15.172:0/1145692739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147db260 0x557b147ec9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.146+0000 7fb0d500e640 1 -- 172.21.15.172:0/1145692739 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b147fcdc0 con 0x557b147e0bc0 2024-09-10T08:53:26.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.146+0000 7fb0d480d640 1 --2- 172.21.15.172:0/1145692739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147db260 0x557b147ec9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:26.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.146+0000 7fb0d500e640 1 --2- 172.21.15.172:0/1145692739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 0x557b147e99c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009bf0 tx=0x7fb0c80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f80f1edf734e40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:26.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.146+0000 7fb0affff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:26.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.146+0000 7fb0affff640 1 -- 172.21.15.172:0/1145692739 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8043050 con 0x557b147e0bc0 2024-09-10T08:53:26.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.146+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/1145692739 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0c8036040 con 0x557b147e0bc0 2024-09-10T08:53:26.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.146+0000 7fb0affff640 1 -- 172.21.15.172:0/1145692739 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb0c8030bc0 con 0x557b147e0bc0 2024-09-10T08:53:26.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.147+0000 7fb0affff640 1 -- 172.21.15.172:0/1145692739 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb0c80305a0 con 0x557b147e0bc0 2024-09-10T08:53:26.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.147+0000 7fb0d8e4a740 10 client.17553 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:26.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.147+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/1145692739 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557b1485f740 con 0x557b147e0bc0 2024-09-10T08:53:26.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.147+0000 7fb0affff640 1 -- 172.21.15.172:0/1145692739 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0c8036040 con 0x557b147e0bc0 2024-09-10T08:53:26.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.147+0000 7fb0affff640 1 -- 172.21.15.172:0/1145692739 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fb0c8030070 con 0x557b147e0bc0 2024-09-10T08:53:26.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.147+0000 7fb0d8e4a740 10 client.17553 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:26.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.147+0000 7fb0d8e4a740 10 client.17553 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:53:26.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.147+0000 7fb0d8e4a740 20 client.17553 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:26.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.147+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/1145692739 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557b1485fe50 0x557b14862270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:26.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.147+0000 7fb0d8e4a740 4 client.17553 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c0f1fdaf", "release": true, "await": true}] 2024-09-10T08:53:26.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.147+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/1145692739 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c0f1fdaf", "release": true, "await": true}) -- 0x557b1473e0a0 con 0x557b1485fe50 2024-09-10T08:53:26.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.147+0000 7fb0d480d640 1 --2- 172.21.15.172:0/1145692739 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557b1485fe50 0x557b14862270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:26.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.148+0000 7fb0d480d640 1 --2- 172.21.15.172:0/1145692739 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557b1485fe50 0x557b14862270 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ac14fcd87b88d2e server_cookie=c771efb9749e9531 in_seq=0 out_seq=0 2024-09-10T08:53:26.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.148+0000 7fb0affff640 10 client.17553 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:26.157 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:53:26.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.166+0000 7fb0affff640 1 -- 172.21.15.172:0/1145692739 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557b1473e0a0 con 0x557b1485fe50 2024-09-10T08:53:26.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.166+0000 7fb0affff640 10 client.17553 handle_command_reply: tid=0 2024-09-10T08:53:26.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.166+0000 7fb0d8e4a740 1 client.17553 shutdown 2024-09-10T08:53:26.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.167+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/1145692739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 msgr2=0x557b147e99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:26.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.167+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/1145692739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 0x557b147e99c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009bf0 tx=0x7fb0c80095c0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.167+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/1145692739 shutdown_connections 2024-09-10T08:53:26.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.167+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/1145692739 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557b1485fe50 0x557b14862270 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.167+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/1145692739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b147e0bc0 0x557b147e99c0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.167+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/1145692739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b147e02a0 0x557b147ecf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.167+0000 7fb0d8e4a740 1 --2- 172.21.15.172:0/1145692739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b147db260 0x557b147ec9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.167+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/1145692739 >> 172.21.15.172:0/1145692739 conn(0x557b147cd0a0 msgr2=0x557b147cbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:26.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.167+0000 7fb0affff640 0 client.17553 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:26.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.167+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/1145692739 shutdown_connections 2024-09-10T08:53:26.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.167+0000 7fb0d8e4a740 1 -- 172.21.15.172:0/1145692739 wait complete. 2024-09-10T08:53:26.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.167+0000 7fb0d8e4a740 20 client.17553 trim_cache size 0 max 16384 2024-09-10T08:53:26.169 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:53:26.169 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 37, 2024-09-10T08:53:26.169 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:53:26.169 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 24, 2024-09-10T08:53:26.169 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:53:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "c0f1fdaf": { 2024-09-10T08:53:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 24, 2024-09-10T08:53:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:53:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:53:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:53:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:53:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:53:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:53:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 140.1, 2024-09-10T08:53:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:53:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:53:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:53:26.170 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:53:26.171 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:53:26.171 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:53:26.171 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:53:26.171 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:53:26.171 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:53:26.171 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:53:26.171 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:53:26.171 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:53:26.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.168+0000 7fb0d6a71640 1 -- 172.21.15.172:0/781319669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0c00a2b80 msgr2=0x7fb0c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:26.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.168+0000 7fb0d6a71640 1 --2- 172.21.15.172:0/781319669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0c00a2b80 0x7fb0c00a5040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8006010 tx=0x7fb0c8009a20 comp rx=0 tx=0).stop 2024-09-10T08:53:26.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.168+0000 7fb0d6a71640 1 -- 172.21.15.172:0/781319669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d0101420 msgr2=0x7fb0d01988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:26.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.168+0000 7fb0d6a71640 1 --2- 172.21.15.172:0/781319669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d0101420 0x7fb0d01988e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00b4a0 tx=0x7fb0bc00b960 comp rx=0 tx=0).stop 2024-09-10T08:53:26.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.168+0000 7fb0d6a71640 1 -- 172.21.15.172:0/781319669 shutdown_connections 2024-09-10T08:53:26.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.168+0000 7fb0d6a71640 1 --2- 172.21.15.172:0/781319669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0c00a2b80 0x7fb0c00a5040 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.168+0000 7fb0d6a71640 1 --2- 172.21.15.172:0/781319669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0d0106a80 0x7fb0d019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.168+0000 7fb0d6a71640 1 --2- 172.21.15.172:0/781319669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0d0101420 0x7fb0d01988e0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.168+0000 7fb0d6a71640 1 --2- 172.21.15.172:0/781319669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d0100a80 0x7fb0d01983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:26.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.168+0000 7fb0d6a71640 1 -- 172.21.15.172:0/781319669 >> 172.21.15.172:0/781319669 conn(0x7fb0d00fc7a0 msgr2=0x7fb0d01048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:26.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.168+0000 7fb0d6a71640 1 -- 172.21.15.172:0/781319669 shutdown_connections 2024-09-10T08:53:26.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:26.169+0000 7fb0d6a71640 1 -- 172.21.15.172:0/781319669 wait complete. 2024-09-10T08:53:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:26 smithi172 ceph-mon[30854]: pgmap v566: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 132 KiB/s wr, 13 op/s 2024-09-10T08:53:26.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:26 smithi174 ceph-mon[34068]: pgmap v566: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 132 KiB/s wr, 13 op/s 2024-09-10T08:53:26.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:26 smithi191 ceph-mon[35314]: pgmap v566: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 132 KiB/s wr, 13 op/s 2024-09-10T08:53:27.283 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c0f1fdaf', total seconds elapsed: 23.8 2024-09-10T08:53:27.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.488+0000 7f2b45f67640 1 -- 172.21.15.172:0/3968535521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b4010c190 msgr2=0x7f2b4010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.488+0000 7f2b45f67640 1 --2- 172.21.15.172:0/3968535521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b4010c190 0x7f2b4010e610 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009ca0 tx=0x7f2b3802a950 comp rx=0 tx=0).stop 2024-09-10T08:53:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.488+0000 7f2b45f67640 1 -- 172.21.15.172:0/3968535521 shutdown_connections 2024-09-10T08:53:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.488+0000 7f2b45f67640 1 --2- 172.21.15.172:0/3968535521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b4010c190 0x7f2b4010e610 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.488+0000 7f2b45f67640 1 --2- 172.21.15.172:0/3968535521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b401036d0 0x7f2b4010bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.488+0000 7f2b45f67640 1 --2- 172.21.15.172:0/3968535521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b40102db0 0x7f2b40103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.488+0000 7f2b45f67640 1 -- 172.21.15.172:0/3968535521 >> 172.21.15.172:0/3968535521 conn(0x7f2b400fc7d0 msgr2=0x7f2b400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.488+0000 7f2b45f67640 1 -- 172.21.15.172:0/3968535521 shutdown_connections 2024-09-10T08:53:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.488+0000 7f2b45f67640 1 -- 172.21.15.172:0/3968535521 wait complete. 2024-09-10T08:53:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.489+0000 7f2b45f67640 1 Processor -- start 2024-09-10T08:53:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.489+0000 7f2b45f67640 1 -- start start 2024-09-10T08:53:27.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.489+0000 7f2b45f67640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b40102db0 0x7f2b401982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.489+0000 7f2b45f67640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b401036d0 0x7f2b40198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.489+0000 7f2b45f67640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b4010c190 0x7f2b4019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.489+0000 7f2b45f67640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b40111e20 con 0x7f2b4010c190 2024-09-10T08:53:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.489+0000 7f2b45f67640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b40111ca0 con 0x7f2b40102db0 2024-09-10T08:53:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.489+0000 7f2b45f67640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2b40111fa0 con 0x7f2b401036d0 2024-09-10T08:53:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.489+0000 7f2b3ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b4010c190 0x7f2b4019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.489+0000 7f2b3ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b4010c190 0x7f2b4019cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40742/0 (socket says 172.21.15.172:40742) 2024-09-10T08:53:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.489+0000 7f2b3ffff640 1 -- 172.21.15.172:0/1451657093 learned_addr learned my addr 172.21.15.172:0/1451657093 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.490+0000 7f2b3effd640 1 --2- 172.21.15.172:0/1451657093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b401036d0 0x7f2b40198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.490+0000 7f2b3f7fe640 1 --2- 172.21.15.172:0/1451657093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b40102db0 0x7f2b401982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.490+0000 7f2b3ffff640 1 -- 172.21.15.172:0/1451657093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b401036d0 msgr2=0x7f2b40198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.490+0000 7f2b3ffff640 1 --2- 172.21.15.172:0/1451657093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b401036d0 0x7f2b40198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.490+0000 7f2b3ffff640 1 -- 172.21.15.172:0/1451657093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b40102db0 msgr2=0x7f2b401982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.490+0000 7f2b3ffff640 1 --2- 172.21.15.172:0/1451657093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b40102db0 0x7f2b401982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.490+0000 7f2b3ffff640 1 -- 172.21.15.172:0/1451657093 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4019d290 con 0x7f2b4010c190 2024-09-10T08:53:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.490+0000 7f2b3ffff640 1 --2- 172.21.15.172:0/1451657093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b4010c190 0x7f2b4019cbb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f2b3802ae70 tx=0x7f2b3802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.490+0000 7f2b3cff9640 1 -- 172.21.15.172:0/1451657093 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38034040 con 0x7f2b4010c190 2024-09-10T08:53:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.490+0000 7f2b45f67640 1 -- 172.21.15.172:0/1451657093 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4019d520 con 0x7f2b4010c190 2024-09-10T08:53:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.490+0000 7f2b3cff9640 1 -- 172.21.15.172:0/1451657093 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b380308f0 con 0x7f2b4010c190 2024-09-10T08:53:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.490+0000 7f2b3cff9640 1 -- 172.21.15.172:0/1451657093 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380047f0 con 0x7f2b4010c190 2024-09-10T08:53:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.491+0000 7f2b45f67640 1 -- 172.21.15.172:0/1451657093 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b401a4e60 con 0x7f2b4010c190 2024-09-10T08:53:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.493+0000 7f2b3cff9640 1 -- 172.21.15.172:0/1451657093 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2b38037020 con 0x7f2b4010c190 2024-09-10T08:53:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b3cff9640 1 --2- 172.21.15.172:0/1451657093 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2b280a2b80 0x7f2b280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b48340740 1 Processor -- start 2024-09-10T08:53:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b48340740 1 -- start start 2024-09-10T08:53:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b3f7fe640 1 --2- 172.21.15.172:0/1451657093 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2b280a2b80 0x7f2b280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b3cff9640 1 -- 172.21.15.172:0/1451657093 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2b380f02d0 con 0x7f2b4010c190 2024-09-10T08:53:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b48340740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddcf4a0 0x55789ddcf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b48340740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddcfdc0 0x55789ddda140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b48340740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddda680 0x55789dddca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b48340740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55789dd352e0 con 0x55789ddda680 2024-09-10T08:53:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b48340740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55789dcdcce0 con 0x55789ddcf4a0 2024-09-10T08:53:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b48340740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55789dd312a0 con 0x55789ddcfdc0 2024-09-10T08:53:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b3ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddda680 0x55789dddca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b3ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddda680 0x55789dddca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40756/0 (socket says 172.21.15.172:40756) 2024-09-10T08:53:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b3ffff640 1 -- 172.21.15.172:0/2824748434 learned_addr learned my addr 172.21.15.172:0/2824748434 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b3f7fe640 1 --2- 172.21.15.172:0/1451657093 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2b280a2b80 0x7f2b280a5040 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000bc30 tx=0x7f2b3000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b3effd640 1 --2- 172.21.15.172:0/2824748434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddcfdc0 0x55789ddda140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b3f7fe640 1 --2- 172.21.15.172:0/2824748434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddcf4a0 0x55789ddcf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b3ffff640 1 -- 172.21.15.172:0/2824748434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddcfdc0 msgr2=0x55789ddda140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:27.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b3ffff640 1 --2- 172.21.15.172:0/2824748434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddcfdc0 0x55789ddda140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b3ffff640 1 -- 172.21.15.172:0/2824748434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddcf4a0 msgr2=0x55789ddcf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:27.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.494+0000 7f2b3ffff640 1 --2- 172.21.15.172:0/2824748434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddcf4a0 0x55789ddcf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.495+0000 7f2b3ffff640 1 -- 172.21.15.172:0/2824748434 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55789dcc1440 con 0x55789ddda680 2024-09-10T08:53:27.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.495+0000 7f2b3effd640 1 --2- 172.21.15.172:0/2824748434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddcfdc0 0x55789ddda140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:27.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.495+0000 7f2b3ffff640 1 --2- 172.21.15.172:0/2824748434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddda680 0x55789dddca70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f2b380bc760 tx=0x7f2b380bc660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4f395cb7419e10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:27.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.495+0000 7f2b25ffb640 1 -- 172.21.15.172:0/2824748434 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3802c040 con 0x55789ddda680 2024-09-10T08:53:27.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.495+0000 7f2b25ffb640 1 -- 172.21.15.172:0/2824748434 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b380bced0 con 0x55789ddda680 2024-09-10T08:53:27.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.495+0000 7f2b25ffb640 1 -- 172.21.15.172:0/2824748434 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380c5440 con 0x55789ddda680 2024-09-10T08:53:27.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.496+0000 7f2b48340740 1 -- 172.21.15.172:0/2824748434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddda680 msgr2=0x55789dddca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:27.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.496+0000 7f2b48340740 1 --2- 172.21.15.172:0/2824748434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddda680 0x55789dddca70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f2b380bc760 tx=0x7f2b380bc660 comp rx=0 tx=0).stop 2024-09-10T08:53:27.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.496+0000 7f2b48340740 1 -- 172.21.15.172:0/2824748434 shutdown_connections 2024-09-10T08:53:27.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.496+0000 7f2b48340740 1 --2- 172.21.15.172:0/2824748434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddda680 0x55789dddca70 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.496+0000 7f2b48340740 1 --2- 172.21.15.172:0/2824748434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddcfdc0 0x55789ddda140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.496+0000 7f2b48340740 1 --2- 172.21.15.172:0/2824748434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddcf4a0 0x55789ddcf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.496+0000 7f2b48340740 1 -- 172.21.15.172:0/2824748434 >> 172.21.15.172:0/2824748434 conn(0x55789ddcba30 msgr2=0x55789ddcbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.496+0000 7f2b48340740 1 -- 172.21.15.172:0/2824748434 shutdown_connections 2024-09-10T08:53:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.496+0000 7f2b48340740 1 -- 172.21.15.172:0/2824748434 wait complete. 2024-09-10T08:53:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.496+0000 7f2b48340740 1 -- start start 2024-09-10T08:53:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.496+0000 7f2b48340740 1 Processor -- start 2024-09-10T08:53:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b48340740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddcf4a0 0x55789ddda030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b48340740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcfdc0 0x55789ddd72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b48340740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddda680 0x55789ddd7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b48340740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55789dddd010 con 0x55789ddcfdc0 2024-09-10T08:53:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b48340740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55789dcf0050 con 0x55789ddda680 2024-09-10T08:53:27.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b48340740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55789dddd190 con 0x55789ddcf4a0 2024-09-10T08:53:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b48340740 10 client.0 _ll_register_callbacks cb 0x55789dd30160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b3effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcfdc0 0x55789ddd72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b3effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcfdc0 0x55789ddd72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40768/0 (socket says 172.21.15.172:40768) 2024-09-10T08:53:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b3effd640 1 -- 172.21.15.172:0/2322020187 learned_addr learned my addr 172.21.15.172:0/2322020187 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b3ffff640 1 --2- 172.21.15.172:0/2322020187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddda680 0x55789ddd7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b3f7fe640 1 --2- 172.21.15.172:0/2322020187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddcf4a0 0x55789ddda030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b3effd640 1 -- 172.21.15.172:0/2322020187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddcf4a0 msgr2=0x55789ddda030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b3effd640 1 --2- 172.21.15.172:0/2322020187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddcf4a0 0x55789ddda030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b3effd640 1 -- 172.21.15.172:0/2322020187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddda680 msgr2=0x55789ddd7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b3effd640 1 --2- 172.21.15.172:0/2322020187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddda680 0x55789ddd7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b3effd640 1 -- 172.21.15.172:0/2322020187 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55789daa63c0 con 0x55789ddcfdc0 2024-09-10T08:53:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b3effd640 1 --2- 172.21.15.172:0/2322020187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcfdc0 0x55789ddd72e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400be30 tx=0x7f2b34012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdef86a9f929c1c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:27.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b1effd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:27.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b1effd640 1 -- 172.21.15.172:0/2322020187 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34004220 con 0x55789ddcfdc0 2024-09-10T08:53:27.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.497+0000 7f2b1effd640 1 -- 172.21.15.172:0/2322020187 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2b340043c0 con 0x55789ddcfdc0 2024-09-10T08:53:27.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.498+0000 7f2b48340740 1 -- 172.21.15.172:0/2322020187 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b380bced0 con 0x55789ddcfdc0 2024-09-10T08:53:27.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.498+0000 7f2b1effd640 1 -- 172.21.15.172:0/2322020187 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2b34010650 con 0x55789ddcfdc0 2024-09-10T08:53:27.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.498+0000 7f2b48340740 10 client.17565 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:27.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.498+0000 7f2b48340740 1 -- 172.21.15.172:0/2322020187 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55789dde89f0 con 0x55789ddcfdc0 2024-09-10T08:53:27.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.498+0000 7f2b1effd640 1 -- 172.21.15.172:0/2322020187 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b380bced0 con 0x55789ddcfdc0 2024-09-10T08:53:27.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.498+0000 7f2b1effd640 1 -- 172.21.15.172:0/2322020187 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f2b34019070 con 0x55789ddcfdc0 2024-09-10T08:53:27.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.498+0000 7f2b48340740 10 client.17565 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:27.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.498+0000 7f2b48340740 10 client.17565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:27.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.498+0000 7f2b48340740 20 client.17565 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:27.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.498+0000 7f2b48340740 1 --2- 172.21.15.172:0/2322020187 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55789dde9f50 0x55789ddec370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.498+0000 7f2b48340740 4 client.17565 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:53:27.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.498+0000 7f2b48340740 1 -- 172.21.15.172:0/2322020187 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55789dd42850 con 0x55789dde9f50 2024-09-10T08:53:27.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.498+0000 7f2b3f7fe640 1 --2- 172.21.15.172:0/2322020187 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55789dde9f50 0x55789ddec370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.499+0000 7f2b1effd640 10 client.17565 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:27.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.499+0000 7f2b3f7fe640 1 --2- 172.21.15.172:0/2322020187 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55789dde9f50 0x55789ddec370 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e97302ec55c15dd0 server_cookie=61f56b2566c6946b in_seq=0 out_seq=0 2024-09-10T08:53:27.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.500+0000 7f2b1effd640 1 -- 172.21.15.172:0/2322020187 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55789dd42850 con 0x55789dde9f50 2024-09-10T08:53:27.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.500+0000 7f2b1effd640 10 client.17565 handle_command_reply: tid=0 2024-09-10T08:53:27.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.500+0000 7f2b48340740 1 client.17565 shutdown 2024-09-10T08:53:27.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.500+0000 7f2b48340740 1 -- 172.21.15.172:0/2322020187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcfdc0 msgr2=0x55789ddd72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:27.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.500+0000 7f2b48340740 1 --2- 172.21.15.172:0/2322020187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcfdc0 0x55789ddd72e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400be30 tx=0x7f2b34012820 comp rx=0 tx=0).stop 2024-09-10T08:53:27.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.500+0000 7f2b48340740 1 -- 172.21.15.172:0/2322020187 shutdown_connections 2024-09-10T08:53:27.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.500+0000 7f2b48340740 1 --2- 172.21.15.172:0/2322020187 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55789dde9f50 0x55789ddec370 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.500+0000 7f2b48340740 1 --2- 172.21.15.172:0/2322020187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddda680 0x55789ddd7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.500+0000 7f2b48340740 1 --2- 172.21.15.172:0/2322020187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcfdc0 0x55789ddd72e0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.500+0000 7f2b48340740 1 --2- 172.21.15.172:0/2322020187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddcf4a0 0x55789ddda030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.500+0000 7f2b1effd640 0 client.17565 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:27.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.500+0000 7f2b48340740 1 -- 172.21.15.172:0/2322020187 >> 172.21.15.172:0/2322020187 conn(0x55789ddcba30 msgr2=0x55789ddd1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:27.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.500+0000 7f2b48340740 1 -- 172.21.15.172:0/2322020187 shutdown_connections 2024-09-10T08:53:27.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.500+0000 7f2b48340740 1 -- 172.21.15.172:0/2322020187 wait complete. 2024-09-10T08:53:27.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.500+0000 7f2b48340740 20 client.17565 trim_cache size 0 max 16384 2024-09-10T08:53:27.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.522+0000 7f2b48340740 1 Processor -- start 2024-09-10T08:53:27.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.522+0000 7f2b48340740 1 -- start start 2024-09-10T08:53:27.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.522+0000 7f2b48340740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddcf4a0 0x55789dde60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.522+0000 7f2b48340740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcfdc0 0x55789ddd9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.522+0000 7f2b48340740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddda680 0x55789ddf1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.522+0000 7f2b48340740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401aea90 con 0x55789ddcfdc0 2024-09-10T08:53:27.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.522+0000 7f2b48340740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401b05d0 con 0x55789ddcf4a0 2024-09-10T08:53:27.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.522+0000 7f2b48340740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401af160 con 0x55789ddda680 2024-09-10T08:53:27.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.522+0000 7f2b3effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcfdc0 0x55789ddd9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.522+0000 7f2b3effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcfdc0 0x55789ddd9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40784/0 (socket says 172.21.15.172:40784) 2024-09-10T08:53:27.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.522+0000 7f2b3effd640 1 -- 172.21.15.172:0/541743660 learned_addr learned my addr 172.21.15.172:0/541743660 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:27.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.522+0000 7f2b3f7fe640 1 --2- 172.21.15.172:0/541743660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddcf4a0 0x55789dde60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.522+0000 7f2b3ffff640 1 --2- 172.21.15.172:0/541743660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddda680 0x55789ddf1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.523+0000 7f2b3effd640 1 -- 172.21.15.172:0/541743660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddda680 msgr2=0x55789ddf1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:27.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.523+0000 7f2b3effd640 1 --2- 172.21.15.172:0/541743660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddda680 0x55789ddf1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.523+0000 7f2b3effd640 1 -- 172.21.15.172:0/541743660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddcf4a0 msgr2=0x55789dde60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:27.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.523+0000 7f2b3effd640 1 --2- 172.21.15.172:0/541743660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddcf4a0 0x55789dde60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.523+0000 7f2b3effd640 1 -- 172.21.15.172:0/541743660 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55789ddec6e0 con 0x55789ddcfdc0 2024-09-10T08:53:27.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.523+0000 7f2b3effd640 1 --2- 172.21.15.172:0/541743660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcfdc0 0x55789ddd9250 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f2b34003cc0 tx=0x7f2b340058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad89aad09c36d0f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:27.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.523+0000 7f2b257fa640 1 -- 172.21.15.172:0/541743660 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b340037f0 con 0x55789ddcfdc0 2024-09-10T08:53:27.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.523+0000 7f2b257fa640 1 -- 172.21.15.172:0/541743660 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b340043e0 con 0x55789ddcfdc0 2024-09-10T08:53:27.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.523+0000 7f2b257fa640 1 -- 172.21.15.172:0/541743660 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34010e90 con 0x55789ddcfdc0 2024-09-10T08:53:27.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.524+0000 7f2b48340740 1 -- 172.21.15.172:0/541743660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcfdc0 msgr2=0x55789ddd9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:27.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.524+0000 7f2b48340740 1 --2- 172.21.15.172:0/541743660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcfdc0 0x55789ddd9250 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f2b34003cc0 tx=0x7f2b340058d0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.524+0000 7f2b48340740 1 -- 172.21.15.172:0/541743660 shutdown_connections 2024-09-10T08:53:27.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.524+0000 7f2b48340740 1 --2- 172.21.15.172:0/541743660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddda680 0x55789ddf1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.524+0000 7f2b48340740 1 --2- 172.21.15.172:0/541743660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcfdc0 0x55789ddd9250 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.524+0000 7f2b48340740 1 --2- 172.21.15.172:0/541743660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddcf4a0 0x55789dde60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.524+0000 7f2b48340740 1 -- 172.21.15.172:0/541743660 >> 172.21.15.172:0/541743660 conn(0x55789ddcba30 msgr2=0x55789ddd8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:27.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.524+0000 7f2b48340740 1 -- 172.21.15.172:0/541743660 shutdown_connections 2024-09-10T08:53:27.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.524+0000 7f2b48340740 1 -- 172.21.15.172:0/541743660 wait complete. 2024-09-10T08:53:27.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.524+0000 7f2b48340740 1 -- start start 2024-09-10T08:53:27.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b48340740 1 Processor -- start 2024-09-10T08:53:27.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b48340740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcf4a0 0x55789dde1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b48340740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddcfdc0 0x55789dde1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b48340740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddda680 0x55789dde1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b48340740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401aff00 con 0x55789ddcf4a0 2024-09-10T08:53:27.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b48340740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401af830 con 0x55789ddcfdc0 2024-09-10T08:53:27.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b48340740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401aca70 con 0x55789ddda680 2024-09-10T08:53:27.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b48340740 10 client.0 _ll_register_callbacks cb 0x7f2b3400b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:27.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b3f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcf4a0 0x55789dde1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b3f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcf4a0 0x55789dde1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40792/0 (socket says 172.21.15.172:40792) 2024-09-10T08:53:27.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b3f7fe640 1 -- 172.21.15.172:0/537313062 learned_addr learned my addr 172.21.15.172:0/537313062 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:27.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b3ffff640 1 --2- 172.21.15.172:0/537313062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddda680 0x55789dde1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b3effd640 1 --2- 172.21.15.172:0/537313062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddcfdc0 0x55789dde1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b3f7fe640 1 -- 172.21.15.172:0/537313062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddda680 msgr2=0x55789dde1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:27.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b3f7fe640 1 --2- 172.21.15.172:0/537313062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddda680 0x55789dde1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b3f7fe640 1 -- 172.21.15.172:0/537313062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddcfdc0 msgr2=0x55789dde1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:27.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b3f7fe640 1 --2- 172.21.15.172:0/537313062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddcfdc0 0x55789dde1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.525+0000 7f2b3f7fe640 1 -- 172.21.15.172:0/537313062 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55789dde2340 con 0x55789ddcf4a0 2024-09-10T08:53:27.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.526+0000 7f2b3f7fe640 1 --2- 172.21.15.172:0/537313062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcf4a0 0x55789dde1220 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x55789ddecbc0 tx=0x7f2b30013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d16095daf36bf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:27.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.526+0000 7f2b1effd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:27.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.526+0000 7f2b1effd640 1 -- 172.21.15.172:0/537313062 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30020540 con 0x55789ddcf4a0 2024-09-10T08:53:27.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.526+0000 7f2b1effd640 1 -- 172.21.15.172:0/537313062 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2b300206e0 con 0x55789ddcf4a0 2024-09-10T08:53:27.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.526+0000 7f2b48340740 1 -- 172.21.15.172:0/537313062 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b340043e0 con 0x55789ddcf4a0 2024-09-10T08:53:27.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.526+0000 7f2b1effd640 1 -- 172.21.15.172:0/537313062 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2b30038050 con 0x55789ddcf4a0 2024-09-10T08:53:27.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.526+0000 7f2b48340740 10 client.17571 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:27.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.526+0000 7f2b48340740 1 -- 172.21.15.172:0/537313062 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55789de59630 con 0x55789ddcf4a0 2024-09-10T08:53:27.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.526+0000 7f2b1effd640 1 -- 172.21.15.172:0/537313062 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b340043e0 con 0x55789ddcf4a0 2024-09-10T08:53:27.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.527+0000 7f2b1effd640 1 -- 172.21.15.172:0/537313062 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f2b30011a20 con 0x55789ddcf4a0 2024-09-10T08:53:27.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.527+0000 7f2b48340740 10 client.17571 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:27.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.527+0000 7f2b48340740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:27.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.527+0000 7f2b48340740 20 client.17571 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:27.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.527+0000 7f2b48340740 1 --2- 172.21.15.172:0/537313062 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55789de59db0 0x55789de5c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:27.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.527+0000 7f2b48340740 4 client.17571 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:53:27.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.527+0000 7f2b48340740 1 -- 172.21.15.172:0/537313062 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55789dd3cd60 con 0x55789de59db0 2024-09-10T08:53:27.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.527+0000 7f2b3effd640 1 --2- 172.21.15.172:0/537313062 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55789de59db0 0x55789de5c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:27.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.527+0000 7f2b1effd640 10 client.17571 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:27.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.527+0000 7f2b3effd640 1 --2- 172.21.15.172:0/537313062 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55789de59db0 0x55789de5c1d0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca27b1a2bf62e052 server_cookie=e0477ff9eb2b62cf in_seq=0 out_seq=0 2024-09-10T08:53:27.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.527+0000 7f2b1effd640 1 -- 172.21.15.172:0/537313062 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55789dd3cd60 con 0x55789de59db0 2024-09-10T08:53:27.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.527+0000 7f2b1effd640 10 client.17571 handle_command_reply: tid=0 2024-09-10T08:53:27.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.527+0000 7f2b48340740 1 client.17571 shutdown 2024-09-10T08:53:27.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.528+0000 7f2b48340740 1 -- 172.21.15.172:0/537313062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcf4a0 msgr2=0x55789dde1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:27.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.528+0000 7f2b48340740 1 --2- 172.21.15.172:0/537313062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcf4a0 0x55789dde1220 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x55789ddecbc0 tx=0x7f2b30013070 comp rx=0 tx=0).stop 2024-09-10T08:53:27.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.528+0000 7f2b48340740 1 -- 172.21.15.172:0/537313062 shutdown_connections 2024-09-10T08:53:27.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.528+0000 7f2b48340740 1 --2- 172.21.15.172:0/537313062 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55789de59db0 0x55789de5c1d0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.528+0000 7f2b48340740 1 --2- 172.21.15.172:0/537313062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55789ddda680 0x55789dde1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.528+0000 7f2b48340740 1 --2- 172.21.15.172:0/537313062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55789ddcfdc0 0x55789dde1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.528+0000 7f2b48340740 1 --2- 172.21.15.172:0/537313062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55789ddcf4a0 0x55789dde1220 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.528+0000 7f2b48340740 1 -- 172.21.15.172:0/537313062 >> 172.21.15.172:0/537313062 conn(0x55789ddcba30 msgr2=0x55789ddd9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:27.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.528+0000 7f2b1effd640 0 client.17571 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:27.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.528+0000 7f2b48340740 1 -- 172.21.15.172:0/537313062 shutdown_connections 2024-09-10T08:53:27.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.528+0000 7f2b48340740 1 -- 172.21.15.172:0/537313062 wait complete. 2024-09-10T08:53:27.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.528+0000 7f2b48340740 20 client.17571 trim_cache size 0 max 16384 2024-09-10T08:53:27.534 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:53:27.534 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T08:53:27.534 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T08:53:27.535 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:53:27.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.528+0000 7f2b45f67640 1 -- 172.21.15.172:0/1451657093 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2b280a2b80 msgr2=0x7f2b280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:27.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.528+0000 7f2b45f67640 1 --2- 172.21.15.172:0/1451657093 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2b280a2b80 0x7f2b280a5040 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000bc30 tx=0x7f2b3000b390 comp rx=0 tx=0).stop 2024-09-10T08:53:27.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.528+0000 7f2b45f67640 1 -- 172.21.15.172:0/1451657093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b4010c190 msgr2=0x7f2b4019cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:27.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.528+0000 7f2b45f67640 1 --2- 172.21.15.172:0/1451657093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b4010c190 0x7f2b4019cbb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f2b3802ae70 tx=0x7f2b3802f5d0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.529+0000 7f2b45f67640 1 -- 172.21.15.172:0/1451657093 shutdown_connections 2024-09-10T08:53:27.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.529+0000 7f2b45f67640 1 --2- 172.21.15.172:0/1451657093 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2b280a2b80 0x7f2b280a5040 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.529+0000 7f2b45f67640 1 --2- 172.21.15.172:0/1451657093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b4010c190 0x7f2b4019cbb0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.529+0000 7f2b45f67640 1 --2- 172.21.15.172:0/1451657093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b401036d0 0x7f2b40198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.529+0000 7f2b45f67640 1 --2- 172.21.15.172:0/1451657093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b40102db0 0x7f2b401982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:27.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.529+0000 7f2b45f67640 1 -- 172.21.15.172:0/1451657093 >> 172.21.15.172:0/1451657093 conn(0x7f2b400fc7d0 msgr2=0x7f2b400fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:27.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.529+0000 7f2b45f67640 1 -- 172.21.15.172:0/1451657093 shutdown_connections 2024-09-10T08:53:27.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:27.529+0000 7f2b45f67640 1 -- 172.21.15.172:0/1451657093 wait complete. 2024-09-10T08:53:28.273 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:53:28.274 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:53:28.551 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:53:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:28 smithi172 ceph-mon[30854]: pgmap v567: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 684 KiB/s wr, 11 op/s 2024-09-10T08:53:28.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:28 smithi174 ceph-mon[34068]: pgmap v567: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 684 KiB/s wr, 11 op/s 2024-09-10T08:53:28.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:28 smithi191 ceph-mon[35314]: pgmap v567: 129 pgs: 129 active+clean; 5.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 684 KiB/s wr, 11 op/s 2024-09-10T08:53:29.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.057+0000 7f0b3f3c8640 1 -- 172.21.15.172:0/410306812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b381027a0 msgr2=0x7f0b381067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:29.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.057+0000 7f0b3f3c8640 1 --2- 172.21.15.172:0/410306812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b381027a0 0x7f0b381067f0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009900 tx=0x7f0b2802a410 comp rx=0 tx=0).stop 2024-09-10T08:53:29.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.058+0000 7f0b3f3c8640 1 -- 172.21.15.172:0/410306812 shutdown_connections 2024-09-10T08:53:29.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.058+0000 7f0b3f3c8640 1 --2- 172.21.15.172:0/410306812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b38106f20 0x7f0b3810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:29.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.058+0000 7f0b3f3c8640 1 --2- 172.21.15.172:0/410306812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b381027a0 0x7f0b381067f0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:29.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.058+0000 7f0b3f3c8640 1 --2- 172.21.15.172:0/410306812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b38101df0 0x7f0b381021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:29.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.058+0000 7f0b3f3c8640 1 -- 172.21.15.172:0/410306812 >> 172.21.15.172:0/410306812 conn(0x7f0b380fd8f0 msgr2=0x7f0b380ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:29.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.058+0000 7f0b3f3c8640 1 -- 172.21.15.172:0/410306812 shutdown_connections 2024-09-10T08:53:29.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.058+0000 7f0b3f3c8640 1 -- 172.21.15.172:0/410306812 wait complete. 2024-09-10T08:53:29.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.058+0000 7f0b3f3c8640 1 Processor -- start 2024-09-10T08:53:29.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.059+0000 7f0b3f3c8640 1 -- start start 2024-09-10T08:53:29.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.059+0000 7f0b3f3c8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b38101df0 0x7f0b381984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:29.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.059+0000 7f0b3f3c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b381027a0 0x7f0b381989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:29.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.059+0000 7f0b3f3c8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b38106f20 0x7f0b3819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:29.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.059+0000 7f0b3f3c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0b38111e30 con 0x7f0b381027a0 2024-09-10T08:53:29.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.059+0000 7f0b3f3c8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b38111cb0 con 0x7f0b38101df0 2024-09-10T08:53:29.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.059+0000 7f0b3f3c8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0b38111fb0 con 0x7f0b38106f20 2024-09-10T08:53:29.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.059+0000 7f0b3c963640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b381027a0 0x7f0b381989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:29.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.059+0000 7f0b3c963640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b381027a0 0x7f0b381989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40818/0 (socket says 172.21.15.172:40818) 2024-09-10T08:53:29.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.059+0000 7f0b3c963640 1 -- 172.21.15.172:0/792134309 learned_addr learned my addr 172.21.15.172:0/792134309 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:29.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.059+0000 7f0b3d164640 1 --2- 172.21.15.172:0/792134309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b38101df0 0x7f0b381984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:29.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.059+0000 7f0b3d965640 1 --2- 172.21.15.172:0/792134309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b38106f20 0x7f0b3819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:29.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.059+0000 7f0b3c963640 1 -- 172.21.15.172:0/792134309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b38106f20 msgr2=0x7f0b3819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:29.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.059+0000 7f0b3c963640 1 --2- 172.21.15.172:0/792134309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b38106f20 0x7f0b3819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:29.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.059+0000 7f0b3c963640 1 -- 172.21.15.172:0/792134309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b38101df0 msgr2=0x7f0b381984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:29.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.059+0000 7f0b3c963640 1 --2- 172.21.15.172:0/792134309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b38101df0 0x7f0b381984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:29.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.059+0000 7f0b3c963640 1 -- 172.21.15.172:0/792134309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3819d460 con 0x7f0b381027a0 2024-09-10T08:53:29.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.060+0000 7f0b3c963640 1 --2- 172.21.15.172:0/792134309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b381027a0 0x7f0b381989f0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0b28005940 tx=0x7f0b280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:29.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.060+0000 7f0b2e7fc640 1 -- 172.21.15.172:0/792134309 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2803f030 con 0x7f0b381027a0 2024-09-10T08:53:29.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.060+0000 7f0b2e7fc640 1 -- 172.21.15.172:0/792134309 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b28003cb0 con 0x7f0b381027a0 2024-09-10T08:53:29.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.060+0000 7f0b3f3c8640 1 -- 172.21.15.172:0/792134309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3819d6f0 con 0x7f0b381027a0 2024-09-10T08:53:29.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.060+0000 7f0b2e7fc640 1 -- 172.21.15.172:0/792134309 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28035070 con 0x7f0b381027a0 2024-09-10T08:53:29.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.060+0000 7f0b3f3c8640 1 -- 172.21.15.172:0/792134309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b381a5030 con 0x7f0b381027a0 2024-09-10T08:53:29.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.061+0000 7f0b3f3c8640 1 -- 172.21.15.172:0/792134309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b381021d0 con 0x7f0b381027a0 2024-09-10T08:53:29.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.063+0000 7f0b2e7fc640 1 -- 172.21.15.172:0/792134309 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0b28042030 con 0x7f0b381027a0 2024-09-10T08:53:29.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.063+0000 7f0b2e7fc640 1 --2- 172.21.15.172:0/792134309 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0b240a2b80 0x7f0b240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:29.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.063+0000 7f0b3d164640 1 --2- 172.21.15.172:0/792134309 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0b240a2b80 0x7f0b240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:29.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.064+0000 7f0b2e7fc640 1 -- 172.21.15.172:0/792134309 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0b280efd60 con 0x7f0b381027a0 2024-09-10T08:53:29.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.064+0000 7f0b3d164640 1 --2- 172.21.15.172:0/792134309 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0b240a2b80 0x7f0b240a5040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0b34005fd0 tx=0x7f0b3400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:29.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.068+0000 7f0b2e7fc640 1 -- 172.21.15.172:0/792134309 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b280bc310 con 0x7f0b381027a0 2024-09-10T08:53:29.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.328+0000 7f0b3f3c8640 1 -- 172.21.15.172:0/792134309 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b3810a100 con 0x7f0b381027a0 2024-09-10T08:53:29.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.329+0000 7f0b2e7fc640 1 -- 172.21.15.172:0/792134309 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+12001 (secure 0 0 0) 0x7f0b280c0610 con 0x7f0b381027a0 2024-09-10T08:53:29.330 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:53:29.330 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":37,"btime":"2024-09-10T08:52:08:313493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:52:07.658341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:53:29.330 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 37 2024-09-10T08:53:29.332 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.331+0000 7f0b3f3c8640 1 -- 172.21.15.172:0/792134309 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0b240a2b80 msgr2=0x7f0b240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:29.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.331+0000 7f0b3f3c8640 1 --2- 172.21.15.172:0/792134309 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0b240a2b80 0x7f0b240a5040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0b34005fd0 tx=0x7f0b3400b390 comp rx=0 tx=0).stop 2024-09-10T08:53:29.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.331+0000 7f0b3f3c8640 1 -- 172.21.15.172:0/792134309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b381027a0 msgr2=0x7f0b381989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:29.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.331+0000 7f0b3f3c8640 1 --2- 172.21.15.172:0/792134309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b381027a0 0x7f0b381989f0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0b28005940 tx=0x7f0b280042f0 comp rx=0 tx=0).stop 2024-09-10T08:53:29.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.331+0000 7f0b3f3c8640 1 -- 172.21.15.172:0/792134309 shutdown_connections 2024-09-10T08:53:29.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.331+0000 7f0b3f3c8640 1 --2- 172.21.15.172:0/792134309 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0b240a2b80 0x7f0b240a5040 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:29.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.331+0000 7f0b3f3c8640 1 --2- 172.21.15.172:0/792134309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b38106f20 0x7f0b3819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:29.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.331+0000 7f0b3f3c8640 1 --2- 172.21.15.172:0/792134309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b381027a0 0x7f0b381989f0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:29.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.331+0000 7f0b3f3c8640 1 --2- 172.21.15.172:0/792134309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b38101df0 0x7f0b381984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:29.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.331+0000 7f0b3f3c8640 1 -- 172.21.15.172:0/792134309 >> 172.21.15.172:0/792134309 conn(0x7f0b380fd8f0 msgr2=0x7f0b380fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:29.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.331+0000 7f0b3f3c8640 1 -- 172.21.15.172:0/792134309 shutdown_connections 2024-09-10T08:53:29.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:29.331+0000 7f0b3f3c8640 1 -- 172.21.15.172:0/792134309 wait complete. 2024-09-10T08:53:29.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:29 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/792134309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:53:29.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:29 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/792134309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:53:29.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:29 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/792134309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:53:30.146 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T08:53:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:53:30.542 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:30 smithi172 ceph-mon[30854]: pgmap v568: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 MiB/s wr, 11 op/s 2024-09-10T08:53:30.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:30 smithi174 ceph-mon[34068]: pgmap v568: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 MiB/s wr, 11 op/s 2024-09-10T08:53:30.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:30 smithi191 ceph-mon[35314]: pgmap v568: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 MiB/s wr, 11 op/s 2024-09-10T08:53:31.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.026+0000 7fb86641b640 1 -- 172.21.15.172:0/2688603039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86010c150 msgr2=0x7fb86010e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:31.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.026+0000 7fb86641b640 1 --2- 172.21.15.172:0/2688603039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86010c150 0x7fb86010e5f0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb858009ca0 tx=0x7fb85802a950 comp rx=0 tx=0).stop 2024-09-10T08:53:31.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.026+0000 7fb86641b640 1 -- 172.21.15.172:0/2688603039 shutdown_connections 2024-09-10T08:53:31.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.026+0000 7fb86641b640 1 --2- 172.21.15.172:0/2688603039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86010c150 0x7fb86010e5f0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.026+0000 7fb86641b640 1 --2- 172.21.15.172:0/2688603039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb860103640 0x7fb86010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.026+0000 7fb86641b640 1 --2- 172.21.15.172:0/2688603039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb860102d20 0x7fb860103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.026+0000 7fb86641b640 1 -- 172.21.15.172:0/2688603039 >> 172.21.15.172:0/2688603039 conn(0x7fb8600fc780 msgr2=0x7fb8600feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:31.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.026+0000 7fb86641b640 1 -- 172.21.15.172:0/2688603039 shutdown_connections 2024-09-10T08:53:31.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.026+0000 7fb86641b640 1 -- 172.21.15.172:0/2688603039 wait complete. 2024-09-10T08:53:31.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.026+0000 7fb86641b640 1 Processor -- start 2024-09-10T08:53:31.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.026+0000 7fb86641b640 1 -- start start 2024-09-10T08:53:31.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.027+0000 7fb86641b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb860102d20 0x7fb860198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.027+0000 7fb86641b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb860103640 0x7fb860198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.027+0000 7fb86641b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86010c150 0x7fb86019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.027+0000 7fb86641b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb860111e50 con 0x7fb86010c150 2024-09-10T08:53:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.027+0000 7fb86641b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb860111cd0 con 0x7fb860102d20 2024-09-10T08:53:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.027+0000 7fb86641b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb860111fd0 con 0x7fb860103640 2024-09-10T08:53:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.027+0000 7fb8649b8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86010c150 0x7fb86019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.027+0000 7fb8649b8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86010c150 0x7fb86019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40828/0 (socket says 172.21.15.172:40828) 2024-09-10T08:53:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.027+0000 7fb8649b8640 1 -- 172.21.15.172:0/2420753817 learned_addr learned my addr 172.21.15.172:0/2420753817 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.027+0000 7fb85ffff640 1 --2- 172.21.15.172:0/2420753817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb860102d20 0x7fb860198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.027+0000 7fb85f7fe640 1 --2- 172.21.15.172:0/2420753817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb860103640 0x7fb860198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.027+0000 7fb8649b8640 1 -- 172.21.15.172:0/2420753817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb860103640 msgr2=0x7fb860198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.027+0000 7fb8649b8640 1 --2- 172.21.15.172:0/2420753817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb860103640 0x7fb860198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.027+0000 7fb8649b8640 1 -- 172.21.15.172:0/2420753817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb860102d20 msgr2=0x7fb860198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:31.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.027+0000 7fb8649b8640 1 --2- 172.21.15.172:0/2420753817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb860102d20 0x7fb860198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.027+0000 7fb8649b8640 1 -- 172.21.15.172:0/2420753817 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86019d3c0 con 0x7fb86010c150 2024-09-10T08:53:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.028+0000 7fb8649b8640 1 --2- 172.21.15.172:0/2420753817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86010c150 0x7fb86019cce0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fb858009dc0 tx=0x7fb858003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.028+0000 7fb85d7fa640 1 -- 172.21.15.172:0/2420753817 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858034040 con 0x7fb86010c150 2024-09-10T08:53:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.028+0000 7fb86641b640 1 -- 172.21.15.172:0/2420753817 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86019d650 con 0x7fb86010c150 2024-09-10T08:53:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.028+0000 7fb85d7fa640 1 -- 172.21.15.172:0/2420753817 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb858030850 con 0x7fb86010c150 2024-09-10T08:53:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.028+0000 7fb85d7fa640 1 -- 172.21.15.172:0/2420753817 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8580047f0 con 0x7fb86010c150 2024-09-10T08:53:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.028+0000 7fb86641b640 1 -- 172.21.15.172:0/2420753817 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8601a4f90 con 0x7fb86010c150 2024-09-10T08:53:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.030+0000 7fb85d7fa640 1 -- 172.21.15.172:0/2420753817 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb858037020 con 0x7fb86010c150 2024-09-10T08:53:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.031+0000 7fb8687f4740 1 Processor -- start 2024-09-10T08:53:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.031+0000 7fb8687f4740 1 -- start start 2024-09-10T08:53:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.031+0000 7fb85d7fa640 1 --2- 172.21.15.172:0/2420753817 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8480a2b80 0x7fb8480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.031+0000 7fb85f7fe640 1 --2- 172.21.15.172:0/2420753817 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8480a2b80 0x7fb8480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.031+0000 7fb8687f4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9aeb40 0x56251f9ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.031+0000 7fb8687f4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9ad5d0 0x56251f9ada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.031+0000 7fb8687f4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9b8370 0x56251f9ba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.031+0000 7fb8687f4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56251f9132f0 con 0x56251f9aeb40 2024-09-10T08:53:31.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.031+0000 7fb8687f4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56251f8bacf0 con 0x56251f9b8370 2024-09-10T08:53:31.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.031+0000 7fb8687f4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56251f90f2b0 con 0x56251f9ad5d0 2024-09-10T08:53:31.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.032+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9aeb40 0x56251f9ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:31.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.032+0000 7fb85d7fa640 1 -- 172.21.15.172:0/2420753817 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb8580f0370 con 0x7fb86010c150 2024-09-10T08:53:31.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.032+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9aeb40 0x56251f9ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40836/0 (socket says 172.21.15.172:40836) 2024-09-10T08:53:31.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.032+0000 7fb85ffff640 1 -- 172.21.15.172:0/1276418371 learned_addr learned my addr 172.21.15.172:0/1276418371 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:31.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.032+0000 7fb85f7fe640 1 --2- 172.21.15.172:0/1276418371 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9ad5d0 0x56251f9ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:31.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.032+0000 7fb8649b8640 1 --2- 172.21.15.172:0/1276418371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9b8370 0x56251f9ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:31.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.032+0000 7fb85ffff640 1 -- 172.21.15.172:0/1276418371 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9ad5d0 msgr2=0x56251f9ada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.032+0000 7fb85ffff640 1 --2- 172.21.15.172:0/1276418371 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9ad5d0 0x56251f9ada10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.032+0000 7fb85ffff640 1 -- 172.21.15.172:0/1276418371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9b8370 msgr2=0x56251f9ba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.032+0000 7fb85ffff640 1 --2- 172.21.15.172:0/1276418371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9b8370 0x56251f9ba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.032+0000 7fb85ffff640 1 -- 172.21.15.172:0/1276418371 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56251f89f3a0 con 0x56251f9aeb40 2024-09-10T08:53:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.032+0000 7fb85f7fe640 1 --2- 172.21.15.172:0/2420753817 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8480a2b80 0x7fb8480a5040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb8600feda0 tx=0x7fb8540127c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.032+0000 7fb85ffff640 1 --2- 172.21.15.172:0/1276418371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9aeb40 0x56251f9ad090 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fb85000b760 tx=0x7fb85000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c55ba408f177882 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.032+0000 7fb8467fc640 1 -- 172.21.15.172:0/1276418371 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850003d50 con 0x56251f9aeb40 2024-09-10T08:53:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.032+0000 7fb8467fc640 1 -- 172.21.15.172:0/1276418371 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb850015070 con 0x56251f9aeb40 2024-09-10T08:53:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.032+0000 7fb8467fc640 1 -- 172.21.15.172:0/1276418371 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850004cd0 con 0x56251f9aeb40 2024-09-10T08:53:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.033+0000 7fb8687f4740 1 -- 172.21.15.172:0/1276418371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9aeb40 msgr2=0x56251f9ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.033+0000 7fb8687f4740 1 --2- 172.21.15.172:0/1276418371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9aeb40 0x56251f9ad090 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fb85000b760 tx=0x7fb85000bc20 comp rx=0 tx=0).stop 2024-09-10T08:53:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.033+0000 7fb8687f4740 1 -- 172.21.15.172:0/1276418371 shutdown_connections 2024-09-10T08:53:31.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.033+0000 7fb8687f4740 1 --2- 172.21.15.172:0/1276418371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9b8370 0x56251f9ba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.033+0000 7fb8687f4740 1 --2- 172.21.15.172:0/1276418371 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9ad5d0 0x56251f9ada10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.033+0000 7fb8687f4740 1 --2- 172.21.15.172:0/1276418371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9aeb40 0x56251f9ad090 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.033+0000 7fb8687f4740 1 -- 172.21.15.172:0/1276418371 >> 172.21.15.172:0/1276418371 conn(0x56251f976310 msgr2=0x56251f9a9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:31.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.033+0000 7fb8687f4740 1 -- 172.21.15.172:0/1276418371 shutdown_connections 2024-09-10T08:53:31.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.033+0000 7fb8687f4740 1 -- 172.21.15.172:0/1276418371 wait complete. 2024-09-10T08:53:31.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.034+0000 7fb8687f4740 1 -- start start 2024-09-10T08:53:31.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.034+0000 7fb8687f4740 1 Processor -- start 2024-09-10T08:53:31.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.034+0000 7fb8687f4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9ad5d0 0x56251f9b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.034+0000 7fb8687f4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9aeb40 0x56251f9b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.034+0000 7fb8687f4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9b8370 0x56251f9b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.034+0000 7fb8687f4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56251f9bb070 con 0x56251f9ad5d0 2024-09-10T08:53:31.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.034+0000 7fb8687f4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56251f8ce060 con 0x56251f9aeb40 2024-09-10T08:53:31.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.034+0000 7fb8687f4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56251f9bb1f0 con 0x56251f9b8370 2024-09-10T08:53:31.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.034+0000 7fb8687f4740 10 client.0 _ll_register_callbacks cb 0x56251f6218c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:31.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.034+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9ad5d0 0x56251f9b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:31.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.034+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9ad5d0 0x56251f9b7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40846/0 (socket says 172.21.15.172:40846) 2024-09-10T08:53:31.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.034+0000 7fb85ffff640 1 -- 172.21.15.172:0/2594801254 learned_addr learned my addr 172.21.15.172:0/2594801254 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:31.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.035+0000 7fb85f7fe640 1 --2- 172.21.15.172:0/2594801254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9aeb40 0x56251f9b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:31.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.035+0000 7fb8649b8640 1 --2- 172.21.15.172:0/2594801254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9b8370 0x56251f9b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:31.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.035+0000 7fb85ffff640 1 -- 172.21.15.172:0/2594801254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9b8370 msgr2=0x56251f9b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:31.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.035+0000 7fb85ffff640 1 --2- 172.21.15.172:0/2594801254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9b8370 0x56251f9b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.035+0000 7fb85ffff640 1 -- 172.21.15.172:0/2594801254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9aeb40 msgr2=0x56251f9b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:31.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.035+0000 7fb85ffff640 1 --2- 172.21.15.172:0/2594801254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9aeb40 0x56251f9b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.035+0000 7fb85ffff640 1 -- 172.21.15.172:0/2594801254 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56251f6843c0 con 0x56251f9ad5d0 2024-09-10T08:53:31.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.035+0000 7fb85f7fe640 1 --2- 172.21.15.172:0/2594801254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9aeb40 0x56251f9b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:31.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.035+0000 7fb85ffff640 1 --2- 172.21.15.172:0/2594801254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9ad5d0 0x56251f9b7d20 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fb850006010 tx=0x7fb850003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31528f5c013a3a7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.035+0000 7fb83f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.035+0000 7fb83f7fe640 1 -- 172.21.15.172:0/2594801254 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850028060 con 0x56251f9ad5d0 2024-09-10T08:53:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.035+0000 7fb83f7fe640 1 -- 172.21.15.172:0/2594801254 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb850011850 con 0x56251f9ad5d0 2024-09-10T08:53:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.035+0000 7fb8687f4740 1 -- 172.21.15.172:0/2594801254 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb850015070 con 0x56251f9ad5d0 2024-09-10T08:53:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.035+0000 7fb83f7fe640 1 -- 172.21.15.172:0/2594801254 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb85002f050 con 0x56251f9ad5d0 2024-09-10T08:53:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.035+0000 7fb8687f4740 10 client.17589 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.036+0000 7fb8687f4740 1 -- 172.21.15.172:0/2594801254 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56251f9c6ac0 con 0x56251f9ad5d0 2024-09-10T08:53:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.036+0000 7fb83f7fe640 1 -- 172.21.15.172:0/2594801254 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb850015070 con 0x56251f9ad5d0 2024-09-10T08:53:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.036+0000 7fb83f7fe640 1 -- 172.21.15.172:0/2594801254 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fb850022030 con 0x56251f9ad5d0 2024-09-10T08:53:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.036+0000 7fb8687f4740 10 client.17589 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.036+0000 7fb8687f4740 10 client.17589 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:53:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.036+0000 7fb8687f4740 20 client.17589 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:31.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.036+0000 7fb8687f4740 1 --2- 172.21.15.172:0/2594801254 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x56251f9c7ec0 0x56251f9ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.036+0000 7fb8687f4740 4 client.17589 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:53:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.036+0000 7fb8687f4740 1 -- 172.21.15.172:0/2594801254 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56251f920860 con 0x56251f9c7ec0 2024-09-10T08:53:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.036+0000 7fb85f7fe640 1 --2- 172.21.15.172:0/2594801254 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x56251f9c7ec0 0x56251f9ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.037+0000 7fb85f7fe640 1 --2- 172.21.15.172:0/2594801254 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x56251f9c7ec0 0x56251f9ca2b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cdcc78d81ef4da06 server_cookie=94af3e0427665d36 in_seq=0 out_seq=0 2024-09-10T08:53:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.037+0000 7fb83f7fe640 10 client.17589 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:53:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.037+0000 7fb83f7fe640 1 -- 172.21.15.172:0/2594801254 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56251f920860 con 0x56251f9c7ec0 2024-09-10T08:53:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.037+0000 7fb83f7fe640 10 client.17589 handle_command_reply: tid=0 2024-09-10T08:53:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.037+0000 7fb8687f4740 1 client.17589 shutdown 2024-09-10T08:53:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.038+0000 7fb8687f4740 1 -- 172.21.15.172:0/2594801254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9ad5d0 msgr2=0x56251f9b7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.038+0000 7fb8687f4740 1 --2- 172.21.15.172:0/2594801254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9ad5d0 0x56251f9b7d20 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fb850006010 tx=0x7fb850003d80 comp rx=0 tx=0).stop 2024-09-10T08:53:31.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.038+0000 7fb8687f4740 1 -- 172.21.15.172:0/2594801254 shutdown_connections 2024-09-10T08:53:31.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.038+0000 7fb8687f4740 1 --2- 172.21.15.172:0/2594801254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9b8370 0x56251f9b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.038+0000 7fb8687f4740 1 --2- 172.21.15.172:0/2594801254 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x56251f9c7ec0 0x56251f9ca2b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.038+0000 7fb83f7fe640 0 client.17589 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:53:31.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.038+0000 7fb8687f4740 1 --2- 172.21.15.172:0/2594801254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9aeb40 0x56251f9b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.038+0000 7fb8687f4740 1 --2- 172.21.15.172:0/2594801254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9ad5d0 0x56251f9b7d20 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.038+0000 7fb8687f4740 1 -- 172.21.15.172:0/2594801254 >> 172.21.15.172:0/2594801254 conn(0x56251f976310 msgr2=0x56251f9af960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:31.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.038+0000 7fb8687f4740 1 -- 172.21.15.172:0/2594801254 shutdown_connections 2024-09-10T08:53:31.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.038+0000 7fb8687f4740 1 -- 172.21.15.172:0/2594801254 wait complete. 2024-09-10T08:53:31.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.038+0000 7fb8687f4740 20 client.17589 trim_cache size 0 max 16384 2024-09-10T08:53:31.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.053+0000 7fb8687f4740 1 Processor -- start 2024-09-10T08:53:31.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.053+0000 7fb8687f4740 1 -- start start 2024-09-10T08:53:31.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.053+0000 7fb8687f4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9ad5d0 0x56251f9c4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.053+0000 7fb8687f4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9aeb40 0x56251f9b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.053+0000 7fb8687f4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9b8370 0x56251f9b0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.054+0000 7fb8687f4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601aeb60 con 0x56251f9ad5d0 2024-09-10T08:53:31.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.054+0000 7fb8687f4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601b06a0 con 0x56251f9aeb40 2024-09-10T08:53:31.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.054+0000 7fb8687f4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601af230 con 0x56251f9b8370 2024-09-10T08:53:31.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.054+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9ad5d0 0x56251f9c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:31.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.054+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9ad5d0 0x56251f9c4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40860/0 (socket says 172.21.15.172:40860) 2024-09-10T08:53:31.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.054+0000 7fb85ffff640 1 -- 172.21.15.172:0/1756609462 learned_addr learned my addr 172.21.15.172:0/1756609462 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:31.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.054+0000 7fb85f7fe640 1 --2- 172.21.15.172:0/1756609462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9aeb40 0x56251f9b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:31.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.054+0000 7fb85f7fe640 1 -- 172.21.15.172:0/1756609462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9b8370 msgr2=0x56251f9b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:31.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.054+0000 7fb85f7fe640 1 --2- 172.21.15.172:0/1756609462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9b8370 0x56251f9b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.054+0000 7fb85f7fe640 1 -- 172.21.15.172:0/1756609462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9ad5d0 msgr2=0x56251f9c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:31.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.054+0000 7fb85f7fe640 1 --2- 172.21.15.172:0/1756609462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9ad5d0 0x56251f9c4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.054+0000 7fb85f7fe640 1 -- 172.21.15.172:0/1756609462 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56251f9b3520 con 0x56251f9aeb40 2024-09-10T08:53:31.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.054+0000 7fb85ffff640 1 --2- 172.21.15.172:0/1756609462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9ad5d0 0x56251f9c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:31.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.054+0000 7fb85f7fe640 1 --2- 172.21.15.172:0/1756609462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9aeb40 0x56251f9b4890 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x56251f9b6580 tx=0x7fb85401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4e85e69737eb1e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:31.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.055+0000 7fb845ffb640 1 -- 172.21.15.172:0/1756609462 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854030030 con 0x56251f9aeb40 2024-09-10T08:53:31.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.055+0000 7fb845ffb640 1 -- 172.21.15.172:0/1756609462 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb85401fdc0 con 0x56251f9aeb40 2024-09-10T08:53:31.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.055+0000 7fb845ffb640 1 -- 172.21.15.172:0/1756609462 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8540047a0 con 0x56251f9aeb40 2024-09-10T08:53:31.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.055+0000 7fb8687f4740 1 -- 172.21.15.172:0/1756609462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9aeb40 msgr2=0x56251f9b4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:31.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.055+0000 7fb8687f4740 1 --2- 172.21.15.172:0/1756609462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9aeb40 0x56251f9b4890 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x56251f9b6580 tx=0x7fb85401a000 comp rx=0 tx=0).stop 2024-09-10T08:53:31.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.055+0000 7fb8687f4740 1 -- 172.21.15.172:0/1756609462 shutdown_connections 2024-09-10T08:53:31.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.055+0000 7fb8687f4740 1 --2- 172.21.15.172:0/1756609462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9b8370 0x56251f9b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.055+0000 7fb8687f4740 1 --2- 172.21.15.172:0/1756609462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9aeb40 0x56251f9b4890 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.055+0000 7fb8687f4740 1 --2- 172.21.15.172:0/1756609462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9ad5d0 0x56251f9c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.055+0000 7fb8687f4740 1 -- 172.21.15.172:0/1756609462 >> 172.21.15.172:0/1756609462 conn(0x56251f976310 msgr2=0x56251f9aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:31.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.056+0000 7fb8687f4740 1 -- 172.21.15.172:0/1756609462 shutdown_connections 2024-09-10T08:53:31.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.056+0000 7fb8687f4740 1 -- 172.21.15.172:0/1756609462 wait complete. 2024-09-10T08:53:31.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.056+0000 7fb8687f4740 1 -- start start 2024-09-10T08:53:31.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.056+0000 7fb8687f4740 1 Processor -- start 2024-09-10T08:53:31.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.056+0000 7fb8687f4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9ad5d0 0x56251f9c0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.056+0000 7fb8687f4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9aeb40 0x56251f9bbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.056+0000 7fb8687f4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9b8370 0x56251f9bc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.056+0000 7fb8687f4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601affd0 con 0x56251f9b8370 2024-09-10T08:53:31.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.056+0000 7fb8687f4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601af900 con 0x56251f9ad5d0 2024-09-10T08:53:31.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.056+0000 7fb8687f4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601acb40 con 0x56251f9aeb40 2024-09-10T08:53:31.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb8687f4740 10 client.0 _ll_register_callbacks cb 0x7fb854012f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:31.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb8649b8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9b8370 0x56251f9bc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:31.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9ad5d0 0x56251f9c0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:31.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb85f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9aeb40 0x56251f9bbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:31.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb8649b8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9b8370 0x56251f9bc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40876/0 (socket says 172.21.15.172:40876) 2024-09-10T08:53:31.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb8649b8640 1 -- 172.21.15.172:0/4186091508 learned_addr learned my addr 172.21.15.172:0/4186091508 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:31.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb8649b8640 1 -- 172.21.15.172:0/4186091508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9aeb40 msgr2=0x56251f9bbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:31.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb8649b8640 1 --2- 172.21.15.172:0/4186091508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9aeb40 0x56251f9bbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb8649b8640 1 -- 172.21.15.172:0/4186091508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9ad5d0 msgr2=0x56251f9c0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:31.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb8649b8640 1 --2- 172.21.15.172:0/4186091508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9ad5d0 0x56251f9c0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb8649b8640 1 -- 172.21.15.172:0/4186091508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56251f9bc780 con 0x56251f9b8370 2024-09-10T08:53:31.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb85ffff640 1 --2- 172.21.15.172:0/4186091508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9ad5d0 0x56251f9c0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:31.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb85f7fe640 1 --2- 172.21.15.172:0/4186091508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9aeb40 0x56251f9bbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:31.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb8649b8640 1 --2- 172.21.15.172:0/4186091508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9b8370 0x56251f9bc1c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb8580edab0 tx=0x7fb8580bc410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dba48accb0adf56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:31.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb83f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:31.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb83f7fe640 1 -- 172.21.15.172:0/4186091508 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8580da060 con 0x56251f9b8370 2024-09-10T08:53:31.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb83f7fe640 1 -- 172.21.15.172:0/4186091508 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb8580bca70 con 0x56251f9b8370 2024-09-10T08:53:31.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.057+0000 7fb8687f4740 1 -- 172.21.15.172:0/4186091508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb85401fdc0 con 0x56251f9b8370 2024-09-10T08:53:31.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.058+0000 7fb83f7fe640 1 -- 172.21.15.172:0/4186091508 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb85802c040 con 0x56251f9b8370 2024-09-10T08:53:31.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.058+0000 7fb8687f4740 10 client.17595 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:31.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.058+0000 7fb8687f4740 1 -- 172.21.15.172:0/4186091508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56251fa325f0 con 0x56251f9b8370 2024-09-10T08:53:31.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.058+0000 7fb83f7fe640 1 -- 172.21.15.172:0/4186091508 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb85401fdc0 con 0x56251f9b8370 2024-09-10T08:53:31.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.058+0000 7fb83f7fe640 1 -- 172.21.15.172:0/4186091508 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fb8580c4ff0 con 0x56251f9b8370 2024-09-10T08:53:31.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.058+0000 7fb8687f4740 10 client.17595 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:31.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.058+0000 7fb8687f4740 10 client.17595 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:53:31.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.058+0000 7fb8687f4740 20 client.17595 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:31.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.058+0000 7fb8687f4740 1 --2- 172.21.15.172:0/4186091508 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x56251f9c7ec0 0x56251fa34d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:31.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.058+0000 7fb8687f4740 4 client.17595 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:53:31.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.058+0000 7fb8687f4740 1 -- 172.21.15.172:0/4186091508 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56251f91ad70 con 0x56251f9c7ec0 2024-09-10T08:53:31.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.058+0000 7fb85ffff640 1 --2- 172.21.15.172:0/4186091508 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x56251f9c7ec0 0x56251fa34d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:31.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.059+0000 7fb85ffff640 1 --2- 172.21.15.172:0/4186091508 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x56251f9c7ec0 0x56251fa34d00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=800ca712fd2911bc server_cookie=5dede8316497d9c1 in_seq=0 out_seq=0 2024-09-10T08:53:31.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.059+0000 7fb83f7fe640 10 client.17595 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:53:31.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.059+0000 7fb83f7fe640 1 -- 172.21.15.172:0/4186091508 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56251f91ad70 con 0x56251f9c7ec0 2024-09-10T08:53:31.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.059+0000 7fb83f7fe640 10 client.17595 handle_command_reply: tid=0 2024-09-10T08:53:31.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.059+0000 7fb8687f4740 1 client.17595 shutdown 2024-09-10T08:53:31.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.059+0000 7fb8687f4740 1 -- 172.21.15.172:0/4186091508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9b8370 msgr2=0x56251f9bc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:31.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.059+0000 7fb8687f4740 1 --2- 172.21.15.172:0/4186091508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9b8370 0x56251f9bc1c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb8580edab0 tx=0x7fb8580bc410 comp rx=0 tx=0).stop 2024-09-10T08:53:31.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.059+0000 7fb8687f4740 1 -- 172.21.15.172:0/4186091508 shutdown_connections 2024-09-10T08:53:31.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.059+0000 7fb8687f4740 1 --2- 172.21.15.172:0/4186091508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56251f9b8370 0x56251f9bc1c0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.059+0000 7fb8687f4740 1 --2- 172.21.15.172:0/4186091508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56251f9aeb40 0x56251f9bbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.059+0000 7fb8687f4740 1 --2- 172.21.15.172:0/4186091508 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x56251f9c7ec0 0x56251fa34d00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.059+0000 7fb8687f4740 1 --2- 172.21.15.172:0/4186091508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56251f9ad5d0 0x56251f9c0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.059+0000 7fb83f7fe640 0 client.17595 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:53:31.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.059+0000 7fb8687f4740 1 -- 172.21.15.172:0/4186091508 >> 172.21.15.172:0/4186091508 conn(0x56251f976310 msgr2=0x56251f9a84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:31.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.059+0000 7fb8687f4740 1 -- 172.21.15.172:0/4186091508 shutdown_connections 2024-09-10T08:53:31.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.060+0000 7fb8687f4740 1 -- 172.21.15.172:0/4186091508 wait complete. 2024-09-10T08:53:31.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.060+0000 7fb8687f4740 20 client.17595 trim_cache size 0 max 16384 2024-09-10T08:53:31.073 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:53:31.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.060+0000 7fb86641b640 1 -- 172.21.15.172:0/2420753817 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8480a2b80 msgr2=0x7fb8480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:31.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.060+0000 7fb86641b640 1 --2- 172.21.15.172:0/2420753817 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8480a2b80 0x7fb8480a5040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb8600feda0 tx=0x7fb8540127c0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.060+0000 7fb86641b640 1 -- 172.21.15.172:0/2420753817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86010c150 msgr2=0x7fb86019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:31.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.060+0000 7fb86641b640 1 --2- 172.21.15.172:0/2420753817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86010c150 0x7fb86019cce0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fb858009dc0 tx=0x7fb858003f10 comp rx=0 tx=0).stop 2024-09-10T08:53:31.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.061+0000 7fb86641b640 1 -- 172.21.15.172:0/2420753817 shutdown_connections 2024-09-10T08:53:31.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.061+0000 7fb86641b640 1 --2- 172.21.15.172:0/2420753817 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8480a2b80 0x7fb8480a5040 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.061+0000 7fb86641b640 1 --2- 172.21.15.172:0/2420753817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb86010c150 0x7fb86019cce0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.061+0000 7fb86641b640 1 --2- 172.21.15.172:0/2420753817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb860103640 0x7fb860198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.061+0000 7fb86641b640 1 --2- 172.21.15.172:0/2420753817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb860102d20 0x7fb860198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:31.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.061+0000 7fb86641b640 1 -- 172.21.15.172:0/2420753817 >> 172.21.15.172:0/2420753817 conn(0x7fb8600fc780 msgr2=0x7fb8600fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:31.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.061+0000 7fb86641b640 1 -- 172.21.15.172:0/2420753817 shutdown_connections 2024-09-10T08:53:31.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:31.061+0000 7fb86641b640 1 -- 172.21.15.172:0/2420753817 wait complete. 2024-09-10T08:53:31.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:31 smithi174 ceph-mon[34068]: pgmap v569: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.9 MiB/s wr, 15 op/s 2024-09-10T08:53:31.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:31 smithi191 ceph-mon[35314]: pgmap v569: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.9 MiB/s wr, 15 op/s 2024-09-10T08:53:31.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:31 smithi172 ceph-mon[30854]: pgmap v569: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.9 MiB/s wr, 15 op/s 2024-09-10T08:53:32.007 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T08:53:32.288 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:53:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.441+0000 7fe02554b640 1 -- 172.21.15.172:0/2975093034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe020101c30 msgr2=0x7fe0201020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.441+0000 7fe02554b640 1 --2- 172.21.15.172:0/2975093034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe020101c30 0x7fe0201020b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe014009900 tx=0x7fe01402a410 comp rx=0 tx=0).stop 2024-09-10T08:53:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.441+0000 7fe02554b640 1 -- 172.21.15.172:0/2975093034 shutdown_connections 2024-09-10T08:53:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.441+0000 7fe02554b640 1 --2- 172.21.15.172:0/2975093034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0201025f0 0x7fe020108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.441+0000 7fe02554b640 1 --2- 172.21.15.172:0/2975093034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe020101c30 0x7fe0201020b0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.441+0000 7fe02554b640 1 --2- 172.21.15.172:0/2975093034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe020100a40 0x7fe020100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.441+0000 7fe02554b640 1 -- 172.21.15.172:0/2975093034 >> 172.21.15.172:0/2975093034 conn(0x7fe0200fc1e0 msgr2=0x7fe0200fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.441+0000 7fe02554b640 1 -- 172.21.15.172:0/2975093034 shutdown_connections 2024-09-10T08:53:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.441+0000 7fe02554b640 1 -- 172.21.15.172:0/2975093034 wait complete. 2024-09-10T08:53:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.442+0000 7fe02554b640 1 Processor -- start 2024-09-10T08:53:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.442+0000 7fe02554b640 1 -- start start 2024-09-10T08:53:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.442+0000 7fe02554b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe020100a40 0x7fe020199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.442+0000 7fe02554b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe020101c30 0x7fe02019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.442+0000 7fe02554b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0201025f0 0x7fe0201a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.442+0000 7fe02554b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe02010b4f0 con 0x7fe020101c30 2024-09-10T08:53:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.442+0000 7fe02554b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe02010b370 con 0x7fe020100a40 2024-09-10T08:53:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.442+0000 7fe02554b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe02010b670 con 0x7fe0201025f0 2024-09-10T08:53:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.442+0000 7fe01e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe020101c30 0x7fe02019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.442+0000 7fe01e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe020101c30 0x7fe02019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40888/0 (socket says 172.21.15.172:40888) 2024-09-10T08:53:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.442+0000 7fe01e7fc640 1 -- 172.21.15.172:0/3206939682 learned_addr learned my addr 172.21.15.172:0/3206939682 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.442+0000 7fe01effd640 1 --2- 172.21.15.172:0/3206939682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe020100a40 0x7fe020199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.442+0000 7fe01f7fe640 1 --2- 172.21.15.172:0/3206939682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0201025f0 0x7fe0201a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.442+0000 7fe01e7fc640 1 -- 172.21.15.172:0/3206939682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0201025f0 msgr2=0x7fe0201a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.442+0000 7fe01e7fc640 1 --2- 172.21.15.172:0/3206939682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0201025f0 0x7fe0201a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.443+0000 7fe01e7fc640 1 -- 172.21.15.172:0/3206939682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe020100a40 msgr2=0x7fe020199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.443+0000 7fe01e7fc640 1 --2- 172.21.15.172:0/3206939682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe020100a40 0x7fe020199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.443+0000 7fe01e7fc640 1 -- 172.21.15.172:0/3206939682 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0201a1b10 con 0x7fe020101c30 2024-09-10T08:53:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.443+0000 7fe01e7fc640 1 --2- 172.21.15.172:0/3206939682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe020101c30 0x7fe02019a430 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fe0140314e0 tx=0x7fe014003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.443+0000 7fe007fff640 1 -- 172.21.15.172:0/3206939682 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01403f070 con 0x7fe020101c30 2024-09-10T08:53:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.443+0000 7fe007fff640 1 -- 172.21.15.172:0/3206939682 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe014031e40 con 0x7fe020101c30 2024-09-10T08:53:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.443+0000 7fe02554b640 1 -- 172.21.15.172:0/3206939682 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0201a1da0 con 0x7fe020101c30 2024-09-10T08:53:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.443+0000 7fe007fff640 1 -- 172.21.15.172:0/3206939682 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014035030 con 0x7fe020101c30 2024-09-10T08:53:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.443+0000 7fe02554b640 1 -- 172.21.15.172:0/3206939682 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0201a2280 con 0x7fe020101c30 2024-09-10T08:53:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.445+0000 7fe007fff640 1 -- 172.21.15.172:0/3206939682 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe014042070 con 0x7fe020101c30 2024-09-10T08:53:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.446+0000 7fe027f22740 1 Processor -- start 2024-09-10T08:53:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.446+0000 7fe007fff640 1 --2- 172.21.15.172:0/3206939682 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0080a2cb0 0x7fe0080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.446+0000 7fe027f22740 1 -- start start 2024-09-10T08:53:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.446+0000 7fe007fff640 1 -- 172.21.15.172:0/3206939682 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe0140efc70 con 0x7fe020101c30 2024-09-10T08:53:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.446+0000 7fe027f22740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f269a0 0x560157f26da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.446+0000 7fe01f7fe640 1 --2- 172.21.15.172:0/3206939682 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0080a2cb0 0x7fe0080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.446+0000 7fe027f22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f272e0 0x560157f2bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.446+0000 7fe027f22740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f2c370 0x560157f2e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.446+0000 7fe027f22740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560157e8a2f0 con 0x560157f269a0 2024-09-10T08:53:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.446+0000 7fe027f22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560157e32270 con 0x560157f272e0 2024-09-10T08:53:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.446+0000 7fe027f22740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560157e862b0 con 0x560157f2c370 2024-09-10T08:53:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.446+0000 7fe01effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f269a0 0x560157f26da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.447+0000 7fe01effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f269a0 0x560157f26da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40896/0 (socket says 172.21.15.172:40896) 2024-09-10T08:53:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.447+0000 7fe01effd640 1 -- 172.21.15.172:0/1414710059 learned_addr learned my addr 172.21.15.172:0/1414710059 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.447+0000 7fe01f7fe640 1 --2- 172.21.15.172:0/1414710059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f2c370 0x560157f2e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.447+0000 7fe01e7fc640 1 --2- 172.21.15.172:0/1414710059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f272e0 0x560157f2bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.447+0000 7fe01effd640 1 -- 172.21.15.172:0/1414710059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f2c370 msgr2=0x560157f2e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.447+0000 7fe01effd640 1 --2- 172.21.15.172:0/1414710059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f2c370 0x560157f2e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.447+0000 7fe01effd640 1 -- 172.21.15.172:0/1414710059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f272e0 msgr2=0x560157f2bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.447+0000 7fe01effd640 1 --2- 172.21.15.172:0/1414710059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f272e0 0x560157f2bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.453 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.447+0000 7fe01effd640 1 -- 172.21.15.172:0/1414710059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560157e31340 con 0x560157f269a0 2024-09-10T08:53:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.447+0000 7fe01f7fe640 1 --2- 172.21.15.172:0/3206939682 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0080a2cb0 0x7fe0080a5170 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe01800bfd0 tx=0x7fe01800b870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.447+0000 7fe01effd640 1 --2- 172.21.15.172:0/1414710059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f269a0 0x560157f26da0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fe01000bcb0 tx=0x7fe0100107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0ad961dc61b573 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.447+0000 7fe0057fa640 1 -- 172.21.15.172:0/1414710059 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010005020 con 0x560157f269a0 2024-09-10T08:53:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.447+0000 7fe0057fa640 1 -- 172.21.15.172:0/1414710059 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0100047a0 con 0x560157f269a0 2024-09-10T08:53:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.447+0000 7fe0057fa640 1 -- 172.21.15.172:0/1414710059 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0100116a0 con 0x560157f269a0 2024-09-10T08:53:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.448+0000 7fe027f22740 1 -- 172.21.15.172:0/1414710059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f269a0 msgr2=0x560157f26da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.448+0000 7fe027f22740 1 --2- 172.21.15.172:0/1414710059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f269a0 0x560157f26da0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fe01000bcb0 tx=0x7fe0100107a0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.448+0000 7fe027f22740 1 -- 172.21.15.172:0/1414710059 shutdown_connections 2024-09-10T08:53:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.448+0000 7fe027f22740 1 --2- 172.21.15.172:0/1414710059 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f2c370 0x560157f2e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.448+0000 7fe027f22740 1 --2- 172.21.15.172:0/1414710059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f272e0 0x560157f2bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.448+0000 7fe027f22740 1 --2- 172.21.15.172:0/1414710059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f269a0 0x560157f26da0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.448+0000 7fe027f22740 1 -- 172.21.15.172:0/1414710059 >> 172.21.15.172:0/1414710059 conn(0x560157f22010 msgr2=0x560157f22410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.448+0000 7fe027f22740 1 -- 172.21.15.172:0/1414710059 shutdown_connections 2024-09-10T08:53:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.448+0000 7fe027f22740 1 -- 172.21.15.172:0/1414710059 wait complete. 2024-09-10T08:53:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe027f22740 1 -- start start 2024-09-10T08:53:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe027f22740 1 Processor -- start 2024-09-10T08:53:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe027f22740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f269a0 0x560157f30b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe027f22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f272e0 0x560157f310d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe027f22740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f2c370 0x560157f38110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe027f22740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560157f2f340 con 0x560157f2c370 2024-09-10T08:53:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe027f22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560157e13290 con 0x560157f272e0 2024-09-10T08:53:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe027f22740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560157f2f4c0 con 0x560157f269a0 2024-09-10T08:53:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe027f22740 10 client.0 _ll_register_callbacks cb 0x560157e44140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe01f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f2c370 0x560157f38110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe01f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f2c370 0x560157f38110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40898/0 (socket says 172.21.15.172:40898) 2024-09-10T08:53:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe01f7fe640 1 -- 172.21.15.172:0/354813390 learned_addr learned my addr 172.21.15.172:0/354813390 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe01effd640 1 --2- 172.21.15.172:0/354813390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f269a0 0x560157f30b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe01e7fc640 1 --2- 172.21.15.172:0/354813390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f272e0 0x560157f310d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe01f7fe640 1 -- 172.21.15.172:0/354813390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f269a0 msgr2=0x560157f30b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe01f7fe640 1 --2- 172.21.15.172:0/354813390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f269a0 0x560157f30b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe01f7fe640 1 -- 172.21.15.172:0/354813390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f272e0 msgr2=0x560157f310d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe01f7fe640 1 --2- 172.21.15.172:0/354813390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f272e0 0x560157f310d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.449+0000 7fe01f7fe640 1 -- 172.21.15.172:0/354813390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560157b9d210 con 0x560157f2c370 2024-09-10T08:53:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.450+0000 7fe01f7fe640 1 --2- 172.21.15.172:0/354813390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f2c370 0x560157f38110 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fe018008fe0 tx=0x7fe01800b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7716a09f80bdcf01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.450+0000 7fe002ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.450+0000 7fe002ff5640 1 -- 172.21.15.172:0/354813390 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01802b060 con 0x560157f2c370 2024-09-10T08:53:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.450+0000 7fe002ff5640 1 -- 172.21.15.172:0/354813390 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe01801c460 con 0x560157f2c370 2024-09-10T08:53:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.450+0000 7fe027f22740 1 -- 172.21.15.172:0/354813390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0100047a0 con 0x560157f2c370 2024-09-10T08:53:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.450+0000 7fe002ff5640 1 -- 172.21.15.172:0/354813390 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe018032050 con 0x560157f2c370 2024-09-10T08:53:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.450+0000 7fe027f22740 10 client.17607 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.450+0000 7fe027f22740 1 -- 172.21.15.172:0/354813390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560157c69620 con 0x560157f2c370 2024-09-10T08:53:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.450+0000 7fe002ff5640 1 -- 172.21.15.172:0/354813390 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0100047a0 con 0x560157f2c370 2024-09-10T08:53:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.450+0000 7fe002ff5640 1 -- 172.21.15.172:0/354813390 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fe01801a040 con 0x560157f2c370 2024-09-10T08:53:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.450+0000 7fe027f22740 10 client.17607 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.450+0000 7fe027f22740 10 client.17607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.450+0000 7fe027f22740 20 client.17607 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.451+0000 7fe027f22740 1 --2- 172.21.15.172:0/354813390 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560157f3c800 0x560157f3ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.451+0000 7fe027f22740 4 client.17607 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:53:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.451+0000 7fe027f22740 1 -- 172.21.15.172:0/354813390 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560157e97a20 con 0x560157f3c800 2024-09-10T08:53:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.451+0000 7fe01effd640 1 --2- 172.21.15.172:0/354813390 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560157f3c800 0x560157f3ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.451+0000 7fe01effd640 1 --2- 172.21.15.172:0/354813390 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560157f3c800 0x560157f3ebf0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e40cafa3e98f2bd1 server_cookie=af5b4837665a079b in_seq=0 out_seq=0 2024-09-10T08:53:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.451+0000 7fe002ff5640 10 client.17607 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.452+0000 7fe002ff5640 1 -- 172.21.15.172:0/354813390 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560157e97a20 con 0x560157f3c800 2024-09-10T08:53:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.452+0000 7fe002ff5640 10 client.17607 handle_command_reply: tid=0 2024-09-10T08:53:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.452+0000 7fe027f22740 1 client.17607 shutdown 2024-09-10T08:53:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.452+0000 7fe027f22740 1 -- 172.21.15.172:0/354813390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f2c370 msgr2=0x560157f38110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.452+0000 7fe027f22740 1 --2- 172.21.15.172:0/354813390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f2c370 0x560157f38110 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fe018008fe0 tx=0x7fe01800b2c0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.452+0000 7fe027f22740 1 -- 172.21.15.172:0/354813390 shutdown_connections 2024-09-10T08:53:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.452+0000 7fe027f22740 1 --2- 172.21.15.172:0/354813390 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560157f3c800 0x560157f3ebf0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.452+0000 7fe002ff5640 0 client.17607 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.452+0000 7fe027f22740 1 --2- 172.21.15.172:0/354813390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f2c370 0x560157f38110 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.452+0000 7fe027f22740 1 --2- 172.21.15.172:0/354813390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f272e0 0x560157f310d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.452+0000 7fe027f22740 1 --2- 172.21.15.172:0/354813390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f269a0 0x560157f30b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.452+0000 7fe027f22740 1 -- 172.21.15.172:0/354813390 >> 172.21.15.172:0/354813390 conn(0x560157f22010 msgr2=0x560157f23820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.452+0000 7fe027f22740 1 -- 172.21.15.172:0/354813390 shutdown_connections 2024-09-10T08:53:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.452+0000 7fe027f22740 1 -- 172.21.15.172:0/354813390 wait complete. 2024-09-10T08:53:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.453+0000 7fe027f22740 20 client.17607 trim_cache size 0 max 16384 2024-09-10T08:53:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.472+0000 7fe027f22740 1 Processor -- start 2024-09-10T08:53:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.472+0000 7fe027f22740 1 -- start start 2024-09-10T08:53:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.472+0000 7fe027f22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f269a0 0x560157f22ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.472+0000 7fe027f22740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f272e0 0x560157f380d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.472+0000 7fe027f22740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f2c370 0x560157f23400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.472+0000 7fe027f22740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe0201ac6f0 con 0x560157f272e0 2024-09-10T08:53:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.472+0000 7fe027f22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe0201ae230 con 0x560157f269a0 2024-09-10T08:53:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.472+0000 7fe027f22740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe0201acdc0 con 0x560157f2c370 2024-09-10T08:53:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.472+0000 7fe01e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f272e0 0x560157f380d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.472+0000 7fe01e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f272e0 0x560157f380d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40904/0 (socket says 172.21.15.172:40904) 2024-09-10T08:53:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.472+0000 7fe01e7fc640 1 -- 172.21.15.172:0/2821812324 learned_addr learned my addr 172.21.15.172:0/2821812324 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.472+0000 7fe01e7fc640 1 -- 172.21.15.172:0/2821812324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f2c370 msgr2=0x560157f23400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.472+0000 7fe01e7fc640 1 --2- 172.21.15.172:0/2821812324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f2c370 0x560157f23400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.472+0000 7fe01e7fc640 1 -- 172.21.15.172:0/2821812324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f269a0 msgr2=0x560157f22ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.472+0000 7fe01e7fc640 1 --2- 172.21.15.172:0/2821812324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f269a0 0x560157f22ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.473+0000 7fe01e7fc640 1 -- 172.21.15.172:0/2821812324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560157f1d2d0 con 0x560157f272e0 2024-09-10T08:53:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.473+0000 7fe01e7fc640 1 --2- 172.21.15.172:0/2821812324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f272e0 0x560157f380d0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fe0140ed410 tx=0x7fe014009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e197968dcb41e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.473+0000 7fe004ff9640 1 -- 172.21.15.172:0/2821812324 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014049040 con 0x560157f272e0 2024-09-10T08:53:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.473+0000 7fe004ff9640 1 -- 172.21.15.172:0/2821812324 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0140c0520 con 0x560157f272e0 2024-09-10T08:53:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.473+0000 7fe004ff9640 1 -- 172.21.15.172:0/2821812324 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0140dc0b0 con 0x560157f272e0 2024-09-10T08:53:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 1 -- 172.21.15.172:0/2821812324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f272e0 msgr2=0x560157f380d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.478 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 1 --2- 172.21.15.172:0/2821812324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f272e0 0x560157f380d0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fe0140ed410 tx=0x7fe014009260 comp rx=0 tx=0).stop 2024-09-10T08:53:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 1 -- 172.21.15.172:0/2821812324 shutdown_connections 2024-09-10T08:53:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 1 --2- 172.21.15.172:0/2821812324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f2c370 0x560157f23400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 1 --2- 172.21.15.172:0/2821812324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f272e0 0x560157f380d0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 1 --2- 172.21.15.172:0/2821812324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f269a0 0x560157f22ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 1 -- 172.21.15.172:0/2821812324 >> 172.21.15.172:0/2821812324 conn(0x560157f22010 msgr2=0x560157f28450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 1 -- 172.21.15.172:0/2821812324 shutdown_connections 2024-09-10T08:53:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 1 -- 172.21.15.172:0/2821812324 wait complete. 2024-09-10T08:53:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 1 -- start start 2024-09-10T08:53:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 1 Processor -- start 2024-09-10T08:53:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f269a0 0x560157f35bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f272e0 0x560157f36130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f2c370 0x560157f370f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe0201adb60 con 0x560157f269a0 2024-09-10T08:53:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe0201ad490 con 0x560157f2c370 2024-09-10T08:53:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe0201aa6d0 con 0x560157f272e0 2024-09-10T08:53:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe027f22740 10 client.0 _ll_register_callbacks cb 0x7fe01800a310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe01effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f269a0 0x560157f35bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.474+0000 7fe01effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f269a0 0x560157f35bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40918/0 (socket says 172.21.15.172:40918) 2024-09-10T08:53:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.475+0000 7fe01effd640 1 -- 172.21.15.172:0/2111233106 learned_addr learned my addr 172.21.15.172:0/2111233106 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.475+0000 7fe01e7fc640 1 --2- 172.21.15.172:0/2111233106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f272e0 0x560157f36130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.475+0000 7fe01f7fe640 1 --2- 172.21.15.172:0/2111233106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f2c370 0x560157f370f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.475+0000 7fe01effd640 1 -- 172.21.15.172:0/2111233106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f272e0 msgr2=0x560157f36130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.475+0000 7fe01effd640 1 --2- 172.21.15.172:0/2111233106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f272e0 0x560157f36130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.475+0000 7fe01effd640 1 -- 172.21.15.172:0/2111233106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f2c370 msgr2=0x560157f370f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.475+0000 7fe01effd640 1 --2- 172.21.15.172:0/2111233106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f2c370 0x560157f370f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.475+0000 7fe01effd640 1 -- 172.21.15.172:0/2111233106 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560157f37700 con 0x560157f269a0 2024-09-10T08:53:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.475+0000 7fe01effd640 1 --2- 172.21.15.172:0/2111233106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f269a0 0x560157f35bf0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fe010010ca0 tx=0x7fe01000b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836422d338c65c4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.475+0000 7fe002ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.475+0000 7fe002ff5640 1 -- 172.21.15.172:0/2111233106 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01000bea0 con 0x560157f269a0 2024-09-10T08:53:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.475+0000 7fe002ff5640 1 -- 172.21.15.172:0/2111233106 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe010004860 con 0x560157f269a0 2024-09-10T08:53:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.475+0000 7fe027f22740 1 -- 172.21.15.172:0/2111233106 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0140c0520 con 0x560157f269a0 2024-09-10T08:53:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.475+0000 7fe002ff5640 1 -- 172.21.15.172:0/2111233106 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe01003a050 con 0x560157f269a0 2024-09-10T08:53:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.475+0000 7fe027f22740 10 client.17613 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.475+0000 7fe027f22740 1 -- 172.21.15.172:0/2111233106 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560157fa7910 con 0x560157f269a0 2024-09-10T08:53:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.475+0000 7fe002ff5640 1 -- 172.21.15.172:0/2111233106 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0140c0520 con 0x560157f269a0 2024-09-10T08:53:32.481 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.476+0000 7fe002ff5640 1 -- 172.21.15.172:0/2111233106 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fe010031470 con 0x560157f269a0 2024-09-10T08:53:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.476+0000 7fe027f22740 10 client.17613 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.476+0000 7fe027f22740 10 client.17613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.476+0000 7fe027f22740 20 client.17613 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.476+0000 7fe027f22740 1 --2- 172.21.15.172:0/2111233106 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560157f3c800 0x560157fa9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.476+0000 7fe027f22740 4 client.17613 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:53:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.476+0000 7fe027f22740 1 -- 172.21.15.172:0/2111233106 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560157f2ec40 con 0x560157f3c800 2024-09-10T08:53:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.476+0000 7fe01e7fc640 1 --2- 172.21.15.172:0/2111233106 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560157f3c800 0x560157fa9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.476+0000 7fe01e7fc640 1 --2- 172.21.15.172:0/2111233106 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560157f3c800 0x560157fa9ff0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f227603825b3f48 server_cookie=cbcd5168fa05a4e4 in_seq=0 out_seq=0 2024-09-10T08:53:32.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.476+0000 7fe002ff5640 10 client.17613 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.480+0000 7fe002ff5640 1 -- 172.21.15.172:0/2111233106 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560157f2ec40 con 0x560157f3c800 2024-09-10T08:53:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.480+0000 7fe002ff5640 10 client.17613 handle_command_reply: tid=0 2024-09-10T08:53:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.480+0000 7fe027f22740 1 client.17613 shutdown 2024-09-10T08:53:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.480+0000 7fe027f22740 1 -- 172.21.15.172:0/2111233106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f269a0 msgr2=0x560157f35bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.480+0000 7fe027f22740 1 --2- 172.21.15.172:0/2111233106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f269a0 0x560157f35bf0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fe010010ca0 tx=0x7fe01000b930 comp rx=0 tx=0).stop 2024-09-10T08:53:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.480+0000 7fe027f22740 1 -- 172.21.15.172:0/2111233106 shutdown_connections 2024-09-10T08:53:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.480+0000 7fe027f22740 1 --2- 172.21.15.172:0/2111233106 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560157f3c800 0x560157fa9ff0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.480+0000 7fe027f22740 1 --2- 172.21.15.172:0/2111233106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560157f2c370 0x560157f370f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.480+0000 7fe027f22740 1 --2- 172.21.15.172:0/2111233106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560157f272e0 0x560157f36130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.480+0000 7fe002ff5640 0 client.17613 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.480+0000 7fe027f22740 1 --2- 172.21.15.172:0/2111233106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560157f269a0 0x560157f35bf0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.480+0000 7fe027f22740 1 -- 172.21.15.172:0/2111233106 >> 172.21.15.172:0/2111233106 conn(0x560157f22010 msgr2=0x560157f209a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.480+0000 7fe027f22740 1 -- 172.21.15.172:0/2111233106 shutdown_connections 2024-09-10T08:53:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.480+0000 7fe027f22740 1 -- 172.21.15.172:0/2111233106 wait complete. 2024-09-10T08:53:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.480+0000 7fe027f22740 20 client.17613 trim_cache size 0 max 16384 2024-09-10T08:53:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:53:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:53:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:53:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:53:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.481+0000 7fe02554b640 1 -- 172.21.15.172:0/3206939682 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0080a2cb0 msgr2=0x7fe0080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.481+0000 7fe02554b640 1 --2- 172.21.15.172:0/3206939682 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0080a2cb0 0x7fe0080a5170 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe01800bfd0 tx=0x7fe01800b870 comp rx=0 tx=0).stop 2024-09-10T08:53:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.481+0000 7fe02554b640 1 -- 172.21.15.172:0/3206939682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe020101c30 msgr2=0x7fe02019a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.481+0000 7fe02554b640 1 --2- 172.21.15.172:0/3206939682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe020101c30 0x7fe02019a430 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fe0140314e0 tx=0x7fe014003c80 comp rx=0 tx=0).stop 2024-09-10T08:53:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.481+0000 7fe02554b640 1 -- 172.21.15.172:0/3206939682 shutdown_connections 2024-09-10T08:53:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.481+0000 7fe02554b640 1 --2- 172.21.15.172:0/3206939682 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0080a2cb0 0x7fe0080a5170 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.481+0000 7fe02554b640 1 --2- 172.21.15.172:0/3206939682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0201025f0 0x7fe0201a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.481+0000 7fe02554b640 1 --2- 172.21.15.172:0/3206939682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe020101c30 0x7fe02019a430 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.481+0000 7fe02554b640 1 --2- 172.21.15.172:0/3206939682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe020100a40 0x7fe020199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.481+0000 7fe02554b640 1 -- 172.21.15.172:0/3206939682 >> 172.21.15.172:0/3206939682 conn(0x7fe0200fc1e0 msgr2=0x7fe020190b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.481+0000 7fe02554b640 1 -- 172.21.15.172:0/3206939682 shutdown_connections 2024-09-10T08:53:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:32.481+0000 7fe02554b640 1 -- 172.21.15.172:0/3206939682 wait complete. 2024-09-10T08:53:32.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.805+0000 7f9db216b640 1 -- 172.21.15.172:0/2457726596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac100e20 msgr2=0x7f9dac101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.805+0000 7f9db216b640 1 --2- 172.21.15.172:0/2457726596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac100e20 0x7f9dac101200 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009900 tx=0x7f9d9c02a480 comp rx=0 tx=0).stop 2024-09-10T08:53:32.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.805+0000 7f9db216b640 1 -- 172.21.15.172:0/2457726596 shutdown_connections 2024-09-10T08:53:32.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.805+0000 7f9db216b640 1 --2- 172.21.15.172:0/2457726596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9dac102ef0 0x7f9dac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.805+0000 7f9db216b640 1 --2- 172.21.15.172:0/2457726596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9dac102570 0x7f9dac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.805+0000 7f9db216b640 1 --2- 172.21.15.172:0/2457726596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac100e20 0x7f9dac101200 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.805+0000 7f9db216b640 1 -- 172.21.15.172:0/2457726596 >> 172.21.15.172:0/2457726596 conn(0x7f9dac0fc7d0 msgr2=0x7f9dac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:32.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.805+0000 7f9db216b640 1 -- 172.21.15.172:0/2457726596 shutdown_connections 2024-09-10T08:53:32.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.805+0000 7f9db216b640 1 -- 172.21.15.172:0/2457726596 wait complete. 2024-09-10T08:53:32.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.806+0000 7f9db216b640 1 Processor -- start 2024-09-10T08:53:32.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.806+0000 7f9db216b640 1 -- start start 2024-09-10T08:53:32.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9db216b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9dac100e20 0x7f9dac1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9db216b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9dac102570 0x7f9dac197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9db216b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac102ef0 0x7f9dac19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9db216b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac110e30 con 0x7f9dac102ef0 2024-09-10T08:53:32.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9db216b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac110cb0 con 0x7f9dac102570 2024-09-10T08:53:32.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9db216b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac110fb0 con 0x7f9dac100e20 2024-09-10T08:53:32.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9dabfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac102ef0 0x7f9dac19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9dabfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac102ef0 0x7f9dac19bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40930/0 (socket says 172.21.15.172:40930) 2024-09-10T08:53:32.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9daaffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9dac102570 0x7f9dac197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9dabfff640 1 -- 172.21.15.172:0/2514845732 learned_addr learned my addr 172.21.15.172:0/2514845732 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9dab7fe640 1 --2- 172.21.15.172:0/2514845732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9dac100e20 0x7f9dac1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9daaffd640 1 -- 172.21.15.172:0/2514845732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9dac100e20 msgr2=0x7f9dac1972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9daaffd640 1 --2- 172.21.15.172:0/2514845732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9dac100e20 0x7f9dac1972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9daaffd640 1 -- 172.21.15.172:0/2514845732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac102ef0 msgr2=0x7f9dac19bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9daaffd640 1 --2- 172.21.15.172:0/2514845732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac102ef0 0x7f9dac19bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9daaffd640 1 -- 172.21.15.172:0/2514845732 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dac19c290 con 0x7f9dac102570 2024-09-10T08:53:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9daaffd640 1 --2- 172.21.15.172:0/2514845732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9dac102570 0x7f9dac197820 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9da000e9f0 tx=0x7f9da000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9da8ff9640 1 -- 172.21.15.172:0/2514845732 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da000ccc0 con 0x7f9dac102570 2024-09-10T08:53:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.807+0000 7f9db216b640 1 -- 172.21.15.172:0/2514845732 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dac19c580 con 0x7f9dac102570 2024-09-10T08:53:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.808+0000 7f9da8ff9640 1 -- 172.21.15.172:0/2514845732 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9da00108e0 con 0x7f9dac102570 2024-09-10T08:53:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.808+0000 7f9da8ff9640 1 -- 172.21.15.172:0/2514845732 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0018880 con 0x7f9dac102570 2024-09-10T08:53:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.808+0000 7f9db216b640 1 -- 172.21.15.172:0/2514845732 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dac1a3e60 con 0x7f9dac102570 2024-09-10T08:53:32.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.811+0000 7f9da8ff9640 1 -- 172.21.15.172:0/2514845732 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9da0010a90 con 0x7f9dac102570 2024-09-10T08:53:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.811+0000 7f9db4544740 1 Processor -- start 2024-09-10T08:53:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.811+0000 7f9db4544740 1 -- start start 2024-09-10T08:53:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.811+0000 7f9db4544740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923070 0x55ff64923450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.811+0000 7f9db4544740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64924220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.811+0000 7f9db4544740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64924760 0x55ff6492af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.811+0000 7f9db4544740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff6487f2f0 con 0x55ff64923de0 2024-09-10T08:53:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.811+0000 7f9db4544740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ff64826cf0 con 0x55ff64923070 2024-09-10T08:53:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.811+0000 7f9db4544740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ff6487b2b0 con 0x55ff64924760 2024-09-10T08:53:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.811+0000 7f9dab7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64924220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.811+0000 7f9dab7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64924220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40942/0 (socket says 172.21.15.172:40942) 2024-09-10T08:53:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.811+0000 7f9dab7fe640 1 -- 172.21.15.172:0/3716899070 learned_addr learned my addr 172.21.15.172:0/3716899070 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:32.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.812+0000 7f9dabfff640 1 --2- 172.21.15.172:0/3716899070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923070 0x55ff64923450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.812+0000 7f9daaffd640 1 --2- 172.21.15.172:0/3716899070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64924760 0x55ff6492af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.812+0000 7f9da8ff9640 1 --2- 172.21.15.172:0/2514845732 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9d940a2b80 0x7f9d940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.812+0000 7f9dab7fe640 1 -- 172.21.15.172:0/3716899070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64924760 msgr2=0x55ff6492af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:32.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.812+0000 7f9dab7fe640 1 --2- 172.21.15.172:0/3716899070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64924760 0x55ff6492af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.812+0000 7f9dab7fe640 1 -- 172.21.15.172:0/3716899070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923070 msgr2=0x55ff64923450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:32.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.812+0000 7f9dab7fe640 1 --2- 172.21.15.172:0/3716899070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923070 0x55ff64923450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.812+0000 7f9dab7fe640 1 -- 172.21.15.172:0/3716899070 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff6480b3a0 con 0x55ff64923de0 2024-09-10T08:53:32.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.812+0000 7f9dabfff640 1 --2- 172.21.15.172:0/2514845732 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9d940a2b80 0x7f9d940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.812+0000 7f9da8ff9640 1 -- 172.21.15.172:0/2514845732 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9da00cf6f0 con 0x7f9dac102570 2024-09-10T08:53:32.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.812+0000 7f9dab7fe640 1 --2- 172.21.15.172:0/3716899070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64924220 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009900 tx=0x7f9d9c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96456668f3acc32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:32.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.812+0000 7f9d91ffb640 1 -- 172.21.15.172:0/3716899070 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c040070 con 0x55ff64923de0 2024-09-10T08:53:32.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.812+0000 7f9dabfff640 1 --2- 172.21.15.172:0/2514845732 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9d940a2b80 0x7f9d940a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009970 tx=0x7f9da4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.812+0000 7f9d91ffb640 1 -- 172.21.15.172:0/3716899070 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d9c004030 con 0x55ff64923de0 2024-09-10T08:53:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.812+0000 7f9d91ffb640 1 -- 172.21.15.172:0/3716899070 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c035040 con 0x55ff64923de0 2024-09-10T08:53:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.813+0000 7f9db4544740 1 -- 172.21.15.172:0/3716899070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 msgr2=0x55ff64924220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.813+0000 7f9db4544740 1 --2- 172.21.15.172:0/3716899070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64924220 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c009900 tx=0x7f9d9c005c00 comp rx=0 tx=0).stop 2024-09-10T08:53:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.813+0000 7f9db4544740 1 -- 172.21.15.172:0/3716899070 shutdown_connections 2024-09-10T08:53:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.813+0000 7f9db4544740 1 --2- 172.21.15.172:0/3716899070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64924760 0x55ff6492af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.813+0000 7f9db4544740 1 --2- 172.21.15.172:0/3716899070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64924220 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.813+0000 7f9db4544740 1 --2- 172.21.15.172:0/3716899070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923070 0x55ff64923450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.813+0000 7f9db4544740 1 -- 172.21.15.172:0/3716899070 >> 172.21.15.172:0/3716899070 conn(0x55ff648e2310 msgr2=0x55ff64915720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.813+0000 7f9db4544740 1 -- 172.21.15.172:0/3716899070 shutdown_connections 2024-09-10T08:53:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.813+0000 7f9db4544740 1 -- 172.21.15.172:0/3716899070 wait complete. 2024-09-10T08:53:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.814+0000 7f9db4544740 1 -- start start 2024-09-10T08:53:32.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.814+0000 7f9db4544740 1 Processor -- start 2024-09-10T08:53:32.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.814+0000 7f9db4544740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64923070 0x55ff6492ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.814+0000 7f9db4544740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923de0 0x55ff64928030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.814+0000 7f9db4544740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64924760 0x55ff64928570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.814+0000 7f9db4544740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff6492b4a0 con 0x55ff64924760 2024-09-10T08:53:32.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.814+0000 7f9db4544740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ff6483a060 con 0x55ff64923de0 2024-09-10T08:53:32.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.814+0000 7f9db4544740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ff6492b620 con 0x55ff64923070 2024-09-10T08:53:32.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.814+0000 7f9db4544740 10 client.0 _ll_register_callbacks cb 0x55ff6458d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:32.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.814+0000 7f9dabfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64924760 0x55ff64928570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.814+0000 7f9dabfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64924760 0x55ff64928570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40946/0 (socket says 172.21.15.172:40946) 2024-09-10T08:53:32.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.815+0000 7f9dabfff640 1 -- 172.21.15.172:0/2762749518 learned_addr learned my addr 172.21.15.172:0/2762749518 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:32.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.815+0000 7f9daaffd640 1 --2- 172.21.15.172:0/2762749518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923de0 0x55ff64928030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.815+0000 7f9dab7fe640 1 --2- 172.21.15.172:0/2762749518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64923070 0x55ff6492ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.815+0000 7f9dabfff640 1 -- 172.21.15.172:0/2762749518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64923070 msgr2=0x55ff6492ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:32.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.815+0000 7f9dabfff640 1 --2- 172.21.15.172:0/2762749518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64923070 0x55ff6492ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.815+0000 7f9dabfff640 1 -- 172.21.15.172:0/2762749518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923de0 msgr2=0x55ff64928030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:32.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.815+0000 7f9dabfff640 1 --2- 172.21.15.172:0/2762749518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923de0 0x55ff64928030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.815+0000 7f9dabfff640 1 -- 172.21.15.172:0/2762749518 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff645f03c0 con 0x55ff64924760 2024-09-10T08:53:32.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.815+0000 7f9dabfff640 1 --2- 172.21.15.172:0/2762749518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64924760 0x55ff64928570 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9da401d8f0 tx=0x7f9da4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0fcde01cede175 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:32.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.815+0000 7f9d8affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:32.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.815+0000 7f9d8affd640 1 -- 172.21.15.172:0/2762749518 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da402b060 con 0x55ff64924760 2024-09-10T08:53:32.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.815+0000 7f9d8affd640 1 -- 172.21.15.172:0/2762749518 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9da4018070 con 0x55ff64924760 2024-09-10T08:53:32.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.815+0000 7f9db4544740 1 -- 172.21.15.172:0/2762749518 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d9c004030 con 0x55ff64924760 2024-09-10T08:53:32.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.816+0000 7f9d8affd640 1 -- 172.21.15.172:0/2762749518 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9da4032050 con 0x55ff64924760 2024-09-10T08:53:32.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.816+0000 7f9db4544740 10 client.17625 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.816+0000 7f9db4544740 1 -- 172.21.15.172:0/2762749518 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ff64936e70 con 0x55ff64924760 2024-09-10T08:53:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.816+0000 7f9d8affd640 1 -- 172.21.15.172:0/2762749518 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d9c004030 con 0x55ff64924760 2024-09-10T08:53:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.816+0000 7f9d8affd640 1 -- 172.21.15.172:0/2762749518 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f9da400f040 con 0x55ff64924760 2024-09-10T08:53:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.816+0000 7f9db4544740 10 client.17625 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.816+0000 7f9db4544740 10 client.17625 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:53:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.816+0000 7f9db4544740 20 client.17625 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.816+0000 7f9db4544740 1 --2- 172.21.15.172:0/2762749518 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55ff64938330 0x55ff6493a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.816+0000 7f9db4544740 4 client.17625 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:53:32.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.816+0000 7f9db4544740 1 -- 172.21.15.172:0/2762749518 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff6488c860 con 0x55ff64938330 2024-09-10T08:53:32.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.817+0000 7f9dab7fe640 1 --2- 172.21.15.172:0/2762749518 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55ff64938330 0x55ff6493a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.817+0000 7f9d8affd640 10 client.17625 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:53:32.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.817+0000 7f9dab7fe640 1 --2- 172.21.15.172:0/2762749518 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55ff64938330 0x55ff6493a780 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3c98cacfb3f3d195 server_cookie=32f516ca25853c8 in_seq=0 out_seq=0 2024-09-10T08:53:32.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.818+0000 7f9d8affd640 1 -- 172.21.15.172:0/2762749518 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff6488c860 con 0x55ff64938330 2024-09-10T08:53:32.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.818+0000 7f9d8affd640 10 client.17625 handle_command_reply: tid=0 2024-09-10T08:53:32.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.818+0000 7f9db4544740 1 client.17625 shutdown 2024-09-10T08:53:32.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.818+0000 7f9db4544740 1 -- 172.21.15.172:0/2762749518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64924760 msgr2=0x55ff64928570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.818+0000 7f9db4544740 1 --2- 172.21.15.172:0/2762749518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64924760 0x55ff64928570 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9da401d8f0 tx=0x7f9da4004ab0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.818+0000 7f9db4544740 1 -- 172.21.15.172:0/2762749518 shutdown_connections 2024-09-10T08:53:32.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.818+0000 7f9db4544740 1 --2- 172.21.15.172:0/2762749518 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55ff64938330 0x55ff6493a780 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.818+0000 7f9db4544740 1 --2- 172.21.15.172:0/2762749518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64924760 0x55ff64928570 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.818+0000 7f9db4544740 1 --2- 172.21.15.172:0/2762749518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923de0 0x55ff64928030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.818+0000 7f9db4544740 1 --2- 172.21.15.172:0/2762749518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64923070 0x55ff6492ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.818+0000 7f9db4544740 1 -- 172.21.15.172:0/2762749518 >> 172.21.15.172:0/2762749518 conn(0x55ff648e2310 msgr2=0x55ff64914d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:32.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.818+0000 7f9d8affd640 0 client.17625 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:53:32.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.818+0000 7f9db4544740 1 -- 172.21.15.172:0/2762749518 shutdown_connections 2024-09-10T08:53:32.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.818+0000 7f9db4544740 1 -- 172.21.15.172:0/2762749518 wait complete. 2024-09-10T08:53:32.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.818+0000 7f9db4544740 20 client.17625 trim_cache size 0 max 16384 2024-09-10T08:53:32.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9db4544740 1 Processor -- start 2024-09-10T08:53:32.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9db4544740 1 -- start start 2024-09-10T08:53:32.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9db4544740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923070 0x55ff64934700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9db4544740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64940050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9db4544740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64924760 0x55ff64940590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9db4544740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac1ac8d0 con 0x55ff64923de0 2024-09-10T08:53:32.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9db4544740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac1ae410 con 0x55ff64923070 2024-09-10T08:53:32.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9db4544740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac1acfa0 con 0x55ff64924760 2024-09-10T08:53:32.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9daaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64940050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9daaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64940050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40956/0 (socket says 172.21.15.172:40956) 2024-09-10T08:53:32.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9daaffd640 1 -- 172.21.15.172:0/866134916 learned_addr learned my addr 172.21.15.172:0/866134916 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:32.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9dab7fe640 1 --2- 172.21.15.172:0/866134916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923070 0x55ff64934700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9dabfff640 1 --2- 172.21.15.172:0/866134916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64924760 0x55ff64940590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9daaffd640 1 -- 172.21.15.172:0/866134916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64924760 msgr2=0x55ff64940590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:32.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9daaffd640 1 --2- 172.21.15.172:0/866134916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64924760 0x55ff64940590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9daaffd640 1 -- 172.21.15.172:0/866134916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923070 msgr2=0x55ff64934700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:32.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9daaffd640 1 --2- 172.21.15.172:0/866134916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923070 0x55ff64934700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9daaffd640 1 -- 172.21.15.172:0/866134916 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff64922790 con 0x55ff64923de0 2024-09-10T08:53:32.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.832+0000 7f9dabfff640 1 --2- 172.21.15.172:0/866134916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64924760 0x55ff64940590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:53:32.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.833+0000 7f9daaffd640 1 --2- 172.21.15.172:0/866134916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64940050 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9da009b9a0 tx=0x7f9da0009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9697c1693343c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:32.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.833+0000 7f9d917fa640 1 -- 172.21.15.172:0/866134916 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da00a4860 con 0x55ff64923de0 2024-09-10T08:53:32.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.833+0000 7f9d917fa640 1 -- 172.21.15.172:0/866134916 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9da0099fa0 con 0x55ff64923de0 2024-09-10T08:53:32.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.833+0000 7f9d917fa640 1 -- 172.21.15.172:0/866134916 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da009b0f0 con 0x55ff64923de0 2024-09-10T08:53:32.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.834+0000 7f9db4544740 1 -- 172.21.15.172:0/866134916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 msgr2=0x55ff64940050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.834+0000 7f9db4544740 1 --2- 172.21.15.172:0/866134916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64940050 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9da009b9a0 tx=0x7f9da0009700 comp rx=0 tx=0).stop 2024-09-10T08:53:32.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.834+0000 7f9db4544740 1 -- 172.21.15.172:0/866134916 shutdown_connections 2024-09-10T08:53:32.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.834+0000 7f9db4544740 1 --2- 172.21.15.172:0/866134916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64924760 0x55ff64940590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.834+0000 7f9db4544740 1 --2- 172.21.15.172:0/866134916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64940050 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.834+0000 7f9db4544740 1 --2- 172.21.15.172:0/866134916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923070 0x55ff64934700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.834+0000 7f9db4544740 1 -- 172.21.15.172:0/866134916 >> 172.21.15.172:0/866134916 conn(0x55ff648e2310 msgr2=0x55ff64926900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:32.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.834+0000 7f9db4544740 1 -- 172.21.15.172:0/866134916 shutdown_connections 2024-09-10T08:53:32.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.834+0000 7f9db4544740 1 -- 172.21.15.172:0/866134916 wait complete. 2024-09-10T08:53:32.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.834+0000 7f9db4544740 1 -- start start 2024-09-10T08:53:32.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.834+0000 7f9db4544740 1 Processor -- start 2024-09-10T08:53:32.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9db4544740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923070 0x55ff64930870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9db4544740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64930db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9db4544740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64924760 0x55ff6492d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9db4544740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac1add40 con 0x55ff64923de0 2024-09-10T08:53:32.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9db4544740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac1ad670 con 0x55ff64923070 2024-09-10T08:53:32.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9db4544740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac1aa8b0 con 0x55ff64924760 2024-09-10T08:53:32.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9db4544740 10 client.0 _ll_register_callbacks cb 0x7f9da4009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:32.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9daaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64930db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9daaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64930db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40964/0 (socket says 172.21.15.172:40964) 2024-09-10T08:53:32.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9daaffd640 1 -- 172.21.15.172:0/2054904558 learned_addr learned my addr 172.21.15.172:0/2054904558 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:32.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9dab7fe640 1 --2- 172.21.15.172:0/2054904558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923070 0x55ff64930870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9dabfff640 1 --2- 172.21.15.172:0/2054904558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64924760 0x55ff6492d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9dab7fe640 1 -- 172.21.15.172:0/2054904558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64924760 msgr2=0x55ff6492d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:32.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9dab7fe640 1 --2- 172.21.15.172:0/2054904558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64924760 0x55ff6492d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9dab7fe640 1 -- 172.21.15.172:0/2054904558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 msgr2=0x55ff64930db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:32.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9dab7fe640 1 --2- 172.21.15.172:0/2054904558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64930db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9dab7fe640 1 -- 172.21.15.172:0/2054904558 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff6492dd80 con 0x55ff64923070 2024-09-10T08:53:32.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9daaffd640 1 --2- 172.21.15.172:0/2054904558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64930db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:32.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9dab7fe640 1 --2- 172.21.15.172:0/2054904558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923070 0x55ff64930870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c02a4c0 tx=0x7f9d9c031520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f957b434f4950838 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:32.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.835+0000 7f9d8affd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:53:32.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.836+0000 7f9d8affd640 1 -- 172.21.15.172:0/2054904558 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c040040 con 0x55ff64923070 2024-09-10T08:53:32.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.836+0000 7f9d8affd640 1 -- 172.21.15.172:0/2054904558 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9d9c005040 con 0x55ff64923070 2024-09-10T08:53:32.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.836+0000 7f9db4544740 1 -- 172.21.15.172:0/2054904558 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9da0099fa0 con 0x55ff64923070 2024-09-10T08:53:32.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.836+0000 7f9d8affd640 1 -- 172.21.15.172:0/2054904558 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9d9c05f050 con 0x55ff64923070 2024-09-10T08:53:32.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.836+0000 7f9db4544740 10 client.26705 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:32.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.836+0000 7f9db4544740 1 -- 172.21.15.172:0/2054904558 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ff649a3690 con 0x55ff64923070 2024-09-10T08:53:32.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.836+0000 7f9d8affd640 1 -- 172.21.15.172:0/2054904558 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9da0099fa0 con 0x55ff64923070 2024-09-10T08:53:32.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.836+0000 7f9d8affd640 1 -- 172.21.15.172:0/2054904558 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f9d9c004c80 con 0x55ff64923070 2024-09-10T08:53:32.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.836+0000 7f9db4544740 10 client.26705 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:32.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.836+0000 7f9db4544740 10 client.26705 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:53:32.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.836+0000 7f9db4544740 20 client.26705 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:32.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.836+0000 7f9db4544740 1 --2- 172.21.15.172:0/2054904558 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55ff649a3a90 0x55ff649a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:32.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.836+0000 7f9db4544740 4 client.26705 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:53:32.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.836+0000 7f9db4544740 1 -- 172.21.15.172:0/2054904558 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff64886d70 con 0x55ff649a3a90 2024-09-10T08:53:32.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.837+0000 7f9daaffd640 1 --2- 172.21.15.172:0/2054904558 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55ff649a3a90 0x55ff649a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:32.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.837+0000 7f9daaffd640 1 --2- 172.21.15.172:0/2054904558 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55ff649a3a90 0x55ff649a5e80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bf782afed6bfbc4d server_cookie=5524506731a5925a in_seq=0 out_seq=0 2024-09-10T08:53:32.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.837+0000 7f9d8affd640 10 client.26705 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:53:32.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.837+0000 7f9d8affd640 1 -- 172.21.15.172:0/2054904558 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff64886d70 con 0x55ff649a3a90 2024-09-10T08:53:32.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.837+0000 7f9d8affd640 10 client.26705 handle_command_reply: tid=0 2024-09-10T08:53:32.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.837+0000 7f9db4544740 1 client.26705 shutdown 2024-09-10T08:53:32.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.837+0000 7f9db4544740 1 -- 172.21.15.172:0/2054904558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923070 msgr2=0x55ff64930870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.837+0000 7f9db4544740 1 --2- 172.21.15.172:0/2054904558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923070 0x55ff64930870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c02a4c0 tx=0x7f9d9c031520 comp rx=0 tx=0).stop 2024-09-10T08:53:32.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.837+0000 7f9db4544740 1 -- 172.21.15.172:0/2054904558 shutdown_connections 2024-09-10T08:53:32.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.837+0000 7f9db4544740 1 --2- 172.21.15.172:0/2054904558 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55ff649a3a90 0x55ff649a5e80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.837+0000 7f9db4544740 1 --2- 172.21.15.172:0/2054904558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff64924760 0x55ff6492d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.837+0000 7f9db4544740 1 --2- 172.21.15.172:0/2054904558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff64923de0 0x55ff64930db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.837+0000 7f9d8affd640 0 client.26705 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:53:32.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.837+0000 7f9db4544740 1 --2- 172.21.15.172:0/2054904558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff64923070 0x55ff64930870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.837+0000 7f9db4544740 1 -- 172.21.15.172:0/2054904558 >> 172.21.15.172:0/2054904558 conn(0x55ff648e2310 msgr2=0x55ff64914d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:32.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.837+0000 7f9db4544740 1 -- 172.21.15.172:0/2054904558 shutdown_connections 2024-09-10T08:53:32.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.838+0000 7f9db4544740 1 -- 172.21.15.172:0/2054904558 wait complete. 2024-09-10T08:53:32.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.838+0000 7f9db4544740 20 client.26705 trim_cache size 0 max 16384 2024-09-10T08:53:32.844 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:53:32.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.838+0000 7f9db216b640 1 -- 172.21.15.172:0/2514845732 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9d940a2b80 msgr2=0x7f9d940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.838+0000 7f9db216b640 1 --2- 172.21.15.172:0/2514845732 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9d940a2b80 0x7f9d940a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009970 tx=0x7f9da4004a40 comp rx=0 tx=0).stop 2024-09-10T08:53:32.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.838+0000 7f9db216b640 1 -- 172.21.15.172:0/2514845732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9dac102570 msgr2=0x7f9dac197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:32.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.838+0000 7f9db216b640 1 --2- 172.21.15.172:0/2514845732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9dac102570 0x7f9dac197820 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9da000e9f0 tx=0x7f9da000eeb0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.838+0000 7f9db216b640 1 -- 172.21.15.172:0/2514845732 shutdown_connections 2024-09-10T08:53:32.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.838+0000 7f9db216b640 1 --2- 172.21.15.172:0/2514845732 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9d940a2b80 0x7f9d940a5040 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.838+0000 7f9db216b640 1 --2- 172.21.15.172:0/2514845732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dac102ef0 0x7f9dac19bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.838+0000 7f9db216b640 1 --2- 172.21.15.172:0/2514845732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9dac102570 0x7f9dac197820 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.838+0000 7f9db216b640 1 --2- 172.21.15.172:0/2514845732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9dac100e20 0x7f9dac1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:32.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.838+0000 7f9db216b640 1 -- 172.21.15.172:0/2514845732 >> 172.21.15.172:0/2514845732 conn(0x7f9dac0fc7d0 msgr2=0x7f9dac0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:32.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.839+0000 7f9db216b640 1 -- 172.21.15.172:0/2514845732 shutdown_connections 2024-09-10T08:53:32.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:32.839+0000 7f9db216b640 1 -- 172.21.15.172:0/2514845732 wait complete. 2024-09-10T08:53:33.642 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:53:33.915 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:53:34.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.420+0000 7f4f12e63640 1 -- 172.21.15.172:0/3898891384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f0c101de0 msgr2=0x7f4f0c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:34.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.420+0000 7f4f12e63640 1 --2- 172.21.15.172:0/3898891384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f0c101de0 0x7f4f0c1021c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009900 tx=0x7f4f0002a410 comp rx=0 tx=0).stop 2024-09-10T08:53:34.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.420+0000 7f4f12e63640 1 -- 172.21.15.172:0/3898891384 shutdown_connections 2024-09-10T08:53:34.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.420+0000 7f4f12e63640 1 --2- 172.21.15.172:0/3898891384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f0c106f10 0x7f4f0c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.420+0000 7f4f12e63640 1 --2- 172.21.15.172:0/3898891384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f0c102790 0x7f4f0c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.420+0000 7f4f12e63640 1 --2- 172.21.15.172:0/3898891384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f0c101de0 0x7f4f0c1021c0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.420+0000 7f4f12e63640 1 -- 172.21.15.172:0/3898891384 >> 172.21.15.172:0/3898891384 conn(0x7f4f0c0fd8c0 msgr2=0x7f4f0c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:34.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.420+0000 7f4f12e63640 1 -- 172.21.15.172:0/3898891384 shutdown_connections 2024-09-10T08:53:34.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.420+0000 7f4f12e63640 1 -- 172.21.15.172:0/3898891384 wait complete. 2024-09-10T08:53:34.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.421+0000 7f4f12e63640 1 Processor -- start 2024-09-10T08:53:34.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.421+0000 7f4f12e63640 1 -- start start 2024-09-10T08:53:34.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.421+0000 7f4f12e63640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f0c101de0 0x7f4f0c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.421+0000 7f4f12e63640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f0c102790 0x7f4f0c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.421+0000 7f4f12e63640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f0c106f10 0x7f4f0c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.421+0000 7f4f12e63640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c111e30 con 0x7f4f0c106f10 2024-09-10T08:53:34.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.421+0000 7f4f12e63640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c111cb0 con 0x7f4f0c101de0 2024-09-10T08:53:34.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.421+0000 7f4f12e63640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c111fb0 con 0x7f4f0c102790 2024-09-10T08:53:34.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.421+0000 7f4f11400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f0c106f10 0x7f4f0c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.421+0000 7f4f11400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f0c106f10 0x7f4f0c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40974/0 (socket says 172.21.15.172:40974) 2024-09-10T08:53:34.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.421+0000 7f4f11400640 1 -- 172.21.15.172:0/2836984529 learned_addr learned my addr 172.21.15.172:0/2836984529 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:34.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.421+0000 7f4f10bff640 1 --2- 172.21.15.172:0/2836984529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f0c101de0 0x7f4f0c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.421+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/2836984529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f0c102790 0x7f4f0c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.422+0000 7f4f11400640 1 -- 172.21.15.172:0/2836984529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f0c102790 msgr2=0x7f4f0c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:34.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.422+0000 7f4f11400640 1 --2- 172.21.15.172:0/2836984529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f0c102790 0x7f4f0c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.422+0000 7f4f11400640 1 -- 172.21.15.172:0/2836984529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f0c101de0 msgr2=0x7f4f0c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:34.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.422+0000 7f4f11400640 1 --2- 172.21.15.172:0/2836984529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f0c101de0 0x7f4f0c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.422+0000 7f4f11400640 1 -- 172.21.15.172:0/2836984529 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0c1a67f0 con 0x7f4f0c106f10 2024-09-10T08:53:34.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.422+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/2836984529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f0c102790 0x7f4f0c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:34.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.422+0000 7f4f11400640 1 --2- 172.21.15.172:0/2836984529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f0c106f10 0x7f4f0c074f70 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400cce0 tx=0x7f4f0400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:34.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.422+0000 7f4f09ffb640 1 -- 172.21.15.172:0/2836984529 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0400f450 con 0x7f4f0c106f10 2024-09-10T08:53:34.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.422+0000 7f4f12e63640 1 -- 172.21.15.172:0/2836984529 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0c1a6ae0 con 0x7f4f0c106f10 2024-09-10T08:53:34.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.422+0000 7f4f09ffb640 1 -- 172.21.15.172:0/2836984529 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f0400faa0 con 0x7f4f0c106f10 2024-09-10T08:53:34.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.422+0000 7f4f09ffb640 1 -- 172.21.15.172:0/2836984529 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04018b70 con 0x7f4f0c106f10 2024-09-10T08:53:34.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.422+0000 7f4f12e63640 1 -- 172.21.15.172:0/2836984529 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f0c1a7020 con 0x7f4f0c106f10 2024-09-10T08:53:34.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.425+0000 7f4f09ffb640 1 -- 172.21.15.172:0/2836984529 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4f04018d10 con 0x7f4f0c106f10 2024-09-10T08:53:34.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.425+0000 7f4f1523c740 1 Processor -- start 2024-09-10T08:53:34.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.425+0000 7f4f1523c740 1 -- start start 2024-09-10T08:53:34.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.425+0000 7f4f1523c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a25b40 0x55c2f0a24090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.425+0000 7f4f1523c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a245d0 0x55c2f0a24a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.425+0000 7f4f1523c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a2f370 0x55c2f0a31760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.425+0000 7f4f1523c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c2f098a2f0 con 0x55c2f0a25b40 2024-09-10T08:53:34.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.425+0000 7f4f1523c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2f0931cf0 con 0x55c2f0a2f370 2024-09-10T08:53:34.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.425+0000 7f4f1523c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c2f09862b0 con 0x55c2f0a245d0 2024-09-10T08:53:34.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.425+0000 7f4f0bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a25b40 0x55c2f0a24090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.426+0000 7f4f0bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a25b40 0x55c2f0a24090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40986/0 (socket says 172.21.15.172:40986) 2024-09-10T08:53:34.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.426+0000 7f4f0bfff640 1 -- 172.21.15.172:0/4090218146 learned_addr learned my addr 172.21.15.172:0/4090218146 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.426+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/4090218146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a2f370 0x55c2f0a31760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.426+0000 7f4f10bff640 1 --2- 172.21.15.172:0/4090218146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a245d0 0x55c2f0a24a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.426+0000 7f4f09ffb640 1 --2- 172.21.15.172:0/2836984529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4efc0a2b80 0x7f4efc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.426+0000 7f4f0bfff640 1 -- 172.21.15.172:0/4090218146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a245d0 msgr2=0x55c2f0a24a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.426+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/4090218146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a245d0 0x55c2f0a24a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.426+0000 7f4f0bfff640 1 -- 172.21.15.172:0/4090218146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a2f370 msgr2=0x55c2f0a31760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.426+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/4090218146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a2f370 0x55c2f0a31760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.426+0000 7f4f11400640 1 --2- 172.21.15.172:0/2836984529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4efc0a2b80 0x7f4efc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.426+0000 7f4f0bfff640 1 -- 172.21.15.172:0/4090218146 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f09163a0 con 0x55c2f0a25b40 2024-09-10T08:53:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.426+0000 7f4f09ffb640 1 -- 172.21.15.172:0/2836984529 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4f040cf750 con 0x7f4f0c106f10 2024-09-10T08:53:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.426+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/4090218146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a25b40 0x55c2f0a24090 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8014960 tx=0x7f4ef800b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a2176a5cd95ee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.427+0000 7f4ef6ffd640 1 -- 172.21.15.172:0/4090218146 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8026070 con 0x55c2f0a25b40 2024-09-10T08:53:34.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.427+0000 7f4ef6ffd640 1 -- 172.21.15.172:0/4090218146 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ef8021950 con 0x55c2f0a25b40 2024-09-10T08:53:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.427+0000 7f4f11400640 1 --2- 172.21.15.172:0/2836984529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4efc0a2b80 0x7f4efc0a5040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4f040980b0 tx=0x7f4f0400af30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.427+0000 7f4ef6ffd640 1 -- 172.21.15.172:0/4090218146 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800d940 con 0x55c2f0a25b40 2024-09-10T08:53:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.427+0000 7f4f1523c740 1 -- 172.21.15.172:0/4090218146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a25b40 msgr2=0x55c2f0a24090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.427+0000 7f4f1523c740 1 --2- 172.21.15.172:0/4090218146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a25b40 0x55c2f0a24090 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8014960 tx=0x7f4ef800b490 comp rx=0 tx=0).stop 2024-09-10T08:53:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.428+0000 7f4f1523c740 1 -- 172.21.15.172:0/4090218146 shutdown_connections 2024-09-10T08:53:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.428+0000 7f4f1523c740 1 --2- 172.21.15.172:0/4090218146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a2f370 0x55c2f0a31760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.428+0000 7f4f1523c740 1 --2- 172.21.15.172:0/4090218146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a245d0 0x55c2f0a24a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.428+0000 7f4f1523c740 1 --2- 172.21.15.172:0/4090218146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a25b40 0x55c2f0a24090 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.428+0000 7f4f1523c740 1 -- 172.21.15.172:0/4090218146 >> 172.21.15.172:0/4090218146 conn(0x55c2f09ed310 msgr2=0x55c2f0a20c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.428+0000 7f4f1523c740 1 -- 172.21.15.172:0/4090218146 shutdown_connections 2024-09-10T08:53:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.428+0000 7f4f1523c740 1 -- 172.21.15.172:0/4090218146 wait complete. 2024-09-10T08:53:34.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.428+0000 7f4f1523c740 1 -- start start 2024-09-10T08:53:34.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.428+0000 7f4f1523c740 1 Processor -- start 2024-09-10T08:53:34.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.428+0000 7f4f1523c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a245d0 0x55c2f0a2ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.428+0000 7f4f1523c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a25b40 0x55c2f0a2c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.428+0000 7f4f1523c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a2f370 0x55c2f0a2c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.428+0000 7f4f1523c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c2f0a32010 con 0x55c2f0a245d0 2024-09-10T08:53:34.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.428+0000 7f4f1523c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2f0945060 con 0x55c2f0a2f370 2024-09-10T08:53:34.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.428+0000 7f4f1523c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c2f0a32190 con 0x55c2f0a25b40 2024-09-10T08:53:34.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.429+0000 7f4f1523c740 10 client.0 _ll_register_callbacks cb 0x55c2f06988c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:34.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.429+0000 7f4f0bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a245d0 0x55c2f0a2ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.429+0000 7f4f0bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a245d0 0x55c2f0a2ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40992/0 (socket says 172.21.15.172:40992) 2024-09-10T08:53:34.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.429+0000 7f4f0bfff640 1 -- 172.21.15.172:0/3842890834 learned_addr learned my addr 172.21.15.172:0/3842890834 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:34.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.429+0000 7f4f10bff640 1 --2- 172.21.15.172:0/3842890834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a25b40 0x55c2f0a2c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.429+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/3842890834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a2f370 0x55c2f0a2c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.429+0000 7f4f0bfff640 1 -- 172.21.15.172:0/3842890834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a25b40 msgr2=0x55c2f0a2c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:34.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.429+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/3842890834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a25b40 0x55c2f0a2c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.429+0000 7f4f0bfff640 1 -- 172.21.15.172:0/3842890834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a2f370 msgr2=0x55c2f0a2c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:34.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.429+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/3842890834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a2f370 0x55c2f0a2c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.429+0000 7f4f0bfff640 1 -- 172.21.15.172:0/3842890834 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f06fb3c0 con 0x55c2f0a245d0 2024-09-10T08:53:34.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.429+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/3842890834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a245d0 0x55c2f0a2ed70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8006010 tx=0x7f4ef80145a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92aeecef69d16dbc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:34.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.429+0000 7f4eebfff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:34.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.429+0000 7f4eebfff640 1 -- 172.21.15.172:0/3842890834 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8032060 con 0x55c2f0a245d0 2024-09-10T08:53:34.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.429+0000 7f4eebfff640 1 -- 172.21.15.172:0/3842890834 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4ef8005040 con 0x55c2f0a245d0 2024-09-10T08:53:34.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.429+0000 7f4f1523c740 1 -- 172.21.15.172:0/3842890834 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef8021950 con 0x55c2f0a245d0 2024-09-10T08:53:34.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.429+0000 7f4eebfff640 1 -- 172.21.15.172:0/3842890834 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4ef8026040 con 0x55c2f0a245d0 2024-09-10T08:53:34.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.430+0000 7f4f1523c740 10 client.17643 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:34.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.430+0000 7f4f1523c740 1 -- 172.21.15.172:0/3842890834 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c2f0a3dab0 con 0x55c2f0a245d0 2024-09-10T08:53:34.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.430+0000 7f4eebfff640 1 -- 172.21.15.172:0/3842890834 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ef8021950 con 0x55c2f0a245d0 2024-09-10T08:53:34.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.430+0000 7f4eebfff640 1 -- 172.21.15.172:0/3842890834 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f4ef8004550 con 0x55c2f0a245d0 2024-09-10T08:53:34.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.430+0000 7f4f1523c740 10 client.17643 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:34.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.430+0000 7f4f1523c740 10 client.17643 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:53:34.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.430+0000 7f4f1523c740 20 client.17643 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:34.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.430+0000 7f4f1523c740 1 --2- 172.21.15.172:0/3842890834 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55c2f0a3eeb0 0x55c2f0a412a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.430+0000 7f4f1523c740 4 client.17643 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:53:34.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.430+0000 7f4f1523c740 1 -- 172.21.15.172:0/3842890834 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2f0997860 con 0x55c2f0a3eeb0 2024-09-10T08:53:34.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.430+0000 7f4f10bff640 1 --2- 172.21.15.172:0/3842890834 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55c2f0a3eeb0 0x55c2f0a412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.431+0000 7f4f10bff640 1 --2- 172.21.15.172:0/3842890834 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55c2f0a3eeb0 0x55c2f0a412a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25c7a65895d60922 server_cookie=3b15f02ace3e2d56 in_seq=0 out_seq=0 2024-09-10T08:53:34.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.431+0000 7f4eebfff640 10 client.17643 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:53:34.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.431+0000 7f4eebfff640 1 -- 172.21.15.172:0/3842890834 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2f0997860 con 0x55c2f0a3eeb0 2024-09-10T08:53:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.431+0000 7f4eebfff640 10 client.17643 handle_command_reply: tid=0 2024-09-10T08:53:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.431+0000 7f4f1523c740 1 client.17643 shutdown 2024-09-10T08:53:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.431+0000 7f4f1523c740 1 -- 172.21.15.172:0/3842890834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a245d0 msgr2=0x55c2f0a2ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.431+0000 7f4f1523c740 1 --2- 172.21.15.172:0/3842890834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a245d0 0x55c2f0a2ed70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8006010 tx=0x7f4ef80145a0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.432+0000 7f4f1523c740 1 -- 172.21.15.172:0/3842890834 shutdown_connections 2024-09-10T08:53:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.432+0000 7f4f1523c740 1 --2- 172.21.15.172:0/3842890834 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55c2f0a3eeb0 0x55c2f0a412a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.432+0000 7f4f1523c740 1 --2- 172.21.15.172:0/3842890834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a2f370 0x55c2f0a2c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.432+0000 7f4f1523c740 1 --2- 172.21.15.172:0/3842890834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a25b40 0x55c2f0a2c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.432+0000 7f4f1523c740 1 --2- 172.21.15.172:0/3842890834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a245d0 0x55c2f0a2ed70 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.432+0000 7f4f1523c740 1 -- 172.21.15.172:0/3842890834 >> 172.21.15.172:0/3842890834 conn(0x55c2f09ed310 msgr2=0x55c2f0a26990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.432+0000 7f4eebfff640 0 client.17643 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:53:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.432+0000 7f4f1523c740 1 -- 172.21.15.172:0/3842890834 shutdown_connections 2024-09-10T08:53:34.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.432+0000 7f4f1523c740 1 -- 172.21.15.172:0/3842890834 wait complete. 2024-09-10T08:53:34.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.432+0000 7f4f1523c740 20 client.17643 trim_cache size 0 max 16384 2024-09-10T08:53:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.446+0000 7f4f1523c740 1 Processor -- start 2024-09-10T08:53:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.446+0000 7f4f1523c740 1 -- start start 2024-09-10T08:53:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.447+0000 7f4f1523c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a245d0 0x55c2f0a2b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.447+0000 7f4f1523c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a25b40 0x55c2f0a43ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.447+0000 7f4f1523c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a2f370 0x55c2f0a44420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.447+0000 7f4f1523c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1ac880 con 0x55c2f0a245d0 2024-09-10T08:53:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.447+0000 7f4f1523c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1ae3c0 con 0x55c2f0a25b40 2024-09-10T08:53:34.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.447+0000 7f4f1523c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1acf50 con 0x55c2f0a2f370 2024-09-10T08:53:34.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.447+0000 7f4f0bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a245d0 0x55c2f0a2b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.447+0000 7f4f0bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a245d0 0x55c2f0a2b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40994/0 (socket says 172.21.15.172:40994) 2024-09-10T08:53:34.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.447+0000 7f4f0bfff640 1 -- 172.21.15.172:0/1064449019 learned_addr learned my addr 172.21.15.172:0/1064449019 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:34.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.447+0000 7f4f10bff640 1 --2- 172.21.15.172:0/1064449019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a25b40 0x55c2f0a43ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.447+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/1064449019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a2f370 0x55c2f0a44420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.447+0000 7f4f0bfff640 1 -- 172.21.15.172:0/1064449019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a2f370 msgr2=0x55c2f0a44420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:34.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.447+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/1064449019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a2f370 0x55c2f0a44420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.447+0000 7f4f0bfff640 1 -- 172.21.15.172:0/1064449019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a25b40 msgr2=0x55c2f0a43ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:34.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.447+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/1064449019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a25b40 0x55c2f0a43ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.447+0000 7f4f0bfff640 1 -- 172.21.15.172:0/1064449019 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f0a2d110 con 0x55c2f0a245d0 2024-09-10T08:53:34.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.447+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/1064449019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a245d0 0x55c2f0a2b000 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x55c2f0a21c00 tx=0x7f4ef800bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad6a3b037d11d4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:34.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.448+0000 7f4ef67fc640 1 -- 172.21.15.172:0/1064449019 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8032040 con 0x55c2f0a245d0 2024-09-10T08:53:34.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.448+0000 7f4ef67fc640 1 -- 172.21.15.172:0/1064449019 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ef80141c0 con 0x55c2f0a245d0 2024-09-10T08:53:34.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.448+0000 7f4ef67fc640 1 -- 172.21.15.172:0/1064449019 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8004fc0 con 0x55c2f0a245d0 2024-09-10T08:53:34.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.448+0000 7f4f1523c740 1 -- 172.21.15.172:0/1064449019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a245d0 msgr2=0x55c2f0a2b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:34.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.448+0000 7f4f1523c740 1 --2- 172.21.15.172:0/1064449019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a245d0 0x55c2f0a2b000 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x55c2f0a21c00 tx=0x7f4ef800bd00 comp rx=0 tx=0).stop 2024-09-10T08:53:34.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.448+0000 7f4f1523c740 1 -- 172.21.15.172:0/1064449019 shutdown_connections 2024-09-10T08:53:34.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.448+0000 7f4f1523c740 1 --2- 172.21.15.172:0/1064449019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a2f370 0x55c2f0a44420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.448+0000 7f4f1523c740 1 --2- 172.21.15.172:0/1064449019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a25b40 0x55c2f0a43ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.448+0000 7f4f1523c740 1 --2- 172.21.15.172:0/1064449019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a245d0 0x55c2f0a2b000 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.448+0000 7f4f1523c740 1 -- 172.21.15.172:0/1064449019 >> 172.21.15.172:0/1064449019 conn(0x55c2f09ed310 msgr2=0x55c2f0a2a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:34.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.449+0000 7f4f1523c740 1 -- 172.21.15.172:0/1064449019 shutdown_connections 2024-09-10T08:53:34.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.449+0000 7f4f1523c740 1 -- 172.21.15.172:0/1064449019 wait complete. 2024-09-10T08:53:34.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.449+0000 7f4f1523c740 1 -- start start 2024-09-10T08:53:34.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.449+0000 7f4f1523c740 1 Processor -- start 2024-09-10T08:53:34.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.449+0000 7f4f1523c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a245d0 0x55c2f0a41080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.449+0000 7f4f1523c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a25b40 0x55c2f0a415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.449+0000 7f4f1523c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a2f370 0x55c2f0aa77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.450+0000 7f4f1523c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1adcf0 con 0x55c2f0a25b40 2024-09-10T08:53:34.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.450+0000 7f4f1523c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1ad620 con 0x55c2f0a245d0 2024-09-10T08:53:34.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.450+0000 7f4f1523c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1aa860 con 0x55c2f0a2f370 2024-09-10T08:53:34.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.450+0000 7f4f1523c740 10 client.0 _ll_register_callbacks cb 0x7f4ef8024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:34.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.450+0000 7f4f10bff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a25b40 0x55c2f0a415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.450+0000 7f4f10bff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a25b40 0x55c2f0a415c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40996/0 (socket says 172.21.15.172:40996) 2024-09-10T08:53:34.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.450+0000 7f4f10bff640 1 -- 172.21.15.172:0/3495249138 learned_addr learned my addr 172.21.15.172:0/3495249138 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:34.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.450+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/3495249138 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a2f370 0x55c2f0aa77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.450+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/3495249138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a245d0 0x55c2f0a41080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.450+0000 7f4f10bff640 1 -- 172.21.15.172:0/3495249138 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a2f370 msgr2=0x55c2f0aa77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.450+0000 7f4f10bff640 1 --2- 172.21.15.172:0/3495249138 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a2f370 0x55c2f0aa77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.450+0000 7f4f10bff640 1 -- 172.21.15.172:0/3495249138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a245d0 msgr2=0x55c2f0a41080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.450+0000 7f4f10bff640 1 --2- 172.21.15.172:0/3495249138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a245d0 0x55c2f0a41080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.450+0000 7f4f10bff640 1 -- 172.21.15.172:0/3495249138 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f0aa7d20 con 0x55c2f0a25b40 2024-09-10T08:53:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.450+0000 7f4f10bff640 1 --2- 172.21.15.172:0/3495249138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a25b40 0x55c2f0a415c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4f0002a450 tx=0x7f4f00005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc9923a36194b34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.450+0000 7f4eebfff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.451+0000 7f4eebfff640 1 -- 172.21.15.172:0/3495249138 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00035040 con 0x55c2f0a25b40 2024-09-10T08:53:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.451+0000 7f4eebfff640 1 -- 172.21.15.172:0/3495249138 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4f00005590 con 0x55c2f0a25b40 2024-09-10T08:53:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.451+0000 7f4f1523c740 1 -- 172.21.15.172:0/3495249138 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef80141c0 con 0x55c2f0a25b40 2024-09-10T08:53:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.451+0000 7f4eebfff640 1 -- 172.21.15.172:0/3495249138 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4f00004770 con 0x55c2f0a25b40 2024-09-10T08:53:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.451+0000 7f4f1523c740 10 client.17649 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:34.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.451+0000 7f4f1523c740 1 -- 172.21.15.172:0/3495249138 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c2f0aa8f40 con 0x55c2f0a25b40 2024-09-10T08:53:34.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.451+0000 7f4eebfff640 1 -- 172.21.15.172:0/3495249138 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ef80141c0 con 0x55c2f0a25b40 2024-09-10T08:53:34.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.451+0000 7f4eebfff640 1 -- 172.21.15.172:0/3495249138 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f4f00003c50 con 0x55c2f0a25b40 2024-09-10T08:53:34.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.451+0000 7f4f1523c740 10 client.17649 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:34.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.451+0000 7f4f1523c740 10 client.17649 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:53:34.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.451+0000 7f4f1523c740 20 client.17649 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:34.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.451+0000 7f4f1523c740 1 --2- 172.21.15.172:0/3495249138 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55c2f0aa9670 0x55c2f0aaba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:34.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.451+0000 7f4f1523c740 4 client.17649 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:53:34.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.451+0000 7f4f1523c740 1 -- 172.21.15.172:0/3495249138 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2f0991d70 con 0x55c2f0aa9670 2024-09-10T08:53:34.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.452+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/3495249138 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55c2f0aa9670 0x55c2f0aaba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:34.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.452+0000 7f4eebfff640 10 client.17649 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:53:34.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.452+0000 7f4f0bfff640 1 --2- 172.21.15.172:0/3495249138 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55c2f0aa9670 0x55c2f0aaba90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee23aca197aa5197 server_cookie=af0f628ef9e46b18 in_seq=0 out_seq=0 2024-09-10T08:53:34.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.452+0000 7f4eebfff640 1 -- 172.21.15.172:0/3495249138 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2f0991d70 con 0x55c2f0aa9670 2024-09-10T08:53:34.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.452+0000 7f4eebfff640 10 client.17649 handle_command_reply: tid=0 2024-09-10T08:53:34.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.452+0000 7f4f1523c740 1 client.17649 shutdown 2024-09-10T08:53:34.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.452+0000 7f4f1523c740 1 -- 172.21.15.172:0/3495249138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a25b40 msgr2=0x55c2f0a415c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:34.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.452+0000 7f4f1523c740 1 --2- 172.21.15.172:0/3495249138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a25b40 0x55c2f0a415c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4f0002a450 tx=0x7f4f00005850 comp rx=0 tx=0).stop 2024-09-10T08:53:34.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.453+0000 7f4f1523c740 1 -- 172.21.15.172:0/3495249138 shutdown_connections 2024-09-10T08:53:34.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.453+0000 7f4f1523c740 1 --2- 172.21.15.172:0/3495249138 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55c2f0aa9670 0x55c2f0aaba90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.453+0000 7f4f1523c740 1 --2- 172.21.15.172:0/3495249138 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c2f0a2f370 0x55c2f0aa77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.453+0000 7f4f1523c740 1 --2- 172.21.15.172:0/3495249138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c2f0a25b40 0x55c2f0a415c0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.453+0000 7f4f1523c740 1 --2- 172.21.15.172:0/3495249138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2f0a245d0 0x55c2f0a41080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.453+0000 7f4eebfff640 0 client.17649 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:53:34.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.453+0000 7f4f1523c740 1 -- 172.21.15.172:0/3495249138 >> 172.21.15.172:0/3495249138 conn(0x55c2f09ed310 msgr2=0x55c2f0a1f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:34.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.453+0000 7f4f1523c740 1 -- 172.21.15.172:0/3495249138 shutdown_connections 2024-09-10T08:53:34.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.453+0000 7f4f1523c740 1 -- 172.21.15.172:0/3495249138 wait complete. 2024-09-10T08:53:34.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.453+0000 7f4f1523c740 20 client.17649 trim_cache size 0 max 16384 2024-09-10T08:53:34.459 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:53:34.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.453+0000 7f4f12e63640 1 -- 172.21.15.172:0/2836984529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4efc0a2b80 msgr2=0x7f4efc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:34.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.453+0000 7f4f12e63640 1 --2- 172.21.15.172:0/2836984529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4efc0a2b80 0x7f4efc0a5040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4f040980b0 tx=0x7f4f0400af30 comp rx=0 tx=0).stop 2024-09-10T08:53:34.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.453+0000 7f4f12e63640 1 -- 172.21.15.172:0/2836984529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f0c106f10 msgr2=0x7f4f0c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:34.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.453+0000 7f4f12e63640 1 --2- 172.21.15.172:0/2836984529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f0c106f10 0x7f4f0c074f70 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400cce0 tx=0x7f4f0400a620 comp rx=0 tx=0).stop 2024-09-10T08:53:34.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.454+0000 7f4f12e63640 1 -- 172.21.15.172:0/2836984529 shutdown_connections 2024-09-10T08:53:34.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.454+0000 7f4f12e63640 1 --2- 172.21.15.172:0/2836984529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4efc0a2b80 0x7f4efc0a5040 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.454+0000 7f4f12e63640 1 --2- 172.21.15.172:0/2836984529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f0c106f10 0x7f4f0c074f70 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.454+0000 7f4f12e63640 1 --2- 172.21.15.172:0/2836984529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f0c102790 0x7f4f0c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.454+0000 7f4f12e63640 1 --2- 172.21.15.172:0/2836984529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f0c101de0 0x7f4f0c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:34.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.454+0000 7f4f12e63640 1 -- 172.21.15.172:0/2836984529 >> 172.21.15.172:0/2836984529 conn(0x7f4f0c0fd8c0 msgr2=0x7f4f0c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:34.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.454+0000 7f4f12e63640 1 -- 172.21.15.172:0/2836984529 shutdown_connections 2024-09-10T08:53:34.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:34.454+0000 7f4f12e63640 1 -- 172.21.15.172:0/2836984529 wait complete. 2024-09-10T08:53:34.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:34 smithi172 ceph-mon[30854]: pgmap v570: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8.9 MiB/s wr, 12 op/s 2024-09-10T08:53:34.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:34 smithi174 ceph-mon[34068]: pgmap v570: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8.9 MiB/s wr, 12 op/s 2024-09-10T08:53:34.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:34 smithi191 ceph-mon[35314]: pgmap v570: 129 pgs: 129 active+clean; 5.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8.9 MiB/s wr, 12 op/s 2024-09-10T08:53:35.161 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:53:35.437 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:53:36.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.188+0000 7f810ee0b640 1 -- 172.21.15.172:0/3246945974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8108102ef0 msgr2=0x7f810810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:36.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.188+0000 7f810ee0b640 1 --2- 172.21.15.172:0/3246945974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8108102ef0 0x7f810810da40 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8100009ca0 tx=0x7f810002a950 comp rx=0 tx=0).stop 2024-09-10T08:53:36.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.189+0000 7f810ee0b640 1 -- 172.21.15.172:0/3246945974 shutdown_connections 2024-09-10T08:53:36.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.189+0000 7f810ee0b640 1 --2- 172.21.15.172:0/3246945974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8108102ef0 0x7f810810da40 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.189+0000 7f810ee0b640 1 --2- 172.21.15.172:0/3246945974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8108102570 0x7f81081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.189+0000 7f810ee0b640 1 --2- 172.21.15.172:0/3246945974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8108100e20 0x7f8108101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.189+0000 7f810ee0b640 1 -- 172.21.15.172:0/3246945974 >> 172.21.15.172:0/3246945974 conn(0x7f81080fc7d0 msgr2=0x7f81080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:36.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.189+0000 7f810ee0b640 1 -- 172.21.15.172:0/3246945974 shutdown_connections 2024-09-10T08:53:36.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.189+0000 7f810ee0b640 1 -- 172.21.15.172:0/3246945974 wait complete. 2024-09-10T08:53:36.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.189+0000 7f810ee0b640 1 Processor -- start 2024-09-10T08:53:36.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.189+0000 7f810ee0b640 1 -- start start 2024-09-10T08:53:36.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.190+0000 7f810ee0b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8108100e20 0x7f81081983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.190+0000 7f810ee0b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8108102570 0x7f81081988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.190+0000 7f810ee0b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8108102ef0 0x7f810819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.190+0000 7f810ee0b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8108110e30 con 0x7f8108102ef0 2024-09-10T08:53:36.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.190+0000 7f810ee0b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8108110cb0 con 0x7f8108100e20 2024-09-10T08:53:36.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.190+0000 7f810ee0b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8108110fb0 con 0x7f8108102570 2024-09-10T08:53:36.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.190+0000 7f810d3a8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8108102ef0 0x7f810819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.190+0000 7f810d3a8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8108102ef0 0x7f810819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35304/0 (socket says 172.21.15.172:35304) 2024-09-10T08:53:36.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.190+0000 7f810d3a8640 1 -- 172.21.15.172:0/1170161103 learned_addr learned my addr 172.21.15.172:0/1170161103 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:36.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.190+0000 7f8107fff640 1 --2- 172.21.15.172:0/1170161103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8108102570 0x7f81081988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.190+0000 7f810cba7640 1 --2- 172.21.15.172:0/1170161103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8108100e20 0x7f81081983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.190+0000 7f810d3a8640 1 -- 172.21.15.172:0/1170161103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8108102570 msgr2=0x7f81081988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:36.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.190+0000 7f810d3a8640 1 --2- 172.21.15.172:0/1170161103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8108102570 0x7f81081988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.190+0000 7f810d3a8640 1 -- 172.21.15.172:0/1170161103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8108100e20 msgr2=0x7f81081983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:36.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.190+0000 7f810d3a8640 1 --2- 172.21.15.172:0/1170161103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8108100e20 0x7f81081983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.190+0000 7f810d3a8640 1 -- 172.21.15.172:0/1170161103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810819d350 con 0x7f8108102ef0 2024-09-10T08:53:36.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.190+0000 7f8107fff640 1 --2- 172.21.15.172:0/1170161103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8108102570 0x7f81081988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:36.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.191+0000 7f810d3a8640 1 --2- 172.21.15.172:0/1170161103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8108102ef0 0x7f810819cc70 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f8100004af0 tx=0x7f8100031f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:36.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.191+0000 7f8105ffb640 1 -- 172.21.15.172:0/1170161103 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100034040 con 0x7f8108102ef0 2024-09-10T08:53:36.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.191+0000 7f8105ffb640 1 -- 172.21.15.172:0/1170161103 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81000308f0 con 0x7f8108102ef0 2024-09-10T08:53:36.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.191+0000 7f810ee0b640 1 -- 172.21.15.172:0/1170161103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810819d5e0 con 0x7f8108102ef0 2024-09-10T08:53:36.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.191+0000 7f8105ffb640 1 -- 172.21.15.172:0/1170161103 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100004600 con 0x7f8108102ef0 2024-09-10T08:53:36.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.191+0000 7f810ee0b640 1 -- 172.21.15.172:0/1170161103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81081a4f20 con 0x7f8108102ef0 2024-09-10T08:53:36.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.193+0000 7f8105ffb640 1 -- 172.21.15.172:0/1170161103 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8100037020 con 0x7f8108102ef0 2024-09-10T08:53:36.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.194+0000 7f8105ffb640 1 --2- 172.21.15.172:0/1170161103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80f80a2c00 0x7f80f80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.194+0000 7f8105ffb640 1 -- 172.21.15.172:0/1170161103 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f81000f00d0 con 0x7f8108102ef0 2024-09-10T08:53:36.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.194+0000 7f810cba7640 1 --2- 172.21.15.172:0/1170161103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80f80a2c00 0x7f80f80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.194+0000 7f81111e4740 1 Processor -- start 2024-09-10T08:53:36.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.194+0000 7f810cba7640 1 --2- 172.21.15.172:0/1170161103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80f80a2c00 0x7f80f80a50c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009bd0 tx=0x7f80fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:36.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.195+0000 7f81111e4740 1 -- start start 2024-09-10T08:53:36.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.195+0000 7f81111e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832b5b40 0x5607832b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.195+0000 7f81111e4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.195+0000 7f81111e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832bf370 0x5607832c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.195+0000 7f81111e4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56078321a2f0 con 0x5607832b45d0 2024-09-10T08:53:36.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.195+0000 7f81111e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5607831c1cf0 con 0x5607832bf370 2024-09-10T08:53:36.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.195+0000 7f81111e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5607832162b0 con 0x5607832b5b40 2024-09-10T08:53:36.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.195+0000 7f8107fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.195+0000 7f8107fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35306/0 (socket says 172.21.15.172:35306) 2024-09-10T08:53:36.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.195+0000 7f8107fff640 1 -- 172.21.15.172:0/2989861351 learned_addr learned my addr 172.21.15.172:0/2989861351 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:36.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.195+0000 7f810d3a8640 1 --2- 172.21.15.172:0/2989861351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832bf370 0x5607832c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.195+0000 7f810cba7640 1 --2- 172.21.15.172:0/2989861351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832b5b40 0x5607832b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.195+0000 7f8107fff640 1 -- 172.21.15.172:0/2989861351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832b5b40 msgr2=0x5607832b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:36.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.195+0000 7f8107fff640 1 --2- 172.21.15.172:0/2989861351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832b5b40 0x5607832b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.195+0000 7f8107fff640 1 -- 172.21.15.172:0/2989861351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832bf370 msgr2=0x5607832c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:36.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.195+0000 7f8107fff640 1 --2- 172.21.15.172:0/2989861351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832bf370 0x5607832c1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.195+0000 7f8107fff640 1 -- 172.21.15.172:0/2989861351 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607831a63a0 con 0x5607832b45d0 2024-09-10T08:53:36.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.196+0000 7f8107fff640 1 --2- 172.21.15.172:0/2989861351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832b4a10 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f80f40114f0 tx=0x7f80f40119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb3878f7e456830 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:36.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.196+0000 7f80f2ffd640 1 -- 172.21.15.172:0/2989861351 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f401e070 con 0x5607832b45d0 2024-09-10T08:53:36.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.196+0000 7f80f2ffd640 1 -- 172.21.15.172:0/2989861351 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80f40035f0 con 0x5607832b45d0 2024-09-10T08:53:36.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.196+0000 7f80f2ffd640 1 -- 172.21.15.172:0/2989861351 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f400f820 con 0x5607832b45d0 2024-09-10T08:53:36.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.197+0000 7f81111e4740 1 -- 172.21.15.172:0/2989861351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 msgr2=0x5607832b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:36.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.197+0000 7f81111e4740 1 --2- 172.21.15.172:0/2989861351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832b4a10 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f80f40114f0 tx=0x7f80f40119b0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.197+0000 7f81111e4740 1 -- 172.21.15.172:0/2989861351 shutdown_connections 2024-09-10T08:53:36.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.197+0000 7f81111e4740 1 --2- 172.21.15.172:0/2989861351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832bf370 0x5607832c1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.197+0000 7f81111e4740 1 --2- 172.21.15.172:0/2989861351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832b4a10 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.197+0000 7f81111e4740 1 --2- 172.21.15.172:0/2989861351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832b5b40 0x5607832b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.197+0000 7f81111e4740 1 -- 172.21.15.172:0/2989861351 >> 172.21.15.172:0/2989861351 conn(0x56078327d310 msgr2=0x5607832b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:36.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.197+0000 7f81111e4740 1 -- 172.21.15.172:0/2989861351 shutdown_connections 2024-09-10T08:53:36.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.197+0000 7f81111e4740 1 -- 172.21.15.172:0/2989861351 wait complete. 2024-09-10T08:53:36.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.198+0000 7f81111e4740 1 -- start start 2024-09-10T08:53:36.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.198+0000 7f81111e4740 1 Processor -- start 2024-09-10T08:53:36.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.198+0000 7f81111e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832b45d0 0x5607832beb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.198+0000 7f81111e4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b5b40 0x5607832bc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.198+0000 7f81111e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832bf370 0x5607832bc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.199+0000 7f81111e4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5607832c2070 con 0x5607832b5b40 2024-09-10T08:53:36.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.199+0000 7f81111e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5607831d5060 con 0x5607832b45d0 2024-09-10T08:53:36.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.199+0000 7f81111e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5607832c21f0 con 0x5607832bf370 2024-09-10T08:53:36.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.199+0000 7f8107fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b5b40 0x5607832bc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.199+0000 7f8107fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b5b40 0x5607832bc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35314/0 (socket says 172.21.15.172:35314) 2024-09-10T08:53:36.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.199+0000 7f8107fff640 1 -- 172.21.15.172:0/325547950 learned_addr learned my addr 172.21.15.172:0/325547950 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:36.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.199+0000 7f81111e4740 10 client.0 _ll_register_callbacks cb 0x560782f288c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:36.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.199+0000 7f810cba7640 1 --2- 172.21.15.172:0/325547950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832b45d0 0x5607832beb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.199+0000 7f8107fff640 1 -- 172.21.15.172:0/325547950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832bf370 msgr2=0x5607832bc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:36.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.199+0000 7f810d3a8640 1 --2- 172.21.15.172:0/325547950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832bf370 0x5607832bc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.199+0000 7f8107fff640 1 --2- 172.21.15.172:0/325547950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832bf370 0x5607832bc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.199+0000 7f8107fff640 1 -- 172.21.15.172:0/325547950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832b45d0 msgr2=0x5607832beb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:36.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.199+0000 7f8107fff640 1 --2- 172.21.15.172:0/325547950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832b45d0 0x5607832beb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.199+0000 7f8107fff640 1 -- 172.21.15.172:0/325547950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560782f8b3c0 con 0x5607832b5b40 2024-09-10T08:53:36.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.199+0000 7f8107fff640 1 --2- 172.21.15.172:0/325547950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b5b40 0x5607832bc080 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f80f401c040 tx=0x7f80f40041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d81fb042b0753ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:36.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.199+0000 7f80e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:36.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.200+0000 7f80e7fff640 1 -- 172.21.15.172:0/325547950 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f401e040 con 0x5607832b5b40 2024-09-10T08:53:36.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.200+0000 7f81111e4740 1 -- 172.21.15.172:0/325547950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80f40035f0 con 0x5607832b5b40 2024-09-10T08:53:36.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.200+0000 7f80e7fff640 1 -- 172.21.15.172:0/325547950 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f80f40043d0 con 0x5607832b5b40 2024-09-10T08:53:36.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.200+0000 7f80e7fff640 1 -- 172.21.15.172:0/325547950 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f80f4032050 con 0x5607832b5b40 2024-09-10T08:53:36.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.200+0000 7f81111e4740 10 client.17661 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:36.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.200+0000 7f81111e4740 1 -- 172.21.15.172:0/325547950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5607832d63b0 con 0x5607832b5b40 2024-09-10T08:53:36.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.200+0000 7f80e7fff640 1 -- 172.21.15.172:0/325547950 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80f40035f0 con 0x5607832b5b40 2024-09-10T08:53:36.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.200+0000 7f80e7fff640 1 -- 172.21.15.172:0/325547950 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f80f4021030 con 0x5607832b5b40 2024-09-10T08:53:36.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.201+0000 7f81111e4740 10 client.17661 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:36.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.201+0000 7f81111e4740 10 client.17661 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:53:36.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.201+0000 7f81111e4740 20 client.17661 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:36.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.201+0000 7f81111e4740 1 --2- 172.21.15.172:0/325547950 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5607832d7760 0x5607832d9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.201+0000 7f81111e4740 4 client.17661 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:53:36.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.201+0000 7f81111e4740 1 -- 172.21.15.172:0/325547950 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560783227860 con 0x5607832d7760 2024-09-10T08:53:36.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.201+0000 7f810cba7640 1 --2- 172.21.15.172:0/325547950 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5607832d7760 0x5607832d9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.202+0000 7f80e7fff640 10 client.17661 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:53:36.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.203+0000 7f810cba7640 1 --2- 172.21.15.172:0/325547950 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5607832d7760 0x5607832d9b50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=457a1d12122deb11 server_cookie=dadd497700929263 in_seq=0 out_seq=0 2024-09-10T08:53:36.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.203+0000 7f80e7fff640 1 -- 172.21.15.172:0/325547950 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560783227860 con 0x5607832d7760 2024-09-10T08:53:36.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.203+0000 7f80e7fff640 10 client.17661 handle_command_reply: tid=0 2024-09-10T08:53:36.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.203+0000 7f81111e4740 1 client.17661 shutdown 2024-09-10T08:53:36.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.203+0000 7f81111e4740 1 -- 172.21.15.172:0/325547950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b5b40 msgr2=0x5607832bc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:36.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.203+0000 7f81111e4740 1 --2- 172.21.15.172:0/325547950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b5b40 0x5607832bc080 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f80f401c040 tx=0x7f80f40041e0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.204+0000 7f81111e4740 1 -- 172.21.15.172:0/325547950 shutdown_connections 2024-09-10T08:53:36.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.204+0000 7f81111e4740 1 --2- 172.21.15.172:0/325547950 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5607832d7760 0x5607832d9b50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.204+0000 7f80e7fff640 0 client.17661 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:53:36.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.204+0000 7f81111e4740 1 --2- 172.21.15.172:0/325547950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832bf370 0x5607832bc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.204+0000 7f81111e4740 1 --2- 172.21.15.172:0/325547950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b5b40 0x5607832bc080 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.204+0000 7f81111e4740 1 --2- 172.21.15.172:0/325547950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832b45d0 0x5607832beb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.204+0000 7f81111e4740 1 -- 172.21.15.172:0/325547950 >> 172.21.15.172:0/325547950 conn(0x56078327d310 msgr2=0x5607832b6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:36.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.204+0000 7f81111e4740 1 -- 172.21.15.172:0/325547950 shutdown_connections 2024-09-10T08:53:36.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.204+0000 7f81111e4740 1 -- 172.21.15.172:0/325547950 wait complete. 2024-09-10T08:53:36.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.204+0000 7f81111e4740 20 client.17661 trim_cache size 0 max 16384 2024-09-10T08:53:36.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.223+0000 7f81111e4740 1 Processor -- start 2024-09-10T08:53:36.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.223+0000 7f81111e4740 1 -- start start 2024-09-10T08:53:36.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.223+0000 7f81111e4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832af500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.223+0000 7f81111e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832b5b40 0x5607832be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.223+0000 7f81111e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832bf370 0x5607832df2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.223+0000 7f81111e4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81081ac7a0 con 0x5607832b45d0 2024-09-10T08:53:36.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.223+0000 7f81111e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81081ae2e0 con 0x5607832bf370 2024-09-10T08:53:36.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.223+0000 7f81111e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f81081ace70 con 0x5607832b5b40 2024-09-10T08:53:36.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.223+0000 7f810cba7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.223+0000 7f810cba7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832af500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35324/0 (socket says 172.21.15.172:35324) 2024-09-10T08:53:36.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.223+0000 7f810cba7640 1 -- 172.21.15.172:0/249216841 learned_addr learned my addr 172.21.15.172:0/249216841 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:36.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.223+0000 7f8107fff640 1 --2- 172.21.15.172:0/249216841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832b5b40 0x5607832be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.223+0000 7f810d3a8640 1 --2- 172.21.15.172:0/249216841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832bf370 0x5607832df2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.224+0000 7f810cba7640 1 -- 172.21.15.172:0/249216841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832b5b40 msgr2=0x5607832be8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:36.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.224+0000 7f810cba7640 1 --2- 172.21.15.172:0/249216841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832b5b40 0x5607832be8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.224+0000 7f810cba7640 1 -- 172.21.15.172:0/249216841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832bf370 msgr2=0x5607832df2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:36.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.224+0000 7f810cba7640 1 --2- 172.21.15.172:0/249216841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832bf370 0x5607832df2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.224+0000 7f810cba7640 1 -- 172.21.15.172:0/249216841 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607832b1960 con 0x5607832b45d0 2024-09-10T08:53:36.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.224+0000 7f8107fff640 1 --2- 172.21.15.172:0/249216841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832b5b40 0x5607832be8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:36.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.224+0000 7f810cba7640 1 --2- 172.21.15.172:0/249216841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832af500 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f80fc01f5c0 tx=0x7f80fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d82e7308440343 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:36.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.224+0000 7f80f27fc640 1 -- 172.21.15.172:0/249216841 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc005220 con 0x5607832b45d0 2024-09-10T08:53:36.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.224+0000 7f80f27fc640 1 -- 172.21.15.172:0/249216841 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80fc004c50 con 0x5607832b45d0 2024-09-10T08:53:36.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.224+0000 7f80f27fc640 1 -- 172.21.15.172:0/249216841 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc01e9a0 con 0x5607832b45d0 2024-09-10T08:53:36.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.225+0000 7f81111e4740 1 -- 172.21.15.172:0/249216841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 msgr2=0x5607832af500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:36.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.225+0000 7f81111e4740 1 --2- 172.21.15.172:0/249216841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832af500 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f80fc01f5c0 tx=0x7f80fc014000 comp rx=0 tx=0).stop 2024-09-10T08:53:36.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.225+0000 7f81111e4740 1 -- 172.21.15.172:0/249216841 shutdown_connections 2024-09-10T08:53:36.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.225+0000 7f81111e4740 1 --2- 172.21.15.172:0/249216841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832bf370 0x5607832df2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.225+0000 7f81111e4740 1 --2- 172.21.15.172:0/249216841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832b5b40 0x5607832be8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.225+0000 7f81111e4740 1 --2- 172.21.15.172:0/249216841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832af500 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.225+0000 7f81111e4740 1 -- 172.21.15.172:0/249216841 >> 172.21.15.172:0/249216841 conn(0x56078327d310 msgr2=0x5607832b6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:36.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.225+0000 7f81111e4740 1 -- 172.21.15.172:0/249216841 shutdown_connections 2024-09-10T08:53:36.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.225+0000 7f81111e4740 1 -- 172.21.15.172:0/249216841 wait complete. 2024-09-10T08:53:36.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.225+0000 7f81111e4740 1 -- start start 2024-09-10T08:53:36.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.225+0000 7f81111e4740 1 Processor -- start 2024-09-10T08:53:36.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.226+0000 7f81111e4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.226+0000 7f81111e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832b5b40 0x5607832b8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.226+0000 7f81111e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832bf370 0x5607832b9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.226+0000 7f81111e4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81081adc10 con 0x5607832b45d0 2024-09-10T08:53:36.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.226+0000 7f81111e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81081ad540 con 0x5607832bf370 2024-09-10T08:53:36.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.226+0000 7f81111e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f81081aa780 con 0x5607832b5b40 2024-09-10T08:53:36.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.226+0000 7f81111e4740 10 client.0 _ll_register_callbacks cb 0x7f80f401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:36.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.226+0000 7f810cba7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.226+0000 7f810cba7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35334/0 (socket says 172.21.15.172:35334) 2024-09-10T08:53:36.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.226+0000 7f810cba7640 1 -- 172.21.15.172:0/1783750077 learned_addr learned my addr 172.21.15.172:0/1783750077 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:36.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.226+0000 7f810d3a8640 1 --2- 172.21.15.172:0/1783750077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832bf370 0x5607832b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.226+0000 7f8107fff640 1 --2- 172.21.15.172:0/1783750077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832b5b40 0x5607832b8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.226+0000 7f810cba7640 1 -- 172.21.15.172:0/1783750077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832b5b40 msgr2=0x5607832b8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:36.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.226+0000 7f810cba7640 1 --2- 172.21.15.172:0/1783750077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832b5b40 0x5607832b8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.226+0000 7f810cba7640 1 -- 172.21.15.172:0/1783750077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832bf370 msgr2=0x5607832b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:36.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.226+0000 7f810cba7640 1 --2- 172.21.15.172:0/1783750077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832bf370 0x5607832b9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.227+0000 7f810cba7640 1 -- 172.21.15.172:0/1783750077 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607832b9800 con 0x5607832b45d0 2024-09-10T08:53:36.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.227+0000 7f8107fff640 1 --2- 172.21.15.172:0/1783750077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832b5b40 0x5607832b8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:36.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.227+0000 7f810cba7640 1 --2- 172.21.15.172:0/1783750077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832d47f0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00a010 tx=0x7f80fc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d222bda7851af0f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:36.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.227+0000 7f80e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:36.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.227+0000 7f80e7fff640 1 -- 172.21.15.172:0/1783750077 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc036020 con 0x5607832b45d0 2024-09-10T08:53:36.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.227+0000 7f80e7fff640 1 -- 172.21.15.172:0/1783750077 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f80fc005000 con 0x5607832b45d0 2024-09-10T08:53:36.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.227+0000 7f81111e4740 1 -- 172.21.15.172:0/1783750077 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80fc004c50 con 0x5607832b45d0 2024-09-10T08:53:36.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.227+0000 7f80e7fff640 1 -- 172.21.15.172:0/1783750077 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f80fc041050 con 0x5607832b45d0 2024-09-10T08:53:36.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.227+0000 7f81111e4740 10 client.17667 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:36.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.227+0000 7f81111e4740 1 -- 172.21.15.172:0/1783750077 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560783342040 con 0x5607832b45d0 2024-09-10T08:53:36.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.227+0000 7f80e7fff640 1 -- 172.21.15.172:0/1783750077 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80fc01e620 con 0x5607832b45d0 2024-09-10T08:53:36.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.228+0000 7f80e7fff640 1 -- 172.21.15.172:0/1783750077 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f80fc03d4c0 con 0x5607832b45d0 2024-09-10T08:53:36.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.228+0000 7f81111e4740 10 client.17667 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:36.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.228+0000 7f81111e4740 10 client.17667 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:53:36.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.228+0000 7f81111e4740 20 client.17667 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:36.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.228+0000 7f81111e4740 1 --2- 172.21.15.172:0/1783750077 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5607832d7760 0x560783344710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:36.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.228+0000 7f81111e4740 4 client.17667 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:53:36.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.228+0000 7f81111e4740 1 -- 172.21.15.172:0/1783750077 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f80fc004c50 con 0x5607832d7760 2024-09-10T08:53:36.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.228+0000 7f8107fff640 1 --2- 172.21.15.172:0/1783750077 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5607832d7760 0x560783344710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:36.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.228+0000 7f8107fff640 1 --2- 172.21.15.172:0/1783750077 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5607832d7760 0x560783344710 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c3a467a7aea741c server_cookie=8b30a8f1228682fb in_seq=0 out_seq=0 2024-09-10T08:53:36.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.228+0000 7f80e7fff640 10 client.17667 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:53:36.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.228+0000 7f80e7fff640 1 -- 172.21.15.172:0/1783750077 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f80fc004c50 con 0x5607832d7760 2024-09-10T08:53:36.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.228+0000 7f80e7fff640 10 client.17667 handle_command_reply: tid=0 2024-09-10T08:53:36.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.228+0000 7f81111e4740 1 client.17667 shutdown 2024-09-10T08:53:36.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.229+0000 7f81111e4740 1 -- 172.21.15.172:0/1783750077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 msgr2=0x5607832d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:36.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.229+0000 7f81111e4740 1 --2- 172.21.15.172:0/1783750077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832d47f0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00a010 tx=0x7f80fc004740 comp rx=0 tx=0).stop 2024-09-10T08:53:36.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.229+0000 7f81111e4740 1 -- 172.21.15.172:0/1783750077 shutdown_connections 2024-09-10T08:53:36.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.229+0000 7f81111e4740 1 --2- 172.21.15.172:0/1783750077 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5607832d7760 0x560783344710 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.229+0000 7f81111e4740 1 --2- 172.21.15.172:0/1783750077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607832bf370 0x5607832b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.229+0000 7f81111e4740 1 --2- 172.21.15.172:0/1783750077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607832b5b40 0x5607832b8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.229+0000 7f80e7fff640 0 client.17667 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:53:36.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.229+0000 7f81111e4740 1 --2- 172.21.15.172:0/1783750077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5607832b45d0 0x5607832d47f0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.229+0000 7f81111e4740 1 -- 172.21.15.172:0/1783750077 >> 172.21.15.172:0/1783750077 conn(0x56078327d310 msgr2=0x5607832b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:36.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.229+0000 7f81111e4740 1 -- 172.21.15.172:0/1783750077 shutdown_connections 2024-09-10T08:53:36.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.229+0000 7f81111e4740 1 -- 172.21.15.172:0/1783750077 wait complete. 2024-09-10T08:53:36.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.229+0000 7f81111e4740 20 client.17667 trim_cache size 0 max 16384 2024-09-10T08:53:36.236 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:53:36.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.230+0000 7f810ee0b640 1 -- 172.21.15.172:0/1170161103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80f80a2c00 msgr2=0x7f80f80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:36.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.230+0000 7f810ee0b640 1 --2- 172.21.15.172:0/1170161103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80f80a2c00 0x7f80f80a50c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009bd0 tx=0x7f80fc005f90 comp rx=0 tx=0).stop 2024-09-10T08:53:36.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.230+0000 7f810ee0b640 1 -- 172.21.15.172:0/1170161103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8108102ef0 msgr2=0x7f810819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:36.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.230+0000 7f810ee0b640 1 --2- 172.21.15.172:0/1170161103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8108102ef0 0x7f810819cc70 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f8100004af0 tx=0x7f8100031f70 comp rx=0 tx=0).stop 2024-09-10T08:53:36.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.230+0000 7f810ee0b640 1 -- 172.21.15.172:0/1170161103 shutdown_connections 2024-09-10T08:53:36.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.230+0000 7f810ee0b640 1 --2- 172.21.15.172:0/1170161103 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f80f80a2c00 0x7f80f80a50c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.230+0000 7f810ee0b640 1 --2- 172.21.15.172:0/1170161103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8108102ef0 0x7f810819cc70 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.230+0000 7f810ee0b640 1 --2- 172.21.15.172:0/1170161103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8108102570 0x7f81081988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.230+0000 7f810ee0b640 1 --2- 172.21.15.172:0/1170161103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8108100e20 0x7f81081983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:36.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.230+0000 7f810ee0b640 1 -- 172.21.15.172:0/1170161103 >> 172.21.15.172:0/1170161103 conn(0x7f81080fc7d0 msgr2=0x7f8108069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:36.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.230+0000 7f810ee0b640 1 -- 172.21.15.172:0/1170161103 shutdown_connections 2024-09-10T08:53:36.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:36.230+0000 7f810ee0b640 1 -- 172.21.15.172:0/1170161103 wait complete. 2024-09-10T08:53:36.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:36 smithi172 ceph-mon[30854]: pgmap v571: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 19 op/s 2024-09-10T08:53:36.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:36 smithi174 ceph-mon[34068]: pgmap v571: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 19 op/s 2024-09-10T08:53:36.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:36 smithi191 ceph-mon[35314]: pgmap v571: 129 pgs: 129 active+clean; 5.8 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 19 op/s 2024-09-10T08:53:36.974 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:53:37.259 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:53:37.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.808+0000 7f314f6a0640 1 -- 172.21.15.172:0/2609726108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3148101400 msgr2=0x7f314810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:37.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.808+0000 7f314f6a0640 1 --2- 172.21.15.172:0/2609726108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3148101400 0x7f314810d930 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f3140009cd0 tx=0x7f314002a8e0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.808+0000 7f314f6a0640 1 -- 172.21.15.172:0/2609726108 shutdown_connections 2024-09-10T08:53:37.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.808+0000 7f314f6a0640 1 --2- 172.21.15.172:0/2609726108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3148101400 0x7f314810d930 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.808+0000 7f314f6a0640 1 --2- 172.21.15.172:0/2609726108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3148100a60 0x7f3148100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.808+0000 7f314f6a0640 1 --2- 172.21.15.172:0/2609726108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3148106a60 0x7f3148106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.808+0000 7f314f6a0640 1 -- 172.21.15.172:0/2609726108 >> 172.21.15.172:0/2609726108 conn(0x7f31480fc780 msgr2=0x7f31480feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:37.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.808+0000 7f314f6a0640 1 -- 172.21.15.172:0/2609726108 shutdown_connections 2024-09-10T08:53:37.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.808+0000 7f314f6a0640 1 -- 172.21.15.172:0/2609726108 wait complete. 2024-09-10T08:53:37.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.809+0000 7f314f6a0640 1 Processor -- start 2024-09-10T08:53:37.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.809+0000 7f314f6a0640 1 -- start start 2024-09-10T08:53:37.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.809+0000 7f314f6a0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3148100a60 0x7f3148198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.809+0000 7f314f6a0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3148101400 0x7f31481988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.809+0000 7f314f6a0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3148106a60 0x7f314819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.809+0000 7f314f6a0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3148110d10 con 0x7f3148106a60 2024-09-10T08:53:37.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.809+0000 7f314f6a0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3148110b90 con 0x7f3148100a60 2024-09-10T08:53:37.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.809+0000 7f314f6a0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3148110e90 con 0x7f3148101400 2024-09-10T08:53:37.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.809+0000 7f314dc3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3148106a60 0x7f314819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.809+0000 7f314dc3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3148106a60 0x7f314819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35352/0 (socket says 172.21.15.172:35352) 2024-09-10T08:53:37.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.809+0000 7f314dc3d640 1 -- 172.21.15.172:0/3026479028 learned_addr learned my addr 172.21.15.172:0/3026479028 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:37.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.809+0000 7f314cc3b640 1 --2- 172.21.15.172:0/3026479028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3148101400 0x7f31481988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.809+0000 7f314d43c640 1 --2- 172.21.15.172:0/3026479028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3148100a60 0x7f3148198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.809+0000 7f314dc3d640 1 -- 172.21.15.172:0/3026479028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3148101400 msgr2=0x7f31481988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:37.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.809+0000 7f314dc3d640 1 --2- 172.21.15.172:0/3026479028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3148101400 0x7f31481988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.809+0000 7f314dc3d640 1 -- 172.21.15.172:0/3026479028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3148100a60 msgr2=0x7f3148198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:37.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.809+0000 7f314dc3d640 1 --2- 172.21.15.172:0/3026479028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3148100a60 0x7f3148198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.810+0000 7f314dc3d640 1 -- 172.21.15.172:0/3026479028 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314819d330 con 0x7f3148106a60 2024-09-10T08:53:37.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.810+0000 7f314cc3b640 1 --2- 172.21.15.172:0/3026479028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3148101400 0x7f31481988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:37.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.810+0000 7f314dc3d640 1 --2- 172.21.15.172:0/3026479028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3148106a60 0x7f314819cc50 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f3140033040 tx=0x7f3140003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:37.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.810+0000 7f313e7fc640 1 -- 172.21.15.172:0/3026479028 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140004040 con 0x7f3148106a60 2024-09-10T08:53:37.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.810+0000 7f313e7fc640 1 -- 172.21.15.172:0/3026479028 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3140004690 con 0x7f3148106a60 2024-09-10T08:53:37.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.810+0000 7f313e7fc640 1 -- 172.21.15.172:0/3026479028 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140030e10 con 0x7f3148106a60 2024-09-10T08:53:37.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.810+0000 7f314f6a0640 1 -- 172.21.15.172:0/3026479028 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314819d5c0 con 0x7f3148106a60 2024-09-10T08:53:37.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.810+0000 7f314f6a0640 1 -- 172.21.15.172:0/3026479028 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31480fcd90 con 0x7f3148106a60 2024-09-10T08:53:37.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.813+0000 7f313e7fc640 1 -- 172.21.15.172:0/3026479028 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f31400305a0 con 0x7f3148106a60 2024-09-10T08:53:37.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.813+0000 7f313e7fc640 1 --2- 172.21.15.172:0/3026479028 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f31340a7010 0x7f31340a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.813+0000 7f314d43c640 1 --2- 172.21.15.172:0/3026479028 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f31340a7010 0x7f31340a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.813+0000 7f313e7fc640 1 -- 172.21.15.172:0/3026479028 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f314002e030 con 0x7f3148106a60 2024-09-10T08:53:37.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.814+0000 7f314d43c640 1 --2- 172.21.15.172:0/3026479028 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f31340a7010 0x7f31340a94d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3144009ac0 tx=0x7f3144005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:37.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.814+0000 7f3151a79740 1 Processor -- start 2024-09-10T08:53:37.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.814+0000 7f3151a79740 1 -- start start 2024-09-10T08:53:37.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.814+0000 7f3151a79740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd3fb40 0x55e9cfd3e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.814+0000 7f3151a79740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd3ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.814+0000 7f3151a79740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd49370 0x55e9cfd4b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.814+0000 7f3151a79740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e9cfca42f0 con 0x55e9cfd49370 2024-09-10T08:53:37.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.814+0000 7f3151a79740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e9cfc4bcf0 con 0x55e9cfd3fb40 2024-09-10T08:53:37.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.814+0000 7f3151a79740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e9cfca02b0 con 0x55e9cfd3e5d0 2024-09-10T08:53:37.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.815+0000 7f314dc3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd49370 0x55e9cfd4b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.815+0000 7f314dc3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd49370 0x55e9cfd4b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35366/0 (socket says 172.21.15.172:35366) 2024-09-10T08:53:37.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.815+0000 7f314dc3d640 1 -- 172.21.15.172:0/587898611 learned_addr learned my addr 172.21.15.172:0/587898611 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:37.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.815+0000 7f314d43c640 1 --2- 172.21.15.172:0/587898611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd3fb40 0x55e9cfd3e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.815+0000 7f314cc3b640 1 --2- 172.21.15.172:0/587898611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd3ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.815+0000 7f314dc3d640 1 -- 172.21.15.172:0/587898611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd3e5d0 msgr2=0x55e9cfd3ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:37.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.815+0000 7f314dc3d640 1 --2- 172.21.15.172:0/587898611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd3ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.815+0000 7f314dc3d640 1 -- 172.21.15.172:0/587898611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd3fb40 msgr2=0x55e9cfd3e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:37.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.815+0000 7f314dc3d640 1 --2- 172.21.15.172:0/587898611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd3fb40 0x55e9cfd3e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.815+0000 7f314dc3d640 1 -- 172.21.15.172:0/587898611 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9cfc303a0 con 0x55e9cfd49370 2024-09-10T08:53:37.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.815+0000 7f314cc3b640 1 --2- 172.21.15.172:0/587898611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd3ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:37.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.815+0000 7f314dc3d640 1 --2- 172.21.15.172:0/587898611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd49370 0x55e9cfd4b760 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f31400b91c0 tx=0x7f3140031e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f75ac38ef5afe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:37.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.815+0000 7f31337fe640 1 -- 172.21.15.172:0/587898611 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31400bd460 con 0x55e9cfd49370 2024-09-10T08:53:37.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.815+0000 7f31337fe640 1 -- 172.21.15.172:0/587898611 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3140036020 con 0x55e9cfd49370 2024-09-10T08:53:37.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.815+0000 7f31337fe640 1 -- 172.21.15.172:0/587898611 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140004b40 con 0x55e9cfd49370 2024-09-10T08:53:37.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.816+0000 7f3151a79740 1 -- 172.21.15.172:0/587898611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd49370 msgr2=0x55e9cfd4b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:37.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.816+0000 7f3151a79740 1 --2- 172.21.15.172:0/587898611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd49370 0x55e9cfd4b760 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f31400b91c0 tx=0x7f3140031e50 comp rx=0 tx=0).stop 2024-09-10T08:53:37.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.816+0000 7f3151a79740 1 -- 172.21.15.172:0/587898611 shutdown_connections 2024-09-10T08:53:37.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.816+0000 7f3151a79740 1 --2- 172.21.15.172:0/587898611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd49370 0x55e9cfd4b760 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.816+0000 7f3151a79740 1 --2- 172.21.15.172:0/587898611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd3ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.816+0000 7f3151a79740 1 --2- 172.21.15.172:0/587898611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd3fb40 0x55e9cfd3e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.816+0000 7f3151a79740 1 -- 172.21.15.172:0/587898611 >> 172.21.15.172:0/587898611 conn(0x55e9cfd07310 msgr2=0x55e9cfd3ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:37.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.816+0000 7f3151a79740 1 -- 172.21.15.172:0/587898611 shutdown_connections 2024-09-10T08:53:37.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.816+0000 7f3151a79740 1 -- 172.21.15.172:0/587898611 wait complete. 2024-09-10T08:53:37.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.817+0000 7f3151a79740 1 -- start start 2024-09-10T08:53:37.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.817+0000 7f3151a79740 1 Processor -- start 2024-09-10T08:53:37.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.817+0000 7f3151a79740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd48e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.817+0000 7f3151a79740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd3fb40 0x55e9cfd460d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.817+0000 7f3151a79740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd49370 0x55e9cfd46860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.817+0000 7f3151a79740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e9cfd4be30 con 0x55e9cfd3e5d0 2024-09-10T08:53:37.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.817+0000 7f3151a79740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e9cfc5f060 con 0x55e9cfd3fb40 2024-09-10T08:53:37.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.817+0000 7f3151a79740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e9cfd4bfb0 con 0x55e9cfd49370 2024-09-10T08:53:37.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.817+0000 7f3151a79740 10 client.0 _ll_register_callbacks cb 0x55e9cf9b28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:37.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.818+0000 7f314d43c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd48e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.818+0000 7f314dc3d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd49370 0x55e9cfd46860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.818+0000 7f314dc3d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd49370 0x55e9cfd46860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33574/0 (socket says 172.21.15.172:33574) 2024-09-10T08:53:37.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.818+0000 7f314dc3d640 1 -- 172.21.15.172:0/3494089438 learned_addr learned my addr 172.21.15.172:0/3494089438 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:37.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.818+0000 7f314d43c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd48e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35376/0 (socket says 172.21.15.172:35376) 2024-09-10T08:53:37.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.818+0000 7f314cc3b640 1 --2- 172.21.15.172:0/3494089438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd3fb40 0x55e9cfd460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.818+0000 7f314dc3d640 1 -- 172.21.15.172:0/3494089438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd3fb40 msgr2=0x55e9cfd460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:37.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.818+0000 7f314dc3d640 1 --2- 172.21.15.172:0/3494089438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd3fb40 0x55e9cfd460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.818+0000 7f314dc3d640 1 -- 172.21.15.172:0/3494089438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd3e5d0 msgr2=0x55e9cfd48e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:37.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.818+0000 7f314dc3d640 1 --2- 172.21.15.172:0/3494089438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd48e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.818+0000 7f314dc3d640 1 -- 172.21.15.172:0/3494089438 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9cfa153c0 con 0x55e9cfd49370 2024-09-10T08:53:37.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.818+0000 7f314d43c640 1 --2- 172.21.15.172:0/3494089438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd48e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:37.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.818+0000 7f314dc3d640 1 --2- 172.21.15.172:0/3494089438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd49370 0x55e9cfd46860 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f31400308f0 tx=0x7f31400b9090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=317211ae2a05dc50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:37.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.818+0000 7f3130ff9640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:53:37.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.818+0000 7f3130ff9640 1 -- 172.21.15.172:0/3494089438 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314009f020 con 0x55e9cfd49370 2024-09-10T08:53:37.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.818+0000 7f3130ff9640 1 -- 172.21.15.172:0/3494089438 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f31400041e0 con 0x55e9cfd49370 2024-09-10T08:53:37.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.818+0000 7f3151a79740 1 -- 172.21.15.172:0/3494089438 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3140036020 con 0x55e9cfd49370 2024-09-10T08:53:37.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.819+0000 7f3130ff9640 1 -- 172.21.15.172:0/3494089438 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f31400ca050 con 0x55e9cfd49370 2024-09-10T08:53:37.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.819+0000 7f3151a79740 10 client.26680 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:37.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.819+0000 7f3151a79740 1 -- 172.21.15.172:0/3494089438 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e9cfd4be30 con 0x55e9cfd49370 2024-09-10T08:53:37.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.819+0000 7f3130ff9640 1 -- 172.21.15.172:0/3494089438 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3140036020 con 0x55e9cfd49370 2024-09-10T08:53:37.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.819+0000 7f3130ff9640 1 -- 172.21.15.172:0/3494089438 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f31400a7050 con 0x55e9cfd49370 2024-09-10T08:53:37.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.819+0000 7f3151a79740 10 client.26680 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:37.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.819+0000 7f3151a79740 10 client.26680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:37.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.819+0000 7f3151a79740 20 client.26680 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:37.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.819+0000 7f3151a79740 1 --2- 172.21.15.172:0/3494089438 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e9cfd58d50 0x55e9cfd5b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.819+0000 7f3151a79740 4 client.26680 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:53:37.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.819+0000 7f3151a79740 1 -- 172.21.15.172:0/3494089438 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9cfcb1860 con 0x55e9cfd58d50 2024-09-10T08:53:37.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.820+0000 7f314d43c640 1 --2- 172.21.15.172:0/3494089438 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e9cfd58d50 0x55e9cfd5b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.821+0000 7f3130ff9640 10 client.26680 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:37.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.821+0000 7f314d43c640 1 --2- 172.21.15.172:0/3494089438 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e9cfd58d50 0x55e9cfd5b170 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56b7da42a5d6bfeb server_cookie=c3605e0b49fe2913 in_seq=0 out_seq=0 2024-09-10T08:53:37.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.822+0000 7f3130ff9640 1 -- 172.21.15.172:0/3494089438 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e9cfcb1860 con 0x55e9cfd58d50 2024-09-10T08:53:37.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.822+0000 7f3130ff9640 10 client.26680 handle_command_reply: tid=0 2024-09-10T08:53:37.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.822+0000 7f3151a79740 1 client.26680 shutdown 2024-09-10T08:53:37.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.822+0000 7f3151a79740 1 -- 172.21.15.172:0/3494089438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd49370 msgr2=0x55e9cfd46860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:37.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.822+0000 7f3151a79740 1 --2- 172.21.15.172:0/3494089438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd49370 0x55e9cfd46860 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f31400308f0 tx=0x7f31400b9090 comp rx=0 tx=0).stop 2024-09-10T08:53:37.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.822+0000 7f3151a79740 1 -- 172.21.15.172:0/3494089438 shutdown_connections 2024-09-10T08:53:37.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.822+0000 7f3151a79740 1 --2- 172.21.15.172:0/3494089438 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e9cfd58d50 0x55e9cfd5b170 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.822+0000 7f3151a79740 1 --2- 172.21.15.172:0/3494089438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd49370 0x55e9cfd46860 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.822+0000 7f3130ff9640 0 client.26680 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:37.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.822+0000 7f3151a79740 1 --2- 172.21.15.172:0/3494089438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd3fb40 0x55e9cfd460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.822+0000 7f3151a79740 1 --2- 172.21.15.172:0/3494089438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd48e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.822+0000 7f3151a79740 1 -- 172.21.15.172:0/3494089438 >> 172.21.15.172:0/3494089438 conn(0x55e9cfd07310 msgr2=0x55e9cfd40990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:37.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.822+0000 7f3151a79740 1 -- 172.21.15.172:0/3494089438 shutdown_connections 2024-09-10T08:53:37.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.822+0000 7f3151a79740 1 -- 172.21.15.172:0/3494089438 wait complete. 2024-09-10T08:53:37.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.822+0000 7f3151a79740 20 client.26680 trim_cache size 0 max 16384 2024-09-10T08:53:37.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.835+0000 7f3151a79740 1 Processor -- start 2024-09-10T08:53:37.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.835+0000 7f3151a79740 1 -- start start 2024-09-10T08:53:37.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.835+0000 7f3151a79740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd45990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.836+0000 7f3151a79740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd3fb40 0x55e9cfd39c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.836+0000 7f3151a79740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd49370 0x55e9cfd60a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.836+0000 7f3151a79740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f31481ac910 con 0x55e9cfd49370 2024-09-10T08:53:37.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.836+0000 7f3151a79740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f31481ae450 con 0x55e9cfd3e5d0 2024-09-10T08:53:37.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.836+0000 7f3151a79740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f31481acfe0 con 0x55e9cfd3fb40 2024-09-10T08:53:37.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.836+0000 7f314d43c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd45990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.836+0000 7f314dc3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd49370 0x55e9cfd60a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.836+0000 7f314d43c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd45990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:51390/0 (socket says 172.21.15.172:51390) 2024-09-10T08:53:37.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.836+0000 7f314d43c640 1 -- 172.21.15.172:0/912526514 learned_addr learned my addr 172.21.15.172:0/912526514 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:37.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.836+0000 7f314cc3b640 1 --2- 172.21.15.172:0/912526514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd3fb40 0x55e9cfd39c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.836+0000 7f314dc3d640 1 -- 172.21.15.172:0/912526514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd3fb40 msgr2=0x55e9cfd39c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:37.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.836+0000 7f314dc3d640 1 --2- 172.21.15.172:0/912526514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd3fb40 0x55e9cfd39c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.836+0000 7f314dc3d640 1 -- 172.21.15.172:0/912526514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd3e5d0 msgr2=0x55e9cfd45990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:37.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.836+0000 7f314dc3d640 1 --2- 172.21.15.172:0/912526514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd45990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.836+0000 7f314dc3d640 1 -- 172.21.15.172:0/912526514 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9cfd5b7d0 con 0x55e9cfd49370 2024-09-10T08:53:37.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.836+0000 7f314d43c640 1 --2- 172.21.15.172:0/912526514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd45990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:37.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.836+0000 7f314dc3d640 1 --2- 172.21.15.172:0/912526514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd49370 0x55e9cfd60a60 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f3140005cc0 tx=0x7f3140009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1a706df12f367d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:37.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.837+0000 7f3132ffd640 1 -- 172.21.15.172:0/912526514 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314009f070 con 0x55e9cfd49370 2024-09-10T08:53:37.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.837+0000 7f3132ffd640 1 -- 172.21.15.172:0/912526514 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3140036070 con 0x55e9cfd49370 2024-09-10T08:53:37.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.837+0000 7f3132ffd640 1 -- 172.21.15.172:0/912526514 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31400db050 con 0x55e9cfd49370 2024-09-10T08:53:37.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.837+0000 7f3151a79740 1 -- 172.21.15.172:0/912526514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd49370 msgr2=0x55e9cfd60a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:37.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.837+0000 7f3151a79740 1 --2- 172.21.15.172:0/912526514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd49370 0x55e9cfd60a60 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f3140005cc0 tx=0x7f3140009960 comp rx=0 tx=0).stop 2024-09-10T08:53:37.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.838+0000 7f3151a79740 1 -- 172.21.15.172:0/912526514 shutdown_connections 2024-09-10T08:53:37.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.838+0000 7f3151a79740 1 --2- 172.21.15.172:0/912526514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd49370 0x55e9cfd60a60 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.838+0000 7f3151a79740 1 --2- 172.21.15.172:0/912526514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd3fb40 0x55e9cfd39c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.838+0000 7f3151a79740 1 --2- 172.21.15.172:0/912526514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd45990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.838+0000 7f3151a79740 1 -- 172.21.15.172:0/912526514 >> 172.21.15.172:0/912526514 conn(0x55e9cfd07310 msgr2=0x55e9cfd39500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.838+0000 7f3151a79740 1 -- 172.21.15.172:0/912526514 shutdown_connections 2024-09-10T08:53:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.838+0000 7f3151a79740 1 -- 172.21.15.172:0/912526514 wait complete. 2024-09-10T08:53:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.838+0000 7f3151a79740 1 -- start start 2024-09-10T08:53:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.838+0000 7f3151a79740 1 Processor -- start 2024-09-10T08:53:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.838+0000 7f3151a79740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd51300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.838+0000 7f3151a79740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd3fb40 0x55e9cfd51840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f3151a79740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd49370 0x55e9cfd428e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f3151a79740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f31481add80 con 0x55e9cfd3fb40 2024-09-10T08:53:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f3151a79740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f31481ad6b0 con 0x55e9cfd49370 2024-09-10T08:53:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f3151a79740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f31481aa8f0 con 0x55e9cfd3e5d0 2024-09-10T08:53:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f3151a79740 10 client.0 _ll_register_callbacks cb 0x7f31440097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f314cc3b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd3fb40 0x55e9cfd51840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f314d43c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd51300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f314cc3b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd3fb40 0x55e9cfd51840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35400/0 (socket says 172.21.15.172:35400) 2024-09-10T08:53:37.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f314cc3b640 1 -- 172.21.15.172:0/619335720 learned_addr learned my addr 172.21.15.172:0/619335720 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:37.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f314dc3d640 1 --2- 172.21.15.172:0/619335720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd49370 0x55e9cfd428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f314d43c640 1 -- 172.21.15.172:0/619335720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd49370 msgr2=0x55e9cfd428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:37.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f314d43c640 1 --2- 172.21.15.172:0/619335720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd49370 0x55e9cfd428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f314d43c640 1 -- 172.21.15.172:0/619335720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd3fb40 msgr2=0x55e9cfd51840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:37.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f314d43c640 1 --2- 172.21.15.172:0/619335720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd3fb40 0x55e9cfd51840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f314d43c640 1 -- 172.21.15.172:0/619335720 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9cfd55000 con 0x55e9cfd3e5d0 2024-09-10T08:53:37.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f314d43c640 1 --2- 172.21.15.172:0/619335720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd51300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55e9cfd5ba00 tx=0x7f3144004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fef24179a98c32ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:37.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f3130ff9640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:53:37.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f3130ff9640 1 -- 172.21.15.172:0/619335720 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144005220 con 0x55e9cfd3e5d0 2024-09-10T08:53:37.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f3151a79740 1 -- 172.21.15.172:0/619335720 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3140036070 con 0x55e9cfd3e5d0 2024-09-10T08:53:37.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.839+0000 7f3130ff9640 1 -- 172.21.15.172:0/619335720 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f31440053c0 con 0x55e9cfd3e5d0 2024-09-10T08:53:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.840+0000 7f3130ff9640 1 -- 172.21.15.172:0/619335720 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f31440055c0 con 0x55e9cfd3e5d0 2024-09-10T08:53:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.840+0000 7f3151a79740 10 client.26686 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.840+0000 7f3151a79740 1 -- 172.21.15.172:0/619335720 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e9cfdc3390 con 0x55e9cfd3e5d0 2024-09-10T08:53:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.840+0000 7f3130ff9640 1 -- 172.21.15.172:0/619335720 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3140036070 con 0x55e9cfd3e5d0 2024-09-10T08:53:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.840+0000 7f3130ff9640 1 -- 172.21.15.172:0/619335720 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f3144004c20 con 0x55e9cfd3e5d0 2024-09-10T08:53:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.840+0000 7f3151a79740 10 client.26686 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.840+0000 7f3151a79740 10 client.26686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.840+0000 7f3151a79740 20 client.26686 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.840+0000 7f3151a79740 1 --2- 172.21.15.172:0/619335720 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e9cfdc3af0 0x55e9cfdc5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.840+0000 7f3151a79740 4 client.26686 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:53:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.840+0000 7f3151a79740 1 -- 172.21.15.172:0/619335720 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9cfcabd70 con 0x55e9cfdc3af0 2024-09-10T08:53:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.840+0000 7f314cc3b640 1 --2- 172.21.15.172:0/619335720 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e9cfdc3af0 0x55e9cfdc5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:37.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.841+0000 7f314cc3b640 1 --2- 172.21.15.172:0/619335720 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e9cfdc3af0 0x55e9cfdc5f10 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45e91368fa4fa5a9 server_cookie=4354491517c42abb in_seq=0 out_seq=0 2024-09-10T08:53:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.841+0000 7f3130ff9640 10 client.26686 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.841+0000 7f3130ff9640 1 -- 172.21.15.172:0/619335720 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9cfcabd70 con 0x55e9cfdc3af0 2024-09-10T08:53:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.841+0000 7f3130ff9640 10 client.26686 handle_command_reply: tid=0 2024-09-10T08:53:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.841+0000 7f3151a79740 1 client.26686 shutdown 2024-09-10T08:53:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.841+0000 7f3151a79740 1 -- 172.21.15.172:0/619335720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd3e5d0 msgr2=0x55e9cfd51300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.841+0000 7f3151a79740 1 --2- 172.21.15.172:0/619335720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd51300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55e9cfd5ba00 tx=0x7f3144004730 comp rx=0 tx=0).stop 2024-09-10T08:53:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.841+0000 7f3151a79740 1 -- 172.21.15.172:0/619335720 shutdown_connections 2024-09-10T08:53:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.841+0000 7f3151a79740 1 --2- 172.21.15.172:0/619335720 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e9cfdc3af0 0x55e9cfdc5f10 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.841+0000 7f3151a79740 1 --2- 172.21.15.172:0/619335720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9cfd49370 0x55e9cfd428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.841+0000 7f3151a79740 1 --2- 172.21.15.172:0/619335720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9cfd3fb40 0x55e9cfd51840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.841+0000 7f3151a79740 1 --2- 172.21.15.172:0/619335720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9cfd3e5d0 0x55e9cfd51300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.841+0000 7f3130ff9640 0 client.26686 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.841+0000 7f3151a79740 1 -- 172.21.15.172:0/619335720 >> 172.21.15.172:0/619335720 conn(0x55e9cfd07310 msgr2=0x55e9cfd47930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:37.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.841+0000 7f3151a79740 1 -- 172.21.15.172:0/619335720 shutdown_connections 2024-09-10T08:53:37.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.841+0000 7f3151a79740 1 -- 172.21.15.172:0/619335720 wait complete. 2024-09-10T08:53:37.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.841+0000 7f3151a79740 20 client.26686 trim_cache size 0 max 16384 2024-09-10T08:53:37.848 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:53:37.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.842+0000 7f314f6a0640 1 -- 172.21.15.172:0/3026479028 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f31340a7010 msgr2=0x7f31340a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:37.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.842+0000 7f314f6a0640 1 --2- 172.21.15.172:0/3026479028 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f31340a7010 0x7f31340a94d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3144009ac0 tx=0x7f3144005f90 comp rx=0 tx=0).stop 2024-09-10T08:53:37.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.842+0000 7f314f6a0640 1 -- 172.21.15.172:0/3026479028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3148106a60 msgr2=0x7f314819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:37.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.842+0000 7f314f6a0640 1 --2- 172.21.15.172:0/3026479028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3148106a60 0x7f314819cc50 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f3140033040 tx=0x7f3140003970 comp rx=0 tx=0).stop 2024-09-10T08:53:37.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.842+0000 7f314f6a0640 1 -- 172.21.15.172:0/3026479028 shutdown_connections 2024-09-10T08:53:37.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.842+0000 7f314f6a0640 1 --2- 172.21.15.172:0/3026479028 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f31340a7010 0x7f31340a94d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.842+0000 7f314f6a0640 1 --2- 172.21.15.172:0/3026479028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3148106a60 0x7f314819cc50 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.842+0000 7f314f6a0640 1 --2- 172.21.15.172:0/3026479028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3148101400 0x7f31481988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.842+0000 7f314f6a0640 1 --2- 172.21.15.172:0/3026479028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3148100a60 0x7f3148198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:37.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.842+0000 7f314f6a0640 1 -- 172.21.15.172:0/3026479028 >> 172.21.15.172:0/3026479028 conn(0x7f31480fc780 msgr2=0x7f314806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:37.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.843+0000 7f314f6a0640 1 -- 172.21.15.172:0/3026479028 shutdown_connections 2024-09-10T08:53:37.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:37.843+0000 7f314f6a0640 1 -- 172.21.15.172:0/3026479028 wait complete. 2024-09-10T08:53:38.648 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:38 smithi172 ceph-mon[30854]: pgmap v572: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-09-10T08:53:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:38 smithi174 ceph-mon[34068]: pgmap v572: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-09-10T08:53:38.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:38 smithi191 ceph-mon[35314]: pgmap v572: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-09-10T08:53:39.676 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:53:39.677 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:53:39.986 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:53:40.515 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:40 smithi172 ceph-mon[30854]: pgmap v573: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-09-10T08:53:40.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.513+0000 7f5daebba640 1 -- 172.21.15.172:0/3394029107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5da81049c0 msgr2=0x7f5da8108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:40.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.513+0000 7f5daebba640 1 --2- 172.21.15.172:0/3394029107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5da81049c0 0x7f5da8108a10 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009900 tx=0x7f5d9402a410 comp rx=0 tx=0).stop 2024-09-10T08:53:40.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.513+0000 7f5daebba640 1 -- 172.21.15.172:0/3394029107 shutdown_connections 2024-09-10T08:53:40.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.513+0000 7f5daebba640 1 --2- 172.21.15.172:0/3394029107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da8109140 0x7f5da8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.513+0000 7f5daebba640 1 --2- 172.21.15.172:0/3394029107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5da81049c0 0x7f5da8108a10 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.513+0000 7f5daebba640 1 --2- 172.21.15.172:0/3394029107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5da8104010 0x7f5da81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.513+0000 7f5daebba640 1 -- 172.21.15.172:0/3394029107 >> 172.21.15.172:0/3394029107 conn(0x7f5da80ffa50 msgr2=0x7f5da8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:40.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.513+0000 7f5daebba640 1 -- 172.21.15.172:0/3394029107 shutdown_connections 2024-09-10T08:53:40.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.513+0000 7f5daebba640 1 -- 172.21.15.172:0/3394029107 wait complete. 2024-09-10T08:53:40.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.514+0000 7f5daebba640 1 Processor -- start 2024-09-10T08:53:40.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.514+0000 7f5daebba640 1 -- start start 2024-09-10T08:53:40.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.514+0000 7f5daebba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5da8104010 0x7f5da819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.514+0000 7f5daebba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da81049c0 0x7f5da819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.514+0000 7f5daebba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5da8109140 0x7f5da819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.529 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.515+0000 7f5daebba640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5da8114010 con 0x7f5da8104010 2024-09-10T08:53:40.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.515+0000 7f5daebba640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5da8113e90 con 0x7f5da81049c0 2024-09-10T08:53:40.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.515+0000 7f5daebba640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5da8114190 con 0x7f5da8109140 2024-09-10T08:53:40.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.515+0000 7f5dac956640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5da8104010 0x7f5da819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:40.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.515+0000 7f5dac956640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5da8104010 0x7f5da819a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35420/0 (socket says 172.21.15.172:35420) 2024-09-10T08:53:40.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.515+0000 7f5dac956640 1 -- 172.21.15.172:0/3435922450 learned_addr learned my addr 172.21.15.172:0/3435922450 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:40.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.515+0000 7f5da7fff640 1 --2- 172.21.15.172:0/3435922450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da81049c0 0x7f5da819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:40.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.515+0000 7f5dad157640 1 --2- 172.21.15.172:0/3435922450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5da8109140 0x7f5da819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:40.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.515+0000 7f5dac956640 1 -- 172.21.15.172:0/3435922450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5da8109140 msgr2=0x7f5da819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:40.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.515+0000 7f5dac956640 1 --2- 172.21.15.172:0/3435922450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5da8109140 0x7f5da819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.515+0000 7f5dac956640 1 -- 172.21.15.172:0/3435922450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da81049c0 msgr2=0x7f5da819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:40.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.515+0000 7f5dac956640 1 --2- 172.21.15.172:0/3435922450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da81049c0 0x7f5da819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.515+0000 7f5dac956640 1 -- 172.21.15.172:0/3435922450 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da819f520 con 0x7f5da8104010 2024-09-10T08:53:40.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.515+0000 7f5da7fff640 1 --2- 172.21.15.172:0/3435922450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da81049c0 0x7f5da819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:40.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.515+0000 7f5dac956640 1 --2- 172.21.15.172:0/3435922450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5da8104010 0x7f5da819a570 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00d770 tx=0x7f5d9c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:40.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.515+0000 7f5da5ffb640 1 -- 172.21.15.172:0/3435922450 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c017070 con 0x7f5da8104010 2024-09-10T08:53:40.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.516+0000 7f5daebba640 1 -- 172.21.15.172:0/3435922450 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da819f810 con 0x7f5da8104010 2024-09-10T08:53:40.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.516+0000 7f5da5ffb640 1 -- 172.21.15.172:0/3435922450 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d9c0054d0 con 0x7f5da8104010 2024-09-10T08:53:40.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.516+0000 7f5da5ffb640 1 -- 172.21.15.172:0/3435922450 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c004cf0 con 0x7f5da8104010 2024-09-10T08:53:40.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.516+0000 7f5daebba640 1 -- 172.21.15.172:0/3435922450 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da81a70f0 con 0x7f5da8104010 2024-09-10T08:53:40.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.519+0000 7f5da5ffb640 1 -- 172.21.15.172:0/3435922450 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5d9c004e90 con 0x7f5da8104010 2024-09-10T08:53:40.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.519+0000 7f5db0f93740 1 Processor -- start 2024-09-10T08:53:40.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.519+0000 7f5db0f93740 1 -- start start 2024-09-10T08:53:40.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.519+0000 7f5db0f93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba24b60 0x55d40ba24f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.519+0000 7f5db0f93740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba25480 0x55d40ba2f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.519+0000 7f5db0f93740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba2fd40 0x55d40ba32130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.519+0000 7f5db0f93740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d40b98a280 con 0x55d40ba2fd40 2024-09-10T08:53:40.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.519+0000 7f5db0f93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d40b931b10 con 0x55d40ba24b60 2024-09-10T08:53:40.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.519+0000 7f5db0f93740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d40b986240 con 0x55d40ba25480 2024-09-10T08:53:40.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.520+0000 7f5da7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba2fd40 0x55d40ba32130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:40.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.520+0000 7f5da7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba2fd40 0x55d40ba32130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35430/0 (socket says 172.21.15.172:35430) 2024-09-10T08:53:40.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.520+0000 7f5da7fff640 1 -- 172.21.15.172:0/463382098 learned_addr learned my addr 172.21.15.172:0/463382098 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:40.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.520+0000 7f5dac956640 1 --2- 172.21.15.172:0/463382098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba25480 0x55d40ba2f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:40.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.520+0000 7f5dad157640 1 --2- 172.21.15.172:0/463382098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba24b60 0x55d40ba24f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:40.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.520+0000 7f5da5ffb640 1 --2- 172.21.15.172:0/3435922450 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5d980a2c00 0x7f5d980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.520+0000 7f5da7fff640 1 -- 172.21.15.172:0/463382098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba25480 msgr2=0x55d40ba2f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:40.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.520+0000 7f5da7fff640 1 --2- 172.21.15.172:0/463382098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba25480 0x55d40ba2f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.520+0000 7f5da5ffb640 1 -- 172.21.15.172:0/3435922450 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5d9c0cedf0 con 0x7f5da8104010 2024-09-10T08:53:40.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.520+0000 7f5da7fff640 1 -- 172.21.15.172:0/463382098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba24b60 msgr2=0x55d40ba24f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:40.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.520+0000 7f5da7fff640 1 --2- 172.21.15.172:0/463382098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba24b60 0x55d40ba24f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.520+0000 7f5da7fff640 1 -- 172.21.15.172:0/463382098 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d40b93e1f0 con 0x55d40ba2fd40 2024-09-10T08:53:40.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.520+0000 7f5dad157640 1 --2- 172.21.15.172:0/463382098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba24b60 0x55d40ba24f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:40.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.520+0000 7f5dad157640 1 --2- 172.21.15.172:0/3435922450 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5d980a2c00 0x7f5d980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:40.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.520+0000 7f5da7fff640 1 --2- 172.21.15.172:0/463382098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba2fd40 0x55d40ba32130 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009900 tx=0x7f5d94032c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a596bd5c007b87a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:40.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.521+0000 7f5d92ffd640 1 -- 172.21.15.172:0/463382098 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9403f070 con 0x55d40ba2fd40 2024-09-10T08:53:40.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.521+0000 7f5d92ffd640 1 -- 172.21.15.172:0/463382098 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d94004030 con 0x55d40ba2fd40 2024-09-10T08:53:40.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.521+0000 7f5d92ffd640 1 -- 172.21.15.172:0/463382098 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94035070 con 0x55d40ba2fd40 2024-09-10T08:53:40.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.521+0000 7f5dad157640 1 --2- 172.21.15.172:0/3435922450 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5d980a2c00 0x7f5d980a50c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5da0005fd0 tx=0x7f5da0004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:40.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.521+0000 7f5db0f93740 1 -- 172.21.15.172:0/463382098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba2fd40 msgr2=0x55d40ba32130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:40.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.521+0000 7f5db0f93740 1 --2- 172.21.15.172:0/463382098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba2fd40 0x55d40ba32130 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009900 tx=0x7f5d94032c00 comp rx=0 tx=0).stop 2024-09-10T08:53:40.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.521+0000 7f5db0f93740 1 -- 172.21.15.172:0/463382098 shutdown_connections 2024-09-10T08:53:40.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.521+0000 7f5db0f93740 1 --2- 172.21.15.172:0/463382098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba2fd40 0x55d40ba32130 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.521+0000 7f5db0f93740 1 --2- 172.21.15.172:0/463382098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba25480 0x55d40ba2f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.521+0000 7f5db0f93740 1 --2- 172.21.15.172:0/463382098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba24b60 0x55d40ba24f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.521+0000 7f5db0f93740 1 -- 172.21.15.172:0/463382098 >> 172.21.15.172:0/463382098 conn(0x55d40ba210f0 msgr2=0x55d40ba214d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:40.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.521+0000 7f5db0f93740 1 -- 172.21.15.172:0/463382098 shutdown_connections 2024-09-10T08:53:40.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.522+0000 7f5db0f93740 1 -- 172.21.15.172:0/463382098 wait complete. 2024-09-10T08:53:40.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.522+0000 7f5db0f93740 1 -- start start 2024-09-10T08:53:40.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.522+0000 7f5db0f93740 1 Processor -- start 2024-09-10T08:53:40.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.522+0000 7f5db0f93740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba24b60 0x55d40ba2cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.522+0000 7f5db0f93740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba25480 0x55d40ba2d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.522+0000 7f5db0f93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba2fd40 0x55d40ba2d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.522+0000 7f5db0f93740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d40ba32910 con 0x55d40ba25480 2024-09-10T08:53:40.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.522+0000 7f5db0f93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d40b944f10 con 0x55d40ba2fd40 2024-09-10T08:53:40.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.522+0000 7f5db0f93740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d40ba32a90 con 0x55d40ba24b60 2024-09-10T08:53:40.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.522+0000 7f5db0f93740 10 client.0 _ll_register_callbacks cb 0x55d40b90da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:40.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.523+0000 7f5da7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba25480 0x55d40ba2d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:40.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.523+0000 7f5da7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba25480 0x55d40ba2d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35438/0 (socket says 172.21.15.172:35438) 2024-09-10T08:53:40.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.523+0000 7f5da7fff640 1 -- 172.21.15.172:0/1365437272 learned_addr learned my addr 172.21.15.172:0/1365437272 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:40.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.523+0000 7f5dad157640 1 --2- 172.21.15.172:0/1365437272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba2fd40 0x55d40ba2d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:40.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.523+0000 7f5dac956640 1 --2- 172.21.15.172:0/1365437272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba24b60 0x55d40ba2cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:40.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.523+0000 7f5da7fff640 1 -- 172.21.15.172:0/1365437272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba24b60 msgr2=0x55d40ba2cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:40.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.523+0000 7f5da7fff640 1 --2- 172.21.15.172:0/1365437272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba24b60 0x55d40ba2cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.523+0000 7f5da7fff640 1 -- 172.21.15.172:0/1365437272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba2fd40 msgr2=0x55d40ba2d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:40.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.523+0000 7f5da7fff640 1 --2- 172.21.15.172:0/1365437272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba2fd40 0x55d40ba2d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.523+0000 7f5da7fff640 1 -- 172.21.15.172:0/1365437272 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d40b6fb6a0 con 0x55d40ba25480 2024-09-10T08:53:40.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.523+0000 7f5dad157640 1 --2- 172.21.15.172:0/1365437272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba2fd40 0x55d40ba2d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:40.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.523+0000 7f5da7fff640 1 --2- 172.21.15.172:0/1365437272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba25480 0x55d40ba2d140 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f5d940052c0 tx=0x7f5d940041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf898a7edfa46b15 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:40.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.523+0000 7f5d87fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:40.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.523+0000 7f5d87fff640 1 -- 172.21.15.172:0/1365437272 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94035070 con 0x55d40ba25480 2024-09-10T08:53:40.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.523+0000 7f5d87fff640 1 -- 172.21.15.172:0/1365437272 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5d940316a0 con 0x55d40ba25480 2024-09-10T08:53:40.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.523+0000 7f5db0f93740 1 -- 172.21.15.172:0/1365437272 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d94004030 con 0x55d40ba25480 2024-09-10T08:53:40.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.524+0000 7f5d87fff640 1 -- 172.21.15.172:0/1365437272 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5d94055050 con 0x55d40ba25480 2024-09-10T08:53:40.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.524+0000 7f5db0f93740 10 client.17697 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:40.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.524+0000 7f5db0f93740 1 -- 172.21.15.172:0/1365437272 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d40ba3e4f0 con 0x55d40ba25480 2024-09-10T08:53:40.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.524+0000 7f5d87fff640 1 -- 172.21.15.172:0/1365437272 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d94004030 con 0x55d40ba25480 2024-09-10T08:53:40.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.524+0000 7f5d87fff640 1 -- 172.21.15.172:0/1365437272 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f5d94003710 con 0x55d40ba25480 2024-09-10T08:53:40.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.524+0000 7f5db0f93740 10 client.17697 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:40.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.524+0000 7f5db0f93740 10 client.17697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:40.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.524+0000 7f5db0f93740 20 client.17697 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:40.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.524+0000 7f5db0f93740 1 --2- 172.21.15.172:0/1365437272 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d40ba3fa90 0x55d40ba41ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.524+0000 7f5db0f93740 4 client.17697 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:53:40.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.524+0000 7f5db0f93740 1 -- 172.21.15.172:0/1365437272 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d40b9977f0 con 0x55d40ba3fa90 2024-09-10T08:53:40.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.525+0000 7f5dac956640 1 --2- 172.21.15.172:0/1365437272 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d40ba3fa90 0x55d40ba41ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:40.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.525+0000 7f5dac956640 1 --2- 172.21.15.172:0/1365437272 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d40ba3fa90 0x55d40ba41ec0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8f093ff714a476e server_cookie=45de3cb9eb520a08 in_seq=0 out_seq=0 2024-09-10T08:53:40.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.525+0000 7f5d87fff640 10 client.17697 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:40.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.526+0000 7f5d87fff640 1 -- 172.21.15.172:0/1365437272 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d40b9977f0 con 0x55d40ba3fa90 2024-09-10T08:53:40.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.526+0000 7f5d87fff640 10 client.17697 handle_command_reply: tid=0 2024-09-10T08:53:40.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.526+0000 7f5db0f93740 1 client.17697 shutdown 2024-09-10T08:53:40.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.526+0000 7f5db0f93740 1 -- 172.21.15.172:0/1365437272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba25480 msgr2=0x55d40ba2d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:40.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.526+0000 7f5db0f93740 1 --2- 172.21.15.172:0/1365437272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba25480 0x55d40ba2d140 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f5d940052c0 tx=0x7f5d940041e0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.526+0000 7f5db0f93740 1 -- 172.21.15.172:0/1365437272 shutdown_connections 2024-09-10T08:53:40.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.526+0000 7f5db0f93740 1 --2- 172.21.15.172:0/1365437272 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d40ba3fa90 0x55d40ba41ec0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.526+0000 7f5db0f93740 1 --2- 172.21.15.172:0/1365437272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba2fd40 0x55d40ba2d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.526+0000 7f5db0f93740 1 --2- 172.21.15.172:0/1365437272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba25480 0x55d40ba2d140 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.526+0000 7f5db0f93740 1 --2- 172.21.15.172:0/1365437272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba24b60 0x55d40ba2cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.526+0000 7f5db0f93740 1 -- 172.21.15.172:0/1365437272 >> 172.21.15.172:0/1365437272 conn(0x55d40ba210f0 msgr2=0x55d40ba26dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:40.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.526+0000 7f5d87fff640 0 client.17697 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:40.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.526+0000 7f5db0f93740 1 -- 172.21.15.172:0/1365437272 shutdown_connections 2024-09-10T08:53:40.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.526+0000 7f5db0f93740 1 -- 172.21.15.172:0/1365437272 wait complete. 2024-09-10T08:53:40.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.526+0000 7f5db0f93740 20 client.17697 trim_cache size 0 max 16384 2024-09-10T08:53:40.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.548+0000 7f5db0f93740 1 Processor -- start 2024-09-10T08:53:40.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.549+0000 7f5db0f93740 1 -- start start 2024-09-10T08:53:40.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.549+0000 7f5db0f93740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba24b60 0x55d40ba2ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.549+0000 7f5db0f93740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba25480 0x55d40ba47790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.549+0000 7f5db0f93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba2fd40 0x55d40ba47cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.549+0000 7f5db0f93740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81b0c10 con 0x55d40ba24b60 2024-09-10T08:53:40.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.549+0000 7f5db0f93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81b2750 con 0x55d40ba2fd40 2024-09-10T08:53:40.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.549+0000 7f5db0f93740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81b12e0 con 0x55d40ba25480 2024-09-10T08:53:40.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.549+0000 7f5dac956640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba24b60 0x55d40ba2ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:40.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.549+0000 7f5dac956640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba24b60 0x55d40ba2ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35450/0 (socket says 172.21.15.172:35450) 2024-09-10T08:53:40.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.549+0000 7f5dac956640 1 -- 172.21.15.172:0/2771572462 learned_addr learned my addr 172.21.15.172:0/2771572462 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:40.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.549+0000 7f5da7fff640 1 --2- 172.21.15.172:0/2771572462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba25480 0x55d40ba47790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:40.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.549+0000 7f5dad157640 1 --2- 172.21.15.172:0/2771572462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba2fd40 0x55d40ba47cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:40.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.550+0000 7f5dac956640 1 -- 172.21.15.172:0/2771572462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba25480 msgr2=0x55d40ba47790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:40.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.550+0000 7f5dac956640 1 --2- 172.21.15.172:0/2771572462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba25480 0x55d40ba47790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.550+0000 7f5dac956640 1 -- 172.21.15.172:0/2771572462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba2fd40 msgr2=0x55d40ba47cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:40.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.550+0000 7f5dac956640 1 --2- 172.21.15.172:0/2771572462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba2fd40 0x55d40ba47cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.550+0000 7f5dac956640 1 -- 172.21.15.172:0/2771572462 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d40ba3f840 con 0x55d40ba24b60 2024-09-10T08:53:40.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.550+0000 7f5da7fff640 1 --2- 172.21.15.172:0/2771572462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba25480 0x55d40ba47790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:40.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.550+0000 7f5dac956640 1 --2- 172.21.15.172:0/2771572462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba24b60 0x55d40ba2ea60 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c09b220 tx=0x7f5d9c004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8827e023c33bcf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:40.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.550+0000 7f5d927fc640 1 -- 172.21.15.172:0/2771572462 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c020040 con 0x55d40ba24b60 2024-09-10T08:53:40.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.550+0000 7f5d927fc640 1 -- 172.21.15.172:0/2771572462 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d9c09b990 con 0x55d40ba24b60 2024-09-10T08:53:40.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.550+0000 7f5d927fc640 1 -- 172.21.15.172:0/2771572462 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0cc640 con 0x55d40ba24b60 2024-09-10T08:53:40.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.551+0000 7f5db0f93740 1 -- 172.21.15.172:0/2771572462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba24b60 msgr2=0x55d40ba2ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:40.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.551+0000 7f5db0f93740 1 --2- 172.21.15.172:0/2771572462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba24b60 0x55d40ba2ea60 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c09b220 tx=0x7f5d9c004780 comp rx=0 tx=0).stop 2024-09-10T08:53:40.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.551+0000 7f5db0f93740 1 -- 172.21.15.172:0/2771572462 shutdown_connections 2024-09-10T08:53:40.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.551+0000 7f5db0f93740 1 --2- 172.21.15.172:0/2771572462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba2fd40 0x55d40ba47cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.551+0000 7f5db0f93740 1 --2- 172.21.15.172:0/2771572462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba25480 0x55d40ba47790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.551+0000 7f5db0f93740 1 --2- 172.21.15.172:0/2771572462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba24b60 0x55d40ba2ea60 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.551+0000 7f5db0f93740 1 -- 172.21.15.172:0/2771572462 >> 172.21.15.172:0/2771572462 conn(0x55d40ba210f0 msgr2=0x55d40ba2e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:40.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.551+0000 7f5db0f93740 1 -- 172.21.15.172:0/2771572462 shutdown_connections 2024-09-10T08:53:40.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.551+0000 7f5db0f93740 1 -- 172.21.15.172:0/2771572462 wait complete. 2024-09-10T08:53:40.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.552+0000 7f5db0f93740 1 -- start start 2024-09-10T08:53:40.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.552+0000 7f5db0f93740 1 Processor -- start 2024-09-10T08:53:40.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.552+0000 7f5db0f93740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba24b60 0x55d40ba37ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.552+0000 7f5db0f93740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba25480 0x55d40ba38220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.552+0000 7f5db0f93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba2fd40 0x55d40ba34cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.552+0000 7f5db0f93740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81b2080 con 0x55d40ba25480 2024-09-10T08:53:40.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.552+0000 7f5db0f93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81b19b0 con 0x55d40ba2fd40 2024-09-10T08:53:40.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.552+0000 7f5db0f93740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5da81aebf0 con 0x55d40ba24b60 2024-09-10T08:53:40.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.552+0000 7f5db0f93740 10 client.0 _ll_register_callbacks cb 0x7f5da0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:40.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.552+0000 7f5da7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba25480 0x55d40ba38220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:40.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.552+0000 7f5da7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba25480 0x55d40ba38220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35464/0 (socket says 172.21.15.172:35464) 2024-09-10T08:53:40.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.552+0000 7f5da7fff640 1 -- 172.21.15.172:0/1867399213 learned_addr learned my addr 172.21.15.172:0/1867399213 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:40.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.553+0000 7f5dad157640 1 --2- 172.21.15.172:0/1867399213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba2fd40 0x55d40ba34cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:40.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.553+0000 7f5da7fff640 1 -- 172.21.15.172:0/1867399213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba24b60 msgr2=0x55d40ba37ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:40.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.553+0000 7f5da7fff640 1 --2- 172.21.15.172:0/1867399213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba24b60 0x55d40ba37ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.553+0000 7f5da7fff640 1 -- 172.21.15.172:0/1867399213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba2fd40 msgr2=0x55d40ba34cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:40.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.553+0000 7f5da7fff640 1 --2- 172.21.15.172:0/1867399213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba2fd40 0x55d40ba34cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.553+0000 7f5da7fff640 1 -- 172.21.15.172:0/1867399213 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d40ba351f0 con 0x55d40ba25480 2024-09-10T08:53:40.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.553+0000 7f5dad157640 1 --2- 172.21.15.172:0/1867399213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba2fd40 0x55d40ba34cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:40.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.553+0000 7f5da7fff640 1 --2- 172.21.15.172:0/1867399213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba25480 0x55d40ba38220 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f5d94003ca0 tx=0x7f5d94003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1487a7895db23d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:40.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.553+0000 7f5d87fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:40.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.553+0000 7f5d87fff640 1 -- 172.21.15.172:0/1867399213 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9405e060 con 0x55d40ba25480 2024-09-10T08:53:40.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.553+0000 7f5d87fff640 1 -- 172.21.15.172:0/1867399213 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5d94032070 con 0x55d40ba25480 2024-09-10T08:53:40.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.553+0000 7f5db0f93740 1 -- 172.21.15.172:0/1867399213 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d9c09b990 con 0x55d40ba25480 2024-09-10T08:53:40.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.554+0000 7f5d87fff640 1 -- 172.21.15.172:0/1867399213 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5d94035070 con 0x55d40ba25480 2024-09-10T08:53:40.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.554+0000 7f5db0f93740 10 client.17703 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:40.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.554+0000 7f5db0f93740 1 -- 172.21.15.172:0/1867399213 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d40baaade0 con 0x55d40ba25480 2024-09-10T08:53:40.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.554+0000 7f5d87fff640 1 -- 172.21.15.172:0/1867399213 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d9c09b990 con 0x55d40ba25480 2024-09-10T08:53:40.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.554+0000 7f5d87fff640 1 -- 172.21.15.172:0/1867399213 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f5d94031600 con 0x55d40ba25480 2024-09-10T08:53:40.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.554+0000 7f5db0f93740 10 client.17703 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:40.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.554+0000 7f5db0f93740 10 client.17703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:40.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.554+0000 7f5db0f93740 20 client.17703 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:40.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.554+0000 7f5db0f93740 1 --2- 172.21.15.172:0/1867399213 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d40baab4a0 0x55d40baad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:40.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.554+0000 7f5db0f93740 4 client.17703 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:53:40.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.554+0000 7f5db0f93740 1 -- 172.21.15.172:0/1867399213 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d40b991d00 con 0x55d40baab4a0 2024-09-10T08:53:40.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.555+0000 7f5dac956640 1 --2- 172.21.15.172:0/1867399213 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d40baab4a0 0x55d40baad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:40.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.555+0000 7f5dac956640 1 --2- 172.21.15.172:0/1867399213 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d40baab4a0 0x55d40baad8c0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d7e12ab2a009efc server_cookie=e95ebaf5ef11aea0 in_seq=0 out_seq=0 2024-09-10T08:53:40.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.555+0000 7f5d87fff640 10 client.17703 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:40.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.556+0000 7f5d87fff640 1 -- 172.21.15.172:0/1867399213 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d40b991d00 con 0x55d40baab4a0 2024-09-10T08:53:40.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.556+0000 7f5d87fff640 10 client.17703 handle_command_reply: tid=0 2024-09-10T08:53:40.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.556+0000 7f5db0f93740 1 client.17703 shutdown 2024-09-10T08:53:40.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.556+0000 7f5db0f93740 1 -- 172.21.15.172:0/1867399213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba25480 msgr2=0x55d40ba38220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:40.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.556+0000 7f5db0f93740 1 --2- 172.21.15.172:0/1867399213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba25480 0x55d40ba38220 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f5d94003ca0 tx=0x7f5d94003920 comp rx=0 tx=0).stop 2024-09-10T08:53:40.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.557+0000 7f5db0f93740 1 -- 172.21.15.172:0/1867399213 shutdown_connections 2024-09-10T08:53:40.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.557+0000 7f5db0f93740 1 --2- 172.21.15.172:0/1867399213 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d40baab4a0 0x55d40baad8c0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.557+0000 7f5db0f93740 1 --2- 172.21.15.172:0/1867399213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40ba2fd40 0x55d40ba34cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.557+0000 7f5d87fff640 0 client.17703 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:40.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.557+0000 7f5db0f93740 1 --2- 172.21.15.172:0/1867399213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d40ba25480 0x55d40ba38220 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.557+0000 7f5db0f93740 1 --2- 172.21.15.172:0/1867399213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d40ba24b60 0x55d40ba37ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.557+0000 7f5db0f93740 1 -- 172.21.15.172:0/1867399213 >> 172.21.15.172:0/1867399213 conn(0x55d40ba210f0 msgr2=0x55d40ba2d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:40.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.557+0000 7f5db0f93740 1 -- 172.21.15.172:0/1867399213 shutdown_connections 2024-09-10T08:53:40.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.557+0000 7f5db0f93740 1 -- 172.21.15.172:0/1867399213 wait complete. 2024-09-10T08:53:40.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.557+0000 7f5db0f93740 20 client.17703 trim_cache size 0 max 16384 2024-09-10T08:53:40.562 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:53:40.562 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T08:53:40.562 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "704d45d1-f294-47d0-9e7a-2817dc8af480", 2024-09-10T08:53:40.562 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T08:53:40.562 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:53:40.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.558+0000 7f5daebba640 1 -- 172.21.15.172:0/3435922450 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5d980a2c00 msgr2=0x7f5d980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:40.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.558+0000 7f5daebba640 1 --2- 172.21.15.172:0/3435922450 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5d980a2c00 0x7f5d980a50c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5da0005fd0 tx=0x7f5da0004ab0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.558+0000 7f5daebba640 1 -- 172.21.15.172:0/3435922450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5da8104010 msgr2=0x7f5da819a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:40.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.558+0000 7f5daebba640 1 --2- 172.21.15.172:0/3435922450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5da8104010 0x7f5da819a570 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00d770 tx=0x7f5d9c00dc30 comp rx=0 tx=0).stop 2024-09-10T08:53:40.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.558+0000 7f5daebba640 1 -- 172.21.15.172:0/3435922450 shutdown_connections 2024-09-10T08:53:40.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.558+0000 7f5daebba640 1 --2- 172.21.15.172:0/3435922450 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5d980a2c00 0x7f5d980a50c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.558+0000 7f5daebba640 1 --2- 172.21.15.172:0/3435922450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5da8109140 0x7f5da819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.558+0000 7f5daebba640 1 --2- 172.21.15.172:0/3435922450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da81049c0 0x7f5da819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.558+0000 7f5daebba640 1 --2- 172.21.15.172:0/3435922450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5da8104010 0x7f5da819a570 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:40.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.558+0000 7f5daebba640 1 -- 172.21.15.172:0/3435922450 >> 172.21.15.172:0/3435922450 conn(0x7f5da80ffa50 msgr2=0x7f5da8100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:40.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.558+0000 7f5daebba640 1 -- 172.21.15.172:0/3435922450 shutdown_connections 2024-09-10T08:53:40.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:40.558+0000 7f5daebba640 1 -- 172.21.15.172:0/3435922450 wait complete. 2024-09-10T08:53:40.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:40 smithi174 ceph-mon[34068]: pgmap v573: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-09-10T08:53:40.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:40 smithi191 ceph-mon[35314]: pgmap v573: 129 pgs: 129 active+clean; 5.9 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-09-10T08:53:41.381 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:53:41.406 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:41 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:53:41.406 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:41 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:53:41.406 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:41 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:53:41.406 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:41 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:41.406 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:41 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:53:41.406 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:41 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:41.406 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:41 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:41.406 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:41 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:53:41.407 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:41 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:41.407 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:41 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:41.668 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:53:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:41 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:53:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:41 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:53:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:41 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:53:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:41 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:41 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:53:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:41 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:41 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:41 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:53:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:41 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:41 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:41.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:41 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:53:41.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:41 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:53:41.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:41 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:53:41.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:41 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:41.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:41 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:53:41.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:41 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:41.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:41 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:41.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:41 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:53:41.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:41 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:41.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:41 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:42.695 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:42 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T08:53:42.695 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:42 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:42.695 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:42 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:53:42.695 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:42 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:42.695 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:42 smithi172 ceph-mon[30854]: pgmap v574: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 26 op/s 2024-09-10T08:53:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:42 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T08:53:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:42 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:42 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:53:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:42 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:42 smithi174 ceph-mon[34068]: pgmap v574: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 26 op/s 2024-09-10T08:53:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:42 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T08:53:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:42 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:42 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:53:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:42 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:53:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:42 smithi191 ceph-mon[35314]: pgmap v574: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 26 op/s 2024-09-10T08:53:44.632 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:44 smithi172 ceph-mon[30854]: pgmap v575: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 MiB/s wr, 27 op/s 2024-09-10T08:53:44.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:44 smithi174 ceph-mon[34068]: pgmap v575: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 MiB/s wr, 27 op/s 2024-09-10T08:53:44.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:44 smithi191 ceph-mon[35314]: pgmap v575: 129 pgs: 129 active+clean; 6.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 MiB/s wr, 27 op/s 2024-09-10T08:53:45.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.629+0000 7f0d8fad0640 1 -- 172.21.15.172:0/1781142516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d88102ef0 msgr2=0x7f0d8810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.629+0000 7f0d8fad0640 1 --2- 172.21.15.172:0/1781142516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d88102ef0 0x7f0d8810da40 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009ca0 tx=0x7f0d8002a950 comp rx=0 tx=0).stop 2024-09-10T08:53:45.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.629+0000 7f0d8fad0640 1 -- 172.21.15.172:0/1781142516 shutdown_connections 2024-09-10T08:53:45.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.629+0000 7f0d8fad0640 1 --2- 172.21.15.172:0/1781142516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d88102ef0 0x7f0d8810da40 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.629+0000 7f0d8fad0640 1 --2- 172.21.15.172:0/1781142516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d88102570 0x7f0d881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.629+0000 7f0d8fad0640 1 --2- 172.21.15.172:0/1781142516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d88100e20 0x7f0d88101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.629+0000 7f0d8fad0640 1 -- 172.21.15.172:0/1781142516 >> 172.21.15.172:0/1781142516 conn(0x7f0d880fc7d0 msgr2=0x7f0d880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:45.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.629+0000 7f0d8fad0640 1 -- 172.21.15.172:0/1781142516 shutdown_connections 2024-09-10T08:53:45.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.630+0000 7f0d8fad0640 1 -- 172.21.15.172:0/1781142516 wait complete. 2024-09-10T08:53:45.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.630+0000 7f0d8fad0640 1 Processor -- start 2024-09-10T08:53:45.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.630+0000 7f0d8fad0640 1 -- start start 2024-09-10T08:53:45.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8fad0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d88100e20 0x7f0d881991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8fad0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d88102570 0x7f0d881996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8fad0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d88102ef0 0x7f0d881a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8fad0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88110e80 con 0x7f0d88102570 2024-09-10T08:53:45.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8fad0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88110d00 con 0x7f0d88100e20 2024-09-10T08:53:45.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8fad0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88111000 con 0x7f0d88102ef0 2024-09-10T08:53:45.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8d06b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d88102570 0x7f0d881996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8d86c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d88100e20 0x7f0d881991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8d86c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d88100e20 0x7f0d881991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:51456/0 (socket says 172.21.15.172:51456) 2024-09-10T08:53:45.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8d06b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d88102570 0x7f0d881996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35484/0 (socket says 172.21.15.172:35484) 2024-09-10T08:53:45.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8d86c640 1 -- 172.21.15.172:0/212552809 learned_addr learned my addr 172.21.15.172:0/212552809 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:45.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8e06d640 1 --2- 172.21.15.172:0/212552809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d88102ef0 0x7f0d881a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8d86c640 1 -- 172.21.15.172:0/212552809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d88102ef0 msgr2=0x7f0d881a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8d86c640 1 --2- 172.21.15.172:0/212552809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d88102ef0 0x7f0d881a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8d86c640 1 -- 172.21.15.172:0/212552809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d88102570 msgr2=0x7f0d881996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8d86c640 1 --2- 172.21.15.172:0/212552809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d88102570 0x7f0d881996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8d86c640 1 -- 172.21.15.172:0/212552809 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d881a4760 con 0x7f0d88100e20 2024-09-10T08:53:45.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8e06d640 1 --2- 172.21.15.172:0/212552809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d88102ef0 0x7f0d881a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:45.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.631+0000 7f0d8d06b640 1 --2- 172.21.15.172:0/212552809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d88102570 0x7f0d881996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:45.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.632+0000 7f0d8d86c640 1 --2- 172.21.15.172:0/212552809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d88100e20 0x7f0d881991a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400c970 tx=0x7f0d8400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:45.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.632+0000 7f0d7effd640 1 -- 172.21.15.172:0/212552809 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84007bb0 con 0x7f0d88100e20 2024-09-10T08:53:45.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.632+0000 7f0d7effd640 1 -- 172.21.15.172:0/212552809 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d84007d50 con 0x7f0d88100e20 2024-09-10T08:53:45.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.632+0000 7f0d7effd640 1 -- 172.21.15.172:0/212552809 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8400f450 con 0x7f0d88100e20 2024-09-10T08:53:45.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.632+0000 7f0d8fad0640 1 -- 172.21.15.172:0/212552809 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d881a4930 con 0x7f0d88100e20 2024-09-10T08:53:45.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.632+0000 7f0d8fad0640 1 -- 172.21.15.172:0/212552809 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d881a4e70 con 0x7f0d88100e20 2024-09-10T08:53:45.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.634+0000 7f0d91ea9740 1 Processor -- start 2024-09-10T08:53:45.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.635+0000 7f0d91ea9740 1 -- start start 2024-09-10T08:53:45.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.635+0000 7f0d91ea9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06e5380 0x55b8e06e5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.635+0000 7f0d91ea9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06e5ca0 0x55b8e06f4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.635+0000 7f0d91ea9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06f4a90 0x55b8e06f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.635+0000 7f0d91ea9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b8e064b2e0 con 0x55b8e06f4a90 2024-09-10T08:53:45.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.635+0000 7f0d91ea9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b8e05f2ce0 con 0x55b8e06e5ca0 2024-09-10T08:53:45.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.635+0000 7f0d91ea9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b8e06472a0 con 0x55b8e06e5380 2024-09-10T08:53:45.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.635+0000 7f0d8d06b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06f4a90 0x55b8e06f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.635+0000 7f0d7effd640 1 -- 172.21.15.172:0/212552809 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0d84004f50 con 0x7f0d88100e20 2024-09-10T08:53:45.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.635+0000 7f0d8d06b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06f4a90 0x55b8e06f6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35494/0 (socket says 172.21.15.172:35494) 2024-09-10T08:53:45.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.635+0000 7f0d8e06d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06e5380 0x55b8e06e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.635+0000 7f0d8d06b640 1 -- 172.21.15.172:0/322791598 learned_addr learned my addr 172.21.15.172:0/322791598 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:45.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.635+0000 7f0d8d86c640 1 --2- 172.21.15.172:0/322791598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06e5ca0 0x55b8e06f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.635+0000 7f0d8d06b640 1 -- 172.21.15.172:0/322791598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06e5380 msgr2=0x55b8e06e5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:45.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.635+0000 7f0d8d06b640 1 --2- 172.21.15.172:0/322791598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06e5380 0x55b8e06e5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.636+0000 7f0d8d06b640 1 -- 172.21.15.172:0/322791598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06e5ca0 msgr2=0x55b8e06f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:45.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.636+0000 7f0d8d06b640 1 --2- 172.21.15.172:0/322791598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06e5ca0 0x55b8e06f4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.636+0000 7f0d8d06b640 1 -- 172.21.15.172:0/322791598 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8e05d7440 con 0x55b8e06f4a90 2024-09-10T08:53:45.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.636+0000 7f0d8d86c640 1 --2- 172.21.15.172:0/322791598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06e5ca0 0x55b8e06f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:53:45.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.636+0000 7f0d8d06b640 1 --2- 172.21.15.172:0/322791598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06f4a90 0x55b8e06f6e80 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f0d780103d0 tx=0x7f0d78010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63019cc371b0c72a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:45.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.636+0000 7f0d73fff640 1 -- 172.21.15.172:0/322791598 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78005220 con 0x55b8e06f4a90 2024-09-10T08:53:45.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.636+0000 7f0d7effd640 1 --2- 172.21.15.172:0/212552809 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0d740a2c00 0x7f0d740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.636+0000 7f0d73fff640 1 -- 172.21.15.172:0/322791598 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d78004c50 con 0x55b8e06f4a90 2024-09-10T08:53:45.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.636+0000 7f0d73fff640 1 -- 172.21.15.172:0/322791598 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78011a50 con 0x55b8e06f4a90 2024-09-10T08:53:45.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.636+0000 7f0d8e06d640 1 --2- 172.21.15.172:0/212552809 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0d740a2c00 0x7f0d740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.636+0000 7f0d7effd640 1 -- 172.21.15.172:0/212552809 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0d840cf120 con 0x7f0d88100e20 2024-09-10T08:53:45.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.637+0000 7f0d8e06d640 1 --2- 172.21.15.172:0/212552809 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0d740a2c00 0x7f0d740a50c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009a60 tx=0x7f0d80009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:45.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.637+0000 7f0d91ea9740 1 -- 172.21.15.172:0/322791598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06f4a90 msgr2=0x55b8e06f6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.637+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/322791598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06f4a90 0x55b8e06f6e80 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f0d780103d0 tx=0x7f0d78010890 comp rx=0 tx=0).stop 2024-09-10T08:53:45.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.637+0000 7f0d91ea9740 1 -- 172.21.15.172:0/322791598 shutdown_connections 2024-09-10T08:53:45.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.637+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/322791598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06f4a90 0x55b8e06f6e80 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.637+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/322791598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06e5ca0 0x55b8e06f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.637+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/322791598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06e5380 0x55b8e06e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.637+0000 7f0d91ea9740 1 -- 172.21.15.172:0/322791598 >> 172.21.15.172:0/322791598 conn(0x55b8e06e1a30 msgr2=0x55b8e06e1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:45.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.637+0000 7f0d91ea9740 1 -- 172.21.15.172:0/322791598 shutdown_connections 2024-09-10T08:53:45.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.637+0000 7f0d91ea9740 1 -- 172.21.15.172:0/322791598 wait complete. 2024-09-10T08:53:45.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.638+0000 7f0d91ea9740 1 -- start start 2024-09-10T08:53:45.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.638+0000 7f0d91ea9740 1 Processor -- start 2024-09-10T08:53:45.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.638+0000 7f0d91ea9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5380 0x55b8e06f4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.638+0000 7f0d91ea9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06e5ca0 0x55b8e06f1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.638+0000 7f0d91ea9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06f4a90 0x55b8e06f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.638+0000 7f0d91ea9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b8e06f7660 con 0x55b8e06e5380 2024-09-10T08:53:45.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.638+0000 7f0d91ea9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b8e0606050 con 0x55b8e06e5ca0 2024-09-10T08:53:45.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.638+0000 7f0d91ea9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b8e06f77e0 con 0x55b8e06f4a90 2024-09-10T08:53:45.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.638+0000 7f0d91ea9740 10 client.0 _ll_register_callbacks cb 0x55b8e0646160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:45.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.638+0000 7f0d8d86c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5380 0x55b8e06f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.638+0000 7f0d8d06b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06e5ca0 0x55b8e06f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.638+0000 7f0d8d86c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5380 0x55b8e06f4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35504/0 (socket says 172.21.15.172:35504) 2024-09-10T08:53:45.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.638+0000 7f0d8d86c640 1 -- 172.21.15.172:0/1402380538 learned_addr learned my addr 172.21.15.172:0/1402380538 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:45.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.638+0000 7f0d8e06d640 1 --2- 172.21.15.172:0/1402380538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06f4a90 0x55b8e06f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.639+0000 7f0d8d06b640 1 -- 172.21.15.172:0/1402380538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06f4a90 msgr2=0x55b8e06f1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:45.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.639+0000 7f0d8d06b640 1 --2- 172.21.15.172:0/1402380538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06f4a90 0x55b8e06f1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.639+0000 7f0d8d06b640 1 -- 172.21.15.172:0/1402380538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5380 msgr2=0x55b8e06f4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:45.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.639+0000 7f0d8d06b640 1 --2- 172.21.15.172:0/1402380538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5380 0x55b8e06f4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.639+0000 7f0d8d06b640 1 -- 172.21.15.172:0/1402380538 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8e03bc3c0 con 0x55b8e06e5ca0 2024-09-10T08:53:45.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.639+0000 7f0d8d86c640 1 --2- 172.21.15.172:0/1402380538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5380 0x55b8e06f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:45.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.639+0000 7f0d717fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:53:45.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.639+0000 7f0d8d06b640 1 --2- 172.21.15.172:0/1402380538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06e5ca0 0x55b8e06f1630 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d78005d00 tx=0x7f0d780048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=951150df7366238e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:45.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.639+0000 7f0d717fa640 1 -- 172.21.15.172:0/1402380538 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78004aa0 con 0x55b8e06e5ca0 2024-09-10T08:53:45.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.639+0000 7f0d91ea9740 1 -- 172.21.15.172:0/1402380538 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d78004c50 con 0x55b8e06e5ca0 2024-09-10T08:53:45.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.639+0000 7f0d717fa640 1 -- 172.21.15.172:0/1402380538 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0d78022070 con 0x55b8e06e5ca0 2024-09-10T08:53:45.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.639+0000 7f0d717fa640 1 -- 172.21.15.172:0/1402380538 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0d7802f050 con 0x55b8e06e5ca0 2024-09-10T08:53:45.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.639+0000 7f0d91ea9740 10 client.26783 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:45.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.639+0000 7f0d91ea9740 1 -- 172.21.15.172:0/1402380538 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b8e0702fb0 con 0x55b8e06e5ca0 2024-09-10T08:53:45.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.640+0000 7f0d717fa640 1 -- 172.21.15.172:0/1402380538 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d78004c50 con 0x55b8e06e5ca0 2024-09-10T08:53:45.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.640+0000 7f0d717fa640 1 -- 172.21.15.172:0/1402380538 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f0d78004e90 con 0x55b8e06e5ca0 2024-09-10T08:53:45.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.640+0000 7f0d91ea9740 10 client.26783 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:45.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.640+0000 7f0d91ea9740 10 client.26783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:45.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.640+0000 7f0d91ea9740 20 client.26783 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:45.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.640+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/1402380538 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b8e07043c0 0x55b8e07067b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.640+0000 7f0d91ea9740 4 client.26783 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:53:45.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.640+0000 7f0d91ea9740 1 -- 172.21.15.172:0/1402380538 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8e0658850 con 0x55b8e07043c0 2024-09-10T08:53:45.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.640+0000 7f0d8d86c640 1 --2- 172.21.15.172:0/1402380538 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b8e07043c0 0x55b8e07067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.641+0000 7f0d8d86c640 1 --2- 172.21.15.172:0/1402380538 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b8e07043c0 0x55b8e07067b0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f1aa37e1f5e6638 server_cookie=610be5fca4225c47 in_seq=0 out_seq=0 2024-09-10T08:53:45.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.641+0000 7f0d717fa640 10 client.26783 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:45.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.641+0000 7f0d717fa640 1 -- 172.21.15.172:0/1402380538 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8e0658850 con 0x55b8e07043c0 2024-09-10T08:53:45.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.641+0000 7f0d717fa640 10 client.26783 handle_command_reply: tid=0 2024-09-10T08:53:45.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.641+0000 7f0d91ea9740 1 client.26783 shutdown 2024-09-10T08:53:45.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.641+0000 7f0d91ea9740 1 -- 172.21.15.172:0/1402380538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06e5ca0 msgr2=0x55b8e06f1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.641+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/1402380538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06e5ca0 0x55b8e06f1630 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d78005d00 tx=0x7f0d780048f0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.642+0000 7f0d91ea9740 1 -- 172.21.15.172:0/1402380538 shutdown_connections 2024-09-10T08:53:45.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.642+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/1402380538 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b8e07043c0 0x55b8e07067b0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.642+0000 7f0d717fa640 0 client.26783 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:45.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.642+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/1402380538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06f4a90 0x55b8e06f1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.642+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/1402380538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06e5ca0 0x55b8e06f1630 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.642+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/1402380538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5380 0x55b8e06f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.642+0000 7f0d91ea9740 1 -- 172.21.15.172:0/1402380538 >> 172.21.15.172:0/1402380538 conn(0x55b8e06e1a30 msgr2=0x55b8e06e7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:45.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.642+0000 7f0d91ea9740 1 -- 172.21.15.172:0/1402380538 shutdown_connections 2024-09-10T08:53:45.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.642+0000 7f0d91ea9740 1 -- 172.21.15.172:0/1402380538 wait complete. 2024-09-10T08:53:45.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.642+0000 7f0d91ea9740 20 client.26783 trim_cache size 0 max 16384 2024-09-10T08:53:45.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.661+0000 7f0d91ea9740 1 Processor -- start 2024-09-10T08:53:45.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.661+0000 7f0d91ea9740 1 -- start start 2024-09-10T08:53:45.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.661+0000 7f0d91ea9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06e5380 0x55b8e0700930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.661+0000 7f0d91ea9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5ca0 0x55b8e070bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.661+0000 7f0d91ea9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06f4a90 0x55b8e070c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.661+0000 7f0d91ea9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881ac970 con 0x55b8e06e5ca0 2024-09-10T08:53:45.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.661+0000 7f0d91ea9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881ae4b0 con 0x55b8e06f4a90 2024-09-10T08:53:45.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.661+0000 7f0d91ea9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881ad040 con 0x55b8e06e5380 2024-09-10T08:53:45.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.662+0000 7f0d8d06b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5ca0 0x55b8e070bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.662+0000 7f0d8d06b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5ca0 0x55b8e070bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35518/0 (socket says 172.21.15.172:35518) 2024-09-10T08:53:45.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.662+0000 7f0d8d86c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06e5380 0x55b8e0700930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.662+0000 7f0d8d06b640 1 -- 172.21.15.172:0/929140699 learned_addr learned my addr 172.21.15.172:0/929140699 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:45.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.662+0000 7f0d8e06d640 1 --2- 172.21.15.172:0/929140699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06f4a90 0x55b8e070c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.662+0000 7f0d8d06b640 1 -- 172.21.15.172:0/929140699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06e5380 msgr2=0x55b8e0700930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:45.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.662+0000 7f0d8d06b640 1 --2- 172.21.15.172:0/929140699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06e5380 0x55b8e0700930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.662+0000 7f0d8d06b640 1 -- 172.21.15.172:0/929140699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06f4a90 msgr2=0x55b8e070c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:45.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.662+0000 7f0d8d06b640 1 --2- 172.21.15.172:0/929140699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06f4a90 0x55b8e070c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.662+0000 7f0d8d06b640 1 -- 172.21.15.172:0/929140699 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8e06e1860 con 0x55b8e06e5ca0 2024-09-10T08:53:45.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.662+0000 7f0d8d06b640 1 --2- 172.21.15.172:0/929140699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5ca0 0x55b8e070bd80 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f0d78000c00 tx=0x7f0d78001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5e449d645fd233 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:45.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.662+0000 7f0d737fe640 1 -- 172.21.15.172:0/929140699 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7802c040 con 0x55b8e06e5ca0 2024-09-10T08:53:45.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.662+0000 7f0d737fe640 1 -- 172.21.15.172:0/929140699 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d78004aa0 con 0x55b8e06e5ca0 2024-09-10T08:53:45.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.662+0000 7f0d737fe640 1 -- 172.21.15.172:0/929140699 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78011d80 con 0x55b8e06e5ca0 2024-09-10T08:53:45.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.663+0000 7f0d91ea9740 1 -- 172.21.15.172:0/929140699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5ca0 msgr2=0x55b8e070bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.663+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/929140699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5ca0 0x55b8e070bd80 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f0d78000c00 tx=0x7f0d78001e50 comp rx=0 tx=0).stop 2024-09-10T08:53:45.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.663+0000 7f0d91ea9740 1 -- 172.21.15.172:0/929140699 shutdown_connections 2024-09-10T08:53:45.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.663+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/929140699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06f4a90 0x55b8e070c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.663+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/929140699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5ca0 0x55b8e070bd80 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.663+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/929140699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06e5380 0x55b8e0700930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.663+0000 7f0d91ea9740 1 -- 172.21.15.172:0/929140699 >> 172.21.15.172:0/929140699 conn(0x55b8e06e1a30 msgr2=0x55b8e06f0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:45.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.663+0000 7f0d91ea9740 1 -- 172.21.15.172:0/929140699 shutdown_connections 2024-09-10T08:53:45.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.663+0000 7f0d91ea9740 1 -- 172.21.15.172:0/929140699 wait complete. 2024-09-10T08:53:45.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.664+0000 7f0d91ea9740 1 -- start start 2024-09-10T08:53:45.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.664+0000 7f0d91ea9740 1 Processor -- start 2024-09-10T08:53:45.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.664+0000 7f0d91ea9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06e5380 0x55b8e06f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.664+0000 7f0d91ea9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5ca0 0x55b8e06f8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.664+0000 7f0d91ea9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06f4a90 0x55b8e06fc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.664+0000 7f0d91ea9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881adde0 con 0x55b8e06e5ca0 2024-09-10T08:53:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.664+0000 7f0d91ea9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881ad710 con 0x55b8e06f4a90 2024-09-10T08:53:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.664+0000 7f0d91ea9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0d881aa950 con 0x55b8e06e5380 2024-09-10T08:53:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.664+0000 7f0d91ea9740 10 client.0 _ll_register_callbacks cb 0x7f0d7800bc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.664+0000 7f0d8d06b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5ca0 0x55b8e06f8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.664+0000 7f0d8d06b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5ca0 0x55b8e06f8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35520/0 (socket says 172.21.15.172:35520) 2024-09-10T08:53:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.664+0000 7f0d8d06b640 1 -- 172.21.15.172:0/1452472396 learned_addr learned my addr 172.21.15.172:0/1452472396 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.664+0000 7f0d8e06d640 1 --2- 172.21.15.172:0/1452472396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06f4a90 0x55b8e06fc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.664+0000 7f0d8d86c640 1 --2- 172.21.15.172:0/1452472396 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06e5380 0x55b8e06f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.665+0000 7f0d8e06d640 1 -- 172.21.15.172:0/1452472396 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06e5380 msgr2=0x55b8e06f81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.665+0000 7f0d8e06d640 1 --2- 172.21.15.172:0/1452472396 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06e5380 0x55b8e06f81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.665+0000 7f0d8e06d640 1 -- 172.21.15.172:0/1452472396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5ca0 msgr2=0x55b8e06f8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:45.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.665+0000 7f0d8e06d640 1 --2- 172.21.15.172:0/1452472396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5ca0 0x55b8e06f8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.665+0000 7f0d8e06d640 1 -- 172.21.15.172:0/1452472396 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8e06fcf30 con 0x55b8e06f4a90 2024-09-10T08:53:45.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.665+0000 7f0d8d06b640 1 --2- 172.21.15.172:0/1452472396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5ca0 0x55b8e06f8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:45.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.665+0000 7f0d8e06d640 1 --2- 172.21.15.172:0/1452472396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06f4a90 0x55b8e06fc840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0d80006010 tx=0x7f0d8002a950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc564eab1b38550c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:45.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.665+0000 7f0d717fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:53:45.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.665+0000 7f0d717fa640 1 -- 172.21.15.172:0/1452472396 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80034040 con 0x55b8e06f4a90 2024-09-10T08:53:45.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.665+0000 7f0d717fa640 1 -- 172.21.15.172:0/1452472396 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0d80041030 con 0x55b8e06f4a90 2024-09-10T08:53:45.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.665+0000 7f0d91ea9740 1 -- 172.21.15.172:0/1452472396 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d78004aa0 con 0x55b8e06f4a90 2024-09-10T08:53:45.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.666+0000 7f0d717fa640 1 -- 172.21.15.172:0/1452472396 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0d80004250 con 0x55b8e06f4a90 2024-09-10T08:53:45.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.666+0000 7f0d91ea9740 10 client.26789 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:45.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.666+0000 7f0d91ea9740 1 -- 172.21.15.172:0/1452472396 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b8e076e890 con 0x55b8e06f4a90 2024-09-10T08:53:45.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.666+0000 7f0d717fa640 1 -- 172.21.15.172:0/1452472396 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d78004aa0 con 0x55b8e06f4a90 2024-09-10T08:53:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.666+0000 7f0d717fa640 1 -- 172.21.15.172:0/1452472396 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f0d80030a30 con 0x55b8e06f4a90 2024-09-10T08:53:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.666+0000 7f0d91ea9740 10 client.26789 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.666+0000 7f0d91ea9740 10 client.26789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.666+0000 7f0d91ea9740 20 client.26789 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.666+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/1452472396 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b8e07043c0 0x55b8e0770ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.666+0000 7f0d91ea9740 4 client.26789 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:53:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.666+0000 7f0d91ea9740 1 -- 172.21.15.172:0/1452472396 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b8e0652d60 con 0x55b8e07043c0 2024-09-10T08:53:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.666+0000 7f0d8d86c640 1 --2- 172.21.15.172:0/1452472396 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b8e07043c0 0x55b8e0770ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.667+0000 7f0d717fa640 10 client.26789 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.667+0000 7f0d8d86c640 1 --2- 172.21.15.172:0/1452472396 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b8e07043c0 0x55b8e0770ff0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bc629cd4ead5094 server_cookie=543a77b34428d162 in_seq=0 out_seq=0 2024-09-10T08:53:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.667+0000 7f0d717fa640 1 -- 172.21.15.172:0/1452472396 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b8e0652d60 con 0x55b8e07043c0 2024-09-10T08:53:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.667+0000 7f0d717fa640 10 client.26789 handle_command_reply: tid=0 2024-09-10T08:53:45.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.667+0000 7f0d91ea9740 1 client.26789 shutdown 2024-09-10T08:53:45.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.667+0000 7f0d91ea9740 1 -- 172.21.15.172:0/1452472396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06f4a90 msgr2=0x55b8e06fc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.667+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/1452472396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06f4a90 0x55b8e06fc840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0d80006010 tx=0x7f0d8002a950 comp rx=0 tx=0).stop 2024-09-10T08:53:45.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.667+0000 7f0d91ea9740 1 -- 172.21.15.172:0/1452472396 shutdown_connections 2024-09-10T08:53:45.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.667+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/1452472396 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b8e07043c0 0x55b8e0770ff0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.667+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/1452472396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8e06f4a90 0x55b8e06fc840 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.667+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/1452472396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8e06e5ca0 0x55b8e06f8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.667+0000 7f0d717fa640 0 client.26789 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:45.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.667+0000 7f0d91ea9740 1 --2- 172.21.15.172:0/1452472396 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8e06e5380 0x55b8e06f81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.667+0000 7f0d91ea9740 1 -- 172.21.15.172:0/1452472396 >> 172.21.15.172:0/1452472396 conn(0x55b8e06e1a30 msgr2=0x55b8e06f2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:45.503 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:53:45.503 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:53:45.503 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T08:53:45.503 INFO:teuthology.orchestra.run.smithi172.stdout: "704d45d1-f294-47d0-9e7a-2817dc8af480": { 2024-09-10T08:53:45.503 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T08:53:45.503 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "704d45d1-f294-47d0-9e7a-2817dc8af480", 2024-09-10T08:53:45.503 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T08:53:45.503 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:53:45.503 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:53:45.503 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:53:45.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.667+0000 7f0d91ea9740 1 -- 172.21.15.172:0/1452472396 shutdown_connections 2024-09-10T08:53:45.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.667+0000 7f0d91ea9740 1 -- 172.21.15.172:0/1452472396 wait complete. 2024-09-10T08:53:45.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.667+0000 7f0d91ea9740 20 client.26789 trim_cache size 0 max 16384 2024-09-10T08:53:45.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.668+0000 7f0d8fad0640 1 -- 172.21.15.172:0/212552809 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0d740a2c00 msgr2=0x7f0d740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.668+0000 7f0d8fad0640 1 --2- 172.21.15.172:0/212552809 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0d740a2c00 0x7f0d740a50c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009a60 tx=0x7f0d80009800 comp rx=0 tx=0).stop 2024-09-10T08:53:45.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.668+0000 7f0d8fad0640 1 -- 172.21.15.172:0/212552809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d88100e20 msgr2=0x7f0d881991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.668+0000 7f0d8fad0640 1 --2- 172.21.15.172:0/212552809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d88100e20 0x7f0d881991a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400c970 tx=0x7f0d8400ce30 comp rx=0 tx=0).stop 2024-09-10T08:53:45.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.668+0000 7f0d8fad0640 1 -- 172.21.15.172:0/212552809 shutdown_connections 2024-09-10T08:53:45.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.668+0000 7f0d8fad0640 1 --2- 172.21.15.172:0/212552809 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0d740a2c00 0x7f0d740a50c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.668+0000 7f0d8fad0640 1 --2- 172.21.15.172:0/212552809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d88102ef0 0x7f0d881a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.668+0000 7f0d8fad0640 1 --2- 172.21.15.172:0/212552809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d88102570 0x7f0d881996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.668+0000 7f0d8fad0640 1 --2- 172.21.15.172:0/212552809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d88100e20 0x7f0d881991a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.668+0000 7f0d8fad0640 1 -- 172.21.15.172:0/212552809 >> 172.21.15.172:0/212552809 conn(0x7f0d880fc7d0 msgr2=0x7f0d880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.668+0000 7f0d8fad0640 1 -- 172.21.15.172:0/212552809 shutdown_connections 2024-09-10T08:53:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:53:44.668+0000 7f0d8fad0640 1 -- 172.21.15.172:0/212552809 wait complete. 2024-09-10T08:53:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.588+0000 7f4188bf8640 1 -- 172.21.15.172:0/4026705044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4184102600 msgr2=0x7f4184108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.588+0000 7f4188bf8640 1 --2- 172.21.15.172:0/4026705044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4184102600 0x7f4184108e90 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f417c009ca0 tx=0x7f417c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.588+0000 7f4188bf8640 1 -- 172.21.15.172:0/4026705044 shutdown_connections 2024-09-10T08:53:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.588+0000 7f4188bf8640 1 --2- 172.21.15.172:0/4026705044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4184102600 0x7f4184108e90 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.588+0000 7f4188bf8640 1 --2- 172.21.15.172:0/4026705044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4184101c40 0x7f41841020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.588+0000 7f4188bf8640 1 --2- 172.21.15.172:0/4026705044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4184100a50 0x7f4184100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.588+0000 7f4188bf8640 1 -- 172.21.15.172:0/4026705044 >> 172.21.15.172:0/4026705044 conn(0x7f41840fc1b0 msgr2=0x7f41840fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.588+0000 7f4188bf8640 1 -- 172.21.15.172:0/4026705044 shutdown_connections 2024-09-10T08:53:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.588+0000 7f4188bf8640 1 -- 172.21.15.172:0/4026705044 wait complete. 2024-09-10T08:53:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.588+0000 7f4188bf8640 1 Processor -- start 2024-09-10T08:53:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.588+0000 7f4188bf8640 1 -- start start 2024-09-10T08:53:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4188bf8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4184100a50 0x7f418406ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4188bf8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4184101c40 0x7f418419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4188bf8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4184102600 0x7f418419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4188bf8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f418410c560 con 0x7f4184101c40 2024-09-10T08:53:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4188bf8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f418410c3e0 con 0x7f4184102600 2024-09-10T08:53:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4188bf8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f418410c6e0 con 0x7f4184100a50 2024-09-10T08:53:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4181d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4184101c40 0x7f418419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4181d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4184101c40 0x7f418419c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35542/0 (socket says 172.21.15.172:35542) 2024-09-10T08:53:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4181d9b640 1 -- 172.21.15.172:0/3912512529 learned_addr learned my addr 172.21.15.172:0/3912512529 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4182d9d640 1 --2- 172.21.15.172:0/3912512529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4184102600 0x7f418419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f418259c640 1 --2- 172.21.15.172:0/3912512529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4184100a50 0x7f418406ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4181d9b640 1 -- 172.21.15.172:0/3912512529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4184100a50 msgr2=0x7f418406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4181d9b640 1 --2- 172.21.15.172:0/3912512529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4184100a50 0x7f418406ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4181d9b640 1 -- 172.21.15.172:0/3912512529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4184102600 msgr2=0x7f418419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4181d9b640 1 --2- 172.21.15.172:0/3912512529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4184102600 0x7f418419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4181d9b640 1 -- 172.21.15.172:0/3912512529 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418419f900 con 0x7f4184101c40 2024-09-10T08:53:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4182d9d640 1 --2- 172.21.15.172:0/3912512529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4184102600 0x7f418419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:53:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4181d9b640 1 --2- 172.21.15.172:0/3912512529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4184101c40 0x7f418419c9d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f417800c890 tx=0x7f417800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f41737fe640 1 -- 172.21.15.172:0/3912512529 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178004220 con 0x7f4184101c40 2024-09-10T08:53:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f4188bf8640 1 -- 172.21.15.172:0/3912512529 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418419fb90 con 0x7f4184101c40 2024-09-10T08:53:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.589+0000 7f41737fe640 1 -- 172.21.15.172:0/3912512529 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41780043c0 con 0x7f4184101c40 2024-09-10T08:53:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.590+0000 7f41737fe640 1 -- 172.21.15.172:0/3912512529 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178004a00 con 0x7f4184101c40 2024-09-10T08:53:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.590+0000 7f4188bf8640 1 -- 172.21.15.172:0/3912512529 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41841a0130 con 0x7f4184101c40 2024-09-10T08:53:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.593+0000 7f418ba68740 1 Processor -- start 2024-09-10T08:53:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.593+0000 7f418ba68740 1 -- start start 2024-09-10T08:53:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.593+0000 7f41737fe640 1 -- 172.21.15.172:0/3912512529 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4178004ba0 con 0x7f4184101c40 2024-09-10T08:53:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.593+0000 7f418ba68740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947129a0 0x55af94712da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.593+0000 7f418ba68740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af947132e0 0x55af94717da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.593+0000 7f418ba68740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af94718370 0x55af9471a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.593+0000 7f418ba68740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55af946762f0 con 0x55af947129a0 2024-09-10T08:53:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.593+0000 7f418ba68740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55af9461e270 con 0x55af947132e0 2024-09-10T08:53:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.593+0000 7f418ba68740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55af946722b0 con 0x55af94718370 2024-09-10T08:53:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.593+0000 7f4182d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947129a0 0x55af94712da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.593+0000 7f4182d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947129a0 0x55af94712da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35546/0 (socket says 172.21.15.172:35546) 2024-09-10T08:53:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.593+0000 7f4182d9d640 1 -- 172.21.15.172:0/1901238268 learned_addr learned my addr 172.21.15.172:0/1901238268 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.593+0000 7f418259c640 1 --2- 172.21.15.172:0/1901238268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af947132e0 0x55af94717da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.593+0000 7f4181d9b640 1 --2- 172.21.15.172:0/1901238268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af94718370 0x55af9471a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.594+0000 7f4182d9d640 1 -- 172.21.15.172:0/1901238268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af94718370 msgr2=0x55af9471a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.594+0000 7f4182d9d640 1 --2- 172.21.15.172:0/1901238268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af94718370 0x55af9471a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.594+0000 7f4182d9d640 1 -- 172.21.15.172:0/1901238268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af947132e0 msgr2=0x55af94717da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.594+0000 7f4182d9d640 1 --2- 172.21.15.172:0/1901238268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af947132e0 0x55af94717da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.594+0000 7f4182d9d640 1 -- 172.21.15.172:0/1901238268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af9461d340 con 0x55af947129a0 2024-09-10T08:53:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.594+0000 7f418259c640 1 --2- 172.21.15.172:0/1901238268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af947132e0 0x55af94717da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:53:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.594+0000 7f4182d9d640 1 --2- 172.21.15.172:0/1901238268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947129a0 0x55af94712da0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f417c005cc0 tx=0x7f417c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbecfe1338c5113 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.594+0000 7f4170ff9640 1 -- 172.21.15.172:0/1901238268 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c031750 con 0x55af947129a0 2024-09-10T08:53:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.594+0000 7f4170ff9640 1 -- 172.21.15.172:0/1901238268 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f417c036070 con 0x55af947129a0 2024-09-10T08:53:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.594+0000 7f4170ff9640 1 -- 172.21.15.172:0/1901238268 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c0044a0 con 0x55af947129a0 2024-09-10T08:53:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.594+0000 7f41737fe640 1 --2- 172.21.15.172:0/3912512529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f416c0a2c30 0x7f416c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.594+0000 7f418259c640 1 --2- 172.21.15.172:0/3912512529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f416c0a2c30 0x7f416c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.594+0000 7f41737fe640 1 -- 172.21.15.172:0/3912512529 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f41780cf110 con 0x7f4184101c40 2024-09-10T08:53:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.595+0000 7f418259c640 1 --2- 172.21.15.172:0/3912512529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f416c0a2c30 0x7f416c0a50f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55af94713d10 tx=0x7f417400b4d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.595+0000 7f418ba68740 1 -- 172.21.15.172:0/1901238268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947129a0 msgr2=0x55af94712da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.595+0000 7f418ba68740 1 --2- 172.21.15.172:0/1901238268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947129a0 0x55af94712da0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f417c005cc0 tx=0x7f417c009a20 comp rx=0 tx=0).stop 2024-09-10T08:53:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.595+0000 7f418ba68740 1 -- 172.21.15.172:0/1901238268 shutdown_connections 2024-09-10T08:53:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.595+0000 7f418ba68740 1 --2- 172.21.15.172:0/1901238268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af94718370 0x55af9471a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.595+0000 7f418ba68740 1 --2- 172.21.15.172:0/1901238268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af947132e0 0x55af94717da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.595+0000 7f418ba68740 1 --2- 172.21.15.172:0/1901238268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947129a0 0x55af94712da0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.595+0000 7f418ba68740 1 -- 172.21.15.172:0/1901238268 >> 172.21.15.172:0/1901238268 conn(0x55af9470e010 msgr2=0x55af9470e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.595+0000 7f418ba68740 1 -- 172.21.15.172:0/1901238268 shutdown_connections 2024-09-10T08:53:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.595+0000 7f418ba68740 1 -- 172.21.15.172:0/1901238268 wait complete. 2024-09-10T08:53:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f418ba68740 1 -- start start 2024-09-10T08:53:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f418ba68740 1 Processor -- start 2024-09-10T08:53:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f418ba68740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af947129a0 0x55af9471cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f418ba68740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af947132e0 0x55af9471d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f418ba68740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af94718370 0x55af947240e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f418ba68740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55af9471b2e0 con 0x55af94718370 2024-09-10T08:53:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f418ba68740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55af945ff290 con 0x55af947129a0 2024-09-10T08:53:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f418ba68740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55af9471b460 con 0x55af947132e0 2024-09-10T08:53:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f418ba68740 10 client.0 _ll_register_callbacks cb 0x55af94630140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f4182d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af94718370 0x55af947240e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f4182d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af94718370 0x55af947240e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35552/0 (socket says 172.21.15.172:35552) 2024-09-10T08:53:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f4182d9d640 1 -- 172.21.15.172:0/2094412355 learned_addr learned my addr 172.21.15.172:0/2094412355 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f4181d9b640 1 --2- 172.21.15.172:0/2094412355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af947132e0 0x55af9471d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f418259c640 1 --2- 172.21.15.172:0/2094412355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af947129a0 0x55af9471cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f4182d9d640 1 -- 172.21.15.172:0/2094412355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af947132e0 msgr2=0x55af9471d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f4182d9d640 1 --2- 172.21.15.172:0/2094412355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af947132e0 0x55af9471d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f4182d9d640 1 -- 172.21.15.172:0/2094412355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af947129a0 msgr2=0x55af9471cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f4182d9d640 1 --2- 172.21.15.172:0/2094412355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af947129a0 0x55af9471cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f4182d9d640 1 -- 172.21.15.172:0/2094412355 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af94389210 con 0x55af94718370 2024-09-10T08:53:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f418259c640 1 --2- 172.21.15.172:0/2094412355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af947129a0 0x55af9471cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:53:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.596+0000 7f4181d9b640 1 --2- 172.21.15.172:0/2094412355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af947132e0 0x55af9471d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.597+0000 7f4182d9d640 1 --2- 172.21.15.172:0/2094412355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af94718370 0x55af947240e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f417c009bf0 tx=0x7f417c0315d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccde68de07f74cce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.597+0000 7f4169ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.597+0000 7f4169ffb640 1 -- 172.21.15.172:0/2094412355 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c049040 con 0x55af94718370 2024-09-10T08:53:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.597+0000 7f418ba68740 1 -- 172.21.15.172:0/2094412355 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f417c036070 con 0x55af94718370 2024-09-10T08:53:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.597+0000 7f4169ffb640 1 -- 172.21.15.172:0/2094412355 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f417c031d40 con 0x55af94718370 2024-09-10T08:53:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.597+0000 7f4169ffb640 1 -- 172.21.15.172:0/2094412355 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f417c05b050 con 0x55af94718370 2024-09-10T08:53:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.597+0000 7f418ba68740 10 client.17733 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.597+0000 7f418ba68740 1 -- 172.21.15.172:0/2094412355 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55af94455620 con 0x55af94718370 2024-09-10T08:53:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.597+0000 7f4169ffb640 1 -- 172.21.15.172:0/2094412355 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f417c036070 con 0x55af94718370 2024-09-10T08:53:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.597+0000 7f4169ffb640 1 -- 172.21.15.172:0/2094412355 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f417c04c070 con 0x55af94718370 2024-09-10T08:53:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.597+0000 7f418ba68740 10 client.17733 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.597+0000 7f418ba68740 10 client.17733 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.597+0000 7f418ba68740 20 client.17733 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.597+0000 7f418ba68740 1 --2- 172.21.15.172:0/2094412355 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55af947283e0 0x55af9472a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.597+0000 7f418ba68740 4 client.17733 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:53:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.597+0000 7f418ba68740 1 -- 172.21.15.172:0/2094412355 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af94683a20 con 0x55af947283e0 2024-09-10T08:53:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.598+0000 7f418259c640 1 --2- 172.21.15.172:0/2094412355 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55af947283e0 0x55af9472a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.598+0000 7f4169ffb640 10 client.17733 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.598+0000 7f418259c640 1 --2- 172.21.15.172:0/2094412355 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55af947283e0 0x55af9472a7d0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3948453d0a52f8b0 server_cookie=5b3d9405628cfe47 in_seq=0 out_seq=0 2024-09-10T08:53:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.599+0000 7f4169ffb640 1 -- 172.21.15.172:0/2094412355 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af94683a20 con 0x55af947283e0 2024-09-10T08:53:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.599+0000 7f4169ffb640 10 client.17733 handle_command_reply: tid=0 2024-09-10T08:53:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.599+0000 7f418ba68740 1 client.17733 shutdown 2024-09-10T08:53:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.599+0000 7f418ba68740 1 -- 172.21.15.172:0/2094412355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af94718370 msgr2=0x55af947240e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.599+0000 7f418ba68740 1 --2- 172.21.15.172:0/2094412355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af94718370 0x55af947240e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f417c009bf0 tx=0x7f417c0315d0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.599+0000 7f418ba68740 1 -- 172.21.15.172:0/2094412355 shutdown_connections 2024-09-10T08:53:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.599+0000 7f418ba68740 1 --2- 172.21.15.172:0/2094412355 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55af947283e0 0x55af9472a7d0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.599+0000 7f418ba68740 1 --2- 172.21.15.172:0/2094412355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af94718370 0x55af947240e0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.599+0000 7f4169ffb640 0 client.17733 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.599+0000 7f418ba68740 1 --2- 172.21.15.172:0/2094412355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af947132e0 0x55af9471d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.599+0000 7f418ba68740 1 --2- 172.21.15.172:0/2094412355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af947129a0 0x55af9471cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.599+0000 7f418ba68740 1 -- 172.21.15.172:0/2094412355 >> 172.21.15.172:0/2094412355 conn(0x55af9470e010 msgr2=0x55af9470f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.599+0000 7f418ba68740 1 -- 172.21.15.172:0/2094412355 shutdown_connections 2024-09-10T08:53:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.599+0000 7f418ba68740 1 -- 172.21.15.172:0/2094412355 wait complete. 2024-09-10T08:53:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.599+0000 7f418ba68740 20 client.17733 trim_cache size 0 max 16384 2024-09-10T08:53:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f418ba68740 1 Processor -- start 2024-09-10T08:53:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f418ba68740 1 -- start start 2024-09-10T08:53:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f418ba68740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af947129a0 0x55af9470eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f418ba68740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947132e0 0x55af94724000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f418ba68740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af94718370 0x55af9470f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f418ba68740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f41841aa4c0 con 0x55af947132e0 2024-09-10T08:53:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f418ba68740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41841ac000 con 0x55af94718370 2024-09-10T08:53:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f418ba68740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f41841aab90 con 0x55af947129a0 2024-09-10T08:53:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f4181d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947132e0 0x55af94724000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f4181d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947132e0 0x55af94724000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35556/0 (socket says 172.21.15.172:35556) 2024-09-10T08:53:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f4181d9b640 1 -- 172.21.15.172:0/3764496085 learned_addr learned my addr 172.21.15.172:0/3764496085 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f4182d9d640 1 --2- 172.21.15.172:0/3764496085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af94718370 0x55af9470f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f418259c640 1 --2- 172.21.15.172:0/3764496085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af947129a0 0x55af9470eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f4181d9b640 1 -- 172.21.15.172:0/3764496085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af947129a0 msgr2=0x55af9470eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f4181d9b640 1 --2- 172.21.15.172:0/3764496085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af947129a0 0x55af9470eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f4181d9b640 1 -- 172.21.15.172:0/3764496085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af94718370 msgr2=0x55af9470f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f4181d9b640 1 --2- 172.21.15.172:0/3764496085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af94718370 0x55af9470f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f4181d9b640 1 -- 172.21.15.172:0/3764496085 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af9471c910 con 0x55af947132e0 2024-09-10T08:53:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f418259c640 1 --2- 172.21.15.172:0/3764496085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af947129a0 0x55af9470eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:53:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f4182d9d640 1 --2- 172.21.15.172:0/3764496085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af94718370 0x55af9470f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.616+0000 7f4181d9b640 1 --2- 172.21.15.172:0/3764496085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947132e0 0x55af94724000 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f417809b5a0 tx=0x7f41780860b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b074e0e0d10249c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.617+0000 7f416bfff640 1 -- 172.21.15.172:0/3764496085 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417801d040 con 0x55af947132e0 2024-09-10T08:53:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.617+0000 7f416bfff640 1 -- 172.21.15.172:0/3764496085 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4178097040 con 0x55af947132e0 2024-09-10T08:53:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.617+0000 7f416bfff640 1 -- 172.21.15.172:0/3764496085 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178099910 con 0x55af947132e0 2024-09-10T08:53:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.617+0000 7f418ba68740 1 -- 172.21.15.172:0/3764496085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947132e0 msgr2=0x55af94724000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.617+0000 7f418ba68740 1 --2- 172.21.15.172:0/3764496085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947132e0 0x55af94724000 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f417809b5a0 tx=0x7f41780860b0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f418ba68740 1 -- 172.21.15.172:0/3764496085 shutdown_connections 2024-09-10T08:53:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f418ba68740 1 --2- 172.21.15.172:0/3764496085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af94718370 0x55af9470f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f418ba68740 1 --2- 172.21.15.172:0/3764496085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947132e0 0x55af94724000 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f418ba68740 1 --2- 172.21.15.172:0/3764496085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af947129a0 0x55af9470eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f418ba68740 1 -- 172.21.15.172:0/3764496085 >> 172.21.15.172:0/3764496085 conn(0x55af9470e010 msgr2=0x55af9471cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f418ba68740 1 -- 172.21.15.172:0/3764496085 shutdown_connections 2024-09-10T08:53:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f418ba68740 1 -- 172.21.15.172:0/3764496085 wait complete. 2024-09-10T08:53:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f418ba68740 1 -- start start 2024-09-10T08:53:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f418ba68740 1 Processor -- start 2024-09-10T08:53:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f418ba68740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af947129a0 0x55af94721940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f418ba68740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947132e0 0x55af94721e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f418ba68740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af94718370 0x55af94722e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f418ba68740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f41841ab930 con 0x55af947132e0 2024-09-10T08:53:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f418ba68740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41841ab260 con 0x55af94718370 2024-09-10T08:53:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f418ba68740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f41841a84a0 con 0x55af947129a0 2024-09-10T08:53:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f418ba68740 10 client.0 _ll_register_callbacks cb 0x7f417400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f4181d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947132e0 0x55af94721e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f4181d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947132e0 0x55af94721e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35572/0 (socket says 172.21.15.172:35572) 2024-09-10T08:53:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f4181d9b640 1 -- 172.21.15.172:0/1925665016 learned_addr learned my addr 172.21.15.172:0/1925665016 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f418259c640 1 --2- 172.21.15.172:0/1925665016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af947129a0 0x55af94721940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.618+0000 7f4182d9d640 1 --2- 172.21.15.172:0/1925665016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af94718370 0x55af94722e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.619+0000 7f4181d9b640 1 -- 172.21.15.172:0/1925665016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af947129a0 msgr2=0x55af94721940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.619+0000 7f4181d9b640 1 --2- 172.21.15.172:0/1925665016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af947129a0 0x55af94721940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.619+0000 7f4181d9b640 1 -- 172.21.15.172:0/1925665016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af94718370 msgr2=0x55af94722e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.619+0000 7f4181d9b640 1 --2- 172.21.15.172:0/1925665016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af94718370 0x55af94722e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.619+0000 7f4181d9b640 1 -- 172.21.15.172:0/1925665016 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af947234a0 con 0x55af947132e0 2024-09-10T08:53:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.619+0000 7f4181d9b640 1 --2- 172.21.15.172:0/1925665016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947132e0 0x55af94721e80 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f41780cf4c0 tx=0x7f417809aff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68acb0cca092131 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.619+0000 7f4169ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.619+0000 7f4169ffb640 1 -- 172.21.15.172:0/1925665016 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41780b4060 con 0x55af947132e0 2024-09-10T08:53:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.619+0000 7f4169ffb640 1 -- 172.21.15.172:0/1925665016 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f41780050b0 con 0x55af947132e0 2024-09-10T08:53:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.619+0000 7f418ba68740 1 -- 172.21.15.172:0/1925665016 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4178097040 con 0x55af947132e0 2024-09-10T08:53:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.619+0000 7f4169ffb640 1 -- 172.21.15.172:0/1925665016 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f417801d040 con 0x55af947132e0 2024-09-10T08:53:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.619+0000 7f418ba68740 10 client.17739 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.619+0000 7f418ba68740 1 -- 172.21.15.172:0/1925665016 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55af947930f0 con 0x55af947132e0 2024-09-10T08:53:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.619+0000 7f4169ffb640 1 -- 172.21.15.172:0/1925665016 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4178097040 con 0x55af947132e0 2024-09-10T08:53:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.620+0000 7f4169ffb640 1 -- 172.21.15.172:0/1925665016 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f4178060b60 con 0x55af947132e0 2024-09-10T08:53:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.620+0000 7f418ba68740 10 client.17739 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.620+0000 7f418ba68740 10 client.17739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.620+0000 7f418ba68740 20 client.17739 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.620+0000 7f418ba68740 1 --2- 172.21.15.172:0/1925665016 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55af947939a0 0x55af94795dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.620+0000 7f418ba68740 4 client.17739 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:53:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.620+0000 7f418ba68740 1 -- 172.21.15.172:0/1925665016 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af9467dd70 con 0x55af947939a0 2024-09-10T08:53:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.620+0000 7f418259c640 1 --2- 172.21.15.172:0/1925665016 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55af947939a0 0x55af94795dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.620+0000 7f4169ffb640 10 client.17739 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.620+0000 7f418259c640 1 --2- 172.21.15.172:0/1925665016 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55af947939a0 0x55af94795dc0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f6bc4ddfae64d1e server_cookie=9cf5209bce8dada3 in_seq=0 out_seq=0 2024-09-10T08:53:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.626+0000 7f4169ffb640 1 -- 172.21.15.172:0/1925665016 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af9467dd70 con 0x55af947939a0 2024-09-10T08:53:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.626+0000 7f4169ffb640 10 client.17739 handle_command_reply: tid=0 2024-09-10T08:53:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.626+0000 7f418ba68740 1 client.17739 shutdown 2024-09-10T08:53:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.626+0000 7f418ba68740 1 -- 172.21.15.172:0/1925665016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947132e0 msgr2=0x55af94721e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.626+0000 7f418ba68740 1 --2- 172.21.15.172:0/1925665016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947132e0 0x55af94721e80 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f41780cf4c0 tx=0x7f417809aff0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.626+0000 7f418ba68740 1 -- 172.21.15.172:0/1925665016 shutdown_connections 2024-09-10T08:53:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.626+0000 7f418ba68740 1 --2- 172.21.15.172:0/1925665016 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55af947939a0 0x55af94795dc0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.626+0000 7f418ba68740 1 --2- 172.21.15.172:0/1925665016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af94718370 0x55af94722e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.626+0000 7f418ba68740 1 --2- 172.21.15.172:0/1925665016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55af947132e0 0x55af94721e80 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.626+0000 7f4169ffb640 0 client.17739 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.626+0000 7f418ba68740 1 --2- 172.21.15.172:0/1925665016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55af947129a0 0x55af94721940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.626+0000 7f418ba68740 1 -- 172.21.15.172:0/1925665016 >> 172.21.15.172:0/1925665016 conn(0x55af9470e010 msgr2=0x55af9470c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.626+0000 7f418ba68740 1 -- 172.21.15.172:0/1925665016 shutdown_connections 2024-09-10T08:53:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.626+0000 7f418ba68740 1 -- 172.21.15.172:0/1925665016 wait complete. 2024-09-10T08:53:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.626+0000 7f418ba68740 20 client.17739 trim_cache size 0 max 16384 2024-09-10T08:53:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:53:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:53:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:53:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:53:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.627+0000 7f4188bf8640 1 -- 172.21.15.172:0/3912512529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f416c0a2c30 msgr2=0x7f416c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.627+0000 7f4188bf8640 1 --2- 172.21.15.172:0/3912512529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f416c0a2c30 0x7f416c0a50f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55af94713d10 tx=0x7f417400b4d0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.627+0000 7f4188bf8640 1 -- 172.21.15.172:0/3912512529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4184101c40 msgr2=0x7f418419c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.627+0000 7f4188bf8640 1 --2- 172.21.15.172:0/3912512529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4184101c40 0x7f418419c9d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f417800c890 tx=0x7f417800cd50 comp rx=0 tx=0).stop 2024-09-10T08:53:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.627+0000 7f4188bf8640 1 -- 172.21.15.172:0/3912512529 shutdown_connections 2024-09-10T08:53:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.627+0000 7f4188bf8640 1 --2- 172.21.15.172:0/3912512529 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f416c0a2c30 0x7f416c0a50f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.627+0000 7f4188bf8640 1 --2- 172.21.15.172:0/3912512529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4184102600 0x7f418419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.627+0000 7f4188bf8640 1 --2- 172.21.15.172:0/3912512529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4184101c40 0x7f418419c9d0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.627+0000 7f4188bf8640 1 --2- 172.21.15.172:0/3912512529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4184100a50 0x7f418406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.627+0000 7f4188bf8640 1 -- 172.21.15.172:0/3912512529 >> 172.21.15.172:0/3912512529 conn(0x7f41840fc1b0 msgr2=0x7f4184104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.627+0000 7f4188bf8640 1 -- 172.21.15.172:0/3912512529 shutdown_connections 2024-09-10T08:53:45.632 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:45.627+0000 7f4188bf8640 1 -- 172.21.15.172:0/3912512529 wait complete. 2024-09-10T08:53:46.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:46 smithi174 ceph-mon[34068]: pgmap v576: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 MiB/s wr, 49 op/s 2024-09-10T08:53:46.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:46 smithi191 ceph-mon[35314]: pgmap v576: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 MiB/s wr, 49 op/s 2024-09-10T08:53:46.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:46 smithi172 ceph-mon[30854]: pgmap v576: 129 pgs: 129 active+clean; 6.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 32 MiB/s wr, 49 op/s 2024-09-10T08:53:47.174 INFO:tasks.cephfs.filesystem:scrub status for tag:704d45d1-f294-47d0-9e7a-2817dc8af480 - {'path': '/', 'tag': '704d45d1-f294-47d0-9e7a-2817dc8af480', 'options': 'recursive,force'} 2024-09-10T08:53:48.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:48 smithi174 ceph-mon[34068]: pgmap v577: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 44 op/s 2024-09-10T08:53:48.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:48 smithi191 ceph-mon[35314]: pgmap v577: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 44 op/s 2024-09-10T08:53:48.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:48 smithi172 ceph-mon[30854]: pgmap v577: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 MiB/s wr, 44 op/s 2024-09-10T08:53:50.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:50 smithi174 ceph-mon[34068]: pgmap v578: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 44 op/s 2024-09-10T08:53:50.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:50 smithi191 ceph-mon[35314]: pgmap v578: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 44 op/s 2024-09-10T08:53:50.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:50 smithi172 ceph-mon[30854]: pgmap v578: 129 pgs: 129 active+clean; 6.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 44 op/s 2024-09-10T08:53:51.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:51 smithi172 ceph-mon[30854]: pgmap v579: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 MiB/s wr, 49 op/s 2024-09-10T08:53:51.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:51 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:53:51.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:51 smithi174 ceph-mon[34068]: pgmap v579: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 MiB/s wr, 49 op/s 2024-09-10T08:53:51.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:51 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:53:51.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:51 smithi191 ceph-mon[35314]: pgmap v579: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 MiB/s wr, 49 op/s 2024-09-10T08:53:51.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:51 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:53:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:52 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:53:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:52 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:53:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:52 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:53:52.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:52 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:53:52.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:52 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:53:52.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:52 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:53:52.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:52 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:53:52.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:52 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:53:52.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:52 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:53:53.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:53 smithi174 ceph-mon[34068]: pgmap v580: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 41 op/s 2024-09-10T08:53:53.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:53 smithi191 ceph-mon[35314]: pgmap v580: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 41 op/s 2024-09-10T08:53:53.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:53 smithi172 ceph-mon[30854]: pgmap v580: 129 pgs: 129 active+clean; 6.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 41 op/s 2024-09-10T08:53:56.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:56 smithi174 ceph-mon[34068]: pgmap v581: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 43 op/s 2024-09-10T08:53:56.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:56 smithi191 ceph-mon[35314]: pgmap v581: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 43 op/s 2024-09-10T08:53:56.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:56 smithi172 ceph-mon[30854]: pgmap v581: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 MiB/s wr, 43 op/s 2024-09-10T08:53:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.717+0000 7fd8af677640 1 -- 172.21.15.172:0/3671937558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8a8101b60 msgr2=0x7fd8a8101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.717+0000 7fd8af677640 1 --2- 172.21.15.172:0/3671937558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8a8101b60 0x7fd8a8101f60 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009900 tx=0x7fd8a402a410 comp rx=0 tx=0).stop 2024-09-10T08:53:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.717+0000 7fd8af677640 1 -- 172.21.15.172:0/3671937558 shutdown_connections 2024-09-10T08:53:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.717+0000 7fd8af677640 1 --2- 172.21.15.172:0/3671937558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a8103710 0x7fd8a8109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.717+0000 7fd8af677640 1 --2- 172.21.15.172:0/3671937558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8a8102d50 0x7fd8a81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.717+0000 7fd8af677640 1 --2- 172.21.15.172:0/3671937558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8a8101b60 0x7fd8a8101f60 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.717+0000 7fd8af677640 1 -- 172.21.15.172:0/3671937558 >> 172.21.15.172:0/3671937558 conn(0x7fd8a80fd2e0 msgr2=0x7fd8a80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.718+0000 7fd8af677640 1 -- 172.21.15.172:0/3671937558 shutdown_connections 2024-09-10T08:53:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.718+0000 7fd8af677640 1 -- 172.21.15.172:0/3671937558 wait complete. 2024-09-10T08:53:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.718+0000 7fd8af677640 1 Processor -- start 2024-09-10T08:53:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.718+0000 7fd8af677640 1 -- start start 2024-09-10T08:53:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.718+0000 7fd8af677640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8a8101b60 0x7fd8a8197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.718+0000 7fd8af677640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a8102d50 0x7fd8a8198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.718+0000 7fd8af677640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8a8103710 0x7fd8a819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.718+0000 7fd8af677640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a810d690 con 0x7fd8a8103710 2024-09-10T08:53:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.718+0000 7fd8af677640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a810d510 con 0x7fd8a8102d50 2024-09-10T08:53:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.718+0000 7fd8af677640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a810d810 con 0x7fd8a8101b60 2024-09-10T08:53:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.719+0000 7fd8adc14640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8a8103710 0x7fd8a819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.719+0000 7fd8adc14640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8a8103710 0x7fd8a819f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58584/0 (socket says 172.21.15.172:58584) 2024-09-10T08:53:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.719+0000 7fd8adc14640 1 -- 172.21.15.172:0/2177889877 learned_addr learned my addr 172.21.15.172:0/2177889877 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.719+0000 7fd8ad413640 1 --2- 172.21.15.172:0/2177889877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8a8101b60 0x7fd8a8197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.719+0000 7fd8acc12640 1 --2- 172.21.15.172:0/2177889877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a8102d50 0x7fd8a8198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.719+0000 7fd8adc14640 1 -- 172.21.15.172:0/2177889877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8a8101b60 msgr2=0x7fd8a8197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.719+0000 7fd8adc14640 1 --2- 172.21.15.172:0/2177889877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8a8101b60 0x7fd8a8197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.719+0000 7fd8adc14640 1 -- 172.21.15.172:0/2177889877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a8102d50 msgr2=0x7fd8a8198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.719+0000 7fd8adc14640 1 --2- 172.21.15.172:0/2177889877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a8102d50 0x7fd8a8198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.719+0000 7fd8adc14640 1 -- 172.21.15.172:0/2177889877 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a819f900 con 0x7fd8a8103710 2024-09-10T08:53:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.719+0000 7fd8adc14640 1 --2- 172.21.15.172:0/2177889877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8a8103710 0x7fd8a819f260 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000cad0 tx=0x7fd8a000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.719+0000 7fd89e7fc640 1 -- 172.21.15.172:0/2177889877 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000ae20 con 0x7fd8a8103710 2024-09-10T08:53:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.719+0000 7fd89e7fc640 1 -- 172.21.15.172:0/2177889877 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8a0005020 con 0x7fd8a8103710 2024-09-10T08:53:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.719+0000 7fd8af677640 1 -- 172.21.15.172:0/2177889877 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a819fbf0 con 0x7fd8a8103710 2024-09-10T08:53:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.719+0000 7fd89e7fc640 1 -- 172.21.15.172:0/2177889877 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000f6b0 con 0x7fd8a8103710 2024-09-10T08:53:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.719+0000 7fd8af677640 1 -- 172.21.15.172:0/2177889877 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a81a00f0 con 0x7fd8a8103710 2024-09-10T08:53:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.722+0000 7fd89e7fc640 1 -- 172.21.15.172:0/2177889877 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd8a00051d0 con 0x7fd8a8103710 2024-09-10T08:53:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.722+0000 7fd89e7fc640 1 --2- 172.21.15.172:0/2177889877 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd8940a2c30 0x7fd8940a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.722+0000 7fd8b240a740 1 Processor -- start 2024-09-10T08:53:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8b240a740 1 -- start start 2024-09-10T08:53:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd89e7fc640 1 -- 172.21.15.172:0/2177889877 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd8a00cf530 con 0x7fd8a8103710 2024-09-10T08:53:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8ad413640 1 --2- 172.21.15.172:0/2177889877 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd8940a2c30 0x7fd8940a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8b240a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b2551b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8b240a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b2556f0 0x55b50b25a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8b240a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b25a780 0x55b50b25cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8b240a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b50b1b42f0 con 0x55b50b254db0 2024-09-10T08:53:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8b240a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b50b15c270 con 0x55b50b2556f0 2024-09-10T08:53:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8b240a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b50b1b02b0 con 0x55b50b25a780 2024-09-10T08:53:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8ad413640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b2551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8ad413640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b2551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58588/0 (socket says 172.21.15.172:58588) 2024-09-10T08:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8ad413640 1 -- 172.21.15.172:0/2139404604 learned_addr learned my addr 172.21.15.172:0/2139404604 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8acc12640 1 --2- 172.21.15.172:0/2139404604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b2556f0 0x55b50b25a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8adc14640 1 --2- 172.21.15.172:0/2139404604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b25a780 0x55b50b25cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8ad413640 1 -- 172.21.15.172:0/2139404604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b25a780 msgr2=0x55b50b25cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8ad413640 1 --2- 172.21.15.172:0/2139404604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b25a780 0x55b50b25cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8ad413640 1 -- 172.21.15.172:0/2139404604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b2556f0 msgr2=0x55b50b25a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8ad413640 1 --2- 172.21.15.172:0/2139404604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b2556f0 0x55b50b25a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8ad413640 1 -- 172.21.15.172:0/2139404604 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b50b15b340 con 0x55b50b254db0 2024-09-10T08:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.723+0000 7fd8ad413640 1 --2- 172.21.15.172:0/2177889877 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd8940a2c30 0x7fd8940a50f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40096c0 tx=0x7fd8a4031c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.724+0000 7fd8ad413640 1 --2- 172.21.15.172:0/2139404604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b2551b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009900 tx=0x7fd8a4046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795dbcb18ca9248b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.724+0000 7fd8937fe640 1 -- 172.21.15.172:0/2139404604 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4050070 con 0x55b50b254db0 2024-09-10T08:53:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.724+0000 7fd8937fe640 1 -- 172.21.15.172:0/2139404604 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8a4003aa0 con 0x55b50b254db0 2024-09-10T08:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.724+0000 7fd8937fe640 1 -- 172.21.15.172:0/2139404604 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a404b970 con 0x55b50b254db0 2024-09-10T08:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.725+0000 7fd8b240a740 1 -- 172.21.15.172:0/2139404604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 msgr2=0x55b50b2551b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.725+0000 7fd8b240a740 1 --2- 172.21.15.172:0/2139404604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b2551b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009900 tx=0x7fd8a4046000 comp rx=0 tx=0).stop 2024-09-10T08:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.725+0000 7fd8b240a740 1 -- 172.21.15.172:0/2139404604 shutdown_connections 2024-09-10T08:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.725+0000 7fd8b240a740 1 --2- 172.21.15.172:0/2139404604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b25a780 0x55b50b25cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.725+0000 7fd8b240a740 1 --2- 172.21.15.172:0/2139404604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b2556f0 0x55b50b25a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.725+0000 7fd8b240a740 1 --2- 172.21.15.172:0/2139404604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b2551b0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.725+0000 7fd8b240a740 1 -- 172.21.15.172:0/2139404604 >> 172.21.15.172:0/2139404604 conn(0x55b50b24c010 msgr2=0x55b50b24c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.725+0000 7fd8b240a740 1 -- 172.21.15.172:0/2139404604 shutdown_connections 2024-09-10T08:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.725+0000 7fd8b240a740 1 -- 172.21.15.172:0/2139404604 wait complete. 2024-09-10T08:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.725+0000 7fd8b240a740 1 -- start start 2024-09-10T08:53:58.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.725+0000 7fd8b240a740 1 Processor -- start 2024-09-10T08:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.725+0000 7fd8b240a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b25f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8b240a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b2556f0 0x55b50b25f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8b240a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b25a780 0x55b50b266610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8b240a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b50b25d6f0 con 0x55b50b254db0 2024-09-10T08:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8b240a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b50b13d290 con 0x55b50b2556f0 2024-09-10T08:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8b240a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b50b25d870 con 0x55b50b25a780 2024-09-10T08:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8b240a740 10 client.0 _ll_register_callbacks cb 0x55b50b16e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8ad413640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b25f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8ad413640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b25f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58602/0 (socket says 172.21.15.172:58602) 2024-09-10T08:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8ad413640 1 -- 172.21.15.172:0/3863965027 learned_addr learned my addr 172.21.15.172:0/3863965027 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8acc12640 1 --2- 172.21.15.172:0/3863965027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b2556f0 0x55b50b25f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8adc14640 1 --2- 172.21.15.172:0/3863965027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b25a780 0x55b50b266610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8ad413640 1 -- 172.21.15.172:0/3863965027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b25a780 msgr2=0x55b50b266610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8ad413640 1 --2- 172.21.15.172:0/3863965027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b25a780 0x55b50b266610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8ad413640 1 -- 172.21.15.172:0/3863965027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b2556f0 msgr2=0x55b50b25f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8ad413640 1 --2- 172.21.15.172:0/3863965027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b2556f0 0x55b50b25f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8ad413640 1 -- 172.21.15.172:0/3863965027 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b50aec7210 con 0x55b50b254db0 2024-09-10T08:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8ad413640 1 --2- 172.21.15.172:0/3863965027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b25f090 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd8a404e040 tx=0x7fd8a402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab7b5babb1a9cb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd890ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd890ff9640 1 -- 172.21.15.172:0/3863965027 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4059060 con 0x55b50b254db0 2024-09-10T08:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd890ff9640 1 -- 172.21.15.172:0/3863965027 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd8a40037f0 con 0x55b50b254db0 2024-09-10T08:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd8b240a740 1 -- 172.21.15.172:0/3863965027 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8a4003aa0 con 0x55b50b254db0 2024-09-10T08:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.726+0000 7fd890ff9640 1 -- 172.21.15.172:0/3863965027 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd8a4060050 con 0x55b50b254db0 2024-09-10T08:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.727+0000 7fd8b240a740 10 client.17751 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.727+0000 7fd8b240a740 1 -- 172.21.15.172:0/3863965027 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b50af93620 con 0x55b50b254db0 2024-09-10T08:53:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.727+0000 7fd890ff9640 1 -- 172.21.15.172:0/3863965027 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8a4003aa0 con 0x55b50b254db0 2024-09-10T08:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.727+0000 7fd890ff9640 1 -- 172.21.15.172:0/3863965027 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fd8a4054570 con 0x55b50b254db0 2024-09-10T08:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.727+0000 7fd8b240a740 10 client.17751 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.727+0000 7fd8b240a740 10 client.17751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.727+0000 7fd8b240a740 20 client.17751 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.727+0000 7fd8b240a740 1 --2- 172.21.15.172:0/3863965027 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b50b26a910 0x55b50b26cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.727+0000 7fd8b240a740 4 client.17751 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.727+0000 7fd8b240a740 1 -- 172.21.15.172:0/3863965027 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b50b1c1a20 con 0x55b50b26a910 2024-09-10T08:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.727+0000 7fd8acc12640 1 --2- 172.21.15.172:0/3863965027 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b50b26a910 0x55b50b26cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.728+0000 7fd8acc12640 1 --2- 172.21.15.172:0/3863965027 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b50b26a910 0x55b50b26cd00 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=299a6eac0d46afb1 server_cookie=9651640f99bd40b6 in_seq=0 out_seq=0 2024-09-10T08:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.728+0000 7fd890ff9640 10 client.17751 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.728+0000 7fd890ff9640 1 -- 172.21.15.172:0/3863965027 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b50b1c1a20 con 0x55b50b26a910 2024-09-10T08:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.728+0000 7fd890ff9640 10 client.17751 handle_command_reply: tid=0 2024-09-10T08:53:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.728+0000 7fd8b240a740 1 client.17751 shutdown 2024-09-10T08:53:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.729+0000 7fd8b240a740 1 -- 172.21.15.172:0/3863965027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 msgr2=0x55b50b25f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.729+0000 7fd8b240a740 1 --2- 172.21.15.172:0/3863965027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b25f090 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd8a404e040 tx=0x7fd8a402a410 comp rx=0 tx=0).stop 2024-09-10T08:53:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.729+0000 7fd8b240a740 1 -- 172.21.15.172:0/3863965027 shutdown_connections 2024-09-10T08:53:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.729+0000 7fd8b240a740 1 --2- 172.21.15.172:0/3863965027 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b50b26a910 0x55b50b26cd00 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.729+0000 7fd8b240a740 1 --2- 172.21.15.172:0/3863965027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b25a780 0x55b50b266610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.729+0000 7fd8b240a740 1 --2- 172.21.15.172:0/3863965027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b2556f0 0x55b50b25f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.729+0000 7fd890ff9640 0 client.17751 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.729+0000 7fd8b240a740 1 --2- 172.21.15.172:0/3863965027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b25f090 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.729+0000 7fd8b240a740 1 -- 172.21.15.172:0/3863965027 >> 172.21.15.172:0/3863965027 conn(0x55b50b24c010 msgr2=0x55b50b24d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.729+0000 7fd8b240a740 1 -- 172.21.15.172:0/3863965027 shutdown_connections 2024-09-10T08:53:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.729+0000 7fd8b240a740 1 -- 172.21.15.172:0/3863965027 wait complete. 2024-09-10T08:53:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.729+0000 7fd8b240a740 20 client.17751 trim_cache size 0 max 16384 2024-09-10T08:53:58.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:53:58 smithi174 ceph-mon[34068]: pgmap v582: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:53:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.743+0000 7fd8b240a740 1 Processor -- start 2024-09-10T08:53:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.743+0000 7fd8b240a740 1 -- start start 2024-09-10T08:53:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.743+0000 7fd8b240a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b254db0 0x55b50b25f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.743+0000 7fd8b240a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b2556f0 0x55b50b2665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.743+0000 7fd8b240a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b25a780 0x55b50b24b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.743+0000 7fd8b240a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81a8280 con 0x55b50b2556f0 2024-09-10T08:53:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.743+0000 7fd8b240a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81a9dc0 con 0x55b50b25a780 2024-09-10T08:53:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.743+0000 7fd8b240a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81a8950 con 0x55b50b254db0 2024-09-10T08:53:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.743+0000 7fd8acc12640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b2556f0 0x55b50b2665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.744+0000 7fd8acc12640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b2556f0 0x55b50b2665a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58608/0 (socket says 172.21.15.172:58608) 2024-09-10T08:53:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.744+0000 7fd8acc12640 1 -- 172.21.15.172:0/2778255732 learned_addr learned my addr 172.21.15.172:0/2778255732 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.744+0000 7fd8ad413640 1 --2- 172.21.15.172:0/2778255732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b254db0 0x55b50b25f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.744+0000 7fd8adc14640 1 --2- 172.21.15.172:0/2778255732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b25a780 0x55b50b24b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.744+0000 7fd8acc12640 1 -- 172.21.15.172:0/2778255732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b254db0 msgr2=0x55b50b25f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.744+0000 7fd8acc12640 1 --2- 172.21.15.172:0/2778255732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b254db0 0x55b50b25f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.744+0000 7fd8acc12640 1 -- 172.21.15.172:0/2778255732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b25a780 msgr2=0x55b50b24b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.744+0000 7fd8acc12640 1 --2- 172.21.15.172:0/2778255732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b25a780 0x55b50b24b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.744+0000 7fd8acc12640 1 -- 172.21.15.172:0/2778255732 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b50b25ebd0 con 0x55b50b2556f0 2024-09-10T08:53:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.744+0000 7fd8ad413640 1 --2- 172.21.15.172:0/2778255732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b254db0 0x55b50b25f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:53:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.744+0000 7fd8adc14640 1 --2- 172.21.15.172:0/2778255732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b25a780 0x55b50b24b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:53:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.744+0000 7fd8acc12640 1 --2- 172.21.15.172:0/2778255732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b2556f0 0x55b50b2665a0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x55b50b256120 tx=0x7fd89800cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e4e03092e249b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.744+0000 7fd892ffd640 1 -- 172.21.15.172:0/2778255732 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898021070 con 0x55b50b2556f0 2024-09-10T08:53:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.744+0000 7fd892ffd640 1 -- 172.21.15.172:0/2778255732 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd898003d20 con 0x55b50b2556f0 2024-09-10T08:53:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.744+0000 7fd892ffd640 1 -- 172.21.15.172:0/2778255732 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898014070 con 0x55b50b2556f0 2024-09-10T08:53:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.745+0000 7fd8b240a740 1 -- 172.21.15.172:0/2778255732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b2556f0 msgr2=0x55b50b2665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.745+0000 7fd8b240a740 1 --2- 172.21.15.172:0/2778255732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b2556f0 0x55b50b2665a0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x55b50b256120 tx=0x7fd89800cd00 comp rx=0 tx=0).stop 2024-09-10T08:53:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.745+0000 7fd8b240a740 1 -- 172.21.15.172:0/2778255732 shutdown_connections 2024-09-10T08:53:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.745+0000 7fd8b240a740 1 --2- 172.21.15.172:0/2778255732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b25a780 0x55b50b24b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.745+0000 7fd8b240a740 1 --2- 172.21.15.172:0/2778255732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b2556f0 0x55b50b2665a0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.745+0000 7fd8b240a740 1 --2- 172.21.15.172:0/2778255732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b254db0 0x55b50b25f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.745+0000 7fd8b240a740 1 -- 172.21.15.172:0/2778255732 >> 172.21.15.172:0/2778255732 conn(0x55b50b24c010 msgr2=0x55b50b24a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.745+0000 7fd8b240a740 1 -- 172.21.15.172:0/2778255732 shutdown_connections 2024-09-10T08:53:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.745+0000 7fd8b240a740 1 -- 172.21.15.172:0/2778255732 wait complete. 2024-09-10T08:53:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.745+0000 7fd8b240a740 1 -- start start 2024-09-10T08:53:58.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.745+0000 7fd8b240a740 1 Processor -- start 2024-09-10T08:53:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8b240a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b263f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8b240a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b2556f0 0x55b50b264440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8b240a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b25a780 0x55b50b265450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8b240a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81a96f0 con 0x55b50b254db0 2024-09-10T08:53:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8b240a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81a9020 con 0x55b50b2556f0 2024-09-10T08:53:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8b240a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81a6260 con 0x55b50b25a780 2024-09-10T08:53:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8b240a740 10 client.0 _ll_register_callbacks cb 0x7fd8a4031f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:53:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8ad413640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b263f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8ad413640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b263f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58616/0 (socket says 172.21.15.172:58616) 2024-09-10T08:53:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8ad413640 1 -- 172.21.15.172:0/2162187991 learned_addr learned my addr 172.21.15.172:0/2162187991 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:53:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8acc12640 1 --2- 172.21.15.172:0/2162187991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b2556f0 0x55b50b264440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8ad413640 1 -- 172.21.15.172:0/2162187991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b25a780 msgr2=0x55b50b265450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8ad413640 1 --2- 172.21.15.172:0/2162187991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b25a780 0x55b50b265450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8ad413640 1 -- 172.21.15.172:0/2162187991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b2556f0 msgr2=0x55b50b264440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:53:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8ad413640 1 --2- 172.21.15.172:0/2162187991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b2556f0 0x55b50b264440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8ad413640 1 -- 172.21.15.172:0/2162187991 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b50b265a60 con 0x55b50b254db0 2024-09-10T08:53:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8acc12640 1 --2- 172.21.15.172:0/2162187991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b2556f0 0x55b50b264440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:53:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd8ad413640 1 --2- 172.21.15.172:0/2162187991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b263f00 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4054990 tx=0x7fd8a4054d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5293f912173b9081 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:53:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd890ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:53:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.746+0000 7fd890ff9640 1 -- 172.21.15.172:0/2162187991 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4050070 con 0x55b50b254db0 2024-09-10T08:53:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.747+0000 7fd890ff9640 1 -- 172.21.15.172:0/2162187991 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd8a4003ee0 con 0x55b50b254db0 2024-09-10T08:53:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.747+0000 7fd8b240a740 1 -- 172.21.15.172:0/2162187991 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd898003d20 con 0x55b50b254db0 2024-09-10T08:53:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.747+0000 7fd890ff9640 1 -- 172.21.15.172:0/2162187991 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd8a406d050 con 0x55b50b254db0 2024-09-10T08:53:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.747+0000 7fd8b240a740 10 client.17757 fetch_fsmap learned FSMap version 37 2024-09-10T08:53:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.747+0000 7fd8b240a740 1 -- 172.21.15.172:0/2162187991 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b50b2d5740 con 0x55b50b254db0 2024-09-10T08:53:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.747+0000 7fd890ff9640 1 -- 172.21.15.172:0/2162187991 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd898003d20 con 0x55b50b254db0 2024-09-10T08:53:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.747+0000 7fd890ff9640 1 -- 172.21.15.172:0/2162187991 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fd8a404b440 con 0x55b50b254db0 2024-09-10T08:53:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.747+0000 7fd8b240a740 10 client.17757 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:53:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.747+0000 7fd8b240a740 10 client.17757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:53:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.747+0000 7fd8b240a740 20 client.17757 populate_metadata read hostname 'smithi172' 2024-09-10T08:53:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.747+0000 7fd8b240a740 1 --2- 172.21.15.172:0/2162187991 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b50b2d6120 0x55b50b2d8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:53:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.747+0000 7fd8b240a740 4 client.17757 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:53:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.747+0000 7fd8b240a740 1 -- 172.21.15.172:0/2162187991 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b50b1bbd70 con 0x55b50b2d6120 2024-09-10T08:53:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.747+0000 7fd8acc12640 1 --2- 172.21.15.172:0/2162187991 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b50b2d6120 0x55b50b2d8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:53:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.748+0000 7fd890ff9640 10 client.17757 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:58.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.748+0000 7fd8acc12640 1 --2- 172.21.15.172:0/2162187991 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b50b2d6120 0x55b50b2d8540 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55a7d47ca4407ff2 server_cookie=b564567dc74641ad in_seq=0 out_seq=0 2024-09-10T08:53:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.755+0000 7fd890ff9640 1 -- 172.21.15.172:0/2162187991 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b50b1bbd70 con 0x55b50b2d6120 2024-09-10T08:53:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.755+0000 7fd890ff9640 10 client.17757 handle_command_reply: tid=0 2024-09-10T08:53:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.755+0000 7fd8b240a740 1 client.17757 shutdown 2024-09-10T08:53:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.755+0000 7fd8b240a740 1 -- 172.21.15.172:0/2162187991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 msgr2=0x55b50b263f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.755+0000 7fd8b240a740 1 --2- 172.21.15.172:0/2162187991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b263f00 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4054990 tx=0x7fd8a4054d90 comp rx=0 tx=0).stop 2024-09-10T08:53:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.755+0000 7fd8b240a740 1 -- 172.21.15.172:0/2162187991 shutdown_connections 2024-09-10T08:53:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.755+0000 7fd8b240a740 1 --2- 172.21.15.172:0/2162187991 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b50b2d6120 0x55b50b2d8540 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.755+0000 7fd8b240a740 1 --2- 172.21.15.172:0/2162187991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b50b25a780 0x55b50b265450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.755+0000 7fd8b240a740 1 --2- 172.21.15.172:0/2162187991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b50b2556f0 0x55b50b264440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.755+0000 7fd8b240a740 1 --2- 172.21.15.172:0/2162187991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b50b254db0 0x55b50b263f00 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.755+0000 7fd890ff9640 0 client.17757 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:53:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.755+0000 7fd8b240a740 1 -- 172.21.15.172:0/2162187991 >> 172.21.15.172:0/2162187991 conn(0x55b50b24c010 msgr2=0x55b50b24b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.755+0000 7fd8b240a740 1 -- 172.21.15.172:0/2162187991 shutdown_connections 2024-09-10T08:53:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.755+0000 7fd8b240a740 1 -- 172.21.15.172:0/2162187991 wait complete. 2024-09-10T08:53:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.755+0000 7fd8b240a740 20 client.17757 trim_cache size 0 max 16384 2024-09-10T08:53:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:53:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:53:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:53:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:53:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.756+0000 7fd8af677640 1 -- 172.21.15.172:0/2177889877 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd8940a2c30 msgr2=0x7fd8940a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.756+0000 7fd8af677640 1 --2- 172.21.15.172:0/2177889877 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd8940a2c30 0x7fd8940a50f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40096c0 tx=0x7fd8a4031c80 comp rx=0 tx=0).stop 2024-09-10T08:53:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.756+0000 7fd8af677640 1 -- 172.21.15.172:0/2177889877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8a8103710 msgr2=0x7fd8a819f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:53:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.756+0000 7fd8af677640 1 --2- 172.21.15.172:0/2177889877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8a8103710 0x7fd8a819f260 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000cad0 tx=0x7fd8a000cf90 comp rx=0 tx=0).stop 2024-09-10T08:53:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.756+0000 7fd8af677640 1 -- 172.21.15.172:0/2177889877 shutdown_connections 2024-09-10T08:53:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.756+0000 7fd8af677640 1 --2- 172.21.15.172:0/2177889877 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd8940a2c30 0x7fd8940a50f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.756+0000 7fd8af677640 1 --2- 172.21.15.172:0/2177889877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8a8103710 0x7fd8a819f260 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.756+0000 7fd8af677640 1 --2- 172.21.15.172:0/2177889877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a8102d50 0x7fd8a8198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.756+0000 7fd8af677640 1 --2- 172.21.15.172:0/2177889877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8a8101b60 0x7fd8a8197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:53:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.756+0000 7fd8af677640 1 -- 172.21.15.172:0/2177889877 >> 172.21.15.172:0/2177889877 conn(0x7fd8a80fd2e0 msgr2=0x7fd8a81060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:53:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.756+0000 7fd8af677640 1 -- 172.21.15.172:0/2177889877 shutdown_connections 2024-09-10T08:53:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:53:58.756+0000 7fd8af677640 1 -- 172.21.15.172:0/2177889877 wait complete. 2024-09-10T08:53:58.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:53:58 smithi191 ceph-mon[35314]: pgmap v582: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:53:58.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:53:58 smithi172 ceph-mon[30854]: pgmap v582: 129 pgs: 129 active+clean; 6.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:54:00.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:00 smithi174 ceph-mon[34068]: pgmap v583: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:54:00.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:00 smithi191 ceph-mon[35314]: pgmap v583: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:54:00.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:00 smithi172 ceph-mon[30854]: pgmap v583: 129 pgs: 129 active+clean; 6.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:54:02.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:02 smithi174 ceph-mon[34068]: pgmap v584: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-10T08:54:02.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:02 smithi191 ceph-mon[35314]: pgmap v584: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-10T08:54:02.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:02 smithi172 ceph-mon[30854]: pgmap v584: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-10T08:54:04.602 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:04 smithi191 ceph-mon[35314]: pgmap v585: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:54:04.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:04 smithi174 ceph-mon[34068]: pgmap v585: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:54:04.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:04 smithi172 ceph-mon[30854]: pgmap v585: 129 pgs: 129 active+clean; 6.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:54:06.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:06 smithi174 ceph-mon[34068]: pgmap v586: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 35 MiB/s wr, 27 op/s 2024-09-10T08:54:06.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:06 smithi191 ceph-mon[35314]: pgmap v586: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 35 MiB/s wr, 27 op/s 2024-09-10T08:54:06.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:06 smithi172 ceph-mon[30854]: pgmap v586: 129 pgs: 129 active+clean; 6.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 35 MiB/s wr, 27 op/s 2024-09-10T08:54:08.634 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:08 smithi172 ceph-mon[30854]: pgmap v587: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-09-10T08:54:08.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:08 smithi174 ceph-mon[34068]: pgmap v587: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-09-10T08:54:08.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:08 smithi191 ceph-mon[35314]: pgmap v587: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 MiB/s wr, 22 op/s 2024-09-10T08:54:10.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:10 smithi174 ceph-mon[34068]: pgmap v588: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 22 op/s 2024-09-10T08:54:10.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:10 smithi191 ceph-mon[35314]: pgmap v588: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 22 op/s 2024-09-10T08:54:10.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:10 smithi172 ceph-mon[30854]: pgmap v588: 129 pgs: 129 active+clean; 6.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 22 op/s 2024-09-10T08:54:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.849+0000 7f53c8e3c640 1 -- 172.21.15.172:0/1974987077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53c41078c0 msgr2=0x7f53c4109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.849+0000 7f53c8e3c640 1 --2- 172.21.15.172:0/1974987077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53c41078c0 0x7f53c4109cb0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009ca0 tx=0x7f53bc02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.850+0000 7f53c8e3c640 1 -- 172.21.15.172:0/1974987077 shutdown_connections 2024-09-10T08:54:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.850+0000 7f53c8e3c640 1 --2- 172.21.15.172:0/1974987077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53c41078c0 0x7f53c4109cb0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.850+0000 7f53c8e3c640 1 --2- 172.21.15.172:0/1974987077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53c4104f90 0x7f53c4107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.850+0000 7f53c8e3c640 1 --2- 172.21.15.172:0/1974987077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53c4102660 0x7f53c4104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.850+0000 7f53c8e3c640 1 -- 172.21.15.172:0/1974987077 >> 172.21.15.172:0/1974987077 conn(0x7f53c40fc1e0 msgr2=0x7f53c40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.850+0000 7f53c8e3c640 1 -- 172.21.15.172:0/1974987077 shutdown_connections 2024-09-10T08:54:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.850+0000 7f53c8e3c640 1 -- 172.21.15.172:0/1974987077 wait complete. 2024-09-10T08:54:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.850+0000 7f53c8e3c640 1 Processor -- start 2024-09-10T08:54:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.850+0000 7f53c8e3c640 1 -- start start 2024-09-10T08:54:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c8e3c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53c4102660 0x7f53c4197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c8e3c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53c4104f90 0x7f53c4198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c8e3c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53c41078c0 0x7f53c419f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c8e3c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f53c410d6b0 con 0x7f53c4104f90 2024-09-10T08:54:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c8e3c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f53c410d530 con 0x7f53c4102660 2024-09-10T08:54:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c8e3c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f53c410d830 con 0x7f53c41078c0 2024-09-10T08:54:11.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53c4104f90 0x7f53c4198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53c4104f90 0x7f53c4198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34668/0 (socket says 172.21.15.172:34668) 2024-09-10T08:54:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c1d9b640 1 -- 172.21.15.172:0/673304991 learned_addr learned my addr 172.21.15.172:0/673304991 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c259c640 1 --2- 172.21.15.172:0/673304991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53c4102660 0x7f53c4197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c2d9d640 1 --2- 172.21.15.172:0/673304991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53c41078c0 0x7f53c419f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c1d9b640 1 -- 172.21.15.172:0/673304991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53c41078c0 msgr2=0x7f53c419f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c1d9b640 1 --2- 172.21.15.172:0/673304991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53c41078c0 0x7f53c419f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c1d9b640 1 -- 172.21.15.172:0/673304991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53c4102660 msgr2=0x7f53c4197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c1d9b640 1 --2- 172.21.15.172:0/673304991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53c4102660 0x7f53c4197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c1d9b640 1 -- 172.21.15.172:0/673304991 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c419f950 con 0x7f53c4104f90 2024-09-10T08:54:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c2d9d640 1 --2- 172.21.15.172:0/673304991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53c41078c0 0x7f53c419f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:54:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53c1d9b640 1 --2- 172.21.15.172:0/673304991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53c4104f90 0x7f53c4198280 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f53b800d740 tx=0x7f53b800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.851+0000 7f53b37fe640 1 -- 172.21.15.172:0/673304991 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b8017070 con 0x7f53c4104f90 2024-09-10T08:54:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.852+0000 7f53b37fe640 1 -- 172.21.15.172:0/673304991 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53b800de50 con 0x7f53c4104f90 2024-09-10T08:54:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.852+0000 7f53c8e3c640 1 -- 172.21.15.172:0/673304991 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c419fc40 con 0x7f53c4104f90 2024-09-10T08:54:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.852+0000 7f53b37fe640 1 -- 172.21.15.172:0/673304991 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b8004d60 con 0x7f53c4104f90 2024-09-10T08:54:11.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.852+0000 7f53c8e3c640 1 -- 172.21.15.172:0/673304991 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c41a0190 con 0x7f53c4104f90 2024-09-10T08:54:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.854+0000 7f53cbbd5740 1 Processor -- start 2024-09-10T08:54:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.854+0000 7f53cbbd5740 1 -- start start 2024-09-10T08:54:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.854+0000 7f53cbbd5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad47fdb0 0x5555ad4801b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53cbbd5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad4806f0 0x5555ad4851b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53cbbd5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad485780 0x5555ad487c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53cbbd5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5555ad3df2f0 con 0x5555ad47fdb0 2024-09-10T08:54:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53cbbd5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5555ad387270 con 0x5555ad485780 2024-09-10T08:54:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53cbbd5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5555ad3db2b0 con 0x5555ad4806f0 2024-09-10T08:54:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53b37fe640 1 -- 172.21.15.172:0/673304991 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f53b8004f00 con 0x7f53c4104f90 2024-09-10T08:54:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53c2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad47fdb0 0x5555ad4801b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53c2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad47fdb0 0x5555ad4801b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34684/0 (socket says 172.21.15.172:34684) 2024-09-10T08:54:11.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53c2d9d640 1 -- 172.21.15.172:0/2256129407 learned_addr learned my addr 172.21.15.172:0/2256129407 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53c259c640 1 --2- 172.21.15.172:0/2256129407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad4806f0 0x5555ad4851b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53c1d9b640 1 --2- 172.21.15.172:0/2256129407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad485780 0x5555ad487c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53c2d9d640 1 -- 172.21.15.172:0/2256129407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad4806f0 msgr2=0x5555ad4851b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53c2d9d640 1 --2- 172.21.15.172:0/2256129407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad4806f0 0x5555ad4851b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53c2d9d640 1 -- 172.21.15.172:0/2256129407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad485780 msgr2=0x5555ad487c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53c2d9d640 1 --2- 172.21.15.172:0/2256129407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad485780 0x5555ad487c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53c2d9d640 1 -- 172.21.15.172:0/2256129407 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ad386340 con 0x5555ad47fdb0 2024-09-10T08:54:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53c1d9b640 1 --2- 172.21.15.172:0/2256129407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad485780 0x5555ad487c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:54:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53c2d9d640 1 --2- 172.21.15.172:0/2256129407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad47fdb0 0x5555ad4801b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f53bc035000 tx=0x7f53bc009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e396f42008b4c82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.855+0000 7f53b37fe640 1 --2- 172.21.15.172:0/673304991 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53ac0a2cb0 0x7f53ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.856+0000 7f53b0ff9640 1 -- 172.21.15.172:0/2256129407 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc031b40 con 0x5555ad47fdb0 2024-09-10T08:54:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.856+0000 7f53c259c640 1 --2- 172.21.15.172:0/673304991 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53ac0a2cb0 0x7f53ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.856+0000 7f53b0ff9640 1 -- 172.21.15.172:0/2256129407 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53bc036040 con 0x5555ad47fdb0 2024-09-10T08:54:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.856+0000 7f53b0ff9640 1 -- 172.21.15.172:0/2256129407 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0044e0 con 0x5555ad47fdb0 2024-09-10T08:54:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.856+0000 7f53b37fe640 1 -- 172.21.15.172:0/673304991 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f53b80cedb0 con 0x7f53c4104f90 2024-09-10T08:54:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.856+0000 7f53cbbd5740 1 -- 172.21.15.172:0/2256129407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad47fdb0 msgr2=0x5555ad4801b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.856+0000 7f53c259c640 1 --2- 172.21.15.172:0/673304991 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53ac0a2cb0 0x7f53ac0a5170 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x5555ad481120 tx=0x7f53b400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.856+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/2256129407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad47fdb0 0x5555ad4801b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f53bc035000 tx=0x7f53bc009a20 comp rx=0 tx=0).stop 2024-09-10T08:54:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.857+0000 7f53cbbd5740 1 -- 172.21.15.172:0/2256129407 shutdown_connections 2024-09-10T08:54:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.857+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/2256129407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad485780 0x5555ad487c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.857+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/2256129407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad4806f0 0x5555ad4851b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.857+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/2256129407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad47fdb0 0x5555ad4801b0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.857+0000 7f53cbbd5740 1 -- 172.21.15.172:0/2256129407 >> 172.21.15.172:0/2256129407 conn(0x5555ad477010 msgr2=0x5555ad477410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.857+0000 7f53cbbd5740 1 -- 172.21.15.172:0/2256129407 shutdown_connections 2024-09-10T08:54:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.857+0000 7f53cbbd5740 1 -- 172.21.15.172:0/2256129407 wait complete. 2024-09-10T08:54:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.857+0000 7f53cbbd5740 1 -- start start 2024-09-10T08:54:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.857+0000 7f53cbbd5740 1 Processor -- start 2024-09-10T08:54:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.859+0000 7f53cbbd5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad47fdb0 0x5555ad489fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.859+0000 7f53cbbd5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad4806f0 0x5555ad48a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.859+0000 7f53cbbd5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad485780 0x5555ad491520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.859+0000 7f53cbbd5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5555ad4886f0 con 0x5555ad4806f0 2024-09-10T08:54:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.859+0000 7f53cbbd5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5555ad368290 con 0x5555ad485780 2024-09-10T08:54:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.859+0000 7f53cbbd5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5555ad488870 con 0x5555ad47fdb0 2024-09-10T08:54:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.859+0000 7f53c1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad4806f0 0x5555ad48a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.859+0000 7f53cbbd5740 10 client.0 _ll_register_callbacks cb 0x5555ad399140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.859+0000 7f53c1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad4806f0 0x5555ad48a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34700/0 (socket says 172.21.15.172:34700) 2024-09-10T08:54:11.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.859+0000 7f53c1d9b640 1 -- 172.21.15.172:0/3461926198 learned_addr learned my addr 172.21.15.172:0/3461926198 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.859+0000 7f53c1d9b640 1 -- 172.21.15.172:0/3461926198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad47fdb0 msgr2=0x5555ad489fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.859+0000 7f53c1d9b640 1 --2- 172.21.15.172:0/3461926198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad47fdb0 0x5555ad489fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.859+0000 7f53c1d9b640 1 -- 172.21.15.172:0/3461926198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad485780 msgr2=0x5555ad491520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.859+0000 7f53c1d9b640 1 --2- 172.21.15.172:0/3461926198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad485780 0x5555ad491520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.859+0000 7f53c1d9b640 1 -- 172.21.15.172:0/3461926198 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ad0f2210 con 0x5555ad4806f0 2024-09-10T08:54:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.859+0000 7f53c1d9b640 1 --2- 172.21.15.172:0/3461926198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad4806f0 0x5555ad48a4e0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f53b8005d00 tx=0x7f53b80993f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d988cff5e7cc7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.859+0000 7f53a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.860+0000 7f53a9ffb640 1 -- 172.21.15.172:0/3461926198 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b8020070 con 0x5555ad4806f0 2024-09-10T08:54:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.860+0000 7f53cbbd5740 1 -- 172.21.15.172:0/3461926198 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53bc036040 con 0x5555ad4806f0 2024-09-10T08:54:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.860+0000 7f53a9ffb640 1 -- 172.21.15.172:0/3461926198 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f53b80a3c90 con 0x5555ad4806f0 2024-09-10T08:54:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.860+0000 7f53a9ffb640 1 -- 172.21.15.172:0/3461926198 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f53b809b9b0 con 0x5555ad4806f0 2024-09-10T08:54:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.860+0000 7f53cbbd5740 10 client.17769 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.860+0000 7f53cbbd5740 1 -- 172.21.15.172:0/3461926198 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5555ad1be620 con 0x5555ad4806f0 2024-09-10T08:54:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.860+0000 7f53a9ffb640 1 -- 172.21.15.172:0/3461926198 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53bc036040 con 0x5555ad4806f0 2024-09-10T08:54:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.860+0000 7f53a9ffb640 1 -- 172.21.15.172:0/3461926198 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f53b80bb0b0 con 0x5555ad4806f0 2024-09-10T08:54:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.860+0000 7f53cbbd5740 10 client.17769 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.860+0000 7f53cbbd5740 10 client.17769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:54:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.860+0000 7f53cbbd5740 20 client.17769 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.860+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/3461926198 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5555ad495bd0 0x5555ad498020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.860+0000 7f53cbbd5740 4 client.17769 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:54:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.860+0000 7f53cbbd5740 1 -- 172.21.15.172:0/3461926198 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555ad3eca20 con 0x5555ad495bd0 2024-09-10T08:54:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.861+0000 7f53c259c640 1 --2- 172.21.15.172:0/3461926198 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5555ad495bd0 0x5555ad498020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.861+0000 7f53a9ffb640 10 client.17769 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.861+0000 7f53c259c640 1 --2- 172.21.15.172:0/3461926198 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5555ad495bd0 0x5555ad498020 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=379f532c9b9e8cb7 server_cookie=c2a377a1e97c565 in_seq=0 out_seq=0 2024-09-10T08:54:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.862+0000 7f53a9ffb640 1 -- 172.21.15.172:0/3461926198 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555ad3eca20 con 0x5555ad495bd0 2024-09-10T08:54:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.862+0000 7f53a9ffb640 10 client.17769 handle_command_reply: tid=0 2024-09-10T08:54:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.862+0000 7f53cbbd5740 1 client.17769 shutdown 2024-09-10T08:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.862+0000 7f53cbbd5740 1 -- 172.21.15.172:0/3461926198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad4806f0 msgr2=0x5555ad48a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.862+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/3461926198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad4806f0 0x5555ad48a4e0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f53b8005d00 tx=0x7f53b80993f0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.862+0000 7f53cbbd5740 1 -- 172.21.15.172:0/3461926198 shutdown_connections 2024-09-10T08:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.862+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/3461926198 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5555ad495bd0 0x5555ad498020 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.862+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/3461926198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad485780 0x5555ad491520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.862+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/3461926198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad4806f0 0x5555ad48a4e0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.862+0000 7f53a9ffb640 0 client.17769 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.862+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/3461926198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad47fdb0 0x5555ad489fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.862+0000 7f53cbbd5740 1 -- 172.21.15.172:0/3461926198 >> 172.21.15.172:0/3461926198 conn(0x5555ad477010 msgr2=0x5555ad478820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.862+0000 7f53cbbd5740 1 -- 172.21.15.172:0/3461926198 shutdown_connections 2024-09-10T08:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.862+0000 7f53cbbd5740 1 -- 172.21.15.172:0/3461926198 wait complete. 2024-09-10T08:54:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.863+0000 7f53cbbd5740 20 client.17769 trim_cache size 0 max 16384 2024-09-10T08:54:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.881+0000 7f53cbbd5740 1 Processor -- start 2024-09-10T08:54:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.881+0000 7f53cbbd5740 1 -- start start 2024-09-10T08:54:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.882+0000 7f53cbbd5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad477010 0x5555ad477ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.882+0000 7f53cbbd5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad47fdb0 0x5555ad4914e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.882+0000 7f53cbbd5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad4806f0 0x5555ad478400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.882+0000 7f53cbbd5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41a8380 con 0x5555ad477010 2024-09-10T08:54:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.882+0000 7f53cbbd5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41a9ec0 con 0x5555ad4806f0 2024-09-10T08:54:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.882+0000 7f53cbbd5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41a8a50 con 0x5555ad47fdb0 2024-09-10T08:54:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.882+0000 7f53c259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad477010 0x5555ad477ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.882+0000 7f53c259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad477010 0x5555ad477ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34702/0 (socket says 172.21.15.172:34702) 2024-09-10T08:54:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.882+0000 7f53c259c640 1 -- 172.21.15.172:0/1501441490 learned_addr learned my addr 172.21.15.172:0/1501441490 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.882+0000 7f53c1d9b640 1 --2- 172.21.15.172:0/1501441490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad47fdb0 0x5555ad4914e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.882+0000 7f53c2d9d640 1 --2- 172.21.15.172:0/1501441490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad4806f0 0x5555ad478400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.882+0000 7f53c259c640 1 -- 172.21.15.172:0/1501441490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad47fdb0 msgr2=0x5555ad4914e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.882+0000 7f53c259c640 1 --2- 172.21.15.172:0/1501441490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad47fdb0 0x5555ad4914e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.882+0000 7f53c259c640 1 -- 172.21.15.172:0/1501441490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad4806f0 msgr2=0x5555ad478400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.882+0000 7f53c259c640 1 --2- 172.21.15.172:0/1501441490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad4806f0 0x5555ad478400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.882+0000 7f53c259c640 1 -- 172.21.15.172:0/1501441490 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ad4986a0 con 0x5555ad477010 2024-09-10T08:54:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.882+0000 7f53c259c640 1 --2- 172.21.15.172:0/1501441490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad477010 0x5555ad477ec0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x5555ad477e90 tx=0x7f53b4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8c8f4719f35e0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.883+0000 7f53abfff640 1 -- 172.21.15.172:0/1501441490 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b401e950 con 0x5555ad477010 2024-09-10T08:54:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.883+0000 7f53abfff640 1 -- 172.21.15.172:0/1501441490 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53b40272c0 con 0x5555ad477010 2024-09-10T08:54:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.883+0000 7f53abfff640 1 -- 172.21.15.172:0/1501441490 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b401d670 con 0x5555ad477010 2024-09-10T08:54:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.883+0000 7f53cbbd5740 1 -- 172.21.15.172:0/1501441490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad477010 msgr2=0x5555ad477ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.883+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/1501441490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad477010 0x5555ad477ec0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x5555ad477e90 tx=0x7f53b4004610 comp rx=0 tx=0).stop 2024-09-10T08:54:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.883+0000 7f53cbbd5740 1 -- 172.21.15.172:0/1501441490 shutdown_connections 2024-09-10T08:54:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.883+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/1501441490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad4806f0 0x5555ad478400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.883+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/1501441490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad47fdb0 0x5555ad4914e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.883+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/1501441490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad477010 0x5555ad477ec0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.883+0000 7f53cbbd5740 1 -- 172.21.15.172:0/1501441490 >> 172.21.15.172:0/1501441490 conn(0x5555ad4852a0 msgr2=0x5555ad4818d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.883+0000 7f53cbbd5740 1 -- 172.21.15.172:0/1501441490 shutdown_connections 2024-09-10T08:54:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.883+0000 7f53cbbd5740 1 -- 172.21.15.172:0/1501441490 wait complete. 2024-09-10T08:54:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53cbbd5740 1 -- start start 2024-09-10T08:54:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53cbbd5740 1 Processor -- start 2024-09-10T08:54:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53cbbd5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad477010 0x5555ad48ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53cbbd5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad47fdb0 0x5555ad48f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53cbbd5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad4806f0 0x5555ad490220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53cbbd5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41a97f0 con 0x5555ad47fdb0 2024-09-10T08:54:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53cbbd5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41a9120 con 0x5555ad4806f0 2024-09-10T08:54:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53cbbd5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41a6360 con 0x5555ad477010 2024-09-10T08:54:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53cbbd5740 10 client.0 _ll_register_callbacks cb 0x7f53b8009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53c1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad47fdb0 0x5555ad48f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53c1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad47fdb0 0x5555ad48f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34712/0 (socket says 172.21.15.172:34712) 2024-09-10T08:54:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53c1d9b640 1 -- 172.21.15.172:0/541998123 learned_addr learned my addr 172.21.15.172:0/541998123 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53c2d9d640 1 --2- 172.21.15.172:0/541998123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad4806f0 0x5555ad490220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53c259c640 1 --2- 172.21.15.172:0/541998123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad477010 0x5555ad48ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53c1d9b640 1 -- 172.21.15.172:0/541998123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad477010 msgr2=0x5555ad48ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53c1d9b640 1 --2- 172.21.15.172:0/541998123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad477010 0x5555ad48ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53c1d9b640 1 -- 172.21.15.172:0/541998123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad4806f0 msgr2=0x5555ad490220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53c1d9b640 1 --2- 172.21.15.172:0/541998123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad4806f0 0x5555ad490220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.884+0000 7f53c1d9b640 1 -- 172.21.15.172:0/541998123 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ad490830 con 0x5555ad47fdb0 2024-09-10T08:54:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.885+0000 7f53c259c640 1 --2- 172.21.15.172:0/541998123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad477010 0x5555ad48ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:54:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.885+0000 7f53c1d9b640 1 --2- 172.21.15.172:0/541998123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad47fdb0 0x5555ad48f210 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f53b809afb0 tx=0x7f53b8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114e0a01efae3dfb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.885+0000 7f53a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.885+0000 7f53a9ffb640 1 -- 172.21.15.172:0/541998123 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b8014030 con 0x5555ad47fdb0 2024-09-10T08:54:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.885+0000 7f53a9ffb640 1 -- 172.21.15.172:0/541998123 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f53b80b9070 con 0x5555ad47fdb0 2024-09-10T08:54:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.885+0000 7f53cbbd5740 1 -- 172.21.15.172:0/541998123 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53b40272c0 con 0x5555ad47fdb0 2024-09-10T08:54:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.885+0000 7f53a9ffb640 1 -- 172.21.15.172:0/541998123 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f53b8083050 con 0x5555ad47fdb0 2024-09-10T08:54:11.891 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.885+0000 7f53cbbd5740 10 client.17775 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.885+0000 7f53cbbd5740 1 -- 172.21.15.172:0/541998123 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5555ad500a60 con 0x5555ad47fdb0 2024-09-10T08:54:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.885+0000 7f53a9ffb640 1 -- 172.21.15.172:0/541998123 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53b40272c0 con 0x5555ad47fdb0 2024-09-10T08:54:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.885+0000 7f53a9ffb640 1 -- 172.21.15.172:0/541998123 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f53b80a3a90 con 0x5555ad47fdb0 2024-09-10T08:54:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.885+0000 7f53cbbd5740 10 client.17775 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.885+0000 7f53cbbd5740 10 client.17775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:54:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.885+0000 7f53cbbd5740 20 client.17775 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.885+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/541998123 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5555ad485780 0x5555ad5034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.885+0000 7f53cbbd5740 4 client.17775 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:54:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.885+0000 7f53cbbd5740 1 -- 172.21.15.172:0/541998123 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5555ad3e6d70 con 0x5555ad485780 2024-09-10T08:54:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.886+0000 7f53c259c640 1 --2- 172.21.15.172:0/541998123 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5555ad485780 0x5555ad5034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.886+0000 7f53a9ffb640 10 client.17775 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.886+0000 7f53c259c640 1 --2- 172.21.15.172:0/541998123 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5555ad485780 0x5555ad5034a0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1f158476ff2dee2 server_cookie=4ed1f338b8278f47 in_seq=0 out_seq=0 2024-09-10T08:54:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.891+0000 7f53a9ffb640 1 -- 172.21.15.172:0/541998123 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5555ad3e6d70 con 0x5555ad485780 2024-09-10T08:54:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.891+0000 7f53a9ffb640 10 client.17775 handle_command_reply: tid=0 2024-09-10T08:54:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.891+0000 7f53cbbd5740 1 client.17775 shutdown 2024-09-10T08:54:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.891+0000 7f53cbbd5740 1 -- 172.21.15.172:0/541998123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad47fdb0 msgr2=0x5555ad48f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.891+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/541998123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad47fdb0 0x5555ad48f210 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f53b809afb0 tx=0x7f53b8009c30 comp rx=0 tx=0).stop 2024-09-10T08:54:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53cbbd5740 1 -- 172.21.15.172:0/541998123 shutdown_connections 2024-09-10T08:54:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/541998123 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5555ad485780 0x5555ad5034a0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/541998123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555ad4806f0 0x5555ad490220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/541998123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ad47fdb0 0x5555ad48f210 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53a9ffb640 0 client.17775 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53cbbd5740 1 --2- 172.21.15.172:0/541998123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555ad477010 0x5555ad48ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53cbbd5740 1 -- 172.21.15.172:0/541998123 >> 172.21.15.172:0/541998123 conn(0x5555ad4852a0 msgr2=0x5555ad4759a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53cbbd5740 1 -- 172.21.15.172:0/541998123 shutdown_connections 2024-09-10T08:54:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53cbbd5740 1 -- 172.21.15.172:0/541998123 wait complete. 2024-09-10T08:54:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53cbbd5740 20 client.17775 trim_cache size 0 max 16384 2024-09-10T08:54:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:54:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:54:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:54:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:54:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53c8e3c640 1 -- 172.21.15.172:0/673304991 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53ac0a2cb0 msgr2=0x7f53ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53c8e3c640 1 --2- 172.21.15.172:0/673304991 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53ac0a2cb0 0x7f53ac0a5170 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x5555ad481120 tx=0x7f53b400b390 comp rx=0 tx=0).stop 2024-09-10T08:54:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53c8e3c640 1 -- 172.21.15.172:0/673304991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53c4104f90 msgr2=0x7f53c4198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53c8e3c640 1 --2- 172.21.15.172:0/673304991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53c4104f90 0x7f53c4198280 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f53b800d740 tx=0x7f53b800dc00 comp rx=0 tx=0).stop 2024-09-10T08:54:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53c8e3c640 1 -- 172.21.15.172:0/673304991 shutdown_connections 2024-09-10T08:54:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53c8e3c640 1 --2- 172.21.15.172:0/673304991 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53ac0a2cb0 0x7f53ac0a5170 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53c8e3c640 1 --2- 172.21.15.172:0/673304991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53c41078c0 0x7f53c419f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53c8e3c640 1 --2- 172.21.15.172:0/673304991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53c4104f90 0x7f53c4198280 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53c8e3c640 1 --2- 172.21.15.172:0/673304991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53c4102660 0x7f53c4197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.892+0000 7f53c8e3c640 1 -- 172.21.15.172:0/673304991 >> 172.21.15.172:0/673304991 conn(0x7f53c40fc1e0 msgr2=0x7f53c4102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.893+0000 7f53c8e3c640 1 -- 172.21.15.172:0/673304991 shutdown_connections 2024-09-10T08:54:11.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:11.893+0000 7f53c8e3c640 1 -- 172.21.15.172:0/673304991 wait complete. 2024-09-10T08:54:12.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:12 smithi174 ceph-mon[34068]: pgmap v589: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-10T08:54:12.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:12 smithi191 ceph-mon[35314]: pgmap v589: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-10T08:54:12.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:12 smithi172 ceph-mon[30854]: pgmap v589: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 33 MiB/s wr, 26 op/s 2024-09-10T08:54:14.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:14 smithi174 ceph-mon[34068]: pgmap v590: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-09-10T08:54:14.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:14 smithi191 ceph-mon[35314]: pgmap v590: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-09-10T08:54:14.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:14 smithi172 ceph-mon[30854]: pgmap v590: 129 pgs: 129 active+clean; 6.9 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-09-10T08:54:16.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:16 smithi174 ceph-mon[34068]: pgmap v591: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 27 op/s 2024-09-10T08:54:16.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:16 smithi191 ceph-mon[35314]: pgmap v591: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 27 op/s 2024-09-10T08:54:16.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:16 smithi172 ceph-mon[30854]: pgmap v591: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 27 op/s 2024-09-10T08:54:17.176 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:54:17.467 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:54:18.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.037+0000 7f36f6a22640 1 -- 172.21.15.172:0/2698366129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36f0102570 msgr2=0x7f36f01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:18.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.037+0000 7f36f6a22640 1 --2- 172.21.15.172:0/2698366129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36f0102570 0x7f36f01029b0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009900 tx=0x7f36e402a480 comp rx=0 tx=0).stop 2024-09-10T08:54:18.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.038+0000 7f36f6a22640 1 -- 172.21.15.172:0/2698366129 shutdown_connections 2024-09-10T08:54:18.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.038+0000 7f36f6a22640 1 --2- 172.21.15.172:0/2698366129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36f0102ef0 0x7f36f010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.038+0000 7f36f6a22640 1 --2- 172.21.15.172:0/2698366129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36f0102570 0x7f36f01029b0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.038+0000 7f36f6a22640 1 --2- 172.21.15.172:0/2698366129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36f0100e20 0x7f36f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.038+0000 7f36f6a22640 1 -- 172.21.15.172:0/2698366129 >> 172.21.15.172:0/2698366129 conn(0x7f36f00fc7d0 msgr2=0x7f36f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:18.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.038+0000 7f36f6a22640 1 -- 172.21.15.172:0/2698366129 shutdown_connections 2024-09-10T08:54:18.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.038+0000 7f36f6a22640 1 -- 172.21.15.172:0/2698366129 wait complete. 2024-09-10T08:54:18.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.038+0000 7f36f6a22640 1 Processor -- start 2024-09-10T08:54:18.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.038+0000 7f36f6a22640 1 -- start start 2024-09-10T08:54:18.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.039+0000 7f36f6a22640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36f0100e20 0x7f36f0199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.039+0000 7f36f6a22640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36f0102570 0x7f36f0199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.039+0000 7f36f6a22640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36f0102ef0 0x7f36f01a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.039+0000 7f36f6a22640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f36f0110e30 con 0x7f36f0100e20 2024-09-10T08:54:18.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.039+0000 7f36f6a22640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36f0110cb0 con 0x7f36f0102ef0 2024-09-10T08:54:18.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.039+0000 7f36f6a22640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f36f0110fb0 con 0x7f36f0102570 2024-09-10T08:54:18.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.039+0000 7f36effff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36f0100e20 0x7f36f0199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.039+0000 7f36effff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36f0100e20 0x7f36f0199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57786/0 (socket says 172.21.15.172:57786) 2024-09-10T08:54:18.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.039+0000 7f36effff640 1 -- 172.21.15.172:0/1556356746 learned_addr learned my addr 172.21.15.172:0/1556356746 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:18.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.039+0000 7f36f4fbf640 1 --2- 172.21.15.172:0/1556356746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36f0102ef0 0x7f36f01a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.039+0000 7f36ef7fe640 1 --2- 172.21.15.172:0/1556356746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36f0102570 0x7f36f0199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.039+0000 7f36effff640 1 -- 172.21.15.172:0/1556356746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36f0102570 msgr2=0x7f36f0199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.039+0000 7f36effff640 1 --2- 172.21.15.172:0/1556356746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36f0102570 0x7f36f0199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.039+0000 7f36effff640 1 -- 172.21.15.172:0/1556356746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36f0102ef0 msgr2=0x7f36f01a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.039+0000 7f36effff640 1 --2- 172.21.15.172:0/1556356746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36f0102ef0 0x7f36f01a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.039+0000 7f36effff640 1 -- 172.21.15.172:0/1556356746 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f01a47c0 con 0x7f36f0100e20 2024-09-10T08:54:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.039+0000 7f36ef7fe640 1 --2- 172.21.15.172:0/1556356746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36f0102570 0x7f36f0199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:54:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.040+0000 7f36effff640 1 --2- 172.21.15.172:0/1556356746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36f0100e20 0x7f36f0199200 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f36e000e9f0 tx=0x7f36e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.040+0000 7f36ed7fa640 1 -- 172.21.15.172:0/1556356746 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e000cd80 con 0x7f36f0100e20 2024-09-10T08:54:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.040+0000 7f36ed7fa640 1 -- 172.21.15.172:0/1556356746 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36e00054d0 con 0x7f36f0100e20 2024-09-10T08:54:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.040+0000 7f36f6a22640 1 -- 172.21.15.172:0/1556356746 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f01a4990 con 0x7f36f0100e20 2024-09-10T08:54:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.040+0000 7f36ed7fa640 1 -- 172.21.15.172:0/1556356746 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0010490 con 0x7f36f0100e20 2024-09-10T08:54:18.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.040+0000 7f36f6a22640 1 -- 172.21.15.172:0/1556356746 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f01a4e50 con 0x7f36f0100e20 2024-09-10T08:54:18.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.043+0000 7f36f8dfb740 1 Processor -- start 2024-09-10T08:54:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.043+0000 7f36f8dfb740 1 -- start start 2024-09-10T08:54:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.043+0000 7f36ed7fa640 1 -- 172.21.15.172:0/1556356746 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f36e0005020 con 0x7f36f0100e20 2024-09-10T08:54:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.043+0000 7f36f8dfb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa05974a0 0x555fa0597880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.043+0000 7f36f8dfb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa0597dc0 0x555fa05a2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.043+0000 7f36f8dfb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05a2680 0x555fa05a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.043+0000 7f36f8dfb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555fa04fd2e0 con 0x555fa0597dc0 2024-09-10T08:54:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.043+0000 7f36f8dfb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555fa04a4ce0 con 0x555fa05a2680 2024-09-10T08:54:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.043+0000 7f36f8dfb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555fa04f92a0 con 0x555fa05974a0 2024-09-10T08:54:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.043+0000 7f36effff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa0597dc0 0x555fa05a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.043+0000 7f36effff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa0597dc0 0x555fa05a2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57796/0 (socket says 172.21.15.172:57796) 2024-09-10T08:54:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.043+0000 7f36effff640 1 -- 172.21.15.172:0/1545464336 learned_addr learned my addr 172.21.15.172:0/1545464336 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:18.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.044+0000 7f36f4fbf640 1 --2- 172.21.15.172:0/1545464336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa05974a0 0x555fa0597880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.044+0000 7f36ef7fe640 1 --2- 172.21.15.172:0/1545464336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05a2680 0x555fa05a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.044+0000 7f36effff640 1 -- 172.21.15.172:0/1545464336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa05974a0 msgr2=0x555fa0597880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.044+0000 7f36effff640 1 --2- 172.21.15.172:0/1545464336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa05974a0 0x555fa0597880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.044+0000 7f36effff640 1 -- 172.21.15.172:0/1545464336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05a2680 msgr2=0x555fa05a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.044+0000 7f36effff640 1 --2- 172.21.15.172:0/1545464336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05a2680 0x555fa05a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.044+0000 7f36effff640 1 -- 172.21.15.172:0/1545464336 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fa0489440 con 0x555fa0597dc0 2024-09-10T08:54:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.044+0000 7f36ef7fe640 1 --2- 172.21.15.172:0/1545464336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05a2680 0x555fa05a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:54:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.044+0000 7f36effff640 1 --2- 172.21.15.172:0/1545464336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa0597dc0 0x555fa05a2140 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f36e00d5a80 tx=0x7f36e00d5b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71654f119ef4dc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.044+0000 7f36ed7fa640 1 --2- 172.21.15.172:0/1556356746 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f36d80a2b80 0x7f36d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.044+0000 7f36f4fbf640 1 --2- 172.21.15.172:0/1556356746 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f36d80a2b80 0x7f36d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.044+0000 7f36ed7fa640 1 -- 172.21.15.172:0/1556356746 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f36e00cf7b0 con 0x7f36f0100e20 2024-09-10T08:54:18.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.044+0000 7f36d67fc640 1 -- 172.21.15.172:0/1545464336 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0097f50 con 0x555fa0597dc0 2024-09-10T08:54:18.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.044+0000 7f36d67fc640 1 -- 172.21.15.172:0/1545464336 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36e00e0070 con 0x555fa0597dc0 2024-09-10T08:54:18.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.045+0000 7f36d67fc640 1 -- 172.21.15.172:0/1545464336 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e009b610 con 0x555fa0597dc0 2024-09-10T08:54:18.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.045+0000 7f36f8dfb740 1 -- 172.21.15.172:0/1545464336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa0597dc0 msgr2=0x555fa05a2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:18.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.045+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/1545464336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa0597dc0 0x555fa05a2140 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f36e00d5a80 tx=0x7f36e00d5b70 comp rx=0 tx=0).stop 2024-09-10T08:54:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.045+0000 7f36f4fbf640 1 --2- 172.21.15.172:0/1556356746 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f36d80a2b80 0x7f36d80a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f36e8007f00 tx=0x7f36e8007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.045+0000 7f36f8dfb740 1 -- 172.21.15.172:0/1545464336 shutdown_connections 2024-09-10T08:54:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.045+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/1545464336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05a2680 0x555fa05a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.045+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/1545464336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa0597dc0 0x555fa05a2140 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.045+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/1545464336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa05974a0 0x555fa0597880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.045+0000 7f36f8dfb740 1 -- 172.21.15.172:0/1545464336 >> 172.21.15.172:0/1545464336 conn(0x555fa0593a30 msgr2=0x555fa0593e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.045+0000 7f36f8dfb740 1 -- 172.21.15.172:0/1545464336 shutdown_connections 2024-09-10T08:54:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.045+0000 7f36f8dfb740 1 -- 172.21.15.172:0/1545464336 wait complete. 2024-09-10T08:54:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.046+0000 7f36f8dfb740 1 -- start start 2024-09-10T08:54:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.046+0000 7f36f8dfb740 1 Processor -- start 2024-09-10T08:54:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.046+0000 7f36f8dfb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05974a0 0x555fa05a2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.046+0000 7f36f8dfb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa0597dc0 0x555fa059f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.046+0000 7f36f8dfb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05a2680 0x555fa059f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.046+0000 7f36f8dfb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555fa05a5250 con 0x555fa05974a0 2024-09-10T08:54:18.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.046+0000 7f36f8dfb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555fa04b8050 con 0x555fa05a2680 2024-09-10T08:54:18.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.046+0000 7f36f8dfb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555fa05a53d0 con 0x555fa0597dc0 2024-09-10T08:54:18.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.046+0000 7f36f8dfb740 10 client.0 _ll_register_callbacks cb 0x555fa04f8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:18.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.047+0000 7f36ef7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa0597dc0 0x555fa059f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.047+0000 7f36effff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05974a0 0x555fa05a2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.047+0000 7f36ef7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa0597dc0 0x555fa059f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:50292/0 (socket says 172.21.15.172:50292) 2024-09-10T08:54:18.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.047+0000 7f36ef7fe640 1 -- 172.21.15.172:0/2170206509 learned_addr learned my addr 172.21.15.172:0/2170206509 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:18.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.047+0000 7f36f4fbf640 1 --2- 172.21.15.172:0/2170206509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05a2680 0x555fa059f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.047+0000 7f36ef7fe640 1 -- 172.21.15.172:0/2170206509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05a2680 msgr2=0x555fa059f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:18.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.047+0000 7f36ef7fe640 1 --2- 172.21.15.172:0/2170206509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05a2680 0x555fa059f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.047+0000 7f36ef7fe640 1 -- 172.21.15.172:0/2170206509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05974a0 msgr2=0x555fa05a2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:18.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.047+0000 7f36ef7fe640 1 --2- 172.21.15.172:0/2170206509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05974a0 0x555fa05a2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.047+0000 7f36ef7fe640 1 -- 172.21.15.172:0/2170206509 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fa026e3c0 con 0x555fa0597dc0 2024-09-10T08:54:18.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.047+0000 7f36effff640 1 --2- 172.21.15.172:0/2170206509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05974a0 0x555fa05a2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:18.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.047+0000 7f36ef7fe640 1 --2- 172.21.15.172:0/2170206509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa0597dc0 0x555fa059f2e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f36e40096c0 tx=0x7f36e4031820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c106507a070507c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:18.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.047+0000 7f36cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:54:18.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.047+0000 7f36cf7fe640 1 -- 172.21.15.172:0/2170206509 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e4035070 con 0x555fa0597dc0 2024-09-10T08:54:18.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.047+0000 7f36cf7fe640 1 -- 172.21.15.172:0/2170206509 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f36e4031ca0 con 0x555fa0597dc0 2024-09-10T08:54:18.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.047+0000 7f36f8dfb740 1 -- 172.21.15.172:0/2170206509 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36e00e0070 con 0x555fa0597dc0 2024-09-10T08:54:18.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.048+0000 7f36cf7fe640 1 -- 172.21.15.172:0/2170206509 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f36e404c050 con 0x555fa0597dc0 2024-09-10T08:54:18.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.048+0000 7f36f8dfb740 10 client.26764 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:18.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.048+0000 7f36f8dfb740 1 -- 172.21.15.172:0/2170206509 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555fa05b0c60 con 0x555fa0597dc0 2024-09-10T08:54:18.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.048+0000 7f36cf7fe640 1 -- 172.21.15.172:0/2170206509 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36e00e0070 con 0x555fa0597dc0 2024-09-10T08:54:18.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.048+0000 7f36cf7fe640 1 -- 172.21.15.172:0/2170206509 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f36e40046c0 con 0x555fa0597dc0 2024-09-10T08:54:18.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.048+0000 7f36f8dfb740 10 client.26764 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:18.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.048+0000 7f36f8dfb740 10 client.26764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:54:18.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.048+0000 7f36f8dfb740 20 client.26764 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:18.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.048+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/2170206509 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555fa05b2120 0x555fa05b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.049+0000 7f36f8dfb740 4 client.26764 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:54:18.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.049+0000 7f36f8dfb740 1 -- 172.21.15.172:0/2170206509 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fa050a850 con 0x555fa05b2120 2024-09-10T08:54:18.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.049+0000 7f36effff640 1 --2- 172.21.15.172:0/2170206509 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555fa05b2120 0x555fa05b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.050+0000 7f36cf7fe640 10 client.26764 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:18.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.050+0000 7f36effff640 1 --2- 172.21.15.172:0/2170206509 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555fa05b2120 0x555fa05b4570 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6736d1e56342cef5 server_cookie=29c28b9b1fad59f0 in_seq=0 out_seq=0 2024-09-10T08:54:18.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.050+0000 7f36cf7fe640 1 -- 172.21.15.172:0/2170206509 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555fa050a850 con 0x555fa05b2120 2024-09-10T08:54:18.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.050+0000 7f36cf7fe640 10 client.26764 handle_command_reply: tid=0 2024-09-10T08:54:18.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.050+0000 7f36f8dfb740 1 client.26764 shutdown 2024-09-10T08:54:18.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.051+0000 7f36f8dfb740 1 -- 172.21.15.172:0/2170206509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa0597dc0 msgr2=0x555fa059f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:18.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.051+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/2170206509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa0597dc0 0x555fa059f2e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f36e40096c0 tx=0x7f36e4031820 comp rx=0 tx=0).stop 2024-09-10T08:54:18.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.051+0000 7f36f8dfb740 1 -- 172.21.15.172:0/2170206509 shutdown_connections 2024-09-10T08:54:18.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.051+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/2170206509 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555fa05b2120 0x555fa05b4570 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.051+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/2170206509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05a2680 0x555fa059f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.051+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/2170206509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa0597dc0 0x555fa059f2e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.051+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/2170206509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05974a0 0x555fa05a2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.051+0000 7f36cf7fe640 0 client.26764 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:18.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.051+0000 7f36f8dfb740 1 -- 172.21.15.172:0/2170206509 >> 172.21.15.172:0/2170206509 conn(0x555fa0593a30 msgr2=0x555fa0599760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:18.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.051+0000 7f36f8dfb740 1 -- 172.21.15.172:0/2170206509 shutdown_connections 2024-09-10T08:54:18.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.051+0000 7f36f8dfb740 1 -- 172.21.15.172:0/2170206509 wait complete. 2024-09-10T08:54:18.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.051+0000 7f36f8dfb740 20 client.26764 trim_cache size 0 max 16384 2024-09-10T08:54:18.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.068+0000 7f36f8dfb740 1 Processor -- start 2024-09-10T08:54:18.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.068+0000 7f36f8dfb740 1 -- start start 2024-09-10T08:54:18.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.068+0000 7f36f8dfb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05974a0 0x555fa059dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.068+0000 7f36f8dfb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa0597dc0 0x555fa05b9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.068+0000 7f36f8dfb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa05a2680 0x555fa05ba380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.068+0000 7f36f8dfb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01ac860 con 0x555fa05974a0 2024-09-10T08:54:18.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.068+0000 7f36f8dfb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01ae3a0 con 0x555fa0597dc0 2024-09-10T08:54:18.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.068+0000 7f36f8dfb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01acf30 con 0x555fa05a2680 2024-09-10T08:54:18.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.068+0000 7f36effff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05974a0 0x555fa059dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.068+0000 7f36effff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05974a0 0x555fa059dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57820/0 (socket says 172.21.15.172:57820) 2024-09-10T08:54:18.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.068+0000 7f36effff640 1 -- 172.21.15.172:0/2568148449 learned_addr learned my addr 172.21.15.172:0/2568148449 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:18.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.068+0000 7f36f4fbf640 1 --2- 172.21.15.172:0/2568148449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa05a2680 0x555fa05ba380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.068+0000 7f36ef7fe640 1 --2- 172.21.15.172:0/2568148449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa0597dc0 0x555fa05b9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.069+0000 7f36effff640 1 -- 172.21.15.172:0/2568148449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa05a2680 msgr2=0x555fa05ba380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:18.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.069+0000 7f36effff640 1 --2- 172.21.15.172:0/2568148449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa05a2680 0x555fa05ba380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.069+0000 7f36effff640 1 -- 172.21.15.172:0/2568148449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa0597dc0 msgr2=0x555fa05b9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:18.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.069+0000 7f36effff640 1 --2- 172.21.15.172:0/2568148449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa0597dc0 0x555fa05b9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.069+0000 7f36effff640 1 -- 172.21.15.172:0/2568148449 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fa059e290 con 0x555fa05974a0 2024-09-10T08:54:18.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.069+0000 7f36f4fbf640 1 --2- 172.21.15.172:0/2568148449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa05a2680 0x555fa05ba380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:18.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.069+0000 7f36effff640 1 --2- 172.21.15.172:0/2568148449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05974a0 0x555fa059dbb0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f36e00ccfa0 tx=0x7f36e0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39d1221cbb16942 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:18.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.069+0000 7f36d5ffb640 1 -- 172.21.15.172:0/2568148449 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e009b7f0 con 0x555fa05974a0 2024-09-10T08:54:18.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.069+0000 7f36d5ffb640 1 -- 172.21.15.172:0/2568148449 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36e00e0040 con 0x555fa05974a0 2024-09-10T08:54:18.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.069+0000 7f36d5ffb640 1 -- 172.21.15.172:0/2568148449 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e009ba80 con 0x555fa05974a0 2024-09-10T08:54:18.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.070+0000 7f36f8dfb740 1 -- 172.21.15.172:0/2568148449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05974a0 msgr2=0x555fa059dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:18.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.070+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/2568148449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05974a0 0x555fa059dbb0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f36e00ccfa0 tx=0x7f36e0009310 comp rx=0 tx=0).stop 2024-09-10T08:54:18.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.070+0000 7f36f8dfb740 1 -- 172.21.15.172:0/2568148449 shutdown_connections 2024-09-10T08:54:18.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.070+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/2568148449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa05a2680 0x555fa05ba380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.070+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/2568148449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa0597dc0 0x555fa05b9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.070+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/2568148449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05974a0 0x555fa059dbb0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.070+0000 7f36f8dfb740 1 -- 172.21.15.172:0/2568148449 >> 172.21.15.172:0/2568148449 conn(0x555fa0593a30 msgr2=0x555fa05a1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:18.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.070+0000 7f36f8dfb740 1 -- 172.21.15.172:0/2568148449 shutdown_connections 2024-09-10T08:54:18.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.070+0000 7f36f8dfb740 1 -- 172.21.15.172:0/2568148449 wait complete. 2024-09-10T08:54:18.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.070+0000 7f36f8dfb740 1 -- start start 2024-09-10T08:54:18.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.071+0000 7f36f8dfb740 1 Processor -- start 2024-09-10T08:54:18.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.071+0000 7f36f8dfb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05974a0 0x555fa05aae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.071+0000 7f36f8dfb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa0597dc0 0x555fa05a81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.071+0000 7f36f8dfb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05a2680 0x555fa05a87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.071+0000 7f36f8dfb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01adcd0 con 0x555fa05a2680 2024-09-10T08:54:18.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.071+0000 7f36f8dfb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01ad600 con 0x555fa05974a0 2024-09-10T08:54:18.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.071+0000 7f36f8dfb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01aa840 con 0x555fa0597dc0 2024-09-10T08:54:18.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.071+0000 7f36f8dfb740 10 client.0 _ll_register_callbacks cb 0x7f36e0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:18.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.071+0000 7f36effff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05974a0 0x555fa05aae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.071+0000 7f36f4fbf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05a2680 0x555fa05a87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.071+0000 7f36effff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05974a0 0x555fa05aae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:39868/0 (socket says 172.21.15.172:39868) 2024-09-10T08:54:18.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.071+0000 7f36effff640 1 -- 172.21.15.172:0/1541223083 learned_addr learned my addr 172.21.15.172:0/1541223083 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:18.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.071+0000 7f36f4fbf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05a2680 0x555fa05a87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57830/0 (socket says 172.21.15.172:57830) 2024-09-10T08:54:18.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.071+0000 7f36ef7fe640 1 --2- 172.21.15.172:0/1541223083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa0597dc0 0x555fa05a81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.072+0000 7f36effff640 1 -- 172.21.15.172:0/1541223083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa0597dc0 msgr2=0x555fa05a81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:18.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.072+0000 7f36effff640 1 --2- 172.21.15.172:0/1541223083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa0597dc0 0x555fa05a81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.072+0000 7f36effff640 1 -- 172.21.15.172:0/1541223083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05a2680 msgr2=0x555fa05a87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:18.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.072+0000 7f36effff640 1 --2- 172.21.15.172:0/1541223083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05a2680 0x555fa05a87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.072+0000 7f36effff640 1 -- 172.21.15.172:0/1541223083 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fa05a9000 con 0x555fa05974a0 2024-09-10T08:54:18.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.072+0000 7f36effff640 1 --2- 172.21.15.172:0/1541223083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05974a0 0x555fa05aae90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f36e00d5230 tx=0x7f36e00d77d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d06283ff26090af3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:18.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.072+0000 7f36cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:54:18.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.072+0000 7f36cf7fe640 1 -- 172.21.15.172:0/1541223083 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e009b7f0 con 0x555fa05974a0 2024-09-10T08:54:18.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.072+0000 7f36cf7fe640 1 -- 172.21.15.172:0/1541223083 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f36e00d5320 con 0x555fa05974a0 2024-09-10T08:54:18.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.072+0000 7f36f8dfb740 1 -- 172.21.15.172:0/1541223083 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36e00e0040 con 0x555fa05974a0 2024-09-10T08:54:18.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.072+0000 7f36cf7fe640 1 -- 172.21.15.172:0/1541223083 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f36e0005310 con 0x555fa05974a0 2024-09-10T08:54:18.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.072+0000 7f36f8dfb740 10 client.26840 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:18.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.072+0000 7f36f8dfb740 1 -- 172.21.15.172:0/1541223083 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555fa061cf40 con 0x555fa05974a0 2024-09-10T08:54:18.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.072+0000 7f36cf7fe640 1 -- 172.21.15.172:0/1541223083 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36e00e0040 con 0x555fa05974a0 2024-09-10T08:54:18.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.073+0000 7f36cf7fe640 1 -- 172.21.15.172:0/1541223083 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f36e009bf10 con 0x555fa05974a0 2024-09-10T08:54:18.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.073+0000 7f36f8dfb740 10 client.26840 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:18.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.073+0000 7f36f8dfb740 10 client.26840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:54:18.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.073+0000 7f36f8dfb740 20 client.26840 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:18.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.073+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/1541223083 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555fa061d170 0x555fa061f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:18.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.073+0000 7f36f8dfb740 4 client.26840 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:54:18.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.073+0000 7f36f8dfb740 1 -- 172.21.15.172:0/1541223083 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555fa0504d60 con 0x555fa061d170 2024-09-10T08:54:18.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.073+0000 7f36ef7fe640 1 --2- 172.21.15.172:0/1541223083 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555fa061d170 0x555fa061f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:18.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.073+0000 7f36cf7fe640 10 client.26840 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:18.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.073+0000 7f36ef7fe640 1 --2- 172.21.15.172:0/1541223083 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555fa061d170 0x555fa061f560 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5a588a536c9fe73 server_cookie=87c6fc7492b6edfe in_seq=0 out_seq=0 2024-09-10T08:54:18.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.073+0000 7f36cf7fe640 1 -- 172.21.15.172:0/1541223083 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555fa0504d60 con 0x555fa061d170 2024-09-10T08:54:18.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.073+0000 7f36cf7fe640 10 client.26840 handle_command_reply: tid=0 2024-09-10T08:54:18.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.073+0000 7f36f8dfb740 1 client.26840 shutdown 2024-09-10T08:54:18.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.074+0000 7f36f8dfb740 1 -- 172.21.15.172:0/1541223083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05974a0 msgr2=0x555fa05aae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:18.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.074+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/1541223083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05974a0 0x555fa05aae90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f36e00d5230 tx=0x7f36e00d77d0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.074+0000 7f36f8dfb740 1 -- 172.21.15.172:0/1541223083 shutdown_connections 2024-09-10T08:54:18.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.074+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/1541223083 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555fa061d170 0x555fa061f560 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.074+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/1541223083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa05a2680 0x555fa05a87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.074+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/1541223083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fa0597dc0 0x555fa05a81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.074+0000 7f36f8dfb740 1 --2- 172.21.15.172:0/1541223083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fa05974a0 0x555fa05aae90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.074+0000 7f36f8dfb740 1 -- 172.21.15.172:0/1541223083 >> 172.21.15.172:0/1541223083 conn(0x555fa0593a30 msgr2=0x555fa05a1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:18.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.074+0000 7f36cf7fe640 0 client.26840 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:18.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.074+0000 7f36f8dfb740 1 -- 172.21.15.172:0/1541223083 shutdown_connections 2024-09-10T08:54:18.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.074+0000 7f36f8dfb740 1 -- 172.21.15.172:0/1541223083 wait complete. 2024-09-10T08:54:18.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.074+0000 7f36f8dfb740 20 client.26840 trim_cache size 0 max 16384 2024-09-10T08:54:18.083 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:54:18.083 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T08:54:18.083 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T08:54:18.083 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:54:18.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.075+0000 7f36f6a22640 1 -- 172.21.15.172:0/1556356746 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f36d80a2b80 msgr2=0x7f36d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:18.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.075+0000 7f36f6a22640 1 --2- 172.21.15.172:0/1556356746 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f36d80a2b80 0x7f36d80a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f36e8007f00 tx=0x7f36e8007520 comp rx=0 tx=0).stop 2024-09-10T08:54:18.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.075+0000 7f36f6a22640 1 -- 172.21.15.172:0/1556356746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36f0100e20 msgr2=0x7f36f0199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:18.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.075+0000 7f36f6a22640 1 --2- 172.21.15.172:0/1556356746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36f0100e20 0x7f36f0199200 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f36e000e9f0 tx=0x7f36e000edf0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.075+0000 7f36f6a22640 1 -- 172.21.15.172:0/1556356746 shutdown_connections 2024-09-10T08:54:18.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.075+0000 7f36f6a22640 1 --2- 172.21.15.172:0/1556356746 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f36d80a2b80 0x7f36d80a5040 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.075+0000 7f36f6a22640 1 --2- 172.21.15.172:0/1556356746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36f0102ef0 0x7f36f01a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.075+0000 7f36f6a22640 1 --2- 172.21.15.172:0/1556356746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36f0102570 0x7f36f0199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.075+0000 7f36f6a22640 1 --2- 172.21.15.172:0/1556356746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36f0100e20 0x7f36f0199200 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:18.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.075+0000 7f36f6a22640 1 -- 172.21.15.172:0/1556356746 >> 172.21.15.172:0/1556356746 conn(0x7f36f00fc7d0 msgr2=0x7f36f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:18.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.075+0000 7f36f6a22640 1 -- 172.21.15.172:0/1556356746 shutdown_connections 2024-09-10T08:54:18.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:18.075+0000 7f36f6a22640 1 -- 172.21.15.172:0/1556356746 wait complete. 2024-09-10T08:54:18.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:18 smithi174 ceph-mon[34068]: pgmap v592: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:54:18.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:18 smithi191 ceph-mon[35314]: pgmap v592: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:54:18.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:18 smithi172 ceph-mon[30854]: pgmap v592: 129 pgs: 129 active+clean; 7.0 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:54:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:19 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:54:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:19 smithi174 ceph-mon[34068]: pgmap v593: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-09-10T08:54:19.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:19 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:54:19.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:19 smithi191 ceph-mon[35314]: pgmap v593: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-09-10T08:54:19.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:19 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:54:19.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:19 smithi172 ceph-mon[30854]: pgmap v593: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-09-10T08:54:22.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:22 smithi174 ceph-mon[34068]: pgmap v594: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-09-10T08:54:22.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:22 smithi191 ceph-mon[35314]: pgmap v594: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-09-10T08:54:22.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:22 smithi172 ceph-mon[30854]: pgmap v594: 129 pgs: 129 active+clean; 7.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-09-10T08:54:24.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:24 smithi174 ceph-mon[34068]: pgmap v595: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-10T08:54:24.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:24 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:54:24.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:24 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:54:24.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:24 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:54:24.764 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:24 smithi172 ceph-mon[30854]: pgmap v595: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-10T08:54:24.764 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:24 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:54:24.764 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:24 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:54:24.764 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:24 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:54:24.796 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:54:24.797 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:54:24.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:24 smithi191 ceph-mon[35314]: pgmap v595: 129 pgs: 129 active+clean; 7.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-10T08:54:24.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:24 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:54:24.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:24 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:54:24.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:24 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:54:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.003+0000 7fa2ac98d640 1 -- 172.21.15.172:0/1426288214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2a80ffe10 msgr2=0x7fa2a8100210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.003+0000 7fa2ac98d640 1 --2- 172.21.15.172:0/1426288214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2a80ffe10 0x7fa2a8100210 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa298009900 tx=0x7fa29802a410 comp rx=0 tx=0).stop 2024-09-10T08:54:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.003+0000 7fa2ac98d640 1 -- 172.21.15.172:0/1426288214 shutdown_connections 2024-09-10T08:54:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.003+0000 7fa2ac98d640 1 --2- 172.21.15.172:0/1426288214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2a81037a0 0x7fa2a810a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.003+0000 7fa2ac98d640 1 --2- 172.21.15.172:0/1426288214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2a8102de0 0x7fa2a8103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.003+0000 7fa2ac98d640 1 --2- 172.21.15.172:0/1426288214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2a80ffe10 0x7fa2a8100210 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.003+0000 7fa2ac98d640 1 -- 172.21.15.172:0/1426288214 >> 172.21.15.172:0/1426288214 conn(0x7fa2a80fb930 msgr2=0x7fa2a80fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.003+0000 7fa2ac98d640 1 -- 172.21.15.172:0/1426288214 shutdown_connections 2024-09-10T08:54:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.003+0000 7fa2ac98d640 1 -- 172.21.15.172:0/1426288214 wait complete. 2024-09-10T08:54:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.004+0000 7fa2ac98d640 1 Processor -- start 2024-09-10T08:54:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.004+0000 7fa2ac98d640 1 -- start start 2024-09-10T08:54:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.004+0000 7fa2ac98d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2a80ffe10 0x7fa2a8197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.004+0000 7fa2ac98d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2a8102de0 0x7fa2a8198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.005+0000 7fa2ac98d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2a81037a0 0x7fa2a819f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.005+0000 7fa2ac98d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a810d740 con 0x7fa2a8102de0 2024-09-10T08:54:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.005+0000 7fa2ac98d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a810d5c0 con 0x7fa2a80ffe10 2024-09-10T08:54:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.005+0000 7fa2ac98d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a810d8c0 con 0x7fa2a81037a0 2024-09-10T08:54:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.005+0000 7fa2a5d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2a8102de0 0x7fa2a8198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.005+0000 7fa2a5d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2a8102de0 0x7fa2a8198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57850/0 (socket says 172.21.15.172:57850) 2024-09-10T08:54:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.005+0000 7fa2a5d9b640 1 -- 172.21.15.172:0/2514252295 learned_addr learned my addr 172.21.15.172:0/2514252295 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.005+0000 7fa2a659c640 1 --2- 172.21.15.172:0/2514252295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2a80ffe10 0x7fa2a8197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.005+0000 7fa2a6d9d640 1 --2- 172.21.15.172:0/2514252295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2a81037a0 0x7fa2a819f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.005+0000 7fa2a659c640 1 -- 172.21.15.172:0/2514252295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2a81037a0 msgr2=0x7fa2a819f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.005+0000 7fa2a659c640 1 --2- 172.21.15.172:0/2514252295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2a81037a0 0x7fa2a819f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.005+0000 7fa2a659c640 1 -- 172.21.15.172:0/2514252295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2a8102de0 msgr2=0x7fa2a8198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.005+0000 7fa2a659c640 1 --2- 172.21.15.172:0/2514252295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2a8102de0 0x7fa2a8198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.005+0000 7fa2a659c640 1 -- 172.21.15.172:0/2514252295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a819f9d0 con 0x7fa2a80ffe10 2024-09-10T08:54:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.005+0000 7fa2a5d9b640 1 --2- 172.21.15.172:0/2514252295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2a8102de0 0x7fa2a8198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.005+0000 7fa2a659c640 1 --2- 172.21.15.172:0/2514252295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2a80ffe10 0x7fa2a8197dc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa298009230 tx=0x7fa29803ac10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.006+0000 7fa2977fe640 1 -- 172.21.15.172:0/2514252295 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298035070 con 0x7fa2a80ffe10 2024-09-10T08:54:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.006+0000 7fa2977fe640 1 -- 172.21.15.172:0/2514252295 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa298003cb0 con 0x7fa2a80ffe10 2024-09-10T08:54:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.006+0000 7fa2ac98d640 1 -- 172.21.15.172:0/2514252295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a8100bd0 con 0x7fa2a80ffe10 2024-09-10T08:54:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.006+0000 7fa2977fe640 1 -- 172.21.15.172:0/2514252295 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298031d10 con 0x7fa2a80ffe10 2024-09-10T08:54:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.006+0000 7fa2ac98d640 1 -- 172.21.15.172:0/2514252295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a8101160 con 0x7fa2a80ffe10 2024-09-10T08:54:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.008+0000 7fa2977fe640 1 -- 172.21.15.172:0/2514252295 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa298031710 con 0x7fa2a80ffe10 2024-09-10T08:54:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.009+0000 7fa2977fe640 1 --2- 172.21.15.172:0/2514252295 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa2900a2cb0 0x7fa2900a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.009+0000 7fa2af807740 1 Processor -- start 2024-09-10T08:54:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.009+0000 7fa2977fe640 1 -- 172.21.15.172:0/2514252295 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa2980f34f0 con 0x7fa2a80ffe10 2024-09-10T08:54:25.011 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.009+0000 7fa2a5d9b640 1 --2- 172.21.15.172:0/2514252295 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa2900a2cb0 0x7fa2900a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2af807740 1 -- start start 2024-09-10T08:54:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2af807740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfd89a0 0x55e3acfd8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2af807740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfd92e0 0x55e3acfddda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2af807740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfde370 0x55e3acfe0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2af807740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e3acf3c2f0 con 0x55e3acfd92e0 2024-09-10T08:54:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2af807740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e3acee4270 con 0x55e3acfd89a0 2024-09-10T08:54:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2af807740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e3acf382b0 con 0x55e3acfde370 2024-09-10T08:54:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2a659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfd89a0 0x55e3acfd8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2a659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfd89a0 0x55e3acfd8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:39896/0 (socket says 172.21.15.172:39896) 2024-09-10T08:54:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2a659c640 1 -- 172.21.15.172:0/3319731981 learned_addr learned my addr 172.21.15.172:0/3319731981 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2a6d9d640 1 --2- 172.21.15.172:0/3319731981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfde370 0x55e3acfe0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2a5d9b640 1 --2- 172.21.15.172:0/2514252295 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa2900a2cb0 0x7fa2900a5170 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa29c011550 tx=0x7fa29c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2a5d9b640 1 --2- 172.21.15.172:0/3319731981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfd92e0 0x55e3acfddda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2a6d9d640 1 -- 172.21.15.172:0/3319731981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfd89a0 msgr2=0x55e3acfd8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2a6d9d640 1 --2- 172.21.15.172:0/3319731981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfd89a0 0x55e3acfd8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2a6d9d640 1 -- 172.21.15.172:0/3319731981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfd92e0 msgr2=0x55e3acfddda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2a6d9d640 1 --2- 172.21.15.172:0/3319731981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfd92e0 0x55e3acfddda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.010+0000 7fa2a6d9d640 1 -- 172.21.15.172:0/3319731981 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3acee3340 con 0x55e3acfde370 2024-09-10T08:54:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.011+0000 7fa2a6d9d640 1 --2- 172.21.15.172:0/3319731981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfde370 0x55e3acfe0830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009f70 tx=0x7fa2a0009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a4e567df0e9401f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.011+0000 7fa294ff9640 1 -- 172.21.15.172:0/3319731981 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0013850 con 0x55e3acfde370 2024-09-10T08:54:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.011+0000 7fa294ff9640 1 -- 172.21.15.172:0/3319731981 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2a0019070 con 0x55e3acfde370 2024-09-10T08:54:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.011+0000 7fa294ff9640 1 -- 172.21.15.172:0/3319731981 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a00122e0 con 0x55e3acfde370 2024-09-10T08:54:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.012+0000 7fa2af807740 1 -- 172.21.15.172:0/3319731981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfde370 msgr2=0x55e3acfe0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.012+0000 7fa2af807740 1 --2- 172.21.15.172:0/3319731981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfde370 0x55e3acfe0830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009f70 tx=0x7fa2a0009650 comp rx=0 tx=0).stop 2024-09-10T08:54:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.012+0000 7fa2af807740 1 -- 172.21.15.172:0/3319731981 shutdown_connections 2024-09-10T08:54:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.012+0000 7fa2af807740 1 --2- 172.21.15.172:0/3319731981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfde370 0x55e3acfe0830 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.012+0000 7fa2af807740 1 --2- 172.21.15.172:0/3319731981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfd92e0 0x55e3acfddda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.012+0000 7fa2af807740 1 --2- 172.21.15.172:0/3319731981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfd89a0 0x55e3acfd8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.012+0000 7fa2af807740 1 -- 172.21.15.172:0/3319731981 >> 172.21.15.172:0/3319731981 conn(0x55e3acfd4010 msgr2=0x55e3acfd4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.012+0000 7fa2af807740 1 -- 172.21.15.172:0/3319731981 shutdown_connections 2024-09-10T08:54:25.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.012+0000 7fa2af807740 1 -- 172.21.15.172:0/3319731981 wait complete. 2024-09-10T08:54:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.013+0000 7fa2af807740 1 -- start start 2024-09-10T08:54:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.013+0000 7fa2af807740 1 Processor -- start 2024-09-10T08:54:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.013+0000 7fa2af807740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd89a0 0x55e3acfe2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.013+0000 7fa2af807740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfd92e0 0x55e3acfe3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.013+0000 7fa2af807740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfde370 0x55e3acfea150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.013+0000 7fa2a659c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd89a0 0x55e3acfe2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.014+0000 7fa2a659c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd89a0 0x55e3acfe2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:50364/0 (socket says 172.21.15.172:50364) 2024-09-10T08:54:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.014+0000 7fa2a659c640 1 -- 172.21.15.172:0/2324091933 learned_addr learned my addr 172.21.15.172:0/2324091933 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.014+0000 7fa2af807740 1 -- 172.21.15.172:0/2324091933 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e3acfe1340 con 0x55e3acfde370 2024-09-10T08:54:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.014+0000 7fa2a6d9d640 1 --2- 172.21.15.172:0/2324091933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfde370 0x55e3acfea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.014+0000 7fa2a5d9b640 1 --2- 172.21.15.172:0/2324091933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfd92e0 0x55e3acfe3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.014+0000 7fa2af807740 1 -- 172.21.15.172:0/2324091933 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e3acec5290 con 0x55e3acfd92e0 2024-09-10T08:54:25.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.014+0000 7fa2af807740 1 -- 172.21.15.172:0/2324091933 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e3acfe14c0 con 0x55e3acfd89a0 2024-09-10T08:54:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.014+0000 7fa2a659c640 1 -- 172.21.15.172:0/2324091933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfd92e0 msgr2=0x55e3acfe3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.014+0000 7fa2a659c640 1 --2- 172.21.15.172:0/2324091933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfd92e0 0x55e3acfe3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.014+0000 7fa2a659c640 1 -- 172.21.15.172:0/2324091933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfde370 msgr2=0x55e3acfea150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.014+0000 7fa2a659c640 1 --2- 172.21.15.172:0/2324091933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfde370 0x55e3acfea150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.014+0000 7fa2a659c640 1 -- 172.21.15.172:0/2324091933 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3acc4f210 con 0x55e3acfd89a0 2024-09-10T08:54:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.014+0000 7fa2af807740 10 client.0 _ll_register_callbacks cb 0x55e3acef6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.014+0000 7fa2af807740 1 -- 172.21.15.172:0/2324091933 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a0019070 con 0x55e3acfd89a0 2024-09-10T08:54:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.014+0000 7fa2a659c640 1 --2- 172.21.15.172:0/2324091933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd89a0 0x55e3acfe2bd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa2980f1000 tx=0x7fa2980bf7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=317d2000d58c795a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.014+0000 7fa28dffb640 10 client.26782 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:54:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.015+0000 7fa28dffb640 1 -- 172.21.15.172:0/2324091933 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2980d7070 con 0x55e3acfd89a0 2024-09-10T08:54:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.015+0000 7fa28dffb640 1 -- 172.21.15.172:0/2324091933 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa2980bfc50 con 0x55e3acfd89a0 2024-09-10T08:54:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.015+0000 7fa28dffb640 1 -- 172.21.15.172:0/2324091933 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa2980bdb30 con 0x55e3acfd89a0 2024-09-10T08:54:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.015+0000 7fa2af807740 10 client.26782 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.015+0000 7fa2af807740 1 -- 172.21.15.172:0/2324091933 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55e3acd1b620 con 0x55e3acfd89a0 2024-09-10T08:54:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.015+0000 7fa28dffb640 1 -- 172.21.15.172:0/2324091933 <== mon.1 v2:172.21.15.191:3300/0 4 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fa2980c81d0 con 0x55e3acfd89a0 2024-09-10T08:54:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.015+0000 7fa2af807740 10 client.26782 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.015+0000 7fa2af807740 10 client.26782 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:54:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.015+0000 7fa2af807740 20 client.26782 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.015+0000 7fa2af807740 1 --2- 172.21.15.172:0/2324091933 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e3acfee650 0x55e3acff0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.015+0000 7fa2af807740 4 client.26782 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:54:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.015+0000 7fa2af807740 1 -- 172.21.15.172:0/2324091933 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3acf49a20 con 0x55e3acfee650 2024-09-10T08:54:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.016+0000 7fa2a5d9b640 1 --2- 172.21.15.172:0/2324091933 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e3acfee650 0x55e3acff0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.016+0000 7fa28dffb640 10 client.26782 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.016+0000 7fa2a5d9b640 1 --2- 172.21.15.172:0/2324091933 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e3acfee650 0x55e3acff0ae0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d812d044161c7b9 server_cookie=29acb4e7ec7fb11c in_seq=0 out_seq=0 2024-09-10T08:54:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.017+0000 7fa28dffb640 1 -- 172.21.15.172:0/2324091933 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3acf49a20 con 0x55e3acfee650 2024-09-10T08:54:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.017+0000 7fa28dffb640 10 client.26782 handle_command_reply: tid=0 2024-09-10T08:54:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.017+0000 7fa2af807740 1 client.26782 shutdown 2024-09-10T08:54:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.017+0000 7fa2af807740 1 -- 172.21.15.172:0/2324091933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd89a0 msgr2=0x55e3acfe2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.017+0000 7fa2af807740 1 --2- 172.21.15.172:0/2324091933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd89a0 0x55e3acfe2bd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa2980f1000 tx=0x7fa2980bf7c0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.017+0000 7fa2af807740 1 -- 172.21.15.172:0/2324091933 shutdown_connections 2024-09-10T08:54:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.017+0000 7fa2af807740 1 --2- 172.21.15.172:0/2324091933 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e3acfee650 0x55e3acff0ae0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.017+0000 7fa28dffb640 0 client.26782 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.017+0000 7fa2af807740 1 --2- 172.21.15.172:0/2324091933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfde370 0x55e3acfea150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.017+0000 7fa2af807740 1 --2- 172.21.15.172:0/2324091933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfd92e0 0x55e3acfe3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.017+0000 7fa2af807740 1 --2- 172.21.15.172:0/2324091933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd89a0 0x55e3acfe2bd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.017+0000 7fa2af807740 1 -- 172.21.15.172:0/2324091933 >> 172.21.15.172:0/2324091933 conn(0x55e3acfd4010 msgr2=0x55e3acfd5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.017+0000 7fa2af807740 1 -- 172.21.15.172:0/2324091933 shutdown_connections 2024-09-10T08:54:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.017+0000 7fa2af807740 1 -- 172.21.15.172:0/2324091933 wait complete. 2024-09-10T08:54:25.024 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.017+0000 7fa2af807740 20 client.26782 trim_cache size 0 max 16384 2024-09-10T08:54:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.034+0000 7fa2af807740 1 Processor -- start 2024-09-10T08:54:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.034+0000 7fa2af807740 1 -- start start 2024-09-10T08:54:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.034+0000 7fa2af807740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfd89a0 0x55e3acfd8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.034+0000 7fa2af807740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd92e0 0x55e3acfea0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.034+0000 7fa2af807740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfde370 0x55e3acfd29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.034+0000 7fa2af807740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81a8300 con 0x55e3acfde370 2024-09-10T08:54:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.034+0000 7fa2af807740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81a9e40 con 0x55e3acfd89a0 2024-09-10T08:54:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.034+0000 7fa2af807740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81a89d0 con 0x55e3acfd92e0 2024-09-10T08:54:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.035+0000 7fa2a6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfde370 0x55e3acfd29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.035+0000 7fa2a6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfde370 0x55e3acfd29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57878/0 (socket says 172.21.15.172:57878) 2024-09-10T08:54:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.035+0000 7fa2a6d9d640 1 -- 172.21.15.172:0/1097928382 learned_addr learned my addr 172.21.15.172:0/1097928382 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.035+0000 7fa2a5d9b640 1 --2- 172.21.15.172:0/1097928382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd92e0 0x55e3acfea0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.035+0000 7fa2a659c640 1 --2- 172.21.15.172:0/1097928382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfd89a0 0x55e3acfd8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.035+0000 7fa2a6d9d640 1 -- 172.21.15.172:0/1097928382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd92e0 msgr2=0x55e3acfea0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.035+0000 7fa2a6d9d640 1 --2- 172.21.15.172:0/1097928382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd92e0 0x55e3acfea0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.035+0000 7fa2a6d9d640 1 -- 172.21.15.172:0/1097928382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfd89a0 msgr2=0x55e3acfd8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.035+0000 7fa2a6d9d640 1 --2- 172.21.15.172:0/1097928382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfd89a0 0x55e3acfd8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.035+0000 7fa2a6d9d640 1 -- 172.21.15.172:0/1097928382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3acfd3e30 con 0x55e3acfde370 2024-09-10T08:54:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.035+0000 7fa2a5d9b640 1 --2- 172.21.15.172:0/1097928382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd92e0 0x55e3acfea0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.035+0000 7fa2a6d9d640 1 --2- 172.21.15.172:0/1097928382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfde370 0x55e3acfd29a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009f70 tx=0x7fa2a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98523caa1cd2eeed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.035+0000 7fa28ffff640 1 -- 172.21.15.172:0/1097928382 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0014620 con 0x55e3acfde370 2024-09-10T08:54:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.035+0000 7fa28ffff640 1 -- 172.21.15.172:0/1097928382 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2a00147c0 con 0x55e3acfde370 2024-09-10T08:54:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.035+0000 7fa28ffff640 1 -- 172.21.15.172:0/1097928382 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a001e780 con 0x55e3acfde370 2024-09-10T08:54:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.036+0000 7fa2af807740 1 -- 172.21.15.172:0/1097928382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfde370 msgr2=0x55e3acfd29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.036+0000 7fa2af807740 1 --2- 172.21.15.172:0/1097928382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfde370 0x55e3acfd29a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009f70 tx=0x7fa2a0004a40 comp rx=0 tx=0).stop 2024-09-10T08:54:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.036+0000 7fa2af807740 1 -- 172.21.15.172:0/1097928382 shutdown_connections 2024-09-10T08:54:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.036+0000 7fa2af807740 1 --2- 172.21.15.172:0/1097928382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfde370 0x55e3acfd29a0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.036+0000 7fa2af807740 1 --2- 172.21.15.172:0/1097928382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd92e0 0x55e3acfea0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.036+0000 7fa2af807740 1 --2- 172.21.15.172:0/1097928382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfd89a0 0x55e3acfd8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.036+0000 7fa2af807740 1 -- 172.21.15.172:0/1097928382 >> 172.21.15.172:0/1097928382 conn(0x55e3acfd4010 msgr2=0x55e3acfe2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.036+0000 7fa2af807740 1 -- 172.21.15.172:0/1097928382 shutdown_connections 2024-09-10T08:54:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.036+0000 7fa2af807740 1 -- 172.21.15.172:0/1097928382 wait complete. 2024-09-10T08:54:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2af807740 1 -- start start 2024-09-10T08:54:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2af807740 1 Processor -- start 2024-09-10T08:54:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2af807740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfd89a0 0x55e3acfe7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2af807740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd92e0 0x55e3acfe7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2af807740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfde370 0x55e3acfe8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2af807740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81a9770 con 0x55e3acfd89a0 2024-09-10T08:54:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2af807740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81a90a0 con 0x55e3acfde370 2024-09-10T08:54:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2af807740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81a62e0 con 0x55e3acfd92e0 2024-09-10T08:54:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2af807740 10 client.0 _ll_register_callbacks cb 0x7fa2980f0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2a659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfd89a0 0x55e3acfe7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2a659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfd89a0 0x55e3acfe7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57886/0 (socket says 172.21.15.172:57886) 2024-09-10T08:54:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2a659c640 1 -- 172.21.15.172:0/680689786 learned_addr learned my addr 172.21.15.172:0/680689786 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2a6d9d640 1 --2- 172.21.15.172:0/680689786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfde370 0x55e3acfe8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2a5d9b640 1 --2- 172.21.15.172:0/680689786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd92e0 0x55e3acfe7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2a659c640 1 -- 172.21.15.172:0/680689786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd92e0 msgr2=0x55e3acfe7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2a659c640 1 --2- 172.21.15.172:0/680689786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd92e0 0x55e3acfe7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2a659c640 1 -- 172.21.15.172:0/680689786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfde370 msgr2=0x55e3acfe8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2a659c640 1 --2- 172.21.15.172:0/680689786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfde370 0x55e3acfe8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2a659c640 1 -- 172.21.15.172:0/680689786 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3acfe8e90 con 0x55e3acfd89a0 2024-09-10T08:54:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.037+0000 7fa2a6d9d640 1 --2- 172.21.15.172:0/680689786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfde370 0x55e3acfe8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.038+0000 7fa2a659c640 1 --2- 172.21.15.172:0/680689786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfd89a0 0x55e3acfe7330 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa298006010 tx=0x7fa2980f0b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc23d5d178c9755b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.038+0000 7fa28dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.038+0000 7fa28dffb640 1 -- 172.21.15.172:0/680689786 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2980d7040 con 0x55e3acfd89a0 2024-09-10T08:54:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.038+0000 7fa28dffb640 1 -- 172.21.15.172:0/680689786 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa2980bfe60 con 0x55e3acfd89a0 2024-09-10T08:54:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.038+0000 7fa2af807740 1 -- 172.21.15.172:0/680689786 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a00147c0 con 0x55e3acfd89a0 2024-09-10T08:54:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.038+0000 7fa28dffb640 1 -- 172.21.15.172:0/680689786 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fa2980c00f0 con 0x55e3acfd89a0 2024-09-10T08:54:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.038+0000 7fa2af807740 10 client.17811 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.038+0000 7fa2af807740 1 -- 172.21.15.172:0/680689786 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e3acfea210 con 0x55e3acfd89a0 2024-09-10T08:54:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.038+0000 7fa28dffb640 1 -- 172.21.15.172:0/680689786 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2a00147c0 con 0x55e3acfd89a0 2024-09-10T08:54:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.038+0000 7fa28dffb640 1 -- 172.21.15.172:0/680689786 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fa2980aa0b0 con 0x55e3acfd89a0 2024-09-10T08:54:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.039+0000 7fa2af807740 10 client.17811 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.039+0000 7fa2af807740 10 client.17811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:54:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.039+0000 7fa2af807740 20 client.17811 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.039+0000 7fa2af807740 1 --2- 172.21.15.172:0/680689786 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e3acfee650 0x55e3ad05b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.039+0000 7fa2af807740 4 client.17811 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:54:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.039+0000 7fa2af807740 1 -- 172.21.15.172:0/680689786 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3acf43d70 con 0x55e3acfee650 2024-09-10T08:54:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.039+0000 7fa2a5d9b640 1 --2- 172.21.15.172:0/680689786 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e3acfee650 0x55e3ad05b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.039+0000 7fa2a5d9b640 1 --2- 172.21.15.172:0/680689786 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e3acfee650 0x55e3ad05b710 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f2a2d8d97823d43 server_cookie=eefe3a5a2d5cb34c in_seq=0 out_seq=0 2024-09-10T08:54:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.039+0000 7fa28dffb640 10 client.17811 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.045+0000 7fa28dffb640 1 -- 172.21.15.172:0/680689786 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e3acf43d70 con 0x55e3acfee650 2024-09-10T08:54:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.045+0000 7fa28dffb640 10 client.17811 handle_command_reply: tid=0 2024-09-10T08:54:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.045+0000 7fa2af807740 1 client.17811 shutdown 2024-09-10T08:54:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.045+0000 7fa2af807740 1 -- 172.21.15.172:0/680689786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfd89a0 msgr2=0x55e3acfe7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.045+0000 7fa2af807740 1 --2- 172.21.15.172:0/680689786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfd89a0 0x55e3acfe7330 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa298006010 tx=0x7fa2980f0b60 comp rx=0 tx=0).stop 2024-09-10T08:54:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.045+0000 7fa2af807740 1 -- 172.21.15.172:0/680689786 shutdown_connections 2024-09-10T08:54:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.046+0000 7fa2af807740 1 --2- 172.21.15.172:0/680689786 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e3acfee650 0x55e3ad05b710 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.046+0000 7fa2af807740 1 --2- 172.21.15.172:0/680689786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3acfde370 0x55e3acfe8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.046+0000 7fa28dffb640 0 client.17811 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.046+0000 7fa2af807740 1 --2- 172.21.15.172:0/680689786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e3acfd92e0 0x55e3acfe7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.046+0000 7fa2af807740 1 --2- 172.21.15.172:0/680689786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3acfd89a0 0x55e3acfe7330 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.046+0000 7fa2af807740 1 -- 172.21.15.172:0/680689786 >> 172.21.15.172:0/680689786 conn(0x55e3acfd4010 msgr2=0x55e3acfd4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.046+0000 7fa2af807740 1 -- 172.21.15.172:0/680689786 shutdown_connections 2024-09-10T08:54:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.046+0000 7fa2af807740 1 -- 172.21.15.172:0/680689786 wait complete. 2024-09-10T08:54:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.046+0000 7fa2af807740 20 client.17811 trim_cache size 0 max 16384 2024-09-10T08:54:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:54:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:54:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:54:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:54:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.047+0000 7fa2ac98d640 1 -- 172.21.15.172:0/2514252295 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa2900a2cb0 msgr2=0x7fa2900a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.047+0000 7fa2ac98d640 1 --2- 172.21.15.172:0/2514252295 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa2900a2cb0 0x7fa2900a5170 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa29c011550 tx=0x7fa29c005f90 comp rx=0 tx=0).stop 2024-09-10T08:54:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.047+0000 7fa2ac98d640 1 -- 172.21.15.172:0/2514252295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2a80ffe10 msgr2=0x7fa2a8197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.047+0000 7fa2ac98d640 1 --2- 172.21.15.172:0/2514252295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2a80ffe10 0x7fa2a8197dc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa298009230 tx=0x7fa29803ac10 comp rx=0 tx=0).stop 2024-09-10T08:54:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.047+0000 7fa2ac98d640 1 -- 172.21.15.172:0/2514252295 shutdown_connections 2024-09-10T08:54:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.047+0000 7fa2ac98d640 1 --2- 172.21.15.172:0/2514252295 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa2900a2cb0 0x7fa2900a5170 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.047+0000 7fa2ac98d640 1 --2- 172.21.15.172:0/2514252295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa2a81037a0 0x7fa2a819f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.047+0000 7fa2ac98d640 1 --2- 172.21.15.172:0/2514252295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2a8102de0 0x7fa2a8198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.047+0000 7fa2ac98d640 1 --2- 172.21.15.172:0/2514252295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2a80ffe10 0x7fa2a8197dc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.047+0000 7fa2ac98d640 1 -- 172.21.15.172:0/2514252295 >> 172.21.15.172:0/2514252295 conn(0x7fa2a80fb930 msgr2=0x7fa2a81061b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.047+0000 7fa2ac98d640 1 -- 172.21.15.172:0/2514252295 shutdown_connections 2024-09-10T08:54:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:25.047+0000 7fa2ac98d640 1 -- 172.21.15.172:0/2514252295 wait complete. 2024-09-10T08:54:25.099 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:54:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:26 smithi172 ceph-mon[30854]: pgmap v596: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-09-10T08:54:26.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:26 smithi174 ceph-mon[34068]: pgmap v596: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-09-10T08:54:26.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:26 smithi191 ceph-mon[35314]: pgmap v596: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-09-10T08:54:28.478 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:28 smithi172 ceph-mon[30854]: pgmap v597: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:54:28.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.476+0000 7fa99bc23640 1 -- 172.21.15.172:0/3896776331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa994108290 msgr2=0x7fa99410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:28.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.476+0000 7fa99bc23640 1 --2- 172.21.15.172:0/3896776331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa994108290 0x7fa99410ec30 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009cd0 tx=0x7fa98c02a8e0 comp rx=0 tx=0).stop 2024-09-10T08:54:28.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.476+0000 7fa99bc23640 1 -- 172.21.15.172:0/3896776331 shutdown_connections 2024-09-10T08:54:28.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.476+0000 7fa99bc23640 1 --2- 172.21.15.172:0/3896776331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa994108290 0x7fa99410ec30 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:28.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.476+0000 7fa99bc23640 1 --2- 172.21.15.172:0/3896776331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa994103740 0x7fa994107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:28.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.476+0000 7fa99bc23640 1 --2- 172.21.15.172:0/3896776331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa994102d90 0x7fa994103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:28.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.476+0000 7fa99bc23640 1 -- 172.21.15.172:0/3896776331 >> 172.21.15.172:0/3896776331 conn(0x7fa9940fc7d0 msgr2=0x7fa9940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:28.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.476+0000 7fa99bc23640 1 -- 172.21.15.172:0/3896776331 shutdown_connections 2024-09-10T08:54:28.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.476+0000 7fa99bc23640 1 -- 172.21.15.172:0/3896776331 wait complete. 2024-09-10T08:54:28.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.477+0000 7fa99bc23640 1 Processor -- start 2024-09-10T08:54:28.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.477+0000 7fa99bc23640 1 -- start start 2024-09-10T08:54:28.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.477+0000 7fa99bc23640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa994102d90 0x7fa994198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:28.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.477+0000 7fa99bc23640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa994103740 0x7fa9941989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:28.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.478+0000 7fa99bc23640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa994108290 0x7fa99419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:28.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.478+0000 7fa99bc23640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa994111f40 con 0x7fa994102d90 2024-09-10T08:54:28.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.478+0000 7fa99bc23640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa994111dc0 con 0x7fa994108290 2024-09-10T08:54:28.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.478+0000 7fa99bc23640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa9941120c0 con 0x7fa994103740 2024-09-10T08:54:28.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.478+0000 7fa9999bf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa994102d90 0x7fa994198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:28.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.478+0000 7fa9999bf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa994102d90 0x7fa994198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57384/0 (socket says 172.21.15.172:57384) 2024-09-10T08:54:28.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.478+0000 7fa9999bf640 1 -- 172.21.15.172:0/2322926628 learned_addr learned my addr 172.21.15.172:0/2322926628 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:28.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.478+0000 7fa99a1c0640 1 --2- 172.21.15.172:0/2322926628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa994108290 0x7fa99419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:28.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.478+0000 7fa9991be640 1 --2- 172.21.15.172:0/2322926628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa994103740 0x7fa9941989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:28.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.478+0000 7fa99a1c0640 1 -- 172.21.15.172:0/2322926628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa994103740 msgr2=0x7fa9941989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:28.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.478+0000 7fa99a1c0640 1 --2- 172.21.15.172:0/2322926628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa994103740 0x7fa9941989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:28.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.478+0000 7fa99a1c0640 1 -- 172.21.15.172:0/2322926628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa994102d90 msgr2=0x7fa994198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:28.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.478+0000 7fa99a1c0640 1 --2- 172.21.15.172:0/2322926628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa994102d90 0x7fa994198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:28.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.478+0000 7fa99a1c0640 1 -- 172.21.15.172:0/2322926628 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99419d410 con 0x7fa994108290 2024-09-10T08:54:28.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.478+0000 7fa9999bf640 1 --2- 172.21.15.172:0/2322926628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa994102d90 0x7fa994198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:28.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.478+0000 7fa99a1c0640 1 --2- 172.21.15.172:0/2322926628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa994108290 0x7fa99419cd30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa98c02ae00 tx=0x7fa98c003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:28.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.479+0000 7fa98affd640 1 -- 172.21.15.172:0/2322926628 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c004240 con 0x7fa994108290 2024-09-10T08:54:28.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.479+0000 7fa98affd640 1 -- 172.21.15.172:0/2322926628 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa98c0305a0 con 0x7fa994108290 2024-09-10T08:54:28.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.479+0000 7fa98affd640 1 -- 172.21.15.172:0/2322926628 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0466e0 con 0x7fa994108290 2024-09-10T08:54:28.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.479+0000 7fa99bc23640 1 -- 172.21.15.172:0/2322926628 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99419d6a0 con 0x7fa994108290 2024-09-10T08:54:28.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.479+0000 7fa99bc23640 1 -- 172.21.15.172:0/2322926628 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9941a4fe0 con 0x7fa994108290 2024-09-10T08:54:28.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.480+0000 7fa99bc23640 1 -- 172.21.15.172:0/2322926628 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa99406a7f0 con 0x7fa994108290 2024-09-10T08:54:28.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.482+0000 7fa98affd640 1 -- 172.21.15.172:0/2322926628 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa98c030aa0 con 0x7fa994108290 2024-09-10T08:54:28.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.482+0000 7fa98affd640 1 --2- 172.21.15.172:0/2322926628 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa9800a2b80 0x7fa9800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:28.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.482+0000 7fa9999bf640 1 --2- 172.21.15.172:0/2322926628 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa9800a2b80 0x7fa9800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:28.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.482+0000 7fa98affd640 1 -- 172.21.15.172:0/2322926628 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa98c02e030 con 0x7fa994108290 2024-09-10T08:54:28.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.483+0000 7fa9999bf640 1 --2- 172.21.15.172:0/2322926628 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa9800a2b80 0x7fa9800a5040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa990005fd0 tx=0x7fa99000b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:28.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.485+0000 7fa98affd640 1 -- 172.21.15.172:0/2322926628 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa98c04b060 con 0x7fa994108290 2024-09-10T08:54:28.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.739+0000 7fa99bc23640 1 -- 172.21.15.172:0/2322926628 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa994103170 con 0x7fa994108290 2024-09-10T08:54:28.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.740+0000 7fa98affd640 1 -- 172.21.15.172:0/2322926628 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+12001 (secure 0 0 0) 0x7fa994103170 con 0x7fa994108290 2024-09-10T08:54:28.742 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:54:28.742 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":37,"btime":"2024-09-10T08:52:08:313493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:52:07.658341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:54:28.742 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 37 2024-09-10T08:54:28.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:28 smithi174 ceph-mon[34068]: pgmap v597: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:54:28.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.742+0000 7fa99bc23640 1 -- 172.21.15.172:0/2322926628 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa9800a2b80 msgr2=0x7fa9800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:28.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.742+0000 7fa99bc23640 1 --2- 172.21.15.172:0/2322926628 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa9800a2b80 0x7fa9800a5040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa990005fd0 tx=0x7fa99000b480 comp rx=0 tx=0).stop 2024-09-10T08:54:28.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.742+0000 7fa99bc23640 1 -- 172.21.15.172:0/2322926628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa994108290 msgr2=0x7fa99419cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:28.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.742+0000 7fa99bc23640 1 --2- 172.21.15.172:0/2322926628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa994108290 0x7fa99419cd30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa98c02ae00 tx=0x7fa98c003ba0 comp rx=0 tx=0).stop 2024-09-10T08:54:28.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.742+0000 7fa99bc23640 1 -- 172.21.15.172:0/2322926628 shutdown_connections 2024-09-10T08:54:28.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.742+0000 7fa99bc23640 1 --2- 172.21.15.172:0/2322926628 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa9800a2b80 0x7fa9800a5040 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:28.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.742+0000 7fa99bc23640 1 --2- 172.21.15.172:0/2322926628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa994108290 0x7fa99419cd30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:28.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.742+0000 7fa99bc23640 1 --2- 172.21.15.172:0/2322926628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa994103740 0x7fa9941989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:28.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.742+0000 7fa99bc23640 1 --2- 172.21.15.172:0/2322926628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa994102d90 0x7fa994198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:28.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.742+0000 7fa99bc23640 1 -- 172.21.15.172:0/2322926628 >> 172.21.15.172:0/2322926628 conn(0x7fa9940fc7d0 msgr2=0x7fa994105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:28.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.742+0000 7fa99bc23640 1 -- 172.21.15.172:0/2322926628 shutdown_connections 2024-09-10T08:54:28.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:28.742+0000 7fa99bc23640 1 -- 172.21.15.172:0/2322926628 wait complete. 2024-09-10T08:54:28.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:28 smithi191 ceph-mon[35314]: pgmap v597: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:54:29.509 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T08:54:29.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:29 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2322926628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:54:29.792 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:54:29.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:29 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2322926628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:54:29.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:29 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2322926628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:54:30.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.320+0000 7f7efd83d640 1 -- 172.21.15.172:0/4169486186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef8100e20 msgr2=0x7f7ef8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:30.333 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.320+0000 7f7efd83d640 1 --2- 172.21.15.172:0/4169486186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef8100e20 0x7f7ef8101200 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009900 tx=0x7f7ee802a480 comp rx=0 tx=0).stop 2024-09-10T08:54:30.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.320+0000 7f7efd83d640 1 -- 172.21.15.172:0/4169486186 shutdown_connections 2024-09-10T08:54:30.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.320+0000 7f7efd83d640 1 --2- 172.21.15.172:0/4169486186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ef8102ef0 0x7f7ef810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.320+0000 7f7efd83d640 1 --2- 172.21.15.172:0/4169486186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ef8102570 0x7f7ef81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.320+0000 7f7efd83d640 1 --2- 172.21.15.172:0/4169486186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef8100e20 0x7f7ef8101200 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.320+0000 7f7efd83d640 1 -- 172.21.15.172:0/4169486186 >> 172.21.15.172:0/4169486186 conn(0x7f7ef80fc7d0 msgr2=0x7f7ef80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:30.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.320+0000 7f7efd83d640 1 -- 172.21.15.172:0/4169486186 shutdown_connections 2024-09-10T08:54:30.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.320+0000 7f7efd83d640 1 -- 172.21.15.172:0/4169486186 wait complete. 2024-09-10T08:54:30.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7efd83d640 1 Processor -- start 2024-09-10T08:54:30.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7efd83d640 1 -- start start 2024-09-10T08:54:30.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7efd83d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef8100e20 0x7f7ef80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7efd83d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ef8102570 0x7f7ef80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.334 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7efd83d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ef8102ef0 0x7f7ef80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7efd83d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef8110e30 con 0x7f7ef8100e20 2024-09-10T08:54:30.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7efd83d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef8110cb0 con 0x7f7ef8102ef0 2024-09-10T08:54:30.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7efd83d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef8110fb0 con 0x7f7ef8102570 2024-09-10T08:54:30.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7ef6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef8100e20 0x7f7ef80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7ef6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef8100e20 0x7f7ef80fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57400/0 (socket says 172.21.15.172:57400) 2024-09-10T08:54:30.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7ef6ffd640 1 -- 172.21.15.172:0/4017050888 learned_addr learned my addr 172.21.15.172:0/4017050888 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:30.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7eeffff640 1 --2- 172.21.15.172:0/4017050888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ef8102570 0x7f7ef80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7ef77fe640 1 --2- 172.21.15.172:0/4017050888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ef8102ef0 0x7f7ef80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7ef6ffd640 1 -- 172.21.15.172:0/4017050888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ef8102570 msgr2=0x7f7ef80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:30.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7ef6ffd640 1 --2- 172.21.15.172:0/4017050888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ef8102570 0x7f7ef80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7ef6ffd640 1 -- 172.21.15.172:0/4017050888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ef8102ef0 msgr2=0x7f7ef80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:30.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7ef6ffd640 1 --2- 172.21.15.172:0/4017050888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ef8102ef0 0x7f7ef80fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.321+0000 7f7ef6ffd640 1 -- 172.21.15.172:0/4017050888 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef81a68e0 con 0x7f7ef8100e20 2024-09-10T08:54:30.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.322+0000 7f7ef6ffd640 1 --2- 172.21.15.172:0/4017050888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef8100e20 0x7f7ef80fcc80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8006010 tx=0x7f7ee80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:30.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.322+0000 7f7efc83b640 1 -- 172.21.15.172:0/4017050888 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee803f030 con 0x7f7ef8100e20 2024-09-10T08:54:30.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.322+0000 7f7efd83d640 1 -- 172.21.15.172:0/4017050888 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef81a6b70 con 0x7f7ef8100e20 2024-09-10T08:54:30.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.322+0000 7f7efc83b640 1 -- 172.21.15.172:0/4017050888 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ee80044c0 con 0x7f7ef8100e20 2024-09-10T08:54:30.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.322+0000 7f7efc83b640 1 -- 172.21.15.172:0/4017050888 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8035070 con 0x7f7ef8100e20 2024-09-10T08:54:30.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.322+0000 7f7efd83d640 1 -- 172.21.15.172:0/4017050888 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef81a7050 con 0x7f7ef8100e20 2024-09-10T08:54:30.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.324+0000 7f7efc83b640 1 -- 172.21.15.172:0/4017050888 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7ee8042030 con 0x7f7ef8100e20 2024-09-10T08:54:30.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.325+0000 7f7effc16740 1 Processor -- start 2024-09-10T08:54:30.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.325+0000 7f7efc83b640 1 --2- 172.21.15.172:0/4017050888 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7ee40a2b80 0x7f7ee40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.325+0000 7f7effc16740 1 -- start start 2024-09-10T08:54:30.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.325+0000 7f7ef77fe640 1 --2- 172.21.15.172:0/4017050888 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7ee40a2b80 0x7f7ee40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.325+0000 7f7efc83b640 1 -- 172.21.15.172:0/4017050888 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f7ee80ef6c0 con 0x7f7ef8100e20 2024-09-10T08:54:30.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.325+0000 7f7effc16740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 0x5592aebb9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.325+0000 7f7effc16740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebb95d0 0x5592aebb9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.325+0000 7f7effc16740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebc4370 0x5592aebc6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.325+0000 7f7effc16740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5592aeb1f2f0 con 0x5592aebbab40 2024-09-10T08:54:30.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.325+0000 7f7effc16740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592aeac6cf0 con 0x5592aebb95d0 2024-09-10T08:54:30.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.325+0000 7f7effc16740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5592aeb1b2b0 con 0x5592aebc4370 2024-09-10T08:54:30.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.326+0000 7f7ef6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 0x5592aebb9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.326+0000 7f7ef6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 0x5592aebb9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57402/0 (socket says 172.21.15.172:57402) 2024-09-10T08:54:30.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.326+0000 7f7ef6ffd640 1 -- 172.21.15.172:0/3876962540 learned_addr learned my addr 172.21.15.172:0/3876962540 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:30.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.326+0000 7f7eeffff640 1 --2- 172.21.15.172:0/3876962540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebb95d0 0x5592aebb9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.326+0000 7f7ef77fe640 1 --2- 172.21.15.172:0/3876962540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebc4370 0x5592aebc6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.326+0000 7f7ef6ffd640 1 -- 172.21.15.172:0/3876962540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebc4370 msgr2=0x5592aebc6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:30.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.326+0000 7f7ef6ffd640 1 --2- 172.21.15.172:0/3876962540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebc4370 0x5592aebc6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.326+0000 7f7ef6ffd640 1 -- 172.21.15.172:0/3876962540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebb95d0 msgr2=0x5592aebb9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:30.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.326+0000 7f7ef6ffd640 1 --2- 172.21.15.172:0/3876962540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebb95d0 0x5592aebb9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.326+0000 7f7ef6ffd640 1 -- 172.21.15.172:0/3876962540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592aeaab3a0 con 0x5592aebbab40 2024-09-10T08:54:30.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.326+0000 7f7ef77fe640 1 --2- 172.21.15.172:0/4017050888 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7ee40a2b80 0x7f7ee40a5040 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0005fd0 tx=0x7f7ef00107a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:30.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.326+0000 7f7ef6ffd640 1 --2- 172.21.15.172:0/3876962540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 0x5592aebb9090 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80ecdd0 tx=0x7f7ee80bb5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d815a94b0be7bed1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:30.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.326+0000 7f7eedffb640 1 -- 172.21.15.172:0/3876962540 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8049040 con 0x5592aebbab40 2024-09-10T08:54:30.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.326+0000 7f7eedffb640 1 -- 172.21.15.172:0/3876962540 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ee80d2070 con 0x5592aebbab40 2024-09-10T08:54:30.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.326+0000 7f7eedffb640 1 -- 172.21.15.172:0/3876962540 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8003e00 con 0x5592aebbab40 2024-09-10T08:54:30.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.327+0000 7f7effc16740 1 -- 172.21.15.172:0/3876962540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 msgr2=0x5592aebb9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:30.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.327+0000 7f7effc16740 1 --2- 172.21.15.172:0/3876962540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 0x5592aebb9090 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80ecdd0 tx=0x7f7ee80bb5e0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.327+0000 7f7effc16740 1 -- 172.21.15.172:0/3876962540 shutdown_connections 2024-09-10T08:54:30.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.327+0000 7f7effc16740 1 --2- 172.21.15.172:0/3876962540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebc4370 0x5592aebc6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.327+0000 7f7effc16740 1 --2- 172.21.15.172:0/3876962540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebb95d0 0x5592aebb9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.327+0000 7f7effc16740 1 --2- 172.21.15.172:0/3876962540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 0x5592aebb9090 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.327+0000 7f7effc16740 1 -- 172.21.15.172:0/3876962540 >> 172.21.15.172:0/3876962540 conn(0x5592aeb82310 msgr2=0x5592aebb5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:30.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.327+0000 7f7effc16740 1 -- 172.21.15.172:0/3876962540 shutdown_connections 2024-09-10T08:54:30.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.327+0000 7f7effc16740 1 -- 172.21.15.172:0/3876962540 wait complete. 2024-09-10T08:54:30.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.328+0000 7f7effc16740 1 -- start start 2024-09-10T08:54:30.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.328+0000 7f7effc16740 1 Processor -- start 2024-09-10T08:54:30.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.328+0000 7f7effc16740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebb95d0 0x5592aebc10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.328+0000 7f7effc16740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 0x5592aebc1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.328+0000 7f7effc16740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebc4370 0x5592aebc1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.328+0000 7f7effc16740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5592aeb1f2f0 con 0x5592aebbab40 2024-09-10T08:54:30.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.328+0000 7f7effc16740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592aeada060 con 0x5592aebb95d0 2024-09-10T08:54:30.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.328+0000 7f7effc16740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5592aebc6e30 con 0x5592aebc4370 2024-09-10T08:54:30.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.328+0000 7f7effc16740 10 client.0 _ll_register_callbacks cb 0x5592ae82d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:30.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.328+0000 7f7eeffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 0x5592aebc1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.328+0000 7f7eeffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 0x5592aebc1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57414/0 (socket says 172.21.15.172:57414) 2024-09-10T08:54:30.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.328+0000 7f7eeffff640 1 -- 172.21.15.172:0/3139601359 learned_addr learned my addr 172.21.15.172:0/3139601359 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:30.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.328+0000 7f7ef6ffd640 1 --2- 172.21.15.172:0/3139601359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebb95d0 0x5592aebc10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.329+0000 7f7ef77fe640 1 --2- 172.21.15.172:0/3139601359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebc4370 0x5592aebc1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.329+0000 7f7eeffff640 1 -- 172.21.15.172:0/3139601359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebc4370 msgr2=0x5592aebc1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:30.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.329+0000 7f7eeffff640 1 --2- 172.21.15.172:0/3139601359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebc4370 0x5592aebc1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.329+0000 7f7eeffff640 1 -- 172.21.15.172:0/3139601359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebb95d0 msgr2=0x5592aebc10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:30.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.329+0000 7f7eeffff640 1 --2- 172.21.15.172:0/3139601359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebb95d0 0x5592aebc10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.329+0000 7f7eeffff640 1 -- 172.21.15.172:0/3139601359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592ae8903c0 con 0x5592aebbab40 2024-09-10T08:54:30.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.329+0000 7f7eeffff640 1 --2- 172.21.15.172:0/3139601359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 0x5592aebc1630 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0012700 tx=0x7f7ee0012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19f774db2288c69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:30.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.329+0000 7f7ed6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:30.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.329+0000 7f7ed6ffd640 1 -- 172.21.15.172:0/3139601359 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0004050 con 0x5592aebbab40 2024-09-10T08:54:30.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.329+0000 7f7ed6ffd640 1 -- 172.21.15.172:0/3139601359 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f7ee00041f0 con 0x5592aebbab40 2024-09-10T08:54:30.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.329+0000 7f7effc16740 1 -- 172.21.15.172:0/3139601359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ee80d2070 con 0x5592aebbab40 2024-09-10T08:54:30.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.329+0000 7f7ed6ffd640 1 -- 172.21.15.172:0/3139601359 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7ee0025050 con 0x5592aebbab40 2024-09-10T08:54:30.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.329+0000 7f7effc16740 10 client.17829 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:30.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.329+0000 7f7effc16740 1 -- 172.21.15.172:0/3139601359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5592aebd2820 con 0x5592aebbab40 2024-09-10T08:54:30.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.330+0000 7f7ed6ffd640 1 -- 172.21.15.172:0/3139601359 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ee80d2070 con 0x5592aebbab40 2024-09-10T08:54:30.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.330+0000 7f7ed6ffd640 1 -- 172.21.15.172:0/3139601359 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f7ee0003ab0 con 0x5592aebbab40 2024-09-10T08:54:30.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.330+0000 7f7effc16740 10 client.17829 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:30.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.330+0000 7f7effc16740 10 client.17829 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:54:30.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.330+0000 7f7effc16740 20 client.17829 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:30.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.330+0000 7f7effc16740 1 --2- 172.21.15.172:0/3139601359 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5592aebd3ce0 0x5592aebd6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.330+0000 7f7effc16740 4 client.17829 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:54:30.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.330+0000 7f7effc16740 1 -- 172.21.15.172:0/3139601359 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592aeb2c860 con 0x5592aebd3ce0 2024-09-10T08:54:30.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.330+0000 7f7ef6ffd640 1 --2- 172.21.15.172:0/3139601359 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5592aebd3ce0 0x5592aebd6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.331+0000 7f7ed6ffd640 10 client.17829 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:54:30.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.331+0000 7f7ef6ffd640 1 --2- 172.21.15.172:0/3139601359 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5592aebd3ce0 0x5592aebd6110 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4fb302bfa4282234 server_cookie=ca7624c0faf8c2f4 in_seq=0 out_seq=0 2024-09-10T08:54:30.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.331+0000 7f7ed6ffd640 1 -- 172.21.15.172:0/3139601359 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592aeb2c860 con 0x5592aebd3ce0 2024-09-10T08:54:30.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.331+0000 7f7ed6ffd640 10 client.17829 handle_command_reply: tid=0 2024-09-10T08:54:30.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.331+0000 7f7effc16740 1 client.17829 shutdown 2024-09-10T08:54:30.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.331+0000 7f7effc16740 1 -- 172.21.15.172:0/3139601359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 msgr2=0x5592aebc1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:30.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.331+0000 7f7effc16740 1 --2- 172.21.15.172:0/3139601359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 0x5592aebc1630 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0012700 tx=0x7f7ee0012bc0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.332+0000 7f7effc16740 1 -- 172.21.15.172:0/3139601359 shutdown_connections 2024-09-10T08:54:30.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.332+0000 7f7effc16740 1 --2- 172.21.15.172:0/3139601359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebc4370 0x5592aebc1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.332+0000 7f7effc16740 1 --2- 172.21.15.172:0/3139601359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 0x5592aebc1630 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.332+0000 7f7effc16740 1 --2- 172.21.15.172:0/3139601359 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5592aebd3ce0 0x5592aebd6110 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.332+0000 7f7effc16740 1 --2- 172.21.15.172:0/3139601359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebb95d0 0x5592aebc10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.332+0000 7f7effc16740 1 -- 172.21.15.172:0/3139601359 >> 172.21.15.172:0/3139601359 conn(0x5592aeb82310 msgr2=0x5592aebbb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:30.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.332+0000 7f7ed6ffd640 0 client.17829 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:54:30.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.332+0000 7f7effc16740 1 -- 172.21.15.172:0/3139601359 shutdown_connections 2024-09-10T08:54:30.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.332+0000 7f7effc16740 1 -- 172.21.15.172:0/3139601359 wait complete. 2024-09-10T08:54:30.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.332+0000 7f7effc16740 20 client.17829 trim_cache size 0 max 16384 2024-09-10T08:54:30.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7effc16740 1 Processor -- start 2024-09-10T08:54:30.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7effc16740 1 -- start start 2024-09-10T08:54:30.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7effc16740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebb95d0 0x5592aebc00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7effc16740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 0x5592aebdb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7effc16740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebc4370 0x5592aebdbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7effc16740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81b0d70 con 0x5592aebbab40 2024-09-10T08:54:30.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7effc16740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81b28b0 con 0x5592aebc4370 2024-09-10T08:54:30.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7effc16740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81b1440 con 0x5592aebb95d0 2024-09-10T08:54:30.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7ef6ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebb95d0 0x5592aebc00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7eeffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 0x5592aebdb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7ef6ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebb95d0 0x5592aebc00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:48446/0 (socket says 172.21.15.172:48446) 2024-09-10T08:54:30.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7ef6ffd640 1 -- 172.21.15.172:0/1912449939 learned_addr learned my addr 172.21.15.172:0/1912449939 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:30.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7ef77fe640 1 --2- 172.21.15.172:0/1912449939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebc4370 0x5592aebdbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7ef6ffd640 1 -- 172.21.15.172:0/1912449939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebc4370 msgr2=0x5592aebdbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:30.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7ef6ffd640 1 --2- 172.21.15.172:0/1912449939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebc4370 0x5592aebdbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7ef6ffd640 1 -- 172.21.15.172:0/1912449939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 msgr2=0x5592aebdb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:30.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7ef6ffd640 1 --2- 172.21.15.172:0/1912449939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 0x5592aebdb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.352+0000 7f7ef6ffd640 1 -- 172.21.15.172:0/1912449939 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592aebb6960 con 0x5592aebb95d0 2024-09-10T08:54:30.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.353+0000 7f7ef6ffd640 1 --2- 172.21.15.172:0/1912449939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebb95d0 0x5592aebc00d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80b8020 tx=0x7f7ee80bbd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f5b63c974971a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:30.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.353+0000 7f7eed7fa640 1 -- 172.21.15.172:0/1912449939 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8049040 con 0x5592aebb95d0 2024-09-10T08:54:30.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.353+0000 7f7eed7fa640 1 -- 172.21.15.172:0/1912449939 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ee80d2040 con 0x5592aebb95d0 2024-09-10T08:54:30.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.353+0000 7f7eed7fa640 1 -- 172.21.15.172:0/1912449939 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee80c4600 con 0x5592aebb95d0 2024-09-10T08:54:30.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.354+0000 7f7effc16740 1 -- 172.21.15.172:0/1912449939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebb95d0 msgr2=0x5592aebc00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:30.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.354+0000 7f7effc16740 1 --2- 172.21.15.172:0/1912449939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebb95d0 0x5592aebc00d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80b8020 tx=0x7f7ee80bbd00 comp rx=0 tx=0).stop 2024-09-10T08:54:30.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.354+0000 7f7effc16740 1 -- 172.21.15.172:0/1912449939 shutdown_connections 2024-09-10T08:54:30.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.354+0000 7f7effc16740 1 --2- 172.21.15.172:0/1912449939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebc4370 0x5592aebdbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.354+0000 7f7effc16740 1 --2- 172.21.15.172:0/1912449939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebbab40 0x5592aebdb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.354+0000 7f7effc16740 1 --2- 172.21.15.172:0/1912449939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebb95d0 0x5592aebc00d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.354+0000 7f7effc16740 1 -- 172.21.15.172:0/1912449939 >> 172.21.15.172:0/1912449939 conn(0x5592aeb82310 msgr2=0x5592aebbf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:30.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.354+0000 7f7effc16740 1 -- 172.21.15.172:0/1912449939 shutdown_connections 2024-09-10T08:54:30.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.354+0000 7f7effc16740 1 -- 172.21.15.172:0/1912449939 wait complete. 2024-09-10T08:54:30.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.354+0000 7f7effc16740 1 -- start start 2024-09-10T08:54:30.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.355+0000 7f7effc16740 1 Processor -- start 2024-09-10T08:54:30.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.355+0000 7f7effc16740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebb95d0 0x5592aebc7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.355+0000 7f7effc16740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebbab40 0x5592aebc76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.355+0000 7f7effc16740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebc4370 0x5592aec3c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.355+0000 7f7effc16740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81b21e0 con 0x5592aebc4370 2024-09-10T08:54:30.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.355+0000 7f7effc16740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81b1b10 con 0x5592aebbab40 2024-09-10T08:54:30.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.355+0000 7f7effc16740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81aed50 con 0x5592aebb95d0 2024-09-10T08:54:30.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.355+0000 7f7effc16740 10 client.0 _ll_register_callbacks cb 0x7f7ee00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:30.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.355+0000 7f7ef77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebc4370 0x5592aec3c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.355+0000 7f7ef77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebc4370 0x5592aec3c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57432/0 (socket says 172.21.15.172:57432) 2024-09-10T08:54:30.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.355+0000 7f7ef77fe640 1 -- 172.21.15.172:0/2344622290 learned_addr learned my addr 172.21.15.172:0/2344622290 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:30.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.355+0000 7f7ef6ffd640 1 --2- 172.21.15.172:0/2344622290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebb95d0 0x5592aebc7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.355+0000 7f7eeffff640 1 --2- 172.21.15.172:0/2344622290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebbab40 0x5592aebc76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.356+0000 7f7ef6ffd640 1 -- 172.21.15.172:0/2344622290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebbab40 msgr2=0x5592aebc76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:30.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.356+0000 7f7ef6ffd640 1 --2- 172.21.15.172:0/2344622290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebbab40 0x5592aebc76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.356+0000 7f7ef6ffd640 1 -- 172.21.15.172:0/2344622290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebc4370 msgr2=0x5592aec3c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:30.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.356+0000 7f7ef6ffd640 1 --2- 172.21.15.172:0/2344622290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebc4370 0x5592aec3c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.356+0000 7f7ef6ffd640 1 -- 172.21.15.172:0/2344622290 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592aebcff10 con 0x5592aebb95d0 2024-09-10T08:54:30.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.356+0000 7f7ef77fe640 1 --2- 172.21.15.172:0/2344622290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebc4370 0x5592aec3c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:30.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.356+0000 7f7ef6ffd640 1 --2- 172.21.15.172:0/2344622290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebb95d0 0x5592aebc7160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80bbb50 tx=0x7f7ee80b9c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d34eca66665ead4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:30.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.356+0000 7f7ed6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:54:30.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.357+0000 7f7ed6ffd640 1 -- 172.21.15.172:0/2344622290 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8049040 con 0x5592aebb95d0 2024-09-10T08:54:30.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.357+0000 7f7ed6ffd640 1 -- 172.21.15.172:0/2344622290 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f7ee80b9ec0 con 0x5592aebb95d0 2024-09-10T08:54:30.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.357+0000 7f7effc16740 1 -- 172.21.15.172:0/2344622290 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ee80d2040 con 0x5592aebb95d0 2024-09-10T08:54:30.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.358+0000 7f7ed6ffd640 1 -- 172.21.15.172:0/2344622290 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7ee80b8230 con 0x5592aebb95d0 2024-09-10T08:54:30.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.358+0000 7f7effc16740 10 client.26812 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:30.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.358+0000 7f7effc16740 1 -- 172.21.15.172:0/2344622290 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5592aec3de30 con 0x5592aebb95d0 2024-09-10T08:54:30.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.358+0000 7f7ed6ffd640 1 -- 172.21.15.172:0/2344622290 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ee80d2040 con 0x5592aebb95d0 2024-09-10T08:54:30.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.358+0000 7f7ed6ffd640 1 -- 172.21.15.172:0/2344622290 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f7ee80b7a30 con 0x5592aebb95d0 2024-09-10T08:54:30.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.358+0000 7f7effc16740 10 client.26812 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:30.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.358+0000 7f7effc16740 10 client.26812 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:54:30.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.358+0000 7f7effc16740 20 client.26812 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:30.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.358+0000 7f7effc16740 1 --2- 172.21.15.172:0/2344622290 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5592aec3e250 0x5592aec40640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:30.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.358+0000 7f7effc16740 4 client.26812 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:54:30.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.358+0000 7f7effc16740 1 -- 172.21.15.172:0/2344622290 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5592aeb26d70 con 0x5592aec3e250 2024-09-10T08:54:30.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.359+0000 7f7eeffff640 1 --2- 172.21.15.172:0/2344622290 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5592aec3e250 0x5592aec40640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:30.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.359+0000 7f7ed6ffd640 10 client.26812 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:54:30.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.359+0000 7f7eeffff640 1 --2- 172.21.15.172:0/2344622290 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5592aec3e250 0x5592aec40640 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4182e1f6e977be8c server_cookie=615f55b76e6d1805 in_seq=0 out_seq=0 2024-09-10T08:54:30.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.359+0000 7f7ed6ffd640 1 -- 172.21.15.172:0/2344622290 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5592aeb26d70 con 0x5592aec3e250 2024-09-10T08:54:30.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.359+0000 7f7ed6ffd640 10 client.26812 handle_command_reply: tid=0 2024-09-10T08:54:30.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.359+0000 7f7effc16740 1 client.26812 shutdown 2024-09-10T08:54:30.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.359+0000 7f7effc16740 1 -- 172.21.15.172:0/2344622290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebb95d0 msgr2=0x5592aebc7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:30.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.359+0000 7f7effc16740 1 --2- 172.21.15.172:0/2344622290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebb95d0 0x5592aebc7160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80bbb50 tx=0x7f7ee80b9c60 comp rx=0 tx=0).stop 2024-09-10T08:54:30.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.359+0000 7f7effc16740 1 -- 172.21.15.172:0/2344622290 shutdown_connections 2024-09-10T08:54:30.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.359+0000 7f7effc16740 1 --2- 172.21.15.172:0/2344622290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592aebc4370 0x5592aec3c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.359+0000 7f7effc16740 1 --2- 172.21.15.172:0/2344622290 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5592aec3e250 0x5592aec40640 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.359+0000 7f7effc16740 1 --2- 172.21.15.172:0/2344622290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592aebbab40 0x5592aebc76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.359+0000 7f7effc16740 1 --2- 172.21.15.172:0/2344622290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5592aebb95d0 0x5592aebc7160 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.359+0000 7f7effc16740 1 -- 172.21.15.172:0/2344622290 >> 172.21.15.172:0/2344622290 conn(0x5592aeb82310 msgr2=0x5592aebb44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:30.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.359+0000 7f7ed6ffd640 0 client.26812 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:54:30.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.360+0000 7f7effc16740 1 -- 172.21.15.172:0/2344622290 shutdown_connections 2024-09-10T08:54:30.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.360+0000 7f7effc16740 1 -- 172.21.15.172:0/2344622290 wait complete. 2024-09-10T08:54:30.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.360+0000 7f7effc16740 20 client.26812 trim_cache size 0 max 16384 2024-09-10T08:54:30.365 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:54:30.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.360+0000 7f7efd83d640 1 -- 172.21.15.172:0/4017050888 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7ee40a2b80 msgr2=0x7f7ee40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:30.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.360+0000 7f7efd83d640 1 --2- 172.21.15.172:0/4017050888 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7ee40a2b80 0x7f7ee40a5040 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0005fd0 tx=0x7f7ef00107a0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.360+0000 7f7efd83d640 1 -- 172.21.15.172:0/4017050888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef8100e20 msgr2=0x7f7ef80fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:30.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.360+0000 7f7efd83d640 1 --2- 172.21.15.172:0/4017050888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef8100e20 0x7f7ef80fcc80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8006010 tx=0x7f7ee80040c0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.360+0000 7f7efd83d640 1 -- 172.21.15.172:0/4017050888 shutdown_connections 2024-09-10T08:54:30.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.360+0000 7f7efd83d640 1 --2- 172.21.15.172:0/4017050888 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f7ee40a2b80 0x7f7ee40a5040 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.360+0000 7f7efd83d640 1 --2- 172.21.15.172:0/4017050888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ef8102ef0 0x7f7ef80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.360+0000 7f7efd83d640 1 --2- 172.21.15.172:0/4017050888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ef8102570 0x7f7ef80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.360+0000 7f7efd83d640 1 --2- 172.21.15.172:0/4017050888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ef8100e20 0x7f7ef80fcc80 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:30.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.360+0000 7f7efd83d640 1 -- 172.21.15.172:0/4017050888 >> 172.21.15.172:0/4017050888 conn(0x7f7ef80fc7d0 msgr2=0x7f7ef806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:30.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.361+0000 7f7efd83d640 1 -- 172.21.15.172:0/4017050888 shutdown_connections 2024-09-10T08:54:30.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:30.361+0000 7f7efd83d640 1 -- 172.21.15.172:0/4017050888 wait complete. 2024-09-10T08:54:30.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:30 smithi172 ceph-mon[30854]: pgmap v598: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:54:30.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:30 smithi174 ceph-mon[34068]: pgmap v598: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:54:30.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:30 smithi191 ceph-mon[35314]: pgmap v598: 129 pgs: 129 active+clean; 7.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:54:31.109 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T08:54:31.390 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:54:31.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.939+0000 7f40314e7640 1 -- 172.21.15.172:0/3519948457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f402c106f20 msgr2=0x7f402c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:31.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.939+0000 7f40314e7640 1 --2- 172.21.15.172:0/3519948457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f402c106f20 0x7f402c10eaa0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f4024009ca0 tx=0x7f402402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.939+0000 7f40314e7640 1 -- 172.21.15.172:0/3519948457 shutdown_connections 2024-09-10T08:54:31.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.939+0000 7f40314e7640 1 --2- 172.21.15.172:0/3519948457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f402c106f20 0x7f402c10eaa0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.939+0000 7f40314e7640 1 --2- 172.21.15.172:0/3519948457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f402c1027a0 0x7f402c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.939+0000 7f40314e7640 1 --2- 172.21.15.172:0/3519948457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f402c101df0 0x7f402c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.939+0000 7f40314e7640 1 -- 172.21.15.172:0/3519948457 >> 172.21.15.172:0/3519948457 conn(0x7f402c0fd8f0 msgr2=0x7f402c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:31.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.939+0000 7f40314e7640 1 -- 172.21.15.172:0/3519948457 shutdown_connections 2024-09-10T08:54:31.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.939+0000 7f40314e7640 1 -- 172.21.15.172:0/3519948457 wait complete. 2024-09-10T08:54:31.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.940+0000 7f40314e7640 1 Processor -- start 2024-09-10T08:54:31.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.940+0000 7f40314e7640 1 -- start start 2024-09-10T08:54:31.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.941+0000 7f40314e7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f402c101df0 0x7f402c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.941+0000 7f40314e7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f402c1027a0 0x7f402c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.941+0000 7f40314e7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f402c106f20 0x7f402c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.941+0000 7f40314e7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f402c111e30 con 0x7f402c106f20 2024-09-10T08:54:31.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.941+0000 7f40314e7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f402c111cb0 con 0x7f402c101df0 2024-09-10T08:54:31.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.941+0000 7f40314e7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f402c111fb0 con 0x7f402c1027a0 2024-09-10T08:54:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.941+0000 7f402b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f402c106f20 0x7f402c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.941+0000 7f402b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f402c106f20 0x7f402c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57444/0 (socket says 172.21.15.172:57444) 2024-09-10T08:54:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.941+0000 7f402b7fe640 1 -- 172.21.15.172:0/3494632849 learned_addr learned my addr 172.21.15.172:0/3494632849 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.941+0000 7f402affd640 1 --2- 172.21.15.172:0/3494632849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f402c101df0 0x7f402c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.941+0000 7f402a7fc640 1 --2- 172.21.15.172:0/3494632849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f402c1027a0 0x7f402c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.942+0000 7f402affd640 1 -- 172.21.15.172:0/3494632849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f402c1027a0 msgr2=0x7f402c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.942+0000 7f402affd640 1 --2- 172.21.15.172:0/3494632849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f402c1027a0 0x7f402c198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.942+0000 7f402affd640 1 -- 172.21.15.172:0/3494632849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f402c106f20 msgr2=0x7f402c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.942+0000 7f402affd640 1 --2- 172.21.15.172:0/3494632849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f402c106f20 0x7f402c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.942+0000 7f402affd640 1 -- 172.21.15.172:0/3494632849 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c19d2e0 con 0x7f402c101df0 2024-09-10T08:54:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.942+0000 7f402b7fe640 1 --2- 172.21.15.172:0/3494632849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f402c106f20 0x7f402c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:31.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.942+0000 7f402affd640 1 --2- 172.21.15.172:0/3494632849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f402c101df0 0x7f402c198330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f401c00d6d0 tx=0x7f401c00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.942+0000 7f4013fff640 1 -- 172.21.15.172:0/3494632849 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c017070 con 0x7f402c101df0 2024-09-10T08:54:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.942+0000 7f40314e7640 1 -- 172.21.15.172:0/3494632849 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402c19d5d0 con 0x7f402c101df0 2024-09-10T08:54:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.942+0000 7f4013fff640 1 -- 172.21.15.172:0/3494632849 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f401c009820 con 0x7f402c101df0 2024-09-10T08:54:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.942+0000 7f40314e7640 1 -- 172.21.15.172:0/3494632849 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402c1a4eb0 con 0x7f402c101df0 2024-09-10T08:54:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.942+0000 7f4013fff640 1 -- 172.21.15.172:0/3494632849 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c004bc0 con 0x7f402c101df0 2024-09-10T08:54:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.945+0000 7f4013fff640 1 -- 172.21.15.172:0/3494632849 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f401c0099d0 con 0x7f402c101df0 2024-09-10T08:54:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.946+0000 7f40338c0740 1 Processor -- start 2024-09-10T08:54:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.946+0000 7f40338c0740 1 -- start start 2024-09-10T08:54:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.946+0000 7f4013fff640 1 --2- 172.21.15.172:0/3494632849 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f40140a7010 0x7f40140a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.946+0000 7f40338c0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65d8b40 0x55bbc65d7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.946+0000 7f40338c0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65d75d0 0x55bbc65d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.946+0000 7f40338c0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65e2370 0x55bbc65e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.946+0000 7f4013fff640 1 -- 172.21.15.172:0/3494632849 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f401c0cee10 con 0x7f402c101df0 2024-09-10T08:54:31.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.946+0000 7f402b7fe640 1 --2- 172.21.15.172:0/3494632849 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f40140a7010 0x7f40140a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.946+0000 7f40338c0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bbc653d2f0 con 0x55bbc65e2370 2024-09-10T08:54:31.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.946+0000 7f40338c0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bbc64e4cf0 con 0x55bbc65d8b40 2024-09-10T08:54:31.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.946+0000 7f40338c0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bbc65392b0 con 0x55bbc65d75d0 2024-09-10T08:54:31.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.946+0000 7f402b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65e2370 0x55bbc65e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.946+0000 7f402affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65d8b40 0x55bbc65d7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.946+0000 7f402affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65d8b40 0x55bbc65d7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:41406/0 (socket says 172.21.15.172:41406) 2024-09-10T08:54:31.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.946+0000 7f402affd640 1 -- 172.21.15.172:0/892755373 learned_addr learned my addr 172.21.15.172:0/892755373 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:31.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.946+0000 7f402a7fc640 1 --2- 172.21.15.172:0/892755373 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65d75d0 0x55bbc65d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.947+0000 7f402affd640 1 -- 172.21.15.172:0/892755373 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65d75d0 msgr2=0x55bbc65d7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:31.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.947+0000 7f402affd640 1 --2- 172.21.15.172:0/892755373 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65d75d0 0x55bbc65d7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.947+0000 7f402affd640 1 -- 172.21.15.172:0/892755373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65e2370 msgr2=0x55bbc65e4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:31.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.947+0000 7f402affd640 1 --2- 172.21.15.172:0/892755373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65e2370 0x55bbc65e4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.947+0000 7f402affd640 1 -- 172.21.15.172:0/892755373 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbc64c93a0 con 0x55bbc65d8b40 2024-09-10T08:54:31.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.947+0000 7f402affd640 1 --2- 172.21.15.172:0/892755373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65d8b40 0x55bbc65d7090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f401c09b0c0 tx=0x7f401c009e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83e1e3b7b6d50c5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:31.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.947+0000 7f402b7fe640 1 --2- 172.21.15.172:0/3494632849 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f40140a7010 0x7f40140a94d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4024005e30 tx=0x7f4024031b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:31.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.947+0000 7f40117fa640 1 -- 172.21.15.172:0/892755373 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c020050 con 0x55bbc65d8b40 2024-09-10T08:54:31.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.947+0000 7f40117fa640 1 -- 172.21.15.172:0/892755373 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f401c0a3f80 con 0x55bbc65d8b40 2024-09-10T08:54:31.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.947+0000 7f40117fa640 1 -- 172.21.15.172:0/892755373 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c0bb0b0 con 0x55bbc65d8b40 2024-09-10T08:54:31.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.948+0000 7f40338c0740 1 -- 172.21.15.172:0/892755373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65d8b40 msgr2=0x55bbc65d7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:31.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.948+0000 7f40338c0740 1 --2- 172.21.15.172:0/892755373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65d8b40 0x55bbc65d7090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f401c09b0c0 tx=0x7f401c009e00 comp rx=0 tx=0).stop 2024-09-10T08:54:31.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.948+0000 7f40338c0740 1 -- 172.21.15.172:0/892755373 shutdown_connections 2024-09-10T08:54:31.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.948+0000 7f40338c0740 1 --2- 172.21.15.172:0/892755373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65e2370 0x55bbc65e4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.948+0000 7f40338c0740 1 --2- 172.21.15.172:0/892755373 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65d75d0 0x55bbc65d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.948+0000 7f40338c0740 1 --2- 172.21.15.172:0/892755373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65d8b40 0x55bbc65d7090 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.948+0000 7f40338c0740 1 -- 172.21.15.172:0/892755373 >> 172.21.15.172:0/892755373 conn(0x55bbc65a0310 msgr2=0x55bbc65d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:31.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.948+0000 7f40338c0740 1 -- 172.21.15.172:0/892755373 shutdown_connections 2024-09-10T08:54:31.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.948+0000 7f40338c0740 1 -- 172.21.15.172:0/892755373 wait complete. 2024-09-10T08:54:31.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.948+0000 7f40338c0740 1 -- start start 2024-09-10T08:54:31.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.948+0000 7f40338c0740 1 Processor -- start 2024-09-10T08:54:31.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.949+0000 7f40338c0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65d75d0 0x55bbc65e1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.949+0000 7f40338c0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d8b40 0x55bbc65df090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.949+0000 7f40338c0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65e2370 0x55bbc65df7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.949+0000 7f40338c0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bbc65e4fb0 con 0x55bbc65d8b40 2024-09-10T08:54:31.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.949+0000 7f40338c0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bbc64f8060 con 0x55bbc65e2370 2024-09-10T08:54:31.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.949+0000 7f40338c0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bbc65e5130 con 0x55bbc65d75d0 2024-09-10T08:54:31.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.949+0000 7f40338c0740 10 client.0 _ll_register_callbacks cb 0x55bbc624b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:31.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.949+0000 7f402a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d8b40 0x55bbc65df090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.949+0000 7f402a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d8b40 0x55bbc65df090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57464/0 (socket says 172.21.15.172:57464) 2024-09-10T08:54:31.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.949+0000 7f402a7fc640 1 -- 172.21.15.172:0/3176097097 learned_addr learned my addr 172.21.15.172:0/3176097097 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:31.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.949+0000 7f402affd640 1 --2- 172.21.15.172:0/3176097097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65d75d0 0x55bbc65e1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.949+0000 7f402b7fe640 1 --2- 172.21.15.172:0/3176097097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65e2370 0x55bbc65df7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.949+0000 7f402a7fc640 1 -- 172.21.15.172:0/3176097097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65d75d0 msgr2=0x55bbc65e1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.949+0000 7f402a7fc640 1 --2- 172.21.15.172:0/3176097097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65d75d0 0x55bbc65e1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.949+0000 7f402a7fc640 1 -- 172.21.15.172:0/3176097097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65e2370 msgr2=0x55bbc65df7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.949+0000 7f402a7fc640 1 --2- 172.21.15.172:0/3176097097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65e2370 0x55bbc65df7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.949+0000 7f402a7fc640 1 -- 172.21.15.172:0/3176097097 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbc62ae3c0 con 0x55bbc65d8b40 2024-09-10T08:54:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.950+0000 7f402a7fc640 1 --2- 172.21.15.172:0/3176097097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d8b40 0x55bbc65df090 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4020012700 tx=0x7f4020012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39924329fca6a67e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.950+0000 7f400eff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.950+0000 7f400eff5640 1 -- 172.21.15.172:0/3176097097 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402000f8c0 con 0x55bbc65d8b40 2024-09-10T08:54:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.950+0000 7f400eff5640 1 -- 172.21.15.172:0/3176097097 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f402000fa60 con 0x55bbc65d8b40 2024-09-10T08:54:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.950+0000 7f40338c0740 1 -- 172.21.15.172:0/3176097097 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f401c0a3f80 con 0x55bbc65d8b40 2024-09-10T08:54:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.950+0000 7f400eff5640 1 -- 172.21.15.172:0/3176097097 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f402000fc60 con 0x55bbc65d8b40 2024-09-10T08:54:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.950+0000 7f40338c0740 10 client.17847 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.950+0000 7f40338c0740 1 -- 172.21.15.172:0/3176097097 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bbc65f0b40 con 0x55bbc65d8b40 2024-09-10T08:54:31.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.950+0000 7f400eff5640 1 -- 172.21.15.172:0/3176097097 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f402000fe00 con 0x55bbc65d8b40 2024-09-10T08:54:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.950+0000 7f400eff5640 1 -- 172.21.15.172:0/3176097097 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f40200055c0 con 0x55bbc65d8b40 2024-09-10T08:54:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.951+0000 7f40338c0740 10 client.17847 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.951+0000 7f40338c0740 10 client.17847 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:54:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.951+0000 7f40338c0740 20 client.17847 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.951+0000 7f40338c0740 1 --2- 172.21.15.172:0/3176097097 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55bbc65f2050 0x55bbc65f4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.951+0000 7f40338c0740 4 client.17847 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:54:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.951+0000 7f40338c0740 1 -- 172.21.15.172:0/3176097097 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f401c0a3f80 con 0x55bbc65f2050 2024-09-10T08:54:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.951+0000 7f402affd640 1 --2- 172.21.15.172:0/3176097097 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55bbc65f2050 0x55bbc65f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.951+0000 7f400eff5640 10 client.17847 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:54:31.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.951+0000 7f402affd640 1 --2- 172.21.15.172:0/3176097097 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55bbc65f2050 0x55bbc65f4470 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ecf047f1f2496721 server_cookie=497a1cc03a03e177 in_seq=0 out_seq=0 2024-09-10T08:54:31.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.952+0000 7f400eff5640 1 -- 172.21.15.172:0/3176097097 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f401c0a3f80 con 0x55bbc65f2050 2024-09-10T08:54:31.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.952+0000 7f400eff5640 10 client.17847 handle_command_reply: tid=0 2024-09-10T08:54:31.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.952+0000 7f40338c0740 1 client.17847 shutdown 2024-09-10T08:54:31.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.953+0000 7f40338c0740 1 -- 172.21.15.172:0/3176097097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d8b40 msgr2=0x55bbc65df090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:31.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.953+0000 7f40338c0740 1 --2- 172.21.15.172:0/3176097097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d8b40 0x55bbc65df090 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4020012700 tx=0x7f4020012bc0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.953+0000 7f40338c0740 1 -- 172.21.15.172:0/3176097097 shutdown_connections 2024-09-10T08:54:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.953+0000 7f40338c0740 1 --2- 172.21.15.172:0/3176097097 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55bbc65f2050 0x55bbc65f4470 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.953+0000 7f40338c0740 1 --2- 172.21.15.172:0/3176097097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65e2370 0x55bbc65df7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.953+0000 7f40338c0740 1 --2- 172.21.15.172:0/3176097097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d8b40 0x55bbc65df090 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.953+0000 7f40338c0740 1 --2- 172.21.15.172:0/3176097097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65d75d0 0x55bbc65e1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.953+0000 7f400eff5640 0 client.17847 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:54:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.953+0000 7f40338c0740 1 -- 172.21.15.172:0/3176097097 >> 172.21.15.172:0/3176097097 conn(0x55bbc65a0310 msgr2=0x55bbc65d99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.953+0000 7f40338c0740 1 -- 172.21.15.172:0/3176097097 shutdown_connections 2024-09-10T08:54:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.953+0000 7f40338c0740 1 -- 172.21.15.172:0/3176097097 wait complete. 2024-09-10T08:54:31.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.953+0000 7f40338c0740 20 client.17847 trim_cache size 0 max 16384 2024-09-10T08:54:31.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f40338c0740 1 Processor -- start 2024-09-10T08:54:31.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f40338c0740 1 -- start start 2024-09-10T08:54:31.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f40338c0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d75d0 0x55bbc65ee250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f40338c0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65d8b40 0x55bbc65da1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f40338c0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65e2370 0x55bbc65da6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f40338c0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f402c1ac940 con 0x55bbc65d75d0 2024-09-10T08:54:31.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f40338c0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f402c1ae480 con 0x55bbc65e2370 2024-09-10T08:54:31.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f40338c0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f402c1ad010 con 0x55bbc65d8b40 2024-09-10T08:54:31.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f402affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d75d0 0x55bbc65ee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f402affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d75d0 0x55bbc65ee250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57472/0 (socket says 172.21.15.172:57472) 2024-09-10T08:54:31.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f402affd640 1 -- 172.21.15.172:0/4224776510 learned_addr learned my addr 172.21.15.172:0/4224776510 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f402a7fc640 1 --2- 172.21.15.172:0/4224776510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65d8b40 0x55bbc65da1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f402b7fe640 1 --2- 172.21.15.172:0/4224776510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65e2370 0x55bbc65da6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f402affd640 1 -- 172.21.15.172:0/4224776510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65d8b40 msgr2=0x55bbc65da1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f402affd640 1 --2- 172.21.15.172:0/4224776510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65d8b40 0x55bbc65da1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f402affd640 1 -- 172.21.15.172:0/4224776510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65e2370 msgr2=0x55bbc65da6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f402affd640 1 --2- 172.21.15.172:0/4224776510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65e2370 0x55bbc65da6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.966+0000 7f402affd640 1 -- 172.21.15.172:0/4224776510 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbc65ee790 con 0x55bbc65d75d0 2024-09-10T08:54:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.967+0000 7f402b7fe640 1 --2- 172.21.15.172:0/4224776510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65e2370 0x55bbc65da6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:54:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.967+0000 7f402affd640 1 --2- 172.21.15.172:0/4224776510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d75d0 0x55bbc65ee250 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f401c097d90 tx=0x7f401c0bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2e03c52f3ead6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:31.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.967+0000 7f4010ff9640 1 -- 172.21.15.172:0/4224776510 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c020070 con 0x55bbc65d75d0 2024-09-10T08:54:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.967+0000 7f4010ff9640 1 -- 172.21.15.172:0/4224776510 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f401c0a3f80 con 0x55bbc65d75d0 2024-09-10T08:54:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.967+0000 7f4010ff9640 1 -- 172.21.15.172:0/4224776510 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c0bb310 con 0x55bbc65d75d0 2024-09-10T08:54:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.968+0000 7f40338c0740 1 -- 172.21.15.172:0/4224776510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d75d0 msgr2=0x55bbc65ee250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.968+0000 7f40338c0740 1 --2- 172.21.15.172:0/4224776510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d75d0 0x55bbc65ee250 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f401c097d90 tx=0x7f401c0bb040 comp rx=0 tx=0).stop 2024-09-10T08:54:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.968+0000 7f40338c0740 1 -- 172.21.15.172:0/4224776510 shutdown_connections 2024-09-10T08:54:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.968+0000 7f40338c0740 1 --2- 172.21.15.172:0/4224776510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65e2370 0x55bbc65da6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.968+0000 7f40338c0740 1 --2- 172.21.15.172:0/4224776510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65d8b40 0x55bbc65da1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.968+0000 7f40338c0740 1 --2- 172.21.15.172:0/4224776510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d75d0 0x55bbc65ee250 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.968+0000 7f40338c0740 1 -- 172.21.15.172:0/4224776510 >> 172.21.15.172:0/4224776510 conn(0x55bbc65a0310 msgr2=0x55bbc65de050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.968+0000 7f40338c0740 1 -- 172.21.15.172:0/4224776510 shutdown_connections 2024-09-10T08:54:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.968+0000 7f40338c0740 1 -- 172.21.15.172:0/4224776510 wait complete. 2024-09-10T08:54:31.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.968+0000 7f40338c0740 1 -- start start 2024-09-10T08:54:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.968+0000 7f40338c0740 1 Processor -- start 2024-09-10T08:54:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.968+0000 7f40338c0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65d75d0 0x55bbc65eab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.968+0000 7f40338c0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d8b40 0x55bbc65e5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.968+0000 7f40338c0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65e2370 0x55bbc65e63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.968+0000 7f40338c0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f402c1addb0 con 0x55bbc65d8b40 2024-09-10T08:54:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.968+0000 7f40338c0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f402c1ad6e0 con 0x55bbc65d75d0 2024-09-10T08:54:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.968+0000 7f40338c0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f402c1aa920 con 0x55bbc65e2370 2024-09-10T08:54:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.969+0000 7f40338c0740 10 client.0 _ll_register_callbacks cb 0x7f4020009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.969+0000 7f402affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65d75d0 0x55bbc65eab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.969+0000 7f402a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d8b40 0x55bbc65e5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.969+0000 7f402affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65d75d0 0x55bbc65eab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:41436/0 (socket says 172.21.15.172:41436) 2024-09-10T08:54:31.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.969+0000 7f402affd640 1 -- 172.21.15.172:0/3809082055 learned_addr learned my addr 172.21.15.172:0/3809082055 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.969+0000 7f402a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d8b40 0x55bbc65e5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57480/0 (socket says 172.21.15.172:57480) 2024-09-10T08:54:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.969+0000 7f402b7fe640 1 --2- 172.21.15.172:0/3809082055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65e2370 0x55bbc65e63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.969+0000 7f402a7fc640 1 -- 172.21.15.172:0/3809082055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65e2370 msgr2=0x55bbc65e63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.969+0000 7f402a7fc640 1 --2- 172.21.15.172:0/3809082055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65e2370 0x55bbc65e63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.969+0000 7f402a7fc640 1 -- 172.21.15.172:0/3809082055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65d75d0 msgr2=0x55bbc65eab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.969+0000 7f402a7fc640 1 --2- 172.21.15.172:0/3809082055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65d75d0 0x55bbc65eab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.969+0000 7f402a7fc640 1 -- 172.21.15.172:0/3809082055 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbc65e6bf0 con 0x55bbc65d8b40 2024-09-10T08:54:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.969+0000 7f402a7fc640 1 --2- 172.21.15.172:0/3809082055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d8b40 0x55bbc65e5e60 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x55bbc65e0640 tx=0x7f40200096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320770f444f47acb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.969+0000 7f400eff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.969+0000 7f400eff5640 1 -- 172.21.15.172:0/3809082055 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020005020 con 0x55bbc65d8b40 2024-09-10T08:54:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.970+0000 7f400eff5640 1 -- 172.21.15.172:0/3809082055 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f402001edd0 con 0x55bbc65d8b40 2024-09-10T08:54:31.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.970+0000 7f40338c0740 1 -- 172.21.15.172:0/3809082055 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f401c0a3f80 con 0x55bbc65d8b40 2024-09-10T08:54:31.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.970+0000 7f400eff5640 1 -- 172.21.15.172:0/3809082055 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f40200051c0 con 0x55bbc65d8b40 2024-09-10T08:54:31.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.970+0000 7f40338c0740 10 client.17853 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:31.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.970+0000 7f40338c0740 1 -- 172.21.15.172:0/3809082055 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bbc665cd20 con 0x55bbc65d8b40 2024-09-10T08:54:31.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.970+0000 7f400eff5640 1 -- 172.21.15.172:0/3809082055 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f401c0a3f80 con 0x55bbc65d8b40 2024-09-10T08:54:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.970+0000 7f400eff5640 1 -- 172.21.15.172:0/3809082055 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f4020003680 con 0x55bbc65d8b40 2024-09-10T08:54:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.971+0000 7f40338c0740 10 client.17853 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.971+0000 7f40338c0740 10 client.17853 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:54:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.971+0000 7f40338c0740 20 client.17853 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.971+0000 7f40338c0740 1 --2- 172.21.15.172:0/3809082055 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55bbc665d4a0 0x55bbc665f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.971+0000 7f40338c0740 4 client.17853 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:54:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.971+0000 7f40338c0740 1 -- 172.21.15.172:0/3809082055 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bbc654a860 con 0x55bbc665d4a0 2024-09-10T08:54:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.971+0000 7f402affd640 1 --2- 172.21.15.172:0/3809082055 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55bbc665d4a0 0x55bbc665f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:31.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.971+0000 7f402affd640 1 --2- 172.21.15.172:0/3809082055 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55bbc665d4a0 0x55bbc665f8c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bc4b46fb73180588 server_cookie=1a07bdd0fc5f2dc4 in_seq=0 out_seq=0 2024-09-10T08:54:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.971+0000 7f400eff5640 10 client.17853 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:54:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.972+0000 7f400eff5640 1 -- 172.21.15.172:0/3809082055 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bbc654a860 con 0x55bbc665d4a0 2024-09-10T08:54:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.972+0000 7f400eff5640 10 client.17853 handle_command_reply: tid=0 2024-09-10T08:54:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.972+0000 7f40338c0740 1 client.17853 shutdown 2024-09-10T08:54:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.972+0000 7f40338c0740 1 -- 172.21.15.172:0/3809082055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d8b40 msgr2=0x55bbc65e5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.972+0000 7f40338c0740 1 --2- 172.21.15.172:0/3809082055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d8b40 0x55bbc65e5e60 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x55bbc65e0640 tx=0x7f40200096c0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.972+0000 7f40338c0740 1 -- 172.21.15.172:0/3809082055 shutdown_connections 2024-09-10T08:54:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.972+0000 7f40338c0740 1 --2- 172.21.15.172:0/3809082055 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55bbc665d4a0 0x55bbc665f8c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.972+0000 7f40338c0740 1 --2- 172.21.15.172:0/3809082055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbc65e2370 0x55bbc65e63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.972+0000 7f40338c0740 1 --2- 172.21.15.172:0/3809082055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbc65d8b40 0x55bbc65e5e60 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.972+0000 7f40338c0740 1 --2- 172.21.15.172:0/3809082055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbc65d75d0 0x55bbc65eab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.972+0000 7f40338c0740 1 -- 172.21.15.172:0/3809082055 >> 172.21.15.172:0/3809082055 conn(0x55bbc65a0310 msgr2=0x55bbc65d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:31.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.972+0000 7f400eff5640 0 client.17853 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:54:31.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.972+0000 7f40338c0740 1 -- 172.21.15.172:0/3809082055 shutdown_connections 2024-09-10T08:54:31.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.972+0000 7f40338c0740 1 -- 172.21.15.172:0/3809082055 wait complete. 2024-09-10T08:54:31.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.972+0000 7f40338c0740 20 client.17853 trim_cache size 0 max 16384 2024-09-10T08:54:31.980 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:54:31.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.973+0000 7f40314e7640 1 -- 172.21.15.172:0/3494632849 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f40140a7010 msgr2=0x7f40140a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:31.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.973+0000 7f40314e7640 1 --2- 172.21.15.172:0/3494632849 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f40140a7010 0x7f40140a94d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4024005e30 tx=0x7f4024031b80 comp rx=0 tx=0).stop 2024-09-10T08:54:31.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.973+0000 7f40314e7640 1 -- 172.21.15.172:0/3494632849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f402c101df0 msgr2=0x7f402c198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:31.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.973+0000 7f40314e7640 1 --2- 172.21.15.172:0/3494632849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f402c101df0 0x7f402c198330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f401c00d6d0 tx=0x7f401c00db90 comp rx=0 tx=0).stop 2024-09-10T08:54:31.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.973+0000 7f40314e7640 1 -- 172.21.15.172:0/3494632849 shutdown_connections 2024-09-10T08:54:31.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.973+0000 7f40314e7640 1 --2- 172.21.15.172:0/3494632849 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f40140a7010 0x7f40140a94d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.973+0000 7f40314e7640 1 --2- 172.21.15.172:0/3494632849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f402c106f20 0x7f402c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.973+0000 7f40314e7640 1 --2- 172.21.15.172:0/3494632849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f402c1027a0 0x7f402c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.973+0000 7f40314e7640 1 --2- 172.21.15.172:0/3494632849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f402c101df0 0x7f402c198330 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:31.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.973+0000 7f40314e7640 1 -- 172.21.15.172:0/3494632849 >> 172.21.15.172:0/3494632849 conn(0x7f402c0fd8f0 msgr2=0x7f402c0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:31.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.973+0000 7f40314e7640 1 -- 172.21.15.172:0/3494632849 shutdown_connections 2024-09-10T08:54:31.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:31.973+0000 7f40314e7640 1 -- 172.21.15.172:0/3494632849 wait complete. 2024-09-10T08:54:32.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:32 smithi174 ceph-mon[34068]: pgmap v599: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-09-10T08:54:32.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:32 smithi191 ceph-mon[35314]: pgmap v599: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-09-10T08:54:32.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:32 smithi172 ceph-mon[30854]: pgmap v599: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-09-10T08:54:32.901 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:54:33.178 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:54:33.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.665+0000 7fdba5e6d640 1 -- 172.21.15.172:0/3815457401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdba0102780 msgr2=0x7fdba01067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:33.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.665+0000 7fdba5e6d640 1 --2- 172.21.15.172:0/3815457401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdba0102780 0x7fdba01067d0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009900 tx=0x7fdb9402a410 comp rx=0 tx=0).stop 2024-09-10T08:54:33.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.665+0000 7fdba5e6d640 1 -- 172.21.15.172:0/3815457401 shutdown_connections 2024-09-10T08:54:33.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.665+0000 7fdba5e6d640 1 --2- 172.21.15.172:0/3815457401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdba0106f00 0x7fdba010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.665+0000 7fdba5e6d640 1 --2- 172.21.15.172:0/3815457401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdba0102780 0x7fdba01067d0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.665+0000 7fdba5e6d640 1 --2- 172.21.15.172:0/3815457401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdba0101dd0 0x7fdba01021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.665+0000 7fdba5e6d640 1 -- 172.21.15.172:0/3815457401 >> 172.21.15.172:0/3815457401 conn(0x7fdba00fd890 msgr2=0x7fdba00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:33.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.665+0000 7fdba5e6d640 1 -- 172.21.15.172:0/3815457401 shutdown_connections 2024-09-10T08:54:33.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.665+0000 7fdba5e6d640 1 -- 172.21.15.172:0/3815457401 wait complete. 2024-09-10T08:54:33.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.666+0000 7fdba5e6d640 1 Processor -- start 2024-09-10T08:54:33.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.666+0000 7fdba5e6d640 1 -- start start 2024-09-10T08:54:33.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.666+0000 7fdba5e6d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdba0101dd0 0x7fdba0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.666+0000 7fdba5e6d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdba0102780 0x7fdba0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.666+0000 7fdba5e6d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdba0106f00 0x7fdba019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.666+0000 7fdba5e6d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdba0111dd0 con 0x7fdba0102780 2024-09-10T08:54:33.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.666+0000 7fdba5e6d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdba0111c50 con 0x7fdba0101dd0 2024-09-10T08:54:33.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.666+0000 7fdba5e6d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdba0111f50 con 0x7fdba0106f00 2024-09-10T08:54:33.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.666+0000 7fdb9effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdba0102780 0x7fdba0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:33.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.666+0000 7fdb9effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdba0102780 0x7fdba0198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57494/0 (socket says 172.21.15.172:57494) 2024-09-10T08:54:33.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.666+0000 7fdb9effd640 1 -- 172.21.15.172:0/2998495838 learned_addr learned my addr 172.21.15.172:0/2998495838 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:33.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.666+0000 7fdb9f7fe640 1 --2- 172.21.15.172:0/2998495838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdba0101dd0 0x7fdba0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:33.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.666+0000 7fdb9ffff640 1 --2- 172.21.15.172:0/2998495838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdba0106f00 0x7fdba019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:33.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.667+0000 7fdb9effd640 1 -- 172.21.15.172:0/2998495838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdba0106f00 msgr2=0x7fdba019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:33.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.667+0000 7fdb9effd640 1 --2- 172.21.15.172:0/2998495838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdba0106f00 0x7fdba019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.667+0000 7fdb9effd640 1 -- 172.21.15.172:0/2998495838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdba0101dd0 msgr2=0x7fdba0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:33.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.667+0000 7fdb9effd640 1 --2- 172.21.15.172:0/2998495838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdba0101dd0 0x7fdba0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.667+0000 7fdb9effd640 1 -- 172.21.15.172:0/2998495838 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba019d2e0 con 0x7fdba0102780 2024-09-10T08:54:33.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.667+0000 7fdb9ffff640 1 --2- 172.21.15.172:0/2998495838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdba0106f00 0x7fdba019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:54:33.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.667+0000 7fdb9effd640 1 --2- 172.21.15.172:0/2998495838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdba0102780 0x7fdba0198870 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fdb94005940 tx=0x7fdb940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:33.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.667+0000 7fdb9cff9640 1 -- 172.21.15.172:0/2998495838 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9403f030 con 0x7fdba0102780 2024-09-10T08:54:33.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.667+0000 7fdba5e6d640 1 -- 172.21.15.172:0/2998495838 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba019d570 con 0x7fdba0102780 2024-09-10T08:54:33.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.667+0000 7fdb9cff9640 1 -- 172.21.15.172:0/2998495838 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb94003cb0 con 0x7fdba0102780 2024-09-10T08:54:33.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.667+0000 7fdb9cff9640 1 -- 172.21.15.172:0/2998495838 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb94035070 con 0x7fdba0102780 2024-09-10T08:54:33.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.667+0000 7fdba5e6d640 1 -- 172.21.15.172:0/2998495838 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba01a4eb0 con 0x7fdba0102780 2024-09-10T08:54:33.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.670+0000 7fdb9cff9640 1 -- 172.21.15.172:0/2998495838 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fdb94042030 con 0x7fdba0102780 2024-09-10T08:54:33.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.670+0000 7fdba8246740 1 Processor -- start 2024-09-10T08:54:33.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.670+0000 7fdba8246740 1 -- start start 2024-09-10T08:54:33.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.670+0000 7fdba8246740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427efb40 0x55ff427ee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.670+0000 7fdba8246740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427ee5d0 0x55ff427eea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.670+0000 7fdba8246740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427f9370 0x55ff427fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.670+0000 7fdba8246740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff427542f0 con 0x55ff427f9370 2024-09-10T08:54:33.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.670+0000 7fdba8246740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ff426fbcf0 con 0x55ff427ee5d0 2024-09-10T08:54:33.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.670+0000 7fdba8246740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ff427502b0 con 0x55ff427efb40 2024-09-10T08:54:33.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.671+0000 7fdb9effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427f9370 0x55ff427fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:33.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.671+0000 7fdb9cff9640 1 --2- 172.21.15.172:0/2998495838 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb880a2b80 0x7fdb880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.671+0000 7fdb9effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427f9370 0x55ff427fb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57510/0 (socket says 172.21.15.172:57510) 2024-09-10T08:54:33.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.671+0000 7fdb9effd640 1 -- 172.21.15.172:0/1898508535 learned_addr learned my addr 172.21.15.172:0/1898508535 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:33.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.671+0000 7fdb9f7fe640 1 --2- 172.21.15.172:0/1898508535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427ee5d0 0x55ff427eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:33.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.671+0000 7fdb9ffff640 1 --2- 172.21.15.172:0/1898508535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427efb40 0x55ff427ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:33.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.671+0000 7fdb9ffff640 1 --2- 172.21.15.172:0/2998495838 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb880a2b80 0x7fdb880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:33.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.671+0000 7fdb9ffff640 1 -- 172.21.15.172:0/1898508535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427ee5d0 msgr2=0x55ff427eea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:33.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.671+0000 7fdb9cff9640 1 -- 172.21.15.172:0/2998495838 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fdb940ef800 con 0x7fdba0102780 2024-09-10T08:54:33.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.671+0000 7fdb9ffff640 1 --2- 172.21.15.172:0/1898508535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427ee5d0 0x55ff427eea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.671+0000 7fdb9ffff640 1 -- 172.21.15.172:0/1898508535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427f9370 msgr2=0x55ff427fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:33.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.671+0000 7fdb9ffff640 1 --2- 172.21.15.172:0/1898508535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427f9370 0x55ff427fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.671+0000 7fdb9ffff640 1 -- 172.21.15.172:0/1898508535 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff426e03a0 con 0x55ff427efb40 2024-09-10T08:54:33.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.671+0000 7fdb9f7fe640 1 --2- 172.21.15.172:0/1898508535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427ee5d0 0x55ff427eea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:33.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.671+0000 7fdb9effd640 1 --2- 172.21.15.172:0/1898508535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427f9370 0x55ff427fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:33.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.671+0000 7fdb9ffff640 1 --2- 172.21.15.172:0/1898508535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427efb40 0x55ff427ee090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdb980129f0 tx=0x7fdb98012eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca50c391cf70219 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:33.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.671+0000 7fdb9ffff640 1 --2- 172.21.15.172:0/2998495838 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb880a2b80 0x7fdb880a5040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdb9801c000 tx=0x7fdb98020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:33.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.672+0000 7fdb85ffb640 1 -- 172.21.15.172:0/1898508535 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98012280 con 0x55ff427efb40 2024-09-10T08:54:33.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.672+0000 7fdb85ffb640 1 -- 172.21.15.172:0/1898508535 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb98023070 con 0x55ff427efb40 2024-09-10T08:54:33.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.672+0000 7fdb85ffb640 1 -- 172.21.15.172:0/1898508535 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98010ad0 con 0x55ff427efb40 2024-09-10T08:54:33.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.672+0000 7fdba8246740 1 -- 172.21.15.172:0/1898508535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427efb40 msgr2=0x55ff427ee090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:33.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.672+0000 7fdba8246740 1 --2- 172.21.15.172:0/1898508535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427efb40 0x55ff427ee090 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdb980129f0 tx=0x7fdb98012eb0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.672+0000 7fdba8246740 1 -- 172.21.15.172:0/1898508535 shutdown_connections 2024-09-10T08:54:33.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.672+0000 7fdba8246740 1 --2- 172.21.15.172:0/1898508535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427f9370 0x55ff427fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.672+0000 7fdba8246740 1 --2- 172.21.15.172:0/1898508535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427ee5d0 0x55ff427eea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.672+0000 7fdba8246740 1 --2- 172.21.15.172:0/1898508535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427efb40 0x55ff427ee090 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.672+0000 7fdba8246740 1 -- 172.21.15.172:0/1898508535 >> 172.21.15.172:0/1898508535 conn(0x55ff427b7310 msgr2=0x55ff427eac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:33.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.673+0000 7fdba8246740 1 -- 172.21.15.172:0/1898508535 shutdown_connections 2024-09-10T08:54:33.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.673+0000 7fdba8246740 1 -- 172.21.15.172:0/1898508535 wait complete. 2024-09-10T08:54:33.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.673+0000 7fdba8246740 1 -- start start 2024-09-10T08:54:33.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.673+0000 7fdba8246740 1 Processor -- start 2024-09-10T08:54:33.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.673+0000 7fdba8246740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427ee5d0 0x55ff427f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.673+0000 7fdba8246740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427efb40 0x55ff427f6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.673+0000 7fdba8246740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427f9370 0x55ff427f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.673+0000 7fdba8246740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff427fc010 con 0x55ff427f9370 2024-09-10T08:54:33.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.673+0000 7fdba8246740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ff4270f060 con 0x55ff427ee5d0 2024-09-10T08:54:33.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.673+0000 7fdba8246740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ff427fc190 con 0x55ff427efb40 2024-09-10T08:54:33.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.674+0000 7fdba8246740 10 client.0 _ll_register_callbacks cb 0x55ff424628c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:33.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.674+0000 7fdb9ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427f9370 0x55ff427f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:33.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.674+0000 7fdb9f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427ee5d0 0x55ff427f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:33.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.674+0000 7fdb9ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427f9370 0x55ff427f66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57516/0 (socket says 172.21.15.172:57516) 2024-09-10T08:54:33.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.674+0000 7fdb9ffff640 1 -- 172.21.15.172:0/1600011864 learned_addr learned my addr 172.21.15.172:0/1600011864 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:33.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.674+0000 7fdb9ffff640 1 -- 172.21.15.172:0/1600011864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427efb40 msgr2=0x55ff427f6040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:54:33.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.674+0000 7fdb9ffff640 1 --2- 172.21.15.172:0/1600011864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427efb40 0x55ff427f6040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.674+0000 7fdb9ffff640 1 -- 172.21.15.172:0/1600011864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427ee5d0 msgr2=0x55ff427f8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:33.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.674+0000 7fdb9ffff640 1 --2- 172.21.15.172:0/1600011864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427ee5d0 0x55ff427f8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.674+0000 7fdb9ffff640 1 -- 172.21.15.172:0/1600011864 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff424c53c0 con 0x55ff427f9370 2024-09-10T08:54:33.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.674+0000 7fdb9f7fe640 1 --2- 172.21.15.172:0/1600011864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427ee5d0 0x55ff427f8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:33.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.674+0000 7fdb9ffff640 1 --2- 172.21.15.172:0/1600011864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427f9370 0x55ff427f66a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fdb98015090 tx=0x7fdb98004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3784bf18fa2cd85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:33.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.674+0000 7fdb7effd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:33.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.674+0000 7fdb7effd640 1 -- 172.21.15.172:0/1600011864 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98035060 con 0x55ff427f9370 2024-09-10T08:54:33.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.674+0000 7fdb7effd640 1 -- 172.21.15.172:0/1600011864 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fdb9801fad0 con 0x55ff427f9370 2024-09-10T08:54:33.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.675+0000 7fdba8246740 1 -- 172.21.15.172:0/1600011864 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb98023070 con 0x55ff427f9370 2024-09-10T08:54:33.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.675+0000 7fdb7effd640 1 -- 172.21.15.172:0/1600011864 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdb9803c050 con 0x55ff427f9370 2024-09-10T08:54:33.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.675+0000 7fdba8246740 10 client.17865 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:33.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.675+0000 7fdba8246740 1 -- 172.21.15.172:0/1600011864 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ff42807ac0 con 0x55ff427f9370 2024-09-10T08:54:33.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.675+0000 7fdb7effd640 1 -- 172.21.15.172:0/1600011864 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb98023070 con 0x55ff427f9370 2024-09-10T08:54:33.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.675+0000 7fdb7effd640 1 -- 172.21.15.172:0/1600011864 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fdb9801e070 con 0x55ff427f9370 2024-09-10T08:54:33.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.675+0000 7fdba8246740 10 client.17865 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:33.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.675+0000 7fdba8246740 10 client.17865 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:54:33.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.675+0000 7fdba8246740 20 client.17865 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:33.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.675+0000 7fdba8246740 1 --2- 172.21.15.172:0/1600011864 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55ff42808f20 0x55ff4280b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.675+0000 7fdba8246740 4 client.17865 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:54:33.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.675+0000 7fdba8246740 1 -- 172.21.15.172:0/1600011864 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff42761860 con 0x55ff42808f20 2024-09-10T08:54:33.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.676+0000 7fdb9f7fe640 1 --2- 172.21.15.172:0/1600011864 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55ff42808f20 0x55ff4280b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:33.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.676+0000 7fdb9f7fe640 1 --2- 172.21.15.172:0/1600011864 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55ff42808f20 0x55ff4280b310 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d0fb56850603f5a server_cookie=2cae4ee11cc0a12b in_seq=0 out_seq=0 2024-09-10T08:54:33.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.676+0000 7fdb7effd640 10 client.17865 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:54:33.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.676+0000 7fdb7effd640 1 -- 172.21.15.172:0/1600011864 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff42761860 con 0x55ff42808f20 2024-09-10T08:54:33.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.676+0000 7fdb7effd640 10 client.17865 handle_command_reply: tid=0 2024-09-10T08:54:33.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.677+0000 7fdba8246740 1 client.17865 shutdown 2024-09-10T08:54:33.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.677+0000 7fdba8246740 1 -- 172.21.15.172:0/1600011864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427f9370 msgr2=0x55ff427f66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:33.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.677+0000 7fdba8246740 1 --2- 172.21.15.172:0/1600011864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427f9370 0x55ff427f66a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fdb98015090 tx=0x7fdb98004c70 comp rx=0 tx=0).stop 2024-09-10T08:54:33.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.677+0000 7fdba8246740 1 -- 172.21.15.172:0/1600011864 shutdown_connections 2024-09-10T08:54:33.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.677+0000 7fdba8246740 1 --2- 172.21.15.172:0/1600011864 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55ff42808f20 0x55ff4280b310 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.677+0000 7fdba8246740 1 --2- 172.21.15.172:0/1600011864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427f9370 0x55ff427f66a0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.677+0000 7fdba8246740 1 --2- 172.21.15.172:0/1600011864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427efb40 0x55ff427f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.677+0000 7fdba8246740 1 --2- 172.21.15.172:0/1600011864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427ee5d0 0x55ff427f8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.677+0000 7fdba8246740 1 -- 172.21.15.172:0/1600011864 >> 172.21.15.172:0/1600011864 conn(0x55ff427b7310 msgr2=0x55ff427f0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:33.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.677+0000 7fdb7effd640 0 client.17865 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:54:33.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.677+0000 7fdba8246740 1 -- 172.21.15.172:0/1600011864 shutdown_connections 2024-09-10T08:54:33.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.677+0000 7fdba8246740 1 -- 172.21.15.172:0/1600011864 wait complete. 2024-09-10T08:54:33.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.677+0000 7fdba8246740 20 client.17865 trim_cache size 0 max 16384 2024-09-10T08:54:33.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.690+0000 7fdba8246740 1 Processor -- start 2024-09-10T08:54:33.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.690+0000 7fdba8246740 1 -- start start 2024-09-10T08:54:33.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.690+0000 7fdba8246740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427ee5d0 0x55ff427f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.690+0000 7fdba8246740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427efb40 0x55ff427f1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.690+0000 7fdba8246740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427f9370 0x55ff427f1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.690+0000 7fdba8246740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01aea60 con 0x55ff427ee5d0 2024-09-10T08:54:33.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.690+0000 7fdba8246740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01b05a0 con 0x55ff427f9370 2024-09-10T08:54:33.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.690+0000 7fdba8246740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01af130 con 0x55ff427efb40 2024-09-10T08:54:33.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.690+0000 7fdb9f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427ee5d0 0x55ff427f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:33.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.690+0000 7fdb9f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427ee5d0 0x55ff427f57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57528/0 (socket says 172.21.15.172:57528) 2024-09-10T08:54:33.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.690+0000 7fdb9f7fe640 1 -- 172.21.15.172:0/3751180892 learned_addr learned my addr 172.21.15.172:0/3751180892 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:33.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.691+0000 7fdb9effd640 1 --2- 172.21.15.172:0/3751180892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427efb40 0x55ff427f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:33.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.691+0000 7fdb9ffff640 1 --2- 172.21.15.172:0/3751180892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427f9370 0x55ff427f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:33.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.691+0000 7fdb9f7fe640 1 -- 172.21.15.172:0/3751180892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427efb40 msgr2=0x55ff427f1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:33.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.691+0000 7fdb9f7fe640 1 --2- 172.21.15.172:0/3751180892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427efb40 0x55ff427f1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.691+0000 7fdb9f7fe640 1 -- 172.21.15.172:0/3751180892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427f9370 msgr2=0x55ff427f1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:33.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.691+0000 7fdb9f7fe640 1 --2- 172.21.15.172:0/3751180892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427f9370 0x55ff427f1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.691+0000 7fdb9f7fe640 1 -- 172.21.15.172:0/3751180892 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff427f6ca0 con 0x55ff427ee5d0 2024-09-10T08:54:33.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.691+0000 7fdb9effd640 1 --2- 172.21.15.172:0/3751180892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427efb40 0x55ff427f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:54:33.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.691+0000 7fdb9f7fe640 1 --2- 172.21.15.172:0/3751180892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427ee5d0 0x55ff427f57f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fdb90005e90 tx=0x7fdb9000a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68bb02d5f883c5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:33.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.691+0000 7fdb857fa640 1 -- 172.21.15.172:0/3751180892 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90004220 con 0x55ff427ee5d0 2024-09-10T08:54:33.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.691+0000 7fdb857fa640 1 -- 172.21.15.172:0/3751180892 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb900043c0 con 0x55ff427ee5d0 2024-09-10T08:54:33.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.691+0000 7fdb857fa640 1 -- 172.21.15.172:0/3751180892 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9001e8e0 con 0x55ff427ee5d0 2024-09-10T08:54:33.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.692+0000 7fdba8246740 1 -- 172.21.15.172:0/3751180892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427ee5d0 msgr2=0x55ff427f57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:33.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.692+0000 7fdba8246740 1 --2- 172.21.15.172:0/3751180892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427ee5d0 0x55ff427f57f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fdb90005e90 tx=0x7fdb9000a2d0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.692+0000 7fdba8246740 1 -- 172.21.15.172:0/3751180892 shutdown_connections 2024-09-10T08:54:33.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.692+0000 7fdba8246740 1 --2- 172.21.15.172:0/3751180892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427f9370 0x55ff427f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.692+0000 7fdba8246740 1 --2- 172.21.15.172:0/3751180892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427efb40 0x55ff427f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.692+0000 7fdba8246740 1 --2- 172.21.15.172:0/3751180892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427ee5d0 0x55ff427f57f0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.692+0000 7fdba8246740 1 -- 172.21.15.172:0/3751180892 >> 172.21.15.172:0/3751180892 conn(0x55ff427b7310 msgr2=0x55ff427f80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:33.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.692+0000 7fdba8246740 1 -- 172.21.15.172:0/3751180892 shutdown_connections 2024-09-10T08:54:33.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.692+0000 7fdba8246740 1 -- 172.21.15.172:0/3751180892 wait complete. 2024-09-10T08:54:33.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.692+0000 7fdba8246740 1 -- start start 2024-09-10T08:54:33.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.693+0000 7fdba8246740 1 Processor -- start 2024-09-10T08:54:33.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.693+0000 7fdba8246740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427ee5d0 0x55ff4286bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.693+0000 7fdba8246740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427efb40 0x55ff4286c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.693+0000 7fdba8246740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427f9370 0x55ff427fca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.693+0000 7fdba8246740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01afed0 con 0x55ff427ee5d0 2024-09-10T08:54:33.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.693+0000 7fdba8246740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01af800 con 0x55ff427efb40 2024-09-10T08:54:33.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.693+0000 7fdba8246740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdba01aca40 con 0x55ff427f9370 2024-09-10T08:54:33.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.693+0000 7fdba8246740 10 client.0 _ll_register_callbacks cb 0x7fdb940b7dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:33.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.693+0000 7fdb9f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427ee5d0 0x55ff4286bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:33.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.693+0000 7fdb9f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427ee5d0 0x55ff4286bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57532/0 (socket says 172.21.15.172:57532) 2024-09-10T08:54:33.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.693+0000 7fdb9f7fe640 1 -- 172.21.15.172:0/2576665988 learned_addr learned my addr 172.21.15.172:0/2576665988 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:33.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.693+0000 7fdb9effd640 1 --2- 172.21.15.172:0/2576665988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427efb40 0x55ff4286c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:33.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.693+0000 7fdb9ffff640 1 --2- 172.21.15.172:0/2576665988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427f9370 0x55ff427fca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:33.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.694+0000 7fdb9f7fe640 1 -- 172.21.15.172:0/2576665988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427f9370 msgr2=0x55ff427fca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:33.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.694+0000 7fdb9f7fe640 1 --2- 172.21.15.172:0/2576665988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427f9370 0x55ff427fca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.694+0000 7fdb9f7fe640 1 -- 172.21.15.172:0/2576665988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427efb40 msgr2=0x55ff4286c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:33.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.694+0000 7fdb9f7fe640 1 --2- 172.21.15.172:0/2576665988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427efb40 0x55ff4286c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.694+0000 7fdb9f7fe640 1 -- 172.21.15.172:0/2576665988 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff427ea510 con 0x55ff427ee5d0 2024-09-10T08:54:33.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.694+0000 7fdb9ffff640 1 --2- 172.21.15.172:0/2576665988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427f9370 0x55ff427fca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:54:33.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.694+0000 7fdb9effd640 1 --2- 172.21.15.172:0/2576665988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427efb40 0x55ff4286c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:33.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.694+0000 7fdb9f7fe640 1 --2- 172.21.15.172:0/2576665988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427ee5d0 0x55ff4286bd60 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fdb9001e4e0 tx=0x7fdb900040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da84ad0d0ef458c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:33.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.694+0000 7fdb7effd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:33.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.694+0000 7fdb7effd640 1 -- 172.21.15.172:0/2576665988 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90034020 con 0x55ff427ee5d0 2024-09-10T08:54:33.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.694+0000 7fdb7effd640 1 -- 172.21.15.172:0/2576665988 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fdb90003800 con 0x55ff427ee5d0 2024-09-10T08:54:33.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.694+0000 7fdba8246740 1 -- 172.21.15.172:0/2576665988 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb900043c0 con 0x55ff427ee5d0 2024-09-10T08:54:33.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.694+0000 7fdb7effd640 1 -- 172.21.15.172:0/2576665988 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdb9003f050 con 0x55ff427ee5d0 2024-09-10T08:54:33.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.694+0000 7fdba8246740 10 client.17871 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:33.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.694+0000 7fdba8246740 1 -- 172.21.15.172:0/2576665988 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ff427eebe0 con 0x55ff427ee5d0 2024-09-10T08:54:33.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.695+0000 7fdb7effd640 1 -- 172.21.15.172:0/2576665988 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb900043c0 con 0x55ff427ee5d0 2024-09-10T08:54:33.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.695+0000 7fdb7effd640 1 -- 172.21.15.172:0/2576665988 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fdb90030460 con 0x55ff427ee5d0 2024-09-10T08:54:33.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.695+0000 7fdba8246740 10 client.17871 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:33.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.695+0000 7fdba8246740 10 client.17871 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:54:33.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.695+0000 7fdba8246740 20 client.17871 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:33.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.695+0000 7fdba8246740 1 --2- 172.21.15.172:0/2576665988 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55ff42877ca0 0x55ff4287a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:33.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.695+0000 7fdba8246740 4 client.17871 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:54:33.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.695+0000 7fdba8246740 1 -- 172.21.15.172:0/2576665988 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff4275bd70 con 0x55ff42877ca0 2024-09-10T08:54:33.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.695+0000 7fdb9effd640 1 --2- 172.21.15.172:0/2576665988 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55ff42877ca0 0x55ff4287a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:33.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.696+0000 7fdb7effd640 10 client.17871 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:54:33.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.696+0000 7fdb9effd640 1 --2- 172.21.15.172:0/2576665988 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55ff42877ca0 0x55ff4287a090 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b2e0c084b79ec9e server_cookie=82118e4214e1360a in_seq=0 out_seq=0 2024-09-10T08:54:33.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.696+0000 7fdb7effd640 1 -- 172.21.15.172:0/2576665988 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff4275bd70 con 0x55ff42877ca0 2024-09-10T08:54:33.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.696+0000 7fdb7effd640 10 client.17871 handle_command_reply: tid=0 2024-09-10T08:54:33.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.696+0000 7fdba8246740 1 client.17871 shutdown 2024-09-10T08:54:33.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.696+0000 7fdba8246740 1 -- 172.21.15.172:0/2576665988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427ee5d0 msgr2=0x55ff4286bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:33.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.696+0000 7fdba8246740 1 --2- 172.21.15.172:0/2576665988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427ee5d0 0x55ff4286bd60 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fdb9001e4e0 tx=0x7fdb900040b0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.696+0000 7fdba8246740 1 -- 172.21.15.172:0/2576665988 shutdown_connections 2024-09-10T08:54:33.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.696+0000 7fdba8246740 1 --2- 172.21.15.172:0/2576665988 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55ff42877ca0 0x55ff4287a090 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.696+0000 7fdba8246740 1 --2- 172.21.15.172:0/2576665988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ff427f9370 0x55ff427fca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.696+0000 7fdba8246740 1 --2- 172.21.15.172:0/2576665988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ff427efb40 0x55ff4286c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.696+0000 7fdba8246740 1 --2- 172.21.15.172:0/2576665988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff427ee5d0 0x55ff4286bd60 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.696+0000 7fdba8246740 1 -- 172.21.15.172:0/2576665988 >> 172.21.15.172:0/2576665988 conn(0x55ff427b7310 msgr2=0x55ff427f78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:33.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.696+0000 7fdb7effd640 0 client.17871 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:54:33.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.696+0000 7fdba8246740 1 -- 172.21.15.172:0/2576665988 shutdown_connections 2024-09-10T08:54:33.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.696+0000 7fdba8246740 1 -- 172.21.15.172:0/2576665988 wait complete. 2024-09-10T08:54:33.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.696+0000 7fdba8246740 20 client.17871 trim_cache size 0 max 16384 2024-09-10T08:54:33.704 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:54:33.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.697+0000 7fdba5e6d640 1 -- 172.21.15.172:0/2998495838 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb880a2b80 msgr2=0x7fdb880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:33.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.697+0000 7fdba5e6d640 1 --2- 172.21.15.172:0/2998495838 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb880a2b80 0x7fdb880a5040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdb9801c000 tx=0x7fdb98020000 comp rx=0 tx=0).stop 2024-09-10T08:54:33.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.697+0000 7fdba5e6d640 1 -- 172.21.15.172:0/2998495838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdba0102780 msgr2=0x7fdba0198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:33.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.697+0000 7fdba5e6d640 1 --2- 172.21.15.172:0/2998495838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdba0102780 0x7fdba0198870 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fdb94005940 tx=0x7fdb940042f0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.697+0000 7fdba5e6d640 1 -- 172.21.15.172:0/2998495838 shutdown_connections 2024-09-10T08:54:33.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.697+0000 7fdba5e6d640 1 --2- 172.21.15.172:0/2998495838 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdb880a2b80 0x7fdb880a5040 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.697+0000 7fdba5e6d640 1 --2- 172.21.15.172:0/2998495838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdba0106f00 0x7fdba019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.697+0000 7fdba5e6d640 1 --2- 172.21.15.172:0/2998495838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdba0102780 0x7fdba0198870 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.697+0000 7fdba5e6d640 1 --2- 172.21.15.172:0/2998495838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdba0101dd0 0x7fdba0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:33.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.697+0000 7fdba5e6d640 1 -- 172.21.15.172:0/2998495838 >> 172.21.15.172:0/2998495838 conn(0x7fdba00fd890 msgr2=0x7fdba00fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:33.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.697+0000 7fdba5e6d640 1 -- 172.21.15.172:0/2998495838 shutdown_connections 2024-09-10T08:54:33.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:33.697+0000 7fdba5e6d640 1 -- 172.21.15.172:0/2998495838 wait complete. 2024-09-10T08:54:34.491 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:54:34.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:34 smithi174 ceph-mon[34068]: pgmap v600: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-09-10T08:54:34.777 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:54:34.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:34 smithi191 ceph-mon[35314]: pgmap v600: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-09-10T08:54:34.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:34 smithi172 ceph-mon[30854]: pgmap v600: 129 pgs: 129 active+clean; 7.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-09-10T08:54:35.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.293+0000 7f1f2cc99640 1 -- 172.21.15.172:0/2186243646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f28102d30 msgr2=0x7f1f28103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:35.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.293+0000 7f1f2cc99640 1 --2- 172.21.15.172:0/2186243646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f28102d30 0x7f1f28103110 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009930 tx=0x7f1f1802a440 comp rx=0 tx=0).stop 2024-09-10T08:54:35.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.293+0000 7f1f2cc99640 1 -- 172.21.15.172:0/2186243646 shutdown_connections 2024-09-10T08:54:35.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.293+0000 7f1f2cc99640 1 --2- 172.21.15.172:0/2186243646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f281082a0 0x7f1f2810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.293+0000 7f1f2cc99640 1 --2- 172.21.15.172:0/2186243646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f281036e0 0x7f1f28107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.293+0000 7f1f2cc99640 1 --2- 172.21.15.172:0/2186243646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f28102d30 0x7f1f28103110 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.293+0000 7f1f2cc99640 1 -- 172.21.15.172:0/2186243646 >> 172.21.15.172:0/2186243646 conn(0x7f1f280fc7d0 msgr2=0x7f1f280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:35.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.293+0000 7f1f2cc99640 1 -- 172.21.15.172:0/2186243646 shutdown_connections 2024-09-10T08:54:35.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.293+0000 7f1f2cc99640 1 -- 172.21.15.172:0/2186243646 wait complete. 2024-09-10T08:54:35.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.294+0000 7f1f2cc99640 1 Processor -- start 2024-09-10T08:54:35.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.294+0000 7f1f2cc99640 1 -- start start 2024-09-10T08:54:35.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f2cc99640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f28102d30 0x7f1f28198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f2cc99640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f281036e0 0x7f1f28198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f2cc99640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f281082a0 0x7f1f2819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f2cc99640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1f28111fb0 con 0x7f1f28102d30 2024-09-10T08:54:35.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f2cc99640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f28111e30 con 0x7f1f281036e0 2024-09-10T08:54:35.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f2cc99640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1f28112130 con 0x7f1f281082a0 2024-09-10T08:54:35.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f2659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f28102d30 0x7f1f28198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f2659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f28102d30 0x7f1f28198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57556/0 (socket says 172.21.15.172:57556) 2024-09-10T08:54:35.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f2659c640 1 -- 172.21.15.172:0/2097277508 learned_addr learned my addr 172.21.15.172:0/2097277508 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:35.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f26d9d640 1 --2- 172.21.15.172:0/2097277508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f281082a0 0x7f1f2819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f25d9b640 1 --2- 172.21.15.172:0/2097277508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f281036e0 0x7f1f28198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f26d9d640 1 -- 172.21.15.172:0/2097277508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f281036e0 msgr2=0x7f1f28198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:35.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f26d9d640 1 --2- 172.21.15.172:0/2097277508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f281036e0 0x7f1f28198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f26d9d640 1 -- 172.21.15.172:0/2097277508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f28102d30 msgr2=0x7f1f28198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:35.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f26d9d640 1 --2- 172.21.15.172:0/2097277508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f28102d30 0x7f1f28198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f26d9d640 1 -- 172.21.15.172:0/2097277508 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2819d500 con 0x7f1f281082a0 2024-09-10T08:54:35.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f2659c640 1 --2- 172.21.15.172:0/2097277508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f28102d30 0x7f1f28198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:35.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f26d9d640 1 --2- 172.21.15.172:0/2097277508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f281082a0 0x7f1f2819ce20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000e940 tx=0x7f1f2000ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:35.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.295+0000 7f1f177fe640 1 -- 172.21.15.172:0/2097277508 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20004d20 con 0x7f1f281082a0 2024-09-10T08:54:35.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.296+0000 7f1f2cc99640 1 -- 172.21.15.172:0/2097277508 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f2819d7f0 con 0x7f1f281082a0 2024-09-10T08:54:35.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.296+0000 7f1f177fe640 1 -- 172.21.15.172:0/2097277508 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f20005020 con 0x7f1f281082a0 2024-09-10T08:54:35.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.296+0000 7f1f177fe640 1 -- 172.21.15.172:0/2097277508 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f200116d0 con 0x7f1f281082a0 2024-09-10T08:54:35.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.296+0000 7f1f2cc99640 1 -- 172.21.15.172:0/2097277508 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f281a50d0 con 0x7f1f281082a0 2024-09-10T08:54:35.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.298+0000 7f1f177fe640 1 -- 172.21.15.172:0/2097277508 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1f2000ca60 con 0x7f1f281082a0 2024-09-10T08:54:35.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.299+0000 7f1f2f072740 1 Processor -- start 2024-09-10T08:54:35.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.299+0000 7f1f2f072740 1 -- start start 2024-09-10T08:54:35.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.299+0000 7f1f2f072740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa953b40 0x55aaaa952090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.299+0000 7f1f2f072740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa9525d0 0x55aaaa952a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.299+0000 7f1f2f072740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa95d370 0x55aaaa95f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.299+0000 7f1f2f072740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55aaaa8b82f0 con 0x55aaaa9525d0 2024-09-10T08:54:35.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.299+0000 7f1f2f072740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aaaa85fcf0 con 0x55aaaa95d370 2024-09-10T08:54:35.321 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.299+0000 7f1f2f072740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55aaaa8b42b0 con 0x55aaaa953b40 2024-09-10T08:54:35.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.299+0000 7f1f2659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa9525d0 0x55aaaa952a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.299+0000 7f1f25d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa953b40 0x55aaaa952090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.299+0000 7f1f177fe640 1 --2- 172.21.15.172:0/2097277508 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1f100a2b80 0x7f1f100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.299+0000 7f1f2659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa9525d0 0x55aaaa952a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57560/0 (socket says 172.21.15.172:57560) 2024-09-10T08:54:35.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.299+0000 7f1f2659c640 1 -- 172.21.15.172:0/3289707596 learned_addr learned my addr 172.21.15.172:0/3289707596 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:35.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.299+0000 7f1f25d9b640 1 --2- 172.21.15.172:0/3289707596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa95d370 0x55aaaa95f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.300+0000 7f1f26d9d640 1 --2- 172.21.15.172:0/2097277508 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1f100a2b80 0x7f1f100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.300+0000 7f1f25d9b640 1 -- 172.21.15.172:0/3289707596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa95d370 msgr2=0x55aaaa95f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:35.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.300+0000 7f1f25d9b640 1 --2- 172.21.15.172:0/3289707596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa95d370 0x55aaaa95f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.300+0000 7f1f25d9b640 1 -- 172.21.15.172:0/3289707596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa9525d0 msgr2=0x55aaaa952a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:35.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.300+0000 7f1f25d9b640 1 --2- 172.21.15.172:0/3289707596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa9525d0 0x55aaaa952a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.322 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.300+0000 7f1f25d9b640 1 -- 172.21.15.172:0/3289707596 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaaa8443a0 con 0x55aaaa953b40 2024-09-10T08:54:35.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.300+0000 7f1f177fe640 1 -- 172.21.15.172:0/2097277508 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1f200cf3d0 con 0x7f1f281082a0 2024-09-10T08:54:35.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.300+0000 7f1f2659c640 1 --2- 172.21.15.172:0/3289707596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa9525d0 0x55aaaa952a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:35.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.300+0000 7f1f25d9b640 1 --2- 172.21.15.172:0/3289707596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa953b40 0x55aaaa952090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1f280feea0 tx=0x7f1f1c009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aac3ff8d286fc21c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:35.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.300+0000 7f1f14ff9640 1 -- 172.21.15.172:0/3289707596 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c017470 con 0x55aaaa953b40 2024-09-10T08:54:35.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.300+0000 7f1f14ff9640 1 -- 172.21.15.172:0/3289707596 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f1c01c070 con 0x55aaaa953b40 2024-09-10T08:54:35.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.300+0000 7f1f14ff9640 1 -- 172.21.15.172:0/3289707596 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c015900 con 0x55aaaa953b40 2024-09-10T08:54:35.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.300+0000 7f1f26d9d640 1 --2- 172.21.15.172:0/2097277508 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1f100a2b80 0x7f1f100a5040 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000c640 tx=0x7f1f20066550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:35.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.301+0000 7f1f2f072740 1 -- 172.21.15.172:0/3289707596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa953b40 msgr2=0x55aaaa952090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:35.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.301+0000 7f1f2f072740 1 --2- 172.21.15.172:0/3289707596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa953b40 0x55aaaa952090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1f280feea0 tx=0x7f1f1c009520 comp rx=0 tx=0).stop 2024-09-10T08:54:35.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.301+0000 7f1f2f072740 1 -- 172.21.15.172:0/3289707596 shutdown_connections 2024-09-10T08:54:35.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.301+0000 7f1f2f072740 1 --2- 172.21.15.172:0/3289707596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa95d370 0x55aaaa95f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.301+0000 7f1f2f072740 1 --2- 172.21.15.172:0/3289707596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa9525d0 0x55aaaa952a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.301+0000 7f1f2f072740 1 --2- 172.21.15.172:0/3289707596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa953b40 0x55aaaa952090 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.301+0000 7f1f2f072740 1 -- 172.21.15.172:0/3289707596 >> 172.21.15.172:0/3289707596 conn(0x55aaaa91b310 msgr2=0x55aaaa94ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:35.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.301+0000 7f1f2f072740 1 -- 172.21.15.172:0/3289707596 shutdown_connections 2024-09-10T08:54:35.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.301+0000 7f1f2f072740 1 -- 172.21.15.172:0/3289707596 wait complete. 2024-09-10T08:54:35.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.301+0000 7f1f2f072740 1 -- start start 2024-09-10T08:54:35.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.301+0000 7f1f2f072740 1 Processor -- start 2024-09-10T08:54:35.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f2f072740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa9525d0 0x55aaaa95cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f2f072740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa95a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f2f072740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa95d370 0x55aaaa95a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f2f072740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55aaaa960070 con 0x55aaaa953b40 2024-09-10T08:54:35.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f2f072740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aaaa873060 con 0x55aaaa9525d0 2024-09-10T08:54:35.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f2f072740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55aaaa9601f0 con 0x55aaaa95d370 2024-09-10T08:54:35.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f2f072740 10 client.0 _ll_register_callbacks cb 0x55aaaa5c68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:35.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f2659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa95a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f2659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa95a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57576/0 (socket says 172.21.15.172:57576) 2024-09-10T08:54:35.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f2659c640 1 -- 172.21.15.172:0/1961389266 learned_addr learned my addr 172.21.15.172:0/1961389266 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:35.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f25d9b640 1 --2- 172.21.15.172:0/1961389266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa9525d0 0x55aaaa95cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f25d9b640 1 --2- 172.21.15.172:0/1961389266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa95d370 0x55aaaa95a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f2659c640 1 -- 172.21.15.172:0/1961389266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa95d370 msgr2=0x55aaaa95a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:35.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f2659c640 1 --2- 172.21.15.172:0/1961389266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa95d370 0x55aaaa95a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f2659c640 1 -- 172.21.15.172:0/1961389266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa9525d0 msgr2=0x55aaaa95cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:35.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f2659c640 1 --2- 172.21.15.172:0/1961389266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa9525d0 0x55aaaa95cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f2659c640 1 -- 172.21.15.172:0/1961389266 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaaa6293c0 con 0x55aaaa953b40 2024-09-10T08:54:35.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f25d9b640 1 --2- 172.21.15.172:0/1961389266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa95d370 0x55aaaa95a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:54:35.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f2659c640 1 --2- 172.21.15.172:0/1961389266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa95a050 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009a50 tx=0x7f1f18031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0b984f764c1871 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:35.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.302+0000 7f1f0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:35.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.303+0000 7f1f0dffb640 1 -- 172.21.15.172:0/1961389266 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18035040 con 0x55aaaa953b40 2024-09-10T08:54:35.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.303+0000 7f1f0dffb640 1 -- 172.21.15.172:0/1961389266 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1f18005450 con 0x55aaaa953b40 2024-09-10T08:54:35.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.303+0000 7f1f2f072740 1 -- 172.21.15.172:0/1961389266 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f1c01c070 con 0x55aaaa953b40 2024-09-10T08:54:35.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.303+0000 7f1f0dffb640 1 -- 172.21.15.172:0/1961389266 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1f18005650 con 0x55aaaa953b40 2024-09-10T08:54:35.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.303+0000 7f1f2f072740 10 client.17883 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:35.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.303+0000 7f1f2f072740 1 -- 172.21.15.172:0/1961389266 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55aaaa96bb50 con 0x55aaaa953b40 2024-09-10T08:54:35.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.303+0000 7f1f0dffb640 1 -- 172.21.15.172:0/1961389266 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f1c01c070 con 0x55aaaa953b40 2024-09-10T08:54:35.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.303+0000 7f1f0dffb640 1 -- 172.21.15.172:0/1961389266 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f1f18048940 con 0x55aaaa953b40 2024-09-10T08:54:35.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.303+0000 7f1f2f072740 10 client.17883 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:35.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.303+0000 7f1f2f072740 10 client.17883 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:54:35.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.303+0000 7f1f2f072740 20 client.17883 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:35.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.303+0000 7f1f2f072740 1 --2- 172.21.15.172:0/1961389266 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55aaaa96d010 0x55aaaa96f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.304+0000 7f1f2f072740 4 client.17883 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:54:35.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.304+0000 7f1f2f072740 1 -- 172.21.15.172:0/1961389266 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaaa8c5860 con 0x55aaaa96d010 2024-09-10T08:54:35.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.304+0000 7f1f25d9b640 1 --2- 172.21.15.172:0/1961389266 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55aaaa96d010 0x55aaaa96f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.304+0000 7f1f0dffb640 10 client.17883 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:54:35.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.304+0000 7f1f25d9b640 1 --2- 172.21.15.172:0/1961389266 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55aaaa96d010 0x55aaaa96f400 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec732b0c48555a24 server_cookie=8db7377b385c5776 in_seq=0 out_seq=0 2024-09-10T08:54:35.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.305+0000 7f1f0dffb640 1 -- 172.21.15.172:0/1961389266 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aaaa8c5860 con 0x55aaaa96d010 2024-09-10T08:54:35.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.305+0000 7f1f0dffb640 10 client.17883 handle_command_reply: tid=0 2024-09-10T08:54:35.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.305+0000 7f1f2f072740 1 client.17883 shutdown 2024-09-10T08:54:35.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.305+0000 7f1f2f072740 1 -- 172.21.15.172:0/1961389266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 msgr2=0x55aaaa95a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:35.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.305+0000 7f1f2f072740 1 --2- 172.21.15.172:0/1961389266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa95a050 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009a50 tx=0x7f1f18031810 comp rx=0 tx=0).stop 2024-09-10T08:54:35.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.305+0000 7f1f2f072740 1 -- 172.21.15.172:0/1961389266 shutdown_connections 2024-09-10T08:54:35.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.305+0000 7f1f2f072740 1 --2- 172.21.15.172:0/1961389266 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55aaaa96d010 0x55aaaa96f400 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.328 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.305+0000 7f1f2f072740 1 --2- 172.21.15.172:0/1961389266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa95d370 0x55aaaa95a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.305+0000 7f1f2f072740 1 --2- 172.21.15.172:0/1961389266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa95a050 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.305+0000 7f1f0dffb640 0 client.17883 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:54:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.305+0000 7f1f2f072740 1 --2- 172.21.15.172:0/1961389266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa9525d0 0x55aaaa95cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.305+0000 7f1f2f072740 1 -- 172.21.15.172:0/1961389266 >> 172.21.15.172:0/1961389266 conn(0x55aaaa91b310 msgr2=0x55aaaa954960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.305+0000 7f1f2f072740 1 -- 172.21.15.172:0/1961389266 shutdown_connections 2024-09-10T08:54:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.305+0000 7f1f2f072740 1 -- 172.21.15.172:0/1961389266 wait complete. 2024-09-10T08:54:35.329 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.305+0000 7f1f2f072740 20 client.17883 trim_cache size 0 max 16384 2024-09-10T08:54:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.318+0000 7f1f2f072740 1 Processor -- start 2024-09-10T08:54:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.318+0000 7f1f2f072740 1 -- start start 2024-09-10T08:54:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f2f072740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa9525d0 0x55aaaa955120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f2f072740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa955660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.335 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f2f072740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa95d370 0x55aaaa974cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f2f072740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281b0df0 con 0x55aaaa953b40 2024-09-10T08:54:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f2f072740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281b2930 con 0x55aaaa95d370 2024-09-10T08:54:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f2f072740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281b14c0 con 0x55aaaa9525d0 2024-09-10T08:54:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f2659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa955660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f2659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa955660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57584/0 (socket says 172.21.15.172:57584) 2024-09-10T08:54:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f2659c640 1 -- 172.21.15.172:0/2091973749 learned_addr learned my addr 172.21.15.172:0/2091973749 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f25d9b640 1 --2- 172.21.15.172:0/2091973749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa95d370 0x55aaaa974cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f25d9b640 1 --2- 172.21.15.172:0/2091973749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa9525d0 0x55aaaa955120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f2659c640 1 -- 172.21.15.172:0/2091973749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa9525d0 msgr2=0x55aaaa955120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:35.336 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f2659c640 1 --2- 172.21.15.172:0/2091973749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa9525d0 0x55aaaa955120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f2659c640 1 -- 172.21.15.172:0/2091973749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa95d370 msgr2=0x55aaaa974cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f2659c640 1 --2- 172.21.15.172:0/2091973749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa95d370 0x55aaaa974cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f2659c640 1 -- 172.21.15.172:0/2091973749 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaaa96fa80 con 0x55aaaa953b40 2024-09-10T08:54:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f25d9b640 1 --2- 172.21.15.172:0/2091973749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa9525d0 0x55aaaa955120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:54:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.319+0000 7f1f2659c640 1 --2- 172.21.15.172:0/2091973749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa955660 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1f18005940 tx=0x7f1f18009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864c270fca051ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.320+0000 7f1f0ffff640 1 -- 172.21.15.172:0/2091973749 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18059020 con 0x55aaaa953b40 2024-09-10T08:54:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.320+0000 7f1f0ffff640 1 -- 172.21.15.172:0/2091973749 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f18005250 con 0x55aaaa953b40 2024-09-10T08:54:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.320+0000 7f1f0ffff640 1 -- 172.21.15.172:0/2091973749 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18035020 con 0x55aaaa953b40 2024-09-10T08:54:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.320+0000 7f1f2f072740 1 -- 172.21.15.172:0/2091973749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 msgr2=0x55aaaa955660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.320+0000 7f1f2f072740 1 --2- 172.21.15.172:0/2091973749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa955660 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1f18005940 tx=0x7f1f18009230 comp rx=0 tx=0).stop 2024-09-10T08:54:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.320+0000 7f1f2f072740 1 -- 172.21.15.172:0/2091973749 shutdown_connections 2024-09-10T08:54:35.337 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.320+0000 7f1f2f072740 1 --2- 172.21.15.172:0/2091973749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa95d370 0x55aaaa974cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.320+0000 7f1f2f072740 1 --2- 172.21.15.172:0/2091973749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa955660 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.320+0000 7f1f2f072740 1 --2- 172.21.15.172:0/2091973749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa9525d0 0x55aaaa955120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.321+0000 7f1f2f072740 1 -- 172.21.15.172:0/2091973749 >> 172.21.15.172:0/2091973749 conn(0x55aaaa91b310 msgr2=0x55aaaa95c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.321+0000 7f1f2f072740 1 -- 172.21.15.172:0/2091973749 shutdown_connections 2024-09-10T08:54:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.321+0000 7f1f2f072740 1 -- 172.21.15.172:0/2091973749 wait complete. 2024-09-10T08:54:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.321+0000 7f1f2f072740 1 -- start start 2024-09-10T08:54:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.321+0000 7f1f2f072740 1 Processor -- start 2024-09-10T08:54:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.321+0000 7f1f2f072740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa9525d0 0x55aaaa965480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.321+0000 7f1f2f072740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa9659c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.321+0000 7f1f2f072740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa95d370 0x55aaaa962450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.321+0000 7f1f2f072740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281b2260 con 0x55aaaa953b40 2024-09-10T08:54:35.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.321+0000 7f1f2f072740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281b1b90 con 0x55aaaa95d370 2024-09-10T08:54:35.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.321+0000 7f1f2f072740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1f281aedd0 con 0x55aaaa9525d0 2024-09-10T08:54:35.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.321+0000 7f1f2f072740 10 client.0 _ll_register_callbacks cb 0x7f1f18009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:35.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.322+0000 7f1f2659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa9659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.322+0000 7f1f2659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa9659c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57598/0 (socket says 172.21.15.172:57598) 2024-09-10T08:54:35.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.322+0000 7f1f2659c640 1 -- 172.21.15.172:0/2893646849 learned_addr learned my addr 172.21.15.172:0/2893646849 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:35.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.322+0000 7f1f25d9b640 1 --2- 172.21.15.172:0/2893646849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa9525d0 0x55aaaa965480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.322+0000 7f1f25d9b640 1 --2- 172.21.15.172:0/2893646849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa95d370 0x55aaaa962450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.322+0000 7f1f2659c640 1 -- 172.21.15.172:0/2893646849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa9525d0 msgr2=0x55aaaa965480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:35.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.322+0000 7f1f2659c640 1 --2- 172.21.15.172:0/2893646849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa9525d0 0x55aaaa965480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.322+0000 7f1f2659c640 1 -- 172.21.15.172:0/2893646849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa95d370 msgr2=0x55aaaa962450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:35.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.322+0000 7f1f2659c640 1 --2- 172.21.15.172:0/2893646849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa95d370 0x55aaaa962450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.322+0000 7f1f2659c640 1 -- 172.21.15.172:0/2893646849 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaaa966030 con 0x55aaaa953b40 2024-09-10T08:54:35.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.322+0000 7f1f2659c640 1 --2- 172.21.15.172:0/2893646849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa9659c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f1f18006010 tx=0x7f1f18000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6cc0d4001c70151 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:35.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.322+0000 7f1f0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:35.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.322+0000 7f1f0dffb640 1 -- 172.21.15.172:0/2893646849 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18035070 con 0x55aaaa953b40 2024-09-10T08:54:35.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.322+0000 7f1f0dffb640 1 -- 172.21.15.172:0/2893646849 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1f1803da00 con 0x55aaaa953b40 2024-09-10T08:54:35.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.322+0000 7f1f2f072740 1 -- 172.21.15.172:0/2893646849 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f18005250 con 0x55aaaa953b40 2024-09-10T08:54:35.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.323+0000 7f1f0dffb640 1 -- 172.21.15.172:0/2893646849 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1f18059030 con 0x55aaaa953b40 2024-09-10T08:54:35.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.323+0000 7f1f2f072740 10 client.17889 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:35.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.323+0000 7f1f2f072740 1 -- 172.21.15.172:0/2893646849 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55aaaa9d8190 con 0x55aaaa953b40 2024-09-10T08:54:35.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.323+0000 7f1f0dffb640 1 -- 172.21.15.172:0/2893646849 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f18005250 con 0x55aaaa953b40 2024-09-10T08:54:35.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.323+0000 7f1f0dffb640 1 -- 172.21.15.172:0/2893646849 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f1f1805e070 con 0x55aaaa953b40 2024-09-10T08:54:35.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.323+0000 7f1f2f072740 10 client.17889 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:35.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.323+0000 7f1f2f072740 10 client.17889 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:54:35.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.323+0000 7f1f2f072740 20 client.17889 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:35.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.323+0000 7f1f2f072740 1 --2- 172.21.15.172:0/2893646849 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55aaaa9d8670 0x55aaaa9daa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:35.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.323+0000 7f1f2f072740 4 client.17889 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:54:35.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.323+0000 7f1f2f072740 1 -- 172.21.15.172:0/2893646849 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aaaa8bfd70 con 0x55aaaa9d8670 2024-09-10T08:54:35.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.323+0000 7f1f25d9b640 1 --2- 172.21.15.172:0/2893646849 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55aaaa9d8670 0x55aaaa9daa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:35.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.324+0000 7f1f0dffb640 10 client.17889 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:54:35.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.324+0000 7f1f25d9b640 1 --2- 172.21.15.172:0/2893646849 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55aaaa9d8670 0x55aaaa9daa90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=daa0480337320c42 server_cookie=194d29f9cccc3f in_seq=0 out_seq=0 2024-09-10T08:54:35.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.324+0000 7f1f0dffb640 1 -- 172.21.15.172:0/2893646849 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aaaa8bfd70 con 0x55aaaa9d8670 2024-09-10T08:54:35.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.324+0000 7f1f0dffb640 10 client.17889 handle_command_reply: tid=0 2024-09-10T08:54:35.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.324+0000 7f1f2f072740 1 client.17889 shutdown 2024-09-10T08:54:35.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.324+0000 7f1f2f072740 1 -- 172.21.15.172:0/2893646849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 msgr2=0x55aaaa9659c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:35.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.324+0000 7f1f2f072740 1 --2- 172.21.15.172:0/2893646849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa9659c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f1f18006010 tx=0x7f1f18000bc0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.324+0000 7f1f2f072740 1 -- 172.21.15.172:0/2893646849 shutdown_connections 2024-09-10T08:54:35.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.324+0000 7f1f2f072740 1 --2- 172.21.15.172:0/2893646849 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55aaaa9d8670 0x55aaaa9daa90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.324+0000 7f1f2f072740 1 --2- 172.21.15.172:0/2893646849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aaaa95d370 0x55aaaa962450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.324+0000 7f1f2f072740 1 --2- 172.21.15.172:0/2893646849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaaa953b40 0x55aaaa9659c0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.324+0000 7f1f0dffb640 0 client.17889 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:54:35.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.324+0000 7f1f2f072740 1 --2- 172.21.15.172:0/2893646849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aaaa9525d0 0x55aaaa965480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.324+0000 7f1f2f072740 1 -- 172.21.15.172:0/2893646849 >> 172.21.15.172:0/2893646849 conn(0x55aaaa91b310 msgr2=0x55aaaa94d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:35.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.325+0000 7f1f2f072740 1 -- 172.21.15.172:0/2893646849 shutdown_connections 2024-09-10T08:54:35.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.325+0000 7f1f2f072740 1 -- 172.21.15.172:0/2893646849 wait complete. 2024-09-10T08:54:35.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.325+0000 7f1f2f072740 20 client.17889 trim_cache size 0 max 16384 2024-09-10T08:54:35.343 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:54:35.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.325+0000 7f1f2cc99640 1 -- 172.21.15.172:0/2097277508 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1f100a2b80 msgr2=0x7f1f100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:35.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.325+0000 7f1f2cc99640 1 --2- 172.21.15.172:0/2097277508 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1f100a2b80 0x7f1f100a5040 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000c640 tx=0x7f1f20066550 comp rx=0 tx=0).stop 2024-09-10T08:54:35.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.325+0000 7f1f2cc99640 1 -- 172.21.15.172:0/2097277508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f281082a0 msgr2=0x7f1f2819ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:35.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.325+0000 7f1f2cc99640 1 --2- 172.21.15.172:0/2097277508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f281082a0 0x7f1f2819ce20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000e940 tx=0x7f1f2000ed40 comp rx=0 tx=0).stop 2024-09-10T08:54:35.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.326+0000 7f1f2cc99640 1 -- 172.21.15.172:0/2097277508 shutdown_connections 2024-09-10T08:54:35.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.326+0000 7f1f2cc99640 1 --2- 172.21.15.172:0/2097277508 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1f100a2b80 0x7f1f100a5040 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.326+0000 7f1f2cc99640 1 --2- 172.21.15.172:0/2097277508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f281082a0 0x7f1f2819ce20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.326+0000 7f1f2cc99640 1 --2- 172.21.15.172:0/2097277508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f281036e0 0x7f1f28198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.326+0000 7f1f2cc99640 1 --2- 172.21.15.172:0/2097277508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f28102d30 0x7f1f28198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:35.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.326+0000 7f1f2cc99640 1 -- 172.21.15.172:0/2097277508 >> 172.21.15.172:0/2097277508 conn(0x7f1f280fc7d0 msgr2=0x7f1f28105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:35.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.326+0000 7f1f2cc99640 1 -- 172.21.15.172:0/2097277508 shutdown_connections 2024-09-10T08:54:35.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:35.326+0000 7f1f2cc99640 1 -- 172.21.15.172:0/2097277508 wait complete. 2024-09-10T08:54:35.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:35 smithi172 ceph-mon[30854]: pgmap v601: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-09-10T08:54:35.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:35 smithi174 ceph-mon[34068]: pgmap v601: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-09-10T08:54:35.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:35 smithi191 ceph-mon[35314]: pgmap v601: 129 pgs: 129 active+clean; 7.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-09-10T08:54:36.037 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:54:36.325 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:54:36.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.838+0000 7f10ec9b3640 1 -- 172.21.15.172:0/425151715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10e8102ef0 msgr2=0x7f10e810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:36.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.838+0000 7f10ec9b3640 1 --2- 172.21.15.172:0/425151715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10e8102ef0 0x7f10e810da40 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009ca0 tx=0x7f10e002a950 comp rx=0 tx=0).stop 2024-09-10T08:54:36.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.838+0000 7f10ec9b3640 1 -- 172.21.15.172:0/425151715 shutdown_connections 2024-09-10T08:54:36.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.838+0000 7f10ec9b3640 1 --2- 172.21.15.172:0/425151715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10e8102ef0 0x7f10e810da40 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.838+0000 7f10ec9b3640 1 --2- 172.21.15.172:0/425151715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10e8102570 0x7f10e81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.838+0000 7f10ec9b3640 1 --2- 172.21.15.172:0/425151715 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e8100e20 0x7f10e8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.838+0000 7f10ec9b3640 1 -- 172.21.15.172:0/425151715 >> 172.21.15.172:0/425151715 conn(0x7f10e80fc7d0 msgr2=0x7f10e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:36.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.838+0000 7f10ec9b3640 1 -- 172.21.15.172:0/425151715 shutdown_connections 2024-09-10T08:54:36.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.838+0000 7f10ec9b3640 1 -- 172.21.15.172:0/425151715 wait complete. 2024-09-10T08:54:36.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.839+0000 7f10ec9b3640 1 Processor -- start 2024-09-10T08:54:36.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.839+0000 7f10ec9b3640 1 -- start start 2024-09-10T08:54:36.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.839+0000 7f10ec9b3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10e8100e20 0x7f10e8197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.839+0000 7f10ec9b3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10e8102570 0x7f10e8197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.839+0000 7f10ec9b3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e8102ef0 0x7f10e819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.839+0000 7f10ec9b3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8110e30 con 0x7f10e8102570 2024-09-10T08:54:36.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.839+0000 7f10ec9b3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8110cb0 con 0x7f10e8100e20 2024-09-10T08:54:36.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.839+0000 7f10ec9b3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8110fb0 con 0x7f10e8102ef0 2024-09-10T08:54:36.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.839+0000 7f10e5d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10e8102570 0x7f10e8197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.840+0000 7f10e5d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10e8102570 0x7f10e8197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52950/0 (socket says 172.21.15.172:52950) 2024-09-10T08:54:36.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.840+0000 7f10e5d9b640 1 -- 172.21.15.172:0/1753032415 learned_addr learned my addr 172.21.15.172:0/1753032415 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:36.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.840+0000 7f10e659c640 1 --2- 172.21.15.172:0/1753032415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10e8100e20 0x7f10e8197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.840+0000 7f10e6d9d640 1 --2- 172.21.15.172:0/1753032415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e8102ef0 0x7f10e819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.840+0000 7f10e5d9b640 1 -- 172.21.15.172:0/1753032415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e8102ef0 msgr2=0x7f10e819bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:36.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.840+0000 7f10e5d9b640 1 --2- 172.21.15.172:0/1753032415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e8102ef0 0x7f10e819bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.840+0000 7f10e5d9b640 1 -- 172.21.15.172:0/1753032415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10e8100e20 msgr2=0x7f10e8197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:36.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.840+0000 7f10e5d9b640 1 --2- 172.21.15.172:0/1753032415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10e8100e20 0x7f10e8197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.840+0000 7f10e5d9b640 1 -- 172.21.15.172:0/1753032415 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e819c2c0 con 0x7f10e8102570 2024-09-10T08:54:36.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.840+0000 7f10e6d9d640 1 --2- 172.21.15.172:0/1753032415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e8102ef0 0x7f10e819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:54:36.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.840+0000 7f10e5d9b640 1 --2- 172.21.15.172:0/1753032415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10e8102570 0x7f10e8197850 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00c920 tx=0x7f10dc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:36.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.840+0000 7f10d77fe640 1 -- 172.21.15.172:0/1753032415 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc007c70 con 0x7f10e8102570 2024-09-10T08:54:36.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.840+0000 7f10d77fe640 1 -- 172.21.15.172:0/1753032415 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10dc007e10 con 0x7f10e8102570 2024-09-10T08:54:36.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.840+0000 7f10ec9b3640 1 -- 172.21.15.172:0/1753032415 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e819c5b0 con 0x7f10e8102570 2024-09-10T08:54:36.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.840+0000 7f10d77fe640 1 -- 172.21.15.172:0/1753032415 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc018820 con 0x7f10e8102570 2024-09-10T08:54:36.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.840+0000 7f10ec9b3640 1 -- 172.21.15.172:0/1753032415 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e8069f70 con 0x7f10e8102570 2024-09-10T08:54:36.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.843+0000 7f10d77fe640 1 -- 172.21.15.172:0/1753032415 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f10dc0189c0 con 0x7f10e8102570 2024-09-10T08:54:36.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.843+0000 7f10eed8c740 1 Processor -- start 2024-09-10T08:54:36.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.843+0000 7f10eed8c740 1 -- start start 2024-09-10T08:54:36.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.844+0000 7f10eed8c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b325913b40 0x55b325912090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.844+0000 7f10eed8c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3259125d0 0x55b325912a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.844+0000 7f10eed8c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b32591d370 0x55b32591f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.844+0000 7f10eed8c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b3258782f0 con 0x55b3259125d0 2024-09-10T08:54:36.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.844+0000 7f10eed8c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b32581fcf0 con 0x55b325913b40 2024-09-10T08:54:36.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.844+0000 7f10eed8c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b3258742b0 con 0x55b32591d370 2024-09-10T08:54:36.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.844+0000 7f10d77fe640 1 --2- 172.21.15.172:0/1753032415 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f10d00a2b80 0x7f10d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.844+0000 7f10e659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3259125d0 0x55b325912a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.844+0000 7f10e659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3259125d0 0x55b325912a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52956/0 (socket says 172.21.15.172:52956) 2024-09-10T08:54:36.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.844+0000 7f10e659c640 1 -- 172.21.15.172:0/4076797325 learned_addr learned my addr 172.21.15.172:0/4076797325 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:36.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.844+0000 7f10e5d9b640 1 --2- 172.21.15.172:0/4076797325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b32591d370 0x55b32591f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.844+0000 7f10e6d9d640 1 --2- 172.21.15.172:0/4076797325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b325913b40 0x55b325912090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.844+0000 7f10d77fe640 1 -- 172.21.15.172:0/1753032415 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f10dc0cf4a0 con 0x7f10e8102570 2024-09-10T08:54:36.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.844+0000 7f10e5d9b640 1 -- 172.21.15.172:0/4076797325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b325913b40 msgr2=0x55b325912090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:36.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.844+0000 7f10e5d9b640 1 --2- 172.21.15.172:0/4076797325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b325913b40 0x55b325912090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.844+0000 7f10e6d9d640 1 --2- 172.21.15.172:0/1753032415 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f10d00a2b80 0x7f10d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.845+0000 7f10e5d9b640 1 -- 172.21.15.172:0/4076797325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3259125d0 msgr2=0x55b325912a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:36.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.845+0000 7f10e5d9b640 1 --2- 172.21.15.172:0/4076797325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3259125d0 0x55b325912a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.845+0000 7f10e5d9b640 1 -- 172.21.15.172:0/4076797325 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3258043a0 con 0x55b32591d370 2024-09-10T08:54:36.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.845+0000 7f10e659c640 1 --2- 172.21.15.172:0/4076797325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3259125d0 0x55b325912a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:36.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.845+0000 7f10e5d9b640 1 --2- 172.21.15.172:0/4076797325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b32591d370 0x55b32591f760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0ccc90 tx=0x7f10dc0c6530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee685f3852e93684 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:36.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.845+0000 7f10e6d9d640 1 --2- 172.21.15.172:0/1753032415 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f10d00a2b80 0x7f10d00a5040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009ca0 tx=0x7f10e00095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:36.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.845+0000 7f10d4ff9640 1 -- 172.21.15.172:0/4076797325 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc018cb0 con 0x55b32591d370 2024-09-10T08:54:36.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.845+0000 7f10d4ff9640 1 -- 172.21.15.172:0/4076797325 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10dc00fb00 con 0x55b32591d370 2024-09-10T08:54:36.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.845+0000 7f10d4ff9640 1 -- 172.21.15.172:0/4076797325 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc0669e0 con 0x55b32591d370 2024-09-10T08:54:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.846+0000 7f10eed8c740 1 -- 172.21.15.172:0/4076797325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b32591d370 msgr2=0x55b32591f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.846+0000 7f10eed8c740 1 --2- 172.21.15.172:0/4076797325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b32591d370 0x55b32591f760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0ccc90 tx=0x7f10dc0c6530 comp rx=0 tx=0).stop 2024-09-10T08:54:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.846+0000 7f10eed8c740 1 -- 172.21.15.172:0/4076797325 shutdown_connections 2024-09-10T08:54:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.846+0000 7f10eed8c740 1 --2- 172.21.15.172:0/4076797325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b32591d370 0x55b32591f760 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.846+0000 7f10eed8c740 1 --2- 172.21.15.172:0/4076797325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3259125d0 0x55b325912a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.846+0000 7f10eed8c740 1 --2- 172.21.15.172:0/4076797325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b325913b40 0x55b325912090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.846+0000 7f10eed8c740 1 -- 172.21.15.172:0/4076797325 >> 172.21.15.172:0/4076797325 conn(0x55b3258db310 msgr2=0x55b32590ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.847+0000 7f10eed8c740 1 -- 172.21.15.172:0/4076797325 shutdown_connections 2024-09-10T08:54:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.847+0000 7f10eed8c740 1 -- 172.21.15.172:0/4076797325 wait complete. 2024-09-10T08:54:36.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.847+0000 7f10eed8c740 1 -- start start 2024-09-10T08:54:36.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.847+0000 7f10eed8c740 1 Processor -- start 2024-09-10T08:54:36.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10eed8c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3259125d0 0x55b32591cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10eed8c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b325913b40 0x55b32591a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10eed8c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b32591d370 0x55b32591a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10eed8c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b325920070 con 0x55b3259125d0 2024-09-10T08:54:36.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10eed8c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b325833060 con 0x55b32591d370 2024-09-10T08:54:36.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10eed8c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b3259201f0 con 0x55b325913b40 2024-09-10T08:54:36.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10eed8c740 10 client.0 _ll_register_callbacks cb 0x55b3255868c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:36.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10e659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3259125d0 0x55b32591cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10e659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3259125d0 0x55b32591cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52968/0 (socket says 172.21.15.172:52968) 2024-09-10T08:54:36.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10e659c640 1 -- 172.21.15.172:0/409717199 learned_addr learned my addr 172.21.15.172:0/409717199 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:36.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10e5d9b640 1 --2- 172.21.15.172:0/409717199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b325913b40 0x55b32591a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10e6d9d640 1 --2- 172.21.15.172:0/409717199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b32591d370 0x55b32591a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10e659c640 1 -- 172.21.15.172:0/409717199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b325913b40 msgr2=0x55b32591a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:36.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10e659c640 1 --2- 172.21.15.172:0/409717199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b325913b40 0x55b32591a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10e659c640 1 -- 172.21.15.172:0/409717199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b32591d370 msgr2=0x55b32591a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:36.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10e659c640 1 --2- 172.21.15.172:0/409717199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b32591d370 0x55b32591a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10e659c640 1 -- 172.21.15.172:0/409717199 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3255e93c0 con 0x55b3259125d0 2024-09-10T08:54:36.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10e659c640 1 --2- 172.21.15.172:0/409717199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3259125d0 0x55b32591cde0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f10d800bc80 tx=0x7f10d80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37f037fc184fcd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:36.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.848+0000 7f10cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:36.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.849+0000 7f10cdffb640 1 -- 172.21.15.172:0/409717199 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8011440 con 0x55b3259125d0 2024-09-10T08:54:36.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.849+0000 7f10cdffb640 1 -- 172.21.15.172:0/409717199 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f10d80115e0 con 0x55b3259125d0 2024-09-10T08:54:36.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.849+0000 7f10eed8c740 1 -- 172.21.15.172:0/409717199 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b325885860 con 0x55b3259125d0 2024-09-10T08:54:36.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.849+0000 7f10cdffb640 1 -- 172.21.15.172:0/409717199 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f10d80117e0 con 0x55b3259125d0 2024-09-10T08:54:36.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.849+0000 7f10eed8c740 10 client.17901 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:36.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.849+0000 7f10eed8c740 1 -- 172.21.15.172:0/409717199 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b32592bc00 con 0x55b3259125d0 2024-09-10T08:54:36.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.849+0000 7f10cdffb640 1 -- 172.21.15.172:0/409717199 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b325885860 con 0x55b3259125d0 2024-09-10T08:54:36.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.849+0000 7f10cdffb640 1 -- 172.21.15.172:0/409717199 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f10d801ba70 con 0x55b3259125d0 2024-09-10T08:54:36.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.849+0000 7f10eed8c740 10 client.17901 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:36.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.849+0000 7f10eed8c740 10 client.17901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:54:36.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.849+0000 7f10eed8c740 20 client.17901 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:36.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.849+0000 7f10eed8c740 1 --2- 172.21.15.172:0/409717199 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b32592d0c0 0x55b32592f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.850+0000 7f10eed8c740 4 client.17901 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:54:36.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.850+0000 7f10eed8c740 1 -- 172.21.15.172:0/409717199 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b32587fd70 con 0x55b32592d0c0 2024-09-10T08:54:36.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.850+0000 7f10e5d9b640 1 --2- 172.21.15.172:0/409717199 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b32592d0c0 0x55b32592f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.850+0000 7f10e5d9b640 1 --2- 172.21.15.172:0/409717199 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b32592d0c0 0x55b32592f4b0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a4ccda3e770c739 server_cookie=f7dfa89f1a22da in_seq=0 out_seq=0 2024-09-10T08:54:36.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.850+0000 7f10cdffb640 10 client.17901 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.851+0000 7f10cdffb640 1 -- 172.21.15.172:0/409717199 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b32587fd70 con 0x55b32592d0c0 2024-09-10T08:54:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.851+0000 7f10cdffb640 10 client.17901 handle_command_reply: tid=0 2024-09-10T08:54:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.851+0000 7f10eed8c740 1 client.17901 shutdown 2024-09-10T08:54:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.851+0000 7f10eed8c740 1 -- 172.21.15.172:0/409717199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3259125d0 msgr2=0x55b32591cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.851+0000 7f10eed8c740 1 --2- 172.21.15.172:0/409717199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3259125d0 0x55b32591cde0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f10d800bc80 tx=0x7f10d80039c0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.851+0000 7f10eed8c740 1 -- 172.21.15.172:0/409717199 shutdown_connections 2024-09-10T08:54:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.851+0000 7f10eed8c740 1 --2- 172.21.15.172:0/409717199 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b32592d0c0 0x55b32592f4b0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.851+0000 7f10eed8c740 1 --2- 172.21.15.172:0/409717199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b32591d370 0x55b32591a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.851+0000 7f10eed8c740 1 --2- 172.21.15.172:0/409717199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b325913b40 0x55b32591a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.851+0000 7f10eed8c740 1 --2- 172.21.15.172:0/409717199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b3259125d0 0x55b32591cde0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.851+0000 7f10eed8c740 1 -- 172.21.15.172:0/409717199 >> 172.21.15.172:0/409717199 conn(0x55b3258db310 msgr2=0x55b325914960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.851+0000 7f10cdffb640 0 client.17901 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.851+0000 7f10eed8c740 1 -- 172.21.15.172:0/409717199 shutdown_connections 2024-09-10T08:54:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.851+0000 7f10eed8c740 1 -- 172.21.15.172:0/409717199 wait complete. 2024-09-10T08:54:36.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.851+0000 7f10eed8c740 20 client.17901 trim_cache size 0 max 16384 2024-09-10T08:54:36.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.867+0000 7f10eed8c740 1 Processor -- start 2024-09-10T08:54:36.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.867+0000 7f10eed8c740 1 -- start start 2024-09-10T08:54:36.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.867+0000 7f10eed8c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b3259125d0 0x55b325929310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.867+0000 7f10eed8c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b325913b40 0x55b325915120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.867+0000 7f10eed8c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b32591d370 0x55b325915660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.867+0000 7f10eed8c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81b0c90 con 0x55b325913b40 2024-09-10T08:54:36.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.867+0000 7f10eed8c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81b27d0 con 0x55b32591d370 2024-09-10T08:54:36.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.867+0000 7f10eed8c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81b1360 con 0x55b3259125d0 2024-09-10T08:54:36.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.867+0000 7f10e5d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b325913b40 0x55b325915120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.867+0000 7f10e5d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b325913b40 0x55b325915120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52982/0 (socket says 172.21.15.172:52982) 2024-09-10T08:54:36.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.867+0000 7f10e5d9b640 1 -- 172.21.15.172:0/1940916366 learned_addr learned my addr 172.21.15.172:0/1940916366 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:36.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.867+0000 7f10e659c640 1 --2- 172.21.15.172:0/1940916366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b3259125d0 0x55b325929310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.867+0000 7f10e6d9d640 1 --2- 172.21.15.172:0/1940916366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b32591d370 0x55b325915660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.868+0000 7f10e659c640 1 -- 172.21.15.172:0/1940916366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b32591d370 msgr2=0x55b325915660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:36.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.868+0000 7f10e659c640 1 --2- 172.21.15.172:0/1940916366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b32591d370 0x55b325915660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.868+0000 7f10e659c640 1 -- 172.21.15.172:0/1940916366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b325913b40 msgr2=0x55b325915120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:36.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.868+0000 7f10e659c640 1 --2- 172.21.15.172:0/1940916366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b325913b40 0x55b325915120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.868+0000 7f10e659c640 1 -- 172.21.15.172:0/1940916366 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b325929850 con 0x55b3259125d0 2024-09-10T08:54:36.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.868+0000 7f10e659c640 1 --2- 172.21.15.172:0/1940916366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b3259125d0 0x55b325929310 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f10d801be90 tx=0x7f10d8003f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3727f977e6374090 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:36.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.868+0000 7f10cffff640 1 -- 172.21.15.172:0/1940916366 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8011440 con 0x55b3259125d0 2024-09-10T08:54:36.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.868+0000 7f10cffff640 1 -- 172.21.15.172:0/1940916366 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10d8011e30 con 0x55b3259125d0 2024-09-10T08:54:36.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.868+0000 7f10cffff640 1 -- 172.21.15.172:0/1940916366 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8005470 con 0x55b3259125d0 2024-09-10T08:54:36.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.869+0000 7f10eed8c740 1 -- 172.21.15.172:0/1940916366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b3259125d0 msgr2=0x55b325929310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:36.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.869+0000 7f10eed8c740 1 --2- 172.21.15.172:0/1940916366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b3259125d0 0x55b325929310 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f10d801be90 tx=0x7f10d8003f00 comp rx=0 tx=0).stop 2024-09-10T08:54:36.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.869+0000 7f10eed8c740 1 -- 172.21.15.172:0/1940916366 shutdown_connections 2024-09-10T08:54:36.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.869+0000 7f10eed8c740 1 --2- 172.21.15.172:0/1940916366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b32591d370 0x55b325915660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.869+0000 7f10eed8c740 1 --2- 172.21.15.172:0/1940916366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b325913b40 0x55b325915120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.869+0000 7f10eed8c740 1 --2- 172.21.15.172:0/1940916366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b3259125d0 0x55b325929310 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.869+0000 7f10eed8c740 1 -- 172.21.15.172:0/1940916366 >> 172.21.15.172:0/1940916366 conn(0x55b3258db310 msgr2=0x55b325919180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:36.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.869+0000 7f10eed8c740 1 -- 172.21.15.172:0/1940916366 shutdown_connections 2024-09-10T08:54:36.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.869+0000 7f10eed8c740 1 -- 172.21.15.172:0/1940916366 wait complete. 2024-09-10T08:54:36.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.869+0000 7f10eed8c740 1 -- start start 2024-09-10T08:54:36.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.869+0000 7f10eed8c740 1 Processor -- start 2024-09-10T08:54:36.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10eed8c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b3259125d0 0x55b325920c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10eed8c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b325913b40 0x55b325921150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10eed8c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b32591d370 0x55b32598fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10eed8c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81b2100 con 0x55b32591d370 2024-09-10T08:54:36.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10eed8c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81b1a30 con 0x55b325913b40 2024-09-10T08:54:36.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10eed8c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81aec70 con 0x55b3259125d0 2024-09-10T08:54:36.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10eed8c740 10 client.0 _ll_register_callbacks cb 0x7f10d8004270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:36.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10e6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b32591d370 0x55b32598fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10e6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b32591d370 0x55b32598fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52988/0 (socket says 172.21.15.172:52988) 2024-09-10T08:54:36.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10e6d9d640 1 -- 172.21.15.172:0/1939490000 learned_addr learned my addr 172.21.15.172:0/1939490000 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:36.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10e659c640 1 --2- 172.21.15.172:0/1939490000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b3259125d0 0x55b325920c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10e5d9b640 1 --2- 172.21.15.172:0/1939490000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b325913b40 0x55b325921150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10e6d9d640 1 -- 172.21.15.172:0/1939490000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b3259125d0 msgr2=0x55b325920c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:36.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10e6d9d640 1 --2- 172.21.15.172:0/1939490000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b3259125d0 0x55b325920c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10e6d9d640 1 -- 172.21.15.172:0/1939490000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b325913b40 msgr2=0x55b325921150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:36.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10e6d9d640 1 --2- 172.21.15.172:0/1939490000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b325913b40 0x55b325921150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10e6d9d640 1 -- 172.21.15.172:0/1939490000 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b325990350 con 0x55b32591d370 2024-09-10T08:54:36.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10e659c640 1 --2- 172.21.15.172:0/1939490000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b3259125d0 0x55b325920c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:36.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10e6d9d640 1 --2- 172.21.15.172:0/1939490000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b32591d370 0x55b32598fd10 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f10e002fd00 tx=0x7f10e002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704eabd9b8afb4d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:36.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.870+0000 7f10cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:36.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.871+0000 7f10cdffb640 1 -- 172.21.15.172:0/1939490000 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e0057060 con 0x55b32591d370 2024-09-10T08:54:36.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.871+0000 7f10cdffb640 1 -- 172.21.15.172:0/1939490000 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f10e00043d0 con 0x55b32591d370 2024-09-10T08:54:36.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.871+0000 7f10eed8c740 1 -- 172.21.15.172:0/1939490000 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d8011e30 con 0x55b32591d370 2024-09-10T08:54:36.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.871+0000 7f10cdffb640 1 -- 172.21.15.172:0/1939490000 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f10e0042070 con 0x55b32591d370 2024-09-10T08:54:36.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.871+0000 7f10eed8c740 10 client.17907 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:36.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.871+0000 7f10eed8c740 1 -- 172.21.15.172:0/1939490000 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b3259976e0 con 0x55b32591d370 2024-09-10T08:54:36.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.871+0000 7f10cdffb640 1 -- 172.21.15.172:0/1939490000 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10d8011e30 con 0x55b32591d370 2024-09-10T08:54:36.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.871+0000 7f10cdffb640 1 -- 172.21.15.172:0/1939490000 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f10e003b440 con 0x55b32591d370 2024-09-10T08:54:36.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.871+0000 7f10eed8c740 10 client.17907 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:36.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.871+0000 7f10eed8c740 10 client.17907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:54:36.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.871+0000 7f10eed8c740 20 client.17907 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:36.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.871+0000 7f10eed8c740 1 --2- 172.21.15.172:0/1939490000 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b325997e90 0x55b32599a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:36.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.871+0000 7f10eed8c740 4 client.17907 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:54:36.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.871+0000 7f10eed8c740 1 -- 172.21.15.172:0/1939490000 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3257fc170 con 0x55b325997e90 2024-09-10T08:54:36.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.872+0000 7f10e659c640 1 --2- 172.21.15.172:0/1939490000 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b325997e90 0x55b32599a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:36.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.872+0000 7f10e659c640 1 --2- 172.21.15.172:0/1939490000 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b325997e90 0x55b32599a2b0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4295a33e51cb34d1 server_cookie=5eec6dad6c78e49e in_seq=0 out_seq=0 2024-09-10T08:54:36.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.872+0000 7f10cdffb640 10 client.17907 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:36.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.872+0000 7f10cdffb640 1 -- 172.21.15.172:0/1939490000 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3257fc170 con 0x55b325997e90 2024-09-10T08:54:36.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.872+0000 7f10cdffb640 10 client.17907 handle_command_reply: tid=0 2024-09-10T08:54:36.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.872+0000 7f10eed8c740 1 client.17907 shutdown 2024-09-10T08:54:36.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.872+0000 7f10eed8c740 1 -- 172.21.15.172:0/1939490000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b32591d370 msgr2=0x55b32598fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:36.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.872+0000 7f10eed8c740 1 --2- 172.21.15.172:0/1939490000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b32591d370 0x55b32598fd10 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f10e002fd00 tx=0x7f10e002a950 comp rx=0 tx=0).stop 2024-09-10T08:54:36.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10eed8c740 1 -- 172.21.15.172:0/1939490000 shutdown_connections 2024-09-10T08:54:36.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10eed8c740 1 --2- 172.21.15.172:0/1939490000 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b325997e90 0x55b32599a2b0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10eed8c740 1 --2- 172.21.15.172:0/1939490000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b32591d370 0x55b32598fd10 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10eed8c740 1 --2- 172.21.15.172:0/1939490000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b325913b40 0x55b325921150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10cdffb640 0 client.17907 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:36.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10eed8c740 1 --2- 172.21.15.172:0/1939490000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b3259125d0 0x55b325920c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10eed8c740 1 -- 172.21.15.172:0/1939490000 >> 172.21.15.172:0/1939490000 conn(0x55b3258db310 msgr2=0x55b32590d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:36.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10eed8c740 1 -- 172.21.15.172:0/1939490000 shutdown_connections 2024-09-10T08:54:36.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10eed8c740 1 -- 172.21.15.172:0/1939490000 wait complete. 2024-09-10T08:54:36.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10eed8c740 20 client.17907 trim_cache size 0 max 16384 2024-09-10T08:54:36.879 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:54:36.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10ec9b3640 1 -- 172.21.15.172:0/1753032415 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f10d00a2b80 msgr2=0x7f10d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:36.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10ec9b3640 1 --2- 172.21.15.172:0/1753032415 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f10d00a2b80 0x7f10d00a5040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009ca0 tx=0x7f10e00095c0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10ec9b3640 1 -- 172.21.15.172:0/1753032415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10e8102570 msgr2=0x7f10e8197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:36.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10ec9b3640 1 --2- 172.21.15.172:0/1753032415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10e8102570 0x7f10e8197850 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00c920 tx=0x7f10dc00cde0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10ec9b3640 1 -- 172.21.15.172:0/1753032415 shutdown_connections 2024-09-10T08:54:36.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10ec9b3640 1 --2- 172.21.15.172:0/1753032415 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f10d00a2b80 0x7f10d00a5040 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10ec9b3640 1 --2- 172.21.15.172:0/1753032415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10e8102ef0 0x7f10e819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10ec9b3640 1 --2- 172.21.15.172:0/1753032415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10e8102570 0x7f10e8197850 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10ec9b3640 1 --2- 172.21.15.172:0/1753032415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10e8100e20 0x7f10e8197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:36.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.873+0000 7f10ec9b3640 1 -- 172.21.15.172:0/1753032415 >> 172.21.15.172:0/1753032415 conn(0x7f10e80fc7d0 msgr2=0x7f10e81068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:36.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.874+0000 7f10ec9b3640 1 -- 172.21.15.172:0/1753032415 shutdown_connections 2024-09-10T08:54:36.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:36.874+0000 7f10ec9b3640 1 -- 172.21.15.172:0/1753032415 wait complete. 2024-09-10T08:54:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.149+0000 7f511dfe4640 1 -- 172.21.15.172:0/417176568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5118104ef0 msgr2=0x7f51181072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.149+0000 7f511dfe4640 1 --2- 172.21.15.172:0/417176568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5118104ef0 0x7f51181072e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f510c009900 tx=0x7f510c02a410 comp rx=0 tx=0).stop 2024-09-10T08:54:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.149+0000 7f511dfe4640 1 -- 172.21.15.172:0/417176568 shutdown_connections 2024-09-10T08:54:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.149+0000 7f511dfe4640 1 --2- 172.21.15.172:0/417176568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5118107820 0x7f5118109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.149+0000 7f511dfe4640 1 --2- 172.21.15.172:0/417176568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5118104ef0 0x7f51181072e0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.149+0000 7f511dfe4640 1 --2- 172.21.15.172:0/417176568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51181025c0 0x7f51181049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.149+0000 7f511dfe4640 1 -- 172.21.15.172:0/417176568 >> 172.21.15.172:0/417176568 conn(0x7f51180fc1e0 msgr2=0x7f51180fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.149+0000 7f511dfe4640 1 -- 172.21.15.172:0/417176568 shutdown_connections 2024-09-10T08:54:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.149+0000 7f511dfe4640 1 -- 172.21.15.172:0/417176568 wait complete. 2024-09-10T08:54:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.150+0000 7f511dfe4640 1 Processor -- start 2024-09-10T08:54:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.150+0000 7f511dfe4640 1 -- start start 2024-09-10T08:54:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.150+0000 7f511dfe4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51181025c0 0x7f5118197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.150+0000 7f511dfe4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5118104ef0 0x7f51181982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.150+0000 7f511dfe4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5118107820 0x7f511819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.150+0000 7f511dfe4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f511810d6a0 con 0x7f51181025c0 2024-09-10T08:54:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.150+0000 7f511dfe4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f511810d520 con 0x7f5118104ef0 2024-09-10T08:54:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.150+0000 7f511dfe4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f511810d820 con 0x7f5118107820 2024-09-10T08:54:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.150+0000 7f51177fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51181025c0 0x7f5118197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.150+0000 7f51177fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51181025c0 0x7f5118197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53018/0 (socket says 172.21.15.172:53018) 2024-09-10T08:54:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.150+0000 7f51177fe640 1 -- 172.21.15.172:0/3360055018 learned_addr learned my addr 172.21.15.172:0/3360055018 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.150+0000 7f5117fff640 1 --2- 172.21.15.172:0/3360055018 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5118107820 0x7f511819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.151+0000 7f5116ffd640 1 --2- 172.21.15.172:0/3360055018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5118104ef0 0x7f51181982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.151+0000 7f51177fe640 1 -- 172.21.15.172:0/3360055018 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5118107820 msgr2=0x7f511819f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.151+0000 7f51177fe640 1 --2- 172.21.15.172:0/3360055018 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5118107820 0x7f511819f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.151+0000 7f51177fe640 1 -- 172.21.15.172:0/3360055018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5118104ef0 msgr2=0x7f51181982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.151+0000 7f51177fe640 1 --2- 172.21.15.172:0/3360055018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5118104ef0 0x7f51181982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.151+0000 7f51177fe640 1 -- 172.21.15.172:0/3360055018 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511819f9b0 con 0x7f51181025c0 2024-09-10T08:54:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.151+0000 7f5117fff640 1 --2- 172.21.15.172:0/3360055018 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5118107820 0x7f511819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:54:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.151+0000 7f5116ffd640 1 --2- 172.21.15.172:0/3360055018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5118104ef0 0x7f51181982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:54:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.151+0000 7f51177fe640 1 --2- 172.21.15.172:0/3360055018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51181025c0 0x7f5118197da0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f510800d770 tx=0x7f510800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.151+0000 7f5114ff9640 1 -- 172.21.15.172:0/3360055018 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108017070 con 0x7f51181025c0 2024-09-10T08:54:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.151+0000 7f511dfe4640 1 -- 172.21.15.172:0/3360055018 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511819fca0 con 0x7f51181025c0 2024-09-10T08:54:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.151+0000 7f5114ff9640 1 -- 172.21.15.172:0/3360055018 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51080054d0 con 0x7f51181025c0 2024-09-10T08:54:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.151+0000 7f5114ff9640 1 -- 172.21.15.172:0/3360055018 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108004cf0 con 0x7f51181025c0 2024-09-10T08:54:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.151+0000 7f511dfe4640 1 -- 172.21.15.172:0/3360055018 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51181a01a0 con 0x7f51181025c0 2024-09-10T08:54:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.154+0000 7f5114ff9640 1 -- 172.21.15.172:0/3360055018 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5108004e90 con 0x7f51181025c0 2024-09-10T08:54:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.154+0000 7f5120e55740 1 Processor -- start 2024-09-10T08:54:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.154+0000 7f5120e55740 1 -- start start 2024-09-10T08:54:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5120e55740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557063534db0 0x5570635351b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5120e55740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570635356f0 0x55706353a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5120e55740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55706353a780 0x55706353cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5120e55740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5570634942f0 con 0x55706353a780 2024-09-10T08:54:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5120e55740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55706343c270 con 0x557063534db0 2024-09-10T08:54:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5120e55740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5570634902b0 con 0x5570635356f0 2024-09-10T08:54:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5116ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55706353a780 0x55706353cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f51177fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570635356f0 0x55706353a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5116ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55706353a780 0x55706353cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53026/0 (socket says 172.21.15.172:53026) 2024-09-10T08:54:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5116ffd640 1 -- 172.21.15.172:0/4073111747 learned_addr learned my addr 172.21.15.172:0/4073111747 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5117fff640 1 --2- 172.21.15.172:0/4073111747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557063534db0 0x5570635351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5114ff9640 1 --2- 172.21.15.172:0/3360055018 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51000a2c30 0x7f51000a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5117fff640 1 --2- 172.21.15.172:0/3360055018 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51000a2c30 0x7f51000a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5116ffd640 1 -- 172.21.15.172:0/4073111747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570635356f0 msgr2=0x55706353a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5116ffd640 1 --2- 172.21.15.172:0/4073111747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570635356f0 0x55706353a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5116ffd640 1 -- 172.21.15.172:0/4073111747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557063534db0 msgr2=0x5570635351b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5116ffd640 1 --2- 172.21.15.172:0/4073111747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557063534db0 0x5570635351b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5116ffd640 1 -- 172.21.15.172:0/4073111747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706343b340 con 0x55706353a780 2024-09-10T08:54:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5114ff9640 1 -- 172.21.15.172:0/3360055018 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f51080cef50 con 0x7f51181025c0 2024-09-10T08:54:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.155+0000 7f5116ffd640 1 --2- 172.21.15.172:0/4073111747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55706353a780 0x55706353cc40 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f510c006010 tx=0x7f510c032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d7276ef7619333 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:38.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.156+0000 7f5117fff640 1 --2- 172.21.15.172:0/3360055018 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51000a2c30 0x7f51000a50f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f511806a980 tx=0x7f5110004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.156+0000 7f50fdffb640 1 -- 172.21.15.172:0/4073111747 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c03f070 con 0x55706353a780 2024-09-10T08:54:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.156+0000 7f50fdffb640 1 -- 172.21.15.172:0/4073111747 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f510c031780 con 0x55706353a780 2024-09-10T08:54:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.156+0000 7f50fdffb640 1 -- 172.21.15.172:0/4073111747 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c035070 con 0x55706353a780 2024-09-10T08:54:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.156+0000 7f5120e55740 1 -- 172.21.15.172:0/4073111747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55706353a780 msgr2=0x55706353cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.156+0000 7f5120e55740 1 --2- 172.21.15.172:0/4073111747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55706353a780 0x55706353cc40 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f510c006010 tx=0x7f510c032cc0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.166 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.157+0000 7f5120e55740 1 -- 172.21.15.172:0/4073111747 shutdown_connections 2024-09-10T08:54:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.157+0000 7f5120e55740 1 --2- 172.21.15.172:0/4073111747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55706353a780 0x55706353cc40 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.157+0000 7f5120e55740 1 --2- 172.21.15.172:0/4073111747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570635356f0 0x55706353a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.157+0000 7f5120e55740 1 --2- 172.21.15.172:0/4073111747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557063534db0 0x5570635351b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.157+0000 7f5120e55740 1 -- 172.21.15.172:0/4073111747 >> 172.21.15.172:0/4073111747 conn(0x55706352c010 msgr2=0x55706352c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.157+0000 7f5120e55740 1 -- 172.21.15.172:0/4073111747 shutdown_connections 2024-09-10T08:54:38.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.157+0000 7f5120e55740 1 -- 172.21.15.172:0/4073111747 wait complete. 2024-09-10T08:54:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.157+0000 7f5120e55740 1 -- start start 2024-09-10T08:54:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.157+0000 7f5120e55740 1 Processor -- start 2024-09-10T08:54:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.157+0000 7f5120e55740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557063534db0 0x55706353eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.157+0000 7f5120e55740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570635356f0 0x55706353f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.157+0000 7f5120e55740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55706353a780 0x557063546440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.157+0000 7f5120e55740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55706353d6f0 con 0x557063534db0 2024-09-10T08:54:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.157+0000 7f5120e55740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55706341d290 con 0x55706353a780 2024-09-10T08:54:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f5120e55740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55706353d870 con 0x5570635356f0 2024-09-10T08:54:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f5120e55740 10 client.0 _ll_register_callbacks cb 0x55706344e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f51177fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557063534db0 0x55706353eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f51177fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557063534db0 0x55706353eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53030/0 (socket says 172.21.15.172:53030) 2024-09-10T08:54:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f51177fe640 1 -- 172.21.15.172:0/3568030554 learned_addr learned my addr 172.21.15.172:0/3568030554 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f5117fff640 1 --2- 172.21.15.172:0/3568030554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55706353a780 0x557063546440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f5116ffd640 1 --2- 172.21.15.172:0/3568030554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570635356f0 0x55706353f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f51177fe640 1 -- 172.21.15.172:0/3568030554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570635356f0 msgr2=0x55706353f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f51177fe640 1 --2- 172.21.15.172:0/3568030554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570635356f0 0x55706353f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f51177fe640 1 -- 172.21.15.172:0/3568030554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55706353a780 msgr2=0x557063546440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f51177fe640 1 --2- 172.21.15.172:0/3568030554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55706353a780 0x557063546440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f51177fe640 1 -- 172.21.15.172:0/3568030554 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570631a7210 con 0x557063534db0 2024-09-10T08:54:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f5117fff640 1 --2- 172.21.15.172:0/3568030554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55706353a780 0x557063546440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:54:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f51177fe640 1 --2- 172.21.15.172:0/3568030554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557063534db0 0x55706353eec0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f5108009610 tx=0x7f51080d2620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f04072ca6d7d43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f50f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f50f6ffd640 1 -- 172.21.15.172:0/3568030554 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108020070 con 0x557063534db0 2024-09-10T08:54:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f50f6ffd640 1 -- 172.21.15.172:0/3568030554 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5108097110 con 0x557063534db0 2024-09-10T08:54:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f5120e55740 1 -- 172.21.15.172:0/3568030554 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f510c031780 con 0x557063534db0 2024-09-10T08:54:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f50f6ffd640 1 -- 172.21.15.172:0/3568030554 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f510809b220 con 0x557063534db0 2024-09-10T08:54:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f5120e55740 10 client.17919 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.158+0000 7f5120e55740 1 -- 172.21.15.172:0/3568030554 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557063273620 con 0x557063534db0 2024-09-10T08:54:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.159+0000 7f50f6ffd640 1 -- 172.21.15.172:0/3568030554 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f510c031780 con 0x557063534db0 2024-09-10T08:54:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.159+0000 7f50f6ffd640 1 -- 172.21.15.172:0/3568030554 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f51080ccbb0 con 0x557063534db0 2024-09-10T08:54:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.159+0000 7f5120e55740 10 client.17919 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.159+0000 7f5120e55740 10 client.17919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:54:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.159+0000 7f5120e55740 20 client.17919 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.159+0000 7f5120e55740 1 --2- 172.21.15.172:0/3568030554 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55706354ab30 0x55706354cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.159+0000 7f5120e55740 4 client.17919 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:54:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.159+0000 7f5120e55740 1 -- 172.21.15.172:0/3568030554 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570634a1a20 con 0x55706354ab30 2024-09-10T08:54:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.159+0000 7f5116ffd640 1 --2- 172.21.15.172:0/3568030554 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55706354ab30 0x55706354cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.160+0000 7f50f6ffd640 10 client.17919 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.160+0000 7f5116ffd640 1 --2- 172.21.15.172:0/3568030554 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55706354ab30 0x55706354cf20 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b21424b6489ec41c server_cookie=9048ec3685f68acc in_seq=0 out_seq=0 2024-09-10T08:54:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.160+0000 7f50f6ffd640 1 -- 172.21.15.172:0/3568030554 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570634a1a20 con 0x55706354ab30 2024-09-10T08:54:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.160+0000 7f50f6ffd640 10 client.17919 handle_command_reply: tid=0 2024-09-10T08:54:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.160+0000 7f5120e55740 1 client.17919 shutdown 2024-09-10T08:54:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.161+0000 7f5120e55740 1 -- 172.21.15.172:0/3568030554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557063534db0 msgr2=0x55706353eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.161+0000 7f5120e55740 1 --2- 172.21.15.172:0/3568030554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557063534db0 0x55706353eec0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f5108009610 tx=0x7f51080d2620 comp rx=0 tx=0).stop 2024-09-10T08:54:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.161+0000 7f5120e55740 1 -- 172.21.15.172:0/3568030554 shutdown_connections 2024-09-10T08:54:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.161+0000 7f5120e55740 1 --2- 172.21.15.172:0/3568030554 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55706354ab30 0x55706354cf20 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.161+0000 7f5120e55740 1 --2- 172.21.15.172:0/3568030554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55706353a780 0x557063546440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.161+0000 7f5120e55740 1 --2- 172.21.15.172:0/3568030554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570635356f0 0x55706353f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.161+0000 7f50f6ffd640 0 client.17919 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.161+0000 7f5120e55740 1 --2- 172.21.15.172:0/3568030554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557063534db0 0x55706353eec0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.161+0000 7f5120e55740 1 -- 172.21.15.172:0/3568030554 >> 172.21.15.172:0/3568030554 conn(0x55706352c010 msgr2=0x55706352d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.161+0000 7f5120e55740 1 -- 172.21.15.172:0/3568030554 shutdown_connections 2024-09-10T08:54:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.161+0000 7f5120e55740 1 -- 172.21.15.172:0/3568030554 wait complete. 2024-09-10T08:54:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.161+0000 7f5120e55740 20 client.17919 trim_cache size 0 max 16384 2024-09-10T08:54:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.180+0000 7f5120e55740 1 Processor -- start 2024-09-10T08:54:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.180+0000 7f5120e55740 1 -- start start 2024-09-10T08:54:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.180+0000 7f5120e55740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557063534db0 0x55706352cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.180+0000 7f5120e55740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570635356f0 0x557063546400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.180+0000 7f5120e55740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55706353a780 0x55706352d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.180+0000 7f5120e55740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f51181a82b0 con 0x55706353a780 2024-09-10T08:54:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.180+0000 7f5120e55740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f51181a9df0 con 0x557063534db0 2024-09-10T08:54:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.180+0000 7f5120e55740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f51181a8980 con 0x5570635356f0 2024-09-10T08:54:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.180+0000 7f5117fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55706353a780 0x55706352d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.180+0000 7f5117fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55706353a780 0x55706352d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53040/0 (socket says 172.21.15.172:53040) 2024-09-10T08:54:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.180+0000 7f5117fff640 1 -- 172.21.15.172:0/1052170184 learned_addr learned my addr 172.21.15.172:0/1052170184 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.180+0000 7f5116ffd640 1 --2- 172.21.15.172:0/1052170184 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570635356f0 0x557063546400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.180+0000 7f5117fff640 1 -- 172.21.15.172:0/1052170184 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570635356f0 msgr2=0x557063546400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.180+0000 7f5117fff640 1 --2- 172.21.15.172:0/1052170184 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570635356f0 0x557063546400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.180+0000 7f5117fff640 1 -- 172.21.15.172:0/1052170184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557063534db0 msgr2=0x55706352cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.180+0000 7f5117fff640 1 --2- 172.21.15.172:0/1052170184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557063534db0 0x55706352cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.180+0000 7f5117fff640 1 -- 172.21.15.172:0/1052170184 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706353ea00 con 0x55706353a780 2024-09-10T08:54:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.181+0000 7f5117fff640 1 --2- 172.21.15.172:0/1052170184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55706353a780 0x55706352d400 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f5110012c20 tx=0x7f5110009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803fbc1fafb2ceb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:38.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.181+0000 7f50fd7fa640 1 -- 172.21.15.172:0/1052170184 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511001a070 con 0x55706353a780 2024-09-10T08:54:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.181+0000 7f50fd7fa640 1 -- 172.21.15.172:0/1052170184 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5110026030 con 0x55706353a780 2024-09-10T08:54:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.181+0000 7f50fd7fa640 1 -- 172.21.15.172:0/1052170184 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110022810 con 0x55706353a780 2024-09-10T08:54:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.182+0000 7f5120e55740 1 -- 172.21.15.172:0/1052170184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55706353a780 msgr2=0x55706352d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.182+0000 7f5120e55740 1 --2- 172.21.15.172:0/1052170184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55706353a780 0x55706352d400 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f5110012c20 tx=0x7f5110009900 comp rx=0 tx=0).stop 2024-09-10T08:54:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.182+0000 7f5120e55740 1 -- 172.21.15.172:0/1052170184 shutdown_connections 2024-09-10T08:54:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.182+0000 7f5120e55740 1 --2- 172.21.15.172:0/1052170184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55706353a780 0x55706352d400 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.182+0000 7f5120e55740 1 --2- 172.21.15.172:0/1052170184 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570635356f0 0x557063546400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.182+0000 7f5120e55740 1 --2- 172.21.15.172:0/1052170184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557063534db0 0x55706352cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.182+0000 7f5120e55740 1 -- 172.21.15.172:0/1052170184 >> 172.21.15.172:0/1052170184 conn(0x55706352c010 msgr2=0x55706353ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.182+0000 7f5120e55740 1 -- 172.21.15.172:0/1052170184 shutdown_connections 2024-09-10T08:54:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.182+0000 7f5120e55740 1 -- 172.21.15.172:0/1052170184 wait complete. 2024-09-10T08:54:38.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.182+0000 7f5120e55740 1 -- start start 2024-09-10T08:54:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f5120e55740 1 Processor -- start 2024-09-10T08:54:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f5120e55740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557063534db0 0x5570635436a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f5120e55740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570635356f0 0x557063543be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f5120e55740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55706353a780 0x557063544bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f5120e55740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f51181a9720 con 0x557063534db0 2024-09-10T08:54:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f5120e55740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f51181a9050 con 0x5570635356f0 2024-09-10T08:54:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f5120e55740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f51181a6290 con 0x55706353a780 2024-09-10T08:54:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f51177fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557063534db0 0x5570635436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f5120e55740 10 client.0 _ll_register_callbacks cb 0x7f510800d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:38.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f51177fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557063534db0 0x5570635436a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53048/0 (socket says 172.21.15.172:53048) 2024-09-10T08:54:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f51177fe640 1 -- 172.21.15.172:0/1188197527 learned_addr learned my addr 172.21.15.172:0/1188197527 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f5116ffd640 1 --2- 172.21.15.172:0/1188197527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570635356f0 0x557063543be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f5117fff640 1 --2- 172.21.15.172:0/1188197527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55706353a780 0x557063544bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f51177fe640 1 -- 172.21.15.172:0/1188197527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55706353a780 msgr2=0x557063544bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f51177fe640 1 --2- 172.21.15.172:0/1188197527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55706353a780 0x557063544bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f51177fe640 1 -- 172.21.15.172:0/1188197527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570635356f0 msgr2=0x557063543be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f51177fe640 1 --2- 172.21.15.172:0/1188197527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570635356f0 0x557063543be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f51177fe640 1 -- 172.21.15.172:0/1188197527 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557063545200 con 0x557063534db0 2024-09-10T08:54:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.183+0000 7f5117fff640 1 --2- 172.21.15.172:0/1188197527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55706353a780 0x557063544bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:54:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.184+0000 7f51177fe640 1 --2- 172.21.15.172:0/1188197527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557063534db0 0x5570635436a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f5108097600 tx=0x7f51080d2600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9eafe347e052492 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.184+0000 7f50f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.184+0000 7f50f6ffd640 1 -- 172.21.15.172:0/1188197527 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108014020 con 0x557063534db0 2024-09-10T08:54:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.184+0000 7f50f6ffd640 1 -- 172.21.15.172:0/1188197527 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f51080d2270 con 0x557063534db0 2024-09-10T08:54:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.184+0000 7f5120e55740 1 -- 172.21.15.172:0/1188197527 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5110026030 con 0x557063534db0 2024-09-10T08:54:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.184+0000 7f50f6ffd640 1 -- 172.21.15.172:0/1188197527 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f51080b9050 con 0x557063534db0 2024-09-10T08:54:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.184+0000 7f5120e55740 10 client.17925 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.184+0000 7f5120e55740 1 -- 172.21.15.172:0/1188197527 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557063546580 con 0x557063534db0 2024-09-10T08:54:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.184+0000 7f50f6ffd640 1 -- 172.21.15.172:0/1188197527 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5110026030 con 0x557063534db0 2024-09-10T08:54:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.184+0000 7f50f6ffd640 1 -- 172.21.15.172:0/1188197527 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f51080b4160 con 0x557063534db0 2024-09-10T08:54:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.184+0000 7f5120e55740 10 client.17925 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.184+0000 7f5120e55740 10 client.17925 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:54:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.184+0000 7f5120e55740 20 client.17925 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:38.191 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.184+0000 7f5120e55740 1 --2- 172.21.15.172:0/1188197527 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5570635b5dc0 0x5570635b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.184+0000 7f5120e55740 4 client.17925 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:54:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.184+0000 7f5120e55740 1 -- 172.21.15.172:0/1188197527 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55706349bd70 con 0x5570635b5dc0 2024-09-10T08:54:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.185+0000 7f5116ffd640 1 --2- 172.21.15.172:0/1188197527 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5570635b5dc0 0x5570635b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.185+0000 7f5116ffd640 1 --2- 172.21.15.172:0/1188197527 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5570635b5dc0 0x5570635b81e0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b0336f75410e42e server_cookie=1702f16da8656d0e in_seq=0 out_seq=0 2024-09-10T08:54:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.185+0000 7f50f6ffd640 10 client.17925 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.191+0000 7f50f6ffd640 1 -- 172.21.15.172:0/1188197527 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55706349bd70 con 0x5570635b5dc0 2024-09-10T08:54:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.191+0000 7f50f6ffd640 10 client.17925 handle_command_reply: tid=0 2024-09-10T08:54:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.191+0000 7f5120e55740 1 client.17925 shutdown 2024-09-10T08:54:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f5120e55740 1 -- 172.21.15.172:0/1188197527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557063534db0 msgr2=0x5570635436a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:38.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f5120e55740 1 --2- 172.21.15.172:0/1188197527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557063534db0 0x5570635436a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f5108097600 tx=0x7f51080d2600 comp rx=0 tx=0).stop 2024-09-10T08:54:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f5120e55740 1 -- 172.21.15.172:0/1188197527 shutdown_connections 2024-09-10T08:54:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f5120e55740 1 --2- 172.21.15.172:0/1188197527 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5570635b5dc0 0x5570635b81e0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f5120e55740 1 --2- 172.21.15.172:0/1188197527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55706353a780 0x557063544bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f5120e55740 1 --2- 172.21.15.172:0/1188197527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570635356f0 0x557063543be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f5120e55740 1 --2- 172.21.15.172:0/1188197527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557063534db0 0x5570635436a0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f5120e55740 1 -- 172.21.15.172:0/1188197527 >> 172.21.15.172:0/1188197527 conn(0x55706352c010 msgr2=0x55706352a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f50f6ffd640 0 client.17925 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f5120e55740 1 -- 172.21.15.172:0/1188197527 shutdown_connections 2024-09-10T08:54:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f5120e55740 1 -- 172.21.15.172:0/1188197527 wait complete. 2024-09-10T08:54:38.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f5120e55740 20 client.17925 trim_cache size 0 max 16384 2024-09-10T08:54:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:54:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:54:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:54:38.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:54:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f511dfe4640 1 -- 172.21.15.172:0/3360055018 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51000a2c30 msgr2=0x7f51000a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f511dfe4640 1 --2- 172.21.15.172:0/3360055018 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51000a2c30 0x7f51000a50f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f511806a980 tx=0x7f5110004ab0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f511dfe4640 1 -- 172.21.15.172:0/3360055018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51181025c0 msgr2=0x7f5118197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f511dfe4640 1 --2- 172.21.15.172:0/3360055018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51181025c0 0x7f5118197da0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f510800d770 tx=0x7f510800dc30 comp rx=0 tx=0).stop 2024-09-10T08:54:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f511dfe4640 1 -- 172.21.15.172:0/3360055018 shutdown_connections 2024-09-10T08:54:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f511dfe4640 1 --2- 172.21.15.172:0/3360055018 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51000a2c30 0x7f51000a50f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f511dfe4640 1 --2- 172.21.15.172:0/3360055018 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5118107820 0x7f511819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.192+0000 7f511dfe4640 1 --2- 172.21.15.172:0/3360055018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5118104ef0 0x7f51181982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.193+0000 7f511dfe4640 1 --2- 172.21.15.172:0/3360055018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51181025c0 0x7f5118197da0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.193+0000 7f511dfe4640 1 -- 172.21.15.172:0/3360055018 >> 172.21.15.172:0/3360055018 conn(0x7f51180fc1e0 msgr2=0x7f5118102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.193+0000 7f511dfe4640 1 -- 172.21.15.172:0/3360055018 shutdown_connections 2024-09-10T08:54:38.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:38.193+0000 7f511dfe4640 1 -- 172.21.15.172:0/3360055018 wait complete. 2024-09-10T08:54:38.583 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:54:38.584 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:54:38.608 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:38 smithi172 ceph-mon[30854]: pgmap v602: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:54:38.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:38 smithi174 ceph-mon[34068]: pgmap v602: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:54:38.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:38 smithi191 ceph-mon[35314]: pgmap v602: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:54:38.878 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:54:39.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.436+0000 7f4cda468640 1 -- 172.21.15.172:0/3698098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cd4101470 msgr2=0x7f4cd4101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:39.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.436+0000 7f4cda468640 1 --2- 172.21.15.172:0/3698098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cd4101470 0x7f4cd4101850 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009900 tx=0x7f4cc402a410 comp rx=0 tx=0).stop 2024-09-10T08:54:39.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.436+0000 7f4cda468640 1 -- 172.21.15.172:0/3698098 shutdown_connections 2024-09-10T08:54:39.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.436+0000 7f4cda468640 1 --2- 172.21.15.172:0/3698098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cd4105de0 0x7f4cd410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.436+0000 7f4cda468640 1 --2- 172.21.15.172:0/3698098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd4101e20 0x7f4cd41056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.436+0000 7f4cda468640 1 --2- 172.21.15.172:0/3698098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cd4101470 0x7f4cd4101850 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.436+0000 7f4cda468640 1 -- 172.21.15.172:0/3698098 >> 172.21.15.172:0/3698098 conn(0x7f4cd4077230 msgr2=0x7f4cd40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:39.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.436+0000 7f4cda468640 1 -- 172.21.15.172:0/3698098 shutdown_connections 2024-09-10T08:54:39.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.436+0000 7f4cda468640 1 -- 172.21.15.172:0/3698098 wait complete. 2024-09-10T08:54:39.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.436+0000 7f4cda468640 1 Processor -- start 2024-09-10T08:54:39.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.437+0000 7f4cda468640 1 -- start start 2024-09-10T08:54:39.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.437+0000 7f4cda468640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd4101470 0x7f4cd4198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.437+0000 7f4cda468640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cd4101e20 0x7f4cd4198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.437+0000 7f4cda468640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cd4105de0 0x7f4cd419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.437+0000 7f4cda468640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd4110cf0 con 0x7f4cd4101e20 2024-09-10T08:54:39.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.437+0000 7f4cda468640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd4110b70 con 0x7f4cd4101470 2024-09-10T08:54:39.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.437+0000 7f4cda468640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd4110e70 con 0x7f4cd4105de0 2024-09-10T08:54:39.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.437+0000 7f4cd3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd4101470 0x7f4cd4198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.437+0000 7f4cd3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd4101470 0x7f4cd4198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:51024/0 (socket says 172.21.15.172:51024) 2024-09-10T08:54:39.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.437+0000 7f4cd3fff640 1 -- 172.21.15.172:0/836670354 learned_addr learned my addr 172.21.15.172:0/836670354 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:39.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.437+0000 7f4cd37fe640 1 --2- 172.21.15.172:0/836670354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cd4101e20 0x7f4cd4198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.438+0000 7f4cd8a05640 1 --2- 172.21.15.172:0/836670354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cd4105de0 0x7f4cd419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.438+0000 7f4cd3fff640 1 -- 172.21.15.172:0/836670354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cd4105de0 msgr2=0x7f4cd419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:39.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.438+0000 7f4cd3fff640 1 --2- 172.21.15.172:0/836670354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cd4105de0 0x7f4cd419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.438+0000 7f4cd3fff640 1 -- 172.21.15.172:0/836670354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cd4101e20 msgr2=0x7f4cd4198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:39.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.438+0000 7f4cd3fff640 1 --2- 172.21.15.172:0/836670354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cd4101e20 0x7f4cd4198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.438+0000 7f4cd3fff640 1 -- 172.21.15.172:0/836670354 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd419d2c0 con 0x7f4cd4101470 2024-09-10T08:54:39.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.438+0000 7f4cd37fe640 1 --2- 172.21.15.172:0/836670354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cd4101e20 0x7f4cd4198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:54:39.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.438+0000 7f4cd3fff640 1 --2- 172.21.15.172:0/836670354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd4101470 0x7f4cd4198310 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4005940 tx=0x7f4cc4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:39.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.439+0000 7f4cd17fa640 1 -- 172.21.15.172:0/836670354 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc403f030 con 0x7f4cd4101470 2024-09-10T08:54:39.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.439+0000 7f4cd17fa640 1 -- 172.21.15.172:0/836670354 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cc4003aa0 con 0x7f4cd4101470 2024-09-10T08:54:39.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.439+0000 7f4cda468640 1 -- 172.21.15.172:0/836670354 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd419d550 con 0x7f4cd4101470 2024-09-10T08:54:39.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.439+0000 7f4cda468640 1 -- 172.21.15.172:0/836670354 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd41a4e90 con 0x7f4cd4101470 2024-09-10T08:54:39.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.439+0000 7f4cd17fa640 1 -- 172.21.15.172:0/836670354 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4035070 con 0x7f4cd4101470 2024-09-10T08:54:39.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.442+0000 7f4cd17fa640 1 -- 172.21.15.172:0/836670354 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4cc4042030 con 0x7f4cd4101470 2024-09-10T08:54:39.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.442+0000 7f4cdc841740 1 Processor -- start 2024-09-10T08:54:39.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.442+0000 7f4cdc841740 1 -- start start 2024-09-10T08:54:39.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cdc841740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 0x556b766e0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cd17fa640 1 --2- 172.21.15.172:0/836670354 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4cbc0a2c00 0x7f4cbc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cdc841740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766e1480 0x556b766eb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cdc841740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766ebd40 0x556b766ee130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cdc841740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556b76646280 con 0x556b766e0b60 2024-09-10T08:54:39.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cdc841740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556b765edb10 con 0x556b766ebd40 2024-09-10T08:54:39.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cdc841740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556b76642240 con 0x556b766e1480 2024-09-10T08:54:39.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cd3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 0x556b766e0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cd3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 0x556b766e0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53082/0 (socket says 172.21.15.172:53082) 2024-09-10T08:54:39.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cd3fff640 1 -- 172.21.15.172:0/635608722 learned_addr learned my addr 172.21.15.172:0/635608722 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:39.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cd37fe640 1 --2- 172.21.15.172:0/635608722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766e1480 0x556b766eb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cd8a05640 1 --2- 172.21.15.172:0/635608722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766ebd40 0x556b766ee130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cd17fa640 1 -- 172.21.15.172:0/836670354 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4cc40ef8e0 con 0x7f4cd4101470 2024-09-10T08:54:39.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cd3fff640 1 -- 172.21.15.172:0/635608722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766e1480 msgr2=0x556b766eb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:39.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cd3fff640 1 --2- 172.21.15.172:0/635608722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766e1480 0x556b766eb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cd3fff640 1 -- 172.21.15.172:0/635608722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766ebd40 msgr2=0x556b766ee130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:39.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cd3fff640 1 --2- 172.21.15.172:0/635608722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766ebd40 0x556b766ee130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cd3fff640 1 -- 172.21.15.172:0/635608722 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b765fa1f0 con 0x556b766e0b60 2024-09-10T08:54:39.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.443+0000 7f4cd8a05640 1 --2- 172.21.15.172:0/836670354 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4cbc0a2c00 0x7f4cbc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.444+0000 7f4cd3fff640 1 --2- 172.21.15.172:0/635608722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 0x556b766e0f40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4cc403b790 tx=0x7f4cc4035000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b939c4b61a5ec9cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:39.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.444+0000 7f4cba7fc640 1 -- 172.21.15.172:0/635608722 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4049070 con 0x556b766e0b60 2024-09-10T08:54:39.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.444+0000 7f4cba7fc640 1 -- 172.21.15.172:0/635608722 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cc40bbde0 con 0x556b766e0b60 2024-09-10T08:54:39.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.444+0000 7f4cd8a05640 1 --2- 172.21.15.172:0/836670354 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4cbc0a2c00 0x7f4cbc0a50c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x556b766de470 tx=0x7f4ccc009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:39.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.444+0000 7f4cba7fc640 1 -- 172.21.15.172:0/635608722 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4095350 con 0x556b766e0b60 2024-09-10T08:54:39.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.445+0000 7f4cdc841740 1 -- 172.21.15.172:0/635608722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 msgr2=0x556b766e0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:39.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.445+0000 7f4cdc841740 1 --2- 172.21.15.172:0/635608722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 0x556b766e0f40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4cc403b790 tx=0x7f4cc4035000 comp rx=0 tx=0).stop 2024-09-10T08:54:39.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.446+0000 7f4cdc841740 1 -- 172.21.15.172:0/635608722 shutdown_connections 2024-09-10T08:54:39.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.446+0000 7f4cdc841740 1 --2- 172.21.15.172:0/635608722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766ebd40 0x556b766ee130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.446+0000 7f4cdc841740 1 --2- 172.21.15.172:0/635608722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766e1480 0x556b766eb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.446+0000 7f4cdc841740 1 --2- 172.21.15.172:0/635608722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 0x556b766e0f40 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.446+0000 7f4cdc841740 1 -- 172.21.15.172:0/635608722 >> 172.21.15.172:0/635608722 conn(0x556b766dd0f0 msgr2=0x556b766dd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:39.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.446+0000 7f4cdc841740 1 -- 172.21.15.172:0/635608722 shutdown_connections 2024-09-10T08:54:39.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.446+0000 7f4cdc841740 1 -- 172.21.15.172:0/635608722 wait complete. 2024-09-10T08:54:39.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.446+0000 7f4cdc841740 1 -- start start 2024-09-10T08:54:39.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.447+0000 7f4cdc841740 1 Processor -- start 2024-09-10T08:54:39.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.447+0000 7f4cdc841740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 0x556b766e8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.447+0000 7f4cdc841740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766e1480 0x556b766e9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.447+0000 7f4cdc841740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766ebd40 0x556b766e9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.447+0000 7f4cdc841740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556b766ee6d0 con 0x556b766e0b60 2024-09-10T08:54:39.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.447+0000 7f4cdc841740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556b76600f10 con 0x556b766e1480 2024-09-10T08:54:39.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.447+0000 7f4cdc841740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556b766ee850 con 0x556b766ebd40 2024-09-10T08:54:39.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.447+0000 7f4cdc841740 10 client.0 _ll_register_callbacks cb 0x556b765c9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:39.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.447+0000 7f4cd3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 0x556b766e8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.447+0000 7f4cd8a05640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766ebd40 0x556b766e9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.447+0000 7f4cd37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766e1480 0x556b766e9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.447+0000 7f4cd8a05640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766ebd40 0x556b766e9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:54080/0 (socket says 172.21.15.172:54080) 2024-09-10T08:54:39.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.447+0000 7f4cd8a05640 1 -- 172.21.15.172:0/2955657780 learned_addr learned my addr 172.21.15.172:0/2955657780 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:39.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.447+0000 7f4cd37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766e1480 0x556b766e9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:51044/0 (socket says 172.21.15.172:51044) 2024-09-10T08:54:39.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.448+0000 7f4cd3fff640 1 -- 172.21.15.172:0/2955657780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766ebd40 msgr2=0x556b766e9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:39.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.448+0000 7f4cd3fff640 1 --2- 172.21.15.172:0/2955657780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766ebd40 0x556b766e9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.448+0000 7f4cd3fff640 1 -- 172.21.15.172:0/2955657780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766e1480 msgr2=0x556b766e9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:39.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.448+0000 7f4cd3fff640 1 --2- 172.21.15.172:0/2955657780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766e1480 0x556b766e9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.448+0000 7f4cd3fff640 1 -- 172.21.15.172:0/2955657780 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b763b76a0 con 0x556b766e0b60 2024-09-10T08:54:39.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.448+0000 7f4cd8a05640 1 --2- 172.21.15.172:0/2955657780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766ebd40 0x556b766e9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:39.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.448+0000 7f4cd37fe640 1 --2- 172.21.15.172:0/2955657780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766e1480 0x556b766e9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:39.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.448+0000 7f4cd3fff640 1 --2- 172.21.15.172:0/2955657780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 0x556b766e8bd0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40ed0d0 tx=0x7f4cc402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4392a3a3922143ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:39.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.448+0000 7f4cb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:39.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.448+0000 7f4cb37fe640 1 -- 172.21.15.172:0/2955657780 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4049070 con 0x556b766e0b60 2024-09-10T08:54:39.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.449+0000 7f4cb37fe640 1 -- 172.21.15.172:0/2955657780 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4cc40b8450 con 0x556b766e0b60 2024-09-10T08:54:39.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.449+0000 7f4cdc841740 1 -- 172.21.15.172:0/2955657780 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cc40bbde0 con 0x556b766e0b60 2024-09-10T08:54:39.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.449+0000 7f4cb37fe640 1 -- 172.21.15.172:0/2955657780 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4cc40b8080 con 0x556b766e0b60 2024-09-10T08:54:39.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.449+0000 7f4cdc841740 10 client.17937 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:39.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.449+0000 7f4cdc841740 1 -- 172.21.15.172:0/2955657780 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556b766ee6d0 con 0x556b766e0b60 2024-09-10T08:54:39.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.449+0000 7f4cb37fe640 1 -- 172.21.15.172:0/2955657780 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cc40bbde0 con 0x556b766e0b60 2024-09-10T08:54:39.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.449+0000 7f4cb37fe640 1 -- 172.21.15.172:0/2955657780 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f4cc4003c50 con 0x556b766e0b60 2024-09-10T08:54:39.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.449+0000 7f4cdc841740 10 client.17937 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:39.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.449+0000 7f4cdc841740 10 client.17937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:54:39.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.449+0000 7f4cdc841740 20 client.17937 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:39.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.449+0000 7f4cdc841740 1 --2- 172.21.15.172:0/2955657780 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556b766fb450 0x556b766fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.449+0000 7f4cdc841740 4 client.17937 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:54:39.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.449+0000 7f4cdc841740 1 -- 172.21.15.172:0/2955657780 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b766537f0 con 0x556b766fb450 2024-09-10T08:54:39.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.450+0000 7f4cd37fe640 1 --2- 172.21.15.172:0/2955657780 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556b766fb450 0x556b766fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.450+0000 7f4cb37fe640 10 client.17937 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:39.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.450+0000 7f4cd37fe640 1 --2- 172.21.15.172:0/2955657780 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556b766fb450 0x556b766fd8d0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac5119454f6d202 server_cookie=53159f03149fdbb0 in_seq=0 out_seq=0 2024-09-10T08:54:39.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.451+0000 7f4cb37fe640 1 -- 172.21.15.172:0/2955657780 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b766537f0 con 0x556b766fb450 2024-09-10T08:54:39.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.451+0000 7f4cb37fe640 10 client.17937 handle_command_reply: tid=0 2024-09-10T08:54:39.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.451+0000 7f4cdc841740 1 client.17937 shutdown 2024-09-10T08:54:39.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.451+0000 7f4cdc841740 1 -- 172.21.15.172:0/2955657780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 msgr2=0x556b766e8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:39.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.451+0000 7f4cdc841740 1 --2- 172.21.15.172:0/2955657780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 0x556b766e8bd0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40ed0d0 tx=0x7f4cc402a410 comp rx=0 tx=0).stop 2024-09-10T08:54:39.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.451+0000 7f4cdc841740 1 -- 172.21.15.172:0/2955657780 shutdown_connections 2024-09-10T08:54:39.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.451+0000 7f4cdc841740 1 --2- 172.21.15.172:0/2955657780 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556b766fb450 0x556b766fd8d0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.451+0000 7f4cdc841740 1 --2- 172.21.15.172:0/2955657780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766ebd40 0x556b766e9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.451+0000 7f4cb37fe640 0 client.17937 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:39.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.451+0000 7f4cdc841740 1 --2- 172.21.15.172:0/2955657780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766e1480 0x556b766e9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.452+0000 7f4cdc841740 1 --2- 172.21.15.172:0/2955657780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 0x556b766e8bd0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.452+0000 7f4cdc841740 1 -- 172.21.15.172:0/2955657780 >> 172.21.15.172:0/2955657780 conn(0x556b766dd0f0 msgr2=0x556b766dc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:39.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.452+0000 7f4cdc841740 1 -- 172.21.15.172:0/2955657780 shutdown_connections 2024-09-10T08:54:39.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.452+0000 7f4cdc841740 1 -- 172.21.15.172:0/2955657780 wait complete. 2024-09-10T08:54:39.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.452+0000 7f4cdc841740 20 client.17937 trim_cache size 0 max 16384 2024-09-10T08:54:39.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.470+0000 7f4cdc841740 1 Processor -- start 2024-09-10T08:54:39.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.470+0000 7f4cdc841740 1 -- start start 2024-09-10T08:54:39.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.471+0000 7f4cdc841740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766e0b60 0x556b767031a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.471+0000 7f4cdc841740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e1480 0x556b767036e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.471+0000 7f4cdc841740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766ebd40 0x556b766e7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.471+0000 7f4cdc841740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc000fa50 con 0x556b766e1480 2024-09-10T08:54:39.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.471+0000 7f4cdc841740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc0011590 con 0x556b766e0b60 2024-09-10T08:54:39.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.471+0000 7f4cdc841740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc0010120 con 0x556b766ebd40 2024-09-10T08:54:39.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.471+0000 7f4cd37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e1480 0x556b767036e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.471+0000 7f4cd37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e1480 0x556b767036e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53094/0 (socket says 172.21.15.172:53094) 2024-09-10T08:54:39.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.471+0000 7f4cd37fe640 1 -- 172.21.15.172:0/4273023453 learned_addr learned my addr 172.21.15.172:0/4273023453 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:39.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.471+0000 7f4cd3fff640 1 --2- 172.21.15.172:0/4273023453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766e0b60 0x556b767031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.471+0000 7f4cd8a05640 1 --2- 172.21.15.172:0/4273023453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766ebd40 0x556b766e7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.471+0000 7f4cd3fff640 1 -- 172.21.15.172:0/4273023453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766ebd40 msgr2=0x556b766e7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:39.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.471+0000 7f4cd3fff640 1 --2- 172.21.15.172:0/4273023453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766ebd40 0x556b766e7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.471+0000 7f4cd3fff640 1 -- 172.21.15.172:0/4273023453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e1480 msgr2=0x556b767036e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:39.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.471+0000 7f4cd3fff640 1 --2- 172.21.15.172:0/4273023453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e1480 0x556b767036e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.471+0000 7f4cd3fff640 1 -- 172.21.15.172:0/4273023453 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b766e12b0 con 0x556b766e0b60 2024-09-10T08:54:39.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.471+0000 7f4cd8a05640 1 --2- 172.21.15.172:0/4273023453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766ebd40 0x556b766e7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:54:39.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.472+0000 7f4cd37fe640 1 --2- 172.21.15.172:0/4273023453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e1480 0x556b767036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:39.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.472+0000 7f4cd3fff640 1 --2- 172.21.15.172:0/4273023453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766e0b60 0x556b767031a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40c4520 tx=0x7f4cc4005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68977a743f74750f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:39.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.472+0000 7f4cb9ffb640 1 -- 172.21.15.172:0/4273023453 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4049070 con 0x556b766e0b60 2024-09-10T08:54:39.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.472+0000 7f4cb9ffb640 1 -- 172.21.15.172:0/4273023453 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cc403b460 con 0x556b766e0b60 2024-09-10T08:54:39.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.472+0000 7f4cb9ffb640 1 -- 172.21.15.172:0/4273023453 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4003c50 con 0x556b766e0b60 2024-09-10T08:54:39.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.473+0000 7f4cdc841740 1 -- 172.21.15.172:0/4273023453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766e0b60 msgr2=0x556b767031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:39.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.473+0000 7f4cdc841740 1 --2- 172.21.15.172:0/4273023453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766e0b60 0x556b767031a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4cc40c4520 tx=0x7f4cc4005c00 comp rx=0 tx=0).stop 2024-09-10T08:54:39.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.473+0000 7f4cdc841740 1 -- 172.21.15.172:0/4273023453 shutdown_connections 2024-09-10T08:54:39.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.473+0000 7f4cdc841740 1 --2- 172.21.15.172:0/4273023453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766ebd40 0x556b766e7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.473+0000 7f4cdc841740 1 --2- 172.21.15.172:0/4273023453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e1480 0x556b767036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.473+0000 7f4cdc841740 1 --2- 172.21.15.172:0/4273023453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766e0b60 0x556b767031a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.473+0000 7f4cdc841740 1 -- 172.21.15.172:0/4273023453 >> 172.21.15.172:0/4273023453 conn(0x556b766dd0f0 msgr2=0x556b766eab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:39.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.473+0000 7f4cdc841740 1 -- 172.21.15.172:0/4273023453 shutdown_connections 2024-09-10T08:54:39.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.473+0000 7f4cdc841740 1 -- 172.21.15.172:0/4273023453 wait complete. 2024-09-10T08:54:39.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.473+0000 7f4cdc841740 1 -- start start 2024-09-10T08:54:39.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.473+0000 7f4cdc841740 1 Processor -- start 2024-09-10T08:54:39.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.474+0000 7f4cdc841740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 0x556b766f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.474+0000 7f4cdc841740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766e1480 0x556b766ef390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.474+0000 7f4cdc841740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766ebd40 0x556b766ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.474+0000 7f4cdc841740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc0010ec0 con 0x556b766e0b60 2024-09-10T08:54:39.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.474+0000 7f4cdc841740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc00107f0 con 0x556b766ebd40 2024-09-10T08:54:39.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.474+0000 7f4cdc841740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc000da30 con 0x556b766e1480 2024-09-10T08:54:39.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.474+0000 7f4cdc841740 10 client.0 _ll_register_callbacks cb 0x7f4cc8008390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:39.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.474+0000 7f4cd3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 0x556b766f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.474+0000 7f4cd3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 0x556b766f3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53106/0 (socket says 172.21.15.172:53106) 2024-09-10T08:54:39.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.474+0000 7f4cd3fff640 1 -- 172.21.15.172:0/1945954856 learned_addr learned my addr 172.21.15.172:0/1945954856 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:39.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.474+0000 7f4cd37fe640 1 --2- 172.21.15.172:0/1945954856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766e1480 0x556b766ef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.474+0000 7f4cd8a05640 1 --2- 172.21.15.172:0/1945954856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766ebd40 0x556b766ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.474+0000 7f4cd37fe640 1 -- 172.21.15.172:0/1945954856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766ebd40 msgr2=0x556b766ef8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:39.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.474+0000 7f4cd37fe640 1 --2- 172.21.15.172:0/1945954856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766ebd40 0x556b766ef8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.474+0000 7f4cd37fe640 1 -- 172.21.15.172:0/1945954856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 msgr2=0x556b766f3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:39.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.474+0000 7f4cd37fe640 1 --2- 172.21.15.172:0/1945954856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 0x556b766f3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.474+0000 7f4cd37fe640 1 -- 172.21.15.172:0/1945954856 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b766efe90 con 0x556b766e1480 2024-09-10T08:54:39.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.475+0000 7f4cd37fe640 1 --2- 172.21.15.172:0/1945954856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766e1480 0x556b766ef390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8013890 tx=0x7f4cc80207d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61a29c40eef7d889 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:39.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.475+0000 7f4cb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:54:39.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.475+0000 7f4cb37fe640 1 -- 172.21.15.172:0/1945954856 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc801e8b0 con 0x556b766e1480 2024-09-10T08:54:39.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.475+0000 7f4cdc841740 1 -- 172.21.15.172:0/1945954856 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cc403b460 con 0x556b766e1480 2024-09-10T08:54:39.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.475+0000 7f4cb37fe640 1 -- 172.21.15.172:0/1945954856 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4cc801ea50 con 0x556b766e1480 2024-09-10T08:54:39.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.475+0000 7f4cb37fe640 1 -- 172.21.15.172:0/1945954856 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4cc8035050 con 0x556b766e1480 2024-09-10T08:54:39.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.475+0000 7f4cdc841740 10 client.26905 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:39.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.475+0000 7f4cdc841740 1 -- 172.21.15.172:0/1945954856 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556b76765f70 con 0x556b766e1480 2024-09-10T08:54:39.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.475+0000 7f4cb37fe640 1 -- 172.21.15.172:0/1945954856 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4cc403b460 con 0x556b766e1480 2024-09-10T08:54:39.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.475+0000 7f4cb37fe640 1 -- 172.21.15.172:0/1945954856 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f4cc80303b0 con 0x556b766e1480 2024-09-10T08:54:39.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.476+0000 7f4cdc841740 10 client.26905 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:39.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.476+0000 7f4cdc841740 10 client.26905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:54:39.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.476+0000 7f4cdc841740 20 client.26905 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:39.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.476+0000 7f4cdc841740 1 --2- 172.21.15.172:0/1945954856 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556b766fb450 0x556b76768730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:39.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.476+0000 7f4cdc841740 4 client.26905 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:54:39.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.476+0000 7f4cdc841740 1 -- 172.21.15.172:0/1945954856 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556b7664dd00 con 0x556b766fb450 2024-09-10T08:54:39.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.476+0000 7f4cd3fff640 1 --2- 172.21.15.172:0/1945954856 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556b766fb450 0x556b76768730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:39.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.476+0000 7f4cb37fe640 10 client.26905 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:39.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.476+0000 7f4cd3fff640 1 --2- 172.21.15.172:0/1945954856 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556b766fb450 0x556b76768730 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=204e523c6ab91dc7 server_cookie=ff859794b932e84f in_seq=0 out_seq=0 2024-09-10T08:54:39.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.477+0000 7f4cb37fe640 1 -- 172.21.15.172:0/1945954856 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556b7664dd00 con 0x556b766fb450 2024-09-10T08:54:39.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.477+0000 7f4cb37fe640 10 client.26905 handle_command_reply: tid=0 2024-09-10T08:54:39.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.477+0000 7f4cdc841740 1 client.26905 shutdown 2024-09-10T08:54:39.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.478+0000 7f4cdc841740 1 -- 172.21.15.172:0/1945954856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766e1480 msgr2=0x556b766ef390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:39.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.478+0000 7f4cdc841740 1 --2- 172.21.15.172:0/1945954856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766e1480 0x556b766ef390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8013890 tx=0x7f4cc80207d0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.478+0000 7f4cdc841740 1 -- 172.21.15.172:0/1945954856 shutdown_connections 2024-09-10T08:54:39.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.478+0000 7f4cdc841740 1 --2- 172.21.15.172:0/1945954856 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556b766fb450 0x556b76768730 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.478+0000 7f4cdc841740 1 --2- 172.21.15.172:0/1945954856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b766ebd40 0x556b766ef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.478+0000 7f4cdc841740 1 --2- 172.21.15.172:0/1945954856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556b766e1480 0x556b766ef390 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.478+0000 7f4cdc841740 1 --2- 172.21.15.172:0/1945954856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556b766e0b60 0x556b766f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.478+0000 7f4cdc841740 1 -- 172.21.15.172:0/1945954856 >> 172.21.15.172:0/1945954856 conn(0x556b766dd0f0 msgr2=0x556b766dba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:39.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.478+0000 7f4cb37fe640 0 client.26905 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:39.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.478+0000 7f4cdc841740 1 -- 172.21.15.172:0/1945954856 shutdown_connections 2024-09-10T08:54:39.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.478+0000 7f4cdc841740 1 -- 172.21.15.172:0/1945954856 wait complete. 2024-09-10T08:54:39.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.478+0000 7f4cdc841740 20 client.26905 trim_cache size 0 max 16384 2024-09-10T08:54:39.484 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:54:39.485 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T08:54:39.485 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "1a9db4bb-ed7f-4f82-bbe2-5657b3f721a4", 2024-09-10T08:54:39.485 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T08:54:39.485 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:54:39.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.479+0000 7f4cda468640 1 -- 172.21.15.172:0/836670354 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4cbc0a2c00 msgr2=0x7f4cbc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:39.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.479+0000 7f4cda468640 1 --2- 172.21.15.172:0/836670354 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4cbc0a2c00 0x7f4cbc0a50c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x556b766de470 tx=0x7f4ccc009600 comp rx=0 tx=0).stop 2024-09-10T08:54:39.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.479+0000 7f4cda468640 1 -- 172.21.15.172:0/836670354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd4101470 msgr2=0x7f4cd4198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:39.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.479+0000 7f4cda468640 1 --2- 172.21.15.172:0/836670354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd4101470 0x7f4cd4198310 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4005940 tx=0x7f4cc4004270 comp rx=0 tx=0).stop 2024-09-10T08:54:39.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.479+0000 7f4cda468640 1 -- 172.21.15.172:0/836670354 shutdown_connections 2024-09-10T08:54:39.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.479+0000 7f4cda468640 1 --2- 172.21.15.172:0/836670354 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4cbc0a2c00 0x7f4cbc0a50c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.479+0000 7f4cda468640 1 --2- 172.21.15.172:0/836670354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cd4105de0 0x7f4cd419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.479+0000 7f4cda468640 1 --2- 172.21.15.172:0/836670354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cd4101e20 0x7f4cd4198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.479+0000 7f4cda468640 1 --2- 172.21.15.172:0/836670354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd4101470 0x7f4cd4198310 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:39.486 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.479+0000 7f4cda468640 1 -- 172.21.15.172:0/836670354 >> 172.21.15.172:0/836670354 conn(0x7f4cd4077230 msgr2=0x7f4cd4104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:39.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.479+0000 7f4cda468640 1 -- 172.21.15.172:0/836670354 shutdown_connections 2024-09-10T08:54:39.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:39.479+0000 7f4cda468640 1 -- 172.21.15.172:0/836670354 wait complete. 2024-09-10T08:54:40.202 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:54:40.555 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:54:40.680 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:40 smithi172 ceph-mon[30854]: pgmap v603: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-09-10T08:54:40.680 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:40 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:54:40.681 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:40 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:54:40.681 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:40 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:54:40.681 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:40 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:40.681 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:40 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:54:40.681 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:40 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:40.682 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:40 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:40.682 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:40 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:54:40.682 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:40 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:40.682 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:40 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:40.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:40 smithi174 ceph-mon[34068]: pgmap v603: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-09-10T08:54:40.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:40 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:54:40.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:40 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:54:40.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:40 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:54:40.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:40 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:40.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:40 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:54:40.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:40 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:40.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:40 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:40.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:40 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:54:40.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:40 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:40.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:40 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:40.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:40 smithi191 ceph-mon[35314]: pgmap v603: 129 pgs: 129 active+clean; 7.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-09-10T08:54:40.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:40 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:54:40.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:40 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:54:40.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:40 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:54:40.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:40 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:40.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:40 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:54:40.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:40 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:40.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:40 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:40.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:40 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:54:40.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:40 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:40.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:40 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:41.111 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.094+0000 7fdbf8f26640 1 -- 172.21.15.172:0/2018376202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbf4102ef0 msgr2=0x7fdbf410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:41.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.094+0000 7fdbf8f26640 1 --2- 172.21.15.172:0/2018376202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbf4102ef0 0x7fdbf410da40 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009ca0 tx=0x7fdbec02a950 comp rx=0 tx=0).stop 2024-09-10T08:54:41.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.094+0000 7fdbf8f26640 1 -- 172.21.15.172:0/2018376202 shutdown_connections 2024-09-10T08:54:41.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.094+0000 7fdbf8f26640 1 --2- 172.21.15.172:0/2018376202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbf4102ef0 0x7fdbf410da40 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.094+0000 7fdbf8f26640 1 --2- 172.21.15.172:0/2018376202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbf4102570 0x7fdbf41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.094+0000 7fdbf8f26640 1 --2- 172.21.15.172:0/2018376202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbf4100e20 0x7fdbf4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.094+0000 7fdbf8f26640 1 -- 172.21.15.172:0/2018376202 >> 172.21.15.172:0/2018376202 conn(0x7fdbf40fc7d0 msgr2=0x7fdbf40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:41.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.094+0000 7fdbf8f26640 1 -- 172.21.15.172:0/2018376202 shutdown_connections 2024-09-10T08:54:41.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.094+0000 7fdbf8f26640 1 -- 172.21.15.172:0/2018376202 wait complete. 2024-09-10T08:54:41.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.095+0000 7fdbf8f26640 1 Processor -- start 2024-09-10T08:54:41.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.095+0000 7fdbf8f26640 1 -- start start 2024-09-10T08:54:41.112 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.095+0000 7fdbf8f26640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbf4100e20 0x7fdbf41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf8f26640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbf4102570 0x7fdbf4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf8f26640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbf4102ef0 0x7fdbf419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf8f26640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf4111f50 con 0x7fdbf4102570 2024-09-10T08:54:41.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf8f26640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf4111dd0 con 0x7fdbf4102ef0 2024-09-10T08:54:41.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf8f26640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41120d0 con 0x7fdbf4100e20 2024-09-10T08:54:41.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbf4102570 0x7fdbf4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbf4102570 0x7fdbf4198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53132/0 (socket says 172.21.15.172:53132) 2024-09-10T08:54:41.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf1d9b640 1 -- 172.21.15.172:0/1958047305 learned_addr learned my addr 172.21.15.172:0/1958047305 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:41.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf2d9d640 1 --2- 172.21.15.172:0/1958047305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbf4102ef0 0x7fdbf419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf259c640 1 --2- 172.21.15.172:0/1958047305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbf4100e20 0x7fdbf41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf1d9b640 1 -- 172.21.15.172:0/1958047305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbf4100e20 msgr2=0x7fdbf41983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:41.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf1d9b640 1 --2- 172.21.15.172:0/1958047305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbf4100e20 0x7fdbf41983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf1d9b640 1 -- 172.21.15.172:0/1958047305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbf4102ef0 msgr2=0x7fdbf419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:41.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf1d9b640 1 --2- 172.21.15.172:0/1958047305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbf4102ef0 0x7fdbf419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf1d9b640 1 -- 172.21.15.172:0/1958047305 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf419d380 con 0x7fdbf4102570 2024-09-10T08:54:41.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf2d9d640 1 --2- 172.21.15.172:0/1958047305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbf4102ef0 0x7fdbf419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:41.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf1d9b640 1 --2- 172.21.15.172:0/1958047305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbf4102570 0x7fdbf4198910 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800e9f0 tx=0x7fdbe800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:41.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbe37fe640 1 -- 172.21.15.172:0/1958047305 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe800cd80 con 0x7fdbf4102570 2024-09-10T08:54:41.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbe37fe640 1 -- 172.21.15.172:0/1958047305 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbe80108e0 con 0x7fdbf4102570 2024-09-10T08:54:41.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbf8f26640 1 -- 172.21.15.172:0/1958047305 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf419d670 con 0x7fdbf4102570 2024-09-10T08:54:41.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.096+0000 7fdbe37fe640 1 -- 172.21.15.172:0/1958047305 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8018920 con 0x7fdbf4102570 2024-09-10T08:54:41.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.097+0000 7fdbf8f26640 1 -- 172.21.15.172:0/1958047305 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf41a4f50 con 0x7fdbf4102570 2024-09-10T08:54:41.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.099+0000 7fdbe37fe640 1 -- 172.21.15.172:0/1958047305 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fdbe8018ac0 con 0x7fdbf4102570 2024-09-10T08:54:41.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.099+0000 7fdbfb2ff740 1 Processor -- start 2024-09-10T08:54:41.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.099+0000 7fdbfb2ff740 1 -- start start 2024-09-10T08:54:41.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbfb2ff740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c308d360 0x55b9c308d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbfb2ff740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c308dc80 0x55b9c309c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbfb2ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c309ca90 0x55b9c309ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbfb2ff740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b9c2ff32e0 con 0x55b9c308dc80 2024-09-10T08:54:41.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbfb2ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b9c2f9ace0 con 0x55b9c309ca90 2024-09-10T08:54:41.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbfb2ff740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b9c2fef2a0 con 0x55b9c308d360 2024-09-10T08:54:41.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbf259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c308dc80 0x55b9c309c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbf259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c308dc80 0x55b9c309c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53148/0 (socket says 172.21.15.172:53148) 2024-09-10T08:54:41.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbf259c640 1 -- 172.21.15.172:0/3108609667 learned_addr learned my addr 172.21.15.172:0/3108609667 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:41.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbf2d9d640 1 --2- 172.21.15.172:0/3108609667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c308d360 0x55b9c308d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbf1d9b640 1 --2- 172.21.15.172:0/3108609667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c309ca90 0x55b9c309ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbf259c640 1 -- 172.21.15.172:0/3108609667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c308d360 msgr2=0x55b9c308d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:41.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbf259c640 1 --2- 172.21.15.172:0/3108609667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c308d360 0x55b9c308d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbf259c640 1 -- 172.21.15.172:0/3108609667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c309ca90 msgr2=0x55b9c309ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:41.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbf259c640 1 --2- 172.21.15.172:0/3108609667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c309ca90 0x55b9c309ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbe37fe640 1 --2- 172.21.15.172:0/1958047305 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdbdc0a2b80 0x7fdbdc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbf259c640 1 -- 172.21.15.172:0/3108609667 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c2f7f440 con 0x55b9c308dc80 2024-09-10T08:54:41.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbe37fe640 1 -- 172.21.15.172:0/1958047305 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fdbe80cf6d0 con 0x7fdbf4102570 2024-09-10T08:54:41.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbf259c640 1 --2- 172.21.15.172:0/3108609667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c308dc80 0x55b9c309c550 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400b760 tx=0x7fdbe400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83d62e88b5f5110 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:41.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.100+0000 7fdbf2d9d640 1 --2- 172.21.15.172:0/1958047305 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdbdc0a2b80 0x7fdbdc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.101+0000 7fdbe0ff9640 1 -- 172.21.15.172:0/3108609667 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4003dd0 con 0x55b9c308dc80 2024-09-10T08:54:41.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.101+0000 7fdbe0ff9640 1 -- 172.21.15.172:0/3108609667 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbe4015070 con 0x55b9c308dc80 2024-09-10T08:54:41.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.101+0000 7fdbe0ff9640 1 -- 172.21.15.172:0/3108609667 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4004d90 con 0x55b9c308dc80 2024-09-10T08:54:41.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.102+0000 7fdbf2d9d640 1 --2- 172.21.15.172:0/1958047305 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdbdc0a2b80 0x7fdbdc0a5040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdbec02fd10 tx=0x7fdbec005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:41.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.102+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/3108609667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c308dc80 msgr2=0x55b9c309c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:41.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.102+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/3108609667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c308dc80 0x55b9c309c550 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400b760 tx=0x7fdbe400bc20 comp rx=0 tx=0).stop 2024-09-10T08:54:41.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.102+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/3108609667 shutdown_connections 2024-09-10T08:54:41.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.102+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/3108609667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c309ca90 0x55b9c309ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.102+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/3108609667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c308dc80 0x55b9c309c550 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.102+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/3108609667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c308d360 0x55b9c308d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.102+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/3108609667 >> 172.21.15.172:0/3108609667 conn(0x55b9c3089a30 msgr2=0x55b9c3089e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:41.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.102+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/3108609667 shutdown_connections 2024-09-10T08:54:41.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.102+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/3108609667 wait complete. 2024-09-10T08:54:41.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.102+0000 7fdbfb2ff740 1 -- start start 2024-09-10T08:54:41.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbfb2ff740 1 Processor -- start 2024-09-10T08:54:41.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbfb2ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308d360 0x55b9c309c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbfb2ff740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c308dc80 0x55b9c3099720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbfb2ff740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c309ca90 0x55b9c3099cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbfb2ff740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b9c309f660 con 0x55b9c309ca90 2024-09-10T08:54:41.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbfb2ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b9c2fae050 con 0x55b9c308d360 2024-09-10T08:54:41.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbfb2ff740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b9c309f7e0 con 0x55b9c308dc80 2024-09-10T08:54:41.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbfb2ff740 10 client.0 _ll_register_callbacks cb 0x55b9c2fee160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:41.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbf259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308d360 0x55b9c309c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbf259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308d360 0x55b9c309c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:51100/0 (socket says 172.21.15.172:51100) 2024-09-10T08:54:41.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbf259c640 1 -- 172.21.15.172:0/2029084469 learned_addr learned my addr 172.21.15.172:0/2029084469 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:41.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbf2d9d640 1 --2- 172.21.15.172:0/2029084469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c309ca90 0x55b9c3099cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbf1d9b640 1 --2- 172.21.15.172:0/2029084469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c308dc80 0x55b9c3099720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbf259c640 1 -- 172.21.15.172:0/2029084469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c308dc80 msgr2=0x55b9c3099720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:41.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbf259c640 1 --2- 172.21.15.172:0/2029084469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c308dc80 0x55b9c3099720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbf259c640 1 -- 172.21.15.172:0/2029084469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c309ca90 msgr2=0x55b9c3099cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:41.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbf259c640 1 --2- 172.21.15.172:0/2029084469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c309ca90 0x55b9c3099cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbf259c640 1 -- 172.21.15.172:0/2029084469 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c2d643c0 con 0x55b9c308d360 2024-09-10T08:54:41.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbf2d9d640 1 --2- 172.21.15.172:0/2029084469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c309ca90 0x55b9c3099cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:54:41.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbf1d9b640 1 --2- 172.21.15.172:0/2029084469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c308dc80 0x55b9c3099720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:54:41.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbf259c640 1 --2- 172.21.15.172:0/2029084469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308d360 0x55b9c309c470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4011670 tx=0x7fdbe40116a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65f1548df608cbdc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:41.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.103+0000 7fdbd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:54:41.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.104+0000 7fdbd9ffb640 1 -- 172.21.15.172:0/2029084469 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4028060 con 0x55b9c308d360 2024-09-10T08:54:41.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.104+0000 7fdbd9ffb640 1 -- 172.21.15.172:0/2029084469 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fdbe4011cd0 con 0x55b9c308d360 2024-09-10T08:54:41.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.104+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/2029084469 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe4015070 con 0x55b9c308d360 2024-09-10T08:54:41.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.104+0000 7fdbd9ffb640 1 -- 172.21.15.172:0/2029084469 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdbe402f050 con 0x55b9c308d360 2024-09-10T08:54:41.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.104+0000 7fdbfb2ff740 10 client.26981 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:41.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.104+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/2029084469 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b9c30900e0 con 0x55b9c308d360 2024-09-10T08:54:41.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.104+0000 7fdbd9ffb640 1 -- 172.21.15.172:0/2029084469 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbe4015070 con 0x55b9c308d360 2024-09-10T08:54:41.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.104+0000 7fdbd9ffb640 1 -- 172.21.15.172:0/2029084469 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fdbe4022070 con 0x55b9c308d360 2024-09-10T08:54:41.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.104+0000 7fdbfb2ff740 10 client.26981 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:41.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.104+0000 7fdbfb2ff740 10 client.26981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:54:41.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.104+0000 7fdbfb2ff740 20 client.26981 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:41.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.105+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/2029084469 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b9c3091490 0x55b9c30938f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.105+0000 7fdbfb2ff740 4 client.26981 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:54:41.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.105+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/2029084469 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9c3000850 con 0x55b9c3091490 2024-09-10T08:54:41.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.105+0000 7fdbf1d9b640 1 --2- 172.21.15.172:0/2029084469 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b9c3091490 0x55b9c30938f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.105+0000 7fdbf1d9b640 1 --2- 172.21.15.172:0/2029084469 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b9c3091490 0x55b9c30938f0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=709b165be9474658 server_cookie=6a9dabb4007273a3 in_seq=0 out_seq=0 2024-09-10T08:54:41.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.105+0000 7fdbd9ffb640 10 client.26981 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:41.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.106+0000 7fdbd9ffb640 1 -- 172.21.15.172:0/2029084469 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9c3000850 con 0x55b9c3091490 2024-09-10T08:54:41.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.106+0000 7fdbd9ffb640 10 client.26981 handle_command_reply: tid=0 2024-09-10T08:54:41.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.106+0000 7fdbfb2ff740 1 client.26981 shutdown 2024-09-10T08:54:41.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.106+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/2029084469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308d360 msgr2=0x55b9c309c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:41.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.106+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/2029084469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308d360 0x55b9c309c470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4011670 tx=0x7fdbe40116a0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.106+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/2029084469 shutdown_connections 2024-09-10T08:54:41.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.106+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/2029084469 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b9c3091490 0x55b9c30938f0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.106+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/2029084469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c309ca90 0x55b9c3099cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.106+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/2029084469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c308dc80 0x55b9c3099720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.106+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/2029084469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308d360 0x55b9c309c470 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.106+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/2029084469 >> 172.21.15.172:0/2029084469 conn(0x55b9c3089a30 msgr2=0x55b9c308f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:41.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.106+0000 7fdbd9ffb640 0 client.26981 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:41.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.106+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/2029084469 shutdown_connections 2024-09-10T08:54:41.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.106+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/2029084469 wait complete. 2024-09-10T08:54:41.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.106+0000 7fdbfb2ff740 20 client.26981 trim_cache size 0 max 16384 2024-09-10T08:54:41.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.126+0000 7fdbfb2ff740 1 Processor -- start 2024-09-10T08:54:41.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.126+0000 7fdbfb2ff740 1 -- start start 2024-09-10T08:54:41.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.126+0000 7fdbfb2ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308d360 0x55b9c3096980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.126+0000 7fdbfb2ff740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c308dc80 0x55b9c3096ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.126+0000 7fdbfb2ff740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c309ca90 0x55b9c309af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.126+0000 7fdbfb2ff740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41b5140 con 0x55b9c308dc80 2024-09-10T08:54:41.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.126+0000 7fdbfb2ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41b6c80 con 0x55b9c308d360 2024-09-10T08:54:41.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.126+0000 7fdbfb2ff740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41b5810 con 0x55b9c309ca90 2024-09-10T08:54:41.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.126+0000 7fdbf1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c308dc80 0x55b9c3096ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.126+0000 7fdbf1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c308dc80 0x55b9c3096ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53164/0 (socket says 172.21.15.172:53164) 2024-09-10T08:54:41.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.126+0000 7fdbf1d9b640 1 -- 172.21.15.172:0/1439771082 learned_addr learned my addr 172.21.15.172:0/1439771082 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:41.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.126+0000 7fdbf259c640 1 --2- 172.21.15.172:0/1439771082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308d360 0x55b9c3096980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.127+0000 7fdbf2d9d640 1 --2- 172.21.15.172:0/1439771082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c309ca90 0x55b9c309af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.127+0000 7fdbf1d9b640 1 -- 172.21.15.172:0/1439771082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c309ca90 msgr2=0x55b9c309af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:41.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.127+0000 7fdbf1d9b640 1 --2- 172.21.15.172:0/1439771082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c309ca90 0x55b9c309af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.127+0000 7fdbf1d9b640 1 -- 172.21.15.172:0/1439771082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308d360 msgr2=0x55b9c3096980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:41.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.127+0000 7fdbf1d9b640 1 --2- 172.21.15.172:0/1439771082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308d360 0x55b9c3096980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.127+0000 7fdbf1d9b640 1 -- 172.21.15.172:0/1439771082 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9c308dab0 con 0x55b9c308dc80 2024-09-10T08:54:41.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.127+0000 7fdbf259c640 1 --2- 172.21.15.172:0/1439771082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308d360 0x55b9c3096980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:54:41.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.127+0000 7fdbf1d9b640 1 --2- 172.21.15.172:0/1439771082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c308dc80 0x55b9c3096ec0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80cce70 tx=0x7fdbe80ccd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff14e76e6e7c493a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:41.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.127+0000 7fdbdbfff640 1 -- 172.21.15.172:0/1439771082 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80a4840 con 0x55b9c308dc80 2024-09-10T08:54:41.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.127+0000 7fdbdbfff640 1 -- 172.21.15.172:0/1439771082 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbe8099d20 con 0x55b9c308dc80 2024-09-10T08:54:41.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.127+0000 7fdbdbfff640 1 -- 172.21.15.172:0/1439771082 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80cfb80 con 0x55b9c308dc80 2024-09-10T08:54:41.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.128+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/1439771082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c308dc80 msgr2=0x55b9c3096ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:41.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.128+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/1439771082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c308dc80 0x55b9c3096ec0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80cce70 tx=0x7fdbe80ccd40 comp rx=0 tx=0).stop 2024-09-10T08:54:41.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.128+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/1439771082 shutdown_connections 2024-09-10T08:54:41.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.128+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/1439771082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c309ca90 0x55b9c309af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.128+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/1439771082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c308dc80 0x55b9c3096ec0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.128+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/1439771082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308d360 0x55b9c3096980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.128+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/1439771082 >> 172.21.15.172:0/1439771082 conn(0x55b9c3089a30 msgr2=0x55b9c308f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:41.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.128+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/1439771082 shutdown_connections 2024-09-10T08:54:41.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.128+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/1439771082 wait complete. 2024-09-10T08:54:41.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.128+0000 7fdbfb2ff740 1 -- start start 2024-09-10T08:54:41.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbfb2ff740 1 Processor -- start 2024-09-10T08:54:41.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbfb2ff740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c308d360 0x55b9c30af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbfb2ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308dc80 0x55b9c30af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbfb2ff740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c309ca90 0x55b9c30afc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbfb2ff740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41b65b0 con 0x55b9c309ca90 2024-09-10T08:54:41.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbfb2ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41b5ee0 con 0x55b9c308dc80 2024-09-10T08:54:41.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbfb2ff740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf41b3120 con 0x55b9c308d360 2024-09-10T08:54:41.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbfb2ff740 10 client.0 _ll_register_callbacks cb 0x7fdbe4004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:41.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbf2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c309ca90 0x55b9c30afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbf2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c309ca90 0x55b9c30afc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53180/0 (socket says 172.21.15.172:53180) 2024-09-10T08:54:41.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbf2d9d640 1 -- 172.21.15.172:0/3845050060 learned_addr learned my addr 172.21.15.172:0/3845050060 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:41.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbf259c640 1 --2- 172.21.15.172:0/3845050060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c308d360 0x55b9c30af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbf1d9b640 1 --2- 172.21.15.172:0/3845050060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308dc80 0x55b9c30af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbf2d9d640 1 -- 172.21.15.172:0/3845050060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c308d360 msgr2=0x55b9c30af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:41.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbf2d9d640 1 --2- 172.21.15.172:0/3845050060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c308d360 0x55b9c30af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbf2d9d640 1 -- 172.21.15.172:0/3845050060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308dc80 msgr2=0x55b9c30af6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:41.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbf2d9d640 1 --2- 172.21.15.172:0/3845050060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308dc80 0x55b9c30af6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.129+0000 7fdbf2d9d640 1 -- 172.21.15.172:0/3845050060 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9c31113e0 con 0x55b9c309ca90 2024-09-10T08:54:41.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.130+0000 7fdbf2d9d640 1 --2- 172.21.15.172:0/3845050060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c309ca90 0x55b9c30afc50 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009ca0 tx=0x7fdbec0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6408fe65863a6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:41.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.130+0000 7fdbd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:41.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.130+0000 7fdbd9ffb640 1 -- 172.21.15.172:0/3845050060 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec057060 con 0x55b9c309ca90 2024-09-10T08:54:41.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.130+0000 7fdbd9ffb640 1 -- 172.21.15.172:0/3845050060 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fdbec004450 con 0x55b9c309ca90 2024-09-10T08:54:41.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.130+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/3845050060 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe8099d20 con 0x55b9c309ca90 2024-09-10T08:54:41.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.130+0000 7fdbd9ffb640 1 -- 172.21.15.172:0/3845050060 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdbec05e050 con 0x55b9c309ca90 2024-09-10T08:54:41.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.130+0000 7fdbfb2ff740 10 client.17958 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:41.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.130+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/3845050060 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b9c31126f0 con 0x55b9c309ca90 2024-09-10T08:54:41.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.130+0000 7fdbd9ffb640 1 -- 172.21.15.172:0/3845050060 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbe8099d20 con 0x55b9c309ca90 2024-09-10T08:54:41.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.131+0000 7fdbd9ffb640 1 -- 172.21.15.172:0/3845050060 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fdbec051070 con 0x55b9c309ca90 2024-09-10T08:54:41.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.131+0000 7fdbfb2ff740 10 client.17958 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:41.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.131+0000 7fdbfb2ff740 10 client.17958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:54:41.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.131+0000 7fdbfb2ff740 20 client.17958 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:41.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.131+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/3845050060 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b9c3112e70 0x55b9c3115290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:41.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.131+0000 7fdbfb2ff740 4 client.17958 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:54:41.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.131+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/3845050060 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b9c2ffad60 con 0x55b9c3112e70 2024-09-10T08:54:41.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.131+0000 7fdbf259c640 1 --2- 172.21.15.172:0/3845050060 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b9c3112e70 0x55b9c3115290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:41.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.131+0000 7fdbd9ffb640 10 client.17958 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:41.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.131+0000 7fdbf259c640 1 --2- 172.21.15.172:0/3845050060 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b9c3112e70 0x55b9c3115290 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46425defc5420465 server_cookie=4f09e9dd6b8b4a33 in_seq=0 out_seq=0 2024-09-10T08:54:41.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.131+0000 7fdbd9ffb640 1 -- 172.21.15.172:0/3845050060 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b9c2ffad60 con 0x55b9c3112e70 2024-09-10T08:54:41.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.131+0000 7fdbd9ffb640 10 client.17958 handle_command_reply: tid=0 2024-09-10T08:54:41.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.131+0000 7fdbfb2ff740 1 client.17958 shutdown 2024-09-10T08:54:41.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.132+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/3845050060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c309ca90 msgr2=0x55b9c30afc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:41.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.132+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/3845050060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c309ca90 0x55b9c30afc50 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009ca0 tx=0x7fdbec0095c0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.132+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/3845050060 shutdown_connections 2024-09-10T08:54:41.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.132+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/3845050060 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b9c3112e70 0x55b9c3115290 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.132+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/3845050060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9c309ca90 0x55b9c30afc50 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.132+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/3845050060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b9c308dc80 0x55b9c30af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.132+0000 7fdbd9ffb640 0 client.17958 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:41.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.132+0000 7fdbfb2ff740 1 --2- 172.21.15.172:0/3845050060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b9c308d360 0x55b9c30af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.132+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/3845050060 >> 172.21.15.172:0/3845050060 conn(0x55b9c3089a30 msgr2=0x55b9c308ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:41.138 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.132+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/3845050060 shutdown_connections 2024-09-10T08:54:41.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.132+0000 7fdbfb2ff740 1 -- 172.21.15.172:0/3845050060 wait complete. 2024-09-10T08:54:41.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.132+0000 7fdbfb2ff740 20 client.17958 trim_cache size 0 max 16384 2024-09-10T08:54:41.139 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:54:41.139 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:54:41.139 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T08:54:41.139 INFO:teuthology.orchestra.run.smithi172.stdout: "1a9db4bb-ed7f-4f82-bbe2-5657b3f721a4": { 2024-09-10T08:54:41.140 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T08:54:41.140 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "1a9db4bb-ed7f-4f82-bbe2-5657b3f721a4", 2024-09-10T08:54:41.140 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T08:54:41.140 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:54:41.140 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:54:41.140 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:54:41.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.132+0000 7fdbf8f26640 1 -- 172.21.15.172:0/1958047305 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdbdc0a2b80 msgr2=0x7fdbdc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:41.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.132+0000 7fdbf8f26640 1 --2- 172.21.15.172:0/1958047305 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdbdc0a2b80 0x7fdbdc0a5040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdbec02fd10 tx=0x7fdbec005c50 comp rx=0 tx=0).stop 2024-09-10T08:54:41.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.132+0000 7fdbf8f26640 1 -- 172.21.15.172:0/1958047305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbf4102570 msgr2=0x7fdbf4198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:41.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.132+0000 7fdbf8f26640 1 --2- 172.21.15.172:0/1958047305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbf4102570 0x7fdbf4198910 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800e9f0 tx=0x7fdbe800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.133+0000 7fdbf8f26640 1 -- 172.21.15.172:0/1958047305 shutdown_connections 2024-09-10T08:54:41.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.133+0000 7fdbf8f26640 1 --2- 172.21.15.172:0/1958047305 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdbdc0a2b80 0x7fdbdc0a5040 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.133+0000 7fdbf8f26640 1 --2- 172.21.15.172:0/1958047305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbf4102ef0 0x7fdbf419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.133+0000 7fdbf8f26640 1 --2- 172.21.15.172:0/1958047305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbf4102570 0x7fdbf4198910 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.133+0000 7fdbf8f26640 1 --2- 172.21.15.172:0/1958047305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbf4100e20 0x7fdbf41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:41.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.133+0000 7fdbf8f26640 1 -- 172.21.15.172:0/1958047305 >> 172.21.15.172:0/1958047305 conn(0x7fdbf40fc7d0 msgr2=0x7fdbf40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:41.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.133+0000 7fdbf8f26640 1 -- 172.21.15.172:0/1958047305 shutdown_connections 2024-09-10T08:54:41.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:41.133+0000 7fdbf8f26640 1 -- 172.21.15.172:0/1958047305 wait complete. 2024-09-10T08:54:41.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:41 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T08:54:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:41 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:41 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:54:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:41 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:41.767 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:41 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T08:54:41.767 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:41 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:41.767 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:41 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:54:41.767 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:41 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:41.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:41 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T08:54:41.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:41 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:41.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:41 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:54:41.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:41 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:54:41.803 INFO:tasks.cephfs.filesystem:scrub status for tag:1a9db4bb-ed7f-4f82-bbe2-5657b3f721a4 - {'path': '/', 'tag': '1a9db4bb-ed7f-4f82-bbe2-5657b3f721a4', 'options': 'recursive,force'} 2024-09-10T08:54:41.941 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-10T08:54:41.941 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:54:42.223 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:54:42.629 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:42 smithi172 ceph-mon[30854]: pgmap v604: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 30 MiB/s wr, 33 op/s 2024-09-10T08:54:42.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.733+0000 7f56c3448640 1 -- 172.21.15.172:0/1707485821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc102570 msgr2=0x7f56bc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:42.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.733+0000 7f56c3448640 1 --2- 172.21.15.172:0/1707485821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc102570 0x7f56bc1029b0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009900 tx=0x7f56ac02a480 comp rx=0 tx=0).stop 2024-09-10T08:54:42.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.733+0000 7f56c3448640 1 -- 172.21.15.172:0/1707485821 shutdown_connections 2024-09-10T08:54:42.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.733+0000 7f56c3448640 1 --2- 172.21.15.172:0/1707485821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56bc102ef0 0x7f56bc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:42.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.733+0000 7f56c3448640 1 --2- 172.21.15.172:0/1707485821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc102570 0x7f56bc1029b0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:42.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.733+0000 7f56c3448640 1 --2- 172.21.15.172:0/1707485821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56bc100e20 0x7f56bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:42.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.733+0000 7f56c3448640 1 -- 172.21.15.172:0/1707485821 >> 172.21.15.172:0/1707485821 conn(0x7f56bc0fc7d0 msgr2=0x7f56bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:42.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.734+0000 7f56c3448640 1 -- 172.21.15.172:0/1707485821 shutdown_connections 2024-09-10T08:54:42.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.734+0000 7f56c3448640 1 -- 172.21.15.172:0/1707485821 wait complete. 2024-09-10T08:54:42.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.734+0000 7f56c3448640 1 Processor -- start 2024-09-10T08:54:42.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.734+0000 7f56c3448640 1 -- start start 2024-09-10T08:54:42.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.734+0000 7f56c3448640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56bc100e20 0x7f56bc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:42.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.734+0000 7f56c3448640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc102570 0x7f56bc19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:42.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.734+0000 7f56c3448640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56bc102ef0 0x7f56bc19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:42.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.734+0000 7f56c3448640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc1140f0 con 0x7f56bc102570 2024-09-10T08:54:42.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.734+0000 7f56c3448640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc113f70 con 0x7f56bc102ef0 2024-09-10T08:54:42.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.734+0000 7f56c3448640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc114270 con 0x7f56bc100e20 2024-09-10T08:54:42.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.735+0000 7f56c09e3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc102570 0x7f56bc19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:42.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.735+0000 7f56c11e4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56bc100e20 0x7f56bc19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:42.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.735+0000 7f56c11e4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56bc100e20 0x7f56bc19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:54180/0 (socket says 172.21.15.172:54180) 2024-09-10T08:54:42.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.735+0000 7f56c11e4640 1 -- 172.21.15.172:0/257070828 learned_addr learned my addr 172.21.15.172:0/257070828 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:42.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.735+0000 7f56c19e5640 1 --2- 172.21.15.172:0/257070828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56bc102ef0 0x7f56bc19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:42.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.735+0000 7f56c11e4640 1 -- 172.21.15.172:0/257070828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56bc102ef0 msgr2=0x7f56bc19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:42.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.735+0000 7f56c11e4640 1 --2- 172.21.15.172:0/257070828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56bc102ef0 0x7f56bc19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:42.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.735+0000 7f56c11e4640 1 -- 172.21.15.172:0/257070828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc102570 msgr2=0x7f56bc19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:42.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.735+0000 7f56c11e4640 1 --2- 172.21.15.172:0/257070828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc102570 0x7f56bc19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:42.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.735+0000 7f56c11e4640 1 -- 172.21.15.172:0/257070828 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc19f5c0 con 0x7f56bc100e20 2024-09-10T08:54:42.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.735+0000 7f56c11e4640 1 --2- 172.21.15.172:0/257070828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56bc100e20 0x7f56bc19a610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f56b800d950 tx=0x7f56b800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:42.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.735+0000 7f56b27fc640 1 -- 172.21.15.172:0/257070828 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8017070 con 0x7f56bc100e20 2024-09-10T08:54:42.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.735+0000 7f56b27fc640 1 -- 172.21.15.172:0/257070828 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56b8009c30 con 0x7f56bc100e20 2024-09-10T08:54:42.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.735+0000 7f56c3448640 1 -- 172.21.15.172:0/257070828 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56bc19f8b0 con 0x7f56bc100e20 2024-09-10T08:54:42.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.735+0000 7f56b27fc640 1 -- 172.21.15.172:0/257070828 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b800f620 con 0x7f56bc100e20 2024-09-10T08:54:42.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.736+0000 7f56c3448640 1 -- 172.21.15.172:0/257070828 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56bc1a7190 con 0x7f56bc100e20 2024-09-10T08:54:42.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.737+0000 7f56c3448640 1 -- 172.21.15.172:0/257070828 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56bc101200 con 0x7f56bc100e20 2024-09-10T08:54:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:42 smithi174 ceph-mon[34068]: pgmap v604: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 30 MiB/s wr, 33 op/s 2024-09-10T08:54:42.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.738+0000 7f56b27fc640 1 -- 172.21.15.172:0/257070828 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f56b80109a0 con 0x7f56bc100e20 2024-09-10T08:54:42.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.739+0000 7f56b27fc640 1 --2- 172.21.15.172:0/257070828 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56a80a2b70 0x7f56a80a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:42.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.739+0000 7f56c09e3640 1 --2- 172.21.15.172:0/257070828 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56a80a2b70 0x7f56a80a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:42.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.739+0000 7f56b27fc640 1 -- 172.21.15.172:0/257070828 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f56b80cef20 con 0x7f56bc100e20 2024-09-10T08:54:42.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.739+0000 7f56c09e3640 1 --2- 172.21.15.172:0/257070828 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56a80a2b70 0x7f56a80a5030 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009230 tx=0x7f56ac0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:42.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.742+0000 7f56b27fc640 1 -- 172.21.15.172:0/257070828 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56b8009de0 con 0x7f56bc100e20 2024-09-10T08:54:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:42 smithi191 ceph-mon[35314]: pgmap v604: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 30 MiB/s wr, 33 op/s 2024-09-10T08:54:43.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:42.999+0000 7f56c3448640 1 -- 172.21.15.172:0/257070828 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56bc109090 con 0x7f56bc100e20 2024-09-10T08:54:43.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:43.000+0000 7f56b27fc640 1 -- 172.21.15.172:0/257070828 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+12001 (secure 0 0 0) 0x7f56b809b4d0 con 0x7f56bc100e20 2024-09-10T08:54:43.001 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:54:43.001 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":37,"btime":"2024-09-10T08:52:08:313493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:52:07.658341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:54:43.001 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 37 2024-09-10T08:54:43.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:43.003+0000 7f56c3448640 1 -- 172.21.15.172:0/257070828 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56a80a2b70 msgr2=0x7f56a80a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:43.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:43.003+0000 7f56c3448640 1 --2- 172.21.15.172:0/257070828 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56a80a2b70 0x7f56a80a5030 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009230 tx=0x7f56ac0058d0 comp rx=0 tx=0).stop 2024-09-10T08:54:43.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:43.003+0000 7f56c3448640 1 -- 172.21.15.172:0/257070828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56bc100e20 msgr2=0x7f56bc19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:43.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:43.003+0000 7f56c3448640 1 --2- 172.21.15.172:0/257070828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56bc100e20 0x7f56bc19a610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f56b800d950 tx=0x7f56b800de10 comp rx=0 tx=0).stop 2024-09-10T08:54:43.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:43.003+0000 7f56c3448640 1 -- 172.21.15.172:0/257070828 shutdown_connections 2024-09-10T08:54:43.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:43.003+0000 7f56c3448640 1 --2- 172.21.15.172:0/257070828 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56a80a2b70 0x7f56a80a5030 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:43.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:43.003+0000 7f56c3448640 1 --2- 172.21.15.172:0/257070828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56bc102ef0 0x7f56bc19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:43.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:43.003+0000 7f56c3448640 1 --2- 172.21.15.172:0/257070828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc102570 0x7f56bc19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:43.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:43.003+0000 7f56c3448640 1 --2- 172.21.15.172:0/257070828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56bc100e20 0x7f56bc19a610 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:43.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:43.003+0000 7f56c3448640 1 -- 172.21.15.172:0/257070828 >> 172.21.15.172:0/257070828 conn(0x7f56bc0fc7d0 msgr2=0x7f56bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:43.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:43.003+0000 7f56c3448640 1 -- 172.21.15.172:0/257070828 shutdown_connections 2024-09-10T08:54:43.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:43.003+0000 7f56c3448640 1 -- 172.21.15.172:0/257070828 wait complete. 2024-09-10T08:54:43.669 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:43 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/257070828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:54:43.700 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-10T08:54:43.701 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-10T08:54:43.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:43 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/257070828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:54:43.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:43 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/257070828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:54:43.980 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:54:44.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.487+0000 7f981f395640 1 -- 172.21.15.172:0/2046014782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9818106f10 msgr2=0x7f981810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:44.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.487+0000 7f981f395640 1 --2- 172.21.15.172:0/2046014782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9818106f10 0x7f981810ea90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f9810009ca0 tx=0x7f981002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.487+0000 7f981f395640 1 -- 172.21.15.172:0/2046014782 shutdown_connections 2024-09-10T08:54:44.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.487+0000 7f981f395640 1 --2- 172.21.15.172:0/2046014782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9818106f10 0x7f981810ea90 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.487+0000 7f981f395640 1 --2- 172.21.15.172:0/2046014782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9818102790 0x7f98181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.487+0000 7f981f395640 1 --2- 172.21.15.172:0/2046014782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9818101de0 0x7f98181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.487+0000 7f981f395640 1 -- 172.21.15.172:0/2046014782 >> 172.21.15.172:0/2046014782 conn(0x7f98180fd880 msgr2=0x7f98180ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:44.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.487+0000 7f981f395640 1 -- 172.21.15.172:0/2046014782 shutdown_connections 2024-09-10T08:54:44.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.487+0000 7f981f395640 1 -- 172.21.15.172:0/2046014782 wait complete. 2024-09-10T08:54:44.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981f395640 1 Processor -- start 2024-09-10T08:54:44.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981f395640 1 -- start start 2024-09-10T08:54:44.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981f395640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9818101de0 0x7f9818198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981f395640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9818102790 0x7f98181987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981f395640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9818106f10 0x7f981819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981f395640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9818111e20 con 0x7f9818106f10 2024-09-10T08:54:44.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981f395640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9818111ca0 con 0x7f9818101de0 2024-09-10T08:54:44.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981f395640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9818111fa0 con 0x7f9818102790 2024-09-10T08:54:44.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981d131640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9818101de0 0x7f9818198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981d131640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9818101de0 0x7f9818198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:51148/0 (socket says 172.21.15.172:51148) 2024-09-10T08:54:44.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981c930640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9818102790 0x7f98181987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981d131640 1 -- 172.21.15.172:0/75760516 learned_addr learned my addr 172.21.15.172:0/75760516 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:44.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981d932640 1 --2- 172.21.15.172:0/75760516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9818106f10 0x7f981819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981d131640 1 -- 172.21.15.172:0/75760516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9818102790 msgr2=0x7f98181987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:44.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981d131640 1 --2- 172.21.15.172:0/75760516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9818102790 0x7f98181987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981d131640 1 -- 172.21.15.172:0/75760516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9818106f10 msgr2=0x7f981819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:44.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981d131640 1 --2- 172.21.15.172:0/75760516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9818106f10 0x7f981819cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.488+0000 7f981d131640 1 -- 172.21.15.172:0/75760516 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f981819d230 con 0x7f9818101de0 2024-09-10T08:54:44.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.489+0000 7f981d131640 1 --2- 172.21.15.172:0/75760516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9818101de0 0x7f9818198280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f981400e9f0 tx=0x7f981400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:44.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.489+0000 7f980e7fc640 1 -- 172.21.15.172:0/75760516 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981400cd80 con 0x7f9818101de0 2024-09-10T08:54:44.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.489+0000 7f981f395640 1 -- 172.21.15.172:0/75760516 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f981819d520 con 0x7f9818101de0 2024-09-10T08:54:44.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.489+0000 7f980e7fc640 1 -- 172.21.15.172:0/75760516 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98140108e0 con 0x7f9818101de0 2024-09-10T08:54:44.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.489+0000 7f980e7fc640 1 -- 172.21.15.172:0/75760516 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9814018920 con 0x7f9818101de0 2024-09-10T08:54:44.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.489+0000 7f981f395640 1 -- 172.21.15.172:0/75760516 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98181a4e00 con 0x7f9818101de0 2024-09-10T08:54:44.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.491+0000 7f980e7fc640 1 -- 172.21.15.172:0/75760516 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9814010a90 con 0x7f9818101de0 2024-09-10T08:54:44.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.492+0000 7f980e7fc640 1 --2- 172.21.15.172:0/75760516 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98040a2b80 0x7f98040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.492+0000 7f982176e740 1 Processor -- start 2024-09-10T08:54:44.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.492+0000 7f982176e740 1 -- start start 2024-09-10T08:54:44.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.492+0000 7f981c930640 1 --2- 172.21.15.172:0/75760516 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98040a2b80 0x7f98040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.492+0000 7f980e7fc640 1 -- 172.21.15.172:0/75760516 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f98140cf6d0 con 0x7f9818101de0 2024-09-10T08:54:44.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f982176e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e67ece0 0x56236e67d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f982176e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e67db40 0x56236e67df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f982176e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e690fe0 0x56236e6933d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f982176e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56236e5e32a0 con 0x56236e67ece0 2024-09-10T08:54:44.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f982176e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56236e58ab10 con 0x56236e690fe0 2024-09-10T08:54:44.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f982176e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56236e5df260 con 0x56236e67db40 2024-09-10T08:54:44.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f981d131640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e67ece0 0x56236e67d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f981d131640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e67ece0 0x56236e67d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53216/0 (socket says 172.21.15.172:53216) 2024-09-10T08:54:44.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f981d131640 1 -- 172.21.15.172:0/4176652463 learned_addr learned my addr 172.21.15.172:0/4176652463 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:44.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f981d932640 1 --2- 172.21.15.172:0/4176652463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e690fe0 0x56236e6933d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f981c930640 1 --2- 172.21.15.172:0/75760516 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98040a2b80 0x7f98040a5040 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f98180fdd80 tx=0x7f9808005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:44.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f981c930640 1 --2- 172.21.15.172:0/4176652463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e67db40 0x56236e67df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f981d131640 1 -- 172.21.15.172:0/4176652463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e67db40 msgr2=0x56236e67df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:44.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f981d131640 1 --2- 172.21.15.172:0/4176652463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e67db40 0x56236e67df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f981d131640 1 -- 172.21.15.172:0/4176652463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e690fe0 msgr2=0x56236e6933d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:44.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f981d131640 1 --2- 172.21.15.172:0/4176652463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e690fe0 0x56236e6933d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f981d131640 1 -- 172.21.15.172:0/4176652463 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56236e354b90 con 0x56236e67ece0 2024-09-10T08:54:44.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.493+0000 7f981d131640 1 --2- 172.21.15.172:0/4176652463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e67ece0 0x56236e67d600 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f98140985f0 tx=0x7f9814010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e378309a43e466 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:44.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.494+0000 7f98037fe640 1 -- 172.21.15.172:0/4176652463 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98140a47e0 con 0x56236e67ece0 2024-09-10T08:54:44.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.494+0000 7f98037fe640 1 -- 172.21.15.172:0/4176652463 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9814099f20 con 0x56236e67ece0 2024-09-10T08:54:44.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.494+0000 7f98037fe640 1 -- 172.21.15.172:0/4176652463 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9814097970 con 0x56236e67ece0 2024-09-10T08:54:44.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.494+0000 7f982176e740 1 -- 172.21.15.172:0/4176652463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e67ece0 msgr2=0x56236e67d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:44.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.494+0000 7f982176e740 1 --2- 172.21.15.172:0/4176652463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e67ece0 0x56236e67d600 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f98140985f0 tx=0x7f9814010f20 comp rx=0 tx=0).stop 2024-09-10T08:54:44.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.495+0000 7f982176e740 1 -- 172.21.15.172:0/4176652463 shutdown_connections 2024-09-10T08:54:44.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.495+0000 7f982176e740 1 --2- 172.21.15.172:0/4176652463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e690fe0 0x56236e6933d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.495+0000 7f982176e740 1 --2- 172.21.15.172:0/4176652463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e67db40 0x56236e67df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.495+0000 7f982176e740 1 --2- 172.21.15.172:0/4176652463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e67ece0 0x56236e67d600 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.495+0000 7f982176e740 1 -- 172.21.15.172:0/4176652463 >> 172.21.15.172:0/4176652463 conn(0x56236e679cf0 msgr2=0x56236e67a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:44.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.495+0000 7f982176e740 1 -- 172.21.15.172:0/4176652463 shutdown_connections 2024-09-10T08:54:44.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.495+0000 7f982176e740 1 -- 172.21.15.172:0/4176652463 wait complete. 2024-09-10T08:54:44.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.495+0000 7f982176e740 1 -- start start 2024-09-10T08:54:44.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.495+0000 7f982176e740 1 Processor -- start 2024-09-10T08:54:44.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f982176e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e67db40 0x56236e681390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f982176e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67ece0 0x56236e6818d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f982176e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e690fe0 0x56236e681e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f982176e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56236e693cb0 con 0x56236e690fe0 2024-09-10T08:54:44.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f982176e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56236e56b6e0 con 0x56236e67ece0 2024-09-10T08:54:44.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f982176e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56236e693e30 con 0x56236e67db40 2024-09-10T08:54:44.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f982176e740 10 client.0 _ll_register_callbacks cb 0x56236e58fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:44.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f981c930640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67ece0 0x56236e6818d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f981c930640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67ece0 0x56236e6818d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:51176/0 (socket says 172.21.15.172:51176) 2024-09-10T08:54:44.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f981c930640 1 -- 172.21.15.172:0/4202521385 learned_addr learned my addr 172.21.15.172:0/4202521385 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:44.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f981d932640 1 --2- 172.21.15.172:0/4202521385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e690fe0 0x56236e681e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f981d131640 1 --2- 172.21.15.172:0/4202521385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e67db40 0x56236e681390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f981c930640 1 -- 172.21.15.172:0/4202521385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e67db40 msgr2=0x56236e681390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:44.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f981c930640 1 --2- 172.21.15.172:0/4202521385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e67db40 0x56236e681390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f981c930640 1 -- 172.21.15.172:0/4202521385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e690fe0 msgr2=0x56236e681e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:44.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f981c930640 1 --2- 172.21.15.172:0/4202521385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e690fe0 0x56236e681e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f981c930640 1 -- 172.21.15.172:0/4202521385 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56236e69d210 con 0x56236e67ece0 2024-09-10T08:54:44.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f981c930640 1 --2- 172.21.15.172:0/4202521385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67ece0 0x56236e6818d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f98080045e0 tx=0x7f9808004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22a4501a187d1c59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:44.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.496+0000 7f9800ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:54:44.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.497+0000 7f9800ff9640 1 -- 172.21.15.172:0/4202521385 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808005220 con 0x56236e67ece0 2024-09-10T08:54:44.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.497+0000 7f982176e740 1 -- 172.21.15.172:0/4202521385 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9814099f20 con 0x56236e67ece0 2024-09-10T08:54:44.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.497+0000 7f9800ff9640 1 -- 172.21.15.172:0/4202521385 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f98080053c0 con 0x56236e67ece0 2024-09-10T08:54:44.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.497+0000 7f9800ff9640 1 -- 172.21.15.172:0/4202521385 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f98080055c0 con 0x56236e67ece0 2024-09-10T08:54:44.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.497+0000 7f982176e740 10 client.27002 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:44.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.497+0000 7f982176e740 1 -- 172.21.15.172:0/4202521385 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56236e69fc50 con 0x56236e67ece0 2024-09-10T08:54:44.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.497+0000 7f9800ff9640 1 -- 172.21.15.172:0/4202521385 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9814099f20 con 0x56236e67ece0 2024-09-10T08:54:44.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.498+0000 7f9800ff9640 1 -- 172.21.15.172:0/4202521385 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f980801fbb0 con 0x56236e67ece0 2024-09-10T08:54:44.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.498+0000 7f982176e740 10 client.27002 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:44.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.498+0000 7f982176e740 10 client.27002 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:54:44.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.498+0000 7f982176e740 20 client.27002 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:44.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.498+0000 7f982176e740 1 --2- 172.21.15.172:0/4202521385 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56236e6a14d0 0x56236e6a3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.498+0000 7f982176e740 4 client.27002 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:54:44.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.498+0000 7f982176e740 1 -- 172.21.15.172:0/4202521385 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56236e5f09d0 con 0x56236e6a14d0 2024-09-10T08:54:44.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.498+0000 7f981d131640 1 --2- 172.21.15.172:0/4202521385 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56236e6a14d0 0x56236e6a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.498+0000 7f981d131640 1 --2- 172.21.15.172:0/4202521385 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56236e6a14d0 0x56236e6a3910 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4ea8e393296fdd4 server_cookie=e306c2bc159e395a in_seq=0 out_seq=0 2024-09-10T08:54:44.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.498+0000 7f9800ff9640 10 client.27002 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:44.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.499+0000 7f9800ff9640 1 -- 172.21.15.172:0/4202521385 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56236e5f09d0 con 0x56236e6a14d0 2024-09-10T08:54:44.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.499+0000 7f9800ff9640 10 client.27002 handle_command_reply: tid=0 2024-09-10T08:54:44.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.499+0000 7f982176e740 1 client.27002 shutdown 2024-09-10T08:54:44.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.499+0000 7f982176e740 1 -- 172.21.15.172:0/4202521385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67ece0 msgr2=0x56236e6818d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:44.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.499+0000 7f982176e740 1 --2- 172.21.15.172:0/4202521385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67ece0 0x56236e6818d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f98080045e0 tx=0x7f9808004730 comp rx=0 tx=0).stop 2024-09-10T08:54:44.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.499+0000 7f982176e740 1 -- 172.21.15.172:0/4202521385 shutdown_connections 2024-09-10T08:54:44.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.499+0000 7f982176e740 1 --2- 172.21.15.172:0/4202521385 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56236e6a14d0 0x56236e6a3910 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.499+0000 7f982176e740 1 --2- 172.21.15.172:0/4202521385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e690fe0 0x56236e681e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.499+0000 7f982176e740 1 --2- 172.21.15.172:0/4202521385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67ece0 0x56236e6818d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.499+0000 7f982176e740 1 --2- 172.21.15.172:0/4202521385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e67db40 0x56236e681390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.499+0000 7f9800ff9640 0 client.27002 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:44.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.499+0000 7f982176e740 1 -- 172.21.15.172:0/4202521385 >> 172.21.15.172:0/4202521385 conn(0x56236e679cf0 msgr2=0x56236e67fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:44.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.500+0000 7f982176e740 1 -- 172.21.15.172:0/4202521385 shutdown_connections 2024-09-10T08:54:44.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.500+0000 7f982176e740 1 -- 172.21.15.172:0/4202521385 wait complete. 2024-09-10T08:54:44.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.500+0000 7f982176e740 20 client.27002 trim_cache size 0 max 16384 2024-09-10T08:54:44.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.526+0000 7f982176e740 1 Processor -- start 2024-09-10T08:54:44.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.526+0000 7f982176e740 1 -- start start 2024-09-10T08:54:44.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.527+0000 7f982176e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67db40 0x56236e683260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.527+0000 7f982176e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e67ece0 0x56236e679660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.527+0000 7f982176e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e690fe0 0x56236e6a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.527+0000 7f982176e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f98181ac8b0 con 0x56236e67ece0 2024-09-10T08:54:44.532 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.527+0000 7f982176e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98181ae3f0 con 0x56236e67db40 2024-09-10T08:54:44.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.527+0000 7f982176e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f98181acf80 con 0x56236e690fe0 2024-09-10T08:54:44.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.527+0000 7f981c930640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e67ece0 0x56236e679660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.527+0000 7f981c930640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e67ece0 0x56236e679660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53226/0 (socket says 172.21.15.172:53226) 2024-09-10T08:54:44.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.527+0000 7f981c930640 1 -- 172.21.15.172:0/262038085 learned_addr learned my addr 172.21.15.172:0/262038085 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:44.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.527+0000 7f981d131640 1 --2- 172.21.15.172:0/262038085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67db40 0x56236e683260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.527+0000 7f981d932640 1 --2- 172.21.15.172:0/262038085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e690fe0 0x56236e6a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.527+0000 7f981c930640 1 -- 172.21.15.172:0/262038085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e690fe0 msgr2=0x56236e6a91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:44.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.527+0000 7f981c930640 1 --2- 172.21.15.172:0/262038085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e690fe0 0x56236e6a91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.527+0000 7f981c930640 1 -- 172.21.15.172:0/262038085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67db40 msgr2=0x56236e683260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:44.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.527+0000 7f981c930640 1 --2- 172.21.15.172:0/262038085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67db40 0x56236e683260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.527+0000 7f981c930640 1 -- 172.21.15.172:0/262038085 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56236e675260 con 0x56236e67ece0 2024-09-10T08:54:44.533 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.527+0000 7f981c930640 1 --2- 172.21.15.172:0/262038085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e67ece0 0x56236e679660 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f9808009930 tx=0x7f9808011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98cd5f82c3400d26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:44.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.528+0000 7f9802ffd640 1 -- 172.21.15.172:0/262038085 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808039070 con 0x56236e67ece0 2024-09-10T08:54:44.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.528+0000 7f9802ffd640 1 -- 172.21.15.172:0/262038085 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9808023070 con 0x56236e67ece0 2024-09-10T08:54:44.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.528+0000 7f9802ffd640 1 -- 172.21.15.172:0/262038085 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980801fc10 con 0x56236e67ece0 2024-09-10T08:54:44.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.528+0000 7f982176e740 1 -- 172.21.15.172:0/262038085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e67ece0 msgr2=0x56236e679660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:44.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.528+0000 7f982176e740 1 --2- 172.21.15.172:0/262038085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e67ece0 0x56236e679660 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f9808009930 tx=0x7f9808011c90 comp rx=0 tx=0).stop 2024-09-10T08:54:44.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.529+0000 7f982176e740 1 -- 172.21.15.172:0/262038085 shutdown_connections 2024-09-10T08:54:44.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.529+0000 7f982176e740 1 --2- 172.21.15.172:0/262038085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e690fe0 0x56236e6a91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.529+0000 7f982176e740 1 --2- 172.21.15.172:0/262038085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e67ece0 0x56236e679660 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.529+0000 7f982176e740 1 --2- 172.21.15.172:0/262038085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67db40 0x56236e683260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.529+0000 7f982176e740 1 -- 172.21.15.172:0/262038085 >> 172.21.15.172:0/262038085 conn(0x56236e679cf0 msgr2=0x56236e67fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:44.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.529+0000 7f982176e740 1 -- 172.21.15.172:0/262038085 shutdown_connections 2024-09-10T08:54:44.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.529+0000 7f982176e740 1 -- 172.21.15.172:0/262038085 wait complete. 2024-09-10T08:54:44.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.529+0000 7f982176e740 1 -- start start 2024-09-10T08:54:44.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.529+0000 7f982176e740 1 Processor -- start 2024-09-10T08:54:44.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.529+0000 7f982176e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67db40 0x56236e6998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.529+0000 7f982176e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e67ece0 0x56236e695c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.529+0000 7f982176e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e690fe0 0x56236e696180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f982176e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f98181add20 con 0x56236e690fe0 2024-09-10T08:54:44.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f982176e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98181ad650 con 0x56236e67db40 2024-09-10T08:54:44.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f982176e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f98181aa890 con 0x56236e67ece0 2024-09-10T08:54:44.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f981d932640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e690fe0 0x56236e696180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f982176e740 10 client.0 _ll_register_callbacks cb 0x7f98080044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:44.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f981c930640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e67ece0 0x56236e695c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f981d932640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e690fe0 0x56236e696180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53234/0 (socket says 172.21.15.172:53234) 2024-09-10T08:54:44.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f981c930640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e67ece0 0x56236e695c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:54222/0 (socket says 172.21.15.172:54222) 2024-09-10T08:54:44.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f981d131640 1 --2- 172.21.15.172:0/2384055976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67db40 0x56236e6998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f981d932640 1 -- 172.21.15.172:0/2384055976 learned_addr learned my addr 172.21.15.172:0/2384055976 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:44.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f981c930640 1 -- 172.21.15.172:0/2384055976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67db40 msgr2=0x56236e6998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:44.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f981c930640 1 --2- 172.21.15.172:0/2384055976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67db40 0x56236e6998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f981c930640 1 -- 172.21.15.172:0/2384055976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e690fe0 msgr2=0x56236e696180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:44.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f981c930640 1 --2- 172.21.15.172:0/2384055976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e690fe0 0x56236e696180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f981c930640 1 -- 172.21.15.172:0/2384055976 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56236e693e20 con 0x56236e67ece0 2024-09-10T08:54:44.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f981d932640 1 --2- 172.21.15.172:0/2384055976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e690fe0 0x56236e696180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:44.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f981d131640 1 --2- 172.21.15.172:0/2384055976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67db40 0x56236e6998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:54:44.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f981c930640 1 --2- 172.21.15.172:0/2384055976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e67ece0 0x56236e695c00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9808004480 tx=0x7f9808003fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=601871e1c958fa7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:44.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f9800ff9640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:54:44.536 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.530+0000 7f9800ff9640 1 -- 172.21.15.172:0/2384055976 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808039070 con 0x56236e67ece0 2024-09-10T08:54:44.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.531+0000 7f9800ff9640 1 -- 172.21.15.172:0/2384055976 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f980801f810 con 0x56236e67ece0 2024-09-10T08:54:44.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.531+0000 7f982176e740 1 -- 172.21.15.172:0/2384055976 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9808023070 con 0x56236e67ece0 2024-09-10T08:54:44.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.531+0000 7f9800ff9640 1 -- 172.21.15.172:0/2384055976 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f980801fa10 con 0x56236e67ece0 2024-09-10T08:54:44.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.531+0000 7f982176e740 10 client.26944 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:44.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.531+0000 7f982176e740 1 -- 172.21.15.172:0/2384055976 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56236e67e550 con 0x56236e67ece0 2024-09-10T08:54:44.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.531+0000 7f9800ff9640 1 -- 172.21.15.172:0/2384055976 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9808023070 con 0x56236e67ece0 2024-09-10T08:54:44.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.531+0000 7f9800ff9640 1 -- 172.21.15.172:0/2384055976 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f980801e430 con 0x56236e67ece0 2024-09-10T08:54:44.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.531+0000 7f982176e740 10 client.26944 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:44.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.531+0000 7f982176e740 10 client.26944 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:54:44.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.531+0000 7f982176e740 20 client.26944 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:44.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.531+0000 7f982176e740 1 --2- 172.21.15.172:0/2384055976 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56236e70d570 0x56236e70f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:44.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.531+0000 7f982176e740 4 client.26944 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-10T08:54:44.537 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.531+0000 7f982176e740 1 -- 172.21.15.172:0/2384055976 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x56236e5ead20 con 0x56236e70d570 2024-09-10T08:54:44.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.532+0000 7f981d131640 1 --2- 172.21.15.172:0/2384055976 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56236e70d570 0x56236e70f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:44.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.532+0000 7f9800ff9640 10 client.26944 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:44.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.532+0000 7f981d131640 1 --2- 172.21.15.172:0/2384055976 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56236e70d570 0x56236e70f990 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b6e56b60444e275 server_cookie=2f727052c68fa4f6 in_seq=0 out_seq=0 2024-09-10T08:54:44.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.603+0000 7f9800ff9640 1 -- 172.21.15.172:0/2384055976 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56236e5ead20 con 0x56236e70d570 2024-09-10T08:54:44.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.603+0000 7f9800ff9640 10 client.26944 handle_command_reply: tid=0 2024-09-10T08:54:44.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.603+0000 7f982176e740 1 client.26944 shutdown 2024-09-10T08:54:44.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.603+0000 7f982176e740 1 -- 172.21.15.172:0/2384055976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e67ece0 msgr2=0x56236e695c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:44.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.603+0000 7f982176e740 1 --2- 172.21.15.172:0/2384055976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e67ece0 0x56236e695c00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9808004480 tx=0x7f9808003fb0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.603+0000 7f982176e740 1 -- 172.21.15.172:0/2384055976 shutdown_connections 2024-09-10T08:54:44.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.603+0000 7f982176e740 1 --2- 172.21.15.172:0/2384055976 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56236e70d570 0x56236e70f990 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.603+0000 7f982176e740 1 --2- 172.21.15.172:0/2384055976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56236e690fe0 0x56236e696180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.603+0000 7f982176e740 1 --2- 172.21.15.172:0/2384055976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56236e67ece0 0x56236e695c00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.603+0000 7f982176e740 1 --2- 172.21.15.172:0/2384055976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56236e67db40 0x56236e6998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.603+0000 7f9800ff9640 0 client.26944 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:44.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.603+0000 7f982176e740 1 -- 172.21.15.172:0/2384055976 >> 172.21.15.172:0/2384055976 conn(0x56236e679cf0 msgr2=0x56236e67f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:44.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.604+0000 7f982176e740 1 -- 172.21.15.172:0/2384055976 shutdown_connections 2024-09-10T08:54:44.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.604+0000 7f982176e740 1 -- 172.21.15.172:0/2384055976 wait complete. 2024-09-10T08:54:44.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.604+0000 7f982176e740 20 client.26944 trim_cache size 0 max 16384 2024-09-10T08:54:44.702 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:54:44.702 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 37, 2024-09-10T08:54:44.702 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:54:44.702 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 26, 2024-09-10T08:54:44.703 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:54:44.703 INFO:teuthology.orchestra.run.smithi172.stdout: "c1e21f5b": { 2024-09-10T08:54:44.703 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 26, 2024-09-10T08:54:44.703 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:54:44.703 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:54:44.703 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:54:44.703 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:54:44.703 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:54:44.703 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:54:44.703 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 133.4, 2024-09-10T08:54:44.703 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:54:44.704 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:54:44.704 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:54:44.704 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:54:44.704 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:54:44.704 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:54:44.704 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:54:44.704 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:54:44.704 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:54:44.704 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:54:44.704 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:54:44.704 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:54:44.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.604+0000 7f981f395640 1 -- 172.21.15.172:0/75760516 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98040a2b80 msgr2=0x7f98040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:44.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.604+0000 7f981f395640 1 --2- 172.21.15.172:0/75760516 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98040a2b80 0x7f98040a5040 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f98180fdd80 tx=0x7f9808005f90 comp rx=0 tx=0).stop 2024-09-10T08:54:44.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.604+0000 7f981f395640 1 -- 172.21.15.172:0/75760516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9818101de0 msgr2=0x7f9818198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:44.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.604+0000 7f981f395640 1 --2- 172.21.15.172:0/75760516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9818101de0 0x7f9818198280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f981400e9f0 tx=0x7f981400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.604+0000 7f981f395640 1 -- 172.21.15.172:0/75760516 shutdown_connections 2024-09-10T08:54:44.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.604+0000 7f981f395640 1 --2- 172.21.15.172:0/75760516 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98040a2b80 0x7f98040a5040 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.604+0000 7f981f395640 1 --2- 172.21.15.172:0/75760516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9818106f10 0x7f981819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.604+0000 7f981f395640 1 --2- 172.21.15.172:0/75760516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9818102790 0x7f98181987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.604+0000 7f981f395640 1 --2- 172.21.15.172:0/75760516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9818101de0 0x7f9818198280 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:44.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.604+0000 7f981f395640 1 -- 172.21.15.172:0/75760516 >> 172.21.15.172:0/75760516 conn(0x7f98180fd880 msgr2=0x7f98180feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:44.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.604+0000 7f981f395640 1 -- 172.21.15.172:0/75760516 shutdown_connections 2024-09-10T08:54:44.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:44.604+0000 7f981f395640 1 -- 172.21.15.172:0/75760516 wait complete. 2024-09-10T08:54:44.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:44 smithi174 ceph-mon[34068]: pgmap v605: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 33 op/s 2024-09-10T08:54:44.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:44 smithi191 ceph-mon[35314]: pgmap v605: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 33 op/s 2024-09-10T08:54:44.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:44 smithi172 ceph-mon[30854]: pgmap v605: 129 pgs: 129 active+clean; 7.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 33 op/s 2024-09-10T08:54:45.313 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c1e21f5b', quiesce took 3.4 seconds. Will release after: 10.0 2024-09-10T08:54:45.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:45 smithi172 ceph-mon[30854]: pgmap v606: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 MiB/s wr, 50 op/s 2024-09-10T08:54:45.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:45 smithi174 ceph-mon[34068]: pgmap v606: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 MiB/s wr, 50 op/s 2024-09-10T08:54:45.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:45 smithi191 ceph-mon[35314]: pgmap v606: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 MiB/s wr, 50 op/s 2024-09-10T08:54:48.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:48 smithi174 ceph-mon[34068]: pgmap v607: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 44 op/s 2024-09-10T08:54:48.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:48 smithi191 ceph-mon[35314]: pgmap v607: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 44 op/s 2024-09-10T08:54:48.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:48 smithi172 ceph-mon[30854]: pgmap v607: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 44 op/s 2024-09-10T08:54:50.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:50 smithi172 ceph-mon[30854]: pgmap v608: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 43 op/s 2024-09-10T08:54:50.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:50 smithi174 ceph-mon[34068]: pgmap v608: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 43 op/s 2024-09-10T08:54:50.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:50 smithi191 ceph-mon[35314]: pgmap v608: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 21 MiB/s wr, 43 op/s 2024-09-10T08:54:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.290+0000 7f53c38d0640 1 -- 172.21.15.172:0/3713080340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53bc104f10 msgr2=0x7f53bc107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.290+0000 7f53c38d0640 1 --2- 172.21.15.172:0/3713080340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53bc104f10 0x7f53bc107300 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009900 tx=0x7f53ac02a410 comp rx=0 tx=0).stop 2024-09-10T08:54:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.291+0000 7f53c38d0640 1 -- 172.21.15.172:0/3713080340 shutdown_connections 2024-09-10T08:54:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.291+0000 7f53c38d0640 1 --2- 172.21.15.172:0/3713080340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53bc107840 0x7f53bc109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.291+0000 7f53c38d0640 1 --2- 172.21.15.172:0/3713080340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53bc104f10 0x7f53bc107300 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.291+0000 7f53c38d0640 1 --2- 172.21.15.172:0/3713080340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53bc1025e0 0x7f53bc1049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.291+0000 7f53c38d0640 1 -- 172.21.15.172:0/3713080340 >> 172.21.15.172:0/3713080340 conn(0x7f53bc0fc1e0 msgr2=0x7f53bc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.291+0000 7f53c38d0640 1 -- 172.21.15.172:0/3713080340 shutdown_connections 2024-09-10T08:54:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.291+0000 7f53c38d0640 1 -- 172.21.15.172:0/3713080340 wait complete. 2024-09-10T08:54:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.291+0000 7f53c38d0640 1 Processor -- start 2024-09-10T08:54:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.291+0000 7f53c38d0640 1 -- start start 2024-09-10T08:54:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.291+0000 7f53c38d0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53bc1025e0 0x7f53bc199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.291+0000 7f53c38d0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53bc104f10 0x7f53bc19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.291+0000 7f53c38d0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53bc107840 0x7f53bc1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.291+0000 7f53c38d0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc10d6d0 con 0x7f53bc1025e0 2024-09-10T08:54:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.291+0000 7f53c38d0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc10d550 con 0x7f53bc104f10 2024-09-10T08:54:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.291+0000 7f53c38d0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc10d850 con 0x7f53bc107840 2024-09-10T08:54:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.292+0000 7f53c166c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53bc1025e0 0x7f53bc199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.292+0000 7f53c166c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53bc1025e0 0x7f53bc199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47260/0 (socket says 172.21.15.172:47260) 2024-09-10T08:54:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.292+0000 7f53c166c640 1 -- 172.21.15.172:0/1233685668 learned_addr learned my addr 172.21.15.172:0/1233685668 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.292+0000 7f53c0e6b640 1 --2- 172.21.15.172:0/1233685668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53bc104f10 0x7f53bc19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.292+0000 7f53c1e6d640 1 --2- 172.21.15.172:0/1233685668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53bc107840 0x7f53bc1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.292+0000 7f53c166c640 1 -- 172.21.15.172:0/1233685668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53bc107840 msgr2=0x7f53bc1a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.292+0000 7f53c166c640 1 --2- 172.21.15.172:0/1233685668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53bc107840 0x7f53bc1a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.292+0000 7f53c166c640 1 -- 172.21.15.172:0/1233685668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53bc104f10 msgr2=0x7f53bc19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.292+0000 7f53c166c640 1 --2- 172.21.15.172:0/1233685668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53bc104f10 0x7f53bc19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.292+0000 7f53c166c640 1 -- 172.21.15.172:0/1233685668 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53bc1a1bd0 con 0x7f53bc1025e0 2024-09-10T08:54:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.292+0000 7f53c166c640 1 --2- 172.21.15.172:0/1233685668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53bc1025e0 0x7f53bc199fb0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f53b800e9f0 tx=0x7f53b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.292+0000 7f53b27fc640 1 -- 172.21.15.172:0/1233685668 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b800cd80 con 0x7f53bc1025e0 2024-09-10T08:54:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.292+0000 7f53b27fc640 1 -- 172.21.15.172:0/1233685668 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53b80108e0 con 0x7f53bc1025e0 2024-09-10T08:54:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.292+0000 7f53c38d0640 1 -- 172.21.15.172:0/1233685668 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53bc1a1ec0 con 0x7f53bc1025e0 2024-09-10T08:54:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.292+0000 7f53b27fc640 1 -- 172.21.15.172:0/1233685668 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b8018920 con 0x7f53bc1025e0 2024-09-10T08:54:51.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.292+0000 7f53c38d0640 1 -- 172.21.15.172:0/1233685668 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53bc1a2320 con 0x7f53bc1025e0 2024-09-10T08:54:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.295+0000 7f53b27fc640 1 -- 172.21.15.172:0/1233685668 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f53b8010a90 con 0x7f53bc1025e0 2024-09-10T08:54:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.295+0000 7f53c6665740 1 Processor -- start 2024-09-10T08:54:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.295+0000 7f53c6665740 1 -- start start 2024-09-10T08:54:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53b27fc640 1 --2- 172.21.15.172:0/1233685668 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53a80a2c30 0x7f53a80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c6665740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd469a0 0x55bbafd46da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c6665740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd472e0 0x55bbafd4bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c6665740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd4c370 0x55bbafd4e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c6665740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bbafcaa2f0 con 0x55bbafd4c370 2024-09-10T08:54:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c6665740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bbafc52270 con 0x55bbafd469a0 2024-09-10T08:54:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c6665740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bbafca62b0 con 0x55bbafd472e0 2024-09-10T08:54:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c1e6d640 1 --2- 172.21.15.172:0/1233685668 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53a80a2c30 0x7f53a80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53b27fc640 1 -- 172.21.15.172:0/1233685668 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f53b80cf740 con 0x7f53bc1025e0 2024-09-10T08:54:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c1e6d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd4c370 0x55bbafd4e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c1e6d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd4c370 0x55bbafd4e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47272/0 (socket says 172.21.15.172:47272) 2024-09-10T08:54:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c1e6d640 1 -- 172.21.15.172:0/1546005483 learned_addr learned my addr 172.21.15.172:0/1546005483 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c166c640 1 --2- 172.21.15.172:0/1546005483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd469a0 0x55bbafd46da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c0e6b640 1 --2- 172.21.15.172:0/1546005483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd472e0 0x55bbafd4bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c166c640 1 -- 172.21.15.172:0/1546005483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd472e0 msgr2=0x55bbafd4bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c166c640 1 --2- 172.21.15.172:0/1546005483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd472e0 0x55bbafd4bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c166c640 1 -- 172.21.15.172:0/1546005483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd4c370 msgr2=0x55bbafd4e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c166c640 1 --2- 172.21.15.172:0/1546005483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd4c370 0x55bbafd4e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c166c640 1 -- 172.21.15.172:0/1546005483 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbafc51340 con 0x55bbafd469a0 2024-09-10T08:54:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c1e6d640 1 --2- 172.21.15.172:0/1233685668 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53a80a2c30 0x7f53a80a50f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009fd0 tx=0x7f53b4009d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.296+0000 7f53c166c640 1 --2- 172.21.15.172:0/1546005483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd469a0 0x55bbafd46da0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f53b80cd250 tx=0x7f53b8073cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7694b442df3c263f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.297+0000 7f53a77fe640 1 -- 172.21.15.172:0/1546005483 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b8070180 con 0x55bbafd469a0 2024-09-10T08:54:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.297+0000 7f53a77fe640 1 -- 172.21.15.172:0/1546005483 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53b809bc40 con 0x55bbafd469a0 2024-09-10T08:54:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.297+0000 7f53a77fe640 1 -- 172.21.15.172:0/1546005483 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b80982b0 con 0x55bbafd469a0 2024-09-10T08:54:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.297+0000 7f53c6665740 1 -- 172.21.15.172:0/1546005483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd469a0 msgr2=0x55bbafd46da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.297+0000 7f53c6665740 1 --2- 172.21.15.172:0/1546005483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd469a0 0x55bbafd46da0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f53b80cd250 tx=0x7f53b8073cd0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.297+0000 7f53c6665740 1 -- 172.21.15.172:0/1546005483 shutdown_connections 2024-09-10T08:54:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.297+0000 7f53c6665740 1 --2- 172.21.15.172:0/1546005483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd4c370 0x55bbafd4e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.297+0000 7f53c6665740 1 --2- 172.21.15.172:0/1546005483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd472e0 0x55bbafd4bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.297+0000 7f53c6665740 1 --2- 172.21.15.172:0/1546005483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd469a0 0x55bbafd46da0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.297+0000 7f53c6665740 1 -- 172.21.15.172:0/1546005483 >> 172.21.15.172:0/1546005483 conn(0x55bbafd42010 msgr2=0x55bbafd42410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.297+0000 7f53c6665740 1 -- 172.21.15.172:0/1546005483 shutdown_connections 2024-09-10T08:54:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.297+0000 7f53c6665740 1 -- 172.21.15.172:0/1546005483 wait complete. 2024-09-10T08:54:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.298+0000 7f53c6665740 1 -- start start 2024-09-10T08:54:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.298+0000 7f53c6665740 1 Processor -- start 2024-09-10T08:54:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.298+0000 7f53c6665740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd469a0 0x55bbafd50b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.298+0000 7f53c6665740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd472e0 0x55bbafd51040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.298+0000 7f53c6665740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd4c370 0x55bbafd58010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.298+0000 7f53c6665740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bbafd4f2e0 con 0x55bbafd469a0 2024-09-10T08:54:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.298+0000 7f53c6665740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bbafc33290 con 0x55bbafd472e0 2024-09-10T08:54:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.298+0000 7f53c6665740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bbafd4f460 con 0x55bbafd4c370 2024-09-10T08:54:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.298+0000 7f53c6665740 10 client.0 _ll_register_callbacks cb 0x55bbafc64140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.298+0000 7f53c166c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd469a0 0x55bbafd50b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.298+0000 7f53c166c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd469a0 0x55bbafd50b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47288/0 (socket says 172.21.15.172:47288) 2024-09-10T08:54:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.298+0000 7f53c166c640 1 -- 172.21.15.172:0/1210903825 learned_addr learned my addr 172.21.15.172:0/1210903825 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.298+0000 7f53c0e6b640 1 --2- 172.21.15.172:0/1210903825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd472e0 0x55bbafd51040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.299+0000 7f53c1e6d640 1 --2- 172.21.15.172:0/1210903825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd4c370 0x55bbafd58010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.299+0000 7f53c166c640 1 -- 172.21.15.172:0/1210903825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd4c370 msgr2=0x55bbafd58010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.299+0000 7f53c166c640 1 --2- 172.21.15.172:0/1210903825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd4c370 0x55bbafd58010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.299+0000 7f53c166c640 1 -- 172.21.15.172:0/1210903825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd472e0 msgr2=0x55bbafd51040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.299+0000 7f53c166c640 1 --2- 172.21.15.172:0/1210903825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd472e0 0x55bbafd51040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.299+0000 7f53c166c640 1 -- 172.21.15.172:0/1210903825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbaf9bd210 con 0x55bbafd469a0 2024-09-10T08:54:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.299+0000 7f53c166c640 1 --2- 172.21.15.172:0/1210903825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd469a0 0x55bbafd50b00 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f53b8098610 tx=0x7f53b809c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76780fe7af04e62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.299+0000 7f53a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.299+0000 7f53a4ff9640 1 -- 172.21.15.172:0/1210903825 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b809fff0 con 0x55bbafd469a0 2024-09-10T08:54:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.299+0000 7f53a4ff9640 1 -- 172.21.15.172:0/1210903825 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f53b8086040 con 0x55bbafd469a0 2024-09-10T08:54:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.299+0000 7f53c6665740 1 -- 172.21.15.172:0/1210903825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53b809bc40 con 0x55bbafd469a0 2024-09-10T08:54:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.299+0000 7f53a4ff9640 1 -- 172.21.15.172:0/1210903825 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f53b80a48b0 con 0x55bbafd469a0 2024-09-10T08:54:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.299+0000 7f53c6665740 10 client.17994 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.299+0000 7f53c6665740 1 -- 172.21.15.172:0/1210903825 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bbafa89620 con 0x55bbafd469a0 2024-09-10T08:54:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.299+0000 7f53a4ff9640 1 -- 172.21.15.172:0/1210903825 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53b809bc40 con 0x55bbafd469a0 2024-09-10T08:54:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.299+0000 7f53a4ff9640 1 -- 172.21.15.172:0/1210903825 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f53b80764a0 con 0x55bbafd469a0 2024-09-10T08:54:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.300+0000 7f53c6665740 10 client.17994 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:51.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.300+0000 7f53c6665740 10 client.17994 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:54:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.300+0000 7f53c6665740 20 client.17994 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.300+0000 7f53c6665740 1 --2- 172.21.15.172:0/1210903825 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bbafd5c310 0x55bbafd5e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.300+0000 7f53c6665740 4 client.17994 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:54:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.300+0000 7f53c6665740 1 -- 172.21.15.172:0/1210903825 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbafcb7a20 con 0x55bbafd5c310 2024-09-10T08:54:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.300+0000 7f53c0e6b640 1 --2- 172.21.15.172:0/1210903825 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bbafd5c310 0x55bbafd5e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.300+0000 7f53a4ff9640 10 client.17994 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.300+0000 7f53c0e6b640 1 --2- 172.21.15.172:0/1210903825 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bbafd5c310 0x55bbafd5e700 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb822b464fcc7dd server_cookie=b5f9121b0e46580 in_seq=0 out_seq=0 2024-09-10T08:54:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.301+0000 7f53a4ff9640 1 -- 172.21.15.172:0/1210903825 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bbafcb7a20 con 0x55bbafd5c310 2024-09-10T08:54:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.301+0000 7f53a4ff9640 10 client.17994 handle_command_reply: tid=0 2024-09-10T08:54:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.301+0000 7f53c6665740 1 client.17994 shutdown 2024-09-10T08:54:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.301+0000 7f53c6665740 1 -- 172.21.15.172:0/1210903825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd469a0 msgr2=0x55bbafd50b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.301+0000 7f53c6665740 1 --2- 172.21.15.172:0/1210903825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd469a0 0x55bbafd50b00 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f53b8098610 tx=0x7f53b809c400 comp rx=0 tx=0).stop 2024-09-10T08:54:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.301+0000 7f53c6665740 1 -- 172.21.15.172:0/1210903825 shutdown_connections 2024-09-10T08:54:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.301+0000 7f53c6665740 1 --2- 172.21.15.172:0/1210903825 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bbafd5c310 0x55bbafd5e700 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.301+0000 7f53c6665740 1 --2- 172.21.15.172:0/1210903825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd4c370 0x55bbafd58010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.301+0000 7f53a4ff9640 0 client.17994 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.301+0000 7f53c6665740 1 --2- 172.21.15.172:0/1210903825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd472e0 0x55bbafd51040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.301+0000 7f53c6665740 1 --2- 172.21.15.172:0/1210903825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd469a0 0x55bbafd50b00 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.301+0000 7f53c6665740 1 -- 172.21.15.172:0/1210903825 >> 172.21.15.172:0/1210903825 conn(0x55bbafd42010 msgr2=0x55bbafd43820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.301+0000 7f53c6665740 1 -- 172.21.15.172:0/1210903825 shutdown_connections 2024-09-10T08:54:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.301+0000 7f53c6665740 1 -- 172.21.15.172:0/1210903825 wait complete. 2024-09-10T08:54:51.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.301+0000 7f53c6665740 20 client.17994 trim_cache size 0 max 16384 2024-09-10T08:54:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c6665740 1 Processor -- start 2024-09-10T08:54:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c6665740 1 -- start start 2024-09-10T08:54:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c6665740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd469a0 0x55bbafd50fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c6665740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd472e0 0x55bbafd57fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c6665740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd4c370 0x55bbafd42f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c6665740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc1ac7b0 con 0x55bbafd4c370 2024-09-10T08:54:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c6665740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc1ae2f0 con 0x55bbafd469a0 2024-09-10T08:54:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c6665740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc1ace80 con 0x55bbafd472e0 2024-09-10T08:54:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c1e6d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd4c370 0x55bbafd42f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c1e6d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd4c370 0x55bbafd42f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47290/0 (socket says 172.21.15.172:47290) 2024-09-10T08:54:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c1e6d640 1 -- 172.21.15.172:0/650024398 learned_addr learned my addr 172.21.15.172:0/650024398 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c166c640 1 --2- 172.21.15.172:0/650024398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd469a0 0x55bbafd50fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c0e6b640 1 --2- 172.21.15.172:0/650024398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd472e0 0x55bbafd57fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c1e6d640 1 -- 172.21.15.172:0/650024398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd472e0 msgr2=0x55bbafd57fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c1e6d640 1 --2- 172.21.15.172:0/650024398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd472e0 0x55bbafd57fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c1e6d640 1 -- 172.21.15.172:0/650024398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd469a0 msgr2=0x55bbafd50fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c1e6d640 1 --2- 172.21.15.172:0/650024398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd469a0 0x55bbafd50fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c1e6d640 1 -- 172.21.15.172:0/650024398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbafd4e760 con 0x55bbafd4c370 2024-09-10T08:54:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c0e6b640 1 --2- 172.21.15.172:0/650024398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd472e0 0x55bbafd57fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:54:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.316+0000 7f53c1e6d640 1 --2- 172.21.15.172:0/650024398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd4c370 0x55bbafd42f20 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f53b40097c0 tx=0x7f53b4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce5ebc4e2226c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:51.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.317+0000 7f53a6ffd640 1 -- 172.21.15.172:0/650024398 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4011960 con 0x55bbafd4c370 2024-09-10T08:54:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.317+0000 7f53a6ffd640 1 -- 172.21.15.172:0/650024398 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53b4020400 con 0x55bbafd4c370 2024-09-10T08:54:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.317+0000 7f53a6ffd640 1 -- 172.21.15.172:0/650024398 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b401e620 con 0x55bbafd4c370 2024-09-10T08:54:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.317+0000 7f53c6665740 1 -- 172.21.15.172:0/650024398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd4c370 msgr2=0x55bbafd42f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.317+0000 7f53c6665740 1 --2- 172.21.15.172:0/650024398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd4c370 0x55bbafd42f20 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f53b40097c0 tx=0x7f53b4004ab0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.317+0000 7f53c6665740 1 -- 172.21.15.172:0/650024398 shutdown_connections 2024-09-10T08:54:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.317+0000 7f53c6665740 1 --2- 172.21.15.172:0/650024398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd4c370 0x55bbafd42f20 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.317+0000 7f53c6665740 1 --2- 172.21.15.172:0/650024398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd472e0 0x55bbafd57fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.317+0000 7f53c6665740 1 --2- 172.21.15.172:0/650024398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd469a0 0x55bbafd50fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.317+0000 7f53c6665740 1 -- 172.21.15.172:0/650024398 >> 172.21.15.172:0/650024398 conn(0x55bbafd42010 msgr2=0x55bbafd508b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.317+0000 7f53c6665740 1 -- 172.21.15.172:0/650024398 shutdown_connections 2024-09-10T08:54:51.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.317+0000 7f53c6665740 1 -- 172.21.15.172:0/650024398 wait complete. 2024-09-10T08:54:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.318+0000 7f53c6665740 1 -- start start 2024-09-10T08:54:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.318+0000 7f53c6665740 1 Processor -- start 2024-09-10T08:54:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.318+0000 7f53c6665740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd469a0 0x55bbafd553c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.318+0000 7f53c6665740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd472e0 0x55bbafd55900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.318+0000 7f53c6665740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd4c370 0x55bbafd568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.318+0000 7f53c6665740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc1adc20 con 0x55bbafd472e0 2024-09-10T08:54:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.318+0000 7f53c6665740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc1ad550 con 0x55bbafd469a0 2024-09-10T08:54:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.318+0000 7f53c6665740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc1aa790 con 0x55bbafd4c370 2024-09-10T08:54:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.318+0000 7f53c6665740 10 client.0 _ll_register_callbacks cb 0x7f53b809b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.318+0000 7f53c0e6b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd472e0 0x55bbafd55900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.318+0000 7f53c0e6b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd472e0 0x55bbafd55900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47296/0 (socket says 172.21.15.172:47296) 2024-09-10T08:54:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.318+0000 7f53c0e6b640 1 -- 172.21.15.172:0/4144250861 learned_addr learned my addr 172.21.15.172:0/4144250861 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.318+0000 7f53c0e6b640 1 -- 172.21.15.172:0/4144250861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd4c370 msgr2=0x55bbafd568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.318+0000 7f53c0e6b640 1 --2- 172.21.15.172:0/4144250861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd4c370 0x55bbafd568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.318+0000 7f53c0e6b640 1 -- 172.21.15.172:0/4144250861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd469a0 msgr2=0x55bbafd553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.318+0000 7f53c0e6b640 1 --2- 172.21.15.172:0/4144250861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd469a0 0x55bbafd553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.318+0000 7f53c0e6b640 1 -- 172.21.15.172:0/4144250861 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbafd56eb0 con 0x55bbafd472e0 2024-09-10T08:54:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.319+0000 7f53c0e6b640 1 --2- 172.21.15.172:0/4144250861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd472e0 0x55bbafd55900 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0096c0 tx=0x7f53ac0317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389939d6c4e47a81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.319+0000 7f53a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.319+0000 7f53a4ff9640 1 -- 172.21.15.172:0/4144250861 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac04a020 con 0x55bbafd472e0 2024-09-10T08:54:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.319+0000 7f53a4ff9640 1 -- 172.21.15.172:0/4144250861 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f53ac004450 con 0x55bbafd472e0 2024-09-10T08:54:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.319+0000 7f53c6665740 1 -- 172.21.15.172:0/4144250861 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53b4020400 con 0x55bbafd472e0 2024-09-10T08:54:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.319+0000 7f53a4ff9640 1 -- 172.21.15.172:0/4144250861 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f53ac055050 con 0x55bbafd472e0 2024-09-10T08:54:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.319+0000 7f53c6665740 10 client.18000 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.319+0000 7f53c6665740 1 -- 172.21.15.172:0/4144250861 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bbafd58200 con 0x55bbafd472e0 2024-09-10T08:54:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.319+0000 7f53a4ff9640 1 -- 172.21.15.172:0/4144250861 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53b4020400 con 0x55bbafd472e0 2024-09-10T08:54:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.319+0000 7f53a4ff9640 1 -- 172.21.15.172:0/4144250861 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f53ac0465b0 con 0x55bbafd472e0 2024-09-10T08:54:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.319+0000 7f53c6665740 10 client.18000 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.319+0000 7f53c6665740 10 client.18000 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:54:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.319+0000 7f53c6665740 20 client.18000 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.319+0000 7f53c6665740 1 --2- 172.21.15.172:0/4144250861 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bbafdc7460 0x55bbafdc9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.319+0000 7f53c6665740 4 client.18000 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:54:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.319+0000 7f53c6665740 1 -- 172.21.15.172:0/4144250861 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bbafcb1d70 con 0x55bbafdc7460 2024-09-10T08:54:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.320+0000 7f53c166c640 1 --2- 172.21.15.172:0/4144250861 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bbafdc7460 0x55bbafdc9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.320+0000 7f53c166c640 1 --2- 172.21.15.172:0/4144250861 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bbafdc7460 0x55bbafdc9880 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cf905dbcdfcab8f server_cookie=616adca5d7ce2482 in_seq=0 out_seq=0 2024-09-10T08:54:51.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.320+0000 7f53a4ff9640 10 client.18000 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.327+0000 7f53a4ff9640 1 -- 172.21.15.172:0/4144250861 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bbafcb1d70 con 0x55bbafdc7460 2024-09-10T08:54:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.327+0000 7f53a4ff9640 10 client.18000 handle_command_reply: tid=0 2024-09-10T08:54:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.327+0000 7f53c6665740 1 client.18000 shutdown 2024-09-10T08:54:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.327+0000 7f53c6665740 1 -- 172.21.15.172:0/4144250861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd472e0 msgr2=0x55bbafd55900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.327+0000 7f53c6665740 1 --2- 172.21.15.172:0/4144250861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd472e0 0x55bbafd55900 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0096c0 tx=0x7f53ac0317d0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.327+0000 7f53c6665740 1 -- 172.21.15.172:0/4144250861 shutdown_connections 2024-09-10T08:54:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.327+0000 7f53c6665740 1 --2- 172.21.15.172:0/4144250861 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bbafdc7460 0x55bbafdc9880 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.327+0000 7f53c6665740 1 --2- 172.21.15.172:0/4144250861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bbafd4c370 0x55bbafd568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.328 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.327+0000 7f53a4ff9640 0 client.18000 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.327+0000 7f53c6665740 1 --2- 172.21.15.172:0/4144250861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bbafd472e0 0x55bbafd55900 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.327+0000 7f53c6665740 1 --2- 172.21.15.172:0/4144250861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbafd469a0 0x55bbafd553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.327+0000 7f53c6665740 1 -- 172.21.15.172:0/4144250861 >> 172.21.15.172:0/4144250861 conn(0x55bbafd42010 msgr2=0x55bbafd409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.327+0000 7f53c6665740 1 -- 172.21.15.172:0/4144250861 shutdown_connections 2024-09-10T08:54:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.327+0000 7f53c6665740 1 -- 172.21.15.172:0/4144250861 wait complete. 2024-09-10T08:54:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.327+0000 7f53c6665740 20 client.18000 trim_cache size 0 max 16384 2024-09-10T08:54:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:54:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:54:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:54:51.329 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:54:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.328+0000 7f53c38d0640 1 -- 172.21.15.172:0/1233685668 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53a80a2c30 msgr2=0x7f53a80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.328+0000 7f53c38d0640 1 --2- 172.21.15.172:0/1233685668 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53a80a2c30 0x7f53a80a50f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009fd0 tx=0x7f53b4009d80 comp rx=0 tx=0).stop 2024-09-10T08:54:51.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.328+0000 7f53c38d0640 1 -- 172.21.15.172:0/1233685668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53bc1025e0 msgr2=0x7f53bc199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.328+0000 7f53c38d0640 1 --2- 172.21.15.172:0/1233685668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53bc1025e0 0x7f53bc199fb0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f53b800e9f0 tx=0x7f53b800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.328+0000 7f53c38d0640 1 -- 172.21.15.172:0/1233685668 shutdown_connections 2024-09-10T08:54:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.328+0000 7f53c38d0640 1 --2- 172.21.15.172:0/1233685668 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53a80a2c30 0x7f53a80a50f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.328+0000 7f53c38d0640 1 --2- 172.21.15.172:0/1233685668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53bc107840 0x7f53bc1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.328+0000 7f53c38d0640 1 --2- 172.21.15.172:0/1233685668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53bc104f10 0x7f53bc19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.328+0000 7f53c38d0640 1 --2- 172.21.15.172:0/1233685668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53bc1025e0 0x7f53bc199fb0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.328+0000 7f53c38d0640 1 -- 172.21.15.172:0/1233685668 >> 172.21.15.172:0/1233685668 conn(0x7f53bc0fc1e0 msgr2=0x7f53bc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.328+0000 7f53c38d0640 1 -- 172.21.15.172:0/1233685668 shutdown_connections 2024-09-10T08:54:51.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:54:51.328+0000 7f53c38d0640 1 -- 172.21.15.172:0/1233685668 wait complete. 2024-09-10T08:54:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:52 smithi174 ceph-mon[34068]: pgmap v609: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 20 MiB/s wr, 45 op/s 2024-09-10T08:54:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:52 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:54:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:52 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:54:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:52 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:54:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:52 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:54:52.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:52 smithi191 ceph-mon[35314]: pgmap v609: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 20 MiB/s wr, 45 op/s 2024-09-10T08:54:52.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:52 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:54:52.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:52 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:54:52.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:52 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:54:52.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:52 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:54:52.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:52 smithi172 ceph-mon[30854]: pgmap v609: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 20 MiB/s wr, 45 op/s 2024-09-10T08:54:52.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:52 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:54:52.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:52 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:54:52.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:52 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:54:52.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:52 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:54:54.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:54 smithi174 ceph-mon[34068]: pgmap v610: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 30 op/s 2024-09-10T08:54:54.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:54 smithi191 ceph-mon[35314]: pgmap v610: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 30 op/s 2024-09-10T08:54:54.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:54 smithi172 ceph-mon[30854]: pgmap v610: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 13 MiB/s wr, 30 op/s 2024-09-10T08:54:55.314 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:54:55.576 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:54:56.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.037+0000 7fba77da9640 1 -- 172.21.15.172:0/1055390798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba70102ef0 msgr2=0x7fba7010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:56.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.037+0000 7fba77da9640 1 --2- 172.21.15.172:0/1055390798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba70102ef0 0x7fba7010da40 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fba68009ca0 tx=0x7fba6802a950 comp rx=0 tx=0).stop 2024-09-10T08:54:56.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.037+0000 7fba77da9640 1 -- 172.21.15.172:0/1055390798 shutdown_connections 2024-09-10T08:54:56.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.037+0000 7fba77da9640 1 --2- 172.21.15.172:0/1055390798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba70102ef0 0x7fba7010da40 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:56.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.037+0000 7fba77da9640 1 --2- 172.21.15.172:0/1055390798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba70102570 0x7fba701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:56.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.037+0000 7fba77da9640 1 --2- 172.21.15.172:0/1055390798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba70100e20 0x7fba70101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:56.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.037+0000 7fba77da9640 1 -- 172.21.15.172:0/1055390798 >> 172.21.15.172:0/1055390798 conn(0x7fba700fc7d0 msgr2=0x7fba700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:56.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.037+0000 7fba77da9640 1 -- 172.21.15.172:0/1055390798 shutdown_connections 2024-09-10T08:54:56.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.037+0000 7fba77da9640 1 -- 172.21.15.172:0/1055390798 wait complete. 2024-09-10T08:54:56.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.037+0000 7fba77da9640 1 Processor -- start 2024-09-10T08:54:56.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.037+0000 7fba77da9640 1 -- start start 2024-09-10T08:54:56.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba77da9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba70100e20 0x7fba70199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:56.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba77da9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba70102570 0x7fba701996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:56.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba77da9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba70102ef0 0x7fba701a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:56.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba77da9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fba70110e30 con 0x7fba70100e20 2024-09-10T08:54:56.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba77da9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fba70110cb0 con 0x7fba70102ef0 2024-09-10T08:54:56.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba77da9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fba70110fb0 con 0x7fba70102570 2024-09-10T08:54:56.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba75b45640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba70100e20 0x7fba70199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:56.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba75b45640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba70100e20 0x7fba70199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47310/0 (socket says 172.21.15.172:47310) 2024-09-10T08:54:56.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba75b45640 1 -- 172.21.15.172:0/3501221697 learned_addr learned my addr 172.21.15.172:0/3501221697 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:56.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba76346640 1 --2- 172.21.15.172:0/3501221697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba70102ef0 0x7fba701a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:56.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba75344640 1 --2- 172.21.15.172:0/3501221697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba70102570 0x7fba701996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:56.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba75b45640 1 -- 172.21.15.172:0/3501221697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba70102570 msgr2=0x7fba701996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:56.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba75b45640 1 --2- 172.21.15.172:0/3501221697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba70102570 0x7fba701996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:56.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba75b45640 1 -- 172.21.15.172:0/3501221697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba70102ef0 msgr2=0x7fba701a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:56.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba75b45640 1 --2- 172.21.15.172:0/3501221697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba70102ef0 0x7fba701a41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:56.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba75b45640 1 -- 172.21.15.172:0/3501221697 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba701a4720 con 0x7fba70100e20 2024-09-10T08:54:56.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba75344640 1 --2- 172.21.15.172:0/3501221697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba70102570 0x7fba701996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:54:56.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba75b45640 1 --2- 172.21.15.172:0/3501221697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba70100e20 0x7fba70199160 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00d950 tx=0x7fba6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:56.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba66ffd640 1 -- 172.21.15.172:0/3501221697 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c017030 con 0x7fba70100e20 2024-09-10T08:54:56.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba77da9640 1 -- 172.21.15.172:0/3501221697 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba701a48f0 con 0x7fba70100e20 2024-09-10T08:54:56.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba66ffd640 1 -- 172.21.15.172:0/3501221697 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba6c004020 con 0x7fba70100e20 2024-09-10T08:54:56.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba66ffd640 1 -- 172.21.15.172:0/3501221697 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0047a0 con 0x7fba70100e20 2024-09-10T08:54:56.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.038+0000 7fba77da9640 1 -- 172.21.15.172:0/3501221697 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba701a4de0 con 0x7fba70100e20 2024-09-10T08:54:56.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.039+0000 7fba77da9640 1 -- 172.21.15.172:0/3501221697 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba70101200 con 0x7fba70100e20 2024-09-10T08:54:56.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.041+0000 7fba66ffd640 1 -- 172.21.15.172:0/3501221697 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fba6c0041d0 con 0x7fba70100e20 2024-09-10T08:54:56.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.041+0000 7fba66ffd640 1 --2- 172.21.15.172:0/3501221697 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fba5c0a2b80 0x7fba5c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:56.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.041+0000 7fba75344640 1 --2- 172.21.15.172:0/3501221697 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fba5c0a2b80 0x7fba5c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:56.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.041+0000 7fba66ffd640 1 -- 172.21.15.172:0/3501221697 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fba6c0cf4a0 con 0x7fba70100e20 2024-09-10T08:54:56.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.042+0000 7fba75344640 1 --2- 172.21.15.172:0/3501221697 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fba5c0a2b80 0x7fba5c0a5040 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fba60005e10 tx=0x7fba60005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:56.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.044+0000 7fba66ffd640 1 -- 172.21.15.172:0/3501221697 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fba6c09c0a0 con 0x7fba70100e20 2024-09-10T08:54:56.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.245+0000 7fba77da9640 1 -- 172.21.15.172:0/3501221697 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba7006a560 con 0x7fba70100e20 2024-09-10T08:54:56.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.245+0000 7fba66ffd640 1 -- 172.21.15.172:0/3501221697 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+12001 (secure 0 0 0) 0x7fba6c09b9a0 con 0x7fba70100e20 2024-09-10T08:54:56.246 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:54:56.246 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":37,"btime":"2024-09-10T08:52:08:313493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:52:07.658341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:54:56.247 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 37 2024-09-10T08:54:56.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.247+0000 7fba77da9640 1 -- 172.21.15.172:0/3501221697 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fba5c0a2b80 msgr2=0x7fba5c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:56.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.247+0000 7fba77da9640 1 --2- 172.21.15.172:0/3501221697 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fba5c0a2b80 0x7fba5c0a5040 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fba60005e10 tx=0x7fba60005da0 comp rx=0 tx=0).stop 2024-09-10T08:54:56.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.247+0000 7fba77da9640 1 -- 172.21.15.172:0/3501221697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba70100e20 msgr2=0x7fba70199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:56.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.247+0000 7fba77da9640 1 --2- 172.21.15.172:0/3501221697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba70100e20 0x7fba70199160 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00d950 tx=0x7fba6c00de10 comp rx=0 tx=0).stop 2024-09-10T08:54:56.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.247+0000 7fba77da9640 1 -- 172.21.15.172:0/3501221697 shutdown_connections 2024-09-10T08:54:56.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.247+0000 7fba77da9640 1 --2- 172.21.15.172:0/3501221697 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fba5c0a2b80 0x7fba5c0a5040 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:56.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.247+0000 7fba77da9640 1 --2- 172.21.15.172:0/3501221697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba70102ef0 0x7fba701a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:56.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.247+0000 7fba77da9640 1 --2- 172.21.15.172:0/3501221697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba70102570 0x7fba701996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:56.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.247+0000 7fba77da9640 1 --2- 172.21.15.172:0/3501221697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba70100e20 0x7fba70199160 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:56.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.247+0000 7fba77da9640 1 -- 172.21.15.172:0/3501221697 >> 172.21.15.172:0/3501221697 conn(0x7fba700fc7d0 msgr2=0x7fba70106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:56.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.248+0000 7fba77da9640 1 -- 172.21.15.172:0/3501221697 shutdown_connections 2024-09-10T08:54:56.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:56.248+0000 7fba77da9640 1 -- 172.21.15.172:0/3501221697 wait complete. 2024-09-10T08:54:56.526 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:56 smithi172 ceph-mon[30854]: pgmap v611: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 29 op/s 2024-09-10T08:54:56.526 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:56 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3501221697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:54:56.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:56 smithi174 ceph-mon[34068]: pgmap v611: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 29 op/s 2024-09-10T08:54:56.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:56 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3501221697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:54:56.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:56 smithi191 ceph-mon[35314]: pgmap v611: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 29 op/s 2024-09-10T08:54:56.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:56 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3501221697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:54:56.961 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id c1e21f5b --release --await' 2024-09-10T08:54:56.961 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id c1e21f5b --release --await 2024-09-10T08:54:57.216 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:54:57.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.756+0000 7f28953b0640 1 -- 172.21.15.172:0/1881444492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2890102d40 msgr2=0x7f2890103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:57.760 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.756+0000 7f28953b0640 1 --2- 172.21.15.172:0/1881444492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2890102d40 0x7f2890103120 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f2884009ca0 tx=0x7f288402a950 comp rx=0 tx=0).stop 2024-09-10T08:54:57.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.756+0000 7f28953b0640 1 -- 172.21.15.172:0/1881444492 shutdown_connections 2024-09-10T08:54:57.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.756+0000 7f28953b0640 1 --2- 172.21.15.172:0/1881444492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f289010c170 0x7f289010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.756+0000 7f28953b0640 1 --2- 172.21.15.172:0/1881444492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2890103660 0x7f289010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.756+0000 7f28953b0640 1 --2- 172.21.15.172:0/1881444492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2890102d40 0x7f2890103120 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.756+0000 7f28953b0640 1 -- 172.21.15.172:0/1881444492 >> 172.21.15.172:0/1881444492 conn(0x7f28900fc7a0 msgr2=0x7f28900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:57.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.756+0000 7f28953b0640 1 -- 172.21.15.172:0/1881444492 shutdown_connections 2024-09-10T08:54:57.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.756+0000 7f28953b0640 1 -- 172.21.15.172:0/1881444492 wait complete. 2024-09-10T08:54:57.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.757+0000 7f28953b0640 1 Processor -- start 2024-09-10T08:54:57.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.757+0000 7f28953b0640 1 -- start start 2024-09-10T08:54:57.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.757+0000 7f28953b0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2890102d40 0x7f28901009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.761 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.757+0000 7f28953b0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2890103660 0x7f2890100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.757+0000 7f28953b0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f289010c170 0x7f2890101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.757+0000 7f28953b0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2890111e20 con 0x7f2890102d40 2024-09-10T08:54:57.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.757+0000 7f28953b0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2890111ca0 con 0x7f289010c170 2024-09-10T08:54:57.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.757+0000 7f28953b0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2890111fa0 con 0x7f2890103660 2024-09-10T08:54:57.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.757+0000 7f288effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2890102d40 0x7f28901009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:57.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.757+0000 7f288effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2890102d40 0x7f28901009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46164/0 (socket says 172.21.15.172:46164) 2024-09-10T08:54:57.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.757+0000 7f288effd640 1 -- 172.21.15.172:0/554205554 learned_addr learned my addr 172.21.15.172:0/554205554 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:57.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.757+0000 7f288e7fc640 1 --2- 172.21.15.172:0/554205554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2890103660 0x7f2890100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:57.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.757+0000 7f288f7fe640 1 --2- 172.21.15.172:0/554205554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f289010c170 0x7f2890101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:57.762 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.757+0000 7f288effd640 1 -- 172.21.15.172:0/554205554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2890103660 msgr2=0x7f2890100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:57.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.757+0000 7f288effd640 1 --2- 172.21.15.172:0/554205554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2890103660 0x7f2890100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.757+0000 7f288effd640 1 -- 172.21.15.172:0/554205554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f289010c170 msgr2=0x7f2890101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:57.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.758+0000 7f288effd640 1 --2- 172.21.15.172:0/554205554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f289010c170 0x7f2890101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.758+0000 7f288effd640 1 -- 172.21.15.172:0/554205554 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28901a68e0 con 0x7f2890102d40 2024-09-10T08:54:57.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.758+0000 7f288f7fe640 1 --2- 172.21.15.172:0/554205554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f289010c170 0x7f2890101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:54:57.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.758+0000 7f288effd640 1 --2- 172.21.15.172:0/554205554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2890102d40 0x7f28901009a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2884005cc0 tx=0x7f288402f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:57.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.758+0000 7f2877fff640 1 -- 172.21.15.172:0/554205554 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884034040 con 0x7f2890102d40 2024-09-10T08:54:57.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.758+0000 7f28953b0640 1 -- 172.21.15.172:0/554205554 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28901a6b70 con 0x7f2890102d40 2024-09-10T08:54:57.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.758+0000 7f2877fff640 1 -- 172.21.15.172:0/554205554 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28840308f0 con 0x7f2890102d40 2024-09-10T08:54:57.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.758+0000 7f2877fff640 1 -- 172.21.15.172:0/554205554 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28840047c0 con 0x7f2890102d40 2024-09-10T08:54:57.763 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.758+0000 7f28953b0640 1 -- 172.21.15.172:0/554205554 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28901a70a0 con 0x7f2890102d40 2024-09-10T08:54:57.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.761+0000 7f2877fff640 1 -- 172.21.15.172:0/554205554 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2884037020 con 0x7f2890102d40 2024-09-10T08:54:57.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.761+0000 7f2897789740 1 Processor -- start 2024-09-10T08:54:57.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.761+0000 7f2897789740 1 -- start start 2024-09-10T08:54:57.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.761+0000 7f2897789740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99459060 0x557b99459440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.761+0000 7f2897789740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99459980 0x557b99463d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.761+0000 7f2897789740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99464240 0x557b99466630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.769 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.761+0000 7f2897789740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557b993be620 con 0x557b99464240 2024-09-10T08:54:57.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.761+0000 7f2897789740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b99365e50 con 0x557b99459060 2024-09-10T08:54:57.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.761+0000 7f2897789740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557b993ba5e0 con 0x557b99459980 2024-09-10T08:54:57.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.761+0000 7f288e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99464240 0x557b99466630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:57.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.761+0000 7f288e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99464240 0x557b99466630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46174/0 (socket says 172.21.15.172:46174) 2024-09-10T08:54:57.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.761+0000 7f288e7fc640 1 -- 172.21.15.172:0/952237740 learned_addr learned my addr 172.21.15.172:0/952237740 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:57.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.761+0000 7f2877fff640 1 --2- 172.21.15.172:0/554205554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f28780a2b80 0x7f28780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.761+0000 7f288effd640 1 --2- 172.21.15.172:0/952237740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99459980 0x557b99463d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:57.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.761+0000 7f288f7fe640 1 --2- 172.21.15.172:0/952237740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99459060 0x557b99459440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:57.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.762+0000 7f288f7fe640 1 --2- 172.21.15.172:0/554205554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f28780a2b80 0x7f28780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:57.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.762+0000 7f2877fff640 1 -- 172.21.15.172:0/554205554 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f28840f0330 con 0x7f2890102d40 2024-09-10T08:54:57.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.762+0000 7f288e7fc640 1 -- 172.21.15.172:0/952237740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99459980 msgr2=0x557b99463d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:57.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.762+0000 7f288e7fc640 1 --2- 172.21.15.172:0/952237740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99459980 0x557b99463d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.770 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.762+0000 7f288e7fc640 1 -- 172.21.15.172:0/952237740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99459060 msgr2=0x557b99459440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:57.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.762+0000 7f288e7fc640 1 --2- 172.21.15.172:0/952237740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99459060 0x557b99459440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.762+0000 7f288e7fc640 1 -- 172.21.15.172:0/952237740 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b99356d40 con 0x557b99464240 2024-09-10T08:54:57.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.762+0000 7f288effd640 1 --2- 172.21.15.172:0/952237740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99459980 0x557b99463d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:54:57.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.762+0000 7f288e7fc640 1 --2- 172.21.15.172:0/952237740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99464240 0x557b99466630 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f288000b730 tx=0x7f288000bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd8471898e76afa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:57.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.762+0000 7f288f7fe640 1 --2- 172.21.15.172:0/554205554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f28780a2b80 0x7f28780a5040 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f288800fdc0 tx=0x7f2888005900 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:57.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.762+0000 7f28757fa640 1 -- 172.21.15.172:0/952237740 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880003dd0 con 0x557b99464240 2024-09-10T08:54:57.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.762+0000 7f28757fa640 1 -- 172.21.15.172:0/952237740 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2880015070 con 0x557b99464240 2024-09-10T08:54:57.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.762+0000 7f28757fa640 1 -- 172.21.15.172:0/952237740 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880004d10 con 0x557b99464240 2024-09-10T08:54:57.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.763+0000 7f2897789740 1 -- 172.21.15.172:0/952237740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99464240 msgr2=0x557b99466630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:57.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.763+0000 7f2897789740 1 --2- 172.21.15.172:0/952237740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99464240 0x557b99466630 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f288000b730 tx=0x7f288000bbf0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.763+0000 7f2897789740 1 -- 172.21.15.172:0/952237740 shutdown_connections 2024-09-10T08:54:57.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.763+0000 7f2897789740 1 --2- 172.21.15.172:0/952237740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99464240 0x557b99466630 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.771 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.763+0000 7f2897789740 1 --2- 172.21.15.172:0/952237740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99459980 0x557b99463d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.763+0000 7f2897789740 1 --2- 172.21.15.172:0/952237740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99459060 0x557b99459440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.763+0000 7f2897789740 1 -- 172.21.15.172:0/952237740 >> 172.21.15.172:0/952237740 conn(0x557b994555f0 msgr2=0x557b994559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:57.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.763+0000 7f2897789740 1 -- 172.21.15.172:0/952237740 shutdown_connections 2024-09-10T08:54:57.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.763+0000 7f2897789740 1 -- 172.21.15.172:0/952237740 wait complete. 2024-09-10T08:54:57.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.763+0000 7f2897789740 1 -- start start 2024-09-10T08:54:57.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.763+0000 7f2897789740 1 Processor -- start 2024-09-10T08:54:57.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f2897789740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99459060 0x557b994610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f2897789740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459980 0x557b99461600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f2897789740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99464240 0x557b99461b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f2897789740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557b99466e10 con 0x557b99459980 2024-09-10T08:54:57.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f2897789740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b99346f10 con 0x557b99464240 2024-09-10T08:54:57.772 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f2897789740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557b99466f90 con 0x557b99459060 2024-09-10T08:54:57.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f2897789740 10 client.0 _ll_register_callbacks cb 0x557b9936adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:57.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f288e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459980 0x557b99461600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:57.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f288e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459980 0x557b99461600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46188/0 (socket says 172.21.15.172:46188) 2024-09-10T08:54:57.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f288e7fc640 1 -- 172.21.15.172:0/3255802303 learned_addr learned my addr 172.21.15.172:0/3255802303 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:57.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f288effd640 1 --2- 172.21.15.172:0/3255802303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99459060 0x557b994610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:57.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f288f7fe640 1 --2- 172.21.15.172:0/3255802303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99464240 0x557b99461b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:57.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f288e7fc640 1 -- 172.21.15.172:0/3255802303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99459060 msgr2=0x557b994610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:57.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f288e7fc640 1 --2- 172.21.15.172:0/3255802303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99459060 0x557b994610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f288e7fc640 1 -- 172.21.15.172:0/3255802303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99464240 msgr2=0x557b99461b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:57.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f288e7fc640 1 --2- 172.21.15.172:0/3255802303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99464240 0x557b99461b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f288e7fc640 1 -- 172.21.15.172:0/3255802303 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b990cd580 con 0x557b99459980 2024-09-10T08:54:57.773 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f288f7fe640 1 --2- 172.21.15.172:0/3255802303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99464240 0x557b99461b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:54:57.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f288e7fc640 1 --2- 172.21.15.172:0/3255802303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459980 0x557b99461600 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f28800057c0 tx=0x7f2880003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebe7d01bd0b4b7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:57.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f2872ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:57.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.764+0000 7f2872ff5640 1 -- 172.21.15.172:0/3255802303 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880028060 con 0x557b99459980 2024-09-10T08:54:57.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.765+0000 7f2872ff5640 1 -- 172.21.15.172:0/3255802303 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2880011650 con 0x557b99459980 2024-09-10T08:54:57.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.765+0000 7f2897789740 1 -- 172.21.15.172:0/3255802303 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2880015070 con 0x557b99459980 2024-09-10T08:54:57.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.765+0000 7f2872ff5640 1 -- 172.21.15.172:0/3255802303 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f288002f050 con 0x557b99459980 2024-09-10T08:54:57.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.765+0000 7f2897789740 10 client.18018 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:57.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.765+0000 7f2897789740 1 -- 172.21.15.172:0/3255802303 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557b99197960 con 0x557b99459980 2024-09-10T08:54:57.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.765+0000 7f2872ff5640 1 -- 172.21.15.172:0/3255802303 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2880015070 con 0x557b99459980 2024-09-10T08:54:57.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.765+0000 7f2872ff5640 1 -- 172.21.15.172:0/3255802303 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f2880022070 con 0x557b99459980 2024-09-10T08:54:57.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.765+0000 7f2897789740 10 client.18018 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:57.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.765+0000 7f2897789740 10 client.18018 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:54:57.774 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.765+0000 7f2897789740 20 client.18018 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:57.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.765+0000 7f2897789740 1 --2- 172.21.15.172:0/3255802303 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557b99473d70 0x557b99476160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.765+0000 7f2897789740 4 client.18018 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:54:57.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.765+0000 7f2897789740 1 -- 172.21.15.172:0/3255802303 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b993cbb90 con 0x557b99473d70 2024-09-10T08:54:57.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.766+0000 7f288effd640 1 --2- 172.21.15.172:0/3255802303 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557b99473d70 0x557b99476160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:57.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.766+0000 7f2872ff5640 10 client.18018 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:57.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.766+0000 7f288effd640 1 --2- 172.21.15.172:0/3255802303 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557b99473d70 0x557b99476160 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b9f4d91c0bee2a9 server_cookie=e589da33e4852af8 in_seq=0 out_seq=0 2024-09-10T08:54:57.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.766+0000 7f2872ff5640 1 -- 172.21.15.172:0/3255802303 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b993cbb90 con 0x557b99473d70 2024-09-10T08:54:57.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.767+0000 7f2872ff5640 10 client.18018 handle_command_reply: tid=0 2024-09-10T08:54:57.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.767+0000 7f2897789740 1 client.18018 shutdown 2024-09-10T08:54:57.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.767+0000 7f2897789740 1 -- 172.21.15.172:0/3255802303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459980 msgr2=0x557b99461600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:57.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.767+0000 7f2897789740 1 --2- 172.21.15.172:0/3255802303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459980 0x557b99461600 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f28800057c0 tx=0x7f2880003e00 comp rx=0 tx=0).stop 2024-09-10T08:54:57.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.767+0000 7f2897789740 1 -- 172.21.15.172:0/3255802303 shutdown_connections 2024-09-10T08:54:57.775 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.767+0000 7f2897789740 1 --2- 172.21.15.172:0/3255802303 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557b99473d70 0x557b99476160 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.767+0000 7f2897789740 1 --2- 172.21.15.172:0/3255802303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99464240 0x557b99461b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.767+0000 7f2897789740 1 --2- 172.21.15.172:0/3255802303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459980 0x557b99461600 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.767+0000 7f2897789740 1 --2- 172.21.15.172:0/3255802303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99459060 0x557b994610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.767+0000 7f2872ff5640 0 client.18018 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:57.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.767+0000 7f2897789740 1 -- 172.21.15.172:0/3255802303 >> 172.21.15.172:0/3255802303 conn(0x557b994555f0 msgr2=0x557b9945b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:57.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.767+0000 7f2897789740 1 -- 172.21.15.172:0/3255802303 shutdown_connections 2024-09-10T08:54:57.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.767+0000 7f2897789740 1 -- 172.21.15.172:0/3255802303 wait complete. 2024-09-10T08:54:57.776 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.767+0000 7f2897789740 20 client.18018 trim_cache size 0 max 16384 2024-09-10T08:54:57.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f2897789740 1 Processor -- start 2024-09-10T08:54:57.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f2897789740 1 -- start start 2024-09-10T08:54:57.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f2897789740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99459060 0x557b9947b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f2897789740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459980 0x557b9947bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f2897789740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99464240 0x557b9945e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f2897789740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f28901b0c40 con 0x557b99459980 2024-09-10T08:54:57.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f2897789740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28901b2780 con 0x557b99464240 2024-09-10T08:54:57.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f2897789740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f28901b1310 con 0x557b99459060 2024-09-10T08:54:57.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f288e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459980 0x557b9947bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:57.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f288e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459980 0x557b9947bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46200/0 (socket says 172.21.15.172:46200) 2024-09-10T08:54:57.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f288e7fc640 1 -- 172.21.15.172:0/1921633292 learned_addr learned my addr 172.21.15.172:0/1921633292 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:57.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f288e7fc640 1 -- 172.21.15.172:0/1921633292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99459060 msgr2=0x557b9947b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:57.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f288e7fc640 1 --2- 172.21.15.172:0/1921633292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99459060 0x557b9947b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f288e7fc640 1 -- 172.21.15.172:0/1921633292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99464240 msgr2=0x557b9945e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:57.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f288e7fc640 1 --2- 172.21.15.172:0/1921633292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99464240 0x557b9945e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f288e7fc640 1 -- 172.21.15.172:0/1921633292 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b99456280 con 0x557b99459980 2024-09-10T08:54:57.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f288e7fc640 1 --2- 172.21.15.172:0/1921633292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459980 0x557b9947bcd0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2880004470 tx=0x7f288000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f215eb9531e72286 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:57.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.784+0000 7f2874ff9640 1 -- 172.21.15.172:0/1921633292 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880038070 con 0x557b99459980 2024-09-10T08:54:57.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.785+0000 7f2874ff9640 1 -- 172.21.15.172:0/1921633292 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2880015040 con 0x557b99459980 2024-09-10T08:54:57.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.785+0000 7f2874ff9640 1 -- 172.21.15.172:0/1921633292 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880004840 con 0x557b99459980 2024-09-10T08:54:57.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.785+0000 7f2897789740 1 -- 172.21.15.172:0/1921633292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459980 msgr2=0x557b9947bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:57.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.785+0000 7f2897789740 1 --2- 172.21.15.172:0/1921633292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459980 0x557b9947bcd0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2880004470 tx=0x7f288000bfd0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.785+0000 7f2897789740 1 -- 172.21.15.172:0/1921633292 shutdown_connections 2024-09-10T08:54:57.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.785+0000 7f2897789740 1 --2- 172.21.15.172:0/1921633292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99464240 0x557b9945e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.785+0000 7f2897789740 1 --2- 172.21.15.172:0/1921633292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459980 0x557b9947bcd0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.785+0000 7f2897789740 1 --2- 172.21.15.172:0/1921633292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99459060 0x557b9947b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.785+0000 7f2897789740 1 -- 172.21.15.172:0/1921633292 >> 172.21.15.172:0/1921633292 conn(0x557b994555f0 msgr2=0x557b99462920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:57.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.785+0000 7f2897789740 1 -- 172.21.15.172:0/1921633292 shutdown_connections 2024-09-10T08:54:57.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.785+0000 7f2897789740 1 -- 172.21.15.172:0/1921633292 wait complete. 2024-09-10T08:54:57.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.785+0000 7f2897789740 1 -- start start 2024-09-10T08:54:57.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f2897789740 1 Processor -- start 2024-09-10T08:54:57.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f2897789740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459060 0x557b9946c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f2897789740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99459980 0x557b9946c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f2897789740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99464240 0x557b994691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f2897789740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f28901b20b0 con 0x557b99459060 2024-09-10T08:54:57.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f2897789740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28901b19e0 con 0x557b99459980 2024-09-10T08:54:57.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f2897789740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f28901aec20 con 0x557b99464240 2024-09-10T08:54:57.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f2897789740 10 client.0 _ll_register_callbacks cb 0x7f28800043c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:54:57.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f288effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459060 0x557b9946c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:57.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f288effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459060 0x557b9946c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46212/0 (socket says 172.21.15.172:46212) 2024-09-10T08:54:57.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f288effd640 1 -- 172.21.15.172:0/3120562424 learned_addr learned my addr 172.21.15.172:0/3120562424 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:54:57.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f288e7fc640 1 --2- 172.21.15.172:0/3120562424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99459980 0x557b9946c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:57.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f288effd640 1 -- 172.21.15.172:0/3120562424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99464240 msgr2=0x557b994691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:57.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f288effd640 1 --2- 172.21.15.172:0/3120562424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99464240 0x557b994691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f288effd640 1 -- 172.21.15.172:0/3120562424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99459980 msgr2=0x557b9946c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:54:57.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f288effd640 1 --2- 172.21.15.172:0/3120562424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99459980 0x557b9946c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f288effd640 1 -- 172.21.15.172:0/3120562424 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b99462090 con 0x557b99459060 2024-09-10T08:54:57.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f288e7fc640 1 --2- 172.21.15.172:0/3120562424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99459980 0x557b9946c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:54:57.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f288effd640 1 --2- 172.21.15.172:0/3120562424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459060 0x557b9946c3f0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f28840b8d30 tx=0x7f288408b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef03d3290c2ccc28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:54:57.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f2872ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:54:57.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.786+0000 7f2872ff5640 1 -- 172.21.15.172:0/3120562424 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28840a5e60 con 0x557b99459060 2024-09-10T08:54:57.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.787+0000 7f2872ff5640 1 -- 172.21.15.172:0/3120562424 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2884004040 con 0x557b99459060 2024-09-10T08:54:57.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.787+0000 7f2897789740 1 -- 172.21.15.172:0/3120562424 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2880015040 con 0x557b99459060 2024-09-10T08:54:57.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.787+0000 7f2872ff5640 1 -- 172.21.15.172:0/3120562424 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f28840bc230 con 0x557b99459060 2024-09-10T08:54:57.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.787+0000 7f2897789740 10 client.18024 fetch_fsmap learned FSMap version 37 2024-09-10T08:54:57.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.787+0000 7f2897789740 1 -- 172.21.15.172:0/3120562424 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557b994df090 con 0x557b99459060 2024-09-10T08:54:57.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.787+0000 7f2872ff5640 1 -- 172.21.15.172:0/3120562424 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2880015040 con 0x557b99459060 2024-09-10T08:54:57.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.787+0000 7f2872ff5640 1 -- 172.21.15.172:0/3120562424 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f28840b8f40 con 0x557b99459060 2024-09-10T08:54:57.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.787+0000 7f2897789740 10 client.18024 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:54:57.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.787+0000 7f2897789740 10 client.18024 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:54:57.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.787+0000 7f2897789740 20 client.18024 populate_metadata read hostname 'smithi172' 2024-09-10T08:54:57.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.787+0000 7f2897789740 1 --2- 172.21.15.172:0/3120562424 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557b994df760 0x557b994e1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:54:57.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.787+0000 7f2897789740 4 client.18024 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c1e21f5b", "release": true, "await": true}] 2024-09-10T08:54:57.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.787+0000 7f2897789740 1 -- 172.21.15.172:0/3120562424 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c1e21f5b", "release": true, "await": true}) -- 0x557b993c60a0 con 0x557b994df760 2024-09-10T08:54:57.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.787+0000 7f288e7fc640 1 --2- 172.21.15.172:0/3120562424 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557b994df760 0x557b994e1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:54:57.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.788+0000 7f288e7fc640 1 --2- 172.21.15.172:0/3120562424 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557b994df760 0x557b994e1b50 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=413dd6e800da2095 server_cookie=fca80b02063909e0 in_seq=0 out_seq=0 2024-09-10T08:54:57.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.788+0000 7f2872ff5640 10 client.18024 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:57.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.804+0000 7f2872ff5640 1 -- 172.21.15.172:0/3120562424 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557b993c60a0 con 0x557b994df760 2024-09-10T08:54:57.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.804+0000 7f2872ff5640 10 client.18024 handle_command_reply: tid=0 2024-09-10T08:54:57.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.804+0000 7f2897789740 1 client.18024 shutdown 2024-09-10T08:54:57.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.804+0000 7f2897789740 1 -- 172.21.15.172:0/3120562424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459060 msgr2=0x557b9946c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:57.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.804+0000 7f2897789740 1 --2- 172.21.15.172:0/3120562424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459060 0x557b9946c3f0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f28840b8d30 tx=0x7f288408b850 comp rx=0 tx=0).stop 2024-09-10T08:54:57.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.804+0000 7f2897789740 1 -- 172.21.15.172:0/3120562424 shutdown_connections 2024-09-10T08:54:57.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.804+0000 7f2897789740 1 --2- 172.21.15.172:0/3120562424 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557b994df760 0x557b994e1b50 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.804+0000 7f2897789740 1 --2- 172.21.15.172:0/3120562424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557b99464240 0x557b994691e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.804+0000 7f2897789740 1 --2- 172.21.15.172:0/3120562424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b99459980 0x557b9946c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.804+0000 7f2897789740 1 --2- 172.21.15.172:0/3120562424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b99459060 0x557b9946c3f0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.804+0000 7f2897789740 1 -- 172.21.15.172:0/3120562424 >> 172.21.15.172:0/3120562424 conn(0x557b994555f0 msgr2=0x557b9945f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:57.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.804+0000 7f2872ff5640 0 client.18024 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:54:57.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.804+0000 7f2897789740 1 -- 172.21.15.172:0/3120562424 shutdown_connections 2024-09-10T08:54:57.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.804+0000 7f2897789740 1 -- 172.21.15.172:0/3120562424 wait complete. 2024-09-10T08:54:57.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.804+0000 7f2897789740 20 client.18024 trim_cache size 0 max 16384 2024-09-10T08:54:57.806 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:54:57.806 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 37, 2024-09-10T08:54:57.806 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:54:57.806 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 28, 2024-09-10T08:54:57.807 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:54:57.807 INFO:teuthology.orchestra.run.smithi172.stdout: "c1e21f5b": { 2024-09-10T08:54:57.807 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 28, 2024-09-10T08:54:57.807 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:54:57.807 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:54:57.807 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:54:57.807 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:54:57.807 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:54:57.807 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:54:57.807 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 133.4, 2024-09-10T08:54:57.807 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:54:57.807 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:54:57.807 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:54:57.807 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:54:57.808 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:54:57.808 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:54:57.808 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:54:57.808 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:54:57.808 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:54:57.808 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:54:57.808 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:54:57.808 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:54:57.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.805+0000 7f28953b0640 1 -- 172.21.15.172:0/554205554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f28780a2b80 msgr2=0x7f28780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:57.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.805+0000 7f28953b0640 1 --2- 172.21.15.172:0/554205554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f28780a2b80 0x7f28780a5040 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f288800fdc0 tx=0x7f2888005900 comp rx=0 tx=0).stop 2024-09-10T08:54:57.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.805+0000 7f28953b0640 1 -- 172.21.15.172:0/554205554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2890102d40 msgr2=0x7f28901009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:54:57.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.805+0000 7f28953b0640 1 --2- 172.21.15.172:0/554205554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2890102d40 0x7f28901009a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2884005cc0 tx=0x7f288402f650 comp rx=0 tx=0).stop 2024-09-10T08:54:57.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.805+0000 7f28953b0640 1 -- 172.21.15.172:0/554205554 shutdown_connections 2024-09-10T08:54:57.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.805+0000 7f28953b0640 1 --2- 172.21.15.172:0/554205554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f28780a2b80 0x7f28780a5040 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.805+0000 7f28953b0640 1 --2- 172.21.15.172:0/554205554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f289010c170 0x7f2890101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.805+0000 7f28953b0640 1 --2- 172.21.15.172:0/554205554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2890103660 0x7f2890100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.805+0000 7f28953b0640 1 --2- 172.21.15.172:0/554205554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2890102d40 0x7f28901009a0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:54:57.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.805+0000 7f28953b0640 1 -- 172.21.15.172:0/554205554 >> 172.21.15.172:0/554205554 conn(0x7f28900fc7a0 msgr2=0x7f28900fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:54:57.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.805+0000 7f28953b0640 1 -- 172.21.15.172:0/554205554 shutdown_connections 2024-09-10T08:54:57.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:54:57.805+0000 7f28953b0640 1 -- 172.21.15.172:0/554205554 wait complete. 2024-09-10T08:54:58.628 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:58 smithi172 ceph-mon[30854]: pgmap v612: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 MiB/s wr, 12 op/s 2024-09-10T08:54:58.629 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c1e21f5b', total seconds elapsed: 16.7 2024-09-10T08:54:58.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:58 smithi174 ceph-mon[34068]: pgmap v612: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 MiB/s wr, 12 op/s 2024-09-10T08:54:58.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:58 smithi191 ceph-mon[35314]: pgmap v612: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.2 MiB/s wr, 12 op/s 2024-09-10T08:54:59.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:54:59 smithi174 ceph-mon[34068]: pgmap v613: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.6 MiB/s wr, 12 op/s 2024-09-10T08:54:59.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:54:59 smithi191 ceph-mon[35314]: pgmap v613: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.6 MiB/s wr, 12 op/s 2024-09-10T08:54:59.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:54:59 smithi172 ceph-mon[30854]: pgmap v613: 129 pgs: 129 active+clean; 7.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.6 MiB/s wr, 12 op/s 2024-09-10T08:55:02.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:02 smithi174 ceph-mon[34068]: pgmap v614: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.2 MiB/s wr, 14 op/s 2024-09-10T08:55:02.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:02 smithi191 ceph-mon[35314]: pgmap v614: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.2 MiB/s wr, 14 op/s 2024-09-10T08:55:02.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:02 smithi172 ceph-mon[30854]: pgmap v614: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.2 MiB/s wr, 14 op/s 2024-09-10T08:55:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.436+0000 7f21e5159640 1 -- 172.21.15.172:0/2672113760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21e0103710 msgr2=0x7f21e0109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.436+0000 7f21e5159640 1 --2- 172.21.15.172:0/2672113760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21e0103710 0x7f21e0109fa0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009ca0 tx=0x7f21d802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.436+0000 7f21e5159640 1 -- 172.21.15.172:0/2672113760 shutdown_connections 2024-09-10T08:55:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.436+0000 7f21e5159640 1 --2- 172.21.15.172:0/2672113760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21e0103710 0x7f21e0109fa0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.436+0000 7f21e5159640 1 --2- 172.21.15.172:0/2672113760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21e0102d50 0x7f21e01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.436+0000 7f21e5159640 1 --2- 172.21.15.172:0/2672113760 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21e0101b60 0x7f21e0101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.436+0000 7f21e5159640 1 -- 172.21.15.172:0/2672113760 >> 172.21.15.172:0/2672113760 conn(0x7f21e00fd2e0 msgr2=0x7f21e00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.436+0000 7f21e5159640 1 -- 172.21.15.172:0/2672113760 shutdown_connections 2024-09-10T08:55:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.436+0000 7f21e5159640 1 -- 172.21.15.172:0/2672113760 wait complete. 2024-09-10T08:55:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.437+0000 7f21e5159640 1 Processor -- start 2024-09-10T08:55:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.437+0000 7f21e5159640 1 -- start start 2024-09-10T08:55:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21e5159640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21e0101b60 0x7f21e0197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21e5159640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21e0102d50 0x7f21e01981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21e5159640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21e0103710 0x7f21e019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21e5159640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f21e010d670 con 0x7f21e0102d50 2024-09-10T08:55:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21e5159640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21e010d4f0 con 0x7f21e0101b60 2024-09-10T08:55:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21e5159640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f21e010d7f0 con 0x7f21e0103710 2024-09-10T08:55:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21de59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21e0102d50 0x7f21e01981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21de59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21e0102d50 0x7f21e01981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46234/0 (socket says 172.21.15.172:46234) 2024-09-10T08:55:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21ded9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21e0101b60 0x7f21e0197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21de59c640 1 -- 172.21.15.172:0/2646777587 learned_addr learned my addr 172.21.15.172:0/2646777587 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21df59e640 1 --2- 172.21.15.172:0/2646777587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21e0103710 0x7f21e019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21ded9d640 1 -- 172.21.15.172:0/2646777587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21e0103710 msgr2=0x7f21e019f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21ded9d640 1 --2- 172.21.15.172:0/2646777587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21e0103710 0x7f21e019f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21ded9d640 1 -- 172.21.15.172:0/2646777587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21e0102d50 msgr2=0x7f21e01981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21ded9d640 1 --2- 172.21.15.172:0/2646777587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21e0102d50 0x7f21e01981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21ded9d640 1 -- 172.21.15.172:0/2646777587 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e019f8d0 con 0x7f21e0101b60 2024-09-10T08:55:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21de59c640 1 --2- 172.21.15.172:0/2646777587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21e0102d50 0x7f21e01981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.438+0000 7f21ded9d640 1 --2- 172.21.15.172:0/2646777587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21e0101b60 0x7f21e0197c90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f21d000d6a0 tx=0x7f21d000db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.439+0000 7f21c7fff640 1 -- 172.21.15.172:0/2646777587 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0017070 con 0x7f21e0101b60 2024-09-10T08:55:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.439+0000 7f21c7fff640 1 -- 172.21.15.172:0/2646777587 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21d0009cd0 con 0x7f21e0101b60 2024-09-10T08:55:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.439+0000 7f21c7fff640 1 -- 172.21.15.172:0/2646777587 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0003bf0 con 0x7f21e0101b60 2024-09-10T08:55:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.439+0000 7f21e5159640 1 -- 172.21.15.172:0/2646777587 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e019fbc0 con 0x7f21e0101b60 2024-09-10T08:55:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.439+0000 7f21e5159640 1 -- 172.21.15.172:0/2646777587 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e01a00c0 con 0x7f21e0101b60 2024-09-10T08:55:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.441+0000 7f21c7fff640 1 -- 172.21.15.172:0/2646777587 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f21d0009820 con 0x7f21e0101b60 2024-09-10T08:55:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21e7fd1740 1 Processor -- start 2024-09-10T08:55:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21e7fd1740 1 -- start start 2024-09-10T08:55:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21c7fff640 1 --2- 172.21.15.172:0/2646777587 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f21c80a2c30 0x7f21c80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21e7fd1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a3db0 0x5630074a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21df59e640 1 --2- 172.21.15.172:0/2646777587 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f21c80a2c30 0x7f21c80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21e7fd1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a46f0 0x5630074a91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21e7fd1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a9780 0x5630074abc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21c7fff640 1 -- 172.21.15.172:0/2646777587 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f21d00cefe0 con 0x7f21e0101b60 2024-09-10T08:55:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21e7fd1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5630074032f0 con 0x5630074a3db0 2024-09-10T08:55:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21e7fd1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5630073ab270 con 0x5630074a46f0 2024-09-10T08:55:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21e7fd1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5630073ff2b0 con 0x5630074a9780 2024-09-10T08:55:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21ded9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a3db0 0x5630074a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21ded9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a3db0 0x5630074a41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46244/0 (socket says 172.21.15.172:46244) 2024-09-10T08:55:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21ded9d640 1 -- 172.21.15.172:0/851304206 learned_addr learned my addr 172.21.15.172:0/851304206 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21de59c640 1 --2- 172.21.15.172:0/851304206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a46f0 0x5630074a91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21ded9d640 1 -- 172.21.15.172:0/851304206 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a9780 msgr2=0x5630074abc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21ded9d640 1 --2- 172.21.15.172:0/851304206 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a9780 0x5630074abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21ded9d640 1 -- 172.21.15.172:0/851304206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a46f0 msgr2=0x5630074a91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21ded9d640 1 --2- 172.21.15.172:0/851304206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a46f0 0x5630074a91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21ded9d640 1 -- 172.21.15.172:0/851304206 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630073aa340 con 0x5630074a3db0 2024-09-10T08:55:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21df59e640 1 --2- 172.21.15.172:0/2646777587 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f21c80a2c30 0x7f21c80a50f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f21d8005e30 tx=0x7f21d8009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.442+0000 7f21ded9d640 1 --2- 172.21.15.172:0/851304206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a3db0 0x5630074a41b0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f21d00979f0 tx=0x7f21d009afb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa40d5ee043a73a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.443+0000 7f21c57fa640 1 -- 172.21.15.172:0/851304206 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0020070 con 0x5630074a3db0 2024-09-10T08:55:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.443+0000 7f21c57fa640 1 -- 172.21.15.172:0/851304206 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21d009f890 con 0x5630074a3db0 2024-09-10T08:55:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.443+0000 7f21c57fa640 1 -- 172.21.15.172:0/851304206 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0097c00 con 0x5630074a3db0 2024-09-10T08:55:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.444+0000 7f21e7fd1740 1 -- 172.21.15.172:0/851304206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a3db0 msgr2=0x5630074a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.444+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/851304206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a3db0 0x5630074a41b0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f21d00979f0 tx=0x7f21d009afb0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.444+0000 7f21e7fd1740 1 -- 172.21.15.172:0/851304206 shutdown_connections 2024-09-10T08:55:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.444+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/851304206 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a9780 0x5630074abc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.444+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/851304206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a46f0 0x5630074a91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.444+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/851304206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a3db0 0x5630074a41b0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.444+0000 7f21e7fd1740 1 -- 172.21.15.172:0/851304206 >> 172.21.15.172:0/851304206 conn(0x56300749b010 msgr2=0x56300749b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.444+0000 7f21e7fd1740 1 -- 172.21.15.172:0/851304206 shutdown_connections 2024-09-10T08:55:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.444+0000 7f21e7fd1740 1 -- 172.21.15.172:0/851304206 wait complete. 2024-09-10T08:55:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.444+0000 7f21e7fd1740 1 -- start start 2024-09-10T08:55:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.444+0000 7f21e7fd1740 1 Processor -- start 2024-09-10T08:55:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21e7fd1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a3db0 0x5630074adf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21e7fd1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a46f0 0x5630074ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21e7fd1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a9780 0x5630074b5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21e7fd1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5630074ac6f0 con 0x5630074a46f0 2024-09-10T08:55:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21e7fd1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56300738c290 con 0x5630074a3db0 2024-09-10T08:55:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21e7fd1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5630074ac870 con 0x5630074a9780 2024-09-10T08:55:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21e7fd1740 10 client.0 _ll_register_callbacks cb 0x5630073bd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21de59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a46f0 0x5630074ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21de59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a46f0 0x5630074ae4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46252/0 (socket says 172.21.15.172:46252) 2024-09-10T08:55:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21de59c640 1 -- 172.21.15.172:0/2372994386 learned_addr learned my addr 172.21.15.172:0/2372994386 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21ded9d640 1 --2- 172.21.15.172:0/2372994386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a3db0 0x5630074adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:04.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21df59e640 1 --2- 172.21.15.172:0/2372994386 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a9780 0x5630074b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21de59c640 1 -- 172.21.15.172:0/2372994386 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a9780 msgr2=0x5630074b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21de59c640 1 --2- 172.21.15.172:0/2372994386 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a9780 0x5630074b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21de59c640 1 -- 172.21.15.172:0/2372994386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a3db0 msgr2=0x5630074adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21de59c640 1 --2- 172.21.15.172:0/2372994386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a3db0 0x5630074adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21de59c640 1 -- 172.21.15.172:0/2372994386 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563007116210 con 0x5630074a46f0 2024-09-10T08:55:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21de59c640 1 --2- 172.21.15.172:0/2372994386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a46f0 0x5630074ae4c0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f21d4004540 tx=0x7f21d40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d629ae373c496a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.445+0000 7f21c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.446+0000 7f21c2ff5640 1 -- 172.21.15.172:0/2372994386 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4011830 con 0x5630074a46f0 2024-09-10T08:55:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.446+0000 7f21c2ff5640 1 -- 172.21.15.172:0/2372994386 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f21d40119d0 con 0x5630074a46f0 2024-09-10T08:55:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.446+0000 7f21e7fd1740 1 -- 172.21.15.172:0/2372994386 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21d009f890 con 0x5630074a46f0 2024-09-10T08:55:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.446+0000 7f21c2ff5640 1 -- 172.21.15.172:0/2372994386 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f21d4011bd0 con 0x5630074a46f0 2024-09-10T08:55:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.446+0000 7f21e7fd1740 10 client.18036 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.446+0000 7f21e7fd1740 1 -- 172.21.15.172:0/2372994386 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5630071e2620 con 0x5630074a46f0 2024-09-10T08:55:04.454 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.446+0000 7f21c2ff5640 1 -- 172.21.15.172:0/2372994386 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21d009f890 con 0x5630074a46f0 2024-09-10T08:55:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.446+0000 7f21c2ff5640 1 -- 172.21.15.172:0/2372994386 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f21d4010440 con 0x5630074a46f0 2024-09-10T08:55:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.446+0000 7f21e7fd1740 10 client.18036 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.446+0000 7f21e7fd1740 10 client.18036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.446+0000 7f21e7fd1740 20 client.18036 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.446+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/2372994386 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5630074b9c00 0x5630074bc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.446+0000 7f21e7fd1740 4 client.18036 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:55:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.446+0000 7f21e7fd1740 1 -- 172.21.15.172:0/2372994386 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563007410a20 con 0x5630074b9c00 2024-09-10T08:55:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.447+0000 7f21ded9d640 1 --2- 172.21.15.172:0/2372994386 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5630074b9c00 0x5630074bc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.448+0000 7f21ded9d640 1 --2- 172.21.15.172:0/2372994386 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5630074b9c00 0x5630074bc030 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3da50abe46355919 server_cookie=e42deb8574d1b1d in_seq=0 out_seq=0 2024-09-10T08:55:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.448+0000 7f21c2ff5640 10 client.18036 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.448+0000 7f21c2ff5640 1 -- 172.21.15.172:0/2372994386 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563007410a20 con 0x5630074b9c00 2024-09-10T08:55:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.448+0000 7f21c2ff5640 10 client.18036 handle_command_reply: tid=0 2024-09-10T08:55:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.448+0000 7f21e7fd1740 1 client.18036 shutdown 2024-09-10T08:55:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.448+0000 7f21e7fd1740 1 -- 172.21.15.172:0/2372994386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a46f0 msgr2=0x5630074ae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.448+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/2372994386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a46f0 0x5630074ae4c0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f21d4004540 tx=0x7f21d40039c0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.449+0000 7f21e7fd1740 1 -- 172.21.15.172:0/2372994386 shutdown_connections 2024-09-10T08:55:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.449+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/2372994386 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5630074b9c00 0x5630074bc030 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.449+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/2372994386 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a9780 0x5630074b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.449+0000 7f21c2ff5640 0 client.18036 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.449+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/2372994386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a46f0 0x5630074ae4c0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.449+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/2372994386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a3db0 0x5630074adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.449+0000 7f21e7fd1740 1 -- 172.21.15.172:0/2372994386 >> 172.21.15.172:0/2372994386 conn(0x56300749b010 msgr2=0x56300749c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.449+0000 7f21e7fd1740 1 -- 172.21.15.172:0/2372994386 shutdown_connections 2024-09-10T08:55:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.449+0000 7f21e7fd1740 1 -- 172.21.15.172:0/2372994386 wait complete. 2024-09-10T08:55:04.457 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.449+0000 7f21e7fd1740 20 client.18036 trim_cache size 0 max 16384 2024-09-10T08:55:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.466+0000 7f21e7fd1740 1 Processor -- start 2024-09-10T08:55:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.466+0000 7f21e7fd1740 1 -- start start 2024-09-10T08:55:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.466+0000 7f21e7fd1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a3db0 0x56300749a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.466+0000 7f21e7fd1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a46f0 0x5630074b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.466+0000 7f21e7fd1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a9780 0x56300749c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.469 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.466+0000 7f21e7fd1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01a83d0 con 0x5630074a3db0 2024-09-10T08:55:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.466+0000 7f21e7fd1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01a9f10 con 0x5630074a46f0 2024-09-10T08:55:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.466+0000 7f21e7fd1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01a8aa0 con 0x5630074a9780 2024-09-10T08:55:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.466+0000 7f21ded9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a3db0 0x56300749a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.467+0000 7f21ded9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a3db0 0x56300749a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46258/0 (socket says 172.21.15.172:46258) 2024-09-10T08:55:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.467+0000 7f21ded9d640 1 -- 172.21.15.172:0/2243007641 learned_addr learned my addr 172.21.15.172:0/2243007641 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.467+0000 7f21df59e640 1 --2- 172.21.15.172:0/2243007641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a9780 0x56300749c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.467+0000 7f21de59c640 1 --2- 172.21.15.172:0/2243007641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a46f0 0x5630074b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.467+0000 7f21ded9d640 1 -- 172.21.15.172:0/2243007641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a9780 msgr2=0x56300749c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.467+0000 7f21ded9d640 1 --2- 172.21.15.172:0/2243007641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a9780 0x56300749c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.467+0000 7f21ded9d640 1 -- 172.21.15.172:0/2243007641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a46f0 msgr2=0x5630074b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.467+0000 7f21ded9d640 1 --2- 172.21.15.172:0/2243007641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a46f0 0x5630074b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.467+0000 7f21ded9d640 1 -- 172.21.15.172:0/2243007641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630074a4bd0 con 0x5630074a3db0 2024-09-10T08:55:04.471 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.467+0000 7f21ded9d640 1 --2- 172.21.15.172:0/2243007641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a3db0 0x56300749a0b0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009610 tx=0x7f21d009b020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7413e00c3966e86d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:04.471 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.467+0000 7f21c4ff9640 1 -- 172.21.15.172:0/2243007641 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0020040 con 0x5630074a3db0 2024-09-10T08:55:04.471 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.467+0000 7f21c4ff9640 1 -- 172.21.15.172:0/2243007641 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21d0099890 con 0x5630074a3db0 2024-09-10T08:55:04.471 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.467+0000 7f21c4ff9640 1 -- 172.21.15.172:0/2243007641 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d00a3cc0 con 0x5630074a3db0 2024-09-10T08:55:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.468+0000 7f21e7fd1740 1 -- 172.21.15.172:0/2243007641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a3db0 msgr2=0x56300749a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.468+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/2243007641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a3db0 0x56300749a0b0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009610 tx=0x7f21d009b020 comp rx=0 tx=0).stop 2024-09-10T08:55:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.468+0000 7f21e7fd1740 1 -- 172.21.15.172:0/2243007641 shutdown_connections 2024-09-10T08:55:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.468+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/2243007641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a9780 0x56300749c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.468+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/2243007641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a46f0 0x5630074b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.468+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/2243007641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a3db0 0x56300749a0b0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.468+0000 7f21e7fd1740 1 -- 172.21.15.172:0/2243007641 >> 172.21.15.172:0/2243007641 conn(0x56300749b010 msgr2=0x5630074999a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:04.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.468+0000 7f21e7fd1740 1 -- 172.21.15.172:0/2243007641 shutdown_connections 2024-09-10T08:55:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.468+0000 7f21e7fd1740 1 -- 172.21.15.172:0/2243007641 wait complete. 2024-09-10T08:55:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.468+0000 7f21e7fd1740 1 -- start start 2024-09-10T08:55:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21e7fd1740 1 Processor -- start 2024-09-10T08:55:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21e7fd1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a3db0 0x5630074b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21e7fd1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a46f0 0x5630074b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21e7fd1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a9780 0x5630074b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21e7fd1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01a9840 con 0x5630074a9780 2024-09-10T08:55:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21e7fd1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01a9170 con 0x5630074a3db0 2024-09-10T08:55:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21e7fd1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f21e01a63b0 con 0x5630074a46f0 2024-09-10T08:55:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21e7fd1740 10 client.0 _ll_register_callbacks cb 0x7f21d400be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:04.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21df59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a9780 0x5630074b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21df59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a9780 0x5630074b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46274/0 (socket says 172.21.15.172:46274) 2024-09-10T08:55:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21df59e640 1 -- 172.21.15.172:0/415222849 learned_addr learned my addr 172.21.15.172:0/415222849 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21ded9d640 1 --2- 172.21.15.172:0/415222849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a3db0 0x5630074b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21de59c640 1 --2- 172.21.15.172:0/415222849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a46f0 0x5630074b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21df59e640 1 -- 172.21.15.172:0/415222849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a46f0 msgr2=0x5630074b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21df59e640 1 --2- 172.21.15.172:0/415222849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a46f0 0x5630074b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21df59e640 1 -- 172.21.15.172:0/415222849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a3db0 msgr2=0x5630074b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21df59e640 1 --2- 172.21.15.172:0/415222849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a3db0 0x5630074b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21df59e640 1 -- 172.21.15.172:0/415222849 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630074b47a0 con 0x5630074a9780 2024-09-10T08:55:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21df59e640 1 --2- 172.21.15.172:0/415222849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a9780 0x5630074b4190 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009ca0 tx=0x7f21d802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca56de2ef105f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.469+0000 7f21c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.470+0000 7f21c2ff5640 1 -- 172.21.15.172:0/415222849 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d8042070 con 0x5630074a9780 2024-09-10T08:55:04.476 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.470+0000 7f21c2ff5640 1 -- 172.21.15.172:0/415222849 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f21d8004500 con 0x5630074a9780 2024-09-10T08:55:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.470+0000 7f21e7fd1740 1 -- 172.21.15.172:0/415222849 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21d0099890 con 0x5630074a9780 2024-09-10T08:55:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.470+0000 7f21c2ff5640 1 -- 172.21.15.172:0/415222849 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f21d8004700 con 0x5630074a9780 2024-09-10T08:55:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.470+0000 7f21e7fd1740 10 client.18042 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.470+0000 7f21e7fd1740 1 -- 172.21.15.172:0/415222849 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563007524a30 con 0x5630074a9780 2024-09-10T08:55:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.470+0000 7f21c2ff5640 1 -- 172.21.15.172:0/415222849 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21d0099890 con 0x5630074a9780 2024-09-10T08:55:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.470+0000 7f21c2ff5640 1 -- 172.21.15.172:0/415222849 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f21d8052020 con 0x5630074a9780 2024-09-10T08:55:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.470+0000 7f21e7fd1740 10 client.18042 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.470+0000 7f21e7fd1740 10 client.18042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.470+0000 7f21e7fd1740 20 client.18042 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.470+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/415222849 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5630075253c0 0x5630075277e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.470+0000 7f21e7fd1740 4 client.18042 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:55:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.470+0000 7f21e7fd1740 1 -- 172.21.15.172:0/415222849 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56300740ad70 con 0x5630075253c0 2024-09-10T08:55:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.471+0000 7f21ded9d640 1 --2- 172.21.15.172:0/415222849 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5630075253c0 0x5630075277e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.471+0000 7f21c2ff5640 10 client.18042 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.471+0000 7f21ded9d640 1 --2- 172.21.15.172:0/415222849 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5630075253c0 0x5630075277e0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a186210853b26b0 server_cookie=6558b4b5301e8b85 in_seq=0 out_seq=0 2024-09-10T08:55:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.475+0000 7f21c2ff5640 1 -- 172.21.15.172:0/415222849 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56300740ad70 con 0x5630075253c0 2024-09-10T08:55:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.475+0000 7f21c2ff5640 10 client.18042 handle_command_reply: tid=0 2024-09-10T08:55:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.475+0000 7f21e7fd1740 1 client.18042 shutdown 2024-09-10T08:55:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.475+0000 7f21e7fd1740 1 -- 172.21.15.172:0/415222849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a9780 msgr2=0x5630074b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.475+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/415222849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a9780 0x5630074b4190 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009ca0 tx=0x7f21d802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.475+0000 7f21e7fd1740 1 -- 172.21.15.172:0/415222849 shutdown_connections 2024-09-10T08:55:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.475+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/415222849 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5630075253c0 0x5630075277e0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.475+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/415222849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630074a9780 0x5630074b4190 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.475+0000 7f21c2ff5640 0 client.18042 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.475+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/415222849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5630074a46f0 0x5630074b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.475+0000 7f21e7fd1740 1 --2- 172.21.15.172:0/415222849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630074a3db0 0x5630074b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.475+0000 7f21e7fd1740 1 -- 172.21.15.172:0/415222849 >> 172.21.15.172:0/415222849 conn(0x56300749b010 msgr2=0x56300749a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.475+0000 7f21e7fd1740 1 -- 172.21.15.172:0/415222849 shutdown_connections 2024-09-10T08:55:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.475+0000 7f21e7fd1740 1 -- 172.21.15.172:0/415222849 wait complete. 2024-09-10T08:55:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.475+0000 7f21e7fd1740 20 client.18042 trim_cache size 0 max 16384 2024-09-10T08:55:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:55:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:55:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:55:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:55:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.476+0000 7f21e5159640 1 -- 172.21.15.172:0/2646777587 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f21c80a2c30 msgr2=0x7f21c80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.476+0000 7f21e5159640 1 --2- 172.21.15.172:0/2646777587 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f21c80a2c30 0x7f21c80a50f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f21d8005e30 tx=0x7f21d8009800 comp rx=0 tx=0).stop 2024-09-10T08:55:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.476+0000 7f21e5159640 1 -- 172.21.15.172:0/2646777587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21e0101b60 msgr2=0x7f21e0197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.476+0000 7f21e5159640 1 --2- 172.21.15.172:0/2646777587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21e0101b60 0x7f21e0197c90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f21d000d6a0 tx=0x7f21d000db60 comp rx=0 tx=0).stop 2024-09-10T08:55:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.476+0000 7f21e5159640 1 -- 172.21.15.172:0/2646777587 shutdown_connections 2024-09-10T08:55:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.476+0000 7f21e5159640 1 --2- 172.21.15.172:0/2646777587 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f21c80a2c30 0x7f21c80a50f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.476+0000 7f21e5159640 1 --2- 172.21.15.172:0/2646777587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21e0103710 0x7f21e019f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.476+0000 7f21e5159640 1 --2- 172.21.15.172:0/2646777587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f21e0102d50 0x7f21e01981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.476+0000 7f21e5159640 1 --2- 172.21.15.172:0/2646777587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21e0101b60 0x7f21e0197c90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.476+0000 7f21e5159640 1 -- 172.21.15.172:0/2646777587 >> 172.21.15.172:0/2646777587 conn(0x7f21e00fd2e0 msgr2=0x7f21e00ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.476+0000 7f21e5159640 1 -- 172.21.15.172:0/2646777587 shutdown_connections 2024-09-10T08:55:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:04.476+0000 7f21e5159640 1 -- 172.21.15.172:0/2646777587 wait complete. 2024-09-10T08:55:04.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:04 smithi174 ceph-mon[34068]: pgmap v615: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-09-10T08:55:04.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:04 smithi191 ceph-mon[35314]: pgmap v615: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-09-10T08:55:04.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:04 smithi172 ceph-mon[30854]: pgmap v615: 129 pgs: 129 active+clean; 7.9 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.7 MiB/s wr, 12 op/s 2024-09-10T08:55:06.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:06 smithi174 ceph-mon[34068]: pgmap v616: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 14 MiB/s wr, 19 op/s 2024-09-10T08:55:06.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:06 smithi191 ceph-mon[35314]: pgmap v616: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 14 MiB/s wr, 19 op/s 2024-09-10T08:55:06.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:06 smithi172 ceph-mon[30854]: pgmap v616: 129 pgs: 129 active+clean; 8.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 14 MiB/s wr, 19 op/s 2024-09-10T08:55:08.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:08 smithi174 ceph-mon[34068]: pgmap v617: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 18 MiB/s wr, 16 op/s 2024-09-10T08:55:08.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:08 smithi191 ceph-mon[35314]: pgmap v617: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 18 MiB/s wr, 16 op/s 2024-09-10T08:55:08.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:08 smithi172 ceph-mon[30854]: pgmap v617: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 18 MiB/s wr, 16 op/s 2024-09-10T08:55:10.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:10 smithi174 ceph-mon[34068]: pgmap v618: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 20 MiB/s wr, 17 op/s 2024-09-10T08:55:10.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:10 smithi191 ceph-mon[35314]: pgmap v618: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 20 MiB/s wr, 17 op/s 2024-09-10T08:55:10.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:10 smithi172 ceph-mon[30854]: pgmap v618: 129 pgs: 129 active+clean; 8.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 20 MiB/s wr, 17 op/s 2024-09-10T08:55:11.804 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:55:12.086 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:55:12.537 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:12 smithi172 ceph-mon[30854]: pgmap v619: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-09-10T08:55:12.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.633+0000 7f91716b5640 1 -- 172.21.15.172:0/1879895209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f916c108300 msgr2=0x7f916c10ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:12.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.633+0000 7f91716b5640 1 --2- 172.21.15.172:0/1879895209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f916c108300 0x7f916c10ec70 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f9164009cd0 tx=0x7f916402a8d0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.633+0000 7f91716b5640 1 -- 172.21.15.172:0/1879895209 shutdown_connections 2024-09-10T08:55:12.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.633+0000 7f91716b5640 1 --2- 172.21.15.172:0/1879895209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f916c108300 0x7f916c10ec70 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.633+0000 7f91716b5640 1 --2- 172.21.15.172:0/1879895209 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f916c103780 0x7f916c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.633+0000 7f91716b5640 1 --2- 172.21.15.172:0/1879895209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f916c102dd0 0x7f916c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.633+0000 7f91716b5640 1 -- 172.21.15.172:0/1879895209 >> 172.21.15.172:0/1879895209 conn(0x7f916c0fc7d0 msgr2=0x7f916c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:12.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.633+0000 7f91716b5640 1 -- 172.21.15.172:0/1879895209 shutdown_connections 2024-09-10T08:55:12.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.633+0000 7f91716b5640 1 -- 172.21.15.172:0/1879895209 wait complete. 2024-09-10T08:55:12.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.634+0000 7f91716b5640 1 Processor -- start 2024-09-10T08:55:12.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.634+0000 7f91716b5640 1 -- start start 2024-09-10T08:55:12.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f91716b5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f916c102dd0 0x7f916c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f91716b5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f916c103780 0x7f916c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f91716b5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f916c108300 0x7f916c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f91716b5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f916c111fd0 con 0x7f916c108300 2024-09-10T08:55:12.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f91716b5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f916c111e50 con 0x7f916c103780 2024-09-10T08:55:12.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f91716b5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f916c112150 con 0x7f916c102dd0 2024-09-10T08:55:12.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f916affd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f916c102dd0 0x7f916c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f916b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f916c108300 0x7f916c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f916affd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f916c102dd0 0x7f916c1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:48588/0 (socket says 172.21.15.172:48588) 2024-09-10T08:55:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f916affd640 1 -- 172.21.15.172:0/1193383281 learned_addr learned my addr 172.21.15.172:0/1193383281 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f916a7fc640 1 --2- 172.21.15.172:0/1193383281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f916c103780 0x7f916c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f916affd640 1 -- 172.21.15.172:0/1193383281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f916c103780 msgr2=0x7f916c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f916affd640 1 --2- 172.21.15.172:0/1193383281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f916c103780 0x7f916c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f916affd640 1 -- 172.21.15.172:0/1193383281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f916c108300 msgr2=0x7f916c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f916affd640 1 --2- 172.21.15.172:0/1193383281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f916c108300 0x7f916c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f916affd640 1 -- 172.21.15.172:0/1193383281 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916c19d3a0 con 0x7f916c102dd0 2024-09-10T08:55:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f916b7fe640 1 --2- 172.21.15.172:0/1193383281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f916c108300 0x7f916c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.635+0000 7f916affd640 1 --2- 172.21.15.172:0/1193383281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f916c102dd0 0x7f916c1983f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f915c00d950 tx=0x7f915c00dd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.636+0000 7f9153fff640 1 -- 172.21.15.172:0/1193383281 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c017070 con 0x7f916c102dd0 2024-09-10T08:55:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.636+0000 7f9153fff640 1 -- 172.21.15.172:0/1193383281 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f915c00de80 con 0x7f916c102dd0 2024-09-10T08:55:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.636+0000 7f91716b5640 1 -- 172.21.15.172:0/1193383281 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916c19d690 con 0x7f916c102dd0 2024-09-10T08:55:12.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.636+0000 7f9153fff640 1 -- 172.21.15.172:0/1193383281 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c00f620 con 0x7f916c102dd0 2024-09-10T08:55:12.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.636+0000 7f91716b5640 1 -- 172.21.15.172:0/1193383281 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f916c1a4f70 con 0x7f916c102dd0 2024-09-10T08:55:12.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.638+0000 7f9153fff640 1 -- 172.21.15.172:0/1193383281 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f915c00f7c0 con 0x7f916c102dd0 2024-09-10T08:55:12.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f9173a8e740 1 Processor -- start 2024-09-10T08:55:12.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f9173a8e740 1 -- start start 2024-09-10T08:55:12.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f9173a8e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984370 0x557ace984750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f9173a8e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace984c90 0x557ace993540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f9173a8e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace993a80 0x557ace995e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f9173a8e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557ace8ea2e0 con 0x557ace984370 2024-09-10T08:55:12.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f9173a8e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557ace891ce0 con 0x557ace984c90 2024-09-10T08:55:12.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f9173a8e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557ace8e62a0 con 0x557ace993a80 2024-09-10T08:55:12.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f916b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984370 0x557ace984750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f9153fff640 1 --2- 172.21.15.172:0/1193383281 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f91540a2b80 0x7f91540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f916b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984370 0x557ace984750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55950/0 (socket says 172.21.15.172:55950) 2024-09-10T08:55:12.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f916b7fe640 1 -- 172.21.15.172:0/3188409989 learned_addr learned my addr 172.21.15.172:0/3188409989 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:12.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f916a7fc640 1 --2- 172.21.15.172:0/3188409989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace993a80 0x557ace995e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f916affd640 1 --2- 172.21.15.172:0/3188409989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace984c90 0x557ace993540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f916b7fe640 1 --2- 172.21.15.172:0/1193383281 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f91540a2b80 0x7f91540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f916a7fc640 1 -- 172.21.15.172:0/3188409989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace984c90 msgr2=0x557ace993540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f9153fff640 1 -- 172.21.15.172:0/1193383281 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f915c0cef70 con 0x7f916c102dd0 2024-09-10T08:55:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f916a7fc640 1 --2- 172.21.15.172:0/3188409989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace984c90 0x557ace993540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f916a7fc640 1 -- 172.21.15.172:0/3188409989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984370 msgr2=0x557ace984750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f916a7fc640 1 --2- 172.21.15.172:0/3188409989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984370 0x557ace984750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f916a7fc640 1 -- 172.21.15.172:0/3188409989 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ace876440 con 0x557ace993a80 2024-09-10T08:55:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.639+0000 7f916affd640 1 --2- 172.21.15.172:0/3188409989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace984c90 0x557ace993540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.640+0000 7f916a7fc640 1 --2- 172.21.15.172:0/3188409989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace993a80 0x557ace995e70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f916c06a2d0 tx=0x7f916000f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76f5854a862f694d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.640+0000 7f91517fa640 1 -- 172.21.15.172:0/3188409989 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9160005020 con 0x557ace993a80 2024-09-10T08:55:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.640+0000 7f91517fa640 1 -- 172.21.15.172:0/3188409989 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91600047a0 con 0x557ace993a80 2024-09-10T08:55:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.640+0000 7f916b7fe640 1 --2- 172.21.15.172:0/1193383281 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f91540a2b80 0x7f91540a5040 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9164005fe0 tx=0x7f916402a8d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.640+0000 7f91517fa640 1 -- 172.21.15.172:0/3188409989 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9160018730 con 0x557ace993a80 2024-09-10T08:55:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.641+0000 7f9173a8e740 1 -- 172.21.15.172:0/3188409989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace993a80 msgr2=0x557ace995e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.641+0000 7f9173a8e740 1 --2- 172.21.15.172:0/3188409989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace993a80 0x557ace995e70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f916c06a2d0 tx=0x7f916000f3c0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.641+0000 7f9173a8e740 1 -- 172.21.15.172:0/3188409989 shutdown_connections 2024-09-10T08:55:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.641+0000 7f9173a8e740 1 --2- 172.21.15.172:0/3188409989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace993a80 0x557ace995e70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.641+0000 7f9173a8e740 1 --2- 172.21.15.172:0/3188409989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace984c90 0x557ace993540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.641+0000 7f9173a8e740 1 --2- 172.21.15.172:0/3188409989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984370 0x557ace984750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.641+0000 7f9173a8e740 1 -- 172.21.15.172:0/3188409989 >> 172.21.15.172:0/3188409989 conn(0x557ace980a20 msgr2=0x557ace980e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:12.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.641+0000 7f9173a8e740 1 -- 172.21.15.172:0/3188409989 shutdown_connections 2024-09-10T08:55:12.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.641+0000 7f9173a8e740 1 -- 172.21.15.172:0/3188409989 wait complete. 2024-09-10T08:55:12.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.641+0000 7f9173a8e740 1 -- start start 2024-09-10T08:55:12.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.641+0000 7f9173a8e740 1 Processor -- start 2024-09-10T08:55:12.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f9173a8e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace984370 0x557ace9933b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f9173a8e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984c90 0x557ace990660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f9173a8e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace993a80 0x557ace990ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f9173a8e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557ace996650 con 0x557ace984c90 2024-09-10T08:55:12.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f9173a8e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557ace8a5050 con 0x557ace993a80 2024-09-10T08:55:12.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f9173a8e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557ace9967d0 con 0x557ace984370 2024-09-10T08:55:12.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f916a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984c90 0x557ace990660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f9173a8e740 10 client.0 _ll_register_callbacks cb 0x557ace8e5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:12.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f916affd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace984370 0x557ace9933b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f916a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984c90 0x557ace990660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55962/0 (socket says 172.21.15.172:55962) 2024-09-10T08:55:12.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f916a7fc640 1 -- 172.21.15.172:0/3782559190 learned_addr learned my addr 172.21.15.172:0/3782559190 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:12.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f916b7fe640 1 --2- 172.21.15.172:0/3782559190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace993a80 0x557ace990ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f916affd640 1 -- 172.21.15.172:0/3782559190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace993a80 msgr2=0x557ace990ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:12.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f916affd640 1 --2- 172.21.15.172:0/3782559190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace993a80 0x557ace990ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f916affd640 1 -- 172.21.15.172:0/3782559190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984c90 msgr2=0x557ace990660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:12.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f916affd640 1 --2- 172.21.15.172:0/3782559190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984c90 0x557ace990660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f916affd640 1 -- 172.21.15.172:0/3782559190 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ace65b3c0 con 0x557ace984370 2024-09-10T08:55:12.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f916a7fc640 1 --2- 172.21.15.172:0/3782559190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984c90 0x557ace990660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:12.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f916affd640 1 --2- 172.21.15.172:0/3782559190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace984370 0x557ace9933b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x557ace987080 tx=0x7f915c06c290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d6063b950e72d18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:12.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.642+0000 7f914eff5640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:55:12.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.643+0000 7f914eff5640 1 -- 172.21.15.172:0/3782559190 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c020070 con 0x557ace984370 2024-09-10T08:55:12.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.643+0000 7f9173a8e740 1 -- 172.21.15.172:0/3782559190 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91600047a0 con 0x557ace984370 2024-09-10T08:55:12.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.643+0000 7f914eff5640 1 -- 172.21.15.172:0/3782559190 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f915c097ae0 con 0x557ace984370 2024-09-10T08:55:12.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.643+0000 7f914eff5640 1 -- 172.21.15.172:0/3782559190 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f915c09bb70 con 0x557ace984370 2024-09-10T08:55:12.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.643+0000 7f9173a8e740 10 client.26998 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:12.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.643+0000 7f9173a8e740 1 -- 172.21.15.172:0/3782559190 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557ace9a2000 con 0x557ace984370 2024-09-10T08:55:12.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.643+0000 7f914eff5640 1 -- 172.21.15.172:0/3782559190 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91600047a0 con 0x557ace984370 2024-09-10T08:55:12.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.643+0000 7f914eff5640 1 -- 172.21.15.172:0/3782559190 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f915c0d10d0 con 0x557ace984370 2024-09-10T08:55:12.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.643+0000 7f9173a8e740 10 client.26998 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:12.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.643+0000 7f9173a8e740 10 client.26998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:12.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.643+0000 7f9173a8e740 20 client.26998 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:12.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.643+0000 7f9173a8e740 1 --2- 172.21.15.172:0/3782559190 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557ace9a3820 0x557ace9a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.643+0000 7f9173a8e740 4 client.26998 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:55:12.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.643+0000 7f9173a8e740 1 -- 172.21.15.172:0/3782559190 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ace8f7850 con 0x557ace9a3820 2024-09-10T08:55:12.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.643+0000 7f916a7fc640 1 --2- 172.21.15.172:0/3782559190 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557ace9a3820 0x557ace9a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.644+0000 7f916a7fc640 1 --2- 172.21.15.172:0/3782559190 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557ace9a3820 0x557ace9a5ce0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a92dca2255ae8e8d server_cookie=4d5ceec06b0adb19 in_seq=0 out_seq=0 2024-09-10T08:55:12.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.644+0000 7f914eff5640 10 client.26998 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:12.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.644+0000 7f914eff5640 1 -- 172.21.15.172:0/3782559190 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ace8f7850 con 0x557ace9a3820 2024-09-10T08:55:12.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.644+0000 7f914eff5640 10 client.26998 handle_command_reply: tid=0 2024-09-10T08:55:12.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.644+0000 7f9173a8e740 1 client.26998 shutdown 2024-09-10T08:55:12.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.645+0000 7f9173a8e740 1 -- 172.21.15.172:0/3782559190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace984370 msgr2=0x557ace9933b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:12.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.645+0000 7f9173a8e740 1 --2- 172.21.15.172:0/3782559190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace984370 0x557ace9933b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x557ace987080 tx=0x7f915c06c290 comp rx=0 tx=0).stop 2024-09-10T08:55:12.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.645+0000 7f9173a8e740 1 -- 172.21.15.172:0/3782559190 shutdown_connections 2024-09-10T08:55:12.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.645+0000 7f9173a8e740 1 --2- 172.21.15.172:0/3782559190 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557ace9a3820 0x557ace9a5ce0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.645+0000 7f9173a8e740 1 --2- 172.21.15.172:0/3782559190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace993a80 0x557ace990ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.645+0000 7f9173a8e740 1 --2- 172.21.15.172:0/3782559190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984c90 0x557ace990660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.645+0000 7f9173a8e740 1 --2- 172.21.15.172:0/3782559190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace984370 0x557ace9933b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.645+0000 7f9173a8e740 1 -- 172.21.15.172:0/3782559190 >> 172.21.15.172:0/3782559190 conn(0x557ace980a20 msgr2=0x557ace986780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.645+0000 7f914eff5640 0 client.26998 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.645+0000 7f9173a8e740 1 -- 172.21.15.172:0/3782559190 shutdown_connections 2024-09-10T08:55:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.645+0000 7f9173a8e740 1 -- 172.21.15.172:0/3782559190 wait complete. 2024-09-10T08:55:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.645+0000 7f9173a8e740 20 client.26998 trim_cache size 0 max 16384 2024-09-10T08:55:12.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f9173a8e740 1 Processor -- start 2024-09-10T08:55:12.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f9173a8e740 1 -- start start 2024-09-10T08:55:12.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f9173a8e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace980a20 0x557ace99f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f9173a8e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace984370 0x557ace98ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f9173a8e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984c90 0x557ace97fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f9173a8e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f916c1ac940 con 0x557ace984c90 2024-09-10T08:55:12.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f9173a8e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f916c1ae480 con 0x557ace984370 2024-09-10T08:55:12.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f9173a8e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f916c1ad010 con 0x557ace980a20 2024-09-10T08:55:12.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f916b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984c90 0x557ace97fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f916b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984c90 0x557ace97fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55976/0 (socket says 172.21.15.172:55976) 2024-09-10T08:55:12.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f916b7fe640 1 -- 172.21.15.172:0/461532766 learned_addr learned my addr 172.21.15.172:0/461532766 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:12.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f916affd640 1 --2- 172.21.15.172:0/461532766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace980a20 0x557ace99f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f916a7fc640 1 --2- 172.21.15.172:0/461532766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace984370 0x557ace98ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f916b7fe640 1 -- 172.21.15.172:0/461532766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace980a20 msgr2=0x557ace99f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:12.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f916b7fe640 1 --2- 172.21.15.172:0/461532766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace980a20 0x557ace99f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f916b7fe640 1 -- 172.21.15.172:0/461532766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace984370 msgr2=0x557ace98ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:12.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f916b7fe640 1 --2- 172.21.15.172:0/461532766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace984370 0x557ace98ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.665+0000 7f916b7fe640 1 -- 172.21.15.172:0/461532766 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ace98f610 con 0x557ace984c90 2024-09-10T08:55:12.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.666+0000 7f916b7fe640 1 --2- 172.21.15.172:0/461532766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984c90 0x557ace97fa90 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f9164009cd0 tx=0x7f9164005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3ab5619dcc1190 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:12.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.666+0000 7f9150ff9640 1 -- 172.21.15.172:0/461532766 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9164003d70 con 0x557ace984c90 2024-09-10T08:55:12.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.666+0000 7f9150ff9640 1 -- 172.21.15.172:0/461532766 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9164036050 con 0x557ace984c90 2024-09-10T08:55:12.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.666+0000 7f9150ff9640 1 -- 172.21.15.172:0/461532766 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9164030ae0 con 0x557ace984c90 2024-09-10T08:55:12.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.667+0000 7f9173a8e740 1 -- 172.21.15.172:0/461532766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984c90 msgr2=0x557ace97fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:12.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.667+0000 7f9173a8e740 1 --2- 172.21.15.172:0/461532766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984c90 0x557ace97fa90 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f9164009cd0 tx=0x7f9164005ef0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.667+0000 7f9173a8e740 1 -- 172.21.15.172:0/461532766 shutdown_connections 2024-09-10T08:55:12.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.667+0000 7f9173a8e740 1 --2- 172.21.15.172:0/461532766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace984c90 0x557ace97fa90 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.667+0000 7f9173a8e740 1 --2- 172.21.15.172:0/461532766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace984370 0x557ace98ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.667+0000 7f9173a8e740 1 --2- 172.21.15.172:0/461532766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace980a20 0x557ace99f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.667+0000 7f9173a8e740 1 -- 172.21.15.172:0/461532766 >> 172.21.15.172:0/461532766 conn(0x557ace990d00 msgr2=0x557ace97f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:12.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.667+0000 7f9173a8e740 1 -- 172.21.15.172:0/461532766 shutdown_connections 2024-09-10T08:55:12.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.667+0000 7f9173a8e740 1 -- 172.21.15.172:0/461532766 wait complete. 2024-09-10T08:55:12.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.667+0000 7f9173a8e740 1 -- start start 2024-09-10T08:55:12.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.667+0000 7f9173a8e740 1 Processor -- start 2024-09-10T08:55:12.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f9173a8e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace980a20 0x557ace997870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f9173a8e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace984370 0x557ace997db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f9173a8e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace984c90 0x557ace9982f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f916affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace980a20 0x557ace997870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f916affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace980a20 0x557ace997870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55988/0 (socket says 172.21.15.172:55988) 2024-09-10T08:55:12.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f916affd640 1 -- 172.21.15.172:0/1768158459 learned_addr learned my addr 172.21.15.172:0/1768158459 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:12.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f9173a8e740 1 -- 172.21.15.172:0/1768158459 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f916c1addb0 con 0x557ace980a20 2024-09-10T08:55:12.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f9173a8e740 1 -- 172.21.15.172:0/1768158459 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f916c1ad6e0 con 0x557ace984370 2024-09-10T08:55:12.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f9173a8e740 1 -- 172.21.15.172:0/1768158459 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f916c1aa920 con 0x557ace984c90 2024-09-10T08:55:12.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f9173a8e740 10 client.0 _ll_register_callbacks cb 0x7f915c097710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:12.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f916b7fe640 1 --2- 172.21.15.172:0/1768158459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace984c90 0x557ace9982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f916a7fc640 1 --2- 172.21.15.172:0/1768158459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace984370 0x557ace997db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f916affd640 1 -- 172.21.15.172:0/1768158459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace984c90 msgr2=0x557ace9982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:12.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f916affd640 1 --2- 172.21.15.172:0/1768158459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace984c90 0x557ace9982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f916affd640 1 -- 172.21.15.172:0/1768158459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace984370 msgr2=0x557ace997db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:12.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f916affd640 1 --2- 172.21.15.172:0/1768158459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace984370 0x557ace997db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f916affd640 1 -- 172.21.15.172:0/1768158459 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ace984ac0 con 0x557ace980a20 2024-09-10T08:55:12.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f916b7fe640 1 --2- 172.21.15.172:0/1768158459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace984c90 0x557ace9982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:55:12.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f916a7fc640 1 --2- 172.21.15.172:0/1768158459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace984370 0x557ace997db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:55:12.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f916affd640 1 --2- 172.21.15.172:0/1768158459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace980a20 0x557ace997870 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f915c09b400 tx=0x7f915c097840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d88f8c7612347a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:12.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.668+0000 7f914eff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:12.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.669+0000 7f914eff5640 1 -- 172.21.15.172:0/1768158459 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c020040 con 0x557ace980a20 2024-09-10T08:55:12.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.669+0000 7f914eff5640 1 -- 172.21.15.172:0/1768158459 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f915c0ccdd0 con 0x557ace980a20 2024-09-10T08:55:12.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.669+0000 7f9173a8e740 1 -- 172.21.15.172:0/1768158459 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9164036050 con 0x557ace980a20 2024-09-10T08:55:12.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.669+0000 7f914eff5640 1 -- 172.21.15.172:0/1768158459 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f915c09f820 con 0x557ace980a20 2024-09-10T08:55:12.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.669+0000 7f9173a8e740 10 client.18060 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:12.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.669+0000 7f9173a8e740 1 -- 172.21.15.172:0/1768158459 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557acea0e770 con 0x557ace980a20 2024-09-10T08:55:12.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.669+0000 7f914eff5640 1 -- 172.21.15.172:0/1768158459 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9164036050 con 0x557ace980a20 2024-09-10T08:55:12.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.669+0000 7f914eff5640 1 -- 172.21.15.172:0/1768158459 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f915c06f040 con 0x557ace980a20 2024-09-10T08:55:12.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.669+0000 7f9173a8e740 10 client.18060 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:12.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.669+0000 7f9173a8e740 10 client.18060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:12.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.669+0000 7f9173a8e740 20 client.18060 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:12.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.669+0000 7f9173a8e740 1 --2- 172.21.15.172:0/1768158459 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557ace993a80 0x557acea10f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:12.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.669+0000 7f9173a8e740 4 client.18060 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:55:12.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.669+0000 7f9173a8e740 1 -- 172.21.15.172:0/1768158459 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557ace8f1d60 con 0x557ace993a80 2024-09-10T08:55:12.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.670+0000 7f916a7fc640 1 --2- 172.21.15.172:0/1768158459 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557ace993a80 0x557acea10f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:12.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.670+0000 7f914eff5640 10 client.18060 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:12.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.670+0000 7f916a7fc640 1 --2- 172.21.15.172:0/1768158459 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557ace993a80 0x557acea10f60 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33d95d36b788cf62 server_cookie=dbaf473f322388b0 in_seq=0 out_seq=0 2024-09-10T08:55:12.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.670+0000 7f914eff5640 1 -- 172.21.15.172:0/1768158459 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557ace8f1d60 con 0x557ace993a80 2024-09-10T08:55:12.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.670+0000 7f914eff5640 10 client.18060 handle_command_reply: tid=0 2024-09-10T08:55:12.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.670+0000 7f9173a8e740 1 client.18060 shutdown 2024-09-10T08:55:12.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.670+0000 7f9173a8e740 1 -- 172.21.15.172:0/1768158459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace980a20 msgr2=0x557ace997870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:12.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.670+0000 7f9173a8e740 1 --2- 172.21.15.172:0/1768158459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace980a20 0x557ace997870 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f915c09b400 tx=0x7f915c097840 comp rx=0 tx=0).stop 2024-09-10T08:55:12.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.671+0000 7f9173a8e740 1 -- 172.21.15.172:0/1768158459 shutdown_connections 2024-09-10T08:55:12.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.671+0000 7f9173a8e740 1 --2- 172.21.15.172:0/1768158459 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557ace993a80 0x557acea10f60 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.671+0000 7f9173a8e740 1 --2- 172.21.15.172:0/1768158459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ace984c90 0x557ace9982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.671+0000 7f9173a8e740 1 --2- 172.21.15.172:0/1768158459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ace984370 0x557ace997db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.671+0000 7f914eff5640 0 client.18060 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:12.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.671+0000 7f9173a8e740 1 --2- 172.21.15.172:0/1768158459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ace980a20 0x557ace997870 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.671+0000 7f9173a8e740 1 -- 172.21.15.172:0/1768158459 >> 172.21.15.172:0/1768158459 conn(0x557ace990d00 msgr2=0x557ace97f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:12.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.671+0000 7f9173a8e740 1 -- 172.21.15.172:0/1768158459 shutdown_connections 2024-09-10T08:55:12.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.671+0000 7f9173a8e740 1 -- 172.21.15.172:0/1768158459 wait complete. 2024-09-10T08:55:12.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.671+0000 7f9173a8e740 20 client.18060 trim_cache size 0 max 16384 2024-09-10T08:55:12.679 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:55:12.679 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T08:55:12.679 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T08:55:12.679 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:55:12.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.672+0000 7f91716b5640 1 -- 172.21.15.172:0/1193383281 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f91540a2b80 msgr2=0x7f91540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:12.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.672+0000 7f91716b5640 1 --2- 172.21.15.172:0/1193383281 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f91540a2b80 0x7f91540a5040 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9164005fe0 tx=0x7f916402a8d0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.672+0000 7f91716b5640 1 -- 172.21.15.172:0/1193383281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f916c102dd0 msgr2=0x7f916c1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:12.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.672+0000 7f91716b5640 1 --2- 172.21.15.172:0/1193383281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f916c102dd0 0x7f916c1983f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f915c00d950 tx=0x7f915c00dd50 comp rx=0 tx=0).stop 2024-09-10T08:55:12.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.672+0000 7f91716b5640 1 -- 172.21.15.172:0/1193383281 shutdown_connections 2024-09-10T08:55:12.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.672+0000 7f91716b5640 1 --2- 172.21.15.172:0/1193383281 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f91540a2b80 0x7f91540a5040 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.672+0000 7f91716b5640 1 --2- 172.21.15.172:0/1193383281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f916c108300 0x7f916c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.672+0000 7f91716b5640 1 --2- 172.21.15.172:0/1193383281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f916c103780 0x7f916c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.672+0000 7f91716b5640 1 --2- 172.21.15.172:0/1193383281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f916c102dd0 0x7f916c1983f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:12.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.672+0000 7f91716b5640 1 -- 172.21.15.172:0/1193383281 >> 172.21.15.172:0/1193383281 conn(0x7f916c0fc7d0 msgr2=0x7f916c1063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:12.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.672+0000 7f91716b5640 1 -- 172.21.15.172:0/1193383281 shutdown_connections 2024-09-10T08:55:12.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:12.672+0000 7f91716b5640 1 -- 172.21.15.172:0/1193383281 wait complete. 2024-09-10T08:55:12.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:12 smithi174 ceph-mon[34068]: pgmap v619: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-09-10T08:55:12.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:12 smithi191 ceph-mon[35314]: pgmap v619: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-09-10T08:55:13.497 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:55:13.497 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:55:13.776 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:55:14.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.304+0000 7fb2ae74e640 1 -- 172.21.15.172:0/2791511813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2a81082a0 msgr2=0x7fb2a810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:14.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.304+0000 7fb2ae74e640 1 --2- 172.21.15.172:0/2791511813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2a81082a0 0x7fb2a810ec40 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009cd0 tx=0x7fb2a002a8e0 comp rx=0 tx=0).stop 2024-09-10T08:55:14.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.304+0000 7fb2ae74e640 1 -- 172.21.15.172:0/2791511813 shutdown_connections 2024-09-10T08:55:14.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.304+0000 7fb2ae74e640 1 --2- 172.21.15.172:0/2791511813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2a81082a0 0x7fb2a810ec40 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:14.323 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.304+0000 7fb2ae74e640 1 --2- 172.21.15.172:0/2791511813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2a81036e0 0x7fb2a8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:14.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.304+0000 7fb2ae74e640 1 --2- 172.21.15.172:0/2791511813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2a8102d30 0x7fb2a8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:14.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.304+0000 7fb2ae74e640 1 -- 172.21.15.172:0/2791511813 >> 172.21.15.172:0/2791511813 conn(0x7fb2a80fc7d0 msgr2=0x7fb2a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:14.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.304+0000 7fb2ae74e640 1 -- 172.21.15.172:0/2791511813 shutdown_connections 2024-09-10T08:55:14.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.304+0000 7fb2ae74e640 1 -- 172.21.15.172:0/2791511813 wait complete. 2024-09-10T08:55:14.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.304+0000 7fb2ae74e640 1 Processor -- start 2024-09-10T08:55:14.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.304+0000 7fb2ae74e640 1 -- start start 2024-09-10T08:55:14.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2ae74e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2a8102d30 0x7fb2a819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:14.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2ae74e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2a81036e0 0x7fb2a819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:14.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2ae74e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2a81082a0 0x7fb2a819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:14.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2ae74e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a8111f50 con 0x7fb2a81036e0 2024-09-10T08:55:14.324 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2ae74e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a8111dd0 con 0x7fb2a8102d30 2024-09-10T08:55:14.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2ae74e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a81120d0 con 0x7fb2a81082a0 2024-09-10T08:55:14.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2a77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2a81036e0 0x7fb2a819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:14.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2a77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2a81036e0 0x7fb2a819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:55994/0 (socket says 172.21.15.172:55994) 2024-09-10T08:55:14.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2a77fe640 1 -- 172.21.15.172:0/1610422590 learned_addr learned my addr 172.21.15.172:0/1610422590 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:14.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2a7fff640 1 --2- 172.21.15.172:0/1610422590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2a8102d30 0x7fb2a819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:14.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2acceb640 1 --2- 172.21.15.172:0/1610422590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2a81082a0 0x7fb2a819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:14.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2a77fe640 1 -- 172.21.15.172:0/1610422590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2a81082a0 msgr2=0x7fb2a819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:14.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2a77fe640 1 --2- 172.21.15.172:0/1610422590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2a81082a0 0x7fb2a819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:14.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2a77fe640 1 -- 172.21.15.172:0/1610422590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2a8102d30 msgr2=0x7fb2a819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:14.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2a77fe640 1 --2- 172.21.15.172:0/1610422590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2a8102d30 0x7fb2a819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:14.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2a77fe640 1 -- 172.21.15.172:0/1610422590 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a819f620 con 0x7fb2a81036e0 2024-09-10T08:55:14.325 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2acceb640 1 --2- 172.21.15.172:0/1610422590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2a81082a0 0x7fb2a819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:55:14.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.305+0000 7fb2a77fe640 1 --2- 172.21.15.172:0/1610422590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2a81036e0 0x7fb2a819abb0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00e9c0 tx=0x7fb29c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:14.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.306+0000 7fb2a57fa640 1 -- 172.21.15.172:0/1610422590 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c00ccc0 con 0x7fb2a81036e0 2024-09-10T08:55:14.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.306+0000 7fb2a57fa640 1 -- 172.21.15.172:0/1610422590 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb29c0054d0 con 0x7fb2a81036e0 2024-09-10T08:55:14.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.306+0000 7fb2a57fa640 1 -- 172.21.15.172:0/1610422590 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c010430 con 0x7fb2a81036e0 2024-09-10T08:55:14.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.306+0000 7fb2ae74e640 1 -- 172.21.15.172:0/1610422590 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a819f910 con 0x7fb2a81036e0 2024-09-10T08:55:14.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.306+0000 7fb2ae74e640 1 -- 172.21.15.172:0/1610422590 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a81a71f0 con 0x7fb2a81036e0 2024-09-10T08:55:14.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.307+0000 7fb2ae74e640 1 -- 172.21.15.172:0/1610422590 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a8103110 con 0x7fb2a81036e0 2024-09-10T08:55:14.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.308+0000 7fb2a57fa640 1 -- 172.21.15.172:0/1610422590 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb29c005020 con 0x7fb2a81036e0 2024-09-10T08:55:14.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.309+0000 7fb2a57fa640 1 --2- 172.21.15.172:0/1610422590 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb2900a2c00 0x7fb2900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:14.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.309+0000 7fb2a7fff640 1 --2- 172.21.15.172:0/1610422590 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb2900a2c00 0x7fb2900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:14.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.309+0000 7fb2a57fa640 1 -- 172.21.15.172:0/1610422590 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb29c0cf860 con 0x7fb2a81036e0 2024-09-10T08:55:14.326 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.309+0000 7fb2a7fff640 1 --2- 172.21.15.172:0/1610422590 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb2900a2c00 0x7fb2900a50c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb29800bbe0 tx=0x7fb29800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:14.327 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.314+0000 7fb2a57fa640 1 -- 172.21.15.172:0/1610422590 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb29c09c460 con 0x7fb2a81036e0 2024-09-10T08:55:14.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.552+0000 7fb2ae74e640 1 -- 172.21.15.172:0/1610422590 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2a810c2c0 con 0x7fb2a81036e0 2024-09-10T08:55:14.553 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:14 smithi172 ceph-mon[30854]: pgmap v620: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 27 MiB/s wr, 19 op/s 2024-09-10T08:55:14.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.553+0000 7fb2a57fa640 1 -- 172.21.15.172:0/1610422590 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+12001 (secure 0 0 0) 0x7fb29c09bd60 con 0x7fb2a81036e0 2024-09-10T08:55:14.554 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:55:14.554 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":37,"btime":"2024-09-10T08:52:08:313493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:52:07.658341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:55:14.555 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 37 2024-09-10T08:55:14.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.555+0000 7fb2ae74e640 1 -- 172.21.15.172:0/1610422590 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb2900a2c00 msgr2=0x7fb2900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:14.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.555+0000 7fb2ae74e640 1 --2- 172.21.15.172:0/1610422590 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb2900a2c00 0x7fb2900a50c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb29800bbe0 tx=0x7fb29800b390 comp rx=0 tx=0).stop 2024-09-10T08:55:14.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.556+0000 7fb2ae74e640 1 -- 172.21.15.172:0/1610422590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2a81036e0 msgr2=0x7fb2a819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:14.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.556+0000 7fb2ae74e640 1 --2- 172.21.15.172:0/1610422590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2a81036e0 0x7fb2a819abb0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00e9c0 tx=0x7fb29c00ee80 comp rx=0 tx=0).stop 2024-09-10T08:55:14.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.556+0000 7fb2ae74e640 1 -- 172.21.15.172:0/1610422590 shutdown_connections 2024-09-10T08:55:14.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.556+0000 7fb2ae74e640 1 --2- 172.21.15.172:0/1610422590 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb2900a2c00 0x7fb2900a50c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:14.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.556+0000 7fb2ae74e640 1 --2- 172.21.15.172:0/1610422590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2a81082a0 0x7fb2a819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:14.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.556+0000 7fb2ae74e640 1 --2- 172.21.15.172:0/1610422590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2a81036e0 0x7fb2a819abb0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:14.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.556+0000 7fb2ae74e640 1 --2- 172.21.15.172:0/1610422590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2a8102d30 0x7fb2a819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:14.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.556+0000 7fb2ae74e640 1 -- 172.21.15.172:0/1610422590 >> 172.21.15.172:0/1610422590 conn(0x7fb2a80fc7d0 msgr2=0x7fb2a8100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:14.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.556+0000 7fb2ae74e640 1 -- 172.21.15.172:0/1610422590 shutdown_connections 2024-09-10T08:55:14.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:14.556+0000 7fb2ae74e640 1 -- 172.21.15.172:0/1610422590 wait complete. 2024-09-10T08:55:14.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:14 smithi174 ceph-mon[34068]: pgmap v620: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 27 MiB/s wr, 19 op/s 2024-09-10T08:55:14.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:14 smithi191 ceph-mon[35314]: pgmap v620: 129 pgs: 129 active+clean; 8.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 27 MiB/s wr, 19 op/s 2024-09-10T08:55:15.411 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T08:55:15.702 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:55:15.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:15 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1610422590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:55:15.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:15 smithi174 ceph-mon[34068]: pgmap v621: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 26 op/s 2024-09-10T08:55:15.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:15 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1610422590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:55:15.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:15 smithi191 ceph-mon[35314]: pgmap v621: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 26 op/s 2024-09-10T08:55:15.824 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:15 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1610422590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:55:15.824 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:15 smithi172 ceph-mon[30854]: pgmap v621: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 26 op/s 2024-09-10T08:55:16.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.176+0000 7f7460df3640 1 -- 172.21.15.172:0/733944265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f745c106a80 msgr2=0x7f745c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:16.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.176+0000 7f7460df3640 1 --2- 172.21.15.172:0/733944265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f745c106a80 0x7f745c106e60 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f744c009930 tx=0x7f744c02a420 comp rx=0 tx=0).stop 2024-09-10T08:55:16.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.176+0000 7f7460df3640 1 -- 172.21.15.172:0/733944265 shutdown_connections 2024-09-10T08:55:16.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.176+0000 7f7460df3640 1 --2- 172.21.15.172:0/733944265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f745c101420 0x7f745c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.176+0000 7f7460df3640 1 --2- 172.21.15.172:0/733944265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f745c100a80 0x7f745c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.176+0000 7f7460df3640 1 --2- 172.21.15.172:0/733944265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f745c106a80 0x7f745c106e60 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.176+0000 7f7460df3640 1 -- 172.21.15.172:0/733944265 >> 172.21.15.172:0/733944265 conn(0x7f745c0fc7a0 msgr2=0x7f745c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:16.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.176+0000 7f7460df3640 1 -- 172.21.15.172:0/733944265 shutdown_connections 2024-09-10T08:55:16.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.176+0000 7f7460df3640 1 -- 172.21.15.172:0/733944265 wait complete. 2024-09-10T08:55:16.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.177+0000 7f7460df3640 1 Processor -- start 2024-09-10T08:55:16.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.177+0000 7f7460df3640 1 -- start start 2024-09-10T08:55:16.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.177+0000 7f7460df3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f745c100a80 0x7f745c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.177+0000 7f7460df3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f745c101420 0x7f745c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.177+0000 7f7460df3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f745c106a80 0x7f745c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.177+0000 7f7460df3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f745c110d30 con 0x7f745c101420 2024-09-10T08:55:16.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.177+0000 7f7460df3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f745c110bb0 con 0x7f745c100a80 2024-09-10T08:55:16.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.177+0000 7f7460df3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f745c110eb0 con 0x7f745c106a80 2024-09-10T08:55:16.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.177+0000 7f7459d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f745c101420 0x7f745c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.178+0000 7f7459d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f745c101420 0x7f745c1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42458/0 (socket says 172.21.15.172:42458) 2024-09-10T08:55:16.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.178+0000 7f7459d9b640 1 -- 172.21.15.172:0/3305588500 learned_addr learned my addr 172.21.15.172:0/3305588500 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:16.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.178+0000 7f745ad9d640 1 --2- 172.21.15.172:0/3305588500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f745c106a80 0x7f745c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.178+0000 7f745a59c640 1 --2- 172.21.15.172:0/3305588500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f745c100a80 0x7f745c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.178+0000 7f7459d9b640 1 -- 172.21.15.172:0/3305588500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f745c106a80 msgr2=0x7f745c19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:16.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.178+0000 7f7459d9b640 1 --2- 172.21.15.172:0/3305588500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f745c106a80 0x7f745c19cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.178+0000 7f7459d9b640 1 -- 172.21.15.172:0/3305588500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f745c100a80 msgr2=0x7f745c1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:16.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.178+0000 7f7459d9b640 1 --2- 172.21.15.172:0/3305588500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f745c100a80 0x7f745c1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.178+0000 7f7459d9b640 1 -- 172.21.15.172:0/3305588500 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c19d260 con 0x7f745c101420 2024-09-10T08:55:16.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.178+0000 7f745ad9d640 1 --2- 172.21.15.172:0/3305588500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f745c106a80 0x7f745c19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:16.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.178+0000 7f7459d9b640 1 --2- 172.21.15.172:0/3305588500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f745c101420 0x7f745c1987f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f745000c920 tx=0x7f745000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:16.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.178+0000 7f744b7fe640 1 -- 172.21.15.172:0/3305588500 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450007c70 con 0x7f745c101420 2024-09-10T08:55:16.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.178+0000 7f7460df3640 1 -- 172.21.15.172:0/3305588500 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745c19d550 con 0x7f745c101420 2024-09-10T08:55:16.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.178+0000 7f744b7fe640 1 -- 172.21.15.172:0/3305588500 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7450007e10 con 0x7f745c101420 2024-09-10T08:55:16.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.178+0000 7f744b7fe640 1 -- 172.21.15.172:0/3305588500 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450018820 con 0x7f745c101420 2024-09-10T08:55:16.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.178+0000 7f7460df3640 1 -- 172.21.15.172:0/3305588500 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745c1a4e30 con 0x7f745c101420 2024-09-10T08:55:16.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.181+0000 7f744b7fe640 1 -- 172.21.15.172:0/3305588500 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f74500189c0 con 0x7f745c101420 2024-09-10T08:55:16.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.182+0000 7f74631cc740 1 Processor -- start 2024-09-10T08:55:16.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.182+0000 7f74631cc740 1 -- start start 2024-09-10T08:55:16.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.182+0000 7f74631cc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd90b40 0x55d64bd8f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.182+0000 7f74631cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd8fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.182+0000 7f74631cc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd9a370 0x55d64bd9c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.182+0000 7f74631cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d64bcf52f0 con 0x55d64bd8f5d0 2024-09-10T08:55:16.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.182+0000 7f74631cc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d64bc9ccf0 con 0x55d64bd9a370 2024-09-10T08:55:16.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.182+0000 7f74631cc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d64bcf12b0 con 0x55d64bd90b40 2024-09-10T08:55:16.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.182+0000 7f744b7fe640 1 --2- 172.21.15.172:0/3305588500 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f74440a2b80 0x7f74440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.183+0000 7f745a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd8fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.183+0000 7f745a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd8fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42474/0 (socket says 172.21.15.172:42474) 2024-09-10T08:55:16.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.183+0000 7f745a59c640 1 -- 172.21.15.172:0/157906380 learned_addr learned my addr 172.21.15.172:0/157906380 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:16.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.183+0000 7f745ad9d640 1 --2- 172.21.15.172:0/3305588500 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f74440a2b80 0x7f74440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.183+0000 7f745ad9d640 1 --2- 172.21.15.172:0/157906380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd90b40 0x55d64bd8f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.183+0000 7f744b7fe640 1 -- 172.21.15.172:0/3305588500 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f74500cf620 con 0x7f745c101420 2024-09-10T08:55:16.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.183+0000 7f7459d9b640 1 --2- 172.21.15.172:0/157906380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd9a370 0x55d64bd9c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.183+0000 7f745ad9d640 1 --2- 172.21.15.172:0/3305588500 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f74440a2b80 0x7f74440a5040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7454005c40 tx=0x7f7454009d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:16.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.183+0000 7f745ad9d640 1 -- 172.21.15.172:0/157906380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd9a370 msgr2=0x55d64bd9c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:16.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.183+0000 7f745ad9d640 1 --2- 172.21.15.172:0/157906380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd9a370 0x55d64bd9c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.183+0000 7f745ad9d640 1 -- 172.21.15.172:0/157906380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 msgr2=0x55d64bd8fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:16.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.183+0000 7f745ad9d640 1 --2- 172.21.15.172:0/157906380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd8fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.183+0000 7f745ad9d640 1 -- 172.21.15.172:0/157906380 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64bc813a0 con 0x55d64bd90b40 2024-09-10T08:55:16.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.184+0000 7f745ad9d640 1 --2- 172.21.15.172:0/157906380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd90b40 0x55d64bd8f090 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7454005fd0 tx=0x7f745401e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c37e06ef7ffde8bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:16.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.184+0000 7f7448ff9640 1 -- 172.21.15.172:0/157906380 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745401e7b0 con 0x55d64bd90b40 2024-09-10T08:55:16.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.184+0000 7f7448ff9640 1 -- 172.21.15.172:0/157906380 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7454023070 con 0x55d64bd90b40 2024-09-10T08:55:16.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.184+0000 7f7448ff9640 1 -- 172.21.15.172:0/157906380 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745401fd80 con 0x55d64bd90b40 2024-09-10T08:55:16.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.185+0000 7f74631cc740 1 -- 172.21.15.172:0/157906380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd90b40 msgr2=0x55d64bd8f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:16.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.185+0000 7f74631cc740 1 --2- 172.21.15.172:0/157906380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd90b40 0x55d64bd8f090 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7454005fd0 tx=0x7f745401e000 comp rx=0 tx=0).stop 2024-09-10T08:55:16.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.185+0000 7f74631cc740 1 -- 172.21.15.172:0/157906380 shutdown_connections 2024-09-10T08:55:16.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.185+0000 7f74631cc740 1 --2- 172.21.15.172:0/157906380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd9a370 0x55d64bd9c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.185+0000 7f74631cc740 1 --2- 172.21.15.172:0/157906380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd8fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.185+0000 7f74631cc740 1 --2- 172.21.15.172:0/157906380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd90b40 0x55d64bd8f090 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.185+0000 7f74631cc740 1 -- 172.21.15.172:0/157906380 >> 172.21.15.172:0/157906380 conn(0x55d64bd58310 msgr2=0x55d64bd8bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:16.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.185+0000 7f74631cc740 1 -- 172.21.15.172:0/157906380 shutdown_connections 2024-09-10T08:55:16.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.185+0000 7f74631cc740 1 -- 172.21.15.172:0/157906380 wait complete. 2024-09-10T08:55:16.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.185+0000 7f74631cc740 1 -- start start 2024-09-10T08:55:16.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.185+0000 7f74631cc740 1 Processor -- start 2024-09-10T08:55:16.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.186+0000 7f74631cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd99d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.186+0000 7f74631cc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd90b40 0x55d64bd97000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.186+0000 7f74631cc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd9a370 0x55d64bd975d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.186+0000 7f74631cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d64bd9d010 con 0x55d64bd8f5d0 2024-09-10T08:55:16.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.186+0000 7f74631cc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d64bcb0060 con 0x55d64bd9a370 2024-09-10T08:55:16.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.186+0000 7f74631cc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d64bd9d190 con 0x55d64bd90b40 2024-09-10T08:55:16.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.186+0000 7f745a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd99d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.186+0000 7f74631cc740 10 client.0 _ll_register_callbacks cb 0x55d64ba038c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:16.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.186+0000 7f745a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd99d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42488/0 (socket says 172.21.15.172:42488) 2024-09-10T08:55:16.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.186+0000 7f745a59c640 1 -- 172.21.15.172:0/1233670627 learned_addr learned my addr 172.21.15.172:0/1233670627 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:16.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.186+0000 7f7459d9b640 1 --2- 172.21.15.172:0/1233670627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd90b40 0x55d64bd97000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.186+0000 7f745ad9d640 1 --2- 172.21.15.172:0/1233670627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd9a370 0x55d64bd975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.186+0000 7f745a59c640 1 -- 172.21.15.172:0/1233670627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd90b40 msgr2=0x55d64bd97000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:16.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.186+0000 7f745a59c640 1 --2- 172.21.15.172:0/1233670627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd90b40 0x55d64bd97000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.186+0000 7f745a59c640 1 -- 172.21.15.172:0/1233670627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd9a370 msgr2=0x55d64bd975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:16.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.186+0000 7f745a59c640 1 --2- 172.21.15.172:0/1233670627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd9a370 0x55d64bd975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.186+0000 7f745a59c640 1 -- 172.21.15.172:0/1233670627 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d64ba663c0 con 0x55d64bd8f5d0 2024-09-10T08:55:16.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.187+0000 7f745a59c640 1 --2- 172.21.15.172:0/1233670627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd99d50 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f744c009900 tx=0x7f744c031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ca4c193b6e8918 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:16.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.187+0000 7f7441ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:16.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.187+0000 7f7441ffb640 1 -- 172.21.15.172:0/1233670627 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c035070 con 0x55d64bd8f5d0 2024-09-10T08:55:16.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.187+0000 7f7441ffb640 1 -- 172.21.15.172:0/1233670627 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f744c004240 con 0x55d64bd8f5d0 2024-09-10T08:55:16.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.187+0000 7f74631cc740 1 -- 172.21.15.172:0/1233670627 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7454023070 con 0x55d64bd8f5d0 2024-09-10T08:55:16.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.187+0000 7f7441ffb640 1 -- 172.21.15.172:0/1233670627 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f744c004440 con 0x55d64bd8f5d0 2024-09-10T08:55:16.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.187+0000 7f74631cc740 10 client.18078 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:16.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.187+0000 7f74631cc740 1 -- 172.21.15.172:0/1233670627 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d64bda8a90 con 0x55d64bd8f5d0 2024-09-10T08:55:16.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.187+0000 7f7441ffb640 1 -- 172.21.15.172:0/1233670627 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7454023070 con 0x55d64bd8f5d0 2024-09-10T08:55:16.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.187+0000 7f7441ffb640 1 -- 172.21.15.172:0/1233670627 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f744c004b50 con 0x55d64bd8f5d0 2024-09-10T08:55:16.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.187+0000 7f74631cc740 10 client.18078 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:16.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.187+0000 7f74631cc740 10 client.18078 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:55:16.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.187+0000 7f74631cc740 20 client.18078 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:16.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.187+0000 7f74631cc740 1 --2- 172.21.15.172:0/1233670627 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d64bdaa1f0 0x55d64bdac620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.187+0000 7f74631cc740 4 client.18078 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:55:16.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.187+0000 7f74631cc740 1 -- 172.21.15.172:0/1233670627 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d64bd02860 con 0x55d64bdaa1f0 2024-09-10T08:55:16.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.188+0000 7f7459d9b640 1 --2- 172.21.15.172:0/1233670627 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d64bdaa1f0 0x55d64bdac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.188+0000 7f7459d9b640 1 --2- 172.21.15.172:0/1233670627 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d64bdaa1f0 0x55d64bdac620 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=849141612673dc90 server_cookie=7896064f209a456b in_seq=0 out_seq=0 2024-09-10T08:55:16.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.188+0000 7f7441ffb640 10 client.18078 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:55:16.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.189+0000 7f7441ffb640 1 -- 172.21.15.172:0/1233670627 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d64bd02860 con 0x55d64bdaa1f0 2024-09-10T08:55:16.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.189+0000 7f7441ffb640 10 client.18078 handle_command_reply: tid=0 2024-09-10T08:55:16.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.189+0000 7f74631cc740 1 client.18078 shutdown 2024-09-10T08:55:16.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.189+0000 7f74631cc740 1 -- 172.21.15.172:0/1233670627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 msgr2=0x55d64bd99d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:16.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.189+0000 7f74631cc740 1 --2- 172.21.15.172:0/1233670627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd99d50 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f744c009900 tx=0x7f744c031750 comp rx=0 tx=0).stop 2024-09-10T08:55:16.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.189+0000 7f74631cc740 1 -- 172.21.15.172:0/1233670627 shutdown_connections 2024-09-10T08:55:16.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.189+0000 7f74631cc740 1 --2- 172.21.15.172:0/1233670627 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d64bdaa1f0 0x55d64bdac620 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.189+0000 7f74631cc740 1 --2- 172.21.15.172:0/1233670627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd9a370 0x55d64bd975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.189+0000 7f74631cc740 1 --2- 172.21.15.172:0/1233670627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd90b40 0x55d64bd97000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.189+0000 7f74631cc740 1 --2- 172.21.15.172:0/1233670627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd99d50 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.189+0000 7f74631cc740 1 -- 172.21.15.172:0/1233670627 >> 172.21.15.172:0/1233670627 conn(0x55d64bd58310 msgr2=0x55d64bd91960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:16.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.189+0000 7f7441ffb640 0 client.18078 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:55:16.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.189+0000 7f74631cc740 1 -- 172.21.15.172:0/1233670627 shutdown_connections 2024-09-10T08:55:16.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.189+0000 7f74631cc740 1 -- 172.21.15.172:0/1233670627 wait complete. 2024-09-10T08:55:16.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.189+0000 7f74631cc740 20 client.18078 trim_cache size 0 max 16384 2024-09-10T08:55:16.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f74631cc740 1 Processor -- start 2024-09-10T08:55:16.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f74631cc740 1 -- start start 2024-09-10T08:55:16.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f74631cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd968c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f74631cc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd90b40 0x55d64bd92120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f74631cc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd9a370 0x55d64bd92660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f74631cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1aea90 con 0x55d64bd8f5d0 2024-09-10T08:55:16.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f74631cc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1b05d0 con 0x55d64bd9a370 2024-09-10T08:55:16.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f74631cc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1af160 con 0x55d64bd90b40 2024-09-10T08:55:16.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f745a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f745a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42504/0 (socket says 172.21.15.172:42504) 2024-09-10T08:55:16.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f745a59c640 1 -- 172.21.15.172:0/564366919 learned_addr learned my addr 172.21.15.172:0/564366919 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:16.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f7459d9b640 1 --2- 172.21.15.172:0/564366919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd90b40 0x55d64bd92120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f745ad9d640 1 --2- 172.21.15.172:0/564366919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd9a370 0x55d64bd92660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f745a59c640 1 -- 172.21.15.172:0/564366919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd90b40 msgr2=0x55d64bd92120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:16.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f745a59c640 1 --2- 172.21.15.172:0/564366919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd90b40 0x55d64bd92120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f745a59c640 1 -- 172.21.15.172:0/564366919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd9a370 msgr2=0x55d64bd92660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:16.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f745a59c640 1 --2- 172.21.15.172:0/564366919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd9a370 0x55d64bd92660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f745a59c640 1 -- 172.21.15.172:0/564366919 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64bd95fb0 con 0x55d64bd8f5d0 2024-09-10T08:55:16.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.206+0000 7f745ad9d640 1 --2- 172.21.15.172:0/564366919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd9a370 0x55d64bd92660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:16.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.207+0000 7f745a59c640 1 --2- 172.21.15.172:0/564366919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd968c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f744c031440 tx=0x7f744c032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc59fbd29de2e473 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:16.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.207+0000 7f7443fff640 1 -- 172.21.15.172:0/564366919 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c035070 con 0x55d64bd8f5d0 2024-09-10T08:55:16.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.207+0000 7f7443fff640 1 -- 172.21.15.172:0/564366919 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f744c004b70 con 0x55d64bd8f5d0 2024-09-10T08:55:16.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.207+0000 7f7443fff640 1 -- 172.21.15.172:0/564366919 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c0053c0 con 0x55d64bd8f5d0 2024-09-10T08:55:16.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.208+0000 7f74631cc740 1 -- 172.21.15.172:0/564366919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 msgr2=0x55d64bd968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:16.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.208+0000 7f74631cc740 1 --2- 172.21.15.172:0/564366919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd968c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f744c031440 tx=0x7f744c032ea0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.208+0000 7f74631cc740 1 -- 172.21.15.172:0/564366919 shutdown_connections 2024-09-10T08:55:16.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.208+0000 7f74631cc740 1 --2- 172.21.15.172:0/564366919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd9a370 0x55d64bd92660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.208+0000 7f74631cc740 1 --2- 172.21.15.172:0/564366919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd90b40 0x55d64bd92120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.208+0000 7f74631cc740 1 --2- 172.21.15.172:0/564366919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd968c0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.208+0000 7f74631cc740 1 -- 172.21.15.172:0/564366919 >> 172.21.15.172:0/564366919 conn(0x55d64bd58310 msgr2=0x55d64bd8ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:16.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.208+0000 7f74631cc740 1 -- 172.21.15.172:0/564366919 shutdown_connections 2024-09-10T08:55:16.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.208+0000 7f74631cc740 1 -- 172.21.15.172:0/564366919 wait complete. 2024-09-10T08:55:16.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.208+0000 7f74631cc740 1 -- start start 2024-09-10T08:55:16.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.208+0000 7f74631cc740 1 Processor -- start 2024-09-10T08:55:16.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.208+0000 7f74631cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd9e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.208+0000 7f74631cc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd90b40 0x55d64bd9e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.208+0000 7f74631cc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd9a370 0x55d64bd9eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f74631cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1aff00 con 0x55d64bd8f5d0 2024-09-10T08:55:16.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f74631cc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1af830 con 0x55d64bd90b40 2024-09-10T08:55:16.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f74631cc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f745c1aca70 con 0x55d64bd9a370 2024-09-10T08:55:16.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f74631cc740 10 client.0 _ll_register_callbacks cb 0x7f744c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:16.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f745a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd9e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f745a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd9e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42514/0 (socket says 172.21.15.172:42514) 2024-09-10T08:55:16.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f745a59c640 1 -- 172.21.15.172:0/859846563 learned_addr learned my addr 172.21.15.172:0/859846563 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:16.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f745ad9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd9a370 0x55d64bd9eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f7459d9b640 1 --2- 172.21.15.172:0/859846563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd90b40 0x55d64bd9e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f745a59c640 1 -- 172.21.15.172:0/859846563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd9a370 msgr2=0x55d64bd9eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:16.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f745a59c640 1 --2- 172.21.15.172:0/859846563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd9a370 0x55d64bd9eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f745a59c640 1 -- 172.21.15.172:0/859846563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd90b40 msgr2=0x55d64bd9e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:16.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f745a59c640 1 --2- 172.21.15.172:0/859846563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd90b40 0x55d64bd9e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f745a59c640 1 -- 172.21.15.172:0/859846563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d64be182d0 con 0x55d64bd8f5d0 2024-09-10T08:55:16.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f745ad9d640 1 --2- 172.21.15.172:0/859846563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd9a370 0x55d64bd9eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:16.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f7459d9b640 1 --2- 172.21.15.172:0/859846563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd90b40 0x55d64bd9e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:16.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f745a59c640 1 --2- 172.21.15.172:0/859846563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd9e0b0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f744c032ad0 tx=0x7f744c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a2117471c914f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:16.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f7441ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:16.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f7441ffb640 1 -- 172.21.15.172:0/859846563 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c035030 con 0x55d64bd8f5d0 2024-09-10T08:55:16.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f7441ffb640 1 -- 172.21.15.172:0/859846563 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f744c005140 con 0x55d64bd8f5d0 2024-09-10T08:55:16.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.209+0000 7f74631cc740 1 -- 172.21.15.172:0/859846563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f744c004b70 con 0x55d64bd8f5d0 2024-09-10T08:55:16.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.210+0000 7f7441ffb640 1 -- 172.21.15.172:0/859846563 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f744c005340 con 0x55d64bd8f5d0 2024-09-10T08:55:16.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.210+0000 7f74631cc740 10 client.18084 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:16.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.210+0000 7f74631cc740 1 -- 172.21.15.172:0/859846563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d64be194f0 con 0x55d64bd8f5d0 2024-09-10T08:55:16.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.210+0000 7f7441ffb640 1 -- 172.21.15.172:0/859846563 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f744c004b70 con 0x55d64bd8f5d0 2024-09-10T08:55:16.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.210+0000 7f7441ffb640 1 -- 172.21.15.172:0/859846563 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f744c044590 con 0x55d64bd8f5d0 2024-09-10T08:55:16.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.210+0000 7f74631cc740 10 client.18084 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:16.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.210+0000 7f74631cc740 10 client.18084 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:55:16.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.210+0000 7f74631cc740 20 client.18084 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:16.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.210+0000 7f74631cc740 1 --2- 172.21.15.172:0/859846563 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d64be199c0 0x55d64be1bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:16.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.210+0000 7f74631cc740 4 client.18084 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:55:16.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.210+0000 7f74631cc740 1 -- 172.21.15.172:0/859846563 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d64bcfcd70 con 0x55d64be199c0 2024-09-10T08:55:16.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.211+0000 7f7459d9b640 1 --2- 172.21.15.172:0/859846563 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d64be199c0 0x55d64be1bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:16.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.211+0000 7f7441ffb640 10 client.18084 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:55:16.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.211+0000 7f7459d9b640 1 --2- 172.21.15.172:0/859846563 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d64be199c0 0x55d64be1bdb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=245f34ef3b9c42b1 server_cookie=78a45d4bfb429ff6 in_seq=0 out_seq=0 2024-09-10T08:55:16.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.211+0000 7f7441ffb640 1 -- 172.21.15.172:0/859846563 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d64bcfcd70 con 0x55d64be199c0 2024-09-10T08:55:16.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.211+0000 7f7441ffb640 10 client.18084 handle_command_reply: tid=0 2024-09-10T08:55:16.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.211+0000 7f74631cc740 1 client.18084 shutdown 2024-09-10T08:55:16.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.211+0000 7f74631cc740 1 -- 172.21.15.172:0/859846563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 msgr2=0x55d64bd9e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:16.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.211+0000 7f74631cc740 1 --2- 172.21.15.172:0/859846563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd9e0b0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f744c032ad0 tx=0x7f744c005fd0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.211+0000 7f74631cc740 1 -- 172.21.15.172:0/859846563 shutdown_connections 2024-09-10T08:55:16.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.211+0000 7f74631cc740 1 --2- 172.21.15.172:0/859846563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d64bd9a370 0x55d64bd9eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.211+0000 7f74631cc740 1 --2- 172.21.15.172:0/859846563 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d64be199c0 0x55d64be1bdb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.211+0000 7f74631cc740 1 --2- 172.21.15.172:0/859846563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d64bd90b40 0x55d64bd9e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.211+0000 7f7441ffb640 0 client.18084 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:55:16.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.212+0000 7f74631cc740 1 --2- 172.21.15.172:0/859846563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d64bd8f5d0 0x55d64bd9e0b0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.212+0000 7f74631cc740 1 -- 172.21.15.172:0/859846563 >> 172.21.15.172:0/859846563 conn(0x55d64bd58310 msgr2=0x55d64bd98860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:16.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.212+0000 7f74631cc740 1 -- 172.21.15.172:0/859846563 shutdown_connections 2024-09-10T08:55:16.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.212+0000 7f74631cc740 1 -- 172.21.15.172:0/859846563 wait complete. 2024-09-10T08:55:16.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.212+0000 7f74631cc740 20 client.18084 trim_cache size 0 max 16384 2024-09-10T08:55:16.223 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:55:16.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.212+0000 7f7460df3640 1 -- 172.21.15.172:0/3305588500 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f74440a2b80 msgr2=0x7f74440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:16.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.212+0000 7f7460df3640 1 --2- 172.21.15.172:0/3305588500 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f74440a2b80 0x7f74440a5040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7454005c40 tx=0x7f7454009d50 comp rx=0 tx=0).stop 2024-09-10T08:55:16.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.212+0000 7f7460df3640 1 -- 172.21.15.172:0/3305588500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f745c101420 msgr2=0x7f745c1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:16.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.212+0000 7f7460df3640 1 --2- 172.21.15.172:0/3305588500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f745c101420 0x7f745c1987f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f745000c920 tx=0x7f745000cde0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.212+0000 7f7460df3640 1 -- 172.21.15.172:0/3305588500 shutdown_connections 2024-09-10T08:55:16.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.212+0000 7f7460df3640 1 --2- 172.21.15.172:0/3305588500 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f74440a2b80 0x7f74440a5040 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.212+0000 7f7460df3640 1 --2- 172.21.15.172:0/3305588500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f745c106a80 0x7f745c19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.212+0000 7f7460df3640 1 --2- 172.21.15.172:0/3305588500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f745c101420 0x7f745c1987f0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.212+0000 7f7460df3640 1 --2- 172.21.15.172:0/3305588500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f745c100a80 0x7f745c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:16.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.212+0000 7f7460df3640 1 -- 172.21.15.172:0/3305588500 >> 172.21.15.172:0/3305588500 conn(0x7f745c0fc7a0 msgr2=0x7f745c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:16.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.213+0000 7f7460df3640 1 -- 172.21.15.172:0/3305588500 shutdown_connections 2024-09-10T08:55:16.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:16.213+0000 7f7460df3640 1 -- 172.21.15.172:0/3305588500 wait complete. 2024-09-10T08:55:16.911 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T08:55:17.181 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:55:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.572+0000 7f90b579f640 1 -- 172.21.15.172:0/2188461221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b0100840 msgr2=0x7f90b0100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.572+0000 7f90b579f640 1 --2- 172.21.15.172:0/2188461221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b0100840 0x7f90b0100c40 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009900 tx=0x7f90a002a410 comp rx=0 tx=0).stop 2024-09-10T08:55:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.572+0000 7f90b579f640 1 -- 172.21.15.172:0/2188461221 shutdown_connections 2024-09-10T08:55:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.572+0000 7f90b579f640 1 --2- 172.21.15.172:0/2188461221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90b01023f0 0x7f90b0108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.572+0000 7f90b579f640 1 --2- 172.21.15.172:0/2188461221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b0101a30 0x7f90b0101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.572+0000 7f90b579f640 1 --2- 172.21.15.172:0/2188461221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b0100840 0x7f90b0100c40 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.572+0000 7f90b579f640 1 -- 172.21.15.172:0/2188461221 >> 172.21.15.172:0/2188461221 conn(0x7f90b00fbfa0 msgr2=0x7f90b00fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.572+0000 7f90b579f640 1 -- 172.21.15.172:0/2188461221 shutdown_connections 2024-09-10T08:55:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.572+0000 7f90b579f640 1 -- 172.21.15.172:0/2188461221 wait complete. 2024-09-10T08:55:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.573+0000 7f90b579f640 1 Processor -- start 2024-09-10T08:55:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.573+0000 7f90b579f640 1 -- start start 2024-09-10T08:55:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.573+0000 7f90b579f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90b0100840 0x7f90b0197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90b579f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b0101a30 0x7f90b0198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90b579f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b01023f0 0x7f90b019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90b579f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f90b010c350 con 0x7f90b0101a30 2024-09-10T08:55:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90b579f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f90b010c1d0 con 0x7f90b01023f0 2024-09-10T08:55:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90b579f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f90b010c4d0 con 0x7f90b0100840 2024-09-10T08:55:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90af7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b0101a30 0x7f90b0198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90af7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b0101a30 0x7f90b0198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42544/0 (socket says 172.21.15.172:42544) 2024-09-10T08:55:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90af7fe640 1 -- 172.21.15.172:0/3111677172 learned_addr learned my addr 172.21.15.172:0/3111677172 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90b4f9e640 1 --2- 172.21.15.172:0/3111677172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b01023f0 0x7f90b019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90affff640 1 --2- 172.21.15.172:0/3111677172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90b0100840 0x7f90b0197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90af7fe640 1 -- 172.21.15.172:0/3111677172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90b0100840 msgr2=0x7f90b0197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90af7fe640 1 --2- 172.21.15.172:0/3111677172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90b0100840 0x7f90b0197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90af7fe640 1 -- 172.21.15.172:0/3111677172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b01023f0 msgr2=0x7f90b019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90af7fe640 1 --2- 172.21.15.172:0/3111677172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b01023f0 0x7f90b019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90af7fe640 1 -- 172.21.15.172:0/3111677172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b019f830 con 0x7f90b0101a30 2024-09-10T08:55:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90af7fe640 1 --2- 172.21.15.172:0/3111677172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b0101a30 0x7f90b0198160 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f90a400d950 tx=0x7f90a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90ad7fa640 1 -- 172.21.15.172:0/3111677172 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4017070 con 0x7f90b0101a30 2024-09-10T08:55:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90ad7fa640 1 -- 172.21.15.172:0/3111677172 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90a4009c30 con 0x7f90b0101a30 2024-09-10T08:55:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90b579f640 1 -- 172.21.15.172:0/3111677172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b019fb20 con 0x7f90b0101a30 2024-09-10T08:55:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90ad7fa640 1 -- 172.21.15.172:0/3111677172 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a400f620 con 0x7f90b0101a30 2024-09-10T08:55:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.574+0000 7f90b579f640 1 -- 172.21.15.172:0/3111677172 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b01a0040 con 0x7f90b0101a30 2024-09-10T08:55:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.577+0000 7f90ad7fa640 1 -- 172.21.15.172:0/3111677172 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f90a40109a0 con 0x7f90b0101a30 2024-09-10T08:55:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.577+0000 7f90b9870740 1 Processor -- start 2024-09-10T08:55:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.577+0000 7f90b9870740 1 -- start start 2024-09-10T08:55:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.577+0000 7f90b9870740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51b6b70 0x559ae51b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.577+0000 7f90b9870740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51b74b0 0x559ae51bbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.577+0000 7f90b9870740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51bc540 0x559ae51bea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.577+0000 7f90b9870740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559ae511a2f0 con 0x559ae51bc540 2024-09-10T08:55:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.577+0000 7f90b9870740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559ae50c2270 con 0x559ae51b6b70 2024-09-10T08:55:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.577+0000 7f90b9870740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559ae51162b0 con 0x559ae51b74b0 2024-09-10T08:55:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.578+0000 7f90af7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51bc540 0x559ae51bea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.578+0000 7f90af7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51bc540 0x559ae51bea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42550/0 (socket says 172.21.15.172:42550) 2024-09-10T08:55:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.578+0000 7f90af7fe640 1 -- 172.21.15.172:0/1690230086 learned_addr learned my addr 172.21.15.172:0/1690230086 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.578+0000 7f90affff640 1 --2- 172.21.15.172:0/1690230086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51b74b0 0x559ae51bbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.578+0000 7f90b4f9e640 1 --2- 172.21.15.172:0/1690230086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51b6b70 0x559ae51b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.578+0000 7f90af7fe640 1 -- 172.21.15.172:0/1690230086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51b74b0 msgr2=0x559ae51bbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.578+0000 7f90af7fe640 1 --2- 172.21.15.172:0/1690230086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51b74b0 0x559ae51bbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.578+0000 7f90af7fe640 1 -- 172.21.15.172:0/1690230086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51b6b70 msgr2=0x559ae51b6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.578+0000 7f90af7fe640 1 --2- 172.21.15.172:0/1690230086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51b6b70 0x559ae51b6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.578+0000 7f90af7fe640 1 -- 172.21.15.172:0/1690230086 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ae50c1340 con 0x559ae51bc540 2024-09-10T08:55:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.578+0000 7f90ad7fa640 1 --2- 172.21.15.172:0/3111677172 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f90980a2b80 0x7f90980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.578+0000 7f90b4f9e640 1 --2- 172.21.15.172:0/3111677172 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f90980a2b80 0x7f90980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.578+0000 7f90af7fe640 1 --2- 172.21.15.172:0/1690230086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51bc540 0x559ae51bea00 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f90a40d7260 tx=0x7f90a40d7720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07fc2e325abe4f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.578+0000 7f90ad7fa640 1 -- 172.21.15.172:0/3111677172 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f90a40cefd0 con 0x7f90b0101a30 2024-09-10T08:55:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.578+0000 7f90967fc640 1 -- 172.21.15.172:0/1690230086 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4020020 con 0x559ae51bc540 2024-09-10T08:55:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.579+0000 7f90967fc640 1 -- 172.21.15.172:0/1690230086 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90a4009de0 con 0x559ae51bc540 2024-09-10T08:55:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.579+0000 7f90967fc640 1 -- 172.21.15.172:0/1690230086 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4097790 con 0x559ae51bc540 2024-09-10T08:55:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.579+0000 7f90b4f9e640 1 --2- 172.21.15.172:0/3111677172 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f90980a2b80 0x7f90980a5040 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f90a8005fd0 tx=0x7f90a8009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.579+0000 7f90b9870740 1 -- 172.21.15.172:0/1690230086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51bc540 msgr2=0x559ae51bea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.579+0000 7f90b9870740 1 --2- 172.21.15.172:0/1690230086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51bc540 0x559ae51bea00 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f90a40d7260 tx=0x7f90a40d7720 comp rx=0 tx=0).stop 2024-09-10T08:55:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.579+0000 7f90b9870740 1 -- 172.21.15.172:0/1690230086 shutdown_connections 2024-09-10T08:55:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.579+0000 7f90b9870740 1 --2- 172.21.15.172:0/1690230086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51bc540 0x559ae51bea00 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.579+0000 7f90b9870740 1 --2- 172.21.15.172:0/1690230086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51b74b0 0x559ae51bbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.579+0000 7f90b9870740 1 --2- 172.21.15.172:0/1690230086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51b6b70 0x559ae51b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.579+0000 7f90b9870740 1 -- 172.21.15.172:0/1690230086 >> 172.21.15.172:0/1690230086 conn(0x559ae51b21e0 msgr2=0x559ae51b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.580+0000 7f90b9870740 1 -- 172.21.15.172:0/1690230086 shutdown_connections 2024-09-10T08:55:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.580+0000 7f90b9870740 1 -- 172.21.15.172:0/1690230086 wait complete. 2024-09-10T08:55:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.580+0000 7f90b9870740 1 -- start start 2024-09-10T08:55:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.580+0000 7f90b9870740 1 Processor -- start 2024-09-10T08:55:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.580+0000 7f90b9870740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51b6b70 0x559ae51c0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.580+0000 7f90b9870740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51b74b0 0x559ae51c1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.580+0000 7f90b9870740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51bc540 0x559ae51c8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.580+0000 7f90b9870740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559ae51bf510 con 0x559ae51b74b0 2024-09-10T08:55:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.580+0000 7f90b9870740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559ae50a3290 con 0x559ae51b6b70 2024-09-10T08:55:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.580+0000 7f90b9870740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559ae51bf690 con 0x559ae51bc540 2024-09-10T08:55:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f90b9870740 10 client.0 _ll_register_callbacks cb 0x559ae50d4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f90af7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51b74b0 0x559ae51c1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f90af7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51b74b0 0x559ae51c1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42560/0 (socket says 172.21.15.172:42560) 2024-09-10T08:55:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f90af7fe640 1 -- 172.21.15.172:0/391135778 learned_addr learned my addr 172.21.15.172:0/391135778 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f90b4f9e640 1 --2- 172.21.15.172:0/391135778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51bc540 0x559ae51c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f90affff640 1 --2- 172.21.15.172:0/391135778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51b6b70 0x559ae51c0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f90af7fe640 1 -- 172.21.15.172:0/391135778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51bc540 msgr2=0x559ae51c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f90af7fe640 1 --2- 172.21.15.172:0/391135778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51bc540 0x559ae51c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f90af7fe640 1 -- 172.21.15.172:0/391135778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51b6b70 msgr2=0x559ae51c0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f90af7fe640 1 --2- 172.21.15.172:0/391135778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51b6b70 0x559ae51c0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f90af7fe640 1 -- 172.21.15.172:0/391135778 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ae4e2d210 con 0x559ae51b74b0 2024-09-10T08:55:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f90af7fe640 1 --2- 172.21.15.172:0/391135778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51b74b0 0x559ae51c1220 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f90a4012040 tx=0x7f90a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee0f23ce8fc9424 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f908f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f908f7fe640 1 -- 172.21.15.172:0/391135778 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4020050 con 0x559ae51b74b0 2024-09-10T08:55:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f908f7fe640 1 -- 172.21.15.172:0/391135778 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f90a409b7e0 con 0x559ae51b74b0 2024-09-10T08:55:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f90b9870740 1 -- 172.21.15.172:0/391135778 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a4009de0 con 0x559ae51b74b0 2024-09-10T08:55:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f908f7fe640 1 -- 172.21.15.172:0/391135778 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f90a40996c0 con 0x559ae51b74b0 2024-09-10T08:55:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.581+0000 7f90b9870740 10 client.18096 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.582+0000 7f90b9870740 1 -- 172.21.15.172:0/391135778 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559ae4ef9620 con 0x559ae51b74b0 2024-09-10T08:55:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.582+0000 7f908f7fe640 1 -- 172.21.15.172:0/391135778 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90a4009de0 con 0x559ae51b74b0 2024-09-10T08:55:17.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.582+0000 7f908f7fe640 1 -- 172.21.15.172:0/391135778 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f90a40d2130 con 0x559ae51b74b0 2024-09-10T08:55:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.582+0000 7f90b9870740 10 client.18096 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.582+0000 7f90b9870740 10 client.18096 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.582+0000 7f90b9870740 20 client.18096 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.582+0000 7f90b9870740 1 --2- 172.21.15.172:0/391135778 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ae51cc560 0x559ae51ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.582+0000 7f90b9870740 4 client.18096 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:55:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.582+0000 7f90b9870740 1 -- 172.21.15.172:0/391135778 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ae5127a20 con 0x559ae51cc560 2024-09-10T08:55:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.582+0000 7f90affff640 1 --2- 172.21.15.172:0/391135778 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ae51cc560 0x559ae51ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.583+0000 7f90affff640 1 --2- 172.21.15.172:0/391135778 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ae51cc560 0x559ae51ce950 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a0c0cec121ad95d server_cookie=661b1a30623dd2da in_seq=0 out_seq=0 2024-09-10T08:55:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.583+0000 7f908f7fe640 10 client.18096 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.583+0000 7f908f7fe640 1 -- 172.21.15.172:0/391135778 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ae5127a20 con 0x559ae51cc560 2024-09-10T08:55:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.583+0000 7f908f7fe640 10 client.18096 handle_command_reply: tid=0 2024-09-10T08:55:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.583+0000 7f90b9870740 1 client.18096 shutdown 2024-09-10T08:55:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.584+0000 7f90b9870740 1 -- 172.21.15.172:0/391135778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51b74b0 msgr2=0x559ae51c1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.584+0000 7f90b9870740 1 --2- 172.21.15.172:0/391135778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51b74b0 0x559ae51c1220 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f90a4012040 tx=0x7f90a4005880 comp rx=0 tx=0).stop 2024-09-10T08:55:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.584+0000 7f90b9870740 1 -- 172.21.15.172:0/391135778 shutdown_connections 2024-09-10T08:55:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.584+0000 7f90b9870740 1 --2- 172.21.15.172:0/391135778 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ae51cc560 0x559ae51ce950 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.584+0000 7f90b9870740 1 --2- 172.21.15.172:0/391135778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51bc540 0x559ae51c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.584+0000 7f90b9870740 1 --2- 172.21.15.172:0/391135778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51b74b0 0x559ae51c1220 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.584+0000 7f90b9870740 1 --2- 172.21.15.172:0/391135778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51b6b70 0x559ae51c0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.584+0000 7f908f7fe640 0 client.18096 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.584+0000 7f90b9870740 1 -- 172.21.15.172:0/391135778 >> 172.21.15.172:0/391135778 conn(0x559ae51b21e0 msgr2=0x559ae51b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.584+0000 7f90b9870740 1 -- 172.21.15.172:0/391135778 shutdown_connections 2024-09-10T08:55:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.584+0000 7f90b9870740 1 -- 172.21.15.172:0/391135778 wait complete. 2024-09-10T08:55:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.584+0000 7f90b9870740 20 client.18096 trim_cache size 0 max 16384 2024-09-10T08:55:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.600+0000 7f90b9870740 1 Processor -- start 2024-09-10T08:55:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.600+0000 7f90b9870740 1 -- start start 2024-09-10T08:55:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.600+0000 7f90b9870740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51b6b70 0x559ae51b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.600+0000 7f90b9870740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51b74b0 0x559ae51c8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.600+0000 7f90b9870740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51bc540 0x559ae51b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.600+0000 7f90b9870740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f90b01aa360 con 0x559ae51bc540 2024-09-10T08:55:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.600+0000 7f90b9870740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f90b01abea0 con 0x559ae51b74b0 2024-09-10T08:55:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.600+0000 7f90b9870740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f90b01aaa30 con 0x559ae51b6b70 2024-09-10T08:55:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.601+0000 7f90b4f9e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51bc540 0x559ae51b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.601+0000 7f90b4f9e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51bc540 0x559ae51b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42576/0 (socket says 172.21.15.172:42576) 2024-09-10T08:55:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.601+0000 7f90b4f9e640 1 -- 172.21.15.172:0/51675028 learned_addr learned my addr 172.21.15.172:0/51675028 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.601+0000 7f90affff640 1 --2- 172.21.15.172:0/51675028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51b6b70 0x559ae51b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.601+0000 7f90af7fe640 1 --2- 172.21.15.172:0/51675028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51b74b0 0x559ae51c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.601+0000 7f90b4f9e640 1 -- 172.21.15.172:0/51675028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51b6b70 msgr2=0x559ae51b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.601+0000 7f90b4f9e640 1 --2- 172.21.15.172:0/51675028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51b6b70 0x559ae51b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.601+0000 7f90b4f9e640 1 -- 172.21.15.172:0/51675028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51b74b0 msgr2=0x559ae51c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.601+0000 7f90b4f9e640 1 --2- 172.21.15.172:0/51675028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51b74b0 0x559ae51c8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.601+0000 7f90b4f9e640 1 -- 172.21.15.172:0/51675028 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ae51b2000 con 0x559ae51bc540 2024-09-10T08:55:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.601+0000 7f90affff640 1 --2- 172.21.15.172:0/51675028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51b6b70 0x559ae51b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.601+0000 7f90b4f9e640 1 --2- 172.21.15.172:0/51675028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51bc540 0x559ae51b16e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f90a8011810 tx=0x7f90a8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa93d632f57c164 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.601+0000 7f9095ffb640 1 -- 172.21.15.172:0/51675028 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8003970 con 0x559ae51bc540 2024-09-10T08:55:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.601+0000 7f9095ffb640 1 -- 172.21.15.172:0/51675028 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90a8004020 con 0x559ae51bc540 2024-09-10T08:55:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.601+0000 7f9095ffb640 1 -- 172.21.15.172:0/51675028 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a801d690 con 0x559ae51bc540 2024-09-10T08:55:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.602+0000 7f90b9870740 1 -- 172.21.15.172:0/51675028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51bc540 msgr2=0x559ae51b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.602+0000 7f90b9870740 1 --2- 172.21.15.172:0/51675028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51bc540 0x559ae51b16e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f90a8011810 tx=0x7f90a8016000 comp rx=0 tx=0).stop 2024-09-10T08:55:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.602+0000 7f90b9870740 1 -- 172.21.15.172:0/51675028 shutdown_connections 2024-09-10T08:55:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.602+0000 7f90b9870740 1 --2- 172.21.15.172:0/51675028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51bc540 0x559ae51b16e0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.602+0000 7f90b9870740 1 --2- 172.21.15.172:0/51675028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51b74b0 0x559ae51c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.602+0000 7f90b9870740 1 --2- 172.21.15.172:0/51675028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51b6b70 0x559ae51b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.602+0000 7f90b9870740 1 -- 172.21.15.172:0/51675028 >> 172.21.15.172:0/51675028 conn(0x559ae51b21e0 msgr2=0x559ae51b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.602+0000 7f90b9870740 1 -- 172.21.15.172:0/51675028 shutdown_connections 2024-09-10T08:55:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.602+0000 7f90b9870740 1 -- 172.21.15.172:0/51675028 wait complete. 2024-09-10T08:55:17.609 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.602+0000 7f90b9870740 1 -- start start 2024-09-10T08:55:17.609 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.603+0000 7f90b9870740 1 Processor -- start 2024-09-10T08:55:17.609 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.603+0000 7f90b9870740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51b6b70 0x559ae51c5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.609 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.603+0000 7f90b9870740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51b74b0 0x559ae51c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.609 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.603+0000 7f90b9870740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51bc540 0x559ae51c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.609 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.603+0000 7f90b9870740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f90b01ab7d0 con 0x559ae51b74b0 2024-09-10T08:55:17.609 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.603+0000 7f90b9870740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f90b01ab100 con 0x559ae51bc540 2024-09-10T08:55:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.603+0000 7f90b9870740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f90b01a8340 con 0x559ae51b6b70 2024-09-10T08:55:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.603+0000 7f90b9870740 10 client.0 _ll_register_callbacks cb 0x7f90a8009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.603+0000 7f90af7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51b74b0 0x559ae51c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.603+0000 7f90af7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51b74b0 0x559ae51c59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42590/0 (socket says 172.21.15.172:42590) 2024-09-10T08:55:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.603+0000 7f90af7fe640 1 -- 172.21.15.172:0/2965523455 learned_addr learned my addr 172.21.15.172:0/2965523455 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.603+0000 7f90affff640 1 --2- 172.21.15.172:0/2965523455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51b6b70 0x559ae51c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.603+0000 7f90b4f9e640 1 --2- 172.21.15.172:0/2965523455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51bc540 0x559ae51c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.603+0000 7f90af7fe640 1 -- 172.21.15.172:0/2965523455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51b6b70 msgr2=0x559ae51c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.603+0000 7f90af7fe640 1 --2- 172.21.15.172:0/2965523455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51b6b70 0x559ae51c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.604+0000 7f90af7fe640 1 -- 172.21.15.172:0/2965523455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51bc540 msgr2=0x559ae51c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:17.610 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.604+0000 7f90af7fe640 1 --2- 172.21.15.172:0/2965523455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51bc540 0x559ae51c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.604+0000 7f90af7fe640 1 -- 172.21.15.172:0/2965523455 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ae51c6ff0 con 0x559ae51b74b0 2024-09-10T08:55:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.604+0000 7f90affff640 1 --2- 172.21.15.172:0/2965523455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51b6b70 0x559ae51c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.604+0000 7f90af7fe640 1 --2- 172.21.15.172:0/2965523455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51b74b0 0x559ae51c59d0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f90a40860b0 tx=0x7f90a4005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9bd8da8f068965 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.604+0000 7f908f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.604+0000 7f908f7fe640 1 -- 172.21.15.172:0/2965523455 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4020070 con 0x559ae51b74b0 2024-09-10T08:55:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.604+0000 7f908f7fe640 1 -- 172.21.15.172:0/2965523455 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f90a40d2560 con 0x559ae51b74b0 2024-09-10T08:55:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.604+0000 7f90b9870740 1 -- 172.21.15.172:0/2965523455 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a8004020 con 0x559ae51b74b0 2024-09-10T08:55:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.604+0000 7f908f7fe640 1 -- 172.21.15.172:0/2965523455 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f90a409aee0 con 0x559ae51b74b0 2024-09-10T08:55:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.604+0000 7f90b9870740 10 client.18102 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.604+0000 7f90b9870740 1 -- 172.21.15.172:0/2965523455 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559ae51c6ff0 con 0x559ae51b74b0 2024-09-10T08:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.604+0000 7f908f7fe640 1 -- 172.21.15.172:0/2965523455 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90a8004020 con 0x559ae51b74b0 2024-09-10T08:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.604+0000 7f908f7fe640 1 -- 172.21.15.172:0/2965523455 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f90a40a3cb0 con 0x559ae51b74b0 2024-09-10T08:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.605+0000 7f90b9870740 10 client.18102 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.605+0000 7f90b9870740 10 client.18102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.605+0000 7f90b9870740 20 client.18102 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.605+0000 7f90b9870740 1 --2- 172.21.15.172:0/2965523455 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ae52372a0 0x559ae5239690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.605+0000 7f90b9870740 4 client.18102 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.605+0000 7f90b9870740 1 -- 172.21.15.172:0/2965523455 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ae5121d70 con 0x559ae52372a0 2024-09-10T08:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.605+0000 7f90affff640 1 --2- 172.21.15.172:0/2965523455 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ae52372a0 0x559ae5239690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.605+0000 7f908f7fe640 10 client.18102 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.605+0000 7f90affff640 1 --2- 172.21.15.172:0/2965523455 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ae52372a0 0x559ae5239690 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=146b825beac91931 server_cookie=b9d37b106f1dd012 in_seq=0 out_seq=0 2024-09-10T08:55:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.611+0000 7f908f7fe640 1 -- 172.21.15.172:0/2965523455 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ae5121d70 con 0x559ae52372a0 2024-09-10T08:55:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.611+0000 7f908f7fe640 10 client.18102 handle_command_reply: tid=0 2024-09-10T08:55:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.611+0000 7f90b9870740 1 client.18102 shutdown 2024-09-10T08:55:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.612+0000 7f90b9870740 1 -- 172.21.15.172:0/2965523455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51b74b0 msgr2=0x559ae51c59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.612+0000 7f90b9870740 1 --2- 172.21.15.172:0/2965523455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51b74b0 0x559ae51c59d0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f90a40860b0 tx=0x7f90a4005bb0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.612+0000 7f90b9870740 1 -- 172.21.15.172:0/2965523455 shutdown_connections 2024-09-10T08:55:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.612+0000 7f90b9870740 1 --2- 172.21.15.172:0/2965523455 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ae52372a0 0x559ae5239690 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.612+0000 7f908f7fe640 0 client.18102 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.612+0000 7f90b9870740 1 --2- 172.21.15.172:0/2965523455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ae51bc540 0x559ae51c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.612+0000 7f90b9870740 1 --2- 172.21.15.172:0/2965523455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ae51b74b0 0x559ae51c59d0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.612+0000 7f90b9870740 1 --2- 172.21.15.172:0/2965523455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ae51b6b70 0x559ae51c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.612+0000 7f90b9870740 1 -- 172.21.15.172:0/2965523455 >> 172.21.15.172:0/2965523455 conn(0x559ae51b21e0 msgr2=0x559ae51b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.612+0000 7f90b9870740 1 -- 172.21.15.172:0/2965523455 shutdown_connections 2024-09-10T08:55:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.612+0000 7f90b9870740 1 -- 172.21.15.172:0/2965523455 wait complete. 2024-09-10T08:55:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.612+0000 7f90b9870740 20 client.18102 trim_cache size 0 max 16384 2024-09-10T08:55:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:55:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:55:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:55:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:55:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.612+0000 7f90b579f640 1 -- 172.21.15.172:0/3111677172 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f90980a2b80 msgr2=0x7f90980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.612+0000 7f90b579f640 1 --2- 172.21.15.172:0/3111677172 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f90980a2b80 0x7f90980a5040 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f90a8005fd0 tx=0x7f90a8009600 comp rx=0 tx=0).stop 2024-09-10T08:55:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.612+0000 7f90b579f640 1 -- 172.21.15.172:0/3111677172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b0101a30 msgr2=0x7f90b0198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.613+0000 7f90b579f640 1 --2- 172.21.15.172:0/3111677172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b0101a30 0x7f90b0198160 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f90a400d950 tx=0x7f90a400de10 comp rx=0 tx=0).stop 2024-09-10T08:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.613+0000 7f90b579f640 1 -- 172.21.15.172:0/3111677172 shutdown_connections 2024-09-10T08:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.613+0000 7f90b579f640 1 --2- 172.21.15.172:0/3111677172 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f90980a2b80 0x7f90980a5040 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.613+0000 7f90b579f640 1 --2- 172.21.15.172:0/3111677172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b01023f0 0x7f90b019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.613+0000 7f90b579f640 1 --2- 172.21.15.172:0/3111677172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b0101a30 0x7f90b0198160 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.613+0000 7f90b579f640 1 --2- 172.21.15.172:0/3111677172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90b0100840 0x7f90b0197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.613+0000 7f90b579f640 1 -- 172.21.15.172:0/3111677172 >> 172.21.15.172:0/3111677172 conn(0x7f90b00fbfa0 msgr2=0x7f90b0069910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.613+0000 7f90b579f640 1 -- 172.21.15.172:0/3111677172 shutdown_connections 2024-09-10T08:55:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:17.613+0000 7f90b579f640 1 -- 172.21.15.172:0/3111677172 wait complete. 2024-09-10T08:55:17.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.713+0000 7f88874cb640 1 -- 172.21.15.172:0/2254434300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88801082a0 msgr2=0x7f888010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.713+0000 7f88874cb640 1 --2- 172.21.15.172:0/2254434300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88801082a0 0x7f888010ec40 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f8878009cd0 tx=0x7f887802a8e0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.713+0000 7f88874cb640 1 -- 172.21.15.172:0/2254434300 shutdown_connections 2024-09-10T08:55:17.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.713+0000 7f88874cb640 1 --2- 172.21.15.172:0/2254434300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f88801082a0 0x7f888010ec40 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.713+0000 7f88874cb640 1 --2- 172.21.15.172:0/2254434300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88801036e0 0x7f8880107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.713+0000 7f88874cb640 1 --2- 172.21.15.172:0/2254434300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8880102d30 0x7f8880103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.713+0000 7f88874cb640 1 -- 172.21.15.172:0/2254434300 >> 172.21.15.172:0/2254434300 conn(0x7f88800fc7d0 msgr2=0x7f88800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:17.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.713+0000 7f88874cb640 1 -- 172.21.15.172:0/2254434300 shutdown_connections 2024-09-10T08:55:17.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.713+0000 7f88874cb640 1 -- 172.21.15.172:0/2254434300 wait complete. 2024-09-10T08:55:17.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.714+0000 7f88874cb640 1 Processor -- start 2024-09-10T08:55:17.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.714+0000 7f88874cb640 1 -- start start 2024-09-10T08:55:17.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.714+0000 7f88874cb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8880102d30 0x7f888019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.714+0000 7f88874cb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88801036e0 0x7f888019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.714+0000 7f88874cb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88801082a0 0x7f888019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f88874cb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8880111f50 con 0x7f8880102d30 2024-09-10T08:55:17.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f8885267640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8880102d30 0x7f888019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f88874cb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8880111dd0 con 0x7f88801082a0 2024-09-10T08:55:17.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f88874cb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f88801120d0 con 0x7f88801036e0 2024-09-10T08:55:17.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f8885267640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8880102d30 0x7f888019a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42610/0 (socket says 172.21.15.172:42610) 2024-09-10T08:55:17.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f8885267640 1 -- 172.21.15.172:0/2807567842 learned_addr learned my addr 172.21.15.172:0/2807567842 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:17.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f8885a68640 1 --2- 172.21.15.172:0/2807567842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88801082a0 0x7f888019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f8884a66640 1 --2- 172.21.15.172:0/2807567842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88801036e0 0x7f888019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f8885267640 1 -- 172.21.15.172:0/2807567842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88801036e0 msgr2=0x7f888019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f8885267640 1 --2- 172.21.15.172:0/2807567842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88801036e0 0x7f888019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f8885267640 1 -- 172.21.15.172:0/2807567842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88801082a0 msgr2=0x7f888019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f8885267640 1 --2- 172.21.15.172:0/2807567842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88801082a0 0x7f888019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f8885267640 1 -- 172.21.15.172:0/2807567842 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888019f6d0 con 0x7f8880102d30 2024-09-10T08:55:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f8885a68640 1 --2- 172.21.15.172:0/2807567842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88801082a0 0x7f888019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f8884a66640 1 --2- 172.21.15.172:0/2807567842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88801036e0 0x7f888019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f8885267640 1 --2- 172.21.15.172:0/2807567842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8880102d30 0x7f888019a720 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f887c00d950 tx=0x7f887c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f88767fc640 1 -- 172.21.15.172:0/2807567842 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c017070 con 0x7f8880102d30 2024-09-10T08:55:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f88767fc640 1 -- 172.21.15.172:0/2807567842 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f887c009c30 con 0x7f8880102d30 2024-09-10T08:55:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f88767fc640 1 -- 172.21.15.172:0/2807567842 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c00f620 con 0x7f8880102d30 2024-09-10T08:55:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.715+0000 7f88874cb640 1 -- 172.21.15.172:0/2807567842 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888019f9c0 con 0x7f8880102d30 2024-09-10T08:55:17.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.716+0000 7f88874cb640 1 -- 172.21.15.172:0/2807567842 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88801a72a0 con 0x7f8880102d30 2024-09-10T08:55:17.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.718+0000 7f88767fc640 1 -- 172.21.15.172:0/2807567842 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f887c0109a0 con 0x7f8880102d30 2024-09-10T08:55:17.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.718+0000 7f88767fc640 1 --2- 172.21.15.172:0/2807567842 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f886c0a2b80 0x7f886c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.719+0000 7f88767fc640 1 -- 172.21.15.172:0/2807567842 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f887c0cf0a0 con 0x7f8880102d30 2024-09-10T08:55:17.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.719+0000 7f8884a66640 1 --2- 172.21.15.172:0/2807567842 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f886c0a2b80 0x7f886c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.719+0000 7f88898a4740 1 Processor -- start 2024-09-10T08:55:17.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.719+0000 7f88898a4740 1 -- start start 2024-09-10T08:55:17.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.719+0000 7f8884a66640 1 --2- 172.21.15.172:0/2807567842 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f886c0a2b80 0x7f886c0a5040 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8870005fd0 tx=0x7f8870005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:17.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.719+0000 7f88898a4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbdb40 0x55c794fbc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.719+0000 7f88898a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbc5d0 0x55c794fbca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.719+0000 7f88898a4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c794fc9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.719+0000 7f88898a4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c794f222f0 con 0x55c794fc7370 2024-09-10T08:55:17.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.719+0000 7f88898a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c794ec9cf0 con 0x55c794fbc5d0 2024-09-10T08:55:17.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.719+0000 7f88898a4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c794f1e2b0 con 0x55c794fbdb40 2024-09-10T08:55:17.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.720+0000 7f8885a68640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c794fc9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.720+0000 7f8885a68640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c794fc9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42622/0 (socket says 172.21.15.172:42622) 2024-09-10T08:55:17.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.720+0000 7f8885a68640 1 -- 172.21.15.172:0/2630007716 learned_addr learned my addr 172.21.15.172:0/2630007716 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:17.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.720+0000 7f8885267640 1 --2- 172.21.15.172:0/2630007716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbdb40 0x55c794fbc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.720+0000 7f8884a66640 1 --2- 172.21.15.172:0/2630007716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbc5d0 0x55c794fbca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.720+0000 7f8885267640 1 -- 172.21.15.172:0/2630007716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbc5d0 msgr2=0x55c794fbca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:17.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.720+0000 7f8885267640 1 --2- 172.21.15.172:0/2630007716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbc5d0 0x55c794fbca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.720+0000 7f8885267640 1 -- 172.21.15.172:0/2630007716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 msgr2=0x55c794fc9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:17.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.720+0000 7f8885267640 1 --2- 172.21.15.172:0/2630007716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c794fc9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.720+0000 7f8885267640 1 -- 172.21.15.172:0/2630007716 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c794eae3a0 con 0x55c794fbdb40 2024-09-10T08:55:17.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.720+0000 7f8885a68640 1 --2- 172.21.15.172:0/2630007716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c794fc9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:17.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.720+0000 7f8885267640 1 --2- 172.21.15.172:0/2630007716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbdb40 0x55c794fbc090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f887c09b2a0 tx=0x7f887c009de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=360e2ecae20c226d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:17.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.720+0000 7f886b7fe640 1 -- 172.21.15.172:0/2630007716 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c020040 con 0x55c794fbdb40 2024-09-10T08:55:17.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.720+0000 7f886b7fe640 1 -- 172.21.15.172:0/2630007716 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f887c09b5a0 con 0x55c794fbdb40 2024-09-10T08:55:17.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.720+0000 7f886b7fe640 1 -- 172.21.15.172:0/2630007716 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c09a9f0 con 0x55c794fbdb40 2024-09-10T08:55:17.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.721+0000 7f88898a4740 1 -- 172.21.15.172:0/2630007716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbdb40 msgr2=0x55c794fbc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.721+0000 7f88898a4740 1 --2- 172.21.15.172:0/2630007716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbdb40 0x55c794fbc090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f887c09b2a0 tx=0x7f887c009de0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.721+0000 7f88898a4740 1 -- 172.21.15.172:0/2630007716 shutdown_connections 2024-09-10T08:55:17.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.721+0000 7f88898a4740 1 --2- 172.21.15.172:0/2630007716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c794fc9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.721+0000 7f88898a4740 1 --2- 172.21.15.172:0/2630007716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbc5d0 0x55c794fbca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.721+0000 7f88898a4740 1 --2- 172.21.15.172:0/2630007716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbdb40 0x55c794fbc090 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.721+0000 7f88898a4740 1 -- 172.21.15.172:0/2630007716 >> 172.21.15.172:0/2630007716 conn(0x55c794f85310 msgr2=0x55c794fb8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:17.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.722+0000 7f88898a4740 1 -- 172.21.15.172:0/2630007716 shutdown_connections 2024-09-10T08:55:17.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.722+0000 7f88898a4740 1 -- 172.21.15.172:0/2630007716 wait complete. 2024-09-10T08:55:17.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.722+0000 7f88898a4740 1 -- start start 2024-09-10T08:55:17.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.722+0000 7f88898a4740 1 Processor -- start 2024-09-10T08:55:17.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.722+0000 7f88898a4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbc5d0 0x55c794fc6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.722+0000 7f88898a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbdb40 0x55c794fc4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f88898a4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c794fc4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f88898a4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c794fca070 con 0x55c794fc7370 2024-09-10T08:55:17.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f88898a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c794edd060 con 0x55c794fbdb40 2024-09-10T08:55:17.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f88898a4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c794fca1f0 con 0x55c794fbc5d0 2024-09-10T08:55:17.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f88898a4740 10 client.0 _ll_register_callbacks cb 0x55c794c308c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:17.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f8885a68640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c794fc4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f8885267640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbc5d0 0x55c794fc6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f8884a66640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbdb40 0x55c794fc4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f8885a68640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c794fc4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42634/0 (socket says 172.21.15.172:42634) 2024-09-10T08:55:17.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f8885a68640 1 -- 172.21.15.172:0/766195280 learned_addr learned my addr 172.21.15.172:0/766195280 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:17.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f8885267640 1 -- 172.21.15.172:0/766195280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbdb40 msgr2=0x55c794fc4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:17.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f8885267640 1 --2- 172.21.15.172:0/766195280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbdb40 0x55c794fc4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f8885267640 1 -- 172.21.15.172:0/766195280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 msgr2=0x55c794fc4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:17.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f8885267640 1 --2- 172.21.15.172:0/766195280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c794fc4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f8885267640 1 -- 172.21.15.172:0/766195280 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c794c933c0 con 0x55c794fbc5d0 2024-09-10T08:55:17.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f8885267640 1 --2- 172.21.15.172:0/766195280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbc5d0 0x55c794fc6d70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f887c097280 tx=0x7f887c0996d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eae5db6a05bc6db9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:17.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f8868ff9640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:55:17.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f8868ff9640 1 -- 172.21.15.172:0/766195280 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c020030 con 0x55c794fbc5d0 2024-09-10T08:55:17.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f88898a4740 1 -- 172.21.15.172:0/766195280 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f887c09b5a0 con 0x55c794fbc5d0 2024-09-10T08:55:17.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.723+0000 7f8868ff9640 1 -- 172.21.15.172:0/766195280 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f887c0b42a0 con 0x55c794fbc5d0 2024-09-10T08:55:17.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.724+0000 7f8868ff9640 1 -- 172.21.15.172:0/766195280 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f887c0a4210 con 0x55c794fbc5d0 2024-09-10T08:55:17.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.724+0000 7f88898a4740 10 client.27052 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:17.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.724+0000 7f88898a4740 1 -- 172.21.15.172:0/766195280 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c794fca0c0 con 0x55c794fbc5d0 2024-09-10T08:55:17.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.724+0000 7f8868ff9640 1 -- 172.21.15.172:0/766195280 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f887c09b5a0 con 0x55c794fbc5d0 2024-09-10T08:55:17.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.724+0000 7f8868ff9640 1 -- 172.21.15.172:0/766195280 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f887c060af0 con 0x55c794fbc5d0 2024-09-10T08:55:17.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.724+0000 7f88898a4740 10 client.27052 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:17.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.724+0000 7f88898a4740 10 client.27052 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:55:17.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.724+0000 7f88898a4740 20 client.27052 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:17.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.724+0000 7f88898a4740 1 --2- 172.21.15.172:0/766195280 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55c794fd6c90 0x55c794fd9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.724+0000 7f88898a4740 4 client.27052 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:55:17.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.724+0000 7f88898a4740 1 -- 172.21.15.172:0/766195280 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c794f2f860 con 0x55c794fd6c90 2024-09-10T08:55:17.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.724+0000 7f8884a66640 1 --2- 172.21.15.172:0/766195280 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55c794fd6c90 0x55c794fd9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.725+0000 7f8868ff9640 10 client.27052 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:55:17.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.725+0000 7f8884a66640 1 --2- 172.21.15.172:0/766195280 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55c794fd6c90 0x55c794fd9080 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=705394253cf97d28 server_cookie=18865129d25c4ef3 in_seq=0 out_seq=0 2024-09-10T08:55:17.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.725+0000 7f8868ff9640 1 -- 172.21.15.172:0/766195280 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c794f2f860 con 0x55c794fd6c90 2024-09-10T08:55:17.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.725+0000 7f8868ff9640 10 client.27052 handle_command_reply: tid=0 2024-09-10T08:55:17.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.725+0000 7f88898a4740 1 client.27052 shutdown 2024-09-10T08:55:17.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.726+0000 7f88898a4740 1 -- 172.21.15.172:0/766195280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbc5d0 msgr2=0x55c794fc6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.726+0000 7f88898a4740 1 --2- 172.21.15.172:0/766195280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbc5d0 0x55c794fc6d70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f887c097280 tx=0x7f887c0996d0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.726+0000 7f88898a4740 1 -- 172.21.15.172:0/766195280 shutdown_connections 2024-09-10T08:55:17.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.726+0000 7f88898a4740 1 --2- 172.21.15.172:0/766195280 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55c794fd6c90 0x55c794fd9080 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.726+0000 7f88898a4740 1 --2- 172.21.15.172:0/766195280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c794fc4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.726+0000 7f88898a4740 1 --2- 172.21.15.172:0/766195280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbdb40 0x55c794fc4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.726+0000 7f8868ff9640 0 client.27052 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:55:17.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.726+0000 7f88898a4740 1 --2- 172.21.15.172:0/766195280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbc5d0 0x55c794fc6d70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.726+0000 7f88898a4740 1 -- 172.21.15.172:0/766195280 >> 172.21.15.172:0/766195280 conn(0x55c794f85310 msgr2=0x55c794fbe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:17.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.726+0000 7f88898a4740 1 -- 172.21.15.172:0/766195280 shutdown_connections 2024-09-10T08:55:17.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.726+0000 7f88898a4740 1 -- 172.21.15.172:0/766195280 wait complete. 2024-09-10T08:55:17.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.726+0000 7f88898a4740 20 client.27052 trim_cache size 0 max 16384 2024-09-10T08:55:17.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.740+0000 7f88898a4740 1 Processor -- start 2024-09-10T08:55:17.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.740+0000 7f88898a4740 1 -- start start 2024-09-10T08:55:17.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.740+0000 7f88898a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbc5d0 0x55c794fd9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.740+0000 7f88898a4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbdb40 0x55c794fd3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.740+0000 7f88898a4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c794fc3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.740+0000 7f88898a4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f88801a84a0 con 0x55c794fc7370 2024-09-10T08:55:17.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.740+0000 7f88898a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f88801a9fe0 con 0x55c794fbc5d0 2024-09-10T08:55:17.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.740+0000 7f88898a4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f88801a8b70 con 0x55c794fbdb40 2024-09-10T08:55:17.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.741+0000 7f8885a68640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c794fc3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.741+0000 7f8884a66640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbdb40 0x55c794fd3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.741+0000 7f8885a68640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c794fc3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42650/0 (socket says 172.21.15.172:42650) 2024-09-10T08:55:17.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.741+0000 7f8885a68640 1 -- 172.21.15.172:0/1879223786 learned_addr learned my addr 172.21.15.172:0/1879223786 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:17.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.741+0000 7f8885267640 1 --2- 172.21.15.172:0/1879223786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbc5d0 0x55c794fd9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.741+0000 7f8884a66640 1 -- 172.21.15.172:0/1879223786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbc5d0 msgr2=0x55c794fd9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:17.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.741+0000 7f8884a66640 1 --2- 172.21.15.172:0/1879223786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbc5d0 0x55c794fd9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.741+0000 7f8884a66640 1 -- 172.21.15.172:0/1879223786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 msgr2=0x55c794fc3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:17.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.741+0000 7f8884a66640 1 --2- 172.21.15.172:0/1879223786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c794fc3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.741+0000 7f8884a66640 1 -- 172.21.15.172:0/1879223786 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c794fbf120 con 0x55c794fbdb40 2024-09-10T08:55:17.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.741+0000 7f8885267640 1 --2- 172.21.15.172:0/1879223786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbc5d0 0x55c794fd9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:17.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.741+0000 7f8884a66640 1 --2- 172.21.15.172:0/1879223786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbdb40 0x55c794fd3310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f88700058c0 tx=0x7f8870012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90faae292f97b31b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:17.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.741+0000 7f886affd640 1 -- 172.21.15.172:0/1879223786 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887002f070 con 0x55c794fbdb40 2024-09-10T08:55:17.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.741+0000 7f886affd640 1 -- 172.21.15.172:0/1879223786 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88700042f0 con 0x55c794fbdb40 2024-09-10T08:55:17.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.741+0000 7f886affd640 1 -- 172.21.15.172:0/1879223786 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8870029700 con 0x55c794fbdb40 2024-09-10T08:55:17.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.742+0000 7f88898a4740 1 -- 172.21.15.172:0/1879223786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbdb40 msgr2=0x55c794fd3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.742+0000 7f88898a4740 1 --2- 172.21.15.172:0/1879223786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbdb40 0x55c794fd3310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f88700058c0 tx=0x7f8870012000 comp rx=0 tx=0).stop 2024-09-10T08:55:17.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.742+0000 7f88898a4740 1 -- 172.21.15.172:0/1879223786 shutdown_connections 2024-09-10T08:55:17.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.742+0000 7f88898a4740 1 --2- 172.21.15.172:0/1879223786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c794fc3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.742+0000 7f88898a4740 1 --2- 172.21.15.172:0/1879223786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbdb40 0x55c794fd3310 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.742+0000 7f88898a4740 1 --2- 172.21.15.172:0/1879223786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbc5d0 0x55c794fd9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.742+0000 7f88898a4740 1 -- 172.21.15.172:0/1879223786 >> 172.21.15.172:0/1879223786 conn(0x55c794f85310 msgr2=0x55c794fb7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:17.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.742+0000 7f88898a4740 1 -- 172.21.15.172:0/1879223786 shutdown_connections 2024-09-10T08:55:17.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.742+0000 7f88898a4740 1 -- 172.21.15.172:0/1879223786 wait complete. 2024-09-10T08:55:17.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.742+0000 7f88898a4740 1 -- start start 2024-09-10T08:55:17.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.743+0000 7f88898a4740 1 Processor -- start 2024-09-10T08:55:17.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.743+0000 7f88898a4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbc5d0 0x55c794fcabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.743+0000 7f88898a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbdb40 0x55c794fcb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.743+0000 7f88898a4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c7950396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.743+0000 7f88898a4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f88801a9910 con 0x55c794fc7370 2024-09-10T08:55:17.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.743+0000 7f88898a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f88801a9240 con 0x55c794fbdb40 2024-09-10T08:55:17.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.743+0000 7f88898a4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8880103110 con 0x55c794fbc5d0 2024-09-10T08:55:17.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.743+0000 7f88898a4740 10 client.0 _ll_register_callbacks cb 0x7f8878009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:17.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.743+0000 7f8885a68640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c7950396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.743+0000 7f8885267640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbc5d0 0x55c794fcabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.743+0000 7f8885a68640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c7950396d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42662/0 (socket says 172.21.15.172:42662) 2024-09-10T08:55:17.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.743+0000 7f8885a68640 1 -- 172.21.15.172:0/1639535563 learned_addr learned my addr 172.21.15.172:0/1639535563 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:17.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.743+0000 7f8884a66640 1 --2- 172.21.15.172:0/1639535563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbdb40 0x55c794fcb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.744+0000 7f8885267640 1 -- 172.21.15.172:0/1639535563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbdb40 msgr2=0x55c794fcb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:17.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.744+0000 7f8885267640 1 --2- 172.21.15.172:0/1639535563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbdb40 0x55c794fcb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.744+0000 7f8885267640 1 -- 172.21.15.172:0/1639535563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 msgr2=0x55c7950396d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:17.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.744+0000 7f8885267640 1 --2- 172.21.15.172:0/1639535563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c7950396d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.744+0000 7f8885267640 1 -- 172.21.15.172:0/1639535563 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c795039d70 con 0x55c794fbc5d0 2024-09-10T08:55:17.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.744+0000 7f8884a66640 1 --2- 172.21.15.172:0/1639535563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbdb40 0x55c794fcb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:17.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.744+0000 7f8885267640 1 --2- 172.21.15.172:0/1639535563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbc5d0 0x55c794fcabe0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f887c0a3ce0 tx=0x7f887c0972f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3feef3bd41cd3228 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:17.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.744+0000 7f8868ff9640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:55:17.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.744+0000 7f8885a68640 1 --2- 172.21.15.172:0/1639535563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c7950396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:17.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.744+0000 7f8868ff9640 1 -- 172.21.15.172:0/1639535563 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c020030 con 0x55c794fbc5d0 2024-09-10T08:55:17.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.744+0000 7f8868ff9640 1 -- 172.21.15.172:0/1639535563 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f887c0b42a0 con 0x55c794fbc5d0 2024-09-10T08:55:17.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.744+0000 7f88898a4740 1 -- 172.21.15.172:0/1639535563 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88700042f0 con 0x55c794fbc5d0 2024-09-10T08:55:17.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.744+0000 7f8868ff9640 1 -- 172.21.15.172:0/1639535563 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f887c084d40 con 0x55c794fbc5d0 2024-09-10T08:55:17.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.744+0000 7f88898a4740 10 client.27058 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:17.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.744+0000 7f88898a4740 1 -- 172.21.15.172:0/1639535563 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c795041140 con 0x55c794fbc5d0 2024-09-10T08:55:17.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.744+0000 7f8868ff9640 1 -- 172.21.15.172:0/1639535563 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88700042f0 con 0x55c794fbc5d0 2024-09-10T08:55:17.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.745+0000 7f8868ff9640 1 -- 172.21.15.172:0/1639535563 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f887c082080 con 0x55c794fbc5d0 2024-09-10T08:55:17.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.745+0000 7f88898a4740 10 client.27058 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:17.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.745+0000 7f88898a4740 10 client.27058 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:55:17.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.745+0000 7f88898a4740 20 client.27058 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:17.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.745+0000 7f88898a4740 1 --2- 172.21.15.172:0/1639535563 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55c795041610 0x55c795043a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:17.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.745+0000 7f88898a4740 4 client.27058 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:55:17.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.745+0000 7f88898a4740 1 -- 172.21.15.172:0/1639535563 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c794f29d70 con 0x55c795041610 2024-09-10T08:55:17.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.745+0000 7f8884a66640 1 --2- 172.21.15.172:0/1639535563 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55c795041610 0x55c795043a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:17.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.746+0000 7f8868ff9640 10 client.27058 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:55:17.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.746+0000 7f8884a66640 1 --2- 172.21.15.172:0/1639535563 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55c795041610 0x55c795043a00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7903e70ffa62913 server_cookie=104cc90c011393d7 in_seq=0 out_seq=0 2024-09-10T08:55:17.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.746+0000 7f8868ff9640 1 -- 172.21.15.172:0/1639535563 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c794f29d70 con 0x55c795041610 2024-09-10T08:55:17.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.746+0000 7f8868ff9640 10 client.27058 handle_command_reply: tid=0 2024-09-10T08:55:17.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.747+0000 7f88898a4740 1 client.27058 shutdown 2024-09-10T08:55:17.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.747+0000 7f88898a4740 1 -- 172.21.15.172:0/1639535563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbc5d0 msgr2=0x55c794fcabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.747+0000 7f88898a4740 1 --2- 172.21.15.172:0/1639535563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbc5d0 0x55c794fcabe0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f887c0a3ce0 tx=0x7f887c0972f0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.747+0000 7f88898a4740 1 -- 172.21.15.172:0/1639535563 shutdown_connections 2024-09-10T08:55:17.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.747+0000 7f88898a4740 1 --2- 172.21.15.172:0/1639535563 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55c795041610 0x55c795043a00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.747+0000 7f88898a4740 1 --2- 172.21.15.172:0/1639535563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c794fc7370 0x55c7950396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.747+0000 7f8868ff9640 0 client.27058 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:55:17.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.747+0000 7f88898a4740 1 --2- 172.21.15.172:0/1639535563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c794fbdb40 0x55c794fcb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.752 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.747+0000 7f88898a4740 1 --2- 172.21.15.172:0/1639535563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c794fbc5d0 0x55c794fcabe0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.747+0000 7f88898a4740 1 -- 172.21.15.172:0/1639535563 >> 172.21.15.172:0/1639535563 conn(0x55c794f85310 msgr2=0x55c794fbe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:17.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.747+0000 7f88898a4740 1 -- 172.21.15.172:0/1639535563 shutdown_connections 2024-09-10T08:55:17.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.747+0000 7f88898a4740 1 -- 172.21.15.172:0/1639535563 wait complete. 2024-09-10T08:55:17.753 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.747+0000 7f88898a4740 20 client.27058 trim_cache size 0 max 16384 2024-09-10T08:55:17.753 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:55:17.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.748+0000 7f88874cb640 1 -- 172.21.15.172:0/2807567842 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f886c0a2b80 msgr2=0x7f886c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.748+0000 7f88874cb640 1 --2- 172.21.15.172:0/2807567842 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f886c0a2b80 0x7f886c0a5040 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8870005fd0 tx=0x7f8870005850 comp rx=0 tx=0).stop 2024-09-10T08:55:17.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.748+0000 7f88874cb640 1 -- 172.21.15.172:0/2807567842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8880102d30 msgr2=0x7f888019a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:17.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.748+0000 7f88874cb640 1 --2- 172.21.15.172:0/2807567842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8880102d30 0x7f888019a720 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f887c00d950 tx=0x7f887c00de10 comp rx=0 tx=0).stop 2024-09-10T08:55:17.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.748+0000 7f88874cb640 1 -- 172.21.15.172:0/2807567842 shutdown_connections 2024-09-10T08:55:17.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.748+0000 7f88874cb640 1 --2- 172.21.15.172:0/2807567842 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f886c0a2b80 0x7f886c0a5040 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.748+0000 7f88874cb640 1 --2- 172.21.15.172:0/2807567842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88801082a0 0x7f888019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.748+0000 7f88874cb640 1 --2- 172.21.15.172:0/2807567842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88801036e0 0x7f888019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.748+0000 7f88874cb640 1 --2- 172.21.15.172:0/2807567842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8880102d30 0x7f888019a720 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:17.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.748+0000 7f88874cb640 1 -- 172.21.15.172:0/2807567842 >> 172.21.15.172:0/2807567842 conn(0x7f88800fc7d0 msgr2=0x7f88800fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:17.754 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.748+0000 7f88874cb640 1 -- 172.21.15.172:0/2807567842 shutdown_connections 2024-09-10T08:55:17.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:17.748+0000 7f88874cb640 1 -- 172.21.15.172:0/2807567842 wait complete. 2024-09-10T08:55:18.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:18 smithi172 ceph-mon[30854]: pgmap v622: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-09-10T08:55:18.656 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:55:18.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:18 smithi174 ceph-mon[34068]: pgmap v622: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-09-10T08:55:18.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:18 smithi191 ceph-mon[35314]: pgmap v622: 129 pgs: 129 active+clean; 8.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 28 MiB/s wr, 20 op/s 2024-09-10T08:55:18.996 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:55:19.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.531+0000 7fcb5349d640 1 -- 172.21.15.172:0/1131218969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4c106a80 msgr2=0x7fcb4c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:19.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.531+0000 7fcb5349d640 1 --2- 172.21.15.172:0/1131218969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4c106a80 0x7fcb4c106e60 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009930 tx=0x7fcb4802a420 comp rx=0 tx=0).stop 2024-09-10T08:55:19.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.531+0000 7fcb5349d640 1 -- 172.21.15.172:0/1131218969 shutdown_connections 2024-09-10T08:55:19.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.531+0000 7fcb5349d640 1 --2- 172.21.15.172:0/1131218969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb4c101420 0x7fcb4c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.531+0000 7fcb5349d640 1 --2- 172.21.15.172:0/1131218969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb4c100a80 0x7fcb4c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.531+0000 7fcb5349d640 1 --2- 172.21.15.172:0/1131218969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4c106a80 0x7fcb4c106e60 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.531+0000 7fcb5349d640 1 -- 172.21.15.172:0/1131218969 >> 172.21.15.172:0/1131218969 conn(0x7fcb4c0fc7a0 msgr2=0x7fcb4c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:19.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.531+0000 7fcb5349d640 1 -- 172.21.15.172:0/1131218969 shutdown_connections 2024-09-10T08:55:19.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.531+0000 7fcb5349d640 1 -- 172.21.15.172:0/1131218969 wait complete. 2024-09-10T08:55:19.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.532+0000 7fcb5349d640 1 Processor -- start 2024-09-10T08:55:19.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.532+0000 7fcb5349d640 1 -- start start 2024-09-10T08:55:19.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.532+0000 7fcb5349d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb4c100a80 0x7fcb4c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.532+0000 7fcb5349d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb4c101420 0x7fcb4c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.532+0000 7fcb5349d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4c106a80 0x7fcb4c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.533+0000 7fcb5349d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c110d30 con 0x7fcb4c106a80 2024-09-10T08:55:19.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.533+0000 7fcb5349d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c110bb0 con 0x7fcb4c101420 2024-09-10T08:55:19.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.533+0000 7fcb5349d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c110eb0 con 0x7fcb4c100a80 2024-09-10T08:55:19.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.533+0000 7fcb51a3a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4c106a80 0x7fcb4c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:19.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.533+0000 7fcb51239640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb4c100a80 0x7fcb4c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:19.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.533+0000 7fcb51a3a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4c106a80 0x7fcb4c19cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42688/0 (socket says 172.21.15.172:42688) 2024-09-10T08:55:19.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.533+0000 7fcb51a3a640 1 -- 172.21.15.172:0/3696912690 learned_addr learned my addr 172.21.15.172:0/3696912690 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:19.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.533+0000 7fcb50a38640 1 --2- 172.21.15.172:0/3696912690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb4c101420 0x7fcb4c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:19.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.533+0000 7fcb51a3a640 1 -- 172.21.15.172:0/3696912690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb4c100a80 msgr2=0x7fcb4c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:19.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.533+0000 7fcb51a3a640 1 --2- 172.21.15.172:0/3696912690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb4c100a80 0x7fcb4c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.533+0000 7fcb51a3a640 1 -- 172.21.15.172:0/3696912690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb4c101420 msgr2=0x7fcb4c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:19.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.533+0000 7fcb51a3a640 1 --2- 172.21.15.172:0/3696912690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb4c101420 0x7fcb4c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.533+0000 7fcb51a3a640 1 -- 172.21.15.172:0/3696912690 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4c19d370 con 0x7fcb4c106a80 2024-09-10T08:55:19.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.533+0000 7fcb50a38640 1 --2- 172.21.15.172:0/3696912690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb4c101420 0x7fcb4c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:19.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.533+0000 7fcb51a3a640 1 --2- 172.21.15.172:0/3696912690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4c106a80 0x7fcb4c19cc90 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fcb44012f20 tx=0x7fcb4400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:19.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.533+0000 7fcb427fc640 1 -- 172.21.15.172:0/3696912690 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44004b20 con 0x7fcb4c106a80 2024-09-10T08:55:19.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.533+0000 7fcb5349d640 1 -- 172.21.15.172:0/3696912690 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4c19d660 con 0x7fcb4c106a80 2024-09-10T08:55:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.534+0000 7fcb427fc640 1 -- 172.21.15.172:0/3696912690 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb44004cc0 con 0x7fcb4c106a80 2024-09-10T08:55:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.534+0000 7fcb427fc640 1 -- 172.21.15.172:0/3696912690 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4401e020 con 0x7fcb4c106a80 2024-09-10T08:55:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.534+0000 7fcb5349d640 1 -- 172.21.15.172:0/3696912690 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb4c0fce00 con 0x7fcb4c106a80 2024-09-10T08:55:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.537+0000 7fcb427fc640 1 -- 172.21.15.172:0/3696912690 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fcb44021020 con 0x7fcb4c106a80 2024-09-10T08:55:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.537+0000 7fcb55876740 1 Processor -- start 2024-09-10T08:55:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.537+0000 7fcb55876740 1 -- start start 2024-09-10T08:55:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.537+0000 7fcb55876740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564302922b40 0x564302921090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.537+0000 7fcb55876740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 0x564302921a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.537+0000 7fcb55876740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56430292c370 0x56430292e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.537+0000 7fcb55876740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5643028872f0 con 0x56430292c370 2024-09-10T08:55:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.537+0000 7fcb55876740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56430282ecf0 con 0x564302922b40 2024-09-10T08:55:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.537+0000 7fcb55876740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5643028832b0 con 0x5643029215d0 2024-09-10T08:55:19.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.538+0000 7fcb50a38640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56430292c370 0x56430292e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:19.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.538+0000 7fcb51239640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 0x564302921a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:19.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.538+0000 7fcb50a38640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56430292c370 0x56430292e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42700/0 (socket says 172.21.15.172:42700) 2024-09-10T08:55:19.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.538+0000 7fcb427fc640 1 --2- 172.21.15.172:0/3696912690 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcb380a2c00 0x7fcb380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.538+0000 7fcb51239640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 0x564302921a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:57686/0 (socket says 172.21.15.172:57686) 2024-09-10T08:55:19.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.538+0000 7fcb50a38640 1 -- 172.21.15.172:0/1536047190 learned_addr learned my addr 172.21.15.172:0/1536047190 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:19.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.538+0000 7fcb50a38640 1 --2- 172.21.15.172:0/1536047190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564302922b40 0x564302921090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:19.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.538+0000 7fcb51a3a640 1 --2- 172.21.15.172:0/3696912690 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcb380a2c00 0x7fcb380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:19.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.538+0000 7fcb51239640 1 -- 172.21.15.172:0/1536047190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564302922b40 msgr2=0x564302921090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:19.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.538+0000 7fcb51239640 1 --2- 172.21.15.172:0/1536047190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564302922b40 0x564302921090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.538+0000 7fcb51239640 1 -- 172.21.15.172:0/1536047190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56430292c370 msgr2=0x56430292e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:19.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.538+0000 7fcb51239640 1 --2- 172.21.15.172:0/1536047190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56430292c370 0x56430292e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.538+0000 7fcb51239640 1 -- 172.21.15.172:0/1536047190 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643028133a0 con 0x5643029215d0 2024-09-10T08:55:19.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.538+0000 7fcb51239640 1 --2- 172.21.15.172:0/1536047190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 0x564302921a10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb48031440 tx=0x7fcb48005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d64e0b5fe276c98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:19.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.538+0000 7fcb427fc640 1 -- 172.21.15.172:0/3696912690 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcb440d3150 con 0x7fcb4c106a80 2024-09-10T08:55:19.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.538+0000 7fcb51a3a640 1 --2- 172.21.15.172:0/3696912690 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcb380a2c00 0x7fcb380a50c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcb440a7d90 tx=0x7fcb4409b1d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:19.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.539+0000 7fcb377fe640 1 -- 172.21.15.172:0/1536047190 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48040070 con 0x5643029215d0 2024-09-10T08:55:19.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.539+0000 7fcb377fe640 1 -- 172.21.15.172:0/1536047190 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb48004030 con 0x5643029215d0 2024-09-10T08:55:19.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.539+0000 7fcb377fe640 1 -- 172.21.15.172:0/1536047190 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48035040 con 0x5643029215d0 2024-09-10T08:55:19.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.539+0000 7fcb55876740 1 -- 172.21.15.172:0/1536047190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 msgr2=0x564302921a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:19.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.539+0000 7fcb55876740 1 --2- 172.21.15.172:0/1536047190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 0x564302921a10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcb48031440 tx=0x7fcb48005be0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.539+0000 7fcb55876740 1 -- 172.21.15.172:0/1536047190 shutdown_connections 2024-09-10T08:55:19.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.539+0000 7fcb55876740 1 --2- 172.21.15.172:0/1536047190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56430292c370 0x56430292e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.539+0000 7fcb55876740 1 --2- 172.21.15.172:0/1536047190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 0x564302921a10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.539+0000 7fcb55876740 1 --2- 172.21.15.172:0/1536047190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564302922b40 0x564302921090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.539+0000 7fcb55876740 1 -- 172.21.15.172:0/1536047190 >> 172.21.15.172:0/1536047190 conn(0x5643028ea310 msgr2=0x56430291dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:19.555 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.539+0000 7fcb55876740 1 -- 172.21.15.172:0/1536047190 shutdown_connections 2024-09-10T08:55:19.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.539+0000 7fcb55876740 1 -- 172.21.15.172:0/1536047190 wait complete. 2024-09-10T08:55:19.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.540+0000 7fcb55876740 1 -- start start 2024-09-10T08:55:19.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.540+0000 7fcb55876740 1 Processor -- start 2024-09-10T08:55:19.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.540+0000 7fcb55876740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 0x56430292bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.540+0000 7fcb55876740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564302922b40 0x564302928fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.540+0000 7fcb55876740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56430292c370 0x564302929570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.540+0000 7fcb55876740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56430292f010 con 0x564302922b40 2024-09-10T08:55:19.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.540+0000 7fcb55876740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564302842060 con 0x56430292c370 2024-09-10T08:55:19.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.540+0000 7fcb55876740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56430292f190 con 0x5643029215d0 2024-09-10T08:55:19.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.541+0000 7fcb55876740 10 client.0 _ll_register_callbacks cb 0x5643025958c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:19.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.541+0000 7fcb50a38640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 0x56430292bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:19.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.541+0000 7fcb50a38640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 0x56430292bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:57702/0 (socket says 172.21.15.172:57702) 2024-09-10T08:55:19.556 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.541+0000 7fcb51239640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564302922b40 0x564302928fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:19.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.541+0000 7fcb50a38640 1 -- 172.21.15.172:0/3425380049 learned_addr learned my addr 172.21.15.172:0/3425380049 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:19.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.541+0000 7fcb50a38640 1 --2- 172.21.15.172:0/3425380049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56430292c370 0x564302929570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:19.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.541+0000 7fcb50a38640 1 -- 172.21.15.172:0/3425380049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56430292c370 msgr2=0x564302929570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:19.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.541+0000 7fcb50a38640 1 --2- 172.21.15.172:0/3425380049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56430292c370 0x564302929570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.541+0000 7fcb50a38640 1 -- 172.21.15.172:0/3425380049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564302922b40 msgr2=0x564302928fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:19.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.541+0000 7fcb50a38640 1 --2- 172.21.15.172:0/3425380049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564302922b40 0x564302928fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.541+0000 7fcb50a38640 1 -- 172.21.15.172:0/3425380049 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643025f83c0 con 0x5643029215d0 2024-09-10T08:55:19.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.541+0000 7fcb51239640 1 --2- 172.21.15.172:0/3425380049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564302922b40 0x564302928fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:19.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.541+0000 7fcb50a38640 1 --2- 172.21.15.172:0/3425380049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 0x56430292bd20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c012de0 tx=0x7fcb3c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de579244152cc9dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:19.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.541+0000 7fcb34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:55:19.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.541+0000 7fcb34ff9640 1 -- 172.21.15.172:0/3425380049 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c00de60 con 0x5643029215d0 2024-09-10T08:55:19.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.541+0000 7fcb55876740 1 -- 172.21.15.172:0/3425380049 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb48004030 con 0x5643029215d0 2024-09-10T08:55:19.557 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.541+0000 7fcb34ff9640 1 -- 172.21.15.172:0/3425380049 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcb3c005020 con 0x5643029215d0 2024-09-10T08:55:19.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.542+0000 7fcb34ff9640 1 -- 172.21.15.172:0/3425380049 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcb3c02b050 con 0x5643029215d0 2024-09-10T08:55:19.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.542+0000 7fcb55876740 10 client.27070 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:19.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.542+0000 7fcb55876740 1 -- 172.21.15.172:0/3425380049 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56430293aab0 con 0x5643029215d0 2024-09-10T08:55:19.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.542+0000 7fcb34ff9640 1 -- 172.21.15.172:0/3425380049 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb48004030 con 0x5643029215d0 2024-09-10T08:55:19.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.542+0000 7fcb34ff9640 1 -- 172.21.15.172:0/3425380049 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fcb3c01e030 con 0x5643029215d0 2024-09-10T08:55:19.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.542+0000 7fcb55876740 10 client.27070 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:19.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.542+0000 7fcb55876740 10 client.27070 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:55:19.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.542+0000 7fcb55876740 20 client.27070 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:19.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.542+0000 7fcb55876740 1 --2- 172.21.15.172:0/3425380049 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x56430293bf70 0x56430293e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.542+0000 7fcb55876740 4 client.27070 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:55:19.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.542+0000 7fcb55876740 1 -- 172.21.15.172:0/3425380049 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564302894860 con 0x56430293bf70 2024-09-10T08:55:19.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.542+0000 7fcb51239640 1 --2- 172.21.15.172:0/3425380049 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x56430293bf70 0x56430293e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:19.558 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.543+0000 7fcb51239640 1 --2- 172.21.15.172:0/3425380049 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x56430293bf70 0x56430293e380 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1116c18795c3957 server_cookie=326e9a5aacf5a4b0 in_seq=0 out_seq=0 2024-09-10T08:55:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.543+0000 7fcb34ff9640 10 client.27070 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:55:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.543+0000 7fcb34ff9640 1 -- 172.21.15.172:0/3425380049 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564302894860 con 0x56430293bf70 2024-09-10T08:55:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.543+0000 7fcb34ff9640 10 client.27070 handle_command_reply: tid=0 2024-09-10T08:55:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.543+0000 7fcb55876740 1 client.27070 shutdown 2024-09-10T08:55:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.543+0000 7fcb55876740 1 -- 172.21.15.172:0/3425380049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 msgr2=0x56430292bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.543+0000 7fcb55876740 1 --2- 172.21.15.172:0/3425380049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 0x56430292bd20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c012de0 tx=0x7fcb3c004730 comp rx=0 tx=0).stop 2024-09-10T08:55:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.544+0000 7fcb55876740 1 -- 172.21.15.172:0/3425380049 shutdown_connections 2024-09-10T08:55:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.544+0000 7fcb55876740 1 --2- 172.21.15.172:0/3425380049 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x56430293bf70 0x56430293e380 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.544+0000 7fcb55876740 1 --2- 172.21.15.172:0/3425380049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56430292c370 0x564302929570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.544+0000 7fcb55876740 1 --2- 172.21.15.172:0/3425380049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564302922b40 0x564302928fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.544+0000 7fcb55876740 1 --2- 172.21.15.172:0/3425380049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 0x56430292bd20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.544+0000 7fcb55876740 1 -- 172.21.15.172:0/3425380049 >> 172.21.15.172:0/3425380049 conn(0x5643028ea310 msgr2=0x564302923960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:19.559 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.544+0000 7fcb34ff9640 0 client.27070 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:55:19.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.544+0000 7fcb55876740 1 -- 172.21.15.172:0/3425380049 shutdown_connections 2024-09-10T08:55:19.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.544+0000 7fcb55876740 1 -- 172.21.15.172:0/3425380049 wait complete. 2024-09-10T08:55:19.560 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.544+0000 7fcb55876740 20 client.27070 trim_cache size 0 max 16384 2024-09-10T08:55:19.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.559+0000 7fcb55876740 1 Processor -- start 2024-09-10T08:55:19.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.559+0000 7fcb55876740 1 -- start start 2024-09-10T08:55:19.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.559+0000 7fcb55876740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5643029215d0 0x56430291de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.561 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.559+0000 7fcb55876740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564302922b40 0x5643029382b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.559+0000 7fcb55876740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56430292c370 0x564302924120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.559+0000 7fcb55876740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c1ac8e0 con 0x5643029215d0 2024-09-10T08:55:19.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.560+0000 7fcb55876740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c1ae420 con 0x564302922b40 2024-09-10T08:55:19.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.560+0000 7fcb55876740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c1acfb0 con 0x56430292c370 2024-09-10T08:55:19.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.560+0000 7fcb50a38640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5643029215d0 0x56430291de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:19.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.560+0000 7fcb50a38640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5643029215d0 0x56430291de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42726/0 (socket says 172.21.15.172:42726) 2024-09-10T08:55:19.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.560+0000 7fcb50a38640 1 -- 172.21.15.172:0/1020910617 learned_addr learned my addr 172.21.15.172:0/1020910617 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:19.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.560+0000 7fcb50a38640 1 --2- 172.21.15.172:0/1020910617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56430292c370 0x564302924120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:19.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.560+0000 7fcb50a38640 1 -- 172.21.15.172:0/1020910617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56430292c370 msgr2=0x564302924120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:19.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.560+0000 7fcb50a38640 1 --2- 172.21.15.172:0/1020910617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56430292c370 0x564302924120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.560+0000 7fcb50a38640 1 -- 172.21.15.172:0/1020910617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564302922b40 msgr2=0x5643029382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:19.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.560+0000 7fcb50a38640 1 --2- 172.21.15.172:0/1020910617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564302922b40 0x5643029382b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.562 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.560+0000 7fcb50a38640 1 -- 172.21.15.172:0/1020910617 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56430292b940 con 0x5643029215d0 2024-09-10T08:55:19.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.560+0000 7fcb50a38640 1 --2- 172.21.15.172:0/1020910617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5643029215d0 0x56430291de10 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c006010 tx=0x7fcb3c0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b7eabe064969c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:19.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.560+0000 7fcb36ffd640 1 -- 172.21.15.172:0/1020910617 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c028040 con 0x5643029215d0 2024-09-10T08:55:19.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.560+0000 7fcb36ffd640 1 -- 172.21.15.172:0/1020910617 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb3c012c10 con 0x5643029215d0 2024-09-10T08:55:19.563 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.560+0000 7fcb36ffd640 1 -- 172.21.15.172:0/1020910617 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c004a00 con 0x5643029215d0 2024-09-10T08:55:19.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.561+0000 7fcb55876740 1 -- 172.21.15.172:0/1020910617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5643029215d0 msgr2=0x56430291de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:19.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.561+0000 7fcb55876740 1 --2- 172.21.15.172:0/1020910617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5643029215d0 0x56430291de10 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c006010 tx=0x7fcb3c0092a0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.561+0000 7fcb55876740 1 -- 172.21.15.172:0/1020910617 shutdown_connections 2024-09-10T08:55:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.561+0000 7fcb55876740 1 --2- 172.21.15.172:0/1020910617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56430292c370 0x564302924120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.561+0000 7fcb55876740 1 --2- 172.21.15.172:0/1020910617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564302922b40 0x5643029382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.561+0000 7fcb55876740 1 --2- 172.21.15.172:0/1020910617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5643029215d0 0x56430291de10 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.561+0000 7fcb55876740 1 -- 172.21.15.172:0/1020910617 >> 172.21.15.172:0/1020910617 conn(0x5643028ea310 msgr2=0x56430291d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.561+0000 7fcb55876740 1 -- 172.21.15.172:0/1020910617 shutdown_connections 2024-09-10T08:55:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.561+0000 7fcb55876740 1 -- 172.21.15.172:0/1020910617 wait complete. 2024-09-10T08:55:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.562+0000 7fcb55876740 1 -- start start 2024-09-10T08:55:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.562+0000 7fcb55876740 1 Processor -- start 2024-09-10T08:55:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.562+0000 7fcb55876740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 0x56430292f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.562+0000 7fcb55876740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564302922b40 0x56430292f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.562+0000 7fcb55876740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56430292c370 0x56430292fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.562+0000 7fcb55876740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c1add50 con 0x564302922b40 2024-09-10T08:55:19.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.562+0000 7fcb55876740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c1ad680 con 0x56430292c370 2024-09-10T08:55:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.562+0000 7fcb55876740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c1aa8c0 con 0x5643029215d0 2024-09-10T08:55:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.562+0000 7fcb55876740 10 client.0 _ll_register_callbacks cb 0x7fcb3c0099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.562+0000 7fcb51239640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564302922b40 0x56430292f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.562+0000 7fcb51239640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564302922b40 0x56430292f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42736/0 (socket says 172.21.15.172:42736) 2024-09-10T08:55:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.562+0000 7fcb51239640 1 -- 172.21.15.172:0/1911619655 learned_addr learned my addr 172.21.15.172:0/1911619655 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.563+0000 7fcb50a38640 1 --2- 172.21.15.172:0/1911619655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56430292c370 0x56430292fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.563+0000 7fcb50a38640 1 --2- 172.21.15.172:0/1911619655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 0x56430292f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.563+0000 7fcb51239640 1 -- 172.21.15.172:0/1911619655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 msgr2=0x56430292f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.563+0000 7fcb51239640 1 --2- 172.21.15.172:0/1911619655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 0x56430292f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.563+0000 7fcb51239640 1 -- 172.21.15.172:0/1911619655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56430292c370 msgr2=0x56430292fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.563+0000 7fcb51239640 1 --2- 172.21.15.172:0/1911619655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56430292c370 0x56430292fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.563+0000 7fcb51239640 1 -- 172.21.15.172:0/1911619655 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643029a4d40 con 0x564302922b40 2024-09-10T08:55:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.563+0000 7fcb50a38640 1 --2- 172.21.15.172:0/1911619655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 0x56430292f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.563+0000 7fcb51239640 1 --2- 172.21.15.172:0/1911619655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564302922b40 0x56430292f920 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009930 tx=0x7fcb48005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67a2a2442e3a46b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.563+0000 7fcb34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.563+0000 7fcb34ff9640 1 -- 172.21.15.172:0/1911619655 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48058060 con 0x564302922b40 2024-09-10T08:55:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.563+0000 7fcb34ff9640 1 -- 172.21.15.172:0/1911619655 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcb480316a0 con 0x564302922b40 2024-09-10T08:55:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.563+0000 7fcb55876740 1 -- 172.21.15.172:0/1911619655 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb3c012c10 con 0x564302922b40 2024-09-10T08:55:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.563+0000 7fcb34ff9640 1 -- 172.21.15.172:0/1911619655 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcb4805f050 con 0x564302922b40 2024-09-10T08:55:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.564+0000 7fcb55876740 10 client.18138 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.564+0000 7fcb55876740 1 -- 172.21.15.172:0/1911619655 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5643029a6000 con 0x564302922b40 2024-09-10T08:55:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.564+0000 7fcb34ff9640 1 -- 172.21.15.172:0/1911619655 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb3c012c10 con 0x564302922b40 2024-09-10T08:55:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.564+0000 7fcb34ff9640 1 -- 172.21.15.172:0/1911619655 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fcb4804e070 con 0x564302922b40 2024-09-10T08:55:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.564+0000 7fcb55876740 10 client.18138 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.564+0000 7fcb55876740 10 client.18138 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:55:19.569 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.564+0000 7fcb55876740 20 client.18138 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:19.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.564+0000 7fcb55876740 1 --2- 172.21.15.172:0/1911619655 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x5643029a6340 0x5643029a8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:19.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.564+0000 7fcb55876740 4 client.18138 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:55:19.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.564+0000 7fcb55876740 1 -- 172.21.15.172:0/1911619655 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56430288ed70 con 0x5643029a6340 2024-09-10T08:55:19.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.564+0000 7fcb50a38640 1 --2- 172.21.15.172:0/1911619655 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x5643029a6340 0x5643029a8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:19.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.565+0000 7fcb50a38640 1 --2- 172.21.15.172:0/1911619655 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x5643029a6340 0x5643029a8730 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1825d9ca5963ee75 server_cookie=5cfa238c60809215 in_seq=0 out_seq=0 2024-09-10T08:55:19.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.565+0000 7fcb34ff9640 10 client.18138 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:55:19.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.565+0000 7fcb34ff9640 1 -- 172.21.15.172:0/1911619655 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56430288ed70 con 0x5643029a6340 2024-09-10T08:55:19.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.565+0000 7fcb34ff9640 10 client.18138 handle_command_reply: tid=0 2024-09-10T08:55:19.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.565+0000 7fcb55876740 1 client.18138 shutdown 2024-09-10T08:55:19.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.565+0000 7fcb55876740 1 -- 172.21.15.172:0/1911619655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564302922b40 msgr2=0x56430292f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:19.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.565+0000 7fcb55876740 1 --2- 172.21.15.172:0/1911619655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564302922b40 0x56430292f920 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009930 tx=0x7fcb48005da0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.565+0000 7fcb55876740 1 -- 172.21.15.172:0/1911619655 shutdown_connections 2024-09-10T08:55:19.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.565+0000 7fcb55876740 1 --2- 172.21.15.172:0/1911619655 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x5643029a6340 0x5643029a8730 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.565+0000 7fcb55876740 1 --2- 172.21.15.172:0/1911619655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56430292c370 0x56430292fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.565+0000 7fcb55876740 1 --2- 172.21.15.172:0/1911619655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564302922b40 0x56430292f920 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.565+0000 7fcb55876740 1 --2- 172.21.15.172:0/1911619655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643029215d0 0x56430292f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.565+0000 7fcb55876740 1 -- 172.21.15.172:0/1911619655 >> 172.21.15.172:0/1911619655 conn(0x5643028ea310 msgr2=0x56430292a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:19.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.565+0000 7fcb34ff9640 0 client.18138 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:55:19.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.565+0000 7fcb55876740 1 -- 172.21.15.172:0/1911619655 shutdown_connections 2024-09-10T08:55:19.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.565+0000 7fcb55876740 1 -- 172.21.15.172:0/1911619655 wait complete. 2024-09-10T08:55:19.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.566+0000 7fcb55876740 20 client.18138 trim_cache size 0 max 16384 2024-09-10T08:55:19.571 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:55:19.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.566+0000 7fcb5349d640 1 -- 172.21.15.172:0/3696912690 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcb380a2c00 msgr2=0x7fcb380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:19.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.566+0000 7fcb5349d640 1 --2- 172.21.15.172:0/3696912690 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcb380a2c00 0x7fcb380a50c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcb440a7d90 tx=0x7fcb4409b1d0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.566+0000 7fcb5349d640 1 -- 172.21.15.172:0/3696912690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4c106a80 msgr2=0x7fcb4c19cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:19.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.566+0000 7fcb5349d640 1 --2- 172.21.15.172:0/3696912690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4c106a80 0x7fcb4c19cc90 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fcb44012f20 tx=0x7fcb4400c760 comp rx=0 tx=0).stop 2024-09-10T08:55:19.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.566+0000 7fcb5349d640 1 -- 172.21.15.172:0/3696912690 shutdown_connections 2024-09-10T08:55:19.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.566+0000 7fcb5349d640 1 --2- 172.21.15.172:0/3696912690 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcb380a2c00 0x7fcb380a50c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.566+0000 7fcb5349d640 1 --2- 172.21.15.172:0/3696912690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4c106a80 0x7fcb4c19cc90 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.566+0000 7fcb5349d640 1 --2- 172.21.15.172:0/3696912690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb4c101420 0x7fcb4c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.566+0000 7fcb5349d640 1 --2- 172.21.15.172:0/3696912690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb4c100a80 0x7fcb4c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:19.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.566+0000 7fcb5349d640 1 -- 172.21.15.172:0/3696912690 >> 172.21.15.172:0/3696912690 conn(0x7fcb4c0fc7a0 msgr2=0x7fcb4c06ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:19.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.566+0000 7fcb5349d640 1 -- 172.21.15.172:0/3696912690 shutdown_connections 2024-09-10T08:55:19.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:19.566+0000 7fcb5349d640 1 -- 172.21.15.172:0/3696912690 wait complete. 2024-09-10T08:55:20.415 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:55:20.437 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:20 smithi172 ceph-mon[30854]: pgmap v623: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:55:20.699 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:55:20.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:20 smithi174 ceph-mon[34068]: pgmap v623: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:55:20.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:20 smithi191 ceph-mon[35314]: pgmap v623: 129 pgs: 129 active+clean; 8.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:55:21.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.229+0000 7f785016a640 1 -- 172.21.15.172:0/2699826515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7848102790 msgr2=0x7f78481067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:21.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.229+0000 7f785016a640 1 --2- 172.21.15.172:0/2699826515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7848102790 0x7f78481067e0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f7838009900 tx=0x7f783802a410 comp rx=0 tx=0).stop 2024-09-10T08:55:21.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.229+0000 7f785016a640 1 -- 172.21.15.172:0/2699826515 shutdown_connections 2024-09-10T08:55:21.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.229+0000 7f785016a640 1 --2- 172.21.15.172:0/2699826515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7848106f10 0x7f784810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.229+0000 7f785016a640 1 --2- 172.21.15.172:0/2699826515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7848102790 0x7f78481067e0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.229+0000 7f785016a640 1 --2- 172.21.15.172:0/2699826515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7848101de0 0x7f78481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.229+0000 7f785016a640 1 -- 172.21.15.172:0/2699826515 >> 172.21.15.172:0/2699826515 conn(0x7f78480fd8e0 msgr2=0x7f78480ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:21.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.229+0000 7f785016a640 1 -- 172.21.15.172:0/2699826515 shutdown_connections 2024-09-10T08:55:21.231 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.229+0000 7f785016a640 1 -- 172.21.15.172:0/2699826515 wait complete. 2024-09-10T08:55:21.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.230+0000 7f785016a640 1 Processor -- start 2024-09-10T08:55:21.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.230+0000 7f785016a640 1 -- start start 2024-09-10T08:55:21.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.230+0000 7f785016a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7848101de0 0x7f78480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.230+0000 7f785016a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7848102790 0x7f7848074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.230+0000 7f785016a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7848106f10 0x7f7848074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.230+0000 7f785016a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7848111e20 con 0x7f7848102790 2024-09-10T08:55:21.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.230+0000 7f785016a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7848111ca0 con 0x7f7848106f10 2024-09-10T08:55:21.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.230+0000 7f785016a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7848111fa0 con 0x7f7848101de0 2024-09-10T08:55:21.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.230+0000 7f784d705640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7848102790 0x7f7848074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.230+0000 7f784d705640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7848102790 0x7f7848074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42752/0 (socket says 172.21.15.172:42752) 2024-09-10T08:55:21.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.230+0000 7f784d705640 1 -- 172.21.15.172:0/389157306 learned_addr learned my addr 172.21.15.172:0/389157306 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:21.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.231+0000 7f784df06640 1 --2- 172.21.15.172:0/389157306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7848101de0 0x7f78480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.231+0000 7f784e707640 1 --2- 172.21.15.172:0/389157306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7848106f10 0x7f7848074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.234 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.231+0000 7f784d705640 1 -- 172.21.15.172:0/389157306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7848101de0 msgr2=0x7f78480744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:21.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.231+0000 7f784d705640 1 --2- 172.21.15.172:0/389157306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7848101de0 0x7f78480744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.231+0000 7f784d705640 1 -- 172.21.15.172:0/389157306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7848106f10 msgr2=0x7f7848074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:21.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.231+0000 7f784d705640 1 --2- 172.21.15.172:0/389157306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7848106f10 0x7f7848074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.231+0000 7f784d705640 1 -- 172.21.15.172:0/389157306 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78481a6860 con 0x7f7848102790 2024-09-10T08:55:21.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.231+0000 7f784e707640 1 --2- 172.21.15.172:0/389157306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7848106f10 0x7f7848074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:21.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.231+0000 7f784d705640 1 --2- 172.21.15.172:0/389157306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7848102790 0x7f7848074a30 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f7838005940 tx=0x7f78380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:21.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.231+0000 7f783effd640 1 -- 172.21.15.172:0/389157306 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783803f030 con 0x7f7848102790 2024-09-10T08:55:21.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.231+0000 7f783effd640 1 -- 172.21.15.172:0/389157306 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7838003cb0 con 0x7f7848102790 2024-09-10T08:55:21.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.231+0000 7f785016a640 1 -- 172.21.15.172:0/389157306 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78481a6af0 con 0x7f7848102790 2024-09-10T08:55:21.235 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.231+0000 7f783effd640 1 -- 172.21.15.172:0/389157306 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838035070 con 0x7f7848102790 2024-09-10T08:55:21.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.231+0000 7f785016a640 1 -- 172.21.15.172:0/389157306 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78481a6fd0 con 0x7f7848102790 2024-09-10T08:55:21.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.234+0000 7f783effd640 1 -- 172.21.15.172:0/389157306 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7838042030 con 0x7f7848102790 2024-09-10T08:55:21.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.234+0000 7f7852543740 1 Processor -- start 2024-09-10T08:55:21.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.234+0000 7f7852543740 1 -- start start 2024-09-10T08:55:21.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f7852543740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf69607b40 0x55bf69606090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f7852543740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf696065d0 0x55bf69606a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f7852543740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69611370 0x55bf69613760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f7852543740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bf6956c2f0 con 0x55bf69607b40 2024-09-10T08:55:21.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f7852543740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf69513cf0 con 0x55bf69611370 2024-09-10T08:55:21.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f7852543740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bf695682b0 con 0x55bf696065d0 2024-09-10T08:55:21.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f784e707640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf69607b40 0x55bf69606090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f784e707640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf69607b40 0x55bf69606090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42756/0 (socket says 172.21.15.172:42756) 2024-09-10T08:55:21.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f784e707640 1 -- 172.21.15.172:0/1668284489 learned_addr learned my addr 172.21.15.172:0/1668284489 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:21.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f784df06640 1 --2- 172.21.15.172:0/1668284489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf696065d0 0x55bf69606a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f783effd640 1 --2- 172.21.15.172:0/389157306 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f78340a2b80 0x7f78340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f784d705640 1 --2- 172.21.15.172:0/1668284489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69611370 0x55bf69613760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f784df06640 1 -- 172.21.15.172:0/1668284489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69611370 msgr2=0x55bf69613760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:21.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f784df06640 1 --2- 172.21.15.172:0/1668284489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69611370 0x55bf69613760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f784df06640 1 -- 172.21.15.172:0/1668284489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf69607b40 msgr2=0x55bf69606090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:21.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f784df06640 1 --2- 172.21.15.172:0/1668284489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf69607b40 0x55bf69606090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f784df06640 1 -- 172.21.15.172:0/1668284489 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf694f83a0 con 0x55bf696065d0 2024-09-10T08:55:21.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f784e707640 1 --2- 172.21.15.172:0/389157306 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f78340a2b80 0x7f78340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.235+0000 7f783effd640 1 -- 172.21.15.172:0/389157306 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f78380ef8a0 con 0x7f7848102790 2024-09-10T08:55:21.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.236+0000 7f784df06640 1 --2- 172.21.15.172:0/1668284489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf696065d0 0x55bf69606a10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7844009f50 tx=0x7f7844010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8398619583fe083e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:21.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.236+0000 7f7833fff640 1 -- 172.21.15.172:0/1668284489 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844003b70 con 0x55bf696065d0 2024-09-10T08:55:21.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.236+0000 7f7833fff640 1 -- 172.21.15.172:0/1668284489 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7844015070 con 0x55bf696065d0 2024-09-10T08:55:21.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.236+0000 7f784e707640 1 --2- 172.21.15.172:0/389157306 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f78340a2b80 0x7f78340a5040 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7840005fd0 tx=0x7f78400046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:21.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.236+0000 7f7833fff640 1 -- 172.21.15.172:0/1668284489 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78440117f0 con 0x55bf696065d0 2024-09-10T08:55:21.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.237+0000 7f7852543740 1 -- 172.21.15.172:0/1668284489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf696065d0 msgr2=0x55bf69606a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:21.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.237+0000 7f7852543740 1 --2- 172.21.15.172:0/1668284489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf696065d0 0x55bf69606a10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7844009f50 tx=0x7f7844010860 comp rx=0 tx=0).stop 2024-09-10T08:55:21.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.237+0000 7f7852543740 1 -- 172.21.15.172:0/1668284489 shutdown_connections 2024-09-10T08:55:21.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.237+0000 7f7852543740 1 --2- 172.21.15.172:0/1668284489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69611370 0x55bf69613760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.237+0000 7f7852543740 1 --2- 172.21.15.172:0/1668284489 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf696065d0 0x55bf69606a10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.237+0000 7f7852543740 1 --2- 172.21.15.172:0/1668284489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf69607b40 0x55bf69606090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.237+0000 7f7852543740 1 -- 172.21.15.172:0/1668284489 >> 172.21.15.172:0/1668284489 conn(0x55bf695cf310 msgr2=0x55bf69602c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:21.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.237+0000 7f7852543740 1 -- 172.21.15.172:0/1668284489 shutdown_connections 2024-09-10T08:55:21.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.237+0000 7f7852543740 1 -- 172.21.15.172:0/1668284489 wait complete. 2024-09-10T08:55:21.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.237+0000 7f7852543740 1 -- start start 2024-09-10T08:55:21.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.237+0000 7f7852543740 1 Processor -- start 2024-09-10T08:55:21.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f7852543740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf696065d0 0x55bf69610cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f7852543740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69607b40 0x55bf6960df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f7852543740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf69611370 0x55bf6960e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f7852543740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bf69614070 con 0x55bf696065d0 2024-09-10T08:55:21.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f7852543740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf69527060 con 0x55bf69607b40 2024-09-10T08:55:21.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f7852543740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bf696141f0 con 0x55bf69611370 2024-09-10T08:55:21.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f7852543740 10 client.0 _ll_register_callbacks cb 0x55bf6927a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:21.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f784df06640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf696065d0 0x55bf69610cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f784df06640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf696065d0 0x55bf69610cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42770/0 (socket says 172.21.15.172:42770) 2024-09-10T08:55:21.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f784df06640 1 -- 172.21.15.172:0/50194141 learned_addr learned my addr 172.21.15.172:0/50194141 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:21.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f784d705640 1 --2- 172.21.15.172:0/50194141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69607b40 0x55bf6960df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f784e707640 1 --2- 172.21.15.172:0/50194141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf69611370 0x55bf6960e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f784df06640 1 -- 172.21.15.172:0/50194141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf69611370 msgr2=0x55bf6960e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:21.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f784df06640 1 --2- 172.21.15.172:0/50194141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf69611370 0x55bf6960e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f784df06640 1 -- 172.21.15.172:0/50194141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69607b40 msgr2=0x55bf6960df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:21.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f784df06640 1 --2- 172.21.15.172:0/50194141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69607b40 0x55bf6960df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f784df06640 1 -- 172.21.15.172:0/50194141 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf692dd3c0 con 0x55bf696065d0 2024-09-10T08:55:21.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.238+0000 7f784e707640 1 --2- 172.21.15.172:0/50194141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf69611370 0x55bf6960e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:21.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.239+0000 7f784df06640 1 --2- 172.21.15.172:0/50194141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf696065d0 0x55bf69610cc0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f78440117a0 tx=0x7f78440117d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748eeaefd38a792e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:21.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.239+0000 7f78317fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:21.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.239+0000 7f78317fa640 1 -- 172.21.15.172:0/50194141 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784402c060 con 0x55bf696065d0 2024-09-10T08:55:21.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.239+0000 7f78317fa640 1 -- 172.21.15.172:0/50194141 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f7844011c00 con 0x55bf696065d0 2024-09-10T08:55:21.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.239+0000 7f7852543740 1 -- 172.21.15.172:0/50194141 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7844015070 con 0x55bf696065d0 2024-09-10T08:55:21.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.239+0000 7f78317fa640 1 -- 172.21.15.172:0/50194141 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7844033050 con 0x55bf696065d0 2024-09-10T08:55:21.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.239+0000 7f7852543740 10 client.18150 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:21.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.239+0000 7f7852543740 1 -- 172.21.15.172:0/50194141 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bf6961fa80 con 0x55bf696065d0 2024-09-10T08:55:21.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.239+0000 7f78317fa640 1 -- 172.21.15.172:0/50194141 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7844015070 con 0x55bf696065d0 2024-09-10T08:55:21.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.240+0000 7f78317fa640 1 -- 172.21.15.172:0/50194141 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f784401a380 con 0x55bf696065d0 2024-09-10T08:55:21.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.240+0000 7f7852543740 10 client.18150 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:21.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.240+0000 7f7852543740 10 client.18150 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:55:21.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.240+0000 7f7852543740 20 client.18150 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:21.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.240+0000 7f7852543740 1 --2- 172.21.15.172:0/50194141 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55bf69620e80 0x55bf69623270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.240+0000 7f7852543740 4 client.18150 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:55:21.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.240+0000 7f7852543740 1 -- 172.21.15.172:0/50194141 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf69579860 con 0x55bf69620e80 2024-09-10T08:55:21.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.240+0000 7f784d705640 1 --2- 172.21.15.172:0/50194141 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55bf69620e80 0x55bf69623270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.240+0000 7f784d705640 1 --2- 172.21.15.172:0/50194141 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55bf69620e80 0x55bf69623270 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60f4501e8ba07003 server_cookie=1019b0254c906d82 in_seq=0 out_seq=0 2024-09-10T08:55:21.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.240+0000 7f78317fa640 10 client.18150 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:55:21.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.241+0000 7f78317fa640 1 -- 172.21.15.172:0/50194141 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf69579860 con 0x55bf69620e80 2024-09-10T08:55:21.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.241+0000 7f78317fa640 10 client.18150 handle_command_reply: tid=0 2024-09-10T08:55:21.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.241+0000 7f7852543740 1 client.18150 shutdown 2024-09-10T08:55:21.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.241+0000 7f7852543740 1 -- 172.21.15.172:0/50194141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf696065d0 msgr2=0x55bf69610cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:21.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.241+0000 7f7852543740 1 --2- 172.21.15.172:0/50194141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf696065d0 0x55bf69610cc0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f78440117a0 tx=0x7f78440117d0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.241+0000 7f7852543740 1 -- 172.21.15.172:0/50194141 shutdown_connections 2024-09-10T08:55:21.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.241+0000 7f7852543740 1 --2- 172.21.15.172:0/50194141 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55bf69620e80 0x55bf69623270 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.241+0000 7f7852543740 1 --2- 172.21.15.172:0/50194141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf69611370 0x55bf6960e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.241+0000 7f7852543740 1 --2- 172.21.15.172:0/50194141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69607b40 0x55bf6960df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.241+0000 7f78317fa640 0 client.18150 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:55:21.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.241+0000 7f7852543740 1 --2- 172.21.15.172:0/50194141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf696065d0 0x55bf69610cc0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.241+0000 7f7852543740 1 -- 172.21.15.172:0/50194141 >> 172.21.15.172:0/50194141 conn(0x55bf695cf310 msgr2=0x55bf69608960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:21.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.241+0000 7f7852543740 1 -- 172.21.15.172:0/50194141 shutdown_connections 2024-09-10T08:55:21.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.241+0000 7f7852543740 1 -- 172.21.15.172:0/50194141 wait complete. 2024-09-10T08:55:21.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.242+0000 7f7852543740 20 client.18150 trim_cache size 0 max 16384 2024-09-10T08:55:21.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.258+0000 7f7852543740 1 Processor -- start 2024-09-10T08:55:21.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.258+0000 7f7852543740 1 -- start start 2024-09-10T08:55:21.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.258+0000 7f7852543740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf696065d0 0x55bf6960fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.258+0000 7f7852543740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf69607b40 0x55bf69609120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.258+0000 7f7852543740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69611370 0x55bf69609660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.258+0000 7f7852543740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b0ce0 con 0x55bf69607b40 2024-09-10T08:55:21.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.258+0000 7f7852543740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b2820 con 0x55bf69611370 2024-09-10T08:55:21.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.258+0000 7f7852543740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b13b0 con 0x55bf696065d0 2024-09-10T08:55:21.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.258+0000 7f784d705640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf69607b40 0x55bf69609120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.258+0000 7f784d705640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf69607b40 0x55bf69609120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42774/0 (socket says 172.21.15.172:42774) 2024-09-10T08:55:21.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.258+0000 7f784d705640 1 -- 172.21.15.172:0/367242704 learned_addr learned my addr 172.21.15.172:0/367242704 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:21.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.259+0000 7f784df06640 1 --2- 172.21.15.172:0/367242704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf696065d0 0x55bf6960fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.259+0000 7f784e707640 1 --2- 172.21.15.172:0/367242704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69611370 0x55bf69609660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.259+0000 7f784d705640 1 -- 172.21.15.172:0/367242704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf696065d0 msgr2=0x55bf6960fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:21.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.259+0000 7f784d705640 1 --2- 172.21.15.172:0/367242704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf696065d0 0x55bf6960fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.259+0000 7f784d705640 1 -- 172.21.15.172:0/367242704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69611370 msgr2=0x55bf69609660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:21.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.259+0000 7f784d705640 1 --2- 172.21.15.172:0/367242704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69611370 0x55bf69609660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.259+0000 7f784d705640 1 -- 172.21.15.172:0/367242704 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf696141e0 con 0x55bf69607b40 2024-09-10T08:55:21.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.259+0000 7f784e707640 1 --2- 172.21.15.172:0/367242704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69611370 0x55bf69609660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:21.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.259+0000 7f784d705640 1 --2- 172.21.15.172:0/367242704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf69607b40 0x55bf69609120 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f78380bbd30 tx=0x7f783802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8f4c37096f9144 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:21.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.259+0000 7f78337fe640 1 -- 172.21.15.172:0/367242704 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838049040 con 0x55bf69607b40 2024-09-10T08:55:21.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.259+0000 7f78337fe640 1 -- 172.21.15.172:0/367242704 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f78380bc4a0 con 0x55bf69607b40 2024-09-10T08:55:21.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.259+0000 7f78337fe640 1 -- 172.21.15.172:0/367242704 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78380b8410 con 0x55bf69607b40 2024-09-10T08:55:21.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.260+0000 7f7852543740 1 -- 172.21.15.172:0/367242704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf69607b40 msgr2=0x55bf69609120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:21.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.260+0000 7f7852543740 1 --2- 172.21.15.172:0/367242704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf69607b40 0x55bf69609120 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f78380bbd30 tx=0x7f783802a410 comp rx=0 tx=0).stop 2024-09-10T08:55:21.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.260+0000 7f7852543740 1 -- 172.21.15.172:0/367242704 shutdown_connections 2024-09-10T08:55:21.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.260+0000 7f7852543740 1 --2- 172.21.15.172:0/367242704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69611370 0x55bf69609660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.260+0000 7f7852543740 1 --2- 172.21.15.172:0/367242704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf69607b40 0x55bf69609120 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.260+0000 7f7852543740 1 --2- 172.21.15.172:0/367242704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf696065d0 0x55bf6960fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.260+0000 7f7852543740 1 -- 172.21.15.172:0/367242704 >> 172.21.15.172:0/367242704 conn(0x55bf695cf310 msgr2=0x55bf6960f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:21.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.260+0000 7f7852543740 1 -- 172.21.15.172:0/367242704 shutdown_connections 2024-09-10T08:55:21.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.260+0000 7f7852543740 1 -- 172.21.15.172:0/367242704 wait complete. 2024-09-10T08:55:21.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.261+0000 7f7852543740 1 -- start start 2024-09-10T08:55:21.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.261+0000 7f7852543740 1 Processor -- start 2024-09-10T08:55:21.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.261+0000 7f7852543740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf696065d0 0x55bf6960b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.261+0000 7f7852543740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf69607b40 0x55bf6960b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.261+0000 7f7852543740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69611370 0x55bf6960ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.261+0000 7f7852543740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b2150 con 0x55bf696065d0 2024-09-10T08:55:21.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.261+0000 7f7852543740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b1a80 con 0x55bf69611370 2024-09-10T08:55:21.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.261+0000 7f7852543740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f78481aecc0 con 0x55bf69607b40 2024-09-10T08:55:21.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.261+0000 7f7852543740 10 client.0 _ll_register_callbacks cb 0x7f784000b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:21.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.261+0000 7f784df06640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf696065d0 0x55bf6960b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.262+0000 7f784df06640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf696065d0 0x55bf6960b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42790/0 (socket says 172.21.15.172:42790) 2024-09-10T08:55:21.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.262+0000 7f784df06640 1 -- 172.21.15.172:0/4131951594 learned_addr learned my addr 172.21.15.172:0/4131951594 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:21.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.262+0000 7f784d705640 1 --2- 172.21.15.172:0/4131951594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf69607b40 0x55bf6960b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.262+0000 7f784e707640 1 --2- 172.21.15.172:0/4131951594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69611370 0x55bf6960ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.262+0000 7f784df06640 1 -- 172.21.15.172:0/4131951594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf69607b40 msgr2=0x55bf6960b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:21.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.262+0000 7f784df06640 1 --2- 172.21.15.172:0/4131951594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf69607b40 0x55bf6960b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.262+0000 7f784df06640 1 -- 172.21.15.172:0/4131951594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69611370 msgr2=0x55bf6960ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:21.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.262+0000 7f784df06640 1 --2- 172.21.15.172:0/4131951594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69611370 0x55bf6960ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.262+0000 7f784df06640 1 -- 172.21.15.172:0/4131951594 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf6968a8c0 con 0x55bf696065d0 2024-09-10T08:55:21.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.262+0000 7f784d705640 1 --2- 172.21.15.172:0/4131951594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf69607b40 0x55bf6960b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:21.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.262+0000 7f784df06640 1 --2- 172.21.15.172:0/4131951594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf696065d0 0x55bf6960b010 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f784401a940 tx=0x7f784401a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea124e8707857d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:21.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.262+0000 7f78317fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:21.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.262+0000 7f78317fa640 1 -- 172.21.15.172:0/4131951594 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784402c070 con 0x55bf696065d0 2024-09-10T08:55:21.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.262+0000 7f78317fa640 1 -- 172.21.15.172:0/4131951594 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f784401a9c0 con 0x55bf696065d0 2024-09-10T08:55:21.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.262+0000 7f7852543740 1 -- 172.21.15.172:0/4131951594 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78380bc4a0 con 0x55bf696065d0 2024-09-10T08:55:21.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.263+0000 7f78317fa640 1 -- 172.21.15.172:0/4131951594 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f784403f050 con 0x55bf696065d0 2024-09-10T08:55:21.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.263+0000 7f7852543740 10 client.18156 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:21.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.263+0000 7f7852543740 1 -- 172.21.15.172:0/4131951594 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bf6968bba0 con 0x55bf696065d0 2024-09-10T08:55:21.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.263+0000 7f78317fa640 1 -- 172.21.15.172:0/4131951594 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7844021070 con 0x55bf696065d0 2024-09-10T08:55:21.270 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.263+0000 7f78317fa640 1 -- 172.21.15.172:0/4131951594 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f7844005110 con 0x55bf696065d0 2024-09-10T08:55:21.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.263+0000 7f7852543740 10 client.18156 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:21.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.263+0000 7f7852543740 10 client.18156 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:55:21.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.263+0000 7f7852543740 20 client.18156 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:21.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.263+0000 7f7852543740 1 --2- 172.21.15.172:0/4131951594 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55bf6968c200 0x55bf6968e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:21.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.263+0000 7f7852543740 4 client.18156 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:55:21.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.263+0000 7f7852543740 1 -- 172.21.15.172:0/4131951594 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f78380bc4a0 con 0x55bf6968c200 2024-09-10T08:55:21.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.263+0000 7f784d705640 1 --2- 172.21.15.172:0/4131951594 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55bf6968c200 0x55bf6968e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:21.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.264+0000 7f784d705640 1 --2- 172.21.15.172:0/4131951594 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55bf6968c200 0x55bf6968e5f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d810f52b2265752 server_cookie=9f7436c50fa154b2 in_seq=0 out_seq=0 2024-09-10T08:55:21.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.264+0000 7f78317fa640 10 client.18156 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:55:21.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.264+0000 7f78317fa640 1 -- 172.21.15.172:0/4131951594 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f78380bc4a0 con 0x55bf6968c200 2024-09-10T08:55:21.271 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.264+0000 7f78317fa640 10 client.18156 handle_command_reply: tid=0 2024-09-10T08:55:21.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.264+0000 7f7852543740 1 client.18156 shutdown 2024-09-10T08:55:21.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.264+0000 7f7852543740 1 -- 172.21.15.172:0/4131951594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf696065d0 msgr2=0x55bf6960b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:21.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.264+0000 7f7852543740 1 --2- 172.21.15.172:0/4131951594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf696065d0 0x55bf6960b010 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f784401a940 tx=0x7f784401a630 comp rx=0 tx=0).stop 2024-09-10T08:55:21.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.264+0000 7f7852543740 1 -- 172.21.15.172:0/4131951594 shutdown_connections 2024-09-10T08:55:21.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.264+0000 7f7852543740 1 --2- 172.21.15.172:0/4131951594 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55bf6968c200 0x55bf6968e5f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.264+0000 7f7852543740 1 --2- 172.21.15.172:0/4131951594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf69611370 0x55bf6960ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.264+0000 7f78317fa640 0 client.18156 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:55:21.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.264+0000 7f7852543740 1 --2- 172.21.15.172:0/4131951594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf69607b40 0x55bf6960b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.264+0000 7f7852543740 1 --2- 172.21.15.172:0/4131951594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf696065d0 0x55bf6960b010 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.264+0000 7f7852543740 1 -- 172.21.15.172:0/4131951594 >> 172.21.15.172:0/4131951594 conn(0x55bf695cf310 msgr2=0x55bf69601d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:21.272 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.264+0000 7f7852543740 1 -- 172.21.15.172:0/4131951594 shutdown_connections 2024-09-10T08:55:21.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.264+0000 7f7852543740 1 -- 172.21.15.172:0/4131951594 wait complete. 2024-09-10T08:55:21.273 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.265+0000 7f7852543740 20 client.18156 trim_cache size 0 max 16384 2024-09-10T08:55:21.273 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:55:21.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.265+0000 7f785016a640 1 -- 172.21.15.172:0/389157306 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f78340a2b80 msgr2=0x7f78340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:21.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.265+0000 7f785016a640 1 --2- 172.21.15.172:0/389157306 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f78340a2b80 0x7f78340a5040 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7840005fd0 tx=0x7f78400046c0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.265+0000 7f785016a640 1 -- 172.21.15.172:0/389157306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7848102790 msgr2=0x7f7848074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:21.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.265+0000 7f785016a640 1 --2- 172.21.15.172:0/389157306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7848102790 0x7f7848074a30 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f7838005940 tx=0x7f78380042f0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.265+0000 7f785016a640 1 -- 172.21.15.172:0/389157306 shutdown_connections 2024-09-10T08:55:21.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.265+0000 7f785016a640 1 --2- 172.21.15.172:0/389157306 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f78340a2b80 0x7f78340a5040 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.265+0000 7f785016a640 1 --2- 172.21.15.172:0/389157306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7848106f10 0x7f7848074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.265+0000 7f785016a640 1 --2- 172.21.15.172:0/389157306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7848102790 0x7f7848074a30 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.265+0000 7f785016a640 1 --2- 172.21.15.172:0/389157306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7848101de0 0x7f78480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:21.274 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.265+0000 7f785016a640 1 -- 172.21.15.172:0/389157306 >> 172.21.15.172:0/389157306 conn(0x7f78480fd8e0 msgr2=0x7f78480fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:21.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.265+0000 7f785016a640 1 -- 172.21.15.172:0/389157306 shutdown_connections 2024-09-10T08:55:21.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:21.265+0000 7f785016a640 1 -- 172.21.15.172:0/389157306 wait complete. 2024-09-10T08:55:22.076 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:55:22.404 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:55:22.674 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:22 smithi172 ceph-mon[30854]: pgmap v624: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-09-10T08:55:22.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:22 smithi174 ceph-mon[34068]: pgmap v624: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-09-10T08:55:22.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:22 smithi191 ceph-mon[35314]: pgmap v624: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 32 MiB/s wr, 26 op/s 2024-09-10T08:55:22.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.927+0000 7f494ca69640 1 -- 172.21.15.172:0/3370752981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4948106f20 msgr2=0x7f494810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:22.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.927+0000 7f494ca69640 1 --2- 172.21.15.172:0/3370752981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4948106f20 0x7f494810eaa0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f4940009ca0 tx=0x7f494002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.927+0000 7f494ca69640 1 -- 172.21.15.172:0/3370752981 shutdown_connections 2024-09-10T08:55:22.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.927+0000 7f494ca69640 1 --2- 172.21.15.172:0/3370752981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4948106f20 0x7f494810eaa0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.927+0000 7f494ca69640 1 --2- 172.21.15.172:0/3370752981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49481027a0 0x7f49481067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.927+0000 7f494ca69640 1 --2- 172.21.15.172:0/3370752981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4948101df0 0x7f49481021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.927+0000 7f494ca69640 1 -- 172.21.15.172:0/3370752981 >> 172.21.15.172:0/3370752981 conn(0x7f49480fd8f0 msgr2=0x7f49480ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:22.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.927+0000 7f494ca69640 1 -- 172.21.15.172:0/3370752981 shutdown_connections 2024-09-10T08:55:22.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.927+0000 7f494ca69640 1 -- 172.21.15.172:0/3370752981 wait complete. 2024-09-10T08:55:22.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.928+0000 7f494ca69640 1 Processor -- start 2024-09-10T08:55:22.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.928+0000 7f494ca69640 1 -- start start 2024-09-10T08:55:22.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.928+0000 7f494ca69640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4948101df0 0x7f4948198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.928+0000 7f494ca69640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49481027a0 0x7f4948198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.928+0000 7f494ca69640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4948106f20 0x7f494819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.928+0000 7f494ca69640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4948111e80 con 0x7f4948101df0 2024-09-10T08:55:22.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.928+0000 7f494ca69640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4948111d00 con 0x7f49481027a0 2024-09-10T08:55:22.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.928+0000 7f494ca69640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4948112000 con 0x7f4948106f20 2024-09-10T08:55:22.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.929+0000 7f494659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4948101df0 0x7f4948198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.929+0000 7f494659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4948101df0 0x7f4948198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42818/0 (socket says 172.21.15.172:42818) 2024-09-10T08:55:22.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.929+0000 7f494659c640 1 -- 172.21.15.172:0/4255362861 learned_addr learned my addr 172.21.15.172:0/4255362861 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:22.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.929+0000 7f4946d9d640 1 --2- 172.21.15.172:0/4255362861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4948106f20 0x7f494819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.929+0000 7f4945d9b640 1 --2- 172.21.15.172:0/4255362861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49481027a0 0x7f4948198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.929+0000 7f494659c640 1 -- 172.21.15.172:0/4255362861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4948106f20 msgr2=0x7f494819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:22.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.929+0000 7f494659c640 1 --2- 172.21.15.172:0/4255362861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4948106f20 0x7f494819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.929+0000 7f494659c640 1 -- 172.21.15.172:0/4255362861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49481027a0 msgr2=0x7f4948198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:22.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.929+0000 7f494659c640 1 --2- 172.21.15.172:0/4255362861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49481027a0 0x7f4948198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.929+0000 7f494659c640 1 -- 172.21.15.172:0/4255362861 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494819d2b0 con 0x7f4948101df0 2024-09-10T08:55:22.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.929+0000 7f4946d9d640 1 --2- 172.21.15.172:0/4255362861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4948106f20 0x7f494819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:22.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.929+0000 7f494659c640 1 --2- 172.21.15.172:0/4255362861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4948101df0 0x7f4948198300 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f493800e9f0 tx=0x7f493800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:22.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.929+0000 7f49377fe640 1 -- 172.21.15.172:0/4255362861 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493800cd80 con 0x7f4948101df0 2024-09-10T08:55:22.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.929+0000 7f494ca69640 1 -- 172.21.15.172:0/4255362861 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494819d5a0 con 0x7f4948101df0 2024-09-10T08:55:22.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.929+0000 7f49377fe640 1 -- 172.21.15.172:0/4255362861 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49380108e0 con 0x7f4948101df0 2024-09-10T08:55:22.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.929+0000 7f49377fe640 1 -- 172.21.15.172:0/4255362861 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938018920 con 0x7f4948101df0 2024-09-10T08:55:22.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.929+0000 7f494ca69640 1 -- 172.21.15.172:0/4255362861 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49481a4e80 con 0x7f4948101df0 2024-09-10T08:55:22.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.932+0000 7f49377fe640 1 -- 172.21.15.172:0/4255362861 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4938010a90 con 0x7f4948101df0 2024-09-10T08:55:22.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.932+0000 7f494ee42740 1 Processor -- start 2024-09-10T08:55:22.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.933+0000 7f494ee42740 1 -- start start 2024-09-10T08:55:22.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.933+0000 7f494ee42740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7d0b40 0x56044f7cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.933+0000 7f494ee42740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7cf5d0 0x56044f7cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.933+0000 7f494ee42740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7da370 0x56044f7dc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.933+0000 7f494ee42740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56044f7352f0 con 0x56044f7d0b40 2024-09-10T08:55:22.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.933+0000 7f494ee42740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56044f6dccf0 con 0x56044f7da370 2024-09-10T08:55:22.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.933+0000 7f494ee42740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56044f7312b0 con 0x56044f7cf5d0 2024-09-10T08:55:22.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.933+0000 7f4946d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7d0b40 0x56044f7cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.933+0000 7f4946d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7d0b40 0x56044f7cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42822/0 (socket says 172.21.15.172:42822) 2024-09-10T08:55:22.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.933+0000 7f4946d9d640 1 -- 172.21.15.172:0/219114029 learned_addr learned my addr 172.21.15.172:0/219114029 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:22.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.933+0000 7f4945d9b640 1 --2- 172.21.15.172:0/219114029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7da370 0x56044f7dc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.933+0000 7f49377fe640 1 --2- 172.21.15.172:0/4255362861 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49300a2b80 0x7f49300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.933+0000 7f494659c640 1 --2- 172.21.15.172:0/219114029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7cf5d0 0x56044f7cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.933+0000 7f4946d9d640 1 -- 172.21.15.172:0/219114029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7cf5d0 msgr2=0x56044f7cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:22.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.934+0000 7f4946d9d640 1 --2- 172.21.15.172:0/219114029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7cf5d0 0x56044f7cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.934+0000 7f4946d9d640 1 -- 172.21.15.172:0/219114029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7da370 msgr2=0x56044f7dc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:22.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.934+0000 7f4946d9d640 1 --2- 172.21.15.172:0/219114029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7da370 0x56044f7dc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.934+0000 7f4946d9d640 1 -- 172.21.15.172:0/219114029 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56044f6c13a0 con 0x56044f7d0b40 2024-09-10T08:55:22.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.934+0000 7f4946d9d640 1 --2- 172.21.15.172:0/4255362861 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49300a2b80 0x7f49300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.934+0000 7f49377fe640 1 -- 172.21.15.172:0/4255362861 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f49380cf7d0 con 0x7f4948101df0 2024-09-10T08:55:22.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.934+0000 7f4945d9b640 1 --2- 172.21.15.172:0/219114029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7da370 0x56044f7dc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:22.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.934+0000 7f4946d9d640 1 --2- 172.21.15.172:0/219114029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7d0b40 0x56044f7cf090 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4940035000 tx=0x7f4940031ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5acbd1798b1d1d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:22.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.934+0000 7f4934ff9640 1 -- 172.21.15.172:0/219114029 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940003b30 con 0x56044f7d0b40 2024-09-10T08:55:22.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.934+0000 7f4934ff9640 1 -- 172.21.15.172:0/219114029 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4940036040 con 0x56044f7d0b40 2024-09-10T08:55:22.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.934+0000 7f4934ff9640 1 -- 172.21.15.172:0/219114029 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940030820 con 0x56044f7d0b40 2024-09-10T08:55:22.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.935+0000 7f494ee42740 1 -- 172.21.15.172:0/219114029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7d0b40 msgr2=0x56044f7cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:22.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.935+0000 7f4946d9d640 1 --2- 172.21.15.172:0/4255362861 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49300a2b80 0x7f49300a5040 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f494004c000 tx=0x7f4940049000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:22.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.935+0000 7f494ee42740 1 --2- 172.21.15.172:0/219114029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7d0b40 0x56044f7cf090 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4940035000 tx=0x7f4940031ac0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.935+0000 7f494ee42740 1 -- 172.21.15.172:0/219114029 shutdown_connections 2024-09-10T08:55:22.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.935+0000 7f494ee42740 1 --2- 172.21.15.172:0/219114029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7da370 0x56044f7dc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.935+0000 7f494ee42740 1 --2- 172.21.15.172:0/219114029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7cf5d0 0x56044f7cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.935+0000 7f494ee42740 1 --2- 172.21.15.172:0/219114029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7d0b40 0x56044f7cf090 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.935+0000 7f494ee42740 1 -- 172.21.15.172:0/219114029 >> 172.21.15.172:0/219114029 conn(0x56044f798310 msgr2=0x56044f7cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:22.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.935+0000 7f494ee42740 1 -- 172.21.15.172:0/219114029 shutdown_connections 2024-09-10T08:55:22.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.935+0000 7f494ee42740 1 -- 172.21.15.172:0/219114029 wait complete. 2024-09-10T08:55:22.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f494ee42740 1 -- start start 2024-09-10T08:55:22.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f494ee42740 1 Processor -- start 2024-09-10T08:55:22.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f494ee42740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7cf5d0 0x56044f7d9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f494ee42740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7d0b40 0x56044f7d6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f494ee42740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7da370 0x56044f7d75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f494ee42740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56044f7dcef0 con 0x56044f7d0b40 2024-09-10T08:55:22.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f494ee42740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56044f6f0060 con 0x56044f7cf5d0 2024-09-10T08:55:22.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f494ee42740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56044f7dd070 con 0x56044f7da370 2024-09-10T08:55:22.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f494ee42740 10 client.0 _ll_register_callbacks cb 0x56044f4438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:22.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f4945d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7d0b40 0x56044f7d6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f4945d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7d0b40 0x56044f7d6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42828/0 (socket says 172.21.15.172:42828) 2024-09-10T08:55:22.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f4945d9b640 1 -- 172.21.15.172:0/2080452546 learned_addr learned my addr 172.21.15.172:0/2080452546 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:22.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f4946d9d640 1 --2- 172.21.15.172:0/2080452546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7da370 0x56044f7d75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f494659c640 1 --2- 172.21.15.172:0/2080452546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7cf5d0 0x56044f7d9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f4945d9b640 1 -- 172.21.15.172:0/2080452546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7da370 msgr2=0x56044f7d75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:22.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f4945d9b640 1 --2- 172.21.15.172:0/2080452546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7da370 0x56044f7d75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f4945d9b640 1 -- 172.21.15.172:0/2080452546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7cf5d0 msgr2=0x56044f7d9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:22.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f4945d9b640 1 --2- 172.21.15.172:0/2080452546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7cf5d0 0x56044f7d9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.936+0000 7f4945d9b640 1 -- 172.21.15.172:0/2080452546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56044f4a63c0 con 0x56044f7d0b40 2024-09-10T08:55:22.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.937+0000 7f4945d9b640 1 --2- 172.21.15.172:0/2080452546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7d0b40 0x56044f7d6fe0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f493c00be40 tx=0x7f493c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebca09f60b83871b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:22.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.937+0000 7f492dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:22.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.937+0000 7f492dffb640 1 -- 172.21.15.172:0/2080452546 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c0035f0 con 0x56044f7d0b40 2024-09-10T08:55:22.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.937+0000 7f492dffb640 1 -- 172.21.15.172:0/2080452546 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f493c003790 con 0x56044f7d0b40 2024-09-10T08:55:22.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.937+0000 7f494ee42740 1 -- 172.21.15.172:0/2080452546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4940036040 con 0x56044f7d0b40 2024-09-10T08:55:22.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.937+0000 7f492dffb640 1 -- 172.21.15.172:0/2080452546 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f493c003990 con 0x56044f7d0b40 2024-09-10T08:55:22.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.937+0000 7f494ee42740 10 client.18168 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:22.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.937+0000 7f494ee42740 1 -- 172.21.15.172:0/2080452546 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56044f7e8a00 con 0x56044f7d0b40 2024-09-10T08:55:22.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.937+0000 7f492dffb640 1 -- 172.21.15.172:0/2080452546 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4940036040 con 0x56044f7d0b40 2024-09-10T08:55:22.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.937+0000 7f492dffb640 1 -- 172.21.15.172:0/2080452546 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f493c019070 con 0x56044f7d0b40 2024-09-10T08:55:22.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.938+0000 7f494ee42740 10 client.18168 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:22.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.938+0000 7f494ee42740 10 client.18168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:22.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.938+0000 7f494ee42740 20 client.18168 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:22.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.938+0000 7f494ee42740 1 --2- 172.21.15.172:0/2080452546 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56044f7e9f10 0x56044f7ec390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.938+0000 7f494ee42740 4 client.18168 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:55:22.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.938+0000 7f494ee42740 1 -- 172.21.15.172:0/2080452546 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56044f742860 con 0x56044f7e9f10 2024-09-10T08:55:22.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.938+0000 7f494659c640 1 --2- 172.21.15.172:0/2080452546 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56044f7e9f10 0x56044f7ec390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.938+0000 7f494659c640 1 --2- 172.21.15.172:0/2080452546 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56044f7e9f10 0x56044f7ec390 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdbde0eba2fea88 server_cookie=9305ddf580492bd1 in_seq=0 out_seq=0 2024-09-10T08:55:22.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.938+0000 7f492dffb640 10 client.18168 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:22.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.939+0000 7f492dffb640 1 -- 172.21.15.172:0/2080452546 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56044f742860 con 0x56044f7e9f10 2024-09-10T08:55:22.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.939+0000 7f492dffb640 10 client.18168 handle_command_reply: tid=0 2024-09-10T08:55:22.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.939+0000 7f494ee42740 1 client.18168 shutdown 2024-09-10T08:55:22.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.939+0000 7f494ee42740 1 -- 172.21.15.172:0/2080452546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7d0b40 msgr2=0x56044f7d6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:22.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.939+0000 7f494ee42740 1 --2- 172.21.15.172:0/2080452546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7d0b40 0x56044f7d6fe0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f493c00be40 tx=0x7f493c012820 comp rx=0 tx=0).stop 2024-09-10T08:55:22.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.939+0000 7f494ee42740 1 -- 172.21.15.172:0/2080452546 shutdown_connections 2024-09-10T08:55:22.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.939+0000 7f494ee42740 1 --2- 172.21.15.172:0/2080452546 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56044f7e9f10 0x56044f7ec390 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.939+0000 7f494ee42740 1 --2- 172.21.15.172:0/2080452546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7da370 0x56044f7d75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.939+0000 7f494ee42740 1 --2- 172.21.15.172:0/2080452546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7d0b40 0x56044f7d6fe0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.939+0000 7f494ee42740 1 --2- 172.21.15.172:0/2080452546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7cf5d0 0x56044f7d9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.939+0000 7f494ee42740 1 -- 172.21.15.172:0/2080452546 >> 172.21.15.172:0/2080452546 conn(0x56044f798310 msgr2=0x56044f7d1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:22.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.939+0000 7f492dffb640 0 client.18168 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:22.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.939+0000 7f494ee42740 1 -- 172.21.15.172:0/2080452546 shutdown_connections 2024-09-10T08:55:22.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.939+0000 7f494ee42740 1 -- 172.21.15.172:0/2080452546 wait complete. 2024-09-10T08:55:22.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.939+0000 7f494ee42740 20 client.18168 trim_cache size 0 max 16384 2024-09-10T08:55:22.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.956+0000 7f494ee42740 1 Processor -- start 2024-09-10T08:55:22.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.956+0000 7f494ee42740 1 -- start start 2024-09-10T08:55:22.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f494ee42740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7cf5d0 0x56044f7cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f494ee42740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7d0b40 0x56044f7e6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f494ee42740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7da370 0x56044f7d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f494ee42740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49481aeaa0 con 0x56044f7da370 2024-09-10T08:55:22.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f494ee42740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f49481b05e0 con 0x56044f7cf5d0 2024-09-10T08:55:22.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f494ee42740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f49481af170 con 0x56044f7d0b40 2024-09-10T08:55:22.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f4946d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7da370 0x56044f7d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f4946d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7da370 0x56044f7d2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42832/0 (socket says 172.21.15.172:42832) 2024-09-10T08:55:22.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f4946d9d640 1 -- 172.21.15.172:0/2287051242 learned_addr learned my addr 172.21.15.172:0/2287051242 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:22.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f494659c640 1 --2- 172.21.15.172:0/2287051242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7cf5d0 0x56044f7cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f4945d9b640 1 --2- 172.21.15.172:0/2287051242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7d0b40 0x56044f7e6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f494659c640 1 -- 172.21.15.172:0/2287051242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7d0b40 msgr2=0x56044f7e6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:22.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f494659c640 1 --2- 172.21.15.172:0/2287051242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7d0b40 0x56044f7e6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f494659c640 1 -- 172.21.15.172:0/2287051242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7da370 msgr2=0x56044f7d2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:22.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f494659c640 1 --2- 172.21.15.172:0/2287051242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7da370 0x56044f7d2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f494659c640 1 -- 172.21.15.172:0/2287051242 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56044f7d5530 con 0x56044f7cf5d0 2024-09-10T08:55:22.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f4946d9d640 1 --2- 172.21.15.172:0/2287051242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7da370 0x56044f7d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:22.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.957+0000 7f494659c640 1 --2- 172.21.15.172:0/2287051242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7cf5d0 0x56044f7cbe10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f49380ccfc0 tx=0x7f49380b2040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=543ae9157e4ebdb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:22.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.958+0000 7f492ffff640 1 -- 172.21.15.172:0/2287051242 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49380a0080 con 0x56044f7cf5d0 2024-09-10T08:55:22.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.958+0000 7f492ffff640 1 -- 172.21.15.172:0/2287051242 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49380a4940 con 0x56044f7cf5d0 2024-09-10T08:55:22.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.958+0000 7f492ffff640 1 -- 172.21.15.172:0/2287051242 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493809b380 con 0x56044f7cf5d0 2024-09-10T08:55:22.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.958+0000 7f494ee42740 1 -- 172.21.15.172:0/2287051242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7cf5d0 msgr2=0x56044f7cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:22.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.958+0000 7f494ee42740 1 --2- 172.21.15.172:0/2287051242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7cf5d0 0x56044f7cbe10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f49380ccfc0 tx=0x7f49380b2040 comp rx=0 tx=0).stop 2024-09-10T08:55:22.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.958+0000 7f494ee42740 1 -- 172.21.15.172:0/2287051242 shutdown_connections 2024-09-10T08:55:22.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.959+0000 7f494ee42740 1 --2- 172.21.15.172:0/2287051242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7da370 0x56044f7d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.959+0000 7f494ee42740 1 --2- 172.21.15.172:0/2287051242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7d0b40 0x56044f7e6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.959+0000 7f494ee42740 1 --2- 172.21.15.172:0/2287051242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7cf5d0 0x56044f7cbe10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.959+0000 7f494ee42740 1 -- 172.21.15.172:0/2287051242 >> 172.21.15.172:0/2287051242 conn(0x56044f798310 msgr2=0x56044f7cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:22.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.959+0000 7f494ee42740 1 -- 172.21.15.172:0/2287051242 shutdown_connections 2024-09-10T08:55:22.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.959+0000 7f494ee42740 1 -- 172.21.15.172:0/2287051242 wait complete. 2024-09-10T08:55:22.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.959+0000 7f494ee42740 1 -- start start 2024-09-10T08:55:22.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.959+0000 7f494ee42740 1 Processor -- start 2024-09-10T08:55:22.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.959+0000 7f494ee42740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7cf5d0 0x56044f7e2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.959+0000 7f494ee42740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7d0b40 0x56044f7dda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.959+0000 7f494ee42740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7da370 0x56044f7ddf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f4945d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7d0b40 0x56044f7dda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f494ee42740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49481aff10 con 0x56044f7da370 2024-09-10T08:55:22.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f494ee42740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f49481af840 con 0x56044f7d0b40 2024-09-10T08:55:22.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f494ee42740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f49481aca80 con 0x56044f7cf5d0 2024-09-10T08:55:22.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f4945d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7d0b40 0x56044f7dda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:33080/0 (socket says 172.21.15.172:33080) 2024-09-10T08:55:22.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f4945d9b640 1 -- 172.21.15.172:0/3353658982 learned_addr learned my addr 172.21.15.172:0/3353658982 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f4946d9d640 1 --2- 172.21.15.172:0/3353658982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7da370 0x56044f7ddf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f494ee42740 10 client.0 _ll_register_callbacks cb 0x7f493c00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f494659c640 1 --2- 172.21.15.172:0/3353658982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7cf5d0 0x56044f7e2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f4945d9b640 1 -- 172.21.15.172:0/3353658982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7cf5d0 msgr2=0x56044f7e2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f4945d9b640 1 --2- 172.21.15.172:0/3353658982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7cf5d0 0x56044f7e2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f4945d9b640 1 -- 172.21.15.172:0/3353658982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7da370 msgr2=0x56044f7ddf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f4945d9b640 1 --2- 172.21.15.172:0/3353658982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7da370 0x56044f7ddf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f4945d9b640 1 -- 172.21.15.172:0/3353658982 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56044f7de4d0 con 0x56044f7d0b40 2024-09-10T08:55:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f4946d9d640 1 --2- 172.21.15.172:0/3353658982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7da370 0x56044f7ddf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f494659c640 1 --2- 172.21.15.172:0/3353658982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7cf5d0 0x56044f7e2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:22.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f4945d9b640 1 --2- 172.21.15.172:0/3353658982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7d0b40 0x56044f7dda50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f493c010d90 tx=0x7f493c004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=649c00a4696dcad9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f492dffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:55:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.960+0000 7f492dffb640 1 -- 172.21.15.172:0/3353658982 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c005040 con 0x56044f7d0b40 2024-09-10T08:55:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.961+0000 7f494ee42740 1 -- 172.21.15.172:0/3353658982 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49380a4940 con 0x56044f7d0b40 2024-09-10T08:55:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.961+0000 7f492dffb640 1 -- 172.21.15.172:0/3353658982 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f493c0051e0 con 0x56044f7d0b40 2024-09-10T08:55:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.961+0000 7f492dffb640 1 -- 172.21.15.172:0/3353658982 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f493c03f050 con 0x56044f7d0b40 2024-09-10T08:55:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.961+0000 7f494ee42740 10 client.27173 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.961+0000 7f494ee42740 1 -- 172.21.15.172:0/3353658982 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56044f854950 con 0x56044f7d0b40 2024-09-10T08:55:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.961+0000 7f492dffb640 1 -- 172.21.15.172:0/3353658982 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49380a4940 con 0x56044f7d0b40 2024-09-10T08:55:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.961+0000 7f492dffb640 1 -- 172.21.15.172:0/3353658982 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f493c034020 con 0x56044f7d0b40 2024-09-10T08:55:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.961+0000 7f494ee42740 10 client.27173 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:22.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.961+0000 7f494ee42740 10 client.27173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.961+0000 7f494ee42740 20 client.27173 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.961+0000 7f494ee42740 1 --2- 172.21.15.172:0/3353658982 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56044f855100 0x56044f857520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.961+0000 7f494ee42740 4 client.27173 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:55:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.961+0000 7f494ee42740 1 -- 172.21.15.172:0/3353658982 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56044f73cd70 con 0x56044f855100 2024-09-10T08:55:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.961+0000 7f494659c640 1 --2- 172.21.15.172:0/3353658982 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56044f855100 0x56044f857520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.962+0000 7f494659c640 1 --2- 172.21.15.172:0/3353658982 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56044f855100 0x56044f857520 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29fcebb38fb9aac1 server_cookie=cd32d37252e81992 in_seq=0 out_seq=0 2024-09-10T08:55:22.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.962+0000 7f492dffb640 10 client.27173 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.962+0000 7f492dffb640 1 -- 172.21.15.172:0/3353658982 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56044f73cd70 con 0x56044f855100 2024-09-10T08:55:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.962+0000 7f492dffb640 10 client.27173 handle_command_reply: tid=0 2024-09-10T08:55:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.962+0000 7f494ee42740 1 client.27173 shutdown 2024-09-10T08:55:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.962+0000 7f494ee42740 1 -- 172.21.15.172:0/3353658982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7d0b40 msgr2=0x56044f7dda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.962+0000 7f494ee42740 1 --2- 172.21.15.172:0/3353658982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7d0b40 0x56044f7dda50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f493c010d90 tx=0x7f493c004530 comp rx=0 tx=0).stop 2024-09-10T08:55:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.962+0000 7f494ee42740 1 -- 172.21.15.172:0/3353658982 shutdown_connections 2024-09-10T08:55:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.962+0000 7f494ee42740 1 --2- 172.21.15.172:0/3353658982 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56044f855100 0x56044f857520 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.962+0000 7f494ee42740 1 --2- 172.21.15.172:0/3353658982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56044f7da370 0x56044f7ddf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.962+0000 7f494ee42740 1 --2- 172.21.15.172:0/3353658982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56044f7d0b40 0x56044f7dda50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.962+0000 7f494ee42740 1 --2- 172.21.15.172:0/3353658982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56044f7cf5d0 0x56044f7e2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.962+0000 7f494ee42740 1 -- 172.21.15.172:0/3353658982 >> 172.21.15.172:0/3353658982 conn(0x56044f798310 msgr2=0x56044f7d8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:22.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.962+0000 7f492dffb640 0 client.27173 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:22.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.962+0000 7f494ee42740 1 -- 172.21.15.172:0/3353658982 shutdown_connections 2024-09-10T08:55:22.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.962+0000 7f494ee42740 1 -- 172.21.15.172:0/3353658982 wait complete. 2024-09-10T08:55:22.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.962+0000 7f494ee42740 20 client.27173 trim_cache size 0 max 16384 2024-09-10T08:55:22.974 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:55:22.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.963+0000 7f494ca69640 1 -- 172.21.15.172:0/4255362861 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49300a2b80 msgr2=0x7f49300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:22.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.963+0000 7f494ca69640 1 --2- 172.21.15.172:0/4255362861 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49300a2b80 0x7f49300a5040 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f494004c000 tx=0x7f4940049000 comp rx=0 tx=0).stop 2024-09-10T08:55:22.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.963+0000 7f494ca69640 1 -- 172.21.15.172:0/4255362861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4948101df0 msgr2=0x7f4948198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:22.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.963+0000 7f494ca69640 1 --2- 172.21.15.172:0/4255362861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4948101df0 0x7f4948198300 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f493800e9f0 tx=0x7f493800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.963+0000 7f494ca69640 1 -- 172.21.15.172:0/4255362861 shutdown_connections 2024-09-10T08:55:22.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.963+0000 7f494ca69640 1 --2- 172.21.15.172:0/4255362861 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49300a2b80 0x7f49300a5040 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.963+0000 7f494ca69640 1 --2- 172.21.15.172:0/4255362861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4948106f20 0x7f494819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.963+0000 7f494ca69640 1 --2- 172.21.15.172:0/4255362861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49481027a0 0x7f4948198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.963+0000 7f494ca69640 1 --2- 172.21.15.172:0/4255362861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4948101df0 0x7f4948198300 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:22.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.963+0000 7f494ca69640 1 -- 172.21.15.172:0/4255362861 >> 172.21.15.172:0/4255362861 conn(0x7f49480fd8f0 msgr2=0x7f49480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:22.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.963+0000 7f494ca69640 1 -- 172.21.15.172:0/4255362861 shutdown_connections 2024-09-10T08:55:22.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:22.963+0000 7f494ca69640 1 -- 172.21.15.172:0/4255362861 wait complete. 2024-09-10T08:55:23.516 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.6 2024-09-10T08:55:23.516 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:55:23.982 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:55:24.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.513+0000 7f9fd8a25640 1 -- 172.21.15.172:0/3582309010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fd4109120 msgr2=0x7f9fd4110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:24.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.513+0000 7f9fd8a25640 1 --2- 172.21.15.172:0/3582309010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fd4109120 0x7f9fd4110ca0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc009ca0 tx=0x7f9fcc02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:55:24.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.513+0000 7f9fd8a25640 1 -- 172.21.15.172:0/3582309010 shutdown_connections 2024-09-10T08:55:24.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.513+0000 7f9fd8a25640 1 --2- 172.21.15.172:0/3582309010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fd4109120 0x7f9fd4110ca0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:24.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.513+0000 7f9fd8a25640 1 --2- 172.21.15.172:0/3582309010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fd41049a0 0x7f9fd41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:24.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.513+0000 7f9fd8a25640 1 --2- 172.21.15.172:0/3582309010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fd4103ff0 0x7f9fd41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:24.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.513+0000 7f9fd8a25640 1 -- 172.21.15.172:0/3582309010 >> 172.21.15.172:0/3582309010 conn(0x7f9fd40ffa70 msgr2=0x7f9fd4101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:24.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.513+0000 7f9fd8a25640 1 -- 172.21.15.172:0/3582309010 shutdown_connections 2024-09-10T08:55:24.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.513+0000 7f9fd8a25640 1 -- 172.21.15.172:0/3582309010 wait complete. 2024-09-10T08:55:24.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.514+0000 7f9fd8a25640 1 Processor -- start 2024-09-10T08:55:24.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.514+0000 7f9fd8a25640 1 -- start start 2024-09-10T08:55:24.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.514+0000 7f9fd8a25640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fd4103ff0 0x7f9fd419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:24.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.514+0000 7f9fd8a25640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fd41049a0 0x7f9fd419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:24.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.514+0000 7f9fd8a25640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fd4109120 0x7f9fd419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:24.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.514+0000 7f9fd8a25640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd4114030 con 0x7f9fd4103ff0 2024-09-10T08:55:24.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.514+0000 7f9fd8a25640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd4113eb0 con 0x7f9fd41049a0 2024-09-10T08:55:24.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.514+0000 7f9fd8a25640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd41141b0 con 0x7f9fd4109120 2024-09-10T08:55:24.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.514+0000 7f9fd259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fd4103ff0 0x7f9fd419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:24.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.514+0000 7f9fd259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fd4103ff0 0x7f9fd419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42850/0 (socket says 172.21.15.172:42850) 2024-09-10T08:55:24.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.514+0000 7f9fd259c640 1 -- 172.21.15.172:0/2611600340 learned_addr learned my addr 172.21.15.172:0/2611600340 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:24.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.514+0000 7f9fd2d9d640 1 --2- 172.21.15.172:0/2611600340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fd4109120 0x7f9fd419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:24.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.514+0000 7f9fd1d9b640 1 --2- 172.21.15.172:0/2611600340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fd41049a0 0x7f9fd419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:24.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.515+0000 7f9fd259c640 1 -- 172.21.15.172:0/2611600340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fd4109120 msgr2=0x7f9fd419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:24.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.515+0000 7f9fd259c640 1 --2- 172.21.15.172:0/2611600340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fd4109120 0x7f9fd419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:24.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.515+0000 7f9fd259c640 1 -- 172.21.15.172:0/2611600340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fd41049a0 msgr2=0x7f9fd419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:24.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.515+0000 7f9fd259c640 1 --2- 172.21.15.172:0/2611600340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fd41049a0 0x7f9fd419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:24.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.515+0000 7f9fd259c640 1 -- 172.21.15.172:0/2611600340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd419f4a0 con 0x7f9fd4103ff0 2024-09-10T08:55:24.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.515+0000 7f9fd259c640 1 --2- 172.21.15.172:0/2611600340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fd4103ff0 0x7f9fd419a4f0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400e9f0 tx=0x7f9fc400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:24.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.515+0000 7f9fc37fe640 1 -- 172.21.15.172:0/2611600340 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc400cd80 con 0x7f9fd4103ff0 2024-09-10T08:55:24.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.515+0000 7f9fc37fe640 1 -- 172.21.15.172:0/2611600340 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fc40108e0 con 0x7f9fd4103ff0 2024-09-10T08:55:24.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.515+0000 7f9fd8a25640 1 -- 172.21.15.172:0/2611600340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd419f790 con 0x7f9fd4103ff0 2024-09-10T08:55:24.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.515+0000 7f9fc37fe640 1 -- 172.21.15.172:0/2611600340 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4018920 con 0x7f9fd4103ff0 2024-09-10T08:55:24.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.515+0000 7f9fd8a25640 1 -- 172.21.15.172:0/2611600340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd41a7070 con 0x7f9fd4103ff0 2024-09-10T08:55:24.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.516+0000 7f9fd8a25640 1 -- 172.21.15.172:0/2611600340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd41043d0 con 0x7f9fd4103ff0 2024-09-10T08:55:24.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.518+0000 7f9fc37fe640 1 -- 172.21.15.172:0/2611600340 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9fc4010a90 con 0x7f9fd4103ff0 2024-09-10T08:55:24.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.518+0000 7f9fc37fe640 1 --2- 172.21.15.172:0/2611600340 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9fbc0a2c80 0x7f9fbc0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:24.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.518+0000 7f9fd1d9b640 1 --2- 172.21.15.172:0/2611600340 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9fbc0a2c80 0x7f9fbc0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:24.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.519+0000 7f9fc37fe640 1 -- 172.21.15.172:0/2611600340 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9fc40cf680 con 0x7f9fd4103ff0 2024-09-10T08:55:24.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.519+0000 7f9fd1d9b640 1 --2- 172.21.15.172:0/2611600340 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9fbc0a2c80 0x7f9fbc0a5140 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9fd410ab20 tx=0x7f9fc800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:24.524 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.523+0000 7f9fc37fe640 1 -- 172.21.15.172:0/2611600340 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9fc409bbd0 con 0x7f9fd4103ff0 2024-09-10T08:55:24.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:24 smithi172 ceph-mon[30854]: pgmap v625: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-10T08:55:24.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:24 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:55:24.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:24 smithi174 ceph-mon[34068]: pgmap v625: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-10T08:55:24.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:24 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:55:24.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.782+0000 7f9fd8a25640 1 -- 172.21.15.172:0/2611600340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fd410c3c0 con 0x7f9fd4103ff0 2024-09-10T08:55:24.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.783+0000 7f9fc37fe640 1 -- 172.21.15.172:0/2611600340 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+12001 (secure 0 0 0) 0x7f9fc409fed0 con 0x7f9fd4103ff0 2024-09-10T08:55:24.784 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:55:24.784 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":37,"btime":"2024-09-10T08:52:08:313493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:52:07.658341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:55:24.784 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 37 2024-09-10T08:55:24.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.785+0000 7f9fd8a25640 1 -- 172.21.15.172:0/2611600340 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9fbc0a2c80 msgr2=0x7f9fbc0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:24.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.785+0000 7f9fd8a25640 1 --2- 172.21.15.172:0/2611600340 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9fbc0a2c80 0x7f9fbc0a5140 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9fd410ab20 tx=0x7f9fc800b410 comp rx=0 tx=0).stop 2024-09-10T08:55:24.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.785+0000 7f9fd8a25640 1 -- 172.21.15.172:0/2611600340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fd4103ff0 msgr2=0x7f9fd419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:24.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.785+0000 7f9fd8a25640 1 --2- 172.21.15.172:0/2611600340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fd4103ff0 0x7f9fd419a4f0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400e9f0 tx=0x7f9fc400eeb0 comp rx=0 tx=0).stop 2024-09-10T08:55:24.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.785+0000 7f9fd8a25640 1 -- 172.21.15.172:0/2611600340 shutdown_connections 2024-09-10T08:55:24.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.785+0000 7f9fd8a25640 1 --2- 172.21.15.172:0/2611600340 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9fbc0a2c80 0x7f9fbc0a5140 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:24.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.785+0000 7f9fd8a25640 1 --2- 172.21.15.172:0/2611600340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fd4109120 0x7f9fd419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:24.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.785+0000 7f9fd8a25640 1 --2- 172.21.15.172:0/2611600340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fd41049a0 0x7f9fd419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:24.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.785+0000 7f9fd8a25640 1 --2- 172.21.15.172:0/2611600340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fd4103ff0 0x7f9fd419a4f0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:24.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.785+0000 7f9fd8a25640 1 -- 172.21.15.172:0/2611600340 >> 172.21.15.172:0/2611600340 conn(0x7f9fd40ffa70 msgr2=0x7f9fd4100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:24.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.785+0000 7f9fd8a25640 1 -- 172.21.15.172:0/2611600340 shutdown_connections 2024-09-10T08:55:24.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:24.785+0000 7f9fd8a25640 1 -- 172.21.15.172:0/2611600340 wait complete. 2024-09-10T08:55:24.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:24 smithi191 ceph-mon[35314]: pgmap v625: 129 pgs: 129 active+clean; 8.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 25 MiB/s wr, 20 op/s 2024-09-10T08:55:24.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:24 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:55:24.941 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:55:24.941 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:55:25.634 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:25 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:55:25.634 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:25 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:55:25.634 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:25 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:55:25.634 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:25 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2611600340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:55:25.635 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 144.6 --await-for 45.0' 2024-09-10T08:55:25.636 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 144.6 --await-for 45.0 2024-09-10T08:55:25.679 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:55:25.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:25 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:55:25.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:25 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:55:25.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:25 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:55:25.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:25 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2611600340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:55:25.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:25 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:55:25.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:25 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:55:25.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:25 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:55:25.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:25 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2611600340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:55:26.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.179+0000 7f1be38e1640 1 -- 172.21.15.172:0/1453305363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bdc102ef0 msgr2=0x7f1bdc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:26.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.179+0000 7f1be38e1640 1 --2- 172.21.15.172:0/1453305363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bdc102ef0 0x7f1bdc10da40 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009ca0 tx=0x7f1bd402a950 comp rx=0 tx=0).stop 2024-09-10T08:55:26.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.179+0000 7f1be38e1640 1 -- 172.21.15.172:0/1453305363 shutdown_connections 2024-09-10T08:55:26.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.179+0000 7f1be38e1640 1 --2- 172.21.15.172:0/1453305363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bdc102ef0 0x7f1bdc10da40 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.179+0000 7f1be38e1640 1 --2- 172.21.15.172:0/1453305363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bdc102570 0x7f1bdc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.179+0000 7f1be38e1640 1 --2- 172.21.15.172:0/1453305363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bdc100e20 0x7f1bdc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.179+0000 7f1be38e1640 1 -- 172.21.15.172:0/1453305363 >> 172.21.15.172:0/1453305363 conn(0x7f1bdc0fc7d0 msgr2=0x7f1bdc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:26.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.179+0000 7f1be38e1640 1 -- 172.21.15.172:0/1453305363 shutdown_connections 2024-09-10T08:55:26.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.179+0000 7f1be38e1640 1 -- 172.21.15.172:0/1453305363 wait complete. 2024-09-10T08:55:26.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.180+0000 7f1be38e1640 1 Processor -- start 2024-09-10T08:55:26.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.180+0000 7f1be38e1640 1 -- start start 2024-09-10T08:55:26.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.180+0000 7f1be38e1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bdc100e20 0x7f1bdc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.180+0000 7f1be38e1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bdc102570 0x7f1bdc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.180+0000 7f1be38e1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bdc102ef0 0x7f1bdc0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.180+0000 7f1be38e1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc110e30 con 0x7f1bdc102570 2024-09-10T08:55:26.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.180+0000 7f1be38e1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc110cb0 con 0x7f1bdc100e20 2024-09-10T08:55:26.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.180+0000 7f1be38e1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc110fb0 con 0x7f1bdc102ef0 2024-09-10T08:55:26.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.180+0000 7f1be0e7c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bdc102570 0x7f1bdc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.180+0000 7f1be0e7c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bdc102570 0x7f1bdc0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42912/0 (socket says 172.21.15.172:42912) 2024-09-10T08:55:26.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.180+0000 7f1be0e7c640 1 -- 172.21.15.172:0/3224084857 learned_addr learned my addr 172.21.15.172:0/3224084857 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:26.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.180+0000 7f1be1e7e640 1 --2- 172.21.15.172:0/3224084857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bdc102ef0 0x7f1bdc0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.180+0000 7f1be167d640 1 --2- 172.21.15.172:0/3224084857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bdc100e20 0x7f1bdc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.181+0000 7f1be0e7c640 1 -- 172.21.15.172:0/3224084857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bdc102ef0 msgr2=0x7f1bdc0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:26.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.181+0000 7f1be0e7c640 1 --2- 172.21.15.172:0/3224084857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bdc102ef0 0x7f1bdc0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.181+0000 7f1be0e7c640 1 -- 172.21.15.172:0/3224084857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bdc100e20 msgr2=0x7f1bdc0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:26.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.181+0000 7f1be0e7c640 1 --2- 172.21.15.172:0/3224084857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bdc100e20 0x7f1bdc0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.181+0000 7f1be0e7c640 1 -- 172.21.15.172:0/3224084857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bdc1a68c0 con 0x7f1bdc102570 2024-09-10T08:55:26.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.181+0000 7f1be167d640 1 --2- 172.21.15.172:0/3224084857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bdc100e20 0x7f1bdc0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:26.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.181+0000 7f1be0e7c640 1 --2- 172.21.15.172:0/3224084857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bdc102570 0x7f1bdc0fd1c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00d950 tx=0x7f1bcc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:26.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.181+0000 7f1bd27fc640 1 -- 172.21.15.172:0/3224084857 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc017030 con 0x7f1bdc102570 2024-09-10T08:55:26.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.181+0000 7f1be38e1640 1 -- 172.21.15.172:0/3224084857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bdc1a6bb0 con 0x7f1bdc102570 2024-09-10T08:55:26.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.181+0000 7f1bd27fc640 1 -- 172.21.15.172:0/3224084857 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bcc004020 con 0x7f1bdc102570 2024-09-10T08:55:26.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.181+0000 7f1bd27fc640 1 -- 172.21.15.172:0/3224084857 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc0047a0 con 0x7f1bdc102570 2024-09-10T08:55:26.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.181+0000 7f1be38e1640 1 -- 172.21.15.172:0/3224084857 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bdc1a7140 con 0x7f1bdc102570 2024-09-10T08:55:26.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.184+0000 7f1bd27fc640 1 -- 172.21.15.172:0/3224084857 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1bcc0041d0 con 0x7f1bdc102570 2024-09-10T08:55:26.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.185+0000 7f1be5cba740 1 Processor -- start 2024-09-10T08:55:26.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.185+0000 7f1be5cba740 1 -- start start 2024-09-10T08:55:26.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.185+0000 7f1be5cba740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e09db60 0x55cb0e09df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.185+0000 7f1be5cba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e09e480 0x55cb0e0a8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.185+0000 7f1be5cba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0ab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.185+0000 7f1be5cba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cb0e003280 con 0x55cb0e0a8d40 2024-09-10T08:55:26.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.185+0000 7f1be5cba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cb0dfaab10 con 0x55cb0e09e480 2024-09-10T08:55:26.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.185+0000 7f1be5cba740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55cb0dfff240 con 0x55cb0e09db60 2024-09-10T08:55:26.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.185+0000 7f1be0e7c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.185+0000 7f1bd27fc640 1 --2- 172.21.15.172:0/3224084857 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1bc80a2b80 0x7f1bc80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.185+0000 7f1be0e7c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0ab130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42914/0 (socket says 172.21.15.172:42914) 2024-09-10T08:55:26.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.185+0000 7f1be0e7c640 1 -- 172.21.15.172:0/2453014498 learned_addr learned my addr 172.21.15.172:0/2453014498 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:26.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.185+0000 7f1be1e7e640 1 --2- 172.21.15.172:0/2453014498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e09db60 0x55cb0e09df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.185+0000 7f1be167d640 1 --2- 172.21.15.172:0/2453014498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e09e480 0x55cb0e0a8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.186+0000 7f1be1e7e640 1 --2- 172.21.15.172:0/3224084857 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1bc80a2b80 0x7f1bc80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.186+0000 7f1be0e7c640 1 -- 172.21.15.172:0/2453014498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e09db60 msgr2=0x55cb0e09df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:26.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.186+0000 7f1be0e7c640 1 --2- 172.21.15.172:0/2453014498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e09db60 0x55cb0e09df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.186+0000 7f1be0e7c640 1 -- 172.21.15.172:0/2453014498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e09e480 msgr2=0x55cb0e0a8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:26.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.186+0000 7f1be0e7c640 1 --2- 172.21.15.172:0/2453014498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e09e480 0x55cb0e0a8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.186+0000 7f1be0e7c640 1 -- 172.21.15.172:0/2453014498 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb0dfb71f0 con 0x55cb0e0a8d40 2024-09-10T08:55:26.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.186+0000 7f1be167d640 1 --2- 172.21.15.172:0/2453014498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e09e480 0x55cb0e0a8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:26.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.186+0000 7f1bd27fc640 1 -- 172.21.15.172:0/3224084857 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1bcc0ced10 con 0x7f1bdc102570 2024-09-10T08:55:26.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.186+0000 7f1be0e7c640 1 --2- 172.21.15.172:0/2453014498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0ab130 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0cc3f0 tx=0x7f1bcc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b32f2510ebb349e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:26.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.186+0000 7f1be1e7e640 1 --2- 172.21.15.172:0/3224084857 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1bc80a2b80 0x7f1bc80a5040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4005fe0 tx=0x7f1bd402a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:26.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.186+0000 7f1bc77fe640 1 -- 172.21.15.172:0/2453014498 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc09b910 con 0x55cb0e0a8d40 2024-09-10T08:55:26.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.186+0000 7f1bc77fe640 1 -- 172.21.15.172:0/2453014498 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bcc0b9e50 con 0x55cb0e0a8d40 2024-09-10T08:55:26.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.186+0000 7f1bc77fe640 1 -- 172.21.15.172:0/2453014498 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc09f5c0 con 0x55cb0e0a8d40 2024-09-10T08:55:26.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.187+0000 7f1be5cba740 1 -- 172.21.15.172:0/2453014498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e0a8d40 msgr2=0x55cb0e0ab130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:26.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.187+0000 7f1be5cba740 1 --2- 172.21.15.172:0/2453014498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0ab130 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0cc3f0 tx=0x7f1bcc01c000 comp rx=0 tx=0).stop 2024-09-10T08:55:26.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.187+0000 7f1be5cba740 1 -- 172.21.15.172:0/2453014498 shutdown_connections 2024-09-10T08:55:26.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.187+0000 7f1be5cba740 1 --2- 172.21.15.172:0/2453014498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0ab130 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.187+0000 7f1be5cba740 1 --2- 172.21.15.172:0/2453014498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e09e480 0x55cb0e0a8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.187+0000 7f1be5cba740 1 --2- 172.21.15.172:0/2453014498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e09db60 0x55cb0e09df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.187+0000 7f1be5cba740 1 -- 172.21.15.172:0/2453014498 >> 172.21.15.172:0/2453014498 conn(0x55cb0e09a0f0 msgr2=0x55cb0e09a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:26.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.187+0000 7f1be5cba740 1 -- 172.21.15.172:0/2453014498 shutdown_connections 2024-09-10T08:55:26.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.187+0000 7f1be5cba740 1 -- 172.21.15.172:0/2453014498 wait complete. 2024-09-10T08:55:26.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.188+0000 7f1be5cba740 1 -- start start 2024-09-10T08:55:26.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.188+0000 7f1be5cba740 1 Processor -- start 2024-09-10T08:55:26.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.188+0000 7f1be5cba740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e09db60 0x55cb0e0a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.188+0000 7f1be5cba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e09e480 0x55cb0e0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.188+0000 7f1be5cba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0a6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.188+0000 7f1be5cba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cb0e0ab910 con 0x55cb0e0a8d40 2024-09-10T08:55:26.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.188+0000 7f1be5cba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cb0dfbdf10 con 0x55cb0e09e480 2024-09-10T08:55:26.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.188+0000 7f1be5cba740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55cb0e0aba90 con 0x55cb0e09db60 2024-09-10T08:55:26.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.189+0000 7f1be5cba740 10 client.0 _ll_register_callbacks cb 0x55cb0df86a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:26.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.189+0000 7f1be1e7e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.189+0000 7f1be1e7e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0a6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42928/0 (socket says 172.21.15.172:42928) 2024-09-10T08:55:26.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.189+0000 7f1be1e7e640 1 -- 172.21.15.172:0/4227011727 learned_addr learned my addr 172.21.15.172:0/4227011727 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:26.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.189+0000 7f1be167d640 1 --2- 172.21.15.172:0/4227011727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e09db60 0x55cb0e0a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.189+0000 7f1be0e7c640 1 --2- 172.21.15.172:0/4227011727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e09e480 0x55cb0e0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.189+0000 7f1be1e7e640 1 -- 172.21.15.172:0/4227011727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e09db60 msgr2=0x55cb0e0a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:26.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.189+0000 7f1be1e7e640 1 --2- 172.21.15.172:0/4227011727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e09db60 0x55cb0e0a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.189+0000 7f1be1e7e640 1 -- 172.21.15.172:0/4227011727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e09e480 msgr2=0x55cb0e0a61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:26.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.189+0000 7f1be1e7e640 1 --2- 172.21.15.172:0/4227011727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e09e480 0x55cb0e0a61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.189+0000 7f1be1e7e640 1 -- 172.21.15.172:0/4227011727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb0dd746a0 con 0x55cb0e0a8d40 2024-09-10T08:55:26.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.189+0000 7f1be0e7c640 1 --2- 172.21.15.172:0/4227011727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e09e480 0x55cb0e0a61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:26.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.189+0000 7f1be1e7e640 1 --2- 172.21.15.172:0/4227011727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0a6700 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1bd402f5b0 tx=0x7f1bd40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba548b631fddd45 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:26.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.189+0000 7f1bc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:26.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.190+0000 7f1bc4ff9640 1 -- 172.21.15.172:0/4227011727 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4043040 con 0x55cb0e0a8d40 2024-09-10T08:55:26.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.190+0000 7f1be5cba740 1 -- 172.21.15.172:0/4227011727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bcc0b9e50 con 0x55cb0e0a8d40 2024-09-10T08:55:26.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.190+0000 7f1bc4ff9640 1 -- 172.21.15.172:0/4227011727 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1bd4004240 con 0x55cb0e0a8d40 2024-09-10T08:55:26.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.190+0000 7f1bc4ff9640 1 -- 172.21.15.172:0/4227011727 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1bd4034040 con 0x55cb0e0a8d40 2024-09-10T08:55:26.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.190+0000 7f1be5cba740 10 client.18192 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:26.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.190+0000 7f1be5cba740 1 -- 172.21.15.172:0/4227011727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cb0e0b75c0 con 0x55cb0e0a8d40 2024-09-10T08:55:26.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.190+0000 7f1bc4ff9640 1 -- 172.21.15.172:0/4227011727 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bcc0b9e50 con 0x55cb0e0a8d40 2024-09-10T08:55:26.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.190+0000 7f1bc4ff9640 1 -- 172.21.15.172:0/4227011727 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f1bd4030a50 con 0x55cb0e0a8d40 2024-09-10T08:55:26.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.190+0000 7f1be5cba740 10 client.18192 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:26.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.190+0000 7f1be5cba740 10 client.18192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:26.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.191+0000 7f1be5cba740 20 client.18192 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:26.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.191+0000 7f1be5cba740 1 --2- 172.21.15.172:0/4227011727 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cb0e0b8db0 0x55cb0e0bb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.191+0000 7f1be5cba740 4 client.18192 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:55:26.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.191+0000 7f1be5cba740 1 -- 172.21.15.172:0/4227011727 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb0e0107f0 con 0x55cb0e0b8db0 2024-09-10T08:55:26.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.191+0000 7f1be167d640 1 --2- 172.21.15.172:0/4227011727 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cb0e0b8db0 0x55cb0e0bb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.191+0000 7f1be167d640 1 --2- 172.21.15.172:0/4227011727 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cb0e0b8db0 0x55cb0e0bb210 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecf32b7ded8fd19c server_cookie=2ba29ca7532baefa in_seq=0 out_seq=0 2024-09-10T08:55:26.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.191+0000 7f1bc4ff9640 10 client.18192 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:26.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.192+0000 7f1bc4ff9640 1 -- 172.21.15.172:0/4227011727 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cb0e0107f0 con 0x55cb0e0b8db0 2024-09-10T08:55:26.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.192+0000 7f1bc4ff9640 10 client.18192 handle_command_reply: tid=0 2024-09-10T08:55:26.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.192+0000 7f1be5cba740 1 client.18192 shutdown 2024-09-10T08:55:26.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.192+0000 7f1be5cba740 1 -- 172.21.15.172:0/4227011727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e0a8d40 msgr2=0x55cb0e0a6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:26.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.192+0000 7f1be5cba740 1 --2- 172.21.15.172:0/4227011727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0a6700 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1bd402f5b0 tx=0x7f1bd40095c0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.192+0000 7f1be5cba740 1 -- 172.21.15.172:0/4227011727 shutdown_connections 2024-09-10T08:55:26.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.192+0000 7f1be5cba740 1 --2- 172.21.15.172:0/4227011727 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cb0e0b8db0 0x55cb0e0bb210 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.192+0000 7f1be5cba740 1 --2- 172.21.15.172:0/4227011727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0a6700 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.193+0000 7f1be5cba740 1 --2- 172.21.15.172:0/4227011727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e09e480 0x55cb0e0a61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.193+0000 7f1bc4ff9640 0 client.18192 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:26.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.193+0000 7f1be5cba740 1 --2- 172.21.15.172:0/4227011727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e09db60 0x55cb0e0a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.193+0000 7f1be5cba740 1 -- 172.21.15.172:0/4227011727 >> 172.21.15.172:0/4227011727 conn(0x55cb0e09a0f0 msgr2=0x55cb0e09fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:26.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.193+0000 7f1be5cba740 1 -- 172.21.15.172:0/4227011727 shutdown_connections 2024-09-10T08:55:26.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.193+0000 7f1be5cba740 1 -- 172.21.15.172:0/4227011727 wait complete. 2024-09-10T08:55:26.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.193+0000 7f1be5cba740 20 client.18192 trim_cache size 0 max 16384 2024-09-10T08:55:26.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.217+0000 7f1be5cba740 1 Processor -- start 2024-09-10T08:55:26.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.217+0000 7f1be5cba740 1 -- start start 2024-09-10T08:55:26.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.217+0000 7f1be5cba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e09db60 0x55cb0e0a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.218+0000 7f1be5cba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e09e480 0x55cb0e0c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.218+0000 7f1be5cba740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0c1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.218+0000 7f1be5cba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc1b0db0 con 0x55cb0e09db60 2024-09-10T08:55:26.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.218+0000 7f1be5cba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc1b28f0 con 0x55cb0e09e480 2024-09-10T08:55:26.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.218+0000 7f1be5cba740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc1b1480 con 0x55cb0e0a8d40 2024-09-10T08:55:26.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.218+0000 7f1be167d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e09db60 0x55cb0e0a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.218+0000 7f1be167d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e09db60 0x55cb0e0a7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42936/0 (socket says 172.21.15.172:42936) 2024-09-10T08:55:26.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.218+0000 7f1be167d640 1 -- 172.21.15.172:0/1792622654 learned_addr learned my addr 172.21.15.172:0/1792622654 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:26.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.218+0000 7f1be167d640 1 -- 172.21.15.172:0/1792622654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e0a8d40 msgr2=0x55cb0e0c1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:26.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.218+0000 7f1be167d640 1 --2- 172.21.15.172:0/1792622654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0c1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.218+0000 7f1be167d640 1 -- 172.21.15.172:0/1792622654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e09e480 msgr2=0x55cb0e0c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:26.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.218+0000 7f1be167d640 1 --2- 172.21.15.172:0/1792622654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e09e480 0x55cb0e0c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.218+0000 7f1be167d640 1 -- 172.21.15.172:0/1792622654 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb0e0a4c30 con 0x55cb0e09db60 2024-09-10T08:55:26.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.218+0000 7f1be167d640 1 --2- 172.21.15.172:0/1792622654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e09db60 0x55cb0e0a7ae0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80123e0 tx=0x7f1bd8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfedba0e2124147 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:26.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.219+0000 7f1bc6ffd640 1 -- 172.21.15.172:0/1792622654 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8004220 con 0x55cb0e09db60 2024-09-10T08:55:26.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.219+0000 7f1bc6ffd640 1 -- 172.21.15.172:0/1792622654 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bd80043c0 con 0x55cb0e09db60 2024-09-10T08:55:26.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.219+0000 7f1bc6ffd640 1 -- 172.21.15.172:0/1792622654 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80035f0 con 0x55cb0e09db60 2024-09-10T08:55:26.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.220+0000 7f1be5cba740 1 -- 172.21.15.172:0/1792622654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e09db60 msgr2=0x55cb0e0a7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:26.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.220+0000 7f1be5cba740 1 --2- 172.21.15.172:0/1792622654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e09db60 0x55cb0e0a7ae0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80123e0 tx=0x7f1bd8005f90 comp rx=0 tx=0).stop 2024-09-10T08:55:26.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.220+0000 7f1be5cba740 1 -- 172.21.15.172:0/1792622654 shutdown_connections 2024-09-10T08:55:26.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.220+0000 7f1be5cba740 1 --2- 172.21.15.172:0/1792622654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0c1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.220+0000 7f1be5cba740 1 --2- 172.21.15.172:0/1792622654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e09e480 0x55cb0e0c0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.220+0000 7f1be5cba740 1 --2- 172.21.15.172:0/1792622654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e09db60 0x55cb0e0a7ae0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.220+0000 7f1be5cba740 1 -- 172.21.15.172:0/1792622654 >> 172.21.15.172:0/1792622654 conn(0x55cb0e09a0f0 msgr2=0x55cb0e0a73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:26.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.220+0000 7f1be5cba740 1 -- 172.21.15.172:0/1792622654 shutdown_connections 2024-09-10T08:55:26.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.220+0000 7f1be5cba740 1 -- 172.21.15.172:0/1792622654 wait complete. 2024-09-10T08:55:26.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.221+0000 7f1be5cba740 1 -- start start 2024-09-10T08:55:26.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.221+0000 7f1be5cba740 1 Processor -- start 2024-09-10T08:55:26.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.221+0000 7f1be5cba740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e09db60 0x55cb0e0acd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.221+0000 7f1be5cba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e09e480 0x55cb0e0ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.221+0000 7f1be5cba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.221+0000 7f1be0e7c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e09e480 0x55cb0e0ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.221+0000 7f1be0e7c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e09e480 0x55cb0e0ad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42942/0 (socket says 172.21.15.172:42942) 2024-09-10T08:55:26.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.221+0000 7f1be0e7c640 1 -- 172.21.15.172:0/1536966367 learned_addr learned my addr 172.21.15.172:0/1536966367 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:26.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.222+0000 7f1be167d640 1 --2- 172.21.15.172:0/1536966367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e09db60 0x55cb0e0acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.222+0000 7f1be5cba740 1 -- 172.21.15.172:0/1536966367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc1b2220 con 0x55cb0e09e480 2024-09-10T08:55:26.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.222+0000 7f1be1e7e640 1 --2- 172.21.15.172:0/1536966367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.222+0000 7f1be5cba740 1 -- 172.21.15.172:0/1536966367 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc1b1b50 con 0x55cb0e0a8d40 2024-09-10T08:55:26.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.222+0000 7f1be5cba740 1 -- 172.21.15.172:0/1536966367 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc1aed90 con 0x55cb0e09db60 2024-09-10T08:55:26.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.222+0000 7f1be5cba740 10 client.0 _ll_register_callbacks cb 0x7f1bd4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:26.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.222+0000 7f1be0e7c640 1 -- 172.21.15.172:0/1536966367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e09db60 msgr2=0x55cb0e0acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:26.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.222+0000 7f1be0e7c640 1 --2- 172.21.15.172:0/1536966367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e09db60 0x55cb0e0acd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.222+0000 7f1be0e7c640 1 -- 172.21.15.172:0/1536966367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e0a8d40 msgr2=0x55cb0e0ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:26.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.222+0000 7f1be0e7c640 1 --2- 172.21.15.172:0/1536966367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.222+0000 7f1be0e7c640 1 -- 172.21.15.172:0/1536966367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb0e122d90 con 0x55cb0e09e480 2024-09-10T08:55:26.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.222+0000 7f1be167d640 1 --2- 172.21.15.172:0/1536966367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e09db60 0x55cb0e0acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:26.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.222+0000 7f1be0e7c640 1 --2- 172.21.15.172:0/1536966367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e09e480 0x55cb0e0ad260 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0cc500 tx=0x7f1bcc09ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e72577e8fd40b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:26.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.222+0000 7f1bc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:26.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.223+0000 7f1bc4ff9640 1 -- 172.21.15.172:0/1536966367 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc0b4210 con 0x55cb0e09e480 2024-09-10T08:55:26.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.223+0000 7f1bc4ff9640 1 -- 172.21.15.172:0/1536966367 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1bcc0af730 con 0x55cb0e09e480 2024-09-10T08:55:26.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.223+0000 7f1be5cba740 1 -- 172.21.15.172:0/1536966367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bd80043c0 con 0x55cb0e09e480 2024-09-10T08:55:26.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.223+0000 7f1bc4ff9640 1 -- 172.21.15.172:0/1536966367 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1bcc0b9e50 con 0x55cb0e09e480 2024-09-10T08:55:26.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.223+0000 7f1be5cba740 10 client.18198 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:26.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.223+0000 7f1be5cba740 1 -- 172.21.15.172:0/1536966367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cb0e1240b0 con 0x55cb0e09e480 2024-09-10T08:55:26.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.223+0000 7f1bc4ff9640 1 -- 172.21.15.172:0/1536966367 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bd80043c0 con 0x55cb0e09e480 2024-09-10T08:55:26.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.223+0000 7f1bc4ff9640 1 -- 172.21.15.172:0/1536966367 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f1bcc0bb0d0 con 0x55cb0e09e480 2024-09-10T08:55:26.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.224+0000 7f1be5cba740 10 client.18198 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:26.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.224+0000 7f1be5cba740 10 client.18198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:26.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.224+0000 7f1be5cba740 20 client.18198 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:26.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.224+0000 7f1be5cba740 1 --2- 172.21.15.172:0/1536966367 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cb0e0b8db0 0x55cb0e1262f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.224+0000 7f1be5cba740 4 client.18198 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:55:26.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.224+0000 7f1be5cba740 1 -- 172.21.15.172:0/1536966367 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cb0e00ad00 con 0x55cb0e0b8db0 2024-09-10T08:55:26.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.224+0000 7f1be167d640 1 --2- 172.21.15.172:0/1536966367 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cb0e0b8db0 0x55cb0e1262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.224+0000 7f1bc4ff9640 10 client.18198 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:26.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.224+0000 7f1be167d640 1 --2- 172.21.15.172:0/1536966367 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cb0e0b8db0 0x55cb0e1262f0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60b5a258e695e286 server_cookie=cb8413b4ea46b291 in_seq=0 out_seq=0 2024-09-10T08:55:26.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.225+0000 7f1bc4ff9640 1 -- 172.21.15.172:0/1536966367 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cb0e00ad00 con 0x55cb0e0b8db0 2024-09-10T08:55:26.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.225+0000 7f1bc4ff9640 10 client.18198 handle_command_reply: tid=0 2024-09-10T08:55:26.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.226+0000 7f1be5cba740 1 client.18198 shutdown 2024-09-10T08:55:26.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.226+0000 7f1be5cba740 1 -- 172.21.15.172:0/1536966367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e09e480 msgr2=0x55cb0e0ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:26.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.226+0000 7f1be5cba740 1 --2- 172.21.15.172:0/1536966367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e09e480 0x55cb0e0ad260 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0cc500 tx=0x7f1bcc09ad50 comp rx=0 tx=0).stop 2024-09-10T08:55:26.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.226+0000 7f1be5cba740 1 -- 172.21.15.172:0/1536966367 shutdown_connections 2024-09-10T08:55:26.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.226+0000 7f1be5cba740 1 --2- 172.21.15.172:0/1536966367 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cb0e0b8db0 0x55cb0e1262f0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.226+0000 7f1be5cba740 1 --2- 172.21.15.172:0/1536966367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb0e0a8d40 0x55cb0e0ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.226+0000 7f1be5cba740 1 --2- 172.21.15.172:0/1536966367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb0e09e480 0x55cb0e0ad260 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.226+0000 7f1be5cba740 1 --2- 172.21.15.172:0/1536966367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cb0e09db60 0x55cb0e0acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.226+0000 7f1bc4ff9640 0 client.18198 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:26.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.226+0000 7f1be5cba740 1 -- 172.21.15.172:0/1536966367 >> 172.21.15.172:0/1536966367 conn(0x55cb0e09a0f0 msgr2=0x55cb0e099270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:26.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.226+0000 7f1be5cba740 1 -- 172.21.15.172:0/1536966367 shutdown_connections 2024-09-10T08:55:26.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.226+0000 7f1be5cba740 1 -- 172.21.15.172:0/1536966367 wait complete. 2024-09-10T08:55:26.230 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.226+0000 7f1be5cba740 20 client.18198 trim_cache size 0 max 16384 2024-09-10T08:55:26.230 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:55:26.231 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:55:26.231 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T08:55:26.231 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "85ac4318-0229-42f2-9e1a-461b926c0735", 2024-09-10T08:55:26.231 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T08:55:26.231 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:55:26.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.228+0000 7f1be38e1640 1 -- 172.21.15.172:0/3224084857 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1bc80a2b80 msgr2=0x7f1bc80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:26.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.228+0000 7f1be38e1640 1 --2- 172.21.15.172:0/3224084857 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1bc80a2b80 0x7f1bc80a5040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4005fe0 tx=0x7f1bd402a950 comp rx=0 tx=0).stop 2024-09-10T08:55:26.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.228+0000 7f1be38e1640 1 -- 172.21.15.172:0/3224084857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bdc102570 msgr2=0x7f1bdc0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:26.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.228+0000 7f1be38e1640 1 --2- 172.21.15.172:0/3224084857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bdc102570 0x7f1bdc0fd1c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00d950 tx=0x7f1bcc00de10 comp rx=0 tx=0).stop 2024-09-10T08:55:26.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.228+0000 7f1be38e1640 1 -- 172.21.15.172:0/3224084857 shutdown_connections 2024-09-10T08:55:26.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.228+0000 7f1be38e1640 1 --2- 172.21.15.172:0/3224084857 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1bc80a2b80 0x7f1bc80a5040 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.232 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.228+0000 7f1be38e1640 1 --2- 172.21.15.172:0/3224084857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1bdc102ef0 0x7f1bdc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.228+0000 7f1be38e1640 1 --2- 172.21.15.172:0/3224084857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1bdc102570 0x7f1bdc0fd1c0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.228+0000 7f1be38e1640 1 --2- 172.21.15.172:0/3224084857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bdc100e20 0x7f1bdc0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.228+0000 7f1be38e1640 1 -- 172.21.15.172:0/3224084857 >> 172.21.15.172:0/3224084857 conn(0x7f1bdc0fc7d0 msgr2=0x7f1bdc06a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:26.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.228+0000 7f1be38e1640 1 -- 172.21.15.172:0/3224084857 shutdown_connections 2024-09-10T08:55:26.233 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.229+0000 7f1be38e1640 1 -- 172.21.15.172:0/3224084857 wait complete. 2024-09-10T08:55:26.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:26 smithi172 ceph-mon[30854]: pgmap v626: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-09-10T08:55:26.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:26 smithi174 ceph-mon[34068]: pgmap v626: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-09-10T08:55:26.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:26 smithi191 ceph-mon[35314]: pgmap v626: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 32 MiB/s wr, 27 op/s 2024-09-10T08:55:26.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.967+0000 7f4f6b159640 1 -- 172.21.15.172:0/3055696132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f641082a0 msgr2=0x7f4f6410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:26.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.967+0000 7f4f6b159640 1 --2- 172.21.15.172:0/3055696132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f641082a0 0x7f4f6410ec40 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009cd0 tx=0x7f4f5c02a8e0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.967+0000 7f4f6b159640 1 -- 172.21.15.172:0/3055696132 shutdown_connections 2024-09-10T08:55:26.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.967+0000 7f4f6b159640 1 --2- 172.21.15.172:0/3055696132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f641082a0 0x7f4f6410ec40 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.967+0000 7f4f6b159640 1 --2- 172.21.15.172:0/3055696132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f641036e0 0x7f4f64107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.967+0000 7f4f6b159640 1 --2- 172.21.15.172:0/3055696132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f64102d30 0x7f4f64103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.967+0000 7f4f6b159640 1 -- 172.21.15.172:0/3055696132 >> 172.21.15.172:0/3055696132 conn(0x7f4f640fc7d0 msgr2=0x7f4f640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:26.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.967+0000 7f4f6b159640 1 -- 172.21.15.172:0/3055696132 shutdown_connections 2024-09-10T08:55:26.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.967+0000 7f4f6b159640 1 -- 172.21.15.172:0/3055696132 wait complete. 2024-09-10T08:55:26.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.967+0000 7f4f6b159640 1 Processor -- start 2024-09-10T08:55:26.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.968+0000 7f4f6b159640 1 -- start start 2024-09-10T08:55:26.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.968+0000 7f4f6b159640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f64102d30 0x7f4f641009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.968+0000 7f4f6b159640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f641036e0 0x7f4f64100f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.968+0000 7f4f6b159640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f641082a0 0x7f4f64101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.968+0000 7f4f6b159640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4f64111f50 con 0x7f4f641036e0 2024-09-10T08:55:26.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.968+0000 7f4f6b159640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f64111dd0 con 0x7f4f64102d30 2024-09-10T08:55:26.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.968+0000 7f4f6b159640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641120d0 con 0x7f4f641082a0 2024-09-10T08:55:26.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.968+0000 7f4f68ef5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f64102d30 0x7f4f641009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.968+0000 7f4f63fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f641036e0 0x7f4f64100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.968+0000 7f4f68ef5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f64102d30 0x7f4f641009d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:45856/0 (socket says 172.21.15.172:45856) 2024-09-10T08:55:26.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.968+0000 7f4f68ef5640 1 -- 172.21.15.172:0/2802754367 learned_addr learned my addr 172.21.15.172:0/2802754367 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:26.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.968+0000 7f4f696f6640 1 --2- 172.21.15.172:0/2802754367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f641082a0 0x7f4f64101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.968+0000 7f4f68ef5640 1 -- 172.21.15.172:0/2802754367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f641082a0 msgr2=0x7f4f64101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:26.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.968+0000 7f4f68ef5640 1 --2- 172.21.15.172:0/2802754367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f641082a0 0x7f4f64101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.968+0000 7f4f68ef5640 1 -- 172.21.15.172:0/2802754367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f641036e0 msgr2=0x7f4f64100f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:26.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.968+0000 7f4f68ef5640 1 --2- 172.21.15.172:0/2802754367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f641036e0 0x7f4f64100f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.968+0000 7f4f68ef5640 1 -- 172.21.15.172:0/2802754367 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f641aade0 con 0x7f4f64102d30 2024-09-10T08:55:26.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.969+0000 7f4f68ef5640 1 --2- 172.21.15.172:0/2802754367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f64102d30 0x7f4f641009d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800e9f0 tx=0x7f4f5800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:26.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.969+0000 7f4f61ffb640 1 -- 172.21.15.172:0/2802754367 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5800cd80 con 0x7f4f64102d30 2024-09-10T08:55:26.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.969+0000 7f4f61ffb640 1 -- 172.21.15.172:0/2802754367 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f580108e0 con 0x7f4f64102d30 2024-09-10T08:55:26.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.969+0000 7f4f6b159640 1 -- 172.21.15.172:0/2802754367 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f641ab070 con 0x7f4f64102d30 2024-09-10T08:55:26.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.969+0000 7f4f61ffb640 1 -- 172.21.15.172:0/2802754367 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f58018920 con 0x7f4f64102d30 2024-09-10T08:55:26.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.969+0000 7f4f6b159640 1 -- 172.21.15.172:0/2802754367 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f641ab4c0 con 0x7f4f64102d30 2024-09-10T08:55:26.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.972+0000 7f4f61ffb640 1 -- 172.21.15.172:0/2802754367 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4f58010430 con 0x7f4f64102d30 2024-09-10T08:55:26.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.972+0000 7f4f6d532740 1 Processor -- start 2024-09-10T08:55:26.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.972+0000 7f4f6d532740 1 -- start start 2024-09-10T08:55:26.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f6d532740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4d760 0x563b24a4db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f6d532740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4e080 0x563b24a58400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f61ffb640 1 --2- 172.21.15.172:0/2802754367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4f540a2b80 0x7f4f540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f6d532740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a58940 0x563b24a5ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f6d532740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563b249b32a0 con 0x563b24a4e080 2024-09-10T08:55:26.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f6d532740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b2495ab10 con 0x563b24a58940 2024-09-10T08:55:26.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f6d532740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563b249af260 con 0x563b24a4d760 2024-09-10T08:55:26.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f696f6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a58940 0x563b24a5ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f696f6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a58940 0x563b24a5ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:45872/0 (socket says 172.21.15.172:45872) 2024-09-10T08:55:26.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f63fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4e080 0x563b24a58400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f696f6640 1 -- 172.21.15.172:0/2799042029 learned_addr learned my addr 172.21.15.172:0/2799042029 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:26.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f68ef5640 1 --2- 172.21.15.172:0/2802754367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4f540a2b80 0x7f4f540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f61ffb640 1 -- 172.21.15.172:0/2802754367 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4f580cf770 con 0x7f4f64102d30 2024-09-10T08:55:26.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f696f6640 1 --2- 172.21.15.172:0/2799042029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4d760 0x563b24a4db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f63fff640 1 -- 172.21.15.172:0/2799042029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4d760 msgr2=0x563b24a4db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:26.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f63fff640 1 --2- 172.21.15.172:0/2799042029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4d760 0x563b24a4db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f63fff640 1 -- 172.21.15.172:0/2799042029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a58940 msgr2=0x563b24a5ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:26.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f63fff640 1 --2- 172.21.15.172:0/2799042029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a58940 0x563b24a5ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f63fff640 1 -- 172.21.15.172:0/2799042029 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b24724b90 con 0x563b24a4e080 2024-09-10T08:55:26.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.973+0000 7f4f696f6640 1 --2- 172.21.15.172:0/2799042029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4d760 0x563b24a4db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:26.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.974+0000 7f4f63fff640 1 --2- 172.21.15.172:0/2799042029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4e080 0x563b24a58400 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f4f64102220 tx=0x7f4f500103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c331f66efebf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:26.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.974+0000 7f4f4effd640 1 -- 172.21.15.172:0/2799042029 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50010e90 con 0x563b24a4e080 2024-09-10T08:55:26.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.974+0000 7f4f4effd640 1 -- 172.21.15.172:0/2799042029 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f500035f0 con 0x563b24a4e080 2024-09-10T08:55:26.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.974+0000 7f4f4effd640 1 -- 172.21.15.172:0/2799042029 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f500044e0 con 0x563b24a4e080 2024-09-10T08:55:26.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.975+0000 7f4f6d532740 1 -- 172.21.15.172:0/2799042029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4e080 msgr2=0x563b24a58400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:26.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.975+0000 7f4f68ef5640 1 --2- 172.21.15.172:0/2802754367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4f540a2b80 0x7f4f540a5040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800cfd0 tx=0x7f4f5800cf20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:26.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.975+0000 7f4f6d532740 1 --2- 172.21.15.172:0/2799042029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4e080 0x563b24a58400 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f4f64102220 tx=0x7f4f500103d0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.975+0000 7f4f6d532740 1 -- 172.21.15.172:0/2799042029 shutdown_connections 2024-09-10T08:55:26.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.975+0000 7f4f6d532740 1 --2- 172.21.15.172:0/2799042029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a58940 0x563b24a5ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.975+0000 7f4f6d532740 1 --2- 172.21.15.172:0/2799042029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4e080 0x563b24a58400 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.975+0000 7f4f6d532740 1 --2- 172.21.15.172:0/2799042029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4d760 0x563b24a4db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.975+0000 7f4f6d532740 1 -- 172.21.15.172:0/2799042029 >> 172.21.15.172:0/2799042029 conn(0x563b24a49cf0 msgr2=0x563b24a4a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:26.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.975+0000 7f4f6d532740 1 -- 172.21.15.172:0/2799042029 shutdown_connections 2024-09-10T08:55:26.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.975+0000 7f4f6d532740 1 -- 172.21.15.172:0/2799042029 wait complete. 2024-09-10T08:55:26.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.975+0000 7f4f6d532740 1 -- start start 2024-09-10T08:55:26.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.976+0000 7f4f6d532740 1 Processor -- start 2024-09-10T08:55:26.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.976+0000 7f4f6d532740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a4d760 0x563b24a55840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.976+0000 7f4f6d532740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4e080 0x563b24a55d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.976+0000 7f4f6d532740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a58940 0x563b24a562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.976+0000 7f4f6d532740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563b24a5b510 con 0x563b24a58940 2024-09-10T08:55:26.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.976+0000 7f4f6d532740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b2493b6e0 con 0x563b24a4d760 2024-09-10T08:55:26.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.976+0000 7f4f6d532740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563b24a5b690 con 0x563b24a4e080 2024-09-10T08:55:26.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.976+0000 7f4f6d532740 10 client.0 _ll_register_callbacks cb 0x563b2495fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:26.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.977+0000 7f4f63fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4e080 0x563b24a55d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.977+0000 7f4f63fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4e080 0x563b24a55d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:35638/0 (socket says 172.21.15.172:35638) 2024-09-10T08:55:26.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.977+0000 7f4f63fff640 1 -- 172.21.15.172:0/1393231329 learned_addr learned my addr 172.21.15.172:0/1393231329 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:26.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.977+0000 7f4f696f6640 1 --2- 172.21.15.172:0/1393231329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a58940 0x563b24a562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.977+0000 7f4f696f6640 1 --2- 172.21.15.172:0/1393231329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a4d760 0x563b24a55840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.977+0000 7f4f63fff640 1 -- 172.21.15.172:0/1393231329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a4d760 msgr2=0x563b24a55840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:26.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.977+0000 7f4f63fff640 1 --2- 172.21.15.172:0/1393231329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a4d760 0x563b24a55840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.977+0000 7f4f63fff640 1 -- 172.21.15.172:0/1393231329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a58940 msgr2=0x563b24a562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:26.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.977+0000 7f4f63fff640 1 --2- 172.21.15.172:0/1393231329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a58940 0x563b24a562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.977+0000 7f4f63fff640 1 -- 172.21.15.172:0/1393231329 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b24a64a30 con 0x563b24a4e080 2024-09-10T08:55:26.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.977+0000 7f4f63fff640 1 --2- 172.21.15.172:0/1393231329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4e080 0x563b24a55d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4f50004030 tx=0x7f4f50004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c27c3657c7dd78f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:26.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.977+0000 7f4f43fff640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:55:26.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.977+0000 7f4f43fff640 1 -- 172.21.15.172:0/1393231329 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50004410 con 0x563b24a4e080 2024-09-10T08:55:26.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.977+0000 7f4f43fff640 1 -- 172.21.15.172:0/1393231329 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4f50005040 con 0x563b24a4e080 2024-09-10T08:55:26.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.977+0000 7f4f6d532740 1 -- 172.21.15.172:0/1393231329 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f500035f0 con 0x563b24a4e080 2024-09-10T08:55:26.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.978+0000 7f4f43fff640 1 -- 172.21.15.172:0/1393231329 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4f50005240 con 0x563b24a4e080 2024-09-10T08:55:26.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.978+0000 7f4f6d532740 10 client.27145 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:26.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.978+0000 7f4f6d532740 1 -- 172.21.15.172:0/1393231329 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563b24a673a0 con 0x563b24a4e080 2024-09-10T08:55:26.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.978+0000 7f4f43fff640 1 -- 172.21.15.172:0/1393231329 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f500035f0 con 0x563b24a4e080 2024-09-10T08:55:26.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.978+0000 7f4f43fff640 1 -- 172.21.15.172:0/1393231329 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f4f50004750 con 0x563b24a4e080 2024-09-10T08:55:26.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.978+0000 7f4f6d532740 10 client.27145 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:26.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.978+0000 7f4f6d532740 10 client.27145 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:55:26.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.978+0000 7f4f6d532740 20 client.27145 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:26.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.978+0000 7f4f6d532740 1 --2- 172.21.15.172:0/1393231329 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563b24a687b0 0x563b24a6aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:26.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.978+0000 7f4f6d532740 4 client.27145 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:55:26.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.978+0000 7f4f6d532740 1 -- 172.21.15.172:0/1393231329 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b249c09d0 con 0x563b24a687b0 2024-09-10T08:55:26.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.979+0000 7f4f696f6640 1 --2- 172.21.15.172:0/1393231329 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563b24a687b0 0x563b24a6aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:26.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.979+0000 7f4f43fff640 10 client.27145 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:26.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.979+0000 7f4f696f6640 1 --2- 172.21.15.172:0/1393231329 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563b24a687b0 0x563b24a6aba0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4980f515bcb22ab5 server_cookie=1e333143ed026a49 in_seq=0 out_seq=0 2024-09-10T08:55:26.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.980+0000 7f4f43fff640 1 -- 172.21.15.172:0/1393231329 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b249c09d0 con 0x563b24a687b0 2024-09-10T08:55:26.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.980+0000 7f4f43fff640 10 client.27145 handle_command_reply: tid=0 2024-09-10T08:55:26.984 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.980+0000 7f4f6d532740 1 client.27145 shutdown 2024-09-10T08:55:26.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.980+0000 7f4f6d532740 1 -- 172.21.15.172:0/1393231329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4e080 msgr2=0x563b24a55d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:26.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.980+0000 7f4f6d532740 1 --2- 172.21.15.172:0/1393231329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4e080 0x563b24a55d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4f50004030 tx=0x7f4f50004060 comp rx=0 tx=0).stop 2024-09-10T08:55:26.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.980+0000 7f4f6d532740 1 -- 172.21.15.172:0/1393231329 shutdown_connections 2024-09-10T08:55:26.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.980+0000 7f4f6d532740 1 --2- 172.21.15.172:0/1393231329 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563b24a687b0 0x563b24a6aba0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.980+0000 7f4f43fff640 0 client.27145 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:26.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.980+0000 7f4f6d532740 1 --2- 172.21.15.172:0/1393231329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a58940 0x563b24a562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.980+0000 7f4f6d532740 1 --2- 172.21.15.172:0/1393231329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4e080 0x563b24a55d80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.980+0000 7f4f6d532740 1 --2- 172.21.15.172:0/1393231329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a4d760 0x563b24a55840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:26.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.980+0000 7f4f6d532740 1 -- 172.21.15.172:0/1393231329 >> 172.21.15.172:0/1393231329 conn(0x563b24a49cf0 msgr2=0x563b24a48e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:26.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.980+0000 7f4f6d532740 1 -- 172.21.15.172:0/1393231329 shutdown_connections 2024-09-10T08:55:26.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.980+0000 7f4f6d532740 1 -- 172.21.15.172:0/1393231329 wait complete. 2024-09-10T08:55:26.985 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:26.980+0000 7f4f6d532740 20 client.27145 trim_cache size 0 max 16384 2024-09-10T08:55:27.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.003+0000 7f4f6d532740 1 Processor -- start 2024-09-10T08:55:27.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.003+0000 7f4f6d532740 1 -- start start 2024-09-10T08:55:27.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.004+0000 7f4f6d532740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a4d760 0x563b24a70310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:27.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.004+0000 7f4f6d532740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4e080 0x563b24a70850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:27.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.004+0000 7f4f6d532740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a58940 0x563b24a52aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:27.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.004+0000 7f4f6d532740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641b5200 con 0x563b24a4e080 2024-09-10T08:55:27.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.004+0000 7f4f6d532740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641b6d40 con 0x563b24a4d760 2024-09-10T08:55:27.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.004+0000 7f4f6d532740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641b58d0 con 0x563b24a58940 2024-09-10T08:55:27.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.004+0000 7f4f63fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4e080 0x563b24a70850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:27.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.004+0000 7f4f63fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4e080 0x563b24a70850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42984/0 (socket says 172.21.15.172:42984) 2024-09-10T08:55:27.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.004+0000 7f4f63fff640 1 -- 172.21.15.172:0/1567767638 learned_addr learned my addr 172.21.15.172:0/1567767638 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:27.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.004+0000 7f4f696f6640 1 --2- 172.21.15.172:0/1567767638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a58940 0x563b24a52aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:27.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.004+0000 7f4f696f6640 1 --2- 172.21.15.172:0/1567767638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a4d760 0x563b24a70310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:27.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.004+0000 7f4f696f6640 1 -- 172.21.15.172:0/1567767638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a4d760 msgr2=0x563b24a70310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:27.006 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.004+0000 7f4f696f6640 1 --2- 172.21.15.172:0/1567767638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a4d760 0x563b24a70310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:27.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.004+0000 7f4f696f6640 1 -- 172.21.15.172:0/1567767638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4e080 msgr2=0x563b24a70850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:27.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.004+0000 7f4f696f6640 1 --2- 172.21.15.172:0/1567767638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4e080 0x563b24a70850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:27.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.004+0000 7f4f696f6640 1 -- 172.21.15.172:0/1567767638 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b24a45260 con 0x563b24a58940 2024-09-10T08:55:27.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.004+0000 7f4f63fff640 1 --2- 172.21.15.172:0/1567767638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4e080 0x563b24a70850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:27.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.005+0000 7f4f696f6640 1 --2- 172.21.15.172:0/1567767638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a58940 0x563b24a52aa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c02ae00 tx=0x7f4f5c032fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25fc40468fcf5224 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:27.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.005+0000 7f4f4e7fc640 1 -- 172.21.15.172:0/1567767638 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c050030 con 0x563b24a58940 2024-09-10T08:55:27.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.005+0000 7f4f4e7fc640 1 -- 172.21.15.172:0/1567767638 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f5c036040 con 0x563b24a58940 2024-09-10T08:55:27.007 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.005+0000 7f4f4e7fc640 1 -- 172.21.15.172:0/1567767638 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c004030 con 0x563b24a58940 2024-09-10T08:55:27.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.006+0000 7f4f6d532740 1 -- 172.21.15.172:0/1567767638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a58940 msgr2=0x563b24a52aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:27.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.006+0000 7f4f6d532740 1 --2- 172.21.15.172:0/1567767638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a58940 0x563b24a52aa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c02ae00 tx=0x7f4f5c032fd0 comp rx=0 tx=0).stop 2024-09-10T08:55:27.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.006+0000 7f4f6d532740 1 -- 172.21.15.172:0/1567767638 shutdown_connections 2024-09-10T08:55:27.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.006+0000 7f4f6d532740 1 --2- 172.21.15.172:0/1567767638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a58940 0x563b24a52aa0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:27.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.006+0000 7f4f6d532740 1 --2- 172.21.15.172:0/1567767638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4e080 0x563b24a70850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:27.008 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.006+0000 7f4f6d532740 1 --2- 172.21.15.172:0/1567767638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a4d760 0x563b24a70310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:27.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.006+0000 7f4f6d532740 1 -- 172.21.15.172:0/1567767638 >> 172.21.15.172:0/1567767638 conn(0x563b24a49cf0 msgr2=0x563b24a578c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:27.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.006+0000 7f4f6d532740 1 -- 172.21.15.172:0/1567767638 shutdown_connections 2024-09-10T08:55:27.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.007+0000 7f4f6d532740 1 -- 172.21.15.172:0/1567767638 wait complete. 2024-09-10T08:55:27.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.007+0000 7f4f6d532740 1 -- start start 2024-09-10T08:55:27.009 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.007+0000 7f4f6d532740 1 Processor -- start 2024-09-10T08:55:27.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.008+0000 7f4f6d532740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4d760 0x563b24a5c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:27.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.008+0000 7f4f6d532740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4e080 0x563b24a5cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:27.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.008+0000 7f4f696f6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4d760 0x563b24a5c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:27.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.008+0000 7f4f696f6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4d760 0x563b24a5c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42992/0 (socket says 172.21.15.172:42992) 2024-09-10T08:55:27.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.008+0000 7f4f696f6640 1 -- 172.21.15.172:0/1560631295 learned_addr learned my addr 172.21.15.172:0/1560631295 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:27.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.009+0000 7f4f63fff640 1 --2- 172.21.15.172:0/1560631295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4e080 0x563b24a5cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:27.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.009+0000 7f4f6d532740 1 --2- 172.21.15.172:0/1560631295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a58940 0x563b24a5d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:27.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.009+0000 7f4f6d532740 1 -- 172.21.15.172:0/1560631295 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641b6670 con 0x563b24a4d760 2024-09-10T08:55:27.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.009+0000 7f4f6d532740 1 -- 172.21.15.172:0/1560631295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641b5fa0 con 0x563b24a58940 2024-09-10T08:55:27.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.009+0000 7f4f6d532740 1 -- 172.21.15.172:0/1560631295 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641b31e0 con 0x563b24a4e080 2024-09-10T08:55:27.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.009+0000 7f4f696f6640 1 -- 172.21.15.172:0/1560631295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4e080 msgr2=0x563b24a5cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:27.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.009+0000 7f4f696f6640 1 --2- 172.21.15.172:0/1560631295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4e080 0x563b24a5cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:27.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.009+0000 7f4f696f6640 1 -- 172.21.15.172:0/1560631295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a58940 msgr2=0x563b24a5d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:27.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.009+0000 7f4f696f6640 1 --2- 172.21.15.172:0/1560631295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a58940 0x563b24a5d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:27.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.009+0000 7f4f696f6640 1 -- 172.21.15.172:0/1560631295 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b24ad24e0 con 0x563b24a4d760 2024-09-10T08:55:27.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.010+0000 7f4f63fff640 1 --2- 172.21.15.172:0/1560631295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4e080 0x563b24a5cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:27.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.010+0000 7f4f6d532740 10 client.0 _ll_register_callbacks cb 0x7f4f50010c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:27.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.010+0000 7f4f6d532740 1 -- 172.21.15.172:0/1560631295 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f5c036040 con 0x563b24a4d760 2024-09-10T08:55:27.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.010+0000 7f4f696f6640 1 --2- 172.21.15.172:0/1560631295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4d760 0x563b24a5c840 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c02a920 tx=0x7f4f5c009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed493ddc11d73b20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:27.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.010+0000 7f4f43fff640 10 client.18216 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:27.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.010+0000 7f4f43fff640 1 -- 172.21.15.172:0/1560631295 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c058030 con 0x563b24a4d760 2024-09-10T08:55:27.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.010+0000 7f4f43fff640 1 -- 172.21.15.172:0/1560631295 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4f5c051070 con 0x563b24a4d760 2024-09-10T08:55:27.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.010+0000 7f4f43fff640 1 -- 172.21.15.172:0/1560631295 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4f5c050030 con 0x563b24a4d760 2024-09-10T08:55:27.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.010+0000 7f4f6d532740 10 client.18216 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:27.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.010+0000 7f4f6d532740 1 -- 172.21.15.172:0/1560631295 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=37}) -- 0x563b24ad34d0 con 0x563b24a4d760 2024-09-10T08:55:27.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.011+0000 7f4f43fff640 1 -- 172.21.15.172:0/1560631295 <== mon.0 v2:172.21.15.172:3300/0 4 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f4f5c049020 con 0x563b24a4d760 2024-09-10T08:55:27.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.011+0000 7f4f6d532740 10 client.18216 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:27.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.011+0000 7f4f6d532740 10 client.18216 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:55:27.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.011+0000 7f4f6d532740 20 client.18216 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:27.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.011+0000 7f4f6d532740 1 --2- 172.21.15.172:0/1560631295 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563b24ad3b30 0x563b24ad5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:27.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.011+0000 7f4f6d532740 4 client.18216 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.6, "await_for": 45.0}] 2024-09-10T08:55:27.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.011+0000 7f4f6d532740 1 -- 172.21.15.172:0/1560631295 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.6, "await_for": 45.0}) -- 0x7f4f5c036040 con 0x563b24ad3b30 2024-09-10T08:55:27.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.011+0000 7f4f63fff640 1 --2- 172.21.15.172:0/1560631295 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563b24ad3b30 0x563b24ad5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:27.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.012+0000 7f4f43fff640 10 client.18216 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:27.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.012+0000 7f4f63fff640 1 --2- 172.21.15.172:0/1560631295 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563b24ad3b30 0x563b24ad5fb0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5806cfe8ee5db53e server_cookie=3d4803fa90db59b7 in_seq=0 out_seq=0 2024-09-10T08:55:27.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.086+0000 7f4f43fff640 1 -- 172.21.15.172:0/1560631295 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f4f5c036040 con 0x563b24ad3b30 2024-09-10T08:55:27.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.086+0000 7f4f43fff640 10 client.18216 handle_command_reply: tid=0 2024-09-10T08:55:27.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.086+0000 7f4f6d532740 1 client.18216 shutdown 2024-09-10T08:55:27.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.086+0000 7f4f6d532740 1 -- 172.21.15.172:0/1560631295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4d760 msgr2=0x563b24a5c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:27.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.086+0000 7f4f6d532740 1 --2- 172.21.15.172:0/1560631295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4d760 0x563b24a5c840 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c02a920 tx=0x7f4f5c009540 comp rx=0 tx=0).stop 2024-09-10T08:55:27.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.086+0000 7f4f6d532740 1 -- 172.21.15.172:0/1560631295 shutdown_connections 2024-09-10T08:55:27.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.086+0000 7f4f6d532740 1 --2- 172.21.15.172:0/1560631295 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563b24ad3b30 0x563b24ad5fb0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:27.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.086+0000 7f4f6d532740 1 --2- 172.21.15.172:0/1560631295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b24a58940 0x563b24a5d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:27.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.086+0000 7f4f43fff640 0 client.18216 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:27.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.086+0000 7f4f6d532740 1 --2- 172.21.15.172:0/1560631295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b24a4e080 0x563b24a5cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:27.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.086+0000 7f4f6d532740 1 --2- 172.21.15.172:0/1560631295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b24a4d760 0x563b24a5c840 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:27.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.086+0000 7f4f6d532740 1 -- 172.21.15.172:0/1560631295 >> 172.21.15.172:0/1560631295 conn(0x563b24a49cf0 msgr2=0x563b24a48e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:27.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.087+0000 7f4f6d532740 1 -- 172.21.15.172:0/1560631295 shutdown_connections 2024-09-10T08:55:27.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.087+0000 7f4f6d532740 1 -- 172.21.15.172:0/1560631295 wait complete. 2024-09-10T08:55:27.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.087+0000 7f4f6d532740 20 client.18216 trim_cache size 0 max 16384 2024-09-10T08:55:27.089 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:55:27.089 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 37, 2024-09-10T08:55:27.089 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:55:27.089 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 30, 2024-09-10T08:55:27.089 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:55:27.089 INFO:teuthology.orchestra.run.smithi172.stdout: "b923df17": { 2024-09-10T08:55:27.090 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 30, 2024-09-10T08:55:27.090 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:55:27.090 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:55:27.090 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:55:27.090 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:55:27.090 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:55:27.090 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:55:27.090 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 144.6, 2024-09-10T08:55:27.090 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:55:27.090 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:55:27.090 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:55:27.090 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:55:27.091 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:55:27.091 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:55:27.091 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:55:27.091 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:55:27.091 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:55:27.091 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:55:27.091 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:55:27.091 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:55:27.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.087+0000 7f4f6b159640 1 -- 172.21.15.172:0/2802754367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4f540a2b80 msgr2=0x7f4f540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:27.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.087+0000 7f4f6b159640 1 --2- 172.21.15.172:0/2802754367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4f540a2b80 0x7f4f540a5040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800cfd0 tx=0x7f4f5800cf20 comp rx=0 tx=0).stop 2024-09-10T08:55:27.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.087+0000 7f4f6b159640 1 -- 172.21.15.172:0/2802754367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f64102d30 msgr2=0x7f4f641009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:27.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.087+0000 7f4f6b159640 1 --2- 172.21.15.172:0/2802754367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f64102d30 0x7f4f641009d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4f5800e9f0 tx=0x7f4f5800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:55:27.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.088+0000 7f4f6b159640 1 -- 172.21.15.172:0/2802754367 shutdown_connections 2024-09-10T08:55:27.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.088+0000 7f4f6b159640 1 --2- 172.21.15.172:0/2802754367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4f540a2b80 0x7f4f540a5040 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:27.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.088+0000 7f4f6b159640 1 --2- 172.21.15.172:0/2802754367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f641082a0 0x7f4f64101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:27.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.088+0000 7f4f6b159640 1 --2- 172.21.15.172:0/2802754367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f641036e0 0x7f4f64100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:27.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.088+0000 7f4f6b159640 1 --2- 172.21.15.172:0/2802754367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f64102d30 0x7f4f641009d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:27.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.088+0000 7f4f6b159640 1 -- 172.21.15.172:0/2802754367 >> 172.21.15.172:0/2802754367 conn(0x7f4f640fc7d0 msgr2=0x7f4f64105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:27.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.088+0000 7f4f6b159640 1 -- 172.21.15.172:0/2802754367 shutdown_connections 2024-09-10T08:55:27.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:27.088+0000 7f4f6b159640 1 -- 172.21.15.172:0/2802754367 wait complete. 2024-09-10T08:55:27.647 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:27 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T08:55:27.647 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:27 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.648 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:27 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:55:27.648 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:27 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:55:27.648 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:27 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:55:27.648 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:27 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:55:27.648 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:27 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.648 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:27 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.648 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:27 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:55:27.648 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:27 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.648 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:27 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:55:27.648 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:27 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.648 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:27 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.648 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:27 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:27 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T08:55:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:27 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:27 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:55:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:27 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:55:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:27 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:55:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:27 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:55:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:27 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:27 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:27 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:55:27.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:27 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:27 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:55:27.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:27 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:27 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:27 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:27 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T08:55:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:27 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:27 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:55:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:27 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:55:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:27 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:55:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:27 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:55:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:27 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:27 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:27 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:55:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:27 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:27 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:55:27.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:27 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:27 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:27 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:55:27.864 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:55:28.275 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b923df17', quiesce took 4.8 seconds. Will release after: 19.8 2024-09-10T08:55:28.335 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:55:28.457 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:28 smithi172 ceph-mon[30854]: pgmap v627: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 25 op/s 2024-09-10T08:55:28.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:28 smithi174 ceph-mon[34068]: pgmap v627: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 25 op/s 2024-09-10T08:55:28.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:28 smithi191 ceph-mon[35314]: pgmap v627: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 MiB/s wr, 25 op/s 2024-09-10T08:55:28.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.842+0000 7f081999d640 1 -- 172.21.15.172:0/1278508099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08141008d0 msgr2=0x7f0814100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:28.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.842+0000 7f081999d640 1 --2- 172.21.15.172:0/1278508099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08141008d0 0x7f0814100d30 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f0804009930 tx=0x7f080402a420 comp rx=0 tx=0).stop 2024-09-10T08:55:28.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.842+0000 7f081999d640 1 -- 172.21.15.172:0/1278508099 shutdown_connections 2024-09-10T08:55:28.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.842+0000 7f081999d640 1 --2- 172.21.15.172:0/1278508099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814101270 0x7f081410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.842+0000 7f081999d640 1 --2- 172.21.15.172:0/1278508099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08141008d0 0x7f0814100d30 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.842+0000 7f081999d640 1 --2- 172.21.15.172:0/1278508099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08141068d0 0x7f0814106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.842+0000 7f081999d640 1 -- 172.21.15.172:0/1278508099 >> 172.21.15.172:0/1278508099 conn(0x7f08140fc630 msgr2=0x7f08140fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:28.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.842+0000 7f081999d640 1 -- 172.21.15.172:0/1278508099 shutdown_connections 2024-09-10T08:55:28.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.842+0000 7f081999d640 1 -- 172.21.15.172:0/1278508099 wait complete. 2024-09-10T08:55:28.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.842+0000 7f081999d640 1 Processor -- start 2024-09-10T08:55:28.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.842+0000 7f081999d640 1 -- start start 2024-09-10T08:55:28.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.842+0000 7f081999d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08141008d0 0x7f08141982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.842+0000 7f081999d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814101270 0x7f0814198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f081999d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08141068d0 0x7f081419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f081999d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0814110ba0 con 0x7f08141008d0 2024-09-10T08:55:28.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f081999d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0814110a20 con 0x7f0814101270 2024-09-10T08:55:28.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f081999d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0814110d20 con 0x7f08141068d0 2024-09-10T08:55:28.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f081899b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08141008d0 0x7f08141982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f081899b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08141008d0 0x7f08141982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43004/0 (socket says 172.21.15.172:43004) 2024-09-10T08:55:28.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f081899b640 1 -- 172.21.15.172:0/3832436228 learned_addr learned my addr 172.21.15.172:0/3832436228 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:28.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f080bfff640 1 --2- 172.21.15.172:0/3832436228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814101270 0x7f0814198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f081919c640 1 --2- 172.21.15.172:0/3832436228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08141068d0 0x7f081419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f081899b640 1 -- 172.21.15.172:0/3832436228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08141068d0 msgr2=0x7f081419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:28.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f081899b640 1 --2- 172.21.15.172:0/3832436228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08141068d0 0x7f081419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f081899b640 1 -- 172.21.15.172:0/3832436228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814101270 msgr2=0x7f0814198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:28.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f081899b640 1 --2- 172.21.15.172:0/3832436228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814101270 0x7f0814198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f081899b640 1 -- 172.21.15.172:0/3832436228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081419d2a0 con 0x7f08141008d0 2024-09-10T08:55:28.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f081899b640 1 --2- 172.21.15.172:0/3832436228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08141008d0 0x7f08141982f0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f081000b4a0 tx=0x7f081000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:28.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f0809ffb640 1 -- 172.21.15.172:0/3832436228 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810014070 con 0x7f08141008d0 2024-09-10T08:55:28.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f081999d640 1 -- 172.21.15.172:0/3832436228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081419d590 con 0x7f08141008d0 2024-09-10T08:55:28.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f0809ffb640 1 -- 172.21.15.172:0/3832436228 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0810010ac0 con 0x7f08141008d0 2024-09-10T08:55:28.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f0809ffb640 1 -- 172.21.15.172:0/3832436228 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810017c30 con 0x7f08141008d0 2024-09-10T08:55:28.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.843+0000 7f081999d640 1 -- 172.21.15.172:0/3832436228 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08141a4e70 con 0x7f08141008d0 2024-09-10T08:55:28.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.846+0000 7f0809ffb640 1 -- 172.21.15.172:0/3832436228 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0810017070 con 0x7f08141008d0 2024-09-10T08:55:28.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.846+0000 7f081cfd8740 1 Processor -- start 2024-09-10T08:55:28.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.846+0000 7f081cfd8740 1 -- start start 2024-09-10T08:55:28.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.846+0000 7f0809ffb640 1 --2- 172.21.15.172:0/3832436228 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f07fc0a2b80 0x7f07fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.846+0000 7f081cfd8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d74670 0x55f4d5d74a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.846+0000 7f081cfd8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d7f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.846+0000 7f081cfd8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d7f850 0x55f4d5d81c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.846+0000 7f081cfd8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f4d5cda2e0 con 0x55f4d5d74f90 2024-09-10T08:55:28.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.846+0000 7f081cfd8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f4d5c81ce0 con 0x55f4d5d7f850 2024-09-10T08:55:28.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.846+0000 7f081cfd8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f4d5cd62a0 con 0x55f4d5d74670 2024-09-10T08:55:28.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.846+0000 7f081919c640 1 --2- 172.21.15.172:0/3832436228 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f07fc0a2b80 0x7f07fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.846+0000 7f0809ffb640 1 -- 172.21.15.172:0/3832436228 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f08100ceb70 con 0x7f08141008d0 2024-09-10T08:55:28.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.846+0000 7f080bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d7f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.846+0000 7f080bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d7f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43016/0 (socket says 172.21.15.172:43016) 2024-09-10T08:55:28.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.846+0000 7f080bfff640 1 -- 172.21.15.172:0/3493763916 learned_addr learned my addr 172.21.15.172:0/3493763916 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:28.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.847+0000 7f081919c640 1 --2- 172.21.15.172:0/3493763916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d7f850 0x55f4d5d81c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.847+0000 7f081899b640 1 --2- 172.21.15.172:0/3493763916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d74670 0x55f4d5d74a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.847+0000 7f080bfff640 1 -- 172.21.15.172:0/3493763916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d74670 msgr2=0x55f4d5d74a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:28.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.847+0000 7f080bfff640 1 --2- 172.21.15.172:0/3493763916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d74670 0x55f4d5d74a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.847+0000 7f080bfff640 1 -- 172.21.15.172:0/3493763916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d7f850 msgr2=0x55f4d5d81c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:28.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.847+0000 7f080bfff640 1 --2- 172.21.15.172:0/3493763916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d7f850 0x55f4d5d81c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.847+0000 7f080bfff640 1 -- 172.21.15.172:0/3493763916 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4d5c66440 con 0x55f4d5d74f90 2024-09-10T08:55:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.847+0000 7f081919c640 1 --2- 172.21.15.172:0/3832436228 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f07fc0a2b80 0x7f07fc0a5040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f080c005c40 tx=0x7f080c00acb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.847+0000 7f080bfff640 1 --2- 172.21.15.172:0/3493763916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d7f310 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f080402a940 tx=0x7f0804031cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5733314be7078403 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.847+0000 7f07faffd640 1 -- 172.21.15.172:0/3493763916 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080403f070 con 0x55f4d5d74f90 2024-09-10T08:55:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.847+0000 7f07faffd640 1 -- 172.21.15.172:0/3493763916 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08040316a0 con 0x55f4d5d74f90 2024-09-10T08:55:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.847+0000 7f07faffd640 1 -- 172.21.15.172:0/3493763916 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0804035070 con 0x55f4d5d74f90 2024-09-10T08:55:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.848+0000 7f081cfd8740 1 -- 172.21.15.172:0/3493763916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 msgr2=0x55f4d5d7f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.848+0000 7f081cfd8740 1 --2- 172.21.15.172:0/3493763916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d7f310 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f080402a940 tx=0x7f0804031cb0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.848+0000 7f081cfd8740 1 -- 172.21.15.172:0/3493763916 shutdown_connections 2024-09-10T08:55:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.848+0000 7f081cfd8740 1 --2- 172.21.15.172:0/3493763916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d7f850 0x55f4d5d81c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.848+0000 7f081cfd8740 1 --2- 172.21.15.172:0/3493763916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d7f310 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.848+0000 7f081cfd8740 1 --2- 172.21.15.172:0/3493763916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d74670 0x55f4d5d74a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.848+0000 7f081cfd8740 1 -- 172.21.15.172:0/3493763916 >> 172.21.15.172:0/3493763916 conn(0x55f4d5d70c00 msgr2=0x55f4d5d70fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:28.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.848+0000 7f081cfd8740 1 -- 172.21.15.172:0/3493763916 shutdown_connections 2024-09-10T08:55:28.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.848+0000 7f081cfd8740 1 -- 172.21.15.172:0/3493763916 wait complete. 2024-09-10T08:55:28.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.848+0000 7f081cfd8740 1 -- start start 2024-09-10T08:55:28.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.848+0000 7f081cfd8740 1 Processor -- start 2024-09-10T08:55:28.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081cfd8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d74670 0x55f4d5d7f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081cfd8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d74f90 0x55f4d5d7c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081cfd8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d7f850 0x55f4d5d7c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081cfd8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f4d5d82420 con 0x55f4d5d7f850 2024-09-10T08:55:28.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081cfd8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f4d5c95050 con 0x55f4d5d74670 2024-09-10T08:55:28.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081cfd8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f4d5d825a0 con 0x55f4d5d74f90 2024-09-10T08:55:28.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081cfd8740 10 client.0 _ll_register_callbacks cb 0x55f4d5cd5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:28.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081919c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d7f850 0x55f4d5d7c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081919c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d7f850 0x55f4d5d7c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43030/0 (socket says 172.21.15.172:43030) 2024-09-10T08:55:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081919c640 1 -- 172.21.15.172:0/859293508 learned_addr learned my addr 172.21.15.172:0/859293508 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081899b640 1 --2- 172.21.15.172:0/859293508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d74670 0x55f4d5d7f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f080bfff640 1 --2- 172.21.15.172:0/859293508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d74f90 0x55f4d5d7c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081919c640 1 -- 172.21.15.172:0/859293508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d74f90 msgr2=0x55f4d5d7c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081919c640 1 --2- 172.21.15.172:0/859293508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d74f90 0x55f4d5d7c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081919c640 1 -- 172.21.15.172:0/859293508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d74670 msgr2=0x55f4d5d7f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081919c640 1 --2- 172.21.15.172:0/859293508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d74670 0x55f4d5d7f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081919c640 1 -- 172.21.15.172:0/859293508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4d5a4b3c0 con 0x55f4d5d7f850 2024-09-10T08:55:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081899b640 1 --2- 172.21.15.172:0/859293508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d74670 0x55f4d5d7f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:55:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f080bfff640 1 --2- 172.21.15.172:0/859293508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d74f90 0x55f4d5d7c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:28.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f081919c640 1 --2- 172.21.15.172:0/859293508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d7f850 0x55f4d5d7c920 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f080c010f90 tx=0x7f080c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb13ebd12571e0ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.849+0000 7f07f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.850+0000 7f07f3fff640 1 -- 172.21.15.172:0/859293508 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c003b70 con 0x55f4d5d7f850 2024-09-10T08:55:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.850+0000 7f07f3fff640 1 -- 172.21.15.172:0/859293508 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f080c003d10 con 0x55f4d5d7f850 2024-09-10T08:55:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.850+0000 7f081cfd8740 1 -- 172.21.15.172:0/859293508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08040316a0 con 0x55f4d5d7f850 2024-09-10T08:55:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.850+0000 7f07f3fff640 1 -- 172.21.15.172:0/859293508 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f080c004b20 con 0x55f4d5d7f850 2024-09-10T08:55:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.850+0000 7f081cfd8740 10 client.18228 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.850+0000 7f081cfd8740 1 -- 172.21.15.172:0/859293508 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f4d5d8ddc0 con 0x55f4d5d7f850 2024-09-10T08:55:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.850+0000 7f07f3fff640 1 -- 172.21.15.172:0/859293508 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08040316a0 con 0x55f4d5d7f850 2024-09-10T08:55:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.850+0000 7f07f3fff640 1 -- 172.21.15.172:0/859293508 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f080c01c040 con 0x55f4d5d7f850 2024-09-10T08:55:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.850+0000 7f081cfd8740 10 client.18228 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.850+0000 7f081cfd8740 10 client.18228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:28.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.850+0000 7f081cfd8740 20 client.18228 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:28.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.850+0000 7f081cfd8740 1 --2- 172.21.15.172:0/859293508 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f4d5d8f280 0x55f4d5d916d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.850+0000 7f081cfd8740 4 client.18228 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:55:28.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.850+0000 7f081cfd8740 1 -- 172.21.15.172:0/859293508 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4d5ce7850 con 0x55f4d5d8f280 2024-09-10T08:55:28.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.851+0000 7f081899b640 1 --2- 172.21.15.172:0/859293508 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f4d5d8f280 0x55f4d5d916d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.851+0000 7f081899b640 1 --2- 172.21.15.172:0/859293508 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f4d5d8f280 0x55f4d5d916d0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffe2373639a8b1dd server_cookie=9c2aa34af5e50069 in_seq=0 out_seq=0 2024-09-10T08:55:28.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.851+0000 7f07f3fff640 10 client.18228 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:28.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.852+0000 7f07f3fff640 1 -- 172.21.15.172:0/859293508 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4d5ce7850 con 0x55f4d5d8f280 2024-09-10T08:55:28.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.852+0000 7f07f3fff640 10 client.18228 handle_command_reply: tid=0 2024-09-10T08:55:28.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.852+0000 7f081cfd8740 1 client.18228 shutdown 2024-09-10T08:55:28.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.852+0000 7f081cfd8740 1 -- 172.21.15.172:0/859293508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d7f850 msgr2=0x55f4d5d7c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:28.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.852+0000 7f081cfd8740 1 --2- 172.21.15.172:0/859293508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d7f850 0x55f4d5d7c920 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f080c010f90 tx=0x7f080c004ab0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.852+0000 7f081cfd8740 1 -- 172.21.15.172:0/859293508 shutdown_connections 2024-09-10T08:55:28.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.852+0000 7f081cfd8740 1 --2- 172.21.15.172:0/859293508 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f4d5d8f280 0x55f4d5d916d0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.852+0000 7f081cfd8740 1 --2- 172.21.15.172:0/859293508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d7f850 0x55f4d5d7c920 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.852+0000 7f081cfd8740 1 --2- 172.21.15.172:0/859293508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d74f90 0x55f4d5d7c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.852+0000 7f081cfd8740 1 --2- 172.21.15.172:0/859293508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d74670 0x55f4d5d7f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.852+0000 7f07f3fff640 0 client.18228 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:28.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.852+0000 7f081cfd8740 1 -- 172.21.15.172:0/859293508 >> 172.21.15.172:0/859293508 conn(0x55f4d5d70c00 msgr2=0x55f4d5d768e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:28.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.852+0000 7f081cfd8740 1 -- 172.21.15.172:0/859293508 shutdown_connections 2024-09-10T08:55:28.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.852+0000 7f081cfd8740 1 -- 172.21.15.172:0/859293508 wait complete. 2024-09-10T08:55:28.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.852+0000 7f081cfd8740 20 client.18228 trim_cache size 0 max 16384 2024-09-10T08:55:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.869+0000 7f081cfd8740 1 Processor -- start 2024-09-10T08:55:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.869+0000 7f081cfd8740 1 -- start start 2024-09-10T08:55:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.869+0000 7f081cfd8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d74670 0x55f4d5d96fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.869+0000 7f081cfd8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d974e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.869+0000 7f081cfd8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d7f850 0x55f4d5d79640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.869+0000 7f081cfd8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f08141aea10 con 0x55f4d5d74f90 2024-09-10T08:55:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.869+0000 7f081cfd8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08141b0550 con 0x55f4d5d74670 2024-09-10T08:55:28.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.869+0000 7f081cfd8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08141af0e0 con 0x55f4d5d7f850 2024-09-10T08:55:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.869+0000 7f080bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.869+0000 7f080bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43036/0 (socket says 172.21.15.172:43036) 2024-09-10T08:55:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.869+0000 7f080bfff640 1 -- 172.21.15.172:0/846035226 learned_addr learned my addr 172.21.15.172:0/846035226 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.870+0000 7f081919c640 1 --2- 172.21.15.172:0/846035226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d7f850 0x55f4d5d79640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.870+0000 7f081899b640 1 --2- 172.21.15.172:0/846035226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d74670 0x55f4d5d96fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.870+0000 7f080bfff640 1 -- 172.21.15.172:0/846035226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d7f850 msgr2=0x55f4d5d79640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.870+0000 7f080bfff640 1 --2- 172.21.15.172:0/846035226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d7f850 0x55f4d5d79640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.870+0000 7f080bfff640 1 -- 172.21.15.172:0/846035226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d74670 msgr2=0x55f4d5d96fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.870+0000 7f080bfff640 1 --2- 172.21.15.172:0/846035226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d74670 0x55f4d5d96fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.870+0000 7f080bfff640 1 -- 172.21.15.172:0/846035226 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4d5d706a0 con 0x55f4d5d74f90 2024-09-10T08:55:28.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.870+0000 7f081899b640 1 --2- 172.21.15.172:0/846035226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d74670 0x55f4d5d96fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:55:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.870+0000 7f080bfff640 1 --2- 172.21.15.172:0/846035226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d974e0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f0804009930 tx=0x7f08040316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7f16e8fb763835 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.870+0000 7f07fa7fc640 1 -- 172.21.15.172:0/846035226 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080404a030 con 0x55f4d5d74f90 2024-09-10T08:55:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.870+0000 7f07fa7fc640 1 -- 172.21.15.172:0/846035226 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08040318a0 con 0x55f4d5d74f90 2024-09-10T08:55:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.870+0000 7f07fa7fc640 1 -- 172.21.15.172:0/846035226 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0804035070 con 0x55f4d5d74f90 2024-09-10T08:55:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.871+0000 7f081cfd8740 1 -- 172.21.15.172:0/846035226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 msgr2=0x55f4d5d974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.871+0000 7f081cfd8740 1 --2- 172.21.15.172:0/846035226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d974e0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f0804009930 tx=0x7f08040316b0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.871+0000 7f081cfd8740 1 -- 172.21.15.172:0/846035226 shutdown_connections 2024-09-10T08:55:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.871+0000 7f081cfd8740 1 --2- 172.21.15.172:0/846035226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d7f850 0x55f4d5d79640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.871+0000 7f081cfd8740 1 --2- 172.21.15.172:0/846035226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d974e0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.871+0000 7f081cfd8740 1 --2- 172.21.15.172:0/846035226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d74670 0x55f4d5d96fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.871+0000 7f081cfd8740 1 -- 172.21.15.172:0/846035226 >> 172.21.15.172:0/846035226 conn(0x55f4d5d70c00 msgr2=0x55f4d5d7e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.871+0000 7f081cfd8740 1 -- 172.21.15.172:0/846035226 shutdown_connections 2024-09-10T08:55:28.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.871+0000 7f081cfd8740 1 -- 172.21.15.172:0/846035226 wait complete. 2024-09-10T08:55:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.871+0000 7f081cfd8740 1 -- start start 2024-09-10T08:55:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.871+0000 7f081cfd8740 1 Processor -- start 2024-09-10T08:55:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.871+0000 7f081cfd8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d74670 0x55f4d5d856c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.871+0000 7f081cfd8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d85c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.871+0000 7f081cfd8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d7f850 0x55f4d5d86140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f081cfd8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f08141afe80 con 0x55f4d5d74f90 2024-09-10T08:55:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f081cfd8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08141af7b0 con 0x55f4d5d74670 2024-09-10T08:55:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f081cfd8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08141ac9f0 con 0x55f4d5d7f850 2024-09-10T08:55:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f081cfd8740 10 client.0 _ll_register_callbacks cb 0x7f080c010c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f080bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d85c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f080bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d85c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43040/0 (socket says 172.21.15.172:43040) 2024-09-10T08:55:28.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f080bfff640 1 -- 172.21.15.172:0/565350391 learned_addr learned my addr 172.21.15.172:0/565350391 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f081899b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d74670 0x55f4d5d856c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f081919c640 1 --2- 172.21.15.172:0/565350391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d7f850 0x55f4d5d86140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f080bfff640 1 -- 172.21.15.172:0/565350391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d7f850 msgr2=0x55f4d5d86140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f080bfff640 1 --2- 172.21.15.172:0/565350391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d7f850 0x55f4d5d86140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f080bfff640 1 -- 172.21.15.172:0/565350391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d74670 msgr2=0x55f4d5d856c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f080bfff640 1 --2- 172.21.15.172:0/565350391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d74670 0x55f4d5d856c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f080bfff640 1 -- 172.21.15.172:0/565350391 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4d5dfd460 con 0x55f4d5d74f90 2024-09-10T08:55:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f080bfff640 1 --2- 172.21.15.172:0/565350391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d85c00 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f0804031440 tx=0x7f08040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b0363046af3ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f07f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f07f3fff640 1 -- 172.21.15.172:0/565350391 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0804052070 con 0x55f4d5d74f90 2024-09-10T08:55:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f07f3fff640 1 -- 172.21.15.172:0/565350391 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0804004ad0 con 0x55f4d5d74f90 2024-09-10T08:55:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f081cfd8740 1 -- 172.21.15.172:0/565350391 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08040318a0 con 0x55f4d5d74f90 2024-09-10T08:55:28.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f07f3fff640 1 -- 172.21.15.172:0/565350391 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f080405e050 con 0x55f4d5d74f90 2024-09-10T08:55:28.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f081cfd8740 10 client.18234 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:28.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.872+0000 7f081cfd8740 1 -- 172.21.15.172:0/565350391 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f4d5dfe6c0 con 0x55f4d5d74f90 2024-09-10T08:55:28.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.873+0000 7f07f3fff640 1 -- 172.21.15.172:0/565350391 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08040318a0 con 0x55f4d5d74f90 2024-09-10T08:55:28.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.873+0000 7f07f3fff640 1 -- 172.21.15.172:0/565350391 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f0804003aa0 con 0x55f4d5d74f90 2024-09-10T08:55:28.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.873+0000 7f081cfd8740 10 client.18234 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:28.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.873+0000 7f081cfd8740 10 client.18234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:28.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.873+0000 7f081cfd8740 20 client.18234 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:28.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.873+0000 7f081cfd8740 1 --2- 172.21.15.172:0/565350391 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f4d5dfe990 0x55f4d5e00d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:28.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.873+0000 7f081cfd8740 4 client.18234 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:55:28.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.873+0000 7f081cfd8740 1 -- 172.21.15.172:0/565350391 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f4d5ce1d60 con 0x55f4d5dfe990 2024-09-10T08:55:28.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.873+0000 7f081899b640 1 --2- 172.21.15.172:0/565350391 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f4d5dfe990 0x55f4d5e00d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:28.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.873+0000 7f07f3fff640 10 client.18234 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:28.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.873+0000 7f081899b640 1 --2- 172.21.15.172:0/565350391 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f4d5dfe990 0x55f4d5e00d80 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df336ee656a17791 server_cookie=e8fabdd6a7a3649e in_seq=0 out_seq=0 2024-09-10T08:55:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.874+0000 7f07f3fff640 1 -- 172.21.15.172:0/565350391 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f4d5ce1d60 con 0x55f4d5dfe990 2024-09-10T08:55:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.874+0000 7f07f3fff640 10 client.18234 handle_command_reply: tid=0 2024-09-10T08:55:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.874+0000 7f081cfd8740 1 client.18234 shutdown 2024-09-10T08:55:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.874+0000 7f081cfd8740 1 -- 172.21.15.172:0/565350391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 msgr2=0x55f4d5d85c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.874+0000 7f081cfd8740 1 --2- 172.21.15.172:0/565350391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d85c00 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f0804031440 tx=0x7f08040042f0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.874+0000 7f081cfd8740 1 -- 172.21.15.172:0/565350391 shutdown_connections 2024-09-10T08:55:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.874+0000 7f081cfd8740 1 --2- 172.21.15.172:0/565350391 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f4d5dfe990 0x55f4d5e00d80 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.874+0000 7f081cfd8740 1 --2- 172.21.15.172:0/565350391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4d5d7f850 0x55f4d5d86140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.874+0000 7f081cfd8740 1 --2- 172.21.15.172:0/565350391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4d5d74f90 0x55f4d5d85c00 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.874+0000 7f081cfd8740 1 --2- 172.21.15.172:0/565350391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d5d74670 0x55f4d5d856c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.874+0000 7f081cfd8740 1 -- 172.21.15.172:0/565350391 >> 172.21.15.172:0/565350391 conn(0x55f4d5d70c00 msgr2=0x55f4d5d6f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:28.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.874+0000 7f07f3fff640 0 client.18234 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:28.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.874+0000 7f081cfd8740 1 -- 172.21.15.172:0/565350391 shutdown_connections 2024-09-10T08:55:28.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.874+0000 7f081cfd8740 1 -- 172.21.15.172:0/565350391 wait complete. 2024-09-10T08:55:28.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.874+0000 7f081cfd8740 20 client.18234 trim_cache size 0 max 16384 2024-09-10T08:55:28.882 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:55:28.882 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:55:28.882 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T08:55:28.882 INFO:teuthology.orchestra.run.smithi172.stdout: "85ac4318-0229-42f2-9e1a-461b926c0735": { 2024-09-10T08:55:28.882 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T08:55:28.882 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "85ac4318-0229-42f2-9e1a-461b926c0735", 2024-09-10T08:55:28.882 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T08:55:28.882 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:55:28.883 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:55:28.883 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:55:28.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.875+0000 7f081999d640 1 -- 172.21.15.172:0/3832436228 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f07fc0a2b80 msgr2=0x7f07fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:28.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.875+0000 7f081999d640 1 --2- 172.21.15.172:0/3832436228 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f07fc0a2b80 0x7f07fc0a5040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f080c005c40 tx=0x7f080c00acb0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.875+0000 7f081999d640 1 -- 172.21.15.172:0/3832436228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08141008d0 msgr2=0x7f08141982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:28.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.875+0000 7f081999d640 1 --2- 172.21.15.172:0/3832436228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08141008d0 0x7f08141982f0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f081000b4a0 tx=0x7f081000b960 comp rx=0 tx=0).stop 2024-09-10T08:55:28.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.875+0000 7f081999d640 1 -- 172.21.15.172:0/3832436228 shutdown_connections 2024-09-10T08:55:28.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.875+0000 7f081999d640 1 --2- 172.21.15.172:0/3832436228 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f07fc0a2b80 0x7f07fc0a5040 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.875+0000 7f081999d640 1 --2- 172.21.15.172:0/3832436228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08141068d0 0x7f081419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.875+0000 7f081999d640 1 --2- 172.21.15.172:0/3832436228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814101270 0x7f0814198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.875+0000 7f081999d640 1 --2- 172.21.15.172:0/3832436228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08141008d0 0x7f08141982f0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:28.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.875+0000 7f081999d640 1 -- 172.21.15.172:0/3832436228 >> 172.21.15.172:0/3832436228 conn(0x7f08140fc630 msgr2=0x7f081406ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:28.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.875+0000 7f081999d640 1 -- 172.21.15.172:0/3832436228 shutdown_connections 2024-09-10T08:55:28.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:28.875+0000 7f081999d640 1 -- 172.21.15.172:0/3832436228 wait complete. 2024-09-10T08:55:29.577 INFO:tasks.cephfs.filesystem:scrub status for tag:85ac4318-0229-42f2-9e1a-461b926c0735 - {'path': '/', 'tag': '85ac4318-0229-42f2-9e1a-461b926c0735', 'options': 'recursive,force'} 2024-09-10T08:55:29.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:29 smithi174 ceph-mon[34068]: pgmap v628: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 MiB/s wr, 38 op/s 2024-09-10T08:55:29.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:29 smithi191 ceph-mon[35314]: pgmap v628: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 MiB/s wr, 38 op/s 2024-09-10T08:55:29.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:29 smithi172 ceph-mon[30854]: pgmap v628: 129 pgs: 129 active+clean; 8.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 MiB/s wr, 38 op/s 2024-09-10T08:55:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.714+0000 7f0c56a5e640 1 -- 172.21.15.172:0/3703499086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c50103710 msgr2=0x7f0c50109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.714+0000 7f0c56a5e640 1 --2- 172.21.15.172:0/3703499086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c50103710 0x7f0c50109fa0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009ca0 tx=0x7f0c4802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.715+0000 7f0c56a5e640 1 -- 172.21.15.172:0/3703499086 shutdown_connections 2024-09-10T08:55:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.715+0000 7f0c56a5e640 1 --2- 172.21.15.172:0/3703499086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c50103710 0x7f0c50109fa0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.715+0000 7f0c56a5e640 1 --2- 172.21.15.172:0/3703499086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c50102d50 0x7f0c501031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.715+0000 7f0c56a5e640 1 --2- 172.21.15.172:0/3703499086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c50101b60 0x7f0c50101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.715+0000 7f0c56a5e640 1 -- 172.21.15.172:0/3703499086 >> 172.21.15.172:0/3703499086 conn(0x7f0c500fd2e0 msgr2=0x7f0c500ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.715+0000 7f0c56a5e640 1 -- 172.21.15.172:0/3703499086 shutdown_connections 2024-09-10T08:55:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.715+0000 7f0c56a5e640 1 -- 172.21.15.172:0/3703499086 wait complete. 2024-09-10T08:55:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.715+0000 7f0c56a5e640 1 Processor -- start 2024-09-10T08:55:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.715+0000 7f0c56a5e640 1 -- start start 2024-09-10T08:55:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c56a5e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c50101b60 0x7f0c50197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c56a5e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c50102d50 0x7f0c50198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c56a5e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c50103710 0x7f0c5019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c56a5e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5010d670 con 0x7f0c50101b60 2024-09-10T08:55:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c56a5e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5010d4f0 con 0x7f0c50102d50 2024-09-10T08:55:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c56a5e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5010d7f0 con 0x7f0c50103710 2024-09-10T08:55:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c4ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c50101b60 0x7f0c50197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c4ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c50101b60 0x7f0c50197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43058/0 (socket says 172.21.15.172:43058) 2024-09-10T08:55:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c4ffff640 1 -- 172.21.15.172:0/3992655510 learned_addr learned my addr 172.21.15.172:0/3992655510 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c4f7fe640 1 --2- 172.21.15.172:0/3992655510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c50102d50 0x7f0c50198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c54ffb640 1 --2- 172.21.15.172:0/3992655510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c50103710 0x7f0c5019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c4ffff640 1 -- 172.21.15.172:0/3992655510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c50103710 msgr2=0x7f0c5019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c4ffff640 1 --2- 172.21.15.172:0/3992655510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c50103710 0x7f0c5019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c4ffff640 1 -- 172.21.15.172:0/3992655510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c50102d50 msgr2=0x7f0c50198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c4ffff640 1 --2- 172.21.15.172:0/3992655510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c50102d50 0x7f0c50198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c4ffff640 1 -- 172.21.15.172:0/3992655510 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5019f800 con 0x7f0c50101b60 2024-09-10T08:55:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c54ffb640 1 --2- 172.21.15.172:0/3992655510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c50103710 0x7f0c5019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c4ffff640 1 --2- 172.21.15.172:0/3992655510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c50101b60 0x7f0c50197be0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000c970 tx=0x7f0c4000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.716+0000 7f0c4d7fa640 1 -- 172.21.15.172:0/3992655510 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c40007c10 con 0x7f0c50101b60 2024-09-10T08:55:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.717+0000 7f0c4d7fa640 1 -- 172.21.15.172:0/3992655510 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c40007db0 con 0x7f0c50101b60 2024-09-10T08:55:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.717+0000 7f0c56a5e640 1 -- 172.21.15.172:0/3992655510 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c5019faf0 con 0x7f0c50101b60 2024-09-10T08:55:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.717+0000 7f0c4d7fa640 1 -- 172.21.15.172:0/3992655510 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4000f870 con 0x7f0c50101b60 2024-09-10T08:55:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.717+0000 7f0c56a5e640 1 -- 172.21.15.172:0/3992655510 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c501a0040 con 0x7f0c50101b60 2024-09-10T08:55:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.719+0000 7f0c4d7fa640 1 -- 172.21.15.172:0/3992655510 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0c4000fa10 con 0x7f0c50101b60 2024-09-10T08:55:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.719+0000 7f0c597fb740 1 Processor -- start 2024-09-10T08:55:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.719+0000 7f0c597fb740 1 -- start start 2024-09-10T08:55:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.719+0000 7f0c597fb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d7599a0 0x56092d759da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.719+0000 7f0c597fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d75a2e0 0x56092d75eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.719+0000 7f0c597fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75f370 0x56092d761830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.719+0000 7f0c597fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56092d6bd2f0 con 0x56092d75f370 2024-09-10T08:55:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.719+0000 7f0c597fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56092d665270 con 0x56092d75a2e0 2024-09-10T08:55:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.719+0000 7f0c597fb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56092d6b92b0 con 0x56092d7599a0 2024-09-10T08:55:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.719+0000 7f0c4f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75f370 0x56092d761830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.720+0000 7f0c4f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75f370 0x56092d761830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43060/0 (socket says 172.21.15.172:43060) 2024-09-10T08:55:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.720+0000 7f0c4f7fe640 1 -- 172.21.15.172:0/3326203163 learned_addr learned my addr 172.21.15.172:0/3326203163 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:30.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.720+0000 7f0c4d7fa640 1 --2- 172.21.15.172:0/3992655510 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c380a2c30 0x7f0c380a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.720+0000 7f0c4ffff640 1 --2- 172.21.15.172:0/3326203163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d75a2e0 0x56092d75eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.720+0000 7f0c54ffb640 1 --2- 172.21.15.172:0/3326203163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d7599a0 0x56092d759da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.720+0000 7f0c4f7fe640 1 -- 172.21.15.172:0/3326203163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d7599a0 msgr2=0x56092d759da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.720+0000 7f0c4f7fe640 1 --2- 172.21.15.172:0/3326203163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d7599a0 0x56092d759da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.720+0000 7f0c4f7fe640 1 -- 172.21.15.172:0/3326203163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d75a2e0 msgr2=0x56092d75eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.720+0000 7f0c4f7fe640 1 --2- 172.21.15.172:0/3326203163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d75a2e0 0x56092d75eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.720+0000 7f0c4f7fe640 1 -- 172.21.15.172:0/3326203163 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56092d664340 con 0x56092d75f370 2024-09-10T08:55:30.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.720+0000 7f0c54ffb640 1 --2- 172.21.15.172:0/3992655510 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c380a2c30 0x7f0c380a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.720+0000 7f0c4d7fa640 1 -- 172.21.15.172:0/3992655510 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0c400cf740 con 0x7f0c50101b60 2024-09-10T08:55:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.720+0000 7f0c4f7fe640 1 --2- 172.21.15.172:0/3326203163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75f370 0x56092d761830 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400abf0 tx=0x7f0c4400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e17d248c3b96f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.720+0000 7f0c367fc640 1 -- 172.21.15.172:0/3326203163 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4401d070 con 0x56092d75f370 2024-09-10T08:55:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.720+0000 7f0c367fc640 1 -- 172.21.15.172:0/3326203163 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c44018af0 con 0x56092d75f370 2024-09-10T08:55:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.720+0000 7f0c367fc640 1 -- 172.21.15.172:0/3326203163 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44010aa0 con 0x56092d75f370 2024-09-10T08:55:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.721+0000 7f0c597fb740 1 -- 172.21.15.172:0/3326203163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75f370 msgr2=0x56092d761830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.721+0000 7f0c54ffb640 1 --2- 172.21.15.172:0/3992655510 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c380a2c30 0x7f0c380a50f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0c48031aa0 tx=0x7f0c480095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.721+0000 7f0c597fb740 1 --2- 172.21.15.172:0/3326203163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75f370 0x56092d761830 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400abf0 tx=0x7f0c4400f790 comp rx=0 tx=0).stop 2024-09-10T08:55:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.721+0000 7f0c597fb740 1 -- 172.21.15.172:0/3326203163 shutdown_connections 2024-09-10T08:55:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.721+0000 7f0c597fb740 1 --2- 172.21.15.172:0/3326203163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75f370 0x56092d761830 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.721+0000 7f0c597fb740 1 --2- 172.21.15.172:0/3326203163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d75a2e0 0x56092d75eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.721+0000 7f0c597fb740 1 --2- 172.21.15.172:0/3326203163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d7599a0 0x56092d759da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.721+0000 7f0c597fb740 1 -- 172.21.15.172:0/3326203163 >> 172.21.15.172:0/3326203163 conn(0x56092d755010 msgr2=0x56092d755410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.721+0000 7f0c597fb740 1 -- 172.21.15.172:0/3326203163 shutdown_connections 2024-09-10T08:55:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.721+0000 7f0c597fb740 1 -- 172.21.15.172:0/3326203163 wait complete. 2024-09-10T08:55:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.721+0000 7f0c597fb740 1 -- start start 2024-09-10T08:55:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c597fb740 1 Processor -- start 2024-09-10T08:55:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c597fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d7599a0 0x56092d763b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c597fb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d75a2e0 0x56092d7640d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c597fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d75f370 0x56092d76b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c597fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56092d762340 con 0x56092d7599a0 2024-09-10T08:55:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c597fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56092d646290 con 0x56092d75f370 2024-09-10T08:55:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c597fb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56092d7624c0 con 0x56092d75a2e0 2024-09-10T08:55:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c597fb740 10 client.0 _ll_register_callbacks cb 0x56092d677140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c4ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d7599a0 0x56092d763b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c4ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d7599a0 0x56092d763b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43066/0 (socket says 172.21.15.172:43066) 2024-09-10T08:55:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c4ffff640 1 -- 172.21.15.172:0/197697877 learned_addr learned my addr 172.21.15.172:0/197697877 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c4f7fe640 1 --2- 172.21.15.172:0/197697877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d75a2e0 0x56092d7640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c54ffb640 1 --2- 172.21.15.172:0/197697877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d75f370 0x56092d76b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c4ffff640 1 -- 172.21.15.172:0/197697877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d75a2e0 msgr2=0x56092d7640d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c4ffff640 1 --2- 172.21.15.172:0/197697877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d75a2e0 0x56092d7640d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c4ffff640 1 -- 172.21.15.172:0/197697877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d75f370 msgr2=0x56092d76b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c4ffff640 1 --2- 172.21.15.172:0/197697877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d75f370 0x56092d76b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.722+0000 7f0c4ffff640 1 -- 172.21.15.172:0/197697877 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56092d3d0210 con 0x56092d7599a0 2024-09-10T08:55:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.723+0000 7f0c4ffff640 1 --2- 172.21.15.172:0/197697877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d7599a0 0x56092d763b90 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f0c400cd250 tx=0x7f0c4009baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16432f7e50b9ce8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.723+0000 7f0c2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.723+0000 7f0c2f7fe640 1 -- 172.21.15.172:0/197697877 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c40018b50 con 0x56092d7599a0 2024-09-10T08:55:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.723+0000 7f0c2f7fe640 1 -- 172.21.15.172:0/197697877 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0c400a4420 con 0x56092d7599a0 2024-09-10T08:55:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.723+0000 7f0c597fb740 1 -- 172.21.15.172:0/197697877 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c44018af0 con 0x56092d7599a0 2024-09-10T08:55:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.723+0000 7f0c2f7fe640 1 -- 172.21.15.172:0/197697877 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0c4009c3d0 con 0x56092d7599a0 2024-09-10T08:55:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.723+0000 7f0c597fb740 10 client.18246 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.723+0000 7f0c597fb740 1 -- 172.21.15.172:0/197697877 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56092d49c620 con 0x56092d7599a0 2024-09-10T08:55:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.723+0000 7f0c2f7fe640 1 -- 172.21.15.172:0/197697877 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c44018af0 con 0x56092d7599a0 2024-09-10T08:55:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.723+0000 7f0c2f7fe640 1 -- 172.21.15.172:0/197697877 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f0c4009b120 con 0x56092d7599a0 2024-09-10T08:55:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.723+0000 7f0c597fb740 10 client.18246 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.723+0000 7f0c597fb740 10 client.18246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.723+0000 7f0c597fb740 20 client.18246 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.723+0000 7f0c597fb740 1 --2- 172.21.15.172:0/197697877 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56092d76f860 0x56092d771c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.723+0000 7f0c597fb740 4 client.18246 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:55:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.723+0000 7f0c597fb740 1 -- 172.21.15.172:0/197697877 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56092d6caa20 con 0x56092d76f860 2024-09-10T08:55:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.724+0000 7f0c4f7fe640 1 --2- 172.21.15.172:0/197697877 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56092d76f860 0x56092d771c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.724+0000 7f0c2f7fe640 10 client.18246 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.724+0000 7f0c4f7fe640 1 --2- 172.21.15.172:0/197697877 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56092d76f860 0x56092d771c50 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd921f89d264b30 server_cookie=40922e2d77e3a8d6 in_seq=0 out_seq=0 2024-09-10T08:55:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.725+0000 7f0c2f7fe640 1 -- 172.21.15.172:0/197697877 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56092d6caa20 con 0x56092d76f860 2024-09-10T08:55:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.725+0000 7f0c2f7fe640 10 client.18246 handle_command_reply: tid=0 2024-09-10T08:55:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.725+0000 7f0c597fb740 1 client.18246 shutdown 2024-09-10T08:55:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.725+0000 7f0c597fb740 1 -- 172.21.15.172:0/197697877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d7599a0 msgr2=0x56092d763b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.725+0000 7f0c597fb740 1 --2- 172.21.15.172:0/197697877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d7599a0 0x56092d763b90 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f0c400cd250 tx=0x7f0c4009baa0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.725+0000 7f0c597fb740 1 -- 172.21.15.172:0/197697877 shutdown_connections 2024-09-10T08:55:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.725+0000 7f0c597fb740 1 --2- 172.21.15.172:0/197697877 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56092d76f860 0x56092d771c50 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.725+0000 7f0c597fb740 1 --2- 172.21.15.172:0/197697877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d75f370 0x56092d76b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.725+0000 7f0c597fb740 1 --2- 172.21.15.172:0/197697877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d75a2e0 0x56092d7640d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.725+0000 7f0c2f7fe640 0 client.18246 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.725+0000 7f0c597fb740 1 --2- 172.21.15.172:0/197697877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d7599a0 0x56092d763b90 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.725+0000 7f0c597fb740 1 -- 172.21.15.172:0/197697877 >> 172.21.15.172:0/197697877 conn(0x56092d755010 msgr2=0x56092d756820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.725+0000 7f0c597fb740 1 -- 172.21.15.172:0/197697877 shutdown_connections 2024-09-10T08:55:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.726+0000 7f0c597fb740 1 -- 172.21.15.172:0/197697877 wait complete. 2024-09-10T08:55:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.726+0000 7f0c597fb740 20 client.18246 trim_cache size 0 max 16384 2024-09-10T08:55:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.740+0000 7f0c597fb740 1 Processor -- start 2024-09-10T08:55:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.740+0000 7f0c597fb740 1 -- start start 2024-09-10T08:55:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.740+0000 7f0c597fb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d7599a0 0x56092d763fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.740+0000 7f0c597fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75a2e0 0x56092d76b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.740+0000 7f0c597fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d75f370 0x56092d776520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.740+0000 7f0c597fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0c501aa450 con 0x56092d75a2e0 2024-09-10T08:55:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.740+0000 7f0c597fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c501abf90 con 0x56092d75f370 2024-09-10T08:55:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.740+0000 7f0c597fb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0c501aab20 con 0x56092d7599a0 2024-09-10T08:55:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.740+0000 7f0c4f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75a2e0 0x56092d76b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.740+0000 7f0c4f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75a2e0 0x56092d76b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43076/0 (socket says 172.21.15.172:43076) 2024-09-10T08:55:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.740+0000 7f0c4f7fe640 1 -- 172.21.15.172:0/3824760901 learned_addr learned my addr 172.21.15.172:0/3824760901 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.740+0000 7f0c4ffff640 1 --2- 172.21.15.172:0/3824760901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d7599a0 0x56092d763fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.740+0000 7f0c4f7fe640 1 -- 172.21.15.172:0/3824760901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d7599a0 msgr2=0x56092d763fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.740+0000 7f0c4f7fe640 1 --2- 172.21.15.172:0/3824760901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d7599a0 0x56092d763fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.740+0000 7f0c4f7fe640 1 -- 172.21.15.172:0/3824760901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d75f370 msgr2=0x56092d776520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.740+0000 7f0c4f7fe640 1 --2- 172.21.15.172:0/3824760901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d75f370 0x56092d776520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.740+0000 7f0c4f7fe640 1 -- 172.21.15.172:0/3824760901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56092d771fc0 con 0x56092d75a2e0 2024-09-10T08:55:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.741+0000 7f0c4f7fe640 1 --2- 172.21.15.172:0/3824760901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75a2e0 0x56092d76b0d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400afd0 tx=0x7f0c4400aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36ed1c93365d80e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.741+0000 7f0c35ffb640 1 -- 172.21.15.172:0/3824760901 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4401d040 con 0x56092d75a2e0 2024-09-10T08:55:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.741+0000 7f0c35ffb640 1 -- 172.21.15.172:0/3824760901 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c4401e040 con 0x56092d75a2e0 2024-09-10T08:55:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.741+0000 7f0c35ffb640 1 -- 172.21.15.172:0/3824760901 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44005330 con 0x56092d75a2e0 2024-09-10T08:55:30.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.741+0000 7f0c597fb740 1 -- 172.21.15.172:0/3824760901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75a2e0 msgr2=0x56092d76b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.741+0000 7f0c597fb740 1 --2- 172.21.15.172:0/3824760901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75a2e0 0x56092d76b0d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400afd0 tx=0x7f0c4400aaf0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.741+0000 7f0c597fb740 1 -- 172.21.15.172:0/3824760901 shutdown_connections 2024-09-10T08:55:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.741+0000 7f0c597fb740 1 --2- 172.21.15.172:0/3824760901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d75f370 0x56092d776520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.741+0000 7f0c597fb740 1 --2- 172.21.15.172:0/3824760901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75a2e0 0x56092d76b0d0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.741+0000 7f0c597fb740 1 --2- 172.21.15.172:0/3824760901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d7599a0 0x56092d763fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.741+0000 7f0c597fb740 1 -- 172.21.15.172:0/3824760901 >> 172.21.15.172:0/3824760901 conn(0x56092d755010 msgr2=0x56092d7541c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.741+0000 7f0c597fb740 1 -- 172.21.15.172:0/3824760901 shutdown_connections 2024-09-10T08:55:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.741+0000 7f0c597fb740 1 -- 172.21.15.172:0/3824760901 wait complete. 2024-09-10T08:55:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.742+0000 7f0c597fb740 1 -- start start 2024-09-10T08:55:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.742+0000 7f0c597fb740 1 Processor -- start 2024-09-10T08:55:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.742+0000 7f0c597fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d7599a0 0x56092d7691b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.742+0000 7f0c597fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75a2e0 0x56092d7696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.742+0000 7f0c597fb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d75f370 0x56092d76a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.742+0000 7f0c597fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0c501ab8c0 con 0x56092d75a2e0 2024-09-10T08:55:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.742+0000 7f0c597fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c501ab1f0 con 0x56092d7599a0 2024-09-10T08:55:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.742+0000 7f0c597fb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0c501a8430 con 0x56092d75f370 2024-09-10T08:55:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.742+0000 7f0c597fb740 10 client.0 _ll_register_callbacks cb 0x7f0c4000c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.742+0000 7f0c4f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75a2e0 0x56092d7696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.742+0000 7f0c4f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75a2e0 0x56092d7696f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43082/0 (socket says 172.21.15.172:43082) 2024-09-10T08:55:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.742+0000 7f0c4f7fe640 1 -- 172.21.15.172:0/2702562217 learned_addr learned my addr 172.21.15.172:0/2702562217 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.742+0000 7f0c54ffb640 1 --2- 172.21.15.172:0/2702562217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d75f370 0x56092d76a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c4ffff640 1 --2- 172.21.15.172:0/2702562217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d7599a0 0x56092d7691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c4f7fe640 1 -- 172.21.15.172:0/2702562217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d75f370 msgr2=0x56092d76a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c4f7fe640 1 --2- 172.21.15.172:0/2702562217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d75f370 0x56092d76a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c4f7fe640 1 -- 172.21.15.172:0/2702562217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d7599a0 msgr2=0x56092d7691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c4f7fe640 1 --2- 172.21.15.172:0/2702562217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d7599a0 0x56092d7691b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c4f7fe640 1 -- 172.21.15.172:0/2702562217 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56092d76ad10 con 0x56092d75a2e0 2024-09-10T08:55:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c4f7fe640 1 --2- 172.21.15.172:0/2702562217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75a2e0 0x56092d7696f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400fc90 tx=0x7f0c4400a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734690dcee88b55f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c2f7fe640 1 -- 172.21.15.172:0/2702562217 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4401d030 con 0x56092d75a2e0 2024-09-10T08:55:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c2f7fe640 1 -- 172.21.15.172:0/2702562217 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0c44018660 con 0x56092d75a2e0 2024-09-10T08:55:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c597fb740 1 -- 172.21.15.172:0/2702562217 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c4401e040 con 0x56092d75a2e0 2024-09-10T08:55:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c2f7fe640 1 -- 172.21.15.172:0/2702562217 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0c44044050 con 0x56092d75a2e0 2024-09-10T08:55:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c597fb740 10 client.18252 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c597fb740 1 -- 172.21.15.172:0/2702562217 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56092d7dae10 con 0x56092d75a2e0 2024-09-10T08:55:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c2f7fe640 1 -- 172.21.15.172:0/2702562217 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c4401e040 con 0x56092d75a2e0 2024-09-10T08:55:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c2f7fe640 1 -- 172.21.15.172:0/2702562217 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f0c44035070 con 0x56092d75a2e0 2024-09-10T08:55:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c597fb740 10 client.18252 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c597fb740 10 client.18252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c597fb740 20 client.18252 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c597fb740 1 --2- 172.21.15.172:0/2702562217 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56092d7db560 0x56092d7dd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c597fb740 4 client.18252 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:55:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.743+0000 7f0c597fb740 1 -- 172.21.15.172:0/2702562217 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56092d6c4d70 con 0x56092d7db560 2024-09-10T08:55:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.744+0000 7f0c4ffff640 1 --2- 172.21.15.172:0/2702562217 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56092d7db560 0x56092d7dd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.744+0000 7f0c2f7fe640 10 client.18252 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.744+0000 7f0c4ffff640 1 --2- 172.21.15.172:0/2702562217 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56092d7db560 0x56092d7dd980 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6777ec6146a1a96a server_cookie=bea4c9e9eef7ec62 in_seq=0 out_seq=0 2024-09-10T08:55:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.751+0000 7f0c2f7fe640 1 -- 172.21.15.172:0/2702562217 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56092d6c4d70 con 0x56092d7db560 2024-09-10T08:55:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.751+0000 7f0c2f7fe640 10 client.18252 handle_command_reply: tid=0 2024-09-10T08:55:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.751+0000 7f0c597fb740 1 client.18252 shutdown 2024-09-10T08:55:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.751+0000 7f0c597fb740 1 -- 172.21.15.172:0/2702562217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75a2e0 msgr2=0x56092d7696f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.751+0000 7f0c597fb740 1 --2- 172.21.15.172:0/2702562217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75a2e0 0x56092d7696f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400fc90 tx=0x7f0c4400a930 comp rx=0 tx=0).stop 2024-09-10T08:55:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.751+0000 7f0c597fb740 1 -- 172.21.15.172:0/2702562217 shutdown_connections 2024-09-10T08:55:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.751+0000 7f0c597fb740 1 --2- 172.21.15.172:0/2702562217 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56092d7db560 0x56092d7dd980 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.751+0000 7f0c597fb740 1 --2- 172.21.15.172:0/2702562217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56092d75f370 0x56092d76a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.751+0000 7f0c597fb740 1 --2- 172.21.15.172:0/2702562217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56092d75a2e0 0x56092d7696f0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.751+0000 7f0c2f7fe640 0 client.18252 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.751+0000 7f0c597fb740 1 --2- 172.21.15.172:0/2702562217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092d7599a0 0x56092d7691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.751+0000 7f0c597fb740 1 -- 172.21.15.172:0/2702562217 >> 172.21.15.172:0/2702562217 conn(0x56092d755010 msgr2=0x56092d7541c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.751+0000 7f0c597fb740 1 -- 172.21.15.172:0/2702562217 shutdown_connections 2024-09-10T08:55:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.751+0000 7f0c597fb740 1 -- 172.21.15.172:0/2702562217 wait complete. 2024-09-10T08:55:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.751+0000 7f0c597fb740 20 client.18252 trim_cache size 0 max 16384 2024-09-10T08:55:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:55:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:55:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:55:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:55:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.752+0000 7f0c56a5e640 1 -- 172.21.15.172:0/3992655510 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c380a2c30 msgr2=0x7f0c380a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.752+0000 7f0c56a5e640 1 --2- 172.21.15.172:0/3992655510 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c380a2c30 0x7f0c380a50f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0c48031aa0 tx=0x7f0c480095c0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.752+0000 7f0c56a5e640 1 -- 172.21.15.172:0/3992655510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c50101b60 msgr2=0x7f0c50197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.752+0000 7f0c56a5e640 1 --2- 172.21.15.172:0/3992655510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c50101b60 0x7f0c50197be0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000c970 tx=0x7f0c4000ce30 comp rx=0 tx=0).stop 2024-09-10T08:55:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.752+0000 7f0c56a5e640 1 -- 172.21.15.172:0/3992655510 shutdown_connections 2024-09-10T08:55:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.752+0000 7f0c56a5e640 1 --2- 172.21.15.172:0/3992655510 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c380a2c30 0x7f0c380a50f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.752+0000 7f0c56a5e640 1 --2- 172.21.15.172:0/3992655510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c50103710 0x7f0c5019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.752+0000 7f0c56a5e640 1 --2- 172.21.15.172:0/3992655510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c50102d50 0x7f0c50198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.752+0000 7f0c56a5e640 1 --2- 172.21.15.172:0/3992655510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c50101b60 0x7f0c50197be0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.752+0000 7f0c56a5e640 1 -- 172.21.15.172:0/3992655510 >> 172.21.15.172:0/3992655510 conn(0x7f0c500fd2e0 msgr2=0x7f0c50105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.752+0000 7f0c56a5e640 1 -- 172.21.15.172:0/3992655510 shutdown_connections 2024-09-10T08:55:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:30.752+0000 7f0c56a5e640 1 -- 172.21.15.172:0/3992655510 wait complete. 2024-09-10T08:55:32.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:32 smithi174 ceph-mon[34068]: pgmap v629: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 25 MiB/s wr, 47 op/s 2024-09-10T08:55:32.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:32 smithi191 ceph-mon[35314]: pgmap v629: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 25 MiB/s wr, 47 op/s 2024-09-10T08:55:32.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:32 smithi172 ceph-mon[30854]: pgmap v629: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 25 MiB/s wr, 47 op/s 2024-09-10T08:55:34.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:34 smithi174 ceph-mon[34068]: pgmap v630: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 40 op/s 2024-09-10T08:55:34.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:34 smithi191 ceph-mon[35314]: pgmap v630: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 40 op/s 2024-09-10T08:55:34.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:34 smithi172 ceph-mon[30854]: pgmap v630: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 40 op/s 2024-09-10T08:55:36.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:36 smithi174 ceph-mon[34068]: pgmap v631: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 44 op/s 2024-09-10T08:55:36.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:36 smithi191 ceph-mon[35314]: pgmap v631: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 44 op/s 2024-09-10T08:55:36.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:36 smithi172 ceph-mon[30854]: pgmap v631: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 44 op/s 2024-09-10T08:55:37.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:37 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:55:37.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:37 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:55:37.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:37 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:55:37.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:37 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:55:37.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:37 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:55:37.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:37 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:55:37.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:37 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:55:37.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:37 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:55:37.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:37 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:55:37.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:37 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:55:37.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:37 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:55:37.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:37 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:55:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:38 smithi174 ceph-mon[34068]: pgmap v632: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.6 MiB/s wr, 37 op/s 2024-09-10T08:55:38.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:38 smithi191 ceph-mon[35314]: pgmap v632: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.6 MiB/s wr, 37 op/s 2024-09-10T08:55:38.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:38 smithi172 ceph-mon[30854]: pgmap v632: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.6 MiB/s wr, 37 op/s 2024-09-10T08:55:39.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:39 smithi172 ceph-mon[30854]: pgmap v633: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.6 MiB/s wr, 32 op/s 2024-09-10T08:55:39.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:39 smithi174 ceph-mon[34068]: pgmap v633: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.6 MiB/s wr, 32 op/s 2024-09-10T08:55:39.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:39 smithi191 ceph-mon[35314]: pgmap v633: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.6 MiB/s wr, 32 op/s 2024-09-10T08:55:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:42 smithi174 ceph-mon[34068]: pgmap v634: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.3 MiB/s wr, 20 op/s 2024-09-10T08:55:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:42 smithi191 ceph-mon[35314]: pgmap v634: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.3 MiB/s wr, 20 op/s 2024-09-10T08:55:42.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:42 smithi172 ceph-mon[30854]: pgmap v634: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.3 MiB/s wr, 20 op/s 2024-09-10T08:55:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.849+0000 7f68bf14b640 1 -- 172.21.15.172:0/1275948904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68b8102660 msgr2=0x7f68b8104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.849+0000 7f68bf14b640 1 --2- 172.21.15.172:0/1275948904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68b8102660 0x7f68b8104a50 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009900 tx=0x7f68ac02a410 comp rx=0 tx=0).stop 2024-09-10T08:55:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.849+0000 7f68bf14b640 1 -- 172.21.15.172:0/1275948904 shutdown_connections 2024-09-10T08:55:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.849+0000 7f68bf14b640 1 --2- 172.21.15.172:0/1275948904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68b81078c0 0x7f68b8109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.849+0000 7f68bf14b640 1 --2- 172.21.15.172:0/1275948904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68b8104f90 0x7f68b8107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.849+0000 7f68bf14b640 1 --2- 172.21.15.172:0/1275948904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68b8102660 0x7f68b8104a50 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.849+0000 7f68bf14b640 1 -- 172.21.15.172:0/1275948904 >> 172.21.15.172:0/1275948904 conn(0x7f68b80fc1e0 msgr2=0x7f68b80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.849+0000 7f68bf14b640 1 -- 172.21.15.172:0/1275948904 shutdown_connections 2024-09-10T08:55:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.849+0000 7f68bf14b640 1 -- 172.21.15.172:0/1275948904 wait complete. 2024-09-10T08:55:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.849+0000 7f68bf14b640 1 Processor -- start 2024-09-10T08:55:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.849+0000 7f68bf14b640 1 -- start start 2024-09-10T08:55:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68bf14b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68b8102660 0x7f68b8197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68bf14b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68b8104f90 0x7f68b8198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68bf14b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68b81078c0 0x7f68b819f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68bf14b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f68b810d6b0 con 0x7f68b8104f90 2024-09-10T08:55:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68bf14b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68b810d530 con 0x7f68b8102660 2024-09-10T08:55:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68bf14b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f68b810d830 con 0x7f68b81078c0 2024-09-10T08:55:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68b7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68b8104f90 0x7f68b8198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68b7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68b8104f90 0x7f68b8198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49636/0 (socket says 172.21.15.172:49636) 2024-09-10T08:55:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68b7fff640 1 -- 172.21.15.172:0/2577214201 learned_addr learned my addr 172.21.15.172:0/2577214201 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68bcee7640 1 --2- 172.21.15.172:0/2577214201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68b8102660 0x7f68b8197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68bd6e8640 1 --2- 172.21.15.172:0/2577214201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68b81078c0 0x7f68b819f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68b7fff640 1 -- 172.21.15.172:0/2577214201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68b81078c0 msgr2=0x7f68b819f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68b7fff640 1 --2- 172.21.15.172:0/2577214201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68b81078c0 0x7f68b819f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68b7fff640 1 -- 172.21.15.172:0/2577214201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68b8102660 msgr2=0x7f68b8197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68b7fff640 1 --2- 172.21.15.172:0/2577214201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68b8102660 0x7f68b8197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68b7fff640 1 -- 172.21.15.172:0/2577214201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b819f950 con 0x7f68b8104f90 2024-09-10T08:55:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68b7fff640 1 --2- 172.21.15.172:0/2577214201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68b8104f90 0x7f68b8198280 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f68a400c970 tx=0x7f68a400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68b5ffb640 1 -- 172.21.15.172:0/2577214201 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4007bb0 con 0x7f68b8104f90 2024-09-10T08:55:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68b5ffb640 1 -- 172.21.15.172:0/2577214201 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68a4007d50 con 0x7f68b8104f90 2024-09-10T08:55:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68bf14b640 1 -- 172.21.15.172:0/2577214201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b819fc40 con 0x7f68b8104f90 2024-09-10T08:55:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.850+0000 7f68b5ffb640 1 -- 172.21.15.172:0/2577214201 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4018730 con 0x7f68b8104f90 2024-09-10T08:55:43.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.851+0000 7f68bf14b640 1 -- 172.21.15.172:0/2577214201 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b81a0190 con 0x7f68b8104f90 2024-09-10T08:55:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.853+0000 7f68b5ffb640 1 -- 172.21.15.172:0/2577214201 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f68a400f450 con 0x7f68b8104f90 2024-09-10T08:55:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.853+0000 7f68c1b22740 1 Processor -- start 2024-09-10T08:55:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.853+0000 7f68c1b22740 1 -- start start 2024-09-10T08:55:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.853+0000 7f68c1b22740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d65849a0 0x5634d6584da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.853+0000 7f68c1b22740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d6589da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68c1b22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d658a370 0x5634d658c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68b5ffb640 1 --2- 172.21.15.172:0/2577214201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68a80a2cb0 0x7f68a80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68c1b22740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5634d64e82f0 con 0x5634d65852e0 2024-09-10T08:55:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68c1b22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5634d6490270 con 0x5634d658a370 2024-09-10T08:55:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68c1b22740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5634d64e42b0 con 0x5634d65849a0 2024-09-10T08:55:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68bcee7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d6589da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68bd6e8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d65849a0 0x5634d6584da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68bcee7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d6589da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49640/0 (socket says 172.21.15.172:49640) 2024-09-10T08:55:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68bcee7640 1 -- 172.21.15.172:0/943383192 learned_addr learned my addr 172.21.15.172:0/943383192 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68b7fff640 1 --2- 172.21.15.172:0/2577214201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68a80a2cb0 0x7f68a80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68bd6e8640 1 --2- 172.21.15.172:0/943383192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d658a370 0x5634d658c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68b5ffb640 1 -- 172.21.15.172:0/2577214201 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f68a40cf550 con 0x7f68b8104f90 2024-09-10T08:55:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68bcee7640 1 -- 172.21.15.172:0/943383192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d65849a0 msgr2=0x5634d6584da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68bcee7640 1 --2- 172.21.15.172:0/943383192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d65849a0 0x5634d6584da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68bcee7640 1 -- 172.21.15.172:0/943383192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d658a370 msgr2=0x5634d658c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:43.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68bcee7640 1 --2- 172.21.15.172:0/943383192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d658a370 0x5634d658c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68bcee7640 1 -- 172.21.15.172:0/943383192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634d648f340 con 0x5634d65852e0 2024-09-10T08:55:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68bd6e8640 1 --2- 172.21.15.172:0/943383192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d65849a0 0x5634d6584da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68b7fff640 1 --2- 172.21.15.172:0/2577214201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68a80a2cb0 0x7f68a80a5170 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f68a40cccf0 tx=0x7f68a4090f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68bcee7640 1 --2- 172.21.15.172:0/943383192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d6589da0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009900 tx=0x7f68ac031cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce92b94ecb6a709 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.854+0000 7f68a2ffd640 1 -- 172.21.15.172:0/943383192 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac035030 con 0x5634d65852e0 2024-09-10T08:55:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.855+0000 7f68a2ffd640 1 -- 172.21.15.172:0/943383192 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68ac031780 con 0x5634d65852e0 2024-09-10T08:55:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.855+0000 7f68a2ffd640 1 -- 172.21.15.172:0/943383192 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac003c90 con 0x5634d65852e0 2024-09-10T08:55:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.855+0000 7f68c1b22740 1 -- 172.21.15.172:0/943383192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 msgr2=0x5634d6589da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.855+0000 7f68c1b22740 1 --2- 172.21.15.172:0/943383192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d6589da0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009900 tx=0x7f68ac031cb0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.855+0000 7f68c1b22740 1 -- 172.21.15.172:0/943383192 shutdown_connections 2024-09-10T08:55:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.855+0000 7f68c1b22740 1 --2- 172.21.15.172:0/943383192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d658a370 0x5634d658c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.855+0000 7f68c1b22740 1 --2- 172.21.15.172:0/943383192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d6589da0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.855+0000 7f68c1b22740 1 --2- 172.21.15.172:0/943383192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d65849a0 0x5634d6584da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.855+0000 7f68c1b22740 1 -- 172.21.15.172:0/943383192 >> 172.21.15.172:0/943383192 conn(0x5634d6580010 msgr2=0x5634d6580410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.855+0000 7f68c1b22740 1 -- 172.21.15.172:0/943383192 shutdown_connections 2024-09-10T08:55:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.855+0000 7f68c1b22740 1 -- 172.21.15.172:0/943383192 wait complete. 2024-09-10T08:55:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.856+0000 7f68c1b22740 1 -- start start 2024-09-10T08:55:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.856+0000 7f68c1b22740 1 Processor -- start 2024-09-10T08:55:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.856+0000 7f68c1b22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d65849a0 0x5634d658eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.856+0000 7f68c1b22740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d658f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.856+0000 7f68c1b22740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d658a370 0x5634d6596110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.856+0000 7f68c1b22740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5634d658d2e0 con 0x5634d65852e0 2024-09-10T08:55:43.864 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.856+0000 7f68c1b22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5634d6471290 con 0x5634d65849a0 2024-09-10T08:55:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.856+0000 7f68c1b22740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5634d658d460 con 0x5634d658a370 2024-09-10T08:55:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.856+0000 7f68c1b22740 10 client.0 _ll_register_callbacks cb 0x5634d64a2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.856+0000 7f68bcee7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d658f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.856+0000 7f68bcee7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d658f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49656/0 (socket says 172.21.15.172:49656) 2024-09-10T08:55:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.856+0000 7f68bcee7640 1 -- 172.21.15.172:0/4271202320 learned_addr learned my addr 172.21.15.172:0/4271202320 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.856+0000 7f68bd6e8640 1 --2- 172.21.15.172:0/4271202320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d65849a0 0x5634d658eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.856+0000 7f68bd6e8640 1 --2- 172.21.15.172:0/4271202320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d658a370 0x5634d6596110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f68bcee7640 1 -- 172.21.15.172:0/4271202320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d658a370 msgr2=0x5634d6596110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f68bcee7640 1 --2- 172.21.15.172:0/4271202320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d658a370 0x5634d6596110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f68bcee7640 1 -- 172.21.15.172:0/4271202320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d65849a0 msgr2=0x5634d658eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f68bcee7640 1 --2- 172.21.15.172:0/4271202320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d65849a0 0x5634d658eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f68bcee7640 1 -- 172.21.15.172:0/4271202320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634d61fb210 con 0x5634d65852e0 2024-09-10T08:55:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f68bd6e8640 1 --2- 172.21.15.172:0/4271202320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d658a370 0x5634d6596110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:55:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f68bcee7640 1 --2- 172.21.15.172:0/4271202320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d658f0d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f68ac000c00 tx=0x7f68ac03c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346758a6083d4b09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f6897fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f6897fff640 1 -- 172.21.15.172:0/4271202320 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac035070 con 0x5634d65852e0 2024-09-10T08:55:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f6897fff640 1 -- 172.21.15.172:0/4271202320 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f68ac031580 con 0x5634d65852e0 2024-09-10T08:55:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f68c1b22740 1 -- 172.21.15.172:0/4271202320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68ac031780 con 0x5634d65852e0 2024-09-10T08:55:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f6897fff640 1 -- 172.21.15.172:0/4271202320 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f68ac057050 con 0x5634d65852e0 2024-09-10T08:55:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f68c1b22740 10 client.18264 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f68c1b22740 1 -- 172.21.15.172:0/4271202320 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5634d62c7620 con 0x5634d65852e0 2024-09-10T08:55:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f6897fff640 1 -- 172.21.15.172:0/4271202320 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68ac031780 con 0x5634d65852e0 2024-09-10T08:55:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f6897fff640 1 -- 172.21.15.172:0/4271202320 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f68ac04e980 con 0x5634d65852e0 2024-09-10T08:55:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f68c1b22740 10 client.18264 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f68c1b22740 10 client.18264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:43.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.857+0000 7f68c1b22740 20 client.18264 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.858+0000 7f68c1b22740 1 --2- 172.21.15.172:0/4271202320 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5634d659a410 0x5634d659c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.858+0000 7f68c1b22740 4 client.18264 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:55:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.858+0000 7f68c1b22740 1 -- 172.21.15.172:0/4271202320 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634d64f5a20 con 0x5634d659a410 2024-09-10T08:55:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.858+0000 7f68bd6e8640 1 --2- 172.21.15.172:0/4271202320 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5634d659a410 0x5634d659c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.858+0000 7f68bd6e8640 1 --2- 172.21.15.172:0/4271202320 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5634d659a410 0x5634d659c800 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7964c34ca712a006 server_cookie=7a8a3b3728444a7c in_seq=0 out_seq=0 2024-09-10T08:55:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.858+0000 7f6897fff640 10 client.18264 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.859+0000 7f6897fff640 1 -- 172.21.15.172:0/4271202320 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634d64f5a20 con 0x5634d659a410 2024-09-10T08:55:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.859+0000 7f6897fff640 10 client.18264 handle_command_reply: tid=0 2024-09-10T08:55:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.859+0000 7f68c1b22740 1 client.18264 shutdown 2024-09-10T08:55:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.859+0000 7f68c1b22740 1 -- 172.21.15.172:0/4271202320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 msgr2=0x5634d658f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.859+0000 7f68c1b22740 1 --2- 172.21.15.172:0/4271202320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d658f0d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f68ac000c00 tx=0x7f68ac03c670 comp rx=0 tx=0).stop 2024-09-10T08:55:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.859+0000 7f68c1b22740 1 -- 172.21.15.172:0/4271202320 shutdown_connections 2024-09-10T08:55:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.859+0000 7f68c1b22740 1 --2- 172.21.15.172:0/4271202320 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5634d659a410 0x5634d659c800 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.859+0000 7f68c1b22740 1 --2- 172.21.15.172:0/4271202320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d658a370 0x5634d6596110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.859+0000 7f68c1b22740 1 --2- 172.21.15.172:0/4271202320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d658f0d0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.859+0000 7f6897fff640 0 client.18264 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.859+0000 7f68c1b22740 1 --2- 172.21.15.172:0/4271202320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d65849a0 0x5634d658eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.859+0000 7f68c1b22740 1 -- 172.21.15.172:0/4271202320 >> 172.21.15.172:0/4271202320 conn(0x5634d6580010 msgr2=0x5634d6581820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.859+0000 7f68c1b22740 1 -- 172.21.15.172:0/4271202320 shutdown_connections 2024-09-10T08:55:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.859+0000 7f68c1b22740 1 -- 172.21.15.172:0/4271202320 wait complete. 2024-09-10T08:55:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.859+0000 7f68c1b22740 20 client.18264 trim_cache size 0 max 16384 2024-09-10T08:55:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.873+0000 7f68c1b22740 1 Processor -- start 2024-09-10T08:55:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.873+0000 7f68c1b22740 1 -- start start 2024-09-10T08:55:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68c1b22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d6580010 0x5634d6581400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68c1b22740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65849a0 0x5634d65960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68c1b22740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d65852e0 0x5634d6581940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68c1b22740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81aa3e0 con 0x5634d65849a0 2024-09-10T08:55:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68c1b22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81abf20 con 0x5634d6580010 2024-09-10T08:55:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68c1b22740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81aaab0 con 0x5634d65852e0 2024-09-10T08:55:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68bcee7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65849a0 0x5634d65960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68bcee7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65849a0 0x5634d65960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49658/0 (socket says 172.21.15.172:49658) 2024-09-10T08:55:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68bcee7640 1 -- 172.21.15.172:0/3986902739 learned_addr learned my addr 172.21.15.172:0/3986902739 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68bd6e8640 1 --2- 172.21.15.172:0/3986902739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d65852e0 0x5634d6581940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68bd6e8640 1 --2- 172.21.15.172:0/3986902739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d6580010 0x5634d6581400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68bcee7640 1 -- 172.21.15.172:0/3986902739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d65852e0 msgr2=0x5634d6581940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68bcee7640 1 --2- 172.21.15.172:0/3986902739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d65852e0 0x5634d6581940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68bcee7640 1 -- 172.21.15.172:0/3986902739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d6580010 msgr2=0x5634d6581400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68bcee7640 1 --2- 172.21.15.172:0/3986902739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d6580010 0x5634d6581400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68bcee7640 1 -- 172.21.15.172:0/3986902739 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634d6581e80 con 0x5634d65849a0 2024-09-10T08:55:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68bd6e8640 1 --2- 172.21.15.172:0/3986902739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d6580010 0x5634d6581400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:55:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68bcee7640 1 --2- 172.21.15.172:0/3986902739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65849a0 0x5634d65960d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f68ac006010 tx=0x7f68ac0319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59ee2400db7921c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68a27fc640 1 -- 172.21.15.172:0/3986902739 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac035040 con 0x5634d65849a0 2024-09-10T08:55:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68a27fc640 1 -- 172.21.15.172:0/3986902739 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68ac03ce20 con 0x5634d65849a0 2024-09-10T08:55:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.874+0000 7f68a27fc640 1 -- 172.21.15.172:0/3986902739 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac04e680 con 0x5634d65849a0 2024-09-10T08:55:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.875+0000 7f68c1b22740 1 -- 172.21.15.172:0/3986902739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65849a0 msgr2=0x5634d65960d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:43.880 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.875+0000 7f68c1b22740 1 --2- 172.21.15.172:0/3986902739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65849a0 0x5634d65960d0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f68ac006010 tx=0x7f68ac0319a0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.875+0000 7f68c1b22740 1 -- 172.21.15.172:0/3986902739 shutdown_connections 2024-09-10T08:55:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.875+0000 7f68c1b22740 1 --2- 172.21.15.172:0/3986902739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d65852e0 0x5634d6581940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.875+0000 7f68c1b22740 1 --2- 172.21.15.172:0/3986902739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65849a0 0x5634d65960d0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.875+0000 7f68c1b22740 1 --2- 172.21.15.172:0/3986902739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d6580010 0x5634d6581400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.875+0000 7f68c1b22740 1 -- 172.21.15.172:0/3986902739 >> 172.21.15.172:0/3986902739 conn(0x5634d6589e90 msgr2=0x5634d657f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.875+0000 7f68c1b22740 1 -- 172.21.15.172:0/3986902739 shutdown_connections 2024-09-10T08:55:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.875+0000 7f68c1b22740 1 -- 172.21.15.172:0/3986902739 wait complete. 2024-09-10T08:55:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.875+0000 7f68c1b22740 1 -- start start 2024-09-10T08:55:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.875+0000 7f68c1b22740 1 Processor -- start 2024-09-10T08:55:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68c1b22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d6580010 0x5634d6593d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68c1b22740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d65849a0 0x5634d6594290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68c1b22740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d65952a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68c1b22740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81ab850 con 0x5634d65852e0 2024-09-10T08:55:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68c1b22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81ab180 con 0x5634d6580010 2024-09-10T08:55:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68c1b22740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f68b81a83c0 con 0x5634d65849a0 2024-09-10T08:55:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68c1b22740 10 client.0 _ll_register_callbacks cb 0x7f68b0013480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68bd6e8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d65952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68bd6e8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d65952a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49666/0 (socket says 172.21.15.172:49666) 2024-09-10T08:55:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68bd6e8640 1 -- 172.21.15.172:0/1326098832 learned_addr learned my addr 172.21.15.172:0/1326098832 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68bcee7640 1 --2- 172.21.15.172:0/1326098832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d65849a0 0x5634d6594290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68bd6e8640 1 --2- 172.21.15.172:0/1326098832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d6580010 0x5634d6593d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68bd6e8640 1 -- 172.21.15.172:0/1326098832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d65849a0 msgr2=0x5634d6594290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:43.882 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68bd6e8640 1 --2- 172.21.15.172:0/1326098832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d65849a0 0x5634d6594290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68bd6e8640 1 -- 172.21.15.172:0/1326098832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d6580010 msgr2=0x5634d6593d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68bd6e8640 1 --2- 172.21.15.172:0/1326098832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d6580010 0x5634d6593d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68bd6e8640 1 -- 172.21.15.172:0/1326098832 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634d65958b0 con 0x5634d65852e0 2024-09-10T08:55:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f68bd6e8640 1 --2- 172.21.15.172:0/1326098832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d65952a0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f68b00132f0 tx=0x7f68b00075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57bd405734d7526 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.876+0000 7f6897fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.877+0000 7f6897fff640 1 -- 172.21.15.172:0/1326098832 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0023030 con 0x5634d65852e0 2024-09-10T08:55:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.877+0000 7f6897fff640 1 -- 172.21.15.172:0/1326098832 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f68b0004720 con 0x5634d65852e0 2024-09-10T08:55:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.877+0000 7f68c1b22740 1 -- 172.21.15.172:0/1326098832 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68ac03ce20 con 0x5634d65852e0 2024-09-10T08:55:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.877+0000 7f6897fff640 1 -- 172.21.15.172:0/1326098832 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f68b0004b20 con 0x5634d65852e0 2024-09-10T08:55:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.877+0000 7f68c1b22740 10 client.18270 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.877+0000 7f68c1b22740 1 -- 172.21.15.172:0/1326098832 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5634d6605580 con 0x5634d65852e0 2024-09-10T08:55:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.877+0000 7f6897fff640 1 -- 172.21.15.172:0/1326098832 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68ac03ce20 con 0x5634d65852e0 2024-09-10T08:55:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.877+0000 7f6897fff640 1 -- 172.21.15.172:0/1326098832 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f68b001e2f0 con 0x5634d65852e0 2024-09-10T08:55:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.877+0000 7f68c1b22740 10 client.18270 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.877+0000 7f68c1b22740 10 client.18270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.877+0000 7f68c1b22740 20 client.18270 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.877+0000 7f68c1b22740 1 --2- 172.21.15.172:0/1326098832 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5634d658a370 0x5634d6607f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.877+0000 7f68c1b22740 4 client.18270 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:55:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.877+0000 7f68c1b22740 1 -- 172.21.15.172:0/1326098832 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634d64efd70 con 0x5634d658a370 2024-09-10T08:55:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.877+0000 7f68bcee7640 1 --2- 172.21.15.172:0/1326098832 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5634d658a370 0x5634d6607f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.878+0000 7f6897fff640 10 client.18270 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.878+0000 7f68bcee7640 1 --2- 172.21.15.172:0/1326098832 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5634d658a370 0x5634d6607f70 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b933b9f5011eeafb server_cookie=b204afaadb420c39 in_seq=0 out_seq=0 2024-09-10T08:55:43.886 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.885+0000 7f6897fff640 1 -- 172.21.15.172:0/1326098832 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634d64efd70 con 0x5634d658a370 2024-09-10T08:55:43.886 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.885+0000 7f6897fff640 10 client.18270 handle_command_reply: tid=0 2024-09-10T08:55:43.886 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.885+0000 7f68c1b22740 1 client.18270 shutdown 2024-09-10T08:55:43.886 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.885+0000 7f68c1b22740 1 -- 172.21.15.172:0/1326098832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 msgr2=0x5634d65952a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:43.886 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.885+0000 7f68c1b22740 1 --2- 172.21.15.172:0/1326098832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d65952a0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f68b00132f0 tx=0x7f68b00075f0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.885+0000 7f68c1b22740 1 -- 172.21.15.172:0/1326098832 shutdown_connections 2024-09-10T08:55:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.885+0000 7f68c1b22740 1 --2- 172.21.15.172:0/1326098832 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5634d658a370 0x5634d6607f70 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.885+0000 7f68c1b22740 1 --2- 172.21.15.172:0/1326098832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634d65852e0 0x5634d65952a0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.885+0000 7f68c1b22740 1 --2- 172.21.15.172:0/1326098832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5634d65849a0 0x5634d6594290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.885+0000 7f6897fff640 0 client.18270 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.885+0000 7f68c1b22740 1 --2- 172.21.15.172:0/1326098832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634d6580010 0x5634d6593d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.885+0000 7f68c1b22740 1 -- 172.21.15.172:0/1326098832 >> 172.21.15.172:0/1326098832 conn(0x5634d6589e90 msgr2=0x5634d65803f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.885+0000 7f68c1b22740 1 -- 172.21.15.172:0/1326098832 shutdown_connections 2024-09-10T08:55:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.885+0000 7f68c1b22740 1 -- 172.21.15.172:0/1326098832 wait complete. 2024-09-10T08:55:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.885+0000 7f68c1b22740 20 client.18270 trim_cache size 0 max 16384 2024-09-10T08:55:43.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:55:43.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:55:43.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:55:43.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:55:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.886+0000 7f68bf14b640 1 -- 172.21.15.172:0/2577214201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68a80a2cb0 msgr2=0x7f68a80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.886+0000 7f68bf14b640 1 --2- 172.21.15.172:0/2577214201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68a80a2cb0 0x7f68a80a5170 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f68a40cccf0 tx=0x7f68a4090f50 comp rx=0 tx=0).stop 2024-09-10T08:55:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.886+0000 7f68bf14b640 1 -- 172.21.15.172:0/2577214201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68b8104f90 msgr2=0x7f68b8198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.886+0000 7f68bf14b640 1 --2- 172.21.15.172:0/2577214201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68b8104f90 0x7f68b8198280 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f68a400c970 tx=0x7f68a400ce30 comp rx=0 tx=0).stop 2024-09-10T08:55:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.886+0000 7f68bf14b640 1 -- 172.21.15.172:0/2577214201 shutdown_connections 2024-09-10T08:55:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.886+0000 7f68bf14b640 1 --2- 172.21.15.172:0/2577214201 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f68a80a2cb0 0x7f68a80a5170 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.886+0000 7f68bf14b640 1 --2- 172.21.15.172:0/2577214201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68b81078c0 0x7f68b819f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.886+0000 7f68bf14b640 1 --2- 172.21.15.172:0/2577214201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68b8104f90 0x7f68b8198280 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.886+0000 7f68bf14b640 1 --2- 172.21.15.172:0/2577214201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68b8102660 0x7f68b8197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.886+0000 7f68bf14b640 1 -- 172.21.15.172:0/2577214201 >> 172.21.15.172:0/2577214201 conn(0x7f68b80fc1e0 msgr2=0x7f68b8102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:43.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.886+0000 7f68bf14b640 1 -- 172.21.15.172:0/2577214201 shutdown_connections 2024-09-10T08:55:43.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:43.886+0000 7f68bf14b640 1 -- 172.21.15.172:0/2577214201 wait complete. 2024-09-10T08:55:44.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:44 smithi174 ceph-mon[34068]: pgmap v635: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1002 KiB/s wr, 10 op/s 2024-09-10T08:55:44.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:44 smithi191 ceph-mon[35314]: pgmap v635: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1002 KiB/s wr, 10 op/s 2024-09-10T08:55:44.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:44 smithi172 ceph-mon[30854]: pgmap v635: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1002 KiB/s wr, 10 op/s 2024-09-10T08:55:45.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:45 smithi174 ceph-mon[34068]: pgmap v636: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 662 KiB/s wr, 13 op/s 2024-09-10T08:55:45.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:45 smithi191 ceph-mon[35314]: pgmap v636: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 662 KiB/s wr, 13 op/s 2024-09-10T08:55:45.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:45 smithi172 ceph-mon[30854]: pgmap v636: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 662 KiB/s wr, 13 op/s 2024-09-10T08:55:48.076 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:55:48.341 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:55:48.635 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:48 smithi172 ceph-mon[30854]: pgmap v637: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 661 KiB/s wr, 10 op/s 2024-09-10T08:55:48.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:48 smithi174 ceph-mon[34068]: pgmap v637: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 661 KiB/s wr, 10 op/s 2024-09-10T08:55:48.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:48 smithi191 ceph-mon[35314]: pgmap v637: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 661 KiB/s wr, 10 op/s 2024-09-10T08:55:48.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.877+0000 7f2e8a960640 1 -- 172.21.15.172:0/3256333186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e84102ef0 msgr2=0x7f2e8410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:48.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.877+0000 7f2e8a960640 1 --2- 172.21.15.172:0/3256333186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e84102ef0 0x7f2e8410da40 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009ca0 tx=0x7f2e7c02a950 comp rx=0 tx=0).stop 2024-09-10T08:55:48.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.877+0000 7f2e8a960640 1 -- 172.21.15.172:0/3256333186 shutdown_connections 2024-09-10T08:55:48.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.877+0000 7f2e8a960640 1 --2- 172.21.15.172:0/3256333186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e84102ef0 0x7f2e8410da40 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:48.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.877+0000 7f2e8a960640 1 --2- 172.21.15.172:0/3256333186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e84102570 0x7f2e841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:48.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.877+0000 7f2e8a960640 1 --2- 172.21.15.172:0/3256333186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e84100e20 0x7f2e84101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:48.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.877+0000 7f2e8a960640 1 -- 172.21.15.172:0/3256333186 >> 172.21.15.172:0/3256333186 conn(0x7f2e840fc7d0 msgr2=0x7f2e840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:48.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.877+0000 7f2e8a960640 1 -- 172.21.15.172:0/3256333186 shutdown_connections 2024-09-10T08:55:48.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.877+0000 7f2e8a960640 1 -- 172.21.15.172:0/3256333186 wait complete. 2024-09-10T08:55:48.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e8a960640 1 Processor -- start 2024-09-10T08:55:48.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e8a960640 1 -- start start 2024-09-10T08:55:48.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e8a960640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e84100e20 0x7f2e84197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:48.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e8a960640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e84102570 0x7f2e841977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:48.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e8a960640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e84102ef0 0x7f2e8419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:48.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e8a960640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2e84110e50 con 0x7f2e84102570 2024-09-10T08:55:48.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e8a960640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2e84110cd0 con 0x7f2e84100e20 2024-09-10T08:55:48.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e8a960640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2e84110fd0 con 0x7f2e84102ef0 2024-09-10T08:55:48.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e837fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e84102570 0x7f2e841977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:48.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e837fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e84102570 0x7f2e841977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54514/0 (socket says 172.21.15.172:54514) 2024-09-10T08:55:48.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e837fe640 1 -- 172.21.15.172:0/1916998118 learned_addr learned my addr 172.21.15.172:0/1916998118 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:48.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e83fff640 1 --2- 172.21.15.172:0/1916998118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e84100e20 0x7f2e84197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:48.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e88efd640 1 --2- 172.21.15.172:0/1916998118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e84102ef0 0x7f2e8419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:48.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e837fe640 1 -- 172.21.15.172:0/1916998118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e84102ef0 msgr2=0x7f2e8419bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:48.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e837fe640 1 --2- 172.21.15.172:0/1916998118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e84102ef0 0x7f2e8419bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:48.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e837fe640 1 -- 172.21.15.172:0/1916998118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e84100e20 msgr2=0x7f2e84197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:48.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e837fe640 1 --2- 172.21.15.172:0/1916998118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e84100e20 0x7f2e84197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:48.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e837fe640 1 -- 172.21.15.172:0/1916998118 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8419c240 con 0x7f2e84102570 2024-09-10T08:55:48.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e88efd640 1 --2- 172.21.15.172:0/1916998118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e84102ef0 0x7f2e8419bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:55:48.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.878+0000 7f2e83fff640 1 --2- 172.21.15.172:0/1916998118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e84100e20 0x7f2e84197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:48.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.879+0000 7f2e837fe640 1 --2- 172.21.15.172:0/1916998118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e84102570 0x7f2e841977d0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00c920 tx=0x7f2e6c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:48.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.879+0000 7f2e817fa640 1 -- 172.21.15.172:0/1916998118 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c007c70 con 0x7f2e84102570 2024-09-10T08:55:48.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.879+0000 7f2e8a960640 1 -- 172.21.15.172:0/1916998118 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8419c530 con 0x7f2e84102570 2024-09-10T08:55:48.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.879+0000 7f2e817fa640 1 -- 172.21.15.172:0/1916998118 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e6c007e10 con 0x7f2e84102570 2024-09-10T08:55:48.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.879+0000 7f2e817fa640 1 -- 172.21.15.172:0/1916998118 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c018820 con 0x7f2e84102570 2024-09-10T08:55:48.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.879+0000 7f2e8a960640 1 -- 172.21.15.172:0/1916998118 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e841a3e10 con 0x7f2e84102570 2024-09-10T08:55:48.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.880+0000 7f2e8a960640 1 -- 172.21.15.172:0/1916998118 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e84101200 con 0x7f2e84102570 2024-09-10T08:55:48.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.881+0000 7f2e817fa640 1 -- 172.21.15.172:0/1916998118 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2e6c0189c0 con 0x7f2e84102570 2024-09-10T08:55:48.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.882+0000 7f2e817fa640 1 --2- 172.21.15.172:0/1916998118 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e700a2b80 0x7f2e700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:48.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.882+0000 7f2e83fff640 1 --2- 172.21.15.172:0/1916998118 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e700a2b80 0x7f2e700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:48.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.882+0000 7f2e817fa640 1 -- 172.21.15.172:0/1916998118 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2e6c0cf850 con 0x7f2e84102570 2024-09-10T08:55:48.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.882+0000 7f2e83fff640 1 --2- 172.21.15.172:0/1916998118 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e700a2b80 0x7f2e700a5040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2e74009930 tx=0x7f2e74008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:48.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:48.885+0000 7f2e817fa640 1 -- 172.21.15.172:0/1916998118 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e6c014030 con 0x7f2e84102570 2024-09-10T08:55:49.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:49.084+0000 7f2e8a960640 1 -- 172.21.15.172:0/1916998118 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e841985f0 con 0x7f2e84102570 2024-09-10T08:55:49.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:49.084+0000 7f2e817fa640 1 -- 172.21.15.172:0/1916998118 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+12001 (secure 0 0 0) 0x7f2e841985f0 con 0x7f2e84102570 2024-09-10T08:55:49.086 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:55:49.087 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":37,"btime":"2024-09-10T08:52:08:313493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:52:07.658341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:55:49.087 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 37 2024-09-10T08:55:49.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:49.087+0000 7f2e8a960640 1 -- 172.21.15.172:0/1916998118 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e700a2b80 msgr2=0x7f2e700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:49.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:49.087+0000 7f2e8a960640 1 --2- 172.21.15.172:0/1916998118 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e700a2b80 0x7f2e700a5040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2e74009930 tx=0x7f2e74008040 comp rx=0 tx=0).stop 2024-09-10T08:55:49.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:49.087+0000 7f2e8a960640 1 -- 172.21.15.172:0/1916998118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e84102570 msgr2=0x7f2e841977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:49.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:49.087+0000 7f2e8a960640 1 --2- 172.21.15.172:0/1916998118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e84102570 0x7f2e841977d0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00c920 tx=0x7f2e6c00cde0 comp rx=0 tx=0).stop 2024-09-10T08:55:49.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:49.087+0000 7f2e8a960640 1 -- 172.21.15.172:0/1916998118 shutdown_connections 2024-09-10T08:55:49.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:49.087+0000 7f2e8a960640 1 --2- 172.21.15.172:0/1916998118 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2e700a2b80 0x7f2e700a5040 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:49.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:49.087+0000 7f2e8a960640 1 --2- 172.21.15.172:0/1916998118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e84102ef0 0x7f2e8419bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:49.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:49.087+0000 7f2e8a960640 1 --2- 172.21.15.172:0/1916998118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e84102570 0x7f2e841977d0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:49.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:49.087+0000 7f2e8a960640 1 --2- 172.21.15.172:0/1916998118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e84100e20 0x7f2e84197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:49.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:49.087+0000 7f2e8a960640 1 -- 172.21.15.172:0/1916998118 >> 172.21.15.172:0/1916998118 conn(0x7f2e840fc7d0 msgr2=0x7f2e84106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:49.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:49.087+0000 7f2e8a960640 1 -- 172.21.15.172:0/1916998118 shutdown_connections 2024-09-10T08:55:49.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:49.087+0000 7f2e8a960640 1 -- 172.21.15.172:0/1916998118 wait complete. 2024-09-10T08:55:49.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:49 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1916998118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:55:49.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:49 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1916998118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:55:49.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:49 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1916998118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:55:49.883 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id b923df17 --release --await' 2024-09-10T08:55:49.883 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id b923df17 --release --await 2024-09-10T08:55:50.131 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:55:50.555 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:50 smithi172 ceph-mon[30854]: pgmap v638: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-10T08:55:50.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.641+0000 7f22c8383640 1 -- 172.21.15.172:0/1987066601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22c0100eb0 msgr2=0x7f22c01067a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:50.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.641+0000 7f22c8383640 1 --2- 172.21.15.172:0/1987066601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22c0100eb0 0x7f22c01067a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009900 tx=0x7f22b002a410 comp rx=0 tx=0).stop 2024-09-10T08:55:50.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.641+0000 7f22c8383640 1 -- 172.21.15.172:0/1987066601 shutdown_connections 2024-09-10T08:55:50.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.641+0000 7f22c8383640 1 --2- 172.21.15.172:0/1987066601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22c0106ed0 0x7f22c010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.641+0000 7f22c8383640 1 --2- 172.21.15.172:0/1987066601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22c0100eb0 0x7f22c01067a0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.641+0000 7f22c8383640 1 --2- 172.21.15.172:0/1987066601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22c0100500 0x7f22c01008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.641+0000 7f22c8383640 1 -- 172.21.15.172:0/1987066601 >> 172.21.15.172:0/1987066601 conn(0x7f22c00fc5c0 msgr2=0x7f22c00fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:50.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.641+0000 7f22c8383640 1 -- 172.21.15.172:0/1987066601 shutdown_connections 2024-09-10T08:55:50.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.641+0000 7f22c8383640 1 -- 172.21.15.172:0/1987066601 wait complete. 2024-09-10T08:55:50.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c8383640 1 Processor -- start 2024-09-10T08:55:50.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c8383640 1 -- start start 2024-09-10T08:55:50.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c8383640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22c0100500 0x7f22c0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c8383640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22c0100eb0 0x7f22c01988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c8383640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22c0106ed0 0x7f22c019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c8383640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f22c0111df0 con 0x7f22c0106ed0 2024-09-10T08:55:50.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c8383640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f22c0111c70 con 0x7f22c0100500 2024-09-10T08:55:50.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c8383640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f22c0111f70 con 0x7f22c0100eb0 2024-09-10T08:55:50.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c6920640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22c0106ed0 0x7f22c019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:50.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c6920640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22c0106ed0 0x7f22c019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54538/0 (socket says 172.21.15.172:54538) 2024-09-10T08:55:50.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c6920640 1 -- 172.21.15.172:0/2013297495 learned_addr learned my addr 172.21.15.172:0/2013297495 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:50.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c611f640 1 --2- 172.21.15.172:0/2013297495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22c0100500 0x7f22c0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:50.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c591e640 1 --2- 172.21.15.172:0/2013297495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22c0100eb0 0x7f22c01988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:50.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c6920640 1 -- 172.21.15.172:0/2013297495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22c0100eb0 msgr2=0x7f22c01988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:50.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c6920640 1 --2- 172.21.15.172:0/2013297495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22c0100eb0 0x7f22c01988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c6920640 1 -- 172.21.15.172:0/2013297495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22c0100500 msgr2=0x7f22c0198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:50.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c6920640 1 --2- 172.21.15.172:0/2013297495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22c0100500 0x7f22c0198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.642+0000 7f22c6920640 1 -- 172.21.15.172:0/2013297495 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c019d310 con 0x7f22c0106ed0 2024-09-10T08:55:50.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.643+0000 7f22c6920640 1 --2- 172.21.15.172:0/2013297495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22c0106ed0 0x7f22c019cc30 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f22b800eaa0 tx=0x7f22b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:50.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.643+0000 7f22b77fe640 1 -- 172.21.15.172:0/2013297495 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b800cdb0 con 0x7f22c0106ed0 2024-09-10T08:55:50.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.643+0000 7f22b77fe640 1 -- 172.21.15.172:0/2013297495 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22b8005020 con 0x7f22c0106ed0 2024-09-10T08:55:50.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.643+0000 7f22c8383640 1 -- 172.21.15.172:0/2013297495 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c019d600 con 0x7f22c0106ed0 2024-09-10T08:55:50.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.643+0000 7f22b77fe640 1 -- 172.21.15.172:0/2013297495 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b80116d0 con 0x7f22c0106ed0 2024-09-10T08:55:50.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.643+0000 7f22c8383640 1 -- 172.21.15.172:0/2013297495 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c01a4ee0 con 0x7f22c0106ed0 2024-09-10T08:55:50.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.645+0000 7f22b77fe640 1 -- 172.21.15.172:0/2013297495 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f22b80051d0 con 0x7f22c0106ed0 2024-09-10T08:55:50.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.645+0000 7f22ca75c740 1 Processor -- start 2024-09-10T08:55:50.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.646+0000 7f22ca75c740 1 -- start start 2024-09-10T08:55:50.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.646+0000 7f22ca75c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cf3060 0x565075cf3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.646+0000 7f22ca75c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3980 0x565075cfdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.646+0000 7f22ca75c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cfe240 0x565075d00630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.646+0000 7f22ca75c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x565075c58620 con 0x565075cfe240 2024-09-10T08:55:50.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.646+0000 7f22ca75c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565075bffe50 con 0x565075cf3980 2024-09-10T08:55:50.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.646+0000 7f22ca75c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x565075c545e0 con 0x565075cf3060 2024-09-10T08:55:50.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.646+0000 7f22c591e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cfe240 0x565075d00630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:50.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.646+0000 7f22c591e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cfe240 0x565075d00630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54550/0 (socket says 172.21.15.172:54550) 2024-09-10T08:55:50.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.646+0000 7f22c591e640 1 -- 172.21.15.172:0/2321593579 learned_addr learned my addr 172.21.15.172:0/2321593579 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:50.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.646+0000 7f22b77fe640 1 --2- 172.21.15.172:0/2013297495 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22ac0a2b80 0x7f22ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.646+0000 7f22c611f640 1 --2- 172.21.15.172:0/2321593579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3980 0x565075cfdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:50.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.646+0000 7f22c591e640 1 --2- 172.21.15.172:0/2321593579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cf3060 0x565075cf3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:50.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.647+0000 7f22b77fe640 1 -- 172.21.15.172:0/2013297495 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f22b80cf4e0 con 0x7f22c0106ed0 2024-09-10T08:55:50.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.647+0000 7f22c6920640 1 --2- 172.21.15.172:0/2013297495 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22ac0a2b80 0x7f22ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:50.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.647+0000 7f22c591e640 1 -- 172.21.15.172:0/2321593579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cf3060 msgr2=0x565075cf3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:50.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.647+0000 7f22c591e640 1 --2- 172.21.15.172:0/2321593579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cf3060 0x565075cf3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.647+0000 7f22c591e640 1 -- 172.21.15.172:0/2321593579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3980 msgr2=0x565075cfdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:50.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.647+0000 7f22c591e640 1 --2- 172.21.15.172:0/2321593579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3980 0x565075cfdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.647+0000 7f22c591e640 1 -- 172.21.15.172:0/2321593579 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565075bf0d40 con 0x565075cfe240 2024-09-10T08:55:50.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.647+0000 7f22c611f640 1 --2- 172.21.15.172:0/2321593579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3980 0x565075cfdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:50.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.647+0000 7f22c591e640 1 --2- 172.21.15.172:0/2321593579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cfe240 0x565075d00630 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f22b0032fd0 tx=0x7f22b0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5aeae5f87f6d79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:50.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.647+0000 7f22c6920640 1 --2- 172.21.15.172:0/2013297495 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22ac0a2b80 0x7f22ac0a5040 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f22b8098400 tx=0x7f22b800cf50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:50.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.647+0000 7f22b4dea640 1 -- 172.21.15.172:0/2321593579 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b0035070 con 0x565075cfe240 2024-09-10T08:55:50.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.647+0000 7f22b4dea640 1 -- 172.21.15.172:0/2321593579 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22b00035f0 con 0x565075cfe240 2024-09-10T08:55:50.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.647+0000 7f22b4dea640 1 -- 172.21.15.172:0/2321593579 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b00044e0 con 0x565075cfe240 2024-09-10T08:55:50.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.648+0000 7f22ca75c740 1 -- 172.21.15.172:0/2321593579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cfe240 msgr2=0x565075d00630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:50.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.648+0000 7f22ca75c740 1 --2- 172.21.15.172:0/2321593579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cfe240 0x565075d00630 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f22b0032fd0 tx=0x7f22b0005b50 comp rx=0 tx=0).stop 2024-09-10T08:55:50.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.648+0000 7f22ca75c740 1 -- 172.21.15.172:0/2321593579 shutdown_connections 2024-09-10T08:55:50.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.648+0000 7f22ca75c740 1 --2- 172.21.15.172:0/2321593579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cfe240 0x565075d00630 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.648+0000 7f22ca75c740 1 --2- 172.21.15.172:0/2321593579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3980 0x565075cfdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.648+0000 7f22ca75c740 1 --2- 172.21.15.172:0/2321593579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cf3060 0x565075cf3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.648+0000 7f22ca75c740 1 -- 172.21.15.172:0/2321593579 >> 172.21.15.172:0/2321593579 conn(0x565075cef5f0 msgr2=0x565075cef9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:50.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.648+0000 7f22ca75c740 1 -- 172.21.15.172:0/2321593579 shutdown_connections 2024-09-10T08:55:50.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.648+0000 7f22ca75c740 1 -- 172.21.15.172:0/2321593579 wait complete. 2024-09-10T08:55:50.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.648+0000 7f22ca75c740 1 -- start start 2024-09-10T08:55:50.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.648+0000 7f22ca75c740 1 Processor -- start 2024-09-10T08:55:50.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22ca75c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3060 0x565075cfb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22ca75c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075cfb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22ca75c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cfe240 0x565075cfbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22ca75c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x565075d00e10 con 0x565075cf3980 2024-09-10T08:55:50.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22ca75c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565075be0f10 con 0x565075cf3060 2024-09-10T08:55:50.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22ca75c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x565075d00f90 con 0x565075cfe240 2024-09-10T08:55:50.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22ca75c740 10 client.0 _ll_register_callbacks cb 0x565075c04db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:50.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22c611f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075cfb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:50.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22c611f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075cfb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54556/0 (socket says 172.21.15.172:54556) 2024-09-10T08:55:50.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22c611f640 1 -- 172.21.15.172:0/3720694507 learned_addr learned my addr 172.21.15.172:0/3720694507 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:50.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22c591e640 1 --2- 172.21.15.172:0/3720694507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3060 0x565075cfb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:50.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22c591e640 1 --2- 172.21.15.172:0/3720694507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cfe240 0x565075cfbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:50.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22c591e640 1 -- 172.21.15.172:0/3720694507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cfe240 msgr2=0x565075cfbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:50.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22c591e640 1 --2- 172.21.15.172:0/3720694507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cfe240 0x565075cfbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22c591e640 1 -- 172.21.15.172:0/3720694507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 msgr2=0x565075cfb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:50.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22c591e640 1 --2- 172.21.15.172:0/3720694507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075cfb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22c591e640 1 -- 172.21.15.172:0/3720694507 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565075967580 con 0x565075cf3060 2024-09-10T08:55:50.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22c611f640 1 --2- 172.21.15.172:0/3720694507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075cfb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:50.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22c591e640 1 --2- 172.21.15.172:0/3720694507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3060 0x565075cfb130 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f22b0005940 tx=0x7f22b0041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e3bfc160f57567 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:50.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.649+0000 7f22a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:55:50.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.650+0000 7f22a9ffb640 1 -- 172.21.15.172:0/3720694507 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b004e030 con 0x565075cf3060 2024-09-10T08:55:50.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.650+0000 7f22a9ffb640 1 -- 172.21.15.172:0/3720694507 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f22b00037a0 con 0x565075cf3060 2024-09-10T08:55:50.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.650+0000 7f22ca75c740 1 -- 172.21.15.172:0/3720694507 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22b00035f0 con 0x565075cf3060 2024-09-10T08:55:50.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.650+0000 7f22a9ffb640 1 -- 172.21.15.172:0/3720694507 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f22b0035070 con 0x565075cf3060 2024-09-10T08:55:50.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.650+0000 7f22ca75c740 10 client.27251 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:50.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.650+0000 7f22ca75c740 1 -- 172.21.15.172:0/3720694507 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x565075a31960 con 0x565075cf3060 2024-09-10T08:55:50.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.650+0000 7f22a9ffb640 1 -- 172.21.15.172:0/3720694507 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22b00035f0 con 0x565075cf3060 2024-09-10T08:55:50.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.650+0000 7f22a9ffb640 1 -- 172.21.15.172:0/3720694507 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f22b003d7f0 con 0x565075cf3060 2024-09-10T08:55:50.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.650+0000 7f22ca75c740 10 client.27251 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:50.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.650+0000 7f22ca75c740 10 client.27251 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:55:50.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.650+0000 7f22ca75c740 20 client.27251 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:50.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.650+0000 7f22ca75c740 1 --2- 172.21.15.172:0/3720694507 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x565075d0dc40 0x565075d10030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.650+0000 7f22ca75c740 4 client.27251 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:55:50.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.650+0000 7f22ca75c740 1 -- 172.21.15.172:0/3720694507 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565075c65b90 con 0x565075d0dc40 2024-09-10T08:55:50.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.651+0000 7f22c611f640 1 --2- 172.21.15.172:0/3720694507 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x565075d0dc40 0x565075d10030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:50.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.651+0000 7f22a9ffb640 10 client.27251 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:50.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.651+0000 7f22c611f640 1 --2- 172.21.15.172:0/3720694507 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x565075d0dc40 0x565075d10030 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8df2437800c9f8c server_cookie=89e7632144a0af2 in_seq=0 out_seq=0 2024-09-10T08:55:50.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.652+0000 7f22a9ffb640 1 -- 172.21.15.172:0/3720694507 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565075c65b90 con 0x565075d0dc40 2024-09-10T08:55:50.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.652+0000 7f22a9ffb640 10 client.27251 handle_command_reply: tid=0 2024-09-10T08:55:50.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.652+0000 7f22ca75c740 1 client.27251 shutdown 2024-09-10T08:55:50.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.652+0000 7f22ca75c740 1 -- 172.21.15.172:0/3720694507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3060 msgr2=0x565075cfb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:50.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.652+0000 7f22ca75c740 1 --2- 172.21.15.172:0/3720694507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3060 0x565075cfb130 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f22b0005940 tx=0x7f22b0041040 comp rx=0 tx=0).stop 2024-09-10T08:55:50.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.652+0000 7f22ca75c740 1 -- 172.21.15.172:0/3720694507 shutdown_connections 2024-09-10T08:55:50.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.652+0000 7f22ca75c740 1 --2- 172.21.15.172:0/3720694507 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x565075d0dc40 0x565075d10030 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.652+0000 7f22ca75c740 1 --2- 172.21.15.172:0/3720694507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cfe240 0x565075cfbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.652+0000 7f22ca75c740 1 --2- 172.21.15.172:0/3720694507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075cfb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.652+0000 7f22a9ffb640 0 client.27251 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:50.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.652+0000 7f22ca75c740 1 --2- 172.21.15.172:0/3720694507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3060 0x565075cfb130 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.652+0000 7f22ca75c740 1 -- 172.21.15.172:0/3720694507 >> 172.21.15.172:0/3720694507 conn(0x565075cef5f0 msgr2=0x565075cf52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:50.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.652+0000 7f22ca75c740 1 -- 172.21.15.172:0/3720694507 shutdown_connections 2024-09-10T08:55:50.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.652+0000 7f22ca75c740 1 -- 172.21.15.172:0/3720694507 wait complete. 2024-09-10T08:55:50.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.652+0000 7f22ca75c740 20 client.27251 trim_cache size 0 max 16384 2024-09-10T08:55:50.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22ca75c740 1 Processor -- start 2024-09-10T08:55:50.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22ca75c740 1 -- start start 2024-09-10T08:55:50.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22ca75c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cf3060 0x565075d13050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22ca75c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075d13590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22ca75c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cfe240 0x565075cf83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22ca75c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f22c01aeb40 con 0x565075cf3980 2024-09-10T08:55:50.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22ca75c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f22c01b0680 con 0x565075cfe240 2024-09-10T08:55:50.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22ca75c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f22c01af210 con 0x565075cf3060 2024-09-10T08:55:50.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22c611f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075d13590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:50.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22c611f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075d13590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54562/0 (socket says 172.21.15.172:54562) 2024-09-10T08:55:50.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22c611f640 1 -- 172.21.15.172:0/1564665225 learned_addr learned my addr 172.21.15.172:0/1564665225 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:50.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22c591e640 1 --2- 172.21.15.172:0/1564665225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cfe240 0x565075cf83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:50.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22c611f640 1 -- 172.21.15.172:0/1564665225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cf3060 msgr2=0x565075d13050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:50.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22c611f640 1 --2- 172.21.15.172:0/1564665225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cf3060 0x565075d13050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22c611f640 1 -- 172.21.15.172:0/1564665225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cfe240 msgr2=0x565075cf83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:50.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22c611f640 1 --2- 172.21.15.172:0/1564665225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cfe240 0x565075cf83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22c611f640 1 -- 172.21.15.172:0/1564665225 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565075cf6d30 con 0x565075cf3980 2024-09-10T08:55:50.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22c591e640 1 --2- 172.21.15.172:0/1564665225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cfe240 0x565075cf83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:55:50.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22c611f640 1 --2- 172.21.15.172:0/1564665225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075d13590 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x565075cfc770 tx=0x7f22bc00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9b8a0e329dcf43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:50.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.669+0000 7f22abfff640 1 -- 172.21.15.172:0/1564665225 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc00be30 con 0x565075cf3980 2024-09-10T08:55:50.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.670+0000 7f22abfff640 1 -- 172.21.15.172:0/1564665225 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22bc0118f0 con 0x565075cf3980 2024-09-10T08:55:50.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.670+0000 7f22abfff640 1 -- 172.21.15.172:0/1564665225 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc026950 con 0x565075cf3980 2024-09-10T08:55:50.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.670+0000 7f22ca75c740 1 -- 172.21.15.172:0/1564665225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 msgr2=0x565075d13590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:50.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.670+0000 7f22ca75c740 1 --2- 172.21.15.172:0/1564665225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075d13590 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x565075cfc770 tx=0x7f22bc00b390 comp rx=0 tx=0).stop 2024-09-10T08:55:50.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.670+0000 7f22ca75c740 1 -- 172.21.15.172:0/1564665225 shutdown_connections 2024-09-10T08:55:50.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.670+0000 7f22ca75c740 1 --2- 172.21.15.172:0/1564665225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cfe240 0x565075cf83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.670+0000 7f22ca75c740 1 --2- 172.21.15.172:0/1564665225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075d13590 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.670+0000 7f22ca75c740 1 --2- 172.21.15.172:0/1564665225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cf3060 0x565075d13050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.670+0000 7f22ca75c740 1 -- 172.21.15.172:0/1564665225 >> 172.21.15.172:0/1564665225 conn(0x565075cef5f0 msgr2=0x565075cfc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:50.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.670+0000 7f22ca75c740 1 -- 172.21.15.172:0/1564665225 shutdown_connections 2024-09-10T08:55:50.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.670+0000 7f22ca75c740 1 -- 172.21.15.172:0/1564665225 wait complete. 2024-09-10T08:55:50.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.670+0000 7f22ca75c740 1 -- start start 2024-09-10T08:55:50.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.670+0000 7f22ca75c740 1 Processor -- start 2024-09-10T08:55:50.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22ca75c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3060 0x565075d11640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22ca75c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075d11b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22ca75c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cfe240 0x565075d71550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22ca75c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f22c01affb0 con 0x565075cf3980 2024-09-10T08:55:50.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22ca75c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f22c01af8e0 con 0x565075cf3060 2024-09-10T08:55:50.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22ca75c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f22c01acb20 con 0x565075cfe240 2024-09-10T08:55:50.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22ca75c740 10 client.0 _ll_register_callbacks cb 0x7f22b0031770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:50.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22c611f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075d11b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:50.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22c611f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075d11b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:54566/0 (socket says 172.21.15.172:54566) 2024-09-10T08:55:50.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22c611f640 1 -- 172.21.15.172:0/2045002542 learned_addr learned my addr 172.21.15.172:0/2045002542 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:50.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22c591e640 1 --2- 172.21.15.172:0/2045002542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cfe240 0x565075d71550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:50.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22c591e640 1 --2- 172.21.15.172:0/2045002542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3060 0x565075d11640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:50.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22c611f640 1 -- 172.21.15.172:0/2045002542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cfe240 msgr2=0x565075d71550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:50.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22c611f640 1 --2- 172.21.15.172:0/2045002542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cfe240 0x565075d71550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22c611f640 1 -- 172.21.15.172:0/2045002542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3060 msgr2=0x565075d11640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:50.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22c611f640 1 --2- 172.21.15.172:0/2045002542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3060 0x565075d11640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22c611f640 1 -- 172.21.15.172:0/2045002542 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565075d71b90 con 0x565075cf3980 2024-09-10T08:55:50.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22c591e640 1 --2- 172.21.15.172:0/2045002542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3060 0x565075d11640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:55:50.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22c611f640 1 --2- 172.21.15.172:0/2045002542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075d11b80 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00b5f0 tx=0x7f22bc00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1668daf424cf08 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:50.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:50.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22a9ffb640 1 -- 172.21.15.172:0/2045002542 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc011aa0 con 0x565075cf3980 2024-09-10T08:55:50.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22a9ffb640 1 -- 172.21.15.172:0/2045002542 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f22bc011c40 con 0x565075cf3980 2024-09-10T08:55:50.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.671+0000 7f22ca75c740 1 -- 172.21.15.172:0/2045002542 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22bc0118f0 con 0x565075cf3980 2024-09-10T08:55:50.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.672+0000 7f22a9ffb640 1 -- 172.21.15.172:0/2045002542 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f22bc005020 con 0x565075cf3980 2024-09-10T08:55:50.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.672+0000 7f22ca75c740 10 client.18294 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:50.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.672+0000 7f22ca75c740 1 -- 172.21.15.172:0/2045002542 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x565075d78eb0 con 0x565075cf3980 2024-09-10T08:55:50.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.672+0000 7f22a9ffb640 1 -- 172.21.15.172:0/2045002542 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22bc0118f0 con 0x565075cf3980 2024-09-10T08:55:50.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.672+0000 7f22a9ffb640 1 -- 172.21.15.172:0/2045002542 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f22bc02e070 con 0x565075cf3980 2024-09-10T08:55:50.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.672+0000 7f22ca75c740 10 client.18294 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:50.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.672+0000 7f22ca75c740 10 client.18294 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:55:50.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.672+0000 7f22ca75c740 20 client.18294 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:50.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.672+0000 7f22ca75c740 1 --2- 172.21.15.172:0/2045002542 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x565075d79550 0x565075d7b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:50.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.672+0000 7f22ca75c740 4 client.18294 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b923df17", "release": true, "await": true}] 2024-09-10T08:55:50.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.672+0000 7f22ca75c740 1 -- 172.21.15.172:0/2045002542 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b923df17", "release": true, "await": true}) -- 0x565075c600a0 con 0x565075d79550 2024-09-10T08:55:50.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.672+0000 7f22c591e640 1 --2- 172.21.15.172:0/2045002542 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x565075d79550 0x565075d7b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:50.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.673+0000 7f22a9ffb640 10 client.18294 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:50.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.673+0000 7f22c591e640 1 --2- 172.21.15.172:0/2045002542 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x565075d79550 0x565075d7b970 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=614e4ff2b38ddfa7 server_cookie=b53826ef1e2a5529 in_seq=0 out_seq=0 2024-09-10T08:55:50.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.691+0000 7f22a9ffb640 1 -- 172.21.15.172:0/2045002542 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x565075c600a0 con 0x565075d79550 2024-09-10T08:55:50.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.691+0000 7f22a9ffb640 10 client.18294 handle_command_reply: tid=0 2024-09-10T08:55:50.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.691+0000 7f22ca75c740 1 client.18294 shutdown 2024-09-10T08:55:50.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.691+0000 7f22ca75c740 1 -- 172.21.15.172:0/2045002542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 msgr2=0x565075d11b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:50.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.691+0000 7f22ca75c740 1 --2- 172.21.15.172:0/2045002542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075d11b80 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00b5f0 tx=0x7f22bc00bf40 comp rx=0 tx=0).stop 2024-09-10T08:55:50.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.691+0000 7f22ca75c740 1 -- 172.21.15.172:0/2045002542 shutdown_connections 2024-09-10T08:55:50.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.691+0000 7f22ca75c740 1 --2- 172.21.15.172:0/2045002542 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x565075d79550 0x565075d7b970 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.691+0000 7f22ca75c740 1 --2- 172.21.15.172:0/2045002542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565075cfe240 0x565075d71550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.691+0000 7f22ca75c740 1 --2- 172.21.15.172:0/2045002542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565075cf3980 0x565075d11b80 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.691+0000 7f22a9ffb640 0 client.18294 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:50.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.691+0000 7f22ca75c740 1 --2- 172.21.15.172:0/2045002542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565075cf3060 0x565075d11640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.691+0000 7f22ca75c740 1 -- 172.21.15.172:0/2045002542 >> 172.21.15.172:0/2045002542 conn(0x565075cef5f0 msgr2=0x565075cfb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:50.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.691+0000 7f22ca75c740 1 -- 172.21.15.172:0/2045002542 shutdown_connections 2024-09-10T08:55:50.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.691+0000 7f22ca75c740 1 -- 172.21.15.172:0/2045002542 wait complete. 2024-09-10T08:55:50.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.692+0000 7f22ca75c740 20 client.18294 trim_cache size 0 max 16384 2024-09-10T08:55:50.695 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:55:50.695 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 37, 2024-09-10T08:55:50.695 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:55:50.695 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 32, 2024-09-10T08:55:50.695 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:55:50.695 INFO:teuthology.orchestra.run.smithi172.stdout: "b923df17": { 2024-09-10T08:55:50.695 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 32, 2024-09-10T08:55:50.695 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:55:50.695 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:55:50.695 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:55:50.695 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:55:50.696 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:55:50.696 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:55:50.696 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 144.6, 2024-09-10T08:55:50.696 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:55:50.696 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:55:50.696 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:55:50.696 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:55:50.696 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:55:50.696 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:55:50.696 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:55:50.696 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:55:50.696 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:55:50.696 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:55:50.697 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:55:50.697 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:55:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.692+0000 7f22c8383640 1 -- 172.21.15.172:0/2013297495 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22ac0a2b80 msgr2=0x7f22ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.692+0000 7f22c8383640 1 --2- 172.21.15.172:0/2013297495 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22ac0a2b80 0x7f22ac0a5040 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f22b8098400 tx=0x7f22b800cf50 comp rx=0 tx=0).stop 2024-09-10T08:55:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.692+0000 7f22c8383640 1 -- 172.21.15.172:0/2013297495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22c0106ed0 msgr2=0x7f22c019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.692+0000 7f22c8383640 1 --2- 172.21.15.172:0/2013297495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22c0106ed0 0x7f22c019cc30 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f22b800eaa0 tx=0x7f22b800ef60 comp rx=0 tx=0).stop 2024-09-10T08:55:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.692+0000 7f22c8383640 1 -- 172.21.15.172:0/2013297495 shutdown_connections 2024-09-10T08:55:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.692+0000 7f22c8383640 1 --2- 172.21.15.172:0/2013297495 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22ac0a2b80 0x7f22ac0a5040 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.692+0000 7f22c8383640 1 --2- 172.21.15.172:0/2013297495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22c0106ed0 0x7f22c019cc30 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.692+0000 7f22c8383640 1 --2- 172.21.15.172:0/2013297495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22c0100eb0 0x7f22c01988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.692+0000 7f22c8383640 1 --2- 172.21.15.172:0/2013297495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22c0100500 0x7f22c0198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.692+0000 7f22c8383640 1 -- 172.21.15.172:0/2013297495 >> 172.21.15.172:0/2013297495 conn(0x7f22c00fc5c0 msgr2=0x7f22c0105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.692+0000 7f22c8383640 1 -- 172.21.15.172:0/2013297495 shutdown_connections 2024-09-10T08:55:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:55:50.693+0000 7f22c8383640 1 -- 172.21.15.172:0/2013297495 wait complete. 2024-09-10T08:55:50.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:50 smithi174 ceph-mon[34068]: pgmap v638: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-10T08:55:50.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:50 smithi191 ceph-mon[35314]: pgmap v638: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-10T08:55:51.522 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b923df17', total seconds elapsed: 28.0 2024-09-10T08:55:51.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:51 smithi174 ceph-mon[34068]: pgmap v639: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T08:55:51.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:51 smithi191 ceph-mon[35314]: pgmap v639: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T08:55:51.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:51 smithi172 ceph-mon[30854]: pgmap v639: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T08:55:54.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:54 smithi174 ceph-mon[34068]: pgmap v640: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 706 KiB/s wr, 10 op/s 2024-09-10T08:55:54.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:54 smithi191 ceph-mon[35314]: pgmap v640: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 706 KiB/s wr, 10 op/s 2024-09-10T08:55:54.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:54 smithi172 ceph-mon[30854]: pgmap v640: 129 pgs: 129 active+clean; 8.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 706 KiB/s wr, 10 op/s 2024-09-10T08:55:56.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:56 smithi174 ceph-mon[34068]: pgmap v641: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.4 MiB/s wr, 15 op/s 2024-09-10T08:55:56.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:56 smithi191 ceph-mon[35314]: pgmap v641: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.4 MiB/s wr, 15 op/s 2024-09-10T08:55:56.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:56 smithi172 ceph-mon[30854]: pgmap v641: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.4 MiB/s wr, 15 op/s 2024-09-10T08:55:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.977+0000 7f17236e4640 1 -- 172.21.15.172:0/734363358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f171c100a40 msgr2=0x7f171c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.977+0000 7f17236e4640 1 --2- 172.21.15.172:0/734363358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f171c100a40 0x7f171c100e40 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f1718009900 tx=0x7f171802a3f0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.977+0000 7f17236e4640 1 -- 172.21.15.172:0/734363358 shutdown_connections 2024-09-10T08:55:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.977+0000 7f17236e4640 1 --2- 172.21.15.172:0/734363358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f171c1025f0 0x7f171c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.977+0000 7f17236e4640 1 --2- 172.21.15.172:0/734363358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f171c101c30 0x7f171c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.977+0000 7f17236e4640 1 --2- 172.21.15.172:0/734363358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f171c100a40 0x7f171c100e40 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.977+0000 7f17236e4640 1 -- 172.21.15.172:0/734363358 >> 172.21.15.172:0/734363358 conn(0x7f171c0fc1e0 msgr2=0x7f171c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.978+0000 7f17236e4640 1 -- 172.21.15.172:0/734363358 shutdown_connections 2024-09-10T08:55:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.978+0000 7f17236e4640 1 -- 172.21.15.172:0/734363358 wait complete. 2024-09-10T08:55:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.978+0000 7f17236e4640 1 Processor -- start 2024-09-10T08:55:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.978+0000 7f17236e4640 1 -- start start 2024-09-10T08:55:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.978+0000 7f17236e4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f171c100a40 0x7f171c06a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.978+0000 7f17236e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f171c101c30 0x7f171c19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.978+0000 7f17236e4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f171c1025f0 0x7f171c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.978+0000 7f17236e4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f171c10c550 con 0x7f171c1025f0 2024-09-10T08:55:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.978+0000 7f17236e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f171c10c3d0 con 0x7f171c101c30 2024-09-10T08:55:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.978+0000 7f17236e4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f171c10c6d0 con 0x7f171c100a40 2024-09-10T08:55:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.979+0000 7f1721c81640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f171c1025f0 0x7f171c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.979+0000 7f1721480640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f171c100a40 0x7f171c06a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.979+0000 7f1721c81640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f171c1025f0 0x7f171c19f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40830/0 (socket says 172.21.15.172:40830) 2024-09-10T08:55:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.979+0000 7f1721480640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f171c100a40 0x7f171c06a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:38732/0 (socket says 172.21.15.172:38732) 2024-09-10T08:55:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.979+0000 7f1721c81640 1 -- 172.21.15.172:0/4191436945 learned_addr learned my addr 172.21.15.172:0/4191436945 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.979+0000 7f1720c7f640 1 --2- 172.21.15.172:0/4191436945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f171c101c30 0x7f171c19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.979+0000 7f1721c81640 1 -- 172.21.15.172:0/4191436945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f171c100a40 msgr2=0x7f171c06a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.979+0000 7f1721c81640 1 --2- 172.21.15.172:0/4191436945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f171c100a40 0x7f171c06a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.979+0000 7f1721c81640 1 -- 172.21.15.172:0/4191436945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f171c101c30 msgr2=0x7f171c19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.979+0000 7f1721c81640 1 --2- 172.21.15.172:0/4191436945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f171c101c30 0x7f171c19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.979+0000 7f1721c81640 1 -- 172.21.15.172:0/4191436945 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171c19f920 con 0x7f171c1025f0 2024-09-10T08:55:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.979+0000 7f1721c81640 1 --2- 172.21.15.172:0/4191436945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f171c1025f0 0x7f171c19f010 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f171400cce0 tx=0x7f171400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.979+0000 7f17127fc640 1 -- 172.21.15.172:0/4191436945 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171400f450 con 0x7f171c1025f0 2024-09-10T08:55:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.979+0000 7f17236e4640 1 -- 172.21.15.172:0/4191436945 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f171c19fbb0 con 0x7f171c1025f0 2024-09-10T08:55:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.979+0000 7f17127fc640 1 -- 172.21.15.172:0/4191436945 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f171400faa0 con 0x7f171c1025f0 2024-09-10T08:55:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.979+0000 7f17127fc640 1 -- 172.21.15.172:0/4191436945 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714018b70 con 0x7f171c1025f0 2024-09-10T08:55:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.979+0000 7f17236e4640 1 -- 172.21.15.172:0/4191436945 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f171c1a0100 con 0x7f171c1025f0 2024-09-10T08:55:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.982+0000 7f17127fc640 1 -- 172.21.15.172:0/4191436945 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1714018d10 con 0x7f171c1025f0 2024-09-10T08:55:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.982+0000 7f1726485740 1 Processor -- start 2024-09-10T08:55:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.982+0000 7f1726485740 1 -- start start 2024-09-10T08:55:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.982+0000 7f1726485740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634de9a0 0x5560634deda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.982+0000 7f1726485740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634df2e0 0x5560634e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.982+0000 7f1726485740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634e4370 0x5560634e6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.982+0000 7f1726485740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5560634422f0 con 0x5560634df2e0 2024-09-10T08:55:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.982+0000 7f1726485740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5560633ea270 con 0x5560634de9a0 2024-09-10T08:55:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.982+0000 7f1726485740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55606343e2b0 con 0x5560634e4370 2024-09-10T08:55:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.983+0000 7f1721480640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634df2e0 0x5560634e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.983+0000 7f17127fc640 1 --2- 172.21.15.172:0/4191436945 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f17080a2c30 0x7f17080a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.983+0000 7f1721480640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634df2e0 0x5560634e3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40834/0 (socket says 172.21.15.172:40834) 2024-09-10T08:55:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.983+0000 7f1721480640 1 -- 172.21.15.172:0/4291861920 learned_addr learned my addr 172.21.15.172:0/4291861920 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.983+0000 7f1720c7f640 1 --2- 172.21.15.172:0/4291861920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634e4370 0x5560634e6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.983+0000 7f1720c7f640 1 --2- 172.21.15.172:0/4291861920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634de9a0 0x5560634deda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.983+0000 7f17127fc640 1 -- 172.21.15.172:0/4191436945 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f17140cf6c0 con 0x7f171c1025f0 2024-09-10T08:55:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.983+0000 7f1721c81640 1 --2- 172.21.15.172:0/4191436945 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f17080a2c30 0x7f17080a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.983+0000 7f1720c7f640 1 -- 172.21.15.172:0/4291861920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634de9a0 msgr2=0x5560634deda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.983+0000 7f1720c7f640 1 --2- 172.21.15.172:0/4291861920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634de9a0 0x5560634deda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.983+0000 7f1720c7f640 1 -- 172.21.15.172:0/4291861920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634df2e0 msgr2=0x5560634e3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.983+0000 7f1720c7f640 1 --2- 172.21.15.172:0/4291861920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634df2e0 0x5560634e3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.983+0000 7f1720c7f640 1 -- 172.21.15.172:0/4291861920 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560633e9340 con 0x5560634e4370 2024-09-10T08:55:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.983+0000 7f1721480640 1 --2- 172.21.15.172:0/4291861920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634df2e0 0x5560634e3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:55:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.983+0000 7f1720c7f640 1 --2- 172.21.15.172:0/4291861920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634e4370 0x5560634e6830 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f170c00b6c0 tx=0x7f170c00b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5405ba8fb820fa69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.983+0000 7f1721c81640 1 --2- 172.21.15.172:0/4191436945 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f17080a2c30 0x7f17080a50f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f17140cce60 tx=0x7f171400af30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.983+0000 7f17077fe640 1 -- 172.21.15.172:0/4291861920 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c012d40 con 0x5560634e4370 2024-09-10T08:55:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.984+0000 7f17077fe640 1 -- 172.21.15.172:0/4291861920 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f170c005020 con 0x5560634e4370 2024-09-10T08:55:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.984+0000 7f17077fe640 1 -- 172.21.15.172:0/4291861920 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c0049c0 con 0x5560634e4370 2024-09-10T08:55:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.984+0000 7f1726485740 1 -- 172.21.15.172:0/4291861920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634e4370 msgr2=0x5560634e6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.984+0000 7f1726485740 1 --2- 172.21.15.172:0/4291861920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634e4370 0x5560634e6830 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f170c00b6c0 tx=0x7f170c00b650 comp rx=0 tx=0).stop 2024-09-10T08:55:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.984+0000 7f1726485740 1 -- 172.21.15.172:0/4291861920 shutdown_connections 2024-09-10T08:55:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.984+0000 7f1726485740 1 --2- 172.21.15.172:0/4291861920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634e4370 0x5560634e6830 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.984+0000 7f1726485740 1 --2- 172.21.15.172:0/4291861920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634df2e0 0x5560634e3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.984+0000 7f1726485740 1 --2- 172.21.15.172:0/4291861920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634de9a0 0x5560634deda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.984+0000 7f1726485740 1 -- 172.21.15.172:0/4291861920 >> 172.21.15.172:0/4291861920 conn(0x5560634da010 msgr2=0x5560634da410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.984+0000 7f1726485740 1 -- 172.21.15.172:0/4291861920 shutdown_connections 2024-09-10T08:55:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.984+0000 7f1726485740 1 -- 172.21.15.172:0/4291861920 wait complete. 2024-09-10T08:55:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.985+0000 7f1726485740 1 -- start start 2024-09-10T08:55:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.985+0000 7f1726485740 1 Processor -- start 2024-09-10T08:55:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.985+0000 7f1726485740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634de9a0 0x5560634e8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.985+0000 7f1726485740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634df2e0 0x5560634e90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.985+0000 7f1726485740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634e4370 0x5560634f0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.985+0000 7f1726485740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5560634e7340 con 0x5560634df2e0 2024-09-10T08:55:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.985+0000 7f1726485740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5560633cb290 con 0x5560634de9a0 2024-09-10T08:55:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.985+0000 7f1726485740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5560634e74c0 con 0x5560634e4370 2024-09-10T08:55:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.985+0000 7f1726485740 10 client.0 _ll_register_callbacks cb 0x5560633fc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.985+0000 7f1721480640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634df2e0 0x5560634e90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.985+0000 7f1721480640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634df2e0 0x5560634e90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40844/0 (socket says 172.21.15.172:40844) 2024-09-10T08:55:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.985+0000 7f1721480640 1 -- 172.21.15.172:0/4025644312 learned_addr learned my addr 172.21.15.172:0/4025644312 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.985+0000 7f1720c7f640 1 --2- 172.21.15.172:0/4025644312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634e4370 0x5560634f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.986+0000 7f1720c7f640 1 --2- 172.21.15.172:0/4025644312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634de9a0 0x5560634e8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.986+0000 7f1721480640 1 -- 172.21.15.172:0/4025644312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634e4370 msgr2=0x5560634f0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.986+0000 7f1721480640 1 --2- 172.21.15.172:0/4025644312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634e4370 0x5560634f0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.986+0000 7f1721480640 1 -- 172.21.15.172:0/4025644312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634de9a0 msgr2=0x5560634e8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.986+0000 7f1721480640 1 --2- 172.21.15.172:0/4025644312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634de9a0 0x5560634e8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.986+0000 7f1721480640 1 -- 172.21.15.172:0/4025644312 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556063155210 con 0x5560634df2e0 2024-09-10T08:55:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.986+0000 7f1720c7f640 1 --2- 172.21.15.172:0/4025644312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634de9a0 0x5560634e8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:55:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.986+0000 7f1721480640 1 --2- 172.21.15.172:0/4025644312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634df2e0 0x5560634e90d0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f17180314e0 tx=0x7f17180317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92344afa057dfbc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.986+0000 7f1704ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.986+0000 7f1704ff9640 1 -- 172.21.15.172:0/4025644312 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718035040 con 0x5560634df2e0 2024-09-10T08:55:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.986+0000 7f1704ff9640 1 -- 172.21.15.172:0/4025644312 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1718004450 con 0x5560634df2e0 2024-09-10T08:55:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.986+0000 7f1726485740 1 -- 172.21.15.172:0/4025644312 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f170c005020 con 0x5560634df2e0 2024-09-10T08:55:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.986+0000 7f1704ff9640 1 -- 172.21.15.172:0/4025644312 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f171804e050 con 0x5560634df2e0 2024-09-10T08:55:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.986+0000 7f1726485740 10 client.18306 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.986+0000 7f1726485740 1 -- 172.21.15.172:0/4025644312 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556063221620 con 0x5560634df2e0 2024-09-10T08:55:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.987+0000 7f1704ff9640 1 -- 172.21.15.172:0/4025644312 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f170c005020 con 0x5560634df2e0 2024-09-10T08:55:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.987+0000 7f1704ff9640 1 -- 172.21.15.172:0/4025644312 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f171803c700 con 0x5560634df2e0 2024-09-10T08:55:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.987+0000 7f1726485740 10 client.18306 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.987+0000 7f1726485740 10 client.18306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.987+0000 7f1726485740 20 client.18306 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.987+0000 7f1726485740 1 --2- 172.21.15.172:0/4025644312 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5560634f4820 0x5560634f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.987+0000 7f1726485740 4 client.18306 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:55:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.987+0000 7f1726485740 1 -- 172.21.15.172:0/4025644312 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55606344fa20 con 0x5560634f4820 2024-09-10T08:55:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.987+0000 7f1720c7f640 1 --2- 172.21.15.172:0/4025644312 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5560634f4820 0x5560634f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.988+0000 7f1704ff9640 10 client.18306 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.987+0000 7f1720c7f640 1 --2- 172.21.15.172:0/4025644312 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5560634f4820 0x5560634f6c50 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef2bd6ba919148e0 server_cookie=d7677029b24f5795 in_seq=0 out_seq=0 2024-09-10T08:55:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.988+0000 7f1704ff9640 1 -- 172.21.15.172:0/4025644312 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55606344fa20 con 0x5560634f4820 2024-09-10T08:55:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.988+0000 7f1704ff9640 10 client.18306 handle_command_reply: tid=0 2024-09-10T08:55:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.988+0000 7f1726485740 1 client.18306 shutdown 2024-09-10T08:55:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.988+0000 7f1726485740 1 -- 172.21.15.172:0/4025644312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634df2e0 msgr2=0x5560634e90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.988+0000 7f1726485740 1 --2- 172.21.15.172:0/4025644312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634df2e0 0x5560634e90d0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f17180314e0 tx=0x7f17180317b0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.988+0000 7f1726485740 1 -- 172.21.15.172:0/4025644312 shutdown_connections 2024-09-10T08:55:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.988+0000 7f1726485740 1 --2- 172.21.15.172:0/4025644312 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5560634f4820 0x5560634f6c50 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.988+0000 7f1726485740 1 --2- 172.21.15.172:0/4025644312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634e4370 0x5560634f0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.988+0000 7f1726485740 1 --2- 172.21.15.172:0/4025644312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634df2e0 0x5560634e90d0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.988+0000 7f1726485740 1 --2- 172.21.15.172:0/4025644312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634de9a0 0x5560634e8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.988+0000 7f1726485740 1 -- 172.21.15.172:0/4025644312 >> 172.21.15.172:0/4025644312 conn(0x5560634da010 msgr2=0x5560634db820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.988+0000 7f1704ff9640 0 client.18306 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.989+0000 7f1726485740 1 -- 172.21.15.172:0/4025644312 shutdown_connections 2024-09-10T08:55:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.989+0000 7f1726485740 1 -- 172.21.15.172:0/4025644312 wait complete. 2024-09-10T08:55:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:56.989+0000 7f1726485740 20 client.18306 trim_cache size 0 max 16384 2024-09-10T08:55:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.006+0000 7f1726485740 1 Processor -- start 2024-09-10T08:55:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.006+0000 7f1726485740 1 -- start start 2024-09-10T08:55:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.006+0000 7f1726485740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634de9a0 0x5560634db400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.006+0000 7f1726485740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634df2e0 0x5560634f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.006+0000 7f1726485740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634e4370 0x5560634db940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.006+0000 7f1726485740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f171c1aa420 con 0x5560634de9a0 2024-09-10T08:55:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.006+0000 7f1726485740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f171c1abf60 con 0x5560634e4370 2024-09-10T08:55:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.006+0000 7f1726485740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f171c1aaaf0 con 0x5560634df2e0 2024-09-10T08:55:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.006+0000 7f1720c7f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634de9a0 0x5560634db400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.007+0000 7f1720c7f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634de9a0 0x5560634db400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40858/0 (socket says 172.21.15.172:40858) 2024-09-10T08:55:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.007+0000 7f1720c7f640 1 -- 172.21.15.172:0/2409746409 learned_addr learned my addr 172.21.15.172:0/2409746409 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.007+0000 7f1720c7f640 1 --2- 172.21.15.172:0/2409746409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634e4370 0x5560634db940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.007+0000 7f1721480640 1 --2- 172.21.15.172:0/2409746409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634df2e0 0x5560634f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.007+0000 7f1720c7f640 1 -- 172.21.15.172:0/2409746409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634df2e0 msgr2=0x5560634f00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.007+0000 7f1720c7f640 1 --2- 172.21.15.172:0/2409746409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634df2e0 0x5560634f00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.007+0000 7f1720c7f640 1 -- 172.21.15.172:0/2409746409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634e4370 msgr2=0x5560634db940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.007+0000 7f1720c7f640 1 --2- 172.21.15.172:0/2409746409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634e4370 0x5560634db940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.007+0000 7f1720c7f640 1 -- 172.21.15.172:0/2409746409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560634e6ce0 con 0x5560634de9a0 2024-09-10T08:55:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.007+0000 7f1720c7f640 1 --2- 172.21.15.172:0/2409746409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634de9a0 0x5560634db400 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f170c005d00 tx=0x7f170c00bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94293ddf6e97a4a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.007+0000 7f1706ffd640 1 -- 172.21.15.172:0/2409746409 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c005590 con 0x5560634de9a0 2024-09-10T08:55:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.007+0000 7f1706ffd640 1 -- 172.21.15.172:0/2409746409 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f170c010bf0 con 0x5560634de9a0 2024-09-10T08:55:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.007+0000 7f1706ffd640 1 -- 172.21.15.172:0/2409746409 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c02f7b0 con 0x5560634de9a0 2024-09-10T08:55:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.008+0000 7f1726485740 1 -- 172.21.15.172:0/2409746409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634de9a0 msgr2=0x5560634db400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.008+0000 7f1726485740 1 --2- 172.21.15.172:0/2409746409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634de9a0 0x5560634db400 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f170c005d00 tx=0x7f170c00bc50 comp rx=0 tx=0).stop 2024-09-10T08:55:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.008+0000 7f1726485740 1 -- 172.21.15.172:0/2409746409 shutdown_connections 2024-09-10T08:55:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.008+0000 7f1726485740 1 --2- 172.21.15.172:0/2409746409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634e4370 0x5560634db940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.008+0000 7f1726485740 1 --2- 172.21.15.172:0/2409746409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634df2e0 0x5560634f00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.008+0000 7f1726485740 1 --2- 172.21.15.172:0/2409746409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634de9a0 0x5560634db400 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.008+0000 7f1726485740 1 -- 172.21.15.172:0/2409746409 >> 172.21.15.172:0/2409746409 conn(0x5560634da010 msgr2=0x5560634d91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.008+0000 7f1726485740 1 -- 172.21.15.172:0/2409746409 shutdown_connections 2024-09-10T08:55:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.008+0000 7f1726485740 1 -- 172.21.15.172:0/2409746409 wait complete. 2024-09-10T08:55:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.008+0000 7f1726485740 1 -- start start 2024-09-10T08:55:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.008+0000 7f1726485740 1 Processor -- start 2024-09-10T08:55:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1726485740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634de9a0 0x5560634ed7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1726485740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634df2e0 0x5560634edd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1726485740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634e4370 0x5560634eed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1726485740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f171c1ab890 con 0x5560634de9a0 2024-09-10T08:55:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1726485740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f171c1ab1c0 con 0x5560634e4370 2024-09-10T08:55:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1726485740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f171c1a8400 con 0x5560634df2e0 2024-09-10T08:55:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1726485740 10 client.0 _ll_register_callbacks cb 0x7f1718031d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:55:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1720c7f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634de9a0 0x5560634ed7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1720c7f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634de9a0 0x5560634ed7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40866/0 (socket says 172.21.15.172:40866) 2024-09-10T08:55:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1720c7f640 1 -- 172.21.15.172:0/700907797 learned_addr learned my addr 172.21.15.172:0/700907797 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:55:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1720c7f640 1 --2- 172.21.15.172:0/700907797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634e4370 0x5560634eed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1721480640 1 --2- 172.21.15.172:0/700907797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634df2e0 0x5560634edd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1720c7f640 1 -- 172.21.15.172:0/700907797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634df2e0 msgr2=0x5560634edd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1720c7f640 1 --2- 172.21.15.172:0/700907797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634df2e0 0x5560634edd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1720c7f640 1 -- 172.21.15.172:0/700907797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634e4370 msgr2=0x5560634eed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:55:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1720c7f640 1 --2- 172.21.15.172:0/700907797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634e4370 0x5560634eed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1720c7f640 1 -- 172.21.15.172:0/700907797 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560634ef320 con 0x5560634de9a0 2024-09-10T08:55:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1720c7f640 1 --2- 172.21.15.172:0/700907797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634de9a0 0x5560634ed7c0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f170c00b690 tx=0x7f170c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28f782987f9c370 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:55:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.009+0000 7f1704ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:55:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.010+0000 7f1704ff9640 1 -- 172.21.15.172:0/700907797 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c0054d0 con 0x5560634de9a0 2024-09-10T08:55:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.010+0000 7f1726485740 1 -- 172.21.15.172:0/700907797 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f170c010bf0 con 0x5560634de9a0 2024-09-10T08:55:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.010+0000 7f1704ff9640 1 -- 172.21.15.172:0/700907797 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f170c010740 con 0x5560634de9a0 2024-09-10T08:55:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.010+0000 7f1704ff9640 1 -- 172.21.15.172:0/700907797 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f170c010940 con 0x5560634de9a0 2024-09-10T08:55:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.010+0000 7f1726485740 10 client.18312 fetch_fsmap learned FSMap version 37 2024-09-10T08:55:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.010+0000 7f1726485740 1 -- 172.21.15.172:0/700907797 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55606355f530 con 0x5560634de9a0 2024-09-10T08:55:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.010+0000 7f1704ff9640 1 -- 172.21.15.172:0/700907797 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f170c010bf0 con 0x5560634de9a0 2024-09-10T08:55:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.010+0000 7f1704ff9640 1 -- 172.21.15.172:0/700907797 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f170c0046f0 con 0x5560634de9a0 2024-09-10T08:55:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.010+0000 7f1726485740 10 client.18312 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:55:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.010+0000 7f1726485740 10 client.18312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:55:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.010+0000 7f1726485740 20 client.18312 populate_metadata read hostname 'smithi172' 2024-09-10T08:55:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.010+0000 7f1726485740 1 --2- 172.21.15.172:0/700907797 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55606355fc50 0x556063562070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:55:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.010+0000 7f1726485740 4 client.18312 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:55:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.010+0000 7f1726485740 1 -- 172.21.15.172:0/700907797 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556063449d70 con 0x55606355fc50 2024-09-10T08:55:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.010+0000 7f1721480640 1 --2- 172.21.15.172:0/700907797 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55606355fc50 0x556063562070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:55:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.011+0000 7f1704ff9640 10 client.18312 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.011+0000 7f1721480640 1 --2- 172.21.15.172:0/700907797 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55606355fc50 0x556063562070 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9caab9c7a66d780f server_cookie=a6958bd6adceb9c in_seq=0 out_seq=0 2024-09-10T08:55:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.014+0000 7f1704ff9640 1 -- 172.21.15.172:0/700907797 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556063449d70 con 0x55606355fc50 2024-09-10T08:55:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.014+0000 7f1704ff9640 10 client.18312 handle_command_reply: tid=0 2024-09-10T08:55:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.015+0000 7f1726485740 1 client.18312 shutdown 2024-09-10T08:55:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.015+0000 7f1726485740 1 -- 172.21.15.172:0/700907797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634de9a0 msgr2=0x5560634ed7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.015+0000 7f1726485740 1 --2- 172.21.15.172:0/700907797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634de9a0 0x5560634ed7c0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f170c00b690 tx=0x7f170c005850 comp rx=0 tx=0).stop 2024-09-10T08:55:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.015+0000 7f1726485740 1 -- 172.21.15.172:0/700907797 shutdown_connections 2024-09-10T08:55:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.015+0000 7f1726485740 1 --2- 172.21.15.172:0/700907797 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55606355fc50 0x556063562070 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.015+0000 7f1726485740 1 --2- 172.21.15.172:0/700907797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560634e4370 0x5560634eed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.015+0000 7f1726485740 1 --2- 172.21.15.172:0/700907797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560634df2e0 0x5560634edd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.015+0000 7f1704ff9640 0 client.18312 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:55:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.015+0000 7f1726485740 1 --2- 172.21.15.172:0/700907797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560634de9a0 0x5560634ed7c0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.015+0000 7f1726485740 1 -- 172.21.15.172:0/700907797 >> 172.21.15.172:0/700907797 conn(0x5560634da010 msgr2=0x5560634da3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.015+0000 7f1726485740 1 -- 172.21.15.172:0/700907797 shutdown_connections 2024-09-10T08:55:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.015+0000 7f1726485740 1 -- 172.21.15.172:0/700907797 wait complete. 2024-09-10T08:55:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.015+0000 7f1726485740 20 client.18312 trim_cache size 0 max 16384 2024-09-10T08:55:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:55:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:55:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:55:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:55:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.015+0000 7f17236e4640 1 -- 172.21.15.172:0/4191436945 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f17080a2c30 msgr2=0x7f17080a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.015+0000 7f17236e4640 1 --2- 172.21.15.172:0/4191436945 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f17080a2c30 0x7f17080a50f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f17140cce60 tx=0x7f171400af30 comp rx=0 tx=0).stop 2024-09-10T08:55:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.015+0000 7f17236e4640 1 -- 172.21.15.172:0/4191436945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f171c1025f0 msgr2=0x7f171c19f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:55:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.015+0000 7f17236e4640 1 --2- 172.21.15.172:0/4191436945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f171c1025f0 0x7f171c19f010 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f171400cce0 tx=0x7f171400a620 comp rx=0 tx=0).stop 2024-09-10T08:55:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.016+0000 7f17236e4640 1 -- 172.21.15.172:0/4191436945 shutdown_connections 2024-09-10T08:55:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.016+0000 7f17236e4640 1 --2- 172.21.15.172:0/4191436945 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f17080a2c30 0x7f17080a50f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.016+0000 7f17236e4640 1 --2- 172.21.15.172:0/4191436945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f171c1025f0 0x7f171c19f010 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.016+0000 7f17236e4640 1 --2- 172.21.15.172:0/4191436945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f171c101c30 0x7f171c19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.016+0000 7f17236e4640 1 --2- 172.21.15.172:0/4191436945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f171c100a40 0x7f171c06a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:55:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.016+0000 7f17236e4640 1 -- 172.21.15.172:0/4191436945 >> 172.21.15.172:0/4191436945 conn(0x7f171c0fc1e0 msgr2=0x7f171c104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:55:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.016+0000 7f17236e4640 1 -- 172.21.15.172:0/4191436945 shutdown_connections 2024-09-10T08:55:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:55:57.016+0000 7f17236e4640 1 -- 172.21.15.172:0/4191436945 wait complete. 2024-09-10T08:55:57.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:55:57 smithi174 ceph-mon[34068]: pgmap v642: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 MiB/s wr, 13 op/s 2024-09-10T08:55:57.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:55:57 smithi191 ceph-mon[35314]: pgmap v642: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 MiB/s wr, 13 op/s 2024-09-10T08:55:57.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:55:57 smithi172 ceph-mon[30854]: pgmap v642: 129 pgs: 129 active+clean; 8.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 MiB/s wr, 13 op/s 2024-09-10T08:55:59.579 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:55:59.869 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:56:00.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.377+0000 7f3e8b258640 1 -- 172.21.15.172:0/545905325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e84106f20 msgr2=0x7f3e8410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:00.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.377+0000 7f3e8b258640 1 --2- 172.21.15.172:0/545905325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e84106f20 0x7f3e8410eaa0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009ca0 tx=0x7f3e7c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.377+0000 7f3e8b258640 1 -- 172.21.15.172:0/545905325 shutdown_connections 2024-09-10T08:56:00.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.377+0000 7f3e8b258640 1 --2- 172.21.15.172:0/545905325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e84106f20 0x7f3e8410eaa0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.377+0000 7f3e8b258640 1 --2- 172.21.15.172:0/545905325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e841027a0 0x7f3e841067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.377+0000 7f3e8b258640 1 --2- 172.21.15.172:0/545905325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e84101df0 0x7f3e841021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.377+0000 7f3e8b258640 1 -- 172.21.15.172:0/545905325 >> 172.21.15.172:0/545905325 conn(0x7f3e840fd8f0 msgr2=0x7f3e840ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:00.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.377+0000 7f3e8b258640 1 -- 172.21.15.172:0/545905325 shutdown_connections 2024-09-10T08:56:00.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.377+0000 7f3e8b258640 1 -- 172.21.15.172:0/545905325 wait complete. 2024-09-10T08:56:00.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.378+0000 7f3e8b258640 1 Processor -- start 2024-09-10T08:56:00.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.378+0000 7f3e8b258640 1 -- start start 2024-09-10T08:56:00.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.378+0000 7f3e8b258640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e84101df0 0x7f3e840744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.378+0000 7f3e8b258640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e841027a0 0x7f3e84074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.378+0000 7f3e8b258640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e84106f20 0x7f3e84074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.378+0000 7f3e8b258640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3e84111e30 con 0x7f3e84101df0 2024-09-10T08:56:00.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.378+0000 7f3e8b258640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3e84111cb0 con 0x7f3e84106f20 2024-09-10T08:56:00.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.378+0000 7f3e8b258640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3e84111fb0 con 0x7f3e841027a0 2024-09-10T08:56:00.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.378+0000 7f3e88ff4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e84101df0 0x7f3e840744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:00.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.378+0000 7f3e88ff4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e84101df0 0x7f3e840744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40892/0 (socket says 172.21.15.172:40892) 2024-09-10T08:56:00.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.378+0000 7f3e88ff4640 1 -- 172.21.15.172:0/3634055525 learned_addr learned my addr 172.21.15.172:0/3634055525 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:00.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.378+0000 7f3e897f5640 1 --2- 172.21.15.172:0/3634055525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e84106f20 0x7f3e84074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:00.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.379+0000 7f3e88ff4640 1 -- 172.21.15.172:0/3634055525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e841027a0 msgr2=0x7f3e84074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:00.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.379+0000 7f3e88ff4640 1 --2- 172.21.15.172:0/3634055525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e841027a0 0x7f3e84074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.379+0000 7f3e88ff4640 1 -- 172.21.15.172:0/3634055525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e84106f20 msgr2=0x7f3e84074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:00.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.379+0000 7f3e88ff4640 1 --2- 172.21.15.172:0/3634055525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e84106f20 0x7f3e84074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.379+0000 7f3e88ff4640 1 -- 172.21.15.172:0/3634055525 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e841a6810 con 0x7f3e84101df0 2024-09-10T08:56:00.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.379+0000 7f3e88ff4640 1 --2- 172.21.15.172:0/3634055525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e84101df0 0x7f3e840744f0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e9f0 tx=0x7f3e7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:00.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.379+0000 7f3e81ffb640 1 -- 172.21.15.172:0/3634055525 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7800cd80 con 0x7f3e84101df0 2024-09-10T08:56:00.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.379+0000 7f3e81ffb640 1 -- 172.21.15.172:0/3634055525 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e780108e0 con 0x7f3e84101df0 2024-09-10T08:56:00.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.379+0000 7f3e8b258640 1 -- 172.21.15.172:0/3634055525 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e841a6b00 con 0x7f3e84101df0 2024-09-10T08:56:00.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.379+0000 7f3e81ffb640 1 -- 172.21.15.172:0/3634055525 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78018920 con 0x7f3e84101df0 2024-09-10T08:56:00.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.379+0000 7f3e8b258640 1 -- 172.21.15.172:0/3634055525 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e841a7090 con 0x7f3e84101df0 2024-09-10T08:56:00.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.382+0000 7f3e81ffb640 1 -- 172.21.15.172:0/3634055525 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3e78010a90 con 0x7f3e84101df0 2024-09-10T08:56:00.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.383+0000 7f3e8d631740 1 Processor -- start 2024-09-10T08:56:00.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.383+0000 7f3e8d631740 1 -- start start 2024-09-10T08:56:00.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.383+0000 7f3e8d631740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3df24a0 0x5577f3df2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.383+0000 7f3e8d631740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df2dc0 0x5577f3dfd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.383+0000 7f3e8d631740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3dfd680 0x5577f3dffa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.383+0000 7f3e8d631740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5577f3d582e0 con 0x5577f3df2dc0 2024-09-10T08:56:00.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.383+0000 7f3e8d631740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5577f3cffce0 con 0x5577f3df24a0 2024-09-10T08:56:00.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.383+0000 7f3e8d631740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5577f3d542a0 con 0x5577f3dfd680 2024-09-10T08:56:00.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.383+0000 7f3e81ffb640 1 --2- 172.21.15.172:0/3634055525 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3e740a2c00 0x7f3e740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.383+0000 7f3e897f5640 1 --2- 172.21.15.172:0/3634055525 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3e740a2c00 0x7f3e740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:00.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.383+0000 7f3e88ff4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df2dc0 0x5577f3dfd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:00.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.383+0000 7f3e88ff4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df2dc0 0x5577f3dfd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40894/0 (socket says 172.21.15.172:40894) 2024-09-10T08:56:00.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.383+0000 7f3e88ff4640 1 -- 172.21.15.172:0/3261317812 learned_addr learned my addr 172.21.15.172:0/3261317812 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:00.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.383+0000 7f3e897f5640 1 --2- 172.21.15.172:0/3261317812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3df24a0 0x5577f3df2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:00.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.383+0000 7f3e83fff640 1 --2- 172.21.15.172:0/3261317812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3dfd680 0x5577f3dffa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:00.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.383+0000 7f3e81ffb640 1 -- 172.21.15.172:0/3634055525 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3e780cf680 con 0x7f3e84101df0 2024-09-10T08:56:00.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.384+0000 7f3e88ff4640 1 -- 172.21.15.172:0/3261317812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3dfd680 msgr2=0x5577f3dffa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:00.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.384+0000 7f3e88ff4640 1 --2- 172.21.15.172:0/3261317812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3dfd680 0x5577f3dffa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.384+0000 7f3e88ff4640 1 -- 172.21.15.172:0/3261317812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3df24a0 msgr2=0x5577f3df2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:00.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.384+0000 7f3e88ff4640 1 --2- 172.21.15.172:0/3261317812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3df24a0 0x5577f3df2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.384+0000 7f3e88ff4640 1 -- 172.21.15.172:0/3261317812 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577f3ce4440 con 0x5577f3df2dc0 2024-09-10T08:56:00.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.384+0000 7f3e897f5640 1 --2- 172.21.15.172:0/3634055525 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3e740a2c00 0x7f3e740a50c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009430 tx=0x7f3e7c009950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:00.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.384+0000 7f3e88ff4640 1 --2- 172.21.15.172:0/3261317812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df2dc0 0x5577f3dfd140 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3e780cce20 tx=0x7f3e78091060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403d46c2eced4436 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:00.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.384+0000 7f3e6effd640 1 -- 172.21.15.172:0/3261317812 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7809c280 con 0x5577f3df2dc0 2024-09-10T08:56:00.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.384+0000 7f3e6effd640 1 -- 172.21.15.172:0/3261317812 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e7809bb80 con 0x5577f3df2dc0 2024-09-10T08:56:00.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.384+0000 7f3e6effd640 1 -- 172.21.15.172:0/3261317812 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7809bde0 con 0x5577f3df2dc0 2024-09-10T08:56:00.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.385+0000 7f3e8d631740 1 -- 172.21.15.172:0/3261317812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df2dc0 msgr2=0x5577f3dfd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:00.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.385+0000 7f3e8d631740 1 --2- 172.21.15.172:0/3261317812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df2dc0 0x5577f3dfd140 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3e780cce20 tx=0x7f3e78091060 comp rx=0 tx=0).stop 2024-09-10T08:56:00.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.385+0000 7f3e8d631740 1 -- 172.21.15.172:0/3261317812 shutdown_connections 2024-09-10T08:56:00.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.385+0000 7f3e8d631740 1 --2- 172.21.15.172:0/3261317812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3dfd680 0x5577f3dffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.385+0000 7f3e8d631740 1 --2- 172.21.15.172:0/3261317812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df2dc0 0x5577f3dfd140 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.385+0000 7f3e8d631740 1 --2- 172.21.15.172:0/3261317812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3df24a0 0x5577f3df2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.385+0000 7f3e8d631740 1 -- 172.21.15.172:0/3261317812 >> 172.21.15.172:0/3261317812 conn(0x5577f3deea30 msgr2=0x5577f3deee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:00.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.385+0000 7f3e8d631740 1 -- 172.21.15.172:0/3261317812 shutdown_connections 2024-09-10T08:56:00.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.385+0000 7f3e8d631740 1 -- 172.21.15.172:0/3261317812 wait complete. 2024-09-10T08:56:00.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.386+0000 7f3e8d631740 1 -- start start 2024-09-10T08:56:00.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.386+0000 7f3e8d631740 1 Processor -- start 2024-09-10T08:56:00.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.386+0000 7f3e8d631740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3df24a0 0x5577f3dfd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.386+0000 7f3e8d631740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df2dc0 0x5577f3dfa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.386+0000 7f3e8d631740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3dfd680 0x5577f3dfa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.386+0000 7f3e8d631740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5577f3e08a60 con 0x5577f3df2dc0 2024-09-10T08:56:00.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.386+0000 7f3e8d631740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5577f3d13050 con 0x5577f3df24a0 2024-09-10T08:56:00.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.386+0000 7f3e8d631740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5577f3e08be0 con 0x5577f3dfd680 2024-09-10T08:56:00.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e83fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df2dc0 0x5577f3dfa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:00.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e8d631740 10 client.0 _ll_register_callbacks cb 0x5577f3d53160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:00.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e83fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df2dc0 0x5577f3dfa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40902/0 (socket says 172.21.15.172:40902) 2024-09-10T08:56:00.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e83fff640 1 -- 172.21.15.172:0/202377814 learned_addr learned my addr 172.21.15.172:0/202377814 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:00.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e88ff4640 1 --2- 172.21.15.172:0/202377814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3df24a0 0x5577f3dfd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:00.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e897f5640 1 --2- 172.21.15.172:0/202377814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3dfd680 0x5577f3dfa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:00.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e88ff4640 1 -- 172.21.15.172:0/202377814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3dfd680 msgr2=0x5577f3dfa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:00.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e88ff4640 1 --2- 172.21.15.172:0/202377814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3dfd680 0x5577f3dfa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e88ff4640 1 -- 172.21.15.172:0/202377814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df2dc0 msgr2=0x5577f3dfa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:00.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e88ff4640 1 --2- 172.21.15.172:0/202377814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df2dc0 0x5577f3dfa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e88ff4640 1 -- 172.21.15.172:0/202377814 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577f3ac93c0 con 0x5577f3df24a0 2024-09-10T08:56:00.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e88ff4640 1 --2- 172.21.15.172:0/202377814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3df24a0 0x5577f3dfd010 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5577f3def890 tx=0x7f3e780a4870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ce76aa9667e1ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:00.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e63fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:56:00.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e63fff640 1 -- 172.21.15.172:0/202377814 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7809ff30 con 0x5577f3df24a0 2024-09-10T08:56:00.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e63fff640 1 -- 172.21.15.172:0/202377814 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3e7806b170 con 0x5577f3df24a0 2024-09-10T08:56:00.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e8d631740 1 -- 172.21.15.172:0/202377814 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e7809bb80 con 0x5577f3df24a0 2024-09-10T08:56:00.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e63fff640 1 -- 172.21.15.172:0/202377814 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3e7809bde0 con 0x5577f3df24a0 2024-09-10T08:56:00.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.387+0000 7f3e8d631740 10 client.27281 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:00.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.388+0000 7f3e8d631740 1 -- 172.21.15.172:0/202377814 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5577f3e14480 con 0x5577f3df24a0 2024-09-10T08:56:00.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.388+0000 7f3e63fff640 1 -- 172.21.15.172:0/202377814 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e7809bb80 con 0x5577f3df24a0 2024-09-10T08:56:00.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.388+0000 7f3e63fff640 1 -- 172.21.15.172:0/202377814 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f3e780a4360 con 0x5577f3df24a0 2024-09-10T08:56:00.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.388+0000 7f3e8d631740 10 client.27281 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:00.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.388+0000 7f3e8d631740 10 client.27281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:00.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.388+0000 7f3e8d631740 20 client.27281 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:00.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.388+0000 7f3e8d631740 1 --2- 172.21.15.172:0/202377814 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5577f3e157d0 0x5577f3e17bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.388+0000 7f3e8d631740 4 client.27281 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:56:00.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.388+0000 7f3e8d631740 1 -- 172.21.15.172:0/202377814 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577f3d65850 con 0x5577f3e157d0 2024-09-10T08:56:00.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.388+0000 7f3e83fff640 1 --2- 172.21.15.172:0/202377814 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5577f3e157d0 0x5577f3e17bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:00.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.389+0000 7f3e63fff640 10 client.27281 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:00.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.389+0000 7f3e83fff640 1 --2- 172.21.15.172:0/202377814 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5577f3e157d0 0x5577f3e17bc0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=177dc7f5a7af08dc server_cookie=2849ab2f57e53535 in_seq=0 out_seq=0 2024-09-10T08:56:00.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.389+0000 7f3e63fff640 1 -- 172.21.15.172:0/202377814 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577f3d65850 con 0x5577f3e157d0 2024-09-10T08:56:00.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.389+0000 7f3e63fff640 10 client.27281 handle_command_reply: tid=0 2024-09-10T08:56:00.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.389+0000 7f3e8d631740 1 client.27281 shutdown 2024-09-10T08:56:00.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.389+0000 7f3e8d631740 1 -- 172.21.15.172:0/202377814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3df24a0 msgr2=0x5577f3dfd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:00.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.389+0000 7f3e8d631740 1 --2- 172.21.15.172:0/202377814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3df24a0 0x5577f3dfd010 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5577f3def890 tx=0x7f3e780a4870 comp rx=0 tx=0).stop 2024-09-10T08:56:00.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.390+0000 7f3e8d631740 1 -- 172.21.15.172:0/202377814 shutdown_connections 2024-09-10T08:56:00.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.390+0000 7f3e8d631740 1 --2- 172.21.15.172:0/202377814 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5577f3e157d0 0x5577f3e17bc0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.390+0000 7f3e8d631740 1 --2- 172.21.15.172:0/202377814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3dfd680 0x5577f3dfa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.390+0000 7f3e8d631740 1 --2- 172.21.15.172:0/202377814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df2dc0 0x5577f3dfa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.390+0000 7f3e63fff640 0 client.27281 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:00.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.390+0000 7f3e8d631740 1 --2- 172.21.15.172:0/202377814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3df24a0 0x5577f3dfd010 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.390+0000 7f3e8d631740 1 -- 172.21.15.172:0/202377814 >> 172.21.15.172:0/202377814 conn(0x5577f3deea30 msgr2=0x5577f3df4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:00.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.390+0000 7f3e8d631740 1 -- 172.21.15.172:0/202377814 shutdown_connections 2024-09-10T08:56:00.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.390+0000 7f3e8d631740 1 -- 172.21.15.172:0/202377814 wait complete. 2024-09-10T08:56:00.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.390+0000 7f3e8d631740 20 client.27281 trim_cache size 0 max 16384 2024-09-10T08:56:00.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.407+0000 7f3e8d631740 1 Processor -- start 2024-09-10T08:56:00.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.407+0000 7f3e8d631740 1 -- start start 2024-09-10T08:56:00.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.408+0000 7f3e8d631740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df24a0 0x5577f3e1abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.408+0000 7f3e8d631740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3df2dc0 0x5577f3e1b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.408+0000 7f3e8d631740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3dfd680 0x5577f3df7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.408+0000 7f3e8d631740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841ac970 con 0x5577f3df24a0 2024-09-10T08:56:00.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.408+0000 7f3e8d631740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841ae4b0 con 0x5577f3dfd680 2024-09-10T08:56:00.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.408+0000 7f3e8d631740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841ad040 con 0x5577f3df2dc0 2024-09-10T08:56:00.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.408+0000 7f3e88ff4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df24a0 0x5577f3e1abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:00.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.408+0000 7f3e88ff4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df24a0 0x5577f3e1abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40904/0 (socket says 172.21.15.172:40904) 2024-09-10T08:56:00.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.408+0000 7f3e88ff4640 1 -- 172.21.15.172:0/1282028806 learned_addr learned my addr 172.21.15.172:0/1282028806 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:00.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.408+0000 7f3e83fff640 1 --2- 172.21.15.172:0/1282028806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3df2dc0 0x5577f3e1b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:00.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.408+0000 7f3e897f5640 1 --2- 172.21.15.172:0/1282028806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3dfd680 0x5577f3df7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:00.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.408+0000 7f3e88ff4640 1 -- 172.21.15.172:0/1282028806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3df2dc0 msgr2=0x5577f3e1b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:00.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.408+0000 7f3e88ff4640 1 --2- 172.21.15.172:0/1282028806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3df2dc0 0x5577f3e1b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.408+0000 7f3e88ff4640 1 -- 172.21.15.172:0/1282028806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3dfd680 msgr2=0x5577f3df7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:00.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.408+0000 7f3e88ff4640 1 --2- 172.21.15.172:0/1282028806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3dfd680 0x5577f3df7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.408+0000 7f3e88ff4640 1 -- 172.21.15.172:0/1282028806 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577f3dee4d0 con 0x5577f3df24a0 2024-09-10T08:56:00.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.408+0000 7f3e88ff4640 1 --2- 172.21.15.172:0/1282028806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df24a0 0x5577f3e1abe0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f3e780cd190 tx=0x7f3e7809b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427a7f5de696f450 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:00.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.409+0000 7f3e6e7fc640 1 -- 172.21.15.172:0/1282028806 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7809bb80 con 0x5577f3df24a0 2024-09-10T08:56:00.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.409+0000 7f3e6e7fc640 1 -- 172.21.15.172:0/1282028806 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e78099e80 con 0x5577f3df24a0 2024-09-10T08:56:00.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.409+0000 7f3e6e7fc640 1 -- 172.21.15.172:0/1282028806 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7809c070 con 0x5577f3df24a0 2024-09-10T08:56:00.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.409+0000 7f3e8d631740 1 -- 172.21.15.172:0/1282028806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df24a0 msgr2=0x5577f3e1abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:00.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.409+0000 7f3e8d631740 1 --2- 172.21.15.172:0/1282028806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df24a0 0x5577f3e1abe0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f3e780cd190 tx=0x7f3e7809b410 comp rx=0 tx=0).stop 2024-09-10T08:56:00.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.410+0000 7f3e8d631740 1 -- 172.21.15.172:0/1282028806 shutdown_connections 2024-09-10T08:56:00.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.410+0000 7f3e8d631740 1 --2- 172.21.15.172:0/1282028806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3dfd680 0x5577f3df7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.410+0000 7f3e8d631740 1 --2- 172.21.15.172:0/1282028806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3df2dc0 0x5577f3e1b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.410+0000 7f3e8d631740 1 --2- 172.21.15.172:0/1282028806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3df24a0 0x5577f3e1abe0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.410+0000 7f3e8d631740 1 -- 172.21.15.172:0/1282028806 >> 172.21.15.172:0/1282028806 conn(0x5577f3deea30 msgr2=0x5577f3dfc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:00.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.410+0000 7f3e8d631740 1 -- 172.21.15.172:0/1282028806 shutdown_connections 2024-09-10T08:56:00.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.410+0000 7f3e8d631740 1 -- 172.21.15.172:0/1282028806 wait complete. 2024-09-10T08:56:00.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.410+0000 7f3e8d631740 1 -- start start 2024-09-10T08:56:00.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.410+0000 7f3e8d631740 1 Processor -- start 2024-09-10T08:56:00.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e8d631740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3df24a0 0x5577f3e18ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e8d631740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3df2dc0 0x5577f3e18ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e8d631740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3dfd680 0x5577f3e19530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e8d631740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841adde0 con 0x5577f3dfd680 2024-09-10T08:56:00.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e8d631740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841ad710 con 0x5577f3df24a0 2024-09-10T08:56:00.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e8d631740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3e841aa950 con 0x5577f3df2dc0 2024-09-10T08:56:00.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e83fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3df2dc0 0x5577f3e18ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:00.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e83fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3df2dc0 0x5577f3e18ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:38820/0 (socket says 172.21.15.172:38820) 2024-09-10T08:56:00.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e83fff640 1 -- 172.21.15.172:0/230084138 learned_addr learned my addr 172.21.15.172:0/230084138 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:00.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e88ff4640 1 --2- 172.21.15.172:0/230084138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3df24a0 0x5577f3e18ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:00.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e897f5640 1 --2- 172.21.15.172:0/230084138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3dfd680 0x5577f3e19530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:00.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e8d631740 10 client.0 _ll_register_callbacks cb 0x7f3e7c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:00.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e83fff640 1 -- 172.21.15.172:0/230084138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3df24a0 msgr2=0x5577f3e18ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:00.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e83fff640 1 --2- 172.21.15.172:0/230084138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3df24a0 0x5577f3e18ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e83fff640 1 -- 172.21.15.172:0/230084138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3dfd680 msgr2=0x5577f3e19530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:00.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e83fff640 1 --2- 172.21.15.172:0/230084138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3dfd680 0x5577f3e19530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e83fff640 1 -- 172.21.15.172:0/230084138 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577f3e7f210 con 0x5577f3df2dc0 2024-09-10T08:56:00.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e88ff4640 1 --2- 172.21.15.172:0/230084138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3df24a0 0x5577f3e18ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:00.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e83fff640 1 --2- 172.21.15.172:0/230084138 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3df2dc0 0x5577f3e18ff0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5577f3df9b50 tx=0x7f3e70013830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=481a6285d80c1ba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:00.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.411+0000 7f3e63fff640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:56:00.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.412+0000 7f3e63fff640 1 -- 172.21.15.172:0/230084138 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70005230 con 0x5577f3df2dc0 2024-09-10T08:56:00.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.412+0000 7f3e63fff640 1 -- 172.21.15.172:0/230084138 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3e700053d0 con 0x5577f3df2dc0 2024-09-10T08:56:00.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.412+0000 7f3e8d631740 1 -- 172.21.15.172:0/230084138 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e78099e80 con 0x5577f3df2dc0 2024-09-10T08:56:00.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.412+0000 7f3e63fff640 1 -- 172.21.15.172:0/230084138 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3e700055f0 con 0x5577f3df2dc0 2024-09-10T08:56:00.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.412+0000 7f3e8d631740 10 client.27235 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:00.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.412+0000 7f3e8d631740 1 -- 172.21.15.172:0/230084138 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5577f3e80570 con 0x5577f3df2dc0 2024-09-10T08:56:00.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.412+0000 7f3e63fff640 1 -- 172.21.15.172:0/230084138 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e70004c50 con 0x5577f3df2dc0 2024-09-10T08:56:00.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.412+0000 7f3e63fff640 1 -- 172.21.15.172:0/230084138 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f3e70021020 con 0x5577f3df2dc0 2024-09-10T08:56:00.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.412+0000 7f3e8d631740 10 client.27235 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:00.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.412+0000 7f3e8d631740 10 client.27235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:00.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.412+0000 7f3e8d631740 20 client.27235 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:00.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.412+0000 7f3e8d631740 1 --2- 172.21.15.172:0/230084138 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5577f3e157d0 0x5577f3e82d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:00.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.412+0000 7f3e8d631740 4 client.27235 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:56:00.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.412+0000 7f3e8d631740 1 -- 172.21.15.172:0/230084138 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f3e78099e80 con 0x5577f3e157d0 2024-09-10T08:56:00.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.413+0000 7f3e88ff4640 1 --2- 172.21.15.172:0/230084138 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5577f3e157d0 0x5577f3e82d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:00.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.413+0000 7f3e88ff4640 1 --2- 172.21.15.172:0/230084138 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5577f3e157d0 0x5577f3e82d60 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36937e2c4033f9a5 server_cookie=386f0745a0d1a96c in_seq=0 out_seq=0 2024-09-10T08:56:00.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.413+0000 7f3e63fff640 10 client.27235 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:00.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.413+0000 7f3e63fff640 1 -- 172.21.15.172:0/230084138 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f3e78099e80 con 0x5577f3e157d0 2024-09-10T08:56:00.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.413+0000 7f3e63fff640 10 client.27235 handle_command_reply: tid=0 2024-09-10T08:56:00.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.413+0000 7f3e8d631740 1 client.27235 shutdown 2024-09-10T08:56:00.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.413+0000 7f3e8d631740 1 -- 172.21.15.172:0/230084138 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3df2dc0 msgr2=0x5577f3e18ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:00.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.413+0000 7f3e8d631740 1 --2- 172.21.15.172:0/230084138 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3df2dc0 0x5577f3e18ff0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5577f3df9b50 tx=0x7f3e70013830 comp rx=0 tx=0).stop 2024-09-10T08:56:00.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.413+0000 7f3e8d631740 1 -- 172.21.15.172:0/230084138 shutdown_connections 2024-09-10T08:56:00.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.413+0000 7f3e8d631740 1 --2- 172.21.15.172:0/230084138 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5577f3e157d0 0x5577f3e82d60 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.413+0000 7f3e8d631740 1 --2- 172.21.15.172:0/230084138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5577f3dfd680 0x5577f3e19530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.413+0000 7f3e8d631740 1 --2- 172.21.15.172:0/230084138 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577f3df2dc0 0x5577f3e18ff0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.413+0000 7f3e8d631740 1 --2- 172.21.15.172:0/230084138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5577f3df24a0 0x5577f3e18ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.413+0000 7f3e8d631740 1 -- 172.21.15.172:0/230084138 >> 172.21.15.172:0/230084138 conn(0x5577f3deea30 msgr2=0x5577f3ded350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:00.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.413+0000 7f3e63fff640 0 client.27235 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:00.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.414+0000 7f3e8d631740 1 -- 172.21.15.172:0/230084138 shutdown_connections 2024-09-10T08:56:00.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.414+0000 7f3e8d631740 1 -- 172.21.15.172:0/230084138 wait complete. 2024-09-10T08:56:00.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.414+0000 7f3e8d631740 20 client.27235 trim_cache size 0 max 16384 2024-09-10T08:56:00.423 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:56:00.423 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T08:56:00.423 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T08:56:00.423 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:56:00.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.414+0000 7f3e8b258640 1 -- 172.21.15.172:0/3634055525 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3e740a2c00 msgr2=0x7f3e740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:00.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.414+0000 7f3e8b258640 1 --2- 172.21.15.172:0/3634055525 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3e740a2c00 0x7f3e740a50c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009430 tx=0x7f3e7c009950 comp rx=0 tx=0).stop 2024-09-10T08:56:00.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.414+0000 7f3e8b258640 1 -- 172.21.15.172:0/3634055525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e84101df0 msgr2=0x7f3e840744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:00.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.414+0000 7f3e8b258640 1 --2- 172.21.15.172:0/3634055525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e84101df0 0x7f3e840744f0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e9f0 tx=0x7f3e7800eeb0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.414+0000 7f3e8b258640 1 -- 172.21.15.172:0/3634055525 shutdown_connections 2024-09-10T08:56:00.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.414+0000 7f3e8b258640 1 --2- 172.21.15.172:0/3634055525 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3e740a2c00 0x7f3e740a50c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.414+0000 7f3e8b258640 1 --2- 172.21.15.172:0/3634055525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e84106f20 0x7f3e84074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.414+0000 7f3e8b258640 1 --2- 172.21.15.172:0/3634055525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e841027a0 0x7f3e84074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.414+0000 7f3e8b258640 1 --2- 172.21.15.172:0/3634055525 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e84101df0 0x7f3e840744f0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:00.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.414+0000 7f3e8b258640 1 -- 172.21.15.172:0/3634055525 >> 172.21.15.172:0/3634055525 conn(0x7f3e840fd8f0 msgr2=0x7f3e840fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:00.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.415+0000 7f3e8b258640 1 -- 172.21.15.172:0/3634055525 shutdown_connections 2024-09-10T08:56:00.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:00.415+0000 7f3e8b258640 1 -- 172.21.15.172:0/3634055525 wait complete. 2024-09-10T08:56:00.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:00 smithi172 ceph-mon[30854]: pgmap v643: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 17 MiB/s wr, 17 op/s 2024-09-10T08:56:00.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:00 smithi174 ceph-mon[34068]: pgmap v643: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 17 MiB/s wr, 17 op/s 2024-09-10T08:56:00.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:00 smithi191 ceph-mon[35314]: pgmap v643: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 17 MiB/s wr, 17 op/s 2024-09-10T08:56:01.254 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:56:01.255 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:56:01.576 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:56:02.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.137+0000 7f3649654640 1 -- 172.21.15.172:0/1423324388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3644102570 msgr2=0x7f36441029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:02.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.137+0000 7f3649654640 1 --2- 172.21.15.172:0/1423324388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3644102570 0x7f36441029b0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f3638009900 tx=0x7f363802a480 comp rx=0 tx=0).stop 2024-09-10T08:56:02.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.137+0000 7f3649654640 1 -- 172.21.15.172:0/1423324388 shutdown_connections 2024-09-10T08:56:02.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.137+0000 7f3649654640 1 --2- 172.21.15.172:0/1423324388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3644102ef0 0x7f364410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:02.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.137+0000 7f3649654640 1 --2- 172.21.15.172:0/1423324388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3644102570 0x7f36441029b0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:02.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.137+0000 7f3649654640 1 --2- 172.21.15.172:0/1423324388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3644100e20 0x7f3644101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:02.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.137+0000 7f3649654640 1 -- 172.21.15.172:0/1423324388 >> 172.21.15.172:0/1423324388 conn(0x7f36440fc7d0 msgr2=0x7f36440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:02.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.137+0000 7f3649654640 1 -- 172.21.15.172:0/1423324388 shutdown_connections 2024-09-10T08:56:02.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.137+0000 7f3649654640 1 -- 172.21.15.172:0/1423324388 wait complete. 2024-09-10T08:56:02.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.138+0000 7f3649654640 1 Processor -- start 2024-09-10T08:56:02.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.138+0000 7f3649654640 1 -- start start 2024-09-10T08:56:02.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.139+0000 7f3649654640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3644100e20 0x7f364419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:02.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.139+0000 7f3649654640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3644102570 0x7f364419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:02.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.139+0000 7f3649654640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3644102ef0 0x7f364419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:02.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.139+0000 7f3649654640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3644110e80 con 0x7f3644102570 2024-09-10T08:56:02.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.139+0000 7f3649654640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3644110d00 con 0x7f3644100e20 2024-09-10T08:56:02.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.139+0000 7f3649654640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3644111000 con 0x7f3644102ef0 2024-09-10T08:56:02.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.139+0000 7f3642ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3644100e20 0x7f364419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:02.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.139+0000 7f36427fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3644102570 0x7f364419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:02.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.139+0000 7f3642ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3644100e20 0x7f364419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:58046/0 (socket says 172.21.15.172:58046) 2024-09-10T08:56:02.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.139+0000 7f3642ffd640 1 -- 172.21.15.172:0/438760927 learned_addr learned my addr 172.21.15.172:0/438760927 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:02.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.139+0000 7f36437fe640 1 --2- 172.21.15.172:0/438760927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3644102ef0 0x7f364419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:02.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.139+0000 7f3642ffd640 1 -- 172.21.15.172:0/438760927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3644102ef0 msgr2=0x7f364419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:02.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.139+0000 7f3642ffd640 1 --2- 172.21.15.172:0/438760927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3644102ef0 0x7f364419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:02.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.139+0000 7f3642ffd640 1 -- 172.21.15.172:0/438760927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3644102570 msgr2=0x7f364419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:02.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.139+0000 7f3642ffd640 1 --2- 172.21.15.172:0/438760927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3644102570 0x7f364419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:02.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.139+0000 7f3642ffd640 1 -- 172.21.15.172:0/438760927 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364419f5f0 con 0x7f3644100e20 2024-09-10T08:56:02.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.139+0000 7f3642ffd640 1 --2- 172.21.15.172:0/438760927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3644100e20 0x7f364419a640 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f363400d770 tx=0x7f363400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:02.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.140+0000 7f362bfff640 1 -- 172.21.15.172:0/438760927 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634017070 con 0x7f3644100e20 2024-09-10T08:56:02.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.140+0000 7f3649654640 1 -- 172.21.15.172:0/438760927 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364419f8e0 con 0x7f3644100e20 2024-09-10T08:56:02.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.140+0000 7f362bfff640 1 -- 172.21.15.172:0/438760927 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36340054d0 con 0x7f3644100e20 2024-09-10T08:56:02.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.140+0000 7f362bfff640 1 -- 172.21.15.172:0/438760927 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634004d50 con 0x7f3644100e20 2024-09-10T08:56:02.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.140+0000 7f3649654640 1 -- 172.21.15.172:0/438760927 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36441a71c0 con 0x7f3644100e20 2024-09-10T08:56:02.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.141+0000 7f3649654640 1 -- 172.21.15.172:0/438760927 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36441021b0 con 0x7f3644100e20 2024-09-10T08:56:02.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.142+0000 7f362bfff640 1 -- 172.21.15.172:0/438760927 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3634005020 con 0x7f3644100e20 2024-09-10T08:56:02.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.143+0000 7f362bfff640 1 --2- 172.21.15.172:0/438760927 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f362c0a2c00 0x7f362c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:02.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.143+0000 7f36427fc640 1 --2- 172.21.15.172:0/438760927 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f362c0a2c00 0x7f362c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:02.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.143+0000 7f362bfff640 1 -- 172.21.15.172:0/438760927 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f36340cee80 con 0x7f3644100e20 2024-09-10T08:56:02.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.144+0000 7f36427fc640 1 --2- 172.21.15.172:0/438760927 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f362c0a2c00 0x7f362c0a50c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3638000c00 tx=0x7f3638005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:02.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.146+0000 7f362bfff640 1 -- 172.21.15.172:0/438760927 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f363409baa0 con 0x7f3644100e20 2024-09-10T08:56:02.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.395+0000 7f3649654640 1 -- 172.21.15.172:0/438760927 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3644101200 con 0x7f3644100e20 2024-09-10T08:56:02.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.396+0000 7f362bfff640 1 -- 172.21.15.172:0/438760927 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+12001 (secure 0 0 0) 0x7f3644101200 con 0x7f3644100e20 2024-09-10T08:56:02.397 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:56:02.397 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":37,"btime":"2024-09-10T08:52:08:313493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:52:07.658341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:56:02.397 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 37 2024-09-10T08:56:02.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.398+0000 7f3649654640 1 -- 172.21.15.172:0/438760927 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f362c0a2c00 msgr2=0x7f362c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:02.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.398+0000 7f3649654640 1 --2- 172.21.15.172:0/438760927 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f362c0a2c00 0x7f362c0a50c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3638000c00 tx=0x7f3638005e80 comp rx=0 tx=0).stop 2024-09-10T08:56:02.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.398+0000 7f3649654640 1 -- 172.21.15.172:0/438760927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3644100e20 msgr2=0x7f364419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:02.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.398+0000 7f3649654640 1 --2- 172.21.15.172:0/438760927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3644100e20 0x7f364419a640 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f363400d770 tx=0x7f363400dc30 comp rx=0 tx=0).stop 2024-09-10T08:56:02.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.398+0000 7f3649654640 1 -- 172.21.15.172:0/438760927 shutdown_connections 2024-09-10T08:56:02.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.398+0000 7f3649654640 1 --2- 172.21.15.172:0/438760927 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f362c0a2c00 0x7f362c0a50c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:02.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.398+0000 7f3649654640 1 --2- 172.21.15.172:0/438760927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3644102ef0 0x7f364419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:02.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.398+0000 7f3649654640 1 --2- 172.21.15.172:0/438760927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3644102570 0x7f364419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:02.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.398+0000 7f3649654640 1 --2- 172.21.15.172:0/438760927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3644100e20 0x7f364419a640 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:02.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.398+0000 7f3649654640 1 -- 172.21.15.172:0/438760927 >> 172.21.15.172:0/438760927 conn(0x7f36440fc7d0 msgr2=0x7f364406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:02.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.399+0000 7f3649654640 1 -- 172.21.15.172:0/438760927 shutdown_connections 2024-09-10T08:56:02.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:02.399+0000 7f3649654640 1 -- 172.21.15.172:0/438760927 wait complete. 2024-09-10T08:56:02.716 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:02 smithi172 ceph-mon[30854]: pgmap v644: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 21 MiB/s wr, 19 op/s 2024-09-10T08:56:02.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:02 smithi174 ceph-mon[34068]: pgmap v644: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 21 MiB/s wr, 19 op/s 2024-09-10T08:56:02.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:02 smithi191 ceph-mon[35314]: pgmap v644: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 21 MiB/s wr, 19 op/s 2024-09-10T08:56:03.170 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T08:56:03.453 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:56:03.577 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:03 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/438760927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:56:03.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:03 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/438760927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:56:03.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:03 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/438760927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:56:03.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.985+0000 7fb5dee33640 1 -- 172.21.15.172:0/3009496165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d8102570 msgr2=0x7fb5d81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:03.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.985+0000 7fb5dee33640 1 --2- 172.21.15.172:0/3009496165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d8102570 0x7fb5d81029b0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009900 tx=0x7fb5c402a480 comp rx=0 tx=0).stop 2024-09-10T08:56:03.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.985+0000 7fb5dee33640 1 -- 172.21.15.172:0/3009496165 shutdown_connections 2024-09-10T08:56:03.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.985+0000 7fb5dee33640 1 --2- 172.21.15.172:0/3009496165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5d8102ef0 0x7fb5d810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:03.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.985+0000 7fb5dee33640 1 --2- 172.21.15.172:0/3009496165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d8102570 0x7fb5d81029b0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:03.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.985+0000 7fb5dee33640 1 --2- 172.21.15.172:0/3009496165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5d8100e20 0x7fb5d8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:03.989 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.985+0000 7fb5dee33640 1 -- 172.21.15.172:0/3009496165 >> 172.21.15.172:0/3009496165 conn(0x7fb5d80fc7d0 msgr2=0x7fb5d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:03.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.985+0000 7fb5dee33640 1 -- 172.21.15.172:0/3009496165 shutdown_connections 2024-09-10T08:56:03.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.985+0000 7fb5dee33640 1 -- 172.21.15.172:0/3009496165 wait complete. 2024-09-10T08:56:03.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.986+0000 7fb5dee33640 1 Processor -- start 2024-09-10T08:56:03.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.986+0000 7fb5dee33640 1 -- start start 2024-09-10T08:56:03.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.986+0000 7fb5dee33640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d8100e20 0x7fb5d81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:03.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.986+0000 7fb5dee33640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5d8102570 0x7fb5d8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:03.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5dee33640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5d8102ef0 0x7fb5d819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:03.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5dee33640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d8110ed0 con 0x7fb5d8100e20 2024-09-10T08:56:03.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5dee33640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d8110d50 con 0x7fb5d8102ef0 2024-09-10T08:56:03.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5dee33640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d8111050 con 0x7fb5d8102570 2024-09-10T08:56:03.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5dcbcf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d8100e20 0x7fb5d81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:03.990 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5dcbcf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d8100e20 0x7fb5d81985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40950/0 (socket says 172.21.15.172:40950) 2024-09-10T08:56:03.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5dcbcf640 1 -- 172.21.15.172:0/2644123781 learned_addr learned my addr 172.21.15.172:0/2644123781 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:03.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5d7fff640 1 --2- 172.21.15.172:0/2644123781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5d8102570 0x7fb5d8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:03.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5dd3d0640 1 --2- 172.21.15.172:0/2644123781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5d8102ef0 0x7fb5d819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:03.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5dcbcf640 1 -- 172.21.15.172:0/2644123781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5d8102570 msgr2=0x7fb5d8198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:03.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5dcbcf640 1 --2- 172.21.15.172:0/2644123781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5d8102570 0x7fb5d8198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:03.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5dcbcf640 1 -- 172.21.15.172:0/2644123781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5d8102ef0 msgr2=0x7fb5d819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:03.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5dcbcf640 1 --2- 172.21.15.172:0/2644123781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5d8102ef0 0x7fb5d819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:03.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5dcbcf640 1 -- 172.21.15.172:0/2644123781 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d819d560 con 0x7fb5d8100e20 2024-09-10T08:56:03.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5dcbcf640 1 --2- 172.21.15.172:0/2644123781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d8100e20 0x7fb5d81985b0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00b4a0 tx=0x7fb5cc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:03.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5d5ffb640 1 -- 172.21.15.172:0/2644123781 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc014070 con 0x7fb5d8100e20 2024-09-10T08:56:03.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5d5ffb640 1 -- 172.21.15.172:0/2644123781 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5cc010ac0 con 0x7fb5d8100e20 2024-09-10T08:56:03.991 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5dee33640 1 -- 172.21.15.172:0/2644123781 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d819d850 con 0x7fb5d8100e20 2024-09-10T08:56:03.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.987+0000 7fb5d5ffb640 1 -- 172.21.15.172:0/2644123781 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc017c30 con 0x7fb5d8100e20 2024-09-10T08:56:03.992 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.988+0000 7fb5dee33640 1 -- 172.21.15.172:0/2644123781 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d81a5130 con 0x7fb5d8100e20 2024-09-10T08:56:03.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.991+0000 7fb5d5ffb640 1 -- 172.21.15.172:0/2644123781 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb5cc017070 con 0x7fb5d8100e20 2024-09-10T08:56:03.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.991+0000 7fb5e120c740 1 Processor -- start 2024-09-10T08:56:03.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.991+0000 7fb5e120c740 1 -- start start 2024-09-10T08:56:03.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.991+0000 7fb5e120c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad005b40 0x556cad004090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:03.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.991+0000 7fb5e120c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad0045d0 0x556cad004a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:03.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.991+0000 7fb5e120c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad00f370 0x556cad011760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:03.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.991+0000 7fb5e120c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556cacf6a2f0 con 0x556cad0045d0 2024-09-10T08:56:03.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.991+0000 7fb5e120c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556cacf11cf0 con 0x556cad00f370 2024-09-10T08:56:03.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.991+0000 7fb5e120c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556cacf662b0 con 0x556cad005b40 2024-09-10T08:56:03.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.991+0000 7fb5dcbcf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad0045d0 0x556cad004a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:03.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.991+0000 7fb5dcbcf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad0045d0 0x556cad004a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40960/0 (socket says 172.21.15.172:40960) 2024-09-10T08:56:03.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.991+0000 7fb5dcbcf640 1 -- 172.21.15.172:0/874644350 learned_addr learned my addr 172.21.15.172:0/874644350 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:03.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.992+0000 7fb5d7fff640 1 --2- 172.21.15.172:0/874644350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad00f370 0x556cad011760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:03.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.992+0000 7fb5dd3d0640 1 --2- 172.21.15.172:0/874644350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad005b40 0x556cad004090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:03.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.992+0000 7fb5d5ffb640 1 --2- 172.21.15.172:0/2644123781 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb5c80a2c00 0x7fb5c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:03.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.992+0000 7fb5d5ffb640 1 -- 172.21.15.172:0/2644123781 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb5cc0ce9e0 con 0x7fb5d8100e20 2024-09-10T08:56:03.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.992+0000 7fb5dcbcf640 1 -- 172.21.15.172:0/874644350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad005b40 msgr2=0x556cad004090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:03.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.992+0000 7fb5dcbcf640 1 --2- 172.21.15.172:0/874644350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad005b40 0x556cad004090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:03.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.992+0000 7fb5dcbcf640 1 -- 172.21.15.172:0/874644350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad00f370 msgr2=0x556cad011760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:03.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.992+0000 7fb5dcbcf640 1 --2- 172.21.15.172:0/874644350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad00f370 0x556cad011760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:03.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.992+0000 7fb5dcbcf640 1 -- 172.21.15.172:0/874644350 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cacef63a0 con 0x556cad0045d0 2024-09-10T08:56:03.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.992+0000 7fb5dd3d0640 1 --2- 172.21.15.172:0/2644123781 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb5c80a2c00 0x7fb5c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:03.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.992+0000 7fb5dcbcf640 1 --2- 172.21.15.172:0/874644350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad0045d0 0x556cad004a10 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc097340 tx=0x7fb5cc00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4bae4afe10ff80b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:03.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.992+0000 7fb5dd3d0640 1 --2- 172.21.15.172:0/2644123781 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb5c80a2c00 0x7fb5c80a50c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0010fd0 tx=0x7fb5d0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:03.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.993+0000 7fb5c2ffd640 1 -- 172.21.15.172:0/874644350 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc01e070 con 0x556cad0045d0 2024-09-10T08:56:03.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.993+0000 7fb5c2ffd640 1 -- 172.21.15.172:0/874644350 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5cc09b5e0 con 0x556cad0045d0 2024-09-10T08:56:03.996 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.993+0000 7fb5c2ffd640 1 -- 172.21.15.172:0/874644350 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc097550 con 0x556cad0045d0 2024-09-10T08:56:03.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.993+0000 7fb5e120c740 1 -- 172.21.15.172:0/874644350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad0045d0 msgr2=0x556cad004a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:04.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.993+0000 7fb5e120c740 1 --2- 172.21.15.172:0/874644350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad0045d0 0x556cad004a10 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc097340 tx=0x7fb5cc00bf90 comp rx=0 tx=0).stop 2024-09-10T08:56:04.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.994+0000 7fb5e120c740 1 -- 172.21.15.172:0/874644350 shutdown_connections 2024-09-10T08:56:04.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.994+0000 7fb5e120c740 1 --2- 172.21.15.172:0/874644350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad00f370 0x556cad011760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.994+0000 7fb5e120c740 1 --2- 172.21.15.172:0/874644350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad0045d0 0x556cad004a10 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.994+0000 7fb5e120c740 1 --2- 172.21.15.172:0/874644350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad005b40 0x556cad004090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.994+0000 7fb5e120c740 1 -- 172.21.15.172:0/874644350 >> 172.21.15.172:0/874644350 conn(0x556cacfcd310 msgr2=0x556cad000c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:04.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.994+0000 7fb5e120c740 1 -- 172.21.15.172:0/874644350 shutdown_connections 2024-09-10T08:56:04.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.994+0000 7fb5e120c740 1 -- 172.21.15.172:0/874644350 wait complete. 2024-09-10T08:56:04.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.994+0000 7fb5e120c740 1 -- start start 2024-09-10T08:56:04.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.994+0000 7fb5e120c740 1 Processor -- start 2024-09-10T08:56:04.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.994+0000 7fb5e120c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad0045d0 0x556cad00edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:04.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5e120c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad005b40 0x556cad00c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:04.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5e120c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad00f370 0x556cad00c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:04.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5e120c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556cad012070 con 0x556cad005b40 2024-09-10T08:56:04.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5e120c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556cacf25060 con 0x556cad0045d0 2024-09-10T08:56:04.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5e120c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556cad0121f0 con 0x556cad00f370 2024-09-10T08:56:04.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad005b40 0x556cad00c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:04.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5e120c740 10 client.0 _ll_register_callbacks cb 0x556cacc788c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:04.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad005b40 0x556cad00c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40968/0 (socket says 172.21.15.172:40968) 2024-09-10T08:56:04.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5d7fff640 1 -- 172.21.15.172:0/2335103954 learned_addr learned my addr 172.21.15.172:0/2335103954 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:04.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5dcbcf640 1 --2- 172.21.15.172:0/2335103954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad0045d0 0x556cad00edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:04.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5dd3d0640 1 --2- 172.21.15.172:0/2335103954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad00f370 0x556cad00c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:04.001 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5d7fff640 1 -- 172.21.15.172:0/2335103954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad00f370 msgr2=0x556cad00c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:04.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5d7fff640 1 --2- 172.21.15.172:0/2335103954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad00f370 0x556cad00c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5d7fff640 1 -- 172.21.15.172:0/2335103954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad0045d0 msgr2=0x556cad00edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:04.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5d7fff640 1 --2- 172.21.15.172:0/2335103954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad0045d0 0x556cad00edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5d7fff640 1 -- 172.21.15.172:0/2335103954 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556caccdb3c0 con 0x556cad005b40 2024-09-10T08:56:04.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5dcbcf640 1 --2- 172.21.15.172:0/2335103954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad0045d0 0x556cad00edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:04.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5d7fff640 1 --2- 172.21.15.172:0/2335103954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad005b40 0x556cad00c080 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fb5c402a9a0 tx=0x7fb5c4031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e055ecb750fbca25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:04.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:04.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.995+0000 7fb5b7fff640 1 -- 172.21.15.172:0/2335103954 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4035040 con 0x556cad005b40 2024-09-10T08:56:04.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.996+0000 7fb5e120c740 1 -- 172.21.15.172:0/2335103954 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5cc09b5e0 con 0x556cad005b40 2024-09-10T08:56:04.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.996+0000 7fb5b7fff640 1 -- 172.21.15.172:0/2335103954 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb5c4004250 con 0x556cad005b40 2024-09-10T08:56:04.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.996+0000 7fb5b7fff640 1 -- 172.21.15.172:0/2335103954 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb5c4004450 con 0x556cad005b40 2024-09-10T08:56:04.002 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.996+0000 7fb5e120c740 10 client.18348 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:04.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.996+0000 7fb5e120c740 1 -- 172.21.15.172:0/2335103954 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556cad01dbe0 con 0x556cad005b40 2024-09-10T08:56:04.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.996+0000 7fb5b7fff640 1 -- 172.21.15.172:0/2335103954 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5cc09b5e0 con 0x556cad005b40 2024-09-10T08:56:04.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.996+0000 7fb5b7fff640 1 -- 172.21.15.172:0/2335103954 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fb5c4044070 con 0x556cad005b40 2024-09-10T08:56:04.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.996+0000 7fb5e120c740 10 client.18348 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:04.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.996+0000 7fb5e120c740 10 client.18348 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:56:04.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.996+0000 7fb5e120c740 20 client.18348 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:04.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.996+0000 7fb5e120c740 1 --2- 172.21.15.172:0/2335103954 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x556cad01eff0 0x556cad021400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:04.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.996+0000 7fb5e120c740 4 client.18348 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:56:04.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.996+0000 7fb5e120c740 1 -- 172.21.15.172:0/2335103954 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cacf77860 con 0x556cad01eff0 2024-09-10T08:56:04.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.997+0000 7fb5dcbcf640 1 --2- 172.21.15.172:0/2335103954 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x556cad01eff0 0x556cad021400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:04.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.997+0000 7fb5dcbcf640 1 --2- 172.21.15.172:0/2335103954 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x556cad01eff0 0x556cad021400 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=849c39df46c82b07 server_cookie=3049b96baf325067 in_seq=0 out_seq=0 2024-09-10T08:56:04.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.997+0000 7fb5b7fff640 10 client.18348 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:56:04.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.998+0000 7fb5b7fff640 1 -- 172.21.15.172:0/2335103954 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556cacf77860 con 0x556cad01eff0 2024-09-10T08:56:04.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.998+0000 7fb5b7fff640 10 client.18348 handle_command_reply: tid=0 2024-09-10T08:56:04.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.998+0000 7fb5e120c740 1 client.18348 shutdown 2024-09-10T08:56:04.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.998+0000 7fb5e120c740 1 -- 172.21.15.172:0/2335103954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad005b40 msgr2=0x556cad00c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:04.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.998+0000 7fb5e120c740 1 --2- 172.21.15.172:0/2335103954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad005b40 0x556cad00c080 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fb5c402a9a0 tx=0x7fb5c4031660 comp rx=0 tx=0).stop 2024-09-10T08:56:04.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.998+0000 7fb5e120c740 1 -- 172.21.15.172:0/2335103954 shutdown_connections 2024-09-10T08:56:04.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.998+0000 7fb5e120c740 1 --2- 172.21.15.172:0/2335103954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad00f370 0x556cad00c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.998+0000 7fb5e120c740 1 --2- 172.21.15.172:0/2335103954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad005b40 0x556cad00c080 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.998+0000 7fb5e120c740 1 --2- 172.21.15.172:0/2335103954 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x556cad01eff0 0x556cad021400 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.998+0000 7fb5e120c740 1 --2- 172.21.15.172:0/2335103954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad0045d0 0x556cad00edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.998+0000 7fb5e120c740 1 -- 172.21.15.172:0/2335103954 >> 172.21.15.172:0/2335103954 conn(0x556cacfcd310 msgr2=0x556cad006990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:04.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.998+0000 7fb5b7fff640 0 client.18348 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:56:04.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.998+0000 7fb5e120c740 1 -- 172.21.15.172:0/2335103954 shutdown_connections 2024-09-10T08:56:04.004 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.999+0000 7fb5e120c740 1 -- 172.21.15.172:0/2335103954 wait complete. 2024-09-10T08:56:04.005 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:03.999+0000 7fb5e120c740 20 client.18348 trim_cache size 0 max 16384 2024-09-10T08:56:04.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.013+0000 7fb5e120c740 1 Processor -- start 2024-09-10T08:56:04.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.013+0000 7fb5e120c740 1 -- start start 2024-09-10T08:56:04.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.013+0000 7fb5e120c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad0045d0 0x556cad00b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:04.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.013+0000 7fb5e120c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad00f370 0x556cad007120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:04.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.013+0000 7fb5e120c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad007660 0x556cad026c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:04.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.013+0000 7fb5e120c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d81ac9b0 con 0x556cad00f370 2024-09-10T08:56:04.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.013+0000 7fb5e120c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d81ae4f0 con 0x556cad007660 2024-09-10T08:56:04.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.013+0000 7fb5e120c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d81ad080 con 0x556cad0045d0 2024-09-10T08:56:04.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.013+0000 7fb5d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad00f370 0x556cad007120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:04.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.014+0000 7fb5d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad00f370 0x556cad007120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40976/0 (socket says 172.21.15.172:40976) 2024-09-10T08:56:04.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.014+0000 7fb5d7fff640 1 -- 172.21.15.172:0/2645332198 learned_addr learned my addr 172.21.15.172:0/2645332198 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:04.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.014+0000 7fb5dcbcf640 1 --2- 172.21.15.172:0/2645332198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad0045d0 0x556cad00b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:04.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.014+0000 7fb5dd3d0640 1 --2- 172.21.15.172:0/2645332198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad007660 0x556cad026c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:04.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.014+0000 7fb5d7fff640 1 -- 172.21.15.172:0/2645332198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad0045d0 msgr2=0x556cad00b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:04.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.014+0000 7fb5d7fff640 1 --2- 172.21.15.172:0/2645332198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad0045d0 0x556cad00b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.014+0000 7fb5d7fff640 1 -- 172.21.15.172:0/2645332198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad007660 msgr2=0x556cad026c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:04.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.014+0000 7fb5d7fff640 1 --2- 172.21.15.172:0/2645332198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad007660 0x556cad026c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.014+0000 7fb5d7fff640 1 -- 172.21.15.172:0/2645332198 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cad00b030 con 0x556cad00f370 2024-09-10T08:56:04.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.014+0000 7fb5dcbcf640 1 --2- 172.21.15.172:0/2645332198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad0045d0 0x556cad00b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:04.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.014+0000 7fb5d7fff640 1 --2- 172.21.15.172:0/2645332198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad00f370 0x556cad007120 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4005ec0 tx=0x7fb5c402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d29d314955c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:04.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.014+0000 7fb5c27fc640 1 -- 172.21.15.172:0/2645332198 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4059070 con 0x556cad00f370 2024-09-10T08:56:04.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.015+0000 7fb5c27fc640 1 -- 172.21.15.172:0/2645332198 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5c4046a70 con 0x556cad00f370 2024-09-10T08:56:04.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.015+0000 7fb5c27fc640 1 -- 172.21.15.172:0/2645332198 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4035070 con 0x556cad00f370 2024-09-10T08:56:04.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.016+0000 7fb5e120c740 1 -- 172.21.15.172:0/2645332198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad00f370 msgr2=0x556cad007120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:04.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.016+0000 7fb5e120c740 1 --2- 172.21.15.172:0/2645332198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad00f370 0x556cad007120 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4005ec0 tx=0x7fb5c402a480 comp rx=0 tx=0).stop 2024-09-10T08:56:04.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.016+0000 7fb5e120c740 1 -- 172.21.15.172:0/2645332198 shutdown_connections 2024-09-10T08:56:04.017 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.016+0000 7fb5e120c740 1 --2- 172.21.15.172:0/2645332198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad007660 0x556cad026c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.016+0000 7fb5e120c740 1 --2- 172.21.15.172:0/2645332198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad00f370 0x556cad007120 secure :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4005ec0 tx=0x7fb5c402a480 comp rx=0 tx=0).stop 2024-09-10T08:56:04.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.016+0000 7fb5e120c740 1 --2- 172.21.15.172:0/2645332198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad0045d0 0x556cad00b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.016+0000 7fb5e120c740 1 -- 172.21.15.172:0/2645332198 >> 172.21.15.172:0/2645332198 conn(0x556cacfcd310 msgr2=0x556cad000700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:04.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.017+0000 7fb5e120c740 1 -- 172.21.15.172:0/2645332198 shutdown_connections 2024-09-10T08:56:04.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.017+0000 7fb5e120c740 1 -- 172.21.15.172:0/2645332198 wait complete. 2024-09-10T08:56:04.018 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.017+0000 7fb5e120c740 1 -- start start 2024-09-10T08:56:04.019 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.017+0000 7fb5e120c740 1 Processor -- start 2024-09-10T08:56:04.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.018+0000 7fb5e120c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad0045d0 0x556cad017cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:04.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.018+0000 7fb5e120c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad007660 0x556cad012f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:04.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.018+0000 7fb5e120c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad0134d0 0x556cad013910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:04.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.018+0000 7fb5e120c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d81ade20 con 0x556cad0045d0 2024-09-10T08:56:04.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.018+0000 7fb5e120c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d81ad750 con 0x556cad0134d0 2024-09-10T08:56:04.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.018+0000 7fb5e120c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d81aa990 con 0x556cad007660 2024-09-10T08:56:04.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.018+0000 7fb5e120c740 10 client.0 _ll_register_callbacks cb 0x7fb5c4032ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:04.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.018+0000 7fb5dcbcf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad0045d0 0x556cad017cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:04.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.018+0000 7fb5dcbcf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad0045d0 0x556cad017cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40982/0 (socket says 172.21.15.172:40982) 2024-09-10T08:56:04.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.018+0000 7fb5dcbcf640 1 -- 172.21.15.172:0/2634172618 learned_addr learned my addr 172.21.15.172:0/2634172618 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:04.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.019+0000 7fb5dd3d0640 1 --2- 172.21.15.172:0/2634172618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad0134d0 0x556cad013910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:04.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.019+0000 7fb5dcbcf640 1 -- 172.21.15.172:0/2634172618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad007660 msgr2=0x556cad012f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:04.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.019+0000 7fb5dcbcf640 1 --2- 172.21.15.172:0/2634172618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad007660 0x556cad012f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.019+0000 7fb5dcbcf640 1 -- 172.21.15.172:0/2634172618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad0134d0 msgr2=0x556cad013910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:04.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.019+0000 7fb5dcbcf640 1 --2- 172.21.15.172:0/2634172618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad0134d0 0x556cad013910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.019+0000 7fb5dcbcf640 1 -- 172.21.15.172:0/2634172618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cad088e20 con 0x556cad0045d0 2024-09-10T08:56:04.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.019+0000 7fb5dcbcf640 1 --2- 172.21.15.172:0/2634172618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad0045d0 0x556cad017cb0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc097340 tx=0x7fb5cc096b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6982438c283aa3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:04.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.019+0000 7fb5b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:04.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.019+0000 7fb5b7fff640 1 -- 172.21.15.172:0/2634172618 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc0b3070 con 0x556cad0045d0 2024-09-10T08:56:04.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.019+0000 7fb5b7fff640 1 -- 172.21.15.172:0/2634172618 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb5cc0991e0 con 0x556cad0045d0 2024-09-10T08:56:04.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.019+0000 7fb5e120c740 1 -- 172.21.15.172:0/2634172618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5c4046a70 con 0x556cad0045d0 2024-09-10T08:56:04.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.019+0000 7fb5b7fff640 1 -- 172.21.15.172:0/2634172618 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb5cc01e040 con 0x556cad0045d0 2024-09-10T08:56:04.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.019+0000 7fb5e120c740 10 client.18354 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:04.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.019+0000 7fb5e120c740 1 -- 172.21.15.172:0/2634172618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556cad08a120 con 0x556cad0045d0 2024-09-10T08:56:04.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.020+0000 7fb5b7fff640 1 -- 172.21.15.172:0/2634172618 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5c4046a70 con 0x556cad0045d0 2024-09-10T08:56:04.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.020+0000 7fb5b7fff640 1 -- 172.21.15.172:0/2634172618 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fb5cc010cb0 con 0x556cad0045d0 2024-09-10T08:56:04.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.020+0000 7fb5e120c740 10 client.18354 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:04.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.020+0000 7fb5e120c740 10 client.18354 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:56:04.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.020+0000 7fb5e120c740 20 client.18354 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:04.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.020+0000 7fb5e120c740 1 --2- 172.21.15.172:0/2634172618 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x556cad08a3b0 0x556cad08c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:04.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.020+0000 7fb5e120c740 4 client.18354 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:56:04.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.020+0000 7fb5e120c740 1 -- 172.21.15.172:0/2634172618 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556cacf71d70 con 0x556cad08a3b0 2024-09-10T08:56:04.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.020+0000 7fb5d7fff640 1 --2- 172.21.15.172:0/2634172618 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x556cad08a3b0 0x556cad08c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:04.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.021+0000 7fb5d7fff640 1 --2- 172.21.15.172:0/2634172618 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x556cad08a3b0 0x556cad08c7a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fa4ffaba486fbc4e server_cookie=a27b2488fb4d5e75 in_seq=0 out_seq=0 2024-09-10T08:56:04.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.021+0000 7fb5b7fff640 10 client.18354 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:56:04.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.021+0000 7fb5b7fff640 1 -- 172.21.15.172:0/2634172618 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556cacf71d70 con 0x556cad08a3b0 2024-09-10T08:56:04.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.021+0000 7fb5b7fff640 10 client.18354 handle_command_reply: tid=0 2024-09-10T08:56:04.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.021+0000 7fb5e120c740 1 client.18354 shutdown 2024-09-10T08:56:04.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.021+0000 7fb5e120c740 1 -- 172.21.15.172:0/2634172618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad0045d0 msgr2=0x556cad017cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:04.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.021+0000 7fb5e120c740 1 --2- 172.21.15.172:0/2634172618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad0045d0 0x556cad017cb0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc097340 tx=0x7fb5cc096b80 comp rx=0 tx=0).stop 2024-09-10T08:56:04.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.021+0000 7fb5e120c740 1 -- 172.21.15.172:0/2634172618 shutdown_connections 2024-09-10T08:56:04.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.021+0000 7fb5e120c740 1 --2- 172.21.15.172:0/2634172618 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x556cad08a3b0 0x556cad08c7a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.021+0000 7fb5e120c740 1 --2- 172.21.15.172:0/2634172618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cad0134d0 0x556cad013910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.021+0000 7fb5e120c740 1 --2- 172.21.15.172:0/2634172618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cad007660 0x556cad012f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.021+0000 7fb5b7fff640 0 client.18354 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:56:04.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.021+0000 7fb5e120c740 1 --2- 172.21.15.172:0/2634172618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cad0045d0 0x556cad017cb0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.021+0000 7fb5e120c740 1 -- 172.21.15.172:0/2634172618 >> 172.21.15.172:0/2634172618 conn(0x556cacfcd310 msgr2=0x556cad00d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:04.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.021+0000 7fb5e120c740 1 -- 172.21.15.172:0/2634172618 shutdown_connections 2024-09-10T08:56:04.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.022+0000 7fb5e120c740 1 -- 172.21.15.172:0/2634172618 wait complete. 2024-09-10T08:56:04.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.022+0000 7fb5e120c740 20 client.18354 trim_cache size 0 max 16384 2024-09-10T08:56:04.026 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:56:04.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.022+0000 7fb5dee33640 1 -- 172.21.15.172:0/2644123781 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb5c80a2c00 msgr2=0x7fb5c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:04.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.022+0000 7fb5dee33640 1 --2- 172.21.15.172:0/2644123781 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb5c80a2c00 0x7fb5c80a50c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0010fd0 tx=0x7fb5d0004a40 comp rx=0 tx=0).stop 2024-09-10T08:56:04.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.022+0000 7fb5dee33640 1 -- 172.21.15.172:0/2644123781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d8100e20 msgr2=0x7fb5d81985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:04.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.022+0000 7fb5dee33640 1 --2- 172.21.15.172:0/2644123781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d8100e20 0x7fb5d81985b0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00b4a0 tx=0x7fb5cc00b960 comp rx=0 tx=0).stop 2024-09-10T08:56:04.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.022+0000 7fb5dee33640 1 -- 172.21.15.172:0/2644123781 shutdown_connections 2024-09-10T08:56:04.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.022+0000 7fb5dee33640 1 --2- 172.21.15.172:0/2644123781 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb5c80a2c00 0x7fb5c80a50c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.022+0000 7fb5dee33640 1 --2- 172.21.15.172:0/2644123781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5d8102ef0 0x7fb5d819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.022+0000 7fb5dee33640 1 --2- 172.21.15.172:0/2644123781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5d8102570 0x7fb5d8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.022+0000 7fb5dee33640 1 --2- 172.21.15.172:0/2644123781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5d8100e20 0x7fb5d81985b0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:04.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.023+0000 7fb5dee33640 1 -- 172.21.15.172:0/2644123781 >> 172.21.15.172:0/2644123781 conn(0x7fb5d80fc7d0 msgr2=0x7fb5d806a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:04.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.023+0000 7fb5dee33640 1 -- 172.21.15.172:0/2644123781 shutdown_connections 2024-09-10T08:56:04.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:04.023+0000 7fb5dee33640 1 -- 172.21.15.172:0/2644123781 wait complete. 2024-09-10T08:56:04.643 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:04 smithi174 ceph-mon[34068]: pgmap v645: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-09-10T08:56:04.701 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:04 smithi191 ceph-mon[35314]: pgmap v645: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-09-10T08:56:04.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:04 smithi172 ceph-mon[30854]: pgmap v645: 129 pgs: 129 active+clean; 8.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 23 MiB/s wr, 18 op/s 2024-09-10T08:56:04.928 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T08:56:05.220 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:56:05.677 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:05 smithi172 ceph-mon[30854]: pgmap v646: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-09-10T08:56:05.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:05 smithi174 ceph-mon[34068]: pgmap v646: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-09-10T08:56:05.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.785+0000 7f5cbe027640 1 -- 172.21.15.172:0/3117314093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cb8106f10 msgr2=0x7f5cb810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:05.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.785+0000 7f5cbe027640 1 --2- 172.21.15.172:0/3117314093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cb8106f10 0x7f5cb810ea90 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0009ca0 tx=0x7f5cb002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.785+0000 7f5cbe027640 1 -- 172.21.15.172:0/3117314093 shutdown_connections 2024-09-10T08:56:05.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.785+0000 7f5cbe027640 1 --2- 172.21.15.172:0/3117314093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cb8106f10 0x7f5cb810ea90 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.785+0000 7f5cbe027640 1 --2- 172.21.15.172:0/3117314093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cb8102790 0x7f5cb81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.785+0000 7f5cbe027640 1 --2- 172.21.15.172:0/3117314093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cb8101de0 0x7f5cb81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.785+0000 7f5cbe027640 1 -- 172.21.15.172:0/3117314093 >> 172.21.15.172:0/3117314093 conn(0x7f5cb80fd8c0 msgr2=0x7f5cb80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:05.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.785+0000 7f5cbe027640 1 -- 172.21.15.172:0/3117314093 shutdown_connections 2024-09-10T08:56:05.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.785+0000 7f5cbe027640 1 -- 172.21.15.172:0/3117314093 wait complete. 2024-09-10T08:56:05.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.786+0000 7f5cbe027640 1 Processor -- start 2024-09-10T08:56:05.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.786+0000 7f5cbe027640 1 -- start start 2024-09-10T08:56:05.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.786+0000 7f5cbe027640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cb8101de0 0x7f5cb8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.786+0000 7f5cbe027640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cb8102790 0x7f5cb8198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.786+0000 7f5cbe027640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cb8106f10 0x7f5cb819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.786+0000 7f5cbe027640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb8111e20 con 0x7f5cb8102790 2024-09-10T08:56:05.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.786+0000 7f5cbe027640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb8111ca0 con 0x7f5cb8101de0 2024-09-10T08:56:05.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.786+0000 7f5cbe027640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb8111fa0 con 0x7f5cb8106f10 2024-09-10T08:56:05.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.786+0000 7f5caffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cb8102790 0x7f5cb8198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.786+0000 7f5caffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cb8102790 0x7f5cb8198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41004/0 (socket says 172.21.15.172:41004) 2024-09-10T08:56:05.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.786+0000 7f5caffff640 1 -- 172.21.15.172:0/2390518950 learned_addr learned my addr 172.21.15.172:0/2390518950 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:05.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.786+0000 7f5cb77fe640 1 --2- 172.21.15.172:0/2390518950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cb8101de0 0x7f5cb8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.786+0000 7f5cb7fff640 1 --2- 172.21.15.172:0/2390518950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cb8106f10 0x7f5cb819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.787+0000 7f5caffff640 1 -- 172.21.15.172:0/2390518950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cb8106f10 msgr2=0x7f5cb819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:05.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.787+0000 7f5caffff640 1 --2- 172.21.15.172:0/2390518950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cb8106f10 0x7f5cb819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.787+0000 7f5caffff640 1 -- 172.21.15.172:0/2390518950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cb8101de0 msgr2=0x7f5cb8198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:05.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.787+0000 7f5caffff640 1 --2- 172.21.15.172:0/2390518950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cb8101de0 0x7f5cb8198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.787+0000 7f5caffff640 1 -- 172.21.15.172:0/2390518950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb819d2f0 con 0x7f5cb8102790 2024-09-10T08:56:05.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.787+0000 7f5cb7fff640 1 --2- 172.21.15.172:0/2390518950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cb8106f10 0x7f5cb819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:05.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.787+0000 7f5caffff640 1 --2- 172.21.15.172:0/2390518950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cb8102790 0x7f5cb8198880 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000d950 tx=0x7f5ca000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:05.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.787+0000 7f5cb57fa640 1 -- 172.21.15.172:0/2390518950 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0017070 con 0x7f5cb8102790 2024-09-10T08:56:05.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.787+0000 7f5cb57fa640 1 -- 172.21.15.172:0/2390518950 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ca0009c30 con 0x7f5cb8102790 2024-09-10T08:56:05.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.787+0000 7f5cb57fa640 1 -- 172.21.15.172:0/2390518950 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca000f620 con 0x7f5cb8102790 2024-09-10T08:56:05.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.787+0000 7f5cbe027640 1 -- 172.21.15.172:0/2390518950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb819d5e0 con 0x7f5cb8102790 2024-09-10T08:56:05.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.787+0000 7f5cbe027640 1 -- 172.21.15.172:0/2390518950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb81a4ec0 con 0x7f5cb8102790 2024-09-10T08:56:05.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.790+0000 7f5cc0400740 1 Processor -- start 2024-09-10T08:56:05.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.790+0000 7f5cb57fa640 1 -- 172.21.15.172:0/2390518950 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5ca00109a0 con 0x7f5cb8102790 2024-09-10T08:56:05.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.790+0000 7f5cc0400740 1 -- start start 2024-09-10T08:56:05.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5cc0400740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 0x563b9b0ab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5cc0400740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0ab5d0 0x563b9b0aba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5cc0400740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0b6370 0x563b9b0b8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5cc0400740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563b9b0112f0 con 0x563b9b0b6370 2024-09-10T08:56:05.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5cc0400740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b9afb8cf0 con 0x563b9b0acb40 2024-09-10T08:56:05.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5cc0400740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563b9b00d2b0 con 0x563b9b0ab5d0 2024-09-10T08:56:05.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5caffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0b6370 0x563b9b0b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5caffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0b6370 0x563b9b0b8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41018/0 (socket says 172.21.15.172:41018) 2024-09-10T08:56:05.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5caffff640 1 -- 172.21.15.172:0/242880381 learned_addr learned my addr 172.21.15.172:0/242880381 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:05.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5cb77fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0ab5d0 0x563b9b0aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5cb7fff640 1 --2- 172.21.15.172:0/242880381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 0x563b9b0ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5cb7fff640 1 -- 172.21.15.172:0/242880381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0ab5d0 msgr2=0x563b9b0aba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:05.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5cb7fff640 1 --2- 172.21.15.172:0/242880381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0ab5d0 0x563b9b0aba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5cb7fff640 1 -- 172.21.15.172:0/242880381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0b6370 msgr2=0x563b9b0b8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:05.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5cb7fff640 1 --2- 172.21.15.172:0/242880381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0b6370 0x563b9b0b8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5cb57fa640 1 --2- 172.21.15.172:0/2390518950 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5ca40a2c00 0x7f5ca40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5cb7fff640 1 -- 172.21.15.172:0/242880381 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b9af9d3a0 con 0x563b9b0acb40 2024-09-10T08:56:05.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5cb77fe640 1 --2- 172.21.15.172:0/2390518950 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5ca40a2c00 0x7f5ca40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.791+0000 7f5cb57fa640 1 -- 172.21.15.172:0/2390518950 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5ca00cef30 con 0x7f5cb8102790 2024-09-10T08:56:05.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.792+0000 7f5cb7fff640 1 --2- 172.21.15.172:0/242880381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 0x563b9b0ab090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0006010 tx=0x7f5cb0009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eff657156bc8f4fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:05.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.792+0000 7f5cadffb640 1 -- 172.21.15.172:0/242880381 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0031d40 con 0x563b9b0acb40 2024-09-10T08:56:05.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.792+0000 7f5cadffb640 1 -- 172.21.15.172:0/242880381 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cb0036040 con 0x563b9b0acb40 2024-09-10T08:56:05.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.792+0000 7f5cadffb640 1 -- 172.21.15.172:0/242880381 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0040a80 con 0x563b9b0acb40 2024-09-10T08:56:05.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.792+0000 7f5cb77fe640 1 --2- 172.21.15.172:0/2390518950 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5ca40a2c00 0x7f5ca40a50c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x563b9b0a8ac0 tx=0x7f5ca800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:05.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.793+0000 7f5cc0400740 1 -- 172.21.15.172:0/242880381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 msgr2=0x563b9b0ab090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:05.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.793+0000 7f5cc0400740 1 --2- 172.21.15.172:0/242880381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 0x563b9b0ab090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0006010 tx=0x7f5cb0009360 comp rx=0 tx=0).stop 2024-09-10T08:56:05.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.793+0000 7f5cc0400740 1 -- 172.21.15.172:0/242880381 shutdown_connections 2024-09-10T08:56:05.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.793+0000 7f5cc0400740 1 --2- 172.21.15.172:0/242880381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0b6370 0x563b9b0b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.793+0000 7f5cc0400740 1 --2- 172.21.15.172:0/242880381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0ab5d0 0x563b9b0aba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.793+0000 7f5cc0400740 1 --2- 172.21.15.172:0/242880381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 0x563b9b0ab090 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.793+0000 7f5cc0400740 1 -- 172.21.15.172:0/242880381 >> 172.21.15.172:0/242880381 conn(0x563b9b074310 msgr2=0x563b9b0a7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:05.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.793+0000 7f5cc0400740 1 -- 172.21.15.172:0/242880381 shutdown_connections 2024-09-10T08:56:05.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.793+0000 7f5cc0400740 1 -- 172.21.15.172:0/242880381 wait complete. 2024-09-10T08:56:05.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.793+0000 7f5cc0400740 1 -- start start 2024-09-10T08:56:05.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.793+0000 7f5cc0400740 1 Processor -- start 2024-09-10T08:56:05.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5cc0400740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0ab5d0 0x563b9b0b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5cc0400740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 0x563b9b0b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5cc0400740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0b6370 0x563b9b0b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5cc0400740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563b9b0b9010 con 0x563b9b0ab5d0 2024-09-10T08:56:05.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5cc0400740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b9afcc060 con 0x563b9b0acb40 2024-09-10T08:56:05.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5cc0400740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563b9b0b9190 con 0x563b9b0b6370 2024-09-10T08:56:05.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5cc0400740 10 client.0 _ll_register_callbacks cb 0x563b9ad1f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:05.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5cb77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0ab5d0 0x563b9b0b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5caffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 0x563b9b0b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5caffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 0x563b9b0b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:58150/0 (socket says 172.21.15.172:58150) 2024-09-10T08:56:05.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5caffff640 1 -- 172.21.15.172:0/2210075899 learned_addr learned my addr 172.21.15.172:0/2210075899 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:05.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5cb7fff640 1 --2- 172.21.15.172:0/2210075899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0b6370 0x563b9b0b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5cb77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0ab5d0 0x563b9b0b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41024/0 (socket says 172.21.15.172:41024) 2024-09-10T08:56:05.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5cb7fff640 1 -- 172.21.15.172:0/2210075899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 msgr2=0x563b9b0b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:05.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5cb7fff640 1 --2- 172.21.15.172:0/2210075899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 0x563b9b0b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5cb7fff640 1 -- 172.21.15.172:0/2210075899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0ab5d0 msgr2=0x563b9b0b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:05.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5cb7fff640 1 --2- 172.21.15.172:0/2210075899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0ab5d0 0x563b9b0b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.794+0000 7f5cb7fff640 1 -- 172.21.15.172:0/2210075899 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b9ad823c0 con 0x563b9b0b6370 2024-09-10T08:56:05.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.795+0000 7f5caffff640 1 --2- 172.21.15.172:0/2210075899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 0x563b9b0b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:05.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.795+0000 7f5cb7fff640 1 --2- 172.21.15.172:0/2210075899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0b6370 0x563b9b0b3530 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0005e30 tx=0x7f5cb0031be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31cdc7d7b2bfc12d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:05.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.795+0000 7f5c96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:56:05.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.795+0000 7f5c96ffd640 1 -- 172.21.15.172:0/2210075899 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0056020 con 0x563b9b0b6370 2024-09-10T08:56:05.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.795+0000 7f5cc0400740 1 -- 172.21.15.172:0/2210075899 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cb0036040 con 0x563b9b0b6370 2024-09-10T08:56:05.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.795+0000 7f5c96ffd640 1 -- 172.21.15.172:0/2210075899 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5cb0003970 con 0x563b9b0b6370 2024-09-10T08:56:05.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.795+0000 7f5c96ffd640 1 -- 172.21.15.172:0/2210075899 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5cb0061050 con 0x563b9b0b6370 2024-09-10T08:56:05.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.795+0000 7f5cc0400740 10 client.27265 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:05.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.795+0000 7f5cc0400740 1 -- 172.21.15.172:0/2210075899 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563b9b0b9010 con 0x563b9b0b6370 2024-09-10T08:56:05.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.796+0000 7f5c96ffd640 1 -- 172.21.15.172:0/2210075899 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cb0036040 con 0x563b9b0b6370 2024-09-10T08:56:05.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.796+0000 7f5c96ffd640 1 -- 172.21.15.172:0/2210075899 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f5cb0051030 con 0x563b9b0b6370 2024-09-10T08:56:05.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.796+0000 7f5cc0400740 10 client.27265 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:05.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.796+0000 7f5cc0400740 10 client.27265 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:56:05.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.796+0000 7f5cc0400740 20 client.27265 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:05.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.796+0000 7f5cc0400740 1 --2- 172.21.15.172:0/2210075899 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x563b9b0c5bc0 0x563b9b0c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.796+0000 7f5cc0400740 4 client.27265 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:56:05.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.796+0000 7f5cc0400740 1 -- 172.21.15.172:0/2210075899 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b9b01e860 con 0x563b9b0c5bc0 2024-09-10T08:56:05.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.796+0000 7f5cb77fe640 1 --2- 172.21.15.172:0/2210075899 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x563b9b0c5bc0 0x563b9b0c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.797+0000 7f5cb77fe640 1 --2- 172.21.15.172:0/2210075899 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x563b9b0c5bc0 0x563b9b0c7fb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b2c8f54c83b621a server_cookie=5c6b4c291efea5e5 in_seq=0 out_seq=0 2024-09-10T08:56:05.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.797+0000 7f5c96ffd640 10 client.27265 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:56:05.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.797+0000 7f5c96ffd640 1 -- 172.21.15.172:0/2210075899 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b9b01e860 con 0x563b9b0c5bc0 2024-09-10T08:56:05.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.797+0000 7f5c96ffd640 10 client.27265 handle_command_reply: tid=0 2024-09-10T08:56:05.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.798+0000 7f5cc0400740 1 client.27265 shutdown 2024-09-10T08:56:05.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.798+0000 7f5cc0400740 1 -- 172.21.15.172:0/2210075899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0b6370 msgr2=0x563b9b0b3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:05.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.798+0000 7f5cc0400740 1 --2- 172.21.15.172:0/2210075899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0b6370 0x563b9b0b3530 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0005e30 tx=0x7f5cb0031be0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.798+0000 7f5cc0400740 1 -- 172.21.15.172:0/2210075899 shutdown_connections 2024-09-10T08:56:05.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.798+0000 7f5cc0400740 1 --2- 172.21.15.172:0/2210075899 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x563b9b0c5bc0 0x563b9b0c7fb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.798+0000 7f5cc0400740 1 --2- 172.21.15.172:0/2210075899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0b6370 0x563b9b0b3530 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.798+0000 7f5c96ffd640 0 client.27265 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:56:05.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.798+0000 7f5cc0400740 1 --2- 172.21.15.172:0/2210075899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 0x563b9b0b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.798+0000 7f5cc0400740 1 --2- 172.21.15.172:0/2210075899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0ab5d0 0x563b9b0b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.798+0000 7f5cc0400740 1 -- 172.21.15.172:0/2210075899 >> 172.21.15.172:0/2210075899 conn(0x563b9b074310 msgr2=0x563b9b0ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:05.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.798+0000 7f5cc0400740 1 -- 172.21.15.172:0/2210075899 shutdown_connections 2024-09-10T08:56:05.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.798+0000 7f5cc0400740 1 -- 172.21.15.172:0/2210075899 wait complete. 2024-09-10T08:56:05.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.798+0000 7f5cc0400740 20 client.27265 trim_cache size 0 max 16384 2024-09-10T08:56:05.806 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:05 smithi191 ceph-mon[35314]: pgmap v646: 129 pgs: 129 active+clean; 9.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 31 MiB/s wr, 25 op/s 2024-09-10T08:56:05.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.811+0000 7f5cc0400740 1 Processor -- start 2024-09-10T08:56:05.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.811+0000 7f5cc0400740 1 -- start start 2024-09-10T08:56:05.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.811+0000 7f5cc0400740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0ab5d0 0x563b9b0c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.811+0000 7f5cc0400740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 0x563b9b0b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.811+0000 7f5cc0400740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0b6370 0x563b9b0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.811+0000 7f5cc0400740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb81ac920 con 0x563b9b0ab5d0 2024-09-10T08:56:05.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.811+0000 7f5cc0400740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb81ae460 con 0x563b9b0acb40 2024-09-10T08:56:05.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.811+0000 7f5cc0400740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb81acff0 con 0x563b9b0b6370 2024-09-10T08:56:05.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.811+0000 7f5cb77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0ab5d0 0x563b9b0c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.811+0000 7f5cb77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0ab5d0 0x563b9b0c22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41032/0 (socket says 172.21.15.172:41032) 2024-09-10T08:56:05.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.811+0000 7f5cb77fe640 1 -- 172.21.15.172:0/345526267 learned_addr learned my addr 172.21.15.172:0/345526267 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:05.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.811+0000 7f5caffff640 1 --2- 172.21.15.172:0/345526267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 0x563b9b0b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.812+0000 7f5cb7fff640 1 --2- 172.21.15.172:0/345526267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0b6370 0x563b9b0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.812+0000 7f5cb77fe640 1 -- 172.21.15.172:0/345526267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0b6370 msgr2=0x563b9b0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:05.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.812+0000 7f5cb77fe640 1 --2- 172.21.15.172:0/345526267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0b6370 0x563b9b0cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.812+0000 7f5cb77fe640 1 -- 172.21.15.172:0/345526267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 msgr2=0x563b9b0b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:05.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.812+0000 7f5cb77fe640 1 --2- 172.21.15.172:0/345526267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 0x563b9b0b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.812+0000 7f5cb77fe640 1 -- 172.21.15.172:0/345526267 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b9b0ae120 con 0x563b9b0ab5d0 2024-09-10T08:56:05.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.812+0000 7f5cb7fff640 1 --2- 172.21.15.172:0/345526267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0b6370 0x563b9b0cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:05.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.812+0000 7f5cb77fe640 1 --2- 172.21.15.172:0/345526267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0ab5d0 0x563b9b0c22b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8003db0 tx=0x7f5ca8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24618e97f02ba88e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:05.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.812+0000 7f5cad7fa640 1 -- 172.21.15.172:0/345526267 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8030070 con 0x563b9b0ab5d0 2024-09-10T08:56:05.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.812+0000 7f5cad7fa640 1 -- 172.21.15.172:0/345526267 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ca802a3b0 con 0x563b9b0ab5d0 2024-09-10T08:56:05.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.812+0000 7f5cad7fa640 1 -- 172.21.15.172:0/345526267 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8029660 con 0x563b9b0ab5d0 2024-09-10T08:56:05.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.813+0000 7f5cc0400740 1 -- 172.21.15.172:0/345526267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0ab5d0 msgr2=0x563b9b0c22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:05.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.813+0000 7f5cc0400740 1 --2- 172.21.15.172:0/345526267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0ab5d0 0x563b9b0c22b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8003db0 tx=0x7f5ca8004610 comp rx=0 tx=0).stop 2024-09-10T08:56:05.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.813+0000 7f5cc0400740 1 -- 172.21.15.172:0/345526267 shutdown_connections 2024-09-10T08:56:05.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.813+0000 7f5cc0400740 1 --2- 172.21.15.172:0/345526267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0b6370 0x563b9b0cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.813+0000 7f5cc0400740 1 --2- 172.21.15.172:0/345526267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0acb40 0x563b9b0b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.813+0000 7f5cc0400740 1 --2- 172.21.15.172:0/345526267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0ab5d0 0x563b9b0c22b0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.813+0000 7f5cc0400740 1 -- 172.21.15.172:0/345526267 >> 172.21.15.172:0/345526267 conn(0x563b9b074310 msgr2=0x563b9b0b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:05.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.813+0000 7f5cc0400740 1 -- 172.21.15.172:0/345526267 shutdown_connections 2024-09-10T08:56:05.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.813+0000 7f5cc0400740 1 -- 172.21.15.172:0/345526267 wait complete. 2024-09-10T08:56:05.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.813+0000 7f5cc0400740 1 -- start start 2024-09-10T08:56:05.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5cc0400740 1 Processor -- start 2024-09-10T08:56:05.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5cc0400740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0acb40 0x563b9b0baa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5cc0400740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0b6370 0x563b9b0baf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5cc0400740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0c5bc0 0x563b9b0be0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5cc0400740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb81add90 con 0x563b9b0b6370 2024-09-10T08:56:05.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5cc0400740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb81ad6c0 con 0x563b9b0c5bc0 2024-09-10T08:56:05.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5cc0400740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb81aa900 con 0x563b9b0acb40 2024-09-10T08:56:05.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5cc0400740 10 client.0 _ll_register_callbacks cb 0x7f5ca00d28e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:05.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5caffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0b6370 0x563b9b0baf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5caffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0b6370 0x563b9b0baf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41044/0 (socket says 172.21.15.172:41044) 2024-09-10T08:56:05.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5caffff640 1 -- 172.21.15.172:0/1080042924 learned_addr learned my addr 172.21.15.172:0/1080042924 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:05.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5cb77fe640 1 --2- 172.21.15.172:0/1080042924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0acb40 0x563b9b0baa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5cb7fff640 1 --2- 172.21.15.172:0/1080042924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0c5bc0 0x563b9b0be0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5caffff640 1 -- 172.21.15.172:0/1080042924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0acb40 msgr2=0x563b9b0baa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:05.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5caffff640 1 --2- 172.21.15.172:0/1080042924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0acb40 0x563b9b0baa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5caffff640 1 -- 172.21.15.172:0/1080042924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0c5bc0 msgr2=0x563b9b0be0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:05.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5caffff640 1 --2- 172.21.15.172:0/1080042924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0c5bc0 0x563b9b0be0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.814+0000 7f5caffff640 1 -- 172.21.15.172:0/1080042924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b9b0be790 con 0x563b9b0b6370 2024-09-10T08:56:05.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.815+0000 7f5cb7fff640 1 --2- 172.21.15.172:0/1080042924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0c5bc0 0x563b9b0be0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:56:05.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.815+0000 7f5caffff640 1 --2- 172.21.15.172:0/1080042924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0b6370 0x563b9b0baf90 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0097890 tx=0x7f5ca00d2600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682f72676543bb88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:05.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.815+0000 7f5c96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:05.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.815+0000 7f5c96ffd640 1 -- 172.21.15.172:0/1080042924 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0020040 con 0x563b9b0b6370 2024-09-10T08:56:05.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.815+0000 7f5c96ffd640 1 -- 172.21.15.172:0/1080042924 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5ca00d2dc0 con 0x563b9b0b6370 2024-09-10T08:56:05.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.815+0000 7f5cc0400740 1 -- 172.21.15.172:0/1080042924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ca802a3b0 con 0x563b9b0b6370 2024-09-10T08:56:05.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.815+0000 7f5c96ffd640 1 -- 172.21.15.172:0/1080042924 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5ca0009de0 con 0x563b9b0b6370 2024-09-10T08:56:05.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.815+0000 7f5cc0400740 10 client.18372 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:05.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.815+0000 7f5cc0400740 1 -- 172.21.15.172:0/1080042924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563b9b1311d0 con 0x563b9b0b6370 2024-09-10T08:56:05.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.815+0000 7f5c96ffd640 1 -- 172.21.15.172:0/1080042924 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ca009bb30 con 0x563b9b0b6370 2024-09-10T08:56:05.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.815+0000 7f5c96ffd640 1 -- 172.21.15.172:0/1080042924 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f5ca0097380 con 0x563b9b0b6370 2024-09-10T08:56:05.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.815+0000 7f5cc0400740 10 client.18372 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:05.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.815+0000 7f5cc0400740 10 client.18372 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:56:05.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.816+0000 7f5cc0400740 20 client.18372 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:05.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.816+0000 7f5cc0400740 1 --2- 172.21.15.172:0/1080042924 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x563b9b131910 0x563b9b133d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:05.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.816+0000 7f5cc0400740 4 client.18372 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:56:05.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.816+0000 7f5cc0400740 1 -- 172.21.15.172:0/1080042924 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5ca802a3b0 con 0x563b9b131910 2024-09-10T08:56:05.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.816+0000 7f5cb77fe640 1 --2- 172.21.15.172:0/1080042924 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x563b9b131910 0x563b9b133d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:05.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.816+0000 7f5cb77fe640 1 --2- 172.21.15.172:0/1080042924 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x563b9b131910 0x563b9b133d30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d45e81f63fd61f15 server_cookie=370e6232c6693784 in_seq=0 out_seq=0 2024-09-10T08:56:05.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.816+0000 7f5c96ffd640 10 client.18372 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:56:05.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.816+0000 7f5c96ffd640 1 -- 172.21.15.172:0/1080042924 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5ca802a3b0 con 0x563b9b131910 2024-09-10T08:56:05.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.816+0000 7f5c96ffd640 10 client.18372 handle_command_reply: tid=0 2024-09-10T08:56:05.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.816+0000 7f5cc0400740 1 client.18372 shutdown 2024-09-10T08:56:05.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.817+0000 7f5cc0400740 1 -- 172.21.15.172:0/1080042924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0b6370 msgr2=0x563b9b0baf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:05.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.817+0000 7f5cc0400740 1 --2- 172.21.15.172:0/1080042924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0b6370 0x563b9b0baf90 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0097890 tx=0x7f5ca00d2600 comp rx=0 tx=0).stop 2024-09-10T08:56:05.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.817+0000 7f5cc0400740 1 -- 172.21.15.172:0/1080042924 shutdown_connections 2024-09-10T08:56:05.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.817+0000 7f5cc0400740 1 --2- 172.21.15.172:0/1080042924 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x563b9b131910 0x563b9b133d30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.817+0000 7f5cc0400740 1 --2- 172.21.15.172:0/1080042924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b9b0c5bc0 0x563b9b0be0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.817+0000 7f5c96ffd640 0 client.18372 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:56:05.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.817+0000 7f5cc0400740 1 --2- 172.21.15.172:0/1080042924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b9b0b6370 0x563b9b0baf90 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.817+0000 7f5cc0400740 1 --2- 172.21.15.172:0/1080042924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b9b0acb40 0x563b9b0baa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.817+0000 7f5cc0400740 1 -- 172.21.15.172:0/1080042924 >> 172.21.15.172:0/1080042924 conn(0x563b9b074310 msgr2=0x563b9b0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:05.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.817+0000 7f5cc0400740 1 -- 172.21.15.172:0/1080042924 shutdown_connections 2024-09-10T08:56:05.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.817+0000 7f5cc0400740 1 -- 172.21.15.172:0/1080042924 wait complete. 2024-09-10T08:56:05.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.817+0000 7f5cc0400740 20 client.18372 trim_cache size 0 max 16384 2024-09-10T08:56:05.825 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:56:05.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.818+0000 7f5cbe027640 1 -- 172.21.15.172:0/2390518950 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5ca40a2c00 msgr2=0x7f5ca40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:05.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.818+0000 7f5cbe027640 1 --2- 172.21.15.172:0/2390518950 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5ca40a2c00 0x7f5ca40a50c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x563b9b0a8ac0 tx=0x7f5ca800b390 comp rx=0 tx=0).stop 2024-09-10T08:56:05.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.818+0000 7f5cbe027640 1 -- 172.21.15.172:0/2390518950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cb8102790 msgr2=0x7f5cb8198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:05.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.818+0000 7f5cbe027640 1 --2- 172.21.15.172:0/2390518950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cb8102790 0x7f5cb8198880 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000d950 tx=0x7f5ca000de10 comp rx=0 tx=0).stop 2024-09-10T08:56:05.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.818+0000 7f5cbe027640 1 -- 172.21.15.172:0/2390518950 shutdown_connections 2024-09-10T08:56:05.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.818+0000 7f5cbe027640 1 --2- 172.21.15.172:0/2390518950 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5ca40a2c00 0x7f5ca40a50c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.818+0000 7f5cbe027640 1 --2- 172.21.15.172:0/2390518950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cb8106f10 0x7f5cb819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.818+0000 7f5cbe027640 1 --2- 172.21.15.172:0/2390518950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cb8102790 0x7f5cb8198880 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.818+0000 7f5cbe027640 1 --2- 172.21.15.172:0/2390518950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cb8101de0 0x7f5cb8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:05.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.818+0000 7f5cbe027640 1 -- 172.21.15.172:0/2390518950 >> 172.21.15.172:0/2390518950 conn(0x7f5cb80fd8c0 msgr2=0x7f5cb80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:05.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.818+0000 7f5cbe027640 1 -- 172.21.15.172:0/2390518950 shutdown_connections 2024-09-10T08:56:05.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:05.818+0000 7f5cbe027640 1 -- 172.21.15.172:0/2390518950 wait complete. 2024-09-10T08:56:06.665 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:56:06.952 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:56:07.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.467+0000 7fd0dda9d640 1 -- 172.21.15.172:0/2861025208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0d8106b80 msgr2=0x7fd0d8106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:07.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.467+0000 7fd0dda9d640 1 --2- 172.21.15.172:0/2861025208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0d8106b80 0x7fd0d8106f60 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8009900 tx=0x7fd0c802a410 comp rx=0 tx=0).stop 2024-09-10T08:56:07.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.467+0000 7fd0dda9d640 1 -- 172.21.15.172:0/2861025208 shutdown_connections 2024-09-10T08:56:07.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.467+0000 7fd0dda9d640 1 --2- 172.21.15.172:0/2861025208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0d8100aa0 0x7fd0d810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.467+0000 7fd0dda9d640 1 --2- 172.21.15.172:0/2861025208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0d8100100 0x7fd0d8100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.467+0000 7fd0dda9d640 1 --2- 172.21.15.172:0/2861025208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0d8106b80 0x7fd0d8106f60 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.467+0000 7fd0dda9d640 1 -- 172.21.15.172:0/2861025208 >> 172.21.15.172:0/2861025208 conn(0x7fd0d8077230 msgr2=0x7fd0d80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:07.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.467+0000 7fd0dda9d640 1 -- 172.21.15.172:0/2861025208 shutdown_connections 2024-09-10T08:56:07.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.467+0000 7fd0dda9d640 1 -- 172.21.15.172:0/2861025208 wait complete. 2024-09-10T08:56:07.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.468+0000 7fd0dda9d640 1 Processor -- start 2024-09-10T08:56:07.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.468+0000 7fd0dda9d640 1 -- start start 2024-09-10T08:56:07.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.468+0000 7fd0dda9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0d8100100 0x7fd0d81012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.468+0000 7fd0dda9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0d8100aa0 0x7fd0d8101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.468+0000 7fd0dda9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0d8106b80 0x7fd0d8101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.468+0000 7fd0dda9d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d8111f00 con 0x7fd0d8100aa0 2024-09-10T08:56:07.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.468+0000 7fd0dda9d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d8111d80 con 0x7fd0d8100100 2024-09-10T08:56:07.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.468+0000 7fd0dda9d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d8112080 con 0x7fd0d8106b80 2024-09-10T08:56:07.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.468+0000 7fd0d6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0d8100aa0 0x7fd0d8101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:07.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.468+0000 7fd0d6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0d8100aa0 0x7fd0d8101820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42890/0 (socket says 172.21.15.172:42890) 2024-09-10T08:56:07.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.468+0000 7fd0d6ffd640 1 -- 172.21.15.172:0/1608887230 learned_addr learned my addr 172.21.15.172:0/1608887230 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:07.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.468+0000 7fd0d77fe640 1 --2- 172.21.15.172:0/1608887230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0d8100100 0x7fd0d81012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:07.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.468+0000 7fd0d7fff640 1 --2- 172.21.15.172:0/1608887230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0d8106b80 0x7fd0d8101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:07.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.469+0000 7fd0d6ffd640 1 -- 172.21.15.172:0/1608887230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0d8106b80 msgr2=0x7fd0d8101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:07.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.469+0000 7fd0d6ffd640 1 --2- 172.21.15.172:0/1608887230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0d8106b80 0x7fd0d8101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.469+0000 7fd0d6ffd640 1 -- 172.21.15.172:0/1608887230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0d8100100 msgr2=0x7fd0d81012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:07.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.469+0000 7fd0d6ffd640 1 --2- 172.21.15.172:0/1608887230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0d8100100 0x7fd0d81012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.469+0000 7fd0d6ffd640 1 -- 172.21.15.172:0/1608887230 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d81a68e0 con 0x7fd0d8100aa0 2024-09-10T08:56:07.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.469+0000 7fd0d6ffd640 1 --2- 172.21.15.172:0/1608887230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0d8100aa0 0x7fd0d8101820 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00d950 tx=0x7fd0cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:07.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.469+0000 7fd0d4ff9640 1 -- 172.21.15.172:0/1608887230 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc017030 con 0x7fd0d8100aa0 2024-09-10T08:56:07.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.469+0000 7fd0d4ff9640 1 -- 172.21.15.172:0/1608887230 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0cc004020 con 0x7fd0d8100aa0 2024-09-10T08:56:07.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.469+0000 7fd0dda9d640 1 -- 172.21.15.172:0/1608887230 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d81a6bd0 con 0x7fd0d8100aa0 2024-09-10T08:56:07.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.469+0000 7fd0d4ff9640 1 -- 172.21.15.172:0/1608887230 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0047a0 con 0x7fd0d8100aa0 2024-09-10T08:56:07.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.469+0000 7fd0dda9d640 1 -- 172.21.15.172:0/1608887230 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d81a7160 con 0x7fd0d8100aa0 2024-09-10T08:56:07.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.472+0000 7fd0d4ff9640 1 -- 172.21.15.172:0/1608887230 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd0cc0041d0 con 0x7fd0d8100aa0 2024-09-10T08:56:07.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.472+0000 7fd0d4ff9640 1 --2- 172.21.15.172:0/1608887230 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd0c00a2c00 0x7fd0c00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.472+0000 7fd0d77fe640 1 --2- 172.21.15.172:0/1608887230 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd0c00a2c00 0x7fd0c00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:07.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.472+0000 7fd0d4ff9640 1 -- 172.21.15.172:0/1608887230 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd0cc0ceea0 con 0x7fd0d8100aa0 2024-09-10T08:56:07.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.472+0000 7fd0dfe76740 1 Processor -- start 2024-09-10T08:56:07.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0dfe76740 1 -- start start 2024-09-10T08:56:07.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0dfe76740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507db40 0x55582507c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0dfe76740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55582507c5d0 0x55582507ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0dfe76740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555825087370 0x555825089760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0dfe76740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555824fe22f0 con 0x55582507c5d0 2024-09-10T08:56:07.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0dfe76740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555824f89cf0 con 0x555825087370 2024-09-10T08:56:07.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0dfe76740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555824fde2b0 con 0x55582507db40 2024-09-10T08:56:07.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0d6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55582507c5d0 0x55582507ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:07.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0d77fe640 1 --2- 172.21.15.172:0/1608887230 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd0c00a2c00 0x7fd0c00a50c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8031440 tx=0x7fd0c80095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:07.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0d6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55582507c5d0 0x55582507ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42898/0 (socket says 172.21.15.172:42898) 2024-09-10T08:56:07.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0d6ffd640 1 -- 172.21.15.172:0/3613935782 learned_addr learned my addr 172.21.15.172:0/3613935782 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:07.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0d77fe640 1 --2- 172.21.15.172:0/3613935782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507db40 0x55582507c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:07.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0d7fff640 1 --2- 172.21.15.172:0/3613935782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555825087370 0x555825089760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:07.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0d6ffd640 1 -- 172.21.15.172:0/3613935782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507db40 msgr2=0x55582507c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:07.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0d6ffd640 1 --2- 172.21.15.172:0/3613935782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507db40 0x55582507c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0d6ffd640 1 -- 172.21.15.172:0/3613935782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555825087370 msgr2=0x555825089760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:07.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0d6ffd640 1 --2- 172.21.15.172:0/3613935782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555825087370 0x555825089760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.473+0000 7fd0d6ffd640 1 -- 172.21.15.172:0/3613935782 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555824f6e3a0 con 0x55582507c5d0 2024-09-10T08:56:07.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.474+0000 7fd0d6ffd640 1 --2- 172.21.15.172:0/3613935782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55582507c5d0 0x55582507ca10 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc097d10 tx=0x7fd0cc09add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0ee445ef21948f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:07.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.474+0000 7fd0bdffb640 1 -- 172.21.15.172:0/3613935782 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0a3fb0 con 0x55582507c5d0 2024-09-10T08:56:07.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.474+0000 7fd0bdffb640 1 -- 172.21.15.172:0/3613935782 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0cc099640 con 0x55582507c5d0 2024-09-10T08:56:07.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.474+0000 7fd0bdffb640 1 -- 172.21.15.172:0/3613935782 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0972f0 con 0x55582507c5d0 2024-09-10T08:56:07.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.475+0000 7fd0dfe76740 1 -- 172.21.15.172:0/3613935782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55582507c5d0 msgr2=0x55582507ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:07.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.475+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/3613935782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55582507c5d0 0x55582507ca10 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc097d10 tx=0x7fd0cc09add0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.475+0000 7fd0dfe76740 1 -- 172.21.15.172:0/3613935782 shutdown_connections 2024-09-10T08:56:07.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.475+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/3613935782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555825087370 0x555825089760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.475+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/3613935782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55582507c5d0 0x55582507ca10 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.475+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/3613935782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507db40 0x55582507c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.475+0000 7fd0dfe76740 1 -- 172.21.15.172:0/3613935782 >> 172.21.15.172:0/3613935782 conn(0x555825045310 msgr2=0x555825078c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:07.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.475+0000 7fd0dfe76740 1 -- 172.21.15.172:0/3613935782 shutdown_connections 2024-09-10T08:56:07.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.475+0000 7fd0dfe76740 1 -- 172.21.15.172:0/3613935782 wait complete. 2024-09-10T08:56:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.475+0000 7fd0dfe76740 1 -- start start 2024-09-10T08:56:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.475+0000 7fd0dfe76740 1 Processor -- start 2024-09-10T08:56:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0dfe76740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507c5d0 0x555825086dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0dfe76740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55582507db40 0x555825084080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0dfe76740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555825087370 0x555825084810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0d6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55582507db40 0x555825084080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0dfe76740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55582508a010 con 0x55582507db40 2024-09-10T08:56:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0d6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55582507db40 0x555825084080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42916/0 (socket says 172.21.15.172:42916) 2024-09-10T08:56:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0dfe76740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555824f9d060 con 0x555825087370 2024-09-10T08:56:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0dfe76740 1 -- 172.21.15.172:0/1848090519 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55582508a190 con 0x55582507c5d0 2024-09-10T08:56:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0d6ffd640 1 -- 172.21.15.172:0/1848090519 learned_addr learned my addr 172.21.15.172:0/1848090519 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0dfe76740 10 client.0 _ll_register_callbacks cb 0x555824cf08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:07.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0d77fe640 1 --2- 172.21.15.172:0/1848090519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507c5d0 0x555825086dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:07.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0d6ffd640 1 -- 172.21.15.172:0/1848090519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507c5d0 msgr2=0x555825086dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:07.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0d6ffd640 1 --2- 172.21.15.172:0/1848090519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507c5d0 0x555825086dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0d6ffd640 1 -- 172.21.15.172:0/1848090519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555825087370 msgr2=0x555825084810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:07.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0d6ffd640 1 --2- 172.21.15.172:0/1848090519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555825087370 0x555825084810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0d6ffd640 1 -- 172.21.15.172:0/1848090519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555824d533c0 con 0x55582507db40 2024-09-10T08:56:07.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0dfe76740 1 -- 172.21.15.172:0/1848090519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0cc099640 con 0x55582507db40 2024-09-10T08:56:07.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0d6ffd640 1 --2- 172.21.15.172:0/1848090519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55582507db40 0x555825084080 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc005170 tx=0x7fd0cc0045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39efa78361ff753d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:07.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0b6ffd640 10 client.18384 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:07.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.476+0000 7fd0b6ffd640 1 -- 172.21.15.172:0/1848090519 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0b9060 con 0x55582507db40 2024-09-10T08:56:07.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.477+0000 7fd0b6ffd640 1 -- 172.21.15.172:0/1848090519 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd0cc0979b0 con 0x55582507db40 2024-09-10T08:56:07.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.477+0000 7fd0b6ffd640 1 -- 172.21.15.172:0/1848090519 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd0cc0a3fb0 con 0x55582507db40 2024-09-10T08:56:07.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.477+0000 7fd0dfe76740 10 client.18384 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:07.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.477+0000 7fd0dfe76740 1 -- 172.21.15.172:0/1848090519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=37}) -- 0x555825095a50 con 0x55582507db40 2024-09-10T08:56:07.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.477+0000 7fd0b6ffd640 1 -- 172.21.15.172:0/1848090519 <== mon.0 v2:172.21.15.172:3300/0 4 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fd0cc09a790 con 0x55582507db40 2024-09-10T08:56:07.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.477+0000 7fd0dfe76740 10 client.18384 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:07.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.477+0000 7fd0dfe76740 10 client.18384 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:56:07.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.477+0000 7fd0dfe76740 20 client.18384 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:07.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.477+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/1848090519 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x555825096da0 0x555825099190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.477+0000 7fd0dfe76740 4 client.18384 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:56:07.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.477+0000 7fd0dfe76740 1 -- 172.21.15.172:0/1848090519 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555824fef860 con 0x555825096da0 2024-09-10T08:56:07.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.478+0000 7fd0d77fe640 1 --2- 172.21.15.172:0/1848090519 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x555825096da0 0x555825099190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:07.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.478+0000 7fd0d77fe640 1 --2- 172.21.15.172:0/1848090519 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x555825096da0 0x555825099190 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f46ccc7fd41c69db server_cookie=810a679e200a1d94 in_seq=0 out_seq=0 2024-09-10T08:56:07.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.478+0000 7fd0b6ffd640 10 client.18384 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:56:07.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.479+0000 7fd0b6ffd640 1 -- 172.21.15.172:0/1848090519 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555824fef860 con 0x555825096da0 2024-09-10T08:56:07.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.479+0000 7fd0b6ffd640 10 client.18384 handle_command_reply: tid=0 2024-09-10T08:56:07.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.479+0000 7fd0dfe76740 1 client.18384 shutdown 2024-09-10T08:56:07.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.479+0000 7fd0dfe76740 1 -- 172.21.15.172:0/1848090519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55582507db40 msgr2=0x555825084080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:07.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.479+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/1848090519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55582507db40 0x555825084080 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc005170 tx=0x7fd0cc0045b0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.479+0000 7fd0dfe76740 1 -- 172.21.15.172:0/1848090519 shutdown_connections 2024-09-10T08:56:07.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.479+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/1848090519 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x555825096da0 0x555825099190 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.479+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/1848090519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555825087370 0x555825084810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.479+0000 7fd0b6ffd640 0 client.18384 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:56:07.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.479+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/1848090519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55582507db40 0x555825084080 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.479+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/1848090519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507c5d0 0x555825086dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.479+0000 7fd0dfe76740 1 -- 172.21.15.172:0/1848090519 >> 172.21.15.172:0/1848090519 conn(0x555825045310 msgr2=0x55582507e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:07.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.479+0000 7fd0dfe76740 1 -- 172.21.15.172:0/1848090519 shutdown_connections 2024-09-10T08:56:07.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.479+0000 7fd0dfe76740 1 -- 172.21.15.172:0/1848090519 wait complete. 2024-09-10T08:56:07.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.479+0000 7fd0dfe76740 20 client.18384 trim_cache size 0 max 16384 2024-09-10T08:56:07.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.495+0000 7fd0dfe76740 1 Processor -- start 2024-09-10T08:56:07.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.495+0000 7fd0dfe76740 1 -- start start 2024-09-10T08:56:07.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.495+0000 7fd0dfe76740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507c5d0 0x55582507d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.495+0000 7fd0dfe76740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582507db40 0x555825083940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.495+0000 7fd0dfe76740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555825087370 0x5558250786e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.495+0000 7fd0dfe76740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d81b0df0 con 0x555825087370 2024-09-10T08:56:07.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.495+0000 7fd0dfe76740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d81b2930 con 0x55582507db40 2024-09-10T08:56:07.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.495+0000 7fd0dfe76740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d81b14c0 con 0x55582507c5d0 2024-09-10T08:56:07.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.495+0000 7fd0d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555825087370 0x5558250786e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:07.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.495+0000 7fd0d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555825087370 0x5558250786e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42918/0 (socket says 172.21.15.172:42918) 2024-09-10T08:56:07.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.495+0000 7fd0d7fff640 1 -- 172.21.15.172:0/3791047216 learned_addr learned my addr 172.21.15.172:0/3791047216 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:07.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.495+0000 7fd0d6ffd640 1 --2- 172.21.15.172:0/3791047216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582507db40 0x555825083940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:07.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.495+0000 7fd0d77fe640 1 --2- 172.21.15.172:0/3791047216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507c5d0 0x55582507d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:07.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.495+0000 7fd0d7fff640 1 -- 172.21.15.172:0/3791047216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507c5d0 msgr2=0x55582507d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:07.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.495+0000 7fd0d7fff640 1 --2- 172.21.15.172:0/3791047216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507c5d0 0x55582507d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.496+0000 7fd0d7fff640 1 -- 172.21.15.172:0/3791047216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582507db40 msgr2=0x555825083940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:07.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.496+0000 7fd0d7fff640 1 --2- 172.21.15.172:0/3791047216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582507db40 0x555825083940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.496+0000 7fd0d7fff640 1 -- 172.21.15.172:0/3791047216 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558250825d0 con 0x555825087370 2024-09-10T08:56:07.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.496+0000 7fd0d7fff640 1 --2- 172.21.15.172:0/3791047216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555825087370 0x5558250786e0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0009e10 tx=0x7fd0d00127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733e98855826258c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:07.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.496+0000 7fd0bd7fa640 1 -- 172.21.15.172:0/3791047216 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0004d20 con 0x555825087370 2024-09-10T08:56:07.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.496+0000 7fd0bd7fa640 1 -- 172.21.15.172:0/3791047216 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0d00054d0 con 0x555825087370 2024-09-10T08:56:07.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.496+0000 7fd0bd7fa640 1 -- 172.21.15.172:0/3791047216 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0010940 con 0x555825087370 2024-09-10T08:56:07.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.497+0000 7fd0dfe76740 1 -- 172.21.15.172:0/3791047216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555825087370 msgr2=0x5558250786e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:07.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.497+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/3791047216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555825087370 0x5558250786e0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0009e10 tx=0x7fd0d00127c0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.497+0000 7fd0dfe76740 1 -- 172.21.15.172:0/3791047216 shutdown_connections 2024-09-10T08:56:07.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.497+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/3791047216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555825087370 0x5558250786e0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.497+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/3791047216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582507db40 0x555825083940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.497+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/3791047216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507c5d0 0x55582507d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.497+0000 7fd0dfe76740 1 -- 172.21.15.172:0/3791047216 >> 172.21.15.172:0/3791047216 conn(0x555825045310 msgr2=0x555825073a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:07.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.497+0000 7fd0dfe76740 1 -- 172.21.15.172:0/3791047216 shutdown_connections 2024-09-10T08:56:07.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.497+0000 7fd0dfe76740 1 -- 172.21.15.172:0/3791047216 wait complete. 2024-09-10T08:56:07.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.497+0000 7fd0dfe76740 1 -- start start 2024-09-10T08:56:07.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0dfe76740 1 Processor -- start 2024-09-10T08:56:07.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0dfe76740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582507c5d0 0x55582508f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0dfe76740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507db40 0x555825080bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0dfe76740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555825087370 0x555825081130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0dfe76740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d81b2260 con 0x555825087370 2024-09-10T08:56:07.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0dfe76740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d81b1b90 con 0x55582507c5d0 2024-09-10T08:56:07.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0dfe76740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d81aedd0 con 0x55582507db40 2024-09-10T08:56:07.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0dfe76740 10 client.0 _ll_register_callbacks cb 0x7fd0d0009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:07.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555825087370 0x555825081130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:07.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555825087370 0x555825081130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42930/0 (socket says 172.21.15.172:42930) 2024-09-10T08:56:07.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0d7fff640 1 -- 172.21.15.172:0/2525240216 learned_addr learned my addr 172.21.15.172:0/2525240216 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:07.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0d77fe640 1 --2- 172.21.15.172:0/2525240216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582507c5d0 0x55582508f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:07.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0d6ffd640 1 --2- 172.21.15.172:0/2525240216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507db40 0x555825080bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:07.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0d7fff640 1 -- 172.21.15.172:0/2525240216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507db40 msgr2=0x555825080bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:07.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0d7fff640 1 --2- 172.21.15.172:0/2525240216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507db40 0x555825080bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0d7fff640 1 -- 172.21.15.172:0/2525240216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582507c5d0 msgr2=0x55582508f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:07.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0d7fff640 1 --2- 172.21.15.172:0/2525240216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582507c5d0 0x55582508f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0d7fff640 1 -- 172.21.15.172:0/2525240216 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555825081720 con 0x555825087370 2024-09-10T08:56:07.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.498+0000 7fd0d77fe640 1 --2- 172.21.15.172:0/2525240216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582507c5d0 0x55582508f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:07.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.499+0000 7fd0d7fff640 1 --2- 172.21.15.172:0/2525240216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555825087370 0x555825081130 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0009fa0 tx=0x7fd0d00059a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0ddec25303c37b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:07.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.499+0000 7fd0b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:07.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.499+0000 7fd0b6ffd640 1 -- 172.21.15.172:0/2525240216 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0029020 con 0x555825087370 2024-09-10T08:56:07.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.499+0000 7fd0b6ffd640 1 -- 172.21.15.172:0/2525240216 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd0d00049e0 con 0x555825087370 2024-09-10T08:56:07.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.499+0000 7fd0dfe76740 1 -- 172.21.15.172:0/2525240216 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0d00054d0 con 0x555825087370 2024-09-10T08:56:07.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.499+0000 7fd0b6ffd640 1 -- 172.21.15.172:0/2525240216 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd0d0034050 con 0x555825087370 2024-09-10T08:56:07.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.499+0000 7fd0dfe76740 10 client.18390 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:07.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.499+0000 7fd0dfe76740 1 -- 172.21.15.172:0/2525240216 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5558251016f0 con 0x555825087370 2024-09-10T08:56:07.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.499+0000 7fd0b6ffd640 1 -- 172.21.15.172:0/2525240216 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0d00054d0 con 0x555825087370 2024-09-10T08:56:07.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.500+0000 7fd0b6ffd640 1 -- 172.21.15.172:0/2525240216 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fd0d0019070 con 0x555825087370 2024-09-10T08:56:07.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.500+0000 7fd0dfe76740 10 client.18390 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:07.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.500+0000 7fd0dfe76740 10 client.18390 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:56:07.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.500+0000 7fd0dfe76740 20 client.18390 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:07.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.500+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/2525240216 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x555825096da0 0x555825103de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:07.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.500+0000 7fd0dfe76740 4 client.18390 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:56:07.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.500+0000 7fd0dfe76740 1 -- 172.21.15.172:0/2525240216 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555824fe9d70 con 0x555825096da0 2024-09-10T08:56:07.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.500+0000 7fd0d77fe640 1 --2- 172.21.15.172:0/2525240216 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x555825096da0 0x555825103de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:07.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.500+0000 7fd0b6ffd640 10 client.18390 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:56:07.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.500+0000 7fd0d77fe640 1 --2- 172.21.15.172:0/2525240216 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x555825096da0 0x555825103de0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35cf5ae8a9763cd9 server_cookie=a565d2bb6a4751e6 in_seq=0 out_seq=0 2024-09-10T08:56:07.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.500+0000 7fd0b6ffd640 1 -- 172.21.15.172:0/2525240216 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555824fe9d70 con 0x555825096da0 2024-09-10T08:56:07.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.500+0000 7fd0b6ffd640 10 client.18390 handle_command_reply: tid=0 2024-09-10T08:56:07.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.500+0000 7fd0dfe76740 1 client.18390 shutdown 2024-09-10T08:56:07.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.501+0000 7fd0dfe76740 1 -- 172.21.15.172:0/2525240216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555825087370 msgr2=0x555825081130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:07.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.501+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/2525240216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555825087370 0x555825081130 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0009fa0 tx=0x7fd0d00059a0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.501+0000 7fd0dfe76740 1 -- 172.21.15.172:0/2525240216 shutdown_connections 2024-09-10T08:56:07.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.501+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/2525240216 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x555825096da0 0x555825103de0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.501+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/2525240216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555825087370 0x555825081130 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.501+0000 7fd0b6ffd640 0 client.18390 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:56:07.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.501+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/2525240216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55582507db40 0x555825080bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.501+0000 7fd0dfe76740 1 --2- 172.21.15.172:0/2525240216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55582507c5d0 0x55582508f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.501+0000 7fd0dfe76740 1 -- 172.21.15.172:0/2525240216 >> 172.21.15.172:0/2525240216 conn(0x555825045310 msgr2=0x555825078650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:07.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.501+0000 7fd0dfe76740 1 -- 172.21.15.172:0/2525240216 shutdown_connections 2024-09-10T08:56:07.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.501+0000 7fd0dfe76740 1 -- 172.21.15.172:0/2525240216 wait complete. 2024-09-10T08:56:07.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.501+0000 7fd0dfe76740 20 client.18390 trim_cache size 0 max 16384 2024-09-10T08:56:07.507 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:56:07.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.502+0000 7fd0dda9d640 1 -- 172.21.15.172:0/1608887230 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd0c00a2c00 msgr2=0x7fd0c00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:07.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.502+0000 7fd0dda9d640 1 --2- 172.21.15.172:0/1608887230 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd0c00a2c00 0x7fd0c00a50c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8031440 tx=0x7fd0c80095f0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.502+0000 7fd0dda9d640 1 -- 172.21.15.172:0/1608887230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0d8100aa0 msgr2=0x7fd0d8101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:07.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.502+0000 7fd0dda9d640 1 --2- 172.21.15.172:0/1608887230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0d8100aa0 0x7fd0d8101820 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00d950 tx=0x7fd0cc00de10 comp rx=0 tx=0).stop 2024-09-10T08:56:07.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.502+0000 7fd0dda9d640 1 -- 172.21.15.172:0/1608887230 shutdown_connections 2024-09-10T08:56:07.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.502+0000 7fd0dda9d640 1 --2- 172.21.15.172:0/1608887230 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd0c00a2c00 0x7fd0c00a50c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.502+0000 7fd0dda9d640 1 --2- 172.21.15.172:0/1608887230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0d8106b80 0x7fd0d8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.502+0000 7fd0dda9d640 1 --2- 172.21.15.172:0/1608887230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0d8100aa0 0x7fd0d8101820 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.502+0000 7fd0dda9d640 1 --2- 172.21.15.172:0/1608887230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0d8100100 0x7fd0d81012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:07.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.502+0000 7fd0dda9d640 1 -- 172.21.15.172:0/1608887230 >> 172.21.15.172:0/1608887230 conn(0x7fd0d8077230 msgr2=0x7fd0d8104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:07.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.502+0000 7fd0dda9d640 1 -- 172.21.15.172:0/1608887230 shutdown_connections 2024-09-10T08:56:07.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:07.502+0000 7fd0dda9d640 1 -- 172.21.15.172:0/1608887230 wait complete. 2024-09-10T08:56:08.376 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:56:08.680 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:56:08.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:08 smithi174 ceph-mon[34068]: pgmap v647: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:56:08.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:08 smithi191 ceph-mon[35314]: pgmap v647: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:56:08.803 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:08 smithi172 ceph-mon[30854]: pgmap v647: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:56:09.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.204+0000 7fb659ebb640 1 -- 172.21.15.172:0/2934869195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb654106f10 msgr2=0x7fb65410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:09.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.204+0000 7fb659ebb640 1 --2- 172.21.15.172:0/2934869195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb654106f10 0x7fb65410ea90 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009ca0 tx=0x7fb64c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.204+0000 7fb659ebb640 1 -- 172.21.15.172:0/2934869195 shutdown_connections 2024-09-10T08:56:09.209 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.204+0000 7fb659ebb640 1 --2- 172.21.15.172:0/2934869195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb654106f10 0x7fb65410ea90 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.204+0000 7fb659ebb640 1 --2- 172.21.15.172:0/2934869195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb654102790 0x7fb6541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.204+0000 7fb659ebb640 1 --2- 172.21.15.172:0/2934869195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb654101de0 0x7fb6541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.204+0000 7fb659ebb640 1 -- 172.21.15.172:0/2934869195 >> 172.21.15.172:0/2934869195 conn(0x7fb6540fd8c0 msgr2=0x7fb6540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:09.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.204+0000 7fb659ebb640 1 -- 172.21.15.172:0/2934869195 shutdown_connections 2024-09-10T08:56:09.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.204+0000 7fb659ebb640 1 -- 172.21.15.172:0/2934869195 wait complete. 2024-09-10T08:56:09.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.205+0000 7fb659ebb640 1 Processor -- start 2024-09-10T08:56:09.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.205+0000 7fb659ebb640 1 -- start start 2024-09-10T08:56:09.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb659ebb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb654101de0 0x7fb6541a3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb659ebb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb654102790 0x7fb6541a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb659ebb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb654106f10 0x7fb65410db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb659ebb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb654111e20 con 0x7fb654101de0 2024-09-10T08:56:09.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb659ebb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb654111ca0 con 0x7fb654106f10 2024-09-10T08:56:09.210 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb659ebb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb654111fa0 con 0x7fb654102790 2024-09-10T08:56:09.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb6537fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb654101de0 0x7fb6541a3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb6537fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb654101de0 0x7fb6541a3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42954/0 (socket says 172.21.15.172:42954) 2024-09-10T08:56:09.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb6537fe640 1 -- 172.21.15.172:0/699623026 learned_addr learned my addr 172.21.15.172:0/699623026 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:09.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb652ffd640 1 --2- 172.21.15.172:0/699623026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb654102790 0x7fb6541a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb653fff640 1 --2- 172.21.15.172:0/699623026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb654106f10 0x7fb65410db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb6537fe640 1 -- 172.21.15.172:0/699623026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb654102790 msgr2=0x7fb6541a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:09.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb6537fe640 1 --2- 172.21.15.172:0/699623026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb654102790 0x7fb6541a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb6537fe640 1 -- 172.21.15.172:0/699623026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb654106f10 msgr2=0x7fb65410db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:09.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb6537fe640 1 --2- 172.21.15.172:0/699623026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb654106f10 0x7fb65410db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb6537fe640 1 -- 172.21.15.172:0/699623026 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65410e260 con 0x7fb654101de0 2024-09-10T08:56:09.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb652ffd640 1 --2- 172.21.15.172:0/699623026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb654102790 0x7fb6541a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:09.211 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb6537fe640 1 --2- 172.21.15.172:0/699623026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb654101de0 0x7fb6541a3500 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fb64400c890 tx=0x7fb64400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:09.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb650ff9640 1 -- 172.21.15.172:0/699623026 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644004220 con 0x7fb654101de0 2024-09-10T08:56:09.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb650ff9640 1 -- 172.21.15.172:0/699623026 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6440043c0 con 0x7fb654101de0 2024-09-10T08:56:09.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.206+0000 7fb650ff9640 1 -- 172.21.15.172:0/699623026 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644004a00 con 0x7fb654101de0 2024-09-10T08:56:09.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.207+0000 7fb659ebb640 1 -- 172.21.15.172:0/699623026 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65410e550 con 0x7fb654101de0 2024-09-10T08:56:09.212 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.207+0000 7fb659ebb640 1 -- 172.21.15.172:0/699623026 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6541ab430 con 0x7fb654101de0 2024-09-10T08:56:09.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.209+0000 7fb65c294740 1 Processor -- start 2024-09-10T08:56:09.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.209+0000 7fb65c294740 1 -- start start 2024-09-10T08:56:09.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.209+0000 7fb650ff9640 1 -- 172.21.15.172:0/699623026 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb644004ba0 con 0x7fb654101de0 2024-09-10T08:56:09.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb65c294740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd986b40 0x5614dd985090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb65c294740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd9855d0 0x5614dd985a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb65c294740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd990370 0x5614dd992760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb65c294740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5614dd8eb2f0 con 0x5614dd990370 2024-09-10T08:56:09.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb65c294740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5614dd892cf0 con 0x5614dd986b40 2024-09-10T08:56:09.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb65c294740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5614dd8e72b0 con 0x5614dd9855d0 2024-09-10T08:56:09.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb652ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd990370 0x5614dd992760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb652ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd990370 0x5614dd992760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42958/0 (socket says 172.21.15.172:42958) 2024-09-10T08:56:09.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb652ffd640 1 -- 172.21.15.172:0/3004022395 learned_addr learned my addr 172.21.15.172:0/3004022395 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:09.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb653fff640 1 --2- 172.21.15.172:0/3004022395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd986b40 0x5614dd985090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb6537fe640 1 --2- 172.21.15.172:0/3004022395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd9855d0 0x5614dd985a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb652ffd640 1 -- 172.21.15.172:0/3004022395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd9855d0 msgr2=0x5614dd985a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:09.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb650ff9640 1 --2- 172.21.15.172:0/699623026 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb63c0a2b80 0x7fb63c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb652ffd640 1 --2- 172.21.15.172:0/3004022395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd9855d0 0x5614dd985a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb652ffd640 1 -- 172.21.15.172:0/3004022395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd986b40 msgr2=0x5614dd985090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:09.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb652ffd640 1 --2- 172.21.15.172:0/3004022395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd986b40 0x5614dd985090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb652ffd640 1 -- 172.21.15.172:0/3004022395 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614dd8773a0 con 0x5614dd990370 2024-09-10T08:56:09.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb6537fe640 1 --2- 172.21.15.172:0/3004022395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd9855d0 0x5614dd985a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:09.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.210+0000 7fb650ff9640 1 -- 172.21.15.172:0/699623026 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb6440cf0a0 con 0x7fb654101de0 2024-09-10T08:56:09.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.211+0000 7fb653fff640 1 --2- 172.21.15.172:0/699623026 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb63c0a2b80 0x7fb63c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.211+0000 7fb652ffd640 1 --2- 172.21.15.172:0/3004022395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd990370 0x5614dd992760 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fb64800bc60 tx=0x7fb64800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7499ecc47be612 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:09.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.211+0000 7fb639ffb640 1 -- 172.21.15.172:0/3004022395 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648015070 con 0x5614dd990370 2024-09-10T08:56:09.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.211+0000 7fb639ffb640 1 -- 172.21.15.172:0/3004022395 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6480118f0 con 0x5614dd990370 2024-09-10T08:56:09.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.211+0000 7fb639ffb640 1 -- 172.21.15.172:0/3004022395 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648010750 con 0x5614dd990370 2024-09-10T08:56:09.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.212+0000 7fb65c294740 1 -- 172.21.15.172:0/3004022395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd990370 msgr2=0x5614dd992760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:09.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.212+0000 7fb653fff640 1 --2- 172.21.15.172:0/699623026 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb63c0a2b80 0x7fb63c0a5040 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb64c005fe0 tx=0x7fb64c02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:09.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.212+0000 7fb65c294740 1 --2- 172.21.15.172:0/3004022395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd990370 0x5614dd992760 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fb64800bc60 tx=0x7fb64800f730 comp rx=0 tx=0).stop 2024-09-10T08:56:09.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.212+0000 7fb65c294740 1 -- 172.21.15.172:0/3004022395 shutdown_connections 2024-09-10T08:56:09.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.212+0000 7fb65c294740 1 --2- 172.21.15.172:0/3004022395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd990370 0x5614dd992760 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.212+0000 7fb65c294740 1 --2- 172.21.15.172:0/3004022395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd9855d0 0x5614dd985a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.212+0000 7fb65c294740 1 --2- 172.21.15.172:0/3004022395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd986b40 0x5614dd985090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.212+0000 7fb65c294740 1 -- 172.21.15.172:0/3004022395 >> 172.21.15.172:0/3004022395 conn(0x5614dd94e310 msgr2=0x5614dd981c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:09.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.212+0000 7fb65c294740 1 -- 172.21.15.172:0/3004022395 shutdown_connections 2024-09-10T08:56:09.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.212+0000 7fb65c294740 1 -- 172.21.15.172:0/3004022395 wait complete. 2024-09-10T08:56:09.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.212+0000 7fb65c294740 1 -- start start 2024-09-10T08:56:09.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb65c294740 1 Processor -- start 2024-09-10T08:56:09.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb65c294740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd9855d0 0x5614dd98fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb65c294740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd986b40 0x5614dd98d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb65c294740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd990370 0x5614dd98d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb65c294740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5614dd993070 con 0x5614dd986b40 2024-09-10T08:56:09.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb65c294740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5614dd8a6060 con 0x5614dd9855d0 2024-09-10T08:56:09.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb65c294740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5614dd9931f0 con 0x5614dd990370 2024-09-10T08:56:09.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb65c294740 10 client.0 _ll_register_callbacks cb 0x5614dd5f98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:09.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb652ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd986b40 0x5614dd98d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb652ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd986b40 0x5614dd98d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42972/0 (socket says 172.21.15.172:42972) 2024-09-10T08:56:09.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb652ffd640 1 -- 172.21.15.172:0/833769998 learned_addr learned my addr 172.21.15.172:0/833769998 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:09.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb6537fe640 1 --2- 172.21.15.172:0/833769998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd9855d0 0x5614dd98fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb653fff640 1 --2- 172.21.15.172:0/833769998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd990370 0x5614dd98d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb652ffd640 1 -- 172.21.15.172:0/833769998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd990370 msgr2=0x5614dd98d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:09.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb652ffd640 1 --2- 172.21.15.172:0/833769998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd990370 0x5614dd98d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb652ffd640 1 -- 172.21.15.172:0/833769998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd9855d0 msgr2=0x5614dd98fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:09.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb652ffd640 1 --2- 172.21.15.172:0/833769998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd9855d0 0x5614dd98fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb652ffd640 1 -- 172.21.15.172:0/833769998 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614dd65c3c0 con 0x5614dd986b40 2024-09-10T08:56:09.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb653fff640 1 --2- 172.21.15.172:0/833769998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd990370 0x5614dd98d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:56:09.223 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.213+0000 7fb6537fe640 1 --2- 172.21.15.172:0/833769998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd9855d0 0x5614dd98fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:09.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.214+0000 7fb652ffd640 1 --2- 172.21.15.172:0/833769998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd986b40 0x5614dd98d010 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb64800c010 tx=0x7fb64800b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95edeb191616258f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:09.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.214+0000 7fb632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:09.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.214+0000 7fb632ffd640 1 -- 172.21.15.172:0/833769998 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648015050 con 0x5614dd986b40 2024-09-10T08:56:09.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.214+0000 7fb632ffd640 1 -- 172.21.15.172:0/833769998 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb648011440 con 0x5614dd986b40 2024-09-10T08:56:09.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.214+0000 7fb65c294740 1 -- 172.21.15.172:0/833769998 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6480118f0 con 0x5614dd986b40 2024-09-10T08:56:09.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.214+0000 7fb632ffd640 1 -- 172.21.15.172:0/833769998 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb64802f050 con 0x5614dd986b40 2024-09-10T08:56:09.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.214+0000 7fb65c294740 10 client.18402 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:09.224 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.214+0000 7fb65c294740 1 -- 172.21.15.172:0/833769998 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5614dd99eb10 con 0x5614dd986b40 2024-09-10T08:56:09.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.214+0000 7fb632ffd640 1 -- 172.21.15.172:0/833769998 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb6480118f0 con 0x5614dd986b40 2024-09-10T08:56:09.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.215+0000 7fb632ffd640 1 -- 172.21.15.172:0/833769998 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fb648003860 con 0x5614dd986b40 2024-09-10T08:56:09.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.215+0000 7fb65c294740 10 client.18402 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:09.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.215+0000 7fb65c294740 10 client.18402 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:56:09.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.215+0000 7fb65c294740 20 client.18402 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:09.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.215+0000 7fb65c294740 1 --2- 172.21.15.172:0/833769998 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5614dd9a0190 0x5614dd9a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.215+0000 7fb65c294740 4 client.18402 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:56:09.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.215+0000 7fb65c294740 1 -- 172.21.15.172:0/833769998 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614dd8f8860 con 0x5614dd9a0190 2024-09-10T08:56:09.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.215+0000 7fb6537fe640 1 --2- 172.21.15.172:0/833769998 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5614dd9a0190 0x5614dd9a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.215+0000 7fb632ffd640 10 client.18402 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:56:09.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.215+0000 7fb6537fe640 1 --2- 172.21.15.172:0/833769998 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5614dd9a0190 0x5614dd9a2580 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1dffba26082de30e server_cookie=807c74ed53d162d8 in_seq=0 out_seq=0 2024-09-10T08:56:09.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.216+0000 7fb632ffd640 1 -- 172.21.15.172:0/833769998 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614dd8f8860 con 0x5614dd9a0190 2024-09-10T08:56:09.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.216+0000 7fb632ffd640 10 client.18402 handle_command_reply: tid=0 2024-09-10T08:56:09.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.216+0000 7fb65c294740 1 client.18402 shutdown 2024-09-10T08:56:09.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.216+0000 7fb65c294740 1 -- 172.21.15.172:0/833769998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd986b40 msgr2=0x5614dd98d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:09.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.216+0000 7fb65c294740 1 --2- 172.21.15.172:0/833769998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd986b40 0x5614dd98d010 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb64800c010 tx=0x7fb64800b9d0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.216+0000 7fb65c294740 1 -- 172.21.15.172:0/833769998 shutdown_connections 2024-09-10T08:56:09.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.216+0000 7fb65c294740 1 --2- 172.21.15.172:0/833769998 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5614dd9a0190 0x5614dd9a2580 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.216+0000 7fb632ffd640 0 client.18402 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:56:09.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.216+0000 7fb65c294740 1 --2- 172.21.15.172:0/833769998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd990370 0x5614dd98d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.216+0000 7fb65c294740 1 --2- 172.21.15.172:0/833769998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd986b40 0x5614dd98d010 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.216+0000 7fb65c294740 1 --2- 172.21.15.172:0/833769998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd9855d0 0x5614dd98fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.216+0000 7fb65c294740 1 -- 172.21.15.172:0/833769998 >> 172.21.15.172:0/833769998 conn(0x5614dd94e310 msgr2=0x5614dd987960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:09.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.217+0000 7fb65c294740 1 -- 172.21.15.172:0/833769998 shutdown_connections 2024-09-10T08:56:09.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.217+0000 7fb65c294740 1 -- 172.21.15.172:0/833769998 wait complete. 2024-09-10T08:56:09.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.217+0000 7fb65c294740 20 client.18402 trim_cache size 0 max 16384 2024-09-10T08:56:09.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.233+0000 7fb65c294740 1 Processor -- start 2024-09-10T08:56:09.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.233+0000 7fb65c294740 1 -- start start 2024-09-10T08:56:09.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb65c294740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd9855d0 0x5614dd98cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb65c294740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd986b40 0x5614dd9863a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb65c294740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd990370 0x5614dd9a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb65c294740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541b51d0 con 0x5614dd986b40 2024-09-10T08:56:09.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb65c294740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541b6d10 con 0x5614dd990370 2024-09-10T08:56:09.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb65c294740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541b58a0 con 0x5614dd9855d0 2024-09-10T08:56:09.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb652ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd986b40 0x5614dd9863a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb652ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd986b40 0x5614dd9863a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42976/0 (socket says 172.21.15.172:42976) 2024-09-10T08:56:09.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb652ffd640 1 -- 172.21.15.172:0/3572733468 learned_addr learned my addr 172.21.15.172:0/3572733468 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:09.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb653fff640 1 --2- 172.21.15.172:0/3572733468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd990370 0x5614dd9a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb6537fe640 1 --2- 172.21.15.172:0/3572733468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd9855d0 0x5614dd98cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb652ffd640 1 -- 172.21.15.172:0/3572733468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd9855d0 msgr2=0x5614dd98cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:09.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb652ffd640 1 --2- 172.21.15.172:0/3572733468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd9855d0 0x5614dd98cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb652ffd640 1 -- 172.21.15.172:0/3572733468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd990370 msgr2=0x5614dd9a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb652ffd640 1 --2- 172.21.15.172:0/3572733468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd990370 0x5614dd9a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb652ffd640 1 -- 172.21.15.172:0/3572733468 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614dd9900d0 con 0x5614dd986b40 2024-09-10T08:56:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb653fff640 1 --2- 172.21.15.172:0/3572733468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd990370 0x5614dd9a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb652ffd640 1 --2- 172.21.15.172:0/3572733468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd986b40 0x5614dd9863a0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fb648003e20 tx=0x7fb648005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9506190f3498ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.234+0000 7fb6397fa640 1 -- 172.21.15.172:0/3572733468 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648038070 con 0x5614dd986b40 2024-09-10T08:56:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.235+0000 7fb6397fa640 1 -- 172.21.15.172:0/3572733468 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb64800bd20 con 0x5614dd986b40 2024-09-10T08:56:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.235+0000 7fb6397fa640 1 -- 172.21.15.172:0/3572733468 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648015070 con 0x5614dd986b40 2024-09-10T08:56:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.235+0000 7fb65c294740 1 -- 172.21.15.172:0/3572733468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd986b40 msgr2=0x5614dd9863a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.235+0000 7fb65c294740 1 --2- 172.21.15.172:0/3572733468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd986b40 0x5614dd9863a0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fb648003e20 tx=0x7fb648005710 comp rx=0 tx=0).stop 2024-09-10T08:56:09.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.235+0000 7fb65c294740 1 -- 172.21.15.172:0/3572733468 shutdown_connections 2024-09-10T08:56:09.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.235+0000 7fb65c294740 1 --2- 172.21.15.172:0/3572733468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd990370 0x5614dd9a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.235+0000 7fb65c294740 1 --2- 172.21.15.172:0/3572733468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd986b40 0x5614dd9863a0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.235+0000 7fb65c294740 1 --2- 172.21.15.172:0/3572733468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd9855d0 0x5614dd98cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.235+0000 7fb65c294740 1 -- 172.21.15.172:0/3572733468 >> 172.21.15.172:0/3572733468 conn(0x5614dd94e310 msgr2=0x5614dd98c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:09.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.235+0000 7fb65c294740 1 -- 172.21.15.172:0/3572733468 shutdown_connections 2024-09-10T08:56:09.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.235+0000 7fb65c294740 1 -- 172.21.15.172:0/3572733468 wait complete. 2024-09-10T08:56:09.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.236+0000 7fb65c294740 1 -- start start 2024-09-10T08:56:09.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.236+0000 7fb65c294740 1 Processor -- start 2024-09-10T08:56:09.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.236+0000 7fb65c294740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd9855d0 0x5614dd9a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.236+0000 7fb65c294740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd986b40 0x5614dd9a3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.236+0000 7fb65c294740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd990370 0x5614dd9a3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.236+0000 7fb65c294740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541b6640 con 0x5614dd9855d0 2024-09-10T08:56:09.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.236+0000 7fb65c294740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541b5f70 con 0x5614dd990370 2024-09-10T08:56:09.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.236+0000 7fb65c294740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541b31b0 con 0x5614dd986b40 2024-09-10T08:56:09.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.236+0000 7fb65c294740 10 client.0 _ll_register_callbacks cb 0x7fb64800bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:09.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.237+0000 7fb6537fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd9855d0 0x5614dd9a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.237+0000 7fb6537fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd9855d0 0x5614dd9a5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42980/0 (socket says 172.21.15.172:42980) 2024-09-10T08:56:09.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.237+0000 7fb6537fe640 1 -- 172.21.15.172:0/2014989220 learned_addr learned my addr 172.21.15.172:0/2014989220 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:09.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.237+0000 7fb653fff640 1 --2- 172.21.15.172:0/2014989220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd990370 0x5614dd9a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.237+0000 7fb652ffd640 1 --2- 172.21.15.172:0/2014989220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd986b40 0x5614dd9a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.237+0000 7fb6537fe640 1 -- 172.21.15.172:0/2014989220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd986b40 msgr2=0x5614dd9a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:09.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.237+0000 7fb6537fe640 1 --2- 172.21.15.172:0/2014989220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd986b40 0x5614dd9a3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.237+0000 7fb6537fe640 1 -- 172.21.15.172:0/2014989220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd990370 msgr2=0x5614dd9a3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:09.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.237+0000 7fb6537fe640 1 --2- 172.21.15.172:0/2014989220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd990370 0x5614dd9a3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.237+0000 7fb6537fe640 1 -- 172.21.15.172:0/2014989220 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614dd9a3d60 con 0x5614dd9855d0 2024-09-10T08:56:09.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.237+0000 7fb653fff640 1 --2- 172.21.15.172:0/2014989220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd990370 0x5614dd9a3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:09.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.237+0000 7fb652ffd640 1 --2- 172.21.15.172:0/2014989220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd986b40 0x5614dd9a3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:09.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.237+0000 7fb6537fe640 1 --2- 172.21.15.172:0/2014989220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd9855d0 0x5614dd9a5cd0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fb644005fd0 tx=0x7fb644086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d9c96dc50a9a82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:09.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.237+0000 7fb632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:09.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.237+0000 7fb632ffd640 1 -- 172.21.15.172:0/2014989220 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6440b4060 con 0x5614dd9855d0 2024-09-10T08:56:09.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.237+0000 7fb632ffd640 1 -- 172.21.15.172:0/2014989220 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb644099640 con 0x5614dd9855d0 2024-09-10T08:56:09.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.237+0000 7fb65c294740 1 -- 172.21.15.172:0/2014989220 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64800bd20 con 0x5614dd9855d0 2024-09-10T08:56:09.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.238+0000 7fb632ffd640 1 -- 172.21.15.172:0/2014989220 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb64401d040 con 0x5614dd9855d0 2024-09-10T08:56:09.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.238+0000 7fb65c294740 10 client.18408 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:09.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.238+0000 7fb65c294740 1 -- 172.21.15.172:0/2014989220 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5614dda0aaa0 con 0x5614dd9855d0 2024-09-10T08:56:09.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.238+0000 7fb632ffd640 1 -- 172.21.15.172:0/2014989220 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb64800bd20 con 0x5614dd9855d0 2024-09-10T08:56:09.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.238+0000 7fb632ffd640 1 -- 172.21.15.172:0/2014989220 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fb644097c10 con 0x5614dd9855d0 2024-09-10T08:56:09.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.238+0000 7fb65c294740 10 client.18408 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:09.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.238+0000 7fb65c294740 10 client.18408 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:56:09.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.238+0000 7fb65c294740 20 client.18408 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:09.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.238+0000 7fb65c294740 1 --2- 172.21.15.172:0/2014989220 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5614dda0b170 0x5614dda0d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:09.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.238+0000 7fb65c294740 4 client.18408 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:56:09.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.238+0000 7fb65c294740 1 -- 172.21.15.172:0/2014989220 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614dd8f2d70 con 0x5614dda0b170 2024-09-10T08:56:09.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.238+0000 7fb652ffd640 1 --2- 172.21.15.172:0/2014989220 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5614dda0b170 0x5614dda0d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:09.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.239+0000 7fb632ffd640 10 client.18408 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:56:09.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.239+0000 7fb652ffd640 1 --2- 172.21.15.172:0/2014989220 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5614dda0b170 0x5614dda0d560 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d054f6ef32ef1445 server_cookie=3111ed005b767947 in_seq=0 out_seq=0 2024-09-10T08:56:09.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.239+0000 7fb632ffd640 1 -- 172.21.15.172:0/2014989220 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614dd8f2d70 con 0x5614dda0b170 2024-09-10T08:56:09.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.239+0000 7fb632ffd640 10 client.18408 handle_command_reply: tid=0 2024-09-10T08:56:09.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.239+0000 7fb65c294740 1 client.18408 shutdown 2024-09-10T08:56:09.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.239+0000 7fb65c294740 1 -- 172.21.15.172:0/2014989220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd9855d0 msgr2=0x5614dd9a5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:09.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.239+0000 7fb65c294740 1 --2- 172.21.15.172:0/2014989220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd9855d0 0x5614dd9a5cd0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fb644005fd0 tx=0x7fb644086040 comp rx=0 tx=0).stop 2024-09-10T08:56:09.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.239+0000 7fb65c294740 1 -- 172.21.15.172:0/2014989220 shutdown_connections 2024-09-10T08:56:09.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.239+0000 7fb65c294740 1 --2- 172.21.15.172:0/2014989220 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5614dda0b170 0x5614dda0d560 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.239+0000 7fb65c294740 1 --2- 172.21.15.172:0/2014989220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614dd990370 0x5614dd9a3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.239+0000 7fb65c294740 1 --2- 172.21.15.172:0/2014989220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5614dd986b40 0x5614dd9a3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.239+0000 7fb65c294740 1 --2- 172.21.15.172:0/2014989220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614dd9855d0 0x5614dd9a5cd0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.239+0000 7fb65c294740 1 -- 172.21.15.172:0/2014989220 >> 172.21.15.172:0/2014989220 conn(0x5614dd94e310 msgr2=0x5614dd98e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:09.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.239+0000 7fb632ffd640 0 client.18408 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:56:09.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.239+0000 7fb65c294740 1 -- 172.21.15.172:0/2014989220 shutdown_connections 2024-09-10T08:56:09.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.239+0000 7fb65c294740 1 -- 172.21.15.172:0/2014989220 wait complete. 2024-09-10T08:56:09.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.239+0000 7fb65c294740 20 client.18408 trim_cache size 0 max 16384 2024-09-10T08:56:09.254 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:56:09.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.240+0000 7fb659ebb640 1 -- 172.21.15.172:0/699623026 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb63c0a2b80 msgr2=0x7fb63c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:09.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.240+0000 7fb659ebb640 1 --2- 172.21.15.172:0/699623026 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb63c0a2b80 0x7fb63c0a5040 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb64c005fe0 tx=0x7fb64c02a8c0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.240+0000 7fb659ebb640 1 -- 172.21.15.172:0/699623026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb654101de0 msgr2=0x7fb6541a3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:09.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.240+0000 7fb659ebb640 1 --2- 172.21.15.172:0/699623026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb654101de0 0x7fb6541a3500 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fb64400c890 tx=0x7fb64400cd50 comp rx=0 tx=0).stop 2024-09-10T08:56:09.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.240+0000 7fb659ebb640 1 -- 172.21.15.172:0/699623026 shutdown_connections 2024-09-10T08:56:09.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.240+0000 7fb659ebb640 1 --2- 172.21.15.172:0/699623026 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb63c0a2b80 0x7fb63c0a5040 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.240+0000 7fb659ebb640 1 --2- 172.21.15.172:0/699623026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb654106f10 0x7fb65410db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.240+0000 7fb659ebb640 1 --2- 172.21.15.172:0/699623026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb654102790 0x7fb6541a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.240+0000 7fb659ebb640 1 --2- 172.21.15.172:0/699623026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb654101de0 0x7fb6541a3500 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:09.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.240+0000 7fb659ebb640 1 -- 172.21.15.172:0/699623026 >> 172.21.15.172:0/699623026 conn(0x7fb6540fd8c0 msgr2=0x7fb6540fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:09.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.240+0000 7fb659ebb640 1 -- 172.21.15.172:0/699623026 shutdown_connections 2024-09-10T08:56:09.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:09.240+0000 7fb659ebb640 1 -- 172.21.15.172:0/699623026 wait complete. 2024-09-10T08:56:09.723 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:09 smithi172 ceph-mon[30854]: pgmap v648: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-09-10T08:56:09.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:09 smithi174 ceph-mon[34068]: pgmap v648: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-09-10T08:56:09.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:09 smithi191 ceph-mon[35314]: pgmap v648: 129 pgs: 129 active+clean; 9.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 23 op/s 2024-09-10T08:56:10.056 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:56:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.118+0000 7f5740766640 1 -- 172.21.15.172:0/3134845004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57381025f0 msgr2=0x7f5738108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.118+0000 7f5740766640 1 --2- 172.21.15.172:0/3134845004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57381025f0 0x7f5738108e80 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f5730009ca0 tx=0x7f573002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.118+0000 7f5740766640 1 -- 172.21.15.172:0/3134845004 shutdown_connections 2024-09-10T08:56:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.118+0000 7f5740766640 1 --2- 172.21.15.172:0/3134845004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57381025f0 0x7f5738108e80 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.118+0000 7f5740766640 1 --2- 172.21.15.172:0/3134845004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5738101c30 0x7f57381020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.118+0000 7f5740766640 1 --2- 172.21.15.172:0/3134845004 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5738100a40 0x7f5738100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.118+0000 7f5740766640 1 -- 172.21.15.172:0/3134845004 >> 172.21.15.172:0/3134845004 conn(0x7f57380fc1e0 msgr2=0x7f57380fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.118+0000 7f5740766640 1 -- 172.21.15.172:0/3134845004 shutdown_connections 2024-09-10T08:56:10.120 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.118+0000 7f5740766640 1 -- 172.21.15.172:0/3134845004 wait complete. 2024-09-10T08:56:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.119+0000 7f5740766640 1 Processor -- start 2024-09-10T08:56:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.119+0000 7f5740766640 1 -- start start 2024-09-10T08:56:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.119+0000 7f5740766640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5738100a40 0x7f5738197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.119+0000 7f5740766640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5738101c30 0x7f57381981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.119+0000 7f5740766640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57381025f0 0x7f573819f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.119+0000 7f5740766640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f573810c550 con 0x7f5738100a40 2024-09-10T08:56:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.119+0000 7f5740766640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f573810c3d0 con 0x7f5738101c30 2024-09-10T08:56:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.119+0000 7f5740766640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f573810c6d0 con 0x7f57381025f0 2024-09-10T08:56:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.119+0000 7f573e502640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5738100a40 0x7f5738197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.119+0000 7f573e502640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5738100a40 0x7f5738197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42998/0 (socket says 172.21.15.172:42998) 2024-09-10T08:56:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.119+0000 7f573e502640 1 -- 172.21.15.172:0/550805811 learned_addr learned my addr 172.21.15.172:0/550805811 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.119+0000 7f573dd01640 1 --2- 172.21.15.172:0/550805811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5738101c30 0x7f57381981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.119+0000 7f573ed03640 1 --2- 172.21.15.172:0/550805811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57381025f0 0x7f573819f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.120+0000 7f573e502640 1 -- 172.21.15.172:0/550805811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57381025f0 msgr2=0x7f573819f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.120+0000 7f573e502640 1 --2- 172.21.15.172:0/550805811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57381025f0 0x7f573819f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.120+0000 7f573e502640 1 -- 172.21.15.172:0/550805811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5738101c30 msgr2=0x7f57381981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.120+0000 7f573e502640 1 --2- 172.21.15.172:0/550805811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5738101c30 0x7f57381981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.120+0000 7f573e502640 1 -- 172.21.15.172:0/550805811 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573819f890 con 0x7f5738100a40 2024-09-10T08:56:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.120+0000 7f573e502640 1 --2- 172.21.15.172:0/550805811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5738100a40 0x7f5738197c80 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f573400d950 tx=0x7f573400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.120+0000 7f572f7fe640 1 -- 172.21.15.172:0/550805811 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734017020 con 0x7f5738100a40 2024-09-10T08:56:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.120+0000 7f5740766640 1 -- 172.21.15.172:0/550805811 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5738102eb0 con 0x7f5738100a40 2024-09-10T08:56:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.120+0000 7f572f7fe640 1 -- 172.21.15.172:0/550805811 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5734009c30 con 0x7f5738100a40 2024-09-10T08:56:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.120+0000 7f5740766640 1 -- 172.21.15.172:0/550805811 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5738103450 con 0x7f5738100a40 2024-09-10T08:56:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.120+0000 7f572f7fe640 1 -- 172.21.15.172:0/550805811 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573400f620 con 0x7f5738100a40 2024-09-10T08:56:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.123+0000 7f572f7fe640 1 -- 172.21.15.172:0/550805811 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f573401a020 con 0x7f5738100a40 2024-09-10T08:56:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.123+0000 7f57435df740 1 Processor -- start 2024-09-10T08:56:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.124+0000 7f57435df740 1 -- start start 2024-09-10T08:56:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.124+0000 7f57435df740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78effda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.124+0000 7f57435df740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f002e0 0x55bf78f04da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.124+0000 7f57435df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f05370 0x55bf78f07830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.124+0000 7f57435df740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bf78e632f0 con 0x55bf78f002e0 2024-09-10T08:56:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.124+0000 7f57435df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf78e0b270 con 0x55bf78f05370 2024-09-10T08:56:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.124+0000 7f57435df740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bf78e5f2b0 con 0x55bf78eff9a0 2024-09-10T08:56:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.124+0000 7f573e502640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f002e0 0x55bf78f04da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.124+0000 7f573ed03640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78effda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.124+0000 7f573ed03640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78effda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:50622/0 (socket says 172.21.15.172:50622) 2024-09-10T08:56:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.124+0000 7f573ed03640 1 -- 172.21.15.172:0/1308478896 learned_addr learned my addr 172.21.15.172:0/1308478896 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.124+0000 7f573dd01640 1 --2- 172.21.15.172:0/1308478896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f05370 0x55bf78f07830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.125+0000 7f573ed03640 1 -- 172.21.15.172:0/1308478896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f05370 msgr2=0x55bf78f07830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.125+0000 7f573ed03640 1 --2- 172.21.15.172:0/1308478896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f05370 0x55bf78f07830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.125+0000 7f573ed03640 1 -- 172.21.15.172:0/1308478896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f002e0 msgr2=0x55bf78f04da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.125+0000 7f573ed03640 1 --2- 172.21.15.172:0/1308478896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f002e0 0x55bf78f04da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.125+0000 7f573ed03640 1 -- 172.21.15.172:0/1308478896 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf78e0a340 con 0x55bf78eff9a0 2024-09-10T08:56:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.125+0000 7f573ed03640 1 --2- 172.21.15.172:0/1308478896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78effda0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5730009ca0 tx=0x7f5730009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9789b402c4edf8da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.125+0000 7f572f7fe640 1 --2- 172.21.15.172:0/550805811 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57240a2cb0 0x7f57240a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.125+0000 7f572cff9640 1 -- 172.21.15.172:0/1308478896 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573004c070 con 0x55bf78eff9a0 2024-09-10T08:56:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.125+0000 7f572cff9640 1 -- 172.21.15.172:0/1308478896 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5730036040 con 0x55bf78eff9a0 2024-09-10T08:56:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.125+0000 7f572cff9640 1 -- 172.21.15.172:0/1308478896 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573003caa0 con 0x55bf78eff9a0 2024-09-10T08:56:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.125+0000 7f573e502640 1 --2- 172.21.15.172:0/550805811 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57240a2cb0 0x7f57240a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.126+0000 7f572f7fe640 1 -- 172.21.15.172:0/550805811 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f57340cf1c0 con 0x7f5738100a40 2024-09-10T08:56:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.126+0000 7f57435df740 1 -- 172.21.15.172:0/1308478896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 msgr2=0x55bf78effda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.126+0000 7f573e502640 1 --2- 172.21.15.172:0/550805811 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57240a2cb0 0x7f57240a5170 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f57340cccd0 tx=0x7f57340d2ca0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:10.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.126+0000 7f57435df740 1 --2- 172.21.15.172:0/1308478896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78effda0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5730009ca0 tx=0x7f5730009a20 comp rx=0 tx=0).stop 2024-09-10T08:56:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.126+0000 7f57435df740 1 -- 172.21.15.172:0/1308478896 shutdown_connections 2024-09-10T08:56:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.126+0000 7f57435df740 1 --2- 172.21.15.172:0/1308478896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f05370 0x55bf78f07830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.126+0000 7f57435df740 1 --2- 172.21.15.172:0/1308478896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f002e0 0x55bf78f04da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.126+0000 7f57435df740 1 --2- 172.21.15.172:0/1308478896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78effda0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.126+0000 7f57435df740 1 -- 172.21.15.172:0/1308478896 >> 172.21.15.172:0/1308478896 conn(0x55bf78efb010 msgr2=0x55bf78efb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.126+0000 7f57435df740 1 -- 172.21.15.172:0/1308478896 shutdown_connections 2024-09-10T08:56:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.126+0000 7f57435df740 1 -- 172.21.15.172:0/1308478896 wait complete. 2024-09-10T08:56:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.127+0000 7f57435df740 1 -- start start 2024-09-10T08:56:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.127+0000 7f57435df740 1 Processor -- start 2024-09-10T08:56:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.127+0000 7f57435df740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78f09b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.127+0000 7f57435df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f002e0 0x55bf78f0a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.127+0000 7f57435df740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f05370 0x55bf78f110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.127+0000 7f57435df740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bf78f082e0 con 0x55bf78f05370 2024-09-10T08:56:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.127+0000 7f57435df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf78dec290 con 0x55bf78f002e0 2024-09-10T08:56:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.127+0000 7f57435df740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bf78f08460 con 0x55bf78eff9a0 2024-09-10T08:56:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.127+0000 7f57435df740 10 client.0 _ll_register_callbacks cb 0x55bf78e1d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.128+0000 7f573ed03640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f05370 0x55bf78f110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.128+0000 7f573ed03640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f05370 0x55bf78f110f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43018/0 (socket says 172.21.15.172:43018) 2024-09-10T08:56:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.128+0000 7f573ed03640 1 -- 172.21.15.172:0/939398813 learned_addr learned my addr 172.21.15.172:0/939398813 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.128+0000 7f573ed03640 1 --2- 172.21.15.172:0/939398813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78f09b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.128+0000 7f573dd01640 1 --2- 172.21.15.172:0/939398813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f002e0 0x55bf78f0a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.128+0000 7f573ed03640 1 -- 172.21.15.172:0/939398813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f002e0 msgr2=0x55bf78f0a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.128+0000 7f573ed03640 1 --2- 172.21.15.172:0/939398813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f002e0 0x55bf78f0a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.128+0000 7f573ed03640 1 -- 172.21.15.172:0/939398813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f05370 msgr2=0x55bf78f110f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.128+0000 7f573ed03640 1 --2- 172.21.15.172:0/939398813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f05370 0x55bf78f110f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.128+0000 7f573ed03640 1 -- 172.21.15.172:0/939398813 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf78b76210 con 0x55bf78eff9a0 2024-09-10T08:56:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.129+0000 7f573ed03640 1 --2- 172.21.15.172:0/939398813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78f09b70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5730009960 tx=0x7f573002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69333b852045f799 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.129+0000 7f5721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:56:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.129+0000 7f5721ffb640 1 -- 172.21.15.172:0/939398813 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573005c060 con 0x55bf78eff9a0 2024-09-10T08:56:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.129+0000 7f5721ffb640 1 -- 172.21.15.172:0/939398813 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5730031b40 con 0x55bf78eff9a0 2024-09-10T08:56:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.129+0000 7f57435df740 1 -- 172.21.15.172:0/939398813 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5730036040 con 0x55bf78eff9a0 2024-09-10T08:56:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.130+0000 7f5721ffb640 1 -- 172.21.15.172:0/939398813 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5730063050 con 0x55bf78eff9a0 2024-09-10T08:56:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.130+0000 7f57435df740 10 client.27313 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.130+0000 7f57435df740 1 -- 172.21.15.172:0/939398813 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bf78c42620 con 0x55bf78eff9a0 2024-09-10T08:56:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.130+0000 7f5721ffb640 1 -- 172.21.15.172:0/939398813 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5730036040 con 0x55bf78eff9a0 2024-09-10T08:56:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.130+0000 7f5721ffb640 1 -- 172.21.15.172:0/939398813 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f57300305c0 con 0x55bf78eff9a0 2024-09-10T08:56:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.131+0000 7f57435df740 10 client.27313 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.131+0000 7f57435df740 10 client.27313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.131+0000 7f57435df740 20 client.27313 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.131+0000 7f57435df740 1 --2- 172.21.15.172:0/939398813 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bf78f153f0 0x55bf78f177e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.131+0000 7f57435df740 4 client.27313 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:56:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.131+0000 7f57435df740 1 -- 172.21.15.172:0/939398813 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf78e70a20 con 0x55bf78f153f0 2024-09-10T08:56:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.131+0000 7f573dd01640 1 --2- 172.21.15.172:0/939398813 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bf78f153f0 0x55bf78f177e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.132+0000 7f573dd01640 1 --2- 172.21.15.172:0/939398813 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bf78f153f0 0x55bf78f177e0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67d51e529220ae19 server_cookie=6d661f9261aed04f in_seq=0 out_seq=0 2024-09-10T08:56:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.132+0000 7f5721ffb640 10 client.27313 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.133+0000 7f5721ffb640 1 -- 172.21.15.172:0/939398813 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf78e70a20 con 0x55bf78f153f0 2024-09-10T08:56:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.133+0000 7f5721ffb640 10 client.27313 handle_command_reply: tid=0 2024-09-10T08:56:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.133+0000 7f57435df740 1 client.27313 shutdown 2024-09-10T08:56:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.133+0000 7f57435df740 1 -- 172.21.15.172:0/939398813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 msgr2=0x55bf78f09b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.133+0000 7f57435df740 1 --2- 172.21.15.172:0/939398813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78f09b70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5730009960 tx=0x7f573002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.133+0000 7f57435df740 1 -- 172.21.15.172:0/939398813 shutdown_connections 2024-09-10T08:56:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.133+0000 7f57435df740 1 --2- 172.21.15.172:0/939398813 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bf78f153f0 0x55bf78f177e0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.133+0000 7f57435df740 1 --2- 172.21.15.172:0/939398813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f05370 0x55bf78f110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.133+0000 7f57435df740 1 --2- 172.21.15.172:0/939398813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f002e0 0x55bf78f0a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.133+0000 7f57435df740 1 --2- 172.21.15.172:0/939398813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78f09b70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.133+0000 7f5721ffb640 0 client.27313 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.133+0000 7f57435df740 1 -- 172.21.15.172:0/939398813 >> 172.21.15.172:0/939398813 conn(0x55bf78efb010 msgr2=0x55bf78efc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.133+0000 7f57435df740 1 -- 172.21.15.172:0/939398813 shutdown_connections 2024-09-10T08:56:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.133+0000 7f57435df740 1 -- 172.21.15.172:0/939398813 wait complete. 2024-09-10T08:56:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.134+0000 7f57435df740 20 client.27313 trim_cache size 0 max 16384 2024-09-10T08:56:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.154+0000 7f57435df740 1 Processor -- start 2024-09-10T08:56:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f57435df740 1 -- start start 2024-09-10T08:56:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f57435df740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78f09f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f57435df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f002e0 0x55bf78f11080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f57435df740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f05370 0x55bf78f1bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f57435df740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f571c00fb40 con 0x55bf78f05370 2024-09-10T08:56:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f57435df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f571c011680 con 0x55bf78f002e0 2024-09-10T08:56:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f57435df740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f571c010210 con 0x55bf78eff9a0 2024-09-10T08:56:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f573ed03640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f05370 0x55bf78f1bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f573ed03640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f05370 0x55bf78f1bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43028/0 (socket says 172.21.15.172:43028) 2024-09-10T08:56:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f573ed03640 1 -- 172.21.15.172:0/403774724 learned_addr learned my addr 172.21.15.172:0/403774724 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f573ed03640 1 --2- 172.21.15.172:0/403774724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78f09f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f573dd01640 1 --2- 172.21.15.172:0/403774724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f002e0 0x55bf78f11080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f573ed03640 1 -- 172.21.15.172:0/403774724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 msgr2=0x55bf78f09f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f573ed03640 1 --2- 172.21.15.172:0/403774724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78f09f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f573ed03640 1 -- 172.21.15.172:0/403774724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f002e0 msgr2=0x55bf78f11080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f573ed03640 1 --2- 172.21.15.172:0/403774724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f002e0 0x55bf78f11080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f573ed03640 1 -- 172.21.15.172:0/403774724 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf78f096b0 con 0x55bf78f05370 2024-09-10T08:56:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.155+0000 7f573dd01640 1 --2- 172.21.15.172:0/403774724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f002e0 0x55bf78f11080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.156+0000 7f573ed03640 1 --2- 172.21.15.172:0/403774724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f05370 0x55bf78f1bfc0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f5730009ca0 tx=0x7f5730009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d740fb6f79762a04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.156+0000 7f5723fff640 1 -- 172.21.15.172:0/403774724 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573005c070 con 0x55bf78f05370 2024-09-10T08:56:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.156+0000 7f5723fff640 1 -- 172.21.15.172:0/403774724 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5730036040 con 0x55bf78f05370 2024-09-10T08:56:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.156+0000 7f5723fff640 1 -- 172.21.15.172:0/403774724 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573004c070 con 0x55bf78f05370 2024-09-10T08:56:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.157+0000 7f57435df740 1 -- 172.21.15.172:0/403774724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f05370 msgr2=0x55bf78f1bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.157+0000 7f57435df740 1 --2- 172.21.15.172:0/403774724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f05370 0x55bf78f1bfc0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f5730009ca0 tx=0x7f5730009a20 comp rx=0 tx=0).stop 2024-09-10T08:56:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.157+0000 7f57435df740 1 -- 172.21.15.172:0/403774724 shutdown_connections 2024-09-10T08:56:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.157+0000 7f57435df740 1 --2- 172.21.15.172:0/403774724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f05370 0x55bf78f1bfc0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.157+0000 7f57435df740 1 --2- 172.21.15.172:0/403774724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f002e0 0x55bf78f11080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.157+0000 7f57435df740 1 --2- 172.21.15.172:0/403774724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78f09f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.157+0000 7f57435df740 1 -- 172.21.15.172:0/403774724 >> 172.21.15.172:0/403774724 conn(0x55bf78efb010 msgr2=0x55bf78efa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.157+0000 7f57435df740 1 -- 172.21.15.172:0/403774724 shutdown_connections 2024-09-10T08:56:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.157+0000 7f57435df740 1 -- 172.21.15.172:0/403774724 wait complete. 2024-09-10T08:56:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.157+0000 7f57435df740 1 -- start start 2024-09-10T08:56:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.157+0000 7f57435df740 1 Processor -- start 2024-09-10T08:56:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.157+0000 7f57435df740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78f0efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.157+0000 7f57435df740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f002e0 0x55bf78f0f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.157+0000 7f57435df740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f05370 0x55bf78f104f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.157+0000 7f57435df740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f571c010fb0 con 0x55bf78f002e0 2024-09-10T08:56:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.157+0000 7f57435df740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f571c0108e0 con 0x55bf78f05370 2024-09-10T08:56:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.157+0000 7f57435df740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f571c00db20 con 0x55bf78eff9a0 2024-09-10T08:56:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.158+0000 7f57435df740 10 client.0 _ll_register_callbacks cb 0x7f57340d2b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.158+0000 7f573dd01640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f002e0 0x55bf78f0f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.158+0000 7f573dd01640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f002e0 0x55bf78f0f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43042/0 (socket says 172.21.15.172:43042) 2024-09-10T08:56:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.158+0000 7f573dd01640 1 -- 172.21.15.172:0/1431566923 learned_addr learned my addr 172.21.15.172:0/1431566923 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.158+0000 7f573ed03640 1 --2- 172.21.15.172:0/1431566923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78f0efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.158+0000 7f573ed03640 1 --2- 172.21.15.172:0/1431566923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f05370 0x55bf78f104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.158+0000 7f573dd01640 1 -- 172.21.15.172:0/1431566923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 msgr2=0x55bf78f0efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.158+0000 7f573dd01640 1 --2- 172.21.15.172:0/1431566923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78f0efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.158+0000 7f573dd01640 1 -- 172.21.15.172:0/1431566923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f05370 msgr2=0x55bf78f104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.158+0000 7f573dd01640 1 --2- 172.21.15.172:0/1431566923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f05370 0x55bf78f104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.158+0000 7f573dd01640 1 -- 172.21.15.172:0/1431566923 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf78f10b00 con 0x55bf78f002e0 2024-09-10T08:56:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.158+0000 7f573ed03640 1 --2- 172.21.15.172:0/1431566923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f05370 0x55bf78f104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.158+0000 7f573dd01640 1 --2- 172.21.15.172:0/1431566923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f002e0 0x55bf78f0f4e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f572800b6b0 tx=0x7f572800bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30e427a0e2770b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.158+0000 7f5721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.158+0000 7f5721ffb640 1 -- 172.21.15.172:0/1431566923 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728003df0 con 0x55bf78f002e0 2024-09-10T08:56:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.158+0000 7f5721ffb640 1 -- 172.21.15.172:0/1431566923 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f572801d420 con 0x55bf78f002e0 2024-09-10T08:56:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.158+0000 7f57435df740 1 -- 172.21.15.172:0/1431566923 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5730036040 con 0x55bf78f002e0 2024-09-10T08:56:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.159+0000 7f5721ffb640 1 -- 172.21.15.172:0/1431566923 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f572802d050 con 0x55bf78f002e0 2024-09-10T08:56:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.159+0000 7f57435df740 10 client.18426 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.159+0000 7f57435df740 1 -- 172.21.15.172:0/1431566923 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bf78f80880 con 0x55bf78f002e0 2024-09-10T08:56:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.159+0000 7f5721ffb640 1 -- 172.21.15.172:0/1431566923 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5730036040 con 0x55bf78f002e0 2024-09-10T08:56:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.159+0000 7f5721ffb640 1 -- 172.21.15.172:0/1431566923 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f57280118c0 con 0x55bf78f002e0 2024-09-10T08:56:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.159+0000 7f57435df740 10 client.18426 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.159+0000 7f57435df740 10 client.18426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.159+0000 7f57435df740 20 client.18426 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.159+0000 7f57435df740 1 --2- 172.21.15.172:0/1431566923 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bf78f812b0 0x55bf78f836d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.159+0000 7f57435df740 4 client.18426 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:56:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.159+0000 7f57435df740 1 -- 172.21.15.172:0/1431566923 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf78e6ad70 con 0x55bf78f812b0 2024-09-10T08:56:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.159+0000 7f573ed03640 1 --2- 172.21.15.172:0/1431566923 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bf78f812b0 0x55bf78f836d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.160+0000 7f573ed03640 1 --2- 172.21.15.172:0/1431566923 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bf78f812b0 0x55bf78f836d0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fa7ea165354f931 server_cookie=9ef574b40dae8846 in_seq=0 out_seq=0 2024-09-10T08:56:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.160+0000 7f5721ffb640 10 client.18426 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.166+0000 7f5721ffb640 1 -- 172.21.15.172:0/1431566923 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf78e6ad70 con 0x55bf78f812b0 2024-09-10T08:56:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.166+0000 7f5721ffb640 10 client.18426 handle_command_reply: tid=0 2024-09-10T08:56:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.166+0000 7f57435df740 1 client.18426 shutdown 2024-09-10T08:56:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.166+0000 7f57435df740 1 -- 172.21.15.172:0/1431566923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f002e0 msgr2=0x55bf78f0f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.166+0000 7f57435df740 1 --2- 172.21.15.172:0/1431566923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f002e0 0x55bf78f0f4e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f572800b6b0 tx=0x7f572800bdb0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.166+0000 7f57435df740 1 -- 172.21.15.172:0/1431566923 shutdown_connections 2024-09-10T08:56:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.166+0000 7f57435df740 1 --2- 172.21.15.172:0/1431566923 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55bf78f812b0 0x55bf78f836d0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.166+0000 7f57435df740 1 --2- 172.21.15.172:0/1431566923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf78f05370 0x55bf78f104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.166+0000 7f57435df740 1 --2- 172.21.15.172:0/1431566923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bf78f002e0 0x55bf78f0f4e0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.166+0000 7f57435df740 1 --2- 172.21.15.172:0/1431566923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf78eff9a0 0x55bf78f0efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.166+0000 7f5721ffb640 0 client.18426 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.166+0000 7f57435df740 1 -- 172.21.15.172:0/1431566923 >> 172.21.15.172:0/1431566923 conn(0x55bf78efb010 msgr2=0x55bf78efa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.166+0000 7f57435df740 1 -- 172.21.15.172:0/1431566923 shutdown_connections 2024-09-10T08:56:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.166+0000 7f57435df740 1 -- 172.21.15.172:0/1431566923 wait complete. 2024-09-10T08:56:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.166+0000 7f57435df740 20 client.18426 trim_cache size 0 max 16384 2024-09-10T08:56:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:56:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:56:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:56:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:56:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.167+0000 7f5740766640 1 -- 172.21.15.172:0/550805811 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57240a2cb0 msgr2=0x7f57240a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.167+0000 7f5740766640 1 --2- 172.21.15.172:0/550805811 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57240a2cb0 0x7f57240a5170 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f57340cccd0 tx=0x7f57340d2ca0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.167+0000 7f5740766640 1 -- 172.21.15.172:0/550805811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5738100a40 msgr2=0x7f5738197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.167+0000 7f5740766640 1 --2- 172.21.15.172:0/550805811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5738100a40 0x7f5738197c80 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f573400d950 tx=0x7f573400de10 comp rx=0 tx=0).stop 2024-09-10T08:56:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.167+0000 7f5740766640 1 -- 172.21.15.172:0/550805811 shutdown_connections 2024-09-10T08:56:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.167+0000 7f5740766640 1 --2- 172.21.15.172:0/550805811 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57240a2cb0 0x7f57240a5170 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.167+0000 7f5740766640 1 --2- 172.21.15.172:0/550805811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57381025f0 0x7f573819f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.167+0000 7f5740766640 1 --2- 172.21.15.172:0/550805811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5738101c30 0x7f57381981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.167+0000 7f5740766640 1 --2- 172.21.15.172:0/550805811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5738100a40 0x7f5738197c80 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.167+0000 7f5740766640 1 -- 172.21.15.172:0/550805811 >> 172.21.15.172:0/550805811 conn(0x7f57380fc1e0 msgr2=0x7f57380fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.167+0000 7f5740766640 1 -- 172.21.15.172:0/550805811 shutdown_connections 2024-09-10T08:56:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:10.167+0000 7f5740766640 1 -- 172.21.15.172:0/550805811 wait complete. 2024-09-10T08:56:10.345 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:56:10.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.912+0000 7fea52409640 1 -- 172.21.15.172:0/3146662320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea4c106f20 msgr2=0x7fea4c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.912+0000 7fea52409640 1 --2- 172.21.15.172:0/3146662320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea4c106f20 0x7fea4c10eaa0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fea44009ca0 tx=0x7fea4402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.912+0000 7fea52409640 1 -- 172.21.15.172:0/3146662320 shutdown_connections 2024-09-10T08:56:10.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.912+0000 7fea52409640 1 --2- 172.21.15.172:0/3146662320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea4c106f20 0x7fea4c10eaa0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.912+0000 7fea52409640 1 --2- 172.21.15.172:0/3146662320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea4c1027a0 0x7fea4c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.912+0000 7fea52409640 1 --2- 172.21.15.172:0/3146662320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea4c101df0 0x7fea4c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.912+0000 7fea52409640 1 -- 172.21.15.172:0/3146662320 >> 172.21.15.172:0/3146662320 conn(0x7fea4c0fd8f0 msgr2=0x7fea4c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:10.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.912+0000 7fea52409640 1 -- 172.21.15.172:0/3146662320 shutdown_connections 2024-09-10T08:56:10.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.912+0000 7fea52409640 1 -- 172.21.15.172:0/3146662320 wait complete. 2024-09-10T08:56:10.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.912+0000 7fea52409640 1 Processor -- start 2024-09-10T08:56:10.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.912+0000 7fea52409640 1 -- start start 2024-09-10T08:56:10.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.913+0000 7fea52409640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea4c101df0 0x7fea4c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.913+0000 7fea52409640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea4c1027a0 0x7fea4c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.913+0000 7fea52409640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea4c106f20 0x7fea4c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.913+0000 7fea52409640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c111e30 con 0x7fea4c101df0 2024-09-10T08:56:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.913+0000 7fea52409640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c111cb0 con 0x7fea4c106f20 2024-09-10T08:56:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.913+0000 7fea52409640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c111fb0 con 0x7fea4c1027a0 2024-09-10T08:56:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.913+0000 7fea4bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea4c101df0 0x7fea4c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.913+0000 7fea4bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea4c101df0 0x7fea4c198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43048/0 (socket says 172.21.15.172:43048) 2024-09-10T08:56:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.913+0000 7fea4b7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea4c1027a0 0x7fea4c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.913+0000 7fea4bfff640 1 -- 172.21.15.172:0/1901028002 learned_addr learned my addr 172.21.15.172:0/1901028002 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.913+0000 7fea509a6640 1 --2- 172.21.15.172:0/1901028002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea4c106f20 0x7fea4c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.913+0000 7fea4bfff640 1 -- 172.21.15.172:0/1901028002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea4c1027a0 msgr2=0x7fea4c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.913+0000 7fea4bfff640 1 --2- 172.21.15.172:0/1901028002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea4c1027a0 0x7fea4c198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.913+0000 7fea4bfff640 1 -- 172.21.15.172:0/1901028002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea4c106f20 msgr2=0x7fea4c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.913+0000 7fea4bfff640 1 --2- 172.21.15.172:0/1901028002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea4c106f20 0x7fea4c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.913+0000 7fea4bfff640 1 -- 172.21.15.172:0/1901028002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4c19d300 con 0x7fea4c101df0 2024-09-10T08:56:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.914+0000 7fea4bfff640 1 --2- 172.21.15.172:0/1901028002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea4c101df0 0x7fea4c198350 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c920 tx=0x7fea3c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.914+0000 7fea497fa640 1 -- 172.21.15.172:0/1901028002 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c007c70 con 0x7fea4c101df0 2024-09-10T08:56:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.914+0000 7fea52409640 1 -- 172.21.15.172:0/1901028002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4c19d5f0 con 0x7fea4c101df0 2024-09-10T08:56:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.914+0000 7fea497fa640 1 -- 172.21.15.172:0/1901028002 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea3c007e10 con 0x7fea4c101df0 2024-09-10T08:56:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.914+0000 7fea497fa640 1 -- 172.21.15.172:0/1901028002 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c018820 con 0x7fea4c101df0 2024-09-10T08:56:10.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.914+0000 7fea52409640 1 -- 172.21.15.172:0/1901028002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea4c074420 con 0x7fea4c101df0 2024-09-10T08:56:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.917+0000 7fea497fa640 1 -- 172.21.15.172:0/1901028002 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fea3c0189c0 con 0x7fea4c101df0 2024-09-10T08:56:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.917+0000 7fea547e2740 1 Processor -- start 2024-09-10T08:56:10.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.917+0000 7fea547e2740 1 -- start start 2024-09-10T08:56:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.917+0000 7fea547e2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556380435b40 0x556380434090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.917+0000 7fea547e2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563804345d0 0x556380434a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea547e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55638043f370 0x556380441760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea497fa640 1 --2- 172.21.15.172:0/1901028002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea340a2b80 0x7fea340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea547e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55638039a2f0 con 0x55638043f370 2024-09-10T08:56:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea547e2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556380341cf0 con 0x556380435b40 2024-09-10T08:56:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea547e2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5563803962b0 con 0x5563804345d0 2024-09-10T08:56:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea509a6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556380435b40 0x556380434090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea4b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55638043f370 0x556380441760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea509a6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556380435b40 0x556380434090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:50714/0 (socket says 172.21.15.172:50714) 2024-09-10T08:56:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea509a6640 1 -- 172.21.15.172:0/1073803352 learned_addr learned my addr 172.21.15.172:0/1073803352 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea509a6640 1 --2- 172.21.15.172:0/1901028002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea340a2b80 0x7fea340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea4bfff640 1 --2- 172.21.15.172:0/1073803352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563804345d0 0x556380434a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea497fa640 1 -- 172.21.15.172:0/1901028002 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fea3c0cf420 con 0x7fea4c101df0 2024-09-10T08:56:10.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea509a6640 1 -- 172.21.15.172:0/1073803352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563804345d0 msgr2=0x556380434a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea509a6640 1 --2- 172.21.15.172:0/1073803352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563804345d0 0x556380434a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea509a6640 1 -- 172.21.15.172:0/1073803352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55638043f370 msgr2=0x556380441760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea509a6640 1 --2- 172.21.15.172:0/1073803352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55638043f370 0x556380441760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea509a6640 1 -- 172.21.15.172:0/1073803352 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563803263a0 con 0x556380435b40 2024-09-10T08:56:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea4b7fe640 1 --2- 172.21.15.172:0/1073803352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55638043f370 0x556380441760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea509a6640 1 --2- 172.21.15.172:0/1073803352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556380435b40 0x556380434090 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fea44035000 tx=0x7fea44032d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf0de9f078bf15e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.918+0000 7fea509a6640 1 --2- 172.21.15.172:0/1901028002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea340a2b80 0x7fea340a5040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fea44004af0 tx=0x7fea4403b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.919+0000 7fea327fc640 1 -- 172.21.15.172:0/1073803352 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4404e070 con 0x556380435b40 2024-09-10T08:56:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.919+0000 7fea327fc640 1 -- 172.21.15.172:0/1073803352 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea44036040 con 0x556380435b40 2024-09-10T08:56:10.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.919+0000 7fea327fc640 1 -- 172.21.15.172:0/1073803352 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea440047e0 con 0x556380435b40 2024-09-10T08:56:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.919+0000 7fea547e2740 1 -- 172.21.15.172:0/1073803352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556380435b40 msgr2=0x556380434090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.919+0000 7fea547e2740 1 --2- 172.21.15.172:0/1073803352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556380435b40 0x556380434090 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fea44035000 tx=0x7fea44032d70 comp rx=0 tx=0).stop 2024-09-10T08:56:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.919+0000 7fea547e2740 1 -- 172.21.15.172:0/1073803352 shutdown_connections 2024-09-10T08:56:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.919+0000 7fea547e2740 1 --2- 172.21.15.172:0/1073803352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55638043f370 0x556380441760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.919+0000 7fea547e2740 1 --2- 172.21.15.172:0/1073803352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563804345d0 0x556380434a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.919+0000 7fea547e2740 1 --2- 172.21.15.172:0/1073803352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556380435b40 0x556380434090 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.919+0000 7fea547e2740 1 -- 172.21.15.172:0/1073803352 >> 172.21.15.172:0/1073803352 conn(0x5563803fd310 msgr2=0x556380430c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.919+0000 7fea547e2740 1 -- 172.21.15.172:0/1073803352 shutdown_connections 2024-09-10T08:56:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.920+0000 7fea547e2740 1 -- 172.21.15.172:0/1073803352 wait complete. 2024-09-10T08:56:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.920+0000 7fea547e2740 1 -- start start 2024-09-10T08:56:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.920+0000 7fea547e2740 1 Processor -- start 2024-09-10T08:56:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.920+0000 7fea547e2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563804345d0 0x55638043ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.920+0000 7fea547e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556380435b40 0x55638043c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.920+0000 7fea547e2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55638043f370 0x55638043c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.920+0000 7fea547e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556380441e90 con 0x556380435b40 2024-09-10T08:56:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.920+0000 7fea547e2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556380355060 con 0x55638043f370 2024-09-10T08:56:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.920+0000 7fea547e2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556380442010 con 0x5563804345d0 2024-09-10T08:56:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.921+0000 7fea547e2740 10 client.0 _ll_register_callbacks cb 0x5563800a88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.921+0000 7fea4b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556380435b40 0x55638043c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.921+0000 7fea4b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556380435b40 0x55638043c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43064/0 (socket says 172.21.15.172:43064) 2024-09-10T08:56:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.921+0000 7fea4b7fe640 1 -- 172.21.15.172:0/1232788724 learned_addr learned my addr 172.21.15.172:0/1232788724 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.921+0000 7fea509a6640 1 --2- 172.21.15.172:0/1232788724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55638043f370 0x55638043c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.921+0000 7fea4bfff640 1 --2- 172.21.15.172:0/1232788724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563804345d0 0x55638043ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.921+0000 7fea509a6640 1 -- 172.21.15.172:0/1232788724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563804345d0 msgr2=0x55638043ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.921+0000 7fea509a6640 1 --2- 172.21.15.172:0/1232788724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563804345d0 0x55638043ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.921+0000 7fea509a6640 1 -- 172.21.15.172:0/1232788724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556380435b40 msgr2=0x55638043c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.921+0000 7fea509a6640 1 --2- 172.21.15.172:0/1232788724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556380435b40 0x55638043c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.921+0000 7fea509a6640 1 -- 172.21.15.172:0/1232788724 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55638010b3c0 con 0x55638043f370 2024-09-10T08:56:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.921+0000 7fea4b7fe640 1 --2- 172.21.15.172:0/1232788724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556380435b40 0x55638043c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.921+0000 7fea509a6640 1 --2- 172.21.15.172:0/1232788724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55638043f370 0x55638043c630 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea44005e30 tx=0x7fea4402a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f30f88713558f51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.921+0000 7fea2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:56:10.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.921+0000 7fea2b7fe640 1 -- 172.21.15.172:0/1232788724 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4404e050 con 0x55638043f370 2024-09-10T08:56:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.921+0000 7fea547e2740 1 -- 172.21.15.172:0/1232788724 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea44036040 con 0x55638043f370 2024-09-10T08:56:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.921+0000 7fea2b7fe640 1 -- 172.21.15.172:0/1232788724 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fea44004040 con 0x55638043f370 2024-09-10T08:56:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.922+0000 7fea2b7fe640 1 -- 172.21.15.172:0/1232788724 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fea44004280 con 0x55638043f370 2024-09-10T08:56:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.922+0000 7fea547e2740 10 client.27383 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.922+0000 7fea547e2740 1 -- 172.21.15.172:0/1232788724 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556380441e90 con 0x55638043f370 2024-09-10T08:56:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.922+0000 7fea2b7fe640 1 -- 172.21.15.172:0/1232788724 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea44036040 con 0x55638043f370 2024-09-10T08:56:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.922+0000 7fea2b7fe640 1 -- 172.21.15.172:0/1232788724 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fea44030c30 con 0x55638043f370 2024-09-10T08:56:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.922+0000 7fea547e2740 10 client.27383 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.922+0000 7fea547e2740 10 client.27383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.922+0000 7fea547e2740 20 client.27383 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.922+0000 7fea547e2740 1 --2- 172.21.15.172:0/1232788724 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55638044eaf0 0x556380450ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.928 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.922+0000 7fea547e2740 4 client.27383 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:56:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.922+0000 7fea547e2740 1 -- 172.21.15.172:0/1232788724 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563803a7860 con 0x55638044eaf0 2024-09-10T08:56:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.922+0000 7fea4bfff640 1 --2- 172.21.15.172:0/1232788724 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55638044eaf0 0x556380450ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.923+0000 7fea2b7fe640 10 client.27383 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.923+0000 7fea4bfff640 1 --2- 172.21.15.172:0/1232788724 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55638044eaf0 0x556380450ee0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6492f82ec77d3859 server_cookie=d01fc11068fca63e in_seq=0 out_seq=0 2024-09-10T08:56:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.923+0000 7fea2b7fe640 1 -- 172.21.15.172:0/1232788724 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563803a7860 con 0x55638044eaf0 2024-09-10T08:56:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.923+0000 7fea2b7fe640 10 client.27383 handle_command_reply: tid=0 2024-09-10T08:56:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.923+0000 7fea547e2740 1 client.27383 shutdown 2024-09-10T08:56:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.924+0000 7fea547e2740 1 -- 172.21.15.172:0/1232788724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55638043f370 msgr2=0x55638043c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.924+0000 7fea547e2740 1 --2- 172.21.15.172:0/1232788724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55638043f370 0x55638043c630 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea44005e30 tx=0x7fea4402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.924+0000 7fea547e2740 1 -- 172.21.15.172:0/1232788724 shutdown_connections 2024-09-10T08:56:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.924+0000 7fea547e2740 1 --2- 172.21.15.172:0/1232788724 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55638044eaf0 0x556380450ee0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.929 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.924+0000 7fea547e2740 1 --2- 172.21.15.172:0/1232788724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55638043f370 0x55638043c630 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.924+0000 7fea547e2740 1 --2- 172.21.15.172:0/1232788724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556380435b40 0x55638043c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.924+0000 7fea547e2740 1 --2- 172.21.15.172:0/1232788724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563804345d0 0x55638043ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.924+0000 7fea547e2740 1 -- 172.21.15.172:0/1232788724 >> 172.21.15.172:0/1232788724 conn(0x5563803fd310 msgr2=0x556380436960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:10.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.924+0000 7fea2b7fe640 0 client.27383 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:10.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.924+0000 7fea547e2740 1 -- 172.21.15.172:0/1232788724 shutdown_connections 2024-09-10T08:56:10.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.924+0000 7fea547e2740 1 -- 172.21.15.172:0/1232788724 wait complete. 2024-09-10T08:56:10.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.924+0000 7fea547e2740 20 client.27383 trim_cache size 0 max 16384 2024-09-10T08:56:10.941 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.939+0000 7fea547e2740 1 Processor -- start 2024-09-10T08:56:10.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea547e2740 1 -- start start 2024-09-10T08:56:10.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea547e2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563804345d0 0x55638044b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea547e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556380435b40 0x556380456760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea547e2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55638043f370 0x556380456ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea547e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c1b0ba0 con 0x556380435b40 2024-09-10T08:56:10.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea547e2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c1b26e0 con 0x5563804345d0 2024-09-10T08:56:10.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea547e2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c1b1270 con 0x55638043f370 2024-09-10T08:56:10.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea4b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556380435b40 0x556380456760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea4b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556380435b40 0x556380456760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43074/0 (socket says 172.21.15.172:43074) 2024-09-10T08:56:10.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea4b7fe640 1 -- 172.21.15.172:0/897539466 learned_addr learned my addr 172.21.15.172:0/897539466 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:10.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea4bfff640 1 --2- 172.21.15.172:0/897539466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563804345d0 0x55638044b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.942 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea4b7fe640 1 -- 172.21.15.172:0/897539466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55638043f370 msgr2=0x556380456ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:10.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea509a6640 1 --2- 172.21.15.172:0/897539466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55638043f370 0x556380456ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea4b7fe640 1 --2- 172.21.15.172:0/897539466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55638043f370 0x556380456ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea4b7fe640 1 -- 172.21.15.172:0/897539466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563804345d0 msgr2=0x55638044b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:10.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea4b7fe640 1 --2- 172.21.15.172:0/897539466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563804345d0 0x55638044b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea4b7fe640 1 -- 172.21.15.172:0/897539466 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556380437120 con 0x556380435b40 2024-09-10T08:56:10.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea4bfff640 1 --2- 172.21.15.172:0/897539466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563804345d0 0x55638044b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:10.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.940+0000 7fea4b7fe640 1 --2- 172.21.15.172:0/897539466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556380435b40 0x556380456760 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fea4000b7c0 tx=0x7fea4000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e572e1641c8e4b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:10.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.941+0000 7fea31ffb640 1 -- 172.21.15.172:0/897539466 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40003d50 con 0x556380435b40 2024-09-10T08:56:10.943 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.941+0000 7fea31ffb640 1 -- 172.21.15.172:0/897539466 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea40020070 con 0x556380435b40 2024-09-10T08:56:10.944 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.941+0000 7fea31ffb640 1 -- 172.21.15.172:0/897539466 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4001b660 con 0x556380435b40 2024-09-10T08:56:10.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.942+0000 7fea547e2740 1 -- 172.21.15.172:0/897539466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556380435b40 msgr2=0x556380456760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.947 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.942+0000 7fea547e2740 1 --2- 172.21.15.172:0/897539466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556380435b40 0x556380456760 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fea4000b7c0 tx=0x7fea4000bc80 comp rx=0 tx=0).stop 2024-09-10T08:56:10.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.942+0000 7fea547e2740 1 -- 172.21.15.172:0/897539466 shutdown_connections 2024-09-10T08:56:10.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.942+0000 7fea547e2740 1 --2- 172.21.15.172:0/897539466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55638043f370 0x556380456ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.942+0000 7fea547e2740 1 --2- 172.21.15.172:0/897539466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556380435b40 0x556380456760 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.942+0000 7fea547e2740 1 --2- 172.21.15.172:0/897539466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563804345d0 0x55638044b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.942+0000 7fea547e2740 1 -- 172.21.15.172:0/897539466 >> 172.21.15.172:0/897539466 conn(0x5563803fd310 msgr2=0x55638043b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:10.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.942+0000 7fea547e2740 1 -- 172.21.15.172:0/897539466 shutdown_connections 2024-09-10T08:56:10.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.942+0000 7fea547e2740 1 -- 172.21.15.172:0/897539466 wait complete. 2024-09-10T08:56:10.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.942+0000 7fea547e2740 1 -- start start 2024-09-10T08:56:10.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.942+0000 7fea547e2740 1 Processor -- start 2024-09-10T08:56:10.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea547e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563804345d0 0x556380442aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.948 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea547e2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556380435b40 0x556380442fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea547e2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55638043f370 0x5563804b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea547e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c1b2010 con 0x5563804345d0 2024-09-10T08:56:10.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea547e2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c1b1940 con 0x556380435b40 2024-09-10T08:56:10.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea547e2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c1aeb80 con 0x55638043f370 2024-09-10T08:56:10.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea547e2740 10 client.0 _ll_register_callbacks cb 0x7fea4000b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:10.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea4bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563804345d0 0x556380442aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea4bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563804345d0 0x556380442aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43090/0 (socket says 172.21.15.172:43090) 2024-09-10T08:56:10.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea4bfff640 1 -- 172.21.15.172:0/1838366566 learned_addr learned my addr 172.21.15.172:0/1838366566 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:10.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea4b7fe640 1 --2- 172.21.15.172:0/1838366566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556380435b40 0x556380442fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea509a6640 1 --2- 172.21.15.172:0/1838366566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55638043f370 0x5563804b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea4bfff640 1 -- 172.21.15.172:0/1838366566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55638043f370 msgr2=0x5563804b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:10.949 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea4bfff640 1 --2- 172.21.15.172:0/1838366566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55638043f370 0x5563804b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea4bfff640 1 -- 172.21.15.172:0/1838366566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556380435b40 msgr2=0x556380442fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:10.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea4bfff640 1 --2- 172.21.15.172:0/1838366566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556380435b40 0x556380442fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea4bfff640 1 -- 172.21.15.172:0/1838366566 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563804b1de0 con 0x5563804345d0 2024-09-10T08:56:10.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea509a6640 1 --2- 172.21.15.172:0/1838366566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55638043f370 0x5563804b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:56:10.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea4bfff640 1 --2- 172.21.15.172:0/1838366566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563804345d0 0x556380442aa0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fea3c007590 tx=0x7fea3c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedcc6b36582bf7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:10.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:10.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.943+0000 7fea2b7fe640 1 -- 172.21.15.172:0/1838366566 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c07f030 con 0x5563804345d0 2024-09-10T08:56:10.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.944+0000 7fea2b7fe640 1 -- 172.21.15.172:0/1838366566 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fea3c099c20 con 0x5563804345d0 2024-09-10T08:56:10.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.944+0000 7fea547e2740 1 -- 172.21.15.172:0/1838366566 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea40020070 con 0x5563804345d0 2024-09-10T08:56:10.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.944+0000 7fea2b7fe640 1 -- 172.21.15.172:0/1838366566 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fea3c0a9050 con 0x5563804345d0 2024-09-10T08:56:10.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.944+0000 7fea547e2740 10 client.18444 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:10.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.944+0000 7fea547e2740 1 -- 172.21.15.172:0/1838366566 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5563804b9130 con 0x5563804345d0 2024-09-10T08:56:10.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.944+0000 7fea2b7fe640 1 -- 172.21.15.172:0/1838366566 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea3c09c020 con 0x5563804345d0 2024-09-10T08:56:10.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.944+0000 7fea2b7fe640 1 -- 172.21.15.172:0/1838366566 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fea3c0ba190 con 0x5563804345d0 2024-09-10T08:56:10.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.944+0000 7fea547e2740 10 client.18444 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:10.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.944+0000 7fea547e2740 10 client.18444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:10.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.944+0000 7fea547e2740 20 client.18444 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:10.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.944+0000 7fea547e2740 1 --2- 172.21.15.172:0/1838366566 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5563804b9850 0x5563804bbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:10.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.944+0000 7fea547e2740 4 client.18444 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:56:10.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.944+0000 7fea547e2740 1 -- 172.21.15.172:0/1838366566 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fea40020070 con 0x5563804b9850 2024-09-10T08:56:10.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.944+0000 7fea4b7fe640 1 --2- 172.21.15.172:0/1838366566 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5563804b9850 0x5563804bbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:10.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.945+0000 7fea4b7fe640 1 --2- 172.21.15.172:0/1838366566 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5563804b9850 0x5563804bbc70 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7c8a59dacc337ea server_cookie=bd25a0826e05fc6e in_seq=0 out_seq=0 2024-09-10T08:56:10.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.945+0000 7fea2b7fe640 10 client.18444 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:10.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.945+0000 7fea2b7fe640 1 -- 172.21.15.172:0/1838366566 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fea40020070 con 0x5563804b9850 2024-09-10T08:56:10.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.945+0000 7fea2b7fe640 10 client.18444 handle_command_reply: tid=0 2024-09-10T08:56:10.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.945+0000 7fea547e2740 1 client.18444 shutdown 2024-09-10T08:56:10.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.945+0000 7fea547e2740 1 -- 172.21.15.172:0/1838366566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563804345d0 msgr2=0x556380442aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.945+0000 7fea547e2740 1 --2- 172.21.15.172:0/1838366566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563804345d0 0x556380442aa0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fea3c007590 tx=0x7fea3c086040 comp rx=0 tx=0).stop 2024-09-10T08:56:10.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.945+0000 7fea547e2740 1 -- 172.21.15.172:0/1838366566 shutdown_connections 2024-09-10T08:56:10.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.945+0000 7fea547e2740 1 --2- 172.21.15.172:0/1838366566 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5563804b9850 0x5563804bbc70 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.945+0000 7fea547e2740 1 --2- 172.21.15.172:0/1838366566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55638043f370 0x5563804b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.945+0000 7fea2b7fe640 0 client.18444 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:10.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.945+0000 7fea547e2740 1 --2- 172.21.15.172:0/1838366566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556380435b40 0x556380442fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.945+0000 7fea547e2740 1 --2- 172.21.15.172:0/1838366566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563804345d0 0x556380442aa0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.945+0000 7fea547e2740 1 -- 172.21.15.172:0/1838366566 >> 172.21.15.172:0/1838366566 conn(0x5563803fd310 msgr2=0x55638043d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:10.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.945+0000 7fea547e2740 1 -- 172.21.15.172:0/1838366566 shutdown_connections 2024-09-10T08:56:10.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.945+0000 7fea547e2740 1 -- 172.21.15.172:0/1838366566 wait complete. 2024-09-10T08:56:10.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.946+0000 7fea547e2740 20 client.18444 trim_cache size 0 max 16384 2024-09-10T08:56:10.953 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:56:10.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.946+0000 7fea52409640 1 -- 172.21.15.172:0/1901028002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea340a2b80 msgr2=0x7fea340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.946+0000 7fea52409640 1 --2- 172.21.15.172:0/1901028002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea340a2b80 0x7fea340a5040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fea44004af0 tx=0x7fea4403b000 comp rx=0 tx=0).stop 2024-09-10T08:56:10.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.946+0000 7fea52409640 1 -- 172.21.15.172:0/1901028002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea4c101df0 msgr2=0x7fea4c198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:10.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.946+0000 7fea52409640 1 --2- 172.21.15.172:0/1901028002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea4c101df0 0x7fea4c198350 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c920 tx=0x7fea3c00cde0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.946+0000 7fea52409640 1 -- 172.21.15.172:0/1901028002 shutdown_connections 2024-09-10T08:56:10.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.946+0000 7fea52409640 1 --2- 172.21.15.172:0/1901028002 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea340a2b80 0x7fea340a5040 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.946+0000 7fea52409640 1 --2- 172.21.15.172:0/1901028002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea4c106f20 0x7fea4c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.946+0000 7fea52409640 1 --2- 172.21.15.172:0/1901028002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea4c1027a0 0x7fea4c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.946+0000 7fea52409640 1 --2- 172.21.15.172:0/1901028002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea4c101df0 0x7fea4c198350 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:10.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.946+0000 7fea52409640 1 -- 172.21.15.172:0/1901028002 >> 172.21.15.172:0/1901028002 conn(0x7fea4c0fd8f0 msgr2=0x7fea4c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:10.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.946+0000 7fea52409640 1 -- 172.21.15.172:0/1901028002 shutdown_connections 2024-09-10T08:56:10.954 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:10.946+0000 7fea52409640 1 -- 172.21.15.172:0/1901028002 wait complete. 2024-09-10T08:56:12.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:12 smithi174 ceph-mon[34068]: pgmap v649: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:56:12.791 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:56:12.792 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:56:12.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:12 smithi191 ceph-mon[35314]: pgmap v649: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:56:12.814 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:12 smithi172 ceph-mon[30854]: pgmap v649: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:56:13.058 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:56:13.498 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:13 smithi172 ceph-mon[30854]: pgmap v650: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-09-10T08:56:13.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.590+0000 7fef1968a640 1 -- 172.21.15.172:0/24191561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef14100e20 msgr2=0x7fef14101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:13.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.590+0000 7fef1968a640 1 --2- 172.21.15.172:0/24191561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef14100e20 0x7fef14101200 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fef04009900 tx=0x7fef0402a480 comp rx=0 tx=0).stop 2024-09-10T08:56:13.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.591+0000 7fef1968a640 1 -- 172.21.15.172:0/24191561 shutdown_connections 2024-09-10T08:56:13.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.591+0000 7fef1968a640 1 --2- 172.21.15.172:0/24191561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef14102ef0 0x7fef1410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.591+0000 7fef1968a640 1 --2- 172.21.15.172:0/24191561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef14102570 0x7fef141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.591+0000 7fef1968a640 1 --2- 172.21.15.172:0/24191561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef14100e20 0x7fef14101200 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.591+0000 7fef1968a640 1 -- 172.21.15.172:0/24191561 >> 172.21.15.172:0/24191561 conn(0x7fef140fc7d0 msgr2=0x7fef140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:13.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.591+0000 7fef1968a640 1 -- 172.21.15.172:0/24191561 shutdown_connections 2024-09-10T08:56:13.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.591+0000 7fef1968a640 1 -- 172.21.15.172:0/24191561 wait complete. 2024-09-10T08:56:13.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.591+0000 7fef1968a640 1 Processor -- start 2024-09-10T08:56:13.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef1968a640 1 -- start start 2024-09-10T08:56:13.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef1968a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef14100e20 0x7fef14198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef1968a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef14102570 0x7fef141989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef1968a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef14102ef0 0x7fef1419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef1968a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fef14110e80 con 0x7fef14102570 2024-09-10T08:56:13.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef1968a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef14110d00 con 0x7fef14100e20 2024-09-10T08:56:13.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef1968a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fef14111000 con 0x7fef14102ef0 2024-09-10T08:56:13.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef127fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef14102570 0x7fef141989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef127fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef14102570 0x7fef141989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43112/0 (socket says 172.21.15.172:43112) 2024-09-10T08:56:13.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef127fc640 1 -- 172.21.15.172:0/1428788067 learned_addr learned my addr 172.21.15.172:0/1428788067 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:13.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef12ffd640 1 --2- 172.21.15.172:0/1428788067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef14100e20 0x7fef14198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef137fe640 1 --2- 172.21.15.172:0/1428788067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef14102ef0 0x7fef1419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef127fc640 1 -- 172.21.15.172:0/1428788067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef14102ef0 msgr2=0x7fef1419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:13.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef127fc640 1 --2- 172.21.15.172:0/1428788067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef14102ef0 0x7fef1419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef127fc640 1 -- 172.21.15.172:0/1428788067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef14100e20 msgr2=0x7fef14198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:13.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef127fc640 1 --2- 172.21.15.172:0/1428788067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef14100e20 0x7fef14198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef127fc640 1 -- 172.21.15.172:0/1428788067 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1419d410 con 0x7fef14102570 2024-09-10T08:56:13.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef12ffd640 1 --2- 172.21.15.172:0/1428788067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef14100e20 0x7fef14198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:13.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.592+0000 7fef137fe640 1 --2- 172.21.15.172:0/1428788067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef14102ef0 0x7fef1419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:13.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.593+0000 7fef127fc640 1 --2- 172.21.15.172:0/1428788067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef14102570 0x7fef141989a0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fef0800e9f0 tx=0x7fef0800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:13.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.593+0000 7feefbfff640 1 -- 172.21.15.172:0/1428788067 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0800cd80 con 0x7fef14102570 2024-09-10T08:56:13.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.593+0000 7fef1968a640 1 -- 172.21.15.172:0/1428788067 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef1419d700 con 0x7fef14102570 2024-09-10T08:56:13.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.593+0000 7feefbfff640 1 -- 172.21.15.172:0/1428788067 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef080054d0 con 0x7fef14102570 2024-09-10T08:56:13.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.593+0000 7feefbfff640 1 -- 172.21.15.172:0/1428788067 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08010490 con 0x7fef14102570 2024-09-10T08:56:13.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.593+0000 7fef1968a640 1 -- 172.21.15.172:0/1428788067 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef141a4fe0 con 0x7fef14102570 2024-09-10T08:56:13.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.595+0000 7feefbfff640 1 -- 172.21.15.172:0/1428788067 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fef08005020 con 0x7fef14102570 2024-09-10T08:56:13.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.596+0000 7fef1ba63740 1 Processor -- start 2024-09-10T08:56:13.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.596+0000 7feefbfff640 1 --2- 172.21.15.172:0/1428788067 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feefc0a2c00 0x7feefc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.596+0000 7fef1ba63740 1 -- start start 2024-09-10T08:56:13.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.596+0000 7fef137fe640 1 --2- 172.21.15.172:0/1428788067 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feefc0a2c00 0x7feefc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.596+0000 7feefbfff640 1 -- 172.21.15.172:0/1428788067 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fef080cf7f0 con 0x7fef14102570 2024-09-10T08:56:13.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.596+0000 7fef1ba63740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632162a40 0x556632162e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.596+0000 7fef1ba63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632163360 0x556632171c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.596+0000 7fef1ba63740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632172150 0x556632174540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.596+0000 7fef1ba63740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5566320c8280 con 0x556632172150 2024-09-10T08:56:13.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.596+0000 7fef1ba63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55663206fb10 con 0x556632163360 2024-09-10T08:56:13.602 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.596+0000 7fef1ba63740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5566320c4240 con 0x556632162a40 2024-09-10T08:56:13.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.597+0000 7fef12ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632162a40 0x556632162e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.597+0000 7fef12ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632162a40 0x556632162e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:50714/0 (socket says 172.21.15.172:50714) 2024-09-10T08:56:13.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.597+0000 7fef12ffd640 1 -- 172.21.15.172:0/1299337218 learned_addr learned my addr 172.21.15.172:0/1299337218 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:13.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.597+0000 7fef137fe640 1 --2- 172.21.15.172:0/1299337218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632172150 0x556632174540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.597+0000 7fef127fc640 1 --2- 172.21.15.172:0/1299337218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632163360 0x556632171c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.597+0000 7fef137fe640 1 --2- 172.21.15.172:0/1428788067 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feefc0a2c00 0x7feefc0a50c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fef0c0126a0 tx=0x7fef0c012aa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:13.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.597+0000 7fef12ffd640 1 -- 172.21.15.172:0/1299337218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632163360 msgr2=0x556632171c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:13.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.597+0000 7fef12ffd640 1 --2- 172.21.15.172:0/1299337218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632163360 0x556632171c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.597+0000 7fef12ffd640 1 -- 172.21.15.172:0/1299337218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632172150 msgr2=0x556632174540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:13.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.597+0000 7fef12ffd640 1 --2- 172.21.15.172:0/1299337218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632172150 0x556632174540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.597+0000 7fef12ffd640 1 -- 172.21.15.172:0/1299337218 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55663207c1f0 con 0x556632162a40 2024-09-10T08:56:13.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.597+0000 7fef137fe640 1 --2- 172.21.15.172:0/1299337218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632172150 0x556632174540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:13.603 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.597+0000 7fef12ffd640 1 --2- 172.21.15.172:0/1299337218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632162a40 0x556632162e20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fef04009900 tx=0x7fef040042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7069cc0521fd43ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:13.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.598+0000 7feef97fa640 1 -- 172.21.15.172:0/1299337218 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef04035030 con 0x556632162a40 2024-09-10T08:56:13.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.598+0000 7feef97fa640 1 -- 172.21.15.172:0/1299337218 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef040054f0 con 0x556632162a40 2024-09-10T08:56:13.604 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.598+0000 7feef97fa640 1 -- 172.21.15.172:0/1299337218 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef040035f0 con 0x556632162a40 2024-09-10T08:56:13.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.598+0000 7fef1ba63740 1 -- 172.21.15.172:0/1299337218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632162a40 msgr2=0x556632162e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:13.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.598+0000 7fef1ba63740 1 --2- 172.21.15.172:0/1299337218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632162a40 0x556632162e20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fef04009900 tx=0x7fef040042e0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.598+0000 7fef1ba63740 1 -- 172.21.15.172:0/1299337218 shutdown_connections 2024-09-10T08:56:13.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.598+0000 7fef1ba63740 1 --2- 172.21.15.172:0/1299337218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632172150 0x556632174540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.598+0000 7fef1ba63740 1 --2- 172.21.15.172:0/1299337218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632163360 0x556632171c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.598+0000 7fef1ba63740 1 --2- 172.21.15.172:0/1299337218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632162a40 0x556632162e20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.598+0000 7fef1ba63740 1 -- 172.21.15.172:0/1299337218 >> 172.21.15.172:0/1299337218 conn(0x55663215f0f0 msgr2=0x55663215f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:13.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.598+0000 7fef1ba63740 1 -- 172.21.15.172:0/1299337218 shutdown_connections 2024-09-10T08:56:13.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.598+0000 7fef1ba63740 1 -- 172.21.15.172:0/1299337218 wait complete. 2024-09-10T08:56:13.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.599+0000 7fef1ba63740 1 -- start start 2024-09-10T08:56:13.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.599+0000 7fef1ba63740 1 Processor -- start 2024-09-10T08:56:13.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.599+0000 7fef1ba63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632162a40 0x5566321718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.599+0000 7fef1ba63740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632163360 0x55663216ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.599+0000 7fef1ba63740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632172150 0x55663216b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.599+0000 7fef1ba63740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556632174d20 con 0x556632172150 2024-09-10T08:56:13.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.599+0000 7fef1ba63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556632082f10 con 0x556632162a40 2024-09-10T08:56:13.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.599+0000 7fef1ba63740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556632174ea0 con 0x556632163360 2024-09-10T08:56:13.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.599+0000 7fef1ba63740 10 client.0 _ll_register_callbacks cb 0x55663204ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:13.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.599+0000 7fef137fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632172150 0x55663216b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.599+0000 7fef137fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632172150 0x55663216b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43128/0 (socket says 172.21.15.172:43128) 2024-09-10T08:56:13.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.599+0000 7fef137fe640 1 -- 172.21.15.172:0/1579855370 learned_addr learned my addr 172.21.15.172:0/1579855370 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:13.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.599+0000 7fef12ffd640 1 --2- 172.21.15.172:0/1579855370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632162a40 0x5566321718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.599+0000 7fef127fc640 1 --2- 172.21.15.172:0/1579855370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632163360 0x55663216ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.600+0000 7fef137fe640 1 -- 172.21.15.172:0/1579855370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632163360 msgr2=0x55663216ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:13.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.600+0000 7fef137fe640 1 --2- 172.21.15.172:0/1579855370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632163360 0x55663216ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.600+0000 7fef137fe640 1 -- 172.21.15.172:0/1579855370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632162a40 msgr2=0x5566321718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:13.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.600+0000 7fef137fe640 1 --2- 172.21.15.172:0/1579855370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632162a40 0x5566321718a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.600+0000 7fef137fe640 1 -- 172.21.15.172:0/1579855370 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556631e396a0 con 0x556632172150 2024-09-10T08:56:13.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.600+0000 7fef137fe640 1 --2- 172.21.15.172:0/1579855370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632172150 0x55663216b180 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fef0c011000 tx=0x7fef0c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d5326cd670aa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:13.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.600+0000 7feef6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:13.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.600+0000 7feef6ff5640 1 -- 172.21.15.172:0/1579855370 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c004d20 con 0x556632172150 2024-09-10T08:56:13.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.600+0000 7fef1ba63740 1 -- 172.21.15.172:0/1579855370 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef040054f0 con 0x556632172150 2024-09-10T08:56:13.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.600+0000 7feef6ff5640 1 -- 172.21.15.172:0/1579855370 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fef0c003970 con 0x556632172150 2024-09-10T08:56:13.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.601+0000 7feef6ff5640 1 -- 172.21.15.172:0/1579855370 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fef0c031050 con 0x556632172150 2024-09-10T08:56:13.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.601+0000 7fef1ba63740 10 client.18456 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:13.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.601+0000 7fef1ba63740 1 -- 172.21.15.172:0/1579855370 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556632170210 con 0x556632172150 2024-09-10T08:56:13.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.601+0000 7feef6ff5640 1 -- 172.21.15.172:0/1579855370 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef040054f0 con 0x556632172150 2024-09-10T08:56:13.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.601+0000 7feef6ff5640 1 -- 172.21.15.172:0/1579855370 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fef0c02b500 con 0x556632172150 2024-09-10T08:56:13.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.601+0000 7fef1ba63740 10 client.18456 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:13.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.601+0000 7fef1ba63740 10 client.18456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:13.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.601+0000 7fef1ba63740 20 client.18456 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:13.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.601+0000 7fef1ba63740 1 --2- 172.21.15.172:0/1579855370 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5566321861e0 0x556632188650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.601+0000 7fef1ba63740 4 client.18456 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:56:13.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.601+0000 7fef1ba63740 1 -- 172.21.15.172:0/1579855370 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566320d57f0 con 0x5566321861e0 2024-09-10T08:56:13.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.602+0000 7fef12ffd640 1 --2- 172.21.15.172:0/1579855370 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5566321861e0 0x556632188650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.602+0000 7feef6ff5640 10 client.18456 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:13.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.602+0000 7fef12ffd640 1 --2- 172.21.15.172:0/1579855370 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5566321861e0 0x556632188650 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5162e34320e25472 server_cookie=1bd0939e45d52580 in_seq=0 out_seq=0 2024-09-10T08:56:13.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.603+0000 7feef6ff5640 1 -- 172.21.15.172:0/1579855370 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566320d57f0 con 0x5566321861e0 2024-09-10T08:56:13.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.603+0000 7feef6ff5640 10 client.18456 handle_command_reply: tid=0 2024-09-10T08:56:13.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.603+0000 7fef1ba63740 1 client.18456 shutdown 2024-09-10T08:56:13.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.603+0000 7fef1ba63740 1 -- 172.21.15.172:0/1579855370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632172150 msgr2=0x55663216b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:13.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.603+0000 7fef1ba63740 1 --2- 172.21.15.172:0/1579855370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632172150 0x55663216b180 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fef0c011000 tx=0x7fef0c004ab0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.603+0000 7fef1ba63740 1 -- 172.21.15.172:0/1579855370 shutdown_connections 2024-09-10T08:56:13.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.603+0000 7fef1ba63740 1 --2- 172.21.15.172:0/1579855370 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5566321861e0 0x556632188650 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.603+0000 7fef1ba63740 1 --2- 172.21.15.172:0/1579855370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632172150 0x55663216b180 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.603+0000 7fef1ba63740 1 --2- 172.21.15.172:0/1579855370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632163360 0x55663216ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.603+0000 7fef1ba63740 1 --2- 172.21.15.172:0/1579855370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632162a40 0x5566321718a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.603+0000 7fef1ba63740 1 -- 172.21.15.172:0/1579855370 >> 172.21.15.172:0/1579855370 conn(0x55663215f0f0 msgr2=0x55663215e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:13.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.603+0000 7feef6ff5640 0 client.18456 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:13.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.603+0000 7fef1ba63740 1 -- 172.21.15.172:0/1579855370 shutdown_connections 2024-09-10T08:56:13.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.603+0000 7fef1ba63740 1 -- 172.21.15.172:0/1579855370 wait complete. 2024-09-10T08:56:13.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.603+0000 7fef1ba63740 20 client.18456 trim_cache size 0 max 16384 2024-09-10T08:56:13.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef1ba63740 1 Processor -- start 2024-09-10T08:56:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef1ba63740 1 -- start start 2024-09-10T08:56:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef1ba63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632162a40 0x55663216f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef1ba63740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632163360 0x556632169b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef1ba63740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632172150 0x556632164c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef1ba63740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fef141ac820 con 0x556632163360 2024-09-10T08:56:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef1ba63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef141ae360 con 0x556632162a40 2024-09-10T08:56:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef1ba63740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fef141acef0 con 0x556632172150 2024-09-10T08:56:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef127fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632163360 0x556632169b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef127fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632163360 0x556632169b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43144/0 (socket says 172.21.15.172:43144) 2024-09-10T08:56:13.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef127fc640 1 -- 172.21.15.172:0/550421072 learned_addr learned my addr 172.21.15.172:0/550421072 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef12ffd640 1 --2- 172.21.15.172:0/550421072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632162a40 0x55663216f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef137fe640 1 --2- 172.21.15.172:0/550421072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632172150 0x556632164c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef127fc640 1 -- 172.21.15.172:0/550421072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632172150 msgr2=0x556632164c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef127fc640 1 --2- 172.21.15.172:0/550421072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632172150 0x556632164c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef127fc640 1 -- 172.21.15.172:0/550421072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632162a40 msgr2=0x55663216f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef127fc640 1 --2- 172.21.15.172:0/550421072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632162a40 0x55663216f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef127fc640 1 -- 172.21.15.172:0/550421072 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556632174880 con 0x556632163360 2024-09-10T08:56:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.630+0000 7fef137fe640 1 --2- 172.21.15.172:0/550421072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632172150 0x556632164c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.631+0000 7fef127fc640 1 --2- 172.21.15.172:0/550421072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632163360 0x556632169b80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fef08009380 tx=0x7fef080cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381a4894aabdf6eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.631+0000 7feef8ff9640 1 -- 172.21.15.172:0/550421072 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef080b8020 con 0x556632163360 2024-09-10T08:56:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.631+0000 7feef8ff9640 1 -- 172.21.15.172:0/550421072 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef08005680 con 0x556632163360 2024-09-10T08:56:13.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.631+0000 7feef8ff9640 1 -- 172.21.15.172:0/550421072 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08016dd0 con 0x556632163360 2024-09-10T08:56:13.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.632+0000 7fef1ba63740 1 -- 172.21.15.172:0/550421072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632163360 msgr2=0x556632169b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:13.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.632+0000 7fef1ba63740 1 --2- 172.21.15.172:0/550421072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632163360 0x556632169b80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fef08009380 tx=0x7fef080cf040 comp rx=0 tx=0).stop 2024-09-10T08:56:13.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.632+0000 7fef1ba63740 1 -- 172.21.15.172:0/550421072 shutdown_connections 2024-09-10T08:56:13.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.632+0000 7fef1ba63740 1 --2- 172.21.15.172:0/550421072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632172150 0x556632164c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.632+0000 7fef1ba63740 1 --2- 172.21.15.172:0/550421072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632163360 0x556632169b80 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.632+0000 7fef1ba63740 1 --2- 172.21.15.172:0/550421072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632162a40 0x55663216f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.632+0000 7fef1ba63740 1 -- 172.21.15.172:0/550421072 >> 172.21.15.172:0/550421072 conn(0x55663215f0f0 msgr2=0x556632164550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:13.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.632+0000 7fef1ba63740 1 -- 172.21.15.172:0/550421072 shutdown_connections 2024-09-10T08:56:13.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.632+0000 7fef1ba63740 1 -- 172.21.15.172:0/550421072 wait complete. 2024-09-10T08:56:13.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.632+0000 7fef1ba63740 1 -- start start 2024-09-10T08:56:13.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.632+0000 7fef1ba63740 1 Processor -- start 2024-09-10T08:56:13.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef1ba63740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632162a40 0x556632183040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef1ba63740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632163360 0x55663217f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef1ba63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632172150 0x55663217f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef1ba63740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fef141adc90 con 0x556632162a40 2024-09-10T08:56:13.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef1ba63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef141ad5c0 con 0x556632172150 2024-09-10T08:56:13.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef1ba63740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fef141aa800 con 0x556632163360 2024-09-10T08:56:13.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef1ba63740 10 client.0 _ll_register_callbacks cb 0x7fef0c010d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:13.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef12ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632162a40 0x556632183040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef12ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632162a40 0x556632183040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43146/0 (socket says 172.21.15.172:43146) 2024-09-10T08:56:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef137fe640 1 --2- 172.21.15.172:0/1494412960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632172150 0x55663217f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef127fc640 1 --2- 172.21.15.172:0/1494412960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632163360 0x55663217f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef12ffd640 1 -- 172.21.15.172:0/1494412960 learned_addr learned my addr 172.21.15.172:0/1494412960 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef137fe640 1 -- 172.21.15.172:0/1494412960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632163360 msgr2=0x55663217f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef137fe640 1 --2- 172.21.15.172:0/1494412960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632163360 0x55663217f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef137fe640 1 -- 172.21.15.172:0/1494412960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632162a40 msgr2=0x556632183040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef137fe640 1 --2- 172.21.15.172:0/1494412960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632162a40 0x556632183040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef137fe640 1 -- 172.21.15.172:0/1494412960 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55663217fe80 con 0x556632172150 2024-09-10T08:56:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef127fc640 1 --2- 172.21.15.172:0/1494412960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632163360 0x55663217f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7fef137fe640 1 --2- 172.21.15.172:0/1494412960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632172150 0x55663217f8c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00a8e0 tx=0x7fef0c02bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcc5bbda404cf214 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:13.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.633+0000 7feef6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:56:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.634+0000 7feef6ff5640 1 -- 172.21.15.172:0/1494412960 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c03a060 con 0x556632172150 2024-09-10T08:56:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.634+0000 7fef1ba63740 1 -- 172.21.15.172:0/1494412960 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef08005680 con 0x556632172150 2024-09-10T08:56:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.634+0000 7feef6ff5640 1 -- 172.21.15.172:0/1494412960 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fef0c02b0b0 con 0x556632172150 2024-09-10T08:56:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.634+0000 7feef6ff5640 1 -- 172.21.15.172:0/1494412960 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fef0c041050 con 0x556632172150 2024-09-10T08:56:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.634+0000 7fef1ba63740 10 client.27404 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.634+0000 7fef1ba63740 1 -- 172.21.15.172:0/1494412960 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5566321f1cf0 con 0x556632172150 2024-09-10T08:56:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.634+0000 7feef6ff5640 1 -- 172.21.15.172:0/1494412960 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef08005680 con 0x556632172150 2024-09-10T08:56:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.634+0000 7feef6ff5640 1 -- 172.21.15.172:0/1494412960 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fef0c01f820 con 0x556632172150 2024-09-10T08:56:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.634+0000 7fef1ba63740 10 client.27404 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.634+0000 7fef1ba63740 10 client.27404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.634+0000 7fef1ba63740 20 client.27404 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.634+0000 7fef1ba63740 1 --2- 172.21.15.172:0/1494412960 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5566321f2410 0x5566321f4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:13.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.634+0000 7fef1ba63740 4 client.27404 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:56:13.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.634+0000 7fef1ba63740 1 -- 172.21.15.172:0/1494412960 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5566320cfd00 con 0x5566321f2410 2024-09-10T08:56:13.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.635+0000 7fef12ffd640 1 --2- 172.21.15.172:0/1494412960 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5566321f2410 0x5566321f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:13.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.635+0000 7feef6ff5640 10 client.27404 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:13.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.635+0000 7fef12ffd640 1 --2- 172.21.15.172:0/1494412960 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5566321f2410 0x5566321f4830 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4633683a99676db server_cookie=32a94e3f47fbd41f in_seq=0 out_seq=0 2024-09-10T08:56:13.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.636+0000 7feef6ff5640 1 -- 172.21.15.172:0/1494412960 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5566320cfd00 con 0x5566321f2410 2024-09-10T08:56:13.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.636+0000 7feef6ff5640 10 client.27404 handle_command_reply: tid=0 2024-09-10T08:56:13.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.636+0000 7fef1ba63740 1 client.27404 shutdown 2024-09-10T08:56:13.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.636+0000 7fef1ba63740 1 -- 172.21.15.172:0/1494412960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632172150 msgr2=0x55663217f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:13.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.636+0000 7fef1ba63740 1 --2- 172.21.15.172:0/1494412960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632172150 0x55663217f8c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00a8e0 tx=0x7fef0c02bd70 comp rx=0 tx=0).stop 2024-09-10T08:56:13.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.636+0000 7fef1ba63740 1 -- 172.21.15.172:0/1494412960 shutdown_connections 2024-09-10T08:56:13.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.636+0000 7fef1ba63740 1 --2- 172.21.15.172:0/1494412960 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5566321f2410 0x5566321f4830 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.637+0000 7fef1ba63740 1 --2- 172.21.15.172:0/1494412960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556632172150 0x55663217f8c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.637+0000 7fef1ba63740 1 --2- 172.21.15.172:0/1494412960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556632163360 0x55663217f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.637+0000 7fef1ba63740 1 --2- 172.21.15.172:0/1494412960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556632162a40 0x556632183040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.637+0000 7fef1ba63740 1 -- 172.21.15.172:0/1494412960 >> 172.21.15.172:0/1494412960 conn(0x55663215f0f0 msgr2=0x556632164d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:13.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.637+0000 7feef6ff5640 0 client.27404 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:13.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.637+0000 7fef1ba63740 1 -- 172.21.15.172:0/1494412960 shutdown_connections 2024-09-10T08:56:13.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.637+0000 7fef1ba63740 1 -- 172.21.15.172:0/1494412960 wait complete. 2024-09-10T08:56:13.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.637+0000 7fef1ba63740 20 client.27404 trim_cache size 0 max 16384 2024-09-10T08:56:13.643 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:56:13.643 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T08:56:13.643 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "c3013e84-7ae7-451b-a669-9e2179fa200d", 2024-09-10T08:56:13.643 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T08:56:13.643 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:56:13.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.637+0000 7fef1968a640 1 -- 172.21.15.172:0/1428788067 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feefc0a2c00 msgr2=0x7feefc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:13.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.637+0000 7fef1968a640 1 --2- 172.21.15.172:0/1428788067 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feefc0a2c00 0x7feefc0a50c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fef0c0126a0 tx=0x7fef0c012aa0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.637+0000 7fef1968a640 1 -- 172.21.15.172:0/1428788067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef14102570 msgr2=0x7fef141989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:13.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.637+0000 7fef1968a640 1 --2- 172.21.15.172:0/1428788067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef14102570 0x7fef141989a0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fef0800e9f0 tx=0x7fef0800edf0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.638+0000 7fef1968a640 1 -- 172.21.15.172:0/1428788067 shutdown_connections 2024-09-10T08:56:13.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.638+0000 7fef1968a640 1 --2- 172.21.15.172:0/1428788067 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feefc0a2c00 0x7feefc0a50c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.638+0000 7fef1968a640 1 --2- 172.21.15.172:0/1428788067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef14102ef0 0x7fef1419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.638+0000 7fef1968a640 1 --2- 172.21.15.172:0/1428788067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef14102570 0x7fef141989a0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.638+0000 7fef1968a640 1 --2- 172.21.15.172:0/1428788067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef14100e20 0x7fef14198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:13.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.638+0000 7fef1968a640 1 -- 172.21.15.172:0/1428788067 >> 172.21.15.172:0/1428788067 conn(0x7fef140fc7d0 msgr2=0x7fef1406a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:13.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.638+0000 7fef1968a640 1 -- 172.21.15.172:0/1428788067 shutdown_connections 2024-09-10T08:56:13.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:13.638+0000 7fef1968a640 1 -- 172.21.15.172:0/1428788067 wait complete. 2024-09-10T08:56:13.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:13 smithi174 ceph-mon[34068]: pgmap v650: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-09-10T08:56:13.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:13 smithi191 ceph-mon[35314]: pgmap v650: 129 pgs: 129 active+clean; 9.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 26 MiB/s wr, 20 op/s 2024-09-10T08:56:14.465 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:14 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T08:56:14.465 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:14 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.465 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:14 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:56:14.465 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:14 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:56:14.466 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:14 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:56:14.466 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:14 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:56:14.466 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:14 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.466 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:14 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.466 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:14 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:56:14.466 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:14 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.466 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:14 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:56:14.466 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:14 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.466 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:14 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.466 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:14 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.495 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:56:14.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:14 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T08:56:14.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:14 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:14 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:56:14.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:14 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:56:14.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:14 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:56:14.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:14 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:56:14.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:14 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:14 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:14 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:56:14.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:14 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:14 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:56:14.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:14 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:14 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:14 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:14 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T08:56:14.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:14 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:14 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:56:14.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:14 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:56:14.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:14 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:56:14.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:14 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:56:14.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:14 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:14 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:14 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:56:14.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:14 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:14 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:56:14.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:14 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:14 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:14 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:56:14.819 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:56:15.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.373+0000 7fcaafda7640 1 -- 172.21.15.172:0/1698821418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa8106a90 msgr2=0x7fcaa8106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:15.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.373+0000 7fcaafda7640 1 --2- 172.21.15.172:0/1698821418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa8106a90 0x7fcaa8106e70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009860 tx=0x7fcaa402a340 comp rx=0 tx=0).stop 2024-09-10T08:56:15.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.374+0000 7fcaafda7640 1 -- 172.21.15.172:0/1698821418 shutdown_connections 2024-09-10T08:56:15.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.374+0000 7fcaafda7640 1 --2- 172.21.15.172:0/1698821418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcaa8101430 0x7fcaa810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.374+0000 7fcaafda7640 1 --2- 172.21.15.172:0/1698821418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaa8100a90 0x7fcaa8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.374+0000 7fcaafda7640 1 --2- 172.21.15.172:0/1698821418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa8106a90 0x7fcaa8106e70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.374+0000 7fcaafda7640 1 -- 172.21.15.172:0/1698821418 >> 172.21.15.172:0/1698821418 conn(0x7fcaa80fc7d0 msgr2=0x7fcaa80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:15.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.374+0000 7fcaafda7640 1 -- 172.21.15.172:0/1698821418 shutdown_connections 2024-09-10T08:56:15.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.374+0000 7fcaafda7640 1 -- 172.21.15.172:0/1698821418 wait complete. 2024-09-10T08:56:15.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.374+0000 7fcaafda7640 1 Processor -- start 2024-09-10T08:56:15.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.374+0000 7fcaafda7640 1 -- start start 2024-09-10T08:56:15.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.375+0000 7fcaafda7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa8100a90 0x7fcaa819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.375+0000 7fcaafda7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcaa8101430 0x7fcaa819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.375+0000 7fcaafda7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaa8106a90 0x7fcaa819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.375+0000 7fcaafda7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa810fc30 con 0x7fcaa8101430 2024-09-10T08:56:15.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.375+0000 7fcaafda7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa810fab0 con 0x7fcaa8100a90 2024-09-10T08:56:15.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.375+0000 7fcaafda7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa810fdb0 con 0x7fcaa8106a90 2024-09-10T08:56:15.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.375+0000 7fcaadb43640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa8100a90 0x7fcaa819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.375+0000 7fcaadb43640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa8100a90 0x7fcaa819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:50808/0 (socket says 172.21.15.172:50808) 2024-09-10T08:56:15.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.375+0000 7fcaadb43640 1 -- 172.21.15.172:0/261320011 learned_addr learned my addr 172.21.15.172:0/261320011 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:15.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.375+0000 7fcaad342640 1 --2- 172.21.15.172:0/261320011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcaa8101430 0x7fcaa819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.375+0000 7fcaae344640 1 --2- 172.21.15.172:0/261320011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaa8106a90 0x7fcaa819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.375+0000 7fcaadb43640 1 -- 172.21.15.172:0/261320011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaa8106a90 msgr2=0x7fcaa819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:15.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.376+0000 7fcaadb43640 1 --2- 172.21.15.172:0/261320011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaa8106a90 0x7fcaa819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.376+0000 7fcaadb43640 1 -- 172.21.15.172:0/261320011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcaa8101430 msgr2=0x7fcaa819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:15.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.376+0000 7fcaadb43640 1 --2- 172.21.15.172:0/261320011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcaa8101430 0x7fcaa819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.376+0000 7fcaadb43640 1 -- 172.21.15.172:0/261320011 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa819f500 con 0x7fcaa8100a90 2024-09-10T08:56:15.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.376+0000 7fcaad342640 1 --2- 172.21.15.172:0/261320011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcaa8101430 0x7fcaa819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:15.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.376+0000 7fcaae344640 1 --2- 172.21.15.172:0/261320011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaa8106a90 0x7fcaa819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:56:15.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.376+0000 7fcaadb43640 1 --2- 172.21.15.172:0/261320011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa8100a90 0x7fcaa819a550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009980 tx=0x7fcaa4030df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:15.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.376+0000 7fca9effd640 1 -- 172.21.15.172:0/261320011 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4037050 con 0x7fcaa8100a90 2024-09-10T08:56:15.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.376+0000 7fcaafda7640 1 -- 172.21.15.172:0/261320011 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa819f790 con 0x7fcaa8100a90 2024-09-10T08:56:15.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.376+0000 7fca9effd640 1 -- 172.21.15.172:0/261320011 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcaa4043070 con 0x7fcaa8100a90 2024-09-10T08:56:15.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.376+0000 7fca9effd640 1 -- 172.21.15.172:0/261320011 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4003c50 con 0x7fcaa8100a90 2024-09-10T08:56:15.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.376+0000 7fcaafda7640 1 -- 172.21.15.172:0/261320011 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa81a70d0 con 0x7fcaa8100a90 2024-09-10T08:56:15.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.379+0000 7fca9effd640 1 -- 172.21.15.172:0/261320011 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fcaa4004040 con 0x7fcaa8100a90 2024-09-10T08:56:15.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.379+0000 7fcab2180740 1 Processor -- start 2024-09-10T08:56:15.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.379+0000 7fcab2180740 1 -- start start 2024-09-10T08:56:15.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.379+0000 7fcab2180740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc384a0 0x55fbdfc38880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fcab2180740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc43140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fcab2180740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc43680 0x55fbdfc45a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fcab2180740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fbdfb9e2e0 con 0x55fbdfc43680 2024-09-10T08:56:15.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fcab2180740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fbdfb45ce0 con 0x55fbdfc38dc0 2024-09-10T08:56:15.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fcab2180740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55fbdfb9a2a0 con 0x55fbdfc384a0 2024-09-10T08:56:15.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fcaadb43640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc43140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fcaad342640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc43680 0x55fbdfc45a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fcaadb43640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc43140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:50822/0 (socket says 172.21.15.172:50822) 2024-09-10T08:56:15.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fcaadb43640 1 -- 172.21.15.172:0/707200889 learned_addr learned my addr 172.21.15.172:0/707200889 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:15.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fcaae344640 1 --2- 172.21.15.172:0/707200889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc384a0 0x55fbdfc38880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fca9effd640 1 --2- 172.21.15.172:0/261320011 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca940a2b80 0x7fca940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.463 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fcaae344640 1 --2- 172.21.15.172:0/261320011 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca940a2b80 0x7fca940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fcaad342640 1 -- 172.21.15.172:0/707200889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc384a0 msgr2=0x55fbdfc38880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:15.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fcaad342640 1 --2- 172.21.15.172:0/707200889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc384a0 0x55fbdfc38880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fcaad342640 1 -- 172.21.15.172:0/707200889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc38dc0 msgr2=0x55fbdfc43140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:15.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fca9effd640 1 -- 172.21.15.172:0/261320011 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcaa40ef9b0 con 0x7fcaa8100a90 2024-09-10T08:56:15.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fcaad342640 1 --2- 172.21.15.172:0/707200889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc43140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fcaad342640 1 -- 172.21.15.172:0/707200889 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbdfb2a440 con 0x55fbdfc43680 2024-09-10T08:56:15.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.380+0000 7fcaad342640 1 --2- 172.21.15.172:0/707200889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc43680 0x55fbdfc45a70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fca9800de90 tx=0x7fca98005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4550dc246a93e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:15.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.381+0000 7fca93fff640 1 -- 172.21.15.172:0/707200889 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98004230 con 0x55fbdfc43680 2024-09-10T08:56:15.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.381+0000 7fcaae344640 1 --2- 172.21.15.172:0/261320011 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca940a2b80 0x7fca940a5040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0005fd0 tx=0x7fcaa0004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:15.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.381+0000 7fca93fff640 1 -- 172.21.15.172:0/707200889 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca980043d0 con 0x55fbdfc43680 2024-09-10T08:56:15.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.381+0000 7fca93fff640 1 -- 172.21.15.172:0/707200889 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca980177a0 con 0x55fbdfc43680 2024-09-10T08:56:15.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.381+0000 7fcab2180740 1 -- 172.21.15.172:0/707200889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc43680 msgr2=0x55fbdfc45a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:15.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.381+0000 7fcab2180740 1 --2- 172.21.15.172:0/707200889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc43680 0x55fbdfc45a70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fca9800de90 tx=0x7fca98005f90 comp rx=0 tx=0).stop 2024-09-10T08:56:15.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.381+0000 7fcab2180740 1 -- 172.21.15.172:0/707200889 shutdown_connections 2024-09-10T08:56:15.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.381+0000 7fcab2180740 1 --2- 172.21.15.172:0/707200889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc43680 0x55fbdfc45a70 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.381+0000 7fcab2180740 1 --2- 172.21.15.172:0/707200889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc43140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.381+0000 7fcab2180740 1 --2- 172.21.15.172:0/707200889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc384a0 0x55fbdfc38880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.382+0000 7fcab2180740 1 -- 172.21.15.172:0/707200889 >> 172.21.15.172:0/707200889 conn(0x55fbdfc34a30 msgr2=0x55fbdfc34e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:15.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.382+0000 7fcab2180740 1 -- 172.21.15.172:0/707200889 shutdown_connections 2024-09-10T08:56:15.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.382+0000 7fcab2180740 1 -- 172.21.15.172:0/707200889 wait complete. 2024-09-10T08:56:15.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.382+0000 7fcab2180740 1 -- start start 2024-09-10T08:56:15.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.382+0000 7fcab2180740 1 Processor -- start 2024-09-10T08:56:15.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcab2180740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc384a0 0x55fbdfc42f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcab2180740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc401b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcab2180740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc43680 0x55fbdfc406f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcab2180740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fbdfc46250 con 0x55fbdfc38dc0 2024-09-10T08:56:15.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcab2180740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fbdfb59050 con 0x55fbdfc384a0 2024-09-10T08:56:15.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcab2180740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55fbdfc463d0 con 0x55fbdfc43680 2024-09-10T08:56:15.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcab2180740 10 client.0 _ll_register_callbacks cb 0x55fbdfb99160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:15.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcaad342640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcaadb43640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc384a0 0x55fbdfc42f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcaad342640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc401b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43178/0 (socket says 172.21.15.172:43178) 2024-09-10T08:56:15.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcaadb43640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc384a0 0x55fbdfc42f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:50834/0 (socket says 172.21.15.172:50834) 2024-09-10T08:56:15.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcaad342640 1 -- 172.21.15.172:0/23588873 learned_addr learned my addr 172.21.15.172:0/23588873 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:15.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcaae344640 1 --2- 172.21.15.172:0/23588873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc43680 0x55fbdfc406f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcaadb43640 1 -- 172.21.15.172:0/23588873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc43680 msgr2=0x55fbdfc406f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:15.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcaadb43640 1 --2- 172.21.15.172:0/23588873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc43680 0x55fbdfc406f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcaadb43640 1 -- 172.21.15.172:0/23588873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc38dc0 msgr2=0x55fbdfc401b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:15.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcaadb43640 1 --2- 172.21.15.172:0/23588873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc401b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcaadb43640 1 -- 172.21.15.172:0/23588873 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbdf90f3c0 con 0x55fbdfc384a0 2024-09-10T08:56:15.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcaad342640 1 --2- 172.21.15.172:0/23588873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:15.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fcaadb43640 1 --2- 172.21.15.172:0/23588873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc384a0 0x55fbdfc42f00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40b8820 tx=0x7fcaa4032880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ee5142c87c45c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:15.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fca917fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:56:15.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.383+0000 7fca917fa640 1 -- 172.21.15.172:0/23588873 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40bb990 con 0x55fbdfc384a0 2024-09-10T08:56:15.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.384+0000 7fca917fa640 1 -- 172.21.15.172:0/23588873 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcaa40ba150 con 0x55fbdfc384a0 2024-09-10T08:56:15.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.384+0000 7fcab2180740 1 -- 172.21.15.172:0/23588873 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca980043d0 con 0x55fbdfc384a0 2024-09-10T08:56:15.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.384+0000 7fca917fa640 1 -- 172.21.15.172:0/23588873 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcaa40bb7b0 con 0x55fbdfc384a0 2024-09-10T08:56:15.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.384+0000 7fcab2180740 10 client.27416 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:15.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.384+0000 7fcab2180740 1 -- 172.21.15.172:0/23588873 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fbdfc51b60 con 0x55fbdfc384a0 2024-09-10T08:56:15.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.384+0000 7fca917fa640 1 -- 172.21.15.172:0/23588873 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca980043d0 con 0x55fbdfc384a0 2024-09-10T08:56:15.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.384+0000 7fca917fa640 1 -- 172.21.15.172:0/23588873 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fcaa40b84c0 con 0x55fbdfc384a0 2024-09-10T08:56:15.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.384+0000 7fcab2180740 10 client.27416 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:15.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.384+0000 7fcab2180740 10 client.27416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:15.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.384+0000 7fcab2180740 20 client.27416 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:15.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.384+0000 7fcab2180740 1 --2- 172.21.15.172:0/23588873 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fbdfc52eb0 0x55fbdfc55340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.384+0000 7fcab2180740 4 client.27416 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:56:15.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.384+0000 7fcab2180740 1 -- 172.21.15.172:0/23588873 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbdfbab850 con 0x55fbdfc52eb0 2024-09-10T08:56:15.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.384+0000 7fcaad342640 1 --2- 172.21.15.172:0/23588873 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fbdfc52eb0 0x55fbdfc55340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.385+0000 7fcaad342640 1 --2- 172.21.15.172:0/23588873 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fbdfc52eb0 0x55fbdfc55340 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=638028cf57ef6724 server_cookie=3879d7d2acaa1939 in_seq=0 out_seq=0 2024-09-10T08:56:15.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.385+0000 7fca917fa640 10 client.27416 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:15.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.385+0000 7fca917fa640 1 -- 172.21.15.172:0/23588873 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fbdfbab850 con 0x55fbdfc52eb0 2024-09-10T08:56:15.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.385+0000 7fca917fa640 10 client.27416 handle_command_reply: tid=0 2024-09-10T08:56:15.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.386+0000 7fcab2180740 1 client.27416 shutdown 2024-09-10T08:56:15.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.386+0000 7fcab2180740 1 -- 172.21.15.172:0/23588873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc384a0 msgr2=0x55fbdfc42f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:15.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.386+0000 7fcab2180740 1 --2- 172.21.15.172:0/23588873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc384a0 0x55fbdfc42f00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40b8820 tx=0x7fcaa4032880 comp rx=0 tx=0).stop 2024-09-10T08:56:15.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.386+0000 7fcab2180740 1 -- 172.21.15.172:0/23588873 shutdown_connections 2024-09-10T08:56:15.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.386+0000 7fcab2180740 1 --2- 172.21.15.172:0/23588873 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fbdfc52eb0 0x55fbdfc55340 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.386+0000 7fcab2180740 1 --2- 172.21.15.172:0/23588873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc43680 0x55fbdfc406f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.386+0000 7fcab2180740 1 --2- 172.21.15.172:0/23588873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.386+0000 7fcab2180740 1 --2- 172.21.15.172:0/23588873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc384a0 0x55fbdfc42f00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.386+0000 7fcab2180740 1 -- 172.21.15.172:0/23588873 >> 172.21.15.172:0/23588873 conn(0x55fbdfc34a30 msgr2=0x55fbdfc3a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:15.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.386+0000 7fca917fa640 0 client.27416 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:15.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.386+0000 7fcab2180740 1 -- 172.21.15.172:0/23588873 shutdown_connections 2024-09-10T08:56:15.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.386+0000 7fcab2180740 1 -- 172.21.15.172:0/23588873 wait complete. 2024-09-10T08:56:15.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.386+0000 7fcab2180740 20 client.27416 trim_cache size 0 max 16384 2024-09-10T08:56:15.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.411+0000 7fcab2180740 1 Processor -- start 2024-09-10T08:56:15.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.411+0000 7fcab2180740 1 -- start start 2024-09-10T08:56:15.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.411+0000 7fcab2180740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc384a0 0x55fbdfc4f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.411+0000 7fcab2180740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc3f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.411+0000 7fcab2180740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc43680 0x55fbdfc3b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.411+0000 7fcab2180740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81ac920 con 0x55fbdfc38dc0 2024-09-10T08:56:15.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.411+0000 7fcab2180740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81ae460 con 0x55fbdfc384a0 2024-09-10T08:56:15.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.411+0000 7fcab2180740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81acff0 con 0x55fbdfc43680 2024-09-10T08:56:15.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.411+0000 7fcaad342640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc3f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.411+0000 7fcaadb43640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc384a0 0x55fbdfc4f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.411+0000 7fcaad342640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc3f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43188/0 (socket says 172.21.15.172:43188) 2024-09-10T08:56:15.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.411+0000 7fcaad342640 1 -- 172.21.15.172:0/2492183842 learned_addr learned my addr 172.21.15.172:0/2492183842 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:15.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.412+0000 7fcaae344640 1 --2- 172.21.15.172:0/2492183842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc43680 0x55fbdfc3b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.412+0000 7fcaad342640 1 -- 172.21.15.172:0/2492183842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc43680 msgr2=0x55fbdfc3b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:15.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.412+0000 7fcaad342640 1 --2- 172.21.15.172:0/2492183842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc43680 0x55fbdfc3b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.412+0000 7fcaad342640 1 -- 172.21.15.172:0/2492183842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc384a0 msgr2=0x55fbdfc4f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:15.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.412+0000 7fcaad342640 1 --2- 172.21.15.172:0/2492183842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc384a0 0x55fbdfc4f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.412+0000 7fcaad342640 1 -- 172.21.15.172:0/2492183842 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbdfc3f160 con 0x55fbdfc38dc0 2024-09-10T08:56:15.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.412+0000 7fcaadb43640 1 --2- 172.21.15.172:0/2492183842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc384a0 0x55fbdfc4f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:15.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.412+0000 7fcaad342640 1 --2- 172.21.15.172:0/2492183842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc3f6f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fca98009fd0 tx=0x7fca9800d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228b83d6e0cb863c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:15.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.412+0000 7fca937fe640 1 -- 172.21.15.172:0/2492183842 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98005610 con 0x55fbdfc38dc0 2024-09-10T08:56:15.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.412+0000 7fca937fe640 1 -- 172.21.15.172:0/2492183842 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca98003c70 con 0x55fbdfc38dc0 2024-09-10T08:56:15.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.412+0000 7fca937fe640 1 -- 172.21.15.172:0/2492183842 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98017ca0 con 0x55fbdfc38dc0 2024-09-10T08:56:15.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.413+0000 7fcab2180740 1 -- 172.21.15.172:0/2492183842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc38dc0 msgr2=0x55fbdfc3f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:15.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.413+0000 7fcab2180740 1 --2- 172.21.15.172:0/2492183842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc3f6f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fca98009fd0 tx=0x7fca9800d630 comp rx=0 tx=0).stop 2024-09-10T08:56:15.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.413+0000 7fcab2180740 1 -- 172.21.15.172:0/2492183842 shutdown_connections 2024-09-10T08:56:15.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.413+0000 7fcab2180740 1 --2- 172.21.15.172:0/2492183842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc43680 0x55fbdfc3b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.413+0000 7fcab2180740 1 --2- 172.21.15.172:0/2492183842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc3f6f0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.413+0000 7fcab2180740 1 --2- 172.21.15.172:0/2492183842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc384a0 0x55fbdfc4f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.413+0000 7fcab2180740 1 -- 172.21.15.172:0/2492183842 >> 172.21.15.172:0/2492183842 conn(0x55fbdfc34a30 msgr2=0x55fbdfc3b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:15.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.413+0000 7fcab2180740 1 -- 172.21.15.172:0/2492183842 shutdown_connections 2024-09-10T08:56:15.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.413+0000 7fcab2180740 1 -- 172.21.15.172:0/2492183842 wait complete. 2024-09-10T08:56:15.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.413+0000 7fcab2180740 1 -- start start 2024-09-10T08:56:15.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.414+0000 7fcab2180740 1 Processor -- start 2024-09-10T08:56:15.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.414+0000 7fcab2180740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc384a0 0x55fbdfc4b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.414+0000 7fcab2180740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc4bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.414+0000 7fcab2180740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc43680 0x55fbdfc46f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.414+0000 7fcaadb43640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc384a0 0x55fbdfc4b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.414+0000 7fcaadb43640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc384a0 0x55fbdfc4b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43194/0 (socket says 172.21.15.172:43194) 2024-09-10T08:56:15.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.414+0000 7fcab2180740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81add90 con 0x55fbdfc384a0 2024-09-10T08:56:15.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.414+0000 7fcab2180740 1 -- 172.21.15.172:0/3418809520 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81ad6c0 con 0x55fbdfc38dc0 2024-09-10T08:56:15.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.414+0000 7fcaadb43640 1 -- 172.21.15.172:0/3418809520 learned_addr learned my addr 172.21.15.172:0/3418809520 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:15.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.414+0000 7fcab2180740 1 -- 172.21.15.172:0/3418809520 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81aa900 con 0x55fbdfc43680 2024-09-10T08:56:15.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.414+0000 7fcab2180740 10 client.0 _ll_register_callbacks cb 0x7fcaa4005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:15.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.414+0000 7fcaad342640 1 --2- 172.21.15.172:0/3418809520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc4bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.414+0000 7fcaae344640 1 --2- 172.21.15.172:0/3418809520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc43680 0x55fbdfc46f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.415+0000 7fcaad342640 1 -- 172.21.15.172:0/3418809520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc43680 msgr2=0x55fbdfc46f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:15.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.415+0000 7fcaad342640 1 --2- 172.21.15.172:0/3418809520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc43680 0x55fbdfc46f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.415+0000 7fcaad342640 1 -- 172.21.15.172:0/3418809520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc384a0 msgr2=0x55fbdfc4b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:15.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.415+0000 7fcaad342640 1 --2- 172.21.15.172:0/3418809520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc384a0 0x55fbdfc4b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.415+0000 7fcaad342640 1 -- 172.21.15.172:0/3418809520 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbdfc47760 con 0x55fbdfc38dc0 2024-09-10T08:56:15.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.415+0000 7fcaad342640 1 --2- 172.21.15.172:0/3418809520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc4bf20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55fbdfc26770 tx=0x7fca98003ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7be1e1c565d51ae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:15.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.415+0000 7fca917fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:56:15.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.415+0000 7fca917fa640 1 -- 172.21.15.172:0/3418809520 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9803a020 con 0x55fbdfc38dc0 2024-09-10T08:56:15.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.415+0000 7fcab2180740 1 -- 172.21.15.172:0/3418809520 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca98003c70 con 0x55fbdfc38dc0 2024-09-10T08:56:15.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.415+0000 7fca917fa640 1 -- 172.21.15.172:0/3418809520 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fca98036660 con 0x55fbdfc38dc0 2024-09-10T08:56:15.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.415+0000 7fca917fa640 1 -- 172.21.15.172:0/3418809520 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fca98003a90 con 0x55fbdfc38dc0 2024-09-10T08:56:15.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.415+0000 7fcab2180740 10 client.27422 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:15.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.415+0000 7fcab2180740 1 -- 172.21.15.172:0/3418809520 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fbdfcbd8c0 con 0x55fbdfc38dc0 2024-09-10T08:56:15.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.416+0000 7fca917fa640 1 -- 172.21.15.172:0/3418809520 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca98003c70 con 0x55fbdfc38dc0 2024-09-10T08:56:15.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.416+0000 7fca917fa640 1 -- 172.21.15.172:0/3418809520 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fcaa40ecc40 con 0x55fbdfc38dc0 2024-09-10T08:56:15.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.416+0000 7fcab2180740 10 client.27422 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:15.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.416+0000 7fcab2180740 10 client.27422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:15.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.416+0000 7fcab2180740 20 client.27422 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:15.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.416+0000 7fcab2180740 1 --2- 172.21.15.172:0/3418809520 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fbdfcbddd0 0x55fbdfcc01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:15.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.416+0000 7fcab2180740 4 client.27422 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:56:15.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.416+0000 7fcab2180740 1 -- 172.21.15.172:0/3418809520 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fbdfba5d60 con 0x55fbdfcbddd0 2024-09-10T08:56:15.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.416+0000 7fcaadb43640 1 --2- 172.21.15.172:0/3418809520 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fbdfcbddd0 0x55fbdfcc01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:15.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.416+0000 7fca917fa640 10 client.27422 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:15.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.416+0000 7fcaadb43640 1 --2- 172.21.15.172:0/3418809520 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fbdfcbddd0 0x55fbdfcc01c0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cde26e4261e9b3a server_cookie=ee51dcbf47a39473 in_seq=0 out_seq=0 2024-09-10T08:56:15.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.417+0000 7fca917fa640 1 -- 172.21.15.172:0/3418809520 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fbdfba5d60 con 0x55fbdfcbddd0 2024-09-10T08:56:15.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.417+0000 7fca917fa640 10 client.27422 handle_command_reply: tid=0 2024-09-10T08:56:15.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.417+0000 7fcab2180740 1 client.27422 shutdown 2024-09-10T08:56:15.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.417+0000 7fcab2180740 1 -- 172.21.15.172:0/3418809520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc38dc0 msgr2=0x55fbdfc4bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:15.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.417+0000 7fcab2180740 1 --2- 172.21.15.172:0/3418809520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc4bf20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55fbdfc26770 tx=0x7fca98003ec0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.417+0000 7fcab2180740 1 -- 172.21.15.172:0/3418809520 shutdown_connections 2024-09-10T08:56:15.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.417+0000 7fcab2180740 1 --2- 172.21.15.172:0/3418809520 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fbdfcbddd0 0x55fbdfcc01c0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.417+0000 7fcab2180740 1 --2- 172.21.15.172:0/3418809520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fbdfc43680 0x55fbdfc46f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.417+0000 7fca917fa640 0 client.27422 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:15.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.417+0000 7fcab2180740 1 --2- 172.21.15.172:0/3418809520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fbdfc38dc0 0x55fbdfc4bf20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.417+0000 7fcab2180740 1 --2- 172.21.15.172:0/3418809520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fbdfc384a0 0x55fbdfc4b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.417+0000 7fcab2180740 1 -- 172.21.15.172:0/3418809520 >> 172.21.15.172:0/3418809520 conn(0x55fbdfc34a30 msgr2=0x55fbdfc33b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:15.477 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:56:15.477 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:56:15.477 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T08:56:15.477 INFO:teuthology.orchestra.run.smithi172.stdout: "c3013e84-7ae7-451b-a669-9e2179fa200d": { 2024-09-10T08:56:15.477 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T08:56:15.477 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "c3013e84-7ae7-451b-a669-9e2179fa200d", 2024-09-10T08:56:15.477 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T08:56:15.477 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:56:15.477 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:56:15.477 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:56:15.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.417+0000 7fcab2180740 1 -- 172.21.15.172:0/3418809520 shutdown_connections 2024-09-10T08:56:15.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.417+0000 7fcab2180740 1 -- 172.21.15.172:0/3418809520 wait complete. 2024-09-10T08:56:15.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.417+0000 7fcab2180740 20 client.27422 trim_cache size 0 max 16384 2024-09-10T08:56:15.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.418+0000 7fcaafda7640 1 -- 172.21.15.172:0/261320011 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca940a2b80 msgr2=0x7fca940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:15.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.418+0000 7fcaafda7640 1 --2- 172.21.15.172:0/261320011 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca940a2b80 0x7fca940a5040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0005fd0 tx=0x7fcaa0004ab0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.418+0000 7fcaafda7640 1 -- 172.21.15.172:0/261320011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa8100a90 msgr2=0x7fcaa819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:15.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.418+0000 7fcaafda7640 1 --2- 172.21.15.172:0/261320011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa8100a90 0x7fcaa819a550 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009980 tx=0x7fcaa4030df0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.418+0000 7fcaafda7640 1 -- 172.21.15.172:0/261320011 shutdown_connections 2024-09-10T08:56:15.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.418+0000 7fcaafda7640 1 --2- 172.21.15.172:0/261320011 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fca940a2b80 0x7fca940a5040 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.418+0000 7fcaafda7640 1 --2- 172.21.15.172:0/261320011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaa8106a90 0x7fcaa819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.418+0000 7fcaafda7640 1 --2- 172.21.15.172:0/261320011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcaa8101430 0x7fcaa819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.418+0000 7fcaafda7640 1 --2- 172.21.15.172:0/261320011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa8100a90 0x7fcaa819a550 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:15.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.418+0000 7fcaafda7640 1 -- 172.21.15.172:0/261320011 >> 172.21.15.172:0/261320011 conn(0x7fcaa80fc7d0 msgr2=0x7fcaa80fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:15.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.418+0000 7fcaafda7640 1 -- 172.21.15.172:0/261320011 shutdown_connections 2024-09-10T08:56:15.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:15.418+0000 7fcaafda7640 1 -- 172.21.15.172:0/261320011 wait complete. 2024-09-10T08:56:15.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:15 smithi172 ceph-mon[30854]: pgmap v651: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 40 op/s 2024-09-10T08:56:15.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:15 smithi174 ceph-mon[34068]: pgmap v651: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 40 op/s 2024-09-10T08:56:15.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:15 smithi191 ceph-mon[35314]: pgmap v651: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 40 op/s 2024-09-10T08:56:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:17 smithi174 ceph-mon[34068]: pgmap v652: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 MiB/s wr, 41 op/s 2024-09-10T08:56:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:17 smithi191 ceph-mon[35314]: pgmap v652: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 MiB/s wr, 41 op/s 2024-09-10T08:56:17.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:17 smithi172 ceph-mon[30854]: pgmap v652: 129 pgs: 129 active+clean; 9.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 MiB/s wr, 41 op/s 2024-09-10T08:56:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:19 smithi174 ceph-mon[34068]: pgmap v653: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 MiB/s wr, 47 op/s 2024-09-10T08:56:19.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:19 smithi191 ceph-mon[35314]: pgmap v653: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 MiB/s wr, 47 op/s 2024-09-10T08:56:19.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:19 smithi172 ceph-mon[30854]: pgmap v653: 129 pgs: 129 active+clean; 9.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 MiB/s wr, 47 op/s 2024-09-10T08:56:21.464 INFO:tasks.cephfs.filesystem:scrub status for tag:c3013e84-7ae7-451b-a669-9e2179fa200d - {'path': '/', 'tag': 'c3013e84-7ae7-451b-a669-9e2179fa200d', 'options': 'recursive,force'} 2024-09-10T08:56:21.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:21 smithi174 ceph-mon[34068]: pgmap v654: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 MiB/s wr, 46 op/s 2024-09-10T08:56:21.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:21 smithi191 ceph-mon[35314]: pgmap v654: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 MiB/s wr, 46 op/s 2024-09-10T08:56:21.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:21 smithi172 ceph-mon[30854]: pgmap v654: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 MiB/s wr, 46 op/s 2024-09-10T08:56:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.264+0000 7f537935b640 1 -- 172.21.15.172:0/991362617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374102600 msgr2=0x7f5374108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.264+0000 7f537935b640 1 --2- 172.21.15.172:0/991362617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374102600 0x7f5374108e90 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f536c009ca0 tx=0x7f536c02c790 comp rx=0 tx=0).stop 2024-09-10T08:56:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.264+0000 7f537935b640 1 -- 172.21.15.172:0/991362617 shutdown_connections 2024-09-10T08:56:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.264+0000 7f537935b640 1 --2- 172.21.15.172:0/991362617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374102600 0x7f5374108e90 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.264+0000 7f537935b640 1 --2- 172.21.15.172:0/991362617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5374101c40 0x7f53741020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.264+0000 7f537935b640 1 --2- 172.21.15.172:0/991362617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5374100a50 0x7f5374100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.264+0000 7f537935b640 1 -- 172.21.15.172:0/991362617 >> 172.21.15.172:0/991362617 conn(0x7f53740fc1b0 msgr2=0x7f53740fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.264+0000 7f537935b640 1 -- 172.21.15.172:0/991362617 shutdown_connections 2024-09-10T08:56:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.264+0000 7f537935b640 1 -- 172.21.15.172:0/991362617 wait complete. 2024-09-10T08:56:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.265+0000 7f537935b640 1 Processor -- start 2024-09-10T08:56:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.265+0000 7f537935b640 1 -- start start 2024-09-10T08:56:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.265+0000 7f537935b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5374100a50 0x7f5374195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.265+0000 7f537935b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374101c40 0x7f5374195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.265+0000 7f537935b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5374102600 0x7f537419d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.265+0000 7f537935b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f537410b480 con 0x7f5374101c40 2024-09-10T08:56:23.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.265+0000 7f537935b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f537410b300 con 0x7f5374100a50 2024-09-10T08:56:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.265+0000 7f537935b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f537410b600 con 0x7f5374102600 2024-09-10T08:56:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.265+0000 7f53727fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374101c40 0x7f5374195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.265+0000 7f53727fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374101c40 0x7f5374195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35794/0 (socket says 172.21.15.172:35794) 2024-09-10T08:56:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.265+0000 7f53727fc640 1 -- 172.21.15.172:0/2880117179 learned_addr learned my addr 172.21.15.172:0/2880117179 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.266+0000 7f5372ffd640 1 --2- 172.21.15.172:0/2880117179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5374100a50 0x7f5374195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.266+0000 7f53737fe640 1 --2- 172.21.15.172:0/2880117179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5374102600 0x7f537419d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.266+0000 7f53727fc640 1 -- 172.21.15.172:0/2880117179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5374102600 msgr2=0x7f537419d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.266+0000 7f53727fc640 1 --2- 172.21.15.172:0/2880117179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5374102600 0x7f537419d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.266+0000 7f53727fc640 1 -- 172.21.15.172:0/2880117179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5374100a50 msgr2=0x7f5374195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.266+0000 7f53727fc640 1 --2- 172.21.15.172:0/2880117179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5374100a50 0x7f5374195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.266+0000 7f53727fc640 1 -- 172.21.15.172:0/2880117179 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537419d6d0 con 0x7f5374101c40 2024-09-10T08:56:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.266+0000 7f53727fc640 1 --2- 172.21.15.172:0/2880117179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374101c40 0x7f5374195ff0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f536800ea60 tx=0x7f536800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:23.269 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.266+0000 7f535bfff640 1 -- 172.21.15.172:0/2880117179 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536800cde0 con 0x7f5374101c40 2024-09-10T08:56:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.266+0000 7f535bfff640 1 -- 172.21.15.172:0/2880117179 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53680035f0 con 0x7f5374101c40 2024-09-10T08:56:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.266+0000 7f537935b640 1 -- 172.21.15.172:0/2880117179 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537419d9c0 con 0x7f5374101c40 2024-09-10T08:56:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.266+0000 7f535bfff640 1 -- 172.21.15.172:0/2880117179 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368010690 con 0x7f5374101c40 2024-09-10T08:56:23.270 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.266+0000 7f537935b640 1 -- 172.21.15.172:0/2880117179 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537419df60 con 0x7f5374101c40 2024-09-10T08:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.269+0000 7f535bfff640 1 -- 172.21.15.172:0/2880117179 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f53680037a0 con 0x7f5374101c40 2024-09-10T08:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.269+0000 7f537c1ce740 1 Processor -- start 2024-09-10T08:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.269+0000 7f537c1ce740 1 -- start start 2024-09-10T08:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.269+0000 7f537c1ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d9989a0 0x56079d998da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.269+0000 7f537c1ce740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d9992e0 0x56079d99dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.269+0000 7f535bfff640 1 --2- 172.21.15.172:0/2880117179 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f535c0a2cb0 0x7f535c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.269+0000 7f537c1ce740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d99e370 0x56079d9a0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.269+0000 7f537c1ce740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56079d8fc2f0 con 0x56079d99e370 2024-09-10T08:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.269+0000 7f537c1ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56079d8a4270 con 0x56079d9989a0 2024-09-10T08:56:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.269+0000 7f537c1ce740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56079d8f82b0 con 0x56079d9992e0 2024-09-10T08:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.269+0000 7f53737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d99e370 0x56079d9a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.269+0000 7f5372ffd640 1 --2- 172.21.15.172:0/2880117179 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f535c0a2cb0 0x7f535c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.269+0000 7f535bfff640 1 -- 172.21.15.172:0/2880117179 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f53680cf710 con 0x7f5374101c40 2024-09-10T08:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.269+0000 7f53727fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d9992e0 0x56079d99dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.269+0000 7f53737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d99e370 0x56079d9a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35810/0 (socket says 172.21.15.172:35810) 2024-09-10T08:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.270+0000 7f53737fe640 1 -- 172.21.15.172:0/127533154 learned_addr learned my addr 172.21.15.172:0/127533154 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.270+0000 7f53737fe640 1 --2- 172.21.15.172:0/127533154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d9989a0 0x56079d998da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.270+0000 7f53727fc640 1 -- 172.21.15.172:0/127533154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d9989a0 msgr2=0x56079d998da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.270+0000 7f53727fc640 1 --2- 172.21.15.172:0/127533154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d9989a0 0x56079d998da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.270+0000 7f53727fc640 1 -- 172.21.15.172:0/127533154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d99e370 msgr2=0x56079d9a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.270+0000 7f53727fc640 1 --2- 172.21.15.172:0/127533154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d99e370 0x56079d9a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.270+0000 7f53727fc640 1 -- 172.21.15.172:0/127533154 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56079d8a3340 con 0x56079d9992e0 2024-09-10T08:56:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.270+0000 7f53737fe640 1 --2- 172.21.15.172:0/127533154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d9989a0 0x56079d998da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.270+0000 7f5372ffd640 1 --2- 172.21.15.172:0/2880117179 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f535c0a2cb0 0x7f535c0a5170 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5374101aa0 tx=0x7f5364005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.270+0000 7f53727fc640 1 --2- 172.21.15.172:0/127533154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d9992e0 0x56079d99dda0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f536800d010 tx=0x7f5368097f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68e7b18362d14e5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.270+0000 7f53597fa640 1 -- 172.21.15.172:0/127533154 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368003b30 con 0x56079d9992e0 2024-09-10T08:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.270+0000 7f53597fa640 1 -- 172.21.15.172:0/127533154 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f536809c310 con 0x56079d9992e0 2024-09-10T08:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.270+0000 7f53597fa640 1 -- 172.21.15.172:0/127533154 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368098280 con 0x56079d9992e0 2024-09-10T08:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.271+0000 7f537c1ce740 1 -- 172.21.15.172:0/127533154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d9992e0 msgr2=0x56079d99dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.271+0000 7f537c1ce740 1 --2- 172.21.15.172:0/127533154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d9992e0 0x56079d99dda0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f536800d010 tx=0x7f5368097f30 comp rx=0 tx=0).stop 2024-09-10T08:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.271+0000 7f537c1ce740 1 -- 172.21.15.172:0/127533154 shutdown_connections 2024-09-10T08:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.271+0000 7f537c1ce740 1 --2- 172.21.15.172:0/127533154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d99e370 0x56079d9a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.271+0000 7f537c1ce740 1 --2- 172.21.15.172:0/127533154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d9992e0 0x56079d99dda0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.271+0000 7f537c1ce740 1 --2- 172.21.15.172:0/127533154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d9989a0 0x56079d998da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.271+0000 7f537c1ce740 1 -- 172.21.15.172:0/127533154 >> 172.21.15.172:0/127533154 conn(0x56079d994010 msgr2=0x56079d994410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.271+0000 7f537c1ce740 1 -- 172.21.15.172:0/127533154 shutdown_connections 2024-09-10T08:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.271+0000 7f537c1ce740 1 -- 172.21.15.172:0/127533154 wait complete. 2024-09-10T08:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.271+0000 7f537c1ce740 1 -- start start 2024-09-10T08:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.271+0000 7f537c1ce740 1 Processor -- start 2024-09-10T08:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f537c1ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d9989a0 0x56079d9a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f537c1ce740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d9992e0 0x56079d9a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f537c1ce740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d99e370 0x56079d9aa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f537c1ce740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56079d9a12e0 con 0x56079d99e370 2024-09-10T08:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f537c1ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56079d885290 con 0x56079d9989a0 2024-09-10T08:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f537c1ce740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56079d9a1460 con 0x56079d9992e0 2024-09-10T08:56:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f537c1ce740 10 client.0 _ll_register_callbacks cb 0x56079d8b6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f53737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d99e370 0x56079d9aa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f53727fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d9992e0 0x56079d9a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f53737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d99e370 0x56079d9aa0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35818/0 (socket says 172.21.15.172:35818) 2024-09-10T08:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f53737fe640 1 -- 172.21.15.172:0/253984809 learned_addr learned my addr 172.21.15.172:0/253984809 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f5372ffd640 1 --2- 172.21.15.172:0/253984809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d9989a0 0x56079d9a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f53737fe640 1 -- 172.21.15.172:0/253984809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d9992e0 msgr2=0x56079d9a30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f53737fe640 1 --2- 172.21.15.172:0/253984809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d9992e0 0x56079d9a30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f53737fe640 1 -- 172.21.15.172:0/253984809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d9989a0 msgr2=0x56079d9a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f53737fe640 1 --2- 172.21.15.172:0/253984809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d9989a0 0x56079d9a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f53737fe640 1 -- 172.21.15.172:0/253984809 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56079d60f210 con 0x56079d99e370 2024-09-10T08:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f53727fc640 1 --2- 172.21.15.172:0/253984809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d9992e0 0x56079d9a30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f53737fe640 1 --2- 172.21.15.172:0/253984809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d99e370 0x56079d9aa0f0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f536c033a80 tx=0x7f536c031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab73c35acc7084b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f5356ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f5356ff5640 1 -- 172.21.15.172:0/253984809 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c04f060 con 0x56079d99e370 2024-09-10T08:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f5356ff5640 1 -- 172.21.15.172:0/253984809 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f536c031bc0 con 0x56079d99e370 2024-09-10T08:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.272+0000 7f537c1ce740 1 -- 172.21.15.172:0/253984809 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f536809c310 con 0x56079d99e370 2024-09-10T08:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.273+0000 7f5356ff5640 1 -- 172.21.15.172:0/253984809 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f536c056050 con 0x56079d99e370 2024-09-10T08:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.273+0000 7f537c1ce740 10 client.18492 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.273+0000 7f537c1ce740 1 -- 172.21.15.172:0/253984809 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56079d6db620 con 0x56079d99e370 2024-09-10T08:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.273+0000 7f5356ff5640 1 -- 172.21.15.172:0/253984809 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f536809c310 con 0x56079d99e370 2024-09-10T08:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.273+0000 7f5356ff5640 1 -- 172.21.15.172:0/253984809 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f536c00b070 con 0x56079d99e370 2024-09-10T08:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.273+0000 7f537c1ce740 10 client.18492 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.273+0000 7f537c1ce740 10 client.18492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.273+0000 7f537c1ce740 20 client.18492 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.273+0000 7f537c1ce740 1 --2- 172.21.15.172:0/253984809 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56079d9ae830 0x56079d9b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.273+0000 7f537c1ce740 4 client.18492 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.273+0000 7f537c1ce740 1 -- 172.21.15.172:0/253984809 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56079d909a20 con 0x56079d9ae830 2024-09-10T08:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.273+0000 7f5372ffd640 1 --2- 172.21.15.172:0/253984809 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56079d9ae830 0x56079d9b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.274+0000 7f5356ff5640 10 client.18492 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.274+0000 7f5372ffd640 1 --2- 172.21.15.172:0/253984809 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56079d9ae830 0x56079d9b0c20 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb4c947654212d4 server_cookie=a89edf688a41c7dc in_seq=0 out_seq=0 2024-09-10T08:56:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.275+0000 7f5356ff5640 1 -- 172.21.15.172:0/253984809 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56079d909a20 con 0x56079d9ae830 2024-09-10T08:56:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.275+0000 7f5356ff5640 10 client.18492 handle_command_reply: tid=0 2024-09-10T08:56:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.275+0000 7f537c1ce740 1 client.18492 shutdown 2024-09-10T08:56:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.275+0000 7f537c1ce740 1 -- 172.21.15.172:0/253984809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d99e370 msgr2=0x56079d9aa0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.275+0000 7f537c1ce740 1 --2- 172.21.15.172:0/253984809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d99e370 0x56079d9aa0f0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f536c033a80 tx=0x7f536c031740 comp rx=0 tx=0).stop 2024-09-10T08:56:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.275+0000 7f537c1ce740 1 -- 172.21.15.172:0/253984809 shutdown_connections 2024-09-10T08:56:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.275+0000 7f537c1ce740 1 --2- 172.21.15.172:0/253984809 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56079d9ae830 0x56079d9b0c20 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.275+0000 7f537c1ce740 1 --2- 172.21.15.172:0/253984809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d99e370 0x56079d9aa0f0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.275+0000 7f537c1ce740 1 --2- 172.21.15.172:0/253984809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d9992e0 0x56079d9a30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.275+0000 7f537c1ce740 1 --2- 172.21.15.172:0/253984809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d9989a0 0x56079d9a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.275+0000 7f537c1ce740 1 -- 172.21.15.172:0/253984809 >> 172.21.15.172:0/253984809 conn(0x56079d994010 msgr2=0x56079d995820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.275+0000 7f5356ff5640 0 client.18492 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.275+0000 7f537c1ce740 1 -- 172.21.15.172:0/253984809 shutdown_connections 2024-09-10T08:56:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.275+0000 7f537c1ce740 1 -- 172.21.15.172:0/253984809 wait complete. 2024-09-10T08:56:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.275+0000 7f537c1ce740 20 client.18492 trim_cache size 0 max 16384 2024-09-10T08:56:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.290+0000 7f537c1ce740 1 Processor -- start 2024-09-10T08:56:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.290+0000 7f537c1ce740 1 -- start start 2024-09-10T08:56:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.290+0000 7f537c1ce740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d9989a0 0x56079d9930b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.290+0000 7f537c1ce740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d9992e0 0x56079d9aa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.290+0000 7f537c1ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d99e370 0x56079d9b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.290+0000 7f537c1ce740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f53741a8350 con 0x56079d9992e0 2024-09-10T08:56:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.290+0000 7f537c1ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f53741a9e90 con 0x56079d99e370 2024-09-10T08:56:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.290+0000 7f537c1ce740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f53741a8a20 con 0x56079d9989a0 2024-09-10T08:56:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.290+0000 7f53727fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d9992e0 0x56079d9aa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.290+0000 7f53727fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d9992e0 0x56079d9aa0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35820/0 (socket says 172.21.15.172:35820) 2024-09-10T08:56:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.290+0000 7f53727fc640 1 -- 172.21.15.172:0/1415325652 learned_addr learned my addr 172.21.15.172:0/1415325652 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.291+0000 7f5372ffd640 1 --2- 172.21.15.172:0/1415325652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d9989a0 0x56079d9930b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.291+0000 7f53737fe640 1 --2- 172.21.15.172:0/1415325652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d99e370 0x56079d9b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.291+0000 7f53727fc640 1 -- 172.21.15.172:0/1415325652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d9989a0 msgr2=0x56079d9930b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.291+0000 7f53727fc640 1 --2- 172.21.15.172:0/1415325652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d9989a0 0x56079d9930b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.291+0000 7f53727fc640 1 -- 172.21.15.172:0/1415325652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d99e370 msgr2=0x56079d9b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.291+0000 7f53727fc640 1 --2- 172.21.15.172:0/1415325652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d99e370 0x56079d9b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.291+0000 7f53727fc640 1 -- 172.21.15.172:0/1415325652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56079d9997c0 con 0x56079d9992e0 2024-09-10T08:56:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.291+0000 7f53737fe640 1 --2- 172.21.15.172:0/1415325652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d99e370 0x56079d9b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:56:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.291+0000 7f53727fc640 1 --2- 172.21.15.172:0/1415325652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d9992e0 0x56079d9aa0b0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f5368098020 tx=0x7f53680045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb009ecf5f879be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.291+0000 7f5358ff9640 1 -- 172.21.15.172:0/1415325652 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536809bc10 con 0x56079d9992e0 2024-09-10T08:56:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.291+0000 7f5358ff9640 1 -- 172.21.15.172:0/1415325652 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f536809ffc0 con 0x56079d9992e0 2024-09-10T08:56:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.291+0000 7f5358ff9640 1 -- 172.21.15.172:0/1415325652 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536801fbe0 con 0x56079d9992e0 2024-09-10T08:56:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.292+0000 7f537c1ce740 1 -- 172.21.15.172:0/1415325652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d9992e0 msgr2=0x56079d9aa0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.292+0000 7f537c1ce740 1 --2- 172.21.15.172:0/1415325652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d9992e0 0x56079d9aa0b0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f5368098020 tx=0x7f53680045d0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.292+0000 7f537c1ce740 1 -- 172.21.15.172:0/1415325652 shutdown_connections 2024-09-10T08:56:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.292+0000 7f537c1ce740 1 --2- 172.21.15.172:0/1415325652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d99e370 0x56079d9b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.292+0000 7f537c1ce740 1 --2- 172.21.15.172:0/1415325652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d9992e0 0x56079d9aa0b0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.292+0000 7f537c1ce740 1 --2- 172.21.15.172:0/1415325652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d9989a0 0x56079d9930b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.292+0000 7f537c1ce740 1 -- 172.21.15.172:0/1415325652 >> 172.21.15.172:0/1415325652 conn(0x56079d994010 msgr2=0x56079d9929a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.292+0000 7f537c1ce740 1 -- 172.21.15.172:0/1415325652 shutdown_connections 2024-09-10T08:56:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.292+0000 7f537c1ce740 1 -- 172.21.15.172:0/1415325652 wait complete. 2024-09-10T08:56:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.292+0000 7f537c1ce740 1 -- start start 2024-09-10T08:56:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.292+0000 7f537c1ce740 1 Processor -- start 2024-09-10T08:56:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f537c1ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d9989a0 0x56079d9a76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f537c1ce740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d9992e0 0x56079d9a7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f537c1ce740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d99e370 0x56079d9a8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f537c1ce740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f53741a97c0 con 0x56079d9992e0 2024-09-10T08:56:23.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f537c1ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f53741a90f0 con 0x56079d9989a0 2024-09-10T08:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f537c1ce740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f53741a6330 con 0x56079d99e370 2024-09-10T08:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f537c1ce740 10 client.0 _ll_register_callbacks cb 0x7f536c033730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f53727fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d9992e0 0x56079d9a7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f53727fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d9992e0 0x56079d9a7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35822/0 (socket says 172.21.15.172:35822) 2024-09-10T08:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f53727fc640 1 -- 172.21.15.172:0/2268420231 learned_addr learned my addr 172.21.15.172:0/2268420231 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f53737fe640 1 --2- 172.21.15.172:0/2268420231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d99e370 0x56079d9a8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f5372ffd640 1 --2- 172.21.15.172:0/2268420231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d9989a0 0x56079d9a76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f53727fc640 1 -- 172.21.15.172:0/2268420231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d99e370 msgr2=0x56079d9a8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f53727fc640 1 --2- 172.21.15.172:0/2268420231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d99e370 0x56079d9a8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f53727fc640 1 -- 172.21.15.172:0/2268420231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d9989a0 msgr2=0x56079d9a76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f53727fc640 1 --2- 172.21.15.172:0/2268420231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d9989a0 0x56079d9a76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f53727fc640 1 -- 172.21.15.172:0/2268420231 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56079d9a91d0 con 0x56079d9992e0 2024-09-10T08:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f5372ffd640 1 --2- 172.21.15.172:0/2268420231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d9989a0 0x56079d9a76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f53727fc640 1 --2- 172.21.15.172:0/2268420231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d9992e0 0x56079d9a7c00 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f536809b910 tx=0x7f5368097f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a29d87f39e398e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f5356ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f5356ff5640 1 -- 172.21.15.172:0/2268420231 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368099cb0 con 0x56079d9992e0 2024-09-10T08:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.293+0000 7f5356ff5640 1 -- 172.21.15.172:0/2268420231 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f536809b060 con 0x56079d9992e0 2024-09-10T08:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.294+0000 7f537c1ce740 1 -- 172.21.15.172:0/2268420231 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f536809ffc0 con 0x56079d9992e0 2024-09-10T08:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.294+0000 7f5356ff5640 1 -- 172.21.15.172:0/2268420231 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5368004000 con 0x56079d9992e0 2024-09-10T08:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.294+0000 7f537c1ce740 10 client.18498 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.294+0000 7f537c1ce740 1 -- 172.21.15.172:0/2268420231 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56079d9aa450 con 0x56079d9992e0 2024-09-10T08:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.294+0000 7f5356ff5640 1 -- 172.21.15.172:0/2268420231 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f536809ffc0 con 0x56079d9992e0 2024-09-10T08:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.294+0000 7f5356ff5640 1 -- 172.21.15.172:0/2268420231 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f5368097af0 con 0x56079d9992e0 2024-09-10T08:56:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.294+0000 7f537c1ce740 10 client.18498 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.294+0000 7f537c1ce740 10 client.18498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.294+0000 7f537c1ce740 20 client.18498 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.294+0000 7f537c1ce740 1 --2- 172.21.15.172:0/2268420231 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56079da197d0 0x56079da1bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.294+0000 7f537c1ce740 4 client.18498 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.294+0000 7f537c1ce740 1 -- 172.21.15.172:0/2268420231 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56079d903d70 con 0x56079da197d0 2024-09-10T08:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.294+0000 7f5372ffd640 1 --2- 172.21.15.172:0/2268420231 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56079da197d0 0x56079da1bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.295+0000 7f5356ff5640 10 client.18498 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.295+0000 7f5372ffd640 1 --2- 172.21.15.172:0/2268420231 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56079da197d0 0x56079da1bbc0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab5457afaf4639a server_cookie=92c0e75d2229d9f5 in_seq=0 out_seq=0 2024-09-10T08:56:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.301+0000 7f5356ff5640 1 -- 172.21.15.172:0/2268420231 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56079d903d70 con 0x56079da197d0 2024-09-10T08:56:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.301+0000 7f5356ff5640 10 client.18498 handle_command_reply: tid=0 2024-09-10T08:56:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.301+0000 7f537c1ce740 1 client.18498 shutdown 2024-09-10T08:56:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.301+0000 7f537c1ce740 1 -- 172.21.15.172:0/2268420231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d9992e0 msgr2=0x56079d9a7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.301+0000 7f537c1ce740 1 --2- 172.21.15.172:0/2268420231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d9992e0 0x56079d9a7c00 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f536809b910 tx=0x7f5368097f20 comp rx=0 tx=0).stop 2024-09-10T08:56:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.301+0000 7f537c1ce740 1 -- 172.21.15.172:0/2268420231 shutdown_connections 2024-09-10T08:56:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.301+0000 7f537c1ce740 1 --2- 172.21.15.172:0/2268420231 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56079da197d0 0x56079da1bbc0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.301+0000 7f537c1ce740 1 --2- 172.21.15.172:0/2268420231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56079d99e370 0x56079d9a8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.301+0000 7f5356ff5640 0 client.18498 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.301+0000 7f537c1ce740 1 --2- 172.21.15.172:0/2268420231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56079d9992e0 0x56079d9a7c00 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.302+0000 7f537c1ce740 1 --2- 172.21.15.172:0/2268420231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56079d9989a0 0x56079d9a76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.302+0000 7f537c1ce740 1 -- 172.21.15.172:0/2268420231 >> 172.21.15.172:0/2268420231 conn(0x56079d994010 msgr2=0x56079d9931c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.302+0000 7f537c1ce740 1 -- 172.21.15.172:0/2268420231 shutdown_connections 2024-09-10T08:56:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.302+0000 7f537c1ce740 1 -- 172.21.15.172:0/2268420231 wait complete. 2024-09-10T08:56:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.302+0000 7f537c1ce740 20 client.18498 trim_cache size 0 max 16384 2024-09-10T08:56:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:56:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:56:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:56:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:56:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.302+0000 7f537935b640 1 -- 172.21.15.172:0/2880117179 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f535c0a2cb0 msgr2=0x7f535c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.302+0000 7f537935b640 1 --2- 172.21.15.172:0/2880117179 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f535c0a2cb0 0x7f535c0a5170 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5374101aa0 tx=0x7f5364005f90 comp rx=0 tx=0).stop 2024-09-10T08:56:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.302+0000 7f537935b640 1 -- 172.21.15.172:0/2880117179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374101c40 msgr2=0x7f5374195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.302+0000 7f537935b640 1 --2- 172.21.15.172:0/2880117179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374101c40 0x7f5374195ff0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f536800ea60 tx=0x7f536800ef20 comp rx=0 tx=0).stop 2024-09-10T08:56:23.305 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.302+0000 7f537935b640 1 -- 172.21.15.172:0/2880117179 shutdown_connections 2024-09-10T08:56:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.302+0000 7f537935b640 1 --2- 172.21.15.172:0/2880117179 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f535c0a2cb0 0x7f535c0a5170 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.302+0000 7f537935b640 1 --2- 172.21.15.172:0/2880117179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5374102600 0x7f537419d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.302+0000 7f537935b640 1 --2- 172.21.15.172:0/2880117179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5374101c40 0x7f5374195ff0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.302+0000 7f537935b640 1 --2- 172.21.15.172:0/2880117179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5374100a50 0x7f5374195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.302+0000 7f537935b640 1 -- 172.21.15.172:0/2880117179 >> 172.21.15.172:0/2880117179 conn(0x7f53740fc1b0 msgr2=0x7f5374105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.302+0000 7f537935b640 1 -- 172.21.15.172:0/2880117179 shutdown_connections 2024-09-10T08:56:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:23.303+0000 7f537935b640 1 -- 172.21.15.172:0/2880117179 wait complete. 2024-09-10T08:56:23.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:23 smithi174 ceph-mon[34068]: pgmap v655: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 MiB/s wr, 43 op/s 2024-09-10T08:56:23.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:23 smithi191 ceph-mon[35314]: pgmap v655: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 MiB/s wr, 43 op/s 2024-09-10T08:56:23.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:23 smithi172 ceph-mon[30854]: pgmap v655: 129 pgs: 129 active+clean; 9.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 28 MiB/s wr, 43 op/s 2024-09-10T08:56:25.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:24 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:56:25.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:24 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:56:25.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:24 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:56:25.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:25 smithi174 ceph-mon[34068]: pgmap v656: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 MiB/s wr, 51 op/s 2024-09-10T08:56:26.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:25 smithi191 ceph-mon[35314]: pgmap v656: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 MiB/s wr, 51 op/s 2024-09-10T08:56:26.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:25 smithi172 ceph-mon[30854]: pgmap v656: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 MiB/s wr, 51 op/s 2024-09-10T08:56:27.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:27 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:56:27.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:27 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:56:27.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:27 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:56:27.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:27 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:56:27.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:56:27.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:56:27.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:27 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:56:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:27 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:56:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:27 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:56:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:27 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:56:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:56:27.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:56:27.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:27 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:56:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:27 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:56:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:27 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:56:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:27 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:56:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:56:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:56:27.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:56:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:28 smithi172 ceph-mon[30854]: pgmap v657: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 MiB/s wr, 31 op/s 2024-09-10T08:56:28.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:28 smithi174 ceph-mon[34068]: pgmap v657: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 MiB/s wr, 31 op/s 2024-09-10T08:56:28.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:28 smithi191 ceph-mon[35314]: pgmap v657: 129 pgs: 129 active+clean; 9.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 28 MiB/s wr, 31 op/s 2024-09-10T08:56:30.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:30 smithi174 ceph-mon[34068]: pgmap v658: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 27 op/s 2024-09-10T08:56:30.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:30 smithi191 ceph-mon[35314]: pgmap v658: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 27 op/s 2024-09-10T08:56:30.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:30 smithi172 ceph-mon[30854]: pgmap v658: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 27 op/s 2024-09-10T08:56:32.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:32 smithi174 ceph-mon[34068]: pgmap v659: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 23 op/s 2024-09-10T08:56:32.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:32 smithi191 ceph-mon[35314]: pgmap v659: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 23 op/s 2024-09-10T08:56:32.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:32 smithi172 ceph-mon[30854]: pgmap v659: 129 pgs: 129 active+clean; 9.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 23 op/s 2024-09-10T08:56:34.602 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:34 smithi191 ceph-mon[35314]: pgmap v660: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-09-10T08:56:34.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:34 smithi174 ceph-mon[34068]: pgmap v660: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-09-10T08:56:34.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:34 smithi172 ceph-mon[30854]: pgmap v660: 129 pgs: 129 active+clean; 9.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 28 MiB/s wr, 21 op/s 2024-09-10T08:56:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.403+0000 7ff16e798640 1 -- 172.21.15.172:0/291657102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff168101c40 msgr2=0x7ff1681020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.403+0000 7ff16e798640 1 --2- 172.21.15.172:0/291657102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff168101c40 0x7ff1681020c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009900 tx=0x7ff15c02a410 comp rx=0 tx=0).stop 2024-09-10T08:56:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.403+0000 7ff16e798640 1 -- 172.21.15.172:0/291657102 shutdown_connections 2024-09-10T08:56:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.403+0000 7ff16e798640 1 --2- 172.21.15.172:0/291657102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff168102600 0x7ff168108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.403+0000 7ff16e798640 1 --2- 172.21.15.172:0/291657102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff168101c40 0x7ff1681020c0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.403+0000 7ff16e798640 1 --2- 172.21.15.172:0/291657102 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff168100a50 0x7ff168100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.403+0000 7ff16e798640 1 -- 172.21.15.172:0/291657102 >> 172.21.15.172:0/291657102 conn(0x7ff1680fc1b0 msgr2=0x7ff1680fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.403+0000 7ff16e798640 1 -- 172.21.15.172:0/291657102 shutdown_connections 2024-09-10T08:56:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.403+0000 7ff16e798640 1 -- 172.21.15.172:0/291657102 wait complete. 2024-09-10T08:56:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.404+0000 7ff16e798640 1 Processor -- start 2024-09-10T08:56:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.404+0000 7ff16e798640 1 -- start start 2024-09-10T08:56:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.404+0000 7ff16e798640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff168100a50 0x7ff168197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.404+0000 7ff16e798640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff168101c40 0x7ff168198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.404+0000 7ff16e798640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff168102600 0x7ff16819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.404+0000 7ff16e798640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff16810c580 con 0x7ff168101c40 2024-09-10T08:56:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.404+0000 7ff16e798640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff16810c400 con 0x7ff168102600 2024-09-10T08:56:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.404+0000 7ff16e798640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff16810c700 con 0x7ff168100a50 2024-09-10T08:56:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.404+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff168101c40 0x7ff168198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.404+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff168101c40 0x7ff168198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51404/0 (socket says 172.21.15.172:51404) 2024-09-10T08:56:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.404+0000 7ff1677fe640 1 -- 172.21.15.172:0/3756277601 learned_addr learned my addr 172.21.15.172:0/3756277601 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.405+0000 7ff16cd35640 1 --2- 172.21.15.172:0/3756277601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff168102600 0x7ff16819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.405+0000 7ff167fff640 1 --2- 172.21.15.172:0/3756277601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff168100a50 0x7ff168197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.405+0000 7ff1677fe640 1 -- 172.21.15.172:0/3756277601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff168100a50 msgr2=0x7ff168197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.405+0000 7ff1677fe640 1 --2- 172.21.15.172:0/3756277601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff168100a50 0x7ff168197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.405+0000 7ff1677fe640 1 -- 172.21.15.172:0/3756277601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff168102600 msgr2=0x7ff16819f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.405+0000 7ff1677fe640 1 --2- 172.21.15.172:0/3756277601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff168102600 0x7ff16819f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.405+0000 7ff1677fe640 1 -- 172.21.15.172:0/3756277601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16819f930 con 0x7ff168101c40 2024-09-10T08:56:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.405+0000 7ff16cd35640 1 --2- 172.21.15.172:0/3756277601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff168102600 0x7ff16819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.405+0000 7ff1677fe640 1 --2- 172.21.15.172:0/3756277601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff168101c40 0x7ff168198260 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7ff15c004770 tx=0x7ff15c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.405+0000 7ff1657fa640 1 -- 172.21.15.172:0/3756277601 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c03f030 con 0x7ff168101c40 2024-09-10T08:56:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.405+0000 7ff1657fa640 1 -- 172.21.15.172:0/3756277601 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff15c003cb0 con 0x7ff168101c40 2024-09-10T08:56:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.405+0000 7ff16e798640 1 -- 172.21.15.172:0/3756277601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16819fbc0 con 0x7ff168101c40 2024-09-10T08:56:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.405+0000 7ff1657fa640 1 -- 172.21.15.172:0/3756277601 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c035070 con 0x7ff168101c40 2024-09-10T08:56:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.405+0000 7ff16e798640 1 -- 172.21.15.172:0/3756277601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1681a00b0 con 0x7ff168101c40 2024-09-10T08:56:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.407+0000 7ff1657fa640 1 -- 172.21.15.172:0/3756277601 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff15c042030 con 0x7ff168101c40 2024-09-10T08:56:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.408+0000 7ff171613740 1 Processor -- start 2024-09-10T08:56:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.408+0000 7ff171613740 1 -- start start 2024-09-10T08:56:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.408+0000 7ff171613740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1719a0 0x55778c171da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.408+0000 7ff171613740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c1722e0 0x55778c176da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.408+0000 7ff171613740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c177370 0x55778c179830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.408+0000 7ff171613740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55778c0d52f0 con 0x55778c1719a0 2024-09-10T08:56:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.408+0000 7ff171613740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55778c07d270 con 0x55778c177370 2024-09-10T08:56:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.408+0000 7ff171613740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55778c0d12b0 con 0x55778c1722e0 2024-09-10T08:56:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.408+0000 7ff1657fa640 1 --2- 172.21.15.172:0/3756277601 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff1500a2c30 0x7ff1500a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.408+0000 7ff16cd35640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1719a0 0x55778c171da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.408+0000 7ff16cd35640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1719a0 0x55778c171da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51414/0 (socket says 172.21.15.172:51414) 2024-09-10T08:56:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.408+0000 7ff16cd35640 1 -- 172.21.15.172:0/1933085186 learned_addr learned my addr 172.21.15.172:0/1933085186 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.408+0000 7ff16cd35640 1 --2- 172.21.15.172:0/3756277601 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff1500a2c30 0x7ff1500a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.409+0000 7ff1657fa640 1 -- 172.21.15.172:0/3756277601 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff15c0efe70 con 0x7ff168101c40 2024-09-10T08:56:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.409+0000 7ff1677fe640 1 --2- 172.21.15.172:0/1933085186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c177370 0x55778c179830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.409+0000 7ff167fff640 1 --2- 172.21.15.172:0/1933085186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c1722e0 0x55778c176da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.409+0000 7ff16cd35640 1 -- 172.21.15.172:0/1933085186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c1722e0 msgr2=0x55778c176da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.409+0000 7ff16cd35640 1 --2- 172.21.15.172:0/1933085186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c1722e0 0x55778c176da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.409+0000 7ff16cd35640 1 -- 172.21.15.172:0/1933085186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c177370 msgr2=0x55778c179830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.409+0000 7ff16cd35640 1 --2- 172.21.15.172:0/1933085186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c177370 0x55778c179830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.409+0000 7ff16cd35640 1 -- 172.21.15.172:0/1933085186 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778c07c340 con 0x55778c1719a0 2024-09-10T08:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.409+0000 7ff1677fe640 1 --2- 172.21.15.172:0/1933085186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c177370 0x55778c179830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.409+0000 7ff16cd35640 1 --2- 172.21.15.172:0/1933085186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1719a0 0x55778c171da0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ff160012d70 tx=0x7ff160010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356886966d16a516 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.409+0000 7ff16cd35640 1 --2- 172.21.15.172:0/3756277601 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff1500a2c30 0x7ff1500a50f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff160010c40 tx=0x7ff160017000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.409+0000 7ff14e7fc640 1 -- 172.21.15.172:0/1933085186 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160023070 con 0x55778c1719a0 2024-09-10T08:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.409+0000 7ff14e7fc640 1 -- 172.21.15.172:0/1933085186 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff160009d20 con 0x55778c1719a0 2024-09-10T08:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.409+0000 7ff14e7fc640 1 -- 172.21.15.172:0/1933085186 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160005280 con 0x55778c1719a0 2024-09-10T08:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.410+0000 7ff171613740 1 -- 172.21.15.172:0/1933085186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1719a0 msgr2=0x55778c171da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.410+0000 7ff171613740 1 --2- 172.21.15.172:0/1933085186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1719a0 0x55778c171da0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ff160012d70 tx=0x7ff160010740 comp rx=0 tx=0).stop 2024-09-10T08:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.410+0000 7ff171613740 1 -- 172.21.15.172:0/1933085186 shutdown_connections 2024-09-10T08:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.410+0000 7ff171613740 1 --2- 172.21.15.172:0/1933085186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c177370 0x55778c179830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.410+0000 7ff171613740 1 --2- 172.21.15.172:0/1933085186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c1722e0 0x55778c176da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.410+0000 7ff171613740 1 --2- 172.21.15.172:0/1933085186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1719a0 0x55778c171da0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.410+0000 7ff171613740 1 -- 172.21.15.172:0/1933085186 >> 172.21.15.172:0/1933085186 conn(0x55778c16d010 msgr2=0x55778c16d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.410+0000 7ff171613740 1 -- 172.21.15.172:0/1933085186 shutdown_connections 2024-09-10T08:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.410+0000 7ff171613740 1 -- 172.21.15.172:0/1933085186 wait complete. 2024-09-10T08:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff171613740 1 -- start start 2024-09-10T08:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff171613740 1 Processor -- start 2024-09-10T08:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff171613740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c1719a0 0x55778c17bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff171613740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1722e0 0x55778c17c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff171613740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c177370 0x55778c1831d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff171613740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55778c17a2e0 con 0x55778c1722e0 2024-09-10T08:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff171613740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55778c05e290 con 0x55778c1719a0 2024-09-10T08:56:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff171613740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55778c17a460 con 0x55778c177370 2024-09-10T08:56:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff171613740 10 client.0 _ll_register_callbacks cb 0x55778c08f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1722e0 0x55778c17c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1722e0 0x55778c17c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51426/0 (socket says 172.21.15.172:51426) 2024-09-10T08:56:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff1677fe640 1 -- 172.21.15.172:0/3143692513 learned_addr learned my addr 172.21.15.172:0/3143692513 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff167fff640 1 --2- 172.21.15.172:0/3143692513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c1719a0 0x55778c17bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff16cd35640 1 --2- 172.21.15.172:0/3143692513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c177370 0x55778c1831d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff1677fe640 1 -- 172.21.15.172:0/3143692513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c177370 msgr2=0x55778c1831d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff1677fe640 1 --2- 172.21.15.172:0/3143692513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c177370 0x55778c1831d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff1677fe640 1 -- 172.21.15.172:0/3143692513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c1719a0 msgr2=0x55778c17bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.411+0000 7ff1677fe640 1 --2- 172.21.15.172:0/3143692513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c1719a0 0x55778c17bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.412+0000 7ff1677fe640 1 -- 172.21.15.172:0/3143692513 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55778bde8210 con 0x55778c1722e0 2024-09-10T08:56:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.412+0000 7ff1677fe640 1 --2- 172.21.15.172:0/3143692513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1722e0 0x55778c17c190 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff15c005940 tx=0x7ff15c0bbeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1379e96303d5c54 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.412+0000 7ff1477fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.412+0000 7ff1477fe640 1 -- 172.21.15.172:0/3143692513 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c049040 con 0x55778c1722e0 2024-09-10T08:56:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.412+0000 7ff1477fe640 1 -- 172.21.15.172:0/3143692513 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff15c03b460 con 0x55778c1722e0 2024-09-10T08:56:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.412+0000 7ff171613740 1 -- 172.21.15.172:0/3143692513 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55778c0e2a20 con 0x55778c1722e0 2024-09-10T08:56:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.412+0000 7ff1477fe640 1 -- 172.21.15.172:0/3143692513 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff15c0bca70 con 0x55778c1722e0 2024-09-10T08:56:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.412+0000 7ff171613740 10 client.18510 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.412+0000 7ff171613740 1 -- 172.21.15.172:0/3143692513 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55778beb4620 con 0x55778c1722e0 2024-09-10T08:56:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.412+0000 7ff1477fe640 1 -- 172.21.15.172:0/3143692513 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55778c0e2a20 con 0x55778c1722e0 2024-09-10T08:56:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.412+0000 7ff1477fe640 1 -- 172.21.15.172:0/3143692513 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7ff15c0d5120 con 0x55778c1722e0 2024-09-10T08:56:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.412+0000 7ff171613740 10 client.18510 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.412+0000 7ff171613740 10 client.18510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.412+0000 7ff171613740 20 client.18510 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.412+0000 7ff171613740 1 --2- 172.21.15.172:0/3143692513 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55778c187760 0x55778c189b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.412+0000 7ff171613740 4 client.18510 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:56:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.412+0000 7ff171613740 1 -- 172.21.15.172:0/3143692513 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55778c0dcd70 con 0x55778c187760 2024-09-10T08:56:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.413+0000 7ff167fff640 1 --2- 172.21.15.172:0/3143692513 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55778c187760 0x55778c189b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.413+0000 7ff167fff640 1 --2- 172.21.15.172:0/3143692513 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55778c187760 0x55778c189b70 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3acd581c13728bb5 server_cookie=85d7ba4814b105a5 in_seq=0 out_seq=0 2024-09-10T08:56:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.413+0000 7ff1477fe640 10 client.18510 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.414+0000 7ff1477fe640 1 -- 172.21.15.172:0/3143692513 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55778c0dcd70 con 0x55778c187760 2024-09-10T08:56:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.414+0000 7ff1477fe640 10 client.18510 handle_command_reply: tid=0 2024-09-10T08:56:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.414+0000 7ff171613740 1 client.18510 shutdown 2024-09-10T08:56:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.414+0000 7ff171613740 1 -- 172.21.15.172:0/3143692513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1722e0 msgr2=0x55778c17c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.414+0000 7ff171613740 1 --2- 172.21.15.172:0/3143692513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1722e0 0x55778c17c190 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff15c005940 tx=0x7ff15c0bbeb0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.414+0000 7ff171613740 1 -- 172.21.15.172:0/3143692513 shutdown_connections 2024-09-10T08:56:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.414+0000 7ff171613740 1 --2- 172.21.15.172:0/3143692513 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55778c187760 0x55778c189b70 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.414+0000 7ff171613740 1 --2- 172.21.15.172:0/3143692513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c177370 0x55778c1831d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.414+0000 7ff171613740 1 --2- 172.21.15.172:0/3143692513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1722e0 0x55778c17c190 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.414+0000 7ff171613740 1 --2- 172.21.15.172:0/3143692513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c1719a0 0x55778c17bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.414+0000 7ff171613740 1 -- 172.21.15.172:0/3143692513 >> 172.21.15.172:0/3143692513 conn(0x55778c16d010 msgr2=0x55778c16e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.414+0000 7ff1477fe640 0 client.18510 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.414+0000 7ff171613740 1 -- 172.21.15.172:0/3143692513 shutdown_connections 2024-09-10T08:56:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.414+0000 7ff171613740 1 -- 172.21.15.172:0/3143692513 wait complete. 2024-09-10T08:56:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.414+0000 7ff171613740 20 client.18510 trim_cache size 0 max 16384 2024-09-10T08:56:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.428+0000 7ff171613740 1 Processor -- start 2024-09-10T08:56:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.428+0000 7ff171613740 1 -- start start 2024-09-10T08:56:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.428+0000 7ff171613740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c1719a0 0x55778c16d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.428+0000 7ff171613740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1722e0 0x55778c183190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.428+0000 7ff171613740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c177370 0x55778c16c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.428+0000 7ff171613740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681aa420 con 0x55778c1722e0 2024-09-10T08:56:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.428+0000 7ff171613740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681abf60 con 0x55778c1719a0 2024-09-10T08:56:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.428+0000 7ff171613740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681aaaf0 con 0x55778c177370 2024-09-10T08:56:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.428+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1722e0 0x55778c183190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.428+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1722e0 0x55778c183190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51432/0 (socket says 172.21.15.172:51432) 2024-09-10T08:56:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.428+0000 7ff1677fe640 1 -- 172.21.15.172:0/311508382 learned_addr learned my addr 172.21.15.172:0/311508382 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.429+0000 7ff16cd35640 1 --2- 172.21.15.172:0/311508382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c177370 0x55778c16c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.429+0000 7ff167fff640 1 --2- 172.21.15.172:0/311508382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c1719a0 0x55778c16d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.429+0000 7ff1677fe640 1 -- 172.21.15.172:0/311508382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c177370 msgr2=0x55778c16c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.429+0000 7ff1677fe640 1 --2- 172.21.15.172:0/311508382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c177370 0x55778c16c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.429+0000 7ff1677fe640 1 -- 172.21.15.172:0/311508382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c1719a0 msgr2=0x55778c16d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.429+0000 7ff1677fe640 1 --2- 172.21.15.172:0/311508382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c1719a0 0x55778c16d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.429+0000 7ff1677fe640 1 -- 172.21.15.172:0/311508382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778c179760 con 0x55778c1722e0 2024-09-10T08:56:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.429+0000 7ff16cd35640 1 --2- 172.21.15.172:0/311508382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c177370 0x55778c16c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.429+0000 7ff1677fe640 1 --2- 172.21.15.172:0/311508382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1722e0 0x55778c183190 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0314e0 tx=0x7ff15c0bbfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd340aeb92cad05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.429+0000 7ff14dffb640 1 -- 172.21.15.172:0/311508382 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0c4fe0 con 0x55778c1722e0 2024-09-10T08:56:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.429+0000 7ff14dffb640 1 -- 172.21.15.172:0/311508382 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff15c0b6e50 con 0x55778c1722e0 2024-09-10T08:56:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.429+0000 7ff14dffb640 1 -- 172.21.15.172:0/311508382 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c03b460 con 0x55778c1722e0 2024-09-10T08:56:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.430+0000 7ff171613740 1 -- 172.21.15.172:0/311508382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1722e0 msgr2=0x55778c183190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.430+0000 7ff171613740 1 --2- 172.21.15.172:0/311508382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1722e0 0x55778c183190 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0314e0 tx=0x7ff15c0bbfe0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.430+0000 7ff171613740 1 -- 172.21.15.172:0/311508382 shutdown_connections 2024-09-10T08:56:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.430+0000 7ff171613740 1 --2- 172.21.15.172:0/311508382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c177370 0x55778c16c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.430+0000 7ff171613740 1 --2- 172.21.15.172:0/311508382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c1722e0 0x55778c183190 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.430+0000 7ff171613740 1 --2- 172.21.15.172:0/311508382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c1719a0 0x55778c16d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.430+0000 7ff171613740 1 -- 172.21.15.172:0/311508382 >> 172.21.15.172:0/311508382 conn(0x55778c16d010 msgr2=0x55778c16b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.430+0000 7ff171613740 1 -- 172.21.15.172:0/311508382 shutdown_connections 2024-09-10T08:56:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.430+0000 7ff171613740 1 -- 172.21.15.172:0/311508382 wait complete. 2024-09-10T08:56:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.430+0000 7ff171613740 1 -- start start 2024-09-10T08:56:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff171613740 1 Processor -- start 2024-09-10T08:56:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff171613740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c1719a0 0x55778c180e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff171613740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c1722e0 0x55778c1813d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff171613740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c177370 0x55778c1823e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff171613740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681ab890 con 0x55778c177370 2024-09-10T08:56:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff171613740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681ab1c0 con 0x55778c1719a0 2024-09-10T08:56:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff171613740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681a8400 con 0x55778c1722e0 2024-09-10T08:56:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff171613740 10 client.0 _ll_register_callbacks cb 0x7ff15c009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff16cd35640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c177370 0x55778c1823e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff16cd35640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c177370 0x55778c1823e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51446/0 (socket says 172.21.15.172:51446) 2024-09-10T08:56:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff16cd35640 1 -- 172.21.15.172:0/1796054064 learned_addr learned my addr 172.21.15.172:0/1796054064 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff1677fe640 1 --2- 172.21.15.172:0/1796054064 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c1722e0 0x55778c1813d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff167fff640 1 --2- 172.21.15.172:0/1796054064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c1719a0 0x55778c180e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff16cd35640 1 -- 172.21.15.172:0/1796054064 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c1722e0 msgr2=0x55778c1813d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff16cd35640 1 --2- 172.21.15.172:0/1796054064 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c1722e0 0x55778c1813d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff16cd35640 1 -- 172.21.15.172:0/1796054064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c1719a0 msgr2=0x55778c180e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff16cd35640 1 --2- 172.21.15.172:0/1796054064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c1719a0 0x55778c180e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff16cd35640 1 -- 172.21.15.172:0/1796054064 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55778c1829f0 con 0x55778c177370 2024-09-10T08:56:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff1677fe640 1 --2- 172.21.15.172:0/1796054064 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c1722e0 0x55778c1813d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.431+0000 7ff167fff640 1 --2- 172.21.15.172:0/1796054064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c1719a0 0x55778c180e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.432+0000 7ff16cd35640 1 --2- 172.21.15.172:0/1796054064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c177370 0x55778c1823e0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7ff160009580 tx=0x7ff16000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75fee1bd8969581 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.432+0000 7ff1477fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.432+0000 7ff1477fe640 1 -- 172.21.15.172:0/1796054064 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160023040 con 0x55778c177370 2024-09-10T08:56:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.432+0000 7ff1477fe640 1 -- 172.21.15.172:0/1796054064 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff160004a00 con 0x55778c177370 2024-09-10T08:56:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.432+0000 7ff171613740 1 -- 172.21.15.172:0/1796054064 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff15c0b6e50 con 0x55778c177370 2024-09-10T08:56:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.432+0000 7ff1477fe640 1 -- 172.21.15.172:0/1796054064 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff160004c00 con 0x55778c177370 2024-09-10T08:56:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.432+0000 7ff171613740 10 client.18516 fetch_fsmap learned FSMap version 37 2024-09-10T08:56:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.432+0000 7ff171613740 1 -- 172.21.15.172:0/1796054064 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55778c1f2ad0 con 0x55778c177370 2024-09-10T08:56:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.432+0000 7ff1477fe640 1 -- 172.21.15.172:0/1796054064 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff15c0b6e50 con 0x55778c177370 2024-09-10T08:56:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.432+0000 7ff1477fe640 1 -- 172.21.15.172:0/1796054064 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7ff16002e070 con 0x55778c177370 2024-09-10T08:56:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.432+0000 7ff171613740 10 client.18516 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T08:56:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.432+0000 7ff171613740 10 client.18516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.432+0000 7ff171613740 20 client.18516 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.432+0000 7ff171613740 1 --2- 172.21.15.172:0/1796054064 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55778c1f3340 0x55778c1f5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.432+0000 7ff171613740 4 client.18516 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:56:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.432+0000 7ff171613740 1 -- 172.21.15.172:0/1796054064 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55778c059900 con 0x55778c1f3340 2024-09-10T08:56:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.433+0000 7ff167fff640 1 --2- 172.21.15.172:0/1796054064 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55778c1f3340 0x55778c1f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.433+0000 7ff1477fe640 10 client.18516 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.433+0000 7ff167fff640 1 --2- 172.21.15.172:0/1796054064 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55778c1f3340 0x55778c1f5760 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57039a0d7ee0d041 server_cookie=8621feddab4159a8 in_seq=0 out_seq=0 2024-09-10T08:56:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.436+0000 7ff1477fe640 1 -- 172.21.15.172:0/1796054064 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55778c059900 con 0x55778c1f3340 2024-09-10T08:56:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.436+0000 7ff1477fe640 10 client.18516 handle_command_reply: tid=0 2024-09-10T08:56:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.436+0000 7ff171613740 1 client.18516 shutdown 2024-09-10T08:56:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.436+0000 7ff171613740 1 -- 172.21.15.172:0/1796054064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c177370 msgr2=0x55778c1823e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.436+0000 7ff171613740 1 --2- 172.21.15.172:0/1796054064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c177370 0x55778c1823e0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7ff160009580 tx=0x7ff16000cb40 comp rx=0 tx=0).stop 2024-09-10T08:56:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.437+0000 7ff171613740 1 -- 172.21.15.172:0/1796054064 shutdown_connections 2024-09-10T08:56:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.437+0000 7ff171613740 1 --2- 172.21.15.172:0/1796054064 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55778c1f3340 0x55778c1f5760 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.437+0000 7ff1477fe640 0 client.18516 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.437+0000 7ff171613740 1 --2- 172.21.15.172:0/1796054064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55778c177370 0x55778c1823e0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.437+0000 7ff171613740 1 --2- 172.21.15.172:0/1796054064 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55778c1722e0 0x55778c1813d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.437+0000 7ff171613740 1 --2- 172.21.15.172:0/1796054064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55778c1719a0 0x55778c180e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.437+0000 7ff171613740 1 -- 172.21.15.172:0/1796054064 >> 172.21.15.172:0/1796054064 conn(0x55778c16d010 msgr2=0x55778c16c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.437+0000 7ff171613740 1 -- 172.21.15.172:0/1796054064 shutdown_connections 2024-09-10T08:56:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.437+0000 7ff171613740 1 -- 172.21.15.172:0/1796054064 wait complete. 2024-09-10T08:56:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.437+0000 7ff171613740 20 client.18516 trim_cache size 0 max 16384 2024-09-10T08:56:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:56:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:56:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:56:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:56:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.437+0000 7ff16e798640 1 -- 172.21.15.172:0/3756277601 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff1500a2c30 msgr2=0x7ff1500a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.437+0000 7ff16e798640 1 --2- 172.21.15.172:0/3756277601 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff1500a2c30 0x7ff1500a50f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff160010c40 tx=0x7ff160017000 comp rx=0 tx=0).stop 2024-09-10T08:56:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.437+0000 7ff16e798640 1 -- 172.21.15.172:0/3756277601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff168101c40 msgr2=0x7ff168198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.437+0000 7ff16e798640 1 --2- 172.21.15.172:0/3756277601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff168101c40 0x7ff168198260 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7ff15c004770 tx=0x7ff15c0042f0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.437+0000 7ff16e798640 1 -- 172.21.15.172:0/3756277601 shutdown_connections 2024-09-10T08:56:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.438+0000 7ff16e798640 1 --2- 172.21.15.172:0/3756277601 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff1500a2c30 0x7ff1500a50f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.438+0000 7ff16e798640 1 --2- 172.21.15.172:0/3756277601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff168102600 0x7ff16819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.438+0000 7ff16e798640 1 --2- 172.21.15.172:0/3756277601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff168101c40 0x7ff168198260 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.438+0000 7ff16e798640 1 --2- 172.21.15.172:0/3756277601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff168100a50 0x7ff168197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.438+0000 7ff16e798640 1 -- 172.21.15.172:0/3756277601 >> 172.21.15.172:0/3756277601 conn(0x7ff1680fc1b0 msgr2=0x7ff1680fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.438+0000 7ff16e798640 1 -- 172.21.15.172:0/3756277601 shutdown_connections 2024-09-10T08:56:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:36.438+0000 7ff16e798640 1 -- 172.21.15.172:0/3756277601 wait complete. 2024-09-10T08:56:36.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:36 smithi174 ceph-mon[34068]: pgmap v661: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 27 op/s 2024-09-10T08:56:36.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:36 smithi191 ceph-mon[35314]: pgmap v661: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 27 op/s 2024-09-10T08:56:36.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:36 smithi172 ceph-mon[30854]: pgmap v661: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 30 MiB/s wr, 27 op/s 2024-09-10T08:56:37.208 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-10T08:56:37.209 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:56:37.500 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:56:38.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.003+0000 7f53a49a3640 1 -- 172.21.15.172:0/4024432467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53a0100a90 msgr2=0x7f53a0100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:38.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.003+0000 7f53a49a3640 1 --2- 172.21.15.172:0/4024432467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53a0100a90 0x7f53a0100ef0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f5394009930 tx=0x7f539402a420 comp rx=0 tx=0).stop 2024-09-10T08:56:38.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.003+0000 7f53a49a3640 1 -- 172.21.15.172:0/4024432467 shutdown_connections 2024-09-10T08:56:38.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.003+0000 7f53a49a3640 1 --2- 172.21.15.172:0/4024432467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53a0101430 0x7f53a010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:38.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.003+0000 7f53a49a3640 1 --2- 172.21.15.172:0/4024432467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53a0100a90 0x7f53a0100ef0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:38.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.003+0000 7f53a49a3640 1 --2- 172.21.15.172:0/4024432467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53a0106a90 0x7f53a0106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:38.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.003+0000 7f53a49a3640 1 -- 172.21.15.172:0/4024432467 >> 172.21.15.172:0/4024432467 conn(0x7f53a00fc7d0 msgr2=0x7f53a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:38.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.003+0000 7f53a49a3640 1 -- 172.21.15.172:0/4024432467 shutdown_connections 2024-09-10T08:56:38.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.003+0000 7f53a49a3640 1 -- 172.21.15.172:0/4024432467 wait complete. 2024-09-10T08:56:38.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.004+0000 7f53a49a3640 1 Processor -- start 2024-09-10T08:56:38.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.004+0000 7f53a49a3640 1 -- start start 2024-09-10T08:56:38.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.004+0000 7f53a49a3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53a0100a90 0x7f53a01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:38.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.004+0000 7f53a49a3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53a0101430 0x7f53a0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:38.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.004+0000 7f53a49a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53a0106a90 0x7f53a019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:38.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.004+0000 7f53a49a3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f53a0110d90 con 0x7f53a0106a90 2024-09-10T08:56:38.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.004+0000 7f53a49a3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f53a0110c10 con 0x7f53a0101430 2024-09-10T08:56:38.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.004+0000 7f53a49a3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f53a0110f10 con 0x7f53a0100a90 2024-09-10T08:56:38.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.004+0000 7f539ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53a0106a90 0x7f53a019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:38.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.005+0000 7f539ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53a0106a90 0x7f53a019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51476/0 (socket says 172.21.15.172:51476) 2024-09-10T08:56:38.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.005+0000 7f539ed9d640 1 -- 172.21.15.172:0/1171683390 learned_addr learned my addr 172.21.15.172:0/1171683390 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:38.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.005+0000 7f539dd9b640 1 --2- 172.21.15.172:0/1171683390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53a0101430 0x7f53a0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:38.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.005+0000 7f539e59c640 1 --2- 172.21.15.172:0/1171683390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53a0100a90 0x7f53a01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:38.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.005+0000 7f539ed9d640 1 -- 172.21.15.172:0/1171683390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53a0100a90 msgr2=0x7f53a01983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:38.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.005+0000 7f539ed9d640 1 --2- 172.21.15.172:0/1171683390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53a0100a90 0x7f53a01983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:38.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.005+0000 7f539ed9d640 1 -- 172.21.15.172:0/1171683390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53a0101430 msgr2=0x7f53a0198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:38.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.005+0000 7f539ed9d640 1 --2- 172.21.15.172:0/1171683390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53a0101430 0x7f53a0198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:38.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.005+0000 7f539ed9d640 1 -- 172.21.15.172:0/1171683390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a019d380 con 0x7f53a0106a90 2024-09-10T08:56:38.012 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.005+0000 7f539ed9d640 1 --2- 172.21.15.172:0/1171683390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53a0106a90 0x7f53a019cca0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f539800cad0 tx=0x7f539800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:38.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.005+0000 7f538f7fe640 1 -- 172.21.15.172:0/1171683390 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539800ae20 con 0x7f53a0106a90 2024-09-10T08:56:38.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.005+0000 7f53a49a3640 1 -- 172.21.15.172:0/1171683390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a019d670 con 0x7f53a0106a90 2024-09-10T08:56:38.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.005+0000 7f53a49a3640 1 -- 172.21.15.172:0/1171683390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a00fcd80 con 0x7f53a0106a90 2024-09-10T08:56:38.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.005+0000 7f538f7fe640 1 -- 172.21.15.172:0/1171683390 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5398005020 con 0x7f53a0106a90 2024-09-10T08:56:38.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.005+0000 7f538f7fe640 1 -- 172.21.15.172:0/1171683390 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539801d060 con 0x7f53a0106a90 2024-09-10T08:56:38.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.007+0000 7f53a49a3640 1 -- 172.21.15.172:0/1171683390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5380005290 con 0x7f53a0106a90 2024-09-10T08:56:38.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.008+0000 7f538f7fe640 1 -- 172.21.15.172:0/1171683390 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5398004a40 con 0x7f53a0106a90 2024-09-10T08:56:38.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.008+0000 7f538f7fe640 1 --2- 172.21.15.172:0/1171683390 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53880a2c00 0x7f53880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:38.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.009+0000 7f539e59c640 1 --2- 172.21.15.172:0/1171683390 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53880a2c00 0x7f53880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:38.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.009+0000 7f538f7fe640 1 -- 172.21.15.172:0/1171683390 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f53980cf310 con 0x7f53a0106a90 2024-09-10T08:56:38.013 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.009+0000 7f539e59c640 1 --2- 172.21.15.172:0/1171683390 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53880a2c00 0x7f53880a50c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f53900058c0 tx=0x7f539000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:38.014 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.013+0000 7f538f7fe640 1 -- 172.21.15.172:0/1171683390 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5398014030 con 0x7f53a0106a90 2024-09-10T08:56:38.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.277+0000 7f53a49a3640 1 -- 172.21.15.172:0/1171683390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5380005580 con 0x7f53a0106a90 2024-09-10T08:56:38.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.278+0000 7f538f7fe640 1 -- 172.21.15.172:0/1171683390 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+12001 (secure 0 0 0) 0x7f539809bf10 con 0x7f53a0106a90 2024-09-10T08:56:38.279 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:56:38.279 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":37,"btime":"2024-09-10T08:52:08:313493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:52:07.658341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:56:38.279 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 37 2024-09-10T08:56:38.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.280+0000 7f53a49a3640 1 -- 172.21.15.172:0/1171683390 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53880a2c00 msgr2=0x7f53880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:38.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.280+0000 7f53a49a3640 1 --2- 172.21.15.172:0/1171683390 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53880a2c00 0x7f53880a50c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f53900058c0 tx=0x7f539000a250 comp rx=0 tx=0).stop 2024-09-10T08:56:38.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.280+0000 7f53a49a3640 1 -- 172.21.15.172:0/1171683390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53a0106a90 msgr2=0x7f53a019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:38.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.280+0000 7f53a49a3640 1 --2- 172.21.15.172:0/1171683390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53a0106a90 0x7f53a019cca0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f539800cad0 tx=0x7f539800cf90 comp rx=0 tx=0).stop 2024-09-10T08:56:38.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.280+0000 7f53a49a3640 1 -- 172.21.15.172:0/1171683390 shutdown_connections 2024-09-10T08:56:38.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.280+0000 7f53a49a3640 1 --2- 172.21.15.172:0/1171683390 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f53880a2c00 0x7f53880a50c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:38.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.280+0000 7f53a49a3640 1 --2- 172.21.15.172:0/1171683390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53a0106a90 0x7f53a019cca0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:38.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.280+0000 7f53a49a3640 1 --2- 172.21.15.172:0/1171683390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53a0101430 0x7f53a0198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:38.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.280+0000 7f53a49a3640 1 --2- 172.21.15.172:0/1171683390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53a0100a90 0x7f53a01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:38.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.280+0000 7f53a49a3640 1 -- 172.21.15.172:0/1171683390 >> 172.21.15.172:0/1171683390 conn(0x7f53a00fc7d0 msgr2=0x7f53a006aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:38.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.281+0000 7f53a49a3640 1 -- 172.21.15.172:0/1171683390 shutdown_connections 2024-09-10T08:56:38.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:38.281+0000 7f53a49a3640 1 -- 172.21.15.172:0/1171683390 wait complete. 2024-09-10T08:56:38.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:38 smithi172 ceph-mon[30854]: pgmap v662: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:56:38.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:38 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1171683390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:56:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:38 smithi174 ceph-mon[34068]: pgmap v662: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:56:38.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:38 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1171683390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:56:38.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:38 smithi191 ceph-mon[35314]: pgmap v662: 129 pgs: 129 active+clean; 9.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:56:38.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:38 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1171683390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:56:39.013 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-10T08:56:39.013 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-10T08:56:39.294 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:56:39.695 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:39 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:56:39.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:39 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:56:39.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:39 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:56:39.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.798+0000 7fe804b4f640 1 -- 172.21.15.172:0/396608151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe800101de0 msgr2=0x7fe8001021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:39.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.798+0000 7fe804b4f640 1 --2- 172.21.15.172:0/396608151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe800101de0 0x7fe8001021c0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009900 tx=0x7fe7f002a410 comp rx=0 tx=0).stop 2024-09-10T08:56:39.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.798+0000 7fe804b4f640 1 -- 172.21.15.172:0/396608151 shutdown_connections 2024-09-10T08:56:39.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.798+0000 7fe804b4f640 1 --2- 172.21.15.172:0/396608151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe800106f10 0x7fe80010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.798+0000 7fe804b4f640 1 --2- 172.21.15.172:0/396608151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe800102790 0x7fe8001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.798+0000 7fe804b4f640 1 --2- 172.21.15.172:0/396608151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe800101de0 0x7fe8001021c0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.798+0000 7fe804b4f640 1 -- 172.21.15.172:0/396608151 >> 172.21.15.172:0/396608151 conn(0x7fe8000fd8c0 msgr2=0x7fe8000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:39.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.798+0000 7fe804b4f640 1 -- 172.21.15.172:0/396608151 shutdown_connections 2024-09-10T08:56:39.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.798+0000 7fe804b4f640 1 -- 172.21.15.172:0/396608151 wait complete. 2024-09-10T08:56:39.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.799+0000 7fe804b4f640 1 Processor -- start 2024-09-10T08:56:39.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.799+0000 7fe804b4f640 1 -- start start 2024-09-10T08:56:39.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.799+0000 7fe804b4f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe800101de0 0x7fe800198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.799+0000 7fe804b4f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe800102790 0x7fe800198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.799+0000 7fe804b4f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe800106f10 0x7fe80019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.799+0000 7fe804b4f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe800111de0 con 0x7fe800102790 2024-09-10T08:56:39.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.799+0000 7fe804b4f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe800111c60 con 0x7fe800106f10 2024-09-10T08:56:39.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.799+0000 7fe804b4f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe800111f60 con 0x7fe800101de0 2024-09-10T08:56:39.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.799+0000 7fe7fdd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe800102790 0x7fe800198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.799+0000 7fe7fdd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe800102790 0x7fe800198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51502/0 (socket says 172.21.15.172:51502) 2024-09-10T08:56:39.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.799+0000 7fe7fdd9b640 1 -- 172.21.15.172:0/1929862391 learned_addr learned my addr 172.21.15.172:0/1929862391 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:39.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.799+0000 7fe7fe59c640 1 --2- 172.21.15.172:0/1929862391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe800101de0 0x7fe800198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.799+0000 7fe7fed9d640 1 --2- 172.21.15.172:0/1929862391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe800106f10 0x7fe80019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.800+0000 7fe7fdd9b640 1 -- 172.21.15.172:0/1929862391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe800101de0 msgr2=0x7fe800198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:39.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.800+0000 7fe7fdd9b640 1 --2- 172.21.15.172:0/1929862391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe800101de0 0x7fe800198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.800+0000 7fe7fdd9b640 1 -- 172.21.15.172:0/1929862391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe800106f10 msgr2=0x7fe80019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:39.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.800+0000 7fe7fdd9b640 1 --2- 172.21.15.172:0/1929862391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe800106f10 0x7fe80019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.800+0000 7fe7fdd9b640 1 -- 172.21.15.172:0/1929862391 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80019d3e0 con 0x7fe800102790 2024-09-10T08:56:39.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.800+0000 7fe7fe59c640 1 --2- 172.21.15.172:0/1929862391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe800101de0 0x7fe800198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:39.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.800+0000 7fe7fed9d640 1 --2- 172.21.15.172:0/1929862391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe800106f10 0x7fe80019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:39.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.800+0000 7fe7fdd9b640 1 --2- 172.21.15.172:0/1929862391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe800102790 0x7fe800198970 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400d950 tx=0x7fe7f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:39.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.800+0000 7fe7ef7fe640 1 -- 172.21.15.172:0/1929862391 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4017030 con 0x7fe800102790 2024-09-10T08:56:39.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.800+0000 7fe7ef7fe640 1 -- 172.21.15.172:0/1929862391 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7f4004020 con 0x7fe800102790 2024-09-10T08:56:39.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.800+0000 7fe804b4f640 1 -- 172.21.15.172:0/1929862391 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80019d6d0 con 0x7fe800102790 2024-09-10T08:56:39.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.800+0000 7fe7ef7fe640 1 -- 172.21.15.172:0/1929862391 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f40047a0 con 0x7fe800102790 2024-09-10T08:56:39.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.800+0000 7fe804b4f640 1 -- 172.21.15.172:0/1929862391 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8001a4fb0 con 0x7fe800102790 2024-09-10T08:56:39.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.803+0000 7fe7ef7fe640 1 -- 172.21.15.172:0/1929862391 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe7f40041d0 con 0x7fe800102790 2024-09-10T08:56:39.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.803+0000 7fe806f28740 1 Processor -- start 2024-09-10T08:56:39.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.803+0000 7fe7ef7fe640 1 --2- 172.21.15.172:0/1929862391 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe7e80a2b80 0x7fe7e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.804+0000 7fe7ef7fe640 1 -- 172.21.15.172:0/1929862391 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe7f40ceee0 con 0x7fe800102790 2024-09-10T08:56:39.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.804+0000 7fe7fed9d640 1 --2- 172.21.15.172:0/1929862391 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe7e80a2b80 0x7fe7e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.804+0000 7fe806f28740 1 -- start start 2024-09-10T08:56:39.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.804+0000 7fe806f28740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17b75e0 0x5601f17b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.804+0000 7fe806f28740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b7f00 0x5601f17c6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.804+0000 7fe806f28740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17c6d50 0x5601f17c9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.804+0000 7fe806f28740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5601f171d2a0 con 0x5601f17b7f00 2024-09-10T08:56:39.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.804+0000 7fe806f28740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5601f16c4b10 con 0x5601f17c6d50 2024-09-10T08:56:39.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.804+0000 7fe806f28740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5601f1719260 con 0x5601f17b75e0 2024-09-10T08:56:39.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.804+0000 7fe7fdd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b7f00 0x5601f17c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.804+0000 7fe7fdd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b7f00 0x5601f17c6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51512/0 (socket says 172.21.15.172:51512) 2024-09-10T08:56:39.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.804+0000 7fe7fdd9b640 1 -- 172.21.15.172:0/3891277234 learned_addr learned my addr 172.21.15.172:0/3891277234 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:39.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.805+0000 7fe7fe59c640 1 --2- 172.21.15.172:0/3891277234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17b75e0 0x5601f17b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.805+0000 7fe7fed9d640 1 --2- 172.21.15.172:0/1929862391 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe7e80a2b80 0x7fe7e80a5040 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80092f0 tx=0x7fe7f8009280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:39.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.805+0000 7fe7fed9d640 1 --2- 172.21.15.172:0/3891277234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17c6d50 0x5601f17c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.805+0000 7fe7fdd9b640 1 -- 172.21.15.172:0/3891277234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17b75e0 msgr2=0x5601f17b79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:39.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.805+0000 7fe7fdd9b640 1 --2- 172.21.15.172:0/3891277234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17b75e0 0x5601f17b79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.805+0000 7fe7fdd9b640 1 -- 172.21.15.172:0/3891277234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17c6d50 msgr2=0x5601f17c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:39.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.805+0000 7fe7fdd9b640 1 --2- 172.21.15.172:0/3891277234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17c6d50 0x5601f17c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.805+0000 7fe7fdd9b640 1 -- 172.21.15.172:0/3891277234 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601f148eb90 con 0x5601f17b7f00 2024-09-10T08:56:39.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.805+0000 7fe7fed9d640 1 --2- 172.21.15.172:0/3891277234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17c6d50 0x5601f17c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:39.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.805+0000 7fe7fdd9b640 1 --2- 172.21.15.172:0/3891277234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b7f00 0x5601f17c6810 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40a3b20 tx=0x7fe7f409ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e7c7012dd76143 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:39.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.805+0000 7fe7ecff9640 1 -- 172.21.15.172:0/3891277234 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f409f790 con 0x5601f17b7f00 2024-09-10T08:56:39.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.805+0000 7fe7ecff9640 1 -- 172.21.15.172:0/3891277234 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7f40a4050 con 0x5601f17b7f00 2024-09-10T08:56:39.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.805+0000 7fe7ecff9640 1 -- 172.21.15.172:0/3891277234 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f409a830 con 0x5601f17b7f00 2024-09-10T08:56:39.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.806+0000 7fe806f28740 1 -- 172.21.15.172:0/3891277234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b7f00 msgr2=0x5601f17c6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:39.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.806+0000 7fe806f28740 1 --2- 172.21.15.172:0/3891277234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b7f00 0x5601f17c6810 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40a3b20 tx=0x7fe7f409ad40 comp rx=0 tx=0).stop 2024-09-10T08:56:39.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.806+0000 7fe806f28740 1 -- 172.21.15.172:0/3891277234 shutdown_connections 2024-09-10T08:56:39.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.806+0000 7fe806f28740 1 --2- 172.21.15.172:0/3891277234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17c6d50 0x5601f17c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.806+0000 7fe806f28740 1 --2- 172.21.15.172:0/3891277234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b7f00 0x5601f17c6810 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.806+0000 7fe806f28740 1 --2- 172.21.15.172:0/3891277234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17b75e0 0x5601f17b79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.806+0000 7fe806f28740 1 -- 172.21.15.172:0/3891277234 >> 172.21.15.172:0/3891277234 conn(0x5601f17b3cf0 msgr2=0x5601f17b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.807+0000 7fe806f28740 1 -- 172.21.15.172:0/3891277234 shutdown_connections 2024-09-10T08:56:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.807+0000 7fe806f28740 1 -- 172.21.15.172:0/3891277234 wait complete. 2024-09-10T08:56:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.807+0000 7fe806f28740 1 -- start start 2024-09-10T08:56:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.807+0000 7fe806f28740 1 Processor -- start 2024-09-10T08:56:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.807+0000 7fe806f28740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b75e0 0x5601f17c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.807+0000 7fe806f28740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17b7f00 0x5601f17c4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.807+0000 7fe806f28740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17c6d50 0x5601f17c4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.807+0000 7fe806f28740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5601f17c9920 con 0x5601f17b75e0 2024-09-10T08:56:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.807+0000 7fe806f28740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5601f16a56e0 con 0x5601f17b7f00 2024-09-10T08:56:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.807+0000 7fe806f28740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5601f17c9aa0 con 0x5601f17c6d50 2024-09-10T08:56:39.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.807+0000 7fe806f28740 10 client.0 _ll_register_callbacks cb 0x5601f16c9a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.808+0000 7fe7fe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b75e0 0x5601f17c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.808+0000 7fe7fe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b75e0 0x5601f17c3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51514/0 (socket says 172.21.15.172:51514) 2024-09-10T08:56:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.808+0000 7fe7fe59c640 1 -- 172.21.15.172:0/1268549703 learned_addr learned my addr 172.21.15.172:0/1268549703 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.808+0000 7fe7fdd9b640 1 --2- 172.21.15.172:0/1268549703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17b7f00 0x5601f17c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.808+0000 7fe7fed9d640 1 --2- 172.21.15.172:0/1268549703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17c6d50 0x5601f17c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.808+0000 7fe7fe59c640 1 -- 172.21.15.172:0/1268549703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17c6d50 msgr2=0x5601f17c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.808+0000 7fe7fe59c640 1 --2- 172.21.15.172:0/1268549703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17c6d50 0x5601f17c4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.808+0000 7fe7fe59c640 1 -- 172.21.15.172:0/1268549703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17b7f00 msgr2=0x5601f17c4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.808+0000 7fe7fe59c640 1 --2- 172.21.15.172:0/1268549703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17b7f00 0x5601f17c4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.808+0000 7fe7fe59c640 1 -- 172.21.15.172:0/1268549703 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601f17d2f30 con 0x5601f17b75e0 2024-09-10T08:56:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.808+0000 7fe7fe59c640 1 --2- 172.21.15.172:0/1268549703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b75e0 0x5601f17c3cf0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009900 tx=0x7fe7f0031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b26dbdddbd5273 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.808+0000 7fe7e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:39.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.808+0000 7fe7e5ffb640 1 -- 172.21.15.172:0/1268549703 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0035040 con 0x5601f17b75e0 2024-09-10T08:56:39.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.808+0000 7fe7e5ffb640 1 -- 172.21.15.172:0/1268549703 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe7f0004240 con 0x5601f17b75e0 2024-09-10T08:56:39.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.808+0000 7fe806f28740 1 -- 172.21.15.172:0/1268549703 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7f40a4050 con 0x5601f17b75e0 2024-09-10T08:56:39.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.809+0000 7fe7e5ffb640 1 -- 172.21.15.172:0/1268549703 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe7f0004440 con 0x5601f17b75e0 2024-09-10T08:56:39.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.809+0000 7fe806f28740 10 client.18534 fetch_fsmap learned FSMap version 38 2024-09-10T08:56:39.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.809+0000 7fe806f28740 1 -- 172.21.15.172:0/1268549703 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5601f17d5880 con 0x5601f17b75e0 2024-09-10T08:56:39.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.809+0000 7fe7e5ffb640 1 -- 172.21.15.172:0/1268549703 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7f40a4050 con 0x5601f17b75e0 2024-09-10T08:56:39.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.809+0000 7fe7e5ffb640 1 -- 172.21.15.172:0/1268549703 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7fe7f0004b50 con 0x5601f17b75e0 2024-09-10T08:56:39.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.809+0000 7fe806f28740 10 client.18534 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T08:56:39.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.809+0000 7fe806f28740 10 client.18534 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:56:39.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.809+0000 7fe806f28740 20 client.18534 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:39.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.809+0000 7fe806f28740 1 --2- 172.21.15.172:0/1268549703 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5601f17d6d40 0x5601f17d9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.809+0000 7fe806f28740 4 client.18534 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:56:39.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.809+0000 7fe806f28740 1 -- 172.21.15.172:0/1268549703 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601f172a9d0 con 0x5601f17d6d40 2024-09-10T08:56:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.810+0000 7fe7fdd9b640 1 --2- 172.21.15.172:0/1268549703 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5601f17d6d40 0x5601f17d9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.810+0000 7fe7e5ffb640 10 client.18534 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.810+0000 7fe7fdd9b640 1 --2- 172.21.15.172:0/1268549703 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5601f17d6d40 0x5601f17d9170 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aa258ea466e1042 server_cookie=c2413197833c340a in_seq=0 out_seq=0 2024-09-10T08:56:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.811+0000 7fe7e5ffb640 1 -- 172.21.15.172:0/1268549703 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5601f172a9d0 con 0x5601f17d6d40 2024-09-10T08:56:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.811+0000 7fe7e5ffb640 10 client.18534 handle_command_reply: tid=0 2024-09-10T08:56:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.811+0000 7fe806f28740 1 client.18534 shutdown 2024-09-10T08:56:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.811+0000 7fe806f28740 1 -- 172.21.15.172:0/1268549703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b75e0 msgr2=0x5601f17c3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.811+0000 7fe806f28740 1 --2- 172.21.15.172:0/1268549703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b75e0 0x5601f17c3cf0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009900 tx=0x7fe7f0031750 comp rx=0 tx=0).stop 2024-09-10T08:56:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.811+0000 7fe806f28740 1 -- 172.21.15.172:0/1268549703 shutdown_connections 2024-09-10T08:56:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.811+0000 7fe806f28740 1 --2- 172.21.15.172:0/1268549703 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5601f17d6d40 0x5601f17d9170 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.811+0000 7fe806f28740 1 --2- 172.21.15.172:0/1268549703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17c6d50 0x5601f17c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.811+0000 7fe806f28740 1 --2- 172.21.15.172:0/1268549703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17b7f00 0x5601f17c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.811+0000 7fe806f28740 1 --2- 172.21.15.172:0/1268549703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b75e0 0x5601f17c3cf0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.811+0000 7fe806f28740 1 -- 172.21.15.172:0/1268549703 >> 172.21.15.172:0/1268549703 conn(0x5601f17b3cf0 msgr2=0x5601f17b99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.811+0000 7fe7e5ffb640 0 client.18534 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.811+0000 7fe806f28740 1 -- 172.21.15.172:0/1268549703 shutdown_connections 2024-09-10T08:56:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.811+0000 7fe806f28740 1 -- 172.21.15.172:0/1268549703 wait complete. 2024-09-10T08:56:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.811+0000 7fe806f28740 20 client.18534 trim_cache size 0 max 16384 2024-09-10T08:56:39.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.831+0000 7fe806f28740 1 Processor -- start 2024-09-10T08:56:39.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.831+0000 7fe806f28740 1 -- start start 2024-09-10T08:56:39.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.831+0000 7fe806f28740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17b75e0 0x5601f17dea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.831+0000 7fe806f28740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17b7f00 0x5601f17def80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.831+0000 7fe806f28740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17c6d50 0x5601f17c0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe806f28740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001ac980 con 0x5601f17c6d50 2024-09-10T08:56:39.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe806f28740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001ae4f0 con 0x5601f17b7f00 2024-09-10T08:56:39.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe806f28740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001ad050 con 0x5601f17b75e0 2024-09-10T08:56:39.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe7fed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17c6d50 0x5601f17c0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe7fed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17c6d50 0x5601f17c0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51522/0 (socket says 172.21.15.172:51522) 2024-09-10T08:56:39.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe7fed9d640 1 -- 172.21.15.172:0/878631312 learned_addr learned my addr 172.21.15.172:0/878631312 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:39.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe7fdd9b640 1 --2- 172.21.15.172:0/878631312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17b7f00 0x5601f17def80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe7fe59c640 1 --2- 172.21.15.172:0/878631312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17b75e0 0x5601f17dea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe7fed9d640 1 -- 172.21.15.172:0/878631312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17b75e0 msgr2=0x5601f17dea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:39.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe7fed9d640 1 --2- 172.21.15.172:0/878631312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17b75e0 0x5601f17dea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe7fed9d640 1 -- 172.21.15.172:0/878631312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17b7f00 msgr2=0x5601f17def80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:39.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe7fed9d640 1 --2- 172.21.15.172:0/878631312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17b7f00 0x5601f17def80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe7fed9d640 1 -- 172.21.15.172:0/878631312 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601f17af260 con 0x5601f17c6d50 2024-09-10T08:56:39.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe7fdd9b640 1 --2- 172.21.15.172:0/878631312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17b7f00 0x5601f17def80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:39.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe7fed9d640 1 --2- 172.21.15.172:0/878631312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17c6d50 0x5601f17c0f50 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8012720 tx=0x7fe7f8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4cff846103fc9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:39.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe7e7fff640 1 -- 172.21.15.172:0/878631312 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8011820 con 0x5601f17c6d50 2024-09-10T08:56:39.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe7e7fff640 1 -- 172.21.15.172:0/878631312 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7f8025070 con 0x5601f17c6d50 2024-09-10T08:56:39.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.832+0000 7fe7e7fff640 1 -- 172.21.15.172:0/878631312 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f802ade0 con 0x5601f17c6d50 2024-09-10T08:56:39.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.833+0000 7fe806f28740 1 -- 172.21.15.172:0/878631312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17c6d50 msgr2=0x5601f17c0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:39.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.833+0000 7fe806f28740 1 --2- 172.21.15.172:0/878631312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17c6d50 0x5601f17c0f50 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8012720 tx=0x7fe7f8013000 comp rx=0 tx=0).stop 2024-09-10T08:56:39.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.833+0000 7fe806f28740 1 -- 172.21.15.172:0/878631312 shutdown_connections 2024-09-10T08:56:39.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.833+0000 7fe806f28740 1 --2- 172.21.15.172:0/878631312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17c6d50 0x5601f17c0f50 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.833+0000 7fe806f28740 1 --2- 172.21.15.172:0/878631312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17b7f00 0x5601f17def80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.833+0000 7fe806f28740 1 --2- 172.21.15.172:0/878631312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17b75e0 0x5601f17dea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.833+0000 7fe806f28740 1 -- 172.21.15.172:0/878631312 >> 172.21.15.172:0/878631312 conn(0x5601f17b3cf0 msgr2=0x5601f17b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:39.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.833+0000 7fe806f28740 1 -- 172.21.15.172:0/878631312 shutdown_connections 2024-09-10T08:56:39.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.833+0000 7fe806f28740 1 -- 172.21.15.172:0/878631312 wait complete. 2024-09-10T08:56:39.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.834+0000 7fe806f28740 1 -- start start 2024-09-10T08:56:39.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.834+0000 7fe806f28740 1 Processor -- start 2024-09-10T08:56:39.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.834+0000 7fe806f28740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17b75e0 0x5601f17cae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.834+0000 7fe806f28740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b7f00 0x5601f17cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.834+0000 7fe806f28740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17c6d50 0x5601f1840a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.834+0000 7fe806f28740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001ade20 con 0x5601f17b7f00 2024-09-10T08:56:39.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.834+0000 7fe806f28740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001ad750 con 0x5601f17b75e0 2024-09-10T08:56:39.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.834+0000 7fe806f28740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001aa960 con 0x5601f17c6d50 2024-09-10T08:56:39.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.834+0000 7fe806f28740 10 client.0 _ll_register_callbacks cb 0x7fe7f0005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:39.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.834+0000 7fe7fdd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b7f00 0x5601f17cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.834+0000 7fe7fdd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b7f00 0x5601f17cb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51534/0 (socket says 172.21.15.172:51534) 2024-09-10T08:56:39.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.834+0000 7fe7fdd9b640 1 -- 172.21.15.172:0/2706527195 learned_addr learned my addr 172.21.15.172:0/2706527195 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:39.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.835+0000 7fe7fed9d640 1 --2- 172.21.15.172:0/2706527195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17c6d50 0x5601f1840a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.835+0000 7fe7fe59c640 1 --2- 172.21.15.172:0/2706527195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17b75e0 0x5601f17cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.835+0000 7fe7fdd9b640 1 -- 172.21.15.172:0/2706527195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17c6d50 msgr2=0x5601f1840a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:39.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.835+0000 7fe7fdd9b640 1 --2- 172.21.15.172:0/2706527195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17c6d50 0x5601f1840a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.835+0000 7fe7fdd9b640 1 -- 172.21.15.172:0/2706527195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17b75e0 msgr2=0x5601f17cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:39.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.835+0000 7fe7fdd9b640 1 --2- 172.21.15.172:0/2706527195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17b75e0 0x5601f17cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.835+0000 7fe7fdd9b640 1 -- 172.21.15.172:0/2706527195 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601f1840f50 con 0x5601f17b7f00 2024-09-10T08:56:39.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.835+0000 7fe7fed9d640 1 --2- 172.21.15.172:0/2706527195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17c6d50 0x5601f1840a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:39.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.835+0000 7fe7fe59c640 1 --2- 172.21.15.172:0/2706527195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17b75e0 0x5601f17cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:39.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.835+0000 7fe7fdd9b640 1 --2- 172.21.15.172:0/2706527195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b7f00 0x5601f17cb3d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009610 tx=0x7fe7f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de4862a94b83057 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:39.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.835+0000 7fe7e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:39.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.835+0000 7fe7e5ffb640 1 -- 172.21.15.172:0/2706527195 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4099500 con 0x5601f17b7f00 2024-09-10T08:56:39.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.835+0000 7fe7e5ffb640 1 -- 172.21.15.172:0/2706527195 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe7f40b4240 con 0x5601f17b7f00 2024-09-10T08:56:39.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.835+0000 7fe806f28740 1 -- 172.21.15.172:0/2706527195 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7f8025070 con 0x5601f17b7f00 2024-09-10T08:56:39.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.835+0000 7fe7e5ffb640 1 -- 172.21.15.172:0/2706527195 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe7f40a4050 con 0x5601f17b7f00 2024-09-10T08:56:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.835+0000 7fe806f28740 10 client.18540 fetch_fsmap learned FSMap version 38 2024-09-10T08:56:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.835+0000 7fe806f28740 1 -- 172.21.15.172:0/2706527195 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5601f1842110 con 0x5601f17b7f00 2024-09-10T08:56:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.836+0000 7fe7e5ffb640 1 -- 172.21.15.172:0/2706527195 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7f8025070 con 0x5601f17b7f00 2024-09-10T08:56:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.836+0000 7fe7e5ffb640 1 -- 172.21.15.172:0/2706527195 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7fe7f409a830 con 0x5601f17b7f00 2024-09-10T08:56:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.836+0000 7fe806f28740 10 client.18540 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T08:56:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.836+0000 7fe806f28740 10 client.18540 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:56:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.836+0000 7fe806f28740 20 client.18540 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.836+0000 7fe806f28740 1 --2- 172.21.15.172:0/2706527195 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5601f1842510 0x5601f1844900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.836+0000 7fe806f28740 4 client.18540 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-10T08:56:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.836+0000 7fe806f28740 1 -- 172.21.15.172:0/2706527195 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x5601f1724d20 con 0x5601f1842510 2024-09-10T08:56:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.836+0000 7fe7fe59c640 1 --2- 172.21.15.172:0/2706527195 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5601f1842510 0x5601f1844900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.836+0000 7fe7fe59c640 1 --2- 172.21.15.172:0/2706527195 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5601f1842510 0x5601f1844900 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1256ea8dc726cb83 server_cookie=7804a6c512b0c74c in_seq=0 out_seq=0 2024-09-10T08:56:39.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.836+0000 7fe7e5ffb640 10 client.18540 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:39.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.933+0000 7fe7e5ffb640 1 -- 172.21.15.172:0/2706527195 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5601f1724d20 con 0x5601f1842510 2024-09-10T08:56:39.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.933+0000 7fe7e5ffb640 10 client.18540 handle_command_reply: tid=0 2024-09-10T08:56:39.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.933+0000 7fe806f28740 1 client.18540 shutdown 2024-09-10T08:56:39.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.933+0000 7fe806f28740 1 -- 172.21.15.172:0/2706527195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b7f00 msgr2=0x5601f17cb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:39.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.933+0000 7fe806f28740 1 --2- 172.21.15.172:0/2706527195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b7f00 0x5601f17cb3d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009610 tx=0x7fe7f4004730 comp rx=0 tx=0).stop 2024-09-10T08:56:39.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe806f28740 1 -- 172.21.15.172:0/2706527195 shutdown_connections 2024-09-10T08:56:39.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe806f28740 1 --2- 172.21.15.172:0/2706527195 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5601f1842510 0x5601f1844900 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe806f28740 1 --2- 172.21.15.172:0/2706527195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5601f17c6d50 0x5601f1840a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe7e5ffb640 0 client.18540 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:39.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe806f28740 1 --2- 172.21.15.172:0/2706527195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601f17b7f00 0x5601f17cb3d0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe806f28740 1 --2- 172.21.15.172:0/2706527195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5601f17b75e0 0x5601f17cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe806f28740 1 -- 172.21.15.172:0/2706527195 >> 172.21.15.172:0/2706527195 conn(0x5601f17b3cf0 msgr2=0x5601f17b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:39.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe806f28740 1 -- 172.21.15.172:0/2706527195 shutdown_connections 2024-09-10T08:56:39.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe806f28740 1 -- 172.21.15.172:0/2706527195 wait complete. 2024-09-10T08:56:39.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe806f28740 20 client.18540 trim_cache size 0 max 16384 2024-09-10T08:56:39.936 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:56:39.936 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 38, 2024-09-10T08:56:39.936 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:56:39.936 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 34, 2024-09-10T08:56:39.936 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:56:39.937 INFO:teuthology.orchestra.run.smithi172.stdout: "46fa34bb": { 2024-09-10T08:56:39.937 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 34, 2024-09-10T08:56:39.937 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:56:39.937 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:56:39.937 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:56:39.937 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:56:39.937 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:56:39.937 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:56:39.937 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 130.0, 2024-09-10T08:56:39.937 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:56:39.937 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:56:39.937 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:56:39.937 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:56:39.937 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:56:39.938 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:56:39.938 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:56:39.938 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:56:39.938 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:56:39.938 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:56:39.938 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:56:39.938 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:56:39.938 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe804b4f640 1 -- 172.21.15.172:0/1929862391 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe7e80a2b80 msgr2=0x7fe7e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:39.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe804b4f640 1 --2- 172.21.15.172:0/1929862391 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe7e80a2b80 0x7fe7e80a5040 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80092f0 tx=0x7fe7f8009280 comp rx=0 tx=0).stop 2024-09-10T08:56:39.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe804b4f640 1 -- 172.21.15.172:0/1929862391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe800102790 msgr2=0x7fe800198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:39.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe804b4f640 1 --2- 172.21.15.172:0/1929862391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe800102790 0x7fe800198970 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400d950 tx=0x7fe7f400de10 comp rx=0 tx=0).stop 2024-09-10T08:56:39.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe804b4f640 1 -- 172.21.15.172:0/1929862391 shutdown_connections 2024-09-10T08:56:39.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe804b4f640 1 --2- 172.21.15.172:0/1929862391 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe7e80a2b80 0x7fe7e80a5040 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe804b4f640 1 --2- 172.21.15.172:0/1929862391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe800106f10 0x7fe80019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe804b4f640 1 --2- 172.21.15.172:0/1929862391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe800102790 0x7fe800198970 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe804b4f640 1 --2- 172.21.15.172:0/1929862391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe800101de0 0x7fe800198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:39.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.934+0000 7fe804b4f640 1 -- 172.21.15.172:0/1929862391 >> 172.21.15.172:0/1929862391 conn(0x7fe8000fd8c0 msgr2=0x7fe8000fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:39.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.935+0000 7fe804b4f640 1 -- 172.21.15.172:0/1929862391 shutdown_connections 2024-09-10T08:56:39.939 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:39.935+0000 7fe804b4f640 1 -- 172.21.15.172:0/1929862391 wait complete. 2024-09-10T08:56:40.685 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:40 smithi172 ceph-mon[30854]: pgmap v663: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 24 op/s 2024-09-10T08:56:40.717 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '46fa34bb', quiesce took 3.5 seconds. Will release after: 6.5 2024-09-10T08:56:40.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:40 smithi174 ceph-mon[34068]: pgmap v663: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 24 op/s 2024-09-10T08:56:40.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:40 smithi191 ceph-mon[35314]: pgmap v663: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 24 op/s 2024-09-10T08:56:42.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:42 smithi174 ceph-mon[34068]: pgmap v664: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-09-10T08:56:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:42 smithi191 ceph-mon[35314]: pgmap v664: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-09-10T08:56:42.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:42 smithi172 ceph-mon[30854]: pgmap v664: 129 pgs: 129 active+clean; 10 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 29 MiB/s wr, 25 op/s 2024-09-10T08:56:44.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:44 smithi174 ceph-mon[34068]: pgmap v665: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:56:44.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:44 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:56:44.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:44 smithi191 ceph-mon[35314]: pgmap v665: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:56:44.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:44 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:56:44.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:44 smithi172 ceph-mon[30854]: pgmap v665: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:56:44.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:44 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:56:46.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:46 smithi174 ceph-mon[34068]: pgmap v666: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-09-10T08:56:46.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:46 smithi191 ceph-mon[35314]: pgmap v666: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-09-10T08:56:46.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:46 smithi172 ceph-mon[30854]: pgmap v666: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 24 op/s 2024-09-10T08:56:47.218 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:56:47.481 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:56:48.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.025+0000 7ff0d805b640 1 -- 172.21.15.172:0/1676940687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0d0106f10 msgr2=0x7ff0d010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:48.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.025+0000 7ff0d805b640 1 --2- 172.21.15.172:0/1676940687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0d0106f10 0x7ff0d010ea90 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009ca0 tx=0x7ff0c802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:56:48.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.025+0000 7ff0d805b640 1 -- 172.21.15.172:0/1676940687 shutdown_connections 2024-09-10T08:56:48.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.025+0000 7ff0d805b640 1 --2- 172.21.15.172:0/1676940687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0d0106f10 0x7ff0d010ea90 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:48.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.025+0000 7ff0d805b640 1 --2- 172.21.15.172:0/1676940687 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0d0102790 0x7ff0d01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:48.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.025+0000 7ff0d805b640 1 --2- 172.21.15.172:0/1676940687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0d0101de0 0x7ff0d01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:48.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.025+0000 7ff0d805b640 1 -- 172.21.15.172:0/1676940687 >> 172.21.15.172:0/1676940687 conn(0x7ff0d00fd880 msgr2=0x7ff0d00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:48.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.025+0000 7ff0d805b640 1 -- 172.21.15.172:0/1676940687 shutdown_connections 2024-09-10T08:56:48.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.025+0000 7ff0d805b640 1 -- 172.21.15.172:0/1676940687 wait complete. 2024-09-10T08:56:48.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.025+0000 7ff0d805b640 1 Processor -- start 2024-09-10T08:56:48.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d805b640 1 -- start start 2024-09-10T08:56:48.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d805b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0d0101de0 0x7ff0d01981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:48.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d805b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0d0102790 0x7ff0d0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:48.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d805b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0d0106f10 0x7ff0d019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:48.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d805b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0111e20 con 0x7ff0d0106f10 2024-09-10T08:56:48.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d805b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0111ca0 con 0x7ff0d0101de0 2024-09-10T08:56:48.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d805b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0111fa0 con 0x7ff0d0102790 2024-09-10T08:56:48.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d65f8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0d0106f10 0x7ff0d019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:48.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d65f8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0d0106f10 0x7ff0d019cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41498/0 (socket says 172.21.15.172:41498) 2024-09-10T08:56:48.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d65f8640 1 -- 172.21.15.172:0/3894156901 learned_addr learned my addr 172.21.15.172:0/3894156901 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:48.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d5df7640 1 --2- 172.21.15.172:0/3894156901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0d0101de0 0x7ff0d01981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:48.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d65f8640 1 -- 172.21.15.172:0/3894156901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0d0102790 msgr2=0x7ff0d0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:48.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d55f6640 1 --2- 172.21.15.172:0/3894156901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0d0102790 0x7ff0d0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:48.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d65f8640 1 --2- 172.21.15.172:0/3894156901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0d0102790 0x7ff0d0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:48.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d65f8640 1 -- 172.21.15.172:0/3894156901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0d0101de0 msgr2=0x7ff0d01981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:48.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d65f8640 1 --2- 172.21.15.172:0/3894156901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0d0101de0 0x7ff0d01981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:48.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d65f8640 1 -- 172.21.15.172:0/3894156901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d019d190 con 0x7ff0d0106f10 2024-09-10T08:56:48.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d5df7640 1 --2- 172.21.15.172:0/3894156901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0d0101de0 0x7ff0d01981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:48.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d55f6640 1 --2- 172.21.15.172:0/3894156901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0d0102790 0x7ff0d0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:56:48.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0d65f8640 1 --2- 172.21.15.172:0/3894156901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0d0106f10 0x7ff0d019cab0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009dc0 tx=0x7ff0c8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:48.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.026+0000 7ff0c6ffd640 1 -- 172.21.15.172:0/3894156901 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8004040 con 0x7ff0d0106f10 2024-09-10T08:56:48.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.027+0000 7ff0d805b640 1 -- 172.21.15.172:0/3894156901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d019d420 con 0x7ff0d0106f10 2024-09-10T08:56:48.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.027+0000 7ff0c6ffd640 1 -- 172.21.15.172:0/3894156901 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0c8004690 con 0x7ff0d0106f10 2024-09-10T08:56:48.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.027+0000 7ff0c6ffd640 1 -- 172.21.15.172:0/3894156901 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8030d90 con 0x7ff0d0106f10 2024-09-10T08:56:48.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.027+0000 7ff0d805b640 1 -- 172.21.15.172:0/3894156901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d01a4d60 con 0x7ff0d0106f10 2024-09-10T08:56:48.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.028+0000 7ff0d805b640 1 -- 172.21.15.172:0/3894156901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d01021c0 con 0x7ff0d0106f10 2024-09-10T08:56:48.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.029+0000 7ff0c6ffd640 1 -- 172.21.15.172:0/3894156901 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff0c80305c0 con 0x7ff0d0106f10 2024-09-10T08:56:48.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.029+0000 7ff0c6ffd640 1 --2- 172.21.15.172:0/3894156901 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff0bc0a2b80 0x7ff0bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:48.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.030+0000 7ff0d5df7640 1 --2- 172.21.15.172:0/3894156901 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff0bc0a2b80 0x7ff0bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:48.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.030+0000 7ff0c6ffd640 1 -- 172.21.15.172:0/3894156901 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff0c80f0c70 con 0x7ff0d0106f10 2024-09-10T08:56:48.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.030+0000 7ff0d5df7640 1 --2- 172.21.15.172:0/3894156901 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff0bc0a2b80 0x7ff0bc0a5040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00abe0 tx=0x7ff0cc00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:48.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.034+0000 7ff0c6ffd640 1 -- 172.21.15.172:0/3894156901 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0c8039e20 con 0x7ff0d0106f10 2024-09-10T08:56:48.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.235+0000 7ff0d805b640 1 -- 172.21.15.172:0/3894156901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0d010a120 con 0x7ff0d0106f10 2024-09-10T08:56:48.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.236+0000 7ff0c6ffd640 1 -- 172.21.15.172:0/3894156901 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+12004 (secure 0 0 0) 0x7ff0c80bd870 con 0x7ff0d0106f10 2024-09-10T08:56:48.237 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:56:48.237 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":39,"btime":"2024-09-10T08:56:43:360461+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:56:42.665159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:56:48.238 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 39 2024-09-10T08:56:48.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.238+0000 7ff0d805b640 1 -- 172.21.15.172:0/3894156901 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff0bc0a2b80 msgr2=0x7ff0bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:48.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.238+0000 7ff0d805b640 1 --2- 172.21.15.172:0/3894156901 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff0bc0a2b80 0x7ff0bc0a5040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00abe0 tx=0x7ff0cc00a5c0 comp rx=0 tx=0).stop 2024-09-10T08:56:48.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.238+0000 7ff0d805b640 1 -- 172.21.15.172:0/3894156901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0d0106f10 msgr2=0x7ff0d019cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:48.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.238+0000 7ff0d805b640 1 --2- 172.21.15.172:0/3894156901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0d0106f10 0x7ff0d019cab0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009dc0 tx=0x7ff0c8003ba0 comp rx=0 tx=0).stop 2024-09-10T08:56:48.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.238+0000 7ff0d805b640 1 -- 172.21.15.172:0/3894156901 shutdown_connections 2024-09-10T08:56:48.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.238+0000 7ff0d805b640 1 --2- 172.21.15.172:0/3894156901 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff0bc0a2b80 0x7ff0bc0a5040 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:48.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.238+0000 7ff0d805b640 1 --2- 172.21.15.172:0/3894156901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0d0106f10 0x7ff0d019cab0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:48.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.238+0000 7ff0d805b640 1 --2- 172.21.15.172:0/3894156901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0d0102790 0x7ff0d0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:48.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.238+0000 7ff0d805b640 1 --2- 172.21.15.172:0/3894156901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0d0101de0 0x7ff0d01981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:48.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.238+0000 7ff0d805b640 1 -- 172.21.15.172:0/3894156901 >> 172.21.15.172:0/3894156901 conn(0x7ff0d00fd880 msgr2=0x7ff0d00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:48.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.238+0000 7ff0d805b640 1 -- 172.21.15.172:0/3894156901 shutdown_connections 2024-09-10T08:56:48.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:48.239+0000 7ff0d805b640 1 -- 172.21.15.172:0/3894156901 wait complete. 2024-09-10T08:56:48.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:48 smithi172 ceph-mon[30854]: pgmap v667: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 19 op/s 2024-09-10T08:56:48.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:48 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3894156901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:56:48.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:48 smithi174 ceph-mon[34068]: pgmap v667: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 19 op/s 2024-09-10T08:56:48.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:48 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3894156901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:56:48.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:48 smithi191 ceph-mon[35314]: pgmap v667: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 17 MiB/s wr, 19 op/s 2024-09-10T08:56:48.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:48 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3894156901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:56:49.082 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 46fa34bb --release --await' 2024-09-10T08:56:49.083 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 46fa34bb --release --await 2024-09-10T08:56:49.331 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:56:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.526+0000 7f0e89de4640 1 -- 172.21.15.172:0/2067139282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e84102d50 msgr2=0x7f0e841031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.526+0000 7f0e89de4640 1 --2- 172.21.15.172:0/2067139282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e84102d50 0x7f0e841031d0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009900 tx=0x7f0e7402a410 comp rx=0 tx=0).stop 2024-09-10T08:56:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.526+0000 7f0e89de4640 1 -- 172.21.15.172:0/2067139282 shutdown_connections 2024-09-10T08:56:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.526+0000 7f0e89de4640 1 --2- 172.21.15.172:0/2067139282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e84103710 0x7f0e84109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.526+0000 7f0e89de4640 1 --2- 172.21.15.172:0/2067139282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e84102d50 0x7f0e841031d0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.526+0000 7f0e89de4640 1 --2- 172.21.15.172:0/2067139282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e84101b60 0x7f0e84101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.526+0000 7f0e89de4640 1 -- 172.21.15.172:0/2067139282 >> 172.21.15.172:0/2067139282 conn(0x7f0e840fd2e0 msgr2=0x7f0e840ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.526+0000 7f0e89de4640 1 -- 172.21.15.172:0/2067139282 shutdown_connections 2024-09-10T08:56:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.526+0000 7f0e89de4640 1 -- 172.21.15.172:0/2067139282 wait complete. 2024-09-10T08:56:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e89de4640 1 Processor -- start 2024-09-10T08:56:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e89de4640 1 -- start start 2024-09-10T08:56:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e89de4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e84101b60 0x7f0e84197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e89de4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e84102d50 0x7f0e84198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e89de4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e84103710 0x7f0e8419f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e89de4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8410d670 con 0x7f0e84101b60 2024-09-10T08:56:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e89de4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8410d4f0 con 0x7f0e84102d50 2024-09-10T08:56:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e89de4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8410d7f0 con 0x7f0e84103710 2024-09-10T08:56:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e837fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e84101b60 0x7f0e84197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e837fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e84101b60 0x7f0e84197d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41514/0 (socket says 172.21.15.172:41514) 2024-09-10T08:56:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e837fe640 1 -- 172.21.15.172:0/3573386420 learned_addr learned my addr 172.21.15.172:0/3573386420 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e837fe640 1 -- 172.21.15.172:0/3573386420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e84103710 msgr2=0x7f0e8419f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e837fe640 1 --2- 172.21.15.172:0/3573386420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e84103710 0x7f0e8419f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e837fe640 1 -- 172.21.15.172:0/3573386420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e84102d50 msgr2=0x7f0e84198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e82ffd640 1 --2- 172.21.15.172:0/3573386420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e84102d50 0x7f0e84198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e837fe640 1 --2- 172.21.15.172:0/3573386420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e84102d50 0x7f0e84198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e837fe640 1 -- 172.21.15.172:0/3573386420 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8419f920 con 0x7f0e84101b60 2024-09-10T08:56:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.527+0000 7f0e837fe640 1 --2- 172.21.15.172:0/3573386420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e84101b60 0x7f0e84197d10 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c970 tx=0x7f0e7800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.528+0000 7f0e80ff9640 1 -- 172.21.15.172:0/3573386420 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78004220 con 0x7f0e84101b60 2024-09-10T08:56:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.528+0000 7f0e80ff9640 1 -- 172.21.15.172:0/3573386420 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e780043c0 con 0x7f0e84101b60 2024-09-10T08:56:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.528+0000 7f0e89de4640 1 -- 172.21.15.172:0/3573386420 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8419fc10 con 0x7f0e84101b60 2024-09-10T08:56:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.528+0000 7f0e80ff9640 1 -- 172.21.15.172:0/3573386420 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7800f450 con 0x7f0e84101b60 2024-09-10T08:56:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.528+0000 7f0e89de4640 1 -- 172.21.15.172:0/3573386420 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e841a0160 con 0x7f0e84101b60 2024-09-10T08:56:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.530+0000 7f0e80ff9640 1 -- 172.21.15.172:0/3573386420 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0e780079b0 con 0x7f0e84101b60 2024-09-10T08:56:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.530+0000 7f0e8cc58740 1 Processor -- start 2024-09-10T08:56:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.530+0000 7f0e8cc58740 1 -- start start 2024-09-10T08:56:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.530+0000 7f0e8cc58740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f9009a0 0x55927f900da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.530+0000 7f0e8cc58740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f905da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.530+0000 7f0e8cc58740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f906370 0x55927f908830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.530+0000 7f0e8cc58740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55927f8642f0 con 0x55927f9012e0 2024-09-10T08:56:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.530+0000 7f0e8cc58740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55927f80c270 con 0x55927f9009a0 2024-09-10T08:56:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.530+0000 7f0e8cc58740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55927f8602b0 con 0x55927f906370 2024-09-10T08:56:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e837fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f905da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e837fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f905da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41526/0 (socket says 172.21.15.172:41526) 2024-09-10T08:56:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e837fe640 1 -- 172.21.15.172:0/428965133 learned_addr learned my addr 172.21.15.172:0/428965133 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e80ff9640 1 --2- 172.21.15.172:0/3573386420 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0e6c0a2c30 0x7f0e6c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e83fff640 1 --2- 172.21.15.172:0/428965133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f9009a0 0x55927f900da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e82ffd640 1 --2- 172.21.15.172:0/428965133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f906370 0x55927f908830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e837fe640 1 -- 172.21.15.172:0/428965133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f906370 msgr2=0x55927f908830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e837fe640 1 --2- 172.21.15.172:0/428965133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f906370 0x55927f908830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e837fe640 1 -- 172.21.15.172:0/428965133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f9009a0 msgr2=0x55927f900da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e837fe640 1 --2- 172.21.15.172:0/428965133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f9009a0 0x55927f900da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e837fe640 1 -- 172.21.15.172:0/428965133 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55927f80b340 con 0x55927f9012e0 2024-09-10T08:56:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e83fff640 1 --2- 172.21.15.172:0/428965133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f9009a0 0x55927f900da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e83fff640 1 --2- 172.21.15.172:0/3573386420 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0e6c0a2c30 0x7f0e6c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e80ff9640 1 -- 172.21.15.172:0/3573386420 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0e780cf2a0 con 0x7f0e84101b60 2024-09-10T08:56:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e837fe640 1 --2- 172.21.15.172:0/428965133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f905da0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f0e780980e0 tx=0x7f0e7805b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d05237664a43ff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e83fff640 1 --2- 172.21.15.172:0/3573386420 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0e6c0a2c30 0x7f0e6c0a50f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c007d80 tx=0x7f0e7c007440 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e69ffb640 1 -- 172.21.15.172:0/428965133 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7809b100 con 0x55927f9012e0 2024-09-10T08:56:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e69ffb640 1 -- 172.21.15.172:0/428965133 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e7809bea0 con 0x55927f9012e0 2024-09-10T08:56:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.531+0000 7f0e69ffb640 1 -- 172.21.15.172:0/428965133 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e780998e0 con 0x55927f9012e0 2024-09-10T08:56:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.532+0000 7f0e8cc58740 1 -- 172.21.15.172:0/428965133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 msgr2=0x55927f905da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.532+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/428965133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f905da0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f0e780980e0 tx=0x7f0e7805b780 comp rx=0 tx=0).stop 2024-09-10T08:56:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.532+0000 7f0e8cc58740 1 -- 172.21.15.172:0/428965133 shutdown_connections 2024-09-10T08:56:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.532+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/428965133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f906370 0x55927f908830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.532+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/428965133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f905da0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.532+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/428965133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f9009a0 0x55927f900da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.532+0000 7f0e8cc58740 1 -- 172.21.15.172:0/428965133 >> 172.21.15.172:0/428965133 conn(0x55927f8fc010 msgr2=0x55927f8fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.532+0000 7f0e8cc58740 1 -- 172.21.15.172:0/428965133 shutdown_connections 2024-09-10T08:56:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.532+0000 7f0e8cc58740 1 -- 172.21.15.172:0/428965133 wait complete. 2024-09-10T08:56:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.532+0000 7f0e8cc58740 1 -- start start 2024-09-10T08:56:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e8cc58740 1 Processor -- start 2024-09-10T08:56:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e8cc58740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f9009a0 0x55927f90ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e8cc58740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f90b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e8cc58740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f906370 0x55927f912080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e8cc58740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55927f9092e0 con 0x55927f9012e0 2024-09-10T08:56:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e8cc58740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55927f7ed290 con 0x55927f9009a0 2024-09-10T08:56:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e8cc58740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55927f909460 con 0x55927f906370 2024-09-10T08:56:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e8cc58740 10 client.0 _ll_register_callbacks cb 0x55927f81e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e82ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f90b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e82ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f90b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41536/0 (socket says 172.21.15.172:41536) 2024-09-10T08:56:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e82ffd640 1 -- 172.21.15.172:0/2093468131 learned_addr learned my addr 172.21.15.172:0/2093468131 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e837fe640 1 --2- 172.21.15.172:0/2093468131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f9009a0 0x55927f90ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e82ffd640 1 -- 172.21.15.172:0/2093468131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f906370 msgr2=0x55927f912080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e82ffd640 1 --2- 172.21.15.172:0/2093468131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f906370 0x55927f912080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e82ffd640 1 -- 172.21.15.172:0/2093468131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f9009a0 msgr2=0x55927f90ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e82ffd640 1 --2- 172.21.15.172:0/2093468131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f9009a0 0x55927f90ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e82ffd640 1 -- 172.21.15.172:0/2093468131 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55927f577210 con 0x55927f9012e0 2024-09-10T08:56:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e82ffd640 1 --2- 172.21.15.172:0/2093468131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f90b040 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f0e7402a930 tx=0x7f0e74031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486e10e4c2772071 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.533+0000 7f0e62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.534+0000 7f0e62ffd640 1 -- 172.21.15.172:0/2093468131 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7403f070 con 0x55927f9012e0 2024-09-10T08:56:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.534+0000 7f0e62ffd640 1 -- 172.21.15.172:0/2093468131 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0e74004450 con 0x55927f9012e0 2024-09-10T08:56:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.534+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2093468131 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e7809bea0 con 0x55927f9012e0 2024-09-10T08:56:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.534+0000 7f0e62ffd640 1 -- 172.21.15.172:0/2093468131 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0e7404c050 con 0x55927f9012e0 2024-09-10T08:56:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.534+0000 7f0e8cc58740 10 client.18558 fetch_fsmap learned FSMap version 39 2024-09-10T08:56:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.534+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2093468131 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55927f643620 con 0x55927f9012e0 2024-09-10T08:56:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.534+0000 7f0e62ffd640 1 -- 172.21.15.172:0/2093468131 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e7809bea0 con 0x55927f9012e0 2024-09-10T08:56:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.534+0000 7f0e62ffd640 1 -- 172.21.15.172:0/2093468131 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f0e74004be0 con 0x55927f9012e0 2024-09-10T08:56:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.534+0000 7f0e8cc58740 10 client.18558 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:56:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.534+0000 7f0e8cc58740 10 client.18558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.534+0000 7f0e8cc58740 20 client.18558 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.534+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/2093468131 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55927f916770 0x55927f918c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.534+0000 7f0e8cc58740 4 client.18558 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:56:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.534+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2093468131 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55927f871a20 con 0x55927f916770 2024-09-10T08:56:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.534+0000 7f0e837fe640 1 --2- 172.21.15.172:0/2093468131 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55927f916770 0x55927f918c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.535+0000 7f0e837fe640 1 --2- 172.21.15.172:0/2093468131 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55927f916770 0x55927f918c00 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f540cca50da653 server_cookie=4cfa4f40d6ea8159 in_seq=0 out_seq=0 2024-09-10T08:56:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.535+0000 7f0e62ffd640 10 client.18558 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.536+0000 7f0e62ffd640 1 -- 172.21.15.172:0/2093468131 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55927f871a20 con 0x55927f916770 2024-09-10T08:56:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.536+0000 7f0e62ffd640 10 client.18558 handle_command_reply: tid=0 2024-09-10T08:56:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.536+0000 7f0e8cc58740 1 client.18558 shutdown 2024-09-10T08:56:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.536+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2093468131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 msgr2=0x55927f90b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.536+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/2093468131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f90b040 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f0e7402a930 tx=0x7f0e74031810 comp rx=0 tx=0).stop 2024-09-10T08:56:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.536+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2093468131 shutdown_connections 2024-09-10T08:56:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.536+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/2093468131 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55927f916770 0x55927f918c00 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.536+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/2093468131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f906370 0x55927f912080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.536+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/2093468131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f90b040 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.536+0000 7f0e62ffd640 0 client.18558 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.536+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/2093468131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f9009a0 0x55927f90ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.536+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2093468131 >> 172.21.15.172:0/2093468131 conn(0x55927f8fc010 msgr2=0x55927f8fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.536+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2093468131 shutdown_connections 2024-09-10T08:56:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.536+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2093468131 wait complete. 2024-09-10T08:56:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.536+0000 7f0e8cc58740 20 client.18558 trim_cache size 0 max 16384 2024-09-10T08:56:49.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.550+0000 7f0e8cc58740 1 Processor -- start 2024-09-10T08:56:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.550+0000 7f0e8cc58740 1 -- start start 2024-09-10T08:56:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.550+0000 7f0e8cc58740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9009a0 0x55927f8fb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.550+0000 7f0e8cc58740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f9012e0 0x55927f912040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.550+0000 7f0e8cc58740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f906370 0x55927f8fd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.550+0000 7f0e8cc58740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841aa5d0 con 0x55927f9009a0 2024-09-10T08:56:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.550+0000 7f0e8cc58740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841ac170 con 0x55927f9012e0 2024-09-10T08:56:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.550+0000 7f0e8cc58740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841aaca0 con 0x55927f906370 2024-09-10T08:56:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.551+0000 7f0e837fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9009a0 0x55927f8fb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.551+0000 7f0e837fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9009a0 0x55927f8fb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41544/0 (socket says 172.21.15.172:41544) 2024-09-10T08:56:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.551+0000 7f0e837fe640 1 -- 172.21.15.172:0/2781440568 learned_addr learned my addr 172.21.15.172:0/2781440568 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:49.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.551+0000 7f0e83fff640 1 --2- 172.21.15.172:0/2781440568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f906370 0x55927f8fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.551+0000 7f0e837fe640 1 -- 172.21.15.172:0/2781440568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f906370 msgr2=0x55927f8fd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.551+0000 7f0e837fe640 1 --2- 172.21.15.172:0/2781440568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f906370 0x55927f8fd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.551+0000 7f0e837fe640 1 -- 172.21.15.172:0/2781440568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f9012e0 msgr2=0x55927f912040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.551+0000 7f0e837fe640 1 --2- 172.21.15.172:0/2781440568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f9012e0 0x55927f912040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.551+0000 7f0e837fe640 1 -- 172.21.15.172:0/2781440568 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55927f8fbe30 con 0x55927f9009a0 2024-09-10T08:56:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.551+0000 7f0e83fff640 1 --2- 172.21.15.172:0/2781440568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f906370 0x55927f8fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.551+0000 7f0e837fe640 1 --2- 172.21.15.172:0/2781440568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9009a0 0x55927f8fb0b0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f0e78097b70 tx=0x7f0e780cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2750e90dc4de1b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.551+0000 7f0e697fa640 1 -- 172.21.15.172:0/2781440568 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7801d040 con 0x55927f9009a0 2024-09-10T08:56:49.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.551+0000 7f0e697fa640 1 -- 172.21.15.172:0/2781440568 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e7809b7a0 con 0x55927f9009a0 2024-09-10T08:56:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.551+0000 7f0e697fa640 1 -- 172.21.15.172:0/2781440568 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78003610 con 0x55927f9009a0 2024-09-10T08:56:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.552+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2781440568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9009a0 msgr2=0x55927f8fb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.552+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/2781440568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9009a0 0x55927f8fb0b0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f0e78097b70 tx=0x7f0e780cf040 comp rx=0 tx=0).stop 2024-09-10T08:56:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.552+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2781440568 shutdown_connections 2024-09-10T08:56:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.552+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/2781440568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f906370 0x55927f8fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.552+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/2781440568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f9012e0 0x55927f912040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.552+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/2781440568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9009a0 0x55927f8fb0b0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.552+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2781440568 >> 172.21.15.172:0/2781440568 conn(0x55927f8fc010 msgr2=0x55927f8fa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.552+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2781440568 shutdown_connections 2024-09-10T08:56:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.552+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2781440568 wait complete. 2024-09-10T08:56:49.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.552+0000 7f0e8cc58740 1 -- start start 2024-09-10T08:56:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.552+0000 7f0e8cc58740 1 Processor -- start 2024-09-10T08:56:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e8cc58740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f9009a0 0x55927f90f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e8cc58740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f90f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e8cc58740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f906370 0x55927f910700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e8cc58740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841abaa0 con 0x55927f9012e0 2024-09-10T08:56:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e8cc58740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841ab3d0 con 0x55927f906370 2024-09-10T08:56:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e8cc58740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0e841a85b0 con 0x55927f9009a0 2024-09-10T08:56:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e8cc58740 10 client.0 _ll_register_callbacks cb 0x7f0e74009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e82ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f90f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.560 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e82ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f90f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41558/0 (socket says 172.21.15.172:41558) 2024-09-10T08:56:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e82ffd640 1 -- 172.21.15.172:0/2755565681 learned_addr learned my addr 172.21.15.172:0/2755565681 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e837fe640 1 --2- 172.21.15.172:0/2755565681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f9009a0 0x55927f90f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e82ffd640 1 -- 172.21.15.172:0/2755565681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f9009a0 msgr2=0x55927f90f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e82ffd640 1 --2- 172.21.15.172:0/2755565681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f9009a0 0x55927f90f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e83fff640 1 --2- 172.21.15.172:0/2755565681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f906370 0x55927f910700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e82ffd640 1 -- 172.21.15.172:0/2755565681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f906370 msgr2=0x55927f910700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e82ffd640 1 --2- 172.21.15.172:0/2755565681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f906370 0x55927f910700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e82ffd640 1 -- 172.21.15.172:0/2755565681 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55927f910d10 con 0x55927f9012e0 2024-09-10T08:56:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e82ffd640 1 --2- 172.21.15.172:0/2755565681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f90f740 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f0e740314e0 tx=0x7f0e74032f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba471b863bc0fe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e62ffd640 1 -- 172.21.15.172:0/2755565681 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7403f070 con 0x55927f9012e0 2024-09-10T08:56:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e62ffd640 1 -- 172.21.15.172:0/2755565681 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0e74003c30 con 0x55927f9012e0 2024-09-10T08:56:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2755565681 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e7809b7a0 con 0x55927f9012e0 2024-09-10T08:56:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e62ffd640 1 -- 172.21.15.172:0/2755565681 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0e7405e050 con 0x55927f9012e0 2024-09-10T08:56:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e8cc58740 10 client.18564 fetch_fsmap learned FSMap version 39 2024-09-10T08:56:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.553+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2755565681 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55927f912020 con 0x55927f9012e0 2024-09-10T08:56:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.554+0000 7f0e62ffd640 1 -- 172.21.15.172:0/2755565681 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e7809b7a0 con 0x55927f9012e0 2024-09-10T08:56:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.554+0000 7f0e62ffd640 1 -- 172.21.15.172:0/2755565681 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f0e740046c0 con 0x55927f9012e0 2024-09-10T08:56:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.554+0000 7f0e8cc58740 10 client.18564 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:56:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.554+0000 7f0e8cc58740 10 client.18564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.554+0000 7f0e8cc58740 20 client.18564 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.554+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/2755565681 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55927f9819c0 0x55927f983de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.554+0000 7f0e8cc58740 4 client.18564 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:56:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.554+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2755565681 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55927f86bd70 con 0x55927f9819c0 2024-09-10T08:56:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.554+0000 7f0e837fe640 1 --2- 172.21.15.172:0/2755565681 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55927f9819c0 0x55927f983de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.555+0000 7f0e62ffd640 10 client.18564 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.555+0000 7f0e837fe640 1 --2- 172.21.15.172:0/2755565681 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55927f9819c0 0x55927f983de0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af8dcee84fe3559 server_cookie=f07d6506aef55032 in_seq=0 out_seq=0 2024-09-10T08:56:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.561+0000 7f0e62ffd640 1 -- 172.21.15.172:0/2755565681 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55927f86bd70 con 0x55927f9819c0 2024-09-10T08:56:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e62ffd640 10 client.18564 handle_command_reply: tid=0 2024-09-10T08:56:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e8cc58740 1 client.18564 shutdown 2024-09-10T08:56:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2755565681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 msgr2=0x55927f90f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/2755565681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f90f740 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f0e740314e0 tx=0x7f0e74032f10 comp rx=0 tx=0).stop 2024-09-10T08:56:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2755565681 shutdown_connections 2024-09-10T08:56:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/2755565681 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55927f9819c0 0x55927f983de0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/2755565681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55927f906370 0x55927f910700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e62ffd640 0 client.18564 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/2755565681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55927f9012e0 0x55927f90f740 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e8cc58740 1 --2- 172.21.15.172:0/2755565681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55927f9009a0 0x55927f90f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2755565681 >> 172.21.15.172:0/2755565681 conn(0x55927f8fc010 msgr2=0x55927f8fb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2755565681 shutdown_connections 2024-09-10T08:56:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e8cc58740 1 -- 172.21.15.172:0/2755565681 wait complete. 2024-09-10T08:56:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e8cc58740 20 client.18564 trim_cache size 0 max 16384 2024-09-10T08:56:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:56:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:56:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:56:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:56:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e89de4640 1 -- 172.21.15.172:0/3573386420 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0e6c0a2c30 msgr2=0x7f0e6c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e89de4640 1 --2- 172.21.15.172:0/3573386420 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0e6c0a2c30 0x7f0e6c0a50f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c007d80 tx=0x7f0e7c007440 comp rx=0 tx=0).stop 2024-09-10T08:56:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e89de4640 1 -- 172.21.15.172:0/3573386420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e84101b60 msgr2=0x7f0e84197d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e89de4640 1 --2- 172.21.15.172:0/3573386420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e84101b60 0x7f0e84197d10 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c970 tx=0x7f0e7800ce30 comp rx=0 tx=0).stop 2024-09-10T08:56:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e89de4640 1 -- 172.21.15.172:0/3573386420 shutdown_connections 2024-09-10T08:56:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e89de4640 1 --2- 172.21.15.172:0/3573386420 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0e6c0a2c30 0x7f0e6c0a50f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e89de4640 1 --2- 172.21.15.172:0/3573386420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e84103710 0x7f0e8419f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e89de4640 1 --2- 172.21.15.172:0/3573386420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e84102d50 0x7f0e84198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.562+0000 7f0e89de4640 1 --2- 172.21.15.172:0/3573386420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e84101b60 0x7f0e84197d10 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.563+0000 7f0e89de4640 1 -- 172.21.15.172:0/3573386420 >> 172.21.15.172:0/3573386420 conn(0x7f0e840fd2e0 msgr2=0x7f0e840ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.563+0000 7f0e89de4640 1 -- 172.21.15.172:0/3573386420 shutdown_connections 2024-09-10T08:56:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:56:49.563+0000 7f0e89de4640 1 -- 172.21.15.172:0/3573386420 wait complete. 2024-09-10T08:56:49.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.834+0000 7f23e8c9c640 1 -- 172.21.15.172:0/484466490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23e4101e10 msgr2=0x7f23e41021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.834+0000 7f23e8c9c640 1 --2- 172.21.15.172:0/484466490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23e4101e10 0x7f23e41021f0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009900 tx=0x7f23d402a410 comp rx=0 tx=0).stop 2024-09-10T08:56:49.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.834+0000 7f23e8c9c640 1 -- 172.21.15.172:0/484466490 shutdown_connections 2024-09-10T08:56:49.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.834+0000 7f23e8c9c640 1 --2- 172.21.15.172:0/484466490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23e4106f40 0x7f23e410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.834+0000 7f23e8c9c640 1 --2- 172.21.15.172:0/484466490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23e41027c0 0x7f23e4106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.834+0000 7f23e8c9c640 1 --2- 172.21.15.172:0/484466490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23e4101e10 0x7f23e41021f0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.834+0000 7f23e8c9c640 1 -- 172.21.15.172:0/484466490 >> 172.21.15.172:0/484466490 conn(0x7f23e40fd8d0 msgr2=0x7f23e40ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:49.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.834+0000 7f23e8c9c640 1 -- 172.21.15.172:0/484466490 shutdown_connections 2024-09-10T08:56:49.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.834+0000 7f23e8c9c640 1 -- 172.21.15.172:0/484466490 wait complete. 2024-09-10T08:56:49.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.834+0000 7f23e8c9c640 1 Processor -- start 2024-09-10T08:56:49.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23e8c9c640 1 -- start start 2024-09-10T08:56:49.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23e8c9c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23e4101e10 0x7f23e4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23e8c9c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23e41027c0 0x7f23e4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23e8c9c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23e4106f40 0x7f23e419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23e8c9c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23e4111e90 con 0x7f23e4101e10 2024-09-10T08:56:49.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23e8c9c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23e4111d10 con 0x7f23e4106f40 2024-09-10T08:56:49.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23e8c9c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f23e4112010 con 0x7f23e41027c0 2024-09-10T08:56:49.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23e259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23e4101e10 0x7f23e4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23e259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23e4101e10 0x7f23e4198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41580/0 (socket says 172.21.15.172:41580) 2024-09-10T08:56:49.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23e259c640 1 -- 172.21.15.172:0/3775139254 learned_addr learned my addr 172.21.15.172:0/3775139254 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:49.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23e2d9d640 1 --2- 172.21.15.172:0/3775139254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23e4106f40 0x7f23e419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23d9d9b640 1 --2- 172.21.15.172:0/3775139254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23e41027c0 0x7f23e4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23e259c640 1 -- 172.21.15.172:0/3775139254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23e41027c0 msgr2=0x7f23e4198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23e259c640 1 --2- 172.21.15.172:0/3775139254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23e41027c0 0x7f23e4198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23e259c640 1 -- 172.21.15.172:0/3775139254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23e4106f40 msgr2=0x7f23e419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23e259c640 1 --2- 172.21.15.172:0/3775139254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23e4106f40 0x7f23e419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23e259c640 1 -- 172.21.15.172:0/3775139254 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e419d3f0 con 0x7f23e4101e10 2024-09-10T08:56:49.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.835+0000 7f23e2d9d640 1 --2- 172.21.15.172:0/3775139254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23e4106f40 0x7f23e419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:49.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.836+0000 7f23e259c640 1 --2- 172.21.15.172:0/3775139254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23e4101e10 0x7f23e4198440 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f23d4006010 tx=0x7f23d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:49.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.836+0000 7f23dbfff640 1 -- 172.21.15.172:0/3775139254 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d403f030 con 0x7f23e4101e10 2024-09-10T08:56:49.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.836+0000 7f23e8c9c640 1 -- 172.21.15.172:0/3775139254 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e419d680 con 0x7f23e4101e10 2024-09-10T08:56:49.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.836+0000 7f23dbfff640 1 -- 172.21.15.172:0/3775139254 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23d40035f0 con 0x7f23e4101e10 2024-09-10T08:56:49.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.836+0000 7f23dbfff640 1 -- 172.21.15.172:0/3775139254 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4035070 con 0x7f23e4101e10 2024-09-10T08:56:49.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.836+0000 7f23e8c9c640 1 -- 172.21.15.172:0/3775139254 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e41a4fc0 con 0x7f23e4101e10 2024-09-10T08:56:49.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.838+0000 7f23dbfff640 1 -- 172.21.15.172:0/3775139254 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f23d4003b10 con 0x7f23e4101e10 2024-09-10T08:56:49.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.839+0000 7f23eb075740 1 Processor -- start 2024-09-10T08:56:49.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.839+0000 7f23eb075740 1 -- start start 2024-09-10T08:56:49.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.839+0000 7f23dbfff640 1 --2- 172.21.15.172:0/3775139254 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f23d00a2b80 0x7f23d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.839+0000 7f23eb075740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007c9060 0x5593007c9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.839+0000 7f23eb075740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007c9980 0x5593007d3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.839+0000 7f23eb075740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007d4240 0x5593007d6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.839+0000 7f23eb075740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55930072e620 con 0x5593007d4240 2024-09-10T08:56:49.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.839+0000 7f23eb075740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5593006d5e50 con 0x5593007c9060 2024-09-10T08:56:49.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.839+0000 7f23eb075740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55930072a5e0 con 0x5593007c9980 2024-09-10T08:56:49.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.839+0000 7f23dbfff640 1 -- 172.21.15.172:0/3775139254 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f23d40efe00 con 0x7f23e4101e10 2024-09-10T08:56:49.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.839+0000 7f23e2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007d4240 0x5593007d6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.839+0000 7f23e2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007d4240 0x5593007d6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41596/0 (socket says 172.21.15.172:41596) 2024-09-10T08:56:49.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.839+0000 7f23e2d9d640 1 -- 172.21.15.172:0/1365696793 learned_addr learned my addr 172.21.15.172:0/1365696793 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:49.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.840+0000 7f23e2d9d640 1 --2- 172.21.15.172:0/3775139254 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f23d00a2b80 0x7f23d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.840+0000 7f23d9d9b640 1 --2- 172.21.15.172:0/1365696793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007c9980 0x5593007d3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.840+0000 7f23e259c640 1 --2- 172.21.15.172:0/1365696793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007c9060 0x5593007c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.840+0000 7f23e2d9d640 1 -- 172.21.15.172:0/1365696793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007c9980 msgr2=0x5593007d3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:49.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.840+0000 7f23e2d9d640 1 --2- 172.21.15.172:0/1365696793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007c9980 0x5593007d3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.840+0000 7f23e2d9d640 1 -- 172.21.15.172:0/1365696793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007c9060 msgr2=0x5593007c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:49.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.840+0000 7f23e2d9d640 1 --2- 172.21.15.172:0/1365696793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007c9060 0x5593007c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.840+0000 7f23e2d9d640 1 -- 172.21.15.172:0/1365696793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593006c6d40 con 0x5593007d4240 2024-09-10T08:56:49.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.840+0000 7f23d9d9b640 1 --2- 172.21.15.172:0/1365696793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007c9980 0x5593007d3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:49.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.840+0000 7f23e259c640 1 --2- 172.21.15.172:0/1365696793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007c9060 0x5593007c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:56:49.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.840+0000 7f23e2d9d640 1 --2- 172.21.15.172:0/1365696793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007d4240 0x5593007d6630 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f23dc013a00 tx=0x7f23dc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac79d5063371bac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:49.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.840+0000 7f23e2d9d640 1 --2- 172.21.15.172:0/3775139254 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f23d00a2b80 0x7f23d00a5040 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f23dc010ca0 tx=0x7f23dc00d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:49.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.840+0000 7f23d8d99640 1 -- 172.21.15.172:0/1365696793 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc013840 con 0x5593007d4240 2024-09-10T08:56:49.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.840+0000 7f23d8d99640 1 -- 172.21.15.172:0/1365696793 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23dc00ac30 con 0x5593007d4240 2024-09-10T08:56:49.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.840+0000 7f23d8d99640 1 -- 172.21.15.172:0/1365696793 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc004620 con 0x5593007d4240 2024-09-10T08:56:49.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.841+0000 7f23eb075740 1 -- 172.21.15.172:0/1365696793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007d4240 msgr2=0x5593007d6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.841+0000 7f23eb075740 1 --2- 172.21.15.172:0/1365696793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007d4240 0x5593007d6630 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f23dc013a00 tx=0x7f23dc0107a0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.841+0000 7f23eb075740 1 -- 172.21.15.172:0/1365696793 shutdown_connections 2024-09-10T08:56:49.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.841+0000 7f23eb075740 1 --2- 172.21.15.172:0/1365696793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007d4240 0x5593007d6630 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.841+0000 7f23eb075740 1 --2- 172.21.15.172:0/1365696793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007c9980 0x5593007d3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.841+0000 7f23eb075740 1 --2- 172.21.15.172:0/1365696793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007c9060 0x5593007c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.841+0000 7f23eb075740 1 -- 172.21.15.172:0/1365696793 >> 172.21.15.172:0/1365696793 conn(0x5593007c55f0 msgr2=0x5593007c59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:49.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.841+0000 7f23eb075740 1 -- 172.21.15.172:0/1365696793 shutdown_connections 2024-09-10T08:56:49.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.841+0000 7f23eb075740 1 -- 172.21.15.172:0/1365696793 wait complete. 2024-09-10T08:56:49.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.841+0000 7f23eb075740 1 -- start start 2024-09-10T08:56:49.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23eb075740 1 Processor -- start 2024-09-10T08:56:49.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23eb075740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007c9060 0x5593007d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23eb075740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007c9980 0x5593007d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23eb075740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007d4240 0x5593007d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23eb075740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5593007d6db0 con 0x5593007d4240 2024-09-10T08:56:49.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23eb075740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5593006b6f10 con 0x5593007c9980 2024-09-10T08:56:49.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23eb075740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5593007d6f30 con 0x5593007c9060 2024-09-10T08:56:49.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23eb075740 10 client.0 _ll_register_callbacks cb 0x5593006dadb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:49.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23e2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007d4240 0x5593007d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23e2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007d4240 0x5593007d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41604/0 (socket says 172.21.15.172:41604) 2024-09-10T08:56:49.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23e2d9d640 1 -- 172.21.15.172:0/475875409 learned_addr learned my addr 172.21.15.172:0/475875409 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:49.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23e259c640 1 --2- 172.21.15.172:0/475875409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007c9060 0x5593007d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23d9d9b640 1 --2- 172.21.15.172:0/475875409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007c9980 0x5593007d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23e2d9d640 1 -- 172.21.15.172:0/475875409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007c9060 msgr2=0x5593007d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:49.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23e2d9d640 1 --2- 172.21.15.172:0/475875409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007c9060 0x5593007d11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23e2d9d640 1 -- 172.21.15.172:0/475875409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007c9980 msgr2=0x5593007d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:49.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23e2d9d640 1 --2- 172.21.15.172:0/475875409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007c9980 0x5593007d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.842+0000 7f23e2d9d640 1 -- 172.21.15.172:0/475875409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55930043d580 con 0x5593007d4240 2024-09-10T08:56:49.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.843+0000 7f23e2d9d640 1 --2- 172.21.15.172:0/475875409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007d4240 0x5593007d1c20 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f23dc003f00 tx=0x7f23dc005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d61b8157f8a2e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:49.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.843+0000 7f23c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:49.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.843+0000 7f23c9ffb640 1 -- 172.21.15.172:0/475875409 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc033020 con 0x5593007d4240 2024-09-10T08:56:49.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.843+0000 7f23c9ffb640 1 -- 172.21.15.172:0/475875409 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f23dc003f70 con 0x5593007d4240 2024-09-10T08:56:49.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.843+0000 7f23eb075740 1 -- 172.21.15.172:0/475875409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55930073bb90 con 0x5593007d4240 2024-09-10T08:56:49.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.843+0000 7f23c9ffb640 1 -- 172.21.15.172:0/475875409 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f23dc03e050 con 0x5593007d4240 2024-09-10T08:56:49.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.843+0000 7f23eb075740 10 client.18576 fetch_fsmap learned FSMap version 39 2024-09-10T08:56:49.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.843+0000 7f23eb075740 1 -- 172.21.15.172:0/475875409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559300507960 con 0x5593007d4240 2024-09-10T08:56:49.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.843+0000 7f23c9ffb640 1 -- 172.21.15.172:0/475875409 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55930073bb90 con 0x5593007d4240 2024-09-10T08:56:49.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.843+0000 7f23c9ffb640 1 -- 172.21.15.172:0/475875409 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f23dc005040 con 0x5593007d4240 2024-09-10T08:56:49.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.843+0000 7f23eb075740 10 client.18576 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:56:49.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.843+0000 7f23eb075740 10 client.18576 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:56:49.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.843+0000 7f23eb075740 20 client.18576 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:49.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.843+0000 7f23eb075740 1 --2- 172.21.15.172:0/475875409 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5593007e3d40 0x5593007e6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.843+0000 7f23eb075740 4 client.18576 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:56:49.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.843+0000 7f23eb075740 1 -- 172.21.15.172:0/475875409 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593007360a0 con 0x5593007e3d40 2024-09-10T08:56:49.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.844+0000 7f23e259c640 1 --2- 172.21.15.172:0/475875409 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5593007e3d40 0x5593007e6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.844+0000 7f23c9ffb640 10 client.18576 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:49.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.844+0000 7f23e259c640 1 --2- 172.21.15.172:0/475875409 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5593007e3d40 0x5593007e6130 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7cedb93dc2bb30c server_cookie=cb79daf6ac48cff5 in_seq=0 out_seq=0 2024-09-10T08:56:49.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.845+0000 7f23c9ffb640 1 -- 172.21.15.172:0/475875409 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5593007360a0 con 0x5593007e3d40 2024-09-10T08:56:49.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.845+0000 7f23c9ffb640 10 client.18576 handle_command_reply: tid=0 2024-09-10T08:56:49.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.845+0000 7f23eb075740 1 client.18576 shutdown 2024-09-10T08:56:49.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.845+0000 7f23eb075740 1 -- 172.21.15.172:0/475875409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007d4240 msgr2=0x5593007d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.845+0000 7f23eb075740 1 --2- 172.21.15.172:0/475875409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007d4240 0x5593007d1c20 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f23dc003f00 tx=0x7f23dc005c10 comp rx=0 tx=0).stop 2024-09-10T08:56:49.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.845+0000 7f23eb075740 1 -- 172.21.15.172:0/475875409 shutdown_connections 2024-09-10T08:56:49.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.845+0000 7f23eb075740 1 --2- 172.21.15.172:0/475875409 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5593007e3d40 0x5593007e6130 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.845+0000 7f23eb075740 1 --2- 172.21.15.172:0/475875409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007d4240 0x5593007d1c20 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.845+0000 7f23eb075740 1 --2- 172.21.15.172:0/475875409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007c9980 0x5593007d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.845+0000 7f23c9ffb640 0 client.18576 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:49.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.845+0000 7f23eb075740 1 --2- 172.21.15.172:0/475875409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007c9060 0x5593007d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.845+0000 7f23eb075740 1 -- 172.21.15.172:0/475875409 >> 172.21.15.172:0/475875409 conn(0x5593007c55f0 msgr2=0x5593007cb320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:49.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.845+0000 7f23eb075740 1 -- 172.21.15.172:0/475875409 shutdown_connections 2024-09-10T08:56:49.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.845+0000 7f23eb075740 1 -- 172.21.15.172:0/475875409 wait complete. 2024-09-10T08:56:49.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.845+0000 7f23eb075740 20 client.18576 trim_cache size 0 max 16384 2024-09-10T08:56:49.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23eb075740 1 Processor -- start 2024-09-10T08:56:49.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23eb075740 1 -- start start 2024-09-10T08:56:49.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23eb075740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007c9060 0x5593007cb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23eb075740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007c9980 0x5593007eb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23eb075740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007d4240 0x5593007ebd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23eb075740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23e41ac9b0 con 0x5593007c9980 2024-09-10T08:56:49.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23eb075740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23e41ae550 con 0x5593007d4240 2024-09-10T08:56:49.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23eb075740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f23e41ad080 con 0x5593007c9060 2024-09-10T08:56:49.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23d9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007c9980 0x5593007eb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23d9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007c9980 0x5593007eb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41620/0 (socket says 172.21.15.172:41620) 2024-09-10T08:56:49.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23d9d9b640 1 -- 172.21.15.172:0/4121598851 learned_addr learned my addr 172.21.15.172:0/4121598851 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:49.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23e2d9d640 1 --2- 172.21.15.172:0/4121598851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007d4240 0x5593007ebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23e259c640 1 --2- 172.21.15.172:0/4121598851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007c9060 0x5593007cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23d9d9b640 1 -- 172.21.15.172:0/4121598851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007c9060 msgr2=0x5593007cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:49.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23d9d9b640 1 --2- 172.21.15.172:0/4121598851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007c9060 0x5593007cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23d9d9b640 1 -- 172.21.15.172:0/4121598851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007d4240 msgr2=0x5593007ebd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:49.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23d9d9b640 1 --2- 172.21.15.172:0/4121598851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007d4240 0x5593007ebd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23d9d9b640 1 -- 172.21.15.172:0/4121598851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593007cf6f0 con 0x5593007c9980 2024-09-10T08:56:49.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23e259c640 1 --2- 172.21.15.172:0/4121598851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007c9060 0x5593007cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:56:49.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.862+0000 7f23d9d9b640 1 --2- 172.21.15.172:0/4121598851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007c9980 0x5593007eb7c0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f23e40fffa0 tx=0x7f23cc00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1725adfaf925c7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:49.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.863+0000 7f23cbfff640 1 -- 172.21.15.172:0/4121598851 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc01a070 con 0x5593007c9980 2024-09-10T08:56:49.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.863+0000 7f23cbfff640 1 -- 172.21.15.172:0/4121598851 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23cc014440 con 0x5593007c9980 2024-09-10T08:56:49.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.863+0000 7f23cbfff640 1 -- 172.21.15.172:0/4121598851 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc00cea0 con 0x5593007c9980 2024-09-10T08:56:49.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.863+0000 7f23eb075740 1 -- 172.21.15.172:0/4121598851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007c9980 msgr2=0x5593007eb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.863+0000 7f23eb075740 1 --2- 172.21.15.172:0/4121598851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007c9980 0x5593007eb7c0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f23e40fffa0 tx=0x7f23cc00cc50 comp rx=0 tx=0).stop 2024-09-10T08:56:49.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.863+0000 7f23eb075740 1 -- 172.21.15.172:0/4121598851 shutdown_connections 2024-09-10T08:56:49.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.863+0000 7f23eb075740 1 --2- 172.21.15.172:0/4121598851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007d4240 0x5593007ebd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.863+0000 7f23eb075740 1 --2- 172.21.15.172:0/4121598851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007c9980 0x5593007eb7c0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.863+0000 7f23eb075740 1 --2- 172.21.15.172:0/4121598851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007c9060 0x5593007cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.863+0000 7f23eb075740 1 -- 172.21.15.172:0/4121598851 >> 172.21.15.172:0/4121598851 conn(0x5593007c55f0 msgr2=0x5593007cab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:49.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.863+0000 7f23eb075740 1 -- 172.21.15.172:0/4121598851 shutdown_connections 2024-09-10T08:56:49.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.863+0000 7f23eb075740 1 -- 172.21.15.172:0/4121598851 wait complete. 2024-09-10T08:56:49.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.863+0000 7f23eb075740 1 -- start start 2024-09-10T08:56:49.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23eb075740 1 Processor -- start 2024-09-10T08:56:49.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23eb075740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007c9060 0x5593007dc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23eb075740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007c9980 0x5593007dc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23eb075740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007d4240 0x5593007d9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23eb075740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23e41ade80 con 0x5593007c9060 2024-09-10T08:56:49.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23eb075740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23e41ad7b0 con 0x5593007c9980 2024-09-10T08:56:49.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23eb075740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f23e41aa990 con 0x5593007d4240 2024-09-10T08:56:49.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23eb075740 10 client.0 _ll_register_callbacks cb 0x7f23dc003d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:49.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23e259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007c9060 0x5593007dc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23e259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007c9060 0x5593007dc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41632/0 (socket says 172.21.15.172:41632) 2024-09-10T08:56:49.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23e259c640 1 -- 172.21.15.172:0/3224914378 learned_addr learned my addr 172.21.15.172:0/3224914378 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:49.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23d9d9b640 1 --2- 172.21.15.172:0/3224914378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007c9980 0x5593007dc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23e259c640 1 -- 172.21.15.172:0/3224914378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007d4240 msgr2=0x5593007d9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:49.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23e259c640 1 --2- 172.21.15.172:0/3224914378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007d4240 0x5593007d9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23e259c640 1 -- 172.21.15.172:0/3224914378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007c9980 msgr2=0x5593007dc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:49.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23e259c640 1 --2- 172.21.15.172:0/3224914378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007c9980 0x5593007dc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23e259c640 1 -- 172.21.15.172:0/3224914378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593007d97c0 con 0x5593007c9060 2024-09-10T08:56:49.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23e259c640 1 --2- 172.21.15.172:0/3224914378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007c9060 0x5593007dc360 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f23d40dc010 tx=0x7f23d402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a55c9fd832b85a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:49.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.864+0000 7f23c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:49.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.865+0000 7f23c9ffb640 1 -- 172.21.15.172:0/3224914378 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4049040 con 0x5593007c9060 2024-09-10T08:56:49.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.865+0000 7f23c9ffb640 1 -- 172.21.15.172:0/3224914378 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f23d40a5450 con 0x5593007c9060 2024-09-10T08:56:49.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.865+0000 7f23eb075740 1 -- 172.21.15.172:0/3224914378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23cc014440 con 0x5593007c9060 2024-09-10T08:56:49.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.865+0000 7f23c9ffb640 1 -- 172.21.15.172:0/3224914378 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f23d40d5050 con 0x5593007c9060 2024-09-10T08:56:49.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.865+0000 7f23eb075740 10 client.18582 fetch_fsmap learned FSMap version 39 2024-09-10T08:56:49.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.865+0000 7f23eb075740 1 -- 172.21.15.172:0/3224914378 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55930084f0b0 con 0x5593007c9060 2024-09-10T08:56:49.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.865+0000 7f23c9ffb640 1 -- 172.21.15.172:0/3224914378 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23cc014440 con 0x5593007c9060 2024-09-10T08:56:49.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.865+0000 7f23c9ffb640 1 -- 172.21.15.172:0/3224914378 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f23d40a5130 con 0x5593007c9060 2024-09-10T08:56:49.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.865+0000 7f23eb075740 10 client.18582 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:56:49.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.865+0000 7f23eb075740 10 client.18582 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:56:49.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.865+0000 7f23eb075740 20 client.18582 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:49.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.865+0000 7f23eb075740 1 --2- 172.21.15.172:0/3224914378 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55930084f780 0x559300851bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:49.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.865+0000 7f23eb075740 4 client.18582 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "46fa34bb", "release": true, "await": true}] 2024-09-10T08:56:49.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.865+0000 7f23eb075740 1 -- 172.21.15.172:0/3224914378 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "set_id": "46fa34bb", "release": true, "await": true}) -- 0x5593006b25a0 con 0x55930084f780 2024-09-10T08:56:49.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.865+0000 7f23d9d9b640 1 --2- 172.21.15.172:0/3224914378 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55930084f780 0x559300851bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:49.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.866+0000 7f23c9ffb640 10 client.18582 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:49.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.866+0000 7f23d9d9b640 1 --2- 172.21.15.172:0/3224914378 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55930084f780 0x559300851bb0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1791b76a6134d2b server_cookie=cf3786854fe96a5 in_seq=0 out_seq=0 2024-09-10T08:56:49.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.891+0000 7f23c9ffb640 1 -- 172.21.15.172:0/3224914378 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5593006b25a0 con 0x55930084f780 2024-09-10T08:56:49.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.891+0000 7f23c9ffb640 10 client.18582 handle_command_reply: tid=0 2024-09-10T08:56:49.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.891+0000 7f23eb075740 1 client.18582 shutdown 2024-09-10T08:56:49.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.891+0000 7f23eb075740 1 -- 172.21.15.172:0/3224914378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007c9060 msgr2=0x5593007dc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.891+0000 7f23eb075740 1 --2- 172.21.15.172:0/3224914378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007c9060 0x5593007dc360 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f23d40dc010 tx=0x7f23d402c000 comp rx=0 tx=0).stop 2024-09-10T08:56:49.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.891+0000 7f23eb075740 1 -- 172.21.15.172:0/3224914378 shutdown_connections 2024-09-10T08:56:49.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.891+0000 7f23eb075740 1 --2- 172.21.15.172:0/3224914378 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55930084f780 0x559300851bb0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.891+0000 7f23eb075740 1 --2- 172.21.15.172:0/3224914378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593007d4240 0x5593007d9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.891+0000 7f23eb075740 1 --2- 172.21.15.172:0/3224914378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593007c9980 0x5593007dc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.891+0000 7f23c9ffb640 0 client.18582 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:49.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.891+0000 7f23eb075740 1 --2- 172.21.15.172:0/3224914378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593007c9060 0x5593007dc360 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.891+0000 7f23eb075740 1 -- 172.21.15.172:0/3224914378 >> 172.21.15.172:0/3224914378 conn(0x5593007c55f0 msgr2=0x5593007cb320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:49.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.891+0000 7f23eb075740 1 -- 172.21.15.172:0/3224914378 shutdown_connections 2024-09-10T08:56:49.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.891+0000 7f23eb075740 1 -- 172.21.15.172:0/3224914378 wait complete. 2024-09-10T08:56:49.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.891+0000 7f23eb075740 20 client.18582 trim_cache size 0 max 16384 2024-09-10T08:56:49.894 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:56:49.894 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 39, 2024-09-10T08:56:49.894 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:56:49.894 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 36, 2024-09-10T08:56:49.894 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:56:49.894 INFO:teuthology.orchestra.run.smithi172.stdout: "46fa34bb": { 2024-09-10T08:56:49.894 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 36, 2024-09-10T08:56:49.894 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:56:49.894 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:56:49.894 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:56:49.894 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:56:49.895 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:56:49.895 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:56:49.895 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 130.0, 2024-09-10T08:56:49.895 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:56:49.895 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:56:49.895 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:56:49.895 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:56:49.895 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:56:49.895 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:56:49.895 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:56:49.895 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:56:49.895 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:56:49.895 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:56:49.895 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:56:49.896 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:56:49.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.892+0000 7f23e8c9c640 1 -- 172.21.15.172:0/3775139254 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f23d00a2b80 msgr2=0x7f23d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.892+0000 7f23e8c9c640 1 --2- 172.21.15.172:0/3775139254 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f23d00a2b80 0x7f23d00a5040 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f23dc010ca0 tx=0x7f23dc00d000 comp rx=0 tx=0).stop 2024-09-10T08:56:49.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.892+0000 7f23e8c9c640 1 -- 172.21.15.172:0/3775139254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23e4101e10 msgr2=0x7f23e4198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:49.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.892+0000 7f23e8c9c640 1 --2- 172.21.15.172:0/3775139254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23e4101e10 0x7f23e4198440 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f23d4006010 tx=0x7f23d4004270 comp rx=0 tx=0).stop 2024-09-10T08:56:49.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.892+0000 7f23e8c9c640 1 -- 172.21.15.172:0/3775139254 shutdown_connections 2024-09-10T08:56:49.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.892+0000 7f23e8c9c640 1 --2- 172.21.15.172:0/3775139254 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f23d00a2b80 0x7f23d00a5040 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.892+0000 7f23e8c9c640 1 --2- 172.21.15.172:0/3775139254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23e4106f40 0x7f23e419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.892+0000 7f23e8c9c640 1 --2- 172.21.15.172:0/3775139254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23e41027c0 0x7f23e4198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.892+0000 7f23e8c9c640 1 --2- 172.21.15.172:0/3775139254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23e4101e10 0x7f23e4198440 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:49.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.892+0000 7f23e8c9c640 1 -- 172.21.15.172:0/3775139254 >> 172.21.15.172:0/3775139254 conn(0x7f23e40fd8d0 msgr2=0x7f23e40fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:49.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.892+0000 7f23e8c9c640 1 -- 172.21.15.172:0/3775139254 shutdown_connections 2024-09-10T08:56:49.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:49.892+0000 7f23e8c9c640 1 -- 172.21.15.172:0/3775139254 wait complete. 2024-09-10T08:56:50.631 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:50 smithi172 ceph-mon[30854]: pgmap v668: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 17 op/s 2024-09-10T08:56:50.632 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '46fa34bb', total seconds elapsed: 13.4 2024-09-10T08:56:50.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:50 smithi174 ceph-mon[34068]: pgmap v668: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 17 op/s 2024-09-10T08:56:50.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:50 smithi191 ceph-mon[35314]: pgmap v668: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 17 op/s 2024-09-10T08:56:51.465 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:56:51.748 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:56:52.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.268+0000 7f0345049640 1 -- 172.21.15.172:0/1074197146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0340102ef0 msgr2=0x7f034010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:52.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.268+0000 7f0345049640 1 --2- 172.21.15.172:0/1074197146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0340102ef0 0x7f034010da40 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f0338009ca0 tx=0x7f033802a950 comp rx=0 tx=0).stop 2024-09-10T08:56:52.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.268+0000 7f0345049640 1 -- 172.21.15.172:0/1074197146 shutdown_connections 2024-09-10T08:56:52.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.268+0000 7f0345049640 1 --2- 172.21.15.172:0/1074197146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0340102ef0 0x7f034010da40 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.268+0000 7f0345049640 1 --2- 172.21.15.172:0/1074197146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0340102570 0x7f03401029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.268+0000 7f0345049640 1 --2- 172.21.15.172:0/1074197146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0340100e20 0x7f0340101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.268+0000 7f0345049640 1 -- 172.21.15.172:0/1074197146 >> 172.21.15.172:0/1074197146 conn(0x7f03400fc7d0 msgr2=0x7f03400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:52.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.269+0000 7f0345049640 1 -- 172.21.15.172:0/1074197146 shutdown_connections 2024-09-10T08:56:52.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.269+0000 7f0345049640 1 -- 172.21.15.172:0/1074197146 wait complete. 2024-09-10T08:56:52.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.269+0000 7f0345049640 1 Processor -- start 2024-09-10T08:56:52.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.269+0000 7f0345049640 1 -- start start 2024-09-10T08:56:52.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.270+0000 7f0345049640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0340100e20 0x7f034019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.270+0000 7f0345049640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0340102570 0x7f034019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.270+0000 7f0345049640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0340102ef0 0x7f034019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.270+0000 7f0345049640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0340110e30 con 0x7f0340102570 2024-09-10T08:56:52.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.270+0000 7f0345049640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0340110cb0 con 0x7f0340100e20 2024-09-10T08:56:52.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.270+0000 7f0345049640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0340110fb0 con 0x7f0340102ef0 2024-09-10T08:56:52.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.270+0000 7f033e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0340102570 0x7f034019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.270+0000 7f033e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0340102570 0x7f034019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41654/0 (socket says 172.21.15.172:41654) 2024-09-10T08:56:52.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.270+0000 7f033e59c640 1 -- 172.21.15.172:0/1678142422 learned_addr learned my addr 172.21.15.172:0/1678142422 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:52.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.270+0000 7f033ed9d640 1 --2- 172.21.15.172:0/1678142422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0340100e20 0x7f034019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.270+0000 7f033f59e640 1 --2- 172.21.15.172:0/1678142422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0340102ef0 0x7f034019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.270+0000 7f033ed9d640 1 -- 172.21.15.172:0/1678142422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0340102ef0 msgr2=0x7f034019edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:52.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.270+0000 7f033ed9d640 1 --2- 172.21.15.172:0/1678142422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0340102ef0 0x7f034019edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.270+0000 7f033ed9d640 1 -- 172.21.15.172:0/1678142422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0340102570 msgr2=0x7f034019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:52.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.270+0000 7f033ed9d640 1 --2- 172.21.15.172:0/1678142422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0340102570 0x7f034019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.270+0000 7f033ed9d640 1 -- 172.21.15.172:0/1678142422 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034019f4b0 con 0x7f0340100e20 2024-09-10T08:56:52.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.270+0000 7f033e59c640 1 --2- 172.21.15.172:0/1678142422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0340102570 0x7f034019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:52.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.271+0000 7f033ed9d640 1 --2- 172.21.15.172:0/1678142422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0340100e20 0x7f034019a500 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f033000d6d0 tx=0x7f033000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:52.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.271+0000 7f0327fff640 1 -- 172.21.15.172:0/1678142422 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330017070 con 0x7f0340100e20 2024-09-10T08:56:52.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.271+0000 7f0345049640 1 -- 172.21.15.172:0/1678142422 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034019f7a0 con 0x7f0340100e20 2024-09-10T08:56:52.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.271+0000 7f0327fff640 1 -- 172.21.15.172:0/1678142422 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0330009820 con 0x7f0340100e20 2024-09-10T08:56:52.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.271+0000 7f0327fff640 1 -- 172.21.15.172:0/1678142422 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330004c40 con 0x7f0340100e20 2024-09-10T08:56:52.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.271+0000 7f0345049640 1 -- 172.21.15.172:0/1678142422 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03401a7080 con 0x7f0340100e20 2024-09-10T08:56:52.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.274+0000 7f0327fff640 1 -- 172.21.15.172:0/1678142422 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f03300099d0 con 0x7f0340100e20 2024-09-10T08:56:52.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.274+0000 7f0347422740 1 Processor -- start 2024-09-10T08:56:52.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.274+0000 7f0347422740 1 -- start start 2024-09-10T08:56:52.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.274+0000 7f0347422740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e21527d4a0 0x55e21527d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.274+0000 7f0347422740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e215288140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.274+0000 7f0347422740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e215288680 0x55e21528aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.274+0000 7f0347422740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e2151e32e0 con 0x55e215288680 2024-09-10T08:56:52.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.274+0000 7f0347422740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e21518ace0 con 0x55e21527ddc0 2024-09-10T08:56:52.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.274+0000 7f0347422740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e2151df2a0 con 0x55e21527d4a0 2024-09-10T08:56:52.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.274+0000 7f033e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e215288680 0x55e21528aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.274+0000 7f033f59e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e21527d4a0 0x55e21527d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.274+0000 7f033e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e215288680 0x55e21528aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41668/0 (socket says 172.21.15.172:41668) 2024-09-10T08:56:52.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.275+0000 7f033e59c640 1 -- 172.21.15.172:0/4031952871 learned_addr learned my addr 172.21.15.172:0/4031952871 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:52.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.275+0000 7f0327fff640 1 --2- 172.21.15.172:0/1678142422 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f03280a2b80 0x7f03280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.275+0000 7f033ed9d640 1 --2- 172.21.15.172:0/4031952871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e215288140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.275+0000 7f033f59e640 1 --2- 172.21.15.172:0/1678142422 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f03280a2b80 0x7f03280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.275+0000 7f033e59c640 1 -- 172.21.15.172:0/4031952871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e21527d4a0 msgr2=0x55e21527d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:52.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.275+0000 7f033e59c640 1 --2- 172.21.15.172:0/4031952871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e21527d4a0 0x55e21527d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.275+0000 7f033e59c640 1 -- 172.21.15.172:0/4031952871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 msgr2=0x55e215288140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:52.296 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.275+0000 7f033e59c640 1 --2- 172.21.15.172:0/4031952871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e215288140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.275+0000 7f033e59c640 1 -- 172.21.15.172:0/4031952871 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e21516f440 con 0x55e215288680 2024-09-10T08:56:52.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.275+0000 7f033ed9d640 1 --2- 172.21.15.172:0/4031952871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e215288140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:56:52.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.275+0000 7f0327fff640 1 -- 172.21.15.172:0/1678142422 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f03300cee80 con 0x7f0340100e20 2024-09-10T08:56:52.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.275+0000 7f033e59c640 1 --2- 172.21.15.172:0/4031952871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e215288680 0x55e21528aa70 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f033400f540 tx=0x7f033400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82265dc7414a504 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:52.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.275+0000 7f03257fa640 1 -- 172.21.15.172:0/4031952871 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334005220 con 0x55e215288680 2024-09-10T08:56:52.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.275+0000 7f033f59e640 1 --2- 172.21.15.172:0/1678142422 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f03280a2b80 0x7f03280a5040 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0338005e30 tx=0x7f033802a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:52.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.275+0000 7f03257fa640 1 -- 172.21.15.172:0/4031952871 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0334004c50 con 0x55e215288680 2024-09-10T08:56:52.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.275+0000 7f03257fa640 1 -- 172.21.15.172:0/4031952871 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334010ba0 con 0x55e215288680 2024-09-10T08:56:52.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.276+0000 7f0347422740 1 -- 172.21.15.172:0/4031952871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e215288680 msgr2=0x55e21528aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:52.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.276+0000 7f0347422740 1 --2- 172.21.15.172:0/4031952871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e215288680 0x55e21528aa70 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f033400f540 tx=0x7f033400fa00 comp rx=0 tx=0).stop 2024-09-10T08:56:52.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.276+0000 7f0347422740 1 -- 172.21.15.172:0/4031952871 shutdown_connections 2024-09-10T08:56:52.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.276+0000 7f0347422740 1 --2- 172.21.15.172:0/4031952871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e215288680 0x55e21528aa70 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.276+0000 7f0347422740 1 --2- 172.21.15.172:0/4031952871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e215288140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.276+0000 7f0347422740 1 --2- 172.21.15.172:0/4031952871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e21527d4a0 0x55e21527d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.276+0000 7f0347422740 1 -- 172.21.15.172:0/4031952871 >> 172.21.15.172:0/4031952871 conn(0x55e215279a30 msgr2=0x55e215279e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:52.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.276+0000 7f0347422740 1 -- 172.21.15.172:0/4031952871 shutdown_connections 2024-09-10T08:56:52.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.276+0000 7f0347422740 1 -- 172.21.15.172:0/4031952871 wait complete. 2024-09-10T08:56:52.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f0347422740 1 -- start start 2024-09-10T08:56:52.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f0347422740 1 Processor -- start 2024-09-10T08:56:52.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f0347422740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e215287f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f0347422740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e215285240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f0347422740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215288680 0x55e215285780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f0347422740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e21528b250 con 0x55e21527d4a0 2024-09-10T08:56:52.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f0347422740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e21519e050 con 0x55e21527ddc0 2024-09-10T08:56:52.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f0347422740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e21528b3d0 con 0x55e215288680 2024-09-10T08:56:52.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f0347422740 10 client.0 _ll_register_callbacks cb 0x55e2151de160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:52.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f033ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e215287f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f033ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e215287f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41682/0 (socket says 172.21.15.172:41682) 2024-09-10T08:56:52.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f033ed9d640 1 -- 172.21.15.172:0/3984625210 learned_addr learned my addr 172.21.15.172:0/3984625210 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:52.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f033f59e640 1 --2- 172.21.15.172:0/3984625210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215288680 0x55e215285780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f033ed9d640 1 -- 172.21.15.172:0/3984625210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215288680 msgr2=0x55e215285780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:52.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f033e59c640 1 --2- 172.21.15.172:0/3984625210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e215285240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f033ed9d640 1 --2- 172.21.15.172:0/3984625210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215288680 0x55e215285780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f033ed9d640 1 -- 172.21.15.172:0/3984625210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 msgr2=0x55e215285240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:52.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f033ed9d640 1 --2- 172.21.15.172:0/3984625210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e215285240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f033ed9d640 1 -- 172.21.15.172:0/3984625210 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e214f543c0 con 0x55e21527d4a0 2024-09-10T08:56:52.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f033f59e640 1 --2- 172.21.15.172:0/3984625210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215288680 0x55e215285780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:56:52.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.277+0000 7f033e59c640 1 --2- 172.21.15.172:0/3984625210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e215285240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:56:52.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.278+0000 7f033ed9d640 1 --2- 172.21.15.172:0/3984625210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e215287f90 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f0330009610 tx=0x7f033006d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ceca2b533abb15a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:52.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.278+0000 7f0322ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:52.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.278+0000 7f0322ff5640 1 -- 172.21.15.172:0/3984625210 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330020040 con 0x55e21527d4a0 2024-09-10T08:56:52.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.278+0000 7f0322ff5640 1 -- 172.21.15.172:0/3984625210 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f033009a820 con 0x55e21527d4a0 2024-09-10T08:56:52.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.278+0000 7f0347422740 1 -- 172.21.15.172:0/3984625210 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0334004c50 con 0x55e21527d4a0 2024-09-10T08:56:52.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.278+0000 7f0322ff5640 1 -- 172.21.15.172:0/3984625210 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f033009b3d0 con 0x55e21527d4a0 2024-09-10T08:56:52.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.278+0000 7f0347422740 10 client.18594 fetch_fsmap learned FSMap version 39 2024-09-10T08:56:52.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.278+0000 7f0347422740 1 -- 172.21.15.172:0/3984625210 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e215296bd0 con 0x55e21527d4a0 2024-09-10T08:56:52.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.278+0000 7f0322ff5640 1 -- 172.21.15.172:0/3984625210 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0334004c50 con 0x55e21527d4a0 2024-09-10T08:56:52.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.279+0000 7f0322ff5640 1 -- 172.21.15.172:0/3984625210 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f033006c070 con 0x55e21527d4a0 2024-09-10T08:56:52.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.279+0000 7f0347422740 10 client.18594 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:56:52.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.279+0000 7f0347422740 10 client.18594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:52.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.279+0000 7f0347422740 20 client.18594 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:52.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.279+0000 7f0347422740 1 --2- 172.21.15.172:0/3984625210 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e215298090 0x55e21529a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.279+0000 7f0347422740 4 client.18594 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:56:52.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.279+0000 7f0347422740 1 -- 172.21.15.172:0/3984625210 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2151f0850 con 0x55e215298090 2024-09-10T08:56:52.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.279+0000 7f033e59c640 1 --2- 172.21.15.172:0/3984625210 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e215298090 0x55e21529a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.279+0000 7f033e59c640 1 --2- 172.21.15.172:0/3984625210 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e215298090 0x55e21529a520 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d0759fb672d7db1 server_cookie=4c1fcce0c7096f13 in_seq=0 out_seq=0 2024-09-10T08:56:52.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.279+0000 7f0322ff5640 10 client.18594 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:52.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.280+0000 7f0322ff5640 1 -- 172.21.15.172:0/3984625210 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e2151f0850 con 0x55e215298090 2024-09-10T08:56:52.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.280+0000 7f0322ff5640 10 client.18594 handle_command_reply: tid=0 2024-09-10T08:56:52.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.280+0000 7f0347422740 1 client.18594 shutdown 2024-09-10T08:56:52.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.280+0000 7f0347422740 1 -- 172.21.15.172:0/3984625210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 msgr2=0x55e215287f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:52.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.280+0000 7f0347422740 1 --2- 172.21.15.172:0/3984625210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e215287f90 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f0330009610 tx=0x7f033006d3e0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.280+0000 7f0347422740 1 -- 172.21.15.172:0/3984625210 shutdown_connections 2024-09-10T08:56:52.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.280+0000 7f0347422740 1 --2- 172.21.15.172:0/3984625210 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e215298090 0x55e21529a520 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.280+0000 7f0347422740 1 --2- 172.21.15.172:0/3984625210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215288680 0x55e215285780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.280+0000 7f0347422740 1 --2- 172.21.15.172:0/3984625210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e215285240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.280+0000 7f0347422740 1 --2- 172.21.15.172:0/3984625210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e215287f90 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.280+0000 7f0347422740 1 -- 172.21.15.172:0/3984625210 >> 172.21.15.172:0/3984625210 conn(0x55e215279a30 msgr2=0x55e21527f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:52.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.280+0000 7f0322ff5640 0 client.18594 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:52.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.280+0000 7f0347422740 1 -- 172.21.15.172:0/3984625210 shutdown_connections 2024-09-10T08:56:52.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.280+0000 7f0347422740 1 -- 172.21.15.172:0/3984625210 wait complete. 2024-09-10T08:56:52.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.280+0000 7f0347422740 20 client.18594 trim_cache size 0 max 16384 2024-09-10T08:56:52.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.305+0000 7f0347422740 1 Processor -- start 2024-09-10T08:56:52.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.305+0000 7f0347422740 1 -- start start 2024-09-10T08:56:52.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.306+0000 7f0347422740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215279a30 0x55e2152944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.306+0000 7f0347422740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e215285200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.306+0000 7f0347422740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e21529fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.306+0000 7f0347422740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f03401b0cb0 con 0x55e21527d4a0 2024-09-10T08:56:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.306+0000 7f0347422740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f03401b2850 con 0x55e21527ddc0 2024-09-10T08:56:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.306+0000 7f0347422740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f03401b1380 con 0x55e215279a30 2024-09-10T08:56:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.306+0000 7f033e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e215285200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.306+0000 7f033e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e215285200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41692/0 (socket says 172.21.15.172:41692) 2024-09-10T08:56:52.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.306+0000 7f033e59c640 1 -- 172.21.15.172:0/3304583872 learned_addr learned my addr 172.21.15.172:0/3304583872 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:52.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.306+0000 7f033ed9d640 1 --2- 172.21.15.172:0/3304583872 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215279a30 0x55e2152944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.306+0000 7f033f59e640 1 --2- 172.21.15.172:0/3304583872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e21529fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.306+0000 7f033e59c640 1 -- 172.21.15.172:0/3304583872 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215279a30 msgr2=0x55e2152944b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:52.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.306+0000 7f033e59c640 1 --2- 172.21.15.172:0/3304583872 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215279a30 0x55e2152944b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.306+0000 7f033e59c640 1 -- 172.21.15.172:0/3304583872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 msgr2=0x55e21529fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:52.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.306+0000 7f033e59c640 1 --2- 172.21.15.172:0/3304583872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e21529fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.306+0000 7f033e59c640 1 -- 172.21.15.172:0/3304583872 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2152794d0 con 0x55e21527d4a0 2024-09-10T08:56:52.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.306+0000 7f033e59c640 1 --2- 172.21.15.172:0/3304583872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e215285200 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f033401a040 tx=0x7f033400f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788be1616cc5ce94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:52.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.307+0000 7f0324ff9640 1 -- 172.21.15.172:0/3304583872 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334010760 con 0x55e21527d4a0 2024-09-10T08:56:52.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.307+0000 7f0324ff9640 1 -- 172.21.15.172:0/3304583872 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f033402d030 con 0x55e21527d4a0 2024-09-10T08:56:52.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.307+0000 7f0324ff9640 1 -- 172.21.15.172:0/3304583872 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03340189f0 con 0x55e21527d4a0 2024-09-10T08:56:52.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.308+0000 7f0347422740 1 -- 172.21.15.172:0/3304583872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 msgr2=0x55e215285200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:52.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.308+0000 7f0347422740 1 --2- 172.21.15.172:0/3304583872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e215285200 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f033401a040 tx=0x7f033400f470 comp rx=0 tx=0).stop 2024-09-10T08:56:52.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.308+0000 7f0347422740 1 -- 172.21.15.172:0/3304583872 shutdown_connections 2024-09-10T08:56:52.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.308+0000 7f0347422740 1 --2- 172.21.15.172:0/3304583872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e21529fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.308+0000 7f0347422740 1 --2- 172.21.15.172:0/3304583872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e215285200 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.308+0000 7f0347422740 1 --2- 172.21.15.172:0/3304583872 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215279a30 0x55e2152944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.308+0000 7f0347422740 1 -- 172.21.15.172:0/3304583872 >> 172.21.15.172:0/3304583872 conn(0x55e2152858e0 msgr2=0x55e21527a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:52.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.308+0000 7f0347422740 1 -- 172.21.15.172:0/3304583872 shutdown_connections 2024-09-10T08:56:52.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.308+0000 7f0347422740 1 -- 172.21.15.172:0/3304583872 wait complete. 2024-09-10T08:56:52.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.308+0000 7f0347422740 1 -- start start 2024-09-10T08:56:52.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.308+0000 7f0347422740 1 Processor -- start 2024-09-10T08:56:52.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f0347422740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215279a30 0x55e21528bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f0347422740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e21528c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f0347422740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e215290440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f0347422740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f03401b2180 con 0x55e21527d4a0 2024-09-10T08:56:52.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f0347422740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f03401b1ab0 con 0x55e21527ddc0 2024-09-10T08:56:52.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f0347422740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f03401aec90 con 0x55e215279a30 2024-09-10T08:56:52.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f033e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e21528c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f033ed9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215279a30 0x55e21528bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f0347422740 10 client.0 _ll_register_callbacks cb 0x7f0330009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:56:52.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f033e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e21528c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41696/0 (socket says 172.21.15.172:41696) 2024-09-10T08:56:52.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f033e59c640 1 -- 172.21.15.172:0/2472141251 learned_addr learned my addr 172.21.15.172:0/2472141251 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:52.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f033f59e640 1 --2- 172.21.15.172:0/2472141251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e215290440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f033ed9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215279a30 0x55e21528bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:53584/0 (socket says 172.21.15.172:53584) 2024-09-10T08:56:52.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f033e59c640 1 -- 172.21.15.172:0/2472141251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215279a30 msgr2=0x55e21528bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:52.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f033e59c640 1 --2- 172.21.15.172:0/2472141251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215279a30 0x55e21528bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f033e59c640 1 -- 172.21.15.172:0/2472141251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 msgr2=0x55e215290440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:56:52.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f033e59c640 1 --2- 172.21.15.172:0/2472141251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e215290440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f033e59c640 1 -- 172.21.15.172:0/2472141251 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e215290b30 con 0x55e21527d4a0 2024-09-10T08:56:52.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.309+0000 7f033ed9d640 1 --2- 172.21.15.172:0/2472141251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215279a30 0x55e21528bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:56:52.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.310+0000 7f033e59c640 1 --2- 172.21.15.172:0/2472141251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e21528c300 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f033402c040 tx=0x7f033400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf44c002b85adce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:52.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.310+0000 7f0322ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:56:52.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.310+0000 7f0322ff5640 1 -- 172.21.15.172:0/2472141251 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334018770 con 0x55e21527d4a0 2024-09-10T08:56:52.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.310+0000 7f0322ff5640 1 -- 172.21.15.172:0/2472141251 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0334018910 con 0x55e21527d4a0 2024-09-10T08:56:52.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.310+0000 7f0347422740 1 -- 172.21.15.172:0/2472141251 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f033402d030 con 0x55e21527d4a0 2024-09-10T08:56:52.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.310+0000 7f0322ff5640 1 -- 172.21.15.172:0/2472141251 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0334018b10 con 0x55e21527d4a0 2024-09-10T08:56:52.315 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.310+0000 7f0347422740 10 client.18600 fetch_fsmap learned FSMap version 39 2024-09-10T08:56:52.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.310+0000 7f0347422740 1 -- 172.21.15.172:0/2472141251 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e2153029c0 con 0x55e21527d4a0 2024-09-10T08:56:52.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.310+0000 7f0322ff5640 1 -- 172.21.15.172:0/2472141251 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f033402d030 con 0x55e21527d4a0 2024-09-10T08:56:52.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.310+0000 7f0322ff5640 1 -- 172.21.15.172:0/2472141251 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f0334043510 con 0x55e21527d4a0 2024-09-10T08:56:52.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.311+0000 7f0347422740 10 client.18600 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:56:52.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.311+0000 7f0347422740 10 client.18600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:56:52.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.311+0000 7f0347422740 20 client.18600 populate_metadata read hostname 'smithi172' 2024-09-10T08:56:52.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.311+0000 7f0347422740 1 --2- 172.21.15.172:0/2472141251 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e215288680 0x55e215304e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:52.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.311+0000 7f0347422740 4 client.18600 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:56:52.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.311+0000 7f0347422740 1 -- 172.21.15.172:0/2472141251 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e2151ead60 con 0x55e215288680 2024-09-10T08:56:52.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.311+0000 7f033ed9d640 1 --2- 172.21.15.172:0/2472141251 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e215288680 0x55e215304e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:52.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.311+0000 7f033ed9d640 1 --2- 172.21.15.172:0/2472141251 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e215288680 0x55e215304e40 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10d68123a216d1af server_cookie=440b6fe90593e0bf in_seq=0 out_seq=0 2024-09-10T08:56:52.316 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.311+0000 7f0322ff5640 10 client.18600 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:52.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.311+0000 7f0322ff5640 1 -- 172.21.15.172:0/2472141251 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e2151ead60 con 0x55e215288680 2024-09-10T08:56:52.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.311+0000 7f0322ff5640 10 client.18600 handle_command_reply: tid=0 2024-09-10T08:56:52.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.311+0000 7f0347422740 1 client.18600 shutdown 2024-09-10T08:56:52.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.311+0000 7f0347422740 1 -- 172.21.15.172:0/2472141251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 msgr2=0x55e21528c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:52.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.311+0000 7f0347422740 1 --2- 172.21.15.172:0/2472141251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e21528c300 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f033402c040 tx=0x7f033400fa00 comp rx=0 tx=0).stop 2024-09-10T08:56:52.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0347422740 1 -- 172.21.15.172:0/2472141251 shutdown_connections 2024-09-10T08:56:52.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0347422740 1 --2- 172.21.15.172:0/2472141251 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e215288680 0x55e215304e40 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0347422740 1 --2- 172.21.15.172:0/2472141251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e21527ddc0 0x55e215290440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0347422740 1 --2- 172.21.15.172:0/2472141251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e21527d4a0 0x55e21528c300 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0347422740 1 --2- 172.21.15.172:0/2472141251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e215279a30 0x55e21528bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0347422740 1 -- 172.21.15.172:0/2472141251 >> 172.21.15.172:0/2472141251 conn(0x55e2152858e0 msgr2=0x55e21527ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:52.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0322ff5640 0 client.18600 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:56:52.317 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0347422740 1 -- 172.21.15.172:0/2472141251 shutdown_connections 2024-09-10T08:56:52.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0347422740 1 -- 172.21.15.172:0/2472141251 wait complete. 2024-09-10T08:56:52.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0347422740 20 client.18600 trim_cache size 0 max 16384 2024-09-10T08:56:52.318 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:56:52.318 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T08:56:52.318 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T08:56:52.318 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:56:52.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0345049640 1 -- 172.21.15.172:0/1678142422 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f03280a2b80 msgr2=0x7f03280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:52.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0345049640 1 --2- 172.21.15.172:0/1678142422 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f03280a2b80 0x7f03280a5040 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0338005e30 tx=0x7f033802a950 comp rx=0 tx=0).stop 2024-09-10T08:56:52.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0345049640 1 -- 172.21.15.172:0/1678142422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0340100e20 msgr2=0x7f034019a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:52.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0345049640 1 --2- 172.21.15.172:0/1678142422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0340100e20 0x7f034019a500 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f033000d6d0 tx=0x7f033000db90 comp rx=0 tx=0).stop 2024-09-10T08:56:52.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0345049640 1 -- 172.21.15.172:0/1678142422 shutdown_connections 2024-09-10T08:56:52.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0345049640 1 --2- 172.21.15.172:0/1678142422 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f03280a2b80 0x7f03280a5040 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0345049640 1 --2- 172.21.15.172:0/1678142422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0340102ef0 0x7f034019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0345049640 1 --2- 172.21.15.172:0/1678142422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0340102570 0x7f034019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0345049640 1 --2- 172.21.15.172:0/1678142422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0340100e20 0x7f034019a500 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:52.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.312+0000 7f0345049640 1 -- 172.21.15.172:0/1678142422 >> 172.21.15.172:0/1678142422 conn(0x7f03400fc7d0 msgr2=0x7f034006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:52.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.313+0000 7f0345049640 1 -- 172.21.15.172:0/1678142422 shutdown_connections 2024-09-10T08:56:52.320 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:52.313+0000 7f0345049640 1 -- 172.21.15.172:0/1678142422 wait complete. 2024-09-10T08:56:52.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:52 smithi172 ceph-mon[30854]: pgmap v669: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 16 op/s 2024-09-10T08:56:52.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:52 smithi174 ceph-mon[34068]: pgmap v669: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 16 op/s 2024-09-10T08:56:52.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:52 smithi191 ceph-mon[35314]: pgmap v669: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.1 MiB/s wr, 16 op/s 2024-09-10T08:56:52.990 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:56:52.990 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:56:53.282 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:56:53.693 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:53 smithi172 ceph-mon[30854]: pgmap v670: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-09-10T08:56:53.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:53 smithi174 ceph-mon[34068]: pgmap v670: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-09-10T08:56:53.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:53 smithi191 ceph-mon[35314]: pgmap v670: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.3 MiB/s wr, 12 op/s 2024-09-10T08:56:53.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.784+0000 7f22662bf640 1 -- 172.21.15.172:0/1165427509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2260102ef0 msgr2=0x7f226010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:53.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.784+0000 7f22662bf640 1 --2- 172.21.15.172:0/1165427509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2260102ef0 0x7f226010da40 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f2258009ca0 tx=0x7f225802a950 comp rx=0 tx=0).stop 2024-09-10T08:56:53.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.785+0000 7f22662bf640 1 -- 172.21.15.172:0/1165427509 shutdown_connections 2024-09-10T08:56:53.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.785+0000 7f22662bf640 1 --2- 172.21.15.172:0/1165427509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2260102ef0 0x7f226010da40 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:53.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.785+0000 7f22662bf640 1 --2- 172.21.15.172:0/1165427509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2260102570 0x7f22601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:53.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.785+0000 7f22662bf640 1 --2- 172.21.15.172:0/1165427509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2260100e20 0x7f2260101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:53.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.785+0000 7f22662bf640 1 -- 172.21.15.172:0/1165427509 >> 172.21.15.172:0/1165427509 conn(0x7f22600fc7d0 msgr2=0x7f22600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:53.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.785+0000 7f22662bf640 1 -- 172.21.15.172:0/1165427509 shutdown_connections 2024-09-10T08:56:53.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.785+0000 7f22662bf640 1 -- 172.21.15.172:0/1165427509 wait complete. 2024-09-10T08:56:53.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.785+0000 7f22662bf640 1 Processor -- start 2024-09-10T08:56:53.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.785+0000 7f22662bf640 1 -- start start 2024-09-10T08:56:53.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.786+0000 7f22662bf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2260100e20 0x7f22601983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:53.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.786+0000 7f22662bf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2260102570 0x7f2260198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:53.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.786+0000 7f22662bf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2260102ef0 0x7f226019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:53.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.786+0000 7f22662bf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2260110e50 con 0x7f2260102ef0 2024-09-10T08:56:53.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.786+0000 7f22662bf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2260110cd0 con 0x7f2260100e20 2024-09-10T08:56:53.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.786+0000 7f22662bf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2260110fd0 con 0x7f2260102570 2024-09-10T08:56:53.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.786+0000 7f226485c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2260102ef0 0x7f226019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:53.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.786+0000 7f225f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2260102570 0x7f2260198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:53.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.786+0000 7f226485c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2260102ef0 0x7f226019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41720/0 (socket says 172.21.15.172:41720) 2024-09-10T08:56:53.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.786+0000 7f226485c640 1 -- 172.21.15.172:0/1979312802 learned_addr learned my addr 172.21.15.172:0/1979312802 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:56:53.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.786+0000 7f225ffff640 1 --2- 172.21.15.172:0/1979312802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2260100e20 0x7f22601983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:53.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.786+0000 7f226485c640 1 -- 172.21.15.172:0/1979312802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2260102570 msgr2=0x7f2260198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:53.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.786+0000 7f226485c640 1 --2- 172.21.15.172:0/1979312802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2260102570 0x7f2260198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:53.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.786+0000 7f226485c640 1 -- 172.21.15.172:0/1979312802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2260100e20 msgr2=0x7f22601983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:53.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.786+0000 7f226485c640 1 --2- 172.21.15.172:0/1979312802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2260100e20 0x7f22601983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:53.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.786+0000 7f226485c640 1 -- 172.21.15.172:0/1979312802 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226019d390 con 0x7f2260102ef0 2024-09-10T08:56:53.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.787+0000 7f226485c640 1 --2- 172.21.15.172:0/1979312802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2260102ef0 0x7f226019ccb0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f2258006010 tx=0x7f225802f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:53.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.787+0000 7f225d7fa640 1 -- 172.21.15.172:0/1979312802 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258034040 con 0x7f2260102ef0 2024-09-10T08:56:53.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.787+0000 7f22662bf640 1 -- 172.21.15.172:0/1979312802 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f226019d620 con 0x7f2260102ef0 2024-09-10T08:56:53.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.787+0000 7f225d7fa640 1 -- 172.21.15.172:0/1979312802 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2258003e00 con 0x7f2260102ef0 2024-09-10T08:56:53.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.787+0000 7f22662bf640 1 -- 172.21.15.172:0/1979312802 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22600fcbb0 con 0x7f2260102ef0 2024-09-10T08:56:53.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.787+0000 7f225d7fa640 1 -- 172.21.15.172:0/1979312802 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258045480 con 0x7f2260102ef0 2024-09-10T08:56:53.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.788+0000 7f22662bf640 1 -- 172.21.15.172:0/1979312802 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2260101200 con 0x7f2260102ef0 2024-09-10T08:56:53.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.789+0000 7f225d7fa640 1 -- 172.21.15.172:0/1979312802 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2258037020 con 0x7f2260102ef0 2024-09-10T08:56:53.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.790+0000 7f225d7fa640 1 --2- 172.21.15.172:0/1979312802 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22480a2b80 0x7f22480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:56:53.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.790+0000 7f225ffff640 1 --2- 172.21.15.172:0/1979312802 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22480a2b80 0x7f22480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:56:53.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.790+0000 7f225d7fa640 1 -- 172.21.15.172:0/1979312802 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f22580f0520 con 0x7f2260102ef0 2024-09-10T08:56:53.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.791+0000 7f225ffff640 1 --2- 172.21.15.172:0/1979312802 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22480a2b80 0x7f22480a5040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f22500058c0 tx=0x7f225000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:56:53.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:53.794+0000 7f225d7fa640 1 -- 172.21.15.172:0/1979312802 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f22580bd1b0 con 0x7f2260102ef0 2024-09-10T08:56:54.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:54.044+0000 7f22662bf640 1 -- 172.21.15.172:0/1979312802 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2260199740 con 0x7f2260102ef0 2024-09-10T08:56:54.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:54.045+0000 7f225d7fa640 1 -- 172.21.15.172:0/1979312802 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+12004 (secure 0 0 0) 0x7f2260199740 con 0x7f2260102ef0 2024-09-10T08:56:54.046 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:56:54.047 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":39,"btime":"2024-09-10T08:56:43:360461+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:56:42.665159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:56:54.047 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 39 2024-09-10T08:56:54.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:54.048+0000 7f22662bf640 1 -- 172.21.15.172:0/1979312802 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22480a2b80 msgr2=0x7f22480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:54.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:54.048+0000 7f22662bf640 1 --2- 172.21.15.172:0/1979312802 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22480a2b80 0x7f22480a5040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f22500058c0 tx=0x7f225000a2d0 comp rx=0 tx=0).stop 2024-09-10T08:56:54.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:54.048+0000 7f22662bf640 1 -- 172.21.15.172:0/1979312802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2260102ef0 msgr2=0x7f226019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:56:54.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:54.048+0000 7f22662bf640 1 --2- 172.21.15.172:0/1979312802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2260102ef0 0x7f226019ccb0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f2258006010 tx=0x7f225802f710 comp rx=0 tx=0).stop 2024-09-10T08:56:54.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:54.048+0000 7f22662bf640 1 -- 172.21.15.172:0/1979312802 shutdown_connections 2024-09-10T08:56:54.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:54.048+0000 7f22662bf640 1 --2- 172.21.15.172:0/1979312802 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f22480a2b80 0x7f22480a5040 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:54.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:54.048+0000 7f22662bf640 1 --2- 172.21.15.172:0/1979312802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2260102ef0 0x7f226019ccb0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:54.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:54.048+0000 7f22662bf640 1 --2- 172.21.15.172:0/1979312802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2260102570 0x7f2260198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:54.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:54.048+0000 7f22662bf640 1 --2- 172.21.15.172:0/1979312802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2260100e20 0x7f22601983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:56:54.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:54.048+0000 7f22662bf640 1 -- 172.21.15.172:0/1979312802 >> 172.21.15.172:0/1979312802 conn(0x7f22600fc7d0 msgr2=0x7f226006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:56:54.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:54.048+0000 7f22662bf640 1 -- 172.21.15.172:0/1979312802 shutdown_connections 2024-09-10T08:56:54.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:56:54.048+0000 7f22662bf640 1 -- 172.21.15.172:0/1979312802 wait complete. 2024-09-10T08:56:54.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:54 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1979312802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:56:54.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:54 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1979312802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:56:54.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:54 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1979312802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:56:55.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:55 smithi174 ceph-mon[34068]: pgmap v671: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-09-10T08:56:55.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:55 smithi191 ceph-mon[35314]: pgmap v671: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-09-10T08:56:55.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:55 smithi172 ceph-mon[30854]: pgmap v671: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-09-10T08:56:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:57 smithi172 ceph-mon[30854]: pgmap v672: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-09-10T08:56:57.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:57 smithi174 ceph-mon[34068]: pgmap v672: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-09-10T08:56:57.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:57 smithi191 ceph-mon[35314]: pgmap v672: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 14 MiB/s wr, 16 op/s 2024-09-10T08:56:59.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:56:59 smithi174 ceph-mon[34068]: pgmap v673: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 19 op/s 2024-09-10T08:56:59.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:56:59 smithi191 ceph-mon[35314]: pgmap v673: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 19 op/s 2024-09-10T08:56:59.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:56:59 smithi172 ceph-mon[30854]: pgmap v673: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 MiB/s wr, 19 op/s 2024-09-10T08:57:01.177 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T08:57:01.486 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:57:01.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:01 smithi174 ceph-mon[34068]: pgmap v674: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 21 op/s 2024-09-10T08:57:01.748 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:01 smithi172 ceph-mon[30854]: pgmap v674: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 21 op/s 2024-09-10T08:57:01.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:01 smithi191 ceph-mon[35314]: pgmap v674: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 21 op/s 2024-09-10T08:57:02.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.977+0000 7f6588ce2640 1 -- 172.21.15.172:0/2673631396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6584102d30 msgr2=0x7f6584103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:02.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.977+0000 7f6588ce2640 1 --2- 172.21.15.172:0/2673631396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6584102d30 0x7f6584103110 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f6574009930 tx=0x7f657402a420 comp rx=0 tx=0).stop 2024-09-10T08:57:02.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.977+0000 7f6588ce2640 1 -- 172.21.15.172:0/2673631396 shutdown_connections 2024-09-10T08:57:02.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.977+0000 7f6588ce2640 1 --2- 172.21.15.172:0/2673631396 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65841082a0 0x7f658410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.977+0000 7f6588ce2640 1 --2- 172.21.15.172:0/2673631396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65841036e0 0x7f6584107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.977+0000 7f6588ce2640 1 --2- 172.21.15.172:0/2673631396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6584102d30 0x7f6584103110 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.977+0000 7f6588ce2640 1 -- 172.21.15.172:0/2673631396 >> 172.21.15.172:0/2673631396 conn(0x7f65840fc7d0 msgr2=0x7f65840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:02.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.977+0000 7f6588ce2640 1 -- 172.21.15.172:0/2673631396 shutdown_connections 2024-09-10T08:57:02.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.977+0000 7f6588ce2640 1 -- 172.21.15.172:0/2673631396 wait complete. 2024-09-10T08:57:02.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.978+0000 7f6588ce2640 1 Processor -- start 2024-09-10T08:57:02.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.978+0000 7f6588ce2640 1 -- start start 2024-09-10T08:57:02.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.978+0000 7f6588ce2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6584102d30 0x7f6584100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.978+0000 7f6588ce2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65841036e0 0x7f6584100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.978+0000 7f6588ce2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65841082a0 0x7f6584101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.978+0000 7f6588ce2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6584111f50 con 0x7f65841082a0 2024-09-10T08:57:02.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.978+0000 7f6588ce2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6584111dd0 con 0x7f6584102d30 2024-09-10T08:57:02.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.978+0000 7f6588ce2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f65841120d0 con 0x7f65841036e0 2024-09-10T08:57:02.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.978+0000 7f6582d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65841082a0 0x7f6584101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.978+0000 7f6582d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65841082a0 0x7f6584101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49280/0 (socket says 172.21.15.172:49280) 2024-09-10T08:57:02.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.978+0000 7f658259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6584102d30 0x7f6584100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.978+0000 7f6582d9d640 1 -- 172.21.15.172:0/2444954177 learned_addr learned my addr 172.21.15.172:0/2444954177 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:02.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.978+0000 7f657bfff640 1 --2- 172.21.15.172:0/2444954177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65841036e0 0x7f6584100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.979+0000 7f6582d9d640 1 -- 172.21.15.172:0/2444954177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65841036e0 msgr2=0x7f6584100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:02.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.979+0000 7f6582d9d640 1 --2- 172.21.15.172:0/2444954177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65841036e0 0x7f6584100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.979+0000 7f6582d9d640 1 -- 172.21.15.172:0/2444954177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6584102d30 msgr2=0x7f6584100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:02.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.979+0000 7f6582d9d640 1 --2- 172.21.15.172:0/2444954177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6584102d30 0x7f6584100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.979+0000 7f6582d9d640 1 -- 172.21.15.172:0/2444954177 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65841a69d0 con 0x7f65841082a0 2024-09-10T08:57:02.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.979+0000 7f658259c640 1 --2- 172.21.15.172:0/2444954177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6584102d30 0x7f6584100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:02.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.979+0000 7f6582d9d640 1 --2- 172.21.15.172:0/2444954177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65841082a0 0x7f6584101410 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f657c00cad0 tx=0x7f657c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:02.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.979+0000 7f657b7fe640 1 -- 172.21.15.172:0/2444954177 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c00ae20 con 0x7f65841082a0 2024-09-10T08:57:02.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.979+0000 7f6588ce2640 1 -- 172.21.15.172:0/2444954177 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65841a6cc0 con 0x7f65841082a0 2024-09-10T08:57:02.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.979+0000 7f6588ce2640 1 -- 172.21.15.172:0/2444954177 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65841a7200 con 0x7f65841082a0 2024-09-10T08:57:02.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.979+0000 7f657b7fe640 1 -- 172.21.15.172:0/2444954177 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f657c005020 con 0x7f65841082a0 2024-09-10T08:57:02.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.979+0000 7f657b7fe640 1 -- 172.21.15.172:0/2444954177 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c00f6b0 con 0x7f65841082a0 2024-09-10T08:57:02.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.982+0000 7f658b0bb740 1 Processor -- start 2024-09-10T08:57:02.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.982+0000 7f657b7fe640 1 -- 172.21.15.172:0/2444954177 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f657c004a40 con 0x7f65841082a0 2024-09-10T08:57:02.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.983+0000 7f658b0bb740 1 -- start start 2024-09-10T08:57:02.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.983+0000 7f658b0bb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c90b40 0x55b1a0c8f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.983+0000 7f658b0bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0c8fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.983+0000 7f658b0bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c9a370 0x55b1a0c9c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.983+0000 7f658b0bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b1a0bf52f0 con 0x55b1a0c9a370 2024-09-10T08:57:02.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.983+0000 7f658b0bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b1a0b9ccf0 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.983+0000 7f658b0bb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b1a0bf12b0 con 0x55b1a0c90b40 2024-09-10T08:57:02.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.983+0000 7f657bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c9a370 0x55b1a0c9c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.983+0000 7f658259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0c8fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.983+0000 7f658259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0c8fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:50606/0 (socket says 172.21.15.172:50606) 2024-09-10T08:57:02.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.983+0000 7f657bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c9a370 0x55b1a0c9c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49296/0 (socket says 172.21.15.172:49296) 2024-09-10T08:57:02.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.983+0000 7f658259c640 1 -- 172.21.15.172:0/2426756545 learned_addr learned my addr 172.21.15.172:0/2426756545 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:02.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.983+0000 7f657bfff640 1 --2- 172.21.15.172:0/2426756545 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c90b40 0x55b1a0c8f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.984+0000 7f658259c640 1 -- 172.21.15.172:0/2426756545 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c90b40 msgr2=0x55b1a0c8f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:02.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.984+0000 7f658259c640 1 --2- 172.21.15.172:0/2426756545 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c90b40 0x55b1a0c8f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.984+0000 7f658259c640 1 -- 172.21.15.172:0/2426756545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c9a370 msgr2=0x55b1a0c9c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:02.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.984+0000 7f658259c640 1 --2- 172.21.15.172:0/2426756545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c9a370 0x55b1a0c9c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.984+0000 7f658259c640 1 -- 172.21.15.172:0/2426756545 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a0b813a0 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.984+0000 7f657b7fe640 1 --2- 172.21.15.172:0/2444954177 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f65700a2c00 0x7f65700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.984+0000 7f658259c640 1 --2- 172.21.15.172:0/2426756545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0c8fa10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f657402a940 tx=0x7f6574031550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1d08d3c3e91dea3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:02.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.984+0000 7f657b7fe640 1 -- 172.21.15.172:0/2444954177 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f657c0cf1d0 con 0x7f65841082a0 2024-09-10T08:57:02.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.984+0000 7f657bfff640 1 --2- 172.21.15.172:0/2444954177 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f65700a2c00 0x7f65700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.984+0000 7f6578ff9640 1 -- 172.21.15.172:0/2426756545 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574035040 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.984+0000 7f6578ff9640 1 -- 172.21.15.172:0/2426756545 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6574005020 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.984+0000 7f6578ff9640 1 -- 172.21.15.172:0/2426756545 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574004a90 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.985+0000 7f657bfff640 1 --2- 172.21.15.172:0/2444954177 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f65700a2c00 0x7f65700a50c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x55b1a0c90070 tx=0x7f656c005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:02.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.985+0000 7f658b0bb740 1 -- 172.21.15.172:0/2426756545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c8f5d0 msgr2=0x55b1a0c8fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:02.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.985+0000 7f658b0bb740 1 --2- 172.21.15.172:0/2426756545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0c8fa10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f657402a940 tx=0x7f6574031550 comp rx=0 tx=0).stop 2024-09-10T08:57:02.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.985+0000 7f658b0bb740 1 -- 172.21.15.172:0/2426756545 shutdown_connections 2024-09-10T08:57:02.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.985+0000 7f658b0bb740 1 --2- 172.21.15.172:0/2426756545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c9a370 0x55b1a0c9c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.985+0000 7f658b0bb740 1 --2- 172.21.15.172:0/2426756545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0c8fa10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.985+0000 7f658b0bb740 1 --2- 172.21.15.172:0/2426756545 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c90b40 0x55b1a0c8f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.985+0000 7f658b0bb740 1 -- 172.21.15.172:0/2426756545 >> 172.21.15.172:0/2426756545 conn(0x55b1a0c58310 msgr2=0x55b1a0c8bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:02.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.985+0000 7f658b0bb740 1 -- 172.21.15.172:0/2426756545 shutdown_connections 2024-09-10T08:57:02.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.986+0000 7f658b0bb740 1 -- 172.21.15.172:0/2426756545 wait complete. 2024-09-10T08:57:02.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.986+0000 7f658b0bb740 1 -- start start 2024-09-10T08:57:02.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.986+0000 7f658b0bb740 1 Processor -- start 2024-09-10T08:57:02.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.987+0000 7f658b0bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0c99ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.987+0000 7f658b0bb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c90b40 0x55b1a0c96f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.987+0000 7f658b0bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c9a370 0x55b1a0c97490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.987+0000 7f658b0bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b1a0c9d010 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.987+0000 7f658b0bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b1a0bb0060 con 0x55b1a0c9a370 2024-09-10T08:57:02.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.987+0000 7f658b0bb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b1a0c9d190 con 0x55b1a0c90b40 2024-09-10T08:57:02.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.987+0000 7f658259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0c99ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.987+0000 7f658b0bb740 10 client.0 _ll_register_callbacks cb 0x55b1a09038c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:02.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.987+0000 7f658259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0c99ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49298/0 (socket says 172.21.15.172:49298) 2024-09-10T08:57:02.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.987+0000 7f658259c640 1 -- 172.21.15.172:0/1636322652 learned_addr learned my addr 172.21.15.172:0/1636322652 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:02.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.987+0000 7f657bfff640 1 --2- 172.21.15.172:0/1636322652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c90b40 0x55b1a0c96f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.987+0000 7f6582d9d640 1 --2- 172.21.15.172:0/1636322652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c9a370 0x55b1a0c97490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.987+0000 7f658259c640 1 -- 172.21.15.172:0/1636322652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c90b40 msgr2=0x55b1a0c96f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:02.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.987+0000 7f658259c640 1 --2- 172.21.15.172:0/1636322652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c90b40 0x55b1a0c96f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.987+0000 7f658259c640 1 -- 172.21.15.172:0/1636322652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c9a370 msgr2=0x55b1a0c97490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:02.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.987+0000 7f658259c640 1 --2- 172.21.15.172:0/1636322652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c9a370 0x55b1a0c97490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.987+0000 7f658259c640 1 -- 172.21.15.172:0/1636322652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a09663c0 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.988+0000 7f6582d9d640 1 --2- 172.21.15.172:0/1636322652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c9a370 0x55b1a0c97490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:57:02.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.988+0000 7f658259c640 1 --2- 172.21.15.172:0/1636322652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0c99ca0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f6574004770 tx=0x7f6574031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169b5ca409510c3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:02.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.988+0000 7f6569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:02.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.988+0000 7f6569ffb640 1 -- 172.21.15.172:0/1636322652 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657404f060 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.988+0000 7f6569ffb640 1 -- 172.21.15.172:0/1636322652 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6574004030 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.988+0000 7f658b0bb740 1 -- 172.21.15.172:0/1636322652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6574005020 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.988+0000 7f6569ffb640 1 -- 172.21.15.172:0/1636322652 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6574056050 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.988+0000 7f658b0bb740 10 client.18618 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:02.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.988+0000 7f658b0bb740 1 -- 172.21.15.172:0/1636322652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b1a0ca8a90 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.989+0000 7f6569ffb640 1 -- 172.21.15.172:0/1636322652 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6574005020 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.989+0000 7f6569ffb640 1 -- 172.21.15.172:0/1636322652 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f6574003760 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.989+0000 7f658b0bb740 10 client.18618 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:02.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.989+0000 7f658b0bb740 10 client.18618 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:57:02.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.989+0000 7f658b0bb740 20 client.18618 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:02.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.989+0000 7f658b0bb740 1 --2- 172.21.15.172:0/1636322652 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55b1a0caa0c0 0x55b1a0cac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.989+0000 7f658b0bb740 4 client.18618 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:57:02.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.989+0000 7f658b0bb740 1 -- 172.21.15.172:0/1636322652 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1a0c02860 con 0x55b1a0caa0c0 2024-09-10T08:57:02.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.989+0000 7f657bfff640 1 --2- 172.21.15.172:0/1636322652 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55b1a0caa0c0 0x55b1a0cac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.990+0000 7f6569ffb640 10 client.18618 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:57:02.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.990+0000 7f657bfff640 1 --2- 172.21.15.172:0/1636322652 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55b1a0caa0c0 0x55b1a0cac560 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bdf9b706b2da4d15 server_cookie=72813a633a3d88f6 in_seq=0 out_seq=0 2024-09-10T08:57:02.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.991+0000 7f6569ffb640 1 -- 172.21.15.172:0/1636322652 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1a0c02860 con 0x55b1a0caa0c0 2024-09-10T08:57:02.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.991+0000 7f6569ffb640 10 client.18618 handle_command_reply: tid=0 2024-09-10T08:57:02.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.991+0000 7f658b0bb740 1 client.18618 shutdown 2024-09-10T08:57:02.055 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.991+0000 7f658b0bb740 1 -- 172.21.15.172:0/1636322652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c8f5d0 msgr2=0x55b1a0c99ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:02.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.991+0000 7f658b0bb740 1 --2- 172.21.15.172:0/1636322652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0c99ca0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f6574004770 tx=0x7f6574031970 comp rx=0 tx=0).stop 2024-09-10T08:57:02.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.991+0000 7f658b0bb740 1 -- 172.21.15.172:0/1636322652 shutdown_connections 2024-09-10T08:57:02.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.991+0000 7f658b0bb740 1 --2- 172.21.15.172:0/1636322652 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55b1a0caa0c0 0x55b1a0cac560 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.991+0000 7f658b0bb740 1 --2- 172.21.15.172:0/1636322652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c9a370 0x55b1a0c97490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.991+0000 7f6569ffb640 0 client.18618 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:57:02.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.991+0000 7f658b0bb740 1 --2- 172.21.15.172:0/1636322652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c90b40 0x55b1a0c96f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.991+0000 7f658b0bb740 1 --2- 172.21.15.172:0/1636322652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0c99ca0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.991+0000 7f658b0bb740 1 -- 172.21.15.172:0/1636322652 >> 172.21.15.172:0/1636322652 conn(0x55b1a0c58310 msgr2=0x55b1a0c91930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:02.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.991+0000 7f658b0bb740 1 -- 172.21.15.172:0/1636322652 shutdown_connections 2024-09-10T08:57:02.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.991+0000 7f658b0bb740 1 -- 172.21.15.172:0/1636322652 wait complete. 2024-09-10T08:57:02.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:01.992+0000 7f658b0bb740 20 client.18618 trim_cache size 0 max 16384 2024-09-10T08:57:02.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.013+0000 7f658b0bb740 1 Processor -- start 2024-09-10T08:57:02.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.013+0000 7f658b0bb740 1 -- start start 2024-09-10T08:57:02.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f658b0bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c58310 0x55b1a0c8be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f658b0bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0ca6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f658b0bb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c90b40 0x55b1a0cb1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f658b0bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f656400faa0 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f658b0bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6564011640 con 0x55b1a0c58310 2024-09-10T08:57:02.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f658b0bb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6564010170 con 0x55b1a0c90b40 2024-09-10T08:57:02.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f657bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0ca6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f657bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0ca6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49306/0 (socket says 172.21.15.172:49306) 2024-09-10T08:57:02.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f657bfff640 1 -- 172.21.15.172:0/216931765 learned_addr learned my addr 172.21.15.172:0/216931765 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:02.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f658259c640 1 --2- 172.21.15.172:0/216931765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c58310 0x55b1a0c8be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f6582d9d640 1 --2- 172.21.15.172:0/216931765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c90b40 0x55b1a0cb1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f657bfff640 1 -- 172.21.15.172:0/216931765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c90b40 msgr2=0x55b1a0cb1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:02.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f657bfff640 1 --2- 172.21.15.172:0/216931765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c90b40 0x55b1a0cb1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f657bfff640 1 -- 172.21.15.172:0/216931765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c58310 msgr2=0x55b1a0c8be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:02.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f657bfff640 1 --2- 172.21.15.172:0/216931765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c58310 0x55b1a0c8be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f657bfff640 1 -- 172.21.15.172:0/216931765 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a0c8ccf0 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f658259c640 1 --2- 172.21.15.172:0/216931765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c58310 0x55b1a0c8be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:02.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.014+0000 7f657bfff640 1 --2- 172.21.15.172:0/216931765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0ca6340 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f6584102420 tx=0x7f656c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f501b674b2e06a46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:02.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.015+0000 7f656bfff640 1 -- 172.21.15.172:0/216931765 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c023a40 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.015+0000 7f656bfff640 1 -- 172.21.15.172:0/216931765 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f656c0054d0 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.015+0000 7f656bfff640 1 -- 172.21.15.172:0/216931765 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c004910 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 1 -- 172.21.15.172:0/216931765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c8f5d0 msgr2=0x55b1a0ca6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:02.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 1 --2- 172.21.15.172:0/216931765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0ca6340 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f6584102420 tx=0x7f656c0046c0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 1 -- 172.21.15.172:0/216931765 shutdown_connections 2024-09-10T08:57:02.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 1 --2- 172.21.15.172:0/216931765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c90b40 0x55b1a0cb1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 1 --2- 172.21.15.172:0/216931765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0ca6340 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 1 --2- 172.21.15.172:0/216931765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c58310 0x55b1a0c8be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 1 -- 172.21.15.172:0/216931765 >> 172.21.15.172:0/216931765 conn(0x55b1a0c975f0 msgr2=0x55b1a0c8b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:02.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 1 -- 172.21.15.172:0/216931765 shutdown_connections 2024-09-10T08:57:02.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 1 -- 172.21.15.172:0/216931765 wait complete. 2024-09-10T08:57:02.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 1 -- start start 2024-09-10T08:57:02.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 1 Processor -- start 2024-09-10T08:57:02.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c58310 0x55b1a0ca2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0c93a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c90b40 0x55b1a0c93fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6564010f70 con 0x55b1a0c90b40 2024-09-10T08:57:02.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f65640108a0 con 0x55b1a0c58310 2024-09-10T08:57:02.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f656400da80 con 0x55b1a0c8f5d0 2024-09-10T08:57:02.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.016+0000 7f658b0bb740 10 client.0 _ll_register_callbacks cb 0x7f6574040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:02.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f6582d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c90b40 0x55b1a0c93fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f6582d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c90b40 0x55b1a0c93fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49320/0 (socket says 172.21.15.172:49320) 2024-09-10T08:57:02.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f6582d9d640 1 -- 172.21.15.172:0/830462712 learned_addr learned my addr 172.21.15.172:0/830462712 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:02.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f658259c640 1 --2- 172.21.15.172:0/830462712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c58310 0x55b1a0ca2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f657bfff640 1 --2- 172.21.15.172:0/830462712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0c93a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f6582d9d640 1 -- 172.21.15.172:0/830462712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c8f5d0 msgr2=0x55b1a0c93a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:02.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f6582d9d640 1 --2- 172.21.15.172:0/830462712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0c93a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f6582d9d640 1 -- 172.21.15.172:0/830462712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c58310 msgr2=0x55b1a0ca2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:02.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f6582d9d640 1 --2- 172.21.15.172:0/830462712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c58310 0x55b1a0ca2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f6582d9d640 1 -- 172.21.15.172:0/830462712 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a0c92120 con 0x55b1a0c90b40 2024-09-10T08:57:02.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f658259c640 1 --2- 172.21.15.172:0/830462712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c58310 0x55b1a0ca2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:02.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f6582d9d640 1 --2- 172.21.15.172:0/830462712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c90b40 0x55b1a0c93fa0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f657c097b80 tx=0x7f657c09fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c67e7b281d2eff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:02.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f6569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:02.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f6569ffb640 1 -- 172.21.15.172:0/830462712 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c0b4060 con 0x55b1a0c90b40 2024-09-10T08:57:02.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f6569ffb640 1 -- 172.21.15.172:0/830462712 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f657c004d30 con 0x55b1a0c90b40 2024-09-10T08:57:02.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f658b0bb740 1 -- 172.21.15.172:0/830462712 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f656c0054d0 con 0x55b1a0c90b40 2024-09-10T08:57:02.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f6569ffb640 1 -- 172.21.15.172:0/830462712 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f657c01d040 con 0x55b1a0c90b40 2024-09-10T08:57:02.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f658b0bb740 10 client.18624 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:02.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.017+0000 7f658b0bb740 1 -- 172.21.15.172:0/830462712 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b1a0d14a40 con 0x55b1a0c90b40 2024-09-10T08:57:02.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.018+0000 7f6569ffb640 1 -- 172.21.15.172:0/830462712 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f656c0054d0 con 0x55b1a0c90b40 2024-09-10T08:57:02.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.018+0000 7f6569ffb640 1 -- 172.21.15.172:0/830462712 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f657c09a5e0 con 0x55b1a0c90b40 2024-09-10T08:57:02.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.018+0000 7f658b0bb740 10 client.18624 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:02.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.018+0000 7f658b0bb740 10 client.18624 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:57:02.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.018+0000 7f658b0bb740 20 client.18624 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:02.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.018+0000 7f658b0bb740 1 --2- 172.21.15.172:0/830462712 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55b1a0c9a370 0x55b1a0d171b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:02.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.018+0000 7f658b0bb740 4 client.18624 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:57:02.062 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:57:02.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.018+0000 7f658b0bb740 1 -- 172.21.15.172:0/830462712 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1a0bfcd70 con 0x55b1a0c9a370 2024-09-10T08:57:02.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.018+0000 7f658259c640 1 --2- 172.21.15.172:0/830462712 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55b1a0c9a370 0x55b1a0d171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:02.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.019+0000 7f658259c640 1 --2- 172.21.15.172:0/830462712 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55b1a0c9a370 0x55b1a0d171b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a68f24c5653427fb server_cookie=783643a1044cb317 in_seq=0 out_seq=0 2024-09-10T08:57:02.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.019+0000 7f6569ffb640 10 client.18624 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:57:02.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.019+0000 7f6569ffb640 1 -- 172.21.15.172:0/830462712 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1a0bfcd70 con 0x55b1a0c9a370 2024-09-10T08:57:02.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.019+0000 7f6569ffb640 10 client.18624 handle_command_reply: tid=0 2024-09-10T08:57:02.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.019+0000 7f658b0bb740 1 client.18624 shutdown 2024-09-10T08:57:02.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.019+0000 7f658b0bb740 1 -- 172.21.15.172:0/830462712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c90b40 msgr2=0x55b1a0c93fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:02.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.019+0000 7f658b0bb740 1 --2- 172.21.15.172:0/830462712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c90b40 0x55b1a0c93fa0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f657c097b80 tx=0x7f657c09fae0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.063 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.019+0000 7f658b0bb740 1 -- 172.21.15.172:0/830462712 shutdown_connections 2024-09-10T08:57:02.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.019+0000 7f658b0bb740 1 --2- 172.21.15.172:0/830462712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1a0c90b40 0x55b1a0c93fa0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.019+0000 7f658b0bb740 1 --2- 172.21.15.172:0/830462712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b1a0c8f5d0 0x55b1a0c93a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.019+0000 7f658b0bb740 1 --2- 172.21.15.172:0/830462712 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55b1a0c9a370 0x55b1a0d171b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.019+0000 7f658b0bb740 1 --2- 172.21.15.172:0/830462712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1a0c58310 0x55b1a0ca2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.019+0000 7f6569ffb640 0 client.18624 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:57:02.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.019+0000 7f658b0bb740 1 -- 172.21.15.172:0/830462712 >> 172.21.15.172:0/830462712 conn(0x55b1a0c975f0 msgr2=0x55b1a0c90fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:02.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.019+0000 7f658b0bb740 1 -- 172.21.15.172:0/830462712 shutdown_connections 2024-09-10T08:57:02.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.019+0000 7f658b0bb740 1 -- 172.21.15.172:0/830462712 wait complete. 2024-09-10T08:57:02.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.019+0000 7f658b0bb740 20 client.18624 trim_cache size 0 max 16384 2024-09-10T08:57:02.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.020+0000 7f6588ce2640 1 -- 172.21.15.172:0/2444954177 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f65700a2c00 msgr2=0x7f65700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:02.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.020+0000 7f6588ce2640 1 --2- 172.21.15.172:0/2444954177 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f65700a2c00 0x7f65700a50c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x55b1a0c90070 tx=0x7f656c005c90 comp rx=0 tx=0).stop 2024-09-10T08:57:02.064 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.020+0000 7f6588ce2640 1 -- 172.21.15.172:0/2444954177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65841082a0 msgr2=0x7f6584101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:02.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.020+0000 7f6588ce2640 1 --2- 172.21.15.172:0/2444954177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65841082a0 0x7f6584101410 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f657c00cad0 tx=0x7f657c00cf90 comp rx=0 tx=0).stop 2024-09-10T08:57:02.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.020+0000 7f6588ce2640 1 -- 172.21.15.172:0/2444954177 shutdown_connections 2024-09-10T08:57:02.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.020+0000 7f6588ce2640 1 --2- 172.21.15.172:0/2444954177 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f65700a2c00 0x7f65700a50c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.020+0000 7f6588ce2640 1 --2- 172.21.15.172:0/2444954177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65841082a0 0x7f6584101410 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.020+0000 7f6588ce2640 1 --2- 172.21.15.172:0/2444954177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65841036e0 0x7f6584100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.020+0000 7f6588ce2640 1 --2- 172.21.15.172:0/2444954177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6584102d30 0x7f6584100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:02.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.020+0000 7f6588ce2640 1 -- 172.21.15.172:0/2444954177 >> 172.21.15.172:0/2444954177 conn(0x7f65840fc7d0 msgr2=0x7f6584105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:02.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.020+0000 7f6588ce2640 1 -- 172.21.15.172:0/2444954177 shutdown_connections 2024-09-10T08:57:02.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:02.020+0000 7f6588ce2640 1 -- 172.21.15.172:0/2444954177 wait complete. 2024-09-10T08:57:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.489+0000 7f02e25d8640 1 -- 172.21.15.172:0/1161042547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02dc100a40 msgr2=0x7f02dc100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.489+0000 7f02e25d8640 1 --2- 172.21.15.172:0/1161042547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02dc100a40 0x7f02dc100e40 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009900 tx=0x7f02cc02a410 comp rx=0 tx=0).stop 2024-09-10T08:57:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.489+0000 7f02e25d8640 1 -- 172.21.15.172:0/1161042547 shutdown_connections 2024-09-10T08:57:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.489+0000 7f02e25d8640 1 --2- 172.21.15.172:0/1161042547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02dc1025f0 0x7f02dc108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.489+0000 7f02e25d8640 1 --2- 172.21.15.172:0/1161042547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02dc101c30 0x7f02dc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.489+0000 7f02e25d8640 1 --2- 172.21.15.172:0/1161042547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02dc100a40 0x7f02dc100e40 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.490+0000 7f02e25d8640 1 -- 172.21.15.172:0/1161042547 >> 172.21.15.172:0/1161042547 conn(0x7f02dc0fc1e0 msgr2=0x7f02dc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.490+0000 7f02e25d8640 1 -- 172.21.15.172:0/1161042547 shutdown_connections 2024-09-10T08:57:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.490+0000 7f02e25d8640 1 -- 172.21.15.172:0/1161042547 wait complete. 2024-09-10T08:57:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.490+0000 7f02e25d8640 1 Processor -- start 2024-09-10T08:57:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.490+0000 7f02e25d8640 1 -- start start 2024-09-10T08:57:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02e25d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02dc100a40 0x7f02dc06a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02e25d8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02dc101c30 0x7f02dc19ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02e25d8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02dc1025f0 0x7f02dc19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02e25d8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc10c570 con 0x7f02dc100a40 2024-09-10T08:57:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02e25d8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc10c3f0 con 0x7f02dc101c30 2024-09-10T08:57:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02e25d8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc10c6f0 con 0x7f02dc1025f0 2024-09-10T08:57:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02dbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02dc100a40 0x7f02dc06a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02db7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02dc101c30 0x7f02dc19ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02dbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02dc100a40 0x7f02dc06a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49352/0 (socket says 172.21.15.172:49352) 2024-09-10T08:57:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02db7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02dc101c30 0x7f02dc19ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:50650/0 (socket says 172.21.15.172:50650) 2024-09-10T08:57:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02dbfff640 1 -- 172.21.15.172:0/64532748 learned_addr learned my addr 172.21.15.172:0/64532748 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02e0b75640 1 --2- 172.21.15.172:0/64532748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02dc1025f0 0x7f02dc19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02dbfff640 1 -- 172.21.15.172:0/64532748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02dc1025f0 msgr2=0x7f02dc19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02dbfff640 1 --2- 172.21.15.172:0/64532748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02dc1025f0 0x7f02dc19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02dbfff640 1 -- 172.21.15.172:0/64532748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02dc101c30 msgr2=0x7f02dc19ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02dbfff640 1 --2- 172.21.15.172:0/64532748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02dc101c30 0x7f02dc19ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02dbfff640 1 -- 172.21.15.172:0/64532748 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02dc19f920 con 0x7f02dc100a40 2024-09-10T08:57:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02db7fe640 1 --2- 172.21.15.172:0/64532748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02dc101c30 0x7f02dc19ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02dbfff640 1 --2- 172.21.15.172:0/64532748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02dc100a40 0x7f02dc06a450 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f02cc0314e0 tx=0x7f02cc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.491+0000 7f02d97fa640 1 -- 172.21.15.172:0/64532748 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc03f030 con 0x7f02dc100a40 2024-09-10T08:57:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.492+0000 7f02d97fa640 1 -- 172.21.15.172:0/64532748 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02cc003aa0 con 0x7f02dc100a40 2024-09-10T08:57:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.492+0000 7f02d97fa640 1 -- 172.21.15.172:0/64532748 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc035070 con 0x7f02dc100a40 2024-09-10T08:57:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.492+0000 7f02e25d8640 1 -- 172.21.15.172:0/64532748 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02dc19fbb0 con 0x7f02dc100a40 2024-09-10T08:57:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.492+0000 7f02e25d8640 1 -- 172.21.15.172:0/64532748 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02dc1a0050 con 0x7f02dc100a40 2024-09-10T08:57:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.494+0000 7f02d97fa640 1 -- 172.21.15.172:0/64532748 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f02cc042030 con 0x7f02dc100a40 2024-09-10T08:57:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.495+0000 7f02d97fa640 1 --2- 172.21.15.172:0/64532748 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f02c40a2cb0 0x7f02c40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.495+0000 7f02db7fe640 1 --2- 172.21.15.172:0/64532748 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f02c40a2cb0 0x7f02c40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.495+0000 7f02d97fa640 1 -- 172.21.15.172:0/64532748 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f02cc0efa10 con 0x7f02dc100a40 2024-09-10T08:57:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.495+0000 7f02db7fe640 1 --2- 172.21.15.172:0/64532748 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f02c40a2cb0 0x7f02c40a5170 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f02dc06a1c0 tx=0x7f02d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:03.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.522+0000 7f02e5449740 1 Processor -- start 2024-09-10T08:57:03.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.522+0000 7f02e5449740 1 -- start start 2024-09-10T08:57:03.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.522+0000 7f02e5449740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c58db0 0x5555d6c591b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.522+0000 7f02e5449740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c596f0 0x5555d6c5e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.522+0000 7f02e5449740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6c60c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.522+0000 7f02e5449740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5555d6bb82f0 con 0x5555d6c5e780 2024-09-10T08:57:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.522+0000 7f02e5449740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5555d6b60270 con 0x5555d6c58db0 2024-09-10T08:57:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.522+0000 7f02e5449740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5555d6bb42b0 con 0x5555d6c596f0 2024-09-10T08:57:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.522+0000 7f02e0b75640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6c60c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.522+0000 7f02e0b75640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6c60c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49364/0 (socket says 172.21.15.172:49364) 2024-09-10T08:57:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.522+0000 7f02e0b75640 1 -- 172.21.15.172:0/18165793 learned_addr learned my addr 172.21.15.172:0/18165793 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.522+0000 7f02dbfff640 1 --2- 172.21.15.172:0/18165793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c58db0 0x5555d6c591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.522+0000 7f02db7fe640 1 --2- 172.21.15.172:0/18165793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c596f0 0x5555d6c5e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.523+0000 7f02e0b75640 1 -- 172.21.15.172:0/18165793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c596f0 msgr2=0x5555d6c5e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.523+0000 7f02e0b75640 1 --2- 172.21.15.172:0/18165793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c596f0 0x5555d6c5e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.523+0000 7f02e0b75640 1 -- 172.21.15.172:0/18165793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c58db0 msgr2=0x5555d6c591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.523+0000 7f02e0b75640 1 --2- 172.21.15.172:0/18165793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c58db0 0x5555d6c591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.523+0000 7f02e0b75640 1 -- 172.21.15.172:0/18165793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d6b5f340 con 0x5555d6c5e780 2024-09-10T08:57:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.523+0000 7f02e0b75640 1 --2- 172.21.15.172:0/18165793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6c60c40 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009fd0 tx=0x7f02d40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab80aa7e6a7699f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.523+0000 7f02c27fc640 1 -- 172.21.15.172:0/18165793 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d4004d20 con 0x5555d6c5e780 2024-09-10T08:57:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.523+0000 7f02c27fc640 1 -- 172.21.15.172:0/18165793 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02d4004050 con 0x5555d6c5e780 2024-09-10T08:57:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.523+0000 7f02c27fc640 1 -- 172.21.15.172:0/18165793 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d4005220 con 0x5555d6c5e780 2024-09-10T08:57:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.524+0000 7f02e5449740 1 -- 172.21.15.172:0/18165793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 msgr2=0x5555d6c60c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.524+0000 7f02e5449740 1 --2- 172.21.15.172:0/18165793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6c60c40 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009fd0 tx=0x7f02d40118d0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.524+0000 7f02e5449740 1 -- 172.21.15.172:0/18165793 shutdown_connections 2024-09-10T08:57:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.524+0000 7f02e5449740 1 --2- 172.21.15.172:0/18165793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6c60c40 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.524+0000 7f02e5449740 1 --2- 172.21.15.172:0/18165793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c596f0 0x5555d6c5e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.524+0000 7f02e5449740 1 --2- 172.21.15.172:0/18165793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c58db0 0x5555d6c591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.524+0000 7f02e5449740 1 -- 172.21.15.172:0/18165793 >> 172.21.15.172:0/18165793 conn(0x5555d6c50010 msgr2=0x5555d6c50410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.524+0000 7f02e5449740 1 -- 172.21.15.172:0/18165793 shutdown_connections 2024-09-10T08:57:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.524+0000 7f02e5449740 1 -- 172.21.15.172:0/18165793 wait complete. 2024-09-10T08:57:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.524+0000 7f02e5449740 1 -- start start 2024-09-10T08:57:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02e5449740 1 Processor -- start 2024-09-10T08:57:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02e5449740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c58db0 0x5555d6c62f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02e5449740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c596f0 0x5555d6c634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02e5449740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6c6a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02e5449740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5555d6c61750 con 0x5555d6c5e780 2024-09-10T08:57:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02e5449740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5555d6b41290 con 0x5555d6c596f0 2024-09-10T08:57:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02e5449740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5555d6c618d0 con 0x5555d6c58db0 2024-09-10T08:57:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02e5449740 10 client.0 _ll_register_callbacks cb 0x5555d6b72140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02e0b75640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6c6a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02e0b75640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6c6a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49380/0 (socket says 172.21.15.172:49380) 2024-09-10T08:57:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02e0b75640 1 -- 172.21.15.172:0/2923761701 learned_addr learned my addr 172.21.15.172:0/2923761701 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02db7fe640 1 --2- 172.21.15.172:0/2923761701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c596f0 0x5555d6c634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02dbfff640 1 --2- 172.21.15.172:0/2923761701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c58db0 0x5555d6c62f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02e0b75640 1 -- 172.21.15.172:0/2923761701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c58db0 msgr2=0x5555d6c62f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:03.532 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02e0b75640 1 --2- 172.21.15.172:0/2923761701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c58db0 0x5555d6c62f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02e0b75640 1 -- 172.21.15.172:0/2923761701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c596f0 msgr2=0x5555d6c634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02e0b75640 1 --2- 172.21.15.172:0/2923761701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c596f0 0x5555d6c634b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02e0b75640 1 -- 172.21.15.172:0/2923761701 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d68cb210 con 0x5555d6c5e780 2024-09-10T08:57:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02dbfff640 1 --2- 172.21.15.172:0/2923761701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c58db0 0x5555d6c62f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:57:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02e0b75640 1 --2- 172.21.15.172:0/2923761701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6c6a4f0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f02d401b040 tx=0x7f02d4004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d8762fe30d6f47 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.525+0000 7f02bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.526+0000 7f02bb7fe640 1 -- 172.21.15.172:0/2923761701 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d4028020 con 0x5555d6c5e780 2024-09-10T08:57:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.526+0000 7f02e5449740 1 -- 172.21.15.172:0/2923761701 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02d4004050 con 0x5555d6c5e780 2024-09-10T08:57:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.526+0000 7f02bb7fe640 1 -- 172.21.15.172:0/2923761701 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f02d4004c50 con 0x5555d6c5e780 2024-09-10T08:57:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.526+0000 7f02bb7fe640 1 -- 172.21.15.172:0/2923761701 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f02d4033050 con 0x5555d6c5e780 2024-09-10T08:57:03.533 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.526+0000 7f02e5449740 10 client.18636 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.526+0000 7f02e5449740 1 -- 172.21.15.172:0/2923761701 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5555d6997620 con 0x5555d6c5e780 2024-09-10T08:57:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.526+0000 7f02bb7fe640 1 -- 172.21.15.172:0/2923761701 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02d4004050 con 0x5555d6c5e780 2024-09-10T08:57:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.526+0000 7f02bb7fe640 1 -- 172.21.15.172:0/2923761701 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f02d400f4e0 con 0x5555d6c5e780 2024-09-10T08:57:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.526+0000 7f02e5449740 10 client.18636 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.526+0000 7f02e5449740 10 client.18636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.526+0000 7f02e5449740 20 client.18636 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.526+0000 7f02e5449740 1 --2- 172.21.15.172:0/2923761701 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5555d6c6eae0 0x5555d6c70ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.526+0000 7f02e5449740 4 client.18636 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:57:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.526+0000 7f02e5449740 1 -- 172.21.15.172:0/2923761701 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555d6bc5a20 con 0x5555d6c6eae0 2024-09-10T08:57:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.527+0000 7f02dbfff640 1 --2- 172.21.15.172:0/2923761701 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5555d6c6eae0 0x5555d6c70ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.527+0000 7f02dbfff640 1 --2- 172.21.15.172:0/2923761701 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5555d6c6eae0 0x5555d6c70ed0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3a8d241d267528e server_cookie=f14afa591dcbb25e in_seq=0 out_seq=0 2024-09-10T08:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.527+0000 7f02bb7fe640 10 client.18636 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.528+0000 7f02bb7fe640 1 -- 172.21.15.172:0/2923761701 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555d6bc5a20 con 0x5555d6c6eae0 2024-09-10T08:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.528+0000 7f02bb7fe640 10 client.18636 handle_command_reply: tid=0 2024-09-10T08:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.528+0000 7f02e5449740 1 client.18636 shutdown 2024-09-10T08:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.528+0000 7f02e5449740 1 -- 172.21.15.172:0/2923761701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 msgr2=0x5555d6c6a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.528+0000 7f02e5449740 1 --2- 172.21.15.172:0/2923761701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6c6a4f0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f02d401b040 tx=0x7f02d4004490 comp rx=0 tx=0).stop 2024-09-10T08:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.528+0000 7f02e5449740 1 -- 172.21.15.172:0/2923761701 shutdown_connections 2024-09-10T08:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.528+0000 7f02e5449740 1 --2- 172.21.15.172:0/2923761701 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5555d6c6eae0 0x5555d6c70ed0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.528+0000 7f02e5449740 1 --2- 172.21.15.172:0/2923761701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6c6a4f0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.528+0000 7f02bb7fe640 0 client.18636 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.528+0000 7f02e5449740 1 --2- 172.21.15.172:0/2923761701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c596f0 0x5555d6c634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.528+0000 7f02e5449740 1 --2- 172.21.15.172:0/2923761701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c58db0 0x5555d6c62f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.528+0000 7f02e5449740 1 -- 172.21.15.172:0/2923761701 >> 172.21.15.172:0/2923761701 conn(0x5555d6c50010 msgr2=0x5555d6c51820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.528+0000 7f02e5449740 1 -- 172.21.15.172:0/2923761701 shutdown_connections 2024-09-10T08:57:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.528+0000 7f02e5449740 1 -- 172.21.15.172:0/2923761701 wait complete. 2024-09-10T08:57:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.528+0000 7f02e5449740 20 client.18636 trim_cache size 0 max 16384 2024-09-10T08:57:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.546+0000 7f02e5449740 1 Processor -- start 2024-09-10T08:57:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.546+0000 7f02e5449740 1 -- start start 2024-09-10T08:57:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.546+0000 7f02e5449740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c58db0 0x5555d6c4f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.546+0000 7f02e5449740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c596f0 0x5555d6c6a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.546+0000 7f02e5449740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c5e780 0x5555d6c757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.546+0000 7f02e5449740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1a8320 con 0x5555d6c596f0 2024-09-10T08:57:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.546+0000 7f02e5449740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1a9ec0 con 0x5555d6c5e780 2024-09-10T08:57:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.546+0000 7f02e5449740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1a89f0 con 0x5555d6c58db0 2024-09-10T08:57:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.546+0000 7f02db7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c596f0 0x5555d6c6a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.547+0000 7f02db7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c596f0 0x5555d6c6a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49392/0 (socket says 172.21.15.172:49392) 2024-09-10T08:57:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.547+0000 7f02db7fe640 1 -- 172.21.15.172:0/1711213387 learned_addr learned my addr 172.21.15.172:0/1711213387 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.547+0000 7f02e0b75640 1 --2- 172.21.15.172:0/1711213387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c5e780 0x5555d6c757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.547+0000 7f02dbfff640 1 --2- 172.21.15.172:0/1711213387 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c58db0 0x5555d6c4f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.547+0000 7f02db7fe640 1 -- 172.21.15.172:0/1711213387 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c58db0 msgr2=0x5555d6c4f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.547+0000 7f02db7fe640 1 --2- 172.21.15.172:0/1711213387 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c58db0 0x5555d6c4f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.547+0000 7f02db7fe640 1 -- 172.21.15.172:0/1711213387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c5e780 msgr2=0x5555d6c757b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.547+0000 7f02db7fe640 1 --2- 172.21.15.172:0/1711213387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c5e780 0x5555d6c757b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.547+0000 7f02db7fe640 1 -- 172.21.15.172:0/1711213387 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555d6c51ea0 con 0x5555d6c596f0 2024-09-10T08:57:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.547+0000 7f02e0b75640 1 --2- 172.21.15.172:0/1711213387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c5e780 0x5555d6c757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:57:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.547+0000 7f02db7fe640 1 --2- 172.21.15.172:0/1711213387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c596f0 0x5555d6c6a4b0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f02d00042d0 tx=0x7f02d0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d7445e5a39ea0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.547+0000 7f02c1ffb640 1 -- 172.21.15.172:0/1711213387 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0004480 con 0x5555d6c596f0 2024-09-10T08:57:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.547+0000 7f02c1ffb640 1 -- 172.21.15.172:0/1711213387 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02d000f930 con 0x5555d6c596f0 2024-09-10T08:57:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.547+0000 7f02c1ffb640 1 -- 172.21.15.172:0/1711213387 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0027840 con 0x5555d6c596f0 2024-09-10T08:57:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.548+0000 7f02e5449740 1 -- 172.21.15.172:0/1711213387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c596f0 msgr2=0x5555d6c6a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.548+0000 7f02e5449740 1 --2- 172.21.15.172:0/1711213387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c596f0 0x5555d6c6a4b0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f02d00042d0 tx=0x7f02d0014000 comp rx=0 tx=0).stop 2024-09-10T08:57:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.548+0000 7f02e5449740 1 -- 172.21.15.172:0/1711213387 shutdown_connections 2024-09-10T08:57:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.548+0000 7f02e5449740 1 --2- 172.21.15.172:0/1711213387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c5e780 0x5555d6c757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.548+0000 7f02e5449740 1 --2- 172.21.15.172:0/1711213387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c596f0 0x5555d6c6a4b0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.548+0000 7f02e5449740 1 --2- 172.21.15.172:0/1711213387 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c58db0 0x5555d6c4f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.548+0000 7f02e5449740 1 -- 172.21.15.172:0/1711213387 >> 172.21.15.172:0/1711213387 conn(0x5555d6c50010 msgr2=0x5555d6c4e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.548+0000 7f02e5449740 1 -- 172.21.15.172:0/1711213387 shutdown_connections 2024-09-10T08:57:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.548+0000 7f02e5449740 1 -- 172.21.15.172:0/1711213387 wait complete. 2024-09-10T08:57:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.548+0000 7f02e5449740 1 -- start start 2024-09-10T08:57:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02e5449740 1 Processor -- start 2024-09-10T08:57:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02e5449740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c58db0 0x5555d6c67450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02e5449740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c596f0 0x5555d6c67990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02e5449740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6cd7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02e5449740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1a97f0 con 0x5555d6c5e780 2024-09-10T08:57:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02e5449740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1a9120 con 0x5555d6c596f0 2024-09-10T08:57:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02e5449740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc1a6300 con 0x5555d6c58db0 2024-09-10T08:57:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02e5449740 10 client.0 _ll_register_callbacks cb 0x7f02cc03b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02e0b75640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6cd7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02e0b75640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6cd7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49400/0 (socket says 172.21.15.172:49400) 2024-09-10T08:57:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02e0b75640 1 -- 172.21.15.172:0/3877753956 learned_addr learned my addr 172.21.15.172:0/3877753956 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02dbfff640 1 --2- 172.21.15.172:0/3877753956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c58db0 0x5555d6c67450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02db7fe640 1 --2- 172.21.15.172:0/3877753956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c596f0 0x5555d6c67990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02e0b75640 1 -- 172.21.15.172:0/3877753956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c58db0 msgr2=0x5555d6c67450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02e0b75640 1 --2- 172.21.15.172:0/3877753956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c58db0 0x5555d6c67450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02e0b75640 1 -- 172.21.15.172:0/3877753956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c596f0 msgr2=0x5555d6c67990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02e0b75640 1 --2- 172.21.15.172:0/3877753956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c596f0 0x5555d6c67990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.549+0000 7f02e0b75640 1 -- 172.21.15.172:0/3877753956 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555d6cd7d10 con 0x5555d6c5e780 2024-09-10T08:57:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.550+0000 7f02e0b75640 1 --2- 172.21.15.172:0/3877753956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6cd7780 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f02d4005c10 tx=0x7f02d4003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299a07c4588038a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.550+0000 7f02bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.550+0000 7f02bb7fe640 1 -- 172.21.15.172:0/3877753956 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d4028040 con 0x5555d6c5e780 2024-09-10T08:57:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.550+0000 7f02bb7fe640 1 -- 172.21.15.172:0/3877753956 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f02d400f4e0 con 0x5555d6c5e780 2024-09-10T08:57:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.550+0000 7f02e5449740 1 -- 172.21.15.172:0/3877753956 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02d000f930 con 0x5555d6c5e780 2024-09-10T08:57:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.550+0000 7f02bb7fe640 1 -- 172.21.15.172:0/3877753956 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f02d4045050 con 0x5555d6c5e780 2024-09-10T08:57:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.550+0000 7f02e5449740 10 client.18642 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.550+0000 7f02e5449740 1 -- 172.21.15.172:0/3877753956 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5555d6cd9020 con 0x5555d6c5e780 2024-09-10T08:57:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.550+0000 7f02bb7fe640 1 -- 172.21.15.172:0/3877753956 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02d000f930 con 0x5555d6c5e780 2024-09-10T08:57:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.550+0000 7f02bb7fe640 1 -- 172.21.15.172:0/3877753956 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f02d4037970 con 0x5555d6c5e780 2024-09-10T08:57:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.550+0000 7f02e5449740 10 client.18642 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.550+0000 7f02e5449740 10 client.18642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.550+0000 7f02e5449740 20 client.18642 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.550+0000 7f02e5449740 1 --2- 172.21.15.172:0/3877753956 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5555d6cd9930 0x5555d6cdbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.550+0000 7f02e5449740 4 client.18642 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:57:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.550+0000 7f02e5449740 1 -- 172.21.15.172:0/3877753956 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5555d6bbfd70 con 0x5555d6cd9930 2024-09-10T08:57:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.551+0000 7f02dbfff640 1 --2- 172.21.15.172:0/3877753956 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5555d6cd9930 0x5555d6cdbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.551+0000 7f02bb7fe640 10 client.18642 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.551+0000 7f02dbfff640 1 --2- 172.21.15.172:0/3877753956 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5555d6cd9930 0x5555d6cdbd50 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bee6fa896122d58 server_cookie=145628abf7b81c3b in_seq=0 out_seq=0 2024-09-10T08:57:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.557+0000 7f02bb7fe640 1 -- 172.21.15.172:0/3877753956 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5555d6bbfd70 con 0x5555d6cd9930 2024-09-10T08:57:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.557+0000 7f02bb7fe640 10 client.18642 handle_command_reply: tid=0 2024-09-10T08:57:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.557+0000 7f02e5449740 1 client.18642 shutdown 2024-09-10T08:57:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.557+0000 7f02e5449740 1 -- 172.21.15.172:0/3877753956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 msgr2=0x5555d6cd7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.557+0000 7f02e5449740 1 --2- 172.21.15.172:0/3877753956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6cd7780 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f02d4005c10 tx=0x7f02d4003e50 comp rx=0 tx=0).stop 2024-09-10T08:57:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.557+0000 7f02e5449740 1 -- 172.21.15.172:0/3877753956 shutdown_connections 2024-09-10T08:57:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.557+0000 7f02e5449740 1 --2- 172.21.15.172:0/3877753956 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5555d6cd9930 0x5555d6cdbd50 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.557+0000 7f02e5449740 1 --2- 172.21.15.172:0/3877753956 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555d6c5e780 0x5555d6cd7780 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.557+0000 7f02bb7fe640 0 client.18642 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.557+0000 7f02e5449740 1 --2- 172.21.15.172:0/3877753956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555d6c596f0 0x5555d6c67990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.557+0000 7f02e5449740 1 --2- 172.21.15.172:0/3877753956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555d6c58db0 0x5555d6c67450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.557+0000 7f02e5449740 1 -- 172.21.15.172:0/3877753956 >> 172.21.15.172:0/3877753956 conn(0x5555d6c50010 msgr2=0x5555d6c4f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.557+0000 7f02e5449740 1 -- 172.21.15.172:0/3877753956 shutdown_connections 2024-09-10T08:57:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.557+0000 7f02e5449740 1 -- 172.21.15.172:0/3877753956 wait complete. 2024-09-10T08:57:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.558+0000 7f02e5449740 20 client.18642 trim_cache size 0 max 16384 2024-09-10T08:57:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:57:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:57:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:57:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:57:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.558+0000 7f02e25d8640 1 -- 172.21.15.172:0/64532748 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f02c40a2cb0 msgr2=0x7f02c40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.558+0000 7f02e25d8640 1 --2- 172.21.15.172:0/64532748 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f02c40a2cb0 0x7f02c40a5170 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f02dc06a1c0 tx=0x7f02d0005f90 comp rx=0 tx=0).stop 2024-09-10T08:57:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.558+0000 7f02e25d8640 1 -- 172.21.15.172:0/64532748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02dc100a40 msgr2=0x7f02dc06a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.558+0000 7f02e25d8640 1 --2- 172.21.15.172:0/64532748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02dc100a40 0x7f02dc06a450 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f02cc0314e0 tx=0x7f02cc004270 comp rx=0 tx=0).stop 2024-09-10T08:57:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.558+0000 7f02e25d8640 1 -- 172.21.15.172:0/64532748 shutdown_connections 2024-09-10T08:57:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.558+0000 7f02e25d8640 1 --2- 172.21.15.172:0/64532748 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f02c40a2cb0 0x7f02c40a5170 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.558+0000 7f02e25d8640 1 --2- 172.21.15.172:0/64532748 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02dc1025f0 0x7f02dc19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.558+0000 7f02e25d8640 1 --2- 172.21.15.172:0/64532748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02dc101c30 0x7f02dc19ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.558+0000 7f02e25d8640 1 --2- 172.21.15.172:0/64532748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02dc100a40 0x7f02dc06a450 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.558+0000 7f02e25d8640 1 -- 172.21.15.172:0/64532748 >> 172.21.15.172:0/64532748 conn(0x7f02dc0fc1e0 msgr2=0x7f02dc104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.558+0000 7f02e25d8640 1 -- 172.21.15.172:0/64532748 shutdown_connections 2024-09-10T08:57:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:03.558+0000 7f02e25d8640 1 -- 172.21.15.172:0/64532748 wait complete. 2024-09-10T08:57:03.622 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T08:57:03.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:03 smithi174 ceph-mon[34068]: pgmap v675: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-09-10T08:57:03.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:03 smithi191 ceph-mon[35314]: pgmap v675: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-09-10T08:57:03.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:03 smithi172 ceph-mon[30854]: pgmap v675: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 23 MiB/s wr, 19 op/s 2024-09-10T08:57:03.981 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:57:05.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.068+0000 7f3bc483d640 1 -- 172.21.15.172:0/1172192607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc0102570 msgr2=0x7f3bc01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:05.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.068+0000 7f3bc483d640 1 --2- 172.21.15.172:0/1172192607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc0102570 0x7f3bc01029b0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009900 tx=0x7f3bb402a480 comp rx=0 tx=0).stop 2024-09-10T08:57:05.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.068+0000 7f3bc483d640 1 -- 172.21.15.172:0/1172192607 shutdown_connections 2024-09-10T08:57:05.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.068+0000 7f3bc483d640 1 --2- 172.21.15.172:0/1172192607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc0102ef0 0x7f3bc010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.068+0000 7f3bc483d640 1 --2- 172.21.15.172:0/1172192607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc0102570 0x7f3bc01029b0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.068+0000 7f3bc483d640 1 --2- 172.21.15.172:0/1172192607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bc0100e20 0x7f3bc0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.068+0000 7f3bc483d640 1 -- 172.21.15.172:0/1172192607 >> 172.21.15.172:0/1172192607 conn(0x7f3bc00fc7d0 msgr2=0x7f3bc00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:05.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.068+0000 7f3bc483d640 1 -- 172.21.15.172:0/1172192607 shutdown_connections 2024-09-10T08:57:05.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.068+0000 7f3bc483d640 1 -- 172.21.15.172:0/1172192607 wait complete. 2024-09-10T08:57:05.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.069+0000 7f3bc483d640 1 Processor -- start 2024-09-10T08:57:05.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.069+0000 7f3bc483d640 1 -- start start 2024-09-10T08:57:05.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bc483d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc0100e20 0x7f3bc0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bc483d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bc0102570 0x7f3bc0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bc483d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc0102ef0 0x7f3bc019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bc483d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc0110e30 con 0x7f3bc0102ef0 2024-09-10T08:57:05.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bc483d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc0110cb0 con 0x7f3bc0100e20 2024-09-10T08:57:05.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bc483d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc0110fb0 con 0x7f3bc0102570 2024-09-10T08:57:05.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bbdd9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bc0102570 0x7f3bc0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bbed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc0102ef0 0x7f3bc019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bbdd9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bc0102570 0x7f3bc0198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:36274/0 (socket says 172.21.15.172:36274) 2024-09-10T08:57:05.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bbdd9b640 1 -- 172.21.15.172:0/1474082312 learned_addr learned my addr 172.21.15.172:0/1474082312 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:05.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bbe59c640 1 --2- 172.21.15.172:0/1474082312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc0100e20 0x7f3bc0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bbed9d640 1 -- 172.21.15.172:0/1474082312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bc0102570 msgr2=0x7f3bc0198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:05.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bbed9d640 1 --2- 172.21.15.172:0/1474082312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bc0102570 0x7f3bc0198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bbed9d640 1 -- 172.21.15.172:0/1474082312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc0100e20 msgr2=0x7f3bc0198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:05.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bbed9d640 1 --2- 172.21.15.172:0/1474082312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc0100e20 0x7f3bc0198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bbed9d640 1 -- 172.21.15.172:0/1474082312 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc019d3b0 con 0x7f3bc0102ef0 2024-09-10T08:57:05.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bbdd9b640 1 --2- 172.21.15.172:0/1474082312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bc0102570 0x7f3bc0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:05.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3bbed9d640 1 --2- 172.21.15.172:0/1474082312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc0102ef0 0x7f3bc019ccd0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800cab0 tx=0x7f3bb800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:05.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.070+0000 7f3baf7fe640 1 -- 172.21.15.172:0/1474082312 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb800ae20 con 0x7f3bc0102ef0 2024-09-10T08:57:05.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.071+0000 7f3bc483d640 1 -- 172.21.15.172:0/1474082312 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc019d6a0 con 0x7f3bc0102ef0 2024-09-10T08:57:05.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.071+0000 7f3bc483d640 1 -- 172.21.15.172:0/1474082312 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc01a4f80 con 0x7f3bc0102ef0 2024-09-10T08:57:05.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.071+0000 7f3baf7fe640 1 -- 172.21.15.172:0/1474082312 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bb800f900 con 0x7f3bc0102ef0 2024-09-10T08:57:05.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.071+0000 7f3baf7fe640 1 -- 172.21.15.172:0/1474082312 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb80189d0 con 0x7f3bc0102ef0 2024-09-10T08:57:05.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.073+0000 7f3baf7fe640 1 -- 172.21.15.172:0/1474082312 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3bb800fab0 con 0x7f3bc0102ef0 2024-09-10T08:57:05.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.073+0000 7f3bc6c16740 1 Processor -- start 2024-09-10T08:57:05.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.074+0000 7f3bc6c16740 1 -- start start 2024-09-10T08:57:05.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.074+0000 7f3baf7fe640 1 --2- 172.21.15.172:0/1474082312 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3ba80a2b80 0x7f3ba80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.074+0000 7f3bc6c16740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556318204f50 0x5563182034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.074+0000 7f3bc6c16740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563182039e0 0x556318203e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.074+0000 7f3bc6c16740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55631820e780 0x556318210b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.074+0000 7f3bc6c16740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5563181652f0 con 0x5563182039e0 2024-09-10T08:57:05.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.074+0000 7f3bc6c16740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55631810ccf0 con 0x55631820e780 2024-09-10T08:57:05.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.074+0000 7f3bc6c16740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5563181612b0 con 0x556318204f50 2024-09-10T08:57:05.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.074+0000 7f3bbdd9b640 1 --2- 172.21.15.172:0/1474082312 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3ba80a2b80 0x7f3ba80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.074+0000 7f3baf7fe640 1 -- 172.21.15.172:0/1474082312 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3bb80cf660 con 0x7f3bc0102ef0 2024-09-10T08:57:05.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.074+0000 7f3bbdd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563182039e0 0x556318203e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.074+0000 7f3bbe59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556318204f50 0x5563182034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.074+0000 7f3bbe59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556318204f50 0x5563182034a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:36290/0 (socket says 172.21.15.172:36290) 2024-09-10T08:57:05.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.074+0000 7f3bbdd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563182039e0 0x556318203e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49424/0 (socket says 172.21.15.172:49424) 2024-09-10T08:57:05.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.074+0000 7f3bbe59c640 1 -- 172.21.15.172:0/52850868 learned_addr learned my addr 172.21.15.172:0/52850868 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:05.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.074+0000 7f3bbed9d640 1 --2- 172.21.15.172:0/52850868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55631820e780 0x556318210b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.075+0000 7f3bbe59c640 1 -- 172.21.15.172:0/52850868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55631820e780 msgr2=0x556318210b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:05.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.075+0000 7f3bbe59c640 1 --2- 172.21.15.172:0/52850868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55631820e780 0x556318210b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.075+0000 7f3bbe59c640 1 -- 172.21.15.172:0/52850868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563182039e0 msgr2=0x556318203e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:05.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.075+0000 7f3bbe59c640 1 --2- 172.21.15.172:0/52850868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563182039e0 0x556318203e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.075+0000 7f3bbe59c640 1 -- 172.21.15.172:0/52850868 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563180f13a0 con 0x556318204f50 2024-09-10T08:57:05.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.075+0000 7f3bbdd9b640 1 --2- 172.21.15.172:0/1474082312 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3ba80a2b80 0x7f3ba80a5040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3bc01000a0 tx=0x7f3bb40317a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:05.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.075+0000 7f3bbe59c640 1 --2- 172.21.15.172:0/52850868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556318204f50 0x5563182034a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3bb000bcd0 tx=0x7f3bb00107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72b2aea29706ea5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:05.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.075+0000 7f3bacff9640 1 -- 172.21.15.172:0/52850868 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0005220 con 0x556318204f50 2024-09-10T08:57:05.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.075+0000 7f3bacff9640 1 -- 172.21.15.172:0/52850868 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bb0004c50 con 0x556318204f50 2024-09-10T08:57:05.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.075+0000 7f3bacff9640 1 -- 172.21.15.172:0/52850868 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb00116a0 con 0x556318204f50 2024-09-10T08:57:05.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.076+0000 7f3bc6c16740 1 -- 172.21.15.172:0/52850868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556318204f50 msgr2=0x5563182034a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:05.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.076+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/52850868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556318204f50 0x5563182034a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3bb000bcd0 tx=0x7f3bb00107a0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.076+0000 7f3bc6c16740 1 -- 172.21.15.172:0/52850868 shutdown_connections 2024-09-10T08:57:05.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.076+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/52850868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55631820e780 0x556318210b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.076+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/52850868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563182039e0 0x556318203e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.076+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/52850868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556318204f50 0x5563182034a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.076+0000 7f3bc6c16740 1 -- 172.21.15.172:0/52850868 >> 172.21.15.172:0/52850868 conn(0x5563181c8310 msgr2=0x556318200070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:05.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.076+0000 7f3bc6c16740 1 -- 172.21.15.172:0/52850868 shutdown_connections 2024-09-10T08:57:05.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.076+0000 7f3bc6c16740 1 -- 172.21.15.172:0/52850868 wait complete. 2024-09-10T08:57:05.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.076+0000 7f3bc6c16740 1 -- start start 2024-09-10T08:57:05.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bc6c16740 1 Processor -- start 2024-09-10T08:57:05.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bc6c16740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563182039e0 0x55631820e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bc6c16740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556318204f50 0x55631820b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bc6c16740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55631820e780 0x55631820b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bc6c16740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556318211480 con 0x5563182039e0 2024-09-10T08:57:05.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bc6c16740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556318120060 con 0x55631820e780 2024-09-10T08:57:05.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bc6c16740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556318211600 con 0x556318204f50 2024-09-10T08:57:05.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bbe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563182039e0 0x55631820e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bbe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563182039e0 0x55631820e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49440/0 (socket says 172.21.15.172:49440) 2024-09-10T08:57:05.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bc6c16740 10 client.0 _ll_register_callbacks cb 0x556317e738c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:05.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bbe59c640 1 -- 172.21.15.172:0/3408859605 learned_addr learned my addr 172.21.15.172:0/3408859605 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:05.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bbdd9b640 1 --2- 172.21.15.172:0/3408859605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556318204f50 0x55631820b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bbed9d640 1 --2- 172.21.15.172:0/3408859605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55631820e780 0x55631820b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bbe59c640 1 -- 172.21.15.172:0/3408859605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556318204f50 msgr2=0x55631820b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:05.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bbe59c640 1 --2- 172.21.15.172:0/3408859605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556318204f50 0x55631820b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bbe59c640 1 -- 172.21.15.172:0/3408859605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55631820e780 msgr2=0x55631820b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:05.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bbe59c640 1 --2- 172.21.15.172:0/3408859605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55631820e780 0x55631820b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bbe59c640 1 -- 172.21.15.172:0/3408859605 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556317ed63c0 con 0x5563182039e0 2024-09-10T08:57:05.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3bbe59c640 1 --2- 172.21.15.172:0/3408859605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563182039e0 0x55631820e140 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0006010 tx=0x7f3bb000bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5560bdd92f20da7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:05.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.077+0000 7f3ba5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:05.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.078+0000 7f3ba5ffb640 1 -- 172.21.15.172:0/3408859605 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0004020 con 0x5563182039e0 2024-09-10T08:57:05.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.078+0000 7f3ba5ffb640 1 -- 172.21.15.172:0/3408859605 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3bb00041c0 con 0x5563182039e0 2024-09-10T08:57:05.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.078+0000 7f3bc6c16740 1 -- 172.21.15.172:0/3408859605 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bb0004c50 con 0x5563182039e0 2024-09-10T08:57:05.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.078+0000 7f3ba5ffb640 1 -- 172.21.15.172:0/3408859605 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3bb00043c0 con 0x5563182039e0 2024-09-10T08:57:05.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.078+0000 7f3bc6c16740 10 client.18654 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:05.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.078+0000 7f3bc6c16740 1 -- 172.21.15.172:0/3408859605 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55631821cec0 con 0x5563182039e0 2024-09-10T08:57:05.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.078+0000 7f3ba5ffb640 1 -- 172.21.15.172:0/3408859605 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bb00035f0 con 0x5563182039e0 2024-09-10T08:57:05.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.078+0000 7f3ba5ffb640 1 -- 172.21.15.172:0/3408859605 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f3bb0011440 con 0x5563182039e0 2024-09-10T08:57:05.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.078+0000 7f3bc6c16740 10 client.18654 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:05.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.078+0000 7f3bc6c16740 10 client.18654 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:57:05.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.078+0000 7f3bc6c16740 20 client.18654 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:05.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.078+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/3408859605 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55631821e210 0x556318220600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.078+0000 7f3bc6c16740 4 client.18654 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:57:05.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.078+0000 7f3bc6c16740 1 -- 172.21.15.172:0/3408859605 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3bb0004c50 con 0x55631821e210 2024-09-10T08:57:05.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.079+0000 7f3bbdd9b640 1 --2- 172.21.15.172:0/3408859605 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55631821e210 0x556318220600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.079+0000 7f3bbdd9b640 1 --2- 172.21.15.172:0/3408859605 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55631821e210 0x556318220600 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=20a12967e6c6932f server_cookie=36ca11023d1dedb9 in_seq=0 out_seq=0 2024-09-10T08:57:05.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.079+0000 7f3ba5ffb640 10 client.18654 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:57:05.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.080+0000 7f3ba5ffb640 1 -- 172.21.15.172:0/3408859605 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f3bb0004c50 con 0x55631821e210 2024-09-10T08:57:05.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.080+0000 7f3ba5ffb640 10 client.18654 handle_command_reply: tid=0 2024-09-10T08:57:05.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.080+0000 7f3bc6c16740 1 client.18654 shutdown 2024-09-10T08:57:05.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.080+0000 7f3bc6c16740 1 -- 172.21.15.172:0/3408859605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563182039e0 msgr2=0x55631820e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:05.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.080+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/3408859605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563182039e0 0x55631820e140 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0006010 tx=0x7f3bb000bef0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.080+0000 7f3bc6c16740 1 -- 172.21.15.172:0/3408859605 shutdown_connections 2024-09-10T08:57:05.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.080+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/3408859605 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55631821e210 0x556318220600 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.080+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/3408859605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55631820e780 0x55631820b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.080+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/3408859605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556318204f50 0x55631820b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.080+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/3408859605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563182039e0 0x55631820e140 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.080+0000 7f3ba5ffb640 0 client.18654 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:57:05.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.080+0000 7f3bc6c16740 1 -- 172.21.15.172:0/3408859605 >> 172.21.15.172:0/3408859605 conn(0x5563181c8310 msgr2=0x556318205d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:05.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.080+0000 7f3bc6c16740 1 -- 172.21.15.172:0/3408859605 shutdown_connections 2024-09-10T08:57:05.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.080+0000 7f3bc6c16740 1 -- 172.21.15.172:0/3408859605 wait complete. 2024-09-10T08:57:05.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.080+0000 7f3bc6c16740 20 client.18654 trim_cache size 0 max 16384 2024-09-10T08:57:05.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.096+0000 7f3bc6c16740 1 Processor -- start 2024-09-10T08:57:05.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.096+0000 7f3bc6c16740 1 -- start start 2024-09-10T08:57:05.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.096+0000 7f3bc6c16740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563182039e0 0x55631821a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.096+0000 7f3bc6c16740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556318204f50 0x556318206530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3bc6c16740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55631820e780 0x556318206a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3bc6c16740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc01ac9b0 con 0x55631820e780 2024-09-10T08:57:05.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3bc6c16740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc01ae550 con 0x556318204f50 2024-09-10T08:57:05.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3bc6c16740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc01ad080 con 0x5563182039e0 2024-09-10T08:57:05.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3bbed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55631820e780 0x556318206a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3bbdd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556318204f50 0x556318206530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3bbdd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556318204f50 0x556318206530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:50730/0 (socket says 172.21.15.172:50730) 2024-09-10T08:57:05.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3bbed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55631820e780 0x556318206a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49444/0 (socket says 172.21.15.172:49444) 2024-09-10T08:57:05.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3bbdd9b640 1 -- 172.21.15.172:0/1856330919 learned_addr learned my addr 172.21.15.172:0/1856330919 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:05.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3bbe59c640 1 --2- 172.21.15.172:0/1856330919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563182039e0 0x55631821a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3bbdd9b640 1 -- 172.21.15.172:0/1856330919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563182039e0 msgr2=0x55631821a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:05.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3bbdd9b640 1 --2- 172.21.15.172:0/1856330919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563182039e0 0x55631821a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3bbdd9b640 1 -- 172.21.15.172:0/1856330919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55631820e780 msgr2=0x556318206a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:05.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3bbdd9b640 1 --2- 172.21.15.172:0/1856330919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55631820e780 0x556318206a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3bbdd9b640 1 -- 172.21.15.172:0/1856330919 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55631820dd60 con 0x556318204f50 2024-09-10T08:57:05.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3bbed9d640 1 --2- 172.21.15.172:0/1856330919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55631820e780 0x556318206a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:05.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3bbdd9b640 1 --2- 172.21.15.172:0/1856330919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556318204f50 0x556318206530 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40096c0 tx=0x7f3bb4048000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28c357053dff638e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:05.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.097+0000 7f3ba7fff640 1 -- 172.21.15.172:0/1856330919 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4003bc0 con 0x556318204f50 2024-09-10T08:57:05.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.098+0000 7f3ba7fff640 1 -- 172.21.15.172:0/1856330919 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bb40044d0 con 0x556318204f50 2024-09-10T08:57:05.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.098+0000 7f3ba7fff640 1 -- 172.21.15.172:0/1856330919 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4004b80 con 0x556318204f50 2024-09-10T08:57:05.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.098+0000 7f3bc6c16740 1 -- 172.21.15.172:0/1856330919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556318204f50 msgr2=0x556318206530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.098+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/1856330919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556318204f50 0x556318206530 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40096c0 tx=0x7f3bb4048000 comp rx=0 tx=0).stop 2024-09-10T08:57:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.098+0000 7f3bc6c16740 1 -- 172.21.15.172:0/1856330919 shutdown_connections 2024-09-10T08:57:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.098+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/1856330919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55631820e780 0x556318206a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.098+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/1856330919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556318204f50 0x556318206530 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.098+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/1856330919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563182039e0 0x55631821a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.098+0000 7f3bc6c16740 1 -- 172.21.15.172:0/1856330919 >> 172.21.15.172:0/1856330919 conn(0x5563181c8310 msgr2=0x55631820a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.098+0000 7f3bc6c16740 1 -- 172.21.15.172:0/1856330919 shutdown_connections 2024-09-10T08:57:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.098+0000 7f3bc6c16740 1 -- 172.21.15.172:0/1856330919 wait complete. 2024-09-10T08:57:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.099+0000 7f3bc6c16740 1 -- start start 2024-09-10T08:57:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.099+0000 7f3bc6c16740 1 Processor -- start 2024-09-10T08:57:05.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.099+0000 7f3bc6c16740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563182039e0 0x556318216ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.099+0000 7f3bc6c16740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556318204f50 0x5563182121d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.099+0000 7f3bc6c16740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55631820e780 0x556318212710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.099+0000 7f3bc6c16740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc01ade80 con 0x55631820e780 2024-09-10T08:57:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.099+0000 7f3bc6c16740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc01ad7b0 con 0x5563182039e0 2024-09-10T08:57:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.099+0000 7f3bc6c16740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc01aa990 con 0x556318204f50 2024-09-10T08:57:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.099+0000 7f3bc6c16740 10 client.0 _ll_register_callbacks cb 0x7f3bb4009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.099+0000 7f3bbe59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563182039e0 0x556318216ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.099+0000 7f3bbe59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563182039e0 0x556318216ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:50746/0 (socket says 172.21.15.172:50746) 2024-09-10T08:57:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.099+0000 7f3bbe59c640 1 -- 172.21.15.172:0/4047536726 learned_addr learned my addr 172.21.15.172:0/4047536726 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.099+0000 7f3bbdd9b640 1 --2- 172.21.15.172:0/4047536726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556318204f50 0x5563182121d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.099+0000 7f3bbed9d640 1 --2- 172.21.15.172:0/4047536726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55631820e780 0x556318212710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.100+0000 7f3bbe59c640 1 -- 172.21.15.172:0/4047536726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556318204f50 msgr2=0x5563182121d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:05.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.100+0000 7f3bbe59c640 1 --2- 172.21.15.172:0/4047536726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556318204f50 0x5563182121d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.100+0000 7f3bbe59c640 1 -- 172.21.15.172:0/4047536726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55631820e780 msgr2=0x556318212710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.100+0000 7f3bbe59c640 1 --2- 172.21.15.172:0/4047536726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55631820e780 0x556318212710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.100+0000 7f3bbe59c640 1 -- 172.21.15.172:0/4047536726 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556318212e00 con 0x5563182039e0 2024-09-10T08:57:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.100+0000 7f3bbe59c640 1 --2- 172.21.15.172:0/4047536726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563182039e0 0x556318216ea0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0005d00 tx=0x7f3bb000be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=742bd8140441363 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.100+0000 7f3ba5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:57:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.100+0000 7f3ba5ffb640 1 -- 172.21.15.172:0/4047536726 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb003a060 con 0x5563182039e0 2024-09-10T08:57:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.100+0000 7f3bc6c16740 1 -- 172.21.15.172:0/4047536726 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bb40044d0 con 0x5563182039e0 2024-09-10T08:57:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.100+0000 7f3ba5ffb640 1 -- 172.21.15.172:0/4047536726 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3bb0011a80 con 0x5563182039e0 2024-09-10T08:57:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.100+0000 7f3ba5ffb640 1 -- 172.21.15.172:0/4047536726 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3bb0041050 con 0x5563182039e0 2024-09-10T08:57:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.100+0000 7f3bc6c16740 10 client.27563 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.100+0000 7f3bc6c16740 1 -- 172.21.15.172:0/4047536726 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556318288b10 con 0x5563182039e0 2024-09-10T08:57:05.106 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.100+0000 7f3ba5ffb640 1 -- 172.21.15.172:0/4047536726 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bb40044d0 con 0x5563182039e0 2024-09-10T08:57:05.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.101+0000 7f3ba5ffb640 1 -- 172.21.15.172:0/4047536726 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f3bb0005030 con 0x5563182039e0 2024-09-10T08:57:05.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.101+0000 7f3bc6c16740 10 client.27563 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:05.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.101+0000 7f3bc6c16740 10 client.27563 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:57:05.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.101+0000 7f3bc6c16740 20 client.27563 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:05.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.101+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/4047536726 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5563182891d0 0x55631828b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:05.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.101+0000 7f3bc6c16740 4 client.27563 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:57:05.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.101+0000 7f3bc6c16740 1 -- 172.21.15.172:0/4047536726 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556318172860 con 0x5563182891d0 2024-09-10T08:57:05.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.101+0000 7f3bbdd9b640 1 --2- 172.21.15.172:0/4047536726 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5563182891d0 0x55631828b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:05.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.101+0000 7f3bbdd9b640 1 --2- 172.21.15.172:0/4047536726 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5563182891d0 0x55631828b5c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ebc80286811f525e server_cookie=579d719864a5400d in_seq=0 out_seq=0 2024-09-10T08:57:05.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.101+0000 7f3ba5ffb640 10 client.27563 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:57:05.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.101+0000 7f3ba5ffb640 1 -- 172.21.15.172:0/4047536726 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556318172860 con 0x5563182891d0 2024-09-10T08:57:05.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.101+0000 7f3ba5ffb640 10 client.27563 handle_command_reply: tid=0 2024-09-10T08:57:05.107 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.101+0000 7f3bc6c16740 1 client.27563 shutdown 2024-09-10T08:57:05.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.102+0000 7f3bc6c16740 1 -- 172.21.15.172:0/4047536726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563182039e0 msgr2=0x556318216ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:05.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.102+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/4047536726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563182039e0 0x556318216ea0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0005d00 tx=0x7f3bb000be80 comp rx=0 tx=0).stop 2024-09-10T08:57:05.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.102+0000 7f3bc6c16740 1 -- 172.21.15.172:0/4047536726 shutdown_connections 2024-09-10T08:57:05.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.102+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/4047536726 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5563182891d0 0x55631828b5c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.102+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/4047536726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55631820e780 0x556318212710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.102+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/4047536726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556318204f50 0x5563182121d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.102+0000 7f3ba5ffb640 0 client.27563 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:57:05.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.102+0000 7f3bc6c16740 1 --2- 172.21.15.172:0/4047536726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563182039e0 0x556318216ea0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.102+0000 7f3bc6c16740 1 -- 172.21.15.172:0/4047536726 >> 172.21.15.172:0/4047536726 conn(0x5563181c8310 msgr2=0x55631820cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:05.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.102+0000 7f3bc6c16740 1 -- 172.21.15.172:0/4047536726 shutdown_connections 2024-09-10T08:57:05.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.102+0000 7f3bc6c16740 1 -- 172.21.15.172:0/4047536726 wait complete. 2024-09-10T08:57:05.108 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.102+0000 7f3bc6c16740 20 client.27563 trim_cache size 0 max 16384 2024-09-10T08:57:05.109 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:57:05.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.102+0000 7f3bc483d640 1 -- 172.21.15.172:0/1474082312 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3ba80a2b80 msgr2=0x7f3ba80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:05.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.102+0000 7f3bc483d640 1 --2- 172.21.15.172:0/1474082312 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3ba80a2b80 0x7f3ba80a5040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3bc01000a0 tx=0x7f3bb40317a0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.109 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.102+0000 7f3bc483d640 1 -- 172.21.15.172:0/1474082312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc0102ef0 msgr2=0x7f3bc019ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:05.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.102+0000 7f3bc483d640 1 --2- 172.21.15.172:0/1474082312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc0102ef0 0x7f3bc019ccd0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800cab0 tx=0x7f3bb800cf70 comp rx=0 tx=0).stop 2024-09-10T08:57:05.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.103+0000 7f3bc483d640 1 -- 172.21.15.172:0/1474082312 shutdown_connections 2024-09-10T08:57:05.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.103+0000 7f3bc483d640 1 --2- 172.21.15.172:0/1474082312 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3ba80a2b80 0x7f3ba80a5040 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.103+0000 7f3bc483d640 1 --2- 172.21.15.172:0/1474082312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc0102ef0 0x7f3bc019ccd0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.103+0000 7f3bc483d640 1 --2- 172.21.15.172:0/1474082312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bc0102570 0x7f3bc0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.103+0000 7f3bc483d640 1 --2- 172.21.15.172:0/1474082312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc0100e20 0x7f3bc0198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:05.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.103+0000 7f3bc483d640 1 -- 172.21.15.172:0/1474082312 >> 172.21.15.172:0/1474082312 conn(0x7f3bc00fc7d0 msgr2=0x7f3bc006a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:05.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.103+0000 7f3bc483d640 1 -- 172.21.15.172:0/1474082312 shutdown_connections 2024-09-10T08:57:05.110 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:05.103+0000 7f3bc483d640 1 -- 172.21.15.172:0/1474082312 wait complete. 2024-09-10T08:57:05.714 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:05 smithi172 ceph-mon[30854]: pgmap v676: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-09-10T08:57:05.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:05 smithi174 ceph-mon[34068]: pgmap v676: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-09-10T08:57:05.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:05 smithi191 ceph-mon[35314]: pgmap v676: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-09-10T08:57:06.208 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:57:06.585 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:57:07.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.117+0000 7f25cfef7640 1 -- 172.21.15.172:0/1321655538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25c8105de0 msgr2=0x7f25c810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:07.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.117+0000 7f25cfef7640 1 --2- 172.21.15.172:0/1321655538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25c8105de0 0x7f25c810d960 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009ca0 tx=0x7f25c002a8b0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.117+0000 7f25cfef7640 1 -- 172.21.15.172:0/1321655538 shutdown_connections 2024-09-10T08:57:07.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.117+0000 7f25cfef7640 1 --2- 172.21.15.172:0/1321655538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25c8105de0 0x7f25c810d960 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.117+0000 7f25cfef7640 1 --2- 172.21.15.172:0/1321655538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25c8101e20 0x7f25c81056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.117+0000 7f25cfef7640 1 --2- 172.21.15.172:0/1321655538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c8101470 0x7f25c8101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.117+0000 7f25cfef7640 1 -- 172.21.15.172:0/1321655538 >> 172.21.15.172:0/1321655538 conn(0x7f25c8077230 msgr2=0x7f25c80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:07.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.117+0000 7f25cfef7640 1 -- 172.21.15.172:0/1321655538 shutdown_connections 2024-09-10T08:57:07.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.117+0000 7f25cfef7640 1 -- 172.21.15.172:0/1321655538 wait complete. 2024-09-10T08:57:07.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25cfef7640 1 Processor -- start 2024-09-10T08:57:07.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25cfef7640 1 -- start start 2024-09-10T08:57:07.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25cfef7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25c8101470 0x7f25c81990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25cfef7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c8101e20 0x7f25c8199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25cfef7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25c8105de0 0x7f25c81a4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25cfef7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f25c8110cf0 con 0x7f25c8105de0 2024-09-10T08:57:07.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25cfef7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f25c8110b70 con 0x7f25c8101470 2024-09-10T08:57:07.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25cfef7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f25c8110e70 con 0x7f25c8101e20 2024-09-10T08:57:07.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25ce494640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25c8105de0 0x7f25c81a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25ce494640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25c8105de0 0x7f25c81a4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35648/0 (socket says 172.21.15.172:35648) 2024-09-10T08:57:07.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25ce494640 1 -- 172.21.15.172:0/2335835248 learned_addr learned my addr 172.21.15.172:0/2335835248 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:07.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25cdc93640 1 --2- 172.21.15.172:0/2335835248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25c8101470 0x7f25c81990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25cd492640 1 --2- 172.21.15.172:0/2335835248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c8101e20 0x7f25c8199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25cdc93640 1 -- 172.21.15.172:0/2335835248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c8101e20 msgr2=0x7f25c8199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:07.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25cdc93640 1 --2- 172.21.15.172:0/2335835248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c8101e20 0x7f25c8199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25cdc93640 1 -- 172.21.15.172:0/2335835248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25c8105de0 msgr2=0x7f25c81a4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:07.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25cdc93640 1 --2- 172.21.15.172:0/2335835248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25c8105de0 0x7f25c81a4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25cdc93640 1 -- 172.21.15.172:0/2335835248 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c81a46b0 con 0x7f25c8101470 2024-09-10T08:57:07.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.118+0000 7f25ce494640 1 --2- 172.21.15.172:0/2335835248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25c8105de0 0x7f25c81a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:07.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.119+0000 7f25cdc93640 1 --2- 172.21.15.172:0/2335835248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25c8101470 0x7f25c81990f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f25c400d6d0 tx=0x7f25c400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:07.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.119+0000 7f25beffd640 1 -- 172.21.15.172:0/2335835248 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4017070 con 0x7f25c8101470 2024-09-10T08:57:07.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.119+0000 7f25cfef7640 1 -- 172.21.15.172:0/2335835248 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c8196ee0 con 0x7f25c8101470 2024-09-10T08:57:07.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.119+0000 7f25beffd640 1 -- 172.21.15.172:0/2335835248 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25c4009820 con 0x7f25c8101470 2024-09-10T08:57:07.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.119+0000 7f25beffd640 1 -- 172.21.15.172:0/2335835248 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4004c40 con 0x7f25c8101470 2024-09-10T08:57:07.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.119+0000 7f25cfef7640 1 -- 172.21.15.172:0/2335835248 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c8197420 con 0x7f25c8101470 2024-09-10T08:57:07.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.122+0000 7f25beffd640 1 -- 172.21.15.172:0/2335835248 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f25c40099d0 con 0x7f25c8101470 2024-09-10T08:57:07.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.122+0000 7f25d22d0740 1 Processor -- start 2024-09-10T08:57:07.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.122+0000 7f25beffd640 1 --2- 172.21.15.172:0/2335835248 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f25b40a2b80 0x7f25b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.122+0000 7f25beffd640 1 -- 172.21.15.172:0/2335835248 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f25c40cedf0 con 0x7f25c8101470 2024-09-10T08:57:07.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.122+0000 7f25cd492640 1 --2- 172.21.15.172:0/2335835248 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f25b40a2b80 0x7f25b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.122+0000 7f25d22d0740 1 -- start start 2024-09-10T08:57:07.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.123+0000 7f25d22d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a3c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.123+0000 7f25d22d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a3c5d0 0x5645e9a3ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.123+0000 7f25d22d0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a47370 0x5645e9a49760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.123+0000 7f25d22d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5645e99a22f0 con 0x5645e9a3c5d0 2024-09-10T08:57:07.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.123+0000 7f25d22d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5645e9949cf0 con 0x5645e9a3db40 2024-09-10T08:57:07.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.123+0000 7f25d22d0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5645e999e2b0 con 0x5645e9a47370 2024-09-10T08:57:07.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.123+0000 7f25cd492640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a3c5d0 0x5645e9a3ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.123+0000 7f25cd492640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a3c5d0 0x5645e9a3ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35664/0 (socket says 172.21.15.172:35664) 2024-09-10T08:57:07.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.123+0000 7f25cd492640 1 -- 172.21.15.172:0/273992089 learned_addr learned my addr 172.21.15.172:0/273992089 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:07.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.123+0000 7f25cdc93640 1 --2- 172.21.15.172:0/273992089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a3c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.123+0000 7f25cd492640 1 --2- 172.21.15.172:0/2335835248 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f25b40a2b80 0x7f25b40a5040 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f25c8100c90 tx=0x7f25b8012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:07.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.123+0000 7f25ce494640 1 --2- 172.21.15.172:0/273992089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a47370 0x5645e9a49760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.123+0000 7f25cdc93640 1 -- 172.21.15.172:0/273992089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a47370 msgr2=0x5645e9a49760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:07.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.123+0000 7f25cdc93640 1 --2- 172.21.15.172:0/273992089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a47370 0x5645e9a49760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.123+0000 7f25cdc93640 1 -- 172.21.15.172:0/273992089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a3c5d0 msgr2=0x5645e9a3ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:07.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.123+0000 7f25cdc93640 1 --2- 172.21.15.172:0/273992089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a3c5d0 0x5645e9a3ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.123+0000 7f25cdc93640 1 -- 172.21.15.172:0/273992089 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645e992e3a0 con 0x5645e9a3db40 2024-09-10T08:57:07.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.124+0000 7f25cdc93640 1 --2- 172.21.15.172:0/273992089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a3c090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f25c400d6a0 tx=0x7f25c4009e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e037c047d785e34b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:07.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.124+0000 7f25b3fff640 1 -- 172.21.15.172:0/273992089 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4020050 con 0x5645e9a3db40 2024-09-10T08:57:07.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.124+0000 7f25b3fff640 1 -- 172.21.15.172:0/273992089 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25c40a3f60 con 0x5645e9a3db40 2024-09-10T08:57:07.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.124+0000 7f25b3fff640 1 -- 172.21.15.172:0/273992089 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c40bb3a0 con 0x5645e9a3db40 2024-09-10T08:57:07.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.124+0000 7f25d22d0740 1 -- 172.21.15.172:0/273992089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 msgr2=0x5645e9a3c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:07.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.124+0000 7f25d22d0740 1 --2- 172.21.15.172:0/273992089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a3c090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f25c400d6a0 tx=0x7f25c4009e80 comp rx=0 tx=0).stop 2024-09-10T08:57:07.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.125+0000 7f25d22d0740 1 -- 172.21.15.172:0/273992089 shutdown_connections 2024-09-10T08:57:07.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.125+0000 7f25d22d0740 1 --2- 172.21.15.172:0/273992089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a47370 0x5645e9a49760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.125+0000 7f25d22d0740 1 --2- 172.21.15.172:0/273992089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a3c5d0 0x5645e9a3ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.125+0000 7f25d22d0740 1 --2- 172.21.15.172:0/273992089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a3c090 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.125+0000 7f25d22d0740 1 -- 172.21.15.172:0/273992089 >> 172.21.15.172:0/273992089 conn(0x5645e9a05310 msgr2=0x5645e9a38c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:07.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.125+0000 7f25d22d0740 1 -- 172.21.15.172:0/273992089 shutdown_connections 2024-09-10T08:57:07.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.125+0000 7f25d22d0740 1 -- 172.21.15.172:0/273992089 wait complete. 2024-09-10T08:57:07.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.125+0000 7f25d22d0740 1 -- start start 2024-09-10T08:57:07.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.125+0000 7f25d22d0740 1 Processor -- start 2024-09-10T08:57:07.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25d22d0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a3c5d0 0x5645e9a46da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25d22d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a44050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25d22d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a47370 0x5645e9a447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25ce494640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a47370 0x5645e9a447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25cd492640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a44050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25ce494640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a47370 0x5645e9a447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35676/0 (socket says 172.21.15.172:35676) 2024-09-10T08:57:07.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25ce494640 1 -- 172.21.15.172:0/3858065961 learned_addr learned my addr 172.21.15.172:0/3858065961 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:07.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25d22d0740 1 -- 172.21.15.172:0/3858065961 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5645e9a49fb0 con 0x5645e9a47370 2024-09-10T08:57:07.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25d22d0740 1 -- 172.21.15.172:0/3858065961 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5645e995d060 con 0x5645e9a3db40 2024-09-10T08:57:07.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25d22d0740 1 -- 172.21.15.172:0/3858065961 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5645e9a4a130 con 0x5645e9a3c5d0 2024-09-10T08:57:07.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25cdc93640 1 --2- 172.21.15.172:0/3858065961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a3c5d0 0x5645e9a46da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25d22d0740 10 client.0 _ll_register_callbacks cb 0x5645e96b08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:07.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25cdc93640 1 -- 172.21.15.172:0/3858065961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 msgr2=0x5645e9a44050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:07.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25cdc93640 1 --2- 172.21.15.172:0/3858065961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a44050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25cdc93640 1 -- 172.21.15.172:0/3858065961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a47370 msgr2=0x5645e9a447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:07.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25cdc93640 1 --2- 172.21.15.172:0/3858065961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a47370 0x5645e9a447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25cdc93640 1 -- 172.21.15.172:0/3858065961 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645e97133c0 con 0x5645e9a3c5d0 2024-09-10T08:57:07.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25cd492640 1 --2- 172.21.15.172:0/3858065961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a44050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:07.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25cdc93640 1 --2- 172.21.15.172:0/3858065961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a3c5d0 0x5645e9a46da0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x5645e9a3e170 tx=0x7f25c4004de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c436020d5aac9882 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:07.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.126+0000 7f25b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:57:07.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.127+0000 7f25b17fa640 1 -- 172.21.15.172:0/3858065961 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c40b9060 con 0x5645e9a3c5d0 2024-09-10T08:57:07.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.127+0000 7f25b17fa640 1 -- 172.21.15.172:0/3858065961 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f25c40b4270 con 0x5645e9a3c5d0 2024-09-10T08:57:07.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.127+0000 7f25d22d0740 1 -- 172.21.15.172:0/3858065961 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25c40a3f60 con 0x5645e9a3c5d0 2024-09-10T08:57:07.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.127+0000 7f25b17fa640 1 -- 172.21.15.172:0/3858065961 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f25c4081050 con 0x5645e9a3c5d0 2024-09-10T08:57:07.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.127+0000 7f25d22d0740 10 client.27511 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:07.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.127+0000 7f25d22d0740 1 -- 172.21.15.172:0/3858065961 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5645e9a49fb0 con 0x5645e9a3c5d0 2024-09-10T08:57:07.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.127+0000 7f25b17fa640 1 -- 172.21.15.172:0/3858065961 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25c40bb0b0 con 0x5645e9a3c5d0 2024-09-10T08:57:07.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.127+0000 7f25b17fa640 1 -- 172.21.15.172:0/3858065961 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f25c4084390 con 0x5645e9a3c5d0 2024-09-10T08:57:07.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.127+0000 7f25d22d0740 10 client.27511 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:07.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.127+0000 7f25d22d0740 10 client.27511 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:57:07.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.127+0000 7f25d22d0740 20 client.27511 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:07.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.127+0000 7f25d22d0740 1 --2- 172.21.15.172:0/3858065961 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x5645e9a56bf0 0x5645e9a58fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.127+0000 7f25d22d0740 4 client.27511 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:57:07.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.127+0000 7f25d22d0740 1 -- 172.21.15.172:0/3858065961 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f25c40a3f60 con 0x5645e9a56bf0 2024-09-10T08:57:07.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.128+0000 7f25cd492640 1 --2- 172.21.15.172:0/3858065961 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x5645e9a56bf0 0x5645e9a58fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.128+0000 7f25b17fa640 10 client.27511 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:57:07.134 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.128+0000 7f25cd492640 1 --2- 172.21.15.172:0/3858065961 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x5645e9a56bf0 0x5645e9a58fe0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b5fe5fefc6a2533f server_cookie=a852b791fbdb800d in_seq=0 out_seq=0 2024-09-10T08:57:07.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.129+0000 7f25b17fa640 1 -- 172.21.15.172:0/3858065961 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f25c40a3f60 con 0x5645e9a56bf0 2024-09-10T08:57:07.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.129+0000 7f25b17fa640 10 client.27511 handle_command_reply: tid=0 2024-09-10T08:57:07.135 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.129+0000 7f25d22d0740 1 client.27511 shutdown 2024-09-10T08:57:07.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.129+0000 7f25d22d0740 1 -- 172.21.15.172:0/3858065961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a3c5d0 msgr2=0x5645e9a46da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:07.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.129+0000 7f25d22d0740 1 --2- 172.21.15.172:0/3858065961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a3c5d0 0x5645e9a46da0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x5645e9a3e170 tx=0x7f25c4004de0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.129+0000 7f25d22d0740 1 -- 172.21.15.172:0/3858065961 shutdown_connections 2024-09-10T08:57:07.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.129+0000 7f25d22d0740 1 --2- 172.21.15.172:0/3858065961 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x5645e9a56bf0 0x5645e9a58fe0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.129+0000 7f25d22d0740 1 --2- 172.21.15.172:0/3858065961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a47370 0x5645e9a447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.129+0000 7f25d22d0740 1 --2- 172.21.15.172:0/3858065961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a44050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.129+0000 7f25b17fa640 0 client.27511 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:57:07.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.129+0000 7f25d22d0740 1 --2- 172.21.15.172:0/3858065961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a3c5d0 0x5645e9a46da0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.129+0000 7f25d22d0740 1 -- 172.21.15.172:0/3858065961 >> 172.21.15.172:0/3858065961 conn(0x5645e9a05310 msgr2=0x5645e9a3e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:07.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.129+0000 7f25d22d0740 1 -- 172.21.15.172:0/3858065961 shutdown_connections 2024-09-10T08:57:07.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.129+0000 7f25d22d0740 1 -- 172.21.15.172:0/3858065961 wait complete. 2024-09-10T08:57:07.136 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.129+0000 7f25d22d0740 20 client.27511 trim_cache size 0 max 16384 2024-09-10T08:57:07.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.143+0000 7f25d22d0740 1 Processor -- start 2024-09-10T08:57:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.143+0000 7f25d22d0740 1 -- start start 2024-09-10T08:57:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.143+0000 7f25d22d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a3c5d0 0x5645e9a43910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.143+0000 7f25d22d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a5e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.143+0000 7f25d22d0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a47370 0x5645e9a5ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.143+0000 7f25d22d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81ac8b0 con 0x5645e9a3c5d0 2024-09-10T08:57:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.143+0000 7f25d22d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81ae450 con 0x5645e9a3db40 2024-09-10T08:57:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.143+0000 7f25d22d0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81acf80 con 0x5645e9a47370 2024-09-10T08:57:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.143+0000 7f25cdc93640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a3c5d0 0x5645e9a43910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.143+0000 7f25cdc93640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a3c5d0 0x5645e9a43910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35692/0 (socket says 172.21.15.172:35692) 2024-09-10T08:57:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.143+0000 7f25cdc93640 1 -- 172.21.15.172:0/2672069948 learned_addr learned my addr 172.21.15.172:0/2672069948 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:07.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.143+0000 7f25cd492640 1 --2- 172.21.15.172:0/2672069948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a5e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.144+0000 7f25ce494640 1 --2- 172.21.15.172:0/2672069948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a47370 0x5645e9a5ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.144+0000 7f25cdc93640 1 -- 172.21.15.172:0/2672069948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a47370 msgr2=0x5645e9a5ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.144+0000 7f25cdc93640 1 --2- 172.21.15.172:0/2672069948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a47370 0x5645e9a5ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.144+0000 7f25cdc93640 1 -- 172.21.15.172:0/2672069948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 msgr2=0x5645e9a5e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.144+0000 7f25cdc93640 1 --2- 172.21.15.172:0/2672069948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a5e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.144+0000 7f25cdc93640 1 -- 172.21.15.172:0/2672069948 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645e9a44b50 con 0x5645e9a3c5d0 2024-09-10T08:57:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.144+0000 7f25cd492640 1 --2- 172.21.15.172:0/2672069948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a5e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.144+0000 7f25cdc93640 1 --2- 172.21.15.172:0/2672069948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a3c5d0 0x5645e9a43910 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f25c40977b0 tx=0x7f25c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77574467273c7f14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.144+0000 7f25b37fe640 1 -- 172.21.15.172:0/2672069948 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c4020070 con 0x5645e9a3c5d0 2024-09-10T08:57:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.144+0000 7f25b37fe640 1 -- 172.21.15.172:0/2672069948 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25c40a3f60 con 0x5645e9a3c5d0 2024-09-10T08:57:07.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.144+0000 7f25b37fe640 1 -- 172.21.15.172:0/2672069948 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c40b9050 con 0x5645e9a3c5d0 2024-09-10T08:57:07.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.145+0000 7f25d22d0740 1 -- 172.21.15.172:0/2672069948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a3c5d0 msgr2=0x5645e9a43910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:07.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.145+0000 7f25d22d0740 1 --2- 172.21.15.172:0/2672069948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a3c5d0 0x5645e9a43910 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f25c40977b0 tx=0x7f25c40046c0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.145+0000 7f25d22d0740 1 -- 172.21.15.172:0/2672069948 shutdown_connections 2024-09-10T08:57:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.145+0000 7f25d22d0740 1 --2- 172.21.15.172:0/2672069948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a47370 0x5645e9a5ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.145+0000 7f25d22d0740 1 --2- 172.21.15.172:0/2672069948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a5e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.145+0000 7f25d22d0740 1 --2- 172.21.15.172:0/2672069948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a3c5d0 0x5645e9a43910 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.145+0000 7f25d22d0740 1 -- 172.21.15.172:0/2672069948 >> 172.21.15.172:0/2672069948 conn(0x5645e9a05310 msgr2=0x5645e9a42920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.145+0000 7f25d22d0740 1 -- 172.21.15.172:0/2672069948 shutdown_connections 2024-09-10T08:57:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.145+0000 7f25d22d0740 1 -- 172.21.15.172:0/2672069948 wait complete. 2024-09-10T08:57:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.146+0000 7f25d22d0740 1 -- start start 2024-09-10T08:57:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.146+0000 7f25d22d0740 1 Processor -- start 2024-09-10T08:57:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.146+0000 7f25d22d0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a3c5d0 0x5645e9a4f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.146+0000 7f25d22d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a4fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.146+0000 7f25d22d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a47370 0x5645e9a40c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.147+0000 7f25cdc93640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a3c5d0 0x5645e9a4f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.147+0000 7f25cdc93640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a3c5d0 0x5645e9a4f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:52610/0 (socket says 172.21.15.172:52610) 2024-09-10T08:57:07.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.147+0000 7f25cd492640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a4fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.147+0000 7f25cd492640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a4fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:51468/0 (socket says 172.21.15.172:51468) 2024-09-10T08:57:07.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.147+0000 7f25ce494640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a47370 0x5645e9a40c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.147+0000 7f25ce494640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a47370 0x5645e9a40c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35700/0 (socket says 172.21.15.172:35700) 2024-09-10T08:57:07.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.147+0000 7f25cdc93640 1 -- 172.21.15.172:0/4217194884 learned_addr learned my addr 172.21.15.172:0/4217194884 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:07.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.147+0000 7f25d22d0740 1 -- 172.21.15.172:0/4217194884 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81add80 con 0x5645e9a47370 2024-09-10T08:57:07.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.147+0000 7f25d22d0740 1 -- 172.21.15.172:0/4217194884 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81ad6b0 con 0x5645e9a3db40 2024-09-10T08:57:07.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.147+0000 7f25d22d0740 1 -- 172.21.15.172:0/4217194884 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81aa890 con 0x5645e9a3c5d0 2024-09-10T08:57:07.152 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.147+0000 7f25d22d0740 10 client.0 _ll_register_callbacks cb 0x7f25b8010aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:07.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.147+0000 7f25cd492640 1 -- 172.21.15.172:0/4217194884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a3c5d0 msgr2=0x5645e9a4f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:07.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.147+0000 7f25cd492640 1 --2- 172.21.15.172:0/4217194884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a3c5d0 0x5645e9a4f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.147+0000 7f25cd492640 1 -- 172.21.15.172:0/4217194884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a47370 msgr2=0x5645e9a40c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:07.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.147+0000 7f25cd492640 1 --2- 172.21.15.172:0/4217194884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a47370 0x5645e9a40c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.147+0000 7f25cd492640 1 -- 172.21.15.172:0/4217194884 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645e9a41480 con 0x5645e9a3db40 2024-09-10T08:57:07.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.148+0000 7f25cd492640 1 --2- 172.21.15.172:0/4217194884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a4fc70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x5645e9a45690 tx=0x7f25b80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d6d957095266581 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:07.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.148+0000 7f25b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:57:07.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.148+0000 7f25b17fa640 1 -- 172.21.15.172:0/4217194884 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8022040 con 0x5645e9a3db40 2024-09-10T08:57:07.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.148+0000 7f25d22d0740 1 -- 172.21.15.172:0/4217194884 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25c40a3f60 con 0x5645e9a3db40 2024-09-10T08:57:07.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.148+0000 7f25b17fa640 1 -- 172.21.15.172:0/4217194884 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f25b8005220 con 0x5645e9a3db40 2024-09-10T08:57:07.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.148+0000 7f25b17fa640 1 -- 172.21.15.172:0/4217194884 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f25b8005420 con 0x5645e9a3db40 2024-09-10T08:57:07.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.148+0000 7f25d22d0740 10 client.27581 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:07.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.148+0000 7f25d22d0740 1 -- 172.21.15.172:0/4217194884 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5645e9ac1380 con 0x5645e9a3db40 2024-09-10T08:57:07.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.148+0000 7f25b17fa640 1 -- 172.21.15.172:0/4217194884 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25b8034070 con 0x5645e9a3db40 2024-09-10T08:57:07.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.148+0000 7f25b17fa640 1 -- 172.21.15.172:0/4217194884 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f25b8004a50 con 0x5645e9a3db40 2024-09-10T08:57:07.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.148+0000 7f25d22d0740 10 client.27581 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:07.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.148+0000 7f25d22d0740 10 client.27581 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:57:07.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.148+0000 7f25d22d0740 20 client.27581 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:07.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.149+0000 7f25d22d0740 1 --2- 172.21.15.172:0/4217194884 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x5645e9ac1b30 0x5645e9ac3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:07.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.149+0000 7f25d22d0740 4 client.27581 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:57:07.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.149+0000 7f25d22d0740 1 -- 172.21.15.172:0/4217194884 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f25c40a3f60 con 0x5645e9ac1b30 2024-09-10T08:57:07.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.149+0000 7f25cdc93640 1 --2- 172.21.15.172:0/4217194884 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x5645e9ac1b30 0x5645e9ac3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:07.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.150+0000 7f25b17fa640 10 client.27581 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:57:07.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.150+0000 7f25cdc93640 1 --2- 172.21.15.172:0/4217194884 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x5645e9ac1b30 0x5645e9ac3f20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9d3c88e56dc549e server_cookie=bcdff960b55d87f8 in_seq=0 out_seq=0 2024-09-10T08:57:07.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.150+0000 7f25b17fa640 1 -- 172.21.15.172:0/4217194884 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f25c40a3f60 con 0x5645e9ac1b30 2024-09-10T08:57:07.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.150+0000 7f25b17fa640 10 client.27581 handle_command_reply: tid=0 2024-09-10T08:57:07.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.150+0000 7f25d22d0740 1 client.27581 shutdown 2024-09-10T08:57:07.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.150+0000 7f25d22d0740 1 -- 172.21.15.172:0/4217194884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 msgr2=0x5645e9a4fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:07.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.150+0000 7f25d22d0740 1 --2- 172.21.15.172:0/4217194884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a4fc70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x5645e9a45690 tx=0x7f25b80046c0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.150+0000 7f25d22d0740 1 -- 172.21.15.172:0/4217194884 shutdown_connections 2024-09-10T08:57:07.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.150+0000 7f25d22d0740 1 --2- 172.21.15.172:0/4217194884 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x5645e9ac1b30 0x5645e9ac3f20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.150+0000 7f25b17fa640 0 client.27581 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:57:07.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.150+0000 7f25d22d0740 1 --2- 172.21.15.172:0/4217194884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645e9a47370 0x5645e9a40c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.150+0000 7f25d22d0740 1 --2- 172.21.15.172:0/4217194884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645e9a3db40 0x5645e9a4fc70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.150+0000 7f25d22d0740 1 --2- 172.21.15.172:0/4217194884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645e9a3c5d0 0x5645e9a4f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.150+0000 7f25d22d0740 1 -- 172.21.15.172:0/4217194884 >> 172.21.15.172:0/4217194884 conn(0x5645e9a05310 msgr2=0x5645e9a42d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:07.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.150+0000 7f25d22d0740 1 -- 172.21.15.172:0/4217194884 shutdown_connections 2024-09-10T08:57:07.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.151+0000 7f25d22d0740 1 -- 172.21.15.172:0/4217194884 wait complete. 2024-09-10T08:57:07.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.151+0000 7f25d22d0740 20 client.27581 trim_cache size 0 max 16384 2024-09-10T08:57:07.157 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:57:07.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.151+0000 7f25cfef7640 1 -- 172.21.15.172:0/2335835248 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f25b40a2b80 msgr2=0x7f25b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:07.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.151+0000 7f25cfef7640 1 --2- 172.21.15.172:0/2335835248 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f25b40a2b80 0x7f25b40a5040 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f25c8100c90 tx=0x7f25b8012040 comp rx=0 tx=0).stop 2024-09-10T08:57:07.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.151+0000 7f25cfef7640 1 -- 172.21.15.172:0/2335835248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25c8101470 msgr2=0x7f25c81990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:07.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.151+0000 7f25cfef7640 1 --2- 172.21.15.172:0/2335835248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25c8101470 0x7f25c81990f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f25c400d6d0 tx=0x7f25c400db90 comp rx=0 tx=0).stop 2024-09-10T08:57:07.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.151+0000 7f25cfef7640 1 -- 172.21.15.172:0/2335835248 shutdown_connections 2024-09-10T08:57:07.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.151+0000 7f25cfef7640 1 --2- 172.21.15.172:0/2335835248 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f25b40a2b80 0x7f25b40a5040 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.151+0000 7f25cfef7640 1 --2- 172.21.15.172:0/2335835248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25c8105de0 0x7f25c81a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.158 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.151+0000 7f25cfef7640 1 --2- 172.21.15.172:0/2335835248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c8101e20 0x7f25c8199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.151+0000 7f25cfef7640 1 --2- 172.21.15.172:0/2335835248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25c8101470 0x7f25c81990f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:07.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.151+0000 7f25cfef7640 1 -- 172.21.15.172:0/2335835248 >> 172.21.15.172:0/2335835248 conn(0x7f25c8077230 msgr2=0x7f25c80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:07.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.152+0000 7f25cfef7640 1 -- 172.21.15.172:0/2335835248 shutdown_connections 2024-09-10T08:57:07.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:07.152+0000 7f25cfef7640 1 -- 172.21.15.172:0/2335835248 wait complete. 2024-09-10T08:57:07.741 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:07 smithi172 ceph-mon[30854]: pgmap v677: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-09-10T08:57:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:07 smithi174 ceph-mon[34068]: pgmap v677: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-09-10T08:57:07.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:07 smithi191 ceph-mon[35314]: pgmap v677: 129 pgs: 129 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-09-10T08:57:08.050 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:57:08.348 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:57:08.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.875+0000 7fea8a5a7640 1 -- 172.21.15.172:0/1351356984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea84108290 msgr2=0x7fea8410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:08.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.875+0000 7fea8a5a7640 1 --2- 172.21.15.172:0/1351356984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea84108290 0x7fea8410ec30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009cd0 tx=0x7fea7c02a8e0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.875+0000 7fea8a5a7640 1 -- 172.21.15.172:0/1351356984 shutdown_connections 2024-09-10T08:57:08.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.875+0000 7fea8a5a7640 1 --2- 172.21.15.172:0/1351356984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea84108290 0x7fea8410ec30 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.875+0000 7fea8a5a7640 1 --2- 172.21.15.172:0/1351356984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea84103740 0x7fea84107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.875+0000 7fea8a5a7640 1 --2- 172.21.15.172:0/1351356984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea84102d90 0x7fea84103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.875+0000 7fea8a5a7640 1 -- 172.21.15.172:0/1351356984 >> 172.21.15.172:0/1351356984 conn(0x7fea840fc7d0 msgr2=0x7fea840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.875+0000 7fea8a5a7640 1 -- 172.21.15.172:0/1351356984 shutdown_connections 2024-09-10T08:57:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.875+0000 7fea8a5a7640 1 -- 172.21.15.172:0/1351356984 wait complete. 2024-09-10T08:57:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.876+0000 7fea8a5a7640 1 Processor -- start 2024-09-10T08:57:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.876+0000 7fea8a5a7640 1 -- start start 2024-09-10T08:57:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.876+0000 7fea8a5a7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea84102d90 0x7fea8419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.876+0000 7fea8a5a7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea84103740 0x7fea8419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.876+0000 7fea8a5a7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea84108290 0x7fea8419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.876+0000 7fea8a5a7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fea84111f40 con 0x7fea84102d90 2024-09-10T08:57:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.876+0000 7fea8a5a7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea84111dc0 con 0x7fea84103740 2024-09-10T08:57:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.876+0000 7fea8a5a7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fea841120c0 con 0x7fea84108290 2024-09-10T08:57:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.877+0000 7fea83fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea84102d90 0x7fea8419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.877+0000 7fea83fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea84102d90 0x7fea8419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35716/0 (socket says 172.21.15.172:35716) 2024-09-10T08:57:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.877+0000 7fea83fff640 1 -- 172.21.15.172:0/3832437372 learned_addr learned my addr 172.21.15.172:0/3832437372 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.877+0000 7fea837fe640 1 --2- 172.21.15.172:0/3832437372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea84103740 0x7fea8419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.877+0000 7fea88b44640 1 --2- 172.21.15.172:0/3832437372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea84108290 0x7fea8419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.877+0000 7fea83fff640 1 -- 172.21.15.172:0/3832437372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea84108290 msgr2=0x7fea8419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:08.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.877+0000 7fea83fff640 1 --2- 172.21.15.172:0/3832437372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea84108290 0x7fea8419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.877+0000 7fea83fff640 1 -- 172.21.15.172:0/3832437372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea84103740 msgr2=0x7fea8419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.877+0000 7fea83fff640 1 --2- 172.21.15.172:0/3832437372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea84103740 0x7fea8419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.877+0000 7fea83fff640 1 -- 172.21.15.172:0/3832437372 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8419f570 con 0x7fea84102d90 2024-09-10T08:57:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.877+0000 7fea83fff640 1 --2- 172.21.15.172:0/3832437372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea84102d90 0x7fea8419a5c0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fea7400c970 tx=0x7fea7400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.877+0000 7fea817fa640 1 -- 172.21.15.172:0/3832437372 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74007bb0 con 0x7fea84102d90 2024-09-10T08:57:08.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.877+0000 7fea817fa640 1 -- 172.21.15.172:0/3832437372 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea74007d50 con 0x7fea84102d90 2024-09-10T08:57:08.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.877+0000 7fea8a5a7640 1 -- 172.21.15.172:0/3832437372 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8419f860 con 0x7fea84102d90 2024-09-10T08:57:08.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.877+0000 7fea817fa640 1 -- 172.21.15.172:0/3832437372 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74004e50 con 0x7fea84102d90 2024-09-10T08:57:08.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.877+0000 7fea8a5a7640 1 -- 172.21.15.172:0/3832437372 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea841a7140 con 0x7fea84102d90 2024-09-10T08:57:08.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.880+0000 7fea817fa640 1 -- 172.21.15.172:0/3832437372 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fea74005020 con 0x7fea84102d90 2024-09-10T08:57:08.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.880+0000 7fea8c980740 1 Processor -- start 2024-09-10T08:57:08.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea8c980740 1 -- start start 2024-09-10T08:57:08.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea8c980740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dfefb40 0x560c0dfee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea8c980740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dfee5d0 0x560c0dfeea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea8c980740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dff9370 0x560c0dffb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea817fa640 1 --2- 172.21.15.172:0/3832437372 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea6c0a2d20 0x7fea6c0a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea8c980740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560c0df542f0 con 0x560c0dff9370 2024-09-10T08:57:08.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea8c980740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560c0defbcf0 con 0x560c0dfefb40 2024-09-10T08:57:08.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea8c980740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560c0df502b0 con 0x560c0dfee5d0 2024-09-10T08:57:08.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea83fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dfee5d0 0x560c0dfeea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea88b44640 1 --2- 172.21.15.172:0/3832437372 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea6c0a2d20 0x7fea6c0a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea83fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dfee5d0 0x560c0dfeea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:52618/0 (socket says 172.21.15.172:52618) 2024-09-10T08:57:08.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea83fff640 1 -- 172.21.15.172:0/3032825941 learned_addr learned my addr 172.21.15.172:0/3032825941 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:08.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea817fa640 1 -- 172.21.15.172:0/3832437372 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fea740cf020 con 0x7fea84102d90 2024-09-10T08:57:08.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea837fe640 1 --2- 172.21.15.172:0/3032825941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dff9370 0x560c0dffb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea88b44640 1 --2- 172.21.15.172:0/3032825941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dfefb40 0x560c0dfee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea83fff640 1 -- 172.21.15.172:0/3032825941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dfefb40 msgr2=0x560c0dfee090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:08.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea83fff640 1 --2- 172.21.15.172:0/3032825941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dfefb40 0x560c0dfee090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea83fff640 1 -- 172.21.15.172:0/3032825941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dff9370 msgr2=0x560c0dffb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:08.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea83fff640 1 --2- 172.21.15.172:0/3032825941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dff9370 0x560c0dffb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea83fff640 1 -- 172.21.15.172:0/3032825941 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c0dee03a0 con 0x560c0dfee5d0 2024-09-10T08:57:08.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea88b44640 1 --2- 172.21.15.172:0/3032825941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dfefb40 0x560c0dfee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:57:08.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.881+0000 7fea837fe640 1 --2- 172.21.15.172:0/3032825941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dff9370 0x560c0dffb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:57:08.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.882+0000 7fea83fff640 1 --2- 172.21.15.172:0/3032825941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dfee5d0 0x560c0dfeea10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fea7400c940 tx=0x7fea7406e5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f156701c7e5ec76d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:08.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.882+0000 7fea88b44640 1 --2- 172.21.15.172:0/3832437372 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea6c0a2d20 0x7fea6c0a51e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009430 tx=0x7fea7c009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:08.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.882+0000 7fea6a7fc640 1 -- 172.21.15.172:0/3032825941 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7401d070 con 0x560c0dfee5d0 2024-09-10T08:57:08.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.882+0000 7fea6a7fc640 1 -- 172.21.15.172:0/3032825941 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea7409bc20 con 0x560c0dfee5d0 2024-09-10T08:57:08.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.882+0000 7fea6a7fc640 1 -- 172.21.15.172:0/3032825941 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74005560 con 0x560c0dfee5d0 2024-09-10T08:57:08.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.883+0000 7fea8c980740 1 -- 172.21.15.172:0/3032825941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dfee5d0 msgr2=0x560c0dfeea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:08.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.883+0000 7fea8c980740 1 --2- 172.21.15.172:0/3032825941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dfee5d0 0x560c0dfeea10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fea7400c940 tx=0x7fea7406e5e0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.883+0000 7fea8c980740 1 -- 172.21.15.172:0/3032825941 shutdown_connections 2024-09-10T08:57:08.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.883+0000 7fea8c980740 1 --2- 172.21.15.172:0/3032825941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dff9370 0x560c0dffb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.883+0000 7fea8c980740 1 --2- 172.21.15.172:0/3032825941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dfee5d0 0x560c0dfeea10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.883+0000 7fea8c980740 1 --2- 172.21.15.172:0/3032825941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dfefb40 0x560c0dfee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.883+0000 7fea8c980740 1 -- 172.21.15.172:0/3032825941 >> 172.21.15.172:0/3032825941 conn(0x560c0dfb7310 msgr2=0x560c0dfeac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:08.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.883+0000 7fea8c980740 1 -- 172.21.15.172:0/3032825941 shutdown_connections 2024-09-10T08:57:08.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.883+0000 7fea8c980740 1 -- 172.21.15.172:0/3032825941 wait complete. 2024-09-10T08:57:08.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.883+0000 7fea8c980740 1 -- start start 2024-09-10T08:57:08.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.883+0000 7fea8c980740 1 Processor -- start 2024-09-10T08:57:08.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.884+0000 7fea8c980740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dfee5d0 0x560c0dff8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.884+0000 7fea8c980740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dfefb40 0x560c0dff6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.884+0000 7fea83fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dfee5d0 0x560c0dff8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.884+0000 7fea83fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dfee5d0 0x560c0dff8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35738/0 (socket says 172.21.15.172:35738) 2024-09-10T08:57:08.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.884+0000 7fea83fff640 1 -- 172.21.15.172:0/2670387256 learned_addr learned my addr 172.21.15.172:0/2670387256 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:08.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.884+0000 7fea8c980740 1 --2- 172.21.15.172:0/2670387256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dff9370 0x560c0dff67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.884+0000 7fea8c980740 1 -- 172.21.15.172:0/2670387256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560c0dffc010 con 0x560c0dfee5d0 2024-09-10T08:57:08.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.884+0000 7fea8c980740 1 -- 172.21.15.172:0/2670387256 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560c0df0f060 con 0x560c0dff9370 2024-09-10T08:57:08.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.884+0000 7fea8c980740 1 -- 172.21.15.172:0/2670387256 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560c0dffc190 con 0x560c0dfefb40 2024-09-10T08:57:08.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.884+0000 7fea837fe640 1 --2- 172.21.15.172:0/2670387256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dfefb40 0x560c0dff6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.884+0000 7fea8c980740 10 client.0 _ll_register_callbacks cb 0x560c0dc628c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:08.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.884+0000 7fea88b44640 1 --2- 172.21.15.172:0/2670387256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dff9370 0x560c0dff67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.884+0000 7fea83fff640 1 -- 172.21.15.172:0/2670387256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dfefb40 msgr2=0x560c0dff6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:08.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.884+0000 7fea83fff640 1 --2- 172.21.15.172:0/2670387256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dfefb40 0x560c0dff6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.884+0000 7fea83fff640 1 -- 172.21.15.172:0/2670387256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dff9370 msgr2=0x560c0dff67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:08.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.884+0000 7fea83fff640 1 --2- 172.21.15.172:0/2670387256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dff9370 0x560c0dff67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.884+0000 7fea83fff640 1 -- 172.21.15.172:0/2670387256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c0dcc53c0 con 0x560c0dfee5d0 2024-09-10T08:57:08.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.885+0000 7fea88b44640 1 --2- 172.21.15.172:0/2670387256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dff9370 0x560c0dff67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:57:08.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.885+0000 7fea83fff640 1 --2- 172.21.15.172:0/2670387256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dfee5d0 0x560c0dff8de0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fea740ccb30 tx=0x7fea7409b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a863a1e3aa72bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:08.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.885+0000 7fea637fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:08.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.885+0000 7fea637fe640 1 -- 172.21.15.172:0/2670387256 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7401d070 con 0x560c0dfee5d0 2024-09-10T08:57:08.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.885+0000 7fea637fe640 1 -- 172.21.15.172:0/2670387256 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fea74099820 con 0x560c0dfee5d0 2024-09-10T08:57:08.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.885+0000 7fea8c980740 1 -- 172.21.15.172:0/2670387256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea7409bc20 con 0x560c0dfee5d0 2024-09-10T08:57:08.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.885+0000 7fea637fe640 1 -- 172.21.15.172:0/2670387256 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fea740a3d00 con 0x560c0dfee5d0 2024-09-10T08:57:08.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.885+0000 7fea8c980740 10 client.18690 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:08.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.885+0000 7fea8c980740 1 -- 172.21.15.172:0/2670387256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560c0e007b20 con 0x560c0dfee5d0 2024-09-10T08:57:08.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.885+0000 7fea637fe640 1 -- 172.21.15.172:0/2670387256 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea7409bc20 con 0x560c0dfee5d0 2024-09-10T08:57:08.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.885+0000 7fea637fe640 1 -- 172.21.15.172:0/2670387256 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fea740972e0 con 0x560c0dfee5d0 2024-09-10T08:57:08.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.886+0000 7fea8c980740 10 client.18690 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:08.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.886+0000 7fea8c980740 10 client.18690 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:57:08.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.886+0000 7fea8c980740 20 client.18690 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:08.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.886+0000 7fea8c980740 1 --2- 172.21.15.172:0/2670387256 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x560c0e008e70 0x560c0e00b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.886+0000 7fea8c980740 4 client.18690 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:57:08.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.886+0000 7fea8c980740 1 -- 172.21.15.172:0/2670387256 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c0df61860 con 0x560c0e008e70 2024-09-10T08:57:08.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.886+0000 7fea837fe640 1 --2- 172.21.15.172:0/2670387256 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x560c0e008e70 0x560c0e00b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.886+0000 7fea837fe640 1 --2- 172.21.15.172:0/2670387256 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x560c0e008e70 0x560c0e00b260 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb4bb95a63c0b790 server_cookie=98f4825d5565eff7 in_seq=0 out_seq=0 2024-09-10T08:57:08.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.886+0000 7fea637fe640 10 client.18690 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:57:08.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.887+0000 7fea637fe640 1 -- 172.21.15.172:0/2670387256 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c0df61860 con 0x560c0e008e70 2024-09-10T08:57:08.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.887+0000 7fea637fe640 10 client.18690 handle_command_reply: tid=0 2024-09-10T08:57:08.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.887+0000 7fea8c980740 1 client.18690 shutdown 2024-09-10T08:57:08.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.887+0000 7fea8c980740 1 -- 172.21.15.172:0/2670387256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dfee5d0 msgr2=0x560c0dff8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:08.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.887+0000 7fea8c980740 1 --2- 172.21.15.172:0/2670387256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dfee5d0 0x560c0dff8de0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fea740ccb30 tx=0x7fea7409b610 comp rx=0 tx=0).stop 2024-09-10T08:57:08.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.887+0000 7fea8c980740 1 -- 172.21.15.172:0/2670387256 shutdown_connections 2024-09-10T08:57:08.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.887+0000 7fea8c980740 1 --2- 172.21.15.172:0/2670387256 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x560c0e008e70 0x560c0e00b260 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.887+0000 7fea8c980740 1 --2- 172.21.15.172:0/2670387256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dff9370 0x560c0dff67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.887+0000 7fea8c980740 1 --2- 172.21.15.172:0/2670387256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dfefb40 0x560c0dff6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.887+0000 7fea8c980740 1 --2- 172.21.15.172:0/2670387256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dfee5d0 0x560c0dff8de0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.887+0000 7fea8c980740 1 -- 172.21.15.172:0/2670387256 >> 172.21.15.172:0/2670387256 conn(0x560c0dfb7310 msgr2=0x560c0dff0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:08.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.887+0000 7fea637fe640 0 client.18690 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:57:08.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.887+0000 7fea8c980740 1 -- 172.21.15.172:0/2670387256 shutdown_connections 2024-09-10T08:57:08.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.887+0000 7fea8c980740 1 -- 172.21.15.172:0/2670387256 wait complete. 2024-09-10T08:57:08.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.887+0000 7fea8c980740 20 client.18690 trim_cache size 0 max 16384 2024-09-10T08:57:08.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.900+0000 7fea8c980740 1 Processor -- start 2024-09-10T08:57:08.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.900+0000 7fea8c980740 1 -- start start 2024-09-10T08:57:08.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea8c980740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dfee5d0 0x560c0e0052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea8c980740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dfefb40 0x560c0dff5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea8c980740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dff9370 0x560c0e010800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea8c980740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fea841b0d90 con 0x560c0dfee5d0 2024-09-10T08:57:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea8c980740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea841b2930 con 0x560c0dfefb40 2024-09-10T08:57:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea8c980740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fea841b1460 con 0x560c0dff9370 2024-09-10T08:57:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea83fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dfee5d0 0x560c0e0052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea83fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dfee5d0 0x560c0e0052b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35740/0 (socket says 172.21.15.172:35740) 2024-09-10T08:57:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea83fff640 1 -- 172.21.15.172:0/3626186578 learned_addr learned my addr 172.21.15.172:0/3626186578 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea837fe640 1 --2- 172.21.15.172:0/3626186578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dfefb40 0x560c0dff5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea88b44640 1 --2- 172.21.15.172:0/3626186578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dff9370 0x560c0e010800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea83fff640 1 -- 172.21.15.172:0/3626186578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dff9370 msgr2=0x560c0e010800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea83fff640 1 --2- 172.21.15.172:0/3626186578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dff9370 0x560c0e010800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea83fff640 1 -- 172.21.15.172:0/3626186578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dfefb40 msgr2=0x560c0dff5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea83fff640 1 --2- 172.21.15.172:0/3626186578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dfefb40 0x560c0dff5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea83fff640 1 -- 172.21.15.172:0/3626186578 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c0dff1120 con 0x560c0dfee5d0 2024-09-10T08:57:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea837fe640 1 --2- 172.21.15.172:0/3626186578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dfefb40 0x560c0dff5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.901+0000 7fea83fff640 1 --2- 172.21.15.172:0/3626186578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dfee5d0 0x560c0e0052b0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fea74097ea0 tx=0x7fea740cc810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ce029478e1841a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.902+0000 7fea69ffb640 1 -- 172.21.15.172:0/3626186578 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7401d070 con 0x560c0dfee5d0 2024-09-10T08:57:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.902+0000 7fea69ffb640 1 -- 172.21.15.172:0/3626186578 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea7409bc20 con 0x560c0dfee5d0 2024-09-10T08:57:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.902+0000 7fea69ffb640 1 -- 172.21.15.172:0/3626186578 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74097270 con 0x560c0dfee5d0 2024-09-10T08:57:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.902+0000 7fea8c980740 1 -- 172.21.15.172:0/3626186578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dfee5d0 msgr2=0x560c0e0052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.902+0000 7fea8c980740 1 --2- 172.21.15.172:0/3626186578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dfee5d0 0x560c0e0052b0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fea74097ea0 tx=0x7fea740cc810 comp rx=0 tx=0).stop 2024-09-10T08:57:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.902+0000 7fea8c980740 1 -- 172.21.15.172:0/3626186578 shutdown_connections 2024-09-10T08:57:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.902+0000 7fea8c980740 1 --2- 172.21.15.172:0/3626186578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dff9370 0x560c0e010800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.902+0000 7fea8c980740 1 --2- 172.21.15.172:0/3626186578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dfefb40 0x560c0dff5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.902+0000 7fea8c980740 1 --2- 172.21.15.172:0/3626186578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dfee5d0 0x560c0e0052b0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.902+0000 7fea8c980740 1 -- 172.21.15.172:0/3626186578 >> 172.21.15.172:0/3626186578 conn(0x560c0dfb7310 msgr2=0x560c0dff4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:08.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.902+0000 7fea8c980740 1 -- 172.21.15.172:0/3626186578 shutdown_connections 2024-09-10T08:57:08.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.902+0000 7fea8c980740 1 -- 172.21.15.172:0/3626186578 wait complete. 2024-09-10T08:57:08.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.903+0000 7fea8c980740 1 -- start start 2024-09-10T08:57:08.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.903+0000 7fea8c980740 1 Processor -- start 2024-09-10T08:57:08.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.903+0000 7fea8c980740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dfee5d0 0x560c0e001960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.903+0000 7fea8c980740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dfefb40 0x560c0dffcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.903+0000 7fea8c980740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dff9370 0x560c0dffd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.903+0000 7fea8c980740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fea841b2260 con 0x560c0dff9370 2024-09-10T08:57:08.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.903+0000 7fea8c980740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea841b1b90 con 0x560c0dfee5d0 2024-09-10T08:57:08.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.903+0000 7fea8c980740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fea841aed70 con 0x560c0dfefb40 2024-09-10T08:57:08.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.903+0000 7fea8c980740 10 client.0 _ll_register_callbacks cb 0x7fea7400a4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:08.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.904+0000 7fea88b44640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dff9370 0x560c0dffd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.904+0000 7fea88b44640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dff9370 0x560c0dffd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35742/0 (socket says 172.21.15.172:35742) 2024-09-10T08:57:08.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.904+0000 7fea88b44640 1 -- 172.21.15.172:0/209990290 learned_addr learned my addr 172.21.15.172:0/209990290 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:08.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.904+0000 7fea83fff640 1 --2- 172.21.15.172:0/209990290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dfee5d0 0x560c0e001960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.904+0000 7fea837fe640 1 --2- 172.21.15.172:0/209990290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dfefb40 0x560c0dffcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.904+0000 7fea88b44640 1 -- 172.21.15.172:0/209990290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dfefb40 msgr2=0x560c0dffcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:08.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.904+0000 7fea88b44640 1 --2- 172.21.15.172:0/209990290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dfefb40 0x560c0dffcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.904+0000 7fea88b44640 1 -- 172.21.15.172:0/209990290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dfee5d0 msgr2=0x560c0e001960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:08.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.904+0000 7fea88b44640 1 --2- 172.21.15.172:0/209990290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dfee5d0 0x560c0e001960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.904+0000 7fea88b44640 1 -- 172.21.15.172:0/209990290 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c0dffd860 con 0x560c0dff9370 2024-09-10T08:57:08.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.904+0000 7fea88b44640 1 --2- 172.21.15.172:0/209990290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dff9370 0x560c0dffd270 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fea7c035000 tx=0x7fea7c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa8b1519981b0d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:08.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.904+0000 7fea637fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:08.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.904+0000 7fea637fe640 1 -- 172.21.15.172:0/209990290 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c056060 con 0x560c0dff9370 2024-09-10T08:57:08.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.904+0000 7fea8c980740 1 -- 172.21.15.172:0/209990290 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea7409bc20 con 0x560c0dff9370 2024-09-10T08:57:08.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.904+0000 7fea637fe640 1 -- 172.21.15.172:0/209990290 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fea7c045070 con 0x560c0dff9370 2024-09-10T08:57:08.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.905+0000 7fea637fe640 1 -- 172.21.15.172:0/209990290 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fea7c05d050 con 0x560c0dff9370 2024-09-10T08:57:08.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.905+0000 7fea8c980740 10 client.18696 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:08.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.905+0000 7fea8c980740 1 -- 172.21.15.172:0/209990290 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560c0e0735d0 con 0x560c0dff9370 2024-09-10T08:57:08.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.905+0000 7fea637fe640 1 -- 172.21.15.172:0/209990290 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea7409bc20 con 0x560c0dff9370 2024-09-10T08:57:08.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.905+0000 7fea637fe640 1 -- 172.21.15.172:0/209990290 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fea7c050070 con 0x560c0dff9370 2024-09-10T08:57:08.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.905+0000 7fea8c980740 10 client.18696 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:08.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.905+0000 7fea8c980740 10 client.18696 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:57:08.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.905+0000 7fea8c980740 20 client.18696 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:08.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.905+0000 7fea8c980740 1 --2- 172.21.15.172:0/209990290 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x560c0e073d50 0x560c0e076170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:08.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.905+0000 7fea8c980740 4 client.18696 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:57:08.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.905+0000 7fea8c980740 1 -- 172.21.15.172:0/209990290 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c0df5bd70 con 0x560c0e073d50 2024-09-10T08:57:08.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.905+0000 7fea83fff640 1 --2- 172.21.15.172:0/209990290 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x560c0e073d50 0x560c0e076170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:08.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.906+0000 7fea637fe640 10 client.18696 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:57:08.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.906+0000 7fea83fff640 1 --2- 172.21.15.172:0/209990290 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x560c0e073d50 0x560c0e076170 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69542a4d03fb558b server_cookie=35c58e1fa2d5ee0f in_seq=0 out_seq=0 2024-09-10T08:57:08.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.906+0000 7fea637fe640 1 -- 172.21.15.172:0/209990290 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c0df5bd70 con 0x560c0e073d50 2024-09-10T08:57:08.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.906+0000 7fea637fe640 10 client.18696 handle_command_reply: tid=0 2024-09-10T08:57:08.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.906+0000 7fea8c980740 1 client.18696 shutdown 2024-09-10T08:57:08.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.906+0000 7fea8c980740 1 -- 172.21.15.172:0/209990290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dff9370 msgr2=0x560c0dffd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:08.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.906+0000 7fea8c980740 1 --2- 172.21.15.172:0/209990290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dff9370 0x560c0dffd270 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fea7c035000 tx=0x7fea7c02a8e0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.906+0000 7fea8c980740 1 -- 172.21.15.172:0/209990290 shutdown_connections 2024-09-10T08:57:08.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.906+0000 7fea8c980740 1 --2- 172.21.15.172:0/209990290 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x560c0e073d50 0x560c0e076170 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.906+0000 7fea8c980740 1 --2- 172.21.15.172:0/209990290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0dff9370 0x560c0dffd270 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.906+0000 7fea8c980740 1 --2- 172.21.15.172:0/209990290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560c0dfefb40 0x560c0dffcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.906+0000 7fea8c980740 1 --2- 172.21.15.172:0/209990290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c0dfee5d0 0x560c0e001960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.906+0000 7fea8c980740 1 -- 172.21.15.172:0/209990290 >> 172.21.15.172:0/209990290 conn(0x560c0dfb7310 msgr2=0x560c0dff0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:08.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.906+0000 7fea637fe640 0 client.18696 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:57:08.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.907+0000 7fea8c980740 1 -- 172.21.15.172:0/209990290 shutdown_connections 2024-09-10T08:57:08.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.907+0000 7fea8c980740 1 -- 172.21.15.172:0/209990290 wait complete. 2024-09-10T08:57:08.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.907+0000 7fea8c980740 20 client.18696 trim_cache size 0 max 16384 2024-09-10T08:57:08.915 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:57:08.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.907+0000 7fea8a5a7640 1 -- 172.21.15.172:0/3832437372 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea6c0a2d20 msgr2=0x7fea6c0a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.907+0000 7fea8a5a7640 1 --2- 172.21.15.172:0/3832437372 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea6c0a2d20 0x7fea6c0a51e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009430 tx=0x7fea7c009360 comp rx=0 tx=0).stop 2024-09-10T08:57:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.907+0000 7fea8a5a7640 1 -- 172.21.15.172:0/3832437372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea84102d90 msgr2=0x7fea8419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.907+0000 7fea8a5a7640 1 --2- 172.21.15.172:0/3832437372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea84102d90 0x7fea8419a5c0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fea7400c970 tx=0x7fea7400cd70 comp rx=0 tx=0).stop 2024-09-10T08:57:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.907+0000 7fea8a5a7640 1 -- 172.21.15.172:0/3832437372 shutdown_connections 2024-09-10T08:57:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.907+0000 7fea8a5a7640 1 --2- 172.21.15.172:0/3832437372 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea6c0a2d20 0x7fea6c0a51e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.907+0000 7fea8a5a7640 1 --2- 172.21.15.172:0/3832437372 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea84108290 0x7fea8419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.907+0000 7fea8a5a7640 1 --2- 172.21.15.172:0/3832437372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea84103740 0x7fea8419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.907+0000 7fea8a5a7640 1 --2- 172.21.15.172:0/3832437372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea84102d90 0x7fea8419a5c0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.907+0000 7fea8a5a7640 1 -- 172.21.15.172:0/3832437372 >> 172.21.15.172:0/3832437372 conn(0x7fea840fc7d0 msgr2=0x7fea84100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.908+0000 7fea8a5a7640 1 -- 172.21.15.172:0/3832437372 shutdown_connections 2024-09-10T08:57:08.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:08.908+0000 7fea8a5a7640 1 -- 172.21.15.172:0/3832437372 wait complete. 2024-09-10T08:57:09.205 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.2 2024-09-10T08:57:09.206 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:57:09.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:09 smithi172 ceph-mon[30854]: pgmap v678: 129 pgs: 129 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:57:09.648 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:57:09.707 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:57:09.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:09 smithi174 ceph-mon[34068]: pgmap v678: 129 pgs: 129 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:57:09.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:09 smithi191 ceph-mon[35314]: pgmap v678: 129 pgs: 129 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:57:10.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.212+0000 7fe393089640 1 -- 172.21.15.172:0/828668551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38c1082a0 msgr2=0x7fe38c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:10.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.212+0000 7fe393089640 1 --2- 172.21.15.172:0/828668551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38c1082a0 0x7fe38c10ec40 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fe384009cd0 tx=0x7fe38402a8e0 comp rx=0 tx=0).stop 2024-09-10T08:57:10.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.212+0000 7fe393089640 1 -- 172.21.15.172:0/828668551 shutdown_connections 2024-09-10T08:57:10.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.212+0000 7fe393089640 1 --2- 172.21.15.172:0/828668551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38c1082a0 0x7fe38c10ec40 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:10.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.212+0000 7fe393089640 1 --2- 172.21.15.172:0/828668551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe38c1036e0 0x7fe38c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:10.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.212+0000 7fe393089640 1 --2- 172.21.15.172:0/828668551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe38c102d30 0x7fe38c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:10.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.212+0000 7fe393089640 1 -- 172.21.15.172:0/828668551 >> 172.21.15.172:0/828668551 conn(0x7fe38c0fc7d0 msgr2=0x7fe38c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:10.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.212+0000 7fe393089640 1 -- 172.21.15.172:0/828668551 shutdown_connections 2024-09-10T08:57:10.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.212+0000 7fe393089640 1 -- 172.21.15.172:0/828668551 wait complete. 2024-09-10T08:57:10.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.212+0000 7fe393089640 1 Processor -- start 2024-09-10T08:57:10.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe393089640 1 -- start start 2024-09-10T08:57:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe393089640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe38c102d30 0x7fe38c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe393089640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38c1036e0 0x7fe38c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe393089640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe38c1082a0 0x7fe38c19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe393089640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c111f50 con 0x7fe38c1036e0 2024-09-10T08:57:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe393089640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c111dd0 con 0x7fe38c102d30 2024-09-10T08:57:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe393089640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c1120d0 con 0x7fe38c1082a0 2024-09-10T08:57:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe38bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38c1036e0 0x7fe38c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe38bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38c1036e0 0x7fe38c198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35772/0 (socket says 172.21.15.172:35772) 2024-09-10T08:57:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe38bfff640 1 -- 172.21.15.172:0/4112845869 learned_addr learned my addr 172.21.15.172:0/4112845869 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe390e25640 1 --2- 172.21.15.172:0/4112845869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe38c102d30 0x7fe38c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe391626640 1 --2- 172.21.15.172:0/4112845869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe38c1082a0 0x7fe38c19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe38bfff640 1 -- 172.21.15.172:0/4112845869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe38c1082a0 msgr2=0x7fe38c19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:10.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe38bfff640 1 --2- 172.21.15.172:0/4112845869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe38c1082a0 0x7fe38c19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:10.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe38bfff640 1 -- 172.21.15.172:0/4112845869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe38c102d30 msgr2=0x7fe38c1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:10.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe38bfff640 1 --2- 172.21.15.172:0/4112845869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe38c102d30 0x7fe38c1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:10.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe38bfff640 1 -- 172.21.15.172:0/4112845869 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38c19d550 con 0x7fe38c1036e0 2024-09-10T08:57:10.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe391626640 1 --2- 172.21.15.172:0/4112845869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe38c1082a0 0x7fe38c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:57:10.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.213+0000 7fe38bfff640 1 --2- 172.21.15.172:0/4112845869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38c1036e0 0x7fe38c198ae0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fe37800d740 tx=0x7fe37800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:10.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.214+0000 7fe389ffb640 1 -- 172.21.15.172:0/4112845869 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe378017070 con 0x7fe38c1036e0 2024-09-10T08:57:10.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.214+0000 7fe389ffb640 1 -- 172.21.15.172:0/4112845869 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe37800de50 con 0x7fe38c1036e0 2024-09-10T08:57:10.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.214+0000 7fe389ffb640 1 -- 172.21.15.172:0/4112845869 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37800f620 con 0x7fe38c1036e0 2024-09-10T08:57:10.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.214+0000 7fe393089640 1 -- 172.21.15.172:0/4112845869 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe38c19d840 con 0x7fe38c1036e0 2024-09-10T08:57:10.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.214+0000 7fe393089640 1 -- 172.21.15.172:0/4112845869 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe38c1a5120 con 0x7fe38c1036e0 2024-09-10T08:57:10.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.215+0000 7fe393089640 1 -- 172.21.15.172:0/4112845869 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe38c103110 con 0x7fe38c1036e0 2024-09-10T08:57:10.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.216+0000 7fe389ffb640 1 -- 172.21.15.172:0/4112845869 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe37801a070 con 0x7fe38c1036e0 2024-09-10T08:57:10.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.217+0000 7fe389ffb640 1 --2- 172.21.15.172:0/4112845869 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe37c0a2c00 0x7fe37c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:10.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.217+0000 7fe389ffb640 1 -- 172.21.15.172:0/4112845869 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe3780cecb0 con 0x7fe38c1036e0 2024-09-10T08:57:10.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.217+0000 7fe390e25640 1 --2- 172.21.15.172:0/4112845869 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe37c0a2c00 0x7fe37c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:10.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.217+0000 7fe390e25640 1 --2- 172.21.15.172:0/4112845869 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe37c0a2c00 0x7fe37c0a50c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe380009bd0 tx=0x7fe380005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:10.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.222+0000 7fe389ffb640 1 -- 172.21.15.172:0/4112845869 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe37809b8b0 con 0x7fe38c1036e0 2024-09-10T08:57:10.282 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:57:10.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.487+0000 7fe393089640 1 -- 172.21.15.172:0/4112845869 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe38c10a2b0 con 0x7fe38c1036e0 2024-09-10T08:57:10.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.488+0000 7fe389ffb640 1 -- 172.21.15.172:0/4112845869 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+12004 (secure 0 0 0) 0x7fe37809b1b0 con 0x7fe38c1036e0 2024-09-10T08:57:10.490 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:57:10.490 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":39,"btime":"2024-09-10T08:56:43:360461+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:56:42.665159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:57:10.491 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 39 2024-09-10T08:57:10.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.490+0000 7fe393089640 1 -- 172.21.15.172:0/4112845869 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe37c0a2c00 msgr2=0x7fe37c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.490+0000 7fe393089640 1 --2- 172.21.15.172:0/4112845869 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe37c0a2c00 0x7fe37c0a50c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe380009bd0 tx=0x7fe380005f90 comp rx=0 tx=0).stop 2024-09-10T08:57:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.490+0000 7fe393089640 1 -- 172.21.15.172:0/4112845869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38c1036e0 msgr2=0x7fe38c198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.490+0000 7fe393089640 1 --2- 172.21.15.172:0/4112845869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38c1036e0 0x7fe38c198ae0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fe37800d740 tx=0x7fe37800dc00 comp rx=0 tx=0).stop 2024-09-10T08:57:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.490+0000 7fe393089640 1 -- 172.21.15.172:0/4112845869 shutdown_connections 2024-09-10T08:57:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.491+0000 7fe393089640 1 --2- 172.21.15.172:0/4112845869 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe37c0a2c00 0x7fe37c0a50c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.491+0000 7fe393089640 1 --2- 172.21.15.172:0/4112845869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe38c1082a0 0x7fe38c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.491+0000 7fe393089640 1 --2- 172.21.15.172:0/4112845869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe38c1036e0 0x7fe38c198ae0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.491+0000 7fe393089640 1 --2- 172.21.15.172:0/4112845869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe38c102d30 0x7fe38c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.491+0000 7fe393089640 1 -- 172.21.15.172:0/4112845869 >> 172.21.15.172:0/4112845869 conn(0x7fe38c0fc7d0 msgr2=0x7fe38c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.491+0000 7fe393089640 1 -- 172.21.15.172:0/4112845869 shutdown_connections 2024-09-10T08:57:10.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:10.491+0000 7fe393089640 1 -- 172.21.15.172:0/4112845869 wait complete. 2024-09-10T08:57:10.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:10 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/4112845869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:57:10.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:10 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/4112845869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:57:10.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:10 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/4112845869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:57:11.190 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0' 2024-09-10T08:57:11.190 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0 2024-09-10T08:57:11.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:11 smithi191 ceph-mon[35314]: pgmap v679: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 23 op/s 2024-09-10T08:57:11.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:11 smithi172 ceph-mon[30854]: pgmap v679: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 23 op/s 2024-09-10T08:57:11.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:11 smithi174 ceph-mon[34068]: pgmap v679: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 23 op/s 2024-09-10T08:57:13.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.260+0000 7fe10f59e640 1 -- 172.21.15.172:0/4173394957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe110102380 msgr2=0x7fe11010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:13.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.260+0000 7fe10f59e640 1 --2- 172.21.15.172:0/4173394957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe110102380 0x7fe11010e9c0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe108009cd0 tx=0x7fe10802a8e0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.260+0000 7fe10f59e640 1 -- 172.21.15.172:0/4173394957 shutdown_connections 2024-09-10T08:57:13.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.260+0000 7fe10f59e640 1 --2- 172.21.15.172:0/4173394957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe110102380 0x7fe11010e9c0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.260+0000 7fe10f59e640 1 --2- 172.21.15.172:0/4173394957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1101019e0 0x7fe110101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.260+0000 7fe10f59e640 1 --2- 172.21.15.172:0/4173394957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe110107af0 0x7fe110107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.260+0000 7fe10f59e640 1 -- 172.21.15.172:0/4173394957 >> 172.21.15.172:0/4173394957 conn(0x7fe1100fd700 msgr2=0x7fe1100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:13.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.260+0000 7fe10f59e640 1 -- 172.21.15.172:0/4173394957 shutdown_connections 2024-09-10T08:57:13.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.260+0000 7fe10f59e640 1 -- 172.21.15.172:0/4173394957 wait complete. 2024-09-10T08:57:13.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.261+0000 7fe10f59e640 1 Processor -- start 2024-09-10T08:57:13.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.261+0000 7fe10f59e640 1 -- start start 2024-09-10T08:57:13.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.261+0000 7fe10f59e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1101019e0 0x7fe1101982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.261+0000 7fe10f59e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe110102380 0x7fe1101987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.261+0000 7fe10f59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe110107af0 0x7fe11019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.261+0000 7fe10f59e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe110111da0 con 0x7fe110107af0 2024-09-10T08:57:13.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.261+0000 7fe10f59e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe110111c20 con 0x7fe110102380 2024-09-10T08:57:13.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.261+0000 7fe10f59e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe110111f20 con 0x7fe1101019e0 2024-09-10T08:57:13.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.262+0000 7fe10ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe110107af0 0x7fe11019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:13.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.262+0000 7fe10ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe110107af0 0x7fe11019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35798/0 (socket says 172.21.15.172:35798) 2024-09-10T08:57:13.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.262+0000 7fe10ed9d640 1 -- 172.21.15.172:0/88963135 learned_addr learned my addr 172.21.15.172:0/88963135 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:13.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.262+0000 7fe10e59c640 1 --2- 172.21.15.172:0/88963135 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1101019e0 0x7fe1101982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:13.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.262+0000 7fe10dd9b640 1 --2- 172.21.15.172:0/88963135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe110102380 0x7fe1101987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:13.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.262+0000 7fe10ed9d640 1 -- 172.21.15.172:0/88963135 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1101019e0 msgr2=0x7fe1101982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:13.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.262+0000 7fe10ed9d640 1 --2- 172.21.15.172:0/88963135 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1101019e0 0x7fe1101982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.262+0000 7fe10ed9d640 1 -- 172.21.15.172:0/88963135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe110102380 msgr2=0x7fe1101987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:13.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.262+0000 7fe10ed9d640 1 --2- 172.21.15.172:0/88963135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe110102380 0x7fe1101987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.262+0000 7fe10ed9d640 1 -- 172.21.15.172:0/88963135 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11019d250 con 0x7fe110107af0 2024-09-10T08:57:13.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.262+0000 7fe10ed9d640 1 --2- 172.21.15.172:0/88963135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe110107af0 0x7fe11019cb70 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe108005cc0 tx=0x7fe108003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:13.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.262+0000 7fe0ff7fe640 1 -- 172.21.15.172:0/88963135 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108004450 con 0x7fe110107af0 2024-09-10T08:57:13.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.262+0000 7fe0ff7fe640 1 -- 172.21.15.172:0/88963135 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe108031c90 con 0x7fe110107af0 2024-09-10T08:57:13.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.262+0000 7fe10f59e640 1 -- 172.21.15.172:0/88963135 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe11019d4e0 con 0x7fe110107af0 2024-09-10T08:57:13.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.262+0000 7fe0ff7fe640 1 -- 172.21.15.172:0/88963135 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10803b400 con 0x7fe110107af0 2024-09-10T08:57:13.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.262+0000 7fe10f59e640 1 -- 172.21.15.172:0/88963135 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1101a4e20 con 0x7fe110107af0 2024-09-10T08:57:13.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.264+0000 7fe0ff7fe640 1 -- 172.21.15.172:0/88963135 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe108004680 con 0x7fe110107af0 2024-09-10T08:57:13.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.265+0000 7fe116bdd740 1 Processor -- start 2024-09-10T08:57:13.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.265+0000 7fe116bdd740 1 -- start start 2024-09-10T08:57:13.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.265+0000 7fe116bdd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d2d20 0x5621756d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.265+0000 7fe116bdd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d17b0 0x5621756d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.265+0000 7fe116bdd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621756dc550 0x5621756de940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.265+0000 7fe116bdd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5621756372f0 con 0x5621756d17b0 2024-09-10T08:57:13.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.265+0000 7fe116bdd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5621755decf0 con 0x5621756dc550 2024-09-10T08:57:13.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.265+0000 7fe116bdd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5621756332b0 con 0x5621756d2d20 2024-09-10T08:57:13.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.265+0000 7fe0ff7fe640 1 --2- 172.21.15.172:0/88963135 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0f80a2b80 0x7fe0f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.266+0000 7fe10e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d17b0 0x5621756d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:13.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.266+0000 7fe10e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d17b0 0x5621756d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35812/0 (socket says 172.21.15.172:35812) 2024-09-10T08:57:13.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.266+0000 7fe10e59c640 1 -- 172.21.15.172:0/1827972447 learned_addr learned my addr 172.21.15.172:0/1827972447 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:13.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.266+0000 7fe10ed9d640 1 --2- 172.21.15.172:0/88963135 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0f80a2b80 0x7fe0f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:13.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.266+0000 7fe10e59c640 1 -- 172.21.15.172:0/1827972447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d2d20 msgr2=0x5621756d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:13.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.266+0000 7fe10dd9b640 1 --2- 172.21.15.172:0/1827972447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d2d20 0x5621756d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:13.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.266+0000 7fe10e59c640 1 --2- 172.21.15.172:0/1827972447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d2d20 0x5621756d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.266+0000 7fe10e59c640 1 -- 172.21.15.172:0/1827972447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621756dc550 msgr2=0x5621756de940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:13.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.266+0000 7fe0ff7fe640 1 -- 172.21.15.172:0/88963135 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe10802e030 con 0x7fe110107af0 2024-09-10T08:57:13.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.266+0000 7fe10e59c640 1 --2- 172.21.15.172:0/1827972447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621756dc550 0x5621756de940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.266+0000 7fe10e59c640 1 -- 172.21.15.172:0/1827972447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621755c33a0 con 0x5621756d17b0 2024-09-10T08:57:13.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.266+0000 7fe10ed9d640 1 --2- 172.21.15.172:0/88963135 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0f80a2b80 0x7fe0f80a5040 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe1080b74a0 tx=0x7fe1080a5f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:13.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.266+0000 7fe10dd9b640 1 --2- 172.21.15.172:0/1827972447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d2d20 0x5621756d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:57:13.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.266+0000 7fe10e59c640 1 --2- 172.21.15.172:0/1827972447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d17b0 0x5621756d1bf0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fe100004610 tx=0x7fe100012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ea3b33fc011230 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:13.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.267+0000 7fe0fcff9640 1 -- 172.21.15.172:0/1827972447 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10000bc70 con 0x5621756d17b0 2024-09-10T08:57:13.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.267+0000 7fe0fcff9640 1 -- 172.21.15.172:0/1827972447 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe10000be10 con 0x5621756d17b0 2024-09-10T08:57:13.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.267+0000 7fe0fcff9640 1 -- 172.21.15.172:0/1827972447 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1000035f0 con 0x5621756d17b0 2024-09-10T08:57:13.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.268+0000 7fe116bdd740 1 -- 172.21.15.172:0/1827972447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d17b0 msgr2=0x5621756d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:13.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.268+0000 7fe116bdd740 1 --2- 172.21.15.172:0/1827972447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d17b0 0x5621756d1bf0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fe100004610 tx=0x7fe100012bf0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.268+0000 7fe116bdd740 1 -- 172.21.15.172:0/1827972447 shutdown_connections 2024-09-10T08:57:13.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.268+0000 7fe116bdd740 1 --2- 172.21.15.172:0/1827972447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621756dc550 0x5621756de940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.268+0000 7fe116bdd740 1 --2- 172.21.15.172:0/1827972447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d17b0 0x5621756d1bf0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.268+0000 7fe116bdd740 1 --2- 172.21.15.172:0/1827972447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d2d20 0x5621756d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.268+0000 7fe116bdd740 1 -- 172.21.15.172:0/1827972447 >> 172.21.15.172:0/1827972447 conn(0x56217569a310 msgr2=0x5621756cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:13.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.268+0000 7fe116bdd740 1 -- 172.21.15.172:0/1827972447 shutdown_connections 2024-09-10T08:57:13.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.268+0000 7fe116bdd740 1 -- 172.21.15.172:0/1827972447 wait complete. 2024-09-10T08:57:13.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.268+0000 7fe116bdd740 1 -- start start 2024-09-10T08:57:13.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.268+0000 7fe116bdd740 1 Processor -- start 2024-09-10T08:57:13.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.268+0000 7fe116bdd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d17b0 0x5621756dbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.268+0000 7fe116bdd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d2d20 0x5621756d9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.268+0000 7fe116bdd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621756dc550 0x5621756d9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.268+0000 7fe116bdd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5621756df2a0 con 0x5621756d17b0 2024-09-10T08:57:13.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.268+0000 7fe116bdd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5621755f2060 con 0x5621756dc550 2024-09-10T08:57:13.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.268+0000 7fe116bdd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5621756df420 con 0x5621756d2d20 2024-09-10T08:57:13.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.269+0000 7fe116bdd740 10 client.0 _ll_register_callbacks cb 0x5621753458c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:13.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.269+0000 7fe10dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d17b0 0x5621756dbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:13.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.269+0000 7fe10dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d17b0 0x5621756dbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35828/0 (socket says 172.21.15.172:35828) 2024-09-10T08:57:13.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.269+0000 7fe10dd9b640 1 -- 172.21.15.172:0/3247558645 learned_addr learned my addr 172.21.15.172:0/3247558645 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:13.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.269+0000 7fe10e59c640 1 --2- 172.21.15.172:0/3247558645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d2d20 0x5621756d9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:13.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.269+0000 7fe10dd9b640 1 --2- 172.21.15.172:0/3247558645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621756dc550 0x5621756d9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:13.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.269+0000 7fe10dd9b640 1 -- 172.21.15.172:0/3247558645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d2d20 msgr2=0x5621756d9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:13.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.269+0000 7fe10dd9b640 1 --2- 172.21.15.172:0/3247558645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d2d20 0x5621756d9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.269+0000 7fe10dd9b640 1 -- 172.21.15.172:0/3247558645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621756dc550 msgr2=0x5621756d9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:13.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.269+0000 7fe10dd9b640 1 --2- 172.21.15.172:0/3247558645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621756dc550 0x5621756d9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.269+0000 7fe10dd9b640 1 -- 172.21.15.172:0/3247558645 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621753a83c0 con 0x5621756d17b0 2024-09-10T08:57:13.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.269+0000 7fe10e59c640 1 --2- 172.21.15.172:0/3247558645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d2d20 0x5621756d9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:13.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.269+0000 7fe10dd9b640 1 --2- 172.21.15.172:0/3247558645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d17b0 0x5621756dbfe0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fe10400b6c0 tx=0x7fe1040052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7543455b51f19f7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:13.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.269+0000 7fe0f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:13.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.269+0000 7fe0f5ffb640 1 -- 172.21.15.172:0/3247558645 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104004020 con 0x5621756d17b0 2024-09-10T08:57:13.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.269+0000 7fe116bdd740 1 -- 172.21.15.172:0/3247558645 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe10000be10 con 0x5621756d17b0 2024-09-10T08:57:13.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.269+0000 7fe0f5ffb640 1 -- 172.21.15.172:0/3247558645 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe1040041c0 con 0x5621756d17b0 2024-09-10T08:57:13.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.270+0000 7fe0f5ffb640 1 -- 172.21.15.172:0/3247558645 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe104024050 con 0x5621756d17b0 2024-09-10T08:57:13.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.270+0000 7fe116bdd740 10 client.18714 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:13.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.270+0000 7fe116bdd740 1 -- 172.21.15.172:0/3247558645 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5621756eae10 con 0x5621756d17b0 2024-09-10T08:57:13.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.270+0000 7fe0f5ffb640 1 -- 172.21.15.172:0/3247558645 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe10000be10 con 0x5621756d17b0 2024-09-10T08:57:13.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.270+0000 7fe0f5ffb640 1 -- 172.21.15.172:0/3247558645 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fe1040035f0 con 0x5621756d17b0 2024-09-10T08:57:13.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.270+0000 7fe116bdd740 10 client.18714 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:13.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.270+0000 7fe116bdd740 10 client.18714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:13.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.270+0000 7fe116bdd740 20 client.18714 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:13.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.270+0000 7fe116bdd740 1 --2- 172.21.15.172:0/3247558645 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5621756ec320 0x5621756ee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.270+0000 7fe116bdd740 4 client.18714 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:57:13.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.270+0000 7fe116bdd740 1 -- 172.21.15.172:0/3247558645 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562175644860 con 0x5621756ec320 2024-09-10T08:57:13.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.271+0000 7fe10e59c640 1 --2- 172.21.15.172:0/3247558645 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5621756ec320 0x5621756ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:13.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.271+0000 7fe10e59c640 1 --2- 172.21.15.172:0/3247558645 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5621756ec320 0x5621756ee760 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3a86d16be856530 server_cookie=10ae22d76e2aab8e in_seq=0 out_seq=0 2024-09-10T08:57:13.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.271+0000 7fe0f5ffb640 10 client.18714 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:13.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.272+0000 7fe0f5ffb640 1 -- 172.21.15.172:0/3247558645 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562175644860 con 0x5621756ec320 2024-09-10T08:57:13.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.272+0000 7fe0f5ffb640 10 client.18714 handle_command_reply: tid=0 2024-09-10T08:57:13.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.272+0000 7fe116bdd740 1 client.18714 shutdown 2024-09-10T08:57:13.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.272+0000 7fe116bdd740 1 -- 172.21.15.172:0/3247558645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d17b0 msgr2=0x5621756dbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:13.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.272+0000 7fe116bdd740 1 --2- 172.21.15.172:0/3247558645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d17b0 0x5621756dbfe0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fe10400b6c0 tx=0x7fe1040052f0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.272+0000 7fe116bdd740 1 -- 172.21.15.172:0/3247558645 shutdown_connections 2024-09-10T08:57:13.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.272+0000 7fe116bdd740 1 --2- 172.21.15.172:0/3247558645 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5621756ec320 0x5621756ee760 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.272+0000 7fe116bdd740 1 --2- 172.21.15.172:0/3247558645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621756dc550 0x5621756d9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.272+0000 7fe116bdd740 1 --2- 172.21.15.172:0/3247558645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d2d20 0x5621756d9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.272+0000 7fe116bdd740 1 --2- 172.21.15.172:0/3247558645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d17b0 0x5621756dbfe0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.272+0000 7fe116bdd740 1 -- 172.21.15.172:0/3247558645 >> 172.21.15.172:0/3247558645 conn(0x56217569a310 msgr2=0x5621756d3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:13.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.272+0000 7fe0f5ffb640 0 client.18714 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:13.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.272+0000 7fe116bdd740 1 -- 172.21.15.172:0/3247558645 shutdown_connections 2024-09-10T08:57:13.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.272+0000 7fe116bdd740 1 -- 172.21.15.172:0/3247558645 wait complete. 2024-09-10T08:57:13.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.272+0000 7fe116bdd740 20 client.18714 trim_cache size 0 max 16384 2024-09-10T08:57:13.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.287+0000 7fe116bdd740 1 Processor -- start 2024-09-10T08:57:13.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.287+0000 7fe116bdd740 1 -- start start 2024-09-10T08:57:13.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe116bdd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217569a310 0x5621756dd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe116bdd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d17b0 0x5621756dde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe116bdd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d2d20 0x5621756d4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe116bdd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe1101ac640 con 0x5621756d2d20 2024-09-10T08:57:13.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe116bdd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe1101ae1e0 con 0x56217569a310 2024-09-10T08:57:13.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe116bdd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe1101acd10 con 0x5621756d17b0 2024-09-10T08:57:13.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe10dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d2d20 0x5621756d4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:13.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe10dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d2d20 0x5621756d4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35838/0 (socket says 172.21.15.172:35838) 2024-09-10T08:57:13.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe10dd9b640 1 -- 172.21.15.172:0/2079646153 learned_addr learned my addr 172.21.15.172:0/2079646153 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:13.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe10dd9b640 1 --2- 172.21.15.172:0/2079646153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217569a310 0x5621756dd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:13.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe10e59c640 1 --2- 172.21.15.172:0/2079646153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d17b0 0x5621756dde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:13.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe10dd9b640 1 -- 172.21.15.172:0/2079646153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d17b0 msgr2=0x5621756dde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:13.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe10dd9b640 1 --2- 172.21.15.172:0/2079646153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d17b0 0x5621756dde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe10dd9b640 1 -- 172.21.15.172:0/2079646153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217569a310 msgr2=0x5621756dd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:13.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe10dd9b640 1 --2- 172.21.15.172:0/2079646153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217569a310 0x5621756dd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe10dd9b640 1 -- 172.21.15.172:0/2079646153 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621756d7880 con 0x5621756d2d20 2024-09-10T08:57:13.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe10e59c640 1 --2- 172.21.15.172:0/2079646153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d17b0 0x5621756dde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:57:13.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.288+0000 7fe10dd9b640 1 --2- 172.21.15.172:0/2079646153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d2d20 0x5621756d4300 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fe10400b690 tx=0x7fe104000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7813c2c8d28e35f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:13.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.289+0000 7fe0f7fff640 1 -- 172.21.15.172:0/2079646153 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104005850 con 0x5621756d2d20 2024-09-10T08:57:13.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.289+0000 7fe0f7fff640 1 -- 172.21.15.172:0/2079646153 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe104011b30 con 0x5621756d2d20 2024-09-10T08:57:13.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.289+0000 7fe0f7fff640 1 -- 172.21.15.172:0/2079646153 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104003f20 con 0x5621756d2d20 2024-09-10T08:57:13.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.289+0000 7fe116bdd740 1 -- 172.21.15.172:0/2079646153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d2d20 msgr2=0x5621756d4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:13.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.289+0000 7fe116bdd740 1 --2- 172.21.15.172:0/2079646153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d2d20 0x5621756d4300 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fe10400b690 tx=0x7fe104000bc0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.289+0000 7fe116bdd740 1 -- 172.21.15.172:0/2079646153 shutdown_connections 2024-09-10T08:57:13.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.289+0000 7fe116bdd740 1 --2- 172.21.15.172:0/2079646153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621756d2d20 0x5621756d4300 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.289+0000 7fe116bdd740 1 --2- 172.21.15.172:0/2079646153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d17b0 0x5621756dde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.289+0000 7fe116bdd740 1 --2- 172.21.15.172:0/2079646153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217569a310 0x5621756dd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.289+0000 7fe116bdd740 1 -- 172.21.15.172:0/2079646153 >> 172.21.15.172:0/2079646153 conn(0x5621756deaf0 msgr2=0x5621756d8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:13.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.290+0000 7fe116bdd740 1 -- 172.21.15.172:0/2079646153 shutdown_connections 2024-09-10T08:57:13.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.290+0000 7fe116bdd740 1 -- 172.21.15.172:0/2079646153 wait complete. 2024-09-10T08:57:13.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.290+0000 7fe116bdd740 1 -- start start 2024-09-10T08:57:13.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.290+0000 7fe116bdd740 1 Processor -- start 2024-09-10T08:57:13.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.290+0000 7fe116bdd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56217569a310 0x5621756e4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.290+0000 7fe116bdd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621756d17b0 0x5621756d6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.290+0000 7fe116bdd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d2d20 0x5621756d6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.290+0000 7fe116bdd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe1101adb10 con 0x56217569a310 2024-09-10T08:57:13.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.290+0000 7fe116bdd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe1101ad440 con 0x5621756d17b0 2024-09-10T08:57:13.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.290+0000 7fe116bdd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe1101aa620 con 0x5621756d2d20 2024-09-10T08:57:13.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.290+0000 7fe116bdd740 10 client.0 _ll_register_callbacks cb 0x7fe104005120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:13.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.290+0000 7fe10dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56217569a310 0x5621756e4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:13.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.290+0000 7fe10dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56217569a310 0x5621756e4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35844/0 (socket says 172.21.15.172:35844) 2024-09-10T08:57:13.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.290+0000 7fe10dd9b640 1 -- 172.21.15.172:0/381324008 learned_addr learned my addr 172.21.15.172:0/381324008 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:13.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.291+0000 7fe10dd9b640 1 --2- 172.21.15.172:0/381324008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d2d20 0x5621756d6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:13.038 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:57:13.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.291+0000 7fe10e59c640 1 --2- 172.21.15.172:0/381324008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621756d17b0 0x5621756d6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:13.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.291+0000 7fe10dd9b640 1 -- 172.21.15.172:0/381324008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d2d20 msgr2=0x5621756d6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:13.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.291+0000 7fe10dd9b640 1 --2- 172.21.15.172:0/381324008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d2d20 0x5621756d6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.291+0000 7fe10dd9b640 1 -- 172.21.15.172:0/381324008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621756d17b0 msgr2=0x5621756d6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:13.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.291+0000 7fe10dd9b640 1 --2- 172.21.15.172:0/381324008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621756d17b0 0x5621756d6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.291+0000 7fe10dd9b640 1 -- 172.21.15.172:0/381324008 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621756d6d30 con 0x56217569a310 2024-09-10T08:57:13.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.291+0000 7fe10e59c640 1 --2- 172.21.15.172:0/381324008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621756d17b0 0x5621756d6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:57:13.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.291+0000 7fe10dd9b640 1 --2- 172.21.15.172:0/381324008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56217569a310 0x5621756e4de0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fe10400b560 tx=0x7fe10400b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60890652060c7d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:13.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.291+0000 7fe0f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:13.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.291+0000 7fe0f5ffb640 1 -- 172.21.15.172:0/381324008 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104030040 con 0x56217569a310 2024-09-10T08:57:13.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.291+0000 7fe0f5ffb640 1 -- 172.21.15.172:0/381324008 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe104011680 con 0x56217569a310 2024-09-10T08:57:13.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.291+0000 7fe116bdd740 1 -- 172.21.15.172:0/381324008 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe104011b30 con 0x56217569a310 2024-09-10T08:57:13.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.291+0000 7fe0f5ffb640 1 -- 172.21.15.172:0/381324008 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe104005850 con 0x56217569a310 2024-09-10T08:57:13.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.291+0000 7fe116bdd740 10 client.18720 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:13.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.291+0000 7fe116bdd740 1 -- 172.21.15.172:0/381324008 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562175757000 con 0x56217569a310 2024-09-10T08:57:13.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.291+0000 7fe0f5ffb640 1 -- 172.21.15.172:0/381324008 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe104011b30 con 0x56217569a310 2024-09-10T08:57:13.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.292+0000 7fe0f5ffb640 1 -- 172.21.15.172:0/381324008 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fe104012450 con 0x56217569a310 2024-09-10T08:57:13.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.292+0000 7fe116bdd740 10 client.18720 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:13.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.292+0000 7fe116bdd740 10 client.18720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:13.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.292+0000 7fe116bdd740 20 client.18720 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:13.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.292+0000 7fe116bdd740 1 --2- 172.21.15.172:0/381324008 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5621756dc550 0x562175759570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:13.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.292+0000 7fe116bdd740 4 client.18720 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:57:13.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.292+0000 7fe116bdd740 1 -- 172.21.15.172:0/381324008 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56217563ed70 con 0x5621756dc550 2024-09-10T08:57:13.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.292+0000 7fe10e59c640 1 --2- 172.21.15.172:0/381324008 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5621756dc550 0x562175759570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:13.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.292+0000 7fe0f5ffb640 10 client.18720 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:13.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.292+0000 7fe10e59c640 1 --2- 172.21.15.172:0/381324008 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5621756dc550 0x562175759570 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5928b22599154da server_cookie=180732b52107e344 in_seq=0 out_seq=0 2024-09-10T08:57:13.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.293+0000 7fe0f5ffb640 1 -- 172.21.15.172:0/381324008 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56217563ed70 con 0x5621756dc550 2024-09-10T08:57:13.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.293+0000 7fe0f5ffb640 10 client.18720 handle_command_reply: tid=0 2024-09-10T08:57:13.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.293+0000 7fe116bdd740 1 client.18720 shutdown 2024-09-10T08:57:13.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.293+0000 7fe116bdd740 1 -- 172.21.15.172:0/381324008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56217569a310 msgr2=0x5621756e4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:13.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.293+0000 7fe116bdd740 1 --2- 172.21.15.172:0/381324008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56217569a310 0x5621756e4de0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fe10400b560 tx=0x7fe10400b600 comp rx=0 tx=0).stop 2024-09-10T08:57:13.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.293+0000 7fe116bdd740 1 -- 172.21.15.172:0/381324008 shutdown_connections 2024-09-10T08:57:13.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.293+0000 7fe116bdd740 1 --2- 172.21.15.172:0/381324008 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5621756dc550 0x562175759570 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.293+0000 7fe116bdd740 1 --2- 172.21.15.172:0/381324008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5621756d2d20 0x5621756d6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.293+0000 7fe116bdd740 1 --2- 172.21.15.172:0/381324008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621756d17b0 0x5621756d6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.293+0000 7fe0f5ffb640 0 client.18720 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:13.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.293+0000 7fe116bdd740 1 --2- 172.21.15.172:0/381324008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56217569a310 0x5621756e4de0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.293+0000 7fe116bdd740 1 -- 172.21.15.172:0/381324008 >> 172.21.15.172:0/381324008 conn(0x5621756deaf0 msgr2=0x5621756cc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:13.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.293+0000 7fe116bdd740 1 -- 172.21.15.172:0/381324008 shutdown_connections 2024-09-10T08:57:13.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.293+0000 7fe116bdd740 1 -- 172.21.15.172:0/381324008 wait complete. 2024-09-10T08:57:13.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.293+0000 7fe116bdd740 20 client.18720 trim_cache size 0 max 16384 2024-09-10T08:57:13.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.293+0000 7fe10f59e640 1 -- 172.21.15.172:0/88963135 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0f80a2b80 msgr2=0x7fe0f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:13.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.294+0000 7fe10f59e640 1 --2- 172.21.15.172:0/88963135 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0f80a2b80 0x7fe0f80a5040 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe1080b74a0 tx=0x7fe1080a5f90 comp rx=0 tx=0).stop 2024-09-10T08:57:13.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.294+0000 7fe10f59e640 1 -- 172.21.15.172:0/88963135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe110107af0 msgr2=0x7fe11019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:13.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.294+0000 7fe10f59e640 1 --2- 172.21.15.172:0/88963135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe110107af0 0x7fe11019cb70 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe108005cc0 tx=0x7fe108003c20 comp rx=0 tx=0).stop 2024-09-10T08:57:13.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.294+0000 7fe10f59e640 1 -- 172.21.15.172:0/88963135 shutdown_connections 2024-09-10T08:57:13.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.294+0000 7fe10f59e640 1 --2- 172.21.15.172:0/88963135 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0f80a2b80 0x7fe0f80a5040 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.294+0000 7fe10f59e640 1 --2- 172.21.15.172:0/88963135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe110107af0 0x7fe11019cb70 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.294+0000 7fe10f59e640 1 --2- 172.21.15.172:0/88963135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe110102380 0x7fe1101987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.294+0000 7fe10f59e640 1 --2- 172.21.15.172:0/88963135 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1101019e0 0x7fe1101982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:13.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.294+0000 7fe10f59e640 1 -- 172.21.15.172:0/88963135 >> 172.21.15.172:0/88963135 conn(0x7fe1100fd700 msgr2=0x7fe1100fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:13.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.294+0000 7fe10f59e640 1 -- 172.21.15.172:0/88963135 shutdown_connections 2024-09-10T08:57:13.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:12.294+0000 7fe10f59e640 1 -- 172.21.15.172:0/88963135 wait complete. 2024-09-10T08:57:13.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:13 smithi174 ceph-mon[34068]: pgmap v680: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:57:13.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:13 smithi191 ceph-mon[35314]: pgmap v680: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:57:13.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:13 smithi172 ceph-mon[30854]: pgmap v680: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:57:14.756 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:57:15.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.258+0000 7f412a0c9640 1 -- 172.21.15.172:0/1596613983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4124102ef0 msgr2=0x7f412410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:15.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.258+0000 7f412a0c9640 1 --2- 172.21.15.172:0/1596613983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4124102ef0 0x7f412410da40 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f411c009ca0 tx=0x7f411c02a950 comp rx=0 tx=0).stop 2024-09-10T08:57:15.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.258+0000 7f412a0c9640 1 -- 172.21.15.172:0/1596613983 shutdown_connections 2024-09-10T08:57:15.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.258+0000 7f412a0c9640 1 --2- 172.21.15.172:0/1596613983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4124102ef0 0x7f412410da40 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.275 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.258+0000 7f412a0c9640 1 --2- 172.21.15.172:0/1596613983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4124102570 0x7f41241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.258+0000 7f412a0c9640 1 --2- 172.21.15.172:0/1596613983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4124100e20 0x7f4124101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.258+0000 7f412a0c9640 1 -- 172.21.15.172:0/1596613983 >> 172.21.15.172:0/1596613983 conn(0x7f41240fc7d0 msgr2=0x7f41240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:15.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.258+0000 7f412a0c9640 1 -- 172.21.15.172:0/1596613983 shutdown_connections 2024-09-10T08:57:15.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.258+0000 7f412a0c9640 1 -- 172.21.15.172:0/1596613983 wait complete. 2024-09-10T08:57:15.276 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.259+0000 7f412a0c9640 1 Processor -- start 2024-09-10T08:57:15.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.259+0000 7f412a0c9640 1 -- start start 2024-09-10T08:57:15.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.259+0000 7f412a0c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4124100e20 0x7f41241983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.259+0000 7f412a0c9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4124102570 0x7f4124198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.259+0000 7f412a0c9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4124102ef0 0x7f412419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.259+0000 7f412a0c9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4124110e30 con 0x7f4124100e20 2024-09-10T08:57:15.277 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.259+0000 7f412a0c9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4124110cb0 con 0x7f4124102570 2024-09-10T08:57:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.259+0000 7f412a0c9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4124110fb0 con 0x7f4124102ef0 2024-09-10T08:57:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.260+0000 7f41237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4124100e20 0x7f41241983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.260+0000 7f41237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4124100e20 0x7f41241983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35862/0 (socket says 172.21.15.172:35862) 2024-09-10T08:57:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.260+0000 7f41237fe640 1 -- 172.21.15.172:0/3151079634 learned_addr learned my addr 172.21.15.172:0/3151079634 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.260+0000 7f4123fff640 1 --2- 172.21.15.172:0/3151079634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4124102ef0 0x7f412419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:15.278 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.260+0000 7f4122ffd640 1 --2- 172.21.15.172:0/3151079634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4124102570 0x7f4124198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:15.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.260+0000 7f41237fe640 1 -- 172.21.15.172:0/3151079634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4124102ef0 msgr2=0x7f412419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:15.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.260+0000 7f41237fe640 1 --2- 172.21.15.172:0/3151079634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4124102ef0 0x7f412419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.260+0000 7f41237fe640 1 -- 172.21.15.172:0/3151079634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4124102570 msgr2=0x7f4124198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:15.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.260+0000 7f41237fe640 1 --2- 172.21.15.172:0/3151079634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4124102570 0x7f4124198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.260+0000 7f41237fe640 1 -- 172.21.15.172:0/3151079634 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412419d370 con 0x7f4124100e20 2024-09-10T08:57:15.279 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.260+0000 7f41237fe640 1 --2- 172.21.15.172:0/3151079634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4124100e20 0x7f41241983c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f411400d770 tx=0x7f411400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:15.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.260+0000 7f4120ff9640 1 -- 172.21.15.172:0/3151079634 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114017070 con 0x7f4124100e20 2024-09-10T08:57:15.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.260+0000 7f412a0c9640 1 -- 172.21.15.172:0/3151079634 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412419d660 con 0x7f4124100e20 2024-09-10T08:57:15.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.260+0000 7f4120ff9640 1 -- 172.21.15.172:0/3151079634 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41140054d0 con 0x7f4124100e20 2024-09-10T08:57:15.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.260+0000 7f4120ff9640 1 -- 172.21.15.172:0/3151079634 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114004cf0 con 0x7f4124100e20 2024-09-10T08:57:15.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.260+0000 7f412a0c9640 1 -- 172.21.15.172:0/3151079634 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41241a4f40 con 0x7f4124100e20 2024-09-10T08:57:15.280 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.263+0000 7f4120ff9640 1 -- 172.21.15.172:0/3151079634 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4114004e90 con 0x7f4124100e20 2024-09-10T08:57:15.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.264+0000 7f412c4a2740 1 Processor -- start 2024-09-10T08:57:15.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.264+0000 7f412c4a2740 1 -- start start 2024-09-10T08:57:15.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.264+0000 7f412c4a2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788423e660 0x55788423ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.264+0000 7f412c4a2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423ef80 0x55788424d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.264+0000 7f412c4a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 0x557884250140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.264+0000 7f412c4a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5578841a42a0 con 0x55788424dd50 2024-09-10T08:57:15.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.264+0000 7f412c4a2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55788414bb10 con 0x55788423ef80 2024-09-10T08:57:15.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.264+0000 7f412c4a2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5578841a0260 con 0x55788423e660 2024-09-10T08:57:15.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.264+0000 7f4122ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 0x557884250140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:15.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.264+0000 7f4122ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 0x557884250140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35870/0 (socket says 172.21.15.172:35870) 2024-09-10T08:57:15.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.264+0000 7f4123fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788423e660 0x55788423ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:15.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.264+0000 7f4122ffd640 1 -- 172.21.15.172:0/2094980430 learned_addr learned my addr 172.21.15.172:0/2094980430 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:15.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.264+0000 7f41237fe640 1 --2- 172.21.15.172:0/2094980430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423ef80 0x55788424d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:15.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.265+0000 7f4120ff9640 1 --2- 172.21.15.172:0/3151079634 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f410c0a2b80 0x7f410c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.265+0000 7f4122ffd640 1 -- 172.21.15.172:0/2094980430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788423e660 msgr2=0x55788423ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:15.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.265+0000 7f4122ffd640 1 --2- 172.21.15.172:0/2094980430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788423e660 0x55788423ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.265+0000 7f4122ffd640 1 -- 172.21.15.172:0/2094980430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423ef80 msgr2=0x55788424d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:15.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.265+0000 7f4122ffd640 1 --2- 172.21.15.172:0/2094980430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423ef80 0x55788424d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.265+0000 7f4122ffd640 1 -- 172.21.15.172:0/2094980430 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557883f15b90 con 0x55788424dd50 2024-09-10T08:57:15.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.265+0000 7f4123fff640 1 --2- 172.21.15.172:0/2094980430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788423e660 0x55788423ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:57:15.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.265+0000 7f4120ff9640 1 -- 172.21.15.172:0/3151079634 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f41140ced60 con 0x7f4124100e20 2024-09-10T08:57:15.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.265+0000 7f4123fff640 1 --2- 172.21.15.172:0/3151079634 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f410c0a2b80 0x7f410c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:15.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.265+0000 7f4122ffd640 1 --2- 172.21.15.172:0/2094980430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 0x557884250140 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f4118009c90 tx=0x7f4118011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548d8e4860bb97c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:15.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.265+0000 7f4109ffb640 1 -- 172.21.15.172:0/2094980430 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118005220 con 0x55788424dd50 2024-09-10T08:57:15.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.265+0000 7f4109ffb640 1 -- 172.21.15.172:0/2094980430 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4118004c50 con 0x55788424dd50 2024-09-10T08:57:15.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.265+0000 7f4109ffb640 1 -- 172.21.15.172:0/2094980430 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118019670 con 0x55788424dd50 2024-09-10T08:57:15.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.266+0000 7f412c4a2740 1 -- 172.21.15.172:0/2094980430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 msgr2=0x557884250140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:15.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.266+0000 7f412c4a2740 1 --2- 172.21.15.172:0/2094980430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 0x557884250140 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f4118009c90 tx=0x7f4118011810 comp rx=0 tx=0).stop 2024-09-10T08:57:15.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.266+0000 7f4123fff640 1 --2- 172.21.15.172:0/3151079634 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f410c0a2b80 0x7f410c0a5040 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f411c005e30 tx=0x7f411c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:15.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.267+0000 7f412c4a2740 1 -- 172.21.15.172:0/2094980430 shutdown_connections 2024-09-10T08:57:15.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.267+0000 7f412c4a2740 1 --2- 172.21.15.172:0/2094980430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 0x557884250140 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.267+0000 7f412c4a2740 1 --2- 172.21.15.172:0/2094980430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423ef80 0x55788424d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.267+0000 7f412c4a2740 1 --2- 172.21.15.172:0/2094980430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788423e660 0x55788423ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.267+0000 7f412c4a2740 1 -- 172.21.15.172:0/2094980430 >> 172.21.15.172:0/2094980430 conn(0x55788423acf0 msgr2=0x55788423b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:15.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.267+0000 7f412c4a2740 1 -- 172.21.15.172:0/2094980430 shutdown_connections 2024-09-10T08:57:15.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.267+0000 7f412c4a2740 1 -- 172.21.15.172:0/2094980430 wait complete. 2024-09-10T08:57:15.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.267+0000 7f412c4a2740 1 -- start start 2024-09-10T08:57:15.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.267+0000 7f412c4a2740 1 Processor -- start 2024-09-10T08:57:15.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f412c4a2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423e660 0x55788424abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f412c4a2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788423ef80 0x55788424b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f412c4a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 0x55788424b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f412c4a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557884250980 con 0x55788424dd50 2024-09-10T08:57:15.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f412c4a2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55788412c6e0 con 0x55788423e660 2024-09-10T08:57:15.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f412c4a2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557884250b00 con 0x55788423ef80 2024-09-10T08:57:15.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f412c4a2740 10 client.0 _ll_register_callbacks cb 0x557884150a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:15.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f41237fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423e660 0x55788424abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:15.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f41237fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423e660 0x55788424abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:51614/0 (socket says 172.21.15.172:51614) 2024-09-10T08:57:15.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f41237fe640 1 -- 172.21.15.172:0/2724742980 learned_addr learned my addr 172.21.15.172:0/2724742980 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:15.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f4122ffd640 1 --2- 172.21.15.172:0/2724742980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788423ef80 0x55788424b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:15.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f41237fe640 1 -- 172.21.15.172:0/2724742980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788423ef80 msgr2=0x55788424b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:15.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f41237fe640 1 --2- 172.21.15.172:0/2724742980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788423ef80 0x55788424b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f41237fe640 1 -- 172.21.15.172:0/2724742980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 msgr2=0x55788424b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:15.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f41237fe640 1 --2- 172.21.15.172:0/2724742980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 0x55788424b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f41237fe640 1 -- 172.21.15.172:0/2724742980 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557884259e80 con 0x55788423e660 2024-09-10T08:57:15.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f4122ffd640 1 --2- 172.21.15.172:0/2724742980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788423ef80 0x55788424b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:57:15.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f41237fe640 1 --2- 172.21.15.172:0/2724742980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423e660 0x55788424abf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f41140cc500 tx=0x7f4114005680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e4aa104b1f9e43c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:15.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.268+0000 7f4102ffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:57:15.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.269+0000 7f4102ffd640 1 -- 172.21.15.172:0/2724742980 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114020050 con 0x55788423e660 2024-09-10T08:57:15.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.269+0000 7f4102ffd640 1 -- 172.21.15.172:0/2724742980 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f41140bb0b0 con 0x55788423e660 2024-09-10T08:57:15.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.269+0000 7f412c4a2740 1 -- 172.21.15.172:0/2724742980 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4118004c50 con 0x55788423e660 2024-09-10T08:57:15.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.269+0000 7f4102ffd640 1 -- 172.21.15.172:0/2724742980 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f411409b0e0 con 0x55788423e660 2024-09-10T08:57:15.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.269+0000 7f412c4a2740 10 client.27623 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:15.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.269+0000 7f412c4a2740 1 -- 172.21.15.172:0/2724742980 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55788425c7f0 con 0x55788423e660 2024-09-10T08:57:15.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.269+0000 7f4102ffd640 1 -- 172.21.15.172:0/2724742980 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4118004c50 con 0x55788423e660 2024-09-10T08:57:15.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.269+0000 7f4102ffd640 1 -- 172.21.15.172:0/2724742980 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f411409f120 con 0x55788423e660 2024-09-10T08:57:15.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.269+0000 7f412c4a2740 10 client.27623 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:15.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.269+0000 7f412c4a2740 10 client.27623 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:57:15.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.269+0000 7f412c4a2740 20 client.27623 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:15.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.270+0000 7f412c4a2740 1 --2- 172.21.15.172:0/2724742980 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55788425dd50 0x557884260210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.270+0000 7f412c4a2740 4 client.27623 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:57:15.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.270+0000 7f412c4a2740 1 -- 172.21.15.172:0/2724742980 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578841b19d0 con 0x55788425dd50 2024-09-10T08:57:15.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.270+0000 7f4122ffd640 1 --2- 172.21.15.172:0/2724742980 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55788425dd50 0x557884260210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:15.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.270+0000 7f4122ffd640 1 --2- 172.21.15.172:0/2724742980 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55788425dd50 0x557884260210 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ebcaa6a8dbf7c1e server_cookie=66e250b38cb9e6d in_seq=0 out_seq=0 2024-09-10T08:57:15.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.270+0000 7f4102ffd640 10 client.27623 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:15.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.271+0000 7f4102ffd640 1 -- 172.21.15.172:0/2724742980 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578841b19d0 con 0x55788425dd50 2024-09-10T08:57:15.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.271+0000 7f4102ffd640 10 client.27623 handle_command_reply: tid=0 2024-09-10T08:57:15.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.271+0000 7f412c4a2740 1 client.27623 shutdown 2024-09-10T08:57:15.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.271+0000 7f412c4a2740 1 -- 172.21.15.172:0/2724742980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423e660 msgr2=0x55788424abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:15.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.271+0000 7f412c4a2740 1 --2- 172.21.15.172:0/2724742980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423e660 0x55788424abf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f41140cc500 tx=0x7f4114005680 comp rx=0 tx=0).stop 2024-09-10T08:57:15.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.271+0000 7f412c4a2740 1 -- 172.21.15.172:0/2724742980 shutdown_connections 2024-09-10T08:57:15.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.271+0000 7f412c4a2740 1 --2- 172.21.15.172:0/2724742980 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55788425dd50 0x557884260210 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.271+0000 7f412c4a2740 1 --2- 172.21.15.172:0/2724742980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 0x55788424b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.271+0000 7f412c4a2740 1 --2- 172.21.15.172:0/2724742980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788423ef80 0x55788424b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.271+0000 7f412c4a2740 1 --2- 172.21.15.172:0/2724742980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423e660 0x55788424abf0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.271+0000 7f4102ffd640 0 client.27623 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:15.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.271+0000 7f412c4a2740 1 -- 172.21.15.172:0/2724742980 >> 172.21.15.172:0/2724742980 conn(0x55788423acf0 msgr2=0x557884240a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:15.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.272+0000 7f412c4a2740 1 -- 172.21.15.172:0/2724742980 shutdown_connections 2024-09-10T08:57:15.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.272+0000 7f412c4a2740 1 -- 172.21.15.172:0/2724742980 wait complete. 2024-09-10T08:57:15.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.272+0000 7f412c4a2740 20 client.27623 trim_cache size 0 max 16384 2024-09-10T08:57:15.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.291+0000 7f412c4a2740 1 Processor -- start 2024-09-10T08:57:15.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.291+0000 7f412c4a2740 1 -- start start 2024-09-10T08:57:15.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.291+0000 7f412c4a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788423e660 0x5578842494c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.291+0000 7f412c4a2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423ef80 0x557884265ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.291+0000 7f412c4a2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788424dd50 0x557884266020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.292+0000 7f412c4a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f41241ac870 con 0x55788423e660 2024-09-10T08:57:15.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.292+0000 7f412c4a2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41241ae410 con 0x55788423ef80 2024-09-10T08:57:15.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.292+0000 7f412c4a2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f41241acf40 con 0x55788424dd50 2024-09-10T08:57:15.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.292+0000 7f41237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788423e660 0x5578842494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:15.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.292+0000 7f41237fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788423e660 0x5578842494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35888/0 (socket says 172.21.15.172:35888) 2024-09-10T08:57:15.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.292+0000 7f41237fe640 1 -- 172.21.15.172:0/823411354 learned_addr learned my addr 172.21.15.172:0/823411354 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:15.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.292+0000 7f4122ffd640 1 --2- 172.21.15.172:0/823411354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423ef80 0x557884265ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:15.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.292+0000 7f4123fff640 1 --2- 172.21.15.172:0/823411354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788424dd50 0x557884266020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:15.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.292+0000 7f41237fe640 1 -- 172.21.15.172:0/823411354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788424dd50 msgr2=0x557884266020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:15.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.292+0000 7f41237fe640 1 --2- 172.21.15.172:0/823411354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788424dd50 0x557884266020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.292+0000 7f41237fe640 1 -- 172.21.15.172:0/823411354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423ef80 msgr2=0x557884265ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:15.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.292+0000 7f41237fe640 1 --2- 172.21.15.172:0/823411354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423ef80 0x557884265ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.292+0000 7f41237fe640 1 -- 172.21.15.172:0/823411354 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55788423b7b0 con 0x55788423e660 2024-09-10T08:57:15.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.292+0000 7f41237fe640 1 --2- 172.21.15.172:0/823411354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788423e660 0x5578842494c0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f411409b2a0 tx=0x7f41140d23b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b42476e6bb7e7b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:15.297 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.292+0000 7f41097fa640 1 -- 172.21.15.172:0/823411354 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41140b9070 con 0x55788423e660 2024-09-10T08:57:15.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.292+0000 7f41097fa640 1 -- 172.21.15.172:0/823411354 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41140b4230 con 0x55788423e660 2024-09-10T08:57:15.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.292+0000 7f41097fa640 1 -- 172.21.15.172:0/823411354 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114014030 con 0x55788423e660 2024-09-10T08:57:15.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.293+0000 7f412c4a2740 1 -- 172.21.15.172:0/823411354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788423e660 msgr2=0x5578842494c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:15.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.293+0000 7f412c4a2740 1 --2- 172.21.15.172:0/823411354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788423e660 0x5578842494c0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f411409b2a0 tx=0x7f41140d23b0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.293+0000 7f412c4a2740 1 -- 172.21.15.172:0/823411354 shutdown_connections 2024-09-10T08:57:15.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.293+0000 7f412c4a2740 1 --2- 172.21.15.172:0/823411354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788424dd50 0x557884266020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.293+0000 7f412c4a2740 1 --2- 172.21.15.172:0/823411354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423ef80 0x557884265ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.293+0000 7f412c4a2740 1 --2- 172.21.15.172:0/823411354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788423e660 0x5578842494c0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.293+0000 7f412c4a2740 1 -- 172.21.15.172:0/823411354 >> 172.21.15.172:0/823411354 conn(0x55788423acf0 msgr2=0x55788424cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:15.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.293+0000 7f412c4a2740 1 -- 172.21.15.172:0/823411354 shutdown_connections 2024-09-10T08:57:15.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.293+0000 7f412c4a2740 1 -- 172.21.15.172:0/823411354 wait complete. 2024-09-10T08:57:15.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.294+0000 7f412c4a2740 1 -- start start 2024-09-10T08:57:15.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.294+0000 7f412c4a2740 1 Processor -- start 2024-09-10T08:57:15.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.294+0000 7f412c4a2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423e660 0x557884251b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.294+0000 7f412c4a2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788423ef80 0x557884252070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.294+0000 7f412c4a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 0x5578842525b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.294+0000 7f412c4a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f41241add40 con 0x55788424dd50 2024-09-10T08:57:15.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.294+0000 7f412c4a2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41241ad670 con 0x55788423e660 2024-09-10T08:57:15.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.294+0000 7f412c4a2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f41241aa850 con 0x55788423ef80 2024-09-10T08:57:15.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.294+0000 7f412c4a2740 10 client.0 _ll_register_callbacks cb 0x7f41140d2530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:15.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.295+0000 7f41237fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423e660 0x557884251b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:15.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.295+0000 7f4123fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 0x5578842525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:15.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.295+0000 7f41237fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423e660 0x557884251b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:51628/0 (socket says 172.21.15.172:51628) 2024-09-10T08:57:15.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.295+0000 7f4123fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 0x5578842525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:35900/0 (socket says 172.21.15.172:35900) 2024-09-10T08:57:15.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.295+0000 7f41237fe640 1 -- 172.21.15.172:0/2421911207 learned_addr learned my addr 172.21.15.172:0/2421911207 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:15.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.295+0000 7f4122ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788423ef80 0x557884252070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:15.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.295+0000 7f41237fe640 1 -- 172.21.15.172:0/2421911207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788423ef80 msgr2=0x557884252070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:15.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.295+0000 7f41237fe640 1 --2- 172.21.15.172:0/2421911207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788423ef80 0x557884252070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.295+0000 7f41237fe640 1 -- 172.21.15.172:0/2421911207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 msgr2=0x5578842525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:15.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.295+0000 7f41237fe640 1 --2- 172.21.15.172:0/2421911207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 0x5578842525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.295+0000 7f41237fe640 1 -- 172.21.15.172:0/2421911207 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557884249ba0 con 0x55788423e660 2024-09-10T08:57:15.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.295+0000 7f4123fff640 1 --2- 172.21.15.172:0/2421911207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 0x5578842525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:15.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.295+0000 7f41237fe640 1 --2- 172.21.15.172:0/2421911207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423e660 0x557884251b30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f41140976a0 tx=0x7f41140976d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cc04857b49a5095 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:15.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.295+0000 7f4102ffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:57:15.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.296+0000 7f4102ffd640 1 -- 172.21.15.172:0/2421911207 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41140b9070 con 0x55788423e660 2024-09-10T08:57:15.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.296+0000 7f412c4a2740 1 -- 172.21.15.172:0/2421911207 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41140b4230 con 0x55788423e660 2024-09-10T08:57:15.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.296+0000 7f4102ffd640 1 -- 172.21.15.172:0/2421911207 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f411409f120 con 0x55788423e660 2024-09-10T08:57:15.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.296+0000 7f4102ffd640 1 -- 172.21.15.172:0/2421911207 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4114020070 con 0x55788423e660 2024-09-10T08:57:15.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.296+0000 7f412c4a2740 10 client.27629 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:15.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.296+0000 7f412c4a2740 1 -- 172.21.15.172:0/2421911207 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5578842c8bf0 con 0x55788423e660 2024-09-10T08:57:15.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.296+0000 7f4102ffd640 1 -- 172.21.15.172:0/2421911207 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41140b4230 con 0x55788423e660 2024-09-10T08:57:15.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.296+0000 7f4102ffd640 1 -- 172.21.15.172:0/2421911207 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f4114097060 con 0x55788423e660 2024-09-10T08:57:15.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.296+0000 7f412c4a2740 10 client.27629 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:15.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.296+0000 7f412c4a2740 10 client.27629 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:57:15.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.296+0000 7f412c4a2740 20 client.27629 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:15.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.296+0000 7f412c4a2740 1 --2- 172.21.15.172:0/2421911207 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5578842c9090 0x5578842cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:15.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.296+0000 7f412c4a2740 4 client.27629 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}] 2024-09-10T08:57:15.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.296+0000 7f412c4a2740 1 -- 172.21.15.172:0/2421911207 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}) -- 0x5578841abd20 con 0x5578842c9090 2024-09-10T08:57:15.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.297+0000 7f4122ffd640 1 --2- 172.21.15.172:0/2421911207 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5578842c9090 0x5578842cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:15.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.297+0000 7f4102ffd640 10 client.27629 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:15.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.297+0000 7f4122ffd640 1 --2- 172.21.15.172:0/2421911207 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5578842c9090 0x5578842cb480 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d5ac312ce117908 server_cookie=c45576a97c706360 in_seq=0 out_seq=0 2024-09-10T08:57:15.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.395+0000 7f4102ffd640 1 -- 172.21.15.172:0/2421911207 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5578841abd20 con 0x5578842c9090 2024-09-10T08:57:15.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.395+0000 7f4102ffd640 10 client.27629 handle_command_reply: tid=0 2024-09-10T08:57:15.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.395+0000 7f412c4a2740 1 client.27629 shutdown 2024-09-10T08:57:15.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.395+0000 7f412c4a2740 1 -- 172.21.15.172:0/2421911207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423e660 msgr2=0x557884251b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:15.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.395+0000 7f412c4a2740 1 --2- 172.21.15.172:0/2421911207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423e660 0x557884251b30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f41140976a0 tx=0x7f41140976d0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.395+0000 7f412c4a2740 1 -- 172.21.15.172:0/2421911207 shutdown_connections 2024-09-10T08:57:15.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.395+0000 7f412c4a2740 1 --2- 172.21.15.172:0/2421911207 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5578842c9090 0x5578842cb480 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.395+0000 7f412c4a2740 1 --2- 172.21.15.172:0/2421911207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55788424dd50 0x5578842525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.395+0000 7f4102ffd640 0 client.27629 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:15.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.395+0000 7f412c4a2740 1 --2- 172.21.15.172:0/2421911207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55788423ef80 0x557884252070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.395+0000 7f412c4a2740 1 --2- 172.21.15.172:0/2421911207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55788423e660 0x557884251b30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.395+0000 7f412c4a2740 1 -- 172.21.15.172:0/2421911207 >> 172.21.15.172:0/2421911207 conn(0x55788423acf0 msgr2=0x55788424cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:15.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.395+0000 7f412c4a2740 1 -- 172.21.15.172:0/2421911207 shutdown_connections 2024-09-10T08:57:15.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.395+0000 7f412c4a2740 1 -- 172.21.15.172:0/2421911207 wait complete. 2024-09-10T08:57:15.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.395+0000 7f412c4a2740 20 client.27629 trim_cache size 0 max 16384 2024-09-10T08:57:15.399 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:57:15.399 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 39, 2024-09-10T08:57:15.399 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:57:15.399 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 38, 2024-09-10T08:57:15.399 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:57:15.399 INFO:teuthology.orchestra.run.smithi172.stdout: "bb1cd559": { 2024-09-10T08:57:15.399 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 38, 2024-09-10T08:57:15.399 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:57:15.399 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:57:15.400 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:57:15.400 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:57:15.400 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:57:15.400 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:57:15.400 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 145.2, 2024-09-10T08:57:15.400 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:57:15.400 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:57:15.400 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:57:15.400 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:57:15.400 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:57:15.400 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:57:15.400 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:57:15.400 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:57:15.401 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:57:15.401 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:57:15.401 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:57:15.401 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:57:15.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.396+0000 7f412a0c9640 1 -- 172.21.15.172:0/3151079634 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f410c0a2b80 msgr2=0x7f410c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:15.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.396+0000 7f412a0c9640 1 --2- 172.21.15.172:0/3151079634 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f410c0a2b80 0x7f410c0a5040 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f411c005e30 tx=0x7f411c0095c0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.396+0000 7f412a0c9640 1 -- 172.21.15.172:0/3151079634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4124100e20 msgr2=0x7f41241983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:15.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.396+0000 7f412a0c9640 1 --2- 172.21.15.172:0/3151079634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4124100e20 0x7f41241983c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f411400d770 tx=0x7f411400dc30 comp rx=0 tx=0).stop 2024-09-10T08:57:15.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.396+0000 7f412a0c9640 1 -- 172.21.15.172:0/3151079634 shutdown_connections 2024-09-10T08:57:15.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.396+0000 7f412a0c9640 1 --2- 172.21.15.172:0/3151079634 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f410c0a2b80 0x7f410c0a5040 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.396+0000 7f412a0c9640 1 --2- 172.21.15.172:0/3151079634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4124102ef0 0x7f412419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.396+0000 7f412a0c9640 1 --2- 172.21.15.172:0/3151079634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4124102570 0x7f4124198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.396+0000 7f412a0c9640 1 --2- 172.21.15.172:0/3151079634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4124100e20 0x7f41241983c0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:15.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.396+0000 7f412a0c9640 1 -- 172.21.15.172:0/3151079634 >> 172.21.15.172:0/3151079634 conn(0x7f41240fc7d0 msgr2=0x7f4124069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:15.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.396+0000 7f412a0c9640 1 -- 172.21.15.172:0/3151079634 shutdown_connections 2024-09-10T08:57:15.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:15.396+0000 7f412a0c9640 1 -- 172.21.15.172:0/3151079634 wait complete. 2024-09-10T08:57:15.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:15 smithi172 ceph-mon[30854]: pgmap v681: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-09-10T08:57:15.729 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:57:15.729 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:57:15.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:15 smithi174 ceph-mon[34068]: pgmap v681: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-09-10T08:57:15.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:15 smithi191 ceph-mon[35314]: pgmap v681: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-09-10T08:57:16.075 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bb1cd559', quiesce took 6.9 seconds. Will release after: 18.299999999999997 2024-09-10T08:57:16.160 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:57:16.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.621+0000 7fd381814640 1 -- 172.21.15.172:0/250673207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd37c10c170 msgr2=0x7fd37c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.621+0000 7fd381814640 1 --2- 172.21.15.172:0/250673207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd37c10c170 0x7fd37c10e610 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fd374009be0 tx=0x7fd37402a750 comp rx=0 tx=0).stop 2024-09-10T08:57:16.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.621+0000 7fd381814640 1 -- 172.21.15.172:0/250673207 shutdown_connections 2024-09-10T08:57:16.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.621+0000 7fd381814640 1 --2- 172.21.15.172:0/250673207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd37c10c170 0x7fd37c10e610 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.621+0000 7fd381814640 1 --2- 172.21.15.172:0/250673207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd37c103660 0x7fd37c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.621+0000 7fd381814640 1 --2- 172.21.15.172:0/250673207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd37c102d40 0x7fd37c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.621+0000 7fd381814640 1 -- 172.21.15.172:0/250673207 >> 172.21.15.172:0/250673207 conn(0x7fd37c0fc7a0 msgr2=0x7fd37c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:16.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.621+0000 7fd381814640 1 -- 172.21.15.172:0/250673207 shutdown_connections 2024-09-10T08:57:16.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.621+0000 7fd381814640 1 -- 172.21.15.172:0/250673207 wait complete. 2024-09-10T08:57:16.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.622+0000 7fd381814640 1 Processor -- start 2024-09-10T08:57:16.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.622+0000 7fd381814640 1 -- start start 2024-09-10T08:57:16.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.622+0000 7fd381814640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd37c102d40 0x7fd37c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.622+0000 7fd381814640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd37c103660 0x7fd37c198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.622+0000 7fd381814640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd37c10c170 0x7fd37c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.622+0000 7fd381814640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c111de0 con 0x7fd37c102d40 2024-09-10T08:57:16.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.622+0000 7fd381814640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c111c60 con 0x7fd37c10c170 2024-09-10T08:57:16.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.622+0000 7fd381814640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c111f60 con 0x7fd37c103660 2024-09-10T08:57:16.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.622+0000 7fd37affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd37c102d40 0x7fd37c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.622+0000 7fd37affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd37c102d40 0x7fd37c1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39304/0 (socket says 172.21.15.172:39304) 2024-09-10T08:57:16.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.622+0000 7fd37affd640 1 -- 172.21.15.172:0/1637290385 learned_addr learned my addr 172.21.15.172:0/1637290385 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:16.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.622+0000 7fd37b7fe640 1 --2- 172.21.15.172:0/1637290385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd37c10c170 0x7fd37c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.622+0000 7fd37a7fc640 1 --2- 172.21.15.172:0/1637290385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd37c103660 0x7fd37c198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.623+0000 7fd37affd640 1 -- 172.21.15.172:0/1637290385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd37c103660 msgr2=0x7fd37c198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.623+0000 7fd37affd640 1 --2- 172.21.15.172:0/1637290385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd37c103660 0x7fd37c198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.623+0000 7fd37affd640 1 -- 172.21.15.172:0/1637290385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd37c10c170 msgr2=0x7fd37c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.623+0000 7fd37affd640 1 --2- 172.21.15.172:0/1637290385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd37c10c170 0x7fd37c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.623+0000 7fd37affd640 1 -- 172.21.15.172:0/1637290385 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37c19d290 con 0x7fd37c102d40 2024-09-10T08:57:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.623+0000 7fd37affd640 1 --2- 172.21.15.172:0/1637290385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd37c102d40 0x7fd37c1982e0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00e9c0 tx=0x7fd36c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.623+0000 7fd380812640 1 -- 172.21.15.172:0/1637290385 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c00cde0 con 0x7fd37c102d40 2024-09-10T08:57:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.623+0000 7fd381814640 1 -- 172.21.15.172:0/1637290385 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd37c19d580 con 0x7fd37c102d40 2024-09-10T08:57:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.623+0000 7fd380812640 1 -- 172.21.15.172:0/1637290385 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd36c0035f0 con 0x7fd37c102d40 2024-09-10T08:57:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.623+0000 7fd380812640 1 -- 172.21.15.172:0/1637290385 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c010690 con 0x7fd37c102d40 2024-09-10T08:57:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.623+0000 7fd381814640 1 -- 172.21.15.172:0/1637290385 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd37c1a4e60 con 0x7fd37c102d40 2024-09-10T08:57:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.625+0000 7fd380812640 1 -- 172.21.15.172:0/1637290385 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd36c0037a0 con 0x7fd37c102d40 2024-09-10T08:57:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd383bed740 1 Processor -- start 2024-09-10T08:57:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd383bed740 1 -- start start 2024-09-10T08:57:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd380812640 1 --2- 172.21.15.172:0/1637290385 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd3640a2b80 0x7fd3640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd383bed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eab60 0x55a0c18eaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd383bed740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eb480 0x55a0c18f5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd383bed740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18f5d40 0x55a0c18f8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd383bed740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a0c1850280 con 0x55a0c18eb480 2024-09-10T08:57:16.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd383bed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a0c17f7b10 con 0x55a0c18eab60 2024-09-10T08:57:16.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd383bed740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a0c184c240 con 0x55a0c18f5d40 2024-09-10T08:57:16.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd380812640 1 -- 172.21.15.172:0/1637290385 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd36c0cf7f0 con 0x7fd37c102d40 2024-09-10T08:57:16.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd37b7fe640 1 --2- 172.21.15.172:0/1637290385 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd3640a2b80 0x7fd3640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd37a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eb480 0x55a0c18f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd37a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eb480 0x55a0c18f5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39308/0 (socket says 172.21.15.172:39308) 2024-09-10T08:57:16.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd37a7fc640 1 -- 172.21.15.172:0/2401336889 learned_addr learned my addr 172.21.15.172:0/2401336889 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:16.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd37affd640 1 --2- 172.21.15.172:0/2401336889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eab60 0x55a0c18eaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd37b7fe640 1 --2- 172.21.15.172:0/2401336889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18f5d40 0x55a0c18f8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd37a7fc640 1 -- 172.21.15.172:0/2401336889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18f5d40 msgr2=0x55a0c18f8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd37a7fc640 1 --2- 172.21.15.172:0/2401336889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18f5d40 0x55a0c18f8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd37a7fc640 1 -- 172.21.15.172:0/2401336889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eab60 msgr2=0x55a0c18eaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd37a7fc640 1 --2- 172.21.15.172:0/2401336889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eab60 0x55a0c18eaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.626+0000 7fd37a7fc640 1 -- 172.21.15.172:0/2401336889 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0c18041f0 con 0x55a0c18eb480 2024-09-10T08:57:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.627+0000 7fd37b7fe640 1 --2- 172.21.15.172:0/1637290385 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd3640a2b80 0x7fd3640a5040 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd374004af0 tx=0x7fd374009330 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.627+0000 7fd37a7fc640 1 --2- 172.21.15.172:0/2401336889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eb480 0x55a0c18f5800 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fd37000bbf0 tx=0x7fd37000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c570453f865a86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.627+0000 7fd361ffb640 1 -- 172.21.15.172:0/2401336889 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370015070 con 0x55a0c18eb480 2024-09-10T08:57:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.627+0000 7fd361ffb640 1 -- 172.21.15.172:0/2401336889 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd37001a910 con 0x55a0c18eb480 2024-09-10T08:57:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.627+0000 7fd361ffb640 1 -- 172.21.15.172:0/2401336889 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3700117c0 con 0x55a0c18eb480 2024-09-10T08:57:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.627+0000 7fd383bed740 1 -- 172.21.15.172:0/2401336889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eb480 msgr2=0x55a0c18f5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.627+0000 7fd383bed740 1 --2- 172.21.15.172:0/2401336889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eb480 0x55a0c18f5800 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fd37000bbf0 tx=0x7fd37000f730 comp rx=0 tx=0).stop 2024-09-10T08:57:16.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.627+0000 7fd383bed740 1 -- 172.21.15.172:0/2401336889 shutdown_connections 2024-09-10T08:57:16.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.627+0000 7fd383bed740 1 --2- 172.21.15.172:0/2401336889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18f5d40 0x55a0c18f8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.628+0000 7fd383bed740 1 --2- 172.21.15.172:0/2401336889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eb480 0x55a0c18f5800 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.628+0000 7fd383bed740 1 --2- 172.21.15.172:0/2401336889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eab60 0x55a0c18eaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.628+0000 7fd383bed740 1 -- 172.21.15.172:0/2401336889 >> 172.21.15.172:0/2401336889 conn(0x55a0c18e70f0 msgr2=0x55a0c18e74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:16.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.628+0000 7fd383bed740 1 -- 172.21.15.172:0/2401336889 shutdown_connections 2024-09-10T08:57:16.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.628+0000 7fd383bed740 1 -- 172.21.15.172:0/2401336889 wait complete. 2024-09-10T08:57:16.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.628+0000 7fd383bed740 1 -- start start 2024-09-10T08:57:16.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.628+0000 7fd383bed740 1 Processor -- start 2024-09-10T08:57:16.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.628+0000 7fd383bed740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eab60 0x55a0c18f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.628+0000 7fd383bed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eb480 0x55a0c18f3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.642 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.628+0000 7fd383bed740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18f5d40 0x55a0c18f3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.628+0000 7fd383bed740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a0c18f8910 con 0x55a0c18eab60 2024-09-10T08:57:16.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.628+0000 7fd383bed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a0c180af10 con 0x55a0c18eb480 2024-09-10T08:57:16.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.628+0000 7fd383bed740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a0c18f8a90 con 0x55a0c18f5d40 2024-09-10T08:57:16.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.628+0000 7fd383bed740 10 client.0 _ll_register_callbacks cb 0x55a0c17d3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:16.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.628+0000 7fd37affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eab60 0x55a0c18f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd37affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eab60 0x55a0c18f2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39316/0 (socket says 172.21.15.172:39316) 2024-09-10T08:57:16.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd37affd640 1 -- 172.21.15.172:0/2529804053 learned_addr learned my addr 172.21.15.172:0/2529804053 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:16.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd37b7fe640 1 --2- 172.21.15.172:0/2529804053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18f5d40 0x55a0c18f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd37affd640 1 -- 172.21.15.172:0/2529804053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18f5d40 msgr2=0x55a0c18f3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:16.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd37a7fc640 1 --2- 172.21.15.172:0/2529804053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eb480 0x55a0c18f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd37affd640 1 --2- 172.21.15.172:0/2529804053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18f5d40 0x55a0c18f3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd37affd640 1 -- 172.21.15.172:0/2529804053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eb480 msgr2=0x55a0c18f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd37affd640 1 --2- 172.21.15.172:0/2529804053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eb480 0x55a0c18f3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd37affd640 1 -- 172.21.15.172:0/2529804053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0c15c16a0 con 0x55a0c18eab60 2024-09-10T08:57:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd37b7fe640 1 --2- 172.21.15.172:0/2529804053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18f5d40 0x55a0c18f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:57:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd37affd640 1 --2- 172.21.15.172:0/2529804053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eab60 0x55a0c18f2bf0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0ccf90 tx=0x7fd36c003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41944623d4c1baa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd35affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd35affd640 1 -- 172.21.15.172:0/2529804053 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c0a00a0 con 0x55a0c18eab60 2024-09-10T08:57:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd35affd640 1 -- 172.21.15.172:0/2529804053 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd36c0a44d0 con 0x55a0c18eab60 2024-09-10T08:57:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd383bed740 1 -- 172.21.15.172:0/2529804053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd37001a910 con 0x55a0c18eab60 2024-09-10T08:57:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd35affd640 1 -- 172.21.15.172:0/2529804053 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd36c0a4960 con 0x55a0c18eab60 2024-09-10T08:57:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd383bed740 10 client.18747 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd383bed740 1 -- 172.21.15.172:0/2529804053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a0c1904540 con 0x55a0c18eab60 2024-09-10T08:57:16.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.629+0000 7fd35affd640 1 -- 172.21.15.172:0/2529804053 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd37001a910 con 0x55a0c18eab60 2024-09-10T08:57:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.630+0000 7fd35affd640 1 -- 172.21.15.172:0/2529804053 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fd36c09abb0 con 0x55a0c18eab60 2024-09-10T08:57:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.630+0000 7fd383bed740 10 client.18747 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.630+0000 7fd383bed740 10 client.18747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.630+0000 7fd383bed740 20 client.18747 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.630+0000 7fd383bed740 1 --2- 172.21.15.172:0/2529804053 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a0c1905950 0x55a0c1907dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.630+0000 7fd383bed740 4 client.18747 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:57:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.630+0000 7fd383bed740 1 -- 172.21.15.172:0/2529804053 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0c185d7f0 con 0x55a0c1905950 2024-09-10T08:57:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.630+0000 7fd37a7fc640 1 --2- 172.21.15.172:0/2529804053 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a0c1905950 0x55a0c1907dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.630+0000 7fd35affd640 10 client.18747 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:16.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.630+0000 7fd37a7fc640 1 --2- 172.21.15.172:0/2529804053 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a0c1905950 0x55a0c1907dc0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ac5a6802157a8d3 server_cookie=f3462c363aec76ee in_seq=0 out_seq=0 2024-09-10T08:57:16.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.631+0000 7fd35affd640 1 -- 172.21.15.172:0/2529804053 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0c185d7f0 con 0x55a0c1905950 2024-09-10T08:57:16.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.631+0000 7fd35affd640 10 client.18747 handle_command_reply: tid=0 2024-09-10T08:57:16.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.631+0000 7fd383bed740 1 client.18747 shutdown 2024-09-10T08:57:16.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.631+0000 7fd383bed740 1 -- 172.21.15.172:0/2529804053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eab60 msgr2=0x55a0c18f2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.631+0000 7fd383bed740 1 --2- 172.21.15.172:0/2529804053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eab60 0x55a0c18f2bf0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0ccf90 tx=0x7fd36c003c50 comp rx=0 tx=0).stop 2024-09-10T08:57:16.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.631+0000 7fd383bed740 1 -- 172.21.15.172:0/2529804053 shutdown_connections 2024-09-10T08:57:16.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.631+0000 7fd383bed740 1 --2- 172.21.15.172:0/2529804053 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a0c1905950 0x55a0c1907dc0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.646 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.631+0000 7fd383bed740 1 --2- 172.21.15.172:0/2529804053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18f5d40 0x55a0c18f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.631+0000 7fd383bed740 1 --2- 172.21.15.172:0/2529804053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eb480 0x55a0c18f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.631+0000 7fd383bed740 1 --2- 172.21.15.172:0/2529804053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eab60 0x55a0c18f2bf0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.631+0000 7fd383bed740 1 -- 172.21.15.172:0/2529804053 >> 172.21.15.172:0/2529804053 conn(0x55a0c18e70f0 msgr2=0x55a0c18ece20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:16.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.631+0000 7fd35affd640 0 client.18747 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:16.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.631+0000 7fd383bed740 1 -- 172.21.15.172:0/2529804053 shutdown_connections 2024-09-10T08:57:16.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.631+0000 7fd383bed740 1 -- 172.21.15.172:0/2529804053 wait complete. 2024-09-10T08:57:16.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.631+0000 7fd383bed740 20 client.18747 trim_cache size 0 max 16384 2024-09-10T08:57:16.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd383bed740 1 Processor -- start 2024-09-10T08:57:16.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd383bed740 1 -- start start 2024-09-10T08:57:16.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd383bed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eab60 0x55a0c18ecd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd383bed740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18eb480 0x55a0c190d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd383bed740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18f5d40 0x55a0c190dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd383bed740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c1aeb10 con 0x55a0c18f5d40 2024-09-10T08:57:16.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd383bed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c1b06b0 con 0x55a0c18eab60 2024-09-10T08:57:16.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd383bed740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c1af1e0 con 0x55a0c18eb480 2024-09-10T08:57:16.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd37b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18f5d40 0x55a0c190dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd37b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18f5d40 0x55a0c190dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39326/0 (socket says 172.21.15.172:39326) 2024-09-10T08:57:16.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd37b7fe640 1 -- 172.21.15.172:0/2150302640 learned_addr learned my addr 172.21.15.172:0/2150302640 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:16.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd37affd640 1 --2- 172.21.15.172:0/2150302640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eab60 0x55a0c18ecd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd37a7fc640 1 --2- 172.21.15.172:0/2150302640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18eb480 0x55a0c190d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd37b7fe640 1 -- 172.21.15.172:0/2150302640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18eb480 msgr2=0x55a0c190d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:16.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd37b7fe640 1 --2- 172.21.15.172:0/2150302640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18eb480 0x55a0c190d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd37b7fe640 1 -- 172.21.15.172:0/2150302640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eab60 msgr2=0x55a0c18ecd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:16.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd37b7fe640 1 --2- 172.21.15.172:0/2150302640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eab60 0x55a0c18ecd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd37b7fe640 1 -- 172.21.15.172:0/2150302640 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0c18e6f20 con 0x55a0c18f5d40 2024-09-10T08:57:16.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.649+0000 7fd37affd640 1 --2- 172.21.15.172:0/2150302640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eab60 0x55a0c18ecd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:16.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.650+0000 7fd37b7fe640 1 --2- 172.21.15.172:0/2150302640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18f5d40 0x55a0c190dbd0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fd374009be0 tx=0x7fd374043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a67d14f821eeeb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:16.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.650+0000 7fd3617fa640 1 -- 172.21.15.172:0/2150302640 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374050070 con 0x55a0c18f5d40 2024-09-10T08:57:16.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.650+0000 7fd3617fa640 1 -- 172.21.15.172:0/2150302640 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd374037040 con 0x55a0c18f5d40 2024-09-10T08:57:16.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.650+0000 7fd3617fa640 1 -- 172.21.15.172:0/2150302640 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374036050 con 0x55a0c18f5d40 2024-09-10T08:57:16.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.651+0000 7fd383bed740 1 -- 172.21.15.172:0/2150302640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18f5d40 msgr2=0x55a0c190dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.651+0000 7fd383bed740 1 --2- 172.21.15.172:0/2150302640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18f5d40 0x55a0c190dbd0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fd374009be0 tx=0x7fd374043000 comp rx=0 tx=0).stop 2024-09-10T08:57:16.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.651+0000 7fd383bed740 1 -- 172.21.15.172:0/2150302640 shutdown_connections 2024-09-10T08:57:16.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.651+0000 7fd383bed740 1 --2- 172.21.15.172:0/2150302640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18f5d40 0x55a0c190dbd0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.651+0000 7fd383bed740 1 --2- 172.21.15.172:0/2150302640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18eb480 0x55a0c190d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.651+0000 7fd383bed740 1 --2- 172.21.15.172:0/2150302640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eab60 0x55a0c18ecd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.651+0000 7fd383bed740 1 -- 172.21.15.172:0/2150302640 >> 172.21.15.172:0/2150302640 conn(0x55a0c18e70f0 msgr2=0x55a0c18ec600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:16.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.651+0000 7fd383bed740 1 -- 172.21.15.172:0/2150302640 shutdown_connections 2024-09-10T08:57:16.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.651+0000 7fd383bed740 1 -- 172.21.15.172:0/2150302640 wait complete. 2024-09-10T08:57:16.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.651+0000 7fd383bed740 1 -- start start 2024-09-10T08:57:16.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.651+0000 7fd383bed740 1 Processor -- start 2024-09-10T08:57:16.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.651+0000 7fd383bed740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eab60 0x55a0c18ef040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.651+0000 7fd383bed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eb480 0x55a0c18ef580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.651+0000 7fd383bed740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18f5d40 0x55a0c18efcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd383bed740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c1affe0 con 0x55a0c18eab60 2024-09-10T08:57:16.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd383bed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c1af910 con 0x55a0c18eb480 2024-09-10T08:57:16.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd383bed740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c1acaf0 con 0x55a0c18f5d40 2024-09-10T08:57:16.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd383bed740 10 client.0 _ll_register_callbacks cb 0x7fd36c09bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:16.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd37affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eab60 0x55a0c18ef040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd37b7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18f5d40 0x55a0c18efcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd37affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eab60 0x55a0c18ef040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39342/0 (socket says 172.21.15.172:39342) 2024-09-10T08:57:16.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd37affd640 1 -- 172.21.15.172:0/1771230112 learned_addr learned my addr 172.21.15.172:0/1771230112 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:16.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd37a7fc640 1 --2- 172.21.15.172:0/1771230112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eb480 0x55a0c18ef580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd37affd640 1 -- 172.21.15.172:0/1771230112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18f5d40 msgr2=0x55a0c18efcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:16.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd37affd640 1 --2- 172.21.15.172:0/1771230112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18f5d40 0x55a0c18efcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd37affd640 1 -- 172.21.15.172:0/1771230112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eb480 msgr2=0x55a0c18ef580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:16.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd37affd640 1 --2- 172.21.15.172:0/1771230112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eb480 0x55a0c18ef580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd37affd640 1 -- 172.21.15.172:0/1771230112 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0c1908130 con 0x55a0c18eab60 2024-09-10T08:57:16.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd37a7fc640 1 --2- 172.21.15.172:0/1771230112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eb480 0x55a0c18ef580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:16.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd37b7fe640 1 --2- 172.21.15.172:0/1771230112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18f5d40 0x55a0c18efcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:16.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd37affd640 1 --2- 172.21.15.172:0/1771230112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eab60 0x55a0c18ef040 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0a4430 tx=0x7fd36c0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd3f52455b74cc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:16.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd35affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:16.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.652+0000 7fd35affd640 1 -- 172.21.15.172:0/1771230112 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c0a46d0 con 0x55a0c18eab60 2024-09-10T08:57:16.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.653+0000 7fd35affd640 1 -- 172.21.15.172:0/1771230112 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd36c09b140 con 0x55a0c18eab60 2024-09-10T08:57:16.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.653+0000 7fd383bed740 1 -- 172.21.15.172:0/1771230112 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd374037040 con 0x55a0c18eab60 2024-09-10T08:57:16.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.653+0000 7fd35affd640 1 -- 172.21.15.172:0/1771230112 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd36c0a00a0 con 0x55a0c18eab60 2024-09-10T08:57:16.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.653+0000 7fd383bed740 10 client.18753 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:16.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.653+0000 7fd383bed740 1 -- 172.21.15.172:0/1771230112 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a0c1974ea0 con 0x55a0c18eab60 2024-09-10T08:57:16.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.653+0000 7fd35affd640 1 -- 172.21.15.172:0/1771230112 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd374037040 con 0x55a0c18eab60 2024-09-10T08:57:16.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.653+0000 7fd35affd640 1 -- 172.21.15.172:0/1771230112 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fd36c064070 con 0x55a0c18eab60 2024-09-10T08:57:16.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.653+0000 7fd383bed740 10 client.18753 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:16.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.653+0000 7fd383bed740 10 client.18753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:16.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.653+0000 7fd383bed740 20 client.18753 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:16.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.653+0000 7fd383bed740 1 --2- 172.21.15.172:0/1771230112 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a0c19755c0 0x55a0c19779e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.653+0000 7fd383bed740 4 client.18753 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:57:16.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.653+0000 7fd383bed740 1 -- 172.21.15.172:0/1771230112 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a0c1857d00 con 0x55a0c19755c0 2024-09-10T08:57:16.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.653+0000 7fd37a7fc640 1 --2- 172.21.15.172:0/1771230112 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a0c19755c0 0x55a0c19779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.654+0000 7fd35affd640 10 client.18753 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:16.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.654+0000 7fd37a7fc640 1 --2- 172.21.15.172:0/1771230112 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a0c19755c0 0x55a0c19779e0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0badb7d345b046 server_cookie=32e986deb785d0fc in_seq=0 out_seq=0 2024-09-10T08:57:16.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.655+0000 7fd35affd640 1 -- 172.21.15.172:0/1771230112 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a0c1857d00 con 0x55a0c19755c0 2024-09-10T08:57:16.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.655+0000 7fd35affd640 10 client.18753 handle_command_reply: tid=0 2024-09-10T08:57:16.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.655+0000 7fd383bed740 1 client.18753 shutdown 2024-09-10T08:57:16.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.655+0000 7fd383bed740 1 -- 172.21.15.172:0/1771230112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eab60 msgr2=0x55a0c18ef040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.655+0000 7fd383bed740 1 --2- 172.21.15.172:0/1771230112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eab60 0x55a0c18ef040 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0a4430 tx=0x7fd36c0092d0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.655+0000 7fd383bed740 1 -- 172.21.15.172:0/1771230112 shutdown_connections 2024-09-10T08:57:16.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.655+0000 7fd383bed740 1 --2- 172.21.15.172:0/1771230112 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a0c19755c0 0x55a0c19779e0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.655+0000 7fd383bed740 1 --2- 172.21.15.172:0/1771230112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a0c18f5d40 0x55a0c18efcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.655+0000 7fd383bed740 1 --2- 172.21.15.172:0/1771230112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0c18eb480 0x55a0c18ef580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.655+0000 7fd35affd640 0 client.18753 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:16.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.655+0000 7fd383bed740 1 --2- 172.21.15.172:0/1771230112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0c18eab60 0x55a0c18ef040 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.655+0000 7fd383bed740 1 -- 172.21.15.172:0/1771230112 >> 172.21.15.172:0/1771230112 conn(0x55a0c18e70f0 msgr2=0x55a0c18f33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:16.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.655+0000 7fd383bed740 1 -- 172.21.15.172:0/1771230112 shutdown_connections 2024-09-10T08:57:16.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.655+0000 7fd383bed740 1 -- 172.21.15.172:0/1771230112 wait complete. 2024-09-10T08:57:16.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.655+0000 7fd383bed740 20 client.18753 trim_cache size 0 max 16384 2024-09-10T08:57:16.663 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:57:16.664 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T08:57:16.664 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "6094529b-e458-4124-983f-dbb8bcbd4a36", 2024-09-10T08:57:16.664 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T08:57:16.664 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:57:16.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.656+0000 7fd381814640 1 -- 172.21.15.172:0/1637290385 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd3640a2b80 msgr2=0x7fd3640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.656+0000 7fd381814640 1 --2- 172.21.15.172:0/1637290385 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd3640a2b80 0x7fd3640a5040 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd374004af0 tx=0x7fd374009330 comp rx=0 tx=0).stop 2024-09-10T08:57:16.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.656+0000 7fd381814640 1 -- 172.21.15.172:0/1637290385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd37c102d40 msgr2=0x7fd37c1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.656+0000 7fd381814640 1 --2- 172.21.15.172:0/1637290385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd37c102d40 0x7fd37c1982e0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00e9c0 tx=0x7fd36c00ee80 comp rx=0 tx=0).stop 2024-09-10T08:57:16.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.656+0000 7fd381814640 1 -- 172.21.15.172:0/1637290385 shutdown_connections 2024-09-10T08:57:16.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.656+0000 7fd381814640 1 --2- 172.21.15.172:0/1637290385 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd3640a2b80 0x7fd3640a5040 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.656+0000 7fd381814640 1 --2- 172.21.15.172:0/1637290385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd37c10c170 0x7fd37c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.656+0000 7fd381814640 1 --2- 172.21.15.172:0/1637290385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd37c103660 0x7fd37c198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.656+0000 7fd381814640 1 --2- 172.21.15.172:0/1637290385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd37c102d40 0x7fd37c1982e0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.656+0000 7fd381814640 1 -- 172.21.15.172:0/1637290385 >> 172.21.15.172:0/1637290385 conn(0x7fd37c0fc7a0 msgr2=0x7fd37c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:16.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.656+0000 7fd381814640 1 -- 172.21.15.172:0/1637290385 shutdown_connections 2024-09-10T08:57:16.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:16.656+0000 7fd381814640 1 -- 172.21.15.172:0/1637290385 wait complete. 2024-09-10T08:57:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.671+0000 7f48703d6640 1 -- 172.21.15.172:0/2818376008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4868101b40 msgr2=0x7f4868101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.671+0000 7f48703d6640 1 --2- 172.21.15.172:0/2818376008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4868101b40 0x7f4868101f40 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f4864009900 tx=0x7f486402a410 comp rx=0 tx=0).stop 2024-09-10T08:57:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.671+0000 7f48703d6640 1 -- 172.21.15.172:0/2818376008 shutdown_connections 2024-09-10T08:57:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.671+0000 7f48703d6640 1 --2- 172.21.15.172:0/2818376008 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48681036f0 0x7f4868109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.671+0000 7f48703d6640 1 --2- 172.21.15.172:0/2818376008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4868102d30 0x7f48681031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.671+0000 7f48703d6640 1 --2- 172.21.15.172:0/2818376008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4868101b40 0x7f4868101f40 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.671+0000 7f48703d6640 1 -- 172.21.15.172:0/2818376008 >> 172.21.15.172:0/2818376008 conn(0x7f48680fd2e0 msgr2=0x7f48680ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.671+0000 7f48703d6640 1 -- 172.21.15.172:0/2818376008 shutdown_connections 2024-09-10T08:57:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.671+0000 7f48703d6640 1 -- 172.21.15.172:0/2818376008 wait complete. 2024-09-10T08:57:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.672+0000 7f48703d6640 1 Processor -- start 2024-09-10T08:57:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.672+0000 7f48703d6640 1 -- start start 2024-09-10T08:57:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.672+0000 7f48703d6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4868101b40 0x7f4868197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.672+0000 7f48703d6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4868102d30 0x7f4868198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.672+0000 7f48703d6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48681036f0 0x7f486819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.672+0000 7f48703d6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f486810d650 con 0x7f4868101b40 2024-09-10T08:57:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.672+0000 7f48703d6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f486810d4d0 con 0x7f4868102d30 2024-09-10T08:57:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.672+0000 7f48703d6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f486810d7d0 con 0x7f48681036f0 2024-09-10T08:57:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.672+0000 7f486e172640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4868101b40 0x7f4868197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.676 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.672+0000 7f486e172640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4868101b40 0x7f4868197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39362/0 (socket says 172.21.15.172:39362) 2024-09-10T08:57:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.672+0000 7f486e172640 1 -- 172.21.15.172:0/2271950971 learned_addr learned my addr 172.21.15.172:0/2271950971 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.672+0000 7f486d971640 1 --2- 172.21.15.172:0/2271950971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4868102d30 0x7f4868198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.672+0000 7f486e973640 1 --2- 172.21.15.172:0/2271950971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48681036f0 0x7f486819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.673+0000 7f486e172640 1 -- 172.21.15.172:0/2271950971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48681036f0 msgr2=0x7f486819f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.673+0000 7f486e172640 1 --2- 172.21.15.172:0/2271950971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48681036f0 0x7f486819f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.673+0000 7f486e172640 1 -- 172.21.15.172:0/2271950971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4868102d30 msgr2=0x7f4868198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.673+0000 7f486e172640 1 --2- 172.21.15.172:0/2271950971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4868102d30 0x7f4868198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.673+0000 7f486e172640 1 -- 172.21.15.172:0/2271950971 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486819f960 con 0x7f4868101b40 2024-09-10T08:57:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.673+0000 7f486e172640 1 --2- 172.21.15.172:0/2271950971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4868101b40 0x7f4868197d50 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f48640314e0 tx=0x7f48640042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.673+0000 7f485f7fe640 1 -- 172.21.15.172:0/2271950971 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486403f030 con 0x7f4868101b40 2024-09-10T08:57:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.673+0000 7f48703d6640 1 -- 172.21.15.172:0/2271950971 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486819fbf0 con 0x7f4868101b40 2024-09-10T08:57:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.673+0000 7f485f7fe640 1 -- 172.21.15.172:0/2271950971 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4864003cb0 con 0x7f4868101b40 2024-09-10T08:57:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.673+0000 7f48703d6640 1 -- 172.21.15.172:0/2271950971 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48681a00e0 con 0x7f4868101b40 2024-09-10T08:57:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.673+0000 7f485f7fe640 1 -- 172.21.15.172:0/2271950971 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864049060 con 0x7f4868101b40 2024-09-10T08:57:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.676+0000 7f485f7fe640 1 -- 172.21.15.172:0/2271950971 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4864042030 con 0x7f4868101b40 2024-09-10T08:57:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.677+0000 7f4873251740 1 Processor -- start 2024-09-10T08:57:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.677+0000 7f4873251740 1 -- start start 2024-09-10T08:57:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.677+0000 7f485f7fe640 1 --2- 172.21.15.172:0/2271950971 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f48540a2c30 0x7f48540a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f486e973640 1 --2- 172.21.15.172:0/2271950971 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f48540a2c30 0x7f48540a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f4873251740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 0x55d713d131b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f4873251740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d136f0 0x55d713d181b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f4873251740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d18780 0x55d713d1ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f4873251740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d713c722f0 con 0x55d713d18780 2024-09-10T08:57:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f4873251740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d713c1a270 con 0x55d713d12db0 2024-09-10T08:57:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f4873251740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d713c6e2b0 con 0x55d713d136f0 2024-09-10T08:57:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f485f7fe640 1 -- 172.21.15.172:0/2271950971 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f48640ef980 con 0x7f4868101b40 2024-09-10T08:57:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f486e973640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d18780 0x55d713d1ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f486d971640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d136f0 0x55d713d181b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f486d971640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d136f0 0x55d713d181b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:35200/0 (socket says 172.21.15.172:35200) 2024-09-10T08:57:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f486d971640 1 -- 172.21.15.172:0/2234773972 learned_addr learned my addr 172.21.15.172:0/2234773972 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f486e172640 1 --2- 172.21.15.172:0/2234773972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 0x55d713d131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f486d971640 1 -- 172.21.15.172:0/2234773972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 msgr2=0x55d713d131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f486d971640 1 --2- 172.21.15.172:0/2234773972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 0x55d713d131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f486d971640 1 -- 172.21.15.172:0/2234773972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d18780 msgr2=0x55d713d1ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f486d971640 1 --2- 172.21.15.172:0/2234773972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d18780 0x55d713d1ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f486d971640 1 -- 172.21.15.172:0/2234773972 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d713c19340 con 0x55d713d136f0 2024-09-10T08:57:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f486e973640 1 --2- 172.21.15.172:0/2271950971 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f48540a2c30 0x7f48540a50f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4860005fd0 tx=0x7f4860014bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f486d971640 1 --2- 172.21.15.172:0/2234773972 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d136f0 0x55d713d181b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f485800bd10 tx=0x7f48580107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d26fa13973c86f77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f485cbfe640 1 -- 172.21.15.172:0/2234773972 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858004030 con 0x55d713d136f0 2024-09-10T08:57:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f485cbfe640 1 -- 172.21.15.172:0/2234773972 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48580035f0 con 0x55d713d136f0 2024-09-10T08:57:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.679+0000 7f485cbfe640 1 -- 172.21.15.172:0/2234773972 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858011680 con 0x55d713d136f0 2024-09-10T08:57:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.680+0000 7f4873251740 1 -- 172.21.15.172:0/2234773972 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d136f0 msgr2=0x55d713d181b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.680+0000 7f4873251740 1 --2- 172.21.15.172:0/2234773972 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d136f0 0x55d713d181b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f485800bd10 tx=0x7f48580107a0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.680+0000 7f4873251740 1 -- 172.21.15.172:0/2234773972 shutdown_connections 2024-09-10T08:57:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.680+0000 7f4873251740 1 --2- 172.21.15.172:0/2234773972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d18780 0x55d713d1ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.680+0000 7f4873251740 1 --2- 172.21.15.172:0/2234773972 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d136f0 0x55d713d181b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.680+0000 7f4873251740 1 --2- 172.21.15.172:0/2234773972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 0x55d713d131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.680+0000 7f4873251740 1 -- 172.21.15.172:0/2234773972 >> 172.21.15.172:0/2234773972 conn(0x55d713d0a010 msgr2=0x55d713d0a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.680+0000 7f4873251740 1 -- 172.21.15.172:0/2234773972 shutdown_connections 2024-09-10T08:57:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.680+0000 7f4873251740 1 -- 172.21.15.172:0/2234773972 wait complete. 2024-09-10T08:57:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.680+0000 7f4873251740 1 -- start start 2024-09-10T08:57:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.680+0000 7f4873251740 1 Processor -- start 2024-09-10T08:57:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f4873251740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 0x55d713d1d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f4873251740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d136f0 0x55d713d1d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f4873251740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d18780 0x55d713d24590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f4873251740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d713d1b750 con 0x55d713d136f0 2024-09-10T08:57:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f4873251740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d713bfb290 con 0x55d713d12db0 2024-09-10T08:57:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f4873251740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d713d1b8d0 con 0x55d713d18780 2024-09-10T08:57:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f486d971640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d136f0 0x55d713d1d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f486e172640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 0x55d713d1d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f486e973640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d18780 0x55d713d24590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f486d971640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d136f0 0x55d713d1d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39380/0 (socket says 172.21.15.172:39380) 2024-09-10T08:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f486e172640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 0x55d713d1d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:34810/0 (socket says 172.21.15.172:34810) 2024-09-10T08:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f486d971640 1 -- 172.21.15.172:0/4242113508 learned_addr learned my addr 172.21.15.172:0/4242113508 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f4873251740 10 client.0 _ll_register_callbacks cb 0x55d713c2c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f486e973640 1 -- 172.21.15.172:0/4242113508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 msgr2=0x55d713d1d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f486e973640 1 --2- 172.21.15.172:0/4242113508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 0x55d713d1d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f486e973640 1 -- 172.21.15.172:0/4242113508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d136f0 msgr2=0x55d713d1d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f486e973640 1 --2- 172.21.15.172:0/4242113508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d136f0 0x55d713d1d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f486e973640 1 -- 172.21.15.172:0/4242113508 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d713985210 con 0x55d713d18780 2024-09-10T08:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.681+0000 7f486e172640 1 --2- 172.21.15.172:0/4242113508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 0x55d713d1d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.682+0000 7f486e973640 1 --2- 172.21.15.172:0/4242113508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d18780 0x55d713d24590 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4860009e10 tx=0x7f4860004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80a7c8f80c594f33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.682+0000 7f4851ffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:57:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.682+0000 7f4851ffb640 1 -- 172.21.15.172:0/4242113508 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860003b70 con 0x55d713d18780 2024-09-10T08:57:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.682+0000 7f4873251740 1 -- 172.21.15.172:0/4242113508 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48580035f0 con 0x55d713d18780 2024-09-10T08:57:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.682+0000 7f4851ffb640 1 -- 172.21.15.172:0/4242113508 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4860003d10 con 0x55d713d18780 2024-09-10T08:57:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.682+0000 7f4851ffb640 1 -- 172.21.15.172:0/4242113508 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4860004020 con 0x55d713d18780 2024-09-10T08:57:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.682+0000 7f4873251740 10 client.27598 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.682+0000 7f4873251740 1 -- 172.21.15.172:0/4242113508 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d713a51620 con 0x55d713d18780 2024-09-10T08:57:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.682+0000 7f4851ffb640 1 -- 172.21.15.172:0/4242113508 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48580035f0 con 0x55d713d18780 2024-09-10T08:57:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.682+0000 7f4851ffb640 1 -- 172.21.15.172:0/4242113508 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f486000f040 con 0x55d713d18780 2024-09-10T08:57:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.682+0000 7f4873251740 10 client.27598 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.682+0000 7f4873251740 10 client.27598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.682+0000 7f4873251740 20 client.27598 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.682+0000 7f4873251740 1 --2- 172.21.15.172:0/4242113508 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d713d28c40 0x55d713d2b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.682+0000 7f4873251740 4 client.27598 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:57:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.682+0000 7f4873251740 1 -- 172.21.15.172:0/4242113508 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d713c7fa20 con 0x55d713d28c40 2024-09-10T08:57:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.683+0000 7f486e172640 1 --2- 172.21.15.172:0/4242113508 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d713d28c40 0x55d713d2b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.683+0000 7f4851ffb640 10 client.27598 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.683+0000 7f486e172640 1 --2- 172.21.15.172:0/4242113508 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d713d28c40 0x55d713d2b090 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e3557f6476f53dd server_cookie=9f2a07a20410cb48 in_seq=0 out_seq=0 2024-09-10T08:57:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.684+0000 7f4851ffb640 1 -- 172.21.15.172:0/4242113508 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d713c7fa20 con 0x55d713d28c40 2024-09-10T08:57:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.684+0000 7f4851ffb640 10 client.27598 handle_command_reply: tid=0 2024-09-10T08:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.684+0000 7f4873251740 1 client.27598 shutdown 2024-09-10T08:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.684+0000 7f4873251740 1 -- 172.21.15.172:0/4242113508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d18780 msgr2=0x55d713d24590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.684+0000 7f4873251740 1 --2- 172.21.15.172:0/4242113508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d18780 0x55d713d24590 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4860009e10 tx=0x7f4860004ab0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.684+0000 7f4873251740 1 -- 172.21.15.172:0/4242113508 shutdown_connections 2024-09-10T08:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.684+0000 7f4873251740 1 --2- 172.21.15.172:0/4242113508 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d713d28c40 0x55d713d2b090 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.684+0000 7f4873251740 1 --2- 172.21.15.172:0/4242113508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d18780 0x55d713d24590 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.684+0000 7f4873251740 1 --2- 172.21.15.172:0/4242113508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d136f0 0x55d713d1d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.684+0000 7f4873251740 1 --2- 172.21.15.172:0/4242113508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 0x55d713d1d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.684+0000 7f4851ffb640 0 client.27598 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.684+0000 7f4873251740 1 -- 172.21.15.172:0/4242113508 >> 172.21.15.172:0/4242113508 conn(0x55d713d0a010 msgr2=0x55d713d0b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.684+0000 7f4873251740 1 -- 172.21.15.172:0/4242113508 shutdown_connections 2024-09-10T08:57:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.684+0000 7f4873251740 1 -- 172.21.15.172:0/4242113508 wait complete. 2024-09-10T08:57:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.684+0000 7f4873251740 20 client.27598 trim_cache size 0 max 16384 2024-09-10T08:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f4873251740 1 Processor -- start 2024-09-10T08:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f4873251740 1 -- start start 2024-09-10T08:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f4873251740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d12db0 0x55d713d090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f4873251740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d136f0 0x55d713d24550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f486e172640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d12db0 0x55d713d090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f486e172640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d12db0 0x55d713d090b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39388/0 (socket says 172.21.15.172:39388) 2024-09-10T08:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f4873251740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d18780 0x55d713d0b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f486e172640 1 -- 172.21.15.172:0/4247339075 learned_addr learned my addr 172.21.15.172:0/4247339075 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f4873251740 1 -- 172.21.15.172:0/4247339075 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f48681a8390 con 0x55d713d12db0 2024-09-10T08:57:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f486d971640 1 --2- 172.21.15.172:0/4247339075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d136f0 0x55d713d24550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f4873251740 1 -- 172.21.15.172:0/4247339075 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f48681a9f30 con 0x55d713d18780 2024-09-10T08:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f4873251740 1 -- 172.21.15.172:0/4247339075 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f48681a8a60 con 0x55d713d136f0 2024-09-10T08:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f486e973640 1 --2- 172.21.15.172:0/4247339075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d18780 0x55d713d0b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f486d971640 1 -- 172.21.15.172:0/4247339075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d18780 msgr2=0x55d713d0b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f486d971640 1 --2- 172.21.15.172:0/4247339075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d18780 0x55d713d0b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f486d971640 1 -- 172.21.15.172:0/4247339075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d12db0 msgr2=0x55d713d090b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f486d971640 1 --2- 172.21.15.172:0/4247339075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d12db0 0x55d713d090b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f486d971640 1 -- 172.21.15.172:0/4247339075 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d713d1bfd0 con 0x55d713d136f0 2024-09-10T08:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.702+0000 7f486e172640 1 --2- 172.21.15.172:0/4247339075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d12db0 0x55d713d090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.703+0000 7f486d971640 1 --2- 172.21.15.172:0/4247339075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d136f0 0x55d713d24550 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4858011010 tx=0x7f485800b570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc97badde87f10ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.703+0000 7f4853fff640 1 -- 172.21.15.172:0/4247339075 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48580037f0 con 0x55d713d136f0 2024-09-10T08:57:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.703+0000 7f4853fff640 1 -- 172.21.15.172:0/4247339075 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4858003990 con 0x55d713d136f0 2024-09-10T08:57:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.703+0000 7f4853fff640 1 -- 172.21.15.172:0/4247339075 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858011900 con 0x55d713d136f0 2024-09-10T08:57:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.704+0000 7f4873251740 1 -- 172.21.15.172:0/4247339075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d136f0 msgr2=0x55d713d24550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.704+0000 7f4873251740 1 --2- 172.21.15.172:0/4247339075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d136f0 0x55d713d24550 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4858011010 tx=0x7f485800b570 comp rx=0 tx=0).stop 2024-09-10T08:57:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.704+0000 7f4873251740 1 -- 172.21.15.172:0/4247339075 shutdown_connections 2024-09-10T08:57:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.704+0000 7f4873251740 1 --2- 172.21.15.172:0/4247339075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d18780 0x55d713d0b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.704+0000 7f4873251740 1 --2- 172.21.15.172:0/4247339075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d136f0 0x55d713d24550 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.704+0000 7f4873251740 1 --2- 172.21.15.172:0/4247339075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d12db0 0x55d713d090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.704+0000 7f4873251740 1 -- 172.21.15.172:0/4247339075 >> 172.21.15.172:0/4247339075 conn(0x55d713d0a010 msgr2=0x55d713d089a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.704+0000 7f4873251740 1 -- 172.21.15.172:0/4247339075 shutdown_connections 2024-09-10T08:57:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.704+0000 7f4873251740 1 -- 172.21.15.172:0/4247339075 wait complete. 2024-09-10T08:57:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.704+0000 7f4873251740 1 -- start start 2024-09-10T08:57:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.704+0000 7f4873251740 1 Processor -- start 2024-09-10T08:57:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f4873251740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 0x55d713d21970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f4873251740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d136f0 0x55d713d21eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f4873251740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d18780 0x55d713d22e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f4873251740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f48681a9860 con 0x55d713d136f0 2024-09-10T08:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f486d971640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d136f0 0x55d713d21eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f486e172640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 0x55d713d21970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f4873251740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f48681a9190 con 0x55d713d12db0 2024-09-10T08:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f486d971640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d136f0 0x55d713d21eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39394/0 (socket says 172.21.15.172:39394) 2024-09-10T08:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f4873251740 1 -- 172.21.15.172:0/316204610 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f48681a6370 con 0x55d713d18780 2024-09-10T08:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f486e172640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 0x55d713d21970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:34820/0 (socket says 172.21.15.172:34820) 2024-09-10T08:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f486d971640 1 -- 172.21.15.172:0/316204610 learned_addr learned my addr 172.21.15.172:0/316204610 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f486e973640 1 --2- 172.21.15.172:0/316204610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d18780 0x55d713d22e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f4873251740 10 client.0 _ll_register_callbacks cb 0x7f4860014ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f486d971640 1 -- 172.21.15.172:0/316204610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d18780 msgr2=0x55d713d22e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f486d971640 1 --2- 172.21.15.172:0/316204610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d18780 0x55d713d22e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f486d971640 1 -- 172.21.15.172:0/316204610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 msgr2=0x55d713d21970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f486d971640 1 --2- 172.21.15.172:0/316204610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 0x55d713d21970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f486d971640 1 -- 172.21.15.172:0/316204610 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d713d23480 con 0x55d713d136f0 2024-09-10T08:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f486e172640 1 --2- 172.21.15.172:0/316204610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 0x55d713d21970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f486d971640 1 --2- 172.21.15.172:0/316204610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d136f0 0x55d713d21eb0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f485800bc30 tx=0x7f4858005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314e506053c55345 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f4851ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f4851ffb640 1 -- 172.21.15.172:0/316204610 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858032060 con 0x55d713d136f0 2024-09-10T08:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f4851ffb640 1 -- 172.21.15.172:0/316204610 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4858011680 con 0x55d713d136f0 2024-09-10T08:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.705+0000 7f4873251740 1 -- 172.21.15.172:0/316204610 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4858003990 con 0x55d713d136f0 2024-09-10T08:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.706+0000 7f4851ffb640 1 -- 172.21.15.172:0/316204610 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4858003b40 con 0x55d713d136f0 2024-09-10T08:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.706+0000 7f4873251740 10 client.18771 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.706+0000 7f4873251740 1 -- 172.21.15.172:0/316204610 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d713d24750 con 0x55d713d136f0 2024-09-10T08:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.706+0000 7f4851ffb640 1 -- 172.21.15.172:0/316204610 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4858003990 con 0x55d713d136f0 2024-09-10T08:57:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.706+0000 7f4851ffb640 1 -- 172.21.15.172:0/316204610 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f4858024070 con 0x55d713d136f0 2024-09-10T08:57:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.706+0000 7f4873251740 10 client.18771 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.706+0000 7f4873251740 10 client.18771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.706+0000 7f4873251740 20 client.18771 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.706+0000 7f4873251740 1 --2- 172.21.15.172:0/316204610 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d713d93da0 0x55d713d96190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.706+0000 7f4873251740 4 client.18771 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:57:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.706+0000 7f4873251740 1 -- 172.21.15.172:0/316204610 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d713c79d70 con 0x55d713d93da0 2024-09-10T08:57:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.706+0000 7f486e172640 1 --2- 172.21.15.172:0/316204610 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d713d93da0 0x55d713d96190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.706+0000 7f486e172640 1 --2- 172.21.15.172:0/316204610 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d713d93da0 0x55d713d96190 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=624cfd9b50094ce1 server_cookie=8261b29f2ea7503e in_seq=0 out_seq=0 2024-09-10T08:57:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.706+0000 7f4851ffb640 10 client.18771 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.712+0000 7f4851ffb640 1 -- 172.21.15.172:0/316204610 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d713c79d70 con 0x55d713d93da0 2024-09-10T08:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.712+0000 7f4851ffb640 10 client.18771 handle_command_reply: tid=0 2024-09-10T08:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.712+0000 7f4873251740 1 client.18771 shutdown 2024-09-10T08:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f4873251740 1 -- 172.21.15.172:0/316204610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d136f0 msgr2=0x55d713d21eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f4873251740 1 --2- 172.21.15.172:0/316204610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d136f0 0x55d713d21eb0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f485800bc30 tx=0x7f4858005300 comp rx=0 tx=0).stop 2024-09-10T08:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f4873251740 1 -- 172.21.15.172:0/316204610 shutdown_connections 2024-09-10T08:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f4873251740 1 --2- 172.21.15.172:0/316204610 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d713d93da0 0x55d713d96190 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f4851ffb640 0 client.18771 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f4873251740 1 --2- 172.21.15.172:0/316204610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d713d18780 0x55d713d22e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f4873251740 1 --2- 172.21.15.172:0/316204610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d713d136f0 0x55d713d21eb0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f4873251740 1 --2- 172.21.15.172:0/316204610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d713d12db0 0x55d713d21970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f4873251740 1 -- 172.21.15.172:0/316204610 >> 172.21.15.172:0/316204610 conn(0x55d713d0a010 msgr2=0x55d713d091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f4873251740 1 -- 172.21.15.172:0/316204610 shutdown_connections 2024-09-10T08:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f4873251740 1 -- 172.21.15.172:0/316204610 wait complete. 2024-09-10T08:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f4873251740 20 client.18771 trim_cache size 0 max 16384 2024-09-10T08:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:57:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:57:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f48703d6640 1 -- 172.21.15.172:0/2271950971 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f48540a2c30 msgr2=0x7f48540a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f48703d6640 1 --2- 172.21.15.172:0/2271950971 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f48540a2c30 0x7f48540a50f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4860005fd0 tx=0x7f4860014bf0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f48703d6640 1 -- 172.21.15.172:0/2271950971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4868101b40 msgr2=0x7f4868197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f48703d6640 1 --2- 172.21.15.172:0/2271950971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4868101b40 0x7f4868197d50 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f48640314e0 tx=0x7f48640042e0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f48703d6640 1 -- 172.21.15.172:0/2271950971 shutdown_connections 2024-09-10T08:57:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f48703d6640 1 --2- 172.21.15.172:0/2271950971 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f48540a2c30 0x7f48540a50f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f48703d6640 1 --2- 172.21.15.172:0/2271950971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48681036f0 0x7f486819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f48703d6640 1 --2- 172.21.15.172:0/2271950971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4868102d30 0x7f4868198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f48703d6640 1 --2- 172.21.15.172:0/2271950971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4868101b40 0x7f4868197d50 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f48703d6640 1 -- 172.21.15.172:0/2271950971 >> 172.21.15.172:0/2271950971 conn(0x7f48680fd2e0 msgr2=0x7f48681061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f48703d6640 1 -- 172.21.15.172:0/2271950971 shutdown_connections 2024-09-10T08:57:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:16.713+0000 7f48703d6640 1 -- 172.21.15.172:0/2271950971 wait complete. 2024-09-10T08:57:17.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:16 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:57:17.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:16 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:17.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:16 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:57:17.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:16 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:17.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:16 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:57:17.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:16 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:17.429 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:57:17.717 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:57:18.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:17 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T08:57:18.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:17 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:18.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:17 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:57:18.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:17 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:57:18.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:17 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:57:18.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:17 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:18.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:17 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:18.084 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:17 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:57:18.084 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:17 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:18.084 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:17 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:18.084 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:17 smithi172 ceph-mon[30854]: pgmap v682: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-09-10T08:57:18.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:17 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T08:57:18.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:17 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:18.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:17 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:57:18.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:17 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:57:18.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:17 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:57:18.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:17 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:18.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:17 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:18.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:17 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:57:18.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:17 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:18.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:17 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:18.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:17 smithi174 ceph-mon[34068]: pgmap v682: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-09-10T08:57:18.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.254+0000 7f847a25c640 1 -- 172.21.15.172:0/1713406952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8474101df0 msgr2=0x7f84741021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:18.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.254+0000 7f847a25c640 1 --2- 172.21.15.172:0/1713406952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8474101df0 0x7f84741021d0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f8464009900 tx=0x7f846402a410 comp rx=0 tx=0).stop 2024-09-10T08:57:18.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.254+0000 7f847a25c640 1 -- 172.21.15.172:0/1713406952 shutdown_connections 2024-09-10T08:57:18.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.254+0000 7f847a25c640 1 --2- 172.21.15.172:0/1713406952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8474106f20 0x7f847410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.254+0000 7f847a25c640 1 --2- 172.21.15.172:0/1713406952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84741027a0 0x7f84741067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.281 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.254+0000 7f847a25c640 1 --2- 172.21.15.172:0/1713406952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8474101df0 0x7f84741021d0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.254+0000 7f847a25c640 1 -- 172.21.15.172:0/1713406952 >> 172.21.15.172:0/1713406952 conn(0x7f84740fd8b0 msgr2=0x7f84740ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:18.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.254+0000 7f847a25c640 1 -- 172.21.15.172:0/1713406952 shutdown_connections 2024-09-10T08:57:18.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.254+0000 7f847a25c640 1 -- 172.21.15.172:0/1713406952 wait complete. 2024-09-10T08:57:18.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.254+0000 7f847a25c640 1 Processor -- start 2024-09-10T08:57:18.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.254+0000 7f847a25c640 1 -- start start 2024-09-10T08:57:18.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.254+0000 7f847a25c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8474101df0 0x7f8474198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.254+0000 7f847a25c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84741027a0 0x7f8474198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.254+0000 7f847a25c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8474106f20 0x7f847419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.254+0000 7f847a25c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8474111e30 con 0x7f84741027a0 2024-09-10T08:57:18.282 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.254+0000 7f847a25c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8474111cb0 con 0x7f8474101df0 2024-09-10T08:57:18.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.254+0000 7f847a25c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8474111fb0 con 0x7f8474106f20 2024-09-10T08:57:18.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.255+0000 7f846bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84741027a0 0x7f8474198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:18.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.255+0000 7f846bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84741027a0 0x7f8474198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39418/0 (socket says 172.21.15.172:39418) 2024-09-10T08:57:18.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.255+0000 7f846bfff640 1 -- 172.21.15.172:0/3758856304 learned_addr learned my addr 172.21.15.172:0/3758856304 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:18.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.255+0000 7f84737fe640 1 --2- 172.21.15.172:0/3758856304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8474101df0 0x7f8474198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:18.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.255+0000 7f8473fff640 1 --2- 172.21.15.172:0/3758856304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8474106f20 0x7f847419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:18.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.255+0000 7f846bfff640 1 -- 172.21.15.172:0/3758856304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8474106f20 msgr2=0x7f847419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:18.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.255+0000 7f846bfff640 1 --2- 172.21.15.172:0/3758856304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8474106f20 0x7f847419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.255+0000 7f846bfff640 1 -- 172.21.15.172:0/3758856304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8474101df0 msgr2=0x7f8474198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:18.283 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.255+0000 7f846bfff640 1 --2- 172.21.15.172:0/3758856304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8474101df0 0x7f8474198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.255+0000 7f846bfff640 1 -- 172.21.15.172:0/3758856304 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847419d300 con 0x7f84741027a0 2024-09-10T08:57:18.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.255+0000 7f846bfff640 1 --2- 172.21.15.172:0/3758856304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84741027a0 0x7f8474198890 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f845c00d770 tx=0x7f845c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:18.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.255+0000 7f84717fa640 1 -- 172.21.15.172:0/3758856304 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c017070 con 0x7f84741027a0 2024-09-10T08:57:18.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.255+0000 7f84717fa640 1 -- 172.21.15.172:0/3758856304 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f845c0054d0 con 0x7f84741027a0 2024-09-10T08:57:18.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.255+0000 7f84717fa640 1 -- 172.21.15.172:0/3758856304 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c004cf0 con 0x7f84741027a0 2024-09-10T08:57:18.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.255+0000 7f847a25c640 1 -- 172.21.15.172:0/3758856304 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f847419d5f0 con 0x7f84741027a0 2024-09-10T08:57:18.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.255+0000 7f847a25c640 1 -- 172.21.15.172:0/3758856304 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84741a4ed0 con 0x7f84741027a0 2024-09-10T08:57:18.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.258+0000 7f84717fa640 1 -- 172.21.15.172:0/3758856304 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f845c004e90 con 0x7f84741027a0 2024-09-10T08:57:18.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.258+0000 7f847c635740 1 Processor -- start 2024-09-10T08:57:18.284 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.258+0000 7f847c635740 1 -- start start 2024-09-10T08:57:18.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.258+0000 7f847c635740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce774a0 0x56219ce77880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.258+0000 7f847c635740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce77dc0 0x56219ce82140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.258+0000 7f847c635740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce82680 0x56219ce84a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.258+0000 7f847c635740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56219cddd2e0 con 0x56219ce77dc0 2024-09-10T08:57:18.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.258+0000 7f847c635740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56219cd84ce0 con 0x56219ce774a0 2024-09-10T08:57:18.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.258+0000 7f847c635740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56219cdd92a0 con 0x56219ce82680 2024-09-10T08:57:18.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f84737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce77dc0 0x56219ce82140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:18.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f84737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce77dc0 0x56219ce82140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39428/0 (socket says 172.21.15.172:39428) 2024-09-10T08:57:18.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f84737fe640 1 -- 172.21.15.172:0/413260860 learned_addr learned my addr 172.21.15.172:0/413260860 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:18.285 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f84717fa640 1 --2- 172.21.15.172:0/3758856304 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84600a2b80 0x7f84600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f846bfff640 1 --2- 172.21.15.172:0/413260860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce82680 0x56219ce84a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:18.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f8473fff640 1 --2- 172.21.15.172:0/413260860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce774a0 0x56219ce77880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:18.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f84737fe640 1 -- 172.21.15.172:0/413260860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce82680 msgr2=0x56219ce84a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:18.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f84737fe640 1 --2- 172.21.15.172:0/413260860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce82680 0x56219ce84a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f84737fe640 1 -- 172.21.15.172:0/413260860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce774a0 msgr2=0x56219ce77880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:18.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f84737fe640 1 --2- 172.21.15.172:0/413260860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce774a0 0x56219ce77880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f84737fe640 1 -- 172.21.15.172:0/413260860 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56219cd69440 con 0x56219ce77dc0 2024-09-10T08:57:18.286 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f8473fff640 1 --2- 172.21.15.172:0/3758856304 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84600a2b80 0x7f84600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:18.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f84717fa640 1 -- 172.21.15.172:0/3758856304 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f845c0cee70 con 0x7f84741027a0 2024-09-10T08:57:18.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f846bfff640 1 --2- 172.21.15.172:0/413260860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce82680 0x56219ce84a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:57:18.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f84737fe640 1 --2- 172.21.15.172:0/413260860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce77dc0 0x56219ce82140 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f8464009900 tx=0x7f8464032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3052b35349ac4581 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:18.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f8469ffb640 1 -- 172.21.15.172:0/413260860 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464040070 con 0x56219ce77dc0 2024-09-10T08:57:18.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f8473fff640 1 --2- 172.21.15.172:0/3758856304 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84600a2b80 0x7f84600a5040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f846c005c40 tx=0x7f846c005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:18.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f8469ffb640 1 -- 172.21.15.172:0/413260860 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8464004030 con 0x56219ce77dc0 2024-09-10T08:57:18.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.259+0000 7f8469ffb640 1 -- 172.21.15.172:0/413260860 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464035040 con 0x56219ce77dc0 2024-09-10T08:57:18.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.260+0000 7f847c635740 1 -- 172.21.15.172:0/413260860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce77dc0 msgr2=0x56219ce82140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:18.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.260+0000 7f847c635740 1 --2- 172.21.15.172:0/413260860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce77dc0 0x56219ce82140 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f8464009900 tx=0x7f8464032bb0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.260+0000 7f847c635740 1 -- 172.21.15.172:0/413260860 shutdown_connections 2024-09-10T08:57:18.287 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.260+0000 7f847c635740 1 --2- 172.21.15.172:0/413260860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce82680 0x56219ce84a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.260+0000 7f847c635740 1 --2- 172.21.15.172:0/413260860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce77dc0 0x56219ce82140 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.260+0000 7f847c635740 1 --2- 172.21.15.172:0/413260860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce774a0 0x56219ce77880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.260+0000 7f847c635740 1 -- 172.21.15.172:0/413260860 >> 172.21.15.172:0/413260860 conn(0x56219ce73a30 msgr2=0x56219ce73e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:18.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.260+0000 7f847c635740 1 -- 172.21.15.172:0/413260860 shutdown_connections 2024-09-10T08:57:18.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.260+0000 7f847c635740 1 -- 172.21.15.172:0/413260860 wait complete. 2024-09-10T08:57:18.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.260+0000 7f847c635740 1 -- start start 2024-09-10T08:57:18.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f847c635740 1 Processor -- start 2024-09-10T08:57:18.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f847c635740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce774a0 0x56219ce81fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f847c635740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce77dc0 0x56219ce7f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.288 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f847c635740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce82680 0x56219ce7f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f847c635740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56219ce85250 con 0x56219ce77dc0 2024-09-10T08:57:18.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f847c635740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56219cd98050 con 0x56219ce82680 2024-09-10T08:57:18.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f847c635740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56219ce853d0 con 0x56219ce774a0 2024-09-10T08:57:18.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f847c635740 10 client.0 _ll_register_callbacks cb 0x56219cdd8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:18.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f846bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce77dc0 0x56219ce7f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:18.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f846bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce77dc0 0x56219ce7f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39434/0 (socket says 172.21.15.172:39434) 2024-09-10T08:57:18.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f846bfff640 1 -- 172.21.15.172:0/3634448603 learned_addr learned my addr 172.21.15.172:0/3634448603 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:18.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f8473fff640 1 --2- 172.21.15.172:0/3634448603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce82680 0x56219ce7f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:18.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f84737fe640 1 --2- 172.21.15.172:0/3634448603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce774a0 0x56219ce81fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:18.289 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f846bfff640 1 -- 172.21.15.172:0/3634448603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce774a0 msgr2=0x56219ce81fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:18.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f846bfff640 1 --2- 172.21.15.172:0/3634448603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce774a0 0x56219ce81fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f846bfff640 1 -- 172.21.15.172:0/3634448603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce82680 msgr2=0x56219ce7f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:18.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f846bfff640 1 --2- 172.21.15.172:0/3634448603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce82680 0x56219ce7f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.261+0000 7f846bfff640 1 -- 172.21.15.172:0/3634448603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56219cb4e3c0 con 0x56219ce77dc0 2024-09-10T08:57:18.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.262+0000 7f846bfff640 1 --2- 172.21.15.172:0/3634448603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce77dc0 0x56219ce7f250 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f845c009610 tx=0x7f845c06b1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed99b88cd7e3459d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:18.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.262+0000 7f8452ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:18.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.262+0000 7f8452ffd640 1 -- 172.21.15.172:0/3634448603 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c020040 con 0x56219ce77dc0 2024-09-10T08:57:18.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.262+0000 7f8452ffd640 1 -- 172.21.15.172:0/3634448603 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f845c09a760 con 0x56219ce77dc0 2024-09-10T08:57:18.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.262+0000 7f847c635740 1 -- 172.21.15.172:0/3634448603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8464004030 con 0x56219ce77dc0 2024-09-10T08:57:18.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.262+0000 7f8452ffd640 1 -- 172.21.15.172:0/3634448603 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f845c09ac70 con 0x56219ce77dc0 2024-09-10T08:57:18.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.262+0000 7f847c635740 10 client.18783 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:18.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.262+0000 7f847c635740 1 -- 172.21.15.172:0/3634448603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56219ce90b90 con 0x56219ce77dc0 2024-09-10T08:57:18.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.262+0000 7f8452ffd640 1 -- 172.21.15.172:0/3634448603 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8464004030 con 0x56219ce77dc0 2024-09-10T08:57:18.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.262+0000 7f8452ffd640 1 -- 172.21.15.172:0/3634448603 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f845c09f150 con 0x56219ce77dc0 2024-09-10T08:57:18.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.262+0000 7f847c635740 10 client.18783 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:18.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.262+0000 7f847c635740 10 client.18783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:18.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.262+0000 7f847c635740 20 client.18783 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:18.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.262+0000 7f847c635740 1 --2- 172.21.15.172:0/3634448603 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56219ce91f30 0x56219ce94360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.262+0000 7f847c635740 4 client.18783 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:57:18.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.262+0000 7f847c635740 1 -- 172.21.15.172:0/3634448603 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56219cdea850 con 0x56219ce91f30 2024-09-10T08:57:18.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.263+0000 7f84737fe640 1 --2- 172.21.15.172:0/3634448603 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56219ce91f30 0x56219ce94360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:18.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.263+0000 7f84737fe640 1 --2- 172.21.15.172:0/3634448603 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56219ce91f30 0x56219ce94360 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc81a696f5bccee9 server_cookie=ed83776f6e225a7f in_seq=0 out_seq=0 2024-09-10T08:57:18.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.263+0000 7f8452ffd640 10 client.18783 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:18.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.264+0000 7f8452ffd640 1 -- 172.21.15.172:0/3634448603 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56219cdea850 con 0x56219ce91f30 2024-09-10T08:57:18.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.264+0000 7f8452ffd640 10 client.18783 handle_command_reply: tid=0 2024-09-10T08:57:18.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.264+0000 7f847c635740 1 client.18783 shutdown 2024-09-10T08:57:18.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.264+0000 7f847c635740 1 -- 172.21.15.172:0/3634448603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce77dc0 msgr2=0x56219ce7f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:18.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.264+0000 7f847c635740 1 --2- 172.21.15.172:0/3634448603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce77dc0 0x56219ce7f250 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f845c009610 tx=0x7f845c06b1c0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.264+0000 7f847c635740 1 -- 172.21.15.172:0/3634448603 shutdown_connections 2024-09-10T08:57:18.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.264+0000 7f847c635740 1 --2- 172.21.15.172:0/3634448603 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56219ce91f30 0x56219ce94360 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.264+0000 7f847c635740 1 --2- 172.21.15.172:0/3634448603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce82680 0x56219ce7f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.264+0000 7f847c635740 1 --2- 172.21.15.172:0/3634448603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce77dc0 0x56219ce7f250 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.264+0000 7f847c635740 1 --2- 172.21.15.172:0/3634448603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce774a0 0x56219ce81fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.264+0000 7f847c635740 1 -- 172.21.15.172:0/3634448603 >> 172.21.15.172:0/3634448603 conn(0x56219ce73a30 msgr2=0x56219ce79760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:18.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.264+0000 7f8452ffd640 0 client.18783 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:18.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.264+0000 7f847c635740 1 -- 172.21.15.172:0/3634448603 shutdown_connections 2024-09-10T08:57:18.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.264+0000 7f847c635740 1 -- 172.21.15.172:0/3634448603 wait complete. 2024-09-10T08:57:18.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.264+0000 7f847c635740 20 client.18783 trim_cache size 0 max 16384 2024-09-10T08:57:18.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.281+0000 7f847c635740 1 Processor -- start 2024-09-10T08:57:18.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.281+0000 7f847c635740 1 -- start start 2024-09-10T08:57:18.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.281+0000 7f847c635740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce73a30 0x56219ce8e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.281+0000 7f847c635740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce774a0 0x56219ce99c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.281+0000 7f847c635740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce77dc0 0x56219ce9a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.281+0000 7f847c635740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84741ae9a0 con 0x56219ce73a30 2024-09-10T08:57:18.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.281+0000 7f847c635740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84741b0540 con 0x56219ce77dc0 2024-09-10T08:57:18.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.281+0000 7f847c635740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f84741af070 con 0x56219ce774a0 2024-09-10T08:57:18.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.281+0000 7f84737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce73a30 0x56219ce8e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:18.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.282+0000 7f84737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce73a30 0x56219ce8e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39450/0 (socket says 172.21.15.172:39450) 2024-09-10T08:57:18.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.282+0000 7f84737fe640 1 -- 172.21.15.172:0/1248586193 learned_addr learned my addr 172.21.15.172:0/1248586193 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:18.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.282+0000 7f846bfff640 1 --2- 172.21.15.172:0/1248586193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce774a0 0x56219ce99c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:18.300 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.282+0000 7f84737fe640 1 -- 172.21.15.172:0/1248586193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce774a0 msgr2=0x56219ce99c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:18.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.282+0000 7f84737fe640 1 --2- 172.21.15.172:0/1248586193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce774a0 0x56219ce99c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.282+0000 7f84737fe640 1 -- 172.21.15.172:0/1248586193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce77dc0 msgr2=0x56219ce9a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:18.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.282+0000 7f84737fe640 1 --2- 172.21.15.172:0/1248586193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce77dc0 0x56219ce9a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.282+0000 7f84737fe640 1 -- 172.21.15.172:0/1248586193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56219ce77bb0 con 0x56219ce73a30 2024-09-10T08:57:18.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.282+0000 7f846bfff640 1 --2- 172.21.15.172:0/1248586193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce774a0 0x56219ce99c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:57:18.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.282+0000 7f84737fe640 1 --2- 172.21.15.172:0/1248586193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce73a30 0x56219ce8e4b0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f8464032bf0 tx=0x7f8464005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119a51f20e8ad9a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:18.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.282+0000 7f84697fa640 1 -- 172.21.15.172:0/1248586193 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464040070 con 0x56219ce73a30 2024-09-10T08:57:18.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.282+0000 7f84697fa640 1 -- 172.21.15.172:0/1248586193 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84640053e0 con 0x56219ce73a30 2024-09-10T08:57:18.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.282+0000 7f84697fa640 1 -- 172.21.15.172:0/1248586193 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464035040 con 0x56219ce73a30 2024-09-10T08:57:18.301 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.283+0000 7f847c635740 1 -- 172.21.15.172:0/1248586193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce73a30 msgr2=0x56219ce8e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:18.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.283+0000 7f847c635740 1 --2- 172.21.15.172:0/1248586193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce73a30 0x56219ce8e4b0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f8464032bf0 tx=0x7f8464005db0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.283+0000 7f847c635740 1 -- 172.21.15.172:0/1248586193 shutdown_connections 2024-09-10T08:57:18.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.283+0000 7f847c635740 1 --2- 172.21.15.172:0/1248586193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce77dc0 0x56219ce9a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.283+0000 7f847c635740 1 --2- 172.21.15.172:0/1248586193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce774a0 0x56219ce99c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.283+0000 7f847c635740 1 --2- 172.21.15.172:0/1248586193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce73a30 0x56219ce8e4b0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.283+0000 7f847c635740 1 -- 172.21.15.172:0/1248586193 >> 172.21.15.172:0/1248586193 conn(0x56219ce7f8f0 msgr2=0x56219ce7db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:18.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.283+0000 7f847c635740 1 -- 172.21.15.172:0/1248586193 shutdown_connections 2024-09-10T08:57:18.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.283+0000 7f847c635740 1 -- 172.21.15.172:0/1248586193 wait complete. 2024-09-10T08:57:18.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.283+0000 7f847c635740 1 -- start start 2024-09-10T08:57:18.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.283+0000 7f847c635740 1 Processor -- start 2024-09-10T08:57:18.302 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f847c635740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce73a30 0x56219ce8a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f847c635740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce774a0 0x56219ce8acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f847c635740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce77dc0 0x56219ce86b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f847c635740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84741afe70 con 0x56219ce73a30 2024-09-10T08:57:18.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f847c635740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84741af7a0 con 0x56219ce774a0 2024-09-10T08:57:18.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f847c635740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f84741ac980 con 0x56219ce77dc0 2024-09-10T08:57:18.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f847c635740 10 client.0 _ll_register_callbacks cb 0x7f845c0b9f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:18.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f84737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce73a30 0x56219ce8a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:18.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f84737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce73a30 0x56219ce8a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39458/0 (socket says 172.21.15.172:39458) 2024-09-10T08:57:18.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f84737fe640 1 -- 172.21.15.172:0/2922290779 learned_addr learned my addr 172.21.15.172:0/2922290779 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:18.303 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f846bfff640 1 --2- 172.21.15.172:0/2922290779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce774a0 0x56219ce8acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:18.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f8473fff640 1 --2- 172.21.15.172:0/2922290779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce77dc0 0x56219ce86b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:18.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f84737fe640 1 -- 172.21.15.172:0/2922290779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce77dc0 msgr2=0x56219ce86b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:18.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f84737fe640 1 --2- 172.21.15.172:0/2922290779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce77dc0 0x56219ce86b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f84737fe640 1 -- 172.21.15.172:0/2922290779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce774a0 msgr2=0x56219ce8acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:18.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f84737fe640 1 --2- 172.21.15.172:0/2922290779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce774a0 0x56219ce8acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f84737fe640 1 -- 172.21.15.172:0/2922290779 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56219ce87380 con 0x56219ce73a30 2024-09-10T08:57:18.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f8473fff640 1 --2- 172.21.15.172:0/2922290779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce77dc0 0x56219ce86b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:57:18.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f84737fe640 1 --2- 172.21.15.172:0/2922290779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce73a30 0x56219ce8a780 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f84640052c0 tx=0x7f8464005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430cb38ee18f1e10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:18.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.284+0000 7f8452ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:18.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.285+0000 7f8452ffd640 1 -- 172.21.15.172:0/2922290779 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464035050 con 0x56219ce73a30 2024-09-10T08:57:18.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.285+0000 7f8452ffd640 1 -- 172.21.15.172:0/2922290779 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f84640314a0 con 0x56219ce73a30 2024-09-10T08:57:18.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.285+0000 7f847c635740 1 -- 172.21.15.172:0/2922290779 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84640053e0 con 0x56219ce73a30 2024-09-10T08:57:18.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.285+0000 7f8452ffd640 1 -- 172.21.15.172:0/2922290779 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8464040040 con 0x56219ce73a30 2024-09-10T08:57:18.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.285+0000 7f847c635740 10 client.18789 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:18.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.285+0000 7f847c635740 1 -- 172.21.15.172:0/2922290779 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56219cf01ad0 con 0x56219ce73a30 2024-09-10T08:57:18.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.285+0000 7f8452ffd640 1 -- 172.21.15.172:0/2922290779 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84640053e0 con 0x56219ce73a30 2024-09-10T08:57:18.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.285+0000 7f8452ffd640 1 -- 172.21.15.172:0/2922290779 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f846404e420 con 0x56219ce73a30 2024-09-10T08:57:18.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.285+0000 7f847c635740 10 client.18789 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:18.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.285+0000 7f847c635740 10 client.18789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:18.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.285+0000 7f847c635740 20 client.18789 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:18.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.285+0000 7f847c635740 1 --2- 172.21.15.172:0/2922290779 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56219ce82680 0x56219cf03d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:18.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.285+0000 7f847c635740 4 client.18789 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:57:18.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.285+0000 7f847c635740 1 -- 172.21.15.172:0/2922290779 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56219cde4d60 con 0x56219ce82680 2024-09-10T08:57:18.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.285+0000 7f846bfff640 1 --2- 172.21.15.172:0/2922290779 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56219ce82680 0x56219cf03d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:18.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.286+0000 7f8452ffd640 10 client.18789 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:18.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.286+0000 7f846bfff640 1 --2- 172.21.15.172:0/2922290779 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56219ce82680 0x56219cf03d60 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ba4d255354d441 server_cookie=e0cd4a6f087209d0 in_seq=0 out_seq=0 2024-09-10T08:57:18.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.286+0000 7f8452ffd640 1 -- 172.21.15.172:0/2922290779 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56219cde4d60 con 0x56219ce82680 2024-09-10T08:57:18.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.286+0000 7f8452ffd640 10 client.18789 handle_command_reply: tid=0 2024-09-10T08:57:18.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.286+0000 7f847c635740 1 client.18789 shutdown 2024-09-10T08:57:18.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.286+0000 7f847c635740 1 -- 172.21.15.172:0/2922290779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce73a30 msgr2=0x56219ce8a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:18.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.286+0000 7f847c635740 1 --2- 172.21.15.172:0/2922290779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce73a30 0x56219ce8a780 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f84640052c0 tx=0x7f8464005fd0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.286+0000 7f847c635740 1 -- 172.21.15.172:0/2922290779 shutdown_connections 2024-09-10T08:57:18.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.286+0000 7f847c635740 1 --2- 172.21.15.172:0/2922290779 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56219ce82680 0x56219cf03d60 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.286+0000 7f847c635740 1 --2- 172.21.15.172:0/2922290779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56219ce77dc0 0x56219ce86b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.286+0000 7f847c635740 1 --2- 172.21.15.172:0/2922290779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56219ce774a0 0x56219ce8acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.286+0000 7f847c635740 1 --2- 172.21.15.172:0/2922290779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56219ce73a30 0x56219ce8a780 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.286+0000 7f847c635740 1 -- 172.21.15.172:0/2922290779 >> 172.21.15.172:0/2922290779 conn(0x56219ce7f8f0 msgr2=0x56219ce72350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:18.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.286+0000 7f8452ffd640 0 client.18789 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:18.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.286+0000 7f847c635740 1 -- 172.21.15.172:0/2922290779 shutdown_connections 2024-09-10T08:57:18.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.286+0000 7f847c635740 1 -- 172.21.15.172:0/2922290779 wait complete. 2024-09-10T08:57:18.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.286+0000 7f847c635740 20 client.18789 trim_cache size 0 max 16384 2024-09-10T08:57:18.308 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:57:18.308 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:57:18.308 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T08:57:18.308 INFO:teuthology.orchestra.run.smithi172.stdout: "6094529b-e458-4124-983f-dbb8bcbd4a36": { 2024-09-10T08:57:18.308 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T08:57:18.308 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "6094529b-e458-4124-983f-dbb8bcbd4a36", 2024-09-10T08:57:18.308 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T08:57:18.308 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:57:18.308 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:57:18.308 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:57:18.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.287+0000 7f847a25c640 1 -- 172.21.15.172:0/3758856304 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84600a2b80 msgr2=0x7f84600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:18.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.287+0000 7f847a25c640 1 --2- 172.21.15.172:0/3758856304 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84600a2b80 0x7f84600a5040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f846c005c40 tx=0x7f846c005bd0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.287+0000 7f847a25c640 1 -- 172.21.15.172:0/3758856304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84741027a0 msgr2=0x7f8474198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:18.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.287+0000 7f847a25c640 1 --2- 172.21.15.172:0/3758856304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84741027a0 0x7f8474198890 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f845c00d770 tx=0x7f845c00dc30 comp rx=0 tx=0).stop 2024-09-10T08:57:18.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.287+0000 7f847a25c640 1 -- 172.21.15.172:0/3758856304 shutdown_connections 2024-09-10T08:57:18.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.287+0000 7f847a25c640 1 --2- 172.21.15.172:0/3758856304 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f84600a2b80 0x7f84600a5040 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.287+0000 7f847a25c640 1 --2- 172.21.15.172:0/3758856304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8474106f20 0x7f847419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.287+0000 7f847a25c640 1 --2- 172.21.15.172:0/3758856304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84741027a0 0x7f8474198890 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.287+0000 7f847a25c640 1 --2- 172.21.15.172:0/3758856304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8474101df0 0x7f8474198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:18.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.287+0000 7f847a25c640 1 -- 172.21.15.172:0/3758856304 >> 172.21.15.172:0/3758856304 conn(0x7f84740fd8b0 msgr2=0x7f84740fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:18.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.287+0000 7f847a25c640 1 -- 172.21.15.172:0/3758856304 shutdown_connections 2024-09-10T08:57:18.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:18.287+0000 7f847a25c640 1 -- 172.21.15.172:0/3758856304 wait complete. 2024-09-10T08:57:18.311 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:17 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T08:57:18.311 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:17 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:18.311 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:17 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:57:18.311 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:17 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:57:18.311 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:17 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:57:18.311 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:17 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:18.311 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:17 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:18.311 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:17 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:57:18.311 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:17 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:18.311 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:17 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:57:18.311 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:17 smithi191 ceph-mon[35314]: pgmap v682: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 MiB/s wr, 25 op/s 2024-09-10T08:57:19.031 INFO:tasks.cephfs.filesystem:scrub status for tag:6094529b-e458-4124-983f-dbb8bcbd4a36 - {'path': '/', 'tag': '6094529b-e458-4124-983f-dbb8bcbd4a36', 'options': 'recursive,force'} 2024-09-10T08:57:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:19 smithi174 ceph-mon[34068]: pgmap v683: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 MiB/s wr, 37 op/s 2024-09-10T08:57:19.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:19 smithi191 ceph-mon[35314]: pgmap v683: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 MiB/s wr, 37 op/s 2024-09-10T08:57:19.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:19 smithi172 ceph-mon[30854]: pgmap v683: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 MiB/s wr, 37 op/s 2024-09-10T08:57:21.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:21 smithi174 ceph-mon[34068]: pgmap v684: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 19 MiB/s wr, 45 op/s 2024-09-10T08:57:21.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:21 smithi191 ceph-mon[35314]: pgmap v684: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 19 MiB/s wr, 45 op/s 2024-09-10T08:57:21.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:21 smithi172 ceph-mon[30854]: pgmap v684: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 19 MiB/s wr, 45 op/s 2024-09-10T08:57:23.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:23 smithi174 ceph-mon[34068]: pgmap v685: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 41 op/s 2024-09-10T08:57:23.802 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:23 smithi191 ceph-mon[35314]: pgmap v685: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 41 op/s 2024-09-10T08:57:23.931 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:23 smithi172 ceph-mon[30854]: pgmap v685: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 41 op/s 2024-09-10T08:57:25.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:25 smithi174 ceph-mon[34068]: pgmap v686: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 43 op/s 2024-09-10T08:57:25.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:25 smithi191 ceph-mon[35314]: pgmap v686: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 43 op/s 2024-09-10T08:57:25.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:25 smithi172 ceph-mon[30854]: pgmap v686: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 12 MiB/s wr, 43 op/s 2024-09-10T08:57:27.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:27 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:57:27.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:27 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:57:27.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:27 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:57:28.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:28 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:57:28.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:28 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:57:28.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:28 smithi191 ceph-mon[35314]: pgmap v687: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.2 MiB/s wr, 37 op/s 2024-09-10T08:57:28.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:28 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:57:28.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:28 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:57:28.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:28 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:57:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:57:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:57:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:28 smithi172 ceph-mon[30854]: pgmap v687: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.2 MiB/s wr, 37 op/s 2024-09-10T08:57:28.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:57:28.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:57:28.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:57:28.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:28 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:57:28.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:28 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:57:28.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:28 smithi174 ceph-mon[34068]: pgmap v687: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.2 MiB/s wr, 37 op/s 2024-09-10T08:57:28.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:28 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:57:28.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:28 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:57:28.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:28 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:57:29.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:29 smithi174 ceph-mon[34068]: pgmap v688: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.5 MiB/s wr, 32 op/s 2024-09-10T08:57:29.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:29 smithi191 ceph-mon[35314]: pgmap v688: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.5 MiB/s wr, 32 op/s 2024-09-10T08:57:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.811+0000 7f00064e6640 1 -- 172.21.15.172:0/966410359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0000102610 msgr2=0x7f0000108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.811+0000 7f00064e6640 1 --2- 172.21.15.172:0/966410359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0000102610 0x7f0000108ea0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7efff8009ca0 tx=0x7efff802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.811+0000 7f00064e6640 1 -- 172.21.15.172:0/966410359 shutdown_connections 2024-09-10T08:57:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.811+0000 7f00064e6640 1 --2- 172.21.15.172:0/966410359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0000102610 0x7f0000108ea0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.811+0000 7f00064e6640 1 --2- 172.21.15.172:0/966410359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0000101c50 0x7f00001020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.811+0000 7f00064e6640 1 --2- 172.21.15.172:0/966410359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0000100a60 0x7f0000100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.811+0000 7f00064e6640 1 -- 172.21.15.172:0/966410359 >> 172.21.15.172:0/966410359 conn(0x7f00000fc1e0 msgr2=0x7f00000fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.811+0000 7f00064e6640 1 -- 172.21.15.172:0/966410359 shutdown_connections 2024-09-10T08:57:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.811+0000 7f00064e6640 1 -- 172.21.15.172:0/966410359 wait complete. 2024-09-10T08:57:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.812+0000 7f00064e6640 1 Processor -- start 2024-09-10T08:57:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.812+0000 7f00064e6640 1 -- start start 2024-09-10T08:57:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.813+0000 7f00064e6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0000100a60 0x7f0000197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.813+0000 7f00064e6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0000101c50 0x7f0000198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.813+0000 7f00064e6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0000102610 0x7f000019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.813+0000 7f00064e6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f000010d6e0 con 0x7f0000102610 2024-09-10T08:57:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.813+0000 7f00064e6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f000010d560 con 0x7f0000100a60 2024-09-10T08:57:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.813+0000 7f00064e6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f000010d860 con 0x7f0000101c50 2024-09-10T08:57:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.813+0000 7f0004a83640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0000102610 0x7f000019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.813+0000 7f0004a83640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0000102610 0x7f000019f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46908/0 (socket says 172.21.15.172:46908) 2024-09-10T08:57:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.813+0000 7f0004a83640 1 -- 172.21.15.172:0/753271830 learned_addr learned my addr 172.21.15.172:0/753271830 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.813+0000 7efffffff640 1 --2- 172.21.15.172:0/753271830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0000100a60 0x7f0000197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.813+0000 7effff7fe640 1 --2- 172.21.15.172:0/753271830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0000101c50 0x7f0000198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.813+0000 7f0004a83640 1 -- 172.21.15.172:0/753271830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0000101c50 msgr2=0x7f0000198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.813+0000 7f0004a83640 1 --2- 172.21.15.172:0/753271830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0000101c50 0x7f0000198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.813+0000 7f0004a83640 1 -- 172.21.15.172:0/753271830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0000100a60 msgr2=0x7f0000197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.813+0000 7f0004a83640 1 --2- 172.21.15.172:0/753271830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0000100a60 0x7f0000197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.813+0000 7f0004a83640 1 -- 172.21.15.172:0/753271830 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000019f900 con 0x7f0000102610 2024-09-10T08:57:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.814+0000 7f0004a83640 1 --2- 172.21.15.172:0/753271830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0000102610 0x7f000019f260 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7efff8009dc0 tx=0x7efff8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.814+0000 7efffd7fa640 1 -- 172.21.15.172:0/753271830 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8032ec0 con 0x7f0000102610 2024-09-10T08:57:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.814+0000 7efffd7fa640 1 -- 172.21.15.172:0/753271830 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efff80044f0 con 0x7f0000102610 2024-09-10T08:57:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.814+0000 7f00064e6640 1 -- 172.21.15.172:0/753271830 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000019fb90 con 0x7f0000102610 2024-09-10T08:57:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.814+0000 7efffd7fa640 1 -- 172.21.15.172:0/753271830 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8030d40 con 0x7f0000102610 2024-09-10T08:57:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.814+0000 7f00064e6640 1 -- 172.21.15.172:0/753271830 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00001a0130 con 0x7f0000102610 2024-09-10T08:57:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.816+0000 7efffd7fa640 1 -- 172.21.15.172:0/753271830 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efff8004040 con 0x7f0000102610 2024-09-10T08:57:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.816+0000 7f000927d740 1 Processor -- start 2024-09-10T08:57:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.816+0000 7f000927d740 1 -- start start 2024-09-10T08:57:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7f000927d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e9961db0 0x5562e99621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7f000927d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e99626f0 0x5562e99671b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7f000927d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9967780 0x5562e9969c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7f000927d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5562e98c12f0 con 0x5562e9967780 2024-09-10T08:57:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7f000927d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5562e9869270 con 0x5562e99626f0 2024-09-10T08:57:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7f000927d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5562e98bd2b0 con 0x5562e9961db0 2024-09-10T08:57:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7efffd7fa640 1 --2- 172.21.15.172:0/753271830 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7effe80a2e50 0x7effe80a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7effff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9967780 0x5562e9969c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7effff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9967780 0x5562e9969c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46914/0 (socket says 172.21.15.172:46914) 2024-09-10T08:57:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7effff7fe640 1 -- 172.21.15.172:0/2809017101 learned_addr learned my addr 172.21.15.172:0/2809017101 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7efffffff640 1 --2- 172.21.15.172:0/2809017101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e99626f0 0x5562e99671b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7f0004a83640 1 --2- 172.21.15.172:0/753271830 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7effe80a2e50 0x7effe80a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7effff7fe640 1 --2- 172.21.15.172:0/2809017101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e9961db0 0x5562e99621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7efffd7fa640 1 -- 172.21.15.172:0/753271830 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7efff80f0860 con 0x7f0000102610 2024-09-10T08:57:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7efffffff640 1 -- 172.21.15.172:0/2809017101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e9961db0 msgr2=0x5562e99621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7efffffff640 1 --2- 172.21.15.172:0/2809017101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e9961db0 0x5562e99621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7efffffff640 1 -- 172.21.15.172:0/2809017101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9967780 msgr2=0x5562e9969c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7efffffff640 1 --2- 172.21.15.172:0/2809017101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9967780 0x5562e9969c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7efffffff640 1 -- 172.21.15.172:0/2809017101 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e9868340 con 0x5562e99626f0 2024-09-10T08:57:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.817+0000 7effff7fe640 1 --2- 172.21.15.172:0/2809017101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9967780 0x5562e9969c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.818+0000 7efffffff640 1 --2- 172.21.15.172:0/2809017101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e99626f0 0x5562e99671b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efff0005fd0 tx=0x7efff00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=137adbf80013d352 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.818+0000 7f0004a83640 1 --2- 172.21.15.172:0/753271830 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7effe80a2e50 0x7effe80a5310 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efff80bce10 tx=0x7efff80880c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.818+0000 7effe67fc640 1 -- 172.21.15.172:0/2809017101 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff001c480 con 0x5562e99626f0 2024-09-10T08:57:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.818+0000 7effe67fc640 1 -- 172.21.15.172:0/2809017101 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efff001cad0 con 0x5562e99626f0 2024-09-10T08:57:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.818+0000 7effe67fc640 1 -- 172.21.15.172:0/2809017101 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff00119a0 con 0x5562e99626f0 2024-09-10T08:57:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.818+0000 7f000927d740 1 -- 172.21.15.172:0/2809017101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e99626f0 msgr2=0x5562e99671b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.818+0000 7f000927d740 1 --2- 172.21.15.172:0/2809017101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e99626f0 0x5562e99671b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efff0005fd0 tx=0x7efff00107a0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.818+0000 7f000927d740 1 -- 172.21.15.172:0/2809017101 shutdown_connections 2024-09-10T08:57:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.818+0000 7f000927d740 1 --2- 172.21.15.172:0/2809017101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9967780 0x5562e9969c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.818+0000 7f000927d740 1 --2- 172.21.15.172:0/2809017101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e99626f0 0x5562e99671b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.818+0000 7f000927d740 1 --2- 172.21.15.172:0/2809017101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e9961db0 0x5562e99621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.818+0000 7f000927d740 1 -- 172.21.15.172:0/2809017101 >> 172.21.15.172:0/2809017101 conn(0x5562e9959010 msgr2=0x5562e9959410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.819+0000 7f000927d740 1 -- 172.21.15.172:0/2809017101 shutdown_connections 2024-09-10T08:57:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.819+0000 7f000927d740 1 -- 172.21.15.172:0/2809017101 wait complete. 2024-09-10T08:57:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.819+0000 7f000927d740 1 -- start start 2024-09-10T08:57:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.819+0000 7f000927d740 1 Processor -- start 2024-09-10T08:57:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.819+0000 7f000927d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e9961db0 0x5562e996bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.819+0000 7f000927d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e99626f0 0x5562e996c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.819+0000 7f000927d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e9967780 0x5562e9973560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.819+0000 7f000927d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5562e996a750 con 0x5562e99626f0 2024-09-10T08:57:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.819+0000 7f000927d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5562e984a290 con 0x5562e9967780 2024-09-10T08:57:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.819+0000 7f000927d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5562e996a8d0 con 0x5562e9961db0 2024-09-10T08:57:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7f000927d740 10 client.0 _ll_register_callbacks cb 0x5562e987b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7efffffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e99626f0 0x5562e996c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7efffffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e99626f0 0x5562e996c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46918/0 (socket says 172.21.15.172:46918) 2024-09-10T08:57:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7efffffff640 1 -- 172.21.15.172:0/255188191 learned_addr learned my addr 172.21.15.172:0/255188191 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7effff7fe640 1 --2- 172.21.15.172:0/255188191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e9967780 0x5562e9973560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7effff7fe640 1 --2- 172.21.15.172:0/255188191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e9961db0 0x5562e996bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7efffffff640 1 -- 172.21.15.172:0/255188191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e9961db0 msgr2=0x5562e996bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7efffffff640 1 --2- 172.21.15.172:0/255188191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e9961db0 0x5562e996bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7efffffff640 1 -- 172.21.15.172:0/255188191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e9967780 msgr2=0x5562e9973560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7efffffff640 1 --2- 172.21.15.172:0/255188191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e9967780 0x5562e9973560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7efffffff640 1 -- 172.21.15.172:0/255188191 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e95d4210 con 0x5562e99626f0 2024-09-10T08:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7effff7fe640 1 --2- 172.21.15.172:0/255188191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e9967780 0x5562e9973560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7efffffff640 1 --2- 172.21.15.172:0/255188191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e99626f0 0x5562e996c520 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7efff000bef0 tx=0x7efff000bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b164624b25538dd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7effdf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7effdf7fe640 1 -- 172.21.15.172:0/255188191 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff002b060 con 0x5562e99626f0 2024-09-10T08:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7effdf7fe640 1 -- 172.21.15.172:0/255188191 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7efff001c790 con 0x5562e99626f0 2024-09-10T08:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7f000927d740 1 -- 172.21.15.172:0/255188191 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff001cad0 con 0x5562e99626f0 2024-09-10T08:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7effdf7fe640 1 -- 172.21.15.172:0/255188191 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efff0032050 con 0x5562e99626f0 2024-09-10T08:57:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7f000927d740 10 client.18801 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.820+0000 7f000927d740 1 -- 172.21.15.172:0/255188191 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5562e96a0620 con 0x5562e99626f0 2024-09-10T08:57:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.821+0000 7effdf7fe640 1 -- 172.21.15.172:0/255188191 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efff001cad0 con 0x5562e99626f0 2024-09-10T08:57:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.821+0000 7effdf7fe640 1 -- 172.21.15.172:0/255188191 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7efff0011ba0 con 0x5562e99626f0 2024-09-10T08:57:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.821+0000 7f000927d740 10 client.18801 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.821+0000 7f000927d740 10 client.18801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.821+0000 7f000927d740 20 client.18801 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.821+0000 7f000927d740 1 --2- 172.21.15.172:0/255188191 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5562e9977970 0x5562e9979d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.821+0000 7f000927d740 4 client.18801 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:57:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.821+0000 7f000927d740 1 -- 172.21.15.172:0/255188191 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562e98cea20 con 0x5562e9977970 2024-09-10T08:57:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.821+0000 7effff7fe640 1 --2- 172.21.15.172:0/255188191 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5562e9977970 0x5562e9979d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.822+0000 7effdf7fe640 10 client.18801 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.822+0000 7effff7fe640 1 --2- 172.21.15.172:0/255188191 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5562e9977970 0x5562e9979d60 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=418242471792c751 server_cookie=f2e6faa2690965d in_seq=0 out_seq=0 2024-09-10T08:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.822+0000 7effdf7fe640 1 -- 172.21.15.172:0/255188191 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5562e98cea20 con 0x5562e9977970 2024-09-10T08:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.822+0000 7effdf7fe640 10 client.18801 handle_command_reply: tid=0 2024-09-10T08:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.822+0000 7f000927d740 1 client.18801 shutdown 2024-09-10T08:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.822+0000 7f000927d740 1 -- 172.21.15.172:0/255188191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e99626f0 msgr2=0x5562e996c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.822+0000 7f000927d740 1 --2- 172.21.15.172:0/255188191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e99626f0 0x5562e996c520 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7efff000bef0 tx=0x7efff000bf20 comp rx=0 tx=0).stop 2024-09-10T08:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.823+0000 7f000927d740 1 -- 172.21.15.172:0/255188191 shutdown_connections 2024-09-10T08:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.823+0000 7f000927d740 1 --2- 172.21.15.172:0/255188191 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5562e9977970 0x5562e9979d60 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.823+0000 7f000927d740 1 --2- 172.21.15.172:0/255188191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e9967780 0x5562e9973560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.823+0000 7f000927d740 1 --2- 172.21.15.172:0/255188191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e99626f0 0x5562e996c520 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.823+0000 7effdf7fe640 0 client.18801 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.823+0000 7f000927d740 1 --2- 172.21.15.172:0/255188191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e9961db0 0x5562e996bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.823+0000 7f000927d740 1 -- 172.21.15.172:0/255188191 >> 172.21.15.172:0/255188191 conn(0x5562e9959010 msgr2=0x5562e995a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.823+0000 7f000927d740 1 -- 172.21.15.172:0/255188191 shutdown_connections 2024-09-10T08:57:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.823+0000 7f000927d740 1 -- 172.21.15.172:0/255188191 wait complete. 2024-09-10T08:57:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.823+0000 7f000927d740 20 client.18801 trim_cache size 0 max 16384 2024-09-10T08:57:29.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:29 smithi172 ceph-mon[30854]: pgmap v688: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.5 MiB/s wr, 32 op/s 2024-09-10T08:57:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.837+0000 7f000927d740 1 Processor -- start 2024-09-10T08:57:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.837+0000 7f000927d740 1 -- start start 2024-09-10T08:57:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.837+0000 7f000927d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e9961db0 0x5562e9963830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.837+0000 7f000927d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e99626f0 0x5562e9963d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.837+0000 7f000927d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9967780 0x5562e9973520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.837+0000 7f000927d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f00001a8260 con 0x5562e9967780 2024-09-10T08:57:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.838+0000 7f000927d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f00001a9e00 con 0x5562e99626f0 2024-09-10T08:57:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.838+0000 7f000927d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f00001a8930 con 0x5562e9961db0 2024-09-10T08:57:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.838+0000 7effff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9967780 0x5562e9973520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.838+0000 7effff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9967780 0x5562e9973520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46934/0 (socket says 172.21.15.172:46934) 2024-09-10T08:57:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.838+0000 7effff7fe640 1 -- 172.21.15.172:0/1079961091 learned_addr learned my addr 172.21.15.172:0/1079961091 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.838+0000 7effff7fe640 1 --2- 172.21.15.172:0/1079961091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e9961db0 0x5562e9963830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.838+0000 7efffffff640 1 --2- 172.21.15.172:0/1079961091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e99626f0 0x5562e9963d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.838+0000 7effff7fe640 1 -- 172.21.15.172:0/1079961091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e9961db0 msgr2=0x5562e9963830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.838+0000 7effff7fe640 1 --2- 172.21.15.172:0/1079961091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e9961db0 0x5562e9963830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.838+0000 7effff7fe640 1 -- 172.21.15.172:0/1079961091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e99626f0 msgr2=0x5562e9963d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.838+0000 7effff7fe640 1 --2- 172.21.15.172:0/1079961091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e99626f0 0x5562e9963d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.838+0000 7effff7fe640 1 -- 172.21.15.172:0/1079961091 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e9969fa0 con 0x5562e9967780 2024-09-10T08:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.838+0000 7effff7fe640 1 --2- 172.21.15.172:0/1079961091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9967780 0x5562e9973520 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7efff4004280 tx=0x7efff400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7243f397cd4cef64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.838+0000 7effe5ffb640 1 -- 172.21.15.172:0/1079961091 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4005220 con 0x5562e9967780 2024-09-10T08:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.838+0000 7effe5ffb640 1 -- 172.21.15.172:0/1079961091 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efff4004c50 con 0x5562e9967780 2024-09-10T08:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.838+0000 7effe5ffb640 1 -- 172.21.15.172:0/1079961091 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff40038e0 con 0x5562e9967780 2024-09-10T08:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.839+0000 7f000927d740 1 -- 172.21.15.172:0/1079961091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9967780 msgr2=0x5562e9973520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.839+0000 7f000927d740 1 --2- 172.21.15.172:0/1079961091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9967780 0x5562e9973520 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7efff4004280 tx=0x7efff400b390 comp rx=0 tx=0).stop 2024-09-10T08:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.839+0000 7f000927d740 1 -- 172.21.15.172:0/1079961091 shutdown_connections 2024-09-10T08:57:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.839+0000 7f000927d740 1 --2- 172.21.15.172:0/1079961091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9967780 0x5562e9973520 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.839+0000 7f000927d740 1 --2- 172.21.15.172:0/1079961091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e99626f0 0x5562e9963d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.839+0000 7f000927d740 1 --2- 172.21.15.172:0/1079961091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e9961db0 0x5562e9963830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.839+0000 7f000927d740 1 -- 172.21.15.172:0/1079961091 >> 172.21.15.172:0/1079961091 conn(0x5562e9959010 msgr2=0x5562e996b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.839+0000 7f000927d740 1 -- 172.21.15.172:0/1079961091 shutdown_connections 2024-09-10T08:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.839+0000 7f000927d740 1 -- 172.21.15.172:0/1079961091 wait complete. 2024-09-10T08:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.839+0000 7f000927d740 1 -- start start 2024-09-10T08:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.839+0000 7f000927d740 1 Processor -- start 2024-09-10T08:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7f000927d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9961db0 0x5562e9970730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7f000927d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e99626f0 0x5562e9970c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7f000927d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e9971c80 0x5562e99720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7f000927d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f00001a9730 con 0x5562e9961db0 2024-09-10T08:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7f000927d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f00001a9060 con 0x5562e9971c80 2024-09-10T08:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7f000927d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f00001a6240 con 0x5562e99626f0 2024-09-10T08:57:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7f000927d740 10 client.0 _ll_register_callbacks cb 0x7efff400bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7effff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9961db0 0x5562e9970730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7effff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9961db0 0x5562e9970730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46948/0 (socket says 172.21.15.172:46948) 2024-09-10T08:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7effff7fe640 1 -- 172.21.15.172:0/3365324591 learned_addr learned my addr 172.21.15.172:0/3365324591 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7efffffff640 1 --2- 172.21.15.172:0/3365324591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e99626f0 0x5562e9970c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7effff7fe640 1 --2- 172.21.15.172:0/3365324591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e9971c80 0x5562e99720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7effff7fe640 1 -- 172.21.15.172:0/3365324591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e99626f0 msgr2=0x5562e9970c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7effff7fe640 1 --2- 172.21.15.172:0/3365324591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e99626f0 0x5562e9970c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7effff7fe640 1 -- 172.21.15.172:0/3365324591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e9971c80 msgr2=0x5562e99720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7effff7fe640 1 --2- 172.21.15.172:0/3365324591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e9971c80 0x5562e99720e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7effff7fe640 1 -- 172.21.15.172:0/3365324591 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e9972670 con 0x5562e9961db0 2024-09-10T08:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7efffffff640 1 --2- 172.21.15.172:0/3365324591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e99626f0 0x5562e9970c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7effff7fe640 1 --2- 172.21.15.172:0/3365324591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9961db0 0x5562e9970730 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7efff40057c0 tx=0x7efff4012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172bdd7aa5044ffa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.840+0000 7effdf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.841+0000 7effdf7fe640 1 -- 172.21.15.172:0/3365324591 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4004940 con 0x5562e9961db0 2024-09-10T08:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.841+0000 7effdf7fe640 1 -- 172.21.15.172:0/3365324591 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7efff4004f70 con 0x5562e9961db0 2024-09-10T08:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.841+0000 7f000927d740 1 -- 172.21.15.172:0/3365324591 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff4004c50 con 0x5562e9961db0 2024-09-10T08:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.841+0000 7effdf7fe640 1 -- 172.21.15.172:0/3365324591 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efff4047050 con 0x5562e9961db0 2024-09-10T08:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.841+0000 7f000927d740 10 client.18807 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.841+0000 7f000927d740 1 -- 172.21.15.172:0/3365324591 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5562e99e25d0 con 0x5562e9961db0 2024-09-10T08:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.841+0000 7effdf7fe640 1 -- 172.21.15.172:0/3365324591 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efff4004c50 con 0x5562e9961db0 2024-09-10T08:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.841+0000 7effdf7fe640 1 -- 172.21.15.172:0/3365324591 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7efff402e020 con 0x5562e9961db0 2024-09-10T08:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.841+0000 7f000927d740 10 client.18807 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.841+0000 7f000927d740 10 client.18807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.841+0000 7f000927d740 20 client.18807 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.841+0000 7f000927d740 1 --2- 172.21.15.172:0/3365324591 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5562e99e2d50 0x5562e99e5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.841+0000 7f000927d740 4 client.18807 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:57:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.841+0000 7f000927d740 1 -- 172.21.15.172:0/3365324591 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562e98c8d70 con 0x5562e99e2d50 2024-09-10T08:57:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.841+0000 7efffffff640 1 --2- 172.21.15.172:0/3365324591 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5562e99e2d50 0x5562e99e5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.842+0000 7effdf7fe640 10 client.18807 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.842+0000 7efffffff640 1 --2- 172.21.15.172:0/3365324591 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5562e99e2d50 0x5562e99e5170 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ad5892126e635c6 server_cookie=5fbd52270dc55eb9 in_seq=0 out_seq=0 2024-09-10T08:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7effdf7fe640 1 -- 172.21.15.172:0/3365324591 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5562e98c8d70 con 0x5562e99e2d50 2024-09-10T08:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7effdf7fe640 10 client.18807 handle_command_reply: tid=0 2024-09-10T08:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7f000927d740 1 client.18807 shutdown 2024-09-10T08:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7f000927d740 1 -- 172.21.15.172:0/3365324591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9961db0 msgr2=0x5562e9970730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7f000927d740 1 --2- 172.21.15.172:0/3365324591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9961db0 0x5562e9970730 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7efff40057c0 tx=0x7efff4012f90 comp rx=0 tx=0).stop 2024-09-10T08:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7f000927d740 1 -- 172.21.15.172:0/3365324591 shutdown_connections 2024-09-10T08:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7f000927d740 1 --2- 172.21.15.172:0/3365324591 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5562e99e2d50 0x5562e99e5170 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7f000927d740 1 --2- 172.21.15.172:0/3365324591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5562e9971c80 0x5562e99720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7f000927d740 1 --2- 172.21.15.172:0/3365324591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5562e99626f0 0x5562e9970c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7effdf7fe640 0 client.18807 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7f000927d740 1 --2- 172.21.15.172:0/3365324591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562e9961db0 0x5562e9970730 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7f000927d740 1 -- 172.21.15.172:0/3365324591 >> 172.21.15.172:0/3365324591 conn(0x5562e9959010 msgr2=0x5562e996bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7f000927d740 1 -- 172.21.15.172:0/3365324591 shutdown_connections 2024-09-10T08:57:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7f000927d740 1 -- 172.21.15.172:0/3365324591 wait complete. 2024-09-10T08:57:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7f000927d740 20 client.18807 trim_cache size 0 max 16384 2024-09-10T08:57:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:57:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:57:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:57:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:57:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7f00064e6640 1 -- 172.21.15.172:0/753271830 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7effe80a2e50 msgr2=0x7effe80a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7f00064e6640 1 --2- 172.21.15.172:0/753271830 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7effe80a2e50 0x7effe80a5310 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efff80bce10 tx=0x7efff80880c0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7f00064e6640 1 -- 172.21.15.172:0/753271830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0000102610 msgr2=0x7f000019f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.849+0000 7f00064e6640 1 --2- 172.21.15.172:0/753271830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0000102610 0x7f000019f260 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7efff8009dc0 tx=0x7efff8003ba0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.850+0000 7f00064e6640 1 -- 172.21.15.172:0/753271830 shutdown_connections 2024-09-10T08:57:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.850+0000 7f00064e6640 1 --2- 172.21.15.172:0/753271830 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7effe80a2e50 0x7effe80a5310 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.850+0000 7f00064e6640 1 --2- 172.21.15.172:0/753271830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0000102610 0x7f000019f260 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.850+0000 7f00064e6640 1 --2- 172.21.15.172:0/753271830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0000101c50 0x7f0000198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.850+0000 7f00064e6640 1 --2- 172.21.15.172:0/753271830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0000100a60 0x7f0000197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.850+0000 7f00064e6640 1 -- 172.21.15.172:0/753271830 >> 172.21.15.172:0/753271830 conn(0x7f00000fc1e0 msgr2=0x7f0000105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.850+0000 7f00064e6640 1 -- 172.21.15.172:0/753271830 shutdown_connections 2024-09-10T08:57:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:29.850+0000 7f00064e6640 1 -- 172.21.15.172:0/753271830 wait complete. 2024-09-10T08:57:31.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:31 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:57:31.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:31 smithi174 ceph-mon[34068]: pgmap v689: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 542 KiB/s wr, 21 op/s 2024-09-10T08:57:31.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:31 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:57:31.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:31 smithi191 ceph-mon[35314]: pgmap v689: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 542 KiB/s wr, 21 op/s 2024-09-10T08:57:31.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:31 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:57:31.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:31 smithi172 ceph-mon[30854]: pgmap v689: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 542 KiB/s wr, 21 op/s 2024-09-10T08:57:32.744 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:32 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:57:32.744 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:32 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:57:32.744 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:32 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:57:32.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:32 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:57:32.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:32 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:57:32.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:32 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:57:32.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:32 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:57:32.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:32 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:57:32.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:32 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:57:33.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:33 smithi174 ceph-mon[34068]: pgmap v690: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 541 KiB/s wr, 10 op/s 2024-09-10T08:57:33.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:33 smithi191 ceph-mon[35314]: pgmap v690: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 541 KiB/s wr, 10 op/s 2024-09-10T08:57:33.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:33 smithi172 ceph-mon[30854]: pgmap v690: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 541 KiB/s wr, 10 op/s 2024-09-10T08:57:34.376 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:57:34.637 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:57:35.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.135+0000 7fe61e580640 1 -- 172.21.15.172:0/3488142772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe618102de0 msgr2=0x7fe6181031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:35.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.135+0000 7fe61e580640 1 --2- 172.21.15.172:0/3488142772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe618102de0 0x7fe6181031c0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fe608009900 tx=0x7fe60802a480 comp rx=0 tx=0).stop 2024-09-10T08:57:35.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.135+0000 7fe61e580640 1 -- 172.21.15.172:0/3488142772 shutdown_connections 2024-09-10T08:57:35.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.135+0000 7fe61e580640 1 --2- 172.21.15.172:0/3488142772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe61810c180 0x7fe61810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:35.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.135+0000 7fe61e580640 1 --2- 172.21.15.172:0/3488142772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe618103700 0x7fe61810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:35.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.135+0000 7fe61e580640 1 --2- 172.21.15.172:0/3488142772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe618102de0 0x7fe6181031c0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:35.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.135+0000 7fe61e580640 1 -- 172.21.15.172:0/3488142772 >> 172.21.15.172:0/3488142772 conn(0x7fe6180fc7a0 msgr2=0x7fe6180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:35.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.135+0000 7fe61e580640 1 -- 172.21.15.172:0/3488142772 shutdown_connections 2024-09-10T08:57:35.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.135+0000 7fe61e580640 1 -- 172.21.15.172:0/3488142772 wait complete. 2024-09-10T08:57:35.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.136+0000 7fe61e580640 1 Processor -- start 2024-09-10T08:57:35.137 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.136+0000 7fe61e580640 1 -- start start 2024-09-10T08:57:35.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.136+0000 7fe61e580640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe618102de0 0x7fe618100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:35.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.136+0000 7fe61e580640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe618103700 0x7fe618100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:35.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.136+0000 7fe61e580640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe61810c180 0x7fe6181014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:35.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.136+0000 7fe61e580640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe618111e80 con 0x7fe61810c180 2024-09-10T08:57:35.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.136+0000 7fe61e580640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe618111d00 con 0x7fe618102de0 2024-09-10T08:57:35.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.136+0000 7fe61e580640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe618112000 con 0x7fe618103700 2024-09-10T08:57:35.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.137+0000 7fe61cb1d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe61810c180 0x7fe6181014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:35.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.137+0000 7fe61cb1d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe61810c180 0x7fe6181014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46968/0 (socket says 172.21.15.172:46968) 2024-09-10T08:57:35.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.137+0000 7fe61cb1d640 1 -- 172.21.15.172:0/1643375810 learned_addr learned my addr 172.21.15.172:0/1643375810 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:35.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.137+0000 7fe6177fe640 1 --2- 172.21.15.172:0/1643375810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe618103700 0x7fe618100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:35.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.137+0000 7fe617fff640 1 --2- 172.21.15.172:0/1643375810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe618102de0 0x7fe618100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:35.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.137+0000 7fe61cb1d640 1 -- 172.21.15.172:0/1643375810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe618103700 msgr2=0x7fe618100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:35.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.137+0000 7fe61cb1d640 1 --2- 172.21.15.172:0/1643375810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe618103700 0x7fe618100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:35.140 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.137+0000 7fe61cb1d640 1 -- 172.21.15.172:0/1643375810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe618102de0 msgr2=0x7fe618100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:35.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.137+0000 7fe61cb1d640 1 --2- 172.21.15.172:0/1643375810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe618102de0 0x7fe618100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:35.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.137+0000 7fe61cb1d640 1 -- 172.21.15.172:0/1643375810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6181a6890 con 0x7fe61810c180 2024-09-10T08:57:35.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.137+0000 7fe61cb1d640 1 --2- 172.21.15.172:0/1643375810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe61810c180 0x7fe6181014c0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fe61000eaa0 tx=0x7fe61000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:35.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.137+0000 7fe6157fa640 1 -- 172.21.15.172:0/1643375810 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610004d20 con 0x7fe61810c180 2024-09-10T08:57:35.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.137+0000 7fe6157fa640 1 -- 172.21.15.172:0/1643375810 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe610005020 con 0x7fe61810c180 2024-09-10T08:57:35.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.137+0000 7fe61e580640 1 -- 172.21.15.172:0/1643375810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6181a6b80 con 0x7fe61810c180 2024-09-10T08:57:35.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.137+0000 7fe6157fa640 1 -- 172.21.15.172:0/1643375810 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6100116b0 con 0x7fe61810c180 2024-09-10T08:57:35.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.137+0000 7fe61e580640 1 -- 172.21.15.172:0/1643375810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6181a70c0 con 0x7fe61810c180 2024-09-10T08:57:35.141 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.138+0000 7fe61e580640 1 -- 172.21.15.172:0/1643375810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6181031c0 con 0x7fe61810c180 2024-09-10T08:57:35.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.140+0000 7fe6157fa640 1 -- 172.21.15.172:0/1643375810 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe6100051d0 con 0x7fe61810c180 2024-09-10T08:57:35.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.140+0000 7fe6157fa640 1 --2- 172.21.15.172:0/1643375810 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe6000a2b80 0x7fe6000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:35.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.140+0000 7fe617fff640 1 --2- 172.21.15.172:0/1643375810 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe6000a2b80 0x7fe6000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:35.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.140+0000 7fe6157fa640 1 -- 172.21.15.172:0/1643375810 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe6100cf520 con 0x7fe61810c180 2024-09-10T08:57:35.142 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.141+0000 7fe617fff640 1 --2- 172.21.15.172:0/1643375810 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe6000a2b80 0x7fe6000a5040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe608031440 tx=0x7fe608005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:35.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.144+0000 7fe6157fa640 1 -- 172.21.15.172:0/1643375810 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe610016030 con 0x7fe61810c180 2024-09-10T08:57:35.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.345+0000 7fe61e580640 1 -- 172.21.15.172:0/1643375810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe618107240 con 0x7fe61810c180 2024-09-10T08:57:35.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.346+0000 7fe6157fa640 1 -- 172.21.15.172:0/1643375810 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+12004 (secure 0 0 0) 0x7fe61009c120 con 0x7fe61810c180 2024-09-10T08:57:35.347 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:57:35.347 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":39,"btime":"2024-09-10T08:56:43:360461+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:56:42.665159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:57:35.347 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 39 2024-09-10T08:57:35.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.348+0000 7fe61e580640 1 -- 172.21.15.172:0/1643375810 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe6000a2b80 msgr2=0x7fe6000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:35.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.348+0000 7fe61e580640 1 --2- 172.21.15.172:0/1643375810 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe6000a2b80 0x7fe6000a5040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe608031440 tx=0x7fe608005b50 comp rx=0 tx=0).stop 2024-09-10T08:57:35.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.348+0000 7fe61e580640 1 -- 172.21.15.172:0/1643375810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe61810c180 msgr2=0x7fe6181014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:35.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.348+0000 7fe61e580640 1 --2- 172.21.15.172:0/1643375810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe61810c180 0x7fe6181014c0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fe61000eaa0 tx=0x7fe61000eea0 comp rx=0 tx=0).stop 2024-09-10T08:57:35.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.348+0000 7fe61e580640 1 -- 172.21.15.172:0/1643375810 shutdown_connections 2024-09-10T08:57:35.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.348+0000 7fe61e580640 1 --2- 172.21.15.172:0/1643375810 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe6000a2b80 0x7fe6000a5040 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:35.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.348+0000 7fe61e580640 1 --2- 172.21.15.172:0/1643375810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe61810c180 0x7fe6181014c0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:35.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.348+0000 7fe61e580640 1 --2- 172.21.15.172:0/1643375810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe618103700 0x7fe618100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:35.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.348+0000 7fe61e580640 1 --2- 172.21.15.172:0/1643375810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe618102de0 0x7fe618100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:35.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.348+0000 7fe61e580640 1 -- 172.21.15.172:0/1643375810 >> 172.21.15.172:0/1643375810 conn(0x7fe6180fc7a0 msgr2=0x7fe6180fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:35.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.348+0000 7fe61e580640 1 -- 172.21.15.172:0/1643375810 shutdown_connections 2024-09-10T08:57:35.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:35.348+0000 7fe61e580640 1 -- 172.21.15.172:0/1643375810 wait complete. 2024-09-10T08:57:35.711 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:35 smithi172 ceph-mon[30854]: pgmap v691: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 541 KiB/s wr, 13 op/s 2024-09-10T08:57:35.711 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:35 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1643375810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:57:35.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:35 smithi174 ceph-mon[34068]: pgmap v691: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 541 KiB/s wr, 13 op/s 2024-09-10T08:57:35.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:35 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1643375810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:57:35.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:35 smithi191 ceph-mon[35314]: pgmap v691: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 541 KiB/s wr, 13 op/s 2024-09-10T08:57:35.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:35 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1643375810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:57:36.059 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id bb1cd559 --release --await' 2024-09-10T08:57:36.059 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id bb1cd559 --release --await 2024-09-10T08:57:36.308 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:57:36.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.796+0000 7f18d37a9640 1 -- 172.21.15.172:0/1386989881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18cc102570 msgr2=0x7f18cc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:36.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.796+0000 7f18d37a9640 1 --2- 172.21.15.172:0/1386989881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18cc102570 0x7f18cc1029b0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009900 tx=0x7f18c002a480 comp rx=0 tx=0).stop 2024-09-10T08:57:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.796+0000 7f18d37a9640 1 -- 172.21.15.172:0/1386989881 shutdown_connections 2024-09-10T08:57:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.796+0000 7f18d37a9640 1 --2- 172.21.15.172:0/1386989881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18cc102ef0 0x7f18cc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.796+0000 7f18d37a9640 1 --2- 172.21.15.172:0/1386989881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18cc102570 0x7f18cc1029b0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.796+0000 7f18d37a9640 1 --2- 172.21.15.172:0/1386989881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18cc100e20 0x7f18cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.796+0000 7f18d37a9640 1 -- 172.21.15.172:0/1386989881 >> 172.21.15.172:0/1386989881 conn(0x7f18cc0fc7d0 msgr2=0x7f18cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.796+0000 7f18d37a9640 1 -- 172.21.15.172:0/1386989881 shutdown_connections 2024-09-10T08:57:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.796+0000 7f18d37a9640 1 -- 172.21.15.172:0/1386989881 wait complete. 2024-09-10T08:57:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.796+0000 7f18d37a9640 1 Processor -- start 2024-09-10T08:57:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.796+0000 7f18d37a9640 1 -- start start 2024-09-10T08:57:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.796+0000 7f18d37a9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18cc100e20 0x7f18cc1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.796+0000 7f18d37a9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18cc102570 0x7f18cc197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.796+0000 7f18d37a9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18cc102ef0 0x7f18cc19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18d37a9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc110e50 con 0x7f18cc102ef0 2024-09-10T08:57:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18d37a9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc110cd0 con 0x7f18cc100e20 2024-09-10T08:57:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18d37a9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc110fd0 con 0x7f18cc102570 2024-09-10T08:57:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18d1d46640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18cc102ef0 0x7f18cc19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18d1d46640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18cc102ef0 0x7f18cc19bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60118/0 (socket says 172.21.15.172:60118) 2024-09-10T08:57:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18d1d46640 1 -- 172.21.15.172:0/3441551400 learned_addr learned my addr 172.21.15.172:0/3441551400 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18d0d44640 1 --2- 172.21.15.172:0/3441551400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18cc102570 0x7f18cc197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18d1545640 1 --2- 172.21.15.172:0/3441551400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18cc100e20 0x7f18cc1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18d1d46640 1 -- 172.21.15.172:0/3441551400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18cc102570 msgr2=0x7f18cc197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18d1d46640 1 --2- 172.21.15.172:0/3441551400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18cc102570 0x7f18cc197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18d1d46640 1 -- 172.21.15.172:0/3441551400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18cc100e20 msgr2=0x7f18cc1972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18d1d46640 1 --2- 172.21.15.172:0/3441551400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18cc100e20 0x7f18cc1972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18d1d46640 1 -- 172.21.15.172:0/3441551400 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18cc19c280 con 0x7f18cc102ef0 2024-09-10T08:57:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18d1d46640 1 --2- 172.21.15.172:0/3441551400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18cc102ef0 0x7f18cc19bba0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009fd0 tx=0x7f18c400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18ca7fc640 1 -- 172.21.15.172:0/3441551400 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4011830 con 0x7f18cc102ef0 2024-09-10T08:57:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18ca7fc640 1 -- 172.21.15.172:0/3441551400 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18c4016070 con 0x7f18cc102ef0 2024-09-10T08:57:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18d37a9640 1 -- 172.21.15.172:0/3441551400 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18cc19c570 con 0x7f18cc102ef0 2024-09-10T08:57:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18ca7fc640 1 -- 172.21.15.172:0/3441551400 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4010de0 con 0x7f18cc102ef0 2024-09-10T08:57:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.797+0000 7f18d37a9640 1 -- 172.21.15.172:0/3441551400 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18cc1a3e50 con 0x7f18cc102ef0 2024-09-10T08:57:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.799+0000 7f18ca7fc640 1 -- 172.21.15.172:0/3441551400 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f18c40119d0 con 0x7f18cc102ef0 2024-09-10T08:57:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.800+0000 7f18d5b82740 1 Processor -- start 2024-09-10T08:57:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.800+0000 7f18d5b82740 1 -- start start 2024-09-10T08:57:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.800+0000 7f18d5b82740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9ef060 0x560a5f9ef440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.800+0000 7f18d5b82740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9ef980 0x560a5f9f9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.800+0000 7f18d5b82740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9fa240 0x560a5f9fc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.800+0000 7f18d5b82740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560a5f954620 con 0x560a5f9ef060 2024-09-10T08:57:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.800+0000 7f18d5b82740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560a5f8fbe50 con 0x560a5f9ef980 2024-09-10T08:57:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.800+0000 7f18d5b82740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560a5f9505e0 con 0x560a5f9fa240 2024-09-10T08:57:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.800+0000 7f18ca7fc640 1 --2- 172.21.15.172:0/3441551400 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f18b40a2b80 0x7f18b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.800+0000 7f18d0d44640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9ef060 0x560a5f9ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.801+0000 7f18d0d44640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9ef060 0x560a5f9ef440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60132/0 (socket says 172.21.15.172:60132) 2024-09-10T08:57:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.801+0000 7f18d0d44640 1 -- 172.21.15.172:0/4154043718 learned_addr learned my addr 172.21.15.172:0/4154043718 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.801+0000 7f18d1545640 1 --2- 172.21.15.172:0/4154043718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9ef980 0x560a5f9f9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.801+0000 7f18d0d44640 1 --2- 172.21.15.172:0/4154043718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9fa240 0x560a5f9fc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.801+0000 7f18ca7fc640 1 -- 172.21.15.172:0/3441551400 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f18c40ceb10 con 0x7f18cc102ef0 2024-09-10T08:57:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.801+0000 7f18d1d46640 1 --2- 172.21.15.172:0/3441551400 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f18b40a2b80 0x7f18b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.801+0000 7f18d0d44640 1 -- 172.21.15.172:0/4154043718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9fa240 msgr2=0x560a5f9fc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.801+0000 7f18d0d44640 1 --2- 172.21.15.172:0/4154043718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9fa240 0x560a5f9fc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.801+0000 7f18d0d44640 1 -- 172.21.15.172:0/4154043718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9ef980 msgr2=0x560a5f9f9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.801+0000 7f18d0d44640 1 --2- 172.21.15.172:0/4154043718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9ef980 0x560a5f9f9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.801+0000 7f18d0d44640 1 -- 172.21.15.172:0/4154043718 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a5f8ecd40 con 0x560a5f9ef060 2024-09-10T08:57:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.801+0000 7f18d0d44640 1 --2- 172.21.15.172:0/4154043718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9ef060 0x560a5f9ef440 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x560a5f9ec970 tx=0x7f18c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85433ce4ed9b1d6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.801+0000 7f18d1d46640 1 --2- 172.21.15.172:0/3441551400 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f18b40a2b80 0x7f18b40a5040 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f18c4097980 tx=0x7f18c4011000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.801+0000 7f18b37fe640 1 -- 172.21.15.172:0/4154043718 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0035070 con 0x560a5f9ef060 2024-09-10T08:57:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.801+0000 7f18b37fe640 1 -- 172.21.15.172:0/4154043718 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18c0031740 con 0x560a5f9ef060 2024-09-10T08:57:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.801+0000 7f18b37fe640 1 -- 172.21.15.172:0/4154043718 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0003bb0 con 0x560a5f9ef060 2024-09-10T08:57:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.802+0000 7f18d5b82740 1 -- 172.21.15.172:0/4154043718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9ef060 msgr2=0x560a5f9ef440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.802+0000 7f18d5b82740 1 --2- 172.21.15.172:0/4154043718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9ef060 0x560a5f9ef440 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x560a5f9ec970 tx=0x7f18c0000bc0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.802+0000 7f18d5b82740 1 -- 172.21.15.172:0/4154043718 shutdown_connections 2024-09-10T08:57:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.802+0000 7f18d5b82740 1 --2- 172.21.15.172:0/4154043718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9fa240 0x560a5f9fc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.802+0000 7f18d5b82740 1 --2- 172.21.15.172:0/4154043718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9ef980 0x560a5f9f9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.802+0000 7f18d5b82740 1 --2- 172.21.15.172:0/4154043718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9ef060 0x560a5f9ef440 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.802+0000 7f18d5b82740 1 -- 172.21.15.172:0/4154043718 >> 172.21.15.172:0/4154043718 conn(0x560a5f9eb5f0 msgr2=0x560a5f9eb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.802+0000 7f18d5b82740 1 -- 172.21.15.172:0/4154043718 shutdown_connections 2024-09-10T08:57:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.802+0000 7f18d5b82740 1 -- 172.21.15.172:0/4154043718 wait complete. 2024-09-10T08:57:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.802+0000 7f18d5b82740 1 -- start start 2024-09-10T08:57:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d5b82740 1 Processor -- start 2024-09-10T08:57:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d5b82740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9ef060 0x560a5f9f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d5b82740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9ef980 0x560a5f9f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d5b82740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9fa240 0x560a5f9f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d5b82740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560a5f9fce10 con 0x560a5f9fa240 2024-09-10T08:57:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d5b82740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560a5f8dcf10 con 0x560a5f9ef060 2024-09-10T08:57:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d5b82740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560a5f9fcf90 con 0x560a5f9ef980 2024-09-10T08:57:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d5b82740 10 client.0 _ll_register_callbacks cb 0x560a5f900db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d0d44640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9fa240 0x560a5f9f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d0d44640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9fa240 0x560a5f9f7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60140/0 (socket says 172.21.15.172:60140) 2024-09-10T08:57:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d0d44640 1 -- 172.21.15.172:0/4046674984 learned_addr learned my addr 172.21.15.172:0/4046674984 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d0d44640 1 --2- 172.21.15.172:0/4046674984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9ef060 0x560a5f9f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d1545640 1 --2- 172.21.15.172:0/4046674984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9ef980 0x560a5f9f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d0d44640 1 -- 172.21.15.172:0/4046674984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9ef980 msgr2=0x560a5f9f76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d0d44640 1 --2- 172.21.15.172:0/4046674984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9ef980 0x560a5f9f76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d0d44640 1 -- 172.21.15.172:0/4046674984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9ef060 msgr2=0x560a5f9f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d0d44640 1 --2- 172.21.15.172:0/4046674984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9ef060 0x560a5f9f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.803+0000 7f18d0d44640 1 -- 172.21.15.172:0/4046674984 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a5f663580 con 0x560a5f9fa240 2024-09-10T08:57:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.804+0000 7f18d0d44640 1 --2- 172.21.15.172:0/4046674984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9fa240 0x560a5f9f7c20 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f18c00096c0 tx=0x7f18c0042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703ea92182aac390 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.804+0000 7f18b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.804+0000 7f18b0ff9640 1 -- 172.21.15.172:0/4046674984 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0035070 con 0x560a5f9fa240 2024-09-10T08:57:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.804+0000 7f18b0ff9640 1 -- 172.21.15.172:0/4046674984 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f18c0003fa0 con 0x560a5f9fa240 2024-09-10T08:57:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.804+0000 7f18d5b82740 1 -- 172.21.15.172:0/4046674984 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18c0031740 con 0x560a5f9fa240 2024-09-10T08:57:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.804+0000 7f18b0ff9640 1 -- 172.21.15.172:0/4046674984 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f18c00041a0 con 0x560a5f9fa240 2024-09-10T08:57:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.804+0000 7f18d5b82740 10 client.18825 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.804+0000 7f18d5b82740 1 -- 172.21.15.172:0/4046674984 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560a5f72d960 con 0x560a5f9fa240 2024-09-10T08:57:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.804+0000 7f18b0ff9640 1 -- 172.21.15.172:0/4046674984 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18c0031740 con 0x560a5f9fa240 2024-09-10T08:57:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.804+0000 7f18b0ff9640 1 -- 172.21.15.172:0/4046674984 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f18c004a460 con 0x560a5f9fa240 2024-09-10T08:57:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.804+0000 7f18d5b82740 10 client.18825 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.804+0000 7f18d5b82740 10 client.18825 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:57:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.804+0000 7f18d5b82740 20 client.18825 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.804+0000 7f18d5b82740 1 --2- 172.21.15.172:0/4046674984 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560a5fa09cc0 0x560a5fa0c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.804+0000 7f18d5b82740 4 client.18825 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:57:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.804+0000 7f18d5b82740 1 -- 172.21.15.172:0/4046674984 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a5f961b90 con 0x560a5fa09cc0 2024-09-10T08:57:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.805+0000 7f18d1545640 1 --2- 172.21.15.172:0/4046674984 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560a5fa09cc0 0x560a5fa0c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.805+0000 7f18d1545640 1 --2- 172.21.15.172:0/4046674984 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560a5fa09cc0 0x560a5fa0c0b0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac5fa752464811b4 server_cookie=6eb066be4392a6b4 in_seq=0 out_seq=0 2024-09-10T08:57:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.805+0000 7f18b0ff9640 10 client.18825 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.806+0000 7f18b0ff9640 1 -- 172.21.15.172:0/4046674984 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a5f961b90 con 0x560a5fa09cc0 2024-09-10T08:57:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.806+0000 7f18b0ff9640 10 client.18825 handle_command_reply: tid=0 2024-09-10T08:57:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.806+0000 7f18d5b82740 1 client.18825 shutdown 2024-09-10T08:57:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.806+0000 7f18d5b82740 1 -- 172.21.15.172:0/4046674984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9fa240 msgr2=0x560a5f9f7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.806+0000 7f18d5b82740 1 --2- 172.21.15.172:0/4046674984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9fa240 0x560a5f9f7c20 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f18c00096c0 tx=0x7f18c0042040 comp rx=0 tx=0).stop 2024-09-10T08:57:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.806+0000 7f18d5b82740 1 -- 172.21.15.172:0/4046674984 shutdown_connections 2024-09-10T08:57:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.806+0000 7f18d5b82740 1 --2- 172.21.15.172:0/4046674984 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560a5fa09cc0 0x560a5fa0c0b0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.806+0000 7f18d5b82740 1 --2- 172.21.15.172:0/4046674984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9fa240 0x560a5f9f7c20 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.806+0000 7f18d5b82740 1 --2- 172.21.15.172:0/4046674984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9ef980 0x560a5f9f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.806+0000 7f18b0ff9640 0 client.18825 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:36.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.806+0000 7f18d5b82740 1 --2- 172.21.15.172:0/4046674984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9ef060 0x560a5f9f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.806+0000 7f18d5b82740 1 -- 172.21.15.172:0/4046674984 >> 172.21.15.172:0/4046674984 conn(0x560a5f9eb5f0 msgr2=0x560a5f9f1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:36.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.806+0000 7f18d5b82740 1 -- 172.21.15.172:0/4046674984 shutdown_connections 2024-09-10T08:57:36.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.806+0000 7f18d5b82740 1 -- 172.21.15.172:0/4046674984 wait complete. 2024-09-10T08:57:36.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.806+0000 7f18d5b82740 20 client.18825 trim_cache size 0 max 16384 2024-09-10T08:57:36.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.823+0000 7f18d5b82740 1 Processor -- start 2024-09-10T08:57:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.823+0000 7f18d5b82740 1 -- start start 2024-09-10T08:57:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.823+0000 7f18d5b82740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9ef060 0x560a5f9f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.823+0000 7f18d5b82740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9ef980 0x560a5f9f6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.823+0000 7f18d5b82740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9fa240 0x560a5fa0f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.823+0000 7f18d5b82740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc1ad980 con 0x560a5f9ef060 2024-09-10T08:57:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.823+0000 7f18d5b82740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc1af520 con 0x560a5f9fa240 2024-09-10T08:57:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.823+0000 7f18d5b82740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc1ae050 con 0x560a5f9ef980 2024-09-10T08:57:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.823+0000 7f18d0d44640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9ef060 0x560a5f9f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.823+0000 7f18d0d44640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9ef060 0x560a5f9f1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60156/0 (socket says 172.21.15.172:60156) 2024-09-10T08:57:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.823+0000 7f18d0d44640 1 -- 172.21.15.172:0/1797626279 learned_addr learned my addr 172.21.15.172:0/1797626279 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.823+0000 7f18d1545640 1 --2- 172.21.15.172:0/1797626279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9ef980 0x560a5f9f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.823+0000 7f18d0d44640 1 -- 172.21.15.172:0/1797626279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9ef980 msgr2=0x560a5f9f6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:36.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.823+0000 7f18d0d44640 1 --2- 172.21.15.172:0/1797626279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9ef980 0x560a5f9f6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.823+0000 7f18d0d44640 1 -- 172.21.15.172:0/1797626279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9fa240 msgr2=0x560a5fa0f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:36.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.823+0000 7f18d0d44640 1 --2- 172.21.15.172:0/1797626279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9fa240 0x560a5fa0f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.823+0000 7f18d0d44640 1 -- 172.21.15.172:0/1797626279 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a5f9ec6a0 con 0x560a5f9ef060 2024-09-10T08:57:36.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.824+0000 7f18d0d44640 1 --2- 172.21.15.172:0/1797626279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9ef060 0x560a5f9f1ac0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f18c002a9a0 tx=0x7f18c002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8385083617aa540f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:36.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.824+0000 7f18b2ffd640 1 -- 172.21.15.172:0/1797626279 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0035070 con 0x560a5f9ef060 2024-09-10T08:57:36.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.824+0000 7f18b2ffd640 1 -- 172.21.15.172:0/1797626279 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18c0050070 con 0x560a5f9ef060 2024-09-10T08:57:36.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.824+0000 7f18b2ffd640 1 -- 172.21.15.172:0/1797626279 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c004aaf0 con 0x560a5f9ef060 2024-09-10T08:57:36.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.824+0000 7f18d5b82740 1 -- 172.21.15.172:0/1797626279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9ef060 msgr2=0x560a5f9f1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:36.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.824+0000 7f18d5b82740 1 --2- 172.21.15.172:0/1797626279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9ef060 0x560a5f9f1ac0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f18c002a9a0 tx=0x7f18c002a480 comp rx=0 tx=0).stop 2024-09-10T08:57:36.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.824+0000 7f18d5b82740 1 -- 172.21.15.172:0/1797626279 shutdown_connections 2024-09-10T08:57:36.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.824+0000 7f18d5b82740 1 --2- 172.21.15.172:0/1797626279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9fa240 0x560a5fa0f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.824+0000 7f18d5b82740 1 --2- 172.21.15.172:0/1797626279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9ef980 0x560a5f9f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.824+0000 7f18d5b82740 1 --2- 172.21.15.172:0/1797626279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9ef060 0x560a5f9f1ac0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.824+0000 7f18d5b82740 1 -- 172.21.15.172:0/1797626279 >> 172.21.15.172:0/1797626279 conn(0x560a5f9eb5f0 msgr2=0x560a5f9f5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:36.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.824+0000 7f18d5b82740 1 -- 172.21.15.172:0/1797626279 shutdown_connections 2024-09-10T08:57:36.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.825+0000 7f18d5b82740 1 -- 172.21.15.172:0/1797626279 wait complete. 2024-09-10T08:57:36.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.825+0000 7f18d5b82740 1 -- start start 2024-09-10T08:57:36.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.825+0000 7f18d5b82740 1 Processor -- start 2024-09-10T08:57:36.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.825+0000 7f18d5b82740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9ef060 0x560a5fa6a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.825+0000 7f18d5b82740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9ef980 0x560a5fa6acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.825+0000 7f18d5b82740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9fa240 0x560a5fa6e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.825+0000 7f18d5b82740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc1aee50 con 0x560a5f9fa240 2024-09-10T08:57:36.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.825+0000 7f18d5b82740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc1ae780 con 0x560a5f9ef060 2024-09-10T08:57:36.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.825+0000 7f18d5b82740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc1ab960 con 0x560a5f9ef980 2024-09-10T08:57:36.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.825+0000 7f18d5b82740 10 client.0 _ll_register_callbacks cb 0x7f18c00035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:36.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.825+0000 7f18d0d44640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9fa240 0x560a5fa6e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:36.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.825+0000 7f18d0d44640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9fa240 0x560a5fa6e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60158/0 (socket says 172.21.15.172:60158) 2024-09-10T08:57:36.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.825+0000 7f18d0d44640 1 -- 172.21.15.172:0/373663174 learned_addr learned my addr 172.21.15.172:0/373663174 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:36.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.826+0000 7f18d1545640 1 --2- 172.21.15.172:0/373663174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9ef980 0x560a5fa6acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:36.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.826+0000 7f18d0d44640 1 --2- 172.21.15.172:0/373663174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9ef060 0x560a5fa6a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:36.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.826+0000 7f18d0d44640 1 -- 172.21.15.172:0/373663174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9ef980 msgr2=0x560a5fa6acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:36.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.826+0000 7f18d0d44640 1 --2- 172.21.15.172:0/373663174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9ef980 0x560a5fa6acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.826+0000 7f18d0d44640 1 -- 172.21.15.172:0/373663174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9ef060 msgr2=0x560a5fa6a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:36.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.826+0000 7f18d0d44640 1 --2- 172.21.15.172:0/373663174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9ef060 0x560a5fa6a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.826+0000 7f18d0d44640 1 -- 172.21.15.172:0/373663174 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a5f9ec280 con 0x560a5f9fa240 2024-09-10T08:57:36.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.826+0000 7f18d0d44640 1 --2- 172.21.15.172:0/373663174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9fa240 0x560a5fa6e7a0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f18c0005940 tx=0x7f18c0005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31006657de7d5300 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:36.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.826+0000 7f18b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:36.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.826+0000 7f18b0ff9640 1 -- 172.21.15.172:0/373663174 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0035030 con 0x560a5f9fa240 2024-09-10T08:57:36.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.826+0000 7f18b0ff9640 1 -- 172.21.15.172:0/373663174 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f18c004ad10 con 0x560a5f9fa240 2024-09-10T08:57:36.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.826+0000 7f18d5b82740 1 -- 172.21.15.172:0/373663174 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18c0050070 con 0x560a5f9fa240 2024-09-10T08:57:36.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.826+0000 7f18b0ff9640 1 -- 172.21.15.172:0/373663174 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f18c0005d50 con 0x560a5f9fa240 2024-09-10T08:57:36.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.826+0000 7f18d5b82740 10 client.18831 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:36.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.826+0000 7f18d5b82740 1 -- 172.21.15.172:0/373663174 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560a5f9ec280 con 0x560a5f9fa240 2024-09-10T08:57:36.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.826+0000 7f18b0ff9640 1 -- 172.21.15.172:0/373663174 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18c0050070 con 0x560a5f9fa240 2024-09-10T08:57:36.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.827+0000 7f18b0ff9640 1 -- 172.21.15.172:0/373663174 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f18c005f030 con 0x560a5f9fa240 2024-09-10T08:57:36.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.827+0000 7f18d5b82740 10 client.18831 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:36.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.827+0000 7f18d5b82740 10 client.18831 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:57:36.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.827+0000 7f18d5b82740 20 client.18831 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:36.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.827+0000 7f18d5b82740 1 --2- 172.21.15.172:0/373663174 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560a5fa77490 0x560a5fa79880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:36.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.827+0000 7f18d5b82740 4 client.18831 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bb1cd559", "release": true, "await": true}] 2024-09-10T08:57:36.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.827+0000 7f18d5b82740 1 -- 172.21.15.172:0/373663174 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bb1cd559", "release": true, "await": true}) -- 0x560a5f95c0a0 con 0x560a5fa77490 2024-09-10T08:57:36.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.827+0000 7f18d1545640 1 --2- 172.21.15.172:0/373663174 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560a5fa77490 0x560a5fa79880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:36.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.827+0000 7f18b0ff9640 10 client.18831 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:36.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.827+0000 7f18d1545640 1 --2- 172.21.15.172:0/373663174 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560a5fa77490 0x560a5fa79880 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98fc1ee23cb13d71 server_cookie=cfcf11e28805789f in_seq=0 out_seq=0 2024-09-10T08:57:36.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.853+0000 7f18b0ff9640 1 -- 172.21.15.172:0/373663174 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560a5f95c0a0 con 0x560a5fa77490 2024-09-10T08:57:36.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.853+0000 7f18b0ff9640 10 client.18831 handle_command_reply: tid=0 2024-09-10T08:57:36.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.853+0000 7f18d5b82740 1 client.18831 shutdown 2024-09-10T08:57:36.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.853+0000 7f18d5b82740 1 -- 172.21.15.172:0/373663174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9fa240 msgr2=0x560a5fa6e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:36.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.853+0000 7f18d5b82740 1 --2- 172.21.15.172:0/373663174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9fa240 0x560a5fa6e7a0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f18c0005940 tx=0x7f18c0005c90 comp rx=0 tx=0).stop 2024-09-10T08:57:36.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.853+0000 7f18d5b82740 1 -- 172.21.15.172:0/373663174 shutdown_connections 2024-09-10T08:57:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.853+0000 7f18d5b82740 1 --2- 172.21.15.172:0/373663174 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560a5fa77490 0x560a5fa79880 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.853+0000 7f18d5b82740 1 --2- 172.21.15.172:0/373663174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a5f9fa240 0x560a5fa6e7a0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.853+0000 7f18d5b82740 1 --2- 172.21.15.172:0/373663174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560a5f9ef980 0x560a5fa6acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.853+0000 7f18d5b82740 1 --2- 172.21.15.172:0/373663174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a5f9ef060 0x560a5fa6a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.853+0000 7f18d5b82740 1 -- 172.21.15.172:0/373663174 >> 172.21.15.172:0/373663174 conn(0x560a5f9eb5f0 msgr2=0x560a5f9f7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.853+0000 7f18b0ff9640 0 client.18831 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.853+0000 7f18d5b82740 1 -- 172.21.15.172:0/373663174 shutdown_connections 2024-09-10T08:57:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.853+0000 7f18d5b82740 1 -- 172.21.15.172:0/373663174 wait complete. 2024-09-10T08:57:36.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.853+0000 7f18d5b82740 20 client.18831 trim_cache size 0 max 16384 2024-09-10T08:57:36.855 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:57:36.856 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 39, 2024-09-10T08:57:36.856 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:57:36.856 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 40, 2024-09-10T08:57:36.856 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:57:36.856 INFO:teuthology.orchestra.run.smithi172.stdout: "bb1cd559": { 2024-09-10T08:57:36.856 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 40, 2024-09-10T08:57:36.856 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:57:36.856 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:57:36.856 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:57:36.856 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:57:36.856 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:57:36.856 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:57:36.857 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 145.2, 2024-09-10T08:57:36.857 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:57:36.857 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:57:36.857 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:57:36.857 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:57:36.857 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:57:36.857 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:57:36.857 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:57:36.857 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:57:36.857 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:57:36.857 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:57:36.857 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:57:36.857 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:57:36.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.854+0000 7f18d37a9640 1 -- 172.21.15.172:0/3441551400 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f18b40a2b80 msgr2=0x7f18b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:36.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.854+0000 7f18d37a9640 1 --2- 172.21.15.172:0/3441551400 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f18b40a2b80 0x7f18b40a5040 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f18c4097980 tx=0x7f18c4011000 comp rx=0 tx=0).stop 2024-09-10T08:57:36.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.854+0000 7f18d37a9640 1 -- 172.21.15.172:0/3441551400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18cc102ef0 msgr2=0x7f18cc19bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:36.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.854+0000 7f18d37a9640 1 --2- 172.21.15.172:0/3441551400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18cc102ef0 0x7f18cc19bba0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009fd0 tx=0x7f18c400b8e0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.854+0000 7f18d37a9640 1 -- 172.21.15.172:0/3441551400 shutdown_connections 2024-09-10T08:57:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.854+0000 7f18d37a9640 1 --2- 172.21.15.172:0/3441551400 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f18b40a2b80 0x7f18b40a5040 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.854+0000 7f18d37a9640 1 --2- 172.21.15.172:0/3441551400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18cc102ef0 0x7f18cc19bba0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.854+0000 7f18d37a9640 1 --2- 172.21.15.172:0/3441551400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18cc102570 0x7f18cc197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.854+0000 7f18d37a9640 1 --2- 172.21.15.172:0/3441551400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18cc100e20 0x7f18cc1972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.854+0000 7f18d37a9640 1 -- 172.21.15.172:0/3441551400 >> 172.21.15.172:0/3441551400 conn(0x7f18cc0fc7d0 msgr2=0x7f18cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.854+0000 7f18d37a9640 1 -- 172.21.15.172:0/3441551400 shutdown_connections 2024-09-10T08:57:36.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:36.854+0000 7f18d37a9640 1 -- 172.21.15.172:0/3441551400 wait complete. 2024-09-10T08:57:37.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:37 smithi174 ceph-mon[34068]: pgmap v692: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-10T08:57:37.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:37 smithi191 ceph-mon[35314]: pgmap v692: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-10T08:57:37.815 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:37 smithi172 ceph-mon[30854]: pgmap v692: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-10T08:57:37.848 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bb1cd559', total seconds elapsed: 28.6 2024-09-10T08:57:39.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:39 smithi174 ceph-mon[34068]: pgmap v693: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 MiB/s wr, 12 op/s 2024-09-10T08:57:39.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:39 smithi191 ceph-mon[35314]: pgmap v693: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 MiB/s wr, 12 op/s 2024-09-10T08:57:39.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:39 smithi172 ceph-mon[30854]: pgmap v693: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 MiB/s wr, 12 op/s 2024-09-10T08:57:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:41 smithi174 ceph-mon[34068]: pgmap v694: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.8 MiB/s wr, 14 op/s 2024-09-10T08:57:41.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:41 smithi191 ceph-mon[35314]: pgmap v694: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.8 MiB/s wr, 14 op/s 2024-09-10T08:57:41.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:41 smithi172 ceph-mon[30854]: pgmap v694: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.8 MiB/s wr, 14 op/s 2024-09-10T08:57:42.987 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.986+0000 7f1067def640 1 -- 172.21.15.172:0/2113545541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1060107890 msgr2=0x7f1060109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.986+0000 7f1067def640 1 --2- 172.21.15.172:0/2113545541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1060107890 0x7f1060109ca0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f1058009ca0 tx=0x7f105802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:57:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.986+0000 7f1067def640 1 -- 172.21.15.172:0/2113545541 shutdown_connections 2024-09-10T08:57:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.986+0000 7f1067def640 1 --2- 172.21.15.172:0/2113545541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1060107890 0x7f1060109ca0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.986+0000 7f1067def640 1 --2- 172.21.15.172:0/2113545541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1060104f60 0x7f1060107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.986+0000 7f1067def640 1 --2- 172.21.15.172:0/2113545541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1060102630 0x7f1060104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.986+0000 7f1067def640 1 -- 172.21.15.172:0/2113545541 >> 172.21.15.172:0/2113545541 conn(0x7f10600fc1b0 msgr2=0x7f10600fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.986+0000 7f1067def640 1 -- 172.21.15.172:0/2113545541 shutdown_connections 2024-09-10T08:57:42.988 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.986+0000 7f1067def640 1 -- 172.21.15.172:0/2113545541 wait complete. 2024-09-10T08:57:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.987+0000 7f1067def640 1 Processor -- start 2024-09-10T08:57:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.987+0000 7f1067def640 1 -- start start 2024-09-10T08:57:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.988+0000 7f1067def640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1060102630 0x7f1060197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.988+0000 7f1067def640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1060104f60 0x7f10601982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.988+0000 7f1067def640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1060107890 0x7f106019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.988+0000 7f1067def640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f106010d6c0 con 0x7f1060102630 2024-09-10T08:57:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.988+0000 7f1067def640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f106010d540 con 0x7f1060107890 2024-09-10T08:57:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.988+0000 7f1067def640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f106010d840 con 0x7f1060104f60 2024-09-10T08:57:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.988+0000 7f1065b8b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1060102630 0x7f1060197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.988+0000 7f1065b8b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1060102630 0x7f1060197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60188/0 (socket says 172.21.15.172:60188) 2024-09-10T08:57:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.988+0000 7f1065b8b640 1 -- 172.21.15.172:0/4086030669 learned_addr learned my addr 172.21.15.172:0/4086030669 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.988+0000 7f106538a640 1 --2- 172.21.15.172:0/4086030669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1060104f60 0x7f10601982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.988+0000 7f1065b8b640 1 -- 172.21.15.172:0/4086030669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1060104f60 msgr2=0x7f10601982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.988+0000 7f1065b8b640 1 --2- 172.21.15.172:0/4086030669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1060104f60 0x7f10601982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.988+0000 7f1065b8b640 1 -- 172.21.15.172:0/4086030669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1060107890 msgr2=0x7f106019f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.988+0000 7f1065b8b640 1 --2- 172.21.15.172:0/4086030669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1060107890 0x7f106019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.988+0000 7f1065b8b640 1 -- 172.21.15.172:0/4086030669 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106019f980 con 0x7f1060102630 2024-09-10T08:57:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.988+0000 7f1065b8b640 1 --2- 172.21.15.172:0/4086030669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1060102630 0x7f1060197d60 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f105c00e9f0 tx=0x7f105c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.989+0000 7f1056ffd640 1 -- 172.21.15.172:0/4086030669 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c00cd80 con 0x7f1060102630 2024-09-10T08:57:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.989+0000 7f1067def640 1 -- 172.21.15.172:0/4086030669 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106019fc70 con 0x7f1060102630 2024-09-10T08:57:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.989+0000 7f1056ffd640 1 -- 172.21.15.172:0/4086030669 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f105c0054d0 con 0x7f1060102630 2024-09-10T08:57:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.989+0000 7f1056ffd640 1 -- 172.21.15.172:0/4086030669 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c010490 con 0x7f1060102630 2024-09-10T08:57:42.992 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.989+0000 7f1067def640 1 -- 172.21.15.172:0/4086030669 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10601a01c0 con 0x7f1060102630 2024-09-10T08:57:42.995 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.992+0000 7f1056ffd640 1 -- 172.21.15.172:0/4086030669 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f105c005000 con 0x7f1060102630 2024-09-10T08:57:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.992+0000 7f106ac65740 1 Processor -- start 2024-09-10T08:57:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.992+0000 7f106ac65740 1 -- start start 2024-09-10T08:57:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f106ac65740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 0x56115ff46da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f106ac65740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff472e0 0x56115ff4bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f106ac65740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff4c370 0x56115ff4e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f106ac65740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56115feaa2f0 con 0x56115ff469a0 2024-09-10T08:57:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f106ac65740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56115fe52270 con 0x56115ff4c370 2024-09-10T08:57:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f106ac65740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56115fea62b0 con 0x56115ff472e0 2024-09-10T08:57:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f106538a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff4c370 0x56115ff4e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f106638c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 0x56115ff46da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f1065b8b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff472e0 0x56115ff4bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f106538a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff4c370 0x56115ff4e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:46722/0 (socket says 172.21.15.172:46722) 2024-09-10T08:57:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f106538a640 1 -- 172.21.15.172:0/1293299141 learned_addr learned my addr 172.21.15.172:0/1293299141 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f106538a640 1 -- 172.21.15.172:0/1293299141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff472e0 msgr2=0x56115ff4bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f106538a640 1 --2- 172.21.15.172:0/1293299141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff472e0 0x56115ff4bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f106538a640 1 -- 172.21.15.172:0/1293299141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 msgr2=0x56115ff46da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f106538a640 1 --2- 172.21.15.172:0/1293299141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 0x56115ff46da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f106538a640 1 -- 172.21.15.172:0/1293299141 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56115fe51340 con 0x56115ff4c370 2024-09-10T08:57:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f1056ffd640 1 --2- 172.21.15.172:0/4086030669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f104c0a2c30 0x7f104c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f106638c640 1 --2- 172.21.15.172:0/4086030669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f104c0a2c30 0x7f104c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f1056ffd640 1 -- 172.21.15.172:0/4086030669 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f105c0cf880 con 0x7f1060102630 2024-09-10T08:57:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.993+0000 7f106538a640 1 --2- 172.21.15.172:0/1293299141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff4c370 0x56115ff4e830 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1050009c60 tx=0x7f1050011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7e94eb76cbb60f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.994+0000 7f104bfff640 1 -- 172.21.15.172:0/1293299141 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050004230 con 0x56115ff4c370 2024-09-10T08:57:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.994+0000 7f104bfff640 1 -- 172.21.15.172:0/1293299141 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10500043d0 con 0x56115ff4c370 2024-09-10T08:57:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.994+0000 7f104bfff640 1 -- 172.21.15.172:0/1293299141 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050004d10 con 0x56115ff4c370 2024-09-10T08:57:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.994+0000 7f106ac65740 1 -- 172.21.15.172:0/1293299141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff4c370 msgr2=0x56115ff4e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.994+0000 7f106ac65740 1 --2- 172.21.15.172:0/1293299141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff4c370 0x56115ff4e830 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1050009c60 tx=0x7f1050011810 comp rx=0 tx=0).stop 2024-09-10T08:57:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.994+0000 7f106638c640 1 --2- 172.21.15.172:0/4086030669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f104c0a2c30 0x7f104c0a50f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1058005e30 tx=0x7f1058005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.994+0000 7f106ac65740 1 -- 172.21.15.172:0/1293299141 shutdown_connections 2024-09-10T08:57:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.994+0000 7f106ac65740 1 --2- 172.21.15.172:0/1293299141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff4c370 0x56115ff4e830 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.994+0000 7f106ac65740 1 --2- 172.21.15.172:0/1293299141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff472e0 0x56115ff4bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.994+0000 7f106ac65740 1 --2- 172.21.15.172:0/1293299141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 0x56115ff46da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.994+0000 7f106ac65740 1 -- 172.21.15.172:0/1293299141 >> 172.21.15.172:0/1293299141 conn(0x56115ff42010 msgr2=0x56115ff42410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.995+0000 7f106ac65740 1 -- 172.21.15.172:0/1293299141 shutdown_connections 2024-09-10T08:57:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.995+0000 7f106ac65740 1 -- 172.21.15.172:0/1293299141 wait complete. 2024-09-10T08:57:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.995+0000 7f106ac65740 1 -- start start 2024-09-10T08:57:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.995+0000 7f106ac65740 1 Processor -- start 2024-09-10T08:57:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.995+0000 7f106ac65740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff469a0 0x56115ff50b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.995+0000 7f106ac65740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff472e0 0x56115ff51070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.995+0000 7f106ac65740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff4c370 0x56115ff580b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.995+0000 7f106ac65740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56115ff4f340 con 0x56115ff4c370 2024-09-10T08:57:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.995+0000 7f106ac65740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56115fe33290 con 0x56115ff472e0 2024-09-10T08:57:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.995+0000 7f106ac65740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56115ff4f4c0 con 0x56115ff469a0 2024-09-10T08:57:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.995+0000 7f106ac65740 10 client.0 _ll_register_callbacks cb 0x56115fe64140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.996+0000 7f106638c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff4c370 0x56115ff580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.996+0000 7f106638c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff4c370 0x56115ff580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60202/0 (socket says 172.21.15.172:60202) 2024-09-10T08:57:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.996+0000 7f106638c640 1 -- 172.21.15.172:0/1805233716 learned_addr learned my addr 172.21.15.172:0/1805233716 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.996+0000 7f1065b8b640 1 --2- 172.21.15.172:0/1805233716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff469a0 0x56115ff50b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.996+0000 7f106538a640 1 --2- 172.21.15.172:0/1805233716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff472e0 0x56115ff51070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.996+0000 7f106638c640 1 -- 172.21.15.172:0/1805233716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff469a0 msgr2=0x56115ff50b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.996+0000 7f106638c640 1 --2- 172.21.15.172:0/1805233716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff469a0 0x56115ff50b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.996+0000 7f106638c640 1 -- 172.21.15.172:0/1805233716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff472e0 msgr2=0x56115ff51070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.996+0000 7f106638c640 1 --2- 172.21.15.172:0/1805233716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff472e0 0x56115ff51070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.996+0000 7f106638c640 1 -- 172.21.15.172:0/1805233716 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56115fbbd210 con 0x56115ff4c370 2024-09-10T08:57:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.996+0000 7f106538a640 1 --2- 172.21.15.172:0/1805233716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff472e0 0x56115ff51070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.996+0000 7f106638c640 1 --2- 172.21.15.172:0/1805233716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff4c370 0x56115ff580b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f1058031da0 tx=0x7f1058031a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9f841c3cb9416 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.996+0000 7f10497fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.996+0000 7f10497fa640 1 -- 172.21.15.172:0/1805233716 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105804e070 con 0x56115ff4c370 2024-09-10T08:57:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.996+0000 7f10497fa640 1 -- 172.21.15.172:0/1805233716 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1058004540 con 0x56115ff4c370 2024-09-10T08:57:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.996+0000 7f106ac65740 1 -- 172.21.15.172:0/1805233716 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10500043d0 con 0x56115ff4c370 2024-09-10T08:57:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.997+0000 7f10497fa640 1 -- 172.21.15.172:0/1805233716 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1058004780 con 0x56115ff4c370 2024-09-10T08:57:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.997+0000 7f106ac65740 10 client.18843 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.997+0000 7f106ac65740 1 -- 172.21.15.172:0/1805233716 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56115fc89620 con 0x56115ff4c370 2024-09-10T08:57:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.997+0000 7f10497fa640 1 -- 172.21.15.172:0/1805233716 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10500043d0 con 0x56115ff4c370 2024-09-10T08:57:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.997+0000 7f10497fa640 1 -- 172.21.15.172:0/1805233716 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f1058043040 con 0x56115ff4c370 2024-09-10T08:57:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.997+0000 7f106ac65740 10 client.18843 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.997+0000 7f106ac65740 10 client.18843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.997+0000 7f106ac65740 20 client.18843 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.997+0000 7f106ac65740 1 --2- 172.21.15.172:0/1805233716 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56115ff5c7d0 0x56115ff5ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.997+0000 7f106ac65740 4 client.18843 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.997+0000 7f106ac65740 1 -- 172.21.15.172:0/1805233716 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56115feb7a20 con 0x56115ff5c7d0 2024-09-10T08:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.997+0000 7f1065b8b640 1 --2- 172.21.15.172:0/1805233716 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56115ff5c7d0 0x56115ff5ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.998+0000 7f1065b8b640 1 --2- 172.21.15.172:0/1805233716 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56115ff5c7d0 0x56115ff5ec30 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b0d01fb59c798c8 server_cookie=400fb7520f5b3c03 in_seq=0 out_seq=0 2024-09-10T08:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.998+0000 7f10497fa640 10 client.18843 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.998+0000 7f10497fa640 1 -- 172.21.15.172:0/1805233716 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56115feb7a20 con 0x56115ff5c7d0 2024-09-10T08:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.998+0000 7f10497fa640 10 client.18843 handle_command_reply: tid=0 2024-09-10T08:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.998+0000 7f106ac65740 1 client.18843 shutdown 2024-09-10T08:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.998+0000 7f106ac65740 1 -- 172.21.15.172:0/1805233716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff4c370 msgr2=0x56115ff580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.998+0000 7f106ac65740 1 --2- 172.21.15.172:0/1805233716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff4c370 0x56115ff580b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f1058031da0 tx=0x7f1058031a40 comp rx=0 tx=0).stop 2024-09-10T08:57:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.999+0000 7f106ac65740 1 -- 172.21.15.172:0/1805233716 shutdown_connections 2024-09-10T08:57:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.999+0000 7f106ac65740 1 --2- 172.21.15.172:0/1805233716 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56115ff5c7d0 0x56115ff5ec30 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.999+0000 7f106ac65740 1 --2- 172.21.15.172:0/1805233716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff4c370 0x56115ff580b0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.999+0000 7f10497fa640 0 client.18843 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.999+0000 7f106ac65740 1 --2- 172.21.15.172:0/1805233716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff472e0 0x56115ff51070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.999+0000 7f106ac65740 1 --2- 172.21.15.172:0/1805233716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff469a0 0x56115ff50b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.999+0000 7f106ac65740 1 -- 172.21.15.172:0/1805233716 >> 172.21.15.172:0/1805233716 conn(0x56115ff42010 msgr2=0x56115ff43820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.999+0000 7f106ac65740 1 -- 172.21.15.172:0/1805233716 shutdown_connections 2024-09-10T08:57:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.999+0000 7f106ac65740 1 -- 172.21.15.172:0/1805233716 wait complete. 2024-09-10T08:57:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:42.999+0000 7f106ac65740 20 client.18843 trim_cache size 0 max 16384 2024-09-10T08:57:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.021+0000 7f106ac65740 1 Processor -- start 2024-09-10T08:57:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.021+0000 7f106ac65740 1 -- start start 2024-09-10T08:57:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.021+0000 7f106ac65740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 0x56115ff42ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.021+0000 7f106ac65740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff472e0 0x56115ff58070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.021+0000 7f106ac65740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff4c370 0x56115ff43400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.021+0000 7f106ac65740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f10601aa580 con 0x56115ff469a0 2024-09-10T08:57:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.021+0000 7f106ac65740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10601ac120 con 0x56115ff4c370 2024-09-10T08:57:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.021+0000 7f106ac65740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f10601aac50 con 0x56115ff472e0 2024-09-10T08:57:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.021+0000 7f1065b8b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 0x56115ff42ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.021+0000 7f1065b8b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 0x56115ff42ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60212/0 (socket says 172.21.15.172:60212) 2024-09-10T08:57:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.021+0000 7f1065b8b640 1 -- 172.21.15.172:0/127697727 learned_addr learned my addr 172.21.15.172:0/127697727 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.022+0000 7f106538a640 1 --2- 172.21.15.172:0/127697727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff472e0 0x56115ff58070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.022+0000 7f106638c640 1 --2- 172.21.15.172:0/127697727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff4c370 0x56115ff43400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.022+0000 7f1065b8b640 1 -- 172.21.15.172:0/127697727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff472e0 msgr2=0x56115ff58070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.022+0000 7f1065b8b640 1 --2- 172.21.15.172:0/127697727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff472e0 0x56115ff58070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.022+0000 7f1065b8b640 1 -- 172.21.15.172:0/127697727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff4c370 msgr2=0x56115ff43400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.022+0000 7f1065b8b640 1 --2- 172.21.15.172:0/127697727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff4c370 0x56115ff43400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.022+0000 7f1065b8b640 1 -- 172.21.15.172:0/127697727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56115ff4eba0 con 0x56115ff469a0 2024-09-10T08:57:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.022+0000 7f106638c640 1 --2- 172.21.15.172:0/127697727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff4c370 0x56115ff43400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:57:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.022+0000 7f106538a640 1 --2- 172.21.15.172:0/127697727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff472e0 0x56115ff58070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:57:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.022+0000 7f1065b8b640 1 --2- 172.21.15.172:0/127697727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 0x56115ff42ec0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f105c098180 tx=0x7f105c0d5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b4a5e60322904 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.022+0000 7f104b7fe640 1 -- 172.21.15.172:0/127697727 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c09b680 con 0x56115ff469a0 2024-09-10T08:57:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.022+0000 7f104b7fe640 1 -- 172.21.15.172:0/127697727 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f105c005680 con 0x56115ff469a0 2024-09-10T08:57:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.022+0000 7f104b7fe640 1 -- 172.21.15.172:0/127697727 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c0a4990 con 0x56115ff469a0 2024-09-10T08:57:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.023+0000 7f106ac65740 1 -- 172.21.15.172:0/127697727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 msgr2=0x56115ff42ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.023+0000 7f106ac65740 1 --2- 172.21.15.172:0/127697727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 0x56115ff42ec0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f105c098180 tx=0x7f105c0d5530 comp rx=0 tx=0).stop 2024-09-10T08:57:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.023+0000 7f106ac65740 1 -- 172.21.15.172:0/127697727 shutdown_connections 2024-09-10T08:57:43.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.023+0000 7f106ac65740 1 --2- 172.21.15.172:0/127697727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff4c370 0x56115ff43400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.023+0000 7f106ac65740 1 --2- 172.21.15.172:0/127697727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff472e0 0x56115ff58070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.023+0000 7f106ac65740 1 --2- 172.21.15.172:0/127697727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 0x56115ff42ec0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.023+0000 7f106ac65740 1 -- 172.21.15.172:0/127697727 >> 172.21.15.172:0/127697727 conn(0x56115ff42010 msgr2=0x56115ff50b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.023+0000 7f106ac65740 1 -- 172.21.15.172:0/127697727 shutdown_connections 2024-09-10T08:57:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.023+0000 7f106ac65740 1 -- 172.21.15.172:0/127697727 wait complete. 2024-09-10T08:57:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.023+0000 7f106ac65740 1 -- start start 2024-09-10T08:57:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.023+0000 7f106ac65740 1 Processor -- start 2024-09-10T08:57:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f106ac65740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 0x56115ff55740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f106ac65740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff472e0 0x56115ff55c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f106ac65740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff4c370 0x56115ff56c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f106ac65740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f10601aba50 con 0x56115ff469a0 2024-09-10T08:57:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f106ac65740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10601ab380 con 0x56115ff472e0 2024-09-10T08:57:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f106ac65740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f10601a8560 con 0x56115ff4c370 2024-09-10T08:57:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f106ac65740 10 client.0 _ll_register_callbacks cb 0x7f105c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f1065b8b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 0x56115ff55740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f1065b8b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 0x56115ff55740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60220/0 (socket says 172.21.15.172:60220) 2024-09-10T08:57:43.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f1065b8b640 1 -- 172.21.15.172:0/3001759249 learned_addr learned my addr 172.21.15.172:0/3001759249 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f106638c640 1 --2- 172.21.15.172:0/3001759249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff4c370 0x56115ff56c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f1065b8b640 1 -- 172.21.15.172:0/3001759249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff4c370 msgr2=0x56115ff56c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f1065b8b640 1 --2- 172.21.15.172:0/3001759249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff4c370 0x56115ff56c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f1065b8b640 1 -- 172.21.15.172:0/3001759249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff472e0 msgr2=0x56115ff55c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f1065b8b640 1 --2- 172.21.15.172:0/3001759249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff472e0 0x56115ff55c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f1065b8b640 1 -- 172.21.15.172:0/3001759249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56115ff572a0 con 0x56115ff469a0 2024-09-10T08:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f106638c640 1 --2- 172.21.15.172:0/3001759249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff4c370 0x56115ff56c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f1065b8b640 1 --2- 172.21.15.172:0/3001759249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 0x56115ff55740 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f105c0a4460 tx=0x7f105c09b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708ea0290531028d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.024+0000 7f10497fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.025+0000 7f10497fa640 1 -- 172.21.15.172:0/3001759249 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c0ba070 con 0x56115ff469a0 2024-09-10T08:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.025+0000 7f10497fa640 1 -- 172.21.15.172:0/3001759249 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f105c0cd0d0 con 0x56115ff469a0 2024-09-10T08:57:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.025+0000 7f106ac65740 1 -- 172.21.15.172:0/3001759249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f105c005680 con 0x56115ff469a0 2024-09-10T08:57:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.025+0000 7f10497fa640 1 -- 172.21.15.172:0/3001759249 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f105c0052f0 con 0x56115ff469a0 2024-09-10T08:57:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.025+0000 7f106ac65740 10 client.18849 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.025+0000 7f106ac65740 1 -- 172.21.15.172:0/3001759249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56115ffc74d0 con 0x56115ff469a0 2024-09-10T08:57:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.025+0000 7f10497fa640 1 -- 172.21.15.172:0/3001759249 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f105c005680 con 0x56115ff469a0 2024-09-10T08:57:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.025+0000 7f10497fa640 1 -- 172.21.15.172:0/3001759249 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f105c097a70 con 0x56115ff469a0 2024-09-10T08:57:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.025+0000 7f106ac65740 10 client.18849 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.025+0000 7f106ac65740 10 client.18849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.025+0000 7f106ac65740 20 client.18849 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.025+0000 7f106ac65740 1 --2- 172.21.15.172:0/3001759249 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56115ffc7d80 0x56115ffca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.025+0000 7f106ac65740 4 client.18849 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:57:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.025+0000 7f106ac65740 1 -- 172.21.15.172:0/3001759249 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56115feb1d70 con 0x56115ffc7d80 2024-09-10T08:57:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.026+0000 7f106538a640 1 --2- 172.21.15.172:0/3001759249 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56115ffc7d80 0x56115ffca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.026+0000 7f10497fa640 10 client.18849 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.026+0000 7f106538a640 1 --2- 172.21.15.172:0/3001759249 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56115ffc7d80 0x56115ffca1a0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a7a34a3b0438038 server_cookie=d9f28ca61e194bae in_seq=0 out_seq=0 2024-09-10T08:57:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.030+0000 7f10497fa640 1 -- 172.21.15.172:0/3001759249 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56115feb1d70 con 0x56115ffc7d80 2024-09-10T08:57:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.030+0000 7f10497fa640 10 client.18849 handle_command_reply: tid=0 2024-09-10T08:57:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.030+0000 7f106ac65740 1 client.18849 shutdown 2024-09-10T08:57:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.030+0000 7f106ac65740 1 -- 172.21.15.172:0/3001759249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 msgr2=0x56115ff55740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.030+0000 7f106ac65740 1 --2- 172.21.15.172:0/3001759249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 0x56115ff55740 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f105c0a4460 tx=0x7f105c09b770 comp rx=0 tx=0).stop 2024-09-10T08:57:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.030+0000 7f106ac65740 1 -- 172.21.15.172:0/3001759249 shutdown_connections 2024-09-10T08:57:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.030+0000 7f106ac65740 1 --2- 172.21.15.172:0/3001759249 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56115ffc7d80 0x56115ffca1a0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.030+0000 7f106ac65740 1 --2- 172.21.15.172:0/3001759249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56115ff4c370 0x56115ff56c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.030+0000 7f10497fa640 0 client.18849 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.030+0000 7f106ac65740 1 --2- 172.21.15.172:0/3001759249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ff472e0 0x56115ff55c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.030+0000 7f106ac65740 1 --2- 172.21.15.172:0/3001759249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56115ff469a0 0x56115ff55740 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.030+0000 7f106ac65740 1 -- 172.21.15.172:0/3001759249 >> 172.21.15.172:0/3001759249 conn(0x56115ff42010 msgr2=0x56115ff409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.030+0000 7f106ac65740 1 -- 172.21.15.172:0/3001759249 shutdown_connections 2024-09-10T08:57:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.030+0000 7f106ac65740 1 -- 172.21.15.172:0/3001759249 wait complete. 2024-09-10T08:57:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.030+0000 7f106ac65740 20 client.18849 trim_cache size 0 max 16384 2024-09-10T08:57:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:57:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:57:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:57:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:57:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.031+0000 7f1067def640 1 -- 172.21.15.172:0/4086030669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f104c0a2c30 msgr2=0x7f104c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.031+0000 7f1067def640 1 --2- 172.21.15.172:0/4086030669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f104c0a2c30 0x7f104c0a50f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1058005e30 tx=0x7f1058005ef0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.031+0000 7f1067def640 1 -- 172.21.15.172:0/4086030669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1060102630 msgr2=0x7f1060197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.031+0000 7f1067def640 1 --2- 172.21.15.172:0/4086030669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1060102630 0x7f1060197d60 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f105c00e9f0 tx=0x7f105c00edf0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.031+0000 7f1067def640 1 -- 172.21.15.172:0/4086030669 shutdown_connections 2024-09-10T08:57:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.031+0000 7f1067def640 1 --2- 172.21.15.172:0/4086030669 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f104c0a2c30 0x7f104c0a50f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.031+0000 7f1067def640 1 --2- 172.21.15.172:0/4086030669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1060107890 0x7f106019f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.031+0000 7f1067def640 1 --2- 172.21.15.172:0/4086030669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1060104f60 0x7f10601982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.031+0000 7f1067def640 1 --2- 172.21.15.172:0/4086030669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1060102630 0x7f1060197d60 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.031+0000 7f1067def640 1 -- 172.21.15.172:0/4086030669 >> 172.21.15.172:0/4086030669 conn(0x7f10600fc1b0 msgr2=0x7f1060102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.031+0000 7f1067def640 1 -- 172.21.15.172:0/4086030669 shutdown_connections 2024-09-10T08:57:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:43.031+0000 7f1067def640 1 -- 172.21.15.172:0/4086030669 wait complete. 2024-09-10T08:57:43.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:43 smithi174 ceph-mon[34068]: pgmap v695: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8.8 MiB/s wr, 12 op/s 2024-09-10T08:57:43.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:43 smithi191 ceph-mon[35314]: pgmap v695: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8.8 MiB/s wr, 12 op/s 2024-09-10T08:57:43.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:43 smithi172 ceph-mon[30854]: pgmap v695: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8.8 MiB/s wr, 12 op/s 2024-09-10T08:57:45.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:45 smithi174 ceph-mon[34068]: pgmap v696: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 18 op/s 2024-09-10T08:57:45.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:45 smithi191 ceph-mon[35314]: pgmap v696: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 18 op/s 2024-09-10T08:57:45.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:45 smithi172 ceph-mon[30854]: pgmap v696: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 16 MiB/s wr, 18 op/s 2024-09-10T08:57:47.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:47 smithi174 ceph-mon[34068]: pgmap v697: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 21 MiB/s wr, 17 op/s 2024-09-10T08:57:47.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:47 smithi191 ceph-mon[35314]: pgmap v697: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 21 MiB/s wr, 17 op/s 2024-09-10T08:57:47.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:47 smithi172 ceph-mon[30854]: pgmap v697: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 21 MiB/s wr, 17 op/s 2024-09-10T08:57:49.032 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:57:49.332 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:57:49.735 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:49 smithi172 ceph-mon[30854]: pgmap v698: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-09-10T08:57:49.741 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:49 smithi174 ceph-mon[34068]: pgmap v698: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-09-10T08:57:49.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:49 smithi191 ceph-mon[35314]: pgmap v698: 129 pgs: 129 active+clean; 11 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 MiB/s wr, 20 op/s 2024-09-10T08:57:50.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.622+0000 7f3fc5498640 1 -- 172.21.15.172:0/2226045191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fc0106b80 msgr2=0x7f3fc0106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:50.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.622+0000 7f3fc5498640 1 --2- 172.21.15.172:0/2226045191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fc0106b80 0x7f3fc0106f60 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009900 tx=0x7f3fb002a410 comp rx=0 tx=0).stop 2024-09-10T08:57:50.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.622+0000 7f3fc5498640 1 -- 172.21.15.172:0/2226045191 shutdown_connections 2024-09-10T08:57:50.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.622+0000 7f3fc5498640 1 --2- 172.21.15.172:0/2226045191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3fc0100aa0 0x7f3fc010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.622+0000 7f3fc5498640 1 --2- 172.21.15.172:0/2226045191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fc0100100 0x7f3fc0100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.622+0000 7f3fc5498640 1 --2- 172.21.15.172:0/2226045191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fc0106b80 0x7f3fc0106f60 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.622+0000 7f3fc5498640 1 -- 172.21.15.172:0/2226045191 >> 172.21.15.172:0/2226045191 conn(0x7f3fc0077230 msgr2=0x7f3fc00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:50.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.622+0000 7f3fc5498640 1 -- 172.21.15.172:0/2226045191 shutdown_connections 2024-09-10T08:57:50.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.622+0000 7f3fc5498640 1 -- 172.21.15.172:0/2226045191 wait complete. 2024-09-10T08:57:50.625 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.622+0000 7f3fc5498640 1 Processor -- start 2024-09-10T08:57:50.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.623+0000 7f3fc5498640 1 -- start start 2024-09-10T08:57:50.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.623+0000 7f3fc5498640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fc0100100 0x7f3fc0198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.626 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.623+0000 7f3fc5498640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fc0100aa0 0x7f3fc0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.623+0000 7f3fc5498640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3fc0106b80 0x7f3fc019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.623+0000 7f3fc5498640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc0111fc0 con 0x7f3fc0100100 2024-09-10T08:57:50.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.623+0000 7f3fc5498640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc0111e40 con 0x7f3fc0100aa0 2024-09-10T08:57:50.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.623+0000 7f3fc5498640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc0112140 con 0x7f3fc0106b80 2024-09-10T08:57:50.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.623+0000 7f3fbeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fc0100100 0x7f3fc0198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.623+0000 7f3fbeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fc0100100 0x7f3fc0198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38642/0 (socket says 172.21.15.172:38642) 2024-09-10T08:57:50.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.623+0000 7f3fbeffd640 1 -- 172.21.15.172:0/976247659 learned_addr learned my addr 172.21.15.172:0/976247659 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:50.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.623+0000 7f3fbf7fe640 1 --2- 172.21.15.172:0/976247659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3fc0106b80 0x7f3fc019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.623+0000 7f3fbe7fc640 1 --2- 172.21.15.172:0/976247659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fc0100aa0 0x7f3fc0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.624+0000 7f3fbeffd640 1 -- 172.21.15.172:0/976247659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3fc0106b80 msgr2=0x7f3fc019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:50.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.624+0000 7f3fbeffd640 1 --2- 172.21.15.172:0/976247659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3fc0106b80 0x7f3fc019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.624+0000 7f3fbeffd640 1 -- 172.21.15.172:0/976247659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fc0100aa0 msgr2=0x7f3fc0198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:50.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.624+0000 7f3fbeffd640 1 --2- 172.21.15.172:0/976247659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fc0100aa0 0x7f3fc0198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.624+0000 7f3fbeffd640 1 -- 172.21.15.172:0/976247659 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc019d4b0 con 0x7f3fc0100100 2024-09-10T08:57:50.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.624+0000 7f3fbeffd640 1 --2- 172.21.15.172:0/976247659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fc0100100 0x7f3fc0198500 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0006010 tx=0x7f3fb0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:50.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.624+0000 7f3fa7fff640 1 -- 172.21.15.172:0/976247659 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb003f030 con 0x7f3fc0100100 2024-09-10T08:57:50.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.624+0000 7f3fa7fff640 1 -- 172.21.15.172:0/976247659 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fb00035f0 con 0x7f3fc0100100 2024-09-10T08:57:50.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.624+0000 7f3fa7fff640 1 -- 172.21.15.172:0/976247659 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0035070 con 0x7f3fc0100100 2024-09-10T08:57:50.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.624+0000 7f3fc5498640 1 -- 172.21.15.172:0/976247659 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc019d740 con 0x7f3fc0100100 2024-09-10T08:57:50.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.624+0000 7f3fc5498640 1 -- 172.21.15.172:0/976247659 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc01a5080 con 0x7f3fc0100100 2024-09-10T08:57:50.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.627+0000 7f3fa7fff640 1 -- 172.21.15.172:0/976247659 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3fb0003b10 con 0x7f3fc0100100 2024-09-10T08:57:50.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.628+0000 7f3fa7fff640 1 --2- 172.21.15.172:0/976247659 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3fa80a2b80 0x7f3fa80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.628+0000 7f3fbe7fc640 1 --2- 172.21.15.172:0/976247659 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3fa80a2b80 0x7f3fa80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.628+0000 7f3fa7fff640 1 -- 172.21.15.172:0/976247659 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3fb00eff20 con 0x7f3fc0100100 2024-09-10T08:57:50.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.629+0000 7f3fbe7fc640 1 --2- 172.21.15.172:0/976247659 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3fa80a2b80 0x7f3fa80a5040 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00ff7f0 tx=0x7f3fb4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.686+0000 7f3fc7871740 1 Processor -- start 2024-09-10T08:57:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.686+0000 7f3fc7871740 1 -- start start 2024-09-10T08:57:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.686+0000 7f3fc7871740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d309034a0 0x561d30903880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.686+0000 7f3fc7871740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d30903dc0 0x561d3090e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.686+0000 7f3fc7871740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d3090e680 0x561d30910a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.686+0000 7f3fc7871740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561d308692e0 con 0x561d30903dc0 2024-09-10T08:57:50.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.686+0000 7f3fc7871740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561d30810ce0 con 0x561d3090e680 2024-09-10T08:57:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.686+0000 7f3fc7871740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561d308652a0 con 0x561d309034a0 2024-09-10T08:57:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.686+0000 7f3fbe7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d30903dc0 0x561d3090e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.686+0000 7f3fbe7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d30903dc0 0x561d3090e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38644/0 (socket says 172.21.15.172:38644) 2024-09-10T08:57:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.686+0000 7f3fbe7fc640 1 -- 172.21.15.172:0/602454291 learned_addr learned my addr 172.21.15.172:0/602454291 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.686+0000 7f3fbf7fe640 1 --2- 172.21.15.172:0/602454291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d3090e680 0x561d30910a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.686+0000 7f3fbeffd640 1 --2- 172.21.15.172:0/602454291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d309034a0 0x561d30903880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.687+0000 7f3fbe7fc640 1 -- 172.21.15.172:0/602454291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d309034a0 msgr2=0x561d30903880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.687+0000 7f3fbe7fc640 1 --2- 172.21.15.172:0/602454291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d309034a0 0x561d30903880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.687+0000 7f3fbe7fc640 1 -- 172.21.15.172:0/602454291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d3090e680 msgr2=0x561d30910a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.687+0000 7f3fbe7fc640 1 --2- 172.21.15.172:0/602454291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d3090e680 0x561d30910a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.687+0000 7f3fbe7fc640 1 -- 172.21.15.172:0/602454291 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d307f5440 con 0x561d30903dc0 2024-09-10T08:57:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.687+0000 7f3fbe7fc640 1 --2- 172.21.15.172:0/602454291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d30903dc0 0x561d3090e140 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400d810 tx=0x7f3fb4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9aa3b99c7ce654 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:50.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.687+0000 7f3fa57fa640 1 -- 172.21.15.172:0/602454291 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4024070 con 0x561d30903dc0 2024-09-10T08:57:50.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.687+0000 7f3fa57fa640 1 -- 172.21.15.172:0/602454291 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fb401f400 con 0x561d30903dc0 2024-09-10T08:57:50.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.687+0000 7f3fa57fa640 1 -- 172.21.15.172:0/602454291 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4014e60 con 0x561d30903dc0 2024-09-10T08:57:50.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.688+0000 7f3fc7871740 1 -- 172.21.15.172:0/602454291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d30903dc0 msgr2=0x561d3090e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:50.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.688+0000 7f3fc7871740 1 --2- 172.21.15.172:0/602454291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d30903dc0 0x561d3090e140 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400d810 tx=0x7f3fb4010000 comp rx=0 tx=0).stop 2024-09-10T08:57:50.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.688+0000 7f3fc7871740 1 -- 172.21.15.172:0/602454291 shutdown_connections 2024-09-10T08:57:50.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.688+0000 7f3fc7871740 1 --2- 172.21.15.172:0/602454291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d3090e680 0x561d30910a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.688+0000 7f3fc7871740 1 --2- 172.21.15.172:0/602454291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d30903dc0 0x561d3090e140 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.688+0000 7f3fc7871740 1 --2- 172.21.15.172:0/602454291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d309034a0 0x561d30903880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.688+0000 7f3fc7871740 1 -- 172.21.15.172:0/602454291 >> 172.21.15.172:0/602454291 conn(0x561d308ffa30 msgr2=0x561d308ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:50.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.688+0000 7f3fc7871740 1 -- 172.21.15.172:0/602454291 shutdown_connections 2024-09-10T08:57:50.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.688+0000 7f3fc7871740 1 -- 172.21.15.172:0/602454291 wait complete. 2024-09-10T08:57:50.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.688+0000 7f3fc7871740 1 -- start start 2024-09-10T08:57:50.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fc7871740 1 Processor -- start 2024-09-10T08:57:50.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fc7871740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d309034a0 0x561d3090e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fc7871740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d30903dc0 0x561d3090b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fc7871740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d3090e680 0x561d3090b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fc7871740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561d30911250 con 0x561d309034a0 2024-09-10T08:57:50.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fc7871740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561d30824050 con 0x561d3090e680 2024-09-10T08:57:50.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fc7871740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561d309113d0 con 0x561d30903dc0 2024-09-10T08:57:50.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fc7871740 10 client.0 _ll_register_callbacks cb 0x561d30864160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:50.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fbeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d309034a0 0x561d3090e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fbeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d309034a0 0x561d3090e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38660/0 (socket says 172.21.15.172:38660) 2024-09-10T08:57:50.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fbeffd640 1 -- 172.21.15.172:0/4199414399 learned_addr learned my addr 172.21.15.172:0/4199414399 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:50.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fbf7fe640 1 --2- 172.21.15.172:0/4199414399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d3090e680 0x561d3090b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fbe7fc640 1 --2- 172.21.15.172:0/4199414399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d30903dc0 0x561d3090b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fbeffd640 1 -- 172.21.15.172:0/4199414399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d30903dc0 msgr2=0x561d3090b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:50.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fbeffd640 1 --2- 172.21.15.172:0/4199414399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d30903dc0 0x561d3090b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fbeffd640 1 -- 172.21.15.172:0/4199414399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d3090e680 msgr2=0x561d3090b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:50.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fbeffd640 1 --2- 172.21.15.172:0/4199414399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d3090e680 0x561d3090b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fbeffd640 1 -- 172.21.15.172:0/4199414399 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d305da3c0 con 0x561d309034a0 2024-09-10T08:57:50.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.689+0000 7f3fbe7fc640 1 --2- 172.21.15.172:0/4199414399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d30903dc0 0x561d3090b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:57:50.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.690+0000 7f3fbeffd640 1 --2- 172.21.15.172:0/4199414399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d309034a0 0x561d3090e040 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00ed6c0 tx=0x7f3fb00bcb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d975b66e186705e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:50.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.690+0000 7f3fa2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:50.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.690+0000 7f3fa2ff5640 1 -- 172.21.15.172:0/4199414399 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00d5060 con 0x561d309034a0 2024-09-10T08:57:50.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.690+0000 7f3fc7871740 1 -- 172.21.15.172:0/4199414399 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fb401f400 con 0x561d309034a0 2024-09-10T08:57:50.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.690+0000 7f3fa2ff5640 1 -- 172.21.15.172:0/4199414399 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3fb00d5200 con 0x561d309034a0 2024-09-10T08:57:50.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.690+0000 7f3fa2ff5640 1 -- 172.21.15.172:0/4199414399 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3fb0049040 con 0x561d309034a0 2024-09-10T08:57:50.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.690+0000 7f3fc7871740 10 client.18861 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:50.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.690+0000 7f3fc7871740 1 -- 172.21.15.172:0/4199414399 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561d3091cc70 con 0x561d309034a0 2024-09-10T08:57:50.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.690+0000 7f3fa2ff5640 1 -- 172.21.15.172:0/4199414399 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fb401f400 con 0x561d309034a0 2024-09-10T08:57:50.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.690+0000 7f3fa2ff5640 1 -- 172.21.15.172:0/4199414399 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f3fb00bc680 con 0x561d309034a0 2024-09-10T08:57:50.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.690+0000 7f3fc7871740 10 client.18861 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:50.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.690+0000 7f3fc7871740 10 client.18861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:50.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.690+0000 7f3fc7871740 20 client.18861 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.690+0000 7f3fc7871740 1 --2- 172.21.15.172:0/4199414399 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561d3091e060 0x561d30920490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.690+0000 7f3fc7871740 4 client.18861 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:57:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.690+0000 7f3fc7871740 1 -- 172.21.15.172:0/4199414399 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d30876850 con 0x561d3091e060 2024-09-10T08:57:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.691+0000 7f3fbe7fc640 1 --2- 172.21.15.172:0/4199414399 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561d3091e060 0x561d30920490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.691+0000 7f3fa2ff5640 10 client.18861 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.691+0000 7f3fbe7fc640 1 --2- 172.21.15.172:0/4199414399 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561d3091e060 0x561d30920490 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcb287eba72c8117 server_cookie=78639ff247a71755 in_seq=0 out_seq=0 2024-09-10T08:57:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.692+0000 7f3fa2ff5640 1 -- 172.21.15.172:0/4199414399 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d30876850 con 0x561d3091e060 2024-09-10T08:57:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.692+0000 7f3fa2ff5640 10 client.18861 handle_command_reply: tid=0 2024-09-10T08:57:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.692+0000 7f3fc7871740 1 client.18861 shutdown 2024-09-10T08:57:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.692+0000 7f3fc7871740 1 -- 172.21.15.172:0/4199414399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d309034a0 msgr2=0x561d3090e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.692+0000 7f3fc7871740 1 --2- 172.21.15.172:0/4199414399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d309034a0 0x561d3090e040 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00ed6c0 tx=0x7f3fb00bcb20 comp rx=0 tx=0).stop 2024-09-10T08:57:50.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.692+0000 7f3fc7871740 1 -- 172.21.15.172:0/4199414399 shutdown_connections 2024-09-10T08:57:50.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.692+0000 7f3fc7871740 1 --2- 172.21.15.172:0/4199414399 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561d3091e060 0x561d30920490 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.692+0000 7f3fc7871740 1 --2- 172.21.15.172:0/4199414399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d3090e680 0x561d3090b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.692+0000 7f3fc7871740 1 --2- 172.21.15.172:0/4199414399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d30903dc0 0x561d3090b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.692+0000 7f3fc7871740 1 --2- 172.21.15.172:0/4199414399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d309034a0 0x561d3090e040 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.692+0000 7f3fc7871740 1 -- 172.21.15.172:0/4199414399 >> 172.21.15.172:0/4199414399 conn(0x561d308ffa30 msgr2=0x561d30905710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:50.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.692+0000 7f3fa2ff5640 0 client.18861 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:50.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.692+0000 7f3fc7871740 1 -- 172.21.15.172:0/4199414399 shutdown_connections 2024-09-10T08:57:50.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.692+0000 7f3fc7871740 1 -- 172.21.15.172:0/4199414399 wait complete. 2024-09-10T08:57:50.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.692+0000 7f3fc7871740 20 client.18861 trim_cache size 0 max 16384 2024-09-10T08:57:50.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.709+0000 7f3fc7871740 1 Processor -- start 2024-09-10T08:57:50.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.709+0000 7f3fc7871740 1 -- start start 2024-09-10T08:57:50.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.709+0000 7f3fc7871740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d309034a0 0x561d308fea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.709+0000 7f3fc7871740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d30903dc0 0x561d30925d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.709+0000 7f3fc7871740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d3090e680 0x561d309262b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.709+0000 7f3fc7871740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc01b0e70 con 0x561d309034a0 2024-09-10T08:57:50.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.709+0000 7f3fc7871740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc01b2a10 con 0x561d3090e680 2024-09-10T08:57:50.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.709+0000 7f3fc7871740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc01b1540 con 0x561d30903dc0 2024-09-10T08:57:50.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.709+0000 7f3fbeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d309034a0 0x561d308fea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.710+0000 7f3fbeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d309034a0 0x561d308fea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38664/0 (socket says 172.21.15.172:38664) 2024-09-10T08:57:50.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.710+0000 7f3fbeffd640 1 -- 172.21.15.172:0/3392678691 learned_addr learned my addr 172.21.15.172:0/3392678691 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:50.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.710+0000 7f3fbf7fe640 1 --2- 172.21.15.172:0/3392678691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d3090e680 0x561d309262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.710+0000 7f3fbe7fc640 1 --2- 172.21.15.172:0/3392678691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d30903dc0 0x561d30925d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.710+0000 7f3fbeffd640 1 -- 172.21.15.172:0/3392678691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d30903dc0 msgr2=0x561d30925d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:50.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.710+0000 7f3fbeffd640 1 --2- 172.21.15.172:0/3392678691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d30903dc0 0x561d30925d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.710+0000 7f3fbeffd640 1 -- 172.21.15.172:0/3392678691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d3090e680 msgr2=0x561d309262b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:50.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.710+0000 7f3fbeffd640 1 --2- 172.21.15.172:0/3392678691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d3090e680 0x561d309262b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.710+0000 7f3fbeffd640 1 -- 172.21.15.172:0/3392678691 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d3090bc10 con 0x561d309034a0 2024-09-10T08:57:50.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.710+0000 7f3fbeffd640 1 --2- 172.21.15.172:0/3392678691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d309034a0 0x561d308fea60 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00ba6d0 tx=0x7f3fb00ba930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a032d76ed2f282f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:50.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.710+0000 7f3fa4ff9640 1 -- 172.21.15.172:0/3392678691 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00c5090 con 0x561d309034a0 2024-09-10T08:57:50.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.710+0000 7f3fa4ff9640 1 -- 172.21.15.172:0/3392678691 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fb0031a20 con 0x561d309034a0 2024-09-10T08:57:50.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.710+0000 7f3fa4ff9640 1 -- 172.21.15.172:0/3392678691 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00edb80 con 0x561d309034a0 2024-09-10T08:57:50.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.711+0000 7f3fc7871740 1 -- 172.21.15.172:0/3392678691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d309034a0 msgr2=0x561d308fea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:50.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.711+0000 7f3fc7871740 1 --2- 172.21.15.172:0/3392678691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d309034a0 0x561d308fea60 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00ba6d0 tx=0x7f3fb00ba930 comp rx=0 tx=0).stop 2024-09-10T08:57:50.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.711+0000 7f3fc7871740 1 -- 172.21.15.172:0/3392678691 shutdown_connections 2024-09-10T08:57:50.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.711+0000 7f3fc7871740 1 --2- 172.21.15.172:0/3392678691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d3090e680 0x561d309262b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.711+0000 7f3fc7871740 1 --2- 172.21.15.172:0/3392678691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d30903dc0 0x561d30925d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.711+0000 7f3fc7871740 1 --2- 172.21.15.172:0/3392678691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d309034a0 0x561d308fea60 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.711+0000 7f3fc7871740 1 -- 172.21.15.172:0/3392678691 >> 172.21.15.172:0/3392678691 conn(0x561d308ffa30 msgr2=0x561d308fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:50.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.711+0000 7f3fc7871740 1 -- 172.21.15.172:0/3392678691 shutdown_connections 2024-09-10T08:57:50.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.711+0000 7f3fc7871740 1 -- 172.21.15.172:0/3392678691 wait complete. 2024-09-10T08:57:50.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.712+0000 7f3fc7871740 1 -- start start 2024-09-10T08:57:50.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.712+0000 7f3fc7871740 1 Processor -- start 2024-09-10T08:57:50.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.712+0000 7f3fc7871740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d309034a0 0x561d30914ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.712+0000 7f3fc7871740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d30903dc0 0x561d30915010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.712+0000 7f3fc7871740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d3090e680 0x561d30987d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.712+0000 7f3fc7871740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc01b2340 con 0x561d30903dc0 2024-09-10T08:57:50.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.712+0000 7f3fc7871740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc01b1c70 con 0x561d309034a0 2024-09-10T08:57:50.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.712+0000 7f3fc7871740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc01aee50 con 0x561d3090e680 2024-09-10T08:57:50.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.712+0000 7f3fc7871740 10 client.0 _ll_register_callbacks cb 0x7f3fb00b8770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:50.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.712+0000 7f3fbe7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d30903dc0 0x561d30915010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.712+0000 7f3fbe7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d30903dc0 0x561d30915010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38672/0 (socket says 172.21.15.172:38672) 2024-09-10T08:57:50.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.712+0000 7f3fbe7fc640 1 -- 172.21.15.172:0/274966651 learned_addr learned my addr 172.21.15.172:0/274966651 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:50.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.712+0000 7f3fbeffd640 1 --2- 172.21.15.172:0/274966651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d309034a0 0x561d30914ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.712+0000 7f3fbf7fe640 1 --2- 172.21.15.172:0/274966651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d3090e680 0x561d30987d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.712+0000 7f3fbe7fc640 1 -- 172.21.15.172:0/274966651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d3090e680 msgr2=0x561d30987d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:50.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.712+0000 7f3fbe7fc640 1 --2- 172.21.15.172:0/274966651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d3090e680 0x561d30987d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.713+0000 7f3fbe7fc640 1 -- 172.21.15.172:0/274966651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d309034a0 msgr2=0x561d30914ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:50.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.713+0000 7f3fbe7fc640 1 --2- 172.21.15.172:0/274966651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d309034a0 0x561d30914ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.713+0000 7f3fbe7fc640 1 -- 172.21.15.172:0/274966651 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d30988290 con 0x561d30903dc0 2024-09-10T08:57:50.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.713+0000 7f3fbe7fc640 1 --2- 172.21.15.172:0/274966651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d30903dc0 0x561d30915010 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400f010 tx=0x7f3fb401fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef364d94ef697394 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:50.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.713+0000 7f3fa2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.713+0000 7f3fa2ff5640 1 -- 172.21.15.172:0/274966651 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4024040 con 0x561d30903dc0 2024-09-10T08:57:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.713+0000 7f3fa2ff5640 1 -- 172.21.15.172:0/274966651 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3fb40149b0 con 0x561d30903dc0 2024-09-10T08:57:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.713+0000 7f3fc7871740 1 -- 172.21.15.172:0/274966651 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fb0031a20 con 0x561d30903dc0 2024-09-10T08:57:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.713+0000 7f3fa2ff5640 1 -- 172.21.15.172:0/274966651 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3fb4014bb0 con 0x561d30903dc0 2024-09-10T08:57:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.713+0000 7f3fc7871740 10 client.18867 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.713+0000 7f3fc7871740 1 -- 172.21.15.172:0/274966651 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561d309895a0 con 0x561d30903dc0 2024-09-10T08:57:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.713+0000 7f3fa2ff5640 1 -- 172.21.15.172:0/274966651 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3fb0031a20 con 0x561d30903dc0 2024-09-10T08:57:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.714+0000 7f3fa2ff5640 1 -- 172.21.15.172:0/274966651 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f3fb4013070 con 0x561d30903dc0 2024-09-10T08:57:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.714+0000 7f3fc7871740 10 client.18867 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.714+0000 7f3fc7871740 10 client.18867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.714+0000 7f3fc7871740 20 client.18867 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:50.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.714+0000 7f3fc7871740 1 --2- 172.21.15.172:0/274966651 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561d30989ad0 0x561d3098bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.714+0000 7f3fc7871740 4 client.18867 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:57:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.714+0000 7f3fc7871740 1 -- 172.21.15.172:0/274966651 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561d30870d60 con 0x561d30989ad0 2024-09-10T08:57:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.714+0000 7f3fbeffd640 1 --2- 172.21.15.172:0/274966651 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561d30989ad0 0x561d3098bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.714+0000 7f3fbeffd640 1 --2- 172.21.15.172:0/274966651 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561d30989ad0 0x561d3098bec0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8f285c0f0902ddf server_cookie=3a3bc273e9c85090 in_seq=0 out_seq=0 2024-09-10T08:57:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.714+0000 7f3fa2ff5640 10 client.18867 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fa2ff5640 1 -- 172.21.15.172:0/274966651 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561d30870d60 con 0x561d30989ad0 2024-09-10T08:57:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fa2ff5640 10 client.18867 handle_command_reply: tid=0 2024-09-10T08:57:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fc7871740 1 client.18867 shutdown 2024-09-10T08:57:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fc7871740 1 -- 172.21.15.172:0/274966651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d30903dc0 msgr2=0x561d30915010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fc7871740 1 --2- 172.21.15.172:0/274966651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d30903dc0 0x561d30915010 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400f010 tx=0x7f3fb401fd60 comp rx=0 tx=0).stop 2024-09-10T08:57:50.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fc7871740 1 -- 172.21.15.172:0/274966651 shutdown_connections 2024-09-10T08:57:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fc7871740 1 --2- 172.21.15.172:0/274966651 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561d30989ad0 0x561d3098bec0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fc7871740 1 --2- 172.21.15.172:0/274966651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561d3090e680 0x561d30987d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fc7871740 1 --2- 172.21.15.172:0/274966651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561d30903dc0 0x561d30915010 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fc7871740 1 --2- 172.21.15.172:0/274966651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d309034a0 0x561d30914ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fc7871740 1 -- 172.21.15.172:0/274966651 >> 172.21.15.172:0/274966651 conn(0x561d308ffa30 msgr2=0x561d308feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fa2ff5640 0 client.18867 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fc7871740 1 -- 172.21.15.172:0/274966651 shutdown_connections 2024-09-10T08:57:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fc7871740 1 -- 172.21.15.172:0/274966651 wait complete. 2024-09-10T08:57:50.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fc7871740 20 client.18867 trim_cache size 0 max 16384 2024-09-10T08:57:50.722 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:57:50.723 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T08:57:50.723 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T08:57:50.723 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:57:50.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fc5498640 1 -- 172.21.15.172:0/976247659 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3fa80a2b80 msgr2=0x7f3fa80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:50.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fc5498640 1 --2- 172.21.15.172:0/976247659 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3fa80a2b80 0x7f3fa80a5040 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00ff7f0 tx=0x7f3fb4008040 comp rx=0 tx=0).stop 2024-09-10T08:57:50.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fc5498640 1 -- 172.21.15.172:0/976247659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fc0100100 msgr2=0x7f3fc0198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:50.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.715+0000 7f3fc5498640 1 --2- 172.21.15.172:0/976247659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fc0100100 0x7f3fc0198500 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0006010 tx=0x7f3fb0004270 comp rx=0 tx=0).stop 2024-09-10T08:57:50.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.716+0000 7f3fc5498640 1 -- 172.21.15.172:0/976247659 shutdown_connections 2024-09-10T08:57:50.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.716+0000 7f3fc5498640 1 --2- 172.21.15.172:0/976247659 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3fa80a2b80 0x7f3fa80a5040 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.716+0000 7f3fc5498640 1 --2- 172.21.15.172:0/976247659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3fc0106b80 0x7f3fc019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.716+0000 7f3fc5498640 1 --2- 172.21.15.172:0/976247659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fc0100aa0 0x7f3fc0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.716+0000 7f3fc5498640 1 --2- 172.21.15.172:0/976247659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fc0100100 0x7f3fc0198500 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:50.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.716+0000 7f3fc5498640 1 -- 172.21.15.172:0/976247659 >> 172.21.15.172:0/976247659 conn(0x7f3fc0077230 msgr2=0x7f3fc0104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:50.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.716+0000 7f3fc5498640 1 -- 172.21.15.172:0/976247659 shutdown_connections 2024-09-10T08:57:50.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:50.716+0000 7f3fc5498640 1 -- 172.21.15.172:0/976247659 wait complete. 2024-09-10T08:57:51.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:51 smithi174 ceph-mon[34068]: pgmap v699: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:57:51.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:51 smithi191 ceph-mon[35314]: pgmap v699: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:57:51.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:51 smithi172 ceph-mon[30854]: pgmap v699: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 MiB/s wr, 21 op/s 2024-09-10T08:57:53.584 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:57:53.585 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:57:53.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:53 smithi174 ceph-mon[34068]: pgmap v700: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:57:53.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:53 smithi191 ceph-mon[35314]: pgmap v700: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:57:53.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:53 smithi172 ceph-mon[30854]: pgmap v700: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 20 op/s 2024-09-10T08:57:53.874 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:57:54.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.528+0000 7f024ef3a640 1 -- 172.21.15.172:0/1100431281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0248101410 msgr2=0x7f024810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:54.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.528+0000 7f024ef3a640 1 --2- 172.21.15.172:0/1100431281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0248101410 0x7f024810d940 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f0240009930 tx=0x7f024002a420 comp rx=0 tx=0).stop 2024-09-10T08:57:54.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.528+0000 7f024ef3a640 1 -- 172.21.15.172:0/1100431281 shutdown_connections 2024-09-10T08:57:54.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.528+0000 7f024ef3a640 1 --2- 172.21.15.172:0/1100431281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0248101410 0x7f024810d940 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:54.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.528+0000 7f024ef3a640 1 --2- 172.21.15.172:0/1100431281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0248100a70 0x7f0248100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:54.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.528+0000 7f024ef3a640 1 --2- 172.21.15.172:0/1100431281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0248106a70 0x7f0248106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:54.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.528+0000 7f024ef3a640 1 -- 172.21.15.172:0/1100431281 >> 172.21.15.172:0/1100431281 conn(0x7f02480fc790 msgr2=0x7f02480febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:54.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.528+0000 7f024ef3a640 1 -- 172.21.15.172:0/1100431281 shutdown_connections 2024-09-10T08:57:54.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.528+0000 7f024ef3a640 1 -- 172.21.15.172:0/1100431281 wait complete. 2024-09-10T08:57:54.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.529+0000 7f024ef3a640 1 Processor -- start 2024-09-10T08:57:54.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.529+0000 7f024ef3a640 1 -- start start 2024-09-10T08:57:54.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.529+0000 7f024ef3a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0248100a70 0x7f02481983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:54.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.529+0000 7f024ef3a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0248101410 0x7f0248198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:54.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.529+0000 7f024ef3a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0248106a70 0x7f024819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:54.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.529+0000 7f024ef3a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0248110d40 con 0x7f0248100a70 2024-09-10T08:57:54.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.529+0000 7f024ef3a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0248110bc0 con 0x7f0248101410 2024-09-10T08:57:54.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.529+0000 7f024ef3a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0248110ec0 con 0x7f0248106a70 2024-09-10T08:57:54.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.529+0000 7f024ccd6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0248100a70 0x7f02481983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:54.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.529+0000 7f024ccd6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0248100a70 0x7f02481983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38696/0 (socket says 172.21.15.172:38696) 2024-09-10T08:57:54.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.529+0000 7f024ccd6640 1 -- 172.21.15.172:0/3984594791 learned_addr learned my addr 172.21.15.172:0/3984594791 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:54.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.529+0000 7f023ffff640 1 --2- 172.21.15.172:0/3984594791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0248101410 0x7f0248198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:54.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.530+0000 7f024ccd6640 1 -- 172.21.15.172:0/3984594791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0248106a70 msgr2=0x7f024819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:54.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.530+0000 7f024ccd6640 1 --2- 172.21.15.172:0/3984594791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0248106a70 0x7f024819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:54.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.530+0000 7f024ccd6640 1 -- 172.21.15.172:0/3984594791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0248101410 msgr2=0x7f0248198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:54.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.530+0000 7f024ccd6640 1 --2- 172.21.15.172:0/3984594791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0248101410 0x7f0248198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:54.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.530+0000 7f024ccd6640 1 -- 172.21.15.172:0/3984594791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f024819d370 con 0x7f0248100a70 2024-09-10T08:57:54.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.530+0000 7f024ccd6640 1 --2- 172.21.15.172:0/3984594791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0248100a70 0x7f02481983c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f024400cce0 tx=0x7f024400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:54.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.530+0000 7f023dffb640 1 -- 172.21.15.172:0/3984594791 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244004b20 con 0x7f0248100a70 2024-09-10T08:57:54.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.530+0000 7f024ef3a640 1 -- 172.21.15.172:0/3984594791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f024819d660 con 0x7f0248100a70 2024-09-10T08:57:54.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.530+0000 7f023dffb640 1 -- 172.21.15.172:0/3984594791 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0244004cc0 con 0x7f0248100a70 2024-09-10T08:57:54.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.530+0000 7f023dffb640 1 -- 172.21.15.172:0/3984594791 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024400f6b0 con 0x7f0248100a70 2024-09-10T08:57:54.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.530+0000 7f024ef3a640 1 -- 172.21.15.172:0/3984594791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02481a4f40 con 0x7f0248100a70 2024-09-10T08:57:54.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.531+0000 7f024ef3a640 1 -- 172.21.15.172:0/3984594791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0248106e50 con 0x7f0248100a70 2024-09-10T08:57:54.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.532+0000 7f023dffb640 1 -- 172.21.15.172:0/3984594791 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f024400f910 con 0x7f0248100a70 2024-09-10T08:57:54.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.533+0000 7f023dffb640 1 --2- 172.21.15.172:0/3984594791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f02300a2c00 0x7f02300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:54.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.533+0000 7f023ffff640 1 --2- 172.21.15.172:0/3984594791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f02300a2c00 0x7f02300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:54.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.533+0000 7f023dffb640 1 -- 172.21.15.172:0/3984594791 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f02440cf480 con 0x7f0248100a70 2024-09-10T08:57:54.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.534+0000 7f023ffff640 1 --2- 172.21.15.172:0/3984594791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f02300a2c00 0x7f02300a50c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f02480fed40 tx=0x7f0238008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:54.601 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.537+0000 7f023dffb640 1 -- 172.21.15.172:0/3984594791 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0244014030 con 0x7f0248100a70 2024-09-10T08:57:54.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.794+0000 7f024ef3a640 1 -- 172.21.15.172:0/3984594791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02481a5310 con 0x7f0248100a70 2024-09-10T08:57:54.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.794+0000 7f023dffb640 1 -- 172.21.15.172:0/3984594791 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+12004 (secure 0 0 0) 0x7f024409c080 con 0x7f0248100a70 2024-09-10T08:57:54.797 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:57:54.797 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":39,"btime":"2024-09-10T08:56:43:360461+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:56:42.665159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:57:54.797 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 39 2024-09-10T08:57:54.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.797+0000 7f024ef3a640 1 -- 172.21.15.172:0/3984594791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f02300a2c00 msgr2=0x7f02300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:54.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.797+0000 7f024ef3a640 1 --2- 172.21.15.172:0/3984594791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f02300a2c00 0x7f02300a50c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f02480fed40 tx=0x7f0238008040 comp rx=0 tx=0).stop 2024-09-10T08:57:54.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.797+0000 7f024ef3a640 1 -- 172.21.15.172:0/3984594791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0248100a70 msgr2=0x7f02481983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:54.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.797+0000 7f024ef3a640 1 --2- 172.21.15.172:0/3984594791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0248100a70 0x7f02481983c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f024400cce0 tx=0x7f024400a620 comp rx=0 tx=0).stop 2024-09-10T08:57:54.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.797+0000 7f024ef3a640 1 -- 172.21.15.172:0/3984594791 shutdown_connections 2024-09-10T08:57:54.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.797+0000 7f024ef3a640 1 --2- 172.21.15.172:0/3984594791 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f02300a2c00 0x7f02300a50c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:54.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.797+0000 7f024ef3a640 1 --2- 172.21.15.172:0/3984594791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0248106a70 0x7f024819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:54.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.797+0000 7f024ef3a640 1 --2- 172.21.15.172:0/3984594791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0248101410 0x7f0248198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:54.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.797+0000 7f024ef3a640 1 --2- 172.21.15.172:0/3984594791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0248100a70 0x7f02481983c0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:54.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.797+0000 7f024ef3a640 1 -- 172.21.15.172:0/3984594791 >> 172.21.15.172:0/3984594791 conn(0x7f02480fc790 msgr2=0x7f02480fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:54.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.797+0000 7f024ef3a640 1 -- 172.21.15.172:0/3984594791 shutdown_connections 2024-09-10T08:57:54.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:54.797+0000 7f024ef3a640 1 -- 172.21.15.172:0/3984594791 wait complete. 2024-09-10T08:57:55.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:54 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3984594791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:57:55.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:54 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3984594791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:57:55.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:54 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3984594791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:57:55.581 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T08:57:55.863 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:57:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.123+0000 7fd302b4b640 1 -- 172.21.15.172:0/3854839044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2fc103150 msgr2=0x7fd2fc103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.123+0000 7fd302b4b640 1 --2- 172.21.15.172:0/3854839044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2fc103150 0x7fd2fc103550 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009930 tx=0x7fd2f002a420 comp rx=0 tx=0).stop 2024-09-10T08:57:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.123+0000 7fd302b4b640 1 -- 172.21.15.172:0/3854839044 shutdown_connections 2024-09-10T08:57:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.123+0000 7fd302b4b640 1 --2- 172.21.15.172:0/3854839044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2fc100580 0x7fd2fc10a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.123+0000 7fd302b4b640 1 --2- 172.21.15.172:0/3854839044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2fc0ffbc0 0x7fd2fc100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.123+0000 7fd302b4b640 1 --2- 172.21.15.172:0/3854839044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2fc103150 0x7fd2fc103550 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.123+0000 7fd302b4b640 1 -- 172.21.15.172:0/3854839044 >> 172.21.15.172:0/3854839044 conn(0x7fd2fc0fb930 msgr2=0x7fd2fc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.123+0000 7fd302b4b640 1 -- 172.21.15.172:0/3854839044 shutdown_connections 2024-09-10T08:57:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.123+0000 7fd302b4b640 1 -- 172.21.15.172:0/3854839044 wait complete. 2024-09-10T08:57:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.124+0000 7fd302b4b640 1 Processor -- start 2024-09-10T08:57:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.124+0000 7fd302b4b640 1 -- start start 2024-09-10T08:57:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.124+0000 7fd302b4b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2fc0ffbc0 0x7fd2fc102850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.124+0000 7fd302b4b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2fc100580 0x7fd2fc100e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.124+0000 7fd302b4b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2fc1013d0 0x7fd2fc101880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.124+0000 7fd302b4b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc10d750 con 0x7fd2fc1013d0 2024-09-10T08:57:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.124+0000 7fd302b4b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc10d5d0 con 0x7fd2fc0ffbc0 2024-09-10T08:57:56.128 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.124+0000 7fd302b4b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc10d8d0 con 0x7fd2fc100580 2024-09-10T08:57:56.128 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:55 smithi172 ceph-mon[30854]: pgmap v701: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-09-10T08:57:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.125+0000 7fd3010e8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2fc1013d0 0x7fd2fc101880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.125+0000 7fd3010e8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2fc1013d0 0x7fd2fc101880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38710/0 (socket says 172.21.15.172:38710) 2024-09-10T08:57:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.125+0000 7fd3010e8640 1 -- 172.21.15.172:0/1780677512 learned_addr learned my addr 172.21.15.172:0/1780677512 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.125+0000 7fd2fbfff640 1 --2- 172.21.15.172:0/1780677512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2fc100580 0x7fd2fc100e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.125+0000 7fd3008e7640 1 --2- 172.21.15.172:0/1780677512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2fc0ffbc0 0x7fd2fc102850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.125+0000 7fd3010e8640 1 -- 172.21.15.172:0/1780677512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2fc100580 msgr2=0x7fd2fc100e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.125+0000 7fd3010e8640 1 --2- 172.21.15.172:0/1780677512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2fc100580 0x7fd2fc100e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.125+0000 7fd3010e8640 1 -- 172.21.15.172:0/1780677512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2fc0ffbc0 msgr2=0x7fd2fc102850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.125+0000 7fd3010e8640 1 --2- 172.21.15.172:0/1780677512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2fc0ffbc0 0x7fd2fc102850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.125+0000 7fd3010e8640 1 -- 172.21.15.172:0/1780677512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc1a2220 con 0x7fd2fc1013d0 2024-09-10T08:57:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.126+0000 7fd3010e8640 1 --2- 172.21.15.172:0/1780677512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2fc1013d0 0x7fd2fc101880 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400b970 tx=0x7fd2f400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.126+0000 7fd2f9ffb640 1 -- 172.21.15.172:0/1780677512 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4019070 con 0x7fd2fc1013d0 2024-09-10T08:57:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.126+0000 7fd2f9ffb640 1 -- 172.21.15.172:0/1780677512 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2f4014440 con 0x7fd2fc1013d0 2024-09-10T08:57:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.126+0000 7fd2f9ffb640 1 -- 172.21.15.172:0/1780677512 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f40075d0 con 0x7fd2fc1013d0 2024-09-10T08:57:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.126+0000 7fd302b4b640 1 -- 172.21.15.172:0/1780677512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2fc1a2510 con 0x7fd2fc1013d0 2024-09-10T08:57:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.127+0000 7fd302b4b640 1 -- 172.21.15.172:0/1780677512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2fc1a2a80 con 0x7fd2fc1013d0 2024-09-10T08:57:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.131+0000 7fd2f9ffb640 1 -- 172.21.15.172:0/1780677512 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd2f401c070 con 0x7fd2fc1013d0 2024-09-10T08:57:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.131+0000 7fd3059c3740 1 Processor -- start 2024-09-10T08:57:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.131+0000 7fd3059c3740 1 -- start start 2024-09-10T08:57:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd3059c3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934ba8fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd3059c3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba902e0 0x55934ba94da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd3059c3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba95370 0x55934ba97830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd3059c3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55934b9f32f0 con 0x55934ba8f9a0 2024-09-10T08:57:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd3059c3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55934b99b270 con 0x55934ba95370 2024-09-10T08:57:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd3059c3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55934b9ef2b0 con 0x55934ba902e0 2024-09-10T08:57:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd2fbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934ba8fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd2fbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934ba8fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38714/0 (socket says 172.21.15.172:38714) 2024-09-10T08:57:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd2fbfff640 1 -- 172.21.15.172:0/1626229515 learned_addr learned my addr 172.21.15.172:0/1626229515 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd2f9ffb640 1 --2- 172.21.15.172:0/1780677512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd2ec0a2da0 0x7fd2ec0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd2fbfff640 1 --2- 172.21.15.172:0/1626229515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba95370 0x55934ba97830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd2f9ffb640 1 -- 172.21.15.172:0/1780677512 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd2f40ce910 con 0x7fd2fc1013d0 2024-09-10T08:57:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd3010e8640 1 --2- 172.21.15.172:0/1780677512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd2ec0a2da0 0x7fd2ec0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd3008e7640 1 --2- 172.21.15.172:0/1626229515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba902e0 0x55934ba94da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd2fbfff640 1 -- 172.21.15.172:0/1626229515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba902e0 msgr2=0x55934ba94da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd2fbfff640 1 --2- 172.21.15.172:0/1626229515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba902e0 0x55934ba94da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd2fbfff640 1 -- 172.21.15.172:0/1626229515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba95370 msgr2=0x55934ba97830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd2fbfff640 1 --2- 172.21.15.172:0/1626229515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba95370 0x55934ba97830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.132+0000 7fd2fbfff640 1 -- 172.21.15.172:0/1626229515 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55934b99a340 con 0x55934ba8f9a0 2024-09-10T08:57:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.133+0000 7fd2fbfff640 1 --2- 172.21.15.172:0/1626229515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934ba8fda0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80058c0 tx=0x7fd2e800b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b39a23ddcb10ab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.133+0000 7fd3010e8640 1 --2- 172.21.15.172:0/1780677512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd2ec0a2da0 0x7fd2ec0a5260 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4097270 tx=0x7fd2f40bb040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.133+0000 7fd2e6ffd640 1 -- 172.21.15.172:0/1626229515 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8004030 con 0x55934ba8f9a0 2024-09-10T08:57:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.133+0000 7fd2e6ffd640 1 -- 172.21.15.172:0/1626229515 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2e800fbb0 con 0x55934ba8f9a0 2024-09-10T08:57:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.133+0000 7fd2e6ffd640 1 -- 172.21.15.172:0/1626229515 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80035f0 con 0x55934ba8f9a0 2024-09-10T08:57:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.134+0000 7fd3059c3740 1 -- 172.21.15.172:0/1626229515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 msgr2=0x55934ba8fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.134+0000 7fd3059c3740 1 --2- 172.21.15.172:0/1626229515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934ba8fda0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80058c0 tx=0x7fd2e800b4c0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.134+0000 7fd3059c3740 1 -- 172.21.15.172:0/1626229515 shutdown_connections 2024-09-10T08:57:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.134+0000 7fd3059c3740 1 --2- 172.21.15.172:0/1626229515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba95370 0x55934ba97830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.134+0000 7fd3059c3740 1 --2- 172.21.15.172:0/1626229515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba902e0 0x55934ba94da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.134+0000 7fd3059c3740 1 --2- 172.21.15.172:0/1626229515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934ba8fda0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.134+0000 7fd3059c3740 1 -- 172.21.15.172:0/1626229515 >> 172.21.15.172:0/1626229515 conn(0x55934ba8b010 msgr2=0x55934ba8b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.134+0000 7fd3059c3740 1 -- 172.21.15.172:0/1626229515 shutdown_connections 2024-09-10T08:57:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.134+0000 7fd3059c3740 1 -- 172.21.15.172:0/1626229515 wait complete. 2024-09-10T08:57:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.134+0000 7fd3059c3740 1 -- start start 2024-09-10T08:57:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.135+0000 7fd3059c3740 1 Processor -- start 2024-09-10T08:57:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.135+0000 7fd3059c3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba8f9a0 0x55934ba99b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.135+0000 7fd3059c3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba902e0 0x55934ba9a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.135+0000 7fd3059c3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba95370 0x55934baa10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.135+0000 7fd2fbfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba8f9a0 0x55934ba99b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.135+0000 7fd2fbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba95370 0x55934baa10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.135+0000 7fd2fbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba95370 0x55934baa10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38728/0 (socket says 172.21.15.172:38728) 2024-09-10T08:57:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.135+0000 7fd2fbfff640 1 -- 172.21.15.172:0/763237949 learned_addr learned my addr 172.21.15.172:0/763237949 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.136+0000 7fd3059c3740 1 -- 172.21.15.172:0/763237949 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55934ba982e0 con 0x55934ba95370 2024-09-10T08:57:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.136+0000 7fd3059c3740 1 -- 172.21.15.172:0/763237949 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55934b97c290 con 0x55934ba8f9a0 2024-09-10T08:57:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.136+0000 7fd3059c3740 1 -- 172.21.15.172:0/763237949 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55934ba98460 con 0x55934ba902e0 2024-09-10T08:57:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.136+0000 7fd3059c3740 10 client.0 _ll_register_callbacks cb 0x55934b9ad140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.136+0000 7fd2fbfff640 1 -- 172.21.15.172:0/763237949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba902e0 msgr2=0x55934ba9a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.136+0000 7fd2fbfff640 1 --2- 172.21.15.172:0/763237949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba902e0 0x55934ba9a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.136+0000 7fd2fbfff640 1 -- 172.21.15.172:0/763237949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba8f9a0 msgr2=0x55934ba99b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.136+0000 7fd2fbfff640 1 --2- 172.21.15.172:0/763237949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba8f9a0 0x55934ba99b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.136+0000 7fd2fbfff640 1 -- 172.21.15.172:0/763237949 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55934b706210 con 0x55934ba95370 2024-09-10T08:57:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.136+0000 7fd2fbfff640 1 --2- 172.21.15.172:0/763237949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba95370 0x55934baa10b0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800bfd0 tx=0x7fd2e8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1937b1f54a2a6cfc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.136+0000 7fd2dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.136+0000 7fd2dbfff640 1 -- 172.21.15.172:0/763237949 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8026070 con 0x55934ba95370 2024-09-10T08:57:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.137+0000 7fd2dbfff640 1 -- 172.21.15.172:0/763237949 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd2e800f700 con 0x55934ba95370 2024-09-10T08:57:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.137+0000 7fd3059c3740 1 -- 172.21.15.172:0/763237949 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2e800fbb0 con 0x55934ba95370 2024-09-10T08:57:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.137+0000 7fd2dbfff640 1 -- 172.21.15.172:0/763237949 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd2e8029040 con 0x55934ba95370 2024-09-10T08:57:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.137+0000 7fd3059c3740 10 client.18885 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.137+0000 7fd3059c3740 1 -- 172.21.15.172:0/763237949 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55934b7d2620 con 0x55934ba95370 2024-09-10T08:57:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.138+0000 7fd2dbfff640 1 -- 172.21.15.172:0/763237949 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2e800f900 con 0x55934ba95370 2024-09-10T08:57:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.138+0000 7fd2dbfff640 1 -- 172.21.15.172:0/763237949 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fd2e802d020 con 0x55934ba95370 2024-09-10T08:57:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.138+0000 7fd3059c3740 10 client.18885 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.138+0000 7fd3059c3740 10 client.18885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.138+0000 7fd3059c3740 20 client.18885 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.138+0000 7fd3059c3740 1 --2- 172.21.15.172:0/763237949 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55934baa5440 0x55934baa7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.138+0000 7fd3059c3740 4 client.18885 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:57:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.138+0000 7fd3059c3740 1 -- 172.21.15.172:0/763237949 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd2e800fbb0 con 0x55934baa5440 2024-09-10T08:57:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.138+0000 7fd3008e7640 1 --2- 172.21.15.172:0/763237949 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55934baa5440 0x55934baa7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.139+0000 7fd3008e7640 1 --2- 172.21.15.172:0/763237949 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55934baa5440 0x55934baa7830 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f4fc768401b3403 server_cookie=10f1d317a4ec5e53 in_seq=0 out_seq=0 2024-09-10T08:57:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.139+0000 7fd2dbfff640 10 client.18885 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.139+0000 7fd2dbfff640 1 -- 172.21.15.172:0/763237949 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd2e800fbb0 con 0x55934baa5440 2024-09-10T08:57:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.139+0000 7fd2dbfff640 10 client.18885 handle_command_reply: tid=0 2024-09-10T08:57:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.139+0000 7fd3059c3740 1 client.18885 shutdown 2024-09-10T08:57:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.139+0000 7fd3059c3740 1 -- 172.21.15.172:0/763237949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba95370 msgr2=0x55934baa10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.139+0000 7fd3059c3740 1 --2- 172.21.15.172:0/763237949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba95370 0x55934baa10b0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800bfd0 tx=0x7fd2e8005850 comp rx=0 tx=0).stop 2024-09-10T08:57:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.140+0000 7fd3059c3740 1 -- 172.21.15.172:0/763237949 shutdown_connections 2024-09-10T08:57:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.140+0000 7fd3059c3740 1 --2- 172.21.15.172:0/763237949 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55934baa5440 0x55934baa7830 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.140+0000 7fd2dbfff640 0 client.18885 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.140+0000 7fd3059c3740 1 --2- 172.21.15.172:0/763237949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba95370 0x55934baa10b0 secure :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800bfd0 tx=0x7fd2e8005850 comp rx=0 tx=0).stop 2024-09-10T08:57:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.140+0000 7fd3059c3740 1 --2- 172.21.15.172:0/763237949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba902e0 0x55934ba9a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.140+0000 7fd3059c3740 1 --2- 172.21.15.172:0/763237949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba8f9a0 0x55934ba99b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.140+0000 7fd3059c3740 1 -- 172.21.15.172:0/763237949 >> 172.21.15.172:0/763237949 conn(0x55934ba8b010 msgr2=0x55934ba8c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.140+0000 7fd3059c3740 1 -- 172.21.15.172:0/763237949 shutdown_connections 2024-09-10T08:57:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.140+0000 7fd3059c3740 1 -- 172.21.15.172:0/763237949 wait complete. 2024-09-10T08:57:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.140+0000 7fd3059c3740 20 client.18885 trim_cache size 0 max 16384 2024-09-10T08:57:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.157+0000 7fd3059c3740 1 Processor -- start 2024-09-10T08:57:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.157+0000 7fd3059c3740 1 -- start start 2024-09-10T08:57:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.157+0000 7fd3059c3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba8b010 0x55934ba99020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.157+0000 7fd3059c3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934baa0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.157+0000 7fd3059c3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba902e0 0x55934ba8a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.157+0000 7fd3059c3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1aced0 con 0x55934ba8f9a0 2024-09-10T08:57:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.157+0000 7fd3059c3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1aea70 con 0x55934ba8b010 2024-09-10T08:57:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.157+0000 7fd3059c3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1ad5a0 con 0x55934ba902e0 2024-09-10T08:57:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.157+0000 7fd3008e7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934baa0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.158+0000 7fd3008e7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934baa0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42816/0 (socket says 172.21.15.172:42816) 2024-09-10T08:57:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.158+0000 7fd3008e7640 1 -- 172.21.15.172:0/3982316648 learned_addr learned my addr 172.21.15.172:0/3982316648 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.158+0000 7fd2fbfff640 1 --2- 172.21.15.172:0/3982316648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba902e0 0x55934ba8a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.158+0000 7fd2fbfff640 1 --2- 172.21.15.172:0/3982316648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba8b010 0x55934ba99020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.158+0000 7fd3008e7640 1 -- 172.21.15.172:0/3982316648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba902e0 msgr2=0x55934ba8a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.158+0000 7fd3008e7640 1 --2- 172.21.15.172:0/3982316648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba902e0 0x55934ba8a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.158+0000 7fd3008e7640 1 -- 172.21.15.172:0/3982316648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba8b010 msgr2=0x55934ba99020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.158+0000 7fd3008e7640 1 --2- 172.21.15.172:0/3982316648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba8b010 0x55934ba99020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.158+0000 7fd3008e7640 1 -- 172.21.15.172:0/3982316648 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55934ba9a210 con 0x55934ba8f9a0 2024-09-10T08:57:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.158+0000 7fd3008e7640 1 --2- 172.21.15.172:0/3982316648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934baa0fd0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009930 tx=0x7fd2f0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfdc1bab17d71e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.158+0000 7fd2e67fc640 1 -- 172.21.15.172:0/3982316648 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0048070 con 0x55934ba8f9a0 2024-09-10T08:57:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.158+0000 7fd2e67fc640 1 -- 172.21.15.172:0/3982316648 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2f00054e0 con 0x55934ba8f9a0 2024-09-10T08:57:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.158+0000 7fd2e67fc640 1 -- 172.21.15.172:0/3982316648 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0035040 con 0x55934ba8f9a0 2024-09-10T08:57:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.159+0000 7fd3059c3740 1 -- 172.21.15.172:0/3982316648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 msgr2=0x55934baa0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.159+0000 7fd3059c3740 1 --2- 172.21.15.172:0/3982316648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934baa0fd0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009930 tx=0x7fd2f0000bc0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.159+0000 7fd3059c3740 1 -- 172.21.15.172:0/3982316648 shutdown_connections 2024-09-10T08:57:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.159+0000 7fd3059c3740 1 --2- 172.21.15.172:0/3982316648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba902e0 0x55934ba8a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.159+0000 7fd3059c3740 1 --2- 172.21.15.172:0/3982316648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934baa0fd0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.159+0000 7fd3059c3740 1 --2- 172.21.15.172:0/3982316648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba8b010 0x55934ba99020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.159+0000 7fd3059c3740 1 -- 172.21.15.172:0/3982316648 >> 172.21.15.172:0/3982316648 conn(0x55934ba97db0 msgr2=0x55934ba899a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.159+0000 7fd3059c3740 1 -- 172.21.15.172:0/3982316648 shutdown_connections 2024-09-10T08:57:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.159+0000 7fd3059c3740 1 -- 172.21.15.172:0/3982316648 wait complete. 2024-09-10T08:57:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.160+0000 7fd3059c3740 1 -- start start 2024-09-10T08:57:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.160+0000 7fd3059c3740 1 Processor -- start 2024-09-10T08:57:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.160+0000 7fd3059c3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba8b010 0x55934ba9e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.160+0000 7fd3059c3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934ba9e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.160+0000 7fd3059c3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba902e0 0x55934ba9f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.160+0000 7fd3059c3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1ae3a0 con 0x55934ba8f9a0 2024-09-10T08:57:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.160+0000 7fd3059c3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1adcd0 con 0x55934ba902e0 2024-09-10T08:57:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.160+0000 7fd3059c3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc1aaeb0 con 0x55934ba8b010 2024-09-10T08:57:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.160+0000 7fd3059c3740 10 client.0 _ll_register_callbacks cb 0x7fd2e800bf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.160+0000 7fd3008e7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934ba9e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.160+0000 7fd3008e7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934ba9e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42826/0 (socket says 172.21.15.172:42826) 2024-09-10T08:57:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.160+0000 7fd3008e7640 1 -- 172.21.15.172:0/3972505765 learned_addr learned my addr 172.21.15.172:0/3972505765 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.160+0000 7fd2fbfff640 1 --2- 172.21.15.172:0/3972505765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba8b010 0x55934ba9e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.160+0000 7fd2fbfff640 1 --2- 172.21.15.172:0/3972505765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba902e0 0x55934ba9f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.161+0000 7fd3008e7640 1 -- 172.21.15.172:0/3972505765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba8b010 msgr2=0x55934ba9e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.161+0000 7fd3008e7640 1 --2- 172.21.15.172:0/3972505765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba8b010 0x55934ba9e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.161+0000 7fd3008e7640 1 -- 172.21.15.172:0/3972505765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba902e0 msgr2=0x55934ba9f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.161+0000 7fd3008e7640 1 --2- 172.21.15.172:0/3972505765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba902e0 0x55934ba9f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.161+0000 7fd3008e7640 1 -- 172.21.15.172:0/3972505765 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55934ba9ff00 con 0x55934ba8f9a0 2024-09-10T08:57:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.161+0000 7fd2fbfff640 1 --2- 172.21.15.172:0/3972505765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba902e0 0x55934ba9f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:57:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.161+0000 7fd3008e7640 1 --2- 172.21.15.172:0/3972505765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934ba9e9a0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fd2f002a460 tx=0x7fd2f00041b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc42baf330fce64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.161+0000 7fd2dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.161+0000 7fd2dbfff640 1 -- 172.21.15.172:0/3972505765 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0048070 con 0x55934ba8f9a0 2024-09-10T08:57:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.161+0000 7fd2dbfff640 1 -- 172.21.15.172:0/3972505765 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd2f0004bd0 con 0x55934ba8f9a0 2024-09-10T08:57:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.161+0000 7fd3059c3740 1 -- 172.21.15.172:0/3972505765 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2f00054e0 con 0x55934ba8f9a0 2024-09-10T08:57:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.161+0000 7fd2dbfff640 1 -- 172.21.15.172:0/3972505765 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd2f0055050 con 0x55934ba8f9a0 2024-09-10T08:57:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.161+0000 7fd3059c3740 10 client.18891 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.161+0000 7fd3059c3740 1 -- 172.21.15.172:0/3972505765 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55934baa1210 con 0x55934ba8f9a0 2024-09-10T08:57:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.161+0000 7fd2dbfff640 1 -- 172.21.15.172:0/3972505765 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd2f00054e0 con 0x55934ba8f9a0 2024-09-10T08:57:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.162+0000 7fd2dbfff640 1 -- 172.21.15.172:0/3972505765 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7fd2f0031a40 con 0x55934ba8f9a0 2024-09-10T08:57:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.162+0000 7fd3059c3740 10 client.18891 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.162+0000 7fd3059c3740 10 client.18891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:57:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.162+0000 7fd3059c3740 20 client.18891 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.162+0000 7fd3059c3740 1 --2- 172.21.15.172:0/3972505765 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55934bb104d0 0x55934bb128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.162+0000 7fd3059c3740 4 client.18891 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:57:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.162+0000 7fd3059c3740 1 -- 172.21.15.172:0/3972505765 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55934ba00a20 con 0x55934bb104d0 2024-09-10T08:57:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.162+0000 7fd2fbfff640 1 --2- 172.21.15.172:0/3972505765 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55934bb104d0 0x55934bb128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.164+0000 7fd2dbfff640 10 client.18891 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.164+0000 7fd2fbfff640 1 --2- 172.21.15.172:0/3972505765 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55934bb104d0 0x55934bb128f0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15af16abdd36e99d server_cookie=4952672fdca0b5af in_seq=0 out_seq=0 2024-09-10T08:57:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.170+0000 7fd2dbfff640 1 -- 172.21.15.172:0/3972505765 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55934ba00a20 con 0x55934bb104d0 2024-09-10T08:57:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.170+0000 7fd2dbfff640 10 client.18891 handle_command_reply: tid=0 2024-09-10T08:57:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.170+0000 7fd3059c3740 1 client.18891 shutdown 2024-09-10T08:57:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.170+0000 7fd3059c3740 1 -- 172.21.15.172:0/3972505765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 msgr2=0x55934ba9e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.170+0000 7fd3059c3740 1 --2- 172.21.15.172:0/3972505765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934ba9e9a0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fd2f002a460 tx=0x7fd2f00041b0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.170+0000 7fd3059c3740 1 -- 172.21.15.172:0/3972505765 shutdown_connections 2024-09-10T08:57:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.170+0000 7fd3059c3740 1 --2- 172.21.15.172:0/3972505765 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55934bb104d0 0x55934bb128f0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.170+0000 7fd3059c3740 1 --2- 172.21.15.172:0/3972505765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934ba902e0 0x55934ba9f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.170+0000 7fd3059c3740 1 --2- 172.21.15.172:0/3972505765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55934ba8f9a0 0x55934ba9e9a0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.170+0000 7fd2dbfff640 0 client.18891 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:57:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.170+0000 7fd3059c3740 1 --2- 172.21.15.172:0/3972505765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934ba8b010 0x55934ba9e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.170+0000 7fd3059c3740 1 -- 172.21.15.172:0/3972505765 >> 172.21.15.172:0/3972505765 conn(0x55934ba97db0 msgr2=0x55934ba8a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.170+0000 7fd3059c3740 1 -- 172.21.15.172:0/3972505765 shutdown_connections 2024-09-10T08:57:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.170+0000 7fd3059c3740 1 -- 172.21.15.172:0/3972505765 wait complete. 2024-09-10T08:57:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.170+0000 7fd3059c3740 20 client.18891 trim_cache size 0 max 16384 2024-09-10T08:57:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:57:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:57:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:57:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:57:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.171+0000 7fd302b4b640 1 -- 172.21.15.172:0/1780677512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd2ec0a2da0 msgr2=0x7fd2ec0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.171+0000 7fd302b4b640 1 --2- 172.21.15.172:0/1780677512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd2ec0a2da0 0x7fd2ec0a5260 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4097270 tx=0x7fd2f40bb040 comp rx=0 tx=0).stop 2024-09-10T08:57:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.171+0000 7fd302b4b640 1 -- 172.21.15.172:0/1780677512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2fc1013d0 msgr2=0x7fd2fc101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.171+0000 7fd302b4b640 1 --2- 172.21.15.172:0/1780677512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2fc1013d0 0x7fd2fc101880 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400b970 tx=0x7fd2f400be30 comp rx=0 tx=0).stop 2024-09-10T08:57:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.171+0000 7fd302b4b640 1 -- 172.21.15.172:0/1780677512 shutdown_connections 2024-09-10T08:57:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.171+0000 7fd302b4b640 1 --2- 172.21.15.172:0/1780677512 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd2ec0a2da0 0x7fd2ec0a5260 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.171+0000 7fd302b4b640 1 --2- 172.21.15.172:0/1780677512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2fc1013d0 0x7fd2fc101880 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.171+0000 7fd302b4b640 1 --2- 172.21.15.172:0/1780677512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2fc100580 0x7fd2fc100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.171+0000 7fd302b4b640 1 --2- 172.21.15.172:0/1780677512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2fc0ffbc0 0x7fd2fc102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.171+0000 7fd302b4b640 1 -- 172.21.15.172:0/1780677512 >> 172.21.15.172:0/1780677512 conn(0x7fd2fc0fb930 msgr2=0x7fd2fc106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.171+0000 7fd302b4b640 1 -- 172.21.15.172:0/1780677512 shutdown_connections 2024-09-10T08:57:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:57:56.171+0000 7fd302b4b640 1 -- 172.21.15.172:0/1780677512 wait complete. 2024-09-10T08:57:56.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:55 smithi174 ceph-mon[34068]: pgmap v701: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-09-10T08:57:56.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:55 smithi191 ceph-mon[35314]: pgmap v701: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 31 MiB/s wr, 26 op/s 2024-09-10T08:57:56.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.357+0000 7f2ab9a33640 1 -- 172.21.15.172:0/4230382421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ab4102d30 msgr2=0x7f2ab4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.357+0000 7f2ab9a33640 1 --2- 172.21.15.172:0/4230382421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ab4102d30 0x7f2ab4103110 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009930 tx=0x7f2aa402a420 comp rx=0 tx=0).stop 2024-09-10T08:57:56.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.357+0000 7f2ab9a33640 1 -- 172.21.15.172:0/4230382421 shutdown_connections 2024-09-10T08:57:56.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.357+0000 7f2ab9a33640 1 --2- 172.21.15.172:0/4230382421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ab41082a0 0x7f2ab410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.357+0000 7f2ab9a33640 1 --2- 172.21.15.172:0/4230382421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ab41036e0 0x7f2ab4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.357+0000 7f2ab9a33640 1 --2- 172.21.15.172:0/4230382421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ab4102d30 0x7f2ab4103110 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.357+0000 7f2ab9a33640 1 -- 172.21.15.172:0/4230382421 >> 172.21.15.172:0/4230382421 conn(0x7f2ab40fc7d0 msgr2=0x7f2ab40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:56.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.357+0000 7f2ab9a33640 1 -- 172.21.15.172:0/4230382421 shutdown_connections 2024-09-10T08:57:56.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.357+0000 7f2ab9a33640 1 -- 172.21.15.172:0/4230382421 wait complete. 2024-09-10T08:57:56.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.358+0000 7f2ab9a33640 1 Processor -- start 2024-09-10T08:57:56.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.358+0000 7f2ab9a33640 1 -- start start 2024-09-10T08:57:56.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.358+0000 7f2ab9a33640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ab4102d30 0x7f2ab419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.358+0000 7f2ab9a33640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ab41036e0 0x7f2ab419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.358+0000 7f2ab9a33640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ab41082a0 0x7f2ab419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.358+0000 7f2ab9a33640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab4111f50 con 0x7f2ab4102d30 2024-09-10T08:57:56.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.358+0000 7f2ab9a33640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab4111dd0 con 0x7f2ab41036e0 2024-09-10T08:57:56.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.358+0000 7f2ab9a33640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41120d0 con 0x7f2ab41082a0 2024-09-10T08:57:56.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.358+0000 7f2ab2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ab4102d30 0x7f2ab419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.358+0000 7f2ab2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ab4102d30 0x7f2ab419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42844/0 (socket says 172.21.15.172:42844) 2024-09-10T08:57:56.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.358+0000 7f2ab2ffd640 1 -- 172.21.15.172:0/2015783726 learned_addr learned my addr 172.21.15.172:0/2015783726 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:56.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.358+0000 7f2ab37fe640 1 --2- 172.21.15.172:0/2015783726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ab41082a0 0x7f2ab419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.358+0000 7f2ab27fc640 1 --2- 172.21.15.172:0/2015783726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ab41036e0 0x7f2ab419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.359+0000 7f2ab2ffd640 1 -- 172.21.15.172:0/2015783726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ab41082a0 msgr2=0x7f2ab419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.359+0000 7f2ab2ffd640 1 --2- 172.21.15.172:0/2015783726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ab41082a0 0x7f2ab419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.359+0000 7f2ab2ffd640 1 -- 172.21.15.172:0/2015783726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ab41036e0 msgr2=0x7f2ab419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.359+0000 7f2ab2ffd640 1 --2- 172.21.15.172:0/2015783726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ab41036e0 0x7f2ab419ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.359+0000 7f2ab2ffd640 1 -- 172.21.15.172:0/2015783726 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab419f700 con 0x7f2ab4102d30 2024-09-10T08:57:56.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.359+0000 7f2ab2ffd640 1 --2- 172.21.15.172:0/2015783726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ab4102d30 0x7f2ab419a750 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4004770 tx=0x7f2aa40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:56.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.359+0000 7f2ab8a31640 1 -- 172.21.15.172:0/2015783726 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa403f070 con 0x7f2ab4102d30 2024-09-10T08:57:56.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.359+0000 7f2ab9a33640 1 -- 172.21.15.172:0/2015783726 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab419f990 con 0x7f2ab4102d30 2024-09-10T08:57:56.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.359+0000 7f2ab8a31640 1 -- 172.21.15.172:0/2015783726 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2aa403b460 con 0x7f2ab4102d30 2024-09-10T08:57:56.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.359+0000 7f2ab8a31640 1 -- 172.21.15.172:0/2015783726 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4035070 con 0x7f2ab4102d30 2024-09-10T08:57:56.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.359+0000 7f2ab9a33640 1 -- 172.21.15.172:0/2015783726 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab41a72d0 con 0x7f2ab4102d30 2024-09-10T08:57:56.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.362+0000 7f2ab8a31640 1 -- 172.21.15.172:0/2015783726 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2aa403b610 con 0x7f2ab4102d30 2024-09-10T08:57:56.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.362+0000 7f2ab8a31640 1 --2- 172.21.15.172:0/2015783726 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2a9c0a2b80 0x7f2a9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.363+0000 7f2ab27fc640 1 --2- 172.21.15.172:0/2015783726 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2a9c0a2b80 0x7f2a9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.363+0000 7f2ab8a31640 1 -- 172.21.15.172:0/2015783726 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2aa40ef680 con 0x7f2ab4102d30 2024-09-10T08:57:56.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.363+0000 7f2ab27fc640 1 --2- 172.21.15.172:0/2015783726 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2a9c0a2b80 0x7f2a9c0a5040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40ff0b0 tx=0x7f2aa800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:56.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.376+0000 7f2abbe0c740 1 Processor -- start 2024-09-10T08:57:56.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.376+0000 7f2abbe0c740 1 -- start start 2024-09-10T08:57:56.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.376+0000 7f2abbe0c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c29b40 0x55a046c28090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.376+0000 7f2abbe0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c285d0 0x55a046c28a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.376+0000 7f2abbe0c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c35760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.376+0000 7f2abbe0c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a046b8e2f0 con 0x55a046c33370 2024-09-10T08:57:56.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.376+0000 7f2abbe0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a046b35cf0 con 0x55a046c285d0 2024-09-10T08:57:56.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.376+0000 7f2abbe0c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a046b8a2b0 con 0x55a046c29b40 2024-09-10T08:57:56.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.376+0000 7f2ab37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c35760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.376+0000 7f2ab37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c35760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42854/0 (socket says 172.21.15.172:42854) 2024-09-10T08:57:56.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.376+0000 7f2ab37fe640 1 -- 172.21.15.172:0/3619208465 learned_addr learned my addr 172.21.15.172:0/3619208465 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:56.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.376+0000 7f2ab2ffd640 1 --2- 172.21.15.172:0/3619208465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c29b40 0x55a046c28090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.376+0000 7f2ab27fc640 1 --2- 172.21.15.172:0/3619208465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c285d0 0x55a046c28a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.377+0000 7f2ab37fe640 1 -- 172.21.15.172:0/3619208465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c29b40 msgr2=0x55a046c28090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:56.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.377+0000 7f2ab37fe640 1 --2- 172.21.15.172:0/3619208465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c29b40 0x55a046c28090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.377+0000 7f2ab37fe640 1 -- 172.21.15.172:0/3619208465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c285d0 msgr2=0x55a046c28a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:56.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.377+0000 7f2ab37fe640 1 --2- 172.21.15.172:0/3619208465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c285d0 0x55a046c28a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.377+0000 7f2ab37fe640 1 -- 172.21.15.172:0/3619208465 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a046b1a3a0 con 0x55a046c33370 2024-09-10T08:57:56.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.377+0000 7f2ab37fe640 1 --2- 172.21.15.172:0/3619208465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c35760 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009f90 tx=0x7f2aac010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804e0404829dc5f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:56.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.377+0000 7f2a99ffb640 1 -- 172.21.15.172:0/3619208465 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac011700 con 0x55a046c33370 2024-09-10T08:57:56.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.377+0000 7f2a99ffb640 1 -- 172.21.15.172:0/3619208465 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2aac011d50 con 0x55a046c33370 2024-09-10T08:57:56.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.377+0000 7f2a99ffb640 1 -- 172.21.15.172:0/3619208465 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac019d40 con 0x55a046c33370 2024-09-10T08:57:56.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.378+0000 7f2abbe0c740 1 -- 172.21.15.172:0/3619208465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 msgr2=0x55a046c35760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.378+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/3619208465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c35760 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009f90 tx=0x7f2aac010860 comp rx=0 tx=0).stop 2024-09-10T08:57:56.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.378+0000 7f2abbe0c740 1 -- 172.21.15.172:0/3619208465 shutdown_connections 2024-09-10T08:57:56.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.378+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/3619208465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c35760 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.378+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/3619208465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c285d0 0x55a046c28a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.378+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/3619208465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c29b40 0x55a046c28090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.378+0000 7f2abbe0c740 1 -- 172.21.15.172:0/3619208465 >> 172.21.15.172:0/3619208465 conn(0x55a046bf1310 msgr2=0x55a046c24c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:56.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.378+0000 7f2abbe0c740 1 -- 172.21.15.172:0/3619208465 shutdown_connections 2024-09-10T08:57:56.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.378+0000 7f2abbe0c740 1 -- 172.21.15.172:0/3619208465 wait complete. 2024-09-10T08:57:56.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.379+0000 7f2abbe0c740 1 -- start start 2024-09-10T08:57:56.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.379+0000 7f2abbe0c740 1 Processor -- start 2024-09-10T08:57:56.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.379+0000 7f2abbe0c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c285d0 0x55a046c32d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.379+0000 7f2abbe0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c29b40 0x55a046c2ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.379+0000 7f2abbe0c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c305d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.379+0000 7f2abbe0c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a046c36070 con 0x55a046c33370 2024-09-10T08:57:56.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.379+0000 7f2abbe0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a046b49060 con 0x55a046c29b40 2024-09-10T08:57:56.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.379+0000 7f2abbe0c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a046c361f0 con 0x55a046c285d0 2024-09-10T08:57:56.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.379+0000 7f2abbe0c740 10 client.0 _ll_register_callbacks cb 0x55a04689c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:56.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.379+0000 7f2ab37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c305d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.379+0000 7f2ab37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c305d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42856/0 (socket says 172.21.15.172:42856) 2024-09-10T08:57:56.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.379+0000 7f2ab37fe640 1 -- 172.21.15.172:0/2191508503 learned_addr learned my addr 172.21.15.172:0/2191508503 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:56.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.379+0000 7f2ab2ffd640 1 --2- 172.21.15.172:0/2191508503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c285d0 0x55a046c32d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.379+0000 7f2ab27fc640 1 --2- 172.21.15.172:0/2191508503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c29b40 0x55a046c2ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.380+0000 7f2ab37fe640 1 -- 172.21.15.172:0/2191508503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c285d0 msgr2=0x55a046c32d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:56.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.380+0000 7f2ab37fe640 1 --2- 172.21.15.172:0/2191508503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c285d0 0x55a046c32d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.380+0000 7f2ab37fe640 1 -- 172.21.15.172:0/2191508503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c29b40 msgr2=0x55a046c2ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:56.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.380+0000 7f2ab37fe640 1 --2- 172.21.15.172:0/2191508503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c29b40 0x55a046c2ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.380+0000 7f2ab37fe640 1 -- 172.21.15.172:0/2191508503 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0468ff3c0 con 0x55a046c33370 2024-09-10T08:57:56.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.380+0000 7f2ab37fe640 1 --2- 172.21.15.172:0/2191508503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c305d0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f2aac011fb0 tx=0x7f2aac019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a9e61521da89ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:56.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.380+0000 7f2a92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:56.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.380+0000 7f2a92ffd640 1 -- 172.21.15.172:0/2191508503 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac027030 con 0x55a046c33370 2024-09-10T08:57:56.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.380+0000 7f2a92ffd640 1 -- 172.21.15.172:0/2191508503 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2aac01f070 con 0x55a046c33370 2024-09-10T08:57:56.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.380+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2191508503 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aac011d50 con 0x55a046c33370 2024-09-10T08:57:56.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.380+0000 7f2a92ffd640 1 -- 172.21.15.172:0/2191508503 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2aac031050 con 0x55a046c33370 2024-09-10T08:57:56.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.380+0000 7f2abbe0c740 10 client.18903 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:56.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.380+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2191508503 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a046c41ad0 con 0x55a046c33370 2024-09-10T08:57:56.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.381+0000 7f2a92ffd640 1 -- 172.21.15.172:0/2191508503 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2aac011d50 con 0x55a046c33370 2024-09-10T08:57:56.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.381+0000 7f2a92ffd640 1 -- 172.21.15.172:0/2191508503 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f2aac011500 con 0x55a046c33370 2024-09-10T08:57:56.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.381+0000 7f2abbe0c740 10 client.18903 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:56.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.381+0000 7f2abbe0c740 10 client.18903 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:57:56.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.381+0000 7f2abbe0c740 20 client.18903 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:56.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.381+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/2191508503 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55a046c42ed0 0x55a046c452c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.381+0000 7f2abbe0c740 4 client.18903 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:57:56.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.381+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2191508503 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a046b9b860 con 0x55a046c42ed0 2024-09-10T08:57:56.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.381+0000 7f2ab2ffd640 1 --2- 172.21.15.172:0/2191508503 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55a046c42ed0 0x55a046c452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.382+0000 7f2a92ffd640 10 client.18903 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:57:56.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.382+0000 7f2ab2ffd640 1 --2- 172.21.15.172:0/2191508503 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55a046c42ed0 0x55a046c452c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=884e426d729f9383 server_cookie=2026e23b31023a55 in_seq=0 out_seq=0 2024-09-10T08:57:56.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.382+0000 7f2a92ffd640 1 -- 172.21.15.172:0/2191508503 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a046b9b860 con 0x55a046c42ed0 2024-09-10T08:57:56.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.382+0000 7f2a92ffd640 10 client.18903 handle_command_reply: tid=0 2024-09-10T08:57:56.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.382+0000 7f2abbe0c740 1 client.18903 shutdown 2024-09-10T08:57:56.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.383+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2191508503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 msgr2=0x55a046c305d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.383+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/2191508503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c305d0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f2aac011fb0 tx=0x7f2aac019740 comp rx=0 tx=0).stop 2024-09-10T08:57:56.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.383+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2191508503 shutdown_connections 2024-09-10T08:57:56.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.383+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/2191508503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c305d0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.383+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/2191508503 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55a046c42ed0 0x55a046c452c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.383+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/2191508503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c29b40 0x55a046c2ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.383+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/2191508503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c285d0 0x55a046c32d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.383+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2191508503 >> 172.21.15.172:0/2191508503 conn(0x55a046bf1310 msgr2=0x55a046c2a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:56.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.383+0000 7f2a92ffd640 0 client.18903 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:57:56.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.383+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2191508503 shutdown_connections 2024-09-10T08:57:56.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.383+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2191508503 wait complete. 2024-09-10T08:57:56.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.383+0000 7f2abbe0c740 20 client.18903 trim_cache size 0 max 16384 2024-09-10T08:57:56.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.397+0000 7f2abbe0c740 1 Processor -- start 2024-09-10T08:57:56.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.397+0000 7f2abbe0c740 1 -- start start 2024-09-10T08:57:56.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.397+0000 7f2abbe0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c285d0 0x55a046c3f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.397+0000 7f2abbe0c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c29b40 0x55a046c2b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.397+0000 7f2abbe0c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c2b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.397+0000 7f2abbe0c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41b0e80 con 0x55a046c33370 2024-09-10T08:57:56.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.397+0000 7f2abbe0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41b2a20 con 0x55a046c285d0 2024-09-10T08:57:56.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.397+0000 7f2abbe0c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41b1550 con 0x55a046c29b40 2024-09-10T08:57:56.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.397+0000 7f2ab37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c2b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.397+0000 7f2ab37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c2b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42860/0 (socket says 172.21.15.172:42860) 2024-09-10T08:57:56.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.397+0000 7f2ab37fe640 1 -- 172.21.15.172:0/2128713133 learned_addr learned my addr 172.21.15.172:0/2128713133 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:56.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.398+0000 7f2ab2ffd640 1 --2- 172.21.15.172:0/2128713133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c285d0 0x55a046c3f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.398+0000 7f2ab27fc640 1 --2- 172.21.15.172:0/2128713133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c29b40 0x55a046c2b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.398+0000 7f2ab37fe640 1 -- 172.21.15.172:0/2128713133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c29b40 msgr2=0x55a046c2b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:56.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.398+0000 7f2ab37fe640 1 --2- 172.21.15.172:0/2128713133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c29b40 0x55a046c2b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.398+0000 7f2ab37fe640 1 -- 172.21.15.172:0/2128713133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c285d0 msgr2=0x55a046c3f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:56.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.398+0000 7f2ab37fe640 1 --2- 172.21.15.172:0/2128713133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c285d0 0x55a046c3f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.398+0000 7f2ab37fe640 1 -- 172.21.15.172:0/2128713133 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a046c30bb0 con 0x55a046c33370 2024-09-10T08:57:56.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.398+0000 7f2ab27fc640 1 --2- 172.21.15.172:0/2128713133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c29b40 0x55a046c2b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:57:56.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.398+0000 7f2ab37fe640 1 --2- 172.21.15.172:0/2128713133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c2b660 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f2aac006010 tx=0x7f2aac0098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695df8c70b7972d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:56.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.398+0000 7f2a997fa640 1 -- 172.21.15.172:0/2128713133 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac027070 con 0x55a046c33370 2024-09-10T08:57:56.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.398+0000 7f2a997fa640 1 -- 172.21.15.172:0/2128713133 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2aac0197b0 con 0x55a046c33370 2024-09-10T08:57:56.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.398+0000 7f2a997fa640 1 -- 172.21.15.172:0/2128713133 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac011b60 con 0x55a046c33370 2024-09-10T08:57:56.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.399+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2128713133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 msgr2=0x55a046c2b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.399+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/2128713133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c2b660 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f2aac006010 tx=0x7f2aac0098e0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.399+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2128713133 shutdown_connections 2024-09-10T08:57:56.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.399+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/2128713133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c33370 0x55a046c2b660 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.399+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/2128713133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c29b40 0x55a046c2b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.399+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/2128713133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c285d0 0x55a046c3f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.399+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2128713133 >> 172.21.15.172:0/2128713133 conn(0x55a046bf1310 msgr2=0x55a046c2f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:56.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.399+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2128713133 shutdown_connections 2024-09-10T08:57:56.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.399+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2128713133 wait complete. 2024-09-10T08:57:56.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.399+0000 7f2abbe0c740 1 -- start start 2024-09-10T08:57:56.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.400+0000 7f2abbe0c740 1 Processor -- start 2024-09-10T08:57:56.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.400+0000 7f2abbe0c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c285d0 0x55a046c3b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.400+0000 7f2abbe0c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c29b40 0x55a046c36c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.400+0000 7f2abbe0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c33370 0x55a046c371b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.400+0000 7f2abbe0c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41b2350 con 0x55a046c29b40 2024-09-10T08:57:56.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.400+0000 7f2abbe0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41b1c80 con 0x55a046c33370 2024-09-10T08:57:56.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.400+0000 7f2abbe0c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41aee60 con 0x55a046c285d0 2024-09-10T08:57:56.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.400+0000 7f2abbe0c740 10 client.0 _ll_register_callbacks cb 0x7f2aa80042a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:56.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.400+0000 7f2ab27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c29b40 0x55a046c36c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.400+0000 7f2ab27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c29b40 0x55a046c36c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42864/0 (socket says 172.21.15.172:42864) 2024-09-10T08:57:56.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.400+0000 7f2ab27fc640 1 -- 172.21.15.172:0/2598568979 learned_addr learned my addr 172.21.15.172:0/2598568979 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:56.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.400+0000 7f2ab37fe640 1 --2- 172.21.15.172:0/2598568979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c33370 0x55a046c371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.400+0000 7f2ab2ffd640 1 --2- 172.21.15.172:0/2598568979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c285d0 0x55a046c3b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.401+0000 7f2ab27fc640 1 -- 172.21.15.172:0/2598568979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c285d0 msgr2=0x55a046c3b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:56.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.401+0000 7f2ab27fc640 1 --2- 172.21.15.172:0/2598568979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c285d0 0x55a046c3b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.401+0000 7f2ab27fc640 1 -- 172.21.15.172:0/2598568979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c33370 msgr2=0x55a046c371b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:56.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.401+0000 7f2ab27fc640 1 --2- 172.21.15.172:0/2598568979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c33370 0x55a046c371b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.401+0000 7f2ab27fc640 1 -- 172.21.15.172:0/2598568979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a046c282d0 con 0x55a046c29b40 2024-09-10T08:57:56.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.401+0000 7f2ab27fc640 1 --2- 172.21.15.172:0/2598568979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c29b40 0x55a046c36c70 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f2ab406a370 tx=0x7f2aa8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c803962faadc47 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:56.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.401+0000 7f2a92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:56.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.401+0000 7f2a92ffd640 1 -- 172.21.15.172:0/2598568979 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa802e060 con 0x55a046c29b40 2024-09-10T08:57:56.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.401+0000 7f2a92ffd640 1 -- 172.21.15.172:0/2598568979 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2aa801e620 con 0x55a046c29b40 2024-09-10T08:57:56.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.401+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2598568979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aac0197b0 con 0x55a046c29b40 2024-09-10T08:57:56.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.401+0000 7f2a92ffd640 1 -- 172.21.15.172:0/2598568979 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2aa8035050 con 0x55a046c29b40 2024-09-10T08:57:56.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.401+0000 7f2abbe0c740 10 client.18909 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:56.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.401+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2598568979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a046cad520 con 0x55a046c29b40 2024-09-10T08:57:56.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.402+0000 7f2a92ffd640 1 -- 172.21.15.172:0/2598568979 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2aac0197b0 con 0x55a046c29b40 2024-09-10T08:57:56.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.402+0000 7f2a92ffd640 1 -- 172.21.15.172:0/2598568979 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f2aa8028070 con 0x55a046c29b40 2024-09-10T08:57:56.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.402+0000 7f2abbe0c740 10 client.18909 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:56.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.402+0000 7f2abbe0c740 10 client.18909 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:57:56.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.402+0000 7f2abbe0c740 20 client.18909 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:56.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.402+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/2598568979 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55a046cadd00 0x55a046cb0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:56.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.402+0000 7f2abbe0c740 4 client.18909 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:57:56.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.402+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2598568979 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a046b95d70 con 0x55a046cadd00 2024-09-10T08:57:56.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.402+0000 7f2ab2ffd640 1 --2- 172.21.15.172:0/2598568979 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55a046cadd00 0x55a046cb0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:56.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.403+0000 7f2a92ffd640 10 client.18909 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:57:56.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.403+0000 7f2ab2ffd640 1 --2- 172.21.15.172:0/2598568979 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55a046cadd00 0x55a046cb0120 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=70e690769e9421ce server_cookie=86fb629ca822ecbd in_seq=0 out_seq=0 2024-09-10T08:57:56.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.403+0000 7f2a92ffd640 1 -- 172.21.15.172:0/2598568979 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a046b95d70 con 0x55a046cadd00 2024-09-10T08:57:56.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.403+0000 7f2a92ffd640 10 client.18909 handle_command_reply: tid=0 2024-09-10T08:57:56.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.403+0000 7f2abbe0c740 1 client.18909 shutdown 2024-09-10T08:57:56.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.403+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2598568979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c29b40 msgr2=0x55a046c36c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.403+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/2598568979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c29b40 0x55a046c36c70 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f2ab406a370 tx=0x7f2aa8004730 comp rx=0 tx=0).stop 2024-09-10T08:57:56.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.403+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2598568979 shutdown_connections 2024-09-10T08:57:56.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.403+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/2598568979 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55a046cadd00 0x55a046cb0120 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.403+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/2598568979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a046c33370 0x55a046c371b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.403+0000 7f2a92ffd640 0 client.18909 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:57:56.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.403+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/2598568979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a046c29b40 0x55a046c36c70 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.403+0000 7f2abbe0c740 1 --2- 172.21.15.172:0/2598568979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a046c285d0 0x55a046c3b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.403+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2598568979 >> 172.21.15.172:0/2598568979 conn(0x55a046bf1310 msgr2=0x55a046c31840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:56.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.403+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2598568979 shutdown_connections 2024-09-10T08:57:56.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.403+0000 7f2abbe0c740 1 -- 172.21.15.172:0/2598568979 wait complete. 2024-09-10T08:57:56.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.403+0000 7f2abbe0c740 20 client.18909 trim_cache size 0 max 16384 2024-09-10T08:57:56.412 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:57:56.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.404+0000 7f2ab9a33640 1 -- 172.21.15.172:0/2015783726 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2a9c0a2b80 msgr2=0x7f2a9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.404+0000 7f2ab9a33640 1 --2- 172.21.15.172:0/2015783726 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2a9c0a2b80 0x7f2a9c0a5040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40ff0b0 tx=0x7f2aa800b390 comp rx=0 tx=0).stop 2024-09-10T08:57:56.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.404+0000 7f2ab9a33640 1 -- 172.21.15.172:0/2015783726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ab4102d30 msgr2=0x7f2ab419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:56.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.404+0000 7f2ab9a33640 1 --2- 172.21.15.172:0/2015783726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ab4102d30 0x7f2ab419a750 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4004770 tx=0x7f2aa40042f0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.404+0000 7f2ab9a33640 1 -- 172.21.15.172:0/2015783726 shutdown_connections 2024-09-10T08:57:56.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.404+0000 7f2ab9a33640 1 --2- 172.21.15.172:0/2015783726 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2a9c0a2b80 0x7f2a9c0a5040 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.404+0000 7f2ab9a33640 1 --2- 172.21.15.172:0/2015783726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ab41082a0 0x7f2ab419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.404+0000 7f2ab9a33640 1 --2- 172.21.15.172:0/2015783726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ab41036e0 0x7f2ab419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.404+0000 7f2ab9a33640 1 --2- 172.21.15.172:0/2015783726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ab4102d30 0x7f2ab419a750 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:56.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.404+0000 7f2ab9a33640 1 -- 172.21.15.172:0/2015783726 >> 172.21.15.172:0/2015783726 conn(0x7f2ab40fc7d0 msgr2=0x7f2ab4100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:56.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.404+0000 7f2ab9a33640 1 -- 172.21.15.172:0/2015783726 shutdown_connections 2024-09-10T08:57:56.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:56.404+0000 7f2ab9a33640 1 -- 172.21.15.172:0/2015783726 wait complete. 2024-09-10T08:57:57.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:57 smithi174 ceph-mon[34068]: pgmap v702: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-10T08:57:57.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:57 smithi191 ceph-mon[35314]: pgmap v702: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-10T08:57:57.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:57 smithi172 ceph-mon[30854]: pgmap v702: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 21 op/s 2024-09-10T08:57:58.559 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T08:57:58.846 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:57:59.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:57:59 smithi172 ceph-mon[30854]: pgmap v703: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 23 op/s 2024-09-10T08:57:59.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:57:59 smithi174 ceph-mon[34068]: pgmap v703: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 23 op/s 2024-09-10T08:57:59.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.378+0000 7ff7acda4640 1 -- 172.21.15.172:0/881444718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7a8101de0 msgr2=0x7ff7a81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:59.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.378+0000 7ff7acda4640 1 --2- 172.21.15.172:0/881444718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7a8101de0 0x7ff7a81021c0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7ff798009900 tx=0x7ff79802a410 comp rx=0 tx=0).stop 2024-09-10T08:57:59.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.378+0000 7ff7acda4640 1 -- 172.21.15.172:0/881444718 shutdown_connections 2024-09-10T08:57:59.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.378+0000 7ff7acda4640 1 --2- 172.21.15.172:0/881444718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7a8106f10 0x7ff7a810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.378+0000 7ff7acda4640 1 --2- 172.21.15.172:0/881444718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a8102790 0x7ff7a81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.378+0000 7ff7acda4640 1 --2- 172.21.15.172:0/881444718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7a8101de0 0x7ff7a81021c0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.378+0000 7ff7acda4640 1 -- 172.21.15.172:0/881444718 >> 172.21.15.172:0/881444718 conn(0x7ff7a80fd8e0 msgr2=0x7ff7a80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:59.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.378+0000 7ff7acda4640 1 -- 172.21.15.172:0/881444718 shutdown_connections 2024-09-10T08:57:59.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.378+0000 7ff7acda4640 1 -- 172.21.15.172:0/881444718 wait complete. 2024-09-10T08:57:59.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.379+0000 7ff7acda4640 1 Processor -- start 2024-09-10T08:57:59.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.379+0000 7ff7acda4640 1 -- start start 2024-09-10T08:57:59.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.380+0000 7ff7acda4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a8101de0 0x7ff7a8198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.380+0000 7ff7acda4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7a8102790 0x7ff7a8198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.380+0000 7ff7acda4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7a8106f10 0x7ff7a819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.380+0000 7ff7acda4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a8111e70 con 0x7ff7a8102790 2024-09-10T08:57:59.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.380+0000 7ff7acda4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a8111cf0 con 0x7ff7a8106f10 2024-09-10T08:57:59.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.380+0000 7ff7acda4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a8111ff0 con 0x7ff7a8101de0 2024-09-10T08:57:59.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.380+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7a8102790 0x7ff7a8198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:59.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.380+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7a8102790 0x7ff7a8198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42878/0 (socket says 172.21.15.172:42878) 2024-09-10T08:57:59.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.380+0000 7ff79ffff640 1 -- 172.21.15.172:0/2737771235 learned_addr learned my addr 172.21.15.172:0/2737771235 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:59.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.380+0000 7ff7a659c640 1 --2- 172.21.15.172:0/2737771235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a8101de0 0x7ff7a8198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:59.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.380+0000 7ff7a6d9d640 1 --2- 172.21.15.172:0/2737771235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7a8106f10 0x7ff7a819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:59.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.380+0000 7ff79ffff640 1 -- 172.21.15.172:0/2737771235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a8101de0 msgr2=0x7ff7a8198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:59.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.380+0000 7ff79ffff640 1 --2- 172.21.15.172:0/2737771235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a8101de0 0x7ff7a8198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.380+0000 7ff79ffff640 1 -- 172.21.15.172:0/2737771235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7a8106f10 msgr2=0x7ff7a819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:59.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.380+0000 7ff79ffff640 1 --2- 172.21.15.172:0/2737771235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7a8106f10 0x7ff7a819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.380+0000 7ff79ffff640 1 -- 172.21.15.172:0/2737771235 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a819d2c0 con 0x7ff7a8102790 2024-09-10T08:57:59.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.380+0000 7ff79ffff640 1 --2- 172.21.15.172:0/2737771235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7a8102790 0x7ff7a8198850 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7ff79000d950 tx=0x7ff79000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:59.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.381+0000 7ff79f7fe640 1 -- 172.21.15.172:0/2737771235 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff790017030 con 0x7ff7a8102790 2024-09-10T08:57:59.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.381+0000 7ff79f7fe640 1 -- 172.21.15.172:0/2737771235 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff790004020 con 0x7ff7a8102790 2024-09-10T08:57:59.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.381+0000 7ff7acda4640 1 -- 172.21.15.172:0/2737771235 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a819d5b0 con 0x7ff7a8102790 2024-09-10T08:57:59.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.381+0000 7ff79f7fe640 1 -- 172.21.15.172:0/2737771235 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7900047a0 con 0x7ff7a8102790 2024-09-10T08:57:59.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.381+0000 7ff7acda4640 1 -- 172.21.15.172:0/2737771235 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a81a4e90 con 0x7ff7a8102790 2024-09-10T08:57:59.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.384+0000 7ff79f7fe640 1 -- 172.21.15.172:0/2737771235 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff7900041d0 con 0x7ff7a8102790 2024-09-10T08:57:59.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.384+0000 7ff7af17d740 1 Processor -- start 2024-09-10T08:57:59.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.384+0000 7ff7af17d740 1 -- start start 2024-09-10T08:57:59.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.384+0000 7ff7af17d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891976cb40 0x55891976b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.384+0000 7ff7af17d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976b5d0 0x55891976ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff7af17d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891977a780 0x55891977cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff7af17d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5589196d12f0 con 0x55891976b5d0 2024-09-10T08:57:59.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff7af17d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558919678cf0 con 0x55891977a780 2024-09-10T08:57:59.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff7af17d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5589196cd2b0 con 0x55891976cb40 2024-09-10T08:57:59.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff7a6d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891976cb40 0x55891976b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:59.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff7a6d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891976cb40 0x55891976b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:54014/0 (socket says 172.21.15.172:54014) 2024-09-10T08:57:59.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff7a659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976b5d0 0x55891976ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:59.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff7a6d9d640 1 -- 172.21.15.172:0/3498401309 learned_addr learned my addr 172.21.15.172:0/3498401309 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:59.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff79f7fe640 1 --2- 172.21.15.172:0/2737771235 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7940a2c00 0x7ff7940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff7a6d9d640 1 -- 172.21.15.172:0/3498401309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891977a780 msgr2=0x55891977cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:59.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff79ffff640 1 --2- 172.21.15.172:0/3498401309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891977a780 0x55891977cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:59.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff7a6d9d640 1 --2- 172.21.15.172:0/3498401309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891977a780 0x55891977cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff7a6d9d640 1 -- 172.21.15.172:0/3498401309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976b5d0 msgr2=0x55891976ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:59.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff7a6d9d640 1 --2- 172.21.15.172:0/3498401309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976b5d0 0x55891976ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff7a6d9d640 1 -- 172.21.15.172:0/3498401309 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55891965d3a0 con 0x55891976cb40 2024-09-10T08:57:59.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff7a659c640 1 --2- 172.21.15.172:0/3498401309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976b5d0 0x55891976ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:59.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff7a6d9d640 1 --2- 172.21.15.172:0/2737771235 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7940a2c00 0x7ff7940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:59.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff79f7fe640 1 -- 172.21.15.172:0/2737771235 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff7900ced00 con 0x7ff7a8102790 2024-09-10T08:57:59.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.385+0000 7ff7a6d9d640 1 --2- 172.21.15.172:0/3498401309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891976cb40 0x55891976b090 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0010440 tx=0x7ff7a0010900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=517e168293f8a369 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:59.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.386+0000 7ff79cff9640 1 -- 172.21.15.172:0/3498401309 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a001d400 con 0x55891976cb40 2024-09-10T08:57:59.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.386+0000 7ff79cff9640 1 -- 172.21.15.172:0/3498401309 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7a001da50 con 0x55891976cb40 2024-09-10T08:57:59.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.386+0000 7ff79cff9640 1 -- 172.21.15.172:0/3498401309 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a001c780 con 0x55891976cb40 2024-09-10T08:57:59.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.387+0000 7ff7af17d740 1 -- 172.21.15.172:0/3498401309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891976cb40 msgr2=0x55891976b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:59.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.387+0000 7ff7af17d740 1 --2- 172.21.15.172:0/3498401309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891976cb40 0x55891976b090 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0010440 tx=0x7ff7a0010900 comp rx=0 tx=0).stop 2024-09-10T08:57:59.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.387+0000 7ff7a6d9d640 1 --2- 172.21.15.172:0/2737771235 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7940a2c00 0x7ff7940a50c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0021000 tx=0x7ff7a001e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:59.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.387+0000 7ff7af17d740 1 -- 172.21.15.172:0/3498401309 shutdown_connections 2024-09-10T08:57:59.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.387+0000 7ff7af17d740 1 --2- 172.21.15.172:0/3498401309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891977a780 0x55891977cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.387+0000 7ff7af17d740 1 --2- 172.21.15.172:0/3498401309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976b5d0 0x55891976ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.387+0000 7ff7af17d740 1 --2- 172.21.15.172:0/3498401309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891976cb40 0x55891976b090 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.387+0000 7ff7af17d740 1 -- 172.21.15.172:0/3498401309 >> 172.21.15.172:0/3498401309 conn(0x558919734310 msgr2=0x558919767c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:59.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.387+0000 7ff7af17d740 1 -- 172.21.15.172:0/3498401309 shutdown_connections 2024-09-10T08:57:59.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.387+0000 7ff7af17d740 1 -- 172.21.15.172:0/3498401309 wait complete. 2024-09-10T08:57:59.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.388+0000 7ff7af17d740 1 -- start start 2024-09-10T08:57:59.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.388+0000 7ff7af17d740 1 Processor -- start 2024-09-10T08:57:59.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.388+0000 7ff7af17d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976b5d0 0x55891977a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.388+0000 7ff7af17d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891976cb40 0x558919777440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.388+0000 7ff7af17d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891977a780 0x558919777aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.388+0000 7ff7af17d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55891977d480 con 0x55891976b5d0 2024-09-10T08:57:59.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.388+0000 7ff7af17d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55891968c060 con 0x55891976cb40 2024-09-10T08:57:59.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.388+0000 7ff7af17d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55891977d600 con 0x55891977a780 2024-09-10T08:57:59.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.388+0000 7ff7af17d740 10 client.0 _ll_register_callbacks cb 0x5589193df8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:59.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.389+0000 7ff7a659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976b5d0 0x55891977a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:59.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.389+0000 7ff7a659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976b5d0 0x55891977a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42894/0 (socket says 172.21.15.172:42894) 2024-09-10T08:57:59.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.389+0000 7ff7a659c640 1 -- 172.21.15.172:0/4124008625 learned_addr learned my addr 172.21.15.172:0/4124008625 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:59.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.389+0000 7ff79ffff640 1 --2- 172.21.15.172:0/4124008625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891976cb40 0x558919777440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:59.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.389+0000 7ff7a659c640 1 -- 172.21.15.172:0/4124008625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891977a780 msgr2=0x558919777aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:57:59.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.389+0000 7ff7a659c640 1 --2- 172.21.15.172:0/4124008625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891977a780 0x558919777aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.389+0000 7ff7a659c640 1 -- 172.21.15.172:0/4124008625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891976cb40 msgr2=0x558919777440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:59.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.389+0000 7ff7a659c640 1 --2- 172.21.15.172:0/4124008625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891976cb40 0x558919777440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.389+0000 7ff7a659c640 1 -- 172.21.15.172:0/4124008625 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589194423c0 con 0x55891976b5d0 2024-09-10T08:57:59.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.390+0000 7ff7a659c640 1 --2- 172.21.15.172:0/4124008625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976b5d0 0x55891977a140 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ff798006010 tx=0x7ff7980045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a16af12e633aac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:59.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.390+0000 7ff78dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:57:59.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.390+0000 7ff78dffb640 1 -- 172.21.15.172:0/4124008625 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798035040 con 0x55891976b5d0 2024-09-10T08:57:59.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.390+0000 7ff7af17d740 1 -- 172.21.15.172:0/4124008625 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7a001da50 con 0x55891976b5d0 2024-09-10T08:57:59.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.390+0000 7ff78dffb640 1 -- 172.21.15.172:0/4124008625 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff798004770 con 0x55891976b5d0 2024-09-10T08:57:59.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.390+0000 7ff78dffb640 1 -- 172.21.15.172:0/4124008625 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff798005610 con 0x55891976b5d0 2024-09-10T08:57:59.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.391+0000 7ff7af17d740 10 client.18921 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:59.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.391+0000 7ff7af17d740 1 -- 172.21.15.172:0/4124008625 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558919788f20 con 0x55891976b5d0 2024-09-10T08:57:59.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.391+0000 7ff78dffb640 1 -- 172.21.15.172:0/4124008625 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7a001da50 con 0x55891976b5d0 2024-09-10T08:57:59.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.391+0000 7ff78dffb640 1 -- 172.21.15.172:0/4124008625 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7ff798003c80 con 0x55891976b5d0 2024-09-10T08:57:59.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.392+0000 7ff7af17d740 10 client.18921 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:59.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.392+0000 7ff7af17d740 10 client.18921 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:57:59.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.392+0000 7ff7af17d740 20 client.18921 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:59.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.392+0000 7ff7af17d740 1 --2- 172.21.15.172:0/4124008625 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55891978a3e0 0x55891978c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.392+0000 7ff7af17d740 4 client.18921 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:57:59.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.392+0000 7ff7af17d740 1 -- 172.21.15.172:0/4124008625 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589196de860 con 0x55891978a3e0 2024-09-10T08:57:59.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.392+0000 7ff79ffff640 1 --2- 172.21.15.172:0/4124008625 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55891978a3e0 0x55891978c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:59.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.392+0000 7ff78dffb640 10 client.18921 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:57:59.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.393+0000 7ff79ffff640 1 --2- 172.21.15.172:0/4124008625 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55891978a3e0 0x55891978c850 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b8ffb50c53a8d58 server_cookie=746e6890276da94c in_seq=0 out_seq=0 2024-09-10T08:57:59.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.393+0000 7ff78dffb640 1 -- 172.21.15.172:0/4124008625 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589196de860 con 0x55891978a3e0 2024-09-10T08:57:59.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.393+0000 7ff78dffb640 10 client.18921 handle_command_reply: tid=0 2024-09-10T08:57:59.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.393+0000 7ff7af17d740 1 client.18921 shutdown 2024-09-10T08:57:59.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.393+0000 7ff7af17d740 1 -- 172.21.15.172:0/4124008625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976b5d0 msgr2=0x55891977a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:59.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.393+0000 7ff7af17d740 1 --2- 172.21.15.172:0/4124008625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976b5d0 0x55891977a140 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ff798006010 tx=0x7ff7980045a0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.394+0000 7ff7af17d740 1 -- 172.21.15.172:0/4124008625 shutdown_connections 2024-09-10T08:57:59.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.394+0000 7ff7af17d740 1 --2- 172.21.15.172:0/4124008625 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55891978a3e0 0x55891978c850 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.394+0000 7ff7af17d740 1 --2- 172.21.15.172:0/4124008625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891977a780 0x558919777aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.394+0000 7ff7af17d740 1 --2- 172.21.15.172:0/4124008625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891976cb40 0x558919777440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.394+0000 7ff78dffb640 0 client.18921 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:57:59.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.394+0000 7ff7af17d740 1 --2- 172.21.15.172:0/4124008625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976b5d0 0x55891977a140 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.394+0000 7ff7af17d740 1 -- 172.21.15.172:0/4124008625 >> 172.21.15.172:0/4124008625 conn(0x558919734310 msgr2=0x55891976d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:59.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.394+0000 7ff7af17d740 1 -- 172.21.15.172:0/4124008625 shutdown_connections 2024-09-10T08:57:59.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.394+0000 7ff7af17d740 1 -- 172.21.15.172:0/4124008625 wait complete. 2024-09-10T08:57:59.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.394+0000 7ff7af17d740 20 client.18921 trim_cache size 0 max 16384 2024-09-10T08:57:59.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.410+0000 7ff7af17d740 1 Processor -- start 2024-09-10T08:57:59.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff7af17d740 1 -- start start 2024-09-10T08:57:59.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff7af17d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891976b5d0 0x558919786720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff7af17d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976cb40 0x558919792120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff7af17d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891977a780 0x558919792660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff7af17d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81ac8e0 con 0x55891976cb40 2024-09-10T08:57:59.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff7af17d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81ae480 con 0x55891976b5d0 2024-09-10T08:57:59.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff7af17d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81acfb0 con 0x55891977a780 2024-09-10T08:57:59.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976cb40 0x558919792120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:59.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976cb40 0x558919792120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42910/0 (socket says 172.21.15.172:42910) 2024-09-10T08:57:59.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff79ffff640 1 -- 172.21.15.172:0/1682292279 learned_addr learned my addr 172.21.15.172:0/1682292279 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:59.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff7a659c640 1 --2- 172.21.15.172:0/1682292279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891976b5d0 0x558919786720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:59.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff7a6d9d640 1 --2- 172.21.15.172:0/1682292279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891977a780 0x558919792660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:59.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff79ffff640 1 -- 172.21.15.172:0/1682292279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891977a780 msgr2=0x558919792660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:59.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff79ffff640 1 --2- 172.21.15.172:0/1682292279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891977a780 0x558919792660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff79ffff640 1 -- 172.21.15.172:0/1682292279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891976b5d0 msgr2=0x558919786720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:59.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff79ffff640 1 --2- 172.21.15.172:0/1682292279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891976b5d0 0x558919786720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff79ffff640 1 -- 172.21.15.172:0/1682292279 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55891977a4b0 con 0x55891976cb40 2024-09-10T08:57:59.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff7a659c640 1 --2- 172.21.15.172:0/1682292279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891976b5d0 0x558919786720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:57:59.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.411+0000 7ff79ffff640 1 --2- 172.21.15.172:0/1682292279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976cb40 0x558919792120 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ff7900cc4a0 tx=0x7ff7900cc370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f193b32c6aea1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:59.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.412+0000 7ff78ffff640 1 -- 172.21.15.172:0/1682292279 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7900a3e70 con 0x55891976cb40 2024-09-10T08:57:59.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.412+0000 7ff78ffff640 1 -- 172.21.15.172:0/1682292279 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff79009b200 con 0x55891976cb40 2024-09-10T08:57:59.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.412+0000 7ff78ffff640 1 -- 172.21.15.172:0/1682292279 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7900a3c40 con 0x55891976cb40 2024-09-10T08:57:59.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.412+0000 7ff7af17d740 1 -- 172.21.15.172:0/1682292279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976cb40 msgr2=0x558919792120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:59.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.412+0000 7ff7af17d740 1 --2- 172.21.15.172:0/1682292279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976cb40 0x558919792120 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ff7900cc4a0 tx=0x7ff7900cc370 comp rx=0 tx=0).stop 2024-09-10T08:57:59.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.413+0000 7ff7af17d740 1 -- 172.21.15.172:0/1682292279 shutdown_connections 2024-09-10T08:57:59.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.413+0000 7ff7af17d740 1 --2- 172.21.15.172:0/1682292279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891977a780 0x558919792660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.413+0000 7ff7af17d740 1 --2- 172.21.15.172:0/1682292279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891976cb40 0x558919792120 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.413+0000 7ff7af17d740 1 --2- 172.21.15.172:0/1682292279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891976b5d0 0x558919786720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.413+0000 7ff7af17d740 1 -- 172.21.15.172:0/1682292279 >> 172.21.15.172:0/1682292279 conn(0x558919734310 msgr2=0x558919776530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:59.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.413+0000 7ff7af17d740 1 -- 172.21.15.172:0/1682292279 shutdown_connections 2024-09-10T08:57:59.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.413+0000 7ff7af17d740 1 -- 172.21.15.172:0/1682292279 wait complete. 2024-09-10T08:57:59.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.413+0000 7ff7af17d740 1 -- start start 2024-09-10T08:57:59.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.413+0000 7ff7af17d740 1 Processor -- start 2024-09-10T08:57:59.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.413+0000 7ff7af17d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891976b5d0 0x55891977e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.413+0000 7ff7af17d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891976cb40 0x55891977eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.413+0000 7ff7af17d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891977a780 0x55891977efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.413+0000 7ff7af17d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81addb0 con 0x55891977a780 2024-09-10T08:57:59.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.413+0000 7ff7af17d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81ad6e0 con 0x55891976cb40 2024-09-10T08:57:59.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.413+0000 7ff7af17d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a81aa8c0 con 0x55891976b5d0 2024-09-10T08:57:59.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.413+0000 7ff7af17d740 10 client.0 _ll_register_callbacks cb 0x7ff798009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:57:59.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.413+0000 7ff7a6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891977a780 0x55891977efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:59.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.414+0000 7ff7a659c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891976b5d0 0x55891977e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:59.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.414+0000 7ff7a6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891977a780 0x55891977efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42912/0 (socket says 172.21.15.172:42912) 2024-09-10T08:57:59.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.414+0000 7ff7a6d9d640 1 -- 172.21.15.172:0/2551969036 learned_addr learned my addr 172.21.15.172:0/2551969036 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:57:59.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.414+0000 7ff79ffff640 1 --2- 172.21.15.172:0/2551969036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891976cb40 0x55891977eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:59.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.414+0000 7ff7a659c640 1 -- 172.21.15.172:0/2551969036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891976cb40 msgr2=0x55891977eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:59.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.414+0000 7ff7a659c640 1 --2- 172.21.15.172:0/2551969036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891976cb40 0x55891977eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.414+0000 7ff7a659c640 1 -- 172.21.15.172:0/2551969036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891977a780 msgr2=0x55891977efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:57:59.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.414+0000 7ff7a659c640 1 --2- 172.21.15.172:0/2551969036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891977a780 0x55891977efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.414+0000 7ff7a659c640 1 -- 172.21.15.172:0/2551969036 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589197f40f0 con 0x55891976b5d0 2024-09-10T08:57:59.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.414+0000 7ff7a6d9d640 1 --2- 172.21.15.172:0/2551969036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891977a780 0x55891977efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:57:59.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.414+0000 7ff7a659c640 1 --2- 172.21.15.172:0/2551969036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891976b5d0 0x55891977e560 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff798031570 tx=0x7ff798004300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf0b587a07911411 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:57:59.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.414+0000 7ff78dffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:57:59.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.414+0000 7ff78dffb640 1 -- 172.21.15.172:0/2551969036 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798035030 con 0x55891976b5d0 2024-09-10T08:57:59.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.414+0000 7ff78dffb640 1 -- 172.21.15.172:0/2551969036 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ff798003800 con 0x55891976b5d0 2024-09-10T08:57:59.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.414+0000 7ff7af17d740 1 -- 172.21.15.172:0/2551969036 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff79009b200 con 0x55891976b5d0 2024-09-10T08:57:59.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.415+0000 7ff78dffb640 1 -- 172.21.15.172:0/2551969036 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff798060050 con 0x55891976b5d0 2024-09-10T08:57:59.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.415+0000 7ff7af17d740 10 client.27727 fetch_fsmap learned FSMap version 39 2024-09-10T08:57:59.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.415+0000 7ff7af17d740 1 -- 172.21.15.172:0/2551969036 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5589197f5350 con 0x55891976b5d0 2024-09-10T08:57:59.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.415+0000 7ff78dffb640 1 -- 172.21.15.172:0/2551969036 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff798041070 con 0x55891976b5d0 2024-09-10T08:57:59.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.415+0000 7ff78dffb640 1 -- 172.21.15.172:0/2551969036 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7ff798004c60 con 0x55891976b5d0 2024-09-10T08:57:59.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.415+0000 7ff7af17d740 10 client.27727 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:57:59.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.415+0000 7ff7af17d740 10 client.27727 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:57:59.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.415+0000 7ff7af17d740 20 client.27727 populate_metadata read hostname 'smithi172' 2024-09-10T08:57:59.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.415+0000 7ff7af17d740 1 --2- 172.21.15.172:0/2551969036 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5589197f5a60 0x5589197f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:57:59.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.415+0000 7ff7af17d740 4 client.27727 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:57:59.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.415+0000 7ff7af17d740 1 -- 172.21.15.172:0/2551969036 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff79009b200 con 0x5589197f5a60 2024-09-10T08:57:59.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.415+0000 7ff79ffff640 1 --2- 172.21.15.172:0/2551969036 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5589197f5a60 0x5589197f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:57:59.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.416+0000 7ff79ffff640 1 --2- 172.21.15.172:0/2551969036 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5589197f5a60 0x5589197f7e50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=59e7bda4930961de server_cookie=288c73af17ccaa7a in_seq=0 out_seq=0 2024-09-10T08:57:59.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.416+0000 7ff78dffb640 10 client.27727 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:57:59.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.416+0000 7ff78dffb640 1 -- 172.21.15.172:0/2551969036 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff79009b200 con 0x5589197f5a60 2024-09-10T08:57:59.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.416+0000 7ff78dffb640 10 client.27727 handle_command_reply: tid=0 2024-09-10T08:57:59.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.416+0000 7ff7af17d740 1 client.27727 shutdown 2024-09-10T08:57:59.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.416+0000 7ff7af17d740 1 -- 172.21.15.172:0/2551969036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891976b5d0 msgr2=0x55891977e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:59.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.416+0000 7ff7af17d740 1 --2- 172.21.15.172:0/2551969036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891976b5d0 0x55891977e560 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff798031570 tx=0x7ff798004300 comp rx=0 tx=0).stop 2024-09-10T08:57:59.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.416+0000 7ff7af17d740 1 -- 172.21.15.172:0/2551969036 shutdown_connections 2024-09-10T08:57:59.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.416+0000 7ff7af17d740 1 --2- 172.21.15.172:0/2551969036 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x5589197f5a60 0x5589197f7e50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.416+0000 7ff78dffb640 0 client.27727 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:57:59.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.416+0000 7ff7af17d740 1 --2- 172.21.15.172:0/2551969036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55891977a780 0x55891977efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.416+0000 7ff7af17d740 1 --2- 172.21.15.172:0/2551969036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55891976cb40 0x55891977eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.416+0000 7ff7af17d740 1 --2- 172.21.15.172:0/2551969036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55891976b5d0 0x55891977e560 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.416+0000 7ff7af17d740 1 -- 172.21.15.172:0/2551969036 >> 172.21.15.172:0/2551969036 conn(0x558919734310 msgr2=0x558919766500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:59.801 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:57:59.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.416+0000 7ff7af17d740 1 -- 172.21.15.172:0/2551969036 shutdown_connections 2024-09-10T08:57:59.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.416+0000 7ff7af17d740 1 -- 172.21.15.172:0/2551969036 wait complete. 2024-09-10T08:57:59.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.416+0000 7ff7af17d740 20 client.27727 trim_cache size 0 max 16384 2024-09-10T08:57:59.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.417+0000 7ff7acda4640 1 -- 172.21.15.172:0/2737771235 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7940a2c00 msgr2=0x7ff7940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:59.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.417+0000 7ff7acda4640 1 --2- 172.21.15.172:0/2737771235 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7940a2c00 0x7ff7940a50c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0021000 tx=0x7ff7a001e000 comp rx=0 tx=0).stop 2024-09-10T08:57:59.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.417+0000 7ff7acda4640 1 -- 172.21.15.172:0/2737771235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7a8102790 msgr2=0x7ff7a8198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:57:59.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.417+0000 7ff7acda4640 1 --2- 172.21.15.172:0/2737771235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7a8102790 0x7ff7a8198850 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7ff79000d950 tx=0x7ff79000de10 comp rx=0 tx=0).stop 2024-09-10T08:57:59.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.417+0000 7ff7acda4640 1 -- 172.21.15.172:0/2737771235 shutdown_connections 2024-09-10T08:57:59.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.417+0000 7ff7acda4640 1 --2- 172.21.15.172:0/2737771235 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ff7940a2c00 0x7ff7940a50c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.417+0000 7ff7acda4640 1 --2- 172.21.15.172:0/2737771235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7a8106f10 0x7ff7a819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.417+0000 7ff7acda4640 1 --2- 172.21.15.172:0/2737771235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7a8102790 0x7ff7a8198850 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.417+0000 7ff7acda4640 1 --2- 172.21.15.172:0/2737771235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a8101de0 0x7ff7a8198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:57:59.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.417+0000 7ff7acda4640 1 -- 172.21.15.172:0/2737771235 >> 172.21.15.172:0/2737771235 conn(0x7ff7a80fd8e0 msgr2=0x7ff7a80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:57:59.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.417+0000 7ff7acda4640 1 -- 172.21.15.172:0/2737771235 shutdown_connections 2024-09-10T08:57:59.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:57:59.417+0000 7ff7acda4640 1 -- 172.21.15.172:0/2737771235 wait complete. 2024-09-10T08:57:59.803 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:57:59 smithi191 ceph-mon[35314]: pgmap v703: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 26 MiB/s wr, 23 op/s 2024-09-10T08:58:00.527 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:58:00.802 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:58:01.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.341+0000 7f93a86f8640 1 -- 172.21.15.172:0/3839818082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a01012f0 msgr2=0x7f93a010dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:01.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.341+0000 7f93a86f8640 1 --2- 172.21.15.172:0/3839818082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a01012f0 0x7f93a010dfb0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f9390009900 tx=0x7f939002a410 comp rx=0 tx=0).stop 2024-09-10T08:58:01.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.341+0000 7f93a86f8640 1 -- 172.21.15.172:0/3839818082 shutdown_connections 2024-09-10T08:58:01.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.341+0000 7f93a86f8640 1 --2- 172.21.15.172:0/3839818082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93a010e4f0 0x7f93a01108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.341+0000 7f93a86f8640 1 --2- 172.21.15.172:0/3839818082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a01012f0 0x7f93a010dfb0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.341+0000 7f93a86f8640 1 --2- 172.21.15.172:0/3839818082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93a0100940 0x7f93a0100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.341+0000 7f93a86f8640 1 -- 172.21.15.172:0/3839818082 >> 172.21.15.172:0/3839818082 conn(0x7f93a00fc7d0 msgr2=0x7f93a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:01.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.341+0000 7f93a86f8640 1 -- 172.21.15.172:0/3839818082 shutdown_connections 2024-09-10T08:58:01.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.341+0000 7f93a86f8640 1 -- 172.21.15.172:0/3839818082 wait complete. 2024-09-10T08:58:01.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.342+0000 7f93a86f8640 1 Processor -- start 2024-09-10T08:58:01.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.342+0000 7f93a86f8640 1 -- start start 2024-09-10T08:58:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.342+0000 7f93a86f8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93a0100940 0x7f93a019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.342+0000 7f93a86f8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a01012f0 0x7f93a019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.342+0000 7f93a86f8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93a010e4f0 0x7f93a019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.342+0000 7f93a86f8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93a0114110 con 0x7f93a01012f0 2024-09-10T08:58:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.342+0000 7f93a86f8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f93a0113f90 con 0x7f93a0100940 2024-09-10T08:58:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.342+0000 7f93a86f8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f93a0114290 con 0x7f93a010e4f0 2024-09-10T08:58:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.342+0000 7f93a5c93640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a01012f0 0x7f93a019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.342+0000 7f93a5c93640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a01012f0 0x7f93a019ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42922/0 (socket says 172.21.15.172:42922) 2024-09-10T08:58:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.342+0000 7f93a5c93640 1 -- 172.21.15.172:0/1472840257 learned_addr learned my addr 172.21.15.172:0/1472840257 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.342+0000 7f93a6494640 1 --2- 172.21.15.172:0/1472840257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93a0100940 0x7f93a019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.342+0000 7f93a6c95640 1 --2- 172.21.15.172:0/1472840257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93a010e4f0 0x7f93a019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.343+0000 7f93a5c93640 1 -- 172.21.15.172:0/1472840257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93a010e4f0 msgr2=0x7f93a019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.343+0000 7f93a5c93640 1 --2- 172.21.15.172:0/1472840257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93a010e4f0 0x7f93a019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.343+0000 7f93a5c93640 1 -- 172.21.15.172:0/1472840257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93a0100940 msgr2=0x7f93a019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.343+0000 7f93a5c93640 1 --2- 172.21.15.172:0/1472840257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93a0100940 0x7f93a019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.343+0000 7f93a5c93640 1 -- 172.21.15.172:0/1472840257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a019f6c0 con 0x7f93a01012f0 2024-09-10T08:58:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.343+0000 7f93a5c93640 1 --2- 172.21.15.172:0/1472840257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a01012f0 0x7f93a019ac50 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f9390006010 tx=0x7f9390004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.343+0000 7f93977fe640 1 -- 172.21.15.172:0/1472840257 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939003f030 con 0x7f93a01012f0 2024-09-10T08:58:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.343+0000 7f93977fe640 1 -- 172.21.15.172:0/1472840257 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93900035f0 con 0x7f93a01012f0 2024-09-10T08:58:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.343+0000 7f93977fe640 1 -- 172.21.15.172:0/1472840257 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390035070 con 0x7f93a01012f0 2024-09-10T08:58:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.343+0000 7f93a86f8640 1 -- 172.21.15.172:0/1472840257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a019f950 con 0x7f93a01012f0 2024-09-10T08:58:01.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.343+0000 7f93a86f8640 1 -- 172.21.15.172:0/1472840257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a01a7290 con 0x7f93a01012f0 2024-09-10T08:58:01.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.346+0000 7f93977fe640 1 -- 172.21.15.172:0/1472840257 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9390003b10 con 0x7f93a01012f0 2024-09-10T08:58:01.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.346+0000 7f93aaad1740 1 Processor -- start 2024-09-10T08:58:01.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.347+0000 7f93aaad1740 1 -- start start 2024-09-10T08:58:01.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.347+0000 7f93977fe640 1 --2- 172.21.15.172:0/1472840257 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f938c0a2c00 0x7f938c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.347+0000 7f93aaad1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9c5b40 0x55e55f9c4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.347+0000 7f93aaad1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c45d0 0x55e55f9c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.347+0000 7f93aaad1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9cf370 0x55e55f9d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.347+0000 7f93aaad1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e55f92a2f0 con 0x55e55f9c5b40 2024-09-10T08:58:01.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.347+0000 7f93aaad1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e55f8d1cf0 con 0x55e55f9cf370 2024-09-10T08:58:01.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.347+0000 7f93aaad1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e55f9262b0 con 0x55e55f9c45d0 2024-09-10T08:58:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.347+0000 7f93a6c95640 1 --2- 172.21.15.172:0/1472840257 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f938c0a2c00 0x7f938c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.347+0000 7f93977fe640 1 -- 172.21.15.172:0/1472840257 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f93900ef940 con 0x7f93a01012f0 2024-09-10T08:58:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.347+0000 7f93a6494640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9c5b40 0x55e55f9c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.348+0000 7f93a6494640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9c5b40 0x55e55f9c4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42938/0 (socket says 172.21.15.172:42938) 2024-09-10T08:58:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.348+0000 7f93a6494640 1 -- 172.21.15.172:0/1640836181 learned_addr learned my addr 172.21.15.172:0/1640836181 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.348+0000 7f93a6c95640 1 --2- 172.21.15.172:0/1640836181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9cf370 0x55e55f9d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.348+0000 7f93a5c93640 1 --2- 172.21.15.172:0/1640836181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c45d0 0x55e55f9c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.348+0000 7f93a6c95640 1 --2- 172.21.15.172:0/1472840257 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f938c0a2c00 0x7f938c0a50c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f93980070b0 tx=0x7f939800c7f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.348+0000 7f93a6494640 1 -- 172.21.15.172:0/1640836181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c45d0 msgr2=0x55e55f9c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.348+0000 7f93a6494640 1 --2- 172.21.15.172:0/1640836181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c45d0 0x55e55f9c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.348+0000 7f93a6494640 1 -- 172.21.15.172:0/1640836181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9cf370 msgr2=0x55e55f9d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.348+0000 7f93a6494640 1 --2- 172.21.15.172:0/1640836181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9cf370 0x55e55f9d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.348+0000 7f93a6494640 1 -- 172.21.15.172:0/1640836181 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e55f8b63a0 con 0x55e55f9c5b40 2024-09-10T08:58:01.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.348+0000 7f93a6494640 1 --2- 172.21.15.172:0/1640836181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9c5b40 0x55e55f9c4090 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f939c009a70 tx=0x7f939c009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6207e98462f6997 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:01.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.348+0000 7f9394ff9640 1 -- 172.21.15.172:0/1640836181 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c017070 con 0x55e55f9c5b40 2024-09-10T08:58:01.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.348+0000 7f9394ff9640 1 -- 172.21.15.172:0/1640836181 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f939c013470 con 0x55e55f9c5b40 2024-09-10T08:58:01.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.348+0000 7f9394ff9640 1 -- 172.21.15.172:0/1640836181 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c012630 con 0x55e55f9c5b40 2024-09-10T08:58:01.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.349+0000 7f93aaad1740 1 -- 172.21.15.172:0/1640836181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9c5b40 msgr2=0x55e55f9c4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:01.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.349+0000 7f93aaad1740 1 --2- 172.21.15.172:0/1640836181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9c5b40 0x55e55f9c4090 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f939c009a70 tx=0x7f939c009f30 comp rx=0 tx=0).stop 2024-09-10T08:58:01.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.349+0000 7f93aaad1740 1 -- 172.21.15.172:0/1640836181 shutdown_connections 2024-09-10T08:58:01.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.349+0000 7f93aaad1740 1 --2- 172.21.15.172:0/1640836181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9cf370 0x55e55f9d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.349+0000 7f93aaad1740 1 --2- 172.21.15.172:0/1640836181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c45d0 0x55e55f9c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.349+0000 7f93aaad1740 1 --2- 172.21.15.172:0/1640836181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9c5b40 0x55e55f9c4090 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.350+0000 7f93aaad1740 1 -- 172.21.15.172:0/1640836181 >> 172.21.15.172:0/1640836181 conn(0x55e55f98d310 msgr2=0x55e55f9c0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:01.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.350+0000 7f93aaad1740 1 -- 172.21.15.172:0/1640836181 shutdown_connections 2024-09-10T08:58:01.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.350+0000 7f93aaad1740 1 -- 172.21.15.172:0/1640836181 wait complete. 2024-09-10T08:58:01.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.350+0000 7f93aaad1740 1 -- start start 2024-09-10T08:58:01.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.350+0000 7f93aaad1740 1 Processor -- start 2024-09-10T08:58:01.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.350+0000 7f93aaad1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c45d0 0x55e55f9cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.350+0000 7f93aaad1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9c5b40 0x55e55f9cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.350+0000 7f93aaad1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 0x55e55f9cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.350+0000 7f93aaad1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e55f9d2070 con 0x55e55f9cf370 2024-09-10T08:58:01.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.350+0000 7f93aaad1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e55f8e5060 con 0x55e55f9c5b40 2024-09-10T08:58:01.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.350+0000 7f93aaad1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e55f9d21f0 con 0x55e55f9c45d0 2024-09-10T08:58:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.351+0000 7f93aaad1740 10 client.0 _ll_register_callbacks cb 0x55e55f6388c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.351+0000 7f93a6494640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c45d0 0x55e55f9cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.351+0000 7f93a6494640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c45d0 0x55e55f9cecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:54078/0 (socket says 172.21.15.172:54078) 2024-09-10T08:58:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.351+0000 7f93a6494640 1 -- 172.21.15.172:0/1460591528 learned_addr learned my addr 172.21.15.172:0/1460591528 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.351+0000 7f93a6c95640 1 --2- 172.21.15.172:0/1460591528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 0x55e55f9cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.351+0000 7f93a5c93640 1 --2- 172.21.15.172:0/1460591528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9c5b40 0x55e55f9cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.351+0000 7f93a6494640 1 -- 172.21.15.172:0/1460591528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9c5b40 msgr2=0x55e55f9cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.351+0000 7f93a6494640 1 --2- 172.21.15.172:0/1460591528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9c5b40 0x55e55f9cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.351+0000 7f93a6494640 1 -- 172.21.15.172:0/1460591528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 msgr2=0x55e55f9cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.351+0000 7f93a6494640 1 --2- 172.21.15.172:0/1460591528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 0x55e55f9cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.351+0000 7f93a6494640 1 -- 172.21.15.172:0/1460591528 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e55f69b3c0 con 0x55e55f9c45d0 2024-09-10T08:58:01.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.351+0000 7f93a6c95640 1 --2- 172.21.15.172:0/1460591528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 0x55e55f9cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.352+0000 7f93a6494640 1 --2- 172.21.15.172:0/1460591528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c45d0 0x55e55f9cecd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f939c004810 tx=0x7f939c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2c00bc2f18af3e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.352+0000 7f9389ffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:58:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.352+0000 7f9389ffb640 1 -- 172.21.15.172:0/1460591528 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c017050 con 0x55e55f9c45d0 2024-09-10T08:58:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.352+0000 7f9389ffb640 1 -- 172.21.15.172:0/1460591528 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f939c01c650 con 0x55e55f9c45d0 2024-09-10T08:58:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.352+0000 7f93aaad1740 1 -- 172.21.15.172:0/1460591528 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f939c013470 con 0x55e55f9c45d0 2024-09-10T08:58:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.352+0000 7f9389ffb640 1 -- 172.21.15.172:0/1460591528 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f939c02f050 con 0x55e55f9c45d0 2024-09-10T08:58:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.352+0000 7f93aaad1740 10 client.27736 fetch_fsmap learned FSMap version 39 2024-09-10T08:58:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.352+0000 7f93aaad1740 1 -- 172.21.15.172:0/1460591528 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e55f9dda80 con 0x55e55f9c45d0 2024-09-10T08:58:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.352+0000 7f9389ffb640 1 -- 172.21.15.172:0/1460591528 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f939c013470 con 0x55e55f9c45d0 2024-09-10T08:58:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.353+0000 7f9389ffb640 1 -- 172.21.15.172:0/1460591528 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f939c0267b0 con 0x55e55f9c45d0 2024-09-10T08:58:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.353+0000 7f93aaad1740 10 client.27736 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:58:01.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.353+0000 7f93aaad1740 10 client.27736 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:58:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.353+0000 7f93aaad1740 20 client.27736 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.353+0000 7f93aaad1740 1 --2- 172.21.15.172:0/1460591528 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e55f9dee80 0x55e55f9e1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.353+0000 7f93aaad1740 4 client.27736 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:58:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.353+0000 7f93aaad1740 1 -- 172.21.15.172:0/1460591528 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e55f937860 con 0x55e55f9dee80 2024-09-10T08:58:01.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.353+0000 7f93a5c93640 1 --2- 172.21.15.172:0/1460591528 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e55f9dee80 0x55e55f9e1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.353+0000 7f93a5c93640 1 --2- 172.21.15.172:0/1460591528 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e55f9dee80 0x55e55f9e1270 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b14608a094ab62ec server_cookie=cb5444e6e5a180d4 in_seq=0 out_seq=0 2024-09-10T08:58:01.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.353+0000 7f9389ffb640 10 client.27736 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:58:01.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.354+0000 7f9389ffb640 1 -- 172.21.15.172:0/1460591528 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e55f937860 con 0x55e55f9dee80 2024-09-10T08:58:01.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.354+0000 7f9389ffb640 10 client.27736 handle_command_reply: tid=0 2024-09-10T08:58:01.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.354+0000 7f93aaad1740 1 client.27736 shutdown 2024-09-10T08:58:01.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.354+0000 7f93aaad1740 1 -- 172.21.15.172:0/1460591528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c45d0 msgr2=0x55e55f9cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:01.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.354+0000 7f93aaad1740 1 --2- 172.21.15.172:0/1460591528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c45d0 0x55e55f9cecd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f939c004810 tx=0x7f939c00edf0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.354+0000 7f93aaad1740 1 -- 172.21.15.172:0/1460591528 shutdown_connections 2024-09-10T08:58:01.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.354+0000 7f93aaad1740 1 --2- 172.21.15.172:0/1460591528 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e55f9dee80 0x55e55f9e1270 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.354+0000 7f93aaad1740 1 --2- 172.21.15.172:0/1460591528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 0x55e55f9cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.354+0000 7f93aaad1740 1 --2- 172.21.15.172:0/1460591528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9c5b40 0x55e55f9cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.354+0000 7f93aaad1740 1 --2- 172.21.15.172:0/1460591528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c45d0 0x55e55f9cecd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.354+0000 7f93aaad1740 1 -- 172.21.15.172:0/1460591528 >> 172.21.15.172:0/1460591528 conn(0x55e55f98d310 msgr2=0x55e55f9c6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:01.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.354+0000 7f9389ffb640 0 client.27736 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:58:01.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.355+0000 7f93aaad1740 1 -- 172.21.15.172:0/1460591528 shutdown_connections 2024-09-10T08:58:01.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.355+0000 7f93aaad1740 1 -- 172.21.15.172:0/1460591528 wait complete. 2024-09-10T08:58:01.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.355+0000 7f93aaad1740 20 client.27736 trim_cache size 0 max 16384 2024-09-10T08:58:01.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.369+0000 7f93aaad1740 1 Processor -- start 2024-09-10T08:58:01.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.369+0000 7f93aaad1740 1 -- start start 2024-09-10T08:58:01.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93aaad1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9c45d0 0x55e55f9db310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93aaad1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c5b40 0x55e55f9cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93aaad1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 0x55e55f9caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93aaad1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01b0ea0 con 0x55e55f9cf370 2024-09-10T08:58:01.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93aaad1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01b2a40 con 0x55e55f9c45d0 2024-09-10T08:58:01.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93aaad1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01b1570 con 0x55e55f9c5b40 2024-09-10T08:58:01.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93a6c95640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 0x55e55f9caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93a6c95640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 0x55e55f9caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42954/0 (socket says 172.21.15.172:42954) 2024-09-10T08:58:01.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93a6c95640 1 -- 172.21.15.172:0/1665794618 learned_addr learned my addr 172.21.15.172:0/1665794618 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:01.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93a5c93640 1 --2- 172.21.15.172:0/1665794618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c5b40 0x55e55f9cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93a6494640 1 --2- 172.21.15.172:0/1665794618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9c45d0 0x55e55f9db310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93a6c95640 1 -- 172.21.15.172:0/1665794618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c5b40 msgr2=0x55e55f9cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:01.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93a6c95640 1 --2- 172.21.15.172:0/1665794618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c5b40 0x55e55f9cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93a6c95640 1 -- 172.21.15.172:0/1665794618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9c45d0 msgr2=0x55e55f9db310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:01.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93a6c95640 1 --2- 172.21.15.172:0/1665794618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9c45d0 0x55e55f9db310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93a6c95640 1 -- 172.21.15.172:0/1665794618 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e55f9d21a0 con 0x55e55f9cf370 2024-09-10T08:58:01.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93a5c93640 1 --2- 172.21.15.172:0/1665794618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c5b40 0x55e55f9cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:01.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.370+0000 7f93a6c95640 1 --2- 172.21.15.172:0/1665794618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 0x55e55f9caf60 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f9398019b80 tx=0x7f9398019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcc2f28592b31ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:01.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.371+0000 7f938bfff640 1 -- 172.21.15.172:0/1665794618 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939802a070 con 0x55e55f9cf370 2024-09-10T08:58:01.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.371+0000 7f938bfff640 1 -- 172.21.15.172:0/1665794618 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9398018d10 con 0x55e55f9cf370 2024-09-10T08:58:01.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.371+0000 7f938bfff640 1 -- 172.21.15.172:0/1665794618 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398024690 con 0x55e55f9cf370 2024-09-10T08:58:01.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.371+0000 7f93aaad1740 1 -- 172.21.15.172:0/1665794618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 msgr2=0x55e55f9caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:01.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.371+0000 7f93aaad1740 1 --2- 172.21.15.172:0/1665794618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 0x55e55f9caf60 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f9398019b80 tx=0x7f9398019000 comp rx=0 tx=0).stop 2024-09-10T08:58:01.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.371+0000 7f93aaad1740 1 -- 172.21.15.172:0/1665794618 shutdown_connections 2024-09-10T08:58:01.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.371+0000 7f93aaad1740 1 --2- 172.21.15.172:0/1665794618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 0x55e55f9caf60 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.371+0000 7f93aaad1740 1 --2- 172.21.15.172:0/1665794618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c5b40 0x55e55f9cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.371+0000 7f93aaad1740 1 --2- 172.21.15.172:0/1665794618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9c45d0 0x55e55f9db310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.371+0000 7f93aaad1740 1 -- 172.21.15.172:0/1665794618 >> 172.21.15.172:0/1665794618 conn(0x55e55f98d310 msgr2=0x55e55f9ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:01.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.371+0000 7f93aaad1740 1 -- 172.21.15.172:0/1665794618 shutdown_connections 2024-09-10T08:58:01.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.372+0000 7f93aaad1740 1 -- 172.21.15.172:0/1665794618 wait complete. 2024-09-10T08:58:01.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.372+0000 7f93aaad1740 1 -- start start 2024-09-10T08:58:01.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.372+0000 7f93aaad1740 1 Processor -- start 2024-09-10T08:58:01.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.372+0000 7f93aaad1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9c45d0 0x55e55f9d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.372+0000 7f93aaad1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c5b40 0x55e55f9d2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.372+0000 7f93aaad1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 0x55e55fa47890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.372+0000 7f93aaad1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01b2370 con 0x55e55f9cf370 2024-09-10T08:58:01.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.372+0000 7f93aaad1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01b1ca0 con 0x55e55f9c45d0 2024-09-10T08:58:01.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.372+0000 7f93aaad1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01aee80 con 0x55e55f9c5b40 2024-09-10T08:58:01.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.372+0000 7f93aaad1740 10 client.0 _ll_register_callbacks cb 0x7f939c0098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:01.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.372+0000 7f93a6c95640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 0x55e55fa47890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.373+0000 7f93a6c95640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 0x55e55fa47890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42970/0 (socket says 172.21.15.172:42970) 2024-09-10T08:58:01.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.373+0000 7f93a6c95640 1 -- 172.21.15.172:0/2003633607 learned_addr learned my addr 172.21.15.172:0/2003633607 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:01.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.373+0000 7f93a5c93640 1 --2- 172.21.15.172:0/2003633607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c5b40 0x55e55f9d2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.373+0000 7f93a6494640 1 --2- 172.21.15.172:0/2003633607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9c45d0 0x55e55f9d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.373+0000 7f93a6c95640 1 -- 172.21.15.172:0/2003633607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c5b40 msgr2=0x55e55f9d2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.373+0000 7f93a6c95640 1 --2- 172.21.15.172:0/2003633607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c5b40 0x55e55f9d2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.373+0000 7f93a6c95640 1 -- 172.21.15.172:0/2003633607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9c45d0 msgr2=0x55e55f9d29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.373+0000 7f93a6c95640 1 --2- 172.21.15.172:0/2003633607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9c45d0 0x55e55f9d29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.373+0000 7f93a6c95640 1 -- 172.21.15.172:0/2003633607 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e55fa47dd0 con 0x55e55f9cf370 2024-09-10T08:58:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.373+0000 7f93a6c95640 1 --2- 172.21.15.172:0/2003633607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 0x55e55fa47890 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f939801b010 tx=0x7f9398025950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8258499cdb32adfb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.373+0000 7f9389ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.373+0000 7f9389ffb640 1 -- 172.21.15.172:0/2003633607 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398034070 con 0x55e55f9cf370 2024-09-10T08:58:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.373+0000 7f9389ffb640 1 -- 172.21.15.172:0/2003633607 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f939802f670 con 0x55e55f9cf370 2024-09-10T08:58:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.373+0000 7f93aaad1740 1 -- 172.21.15.172:0/2003633607 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9398018d10 con 0x55e55f9cf370 2024-09-10T08:58:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.374+0000 7f9389ffb640 1 -- 172.21.15.172:0/2003633607 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9398040050 con 0x55e55f9cf370 2024-09-10T08:58:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.374+0000 7f93aaad1740 10 client.18945 fetch_fsmap learned FSMap version 39 2024-09-10T08:58:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.374+0000 7f93aaad1740 1 -- 172.21.15.172:0/2003633607 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e55fa490d0 con 0x55e55f9cf370 2024-09-10T08:58:01.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.374+0000 7f9389ffb640 1 -- 172.21.15.172:0/2003633607 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9398025b40 con 0x55e55f9cf370 2024-09-10T08:58:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.374+0000 7f9389ffb640 1 -- 172.21.15.172:0/2003633607 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f93980305b0 con 0x55e55f9cf370 2024-09-10T08:58:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.374+0000 7f93aaad1740 10 client.18945 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T08:58:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.374+0000 7f93aaad1740 10 client.18945 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:58:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.374+0000 7f93aaad1740 20 client.18945 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.374+0000 7f93aaad1740 1 --2- 172.21.15.172:0/2003633607 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e55fa49740 0x55e55fa4bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.374+0000 7f93aaad1740 4 client.18945 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:58:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.374+0000 7f93aaad1740 1 -- 172.21.15.172:0/2003633607 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9398018d10 con 0x55e55fa49740 2024-09-10T08:58:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.374+0000 7f93a6494640 1 --2- 172.21.15.172:0/2003633607 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e55fa49740 0x55e55fa4bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.375+0000 7f93a6494640 1 --2- 172.21.15.172:0/2003633607 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e55fa49740 0x55e55fa4bb60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21de3e9184433099 server_cookie=8b8f72353eea7d04 in_seq=0 out_seq=0 2024-09-10T08:58:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.375+0000 7f9389ffb640 10 client.18945 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:58:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.375+0000 7f9389ffb640 1 -- 172.21.15.172:0/2003633607 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9398018d10 con 0x55e55fa49740 2024-09-10T08:58:01.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.375+0000 7f9389ffb640 10 client.18945 handle_command_reply: tid=0 2024-09-10T08:58:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.375+0000 7f93aaad1740 1 client.18945 shutdown 2024-09-10T08:58:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.375+0000 7f93aaad1740 1 -- 172.21.15.172:0/2003633607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 msgr2=0x55e55fa47890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.375+0000 7f93aaad1740 1 --2- 172.21.15.172:0/2003633607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 0x55e55fa47890 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f939801b010 tx=0x7f9398025950 comp rx=0 tx=0).stop 2024-09-10T08:58:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.375+0000 7f93aaad1740 1 -- 172.21.15.172:0/2003633607 shutdown_connections 2024-09-10T08:58:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.375+0000 7f93aaad1740 1 --2- 172.21.15.172:0/2003633607 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55e55fa49740 0x55e55fa4bb60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.375+0000 7f93aaad1740 1 --2- 172.21.15.172:0/2003633607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e55f9cf370 0x55e55fa47890 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.375+0000 7f9389ffb640 0 client.18945 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:58:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.375+0000 7f93aaad1740 1 --2- 172.21.15.172:0/2003633607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e55f9c5b40 0x55e55f9d2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.375+0000 7f93aaad1740 1 --2- 172.21.15.172:0/2003633607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e55f9c45d0 0x55e55f9d29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.375+0000 7f93aaad1740 1 -- 172.21.15.172:0/2003633607 >> 172.21.15.172:0/2003633607 conn(0x55e55f98d310 msgr2=0x55e55f9bfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.375+0000 7f93aaad1740 1 -- 172.21.15.172:0/2003633607 shutdown_connections 2024-09-10T08:58:01.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.375+0000 7f93aaad1740 1 -- 172.21.15.172:0/2003633607 wait complete. 2024-09-10T08:58:01.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.375+0000 7f93aaad1740 20 client.18945 trim_cache size 0 max 16384 2024-09-10T08:58:01.382 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:58:01.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.376+0000 7f93a86f8640 1 -- 172.21.15.172:0/1472840257 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f938c0a2c00 msgr2=0x7f938c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:01.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.376+0000 7f93a86f8640 1 --2- 172.21.15.172:0/1472840257 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f938c0a2c00 0x7f938c0a50c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f93980070b0 tx=0x7f939800c7f0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.376+0000 7f93a86f8640 1 -- 172.21.15.172:0/1472840257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a01012f0 msgr2=0x7f93a019ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:01.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.376+0000 7f93a86f8640 1 --2- 172.21.15.172:0/1472840257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a01012f0 0x7f93a019ac50 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f9390006010 tx=0x7f9390004270 comp rx=0 tx=0).stop 2024-09-10T08:58:01.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.376+0000 7f93a86f8640 1 -- 172.21.15.172:0/1472840257 shutdown_connections 2024-09-10T08:58:01.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.376+0000 7f93a86f8640 1 --2- 172.21.15.172:0/1472840257 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f938c0a2c00 0x7f938c0a50c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.376+0000 7f93a86f8640 1 --2- 172.21.15.172:0/1472840257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93a010e4f0 0x7f93a019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.376+0000 7f93a86f8640 1 --2- 172.21.15.172:0/1472840257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93a01012f0 0x7f93a019ac50 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.376+0000 7f93a86f8640 1 --2- 172.21.15.172:0/1472840257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93a0100940 0x7f93a019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:01.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.376+0000 7f93a86f8640 1 -- 172.21.15.172:0/1472840257 >> 172.21.15.172:0/1472840257 conn(0x7f93a00fc7d0 msgr2=0x7f93a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:01.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.376+0000 7f93a86f8640 1 -- 172.21.15.172:0/1472840257 shutdown_connections 2024-09-10T08:58:01.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:01.376+0000 7f93a86f8640 1 -- 172.21.15.172:0/1472840257 wait complete. 2024-09-10T08:58:01.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:01 smithi172 ceph-mon[30854]: pgmap v704: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-09-10T08:58:01.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:01 smithi174 ceph-mon[34068]: pgmap v704: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-09-10T08:58:01.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:01 smithi191 ceph-mon[35314]: pgmap v704: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-09-10T08:58:02.338 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:58:02.620 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:58:03.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.157+0000 7fd1f95a1640 1 -- 172.21.15.172:0/772172491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f4102ef0 msgr2=0x7fd1f410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:03.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.157+0000 7fd1f95a1640 1 --2- 172.21.15.172:0/772172491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f4102ef0 0x7fd1f410da40 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009ca0 tx=0x7fd1ec02a950 comp rx=0 tx=0).stop 2024-09-10T08:58:03.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.157+0000 7fd1f95a1640 1 -- 172.21.15.172:0/772172491 shutdown_connections 2024-09-10T08:58:03.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.157+0000 7fd1f95a1640 1 --2- 172.21.15.172:0/772172491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f4102ef0 0x7fd1f410da40 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.157+0000 7fd1f95a1640 1 --2- 172.21.15.172:0/772172491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1f4102570 0x7fd1f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.157+0000 7fd1f95a1640 1 --2- 172.21.15.172:0/772172491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1f4100e20 0x7fd1f4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.157+0000 7fd1f95a1640 1 -- 172.21.15.172:0/772172491 >> 172.21.15.172:0/772172491 conn(0x7fd1f40fc7d0 msgr2=0x7fd1f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:03.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.157+0000 7fd1f95a1640 1 -- 172.21.15.172:0/772172491 shutdown_connections 2024-09-10T08:58:03.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.157+0000 7fd1f95a1640 1 -- 172.21.15.172:0/772172491 wait complete. 2024-09-10T08:58:03.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.158+0000 7fd1f95a1640 1 Processor -- start 2024-09-10T08:58:03.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.158+0000 7fd1f95a1640 1 -- start start 2024-09-10T08:58:03.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.158+0000 7fd1f95a1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1f4100e20 0x7fd1f40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.158+0000 7fd1f95a1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f4102570 0x7fd1f40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.158+0000 7fd1f95a1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1f4102ef0 0x7fd1f40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.158+0000 7fd1f95a1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4110e30 con 0x7fd1f4102570 2024-09-10T08:58:03.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.158+0000 7fd1f95a1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4110cb0 con 0x7fd1f4100e20 2024-09-10T08:58:03.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.158+0000 7fd1f95a1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4110fb0 con 0x7fd1f4102ef0 2024-09-10T08:58:03.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.158+0000 7fd1ebfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f4102570 0x7fd1f40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:03.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.158+0000 7fd1ebfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f4102570 0x7fd1f40fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42984/0 (socket says 172.21.15.172:42984) 2024-09-10T08:58:03.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.158+0000 7fd1ebfff640 1 -- 172.21.15.172:0/111129049 learned_addr learned my addr 172.21.15.172:0/111129049 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:03.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.159+0000 7fd1f2ffd640 1 --2- 172.21.15.172:0/111129049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1f4100e20 0x7fd1f40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:03.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.159+0000 7fd1ebfff640 1 -- 172.21.15.172:0/111129049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1f4102ef0 msgr2=0x7fd1f40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:03.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.159+0000 7fd1ebfff640 1 --2- 172.21.15.172:0/111129049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1f4102ef0 0x7fd1f40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.159+0000 7fd1ebfff640 1 -- 172.21.15.172:0/111129049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1f4100e20 msgr2=0x7fd1f40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:03.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.159+0000 7fd1ebfff640 1 --2- 172.21.15.172:0/111129049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1f4100e20 0x7fd1f40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.159+0000 7fd1ebfff640 1 -- 172.21.15.172:0/111129049 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f41a6830 con 0x7fd1f4102570 2024-09-10T08:58:03.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.159+0000 7fd1ebfff640 1 --2- 172.21.15.172:0/111129049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f4102570 0x7fd1f40fd1c0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00e9f0 tx=0x7fd1dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:03.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.159+0000 7fd1f0ff9640 1 -- 172.21.15.172:0/111129049 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc00cd80 con 0x7fd1f4102570 2024-09-10T08:58:03.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.159+0000 7fd1f0ff9640 1 -- 172.21.15.172:0/111129049 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1dc0108e0 con 0x7fd1f4102570 2024-09-10T08:58:03.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.159+0000 7fd1f95a1640 1 -- 172.21.15.172:0/111129049 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f41a6b20 con 0x7fd1f4102570 2024-09-10T08:58:03.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.159+0000 7fd1f0ff9640 1 -- 172.21.15.172:0/111129049 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc018920 con 0x7fd1f4102570 2024-09-10T08:58:03.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.159+0000 7fd1f95a1640 1 -- 172.21.15.172:0/111129049 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f41a70b0 con 0x7fd1f4102570 2024-09-10T08:58:03.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.162+0000 7fd1fb97a740 1 Processor -- start 2024-09-10T08:58:03.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.162+0000 7fd1f0ff9640 1 -- 172.21.15.172:0/111129049 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd1dc010a90 con 0x7fd1f4102570 2024-09-10T08:58:03.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.162+0000 7fd1fb97a740 1 -- start start 2024-09-10T08:58:03.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1fb97a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d839b40 0x562f3d838090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1fb97a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d838a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1fb97a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d84bb80 0x562f3d84df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1fb97a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562f3d79e2f0 con 0x562f3d8385d0 2024-09-10T08:58:03.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1fb97a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562f3d745cf0 con 0x562f3d84bb80 2024-09-10T08:58:03.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1fb97a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562f3d79a2b0 con 0x562f3d839b40 2024-09-10T08:58:03.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1f2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d838a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:03.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1f2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d838a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42994/0 (socket says 172.21.15.172:42994) 2024-09-10T08:58:03.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1f2ffd640 1 -- 172.21.15.172:0/186447238 learned_addr learned my addr 172.21.15.172:0/186447238 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:03.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1f37fe640 1 --2- 172.21.15.172:0/186447238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d839b40 0x562f3d838090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:03.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1ebfff640 1 --2- 172.21.15.172:0/186447238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d84bb80 0x562f3d84df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:03.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1f2ffd640 1 -- 172.21.15.172:0/186447238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d839b40 msgr2=0x562f3d838090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:03.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1f2ffd640 1 --2- 172.21.15.172:0/186447238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d839b40 0x562f3d838090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1f2ffd640 1 -- 172.21.15.172:0/186447238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d84bb80 msgr2=0x562f3d84df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:03.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1f2ffd640 1 --2- 172.21.15.172:0/186447238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d84bb80 0x562f3d84df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1f2ffd640 1 -- 172.21.15.172:0/186447238 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f3d72a3a0 con 0x562f3d8385d0 2024-09-10T08:58:03.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1ebfff640 1 --2- 172.21.15.172:0/186447238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d84bb80 0x562f3d84df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:03.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1f37fe640 1 --2- 172.21.15.172:0/186447238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d839b40 0x562f3d838090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:03.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.163+0000 7fd1f0ff9640 1 --2- 172.21.15.172:0/111129049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1e00a2c00 0x7fd1e00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.164+0000 7fd1f37fe640 1 --2- 172.21.15.172:0/111129049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1e00a2c00 0x7fd1e00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:03.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.164+0000 7fd1f2ffd640 1 --2- 172.21.15.172:0/186447238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d838a10 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400ac20 tx=0x7fd1e400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dc214420aac620 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:03.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.164+0000 7fd1f0ff9640 1 -- 172.21.15.172:0/111129049 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd1dc0cf630 con 0x7fd1f4102570 2024-09-10T08:58:03.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.164+0000 7fd1e97fa640 1 -- 172.21.15.172:0/186447238 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e401d070 con 0x562f3d8385d0 2024-09-10T08:58:03.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.164+0000 7fd1e97fa640 1 -- 172.21.15.172:0/186447238 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1e4018af0 con 0x562f3d8385d0 2024-09-10T08:58:03.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.164+0000 7fd1e97fa640 1 -- 172.21.15.172:0/186447238 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4010ad0 con 0x562f3d8385d0 2024-09-10T08:58:03.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.165+0000 7fd1fb97a740 1 -- 172.21.15.172:0/186447238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 msgr2=0x562f3d838a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:03.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.165+0000 7fd1f37fe640 1 --2- 172.21.15.172:0/111129049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1e00a2c00 0x7fd1e00a50c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec004af0 tx=0x7fd1ec004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:03.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.165+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/186447238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d838a10 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fd1e400ac20 tx=0x7fd1e400f790 comp rx=0 tx=0).stop 2024-09-10T08:58:03.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.165+0000 7fd1fb97a740 1 -- 172.21.15.172:0/186447238 shutdown_connections 2024-09-10T08:58:03.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.165+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/186447238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d84bb80 0x562f3d84df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.176 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.165+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/186447238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d838a10 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.165+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/186447238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d839b40 0x562f3d838090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.165+0000 7fd1fb97a740 1 -- 172.21.15.172:0/186447238 >> 172.21.15.172:0/186447238 conn(0x562f3d801310 msgr2=0x562f3d834c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:03.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.165+0000 7fd1fb97a740 1 -- 172.21.15.172:0/186447238 shutdown_connections 2024-09-10T08:58:03.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.165+0000 7fd1fb97a740 1 -- 172.21.15.172:0/186447238 wait complete. 2024-09-10T08:58:03.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.165+0000 7fd1fb97a740 1 -- start start 2024-09-10T08:58:03.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.165+0000 7fd1fb97a740 1 Processor -- start 2024-09-10T08:58:03.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1fb97a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d84b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1fb97a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d839b40 0x562f3d848760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1fb97a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d84bb80 0x562f3d848ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1fb97a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562f3d84e880 con 0x562f3d8385d0 2024-09-10T08:58:03.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1fb97a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562f3d759060 con 0x562f3d84bb80 2024-09-10T08:58:03.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1fb97a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562f3d84ea00 con 0x562f3d839b40 2024-09-10T08:58:03.177 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1fb97a740 10 client.0 _ll_register_callbacks cb 0x562f3d4ac8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:03.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1f2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d84b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:03.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1f2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d84b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43004/0 (socket says 172.21.15.172:43004) 2024-09-10T08:58:03.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1f2ffd640 1 -- 172.21.15.172:0/533763580 learned_addr learned my addr 172.21.15.172:0/533763580 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:03.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1ebfff640 1 --2- 172.21.15.172:0/533763580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d839b40 0x562f3d848760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:03.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1f37fe640 1 --2- 172.21.15.172:0/533763580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d84bb80 0x562f3d848ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:03.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1f2ffd640 1 -- 172.21.15.172:0/533763580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d839b40 msgr2=0x562f3d848760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:03.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1f2ffd640 1 --2- 172.21.15.172:0/533763580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d839b40 0x562f3d848760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1f2ffd640 1 -- 172.21.15.172:0/533763580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d84bb80 msgr2=0x562f3d848ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:03.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1f2ffd640 1 --2- 172.21.15.172:0/533763580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d84bb80 0x562f3d848ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1f2ffd640 1 -- 172.21.15.172:0/533763580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f3d50f3c0 con 0x562f3d8385d0 2024-09-10T08:58:03.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1f2ffd640 1 --2- 172.21.15.172:0/533763580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d84b4b0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4010470 tx=0x7fd1e4018440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f92f88595a92d3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:03.178 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.166+0000 7fd1da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:03.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.167+0000 7fd1da7fc640 1 -- 172.21.15.172:0/533763580 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e401d050 con 0x562f3d8385d0 2024-09-10T08:58:03.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.167+0000 7fd1fb97a740 1 -- 172.21.15.172:0/533763580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1e4018af0 con 0x562f3d8385d0 2024-09-10T08:58:03.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.167+0000 7fd1da7fc640 1 -- 172.21.15.172:0/533763580 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd1e4010810 con 0x562f3d8385d0 2024-09-10T08:58:03.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.167+0000 7fd1da7fc640 1 -- 172.21.15.172:0/533763580 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd1e4031050 con 0x562f3d8385d0 2024-09-10T08:58:03.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.167+0000 7fd1fb97a740 10 client.18957 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:03.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.167+0000 7fd1fb97a740 1 -- 172.21.15.172:0/533763580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562f3d85a210 con 0x562f3d8385d0 2024-09-10T08:58:03.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.167+0000 7fd1da7fc640 1 -- 172.21.15.172:0/533763580 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1e4018af0 con 0x562f3d8385d0 2024-09-10T08:58:03.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.167+0000 7fd1da7fc640 1 -- 172.21.15.172:0/533763580 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fd1e40046c0 con 0x562f3d8385d0 2024-09-10T08:58:03.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.167+0000 7fd1fb97a740 10 client.18957 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:03.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.167+0000 7fd1fb97a740 10 client.18957 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:58:03.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.167+0000 7fd1fb97a740 20 client.18957 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:03.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.167+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/533763580 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x562f3d85b9a0 0x562f3d85de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.179 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.167+0000 7fd1fb97a740 4 client.18957 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:58:03.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.167+0000 7fd1fb97a740 1 -- 172.21.15.172:0/533763580 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f3d7ab860 con 0x562f3d85b9a0 2024-09-10T08:58:03.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.168+0000 7fd1ebfff640 1 --2- 172.21.15.172:0/533763580 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x562f3d85b9a0 0x562f3d85de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:03.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.168+0000 7fd1da7fc640 10 client.18957 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:58:03.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.168+0000 7fd1ebfff640 1 --2- 172.21.15.172:0/533763580 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x562f3d85b9a0 0x562f3d85de30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d18ef84ebc70d6a server_cookie=39c9be3870a8a468 in_seq=0 out_seq=0 2024-09-10T08:58:03.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.169+0000 7fd1da7fc640 1 -- 172.21.15.172:0/533763580 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f3d7ab860 con 0x562f3d85b9a0 2024-09-10T08:58:03.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.169+0000 7fd1da7fc640 10 client.18957 handle_command_reply: tid=0 2024-09-10T08:58:03.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.169+0000 7fd1fb97a740 1 client.18957 shutdown 2024-09-10T08:58:03.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.169+0000 7fd1fb97a740 1 -- 172.21.15.172:0/533763580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 msgr2=0x562f3d84b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:03.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.169+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/533763580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d84b4b0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4010470 tx=0x7fd1e4018440 comp rx=0 tx=0).stop 2024-09-10T08:58:03.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.169+0000 7fd1fb97a740 1 -- 172.21.15.172:0/533763580 shutdown_connections 2024-09-10T08:58:03.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.169+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/533763580 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x562f3d85b9a0 0x562f3d85de30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.169+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/533763580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d84bb80 0x562f3d848ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.169+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/533763580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d839b40 0x562f3d848760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.169+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/533763580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d84b4b0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.170+0000 7fd1fb97a740 1 -- 172.21.15.172:0/533763580 >> 172.21.15.172:0/533763580 conn(0x562f3d801310 msgr2=0x562f3d83a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:03.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.170+0000 7fd1da7fc640 0 client.18957 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:58:03.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.170+0000 7fd1fb97a740 1 -- 172.21.15.172:0/533763580 shutdown_connections 2024-09-10T08:58:03.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.170+0000 7fd1fb97a740 1 -- 172.21.15.172:0/533763580 wait complete. 2024-09-10T08:58:03.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.170+0000 7fd1fb97a740 20 client.18957 trim_cache size 0 max 16384 2024-09-10T08:58:03.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.182+0000 7fd1fb97a740 1 Processor -- start 2024-09-10T08:58:03.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1fb97a740 1 -- start start 2024-09-10T08:58:03.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1fb97a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d857b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1fb97a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d839b40 0x562f3d843b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1fb97a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d84bb80 0x562f3d84cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1fb97a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41ac940 con 0x562f3d8385d0 2024-09-10T08:58:03.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1fb97a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41ae510 con 0x562f3d839b40 2024-09-10T08:58:03.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1fb97a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41ad010 con 0x562f3d84bb80 2024-09-10T08:58:03.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1f2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d857b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:03.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1f2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d857b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43008/0 (socket says 172.21.15.172:43008) 2024-09-10T08:58:03.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1f2ffd640 1 -- 172.21.15.172:0/2236446379 learned_addr learned my addr 172.21.15.172:0/2236446379 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:03.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1ebfff640 1 --2- 172.21.15.172:0/2236446379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d839b40 0x562f3d843b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:03.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1f37fe640 1 --2- 172.21.15.172:0/2236446379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d84bb80 0x562f3d84cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:03.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1f2ffd640 1 -- 172.21.15.172:0/2236446379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d84bb80 msgr2=0x562f3d84cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:03.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1f2ffd640 1 --2- 172.21.15.172:0/2236446379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d84bb80 0x562f3d84cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1f2ffd640 1 -- 172.21.15.172:0/2236446379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d839b40 msgr2=0x562f3d843b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:03.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1f2ffd640 1 --2- 172.21.15.172:0/2236446379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d839b40 0x562f3d843b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1f2ffd640 1 -- 172.21.15.172:0/2236446379 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f3d85e180 con 0x562f3d8385d0 2024-09-10T08:58:03.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1ebfff640 1 --2- 172.21.15.172:0/2236446379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d839b40 0x562f3d843b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:03.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.183+0000 7fd1f37fe640 1 --2- 172.21.15.172:0/2236446379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d84bb80 0x562f3d84cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:03.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.184+0000 7fd1f2ffd640 1 --2- 172.21.15.172:0/2236446379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d857b50 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40058c0 tx=0x7fd1e400a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78aad5b84cfbf00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:03.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.184+0000 7fd1e8ff9640 1 -- 172.21.15.172:0/2236446379 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e401d040 con 0x562f3d8385d0 2024-09-10T08:58:03.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.184+0000 7fd1e8ff9640 1 -- 172.21.15.172:0/2236446379 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1e403a070 con 0x562f3d8385d0 2024-09-10T08:58:03.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.184+0000 7fd1e8ff9640 1 -- 172.21.15.172:0/2236446379 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4004f30 con 0x562f3d8385d0 2024-09-10T08:58:03.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.185+0000 7fd1fb97a740 1 -- 172.21.15.172:0/2236446379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 msgr2=0x562f3d857b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:03.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.185+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/2236446379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d857b50 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40058c0 tx=0x7fd1e400a3c0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.185+0000 7fd1fb97a740 1 -- 172.21.15.172:0/2236446379 shutdown_connections 2024-09-10T08:58:03.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.185+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/2236446379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d84bb80 0x562f3d84cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.185+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/2236446379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d839b40 0x562f3d843b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.185+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/2236446379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d8385d0 0x562f3d857b50 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.185+0000 7fd1fb97a740 1 -- 172.21.15.172:0/2236446379 >> 172.21.15.172:0/2236446379 conn(0x562f3d801310 msgr2=0x562f3d847030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:03.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.185+0000 7fd1fb97a740 1 -- 172.21.15.172:0/2236446379 shutdown_connections 2024-09-10T08:58:03.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.185+0000 7fd1fb97a740 1 -- 172.21.15.172:0/2236446379 wait complete. 2024-09-10T08:58:03.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.185+0000 7fd1fb97a740 1 -- start start 2024-09-10T08:58:03.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.185+0000 7fd1fb97a740 1 Processor -- start 2024-09-10T08:58:03.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1fb97a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d8385d0 0x562f3d853cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1fb97a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d839b40 0x562f3d854200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1fb97a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d84bb80 0x562f3d8459e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1fb97a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41ade40 con 0x562f3d84bb80 2024-09-10T08:58:03.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1fb97a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41ad770 con 0x562f3d8385d0 2024-09-10T08:58:03.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1fb97a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41aa920 con 0x562f3d839b40 2024-09-10T08:58:03.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1fb97a740 10 client.0 _ll_register_callbacks cb 0x7fd1e401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:03.191 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1f37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d84bb80 0x562f3d8459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1f37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d84bb80 0x562f3d8459e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43022/0 (socket says 172.21.15.172:43022) 2024-09-10T08:58:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1f37fe640 1 -- 172.21.15.172:0/2572118230 learned_addr learned my addr 172.21.15.172:0/2572118230 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1ebfff640 1 --2- 172.21.15.172:0/2572118230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d839b40 0x562f3d854200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1f2ffd640 1 --2- 172.21.15.172:0/2572118230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d8385d0 0x562f3d853cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1f37fe640 1 -- 172.21.15.172:0/2572118230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d839b40 msgr2=0x562f3d854200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1f37fe640 1 --2- 172.21.15.172:0/2572118230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d839b40 0x562f3d854200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1f37fe640 1 -- 172.21.15.172:0/2572118230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d8385d0 msgr2=0x562f3d853cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1f37fe640 1 --2- 172.21.15.172:0/2572118230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d8385d0 0x562f3d853cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1f37fe640 1 -- 172.21.15.172:0/2572118230 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f3d8548f0 con 0x562f3d84bb80 2024-09-10T08:58:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1f37fe640 1 --2- 172.21.15.172:0/2572118230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d84bb80 0x562f3d8459e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009ca0 tx=0x7fd1ec0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d4a7d7d95f221a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.186+0000 7fd1da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:03.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.187+0000 7fd1da7fc640 1 -- 172.21.15.172:0/2572118230 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec034070 con 0x562f3d84bb80 2024-09-10T08:58:03.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.187+0000 7fd1fb97a740 1 -- 172.21.15.172:0/2572118230 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1e403a070 con 0x562f3d84bb80 2024-09-10T08:58:03.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.187+0000 7fd1da7fc640 1 -- 172.21.15.172:0/2572118230 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd1ec004040 con 0x562f3d84bb80 2024-09-10T08:58:03.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.187+0000 7fd1da7fc640 1 -- 172.21.15.172:0/2572118230 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd1ec02fea0 con 0x562f3d84bb80 2024-09-10T08:58:03.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.187+0000 7fd1fb97a740 10 client.18963 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:03.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.187+0000 7fd1fb97a740 1 -- 172.21.15.172:0/2572118230 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562f3d862c20 con 0x562f3d84bb80 2024-09-10T08:58:03.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.187+0000 7fd1da7fc640 1 -- 172.21.15.172:0/2572118230 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1e403a070 con 0x562f3d84bb80 2024-09-10T08:58:03.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.187+0000 7fd1da7fc640 1 -- 172.21.15.172:0/2572118230 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fd1ec041040 con 0x562f3d84bb80 2024-09-10T08:58:03.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.187+0000 7fd1fb97a740 10 client.18963 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:03.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.187+0000 7fd1fb97a740 10 client.18963 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:58:03.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.187+0000 7fd1fb97a740 20 client.18963 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:03.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.187+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/2572118230 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x562f3d8c66d0 0x562f3d8c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:03.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.187+0000 7fd1fb97a740 4 client.18963 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:58:03.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.187+0000 7fd1fb97a740 1 -- 172.21.15.172:0/2572118230 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f3d7a5d70 con 0x562f3d8c66d0 2024-09-10T08:58:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.187+0000 7fd1f2ffd640 1 --2- 172.21.15.172:0/2572118230 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x562f3d8c66d0 0x562f3d8c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.188+0000 7fd1f2ffd640 1 --2- 172.21.15.172:0/2572118230 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x562f3d8c66d0 0x562f3d8c8ac0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e75a54e019a1a09 server_cookie=5d38d2d306505036 in_seq=0 out_seq=0 2024-09-10T08:58:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.188+0000 7fd1da7fc640 10 client.18963 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:58:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.188+0000 7fd1da7fc640 1 -- 172.21.15.172:0/2572118230 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f3d7a5d70 con 0x562f3d8c66d0 2024-09-10T08:58:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.188+0000 7fd1da7fc640 10 client.18963 handle_command_reply: tid=0 2024-09-10T08:58:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.188+0000 7fd1fb97a740 1 client.18963 shutdown 2024-09-10T08:58:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.188+0000 7fd1fb97a740 1 -- 172.21.15.172:0/2572118230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d84bb80 msgr2=0x562f3d8459e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.188+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/2572118230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d84bb80 0x562f3d8459e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009ca0 tx=0x7fd1ec0095c0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.188+0000 7fd1fb97a740 1 -- 172.21.15.172:0/2572118230 shutdown_connections 2024-09-10T08:58:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.188+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/2572118230 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x562f3d8c66d0 0x562f3d8c8ac0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.188+0000 7fd1da7fc640 0 client.18963 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:58:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.188+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/2572118230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f3d84bb80 0x562f3d8459e0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.194 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.188+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/2572118230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562f3d839b40 0x562f3d854200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.188+0000 7fd1fb97a740 1 --2- 172.21.15.172:0/2572118230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562f3d8385d0 0x562f3d853cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.188+0000 7fd1fb97a740 1 -- 172.21.15.172:0/2572118230 >> 172.21.15.172:0/2572118230 conn(0x562f3d801310 msgr2=0x562f3d846fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:03.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.188+0000 7fd1fb97a740 1 -- 172.21.15.172:0/2572118230 shutdown_connections 2024-09-10T08:58:03.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.188+0000 7fd1fb97a740 1 -- 172.21.15.172:0/2572118230 wait complete. 2024-09-10T08:58:03.195 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.189+0000 7fd1fb97a740 20 client.18963 trim_cache size 0 max 16384 2024-09-10T08:58:03.195 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:58:03.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.189+0000 7fd1f95a1640 1 -- 172.21.15.172:0/111129049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1e00a2c00 msgr2=0x7fd1e00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:03.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.189+0000 7fd1f95a1640 1 --2- 172.21.15.172:0/111129049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1e00a2c00 0x7fd1e00a50c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec004af0 tx=0x7fd1ec004a40 comp rx=0 tx=0).stop 2024-09-10T08:58:03.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.189+0000 7fd1f95a1640 1 -- 172.21.15.172:0/111129049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f4102570 msgr2=0x7fd1f40fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:03.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.189+0000 7fd1f95a1640 1 --2- 172.21.15.172:0/111129049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f4102570 0x7fd1f40fd1c0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00e9f0 tx=0x7fd1dc00eeb0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.189+0000 7fd1f95a1640 1 -- 172.21.15.172:0/111129049 shutdown_connections 2024-09-10T08:58:03.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.189+0000 7fd1f95a1640 1 --2- 172.21.15.172:0/111129049 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1e00a2c00 0x7fd1e00a50c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.189+0000 7fd1f95a1640 1 --2- 172.21.15.172:0/111129049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1f4102ef0 0x7fd1f40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.196 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.189+0000 7fd1f95a1640 1 --2- 172.21.15.172:0/111129049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1f4102570 0x7fd1f40fd1c0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.189+0000 7fd1f95a1640 1 --2- 172.21.15.172:0/111129049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1f4100e20 0x7fd1f40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:03.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.189+0000 7fd1f95a1640 1 -- 172.21.15.172:0/111129049 >> 172.21.15.172:0/111129049 conn(0x7fd1f40fc7d0 msgr2=0x7fd1f406a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:03.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.189+0000 7fd1f95a1640 1 -- 172.21.15.172:0/111129049 shutdown_connections 2024-09-10T08:58:03.197 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:03.189+0000 7fd1f95a1640 1 -- 172.21.15.172:0/111129049 wait complete. 2024-09-10T08:58:03.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:03 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:58:03.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:03 smithi174 ceph-mon[34068]: pgmap v705: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:58:03.999 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:03 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:58:03.999 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:03 smithi172 ceph-mon[30854]: pgmap v705: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:58:04.000 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:58:04.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:03 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:58:04.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:03 smithi191 ceph-mon[35314]: pgmap v705: 129 pgs: 129 active+clean; 11 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:58:04.293 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:58:04.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.841+0000 7fb09af3c640 1 -- 172.21.15.172:0/3326705888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb094102df0 msgr2=0x7fb0941031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:04.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.841+0000 7fb09af3c640 1 --2- 172.21.15.172:0/3326705888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb094102df0 0x7fb0941031d0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fb088009900 tx=0x7fb08802a480 comp rx=0 tx=0).stop 2024-09-10T08:58:04.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.841+0000 7fb09af3c640 1 -- 172.21.15.172:0/3326705888 shutdown_connections 2024-09-10T08:58:04.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.841+0000 7fb09af3c640 1 --2- 172.21.15.172:0/3326705888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb09410c190 0x7fb09410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.841+0000 7fb09af3c640 1 --2- 172.21.15.172:0/3326705888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb094103710 0x7fb09410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.841+0000 7fb09af3c640 1 --2- 172.21.15.172:0/3326705888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb094102df0 0x7fb0941031d0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.841+0000 7fb09af3c640 1 -- 172.21.15.172:0/3326705888 >> 172.21.15.172:0/3326705888 conn(0x7fb0940fc790 msgr2=0x7fb0940febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:04.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.841+0000 7fb09af3c640 1 -- 172.21.15.172:0/3326705888 shutdown_connections 2024-09-10T08:58:04.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.841+0000 7fb09af3c640 1 -- 172.21.15.172:0/3326705888 wait complete. 2024-09-10T08:58:04.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.842+0000 7fb09af3c640 1 Processor -- start 2024-09-10T08:58:04.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.842+0000 7fb09af3c640 1 -- start start 2024-09-10T08:58:04.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.842+0000 7fb09af3c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb094102df0 0x7fb0941983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.842+0000 7fb09af3c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb094103710 0x7fb0941988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.842+0000 7fb09af3c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb09410c190 0x7fb09419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.842+0000 7fb09af3c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb094111e10 con 0x7fb094103710 2024-09-10T08:58:04.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.842+0000 7fb09af3c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb094111c90 con 0x7fb09410c190 2024-09-10T08:58:04.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.842+0000 7fb09af3c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb094111f90 con 0x7fb094102df0 2024-09-10T08:58:04.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.842+0000 7fb093fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb094103710 0x7fb0941988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:04.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.842+0000 7fb093fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb094103710 0x7fb0941988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43036/0 (socket says 172.21.15.172:43036) 2024-09-10T08:58:04.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.842+0000 7fb093fff640 1 -- 172.21.15.172:0/83098132 learned_addr learned my addr 172.21.15.172:0/83098132 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:04.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.842+0000 7fb098cd8640 1 --2- 172.21.15.172:0/83098132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb094102df0 0x7fb0941983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:04.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.843+0000 7fb0994d9640 1 --2- 172.21.15.172:0/83098132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb09410c190 0x7fb09419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:04.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.843+0000 7fb093fff640 1 -- 172.21.15.172:0/83098132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb094102df0 msgr2=0x7fb0941983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:04.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.843+0000 7fb093fff640 1 --2- 172.21.15.172:0/83098132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb094102df0 0x7fb0941983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.843+0000 7fb093fff640 1 -- 172.21.15.172:0/83098132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb09410c190 msgr2=0x7fb09419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:04.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.843+0000 7fb093fff640 1 --2- 172.21.15.172:0/83098132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb09410c190 0x7fb09419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.843+0000 7fb093fff640 1 -- 172.21.15.172:0/83098132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09419d360 con 0x7fb094103710 2024-09-10T08:58:04.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.843+0000 7fb093fff640 1 --2- 172.21.15.172:0/83098132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb094103710 0x7fb0941988f0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fb08000d950 tx=0x7fb08000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:04.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.843+0000 7fb091ffb640 1 -- 172.21.15.172:0/83098132 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080017030 con 0x7fb094103710 2024-09-10T08:58:04.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.843+0000 7fb09af3c640 1 -- 172.21.15.172:0/83098132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09419d650 con 0x7fb094103710 2024-09-10T08:58:04.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.843+0000 7fb091ffb640 1 -- 172.21.15.172:0/83098132 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb080004020 con 0x7fb094103710 2024-09-10T08:58:04.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.843+0000 7fb091ffb640 1 -- 172.21.15.172:0/83098132 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0800047a0 con 0x7fb094103710 2024-09-10T08:58:04.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.843+0000 7fb09af3c640 1 -- 172.21.15.172:0/83098132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0941a4f30 con 0x7fb094103710 2024-09-10T08:58:04.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.846+0000 7fb091ffb640 1 -- 172.21.15.172:0/83098132 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb0800041d0 con 0x7fb094103710 2024-09-10T08:58:04.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.846+0000 7fb09d315740 1 Processor -- start 2024-09-10T08:58:04.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.846+0000 7fb09d315740 1 -- start start 2024-09-10T08:58:04.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb091ffb640 1 --2- 172.21.15.172:0/83098132 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0840a2b80 0x7fb0840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb09d315740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009add6b40 0x56009add5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb09d315740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add55d0 0x56009add5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb09d315740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009ade0370 0x56009ade2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb09d315740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56009ad3b2f0 con 0x56009add6b40 2024-09-10T08:58:04.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb09d315740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56009ace2cf0 con 0x56009ade0370 2024-09-10T08:58:04.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb09d315740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56009ad372b0 con 0x56009add55d0 2024-09-10T08:58:04.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb098cd8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009add6b40 0x56009add5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:04.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb098cd8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009add6b40 0x56009add5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43038/0 (socket says 172.21.15.172:43038) 2024-09-10T08:58:04.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb098cd8640 1 -- 172.21.15.172:0/3842806572 learned_addr learned my addr 172.21.15.172:0/3842806572 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:04.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb091ffb640 1 -- 172.21.15.172:0/83098132 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb0800cedf0 con 0x7fb094103710 2024-09-10T08:58:04.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb0994d9640 1 --2- 172.21.15.172:0/3842806572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009ade0370 0x56009ade2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:04.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb098cd8640 1 -- 172.21.15.172:0/3842806572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add55d0 msgr2=0x56009add5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:04.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb098cd8640 1 --2- 172.21.15.172:0/3842806572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add55d0 0x56009add5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb098cd8640 1 -- 172.21.15.172:0/3842806572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009ade0370 msgr2=0x56009ade2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:04.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb098cd8640 1 --2- 172.21.15.172:0/3842806572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009ade0370 0x56009ade2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb098cd8640 1 -- 172.21.15.172:0/3842806572 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56009acc73a0 con 0x56009add6b40 2024-09-10T08:58:04.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.847+0000 7fb0994d9640 1 --2- 172.21.15.172:0/83098132 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0840a2b80 0x7fb0840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:04.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.848+0000 7fb098cd8640 1 --2- 172.21.15.172:0/3842806572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009add6b40 0x56009add5090 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fb088009900 tx=0x7fb088032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ef428ab43764bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:04.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.848+0000 7fb0994d9640 1 --2- 172.21.15.172:0/83098132 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0840a2b80 0x7fb0840a5040 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb08c005c40 tx=0x7fb08c004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:04.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.848+0000 7fb07effd640 1 -- 172.21.15.172:0/3842806572 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088040070 con 0x56009add6b40 2024-09-10T08:58:04.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.848+0000 7fb07effd640 1 -- 172.21.15.172:0/3842806572 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb088004030 con 0x56009add6b40 2024-09-10T08:58:04.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.848+0000 7fb07effd640 1 -- 172.21.15.172:0/3842806572 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088035040 con 0x56009add6b40 2024-09-10T08:58:04.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.849+0000 7fb09d315740 1 -- 172.21.15.172:0/3842806572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009add6b40 msgr2=0x56009add5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:04.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.849+0000 7fb09d315740 1 --2- 172.21.15.172:0/3842806572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009add6b40 0x56009add5090 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fb088009900 tx=0x7fb088032bb0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.849+0000 7fb09d315740 1 -- 172.21.15.172:0/3842806572 shutdown_connections 2024-09-10T08:58:04.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.849+0000 7fb09d315740 1 --2- 172.21.15.172:0/3842806572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009ade0370 0x56009ade2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.849+0000 7fb09d315740 1 --2- 172.21.15.172:0/3842806572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add55d0 0x56009add5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.849+0000 7fb09d315740 1 --2- 172.21.15.172:0/3842806572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009add6b40 0x56009add5090 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.849+0000 7fb09d315740 1 -- 172.21.15.172:0/3842806572 >> 172.21.15.172:0/3842806572 conn(0x56009ad9e310 msgr2=0x56009add1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:04.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.849+0000 7fb09d315740 1 -- 172.21.15.172:0/3842806572 shutdown_connections 2024-09-10T08:58:04.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.849+0000 7fb09d315740 1 -- 172.21.15.172:0/3842806572 wait complete. 2024-09-10T08:58:04.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.849+0000 7fb09d315740 1 -- start start 2024-09-10T08:58:04.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.850+0000 7fb09d315740 1 Processor -- start 2024-09-10T08:58:04.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.850+0000 7fb09d315740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009add55d0 0x56009addfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.850+0000 7fb09d315740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add6b40 0x56009addd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.850+0000 7fb09d315740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009ade0370 0x56009addd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.850+0000 7fb09d315740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56009ade7420 con 0x56009ade0370 2024-09-10T08:58:04.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.850+0000 7fb09d315740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56009acf6060 con 0x56009add55d0 2024-09-10T08:58:04.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.850+0000 7fb09d315740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56009ade75a0 con 0x56009add6b40 2024-09-10T08:58:04.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.850+0000 7fb09d315740 10 client.0 _ll_register_callbacks cb 0x56009aa498c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:04.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.850+0000 7fb0994d9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009ade0370 0x56009addd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:04.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.850+0000 7fb093fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add6b40 0x56009addd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:04.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.850+0000 7fb0994d9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009ade0370 0x56009addd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43054/0 (socket says 172.21.15.172:43054) 2024-09-10T08:58:04.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.850+0000 7fb0994d9640 1 -- 172.21.15.172:0/3559653025 learned_addr learned my addr 172.21.15.172:0/3559653025 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:04.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.850+0000 7fb098cd8640 1 --2- 172.21.15.172:0/3559653025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009add55d0 0x56009addfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:04.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.851+0000 7fb093fff640 1 -- 172.21.15.172:0/3559653025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009add55d0 msgr2=0x56009addfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:04.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.851+0000 7fb093fff640 1 --2- 172.21.15.172:0/3559653025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009add55d0 0x56009addfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.851+0000 7fb093fff640 1 -- 172.21.15.172:0/3559653025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009ade0370 msgr2=0x56009addd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:04.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.851+0000 7fb093fff640 1 --2- 172.21.15.172:0/3559653025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009ade0370 0x56009addd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.851+0000 7fb093fff640 1 -- 172.21.15.172:0/3559653025 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56009aaac3c0 con 0x56009add6b40 2024-09-10T08:58:04.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.851+0000 7fb0994d9640 1 --2- 172.21.15.172:0/3559653025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009ade0370 0x56009addd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:04.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.851+0000 7fb093fff640 1 --2- 172.21.15.172:0/3559653025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add6b40 0x56009addd0d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x56009add2ac0 tx=0x7fb08009bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=610fc64a2108db32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:04.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.851+0000 7fb073fff640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:58:04.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.851+0000 7fb073fff640 1 -- 172.21.15.172:0/3559653025 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080095e60 con 0x56009add6b40 2024-09-10T08:58:04.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.851+0000 7fb09d315740 1 -- 172.21.15.172:0/3559653025 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb088004030 con 0x56009add6b40 2024-09-10T08:58:04.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.851+0000 7fb073fff640 1 -- 172.21.15.172:0/3559653025 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb08009a740 con 0x56009add6b40 2024-09-10T08:58:04.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.852+0000 7fb073fff640 1 -- 172.21.15.172:0/3559653025 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb0800a3f60 con 0x56009add6b40 2024-09-10T08:58:04.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.852+0000 7fb09d315740 10 client.27757 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:04.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.852+0000 7fb09d315740 1 -- 172.21.15.172:0/3559653025 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56009adf3040 con 0x56009add6b40 2024-09-10T08:58:04.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.852+0000 7fb073fff640 1 -- 172.21.15.172:0/3559653025 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb088004030 con 0x56009add6b40 2024-09-10T08:58:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.852+0000 7fb073fff640 1 -- 172.21.15.172:0/3559653025 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fb0800bb070 con 0x56009add6b40 2024-09-10T08:58:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.852+0000 7fb09d315740 10 client.27757 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.852+0000 7fb09d315740 10 client.27757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:58:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.852+0000 7fb09d315740 20 client.27757 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.852+0000 7fb09d315740 1 --2- 172.21.15.172:0/3559653025 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56009adf4450 0x56009adf6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.852+0000 7fb09d315740 4 client.27757 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:58:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.852+0000 7fb09d315740 1 -- 172.21.15.172:0/3559653025 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56009ad48860 con 0x56009adf4450 2024-09-10T08:58:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.852+0000 7fb098cd8640 1 --2- 172.21.15.172:0/3559653025 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56009adf4450 0x56009adf6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.853+0000 7fb098cd8640 1 --2- 172.21.15.172:0/3559653025 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56009adf4450 0x56009adf6840 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd7e69b6f6a60376 server_cookie=805415961f5c4e89 in_seq=0 out_seq=0 2024-09-10T08:58:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.853+0000 7fb073fff640 10 client.27757 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.854+0000 7fb073fff640 1 -- 172.21.15.172:0/3559653025 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56009ad48860 con 0x56009adf4450 2024-09-10T08:58:04.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.854+0000 7fb073fff640 10 client.27757 handle_command_reply: tid=0 2024-09-10T08:58:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.854+0000 7fb09d315740 1 client.27757 shutdown 2024-09-10T08:58:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.854+0000 7fb09d315740 1 -- 172.21.15.172:0/3559653025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add6b40 msgr2=0x56009addd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.854+0000 7fb09d315740 1 --2- 172.21.15.172:0/3559653025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add6b40 0x56009addd0d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x56009add2ac0 tx=0x7fb08009bab0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.854+0000 7fb09d315740 1 -- 172.21.15.172:0/3559653025 shutdown_connections 2024-09-10T08:58:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.854+0000 7fb09d315740 1 --2- 172.21.15.172:0/3559653025 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56009adf4450 0x56009adf6840 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.854+0000 7fb09d315740 1 --2- 172.21.15.172:0/3559653025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009ade0370 0x56009addd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.854+0000 7fb09d315740 1 --2- 172.21.15.172:0/3559653025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add6b40 0x56009addd0d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.854+0000 7fb09d315740 1 --2- 172.21.15.172:0/3559653025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009add55d0 0x56009addfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.854+0000 7fb09d315740 1 -- 172.21.15.172:0/3559653025 >> 172.21.15.172:0/3559653025 conn(0x56009ad9e310 msgr2=0x56009add7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.854+0000 7fb073fff640 0 client.27757 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.854+0000 7fb09d315740 1 -- 172.21.15.172:0/3559653025 shutdown_connections 2024-09-10T08:58:04.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.854+0000 7fb09d315740 1 -- 172.21.15.172:0/3559653025 wait complete. 2024-09-10T08:58:04.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.854+0000 7fb09d315740 20 client.27757 trim_cache size 0 max 16384 2024-09-10T08:58:04.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.874+0000 7fb09d315740 1 Processor -- start 2024-09-10T08:58:04.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.875+0000 7fb09d315740 1 -- start start 2024-09-10T08:58:04.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.875+0000 7fb09d315740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add55d0 0x56009addfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.875+0000 7fb09d315740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009add6b40 0x56009adfc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.875+0000 7fb09d315740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009ade0370 0x56009adfc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.875+0000 7fb09d315740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941ac930 con 0x56009add6b40 2024-09-10T08:58:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.875+0000 7fb09d315740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941ae500 con 0x56009ade0370 2024-09-10T08:58:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.875+0000 7fb09d315740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941ad000 con 0x56009add55d0 2024-09-10T08:58:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.875+0000 7fb093fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009add6b40 0x56009adfc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.875+0000 7fb093fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009add6b40 0x56009adfc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43070/0 (socket says 172.21.15.172:43070) 2024-09-10T08:58:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.875+0000 7fb093fff640 1 -- 172.21.15.172:0/3892973596 learned_addr learned my addr 172.21.15.172:0/3892973596 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.875+0000 7fb0994d9640 1 --2- 172.21.15.172:0/3892973596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009ade0370 0x56009adfc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.875+0000 7fb098cd8640 1 --2- 172.21.15.172:0/3892973596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add55d0 0x56009addfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:04.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.875+0000 7fb093fff640 1 -- 172.21.15.172:0/3892973596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add55d0 msgr2=0x56009addfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:04.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.875+0000 7fb093fff640 1 --2- 172.21.15.172:0/3892973596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add55d0 0x56009addfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.875+0000 7fb093fff640 1 -- 172.21.15.172:0/3892973596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009ade0370 msgr2=0x56009adfc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:04.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.875+0000 7fb093fff640 1 --2- 172.21.15.172:0/3892973596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009ade0370 0x56009adfc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.875+0000 7fb093fff640 1 -- 172.21.15.172:0/3892973596 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56009add8120 con 0x56009add6b40 2024-09-10T08:58:04.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.876+0000 7fb093fff640 1 --2- 172.21.15.172:0/3892973596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009add6b40 0x56009adfc0c0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fb080097750 tx=0x7fb08009f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3bb6c7b7cb8c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:04.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.876+0000 7fb07e7fc640 1 -- 172.21.15.172:0/3892973596 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080095e60 con 0x56009add6b40 2024-09-10T08:58:04.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.876+0000 7fb07e7fc640 1 -- 172.21.15.172:0/3892973596 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0800bb0f0 con 0x56009add6b40 2024-09-10T08:58:04.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.876+0000 7fb07e7fc640 1 -- 172.21.15.172:0/3892973596 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0800972a0 con 0x56009add6b40 2024-09-10T08:58:04.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.877+0000 7fb09d315740 1 -- 172.21.15.172:0/3892973596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009add6b40 msgr2=0x56009adfc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:04.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.877+0000 7fb09d315740 1 --2- 172.21.15.172:0/3892973596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009add6b40 0x56009adfc0c0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fb080097750 tx=0x7fb08009f820 comp rx=0 tx=0).stop 2024-09-10T08:58:04.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.877+0000 7fb09d315740 1 -- 172.21.15.172:0/3892973596 shutdown_connections 2024-09-10T08:58:04.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.877+0000 7fb09d315740 1 --2- 172.21.15.172:0/3892973596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009ade0370 0x56009adfc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.877+0000 7fb09d315740 1 --2- 172.21.15.172:0/3892973596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009add6b40 0x56009adfc0c0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.877+0000 7fb09d315740 1 --2- 172.21.15.172:0/3892973596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add55d0 0x56009addfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.877+0000 7fb09d315740 1 -- 172.21.15.172:0/3892973596 >> 172.21.15.172:0/3892973596 conn(0x56009ad9e310 msgr2=0x56009addc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:04.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.877+0000 7fb09d315740 1 -- 172.21.15.172:0/3892973596 shutdown_connections 2024-09-10T08:58:04.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.877+0000 7fb09d315740 1 -- 172.21.15.172:0/3892973596 wait complete. 2024-09-10T08:58:04.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.877+0000 7fb09d315740 1 -- start start 2024-09-10T08:58:04.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.877+0000 7fb09d315740 1 Processor -- start 2024-09-10T08:58:04.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb09d315740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009add55d0 0x56009adece40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb09d315740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add6b40 0x56009ade8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb09d315740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009ade0370 0x56009ade86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb09d315740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941ade30 con 0x56009ade0370 2024-09-10T08:58:04.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb09d315740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941ad760 con 0x56009add55d0 2024-09-10T08:58:04.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb09d315740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941aa910 con 0x56009add6b40 2024-09-10T08:58:04.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb09d315740 10 client.0 _ll_register_callbacks cb 0x7fb0800050c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:04.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb093fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add6b40 0x56009ade8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:04.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb0994d9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009ade0370 0x56009ade86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:04.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb098cd8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009add55d0 0x56009adece40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:04.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb093fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add6b40 0x56009ade8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:54244/0 (socket says 172.21.15.172:54244) 2024-09-10T08:58:04.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb098cd8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009add55d0 0x56009adece40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:33770/0 (socket says 172.21.15.172:33770) 2024-09-10T08:58:04.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb093fff640 1 -- 172.21.15.172:0/456666614 learned_addr learned my addr 172.21.15.172:0/456666614 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:04.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb098cd8640 1 -- 172.21.15.172:0/456666614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add6b40 msgr2=0x56009ade8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:04.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb098cd8640 1 --2- 172.21.15.172:0/456666614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add6b40 0x56009ade8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb098cd8640 1 -- 172.21.15.172:0/456666614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009ade0370 msgr2=0x56009ade86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:04.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb098cd8640 1 --2- 172.21.15.172:0/456666614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009ade0370 0x56009ade86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb098cd8640 1 -- 172.21.15.172:0/456666614 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56009ade8da0 con 0x56009add55d0 2024-09-10T08:58:04.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.878+0000 7fb0994d9640 1 --2- 172.21.15.172:0/456666614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009ade0370 0x56009ade86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:04.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.879+0000 7fb098cd8640 1 --2- 172.21.15.172:0/456666614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009add55d0 0x56009adece40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x56009adf7060 tx=0x7fb088031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a20ded39f87a435 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:04.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.879+0000 7fb073fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:58:04.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.879+0000 7fb073fff640 1 -- 172.21.15.172:0/456666614 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088053020 con 0x56009add55d0 2024-09-10T08:58:04.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.879+0000 7fb073fff640 1 -- 172.21.15.172:0/456666614 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb0880314a0 con 0x56009add55d0 2024-09-10T08:58:04.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.879+0000 7fb09d315740 1 -- 172.21.15.172:0/456666614 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0800bb0f0 con 0x56009add55d0 2024-09-10T08:58:04.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.879+0000 7fb073fff640 1 -- 172.21.15.172:0/456666614 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb08805e050 con 0x56009add55d0 2024-09-10T08:58:04.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.879+0000 7fb09d315740 10 client.27830 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:04.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.879+0000 7fb09d315740 1 -- 172.21.15.172:0/456666614 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56009ae5ef50 con 0x56009add55d0 2024-09-10T08:58:04.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.879+0000 7fb073fff640 1 -- 172.21.15.172:0/456666614 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0800bb0f0 con 0x56009add55d0 2024-09-10T08:58:04.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.879+0000 7fb073fff640 1 -- 172.21.15.172:0/456666614 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fb08804fa20 con 0x56009add55d0 2024-09-10T08:58:04.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.880+0000 7fb09d315740 10 client.27830 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:04.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.880+0000 7fb09d315740 10 client.27830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:58:04.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.880+0000 7fb09d315740 20 client.27830 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:04.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.880+0000 7fb09d315740 1 --2- 172.21.15.172:0/456666614 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56009ae5f2e0 0x56009ae616d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:04.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.880+0000 7fb09d315740 4 client.27830 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:58:04.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.880+0000 7fb09d315740 1 -- 172.21.15.172:0/456666614 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56009ad42d70 con 0x56009ae5f2e0 2024-09-10T08:58:04.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.880+0000 7fb093fff640 1 --2- 172.21.15.172:0/456666614 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56009ae5f2e0 0x56009ae616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:04.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.880+0000 7fb073fff640 10 client.27830 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:04.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.880+0000 7fb093fff640 1 --2- 172.21.15.172:0/456666614 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56009ae5f2e0 0x56009ae616d0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=652f1cab5b4b7a76 server_cookie=677b26cf6e7718df in_seq=0 out_seq=0 2024-09-10T08:58:04.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.880+0000 7fb073fff640 1 -- 172.21.15.172:0/456666614 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56009ad42d70 con 0x56009ae5f2e0 2024-09-10T08:58:04.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.880+0000 7fb073fff640 10 client.27830 handle_command_reply: tid=0 2024-09-10T08:58:04.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.880+0000 7fb09d315740 1 client.27830 shutdown 2024-09-10T08:58:04.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.880+0000 7fb09d315740 1 -- 172.21.15.172:0/456666614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009add55d0 msgr2=0x56009adece40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:04.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.881+0000 7fb09d315740 1 --2- 172.21.15.172:0/456666614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009add55d0 0x56009adece40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x56009adf7060 tx=0x7fb088031400 comp rx=0 tx=0).stop 2024-09-10T08:58:04.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.881+0000 7fb09d315740 1 -- 172.21.15.172:0/456666614 shutdown_connections 2024-09-10T08:58:04.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.881+0000 7fb09d315740 1 --2- 172.21.15.172:0/456666614 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56009ae5f2e0 0x56009ae616d0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.881+0000 7fb09d315740 1 --2- 172.21.15.172:0/456666614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56009ade0370 0x56009ade86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.881+0000 7fb09d315740 1 --2- 172.21.15.172:0/456666614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56009add6b40 0x56009ade8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.881+0000 7fb073fff640 0 client.27830 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:04.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.881+0000 7fb09d315740 1 --2- 172.21.15.172:0/456666614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56009add55d0 0x56009adece40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.881+0000 7fb09d315740 1 -- 172.21.15.172:0/456666614 >> 172.21.15.172:0/456666614 conn(0x56009ad9e310 msgr2=0x56009add7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:04.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.881+0000 7fb09d315740 1 -- 172.21.15.172:0/456666614 shutdown_connections 2024-09-10T08:58:04.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.881+0000 7fb09d315740 1 -- 172.21.15.172:0/456666614 wait complete. 2024-09-10T08:58:04.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.881+0000 7fb09d315740 20 client.27830 trim_cache size 0 max 16384 2024-09-10T08:58:04.886 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:58:04.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.881+0000 7fb09af3c640 1 -- 172.21.15.172:0/83098132 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0840a2b80 msgr2=0x7fb0840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:04.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.881+0000 7fb09af3c640 1 --2- 172.21.15.172:0/83098132 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0840a2b80 0x7fb0840a5040 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb08c005c40 tx=0x7fb08c004ab0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.881+0000 7fb09af3c640 1 -- 172.21.15.172:0/83098132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb094103710 msgr2=0x7fb0941988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:04.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.881+0000 7fb09af3c640 1 --2- 172.21.15.172:0/83098132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb094103710 0x7fb0941988f0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fb08000d950 tx=0x7fb08000de10 comp rx=0 tx=0).stop 2024-09-10T08:58:04.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.882+0000 7fb09af3c640 1 -- 172.21.15.172:0/83098132 shutdown_connections 2024-09-10T08:58:04.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.882+0000 7fb09af3c640 1 --2- 172.21.15.172:0/83098132 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0840a2b80 0x7fb0840a5040 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.882+0000 7fb09af3c640 1 --2- 172.21.15.172:0/83098132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb09410c190 0x7fb09419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.882+0000 7fb09af3c640 1 --2- 172.21.15.172:0/83098132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb094103710 0x7fb0941988f0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.882+0000 7fb09af3c640 1 --2- 172.21.15.172:0/83098132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb094102df0 0x7fb0941983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:04.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.882+0000 7fb09af3c640 1 -- 172.21.15.172:0/83098132 >> 172.21.15.172:0/83098132 conn(0x7fb0940fc790 msgr2=0x7fb0940fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:04.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.882+0000 7fb09af3c640 1 -- 172.21.15.172:0/83098132 shutdown_connections 2024-09-10T08:58:04.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:04.882+0000 7fb09af3c640 1 -- 172.21.15.172:0/83098132 wait complete. 2024-09-10T08:58:05.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:05 smithi174 ceph-mon[34068]: pgmap v706: 129 pgs: 129 active+clean; 12 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-09-10T08:58:05.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:05 smithi191 ceph-mon[35314]: pgmap v706: 129 pgs: 129 active+clean; 12 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-09-10T08:58:05.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:05 smithi172 ceph-mon[30854]: pgmap v706: 129 pgs: 129 active+clean; 12 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 32 MiB/s wr, 28 op/s 2024-09-10T08:58:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:07 smithi174 ceph-mon[34068]: pgmap v707: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-09-10T08:58:07.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:07 smithi191 ceph-mon[35314]: pgmap v707: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-09-10T08:58:07.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:07 smithi172 ceph-mon[30854]: pgmap v707: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 MiB/s wr, 22 op/s 2024-09-10T08:58:08.613 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:58:08.613 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:58:08.895 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:58:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.277+0000 7f6aff43c640 1 -- 172.21.15.172:0/888896915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af8101c60 msgr2=0x7f6af81020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.277+0000 7f6aff43c640 1 --2- 172.21.15.172:0/888896915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af8101c60 0x7f6af81020e0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009900 tx=0x7f6ae802a410 comp rx=0 tx=0).stop 2024-09-10T08:58:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.277+0000 7f6aff43c640 1 -- 172.21.15.172:0/888896915 shutdown_connections 2024-09-10T08:58:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.277+0000 7f6aff43c640 1 --2- 172.21.15.172:0/888896915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af8102620 0x7f6af8108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.277+0000 7f6aff43c640 1 --2- 172.21.15.172:0/888896915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af8101c60 0x7f6af81020e0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.277+0000 7f6aff43c640 1 --2- 172.21.15.172:0/888896915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6af8100a70 0x7f6af8100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.277+0000 7f6aff43c640 1 -- 172.21.15.172:0/888896915 >> 172.21.15.172:0/888896915 conn(0x7f6af80fc1b0 msgr2=0x7f6af80fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.277+0000 7f6aff43c640 1 -- 172.21.15.172:0/888896915 shutdown_connections 2024-09-10T08:58:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.277+0000 7f6aff43c640 1 -- 172.21.15.172:0/888896915 wait complete. 2024-09-10T08:58:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.278+0000 7f6aff43c640 1 Processor -- start 2024-09-10T08:58:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.278+0000 7f6aff43c640 1 -- start start 2024-09-10T08:58:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.278+0000 7f6aff43c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6af8100a70 0x7f6af8196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.278+0000 7f6aff43c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af8101c60 0x7f6af8197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.278+0000 7f6aff43c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af8102620 0x7f6af819e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.278+0000 7f6aff43c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6af810c580 con 0x7f6af8101c60 2024-09-10T08:58:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.278+0000 7f6aff43c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6af810c400 con 0x7f6af8102620 2024-09-10T08:58:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.278+0000 7f6aff43c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6af810c700 con 0x7f6af8100a70 2024-09-10T08:58:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.278+0000 7f6afc9d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af8101c60 0x7f6af8197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.278+0000 7f6afd9d9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af8102620 0x7f6af819e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.278+0000 7f6afd9d9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af8102620 0x7f6af819e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:54620/0 (socket says 172.21.15.172:54620) 2024-09-10T08:58:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.278+0000 7f6afc9d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af8101c60 0x7f6af8197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39956/0 (socket says 172.21.15.172:39956) 2024-09-10T08:58:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.278+0000 7f6afd9d9640 1 -- 172.21.15.172:0/645732964 learned_addr learned my addr 172.21.15.172:0/645732964 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.279+0000 7f6afd1d8640 1 --2- 172.21.15.172:0/645732964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6af8100a70 0x7f6af8196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.279+0000 7f6afd9d9640 1 -- 172.21.15.172:0/645732964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6af8100a70 msgr2=0x7f6af8196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.279+0000 7f6afd9d9640 1 --2- 172.21.15.172:0/645732964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6af8100a70 0x7f6af8196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.279+0000 7f6afd9d9640 1 -- 172.21.15.172:0/645732964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af8101c60 msgr2=0x7f6af8197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.279+0000 7f6afd9d9640 1 --2- 172.21.15.172:0/645732964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af8101c60 0x7f6af8197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.279+0000 7f6afd9d9640 1 -- 172.21.15.172:0/645732964 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af819e830 con 0x7f6af8102620 2024-09-10T08:58:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.279+0000 7f6afd1d8640 1 --2- 172.21.15.172:0/645732964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6af8100a70 0x7f6af8196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.279+0000 7f6afd9d9640 1 --2- 172.21.15.172:0/645732964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af8102620 0x7f6af819e190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009fd0 tx=0x7f6af000b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.280+0000 7f6aee7fc640 1 -- 172.21.15.172:0/645732964 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0017070 con 0x7f6af8102620 2024-09-10T08:58:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.280+0000 7f6aee7fc640 1 -- 172.21.15.172:0/645732964 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6af0003e20 con 0x7f6af8102620 2024-09-10T08:58:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.280+0000 7f6aee7fc640 1 -- 172.21.15.172:0/645732964 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0010460 con 0x7f6af8102620 2024-09-10T08:58:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.280+0000 7f6aff43c640 1 -- 172.21.15.172:0/645732964 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af806a300 con 0x7f6af8102620 2024-09-10T08:58:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.280+0000 7f6aff43c640 1 -- 172.21.15.172:0/645732964 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af806a850 con 0x7f6af8102620 2024-09-10T08:58:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.284+0000 7f6b021d8740 1 Processor -- start 2024-09-10T08:58:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.284+0000 7f6aee7fc640 1 -- 172.21.15.172:0/645732964 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6af0003970 con 0x7f6af8102620 2024-09-10T08:58:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.284+0000 7f6b021d8740 1 -- start start 2024-09-10T08:58:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.284+0000 7f6b021d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53bd9a0 0x55ffe53bdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.284+0000 7f6b021d8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53be2e0 0x55ffe53c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.284+0000 7f6b021d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53c3370 0x55ffe53c5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.284+0000 7f6b021d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ffe53212f0 con 0x55ffe53c3370 2024-09-10T08:58:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.284+0000 7f6b021d8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ffe52c9270 con 0x55ffe53be2e0 2024-09-10T08:58:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.284+0000 7f6b021d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ffe531d2b0 con 0x55ffe53bd9a0 2024-09-10T08:58:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.284+0000 7f6afc9d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53c3370 0x55ffe53c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.284+0000 7f6afc9d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53c3370 0x55ffe53c5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39958/0 (socket says 172.21.15.172:39958) 2024-09-10T08:58:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.284+0000 7f6afc9d7640 1 -- 172.21.15.172:0/1586563094 learned_addr learned my addr 172.21.15.172:0/1586563094 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.284+0000 7f6afc9d7640 1 --2- 172.21.15.172:0/1586563094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53bd9a0 0x55ffe53bdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.285+0000 7f6afd1d8640 1 --2- 172.21.15.172:0/1586563094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53be2e0 0x55ffe53c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.285+0000 7f6aee7fc640 1 --2- 172.21.15.172:0/645732964 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6ae40a2cb0 0x7f6ae40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.285+0000 7f6afc9d7640 1 -- 172.21.15.172:0/1586563094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53bd9a0 msgr2=0x55ffe53bdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.285+0000 7f6afc9d7640 1 --2- 172.21.15.172:0/1586563094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53bd9a0 0x55ffe53bdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.285+0000 7f6afc9d7640 1 -- 172.21.15.172:0/1586563094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53be2e0 msgr2=0x55ffe53c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.285+0000 7f6afc9d7640 1 --2- 172.21.15.172:0/1586563094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53be2e0 0x55ffe53c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.285+0000 7f6afc9d7640 1 -- 172.21.15.172:0/1586563094 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe52c8340 con 0x55ffe53c3370 2024-09-10T08:58:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.285+0000 7f6afd9d9640 1 --2- 172.21.15.172:0/645732964 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6ae40a2cb0 0x7f6ae40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.285+0000 7f6aee7fc640 1 -- 172.21.15.172:0/645732964 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6af00cf160 con 0x7f6af8102620 2024-09-10T08:58:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.285+0000 7f6afc9d7640 1 --2- 172.21.15.172:0/1586563094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53c3370 0x55ffe53c5830 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009900 tx=0x7f6ae803a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c68a35da1f55a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.285+0000 7f6ae37fe640 1 -- 172.21.15.172:0/1586563094 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8035070 con 0x55ffe53c3370 2024-09-10T08:58:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.285+0000 7f6ae37fe640 1 -- 172.21.15.172:0/1586563094 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ae80316f0 con 0x55ffe53c3370 2024-09-10T08:58:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.285+0000 7f6ae37fe640 1 -- 172.21.15.172:0/1586563094 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8003810 con 0x55ffe53c3370 2024-09-10T08:58:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.286+0000 7f6b021d8740 1 -- 172.21.15.172:0/1586563094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53c3370 msgr2=0x55ffe53c5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.286+0000 7f6afd9d9640 1 --2- 172.21.15.172:0/645732964 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6ae40a2cb0 0x7f6ae40a5170 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6af009b410 tx=0x7f6af0066300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.286+0000 7f6b021d8740 1 --2- 172.21.15.172:0/1586563094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53c3370 0x55ffe53c5830 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009900 tx=0x7f6ae803a040 comp rx=0 tx=0).stop 2024-09-10T08:58:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.286+0000 7f6b021d8740 1 -- 172.21.15.172:0/1586563094 shutdown_connections 2024-09-10T08:58:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.286+0000 7f6b021d8740 1 --2- 172.21.15.172:0/1586563094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53c3370 0x55ffe53c5830 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.286+0000 7f6b021d8740 1 --2- 172.21.15.172:0/1586563094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53be2e0 0x55ffe53c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.286+0000 7f6b021d8740 1 --2- 172.21.15.172:0/1586563094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53bd9a0 0x55ffe53bdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.286+0000 7f6b021d8740 1 -- 172.21.15.172:0/1586563094 >> 172.21.15.172:0/1586563094 conn(0x55ffe53b9010 msgr2=0x55ffe53b9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.286+0000 7f6b021d8740 1 -- 172.21.15.172:0/1586563094 shutdown_connections 2024-09-10T08:58:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.286+0000 7f6b021d8740 1 -- 172.21.15.172:0/1586563094 wait complete. 2024-09-10T08:58:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.287+0000 7f6b021d8740 1 -- start start 2024-09-10T08:58:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.287+0000 7f6b021d8740 1 Processor -- start 2024-09-10T08:58:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.287+0000 7f6b021d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53bd9a0 0x55ffe53c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.287+0000 7f6b021d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53be2e0 0x55ffe53c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.287+0000 7f6b021d8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53c3370 0x55ffe53cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.287+0000 7f6b021d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ffe53c62e0 con 0x55ffe53bd9a0 2024-09-10T08:58:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.287+0000 7f6b021d8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ffe52aa290 con 0x55ffe53c3370 2024-09-10T08:58:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.287+0000 7f6b021d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ffe53c6460 con 0x55ffe53be2e0 2024-09-10T08:58:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.287+0000 7f6b021d8740 10 client.0 _ll_register_callbacks cb 0x55ffe52db140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.287+0000 7f6afc9d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53bd9a0 0x55ffe53c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.287+0000 7f6afc9d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53bd9a0 0x55ffe53c7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39960/0 (socket says 172.21.15.172:39960) 2024-09-10T08:58:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.287+0000 7f6afc9d7640 1 -- 172.21.15.172:0/3345356089 learned_addr learned my addr 172.21.15.172:0/3345356089 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.287+0000 7f6afc9d7640 1 --2- 172.21.15.172:0/3345356089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53c3370 0x55ffe53cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.287+0000 7f6afd1d8640 1 --2- 172.21.15.172:0/3345356089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53be2e0 0x55ffe53c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.288+0000 7f6afd1d8640 1 -- 172.21.15.172:0/3345356089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53c3370 msgr2=0x55ffe53cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.288+0000 7f6afd1d8640 1 --2- 172.21.15.172:0/3345356089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53c3370 0x55ffe53cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.288+0000 7f6afd1d8640 1 -- 172.21.15.172:0/3345356089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53bd9a0 msgr2=0x55ffe53c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.288+0000 7f6afd1d8640 1 --2- 172.21.15.172:0/3345356089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53bd9a0 0x55ffe53c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.288+0000 7f6afd1d8640 1 -- 172.21.15.172:0/3345356089 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffe5034210 con 0x55ffe53be2e0 2024-09-10T08:58:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.288+0000 7f6afc9d7640 1 --2- 172.21.15.172:0/3345356089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53bd9a0 0x55ffe53c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.288+0000 7f6afd1d8640 1 --2- 172.21.15.172:0/3345356089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53be2e0 0x55ffe53c8130 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55ffe53bed10 tx=0x7f6af400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41e57dd37bbc2c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.288+0000 7f6ae0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:58:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.288+0000 7f6ae0ff9640 1 -- 172.21.15.172:0/3345356089 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af40120d0 con 0x55ffe53be2e0 2024-09-10T08:58:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.288+0000 7f6ae0ff9640 1 -- 172.21.15.172:0/3345356089 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6af4004030 con 0x55ffe53be2e0 2024-09-10T08:58:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.288+0000 7f6b021d8740 1 -- 172.21.15.172:0/3345356089 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae80316f0 con 0x55ffe53be2e0 2024-09-10T08:58:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.289+0000 7f6ae0ff9640 1 -- 172.21.15.172:0/3345356089 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6af4025050 con 0x55ffe53be2e0 2024-09-10T08:58:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.289+0000 7f6b021d8740 10 client.27775 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.289+0000 7f6b021d8740 1 -- 172.21.15.172:0/3345356089 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ffe5100620 con 0x55ffe53be2e0 2024-09-10T08:58:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.289+0000 7f6ae0ff9640 1 -- 172.21.15.172:0/3345356089 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ae80316f0 con 0x55ffe53be2e0 2024-09-10T08:58:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.289+0000 7f6ae0ff9640 1 -- 172.21.15.172:0/3345356089 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f6af401c460 con 0x55ffe53be2e0 2024-09-10T08:58:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.289+0000 7f6b021d8740 10 client.27775 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.289+0000 7f6b021d8740 10 client.27775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:58:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.289+0000 7f6b021d8740 20 client.27775 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.289+0000 7f6b021d8740 1 --2- 172.21.15.172:0/3345356089 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ffe53d38c0 0x55ffe53d5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.289+0000 7f6b021d8740 4 client.27775 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:58:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.289+0000 7f6b021d8740 1 -- 172.21.15.172:0/3345356089 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffe532ea20 con 0x55ffe53d38c0 2024-09-10T08:58:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.289+0000 7f6afc9d7640 1 --2- 172.21.15.172:0/3345356089 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ffe53d38c0 0x55ffe53d5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.290+0000 7f6ae0ff9640 10 client.27775 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.290+0000 7f6afc9d7640 1 --2- 172.21.15.172:0/3345356089 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ffe53d38c0 0x55ffe53d5d10 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14b4d30684422da server_cookie=c3c7dae85ccd585f in_seq=0 out_seq=0 2024-09-10T08:58:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.290+0000 7f6ae0ff9640 1 -- 172.21.15.172:0/3345356089 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffe532ea20 con 0x55ffe53d38c0 2024-09-10T08:58:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.290+0000 7f6ae0ff9640 10 client.27775 handle_command_reply: tid=0 2024-09-10T08:58:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.290+0000 7f6b021d8740 1 client.27775 shutdown 2024-09-10T08:58:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.290+0000 7f6b021d8740 1 -- 172.21.15.172:0/3345356089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53be2e0 msgr2=0x55ffe53c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.290+0000 7f6b021d8740 1 --2- 172.21.15.172:0/3345356089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53be2e0 0x55ffe53c8130 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55ffe53bed10 tx=0x7f6af400f790 comp rx=0 tx=0).stop 2024-09-10T08:58:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.291+0000 7f6b021d8740 1 -- 172.21.15.172:0/3345356089 shutdown_connections 2024-09-10T08:58:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.291+0000 7f6b021d8740 1 --2- 172.21.15.172:0/3345356089 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ffe53d38c0 0x55ffe53d5d10 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.291+0000 7f6b021d8740 1 --2- 172.21.15.172:0/3345356089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53c3370 0x55ffe53cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.291+0000 7f6ae0ff9640 0 client.27775 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.291+0000 7f6b021d8740 1 --2- 172.21.15.172:0/3345356089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53be2e0 0x55ffe53c8130 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.291+0000 7f6b021d8740 1 --2- 172.21.15.172:0/3345356089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53bd9a0 0x55ffe53c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.291+0000 7f6b021d8740 1 -- 172.21.15.172:0/3345356089 >> 172.21.15.172:0/3345356089 conn(0x55ffe53b9010 msgr2=0x55ffe53ba820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.291+0000 7f6b021d8740 1 -- 172.21.15.172:0/3345356089 shutdown_connections 2024-09-10T08:58:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.291+0000 7f6b021d8740 1 -- 172.21.15.172:0/3345356089 wait complete. 2024-09-10T08:58:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.291+0000 7f6b021d8740 20 client.27775 trim_cache size 0 max 16384 2024-09-10T08:58:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.308+0000 7f6b021d8740 1 Processor -- start 2024-09-10T08:58:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.308+0000 7f6b021d8740 1 -- start start 2024-09-10T08:58:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.308+0000 7f6b021d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53bd9a0 0x55ffe53bdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.308+0000 7f6b021d8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53be2e0 0x55ffe53cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.308+0000 7f6b021d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53c3370 0x55ffe53b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.308+0000 7f6b021d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81a8310 con 0x55ffe53bd9a0 2024-09-10T08:58:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.308+0000 7f6b021d8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81a9ee0 con 0x55ffe53be2e0 2024-09-10T08:58:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.308+0000 7f6b021d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81a89e0 con 0x55ffe53c3370 2024-09-10T08:58:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.309+0000 7f6afc9d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53bd9a0 0x55ffe53bdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.309+0000 7f6afc9d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53bd9a0 0x55ffe53bdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39964/0 (socket says 172.21.15.172:39964) 2024-09-10T08:58:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.309+0000 7f6afc9d7640 1 -- 172.21.15.172:0/3142928152 learned_addr learned my addr 172.21.15.172:0/3142928152 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.309+0000 7f6afc9d7640 1 --2- 172.21.15.172:0/3142928152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53c3370 0x55ffe53b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.309+0000 7f6afd1d8640 1 --2- 172.21.15.172:0/3142928152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53be2e0 0x55ffe53cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.309+0000 7f6afc9d7640 1 -- 172.21.15.172:0/3142928152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53c3370 msgr2=0x55ffe53b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.309+0000 7f6afc9d7640 1 --2- 172.21.15.172:0/3142928152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53c3370 0x55ffe53b80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.309+0000 7f6afc9d7640 1 -- 172.21.15.172:0/3142928152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53be2e0 msgr2=0x55ffe53cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.309+0000 7f6afc9d7640 1 --2- 172.21.15.172:0/3142928152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53be2e0 0x55ffe53cf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.309+0000 7f6afc9d7640 1 -- 172.21.15.172:0/3142928152 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe53d6390 con 0x55ffe53bd9a0 2024-09-10T08:58:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.309+0000 7f6afd1d8640 1 --2- 172.21.15.172:0/3142928152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53be2e0 0x55ffe53cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.309+0000 7f6afc9d7640 1 --2- 172.21.15.172:0/3142928152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53bd9a0 0x55ffe53bdd80 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009900 tx=0x7f6ae8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acba87b3616a5cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.309+0000 7f6ae2ffd640 1 -- 172.21.15.172:0/3142928152 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8035070 con 0x55ffe53bd9a0 2024-09-10T08:58:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.309+0000 7f6ae2ffd640 1 -- 172.21.15.172:0/3142928152 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ae80316f0 con 0x55ffe53bd9a0 2024-09-10T08:58:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.309+0000 7f6ae2ffd640 1 -- 172.21.15.172:0/3142928152 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8058790 con 0x55ffe53bd9a0 2024-09-10T08:58:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.310+0000 7f6b021d8740 1 -- 172.21.15.172:0/3142928152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53bd9a0 msgr2=0x55ffe53bdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.310+0000 7f6b021d8740 1 --2- 172.21.15.172:0/3142928152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53bd9a0 0x55ffe53bdd80 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009900 tx=0x7f6ae8000bc0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.310+0000 7f6b021d8740 1 -- 172.21.15.172:0/3142928152 shutdown_connections 2024-09-10T08:58:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.310+0000 7f6b021d8740 1 --2- 172.21.15.172:0/3142928152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53c3370 0x55ffe53b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.310+0000 7f6b021d8740 1 --2- 172.21.15.172:0/3142928152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53be2e0 0x55ffe53cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.310+0000 7f6b021d8740 1 --2- 172.21.15.172:0/3142928152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53bd9a0 0x55ffe53bdd80 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.310+0000 7f6b021d8740 1 -- 172.21.15.172:0/3142928152 >> 172.21.15.172:0/3142928152 conn(0x55ffe53b9010 msgr2=0x55ffe53b79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.310+0000 7f6b021d8740 1 -- 172.21.15.172:0/3142928152 shutdown_connections 2024-09-10T08:58:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.310+0000 7f6b021d8740 1 -- 172.21.15.172:0/3142928152 wait complete. 2024-09-10T08:58:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.311+0000 7f6b021d8740 1 -- start start 2024-09-10T08:58:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.311+0000 7f6b021d8740 1 Processor -- start 2024-09-10T08:58:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.311+0000 7f6b021d8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53bd9a0 0x55ffe53cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.311+0000 7f6b021d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53be2e0 0x55ffe53ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.311+0000 7f6b021d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53c3370 0x55ffe53cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.311+0000 7f6b021d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81a9810 con 0x55ffe53be2e0 2024-09-10T08:58:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.311+0000 7f6b021d8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81a9140 con 0x55ffe53bd9a0 2024-09-10T08:58:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.311+0000 7f6b021d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81a62f0 con 0x55ffe53c3370 2024-09-10T08:58:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.311+0000 7f6b021d8740 10 client.0 _ll_register_callbacks cb 0x7f6af4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.311+0000 7f6afd1d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53be2e0 0x55ffe53ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.311+0000 7f6afd1d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53be2e0 0x55ffe53ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39972/0 (socket says 172.21.15.172:39972) 2024-09-10T08:58:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.311+0000 7f6afd1d8640 1 -- 172.21.15.172:0/2897655221 learned_addr learned my addr 172.21.15.172:0/2897655221 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.311+0000 7f6afc9d7640 1 --2- 172.21.15.172:0/2897655221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53c3370 0x55ffe53cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.311+0000 7f6afc9d7640 1 --2- 172.21.15.172:0/2897655221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53bd9a0 0x55ffe53cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.311+0000 7f6afd1d8640 1 -- 172.21.15.172:0/2897655221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53c3370 msgr2=0x55ffe53cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.312+0000 7f6afd1d8640 1 --2- 172.21.15.172:0/2897655221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53c3370 0x55ffe53cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.312+0000 7f6afd1d8640 1 -- 172.21.15.172:0/2897655221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53bd9a0 msgr2=0x55ffe53cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.312+0000 7f6afd1d8640 1 --2- 172.21.15.172:0/2897655221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53bd9a0 0x55ffe53cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.312+0000 7f6afd1d8640 1 -- 172.21.15.172:0/2897655221 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffe53ce0e0 con 0x55ffe53be2e0 2024-09-10T08:58:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.312+0000 7f6afc9d7640 1 --2- 172.21.15.172:0/2897655221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53bd9a0 0x55ffe53cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.312+0000 7f6afd1d8640 1 --2- 172.21.15.172:0/2897655221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53be2e0 0x55ffe53ccb30 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f6af40059f0 tx=0x7f6af4012860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89f1fb40b3bb4b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.312+0000 7f6ae0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.312+0000 7f6ae0ff9640 1 -- 172.21.15.172:0/2897655221 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af402e060 con 0x55ffe53be2e0 2024-09-10T08:58:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.312+0000 7f6ae0ff9640 1 -- 172.21.15.172:0/2897655221 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6af4005110 con 0x55ffe53be2e0 2024-09-10T08:58:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.312+0000 7f6b021d8740 1 -- 172.21.15.172:0/2897655221 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae80316f0 con 0x55ffe53be2e0 2024-09-10T08:58:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.312+0000 7f6ae0ff9640 1 -- 172.21.15.172:0/2897655221 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6af4035050 con 0x55ffe53be2e0 2024-09-10T08:58:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.312+0000 7f6b021d8740 10 client.18999 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.312+0000 7f6b021d8740 1 -- 172.21.15.172:0/2897655221 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ffe543e640 con 0x55ffe53be2e0 2024-09-10T08:58:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.312+0000 7f6ae0ff9640 1 -- 172.21.15.172:0/2897655221 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ae80316f0 con 0x55ffe53be2e0 2024-09-10T08:58:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.312+0000 7f6ae0ff9640 1 -- 172.21.15.172:0/2897655221 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f6af4028030 con 0x55ffe53be2e0 2024-09-10T08:58:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.313+0000 7f6b021d8740 10 client.18999 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:09.319 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.313+0000 7f6b021d8740 10 client.18999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:58:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.313+0000 7f6b021d8740 20 client.18999 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.313+0000 7f6b021d8740 1 --2- 172.21.15.172:0/2897655221 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ffe543eeb0 0x55ffe54412d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.313+0000 7f6b021d8740 4 client.18999 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:58:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.313+0000 7f6b021d8740 1 -- 172.21.15.172:0/2897655221 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ffe5328d70 con 0x55ffe543eeb0 2024-09-10T08:58:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.313+0000 7f6afc9d7640 1 --2- 172.21.15.172:0/2897655221 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ffe543eeb0 0x55ffe54412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.313+0000 7f6ae0ff9640 10 client.18999 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.313+0000 7f6afc9d7640 1 --2- 172.21.15.172:0/2897655221 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ffe543eeb0 0x55ffe54412d0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba6059242d7a3e6b server_cookie=2ed9a331e96c75cf in_seq=0 out_seq=0 2024-09-10T08:58:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.319+0000 7f6ae0ff9640 1 -- 172.21.15.172:0/2897655221 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ffe5328d70 con 0x55ffe543eeb0 2024-09-10T08:58:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.319+0000 7f6ae0ff9640 10 client.18999 handle_command_reply: tid=0 2024-09-10T08:58:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.319+0000 7f6b021d8740 1 client.18999 shutdown 2024-09-10T08:58:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.319+0000 7f6b021d8740 1 -- 172.21.15.172:0/2897655221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53be2e0 msgr2=0x55ffe53ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.319+0000 7f6b021d8740 1 --2- 172.21.15.172:0/2897655221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53be2e0 0x55ffe53ccb30 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f6af40059f0 tx=0x7f6af4012860 comp rx=0 tx=0).stop 2024-09-10T08:58:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.319+0000 7f6b021d8740 1 -- 172.21.15.172:0/2897655221 shutdown_connections 2024-09-10T08:58:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.319+0000 7f6b021d8740 1 --2- 172.21.15.172:0/2897655221 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ffe543eeb0 0x55ffe54412d0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.319+0000 7f6b021d8740 1 --2- 172.21.15.172:0/2897655221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ffe53c3370 0x55ffe53cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.319+0000 7f6b021d8740 1 --2- 172.21.15.172:0/2897655221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffe53be2e0 0x55ffe53ccb30 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.319+0000 7f6b021d8740 1 --2- 172.21.15.172:0/2897655221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffe53bd9a0 0x55ffe53cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.319+0000 7f6b021d8740 1 -- 172.21.15.172:0/2897655221 >> 172.21.15.172:0/2897655221 conn(0x55ffe53b9010 msgr2=0x55ffe53b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.319+0000 7f6ae0ff9640 0 client.18999 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.322+0000 7f6b021d8740 1 -- 172.21.15.172:0/2897655221 shutdown_connections 2024-09-10T08:58:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.322+0000 7f6b021d8740 1 -- 172.21.15.172:0/2897655221 wait complete. 2024-09-10T08:58:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.322+0000 7f6b021d8740 20 client.18999 trim_cache size 0 max 16384 2024-09-10T08:58:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:58:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:58:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:58:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:58:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.323+0000 7f6aff43c640 1 -- 172.21.15.172:0/645732964 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6ae40a2cb0 msgr2=0x7f6ae40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.323+0000 7f6aff43c640 1 --2- 172.21.15.172:0/645732964 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6ae40a2cb0 0x7f6ae40a5170 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6af009b410 tx=0x7f6af0066300 comp rx=0 tx=0).stop 2024-09-10T08:58:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.323+0000 7f6aff43c640 1 -- 172.21.15.172:0/645732964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af8102620 msgr2=0x7f6af819e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.323+0000 7f6aff43c640 1 --2- 172.21.15.172:0/645732964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af8102620 0x7f6af819e190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009fd0 tx=0x7f6af000b260 comp rx=0 tx=0).stop 2024-09-10T08:58:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.323+0000 7f6aff43c640 1 -- 172.21.15.172:0/645732964 shutdown_connections 2024-09-10T08:58:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.323+0000 7f6aff43c640 1 --2- 172.21.15.172:0/645732964 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6ae40a2cb0 0x7f6ae40a5170 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.323+0000 7f6aff43c640 1 --2- 172.21.15.172:0/645732964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af8102620 0x7f6af819e190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.323+0000 7f6aff43c640 1 --2- 172.21.15.172:0/645732964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af8101c60 0x7f6af8197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.325 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.323+0000 7f6aff43c640 1 --2- 172.21.15.172:0/645732964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6af8100a70 0x7f6af8196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.323+0000 7f6aff43c640 1 -- 172.21.15.172:0/645732964 >> 172.21.15.172:0/645732964 conn(0x7f6af80fc1b0 msgr2=0x7f6af80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.323+0000 7f6aff43c640 1 -- 172.21.15.172:0/645732964 shutdown_connections 2024-09-10T08:58:09.326 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:09.323+0000 7f6aff43c640 1 -- 172.21.15.172:0/645732964 wait complete. 2024-09-10T08:58:09.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.419+0000 7f5c367d5640 1 -- 172.21.15.172:0/1755599333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c30106b80 msgr2=0x7f5c30106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.419+0000 7f5c367d5640 1 --2- 172.21.15.172:0/1755599333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c30106b80 0x7f5c30106f60 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009900 tx=0x7f5c2002a410 comp rx=0 tx=0).stop 2024-09-10T08:58:09.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.419+0000 7f5c367d5640 1 -- 172.21.15.172:0/1755599333 shutdown_connections 2024-09-10T08:58:09.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.419+0000 7f5c367d5640 1 --2- 172.21.15.172:0/1755599333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c30100aa0 0x7f5c3010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.419+0000 7f5c367d5640 1 --2- 172.21.15.172:0/1755599333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c30100100 0x7f5c30100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.419+0000 7f5c367d5640 1 --2- 172.21.15.172:0/1755599333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c30106b80 0x7f5c30106f60 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.419+0000 7f5c367d5640 1 -- 172.21.15.172:0/1755599333 >> 172.21.15.172:0/1755599333 conn(0x7f5c30077230 msgr2=0x7f5c300fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:09.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.419+0000 7f5c367d5640 1 -- 172.21.15.172:0/1755599333 shutdown_connections 2024-09-10T08:58:09.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.419+0000 7f5c367d5640 1 -- 172.21.15.172:0/1755599333 wait complete. 2024-09-10T08:58:09.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.419+0000 7f5c367d5640 1 Processor -- start 2024-09-10T08:58:09.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.419+0000 7f5c367d5640 1 -- start start 2024-09-10T08:58:09.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.420+0000 7f5c367d5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c30100100 0x7f5c30198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.420+0000 7f5c367d5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c30100aa0 0x7f5c301988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.420+0000 7f5c367d5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c30106b80 0x7f5c3019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.420+0000 7f5c367d5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5c30111f00 con 0x7f5c30100100 2024-09-10T08:58:09.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.420+0000 7f5c367d5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c30111d80 con 0x7f5c30106b80 2024-09-10T08:58:09.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.420+0000 7f5c367d5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5c30112080 con 0x7f5c30100aa0 2024-09-10T08:58:09.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.420+0000 7f5c2ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c30100100 0x7f5c30198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.420+0000 7f5c2ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c30100100 0x7f5c30198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39980/0 (socket says 172.21.15.172:39980) 2024-09-10T08:58:09.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.420+0000 7f5c2ffff640 1 -- 172.21.15.172:0/2363944016 learned_addr learned my addr 172.21.15.172:0/2363944016 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:09.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.420+0000 7f5c34d72640 1 --2- 172.21.15.172:0/2363944016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c30106b80 0x7f5c3019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.420+0000 7f5c2f7fe640 1 --2- 172.21.15.172:0/2363944016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c30100aa0 0x7f5c301988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.420+0000 7f5c2ffff640 1 -- 172.21.15.172:0/2363944016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c30100aa0 msgr2=0x7f5c301988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.420+0000 7f5c2ffff640 1 --2- 172.21.15.172:0/2363944016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c30100aa0 0x7f5c301988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.420+0000 7f5c2ffff640 1 -- 172.21.15.172:0/2363944016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c30106b80 msgr2=0x7f5c3019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.420+0000 7f5c2ffff640 1 --2- 172.21.15.172:0/2363944016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c30106b80 0x7f5c3019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.420+0000 7f5c2ffff640 1 -- 172.21.15.172:0/2363944016 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3019d330 con 0x7f5c30100100 2024-09-10T08:58:09.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.421+0000 7f5c2ffff640 1 --2- 172.21.15.172:0/2363944016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c30100100 0x7f5c30198380 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f5c20006010 tx=0x7f5c20004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:09.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.421+0000 7f5c2d7fa640 1 -- 172.21.15.172:0/2363944016 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2003f030 con 0x7f5c30100100 2024-09-10T08:58:09.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.421+0000 7f5c367d5640 1 -- 172.21.15.172:0/2363944016 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c3019d5c0 con 0x7f5c30100100 2024-09-10T08:58:09.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.421+0000 7f5c2d7fa640 1 -- 172.21.15.172:0/2363944016 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c200035f0 con 0x7f5c30100100 2024-09-10T08:58:09.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.421+0000 7f5c2d7fa640 1 -- 172.21.15.172:0/2363944016 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c20035070 con 0x7f5c30100100 2024-09-10T08:58:09.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.421+0000 7f5c367d5640 1 -- 172.21.15.172:0/2363944016 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c301a4f00 con 0x7f5c30100100 2024-09-10T08:58:09.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.423+0000 7f5c2d7fa640 1 -- 172.21.15.172:0/2363944016 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5c20003b10 con 0x7f5c30100100 2024-09-10T08:58:09.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.424+0000 7f5c38bae740 1 Processor -- start 2024-09-10T08:58:09.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.424+0000 7f5c38bae740 1 -- start start 2024-09-10T08:58:09.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.424+0000 7f5c38bae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dceb60 0x5580e0dcef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.424+0000 7f5c38bae740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dcf480 0x5580e0dd9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.424+0000 7f5c38bae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dd9d40 0x5580e0ddc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.424+0000 7f5c38bae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5580e0d34280 con 0x5580e0dceb60 2024-09-10T08:58:09.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.424+0000 7f5c38bae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5580e0cdbb10 con 0x5580e0dd9d40 2024-09-10T08:58:09.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.424+0000 7f5c38bae740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5580e0d30240 con 0x5580e0dcf480 2024-09-10T08:58:09.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.425+0000 7f5c2d7fa640 1 --2- 172.21.15.172:0/2363944016 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5c180a2c00 0x7f5c180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.425+0000 7f5c2f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dd9d40 0x5580e0ddc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.425+0000 7f5c2ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dcf480 0x5580e0dd9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.425+0000 7f5c2f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dd9d40 0x5580e0ddc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:54676/0 (socket says 172.21.15.172:54676) 2024-09-10T08:58:09.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.425+0000 7f5c2f7fe640 1 -- 172.21.15.172:0/2890050936 learned_addr learned my addr 172.21.15.172:0/2890050936 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:09.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.425+0000 7f5c34d72640 1 --2- 172.21.15.172:0/2363944016 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5c180a2c00 0x7f5c180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.425+0000 7f5c2d7fa640 1 -- 172.21.15.172:0/2363944016 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5c200efdf0 con 0x7f5c30100100 2024-09-10T08:58:09.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.425+0000 7f5c34d72640 1 --2- 172.21.15.172:0/2890050936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dceb60 0x5580e0dcef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.425+0000 7f5c2f7fe640 1 -- 172.21.15.172:0/2890050936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dcf480 msgr2=0x5580e0dd9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:09.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.425+0000 7f5c2f7fe640 1 --2- 172.21.15.172:0/2890050936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dcf480 0x5580e0dd9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.425+0000 7f5c2f7fe640 1 -- 172.21.15.172:0/2890050936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dceb60 msgr2=0x5580e0dcef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:09.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.425+0000 7f5c2f7fe640 1 --2- 172.21.15.172:0/2890050936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dceb60 0x5580e0dcef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.425+0000 7f5c2f7fe640 1 -- 172.21.15.172:0/2890050936 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580e0ce81f0 con 0x5580e0dd9d40 2024-09-10T08:58:09.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.426+0000 7f5c2f7fe640 1 --2- 172.21.15.172:0/2890050936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dd9d40 0x5580e0ddc130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400abf0 tx=0x7f5c2400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73ce275bf7bdf8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:09.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.426+0000 7f5c167fc640 1 -- 172.21.15.172:0/2890050936 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2401d070 con 0x5580e0dd9d40 2024-09-10T08:58:09.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.426+0000 7f5c167fc640 1 -- 172.21.15.172:0/2890050936 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c24018af0 con 0x5580e0dd9d40 2024-09-10T08:58:09.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.426+0000 7f5c167fc640 1 -- 172.21.15.172:0/2890050936 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c240109d0 con 0x5580e0dd9d40 2024-09-10T08:58:09.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.427+0000 7f5c38bae740 1 -- 172.21.15.172:0/2890050936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dd9d40 msgr2=0x5580e0ddc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.427+0000 7f5c38bae740 1 --2- 172.21.15.172:0/2890050936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dd9d40 0x5580e0ddc130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400abf0 tx=0x7f5c2400f790 comp rx=0 tx=0).stop 2024-09-10T08:58:09.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.427+0000 7f5c34d72640 1 --2- 172.21.15.172:0/2363944016 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5c180a2c00 0x7f5c180a50c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5c28007330 tx=0x7f5c28007e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:09.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.427+0000 7f5c38bae740 1 -- 172.21.15.172:0/2890050936 shutdown_connections 2024-09-10T08:58:09.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.427+0000 7f5c38bae740 1 --2- 172.21.15.172:0/2890050936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dd9d40 0x5580e0ddc130 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.427+0000 7f5c38bae740 1 --2- 172.21.15.172:0/2890050936 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dcf480 0x5580e0dd9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.427+0000 7f5c38bae740 1 --2- 172.21.15.172:0/2890050936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dceb60 0x5580e0dcef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.427+0000 7f5c38bae740 1 -- 172.21.15.172:0/2890050936 >> 172.21.15.172:0/2890050936 conn(0x5580e0dcb0f0 msgr2=0x5580e0dcb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:09.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.427+0000 7f5c38bae740 1 -- 172.21.15.172:0/2890050936 shutdown_connections 2024-09-10T08:58:09.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.427+0000 7f5c38bae740 1 -- 172.21.15.172:0/2890050936 wait complete. 2024-09-10T08:58:09.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.427+0000 7f5c38bae740 1 -- start start 2024-09-10T08:58:09.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.428+0000 7f5c38bae740 1 Processor -- start 2024-09-10T08:58:09.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.428+0000 7f5c38bae740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dceb60 0x5580e0dd6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.428+0000 7f5c38bae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dcf480 0x5580e0dd7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.428+0000 7f5c38bae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dd9d40 0x5580e0dd7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.428+0000 7f5c2f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dcf480 0x5580e0dd7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.433 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.428+0000 7f5c38bae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5580e0ddc9b0 con 0x5580e0dd9d40 2024-09-10T08:58:09.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.428+0000 7f5c38bae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5580e0ceef10 con 0x5580e0dcf480 2024-09-10T08:58:09.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.428+0000 7f5c38bae740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5580e0ddcb30 con 0x5580e0dceb60 2024-09-10T08:58:09.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.428+0000 7f5c2ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dceb60 0x5580e0dd6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.428+0000 7f5c2f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dcf480 0x5580e0dd7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:54678/0 (socket says 172.21.15.172:54678) 2024-09-10T08:58:09.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.428+0000 7f5c2f7fe640 1 -- 172.21.15.172:0/1216876341 learned_addr learned my addr 172.21.15.172:0/1216876341 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:09.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.428+0000 7f5c34d72640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dd9d40 0x5580e0dd7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.428+0000 7f5c38bae740 10 client.0 _ll_register_callbacks cb 0x5580e0cb7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:09.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.429+0000 7f5c2ffff640 1 -- 172.21.15.172:0/1216876341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dcf480 msgr2=0x5580e0dd7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:09.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.429+0000 7f5c2ffff640 1 --2- 172.21.15.172:0/1216876341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dcf480 0x5580e0dd7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.429+0000 7f5c2ffff640 1 -- 172.21.15.172:0/1216876341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dd9d40 msgr2=0x5580e0dd7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:09.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.429+0000 7f5c2ffff640 1 --2- 172.21.15.172:0/1216876341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dd9d40 0x5580e0dd7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.429+0000 7f5c2ffff640 1 -- 172.21.15.172:0/1216876341 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580e0aa56a0 con 0x5580e0dceb60 2024-09-10T08:58:09.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.429+0000 7f5c2f7fe640 1 --2- 172.21.15.172:0/1216876341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dcf480 0x5580e0dd7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:09.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.429+0000 7f5c2ffff640 1 --2- 172.21.15.172:0/1216876341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dceb60 0x5580e0dd6be0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c200bc0a0 tx=0x7f5c200037a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed60248e50834251 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:09.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.429+0000 7f5c0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:58:09.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.429+0000 7f5c0f7fe640 1 -- 172.21.15.172:0/1216876341 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c20049040 con 0x5580e0dceb60 2024-09-10T08:58:09.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.429+0000 7f5c0f7fe640 1 -- 172.21.15.172:0/1216876341 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5c200bc600 con 0x5580e0dceb60 2024-09-10T08:58:09.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.429+0000 7f5c38bae740 1 -- 172.21.15.172:0/1216876341 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c24018af0 con 0x5580e0dceb60 2024-09-10T08:58:09.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.429+0000 7f5c0f7fe640 1 -- 172.21.15.172:0/1216876341 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5c200c4f60 con 0x5580e0dceb60 2024-09-10T08:58:09.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.429+0000 7f5c38bae740 10 client.27790 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:09.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.429+0000 7f5c38bae740 1 -- 172.21.15.172:0/1216876341 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5580e0ddc9b0 con 0x5580e0dceb60 2024-09-10T08:58:09.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.429+0000 7f5c0f7fe640 1 -- 172.21.15.172:0/1216876341 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c24018af0 con 0x5580e0dceb60 2024-09-10T08:58:09.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.430+0000 7f5c0f7fe640 1 -- 172.21.15.172:0/1216876341 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f5c200c4b40 con 0x5580e0dceb60 2024-09-10T08:58:09.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.430+0000 7f5c38bae740 10 client.27790 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:09.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.430+0000 7f5c38bae740 10 client.27790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:58:09.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.430+0000 7f5c38bae740 20 client.27790 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:09.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.430+0000 7f5c38bae740 1 --2- 172.21.15.172:0/1216876341 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5580e0de9980 0x5580e0debdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.430+0000 7f5c38bae740 4 client.27790 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:58:09.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.430+0000 7f5c38bae740 1 -- 172.21.15.172:0/1216876341 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580e0d417f0 con 0x5580e0de9980 2024-09-10T08:58:09.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.430+0000 7f5c2f7fe640 1 --2- 172.21.15.172:0/1216876341 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5580e0de9980 0x5580e0debdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.430+0000 7f5c0f7fe640 10 client.27790 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:09.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.430+0000 7f5c2f7fe640 1 --2- 172.21.15.172:0/1216876341 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5580e0de9980 0x5580e0debdf0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8c31cebc036f186 server_cookie=1cfd0f0bdfd9dcdb in_seq=0 out_seq=0 2024-09-10T08:58:09.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.431+0000 7f5c0f7fe640 1 -- 172.21.15.172:0/1216876341 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5580e0d417f0 con 0x5580e0de9980 2024-09-10T08:58:09.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.431+0000 7f5c0f7fe640 10 client.27790 handle_command_reply: tid=0 2024-09-10T08:58:09.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.431+0000 7f5c38bae740 1 client.27790 shutdown 2024-09-10T08:58:09.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.431+0000 7f5c38bae740 1 -- 172.21.15.172:0/1216876341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dceb60 msgr2=0x5580e0dd6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.431+0000 7f5c38bae740 1 --2- 172.21.15.172:0/1216876341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dceb60 0x5580e0dd6be0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c200bc0a0 tx=0x7f5c200037a0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.431+0000 7f5c38bae740 1 -- 172.21.15.172:0/1216876341 shutdown_connections 2024-09-10T08:58:09.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.431+0000 7f5c38bae740 1 --2- 172.21.15.172:0/1216876341 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5580e0de9980 0x5580e0debdf0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.431+0000 7f5c38bae740 1 --2- 172.21.15.172:0/1216876341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dd9d40 0x5580e0dd7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.431+0000 7f5c38bae740 1 --2- 172.21.15.172:0/1216876341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dcf480 0x5580e0dd7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.431+0000 7f5c38bae740 1 --2- 172.21.15.172:0/1216876341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dceb60 0x5580e0dd6be0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.431+0000 7f5c38bae740 1 -- 172.21.15.172:0/1216876341 >> 172.21.15.172:0/1216876341 conn(0x5580e0dcb0f0 msgr2=0x5580e0dd0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:09.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.431+0000 7f5c0f7fe640 0 client.27790 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:09.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.431+0000 7f5c38bae740 1 -- 172.21.15.172:0/1216876341 shutdown_connections 2024-09-10T08:58:09.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.431+0000 7f5c38bae740 1 -- 172.21.15.172:0/1216876341 wait complete. 2024-09-10T08:58:09.437 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.431+0000 7f5c38bae740 20 client.27790 trim_cache size 0 max 16384 2024-09-10T08:58:09.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c38bae740 1 Processor -- start 2024-09-10T08:58:09.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c38bae740 1 -- start start 2024-09-10T08:58:09.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c38bae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dceb60 0x5580e0dd15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c38bae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dcf480 0x5580e0dd8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c38bae740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dd9d40 0x5580e0df1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c38bae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5c301aea00 con 0x5580e0dcf480 2024-09-10T08:58:09.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c38bae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c301b05d0 con 0x5580e0dceb60 2024-09-10T08:58:09.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c38bae740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5c301af0d0 con 0x5580e0dd9d40 2024-09-10T08:58:09.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c2f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dcf480 0x5580e0dd8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c2f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dcf480 0x5580e0dd8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40000/0 (socket says 172.21.15.172:40000) 2024-09-10T08:58:09.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c2f7fe640 1 -- 172.21.15.172:0/1161412222 learned_addr learned my addr 172.21.15.172:0/1161412222 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:09.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c2ffff640 1 --2- 172.21.15.172:0/1161412222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dceb60 0x5580e0dd15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c34d72640 1 --2- 172.21.15.172:0/1161412222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dd9d40 0x5580e0df1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c2f7fe640 1 -- 172.21.15.172:0/1161412222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dd9d40 msgr2=0x5580e0df1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:09.461 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c2f7fe640 1 --2- 172.21.15.172:0/1161412222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dd9d40 0x5580e0df1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c2f7fe640 1 -- 172.21.15.172:0/1161412222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dceb60 msgr2=0x5580e0dd15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:09.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c2f7fe640 1 --2- 172.21.15.172:0/1161412222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dceb60 0x5580e0dd15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c2f7fe640 1 -- 172.21.15.172:0/1161412222 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580e0dcaf20 con 0x5580e0dcf480 2024-09-10T08:58:09.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c34d72640 1 --2- 172.21.15.172:0/1161412222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dd9d40 0x5580e0df1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:09.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.459+0000 7f5c2ffff640 1 --2- 172.21.15.172:0/1161412222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dceb60 0x5580e0dd15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:09.462 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.460+0000 7f5c2f7fe640 1 --2- 172.21.15.172:0/1161412222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dcf480 0x5580e0dd8a40 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400abf0 tx=0x7f5c24005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d354457135e9559 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:09.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.460+0000 7f5c15ffb640 1 -- 172.21.15.172:0/1161412222 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2401d040 con 0x5580e0dcf480 2024-09-10T08:58:09.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.460+0000 7f5c15ffb640 1 -- 172.21.15.172:0/1161412222 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c2401e040 con 0x5580e0dcf480 2024-09-10T08:58:09.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.460+0000 7f5c15ffb640 1 -- 172.21.15.172:0/1161412222 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c240052a0 con 0x5580e0dcf480 2024-09-10T08:58:09.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.461+0000 7f5c38bae740 1 -- 172.21.15.172:0/1161412222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dcf480 msgr2=0x5580e0dd8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.461+0000 7f5c38bae740 1 --2- 172.21.15.172:0/1161412222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dcf480 0x5580e0dd8a40 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400abf0 tx=0x7f5c24005dc0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.464 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.461+0000 7f5c38bae740 1 -- 172.21.15.172:0/1161412222 shutdown_connections 2024-09-10T08:58:09.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.461+0000 7f5c38bae740 1 --2- 172.21.15.172:0/1161412222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dd9d40 0x5580e0df1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.461+0000 7f5c38bae740 1 --2- 172.21.15.172:0/1161412222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dcf480 0x5580e0dd8a40 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.461+0000 7f5c38bae740 1 --2- 172.21.15.172:0/1161412222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dceb60 0x5580e0dd15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.461+0000 7f5c38bae740 1 -- 172.21.15.172:0/1161412222 >> 172.21.15.172:0/1161412222 conn(0x5580e0dcb0f0 msgr2=0x5580e0dd8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:09.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.461+0000 7f5c38bae740 1 -- 172.21.15.172:0/1161412222 shutdown_connections 2024-09-10T08:58:09.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.461+0000 7f5c38bae740 1 -- 172.21.15.172:0/1161412222 wait complete. 2024-09-10T08:58:09.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.461+0000 7f5c38bae740 1 -- start start 2024-09-10T08:58:09.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.461+0000 7f5c38bae740 1 Processor -- start 2024-09-10T08:58:09.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c38bae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dceb60 0x5580e0de2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c38bae740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dcf480 0x5580e0ddd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c38bae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dd9d40 0x5580e0dddc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.465 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c38bae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5c301aff00 con 0x5580e0dceb60 2024-09-10T08:58:09.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c38bae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c301af830 con 0x5580e0dd9d40 2024-09-10T08:58:09.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c38bae740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5c301ac9e0 con 0x5580e0dcf480 2024-09-10T08:58:09.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c38bae740 10 client.0 _ll_register_callbacks cb 0x7f5c200bbee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:09.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c2ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dceb60 0x5580e0de2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c2ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dceb60 0x5580e0de2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40002/0 (socket says 172.21.15.172:40002) 2024-09-10T08:58:09.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c2ffff640 1 -- 172.21.15.172:0/1871488249 learned_addr learned my addr 172.21.15.172:0/1871488249 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:09.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c34d72640 1 --2- 172.21.15.172:0/1871488249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dd9d40 0x5580e0dddc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c2f7fe640 1 --2- 172.21.15.172:0/1871488249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dcf480 0x5580e0ddd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c2ffff640 1 -- 172.21.15.172:0/1871488249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dcf480 msgr2=0x5580e0ddd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:09.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c2ffff640 1 --2- 172.21.15.172:0/1871488249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dcf480 0x5580e0ddd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c2ffff640 1 -- 172.21.15.172:0/1871488249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dd9d40 msgr2=0x5580e0dddc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:09.466 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c2ffff640 1 --2- 172.21.15.172:0/1871488249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dd9d40 0x5580e0dddc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c2ffff640 1 -- 172.21.15.172:0/1871488249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580e0dde2f0 con 0x5580e0dceb60 2024-09-10T08:58:09.467 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:09 smithi172 ceph-mon[30854]: pgmap v708: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-09-10T08:58:09.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c2ffff640 1 --2- 172.21.15.172:0/1871488249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dceb60 0x5580e0de2390 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f5c200bc330 tx=0x7f5c200038d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5580e720ad8a10d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:09.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.462+0000 7f5c0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:09.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.463+0000 7f5c0f7fe640 1 -- 172.21.15.172:0/1871488249 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c20049070 con 0x5580e0dceb60 2024-09-10T08:58:09.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.463+0000 7f5c0f7fe640 1 -- 172.21.15.172:0/1871488249 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5c200bc800 con 0x5580e0dceb60 2024-09-10T08:58:09.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.463+0000 7f5c38bae740 1 -- 172.21.15.172:0/1871488249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c2401e040 con 0x5580e0dceb60 2024-09-10T08:58:09.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.463+0000 7f5c0f7fe640 1 -- 172.21.15.172:0/1871488249 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5c200a4050 con 0x5580e0dceb60 2024-09-10T08:58:09.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.463+0000 7f5c38bae740 10 client.19017 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:09.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.463+0000 7f5c38bae740 1 -- 172.21.15.172:0/1871488249 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5580e0e54540 con 0x5580e0dceb60 2024-09-10T08:58:09.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.463+0000 7f5c0f7fe640 1 -- 172.21.15.172:0/1871488249 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c2401e040 con 0x5580e0dceb60 2024-09-10T08:58:09.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.463+0000 7f5c0f7fe640 1 -- 172.21.15.172:0/1871488249 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f5c200b80f0 con 0x5580e0dceb60 2024-09-10T08:58:09.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.463+0000 7f5c38bae740 10 client.19017 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:09.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.463+0000 7f5c38bae740 10 client.19017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:58:09.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.463+0000 7f5c38bae740 20 client.19017 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:09.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.463+0000 7f5c38bae740 1 --2- 172.21.15.172:0/1871488249 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5580e0e54ad0 0x5580e0e56ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:09.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.463+0000 7f5c38bae740 4 client.19017 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:58:09.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.463+0000 7f5c38bae740 1 -- 172.21.15.172:0/1871488249 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5580e0d3bd00 con 0x5580e0e54ad0 2024-09-10T08:58:09.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.464+0000 7f5c2f7fe640 1 --2- 172.21.15.172:0/1871488249 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5580e0e54ad0 0x5580e0e56ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:09.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.464+0000 7f5c2f7fe640 1 --2- 172.21.15.172:0/1871488249 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5580e0e54ad0 0x5580e0e56ec0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0991f656756c62d server_cookie=a119c641c1e985df in_seq=0 out_seq=0 2024-09-10T08:58:09.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.464+0000 7f5c0f7fe640 10 client.19017 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:09.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.465+0000 7f5c0f7fe640 1 -- 172.21.15.172:0/1871488249 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5580e0d3bd00 con 0x5580e0e54ad0 2024-09-10T08:58:09.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.465+0000 7f5c0f7fe640 10 client.19017 handle_command_reply: tid=0 2024-09-10T08:58:09.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.465+0000 7f5c38bae740 1 client.19017 shutdown 2024-09-10T08:58:09.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.466+0000 7f5c38bae740 1 -- 172.21.15.172:0/1871488249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dceb60 msgr2=0x5580e0de2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.466+0000 7f5c38bae740 1 --2- 172.21.15.172:0/1871488249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dceb60 0x5580e0de2390 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f5c200bc330 tx=0x7f5c200038d0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.466+0000 7f5c38bae740 1 -- 172.21.15.172:0/1871488249 shutdown_connections 2024-09-10T08:58:09.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.466+0000 7f5c38bae740 1 --2- 172.21.15.172:0/1871488249 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5580e0e54ad0 0x5580e0e56ec0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.466+0000 7f5c38bae740 1 --2- 172.21.15.172:0/1871488249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580e0dd9d40 0x5580e0dddc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.466+0000 7f5c38bae740 1 --2- 172.21.15.172:0/1871488249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580e0dcf480 0x5580e0ddd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.466+0000 7f5c38bae740 1 --2- 172.21.15.172:0/1871488249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5580e0dceb60 0x5580e0de2390 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.466+0000 7f5c38bae740 1 -- 172.21.15.172:0/1871488249 >> 172.21.15.172:0/1871488249 conn(0x5580e0dcb0f0 msgr2=0x5580e0dca270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:09.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.466+0000 7f5c0f7fe640 0 client.19017 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:09.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.466+0000 7f5c38bae740 1 -- 172.21.15.172:0/1871488249 shutdown_connections 2024-09-10T08:58:09.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.466+0000 7f5c38bae740 1 -- 172.21.15.172:0/1871488249 wait complete. 2024-09-10T08:58:09.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.466+0000 7f5c38bae740 20 client.19017 trim_cache size 0 max 16384 2024-09-10T08:58:09.472 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:58:09.472 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T08:58:09.472 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "9a7709d9-167f-4525-ac0b-a36060301a30", 2024-09-10T08:58:09.472 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T08:58:09.472 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:58:09.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.466+0000 7f5c367d5640 1 -- 172.21.15.172:0/2363944016 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5c180a2c00 msgr2=0x7f5c180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.466+0000 7f5c367d5640 1 --2- 172.21.15.172:0/2363944016 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5c180a2c00 0x7f5c180a50c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5c28007330 tx=0x7f5c28007e70 comp rx=0 tx=0).stop 2024-09-10T08:58:09.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.466+0000 7f5c367d5640 1 -- 172.21.15.172:0/2363944016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c30100100 msgr2=0x7f5c30198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:09.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.466+0000 7f5c367d5640 1 --2- 172.21.15.172:0/2363944016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c30100100 0x7f5c30198380 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f5c20006010 tx=0x7f5c20004270 comp rx=0 tx=0).stop 2024-09-10T08:58:09.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.467+0000 7f5c367d5640 1 -- 172.21.15.172:0/2363944016 shutdown_connections 2024-09-10T08:58:09.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.467+0000 7f5c367d5640 1 --2- 172.21.15.172:0/2363944016 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5c180a2c00 0x7f5c180a50c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.467+0000 7f5c367d5640 1 --2- 172.21.15.172:0/2363944016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c30106b80 0x7f5c3019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.467+0000 7f5c367d5640 1 --2- 172.21.15.172:0/2363944016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c30100aa0 0x7f5c301988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.467+0000 7f5c367d5640 1 --2- 172.21.15.172:0/2363944016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c30100100 0x7f5c30198380 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:09.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.467+0000 7f5c367d5640 1 -- 172.21.15.172:0/2363944016 >> 172.21.15.172:0/2363944016 conn(0x7f5c30077230 msgr2=0x7f5c30104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:09.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.467+0000 7f5c367d5640 1 -- 172.21.15.172:0/2363944016 shutdown_connections 2024-09-10T08:58:09.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:09.467+0000 7f5c367d5640 1 -- 172.21.15.172:0/2363944016 wait complete. 2024-09-10T08:58:09.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:09 smithi174 ceph-mon[34068]: pgmap v708: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-09-10T08:58:09.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:09 smithi191 ceph-mon[35314]: pgmap v708: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-09-10T08:58:10.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:10 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T08:58:10.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:10 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:10 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:58:10.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:10 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:58:10.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:10 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:58:10.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:10 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:58:10.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:10 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:58:10.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:10 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:10 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:10 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:10 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:58:10.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:10 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:10 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:10 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:10 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T08:58:10.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:10 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:10 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:58:10.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:10 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:58:10.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:10 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:58:10.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:10 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:58:10.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:10 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:58:10.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:10 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:10 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:10 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:10 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:58:10.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:10 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:10 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:10 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:10 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T08:58:10.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:10 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:10 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:58:10.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:10 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:58:10.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:10 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:58:10.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:10 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:58:10.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:10 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:58:10.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:10 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:10 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:10 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:10 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:58:10.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:10 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:10 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:10.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:10 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:58:11.663 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:58:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:11 smithi174 ceph-mon[34068]: pgmap v709: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 MiB/s wr, 34 op/s 2024-09-10T08:58:11.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:11 smithi191 ceph-mon[35314]: pgmap v709: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 MiB/s wr, 34 op/s 2024-09-10T08:58:11.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:11 smithi172 ceph-mon[30854]: pgmap v709: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 MiB/s wr, 34 op/s 2024-09-10T08:58:11.948 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:58:12.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.639+0000 7fd92c340640 1 -- 172.21.15.172:0/1810597390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd924100e20 msgr2=0x7fd924101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:12.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.639+0000 7fd92c340640 1 --2- 172.21.15.172:0/1810597390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd924100e20 0x7fd924101200 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fd920009900 tx=0x7fd92002a480 comp rx=0 tx=0).stop 2024-09-10T08:58:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.639+0000 7fd92c340640 1 -- 172.21.15.172:0/1810597390 shutdown_connections 2024-09-10T08:58:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.639+0000 7fd92c340640 1 --2- 172.21.15.172:0/1810597390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd924102ef0 0x7fd92410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.639+0000 7fd92c340640 1 --2- 172.21.15.172:0/1810597390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd924102570 0x7fd9241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.639+0000 7fd92c340640 1 --2- 172.21.15.172:0/1810597390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd924100e20 0x7fd924101200 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.639+0000 7fd92c340640 1 -- 172.21.15.172:0/1810597390 >> 172.21.15.172:0/1810597390 conn(0x7fd9240fc7d0 msgr2=0x7fd9240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.639+0000 7fd92c340640 1 -- 172.21.15.172:0/1810597390 shutdown_connections 2024-09-10T08:58:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.639+0000 7fd92c340640 1 -- 172.21.15.172:0/1810597390 wait complete. 2024-09-10T08:58:12.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.640+0000 7fd92c340640 1 Processor -- start 2024-09-10T08:58:12.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.640+0000 7fd92c340640 1 -- start start 2024-09-10T08:58:12.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.640+0000 7fd92c340640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd924100e20 0x7fd9241984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.640+0000 7fd92c340640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd924102570 0x7fd924198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.640+0000 7fd92c340640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd924102ef0 0x7fd92419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.640+0000 7fd92c340640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd924110ed0 con 0x7fd924102ef0 2024-09-10T08:58:12.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.640+0000 7fd92c340640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd924110d50 con 0x7fd924102570 2024-09-10T08:58:12.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.640+0000 7fd92c340640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd924111050 con 0x7fd924100e20 2024-09-10T08:58:12.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.640+0000 7fd92a8dd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd924102ef0 0x7fd92419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.640+0000 7fd92a0dc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd924100e20 0x7fd9241984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.643 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.640+0000 7fd92a8dd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd924102ef0 0x7fd92419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40028/0 (socket says 172.21.15.172:40028) 2024-09-10T08:58:12.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.640+0000 7fd92a8dd640 1 -- 172.21.15.172:0/1412962039 learned_addr learned my addr 172.21.15.172:0/1412962039 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:12.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.641+0000 7fd9298db640 1 --2- 172.21.15.172:0/1412962039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd924102570 0x7fd924198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.641+0000 7fd92a0dc640 1 -- 172.21.15.172:0/1412962039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd924102570 msgr2=0x7fd924198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:12.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.641+0000 7fd92a0dc640 1 --2- 172.21.15.172:0/1412962039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd924102570 0x7fd924198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.641+0000 7fd92a0dc640 1 -- 172.21.15.172:0/1412962039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd924102ef0 msgr2=0x7fd92419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:12.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.641+0000 7fd92a0dc640 1 --2- 172.21.15.172:0/1412962039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd924102ef0 0x7fd92419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.641+0000 7fd92a0dc640 1 -- 172.21.15.172:0/1412962039 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd92419d470 con 0x7fd924100e20 2024-09-10T08:58:12.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.641+0000 7fd92a8dd640 1 --2- 172.21.15.172:0/1412962039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd924102ef0 0x7fd92419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:12.644 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.641+0000 7fd92a0dc640 1 --2- 172.21.15.172:0/1412962039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd924100e20 0x7fd9241984c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd920005940 tx=0x7fd9200042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:12.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.641+0000 7fd91b7fe640 1 -- 172.21.15.172:0/1412962039 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92003f030 con 0x7fd924100e20 2024-09-10T08:58:12.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.641+0000 7fd92c340640 1 -- 172.21.15.172:0/1412962039 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd92419d700 con 0x7fd924100e20 2024-09-10T08:58:12.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.641+0000 7fd91b7fe640 1 -- 172.21.15.172:0/1412962039 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd920003cb0 con 0x7fd924100e20 2024-09-10T08:58:12.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.641+0000 7fd91b7fe640 1 -- 172.21.15.172:0/1412962039 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920035070 con 0x7fd924100e20 2024-09-10T08:58:12.645 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.641+0000 7fd92c340640 1 -- 172.21.15.172:0/1412962039 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9240fcbb0 con 0x7fd924100e20 2024-09-10T08:58:12.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.644+0000 7fd91b7fe640 1 -- 172.21.15.172:0/1412962039 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd920042030 con 0x7fd924100e20 2024-09-10T08:58:12.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.645+0000 7fd92e719740 1 Processor -- start 2024-09-10T08:58:12.647 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.645+0000 7fd92e719740 1 -- start start 2024-09-10T08:58:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.645+0000 7fd91b7fe640 1 --2- 172.21.15.172:0/1412962039 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd9100a2b80 0x7fd9100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.645+0000 7fd92e719740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eae4a0 0x55f969eae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.645+0000 7fd92a8dd640 1 --2- 172.21.15.172:0/1412962039 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd9100a2b80 0x7fd9100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.645+0000 7fd92e719740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eaedc0 0x55f969eb9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.645+0000 7fd91b7fe640 1 -- 172.21.15.172:0/1412962039 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd9200ef920 con 0x7fd924100e20 2024-09-10T08:58:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.645+0000 7fd92e719740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eb9680 0x55f969ebba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.645+0000 7fd92e719740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f969e142e0 con 0x55f969eb9680 2024-09-10T08:58:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.645+0000 7fd92e719740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f969dbbce0 con 0x55f969eaedc0 2024-09-10T08:58:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.645+0000 7fd92e719740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f969e102a0 con 0x55f969eae4a0 2024-09-10T08:58:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.645+0000 7fd92a0dc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eae4a0 0x55f969eae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.645+0000 7fd9298db640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eaedc0 0x55f969eb9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.646+0000 7fd9298db640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eaedc0 0x55f969eb9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:54726/0 (socket says 172.21.15.172:54726) 2024-09-10T08:58:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.646+0000 7fd9298db640 1 -- 172.21.15.172:0/3770816829 learned_addr learned my addr 172.21.15.172:0/3770816829 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.646+0000 7fd92a8dd640 1 --2- 172.21.15.172:0/3770816829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eb9680 0x55f969ebba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.646+0000 7fd92a8dd640 1 --2- 172.21.15.172:0/1412962039 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd9100a2b80 0x7fd9100a5040 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00e890 tx=0x7fd91c009710 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.646+0000 7fd9298db640 1 -- 172.21.15.172:0/3770816829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eae4a0 msgr2=0x55f969eae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.646+0000 7fd9298db640 1 --2- 172.21.15.172:0/3770816829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eae4a0 0x55f969eae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.646+0000 7fd9298db640 1 -- 172.21.15.172:0/3770816829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eb9680 msgr2=0x55f969ebba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.646+0000 7fd9298db640 1 --2- 172.21.15.172:0/3770816829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eb9680 0x55f969ebba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.646+0000 7fd9298db640 1 -- 172.21.15.172:0/3770816829 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f969da0440 con 0x55f969eaedc0 2024-09-10T08:58:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.646+0000 7fd92a0dc640 1 --2- 172.21.15.172:0/3770816829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eae4a0 0x55f969eae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.646+0000 7fd9298db640 1 --2- 172.21.15.172:0/3770816829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eaedc0 0x55f969eb9140 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd914009a40 tx=0x7fd914009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db890e1217d4fa87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.646+0000 7fd918dea640 1 -- 172.21.15.172:0/3770816829 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914017070 con 0x55f969eaedc0 2024-09-10T08:58:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.646+0000 7fd918dea640 1 -- 172.21.15.172:0/3770816829 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd914013470 con 0x55f969eaedc0 2024-09-10T08:58:12.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.646+0000 7fd918dea640 1 -- 172.21.15.172:0/3770816829 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914012630 con 0x55f969eaedc0 2024-09-10T08:58:12.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.647+0000 7fd92e719740 1 -- 172.21.15.172:0/3770816829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eaedc0 msgr2=0x55f969eb9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:12.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.647+0000 7fd92e719740 1 --2- 172.21.15.172:0/3770816829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eaedc0 0x55f969eb9140 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd914009a40 tx=0x7fd914009f00 comp rx=0 tx=0).stop 2024-09-10T08:58:12.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.647+0000 7fd92e719740 1 -- 172.21.15.172:0/3770816829 shutdown_connections 2024-09-10T08:58:12.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.647+0000 7fd92e719740 1 --2- 172.21.15.172:0/3770816829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eb9680 0x55f969ebba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.647+0000 7fd92e719740 1 --2- 172.21.15.172:0/3770816829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eaedc0 0x55f969eb9140 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.647+0000 7fd92e719740 1 --2- 172.21.15.172:0/3770816829 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eae4a0 0x55f969eae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.647+0000 7fd92e719740 1 -- 172.21.15.172:0/3770816829 >> 172.21.15.172:0/3770816829 conn(0x55f969eaaa30 msgr2=0x55f969eaae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.647+0000 7fd92e719740 1 -- 172.21.15.172:0/3770816829 shutdown_connections 2024-09-10T08:58:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.647+0000 7fd92e719740 1 -- 172.21.15.172:0/3770816829 wait complete. 2024-09-10T08:58:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.648+0000 7fd92e719740 1 -- start start 2024-09-10T08:58:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.648+0000 7fd92e719740 1 Processor -- start 2024-09-10T08:58:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.648+0000 7fd92e719740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eae4a0 0x55f969eb8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.648+0000 7fd92e719740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eaedc0 0x55f969eb61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.648+0000 7fd92e719740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eb9680 0x55f969eb66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.648+0000 7fd92e719740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f969ebc250 con 0x55f969eae4a0 2024-09-10T08:58:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.648+0000 7fd92e719740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f969dcf050 con 0x55f969eaedc0 2024-09-10T08:58:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.648+0000 7fd92e719740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f969ebc3d0 con 0x55f969eb9680 2024-09-10T08:58:12.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.648+0000 7fd92e719740 10 client.0 _ll_register_callbacks cb 0x55f969e0f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:12.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.648+0000 7fd92a0dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eae4a0 0x55f969eb8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.648+0000 7fd92a0dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eae4a0 0x55f969eb8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40042/0 (socket says 172.21.15.172:40042) 2024-09-10T08:58:12.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.649+0000 7fd92a0dc640 1 -- 172.21.15.172:0/2131543361 learned_addr learned my addr 172.21.15.172:0/2131543361 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:12.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.649+0000 7fd9298db640 1 --2- 172.21.15.172:0/2131543361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eaedc0 0x55f969eb61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.649+0000 7fd92a8dd640 1 --2- 172.21.15.172:0/2131543361 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eb9680 0x55f969eb66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.649+0000 7fd92a0dc640 1 -- 172.21.15.172:0/2131543361 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eb9680 msgr2=0x55f969eb66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:12.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.649+0000 7fd92a0dc640 1 --2- 172.21.15.172:0/2131543361 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eb9680 0x55f969eb66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.649+0000 7fd92a0dc640 1 -- 172.21.15.172:0/2131543361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eaedc0 msgr2=0x55f969eb61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:12.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.649+0000 7fd92a0dc640 1 --2- 172.21.15.172:0/2131543361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eaedc0 0x55f969eb61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.649+0000 7fd92a0dc640 1 -- 172.21.15.172:0/2131543361 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f969b853c0 con 0x55f969eae4a0 2024-09-10T08:58:12.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.649+0000 7fd92a0dc640 1 --2- 172.21.15.172:0/2131543361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eae4a0 0x55f969eb8ef0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fd9200bbac0 tx=0x7fd9200045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d0a94d32f4567b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:12.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.649+0000 7fd90dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:12.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.649+0000 7fd90dffb640 1 -- 172.21.15.172:0/2131543361 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920049040 con 0x55f969eae4a0 2024-09-10T08:58:12.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.649+0000 7fd92e719740 1 -- 172.21.15.172:0/2131543361 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd914013470 con 0x55f969eae4a0 2024-09-10T08:58:12.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.649+0000 7fd90dffb640 1 -- 172.21.15.172:0/2131543361 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd92003bb00 con 0x55f969eae4a0 2024-09-10T08:58:12.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.650+0000 7fd90dffb640 1 -- 172.21.15.172:0/2131543361 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd9200bbdc0 con 0x55f969eae4a0 2024-09-10T08:58:12.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.650+0000 7fd92e719740 10 client.19029 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:12.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.650+0000 7fd92e719740 1 -- 172.21.15.172:0/2131543361 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f969ec7b40 con 0x55f969eae4a0 2024-09-10T08:58:12.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.650+0000 7fd90dffb640 1 -- 172.21.15.172:0/2131543361 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd914013470 con 0x55f969eae4a0 2024-09-10T08:58:12.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.650+0000 7fd90dffb640 1 -- 172.21.15.172:0/2131543361 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fd920074870 con 0x55f969eae4a0 2024-09-10T08:58:12.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.650+0000 7fd92e719740 10 client.19029 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:12.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.650+0000 7fd92e719740 10 client.19029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:58:12.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.650+0000 7fd92e719740 20 client.19029 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:12.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.650+0000 7fd92e719740 1 --2- 172.21.15.172:0/2131543361 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f969ec8fa0 0x55f969ecb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.650+0000 7fd92e719740 4 client.19029 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:58:12.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.650+0000 7fd92e719740 1 -- 172.21.15.172:0/2131543361 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f969e21850 con 0x55f969ec8fa0 2024-09-10T08:58:12.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.651+0000 7fd9298db640 1 --2- 172.21.15.172:0/2131543361 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f969ec8fa0 0x55f969ecb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.651+0000 7fd90dffb640 10 client.19029 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:12.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.651+0000 7fd9298db640 1 --2- 172.21.15.172:0/2131543361 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f969ec8fa0 0x55f969ecb3b0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75530ce1699ade46 server_cookie=e578c3fca1369a01 in_seq=0 out_seq=0 2024-09-10T08:58:12.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.652+0000 7fd90dffb640 1 -- 172.21.15.172:0/2131543361 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f969e21850 con 0x55f969ec8fa0 2024-09-10T08:58:12.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.652+0000 7fd90dffb640 10 client.19029 handle_command_reply: tid=0 2024-09-10T08:58:12.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.652+0000 7fd92e719740 1 client.19029 shutdown 2024-09-10T08:58:12.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.652+0000 7fd92e719740 1 -- 172.21.15.172:0/2131543361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eae4a0 msgr2=0x55f969eb8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:12.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.652+0000 7fd92e719740 1 --2- 172.21.15.172:0/2131543361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eae4a0 0x55f969eb8ef0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fd9200bbac0 tx=0x7fd9200045a0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.652+0000 7fd92e719740 1 -- 172.21.15.172:0/2131543361 shutdown_connections 2024-09-10T08:58:12.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.652+0000 7fd92e719740 1 --2- 172.21.15.172:0/2131543361 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f969ec8fa0 0x55f969ecb3b0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.652+0000 7fd92e719740 1 --2- 172.21.15.172:0/2131543361 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eb9680 0x55f969eb66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.652+0000 7fd92e719740 1 --2- 172.21.15.172:0/2131543361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eaedc0 0x55f969eb61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.652+0000 7fd90dffb640 0 client.19029 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:12.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.652+0000 7fd92e719740 1 --2- 172.21.15.172:0/2131543361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eae4a0 0x55f969eb8ef0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.652+0000 7fd92e719740 1 -- 172.21.15.172:0/2131543361 >> 172.21.15.172:0/2131543361 conn(0x55f969eaaa30 msgr2=0x55f969ea9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:12.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.652+0000 7fd92e719740 1 -- 172.21.15.172:0/2131543361 shutdown_connections 2024-09-10T08:58:12.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.652+0000 7fd92e719740 1 -- 172.21.15.172:0/2131543361 wait complete. 2024-09-10T08:58:12.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.653+0000 7fd92e719740 20 client.19029 trim_cache size 0 max 16384 2024-09-10T08:58:12.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.679+0000 7fd92e719740 1 Processor -- start 2024-09-10T08:58:12.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.679+0000 7fd92e719740 1 -- start start 2024-09-10T08:58:12.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.679+0000 7fd92e719740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eaedc0 0x55f969eb0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.679+0000 7fd92e719740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eb9680 0x55f969ed0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.679+0000 7fd92e719740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eb4ed0 0x55f969ed1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.679+0000 7fd92e719740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd9241ac800 con 0x55f969eb4ed0 2024-09-10T08:58:12.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.679+0000 7fd92e719740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9241ae3d0 con 0x55f969eaedc0 2024-09-10T08:58:12.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.679+0000 7fd92e719740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd9241aced0 con 0x55f969eb9680 2024-09-10T08:58:12.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.679+0000 7fd92a8dd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eb4ed0 0x55f969ed1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.679+0000 7fd92a8dd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eb4ed0 0x55f969ed1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40046/0 (socket says 172.21.15.172:40046) 2024-09-10T08:58:12.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.679+0000 7fd92a8dd640 1 -- 172.21.15.172:0/1705688400 learned_addr learned my addr 172.21.15.172:0/1705688400 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:12.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.680+0000 7fd92a0dc640 1 --2- 172.21.15.172:0/1705688400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eaedc0 0x55f969eb0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.680+0000 7fd9298db640 1 --2- 172.21.15.172:0/1705688400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eb9680 0x55f969ed0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.680+0000 7fd92a8dd640 1 -- 172.21.15.172:0/1705688400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eb9680 msgr2=0x55f969ed0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:12.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.680+0000 7fd92a8dd640 1 --2- 172.21.15.172:0/1705688400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eb9680 0x55f969ed0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.680+0000 7fd92a8dd640 1 -- 172.21.15.172:0/1705688400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eaedc0 msgr2=0x55f969eb0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:12.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.680+0000 7fd92a8dd640 1 --2- 172.21.15.172:0/1705688400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eaedc0 0x55f969eb0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.680+0000 7fd92a8dd640 1 -- 172.21.15.172:0/1705688400 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f969eaa4d0 con 0x55f969eb4ed0 2024-09-10T08:58:12.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.680+0000 7fd92a8dd640 1 --2- 172.21.15.172:0/1705688400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eb4ed0 0x55f969ed1200 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fd91c012bf0 tx=0x7fd91c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b343cb3f829d03ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:12.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.680+0000 7fd90ffff640 1 -- 172.21.15.172:0/1705688400 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c02a030 con 0x55f969eb4ed0 2024-09-10T08:58:12.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.680+0000 7fd90ffff640 1 -- 172.21.15.172:0/1705688400 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd91c020040 con 0x55f969eb4ed0 2024-09-10T08:58:12.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.680+0000 7fd90ffff640 1 -- 172.21.15.172:0/1705688400 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c02e620 con 0x55f969eb4ed0 2024-09-10T08:58:12.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.681+0000 7fd92e719740 1 -- 172.21.15.172:0/1705688400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eb4ed0 msgr2=0x55f969ed1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:12.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.681+0000 7fd92e719740 1 --2- 172.21.15.172:0/1705688400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eb4ed0 0x55f969ed1200 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fd91c012bf0 tx=0x7fd91c01d000 comp rx=0 tx=0).stop 2024-09-10T08:58:12.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.681+0000 7fd92e719740 1 -- 172.21.15.172:0/1705688400 shutdown_connections 2024-09-10T08:58:12.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.681+0000 7fd92e719740 1 --2- 172.21.15.172:0/1705688400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eb4ed0 0x55f969ed1200 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.681+0000 7fd92e719740 1 --2- 172.21.15.172:0/1705688400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eb9680 0x55f969ed0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.681+0000 7fd92e719740 1 --2- 172.21.15.172:0/1705688400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eaedc0 0x55f969eb0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.681+0000 7fd92e719740 1 -- 172.21.15.172:0/1705688400 >> 172.21.15.172:0/1705688400 conn(0x55f969eaaa30 msgr2=0x55f969eafef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:12.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.681+0000 7fd92e719740 1 -- 172.21.15.172:0/1705688400 shutdown_connections 2024-09-10T08:58:12.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.681+0000 7fd92e719740 1 -- 172.21.15.172:0/1705688400 wait complete. 2024-09-10T08:58:12.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92e719740 1 -- start start 2024-09-10T08:58:12.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92e719740 1 Processor -- start 2024-09-10T08:58:12.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92e719740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eaedc0 0x55f969ec1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92e719740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eb4ed0 0x55f969ebdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92e719740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eb9680 0x55f969ebe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92e719740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd9241add00 con 0x55f969eaedc0 2024-09-10T08:58:12.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92e719740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9241ad630 con 0x55f969eb4ed0 2024-09-10T08:58:12.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92e719740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd9241aa7e0 con 0x55f969eb9680 2024-09-10T08:58:12.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92e719740 10 client.0 _ll_register_callbacks cb 0x7fd9200b79d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:12.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92a0dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eaedc0 0x55f969ec1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92a0dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eaedc0 0x55f969ec1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40054/0 (socket says 172.21.15.172:40054) 2024-09-10T08:58:12.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92a0dc640 1 -- 172.21.15.172:0/2771811672 learned_addr learned my addr 172.21.15.172:0/2771811672 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:12.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92a8dd640 1 --2- 172.21.15.172:0/2771811672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eb9680 0x55f969ebe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd9298db640 1 --2- 172.21.15.172:0/2771811672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eb4ed0 0x55f969ebdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92a0dc640 1 -- 172.21.15.172:0/2771811672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eb9680 msgr2=0x55f969ebe3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:12.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92a0dc640 1 --2- 172.21.15.172:0/2771811672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eb9680 0x55f969ebe3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92a0dc640 1 -- 172.21.15.172:0/2771811672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eb4ed0 msgr2=0x55f969ebdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:12.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92a0dc640 1 --2- 172.21.15.172:0/2771811672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eb4ed0 0x55f969ebdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.682+0000 7fd92a0dc640 1 -- 172.21.15.172:0/2771811672 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f969ebe9e0 con 0x55f969eaedc0 2024-09-10T08:58:12.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.683+0000 7fd92a0dc640 1 --2- 172.21.15.172:0/2771811672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eaedc0 0x55f969ec1b70 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fd920006010 tx=0x7fd9200c48f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9740b041d2e8681 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:12.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.683+0000 7fd90dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:12.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.683+0000 7fd90dffb640 1 -- 172.21.15.172:0/2771811672 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920049040 con 0x55f969eaedc0 2024-09-10T08:58:12.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.683+0000 7fd90dffb640 1 -- 172.21.15.172:0/2771811672 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd92003bb00 con 0x55f969eaedc0 2024-09-10T08:58:12.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.683+0000 7fd92e719740 1 -- 172.21.15.172:0/2771811672 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd91c020040 con 0x55f969eaedc0 2024-09-10T08:58:12.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.683+0000 7fd90dffb640 1 -- 172.21.15.172:0/2771811672 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd9200cb050 con 0x55f969eaedc0 2024-09-10T08:58:12.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.683+0000 7fd92e719740 10 client.19035 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:12.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.683+0000 7fd92e719740 1 -- 172.21.15.172:0/2771811672 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f969ebe9e0 con 0x55f969eaedc0 2024-09-10T08:58:12.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.683+0000 7fd90dffb640 1 -- 172.21.15.172:0/2771811672 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd91c020040 con 0x55f969eaedc0 2024-09-10T08:58:12.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.684+0000 7fd90dffb640 1 -- 172.21.15.172:0/2771811672 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fd9200a2070 con 0x55f969eaedc0 2024-09-10T08:58:12.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.684+0000 7fd92e719740 10 client.19035 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:12.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.684+0000 7fd92e719740 10 client.19035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:58:12.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.684+0000 7fd92e719740 20 client.19035 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:12.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.684+0000 7fd92e719740 1 --2- 172.21.15.172:0/2771811672 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f969f34e70 0x55f969f37260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:12.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.684+0000 7fd92e719740 4 client.19035 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:58:12.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.684+0000 7fd92e719740 1 -- 172.21.15.172:0/2771811672 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f969e1bd60 con 0x55f969f34e70 2024-09-10T08:58:12.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.684+0000 7fd9298db640 1 --2- 172.21.15.172:0/2771811672 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f969f34e70 0x55f969f37260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:12.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.684+0000 7fd90dffb640 10 client.19035 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:12.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.684+0000 7fd9298db640 1 --2- 172.21.15.172:0/2771811672 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f969f34e70 0x55f969f37260 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=258d8872c7bcb26a server_cookie=2214405bb6ca1334 in_seq=0 out_seq=0 2024-09-10T08:58:12.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.685+0000 7fd90dffb640 1 -- 172.21.15.172:0/2771811672 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f969e1bd60 con 0x55f969f34e70 2024-09-10T08:58:12.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.685+0000 7fd90dffb640 10 client.19035 handle_command_reply: tid=0 2024-09-10T08:58:12.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.685+0000 7fd92e719740 1 client.19035 shutdown 2024-09-10T08:58:12.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.685+0000 7fd92e719740 1 -- 172.21.15.172:0/2771811672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eaedc0 msgr2=0x55f969ec1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:12.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.685+0000 7fd92e719740 1 --2- 172.21.15.172:0/2771811672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eaedc0 0x55f969ec1b70 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fd920006010 tx=0x7fd9200c48f0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.685+0000 7fd92e719740 1 -- 172.21.15.172:0/2771811672 shutdown_connections 2024-09-10T08:58:12.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.685+0000 7fd92e719740 1 --2- 172.21.15.172:0/2771811672 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f969f34e70 0x55f969f37260 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.685+0000 7fd92e719740 1 --2- 172.21.15.172:0/2771811672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f969eb9680 0x55f969ebe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.685+0000 7fd92e719740 1 --2- 172.21.15.172:0/2771811672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f969eb4ed0 0x55f969ebdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.685+0000 7fd92e719740 1 --2- 172.21.15.172:0/2771811672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f969eaedc0 0x55f969ec1b70 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.685+0000 7fd92e719740 1 -- 172.21.15.172:0/2771811672 >> 172.21.15.172:0/2771811672 conn(0x55f969eaaa30 msgr2=0x55f969eafef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:12.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.685+0000 7fd90dffb640 0 client.19035 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:12.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.685+0000 7fd92e719740 1 -- 172.21.15.172:0/2771811672 shutdown_connections 2024-09-10T08:58:12.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.685+0000 7fd92e719740 1 -- 172.21.15.172:0/2771811672 wait complete. 2024-09-10T08:58:12.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.685+0000 7fd92e719740 20 client.19035 trim_cache size 0 max 16384 2024-09-10T08:58:12.691 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:58:12.692 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T08:58:12.692 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T08:58:12.692 INFO:teuthology.orchestra.run.smithi172.stdout: "9a7709d9-167f-4525-ac0b-a36060301a30": { 2024-09-10T08:58:12.692 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T08:58:12.692 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "9a7709d9-167f-4525-ac0b-a36060301a30", 2024-09-10T08:58:12.692 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T08:58:12.692 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:58:12.692 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:58:12.692 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:58:12.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.686+0000 7fd92c340640 1 -- 172.21.15.172:0/1412962039 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd9100a2b80 msgr2=0x7fd9100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:12.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.686+0000 7fd92c340640 1 --2- 172.21.15.172:0/1412962039 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd9100a2b80 0x7fd9100a5040 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00e890 tx=0x7fd91c009710 comp rx=0 tx=0).stop 2024-09-10T08:58:12.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.686+0000 7fd92c340640 1 -- 172.21.15.172:0/1412962039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd924100e20 msgr2=0x7fd9241984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:12.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.686+0000 7fd92c340640 1 --2- 172.21.15.172:0/1412962039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd924100e20 0x7fd9241984c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd920005940 tx=0x7fd9200042f0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.686+0000 7fd92c340640 1 -- 172.21.15.172:0/1412962039 shutdown_connections 2024-09-10T08:58:12.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.686+0000 7fd92c340640 1 --2- 172.21.15.172:0/1412962039 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd9100a2b80 0x7fd9100a5040 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.686+0000 7fd92c340640 1 --2- 172.21.15.172:0/1412962039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd924102ef0 0x7fd92419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.686+0000 7fd92c340640 1 --2- 172.21.15.172:0/1412962039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd924102570 0x7fd924198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.686+0000 7fd92c340640 1 --2- 172.21.15.172:0/1412962039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd924100e20 0x7fd9241984c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:12.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.686+0000 7fd92c340640 1 -- 172.21.15.172:0/1412962039 >> 172.21.15.172:0/1412962039 conn(0x7fd9240fc7d0 msgr2=0x7fd92406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:12.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.686+0000 7fd92c340640 1 -- 172.21.15.172:0/1412962039 shutdown_connections 2024-09-10T08:58:12.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:12.686+0000 7fd92c340640 1 -- 172.21.15.172:0/1412962039 wait complete. 2024-09-10T08:58:13.383 INFO:tasks.cephfs.filesystem:scrub status for tag:9a7709d9-167f-4525-ac0b-a36060301a30 - {'path': '/', 'tag': '9a7709d9-167f-4525-ac0b-a36060301a30', 'options': 'recursive,force'} 2024-09-10T08:58:13.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:13 smithi172 ceph-mon[30854]: pgmap v710: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 39 op/s 2024-09-10T08:58:13.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:13 smithi174 ceph-mon[34068]: pgmap v710: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 39 op/s 2024-09-10T08:58:13.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:13 smithi191 ceph-mon[35314]: pgmap v710: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 39 op/s 2024-09-10T08:58:15.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:15 smithi174 ceph-mon[34068]: pgmap v711: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 31 MiB/s wr, 52 op/s 2024-09-10T08:58:15.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:15 smithi191 ceph-mon[35314]: pgmap v711: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 31 MiB/s wr, 52 op/s 2024-09-10T08:58:15.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:15 smithi172 ceph-mon[30854]: pgmap v711: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 31 MiB/s wr, 52 op/s 2024-09-10T08:58:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:17 smithi174 ceph-mon[34068]: pgmap v712: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 46 op/s 2024-09-10T08:58:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:17 smithi191 ceph-mon[35314]: pgmap v712: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 46 op/s 2024-09-10T08:58:17.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:17 smithi172 ceph-mon[30854]: pgmap v712: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 26 MiB/s wr, 46 op/s 2024-09-10T08:58:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:19 smithi174 ceph-mon[34068]: pgmap v713: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 49 op/s 2024-09-10T08:58:19.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:19 smithi191 ceph-mon[35314]: pgmap v713: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 49 op/s 2024-09-10T08:58:19.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:19 smithi172 ceph-mon[30854]: pgmap v713: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 49 op/s 2024-09-10T08:58:21.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:21 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:58:21.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:21 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:58:21.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:21 smithi174 ceph-mon[34068]: pgmap v714: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 48 op/s 2024-09-10T08:58:21.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:21 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:58:21.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:21 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:58:21.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:21 smithi191 ceph-mon[35314]: pgmap v714: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 48 op/s 2024-09-10T08:58:21.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:21 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:58:21.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:21 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:58:21.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:21 smithi172 ceph-mon[30854]: pgmap v714: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 48 op/s 2024-09-10T08:58:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.431+0000 7fbca7427640 1 -- 172.21.15.172:0/579743090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbca01025f0 msgr2=0x7fbca0108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.431+0000 7fbca7427640 1 --2- 172.21.15.172:0/579743090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbca01025f0 0x7fbca0108e80 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009ca0 tx=0x7fbc9802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.431+0000 7fbca7427640 1 -- 172.21.15.172:0/579743090 shutdown_connections 2024-09-10T08:58:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.431+0000 7fbca7427640 1 --2- 172.21.15.172:0/579743090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbca01025f0 0x7fbca0108e80 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.431+0000 7fbca7427640 1 --2- 172.21.15.172:0/579743090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbca0101c30 0x7fbca01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.431+0000 7fbca7427640 1 --2- 172.21.15.172:0/579743090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbca0100a40 0x7fbca0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.431+0000 7fbca7427640 1 -- 172.21.15.172:0/579743090 >> 172.21.15.172:0/579743090 conn(0x7fbca00fc1e0 msgr2=0x7fbca00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.431+0000 7fbca7427640 1 -- 172.21.15.172:0/579743090 shutdown_connections 2024-09-10T08:58:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.431+0000 7fbca7427640 1 -- 172.21.15.172:0/579743090 wait complete. 2024-09-10T08:58:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.431+0000 7fbca7427640 1 Processor -- start 2024-09-10T08:58:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.431+0000 7fbca7427640 1 -- start start 2024-09-10T08:58:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca7427640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbca0100a40 0x7fbca0197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca7427640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbca0101c30 0x7fbca0198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca7427640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbca01025f0 0x7fbca019f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca7427640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbca010c550 con 0x7fbca01025f0 2024-09-10T08:58:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca7427640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbca010c3d0 con 0x7fbca0101c30 2024-09-10T08:58:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca7427640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbca010c6d0 con 0x7fbca0100a40 2024-09-10T08:58:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca59c4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbca01025f0 0x7fbca019f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca59c4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbca01025f0 0x7fbca019f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43012/0 (socket says 172.21.15.172:43012) 2024-09-10T08:58:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca59c4640 1 -- 172.21.15.172:0/3814279316 learned_addr learned my addr 172.21.15.172:0/3814279316 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca49c2640 1 --2- 172.21.15.172:0/3814279316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbca0101c30 0x7fbca0198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca51c3640 1 --2- 172.21.15.172:0/3814279316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbca0100a40 0x7fbca0197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca59c4640 1 -- 172.21.15.172:0/3814279316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbca0100a40 msgr2=0x7fbca0197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca59c4640 1 --2- 172.21.15.172:0/3814279316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbca0100a40 0x7fbca0197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca59c4640 1 -- 172.21.15.172:0/3814279316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbca0101c30 msgr2=0x7fbca0198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca59c4640 1 --2- 172.21.15.172:0/3814279316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbca0101c30 0x7fbca0198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca59c4640 1 -- 172.21.15.172:0/3814279316 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca019f860 con 0x7fbca01025f0 2024-09-10T08:58:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca49c2640 1 --2- 172.21.15.172:0/3814279316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbca0101c30 0x7fbca0198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.432+0000 7fbca59c4640 1 --2- 172.21.15.172:0/3814279316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbca01025f0 0x7fbca019f1c0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009dc0 tx=0x7fbc98003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.433+0000 7fbc967fc640 1 -- 172.21.15.172:0/3814279316 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98032ec0 con 0x7fbca01025f0 2024-09-10T08:58:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.433+0000 7fbc967fc640 1 -- 172.21.15.172:0/3814279316 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc980044f0 con 0x7fbca01025f0 2024-09-10T08:58:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.433+0000 7fbc967fc640 1 -- 172.21.15.172:0/3814279316 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98030d40 con 0x7fbca01025f0 2024-09-10T08:58:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.433+0000 7fbca7427640 1 -- 172.21.15.172:0/3814279316 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca019faf0 con 0x7fbca01025f0 2024-09-10T08:58:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.433+0000 7fbca7427640 1 -- 172.21.15.172:0/3814279316 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca01a0040 con 0x7fbca01025f0 2024-09-10T08:58:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.435+0000 7fbc967fc640 1 -- 172.21.15.172:0/3814279316 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbc98004040 con 0x7fbca01025f0 2024-09-10T08:58:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.435+0000 7fbcaa1c5740 1 Processor -- start 2024-09-10T08:58:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.435+0000 7fbcaa1c5740 1 -- start start 2024-09-10T08:58:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbcaa1c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a26db0 0x561543a271b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbcaa1c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a276f0 0x561543a2c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbcaa1c5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a2c780 0x561543a2ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbcaa1c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5615439862f0 con 0x561543a26db0 2024-09-10T08:58:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbcaa1c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56154392e270 con 0x561543a276f0 2024-09-10T08:58:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbcaa1c5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5615439822b0 con 0x561543a2c780 2024-09-10T08:58:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbc967fc640 1 --2- 172.21.15.172:0/3814279316 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbc8c0a2e50 0x7fbc8c0a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbca49c2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a26db0 0x561543a271b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbca51c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a276f0 0x561543a2c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbca49c2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a26db0 0x561543a271b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43028/0 (socket says 172.21.15.172:43028) 2024-09-10T08:58:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbca49c2640 1 -- 172.21.15.172:0/858023186 learned_addr learned my addr 172.21.15.172:0/858023186 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbca59c4640 1 --2- 172.21.15.172:0/3814279316 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbc8c0a2e50 0x7fbc8c0a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbc967fc640 1 -- 172.21.15.172:0/3814279316 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fbc980f0990 con 0x7fbca01025f0 2024-09-10T08:58:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbca49c2640 1 -- 172.21.15.172:0/858023186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a2c780 msgr2=0x561543a2ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbca49c2640 1 --2- 172.21.15.172:0/858023186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a2c780 0x561543a2ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbca49c2640 1 -- 172.21.15.172:0/858023186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a276f0 msgr2=0x561543a2c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbca49c2640 1 --2- 172.21.15.172:0/858023186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a276f0 0x561543a2c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.436+0000 7fbca49c2640 1 -- 172.21.15.172:0/858023186 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56154392d340 con 0x561543a26db0 2024-09-10T08:58:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.437+0000 7fbca49c2640 1 --2- 172.21.15.172:0/858023186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a26db0 0x561543a271b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fbc90008d00 tx=0x7fbc90010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e71baf81fa8107 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.437+0000 7fbca59c4640 1 --2- 172.21.15.172:0/3814279316 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbc8c0a2e50 0x7fbc8c0a5310 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbc980bcb90 tx=0x7fbc9802ec40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.437+0000 7fbc8b7fe640 1 -- 172.21.15.172:0/858023186 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90016070 con 0x561543a26db0 2024-09-10T08:58:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.437+0000 7fbc8b7fe640 1 -- 172.21.15.172:0/858023186 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc900056d0 con 0x561543a26db0 2024-09-10T08:58:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.437+0000 7fbc8b7fe640 1 -- 172.21.15.172:0/858023186 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc900126e0 con 0x561543a26db0 2024-09-10T08:58:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.438+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/858023186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a26db0 msgr2=0x561543a271b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.438+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/858023186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a26db0 0x561543a271b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fbc90008d00 tx=0x7fbc90010740 comp rx=0 tx=0).stop 2024-09-10T08:58:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.438+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/858023186 shutdown_connections 2024-09-10T08:58:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.438+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/858023186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a2c780 0x561543a2ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.438+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/858023186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a276f0 0x561543a2c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.438+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/858023186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a26db0 0x561543a271b0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.438+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/858023186 >> 172.21.15.172:0/858023186 conn(0x561543a1e010 msgr2=0x561543a1e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.438+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/858023186 shutdown_connections 2024-09-10T08:58:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.438+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/858023186 wait complete. 2024-09-10T08:58:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.438+0000 7fbcaa1c5740 1 -- start start 2024-09-10T08:58:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.438+0000 7fbcaa1c5740 1 Processor -- start 2024-09-10T08:58:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.439+0000 7fbcaa1c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a26db0 0x561543a30fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.439+0000 7fbcaa1c5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a276f0 0x561543a314e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.439+0000 7fbcaa1c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 0x561543a38520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.439+0000 7fbcaa1c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561543a2f750 con 0x561543a2c780 2024-09-10T08:58:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.439+0000 7fbcaa1c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56154390f290 con 0x561543a26db0 2024-09-10T08:58:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.439+0000 7fbcaa1c5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561543a2f8d0 con 0x561543a276f0 2024-09-10T08:58:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.439+0000 7fbcaa1c5740 10 client.0 _ll_register_callbacks cb 0x561543940140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.439+0000 7fbca49c2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 0x561543a38520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.439+0000 7fbca49c2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 0x561543a38520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43040/0 (socket says 172.21.15.172:43040) 2024-09-10T08:58:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.439+0000 7fbca51c3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a276f0 0x561543a314e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.439+0000 7fbca49c2640 1 -- 172.21.15.172:0/1377660063 learned_addr learned my addr 172.21.15.172:0/1377660063 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.439+0000 7fbca49c2640 1 --2- 172.21.15.172:0/1377660063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a26db0 0x561543a30fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.439+0000 7fbca51c3640 1 -- 172.21.15.172:0/1377660063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a26db0 msgr2=0x561543a30fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.439+0000 7fbca51c3640 1 --2- 172.21.15.172:0/1377660063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a26db0 0x561543a30fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.439+0000 7fbca51c3640 1 -- 172.21.15.172:0/1377660063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 msgr2=0x561543a38520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.439+0000 7fbca51c3640 1 --2- 172.21.15.172:0/1377660063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 0x561543a38520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.439+0000 7fbca51c3640 1 -- 172.21.15.172:0/1377660063 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561543699210 con 0x561543a276f0 2024-09-10T08:58:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.440+0000 7fbca51c3640 1 --2- 172.21.15.172:0/1377660063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a276f0 0x561543a314e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x561543a28120 tx=0x7fbc9c00bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5db140134907e6d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.440+0000 7fbc88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:58:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.441+0000 7fbc88ff9640 1 -- 172.21.15.172:0/1377660063 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c003e50 con 0x561543a276f0 2024-09-10T08:58:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.441+0000 7fbc88ff9640 1 -- 172.21.15.172:0/1377660063 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fbc9c005020 con 0x561543a276f0 2024-09-10T08:58:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.441+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/1377660063 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561543993a20 con 0x561543a276f0 2024-09-10T08:58:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.441+0000 7fbc88ff9640 1 -- 172.21.15.172:0/1377660063 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbc9c024050 con 0x561543a276f0 2024-09-10T08:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.441+0000 7fbcaa1c5740 10 client.27814 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.441+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/1377660063 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561543765620 con 0x561543a276f0 2024-09-10T08:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.442+0000 7fbc88ff9640 1 -- 172.21.15.172:0/1377660063 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561543993a20 con 0x561543a276f0 2024-09-10T08:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.442+0000 7fbc88ff9640 1 -- 172.21.15.172:0/1377660063 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fbc9c01b470 con 0x561543a276f0 2024-09-10T08:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.442+0000 7fbcaa1c5740 10 client.27814 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.442+0000 7fbcaa1c5740 10 client.27814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.442+0000 7fbcaa1c5740 20 client.27814 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.442+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/1377660063 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561543a3cd10 0x561543a3f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.442+0000 7fbcaa1c5740 4 client.27814 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.442+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/1377660063 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56154398dd70 con 0x561543a3cd10 2024-09-10T08:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.442+0000 7fbca49c2640 1 --2- 172.21.15.172:0/1377660063 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561543a3cd10 0x561543a3f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.442+0000 7fbc88ff9640 10 client.27814 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.442+0000 7fbca49c2640 1 --2- 172.21.15.172:0/1377660063 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561543a3cd10 0x561543a3f120 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=786eb4ebf696d3ff server_cookie=a484ba6779c5dd4 in_seq=0 out_seq=0 2024-09-10T08:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.443+0000 7fbc88ff9640 1 -- 172.21.15.172:0/1377660063 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56154398dd70 con 0x561543a3cd10 2024-09-10T08:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.443+0000 7fbc88ff9640 10 client.27814 handle_command_reply: tid=0 2024-09-10T08:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.443+0000 7fbcaa1c5740 1 client.27814 shutdown 2024-09-10T08:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.443+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/1377660063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a276f0 msgr2=0x561543a314e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.443+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/1377660063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a276f0 0x561543a314e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x561543a28120 tx=0x7fbc9c00bf30 comp rx=0 tx=0).stop 2024-09-10T08:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.443+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/1377660063 shutdown_connections 2024-09-10T08:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.443+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/1377660063 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561543a3cd10 0x561543a3f120 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.443+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/1377660063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 0x561543a38520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.443+0000 7fbc88ff9640 0 client.27814 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.443+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/1377660063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a276f0 0x561543a314e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.443+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/1377660063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a26db0 0x561543a30fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.443+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/1377660063 >> 172.21.15.172:0/1377660063 conn(0x561543a1e010 msgr2=0x561543a1f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:22.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.443+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/1377660063 shutdown_connections 2024-09-10T08:58:22.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.443+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/1377660063 wait complete. 2024-09-10T08:58:22.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.444+0000 7fbcaa1c5740 20 client.27814 trim_cache size 0 max 16384 2024-09-10T08:58:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.457+0000 7fbcaa1c5740 1 Processor -- start 2024-09-10T08:58:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.457+0000 7fbcaa1c5740 1 -- start start 2024-09-10T08:58:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbcaa1c5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a26db0 0x561543a1fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbcaa1c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a276f0 0x561543a384e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbcaa1c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 0x561543a43a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbcaa1c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01a81f0 con 0x561543a2c780 2024-09-10T08:58:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbcaa1c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01a9dc0 con 0x561543a276f0 2024-09-10T08:58:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbcaa1c5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01a88c0 con 0x561543a26db0 2024-09-10T08:58:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbca49c2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 0x561543a43a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbca49c2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 0x561543a43a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43042/0 (socket says 172.21.15.172:43042) 2024-09-10T08:58:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbca49c2640 1 -- 172.21.15.172:0/416072426 learned_addr learned my addr 172.21.15.172:0/416072426 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbca51c3640 1 --2- 172.21.15.172:0/416072426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a276f0 0x561543a384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbca49c2640 1 --2- 172.21.15.172:0/416072426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a26db0 0x561543a1fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbca49c2640 1 -- 172.21.15.172:0/416072426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a26db0 msgr2=0x561543a1fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbca49c2640 1 --2- 172.21.15.172:0/416072426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a26db0 0x561543a1fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbca49c2640 1 -- 172.21.15.172:0/416072426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a276f0 msgr2=0x561543a384e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbca49c2640 1 --2- 172.21.15.172:0/416072426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a276f0 0x561543a384e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbca49c2640 1 -- 172.21.15.172:0/416072426 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561543a3f7a0 con 0x561543a2c780 2024-09-10T08:58:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbca51c3640 1 --2- 172.21.15.172:0/416072426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a276f0 0x561543a384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.458+0000 7fbca49c2640 1 --2- 172.21.15.172:0/416072426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 0x561543a43a20 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fbc90008990 tx=0x7fbc90008680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a861df12c17471c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.459+0000 7fbc8affd640 1 -- 172.21.15.172:0/416072426 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90016040 con 0x561543a2c780 2024-09-10T08:58:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.459+0000 7fbc8affd640 1 -- 172.21.15.172:0/416072426 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc900054b0 con 0x561543a2c780 2024-09-10T08:58:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.459+0000 7fbc8affd640 1 -- 172.21.15.172:0/416072426 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90024040 con 0x561543a2c780 2024-09-10T08:58:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.459+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/416072426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 msgr2=0x561543a43a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.459+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/416072426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 0x561543a43a20 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fbc90008990 tx=0x7fbc90008680 comp rx=0 tx=0).stop 2024-09-10T08:58:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.459+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/416072426 shutdown_connections 2024-09-10T08:58:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.459+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/416072426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 0x561543a43a20 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.459+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/416072426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a276f0 0x561543a384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.459+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/416072426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a26db0 0x561543a1fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.459+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/416072426 >> 172.21.15.172:0/416072426 conn(0x561543a1e010 msgr2=0x561543a1e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:22.465 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.460+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/416072426 shutdown_connections 2024-09-10T08:58:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.460+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/416072426 wait complete. 2024-09-10T08:58:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.460+0000 7fbcaa1c5740 1 -- start start 2024-09-10T08:58:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.460+0000 7fbcaa1c5740 1 Processor -- start 2024-09-10T08:58:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.460+0000 7fbcaa1c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a26db0 0x561543a36200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.460+0000 7fbcaa1c5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a276f0 0x561543a36740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.460+0000 7fbcaa1c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 0x561543a37700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.460+0000 7fbcaa1c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01a96f0 con 0x561543a2c780 2024-09-10T08:58:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.460+0000 7fbcaa1c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01a9020 con 0x561543a26db0 2024-09-10T08:58:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.460+0000 7fbcaa1c5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01a61d0 con 0x561543a276f0 2024-09-10T08:58:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.460+0000 7fbcaa1c5740 10 client.0 _ll_register_callbacks cb 0x7fbc9c0040c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.460+0000 7fbca49c2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 0x561543a37700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.461+0000 7fbca49c2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 0x561543a37700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43056/0 (socket says 172.21.15.172:43056) 2024-09-10T08:58:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.461+0000 7fbca49c2640 1 -- 172.21.15.172:0/2609110747 learned_addr learned my addr 172.21.15.172:0/2609110747 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.461+0000 7fbca49c2640 1 --2- 172.21.15.172:0/2609110747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a26db0 0x561543a36200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.461+0000 7fbca51c3640 1 --2- 172.21.15.172:0/2609110747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a276f0 0x561543a36740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.461+0000 7fbca49c2640 1 -- 172.21.15.172:0/2609110747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a276f0 msgr2=0x561543a36740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.461+0000 7fbca49c2640 1 --2- 172.21.15.172:0/2609110747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a276f0 0x561543a36740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.461+0000 7fbca49c2640 1 -- 172.21.15.172:0/2609110747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a26db0 msgr2=0x561543a36200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.461+0000 7fbca49c2640 1 --2- 172.21.15.172:0/2609110747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a26db0 0x561543a36200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.461+0000 7fbca49c2640 1 -- 172.21.15.172:0/2609110747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561543a37d10 con 0x561543a2c780 2024-09-10T08:58:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.461+0000 7fbca49c2640 1 --2- 172.21.15.172:0/2609110747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 0x561543a37700 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x561543a357d0 tx=0x7fbc90005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fbc74f27c6ca13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.461+0000 7fbc88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.461+0000 7fbc88ff9640 1 -- 172.21.15.172:0/2609110747 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90016070 con 0x561543a2c780 2024-09-10T08:58:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.461+0000 7fbc88ff9640 1 -- 172.21.15.172:0/2609110747 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fbc90012680 con 0x561543a2c780 2024-09-10T08:58:22.467 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.461+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/2609110747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc900054b0 con 0x561543a2c780 2024-09-10T08:58:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.461+0000 7fbc88ff9640 1 -- 172.21.15.172:0/2609110747 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbc90014e60 con 0x561543a2c780 2024-09-10T08:58:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.461+0000 7fbcaa1c5740 10 client.19053 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.461+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/2609110747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561543aa7fd0 con 0x561543a2c780 2024-09-10T08:58:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.462+0000 7fbc88ff9640 1 -- 172.21.15.172:0/2609110747 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc900054b0 con 0x561543a2c780 2024-09-10T08:58:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.462+0000 7fbc88ff9640 1 -- 172.21.15.172:0/2609110747 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fbc90037950 con 0x561543a2c780 2024-09-10T08:58:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.462+0000 7fbcaa1c5740 10 client.19053 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.462+0000 7fbcaa1c5740 10 client.19053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:58:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.462+0000 7fbcaa1c5740 20 client.19053 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.462+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/2609110747 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561543aa8790 0x561543aaabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.462+0000 7fbcaa1c5740 4 client.19053 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:58:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.462+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/2609110747 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56154390a900 con 0x561543aa8790 2024-09-10T08:58:22.468 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.462+0000 7fbca51c3640 1 --2- 172.21.15.172:0/2609110747 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561543aa8790 0x561543aaabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.463+0000 7fbca51c3640 1 --2- 172.21.15.172:0/2609110747 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561543aa8790 0x561543aaabb0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=629235724f9e764e server_cookie=c33c32633d32925e in_seq=0 out_seq=0 2024-09-10T08:58:22.469 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.463+0000 7fbc88ff9640 10 client.19053 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.468+0000 7fbc88ff9640 1 -- 172.21.15.172:0/2609110747 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56154390a900 con 0x561543aa8790 2024-09-10T08:58:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.468+0000 7fbc88ff9640 10 client.19053 handle_command_reply: tid=0 2024-09-10T08:58:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.468+0000 7fbcaa1c5740 1 client.19053 shutdown 2024-09-10T08:58:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.469+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/2609110747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 msgr2=0x561543a37700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.469+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/2609110747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 0x561543a37700 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x561543a357d0 tx=0x7fbc90005b10 comp rx=0 tx=0).stop 2024-09-10T08:58:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.469+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/2609110747 shutdown_connections 2024-09-10T08:58:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.469+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/2609110747 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561543aa8790 0x561543aaabb0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.471 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.469+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/2609110747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561543a2c780 0x561543a37700 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.471 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.469+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/2609110747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561543a276f0 0x561543a36740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.471 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.469+0000 7fbcaa1c5740 1 --2- 172.21.15.172:0/2609110747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561543a26db0 0x561543a36200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.471 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.469+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/2609110747 >> 172.21.15.172:0/2609110747 conn(0x561543a1e010 msgr2=0x561543a1e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:22.471 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.469+0000 7fbc88ff9640 0 client.19053 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:22.471 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.469+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/2609110747 shutdown_connections 2024-09-10T08:58:22.471 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.469+0000 7fbcaa1c5740 1 -- 172.21.15.172:0/2609110747 wait complete. 2024-09-10T08:58:22.471 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.469+0000 7fbcaa1c5740 20 client.19053 trim_cache size 0 max 16384 2024-09-10T08:58:22.471 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:58:22.471 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:58:22.472 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:58:22.472 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:58:22.472 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.470+0000 7fbca7427640 1 -- 172.21.15.172:0/3814279316 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbc8c0a2e50 msgr2=0x7fbc8c0a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.470+0000 7fbca7427640 1 --2- 172.21.15.172:0/3814279316 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbc8c0a2e50 0x7fbc8c0a5310 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbc980bcb90 tx=0x7fbc9802ec40 comp rx=0 tx=0).stop 2024-09-10T08:58:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.470+0000 7fbca7427640 1 -- 172.21.15.172:0/3814279316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbca01025f0 msgr2=0x7fbca019f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.470+0000 7fbca7427640 1 --2- 172.21.15.172:0/3814279316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbca01025f0 0x7fbca019f1c0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009dc0 tx=0x7fbc98003ba0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.470+0000 7fbca7427640 1 -- 172.21.15.172:0/3814279316 shutdown_connections 2024-09-10T08:58:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.470+0000 7fbca7427640 1 --2- 172.21.15.172:0/3814279316 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbc8c0a2e50 0x7fbc8c0a5310 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.470+0000 7fbca7427640 1 --2- 172.21.15.172:0/3814279316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbca01025f0 0x7fbca019f1c0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.470+0000 7fbca7427640 1 --2- 172.21.15.172:0/3814279316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbca0101c30 0x7fbca0198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.470+0000 7fbca7427640 1 --2- 172.21.15.172:0/3814279316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbca0100a40 0x7fbca0197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.470+0000 7fbca7427640 1 -- 172.21.15.172:0/3814279316 >> 172.21.15.172:0/3814279316 conn(0x7fbca00fc1e0 msgr2=0x7fbca00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.470+0000 7fbca7427640 1 -- 172.21.15.172:0/3814279316 shutdown_connections 2024-09-10T08:58:22.473 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:22.470+0000 7fbca7427640 1 -- 172.21.15.172:0/3814279316 wait complete. 2024-09-10T08:58:22.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:22 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:58:22.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:22 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:58:22.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:22 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:58:22.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:22 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:58:22.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:22 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:58:22.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:22 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:58:22.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:22 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:58:22.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:22 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:58:22.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:22 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:58:23.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:23 smithi174 ceph-mon[34068]: pgmap v715: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 MiB/s wr, 38 op/s 2024-09-10T08:58:23.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:23 smithi191 ceph-mon[35314]: pgmap v715: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 MiB/s wr, 38 op/s 2024-09-10T08:58:23.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:23 smithi172 ceph-mon[30854]: pgmap v715: 129 pgs: 129 active+clean; 12 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 MiB/s wr, 38 op/s 2024-09-10T08:58:24.649 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.1 2024-09-10T08:58:24.650 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:58:24.935 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:58:25.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.486+0000 7f208eb8f640 1 -- 172.21.15.172:0/3731679152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2088100a70 msgr2=0x7f2088100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:25.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.486+0000 7f208eb8f640 1 --2- 172.21.15.172:0/3731679152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2088100a70 0x7f2088100ed0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f2074009930 tx=0x7f207402a420 comp rx=0 tx=0).stop 2024-09-10T08:58:25.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.486+0000 7f208eb8f640 1 -- 172.21.15.172:0/3731679152 shutdown_connections 2024-09-10T08:58:25.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.486+0000 7f208eb8f640 1 --2- 172.21.15.172:0/3731679152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2088101410 0x7f208810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:25.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.486+0000 7f208eb8f640 1 --2- 172.21.15.172:0/3731679152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2088100a70 0x7f2088100ed0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:25.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.486+0000 7f208eb8f640 1 --2- 172.21.15.172:0/3731679152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2088106a70 0x7f2088106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:25.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.486+0000 7f208eb8f640 1 -- 172.21.15.172:0/3731679152 >> 172.21.15.172:0/3731679152 conn(0x7f20880fc790 msgr2=0x7f20880febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:25.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.486+0000 7f208eb8f640 1 -- 172.21.15.172:0/3731679152 shutdown_connections 2024-09-10T08:58:25.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.486+0000 7f208eb8f640 1 -- 172.21.15.172:0/3731679152 wait complete. 2024-09-10T08:58:25.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.487+0000 7f208eb8f640 1 Processor -- start 2024-09-10T08:58:25.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.487+0000 7f208eb8f640 1 -- start start 2024-09-10T08:58:25.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.487+0000 7f208eb8f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2088100a70 0x7f2088198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:25.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.487+0000 7f208eb8f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2088101410 0x7f2088198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:25.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.487+0000 7f208eb8f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2088106a70 0x7f208819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:25.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.487+0000 7f208eb8f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2088110d90 con 0x7f2088100a70 2024-09-10T08:58:25.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.487+0000 7f208eb8f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2088110c10 con 0x7f2088106a70 2024-09-10T08:58:25.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.487+0000 7f208eb8f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2088110f10 con 0x7f2088101410 2024-09-10T08:58:25.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.488+0000 7f208c92b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2088100a70 0x7f2088198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:25.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.488+0000 7f208c92b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2088100a70 0x7f2088198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43080/0 (socket says 172.21.15.172:43080) 2024-09-10T08:58:25.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.488+0000 7f208c92b640 1 -- 172.21.15.172:0/3992573046 learned_addr learned my addr 172.21.15.172:0/3992573046 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:25.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.488+0000 7f208d12c640 1 --2- 172.21.15.172:0/3992573046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2088106a70 0x7f208819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:25.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.488+0000 7f2087fff640 1 --2- 172.21.15.172:0/3992573046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2088101410 0x7f2088198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:25.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.488+0000 7f208c92b640 1 -- 172.21.15.172:0/3992573046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2088101410 msgr2=0x7f2088198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:25.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.488+0000 7f208c92b640 1 --2- 172.21.15.172:0/3992573046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2088101410 0x7f2088198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:25.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.488+0000 7f208c92b640 1 -- 172.21.15.172:0/3992573046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2088106a70 msgr2=0x7f208819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:25.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.488+0000 7f208c92b640 1 --2- 172.21.15.172:0/3992573046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2088106a70 0x7f208819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:25.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.488+0000 7f208c92b640 1 -- 172.21.15.172:0/3992573046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208819d3c0 con 0x7f2088100a70 2024-09-10T08:58:25.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.488+0000 7f2087fff640 1 --2- 172.21.15.172:0/3992573046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2088101410 0x7f2088198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:25.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.488+0000 7f208c92b640 1 --2- 172.21.15.172:0/3992573046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2088100a70 0x7f2088198640 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f207c00b4a0 tx=0x7f207c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:25.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.489+0000 7f2085ffb640 1 -- 172.21.15.172:0/3992573046 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c014070 con 0x7f2088100a70 2024-09-10T08:58:25.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.489+0000 7f2085ffb640 1 -- 172.21.15.172:0/3992573046 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f207c010ac0 con 0x7f2088100a70 2024-09-10T08:58:25.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.489+0000 7f2085ffb640 1 -- 172.21.15.172:0/3992573046 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c017c30 con 0x7f2088100a70 2024-09-10T08:58:25.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.489+0000 7f208eb8f640 1 -- 172.21.15.172:0/3992573046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208819d6b0 con 0x7f2088100a70 2024-09-10T08:58:25.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.489+0000 7f208eb8f640 1 -- 172.21.15.172:0/3992573046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2088111550 con 0x7f2088100a70 2024-09-10T08:58:25.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.490+0000 7f208eb8f640 1 -- 172.21.15.172:0/3992573046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2088106e50 con 0x7f2088100a70 2024-09-10T08:58:25.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.492+0000 7f2085ffb640 1 -- 172.21.15.172:0/3992573046 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f207c017070 con 0x7f2088100a70 2024-09-10T08:58:25.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.493+0000 7f2085ffb640 1 --2- 172.21.15.172:0/3992573046 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f20780a2c00 0x7f20780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:25.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.493+0000 7f2085ffb640 1 -- 172.21.15.172:0/3992573046 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f207c0ce9e0 con 0x7f2088100a70 2024-09-10T08:58:25.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.493+0000 7f2087fff640 1 --2- 172.21.15.172:0/3992573046 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f20780a2c00 0x7f20780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:25.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.494+0000 7f2087fff640 1 --2- 172.21.15.172:0/3992573046 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f20780a2c00 0x7f20780a50c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f207402a940 tx=0x7f2074005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:25.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.498+0000 7f2085ffb640 1 -- 172.21.15.172:0/3992573046 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f207c09b5e0 con 0x7f2088100a70 2024-09-10T08:58:25.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.740+0000 7f208eb8f640 1 -- 172.21.15.172:0/3992573046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2088111870 con 0x7f2088100a70 2024-09-10T08:58:25.742 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:25 smithi172 ceph-mon[30854]: pgmap v716: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 MiB/s wr, 35 op/s 2024-09-10T08:58:25.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.741+0000 7f2085ffb640 1 -- 172.21.15.172:0/3992573046 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+12006 (secure 0 0 0) 0x7f207c09aee0 con 0x7f2088100a70 2024-09-10T08:58:25.743 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:58:25.743 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":40,"btime":"2024-09-10T08:58:02:716535+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:58:02.666404+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T08:58:25.743 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 40 2024-09-10T08:58:25.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:25 smithi174 ceph-mon[34068]: pgmap v716: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 MiB/s wr, 35 op/s 2024-09-10T08:58:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.744+0000 7f208eb8f640 1 -- 172.21.15.172:0/3992573046 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f20780a2c00 msgr2=0x7f20780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.744+0000 7f208eb8f640 1 --2- 172.21.15.172:0/3992573046 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f20780a2c00 0x7f20780a50c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f207402a940 tx=0x7f2074005fd0 comp rx=0 tx=0).stop 2024-09-10T08:58:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.744+0000 7f208eb8f640 1 -- 172.21.15.172:0/3992573046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2088100a70 msgr2=0x7f2088198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.744+0000 7f208eb8f640 1 --2- 172.21.15.172:0/3992573046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2088100a70 0x7f2088198640 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f207c00b4a0 tx=0x7f207c00b960 comp rx=0 tx=0).stop 2024-09-10T08:58:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.744+0000 7f208eb8f640 1 -- 172.21.15.172:0/3992573046 shutdown_connections 2024-09-10T08:58:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.744+0000 7f208eb8f640 1 --2- 172.21.15.172:0/3992573046 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f20780a2c00 0x7f20780a50c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.744+0000 7f208eb8f640 1 --2- 172.21.15.172:0/3992573046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2088106a70 0x7f208819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.744+0000 7f208eb8f640 1 --2- 172.21.15.172:0/3992573046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2088101410 0x7f2088198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.744+0000 7f208eb8f640 1 --2- 172.21.15.172:0/3992573046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2088100a70 0x7f2088198640 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.744+0000 7f208eb8f640 1 -- 172.21.15.172:0/3992573046 >> 172.21.15.172:0/3992573046 conn(0x7f20880fc790 msgr2=0x7f2088104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.744+0000 7f208eb8f640 1 -- 172.21.15.172:0/3992573046 shutdown_connections 2024-09-10T08:58:25.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:25.744+0000 7f208eb8f640 1 -- 172.21.15.172:0/3992573046 wait complete. 2024-09-10T08:58:25.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:25 smithi191 ceph-mon[35314]: pgmap v716: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 31 MiB/s wr, 35 op/s 2024-09-10T08:58:26.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:26 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3992573046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:58:26.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:26 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3992573046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:58:26.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:26 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3992573046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:58:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:27 smithi174 ceph-mon[34068]: pgmap v717: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-09-10T08:58:27.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:27 smithi191 ceph-mon[35314]: pgmap v717: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-09-10T08:58:27.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:27 smithi172 ceph-mon[30854]: pgmap v717: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 23 op/s 2024-09-10T08:58:28.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:28 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:58:28.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:28 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:58:28.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:28 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:58:28.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:28 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:58:28.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:28 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:58:28.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:28 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:58:28.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:28 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:58:28.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:28 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:58:28.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:28 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:58:28.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:28 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:58:28.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:58:28.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:58:28.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:58:28.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:58:28.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:28 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:58:29.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:29 smithi174 ceph-mon[34068]: pgmap v718: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-09-10T08:58:29.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:29 smithi191 ceph-mon[35314]: pgmap v718: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-09-10T08:58:29.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:29 smithi172 ceph-mon[30854]: pgmap v718: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 30 MiB/s wr, 25 op/s 2024-09-10T08:58:30.363 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 138.1 --await-for 45.0' 2024-09-10T08:58:30.364 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 138.1 --await-for 45.0 2024-09-10T08:58:30.674 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:58:30.960 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:58:30.960 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:58:30.960 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:58:30.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:58:30.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:58:30.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:58:31.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:58:31.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:58:31.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:58:31.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.399+0000 7fb260ba8640 1 -- 172.21.15.172:0/2367599508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb25c102ef0 msgr2=0x7fb25c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:31.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.399+0000 7fb260ba8640 1 --2- 172.21.15.172:0/2367599508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb25c102ef0 0x7fb25c10da40 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fb254009ca0 tx=0x7fb25402a950 comp rx=0 tx=0).stop 2024-09-10T08:58:31.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.400+0000 7fb260ba8640 1 -- 172.21.15.172:0/2367599508 shutdown_connections 2024-09-10T08:58:31.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.400+0000 7fb260ba8640 1 --2- 172.21.15.172:0/2367599508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb25c102ef0 0x7fb25c10da40 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.400+0000 7fb260ba8640 1 --2- 172.21.15.172:0/2367599508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb25c102570 0x7fb25c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.400+0000 7fb260ba8640 1 --2- 172.21.15.172:0/2367599508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb25c100e20 0x7fb25c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.400+0000 7fb260ba8640 1 -- 172.21.15.172:0/2367599508 >> 172.21.15.172:0/2367599508 conn(0x7fb25c0fc7d0 msgr2=0x7fb25c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:31.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.400+0000 7fb260ba8640 1 -- 172.21.15.172:0/2367599508 shutdown_connections 2024-09-10T08:58:31.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.400+0000 7fb260ba8640 1 -- 172.21.15.172:0/2367599508 wait complete. 2024-09-10T08:58:31.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.400+0000 7fb260ba8640 1 Processor -- start 2024-09-10T08:58:31.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb260ba8640 1 -- start start 2024-09-10T08:58:31.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb260ba8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb25c100e20 0x7fb25c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb260ba8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb25c102570 0x7fb25c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb260ba8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb25c102ef0 0x7fb25c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb260ba8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c110e70 con 0x7fb25c102570 2024-09-10T08:58:31.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb260ba8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c110cf0 con 0x7fb25c102ef0 2024-09-10T08:58:31.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb260ba8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c110ff0 con 0x7fb25c100e20 2024-09-10T08:58:31.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb25a59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb25c100e20 0x7fb25c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb259d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb25c102570 0x7fb25c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb25a59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb25c100e20 0x7fb25c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:55400/0 (socket says 172.21.15.172:55400) 2024-09-10T08:58:31.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb25a59c640 1 -- 172.21.15.172:0/1727277333 learned_addr learned my addr 172.21.15.172:0/1727277333 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:31.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb25ad9d640 1 --2- 172.21.15.172:0/1727277333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb25c102ef0 0x7fb25c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb25a59c640 1 -- 172.21.15.172:0/1727277333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb25c102ef0 msgr2=0x7fb25c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:31.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb25a59c640 1 --2- 172.21.15.172:0/1727277333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb25c102ef0 0x7fb25c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb25a59c640 1 -- 172.21.15.172:0/1727277333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb25c102570 msgr2=0x7fb25c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:31.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb25a59c640 1 --2- 172.21.15.172:0/1727277333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb25c102570 0x7fb25c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb25a59c640 1 -- 172.21.15.172:0/1727277333 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25c1a6980 con 0x7fb25c100e20 2024-09-10T08:58:31.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.401+0000 7fb259d9b640 1 --2- 172.21.15.172:0/1727277333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb25c102570 0x7fb25c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:31.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.402+0000 7fb25a59c640 1 --2- 172.21.15.172:0/1727277333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb25c100e20 0x7fb25c0fcc80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00b4a0 tx=0x7fb24c00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:31.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.402+0000 7fb24b7fe640 1 -- 172.21.15.172:0/1727277333 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c014070 con 0x7fb25c100e20 2024-09-10T08:58:31.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.402+0000 7fb260ba8640 1 -- 172.21.15.172:0/1727277333 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb25c1a6c70 con 0x7fb25c100e20 2024-09-10T08:58:31.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.402+0000 7fb24b7fe640 1 -- 172.21.15.172:0/1727277333 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb24c010ac0 con 0x7fb25c100e20 2024-09-10T08:58:31.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.402+0000 7fb260ba8640 1 -- 172.21.15.172:0/1727277333 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb25c1a71b0 con 0x7fb25c100e20 2024-09-10T08:58:31.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.402+0000 7fb24b7fe640 1 -- 172.21.15.172:0/1727277333 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c017c30 con 0x7fb25c100e20 2024-09-10T08:58:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.404+0000 7fb24b7fe640 1 -- 172.21.15.172:0/1727277333 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb24c017070 con 0x7fb25c100e20 2024-09-10T08:58:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.405+0000 7fb24b7fe640 1 --2- 172.21.15.172:0/1727277333 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb2440a2b80 0x7fb2440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.405+0000 7fb24b7fe640 1 -- 172.21.15.172:0/1727277333 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb24c0cea10 con 0x7fb25c100e20 2024-09-10T08:58:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.405+0000 7fb259d9b640 1 --2- 172.21.15.172:0/1727277333 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb2440a2b80 0x7fb2440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.405+0000 7fb262f81740 1 Processor -- start 2024-09-10T08:58:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb262f81740 1 -- start start 2024-09-10T08:58:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb259d9b640 1 --2- 172.21.15.172:0/1727277333 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb2440a2b80 0x7fb2440a5040 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0fe710 tx=0x7fb25000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb262f81740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c41fa760 0x5629c41fab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb262f81740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c41fb080 0x5629c4205400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb262f81740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c4205940 0x5629c4207d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb262f81740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5629c41602a0 con 0x5629c41fa760 2024-09-10T08:58:31.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb262f81740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5629c4107b10 con 0x5629c4205940 2024-09-10T08:58:31.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb262f81740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5629c415c260 con 0x5629c41fb080 2024-09-10T08:58:31.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb25a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c41fa760 0x5629c41fab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb25a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c41fa760 0x5629c41fab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51748/0 (socket says 172.21.15.172:51748) 2024-09-10T08:58:31.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb25a59c640 1 -- 172.21.15.172:0/581130192 learned_addr learned my addr 172.21.15.172:0/581130192 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:31.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb259d9b640 1 --2- 172.21.15.172:0/581130192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c41fb080 0x5629c4205400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb25ad9d640 1 --2- 172.21.15.172:0/581130192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c4205940 0x5629c4207d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb25a59c640 1 -- 172.21.15.172:0/581130192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c41fb080 msgr2=0x5629c4205400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:31.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb25a59c640 1 --2- 172.21.15.172:0/581130192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c41fb080 0x5629c4205400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb25a59c640 1 -- 172.21.15.172:0/581130192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c4205940 msgr2=0x5629c4207d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:31.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb25a59c640 1 --2- 172.21.15.172:0/581130192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c4205940 0x5629c4207d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb25a59c640 1 -- 172.21.15.172:0/581130192 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c3ed1b90 con 0x5629c41fa760 2024-09-10T08:58:31.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.406+0000 7fb25a59c640 1 --2- 172.21.15.172:0/581130192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c41fa760 0x5629c41fab40 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0a35f0 tx=0x7fb24c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3f1733731d749b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:31.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.407+0000 7fb248ff9640 1 -- 172.21.15.172:0/581130192 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c01e050 con 0x5629c41fa760 2024-09-10T08:58:31.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.407+0000 7fb248ff9640 1 -- 172.21.15.172:0/581130192 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb24c0a3b20 con 0x5629c41fa760 2024-09-10T08:58:31.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.407+0000 7fb248ff9640 1 -- 172.21.15.172:0/581130192 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c0a3690 con 0x5629c41fa760 2024-09-10T08:58:31.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.407+0000 7fb262f81740 1 -- 172.21.15.172:0/581130192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c41fa760 msgr2=0x5629c41fab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:31.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.407+0000 7fb262f81740 1 --2- 172.21.15.172:0/581130192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c41fa760 0x5629c41fab40 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0a35f0 tx=0x7fb24c0046c0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.408+0000 7fb262f81740 1 -- 172.21.15.172:0/581130192 shutdown_connections 2024-09-10T08:58:31.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.408+0000 7fb262f81740 1 --2- 172.21.15.172:0/581130192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c4205940 0x5629c4207d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.408+0000 7fb262f81740 1 --2- 172.21.15.172:0/581130192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c41fb080 0x5629c4205400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.408+0000 7fb262f81740 1 --2- 172.21.15.172:0/581130192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c41fa760 0x5629c41fab40 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.408+0000 7fb262f81740 1 -- 172.21.15.172:0/581130192 >> 172.21.15.172:0/581130192 conn(0x5629c41f6cf0 msgr2=0x5629c41f70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.408+0000 7fb262f81740 1 -- 172.21.15.172:0/581130192 shutdown_connections 2024-09-10T08:58:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.408+0000 7fb262f81740 1 -- 172.21.15.172:0/581130192 wait complete. 2024-09-10T08:58:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.408+0000 7fb262f81740 1 -- start start 2024-09-10T08:58:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.408+0000 7fb262f81740 1 Processor -- start 2024-09-10T08:58:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.408+0000 7fb262f81740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c41fa760 0x5629c42027f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.408+0000 7fb262f81740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c41fb080 0x5629c4202d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.408+0000 7fb262f81740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c4205940 0x5629c4203270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.409+0000 7fb262f81740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5629c4208510 con 0x5629c4205940 2024-09-10T08:58:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.409+0000 7fb262f81740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5629c40e86e0 con 0x5629c41fa760 2024-09-10T08:58:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.409+0000 7fb262f81740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5629c4208690 con 0x5629c41fb080 2024-09-10T08:58:31.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.409+0000 7fb25ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c4205940 0x5629c4203270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.409+0000 7fb262f81740 10 client.0 _ll_register_callbacks cb 0x5629c410ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:31.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.409+0000 7fb25ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c4205940 0x5629c4203270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51758/0 (socket says 172.21.15.172:51758) 2024-09-10T08:58:31.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.409+0000 7fb259d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c41fb080 0x5629c4202d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.409+0000 7fb25ad9d640 1 -- 172.21.15.172:0/1351500990 learned_addr learned my addr 172.21.15.172:0/1351500990 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:31.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.409+0000 7fb25a59c640 1 --2- 172.21.15.172:0/1351500990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c41fa760 0x5629c42027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.409+0000 7fb259d9b640 1 -- 172.21.15.172:0/1351500990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c41fa760 msgr2=0x5629c42027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:31.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.409+0000 7fb259d9b640 1 --2- 172.21.15.172:0/1351500990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c41fa760 0x5629c42027f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.409+0000 7fb259d9b640 1 -- 172.21.15.172:0/1351500990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c4205940 msgr2=0x5629c4203270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:31.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.409+0000 7fb259d9b640 1 --2- 172.21.15.172:0/1351500990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c4205940 0x5629c4203270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.409+0000 7fb259d9b640 1 -- 172.21.15.172:0/1351500990 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c42119d0 con 0x5629c41fb080 2024-09-10T08:58:31.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.409+0000 7fb25ad9d640 1 --2- 172.21.15.172:0/1351500990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c4205940 0x5629c4203270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:31.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.409+0000 7fb259d9b640 1 --2- 172.21.15.172:0/1351500990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c41fb080 0x5629c4202d30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb25000afd0 tx=0x7fb250004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f5e4f4dd3d93ef8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:31.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.409+0000 7fb241ffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:58:31.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.410+0000 7fb241ffb640 1 -- 172.21.15.172:0/1351500990 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb250016070 con 0x5629c41fb080 2024-09-10T08:58:31.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.410+0000 7fb241ffb640 1 -- 172.21.15.172:0/1351500990 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb25001cc80 con 0x5629c41fb080 2024-09-10T08:58:31.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.410+0000 7fb262f81740 1 -- 172.21.15.172:0/1351500990 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb24c0a3b20 con 0x5629c41fb080 2024-09-10T08:58:31.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.410+0000 7fb241ffb640 1 -- 172.21.15.172:0/1351500990 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb250030050 con 0x5629c41fb080 2024-09-10T08:58:31.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.410+0000 7fb262f81740 10 client.27832 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:31.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.410+0000 7fb262f81740 1 -- 172.21.15.172:0/1351500990 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5629c4214340 con 0x5629c41fb080 2024-09-10T08:58:31.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.410+0000 7fb241ffb640 1 -- 172.21.15.172:0/1351500990 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb24c0a3b20 con 0x5629c41fb080 2024-09-10T08:58:31.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.410+0000 7fb241ffb640 1 -- 172.21.15.172:0/1351500990 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fb250004af0 con 0x5629c41fb080 2024-09-10T08:58:31.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.410+0000 7fb262f81740 10 client.27832 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:31.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.410+0000 7fb262f81740 10 client.27832 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:58:31.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.410+0000 7fb262f81740 20 client.27832 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:31.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.410+0000 7fb262f81740 1 --2- 172.21.15.172:0/1351500990 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5629c42158a0 0x5629c4217ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.410+0000 7fb262f81740 4 client.27832 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:58:31.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.410+0000 7fb262f81740 1 -- 172.21.15.172:0/1351500990 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629c416d9d0 con 0x5629c42158a0 2024-09-10T08:58:31.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.411+0000 7fb25a59c640 1 --2- 172.21.15.172:0/1351500990 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5629c42158a0 0x5629c4217ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.411+0000 7fb25a59c640 1 --2- 172.21.15.172:0/1351500990 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5629c42158a0 0x5629c4217ce0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4816e8008f5b746 server_cookie=cbc1e7a49f99848b in_seq=0 out_seq=0 2024-09-10T08:58:31.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.411+0000 7fb241ffb640 10 client.27832 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:31.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.412+0000 7fb241ffb640 1 -- 172.21.15.172:0/1351500990 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629c416d9d0 con 0x5629c42158a0 2024-09-10T08:58:31.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.412+0000 7fb241ffb640 10 client.27832 handle_command_reply: tid=0 2024-09-10T08:58:31.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.412+0000 7fb262f81740 1 client.27832 shutdown 2024-09-10T08:58:31.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.412+0000 7fb262f81740 1 -- 172.21.15.172:0/1351500990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c41fb080 msgr2=0x5629c4202d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:31.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.412+0000 7fb262f81740 1 --2- 172.21.15.172:0/1351500990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c41fb080 0x5629c4202d30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb25000afd0 tx=0x7fb250004730 comp rx=0 tx=0).stop 2024-09-10T08:58:31.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.412+0000 7fb262f81740 1 -- 172.21.15.172:0/1351500990 shutdown_connections 2024-09-10T08:58:31.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.412+0000 7fb262f81740 1 --2- 172.21.15.172:0/1351500990 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5629c42158a0 0x5629c4217ce0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.412+0000 7fb241ffb640 0 client.27832 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:31.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.412+0000 7fb262f81740 1 --2- 172.21.15.172:0/1351500990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c4205940 0x5629c4203270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.412+0000 7fb262f81740 1 --2- 172.21.15.172:0/1351500990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c41fb080 0x5629c4202d30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.412+0000 7fb262f81740 1 --2- 172.21.15.172:0/1351500990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c41fa760 0x5629c42027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.412+0000 7fb262f81740 1 -- 172.21.15.172:0/1351500990 >> 172.21.15.172:0/1351500990 conn(0x5629c41f6cf0 msgr2=0x5629c41f5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:31.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.412+0000 7fb262f81740 1 -- 172.21.15.172:0/1351500990 shutdown_connections 2024-09-10T08:58:31.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.412+0000 7fb262f81740 1 -- 172.21.15.172:0/1351500990 wait complete. 2024-09-10T08:58:31.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.413+0000 7fb262f81740 20 client.27832 trim_cache size 0 max 16384 2024-09-10T08:58:31.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.432+0000 7fb262f81740 1 Processor -- start 2024-09-10T08:58:31.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.432+0000 7fb262f81740 1 -- start start 2024-09-10T08:58:31.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.432+0000 7fb262f81740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c41fa760 0x5629c41fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.432+0000 7fb262f81740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c41fb080 0x5629c4204760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.432+0000 7fb262f81740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c4205940 0x5629c421d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.432+0000 7fb262f81740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c1b0e40 con 0x5629c41fb080 2024-09-10T08:58:31.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.432+0000 7fb262f81740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c1b2a10 con 0x5629c41fa760 2024-09-10T08:58:31.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.432+0000 7fb262f81740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c1b1510 con 0x5629c4205940 2024-09-10T08:58:31.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.433+0000 7fb259d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c41fb080 0x5629c4204760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.433+0000 7fb259d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c41fb080 0x5629c4204760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51766/0 (socket says 172.21.15.172:51766) 2024-09-10T08:58:31.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.433+0000 7fb259d9b640 1 -- 172.21.15.172:0/156210277 learned_addr learned my addr 172.21.15.172:0/156210277 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:31.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.433+0000 7fb25ad9d640 1 --2- 172.21.15.172:0/156210277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c4205940 0x5629c421d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.433+0000 7fb25a59c640 1 --2- 172.21.15.172:0/156210277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c41fa760 0x5629c41fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.433+0000 7fb259d9b640 1 -- 172.21.15.172:0/156210277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c4205940 msgr2=0x5629c421d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:31.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.433+0000 7fb259d9b640 1 --2- 172.21.15.172:0/156210277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c4205940 0x5629c421d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.433+0000 7fb259d9b640 1 -- 172.21.15.172:0/156210277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c41fa760 msgr2=0x5629c41fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:31.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.433+0000 7fb259d9b640 1 --2- 172.21.15.172:0/156210277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c41fa760 0x5629c41fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.433+0000 7fb259d9b640 1 -- 172.21.15.172:0/156210277 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c41f6b20 con 0x5629c41fb080 2024-09-10T08:58:31.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.433+0000 7fb25a59c640 1 --2- 172.21.15.172:0/156210277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c41fa760 0x5629c41fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:31.436 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.433+0000 7fb259d9b640 1 --2- 172.21.15.172:0/156210277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c41fb080 0x5629c4204760 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fb250004f10 tx=0x7fb25000fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fefcd0c5171c39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.433+0000 7fb243fff640 1 -- 172.21.15.172:0/156210277 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb250038020 con 0x5629c41fb080 2024-09-10T08:58:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.433+0000 7fb243fff640 1 -- 172.21.15.172:0/156210277 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb250004c50 con 0x5629c41fb080 2024-09-10T08:58:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.433+0000 7fb243fff640 1 -- 172.21.15.172:0/156210277 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2500258f0 con 0x5629c41fb080 2024-09-10T08:58:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.434+0000 7fb262f81740 1 -- 172.21.15.172:0/156210277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c41fb080 msgr2=0x5629c4204760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.434+0000 7fb262f81740 1 --2- 172.21.15.172:0/156210277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c41fb080 0x5629c4204760 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fb250004f10 tx=0x7fb25000fc70 comp rx=0 tx=0).stop 2024-09-10T08:58:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.434+0000 7fb262f81740 1 -- 172.21.15.172:0/156210277 shutdown_connections 2024-09-10T08:58:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.434+0000 7fb262f81740 1 --2- 172.21.15.172:0/156210277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c4205940 0x5629c421d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.434+0000 7fb262f81740 1 --2- 172.21.15.172:0/156210277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c41fb080 0x5629c4204760 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.434+0000 7fb262f81740 1 --2- 172.21.15.172:0/156210277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c41fa760 0x5629c41fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.434+0000 7fb262f81740 1 -- 172.21.15.172:0/156210277 >> 172.21.15.172:0/156210277 conn(0x5629c41f6cf0 msgr2=0x5629c4204050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:31.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.434+0000 7fb262f81740 1 -- 172.21.15.172:0/156210277 shutdown_connections 2024-09-10T08:58:31.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.434+0000 7fb262f81740 1 -- 172.21.15.172:0/156210277 wait complete. 2024-09-10T08:58:31.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.435+0000 7fb262f81740 1 -- start start 2024-09-10T08:58:31.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.435+0000 7fb262f81740 1 Processor -- start 2024-09-10T08:58:31.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.435+0000 7fb262f81740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c41fa760 0x5629c420de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.435+0000 7fb262f81740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c41fb080 0x5629c42091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.435+0000 7fb262f81740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c4205940 0x5629c42096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.435+0000 7fb262f81740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c1b2340 con 0x5629c4205940 2024-09-10T08:58:31.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.435+0000 7fb262f81740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c1b1c70 con 0x5629c41fb080 2024-09-10T08:58:31.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.435+0000 7fb262f81740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c1aee20 con 0x5629c41fa760 2024-09-10T08:58:31.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.435+0000 7fb262f81740 10 client.0 _ll_register_callbacks cb 0x7fb2500108d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:31.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.435+0000 7fb25ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c4205940 0x5629c42096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.435+0000 7fb25ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c4205940 0x5629c42096f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51772/0 (socket says 172.21.15.172:51772) 2024-09-10T08:58:31.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.435+0000 7fb25ad9d640 1 -- 172.21.15.172:0/368400284 learned_addr learned my addr 172.21.15.172:0/368400284 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:31.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.435+0000 7fb25a59c640 1 --2- 172.21.15.172:0/368400284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c41fa760 0x5629c420de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.435+0000 7fb259d9b640 1 --2- 172.21.15.172:0/368400284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c41fb080 0x5629c42091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.436+0000 7fb25a59c640 1 -- 172.21.15.172:0/368400284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c41fb080 msgr2=0x5629c42091b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:31.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.436+0000 7fb25a59c640 1 --2- 172.21.15.172:0/368400284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c41fb080 0x5629c42091b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.436+0000 7fb25a59c640 1 -- 172.21.15.172:0/368400284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c4205940 msgr2=0x5629c42096f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:31.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.436+0000 7fb25a59c640 1 --2- 172.21.15.172:0/368400284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c4205940 0x5629c42096f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.436+0000 7fb25a59c640 1 -- 172.21.15.172:0/368400284 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c41f2260 con 0x5629c41fa760 2024-09-10T08:58:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.436+0000 7fb25ad9d640 1 --2- 172.21.15.172:0/368400284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c4205940 0x5629c42096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.436+0000 7fb25a59c640 1 --2- 172.21.15.172:0/368400284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c41fa760 0x5629c420de80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00be60 tx=0x7fb24c09f3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4fae26552658e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.436+0000 7fb241ffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:58:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.436+0000 7fb241ffb640 1 -- 172.21.15.172:0/368400284 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c07e060 con 0x5629c41fa760 2024-09-10T08:58:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.436+0000 7fb262f81740 1 -- 172.21.15.172:0/368400284 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb250004c50 con 0x5629c41fa760 2024-09-10T08:58:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.436+0000 7fb241ffb640 1 -- 172.21.15.172:0/368400284 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb24c099260 con 0x5629c41fa760 2024-09-10T08:58:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.436+0000 7fb241ffb640 1 -- 172.21.15.172:0/368400284 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb24c0b9070 con 0x5629c41fa760 2024-09-10T08:58:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.436+0000 7fb262f81740 10 client.27838 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.436+0000 7fb262f81740 1 -- 172.21.15.172:0/368400284 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5629c4280240 con 0x5629c41fa760 2024-09-10T08:58:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.437+0000 7fb241ffb640 1 -- 172.21.15.172:0/368400284 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb250004c50 con 0x5629c41fa760 2024-09-10T08:58:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.437+0000 7fb241ffb640 1 -- 172.21.15.172:0/368400284 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fb24c0a3690 con 0x5629c41fa760 2024-09-10T08:58:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.437+0000 7fb262f81740 10 client.27838 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:31.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.437+0000 7fb262f81740 10 client.27838 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:58:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.437+0000 7fb262f81740 20 client.27838 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.437+0000 7fb262f81740 1 --2- 172.21.15.172:0/368400284 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5629c42158a0 0x5629c4282590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.437+0000 7fb262f81740 4 client.27838 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.1, "await_for": 45.0}] 2024-09-10T08:58:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.437+0000 7fb262f81740 1 -- 172.21.15.172:0/368400284 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.1, "await_for": 45.0}) -- 0x5629c4167d20 con 0x5629c42158a0 2024-09-10T08:58:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.437+0000 7fb259d9b640 1 --2- 172.21.15.172:0/368400284 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5629c42158a0 0x5629c4282590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.438+0000 7fb259d9b640 1 --2- 172.21.15.172:0/368400284 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5629c42158a0 0x5629c4282590 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bc672ed942d60f1 server_cookie=832390c89b60c584 in_seq=0 out_seq=0 2024-09-10T08:58:31.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.438+0000 7fb241ffb640 10 client.27838 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:31.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.528+0000 7fb241ffb640 1 -- 172.21.15.172:0/368400284 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5629c4167d20 con 0x5629c42158a0 2024-09-10T08:58:31.530 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.528+0000 7fb241ffb640 10 client.27838 handle_command_reply: tid=0 2024-09-10T08:58:31.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.528+0000 7fb262f81740 1 client.27838 shutdown 2024-09-10T08:58:31.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.528+0000 7fb262f81740 1 -- 172.21.15.172:0/368400284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c41fa760 msgr2=0x5629c420de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:31.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.528+0000 7fb262f81740 1 --2- 172.21.15.172:0/368400284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c41fa760 0x5629c420de80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00be60 tx=0x7fb24c09f3a0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.528+0000 7fb262f81740 1 -- 172.21.15.172:0/368400284 shutdown_connections 2024-09-10T08:58:31.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.528+0000 7fb262f81740 1 --2- 172.21.15.172:0/368400284 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5629c42158a0 0x5629c4282590 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.528+0000 7fb262f81740 1 --2- 172.21.15.172:0/368400284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629c4205940 0x5629c42096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.528+0000 7fb241ffb640 0 client.27838 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:31.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.528+0000 7fb262f81740 1 --2- 172.21.15.172:0/368400284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629c41fb080 0x5629c42091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.528+0000 7fb262f81740 1 --2- 172.21.15.172:0/368400284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629c41fa760 0x5629c420de80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.528+0000 7fb262f81740 1 -- 172.21.15.172:0/368400284 >> 172.21.15.172:0/368400284 conn(0x5629c41f6cf0 msgr2=0x5629c41f5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:31.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.528+0000 7fb262f81740 1 -- 172.21.15.172:0/368400284 shutdown_connections 2024-09-10T08:58:31.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.528+0000 7fb262f81740 1 -- 172.21.15.172:0/368400284 wait complete. 2024-09-10T08:58:31.531 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.528+0000 7fb262f81740 20 client.27838 trim_cache size 0 max 16384 2024-09-10T08:58:31.532 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:58:31.532 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 40, 2024-09-10T08:58:31.532 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:58:31.532 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 42, 2024-09-10T08:58:31.532 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:58:31.532 INFO:teuthology.orchestra.run.smithi172.stdout: "4766dcba": { 2024-09-10T08:58:31.532 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 42, 2024-09-10T08:58:31.532 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:58:31.533 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:58:31.533 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:58:31.533 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:58:31.533 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:58:31.533 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:58:31.533 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 138.1, 2024-09-10T08:58:31.533 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:58:31.533 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:58:31.533 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:58:31.533 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:58:31.533 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T08:58:31.533 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:58:31.534 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:58:31.534 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:58:31.534 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:58:31.534 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:58:31.534 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:58:31.534 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:58:31.534 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.529+0000 7fb260ba8640 1 -- 172.21.15.172:0/1727277333 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb2440a2b80 msgr2=0x7fb2440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:31.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.529+0000 7fb260ba8640 1 --2- 172.21.15.172:0/1727277333 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb2440a2b80 0x7fb2440a5040 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0fe710 tx=0x7fb25000a250 comp rx=0 tx=0).stop 2024-09-10T08:58:31.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.529+0000 7fb260ba8640 1 -- 172.21.15.172:0/1727277333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb25c100e20 msgr2=0x7fb25c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:31.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.529+0000 7fb260ba8640 1 --2- 172.21.15.172:0/1727277333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb25c100e20 0x7fb25c0fcc80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00b4a0 tx=0x7fb24c00b960 comp rx=0 tx=0).stop 2024-09-10T08:58:31.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.529+0000 7fb260ba8640 1 -- 172.21.15.172:0/1727277333 shutdown_connections 2024-09-10T08:58:31.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.529+0000 7fb260ba8640 1 --2- 172.21.15.172:0/1727277333 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb2440a2b80 0x7fb2440a5040 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.529+0000 7fb260ba8640 1 --2- 172.21.15.172:0/1727277333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb25c102ef0 0x7fb25c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.529+0000 7fb260ba8640 1 --2- 172.21.15.172:0/1727277333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb25c102570 0x7fb25c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.529+0000 7fb260ba8640 1 --2- 172.21.15.172:0/1727277333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb25c100e20 0x7fb25c0fcc80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:31.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.529+0000 7fb260ba8640 1 -- 172.21.15.172:0/1727277333 >> 172.21.15.172:0/1727277333 conn(0x7fb25c0fc7d0 msgr2=0x7fb25c06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:31.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.529+0000 7fb260ba8640 1 -- 172.21.15.172:0/1727277333 shutdown_connections 2024-09-10T08:58:31.535 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:31.529+0000 7fb260ba8640 1 -- 172.21.15.172:0/1727277333 wait complete. 2024-09-10T08:58:31.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:31 smithi172 ceph-mon[30854]: pgmap v719: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-09-10T08:58:31.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:31 smithi174 ceph-mon[34068]: pgmap v719: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-09-10T08:58:32.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:31 smithi191 ceph-mon[35314]: pgmap v719: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 28 MiB/s wr, 24 op/s 2024-09-10T08:58:32.388 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4766dcba', quiesce took 7.7 seconds. Will release after: 10.400000000000002 2024-09-10T08:58:33.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:33 smithi174 ceph-mon[34068]: pgmap v720: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:58:33.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:33 smithi191 ceph-mon[35314]: pgmap v720: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:58:33.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:33 smithi172 ceph-mon[30854]: pgmap v720: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-09-10T08:58:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.570+0000 7fb8b0949640 1 -- 172.21.15.172:0/2660469265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8ac104ef0 msgr2=0x7fb8ac1072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.570+0000 7fb8b0949640 1 --2- 172.21.15.172:0/2660469265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8ac104ef0 0x7fb8ac1072e0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fb894009900 tx=0x7fb89402a410 comp rx=0 tx=0).stop 2024-09-10T08:58:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.570+0000 7fb8b0949640 1 -- 172.21.15.172:0/2660469265 shutdown_connections 2024-09-10T08:58:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.570+0000 7fb8b0949640 1 --2- 172.21.15.172:0/2660469265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ac107820 0x7fb8ac109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.570+0000 7fb8b0949640 1 --2- 172.21.15.172:0/2660469265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8ac104ef0 0x7fb8ac1072e0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.570+0000 7fb8b0949640 1 --2- 172.21.15.172:0/2660469265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8ac1025c0 0x7fb8ac1049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.570+0000 7fb8b0949640 1 -- 172.21.15.172:0/2660469265 >> 172.21.15.172:0/2660469265 conn(0x7fb8ac0fc1e0 msgr2=0x7fb8ac0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.570+0000 7fb8b0949640 1 -- 172.21.15.172:0/2660469265 shutdown_connections 2024-09-10T08:58:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.570+0000 7fb8b0949640 1 -- 172.21.15.172:0/2660469265 wait complete. 2024-09-10T08:58:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.571+0000 7fb8b0949640 1 Processor -- start 2024-09-10T08:58:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.571+0000 7fb8b0949640 1 -- start start 2024-09-10T08:58:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.571+0000 7fb8b0949640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8ac1025c0 0x7fb8ac197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.571+0000 7fb8b0949640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8ac104ef0 0x7fb8ac198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.571+0000 7fb8b0949640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ac107820 0x7fb8ac19f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.576 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.571+0000 7fb8b0949640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac10d730 con 0x7fb8ac104ef0 2024-09-10T08:58:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.571+0000 7fb8b0949640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac10d5b0 con 0x7fb8ac1025c0 2024-09-10T08:58:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.571+0000 7fb8b0949640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac10d8b0 con 0x7fb8ac107820 2024-09-10T08:58:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.571+0000 7fb8a1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8ac104ef0 0x7fb8ac198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.571+0000 7fb8a1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8ac104ef0 0x7fb8ac198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51794/0 (socket says 172.21.15.172:51794) 2024-09-10T08:58:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.571+0000 7fb8a1d9b640 1 -- 172.21.15.172:0/3514735604 learned_addr learned my addr 172.21.15.172:0/3514735604 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.571+0000 7fb8aa59c640 1 --2- 172.21.15.172:0/3514735604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8ac1025c0 0x7fb8ac197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.571+0000 7fb8aad9d640 1 --2- 172.21.15.172:0/3514735604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ac107820 0x7fb8ac19f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.572+0000 7fb8a1d9b640 1 -- 172.21.15.172:0/3514735604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ac107820 msgr2=0x7fb8ac19f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.572+0000 7fb8a1d9b640 1 --2- 172.21.15.172:0/3514735604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ac107820 0x7fb8ac19f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.572+0000 7fb8a1d9b640 1 -- 172.21.15.172:0/3514735604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8ac1025c0 msgr2=0x7fb8ac197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.572+0000 7fb8a1d9b640 1 --2- 172.21.15.172:0/3514735604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8ac1025c0 0x7fb8ac197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.572+0000 7fb8a1d9b640 1 -- 172.21.15.172:0/3514735604 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ac19fa10 con 0x7fb8ac104ef0 2024-09-10T08:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.572+0000 7fb8aa59c640 1 --2- 172.21.15.172:0/3514735604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8ac1025c0 0x7fb8ac197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.572+0000 7fb8a1d9b640 1 --2- 172.21.15.172:0/3514735604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8ac104ef0 0x7fb8ac198330 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fb8940314e0 tx=0x7fb8940042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.572+0000 7fb8a3fff640 1 -- 172.21.15.172:0/3514735604 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89403f030 con 0x7fb8ac104ef0 2024-09-10T08:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.572+0000 7fb8a3fff640 1 -- 172.21.15.172:0/3514735604 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb894003cb0 con 0x7fb8ac104ef0 2024-09-10T08:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.572+0000 7fb8b0949640 1 -- 172.21.15.172:0/3514735604 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8ac19fca0 con 0x7fb8ac104ef0 2024-09-10T08:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.572+0000 7fb8a3fff640 1 -- 172.21.15.172:0/3514735604 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894035070 con 0x7fb8ac104ef0 2024-09-10T08:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.572+0000 7fb8b0949640 1 -- 172.21.15.172:0/3514735604 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8ac1a0190 con 0x7fb8ac104ef0 2024-09-10T08:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.574+0000 7fb8a3fff640 1 -- 172.21.15.172:0/3514735604 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb894042030 con 0x7fb8ac104ef0 2024-09-10T08:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.574+0000 7fb8b37bd740 1 Processor -- start 2024-09-10T08:58:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.575+0000 7fb8b37bd740 1 -- start start 2024-09-10T08:58:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.575+0000 7fb8b37bd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff42419a0 0x561ff4241da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.575+0000 7fb8b37bd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42422e0 0x561ff4246da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.575+0000 7fb8b37bd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff4247370 0x561ff4249830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.575+0000 7fb8b37bd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561ff41a52f0 con 0x561ff42422e0 2024-09-10T08:58:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.575+0000 7fb8b37bd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561ff414d270 con 0x561ff42419a0 2024-09-10T08:58:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.575+0000 7fb8b37bd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561ff41a12b0 con 0x561ff4247370 2024-09-10T08:58:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.575+0000 7fb8aa59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42422e0 0x561ff4246da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.575+0000 7fb8aa59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42422e0 0x561ff4246da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51798/0 (socket says 172.21.15.172:51798) 2024-09-10T08:58:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.575+0000 7fb8aa59c640 1 -- 172.21.15.172:0/1338691826 learned_addr learned my addr 172.21.15.172:0/1338691826 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.575+0000 7fb8a3fff640 1 --2- 172.21.15.172:0/3514735604 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8980a2c30 0x7fb8980a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.575+0000 7fb8aad9d640 1 --2- 172.21.15.172:0/1338691826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff42419a0 0x561ff4241da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.575+0000 7fb8a1d9b640 1 --2- 172.21.15.172:0/1338691826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff4247370 0x561ff4249830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.575+0000 7fb8aad9d640 1 --2- 172.21.15.172:0/3514735604 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8980a2c30 0x7fb8980a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.575+0000 7fb8aa59c640 1 -- 172.21.15.172:0/1338691826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff4247370 msgr2=0x561ff4249830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.575+0000 7fb8aa59c640 1 --2- 172.21.15.172:0/1338691826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff4247370 0x561ff4249830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.576+0000 7fb8aa59c640 1 -- 172.21.15.172:0/1338691826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff42419a0 msgr2=0x561ff4241da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.576+0000 7fb8aa59c640 1 --2- 172.21.15.172:0/1338691826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff42419a0 0x561ff4241da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.576+0000 7fb8a3fff640 1 -- 172.21.15.172:0/3514735604 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb8940efa80 con 0x7fb8ac104ef0 2024-09-10T08:58:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.576+0000 7fb8aa59c640 1 -- 172.21.15.172:0/1338691826 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ff414c340 con 0x561ff42422e0 2024-09-10T08:58:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.576+0000 7fb8a1d9b640 1 --2- 172.21.15.172:0/1338691826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff4247370 0x561ff4249830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.576+0000 7fb8aa59c640 1 --2- 172.21.15.172:0/1338691826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42422e0 0x561ff4246da0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fb89c007850 tx=0x7fb89c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a80f9a82909c10c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.576+0000 7fb8aad9d640 1 --2- 172.21.15.172:0/3514735604 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8980a2c30 0x7fb8980a50f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009270 tx=0x7fb8a4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.576+0000 7fb8a0d99640 1 -- 172.21.15.172:0/1338691826 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c00d950 con 0x561ff42422e0 2024-09-10T08:58:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.576+0000 7fb8a0d99640 1 -- 172.21.15.172:0/1338691826 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb89c018070 con 0x561ff42422e0 2024-09-10T08:58:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.576+0000 7fb8a0d99640 1 -- 172.21.15.172:0/1338691826 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c013de0 con 0x561ff42422e0 2024-09-10T08:58:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.577+0000 7fb8b37bd740 1 -- 172.21.15.172:0/1338691826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42422e0 msgr2=0x561ff4246da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.577+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/1338691826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42422e0 0x561ff4246da0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fb89c007850 tx=0x7fb89c007d10 comp rx=0 tx=0).stop 2024-09-10T08:58:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.577+0000 7fb8b37bd740 1 -- 172.21.15.172:0/1338691826 shutdown_connections 2024-09-10T08:58:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.577+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/1338691826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff4247370 0x561ff4249830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.577+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/1338691826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42422e0 0x561ff4246da0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.577+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/1338691826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff42419a0 0x561ff4241da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.577+0000 7fb8b37bd740 1 -- 172.21.15.172:0/1338691826 >> 172.21.15.172:0/1338691826 conn(0x561ff423d010 msgr2=0x561ff423d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.577+0000 7fb8b37bd740 1 -- 172.21.15.172:0/1338691826 shutdown_connections 2024-09-10T08:58:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.577+0000 7fb8b37bd740 1 -- 172.21.15.172:0/1338691826 wait complete. 2024-09-10T08:58:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.577+0000 7fb8b37bd740 1 -- start start 2024-09-10T08:58:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.577+0000 7fb8b37bd740 1 Processor -- start 2024-09-10T08:58:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.577+0000 7fb8b37bd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42419a0 0x561ff424bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.577+0000 7fb8b37bd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff42422e0 0x561ff424c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.577+0000 7fb8b37bd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff4247370 0x561ff42530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.577+0000 7fb8b37bd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561ff424a340 con 0x561ff42419a0 2024-09-10T08:58:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb8b37bd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561ff412e290 con 0x561ff42422e0 2024-09-10T08:58:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb8b37bd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561ff424a4c0 con 0x561ff4247370 2024-09-10T08:58:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb8b37bd740 10 client.0 _ll_register_callbacks cb 0x561ff415f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb8aa59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42419a0 0x561ff424bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb8aa59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42419a0 0x561ff424bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51814/0 (socket says 172.21.15.172:51814) 2024-09-10T08:58:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb8aa59c640 1 -- 172.21.15.172:0/4184445206 learned_addr learned my addr 172.21.15.172:0/4184445206 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb8aad9d640 1 --2- 172.21.15.172:0/4184445206 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff4247370 0x561ff42530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb8a1d9b640 1 --2- 172.21.15.172:0/4184445206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff42422e0 0x561ff424c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb8aa59c640 1 -- 172.21.15.172:0/4184445206 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff4247370 msgr2=0x561ff42530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb8aa59c640 1 --2- 172.21.15.172:0/4184445206 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff4247370 0x561ff42530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb8aa59c640 1 -- 172.21.15.172:0/4184445206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff42422e0 msgr2=0x561ff424c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb8aa59c640 1 --2- 172.21.15.172:0/4184445206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff42422e0 0x561ff424c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb8aa59c640 1 -- 172.21.15.172:0/4184445206 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ff3eb8210 con 0x561ff42419a0 2024-09-10T08:58:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb8aa59c640 1 --2- 172.21.15.172:0/4184445206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42419a0 0x561ff424bbc0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fb89c004700 tx=0x7fb89c013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52097f8e932cde6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb891ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb891ffb640 1 -- 172.21.15.172:0/4184445206 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c00db50 con 0x561ff42419a0 2024-09-10T08:58:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb891ffb640 1 -- 172.21.15.172:0/4184445206 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb89c00dcf0 con 0x561ff42419a0 2024-09-10T08:58:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb8b37bd740 1 -- 172.21.15.172:0/4184445206 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb89c018070 con 0x561ff42419a0 2024-09-10T08:58:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.578+0000 7fb891ffb640 1 -- 172.21.15.172:0/4184445206 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb89c030050 con 0x561ff42419a0 2024-09-10T08:58:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.579+0000 7fb8b37bd740 10 client.19089 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.579+0000 7fb8b37bd740 1 -- 172.21.15.172:0/4184445206 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561ff3f84620 con 0x561ff42419a0 2024-09-10T08:58:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.579+0000 7fb891ffb640 1 -- 172.21.15.172:0/4184445206 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb89c018070 con 0x561ff42419a0 2024-09-10T08:58:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.579+0000 7fb891ffb640 1 -- 172.21.15.172:0/4184445206 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fb89c01d400 con 0x561ff42419a0 2024-09-10T08:58:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.579+0000 7fb8b37bd740 10 client.19089 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.579+0000 7fb8b37bd740 10 client.19089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:58:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.579+0000 7fb8b37bd740 20 client.19089 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.579+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/4184445206 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561ff42574b0 0x561ff42598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.579+0000 7fb8b37bd740 4 client.19089 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:58:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.579+0000 7fb8b37bd740 1 -- 172.21.15.172:0/4184445206 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ff41b2a20 con 0x561ff42574b0 2024-09-10T08:58:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.579+0000 7fb8a1d9b640 1 --2- 172.21.15.172:0/4184445206 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561ff42574b0 0x561ff42598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.580+0000 7fb8a1d9b640 1 --2- 172.21.15.172:0/4184445206 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561ff42574b0 0x561ff42598f0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a10da8e88b76b43 server_cookie=b359a76f389b39e4 in_seq=0 out_seq=0 2024-09-10T08:58:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.580+0000 7fb891ffb640 10 client.19089 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.580+0000 7fb891ffb640 1 -- 172.21.15.172:0/4184445206 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ff41b2a20 con 0x561ff42574b0 2024-09-10T08:58:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.580+0000 7fb891ffb640 10 client.19089 handle_command_reply: tid=0 2024-09-10T08:58:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.580+0000 7fb8b37bd740 1 client.19089 shutdown 2024-09-10T08:58:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.581+0000 7fb8b37bd740 1 -- 172.21.15.172:0/4184445206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42419a0 msgr2=0x561ff424bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.581+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/4184445206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42419a0 0x561ff424bbc0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fb89c004700 tx=0x7fb89c013f50 comp rx=0 tx=0).stop 2024-09-10T08:58:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.581+0000 7fb8b37bd740 1 -- 172.21.15.172:0/4184445206 shutdown_connections 2024-09-10T08:58:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.581+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/4184445206 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561ff42574b0 0x561ff42598f0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.581+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/4184445206 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff4247370 0x561ff42530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.581+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/4184445206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff42422e0 0x561ff424c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.581+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/4184445206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42419a0 0x561ff424bbc0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.581+0000 7fb891ffb640 0 client.19089 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.581+0000 7fb8b37bd740 1 -- 172.21.15.172:0/4184445206 >> 172.21.15.172:0/4184445206 conn(0x561ff423d010 msgr2=0x561ff423e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.581+0000 7fb8b37bd740 1 -- 172.21.15.172:0/4184445206 shutdown_connections 2024-09-10T08:58:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.581+0000 7fb8b37bd740 1 -- 172.21.15.172:0/4184445206 wait complete. 2024-09-10T08:58:35.593 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.581+0000 7fb8b37bd740 20 client.19089 trim_cache size 0 max 16384 2024-09-10T08:58:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8b37bd740 1 Processor -- start 2024-09-10T08:58:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8b37bd740 1 -- start start 2024-09-10T08:58:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8b37bd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42419a0 0x561ff424bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8b37bd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff42422e0 0x561ff4253090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8b37bd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff4247370 0x561ff423e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8b37bd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac1aa5d0 con 0x561ff42419a0 2024-09-10T08:58:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8b37bd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac1ac1a0 con 0x561ff4247370 2024-09-10T08:58:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8b37bd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac1aaca0 con 0x561ff42422e0 2024-09-10T08:58:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8aa59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42419a0 0x561ff424bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8aa59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42419a0 0x561ff424bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51826/0 (socket says 172.21.15.172:51826) 2024-09-10T08:58:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8aa59c640 1 -- 172.21.15.172:0/3812714720 learned_addr learned my addr 172.21.15.172:0/3812714720 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8a1d9b640 1 --2- 172.21.15.172:0/3812714720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff42422e0 0x561ff4253090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8aad9d640 1 --2- 172.21.15.172:0/3812714720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff4247370 0x561ff423e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8aa59c640 1 -- 172.21.15.172:0/3812714720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff42422e0 msgr2=0x561ff4253090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8aa59c640 1 --2- 172.21.15.172:0/3812714720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff42422e0 0x561ff4253090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8aa59c640 1 -- 172.21.15.172:0/3812714720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff4247370 msgr2=0x561ff423e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8aa59c640 1 --2- 172.21.15.172:0/3812714720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff4247370 0x561ff423e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.595+0000 7fb8aa59c640 1 -- 172.21.15.172:0/3812714720 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ff424a620 con 0x561ff42419a0 2024-09-10T08:58:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.596+0000 7fb8aa59c640 1 --2- 172.21.15.172:0/3812714720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42419a0 0x561ff424bfe0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0077a0 tx=0x7fb89c004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f6bb415128105f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.596+0000 7fb893fff640 1 -- 172.21.15.172:0/3812714720 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c01da40 con 0x561ff42419a0 2024-09-10T08:58:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.596+0000 7fb893fff640 1 -- 172.21.15.172:0/3812714720 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb89c018040 con 0x561ff42419a0 2024-09-10T08:58:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.596+0000 7fb893fff640 1 -- 172.21.15.172:0/3812714720 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c027410 con 0x561ff42419a0 2024-09-10T08:58:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.596+0000 7fb8b37bd740 1 -- 172.21.15.172:0/3812714720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42419a0 msgr2=0x561ff424bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.596+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/3812714720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42419a0 0x561ff424bfe0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0077a0 tx=0x7fb89c004640 comp rx=0 tx=0).stop 2024-09-10T08:58:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.596+0000 7fb8b37bd740 1 -- 172.21.15.172:0/3812714720 shutdown_connections 2024-09-10T08:58:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.596+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/3812714720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff4247370 0x561ff423e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.596+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/3812714720 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff42422e0 0x561ff4253090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.596+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/3812714720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42419a0 0x561ff424bfe0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.596+0000 7fb8b37bd740 1 -- 172.21.15.172:0/3812714720 >> 172.21.15.172:0/3812714720 conn(0x561ff423d010 msgr2=0x561ff423c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.596+0000 7fb8b37bd740 1 -- 172.21.15.172:0/3812714720 shutdown_connections 2024-09-10T08:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.596+0000 7fb8b37bd740 1 -- 172.21.15.172:0/3812714720 wait complete. 2024-09-10T08:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8b37bd740 1 -- start start 2024-09-10T08:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8b37bd740 1 Processor -- start 2024-09-10T08:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8b37bd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff42419a0 0x561ff4250a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8b37bd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42422e0 0x561ff4250fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8b37bd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff4247370 0x561ff4251f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8b37bd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac1abad0 con 0x561ff42422e0 2024-09-10T08:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8b37bd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac1ab400 con 0x561ff42419a0 2024-09-10T08:58:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8b37bd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac1a85b0 con 0x561ff4247370 2024-09-10T08:58:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8b37bd740 10 client.0 _ll_register_callbacks cb 0x7fb89c0076d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8a1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42422e0 0x561ff4250fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8a1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42422e0 0x561ff4250fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:51836/0 (socket says 172.21.15.172:51836) 2024-09-10T08:58:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8a1d9b640 1 -- 172.21.15.172:0/282796436 learned_addr learned my addr 172.21.15.172:0/282796436 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8aa59c640 1 --2- 172.21.15.172:0/282796436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff42419a0 0x561ff4250a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8a1d9b640 1 -- 172.21.15.172:0/282796436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff4247370 msgr2=0x561ff4251f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8aad9d640 1 --2- 172.21.15.172:0/282796436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff4247370 0x561ff4251f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8a1d9b640 1 --2- 172.21.15.172:0/282796436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff4247370 0x561ff4251f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8a1d9b640 1 -- 172.21.15.172:0/282796436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff42419a0 msgr2=0x561ff4250a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8a1d9b640 1 --2- 172.21.15.172:0/282796436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff42419a0 0x561ff4250a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8a1d9b640 1 -- 172.21.15.172:0/282796436 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ff4252560 con 0x561ff42422e0 2024-09-10T08:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.597+0000 7fb8aad9d640 1 --2- 172.21.15.172:0/282796436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff4247370 0x561ff4251f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.598+0000 7fb8a1d9b640 1 --2- 172.21.15.172:0/282796436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42422e0 0x561ff4250fb0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fb894005ec0 tx=0x7fb894040fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e47514b1f450a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.598+0000 7fb891ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.598+0000 7fb891ffb640 1 -- 172.21.15.172:0/282796436 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894049040 con 0x561ff42422e0 2024-09-10T08:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.598+0000 7fb891ffb640 1 -- 172.21.15.172:0/282796436 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb8940ba280 con 0x561ff42422e0 2024-09-10T08:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.598+0000 7fb8b37bd740 1 -- 172.21.15.172:0/282796436 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb89c018040 con 0x561ff42422e0 2024-09-10T08:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.598+0000 7fb891ffb640 1 -- 172.21.15.172:0/282796436 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb8940bc680 con 0x561ff42422e0 2024-09-10T08:58:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.598+0000 7fb8b37bd740 10 client.19095 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.598+0000 7fb8b37bd740 1 -- 172.21.15.172:0/282796436 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561ff42c2450 con 0x561ff42422e0 2024-09-10T08:58:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.598+0000 7fb891ffb640 1 -- 172.21.15.172:0/282796436 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb89c018040 con 0x561ff42422e0 2024-09-10T08:58:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.598+0000 7fb891ffb640 1 -- 172.21.15.172:0/282796436 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fb89403bc90 con 0x561ff42422e0 2024-09-10T08:58:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.598+0000 7fb8b37bd740 10 client.19095 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.598+0000 7fb8b37bd740 10 client.19095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:58:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.598+0000 7fb8b37bd740 20 client.19095 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.598+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/282796436 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561ff42c2e80 0x561ff42c5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.598+0000 7fb8b37bd740 4 client.19095 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:58:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.598+0000 7fb8b37bd740 1 -- 172.21.15.172:0/282796436 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ff41acd70 con 0x561ff42c2e80 2024-09-10T08:58:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.599+0000 7fb8aa59c640 1 --2- 172.21.15.172:0/282796436 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561ff42c2e80 0x561ff42c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.599+0000 7fb8aa59c640 1 --2- 172.21.15.172:0/282796436 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561ff42c2e80 0x561ff42c5270 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf06a24c9cfb2265 server_cookie=ce95a4d64a9ecde6 in_seq=0 out_seq=0 2024-09-10T08:58:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.599+0000 7fb891ffb640 10 client.19095 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.605+0000 7fb891ffb640 1 -- 172.21.15.172:0/282796436 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561ff41acd70 con 0x561ff42c2e80 2024-09-10T08:58:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.605+0000 7fb891ffb640 10 client.19095 handle_command_reply: tid=0 2024-09-10T08:58:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.605+0000 7fb8b37bd740 1 client.19095 shutdown 2024-09-10T08:58:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b37bd740 1 -- 172.21.15.172:0/282796436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42422e0 msgr2=0x561ff4250fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/282796436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42422e0 0x561ff4250fb0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fb894005ec0 tx=0x7fb894040fb0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b37bd740 1 -- 172.21.15.172:0/282796436 shutdown_connections 2024-09-10T08:58:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/282796436 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561ff42c2e80 0x561ff42c5270 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/282796436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561ff4247370 0x561ff4251f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/282796436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561ff42422e0 0x561ff4250fb0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb891ffb640 0 client.19095 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b37bd740 1 --2- 172.21.15.172:0/282796436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ff42419a0 0x561ff4250a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b37bd740 1 -- 172.21.15.172:0/282796436 >> 172.21.15.172:0/282796436 conn(0x561ff423d010 msgr2=0x561ff423c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b37bd740 1 -- 172.21.15.172:0/282796436 shutdown_connections 2024-09-10T08:58:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b37bd740 1 -- 172.21.15.172:0/282796436 wait complete. 2024-09-10T08:58:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b37bd740 20 client.19095 trim_cache size 0 max 16384 2024-09-10T08:58:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:58:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:58:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:58:35.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:58:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b0949640 1 -- 172.21.15.172:0/3514735604 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8980a2c30 msgr2=0x7fb8980a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b0949640 1 --2- 172.21.15.172:0/3514735604 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8980a2c30 0x7fb8980a50f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009270 tx=0x7fb8a4004a40 comp rx=0 tx=0).stop 2024-09-10T08:58:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b0949640 1 -- 172.21.15.172:0/3514735604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8ac104ef0 msgr2=0x7fb8ac198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b0949640 1 --2- 172.21.15.172:0/3514735604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8ac104ef0 0x7fb8ac198330 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fb8940314e0 tx=0x7fb8940042e0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b0949640 1 -- 172.21.15.172:0/3514735604 shutdown_connections 2024-09-10T08:58:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b0949640 1 --2- 172.21.15.172:0/3514735604 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb8980a2c30 0x7fb8980a50f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b0949640 1 --2- 172.21.15.172:0/3514735604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8ac107820 0x7fb8ac19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b0949640 1 --2- 172.21.15.172:0/3514735604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8ac104ef0 0x7fb8ac198330 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b0949640 1 --2- 172.21.15.172:0/3514735604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8ac1025c0 0x7fb8ac197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.606+0000 7fb8b0949640 1 -- 172.21.15.172:0/3514735604 >> 172.21.15.172:0/3514735604 conn(0x7fb8ac0fc1e0 msgr2=0x7fb8ac102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.607+0000 7fb8b0949640 1 -- 172.21.15.172:0/3514735604 shutdown_connections 2024-09-10T08:58:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:35.607+0000 7fb8b0949640 1 -- 172.21.15.172:0/3514735604 wait complete. 2024-09-10T08:58:35.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:35 smithi174 ceph-mon[34068]: pgmap v721: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 25 op/s 2024-09-10T08:58:35.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:35 smithi191 ceph-mon[35314]: pgmap v721: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 25 op/s 2024-09-10T08:58:35.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:35 smithi172 ceph-mon[30854]: pgmap v721: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 25 op/s 2024-09-10T08:58:37.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:37 smithi174 ceph-mon[34068]: pgmap v722: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 17 MiB/s wr, 19 op/s 2024-09-10T08:58:37.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:37 smithi191 ceph-mon[35314]: pgmap v722: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 17 MiB/s wr, 19 op/s 2024-09-10T08:58:37.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:37 smithi172 ceph-mon[30854]: pgmap v722: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 17 MiB/s wr, 19 op/s 2024-09-10T08:58:39.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:39 smithi174 ceph-mon[34068]: pgmap v723: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 15 MiB/s wr, 19 op/s 2024-09-10T08:58:39.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:39 smithi191 ceph-mon[35314]: pgmap v723: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 15 MiB/s wr, 19 op/s 2024-09-10T08:58:39.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:39 smithi172 ceph-mon[30854]: pgmap v723: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 15 MiB/s wr, 19 op/s 2024-09-10T08:58:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:41 smithi174 ceph-mon[34068]: pgmap v724: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-09-10T08:58:41.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:41 smithi191 ceph-mon[35314]: pgmap v724: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-09-10T08:58:41.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:41 smithi172 ceph-mon[30854]: pgmap v724: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-09-10T08:58:42.789 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:58:43.052 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:58:43.384 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:58:43.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.551+0000 7f0c72565640 1 -- 172.21.15.172:0/338426038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c6c106f20 msgr2=0x7f0c6c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:43.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.551+0000 7f0c72565640 1 --2- 172.21.15.172:0/338426038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c6c106f20 0x7f0c6c10eaa0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009ca0 tx=0x7f0c6402a8c0 comp rx=0 tx=0).stop 2024-09-10T08:58:43.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.551+0000 7f0c72565640 1 -- 172.21.15.172:0/338426038 shutdown_connections 2024-09-10T08:58:43.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.551+0000 7f0c72565640 1 --2- 172.21.15.172:0/338426038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c6c106f20 0x7f0c6c10eaa0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:43.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.551+0000 7f0c72565640 1 --2- 172.21.15.172:0/338426038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c6c1027a0 0x7f0c6c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:43.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.551+0000 7f0c72565640 1 --2- 172.21.15.172:0/338426038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c6c101df0 0x7f0c6c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:43.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.551+0000 7f0c72565640 1 -- 172.21.15.172:0/338426038 >> 172.21.15.172:0/338426038 conn(0x7f0c6c0fd8f0 msgr2=0x7f0c6c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:43.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.551+0000 7f0c72565640 1 -- 172.21.15.172:0/338426038 shutdown_connections 2024-09-10T08:58:43.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.551+0000 7f0c72565640 1 -- 172.21.15.172:0/338426038 wait complete. 2024-09-10T08:58:43.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.552+0000 7f0c72565640 1 Processor -- start 2024-09-10T08:58:43.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.552+0000 7f0c72565640 1 -- start start 2024-09-10T08:58:43.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.552+0000 7f0c72565640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c6c101df0 0x7f0c6c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:43.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.552+0000 7f0c72565640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c6c1027a0 0x7f0c6c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:43.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.552+0000 7f0c72565640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c6c106f20 0x7f0c6c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:43.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.552+0000 7f0c72565640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c111e30 con 0x7f0c6c106f20 2024-09-10T08:58:43.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.552+0000 7f0c72565640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c111cb0 con 0x7f0c6c101df0 2024-09-10T08:58:43.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.552+0000 7f0c72565640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c111fb0 con 0x7f0c6c1027a0 2024-09-10T08:58:43.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.552+0000 7f0c70b02640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c6c106f20 0x7f0c6c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:43.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.552+0000 7f0c70b02640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c6c106f20 0x7f0c6c19cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56132/0 (socket says 172.21.15.172:56132) 2024-09-10T08:58:43.573 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.552+0000 7f0c70b02640 1 -- 172.21.15.172:0/3707787793 learned_addr learned my addr 172.21.15.172:0/3707787793 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:43.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.553+0000 7f0c6bfff640 1 --2- 172.21.15.172:0/3707787793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c6c101df0 0x7f0c6c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:43.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.553+0000 7f0c6b7fe640 1 --2- 172.21.15.172:0/3707787793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c6c1027a0 0x7f0c6c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:43.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.553+0000 7f0c70b02640 1 -- 172.21.15.172:0/3707787793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c6c1027a0 msgr2=0x7f0c6c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:43.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.553+0000 7f0c70b02640 1 --2- 172.21.15.172:0/3707787793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c6c1027a0 0x7f0c6c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:43.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.553+0000 7f0c70b02640 1 -- 172.21.15.172:0/3707787793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c6c101df0 msgr2=0x7f0c6c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:43.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.553+0000 7f0c70b02640 1 --2- 172.21.15.172:0/3707787793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c6c101df0 0x7f0c6c1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:43.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.553+0000 7f0c70b02640 1 -- 172.21.15.172:0/3707787793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6c19d1a0 con 0x7f0c6c106f20 2024-09-10T08:58:43.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.553+0000 7f0c70b02640 1 --2- 172.21.15.172:0/3707787793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c6c106f20 0x7f0c6c19cac0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009dc0 tx=0x7f0c64003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:43.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.553+0000 7f0c697fa640 1 -- 172.21.15.172:0/3707787793 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c64004040 con 0x7f0c6c106f20 2024-09-10T08:58:43.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.553+0000 7f0c72565640 1 -- 172.21.15.172:0/3707787793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c6c19d430 con 0x7f0c6c106f20 2024-09-10T08:58:43.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.553+0000 7f0c697fa640 1 -- 172.21.15.172:0/3707787793 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c64004690 con 0x7f0c6c106f20 2024-09-10T08:58:43.574 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.553+0000 7f0c697fa640 1 -- 172.21.15.172:0/3707787793 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c64030d90 con 0x7f0c6c106f20 2024-09-10T08:58:43.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.553+0000 7f0c72565640 1 -- 172.21.15.172:0/3707787793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c6c1a4d70 con 0x7f0c6c106f20 2024-09-10T08:58:43.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.554+0000 7f0c72565640 1 -- 172.21.15.172:0/3707787793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c6c1021d0 con 0x7f0c6c106f20 2024-09-10T08:58:43.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.555+0000 7f0c697fa640 1 -- 172.21.15.172:0/3707787793 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0c640305c0 con 0x7f0c6c106f20 2024-09-10T08:58:43.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.556+0000 7f0c697fa640 1 --2- 172.21.15.172:0/3707787793 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c540a2b80 0x7f0c540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:43.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.556+0000 7f0c697fa640 1 -- 172.21.15.172:0/3707787793 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0c640f0e00 con 0x7f0c6c106f20 2024-09-10T08:58:43.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.556+0000 7f0c6bfff640 1 --2- 172.21.15.172:0/3707787793 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c540a2b80 0x7f0c540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:43.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.557+0000 7f0c6bfff640 1 --2- 172.21.15.172:0/3707787793 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c540a2b80 0x7f0c540a5040 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c005e10 tx=0x7f0c5c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:43.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.559+0000 7f0c697fa640 1 -- 172.21.15.172:0/3707787793 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c640bda00 con 0x7f0c6c106f20 2024-09-10T08:58:43.646 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:58:43.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:43 smithi174 ceph-mon[34068]: pgmap v725: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.5 MiB/s wr, 14 op/s 2024-09-10T08:58:43.766 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:43 smithi172 ceph-mon[30854]: pgmap v725: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.5 MiB/s wr, 14 op/s 2024-09-10T08:58:43.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.778+0000 7f0c72565640 1 -- 172.21.15.172:0/3707787793 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c6c10a190 con 0x7f0c6c106f20 2024-09-10T08:58:43.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.779+0000 7f0c697fa640 1 -- 172.21.15.172:0/3707787793 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+12006 (secure 0 0 0) 0x7f0c640bd300 con 0x7f0c6c106f20 2024-09-10T08:58:43.780 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:58:43.780 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":40,"btime":"2024-09-10T08:58:02:716535+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:58:02.666404+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T08:58:43.781 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 40 2024-09-10T08:58:43.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.781+0000 7f0c72565640 1 -- 172.21.15.172:0/3707787793 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c540a2b80 msgr2=0x7f0c540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:43.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.781+0000 7f0c72565640 1 --2- 172.21.15.172:0/3707787793 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c540a2b80 0x7f0c540a5040 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c005e10 tx=0x7f0c5c005da0 comp rx=0 tx=0).stop 2024-09-10T08:58:43.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.781+0000 7f0c72565640 1 -- 172.21.15.172:0/3707787793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c6c106f20 msgr2=0x7f0c6c19cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:43.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.781+0000 7f0c72565640 1 --2- 172.21.15.172:0/3707787793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c6c106f20 0x7f0c6c19cac0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009dc0 tx=0x7f0c64003ba0 comp rx=0 tx=0).stop 2024-09-10T08:58:43.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.781+0000 7f0c72565640 1 -- 172.21.15.172:0/3707787793 shutdown_connections 2024-09-10T08:58:43.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.781+0000 7f0c72565640 1 --2- 172.21.15.172:0/3707787793 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0c540a2b80 0x7f0c540a5040 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:43.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.781+0000 7f0c72565640 1 --2- 172.21.15.172:0/3707787793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c6c106f20 0x7f0c6c19cac0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:43.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.781+0000 7f0c72565640 1 --2- 172.21.15.172:0/3707787793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c6c1027a0 0x7f0c6c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:43.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.781+0000 7f0c72565640 1 --2- 172.21.15.172:0/3707787793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c6c101df0 0x7f0c6c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:43.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.781+0000 7f0c72565640 1 -- 172.21.15.172:0/3707787793 >> 172.21.15.172:0/3707787793 conn(0x7f0c6c0fd8f0 msgr2=0x7f0c6c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:43.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.781+0000 7f0c72565640 1 -- 172.21.15.172:0/3707787793 shutdown_connections 2024-09-10T08:58:43.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:43.781+0000 7f0c72565640 1 -- 172.21.15.172:0/3707787793 wait complete. 2024-09-10T08:58:43.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:43 smithi191 ceph-mon[35314]: pgmap v725: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.5 MiB/s wr, 14 op/s 2024-09-10T08:58:44.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:44 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3707787793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:58:44.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:44 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3707787793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:58:44.796 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 4766dcba --release --await' 2024-09-10T08:58:44.797 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 4766dcba --release --await 2024-09-10T08:58:44.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:44 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3707787793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:58:45.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:44.999+0000 7fa4c4551640 1 -- 172.21.15.172:0/707411878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4bc1082a0 msgr2=0x7fa4bc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:45.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:44.999+0000 7fa4c4551640 1 --2- 172.21.15.172:0/707411878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4bc1082a0 0x7fa4bc10ec40 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4009cd0 tx=0x7fa4b402a8e0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:44.999+0000 7fa4c4551640 1 -- 172.21.15.172:0/707411878 shutdown_connections 2024-09-10T08:58:45.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:44.999+0000 7fa4c4551640 1 --2- 172.21.15.172:0/707411878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4bc1082a0 0x7fa4bc10ec40 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:44.999+0000 7fa4c4551640 1 --2- 172.21.15.172:0/707411878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4bc1036e0 0x7fa4bc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:44.999+0000 7fa4c4551640 1 --2- 172.21.15.172:0/707411878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4bc102d30 0x7fa4bc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:44.999+0000 7fa4c4551640 1 -- 172.21.15.172:0/707411878 >> 172.21.15.172:0/707411878 conn(0x7fa4bc0fc7d0 msgr2=0x7fa4bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:45.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:44.999+0000 7fa4c4551640 1 -- 172.21.15.172:0/707411878 shutdown_connections 2024-09-10T08:58:45.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:44.999+0000 7fa4c4551640 1 -- 172.21.15.172:0/707411878 wait complete. 2024-09-10T08:58:45.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.000+0000 7fa4c4551640 1 Processor -- start 2024-09-10T08:58:45.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.000+0000 7fa4c4551640 1 -- start start 2024-09-10T08:58:45.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.000+0000 7fa4c4551640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4bc102d30 0x7fa4bc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.000+0000 7fa4c4551640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4bc1036e0 0x7fa4bc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.000+0000 7fa4c4551640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4bc1082a0 0x7fa4bc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.000+0000 7fa4c4551640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc111f50 con 0x7fa4bc1082a0 2024-09-10T08:58:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.000+0000 7fa4c4551640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc111dd0 con 0x7fa4bc102d30 2024-09-10T08:58:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.000+0000 7fa4c4551640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc1120d0 con 0x7fa4bc1036e0 2024-09-10T08:58:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.000+0000 7fa4c2aee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4bc1082a0 0x7fa4bc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.000+0000 7fa4c2aee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4bc1082a0 0x7fa4bc101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56150/0 (socket says 172.21.15.172:56150) 2024-09-10T08:58:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.000+0000 7fa4c2aee640 1 -- 172.21.15.172:0/1051349785 learned_addr learned my addr 172.21.15.172:0/1051349785 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.000+0000 7fa4c1aec640 1 --2- 172.21.15.172:0/1051349785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4bc1036e0 0x7fa4bc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.000+0000 7fa4c22ed640 1 --2- 172.21.15.172:0/1051349785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4bc102d30 0x7fa4bc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.001+0000 7fa4c2aee640 1 -- 172.21.15.172:0/1051349785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4bc1036e0 msgr2=0x7fa4bc100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:45.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.001+0000 7fa4c2aee640 1 --2- 172.21.15.172:0/1051349785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4bc1036e0 0x7fa4bc100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.001+0000 7fa4c2aee640 1 -- 172.21.15.172:0/1051349785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4bc102d30 msgr2=0x7fa4bc100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:45.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.001+0000 7fa4c2aee640 1 --2- 172.21.15.172:0/1051349785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4bc102d30 0x7fa4bc100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.001+0000 7fa4c2aee640 1 -- 172.21.15.172:0/1051349785 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4bc1a6960 con 0x7fa4bc1082a0 2024-09-10T08:58:45.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.001+0000 7fa4c22ed640 1 --2- 172.21.15.172:0/1051349785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4bc102d30 0x7fa4bc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:45.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.001+0000 7fa4c2aee640 1 --2- 172.21.15.172:0/1051349785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4bc1082a0 0x7fa4bc101410 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4005cc0 tx=0x7fa4b4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:45.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.001+0000 7fa4b37fe640 1 -- 172.21.15.172:0/1051349785 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4004450 con 0x7fa4bc1082a0 2024-09-10T08:58:45.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.001+0000 7fa4b37fe640 1 -- 172.21.15.172:0/1051349785 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4b4031c90 con 0x7fa4bc1082a0 2024-09-10T08:58:45.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.001+0000 7fa4c4551640 1 -- 172.21.15.172:0/1051349785 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4bc1a6bf0 con 0x7fa4bc1082a0 2024-09-10T08:58:45.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.001+0000 7fa4b37fe640 1 -- 172.21.15.172:0/1051349785 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b403b400 con 0x7fa4bc1082a0 2024-09-10T08:58:45.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.001+0000 7fa4c4551640 1 -- 172.21.15.172:0/1051349785 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4bc1a7130 con 0x7fa4bc1082a0 2024-09-10T08:58:45.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.003+0000 7fa4b37fe640 1 -- 172.21.15.172:0/1051349785 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa4b4004680 con 0x7fa4bc1082a0 2024-09-10T08:58:45.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.004+0000 7fa4c692a740 1 Processor -- start 2024-09-10T08:58:45.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.004+0000 7fa4c692a740 1 -- start start 2024-09-10T08:58:45.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.004+0000 7fa4b37fe640 1 --2- 172.21.15.172:0/1051349785 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4a80a2b80 0x7fa4a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.004+0000 7fa4c692a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0db4a0 0x561b8c0db880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.004+0000 7fa4c692a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0dbdc0 0x561b8c0e6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.004+0000 7fa4c692a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0e6680 0x561b8c0e8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.004+0000 7fa4c692a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561b8c0412e0 con 0x561b8c0db4a0 2024-09-10T08:58:45.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.004+0000 7fa4c692a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561b8bfe8ce0 con 0x561b8c0dbdc0 2024-09-10T08:58:45.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.004+0000 7fa4c692a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561b8c03d2a0 con 0x561b8c0e6680 2024-09-10T08:58:45.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.004+0000 7fa4c1aec640 1 --2- 172.21.15.172:0/1051349785 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4a80a2b80 0x7fa4a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:45.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.004+0000 7fa4c22ed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0db4a0 0x561b8c0db880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:45.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.004+0000 7fa4c22ed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0db4a0 0x561b8c0db880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56156/0 (socket says 172.21.15.172:56156) 2024-09-10T08:58:45.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.004+0000 7fa4c22ed640 1 -- 172.21.15.172:0/432162390 learned_addr learned my addr 172.21.15.172:0/432162390 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:45.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.004+0000 7fa4b37fe640 1 -- 172.21.15.172:0/1051349785 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa4b402e030 con 0x7fa4bc1082a0 2024-09-10T08:58:45.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.004+0000 7fa4c2aee640 1 --2- 172.21.15.172:0/432162390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0e6680 0x561b8c0e8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:45.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.005+0000 7fa4c22ed640 1 -- 172.21.15.172:0/432162390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0e6680 msgr2=0x561b8c0e8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:45.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.005+0000 7fa4c22ed640 1 --2- 172.21.15.172:0/432162390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0e6680 0x561b8c0e8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.005+0000 7fa4c22ed640 1 -- 172.21.15.172:0/432162390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0dbdc0 msgr2=0x561b8c0e6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:45.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.005+0000 7fa4c22ed640 1 --2- 172.21.15.172:0/432162390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0dbdc0 0x561b8c0e6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.005+0000 7fa4c22ed640 1 -- 172.21.15.172:0/432162390 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b8bfcd440 con 0x561b8c0db4a0 2024-09-10T08:58:45.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.005+0000 7fa4c1aec640 1 --2- 172.21.15.172:0/1051349785 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4a80a2b80 0x7fa4a80a5040 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc102420 tx=0x7fa4ac00a670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:45.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.005+0000 7fa4c22ed640 1 --2- 172.21.15.172:0/432162390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0db4a0 0x561b8c0db880 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800bd40 tx=0x7fa4b800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291f31bd5978c51c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:45.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.005+0000 7fa4b0ff9640 1 -- 172.21.15.172:0/432162390 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8015070 con 0x561b8c0db4a0 2024-09-10T08:58:45.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.005+0000 7fa4b0ff9640 1 -- 172.21.15.172:0/432162390 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4b80118f0 con 0x561b8c0db4a0 2024-09-10T08:58:45.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.005+0000 7fa4b0ff9640 1 -- 172.21.15.172:0/432162390 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8010870 con 0x561b8c0db4a0 2024-09-10T08:58:45.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.006+0000 7fa4c692a740 1 -- 172.21.15.172:0/432162390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0db4a0 msgr2=0x561b8c0db880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:45.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.006+0000 7fa4c692a740 1 --2- 172.21.15.172:0/432162390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0db4a0 0x561b8c0db880 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800bd40 tx=0x7fa4b800f730 comp rx=0 tx=0).stop 2024-09-10T08:58:45.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.006+0000 7fa4c692a740 1 -- 172.21.15.172:0/432162390 shutdown_connections 2024-09-10T08:58:45.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.006+0000 7fa4c692a740 1 --2- 172.21.15.172:0/432162390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0e6680 0x561b8c0e8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.006+0000 7fa4c692a740 1 --2- 172.21.15.172:0/432162390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0dbdc0 0x561b8c0e6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.006+0000 7fa4c692a740 1 --2- 172.21.15.172:0/432162390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0db4a0 0x561b8c0db880 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.006+0000 7fa4c692a740 1 -- 172.21.15.172:0/432162390 >> 172.21.15.172:0/432162390 conn(0x561b8c0d7a30 msgr2=0x561b8c0d7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:45.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.006+0000 7fa4c692a740 1 -- 172.21.15.172:0/432162390 shutdown_connections 2024-09-10T08:58:45.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.006+0000 7fa4c692a740 1 -- 172.21.15.172:0/432162390 wait complete. 2024-09-10T08:58:45.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.006+0000 7fa4c692a740 1 -- start start 2024-09-10T08:58:45.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.006+0000 7fa4c692a740 1 Processor -- start 2024-09-10T08:58:45.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.006+0000 7fa4c692a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0db4a0 0x561b8c0e5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.006+0000 7fa4c692a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0dbdc0 0x561b8c0e3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.006+0000 7fa4c692a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0e6680 0x561b8c0e3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.006+0000 7fa4c692a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561b8c0e9250 con 0x561b8c0dbdc0 2024-09-10T08:58:45.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.006+0000 7fa4c692a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561b8bffc050 con 0x561b8c0db4a0 2024-09-10T08:58:45.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.006+0000 7fa4c692a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561b8c0e93d0 con 0x561b8c0e6680 2024-09-10T08:58:45.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4c692a740 10 client.0 _ll_register_callbacks cb 0x561b8c03c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:45.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4c1aec640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0dbdc0 0x561b8c0e3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:45.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4c1aec640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0dbdc0 0x561b8c0e3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56166/0 (socket says 172.21.15.172:56166) 2024-09-10T08:58:45.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4c1aec640 1 -- 172.21.15.172:0/2523291449 learned_addr learned my addr 172.21.15.172:0/2523291449 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:45.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4c2aee640 1 --2- 172.21.15.172:0/2523291449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0e6680 0x561b8c0e3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:45.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4c22ed640 1 --2- 172.21.15.172:0/2523291449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0db4a0 0x561b8c0e5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:45.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4c1aec640 1 -- 172.21.15.172:0/2523291449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0e6680 msgr2=0x561b8c0e3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:45.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4c1aec640 1 --2- 172.21.15.172:0/2523291449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0e6680 0x561b8c0e3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4c1aec640 1 -- 172.21.15.172:0/2523291449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0db4a0 msgr2=0x561b8c0e5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:45.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4c1aec640 1 --2- 172.21.15.172:0/2523291449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0db4a0 0x561b8c0e5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4c1aec640 1 -- 172.21.15.172:0/2523291449 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b8bdb23c0 con 0x561b8c0dbdc0 2024-09-10T08:58:45.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4c22ed640 1 --2- 172.21.15.172:0/2523291449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0db4a0 0x561b8c0e5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:45.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4c2aee640 1 --2- 172.21.15.172:0/2523291449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0e6680 0x561b8c0e3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:45.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4c1aec640 1 --2- 172.21.15.172:0/2523291449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0dbdc0 0x561b8c0e3250 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0129d0 tx=0x7fa4ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9500b14eecf11238 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:45.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:45.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4a5ffb640 1 -- 172.21.15.172:0/2523291449 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac022070 con 0x561b8c0dbdc0 2024-09-10T08:58:45.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4c692a740 1 -- 172.21.15.172:0/2523291449 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4b80118f0 con 0x561b8c0dbdc0 2024-09-10T08:58:45.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4a5ffb640 1 -- 172.21.15.172:0/2523291449 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa4ac00ed40 con 0x561b8c0dbdc0 2024-09-10T08:58:45.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4a5ffb640 1 -- 172.21.15.172:0/2523291449 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa4ac02f050 con 0x561b8c0dbdc0 2024-09-10T08:58:45.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4c692a740 10 client.19113 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:45.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.007+0000 7fa4c692a740 1 -- 172.21.15.172:0/2523291449 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561b8c0f4c80 con 0x561b8c0dbdc0 2024-09-10T08:58:45.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.008+0000 7fa4a5ffb640 1 -- 172.21.15.172:0/2523291449 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4b80118f0 con 0x561b8c0dbdc0 2024-09-10T08:58:45.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.008+0000 7fa4a5ffb640 1 -- 172.21.15.172:0/2523291449 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fa4ac01e4f0 con 0x561b8c0dbdc0 2024-09-10T08:58:45.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.008+0000 7fa4c692a740 10 client.19113 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:45.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.008+0000 7fa4c692a740 10 client.19113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:58:45.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.008+0000 7fa4c692a740 20 client.19113 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:45.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.008+0000 7fa4c692a740 1 --2- 172.21.15.172:0/2523291449 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561b8c0f60e0 0x561b8c0f8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.008+0000 7fa4c692a740 4 client.19113 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:58:45.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.008+0000 7fa4c692a740 1 -- 172.21.15.172:0/2523291449 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b8c04e850 con 0x561b8c0f60e0 2024-09-10T08:58:45.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.008+0000 7fa4c22ed640 1 --2- 172.21.15.172:0/2523291449 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561b8c0f60e0 0x561b8c0f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:45.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.009+0000 7fa4a5ffb640 10 client.19113 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:45.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.009+0000 7fa4c22ed640 1 --2- 172.21.15.172:0/2523291449 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561b8c0f60e0 0x561b8c0f8520 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ec1397c0445d583 server_cookie=2c9d649ceffc1d72 in_seq=0 out_seq=0 2024-09-10T08:58:45.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.009+0000 7fa4a5ffb640 1 -- 172.21.15.172:0/2523291449 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b8c04e850 con 0x561b8c0f60e0 2024-09-10T08:58:45.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.009+0000 7fa4a5ffb640 10 client.19113 handle_command_reply: tid=0 2024-09-10T08:58:45.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.009+0000 7fa4c692a740 1 client.19113 shutdown 2024-09-10T08:58:45.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.009+0000 7fa4c692a740 1 -- 172.21.15.172:0/2523291449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0dbdc0 msgr2=0x561b8c0e3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:45.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.009+0000 7fa4c692a740 1 --2- 172.21.15.172:0/2523291449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0dbdc0 0x561b8c0e3250 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0129d0 tx=0x7fa4ac004730 comp rx=0 tx=0).stop 2024-09-10T08:58:45.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.009+0000 7fa4c692a740 1 -- 172.21.15.172:0/2523291449 shutdown_connections 2024-09-10T08:58:45.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.009+0000 7fa4c692a740 1 --2- 172.21.15.172:0/2523291449 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561b8c0f60e0 0x561b8c0f8520 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.009+0000 7fa4c692a740 1 --2- 172.21.15.172:0/2523291449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0e6680 0x561b8c0e3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.009+0000 7fa4c692a740 1 --2- 172.21.15.172:0/2523291449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0dbdc0 0x561b8c0e3250 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.009+0000 7fa4a5ffb640 0 client.19113 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:45.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.009+0000 7fa4c692a740 1 --2- 172.21.15.172:0/2523291449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0db4a0 0x561b8c0e5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.010+0000 7fa4c692a740 1 -- 172.21.15.172:0/2523291449 >> 172.21.15.172:0/2523291449 conn(0x561b8c0d7a30 msgr2=0x561b8c0d6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:45.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.010+0000 7fa4c692a740 1 -- 172.21.15.172:0/2523291449 shutdown_connections 2024-09-10T08:58:45.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.010+0000 7fa4c692a740 1 -- 172.21.15.172:0/2523291449 wait complete. 2024-09-10T08:58:45.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.010+0000 7fa4c692a740 20 client.19113 trim_cache size 0 max 16384 2024-09-10T08:58:45.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c692a740 1 Processor -- start 2024-09-10T08:58:45.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c692a740 1 -- start start 2024-09-10T08:58:45.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c692a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0d7a30 0x561b8c0e51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c692a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0db4a0 0x561b8c0fddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c692a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0dbdc0 0x561b8c0fe330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c692a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc1b0cc0 con 0x561b8c0dbdc0 2024-09-10T08:58:45.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c692a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc1b2890 con 0x561b8c0db4a0 2024-09-10T08:58:45.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c692a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc1b1390 con 0x561b8c0d7a30 2024-09-10T08:58:45.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c2aee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0dbdc0 0x561b8c0fe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:45.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c2aee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0dbdc0 0x561b8c0fe330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56168/0 (socket says 172.21.15.172:56168) 2024-09-10T08:58:45.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c2aee640 1 -- 172.21.15.172:0/2199839303 learned_addr learned my addr 172.21.15.172:0/2199839303 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:45.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c22ed640 1 --2- 172.21.15.172:0/2199839303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0d7a30 0x561b8c0e51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:45.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c1aec640 1 --2- 172.21.15.172:0/2199839303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0db4a0 0x561b8c0fddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:45.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c2aee640 1 -- 172.21.15.172:0/2199839303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0d7a30 msgr2=0x561b8c0e51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:45.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c2aee640 1 --2- 172.21.15.172:0/2199839303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0d7a30 0x561b8c0e51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c2aee640 1 -- 172.21.15.172:0/2199839303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0db4a0 msgr2=0x561b8c0fddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:45.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c2aee640 1 --2- 172.21.15.172:0/2199839303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0db4a0 0x561b8c0fddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.027+0000 7fa4c2aee640 1 -- 172.21.15.172:0/2199839303 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b8c0db2d0 con 0x561b8c0dbdc0 2024-09-10T08:58:45.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.028+0000 7fa4c2aee640 1 --2- 172.21.15.172:0/2199839303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0dbdc0 0x561b8c0fe330 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40b8e90 tx=0x7fa4b40045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fe840fc729d3e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:45.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.028+0000 7fa4a7fff640 1 -- 172.21.15.172:0/2199839303 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b40098b0 con 0x561b8c0dbdc0 2024-09-10T08:58:45.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.028+0000 7fa4a7fff640 1 -- 172.21.15.172:0/2199839303 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4b4036040 con 0x561b8c0dbdc0 2024-09-10T08:58:45.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.028+0000 7fa4a7fff640 1 -- 172.21.15.172:0/2199839303 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b40f02b0 con 0x561b8c0dbdc0 2024-09-10T08:58:45.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.028+0000 7fa4c692a740 1 -- 172.21.15.172:0/2199839303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0dbdc0 msgr2=0x561b8c0fe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:45.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.028+0000 7fa4c692a740 1 --2- 172.21.15.172:0/2199839303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0dbdc0 0x561b8c0fe330 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40b8e90 tx=0x7fa4b40045f0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.028+0000 7fa4c692a740 1 -- 172.21.15.172:0/2199839303 shutdown_connections 2024-09-10T08:58:45.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.028+0000 7fa4c692a740 1 --2- 172.21.15.172:0/2199839303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0dbdc0 0x561b8c0fe330 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.028+0000 7fa4c692a740 1 --2- 172.21.15.172:0/2199839303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0db4a0 0x561b8c0fddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.028+0000 7fa4c692a740 1 --2- 172.21.15.172:0/2199839303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0d7a30 0x561b8c0e51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.029+0000 7fa4c692a740 1 -- 172.21.15.172:0/2199839303 >> 172.21.15.172:0/2199839303 conn(0x561b8c0e38f0 msgr2=0x561b8c0e4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:45.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.029+0000 7fa4c692a740 1 -- 172.21.15.172:0/2199839303 shutdown_connections 2024-09-10T08:58:45.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.029+0000 7fa4c692a740 1 -- 172.21.15.172:0/2199839303 wait complete. 2024-09-10T08:58:45.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.029+0000 7fa4c692a740 1 -- start start 2024-09-10T08:58:45.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.029+0000 7fa4c692a740 1 Processor -- start 2024-09-10T08:58:45.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.029+0000 7fa4c692a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0d7a30 0x561b8c0eede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.029+0000 7fa4c692a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0db4a0 0x561b8c0ec180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.029+0000 7fa4c692a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0dbdc0 0x561b8c0ec6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.029+0000 7fa4c692a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc1b21c0 con 0x561b8c0db4a0 2024-09-10T08:58:45.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.029+0000 7fa4c692a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc1b1af0 con 0x561b8c0d7a30 2024-09-10T08:58:45.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.029+0000 7fa4c692a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc1aeca0 con 0x561b8c0dbdc0 2024-09-10T08:58:45.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.029+0000 7fa4c692a740 10 client.0 _ll_register_callbacks cb 0x7fa4ac00a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:45.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.029+0000 7fa4c1aec640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0db4a0 0x561b8c0ec180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:45.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.029+0000 7fa4c1aec640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0db4a0 0x561b8c0ec180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56182/0 (socket says 172.21.15.172:56182) 2024-09-10T08:58:45.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.029+0000 7fa4c1aec640 1 -- 172.21.15.172:0/2166711758 learned_addr learned my addr 172.21.15.172:0/2166711758 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:45.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4c22ed640 1 --2- 172.21.15.172:0/2166711758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0d7a30 0x561b8c0eede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:45.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4c2aee640 1 --2- 172.21.15.172:0/2166711758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0dbdc0 0x561b8c0ec6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:45.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4c1aec640 1 -- 172.21.15.172:0/2166711758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0dbdc0 msgr2=0x561b8c0ec6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:45.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4c1aec640 1 --2- 172.21.15.172:0/2166711758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0dbdc0 0x561b8c0ec6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4c1aec640 1 -- 172.21.15.172:0/2166711758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0d7a30 msgr2=0x561b8c0eede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:45.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4c1aec640 1 --2- 172.21.15.172:0/2166711758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0d7a30 0x561b8c0eede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4c1aec640 1 -- 172.21.15.172:0/2166711758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b8c0ecf10 con 0x561b8c0db4a0 2024-09-10T08:58:45.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4c2aee640 1 --2- 172.21.15.172:0/2166711758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0dbdc0 0x561b8c0ec6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:45.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4c22ed640 1 --2- 172.21.15.172:0/2166711758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0d7a30 0x561b8c0eede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:45.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4c1aec640 1 --2- 172.21.15.172:0/2166711758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0db4a0 0x561b8c0ec180 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac004700 tx=0x7fa4ac01ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e19eaf1d6b916aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:45.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:45.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4a5ffb640 1 -- 172.21.15.172:0/2166711758 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac02c040 con 0x561b8c0db4a0 2024-09-10T08:58:45.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4a5ffb640 1 -- 172.21.15.172:0/2166711758 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa4ac01e5f0 con 0x561b8c0db4a0 2024-09-10T08:58:45.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4c692a740 1 -- 172.21.15.172:0/2166711758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4b4036040 con 0x561b8c0db4a0 2024-09-10T08:58:45.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4a5ffb640 1 -- 172.21.15.172:0/2166711758 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa4ac022040 con 0x561b8c0db4a0 2024-09-10T08:58:45.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4c692a740 10 client.19119 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:45.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4c692a740 1 -- 172.21.15.172:0/2166711758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561b8c160e70 con 0x561b8c0db4a0 2024-09-10T08:58:45.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.030+0000 7fa4a5ffb640 1 -- 172.21.15.172:0/2166711758 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4b4036040 con 0x561b8c0db4a0 2024-09-10T08:58:45.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.031+0000 7fa4a5ffb640 1 -- 172.21.15.172:0/2166711758 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fa4ac0054a0 con 0x561b8c0db4a0 2024-09-10T08:58:45.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.031+0000 7fa4c692a740 10 client.19119 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:45.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.031+0000 7fa4c692a740 10 client.19119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:58:45.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.031+0000 7fa4c692a740 20 client.19119 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:45.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.031+0000 7fa4c692a740 1 --2- 172.21.15.172:0/2166711758 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561b8c0e6680 0x561b8c1635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:45.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.031+0000 7fa4c692a740 4 client.19119 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T08:58:45.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.031+0000 7fa4c692a740 1 -- 172.21.15.172:0/2166711758 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561b8c048d60 con 0x561b8c0e6680 2024-09-10T08:58:45.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.031+0000 7fa4c22ed640 1 --2- 172.21.15.172:0/2166711758 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561b8c0e6680 0x561b8c1635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:45.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.031+0000 7fa4a5ffb640 10 client.19119 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:45.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.031+0000 7fa4c22ed640 1 --2- 172.21.15.172:0/2166711758 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561b8c0e6680 0x561b8c1635d0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6494fc8a504d4d4e server_cookie=dfac82a2eebaa20a in_seq=0 out_seq=0 2024-09-10T08:58:45.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4a5ffb640 1 -- 172.21.15.172:0/2166711758 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561b8c048d60 con 0x561b8c0e6680 2024-09-10T08:58:45.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4a5ffb640 10 client.19119 handle_command_reply: tid=0 2024-09-10T08:58:45.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4c692a740 1 client.19119 shutdown 2024-09-10T08:58:45.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4c692a740 1 -- 172.21.15.172:0/2166711758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0db4a0 msgr2=0x561b8c0ec180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:45.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4c692a740 1 --2- 172.21.15.172:0/2166711758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0db4a0 0x561b8c0ec180 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac004700 tx=0x7fa4ac01ede0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4c692a740 1 -- 172.21.15.172:0/2166711758 shutdown_connections 2024-09-10T08:58:45.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4c692a740 1 --2- 172.21.15.172:0/2166711758 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x561b8c0e6680 0x561b8c1635d0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4c692a740 1 --2- 172.21.15.172:0/2166711758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b8c0dbdc0 0x561b8c0ec6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4c692a740 1 --2- 172.21.15.172:0/2166711758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561b8c0db4a0 0x561b8c0ec180 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4a5ffb640 0 client.19119 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:45.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4c692a740 1 --2- 172.21.15.172:0/2166711758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b8c0d7a30 0x561b8c0eede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4c692a740 1 -- 172.21.15.172:0/2166711758 >> 172.21.15.172:0/2166711758 conn(0x561b8c0e38f0 msgr2=0x561b8c0e52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:45.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4c692a740 1 -- 172.21.15.172:0/2166711758 shutdown_connections 2024-09-10T08:58:45.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4c692a740 1 -- 172.21.15.172:0/2166711758 wait complete. 2024-09-10T08:58:45.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4c692a740 20 client.19119 trim_cache size 0 max 16384 2024-09-10T08:58:45.049 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:58:45.049 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T08:58:45.049 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T08:58:45.050 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:58:45.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4c4551640 1 -- 172.21.15.172:0/1051349785 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4a80a2b80 msgr2=0x7fa4a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:45.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4c4551640 1 --2- 172.21.15.172:0/1051349785 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4a80a2b80 0x7fa4a80a5040 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc102420 tx=0x7fa4ac00a670 comp rx=0 tx=0).stop 2024-09-10T08:58:45.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4c4551640 1 -- 172.21.15.172:0/1051349785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4bc1082a0 msgr2=0x7fa4bc101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:45.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.032+0000 7fa4c4551640 1 --2- 172.21.15.172:0/1051349785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4bc1082a0 0x7fa4bc101410 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4005cc0 tx=0x7fa4b4003c20 comp rx=0 tx=0).stop 2024-09-10T08:58:45.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.033+0000 7fa4c4551640 1 -- 172.21.15.172:0/1051349785 shutdown_connections 2024-09-10T08:58:45.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.033+0000 7fa4c4551640 1 --2- 172.21.15.172:0/1051349785 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4a80a2b80 0x7fa4a80a5040 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.033+0000 7fa4c4551640 1 --2- 172.21.15.172:0/1051349785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4bc1082a0 0x7fa4bc101410 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.033+0000 7fa4c4551640 1 --2- 172.21.15.172:0/1051349785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4bc1036e0 0x7fa4bc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.033+0000 7fa4c4551640 1 --2- 172.21.15.172:0/1051349785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4bc102d30 0x7fa4bc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:45.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.033+0000 7fa4c4551640 1 -- 172.21.15.172:0/1051349785 >> 172.21.15.172:0/1051349785 conn(0x7fa4bc0fc7d0 msgr2=0x7fa4bc105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:45.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.033+0000 7fa4c4551640 1 -- 172.21.15.172:0/1051349785 shutdown_connections 2024-09-10T08:58:45.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:45.033+0000 7fa4c4551640 1 -- 172.21.15.172:0/1051349785 wait complete. 2024-09-10T08:58:45.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:45 smithi174 ceph-mon[34068]: pgmap v726: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.5 MiB/s wr, 16 op/s 2024-09-10T08:58:45.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:45 smithi191 ceph-mon[35314]: pgmap v726: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.5 MiB/s wr, 16 op/s 2024-09-10T08:58:45.822 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:45 smithi172 ceph-mon[30854]: pgmap v726: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.5 MiB/s wr, 16 op/s 2024-09-10T08:58:45.857 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T08:58:45.858 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:58:45.861 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:58:46.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.386+0000 7f0d1aa4e640 1 -- 172.21.15.172:0/1222834305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d14102570 msgr2=0x7f0d141029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:46.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.386+0000 7f0d1aa4e640 1 --2- 172.21.15.172:0/1222834305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d14102570 0x7f0d141029b0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009900 tx=0x7f0cfc02a480 comp rx=0 tx=0).stop 2024-09-10T08:58:46.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.386+0000 7f0d1aa4e640 1 -- 172.21.15.172:0/1222834305 shutdown_connections 2024-09-10T08:58:46.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.386+0000 7f0d1aa4e640 1 --2- 172.21.15.172:0/1222834305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d14102ef0 0x7f0d1410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.386+0000 7f0d1aa4e640 1 --2- 172.21.15.172:0/1222834305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d14102570 0x7f0d141029b0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.386+0000 7f0d1aa4e640 1 --2- 172.21.15.172:0/1222834305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d14100e20 0x7f0d14101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.386+0000 7f0d1aa4e640 1 -- 172.21.15.172:0/1222834305 >> 172.21.15.172:0/1222834305 conn(0x7f0d140fc7d0 msgr2=0x7f0d140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:46.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.386+0000 7f0d1aa4e640 1 -- 172.21.15.172:0/1222834305 shutdown_connections 2024-09-10T08:58:46.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.386+0000 7f0d1aa4e640 1 -- 172.21.15.172:0/1222834305 wait complete. 2024-09-10T08:58:46.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d1aa4e640 1 Processor -- start 2024-09-10T08:58:46.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d1aa4e640 1 -- start start 2024-09-10T08:58:46.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d1aa4e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d14100e20 0x7f0d141972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d1aa4e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d14102570 0x7f0d141977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d1aa4e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d14102ef0 0x7f0d1419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d1aa4e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d14110e30 con 0x7f0d14102ef0 2024-09-10T08:58:46.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d1aa4e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d14110cb0 con 0x7f0d14100e20 2024-09-10T08:58:46.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d1aa4e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0d14110fb0 con 0x7f0d14102570 2024-09-10T08:58:46.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d18feb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d14102ef0 0x7f0d1419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:46.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d18feb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d14102ef0 0x7f0d1419bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49378/0 (socket says 172.21.15.172:49378) 2024-09-10T08:58:46.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d18feb640 1 -- 172.21.15.172:0/3241527548 learned_addr learned my addr 172.21.15.172:0/3241527548 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:46.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d0bfff640 1 --2- 172.21.15.172:0/3241527548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d14102570 0x7f0d141977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:46.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d13fff640 1 --2- 172.21.15.172:0/3241527548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d14100e20 0x7f0d141972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:46.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d18feb640 1 -- 172.21.15.172:0/3241527548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d14102570 msgr2=0x7f0d141977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:46.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d18feb640 1 --2- 172.21.15.172:0/3241527548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d14102570 0x7f0d141977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d18feb640 1 -- 172.21.15.172:0/3241527548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d14100e20 msgr2=0x7f0d141972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:46.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d18feb640 1 --2- 172.21.15.172:0/3241527548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d14100e20 0x7f0d141972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d18feb640 1 -- 172.21.15.172:0/3241527548 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1419c260 con 0x7f0d14102ef0 2024-09-10T08:58:46.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.387+0000 7f0d13fff640 1 --2- 172.21.15.172:0/3241527548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d14100e20 0x7f0d141972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:46.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.388+0000 7f0d18feb640 1 --2- 172.21.15.172:0/3241527548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d14102ef0 0x7f0d1419bb80 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00cad0 tx=0x7f0d0c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:46.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.388+0000 7f0d11ffb640 1 -- 172.21.15.172:0/3241527548 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c00ae20 con 0x7f0d14102ef0 2024-09-10T08:58:46.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.388+0000 7f0d11ffb640 1 -- 172.21.15.172:0/3241527548 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d0c005020 con 0x7f0d14102ef0 2024-09-10T08:58:46.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.388+0000 7f0d1aa4e640 1 -- 172.21.15.172:0/3241527548 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1419c550 con 0x7f0d14102ef0 2024-09-10T08:58:46.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.388+0000 7f0d11ffb640 1 -- 172.21.15.172:0/3241527548 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c00f6b0 con 0x7f0d14102ef0 2024-09-10T08:58:46.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.388+0000 7f0d1aa4e640 1 -- 172.21.15.172:0/3241527548 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d141a3e30 con 0x7f0d14102ef0 2024-09-10T08:58:46.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.390+0000 7f0d11ffb640 1 -- 172.21.15.172:0/3241527548 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0d0c0051d0 con 0x7f0d14102ef0 2024-09-10T08:58:46.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.390+0000 7f0d1ce27740 1 Processor -- start 2024-09-10T08:58:46.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d1ce27740 1 -- start start 2024-09-10T08:58:46.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d1ce27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f00f40 0x55a787f01320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d1ce27740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f01860 0x55a787f0df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d1ce27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f0e450 0x55a787f10840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d1ce27740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a787e66620 con 0x55a787f00f40 2024-09-10T08:58:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d1ce27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a787e0de50 con 0x55a787f0e450 2024-09-10T08:58:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d1ce27740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a787e625e0 con 0x55a787f01860 2024-09-10T08:58:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f00f40 0x55a787f01320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f00f40 0x55a787f01320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49388/0 (socket says 172.21.15.172:49388) 2024-09-10T08:58:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d0bfff640 1 -- 172.21.15.172:0/4129822791 learned_addr learned my addr 172.21.15.172:0/4129822791 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d11ffb640 1 --2- 172.21.15.172:0/3241527548 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0d000a2c80 0x7f0d000a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d13fff640 1 --2- 172.21.15.172:0/4129822791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f01860 0x55a787f0df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:46.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d0bfff640 1 --2- 172.21.15.172:0/4129822791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f0e450 0x55a787f10840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d11ffb640 1 -- 172.21.15.172:0/3241527548 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0d0c0cf400 con 0x7f0d14102ef0 2024-09-10T08:58:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d0bfff640 1 -- 172.21.15.172:0/4129822791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f01860 msgr2=0x55a787f0df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d0bfff640 1 --2- 172.21.15.172:0/4129822791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f01860 0x55a787f0df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d0bfff640 1 -- 172.21.15.172:0/4129822791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f0e450 msgr2=0x55a787f10840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d0bfff640 1 --2- 172.21.15.172:0/4129822791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f0e450 0x55a787f10840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d0bfff640 1 -- 172.21.15.172:0/4129822791 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a787dfed40 con 0x55a787f00f40 2024-09-10T08:58:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.391+0000 7f0d18feb640 1 --2- 172.21.15.172:0/3241527548 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0d000a2c80 0x7f0d000a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.392+0000 7f0d0bfff640 1 --2- 172.21.15.172:0/4129822791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f00f40 0x55a787f01320 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc02a9a0 tx=0x7f0cfc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37e56a1e8dd6aab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.392+0000 7f0d0a7fc640 1 -- 172.21.15.172:0/4129822791 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc044030 con 0x55a787f00f40 2024-09-10T08:58:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.393+0000 7f0d0a7fc640 1 -- 172.21.15.172:0/4129822791 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cfc0315b0 con 0x55a787f00f40 2024-09-10T08:58:46.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.393+0000 7f0d0a7fc640 1 -- 172.21.15.172:0/4129822791 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc035040 con 0x55a787f00f40 2024-09-10T08:58:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.393+0000 7f0d1ce27740 1 -- 172.21.15.172:0/4129822791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f00f40 msgr2=0x55a787f01320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.393+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/4129822791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f00f40 0x55a787f01320 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc02a9a0 tx=0x7f0cfc000bc0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.393+0000 7f0d18feb640 1 --2- 172.21.15.172:0/3241527548 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0d000a2c80 0x7f0d000a5140 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c09b830 tx=0x7f0d0c00a890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.394+0000 7f0d1ce27740 1 -- 172.21.15.172:0/4129822791 shutdown_connections 2024-09-10T08:58:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.394+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/4129822791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f0e450 0x55a787f10840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.394+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/4129822791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f01860 0x55a787f0df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.394+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/4129822791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f00f40 0x55a787f01320 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.394+0000 7f0d1ce27740 1 -- 172.21.15.172:0/4129822791 >> 172.21.15.172:0/4129822791 conn(0x55a787efd5f0 msgr2=0x55a787efd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.394+0000 7f0d1ce27740 1 -- 172.21.15.172:0/4129822791 shutdown_connections 2024-09-10T08:58:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.394+0000 7f0d1ce27740 1 -- 172.21.15.172:0/4129822791 wait complete. 2024-09-10T08:58:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.394+0000 7f0d1ce27740 1 -- start start 2024-09-10T08:58:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d1ce27740 1 Processor -- start 2024-09-10T08:58:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d1ce27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f00f40 0x55a787f0b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d1ce27740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f01860 0x55a787f0b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d1ce27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f0e450 0x55a787f0bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d1ce27740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a787f11020 con 0x55a787f00f40 2024-09-10T08:58:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d1ce27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a787deef10 con 0x55a787f0e450 2024-09-10T08:58:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d1ce27740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a787f111a0 con 0x55a787f01860 2024-09-10T08:58:46.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d1ce27740 10 client.0 _ll_register_callbacks cb 0x55a787e12db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f00f40 0x55a787f0b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f00f40 0x55a787f0b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49404/0 (socket says 172.21.15.172:49404) 2024-09-10T08:58:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d0bfff640 1 -- 172.21.15.172:0/2819021365 learned_addr learned my addr 172.21.15.172:0/2819021365 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d13fff640 1 --2- 172.21.15.172:0/2819021365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f01860 0x55a787f0b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d0bfff640 1 --2- 172.21.15.172:0/2819021365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f0e450 0x55a787f0bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d0bfff640 1 -- 172.21.15.172:0/2819021365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f01860 msgr2=0x55a787f0b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d0bfff640 1 --2- 172.21.15.172:0/2819021365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f01860 0x55a787f0b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d0bfff640 1 -- 172.21.15.172:0/2819021365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f0e450 msgr2=0x55a787f0bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d0bfff640 1 --2- 172.21.15.172:0/2819021365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f0e450 0x55a787f0bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.395+0000 7f0d0bfff640 1 -- 172.21.15.172:0/2819021365 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a787b75580 con 0x55a787f00f40 2024-09-10T08:58:46.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.396+0000 7f0d0bfff640 1 --2- 172.21.15.172:0/2819021365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f00f40 0x55a787f0b2f0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc005940 tx=0x7f0cfc0051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37ba2f20622ed2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.396+0000 7f0cf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.396+0000 7f0cf37fe640 1 -- 172.21.15.172:0/2819021365 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc035040 con 0x55a787f00f40 2024-09-10T08:58:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.396+0000 7f0d1ce27740 1 -- 172.21.15.172:0/2819021365 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cfc0315b0 con 0x55a787f00f40 2024-09-10T08:58:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.396+0000 7f0cf37fe640 1 -- 172.21.15.172:0/2819021365 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0cfc005550 con 0x55a787f00f40 2024-09-10T08:58:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.396+0000 7f0cf37fe640 1 -- 172.21.15.172:0/2819021365 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0cfc004040 con 0x55a787f00f40 2024-09-10T08:58:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.396+0000 7f0d1ce27740 10 client.19131 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.396+0000 7f0d1ce27740 1 -- 172.21.15.172:0/2819021365 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a787c3f960 con 0x55a787f00f40 2024-09-10T08:58:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.396+0000 7f0cf37fe640 1 -- 172.21.15.172:0/2819021365 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cfc0315b0 con 0x55a787f00f40 2024-09-10T08:58:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.396+0000 7f0cf37fe640 1 -- 172.21.15.172:0/2819021365 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f0cfc0046c0 con 0x55a787f00f40 2024-09-10T08:58:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.397+0000 7f0d1ce27740 10 client.19131 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.397+0000 7f0d1ce27740 10 client.19131 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:58:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.397+0000 7f0d1ce27740 20 client.19131 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:46.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.397+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/2819021365 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a787f04e00 0x55a787f05230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.397+0000 7f0d1ce27740 4 client.19131 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:58:46.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.397+0000 7f0d1ce27740 1 -- 172.21.15.172:0/2819021365 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a787e73b90 con 0x55a787f04e00 2024-09-10T08:58:46.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.397+0000 7f0d13fff640 1 --2- 172.21.15.172:0/2819021365 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a787f04e00 0x55a787f05230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:46.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.397+0000 7f0d13fff640 1 --2- 172.21.15.172:0/2819021365 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a787f04e00 0x55a787f05230 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1a1ed02b51a9436 server_cookie=8052f76b8c99d6bf in_seq=0 out_seq=0 2024-09-10T08:58:46.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.397+0000 7f0cf37fe640 10 client.19131 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:46.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.398+0000 7f0cf37fe640 1 -- 172.21.15.172:0/2819021365 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a787e73b90 con 0x55a787f04e00 2024-09-10T08:58:46.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.398+0000 7f0cf37fe640 10 client.19131 handle_command_reply: tid=0 2024-09-10T08:58:46.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.398+0000 7f0d1ce27740 1 client.19131 shutdown 2024-09-10T08:58:46.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.398+0000 7f0d1ce27740 1 -- 172.21.15.172:0/2819021365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f00f40 msgr2=0x55a787f0b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:46.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.398+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/2819021365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f00f40 0x55a787f0b2f0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc005940 tx=0x7f0cfc0051a0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.399+0000 7f0d1ce27740 1 -- 172.21.15.172:0/2819021365 shutdown_connections 2024-09-10T08:58:46.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.399+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/2819021365 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a787f04e00 0x55a787f05230 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.399+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/2819021365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f0e450 0x55a787f0bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.399+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/2819021365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f01860 0x55a787f0b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.399+0000 7f0cf37fe640 0 client.19131 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:46.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.399+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/2819021365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f00f40 0x55a787f0b2f0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.399+0000 7f0d1ce27740 1 -- 172.21.15.172:0/2819021365 >> 172.21.15.172:0/2819021365 conn(0x55a787efd5f0 msgr2=0x55a787f03350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:46.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.399+0000 7f0d1ce27740 1 -- 172.21.15.172:0/2819021365 shutdown_connections 2024-09-10T08:58:46.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.399+0000 7f0d1ce27740 1 -- 172.21.15.172:0/2819021365 wait complete. 2024-09-10T08:58:46.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.399+0000 7f0d1ce27740 20 client.19131 trim_cache size 0 max 16384 2024-09-10T08:58:46.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.416+0000 7f0d1ce27740 1 Processor -- start 2024-09-10T08:58:46.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.417+0000 7f0d1ce27740 1 -- start start 2024-09-10T08:58:46.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.417+0000 7f0d1ce27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f00f40 0x55a787f0d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.417+0000 7f0d1ce27740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f01860 0x55a787f21c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.417+0000 7f0d1ce27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f0e450 0x55a787f22140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.417+0000 7f0d1ce27740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141ac760 con 0x55a787f0e450 2024-09-10T08:58:46.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.417+0000 7f0d1ce27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141ae330 con 0x55a787f00f40 2024-09-10T08:58:46.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.417+0000 7f0d1ce27740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141ace30 con 0x55a787f01860 2024-09-10T08:58:46.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.417+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f0e450 0x55a787f22140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:46.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.417+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f0e450 0x55a787f22140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49414/0 (socket says 172.21.15.172:49414) 2024-09-10T08:58:46.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.417+0000 7f0d0bfff640 1 -- 172.21.15.172:0/3143591409 learned_addr learned my addr 172.21.15.172:0/3143591409 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:46.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.417+0000 7f0d0bfff640 1 --2- 172.21.15.172:0/3143591409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f00f40 0x55a787f0d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:46.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.418+0000 7f0d0bfff640 1 -- 172.21.15.172:0/3143591409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f01860 msgr2=0x55a787f21c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:46.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.418+0000 7f0d13fff640 1 --2- 172.21.15.172:0/3143591409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f01860 0x55a787f21c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:46.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.418+0000 7f0d0bfff640 1 --2- 172.21.15.172:0/3143591409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f01860 0x55a787f21c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.418+0000 7f0d0bfff640 1 -- 172.21.15.172:0/3143591409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f00f40 msgr2=0x55a787f0d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:46.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.418+0000 7f0d0bfff640 1 --2- 172.21.15.172:0/3143591409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f00f40 0x55a787f0d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.418+0000 7f0d0bfff640 1 -- 172.21.15.172:0/3143591409 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a787f022b0 con 0x55a787f0e450 2024-09-10T08:58:46.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.418+0000 7f0d13fff640 1 --2- 172.21.15.172:0/3143591409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f01860 0x55a787f21c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:46.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.418+0000 7f0d0bfff640 1 --2- 172.21.15.172:0/3143591409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f0e450 0x55a787f22140 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009850 tx=0x7f0cfc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624fdbbd4888e8b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:46.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.418+0000 7f0d09ffb640 1 -- 172.21.15.172:0/3143591409 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc044020 con 0x55a787f0e450 2024-09-10T08:58:46.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.418+0000 7f0d09ffb640 1 -- 172.21.15.172:0/3143591409 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cfc051020 con 0x55a787f0e450 2024-09-10T08:58:46.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.418+0000 7f0d09ffb640 1 -- 172.21.15.172:0/3143591409 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc035040 con 0x55a787f0e450 2024-09-10T08:58:46.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.419+0000 7f0d1ce27740 1 -- 172.21.15.172:0/3143591409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f0e450 msgr2=0x55a787f22140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:46.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.419+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/3143591409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f0e450 0x55a787f22140 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009850 tx=0x7f0cfc000bc0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.419+0000 7f0d1ce27740 1 -- 172.21.15.172:0/3143591409 shutdown_connections 2024-09-10T08:58:46.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.419+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/3143591409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f0e450 0x55a787f22140 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.419+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/3143591409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f01860 0x55a787f21c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.419+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/3143591409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f00f40 0x55a787f0d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.419+0000 7f0d1ce27740 1 -- 172.21.15.172:0/3143591409 >> 172.21.15.172:0/3143591409 conn(0x55a787efd5f0 msgr2=0x55a787f05a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:46.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.419+0000 7f0d1ce27740 1 -- 172.21.15.172:0/3143591409 shutdown_connections 2024-09-10T08:58:46.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.419+0000 7f0d1ce27740 1 -- 172.21.15.172:0/3143591409 wait complete. 2024-09-10T08:58:46.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.419+0000 7f0d1ce27740 1 -- start start 2024-09-10T08:58:46.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.420+0000 7f0d1ce27740 1 Processor -- start 2024-09-10T08:58:46.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.420+0000 7f0d1ce27740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f00f40 0x55a787f868a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.420+0000 7f0d1ce27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f01860 0x55a787f86de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.420+0000 7f0d1ce27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f0e450 0x55a787f1f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.420+0000 7f0d1ce27740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141adc60 con 0x55a787f01860 2024-09-10T08:58:46.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.420+0000 7f0d1ce27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141ad590 con 0x55a787f0e450 2024-09-10T08:58:46.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.420+0000 7f0d1ce27740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0d141aa740 con 0x55a787f00f40 2024-09-10T08:58:46.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.420+0000 7f0d1ce27740 10 client.0 _ll_register_callbacks cb 0x7f0cfc031ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:46.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.420+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f00f40 0x55a787f868a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:46.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.420+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f00f40 0x55a787f868a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:45110/0 (socket says 172.21.15.172:45110) 2024-09-10T08:58:46.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.420+0000 7f0d0bfff640 1 -- 172.21.15.172:0/4108891151 learned_addr learned my addr 172.21.15.172:0/4108891151 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:46.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.420+0000 7f0d13fff640 1 --2- 172.21.15.172:0/4108891151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f01860 0x55a787f86de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:46.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.420+0000 7f0d0bfff640 1 -- 172.21.15.172:0/4108891151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f0e450 msgr2=0x55a787f1f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:46.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.420+0000 7f0d0bfff640 1 --2- 172.21.15.172:0/4108891151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f0e450 0x55a787f1f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.420+0000 7f0d0bfff640 1 -- 172.21.15.172:0/4108891151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f01860 msgr2=0x55a787f86de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:46.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.420+0000 7f0d0bfff640 1 --2- 172.21.15.172:0/4108891151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f01860 0x55a787f86de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.421+0000 7f0d0bfff640 1 -- 172.21.15.172:0/4108891151 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a787f1fa80 con 0x55a787f00f40 2024-09-10T08:58:46.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.421+0000 7f0d13fff640 1 --2- 172.21.15.172:0/4108891151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f01860 0x55a787f86de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:46.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.421+0000 7f0d0bfff640 1 --2- 172.21.15.172:0/4108891151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f00f40 0x55a787f868a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc02ac00 tx=0x7f0cfc0045f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd887c70212459a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:46.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.421+0000 7f0cf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:58:46.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.421+0000 7f0cf37fe640 1 -- 172.21.15.172:0/4108891151 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc044070 con 0x55a787f00f40 2024-09-10T08:58:46.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.421+0000 7f0d1ce27740 1 -- 172.21.15.172:0/4108891151 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cfc051020 con 0x55a787f00f40 2024-09-10T08:58:46.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.421+0000 7f0cf37fe640 1 -- 172.21.15.172:0/4108891151 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0cfc031640 con 0x55a787f00f40 2024-09-10T08:58:46.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.421+0000 7f0cf37fe640 1 -- 172.21.15.172:0/4108891151 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0cfc035040 con 0x55a787f00f40 2024-09-10T08:58:46.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.421+0000 7f0d1ce27740 10 client.27880 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:46.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.421+0000 7f0d1ce27740 1 -- 172.21.15.172:0/4108891151 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a787f8f030 con 0x55a787f00f40 2024-09-10T08:58:46.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.421+0000 7f0cf37fe640 1 -- 172.21.15.172:0/4108891151 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0cfc051020 con 0x55a787f00f40 2024-09-10T08:58:46.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.421+0000 7f0cf37fe640 1 -- 172.21.15.172:0/4108891151 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f0cfc05e020 con 0x55a787f00f40 2024-09-10T08:58:46.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.422+0000 7f0d1ce27740 10 client.27880 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:46.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.422+0000 7f0d1ce27740 10 client.27880 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T08:58:46.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.422+0000 7f0d1ce27740 20 client.27880 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:46.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.422+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/4108891151 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a787f8f720 0x55a787f91b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:46.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.422+0000 7f0d1ce27740 4 client.27880 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4766dcba", "release": true, "await": true}] 2024-09-10T08:58:46.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.422+0000 7f0d1ce27740 1 -- 172.21.15.172:0/4108891151 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4766dcba", "release": true, "await": true}) -- 0x55a787e6e0a0 con 0x55a787f8f720 2024-09-10T08:58:46.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.422+0000 7f0d13fff640 1 --2- 172.21.15.172:0/4108891151 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a787f8f720 0x55a787f91b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:46.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.422+0000 7f0cf37fe640 10 client.27880 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:46.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.422+0000 7f0d13fff640 1 --2- 172.21.15.172:0/4108891151 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a787f8f720 0x55a787f91b40 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c2f9fbee4843ef5 server_cookie=e5e1ae8a6f911179 in_seq=0 out_seq=0 2024-09-10T08:58:46.432 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:58:46.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.438+0000 7f0cf37fe640 1 -- 172.21.15.172:0/4108891151 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a787e6e0a0 con 0x55a787f8f720 2024-09-10T08:58:46.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.438+0000 7f0cf37fe640 10 client.27880 handle_command_reply: tid=0 2024-09-10T08:58:46.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.438+0000 7f0d1ce27740 1 client.27880 shutdown 2024-09-10T08:58:46.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.438+0000 7f0d1ce27740 1 -- 172.21.15.172:0/4108891151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f00f40 msgr2=0x55a787f868a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:46.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.438+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/4108891151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f00f40 0x55a787f868a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc02ac00 tx=0x7f0cfc0045f0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.438+0000 7f0d1ce27740 1 -- 172.21.15.172:0/4108891151 shutdown_connections 2024-09-10T08:58:46.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.438+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/4108891151 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a787f8f720 0x55a787f91b40 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.438+0000 7f0cf37fe640 0 client.27880 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:46.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.438+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/4108891151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a787f0e450 0x55a787f1f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.438+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/4108891151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a787f01860 0x55a787f86de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.438+0000 7f0d1ce27740 1 --2- 172.21.15.172:0/4108891151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a787f00f40 0x55a787f868a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.438+0000 7f0d1ce27740 1 -- 172.21.15.172:0/4108891151 >> 172.21.15.172:0/4108891151 conn(0x55a787efd5f0 msgr2=0x55a787f0b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:46.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.438+0000 7f0d1ce27740 1 -- 172.21.15.172:0/4108891151 shutdown_connections 2024-09-10T08:58:46.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.438+0000 7f0d1ce27740 1 -- 172.21.15.172:0/4108891151 wait complete. 2024-09-10T08:58:46.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.439+0000 7f0d1ce27740 20 client.27880 trim_cache size 0 max 16384 2024-09-10T08:58:46.441 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:58:46.441 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 40, 2024-09-10T08:58:46.441 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T08:58:46.441 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 44, 2024-09-10T08:58:46.441 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T08:58:46.441 INFO:teuthology.orchestra.run.smithi172.stdout: "4766dcba": { 2024-09-10T08:58:46.441 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 44, 2024-09-10T08:58:46.441 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T08:58:46.441 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:58:46.441 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:58:46.442 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:58:46.442 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T08:58:46.442 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T08:58:46.442 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 138.1, 2024-09-10T08:58:46.442 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T08:58:46.442 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T08:58:46.442 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T08:58:46.442 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T08:58:46.442 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T08:58:46.442 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T08:58:46.442 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:58:46.443 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:58:46.443 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:58:46.443 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:58:46.443 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T08:58:46.443 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:58:46.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.439+0000 7f0d1aa4e640 1 -- 172.21.15.172:0/3241527548 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0d000a2c80 msgr2=0x7f0d000a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:46.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.439+0000 7f0d1aa4e640 1 --2- 172.21.15.172:0/3241527548 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0d000a2c80 0x7f0d000a5140 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c09b830 tx=0x7f0d0c00a890 comp rx=0 tx=0).stop 2024-09-10T08:58:46.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.439+0000 7f0d1aa4e640 1 -- 172.21.15.172:0/3241527548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d14102ef0 msgr2=0x7f0d1419bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:46.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.439+0000 7f0d1aa4e640 1 --2- 172.21.15.172:0/3241527548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d14102ef0 0x7f0d1419bb80 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00cad0 tx=0x7f0d0c00cf90 comp rx=0 tx=0).stop 2024-09-10T08:58:46.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.439+0000 7f0d1aa4e640 1 -- 172.21.15.172:0/3241527548 shutdown_connections 2024-09-10T08:58:46.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.439+0000 7f0d1aa4e640 1 --2- 172.21.15.172:0/3241527548 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0d000a2c80 0x7f0d000a5140 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.439+0000 7f0d1aa4e640 1 --2- 172.21.15.172:0/3241527548 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d14102ef0 0x7f0d1419bb80 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.439+0000 7f0d1aa4e640 1 --2- 172.21.15.172:0/3241527548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d14102570 0x7f0d141977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.439+0000 7f0d1aa4e640 1 --2- 172.21.15.172:0/3241527548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d14100e20 0x7f0d141972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:46.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.439+0000 7f0d1aa4e640 1 -- 172.21.15.172:0/3241527548 >> 172.21.15.172:0/3241527548 conn(0x7f0d140fc7d0 msgr2=0x7f0d140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:46.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.439+0000 7f0d1aa4e640 1 -- 172.21.15.172:0/3241527548 shutdown_connections 2024-09-10T08:58:46.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:46.440+0000 7f0d1aa4e640 1 -- 172.21.15.172:0/3241527548 wait complete. 2024-09-10T08:58:47.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.159+0000 7f3824dbb640 1 -- 172.21.15.172:0/1617073872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3820106f20 msgr2=0x7f382010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:47.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.159+0000 7f3824dbb640 1 --2- 172.21.15.172:0/1617073872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3820106f20 0x7f382010eaa0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f3818009ca0 tx=0x7f381802a8c0 comp rx=0 tx=0).stop 2024-09-10T08:58:47.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.159+0000 7f3824dbb640 1 -- 172.21.15.172:0/1617073872 shutdown_connections 2024-09-10T08:58:47.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.159+0000 7f3824dbb640 1 --2- 172.21.15.172:0/1617073872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3820106f20 0x7f382010eaa0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:47.160 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.159+0000 7f3824dbb640 1 --2- 172.21.15.172:0/1617073872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38201027a0 0x7f38201067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:47.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.159+0000 7f3824dbb640 1 --2- 172.21.15.172:0/1617073872 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3820101df0 0x7f38201021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:47.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.159+0000 7f3824dbb640 1 -- 172.21.15.172:0/1617073872 >> 172.21.15.172:0/1617073872 conn(0x7f38200fd8f0 msgr2=0x7f38200ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:47.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.159+0000 7f3824dbb640 1 -- 172.21.15.172:0/1617073872 shutdown_connections 2024-09-10T08:58:47.161 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.159+0000 7f3824dbb640 1 -- 172.21.15.172:0/1617073872 wait complete. 2024-09-10T08:58:47.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.160+0000 7f3824dbb640 1 Processor -- start 2024-09-10T08:58:47.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.160+0000 7f3824dbb640 1 -- start start 2024-09-10T08:58:47.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.160+0000 7f3824dbb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3820101df0 0x7f38201961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:47.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.160+0000 7f3824dbb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38201027a0 0x7f38201966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:47.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.160+0000 7f3824dbb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3820106f20 0x7f382019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:47.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.160+0000 7f3824dbb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f382010fc40 con 0x7f3820106f20 2024-09-10T08:58:47.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.160+0000 7f3824dbb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f382010fac0 con 0x7f3820101df0 2024-09-10T08:58:47.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.160+0000 7f3824dbb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f382010fdc0 con 0x7f38201027a0 2024-09-10T08:58:47.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.160+0000 7f381ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3820106f20 0x7f382019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:47.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.160+0000 7f381ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3820106f20 0x7f382019aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49432/0 (socket says 172.21.15.172:49432) 2024-09-10T08:58:47.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.160+0000 7f381ed9d640 1 -- 172.21.15.172:0/1190821082 learned_addr learned my addr 172.21.15.172:0/1190821082 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:47.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.160+0000 7f381e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3820101df0 0x7f38201961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:47.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.160+0000 7f381dd9b640 1 --2- 172.21.15.172:0/1190821082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38201027a0 0x7f38201966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:47.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.161+0000 7f381e59c640 1 -- 172.21.15.172:0/1190821082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38201027a0 msgr2=0x7f38201966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:47.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.161+0000 7f381e59c640 1 --2- 172.21.15.172:0/1190821082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38201027a0 0x7f38201966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:47.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.161+0000 7f381e59c640 1 -- 172.21.15.172:0/1190821082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3820106f20 msgr2=0x7f382019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:47.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.161+0000 7f381e59c640 1 --2- 172.21.15.172:0/1190821082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3820106f20 0x7f382019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:47.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.161+0000 7f381e59c640 1 -- 172.21.15.172:0/1190821082 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382019b150 con 0x7f3820101df0 2024-09-10T08:58:47.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.161+0000 7f381e59c640 1 --2- 172.21.15.172:0/1190821082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3820101df0 0x7f38201961a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3810011ae0 tx=0x7f381000bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:47.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.161+0000 7f380f7fe640 1 -- 172.21.15.172:0/1190821082 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381000fb00 con 0x7f3820101df0 2024-09-10T08:58:47.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.161+0000 7f380f7fe640 1 -- 172.21.15.172:0/1190821082 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38100054d0 con 0x7f3820101df0 2024-09-10T08:58:47.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.161+0000 7f3824dbb640 1 -- 172.21.15.172:0/1190821082 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f382019b440 con 0x7f3820101df0 2024-09-10T08:58:47.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.161+0000 7f380f7fe640 1 -- 172.21.15.172:0/1190821082 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810019400 con 0x7f3820101df0 2024-09-10T08:58:47.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.161+0000 7f3824dbb640 1 -- 172.21.15.172:0/1190821082 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38201a2d20 con 0x7f3820101df0 2024-09-10T08:58:47.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.162+0000 7f3824dbb640 1 -- 172.21.15.172:0/1190821082 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38201021d0 con 0x7f3820101df0 2024-09-10T08:58:47.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.164+0000 7f380f7fe640 1 -- 172.21.15.172:0/1190821082 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3810004f80 con 0x7f3820101df0 2024-09-10T08:58:47.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.164+0000 7f380f7fe640 1 --2- 172.21.15.172:0/1190821082 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38080a2b80 0x7f38080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:47.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.164+0000 7f381dd9b640 1 --2- 172.21.15.172:0/1190821082 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38080a2b80 0x7f38080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:47.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.164+0000 7f380f7fe640 1 -- 172.21.15.172:0/1190821082 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f38100d3050 con 0x7f3820101df0 2024-09-10T08:58:47.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.165+0000 7f381dd9b640 1 --2- 172.21.15.172:0/1190821082 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38080a2b80 0x7f38080a5040 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f38200fdda0 tx=0x7f3814005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:47.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.168+0000 7f380f7fe640 1 -- 172.21.15.172:0/1190821082 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3810014070 con 0x7f3820101df0 2024-09-10T08:58:47.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.414+0000 7f3824dbb640 1 -- 172.21.15.172:0/1190821082 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3820197500 con 0x7f3820101df0 2024-09-10T08:58:47.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.417+0000 7f380f7fe640 1 -- 172.21.15.172:0/1190821082 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+12006 (secure 0 0 0) 0x7f381009f600 con 0x7f3820101df0 2024-09-10T08:58:47.418 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:58:47.419 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":40,"btime":"2024-09-10T08:58:02:716535+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:58:02.666404+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T08:58:47.419 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 40 2024-09-10T08:58:47.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.420+0000 7f3824dbb640 1 -- 172.21.15.172:0/1190821082 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38080a2b80 msgr2=0x7f38080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:47.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.420+0000 7f3824dbb640 1 --2- 172.21.15.172:0/1190821082 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38080a2b80 0x7f38080a5040 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f38200fdda0 tx=0x7f3814005f90 comp rx=0 tx=0).stop 2024-09-10T08:58:47.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.420+0000 7f3824dbb640 1 -- 172.21.15.172:0/1190821082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3820101df0 msgr2=0x7f38201961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:47.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.420+0000 7f3824dbb640 1 --2- 172.21.15.172:0/1190821082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3820101df0 0x7f38201961a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3810011ae0 tx=0x7f381000bed0 comp rx=0 tx=0).stop 2024-09-10T08:58:47.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.420+0000 7f3824dbb640 1 -- 172.21.15.172:0/1190821082 shutdown_connections 2024-09-10T08:58:47.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.420+0000 7f3824dbb640 1 --2- 172.21.15.172:0/1190821082 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f38080a2b80 0x7f38080a5040 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:47.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.420+0000 7f3824dbb640 1 --2- 172.21.15.172:0/1190821082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3820106f20 0x7f382019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:47.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.420+0000 7f3824dbb640 1 --2- 172.21.15.172:0/1190821082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38201027a0 0x7f38201966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:47.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.420+0000 7f3824dbb640 1 --2- 172.21.15.172:0/1190821082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3820101df0 0x7f38201961a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:47.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.420+0000 7f3824dbb640 1 -- 172.21.15.172:0/1190821082 >> 172.21.15.172:0/1190821082 conn(0x7f38200fd8f0 msgr2=0x7f38200fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:47.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.420+0000 7f3824dbb640 1 -- 172.21.15.172:0/1190821082 shutdown_connections 2024-09-10T08:58:47.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:47.420+0000 7f3824dbb640 1 -- 172.21.15.172:0/1190821082 wait complete. 2024-09-10T08:58:47.636 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4766dcba', total seconds elapsed: 23.0 2024-09-10T08:58:47.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:47 smithi174 ceph-mon[34068]: pgmap v727: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 516 KiB/s wr, 12 op/s 2024-09-10T08:58:47.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:47 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1190821082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:58:47.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:47 smithi191 ceph-mon[35314]: pgmap v727: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 516 KiB/s wr, 12 op/s 2024-09-10T08:58:47.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:47 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1190821082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:58:47.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:47 smithi172 ceph-mon[30854]: pgmap v727: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 516 KiB/s wr, 12 op/s 2024-09-10T08:58:47.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:47 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1190821082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:58:48.329 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T08:58:48.608 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:58:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.708+0000 7f5de7778640 1 -- 172.21.15.172:0/3591442841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5de0103150 msgr2=0x7f5de0103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.708+0000 7f5de7778640 1 --2- 172.21.15.172:0/3591442841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5de0103150 0x7f5de0103550 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009930 tx=0x7f5ddc02a420 comp rx=0 tx=0).stop 2024-09-10T08:58:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.708+0000 7f5de7778640 1 -- 172.21.15.172:0/3591442841 shutdown_connections 2024-09-10T08:58:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.708+0000 7f5de7778640 1 --2- 172.21.15.172:0/3591442841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5de0100580 0x7f5de010a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.708+0000 7f5de7778640 1 --2- 172.21.15.172:0/3591442841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5de00ffbc0 0x7f5de0100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.708+0000 7f5de7778640 1 --2- 172.21.15.172:0/3591442841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5de0103150 0x7f5de0103550 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.708+0000 7f5de7778640 1 -- 172.21.15.172:0/3591442841 >> 172.21.15.172:0/3591442841 conn(0x7f5de00fb930 msgr2=0x7f5de00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.708+0000 7f5de7778640 1 -- 172.21.15.172:0/3591442841 shutdown_connections 2024-09-10T08:58:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.708+0000 7f5de7778640 1 -- 172.21.15.172:0/3591442841 wait complete. 2024-09-10T08:58:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.709+0000 7f5de7778640 1 Processor -- start 2024-09-10T08:58:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.709+0000 7f5de7778640 1 -- start start 2024-09-10T08:58:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.709+0000 7f5de7778640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5de00ffbc0 0x7f5de0197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.709+0000 7f5de7778640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5de0100580 0x7f5de0198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.709+0000 7f5de7778640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5de0103150 0x7f5de019f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.709+0000 7f5de7778640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5de010d6a0 con 0x7f5de0100580 2024-09-10T08:58:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.709+0000 7f5de7778640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5de010d520 con 0x7f5de00ffbc0 2024-09-10T08:58:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.709+0000 7f5de7778640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5de010d820 con 0x7f5de0103150 2024-09-10T08:58:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.709+0000 7f5de4d13640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5de0100580 0x7f5de0198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:48.712 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.709+0000 7f5de4d13640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5de0100580 0x7f5de0198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49438/0 (socket says 172.21.15.172:49438) 2024-09-10T08:58:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.709+0000 7f5de4d13640 1 -- 172.21.15.172:0/4000424684 learned_addr learned my addr 172.21.15.172:0/4000424684 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.709+0000 7f5de5514640 1 --2- 172.21.15.172:0/4000424684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5de00ffbc0 0x7f5de0197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.709+0000 7f5de5d15640 1 --2- 172.21.15.172:0/4000424684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5de0103150 0x7f5de019f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.709+0000 7f5de4d13640 1 -- 172.21.15.172:0/4000424684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5de0103150 msgr2=0x7f5de019f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.710+0000 7f5de4d13640 1 --2- 172.21.15.172:0/4000424684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5de0103150 0x7f5de019f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.710+0000 7f5de4d13640 1 -- 172.21.15.172:0/4000424684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5de00ffbc0 msgr2=0x7f5de0197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.710+0000 7f5de4d13640 1 --2- 172.21.15.172:0/4000424684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5de00ffbc0 0x7f5de0197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.710+0000 7f5de4d13640 1 -- 172.21.15.172:0/4000424684 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de019f970 con 0x7f5de0100580 2024-09-10T08:58:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.710+0000 7f5de4d13640 1 --2- 172.21.15.172:0/4000424684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5de0100580 0x7f5de0198290 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000d950 tx=0x7f5dd000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.710+0000 7f5dd67fc640 1 -- 172.21.15.172:0/4000424684 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0017070 con 0x7f5de0100580 2024-09-10T08:58:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.710+0000 7f5dd67fc640 1 -- 172.21.15.172:0/4000424684 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dd0009c30 con 0x7f5de0100580 2024-09-10T08:58:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.710+0000 7f5de7778640 1 -- 172.21.15.172:0/4000424684 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de019fc60 con 0x7f5de0100580 2024-09-10T08:58:48.713 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.710+0000 7f5dd67fc640 1 -- 172.21.15.172:0/4000424684 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd000f620 con 0x7f5de0100580 2024-09-10T08:58:48.714 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.710+0000 7f5de7778640 1 -- 172.21.15.172:0/4000424684 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de01a01b0 con 0x7f5de0100580 2024-09-10T08:58:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.713+0000 7f5dea5ee740 1 Processor -- start 2024-09-10T08:58:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.713+0000 7f5dea5ee740 1 -- start start 2024-09-10T08:58:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.713+0000 7f5dea5ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab3a9a0 0x563edab3ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.713+0000 7f5dea5ee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab3b2e0 0x563edab3fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.713+0000 7f5dea5ee740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab40370 0x563edab42830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.713+0000 7f5dea5ee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563edaa9e2f0 con 0x563edab3b2e0 2024-09-10T08:58:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.713+0000 7f5dea5ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563edaa46270 con 0x563edab3a9a0 2024-09-10T08:58:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.713+0000 7f5dea5ee740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563edaa9a2b0 con 0x563edab40370 2024-09-10T08:58:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.713+0000 7f5dd67fc640 1 -- 172.21.15.172:0/4000424684 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5dd00109a0 con 0x7f5de0100580 2024-09-10T08:58:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.713+0000 7f5de5514640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab3b2e0 0x563edab3fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.713+0000 7f5de5514640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab3b2e0 0x563edab3fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49444/0 (socket says 172.21.15.172:49444) 2024-09-10T08:58:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.713+0000 7f5de5514640 1 -- 172.21.15.172:0/1453701274 learned_addr learned my addr 172.21.15.172:0/1453701274 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.713+0000 7f5de5d15640 1 --2- 172.21.15.172:0/1453701274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab3a9a0 0x563edab3ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.714+0000 7f5de4d13640 1 --2- 172.21.15.172:0/1453701274 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab40370 0x563edab42830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.714+0000 7f5de5514640 1 -- 172.21.15.172:0/1453701274 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab40370 msgr2=0x563edab42830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.714+0000 7f5de5514640 1 --2- 172.21.15.172:0/1453701274 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab40370 0x563edab42830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.714+0000 7f5de5514640 1 -- 172.21.15.172:0/1453701274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab3a9a0 msgr2=0x563edab3ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.714+0000 7f5de5514640 1 --2- 172.21.15.172:0/1453701274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab3a9a0 0x563edab3ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.720 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.714+0000 7f5de5514640 1 -- 172.21.15.172:0/1453701274 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563edaa45340 con 0x563edab3b2e0 2024-09-10T08:58:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.714+0000 7f5de4d13640 1 --2- 172.21.15.172:0/1453701274 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab40370 0x563edab42830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.714+0000 7f5de5d15640 1 --2- 172.21.15.172:0/1453701274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab3a9a0 0x563edab3ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.714+0000 7f5de5514640 1 --2- 172.21.15.172:0/1453701274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab3b2e0 0x563edab3fda0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009930 tx=0x7f5ddc005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436a1aef487561d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.714+0000 7f5dcb7fe640 1 -- 172.21.15.172:0/1453701274 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc035030 con 0x563edab3b2e0 2024-09-10T08:58:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.714+0000 7f5dcb7fe640 1 -- 172.21.15.172:0/1453701274 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ddc040070 con 0x563edab3b2e0 2024-09-10T08:58:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.714+0000 7f5dcb7fe640 1 -- 172.21.15.172:0/1453701274 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc003b30 con 0x563edab3b2e0 2024-09-10T08:58:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.714+0000 7f5dd67fc640 1 --2- 172.21.15.172:0/4000424684 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5dcc0a2c30 0x7f5dcc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.714+0000 7f5de5d15640 1 --2- 172.21.15.172:0/4000424684 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5dcc0a2c30 0x7f5dcc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.715+0000 7f5dd67fc640 1 -- 172.21.15.172:0/4000424684 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5dd00cef20 con 0x7f5de0100580 2024-09-10T08:58:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.715+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1453701274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab3b2e0 msgr2=0x563edab3fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.715+0000 7f5de5d15640 1 --2- 172.21.15.172:0/4000424684 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5dcc0a2c30 0x7f5dcc0a50f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8005fd0 tx=0x7f5dd8009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.715+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/1453701274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab3b2e0 0x563edab3fda0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009930 tx=0x7f5ddc005c00 comp rx=0 tx=0).stop 2024-09-10T08:58:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.715+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1453701274 shutdown_connections 2024-09-10T08:58:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.715+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/1453701274 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab40370 0x563edab42830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.715+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/1453701274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab3b2e0 0x563edab3fda0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.715+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/1453701274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab3a9a0 0x563edab3ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.715+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1453701274 >> 172.21.15.172:0/1453701274 conn(0x563edab36010 msgr2=0x563edab36410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.715+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1453701274 shutdown_connections 2024-09-10T08:58:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.715+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1453701274 wait complete. 2024-09-10T08:58:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.716+0000 7f5dea5ee740 1 -- start start 2024-09-10T08:58:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.716+0000 7f5dea5ee740 1 Processor -- start 2024-09-10T08:58:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.716+0000 7f5dea5ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab3a9a0 0x563edab44b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.716+0000 7f5dea5ee740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab3b2e0 0x563edab45050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.716+0000 7f5dea5ee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab40370 0x563edab4c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.716+0000 7f5dea5ee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563edab432e0 con 0x563edab40370 2024-09-10T08:58:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.716+0000 7f5dea5ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563edaa27290 con 0x563edab3a9a0 2024-09-10T08:58:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.716+0000 7f5dea5ee740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563edab43460 con 0x563edab3b2e0 2024-09-10T08:58:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.716+0000 7f5dea5ee740 10 client.0 _ll_register_callbacks cb 0x563edaa58140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.716+0000 7f5de5d15640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab40370 0x563edab4c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.716+0000 7f5de5d15640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab40370 0x563edab4c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49450/0 (socket says 172.21.15.172:49450) 2024-09-10T08:58:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.716+0000 7f5de5d15640 1 -- 172.21.15.172:0/1789346340 learned_addr learned my addr 172.21.15.172:0/1789346340 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.716+0000 7f5de5514640 1 --2- 172.21.15.172:0/1789346340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab3a9a0 0x563edab44b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.716+0000 7f5de4d13640 1 --2- 172.21.15.172:0/1789346340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab3b2e0 0x563edab45050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.717+0000 7f5de5d15640 1 -- 172.21.15.172:0/1789346340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab3b2e0 msgr2=0x563edab45050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.717+0000 7f5de5d15640 1 --2- 172.21.15.172:0/1789346340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab3b2e0 0x563edab45050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.717+0000 7f5de5d15640 1 -- 172.21.15.172:0/1789346340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab3a9a0 msgr2=0x563edab44b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.717+0000 7f5de5d15640 1 --2- 172.21.15.172:0/1789346340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab3a9a0 0x563edab44b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.717+0000 7f5de5d15640 1 -- 172.21.15.172:0/1789346340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eda7b1210 con 0x563edab40370 2024-09-10T08:58:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.717+0000 7f5de5d15640 1 --2- 172.21.15.172:0/1789346340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab40370 0x563edab4c090 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009b40 tx=0x7f5dd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc5cc7092cdbb84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.717+0000 7f5dc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.717+0000 7f5dc8ff9640 1 -- 172.21.15.172:0/1789346340 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8020620 con 0x563edab40370 2024-09-10T08:58:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.717+0000 7f5dc8ff9640 1 -- 172.21.15.172:0/1789346340 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5dd80207c0 con 0x563edab40370 2024-09-10T08:58:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.717+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1789346340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ddc040070 con 0x563edab40370 2024-09-10T08:58:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.717+0000 7f5dc8ff9640 1 -- 172.21.15.172:0/1789346340 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5dd8033050 con 0x563edab40370 2024-09-10T08:58:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.717+0000 7f5dea5ee740 10 client.19155 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.717+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1789346340 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563eda87d620 con 0x563edab40370 2024-09-10T08:58:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.717+0000 7f5dc8ff9640 1 -- 172.21.15.172:0/1789346340 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ddc040070 con 0x563edab40370 2024-09-10T08:58:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.717+0000 7f5dc8ff9640 1 -- 172.21.15.172:0/1789346340 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f5dd800f040 con 0x563edab40370 2024-09-10T08:58:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.718+0000 7f5dea5ee740 10 client.19155 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.718+0000 7f5dea5ee740 10 client.19155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:58:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.718+0000 7f5dea5ee740 20 client.19155 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.718+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/1789346340 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563edab508e0 0x563edab52cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.718+0000 7f5dea5ee740 4 client.19155 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:58:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.718+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1789346340 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563edaaaba20 con 0x563edab508e0 2024-09-10T08:58:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.718+0000 7f5de5514640 1 --2- 172.21.15.172:0/1789346340 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563edab508e0 0x563edab52cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.718+0000 7f5de5514640 1 --2- 172.21.15.172:0/1789346340 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563edab508e0 0x563edab52cd0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=844213350aefff87 server_cookie=7438da0e30063327 in_seq=0 out_seq=0 2024-09-10T08:58:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.718+0000 7f5dc8ff9640 10 client.19155 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.719+0000 7f5dc8ff9640 1 -- 172.21.15.172:0/1789346340 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563edaaaba20 con 0x563edab508e0 2024-09-10T08:58:48.726 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.719+0000 7f5dc8ff9640 10 client.19155 handle_command_reply: tid=0 2024-09-10T08:58:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.719+0000 7f5dea5ee740 1 client.19155 shutdown 2024-09-10T08:58:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.719+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1789346340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab40370 msgr2=0x563edab4c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.719+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/1789346340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab40370 0x563edab4c090 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009b40 tx=0x7f5dd8004a40 comp rx=0 tx=0).stop 2024-09-10T08:58:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.719+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1789346340 shutdown_connections 2024-09-10T08:58:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.719+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/1789346340 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563edab508e0 0x563edab52cd0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.719+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/1789346340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab40370 0x563edab4c090 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.719+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/1789346340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab3b2e0 0x563edab45050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.719+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/1789346340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab3a9a0 0x563edab44b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.719+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1789346340 >> 172.21.15.172:0/1789346340 conn(0x563edab36010 msgr2=0x563edab37820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.719+0000 7f5dc8ff9640 0 client.19155 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.719+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1789346340 shutdown_connections 2024-09-10T08:58:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.719+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1789346340 wait complete. 2024-09-10T08:58:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.719+0000 7f5dea5ee740 20 client.19155 trim_cache size 0 max 16384 2024-09-10T08:58:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.735+0000 7f5dea5ee740 1 Processor -- start 2024-09-10T08:58:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.736+0000 7f5dea5ee740 1 -- start start 2024-09-10T08:58:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.736+0000 7f5dea5ee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab3a9a0 0x563edab349a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.736+0000 7f5dea5ee740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab3b2e0 0x563edab4c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.736+0000 7f5dea5ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab40370 0x563edab34ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.736+0000 7f5dea5ee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01aa5e0 con 0x563edab3a9a0 2024-09-10T08:58:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.736+0000 7f5dea5ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01ac1b0 con 0x563edab40370 2024-09-10T08:58:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.736+0000 7f5dea5ee740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01aacb0 con 0x563edab3b2e0 2024-09-10T08:58:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.736+0000 7f5de5514640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab3a9a0 0x563edab349a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.736+0000 7f5de5514640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab3a9a0 0x563edab349a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49454/0 (socket says 172.21.15.172:49454) 2024-09-10T08:58:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.736+0000 7f5de5514640 1 -- 172.21.15.172:0/1882558789 learned_addr learned my addr 172.21.15.172:0/1882558789 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.736+0000 7f5de4d13640 1 --2- 172.21.15.172:0/1882558789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab3b2e0 0x563edab4c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.736+0000 7f5de5514640 1 -- 172.21.15.172:0/1882558789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab3b2e0 msgr2=0x563edab4c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.736+0000 7f5de5514640 1 --2- 172.21.15.172:0/1882558789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab3b2e0 0x563edab4c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.736+0000 7f5de5514640 1 -- 172.21.15.172:0/1882558789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab40370 msgr2=0x563edab34ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.736+0000 7f5de5514640 1 --2- 172.21.15.172:0/1882558789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab40370 0x563edab34ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.736+0000 7f5de5514640 1 -- 172.21.15.172:0/1882558789 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563edab4f0c0 con 0x563edab3a9a0 2024-09-10T08:58:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.737+0000 7f5de5514640 1 --2- 172.21.15.172:0/1882558789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab3a9a0 0x563edab349a0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009900 tx=0x7f5ddc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9645aa7c16632187 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.737+0000 7f5dcaffd640 1 -- 172.21.15.172:0/1882558789 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc035040 con 0x563edab3a9a0 2024-09-10T08:58:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.737+0000 7f5dcaffd640 1 -- 172.21.15.172:0/1882558789 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ddc040040 con 0x563edab3a9a0 2024-09-10T08:58:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.737+0000 7f5dcaffd640 1 -- 172.21.15.172:0/1882558789 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc03c440 con 0x563edab3a9a0 2024-09-10T08:58:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.738+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1882558789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab3a9a0 msgr2=0x563edab349a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.738+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/1882558789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab3a9a0 0x563edab349a0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009900 tx=0x7f5ddc005fd0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.738+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1882558789 shutdown_connections 2024-09-10T08:58:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.738+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/1882558789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab40370 0x563edab34ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.738+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/1882558789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab3b2e0 0x563edab4c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.738+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/1882558789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab3a9a0 0x563edab349a0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.738+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1882558789 >> 172.21.15.172:0/1882558789 conn(0x563edab36010 msgr2=0x563edab528a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.738+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1882558789 shutdown_connections 2024-09-10T08:58:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.738+0000 7f5dea5ee740 1 -- 172.21.15.172:0/1882558789 wait complete. 2024-09-10T08:58:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.738+0000 7f5dea5ee740 1 -- start start 2024-09-10T08:58:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.738+0000 7f5dea5ee740 1 Processor -- start 2024-09-10T08:58:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5dea5ee740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab3a9a0 0x563edab475a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5dea5ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab3b2e0 0x563edab47ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5dea5ee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab40370 0x563edab4ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5dea5ee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01abae0 con 0x563edab40370 2024-09-10T08:58:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5dea5ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01ab410 con 0x563edab3b2e0 2024-09-10T08:58:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5dea5ee740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5de01a85c0 con 0x563edab3a9a0 2024-09-10T08:58:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5dea5ee740 10 client.0 _ll_register_callbacks cb 0x7f5dd8009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5de5d15640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab40370 0x563edab4ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5de5d15640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab40370 0x563edab4ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49458/0 (socket says 172.21.15.172:49458) 2024-09-10T08:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5de5d15640 1 -- 172.21.15.172:0/4250636249 learned_addr learned my addr 172.21.15.172:0/4250636249 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5de5514640 1 --2- 172.21.15.172:0/4250636249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab3a9a0 0x563edab475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5de4d13640 1 --2- 172.21.15.172:0/4250636249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab3b2e0 0x563edab47ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5de5514640 1 -- 172.21.15.172:0/4250636249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab3b2e0 msgr2=0x563edab47ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5de5514640 1 --2- 172.21.15.172:0/4250636249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab3b2e0 0x563edab47ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5de5514640 1 -- 172.21.15.172:0/4250636249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab40370 msgr2=0x563edab4ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5de5514640 1 --2- 172.21.15.172:0/4250636249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab40370 0x563edab4ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5de5514640 1 -- 172.21.15.172:0/4250636249 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563edab4b110 con 0x563edab3a9a0 2024-09-10T08:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.739+0000 7f5de5d15640 1 --2- 172.21.15.172:0/4250636249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab40370 0x563edab4ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.740+0000 7f5de5514640 1 --2- 172.21.15.172:0/4250636249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab3a9a0 0x563edab475a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009850 tx=0x7f5ddc005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb820ffee1cb7df3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.740+0000 7f5dc8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.740+0000 7f5dc8ff9640 1 -- 172.21.15.172:0/4250636249 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc035040 con 0x563edab3a9a0 2024-09-10T08:58:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.740+0000 7f5dea5ee740 1 -- 172.21.15.172:0/4250636249 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ddc040040 con 0x563edab3a9a0 2024-09-10T08:58:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.740+0000 7f5dc8ff9640 1 -- 172.21.15.172:0/4250636249 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5ddc031780 con 0x563edab3a9a0 2024-09-10T08:58:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.740+0000 7f5dc8ff9640 1 -- 172.21.15.172:0/4250636249 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5ddc069050 con 0x563edab3a9a0 2024-09-10T08:58:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.740+0000 7f5dea5ee740 10 client.27898 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.740+0000 7f5dea5ee740 1 -- 172.21.15.172:0/4250636249 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563edabbb4f0 con 0x563edab3a9a0 2024-09-10T08:58:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.740+0000 7f5dc8ff9640 1 -- 172.21.15.172:0/4250636249 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ddc040040 con 0x563edab3a9a0 2024-09-10T08:58:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.741+0000 7f5dc8ff9640 1 -- 172.21.15.172:0/4250636249 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f5ddc003660 con 0x563edab3a9a0 2024-09-10T08:58:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.741+0000 7f5dea5ee740 10 client.27898 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.741+0000 7f5dea5ee740 10 client.27898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:58:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.741+0000 7f5dea5ee740 20 client.27898 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.741+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/4250636249 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563edabbb8d0 0x563edabbdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.741+0000 7f5dea5ee740 4 client.27898 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:58:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.741+0000 7f5dea5ee740 1 -- 172.21.15.172:0/4250636249 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563edaaa5d70 con 0x563edabbb8d0 2024-09-10T08:58:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.741+0000 7f5de4d13640 1 --2- 172.21.15.172:0/4250636249 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563edabbb8d0 0x563edabbdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.741+0000 7f5dc8ff9640 10 client.27898 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.741+0000 7f5de4d13640 1 --2- 172.21.15.172:0/4250636249 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563edabbb8d0 0x563edabbdcc0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f14dc4050bcd0219 server_cookie=6ba227c3173b465 in_seq=0 out_seq=0 2024-09-10T08:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.753+0000 7f5dc8ff9640 1 -- 172.21.15.172:0/4250636249 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563edaaa5d70 con 0x563edabbb8d0 2024-09-10T08:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.753+0000 7f5dc8ff9640 10 client.27898 handle_command_reply: tid=0 2024-09-10T08:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.753+0000 7f5dea5ee740 1 client.27898 shutdown 2024-09-10T08:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.753+0000 7f5dea5ee740 1 -- 172.21.15.172:0/4250636249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab3a9a0 msgr2=0x563edab475a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.753+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/4250636249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab3a9a0 0x563edab475a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009850 tx=0x7f5ddc005850 comp rx=0 tx=0).stop 2024-09-10T08:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.753+0000 7f5dea5ee740 1 -- 172.21.15.172:0/4250636249 shutdown_connections 2024-09-10T08:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.753+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/4250636249 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563edabbb8d0 0x563edabbdcc0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.753+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/4250636249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563edab40370 0x563edab4ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.753+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/4250636249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563edab3b2e0 0x563edab47ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.753+0000 7f5dc8ff9640 0 client.27898 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:58:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.753+0000 7f5dea5ee740 1 --2- 172.21.15.172:0/4250636249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563edab3a9a0 0x563edab475a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.753+0000 7f5dea5ee740 1 -- 172.21.15.172:0/4250636249 >> 172.21.15.172:0/4250636249 conn(0x563edab36010 msgr2=0x563edab349a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.753+0000 7f5dea5ee740 1 -- 172.21.15.172:0/4250636249 shutdown_connections 2024-09-10T08:58:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.753+0000 7f5dea5ee740 1 -- 172.21.15.172:0/4250636249 wait complete. 2024-09-10T08:58:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.754+0000 7f5dea5ee740 20 client.27898 trim_cache size 0 max 16384 2024-09-10T08:58:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:58:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:58:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:58:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:58:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.754+0000 7f5de7778640 1 -- 172.21.15.172:0/4000424684 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5dcc0a2c30 msgr2=0x7f5dcc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.754+0000 7f5de7778640 1 --2- 172.21.15.172:0/4000424684 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5dcc0a2c30 0x7f5dcc0a50f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8005fd0 tx=0x7f5dd8009600 comp rx=0 tx=0).stop 2024-09-10T08:58:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.754+0000 7f5de7778640 1 -- 172.21.15.172:0/4000424684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5de0100580 msgr2=0x7f5de0198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.754+0000 7f5de7778640 1 --2- 172.21.15.172:0/4000424684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5de0100580 0x7f5de0198290 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000d950 tx=0x7f5dd000de10 comp rx=0 tx=0).stop 2024-09-10T08:58:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.754+0000 7f5de7778640 1 -- 172.21.15.172:0/4000424684 shutdown_connections 2024-09-10T08:58:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.754+0000 7f5de7778640 1 --2- 172.21.15.172:0/4000424684 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5dcc0a2c30 0x7f5dcc0a50f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.754+0000 7f5de7778640 1 --2- 172.21.15.172:0/4000424684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5de0103150 0x7f5de019f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.754+0000 7f5de7778640 1 --2- 172.21.15.172:0/4000424684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5de0100580 0x7f5de0198290 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.754+0000 7f5de7778640 1 --2- 172.21.15.172:0/4000424684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5de00ffbc0 0x7f5de0197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.754+0000 7f5de7778640 1 -- 172.21.15.172:0/4000424684 >> 172.21.15.172:0/4000424684 conn(0x7f5de00fb930 msgr2=0x7f5de0106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.754+0000 7f5de7778640 1 -- 172.21.15.172:0/4000424684 shutdown_connections 2024-09-10T08:58:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:58:48.754+0000 7f5de7778640 1 -- 172.21.15.172:0/4000424684 wait complete. 2024-09-10T08:58:49.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.170+0000 7f51db916640 1 -- 172.21.15.172:0/1020582849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51d410c180 msgr2=0x7f51d410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:49.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.170+0000 7f51db916640 1 --2- 172.21.15.172:0/1020582849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51d410c180 0x7f51d410e600 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009ca0 tx=0x7f51cc02a950 comp rx=0 tx=0).stop 2024-09-10T08:58:49.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.170+0000 7f51db916640 1 -- 172.21.15.172:0/1020582849 shutdown_connections 2024-09-10T08:58:49.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.170+0000 7f51db916640 1 --2- 172.21.15.172:0/1020582849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51d410c180 0x7f51d410e600 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.170+0000 7f51db916640 1 --2- 172.21.15.172:0/1020582849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51d41036c0 0x7f51d410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.170+0000 7f51db916640 1 --2- 172.21.15.172:0/1020582849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51d4102da0 0x7f51d4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.170+0000 7f51db916640 1 -- 172.21.15.172:0/1020582849 >> 172.21.15.172:0/1020582849 conn(0x7f51d40fc780 msgr2=0x7f51d40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:49.198 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.170+0000 7f51db916640 1 -- 172.21.15.172:0/1020582849 shutdown_connections 2024-09-10T08:58:49.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.170+0000 7f51db916640 1 -- 172.21.15.172:0/1020582849 wait complete. 2024-09-10T08:58:49.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.171+0000 7f51db916640 1 Processor -- start 2024-09-10T08:58:49.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.171+0000 7f51db916640 1 -- start start 2024-09-10T08:58:49.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.171+0000 7f51db916640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51d4102da0 0x7f51d4100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.171+0000 7f51db916640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51d41036c0 0x7f51d4100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.171+0000 7f51db916640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51d410c180 0x7f51d4101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.172+0000 7f51db916640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4111e60 con 0x7f51d410c180 2024-09-10T08:58:49.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.172+0000 7f51db916640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4111ce0 con 0x7f51d41036c0 2024-09-10T08:58:49.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.172+0000 7f51db916640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4111fe0 con 0x7f51d4102da0 2024-09-10T08:58:49.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.172+0000 7f51d8eb1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51d41036c0 0x7f51d4100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.172+0000 7f51d8eb1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51d41036c0 0x7f51d4100f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:54462/0 (socket says 172.21.15.172:54462) 2024-09-10T08:58:49.199 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.172+0000 7f51d8eb1640 1 -- 172.21.15.172:0/4057276774 learned_addr learned my addr 172.21.15.172:0/4057276774 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:49.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.172+0000 7f51d9eb3640 1 --2- 172.21.15.172:0/4057276774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51d410c180 0x7f51d4101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.172+0000 7f51d96b2640 1 --2- 172.21.15.172:0/4057276774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51d4102da0 0x7f51d4100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.172+0000 7f51d9eb3640 1 -- 172.21.15.172:0/4057276774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51d4102da0 msgr2=0x7f51d4100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:49.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.172+0000 7f51d9eb3640 1 --2- 172.21.15.172:0/4057276774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51d4102da0 0x7f51d4100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.172+0000 7f51d9eb3640 1 -- 172.21.15.172:0/4057276774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51d41036c0 msgr2=0x7f51d4100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:49.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.172+0000 7f51d9eb3640 1 --2- 172.21.15.172:0/4057276774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51d41036c0 0x7f51d4100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.172+0000 7f51d9eb3640 1 -- 172.21.15.172:0/4057276774 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d41a68a0 con 0x7f51d410c180 2024-09-10T08:58:49.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.172+0000 7f51d9eb3640 1 --2- 172.21.15.172:0/4057276774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51d410c180 0x7f51d4101480 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f51cc005e30 tx=0x7f51cc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:49.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.173+0000 7f51ca7fc640 1 -- 172.21.15.172:0/4057276774 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc034040 con 0x7f51d410c180 2024-09-10T08:58:49.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.173+0000 7f51ca7fc640 1 -- 172.21.15.172:0/4057276774 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51cc030850 con 0x7f51d410c180 2024-09-10T08:58:49.200 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.173+0000 7f51db916640 1 -- 172.21.15.172:0/4057276774 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d41a6b30 con 0x7f51d410c180 2024-09-10T08:58:49.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.173+0000 7f51ca7fc640 1 -- 172.21.15.172:0/4057276774 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0046e0 con 0x7f51d410c180 2024-09-10T08:58:49.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.173+0000 7f51db916640 1 -- 172.21.15.172:0/4057276774 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d41a7010 con 0x7f51d410c180 2024-09-10T08:58:49.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.175+0000 7f51ca7fc640 1 -- 172.21.15.172:0/4057276774 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f51cc037020 con 0x7f51d410c180 2024-09-10T08:58:49.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.176+0000 7f51ddcef740 1 Processor -- start 2024-09-10T08:58:49.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.176+0000 7f51ddcef740 1 -- start start 2024-09-10T08:58:49.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.176+0000 7f51ddcef740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cef8b40 0x55d34cef7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.176+0000 7f51ddcef740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef75d0 0x55d34cef7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.176+0000 7f51ddcef740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cf02370 0x55d34cf04760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.176+0000 7f51ddcef740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d34ce5d2f0 con 0x55d34cef75d0 2024-09-10T08:58:49.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.176+0000 7f51ddcef740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d34ce04cf0 con 0x55d34cf02370 2024-09-10T08:58:49.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.176+0000 7f51ddcef740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d34ce592b0 con 0x55d34cef8b40 2024-09-10T08:58:49.201 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.176+0000 7f51ca7fc640 1 --2- 172.21.15.172:0/4057276774 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51c00a2b80 0x7f51c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.176+0000 7f51ca7fc640 1 -- 172.21.15.172:0/4057276774 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f51cc0f0220 con 0x7f51d410c180 2024-09-10T08:58:49.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.176+0000 7f51d9eb3640 1 --2- 172.21.15.172:0/4057276774 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51c00a2b80 0x7f51c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.177+0000 7f51d8eb1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cf02370 0x55d34cf04760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.177+0000 7f51d8eb1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cef8b40 0x55d34cef7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.177+0000 7f51d8eb1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cf02370 0x55d34cf04760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:54468/0 (socket says 172.21.15.172:54468) 2024-09-10T08:58:49.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.177+0000 7f51d8eb1640 1 -- 172.21.15.172:0/3075496871 learned_addr learned my addr 172.21.15.172:0/3075496871 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:49.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.177+0000 7f51d96b2640 1 --2- 172.21.15.172:0/3075496871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef75d0 0x55d34cef7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.177+0000 7f51d8eb1640 1 -- 172.21.15.172:0/3075496871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cf02370 msgr2=0x55d34cf04760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:49.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.177+0000 7f51d9eb3640 1 --2- 172.21.15.172:0/4057276774 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51c00a2b80 0x7f51c00a5040 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f51cc006010 tx=0x7f51cc031f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:49.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.177+0000 7f51d8eb1640 1 --2- 172.21.15.172:0/3075496871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cf02370 0x55d34cf04760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.177+0000 7f51d8eb1640 1 -- 172.21.15.172:0/3075496871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef75d0 msgr2=0x55d34cef7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:49.202 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.177+0000 7f51d8eb1640 1 --2- 172.21.15.172:0/3075496871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef75d0 0x55d34cef7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.177+0000 7f51d8eb1640 1 -- 172.21.15.172:0/3075496871 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d34cde93a0 con 0x55d34cef8b40 2024-09-10T08:58:49.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.177+0000 7f51d96b2640 1 --2- 172.21.15.172:0/3075496871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef75d0 0x55d34cef7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:49.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.177+0000 7f51d8eb1640 1 --2- 172.21.15.172:0/3075496871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cef8b40 0x55d34cef7090 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f51c40127d0 tx=0x7f51c400b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=becb84a44bb755f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:49.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.177+0000 7f51bf7fe640 1 -- 172.21.15.172:0/3075496871 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4012e10 con 0x55d34cef8b40 2024-09-10T08:58:49.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.178+0000 7f51bf7fe640 1 -- 172.21.15.172:0/3075496871 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51c4012470 con 0x55d34cef8b40 2024-09-10T08:58:49.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.178+0000 7f51bf7fe640 1 -- 172.21.15.172:0/3075496871 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4005650 con 0x55d34cef8b40 2024-09-10T08:58:49.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.178+0000 7f51ddcef740 1 -- 172.21.15.172:0/3075496871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cef8b40 msgr2=0x55d34cef7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:49.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.178+0000 7f51ddcef740 1 --2- 172.21.15.172:0/3075496871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cef8b40 0x55d34cef7090 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f51c40127d0 tx=0x7f51c400b480 comp rx=0 tx=0).stop 2024-09-10T08:58:49.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.178+0000 7f51ddcef740 1 -- 172.21.15.172:0/3075496871 shutdown_connections 2024-09-10T08:58:49.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.178+0000 7f51ddcef740 1 --2- 172.21.15.172:0/3075496871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cf02370 0x55d34cf04760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.178+0000 7f51ddcef740 1 --2- 172.21.15.172:0/3075496871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef75d0 0x55d34cef7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.203 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.178+0000 7f51ddcef740 1 --2- 172.21.15.172:0/3075496871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cef8b40 0x55d34cef7090 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.178+0000 7f51ddcef740 1 -- 172.21.15.172:0/3075496871 >> 172.21.15.172:0/3075496871 conn(0x55d34cec0310 msgr2=0x55d34cef3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:49.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.178+0000 7f51ddcef740 1 -- 172.21.15.172:0/3075496871 shutdown_connections 2024-09-10T08:58:49.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.178+0000 7f51ddcef740 1 -- 172.21.15.172:0/3075496871 wait complete. 2024-09-10T08:58:49.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.179+0000 7f51ddcef740 1 -- start start 2024-09-10T08:58:49.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.179+0000 7f51ddcef740 1 Processor -- start 2024-09-10T08:58:49.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.179+0000 7f51ddcef740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cef75d0 0x55d34cf01d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.179+0000 7f51ddcef740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cef8b40 0x55d34cefefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.179+0000 7f51ddcef740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cf02370 0x55d34ceff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.179+0000 7f51ddcef740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d34cf05070 con 0x55d34cf02370 2024-09-10T08:58:49.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.179+0000 7f51ddcef740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d34ce18060 con 0x55d34cef8b40 2024-09-10T08:58:49.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.179+0000 7f51ddcef740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d34cf051f0 con 0x55d34cef75d0 2024-09-10T08:58:49.204 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.179+0000 7f51ddcef740 10 client.0 _ll_register_callbacks cb 0x55d34cb6b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:49.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.179+0000 7f51d8eb1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cf02370 0x55d34ceff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.179+0000 7f51d8eb1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cf02370 0x55d34ceff5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49478/0 (socket says 172.21.15.172:49478) 2024-09-10T08:58:49.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.179+0000 7f51d8eb1640 1 -- 172.21.15.172:0/213353046 learned_addr learned my addr 172.21.15.172:0/213353046 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:49.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.180+0000 7f51d96b2640 1 --2- 172.21.15.172:0/213353046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cef8b40 0x55d34cefefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.180+0000 7f51d8eb1640 1 --2- 172.21.15.172:0/213353046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cef75d0 0x55d34cf01d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.180+0000 7f51d8eb1640 1 -- 172.21.15.172:0/213353046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cef75d0 msgr2=0x55d34cf01d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:49.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.180+0000 7f51d8eb1640 1 --2- 172.21.15.172:0/213353046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cef75d0 0x55d34cf01d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.180+0000 7f51d8eb1640 1 -- 172.21.15.172:0/213353046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cef8b40 msgr2=0x55d34cefefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:49.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.180+0000 7f51d8eb1640 1 --2- 172.21.15.172:0/213353046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cef8b40 0x55d34cefefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.180+0000 7f51d8eb1640 1 -- 172.21.15.172:0/213353046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d34cbce3c0 con 0x55d34cf02370 2024-09-10T08:58:49.205 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.180+0000 7f51d8eb1640 1 --2- 172.21.15.172:0/213353046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cf02370 0x55d34ceff5d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f51c4005fd0 tx=0x7f51c4005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc1674185bfb5d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:49.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.180+0000 7f51bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:49.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.180+0000 7f51bcff9640 1 -- 172.21.15.172:0/213353046 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c40052d0 con 0x55d34cf02370 2024-09-10T08:58:49.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.180+0000 7f51bcff9640 1 -- 172.21.15.172:0/213353046 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f51c401a040 con 0x55d34cf02370 2024-09-10T08:58:49.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.180+0000 7f51ddcef740 1 -- 172.21.15.172:0/213353046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51c4012470 con 0x55d34cf02370 2024-09-10T08:58:49.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.180+0000 7f51bcff9640 1 -- 172.21.15.172:0/213353046 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f51c4035050 con 0x55d34cf02370 2024-09-10T08:58:49.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.180+0000 7f51ddcef740 10 client.19173 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:49.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.180+0000 7f51ddcef740 1 -- 172.21.15.172:0/213353046 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d34cf10b00 con 0x55d34cf02370 2024-09-10T08:58:49.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.181+0000 7f51bcff9640 1 -- 172.21.15.172:0/213353046 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51c4012470 con 0x55d34cf02370 2024-09-10T08:58:49.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.181+0000 7f51bcff9640 1 -- 172.21.15.172:0/213353046 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f51c40046c0 con 0x55d34cf02370 2024-09-10T08:58:49.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.181+0000 7f51ddcef740 10 client.19173 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:49.206 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.181+0000 7f51ddcef740 10 client.19173 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:58:49.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.181+0000 7f51ddcef740 20 client.19173 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:49.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.181+0000 7f51ddcef740 1 --2- 172.21.15.172:0/213353046 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d34cf11e50 0x55d34cf14240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.181+0000 7f51ddcef740 4 client.19173 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:58:49.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.181+0000 7f51ddcef740 1 -- 172.21.15.172:0/213353046 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d34ce6a860 con 0x55d34cf11e50 2024-09-10T08:58:49.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.181+0000 7f51d96b2640 1 --2- 172.21.15.172:0/213353046 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d34cf11e50 0x55d34cf14240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.182+0000 7f51bcff9640 10 client.19173 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:58:49.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.182+0000 7f51d96b2640 1 --2- 172.21.15.172:0/213353046 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d34cf11e50 0x55d34cf14240 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b5ca83c02df85ecf server_cookie=7e389889726c3df6 in_seq=0 out_seq=0 2024-09-10T08:58:49.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.183+0000 7f51bcff9640 1 -- 172.21.15.172:0/213353046 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d34ce6a860 con 0x55d34cf11e50 2024-09-10T08:58:49.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.183+0000 7f51bcff9640 10 client.19173 handle_command_reply: tid=0 2024-09-10T08:58:49.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.183+0000 7f51ddcef740 1 client.19173 shutdown 2024-09-10T08:58:49.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.183+0000 7f51ddcef740 1 -- 172.21.15.172:0/213353046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cf02370 msgr2=0x55d34ceff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:49.207 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.183+0000 7f51ddcef740 1 --2- 172.21.15.172:0/213353046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cf02370 0x55d34ceff5d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f51c4005fd0 tx=0x7f51c4005550 comp rx=0 tx=0).stop 2024-09-10T08:58:49.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.183+0000 7f51ddcef740 1 -- 172.21.15.172:0/213353046 shutdown_connections 2024-09-10T08:58:49.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.183+0000 7f51ddcef740 1 --2- 172.21.15.172:0/213353046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cf02370 0x55d34ceff5d0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.183+0000 7f51ddcef740 1 --2- 172.21.15.172:0/213353046 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d34cf11e50 0x55d34cf14240 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.183+0000 7f51ddcef740 1 --2- 172.21.15.172:0/213353046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cef8b40 0x55d34cefefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.183+0000 7f51ddcef740 1 --2- 172.21.15.172:0/213353046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cef75d0 0x55d34cf01d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.183+0000 7f51ddcef740 1 -- 172.21.15.172:0/213353046 >> 172.21.15.172:0/213353046 conn(0x55d34cec0310 msgr2=0x55d34cef9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:49.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.183+0000 7f51bcff9640 0 client.19173 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:58:49.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.183+0000 7f51ddcef740 1 -- 172.21.15.172:0/213353046 shutdown_connections 2024-09-10T08:58:49.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.183+0000 7f51ddcef740 1 -- 172.21.15.172:0/213353046 wait complete. 2024-09-10T08:58:49.208 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.184+0000 7f51ddcef740 20 client.19173 trim_cache size 0 max 16384 2024-09-10T08:58:49.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.199+0000 7f51ddcef740 1 Processor -- start 2024-09-10T08:58:49.213 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.199+0000 7f51ddcef740 1 -- start start 2024-09-10T08:58:49.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.199+0000 7f51ddcef740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cef75d0 0x55d34cf0e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.199+0000 7f51ddcef740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef8b40 0x55d34cefdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.199+0000 7f51ddcef740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cf02370 0x55d34cf199d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.199+0000 7f51ddcef740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f51d41b0cb0 con 0x55d34cef8b40 2024-09-10T08:58:49.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.199+0000 7f51ddcef740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f51d41b2880 con 0x55d34cef75d0 2024-09-10T08:58:49.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.199+0000 7f51ddcef740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f51d41b1380 con 0x55d34cf02370 2024-09-10T08:58:49.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.199+0000 7f51d96b2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef8b40 0x55d34cefdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.199+0000 7f51d96b2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef8b40 0x55d34cefdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49484/0 (socket says 172.21.15.172:49484) 2024-09-10T08:58:49.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.199+0000 7f51d96b2640 1 -- 172.21.15.172:0/2304620126 learned_addr learned my addr 172.21.15.172:0/2304620126 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:49.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.199+0000 7f51d8eb1640 1 --2- 172.21.15.172:0/2304620126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cef75d0 0x55d34cf0e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.199+0000 7f51d8eb1640 1 --2- 172.21.15.172:0/2304620126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cf02370 0x55d34cf199d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.214 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.200+0000 7f51d96b2640 1 -- 172.21.15.172:0/2304620126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cf02370 msgr2=0x55d34cf199d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:49.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.200+0000 7f51d96b2640 1 --2- 172.21.15.172:0/2304620126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cf02370 0x55d34cf199d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.200+0000 7f51d96b2640 1 -- 172.21.15.172:0/2304620126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cef75d0 msgr2=0x55d34cf0e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:49.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.200+0000 7f51d96b2640 1 --2- 172.21.15.172:0/2304620126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cef75d0 0x55d34cf0e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.200+0000 7f51d96b2640 1 -- 172.21.15.172:0/2304620126 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d34ceff7c0 con 0x55d34cef8b40 2024-09-10T08:58:49.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.200+0000 7f51d8eb1640 1 --2- 172.21.15.172:0/2304620126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cf02370 0x55d34cf199d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:49.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.200+0000 7f51d96b2640 1 --2- 172.21.15.172:0/2304620126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef8b40 0x55d34cefdfc0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x55d34cefe870 tx=0x7f51d0013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d54f89015ee155 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:49.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.200+0000 7f51beffd640 1 -- 172.21.15.172:0/2304620126 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d00130d0 con 0x55d34cef8b40 2024-09-10T08:58:49.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.200+0000 7f51beffd640 1 -- 172.21.15.172:0/2304620126 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51d0013270 con 0x55d34cef8b40 2024-09-10T08:58:49.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.200+0000 7f51beffd640 1 -- 172.21.15.172:0/2304620126 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0011880 con 0x55d34cef8b40 2024-09-10T08:58:49.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.201+0000 7f51ddcef740 1 -- 172.21.15.172:0/2304620126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef8b40 msgr2=0x55d34cefdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:49.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.201+0000 7f51ddcef740 1 --2- 172.21.15.172:0/2304620126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef8b40 0x55d34cefdfc0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x55d34cefe870 tx=0x7f51d0013830 comp rx=0 tx=0).stop 2024-09-10T08:58:49.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.201+0000 7f51ddcef740 1 -- 172.21.15.172:0/2304620126 shutdown_connections 2024-09-10T08:58:49.215 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.201+0000 7f51ddcef740 1 --2- 172.21.15.172:0/2304620126 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cf02370 0x55d34cf199d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.201+0000 7f51ddcef740 1 --2- 172.21.15.172:0/2304620126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef8b40 0x55d34cefdfc0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.201+0000 7f51ddcef740 1 --2- 172.21.15.172:0/2304620126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cef75d0 0x55d34cf0e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.201+0000 7f51ddcef740 1 -- 172.21.15.172:0/2304620126 >> 172.21.15.172:0/2304620126 conn(0x55d34cec0310 msgr2=0x55d34cefd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:49.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.201+0000 7f51ddcef740 1 -- 172.21.15.172:0/2304620126 shutdown_connections 2024-09-10T08:58:49.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.201+0000 7f51ddcef740 1 -- 172.21.15.172:0/2304620126 wait complete. 2024-09-10T08:58:49.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.201+0000 7f51ddcef740 1 -- start start 2024-09-10T08:58:49.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.201+0000 7f51ddcef740 1 Processor -- start 2024-09-10T08:58:49.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51ddcef740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cef75d0 0x55d34cf0a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51ddcef740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef8b40 0x55d34cf05be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51ddcef740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cf02370 0x55d34cf06120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51ddcef740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f51d41b21b0 con 0x55d34cef8b40 2024-09-10T08:58:49.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51ddcef740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f51d41b1ae0 con 0x55d34cef75d0 2024-09-10T08:58:49.216 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51ddcef740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f51d41aec90 con 0x55d34cf02370 2024-09-10T08:58:49.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51ddcef740 10 client.0 _ll_register_callbacks cb 0x7f51c400baf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:49.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51d96b2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef8b40 0x55d34cf05be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51d96b2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef8b40 0x55d34cf05be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49492/0 (socket says 172.21.15.172:49492) 2024-09-10T08:58:49.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51d96b2640 1 -- 172.21.15.172:0/3337513480 learned_addr learned my addr 172.21.15.172:0/3337513480 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:49.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51d8eb1640 1 --2- 172.21.15.172:0/3337513480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cf02370 0x55d34cf06120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51d8eb1640 1 --2- 172.21.15.172:0/3337513480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cef75d0 0x55d34cf0a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51d96b2640 1 -- 172.21.15.172:0/3337513480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cf02370 msgr2=0x55d34cf06120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:49.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51d96b2640 1 --2- 172.21.15.172:0/3337513480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cf02370 0x55d34cf06120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51d96b2640 1 -- 172.21.15.172:0/3337513480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cef75d0 msgr2=0x55d34cf0a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:49.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51d96b2640 1 --2- 172.21.15.172:0/3337513480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cef75d0 0x55d34cf0a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.217 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51d96b2640 1 -- 172.21.15.172:0/3337513480 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d34cf06660 con 0x55d34cef8b40 2024-09-10T08:58:49.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.202+0000 7f51d8eb1640 1 --2- 172.21.15.172:0/3337513480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cef75d0 0x55d34cf0a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:49.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.203+0000 7f51d96b2640 1 --2- 172.21.15.172:0/3337513480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef8b40 0x55d34cf05be0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x55d34cefe870 tx=0x7f51d000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbaac0abe01d9ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:49.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.203+0000 7f51bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:49.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.203+0000 7f51bcff9640 1 -- 172.21.15.172:0/3337513480 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0005450 con 0x55d34cef8b40 2024-09-10T08:58:49.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.203+0000 7f51bcff9640 1 -- 172.21.15.172:0/3337513480 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f51d00055f0 con 0x55d34cef8b40 2024-09-10T08:58:49.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.203+0000 7f51ddcef740 1 -- 172.21.15.172:0/3337513480 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51d0013270 con 0x55d34cef8b40 2024-09-10T08:58:49.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.203+0000 7f51bcff9640 1 -- 172.21.15.172:0/3337513480 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f51d0004020 con 0x55d34cef8b40 2024-09-10T08:58:49.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.203+0000 7f51ddcef740 10 client.19179 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:49.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.203+0000 7f51ddcef740 1 -- 172.21.15.172:0/3337513480 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d34cf7c650 con 0x55d34cef8b40 2024-09-10T08:58:49.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.203+0000 7f51bcff9640 1 -- 172.21.15.172:0/3337513480 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51d0013270 con 0x55d34cef8b40 2024-09-10T08:58:49.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.204+0000 7f51bcff9640 1 -- 172.21.15.172:0/3337513480 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f51d001c460 con 0x55d34cef8b40 2024-09-10T08:58:49.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.204+0000 7f51ddcef740 10 client.19179 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:49.218 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.204+0000 7f51ddcef740 10 client.19179 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T08:58:49.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.204+0000 7f51ddcef740 20 client.19179 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:49.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.204+0000 7f51ddcef740 1 --2- 172.21.15.172:0/3337513480 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d34cf7cd60 0x55d34cf7f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:49.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.204+0000 7f51ddcef740 4 client.19179 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:58:49.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.204+0000 7f51ddcef740 1 -- 172.21.15.172:0/3337513480 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d34ce64d70 con 0x55d34cf7cd60 2024-09-10T08:58:49.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.204+0000 7f51d8eb1640 1 --2- 172.21.15.172:0/3337513480 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d34cf7cd60 0x55d34cf7f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:49.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.204+0000 7f51d8eb1640 1 --2- 172.21.15.172:0/3337513480 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d34cf7cd60 0x55d34cf7f180 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5d70ea014a63cf61 server_cookie=f3d4a2bacac0874d in_seq=0 out_seq=0 2024-09-10T08:58:49.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.204+0000 7f51bcff9640 10 client.19179 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T08:58:49.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.204+0000 7f51bcff9640 1 -- 172.21.15.172:0/3337513480 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d34ce64d70 con 0x55d34cf7cd60 2024-09-10T08:58:49.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.204+0000 7f51bcff9640 10 client.19179 handle_command_reply: tid=0 2024-09-10T08:58:49.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.205+0000 7f51ddcef740 1 client.19179 shutdown 2024-09-10T08:58:49.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.205+0000 7f51ddcef740 1 -- 172.21.15.172:0/3337513480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef8b40 msgr2=0x55d34cf05be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:49.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.205+0000 7f51ddcef740 1 --2- 172.21.15.172:0/3337513480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef8b40 0x55d34cf05be0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x55d34cefe870 tx=0x7f51d000a7d0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.219 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.205+0000 7f51ddcef740 1 -- 172.21.15.172:0/3337513480 shutdown_connections 2024-09-10T08:58:49.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.205+0000 7f51ddcef740 1 --2- 172.21.15.172:0/3337513480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d34cf02370 0x55d34cf06120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.205+0000 7f51ddcef740 1 --2- 172.21.15.172:0/3337513480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d34cef8b40 0x55d34cf05be0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.205+0000 7f51ddcef740 1 --2- 172.21.15.172:0/3337513480 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55d34cf7cd60 0x55d34cf7f180 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.205+0000 7f51bcff9640 0 client.19179 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T08:58:49.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.205+0000 7f51ddcef740 1 --2- 172.21.15.172:0/3337513480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d34cef75d0 0x55d34cf0a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.205+0000 7f51ddcef740 1 -- 172.21.15.172:0/3337513480 >> 172.21.15.172:0/3337513480 conn(0x55d34cec0310 msgr2=0x55d34cefe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:49.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.205+0000 7f51ddcef740 1 -- 172.21.15.172:0/3337513480 shutdown_connections 2024-09-10T08:58:49.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.205+0000 7f51ddcef740 1 -- 172.21.15.172:0/3337513480 wait complete. 2024-09-10T08:58:49.220 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.205+0000 7f51ddcef740 20 client.19179 trim_cache size 0 max 16384 2024-09-10T08:58:49.220 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:58:49.221 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.206+0000 7f51db916640 1 -- 172.21.15.172:0/4057276774 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51c00a2b80 msgr2=0x7f51c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:49.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.206+0000 7f51db916640 1 --2- 172.21.15.172:0/4057276774 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51c00a2b80 0x7f51c00a5040 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f51cc006010 tx=0x7f51cc031f80 comp rx=0 tx=0).stop 2024-09-10T08:58:49.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.206+0000 7f51db916640 1 -- 172.21.15.172:0/4057276774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51d410c180 msgr2=0x7f51d4101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:49.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.206+0000 7f51db916640 1 --2- 172.21.15.172:0/4057276774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51d410c180 0x7f51d4101480 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f51cc005e30 tx=0x7f51cc003f10 comp rx=0 tx=0).stop 2024-09-10T08:58:49.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.206+0000 7f51db916640 1 -- 172.21.15.172:0/4057276774 shutdown_connections 2024-09-10T08:58:49.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.206+0000 7f51db916640 1 --2- 172.21.15.172:0/4057276774 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51c00a2b80 0x7f51c00a5040 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.206+0000 7f51db916640 1 --2- 172.21.15.172:0/4057276774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51d410c180 0x7f51d4101480 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.206+0000 7f51db916640 1 --2- 172.21.15.172:0/4057276774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51d41036c0 0x7f51d4100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.206+0000 7f51db916640 1 --2- 172.21.15.172:0/4057276774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51d4102da0 0x7f51d4100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:49.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.206+0000 7f51db916640 1 -- 172.21.15.172:0/4057276774 >> 172.21.15.172:0/4057276774 conn(0x7f51d40fc780 msgr2=0x7f51d40fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:49.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.206+0000 7f51db916640 1 -- 172.21.15.172:0/4057276774 shutdown_connections 2024-09-10T08:58:49.222 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:49.206+0000 7f51db916640 1 -- 172.21.15.172:0/4057276774 wait complete. 2024-09-10T08:58:49.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:49 smithi174 ceph-mon[34068]: pgmap v728: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 MiB/s wr, 13 op/s 2024-09-10T08:58:49.780 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:49 smithi172 ceph-mon[30854]: pgmap v728: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 MiB/s wr, 13 op/s 2024-09-10T08:58:49.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:49 smithi191 ceph-mon[35314]: pgmap v728: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.0 MiB/s wr, 13 op/s 2024-09-10T08:58:49.906 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T08:58:50.210 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:58:51.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.735+0000 7f16785ee640 1 -- 172.21.15.172:0/2356032656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1670102d90 msgr2=0x7f1670103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:51.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.735+0000 7f16785ee640 1 --2- 172.21.15.172:0/2356032656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1670102d90 0x7f1670103170 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f166c009930 tx=0x7f166c02a420 comp rx=0 tx=0).stop 2024-09-10T08:58:51.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.735+0000 7f16785ee640 1 -- 172.21.15.172:0/2356032656 shutdown_connections 2024-09-10T08:58:51.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.735+0000 7f16785ee640 1 --2- 172.21.15.172:0/2356032656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1670108290 0x7f167010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.735+0000 7f16785ee640 1 --2- 172.21.15.172:0/2356032656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1670103740 0x7f1670107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.735+0000 7f16785ee640 1 --2- 172.21.15.172:0/2356032656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1670102d90 0x7f1670103170 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.735+0000 7f16785ee640 1 -- 172.21.15.172:0/2356032656 >> 172.21.15.172:0/2356032656 conn(0x7f16700fc7d0 msgr2=0x7f16700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:51.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.735+0000 7f16785ee640 1 -- 172.21.15.172:0/2356032656 shutdown_connections 2024-09-10T08:58:51.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.735+0000 7f16785ee640 1 -- 172.21.15.172:0/2356032656 wait complete. 2024-09-10T08:58:51.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.736+0000 7f16785ee640 1 Processor -- start 2024-09-10T08:58:51.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.736+0000 7f16785ee640 1 -- start start 2024-09-10T08:58:51.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.736+0000 7f16785ee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1670102d90 0x7f16701984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.736+0000 7f16785ee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1670103740 0x7f1670198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.736+0000 7f16785ee640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1670108290 0x7f167019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.736+0000 7f16785ee640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1670111f40 con 0x7f1670102d90 2024-09-10T08:58:51.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.736+0000 7f16785ee640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1670111dc0 con 0x7f1670103740 2024-09-10T08:58:51.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.736+0000 7f16785ee640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f16701120c0 con 0x7f1670108290 2024-09-10T08:58:51.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.736+0000 7f167638a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1670102d90 0x7f16701984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.736+0000 7f167638a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1670102d90 0x7f16701984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49508/0 (socket says 172.21.15.172:49508) 2024-09-10T08:58:51.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.736+0000 7f167638a640 1 -- 172.21.15.172:0/3003185133 learned_addr learned my addr 172.21.15.172:0/3003185133 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:51.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.736+0000 7f1675b89640 1 --2- 172.21.15.172:0/3003185133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1670103740 0x7f1670198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.736+0000 7f1676b8b640 1 --2- 172.21.15.172:0/3003185133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1670108290 0x7f167019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.736+0000 7f167638a640 1 -- 172.21.15.172:0/3003185133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1670108290 msgr2=0x7f167019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:51.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.736+0000 7f167638a640 1 --2- 172.21.15.172:0/3003185133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1670108290 0x7f167019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.737+0000 7f167638a640 1 -- 172.21.15.172:0/3003185133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1670103740 msgr2=0x7f1670198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:51.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.737+0000 7f167638a640 1 --2- 172.21.15.172:0/3003185133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1670103740 0x7f1670198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.737+0000 7f167638a640 1 -- 172.21.15.172:0/3003185133 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167019d470 con 0x7f1670102d90 2024-09-10T08:58:51.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.737+0000 7f1676b8b640 1 --2- 172.21.15.172:0/3003185133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1670108290 0x7f167019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:51.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.737+0000 7f1675b89640 1 --2- 172.21.15.172:0/3003185133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1670103740 0x7f1670198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:51.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.737+0000 7f167638a640 1 --2- 172.21.15.172:0/3003185133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1670102d90 0x7f16701984c0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f166c004770 tx=0x7f166c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:51.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.737+0000 7f16677fe640 1 -- 172.21.15.172:0/3003185133 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c03f070 con 0x7f1670102d90 2024-09-10T08:58:51.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.737+0000 7f16785ee640 1 -- 172.21.15.172:0/3003185133 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167019d700 con 0x7f1670102d90 2024-09-10T08:58:51.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.737+0000 7f16677fe640 1 -- 172.21.15.172:0/3003185133 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f166c03b460 con 0x7f1670102d90 2024-09-10T08:58:51.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.737+0000 7f16677fe640 1 -- 172.21.15.172:0/3003185133 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c035070 con 0x7f1670102d90 2024-09-10T08:58:51.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.737+0000 7f16785ee640 1 -- 172.21.15.172:0/3003185133 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16701a5040 con 0x7f1670102d90 2024-09-10T08:58:51.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.739+0000 7f16677fe640 1 -- 172.21.15.172:0/3003185133 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f166c03b610 con 0x7f1670102d90 2024-09-10T08:58:51.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.740+0000 7f167a9c7740 1 Processor -- start 2024-09-10T08:58:51.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.740+0000 7f167a9c7740 1 -- start start 2024-09-10T08:58:51.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.740+0000 7f16677fe640 1 --2- 172.21.15.172:0/3003185133 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f165c0a2b80 0x7f165c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.740+0000 7f167a9c7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 0x55da67725090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.740+0000 7f167a9c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da677255d0 0x55da67725a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.740+0000 7f167a9c7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67734780 0x55da67736b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.740+0000 7f167a9c7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55da6768b2f0 con 0x55da67734780 2024-09-10T08:58:51.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.740+0000 7f167a9c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55da67632cf0 con 0x55da677255d0 2024-09-10T08:58:51.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.740+0000 7f167a9c7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55da676872b0 con 0x55da67726b40 2024-09-10T08:58:51.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.741+0000 7f1676b8b640 1 --2- 172.21.15.172:0/3003185133 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f165c0a2b80 0x7f165c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.741+0000 7f16677fe640 1 -- 172.21.15.172:0/3003185133 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f166c0ef870 con 0x7f1670102d90 2024-09-10T08:58:51.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.741+0000 7f1676b8b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67734780 0x55da67736b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.741+0000 7f1675b89640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da677255d0 0x55da67725a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.741+0000 7f1676b8b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67734780 0x55da67736b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49512/0 (socket says 172.21.15.172:49512) 2024-09-10T08:58:51.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.741+0000 7f1676b8b640 1 -- 172.21.15.172:0/1727057780 learned_addr learned my addr 172.21.15.172:0/1727057780 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:51.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.741+0000 7f167638a640 1 --2- 172.21.15.172:0/1727057780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 0x55da67725090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.741+0000 7f1676b8b640 1 -- 172.21.15.172:0/1727057780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 msgr2=0x55da67725090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:51.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.741+0000 7f1676b8b640 1 --2- 172.21.15.172:0/1727057780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 0x55da67725090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.741+0000 7f1676b8b640 1 -- 172.21.15.172:0/1727057780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da677255d0 msgr2=0x55da67725a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:51.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.741+0000 7f1676b8b640 1 --2- 172.21.15.172:0/1727057780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da677255d0 0x55da67725a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.741+0000 7f1676b8b640 1 -- 172.21.15.172:0/1727057780 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da676173a0 con 0x55da67734780 2024-09-10T08:58:51.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.741+0000 7f1676b8b640 1 --2- 172.21.15.172:0/1727057780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67734780 0x55da67736b70 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f1668013e50 tx=0x7f16680107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc494a0a6e5428 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:51.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.741+0000 7f1676b8b640 1 --2- 172.21.15.172:0/3003185133 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f165c0a2b80 0x7f165c0a5040 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1668010be0 tx=0x7f166801b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:51.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.741+0000 7f1664ff9640 1 -- 172.21.15.172:0/1727057780 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668021070 con 0x55da67734780 2024-09-10T08:58:51.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.741+0000 7f1664ff9640 1 -- 172.21.15.172:0/1727057780 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1668007650 con 0x55da67734780 2024-09-10T08:58:51.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.742+0000 7f1664ff9640 1 -- 172.21.15.172:0/1727057780 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16680053a0 con 0x55da67734780 2024-09-10T08:58:51.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.742+0000 7f167a9c7740 1 -- 172.21.15.172:0/1727057780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67734780 msgr2=0x55da67736b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:51.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.742+0000 7f167a9c7740 1 --2- 172.21.15.172:0/1727057780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67734780 0x55da67736b70 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f1668013e50 tx=0x7f16680107a0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.742+0000 7f167a9c7740 1 -- 172.21.15.172:0/1727057780 shutdown_connections 2024-09-10T08:58:51.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.742+0000 7f167a9c7740 1 --2- 172.21.15.172:0/1727057780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67734780 0x55da67736b70 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.742+0000 7f167a9c7740 1 --2- 172.21.15.172:0/1727057780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da677255d0 0x55da67725a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.742+0000 7f167a9c7740 1 --2- 172.21.15.172:0/1727057780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 0x55da67725090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.742+0000 7f167a9c7740 1 -- 172.21.15.172:0/1727057780 >> 172.21.15.172:0/1727057780 conn(0x55da676ee310 msgr2=0x55da67721c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:51.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.743+0000 7f167a9c7740 1 -- 172.21.15.172:0/1727057780 shutdown_connections 2024-09-10T08:58:51.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.743+0000 7f167a9c7740 1 -- 172.21.15.172:0/1727057780 wait complete. 2024-09-10T08:58:51.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.743+0000 7f167a9c7740 1 -- start start 2024-09-10T08:58:51.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.743+0000 7f167a9c7740 1 Processor -- start 2024-09-10T08:58:51.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.743+0000 7f167a9c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da677255d0 0x55da67731500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.743+0000 7f167a9c7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 0x55da67731a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.743+0000 7f167a9c7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67734780 0x55da67731f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.743+0000 7f167a9c7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55da67737420 con 0x55da67734780 2024-09-10T08:58:51.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.743+0000 7f167a9c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55da67646060 con 0x55da677255d0 2024-09-10T08:58:51.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.743+0000 7f167a9c7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55da677375a0 con 0x55da67726b40 2024-09-10T08:58:51.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.743+0000 7f167a9c7740 10 client.0 _ll_register_callbacks cb 0x55da673998c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:51.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.744+0000 7f1675b89640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 0x55da67731a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.744+0000 7f1676b8b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67734780 0x55da67731f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.744+0000 7f1675b89640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 0x55da67731a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:45268/0 (socket says 172.21.15.172:45268) 2024-09-10T08:58:51.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.744+0000 7f1675b89640 1 -- 172.21.15.172:0/1563076696 learned_addr learned my addr 172.21.15.172:0/1563076696 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:51.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.744+0000 7f167638a640 1 --2- 172.21.15.172:0/1563076696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da677255d0 0x55da67731500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.744+0000 7f1676b8b640 1 -- 172.21.15.172:0/1563076696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 msgr2=0x55da67731a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:51.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.744+0000 7f1676b8b640 1 --2- 172.21.15.172:0/1563076696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 0x55da67731a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.744+0000 7f1676b8b640 1 -- 172.21.15.172:0/1563076696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da677255d0 msgr2=0x55da67731500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:51.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.744+0000 7f1676b8b640 1 --2- 172.21.15.172:0/1563076696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da677255d0 0x55da67731500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.744+0000 7f1676b8b640 1 -- 172.21.15.172:0/1563076696 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da673fc3c0 con 0x55da67734780 2024-09-10T08:58:51.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.744+0000 7f1675b89640 1 --2- 172.21.15.172:0/1563076696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 0x55da67731a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:51.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.744+0000 7f167638a640 1 --2- 172.21.15.172:0/1563076696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da677255d0 0x55da67731500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:51.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.744+0000 7f1676b8b640 1 --2- 172.21.15.172:0/1563076696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67734780 0x55da67731f80 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f16680139d0 tx=0x7f1668005330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aeb4364cb7f9e77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:51.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.744+0000 7f1659ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:51.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.744+0000 7f1659ffb640 1 -- 172.21.15.172:0/1563076696 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1668029030 con 0x55da67734780 2024-09-10T08:58:51.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.744+0000 7f1659ffb640 1 -- 172.21.15.172:0/1563076696 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1668005820 con 0x55da67734780 2024-09-10T08:58:51.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.744+0000 7f167a9c7740 1 -- 172.21.15.172:0/1563076696 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da67698860 con 0x55da67734780 2024-09-10T08:58:51.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.745+0000 7f1659ffb640 1 -- 172.21.15.172:0/1563076696 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1668035050 con 0x55da67734780 2024-09-10T08:58:51.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.745+0000 7f167a9c7740 10 client.19191 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:51.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.745+0000 7f167a9c7740 1 -- 172.21.15.172:0/1563076696 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55da67742fb0 con 0x55da67734780 2024-09-10T08:58:51.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.745+0000 7f1659ffb640 1 -- 172.21.15.172:0/1563076696 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55da67698860 con 0x55da67734780 2024-09-10T08:58:51.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.745+0000 7f1659ffb640 1 -- 172.21.15.172:0/1563076696 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f166801c050 con 0x55da67734780 2024-09-10T08:58:51.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.745+0000 7f167a9c7740 10 client.19191 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:51.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.745+0000 7f167a9c7740 10 client.19191 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:58:51.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.745+0000 7f167a9c7740 20 client.19191 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:51.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.745+0000 7f167a9c7740 1 --2- 172.21.15.172:0/1563076696 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55da677443c0 0x55da677467b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.745+0000 7f167a9c7740 4 client.19191 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:58:51.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.745+0000 7f167a9c7740 1 -- 172.21.15.172:0/1563076696 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da67692d70 con 0x55da677443c0 2024-09-10T08:58:51.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.745+0000 7f167638a640 1 --2- 172.21.15.172:0/1563076696 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55da677443c0 0x55da677467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.746+0000 7f1659ffb640 10 client.19191 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:58:51.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.746+0000 7f167638a640 1 --2- 172.21.15.172:0/1563076696 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55da677443c0 0x55da677467b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=39a5a76eee90f184 server_cookie=8bb0c6ce56d77e06 in_seq=0 out_seq=0 2024-09-10T08:58:51.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.747+0000 7f1659ffb640 1 -- 172.21.15.172:0/1563076696 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da67692d70 con 0x55da677443c0 2024-09-10T08:58:51.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.747+0000 7f1659ffb640 10 client.19191 handle_command_reply: tid=0 2024-09-10T08:58:51.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.747+0000 7f167a9c7740 1 client.19191 shutdown 2024-09-10T08:58:51.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.747+0000 7f167a9c7740 1 -- 172.21.15.172:0/1563076696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67734780 msgr2=0x55da67731f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:51.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.747+0000 7f167a9c7740 1 --2- 172.21.15.172:0/1563076696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67734780 0x55da67731f80 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f16680139d0 tx=0x7f1668005330 comp rx=0 tx=0).stop 2024-09-10T08:58:51.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.747+0000 7f167a9c7740 1 -- 172.21.15.172:0/1563076696 shutdown_connections 2024-09-10T08:58:51.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.747+0000 7f167a9c7740 1 --2- 172.21.15.172:0/1563076696 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55da677443c0 0x55da677467b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.747+0000 7f167a9c7740 1 --2- 172.21.15.172:0/1563076696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67734780 0x55da67731f80 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.747+0000 7f167a9c7740 1 --2- 172.21.15.172:0/1563076696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 0x55da67731a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.747+0000 7f1659ffb640 0 client.19191 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:58:51.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.747+0000 7f167a9c7740 1 --2- 172.21.15.172:0/1563076696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da677255d0 0x55da67731500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.747+0000 7f167a9c7740 1 -- 172.21.15.172:0/1563076696 >> 172.21.15.172:0/1563076696 conn(0x55da676ee310 msgr2=0x55da67727960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:51.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.747+0000 7f167a9c7740 1 -- 172.21.15.172:0/1563076696 shutdown_connections 2024-09-10T08:58:51.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.747+0000 7f167a9c7740 1 -- 172.21.15.172:0/1563076696 wait complete. 2024-09-10T08:58:51.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.747+0000 7f167a9c7740 20 client.19191 trim_cache size 0 max 16384 2024-09-10T08:58:51.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.760+0000 7f167a9c7740 1 Processor -- start 2024-09-10T08:58:51.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f167a9c7740 1 -- start start 2024-09-10T08:58:51.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f167a9c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da677255d0 0x55da67733830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f167a9c7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67726b40 0x55da6774be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f167a9c7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67734780 0x55da6774c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f167a9c7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16701ac980 con 0x55da67726b40 2024-09-10T08:58:51.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f167a9c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16701ae550 con 0x55da677255d0 2024-09-10T08:58:51.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f167a9c7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f16701ad050 con 0x55da67734780 2024-09-10T08:58:51.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f1675b89640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67726b40 0x55da6774be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f1675b89640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67726b40 0x55da6774be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49536/0 (socket says 172.21.15.172:49536) 2024-09-10T08:58:51.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f1675b89640 1 -- 172.21.15.172:0/2544702545 learned_addr learned my addr 172.21.15.172:0/2544702545 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:51.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f167638a640 1 --2- 172.21.15.172:0/2544702545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da677255d0 0x55da67733830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f1676b8b640 1 --2- 172.21.15.172:0/2544702545 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67734780 0x55da6774c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f1675b89640 1 -- 172.21.15.172:0/2544702545 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67734780 msgr2=0x55da6774c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:51.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f1675b89640 1 --2- 172.21.15.172:0/2544702545 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67734780 0x55da6774c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f1675b89640 1 -- 172.21.15.172:0/2544702545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da677255d0 msgr2=0x55da67733830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:51.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f1675b89640 1 --2- 172.21.15.172:0/2544702545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da677255d0 0x55da67733830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f1675b89640 1 -- 172.21.15.172:0/2544702545 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da67728120 con 0x55da67726b40 2024-09-10T08:58:51.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.761+0000 7f1675b89640 1 --2- 172.21.15.172:0/2544702545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67726b40 0x55da6774be40 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f166000bd30 tx=0x7f1660012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e89d0991996a2ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:51.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.762+0000 7f165bfff640 1 -- 172.21.15.172:0/2544702545 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166000be20 con 0x55da67726b40 2024-09-10T08:58:51.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.762+0000 7f165bfff640 1 -- 172.21.15.172:0/2544702545 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16600054d0 con 0x55da67726b40 2024-09-10T08:58:51.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.762+0000 7f165bfff640 1 -- 172.21.15.172:0/2544702545 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166001b670 con 0x55da67726b40 2024-09-10T08:58:51.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.762+0000 7f167a9c7740 1 -- 172.21.15.172:0/2544702545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67726b40 msgr2=0x55da6774be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:51.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.762+0000 7f167a9c7740 1 --2- 172.21.15.172:0/2544702545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67726b40 0x55da6774be40 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f166000bd30 tx=0x7f1660012820 comp rx=0 tx=0).stop 2024-09-10T08:58:51.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.762+0000 7f167a9c7740 1 -- 172.21.15.172:0/2544702545 shutdown_connections 2024-09-10T08:58:51.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.762+0000 7f167a9c7740 1 --2- 172.21.15.172:0/2544702545 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67734780 0x55da6774c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.762+0000 7f167a9c7740 1 --2- 172.21.15.172:0/2544702545 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da67726b40 0x55da6774be40 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.762+0000 7f167a9c7740 1 --2- 172.21.15.172:0/2544702545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da677255d0 0x55da67733830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.762+0000 7f167a9c7740 1 -- 172.21.15.172:0/2544702545 >> 172.21.15.172:0/2544702545 conn(0x55da676ee310 msgr2=0x55da67720d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:51.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.762+0000 7f167a9c7740 1 -- 172.21.15.172:0/2544702545 shutdown_connections 2024-09-10T08:58:51.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.762+0000 7f167a9c7740 1 -- 172.21.15.172:0/2544702545 wait complete. 2024-09-10T08:58:51.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.763+0000 7f167a9c7740 1 -- start start 2024-09-10T08:58:51.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.763+0000 7f167a9c7740 1 Processor -- start 2024-09-10T08:58:51.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.763+0000 7f167a9c7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da677255d0 0x55da6773d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.763+0000 7f167a9c7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 0x55da677383b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.763+0000 7f167a9c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da67734780 0x55da67738980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.763+0000 7f167a9c7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16701ade80 con 0x55da677255d0 2024-09-10T08:58:51.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.763+0000 7f167a9c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16701ad7b0 con 0x55da67734780 2024-09-10T08:58:51.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.763+0000 7f167a9c7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f16701aa960 con 0x55da67726b40 2024-09-10T08:58:51.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.763+0000 7f167a9c7740 10 client.0 _ll_register_callbacks cb 0x7f166000b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:51.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.763+0000 7f1675b89640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 0x55da677383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.763+0000 7f167638a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da677255d0 0x55da6773d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.763+0000 7f1676b8b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da67734780 0x55da67738980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.763+0000 7f1675b89640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 0x55da677383b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:45296/0 (socket says 172.21.15.172:45296) 2024-09-10T08:58:51.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.763+0000 7f1675b89640 1 -- 172.21.15.172:0/2396610617 learned_addr learned my addr 172.21.15.172:0/2396610617 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:51.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.764+0000 7f1675b89640 1 -- 172.21.15.172:0/2396610617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da67734780 msgr2=0x55da67738980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:51.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.764+0000 7f1675b89640 1 --2- 172.21.15.172:0/2396610617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da67734780 0x55da67738980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.764+0000 7f1675b89640 1 -- 172.21.15.172:0/2396610617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da677255d0 msgr2=0x55da6773d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:51.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.764+0000 7f1675b89640 1 --2- 172.21.15.172:0/2396610617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da677255d0 0x55da6773d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.764+0000 7f1675b89640 1 -- 172.21.15.172:0/2396610617 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da677391b0 con 0x55da67726b40 2024-09-10T08:58:51.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.764+0000 7f167638a640 1 --2- 172.21.15.172:0/2396610617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da677255d0 0x55da6773d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:51.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.764+0000 7f1675b89640 1 --2- 172.21.15.172:0/2396610617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 0x55da677383b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f166000b7a0 tx=0x7f1660005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6acbe3a534a070b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:51.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.764+0000 7f1659ffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:58:51.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.764+0000 7f1659ffb640 1 -- 172.21.15.172:0/2396610617 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1660029020 con 0x55da67726b40 2024-09-10T08:58:51.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.764+0000 7f167a9c7740 1 -- 172.21.15.172:0/2396610617 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16600054d0 con 0x55da67726b40 2024-09-10T08:58:51.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.764+0000 7f1659ffb640 1 -- 172.21.15.172:0/2396610617 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f16600046c0 con 0x55da67726b40 2024-09-10T08:58:51.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.765+0000 7f1659ffb640 1 -- 172.21.15.172:0/2396610617 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1660034050 con 0x55da67726b40 2024-09-10T08:58:51.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.765+0000 7f167a9c7740 10 client.27928 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:51.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.765+0000 7f167a9c7740 1 -- 172.21.15.172:0/2396610617 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55da677aef10 con 0x55da67726b40 2024-09-10T08:58:51.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.765+0000 7f1659ffb640 1 -- 172.21.15.172:0/2396610617 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16600054d0 con 0x55da67726b40 2024-09-10T08:58:51.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.765+0000 7f1659ffb640 1 -- 172.21.15.172:0/2396610617 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f166001b410 con 0x55da67726b40 2024-09-10T08:58:51.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.765+0000 7f167a9c7740 10 client.27928 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:51.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.765+0000 7f167a9c7740 10 client.27928 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T08:58:51.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.765+0000 7f167a9c7740 20 client.27928 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:51.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.765+0000 7f167a9c7740 1 --2- 172.21.15.172:0/2396610617 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55da677af6f0 0x55da677b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:51.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.765+0000 7f167a9c7740 4 client.27928 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:58:51.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.765+0000 7f167a9c7740 1 -- 172.21.15.172:0/2396610617 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da6760f170 con 0x55da677af6f0 2024-09-10T08:58:51.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.765+0000 7f167638a640 1 --2- 172.21.15.172:0/2396610617 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55da677af6f0 0x55da677b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:51.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.766+0000 7f167638a640 1 --2- 172.21.15.172:0/2396610617 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55da677af6f0 0x55da677b1b10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7691349113e126c7 server_cookie=ae83402f9bcb9c6d in_seq=0 out_seq=0 2024-09-10T08:58:51.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.766+0000 7f1659ffb640 10 client.27928 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T08:58:51.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.766+0000 7f1659ffb640 1 -- 172.21.15.172:0/2396610617 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da6760f170 con 0x55da677af6f0 2024-09-10T08:58:51.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.766+0000 7f1659ffb640 10 client.27928 handle_command_reply: tid=0 2024-09-10T08:58:51.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.766+0000 7f167a9c7740 1 client.27928 shutdown 2024-09-10T08:58:51.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.766+0000 7f167a9c7740 1 -- 172.21.15.172:0/2396610617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 msgr2=0x55da677383b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:51.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.766+0000 7f167a9c7740 1 --2- 172.21.15.172:0/2396610617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 0x55da677383b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f166000b7a0 tx=0x7f1660005770 comp rx=0 tx=0).stop 2024-09-10T08:58:51.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.766+0000 7f167a9c7740 1 -- 172.21.15.172:0/2396610617 shutdown_connections 2024-09-10T08:58:51.102 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.766+0000 7f167a9c7740 1 --2- 172.21.15.172:0/2396610617 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55da677af6f0 0x55da677b1b10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.766+0000 7f167a9c7740 1 --2- 172.21.15.172:0/2396610617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55da67734780 0x55da67738980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.766+0000 7f167a9c7740 1 --2- 172.21.15.172:0/2396610617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55da67726b40 0x55da677383b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.766+0000 7f167a9c7740 1 --2- 172.21.15.172:0/2396610617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da677255d0 0x55da6773d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.766+0000 7f167a9c7740 1 -- 172.21.15.172:0/2396610617 >> 172.21.15.172:0/2396610617 conn(0x55da676ee310 msgr2=0x55da6772fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:51.103 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:58:51.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.766+0000 7f1659ffb640 0 client.27928 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T08:58:51.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.766+0000 7f167a9c7740 1 -- 172.21.15.172:0/2396610617 shutdown_connections 2024-09-10T08:58:51.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.766+0000 7f167a9c7740 1 -- 172.21.15.172:0/2396610617 wait complete. 2024-09-10T08:58:51.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.766+0000 7f167a9c7740 20 client.27928 trim_cache size 0 max 16384 2024-09-10T08:58:51.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.767+0000 7f16785ee640 1 -- 172.21.15.172:0/3003185133 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f165c0a2b80 msgr2=0x7f165c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:51.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.767+0000 7f16785ee640 1 --2- 172.21.15.172:0/3003185133 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f165c0a2b80 0x7f165c0a5040 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1668010be0 tx=0x7f166801b000 comp rx=0 tx=0).stop 2024-09-10T08:58:51.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.767+0000 7f16785ee640 1 -- 172.21.15.172:0/3003185133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1670102d90 msgr2=0x7f16701984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:51.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.767+0000 7f16785ee640 1 --2- 172.21.15.172:0/3003185133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1670102d90 0x7f16701984c0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f166c004770 tx=0x7f166c0042f0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.767+0000 7f16785ee640 1 -- 172.21.15.172:0/3003185133 shutdown_connections 2024-09-10T08:58:51.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.767+0000 7f16785ee640 1 --2- 172.21.15.172:0/3003185133 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f165c0a2b80 0x7f165c0a5040 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.767+0000 7f16785ee640 1 --2- 172.21.15.172:0/3003185133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1670108290 0x7f167019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.767+0000 7f16785ee640 1 --2- 172.21.15.172:0/3003185133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1670103740 0x7f1670198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.767+0000 7f16785ee640 1 --2- 172.21.15.172:0/3003185133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1670102d90 0x7f16701984c0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:51.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.767+0000 7f16785ee640 1 -- 172.21.15.172:0/3003185133 >> 172.21.15.172:0/3003185133 conn(0x7f16700fc7d0 msgr2=0x7f1670105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:51.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.767+0000 7f16785ee640 1 -- 172.21.15.172:0/3003185133 shutdown_connections 2024-09-10T08:58:51.105 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:50.767+0000 7f16785ee640 1 -- 172.21.15.172:0/3003185133 wait complete. 2024-09-10T08:58:51.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:51 smithi174 ceph-mon[34068]: pgmap v729: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.8 MiB/s wr, 14 op/s 2024-09-10T08:58:51.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:51 smithi191 ceph-mon[35314]: pgmap v729: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.8 MiB/s wr, 14 op/s 2024-09-10T08:58:51.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:51 smithi172 ceph-mon[30854]: pgmap v729: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.8 MiB/s wr, 14 op/s 2024-09-10T08:58:51.897 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T08:58:52.179 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:58:53.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.706+0000 7f06f0cec640 1 -- 172.21.15.172:0/1964738464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06ec102570 msgr2=0x7f06ec1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:53.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.706+0000 7f06f0cec640 1 --2- 172.21.15.172:0/1964738464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06ec102570 0x7f06ec1029b0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009900 tx=0x7f06e002a480 comp rx=0 tx=0).stop 2024-09-10T08:58:53.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.706+0000 7f06f0cec640 1 -- 172.21.15.172:0/1964738464 shutdown_connections 2024-09-10T08:58:53.020 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.706+0000 7f06f0cec640 1 --2- 172.21.15.172:0/1964738464 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06ec102ef0 0x7f06ec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.706+0000 7f06f0cec640 1 --2- 172.21.15.172:0/1964738464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06ec102570 0x7f06ec1029b0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.706+0000 7f06f0cec640 1 --2- 172.21.15.172:0/1964738464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06ec100e20 0x7f06ec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.706+0000 7f06f0cec640 1 -- 172.21.15.172:0/1964738464 >> 172.21.15.172:0/1964738464 conn(0x7f06ec0fc7d0 msgr2=0x7f06ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:53.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.706+0000 7f06f0cec640 1 -- 172.21.15.172:0/1964738464 shutdown_connections 2024-09-10T08:58:53.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.707+0000 7f06f0cec640 1 -- 172.21.15.172:0/1964738464 wait complete. 2024-09-10T08:58:53.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.707+0000 7f06f0cec640 1 Processor -- start 2024-09-10T08:58:53.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.707+0000 7f06f0cec640 1 -- start start 2024-09-10T08:58:53.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06f0cec640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06ec100e20 0x7f06ec199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06f0cec640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06ec102570 0x7f06ec1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06f0cec640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06ec102ef0 0x7f06ec1a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.021 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06f0cec640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec110e00 con 0x7f06ec102570 2024-09-10T08:58:53.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06f0cec640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec110c80 con 0x7f06ec102ef0 2024-09-10T08:58:53.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06f0cec640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec110f80 con 0x7f06ec100e20 2024-09-10T08:58:53.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06e9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06ec102570 0x7f06ec1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06e9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06ec102570 0x7f06ec1996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49574/0 (socket says 172.21.15.172:49574) 2024-09-10T08:58:53.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06e9d9b640 1 -- 172.21.15.172:0/3454193203 learned_addr learned my addr 172.21.15.172:0/3454193203 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:53.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06ea59c640 1 --2- 172.21.15.172:0/3454193203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06ec100e20 0x7f06ec199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06ead9d640 1 --2- 172.21.15.172:0/3454193203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06ec102ef0 0x7f06ec1a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06e9d9b640 1 -- 172.21.15.172:0/3454193203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06ec100e20 msgr2=0x7f06ec199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:53.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06e9d9b640 1 --2- 172.21.15.172:0/3454193203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06ec100e20 0x7f06ec199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06e9d9b640 1 -- 172.21.15.172:0/3454193203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06ec102ef0 msgr2=0x7f06ec1a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:53.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06e9d9b640 1 --2- 172.21.15.172:0/3454193203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06ec102ef0 0x7f06ec1a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.022 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06e9d9b640 1 -- 172.21.15.172:0/3454193203 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ec1a4750 con 0x7f06ec102570 2024-09-10T08:58:53.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06ead9d640 1 --2- 172.21.15.172:0/3454193203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06ec102ef0 0x7f06ec1a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:53.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.708+0000 7f06e9d9b640 1 --2- 172.21.15.172:0/3454193203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06ec102570 0x7f06ec1996d0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f06e0006010 tx=0x7f06e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:53.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.709+0000 7f06db7fe640 1 -- 172.21.15.172:0/3454193203 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e003f070 con 0x7f06ec102570 2024-09-10T08:58:53.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.709+0000 7f06f0cec640 1 -- 172.21.15.172:0/3454193203 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ec1a4920 con 0x7f06ec102570 2024-09-10T08:58:53.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.709+0000 7f06db7fe640 1 -- 172.21.15.172:0/3454193203 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06e0003aa0 con 0x7f06ec102570 2024-09-10T08:58:53.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.709+0000 7f06db7fe640 1 -- 172.21.15.172:0/3454193203 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0035030 con 0x7f06ec102570 2024-09-10T08:58:53.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.709+0000 7f06f0cec640 1 -- 172.21.15.172:0/3454193203 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ec1a4d70 con 0x7f06ec102570 2024-09-10T08:58:53.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.711+0000 7f06db7fe640 1 -- 172.21.15.172:0/3454193203 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f06e0042070 con 0x7f06ec102570 2024-09-10T08:58:53.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.712+0000 7f06f30c5740 1 Processor -- start 2024-09-10T08:58:53.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.712+0000 7f06f30c5740 1 -- start start 2024-09-10T08:58:53.023 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.712+0000 7f06f30c5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb77cb40 0x55d8bb77b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.712+0000 7f06f30c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb77ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.712+0000 7f06f30c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb78a780 0x55d8bb78cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.712+0000 7f06f30c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d8bb6e12f0 con 0x55d8bb77b5d0 2024-09-10T08:58:53.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.712+0000 7f06f30c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d8bb688cf0 con 0x55d8bb78a780 2024-09-10T08:58:53.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.712+0000 7f06f30c5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d8bb6dd2b0 con 0x55d8bb77cb40 2024-09-10T08:58:53.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.712+0000 7f06ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb77ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.712+0000 7f06ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb77ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49588/0 (socket says 172.21.15.172:49588) 2024-09-10T08:58:53.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.712+0000 7f06ea59c640 1 -- 172.21.15.172:0/1208686040 learned_addr learned my addr 172.21.15.172:0/1208686040 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:53.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.712+0000 7f06db7fe640 1 --2- 172.21.15.172:0/3454193203 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f06d40a2b80 0x7f06d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.713+0000 7f06ead9d640 1 --2- 172.21.15.172:0/1208686040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb77cb40 0x55d8bb77b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.024 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.713+0000 7f06ead9d640 1 --2- 172.21.15.172:0/3454193203 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f06d40a2b80 0x7f06d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.713+0000 7f06db7fe640 1 -- 172.21.15.172:0/3454193203 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f06e00efcb0 con 0x7f06ec102570 2024-09-10T08:58:53.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.713+0000 7f06ea59c640 1 -- 172.21.15.172:0/1208686040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb77cb40 msgr2=0x55d8bb77b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:53.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.713+0000 7f06ea59c640 1 --2- 172.21.15.172:0/1208686040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb77cb40 0x55d8bb77b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.713+0000 7f06ea59c640 1 -- 172.21.15.172:0/1208686040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb78a780 msgr2=0x55d8bb78cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:53.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.713+0000 7f06e9d9b640 1 --2- 172.21.15.172:0/1208686040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb78a780 0x55d8bb78cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.025 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.713+0000 7f06ea59c640 1 --2- 172.21.15.172:0/1208686040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb78a780 0x55d8bb78cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.713+0000 7f06ea59c640 1 -- 172.21.15.172:0/1208686040 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8bb66d3a0 con 0x55d8bb77b5d0 2024-09-10T08:58:53.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.713+0000 7f06ead9d640 1 --2- 172.21.15.172:0/1208686040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb77cb40 0x55d8bb77b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:53.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.713+0000 7f06ead9d640 1 --2- 172.21.15.172:0/3454193203 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f06d40a2b80 0x7f06d40a5040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f06e4007fd0 tx=0x7f06e40072a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:53.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.713+0000 7f06ea59c640 1 --2- 172.21.15.172:0/1208686040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb77ba10 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00ad00 tx=0x7f06dc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ffb074c935ec8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:53.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.713+0000 7f06d8ff9640 1 -- 172.21.15.172:0/1208686040 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc01d070 con 0x55d8bb77b5d0 2024-09-10T08:58:53.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.714+0000 7f06d8ff9640 1 -- 172.21.15.172:0/1208686040 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06dc018af0 con 0x55d8bb77b5d0 2024-09-10T08:58:53.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.714+0000 7f06d8ff9640 1 -- 172.21.15.172:0/1208686040 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc010ba0 con 0x55d8bb77b5d0 2024-09-10T08:58:53.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.714+0000 7f06f30c5740 1 -- 172.21.15.172:0/1208686040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 msgr2=0x55d8bb77ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:53.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.714+0000 7f06f30c5740 1 --2- 172.21.15.172:0/1208686040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb77ba10 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00ad00 tx=0x7f06dc00f790 comp rx=0 tx=0).stop 2024-09-10T08:58:53.026 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.714+0000 7f06f30c5740 1 -- 172.21.15.172:0/1208686040 shutdown_connections 2024-09-10T08:58:53.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.714+0000 7f06f30c5740 1 --2- 172.21.15.172:0/1208686040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb78a780 0x55d8bb78cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.714+0000 7f06f30c5740 1 --2- 172.21.15.172:0/1208686040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb77ba10 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.714+0000 7f06f30c5740 1 --2- 172.21.15.172:0/1208686040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb77cb40 0x55d8bb77b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.714+0000 7f06f30c5740 1 -- 172.21.15.172:0/1208686040 >> 172.21.15.172:0/1208686040 conn(0x55d8bb744310 msgr2=0x55d8bb777c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:53.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.714+0000 7f06f30c5740 1 -- 172.21.15.172:0/1208686040 shutdown_connections 2024-09-10T08:58:53.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.715+0000 7f06f30c5740 1 -- 172.21.15.172:0/1208686040 wait complete. 2024-09-10T08:58:53.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.715+0000 7f06f30c5740 1 -- start start 2024-09-10T08:58:53.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.715+0000 7f06f30c5740 1 Processor -- start 2024-09-10T08:58:53.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.715+0000 7f06f30c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb77b5d0 0x55d8bb78a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.715+0000 7f06f30c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77cb40 0x55d8bb787540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.715+0000 7f06f30c5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb78a780 0x55d8bb787a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.027 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.715+0000 7f06f30c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d8bb78d480 con 0x55d8bb77cb40 2024-09-10T08:58:53.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.715+0000 7f06f30c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d8bb69c060 con 0x55d8bb77b5d0 2024-09-10T08:58:53.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.715+0000 7f06f30c5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d8bb78d600 con 0x55d8bb78a780 2024-09-10T08:58:53.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.715+0000 7f06f30c5740 10 client.0 _ll_register_callbacks cb 0x55d8bb3ef8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:53.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.715+0000 7f06e9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77cb40 0x55d8bb787540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.715+0000 7f06e9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77cb40 0x55d8bb787540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49602/0 (socket says 172.21.15.172:49602) 2024-09-10T08:58:53.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.715+0000 7f06e9d9b640 1 -- 172.21.15.172:0/4002844315 learned_addr learned my addr 172.21.15.172:0/4002844315 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:53.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.715+0000 7f06ea59c640 1 --2- 172.21.15.172:0/4002844315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb77b5d0 0x55d8bb78a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.716+0000 7f06e9d9b640 1 -- 172.21.15.172:0/4002844315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb78a780 msgr2=0x55d8bb787a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:53.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.716+0000 7f06ead9d640 1 --2- 172.21.15.172:0/4002844315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb78a780 0x55d8bb787a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.716+0000 7f06e9d9b640 1 --2- 172.21.15.172:0/4002844315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb78a780 0x55d8bb787a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.716+0000 7f06e9d9b640 1 -- 172.21.15.172:0/4002844315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb77b5d0 msgr2=0x55d8bb78a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:53.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.716+0000 7f06e9d9b640 1 --2- 172.21.15.172:0/4002844315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb77b5d0 0x55d8bb78a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.716+0000 7f06e9d9b640 1 -- 172.21.15.172:0/4002844315 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8bb4523c0 con 0x55d8bb77cb40 2024-09-10T08:58:53.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.716+0000 7f06ea59c640 1 --2- 172.21.15.172:0/4002844315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb77b5d0 0x55d8bb78a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:53.029 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:58:53.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.716+0000 7f06ead9d640 1 --2- 172.21.15.172:0/4002844315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb78a780 0x55d8bb787a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:53.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.716+0000 7f06e9d9b640 1 --2- 172.21.15.172:0/4002844315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77cb40 0x55d8bb787540 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f06e00b8b80 tx=0x7f06e00bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1459a75bc677ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:53.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.716+0000 7f06d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:53.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.716+0000 7f06d1ffb640 1 -- 172.21.15.172:0/4002844315 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0049040 con 0x55d8bb77cb40 2024-09-10T08:58:53.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.716+0000 7f06d1ffb640 1 -- 172.21.15.172:0/4002844315 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f06e00b8820 con 0x55d8bb77cb40 2024-09-10T08:58:53.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.716+0000 7f06f30c5740 1 -- 172.21.15.172:0/4002844315 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06dc018af0 con 0x55d8bb77cb40 2024-09-10T08:58:53.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.716+0000 7f06d1ffb640 1 -- 172.21.15.172:0/4002844315 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f06e00bc1b0 con 0x55d8bb77cb40 2024-09-10T08:58:53.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.716+0000 7f06f30c5740 10 client.19209 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:53.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.716+0000 7f06f30c5740 1 -- 172.21.15.172:0/4002844315 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d8bb782040 con 0x55d8bb77cb40 2024-09-10T08:58:53.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.717+0000 7f06d1ffb640 1 -- 172.21.15.172:0/4002844315 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06dc018af0 con 0x55d8bb77cb40 2024-09-10T08:58:53.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.717+0000 7f06d1ffb640 1 -- 172.21.15.172:0/4002844315 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f06e0003c50 con 0x55d8bb77cb40 2024-09-10T08:58:53.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.717+0000 7f06f30c5740 10 client.19209 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:53.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.717+0000 7f06f30c5740 10 client.19209 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:58:53.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.717+0000 7f06f30c5740 20 client.19209 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:53.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.717+0000 7f06f30c5740 1 --2- 172.21.15.172:0/4002844315 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55d8bb79e930 0x55d8bb7a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.717+0000 7f06f30c5740 4 client.19209 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:58:53.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.717+0000 7f06f30c5740 1 -- 172.21.15.172:0/4002844315 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8bb6ee860 con 0x55d8bb79e930 2024-09-10T08:58:53.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.717+0000 7f06ea59c640 1 --2- 172.21.15.172:0/4002844315 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55d8bb79e930 0x55d8bb7a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.717+0000 7f06ea59c640 1 --2- 172.21.15.172:0/4002844315 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55d8bb79e930 0x55d8bb7a0d50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50247983d4bb04f6 server_cookie=d2a94dd787c92f5e in_seq=0 out_seq=0 2024-09-10T08:58:53.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.717+0000 7f06d1ffb640 10 client.19209 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:58:53.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.718+0000 7f06d1ffb640 1 -- 172.21.15.172:0/4002844315 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8bb6ee860 con 0x55d8bb79e930 2024-09-10T08:58:53.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.718+0000 7f06d1ffb640 10 client.19209 handle_command_reply: tid=0 2024-09-10T08:58:53.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.718+0000 7f06f30c5740 1 client.19209 shutdown 2024-09-10T08:58:53.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.718+0000 7f06f30c5740 1 -- 172.21.15.172:0/4002844315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77cb40 msgr2=0x55d8bb787540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:53.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.718+0000 7f06f30c5740 1 --2- 172.21.15.172:0/4002844315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77cb40 0x55d8bb787540 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f06e00b8b80 tx=0x7f06e00bc040 comp rx=0 tx=0).stop 2024-09-10T08:58:53.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.718+0000 7f06f30c5740 1 -- 172.21.15.172:0/4002844315 shutdown_connections 2024-09-10T08:58:53.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.718+0000 7f06f30c5740 1 --2- 172.21.15.172:0/4002844315 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55d8bb79e930 0x55d8bb7a0d50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.718+0000 7f06f30c5740 1 --2- 172.21.15.172:0/4002844315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb78a780 0x55d8bb787a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.718+0000 7f06f30c5740 1 --2- 172.21.15.172:0/4002844315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77cb40 0x55d8bb787540 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.718+0000 7f06d1ffb640 0 client.19209 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:58:53.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.718+0000 7f06f30c5740 1 --2- 172.21.15.172:0/4002844315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb77b5d0 0x55d8bb78a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.718+0000 7f06f30c5740 1 -- 172.21.15.172:0/4002844315 >> 172.21.15.172:0/4002844315 conn(0x55d8bb744310 msgr2=0x55d8bb77d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:53.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.718+0000 7f06f30c5740 1 -- 172.21.15.172:0/4002844315 shutdown_connections 2024-09-10T08:58:53.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.718+0000 7f06f30c5740 1 -- 172.21.15.172:0/4002844315 wait complete. 2024-09-10T08:58:53.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.719+0000 7f06f30c5740 20 client.19209 trim_cache size 0 max 16384 2024-09-10T08:58:53.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.731+0000 7f06f30c5740 1 Processor -- start 2024-09-10T08:58:53.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.731+0000 7f06f30c5740 1 -- start start 2024-09-10T08:58:53.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.731+0000 7f06f30c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb7a6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.731+0000 7f06f30c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb77cb40 0x55d8bb7a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.731+0000 7f06f30c5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb78a780 0x55d8bb7847a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.731+0000 7f06f30c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1ac860 con 0x55d8bb77b5d0 2024-09-10T08:58:53.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.731+0000 7f06f30c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1ae430 con 0x55d8bb77cb40 2024-09-10T08:58:53.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.731+0000 7f06f30c5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1acf30 con 0x55d8bb78a780 2024-09-10T08:58:53.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.731+0000 7f06ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb7a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.731+0000 7f06ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb7a6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49614/0 (socket says 172.21.15.172:49614) 2024-09-10T08:58:53.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.731+0000 7f06ea59c640 1 -- 172.21.15.172:0/2143300871 learned_addr learned my addr 172.21.15.172:0/2143300871 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:53.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.732+0000 7f06e9d9b640 1 --2- 172.21.15.172:0/2143300871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb77cb40 0x55d8bb7a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.732+0000 7f06ead9d640 1 --2- 172.21.15.172:0/2143300871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb78a780 0x55d8bb7847a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.732+0000 7f06ea59c640 1 -- 172.21.15.172:0/2143300871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb78a780 msgr2=0x55d8bb7847a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:53.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.732+0000 7f06ea59c640 1 --2- 172.21.15.172:0/2143300871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb78a780 0x55d8bb7847a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.732+0000 7f06ea59c640 1 -- 172.21.15.172:0/2143300871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb77cb40 msgr2=0x55d8bb7a6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:53.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.732+0000 7f06ea59c640 1 --2- 172.21.15.172:0/2143300871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb77cb40 0x55d8bb7a6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.732+0000 7f06ea59c640 1 -- 172.21.15.172:0/2143300871 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8bb7888c0 con 0x55d8bb77b5d0 2024-09-10T08:58:53.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.732+0000 7f06e9d9b640 1 --2- 172.21.15.172:0/2143300871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb77cb40 0x55d8bb7a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:53.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.732+0000 7f06ead9d640 1 --2- 172.21.15.172:0/2143300871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb78a780 0x55d8bb7847a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:53.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.732+0000 7f06ea59c640 1 --2- 172.21.15.172:0/2143300871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb7a6620 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x55d8bb778ac0 tx=0x7f06dc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbc3217920b90b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:53.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.732+0000 7f06d3fff640 1 -- 172.21.15.172:0/2143300871 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc032070 con 0x55d8bb77b5d0 2024-09-10T08:58:53.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.732+0000 7f06d3fff640 1 -- 172.21.15.172:0/2143300871 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06dc005040 con 0x55d8bb77b5d0 2024-09-10T08:58:53.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.732+0000 7f06d3fff640 1 -- 172.21.15.172:0/2143300871 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc01d040 con 0x55d8bb77b5d0 2024-09-10T08:58:53.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.733+0000 7f06f30c5740 1 -- 172.21.15.172:0/2143300871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 msgr2=0x55d8bb7a6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:53.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.733+0000 7f06f30c5740 1 --2- 172.21.15.172:0/2143300871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb7a6620 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x55d8bb778ac0 tx=0x7f06dc010740 comp rx=0 tx=0).stop 2024-09-10T08:58:53.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.733+0000 7f06f30c5740 1 -- 172.21.15.172:0/2143300871 shutdown_connections 2024-09-10T08:58:53.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.733+0000 7f06f30c5740 1 --2- 172.21.15.172:0/2143300871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb78a780 0x55d8bb7847a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.733+0000 7f06f30c5740 1 --2- 172.21.15.172:0/2143300871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb77cb40 0x55d8bb7a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.733+0000 7f06f30c5740 1 --2- 172.21.15.172:0/2143300871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb7a6620 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.733+0000 7f06f30c5740 1 -- 172.21.15.172:0/2143300871 >> 172.21.15.172:0/2143300871 conn(0x55d8bb744310 msgr2=0x55d8bb7895c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.733+0000 7f06f30c5740 1 -- 172.21.15.172:0/2143300871 shutdown_connections 2024-09-10T08:58:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.733+0000 7f06f30c5740 1 -- 172.21.15.172:0/2143300871 wait complete. 2024-09-10T08:58:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.733+0000 7f06f30c5740 1 -- start start 2024-09-10T08:58:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06f30c5740 1 Processor -- start 2024-09-10T08:58:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06f30c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb7997d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06f30c5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb77cb40 0x55d8bb799d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06f30c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb78a780 0x55d8bb808600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06f30c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1add60 con 0x55d8bb77b5d0 2024-09-10T08:58:53.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06f30c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1ad690 con 0x55d8bb78a780 2024-09-10T08:58:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06f30c5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1aa840 con 0x55d8bb77cb40 2024-09-10T08:58:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06f30c5740 10 client.0 _ll_register_callbacks cb 0x7f06e0005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb7997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb7997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49622/0 (socket says 172.21.15.172:49622) 2024-09-10T08:58:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06ea59c640 1 -- 172.21.15.172:0/4136127349 learned_addr learned my addr 172.21.15.172:0/4136127349 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06e9d9b640 1 --2- 172.21.15.172:0/4136127349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb77cb40 0x55d8bb799d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06ead9d640 1 --2- 172.21.15.172:0/4136127349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb78a780 0x55d8bb808600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06ea59c640 1 -- 172.21.15.172:0/4136127349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb77cb40 msgr2=0x55d8bb799d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06ea59c640 1 --2- 172.21.15.172:0/4136127349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb77cb40 0x55d8bb799d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06ea59c640 1 -- 172.21.15.172:0/4136127349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb78a780 msgr2=0x55d8bb808600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06ea59c640 1 --2- 172.21.15.172:0/4136127349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb78a780 0x55d8bb808600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06ea59c640 1 -- 172.21.15.172:0/4136127349 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8bb808bf0 con 0x55d8bb77b5d0 2024-09-10T08:58:53.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06ea59c640 1 --2- 172.21.15.172:0/4136127349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb7997d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f06dc01b040 tx=0x7f06dc00aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1376368a1e14b781 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:53.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.734+0000 7f06d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:53.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.735+0000 7f06d1ffb640 1 -- 172.21.15.172:0/4136127349 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc032070 con 0x55d8bb77b5d0 2024-09-10T08:58:53.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.735+0000 7f06d1ffb640 1 -- 172.21.15.172:0/4136127349 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f06dc005570 con 0x55d8bb77b5d0 2024-09-10T08:58:53.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.735+0000 7f06f30c5740 1 -- 172.21.15.172:0/4136127349 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06dc005040 con 0x55d8bb77b5d0 2024-09-10T08:58:53.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.735+0000 7f06d1ffb640 1 -- 172.21.15.172:0/4136127349 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f06dc01d040 con 0x55d8bb77b5d0 2024-09-10T08:58:53.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.735+0000 7f06f30c5740 10 client.19215 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:53.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.735+0000 7f06f30c5740 1 -- 172.21.15.172:0/4136127349 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d8bb809e20 con 0x55d8bb77b5d0 2024-09-10T08:58:53.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.735+0000 7f06d1ffb640 1 -- 172.21.15.172:0/4136127349 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06dc005040 con 0x55d8bb77b5d0 2024-09-10T08:58:53.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.735+0000 7f06d1ffb640 1 -- 172.21.15.172:0/4136127349 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7f06dc010c30 con 0x55d8bb77b5d0 2024-09-10T08:58:53.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.735+0000 7f06f30c5740 10 client.19215 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:53.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.735+0000 7f06f30c5740 10 client.19215 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T08:58:53.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.735+0000 7f06f30c5740 20 client.19215 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:53.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.735+0000 7f06f30c5740 1 --2- 172.21.15.172:0/4136127349 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55d8bb79e930 0x55d8bb80c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:53.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.735+0000 7f06f30c5740 4 client.19215 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:58:53.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.735+0000 7f06f30c5740 1 -- 172.21.15.172:0/4136127349 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8bb6e8d70 con 0x55d8bb79e930 2024-09-10T08:58:53.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06e9d9b640 1 --2- 172.21.15.172:0/4136127349 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55d8bb79e930 0x55d8bb80c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:53.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06e9d9b640 1 --2- 172.21.15.172:0/4136127349 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55d8bb79e930 0x55d8bb80c370 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7eab484c43c0de2 server_cookie=c99feace9d64ea7f in_seq=0 out_seq=0 2024-09-10T08:58:53.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06d1ffb640 10 client.19215 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T08:58:53.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06d1ffb640 1 -- 172.21.15.172:0/4136127349 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8bb6e8d70 con 0x55d8bb79e930 2024-09-10T08:58:53.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06d1ffb640 10 client.19215 handle_command_reply: tid=0 2024-09-10T08:58:53.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06f30c5740 1 client.19215 shutdown 2024-09-10T08:58:53.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06f30c5740 1 -- 172.21.15.172:0/4136127349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 msgr2=0x55d8bb7997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:53.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06f30c5740 1 --2- 172.21.15.172:0/4136127349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb7997d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f06dc01b040 tx=0x7f06dc00aa10 comp rx=0 tx=0).stop 2024-09-10T08:58:53.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06f30c5740 1 -- 172.21.15.172:0/4136127349 shutdown_connections 2024-09-10T08:58:53.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06f30c5740 1 --2- 172.21.15.172:0/4136127349 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55d8bb79e930 0x55d8bb80c370 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06f30c5740 1 --2- 172.21.15.172:0/4136127349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bb78a780 0x55d8bb808600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06f30c5740 1 --2- 172.21.15.172:0/4136127349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8bb77cb40 0x55d8bb799d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06d1ffb640 0 client.19215 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T08:58:53.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06f30c5740 1 --2- 172.21.15.172:0/4136127349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8bb77b5d0 0x55d8bb7997d0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06f30c5740 1 -- 172.21.15.172:0/4136127349 >> 172.21.15.172:0/4136127349 conn(0x55d8bb744310 msgr2=0x55d8bb786e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:53.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06f30c5740 1 -- 172.21.15.172:0/4136127349 shutdown_connections 2024-09-10T08:58:53.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06f30c5740 1 -- 172.21.15.172:0/4136127349 wait complete. 2024-09-10T08:58:53.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.736+0000 7f06f30c5740 20 client.19215 trim_cache size 0 max 16384 2024-09-10T08:58:53.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.737+0000 7f06f0cec640 1 -- 172.21.15.172:0/3454193203 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f06d40a2b80 msgr2=0x7f06d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:53.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.737+0000 7f06f0cec640 1 --2- 172.21.15.172:0/3454193203 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f06d40a2b80 0x7f06d40a5040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f06e4007fd0 tx=0x7f06e40072a0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.737+0000 7f06f0cec640 1 -- 172.21.15.172:0/3454193203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06ec102570 msgr2=0x7f06ec1996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:53.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.737+0000 7f06f0cec640 1 --2- 172.21.15.172:0/3454193203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06ec102570 0x7f06ec1996d0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f06e0006010 tx=0x7f06e00042f0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.737+0000 7f06f0cec640 1 -- 172.21.15.172:0/3454193203 shutdown_connections 2024-09-10T08:58:53.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.737+0000 7f06f0cec640 1 --2- 172.21.15.172:0/3454193203 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f06d40a2b80 0x7f06d40a5040 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.737+0000 7f06f0cec640 1 --2- 172.21.15.172:0/3454193203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06ec102ef0 0x7f06ec1a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.737+0000 7f06f0cec640 1 --2- 172.21.15.172:0/3454193203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06ec102570 0x7f06ec1996d0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.737+0000 7f06f0cec640 1 --2- 172.21.15.172:0/3454193203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06ec100e20 0x7f06ec199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:53.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.737+0000 7f06f0cec640 1 -- 172.21.15.172:0/3454193203 >> 172.21.15.172:0/3454193203 conn(0x7f06ec0fc7d0 msgr2=0x7f06ec106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:53.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.737+0000 7f06f0cec640 1 -- 172.21.15.172:0/3454193203 shutdown_connections 2024-09-10T08:58:53.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:52.737+0000 7f06f0cec640 1 -- 172.21.15.172:0/3454193203 wait complete. 2024-09-10T08:58:53.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:53 smithi174 ceph-mon[34068]: pgmap v730: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9.5 MiB/s wr, 14 op/s 2024-09-10T08:58:53.751 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:53 smithi172 ceph-mon[30854]: pgmap v730: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9.5 MiB/s wr, 14 op/s 2024-09-10T08:58:53.782 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T08:58:53.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:53 smithi191 ceph-mon[35314]: pgmap v730: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 9.5 MiB/s wr, 14 op/s 2024-09-10T08:58:54.063 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:58:54.414 INFO:tasks.workunit.client.0.smithi172.stdout:fs setup took 609 secs 2024-09-10T08:58:55.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:55 smithi174 ceph-mon[34068]: pgmap v731: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 16 MiB/s wr, 20 op/s 2024-09-10T08:58:55.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:55 smithi191 ceph-mon[35314]: pgmap v731: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 16 MiB/s wr, 20 op/s 2024-09-10T08:58:55.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:55 smithi172 ceph-mon[30854]: pgmap v731: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 16 MiB/s wr, 20 op/s 2024-09-10T08:58:56.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.891+0000 7fcf31f81640 1 -- 172.21.15.172:0/2018309921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf2c102790 msgr2=0x7fcf2c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:56.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.891+0000 7fcf31f81640 1 --2- 172.21.15.172:0/2018309921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf2c102790 0x7fcf2c1067e0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009900 tx=0x7fcf1402a410 comp rx=0 tx=0).stop 2024-09-10T08:58:56.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.891+0000 7fcf31f81640 1 -- 172.21.15.172:0/2018309921 shutdown_connections 2024-09-10T08:58:56.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.891+0000 7fcf31f81640 1 --2- 172.21.15.172:0/2018309921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf2c106f10 0x7fcf2c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.891+0000 7fcf31f81640 1 --2- 172.21.15.172:0/2018309921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf2c102790 0x7fcf2c1067e0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.891+0000 7fcf31f81640 1 --2- 172.21.15.172:0/2018309921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf2c101de0 0x7fcf2c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.891+0000 7fcf31f81640 1 -- 172.21.15.172:0/2018309921 >> 172.21.15.172:0/2018309921 conn(0x7fcf2c0fd8c0 msgr2=0x7fcf2c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:56.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.891+0000 7fcf31f81640 1 -- 172.21.15.172:0/2018309921 shutdown_connections 2024-09-10T08:58:56.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.891+0000 7fcf31f81640 1 -- 172.21.15.172:0/2018309921 wait complete. 2024-09-10T08:58:56.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.891+0000 7fcf31f81640 1 Processor -- start 2024-09-10T08:58:56.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.891+0000 7fcf31f81640 1 -- start start 2024-09-10T08:58:56.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf31f81640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf2c101de0 0x7fcf2c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf31f81640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf2c102790 0x7fcf2c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf31f81640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf2c106f10 0x7fcf2c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf31f81640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c111e20 con 0x7fcf2c102790 2024-09-10T08:58:56.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf31f81640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c111ca0 con 0x7fcf2c106f10 2024-09-10T08:58:56.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf31f81640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c111fa0 con 0x7fcf2c101de0 2024-09-10T08:58:56.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf23fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf2c102790 0x7fcf2c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf23fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf2c102790 0x7fcf2c1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49626/0 (socket says 172.21.15.172:49626) 2024-09-10T08:58:56.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf23fff640 1 -- 172.21.15.172:0/3940586370 learned_addr learned my addr 172.21.15.172:0/3940586370 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:56.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf2bfff640 1 --2- 172.21.15.172:0/3940586370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf2c106f10 0x7fcf2c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf2b7fe640 1 --2- 172.21.15.172:0/3940586370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf2c101de0 0x7fcf2c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf23fff640 1 -- 172.21.15.172:0/3940586370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf2c101de0 msgr2=0x7fcf2c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:56.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf23fff640 1 --2- 172.21.15.172:0/3940586370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf2c101de0 0x7fcf2c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf23fff640 1 -- 172.21.15.172:0/3940586370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf2c106f10 msgr2=0x7fcf2c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:56.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf23fff640 1 --2- 172.21.15.172:0/3940586370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf2c106f10 0x7fcf2c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf23fff640 1 -- 172.21.15.172:0/3940586370 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2c19d350 con 0x7fcf2c102790 2024-09-10T08:58:56.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf23fff640 1 --2- 172.21.15.172:0/3940586370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf2c102790 0x7fcf2c1988e0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fcf14006010 tx=0x7fcf14004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:56.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.892+0000 7fcf297fa640 1 -- 172.21.15.172:0/3940586370 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1403f030 con 0x7fcf2c102790 2024-09-10T08:58:56.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.893+0000 7fcf31f81640 1 -- 172.21.15.172:0/3940586370 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2c19d5e0 con 0x7fcf2c102790 2024-09-10T08:58:56.443 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.893+0000 7fcf297fa640 1 -- 172.21.15.172:0/3940586370 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf140035f0 con 0x7fcf2c102790 2024-09-10T08:58:56.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.893+0000 7fcf297fa640 1 -- 172.21.15.172:0/3940586370 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14035070 con 0x7fcf2c102790 2024-09-10T08:58:56.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.893+0000 7fcf31f81640 1 -- 172.21.15.172:0/3940586370 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf2c1a4f20 con 0x7fcf2c102790 2024-09-10T08:58:56.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.895+0000 7fcf297fa640 1 -- 172.21.15.172:0/3940586370 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fcf14003b10 con 0x7fcf2c102790 2024-09-10T08:58:56.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.895+0000 7fcf3435a740 1 Processor -- start 2024-09-10T08:58:56.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.896+0000 7fcf3435a740 1 -- start start 2024-09-10T08:58:56.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.896+0000 7fcf3435a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f4086b40 0x5599f4085090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.896+0000 7fcf3435a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f40855d0 0x5599f4085a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.896+0000 7fcf3435a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f4092760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.896+0000 7fcf3435a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5599f3feb2f0 con 0x5599f4090370 2024-09-10T08:58:56.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.896+0000 7fcf3435a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5599f3f92cf0 con 0x5599f40855d0 2024-09-10T08:58:56.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.896+0000 7fcf3435a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5599f3fe72b0 con 0x5599f4086b40 2024-09-10T08:58:56.444 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.896+0000 7fcf297fa640 1 --2- 172.21.15.172:0/3940586370 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcf180a2b80 0x7fcf180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.896+0000 7fcf23fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f4092760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.896+0000 7fcf23fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f4092760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49634/0 (socket says 172.21.15.172:49634) 2024-09-10T08:58:56.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.896+0000 7fcf23fff640 1 -- 172.21.15.172:0/614837085 learned_addr learned my addr 172.21.15.172:0/614837085 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:56.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.896+0000 7fcf2b7fe640 1 --2- 172.21.15.172:0/614837085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f40855d0 0x5599f4085a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.896+0000 7fcf2bfff640 1 --2- 172.21.15.172:0/3940586370 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcf180a2b80 0x7fcf180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.896+0000 7fcf2bfff640 1 --2- 172.21.15.172:0/614837085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f4086b40 0x5599f4085090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.896+0000 7fcf297fa640 1 -- 172.21.15.172:0/3940586370 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcf140efff0 con 0x7fcf2c102790 2024-09-10T08:58:56.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.897+0000 7fcf23fff640 1 -- 172.21.15.172:0/614837085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f4086b40 msgr2=0x5599f4085090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:56.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.897+0000 7fcf23fff640 1 --2- 172.21.15.172:0/614837085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f4086b40 0x5599f4085090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.897+0000 7fcf23fff640 1 -- 172.21.15.172:0/614837085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f40855d0 msgr2=0x5599f4085a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:56.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.897+0000 7fcf23fff640 1 --2- 172.21.15.172:0/614837085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f40855d0 0x5599f4085a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.897+0000 7fcf23fff640 1 -- 172.21.15.172:0/614837085 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599f3f773a0 con 0x5599f4090370 2024-09-10T08:58:56.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.897+0000 7fcf2b7fe640 1 --2- 172.21.15.172:0/614837085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f40855d0 0x5599f4085a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:58:56.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.897+0000 7fcf23fff640 1 --2- 172.21.15.172:0/614837085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f4092760 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fcf140c4c10 tx=0x7fcf140e7060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b260e0cf259678 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:56.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.897+0000 7fcf2bfff640 1 --2- 172.21.15.172:0/3940586370 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcf180a2b80 0x7fcf180a5040 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcf24005cc0 tx=0x7fcf2400a340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:56.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.897+0000 7fcf21ffb640 1 -- 172.21.15.172:0/614837085 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14049030 con 0x5599f4090370 2024-09-10T08:58:56.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.897+0000 7fcf21ffb640 1 -- 172.21.15.172:0/614837085 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf140875b0 con 0x5599f4090370 2024-09-10T08:58:56.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.897+0000 7fcf21ffb640 1 -- 172.21.15.172:0/614837085 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf140b8b40 con 0x5599f4090370 2024-09-10T08:58:56.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.898+0000 7fcf3435a740 1 -- 172.21.15.172:0/614837085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 msgr2=0x5599f4092760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:56.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.898+0000 7fcf3435a740 1 --2- 172.21.15.172:0/614837085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f4092760 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fcf140c4c10 tx=0x7fcf140e7060 comp rx=0 tx=0).stop 2024-09-10T08:58:56.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.898+0000 7fcf3435a740 1 -- 172.21.15.172:0/614837085 shutdown_connections 2024-09-10T08:58:56.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.898+0000 7fcf3435a740 1 --2- 172.21.15.172:0/614837085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f4092760 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.898+0000 7fcf3435a740 1 --2- 172.21.15.172:0/614837085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f40855d0 0x5599f4085a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.446 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.898+0000 7fcf3435a740 1 --2- 172.21.15.172:0/614837085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f4086b40 0x5599f4085090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.898+0000 7fcf3435a740 1 -- 172.21.15.172:0/614837085 >> 172.21.15.172:0/614837085 conn(0x5599f404e310 msgr2=0x5599f4081c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.898+0000 7fcf3435a740 1 -- 172.21.15.172:0/614837085 shutdown_connections 2024-09-10T08:58:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.898+0000 7fcf3435a740 1 -- 172.21.15.172:0/614837085 wait complete. 2024-09-10T08:58:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.898+0000 7fcf3435a740 1 -- start start 2024-09-10T08:58:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.898+0000 7fcf3435a740 1 Processor -- start 2024-09-10T08:58:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf3435a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f40855d0 0x5599f408fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf3435a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f4086b40 0x5599f408d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf3435a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f408d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf3435a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5599f3feb2f0 con 0x5599f4090370 2024-09-10T08:58:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf3435a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5599f3fa6060 con 0x5599f4086b40 2024-09-10T08:58:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf3435a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5599f4092e30 con 0x5599f40855d0 2024-09-10T08:58:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf3435a740 10 client.0 _ll_register_callbacks cb 0x5599f3cf98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:56.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf2bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f408d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf2b7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f40855d0 0x5599f408fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf2bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f408d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49644/0 (socket says 172.21.15.172:49644) 2024-09-10T08:58:56.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf2bfff640 1 -- 172.21.15.172:0/376342429 learned_addr learned my addr 172.21.15.172:0/376342429 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:56.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf23fff640 1 --2- 172.21.15.172:0/376342429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f4086b40 0x5599f408d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf2bfff640 1 -- 172.21.15.172:0/376342429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f40855d0 msgr2=0x5599f408fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:56.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf2bfff640 1 --2- 172.21.15.172:0/376342429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f40855d0 0x5599f408fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf2bfff640 1 -- 172.21.15.172:0/376342429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f4086b40 msgr2=0x5599f408d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:56.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf2bfff640 1 --2- 172.21.15.172:0/376342429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f4086b40 0x5599f408d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf2bfff640 1 -- 172.21.15.172:0/376342429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599f3d5c3c0 con 0x5599f4090370 2024-09-10T08:58:56.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf23fff640 1 --2- 172.21.15.172:0/376342429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f4086b40 0x5599f408d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:58:56.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf2bfff640 1 --2- 172.21.15.172:0/376342429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f408d860 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fcf24011ab0 tx=0x7fcf24004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40434c7d83999d4b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:56.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf0affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:56.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf0affd640 1 -- 172.21.15.172:0/376342429 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24004b20 con 0x5599f4090370 2024-09-10T08:58:56.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf0affd640 1 -- 172.21.15.172:0/376342429 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcf24004cc0 con 0x5599f4090370 2024-09-10T08:58:56.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.899+0000 7fcf3435a740 1 -- 172.21.15.172:0/376342429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf140875b0 con 0x5599f4090370 2024-09-10T08:58:56.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.900+0000 7fcf0affd640 1 -- 172.21.15.172:0/376342429 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcf24005020 con 0x5599f4090370 2024-09-10T08:58:56.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.900+0000 7fcf3435a740 10 client.19227 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:56.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.900+0000 7fcf3435a740 1 -- 172.21.15.172:0/376342429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5599f409e880 con 0x5599f4090370 2024-09-10T08:58:56.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.900+0000 7fcf0affd640 1 -- 172.21.15.172:0/376342429 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf140875b0 con 0x5599f4090370 2024-09-10T08:58:56.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.900+0000 7fcf0affd640 1 -- 172.21.15.172:0/376342429 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fcf24027550 con 0x5599f4090370 2024-09-10T08:58:56.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.900+0000 7fcf3435a740 10 client.19227 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:56.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.900+0000 7fcf3435a740 10 client.19227 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:58:56.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.900+0000 7fcf3435a740 20 client.19227 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:56.449 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.900+0000 7fcf3435a740 1 --2- 172.21.15.172:0/376342429 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5599f409fd40 0x5599f40a2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.900+0000 7fcf3435a740 4 client.19227 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:58:56.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.900+0000 7fcf3435a740 1 -- 172.21.15.172:0/376342429 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599f3ff8860 con 0x5599f409fd40 2024-09-10T08:58:56.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.900+0000 7fcf2b7fe640 1 --2- 172.21.15.172:0/376342429 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5599f409fd40 0x5599f40a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.901+0000 7fcf0affd640 10 client.19227 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:58:56.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.901+0000 7fcf2b7fe640 1 --2- 172.21.15.172:0/376342429 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5599f409fd40 0x5599f40a2130 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e2b84cd9d8b039 server_cookie=d600b88b3aa826d4 in_seq=0 out_seq=0 2024-09-10T08:58:56.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.901+0000 7fcf0affd640 1 -- 172.21.15.172:0/376342429 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599f3ff8860 con 0x5599f409fd40 2024-09-10T08:58:56.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.901+0000 7fcf0affd640 10 client.19227 handle_command_reply: tid=0 2024-09-10T08:58:56.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.902+0000 7fcf3435a740 1 client.19227 shutdown 2024-09-10T08:58:56.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.902+0000 7fcf3435a740 1 -- 172.21.15.172:0/376342429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 msgr2=0x5599f408d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:56.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.902+0000 7fcf3435a740 1 --2- 172.21.15.172:0/376342429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f408d860 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fcf24011ab0 tx=0x7fcf24004ab0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.902+0000 7fcf3435a740 1 -- 172.21.15.172:0/376342429 shutdown_connections 2024-09-10T08:58:56.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.902+0000 7fcf3435a740 1 --2- 172.21.15.172:0/376342429 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5599f409fd40 0x5599f40a2130 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.450 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.902+0000 7fcf3435a740 1 --2- 172.21.15.172:0/376342429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f408d860 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.902+0000 7fcf3435a740 1 --2- 172.21.15.172:0/376342429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f4086b40 0x5599f408d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.902+0000 7fcf3435a740 1 --2- 172.21.15.172:0/376342429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f40855d0 0x5599f408fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.902+0000 7fcf3435a740 1 -- 172.21.15.172:0/376342429 >> 172.21.15.172:0/376342429 conn(0x5599f404e310 msgr2=0x5599f4087990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:56.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.902+0000 7fcf0affd640 0 client.19227 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:58:56.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.902+0000 7fcf3435a740 1 -- 172.21.15.172:0/376342429 shutdown_connections 2024-09-10T08:58:56.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.902+0000 7fcf3435a740 1 -- 172.21.15.172:0/376342429 wait complete. 2024-09-10T08:58:56.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.902+0000 7fcf3435a740 20 client.19227 trim_cache size 0 max 16384 2024-09-10T08:58:56.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf3435a740 1 Processor -- start 2024-09-10T08:58:56.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf3435a740 1 -- start start 2024-09-10T08:58:56.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf3435a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f40855d0 0x5599f40a79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.451 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf3435a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f4086b40 0x5599f40a7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf3435a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f4090370 0x5599f408a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf3435a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1aeb50 con 0x5599f40855d0 2024-09-10T08:58:56.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf3435a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1b0720 con 0x5599f4090370 2024-09-10T08:58:56.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf3435a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1af220 con 0x5599f4086b40 2024-09-10T08:58:56.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf2b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f40855d0 0x5599f40a79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf2b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f40855d0 0x5599f40a79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49648/0 (socket says 172.21.15.172:49648) 2024-09-10T08:58:56.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf2b7fe640 1 -- 172.21.15.172:0/1164322268 learned_addr learned my addr 172.21.15.172:0/1164322268 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:56.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf2bfff640 1 --2- 172.21.15.172:0/1164322268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f4090370 0x5599f408a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf23fff640 1 --2- 172.21.15.172:0/1164322268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f4086b40 0x5599f40a7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf2b7fe640 1 -- 172.21.15.172:0/1164322268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f4086b40 msgr2=0x5599f40a7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:56.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf2b7fe640 1 --2- 172.21.15.172:0/1164322268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f4086b40 0x5599f40a7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf2b7fe640 1 -- 172.21.15.172:0/1164322268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f4090370 msgr2=0x5599f408a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:56.452 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf2b7fe640 1 --2- 172.21.15.172:0/1164322268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f4090370 0x5599f408a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf2b7fe640 1 -- 172.21.15.172:0/1164322268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599f408b620 con 0x5599f40855d0 2024-09-10T08:58:56.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf2bfff640 1 --2- 172.21.15.172:0/1164322268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f4090370 0x5599f408a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:56.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.915+0000 7fcf23fff640 1 --2- 172.21.15.172:0/1164322268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f4086b40 0x5599f40a7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:56.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.916+0000 7fcf2b7fe640 1 --2- 172.21.15.172:0/1164322268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f40855d0 0x5599f40a79e0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00b980 tx=0x7fcf1c00b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52440f0e9b6220ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:56.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.916+0000 7fcf217fa640 1 -- 172.21.15.172:0/1164322268 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c020020 con 0x5599f40855d0 2024-09-10T08:58:56.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.916+0000 7fcf217fa640 1 -- 172.21.15.172:0/1164322268 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf1c00fe10 con 0x5599f40855d0 2024-09-10T08:58:56.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.916+0000 7fcf217fa640 1 -- 172.21.15.172:0/1164322268 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c01ab80 con 0x5599f40855d0 2024-09-10T08:58:56.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.916+0000 7fcf3435a740 1 -- 172.21.15.172:0/1164322268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f40855d0 msgr2=0x5599f40a79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:56.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.916+0000 7fcf3435a740 1 --2- 172.21.15.172:0/1164322268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f40855d0 0x5599f40a79e0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00b980 tx=0x7fcf1c00b340 comp rx=0 tx=0).stop 2024-09-10T08:58:56.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.916+0000 7fcf3435a740 1 -- 172.21.15.172:0/1164322268 shutdown_connections 2024-09-10T08:58:56.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.916+0000 7fcf3435a740 1 --2- 172.21.15.172:0/1164322268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f4090370 0x5599f408a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.453 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.916+0000 7fcf3435a740 1 --2- 172.21.15.172:0/1164322268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f4086b40 0x5599f40a7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.916+0000 7fcf3435a740 1 --2- 172.21.15.172:0/1164322268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f40855d0 0x5599f40a79e0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.916+0000 7fcf3435a740 1 -- 172.21.15.172:0/1164322268 >> 172.21.15.172:0/1164322268 conn(0x5599f404e310 msgr2=0x5599f408b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:56.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.916+0000 7fcf3435a740 1 -- 172.21.15.172:0/1164322268 shutdown_connections 2024-09-10T08:58:56.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.916+0000 7fcf3435a740 1 -- 172.21.15.172:0/1164322268 wait complete. 2024-09-10T08:58:56.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.917+0000 7fcf3435a740 1 -- start start 2024-09-10T08:58:56.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.917+0000 7fcf3435a740 1 Processor -- start 2024-09-10T08:58:56.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.917+0000 7fcf3435a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f40855d0 0x5599f4098900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.917+0000 7fcf3435a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f4086b40 0x5599f4093c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.917+0000 7fcf3435a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f4094220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.917+0000 7fcf3435a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1b0050 con 0x5599f4090370 2024-09-10T08:58:56.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.917+0000 7fcf3435a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1af980 con 0x5599f40855d0 2024-09-10T08:58:56.454 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.917+0000 7fcf3435a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1acb30 con 0x5599f4086b40 2024-09-10T08:58:56.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.917+0000 7fcf3435a740 10 client.0 _ll_register_callbacks cb 0x7fcf2400ae20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:58:56.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.917+0000 7fcf2bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f4094220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.917+0000 7fcf2bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f4094220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49664/0 (socket says 172.21.15.172:49664) 2024-09-10T08:58:56.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.917+0000 7fcf2bfff640 1 -- 172.21.15.172:0/2018164728 learned_addr learned my addr 172.21.15.172:0/2018164728 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:58:56.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.917+0000 7fcf2b7fe640 1 --2- 172.21.15.172:0/2018164728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f40855d0 0x5599f4098900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.918+0000 7fcf23fff640 1 --2- 172.21.15.172:0/2018164728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f4086b40 0x5599f4093c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.918+0000 7fcf2bfff640 1 -- 172.21.15.172:0/2018164728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f4086b40 msgr2=0x5599f4093c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:56.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.918+0000 7fcf2bfff640 1 --2- 172.21.15.172:0/2018164728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f4086b40 0x5599f4093c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.918+0000 7fcf2bfff640 1 -- 172.21.15.172:0/2018164728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f40855d0 msgr2=0x5599f4098900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:58:56.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.918+0000 7fcf2bfff640 1 --2- 172.21.15.172:0/2018164728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f40855d0 0x5599f4098900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.918+0000 7fcf2bfff640 1 -- 172.21.15.172:0/2018164728 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599f4094a20 con 0x5599f4090370 2024-09-10T08:58:56.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.918+0000 7fcf23fff640 1 --2- 172.21.15.172:0/2018164728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f4086b40 0x5599f4093c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:58:56.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.918+0000 7fcf2bfff640 1 --2- 172.21.15.172:0/2018164728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f4094220 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400a2a0 tx=0x7fcf24027e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec78051dc576bb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:58:56.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.918+0000 7fcf0affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:58:56.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.918+0000 7fcf0affd640 1 -- 172.21.15.172:0/2018164728 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24013510 con 0x5599f4090370 2024-09-10T08:58:56.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.918+0000 7fcf0affd640 1 -- 172.21.15.172:0/2018164728 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcf24004b20 con 0x5599f4090370 2024-09-10T08:58:56.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.918+0000 7fcf3435a740 1 -- 172.21.15.172:0/2018164728 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf1c00fe10 con 0x5599f4090370 2024-09-10T08:58:56.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.918+0000 7fcf0affd640 1 -- 172.21.15.172:0/2018164728 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcf24010800 con 0x5599f4090370 2024-09-10T08:58:56.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.918+0000 7fcf3435a740 10 client.19233 fetch_fsmap learned FSMap version 40 2024-09-10T08:58:56.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.918+0000 7fcf3435a740 1 -- 172.21.15.172:0/2018164728 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5599f410ee10 con 0x5599f4090370 2024-09-10T08:58:56.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.918+0000 7fcf0affd640 1 -- 172.21.15.172:0/2018164728 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf1c00fe10 con 0x5599f4090370 2024-09-10T08:58:56.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.919+0000 7fcf0affd640 1 -- 172.21.15.172:0/2018164728 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fcf2403d020 con 0x5599f4090370 2024-09-10T08:58:56.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.919+0000 7fcf3435a740 10 client.19233 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:58:56.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.919+0000 7fcf3435a740 10 client.19233 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T08:58:56.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.919+0000 7fcf3435a740 20 client.19233 populate_metadata read hostname 'smithi172' 2024-09-10T08:58:56.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.919+0000 7fcf3435a740 1 --2- 172.21.15.172:0/2018164728 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5599f410f520 0x5599f4111940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:58:56.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.919+0000 7fcf3435a740 4 client.19233 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:58:56.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.919+0000 7fcf3435a740 1 -- 172.21.15.172:0/2018164728 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599f3ff2d70 con 0x5599f410f520 2024-09-10T08:58:56.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.919+0000 7fcf2b7fe640 1 --2- 172.21.15.172:0/2018164728 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5599f410f520 0x5599f4111940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:58:56.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.919+0000 7fcf2b7fe640 1 --2- 172.21.15.172:0/2018164728 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5599f410f520 0x5599f4111940 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4661cf76659cc47 server_cookie=639ca8ea75ba81b5 in_seq=0 out_seq=0 2024-09-10T08:58:56.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.919+0000 7fcf0affd640 10 client.19233 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T08:58:56.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.919+0000 7fcf0affd640 1 -- 172.21.15.172:0/2018164728 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599f3ff2d70 con 0x5599f410f520 2024-09-10T08:58:56.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.919+0000 7fcf0affd640 10 client.19233 handle_command_reply: tid=0 2024-09-10T08:58:56.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.919+0000 7fcf3435a740 1 client.19233 shutdown 2024-09-10T08:58:56.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf3435a740 1 -- 172.21.15.172:0/2018164728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 msgr2=0x5599f4094220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:56.457 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf3435a740 1 --2- 172.21.15.172:0/2018164728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f4094220 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400a2a0 tx=0x7fcf24027e10 comp rx=0 tx=0).stop 2024-09-10T08:58:56.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf3435a740 1 -- 172.21.15.172:0/2018164728 shutdown_connections 2024-09-10T08:58:56.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf3435a740 1 --2- 172.21.15.172:0/2018164728 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5599f410f520 0x5599f4111940 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf3435a740 1 --2- 172.21.15.172:0/2018164728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5599f4090370 0x5599f4094220 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf3435a740 1 --2- 172.21.15.172:0/2018164728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5599f4086b40 0x5599f4093c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf0affd640 0 client.19233 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T08:58:56.458 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:58:56.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf3435a740 1 --2- 172.21.15.172:0/2018164728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599f40855d0 0x5599f4098900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf3435a740 1 -- 172.21.15.172:0/2018164728 >> 172.21.15.172:0/2018164728 conn(0x5599f404e310 msgr2=0x5599f408e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:56.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf3435a740 1 -- 172.21.15.172:0/2018164728 shutdown_connections 2024-09-10T08:58:56.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf3435a740 1 -- 172.21.15.172:0/2018164728 wait complete. 2024-09-10T08:58:56.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf3435a740 20 client.19233 trim_cache size 0 max 16384 2024-09-10T08:58:56.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf31f81640 1 -- 172.21.15.172:0/3940586370 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcf180a2b80 msgr2=0x7fcf180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:56.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf31f81640 1 --2- 172.21.15.172:0/3940586370 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcf180a2b80 0x7fcf180a5040 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcf24005cc0 tx=0x7fcf2400a340 comp rx=0 tx=0).stop 2024-09-10T08:58:56.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf31f81640 1 -- 172.21.15.172:0/3940586370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf2c102790 msgr2=0x7fcf2c1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:58:56.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf31f81640 1 --2- 172.21.15.172:0/3940586370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf2c102790 0x7fcf2c1988e0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fcf14006010 tx=0x7fcf14004270 comp rx=0 tx=0).stop 2024-09-10T08:58:56.459 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf31f81640 1 -- 172.21.15.172:0/3940586370 shutdown_connections 2024-09-10T08:58:56.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf31f81640 1 --2- 172.21.15.172:0/3940586370 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcf180a2b80 0x7fcf180a5040 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf31f81640 1 --2- 172.21.15.172:0/3940586370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf2c106f10 0x7fcf2c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf31f81640 1 --2- 172.21.15.172:0/3940586370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf2c102790 0x7fcf2c1988e0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf31f81640 1 --2- 172.21.15.172:0/3940586370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf2c101de0 0x7fcf2c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:58:56.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.920+0000 7fcf31f81640 1 -- 172.21.15.172:0/3940586370 >> 172.21.15.172:0/3940586370 conn(0x7fcf2c0fd8c0 msgr2=0x7fcf2c0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:58:56.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.921+0000 7fcf31f81640 1 -- 172.21.15.172:0/3940586370 shutdown_connections 2024-09-10T08:58:56.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:58:54.921+0000 7fcf31f81640 1 -- 172.21.15.172:0/3940586370 wait complete. 2024-09-10T08:58:56.832 INFO:tasks.workunit.client.0.smithi172.stdout:Syncing()...2 sec 2024-09-10T08:58:56.833 INFO:tasks.workunit.client.0.smithi172.stdout:Starting Actual Benchmark At: Tue Sep 10 08:58:56 2024 2024-09-10T08:58:56.833 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T08:58:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:57 smithi172 ceph-mon[30854]: pgmap v732: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 18 MiB/s wr, 26 op/s 2024-09-10T08:58:57.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:57 smithi174 ceph-mon[34068]: pgmap v732: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 18 MiB/s wr, 26 op/s 2024-09-10T08:58:57.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:57 smithi191 ceph-mon[35314]: pgmap v732: 129 pgs: 129 active+clean; 12 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 18 MiB/s wr, 26 op/s 2024-09-10T08:58:58.782 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T08:58:59.367 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:58:59.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:58:59 smithi174 ceph-mon[34068]: pgmap v733: 129 pgs: 129 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 335 op/s 2024-09-10T08:58:59.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:58:59 smithi191 ceph-mon[35314]: pgmap v733: 129 pgs: 129 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 335 op/s 2024-09-10T08:58:59.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:58:59 smithi172 ceph-mon[30854]: pgmap v733: 129 pgs: 129 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 335 op/s 2024-09-10T08:59:01.585 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:01 smithi172 ceph-mon[30854]: pgmap v734: 129 pgs: 129 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 758 op/s 2024-09-10T08:59:01.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:01 smithi174 ceph-mon[34068]: pgmap v734: 129 pgs: 129 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 758 op/s 2024-09-10T08:59:01.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:01 smithi191 ceph-mon[35314]: pgmap v734: 129 pgs: 129 active+clean; 13 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 20 MiB/s wr, 758 op/s 2024-09-10T08:59:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.874+0000 7fa316a2b640 1 -- 172.21.15.172:0/3369687762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3100ff810 msgr2=0x7fa3100ffc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.874+0000 7fa316a2b640 1 --2- 172.21.15.172:0/3369687762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3100ff810 0x7fa3100ffc10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa308009ca0 tx=0x7fa30802a940 comp rx=0 tx=0).stop 2024-09-10T08:59:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.874+0000 7fa316a2b640 1 -- 172.21.15.172:0/3369687762 shutdown_connections 2024-09-10T08:59:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.874+0000 7fa316a2b640 1 --2- 172.21.15.172:0/3369687762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa310101300 0x7fa310105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.874+0000 7fa316a2b640 1 --2- 172.21.15.172:0/3369687762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa310100940 0x7fa310100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.874+0000 7fa316a2b640 1 --2- 172.21.15.172:0/3369687762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3100ff810 0x7fa3100ffc10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.874+0000 7fa316a2b640 1 -- 172.21.15.172:0/3369687762 >> 172.21.15.172:0/3369687762 conn(0x7fa3100fb2e0 msgr2=0x7fa3100fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.877+0000 7fa316a2b640 1 -- 172.21.15.172:0/3369687762 shutdown_connections 2024-09-10T08:59:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.877+0000 7fa316a2b640 1 -- 172.21.15.172:0/3369687762 wait complete. 2024-09-10T08:59:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.878+0000 7fa316a2b640 1 Processor -- start 2024-09-10T08:59:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.879+0000 7fa316a2b640 1 -- start start 2024-09-10T08:59:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.880+0000 7fa316a2b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3100ff810 0x7fa310070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.880+0000 7fa316a2b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa310100940 0x7fa3100711f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.880+0000 7fa316a2b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa310101300 0x7fa3101ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.880+0000 7fa316a2b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa310113af0 con 0x7fa3100ff810 2024-09-10T08:59:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.880+0000 7fa316a2b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa310113970 con 0x7fa310101300 2024-09-10T08:59:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.880+0000 7fa316a2b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa310113c70 con 0x7fa310100940 2024-09-10T08:59:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.880+0000 7fa315228640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa310100940 0x7fa3100711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.880+0000 7fa315228640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa310100940 0x7fa3100711f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:41722/0 (socket says 172.21.15.172:41722) 2024-09-10T08:59:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.880+0000 7fa315228640 1 -- 172.21.15.172:0/3381562624 learned_addr learned my addr 172.21.15.172:0/3381562624 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.880+0000 7fa31622a640 1 --2- 172.21.15.172:0/3381562624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa310101300 0x7fa3101ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.880+0000 7fa315228640 1 -- 172.21.15.172:0/3381562624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa310101300 msgr2=0x7fa3101ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.880+0000 7fa315228640 1 --2- 172.21.15.172:0/3381562624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa310101300 0x7fa3101ae200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.880+0000 7fa315228640 1 -- 172.21.15.172:0/3381562624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3100ff810 msgr2=0x7fa310070cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T08:59:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.880+0000 7fa315228640 1 --2- 172.21.15.172:0/3381562624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3100ff810 0x7fa310070cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.880+0000 7fa315228640 1 -- 172.21.15.172:0/3381562624 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3101ae740 con 0x7fa310100940 2024-09-10T08:59:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.881+0000 7fa315228640 1 --2- 172.21.15.172:0/3381562624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa310100940 0x7fa3100711f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00e9c0 tx=0x7fa30c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.881+0000 7fa306ffd640 1 -- 172.21.15.172:0/3381562624 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c00cd80 con 0x7fa310100940 2024-09-10T08:59:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.881+0000 7fa306ffd640 1 -- 172.21.15.172:0/3381562624 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa30c0035f0 con 0x7fa310100940 2024-09-10T08:59:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.881+0000 7fa306ffd640 1 -- 172.21.15.172:0/3381562624 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c010690 con 0x7fa310100940 2024-09-10T08:59:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.881+0000 7fa316a2b640 1 -- 172.21.15.172:0/3381562624 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3101ae9d0 con 0x7fa310100940 2024-09-10T08:59:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.881+0000 7fa316a2b640 1 -- 172.21.15.172:0/3381562624 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3101aeec0 con 0x7fa310100940 2024-09-10T08:59:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.885+0000 7fa31aa28740 1 Processor -- start 2024-09-10T08:59:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.886+0000 7fa31aa28740 1 -- start start 2024-09-10T08:59:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.886+0000 7fa31aa28740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 0x56545659cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.886+0000 7fa31aa28740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 0x5654565a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.886+0000 7fa31aa28740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654565a2540 0x5654565a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.886+0000 7fa31aa28740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5654565002f0 con 0x5654565a2540 2024-09-10T08:59:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.886+0000 7fa31aa28740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5654564a8270 con 0x56545659d4b0 2024-09-10T08:59:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.886+0000 7fa31aa28740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5654564fc2b0 con 0x56545659cb70 2024-09-10T08:59:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.886+0000 7fa306ffd640 1 -- 172.21.15.172:0/3381562624 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa30c0037a0 con 0x7fa310100940 2024-09-10T08:59:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.887+0000 7fa31622a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 0x56545659cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.887+0000 7fa31622a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 0x56545659cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:41738/0 (socket says 172.21.15.172:41738) 2024-09-10T08:59:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.887+0000 7fa31622a640 1 -- 172.21.15.172:0/161378389 learned_addr learned my addr 172.21.15.172:0/161378389 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:01.888 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.887+0000 7fa315228640 1 --2- 172.21.15.172:0/161378389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654565a2540 0x5654565a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.888+0000 7fa306ffd640 1 --2- 172.21.15.172:0/3381562624 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa2fc0a2b80 0x7fa2fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.888+0000 7fa306ffd640 1 -- 172.21.15.172:0/3381562624 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa30c0cf850 con 0x7fa310100940 2024-09-10T08:59:01.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.888+0000 7fa315228640 1 -- 172.21.15.172:0/161378389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 msgr2=0x56545659cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.888+0000 7fa315228640 1 --2- 172.21.15.172:0/161378389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 0x56545659cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.888+0000 7fa315228640 1 -- 172.21.15.172:0/161378389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 msgr2=0x5654565a1f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:59:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.888+0000 7fa315a29640 1 --2- 172.21.15.172:0/161378389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 0x5654565a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.888+0000 7fa315228640 1 --2- 172.21.15.172:0/161378389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 0x5654565a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.888+0000 7fa315228640 1 -- 172.21.15.172:0/161378389 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654564a7340 con 0x5654565a2540 2024-09-10T08:59:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.888+0000 7fa315a29640 1 --2- 172.21.15.172:0/161378389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 0x5654565a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:59:01.890 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.888+0000 7fa31622a640 1 --2- 172.21.15.172:0/3381562624 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa2fc0a2b80 0x7fa2fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.889+0000 7fa315228640 1 --2- 172.21.15.172:0/161378389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654565a2540 0x5654565a4a00 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0ccf30 tx=0x7fa30c003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e10538f8b7d454e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.889+0000 7fa2fbfff640 1 -- 172.21.15.172:0/161378389 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c09b650 con 0x5654565a2540 2024-09-10T08:59:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.889+0000 7fa2fbfff640 1 -- 172.21.15.172:0/161378389 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa30c003b10 con 0x5654565a2540 2024-09-10T08:59:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.889+0000 7fa2fbfff640 1 -- 172.21.15.172:0/161378389 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0a44d0 con 0x5654565a2540 2024-09-10T08:59:01.891 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.890+0000 7fa31aa28740 1 -- 172.21.15.172:0/161378389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654565a2540 msgr2=0x5654565a4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.890+0000 7fa31622a640 1 --2- 172.21.15.172:0/3381562624 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa2fc0a2b80 0x7fa2fc0a5040 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa300006aa0 tx=0x7fa300008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.890+0000 7fa31aa28740 1 --2- 172.21.15.172:0/161378389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654565a2540 0x5654565a4a00 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0ccf30 tx=0x7fa30c003e10 comp rx=0 tx=0).stop 2024-09-10T08:59:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.891+0000 7fa31aa28740 1 -- 172.21.15.172:0/161378389 shutdown_connections 2024-09-10T08:59:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.891+0000 7fa31aa28740 1 --2- 172.21.15.172:0/161378389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654565a2540 0x5654565a4a00 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.891+0000 7fa31aa28740 1 --2- 172.21.15.172:0/161378389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 0x5654565a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.891+0000 7fa31aa28740 1 --2- 172.21.15.172:0/161378389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 0x56545659cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.892 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.891+0000 7fa31aa28740 1 -- 172.21.15.172:0/161378389 >> 172.21.15.172:0/161378389 conn(0x5654565981e0 msgr2=0x5654565985e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.891+0000 7fa31aa28740 1 -- 172.21.15.172:0/161378389 shutdown_connections 2024-09-10T08:59:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.891+0000 7fa31aa28740 1 -- 172.21.15.172:0/161378389 wait complete. 2024-09-10T08:59:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.892+0000 7fa31aa28740 1 -- start start 2024-09-10T08:59:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.892+0000 7fa31aa28740 1 Processor -- start 2024-09-10T08:59:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa31aa28740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 0x5654565a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa31aa28740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 0x5654565a7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa31aa28740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654565a2540 0x5654565ae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa31aa28740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5654565a54b0 con 0x5654565a2540 2024-09-10T08:59:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa31aa28740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565456489290 con 0x56545659d4b0 2024-09-10T08:59:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa31aa28740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5654565a5630 con 0x56545659cb70 2024-09-10T08:59:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa31aa28740 10 client.0 _ll_register_callbacks cb 0x5654564ba140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:59:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa315a29640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 0x5654565a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa315a29640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 0x5654565a6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:41750/0 (socket says 172.21.15.172:41750) 2024-09-10T08:59:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa315a29640 1 -- 172.21.15.172:0/1750655540 learned_addr learned my addr 172.21.15.172:0/1750655540 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa315228640 1 --2- 172.21.15.172:0/1750655540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 0x5654565a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa31622a640 1 --2- 172.21.15.172:0/1750655540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654565a2540 0x5654565ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa315a29640 1 -- 172.21.15.172:0/1750655540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 msgr2=0x5654565a7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa315a29640 1 --2- 172.21.15.172:0/1750655540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 0x5654565a7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa315a29640 1 -- 172.21.15.172:0/1750655540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654565a2540 msgr2=0x5654565ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa315a29640 1 --2- 172.21.15.172:0/1750655540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654565a2540 0x5654565ae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa315a29640 1 -- 172.21.15.172:0/1750655540 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565456213210 con 0x56545659cb70 2024-09-10T08:59:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.893+0000 7fa31622a640 1 --2- 172.21.15.172:0/1750655540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654565a2540 0x5654565ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:59:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.894+0000 7fa315a29640 1 --2- 172.21.15.172:0/1750655540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 0x5654565a6df0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa308005cc0 tx=0x7fa30802fed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bec613ae816a088 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.894+0000 7fa2f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:59:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.894+0000 7fa2f97fa640 1 -- 172.21.15.172:0/1750655540 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308034070 con 0x56545659cb70 2024-09-10T08:59:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.894+0000 7fa2f97fa640 1 -- 172.21.15.172:0/1750655540 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa308003970 con 0x56545659cb70 2024-09-10T08:59:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.895+0000 7fa31aa28740 1 -- 172.21.15.172:0/1750655540 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56545650da20 con 0x56545659cb70 2024-09-10T08:59:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.895+0000 7fa2f97fa640 1 -- 172.21.15.172:0/1750655540 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa308003b70 con 0x56545659cb70 2024-09-10T08:59:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.895+0000 7fa31aa28740 10 client.27955 fetch_fsmap learned FSMap version 40 2024-09-10T08:59:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.895+0000 7fa31aa28740 1 -- 172.21.15.172:0/1750655540 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5654562df620 con 0x56545659cb70 2024-09-10T08:59:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.895+0000 7fa2f97fa640 1 -- 172.21.15.172:0/1750655540 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56545650da20 con 0x56545659cb70 2024-09-10T08:59:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.896+0000 7fa2f97fa640 1 -- 172.21.15.172:0/1750655540 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fa308041020 con 0x56545659cb70 2024-09-10T08:59:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.897+0000 7fa31aa28740 10 client.27955 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:59:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.897+0000 7fa31aa28740 10 client.27955 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:59:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.897+0000 7fa31aa28740 20 client.27955 populate_metadata read hostname 'smithi172' 2024-09-10T08:59:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.897+0000 7fa31aa28740 1 --2- 172.21.15.172:0/1750655540 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5654565b2960 0x5654565b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.897+0000 7fa31aa28740 4 client.27955 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:59:01.899 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.897+0000 7fa31aa28740 1 -- 172.21.15.172:0/1750655540 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565456507d70 con 0x5654565b2960 2024-09-10T08:59:01.900 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.899+0000 7fa315228640 1 --2- 172.21.15.172:0/1750655540 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5654565b2960 0x5654565b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.900+0000 7fa315228640 1 --2- 172.21.15.172:0/1750655540 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5654565b2960 0x5654565b4dd0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddfdef6c1cef2f10 server_cookie=9f07a293e0692973 in_seq=0 out_seq=0 2024-09-10T08:59:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.900+0000 7fa2f97fa640 10 client.27955 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.901+0000 7fa2f97fa640 1 -- 172.21.15.172:0/1750655540 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565456507d70 con 0x5654565b2960 2024-09-10T08:59:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.901+0000 7fa2f97fa640 10 client.27955 handle_command_reply: tid=0 2024-09-10T08:59:01.902 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.901+0000 7fa31aa28740 1 client.27955 shutdown 2024-09-10T08:59:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.901+0000 7fa31aa28740 1 -- 172.21.15.172:0/1750655540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 msgr2=0x5654565a6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.901+0000 7fa31aa28740 1 --2- 172.21.15.172:0/1750655540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 0x5654565a6df0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa308005cc0 tx=0x7fa30802fed0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.902+0000 7fa31aa28740 1 -- 172.21.15.172:0/1750655540 shutdown_connections 2024-09-10T08:59:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.902+0000 7fa31aa28740 1 --2- 172.21.15.172:0/1750655540 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5654565b2960 0x5654565b4dd0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.902+0000 7fa31aa28740 1 --2- 172.21.15.172:0/1750655540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654565a2540 0x5654565ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.902+0000 7fa31aa28740 1 --2- 172.21.15.172:0/1750655540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 0x5654565a7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.902+0000 7fa31aa28740 1 --2- 172.21.15.172:0/1750655540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 0x5654565a6df0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.902+0000 7fa31aa28740 1 -- 172.21.15.172:0/1750655540 >> 172.21.15.172:0/1750655540 conn(0x5654565981e0 msgr2=0x5654565999f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.902+0000 7fa2f97fa640 0 client.27955 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.903+0000 7fa31aa28740 1 -- 172.21.15.172:0/1750655540 shutdown_connections 2024-09-10T08:59:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.903+0000 7fa31aa28740 1 -- 172.21.15.172:0/1750655540 wait complete. 2024-09-10T08:59:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.903+0000 7fa31aa28740 20 client.27955 trim_cache size 0 max 16384 2024-09-10T08:59:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.934+0000 7fa31aa28740 1 Processor -- start 2024-09-10T08:59:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.934+0000 7fa31aa28740 1 -- start start 2024-09-10T08:59:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.934+0000 7fa31aa28740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 0x5654565a73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.934+0000 7fa31aa28740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 0x5654565ae330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.934+0000 7fa31aa28740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654565a2540 0x5654565971a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.934+0000 7fa31aa28740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3101b9470 con 0x5654565a2540 2024-09-10T08:59:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.934+0000 7fa31aa28740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3101bb040 con 0x56545659d4b0 2024-09-10T08:59:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.934+0000 7fa31aa28740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa3101b9b40 con 0x56545659cb70 2024-09-10T08:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.935+0000 7fa315228640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 0x5654565ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.935+0000 7fa315228640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 0x5654565ae330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:50466/0 (socket says 172.21.15.172:50466) 2024-09-10T08:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.935+0000 7fa315228640 1 -- 172.21.15.172:0/3813384153 learned_addr learned my addr 172.21.15.172:0/3813384153 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.935+0000 7fa315a29640 1 --2- 172.21.15.172:0/3813384153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 0x5654565a73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.935+0000 7fa315228640 1 -- 172.21.15.172:0/3813384153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 msgr2=0x5654565a73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.935+0000 7fa315228640 1 --2- 172.21.15.172:0/3813384153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 0x5654565a73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.935+0000 7fa315228640 1 -- 172.21.15.172:0/3813384153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654565a2540 msgr2=0x5654565971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.935+0000 7fa315228640 1 --2- 172.21.15.172:0/3813384153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654565a2540 0x5654565971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.935+0000 7fa315228640 1 -- 172.21.15.172:0/3813384153 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565456598000 con 0x56545659d4b0 2024-09-10T08:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.935+0000 7fa315228640 1 --2- 172.21.15.172:0/3813384153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 0x5654565ae330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa30c099fa0 tx=0x7fa30c005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e38e275c853f71ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.936+0000 7fa2fb7fe640 1 -- 172.21.15.172:0/3813384153 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c09c3f0 con 0x56545659d4b0 2024-09-10T08:59:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.936+0000 7fa2fb7fe640 1 -- 172.21.15.172:0/3813384153 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa30c0a00a0 con 0x56545659d4b0 2024-09-10T08:59:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.936+0000 7fa2fb7fe640 1 -- 172.21.15.172:0/3813384153 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0a44d0 con 0x56545659d4b0 2024-09-10T08:59:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.937+0000 7fa31aa28740 1 -- 172.21.15.172:0/3813384153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 msgr2=0x5654565ae330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.937+0000 7fa31aa28740 1 --2- 172.21.15.172:0/3813384153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 0x5654565ae330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa30c099fa0 tx=0x7fa30c005f10 comp rx=0 tx=0).stop 2024-09-10T08:59:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.937+0000 7fa31aa28740 1 -- 172.21.15.172:0/3813384153 shutdown_connections 2024-09-10T08:59:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.937+0000 7fa31aa28740 1 --2- 172.21.15.172:0/3813384153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654565a2540 0x5654565971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.937+0000 7fa31aa28740 1 --2- 172.21.15.172:0/3813384153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56545659d4b0 0x5654565ae330 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.937+0000 7fa31aa28740 1 --2- 172.21.15.172:0/3813384153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659cb70 0x5654565a73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.937+0000 7fa31aa28740 1 -- 172.21.15.172:0/3813384153 >> 172.21.15.172:0/3813384153 conn(0x5654565981e0 msgr2=0x565456596a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:01.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.938+0000 7fa31aa28740 1 -- 172.21.15.172:0/3813384153 shutdown_connections 2024-09-10T08:59:01.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.938+0000 7fa31aa28740 1 -- 172.21.15.172:0/3813384153 wait complete. 2024-09-10T08:59:01.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.938+0000 7fa31aa28740 1 -- start start 2024-09-10T08:59:01.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.938+0000 7fa31aa28740 1 Processor -- start 2024-09-10T08:59:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.940+0000 7fa31aa28740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56545659cb70 0x5654565ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.940+0000 7fa31aa28740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659d4b0 0x5654565abf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.940+0000 7fa31aa28740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654565a2540 0x5654565acf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.940+0000 7fa31aa28740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3101ba970 con 0x56545659cb70 2024-09-10T08:59:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.940+0000 7fa31aa28740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3101ba2a0 con 0x5654565a2540 2024-09-10T08:59:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.940+0000 7fa31aa28740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa3101b7450 con 0x56545659d4b0 2024-09-10T08:59:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.940+0000 7fa31aa28740 10 client.0 _ll_register_callbacks cb 0x7fa30c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:59:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.940+0000 7fa315228640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659d4b0 0x5654565abf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.940+0000 7fa315228640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659d4b0 0x5654565abf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:41760/0 (socket says 172.21.15.172:41760) 2024-09-10T08:59:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.940+0000 7fa315228640 1 -- 172.21.15.172:0/1103237558 learned_addr learned my addr 172.21.15.172:0/1103237558 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.941+0000 7fa31622a640 1 --2- 172.21.15.172:0/1103237558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654565a2540 0x5654565acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.941+0000 7fa315a29640 1 --2- 172.21.15.172:0/1103237558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56545659cb70 0x5654565ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.941+0000 7fa315228640 1 -- 172.21.15.172:0/1103237558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654565a2540 msgr2=0x5654565acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.941+0000 7fa315228640 1 --2- 172.21.15.172:0/1103237558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654565a2540 0x5654565acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.941+0000 7fa315228640 1 -- 172.21.15.172:0/1103237558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56545659cb70 msgr2=0x5654565ab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.941+0000 7fa315228640 1 --2- 172.21.15.172:0/1103237558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56545659cb70 0x5654565ab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.941+0000 7fa315228640 1 -- 172.21.15.172:0/1103237558 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654565ad530 con 0x56545659d4b0 2024-09-10T08:59:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.941+0000 7fa315228640 1 --2- 172.21.15.172:0/1103237558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659d4b0 0x5654565abf10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0986c0 tx=0x7fa30c09bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47d91ede5f8fd677 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.941+0000 7fa2f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.942+0000 7fa2f97fa640 1 -- 172.21.15.172:0/1103237558 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0cd0a0 con 0x56545659d4b0 2024-09-10T08:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.942+0000 7fa2f97fa640 1 -- 172.21.15.172:0/1103237558 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa30c0a44d0 con 0x56545659d4b0 2024-09-10T08:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.942+0000 7fa31aa28740 1 -- 172.21.15.172:0/1103237558 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa30c0a00a0 con 0x56545659d4b0 2024-09-10T08:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.942+0000 7fa2f97fa640 1 -- 172.21.15.172:0/1103237558 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa30c0a46d0 con 0x56545659d4b0 2024-09-10T08:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.942+0000 7fa31aa28740 10 client.27961 fetch_fsmap learned FSMap version 40 2024-09-10T08:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.942+0000 7fa31aa28740 1 -- 172.21.15.172:0/1103237558 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56545662e6c0 con 0x56545659d4b0 2024-09-10T08:59:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.943+0000 7fa2f97fa640 1 -- 172.21.15.172:0/1103237558 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa30c0a00a0 con 0x56545659d4b0 2024-09-10T08:59:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.943+0000 7fa2f97fa640 1 -- 172.21.15.172:0/1103237558 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fa30c09abb0 con 0x56545659d4b0 2024-09-10T08:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.944+0000 7fa31aa28740 10 client.27961 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.944+0000 7fa31aa28740 10 client.27961 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.944+0000 7fa31aa28740 20 client.27961 populate_metadata read hostname 'smithi172' 2024-09-10T08:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.944+0000 7fa31aa28740 1 --2- 172.21.15.172:0/1103237558 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56545662eab0 0x565456630ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.944+0000 7fa31aa28740 4 client.27961 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.944+0000 7fa31aa28740 1 -- 172.21.15.172:0/1103237558 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565456484900 con 0x56545662eab0 2024-09-10T08:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.944+0000 7fa315a29640 1 --2- 172.21.15.172:0/1103237558 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56545662eab0 0x565456630ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.945+0000 7fa2f97fa640 10 client.27961 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.945+0000 7fa315a29640 1 --2- 172.21.15.172:0/1103237558 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56545662eab0 0x565456630ea0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8aca41659e20eb0 server_cookie=f12d0a00c32730d0 in_seq=0 out_seq=0 2024-09-10T08:59:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.949+0000 7fa2f97fa640 1 -- 172.21.15.172:0/1103237558 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565456484900 con 0x56545662eab0 2024-09-10T08:59:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.949+0000 7fa2f97fa640 10 client.27961 handle_command_reply: tid=0 2024-09-10T08:59:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.950+0000 7fa31aa28740 1 client.27961 shutdown 2024-09-10T08:59:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.950+0000 7fa31aa28740 1 -- 172.21.15.172:0/1103237558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659d4b0 msgr2=0x5654565abf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.950+0000 7fa31aa28740 1 --2- 172.21.15.172:0/1103237558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659d4b0 0x5654565abf10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0986c0 tx=0x7fa30c09bd60 comp rx=0 tx=0).stop 2024-09-10T08:59:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.950+0000 7fa31aa28740 1 -- 172.21.15.172:0/1103237558 shutdown_connections 2024-09-10T08:59:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.950+0000 7fa31aa28740 1 --2- 172.21.15.172:0/1103237558 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56545662eab0 0x565456630ea0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.950+0000 7fa31aa28740 1 --2- 172.21.15.172:0/1103237558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5654565a2540 0x5654565acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.950+0000 7fa31aa28740 1 --2- 172.21.15.172:0/1103237558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56545659d4b0 0x5654565abf10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.950+0000 7fa31aa28740 1 --2- 172.21.15.172:0/1103237558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56545659cb70 0x5654565ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.950+0000 7fa31aa28740 1 -- 172.21.15.172:0/1103237558 >> 172.21.15.172:0/1103237558 conn(0x5654565981e0 msgr2=0x5654565972b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:01.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.950+0000 7fa2f97fa640 0 client.27961 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.950+0000 7fa31aa28740 1 -- 172.21.15.172:0/1103237558 shutdown_connections 2024-09-10T08:59:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.950+0000 7fa31aa28740 1 -- 172.21.15.172:0/1103237558 wait complete. 2024-09-10T08:59:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.952+0000 7fa31aa28740 20 client.27961 trim_cache size 0 max 16384 2024-09-10T08:59:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:59:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:59:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:59:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:59:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.953+0000 7fa316a2b640 1 -- 172.21.15.172:0/3381562624 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa2fc0a2b80 msgr2=0x7fa2fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.953+0000 7fa316a2b640 1 --2- 172.21.15.172:0/3381562624 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa2fc0a2b80 0x7fa2fc0a5040 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa300006aa0 tx=0x7fa300008040 comp rx=0 tx=0).stop 2024-09-10T08:59:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.953+0000 7fa316a2b640 1 -- 172.21.15.172:0/3381562624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa310100940 msgr2=0x7fa3100711f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.953+0000 7fa316a2b640 1 --2- 172.21.15.172:0/3381562624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa310100940 0x7fa3100711f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00e9c0 tx=0x7fa30c00ee80 comp rx=0 tx=0).stop 2024-09-10T08:59:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.953+0000 7fa316a2b640 1 -- 172.21.15.172:0/3381562624 shutdown_connections 2024-09-10T08:59:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.953+0000 7fa316a2b640 1 --2- 172.21.15.172:0/3381562624 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa2fc0a2b80 0x7fa2fc0a5040 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.953+0000 7fa316a2b640 1 --2- 172.21.15.172:0/3381562624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa310101300 0x7fa3101ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.953+0000 7fa316a2b640 1 --2- 172.21.15.172:0/3381562624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa310100940 0x7fa3100711f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.953+0000 7fa316a2b640 1 --2- 172.21.15.172:0/3381562624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3100ff810 0x7fa310070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.953+0000 7fa316a2b640 1 -- 172.21.15.172:0/3381562624 >> 172.21.15.172:0/3381562624 conn(0x7fa3100fb2e0 msgr2=0x7fa3100fbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.953+0000 7fa316a2b640 1 -- 172.21.15.172:0/3381562624 shutdown_connections 2024-09-10T08:59:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:01.953+0000 7fa316a2b640 1 -- 172.21.15.172:0/3381562624 wait complete. 2024-09-10T08:59:03.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:03 smithi174 ceph-mon[34068]: pgmap v735: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 1.28k op/s 2024-09-10T08:59:03.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:03 smithi191 ceph-mon[35314]: pgmap v735: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 1.28k op/s 2024-09-10T08:59:03.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:03 smithi172 ceph-mon[30854]: pgmap v735: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 MiB/s wr, 1.28k op/s 2024-09-10T08:59:05.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:05 smithi174 ceph-mon[34068]: pgmap v736: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 2.15k op/s 2024-09-10T08:59:05.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:05 smithi191 ceph-mon[35314]: pgmap v736: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 2.15k op/s 2024-09-10T08:59:05.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:05 smithi172 ceph-mon[30854]: pgmap v736: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 18 MiB/s wr, 2.15k op/s 2024-09-10T08:59:06.248 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T08:59:06.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.423+0000 7fd13c567640 1 -- 172.21.15.172:0/2091417410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd134101420 msgr2=0x7fd13410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:06.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.423+0000 7fd13c567640 1 --2- 172.21.15.172:0/2091417410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd134101420 0x7fd13410d950 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009cd0 tx=0x7fd12c02c7e0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.249 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.423+0000 7fd13c567640 1 -- 172.21.15.172:0/2091417410 shutdown_connections 2024-09-10T08:59:06.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.423+0000 7fd13c567640 1 --2- 172.21.15.172:0/2091417410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd134101420 0x7fd13410d950 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.423+0000 7fd13c567640 1 --2- 172.21.15.172:0/2091417410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd134100a80 0x7fd134100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.423+0000 7fd13c567640 1 --2- 172.21.15.172:0/2091417410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd134106a80 0x7fd134106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.423+0000 7fd13c567640 1 -- 172.21.15.172:0/2091417410 >> 172.21.15.172:0/2091417410 conn(0x7fd1340fc7a0 msgr2=0x7fd1340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:06.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.423+0000 7fd13c567640 1 -- 172.21.15.172:0/2091417410 shutdown_connections 2024-09-10T08:59:06.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.423+0000 7fd13c567640 1 -- 172.21.15.172:0/2091417410 wait complete. 2024-09-10T08:59:06.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.424+0000 7fd13c567640 1 Processor -- start 2024-09-10T08:59:06.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.424+0000 7fd13c567640 1 -- start start 2024-09-10T08:59:06.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.425+0000 7fd13c567640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd134100a80 0x7fd134196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.425+0000 7fd13c567640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd134101420 0x7fd134196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.250 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.425+0000 7fd13c567640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd134106a80 0x7fd13419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.425+0000 7fd13c567640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd13410fc20 con 0x7fd134100a80 2024-09-10T08:59:06.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.425+0000 7fd13c567640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd13410faa0 con 0x7fd134106a80 2024-09-10T08:59:06.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.425+0000 7fd13c567640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd13410fda0 con 0x7fd134101420 2024-09-10T08:59:06.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.425+0000 7fd13a303640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd134100a80 0x7fd134196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.425+0000 7fd13a303640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd134100a80 0x7fd134196140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40720/0 (socket says 172.21.15.172:40720) 2024-09-10T08:59:06.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.425+0000 7fd13a303640 1 -- 172.21.15.172:0/845507219 learned_addr learned my addr 172.21.15.172:0/845507219 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:06.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.425+0000 7fd13ab04640 1 --2- 172.21.15.172:0/845507219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd134106a80 0x7fd13419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.425+0000 7fd139b02640 1 --2- 172.21.15.172:0/845507219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd134101420 0x7fd134196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.425+0000 7fd13a303640 1 -- 172.21.15.172:0/845507219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd134101420 msgr2=0x7fd134196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:06.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.425+0000 7fd13a303640 1 --2- 172.21.15.172:0/845507219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd134101420 0x7fd134196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.425+0000 7fd13a303640 1 -- 172.21.15.172:0/845507219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd134106a80 msgr2=0x7fd13419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:06.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.425+0000 7fd13a303640 1 --2- 172.21.15.172:0/845507219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd134106a80 0x7fd13419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.425+0000 7fd13a303640 1 -- 172.21.15.172:0/845507219 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13419b0f0 con 0x7fd134100a80 2024-09-10T08:59:06.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.425+0000 7fd13a303640 1 --2- 172.21.15.172:0/845507219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd134100a80 0x7fd134196140 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fd13000e9f0 tx=0x7fd13000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:06.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.426+0000 7fd12b7fe640 1 -- 172.21.15.172:0/845507219 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13000cd80 con 0x7fd134100a80 2024-09-10T08:59:06.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.426+0000 7fd12b7fe640 1 -- 172.21.15.172:0/845507219 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1300054d0 con 0x7fd134100a80 2024-09-10T08:59:06.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.426+0000 7fd13c567640 1 -- 172.21.15.172:0/845507219 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13419b3e0 con 0x7fd134100a80 2024-09-10T08:59:06.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.426+0000 7fd13c567640 1 -- 172.21.15.172:0/845507219 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1341a2cc0 con 0x7fd134100a80 2024-09-10T08:59:06.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.426+0000 7fd12b7fe640 1 -- 172.21.15.172:0/845507219 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd130010490 con 0x7fd134100a80 2024-09-10T08:59:06.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.428+0000 7fd12b7fe640 1 -- 172.21.15.172:0/845507219 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd130005020 con 0x7fd134100a80 2024-09-10T08:59:06.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.429+0000 7fd13e940740 1 Processor -- start 2024-09-10T08:59:06.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.429+0000 7fd12b7fe640 1 --2- 172.21.15.172:0/845507219 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1200a2c00 0x7fd1200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.429+0000 7fd12b7fe640 1 -- 172.21.15.172:0/845507219 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd1300cf740 con 0x7fd134100a80 2024-09-10T08:59:06.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.429+0000 7fd13ab04640 1 --2- 172.21.15.172:0/845507219 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1200a2c00 0x7fd1200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.429+0000 7fd13e940740 1 -- start start 2024-09-10T08:59:06.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.429+0000 7fd13e940740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2adb40 0x563cef2ac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.429+0000 7fd13e940740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2ac5d0 0x563cef2aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.429+0000 7fd13ab04640 1 --2- 172.21.15.172:0/845507219 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1200a2c00 0x7fd1200a50c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd12c032040 tx=0x7fd12c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:06.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.429+0000 7fd13e940740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2b7370 0x563cef2b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.429+0000 7fd13e940740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563cef2122f0 con 0x563cef2b7370 2024-09-10T08:59:06.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd13e940740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563cef1b9cf0 con 0x563cef2ac5d0 2024-09-10T08:59:06.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd13e940740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563cef20e2b0 con 0x563cef2adb40 2024-09-10T08:59:06.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd13ab04640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2b7370 0x563cef2b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd13ab04640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2b7370 0x563cef2b9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40732/0 (socket says 172.21.15.172:40732) 2024-09-10T08:59:06.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd13ab04640 1 -- 172.21.15.172:0/3402255733 learned_addr learned my addr 172.21.15.172:0/3402255733 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:06.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd13a303640 1 --2- 172.21.15.172:0/3402255733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2adb40 0x563cef2ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd139b02640 1 --2- 172.21.15.172:0/3402255733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2ac5d0 0x563cef2aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd13ab04640 1 -- 172.21.15.172:0/3402255733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2adb40 msgr2=0x563cef2ac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:06.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd13ab04640 1 --2- 172.21.15.172:0/3402255733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2adb40 0x563cef2ac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd13ab04640 1 -- 172.21.15.172:0/3402255733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2ac5d0 msgr2=0x563cef2aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:06.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd13ab04640 1 --2- 172.21.15.172:0/3402255733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2ac5d0 0x563cef2aca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd13ab04640 1 -- 172.21.15.172:0/3402255733 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cef19e3a0 con 0x563cef2b7370 2024-09-10T08:59:06.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd139b02640 1 --2- 172.21.15.172:0/3402255733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2ac5d0 0x563cef2aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:59:06.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd13a303640 1 --2- 172.21.15.172:0/3402255733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2adb40 0x563cef2ac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:59:06.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd13ab04640 1 --2- 172.21.15.172:0/3402255733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2b7370 0x563cef2b9760 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fd12c032530 tx=0x7fd12c042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df1a6e8d044e150 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:06.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd128ff9640 1 -- 172.21.15.172:0/3402255733 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c032810 con 0x563cef2b7370 2024-09-10T08:59:06.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd128ff9640 1 -- 172.21.15.172:0/3402255733 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd12c04e070 con 0x563cef2b7370 2024-09-10T08:59:06.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.430+0000 7fd128ff9640 1 -- 172.21.15.172:0/3402255733 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c031c90 con 0x563cef2b7370 2024-09-10T08:59:06.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.431+0000 7fd13e940740 1 -- 172.21.15.172:0/3402255733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2b7370 msgr2=0x563cef2b9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:06.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.431+0000 7fd13e940740 1 --2- 172.21.15.172:0/3402255733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2b7370 0x563cef2b9760 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fd12c032530 tx=0x7fd12c042000 comp rx=0 tx=0).stop 2024-09-10T08:59:06.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.431+0000 7fd13e940740 1 -- 172.21.15.172:0/3402255733 shutdown_connections 2024-09-10T08:59:06.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.431+0000 7fd13e940740 1 --2- 172.21.15.172:0/3402255733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2b7370 0x563cef2b9760 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.431+0000 7fd13e940740 1 --2- 172.21.15.172:0/3402255733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2ac5d0 0x563cef2aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.431+0000 7fd13e940740 1 --2- 172.21.15.172:0/3402255733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2adb40 0x563cef2ac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.431+0000 7fd13e940740 1 -- 172.21.15.172:0/3402255733 >> 172.21.15.172:0/3402255733 conn(0x563cef275310 msgr2=0x563cef2a8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:06.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.432+0000 7fd13e940740 1 -- 172.21.15.172:0/3402255733 shutdown_connections 2024-09-10T08:59:06.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.432+0000 7fd13e940740 1 -- 172.21.15.172:0/3402255733 wait complete. 2024-09-10T08:59:06.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.432+0000 7fd13e940740 1 -- start start 2024-09-10T08:59:06.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.432+0000 7fd13e940740 1 Processor -- start 2024-09-10T08:59:06.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd13e940740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2ac5d0 0x563cef2b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd13e940740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2adb40 0x563cef2b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd13e940740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2b7370 0x563cef2b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd139b02640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2adb40 0x563cef2b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd13a303640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2ac5d0 0x563cef2b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd139b02640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2adb40 0x563cef2b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40740/0 (socket says 172.21.15.172:40740) 2024-09-10T08:59:06.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd13a303640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2ac5d0 0x563cef2b6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:41794/0 (socket says 172.21.15.172:41794) 2024-09-10T08:59:06.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd139b02640 1 -- 172.21.15.172:0/121435861 learned_addr learned my addr 172.21.15.172:0/121435861 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:06.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd13e940740 1 -- 172.21.15.172:0/121435861 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563cef2ba060 con 0x563cef2adb40 2024-09-10T08:59:06.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd13e940740 1 -- 172.21.15.172:0/121435861 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563cef1cd060 con 0x563cef2b7370 2024-09-10T08:59:06.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd13e940740 1 -- 172.21.15.172:0/121435861 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563cef2ba1e0 con 0x563cef2ac5d0 2024-09-10T08:59:06.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd13ab04640 1 --2- 172.21.15.172:0/121435861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2b7370 0x563cef2b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd139b02640 1 -- 172.21.15.172:0/121435861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2ac5d0 msgr2=0x563cef2b6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:06.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd139b02640 1 --2- 172.21.15.172:0/121435861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2ac5d0 0x563cef2b6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd139b02640 1 -- 172.21.15.172:0/121435861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2b7370 msgr2=0x563cef2b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:06.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd139b02640 1 --2- 172.21.15.172:0/121435861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2b7370 0x563cef2b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd139b02640 1 -- 172.21.15.172:0/121435861 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ceef833c0 con 0x563cef2adb40 2024-09-10T08:59:06.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd139b02640 1 --2- 172.21.15.172:0/121435861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2adb40 0x563cef2b3fc0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fd12400de10 tx=0x7fd12400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b84bfc0627f4d06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:06.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.433+0000 7fd11dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:59:06.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.434+0000 7fd13e940740 10 client.0 _ll_register_callbacks cb 0x563ceef208c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:59:06.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.434+0000 7fd13e940740 1 -- 172.21.15.172:0/121435861 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd12c04e070 con 0x563cef2adb40 2024-09-10T08:59:06.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.434+0000 7fd11dffb640 1 -- 172.21.15.172:0/121435861 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124005020 con 0x563cef2adb40 2024-09-10T08:59:06.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.434+0000 7fd11dffb640 1 -- 172.21.15.172:0/121435861 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd1240051c0 con 0x563cef2adb40 2024-09-10T08:59:06.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.434+0000 7fd11dffb640 1 -- 172.21.15.172:0/121435861 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd1240053c0 con 0x563cef2adb40 2024-09-10T08:59:06.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.434+0000 7fd13e940740 10 client.19254 fetch_fsmap learned FSMap version 40 2024-09-10T08:59:06.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.434+0000 7fd13e940740 1 -- 172.21.15.172:0/121435861 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=40}) -- 0x563cef2c9d50 con 0x563cef2adb40 2024-09-10T08:59:06.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.434+0000 7fd11dffb640 1 -- 172.21.15.172:0/121435861 <== mon.0 v2:172.21.15.172:3300/0 4 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fd1240047a0 con 0x563cef2adb40 2024-09-10T08:59:06.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.434+0000 7fd13e940740 10 client.19254 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:59:06.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.434+0000 7fd13e940740 10 client.19254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:59:06.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.434+0000 7fd13e940740 20 client.19254 populate_metadata read hostname 'smithi172' 2024-09-10T08:59:06.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.434+0000 7fd13e940740 1 --2- 172.21.15.172:0/121435861 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563cef2cb170 0x563cef2cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.434+0000 7fd13e940740 4 client.19254 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:59:06.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.434+0000 7fd13e940740 1 -- 172.21.15.172:0/121435861 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cef21f860 con 0x563cef2cb170 2024-09-10T08:59:06.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.434+0000 7fd13a303640 1 --2- 172.21.15.172:0/121435861 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563cef2cb170 0x563cef2cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.435+0000 7fd11dffb640 10 client.19254 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:06.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.435+0000 7fd13a303640 1 --2- 172.21.15.172:0/121435861 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563cef2cb170 0x563cef2cd600 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d803851e3e30016 server_cookie=c2cb39144cf71414 in_seq=0 out_seq=0 2024-09-10T08:59:06.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.435+0000 7fd11dffb640 1 -- 172.21.15.172:0/121435861 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563cef21f860 con 0x563cef2cb170 2024-09-10T08:59:06.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.435+0000 7fd11dffb640 10 client.19254 handle_command_reply: tid=0 2024-09-10T08:59:06.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.435+0000 7fd13e940740 1 client.19254 shutdown 2024-09-10T08:59:06.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.435+0000 7fd13e940740 1 -- 172.21.15.172:0/121435861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2adb40 msgr2=0x563cef2b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:06.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.435+0000 7fd13e940740 1 --2- 172.21.15.172:0/121435861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2adb40 0x563cef2b3fc0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fd12400de10 tx=0x7fd12400f730 comp rx=0 tx=0).stop 2024-09-10T08:59:06.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.435+0000 7fd13e940740 1 -- 172.21.15.172:0/121435861 shutdown_connections 2024-09-10T08:59:06.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.435+0000 7fd13e940740 1 --2- 172.21.15.172:0/121435861 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563cef2cb170 0x563cef2cd600 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.435+0000 7fd11dffb640 0 client.19254 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:06.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.435+0000 7fd13e940740 1 --2- 172.21.15.172:0/121435861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2b7370 0x563cef2b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.436+0000 7fd13e940740 1 --2- 172.21.15.172:0/121435861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2adb40 0x563cef2b3fc0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.436+0000 7fd13e940740 1 --2- 172.21.15.172:0/121435861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2ac5d0 0x563cef2b6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.436+0000 7fd13e940740 1 -- 172.21.15.172:0/121435861 >> 172.21.15.172:0/121435861 conn(0x563cef275310 msgr2=0x563cef2ae960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:06.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.436+0000 7fd13e940740 1 -- 172.21.15.172:0/121435861 shutdown_connections 2024-09-10T08:59:06.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.436+0000 7fd13e940740 1 -- 172.21.15.172:0/121435861 wait complete. 2024-09-10T08:59:06.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.436+0000 7fd13e940740 20 client.19254 trim_cache size 0 max 16384 2024-09-10T08:59:06.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.448+0000 7fd13e940740 1 Processor -- start 2024-09-10T08:59:06.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.448+0000 7fd13e940740 1 -- start start 2024-09-10T08:59:06.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd13e940740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2ac5d0 0x563cef2b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd13e940740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2adb40 0x563cef2d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd13e940740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2b7370 0x563cef2d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd13e940740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1341ac880 con 0x563cef2ac5d0 2024-09-10T08:59:06.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd13e940740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1341ae450 con 0x563cef2adb40 2024-09-10T08:59:06.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd13e940740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd1341acf50 con 0x563cef2b7370 2024-09-10T08:59:06.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd13a303640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2ac5d0 0x563cef2b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd13a303640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2ac5d0 0x563cef2b2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40752/0 (socket says 172.21.15.172:40752) 2024-09-10T08:59:06.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd13a303640 1 -- 172.21.15.172:0/3994460906 learned_addr learned my addr 172.21.15.172:0/3994460906 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:06.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd139b02640 1 --2- 172.21.15.172:0/3994460906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2adb40 0x563cef2d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd13ab04640 1 --2- 172.21.15.172:0/3994460906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2b7370 0x563cef2d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd13a303640 1 -- 172.21.15.172:0/3994460906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2b7370 msgr2=0x563cef2d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:06.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd13a303640 1 --2- 172.21.15.172:0/3994460906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2b7370 0x563cef2d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd13a303640 1 -- 172.21.15.172:0/3994460906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2adb40 msgr2=0x563cef2d2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:06.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd13a303640 1 --2- 172.21.15.172:0/3994460906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2adb40 0x563cef2d2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd13a303640 1 -- 172.21.15.172:0/3994460906 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cef2a9df0 con 0x563cef2ac5d0 2024-09-10T08:59:06.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd139b02640 1 --2- 172.21.15.172:0/3994460906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2adb40 0x563cef2d2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:59:06.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd13ab04640 1 --2- 172.21.15.172:0/3994460906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2b7370 0x563cef2d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:59:06.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd13a303640 1 --2- 172.21.15.172:0/3994460906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2ac5d0 0x563cef2b2f00 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd1300cce80 tx=0x7fd13009ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7531ff607623f3da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:06.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd11ffff640 1 -- 172.21.15.172:0/3994460906 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1300b2040 con 0x563cef2ac5d0 2024-09-10T08:59:06.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd11ffff640 1 -- 172.21.15.172:0/3994460906 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1300b4040 con 0x563cef2ac5d0 2024-09-10T08:59:06.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.449+0000 7fd11ffff640 1 -- 172.21.15.172:0/3994460906 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd130086040 con 0x563cef2ac5d0 2024-09-10T08:59:06.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.450+0000 7fd13e940740 1 -- 172.21.15.172:0/3994460906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2ac5d0 msgr2=0x563cef2b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:06.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.450+0000 7fd13e940740 1 --2- 172.21.15.172:0/3994460906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2ac5d0 0x563cef2b2f00 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd1300cce80 tx=0x7fd13009ba20 comp rx=0 tx=0).stop 2024-09-10T08:59:06.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.450+0000 7fd13e940740 1 -- 172.21.15.172:0/3994460906 shutdown_connections 2024-09-10T08:59:06.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.450+0000 7fd13e940740 1 --2- 172.21.15.172:0/3994460906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2b7370 0x563cef2d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.450+0000 7fd13e940740 1 --2- 172.21.15.172:0/3994460906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2adb40 0x563cef2d2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.450+0000 7fd13e940740 1 --2- 172.21.15.172:0/3994460906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2ac5d0 0x563cef2b2f00 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.450+0000 7fd13e940740 1 -- 172.21.15.172:0/3994460906 >> 172.21.15.172:0/3994460906 conn(0x563cef275310 msgr2=0x563cef2a7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:06.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.450+0000 7fd13e940740 1 -- 172.21.15.172:0/3994460906 shutdown_connections 2024-09-10T08:59:06.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.450+0000 7fd13e940740 1 -- 172.21.15.172:0/3994460906 wait complete. 2024-09-10T08:59:06.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.450+0000 7fd13e940740 1 -- start start 2024-09-10T08:59:06.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.450+0000 7fd13e940740 1 Processor -- start 2024-09-10T08:59:06.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.450+0000 7fd13e940740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2ac5d0 0x563cef2b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd13e940740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2adb40 0x563cef2b1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd13e940740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2b7370 0x563cef2b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd13e940740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1341add80 con 0x563cef2adb40 2024-09-10T08:59:06.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd13e940740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1341ad6b0 con 0x563cef2b7370 2024-09-10T08:59:06.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd13e940740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd1341aa860 con 0x563cef2ac5d0 2024-09-10T08:59:06.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd13e940740 10 client.0 _ll_register_callbacks cb 0x7fd12400d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:59:06.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd139b02640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2adb40 0x563cef2b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd139b02640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2adb40 0x563cef2b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40762/0 (socket says 172.21.15.172:40762) 2024-09-10T08:59:06.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd139b02640 1 -- 172.21.15.172:0/1303046634 learned_addr learned my addr 172.21.15.172:0/1303046634 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:06.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd13a303640 1 --2- 172.21.15.172:0/1303046634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2ac5d0 0x563cef2b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd13ab04640 1 --2- 172.21.15.172:0/1303046634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2b7370 0x563cef2b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd139b02640 1 -- 172.21.15.172:0/1303046634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2ac5d0 msgr2=0x563cef2b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:06.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd139b02640 1 --2- 172.21.15.172:0/1303046634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2ac5d0 0x563cef2b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd139b02640 1 -- 172.21.15.172:0/1303046634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2b7370 msgr2=0x563cef2b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:06.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd139b02640 1 --2- 172.21.15.172:0/1303046634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2b7370 0x563cef2b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd139b02640 1 -- 172.21.15.172:0/1303046634 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cef3350e0 con 0x563cef2adb40 2024-09-10T08:59:06.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd13a303640 1 --2- 172.21.15.172:0/1303046634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2ac5d0 0x563cef2b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:59:06.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd139b02640 1 --2- 172.21.15.172:0/1303046634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2adb40 0x563cef2b1710 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fd12400d730 tx=0x7fd1240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8cabf712abd78e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:06.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd11dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:59:06.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd11dffb640 1 -- 172.21.15.172:0/1303046634 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124005770 con 0x563cef2adb40 2024-09-10T08:59:06.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd11dffb640 1 -- 172.21.15.172:0/1303046634 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd124004ca0 con 0x563cef2adb40 2024-09-10T08:59:06.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd13e940740 1 -- 172.21.15.172:0/1303046634 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1300b4040 con 0x563cef2adb40 2024-09-10T08:59:06.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd11dffb640 1 -- 172.21.15.172:0/1303046634 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd12403b050 con 0x563cef2adb40 2024-09-10T08:59:06.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd13e940740 10 client.19260 fetch_fsmap learned FSMap version 40 2024-09-10T08:59:06.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.451+0000 7fd13e940740 1 -- 172.21.15.172:0/1303046634 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563cef3363c0 con 0x563cef2adb40 2024-09-10T08:59:06.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.452+0000 7fd11dffb640 1 -- 172.21.15.172:0/1303046634 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1300b4040 con 0x563cef2adb40 2024-09-10T08:59:06.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.452+0000 7fd11dffb640 1 -- 172.21.15.172:0/1303046634 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8382+0+0 (secure 0 0 0) 0x7fd124019430 con 0x563cef2adb40 2024-09-10T08:59:06.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.452+0000 7fd13e940740 10 client.19260 fetch_fsmap finished waiting for FSMap version 40 2024-09-10T08:59:06.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.452+0000 7fd13e940740 10 client.19260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:59:06.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.452+0000 7fd13e940740 20 client.19260 populate_metadata read hostname 'smithi172' 2024-09-10T08:59:06.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.452+0000 7fd13e940740 1 --2- 172.21.15.172:0/1303046634 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563cef336ad0 0x563cef338ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:06.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.452+0000 7fd13e940740 4 client.19260 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T08:59:06.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.452+0000 7fd13e940740 1 -- 172.21.15.172:0/1303046634 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563cef219d70 con 0x563cef336ad0 2024-09-10T08:59:06.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.452+0000 7fd13a303640 1 --2- 172.21.15.172:0/1303046634 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563cef336ad0 0x563cef338ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:06.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.452+0000 7fd11dffb640 10 client.19260 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:06.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.452+0000 7fd13a303640 1 --2- 172.21.15.172:0/1303046634 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563cef336ad0 0x563cef338ec0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc8509d7b2818111 server_cookie=6caf22988955c14f in_seq=0 out_seq=0 2024-09-10T08:59:06.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd11dffb640 1 -- 172.21.15.172:0/1303046634 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563cef219d70 con 0x563cef336ad0 2024-09-10T08:59:06.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd11dffb640 10 client.19260 handle_command_reply: tid=0 2024-09-10T08:59:06.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13e940740 1 client.19260 shutdown 2024-09-10T08:59:06.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13e940740 1 -- 172.21.15.172:0/1303046634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2adb40 msgr2=0x563cef2b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:06.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13e940740 1 --2- 172.21.15.172:0/1303046634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2adb40 0x563cef2b1710 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fd12400d730 tx=0x7fd1240046c0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13e940740 1 -- 172.21.15.172:0/1303046634 shutdown_connections 2024-09-10T08:59:06.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13e940740 1 --2- 172.21.15.172:0/1303046634 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563cef336ad0 0x563cef338ec0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13e940740 1 --2- 172.21.15.172:0/1303046634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563cef2b7370 0x563cef2b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13e940740 1 --2- 172.21.15.172:0/1303046634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cef2adb40 0x563cef2b1710 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd11dffb640 0 client.19260 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:06.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13e940740 1 --2- 172.21.15.172:0/1303046634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563cef2ac5d0 0x563cef2b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13e940740 1 -- 172.21.15.172:0/1303046634 >> 172.21.15.172:0/1303046634 conn(0x563cef275310 msgr2=0x563cef2a74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:06.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13e940740 1 -- 172.21.15.172:0/1303046634 shutdown_connections 2024-09-10T08:59:06.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13e940740 1 -- 172.21.15.172:0/1303046634 wait complete. 2024-09-10T08:59:06.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13e940740 20 client.19260 trim_cache size 0 max 16384 2024-09-10T08:59:06.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13c567640 1 -- 172.21.15.172:0/845507219 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1200a2c00 msgr2=0x7fd1200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:06.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13c567640 1 --2- 172.21.15.172:0/845507219 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1200a2c00 0x7fd1200a50c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd12c032040 tx=0x7fd12c005da0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13c567640 1 -- 172.21.15.172:0/845507219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd134100a80 msgr2=0x7fd134196140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:06.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13c567640 1 --2- 172.21.15.172:0/845507219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd134100a80 0x7fd134196140 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fd13000e9f0 tx=0x7fd13000edf0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13c567640 1 -- 172.21.15.172:0/845507219 shutdown_connections 2024-09-10T08:59:06.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13c567640 1 --2- 172.21.15.172:0/845507219 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1200a2c00 0x7fd1200a50c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13c567640 1 --2- 172.21.15.172:0/845507219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd134106a80 0x7fd13419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13c567640 1 --2- 172.21.15.172:0/845507219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd134101420 0x7fd134196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.268 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13c567640 1 --2- 172.21.15.172:0/845507219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd134100a80 0x7fd134196140 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:06.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.453+0000 7fd13c567640 1 -- 172.21.15.172:0/845507219 >> 172.21.15.172:0/845507219 conn(0x7fd1340fc7a0 msgr2=0x7fd1340fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:06.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.454+0000 7fd13c567640 1 -- 172.21.15.172:0/845507219 shutdown_connections 2024-09-10T08:59:06.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:04.454+0000 7fd13c567640 1 -- 172.21.15.172:0/845507219 wait complete. 2024-09-10T08:59:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:07 smithi174 ceph-mon[34068]: pgmap v737: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 2.71k op/s 2024-09-10T08:59:07.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:07 smithi191 ceph-mon[35314]: pgmap v737: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 2.71k op/s 2024-09-10T08:59:07.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:07 smithi172 ceph-mon[30854]: pgmap v737: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 2.71k op/s 2024-09-10T08:59:09.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:09 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:59:09.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:09 smithi174 ceph-mon[34068]: pgmap v738: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 3.29k op/s 2024-09-10T08:59:09.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:09 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:59:09.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:09 smithi191 ceph-mon[35314]: pgmap v738: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 3.29k op/s 2024-09-10T08:59:09.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:09 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T08:59:09.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:09 smithi172 ceph-mon[30854]: pgmap v738: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 3.29k op/s 2024-09-10T08:59:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:11 smithi174 ceph-mon[34068]: pgmap v739: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 3.44k op/s 2024-09-10T08:59:11.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:11 smithi191 ceph-mon[35314]: pgmap v739: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 3.44k op/s 2024-09-10T08:59:11.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:11 smithi172 ceph-mon[30854]: pgmap v739: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 3.44k op/s 2024-09-10T08:59:13.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:13 smithi174 ceph-mon[34068]: pgmap v740: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-10T08:59:13.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:13 smithi191 ceph-mon[35314]: pgmap v740: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-10T08:59:13.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:13 smithi172 ceph-mon[30854]: pgmap v740: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-10T08:59:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.146+0000 7f308f18d640 1 -- 172.21.15.172:0/3772306765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f308810dbb0 msgr2=0x7f3088110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.146+0000 7f308f18d640 1 --2- 172.21.15.172:0/3772306765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f308810dbb0 0x7f3088110070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3084009930 tx=0x7f308402a460 comp rx=0 tx=0).stop 2024-09-10T08:59:15.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.159+0000 7f308f18d640 1 -- 172.21.15.172:0/3772306765 shutdown_connections 2024-09-10T08:59:15.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.159+0000 7f308f18d640 1 --2- 172.21.15.172:0/3772306765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f308810dbb0 0x7f3088110070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.159+0000 7f308f18d640 1 --2- 172.21.15.172:0/3772306765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3088073680 0x7f308810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.159+0000 7f308f18d640 1 --2- 172.21.15.172:0/3772306765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3088072cb0 0x7f30880730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.159+0000 7f308f18d640 1 -- 172.21.15.172:0/3772306765 >> 172.21.15.172:0/3772306765 conn(0x7f308806eb50 msgr2=0x7f3088070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:15.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.164+0000 7f308f18d640 1 -- 172.21.15.172:0/3772306765 shutdown_connections 2024-09-10T08:59:15.171 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.168+0000 7f308f18d640 1 -- 172.21.15.172:0/3772306765 wait complete. 2024-09-10T08:59:15.171 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.169+0000 7f308f18d640 1 Processor -- start 2024-09-10T08:59:15.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.169+0000 7f308f18d640 1 -- start start 2024-09-10T08:59:15.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.169+0000 7f308f18d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3088072cb0 0x7f30881a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.170+0000 7f308f18d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3088073680 0x7f30881a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.170+0000 7f308f18d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f308810dbb0 0x7f30881adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.170+0000 7f308f18d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f308811c3d0 con 0x7f308810dbb0 2024-09-10T08:59:15.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.170+0000 7f308f18d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f308811c250 con 0x7f3088073680 2024-09-10T08:59:15.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.170+0000 7f308f18d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f308811c550 con 0x7f3088072cb0 2024-09-10T08:59:15.176 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.171+0000 7f308e18b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3088072cb0 0x7f30881a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:15.176 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.171+0000 7f308e18b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3088072cb0 0x7f30881a6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:38896/0 (socket says 172.21.15.172:38896) 2024-09-10T08:59:15.177 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.171+0000 7f308e18b640 1 -- 172.21.15.172:0/1114682806 learned_addr learned my addr 172.21.15.172:0/1114682806 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:15.177 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.173+0000 7f308e18b640 1 -- 172.21.15.172:0/1114682806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3088073680 msgr2=0x7f30881a6f60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T08:59:15.177 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.173+0000 7f308e18b640 1 --2- 172.21.15.172:0/1114682806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3088073680 0x7f30881a6f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.173+0000 7f308e18b640 1 -- 172.21.15.172:0/1114682806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f308810dbb0 msgr2=0x7f30881adfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.173+0000 7f308e18b640 1 --2- 172.21.15.172:0/1114682806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f308810dbb0 0x7f30881adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.173+0000 7f308e18b640 1 -- 172.21.15.172:0/1114682806 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3088110a50 con 0x7f3088072cb0 2024-09-10T08:59:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.174+0000 7f308e18b640 1 --2- 172.21.15.172:0/1114682806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3088072cb0 0x7f30881a6a20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f307800b4a0 tx=0x7f307800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.174+0000 7f307f7fe640 1 -- 172.21.15.172:0/1114682806 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078014070 con 0x7f3088072cb0 2024-09-10T08:59:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.174+0000 7f307f7fe640 1 -- 172.21.15.172:0/1114682806 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3078010ac0 con 0x7f3088072cb0 2024-09-10T08:59:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.174+0000 7f307f7fe640 1 -- 172.21.15.172:0/1114682806 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078017c20 con 0x7f3088072cb0 2024-09-10T08:59:15.178 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.176+0000 7f308f18d640 1 -- 172.21.15.172:0/1114682806 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3088110d40 con 0x7f3088072cb0 2024-09-10T08:59:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.176+0000 7f308f18d640 1 -- 172.21.15.172:0/1114682806 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3088111290 con 0x7f3088072cb0 2024-09-10T08:59:15.181 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.180+0000 7f3093265740 1 Processor -- start 2024-09-10T08:59:15.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.180+0000 7f3093265740 1 -- start start 2024-09-10T08:59:15.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.180+0000 7f3093265740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e750b70 0x56319e750f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.180+0000 7f3093265740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e7514b0 0x56319e755f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.180+0000 7f3093265740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e756540 0x56319e758a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.180+0000 7f3093265740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56319e6b42f0 con 0x56319e756540 2024-09-10T08:59:15.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.180+0000 7f3093265740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56319e65c270 con 0x56319e7514b0 2024-09-10T08:59:15.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.180+0000 7f3093265740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56319e6b02b0 con 0x56319e750b70 2024-09-10T08:59:15.183 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.181+0000 7f308e98c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e750b70 0x56319e750f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.181+0000 7f308e98c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e750b70 0x56319e750f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:38904/0 (socket says 172.21.15.172:38904) 2024-09-10T08:59:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.181+0000 7f308e98c640 1 -- 172.21.15.172:0/2470538681 learned_addr learned my addr 172.21.15.172:0/2470538681 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.181+0000 7f308d98a640 1 --2- 172.21.15.172:0/2470538681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e756540 0x56319e758a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.182+0000 7f308e98c640 1 -- 172.21.15.172:0/2470538681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e7514b0 msgr2=0x56319e755f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:59:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.182+0000 7f308e98c640 1 --2- 172.21.15.172:0/2470538681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e7514b0 0x56319e755f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.182+0000 7f308e98c640 1 -- 172.21.15.172:0/2470538681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e756540 msgr2=0x56319e758a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.182+0000 7f308e98c640 1 --2- 172.21.15.172:0/2470538681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e756540 0x56319e758a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.182+0000 7f308e98c640 1 -- 172.21.15.172:0/2470538681 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56319e65b340 con 0x56319e750b70 2024-09-10T08:59:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.182+0000 7f308e98c640 1 --2- 172.21.15.172:0/2470538681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e750b70 0x56319e750f70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3084031850 tx=0x7f3084031880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bd0db47ca975420 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:15.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.183+0000 7f307cbfe640 1 -- 172.21.15.172:0/2470538681 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308403f070 con 0x56319e750b70 2024-09-10T08:59:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.183+0000 7f307cbfe640 1 -- 172.21.15.172:0/2470538681 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30840054f0 con 0x56319e750b70 2024-09-10T08:59:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.184+0000 7f3093265740 1 -- 172.21.15.172:0/2470538681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e750b70 msgr2=0x56319e750f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.184+0000 7f3093265740 1 --2- 172.21.15.172:0/2470538681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e750b70 0x56319e750f70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3084031850 tx=0x7f3084031880 comp rx=0 tx=0).stop 2024-09-10T08:59:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.184+0000 7f307f7fe640 1 -- 172.21.15.172:0/1114682806 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3078017070 con 0x7f3088072cb0 2024-09-10T08:59:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.185+0000 7f3093265740 1 -- 172.21.15.172:0/2470538681 shutdown_connections 2024-09-10T08:59:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.185+0000 7f3093265740 1 --2- 172.21.15.172:0/2470538681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e756540 0x56319e758a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.185+0000 7f3093265740 1 --2- 172.21.15.172:0/2470538681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e7514b0 0x56319e755f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.187 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.185+0000 7f3093265740 1 --2- 172.21.15.172:0/2470538681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e750b70 0x56319e750f70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.185+0000 7f3093265740 1 -- 172.21.15.172:0/2470538681 >> 172.21.15.172:0/2470538681 conn(0x56319e74c1e0 msgr2=0x56319e74c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:15.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.185+0000 7f3093265740 1 -- 172.21.15.172:0/2470538681 shutdown_connections 2024-09-10T08:59:15.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.185+0000 7f3093265740 1 -- 172.21.15.172:0/2470538681 wait complete. 2024-09-10T08:59:15.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.185+0000 7f3093265740 1 -- start start 2024-09-10T08:59:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.186+0000 7f3093265740 1 Processor -- start 2024-09-10T08:59:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.186+0000 7f307f7fe640 1 --2- 172.21.15.172:0/1114682806 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30740a2bd0 0x7f30740a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.186+0000 7f307f7fe640 1 -- 172.21.15.172:0/1114682806 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f30780ce710 con 0x7f3088072cb0 2024-09-10T08:59:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.187+0000 7f3093265740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e750b70 0x56319e75af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.187+0000 7f3093265740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e756540 0x56319e75b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.187+0000 7f308e98c640 1 --2- 172.21.15.172:0/1114682806 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30740a2bd0 0x7f30740a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.187+0000 7f308e18b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e750b70 0x56319e75af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.187+0000 7f308e18b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e750b70 0x56319e75af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:53308/0 (socket says 172.21.15.172:53308) 2024-09-10T08:59:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.188+0000 7f3093265740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e760480 0x56319e762870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.188+0000 7f3093265740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56319e759550 con 0x56319e760480 2024-09-10T08:59:15.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.188+0000 7f3093265740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56319e63d290 con 0x56319e750b70 2024-09-10T08:59:15.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.188+0000 7f3093265740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56319e7596d0 con 0x56319e756540 2024-09-10T08:59:15.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.188+0000 7f308d98a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e756540 0x56319e75b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:15.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.188+0000 7f308d98a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e756540 0x56319e75b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:38912/0 (socket says 172.21.15.172:38912) 2024-09-10T08:59:15.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.188+0000 7f308d98a640 1 -- 172.21.15.172:0/3213159937 learned_addr learned my addr 172.21.15.172:0/3213159937 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:15.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.188+0000 7f308e98c640 1 --2- 172.21.15.172:0/3213159937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e760480 0x56319e762870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:15.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.188+0000 7f3093265740 10 client.0 _ll_register_callbacks cb 0x56319e66e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:59:15.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.188+0000 7f308e98c640 1 --2- 172.21.15.172:0/1114682806 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30740a2bd0 0x7f30740a5090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f30840316e0 tx=0x7f3084005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:15.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.188+0000 7f308d98a640 1 -- 172.21.15.172:0/3213159937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e750b70 msgr2=0x56319e75af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:15.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.188+0000 7f308d98a640 1 --2- 172.21.15.172:0/3213159937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e750b70 0x56319e75af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.188+0000 7f308d98a640 1 -- 172.21.15.172:0/3213159937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e760480 msgr2=0x56319e762870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.188+0000 7f308d98a640 1 --2- 172.21.15.172:0/3213159937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e760480 0x56319e762870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.188+0000 7f308d98a640 1 -- 172.21.15.172:0/3213159937 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56319e3c7210 con 0x56319e756540 2024-09-10T08:59:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.189+0000 7f308e18b640 1 --2- 172.21.15.172:0/3213159937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e750b70 0x56319e75af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-10T08:59:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.189+0000 7f3093265740 1 -- 172.21.15.172:0/3213159937 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30840054f0 con 0x56319e756540 2024-09-10T08:59:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.189+0000 7f308d98a640 1 --2- 172.21.15.172:0/3213159937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e756540 0x56319e75b450 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f308000c5e0 tx=0x7f308000caa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b6b9275590cd105 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.189+0000 7f3071ffb640 10 client.27988 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:59:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.189+0000 7f3071ffb640 1 -- 172.21.15.172:0/3213159937 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308001f050 con 0x56319e756540 2024-09-10T08:59:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.189+0000 7f3071ffb640 1 -- 172.21.15.172:0/3213159937 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f308001a640 con 0x56319e756540 2024-09-10T08:59:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.189+0000 7f3071ffb640 1 -- 172.21.15.172:0/3213159937 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3080015070 con 0x56319e756540 2024-09-10T08:59:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.189+0000 7f3093265740 10 client.27988 fetch_fsmap learned FSMap version 41 2024-09-10T08:59:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.189+0000 7f3093265740 1 -- 172.21.15.172:0/3213159937 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({fsmap=41}) -- 0x56319e493620 con 0x56319e756540 2024-09-10T08:59:15.194 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.190+0000 7f3071ffb640 1 -- 172.21.15.172:0/3213159937 <== mon.1 v2:172.21.15.191:3300/0 4 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f3080010050 con 0x56319e756540 2024-09-10T08:59:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.190+0000 7f3093265740 10 client.27988 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T08:59:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.190+0000 7f3093265740 10 client.27988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:59:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.190+0000 7f3093265740 20 client.27988 populate_metadata read hostname 'smithi172' 2024-09-10T08:59:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.190+0000 7f3093265740 1 --2- 172.21.15.172:0/3213159937 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56319e766e00 0x56319e7691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.190+0000 7f3093265740 4 client.27988 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:59:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.190+0000 7f3093265740 1 -- 172.21.15.172:0/3213159937 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56319e6c1a20 con 0x56319e766e00 2024-09-10T08:59:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.190+0000 7f308e18b640 1 --2- 172.21.15.172:0/3213159937 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56319e766e00 0x56319e7691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:15.196 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.191+0000 7f308e18b640 1 --2- 172.21.15.172:0/3213159937 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56319e766e00 0x56319e7691f0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=620548dc09d15062 server_cookie=faa0c8773912ec43 in_seq=0 out_seq=0 2024-09-10T08:59:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.191+0000 7f3071ffb640 10 client.27988 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.193+0000 7f3071ffb640 1 -- 172.21.15.172:0/3213159937 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56319e6c1a20 con 0x56319e766e00 2024-09-10T08:59:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.193+0000 7f3071ffb640 10 client.27988 handle_command_reply: tid=0 2024-09-10T08:59:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.193+0000 7f3093265740 1 client.27988 shutdown 2024-09-10T08:59:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.193+0000 7f3093265740 1 -- 172.21.15.172:0/3213159937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e756540 msgr2=0x56319e75b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.193+0000 7f3093265740 1 --2- 172.21.15.172:0/3213159937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e756540 0x56319e75b450 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f308000c5e0 tx=0x7f308000caa0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.194+0000 7f3093265740 1 -- 172.21.15.172:0/3213159937 shutdown_connections 2024-09-10T08:59:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.194+0000 7f3093265740 1 --2- 172.21.15.172:0/3213159937 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56319e766e00 0x56319e7691f0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.194+0000 7f3093265740 1 --2- 172.21.15.172:0/3213159937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e760480 0x56319e762870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.194+0000 7f3093265740 1 --2- 172.21.15.172:0/3213159937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e756540 0x56319e75b450 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.194+0000 7f3093265740 1 --2- 172.21.15.172:0/3213159937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e750b70 0x56319e75af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.197 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.194+0000 7f3071ffb640 0 client.27988 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.194+0000 7f3093265740 1 -- 172.21.15.172:0/3213159937 >> 172.21.15.172:0/3213159937 conn(0x56319e74c1e0 msgr2=0x56319e74d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.194+0000 7f3093265740 1 -- 172.21.15.172:0/3213159937 shutdown_connections 2024-09-10T08:59:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.195+0000 7f3093265740 1 -- 172.21.15.172:0/3213159937 wait complete. 2024-09-10T08:59:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.195+0000 7f3093265740 20 client.27988 trim_cache size 0 max 16384 2024-09-10T08:59:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.232+0000 7f3093265740 1 Processor -- start 2024-09-10T08:59:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.232+0000 7f3093265740 1 -- start start 2024-09-10T08:59:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.232+0000 7f3093265740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e74c1e0 0x56319e762870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.235+0000 7f3093265740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e750b70 0x56319e74b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.235+0000 7f3093265740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e756540 0x56319e74b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.235+0000 7f308e18b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e74c1e0 0x56319e762870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.235+0000 7f308e18b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e74c1e0 0x56319e762870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:38922/0 (socket says 172.21.15.172:38922) 2024-09-10T08:59:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.235+0000 7f308e18b640 1 -- 172.21.15.172:0/3007868646 learned_addr learned my addr 172.21.15.172:0/3007868646 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.235+0000 7f3093265740 1 -- 172.21.15.172:0/3007868646 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30881c1ac0 con 0x56319e750b70 2024-09-10T08:59:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.235+0000 7f3093265740 1 -- 172.21.15.172:0/3007868646 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30881c36c0 con 0x56319e756540 2024-09-10T08:59:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.235+0000 7f3093265740 1 -- 172.21.15.172:0/3007868646 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f30881c2190 con 0x56319e74c1e0 2024-09-10T08:59:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.236+0000 7f308d98a640 1 --2- 172.21.15.172:0/3007868646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e750b70 0x56319e74b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.236+0000 7f308e18b640 1 -- 172.21.15.172:0/3007868646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e756540 msgr2=0x56319e74b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.236+0000 7f308e18b640 1 --2- 172.21.15.172:0/3007868646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e756540 0x56319e74b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.236+0000 7f308e18b640 1 -- 172.21.15.172:0/3007868646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e750b70 msgr2=0x56319e74b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.236+0000 7f308e18b640 1 --2- 172.21.15.172:0/3007868646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e750b70 0x56319e74b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.236+0000 7f308e18b640 1 -- 172.21.15.172:0/3007868646 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56319e769560 con 0x56319e74c1e0 2024-09-10T08:59:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.237+0000 7f308e18b640 1 --2- 172.21.15.172:0/3007868646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e74c1e0 0x56319e762870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3078097010 tx=0x7f30780047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74dbce8ac23c954b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.238+0000 7f3073fff640 1 -- 172.21.15.172:0/3007868646 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30780b1070 con 0x56319e74c1e0 2024-09-10T08:59:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.238+0000 7f3073fff640 1 -- 172.21.15.172:0/3007868646 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3078017dc0 con 0x56319e74c1e0 2024-09-10T08:59:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.239+0000 7f3073fff640 1 -- 172.21.15.172:0/3007868646 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078097220 con 0x56319e74c1e0 2024-09-10T08:59:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.239+0000 7f3093265740 1 -- 172.21.15.172:0/3007868646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e74c1e0 msgr2=0x56319e762870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.239+0000 7f3093265740 1 --2- 172.21.15.172:0/3007868646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e74c1e0 0x56319e762870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3078097010 tx=0x7f30780047d0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.239+0000 7f3093265740 1 -- 172.21.15.172:0/3007868646 shutdown_connections 2024-09-10T08:59:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.239+0000 7f3093265740 1 --2- 172.21.15.172:0/3007868646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e756540 0x56319e74b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.239+0000 7f3093265740 1 --2- 172.21.15.172:0/3007868646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e750b70 0x56319e74b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.239+0000 7f3093265740 1 --2- 172.21.15.172:0/3007868646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e74c1e0 0x56319e762870 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.239+0000 7f3093265740 1 -- 172.21.15.172:0/3007868646 >> 172.21.15.172:0/3007868646 conn(0x56319e7560d0 msgr2=0x56319e74aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.239+0000 7f3093265740 1 -- 172.21.15.172:0/3007868646 shutdown_connections 2024-09-10T08:59:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.239+0000 7f3093265740 1 -- 172.21.15.172:0/3007868646 wait complete. 2024-09-10T08:59:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.240+0000 7f3093265740 1 -- start start 2024-09-10T08:59:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.240+0000 7f3093265740 1 Processor -- start 2024-09-10T08:59:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.240+0000 7f3093265740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e74c1e0 0x56319e7789d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.240+0000 7f3093265740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e750b70 0x56319e778f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.240+0000 7f3093265740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e756540 0x56319e779eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.240+0000 7f3093265740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30881c2ff0 con 0x56319e750b70 2024-09-10T08:59:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.240+0000 7f3093265740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30881c2920 con 0x56319e756540 2024-09-10T08:59:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.240+0000 7f3093265740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3088118da0 con 0x56319e74c1e0 2024-09-10T08:59:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.241+0000 7f308e18b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e74c1e0 0x56319e7789d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.241+0000 7f3093265740 10 client.0 _ll_register_callbacks cb 0x7f3080007410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:59:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.241+0000 7f308d98a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e750b70 0x56319e778f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.241+0000 7f308e18b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e74c1e0 0x56319e7789d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:38938/0 (socket says 172.21.15.172:38938) 2024-09-10T08:59:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.241+0000 7f308e18b640 1 -- 172.21.15.172:0/1820070582 learned_addr learned my addr 172.21.15.172:0/1820070582 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.241+0000 7f308e98c640 1 --2- 172.21.15.172:0/1820070582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e756540 0x56319e779eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.241+0000 7f308d98a640 1 -- 172.21.15.172:0/1820070582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e74c1e0 msgr2=0x56319e7789d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.241+0000 7f308d98a640 1 --2- 172.21.15.172:0/1820070582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e74c1e0 0x56319e7789d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.241+0000 7f308d98a640 1 -- 172.21.15.172:0/1820070582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e756540 msgr2=0x56319e779eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.241+0000 7f308d98a640 1 --2- 172.21.15.172:0/1820070582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e756540 0x56319e779eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.241+0000 7f308d98a640 1 -- 172.21.15.172:0/1820070582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56319e77a4c0 con 0x56319e750b70 2024-09-10T08:59:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.241+0000 7f308d98a640 1 --2- 172.21.15.172:0/1820070582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e750b70 0x56319e778f10 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f3080007cd0 tx=0x7f308000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9bb4c13e84631a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.241+0000 7f3071ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:59:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.242+0000 7f3071ffb640 1 -- 172.21.15.172:0/1820070582 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308001f050 con 0x56319e750b70 2024-09-10T08:59:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.242+0000 7f3093265740 1 -- 172.21.15.172:0/1820070582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3078017dc0 con 0x56319e750b70 2024-09-10T08:59:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.242+0000 7f3071ffb640 1 -- 172.21.15.172:0/1820070582 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f308001a440 con 0x56319e750b70 2024-09-10T08:59:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.242+0000 7f3071ffb640 1 -- 172.21.15.172:0/1820070582 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3080038050 con 0x56319e750b70 2024-09-10T08:59:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.242+0000 7f3093265740 10 client.19269 fetch_fsmap learned FSMap version 41 2024-09-10T08:59:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.242+0000 7f3093265740 1 -- 172.21.15.172:0/1820070582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56319e7d1b00 con 0x56319e750b70 2024-09-10T08:59:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.242+0000 7f3071ffb640 1 -- 172.21.15.172:0/1820070582 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30840054f0 con 0x56319e750b70 2024-09-10T08:59:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.242+0000 7f3071ffb640 1 -- 172.21.15.172:0/1820070582 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f3080010040 con 0x56319e750b70 2024-09-10T08:59:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.242+0000 7f3093265740 10 client.19269 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T08:59:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.242+0000 7f3093265740 10 client.19269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:59:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.242+0000 7f3093265740 20 client.19269 populate_metadata read hostname 'smithi172' 2024-09-10T08:59:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.242+0000 7f3093265740 1 --2- 172.21.15.172:0/1820070582 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56319e760480 0x56319e7d4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.242+0000 7f3093265740 4 client.19269 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:59:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.243+0000 7f3093265740 1 -- 172.21.15.172:0/1820070582 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f3078017dc0 con 0x56319e760480 2024-09-10T08:59:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.243+0000 7f308e18b640 1 --2- 172.21.15.172:0/1820070582 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56319e760480 0x56319e7d4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.243+0000 7f3071ffb640 10 client.19269 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.243+0000 7f308e18b640 1 --2- 172.21.15.172:0/1820070582 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56319e760480 0x56319e7d4460 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=648534c2b8827713 server_cookie=e7b9289cbb519290 in_seq=0 out_seq=0 2024-09-10T08:59:15.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:15 smithi174 ceph-mon[34068]: pgmap v741: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.61k op/s 2024-09-10T08:59:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.765+0000 7f3071ffb640 1 -- 172.21.15.172:0/1820070582 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f3078017dc0 con 0x56319e760480 2024-09-10T08:59:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.765+0000 7f3071ffb640 10 client.19269 handle_command_reply: tid=0 2024-09-10T08:59:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.765+0000 7f3093265740 1 client.19269 shutdown 2024-09-10T08:59:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.770+0000 7f3093265740 1 -- 172.21.15.172:0/1820070582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e750b70 msgr2=0x56319e778f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.770+0000 7f3093265740 1 --2- 172.21.15.172:0/1820070582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e750b70 0x56319e778f10 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f3080007cd0 tx=0x7f308000f000 comp rx=0 tx=0).stop 2024-09-10T08:59:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.770+0000 7f3093265740 1 -- 172.21.15.172:0/1820070582 shutdown_connections 2024-09-10T08:59:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.770+0000 7f3093265740 1 --2- 172.21.15.172:0/1820070582 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56319e760480 0x56319e7d4460 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.770+0000 7f3093265740 1 --2- 172.21.15.172:0/1820070582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56319e756540 0x56319e779eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.770+0000 7f3093265740 1 --2- 172.21.15.172:0/1820070582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56319e750b70 0x56319e778f10 secure :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f3080007cd0 tx=0x7f308000f000 comp rx=0 tx=0).stop 2024-09-10T08:59:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.770+0000 7f3093265740 1 --2- 172.21.15.172:0/1820070582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56319e74c1e0 0x56319e7789d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.770+0000 7f3093265740 1 -- 172.21.15.172:0/1820070582 >> 172.21.15.172:0/1820070582 conn(0x56319e7560d0 msgr2=0x56319e74aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.770+0000 7f3071ffb640 0 client.19269 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.776+0000 7f3093265740 1 -- 172.21.15.172:0/1820070582 shutdown_connections 2024-09-10T08:59:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.781+0000 7f3093265740 1 -- 172.21.15.172:0/1820070582 wait complete. 2024-09-10T08:59:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.781+0000 7f3093265740 20 client.19269 trim_cache size 0 max 16384 2024-09-10T08:59:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:59:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:59:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:59:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:59:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.784+0000 7f308f18d640 1 -- 172.21.15.172:0/1114682806 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30740a2bd0 msgr2=0x7f30740a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.784+0000 7f308f18d640 1 --2- 172.21.15.172:0/1114682806 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30740a2bd0 0x7f30740a5090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f30840316e0 tx=0x7f3084005fd0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.784+0000 7f308f18d640 1 -- 172.21.15.172:0/1114682806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3088072cb0 msgr2=0x7f30881a6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.784+0000 7f308f18d640 1 --2- 172.21.15.172:0/1114682806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3088072cb0 0x7f30881a6a20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f307800b4a0 tx=0x7f307800b960 comp rx=0 tx=0).stop 2024-09-10T08:59:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.787+0000 7f308f18d640 1 -- 172.21.15.172:0/1114682806 shutdown_connections 2024-09-10T08:59:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.787+0000 7f308f18d640 1 --2- 172.21.15.172:0/1114682806 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f30740a2bd0 0x7f30740a5090 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.787+0000 7f308f18d640 1 --2- 172.21.15.172:0/1114682806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f308810dbb0 0x7f30881adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.787+0000 7f308f18d640 1 --2- 172.21.15.172:0/1114682806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3088073680 0x7f30881a6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.787+0000 7f308f18d640 1 --2- 172.21.15.172:0/1114682806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3088072cb0 0x7f30881a6a20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:15.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.787+0000 7f308f18d640 1 -- 172.21.15.172:0/1114682806 >> 172.21.15.172:0/1114682806 conn(0x7f308806eb50 msgr2=0x7f308806f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.794+0000 7f308f18d640 1 -- 172.21.15.172:0/1114682806 shutdown_connections 2024-09-10T08:59:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:15.795+0000 7f308f18d640 1 -- 172.21.15.172:0/1114682806 wait complete. 2024-09-10T08:59:15.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:15 smithi191 ceph-mon[35314]: pgmap v741: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.61k op/s 2024-09-10T08:59:15.835 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:15 smithi172 ceph-mon[30854]: pgmap v741: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.61k op/s 2024-09-10T08:59:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:17 smithi174 ceph-mon[34068]: pgmap v742: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 3.33k op/s 2024-09-10T08:59:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:17 smithi191 ceph-mon[35314]: pgmap v742: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 3.33k op/s 2024-09-10T08:59:17.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:17 smithi172 ceph-mon[30854]: pgmap v742: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 3.33k op/s 2024-09-10T08:59:18.242 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T08:59:18.243 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T08:59:19.058 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:59:19.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:19 smithi172 ceph-mon[30854]: pgmap v743: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 3.51k op/s 2024-09-10T08:59:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:19 smithi174 ceph-mon[34068]: pgmap v743: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 3.51k op/s 2024-09-10T08:59:19.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:19 smithi191 ceph-mon[35314]: pgmap v743: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 3.51k op/s 2024-09-10T08:59:21.251 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T08:59:21.251 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.5 2024-09-10T08:59:21.251 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T08:59:21.450 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:21 smithi172 ceph-mon[30854]: pgmap v744: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 3.34k op/s 2024-09-10T08:59:21.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:21 smithi174 ceph-mon[34068]: pgmap v744: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 3.34k op/s 2024-09-10T08:59:21.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:21 smithi191 ceph-mon[35314]: pgmap v744: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 3.34k op/s 2024-09-10T08:59:23.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:23 smithi174 ceph-mon[34068]: pgmap v745: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.40k op/s 2024-09-10T08:59:23.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:23 smithi191 ceph-mon[35314]: pgmap v745: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.40k op/s 2024-09-10T08:59:23.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:23 smithi172 ceph-mon[30854]: pgmap v745: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.40k op/s 2024-09-10T08:59:25.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:24 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:59:25.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:24 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:25.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:24 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:59:25.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:24 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:25.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:24 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:59:25.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:24 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:26.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.619+0000 7f98c45f9640 1 -- 172.21.15.172:0/137235082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98bc102570 msgr2=0x7f98bc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:26.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.619+0000 7f98c45f9640 1 --2- 172.21.15.172:0/137235082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98bc102570 0x7f98bc1029b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009890 tx=0x7f98ac02a410 comp rx=0 tx=0).stop 2024-09-10T08:59:26.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.619+0000 7f98c45f9640 1 -- 172.21.15.172:0/137235082 shutdown_connections 2024-09-10T08:59:26.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.619+0000 7f98c45f9640 1 --2- 172.21.15.172:0/137235082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98bc102ef0 0x7f98bc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.619+0000 7f98c45f9640 1 --2- 172.21.15.172:0/137235082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98bc102570 0x7f98bc1029b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.619+0000 7f98c45f9640 1 --2- 172.21.15.172:0/137235082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98bc100e20 0x7f98bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.619+0000 7f98c45f9640 1 -- 172.21.15.172:0/137235082 >> 172.21.15.172:0/137235082 conn(0x7f98bc0fc7d0 msgr2=0x7f98bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:26.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.619+0000 7f98c45f9640 1 -- 172.21.15.172:0/137235082 shutdown_connections 2024-09-10T08:59:26.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.619+0000 7f98c45f9640 1 -- 172.21.15.172:0/137235082 wait complete. 2024-09-10T08:59:26.028 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.619+0000 7f98c45f9640 1 Processor -- start 2024-09-10T08:59:26.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98c45f9640 1 -- start start 2024-09-10T08:59:26.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98c45f9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98bc100e20 0x7f98bc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98c45f9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98bc102570 0x7f98bc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98c45f9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98bc102ef0 0x7f98bc0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98c45f9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc110e30 con 0x7f98bc100e20 2024-09-10T08:59:26.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98c45f9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc110cb0 con 0x7f98bc102ef0 2024-09-10T08:59:26.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98c45f9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc110fb0 con 0x7f98bc102570 2024-09-10T08:59:26.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98c2395640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98bc100e20 0x7f98bc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:26.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98c1b94640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98bc102570 0x7f98bc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:26.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98c2395640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98bc100e20 0x7f98bc0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57808/0 (socket says 172.21.15.172:57808) 2024-09-10T08:59:26.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98c2395640 1 -- 172.21.15.172:0/1659140062 learned_addr learned my addr 172.21.15.172:0/1659140062 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:26.029 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98c1b94640 1 -- 172.21.15.172:0/1659140062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98bc102ef0 msgr2=0x7f98bc0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:26.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98c1b94640 1 --2- 172.21.15.172:0/1659140062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98bc102ef0 0x7f98bc0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98c1b94640 1 -- 172.21.15.172:0/1659140062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98bc100e20 msgr2=0x7f98bc0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:26.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98c1b94640 1 --2- 172.21.15.172:0/1659140062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98bc100e20 0x7f98bc0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98c1b94640 1 -- 172.21.15.172:0/1659140062 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc1a69c0 con 0x7f98bc102570 2024-09-10T08:59:26.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98c1b94640 1 --2- 172.21.15.172:0/1659140062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98bc102570 0x7f98bc0fd1c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009890 tx=0x7f98ac030ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:26.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.620+0000 7f98b37fe640 1 -- 172.21.15.172:0/1659140062 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac040070 con 0x7f98bc102570 2024-09-10T08:59:26.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.621+0000 7f98c45f9640 1 -- 172.21.15.172:0/1659140062 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc1a6c50 con 0x7f98bc102570 2024-09-10T08:59:26.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.621+0000 7f98b37fe640 1 -- 172.21.15.172:0/1659140062 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98ac004460 con 0x7f98bc102570 2024-09-10T08:59:26.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.621+0000 7f98b37fe640 1 -- 172.21.15.172:0/1659140062 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0046c0 con 0x7f98bc102570 2024-09-10T08:59:26.030 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.621+0000 7f98c45f9640 1 -- 172.21.15.172:0/1659140062 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc1a7130 con 0x7f98bc102570 2024-09-10T08:59:26.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.623+0000 7f98b37fe640 1 -- 172.21.15.172:0/1659140062 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f98ac043070 con 0x7f98bc102570 2024-09-10T08:59:26.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.623+0000 7f98c69d2740 1 Processor -- start 2024-09-10T08:59:26.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.623+0000 7f98c69d2740 1 -- start start 2024-09-10T08:59:26.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98b37fe640 1 --2- 172.21.15.172:0/1659140062 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98a80a2b80 0x7f98a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c69d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c2b60 0x5587962c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c69d2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587962c3480 0x5587962cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c69d2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962cdd40 0x5587962d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c69d2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558796228280 con 0x5587962cdd40 2024-09-10T08:59:26.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c69d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5587961cfb10 con 0x5587962c2b60 2024-09-10T08:59:26.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c69d2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558796224240 con 0x5587962c3480 2024-09-10T08:59:26.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c1b94640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587962c3480 0x5587962cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:26.031 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c2b96640 1 --2- 172.21.15.172:0/1659140062 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98a80a2b80 0x7f98a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c1b94640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587962c3480 0x5587962cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:56434/0 (socket says 172.21.15.172:56434) 2024-09-10T08:59:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c2395640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c2b60 0x5587962c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c1b94640 1 -- 172.21.15.172:0/2691902563 learned_addr learned my addr 172.21.15.172:0/2691902563 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98b37fe640 1 -- 172.21.15.172:0/1659140062 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f98ac0eff40 con 0x7f98bc102570 2024-09-10T08:59:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c2b96640 1 --2- 172.21.15.172:0/2691902563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962cdd40 0x5587962d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c2395640 1 -- 172.21.15.172:0/2691902563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587962c3480 msgr2=0x5587962cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c2395640 1 --2- 172.21.15.172:0/2691902563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587962c3480 0x5587962cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c2395640 1 -- 172.21.15.172:0/2691902563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962cdd40 msgr2=0x5587962d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c2395640 1 --2- 172.21.15.172:0/2691902563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962cdd40 0x5587962d0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c2395640 1 -- 172.21.15.172:0/2691902563 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587961dc1f0 con 0x5587962c2b60 2024-09-10T08:59:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c2b96640 1 --2- 172.21.15.172:0/1659140062 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98a80a2b80 0x7f98a80a5040 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009fd0 tx=0x7f98b4009d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.624+0000 7f98c2395640 1 --2- 172.21.15.172:0/2691902563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c2b60 0x5587962c2f40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009cf0 tx=0x7f98b8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b38d8fa55783bab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:26.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.625+0000 7f98b0ff9640 1 -- 172.21.15.172:0/2691902563 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b802e020 con 0x5587962c2b60 2024-09-10T08:59:26.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.625+0000 7f98b0ff9640 1 -- 172.21.15.172:0/2691902563 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98b80047a0 con 0x5587962c2b60 2024-09-10T08:59:26.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.625+0000 7f98b0ff9640 1 -- 172.21.15.172:0/2691902563 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b802a680 con 0x5587962c2b60 2024-09-10T08:59:26.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.626+0000 7f98c69d2740 1 -- 172.21.15.172:0/2691902563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c2b60 msgr2=0x5587962c2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:26.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.626+0000 7f98c69d2740 1 --2- 172.21.15.172:0/2691902563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c2b60 0x5587962c2f40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009cf0 tx=0x7f98b8011810 comp rx=0 tx=0).stop 2024-09-10T08:59:26.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.626+0000 7f98c69d2740 1 -- 172.21.15.172:0/2691902563 shutdown_connections 2024-09-10T08:59:26.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.626+0000 7f98c69d2740 1 --2- 172.21.15.172:0/2691902563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962cdd40 0x5587962d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.626+0000 7f98c69d2740 1 --2- 172.21.15.172:0/2691902563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587962c3480 0x5587962cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.626+0000 7f98c69d2740 1 --2- 172.21.15.172:0/2691902563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c2b60 0x5587962c2f40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.626+0000 7f98c69d2740 1 -- 172.21.15.172:0/2691902563 >> 172.21.15.172:0/2691902563 conn(0x5587962bf0f0 msgr2=0x5587962bf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:26.033 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.626+0000 7f98c69d2740 1 -- 172.21.15.172:0/2691902563 shutdown_connections 2024-09-10T08:59:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.626+0000 7f98c69d2740 1 -- 172.21.15.172:0/2691902563 wait complete. 2024-09-10T08:59:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c69d2740 1 -- start start 2024-09-10T08:59:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c69d2740 1 Processor -- start 2024-09-10T08:59:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c69d2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c2b60 0x5587962cac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c69d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c3480 0x5587962cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c69d2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587962cdd40 0x5587962cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c69d2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5587962d0910 con 0x5587962c2b60 2024-09-10T08:59:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c69d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5587961e2f10 con 0x5587962c3480 2024-09-10T08:59:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c69d2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5587962d0a90 con 0x5587962cdd40 2024-09-10T08:59:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c69d2740 10 client.0 _ll_register_callbacks cb 0x5587961aba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:59:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c2395640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c2b60 0x5587962cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c2395640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c2b60 0x5587962cac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57822/0 (socket says 172.21.15.172:57822) 2024-09-10T08:59:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c2395640 1 -- 172.21.15.172:0/3983101611 learned_addr learned my addr 172.21.15.172:0/3983101611 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:26.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c2b96640 1 --2- 172.21.15.172:0/3983101611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587962cdd40 0x5587962cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c1b94640 1 --2- 172.21.15.172:0/3983101611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c3480 0x5587962cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c2395640 1 -- 172.21.15.172:0/3983101611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587962cdd40 msgr2=0x5587962cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c2395640 1 --2- 172.21.15.172:0/3983101611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587962cdd40 0x5587962cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c2395640 1 -- 172.21.15.172:0/3983101611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c3480 msgr2=0x5587962cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c2395640 1 --2- 172.21.15.172:0/3983101611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c3480 0x5587962cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c2395640 1 -- 172.21.15.172:0/3983101611 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558795f996a0 con 0x5587962c2b60 2024-09-10T08:59:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.627+0000 7f98c1b94640 1 --2- 172.21.15.172:0/3983101611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c3480 0x5587962cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:59:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.628+0000 7f98c2395640 1 --2- 172.21.15.172:0/3983101611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c2b60 0x5587962cac40 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f98b8004da0 tx=0x7f98b8004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719400f4914720ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.628+0000 7f98a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:59:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.628+0000 7f98a5ffb640 1 -- 172.21.15.172:0/3983101611 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8039070 con 0x5587962c2b60 2024-09-10T08:59:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.628+0000 7f98a5ffb640 1 -- 172.21.15.172:0/3983101611 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f98b8005410 con 0x5587962c2b60 2024-09-10T08:59:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.628+0000 7f98c69d2740 1 -- 172.21.15.172:0/3983101611 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98b80047a0 con 0x5587962c2b60 2024-09-10T08:59:26.035 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.628+0000 7f98a5ffb640 1 -- 172.21.15.172:0/3983101611 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f98b8045050 con 0x5587962c2b60 2024-09-10T08:59:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.628+0000 7f98c69d2740 10 client.19281 fetch_fsmap learned FSMap version 41 2024-09-10T08:59:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.628+0000 7f98c69d2740 1 -- 172.21.15.172:0/3983101611 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5587962dc540 con 0x5587962c2b60 2024-09-10T08:59:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.628+0000 7f98a5ffb640 1 -- 172.21.15.172:0/3983101611 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98b80047a0 con 0x5587962c2b60 2024-09-10T08:59:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.628+0000 7f98a5ffb640 1 -- 172.21.15.172:0/3983101611 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f98b8033040 con 0x5587962c2b60 2024-09-10T08:59:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.628+0000 7f98c69d2740 10 client.19281 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T08:59:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.628+0000 7f98c69d2740 10 client.19281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:59:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.628+0000 7f98c69d2740 20 client.19281 populate_metadata read hostname 'smithi172' 2024-09-10T08:59:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.629+0000 7f98c69d2740 1 --2- 172.21.15.172:0/3983101611 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5587962dd8f0 0x5587962dfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.629+0000 7f98c69d2740 4 client.19281 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:59:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.629+0000 7f98c69d2740 1 -- 172.21.15.172:0/3983101611 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587962357f0 con 0x5587962dd8f0 2024-09-10T08:59:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.629+0000 7f98c1b94640 1 --2- 172.21.15.172:0/3983101611 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5587962dd8f0 0x5587962dfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.629+0000 7f98a5ffb640 10 client.19281 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.629+0000 7f98c1b94640 1 --2- 172.21.15.172:0/3983101611 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5587962dd8f0 0x5587962dfce0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0ba3264dd9f02aa server_cookie=4e25ab6bdd9d27fa in_seq=0 out_seq=0 2024-09-10T08:59:26.036 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.630+0000 7f98a5ffb640 1 -- 172.21.15.172:0/3983101611 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587962357f0 con 0x5587962dd8f0 2024-09-10T08:59:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.630+0000 7f98a5ffb640 10 client.19281 handle_command_reply: tid=0 2024-09-10T08:59:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.630+0000 7f98c69d2740 1 client.19281 shutdown 2024-09-10T08:59:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.630+0000 7f98c69d2740 1 -- 172.21.15.172:0/3983101611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c2b60 msgr2=0x5587962cac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.630+0000 7f98c69d2740 1 --2- 172.21.15.172:0/3983101611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c2b60 0x5587962cac40 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f98b8004da0 tx=0x7f98b8004dd0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.630+0000 7f98c69d2740 1 -- 172.21.15.172:0/3983101611 shutdown_connections 2024-09-10T08:59:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.630+0000 7f98c69d2740 1 --2- 172.21.15.172:0/3983101611 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5587962dd8f0 0x5587962dfce0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.630+0000 7f98c69d2740 1 --2- 172.21.15.172:0/3983101611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587962cdd40 0x5587962cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.630+0000 7f98a5ffb640 0 client.19281 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.630+0000 7f98c69d2740 1 --2- 172.21.15.172:0/3983101611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c3480 0x5587962cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.630+0000 7f98c69d2740 1 --2- 172.21.15.172:0/3983101611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c2b60 0x5587962cac40 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.630+0000 7f98c69d2740 1 -- 172.21.15.172:0/3983101611 >> 172.21.15.172:0/3983101611 conn(0x5587962bf0f0 msgr2=0x5587962be270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.630+0000 7f98c69d2740 1 -- 172.21.15.172:0/3983101611 shutdown_connections 2024-09-10T08:59:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.630+0000 7f98c69d2740 1 -- 172.21.15.172:0/3983101611 wait complete. 2024-09-10T08:59:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.630+0000 7f98c69d2740 20 client.19281 trim_cache size 0 max 16384 2024-09-10T08:59:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.647+0000 7f98c69d2740 1 Processor -- start 2024-09-10T08:59:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.647+0000 7f98c69d2740 1 -- start start 2024-09-10T08:59:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.647+0000 7f98c69d2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c2b60 0x5587962c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.647+0000 7f98c69d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c3480 0x5587962e5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.647+0000 7f98c69d2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587962cdd40 0x5587962e5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98c69d2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1ac940 con 0x5587962c2b60 2024-09-10T08:59:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98c69d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1ae540 con 0x5587962c3480 2024-09-10T08:59:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98c69d2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1ad010 con 0x5587962cdd40 2024-09-10T08:59:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98c2395640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c2b60 0x5587962c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98c2395640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c2b60 0x5587962c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57824/0 (socket says 172.21.15.172:57824) 2024-09-10T08:59:26.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98c2395640 1 -- 172.21.15.172:0/1725615566 learned_addr learned my addr 172.21.15.172:0/1725615566 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98c1b94640 1 --2- 172.21.15.172:0/1725615566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c3480 0x5587962e5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98c2b96640 1 --2- 172.21.15.172:0/1725615566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587962cdd40 0x5587962e5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98c2395640 1 -- 172.21.15.172:0/1725615566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587962cdd40 msgr2=0x5587962e5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98c2395640 1 --2- 172.21.15.172:0/1725615566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587962cdd40 0x5587962e5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98c2395640 1 -- 172.21.15.172:0/1725615566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c3480 msgr2=0x5587962e5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98c2395640 1 --2- 172.21.15.172:0/1725615566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c3480 0x5587962e5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98c2395640 1 -- 172.21.15.172:0/1725615566 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587962bfd80 con 0x5587962c2b60 2024-09-10T08:59:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98c1b94640 1 --2- 172.21.15.172:0/1725615566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c3480 0x5587962e5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T08:59:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98c2b96640 1 --2- 172.21.15.172:0/1725615566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587962cdd40 0x5587962e5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:59:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98c2395640 1 --2- 172.21.15.172:0/1725615566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c2b60 0x5587962c9af0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009250 tx=0x7f98b8011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271b8c3e66bb30fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98a7fff640 1 -- 172.21.15.172:0/1725615566 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8039070 con 0x5587962c2b60 2024-09-10T08:59:26.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98a7fff640 1 -- 172.21.15.172:0/1725615566 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98b8042030 con 0x5587962c2b60 2024-09-10T08:59:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.648+0000 7f98a7fff640 1 -- 172.21.15.172:0/1725615566 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b802e030 con 0x5587962c2b60 2024-09-10T08:59:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.649+0000 7f98c69d2740 1 -- 172.21.15.172:0/1725615566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c2b60 msgr2=0x5587962c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.649+0000 7f98c69d2740 1 --2- 172.21.15.172:0/1725615566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c2b60 0x5587962c9af0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009250 tx=0x7f98b8011d10 comp rx=0 tx=0).stop 2024-09-10T08:59:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.649+0000 7f98c69d2740 1 -- 172.21.15.172:0/1725615566 shutdown_connections 2024-09-10T08:59:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.649+0000 7f98c69d2740 1 --2- 172.21.15.172:0/1725615566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587962cdd40 0x5587962e5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.649+0000 7f98c69d2740 1 --2- 172.21.15.172:0/1725615566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962c3480 0x5587962e5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.649+0000 7f98c69d2740 1 --2- 172.21.15.172:0/1725615566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c2b60 0x5587962c9af0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.649+0000 7f98c69d2740 1 -- 172.21.15.172:0/1725615566 >> 172.21.15.172:0/1725615566 conn(0x5587962bf0f0 msgr2=0x5587962c93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.649+0000 7f98c69d2740 1 -- 172.21.15.172:0/1725615566 shutdown_connections 2024-09-10T08:59:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.649+0000 7f98c69d2740 1 -- 172.21.15.172:0/1725615566 wait complete. 2024-09-10T08:59:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.649+0000 7f98c69d2740 1 -- start start 2024-09-10T08:59:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.649+0000 7f98c69d2740 1 Processor -- start 2024-09-10T08:59:26.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c69d2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c3480 0x5587962d2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c69d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962cdd40 0x5587962d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c69d2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587963452d0 0x5587963476c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c69d2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1ade70 con 0x5587962c3480 2024-09-10T08:59:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c69d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1ad7a0 con 0x5587962cdd40 2024-09-10T08:59:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c69d2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1aa940 con 0x5587963452d0 2024-09-10T08:59:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c2395640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c3480 0x5587962d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c69d2740 10 client.0 _ll_register_callbacks cb 0x7f98b401bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:59:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c2395640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c3480 0x5587962d2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57830/0 (socket says 172.21.15.172:57830) 2024-09-10T08:59:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c2395640 1 -- 172.21.15.172:0/854945015 learned_addr learned my addr 172.21.15.172:0/854945015 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c1b94640 1 --2- 172.21.15.172:0/854945015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962cdd40 0x5587962d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c2b96640 1 --2- 172.21.15.172:0/854945015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587963452d0 0x5587963476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c2395640 1 -- 172.21.15.172:0/854945015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587963452d0 msgr2=0x5587963476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:26.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c2395640 1 --2- 172.21.15.172:0/854945015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587963452d0 0x5587963476c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c2395640 1 -- 172.21.15.172:0/854945015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962cdd40 msgr2=0x5587962d25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c2395640 1 --2- 172.21.15.172:0/854945015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962cdd40 0x5587962d25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c2395640 1 -- 172.21.15.172:0/854945015 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558796347c00 con 0x5587962c3480 2024-09-10T08:59:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c2b96640 1 --2- 172.21.15.172:0/854945015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587963452d0 0x5587963476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T08:59:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c2395640 1 --2- 172.21.15.172:0/854945015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c3480 0x5587962d2080 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f98b8037040 tx=0x7f98b8005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4700c369ea390a12 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:59:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98a5ffb640 1 -- 172.21.15.172:0/854945015 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b802e070 con 0x5587962c3480 2024-09-10T08:59:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98a5ffb640 1 -- 172.21.15.172:0/854945015 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f98b8005210 con 0x5587962c3480 2024-09-10T08:59:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.650+0000 7f98c69d2740 1 -- 172.21.15.172:0/854945015 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98b8042030 con 0x5587962c3480 2024-09-10T08:59:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.651+0000 7f98a5ffb640 1 -- 172.21.15.172:0/854945015 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f98b805b050 con 0x5587962c3480 2024-09-10T08:59:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.651+0000 7f98c69d2740 10 client.19287 fetch_fsmap learned FSMap version 41 2024-09-10T08:59:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.651+0000 7f98c69d2740 1 -- 172.21.15.172:0/854945015 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558796348e50 con 0x5587962c3480 2024-09-10T08:59:26.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.651+0000 7f98a5ffb640 1 -- 172.21.15.172:0/854945015 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98b8042030 con 0x5587962c3480 2024-09-10T08:59:26.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.651+0000 7f98a5ffb640 1 -- 172.21.15.172:0/854945015 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f98b8051030 con 0x5587962c3480 2024-09-10T08:59:26.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.651+0000 7f98c69d2740 10 client.19287 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T08:59:26.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.651+0000 7f98c69d2740 10 client.19287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T08:59:26.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.651+0000 7f98c69d2740 20 client.19287 populate_metadata read hostname 'smithi172' 2024-09-10T08:59:26.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.651+0000 7f98c69d2740 1 --2- 172.21.15.172:0/854945015 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5587963493b0 0x55879634b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:26.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.651+0000 7f98c69d2740 4 client.19287 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T08:59:26.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.651+0000 7f98c69d2740 1 -- 172.21.15.172:0/854945015 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55879622fd00 con 0x5587963493b0 2024-09-10T08:59:26.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.651+0000 7f98c1b94640 1 --2- 172.21.15.172:0/854945015 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5587963493b0 0x55879634b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:26.043 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.652+0000 7f98c1b94640 1 --2- 172.21.15.172:0/854945015 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5587963493b0 0x55879634b7a0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e48c0511b72c3c8 server_cookie=53ca9c30667f5411 in_seq=0 out_seq=0 2024-09-10T08:59:26.043 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T08:59:26.044 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T08:59:26.044 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "791d24f9-bd30-4b82-85de-e37e74b77a02", 2024-09-10T08:59:26.044 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T08:59:26.044 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T08:59:26.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.652+0000 7f98a5ffb640 10 client.19287 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:26.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.652+0000 7f98a5ffb640 1 -- 172.21.15.172:0/854945015 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55879622fd00 con 0x5587963493b0 2024-09-10T08:59:26.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.652+0000 7f98a5ffb640 10 client.19287 handle_command_reply: tid=0 2024-09-10T08:59:26.044 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.652+0000 7f98c69d2740 1 client.19287 shutdown 2024-09-10T08:59:26.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.652+0000 7f98c69d2740 1 -- 172.21.15.172:0/854945015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c3480 msgr2=0x5587962d2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:26.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.652+0000 7f98c69d2740 1 --2- 172.21.15.172:0/854945015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c3480 0x5587962d2080 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f98b8037040 tx=0x7f98b8005f80 comp rx=0 tx=0).stop 2024-09-10T08:59:26.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.652+0000 7f98c69d2740 1 -- 172.21.15.172:0/854945015 shutdown_connections 2024-09-10T08:59:26.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c69d2740 1 --2- 172.21.15.172:0/854945015 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5587963493b0 0x55879634b7a0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c69d2740 1 --2- 172.21.15.172:0/854945015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587963452d0 0x5587963476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c69d2740 1 --2- 172.21.15.172:0/854945015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587962cdd40 0x5587962d25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98a5ffb640 0 client.19287 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:26.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c69d2740 1 --2- 172.21.15.172:0/854945015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587962c3480 0x5587962d2080 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c69d2740 1 -- 172.21.15.172:0/854945015 >> 172.21.15.172:0/854945015 conn(0x5587962bf0f0 msgr2=0x5587962c4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:26.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c69d2740 1 -- 172.21.15.172:0/854945015 shutdown_connections 2024-09-10T08:59:26.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c69d2740 1 -- 172.21.15.172:0/854945015 wait complete. 2024-09-10T08:59:26.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c69d2740 20 client.19287 trim_cache size 0 max 16384 2024-09-10T08:59:26.045 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c45f9640 1 -- 172.21.15.172:0/1659140062 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98a80a2b80 msgr2=0x7f98a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:26.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c45f9640 1 --2- 172.21.15.172:0/1659140062 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98a80a2b80 0x7f98a80a5040 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009fd0 tx=0x7f98b4009d40 comp rx=0 tx=0).stop 2024-09-10T08:59:26.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c45f9640 1 -- 172.21.15.172:0/1659140062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98bc102570 msgr2=0x7f98bc0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:26.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c45f9640 1 --2- 172.21.15.172:0/1659140062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98bc102570 0x7f98bc0fd1c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009890 tx=0x7f98ac030ce0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c45f9640 1 -- 172.21.15.172:0/1659140062 shutdown_connections 2024-09-10T08:59:26.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c45f9640 1 --2- 172.21.15.172:0/1659140062 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f98a80a2b80 0x7f98a80a5040 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c45f9640 1 --2- 172.21.15.172:0/1659140062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98bc102ef0 0x7f98bc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c45f9640 1 --2- 172.21.15.172:0/1659140062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98bc102570 0x7f98bc0fd1c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c45f9640 1 --2- 172.21.15.172:0/1659140062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98bc100e20 0x7f98bc0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:26.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c45f9640 1 -- 172.21.15.172:0/1659140062 >> 172.21.15.172:0/1659140062 conn(0x7f98bc0fc7d0 msgr2=0x7f98bc06a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:26.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c45f9640 1 -- 172.21.15.172:0/1659140062 shutdown_connections 2024-09-10T08:59:26.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:24.653+0000 7f98c45f9640 1 -- 172.21.15.172:0/1659140062 wait complete. 2024-09-10T08:59:26.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:25 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T08:59:26.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:25 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:26.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:25 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:59:26.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:25 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:59:26.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:25 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:59:26.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:25 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T08:59:26.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:25 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:26.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:25 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:26.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:25 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:26.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:25 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:26.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:25 smithi172 ceph-mon[30854]: pgmap v746: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.95k op/s 2024-09-10T08:59:26.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:25 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T08:59:26.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:25 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:26.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:25 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:59:26.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:25 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:59:26.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:25 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:59:26.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:25 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T08:59:26.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:25 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:26.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:25 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:26.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:25 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:26.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:25 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:26.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:25 smithi174 ceph-mon[34068]: pgmap v746: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.95k op/s 2024-09-10T08:59:26.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:25 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T08:59:26.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:25 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:26.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:25 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:59:26.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:25 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:59:26.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:25 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:59:26.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:25 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T08:59:26.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:25 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:26.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:25 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:26.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:25 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:26.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:25 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T08:59:26.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:25 smithi191 ceph-mon[35314]: pgmap v746: 129 pgs: 129 active+clean; 13 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 15 MiB/s wr, 3.95k op/s 2024-09-10T08:59:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:27 smithi174 ceph-mon[34068]: pgmap v747: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 3.60k op/s 2024-09-10T08:59:27.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:27 smithi191 ceph-mon[35314]: pgmap v747: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 3.60k op/s 2024-09-10T08:59:27.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:27 smithi172 ceph-mon[30854]: pgmap v747: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 3.60k op/s 2024-09-10T08:59:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.903+0000 7f5ece791640 1 -- 172.21.15.172:0/1001239226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec8101690 msgr2=0x7f5ec8108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.903+0000 7f5ece791640 1 --2- 172.21.15.172:0/1001239226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec8101690 0x7f5ec8108f30 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009ca0 tx=0x7f5ec002a8c0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.903+0000 7f5ece791640 1 -- 172.21.15.172:0/1001239226 shutdown_connections 2024-09-10T08:59:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.903+0000 7f5ece791640 1 --2- 172.21.15.172:0/1001239226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec8101690 0x7f5ec8108f30 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.903+0000 7f5ece791640 1 --2- 172.21.15.172:0/1001239226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ec8100cd0 0x7f5ec8101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.903+0000 7f5ece791640 1 --2- 172.21.15.172:0/1001239226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ec8102030 0x7f5ec8102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.903+0000 7f5ece791640 1 -- 172.21.15.172:0/1001239226 >> 172.21.15.172:0/1001239226 conn(0x7f5ec80fb970 msgr2=0x7f5ec80fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.903+0000 7f5ece791640 1 -- 172.21.15.172:0/1001239226 shutdown_connections 2024-09-10T08:59:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.903+0000 7f5ece791640 1 -- 172.21.15.172:0/1001239226 wait complete. 2024-09-10T08:59:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5ece791640 1 Processor -- start 2024-09-10T08:59:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5ece791640 1 -- start start 2024-09-10T08:59:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5ece791640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ec8100cd0 0x7f5ec8197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5ece791640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec8101690 0x7f5ec8198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5ece791640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ec8102030 0x7f5ec819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5ece791640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec810c5c0 con 0x7f5ec8101690 2024-09-10T08:59:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5ece791640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec810c440 con 0x7f5ec8102030 2024-09-10T08:59:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5ece791640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec810c740 con 0x7f5ec8100cd0 2024-09-10T08:59:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ec8100cd0 0x7f5ec8197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5ec77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec8101690 0x7f5ec8198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ec8100cd0 0x7f5ec8197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:40748/0 (socket says 172.21.15.172:40748) 2024-09-10T08:59:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5ec7fff640 1 -- 172.21.15.172:0/2529755248 learned_addr learned my addr 172.21.15.172:0/2529755248 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5eccd2e640 1 --2- 172.21.15.172:0/2529755248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ec8102030 0x7f5ec819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5ec77fe640 1 -- 172.21.15.172:0/2529755248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ec8100cd0 msgr2=0x7f5ec8197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5ec77fe640 1 --2- 172.21.15.172:0/2529755248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ec8100cd0 0x7f5ec8197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5ec77fe640 1 -- 172.21.15.172:0/2529755248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ec8102030 msgr2=0x7f5ec819f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5ec77fe640 1 --2- 172.21.15.172:0/2529755248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ec8102030 0x7f5ec819f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.904+0000 7f5ec77fe640 1 -- 172.21.15.172:0/2529755248 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec819f860 con 0x7f5ec8101690 2024-09-10T08:59:28.909 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.905+0000 7f5ec77fe640 1 --2- 172.21.15.172:0/2529755248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec8101690 0x7f5ec8198190 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00d6d0 tx=0x7f5ebc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:28.909 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.905+0000 7f5ec57fa640 1 -- 172.21.15.172:0/2529755248 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc017070 con 0x7f5ec8101690 2024-09-10T08:59:28.909 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.905+0000 7f5ec57fa640 1 -- 172.21.15.172:0/2529755248 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ebc009820 con 0x7f5ec8101690 2024-09-10T08:59:28.909 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.905+0000 7f5ece791640 1 -- 172.21.15.172:0/2529755248 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec819fb50 con 0x7f5ec8101690 2024-09-10T08:59:28.909 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.905+0000 7f5ec57fa640 1 -- 172.21.15.172:0/2529755248 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc004c40 con 0x7f5ec8101690 2024-09-10T08:59:28.909 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.905+0000 7f5ece791640 1 -- 172.21.15.172:0/2529755248 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec81a00a0 con 0x7f5ec8101690 2024-09-10T08:59:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.907+0000 7f5ec57fa640 1 -- 172.21.15.172:0/2529755248 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5ebc0099d0 con 0x7f5ec8101690 2024-09-10T08:59:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.907+0000 7f5ed160c740 1 Processor -- start 2024-09-10T08:59:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5ed160c740 1 -- start start 2024-09-10T08:59:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5ed160c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae239cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5ed160c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae239d2e0 0x55cae23a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5ed160c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae23a2370 0x55cae23a4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5ed160c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cae23002f0 con 0x55cae239c9a0 2024-09-10T08:59:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5ed160c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cae22a8270 con 0x55cae239d2e0 2024-09-10T08:59:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5ed160c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55cae22fc2b0 con 0x55cae23a2370 2024-09-10T08:59:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5ec57fa640 1 --2- 172.21.15.172:0/2529755248 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5eb00a2c30 0x7f5eb00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5eccd2e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae239cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5eccd2e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae239cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42758/0 (socket says 172.21.15.172:42758) 2024-09-10T08:59:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5eccd2e640 1 -- 172.21.15.172:0/1527575897 learned_addr learned my addr 172.21.15.172:0/1527575897 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5ec57fa640 1 -- 172.21.15.172:0/2529755248 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5ebc0ceeb0 con 0x7f5ec8101690 2024-09-10T08:59:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5ec77fe640 1 --2- 172.21.15.172:0/1527575897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae23a2370 0x55cae23a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5ec7fff640 1 --2- 172.21.15.172:0/1527575897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae239d2e0 0x55cae23a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5eccd2e640 1 --2- 172.21.15.172:0/2529755248 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5eb00a2c30 0x7f5eb00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5eccd2e640 1 -- 172.21.15.172:0/1527575897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae23a2370 msgr2=0x55cae23a4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5eccd2e640 1 --2- 172.21.15.172:0/1527575897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae23a2370 0x55cae23a4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5eccd2e640 1 -- 172.21.15.172:0/1527575897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae239d2e0 msgr2=0x55cae23a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5eccd2e640 1 --2- 172.21.15.172:0/1527575897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae239d2e0 0x55cae23a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5eccd2e640 1 -- 172.21.15.172:0/1527575897 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae22a7340 con 0x55cae239c9a0 2024-09-10T08:59:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5eccd2e640 1 --2- 172.21.15.172:0/1527575897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae239cda0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f5ec002aeb0 tx=0x7f5ec0032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af88f0bb50c849f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.908+0000 7f5eccd2e640 1 --2- 172.21.15.172:0/2529755248 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5eb00a2c30 0x7f5eb00a50f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009ca0 tx=0x7f5ec003b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.909+0000 7f5eae7fc640 1 -- 172.21.15.172:0/1527575897 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec004e070 con 0x55cae239c9a0 2024-09-10T08:59:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.909+0000 7f5eae7fc640 1 -- 172.21.15.172:0/1527575897 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ec0036040 con 0x55cae239c9a0 2024-09-10T08:59:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.909+0000 7f5eae7fc640 1 -- 172.21.15.172:0/1527575897 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0004830 con 0x55cae239c9a0 2024-09-10T08:59:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.909+0000 7f5ed160c740 1 -- 172.21.15.172:0/1527575897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 msgr2=0x55cae239cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.909+0000 7f5ed160c740 1 --2- 172.21.15.172:0/1527575897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae239cda0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f5ec002aeb0 tx=0x7f5ec0032df0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.909+0000 7f5ed160c740 1 -- 172.21.15.172:0/1527575897 shutdown_connections 2024-09-10T08:59:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.909+0000 7f5ed160c740 1 --2- 172.21.15.172:0/1527575897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae23a2370 0x55cae23a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.909+0000 7f5ed160c740 1 --2- 172.21.15.172:0/1527575897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae239d2e0 0x55cae23a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.909+0000 7f5ed160c740 1 --2- 172.21.15.172:0/1527575897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae239cda0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.909+0000 7f5ed160c740 1 -- 172.21.15.172:0/1527575897 >> 172.21.15.172:0/1527575897 conn(0x55cae2398010 msgr2=0x55cae2398410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.909+0000 7f5ed160c740 1 -- 172.21.15.172:0/1527575897 shutdown_connections 2024-09-10T08:59:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.909+0000 7f5ed160c740 1 -- 172.21.15.172:0/1527575897 wait complete. 2024-09-10T08:59:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ed160c740 1 -- start start 2024-09-10T08:59:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ed160c740 1 Processor -- start 2024-09-10T08:59:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ed160c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ed160c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae239d2e0 0x55cae23a6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ed160c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae23a2370 0x55cae23adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ed160c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cae23a5160 con 0x55cae239c9a0 2024-09-10T08:59:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ed160c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cae2289290 con 0x55cae239d2e0 2024-09-10T08:59:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ed160c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55cae23a52e0 con 0x55cae23a2370 2024-09-10T08:59:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ed160c740 10 client.0 _ll_register_callbacks cb 0x55cae22ba140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:59:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23a6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42772/0 (socket says 172.21.15.172:42772) 2024-09-10T08:59:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ec7fff640 1 -- 172.21.15.172:0/1109329670 learned_addr learned my addr 172.21.15.172:0/1109329670 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5eccd2e640 1 --2- 172.21.15.172:0/1109329670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae23a2370 0x55cae23adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ec77fe640 1 --2- 172.21.15.172:0/1109329670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae239d2e0 0x55cae23a6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ec7fff640 1 -- 172.21.15.172:0/1109329670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae23a2370 msgr2=0x55cae23adfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ec7fff640 1 --2- 172.21.15.172:0/1109329670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae23a2370 0x55cae23adfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ec7fff640 1 -- 172.21.15.172:0/1109329670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae239d2e0 msgr2=0x55cae23a6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ec7fff640 1 --2- 172.21.15.172:0/1109329670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae239d2e0 0x55cae23a6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.910+0000 7f5ec7fff640 1 -- 172.21.15.172:0/1109329670 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae2013210 con 0x55cae239c9a0 2024-09-10T08:59:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.911+0000 7f5ec7fff640 1 --2- 172.21.15.172:0/1109329670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23a6a50 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8011470 tx=0x7f5eb8011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1dbf4f257f05dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.911+0000 7f5ea77fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:59:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.911+0000 7f5ea77fe640 1 -- 172.21.15.172:0/1109329670 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8004020 con 0x55cae239c9a0 2024-09-10T08:59:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.911+0000 7f5ea77fe640 1 -- 172.21.15.172:0/1109329670 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5eb80041c0 con 0x55cae239c9a0 2024-09-10T08:59:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.911+0000 7f5ed160c740 1 -- 172.21.15.172:0/1109329670 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ec0036040 con 0x55cae239c9a0 2024-09-10T08:59:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.911+0000 7f5ea77fe640 1 -- 172.21.15.172:0/1109329670 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5eb8025050 con 0x55cae239c9a0 2024-09-10T08:59:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.911+0000 7f5ed160c740 10 client.19299 fetch_fsmap learned FSMap version 41 2024-09-10T08:59:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.911+0000 7f5ed160c740 1 -- 172.21.15.172:0/1109329670 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55cae20df620 con 0x55cae239c9a0 2024-09-10T08:59:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.911+0000 7f5ea77fe640 1 -- 172.21.15.172:0/1109329670 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ec0036040 con 0x55cae239c9a0 2024-09-10T08:59:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.911+0000 7f5ea77fe640 1 -- 172.21.15.172:0/1109329670 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f5eb8003780 con 0x55cae239c9a0 2024-09-10T08:59:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.911+0000 7f5ed160c740 10 client.19299 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T08:59:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.911+0000 7f5ed160c740 10 client.19299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:59:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.911+0000 7f5ed160c740 20 client.19299 populate_metadata read hostname 'smithi172' 2024-09-10T08:59:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.911+0000 7f5ed160c740 1 --2- 172.21.15.172:0/1109329670 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cae23b2730 0x55cae23b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.911+0000 7f5ed160c740 4 client.19299 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:59:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.911+0000 7f5ed160c740 1 -- 172.21.15.172:0/1109329670 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cae230da20 con 0x55cae23b2730 2024-09-10T08:59:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.912+0000 7f5ec77fe640 1 --2- 172.21.15.172:0/1109329670 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cae23b2730 0x55cae23b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.912+0000 7f5ec77fe640 1 --2- 172.21.15.172:0/1109329670 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cae23b2730 0x55cae23b4bb0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54b0913ed55e96ae server_cookie=7f1a06c28efd87fd in_seq=0 out_seq=0 2024-09-10T08:59:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.912+0000 7f5ea77fe640 10 client.19299 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.913+0000 7f5ea77fe640 1 -- 172.21.15.172:0/1109329670 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cae230da20 con 0x55cae23b2730 2024-09-10T08:59:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.913+0000 7f5ea77fe640 10 client.19299 handle_command_reply: tid=0 2024-09-10T08:59:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.913+0000 7f5ed160c740 1 client.19299 shutdown 2024-09-10T08:59:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.913+0000 7f5ed160c740 1 -- 172.21.15.172:0/1109329670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 msgr2=0x55cae23a6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.913+0000 7f5ed160c740 1 --2- 172.21.15.172:0/1109329670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23a6a50 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8011470 tx=0x7f5eb8011930 comp rx=0 tx=0).stop 2024-09-10T08:59:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.913+0000 7f5ed160c740 1 -- 172.21.15.172:0/1109329670 shutdown_connections 2024-09-10T08:59:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.913+0000 7f5ed160c740 1 --2- 172.21.15.172:0/1109329670 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cae23b2730 0x55cae23b4bb0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.913+0000 7f5ed160c740 1 --2- 172.21.15.172:0/1109329670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae23a2370 0x55cae23adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.913+0000 7f5ed160c740 1 --2- 172.21.15.172:0/1109329670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae239d2e0 0x55cae23a6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.913+0000 7f5ea77fe640 0 client.19299 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.913+0000 7f5ed160c740 1 --2- 172.21.15.172:0/1109329670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23a6a50 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.913+0000 7f5ed160c740 1 -- 172.21.15.172:0/1109329670 >> 172.21.15.172:0/1109329670 conn(0x55cae2398010 msgr2=0x55cae2399820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.913+0000 7f5ed160c740 1 -- 172.21.15.172:0/1109329670 shutdown_connections 2024-09-10T08:59:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.913+0000 7f5ed160c740 1 -- 172.21.15.172:0/1109329670 wait complete. 2024-09-10T08:59:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.913+0000 7f5ed160c740 20 client.19299 trim_cache size 0 max 16384 2024-09-10T08:59:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ed160c740 1 Processor -- start 2024-09-10T08:59:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ed160c740 1 -- start start 2024-09-10T08:59:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ed160c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ed160c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae239d2e0 0x55cae23adf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ed160c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae23a2370 0x55cae23b94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ed160c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81aa4b0 con 0x55cae239c9a0 2024-09-10T08:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ed160c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81ac0b0 con 0x55cae239d2e0 2024-09-10T08:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ed160c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81aab80 con 0x55cae23a2370 2024-09-10T08:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23a6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42778/0 (socket says 172.21.15.172:42778) 2024-09-10T08:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ec7fff640 1 -- 172.21.15.172:0/975051894 learned_addr learned my addr 172.21.15.172:0/975051894 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ec77fe640 1 --2- 172.21.15.172:0/975051894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae239d2e0 0x55cae23adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ec7fff640 1 -- 172.21.15.172:0/975051894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae23a2370 msgr2=0x55cae23b94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ec7fff640 1 --2- 172.21.15.172:0/975051894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae23a2370 0x55cae23b94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ec7fff640 1 -- 172.21.15.172:0/975051894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae239d2e0 msgr2=0x55cae23adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ec7fff640 1 --2- 172.21.15.172:0/975051894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae239d2e0 0x55cae23adf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ec7fff640 1 -- 172.21.15.172:0/975051894 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cae2399ea0 con 0x55cae239c9a0 2024-09-10T08:59:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5ec7fff640 1 --2- 172.21.15.172:0/975051894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23a6e70 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8000c00 tx=0x7f5eb800bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3b359d9590dc7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.927+0000 7f5eadffb640 1 -- 172.21.15.172:0/975051894 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb80035f0 con 0x55cae239c9a0 2024-09-10T08:59:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.928+0000 7f5eadffb640 1 -- 172.21.15.172:0/975051894 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5eb80054e0 con 0x55cae239c9a0 2024-09-10T08:59:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.928+0000 7f5eadffb640 1 -- 172.21.15.172:0/975051894 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8004220 con 0x55cae239c9a0 2024-09-10T08:59:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.928+0000 7f5ed160c740 1 -- 172.21.15.172:0/975051894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 msgr2=0x55cae23a6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.928+0000 7f5ed160c740 1 --2- 172.21.15.172:0/975051894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23a6e70 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8000c00 tx=0x7f5eb800bc60 comp rx=0 tx=0).stop 2024-09-10T08:59:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.928+0000 7f5ed160c740 1 -- 172.21.15.172:0/975051894 shutdown_connections 2024-09-10T08:59:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.928+0000 7f5ed160c740 1 --2- 172.21.15.172:0/975051894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae23a2370 0x55cae23b94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.928+0000 7f5ed160c740 1 --2- 172.21.15.172:0/975051894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae239d2e0 0x55cae23adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.928+0000 7f5ed160c740 1 --2- 172.21.15.172:0/975051894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23a6e70 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.928+0000 7f5ed160c740 1 -- 172.21.15.172:0/975051894 >> 172.21.15.172:0/975051894 conn(0x55cae2398010 msgr2=0x55cae23971c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.928+0000 7f5ed160c740 1 -- 172.21.15.172:0/975051894 shutdown_connections 2024-09-10T08:59:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.928+0000 7f5ed160c740 1 -- 172.21.15.172:0/975051894 wait complete. 2024-09-10T08:59:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.928+0000 7f5ed160c740 1 -- start start 2024-09-10T08:59:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ed160c740 1 Processor -- start 2024-09-10T08:59:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ed160c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ed160c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae239d2e0 0x55cae23ac480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ed160c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae23a2370 0x55cae23ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ed160c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81ab9e0 con 0x55cae239c9a0 2024-09-10T08:59:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ed160c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81ab310 con 0x55cae23a2370 2024-09-10T08:59:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ed160c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec81a8490 con 0x55cae239d2e0 2024-09-10T08:59:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ed160c740 10 client.0 _ll_register_callbacks cb 0x7f5eb800b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:59:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23ade70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42784/0 (socket says 172.21.15.172:42784) 2024-09-10T08:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ec7fff640 1 -- 172.21.15.172:0/549608543 learned_addr learned my addr 172.21.15.172:0/549608543 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5eccd2e640 1 --2- 172.21.15.172:0/549608543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae23a2370 0x55cae23ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ec77fe640 1 --2- 172.21.15.172:0/549608543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae239d2e0 0x55cae23ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ec7fff640 1 -- 172.21.15.172:0/549608543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae239d2e0 msgr2=0x55cae23ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ec7fff640 1 --2- 172.21.15.172:0/549608543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae239d2e0 0x55cae23ac480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ec7fff640 1 -- 172.21.15.172:0/549608543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae23a2370 msgr2=0x55cae23ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ec7fff640 1 --2- 172.21.15.172:0/549608543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae23a2370 0x55cae23ac9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ec7fff640 1 -- 172.21.15.172:0/549608543 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cae23ad1c0 con 0x55cae239c9a0 2024-09-10T08:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ec77fe640 1 --2- 172.21.15.172:0/549608543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae239d2e0 0x55cae23ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T08:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ec7fff640 1 --2- 172.21.15.172:0/549608543 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23ade70 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8005bc0 tx=0x7f5eb80041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cf6f2129d56021 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ea77fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T08:59:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ea77fe640 1 -- 172.21.15.172:0/549608543 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb80035f0 con 0x55cae239c9a0 2024-09-10T08:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ea77fe640 1 -- 172.21.15.172:0/549608543 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5eb802f050 con 0x55cae239c9a0 2024-09-10T08:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.929+0000 7f5ed160c740 1 -- 172.21.15.172:0/549608543 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eb80054e0 con 0x55cae239c9a0 2024-09-10T08:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.930+0000 7f5ea77fe640 1 -- 172.21.15.172:0/549608543 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5eb8040050 con 0x55cae239c9a0 2024-09-10T08:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.930+0000 7f5ed160c740 10 client.19305 fetch_fsmap learned FSMap version 41 2024-09-10T08:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.930+0000 7f5ed160c740 1 -- 172.21.15.172:0/549608543 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55cae241e4d0 con 0x55cae239c9a0 2024-09-10T08:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.930+0000 7f5ea77fe640 1 -- 172.21.15.172:0/549608543 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5eb80054e0 con 0x55cae239c9a0 2024-09-10T08:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.930+0000 7f5ea77fe640 1 -- 172.21.15.172:0/549608543 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f5eb801a560 con 0x55cae239c9a0 2024-09-10T08:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.930+0000 7f5ed160c740 10 client.19305 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T08:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.930+0000 7f5ed160c740 10 client.19305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.930+0000 7f5ed160c740 20 client.19305 populate_metadata read hostname 'smithi172' 2024-09-10T08:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.930+0000 7f5ed160c740 1 --2- 172.21.15.172:0/549608543 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cae241ed90 0x55cae24211b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.930+0000 7f5ed160c740 4 client.19305 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:59:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.930+0000 7f5ed160c740 1 -- 172.21.15.172:0/549608543 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cae2307d70 con 0x55cae241ed90 2024-09-10T08:59:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.930+0000 7f5ec77fe640 1 --2- 172.21.15.172:0/549608543 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cae241ed90 0x55cae24211b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.930+0000 7f5ea77fe640 10 client.19305 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:28.930+0000 7f5ec77fe640 1 --2- 172.21.15.172:0/549608543 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cae241ed90 0x55cae24211b0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20b7bfd5fc485912 server_cookie=80d4d75227b78365 in_seq=0 out_seq=0 2024-09-10T08:59:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.009+0000 7f5ea77fe640 1 -- 172.21.15.172:0/549608543 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cae2307d70 con 0x55cae241ed90 2024-09-10T08:59:29.010 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.009+0000 7f5ea77fe640 10 client.19305 handle_command_reply: tid=0 2024-09-10T08:59:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.013+0000 7f5ed160c740 1 client.19305 shutdown 2024-09-10T08:59:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.024+0000 7f5ed160c740 1 -- 172.21.15.172:0/549608543 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 msgr2=0x55cae23ade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.024+0000 7f5ed160c740 1 --2- 172.21.15.172:0/549608543 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23ade70 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8005bc0 tx=0x7f5eb80041a0 comp rx=0 tx=0).stop 2024-09-10T08:59:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.027+0000 7f5ed160c740 1 -- 172.21.15.172:0/549608543 shutdown_connections 2024-09-10T08:59:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.027+0000 7f5ed160c740 1 --2- 172.21.15.172:0/549608543 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55cae241ed90 0x55cae24211b0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.027+0000 7f5ed160c740 1 --2- 172.21.15.172:0/549608543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cae23a2370 0x55cae23ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.027+0000 7f5ed160c740 1 --2- 172.21.15.172:0/549608543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cae239d2e0 0x55cae23ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.027+0000 7f5ed160c740 1 --2- 172.21.15.172:0/549608543 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cae239c9a0 0x55cae23ade70 secure :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8005bc0 tx=0x7f5eb80041a0 comp rx=0 tx=0).stop 2024-09-10T08:59:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.027+0000 7f5ed160c740 1 -- 172.21.15.172:0/549608543 >> 172.21.15.172:0/549608543 conn(0x55cae2398010 msgr2=0x55cae23971c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.027+0000 7f5ea77fe640 0 client.19305 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.032+0000 7f5ed160c740 1 -- 172.21.15.172:0/549608543 shutdown_connections 2024-09-10T08:59:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.032+0000 7f5ed160c740 1 -- 172.21.15.172:0/549608543 wait complete. 2024-09-10T08:59:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.034+0000 7f5ed160c740 20 client.19305 trim_cache size 0 max 16384 2024-09-10T08:59:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:59:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:59:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:59:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:59:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.047+0000 7f5ece791640 1 -- 172.21.15.172:0/2529755248 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5eb00a2c30 msgr2=0x7f5eb00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.047+0000 7f5ece791640 1 --2- 172.21.15.172:0/2529755248 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5eb00a2c30 0x7f5eb00a50f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009ca0 tx=0x7f5ec003b000 comp rx=0 tx=0).stop 2024-09-10T08:59:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.047+0000 7f5ece791640 1 -- 172.21.15.172:0/2529755248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec8101690 msgr2=0x7f5ec8198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.047+0000 7f5ece791640 1 --2- 172.21.15.172:0/2529755248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec8101690 0x7f5ec8198190 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00d6d0 tx=0x7f5ebc00db90 comp rx=0 tx=0).stop 2024-09-10T08:59:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.047+0000 7f5ece791640 1 -- 172.21.15.172:0/2529755248 shutdown_connections 2024-09-10T08:59:29.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.047+0000 7f5ece791640 1 --2- 172.21.15.172:0/2529755248 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5eb00a2c30 0x7f5eb00a50f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.047+0000 7f5ece791640 1 --2- 172.21.15.172:0/2529755248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ec8102030 0x7f5ec819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.047+0000 7f5ece791640 1 --2- 172.21.15.172:0/2529755248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec8101690 0x7f5ec8198190 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.047+0000 7f5ece791640 1 --2- 172.21.15.172:0/2529755248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ec8100cd0 0x7f5ec8197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:29.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.047+0000 7f5ece791640 1 -- 172.21.15.172:0/2529755248 >> 172.21.15.172:0/2529755248 conn(0x7f5ec80fb970 msgr2=0x7f5ec80fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.067+0000 7f5ece791640 1 -- 172.21.15.172:0/2529755248 shutdown_connections 2024-09-10T08:59:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:29.067+0000 7f5ece791640 1 -- 172.21.15.172:0/2529755248 wait complete. 2024-09-10T08:59:29.359 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:59:29.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:29 smithi174 ceph-mon[34068]: pgmap v748: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 3.67k op/s 2024-09-10T08:59:29.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:29 smithi191 ceph-mon[35314]: pgmap v748: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 3.67k op/s 2024-09-10T08:59:29.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:29 smithi172 ceph-mon[30854]: pgmap v748: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 3.67k op/s 2024-09-10T08:59:30.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:30 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:59:30.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:30 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:59:31.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:30 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T08:59:31.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:31 smithi174 ceph-mon[34068]: pgmap v749: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 3.33k op/s 2024-09-10T08:59:32.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:31 smithi191 ceph-mon[35314]: pgmap v749: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 3.33k op/s 2024-09-10T08:59:32.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:31 smithi172 ceph-mon[30854]: pgmap v749: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 3.33k op/s 2024-09-10T08:59:33.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:33 smithi174 ceph-mon[34068]: pgmap v750: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 3.38k op/s 2024-09-10T08:59:33.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:33 smithi191 ceph-mon[35314]: pgmap v750: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 3.38k op/s 2024-09-10T08:59:33.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:33 smithi172 ceph-mon[30854]: pgmap v750: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 3.38k op/s 2024-09-10T08:59:35.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:35 smithi174 ceph-mon[34068]: pgmap v751: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 3.80k op/s 2024-09-10T08:59:35.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:35 smithi191 ceph-mon[35314]: pgmap v751: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 3.80k op/s 2024-09-10T08:59:35.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:35 smithi172 ceph-mon[30854]: pgmap v751: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 3.80k op/s 2024-09-10T08:59:36.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:36 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:59:36.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:36 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:59:36.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:36 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:59:36.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:36 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:59:36.743 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:36 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:59:36.768 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:36 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:59:36.769 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:36 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:59:36.769 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:36 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:59:36.769 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:36 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:59:36.769 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:36 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:59:36.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:36 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T08:59:36.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:36 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T08:59:36.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:36 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T08:59:36.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:36 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:59:36.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:36 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:59:36.827 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T08:59:37.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:37 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:59:37.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:37 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:59:37.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:37 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T08:59:37.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:37 smithi174 ceph-mon[34068]: pgmap v752: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 3.43k op/s 2024-09-10T08:59:37.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:37 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:59:37.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:37 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:59:37.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:37 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T08:59:37.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:37 smithi191 ceph-mon[35314]: pgmap v752: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 3.43k op/s 2024-09-10T08:59:37.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:37 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:59:37.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:37 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:59:37.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:37 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T08:59:37.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:37 smithi172 ceph-mon[30854]: pgmap v752: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 3.43k op/s 2024-09-10T08:59:39.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:39 smithi174 ceph-mon[34068]: pgmap v753: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-09-10T08:59:39.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:39 smithi191 ceph-mon[35314]: pgmap v753: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-09-10T08:59:39.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:39 smithi172 ceph-mon[30854]: pgmap v753: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-09-10T08:59:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:41 smithi174 ceph-mon[34068]: pgmap v754: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 3.40k op/s 2024-09-10T08:59:41.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:41 smithi191 ceph-mon[35314]: pgmap v754: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 3.40k op/s 2024-09-10T08:59:41.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:41 smithi172 ceph-mon[30854]: pgmap v754: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 3.40k op/s 2024-09-10T08:59:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.275+0000 7f60cd6af640 1 -- 172.21.15.172:0/469332044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60c8072cb0 msgr2=0x7f60c80730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.275+0000 7f60cd6af640 1 --2- 172.21.15.172:0/469332044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60c8072cb0 0x7f60c80730b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009930 tx=0x7f60bc02a480 comp rx=0 tx=0).stop 2024-09-10T08:59:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.275+0000 7f60cd6af640 1 -- 172.21.15.172:0/469332044 shutdown_connections 2024-09-10T08:59:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.275+0000 7f60cd6af640 1 --2- 172.21.15.172:0/469332044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60c810dbb0 0x7f60c8110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.275+0000 7f60cd6af640 1 --2- 172.21.15.172:0/469332044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60c8073680 0x7f60c810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.275+0000 7f60cd6af640 1 --2- 172.21.15.172:0/469332044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60c8072cb0 0x7f60c80730b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.275+0000 7f60cd6af640 1 -- 172.21.15.172:0/469332044 >> 172.21.15.172:0/469332044 conn(0x7f60c806eb50 msgr2=0x7f60c8070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.275+0000 7f60cd6af640 1 -- 172.21.15.172:0/469332044 shutdown_connections 2024-09-10T08:59:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.275+0000 7f60cd6af640 1 -- 172.21.15.172:0/469332044 wait complete. 2024-09-10T08:59:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.276+0000 7f60cd6af640 1 Processor -- start 2024-09-10T08:59:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.276+0000 7f60cd6af640 1 -- start start 2024-09-10T08:59:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.277+0000 7f60cd6af640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60c8073680 0x7f60c8115830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.277+0000 7f60cd6af640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60c810dbb0 0x7f60c8115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.277+0000 7f60cd6af640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60c81b49d0 0x7f60c81b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.277+0000 7f60cd6af640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f60c811c2d0 con 0x7f60c8073680 2024-09-10T08:59:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.277+0000 7f60cd6af640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60c811c150 con 0x7f60c810dbb0 2024-09-10T08:59:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.277+0000 7f60cd6af640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f60c811c450 con 0x7f60c81b49d0 2024-09-10T08:59:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.277+0000 7f60cceae640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60c81b49d0 0x7f60c81b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.277+0000 7f60cceae640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60c81b49d0 0x7f60c81b6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:44218/0 (socket says 172.21.15.172:44218) 2024-09-10T08:59:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.277+0000 7f60cceae640 1 -- 172.21.15.172:0/2822401746 learned_addr learned my addr 172.21.15.172:0/2822401746 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.277+0000 7f60c77fe640 1 --2- 172.21.15.172:0/2822401746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60c810dbb0 0x7f60c8115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.278+0000 7f60c77fe640 1 -- 172.21.15.172:0/2822401746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60c81b49d0 msgr2=0x7f60c81b6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.278+0000 7f60c77fe640 1 --2- 172.21.15.172:0/2822401746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60c81b49d0 0x7f60c81b6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.278+0000 7f60c77fe640 1 -- 172.21.15.172:0/2822401746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60c8073680 msgr2=0x7f60c8115830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.278+0000 7f60c77fe640 1 --2- 172.21.15.172:0/2822401746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60c8073680 0x7f60c8115830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.278+0000 7f60c77fe640 1 -- 172.21.15.172:0/2822401746 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c81b7300 con 0x7f60c810dbb0 2024-09-10T08:59:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.278+0000 7f60c77fe640 1 --2- 172.21.15.172:0/2822401746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60c810dbb0 0x7f60c8115d70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f60c0005fd0 tx=0x7f60c000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:42.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.281+0000 7f60c57fa640 1 -- 172.21.15.172:0/2822401746 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c000cd80 con 0x7f60c810dbb0 2024-09-10T08:59:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.281+0000 7f60cd6af640 1 -- 172.21.15.172:0/2822401746 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c81b74d0 con 0x7f60c810dbb0 2024-09-10T08:59:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.281+0000 7f60cd6af640 1 -- 172.21.15.172:0/2822401746 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c81b7990 con 0x7f60c810dbb0 2024-09-10T08:59:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.281+0000 7f60c57fa640 1 -- 172.21.15.172:0/2822401746 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60c0011920 con 0x7f60c810dbb0 2024-09-10T08:59:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.281+0000 7f60c57fa640 1 -- 172.21.15.172:0/2822401746 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c001a9d0 con 0x7f60c810dbb0 2024-09-10T08:59:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.286+0000 7f60d16a5740 1 Processor -- start 2024-09-10T08:59:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.288+0000 7f60d16a5740 1 -- start start 2024-09-10T08:59:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.288+0000 7f60d16a5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587012deb70 0x5587012def70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.288+0000 7f60d16a5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012df4b0 0x5587012e3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.288+0000 7f60d16a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012e4540 0x5587012e6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.288+0000 7f60d16a5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5587012422f0 con 0x5587012deb70 2024-09-10T08:59:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.288+0000 7f60d16a5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5587011ea270 con 0x5587012e4540 2024-09-10T08:59:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.288+0000 7f60d16a5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55870123e2b0 con 0x5587012df4b0 2024-09-10T08:59:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.288+0000 7f60c7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012df4b0 0x5587012e3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.288+0000 7f60c7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012df4b0 0x5587012e3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:44232/0 (socket says 172.21.15.172:44232) 2024-09-10T08:59:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.288+0000 7f60c7fff640 1 -- 172.21.15.172:0/3556758316 learned_addr learned my addr 172.21.15.172:0/3556758316 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.289+0000 7f60c7fff640 1 -- 172.21.15.172:0/3556758316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012e4540 msgr2=0x5587012e6a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T08:59:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.289+0000 7f60c7fff640 1 --2- 172.21.15.172:0/3556758316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012e4540 0x5587012e6a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.289+0000 7f60c7fff640 1 -- 172.21.15.172:0/3556758316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587012deb70 msgr2=0x5587012def70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.289+0000 7f60c7fff640 1 --2- 172.21.15.172:0/3556758316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587012deb70 0x5587012def70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.289+0000 7f60c7fff640 1 -- 172.21.15.172:0/3556758316 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587011e9340 con 0x5587012df4b0 2024-09-10T08:59:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.289+0000 7f60c57fa640 1 -- 172.21.15.172:0/2822401746 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f60c001ab70 con 0x7f60c810dbb0 2024-09-10T08:59:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.289+0000 7f60c7fff640 1 --2- 172.21.15.172:0/3556758316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012df4b0 0x5587012e3f70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f60bc033040 tx=0x7f60bc009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0b90bfb66a23dba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.291+0000 7f60c57fa640 1 --2- 172.21.15.172:0/2822401746 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f60b00a2d90 0x7f60b00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.291+0000 7f60c57fa640 1 -- 172.21.15.172:0/2822401746 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f60c00cfbc0 con 0x7f60c810dbb0 2024-09-10T08:59:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.291+0000 7f60ae7fc640 1 -- 172.21.15.172:0/3556758316 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc040070 con 0x5587012df4b0 2024-09-10T08:59:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.292+0000 7f60ae7fc640 1 -- 172.21.15.172:0/3556758316 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60bc0054f0 con 0x5587012df4b0 2024-09-10T08:59:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.292+0000 7f60cceae640 1 --2- 172.21.15.172:0/2822401746 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f60b00a2d90 0x7f60b00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.293+0000 7f60d16a5740 1 -- 172.21.15.172:0/3556758316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012df4b0 msgr2=0x5587012e3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.293+0000 7f60d16a5740 1 --2- 172.21.15.172:0/3556758316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012df4b0 0x5587012e3f70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f60bc033040 tx=0x7f60bc009400 comp rx=0 tx=0).stop 2024-09-10T08:59:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.293+0000 7f60d16a5740 1 -- 172.21.15.172:0/3556758316 shutdown_connections 2024-09-10T08:59:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.293+0000 7f60d16a5740 1 --2- 172.21.15.172:0/3556758316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012e4540 0x5587012e6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.293+0000 7f60d16a5740 1 --2- 172.21.15.172:0/3556758316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012df4b0 0x5587012e3f70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.293+0000 7f60d16a5740 1 --2- 172.21.15.172:0/3556758316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587012deb70 0x5587012def70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.293+0000 7f60d16a5740 1 -- 172.21.15.172:0/3556758316 >> 172.21.15.172:0/3556758316 conn(0x5587012da1e0 msgr2=0x5587012da5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.295+0000 7f60cceae640 1 --2- 172.21.15.172:0/2822401746 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f60b00a2d90 0x7f60b00a5250 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f60b800bcf0 tx=0x7f60b800b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.297+0000 7f60d16a5740 1 -- 172.21.15.172:0/3556758316 shutdown_connections 2024-09-10T08:59:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.297+0000 7f60d16a5740 1 -- 172.21.15.172:0/3556758316 wait complete. 2024-09-10T08:59:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.297+0000 7f60d16a5740 1 -- start start 2024-09-10T08:59:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.298+0000 7f60d16a5740 1 Processor -- start 2024-09-10T08:59:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.298+0000 7f60d16a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012deb70 0x5587012e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.298+0000 7f60d16a5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012df4b0 0x5587012e9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.298+0000 7f60d16a5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587012e4540 0x5587012f0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.298+0000 7f60d16a5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5587012e7500 con 0x5587012e4540 2024-09-10T08:59:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.298+0000 7f60d16a5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5587011cb290 con 0x5587012deb70 2024-09-10T08:59:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.298+0000 7f60d16a5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5587012e7680 con 0x5587012df4b0 2024-09-10T08:59:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.298+0000 7f60d16a5740 10 client.0 _ll_register_callbacks cb 0x5587011fc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:59:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.298+0000 7f60c77fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012df4b0 0x5587012e9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.298+0000 7f60c77fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012df4b0 0x5587012e9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:44246/0 (socket says 172.21.15.172:44246) 2024-09-10T08:59:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.299+0000 7f60c77fe640 1 -- 172.21.15.172:0/3508566082 learned_addr learned my addr 172.21.15.172:0/3508566082 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.299+0000 7f60c7fff640 1 --2- 172.21.15.172:0/3508566082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012deb70 0x5587012e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.299+0000 7f60c77fe640 1 -- 172.21.15.172:0/3508566082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012deb70 msgr2=0x5587012e8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.299+0000 7f60c77fe640 1 --2- 172.21.15.172:0/3508566082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012deb70 0x5587012e8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.299+0000 7f60c77fe640 1 -- 172.21.15.172:0/3508566082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587012e4540 msgr2=0x5587012f0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.299+0000 7f60c77fe640 1 --2- 172.21.15.172:0/3508566082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587012e4540 0x5587012f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.299+0000 7f60c77fe640 1 -- 172.21.15.172:0/3508566082 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558700f55210 con 0x5587012df4b0 2024-09-10T08:59:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.299+0000 7f60c77fe640 1 --2- 172.21.15.172:0/3508566082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012df4b0 0x5587012e9340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f60c00a47a0 tx=0x7f60c009bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ea01f17aa346122 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.300+0000 7f60a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:59:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.300+0000 7f60a77fe640 1 -- 172.21.15.172:0/3508566082 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c00a0410 con 0x5587012df4b0 2024-09-10T08:59:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.300+0000 7f60d16a5740 1 -- 172.21.15.172:0/3508566082 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60bc0054f0 con 0x5587012df4b0 2024-09-10T08:59:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.300+0000 7f60a77fe640 1 -- 172.21.15.172:0/3508566082 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f60c00a00d0 con 0x5587012df4b0 2024-09-10T08:59:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.300+0000 7f60a77fe640 1 -- 172.21.15.172:0/3508566082 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f60c00a4cd0 con 0x5587012df4b0 2024-09-10T08:59:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.305+0000 7f60d16a5740 10 client.28039 fetch_fsmap learned FSMap version 41 2024-09-10T08:59:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.305+0000 7f60d16a5740 1 -- 172.21.15.172:0/3508566082 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558701021620 con 0x5587012df4b0 2024-09-10T08:59:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.305+0000 7f60a77fe640 1 -- 172.21.15.172:0/3508566082 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60bc0054f0 con 0x5587012df4b0 2024-09-10T08:59:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.306+0000 7f60a77fe640 1 -- 172.21.15.172:0/3508566082 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f60c0022030 con 0x5587012df4b0 2024-09-10T08:59:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.307+0000 7f60d16a5740 10 client.28039 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T08:59:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.307+0000 7f60d16a5740 10 client.28039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:59:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.307+0000 7f60d16a5740 20 client.28039 populate_metadata read hostname 'smithi172' 2024-09-10T08:59:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.307+0000 7f60d16a5740 1 --2- 172.21.15.172:0/3508566082 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5587012f4ed0 0x5587012f7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.307+0000 7f60d16a5740 4 client.28039 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:59:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.307+0000 7f60d16a5740 1 -- 172.21.15.172:0/3508566082 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55870124fa20 con 0x5587012f4ed0 2024-09-10T08:59:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.307+0000 7f60c7fff640 1 --2- 172.21.15.172:0/3508566082 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5587012f4ed0 0x5587012f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.308+0000 7f60c7fff640 1 --2- 172.21.15.172:0/3508566082 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5587012f4ed0 0x5587012f7310 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37db162a41ace461 server_cookie=d48ef9381adf1cb5 in_seq=0 out_seq=0 2024-09-10T08:59:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.308+0000 7f60a77fe640 10 client.28039 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.311+0000 7f60a77fe640 1 -- 172.21.15.172:0/3508566082 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55870124fa20 con 0x5587012f4ed0 2024-09-10T08:59:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.311+0000 7f60a77fe640 10 client.28039 handle_command_reply: tid=0 2024-09-10T08:59:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.311+0000 7f60d16a5740 1 client.28039 shutdown 2024-09-10T08:59:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.311+0000 7f60d16a5740 1 -- 172.21.15.172:0/3508566082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012df4b0 msgr2=0x5587012e9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.311+0000 7f60d16a5740 1 --2- 172.21.15.172:0/3508566082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012df4b0 0x5587012e9340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f60c00a47a0 tx=0x7f60c009bab0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.311+0000 7f60d16a5740 1 -- 172.21.15.172:0/3508566082 shutdown_connections 2024-09-10T08:59:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.311+0000 7f60d16a5740 1 --2- 172.21.15.172:0/3508566082 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5587012f4ed0 0x5587012f7310 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.311+0000 7f60d16a5740 1 --2- 172.21.15.172:0/3508566082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587012e4540 0x5587012f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.311+0000 7f60d16a5740 1 --2- 172.21.15.172:0/3508566082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012df4b0 0x5587012e9340 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.311+0000 7f60d16a5740 1 --2- 172.21.15.172:0/3508566082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012deb70 0x5587012e8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.311+0000 7f60d16a5740 1 -- 172.21.15.172:0/3508566082 >> 172.21.15.172:0/3508566082 conn(0x5587012da1e0 msgr2=0x5587012db9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.312+0000 7f60a77fe640 0 client.28039 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.312+0000 7f60d16a5740 1 -- 172.21.15.172:0/3508566082 shutdown_connections 2024-09-10T08:59:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.312+0000 7f60d16a5740 1 -- 172.21.15.172:0/3508566082 wait complete. 2024-09-10T08:59:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.316+0000 7f60d16a5740 20 client.28039 trim_cache size 0 max 16384 2024-09-10T08:59:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.372+0000 7f60d16a5740 1 Processor -- start 2024-09-10T08:59:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.372+0000 7f60d16a5740 1 -- start start 2024-09-10T08:59:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.372+0000 7f60d16a5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012deb70 0x5587012e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.373+0000 7f60d16a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012df4b0 0x5587012e9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.373+0000 7f60d16a5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587012e4540 0x5587012f0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.373+0000 7f60c7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012deb70 0x5587012e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.373+0000 7f60c7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012deb70 0x5587012e8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:44254/0 (socket says 172.21.15.172:44254) 2024-09-10T08:59:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.373+0000 7f60c77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012df4b0 0x5587012e9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:42.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.373+0000 7f60c77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012df4b0 0x5587012e9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:53826/0 (socket says 172.21.15.172:53826) 2024-09-10T08:59:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.373+0000 7f60c7fff640 1 -- 172.21.15.172:0/1263955101 learned_addr learned my addr 172.21.15.172:0/1263955101 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.373+0000 7f60d16a5740 1 -- 172.21.15.172:0/1263955101 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f60b400fbe0 con 0x5587012e4540 2024-09-10T08:59:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.373+0000 7f60d16a5740 1 -- 172.21.15.172:0/1263955101 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60b40117e0 con 0x5587012df4b0 2024-09-10T08:59:42.376 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.373+0000 7f60d16a5740 1 -- 172.21.15.172:0/1263955101 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f60b40102b0 con 0x5587012deb70 2024-09-10T08:59:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.376+0000 7f60cceae640 1 --2- 172.21.15.172:0/1263955101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587012e4540 0x5587012f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.376+0000 7f60c7fff640 1 -- 172.21.15.172:0/1263955101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012df4b0 msgr2=0x5587012e9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.376+0000 7f60c7fff640 1 --2- 172.21.15.172:0/1263955101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012df4b0 0x5587012e9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.376+0000 7f60c7fff640 1 -- 172.21.15.172:0/1263955101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587012e4540 msgr2=0x5587012f0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.376+0000 7f60c7fff640 1 --2- 172.21.15.172:0/1263955101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587012e4540 0x5587012f0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.377 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.376+0000 7f60c7fff640 1 -- 172.21.15.172:0/1263955101 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587012da000 con 0x5587012deb70 2024-09-10T08:59:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.377+0000 7f60c7fff640 1 --2- 172.21.15.172:0/1263955101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012deb70 0x5587012e8e00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0057c0 tx=0x7f60bc005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6fc2bdaef588731 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.378+0000 7f60adffb640 1 -- 172.21.15.172:0/1263955101 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc040040 con 0x5587012deb70 2024-09-10T08:59:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.378+0000 7f60adffb640 1 -- 172.21.15.172:0/1263955101 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60bc077070 con 0x5587012deb70 2024-09-10T08:59:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.380+0000 7f60d16a5740 1 -- 172.21.15.172:0/1263955101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012deb70 msgr2=0x5587012e8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.381+0000 7f60d16a5740 1 --2- 172.21.15.172:0/1263955101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012deb70 0x5587012e8e00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0057c0 tx=0x7f60bc005b50 comp rx=0 tx=0).stop 2024-09-10T08:59:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.381+0000 7f60d16a5740 1 -- 172.21.15.172:0/1263955101 shutdown_connections 2024-09-10T08:59:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.381+0000 7f60d16a5740 1 --2- 172.21.15.172:0/1263955101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587012e4540 0x5587012f0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.381+0000 7f60d16a5740 1 --2- 172.21.15.172:0/1263955101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012df4b0 0x5587012e9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.381+0000 7f60d16a5740 1 --2- 172.21.15.172:0/1263955101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012deb70 0x5587012e8e00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.381+0000 7f60d16a5740 1 -- 172.21.15.172:0/1263955101 >> 172.21.15.172:0/1263955101 conn(0x5587012da1e0 msgr2=0x5587012e3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.392+0000 7f60d16a5740 1 -- 172.21.15.172:0/1263955101 shutdown_connections 2024-09-10T08:59:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.392+0000 7f60d16a5740 1 -- 172.21.15.172:0/1263955101 wait complete. 2024-09-10T08:59:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.392+0000 7f60d16a5740 1 -- start start 2024-09-10T08:59:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.393+0000 7f60d16a5740 1 Processor -- start 2024-09-10T08:59:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.393+0000 7f60d16a5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012deb70 0x5587012f0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.393+0000 7f60d16a5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587012df4b0 0x5587012ecde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.394+0000 7f60c7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012deb70 0x5587012f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.394+0000 7f60c7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012deb70 0x5587012f0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:53828/0 (socket says 172.21.15.172:53828) 2024-09-10T08:59:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.394+0000 7f60c7fff640 1 -- 172.21.15.172:0/378356356 learned_addr learned my addr 172.21.15.172:0/378356356 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.394+0000 7f60d16a5740 1 --2- 172.21.15.172:0/378356356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012e4540 0x5587012ed340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.394+0000 7f60d16a5740 1 -- 172.21.15.172:0/378356356 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f60b4011110 con 0x5587012df4b0 2024-09-10T08:59:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.394+0000 7f60d16a5740 1 -- 172.21.15.172:0/378356356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60b4010a40 con 0x5587012deb70 2024-09-10T08:59:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.394+0000 7f60cceae640 1 --2- 172.21.15.172:0/378356356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012e4540 0x5587012ed340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.394+0000 7f60d16a5740 1 -- 172.21.15.172:0/378356356 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f60b400dbc0 con 0x5587012e4540 2024-09-10T08:59:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.395+0000 7f60d16a5740 10 client.0 _ll_register_callbacks cb 0x7f60c000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:59:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.395+0000 7f60c7fff640 1 -- 172.21.15.172:0/378356356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012e4540 msgr2=0x5587012ed340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.395+0000 7f60c7fff640 1 --2- 172.21.15.172:0/378356356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012e4540 0x5587012ed340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.395+0000 7f60c7fff640 1 -- 172.21.15.172:0/378356356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587012df4b0 msgr2=0x5587012ecde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.395+0000 7f60c7fff640 1 --2- 172.21.15.172:0/378356356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587012df4b0 0x5587012ecde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.395+0000 7f60c7fff640 1 -- 172.21.15.172:0/378356356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55870135dbe0 con 0x5587012deb70 2024-09-10T08:59:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.395+0000 7f60c7fff640 1 --2- 172.21.15.172:0/378356356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012deb70 0x5587012f0340 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009850 tx=0x7f60bc031810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5a922602f7e6afd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:42.397 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.395+0000 7f60d16a5740 1 -- 172.21.15.172:0/378356356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60bc077070 con 0x5587012deb70 2024-09-10T08:59:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.397+0000 7f60a77fe640 10 client.28100 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T08:59:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.397+0000 7f60a77fe640 1 -- 172.21.15.172:0/378356356 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc080020 con 0x5587012deb70 2024-09-10T08:59:42.398 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.397+0000 7f60a77fe640 1 -- 172.21.15.172:0/378356356 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f60bc004f20 con 0x5587012deb70 2024-09-10T08:59:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.398+0000 7f60a77fe640 1 -- 172.21.15.172:0/378356356 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f60bc08b050 con 0x5587012deb70 2024-09-10T08:59:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.398+0000 7f60d16a5740 10 client.28100 fetch_fsmap learned FSMap version 41 2024-09-10T08:59:42.399 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.398+0000 7f60d16a5740 1 -- 172.21.15.172:0/378356356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=41}) -- 0x55870135ede0 con 0x5587012deb70 2024-09-10T08:59:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.399+0000 7f60a77fe640 1 -- 172.21.15.172:0/378356356 <== mon.2 v2:172.21.15.174:3300/0 4 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f60bc005120 con 0x5587012deb70 2024-09-10T08:59:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.399+0000 7f60d16a5740 10 client.28100 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T08:59:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.399+0000 7f60d16a5740 10 client.28100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:59:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.399+0000 7f60d16a5740 20 client.28100 populate_metadata read hostname 'smithi172' 2024-09-10T08:59:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.399+0000 7f60d16a5740 1 --2- 172.21.15.172:0/378356356 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55870135f730 0x558701361b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.399+0000 7f60d16a5740 4 client.28100 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:59:42.400 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.399+0000 7f60d16a5740 1 -- 172.21.15.172:0/378356356 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f60bc077070 con 0x55870135f730 2024-09-10T08:59:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.400+0000 7f60c77fe640 1 --2- 172.21.15.172:0/378356356 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55870135f730 0x558701361b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.400+0000 7f60c77fe640 1 --2- 172.21.15.172:0/378356356 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55870135f730 0x558701361b90 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5571f52c8368029b server_cookie=27dd7f13de0e6253 in_seq=0 out_seq=0 2024-09-10T08:59:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:42.400+0000 7f60a77fe640 10 client.28100 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:42.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:42 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2042890454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:59:42.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:42 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2042890454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:59:42.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:42 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2042890454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T08:59:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.221+0000 7f60a77fe640 1 -- 172.21.15.172:0/378356356 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f60bc077070 con 0x55870135f730 2024-09-10T08:59:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.221+0000 7f60a77fe640 10 client.28100 handle_command_reply: tid=0 2024-09-10T08:59:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.221+0000 7f60d16a5740 1 client.28100 shutdown 2024-09-10T08:59:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.222+0000 7f60d16a5740 1 -- 172.21.15.172:0/378356356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012deb70 msgr2=0x5587012f0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.222+0000 7f60d16a5740 1 --2- 172.21.15.172:0/378356356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012deb70 0x5587012f0340 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009850 tx=0x7f60bc031810 comp rx=0 tx=0).stop 2024-09-10T08:59:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.222+0000 7f60d16a5740 1 -- 172.21.15.172:0/378356356 shutdown_connections 2024-09-10T08:59:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.222+0000 7f60d16a5740 1 --2- 172.21.15.172:0/378356356 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55870135f730 0x558701361b90 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.222+0000 7f60d16a5740 1 --2- 172.21.15.172:0/378356356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5587012e4540 0x5587012ed340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.222+0000 7f60d16a5740 1 --2- 172.21.15.172:0/378356356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5587012df4b0 0x5587012ecde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.222+0000 7f60d16a5740 1 --2- 172.21.15.172:0/378356356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5587012deb70 0x5587012f0340 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.222+0000 7f60d16a5740 1 -- 172.21.15.172:0/378356356 >> 172.21.15.172:0/378356356 conn(0x5587012da1e0 msgr2=0x5587012e0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.222+0000 7f60a77fe640 0 client.28100 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.222+0000 7f60d16a5740 1 -- 172.21.15.172:0/378356356 shutdown_connections 2024-09-10T08:59:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.222+0000 7f60d16a5740 1 -- 172.21.15.172:0/378356356 wait complete. 2024-09-10T08:59:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.223+0000 7f60d16a5740 20 client.28100 trim_cache size 0 max 16384 2024-09-10T08:59:43.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:59:43.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:59:43.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:59:43.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:59:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.224+0000 7f60cd6af640 1 -- 172.21.15.172:0/2822401746 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f60b00a2d90 msgr2=0x7f60b00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.224+0000 7f60cd6af640 1 --2- 172.21.15.172:0/2822401746 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f60b00a2d90 0x7f60b00a5250 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f60b800bcf0 tx=0x7f60b800b7b0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.224+0000 7f60cd6af640 1 -- 172.21.15.172:0/2822401746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60c810dbb0 msgr2=0x7f60c8115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.224+0000 7f60cd6af640 1 --2- 172.21.15.172:0/2822401746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60c810dbb0 0x7f60c8115d70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f60c0005fd0 tx=0x7f60c000ee80 comp rx=0 tx=0).stop 2024-09-10T08:59:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.224+0000 7f60cd6af640 1 -- 172.21.15.172:0/2822401746 shutdown_connections 2024-09-10T08:59:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.224+0000 7f60cd6af640 1 --2- 172.21.15.172:0/2822401746 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f60b00a2d90 0x7f60b00a5250 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.224+0000 7f60cd6af640 1 --2- 172.21.15.172:0/2822401746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60c81b49d0 0x7f60c81b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.226+0000 7f60cd6af640 1 --2- 172.21.15.172:0/2822401746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60c810dbb0 0x7f60c8115d70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.226+0000 7f60cd6af640 1 --2- 172.21.15.172:0/2822401746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60c8073680 0x7f60c8115830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.226+0000 7f60cd6af640 1 -- 172.21.15.172:0/2822401746 >> 172.21.15.172:0/2822401746 conn(0x7f60c806eb50 msgr2=0x7f60c806f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.239+0000 7f60cd6af640 1 -- 172.21.15.172:0/2822401746 shutdown_connections 2024-09-10T08:59:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:43.242+0000 7f60cd6af640 1 -- 172.21.15.172:0/2822401746 wait complete. 2024-09-10T08:59:43.342 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T08:59:43.343 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":41,"btime":"2024-09-10T08:59:08:444666+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T08:59:07.669792+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T08:59:43.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.085+0000 7f8621650640 1 -- 172.21.15.172:0/2165177795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f861c074510 msgr2=0x7f861c0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:43.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.085+0000 7f8621650640 1 --2- 172.21.15.172:0/2165177795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f861c074510 0x7f861c0748f0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f860c009900 tx=0x7f860c02a480 comp rx=0 tx=0).stop 2024-09-10T08:59:43.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.085+0000 7f8621650640 1 -- 172.21.15.172:0/2165177795 shutdown_connections 2024-09-10T08:59:43.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.085+0000 7f8621650640 1 --2- 172.21.15.172:0/2165177795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f861c10e440 0x7f861c110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.085+0000 7f8621650640 1 --2- 172.21.15.172:0/2165177795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f861c074e30 0x7f861c10df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.085+0000 7f8621650640 1 --2- 172.21.15.172:0/2165177795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f861c074510 0x7f861c0748f0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.085+0000 7f8621650640 1 -- 172.21.15.172:0/2165177795 >> 172.21.15.172:0/2165177795 conn(0x7f861c0fd8c0 msgr2=0x7f861c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:43.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.085+0000 7f8621650640 1 -- 172.21.15.172:0/2165177795 shutdown_connections 2024-09-10T08:59:43.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.085+0000 7f8621650640 1 -- 172.21.15.172:0/2165177795 wait complete. 2024-09-10T08:59:43.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.085+0000 7f8621650640 1 Processor -- start 2024-09-10T08:59:43.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.085+0000 7f8621650640 1 -- start start 2024-09-10T08:59:43.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f8621650640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f861c074510 0x7f861c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:43.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f8621650640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f861c074e30 0x7f861c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:43.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f8621650640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f861c10e440 0x7f861c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:43.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f8621650640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f861c114000 con 0x7f861c074e30 2024-09-10T08:59:43.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f8621650640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f861c113e80 con 0x7f861c074510 2024-09-10T08:59:43.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f8621650640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f861c114180 con 0x7f861c10e440 2024-09-10T08:59:43.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f86127fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f861c074e30 0x7f861c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:43.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f86127fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f861c074e30 0x7f861c19aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60608/0 (socket says 172.21.15.172:60608) 2024-09-10T08:59:43.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f86127fc640 1 -- 172.21.15.172:0/2042890454 learned_addr learned my addr 172.21.15.172:0/2042890454 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f861affd640 1 --2- 172.21.15.172:0/2042890454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f861c074510 0x7f861c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f861b7fe640 1 --2- 172.21.15.172:0/2042890454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f861c10e440 0x7f861c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f86127fc640 1 -- 172.21.15.172:0/2042890454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f861c10e440 msgr2=0x7f861c19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f86127fc640 1 --2- 172.21.15.172:0/2042890454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f861c10e440 0x7f861c19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f86127fc640 1 -- 172.21.15.172:0/2042890454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f861c074510 msgr2=0x7f861c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f86127fc640 1 --2- 172.21.15.172:0/2042890454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f861c074510 0x7f861c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f86127fc640 1 -- 172.21.15.172:0/2042890454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c19f540 con 0x7f861c074e30 2024-09-10T08:59:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f86127fc640 1 --2- 172.21.15.172:0/2042890454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f861c074e30 0x7f861c19aad0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f860400d950 tx=0x7f860400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f8618ff9640 1 -- 172.21.15.172:0/2042890454 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604017030 con 0x7f861c074e30 2024-09-10T08:59:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.086+0000 7f8621650640 1 -- 172.21.15.172:0/2042890454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861c19f830 con 0x7f861c074e30 2024-09-10T08:59:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.087+0000 7f8618ff9640 1 -- 172.21.15.172:0/2042890454 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8604004020 con 0x7f861c074e30 2024-09-10T08:59:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.087+0000 7f8618ff9640 1 -- 172.21.15.172:0/2042890454 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86040047a0 con 0x7f861c074e30 2024-09-10T08:59:43.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.087+0000 7f8621650640 1 -- 172.21.15.172:0/2042890454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f861c1a7110 con 0x7f861c074e30 2024-09-10T08:59:43.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.088+0000 7f8621650640 1 -- 172.21.15.172:0/2042890454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f861c0748f0 con 0x7f861c074e30 2024-09-10T08:59:43.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.089+0000 7f8618ff9640 1 -- 172.21.15.172:0/2042890454 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f86040041d0 con 0x7f861c074e30 2024-09-10T08:59:43.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.089+0000 7f8618ff9640 1 --2- 172.21.15.172:0/2042890454 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86080a2b80 0x7f86080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:43.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.089+0000 7f861affd640 1 --2- 172.21.15.172:0/2042890454 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86080a2b80 0x7f86080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:43.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.089+0000 7f8618ff9640 1 -- 172.21.15.172:0/2042890454 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f86040ceee0 con 0x7f861c074e30 2024-09-10T08:59:43.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.090+0000 7f861affd640 1 --2- 172.21.15.172:0/2042890454 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86080a2b80 0x7f86080a5040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f860c031440 tx=0x7f860c0095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:43.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.093+0000 7f8618ff9640 1 -- 172.21.15.172:0/2042890454 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f860409bae0 con 0x7f861c074e30 2024-09-10T08:59:43.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.486+0000 7f8621650640 1 -- 172.21.15.172:0/2042890454 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f861c109620 con 0x7f861c074e30 2024-09-10T08:59:43.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.487+0000 7f8618ff9640 1 -- 172.21.15.172:0/2042890454 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+12008 (secure 0 0 0) 0x7f860409b3e0 con 0x7f861c074e30 2024-09-10T08:59:43.347 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 41 2024-09-10T08:59:43.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.490+0000 7f8621650640 1 -- 172.21.15.172:0/2042890454 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86080a2b80 msgr2=0x7f86080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:43.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.490+0000 7f8621650640 1 --2- 172.21.15.172:0/2042890454 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86080a2b80 0x7f86080a5040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f860c031440 tx=0x7f860c0095f0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.490+0000 7f8621650640 1 -- 172.21.15.172:0/2042890454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f861c074e30 msgr2=0x7f861c19aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:43.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.490+0000 7f8621650640 1 --2- 172.21.15.172:0/2042890454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f861c074e30 0x7f861c19aad0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f860400d950 tx=0x7f860400de10 comp rx=0 tx=0).stop 2024-09-10T08:59:43.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.490+0000 7f8621650640 1 -- 172.21.15.172:0/2042890454 shutdown_connections 2024-09-10T08:59:43.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.490+0000 7f8621650640 1 --2- 172.21.15.172:0/2042890454 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86080a2b80 0x7f86080a5040 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.490+0000 7f8621650640 1 --2- 172.21.15.172:0/2042890454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f861c10e440 0x7f861c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.490+0000 7f8621650640 1 --2- 172.21.15.172:0/2042890454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f861c074e30 0x7f861c19aad0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.490+0000 7f8621650640 1 --2- 172.21.15.172:0/2042890454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f861c074510 0x7f861c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:43.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.490+0000 7f8621650640 1 -- 172.21.15.172:0/2042890454 >> 172.21.15.172:0/2042890454 conn(0x7f861c0fd8c0 msgr2=0x7f861c0ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:43.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.490+0000 7f8621650640 1 -- 172.21.15.172:0/2042890454 shutdown_connections 2024-09-10T08:59:43.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:42.490+0000 7f8621650640 1 -- 172.21.15.172:0/2042890454 wait complete. 2024-09-10T08:59:43.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:43 smithi191 ceph-mon[35314]: pgmap v755: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 3.46k op/s 2024-09-10T08:59:43.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:43 smithi172 ceph-mon[30854]: pgmap v755: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 3.46k op/s 2024-09-10T08:59:43.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:43 smithi174 ceph-mon[34068]: pgmap v755: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 3.46k op/s 2024-09-10T08:59:45.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:45 smithi174 ceph-mon[34068]: pgmap v756: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.78k op/s 2024-09-10T08:59:45.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:45 smithi191 ceph-mon[35314]: pgmap v756: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.78k op/s 2024-09-10T08:59:45.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:45 smithi172 ceph-mon[30854]: pgmap v756: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.78k op/s 2024-09-10T08:59:46.567 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T08:59:47.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:47 smithi174 ceph-mon[34068]: pgmap v757: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 3.17k op/s 2024-09-10T08:59:47.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:47 smithi191 ceph-mon[35314]: pgmap v757: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 3.17k op/s 2024-09-10T08:59:47.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:47 smithi172 ceph-mon[30854]: pgmap v757: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 3.17k op/s 2024-09-10T08:59:49.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:49 smithi174 ceph-mon[34068]: pgmap v758: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.41k op/s 2024-09-10T08:59:49.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:49 smithi191 ceph-mon[35314]: pgmap v758: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.41k op/s 2024-09-10T08:59:49.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:49 smithi172 ceph-mon[30854]: pgmap v758: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.41k op/s 2024-09-10T08:59:51.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:51 smithi174 ceph-mon[34068]: pgmap v759: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 3.15k op/s 2024-09-10T08:59:51.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:51 smithi191 ceph-mon[35314]: pgmap v759: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 3.15k op/s 2024-09-10T08:59:51.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:51 smithi172 ceph-mon[30854]: pgmap v759: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 3.15k op/s 2024-09-10T08:59:53.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:53 smithi174 ceph-mon[34068]: pgmap v760: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.20k op/s 2024-09-10T08:59:53.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:53 smithi191 ceph-mon[35314]: pgmap v760: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.20k op/s 2024-09-10T08:59:53.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:53 smithi172 ceph-mon[30854]: pgmap v760: 129 pgs: 129 active+clean; 13 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.20k op/s 2024-09-10T08:59:54.469 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 141.5 --await-for 45.0' 2024-09-10T08:59:54.470 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 141.5 --await-for 45.0 2024-09-10T08:59:55.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:55 smithi174 ceph-mon[34068]: pgmap v761: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.74k op/s 2024-09-10T08:59:55.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:55 smithi191 ceph-mon[35314]: pgmap v761: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.74k op/s 2024-09-10T08:59:55.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:55 smithi172 ceph-mon[30854]: pgmap v761: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.74k op/s 2024-09-10T08:59:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.414+0000 7fd1e3a4c640 1 -- 172.21.15.172:0/2964796923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1dc101b60 msgr2=0x7fd1dc101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.414+0000 7fd1e3a4c640 1 --2- 172.21.15.172:0/2964796923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1dc101b60 0x7fd1dc101f60 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009860 tx=0x7fd1d802a340 comp rx=0 tx=0).stop 2024-09-10T08:59:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.415+0000 7fd1e3a4c640 1 -- 172.21.15.172:0/2964796923 shutdown_connections 2024-09-10T08:59:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.415+0000 7fd1e3a4c640 1 --2- 172.21.15.172:0/2964796923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1dc103710 0x7fd1dc109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.415+0000 7fd1e3a4c640 1 --2- 172.21.15.172:0/2964796923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1dc102d50 0x7fd1dc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.415+0000 7fd1e3a4c640 1 --2- 172.21.15.172:0/2964796923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1dc101b60 0x7fd1dc101f60 secure :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009860 tx=0x7fd1d802a340 comp rx=0 tx=0).stop 2024-09-10T08:59:56.417 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.415+0000 7fd1e3a4c640 1 -- 172.21.15.172:0/2964796923 >> 172.21.15.172:0/2964796923 conn(0x7fd1dc0fd2e0 msgr2=0x7fd1dc0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.422+0000 7fd1e3a4c640 1 -- 172.21.15.172:0/2964796923 shutdown_connections 2024-09-10T08:59:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.423+0000 7fd1e3a4c640 1 -- 172.21.15.172:0/2964796923 wait complete. 2024-09-10T08:59:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.424+0000 7fd1e3a4c640 1 Processor -- start 2024-09-10T08:59:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.424+0000 7fd1e3a4c640 1 -- start start 2024-09-10T08:59:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.425+0000 7fd1e3a4c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1dc102d50 0x7fd1dc195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.425+0000 7fd1e3a4c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1dc103710 0x7fd1dc1962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.425+0000 7fd1e3a4c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1dc1ac5c0 0x7fd1dc1ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.425+0000 7fd1e3a4c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc10d6c0 con 0x7fd1dc103710 2024-09-10T08:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.425+0000 7fd1e1fe9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1dc1ac5c0 0x7fd1dc1ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.425+0000 7fd1e1fe9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1dc1ac5c0 0x7fd1dc1ae9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:36478/0 (socket says 172.21.15.172:36478) 2024-09-10T08:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.426+0000 7fd1e1fe9640 1 -- 172.21.15.172:0/2007037303 learned_addr learned my addr 172.21.15.172:0/2007037303 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.426+0000 7fd1e3a4c640 1 -- 172.21.15.172:0/2007037303 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc10d540 con 0x7fd1dc1ac5c0 2024-09-10T08:59:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.426+0000 7fd1e3a4c640 1 -- 172.21.15.172:0/2007037303 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc10d840 con 0x7fd1dc102d50 2024-09-10T08:59:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.426+0000 7fd1e17e8640 1 --2- 172.21.15.172:0/2007037303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1dc102d50 0x7fd1dc195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.427+0000 7fd1e0fe7640 1 --2- 172.21.15.172:0/2007037303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1dc103710 0x7fd1dc1962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.427+0000 7fd1e17e8640 1 -- 172.21.15.172:0/2007037303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1dc1ac5c0 msgr2=0x7fd1dc1ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.427+0000 7fd1e17e8640 1 --2- 172.21.15.172:0/2007037303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1dc1ac5c0 0x7fd1dc1ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.427+0000 7fd1e17e8640 1 -- 172.21.15.172:0/2007037303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1dc103710 msgr2=0x7fd1dc1962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.427+0000 7fd1e17e8640 1 --2- 172.21.15.172:0/2007037303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1dc103710 0x7fd1dc1962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.427+0000 7fd1e17e8640 1 -- 172.21.15.172:0/2007037303 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1dc1aeef0 con 0x7fd1dc102d50 2024-09-10T08:59:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.428+0000 7fd1e17e8640 1 --2- 172.21.15.172:0/2007037303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1dc102d50 0x7fd1dc195d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc102810 tx=0x7fd1d80042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.433+0000 7fd1d27fc640 1 -- 172.21.15.172:0/2007037303 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8040020 con 0x7fd1dc102d50 2024-09-10T08:59:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.433+0000 7fd1d27fc640 1 -- 172.21.15.172:0/2007037303 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1d8038040 con 0x7fd1dc102d50 2024-09-10T08:59:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.433+0000 7fd1d27fc640 1 -- 172.21.15.172:0/2007037303 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8037070 con 0x7fd1dc102d50 2024-09-10T08:59:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.433+0000 7fd1e3a4c640 1 -- 172.21.15.172:0/2007037303 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1dc1af0c0 con 0x7fd1dc102d50 2024-09-10T08:59:56.435 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.433+0000 7fd1e3a4c640 1 -- 172.21.15.172:0/2007037303 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1dc1af4d0 con 0x7fd1dc102d50 2024-09-10T08:59:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.440+0000 7fd1d27fc640 1 -- 172.21.15.172:0/2007037303 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd1d8043070 con 0x7fd1dc102d50 2024-09-10T08:59:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.442+0000 7fd1e67e6740 1 Processor -- start 2024-09-10T08:59:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.442+0000 7fd1e67e6740 1 -- start start 2024-09-10T08:59:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.442+0000 7fd1e67e6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5588307349a0 0x558830734da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.442+0000 7fd1e67e6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588307352e0 0x558830739da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.442+0000 7fd1e67e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55883073a370 0x55883073c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.442+0000 7fd1e67e6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5588306982f0 con 0x5588307352e0 2024-09-10T08:59:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.442+0000 7fd1e67e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558830640270 con 0x55883073a370 2024-09-10T08:59:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.442+0000 7fd1e67e6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5588306942b0 con 0x5588307349a0 2024-09-10T08:59:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.443+0000 7fd1e1fe9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5588307349a0 0x558830734da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.443+0000 7fd1e1fe9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5588307349a0 0x558830734da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:56820/0 (socket says 172.21.15.172:56820) 2024-09-10T08:59:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.443+0000 7fd1e1fe9640 1 -- 172.21.15.172:0/2011543105 learned_addr learned my addr 172.21.15.172:0/2011543105 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.443+0000 7fd1e1fe9640 1 -- 172.21.15.172:0/2011543105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55883073a370 msgr2=0x55883073c830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T08:59:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.443+0000 7fd1e1fe9640 1 --2- 172.21.15.172:0/2011543105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55883073a370 0x55883073c830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.443+0000 7fd1e1fe9640 1 -- 172.21.15.172:0/2011543105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588307352e0 msgr2=0x558830739da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.443+0000 7fd1e1fe9640 1 --2- 172.21.15.172:0/2011543105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588307352e0 0x558830739da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.443+0000 7fd1e1fe9640 1 -- 172.21.15.172:0/2011543105 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55883063f340 con 0x5588307349a0 2024-09-10T08:59:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.443+0000 7fd1e1fe9640 1 --2- 172.21.15.172:0/2011543105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5588307349a0 0x558830734da0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400e550 tx=0x7fd1d40095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31df8f92d1186ff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.444+0000 7fd1d27fc640 1 --2- 172.21.15.172:0/2007037303 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1c80a2e70 0x7fd1c80a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.444+0000 7fd1d27fc640 1 -- 172.21.15.172:0/2007037303 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd1d80ee780 con 0x7fd1dc102d50 2024-09-10T08:59:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.444+0000 7fd1c77fe640 1 -- 172.21.15.172:0/2011543105 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d400d990 con 0x5588307349a0 2024-09-10T08:59:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.445+0000 7fd1c77fe640 1 -- 172.21.15.172:0/2011543105 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1d401c070 con 0x5588307349a0 2024-09-10T08:59:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.445+0000 7fd1e17e8640 1 --2- 172.21.15.172:0/2007037303 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1c80a2e70 0x7fd1c80a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.446+0000 7fd1e67e6740 1 -- 172.21.15.172:0/2011543105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5588307349a0 msgr2=0x558830734da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:56.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.446+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/2011543105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5588307349a0 0x558830734da0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400e550 tx=0x7fd1d40095c0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.449+0000 7fd1e67e6740 1 -- 172.21.15.172:0/2011543105 shutdown_connections 2024-09-10T08:59:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.449+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/2011543105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55883073a370 0x55883073c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.449+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/2011543105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588307352e0 0x558830739da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.449+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/2011543105 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5588307349a0 0x558830734da0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.449+0000 7fd1e67e6740 1 -- 172.21.15.172:0/2011543105 >> 172.21.15.172:0/2011543105 conn(0x558830730010 msgr2=0x558830730410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.450+0000 7fd1e17e8640 1 --2- 172.21.15.172:0/2007037303 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1c80a2e70 0x7fd1c80a5330 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80b7670 tx=0x7fd1d8005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.453+0000 7fd1e67e6740 1 -- 172.21.15.172:0/2011543105 shutdown_connections 2024-09-10T08:59:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.455+0000 7fd1e67e6740 1 -- 172.21.15.172:0/2011543105 wait complete. 2024-09-10T08:59:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.455+0000 7fd1e67e6740 1 -- start start 2024-09-10T08:59:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.455+0000 7fd1e67e6740 1 Processor -- start 2024-09-10T08:59:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.456+0000 7fd1e67e6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5588307349a0 0x55883074fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.456+0000 7fd1e67e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307352e0 0x5588307500e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.456+0000 7fd1e67e6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55883073a370 0x558830757120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.456+0000 7fd1e67e6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55883074e3a0 con 0x55883073a370 2024-09-10T08:59:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.456+0000 7fd1e67e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558830621290 con 0x5588307352e0 2024-09-10T08:59:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.456+0000 7fd1e67e6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55883074e520 con 0x5588307349a0 2024-09-10T08:59:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.456+0000 7fd1e67e6740 10 client.0 _ll_register_callbacks cb 0x558830652140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:59:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.457+0000 7fd1e0fe7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307352e0 0x5588307500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.457+0000 7fd1e0fe7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307352e0 0x5588307500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:36492/0 (socket says 172.21.15.172:36492) 2024-09-10T08:59:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.457+0000 7fd1e0fe7640 1 -- 172.21.15.172:0/825166394 learned_addr learned my addr 172.21.15.172:0/825166394 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.457+0000 7fd1e1fe9640 1 --2- 172.21.15.172:0/825166394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5588307349a0 0x55883074fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.457+0000 7fd1e1fe9640 1 --2- 172.21.15.172:0/825166394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55883073a370 0x558830757120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.457+0000 7fd1e1fe9640 1 -- 172.21.15.172:0/825166394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307352e0 msgr2=0x5588307500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.457+0000 7fd1e1fe9640 1 --2- 172.21.15.172:0/825166394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307352e0 0x5588307500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.457+0000 7fd1e1fe9640 1 -- 172.21.15.172:0/825166394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55883073a370 msgr2=0x558830757120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.457+0000 7fd1e1fe9640 1 --2- 172.21.15.172:0/825166394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55883073a370 0x558830757120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.457+0000 7fd1e1fe9640 1 -- 172.21.15.172:0/825166394 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588303ab210 con 0x5588307349a0 2024-09-10T08:59:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.458+0000 7fd1e1fe9640 1 --2- 172.21.15.172:0/825166394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5588307349a0 0x55883074fba0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400e550 tx=0x7fd1d40095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3bb502f6e5aa23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.459+0000 7fd1c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:59:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.463+0000 7fd1c4ff9640 1 -- 172.21.15.172:0/825166394 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d400dc40 con 0x5588307349a0 2024-09-10T08:59:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.463+0000 7fd1e67e6740 1 -- 172.21.15.172:0/825166394 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1d401c070 con 0x5588307349a0 2024-09-10T08:59:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.463+0000 7fd1c4ff9640 1 -- 172.21.15.172:0/825166394 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd1d400dde0 con 0x5588307349a0 2024-09-10T08:59:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.471+0000 7fd1c4ff9640 1 -- 172.21.15.172:0/825166394 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd1d4003970 con 0x5588307349a0 2024-09-10T08:59:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.473+0000 7fd1e67e6740 10 client.28057 fetch_fsmap learned FSMap version 41 2024-09-10T08:59:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.473+0000 7fd1e67e6740 1 -- 172.21.15.172:0/825166394 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558830477620 con 0x5588307349a0 2024-09-10T08:59:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.480+0000 7fd1c4ff9640 1 -- 172.21.15.172:0/825166394 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1d401c070 con 0x5588307349a0 2024-09-10T08:59:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.481+0000 7fd1c4ff9640 1 -- 172.21.15.172:0/825166394 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fd1d400b040 con 0x5588307349a0 2024-09-10T08:59:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.481+0000 7fd1e67e6740 10 client.28057 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T08:59:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.481+0000 7fd1e67e6740 10 client.28057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:59:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.481+0000 7fd1e67e6740 20 client.28057 populate_metadata read hostname 'smithi172' 2024-09-10T08:59:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.481+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/825166394 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55883075bc40 0x55883075e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.481+0000 7fd1e67e6740 4 client.28057 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T08:59:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.481+0000 7fd1e67e6740 1 -- 172.21.15.172:0/825166394 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588306a5a20 con 0x55883075bc40 2024-09-10T08:59:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.486+0000 7fd1e0fe7640 1 --2- 172.21.15.172:0/825166394 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55883075bc40 0x55883075e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.489+0000 7fd1e0fe7640 1 --2- 172.21.15.172:0/825166394 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55883075bc40 0x55883075e080 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9497996e87584967 server_cookie=37a7356326a815a6 in_seq=0 out_seq=0 2024-09-10T08:59:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.492+0000 7fd1c4ff9640 10 client.28057 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:56.493 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.492+0000 7fd1c4ff9640 1 -- 172.21.15.172:0/825166394 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5588306a5a20 con 0x55883075bc40 2024-09-10T08:59:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.492+0000 7fd1c4ff9640 10 client.28057 handle_command_reply: tid=0 2024-09-10T08:59:56.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.492+0000 7fd1e67e6740 1 client.28057 shutdown 2024-09-10T08:59:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.498+0000 7fd1e67e6740 1 -- 172.21.15.172:0/825166394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5588307349a0 msgr2=0x55883074fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.498+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/825166394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5588307349a0 0x55883074fba0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400e550 tx=0x7fd1d40095c0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.501+0000 7fd1e67e6740 1 -- 172.21.15.172:0/825166394 shutdown_connections 2024-09-10T08:59:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.501+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/825166394 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55883075bc40 0x55883075e080 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.501+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/825166394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55883073a370 0x558830757120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.501+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/825166394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307352e0 0x5588307500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.501+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/825166394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5588307349a0 0x55883074fba0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.501+0000 7fd1e67e6740 1 -- 172.21.15.172:0/825166394 >> 172.21.15.172:0/825166394 conn(0x558830730010 msgr2=0x558830731820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.501+0000 7fd1c4ff9640 0 client.28057 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.505+0000 7fd1e67e6740 1 -- 172.21.15.172:0/825166394 shutdown_connections 2024-09-10T08:59:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.508+0000 7fd1e67e6740 1 -- 172.21.15.172:0/825166394 wait complete. 2024-09-10T08:59:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.508+0000 7fd1e67e6740 20 client.28057 trim_cache size 0 max 16384 2024-09-10T08:59:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.552+0000 7fd1e67e6740 1 Processor -- start 2024-09-10T08:59:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.553+0000 7fd1e67e6740 1 -- start start 2024-09-10T08:59:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.553+0000 7fd1e67e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307349a0 0x558830731400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.553+0000 7fd1e67e6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588307352e0 0x5588307570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.553+0000 7fd1e67e6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55883073a370 0x558830731940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.553+0000 7fd1e1fe9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307349a0 0x558830731400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.553+0000 7fd1e1fe9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307349a0 0x558830731400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:36500/0 (socket says 172.21.15.172:36500) 2024-09-10T08:59:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.553+0000 7fd1e0fe7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588307352e0 0x5588307570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.553+0000 7fd1e0fe7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588307352e0 0x5588307570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49632/0 (socket says 172.21.15.172:49632) 2024-09-10T08:59:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.553+0000 7fd1e1fe9640 1 -- 172.21.15.172:0/1007710580 learned_addr learned my addr 172.21.15.172:0/1007710580 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.553+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1007710580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c000faa0 con 0x5588307352e0 2024-09-10T08:59:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.553+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1007710580 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c00116a0 con 0x5588307349a0 2024-09-10T08:59:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.553+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1007710580 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c0010170 con 0x55883073a370 2024-09-10T08:59:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.553+0000 7fd1e1fe9640 1 --2- 172.21.15.172:0/1007710580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55883073a370 0x558830731940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.561+0000 7fd1e0fe7640 1 -- 172.21.15.172:0/1007710580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55883073a370 msgr2=0x558830731940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.561+0000 7fd1e0fe7640 1 --2- 172.21.15.172:0/1007710580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55883073a370 0x558830731940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.561+0000 7fd1e0fe7640 1 -- 172.21.15.172:0/1007710580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307349a0 msgr2=0x558830731400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.561+0000 7fd1e0fe7640 1 --2- 172.21.15.172:0/1007710580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307349a0 0x558830731400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.561+0000 7fd1e0fe7640 1 -- 172.21.15.172:0/1007710580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55883074f6e0 con 0x5588307352e0 2024-09-10T08:59:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.575+0000 7fd1e0fe7640 1 --2- 172.21.15.172:0/1007710580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588307352e0 0x5588307570b0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x55883075e900 tx=0x7fd1cc014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a8feffc75a8526 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.579+0000 7fd1c6ffd640 1 -- 172.21.15.172:0/1007710580 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc014110 con 0x5588307352e0 2024-09-10T08:59:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.579+0000 7fd1c6ffd640 1 -- 172.21.15.172:0/1007710580 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1cc0142b0 con 0x5588307352e0 2024-09-10T08:59:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.579+0000 7fd1c6ffd640 1 -- 172.21.15.172:0/1007710580 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc012450 con 0x5588307352e0 2024-09-10T08:59:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.580+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1007710580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588307352e0 msgr2=0x5588307570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.580+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/1007710580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588307352e0 0x5588307570b0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x55883075e900 tx=0x7fd1cc014840 comp rx=0 tx=0).stop 2024-09-10T08:59:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.580+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1007710580 shutdown_connections 2024-09-10T08:59:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.580+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/1007710580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55883073a370 0x558830731940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.580+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/1007710580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588307352e0 0x5588307570b0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.580+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/1007710580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307349a0 0x558830731400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.580+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1007710580 >> 172.21.15.172:0/1007710580 conn(0x558830730010 msgr2=0x55883074ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.580+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1007710580 shutdown_connections 2024-09-10T08:59:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.580+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1007710580 wait complete. 2024-09-10T08:59:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.581+0000 7fd1e67e6740 1 -- start start 2024-09-10T08:59:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.581+0000 7fd1e67e6740 1 Processor -- start 2024-09-10T08:59:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.582+0000 7fd1e67e6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588307349a0 0x5588307541b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.582+0000 7fd1e67e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307352e0 0x5588307546f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.582+0000 7fd1e67e6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55883073a370 0x558830755700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.582+0000 7fd1e67e6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c0010fd0 con 0x5588307349a0 2024-09-10T08:59:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.582+0000 7fd1e0fe7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307352e0 0x5588307546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.582+0000 7fd1e0fe7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307352e0 0x5588307546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:36504/0 (socket says 172.21.15.172:36504) 2024-09-10T08:59:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.582+0000 7fd1e0fe7640 1 -- 172.21.15.172:0/1936534830 learned_addr learned my addr 172.21.15.172:0/1936534830 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T08:59:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.582+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1936534830 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c0010900 con 0x5588307352e0 2024-09-10T08:59:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.582+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1936534830 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c000db30 con 0x55883073a370 2024-09-10T08:59:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.582+0000 7fd1e1fe9640 1 --2- 172.21.15.172:0/1936534830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55883073a370 0x558830755700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.583+0000 7fd1e67e6740 10 client.0 _ll_register_callbacks cb 0x7fd1d400e450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T08:59:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.583+0000 7fd1e1fe9640 1 -- 172.21.15.172:0/1936534830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307352e0 msgr2=0x5588307546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.583+0000 7fd1e1fe9640 1 --2- 172.21.15.172:0/1936534830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307352e0 0x5588307546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.583+0000 7fd1e1fe9640 1 -- 172.21.15.172:0/1936534830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588307349a0 msgr2=0x5588307541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T08:59:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.583+0000 7fd1e1fe9640 1 --2- 172.21.15.172:0/1936534830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588307349a0 0x5588307541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.583+0000 7fd1e1fe9640 1 -- 172.21.15.172:0/1936534830 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558830755d10 con 0x55883073a370 2024-09-10T08:59:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.583+0000 7fd1e1fe9640 1 --2- 172.21.15.172:0/1936534830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55883073a370 0x558830755700 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4002230 tx=0x7fd1d4005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d6a3b36dd04afe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T08:59:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.583+0000 7fd1c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T08:59:56.590 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.583+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1936534830 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55883069fd70 con 0x55883073a370 2024-09-10T08:59:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.584+0000 7fd1c4ff9640 1 -- 172.21.15.172:0/1936534830 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4061050 con 0x55883073a370 2024-09-10T08:59:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.584+0000 7fd1c4ff9640 1 -- 172.21.15.172:0/1936534830 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd1d402d040 con 0x55883073a370 2024-09-10T08:59:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.584+0000 7fd1c4ff9640 1 -- 172.21.15.172:0/1936534830 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd1d400de50 con 0x55883073a370 2024-09-10T08:59:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.587+0000 7fd1e67e6740 10 client.28063 fetch_fsmap learned FSMap version 41 2024-09-10T08:59:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.587+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1936534830 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({fsmap=41}) -- 0x558830756ec0 con 0x55883073a370 2024-09-10T08:59:56.591 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.589+0000 7fd1c4ff9640 1 -- 172.21.15.172:0/1936534830 <== mon.1 v2:172.21.15.191:3300/0 4 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7fd1d400b040 con 0x55883073a370 2024-09-10T08:59:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.589+0000 7fd1e67e6740 10 client.28063 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T08:59:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.589+0000 7fd1e67e6740 10 client.28063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T08:59:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.589+0000 7fd1e67e6740 20 client.28063 populate_metadata read hostname 'smithi172' 2024-09-10T08:59:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.589+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/1936534830 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558830743110 0x558830745500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T08:59:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.589+0000 7fd1e67e6740 4 client.28063 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T08:59:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.589+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1936534830 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55883069fd70 con 0x558830743110 2024-09-10T08:59:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.590+0000 7fd1e0fe7640 1 --2- 172.21.15.172:0/1936534830 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558830743110 0x558830745500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T08:59:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.604+0000 7fd1e0fe7640 1 --2- 172.21.15.172:0/1936534830 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558830743110 0x558830745500 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=675518976a305488 server_cookie=51f6e6b0096c198c in_seq=0 out_seq=0 2024-09-10T08:59:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:56.604+0000 7fd1c4ff9640 10 client.28063 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.498+0000 7fd1c4ff9640 1 -- 172.21.15.172:0/1936534830 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55883069fd70 con 0x558830743110 2024-09-10T08:59:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.498+0000 7fd1c4ff9640 10 client.28063 handle_command_reply: tid=0 2024-09-10T08:59:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.498+0000 7fd1e67e6740 1 client.28063 shutdown 2024-09-10T08:59:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.500+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1936534830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55883073a370 msgr2=0x558830755700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.500+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/1936534830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55883073a370 0x558830755700 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4002230 tx=0x7fd1d4005b50 comp rx=0 tx=0).stop 2024-09-10T08:59:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.502+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1936534830 shutdown_connections 2024-09-10T08:59:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.502+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/1936534830 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558830743110 0x558830745500 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.502+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/1936534830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55883073a370 0x558830755700 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.502+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/1936534830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588307352e0 0x5588307546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.502+0000 7fd1e67e6740 1 --2- 172.21.15.172:0/1936534830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588307349a0 0x5588307541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.502+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1936534830 >> 172.21.15.172:0/1936534830 conn(0x558830730010 msgr2=0x55883072f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.502+0000 7fd1c4ff9640 0 client.28063 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T08:59:57.506 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.505+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1936534830 shutdown_connections 2024-09-10T08:59:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.507+0000 7fd1e67e6740 1 -- 172.21.15.172:0/1936534830 wait complete. 2024-09-10T08:59:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.507+0000 7fd1e67e6740 20 client.28063 trim_cache size 0 max 16384 2024-09-10T08:59:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T08:59:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T08:59:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T08:59:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T08:59:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.513+0000 7fd1e3a4c640 1 -- 172.21.15.172:0/2007037303 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1c80a2e70 msgr2=0x7fd1c80a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.513+0000 7fd1e3a4c640 1 --2- 172.21.15.172:0/2007037303 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1c80a2e70 0x7fd1c80a5330 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80b7670 tx=0x7fd1d8005c90 comp rx=0 tx=0).stop 2024-09-10T08:59:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.513+0000 7fd1e3a4c640 1 -- 172.21.15.172:0/2007037303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1dc102d50 msgr2=0x7fd1dc195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T08:59:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.513+0000 7fd1e3a4c640 1 --2- 172.21.15.172:0/2007037303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1dc102d50 0x7fd1dc195d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc102810 tx=0x7fd1d80042e0 comp rx=0 tx=0).stop 2024-09-10T08:59:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.513+0000 7fd1e3a4c640 1 -- 172.21.15.172:0/2007037303 shutdown_connections 2024-09-10T08:59:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.513+0000 7fd1e3a4c640 1 --2- 172.21.15.172:0/2007037303 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd1c80a2e70 0x7fd1c80a5330 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.514+0000 7fd1e3a4c640 1 --2- 172.21.15.172:0/2007037303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1dc1ac5c0 0x7fd1dc1ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.514+0000 7fd1e3a4c640 1 --2- 172.21.15.172:0/2007037303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1dc103710 0x7fd1dc1962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.514+0000 7fd1e3a4c640 1 --2- 172.21.15.172:0/2007037303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1dc102d50 0x7fd1dc195d80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T08:59:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.514+0000 7fd1e3a4c640 1 -- 172.21.15.172:0/2007037303 >> 172.21.15.172:0/2007037303 conn(0x7fd1dc0fd2e0 msgr2=0x7fd1dc074660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T08:59:57.517 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.516+0000 7fd1e3a4c640 1 -- 172.21.15.172:0/2007037303 shutdown_connections 2024-09-10T08:59:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T08:59:57.516+0000 7fd1e3a4c640 1 -- 172.21.15.172:0/2007037303 wait complete. 2024-09-10T08:59:57.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:57 smithi174 ceph-mon[34068]: pgmap v762: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-09-10T08:59:57.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:57 smithi191 ceph-mon[35314]: pgmap v762: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-09-10T08:59:57.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:57 smithi172 ceph-mon[30854]: pgmap v762: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-09-10T08:59:59.471 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 08:59:59 smithi172 ceph-mon[30854]: pgmap v763: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.84k op/s 2024-09-10T08:59:59.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 08:59:59 smithi174 ceph-mon[34068]: pgmap v763: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.84k op/s 2024-09-10T08:59:59.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 08:59:59 smithi191 ceph-mon[35314]: pgmap v763: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.84k op/s 2024-09-10T09:00:00.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:00 smithi174 ceph-mon[34068]: overall HEALTH_OK 2024-09-10T09:00:00.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:00 smithi191 ceph-mon[35314]: overall HEALTH_OK 2024-09-10T09:00:00.825 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:00:00.826 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T09:00:00.826 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T09:00:00.826 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:00:00.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.334+0000 7f66fa6ab640 1 -- 172.21.15.172:0/892803461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f41036e0 msgr2=0x7f66f4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:00.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.334+0000 7f66fa6ab640 1 --2- 172.21.15.172:0/892803461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f41036e0 0x7f66f4107b90 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009930 tx=0x7f66e402a420 comp rx=0 tx=0).stop 2024-09-10T09:00:00.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.334+0000 7f66fa6ab640 1 -- 172.21.15.172:0/892803461 shutdown_connections 2024-09-10T09:00:00.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.334+0000 7f66fa6ab640 1 --2- 172.21.15.172:0/892803461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66f41082a0 0x7f66f410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.334+0000 7f66fa6ab640 1 --2- 172.21.15.172:0/892803461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f41036e0 0x7f66f4107b90 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.334+0000 7f66fa6ab640 1 --2- 172.21.15.172:0/892803461 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66f4102d30 0x7f66f4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.334+0000 7f66fa6ab640 1 -- 172.21.15.172:0/892803461 >> 172.21.15.172:0/892803461 conn(0x7f66f40fc7d0 msgr2=0x7f66f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:00.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.334+0000 7f66fa6ab640 1 -- 172.21.15.172:0/892803461 shutdown_connections 2024-09-10T09:00:00.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.334+0000 7f66fa6ab640 1 -- 172.21.15.172:0/892803461 wait complete. 2024-09-10T09:00:00.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.334+0000 7f66fa6ab640 1 Processor -- start 2024-09-10T09:00:00.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.334+0000 7f66fa6ab640 1 -- start start 2024-09-10T09:00:00.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66fa6ab640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66f4102d30 0x7f66f41984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66fa6ab640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66f41036e0 0x7f66f4198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66fa6ab640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f41082a0 0x7f66f419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66fa6ab640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f66f4111f10 con 0x7f66f41082a0 2024-09-10T09:00:00.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66fa6ab640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66f4111d90 con 0x7f66f41036e0 2024-09-10T09:00:00.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66fa6ab640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f66f4112090 con 0x7f66f4102d30 2024-09-10T09:00:00.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66f8c48640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f41082a0 0x7f66f419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66f3fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66f4102d30 0x7f66f41984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66f8c48640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f41082a0 0x7f66f419cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49650/0 (socket says 172.21.15.172:49650) 2024-09-10T09:00:00.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66f8c48640 1 -- 172.21.15.172:0/3304826615 learned_addr learned my addr 172.21.15.172:0/3304826615 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:00.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66f3fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66f4102d30 0x7f66f41984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:56868/0 (socket says 172.21.15.172:56868) 2024-09-10T09:00:00.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66f37fe640 1 --2- 172.21.15.172:0/3304826615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66f41036e0 0x7f66f4198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66f8c48640 1 -- 172.21.15.172:0/3304826615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66f4102d30 msgr2=0x7f66f41984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:00.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66f8c48640 1 --2- 172.21.15.172:0/3304826615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66f4102d30 0x7f66f41984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66f8c48640 1 -- 172.21.15.172:0/3304826615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66f41036e0 msgr2=0x7f66f4198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:00.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66f8c48640 1 --2- 172.21.15.172:0/3304826615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66f41036e0 0x7f66f4198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66f8c48640 1 -- 172.21.15.172:0/3304826615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f419d4a0 con 0x7f66f41082a0 2024-09-10T09:00:00.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66f37fe640 1 --2- 172.21.15.172:0/3304826615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66f41036e0 0x7f66f4198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:00:00.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.335+0000 7f66f8c48640 1 --2- 172.21.15.172:0/3304826615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f41082a0 0x7f66f419cdc0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00eaa0 tx=0x7f66ec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:00.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.336+0000 7f66f17fa640 1 -- 172.21.15.172:0/3304826615 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec00cdb0 con 0x7f66f41082a0 2024-09-10T09:00:00.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.336+0000 7f66f17fa640 1 -- 172.21.15.172:0/3304826615 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66ec005020 con 0x7f66f41082a0 2024-09-10T09:00:00.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.336+0000 7f66fa6ab640 1 -- 172.21.15.172:0/3304826615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f419d790 con 0x7f66f41082a0 2024-09-10T09:00:00.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.336+0000 7f66f17fa640 1 -- 172.21.15.172:0/3304826615 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec0116d0 con 0x7f66f41082a0 2024-09-10T09:00:00.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.336+0000 7f66fa6ab640 1 -- 172.21.15.172:0/3304826615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f41a5070 con 0x7f66f41082a0 2024-09-10T09:00:00.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.338+0000 7f66f17fa640 1 -- 172.21.15.172:0/3304826615 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f66ec0051d0 con 0x7f66f41082a0 2024-09-10T09:00:00.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.338+0000 7f66f17fa640 1 --2- 172.21.15.172:0/3304826615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66e00a2c00 0x7f66e00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.338+0000 7f66fca84740 1 Processor -- start 2024-09-10T09:00:00.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66fca84740 1 -- start start 2024-09-10T09:00:00.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66f3fff640 1 --2- 172.21.15.172:0/3304826615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66e00a2c00 0x7f66e00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66f17fa640 1 -- 172.21.15.172:0/3304826615 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f66ec0cf4e0 con 0x7f66f41082a0 2024-09-10T09:00:00.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66fca84740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d099a4a0 0x55e1d099a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66fca84740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099adc0 0x55e1d09a5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66fca84740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d09a5680 0x55e1d09a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66fca84740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e1d09002e0 con 0x55e1d099adc0 2024-09-10T09:00:00.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66fca84740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e1d08a7ce0 con 0x55e1d09a5680 2024-09-10T09:00:00.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66fca84740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e1d08fc2a0 con 0x55e1d099a4a0 2024-09-10T09:00:00.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66f37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099adc0 0x55e1d09a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66f37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099adc0 0x55e1d09a5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49658/0 (socket says 172.21.15.172:49658) 2024-09-10T09:00:00.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66f37fe640 1 -- 172.21.15.172:0/3687542319 learned_addr learned my addr 172.21.15.172:0/3687542319 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:00.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66f8c48640 1 --2- 172.21.15.172:0/3687542319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d09a5680 0x55e1d09a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66f3fff640 1 --2- 172.21.15.172:0/3304826615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66e00a2c00 0x7f66e00a50c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f66dc005d00 tx=0x7f66dc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:00.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66f3fff640 1 --2- 172.21.15.172:0/3687542319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d099a4a0 0x55e1d099a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66f37fe640 1 -- 172.21.15.172:0/3687542319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d099a4a0 msgr2=0x55e1d099a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:00.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66f37fe640 1 --2- 172.21.15.172:0/3687542319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d099a4a0 0x55e1d099a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66f37fe640 1 -- 172.21.15.172:0/3687542319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d09a5680 msgr2=0x55e1d09a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:00.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66f37fe640 1 --2- 172.21.15.172:0/3687542319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d09a5680 0x55e1d09a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66f37fe640 1 -- 172.21.15.172:0/3687542319 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1d088c440 con 0x55e1d099adc0 2024-09-10T09:00:00.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66f8c48640 1 --2- 172.21.15.172:0/3687542319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d09a5680 0x55e1d09a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:00:00.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.339+0000 7f66f37fe640 1 --2- 172.21.15.172:0/3687542319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099adc0 0x55e1d09a5140 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f66e4000c00 tx=0x7f66e4032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef075807da2f51a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:00.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.340+0000 7f66ea7fc640 1 -- 172.21.15.172:0/3687542319 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e403f070 con 0x55e1d099adc0 2024-09-10T09:00:00.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.340+0000 7f66ea7fc640 1 -- 172.21.15.172:0/3687542319 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66e4032e30 con 0x55e1d099adc0 2024-09-10T09:00:00.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.340+0000 7f66ea7fc640 1 -- 172.21.15.172:0/3687542319 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4035070 con 0x55e1d099adc0 2024-09-10T09:00:00.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.340+0000 7f66fca84740 1 -- 172.21.15.172:0/3687542319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099adc0 msgr2=0x55e1d09a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:00.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.340+0000 7f66fca84740 1 --2- 172.21.15.172:0/3687542319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099adc0 0x55e1d09a5140 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f66e4000c00 tx=0x7f66e4032c40 comp rx=0 tx=0).stop 2024-09-10T09:00:00.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.340+0000 7f66fca84740 1 -- 172.21.15.172:0/3687542319 shutdown_connections 2024-09-10T09:00:00.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.340+0000 7f66fca84740 1 --2- 172.21.15.172:0/3687542319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d09a5680 0x55e1d09a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.340+0000 7f66fca84740 1 --2- 172.21.15.172:0/3687542319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099adc0 0x55e1d09a5140 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.340+0000 7f66fca84740 1 --2- 172.21.15.172:0/3687542319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d099a4a0 0x55e1d099a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.340+0000 7f66fca84740 1 -- 172.21.15.172:0/3687542319 >> 172.21.15.172:0/3687542319 conn(0x55e1d0996a30 msgr2=0x55e1d0996e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:00.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.340+0000 7f66fca84740 1 -- 172.21.15.172:0/3687542319 shutdown_connections 2024-09-10T09:00:00.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.340+0000 7f66fca84740 1 -- 172.21.15.172:0/3687542319 wait complete. 2024-09-10T09:00:00.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.341+0000 7f66fca84740 1 -- start start 2024-09-10T09:00:00.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.341+0000 7f66fca84740 1 Processor -- start 2024-09-10T09:00:00.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.341+0000 7f66fca84740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d099a4a0 0x55e1d09a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.341+0000 7f66fca84740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099adc0 0x55e1d09a2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.341+0000 7f66fca84740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d09a5680 0x55e1d09a2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.341+0000 7f66fca84740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e1d09a8250 con 0x55e1d099adc0 2024-09-10T09:00:00.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.341+0000 7f66fca84740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e1d08bb050 con 0x55e1d09a5680 2024-09-10T09:00:00.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.341+0000 7f66fca84740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e1d09a83d0 con 0x55e1d099a4a0 2024-09-10T09:00:00.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.341+0000 7f66fca84740 10 client.0 _ll_register_callbacks cb 0x55e1d08fb160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:00.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.341+0000 7f66f37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099adc0 0x55e1d09a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.341+0000 7f66f37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099adc0 0x55e1d09a2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49670/0 (socket says 172.21.15.172:49670) 2024-09-10T09:00:00.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.341+0000 7f66f37fe640 1 -- 172.21.15.172:0/2219196103 learned_addr learned my addr 172.21.15.172:0/2219196103 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:00.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.341+0000 7f66f3fff640 1 --2- 172.21.15.172:0/2219196103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d099a4a0 0x55e1d09a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.342+0000 7f66f8c48640 1 --2- 172.21.15.172:0/2219196103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d09a5680 0x55e1d09a2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.342+0000 7f66f37fe640 1 -- 172.21.15.172:0/2219196103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d099a4a0 msgr2=0x55e1d09a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:00.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.342+0000 7f66f37fe640 1 --2- 172.21.15.172:0/2219196103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d099a4a0 0x55e1d09a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.342+0000 7f66f37fe640 1 -- 172.21.15.172:0/2219196103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d09a5680 msgr2=0x55e1d09a2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:00.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.342+0000 7f66f37fe640 1 --2- 172.21.15.172:0/2219196103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d09a5680 0x55e1d09a2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.342+0000 7f66f37fe640 1 -- 172.21.15.172:0/2219196103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1d06713c0 con 0x55e1d099adc0 2024-09-10T09:00:00.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.342+0000 7f66f37fe640 1 --2- 172.21.15.172:0/2219196103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099adc0 0x55e1d09a2250 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f66e4031440 tx=0x7f66e40055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5fb1b1b5993c4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:00.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.342+0000 7f66d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:00:00.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.342+0000 7f66d37fe640 1 -- 172.21.15.172:0/2219196103 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4035070 con 0x55e1d099adc0 2024-09-10T09:00:00.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.342+0000 7f66d37fe640 1 -- 172.21.15.172:0/2219196103 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f66e4004440 con 0x55e1d099adc0 2024-09-10T09:00:00.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.342+0000 7f66fca84740 1 -- 172.21.15.172:0/2219196103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66e4032e30 con 0x55e1d099adc0 2024-09-10T09:00:00.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.342+0000 7f66d37fe640 1 -- 172.21.15.172:0/2219196103 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f66e4055050 con 0x55e1d099adc0 2024-09-10T09:00:00.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.342+0000 7f66fca84740 10 client.19341 fetch_fsmap learned FSMap version 41 2024-09-10T09:00:00.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.342+0000 7f66fca84740 1 -- 172.21.15.172:0/2219196103 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e1d09b3bd0 con 0x55e1d099adc0 2024-09-10T09:00:00.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.342+0000 7f66d37fe640 1 -- 172.21.15.172:0/2219196103 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66e4032e30 con 0x55e1d099adc0 2024-09-10T09:00:00.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.343+0000 7f66d37fe640 1 -- 172.21.15.172:0/2219196103 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f66e4004950 con 0x55e1d099adc0 2024-09-10T09:00:00.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.343+0000 7f66fca84740 10 client.19341 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T09:00:00.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.343+0000 7f66fca84740 10 client.19341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:00:00.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.343+0000 7f66fca84740 20 client.19341 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:00.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.343+0000 7f66fca84740 1 --2- 172.21.15.172:0/2219196103 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e1d09b4f20 0x55e1d09b7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.343+0000 7f66fca84740 4 client.19341 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:00:00.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.343+0000 7f66fca84740 1 -- 172.21.15.172:0/2219196103 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1d090d850 con 0x55e1d09b4f20 2024-09-10T09:00:00.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.343+0000 7f66f3fff640 1 --2- 172.21.15.172:0/2219196103 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e1d09b4f20 0x55e1d09b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.343+0000 7f66d37fe640 10 client.19341 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:00.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.343+0000 7f66f3fff640 1 --2- 172.21.15.172:0/2219196103 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e1d09b4f20 0x55e1d09b7310 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dfeb42d41d52723 server_cookie=8475ef9023b0430a in_seq=0 out_seq=0 2024-09-10T09:00:00.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.344+0000 7f66d37fe640 1 -- 172.21.15.172:0/2219196103 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e1d090d850 con 0x55e1d09b4f20 2024-09-10T09:00:00.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.344+0000 7f66d37fe640 10 client.19341 handle_command_reply: tid=0 2024-09-10T09:00:00.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.344+0000 7f66fca84740 1 client.19341 shutdown 2024-09-10T09:00:00.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.344+0000 7f66fca84740 1 -- 172.21.15.172:0/2219196103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099adc0 msgr2=0x55e1d09a2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:00.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.344+0000 7f66fca84740 1 --2- 172.21.15.172:0/2219196103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099adc0 0x55e1d09a2250 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f66e4031440 tx=0x7f66e40055e0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.344+0000 7f66fca84740 1 -- 172.21.15.172:0/2219196103 shutdown_connections 2024-09-10T09:00:00.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.344+0000 7f66fca84740 1 --2- 172.21.15.172:0/2219196103 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e1d09b4f20 0x55e1d09b7310 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.344+0000 7f66d37fe640 0 client.19341 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:00.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.344+0000 7f66fca84740 1 --2- 172.21.15.172:0/2219196103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d09a5680 0x55e1d09a2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.344+0000 7f66fca84740 1 --2- 172.21.15.172:0/2219196103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099adc0 0x55e1d09a2250 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.344+0000 7f66fca84740 1 --2- 172.21.15.172:0/2219196103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d099a4a0 0x55e1d09a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.344+0000 7f66fca84740 1 -- 172.21.15.172:0/2219196103 >> 172.21.15.172:0/2219196103 conn(0x55e1d0996a30 msgr2=0x55e1d099c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:00.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.344+0000 7f66fca84740 1 -- 172.21.15.172:0/2219196103 shutdown_connections 2024-09-10T09:00:00.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.344+0000 7f66fca84740 1 -- 172.21.15.172:0/2219196103 wait complete. 2024-09-10T09:00:00.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.344+0000 7f66fca84740 20 client.19341 trim_cache size 0 max 16384 2024-09-10T09:00:00.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.361+0000 7f66fca84740 1 Processor -- start 2024-09-10T09:00:00.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.361+0000 7f66fca84740 1 -- start start 2024-09-10T09:00:00.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.361+0000 7f66fca84740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d099a4a0 0x55e1d0996e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.361+0000 7f66fca84740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d099adc0 0x55e1d099f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.361+0000 7f66fca84740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d09a5680 0x55e1d099f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.361+0000 7f66fca84740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f66f41aeab0 con 0x55e1d09a5680 2024-09-10T09:00:00.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.361+0000 7f66fca84740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66f41b06b0 con 0x55e1d099a4a0 2024-09-10T09:00:00.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.361+0000 7f66fca84740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f66f41af180 con 0x55e1d099adc0 2024-09-10T09:00:00.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.361+0000 7f66f8c48640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d09a5680 0x55e1d099f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.362+0000 7f66f8c48640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d09a5680 0x55e1d099f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49684/0 (socket says 172.21.15.172:49684) 2024-09-10T09:00:00.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.362+0000 7f66f8c48640 1 -- 172.21.15.172:0/1809904878 learned_addr learned my addr 172.21.15.172:0/1809904878 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:00.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.362+0000 7f66f3fff640 1 --2- 172.21.15.172:0/1809904878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d099a4a0 0x55e1d0996e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.362+0000 7f66f37fe640 1 --2- 172.21.15.172:0/1809904878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d099adc0 0x55e1d099f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.362+0000 7f66f8c48640 1 -- 172.21.15.172:0/1809904878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d099adc0 msgr2=0x55e1d099f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:00.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.362+0000 7f66f8c48640 1 --2- 172.21.15.172:0/1809904878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d099adc0 0x55e1d099f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.362+0000 7f66f8c48640 1 -- 172.21.15.172:0/1809904878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d099a4a0 msgr2=0x55e1d0996e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:00.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.362+0000 7f66f8c48640 1 --2- 172.21.15.172:0/1809904878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d099a4a0 0x55e1d0996e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.362+0000 7f66f8c48640 1 -- 172.21.15.172:0/1809904878 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1d099abf0 con 0x55e1d09a5680 2024-09-10T09:00:00.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.362+0000 7f66f37fe640 1 --2- 172.21.15.172:0/1809904878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d099adc0 0x55e1d099f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:00:00.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.362+0000 7f66f8c48640 1 --2- 172.21.15.172:0/1809904878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d09a5680 0x55e1d099f9f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0ccc80 tx=0x7f66ec09c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d17c086309ff2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:00.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.362+0000 7f66e9ffb640 1 -- 172.21.15.172:0/1809904878 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec01f040 con 0x55e1d09a5680 2024-09-10T09:00:00.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.362+0000 7f66e9ffb640 1 -- 172.21.15.172:0/1809904878 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66ec0a4650 con 0x55e1d09a5680 2024-09-10T09:00:00.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.362+0000 7f66e9ffb640 1 -- 172.21.15.172:0/1809904878 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec080b00 con 0x55e1d09a5680 2024-09-10T09:00:00.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.363+0000 7f66fca84740 1 -- 172.21.15.172:0/1809904878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d09a5680 msgr2=0x55e1d099f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:00.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.363+0000 7f66fca84740 1 --2- 172.21.15.172:0/1809904878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d09a5680 0x55e1d099f9f0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0ccc80 tx=0x7f66ec09c1a0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.363+0000 7f66fca84740 1 -- 172.21.15.172:0/1809904878 shutdown_connections 2024-09-10T09:00:00.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.363+0000 7f66fca84740 1 --2- 172.21.15.172:0/1809904878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d09a5680 0x55e1d099f9f0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.363+0000 7f66fca84740 1 --2- 172.21.15.172:0/1809904878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d099adc0 0x55e1d099f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.363+0000 7f66fca84740 1 --2- 172.21.15.172:0/1809904878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d099a4a0 0x55e1d0996e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.363+0000 7f66fca84740 1 -- 172.21.15.172:0/1809904878 >> 172.21.15.172:0/1809904878 conn(0x55e1d0996a30 msgr2=0x55e1d099d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:00.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.363+0000 7f66fca84740 1 -- 172.21.15.172:0/1809904878 shutdown_connections 2024-09-10T09:00:00.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.363+0000 7f66fca84740 1 -- 172.21.15.172:0/1809904878 wait complete. 2024-09-10T09:00:00.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.363+0000 7f66fca84740 1 -- start start 2024-09-10T09:00:00.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.363+0000 7f66fca84740 1 Processor -- start 2024-09-10T09:00:00.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66fca84740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099a4a0 0x55e1d09bab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66fca84740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d099adc0 0x55e1d09b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66fca84740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d09a5680 0x55e1d09b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66fca84740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f66f41affe0 con 0x55e1d099a4a0 2024-09-10T09:00:00.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66fca84740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66f41af910 con 0x55e1d099adc0 2024-09-10T09:00:00.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66fca84740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f66f41aca90 con 0x55e1d09a5680 2024-09-10T09:00:00.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66fca84740 10 client.0 _ll_register_callbacks cb 0x7f66ec009270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:00.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099a4a0 0x55e1d09bab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099a4a0 0x55e1d09bab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49688/0 (socket says 172.21.15.172:49688) 2024-09-10T09:00:00.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66f3fff640 1 -- 172.21.15.172:0/4187422124 learned_addr learned my addr 172.21.15.172:0/4187422124 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:00.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66f37fe640 1 --2- 172.21.15.172:0/4187422124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d099adc0 0x55e1d09b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66f8c48640 1 --2- 172.21.15.172:0/4187422124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d09a5680 0x55e1d09b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66f3fff640 1 -- 172.21.15.172:0/4187422124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d09a5680 msgr2=0x55e1d09b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:00.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66f3fff640 1 --2- 172.21.15.172:0/4187422124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d09a5680 0x55e1d09b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66f3fff640 1 -- 172.21.15.172:0/4187422124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d099adc0 msgr2=0x55e1d09b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:00.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66f3fff640 1 --2- 172.21.15.172:0/4187422124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d099adc0 0x55e1d09b7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66f3fff640 1 -- 172.21.15.172:0/4187422124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1d09b8ce0 con 0x55e1d099a4a0 2024-09-10T09:00:00.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66f3fff640 1 --2- 172.21.15.172:0/4187422124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099a4a0 0x55e1d09bab70 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x55e1d09a8490 tx=0x7f66dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69241c325edfb231 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:00.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:00:00.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66d37fe640 1 -- 172.21.15.172:0/4187422124 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc01eb70 con 0x55e1d099a4a0 2024-09-10T09:00:00.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66d37fe640 1 -- 172.21.15.172:0/4187422124 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f66dc01ed10 con 0x55e1d099a4a0 2024-09-10T09:00:00.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.364+0000 7f66fca84740 1 -- 172.21.15.172:0/4187422124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66ec0a4650 con 0x55e1d099a4a0 2024-09-10T09:00:00.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.365+0000 7f66d37fe640 1 -- 172.21.15.172:0/4187422124 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f66dc03a050 con 0x55e1d099a4a0 2024-09-10T09:00:00.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.365+0000 7f66fca84740 10 client.19347 fetch_fsmap learned FSMap version 41 2024-09-10T09:00:00.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.365+0000 7f66fca84740 1 -- 172.21.15.172:0/4187422124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e1d0a1f920 con 0x55e1d099a4a0 2024-09-10T09:00:00.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.365+0000 7f66d37fe640 1 -- 172.21.15.172:0/4187422124 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66ec0a4650 con 0x55e1d099a4a0 2024-09-10T09:00:00.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.365+0000 7f66d37fe640 1 -- 172.21.15.172:0/4187422124 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f66dc033760 con 0x55e1d099a4a0 2024-09-10T09:00:00.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.365+0000 7f66fca84740 10 client.19347 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T09:00:00.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.365+0000 7f66fca84740 10 client.19347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:00:00.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.365+0000 7f66fca84740 20 client.19347 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:00.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.365+0000 7f66fca84740 1 --2- 172.21.15.172:0/4187422124 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e1d09b4f20 0x55e1d0a22050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:00.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.365+0000 7f66fca84740 4 client.19347 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:00:00.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.365+0000 7f66fca84740 1 -- 172.21.15.172:0/4187422124 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e1d0907d60 con 0x55e1d09b4f20 2024-09-10T09:00:00.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.365+0000 7f66f37fe640 1 --2- 172.21.15.172:0/4187422124 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e1d09b4f20 0x55e1d0a22050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:00.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.365+0000 7f66d37fe640 10 client.19347 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:00.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.365+0000 7f66f37fe640 1 --2- 172.21.15.172:0/4187422124 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e1d09b4f20 0x55e1d0a22050 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c52fa95e643be6 server_cookie=61f3b3b7856a8643 in_seq=0 out_seq=0 2024-09-10T09:00:00.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66d37fe640 1 -- 172.21.15.172:0/4187422124 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e1d0907d60 con 0x55e1d09b4f20 2024-09-10T09:00:00.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66d37fe640 10 client.19347 handle_command_reply: tid=0 2024-09-10T09:00:00.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66fca84740 1 client.19347 shutdown 2024-09-10T09:00:00.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66fca84740 1 -- 172.21.15.172:0/4187422124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099a4a0 msgr2=0x55e1d09bab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:00.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66fca84740 1 --2- 172.21.15.172:0/4187422124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099a4a0 0x55e1d09bab70 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x55e1d09a8490 tx=0x7f66dc004730 comp rx=0 tx=0).stop 2024-09-10T09:00:00.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66fca84740 1 -- 172.21.15.172:0/4187422124 shutdown_connections 2024-09-10T09:00:00.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66fca84740 1 --2- 172.21.15.172:0/4187422124 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e1d09b4f20 0x55e1d0a22050 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66fca84740 1 --2- 172.21.15.172:0/4187422124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e1d09a5680 0x55e1d09b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66fca84740 1 --2- 172.21.15.172:0/4187422124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1d099adc0 0x55e1d09b7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66fca84740 1 --2- 172.21.15.172:0/4187422124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1d099a4a0 0x55e1d09bab70 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66d37fe640 0 client.19347 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:00.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66fca84740 1 -- 172.21.15.172:0/4187422124 >> 172.21.15.172:0/4187422124 conn(0x55e1d0996a30 msgr2=0x55e1d099cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:00.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66fca84740 1 -- 172.21.15.172:0/4187422124 shutdown_connections 2024-09-10T09:00:00.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66fca84740 1 -- 172.21.15.172:0/4187422124 wait complete. 2024-09-10T09:00:00.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66fca84740 20 client.19347 trim_cache size 0 max 16384 2024-09-10T09:00:00.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66fa6ab640 1 -- 172.21.15.172:0/3304826615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66e00a2c00 msgr2=0x7f66e00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:00.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66fa6ab640 1 --2- 172.21.15.172:0/3304826615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66e00a2c00 0x7f66e00a50c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f66dc005d00 tx=0x7f66dc00b410 comp rx=0 tx=0).stop 2024-09-10T09:00:00.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66fa6ab640 1 -- 172.21.15.172:0/3304826615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f41082a0 msgr2=0x7f66f419cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:00.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.366+0000 7f66fa6ab640 1 --2- 172.21.15.172:0/3304826615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f41082a0 0x7f66f419cdc0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00eaa0 tx=0x7f66ec00ef60 comp rx=0 tx=0).stop 2024-09-10T09:00:00.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.367+0000 7f66fa6ab640 1 -- 172.21.15.172:0/3304826615 shutdown_connections 2024-09-10T09:00:00.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.367+0000 7f66fa6ab640 1 --2- 172.21.15.172:0/3304826615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f66e00a2c00 0x7f66e00a50c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.367+0000 7f66fa6ab640 1 --2- 172.21.15.172:0/3304826615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66f41082a0 0x7f66f419cdc0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.367+0000 7f66fa6ab640 1 --2- 172.21.15.172:0/3304826615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66f41036e0 0x7f66f4198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.367+0000 7f66fa6ab640 1 --2- 172.21.15.172:0/3304826615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66f4102d30 0x7f66f41984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:00.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.367+0000 7f66fa6ab640 1 -- 172.21.15.172:0/3304826615 >> 172.21.15.172:0/3304826615 conn(0x7f66f40fc7d0 msgr2=0x7f66f4105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:00.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.367+0000 7f66fa6ab640 1 -- 172.21.15.172:0/3304826615 shutdown_connections 2024-09-10T09:00:00.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T08:59:59.367+0000 7f66fa6ab640 1 -- 172.21.15.172:0/3304826615 wait complete. 2024-09-10T09:00:00.845 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:00 smithi172 ceph-mon[30854]: overall HEALTH_OK 2024-09-10T09:00:01.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:01 smithi174 ceph-mon[34068]: pgmap v764: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-09-10T09:00:01.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:01 smithi191 ceph-mon[35314]: pgmap v764: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-09-10T09:00:01.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:01 smithi172 ceph-mon[30854]: pgmap v764: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 3.68k op/s 2024-09-10T09:00:03.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:03 smithi174 ceph-mon[34068]: pgmap v765: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 3.52k op/s 2024-09-10T09:00:03.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:03 smithi191 ceph-mon[35314]: pgmap v765: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 3.52k op/s 2024-09-10T09:00:03.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:03 smithi172 ceph-mon[30854]: pgmap v765: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 3.52k op/s 2024-09-10T09:00:05.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:05 smithi174 ceph-mon[34068]: pgmap v766: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 4.15k op/s 2024-09-10T09:00:05.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:05 smithi191 ceph-mon[35314]: pgmap v766: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 4.15k op/s 2024-09-10T09:00:05.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:05 smithi172 ceph-mon[30854]: pgmap v766: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 4.15k op/s 2024-09-10T09:00:07.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:07 smithi174 ceph-mon[34068]: pgmap v767: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.44k op/s 2024-09-10T09:00:07.804 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:07 smithi191 ceph-mon[35314]: pgmap v767: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.44k op/s 2024-09-10T09:00:07.813 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:07 smithi172 ceph-mon[30854]: pgmap v767: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.44k op/s 2024-09-10T09:00:09.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:09 smithi174 ceph-mon[34068]: pgmap v768: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 3.69k op/s 2024-09-10T09:00:09.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:09 smithi191 ceph-mon[35314]: pgmap v768: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 3.69k op/s 2024-09-10T09:00:09.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:09 smithi172 ceph-mon[30854]: pgmap v768: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 3.69k op/s 2024-09-10T09:00:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.732+0000 7f83209f7640 1 -- 172.21.15.172:0/306758164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83140a5670 msgr2=0x7f83140a5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.732+0000 7f83209f7640 1 --2- 172.21.15.172:0/306758164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83140a5670 0x7f83140a5af0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f830c009900 tx=0x7f830c02a480 comp rx=0 tx=0).stop 2024-09-10T09:00:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.734+0000 7f83209f7640 1 -- 172.21.15.172:0/306758164 shutdown_connections 2024-09-10T09:00:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.734+0000 7f83209f7640 1 --2- 172.21.15.172:0/306758164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83140a6030 0x7f83140aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.734+0000 7f83209f7640 1 --2- 172.21.15.172:0/306758164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83140a5670 0x7f83140a5af0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.734+0000 7f83209f7640 1 --2- 172.21.15.172:0/306758164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83140a4540 0x7f83140a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.734+0000 7f83209f7640 1 -- 172.21.15.172:0/306758164 >> 172.21.15.172:0/306758164 conn(0x7f831409fba0 msgr2=0x7f83140a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.740+0000 7f83209f7640 1 -- 172.21.15.172:0/306758164 shutdown_connections 2024-09-10T09:00:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.740+0000 7f83209f7640 1 -- 172.21.15.172:0/306758164 wait complete. 2024-09-10T09:00:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.741+0000 7f83209f7640 1 Processor -- start 2024-09-10T09:00:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.741+0000 7f83209f7640 1 -- start start 2024-09-10T09:00:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.741+0000 7f83209f7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83140a4540 0x7f8314143040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.741+0000 7f83209f7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83140a5670 0x7f8314143580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.741+0000 7f83209f7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83140a6030 0x7f831414a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.741+0000 7f83209f7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83140b8860 con 0x7f83140a5670 2024-09-10T09:00:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.741+0000 7f83209f7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83140b86e0 con 0x7f83140a4540 2024-09-10T09:00:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.741+0000 7f83209f7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f83140b89e0 con 0x7f83140a6030 2024-09-10T09:00:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.741+0000 7f831a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83140a5670 0x7f8314143580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.741+0000 7f831a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83140a5670 0x7f8314143580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37900/0 (socket says 172.21.15.172:37900) 2024-09-10T09:00:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.741+0000 7f831a59c640 1 -- 172.21.15.172:0/3793964371 learned_addr learned my addr 172.21.15.172:0/3793964371 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.741+0000 7f831ad9d640 1 --2- 172.21.15.172:0/3793964371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83140a4540 0x7f8314143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.741+0000 7f831b59e640 1 --2- 172.21.15.172:0/3793964371 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83140a6030 0x7f831414a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.742+0000 7f831a59c640 1 -- 172.21.15.172:0/3793964371 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83140a6030 msgr2=0x7f831414a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.742+0000 7f831a59c640 1 --2- 172.21.15.172:0/3793964371 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83140a6030 0x7f831414a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.742+0000 7f831a59c640 1 -- 172.21.15.172:0/3793964371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83140a4540 msgr2=0x7f8314143040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.742+0000 7f831a59c640 1 --2- 172.21.15.172:0/3793964371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83140a4540 0x7f8314143040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.742+0000 7f831a59c640 1 -- 172.21.15.172:0/3793964371 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831414acf0 con 0x7f83140a5670 2024-09-10T09:00:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.742+0000 7f831ad9d640 1 --2- 172.21.15.172:0/3793964371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83140a4540 0x7f8314143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:00:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.742+0000 7f831a59c640 1 --2- 172.21.15.172:0/3793964371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83140a5670 0x7f8314143580 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f830c005750 tx=0x7f830c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.742+0000 7f8303fff640 1 -- 172.21.15.172:0/3793964371 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c03f070 con 0x7f83140a5670 2024-09-10T09:00:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.742+0000 7f8303fff640 1 -- 172.21.15.172:0/3793964371 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f830c004c20 con 0x7f83140a5670 2024-09-10T09:00:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.742+0000 7f83209f7640 1 -- 172.21.15.172:0/3793964371 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831414af80 con 0x7f83140a5670 2024-09-10T09:00:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.743+0000 7f83209f7640 1 -- 172.21.15.172:0/3793964371 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831414b3e0 con 0x7f83140a5670 2024-09-10T09:00:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.743+0000 7f8303fff640 1 -- 172.21.15.172:0/3793964371 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c035030 con 0x7f83140a5670 2024-09-10T09:00:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.746+0000 7f8324067740 1 Processor -- start 2024-09-10T09:00:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.746+0000 7f8303fff640 1 -- 172.21.15.172:0/3793964371 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f830c042070 con 0x7f83140a5670 2024-09-10T09:00:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.746+0000 7f8324067740 1 -- start start 2024-09-10T09:00:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.746+0000 7f8324067740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e59b70 0x557e18e59f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.746+0000 7f8324067740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e5a4b0 0x557e18e5ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.746+0000 7f8324067740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e5f540 0x557e18e61a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.746+0000 7f8324067740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557e18dbd2f0 con 0x557e18e59b70 2024-09-10T09:00:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.746+0000 7f8324067740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557e18d65270 con 0x557e18e5a4b0 2024-09-10T09:00:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.746+0000 7f8324067740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557e18db92b0 con 0x557e18e5f540 2024-09-10T09:00:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.746+0000 7f831b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e59b70 0x557e18e59f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.746+0000 7f831b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e59b70 0x557e18e59f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37910/0 (socket says 172.21.15.172:37910) 2024-09-10T09:00:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.746+0000 7f831b59e640 1 -- 172.21.15.172:0/3644457348 learned_addr learned my addr 172.21.15.172:0/3644457348 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.746+0000 7f831ad9d640 1 --2- 172.21.15.172:0/3644457348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e5a4b0 0x557e18e5ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.747+0000 7f831a59c640 1 --2- 172.21.15.172:0/3644457348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e5f540 0x557e18e61a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.747+0000 7f831b59e640 1 -- 172.21.15.172:0/3644457348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e5f540 msgr2=0x557e18e61a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.747+0000 7f831b59e640 1 --2- 172.21.15.172:0/3644457348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e5f540 0x557e18e61a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.747+0000 7f831b59e640 1 -- 172.21.15.172:0/3644457348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e5a4b0 msgr2=0x557e18e5ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.747+0000 7f831b59e640 1 --2- 172.21.15.172:0/3644457348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e5a4b0 0x557e18e5ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.747+0000 7f831b59e640 1 -- 172.21.15.172:0/3644457348 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e18d64340 con 0x557e18e59b70 2024-09-10T09:00:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.747+0000 7f831a59c640 1 --2- 172.21.15.172:0/3644457348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e5f540 0x557e18e61a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:00:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.747+0000 7f8303fff640 1 --2- 172.21.15.172:0/3793964371 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83040a2c50 0x7f83040a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.747+0000 7f8303fff640 1 -- 172.21.15.172:0/3793964371 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f830c0efc10 con 0x7f83140a5670 2024-09-10T09:00:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.747+0000 7f831ad9d640 1 --2- 172.21.15.172:0/3793964371 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83040a2c50 0x7f83040a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.747+0000 7f831b59e640 1 --2- 172.21.15.172:0/3644457348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e59b70 0x557e18e59f70 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f831c051790 tx=0x7f831c0757a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aaa59c98131c667 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.747+0000 7f83017fa640 1 -- 172.21.15.172:0/3644457348 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c068bc0 con 0x557e18e59b70 2024-09-10T09:00:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.747+0000 7f83017fa640 1 -- 172.21.15.172:0/3644457348 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f831c07a030 con 0x557e18e59b70 2024-09-10T09:00:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.747+0000 7f83017fa640 1 -- 172.21.15.172:0/3644457348 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c0766a0 con 0x557e18e59b70 2024-09-10T09:00:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.748+0000 7f8324067740 1 -- 172.21.15.172:0/3644457348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e59b70 msgr2=0x557e18e59f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.748+0000 7f831ad9d640 1 --2- 172.21.15.172:0/3793964371 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83040a2c50 0x7f83040a5110 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x557e18e5aee0 tx=0x7f831000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:10.755 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.748+0000 7f8324067740 1 --2- 172.21.15.172:0/3644457348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e59b70 0x557e18e59f70 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f831c051790 tx=0x7f831c0757a0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.748+0000 7f8324067740 1 -- 172.21.15.172:0/3644457348 shutdown_connections 2024-09-10T09:00:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.748+0000 7f8324067740 1 --2- 172.21.15.172:0/3644457348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e5f540 0x557e18e61a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.748+0000 7f8324067740 1 --2- 172.21.15.172:0/3644457348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e5a4b0 0x557e18e5ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.748+0000 7f8324067740 1 --2- 172.21.15.172:0/3644457348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e59b70 0x557e18e59f70 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.748+0000 7f8324067740 1 -- 172.21.15.172:0/3644457348 >> 172.21.15.172:0/3644457348 conn(0x557e18e551e0 msgr2=0x557e18e555e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.748+0000 7f8324067740 1 -- 172.21.15.172:0/3644457348 shutdown_connections 2024-09-10T09:00:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.748+0000 7f8324067740 1 -- 172.21.15.172:0/3644457348 wait complete. 2024-09-10T09:00:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.748+0000 7f8324067740 1 -- start start 2024-09-10T09:00:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.748+0000 7f8324067740 1 Processor -- start 2024-09-10T09:00:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f8324067740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e59b70 0x557e18e63df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f8324067740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e5a4b0 0x557e18e64330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.756 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f8324067740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e6b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f8324067740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557e18e62510 con 0x557e18e5f540 2024-09-10T09:00:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f8324067740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557e18d46290 con 0x557e18e59b70 2024-09-10T09:00:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f8324067740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557e18e62690 con 0x557e18e5a4b0 2024-09-10T09:00:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f831b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e6b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f831b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e6b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37920/0 (socket says 172.21.15.172:37920) 2024-09-10T09:00:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f831b59e640 1 -- 172.21.15.172:0/799635037 learned_addr learned my addr 172.21.15.172:0/799635037 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f8324067740 10 client.0 _ll_register_callbacks cb 0x557e18d77140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f831ad9d640 1 --2- 172.21.15.172:0/799635037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e59b70 0x557e18e63df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f831b59e640 1 -- 172.21.15.172:0/799635037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e5a4b0 msgr2=0x557e18e64330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f831b59e640 1 --2- 172.21.15.172:0/799635037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e5a4b0 0x557e18e64330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f831b59e640 1 -- 172.21.15.172:0/799635037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e59b70 msgr2=0x557e18e63df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f831b59e640 1 --2- 172.21.15.172:0/799635037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e59b70 0x557e18e63df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f831b59e640 1 -- 172.21.15.172:0/799635037 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e18ad0210 con 0x557e18e5f540 2024-09-10T09:00:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f831b59e640 1 --2- 172.21.15.172:0/799635037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e6b300 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f831c050d60 tx=0x7f831c06ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b0d9507853aec8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f82feff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:00:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f82feff5640 1 -- 172.21.15.172:0/799635037 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c068b70 con 0x557e18e5f540 2024-09-10T09:00:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f82feff5640 1 -- 172.21.15.172:0/799635037 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f831c068d10 con 0x557e18e5f540 2024-09-10T09:00:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f8324067740 1 -- 172.21.15.172:0/799635037 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f831c07a030 con 0x557e18e5f540 2024-09-10T09:00:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f82feff5640 1 -- 172.21.15.172:0/799635037 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f831c06a0d0 con 0x557e18e5f540 2024-09-10T09:00:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.749+0000 7f8324067740 10 client.19359 fetch_fsmap learned FSMap version 41 2024-09-10T09:00:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.750+0000 7f8324067740 1 -- 172.21.15.172:0/799635037 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557e18b9c620 con 0x557e18e5f540 2024-09-10T09:00:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.750+0000 7f82feff5640 1 -- 172.21.15.172:0/799635037 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f831c07a030 con 0x557e18e5f540 2024-09-10T09:00:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.750+0000 7f82feff5640 1 -- 172.21.15.172:0/799635037 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f831c076a20 con 0x557e18e5f540 2024-09-10T09:00:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.750+0000 7f8324067740 10 client.19359 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T09:00:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.750+0000 7f8324067740 10 client.19359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:00:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.750+0000 7f8324067740 20 client.19359 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.750+0000 7f8324067740 1 --2- 172.21.15.172:0/799635037 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557e18e6f8f0 0x557e18e71ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.750+0000 7f8324067740 4 client.19359 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:00:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.750+0000 7f8324067740 1 -- 172.21.15.172:0/799635037 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e18dcaa20 con 0x557e18e6f8f0 2024-09-10T09:00:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.750+0000 7f831ad9d640 1 --2- 172.21.15.172:0/799635037 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557e18e6f8f0 0x557e18e71ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.751+0000 7f82feff5640 10 client.19359 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.751+0000 7f831ad9d640 1 --2- 172.21.15.172:0/799635037 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557e18e6f8f0 0x557e18e71ce0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1819204152c531ce server_cookie=484da6242277c32b in_seq=0 out_seq=0 2024-09-10T09:00:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.751+0000 7f82feff5640 1 -- 172.21.15.172:0/799635037 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e18dcaa20 con 0x557e18e6f8f0 2024-09-10T09:00:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.751+0000 7f82feff5640 10 client.19359 handle_command_reply: tid=0 2024-09-10T09:00:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.752+0000 7f8324067740 1 client.19359 shutdown 2024-09-10T09:00:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.752+0000 7f8324067740 1 -- 172.21.15.172:0/799635037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 msgr2=0x557e18e6b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.752+0000 7f8324067740 1 --2- 172.21.15.172:0/799635037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e6b300 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f831c050d60 tx=0x7f831c06ab80 comp rx=0 tx=0).stop 2024-09-10T09:00:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.752+0000 7f8324067740 1 -- 172.21.15.172:0/799635037 shutdown_connections 2024-09-10T09:00:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.752+0000 7f8324067740 1 --2- 172.21.15.172:0/799635037 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557e18e6f8f0 0x557e18e71ce0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.752+0000 7f8324067740 1 --2- 172.21.15.172:0/799635037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e6b300 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.752+0000 7f82feff5640 0 client.19359 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.752+0000 7f8324067740 1 --2- 172.21.15.172:0/799635037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e5a4b0 0x557e18e64330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.752+0000 7f8324067740 1 --2- 172.21.15.172:0/799635037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e59b70 0x557e18e63df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.752+0000 7f8324067740 1 -- 172.21.15.172:0/799635037 >> 172.21.15.172:0/799635037 conn(0x557e18e551e0 msgr2=0x557e18e569f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.752+0000 7f8324067740 1 -- 172.21.15.172:0/799635037 shutdown_connections 2024-09-10T09:00:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.752+0000 7f8324067740 1 -- 172.21.15.172:0/799635037 wait complete. 2024-09-10T09:00:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.752+0000 7f8324067740 20 client.19359 trim_cache size 0 max 16384 2024-09-10T09:00:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.766+0000 7f8324067740 1 Processor -- start 2024-09-10T09:00:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.766+0000 7f8324067740 1 -- start start 2024-09-10T09:00:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.766+0000 7f8324067740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e59b70 0x557e18e4fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.766+0000 7f8324067740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e5a4b0 0x557e18e6b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.766+0000 7f8324067740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e63df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.771 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.766+0000 7f8324067740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83141557f0 con 0x557e18e5f540 2024-09-10T09:00:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.766+0000 7f8324067740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83141573f0 con 0x557e18e5a4b0 2024-09-10T09:00:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.766+0000 7f8324067740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8314155ec0 con 0x557e18e59b70 2024-09-10T09:00:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.766+0000 7f831b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e63df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.766+0000 7f831b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e63df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37930/0 (socket says 172.21.15.172:37930) 2024-09-10T09:00:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.766+0000 7f831b59e640 1 -- 172.21.15.172:0/4279496200 learned_addr learned my addr 172.21.15.172:0/4279496200 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.766+0000 7f831ad9d640 1 --2- 172.21.15.172:0/4279496200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e59b70 0x557e18e4fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.766+0000 7f831a59c640 1 --2- 172.21.15.172:0/4279496200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e5a4b0 0x557e18e6b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.766+0000 7f831b59e640 1 -- 172.21.15.172:0/4279496200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e59b70 msgr2=0x557e18e4fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.766+0000 7f831b59e640 1 --2- 172.21.15.172:0/4279496200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e59b70 0x557e18e4fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.766+0000 7f831b59e640 1 -- 172.21.15.172:0/4279496200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e5a4b0 msgr2=0x557e18e6b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:10.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.766+0000 7f831b59e640 1 --2- 172.21.15.172:0/4279496200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e5a4b0 0x557e18e6b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.767+0000 7f831b59e640 1 -- 172.21.15.172:0/4279496200 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e18e62dc0 con 0x557e18e5f540 2024-09-10T09:00:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.767+0000 7f831b59e640 1 --2- 172.21.15.172:0/4279496200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e63df0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f831c051790 tx=0x7f831c068340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b365e97401ee9cbd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.767+0000 7f8300ff9640 1 -- 172.21.15.172:0/4279496200 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c08b420 con 0x557e18e5f540 2024-09-10T09:00:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.767+0000 7f8300ff9640 1 -- 172.21.15.172:0/4279496200 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f831c083070 con 0x557e18e5f540 2024-09-10T09:00:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.767+0000 7f8300ff9640 1 -- 172.21.15.172:0/4279496200 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c08ba50 con 0x557e18e5f540 2024-09-10T09:00:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.767+0000 7f8324067740 1 -- 172.21.15.172:0/4279496200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 msgr2=0x557e18e63df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.767+0000 7f8324067740 1 --2- 172.21.15.172:0/4279496200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e63df0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f831c051790 tx=0x7f831c068340 comp rx=0 tx=0).stop 2024-09-10T09:00:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f8324067740 1 -- 172.21.15.172:0/4279496200 shutdown_connections 2024-09-10T09:00:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f8324067740 1 --2- 172.21.15.172:0/4279496200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e63df0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f8324067740 1 --2- 172.21.15.172:0/4279496200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e5a4b0 0x557e18e6b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f8324067740 1 --2- 172.21.15.172:0/4279496200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e59b70 0x557e18e4fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f8324067740 1 -- 172.21.15.172:0/4279496200 >> 172.21.15.172:0/4279496200 conn(0x557e18e551e0 msgr2=0x557e18e5ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f8324067740 1 -- 172.21.15.172:0/4279496200 shutdown_connections 2024-09-10T09:00:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f8324067740 1 -- 172.21.15.172:0/4279496200 wait complete. 2024-09-10T09:00:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f8324067740 1 -- start start 2024-09-10T09:00:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f8324067740 1 Processor -- start 2024-09-10T09:00:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f8324067740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e59b70 0x557e18e6ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f8324067740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e5a4b0 0x557e18e6b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f8324067740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e676a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f8324067740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8314156d20 con 0x557e18e5f540 2024-09-10T09:00:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f8324067740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8314156650 con 0x557e18e5a4b0 2024-09-10T09:00:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f8324067740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f83141537d0 con 0x557e18e59b70 2024-09-10T09:00:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f831b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e676a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:10.774 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f8324067740 10 client.0 _ll_register_callbacks cb 0x7f831000a980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f831b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e676a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:37942/0 (socket says 172.21.15.172:37942) 2024-09-10T09:00:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f831b59e640 1 -- 172.21.15.172:0/1281337287 learned_addr learned my addr 172.21.15.172:0/1281337287 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f831ad9d640 1 --2- 172.21.15.172:0/1281337287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e59b70 0x557e18e6ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.768+0000 7f831a59c640 1 --2- 172.21.15.172:0/1281337287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e5a4b0 0x557e18e6b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f831b59e640 1 -- 172.21.15.172:0/1281337287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e59b70 msgr2=0x557e18e6ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f831b59e640 1 --2- 172.21.15.172:0/1281337287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e59b70 0x557e18e6ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f831b59e640 1 -- 172.21.15.172:0/1281337287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e5a4b0 msgr2=0x557e18e6b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f831b59e640 1 --2- 172.21.15.172:0/1281337287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e5a4b0 0x557e18e6b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f831b59e640 1 -- 172.21.15.172:0/1281337287 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e18e67e30 con 0x557e18e5f540 2024-09-10T09:00:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f831b59e640 1 --2- 172.21.15.172:0/1281337287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e676a0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f831c050d60 tx=0x7f831c0684c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7674583ec6cb80f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f82feff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:00:10.775 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f82feff5640 1 -- 172.21.15.172:0/1281337287 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c068da0 con 0x557e18e5f540 2024-09-10T09:00:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f8324067740 1 -- 172.21.15.172:0/1281337287 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f831c083070 con 0x557e18e5f540 2024-09-10T09:00:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f82feff5640 1 -- 172.21.15.172:0/1281337287 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f831c08b7a0 con 0x557e18e5f540 2024-09-10T09:00:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f82feff5640 1 -- 172.21.15.172:0/1281337287 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f831c08b9a0 con 0x557e18e5f540 2024-09-10T09:00:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f8324067740 10 client.19365 fetch_fsmap learned FSMap version 41 2024-09-10T09:00:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f8324067740 1 -- 172.21.15.172:0/1281337287 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557e18ed9250 con 0x557e18e5f540 2024-09-10T09:00:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f82feff5640 1 -- 172.21.15.172:0/1281337287 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f831c083070 con 0x557e18e5f540 2024-09-10T09:00:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f82feff5640 1 -- 172.21.15.172:0/1281337287 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f831c099440 con 0x557e18e5f540 2024-09-10T09:00:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f8324067740 10 client.19365 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T09:00:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f8324067740 10 client.19365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:00:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f8324067740 20 client.19365 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f8324067740 1 --2- 172.21.15.172:0/1281337287 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557e18e6f8f0 0x557e18edb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:10.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f8324067740 4 client.19365 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:00:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.769+0000 7f8324067740 1 -- 172.21.15.172:0/1281337287 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e18dc4d70 con 0x557e18e6f8f0 2024-09-10T09:00:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.770+0000 7f831ad9d640 1 --2- 172.21.15.172:0/1281337287 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557e18e6f8f0 0x557e18edb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.770+0000 7f82feff5640 10 client.19365 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:10.770+0000 7f831ad9d640 1 --2- 172.21.15.172:0/1281337287 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557e18e6f8f0 0x557e18edb970 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52029c8b8addfd8b server_cookie=5902ffacc3b4d0e9 in_seq=0 out_seq=0 2024-09-10T09:00:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:11 smithi174 ceph-mon[34068]: pgmap v769: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 3.40k op/s 2024-09-10T09:00:11.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:11 smithi191 ceph-mon[35314]: pgmap v769: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 3.40k op/s 2024-09-10T09:00:11.837 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:11 smithi172 ceph-mon[30854]: pgmap v769: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 3.40k op/s 2024-09-10T09:00:12.997 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:00:13.051 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:00:13.052 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:00:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.446+0000 7f82feff5640 1 -- 172.21.15.172:0/1281337287 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e18dc4d70 con 0x557e18e6f8f0 2024-09-10T09:00:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.446+0000 7f82feff5640 10 client.19365 handle_command_reply: tid=0 2024-09-10T09:00:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.446+0000 7f8324067740 1 client.19365 shutdown 2024-09-10T09:00:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.446+0000 7f8324067740 1 -- 172.21.15.172:0/1281337287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 msgr2=0x557e18e676a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.446+0000 7f8324067740 1 --2- 172.21.15.172:0/1281337287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e676a0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f831c050d60 tx=0x7f831c0684c0 comp rx=0 tx=0).stop 2024-09-10T09:00:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.446+0000 7f8324067740 1 -- 172.21.15.172:0/1281337287 shutdown_connections 2024-09-10T09:00:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.446+0000 7f8324067740 1 --2- 172.21.15.172:0/1281337287 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557e18e6f8f0 0x557e18edb970 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.446+0000 7f8324067740 1 --2- 172.21.15.172:0/1281337287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e18e5f540 0x557e18e676a0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.446+0000 7f8324067740 1 --2- 172.21.15.172:0/1281337287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e18e5a4b0 0x557e18e6b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.446+0000 7f8324067740 1 --2- 172.21.15.172:0/1281337287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557e18e59b70 0x557e18e6ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.446+0000 7f8324067740 1 -- 172.21.15.172:0/1281337287 >> 172.21.15.172:0/1281337287 conn(0x557e18e551e0 msgr2=0x557e18e63df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.451+0000 7f82feff5640 0 client.19365 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.451+0000 7f8324067740 1 -- 172.21.15.172:0/1281337287 shutdown_connections 2024-09-10T09:00:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.451+0000 7f8324067740 1 -- 172.21.15.172:0/1281337287 wait complete. 2024-09-10T09:00:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.451+0000 7f8324067740 20 client.19365 trim_cache size 0 max 16384 2024-09-10T09:00:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:00:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:00:13.452 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:00:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:00:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.469+0000 7f83209f7640 1 -- 172.21.15.172:0/3793964371 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83040a2c50 msgr2=0x7f83040a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.469+0000 7f83209f7640 1 --2- 172.21.15.172:0/3793964371 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83040a2c50 0x7f83040a5110 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x557e18e5aee0 tx=0x7f831000a2d0 comp rx=0 tx=0).stop 2024-09-10T09:00:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.469+0000 7f83209f7640 1 -- 172.21.15.172:0/3793964371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83140a5670 msgr2=0x7f8314143580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.469+0000 7f83209f7640 1 --2- 172.21.15.172:0/3793964371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83140a5670 0x7f8314143580 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f830c005750 tx=0x7f830c005780 comp rx=0 tx=0).stop 2024-09-10T09:00:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.469+0000 7f83209f7640 1 -- 172.21.15.172:0/3793964371 shutdown_connections 2024-09-10T09:00:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.469+0000 7f83209f7640 1 --2- 172.21.15.172:0/3793964371 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f83040a2c50 0x7f83040a5110 secure :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0x557e18e5aee0 tx=0x7f831000a2d0 comp rx=0 tx=0).stop 2024-09-10T09:00:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.469+0000 7f83209f7640 1 --2- 172.21.15.172:0/3793964371 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83140a6030 0x7f831414a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.469+0000 7f83209f7640 1 --2- 172.21.15.172:0/3793964371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83140a5670 0x7f8314143580 secure :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f830c005750 tx=0x7f830c005780 comp rx=0 tx=0).stop 2024-09-10T09:00:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.469+0000 7f83209f7640 1 --2- 172.21.15.172:0/3793964371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83140a4540 0x7f8314143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.469+0000 7f83209f7640 1 -- 172.21.15.172:0/3793964371 >> 172.21.15.172:0/3793964371 conn(0x7f831409fba0 msgr2=0x7f83140a6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.475+0000 7f83209f7640 1 -- 172.21.15.172:0/3793964371 shutdown_connections 2024-09-10T09:00:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:13.475+0000 7f83209f7640 1 -- 172.21.15.172:0/3793964371 wait complete. 2024-09-10T09:00:13.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:13 smithi174 ceph-mon[34068]: pgmap v770: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 3.05k op/s 2024-09-10T09:00:13.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:13 smithi191 ceph-mon[35314]: pgmap v770: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 3.05k op/s 2024-09-10T09:00:13.835 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:13 smithi172 ceph-mon[30854]: pgmap v770: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 3.05k op/s 2024-09-10T09:00:15.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:15 smithi174 ceph-mon[34068]: pgmap v771: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.79k op/s 2024-09-10T09:00:15.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:15 smithi191 ceph-mon[35314]: pgmap v771: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.79k op/s 2024-09-10T09:00:15.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:15 smithi172 ceph-mon[30854]: pgmap v771: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.79k op/s 2024-09-10T09:00:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:17 smithi174 ceph-mon[34068]: pgmap v772: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 3.05k op/s 2024-09-10T09:00:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:17 smithi191 ceph-mon[35314]: pgmap v772: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 3.05k op/s 2024-09-10T09:00:17.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:17 smithi172 ceph-mon[30854]: pgmap v772: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 3.05k op/s 2024-09-10T09:00:19.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.356+0000 7f9af8b13640 1 -- 172.21.15.172:0/3420508182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af4102d30 msgr2=0x7f9af4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:19.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.356+0000 7f9af8b13640 1 --2- 172.21.15.172:0/3420508182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af4102d30 0x7f9af4103110 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009930 tx=0x7f9ae402a440 comp rx=0 tx=0).stop 2024-09-10T09:00:19.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.356+0000 7f9af8b13640 1 -- 172.21.15.172:0/3420508182 shutdown_connections 2024-09-10T09:00:19.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.356+0000 7f9af8b13640 1 --2- 172.21.15.172:0/3420508182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9af41082a0 0x7f9af410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.356+0000 7f9af8b13640 1 --2- 172.21.15.172:0/3420508182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9af41036e0 0x7f9af4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.356+0000 7f9af8b13640 1 --2- 172.21.15.172:0/3420508182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af4102d30 0x7f9af4103110 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.356+0000 7f9af8b13640 1 -- 172.21.15.172:0/3420508182 >> 172.21.15.172:0/3420508182 conn(0x7f9af40fc7d0 msgr2=0x7f9af40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:19.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.356+0000 7f9af8b13640 1 -- 172.21.15.172:0/3420508182 shutdown_connections 2024-09-10T09:00:19.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.356+0000 7f9af8b13640 1 -- 172.21.15.172:0/3420508182 wait complete. 2024-09-10T09:00:19.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.357+0000 7f9af8b13640 1 Processor -- start 2024-09-10T09:00:19.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.357+0000 7f9af8b13640 1 -- start start 2024-09-10T09:00:19.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.357+0000 7f9af8b13640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9af4102d30 0x7f9af4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.357+0000 7f9af8b13640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af41036e0 0x7f9af4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.357+0000 7f9af8b13640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9af41082a0 0x7f9af419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.357+0000 7f9af8b13640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4111f90 con 0x7f9af41036e0 2024-09-10T09:00:19.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.357+0000 7f9af8b13640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4111e10 con 0x7f9af4102d30 2024-09-10T09:00:19.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.357+0000 7f9af8b13640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4112110 con 0x7f9af41082a0 2024-09-10T09:00:19.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.357+0000 7f9af2d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9af41082a0 0x7f9af419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.357+0000 7f9ae9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af41036e0 0x7f9af4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.357+0000 7f9af2d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9af41082a0 0x7f9af419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:52016/0 (socket says 172.21.15.172:52016) 2024-09-10T09:00:19.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.357+0000 7f9af2d9d640 1 -- 172.21.15.172:0/3953887194 learned_addr learned my addr 172.21.15.172:0/3953887194 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:19.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.357+0000 7f9af259c640 1 --2- 172.21.15.172:0/3953887194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9af4102d30 0x7f9af4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.358+0000 7f9af2d9d640 1 -- 172.21.15.172:0/3953887194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9af4102d30 msgr2=0x7f9af4198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:19.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.358+0000 7f9af2d9d640 1 --2- 172.21.15.172:0/3953887194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9af4102d30 0x7f9af4198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.358+0000 7f9af2d9d640 1 -- 172.21.15.172:0/3953887194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af41036e0 msgr2=0x7f9af4198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:19.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.358+0000 7f9af2d9d640 1 --2- 172.21.15.172:0/3953887194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af41036e0 0x7f9af4198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.358+0000 7f9af2d9d640 1 -- 172.21.15.172:0/3953887194 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af419d520 con 0x7f9af41082a0 2024-09-10T09:00:19.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.358+0000 7f9ae9d9b640 1 --2- 172.21.15.172:0/3953887194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af41036e0 0x7f9af4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:00:19.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.358+0000 7f9af2d9d640 1 --2- 172.21.15.172:0/3953887194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9af41082a0 0x7f9af419ce40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009fd0 tx=0x7f9aec00def0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:19.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.358+0000 7f9aebfff640 1 -- 172.21.15.172:0/3953887194 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec017070 con 0x7f9af41082a0 2024-09-10T09:00:19.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.358+0000 7f9af8b13640 1 -- 172.21.15.172:0/3953887194 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af419d810 con 0x7f9af41082a0 2024-09-10T09:00:19.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.358+0000 7f9aebfff640 1 -- 172.21.15.172:0/3953887194 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9aec003e20 con 0x7f9af41082a0 2024-09-10T09:00:19.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.358+0000 7f9aebfff640 1 -- 172.21.15.172:0/3953887194 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec010460 con 0x7f9af41082a0 2024-09-10T09:00:19.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.358+0000 7f9af8b13640 1 -- 172.21.15.172:0/3953887194 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af41a50f0 con 0x7f9af41082a0 2024-09-10T09:00:19.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.360+0000 7f9aebfff640 1 -- 172.21.15.172:0/3953887194 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9aec003970 con 0x7f9af41082a0 2024-09-10T09:00:19.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.360+0000 7f9afaeec740 1 Processor -- start 2024-09-10T09:00:19.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9afaeec740 1 -- start start 2024-09-10T09:00:19.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9afaeec740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef33760 0x55fd8ef33b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9afaeec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef3e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9afaeec740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef3e940 0x55fd8ef40d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9afaeec740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fd8ee992a0 con 0x55fd8ef33760 2024-09-10T09:00:19.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9afaeec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fd8ee40b10 con 0x55fd8ef34080 2024-09-10T09:00:19.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9afaeec740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55fd8ee95260 con 0x55fd8ef3e940 2024-09-10T09:00:19.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9aebfff640 1 --2- 172.21.15.172:0/3953887194 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9ae00a2b80 0x7f9ae00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9ae9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef33760 0x55fd8ef33b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9ae9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef33760 0x55fd8ef33b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41372/0 (socket says 172.21.15.172:41372) 2024-09-10T09:00:19.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9ae9d9b640 1 -- 172.21.15.172:0/288596599 learned_addr learned my addr 172.21.15.172:0/288596599 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:19.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9af259c640 1 --2- 172.21.15.172:0/288596599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef3e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9af2d9d640 1 --2- 172.21.15.172:0/3953887194 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9ae00a2b80 0x7f9ae00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9aebfff640 1 -- 172.21.15.172:0/3953887194 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9aec0cf040 con 0x7f9af41082a0 2024-09-10T09:00:19.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9ae9d9b640 1 --2- 172.21.15.172:0/288596599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef3e940 0x55fd8ef40d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9af259c640 1 -- 172.21.15.172:0/288596599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef3e940 msgr2=0x55fd8ef40d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:19.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9af259c640 1 --2- 172.21.15.172:0/288596599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef3e940 0x55fd8ef40d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9af259c640 1 -- 172.21.15.172:0/288596599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef33760 msgr2=0x55fd8ef33b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:19.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9af259c640 1 --2- 172.21.15.172:0/288596599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef33760 0x55fd8ef33b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.361+0000 7f9af259c640 1 -- 172.21.15.172:0/288596599 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd8ec0ab90 con 0x55fd8ef34080 2024-09-10T09:00:19.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.363+0000 7f9af259c640 1 --2- 172.21.15.172:0/288596599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef3e400 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4031440 tx=0x7f9ae4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b35dbd8bd11a33a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:19.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.363+0000 7f9af2d9d640 1 --2- 172.21.15.172:0/3953887194 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9ae00a2b80 0x7f9ae00a5040 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0979a0 tx=0x7f9aec016b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:19.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.364+0000 7f9ae8d99640 1 -- 172.21.15.172:0/288596599 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4040070 con 0x55fd8ef34080 2024-09-10T09:00:19.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.364+0000 7f9ae8d99640 1 -- 172.21.15.172:0/288596599 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ae40316a0 con 0x55fd8ef34080 2024-09-10T09:00:19.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.364+0000 7f9ae8d99640 1 -- 172.21.15.172:0/288596599 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4035040 con 0x55fd8ef34080 2024-09-10T09:00:19.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.365+0000 7f9afaeec740 1 -- 172.21.15.172:0/288596599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 msgr2=0x55fd8ef3e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:19.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.365+0000 7f9afaeec740 1 --2- 172.21.15.172:0/288596599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef3e400 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4031440 tx=0x7f9ae4005fd0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.365+0000 7f9afaeec740 1 -- 172.21.15.172:0/288596599 shutdown_connections 2024-09-10T09:00:19.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.365+0000 7f9afaeec740 1 --2- 172.21.15.172:0/288596599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef3e940 0x55fd8ef40d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.365+0000 7f9afaeec740 1 --2- 172.21.15.172:0/288596599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef3e400 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.685 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.365+0000 7f9afaeec740 1 --2- 172.21.15.172:0/288596599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef33760 0x55fd8ef33b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.365+0000 7f9afaeec740 1 -- 172.21.15.172:0/288596599 >> 172.21.15.172:0/288596599 conn(0x55fd8ef2fcf0 msgr2=0x55fd8ef300d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:19.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.365+0000 7f9afaeec740 1 -- 172.21.15.172:0/288596599 shutdown_connections 2024-09-10T09:00:19.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.365+0000 7f9afaeec740 1 -- 172.21.15.172:0/288596599 wait complete. 2024-09-10T09:00:19.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9afaeec740 1 -- start start 2024-09-10T09:00:19.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9afaeec740 1 Processor -- start 2024-09-10T09:00:19.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9afaeec740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef33760 0x55fd8ef3b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9afaeec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef3bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9afaeec740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef3e940 0x55fd8ef3c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9afaeec740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fd8ef41510 con 0x55fd8ef33760 2024-09-10T09:00:19.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9afaeec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fd8ee216e0 con 0x55fd8ef34080 2024-09-10T09:00:19.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9afaeec740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55fd8ef41690 con 0x55fd8ef3e940 2024-09-10T09:00:19.686 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9afaeec740 10 client.0 _ll_register_callbacks cb 0x55fd8ee45a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:19.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9ae9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef33760 0x55fd8ef3b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9ae9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef33760 0x55fd8ef3b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41374/0 (socket says 172.21.15.172:41374) 2024-09-10T09:00:19.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9ae9d9b640 1 -- 172.21.15.172:0/3517908656 learned_addr learned my addr 172.21.15.172:0/3517908656 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:19.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9af259c640 1 --2- 172.21.15.172:0/3517908656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef3bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9ae9d9b640 1 --2- 172.21.15.172:0/3517908656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef3e940 0x55fd8ef3c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9ae9d9b640 1 -- 172.21.15.172:0/3517908656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef3e940 msgr2=0x55fd8ef3c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:19.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9ae9d9b640 1 --2- 172.21.15.172:0/3517908656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef3e940 0x55fd8ef3c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9ae9d9b640 1 -- 172.21.15.172:0/3517908656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 msgr2=0x55fd8ef3bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:19.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9ae9d9b640 1 --2- 172.21.15.172:0/3517908656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef3bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9ae9d9b640 1 -- 172.21.15.172:0/3517908656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd8ef4aa50 con 0x55fd8ef33760 2024-09-10T09:00:19.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.366+0000 7f9af259c640 1 --2- 172.21.15.172:0/3517908656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef3bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:00:19.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.367+0000 7f9ae9d9b640 1 --2- 172.21.15.172:0/3517908656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef33760 0x55fd8ef3b880 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f9adc019e10 tx=0x7f9adc009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2a38e8ab540ad8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:19.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.367+0000 7f9ad9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:00:19.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.367+0000 7f9ad9ffb640 1 -- 172.21.15.172:0/3517908656 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc022620 con 0x55fd8ef33760 2024-09-10T09:00:19.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.367+0000 7f9ad9ffb640 1 -- 172.21.15.172:0/3517908656 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9adc0227c0 con 0x55fd8ef33760 2024-09-10T09:00:19.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.367+0000 7f9afaeec740 1 -- 172.21.15.172:0/3517908656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ae40316a0 con 0x55fd8ef33760 2024-09-10T09:00:19.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.367+0000 7f9ad9ffb640 1 -- 172.21.15.172:0/3517908656 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9adc02a050 con 0x55fd8ef33760 2024-09-10T09:00:19.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.367+0000 7f9afaeec740 10 client.19377 fetch_fsmap learned FSMap version 41 2024-09-10T09:00:19.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.367+0000 7f9afaeec740 1 -- 172.21.15.172:0/3517908656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55fd8ef4d440 con 0x55fd8ef33760 2024-09-10T09:00:19.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.367+0000 7f9ad9ffb640 1 -- 172.21.15.172:0/3517908656 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ae40316a0 con 0x55fd8ef33760 2024-09-10T09:00:19.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.367+0000 7f9ad9ffb640 1 -- 172.21.15.172:0/3517908656 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f9adc015040 con 0x55fd8ef33760 2024-09-10T09:00:19.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.367+0000 7f9afaeec740 10 client.19377 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T09:00:19.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.367+0000 7f9afaeec740 10 client.19377 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:00:19.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.367+0000 7f9afaeec740 20 client.19377 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:19.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.367+0000 7f9afaeec740 1 --2- 172.21.15.172:0/3517908656 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fd8ef4e850 0x55fd8ef50c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.368+0000 7f9afaeec740 4 client.19377 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:00:19.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.368+0000 7f9afaeec740 1 -- 172.21.15.172:0/3517908656 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd8eea69d0 con 0x55fd8ef4e850 2024-09-10T09:00:19.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.368+0000 7f9af259c640 1 --2- 172.21.15.172:0/3517908656 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fd8ef4e850 0x55fd8ef50c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.368+0000 7f9af259c640 1 --2- 172.21.15.172:0/3517908656 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fd8ef4e850 0x55fd8ef50c40 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4ed4f02fc0acad server_cookie=e0af49902553ac3b in_seq=0 out_seq=0 2024-09-10T09:00:19.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.368+0000 7f9ad9ffb640 10 client.19377 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:19.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.369+0000 7f9ad9ffb640 1 -- 172.21.15.172:0/3517908656 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd8eea69d0 con 0x55fd8ef4e850 2024-09-10T09:00:19.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.369+0000 7f9ad9ffb640 10 client.19377 handle_command_reply: tid=0 2024-09-10T09:00:19.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.369+0000 7f9afaeec740 1 client.19377 shutdown 2024-09-10T09:00:19.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.369+0000 7f9afaeec740 1 -- 172.21.15.172:0/3517908656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef33760 msgr2=0x55fd8ef3b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:19.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.369+0000 7f9afaeec740 1 --2- 172.21.15.172:0/3517908656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef33760 0x55fd8ef3b880 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f9adc019e10 tx=0x7f9adc009110 comp rx=0 tx=0).stop 2024-09-10T09:00:19.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.369+0000 7f9afaeec740 1 -- 172.21.15.172:0/3517908656 shutdown_connections 2024-09-10T09:00:19.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.369+0000 7f9afaeec740 1 --2- 172.21.15.172:0/3517908656 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fd8ef4e850 0x55fd8ef50c40 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.369+0000 7f9afaeec740 1 --2- 172.21.15.172:0/3517908656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef3e940 0x55fd8ef3c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.369+0000 7f9afaeec740 1 --2- 172.21.15.172:0/3517908656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef3bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.369+0000 7f9ad9ffb640 0 client.19377 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:19.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.369+0000 7f9afaeec740 1 --2- 172.21.15.172:0/3517908656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef33760 0x55fd8ef3b880 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.369+0000 7f9afaeec740 1 -- 172.21.15.172:0/3517908656 >> 172.21.15.172:0/3517908656 conn(0x55fd8ef2fcf0 msgr2=0x55fd8ef2ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:19.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.369+0000 7f9afaeec740 1 -- 172.21.15.172:0/3517908656 shutdown_connections 2024-09-10T09:00:19.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.369+0000 7f9afaeec740 1 -- 172.21.15.172:0/3517908656 wait complete. 2024-09-10T09:00:19.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.369+0000 7f9afaeec740 20 client.19377 trim_cache size 0 max 16384 2024-09-10T09:00:19.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.416+0000 7f9afaeec740 1 Processor -- start 2024-09-10T09:00:19.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.417+0000 7f9afaeec740 1 -- start start 2024-09-10T09:00:19.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.417+0000 7f9afaeec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef361c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.417+0000 7f9afaeec740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef3e940 0x55fd8ef564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.417+0000 7f9afaeec740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef56a00 0x55fd8ef38ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.417+0000 7f9afaeec740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41aeaf0 con 0x55fd8ef3e940 2024-09-10T09:00:19.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.417+0000 7f9afaeec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41b06f0 con 0x55fd8ef34080 2024-09-10T09:00:19.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.417+0000 7f9afaeec740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41af1c0 con 0x55fd8ef56a00 2024-09-10T09:00:19.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.420+0000 7f9ae9d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.420+0000 7f9ae9d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef56a00 0x55fd8ef38ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.420+0000 7f9af259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef3e940 0x55fd8ef564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.420+0000 7f9af259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef3e940 0x55fd8ef564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41382/0 (socket says 172.21.15.172:41382) 2024-09-10T09:00:19.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.420+0000 7f9af259c640 1 -- 172.21.15.172:0/387275181 learned_addr learned my addr 172.21.15.172:0/387275181 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:19.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.423+0000 7f9ae9d9b640 1 -- 172.21.15.172:0/387275181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef56a00 msgr2=0x55fd8ef38ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:19.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.423+0000 7f9ae9d9b640 1 --2- 172.21.15.172:0/387275181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef56a00 0x55fd8ef38ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.423+0000 7f9ae9d9b640 1 -- 172.21.15.172:0/387275181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef3e940 msgr2=0x55fd8ef564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:19.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.423+0000 7f9ae9d9b640 1 --2- 172.21.15.172:0/387275181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef3e940 0x55fd8ef564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.423+0000 7f9ae9d9b640 1 -- 172.21.15.172:0/387275181 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd8ef2b260 con 0x55fd8ef34080 2024-09-10T09:00:19.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.425+0000 7f9ae9d9b640 1 --2- 172.21.15.172:0/387275181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef361c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9adc018000 tx=0x7f9adc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1ea57a5b899b99d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:19.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.425+0000 7f9adbfff640 1 -- 172.21.15.172:0/387275181 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc026030 con 0x55fd8ef34080 2024-09-10T09:00:19.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.425+0000 7f9adbfff640 1 -- 172.21.15.172:0/387275181 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9adc01d030 con 0x55fd8ef34080 2024-09-10T09:00:19.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.425+0000 7f9adbfff640 1 -- 172.21.15.172:0/387275181 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc022980 con 0x55fd8ef34080 2024-09-10T09:00:19.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.429+0000 7f9afaeec740 1 -- 172.21.15.172:0/387275181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 msgr2=0x55fd8ef361c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:19.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.429+0000 7f9afaeec740 1 --2- 172.21.15.172:0/387275181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef361c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9adc018000 tx=0x7f9adc0046c0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.435+0000 7f9afaeec740 1 -- 172.21.15.172:0/387275181 shutdown_connections 2024-09-10T09:00:19.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.435+0000 7f9afaeec740 1 --2- 172.21.15.172:0/387275181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef56a00 0x55fd8ef38ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.435+0000 7f9afaeec740 1 --2- 172.21.15.172:0/387275181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef3e940 0x55fd8ef564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.435+0000 7f9afaeec740 1 --2- 172.21.15.172:0/387275181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef361c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.435+0000 7f9afaeec740 1 -- 172.21.15.172:0/387275181 >> 172.21.15.172:0/387275181 conn(0x55fd8ef2fcf0 msgr2=0x55fd8ef3d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:19.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.445+0000 7f9afaeec740 1 -- 172.21.15.172:0/387275181 shutdown_connections 2024-09-10T09:00:19.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.445+0000 7f9afaeec740 1 -- 172.21.15.172:0/387275181 wait complete. 2024-09-10T09:00:19.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.446+0000 7f9afaeec740 1 -- start start 2024-09-10T09:00:19.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.446+0000 7f9afaeec740 1 Processor -- start 2024-09-10T09:00:19.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.446+0000 7f9afaeec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef42da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.446+0000 7f9afaeec740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef3e940 0x55fd8ef432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.446+0000 7f9afaeec740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef56a00 0x55fd8efc0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.446+0000 7f9afaeec740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41b0020 con 0x55fd8ef3e940 2024-09-10T09:00:19.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.446+0000 7f9afaeec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41af950 con 0x55fd8ef34080 2024-09-10T09:00:19.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.446+0000 7f9afaeec740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41acad0 con 0x55fd8ef56a00 2024-09-10T09:00:19.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.446+0000 7f9afaeec740 10 client.0 _ll_register_callbacks cb 0x7f9adc019a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:19.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.447+0000 7f9ae9d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef42da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.447+0000 7f9ae9d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef56a00 0x55fd8efc0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.448+0000 7f9af259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef3e940 0x55fd8ef432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.448+0000 7f9af259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef3e940 0x55fd8ef432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41396/0 (socket says 172.21.15.172:41396) 2024-09-10T09:00:19.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.448+0000 7f9af259c640 1 -- 172.21.15.172:0/2923995681 learned_addr learned my addr 172.21.15.172:0/2923995681 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:19.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.454+0000 7f9ae9d9b640 1 -- 172.21.15.172:0/2923995681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef56a00 msgr2=0x55fd8efc0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:19.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.454+0000 7f9ae9d9b640 1 --2- 172.21.15.172:0/2923995681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef56a00 0x55fd8efc0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.454+0000 7f9ae9d9b640 1 -- 172.21.15.172:0/2923995681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef3e940 msgr2=0x55fd8ef432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:19.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.454+0000 7f9ae9d9b640 1 --2- 172.21.15.172:0/2923995681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef3e940 0x55fd8ef432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.454+0000 7f9ae9d9b640 1 -- 172.21.15.172:0/2923995681 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd8efd21f0 con 0x55fd8ef34080 2024-09-10T09:00:19.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.457+0000 7f9ae9d9b640 1 --2- 172.21.15.172:0/2923995681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef42da0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9adc019b60 tx=0x7f9adc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f59fce31c6df343 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:19.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.461+0000 7f9ad9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:00:19.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.461+0000 7f9ad9ffb640 1 -- 172.21.15.172:0/2923995681 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc026070 con 0x55fd8ef34080 2024-09-10T09:00:19.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.461+0000 7f9afaeec740 1 -- 172.21.15.172:0/2923995681 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9adc01d030 con 0x55fd8ef34080 2024-09-10T09:00:19.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.461+0000 7f9ad9ffb640 1 -- 172.21.15.172:0/2923995681 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9adc017430 con 0x55fd8ef34080 2024-09-10T09:00:19.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.467+0000 7f9ad9ffb640 1 -- 172.21.15.172:0/2923995681 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9adc017e60 con 0x55fd8ef34080 2024-09-10T09:00:19.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.467+0000 7f9afaeec740 10 client.28163 fetch_fsmap learned FSMap version 41 2024-09-10T09:00:19.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.467+0000 7f9afaeec740 1 -- 172.21.15.172:0/2923995681 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55fd8efd34a0 con 0x55fd8ef34080 2024-09-10T09:00:19.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.471+0000 7f9ad9ffb640 1 -- 172.21.15.172:0/2923995681 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9adc01d030 con 0x55fd8ef34080 2024-09-10T09:00:19.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.472+0000 7f9ad9ffb640 1 -- 172.21.15.172:0/2923995681 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 41) ==== 8386+0+0 (secure 0 0 0) 0x7f9adc03d430 con 0x55fd8ef34080 2024-09-10T09:00:19.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.472+0000 7f9afaeec740 10 client.28163 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T09:00:19.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.472+0000 7f9afaeec740 10 client.28163 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:00:19.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.472+0000 7f9afaeec740 20 client.28163 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:19.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.472+0000 7f9afaeec740 1 --2- 172.21.15.172:0/2923995681 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fd8efd3ad0 0x55fd8efd5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:19.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.472+0000 7f9afaeec740 4 client.28163 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.5, "await_for": 45.0}] 2024-09-10T09:00:19.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.472+0000 7f9afaeec740 1 -- 172.21.15.172:0/2923995681 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.5, "await_for": 45.0}) -- 0x55fd8eea0d20 con 0x55fd8efd3ad0 2024-09-10T09:00:19.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.474+0000 7f9af259c640 1 --2- 172.21.15.172:0/2923995681 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fd8efd3ad0 0x55fd8efd5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:19.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.476+0000 7f9ad9ffb640 10 client.28163 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:19.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.476+0000 7f9af259c640 1 --2- 172.21.15.172:0/2923995681 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fd8efd3ad0 0x55fd8efd5ec0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=290da21457e1c330 server_cookie=a9ef20b6fcdfd566 in_seq=0 out_seq=0 2024-09-10T09:00:19.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.810+0000 7f9ad9ffb640 1 -- 172.21.15.172:0/2923995681 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fd8eea0d20 con 0x55fd8efd3ad0 2024-09-10T09:00:19.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.810+0000 7f9ad9ffb640 10 client.28163 handle_command_reply: tid=0 2024-09-10T09:00:19.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.810+0000 7f9afaeec740 1 client.28163 shutdown 2024-09-10T09:00:19.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.810+0000 7f9afaeec740 1 -- 172.21.15.172:0/2923995681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 msgr2=0x55fd8ef42da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:19.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.810+0000 7f9afaeec740 1 --2- 172.21.15.172:0/2923995681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef42da0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9adc019b60 tx=0x7f9adc000bc0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.810+0000 7f9afaeec740 1 -- 172.21.15.172:0/2923995681 shutdown_connections 2024-09-10T09:00:19.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.810+0000 7f9afaeec740 1 --2- 172.21.15.172:0/2923995681 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fd8efd3ad0 0x55fd8efd5ec0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.810+0000 7f9afaeec740 1 --2- 172.21.15.172:0/2923995681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fd8ef56a00 0x55fd8efc0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.810+0000 7f9afaeec740 1 --2- 172.21.15.172:0/2923995681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fd8ef3e940 0x55fd8ef432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.697 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:00:19.697 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 42, 2024-09-10T09:00:19.697 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T09:00:19.697 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 46, 2024-09-10T09:00:19.697 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T09:00:19.698 INFO:teuthology.orchestra.run.smithi172.stdout: "7787905e": { 2024-09-10T09:00:19.698 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 46, 2024-09-10T09:00:19.698 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T09:00:19.698 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T09:00:19.698 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T09:00:19.698 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T09:00:19.698 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:00:19.698 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T09:00:19.698 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 141.5, 2024-09-10T09:00:19.698 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T09:00:19.698 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T09:00:19.698 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T09:00:19.698 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T09:00:19.699 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T09:00:19.699 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T09:00:19.699 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:00:19.699 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:00:19.699 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:00:19.699 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:00:19.699 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:00:19.699 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:00:19.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.810+0000 7f9afaeec740 1 --2- 172.21.15.172:0/2923995681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd8ef34080 0x55fd8ef42da0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.810+0000 7f9afaeec740 1 -- 172.21.15.172:0/2923995681 >> 172.21.15.172:0/2923995681 conn(0x55fd8ef2fcf0 msgr2=0x55fd8ef2e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:19.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.810+0000 7f9ad9ffb640 0 client.28163 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:19.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.811+0000 7f9afaeec740 1 -- 172.21.15.172:0/2923995681 shutdown_connections 2024-09-10T09:00:19.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.811+0000 7f9afaeec740 1 -- 172.21.15.172:0/2923995681 wait complete. 2024-09-10T09:00:19.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.811+0000 7f9afaeec740 20 client.28163 trim_cache size 0 max 16384 2024-09-10T09:00:19.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.812+0000 7f9af8b13640 1 -- 172.21.15.172:0/3953887194 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9ae00a2b80 msgr2=0x7f9ae00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:19.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.812+0000 7f9af8b13640 1 --2- 172.21.15.172:0/3953887194 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9ae00a2b80 0x7f9ae00a5040 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0979a0 tx=0x7f9aec016b40 comp rx=0 tx=0).stop 2024-09-10T09:00:19.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.812+0000 7f9af8b13640 1 -- 172.21.15.172:0/3953887194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9af41082a0 msgr2=0x7f9af419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:19.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.812+0000 7f9af8b13640 1 --2- 172.21.15.172:0/3953887194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9af41082a0 0x7f9af419ce40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009fd0 tx=0x7f9aec00def0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.812+0000 7f9af8b13640 1 -- 172.21.15.172:0/3953887194 shutdown_connections 2024-09-10T09:00:19.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.812+0000 7f9af8b13640 1 --2- 172.21.15.172:0/3953887194 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9ae00a2b80 0x7f9ae00a5040 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.812+0000 7f9af8b13640 1 --2- 172.21.15.172:0/3953887194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9af41082a0 0x7f9af419ce40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.812+0000 7f9af8b13640 1 --2- 172.21.15.172:0/3953887194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9af41036e0 0x7f9af4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.812+0000 7f9af8b13640 1 --2- 172.21.15.172:0/3953887194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9af4102d30 0x7f9af4198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:19.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.812+0000 7f9af8b13640 1 -- 172.21.15.172:0/3953887194 >> 172.21.15.172:0/3953887194 conn(0x7f9af40fc7d0 msgr2=0x7f9af4105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:19.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.812+0000 7f9af8b13640 1 -- 172.21.15.172:0/3953887194 shutdown_connections 2024-09-10T09:00:19.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:18.812+0000 7f9af8b13640 1 -- 172.21.15.172:0/3953887194 wait complete. 2024-09-10T09:00:19.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:19 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:00:19.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:19 smithi191 ceph-mon[35314]: pgmap v773: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-10T09:00:19.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:19 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:00:19.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:19 smithi172 ceph-mon[30854]: pgmap v773: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-10T09:00:19.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:19 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:00:19.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:19 smithi174 ceph-mon[34068]: pgmap v773: 129 pgs: 129 active+clean; 13 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-10T09:00:21.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:21 smithi174 ceph-mon[34068]: pgmap v774: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 3.22k op/s 2024-09-10T09:00:21.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:21 smithi191 ceph-mon[35314]: pgmap v774: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 3.22k op/s 2024-09-10T09:00:21.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:21 smithi172 ceph-mon[30854]: pgmap v774: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 MiB/s wr, 3.22k op/s 2024-09-10T09:00:22.689 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:00:23.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:23 smithi174 ceph-mon[34068]: pgmap v775: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 2.71k op/s 2024-09-10T09:00:23.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:23 smithi191 ceph-mon[35314]: pgmap v775: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 2.71k op/s 2024-09-10T09:00:23.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:23 smithi172 ceph-mon[30854]: pgmap v775: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 2.71k op/s 2024-09-10T09:00:25.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:25 smithi174 ceph-mon[34068]: pgmap v776: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.9 MiB/s wr, 2.54k op/s 2024-09-10T09:00:25.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:25 smithi191 ceph-mon[35314]: pgmap v776: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.9 MiB/s wr, 2.54k op/s 2024-09-10T09:00:25.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:25 smithi172 ceph-mon[30854]: pgmap v776: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.9 MiB/s wr, 2.54k op/s 2024-09-10T09:00:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.583+0000 7f1e657af640 1 -- 172.21.15.172:0/774639384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e601037b0 msgr2=0x7f1e6010a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.583+0000 7f1e657af640 1 --2- 172.21.15.172:0/774639384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e601037b0 0x7f1e6010a040 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009ca0 tx=0x7f1e5802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.583+0000 7f1e657af640 1 -- 172.21.15.172:0/774639384 shutdown_connections 2024-09-10T09:00:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.583+0000 7f1e657af640 1 --2- 172.21.15.172:0/774639384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e601037b0 0x7f1e6010a040 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.583+0000 7f1e657af640 1 --2- 172.21.15.172:0/774639384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e60102df0 0x7f1e60103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.583+0000 7f1e657af640 1 --2- 172.21.15.172:0/774639384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e600ffe20 0x7f1e60100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.583+0000 7f1e657af640 1 -- 172.21.15.172:0/774639384 >> 172.21.15.172:0/774639384 conn(0x7f1e600fb980 msgr2=0x7f1e600fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.583+0000 7f1e657af640 1 -- 172.21.15.172:0/774639384 shutdown_connections 2024-09-10T09:00:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.583+0000 7f1e657af640 1 -- 172.21.15.172:0/774639384 wait complete. 2024-09-10T09:00:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e657af640 1 Processor -- start 2024-09-10T09:00:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e657af640 1 -- start start 2024-09-10T09:00:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e657af640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e600ffe20 0x7f1e60102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e657af640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e60102df0 0x7f1e60100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e657af640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e601037b0 0x7f1e601012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e657af640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6010d680 con 0x7f1e601037b0 2024-09-10T09:00:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e657af640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6010d500 con 0x7f1e60102df0 2024-09-10T09:00:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e657af640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6010d800 con 0x7f1e600ffe20 2024-09-10T09:00:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e5f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e601037b0 0x7f1e601012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e5f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e601037b0 0x7f1e601012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33702/0 (socket says 172.21.15.172:33702) 2024-09-10T09:00:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e5f7fe640 1 -- 172.21.15.172:0/1093955828 learned_addr learned my addr 172.21.15.172:0/1093955828 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e5effd640 1 --2- 172.21.15.172:0/1093955828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e600ffe20 0x7f1e60102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e57fff640 1 --2- 172.21.15.172:0/1093955828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e60102df0 0x7f1e60100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e5f7fe640 1 -- 172.21.15.172:0/1093955828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e600ffe20 msgr2=0x7f1e60102760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e5f7fe640 1 --2- 172.21.15.172:0/1093955828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e600ffe20 0x7f1e60102760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e5f7fe640 1 -- 172.21.15.172:0/1093955828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e60102df0 msgr2=0x7f1e60100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e5f7fe640 1 --2- 172.21.15.172:0/1093955828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e60102df0 0x7f1e60100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e5f7fe640 1 -- 172.21.15.172:0/1093955828 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e60101b70 con 0x7f1e601037b0 2024-09-10T09:00:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e5effd640 1 --2- 172.21.15.172:0/1093955828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e600ffe20 0x7f1e60102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:00:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.584+0000 7f1e57fff640 1 --2- 172.21.15.172:0/1093955828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e60102df0 0x7f1e60100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:00:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.585+0000 7f1e5f7fe640 1 --2- 172.21.15.172:0/1093955828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e601037b0 0x7f1e601012e0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f1e58031970 tx=0x7f1e58003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.585+0000 7f1e5cff9640 1 -- 172.21.15.172:0/1093955828 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58004040 con 0x7f1e601037b0 2024-09-10T09:00:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.585+0000 7f1e5cff9640 1 -- 172.21.15.172:0/1093955828 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e58004690 con 0x7f1e601037b0 2024-09-10T09:00:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.585+0000 7f1e5cff9640 1 -- 172.21.15.172:0/1093955828 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5803c410 con 0x7f1e601037b0 2024-09-10T09:00:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.585+0000 7f1e657af640 1 -- 172.21.15.172:0/1093955828 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e601a1d00 con 0x7f1e601037b0 2024-09-10T09:00:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.585+0000 7f1e657af640 1 -- 172.21.15.172:0/1093955828 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e601a21e0 con 0x7f1e601037b0 2024-09-10T09:00:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.588+0000 7f1e5cff9640 1 -- 172.21.15.172:0/1093955828 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1e580305c0 con 0x7f1e601037b0 2024-09-10T09:00:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.588+0000 7f1e68624740 1 Processor -- start 2024-09-10T09:00:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.588+0000 7f1e68624740 1 -- start start 2024-09-10T09:00:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.588+0000 7f1e68624740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec8069a0 0x55b6ec806da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.588+0000 7f1e68624740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec8072e0 0x55b6ec80bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.588+0000 7f1e68624740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec80c370 0x55b6ec80e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.588+0000 7f1e68624740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b6ec76a2f0 con 0x55b6ec8069a0 2024-09-10T09:00:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.588+0000 7f1e68624740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b6ec712270 con 0x55b6ec8072e0 2024-09-10T09:00:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.588+0000 7f1e68624740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b6ec7662b0 con 0x55b6ec80c370 2024-09-10T09:00:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.588+0000 7f1e57fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec8069a0 0x55b6ec806da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.589+0000 7f1e57fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec8069a0 0x55b6ec806da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33714/0 (socket says 172.21.15.172:33714) 2024-09-10T09:00:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.589+0000 7f1e57fff640 1 -- 172.21.15.172:0/1161986582 learned_addr learned my addr 172.21.15.172:0/1161986582 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.589+0000 7f1e5cff9640 1 --2- 172.21.15.172:0/1093955828 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1e4c0a2cb0 0x7f1e4c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.589+0000 7f1e57fff640 1 --2- 172.21.15.172:0/1161986582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec80c370 0x55b6ec80e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.589+0000 7f1e5effd640 1 --2- 172.21.15.172:0/1161986582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec8072e0 0x55b6ec80bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.589+0000 7f1e57fff640 1 -- 172.21.15.172:0/1161986582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec80c370 msgr2=0x55b6ec80e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.589+0000 7f1e57fff640 1 --2- 172.21.15.172:0/1161986582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec80c370 0x55b6ec80e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.589+0000 7f1e57fff640 1 -- 172.21.15.172:0/1161986582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec8072e0 msgr2=0x55b6ec80bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.589+0000 7f1e57fff640 1 --2- 172.21.15.172:0/1161986582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec8072e0 0x55b6ec80bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.589+0000 7f1e57fff640 1 -- 172.21.15.172:0/1161986582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ec711340 con 0x55b6ec8069a0 2024-09-10T09:00:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.589+0000 7f1e5cff9640 1 -- 172.21.15.172:0/1093955828 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1e580f0bd0 con 0x7f1e601037b0 2024-09-10T09:00:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.589+0000 7f1e5f7fe640 1 --2- 172.21.15.172:0/1093955828 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1e4c0a2cb0 0x7f1e4c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.589+0000 7f1e57fff640 1 --2- 172.21.15.172:0/1161986582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec8069a0 0x55b6ec806da0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f1e48014570 tx=0x7f1e4800b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf9a03447264ecc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.589+0000 7f1e557fa640 1 -- 172.21.15.172:0/1161986582 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48005220 con 0x55b6ec8069a0 2024-09-10T09:00:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.589+0000 7f1e557fa640 1 -- 172.21.15.172:0/1161986582 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e48005e80 con 0x55b6ec8069a0 2024-09-10T09:00:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.589+0000 7f1e557fa640 1 -- 172.21.15.172:0/1161986582 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48012770 con 0x55b6ec8069a0 2024-09-10T09:00:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.590+0000 7f1e68624740 1 -- 172.21.15.172:0/1161986582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec8069a0 msgr2=0x55b6ec806da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.590+0000 7f1e68624740 1 --2- 172.21.15.172:0/1161986582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec8069a0 0x55b6ec806da0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f1e48014570 tx=0x7f1e4800b5d0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.590+0000 7f1e5f7fe640 1 --2- 172.21.15.172:0/1093955828 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1e4c0a2cb0 0x7f1e4c0a5170 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1e58005e30 tx=0x7f1e5802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.590+0000 7f1e68624740 1 -- 172.21.15.172:0/1161986582 shutdown_connections 2024-09-10T09:00:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.590+0000 7f1e68624740 1 --2- 172.21.15.172:0/1161986582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec80c370 0x55b6ec80e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.590+0000 7f1e68624740 1 --2- 172.21.15.172:0/1161986582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec8072e0 0x55b6ec80bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.590+0000 7f1e68624740 1 --2- 172.21.15.172:0/1161986582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec8069a0 0x55b6ec806da0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.590+0000 7f1e68624740 1 -- 172.21.15.172:0/1161986582 >> 172.21.15.172:0/1161986582 conn(0x55b6ec802010 msgr2=0x55b6ec802410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.590+0000 7f1e68624740 1 -- 172.21.15.172:0/1161986582 shutdown_connections 2024-09-10T09:00:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.590+0000 7f1e68624740 1 -- 172.21.15.172:0/1161986582 wait complete. 2024-09-10T09:00:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e68624740 1 -- start start 2024-09-10T09:00:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e68624740 1 Processor -- start 2024-09-10T09:00:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e68624740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec8069a0 0x55b6ec810ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e68624740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec8072e0 0x55b6ec8110e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e68624740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec80c370 0x55b6ec818120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e68624740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b6ec80f340 con 0x55b6ec8072e0 2024-09-10T09:00:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e68624740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b6ec6f3290 con 0x55b6ec80c370 2024-09-10T09:00:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e68624740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b6ec80f4c0 con 0x55b6ec8069a0 2024-09-10T09:00:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e68624740 10 client.0 _ll_register_callbacks cb 0x55b6ec724140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e5effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec8072e0 0x55b6ec8110e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e5effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec8072e0 0x55b6ec8110e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33720/0 (socket says 172.21.15.172:33720) 2024-09-10T09:00:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e5effd640 1 -- 172.21.15.172:0/1981719551 learned_addr learned my addr 172.21.15.172:0/1981719551 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e57fff640 1 --2- 172.21.15.172:0/1981719551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec80c370 0x55b6ec818120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e5effd640 1 -- 172.21.15.172:0/1981719551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec8069a0 msgr2=0x55b6ec810ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e5effd640 1 --2- 172.21.15.172:0/1981719551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec8069a0 0x55b6ec810ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e5effd640 1 -- 172.21.15.172:0/1981719551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec80c370 msgr2=0x55b6ec818120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e5effd640 1 --2- 172.21.15.172:0/1981719551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec80c370 0x55b6ec818120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e5effd640 1 -- 172.21.15.172:0/1981719551 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ec47d210 con 0x55b6ec8072e0 2024-09-10T09:00:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e57fff640 1 --2- 172.21.15.172:0/1981719551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec80c370 0x55b6ec818120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:00:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e5effd640 1 --2- 172.21.15.172:0/1981719551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec8072e0 0x55b6ec8110e0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000bf00 tx=0x7f1e500117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b24221f75e9b1c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.591+0000 7f1e467fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:00:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.592+0000 7f1e467fc640 1 -- 172.21.15.172:0/1981719551 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e50004220 con 0x55b6ec8072e0 2024-09-10T09:00:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.592+0000 7f1e467fc640 1 -- 172.21.15.172:0/1981719551 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1e500043c0 con 0x55b6ec8072e0 2024-09-10T09:00:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.592+0000 7f1e68624740 1 -- 172.21.15.172:0/1981719551 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b6ec777a20 con 0x55b6ec8072e0 2024-09-10T09:00:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.592+0000 7f1e467fc640 1 -- 172.21.15.172:0/1981719551 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1e50005020 con 0x55b6ec8072e0 2024-09-10T09:00:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.592+0000 7f1e68624740 10 client.19395 fetch_fsmap learned FSMap version 42 2024-09-10T09:00:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.592+0000 7f1e68624740 1 -- 172.21.15.172:0/1981719551 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b6ec549620 con 0x55b6ec8072e0 2024-09-10T09:00:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.592+0000 7f1e467fc640 1 -- 172.21.15.172:0/1981719551 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b6ec777a20 con 0x55b6ec8072e0 2024-09-10T09:00:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.592+0000 7f1e467fc640 1 -- 172.21.15.172:0/1981719551 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7f1e500035f0 con 0x55b6ec8072e0 2024-09-10T09:00:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.592+0000 7f1e68624740 10 client.19395 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:00:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.592+0000 7f1e68624740 10 client.19395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:00:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.592+0000 7f1e68624740 20 client.19395 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.592+0000 7f1e68624740 1 --2- 172.21.15.172:0/1981719551 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b6ec81c950 0x55b6ec81eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.592+0000 7f1e68624740 4 client.19395 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:00:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.592+0000 7f1e68624740 1 -- 172.21.15.172:0/1981719551 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6ec771d70 con 0x55b6ec81c950 2024-09-10T09:00:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.593+0000 7f1e57fff640 1 --2- 172.21.15.172:0/1981719551 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b6ec81c950 0x55b6ec81eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.593+0000 7f1e57fff640 1 --2- 172.21.15.172:0/1981719551 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b6ec81c950 0x55b6ec81eda0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54a0e581cbf53afc server_cookie=3bc49468dec561f5 in_seq=0 out_seq=0 2024-09-10T09:00:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.593+0000 7f1e467fc640 10 client.19395 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.594+0000 7f1e467fc640 1 -- 172.21.15.172:0/1981719551 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6ec771d70 con 0x55b6ec81c950 2024-09-10T09:00:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.594+0000 7f1e467fc640 10 client.19395 handle_command_reply: tid=0 2024-09-10T09:00:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.594+0000 7f1e68624740 1 client.19395 shutdown 2024-09-10T09:00:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.594+0000 7f1e68624740 1 -- 172.21.15.172:0/1981719551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec8072e0 msgr2=0x55b6ec8110e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.594+0000 7f1e68624740 1 --2- 172.21.15.172:0/1981719551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec8072e0 0x55b6ec8110e0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f1e5000bf00 tx=0x7f1e500117b0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.594+0000 7f1e68624740 1 -- 172.21.15.172:0/1981719551 shutdown_connections 2024-09-10T09:00:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.594+0000 7f1e68624740 1 --2- 172.21.15.172:0/1981719551 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b6ec81c950 0x55b6ec81eda0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.594+0000 7f1e68624740 1 --2- 172.21.15.172:0/1981719551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec80c370 0x55b6ec818120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.594+0000 7f1e68624740 1 --2- 172.21.15.172:0/1981719551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec8072e0 0x55b6ec8110e0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.594+0000 7f1e467fc640 0 client.19395 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.594+0000 7f1e68624740 1 --2- 172.21.15.172:0/1981719551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec8069a0 0x55b6ec810ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.594+0000 7f1e68624740 1 -- 172.21.15.172:0/1981719551 >> 172.21.15.172:0/1981719551 conn(0x55b6ec802010 msgr2=0x55b6ec803820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.595+0000 7f1e68624740 1 -- 172.21.15.172:0/1981719551 shutdown_connections 2024-09-10T09:00:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.595+0000 7f1e68624740 1 -- 172.21.15.172:0/1981719551 wait complete. 2024-09-10T09:00:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.595+0000 7f1e68624740 20 client.19395 trim_cache size 0 max 16384 2024-09-10T09:00:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.609+0000 7f1e68624740 1 Processor -- start 2024-09-10T09:00:26.610 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.609+0000 7f1e68624740 1 -- start start 2024-09-10T09:00:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.610+0000 7f1e68624740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec8069a0 0x55b6ec8010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.610+0000 7f1e68624740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec8072e0 0x55b6ec8180e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.610+0000 7f1e68624740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec80c370 0x55b6ec803400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.610+0000 7f1e68624740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601ac5c0 con 0x55b6ec80c370 2024-09-10T09:00:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.610+0000 7f1e68624740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601ae160 con 0x55b6ec8072e0 2024-09-10T09:00:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.610+0000 7f1e68624740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601acc90 con 0x55b6ec8069a0 2024-09-10T09:00:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.610+0000 7f1e57fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec80c370 0x55b6ec803400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.610+0000 7f1e57fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec80c370 0x55b6ec803400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33732/0 (socket says 172.21.15.172:33732) 2024-09-10T09:00:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.610+0000 7f1e57fff640 1 -- 172.21.15.172:0/2791556198 learned_addr learned my addr 172.21.15.172:0/2791556198 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.610+0000 7f1e57fff640 1 --2- 172.21.15.172:0/2791556198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec8069a0 0x55b6ec8010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.610+0000 7f1e5effd640 1 --2- 172.21.15.172:0/2791556198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec8072e0 0x55b6ec8180e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:26.612 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.610+0000 7f1e57fff640 1 -- 172.21.15.172:0/2791556198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec8069a0 msgr2=0x55b6ec8010b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.610+0000 7f1e57fff640 1 --2- 172.21.15.172:0/2791556198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec8069a0 0x55b6ec8010b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.610+0000 7f1e57fff640 1 -- 172.21.15.172:0/2791556198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec8072e0 msgr2=0x55b6ec8180e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.610+0000 7f1e57fff640 1 --2- 172.21.15.172:0/2791556198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec8072e0 0x55b6ec8180e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.610+0000 7f1e57fff640 1 -- 172.21.15.172:0/2791556198 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ec807f10 con 0x55b6ec80c370 2024-09-10T09:00:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.610+0000 7f1e57fff640 1 --2- 172.21.15.172:0/2791556198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec80c370 0x55b6ec803400 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1e480056b0 tx=0x7f1e48004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0830735f297c58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.611+0000 7f1e54ff9640 1 -- 172.21.15.172:0/2791556198 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48004860 con 0x55b6ec80c370 2024-09-10T09:00:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.611+0000 7f1e54ff9640 1 -- 172.21.15.172:0/2791556198 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e48005380 con 0x55b6ec80c370 2024-09-10T09:00:26.613 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.611+0000 7f1e54ff9640 1 -- 172.21.15.172:0/2791556198 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48029bb0 con 0x55b6ec80c370 2024-09-10T09:00:26.616 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.611+0000 7f1e68624740 1 -- 172.21.15.172:0/2791556198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec80c370 msgr2=0x55b6ec803400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.611+0000 7f1e68624740 1 --2- 172.21.15.172:0/2791556198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec80c370 0x55b6ec803400 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1e480056b0 tx=0x7f1e48004ed0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.611+0000 7f1e68624740 1 -- 172.21.15.172:0/2791556198 shutdown_connections 2024-09-10T09:00:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.611+0000 7f1e68624740 1 --2- 172.21.15.172:0/2791556198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec80c370 0x55b6ec803400 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.611+0000 7f1e68624740 1 --2- 172.21.15.172:0/2791556198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec8072e0 0x55b6ec8180e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.611+0000 7f1e68624740 1 --2- 172.21.15.172:0/2791556198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec8069a0 0x55b6ec8010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.611+0000 7f1e68624740 1 -- 172.21.15.172:0/2791556198 >> 172.21.15.172:0/2791556198 conn(0x55b6ec802010 msgr2=0x55b6ec8009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e68624740 1 -- 172.21.15.172:0/2791556198 shutdown_connections 2024-09-10T09:00:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e68624740 1 -- 172.21.15.172:0/2791556198 wait complete. 2024-09-10T09:00:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e68624740 1 -- start start 2024-09-10T09:00:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e68624740 1 Processor -- start 2024-09-10T09:00:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e68624740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec8069a0 0x55b6ec8159b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e68624740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec8072e0 0x55b6ec815ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e68624740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec80c370 0x55b6ec816f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e68624740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601ada90 con 0x55b6ec80c370 2024-09-10T09:00:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e68624740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601ad3c0 con 0x55b6ec8072e0 2024-09-10T09:00:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e68624740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1e601aa5a0 con 0x55b6ec8069a0 2024-09-10T09:00:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e68624740 10 client.0 _ll_register_callbacks cb 0x7f1e5000b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e57fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec80c370 0x55b6ec816f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e57fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec80c370 0x55b6ec816f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33748/0 (socket says 172.21.15.172:33748) 2024-09-10T09:00:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e57fff640 1 -- 172.21.15.172:0/222636997 learned_addr learned my addr 172.21.15.172:0/222636997 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e5effd640 1 --2- 172.21.15.172:0/222636997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec8072e0 0x55b6ec815ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e57fff640 1 -- 172.21.15.172:0/222636997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec8069a0 msgr2=0x55b6ec8159b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e57fff640 1 --2- 172.21.15.172:0/222636997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec8069a0 0x55b6ec8159b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e57fff640 1 -- 172.21.15.172:0/222636997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec8072e0 msgr2=0x55b6ec815ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e57fff640 1 --2- 172.21.15.172:0/222636997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec8072e0 0x55b6ec815ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e57fff640 1 -- 172.21.15.172:0/222636997 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ec817510 con 0x55b6ec80c370 2024-09-10T09:00:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.612+0000 7f1e5effd640 1 --2- 172.21.15.172:0/222636997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec8072e0 0x55b6ec815ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:00:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.613+0000 7f1e57fff640 1 --2- 172.21.15.172:0/222636997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec80c370 0x55b6ec816f00 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f1e480057a0 tx=0x7f1e48005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f408eff71db036 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.613+0000 7f1e467fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:00:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.613+0000 7f1e467fc640 1 -- 172.21.15.172:0/222636997 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48005080 con 0x55b6ec80c370 2024-09-10T09:00:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.613+0000 7f1e467fc640 1 -- 172.21.15.172:0/222636997 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1e48029410 con 0x55b6ec80c370 2024-09-10T09:00:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.613+0000 7f1e68624740 1 -- 172.21.15.172:0/222636997 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b6ec6ee900 con 0x55b6ec80c370 2024-09-10T09:00:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.613+0000 7f1e467fc640 1 -- 172.21.15.172:0/222636997 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1e48029930 con 0x55b6ec80c370 2024-09-10T09:00:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.613+0000 7f1e68624740 10 client.19401 fetch_fsmap learned FSMap version 42 2024-09-10T09:00:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.613+0000 7f1e68624740 1 -- 172.21.15.172:0/222636997 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b6ec8877c0 con 0x55b6ec80c370 2024-09-10T09:00:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.613+0000 7f1e467fc640 1 -- 172.21.15.172:0/222636997 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b6ec6ee900 con 0x55b6ec80c370 2024-09-10T09:00:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.613+0000 7f1e467fc640 1 -- 172.21.15.172:0/222636997 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7f1e48012510 con 0x55b6ec80c370 2024-09-10T09:00:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.613+0000 7f1e68624740 10 client.19401 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:00:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.613+0000 7f1e68624740 10 client.19401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:00:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.613+0000 7f1e68624740 20 client.19401 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.613+0000 7f1e68624740 1 --2- 172.21.15.172:0/222636997 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b6ec887ef0 0x55b6ec88a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.613+0000 7f1e68624740 4 client.19401 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:00:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.613+0000 7f1e68624740 1 -- 172.21.15.172:0/222636997 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b6ec73b820 con 0x55b6ec887ef0 2024-09-10T09:00:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.614+0000 7f1e5effd640 1 --2- 172.21.15.172:0/222636997 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b6ec887ef0 0x55b6ec88a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.614+0000 7f1e5effd640 1 --2- 172.21.15.172:0/222636997 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b6ec887ef0 0x55b6ec88a310 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=158d42d9840af721 server_cookie=24838eb1e00f8ec in_seq=0 out_seq=0 2024-09-10T09:00:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.614+0000 7f1e467fc640 10 client.19401 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.622+0000 7f1e467fc640 1 -- 172.21.15.172:0/222636997 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b6ec73b820 con 0x55b6ec887ef0 2024-09-10T09:00:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.622+0000 7f1e467fc640 10 client.19401 handle_command_reply: tid=0 2024-09-10T09:00:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.622+0000 7f1e68624740 1 client.19401 shutdown 2024-09-10T09:00:26.623 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.622+0000 7f1e68624740 1 -- 172.21.15.172:0/222636997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec80c370 msgr2=0x55b6ec816f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.622+0000 7f1e68624740 1 --2- 172.21.15.172:0/222636997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec80c370 0x55b6ec816f00 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f1e480057a0 tx=0x7f1e48005630 comp rx=0 tx=0).stop 2024-09-10T09:00:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.622+0000 7f1e68624740 1 -- 172.21.15.172:0/222636997 shutdown_connections 2024-09-10T09:00:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.622+0000 7f1e68624740 1 --2- 172.21.15.172:0/222636997 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b6ec887ef0 0x55b6ec88a310 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.622+0000 7f1e68624740 1 --2- 172.21.15.172:0/222636997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6ec80c370 0x55b6ec816f00 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.622+0000 7f1e68624740 1 --2- 172.21.15.172:0/222636997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ec8072e0 0x55b6ec815ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.622+0000 7f1e68624740 1 --2- 172.21.15.172:0/222636997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ec8069a0 0x55b6ec8159b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.622+0000 7f1e467fc640 0 client.19401 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.622+0000 7f1e68624740 1 -- 172.21.15.172:0/222636997 >> 172.21.15.172:0/222636997 conn(0x55b6ec802010 msgr2=0x55b6ec8011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.622+0000 7f1e68624740 1 -- 172.21.15.172:0/222636997 shutdown_connections 2024-09-10T09:00:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.622+0000 7f1e68624740 1 -- 172.21.15.172:0/222636997 wait complete. 2024-09-10T09:00:26.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.622+0000 7f1e68624740 20 client.19401 trim_cache size 0 max 16384 2024-09-10T09:00:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:00:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:00:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:00:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:00:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.623+0000 7f1e657af640 1 -- 172.21.15.172:0/1093955828 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1e4c0a2cb0 msgr2=0x7f1e4c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.623+0000 7f1e657af640 1 --2- 172.21.15.172:0/1093955828 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1e4c0a2cb0 0x7f1e4c0a5170 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1e58005e30 tx=0x7f1e5802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.623+0000 7f1e657af640 1 -- 172.21.15.172:0/1093955828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e601037b0 msgr2=0x7f1e601012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.623+0000 7f1e657af640 1 --2- 172.21.15.172:0/1093955828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e601037b0 0x7f1e601012e0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f1e58031970 tx=0x7f1e58003ba0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.623+0000 7f1e657af640 1 -- 172.21.15.172:0/1093955828 shutdown_connections 2024-09-10T09:00:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.623+0000 7f1e657af640 1 --2- 172.21.15.172:0/1093955828 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1e4c0a2cb0 0x7f1e4c0a5170 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.623+0000 7f1e657af640 1 --2- 172.21.15.172:0/1093955828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e601037b0 0x7f1e601012e0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.623+0000 7f1e657af640 1 --2- 172.21.15.172:0/1093955828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e60102df0 0x7f1e60100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.623+0000 7f1e657af640 1 --2- 172.21.15.172:0/1093955828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e600ffe20 0x7f1e60102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.623+0000 7f1e657af640 1 -- 172.21.15.172:0/1093955828 >> 172.21.15.172:0/1093955828 conn(0x7f1e600fb980 msgr2=0x7f1e601060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.623+0000 7f1e657af640 1 -- 172.21.15.172:0/1093955828 shutdown_connections 2024-09-10T09:00:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:26.623+0000 7f1e657af640 1 -- 172.21.15.172:0/1093955828 wait complete. 2024-09-10T09:00:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:27 smithi174 ceph-mon[34068]: pgmap v777: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 1.47k op/s 2024-09-10T09:00:27.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:27 smithi191 ceph-mon[35314]: pgmap v777: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 1.47k op/s 2024-09-10T09:00:27.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:27 smithi172 ceph-mon[30854]: pgmap v777: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 1.47k op/s 2024-09-10T09:00:29.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:29 smithi172 ceph-mon[30854]: pgmap v778: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.5 MiB/s wr, 1.16k op/s 2024-09-10T09:00:29.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:29 smithi174 ceph-mon[34068]: pgmap v778: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.5 MiB/s wr, 1.16k op/s 2024-09-10T09:00:29.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:29 smithi191 ceph-mon[35314]: pgmap v778: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.5 MiB/s wr, 1.16k op/s 2024-09-10T09:00:31.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:31 smithi174 ceph-mon[34068]: pgmap v779: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.4 MiB/s wr, 627 op/s 2024-09-10T09:00:31.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:31 smithi191 ceph-mon[35314]: pgmap v779: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.4 MiB/s wr, 627 op/s 2024-09-10T09:00:31.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:31 smithi172 ceph-mon[30854]: pgmap v779: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.4 MiB/s wr, 627 op/s 2024-09-10T09:00:32.238 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7787905e', quiesce took 71.0 seconds. Will release after: -49.5 2024-09-10T09:00:32.239 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:00:33.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:33 smithi174 ceph-mon[34068]: pgmap v780: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 443 KiB/s wr, 121 op/s 2024-09-10T09:00:33.786 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:33 smithi172 ceph-mon[30854]: pgmap v780: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 443 KiB/s wr, 121 op/s 2024-09-10T09:00:33.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:33 smithi191 ceph-mon[35314]: pgmap v780: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 443 KiB/s wr, 121 op/s 2024-09-10T09:00:34.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:34 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/498772001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:00:34.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:34 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/498772001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:00:34.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:34 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/498772001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:00:34.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.570+0000 7f0431a9e640 1 -- 172.21.15.172:0/2292246605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c1036e0 msgr2=0x7f042c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:34.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.570+0000 7f0431a9e640 1 --2- 172.21.15.172:0/2292246605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c1036e0 0x7f042c107b90 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f0414009930 tx=0x7f041402a420 comp rx=0 tx=0).stop 2024-09-10T09:00:34.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.570+0000 7f0431a9e640 1 -- 172.21.15.172:0/2292246605 shutdown_connections 2024-09-10T09:00:34.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.570+0000 7f0431a9e640 1 --2- 172.21.15.172:0/2292246605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f042c1082a0 0x7f042c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:34.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.570+0000 7f0431a9e640 1 --2- 172.21.15.172:0/2292246605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c1036e0 0x7f042c107b90 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:34.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.570+0000 7f0431a9e640 1 --2- 172.21.15.172:0/2292246605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f042c102d30 0x7f042c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:34.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.570+0000 7f0431a9e640 1 -- 172.21.15.172:0/2292246605 >> 172.21.15.172:0/2292246605 conn(0x7f042c0fc7d0 msgr2=0x7f042c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:34.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.570+0000 7f0431a9e640 1 -- 172.21.15.172:0/2292246605 shutdown_connections 2024-09-10T09:00:34.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.570+0000 7f0431a9e640 1 -- 172.21.15.172:0/2292246605 wait complete. 2024-09-10T09:00:34.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.571+0000 7f0431a9e640 1 Processor -- start 2024-09-10T09:00:34.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.571+0000 7f0431a9e640 1 -- start start 2024-09-10T09:00:34.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.571+0000 7f0431a9e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f042c102d30 0x7f042c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:34.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.571+0000 7f0431a9e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f042c1036e0 0x7f042c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:34.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.571+0000 7f0431a9e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c1082a0 0x7f042c19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:34.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.571+0000 7f0431a9e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f042c111f50 con 0x7f042c1082a0 2024-09-10T09:00:34.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.571+0000 7f0431a9e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f042c111dd0 con 0x7f042c1036e0 2024-09-10T09:00:34.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.571+0000 7f0431a9e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f042c1120d0 con 0x7f042c102d30 2024-09-10T09:00:34.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.571+0000 7f042bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c1082a0 0x7f042c19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:34.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.571+0000 7f042bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c1082a0 0x7f042c19ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:33776/0 (socket says 172.21.15.172:33776) 2024-09-10T09:00:34.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.571+0000 7f042bfff640 1 -- 172.21.15.172:0/498772001 learned_addr learned my addr 172.21.15.172:0/498772001 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:34.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.571+0000 7f042b7fe640 1 --2- 172.21.15.172:0/498772001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f042c102d30 0x7f042c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:34.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.571+0000 7f0423fff640 1 --2- 172.21.15.172:0/498772001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f042c1036e0 0x7f042c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:34.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.572+0000 7f042bfff640 1 -- 172.21.15.172:0/498772001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f042c102d30 msgr2=0x7f042c198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:34.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.572+0000 7f042bfff640 1 --2- 172.21.15.172:0/498772001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f042c102d30 0x7f042c198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:34.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.572+0000 7f042bfff640 1 -- 172.21.15.172:0/498772001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f042c1036e0 msgr2=0x7f042c198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:34.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.572+0000 7f042bfff640 1 --2- 172.21.15.172:0/498772001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f042c1036e0 0x7f042c198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:34.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.572+0000 7f042bfff640 1 -- 172.21.15.172:0/498772001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042c19d530 con 0x7f042c1082a0 2024-09-10T09:00:34.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.572+0000 7f042b7fe640 1 --2- 172.21.15.172:0/498772001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f042c102d30 0x7f042c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:00:34.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.572+0000 7f042bfff640 1 --2- 172.21.15.172:0/498772001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c1082a0 0x7f042c19ce50 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f042400cad0 tx=0x7f042400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:34.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.572+0000 7f04297fa640 1 -- 172.21.15.172:0/498772001 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042400ae20 con 0x7f042c1082a0 2024-09-10T09:00:34.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.572+0000 7f0431a9e640 1 -- 172.21.15.172:0/498772001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042c19d820 con 0x7f042c1082a0 2024-09-10T09:00:34.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.572+0000 7f04297fa640 1 -- 172.21.15.172:0/498772001 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0424005020 con 0x7f042c1082a0 2024-09-10T09:00:34.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.572+0000 7f04297fa640 1 -- 172.21.15.172:0/498772001 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042400f6b0 con 0x7f042c1082a0 2024-09-10T09:00:34.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.572+0000 7f0431a9e640 1 -- 172.21.15.172:0/498772001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f042c1a5100 con 0x7f042c1082a0 2024-09-10T09:00:34.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.573+0000 7f0431a9e640 1 -- 172.21.15.172:0/498772001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f042c103110 con 0x7f042c1082a0 2024-09-10T09:00:34.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.574+0000 7f04297fa640 1 -- 172.21.15.172:0/498772001 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f04240051d0 con 0x7f042c1082a0 2024-09-10T09:00:34.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.575+0000 7f04297fa640 1 --2- 172.21.15.172:0/498772001 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04180a2b80 0x7f04180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:34.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.575+0000 7f042b7fe640 1 --2- 172.21.15.172:0/498772001 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04180a2b80 0x7f04180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:34.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.575+0000 7f04297fa640 1 -- 172.21.15.172:0/498772001 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f04240cf410 con 0x7f042c1082a0 2024-09-10T09:00:34.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.575+0000 7f042b7fe640 1 --2- 172.21.15.172:0/498772001 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04180a2b80 0x7f04180a5040 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f041c005d00 tx=0x7f041c009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:34.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.579+0000 7f04297fa640 1 -- 172.21.15.172:0/498772001 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0424014030 con 0x7f042c1082a0 2024-09-10T09:00:34.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.784+0000 7f0431a9e640 1 -- 172.21.15.172:0/498772001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f042c10a2b0 con 0x7f042c1082a0 2024-09-10T09:00:34.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.785+0000 7f04297fa640 1 -- 172.21.15.172:0/498772001 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+12004 (secure 0 0 0) 0x7f042409c010 con 0x7f042c1082a0 2024-09-10T09:00:34.909 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T09:00:34.909 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":42,"btime":"2024-09-10T09:00:18:484858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T09:00:17.670862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T09:00:34.910 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 42 2024-09-10T09:00:34.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.787+0000 7f0431a9e640 1 -- 172.21.15.172:0/498772001 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04180a2b80 msgr2=0x7f04180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:34.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.787+0000 7f0431a9e640 1 --2- 172.21.15.172:0/498772001 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04180a2b80 0x7f04180a5040 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f041c005d00 tx=0x7f041c009550 comp rx=0 tx=0).stop 2024-09-10T09:00:34.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.787+0000 7f0431a9e640 1 -- 172.21.15.172:0/498772001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c1082a0 msgr2=0x7f042c19ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:34.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.787+0000 7f0431a9e640 1 --2- 172.21.15.172:0/498772001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c1082a0 0x7f042c19ce50 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f042400cad0 tx=0x7f042400cf90 comp rx=0 tx=0).stop 2024-09-10T09:00:34.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.787+0000 7f0431a9e640 1 -- 172.21.15.172:0/498772001 shutdown_connections 2024-09-10T09:00:34.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.787+0000 7f0431a9e640 1 --2- 172.21.15.172:0/498772001 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f04180a2b80 0x7f04180a5040 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:34.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.787+0000 7f0431a9e640 1 --2- 172.21.15.172:0/498772001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f042c1082a0 0x7f042c19ce50 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:34.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.787+0000 7f0431a9e640 1 --2- 172.21.15.172:0/498772001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f042c1036e0 0x7f042c198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:34.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.787+0000 7f0431a9e640 1 --2- 172.21.15.172:0/498772001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f042c102d30 0x7f042c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:34.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.787+0000 7f0431a9e640 1 -- 172.21.15.172:0/498772001 >> 172.21.15.172:0/498772001 conn(0x7f042c0fc7d0 msgr2=0x7f042c105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:34.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.787+0000 7f0431a9e640 1 -- 172.21.15.172:0/498772001 shutdown_connections 2024-09-10T09:00:34.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:33.787+0000 7f0431a9e640 1 -- 172.21.15.172:0/498772001 wait complete. 2024-09-10T09:00:35.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:35 smithi191 ceph-mon[35314]: pgmap v781: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 156 KiB/s wr, 53 op/s 2024-09-10T09:00:35.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:35 smithi172 ceph-mon[30854]: pgmap v781: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 156 KiB/s wr, 53 op/s 2024-09-10T09:00:35.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:35 smithi174 ceph-mon[34068]: pgmap v781: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 156 KiB/s wr, 53 op/s 2024-09-10T09:00:36.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:36 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:00:36.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:36 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:00:36.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:36 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:00:37.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:37 smithi191 ceph-mon[35314]: pgmap v782: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 83 KiB/s wr, 31 op/s 2024-09-10T09:00:37.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:37 smithi172 ceph-mon[30854]: pgmap v782: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 83 KiB/s wr, 31 op/s 2024-09-10T09:00:37.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:37 smithi174 ceph-mon[34068]: pgmap v782: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 83 KiB/s wr, 31 op/s 2024-09-10T09:00:38.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:38 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:00:38.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:38 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:00:38.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:38 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:00:38.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:38 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:00:38.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:38 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:00:38.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:38 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:00:38.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:38 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:00:38.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:38 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:00:38.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:38 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:00:39.726 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.722+0000 7fe23d5fd640 1 -- 172.21.15.172:0/4195893758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe238100a60 msgr2=0x7fe238100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.722+0000 7fe23d5fd640 1 --2- 172.21.15.172:0/4195893758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe238100a60 0x7fe238100e60 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fe228009900 tx=0x7fe22802a410 comp rx=0 tx=0).stop 2024-09-10T09:00:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.722+0000 7fe23d5fd640 1 -- 172.21.15.172:0/4195893758 shutdown_connections 2024-09-10T09:00:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.722+0000 7fe23d5fd640 1 --2- 172.21.15.172:0/4195893758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe238102610 0x7fe238108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.722+0000 7fe23d5fd640 1 --2- 172.21.15.172:0/4195893758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe238101c50 0x7fe2381020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.722+0000 7fe23d5fd640 1 --2- 172.21.15.172:0/4195893758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe238100a60 0x7fe238100e60 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.722+0000 7fe23d5fd640 1 -- 172.21.15.172:0/4195893758 >> 172.21.15.172:0/4195893758 conn(0x7fe2380fc1e0 msgr2=0x7fe2380fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.722+0000 7fe23d5fd640 1 -- 172.21.15.172:0/4195893758 shutdown_connections 2024-09-10T09:00:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.722+0000 7fe23d5fd640 1 -- 172.21.15.172:0/4195893758 wait complete. 2024-09-10T09:00:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.722+0000 7fe23d5fd640 1 Processor -- start 2024-09-10T09:00:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.723+0000 7fe23d5fd640 1 -- start start 2024-09-10T09:00:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.723+0000 7fe23d5fd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe238100a60 0x7fe23806ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.723+0000 7fe23d5fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe238101c50 0x7fe23819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.723+0000 7fe23d5fd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe238102610 0x7fe23819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.723+0000 7fe23d5fd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe23810c570 con 0x7fe238101c50 2024-09-10T09:00:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.723+0000 7fe23d5fd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe23810c3f0 con 0x7fe238100a60 2024-09-10T09:00:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.723+0000 7fe23d5fd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe23810c6f0 con 0x7fe238102610 2024-09-10T09:00:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.723+0000 7fe2367fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe238101c50 0x7fe23819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.723+0000 7fe2367fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe238101c50 0x7fe23819caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38588/0 (socket says 172.21.15.172:38588) 2024-09-10T09:00:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.723+0000 7fe2367fc640 1 -- 172.21.15.172:0/1422680022 learned_addr learned my addr 172.21.15.172:0/1422680022 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.723+0000 7fe2377fe640 1 --2- 172.21.15.172:0/1422680022 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe238102610 0x7fe23819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.723+0000 7fe236ffd640 1 --2- 172.21.15.172:0/1422680022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe238100a60 0x7fe23806ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.723+0000 7fe2367fc640 1 -- 172.21.15.172:0/1422680022 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe238102610 msgr2=0x7fe23819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.723+0000 7fe2367fc640 1 --2- 172.21.15.172:0/1422680022 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe238102610 0x7fe23819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.723+0000 7fe2367fc640 1 -- 172.21.15.172:0/1422680022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe238100a60 msgr2=0x7fe23806ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.723+0000 7fe2367fc640 1 --2- 172.21.15.172:0/1422680022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe238100a60 0x7fe23806ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.723+0000 7fe2367fc640 1 -- 172.21.15.172:0/1422680022 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23819f9d0 con 0x7fe238101c50 2024-09-10T09:00:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.724+0000 7fe2367fc640 1 --2- 172.21.15.172:0/1422680022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe238101c50 0x7fe23819caa0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fe22000e9f0 tx=0x7fe22000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.724+0000 7fe22ffff640 1 -- 172.21.15.172:0/1422680022 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22000cd80 con 0x7fe238101c50 2024-09-10T09:00:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.724+0000 7fe22ffff640 1 -- 172.21.15.172:0/1422680022 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2200108e0 con 0x7fe238101c50 2024-09-10T09:00:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.724+0000 7fe23d5fd640 1 -- 172.21.15.172:0/1422680022 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe23819fc60 con 0x7fe238101c50 2024-09-10T09:00:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.724+0000 7fe22ffff640 1 -- 172.21.15.172:0/1422680022 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220018920 con 0x7fe238101c50 2024-09-10T09:00:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.724+0000 7fe23d5fd640 1 -- 172.21.15.172:0/1422680022 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2381a01b0 con 0x7fe238101c50 2024-09-10T09:00:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.726+0000 7fe22ffff640 1 -- 172.21.15.172:0/1422680022 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe220010a90 con 0x7fe238101c50 2024-09-10T09:00:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.726+0000 7fe240480740 1 Processor -- start 2024-09-10T09:00:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.727+0000 7fe240480740 1 -- start start 2024-09-10T09:00:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.727+0000 7fe240480740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55bddb0 0x564be55be1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.727+0000 7fe240480740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55be6f0 0x564be55c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.727+0000 7fe240480740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55c3780 0x564be55c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.727+0000 7fe240480740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564be551d2f0 con 0x564be55be6f0 2024-09-10T09:00:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.727+0000 7fe240480740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564be54c5270 con 0x564be55c3780 2024-09-10T09:00:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.727+0000 7fe240480740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564be55192b0 con 0x564be55bddb0 2024-09-10T09:00:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.727+0000 7fe22ffff640 1 --2- 172.21.15.172:0/1422680022 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe2240a2cb0 0x7fe2240a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.727+0000 7fe236ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55be6f0 0x564be55c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.727+0000 7fe236ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55be6f0 0x564be55c31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38604/0 (socket says 172.21.15.172:38604) 2024-09-10T09:00:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.727+0000 7fe236ffd640 1 -- 172.21.15.172:0/239953325 learned_addr learned my addr 172.21.15.172:0/239953325 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.727+0000 7fe2377fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55bddb0 0x564be55be1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.727+0000 7fe2367fc640 1 --2- 172.21.15.172:0/239953325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55c3780 0x564be55c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.727+0000 7fe2377fe640 1 --2- 172.21.15.172:0/1422680022 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe2240a2cb0 0x7fe2240a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.727+0000 7fe22ffff640 1 -- 172.21.15.172:0/1422680022 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe2200cf680 con 0x7fe238101c50 2024-09-10T09:00:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.728+0000 7fe236ffd640 1 -- 172.21.15.172:0/239953325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55bddb0 msgr2=0x564be55be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.728+0000 7fe236ffd640 1 --2- 172.21.15.172:0/239953325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55bddb0 0x564be55be1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.728+0000 7fe236ffd640 1 -- 172.21.15.172:0/239953325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55c3780 msgr2=0x564be55c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.728+0000 7fe236ffd640 1 --2- 172.21.15.172:0/239953325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55c3780 0x564be55c5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.728+0000 7fe236ffd640 1 -- 172.21.15.172:0/239953325 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564be54c4340 con 0x564be55be6f0 2024-09-10T09:00:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.728+0000 7fe2367fc640 1 --2- 172.21.15.172:0/239953325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55c3780 0x564be55c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:00:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.728+0000 7fe2377fe640 1 --2- 172.21.15.172:0/1422680022 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe2240a2cb0 0x7fe2240a5170 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe230009990 tx=0x7fe2300096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.728+0000 7fe236ffd640 1 --2- 172.21.15.172:0/239953325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55be6f0 0x564be55c31b0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fe228009900 tx=0x7fe228032b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202060dcdaed797d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.728+0000 7fe22d7fa640 1 -- 172.21.15.172:0/239953325 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22803f070 con 0x564be55be6f0 2024-09-10T09:00:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.728+0000 7fe22d7fa640 1 -- 172.21.15.172:0/239953325 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe228004030 con 0x564be55be6f0 2024-09-10T09:00:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.728+0000 7fe22d7fa640 1 -- 172.21.15.172:0/239953325 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228035070 con 0x564be55be6f0 2024-09-10T09:00:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.729+0000 7fe240480740 1 -- 172.21.15.172:0/239953325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55be6f0 msgr2=0x564be55c31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.729+0000 7fe240480740 1 --2- 172.21.15.172:0/239953325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55be6f0 0x564be55c31b0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fe228009900 tx=0x7fe228032b00 comp rx=0 tx=0).stop 2024-09-10T09:00:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.729+0000 7fe240480740 1 -- 172.21.15.172:0/239953325 shutdown_connections 2024-09-10T09:00:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.729+0000 7fe240480740 1 --2- 172.21.15.172:0/239953325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55c3780 0x564be55c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.729+0000 7fe240480740 1 --2- 172.21.15.172:0/239953325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55be6f0 0x564be55c31b0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.729+0000 7fe240480740 1 --2- 172.21.15.172:0/239953325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55bddb0 0x564be55be1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.729+0000 7fe240480740 1 -- 172.21.15.172:0/239953325 >> 172.21.15.172:0/239953325 conn(0x564be55b5010 msgr2=0x564be55b5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.729+0000 7fe240480740 1 -- 172.21.15.172:0/239953325 shutdown_connections 2024-09-10T09:00:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.729+0000 7fe240480740 1 -- 172.21.15.172:0/239953325 wait complete. 2024-09-10T09:00:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.729+0000 7fe240480740 1 -- start start 2024-09-10T09:00:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.729+0000 7fe240480740 1 Processor -- start 2024-09-10T09:00:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe240480740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55bddb0 0x564be55c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe240480740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55be6f0 0x564be55c8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe240480740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55c3780 0x564be55cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe240480740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564be55c66f0 con 0x564be55bddb0 2024-09-10T09:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe240480740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564be54a6290 con 0x564be55be6f0 2024-09-10T09:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe240480740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564be55c6870 con 0x564be55c3780 2024-09-10T09:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe240480740 10 client.0 _ll_register_callbacks cb 0x564be54d7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe236ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55bddb0 0x564be55c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe236ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55bddb0 0x564be55c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38610/0 (socket says 172.21.15.172:38610) 2024-09-10T09:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe236ffd640 1 -- 172.21.15.172:0/1868139011 learned_addr learned my addr 172.21.15.172:0/1868139011 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe2367fc640 1 --2- 172.21.15.172:0/1868139011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55be6f0 0x564be55c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe2377fe640 1 --2- 172.21.15.172:0/1868139011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55c3780 0x564be55cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe236ffd640 1 -- 172.21.15.172:0/1868139011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55c3780 msgr2=0x564be55cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe236ffd640 1 --2- 172.21.15.172:0/1868139011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55c3780 0x564be55cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe236ffd640 1 -- 172.21.15.172:0/1868139011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55be6f0 msgr2=0x564be55c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe236ffd640 1 --2- 172.21.15.172:0/1868139011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55be6f0 0x564be55c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe236ffd640 1 -- 172.21.15.172:0/1868139011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564be5230210 con 0x564be55bddb0 2024-09-10T09:00:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe2377fe640 1 --2- 172.21.15.172:0/1868139011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55c3780 0x564be55cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:00:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe2367fc640 1 --2- 172.21.15.172:0/1868139011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55be6f0 0x564be55c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:00:39.746 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe236ffd640 1 --2- 172.21.15.172:0/1868139011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55bddb0 0x564be55c7f40 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fe22802a930 tx=0x7fe228004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20219a307bd9c7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe21e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe21e7fc640 1 -- 172.21.15.172:0/1868139011 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228035070 con 0x564be55bddb0 2024-09-10T09:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe21e7fc640 1 -- 172.21.15.172:0/1868139011 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe2280035f0 con 0x564be55bddb0 2024-09-10T09:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.730+0000 7fe240480740 1 -- 172.21.15.172:0/1868139011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe228004030 con 0x564be55bddb0 2024-09-10T09:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.731+0000 7fe21e7fc640 1 -- 172.21.15.172:0/1868139011 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe228055050 con 0x564be55bddb0 2024-09-10T09:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.731+0000 7fe240480740 10 client.19419 fetch_fsmap learned FSMap version 42 2024-09-10T09:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.731+0000 7fe240480740 1 -- 172.21.15.172:0/1868139011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564be52fc620 con 0x564be55bddb0 2024-09-10T09:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.731+0000 7fe21e7fc640 1 -- 172.21.15.172:0/1868139011 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe228004030 con 0x564be55bddb0 2024-09-10T09:00:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.731+0000 7fe21e7fc640 1 -- 172.21.15.172:0/1868139011 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7fe228004a30 con 0x564be55bddb0 2024-09-10T09:00:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.731+0000 7fe240480740 10 client.19419 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:00:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.731+0000 7fe240480740 10 client.19419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:00:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.731+0000 7fe240480740 20 client.19419 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.731+0000 7fe240480740 1 --2- 172.21.15.172:0/1868139011 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564be55d37c0 0x564be55d5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.731+0000 7fe240480740 4 client.19419 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:00:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.731+0000 7fe240480740 1 -- 172.21.15.172:0/1868139011 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564be552aa20 con 0x564be55d37c0 2024-09-10T09:00:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.731+0000 7fe2367fc640 1 --2- 172.21.15.172:0/1868139011 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564be55d37c0 0x564be55d5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.732+0000 7fe2367fc640 1 --2- 172.21.15.172:0/1868139011 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564be55d37c0 0x564be55d5bb0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8c2f4222b4c0ac7 server_cookie=77886aa2dc01fef9 in_seq=0 out_seq=0 2024-09-10T09:00:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.732+0000 7fe21e7fc640 10 client.19419 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.732+0000 7fe21e7fc640 1 -- 172.21.15.172:0/1868139011 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564be552aa20 con 0x564be55d37c0 2024-09-10T09:00:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.732+0000 7fe21e7fc640 10 client.19419 handle_command_reply: tid=0 2024-09-10T09:00:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.732+0000 7fe240480740 1 client.19419 shutdown 2024-09-10T09:00:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.733+0000 7fe240480740 1 -- 172.21.15.172:0/1868139011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55bddb0 msgr2=0x564be55c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.733+0000 7fe240480740 1 --2- 172.21.15.172:0/1868139011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55bddb0 0x564be55c7f40 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fe22802a930 tx=0x7fe228004470 comp rx=0 tx=0).stop 2024-09-10T09:00:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.733+0000 7fe240480740 1 -- 172.21.15.172:0/1868139011 shutdown_connections 2024-09-10T09:00:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.733+0000 7fe240480740 1 --2- 172.21.15.172:0/1868139011 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564be55d37c0 0x564be55d5bb0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.733+0000 7fe240480740 1 --2- 172.21.15.172:0/1868139011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55c3780 0x564be55cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.733+0000 7fe240480740 1 --2- 172.21.15.172:0/1868139011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55be6f0 0x564be55c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.733+0000 7fe240480740 1 --2- 172.21.15.172:0/1868139011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55bddb0 0x564be55c7f40 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.733+0000 7fe240480740 1 -- 172.21.15.172:0/1868139011 >> 172.21.15.172:0/1868139011 conn(0x564be55b5010 msgr2=0x564be55b6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.733+0000 7fe21e7fc640 0 client.19419 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.733+0000 7fe240480740 1 -- 172.21.15.172:0/1868139011 shutdown_connections 2024-09-10T09:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.733+0000 7fe240480740 1 -- 172.21.15.172:0/1868139011 wait complete. 2024-09-10T09:00:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.733+0000 7fe240480740 20 client.19419 trim_cache size 0 max 16384 2024-09-10T09:00:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.747+0000 7fe240480740 1 Processor -- start 2024-09-10T09:00:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.747+0000 7fe240480740 1 -- start start 2024-09-10T09:00:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe240480740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55bddb0 0x564be55c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe240480740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55be6f0 0x564be55cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe240480740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55c3780 0x564be55b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe240480740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe2381a82f0 con 0x564be55c3780 2024-09-10T09:00:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe240480740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2381a9e90 con 0x564be55be6f0 2024-09-10T09:00:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe240480740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe2381a89c0 con 0x564be55bddb0 2024-09-10T09:00:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe2377fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55c3780 0x564be55b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe2377fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55c3780 0x564be55b53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38624/0 (socket says 172.21.15.172:38624) 2024-09-10T09:00:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe2377fe640 1 -- 172.21.15.172:0/3293446819 learned_addr learned my addr 172.21.15.172:0/3293446819 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe236ffd640 1 --2- 172.21.15.172:0/3293446819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55bddb0 0x564be55c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe2367fc640 1 --2- 172.21.15.172:0/3293446819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55be6f0 0x564be55cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe2377fe640 1 -- 172.21.15.172:0/3293446819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55bddb0 msgr2=0x564be55c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe2377fe640 1 --2- 172.21.15.172:0/3293446819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55bddb0 0x564be55c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe2377fe640 1 -- 172.21.15.172:0/3293446819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55be6f0 msgr2=0x564be55cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe2377fe640 1 --2- 172.21.15.172:0/3293446819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55be6f0 0x564be55cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe2377fe640 1 -- 172.21.15.172:0/3293446819 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564be55c2b30 con 0x564be55c3780 2024-09-10T09:00:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe2367fc640 1 --2- 172.21.15.172:0/3293446819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55be6f0 0x564be55cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:00:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.748+0000 7fe236ffd640 1 --2- 172.21.15.172:0/3293446819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55bddb0 0x564be55c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:00:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.749+0000 7fe2377fe640 1 --2- 172.21.15.172:0/3293446819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55c3780 0x564be55b53f0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fe230012a60 tx=0x7fe230009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d14e0b4172d2f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.749+0000 7fe22cff9640 1 -- 172.21.15.172:0/3293446819 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230012e20 con 0x564be55c3780 2024-09-10T09:00:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.749+0000 7fe22cff9640 1 -- 172.21.15.172:0/3293446819 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe230025070 con 0x564be55c3780 2024-09-10T09:00:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.749+0000 7fe22cff9640 1 -- 172.21.15.172:0/3293446819 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23002abe0 con 0x564be55c3780 2024-09-10T09:00:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 1 -- 172.21.15.172:0/3293446819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55c3780 msgr2=0x564be55b53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 1 --2- 172.21.15.172:0/3293446819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55c3780 0x564be55b53f0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fe230012a60 tx=0x7fe230009920 comp rx=0 tx=0).stop 2024-09-10T09:00:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 1 -- 172.21.15.172:0/3293446819 shutdown_connections 2024-09-10T09:00:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 1 --2- 172.21.15.172:0/3293446819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55c3780 0x564be55b53f0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 1 --2- 172.21.15.172:0/3293446819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55be6f0 0x564be55cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 1 --2- 172.21.15.172:0/3293446819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55bddb0 0x564be55c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 1 -- 172.21.15.172:0/3293446819 >> 172.21.15.172:0/3293446819 conn(0x564be55b5010 msgr2=0x564be55c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 1 -- 172.21.15.172:0/3293446819 shutdown_connections 2024-09-10T09:00:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 1 -- 172.21.15.172:0/3293446819 wait complete. 2024-09-10T09:00:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 1 -- start start 2024-09-10T09:00:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 1 Processor -- start 2024-09-10T09:00:39.762 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55bddb0 0x564be55cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.762 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55be6f0 0x564be55ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.762 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55c3780 0x564be55cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.762 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe2381a97c0 con 0x564be55c3780 2024-09-10T09:00:39.762 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2381a90f0 con 0x564be55be6f0 2024-09-10T09:00:39.762 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe2381a62d0 con 0x564be55bddb0 2024-09-10T09:00:39.762 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.750+0000 7fe240480740 10 client.0 _ll_register_callbacks cb 0x7fe22000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.751+0000 7fe2377fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55c3780 0x564be55cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.751+0000 7fe2377fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55c3780 0x564be55cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38626/0 (socket says 172.21.15.172:38626) 2024-09-10T09:00:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.751+0000 7fe2377fe640 1 -- 172.21.15.172:0/577824171 learned_addr learned my addr 172.21.15.172:0/577824171 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.751+0000 7fe2367fc640 1 --2- 172.21.15.172:0/577824171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55be6f0 0x564be55ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.751+0000 7fe236ffd640 1 --2- 172.21.15.172:0/577824171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55bddb0 0x564be55cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.751+0000 7fe2377fe640 1 -- 172.21.15.172:0/577824171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55bddb0 msgr2=0x564be55cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.751+0000 7fe2377fe640 1 --2- 172.21.15.172:0/577824171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55bddb0 0x564be55cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.764 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.751+0000 7fe2377fe640 1 -- 172.21.15.172:0/577824171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55be6f0 msgr2=0x564be55ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:39.764 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.751+0000 7fe2377fe640 1 --2- 172.21.15.172:0/577824171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55be6f0 0x564be55ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.764 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.751+0000 7fe2377fe640 1 -- 172.21.15.172:0/577824171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564be55ce4c0 con 0x564be55c3780 2024-09-10T09:00:39.764 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.751+0000 7fe236ffd640 1 --2- 172.21.15.172:0/577824171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55bddb0 0x564be55cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:00:39.764 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.751+0000 7fe2377fe640 1 --2- 172.21.15.172:0/577824171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55c3780 0x564be55cdeb0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fe230010820 tx=0x7fe230012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2862c11839872 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:39.764 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.751+0000 7fe21e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:00:39.764 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.751+0000 7fe21e7fc640 1 -- 172.21.15.172:0/577824171 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23002abf0 con 0x564be55c3780 2024-09-10T09:00:39.765 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.751+0000 7fe21e7fc640 1 -- 172.21.15.172:0/577824171 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe23002ad90 con 0x564be55c3780 2024-09-10T09:00:39.765 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.751+0000 7fe240480740 1 -- 172.21.15.172:0/577824171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe230025070 con 0x564be55c3780 2024-09-10T09:00:39.765 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.752+0000 7fe21e7fc640 1 -- 172.21.15.172:0/577824171 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe23003b050 con 0x564be55c3780 2024-09-10T09:00:39.765 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.752+0000 7fe240480740 10 client.19425 fetch_fsmap learned FSMap version 42 2024-09-10T09:00:39.765 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.752+0000 7fe240480740 1 -- 172.21.15.172:0/577824171 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564be55cf7d0 con 0x564be55c3780 2024-09-10T09:00:39.765 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.752+0000 7fe21e7fc640 1 -- 172.21.15.172:0/577824171 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe230025070 con 0x564be55c3780 2024-09-10T09:00:39.765 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.752+0000 7fe21e7fc640 1 -- 172.21.15.172:0/577824171 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7fe2300218c0 con 0x564be55c3780 2024-09-10T09:00:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.752+0000 7fe240480740 10 client.19425 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:00:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.752+0000 7fe240480740 10 client.19425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:00:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.752+0000 7fe240480740 20 client.19425 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.752+0000 7fe240480740 1 --2- 172.21.15.172:0/577824171 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564be563eb00 0x564be5640f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.752+0000 7fe240480740 4 client.19425 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:00:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.752+0000 7fe240480740 1 -- 172.21.15.172:0/577824171 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564be5524d70 con 0x564be563eb00 2024-09-10T09:00:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.752+0000 7fe236ffd640 1 --2- 172.21.15.172:0/577824171 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564be563eb00 0x564be5640f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.753+0000 7fe21e7fc640 10 client.19425 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.753+0000 7fe236ffd640 1 --2- 172.21.15.172:0/577824171 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564be563eb00 0x564be5640f20 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=586df3fd53a4a115 server_cookie=96dedcc5a7678989 in_seq=0 out_seq=0 2024-09-10T09:00:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.759+0000 7fe21e7fc640 1 -- 172.21.15.172:0/577824171 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564be5524d70 con 0x564be563eb00 2024-09-10T09:00:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.759+0000 7fe21e7fc640 10 client.19425 handle_command_reply: tid=0 2024-09-10T09:00:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.759+0000 7fe240480740 1 client.19425 shutdown 2024-09-10T09:00:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.759+0000 7fe240480740 1 -- 172.21.15.172:0/577824171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55c3780 msgr2=0x564be55cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.759+0000 7fe240480740 1 --2- 172.21.15.172:0/577824171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55c3780 0x564be55cdeb0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fe230010820 tx=0x7fe230012b00 comp rx=0 tx=0).stop 2024-09-10T09:00:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe240480740 1 -- 172.21.15.172:0/577824171 shutdown_connections 2024-09-10T09:00:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe240480740 1 --2- 172.21.15.172:0/577824171 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564be563eb00 0x564be5640f20 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe240480740 1 --2- 172.21.15.172:0/577824171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564be55c3780 0x564be55cdeb0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe240480740 1 --2- 172.21.15.172:0/577824171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564be55be6f0 0x564be55ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe21e7fc640 0 client.19425 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe240480740 1 --2- 172.21.15.172:0/577824171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564be55bddb0 0x564be55cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe240480740 1 -- 172.21.15.172:0/577824171 >> 172.21.15.172:0/577824171 conn(0x564be55b5010 msgr2=0x564be55b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe240480740 1 -- 172.21.15.172:0/577824171 shutdown_connections 2024-09-10T09:00:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe240480740 1 -- 172.21.15.172:0/577824171 wait complete. 2024-09-10T09:00:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe240480740 20 client.19425 trim_cache size 0 max 16384 2024-09-10T09:00:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:00:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:00:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:00:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:00:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe23d5fd640 1 -- 172.21.15.172:0/1422680022 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe2240a2cb0 msgr2=0x7fe2240a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:39.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe23d5fd640 1 --2- 172.21.15.172:0/1422680022 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe2240a2cb0 0x7fe2240a5170 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe230009990 tx=0x7fe2300096a0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe23d5fd640 1 -- 172.21.15.172:0/1422680022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe238101c50 msgr2=0x7fe23819caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:39.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe23d5fd640 1 --2- 172.21.15.172:0/1422680022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe238101c50 0x7fe23819caa0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fe22000e9f0 tx=0x7fe22000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe23d5fd640 1 -- 172.21.15.172:0/1422680022 shutdown_connections 2024-09-10T09:00:39.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe23d5fd640 1 --2- 172.21.15.172:0/1422680022 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe2240a2cb0 0x7fe2240a5170 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe23d5fd640 1 --2- 172.21.15.172:0/1422680022 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe238102610 0x7fe23819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe23d5fd640 1 --2- 172.21.15.172:0/1422680022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe238101c50 0x7fe23819caa0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe23d5fd640 1 --2- 172.21.15.172:0/1422680022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe238100a60 0x7fe23806ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.760+0000 7fe23d5fd640 1 -- 172.21.15.172:0/1422680022 >> 172.21.15.172:0/1422680022 conn(0x7fe2380fc1e0 msgr2=0x7fe238105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.761+0000 7fe23d5fd640 1 -- 172.21.15.172:0/1422680022 shutdown_connections 2024-09-10T09:00:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:39.761+0000 7fe23d5fd640 1 -- 172.21.15.172:0/1422680022 wait complete. 2024-09-10T09:00:39.791 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:39 smithi172 ceph-mon[30854]: pgmap v783: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 95 KiB/s wr, 35 op/s 2024-09-10T09:00:39.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:39 smithi191 ceph-mon[35314]: pgmap v783: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 95 KiB/s wr, 35 op/s 2024-09-10T09:00:39.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:39 smithi174 ceph-mon[34068]: pgmap v783: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 95 KiB/s wr, 35 op/s 2024-09-10T09:00:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:41 smithi174 ceph-mon[34068]: pgmap v784: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 98 KiB/s wr, 37 op/s 2024-09-10T09:00:41.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:41 smithi191 ceph-mon[35314]: pgmap v784: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 98 KiB/s wr, 37 op/s 2024-09-10T09:00:41.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:41 smithi172 ceph-mon[30854]: pgmap v784: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 98 KiB/s wr, 37 op/s 2024-09-10T09:00:42.629 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:00:42.676 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:00:43.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:43 smithi174 ceph-mon[34068]: pgmap v785: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 86 KiB/s wr, 31 op/s 2024-09-10T09:00:43.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:43 smithi191 ceph-mon[35314]: pgmap v785: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 86 KiB/s wr, 31 op/s 2024-09-10T09:00:43.813 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:43 smithi172 ceph-mon[30854]: pgmap v785: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 86 KiB/s wr, 31 op/s 2024-09-10T09:00:45.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.275+0000 7f0ee32a1640 1 -- 172.21.15.172:0/489804544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0edc074620 msgr2=0x7f0edc074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:45.292 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.275+0000 7f0ee32a1640 1 --2- 172.21.15.172:0/489804544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0edc074620 0x7f0edc074a00 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009900 tx=0x7f0ed802a480 comp rx=0 tx=0).stop 2024-09-10T09:00:45.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.275+0000 7f0ee32a1640 1 -- 172.21.15.172:0/489804544 shutdown_connections 2024-09-10T09:00:45.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.275+0000 7f0ee32a1640 1 --2- 172.21.15.172:0/489804544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0edc10e3c0 0x7f0edc110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:45.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.275+0000 7f0ee32a1640 1 --2- 172.21.15.172:0/489804544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0edc074f40 0x7f0edc10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:45.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.275+0000 7f0ee32a1640 1 --2- 172.21.15.172:0/489804544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0edc074620 0x7f0edc074a00 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:45.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.275+0000 7f0ee32a1640 1 -- 172.21.15.172:0/489804544 >> 172.21.15.172:0/489804544 conn(0x7f0edc0fd8c0 msgr2=0x7f0edc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:45.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.275+0000 7f0ee32a1640 1 -- 172.21.15.172:0/489804544 shutdown_connections 2024-09-10T09:00:45.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.275+0000 7f0ee32a1640 1 -- 172.21.15.172:0/489804544 wait complete. 2024-09-10T09:00:45.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.276+0000 7f0ee32a1640 1 Processor -- start 2024-09-10T09:00:45.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.276+0000 7f0ee32a1640 1 -- start start 2024-09-10T09:00:45.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.276+0000 7f0ee32a1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0edc074620 0x7f0edc19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:45.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.276+0000 7f0ee32a1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0edc074f40 0x7f0edc19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:45.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.276+0000 7f0ee32a1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0edc10e3c0 0x7f0edc19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:45.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.276+0000 7f0ee32a1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc114000 con 0x7f0edc074620 2024-09-10T09:00:45.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.276+0000 7f0ee32a1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc113e80 con 0x7f0edc074f40 2024-09-10T09:00:45.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.276+0000 7f0ee32a1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0edc114180 con 0x7f0edc10e3c0 2024-09-10T09:00:45.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.276+0000 7f0ee103d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0edc074620 0x7f0edc19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:45.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.276+0000 7f0ee103d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0edc074620 0x7f0edc19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:38636/0 (socket says 172.21.15.172:38636) 2024-09-10T09:00:45.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.276+0000 7f0ee103d640 1 -- 172.21.15.172:0/2179471208 learned_addr learned my addr 172.21.15.172:0/2179471208 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:45.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.277+0000 7f0ee183e640 1 --2- 172.21.15.172:0/2179471208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0edc10e3c0 0x7f0edc19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:45.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.277+0000 7f0ee083c640 1 --2- 172.21.15.172:0/2179471208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0edc074f40 0x7f0edc19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:45.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.277+0000 7f0ee103d640 1 -- 172.21.15.172:0/2179471208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0edc10e3c0 msgr2=0x7f0edc19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:45.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.277+0000 7f0ee103d640 1 --2- 172.21.15.172:0/2179471208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0edc10e3c0 0x7f0edc19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:45.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.277+0000 7f0ee103d640 1 -- 172.21.15.172:0/2179471208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0edc074f40 msgr2=0x7f0edc19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:45.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.277+0000 7f0ee103d640 1 --2- 172.21.15.172:0/2179471208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0edc074f40 0x7f0edc19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:45.294 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.277+0000 7f0ee103d640 1 -- 172.21.15.172:0/2179471208 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0edc19f560 con 0x7f0edc074620 2024-09-10T09:00:45.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.277+0000 7f0ee103d640 1 --2- 172.21.15.172:0/2179471208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0edc074620 0x7f0edc19a5b0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8006010 tx=0x7f0ed8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:45.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.277+0000 7f0ece7fc640 1 -- 172.21.15.172:0/2179471208 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed803f030 con 0x7f0edc074620 2024-09-10T09:00:45.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.277+0000 7f0ee32a1640 1 -- 172.21.15.172:0/2179471208 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0edc19f7f0 con 0x7f0edc074620 2024-09-10T09:00:45.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.277+0000 7f0ece7fc640 1 -- 172.21.15.172:0/2179471208 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ed80035f0 con 0x7f0edc074620 2024-09-10T09:00:45.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.277+0000 7f0ece7fc640 1 -- 172.21.15.172:0/2179471208 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8035070 con 0x7f0edc074620 2024-09-10T09:00:45.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.277+0000 7f0ee32a1640 1 -- 172.21.15.172:0/2179471208 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0edc1a7130 con 0x7f0edc074620 2024-09-10T09:00:45.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.279+0000 7f0ee32a1640 1 -- 172.21.15.172:0/2179471208 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0edc075cd0 con 0x7f0edc074620 2024-09-10T09:00:45.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.280+0000 7f0ece7fc640 1 -- 172.21.15.172:0/2179471208 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0ed8003b10 con 0x7f0edc074620 2024-09-10T09:00:45.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.280+0000 7f0ece7fc640 1 --2- 172.21.15.172:0/2179471208 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0ec80a2b80 0x7f0ec80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:45.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.280+0000 7f0ece7fc640 1 -- 172.21.15.172:0/2179471208 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f0ed80f0080 con 0x7f0edc074620 2024-09-10T09:00:45.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.280+0000 7f0ee083c640 1 --2- 172.21.15.172:0/2179471208 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0ec80a2b80 0x7f0ec80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:45.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.281+0000 7f0ee083c640 1 --2- 172.21.15.172:0/2179471208 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0ec80a2b80 0x7f0ec80a5040 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0fffa0 tx=0x7f0ed000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:45.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.284+0000 7f0ece7fc640 1 -- 172.21.15.172:0/2179471208 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ed80bc630 con 0x7f0edc074620 2024-09-10T09:00:45.342 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:00:45.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.503+0000 7f0ee32a1640 1 -- 172.21.15.172:0/2179471208 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0edc074a00 con 0x7f0edc074620 2024-09-10T09:00:45.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.504+0000 7f0ece7fc640 1 -- 172.21.15.172:0/2179471208 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+12004 (secure 0 0 0) 0x7f0edc074a00 con 0x7f0edc074620 2024-09-10T09:00:45.506 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T09:00:45.506 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":42,"btime":"2024-09-10T09:00:18:484858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T09:00:17.670862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T09:00:45.506 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 42 2024-09-10T09:00:45.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.506+0000 7f0ee32a1640 1 -- 172.21.15.172:0/2179471208 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0ec80a2b80 msgr2=0x7f0ec80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:45.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.506+0000 7f0ee32a1640 1 --2- 172.21.15.172:0/2179471208 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0ec80a2b80 0x7f0ec80a5040 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0fffa0 tx=0x7f0ed000a2d0 comp rx=0 tx=0).stop 2024-09-10T09:00:45.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.506+0000 7f0ee32a1640 1 -- 172.21.15.172:0/2179471208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0edc074620 msgr2=0x7f0edc19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:45.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.506+0000 7f0ee32a1640 1 --2- 172.21.15.172:0/2179471208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0edc074620 0x7f0edc19a5b0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8006010 tx=0x7f0ed8004270 comp rx=0 tx=0).stop 2024-09-10T09:00:45.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.506+0000 7f0ee32a1640 1 -- 172.21.15.172:0/2179471208 shutdown_connections 2024-09-10T09:00:45.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.506+0000 7f0ee32a1640 1 --2- 172.21.15.172:0/2179471208 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f0ec80a2b80 0x7f0ec80a5040 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:45.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.506+0000 7f0ee32a1640 1 --2- 172.21.15.172:0/2179471208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0edc10e3c0 0x7f0edc19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:45.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.506+0000 7f0ee32a1640 1 --2- 172.21.15.172:0/2179471208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0edc074f40 0x7f0edc19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:45.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.506+0000 7f0ee32a1640 1 --2- 172.21.15.172:0/2179471208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0edc074620 0x7f0edc19a5b0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:45.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.506+0000 7f0ee32a1640 1 -- 172.21.15.172:0/2179471208 >> 172.21.15.172:0/2179471208 conn(0x7f0edc0fd8c0 msgr2=0x7f0edc0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:45.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.507+0000 7f0ee32a1640 1 -- 172.21.15.172:0/2179471208 shutdown_connections 2024-09-10T09:00:45.508 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:45.507+0000 7f0ee32a1640 1 -- 172.21.15.172:0/2179471208 wait complete. 2024-09-10T09:00:45.737 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:45 smithi172 ceph-mon[30854]: pgmap v786: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 110 KiB/s wr, 40 op/s 2024-09-10T09:00:45.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:45 smithi174 ceph-mon[34068]: pgmap v786: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 110 KiB/s wr, 40 op/s 2024-09-10T09:00:45.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:45 smithi191 ceph-mon[35314]: pgmap v786: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 110 KiB/s wr, 40 op/s 2024-09-10T09:00:46.404 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 7787905e --release --await' 2024-09-10T09:00:46.404 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 7787905e --release --await 2024-09-10T09:00:46.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.595+0000 7f090207b640 1 -- 172.21.15.172:0/500215972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc101410 msgr2=0x7f08fc10d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:46.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.595+0000 7f090207b640 1 --2- 172.21.15.172:0/500215972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc101410 0x7f08fc10d940 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009cd0 tx=0x7f08f402a8e0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.595+0000 7f090207b640 1 -- 172.21.15.172:0/500215972 shutdown_connections 2024-09-10T09:00:46.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.595+0000 7f090207b640 1 --2- 172.21.15.172:0/500215972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc101410 0x7f08fc10d940 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.595+0000 7f090207b640 1 --2- 172.21.15.172:0/500215972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08fc100a70 0x7f08fc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.595+0000 7f090207b640 1 --2- 172.21.15.172:0/500215972 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08fc106a70 0x7f08fc106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.595+0000 7f090207b640 1 -- 172.21.15.172:0/500215972 >> 172.21.15.172:0/500215972 conn(0x7f08fc0fc7d0 msgr2=0x7f08fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:46.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.595+0000 7f090207b640 1 -- 172.21.15.172:0/500215972 shutdown_connections 2024-09-10T09:00:46.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.595+0000 7f090207b640 1 -- 172.21.15.172:0/500215972 wait complete. 2024-09-10T09:00:46.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.596+0000 7f090207b640 1 Processor -- start 2024-09-10T09:00:46.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.596+0000 7f090207b640 1 -- start start 2024-09-10T09:00:46.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.596+0000 7f090207b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08fc100a70 0x7f08fc1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.596+0000 7f090207b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08fc101410 0x7f08fc199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.596+0000 7f090207b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc106a70 0x7f08fc1a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.596+0000 7f090207b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc110d20 con 0x7f08fc106a70 2024-09-10T09:00:46.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.596+0000 7f090207b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc110ba0 con 0x7f08fc100a70 2024-09-10T09:00:46.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.596+0000 7f090207b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc110ea0 con 0x7f08fc101410 2024-09-10T09:00:46.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.596+0000 7f08fbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc106a70 0x7f08fc1a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:46.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.596+0000 7f08fbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc106a70 0x7f08fc1a4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34356/0 (socket says 172.21.15.172:34356) 2024-09-10T09:00:46.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.596+0000 7f08fbfff640 1 -- 172.21.15.172:0/384294516 learned_addr learned my addr 172.21.15.172:0/384294516 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:46.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.597+0000 7f08fb7fe640 1 --2- 172.21.15.172:0/384294516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08fc100a70 0x7f08fc1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:46.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.597+0000 7f08f3fff640 1 --2- 172.21.15.172:0/384294516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08fc101410 0x7f08fc199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:46.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.597+0000 7f08fbfff640 1 -- 172.21.15.172:0/384294516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08fc101410 msgr2=0x7f08fc199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:46.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.597+0000 7f08fbfff640 1 --2- 172.21.15.172:0/384294516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08fc101410 0x7f08fc199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.597+0000 7f08fbfff640 1 -- 172.21.15.172:0/384294516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08fc100a70 msgr2=0x7f08fc1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:46.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.597+0000 7f08fbfff640 1 --2- 172.21.15.172:0/384294516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08fc100a70 0x7f08fc1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.597+0000 7f08fbfff640 1 -- 172.21.15.172:0/384294516 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc1a4790 con 0x7f08fc106a70 2024-09-10T09:00:46.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.597+0000 7f08f3fff640 1 --2- 172.21.15.172:0/384294516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08fc101410 0x7f08fc199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:00:46.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.597+0000 7f08fbfff640 1 --2- 172.21.15.172:0/384294516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc106a70 0x7f08fc1a4250 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f08f4005cc0 tx=0x7f08f4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:46.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.597+0000 7f08f97fa640 1 -- 172.21.15.172:0/384294516 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4004240 con 0x7f08fc106a70 2024-09-10T09:00:46.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.597+0000 7f08f97fa640 1 -- 172.21.15.172:0/384294516 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08f40305a0 con 0x7f08fc106a70 2024-09-10T09:00:46.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.597+0000 7f08f97fa640 1 -- 172.21.15.172:0/384294516 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f40466c0 con 0x7f08fc106a70 2024-09-10T09:00:46.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.597+0000 7f090207b640 1 -- 172.21.15.172:0/384294516 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08fc1a4960 con 0x7f08fc106a70 2024-09-10T09:00:46.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.597+0000 7f090207b640 1 -- 172.21.15.172:0/384294516 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08fc1a4e40 con 0x7f08fc106a70 2024-09-10T09:00:46.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.599+0000 7f08f97fa640 1 -- 172.21.15.172:0/384294516 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f08f4030ac0 con 0x7f08fc106a70 2024-09-10T09:00:46.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f0904454740 1 Processor -- start 2024-09-10T09:00:46.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f0904454740 1 -- start start 2024-09-10T09:00:46.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f0904454740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 0x561a10956090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f0904454740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a109565d0 0x561a10956a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f0904454740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a10961370 0x561a10963760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f0904454740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561a108bc2f0 con 0x561a10961370 2024-09-10T09:00:46.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f0904454740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561a10863cf0 con 0x561a10957b40 2024-09-10T09:00:46.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f0904454740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561a108b82b0 con 0x561a109565d0 2024-09-10T09:00:46.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f08f97fa640 1 --2- 172.21.15.172:0/384294516 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08e80a2c00 0x7f08e80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f08f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a10961370 0x561a10963760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:46.611 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f08f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a10961370 0x561a10963760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34360/0 (socket says 172.21.15.172:34360) 2024-09-10T09:00:46.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f08f3fff640 1 -- 172.21.15.172:0/857738830 learned_addr learned my addr 172.21.15.172:0/857738830 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:46.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f08fbfff640 1 --2- 172.21.15.172:0/384294516 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08e80a2c00 0x7f08e80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:46.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f08f3fff640 1 --2- 172.21.15.172:0/857738830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 0x561a10956090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:46.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f08fb7fe640 1 --2- 172.21.15.172:0/857738830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a109565d0 0x561a10956a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:46.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f08f97fa640 1 -- 172.21.15.172:0/384294516 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f08f402e030 con 0x7f08fc106a70 2024-09-10T09:00:46.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f08f3fff640 1 -- 172.21.15.172:0/857738830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a109565d0 msgr2=0x561a10956a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:46.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f08f3fff640 1 --2- 172.21.15.172:0/857738830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a109565d0 0x561a10956a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f08f3fff640 1 -- 172.21.15.172:0/857738830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 msgr2=0x561a10956090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:46.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f08f3fff640 1 --2- 172.21.15.172:0/857738830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 0x561a10956090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.600+0000 7f08f3fff640 1 -- 172.21.15.172:0/857738830 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a108483a0 con 0x561a10961370 2024-09-10T09:00:46.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.601+0000 7f08f3fff640 1 --2- 172.21.15.172:0/857738830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a10961370 0x561a10963760 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f08e4014590 tx=0x7f08e400d5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716aa3015a3e9f84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:46.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.601+0000 7f08fbfff640 1 --2- 172.21.15.172:0/384294516 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08e80a2c00 0x7f08e80a50c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f08f40b9490 tx=0x7f08f40b1fa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:46.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.601+0000 7f08f1ffb640 1 -- 172.21.15.172:0/857738830 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4012710 con 0x561a10961370 2024-09-10T09:00:46.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.601+0000 7f08f1ffb640 1 -- 172.21.15.172:0/857738830 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08e401a070 con 0x561a10961370 2024-09-10T09:00:46.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.601+0000 7f08f1ffb640 1 -- 172.21.15.172:0/857738830 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4011b60 con 0x561a10961370 2024-09-10T09:00:46.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.601+0000 7f0904454740 1 -- 172.21.15.172:0/857738830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a10961370 msgr2=0x561a10963760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:46.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.601+0000 7f0904454740 1 --2- 172.21.15.172:0/857738830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a10961370 0x561a10963760 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f08e4014590 tx=0x7f08e400d5a0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.602+0000 7f0904454740 1 -- 172.21.15.172:0/857738830 shutdown_connections 2024-09-10T09:00:46.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.602+0000 7f0904454740 1 --2- 172.21.15.172:0/857738830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a10961370 0x561a10963760 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.602+0000 7f0904454740 1 --2- 172.21.15.172:0/857738830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a109565d0 0x561a10956a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.602+0000 7f0904454740 1 --2- 172.21.15.172:0/857738830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 0x561a10956090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.602+0000 7f0904454740 1 -- 172.21.15.172:0/857738830 >> 172.21.15.172:0/857738830 conn(0x561a1091f310 msgr2=0x561a10952c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:46.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.602+0000 7f0904454740 1 -- 172.21.15.172:0/857738830 shutdown_connections 2024-09-10T09:00:46.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.602+0000 7f0904454740 1 -- 172.21.15.172:0/857738830 wait complete. 2024-09-10T09:00:46.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.602+0000 7f0904454740 1 -- start start 2024-09-10T09:00:46.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.602+0000 7f0904454740 1 Processor -- start 2024-09-10T09:00:46.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f0904454740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a109565d0 0x561a10960d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f0904454740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a10957b40 0x561a1095e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f0904454740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10961370 0x561a1095e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f0904454740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561a10964070 con 0x561a109565d0 2024-09-10T09:00:46.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f0904454740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561a10877060 con 0x561a10961370 2024-09-10T09:00:46.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f08f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a109565d0 0x561a10960d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:46.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f0904454740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561a109641f0 con 0x561a10957b40 2024-09-10T09:00:46.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f08f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a109565d0 0x561a10960d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34364/0 (socket says 172.21.15.172:34364) 2024-09-10T09:00:46.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f08f3fff640 1 -- 172.21.15.172:0/2627047299 learned_addr learned my addr 172.21.15.172:0/2627047299 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:46.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f0904454740 10 client.0 _ll_register_callbacks cb 0x561a105ca8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:46.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f08fb7fe640 1 --2- 172.21.15.172:0/2627047299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a10957b40 0x561a1095e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:46.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f08f3fff640 1 --2- 172.21.15.172:0/2627047299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10961370 0x561a1095e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:46.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f08f3fff640 1 -- 172.21.15.172:0/2627047299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a10957b40 msgr2=0x561a1095e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:46.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f08f3fff640 1 --2- 172.21.15.172:0/2627047299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a10957b40 0x561a1095e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f08f3fff640 1 -- 172.21.15.172:0/2627047299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10961370 msgr2=0x561a1095e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:46.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f08f3fff640 1 --2- 172.21.15.172:0/2627047299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10961370 0x561a1095e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f08f3fff640 1 -- 172.21.15.172:0/2627047299 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a1062d3c0 con 0x561a109565d0 2024-09-10T09:00:46.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f08f3fff640 1 --2- 172.21.15.172:0/2627047299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a109565d0 0x561a10960d70 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f08e40058c0 tx=0x7f08e4011840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450e29a13566b8f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:46.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f08daffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:00:46.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.603+0000 7f08daffd640 1 -- 172.21.15.172:0/2627047299 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4012510 con 0x561a109565d0 2024-09-10T09:00:46.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.604+0000 7f08daffd640 1 -- 172.21.15.172:0/2627047299 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f08e40126b0 con 0x561a109565d0 2024-09-10T09:00:46.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.604+0000 7f0904454740 1 -- 172.21.15.172:0/2627047299 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08e401a070 con 0x561a109565d0 2024-09-10T09:00:46.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.604+0000 7f08daffd640 1 -- 172.21.15.172:0/2627047299 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f08e40128b0 con 0x561a109565d0 2024-09-10T09:00:46.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.604+0000 7f0904454740 10 client.19443 fetch_fsmap learned FSMap version 42 2024-09-10T09:00:46.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.604+0000 7f0904454740 1 -- 172.21.15.172:0/2627047299 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561a1096faf0 con 0x561a109565d0 2024-09-10T09:00:46.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.604+0000 7f08daffd640 1 -- 172.21.15.172:0/2627047299 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08e401a070 con 0x561a109565d0 2024-09-10T09:00:46.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.604+0000 7f08daffd640 1 -- 172.21.15.172:0/2627047299 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7f08e4021ab0 con 0x561a109565d0 2024-09-10T09:00:46.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.604+0000 7f0904454740 10 client.19443 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:00:46.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.604+0000 7f0904454740 10 client.19443 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T09:00:46.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.604+0000 7f0904454740 20 client.19443 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:46.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.604+0000 7f0904454740 1 --2- 172.21.15.172:0/2627047299 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x561a10970f50 0x561a10973340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.604+0000 7f0904454740 4 client.19443 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:00:46.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.604+0000 7f0904454740 1 -- 172.21.15.172:0/2627047299 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a108c9860 con 0x561a10970f50 2024-09-10T09:00:46.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.605+0000 7f08fb7fe640 1 --2- 172.21.15.172:0/2627047299 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x561a10970f50 0x561a10973340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:46.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.605+0000 7f08daffd640 10 client.19443 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T09:00:46.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.605+0000 7f08fb7fe640 1 --2- 172.21.15.172:0/2627047299 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x561a10970f50 0x561a10973340 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d1e73366fc25f892 server_cookie=cc70287b7d264864 in_seq=0 out_seq=0 2024-09-10T09:00:46.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.606+0000 7f08daffd640 1 -- 172.21.15.172:0/2627047299 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a108c9860 con 0x561a10970f50 2024-09-10T09:00:46.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.606+0000 7f08daffd640 10 client.19443 handle_command_reply: tid=0 2024-09-10T09:00:46.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.606+0000 7f0904454740 1 client.19443 shutdown 2024-09-10T09:00:46.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.606+0000 7f0904454740 1 -- 172.21.15.172:0/2627047299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a109565d0 msgr2=0x561a10960d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:46.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.606+0000 7f0904454740 1 --2- 172.21.15.172:0/2627047299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a109565d0 0x561a10960d70 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f08e40058c0 tx=0x7f08e4011840 comp rx=0 tx=0).stop 2024-09-10T09:00:46.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.606+0000 7f0904454740 1 -- 172.21.15.172:0/2627047299 shutdown_connections 2024-09-10T09:00:46.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.606+0000 7f0904454740 1 --2- 172.21.15.172:0/2627047299 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x561a10970f50 0x561a10973340 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.606+0000 7f0904454740 1 --2- 172.21.15.172:0/2627047299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10961370 0x561a1095e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.606+0000 7f0904454740 1 --2- 172.21.15.172:0/2627047299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a10957b40 0x561a1095e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.606+0000 7f0904454740 1 --2- 172.21.15.172:0/2627047299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a109565d0 0x561a10960d70 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.606+0000 7f08daffd640 0 client.19443 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T09:00:46.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.606+0000 7f0904454740 1 -- 172.21.15.172:0/2627047299 >> 172.21.15.172:0/2627047299 conn(0x561a1091f310 msgr2=0x561a10958960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:46.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.606+0000 7f0904454740 1 -- 172.21.15.172:0/2627047299 shutdown_connections 2024-09-10T09:00:46.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.606+0000 7f0904454740 1 -- 172.21.15.172:0/2627047299 wait complete. 2024-09-10T09:00:46.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.606+0000 7f0904454740 20 client.19443 trim_cache size 0 max 16384 2024-09-10T09:00:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f0904454740 1 Processor -- start 2024-09-10T09:00:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f0904454740 1 -- start start 2024-09-10T09:00:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f0904454740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a109565d0 0x561a1096d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f0904454740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 0x561a1095d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f0904454740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a10961370 0x561a10978940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f0904454740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1aeb00 con 0x561a109565d0 2024-09-10T09:00:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f0904454740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1b06a0 con 0x561a10957b40 2024-09-10T09:00:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f0904454740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1af1d0 con 0x561a10961370 2024-09-10T09:00:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f08f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a109565d0 0x561a1096d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f08f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a109565d0 0x561a1096d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34376/0 (socket says 172.21.15.172:34376) 2024-09-10T09:00:46.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f08f3fff640 1 -- 172.21.15.172:0/1246806628 learned_addr learned my addr 172.21.15.172:0/1246806628 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:46.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f08fb7fe640 1 --2- 172.21.15.172:0/1246806628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 0x561a1095d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:46.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f08f3fff640 1 --2- 172.21.15.172:0/1246806628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a10961370 0x561a10978940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:46.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f08f3fff640 1 -- 172.21.15.172:0/1246806628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a10961370 msgr2=0x561a10978940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:46.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f08f3fff640 1 --2- 172.21.15.172:0/1246806628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a10961370 0x561a10978940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f08f3fff640 1 -- 172.21.15.172:0/1246806628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 msgr2=0x561a1095d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:46.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f08f3fff640 1 --2- 172.21.15.172:0/1246806628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 0x561a1095d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f08f3fff640 1 -- 172.21.15.172:0/1246806628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a109641e0 con 0x561a109565d0 2024-09-10T09:00:46.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.620+0000 7f08fb7fe640 1 --2- 172.21.15.172:0/1246806628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 0x561a1095d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:00:46.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.621+0000 7f08f3fff640 1 --2- 172.21.15.172:0/1246806628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a109565d0 0x561a1096d310 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f08e40206b0 tx=0x7f08e4022fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe0b92ed5bb562f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:46.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.621+0000 7f08f17fa640 1 -- 172.21.15.172:0/1246806628 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e403b020 con 0x561a109565d0 2024-09-10T09:00:46.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.621+0000 7f08f17fa640 1 -- 172.21.15.172:0/1246806628 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08e401a040 con 0x561a109565d0 2024-09-10T09:00:46.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.621+0000 7f08f17fa640 1 -- 172.21.15.172:0/1246806628 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4005040 con 0x561a109565d0 2024-09-10T09:00:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.622+0000 7f0904454740 1 -- 172.21.15.172:0/1246806628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a109565d0 msgr2=0x561a1096d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.622+0000 7f0904454740 1 --2- 172.21.15.172:0/1246806628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a109565d0 0x561a1096d310 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f08e40206b0 tx=0x7f08e4022fb0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.622+0000 7f0904454740 1 -- 172.21.15.172:0/1246806628 shutdown_connections 2024-09-10T09:00:46.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.622+0000 7f0904454740 1 --2- 172.21.15.172:0/1246806628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a10961370 0x561a10978940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.622+0000 7f0904454740 1 --2- 172.21.15.172:0/1246806628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 0x561a1095d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.622+0000 7f0904454740 1 --2- 172.21.15.172:0/1246806628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a109565d0 0x561a1096d310 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.622+0000 7f0904454740 1 -- 172.21.15.172:0/1246806628 >> 172.21.15.172:0/1246806628 conn(0x561a1091f310 msgr2=0x561a1095c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.622+0000 7f0904454740 1 -- 172.21.15.172:0/1246806628 shutdown_connections 2024-09-10T09:00:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.622+0000 7f0904454740 1 -- 172.21.15.172:0/1246806628 wait complete. 2024-09-10T09:00:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.622+0000 7f0904454740 1 -- start start 2024-09-10T09:00:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.622+0000 7f0904454740 1 Processor -- start 2024-09-10T09:00:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.623+0000 7f0904454740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a109565d0 0x561a10969710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.623+0000 7f0904454740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 0x561a10969c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.623+0000 7f0904454740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a10961370 0x561a1095a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.623+0000 7f0904454740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1affd0 con 0x561a10961370 2024-09-10T09:00:46.628 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.623+0000 7f0904454740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1af900 con 0x561a10957b40 2024-09-10T09:00:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.623+0000 7f0904454740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc1acae0 con 0x561a109565d0 2024-09-10T09:00:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.623+0000 7f08fb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 0x561a10969c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.623+0000 7f08f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a10961370 0x561a1095a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.623+0000 7f08fb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 0x561a10969c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:53412/0 (socket says 172.21.15.172:53412) 2024-09-10T09:00:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.623+0000 7f08fb7fe640 1 -- 172.21.15.172:0/3414796916 learned_addr learned my addr 172.21.15.172:0/3414796916 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.623+0000 7f0904454740 10 client.0 _ll_register_callbacks cb 0x7f08e4011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.623+0000 7f08fb7fe640 1 -- 172.21.15.172:0/3414796916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a109565d0 msgr2=0x561a10969710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.623+0000 7f08fb7fe640 1 --2- 172.21.15.172:0/3414796916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a109565d0 0x561a10969710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.623+0000 7f08fb7fe640 1 -- 172.21.15.172:0/3414796916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a10961370 msgr2=0x561a1095a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.624+0000 7f08fb7fe640 1 --2- 172.21.15.172:0/3414796916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a10961370 0x561a1095a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.624+0000 7f08fb7fe640 1 -- 172.21.15.172:0/3414796916 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a10959120 con 0x561a10957b40 2024-09-10T09:00:46.629 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.624+0000 7f08f3fff640 1 --2- 172.21.15.172:0/3414796916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a10961370 0x561a1095a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:00:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.624+0000 7f08fb7fe640 1 --2- 172.21.15.172:0/3414796916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 0x561a10969c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x561a1095d8b0 tx=0x7f08ec01da40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf9637c1dff7c980 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.624+0000 7f08daffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:00:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.624+0000 7f08daffd640 1 -- 172.21.15.172:0/3414796916 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec004220 con 0x561a10957b40 2024-09-10T09:00:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.624+0000 7f0904454740 1 -- 172.21.15.172:0/3414796916 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08e401a040 con 0x561a10957b40 2024-09-10T09:00:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.624+0000 7f08daffd640 1 -- 172.21.15.172:0/3414796916 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f08ec0043c0 con 0x561a10957b40 2024-09-10T09:00:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.624+0000 7f08daffd640 1 -- 172.21.15.172:0/3414796916 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f08ec02e050 con 0x561a10957b40 2024-09-10T09:00:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.624+0000 7f0904454740 10 client.28217 fetch_fsmap learned FSMap version 42 2024-09-10T09:00:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.624+0000 7f0904454740 1 -- 172.21.15.172:0/3414796916 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561a109db340 con 0x561a10957b40 2024-09-10T09:00:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.624+0000 7f08daffd640 1 -- 172.21.15.172:0/3414796916 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08e401a040 con 0x561a10957b40 2024-09-10T09:00:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.625+0000 7f08daffd640 1 -- 172.21.15.172:0/3414796916 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7f08ec003b10 con 0x561a10957b40 2024-09-10T09:00:46.630 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.625+0000 7f0904454740 10 client.28217 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:00:46.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.625+0000 7f0904454740 10 client.28217 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T09:00:46.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.625+0000 7f0904454740 20 client.28217 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:46.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.625+0000 7f0904454740 1 --2- 172.21.15.172:0/3414796916 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x561a109dbaa0 0x561a109ddee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:46.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.625+0000 7f0904454740 4 client.28217 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:00:46.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.625+0000 7f0904454740 1 -- 172.21.15.172:0/3414796916 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a108c3d70 con 0x561a109dbaa0 2024-09-10T09:00:46.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.625+0000 7f08f3fff640 1 --2- 172.21.15.172:0/3414796916 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x561a109dbaa0 0x561a109ddee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:46.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.625+0000 7f08f3fff640 1 --2- 172.21.15.172:0/3414796916 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x561a109dbaa0 0x561a109ddee0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=56fab17eed4cab87 server_cookie=a49da5b4415fb813 in_seq=0 out_seq=0 2024-09-10T09:00:46.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.625+0000 7f08daffd640 10 client.28217 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T09:00:46.631 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:46 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2179471208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:00:46.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.625+0000 7f08daffd640 1 -- 172.21.15.172:0/3414796916 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a108c3d70 con 0x561a109dbaa0 2024-09-10T09:00:46.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.625+0000 7f08daffd640 10 client.28217 handle_command_reply: tid=0 2024-09-10T09:00:46.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.626+0000 7f0904454740 1 client.28217 shutdown 2024-09-10T09:00:46.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.626+0000 7f0904454740 1 -- 172.21.15.172:0/3414796916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 msgr2=0x561a10969c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:46.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.626+0000 7f0904454740 1 --2- 172.21.15.172:0/3414796916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 0x561a10969c50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x561a1095d8b0 tx=0x7f08ec01da40 comp rx=0 tx=0).stop 2024-09-10T09:00:46.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.626+0000 7f0904454740 1 -- 172.21.15.172:0/3414796916 shutdown_connections 2024-09-10T09:00:46.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.626+0000 7f0904454740 1 --2- 172.21.15.172:0/3414796916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a10961370 0x561a1095a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.626+0000 7f0904454740 1 --2- 172.21.15.172:0/3414796916 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x561a109dbaa0 0x561a109ddee0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.626+0000 7f0904454740 1 --2- 172.21.15.172:0/3414796916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a10957b40 0x561a10969c50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.626+0000 7f0904454740 1 --2- 172.21.15.172:0/3414796916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561a109565d0 0x561a10969710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.626+0000 7f08daffd640 0 client.28217 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T09:00:46.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.626+0000 7f0904454740 1 -- 172.21.15.172:0/3414796916 >> 172.21.15.172:0/3414796916 conn(0x561a1091f310 msgr2=0x561a1095d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:46.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.626+0000 7f0904454740 1 -- 172.21.15.172:0/3414796916 shutdown_connections 2024-09-10T09:00:46.633 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.626+0000 7f0904454740 1 -- 172.21.15.172:0/3414796916 wait complete. 2024-09-10T09:00:46.634 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.626+0000 7f0904454740 20 client.28217 trim_cache size 0 max 16384 2024-09-10T09:00:46.634 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:00:46.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.627+0000 7f090207b640 1 -- 172.21.15.172:0/384294516 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08e80a2c00 msgr2=0x7f08e80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:46.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.627+0000 7f090207b640 1 --2- 172.21.15.172:0/384294516 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08e80a2c00 0x7f08e80a50c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f08f40b9490 tx=0x7f08f40b1fa0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.627+0000 7f090207b640 1 -- 172.21.15.172:0/384294516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc106a70 msgr2=0x7f08fc1a4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:46.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.627+0000 7f090207b640 1 --2- 172.21.15.172:0/384294516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc106a70 0x7f08fc1a4250 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f08f4005cc0 tx=0x7f08f4003ba0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.627+0000 7f090207b640 1 -- 172.21.15.172:0/384294516 shutdown_connections 2024-09-10T09:00:46.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.627+0000 7f090207b640 1 --2- 172.21.15.172:0/384294516 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08e80a2c00 0x7f08e80a50c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.627+0000 7f090207b640 1 --2- 172.21.15.172:0/384294516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08fc106a70 0x7f08fc1a4250 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.627+0000 7f090207b640 1 --2- 172.21.15.172:0/384294516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08fc101410 0x7f08fc199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.627+0000 7f090207b640 1 --2- 172.21.15.172:0/384294516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08fc100a70 0x7f08fc1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:46.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.627+0000 7f090207b640 1 -- 172.21.15.172:0/384294516 >> 172.21.15.172:0/384294516 conn(0x7f08fc0fc7d0 msgr2=0x7f08fc0fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:46.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.627+0000 7f090207b640 1 -- 172.21.15.172:0/384294516 shutdown_connections 2024-09-10T09:00:46.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:46.627+0000 7f090207b640 1 -- 172.21.15.172:0/384294516 wait complete. 2024-09-10T09:00:46.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:46 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2179471208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:00:46.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:46 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2179471208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:00:47.328 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:00:47.343 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:00:47.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:47 smithi172 ceph-mon[30854]: pgmap v787: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 97 KiB/s wr, 34 op/s 2024-09-10T09:00:47.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:47 smithi174 ceph-mon[34068]: pgmap v787: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 97 KiB/s wr, 34 op/s 2024-09-10T09:00:47.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:47 smithi191 ceph-mon[35314]: pgmap v787: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 97 KiB/s wr, 34 op/s 2024-09-10T09:00:47.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.872+0000 7f83750ab640 1 -- 172.21.15.172:0/2025198929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8370100a80 msgr2=0x7f8370100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:47.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.872+0000 7f83750ab640 1 --2- 172.21.15.172:0/2025198929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8370100a80 0x7f8370100ee0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f8358009930 tx=0x7f835802a420 comp rx=0 tx=0).stop 2024-09-10T09:00:47.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.872+0000 7f83750ab640 1 -- 172.21.15.172:0/2025198929 shutdown_connections 2024-09-10T09:00:47.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.872+0000 7f83750ab640 1 --2- 172.21.15.172:0/2025198929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8370101420 0x7f837010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.872+0000 7f83750ab640 1 --2- 172.21.15.172:0/2025198929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8370100a80 0x7f8370100ee0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.872+0000 7f83750ab640 1 --2- 172.21.15.172:0/2025198929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8370106a80 0x7f8370106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.872+0000 7f83750ab640 1 -- 172.21.15.172:0/2025198929 >> 172.21.15.172:0/2025198929 conn(0x7f83700fc7a0 msgr2=0x7f83700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:47.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.872+0000 7f83750ab640 1 -- 172.21.15.172:0/2025198929 shutdown_connections 2024-09-10T09:00:47.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.873+0000 7f83750ab640 1 -- 172.21.15.172:0/2025198929 wait complete. 2024-09-10T09:00:47.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.874+0000 7f83750ab640 1 Processor -- start 2024-09-10T09:00:47.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.874+0000 7f83750ab640 1 -- start start 2024-09-10T09:00:47.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.874+0000 7f83750ab640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8370100a80 0x7f83701972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.874+0000 7f83750ab640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8370101420 0x7f8370197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.874+0000 7f83750ab640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8370106a80 0x7f837019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.874+0000 7f83750ab640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8370110d30 con 0x7f8370100a80 2024-09-10T09:00:47.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.874+0000 7f83750ab640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8370110bb0 con 0x7f8370106a80 2024-09-10T09:00:47.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.874+0000 7f83750ab640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8370110eb0 con 0x7f8370101420 2024-09-10T09:00:47.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.874+0000 7f836ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8370100a80 0x7f83701972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.874+0000 7f836ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8370100a80 0x7f83701972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34404/0 (socket says 172.21.15.172:34404) 2024-09-10T09:00:47.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.874+0000 7f836ed9d640 1 -- 172.21.15.172:0/3698931693 learned_addr learned my addr 172.21.15.172:0/3698931693 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:47.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.874+0000 7f836f59e640 1 --2- 172.21.15.172:0/3698931693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8370106a80 0x7f837019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.874+0000 7f8367fff640 1 --2- 172.21.15.172:0/3698931693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8370101420 0x7f8370197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.874+0000 7f836ed9d640 1 -- 172.21.15.172:0/3698931693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8370101420 msgr2=0x7f8370197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:47.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.874+0000 7f836ed9d640 1 --2- 172.21.15.172:0/3698931693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8370101420 0x7f8370197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.874+0000 7f836ed9d640 1 -- 172.21.15.172:0/3698931693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8370106a80 msgr2=0x7f837019bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:47.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.874+0000 7f836ed9d640 1 --2- 172.21.15.172:0/3698931693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8370106a80 0x7f837019bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.875+0000 7f836ed9d640 1 -- 172.21.15.172:0/3698931693 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837019c280 con 0x7f8370100a80 2024-09-10T09:00:47.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.875+0000 7f836ed9d640 1 --2- 172.21.15.172:0/3698931693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8370100a80 0x7f83701972d0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f836000d950 tx=0x7f836000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:47.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.875+0000 7f836cd99640 1 -- 172.21.15.172:0/3698931693 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360017030 con 0x7f8370100a80 2024-09-10T09:00:47.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.875+0000 7f836cd99640 1 -- 172.21.15.172:0/3698931693 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8360004020 con 0x7f8370100a80 2024-09-10T09:00:47.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.875+0000 7f83750ab640 1 -- 172.21.15.172:0/3698931693 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f837019c570 con 0x7f8370100a80 2024-09-10T09:00:47.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.875+0000 7f836cd99640 1 -- 172.21.15.172:0/3698931693 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83600047a0 con 0x7f8370100a80 2024-09-10T09:00:47.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.875+0000 7f83750ab640 1 -- 172.21.15.172:0/3698931693 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83701a3e50 con 0x7f8370100a80 2024-09-10T09:00:47.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.877+0000 7f836cd99640 1 -- 172.21.15.172:0/3698931693 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f83600041d0 con 0x7f8370100a80 2024-09-10T09:00:47.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.878+0000 7f8377484740 1 Processor -- start 2024-09-10T09:00:47.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.878+0000 7f8377484740 1 -- start start 2024-09-10T09:00:47.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.878+0000 7f8377484740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8bdb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.878+0000 7f8377484740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8bdb820 0x5595d8be7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.878+0000 7f8377484740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8be8450 0x5595d8bea840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.878+0000 7f8377484740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5595d8b40620 con 0x5595d8bdaf00 2024-09-10T09:00:47.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.878+0000 7f8377484740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5595d8ae7e50 con 0x5595d8bdb820 2024-09-10T09:00:47.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.878+0000 7f8377484740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5595d8b3c5e0 con 0x5595d8be8450 2024-09-10T09:00:47.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.878+0000 7f836f59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8bdb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.878+0000 7f836f59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8bdb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34408/0 (socket says 172.21.15.172:34408) 2024-09-10T09:00:47.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.878+0000 7f836cd99640 1 --2- 172.21.15.172:0/3698931693 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f835c0a2b80 0x7f835c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.878+0000 7f836f59e640 1 -- 172.21.15.172:0/2445098500 learned_addr learned my addr 172.21.15.172:0/2445098500 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:47.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.879+0000 7f836f59e640 1 --2- 172.21.15.172:0/3698931693 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f835c0a2b80 0x7f835c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.879+0000 7f8367fff640 1 --2- 172.21.15.172:0/2445098500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8be8450 0x5595d8bea840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.879+0000 7f836cd99640 1 -- 172.21.15.172:0/3698931693 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f83600ced70 con 0x7f8370100a80 2024-09-10T09:00:47.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.879+0000 7f836ed9d640 1 --2- 172.21.15.172:0/2445098500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8bdb820 0x5595d8be7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.879+0000 7f836f59e640 1 -- 172.21.15.172:0/2445098500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8be8450 msgr2=0x5595d8bea840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:47.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.879+0000 7f836f59e640 1 --2- 172.21.15.172:0/2445098500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8be8450 0x5595d8bea840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.879+0000 7f836f59e640 1 -- 172.21.15.172:0/2445098500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8bdb820 msgr2=0x5595d8be7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:47.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.879+0000 7f836f59e640 1 --2- 172.21.15.172:0/2445098500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8bdb820 0x5595d8be7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.879+0000 7f836f59e640 1 -- 172.21.15.172:0/2445098500 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595d8ad8d40 con 0x5595d8bdaf00 2024-09-10T09:00:47.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.879+0000 7f836f59e640 1 --2- 172.21.15.172:0/2445098500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8bdb2e0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f8368004140 tx=0x7f8368003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330c5e4215cfbcbd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:47.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.879+0000 7f83657fa640 1 -- 172.21.15.172:0/2445098500 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368012e60 con 0x5595d8bdaf00 2024-09-10T09:00:47.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.879+0000 7f83657fa640 1 -- 172.21.15.172:0/2445098500 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8368023070 con 0x5595d8bdaf00 2024-09-10T09:00:47.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.879+0000 7f83657fa640 1 -- 172.21.15.172:0/2445098500 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368003ad0 con 0x5595d8bdaf00 2024-09-10T09:00:47.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.880+0000 7f8377484740 1 -- 172.21.15.172:0/2445098500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 msgr2=0x5595d8bdb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:47.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.880+0000 7f836f59e640 1 --2- 172.21.15.172:0/3698931693 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f835c0a2b80 0x7f835c0a5040 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8368022000 tx=0x7f8368020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:47.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.880+0000 7f8377484740 1 --2- 172.21.15.172:0/2445098500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8bdb2e0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f8368004140 tx=0x7f8368003f90 comp rx=0 tx=0).stop 2024-09-10T09:00:47.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.880+0000 7f8377484740 1 -- 172.21.15.172:0/2445098500 shutdown_connections 2024-09-10T09:00:47.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.880+0000 7f8377484740 1 --2- 172.21.15.172:0/2445098500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8be8450 0x5595d8bea840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.880+0000 7f8377484740 1 --2- 172.21.15.172:0/2445098500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8bdb820 0x5595d8be7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.880+0000 7f8377484740 1 --2- 172.21.15.172:0/2445098500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8bdb2e0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.880+0000 7f8377484740 1 -- 172.21.15.172:0/2445098500 >> 172.21.15.172:0/2445098500 conn(0x5595d8bd75f0 msgr2=0x5595d8bd79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:47.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.881+0000 7f8377484740 1 -- 172.21.15.172:0/2445098500 shutdown_connections 2024-09-10T09:00:47.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.881+0000 7f8377484740 1 -- 172.21.15.172:0/2445098500 wait complete. 2024-09-10T09:00:47.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.881+0000 7f8377484740 1 -- start start 2024-09-10T09:00:47.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.881+0000 7f8377484740 1 Processor -- start 2024-09-10T09:00:47.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.881+0000 7f8377484740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8bdaf00 0x5595d8be36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.881+0000 7f8377484740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8bdb820 0x5595d8be3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.881+0000 7f8377484740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8be8450 0x5595d8bf1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.881+0000 7f8377484740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5595d8bdddc0 con 0x5595d8be8450 2024-09-10T09:00:47.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.881+0000 7f8377484740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5595d8ac8f10 con 0x5595d8bdb820 2024-09-10T09:00:47.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.881+0000 7f8377484740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5595d8bddf40 con 0x5595d8bdaf00 2024-09-10T09:00:47.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.881+0000 7f8377484740 10 client.0 _ll_register_callbacks cb 0x5595d8aecdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:47.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.881+0000 7f836f59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8be8450 0x5595d8bf1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.881+0000 7f836f59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8be8450 0x5595d8bf1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34412/0 (socket says 172.21.15.172:34412) 2024-09-10T09:00:47.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.881+0000 7f836f59e640 1 -- 172.21.15.172:0/671341920 learned_addr learned my addr 172.21.15.172:0/671341920 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:47.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.881+0000 7f8367fff640 1 --2- 172.21.15.172:0/671341920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8bdb820 0x5595d8be3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.882+0000 7f836ed9d640 1 --2- 172.21.15.172:0/671341920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8bdaf00 0x5595d8be36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.882+0000 7f836f59e640 1 -- 172.21.15.172:0/671341920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8bdaf00 msgr2=0x5595d8be36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:47.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.882+0000 7f836f59e640 1 --2- 172.21.15.172:0/671341920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8bdaf00 0x5595d8be36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.882+0000 7f836f59e640 1 -- 172.21.15.172:0/671341920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8bdb820 msgr2=0x5595d8be3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:47.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.882+0000 7f836f59e640 1 --2- 172.21.15.172:0/671341920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8bdb820 0x5595d8be3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.882+0000 7f836f59e640 1 -- 172.21.15.172:0/671341920 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595d884f580 con 0x5595d8be8450 2024-09-10T09:00:47.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.882+0000 7f836ed9d640 1 --2- 172.21.15.172:0/671341920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8bdaf00 0x5595d8be36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:00:47.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.882+0000 7f836f59e640 1 --2- 172.21.15.172:0/671341920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8be8450 0x5595d8bf1f40 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f83680127c0 tx=0x7f836801d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2c88e7cef064be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:47.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.882+0000 7f83567fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:00:47.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.882+0000 7f83567fc640 1 -- 172.21.15.172:0/671341920 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368035020 con 0x5595d8be8450 2024-09-10T09:00:47.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.882+0000 7f83567fc640 1 -- 172.21.15.172:0/671341920 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8368003a10 con 0x5595d8be8450 2024-09-10T09:00:47.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.882+0000 7f8377484740 1 -- 172.21.15.172:0/671341920 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8368023070 con 0x5595d8be8450 2024-09-10T09:00:47.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.882+0000 7f83567fc640 1 -- 172.21.15.172:0/671341920 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f836803d050 con 0x5595d8be8450 2024-09-10T09:00:47.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.882+0000 7f8377484740 10 client.19461 fetch_fsmap learned FSMap version 42 2024-09-10T09:00:47.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.882+0000 7f8377484740 1 -- 172.21.15.172:0/671341920 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5595d8919960 con 0x5595d8be8450 2024-09-10T09:00:47.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.882+0000 7f83567fc640 1 -- 172.21.15.172:0/671341920 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8368023070 con 0x5595d8be8450 2024-09-10T09:00:47.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.883+0000 7f83567fc640 1 -- 172.21.15.172:0/671341920 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7f836802e070 con 0x5595d8be8450 2024-09-10T09:00:47.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.883+0000 7f8377484740 10 client.19461 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:00:47.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.883+0000 7f8377484740 10 client.19461 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:00:47.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.883+0000 7f8377484740 20 client.19461 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:47.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.883+0000 7f8377484740 1 --2- 172.21.15.172:0/671341920 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5595d8bf5cc0 0x5595d8bf80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.883+0000 7f8377484740 4 client.19461 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:00:47.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.883+0000 7f8377484740 1 -- 172.21.15.172:0/671341920 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595d8b4db90 con 0x5595d8bf5cc0 2024-09-10T09:00:47.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.883+0000 7f836ed9d640 1 --2- 172.21.15.172:0/671341920 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5595d8bf5cc0 0x5595d8bf80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.883+0000 7f83567fc640 10 client.19461 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:47.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.883+0000 7f836ed9d640 1 --2- 172.21.15.172:0/671341920 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5595d8bf5cc0 0x5595d8bf80b0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9328a58c56a01d3 server_cookie=354aacbf1dfc662c in_seq=0 out_seq=0 2024-09-10T09:00:47.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.884+0000 7f83567fc640 1 -- 172.21.15.172:0/671341920 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595d8b4db90 con 0x5595d8bf5cc0 2024-09-10T09:00:47.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.884+0000 7f83567fc640 10 client.19461 handle_command_reply: tid=0 2024-09-10T09:00:47.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.884+0000 7f8377484740 1 client.19461 shutdown 2024-09-10T09:00:47.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.884+0000 7f8377484740 1 -- 172.21.15.172:0/671341920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8be8450 msgr2=0x5595d8bf1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:47.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.884+0000 7f8377484740 1 --2- 172.21.15.172:0/671341920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8be8450 0x5595d8bf1f40 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f83680127c0 tx=0x7f836801d450 comp rx=0 tx=0).stop 2024-09-10T09:00:47.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.884+0000 7f8377484740 1 -- 172.21.15.172:0/671341920 shutdown_connections 2024-09-10T09:00:47.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.884+0000 7f8377484740 1 --2- 172.21.15.172:0/671341920 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5595d8bf5cc0 0x5595d8bf80b0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.884+0000 7f8377484740 1 --2- 172.21.15.172:0/671341920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8be8450 0x5595d8bf1f40 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.884+0000 7f8377484740 1 --2- 172.21.15.172:0/671341920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8bdb820 0x5595d8be3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.884+0000 7f8377484740 1 --2- 172.21.15.172:0/671341920 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8bdaf00 0x5595d8be36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.884+0000 7f83567fc640 0 client.19461 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:47.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.884+0000 7f8377484740 1 -- 172.21.15.172:0/671341920 >> 172.21.15.172:0/671341920 conn(0x5595d8bd75f0 msgr2=0x5595d8bdd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:47.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.884+0000 7f8377484740 1 -- 172.21.15.172:0/671341920 shutdown_connections 2024-09-10T09:00:47.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.884+0000 7f8377484740 1 -- 172.21.15.172:0/671341920 wait complete. 2024-09-10T09:00:47.918 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.884+0000 7f8377484740 20 client.19461 trim_cache size 0 max 16384 2024-09-10T09:00:47.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.901+0000 7f8377484740 1 Processor -- start 2024-09-10T09:00:47.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.901+0000 7f8377484740 1 -- start start 2024-09-10T09:00:47.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.902+0000 7f8377484740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8be7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.902+0000 7f8377484740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8bdb820 0x5595d8bfd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.902+0000 7f8377484740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8be8450 0x5595d8bfdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.902+0000 7f8377484740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83701b0db0 con 0x5595d8bdaf00 2024-09-10T09:00:47.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.902+0000 7f8377484740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83701b2950 con 0x5595d8be8450 2024-09-10T09:00:47.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.902+0000 7f8377484740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f83701b1480 con 0x5595d8bdb820 2024-09-10T09:00:47.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.902+0000 7f836ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8be7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.902+0000 7f836ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8be7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34416/0 (socket says 172.21.15.172:34416) 2024-09-10T09:00:47.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.902+0000 7f836ed9d640 1 -- 172.21.15.172:0/4266203077 learned_addr learned my addr 172.21.15.172:0/4266203077 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:47.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.902+0000 7f8367fff640 1 --2- 172.21.15.172:0/4266203077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8bdb820 0x5595d8bfd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.902+0000 7f836f59e640 1 --2- 172.21.15.172:0/4266203077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8be8450 0x5595d8bfdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.902+0000 7f836ed9d640 1 -- 172.21.15.172:0/4266203077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8bdb820 msgr2=0x5595d8bfd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:47.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.902+0000 7f836ed9d640 1 --2- 172.21.15.172:0/4266203077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8bdb820 0x5595d8bfd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.902+0000 7f836ed9d640 1 -- 172.21.15.172:0/4266203077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8be8450 msgr2=0x5595d8bfdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:47.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.902+0000 7f836ed9d640 1 --2- 172.21.15.172:0/4266203077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8be8450 0x5595d8bfdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.902+0000 7f836ed9d640 1 -- 172.21.15.172:0/4266203077 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595d8be4a50 con 0x5595d8bdaf00 2024-09-10T09:00:47.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.902+0000 7f836ed9d640 1 --2- 172.21.15.172:0/4266203077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8be7650 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f83600cc4b0 tx=0x7f8360005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19dbed790453ac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:47.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.903+0000 7f8364ff9640 1 -- 172.21.15.172:0/4266203077 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83600b4070 con 0x5595d8bdaf00 2024-09-10T09:00:47.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.903+0000 7f8364ff9640 1 -- 172.21.15.172:0/4266203077 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8360014040 con 0x5595d8bdaf00 2024-09-10T09:00:47.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.903+0000 7f8364ff9640 1 -- 172.21.15.172:0/4266203077 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360084070 con 0x5595d8bdaf00 2024-09-10T09:00:47.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.903+0000 7f8377484740 1 -- 172.21.15.172:0/4266203077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 msgr2=0x5595d8be7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:47.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.903+0000 7f8377484740 1 --2- 172.21.15.172:0/4266203077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8be7650 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f83600cc4b0 tx=0x7f8360005d50 comp rx=0 tx=0).stop 2024-09-10T09:00:47.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f8377484740 1 -- 172.21.15.172:0/4266203077 shutdown_connections 2024-09-10T09:00:47.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f8377484740 1 --2- 172.21.15.172:0/4266203077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8be8450 0x5595d8bfdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f8377484740 1 --2- 172.21.15.172:0/4266203077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8bdb820 0x5595d8bfd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f8377484740 1 --2- 172.21.15.172:0/4266203077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8be7650 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f8377484740 1 -- 172.21.15.172:0/4266203077 >> 172.21.15.172:0/4266203077 conn(0x5595d8bd75f0 msgr2=0x5595d8be6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:47.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f8377484740 1 -- 172.21.15.172:0/4266203077 shutdown_connections 2024-09-10T09:00:47.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f8377484740 1 -- 172.21.15.172:0/4266203077 wait complete. 2024-09-10T09:00:47.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f8377484740 1 -- start start 2024-09-10T09:00:47.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f8377484740 1 Processor -- start 2024-09-10T09:00:47.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f8377484740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8bedac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f8377484740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8bdb820 0x5595d8bee000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f8377484740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8be8450 0x5595d8c5f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f8377484740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83701b2280 con 0x5595d8bdaf00 2024-09-10T09:00:47.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f8377484740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83701b1bb0 con 0x5595d8be8450 2024-09-10T09:00:47.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f8377484740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f83701aed90 con 0x5595d8bdb820 2024-09-10T09:00:47.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f8377484740 10 client.0 _ll_register_callbacks cb 0x7f8368003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:47.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f836ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8bedac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f836ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8bedac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34424/0 (socket says 172.21.15.172:34424) 2024-09-10T09:00:47.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f836ed9d640 1 -- 172.21.15.172:0/979986970 learned_addr learned my addr 172.21.15.172:0/979986970 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:47.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.904+0000 7f836f59e640 1 --2- 172.21.15.172:0/979986970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8be8450 0x5595d8c5f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.905+0000 7f8367fff640 1 --2- 172.21.15.172:0/979986970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8bdb820 0x5595d8bee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.905+0000 7f836ed9d640 1 -- 172.21.15.172:0/979986970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8bdb820 msgr2=0x5595d8bee000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:47.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.905+0000 7f836ed9d640 1 --2- 172.21.15.172:0/979986970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8bdb820 0x5595d8bee000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.905+0000 7f836ed9d640 1 -- 172.21.15.172:0/979986970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8be8450 msgr2=0x5595d8c5f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:47.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.905+0000 7f836ed9d640 1 --2- 172.21.15.172:0/979986970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8be8450 0x5595d8c5f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.905+0000 7f836ed9d640 1 -- 172.21.15.172:0/979986970 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595d8c5fd80 con 0x5595d8bdaf00 2024-09-10T09:00:47.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.905+0000 7f836ed9d640 1 --2- 172.21.15.172:0/979986970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8bedac0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f8360097be0 tx=0x7f8360005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dac0c059e2035d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:47.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.905+0000 7f83567fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:00:47.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.905+0000 7f83567fc640 1 -- 172.21.15.172:0/979986970 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360084040 con 0x5595d8bdaf00 2024-09-10T09:00:47.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.905+0000 7f83567fc640 1 -- 172.21.15.172:0/979986970 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8360097880 con 0x5595d8bdaf00 2024-09-10T09:00:47.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.905+0000 7f8377484740 1 -- 172.21.15.172:0/979986970 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8360014040 con 0x5595d8bdaf00 2024-09-10T09:00:47.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.905+0000 7f83567fc640 1 -- 172.21.15.172:0/979986970 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f83600b4040 con 0x5595d8bdaf00 2024-09-10T09:00:47.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.905+0000 7f8377484740 10 client.19467 fetch_fsmap learned FSMap version 42 2024-09-10T09:00:47.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.905+0000 7f8377484740 1 -- 172.21.15.172:0/979986970 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5595d8c610a0 con 0x5595d8bdaf00 2024-09-10T09:00:47.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.906+0000 7f83567fc640 1 -- 172.21.15.172:0/979986970 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8360014040 con 0x5595d8bdaf00 2024-09-10T09:00:47.923 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.906+0000 7f83567fc640 1 -- 172.21.15.172:0/979986970 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7f83600a39f0 con 0x5595d8bdaf00 2024-09-10T09:00:47.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.906+0000 7f8377484740 10 client.19467 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:00:47.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.906+0000 7f8377484740 10 client.19467 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:00:47.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.906+0000 7f8377484740 20 client.19467 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:47.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.906+0000 7f8377484740 1 --2- 172.21.15.172:0/979986970 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5595d8c61760 0x5595d8c63b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:47.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.906+0000 7f8377484740 4 client.19467 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7787905e", "release": true, "await": true}] 2024-09-10T09:00:47.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.906+0000 7f8377484740 1 -- 172.21.15.172:0/979986970 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7787905e", "release": true, "await": true}) -- 0x5595d8b480a0 con 0x5595d8c61760 2024-09-10T09:00:47.924 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.906+0000 7f8367fff640 1 --2- 172.21.15.172:0/979986970 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5595d8c61760 0x5595d8c63b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:47.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.907+0000 7f83567fc640 10 client.19467 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:47.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.907+0000 7f8367fff640 1 --2- 172.21.15.172:0/979986970 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5595d8c61760 0x5595d8c63b80 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b39e7df6066bf76 server_cookie=3de1e9087d8d78bb in_seq=0 out_seq=0 2024-09-10T09:00:47.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.923+0000 7f83567fc640 1 -- 172.21.15.172:0/979986970 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5595d8b480a0 con 0x5595d8c61760 2024-09-10T09:00:47.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.923+0000 7f83567fc640 10 client.19467 handle_command_reply: tid=0 2024-09-10T09:00:47.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.923+0000 7f8377484740 1 client.19467 shutdown 2024-09-10T09:00:47.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.923+0000 7f8377484740 1 -- 172.21.15.172:0/979986970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 msgr2=0x5595d8bedac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:47.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.923+0000 7f8377484740 1 --2- 172.21.15.172:0/979986970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8bedac0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f8360097be0 tx=0x7f8360005af0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.924+0000 7f8377484740 1 -- 172.21.15.172:0/979986970 shutdown_connections 2024-09-10T09:00:47.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.924+0000 7f8377484740 1 --2- 172.21.15.172:0/979986970 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5595d8c61760 0x5595d8c63b80 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.924+0000 7f8377484740 1 --2- 172.21.15.172:0/979986970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595d8be8450 0x5595d8c5f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.924+0000 7f8377484740 1 --2- 172.21.15.172:0/979986970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5595d8bdb820 0x5595d8bee000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.924+0000 7f83567fc640 0 client.19467 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:47.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.924+0000 7f8377484740 1 --2- 172.21.15.172:0/979986970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5595d8bdaf00 0x5595d8bedac0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.924+0000 7f8377484740 1 -- 172.21.15.172:0/979986970 >> 172.21.15.172:0/979986970 conn(0x5595d8bd75f0 msgr2=0x5595d8be7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:47.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.924+0000 7f8377484740 1 -- 172.21.15.172:0/979986970 shutdown_connections 2024-09-10T09:00:47.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.924+0000 7f8377484740 1 -- 172.21.15.172:0/979986970 wait complete. 2024-09-10T09:00:47.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.924+0000 7f8377484740 20 client.19467 trim_cache size 0 max 16384 2024-09-10T09:00:47.927 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:00:47.927 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 42, 2024-09-10T09:00:47.927 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T09:00:47.927 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 48, 2024-09-10T09:00:47.927 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T09:00:47.927 INFO:teuthology.orchestra.run.smithi172.stdout: "7787905e": { 2024-09-10T09:00:47.927 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 48, 2024-09-10T09:00:47.928 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T09:00:47.928 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T09:00:47.928 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T09:00:47.928 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T09:00:47.928 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:00:47.928 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T09:00:47.928 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 141.5, 2024-09-10T09:00:47.928 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T09:00:47.928 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T09:00:47.928 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T09:00:47.928 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T09:00:47.928 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T09:00:47.928 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T09:00:47.928 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:00:47.929 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:00:47.929 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:00:47.929 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:00:47.929 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:00:47.929 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:00:47.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.925+0000 7f83750ab640 1 -- 172.21.15.172:0/3698931693 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f835c0a2b80 msgr2=0x7f835c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:47.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.925+0000 7f83750ab640 1 --2- 172.21.15.172:0/3698931693 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f835c0a2b80 0x7f835c0a5040 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8368022000 tx=0x7f8368020000 comp rx=0 tx=0).stop 2024-09-10T09:00:47.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.925+0000 7f83750ab640 1 -- 172.21.15.172:0/3698931693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8370100a80 msgr2=0x7f83701972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:47.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.925+0000 7f83750ab640 1 --2- 172.21.15.172:0/3698931693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8370100a80 0x7f83701972d0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f836000d950 tx=0x7f836000de10 comp rx=0 tx=0).stop 2024-09-10T09:00:47.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.926+0000 7f83750ab640 1 -- 172.21.15.172:0/3698931693 shutdown_connections 2024-09-10T09:00:47.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.926+0000 7f83750ab640 1 --2- 172.21.15.172:0/3698931693 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f835c0a2b80 0x7f835c0a5040 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.926+0000 7f83750ab640 1 --2- 172.21.15.172:0/3698931693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8370106a80 0x7f837019bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.926+0000 7f83750ab640 1 --2- 172.21.15.172:0/3698931693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8370101420 0x7f8370197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.926+0000 7f83750ab640 1 --2- 172.21.15.172:0/3698931693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8370100a80 0x7f83701972d0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:47.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.926+0000 7f83750ab640 1 -- 172.21.15.172:0/3698931693 >> 172.21.15.172:0/3698931693 conn(0x7f83700fc7a0 msgr2=0x7f83700fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:47.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.926+0000 7f83750ab640 1 -- 172.21.15.172:0/3698931693 shutdown_connections 2024-09-10T09:00:47.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:47.926+0000 7f83750ab640 1 -- 172.21.15.172:0/3698931693 wait complete. 2024-09-10T09:00:48.745 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7787905e', total seconds elapsed: 87.5 2024-09-10T09:00:48.745 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.5 2024-09-10T09:00:48.745 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:00:48.762 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:00:49.628 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:49 smithi172 ceph-mon[30854]: pgmap v788: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 663 KiB/s wr, 175 op/s 2024-09-10T09:00:49.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:49 smithi174 ceph-mon[34068]: pgmap v788: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 663 KiB/s wr, 175 op/s 2024-09-10T09:00:49.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:49 smithi191 ceph-mon[35314]: pgmap v788: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 663 KiB/s wr, 175 op/s 2024-09-10T09:00:50.787 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:00:50.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.525+0000 7f6e6a6d7640 1 -- 172.21.15.172:0/1858606100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e64074c90 msgr2=0x7f6e64075390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:50.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.525+0000 7f6e6a6d7640 1 --2- 172.21.15.172:0/1858606100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e64074c90 0x7f6e64075390 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009860 tx=0x7f6e58032b50 comp rx=0 tx=0).stop 2024-09-10T09:00:50.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.526+0000 7f6e6a6d7640 1 -- 172.21.15.172:0/1858606100 shutdown_connections 2024-09-10T09:00:50.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.526+0000 7f6e6a6d7640 1 --2- 172.21.15.172:0/1858606100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e64074c90 0x7f6e64075390 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.526+0000 7f6e6a6d7640 1 --2- 172.21.15.172:0/1858606100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e640742d0 0x7f6e64074750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.526+0000 7f6e6a6d7640 1 --2- 172.21.15.172:0/1858606100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e641098a0 0x7f6e64109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.526+0000 7f6e6a6d7640 1 -- 172.21.15.172:0/1858606100 >> 172.21.15.172:0/1858606100 conn(0x7f6e6406e140 msgr2=0x7f6e6406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:50.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.527+0000 7f6e6a6d7640 1 -- 172.21.15.172:0/1858606100 shutdown_connections 2024-09-10T09:00:50.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.528+0000 7f6e6a6d7640 1 -- 172.21.15.172:0/1858606100 wait complete. 2024-09-10T09:00:50.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.528+0000 7f6e6a6d7640 1 Processor -- start 2024-09-10T09:00:50.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.529+0000 7f6e6a6d7640 1 -- start start 2024-09-10T09:00:50.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.529+0000 7f6e6a6d7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e640742d0 0x7f6e64084ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.529+0000 7f6e6a6d7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e64074c90 0x7f6e6407fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.529+0000 7f6e6a6d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e641098a0 0x7f6e640800b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.529+0000 7f6e6a6d7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6407e6e0 con 0x7f6e641098a0 2024-09-10T09:00:50.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.529+0000 7f6e6a6d7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6407c500 con 0x7f6e640742d0 2024-09-10T09:00:50.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.529+0000 7f6e6a6d7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6407e860 con 0x7f6e64074c90 2024-09-10T09:00:50.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.529+0000 7f6e68c74640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e641098a0 0x7f6e640800b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:50.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.530+0000 7f6e68c74640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e641098a0 0x7f6e640800b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34434/0 (socket says 172.21.15.172:34434) 2024-09-10T09:00:50.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.530+0000 7f6e68c74640 1 -- 172.21.15.172:0/3629406960 learned_addr learned my addr 172.21.15.172:0/3629406960 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:50.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.530+0000 7f6e63fff640 1 --2- 172.21.15.172:0/3629406960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e640742d0 0x7f6e64084ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:50.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.530+0000 7f6e68c74640 1 -- 172.21.15.172:0/3629406960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e64074c90 msgr2=0x7f6e6407fb70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:00:50.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.530+0000 7f6e68c74640 1 --2- 172.21.15.172:0/3629406960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e64074c90 0x7f6e6407fb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.530+0000 7f6e68c74640 1 -- 172.21.15.172:0/3629406960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e640742d0 msgr2=0x7f6e64084ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:50.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.530+0000 7f6e68c74640 1 --2- 172.21.15.172:0/3629406960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e640742d0 0x7f6e64084ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.530+0000 7f6e68c74640 1 -- 172.21.15.172:0/3629406960 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e64080790 con 0x7f6e641098a0 2024-09-10T09:00:50.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.530+0000 7f6e63fff640 1 --2- 172.21.15.172:0/3629406960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e640742d0 0x7f6e64084ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:00:50.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.530+0000 7f6e68c74640 1 --2- 172.21.15.172:0/3629406960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e641098a0 0x7f6e640800b0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009830 tx=0x7f6e58035730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:50.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.534+0000 7f6e617fa640 1 -- 172.21.15.172:0/3629406960 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e580346a0 con 0x7f6e641098a0 2024-09-10T09:00:50.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.534+0000 7f6e6a6d7640 1 -- 172.21.15.172:0/3629406960 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e641ccfb0 con 0x7f6e641098a0 2024-09-10T09:00:50.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.534+0000 7f6e6a6d7640 1 -- 172.21.15.172:0/3629406960 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e641cd350 con 0x7f6e641098a0 2024-09-10T09:00:50.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.534+0000 7f6e617fa640 1 -- 172.21.15.172:0/3629406960 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e58004020 con 0x7f6e641098a0 2024-09-10T09:00:50.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.534+0000 7f6e617fa640 1 -- 172.21.15.172:0/3629406960 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e580144c0 con 0x7f6e641098a0 2024-09-10T09:00:50.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.541+0000 7f6e617fa640 1 -- 172.21.15.172:0/3629406960 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6e58045020 con 0x7f6e641098a0 2024-09-10T09:00:50.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.542+0000 7f6e617fa640 1 --2- 172.21.15.172:0/3629406960 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e4c0a2d50 0x7f6e4c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.543+0000 7f6e617fa640 1 -- 172.21.15.172:0/3629406960 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6e580ef790 con 0x7f6e641098a0 2024-09-10T09:00:50.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.544+0000 7f6e63fff640 1 --2- 172.21.15.172:0/3629406960 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e4c0a2d50 0x7f6e4c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:50.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.547+0000 7f6e63fff640 1 --2- 172.21.15.172:0/3629406960 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e4c0a2d50 0x7f6e4c0a5210 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009ab0 tx=0x7f6e540094b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:50.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.548+0000 7f6e6cab0740 1 Processor -- start 2024-09-10T09:00:50.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.549+0000 7f6e6cab0740 1 -- start start 2024-09-10T09:00:50.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.549+0000 7f6e6cab0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d8263b40 0x55a8d8262090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.549+0000 7f6e6cab0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 0x55a8d8262a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.550+0000 7f6e63fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d8263b40 0x55a8d8262090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:50.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.550+0000 7f6e63fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d8263b40 0x55a8d8262090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:53494/0 (socket says 172.21.15.172:53494) 2024-09-10T09:00:50.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.550+0000 7f6e6cab0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 0x55a8d826f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.550+0000 7f6e63fff640 1 -- 172.21.15.172:0/96022594 learned_addr learned my addr 172.21.15.172:0/96022594 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:50.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.550+0000 7f6e637fe640 1 --2- 172.21.15.172:0/96022594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 0x55a8d8262a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:50.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.550+0000 7f6e6cab0740 1 -- 172.21.15.172:0/96022594 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a8d81c82f0 con 0x55a8d826d370 2024-09-10T09:00:50.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.550+0000 7f6e6cab0740 1 -- 172.21.15.172:0/96022594 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a8d816fcf0 con 0x55a8d8263b40 2024-09-10T09:00:50.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.550+0000 7f6e6cab0740 1 -- 172.21.15.172:0/96022594 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a8d81c42b0 con 0x55a8d82625d0 2024-09-10T09:00:50.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.551+0000 7f6e63fff640 1 -- 172.21.15.172:0/96022594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 msgr2=0x55a8d8262a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:50.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.551+0000 7f6e63fff640 1 --2- 172.21.15.172:0/96022594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 0x55a8d8262a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.551+0000 7f6e63fff640 1 -- 172.21.15.172:0/96022594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 msgr2=0x55a8d826f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:50.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.551+0000 7f6e63fff640 1 --2- 172.21.15.172:0/96022594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 0x55a8d826f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.551+0000 7f6e63fff640 1 -- 172.21.15.172:0/96022594 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8d81543a0 con 0x55a8d8263b40 2024-09-10T09:00:50.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.551+0000 7f6e63fff640 1 --2- 172.21.15.172:0/96022594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d8263b40 0x55a8d8262090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400ee00 tx=0x7f6e54011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6055ebd95087e2c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:50.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.554+0000 7f6e4a7fc640 1 -- 172.21.15.172:0/96022594 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5401e440 con 0x55a8d8263b40 2024-09-10T09:00:50.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.554+0000 7f6e4a7fc640 1 -- 172.21.15.172:0/96022594 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e54023070 con 0x55a8d8263b40 2024-09-10T09:00:50.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.555+0000 7f6e6cab0740 1 -- 172.21.15.172:0/96022594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d8263b40 msgr2=0x55a8d8262090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:50.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.555+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/96022594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d8263b40 0x55a8d8262090 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400ee00 tx=0x7f6e54011000 comp rx=0 tx=0).stop 2024-09-10T09:00:50.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.556+0000 7f6e6cab0740 1 -- 172.21.15.172:0/96022594 shutdown_connections 2024-09-10T09:00:50.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.556+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/96022594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 0x55a8d826f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.556+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/96022594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 0x55a8d8262a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.556+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/96022594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d8263b40 0x55a8d8262090 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.556+0000 7f6e6cab0740 1 -- 172.21.15.172:0/96022594 >> 172.21.15.172:0/96022594 conn(0x55a8d822b310 msgr2=0x55a8d825ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:50.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.559+0000 7f6e6cab0740 1 -- 172.21.15.172:0/96022594 shutdown_connections 2024-09-10T09:00:50.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.559+0000 7f6e6cab0740 1 -- 172.21.15.172:0/96022594 wait complete. 2024-09-10T09:00:50.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.560+0000 7f6e6cab0740 1 -- start start 2024-09-10T09:00:50.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.560+0000 7f6e6cab0740 1 Processor -- start 2024-09-10T09:00:50.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.561+0000 7f6e6cab0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d82625d0 0x55a8d826caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.561+0000 7f6e6cab0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 0x55a8d8269da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.561+0000 7f6e6cab0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d826a2e0 0x55a8d826a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.561+0000 7f6e6cab0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a8d8270060 con 0x55a8d826d370 2024-09-10T09:00:50.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.561+0000 7f6e6cab0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a8d8183060 con 0x55a8d82625d0 2024-09-10T09:00:50.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.561+0000 7f6e6cab0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a8d82701e0 con 0x55a8d826a2e0 2024-09-10T09:00:50.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.561+0000 7f6e6cab0740 10 client.0 _ll_register_callbacks cb 0x55a8d7ed68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:50.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.561+0000 7f6e63fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d82625d0 0x55a8d826caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:50.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.561+0000 7f6e63fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d82625d0 0x55a8d826caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:53500/0 (socket says 172.21.15.172:53500) 2024-09-10T09:00:50.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.561+0000 7f6e63fff640 1 -- 172.21.15.172:0/1119525412 learned_addr learned my addr 172.21.15.172:0/1119525412 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:50.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.561+0000 7f6e68c74640 1 --2- 172.21.15.172:0/1119525412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d826a2e0 0x55a8d826a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:50.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.563+0000 7f6e637fe640 1 --2- 172.21.15.172:0/1119525412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 0x55a8d8269da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:50.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.563+0000 7f6e68c74640 1 -- 172.21.15.172:0/1119525412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d82625d0 msgr2=0x55a8d826caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:50.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.563+0000 7f6e68c74640 1 --2- 172.21.15.172:0/1119525412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d82625d0 0x55a8d826caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.563+0000 7f6e68c74640 1 -- 172.21.15.172:0/1119525412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 msgr2=0x55a8d8269da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:50.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.563+0000 7f6e68c74640 1 --2- 172.21.15.172:0/1119525412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 0x55a8d8269da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.563+0000 7f6e68c74640 1 -- 172.21.15.172:0/1119525412 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8d7f393c0 con 0x55a8d826a2e0 2024-09-10T09:00:50.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.566+0000 7f6e68c74640 1 --2- 172.21.15.172:0/1119525412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d826a2e0 0x55a8d826a760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55a8d8263070 tx=0x7f6e580bb950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd65fb88ef34a011 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:50.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.569+0000 7f6e437fe640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:00:50.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.569+0000 7f6e437fe640 1 -- 172.21.15.172:0/1119525412 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e580bbd50 con 0x55a8d826a2e0 2024-09-10T09:00:50.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.569+0000 7f6e437fe640 1 -- 172.21.15.172:0/1119525412 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6e580b83c0 con 0x55a8d826a2e0 2024-09-10T09:00:50.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.571+0000 7f6e6cab0740 1 -- 172.21.15.172:0/1119525412 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a8d81d5860 con 0x55a8d826a2e0 2024-09-10T09:00:50.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.572+0000 7f6e437fe640 1 -- 172.21.15.172:0/1119525412 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6e580c0100 con 0x55a8d826a2e0 2024-09-10T09:00:50.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.579+0000 7f6e6cab0740 10 client.28171 fetch_fsmap learned FSMap version 42 2024-09-10T09:00:50.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.579+0000 7f6e6cab0740 1 -- 172.21.15.172:0/1119525412 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a8d8270060 con 0x55a8d826a2e0 2024-09-10T09:00:50.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.580+0000 7f6e437fe640 1 -- 172.21.15.172:0/1119525412 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a8d81d5860 con 0x55a8d826a2e0 2024-09-10T09:00:50.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.580+0000 7f6e437fe640 1 -- 172.21.15.172:0/1119525412 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7f6e58014660 con 0x55a8d826a2e0 2024-09-10T09:00:50.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.580+0000 7f6e6cab0740 10 client.28171 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:00:50.795 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.580+0000 7f6e6cab0740 10 client.28171 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T09:00:50.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.580+0000 7f6e6cab0740 20 client.28171 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:50.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.580+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/1119525412 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55a8d827d480 0x55a8d827f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.580+0000 7f6e6cab0740 4 client.28171 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:00:50.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.580+0000 7f6e6cab0740 1 -- 172.21.15.172:0/1119525412 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8d81cfd70 con 0x55a8d827d480 2024-09-10T09:00:50.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.581+0000 7f6e63fff640 1 --2- 172.21.15.172:0/1119525412 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55a8d827d480 0x55a8d827f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:50.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.581+0000 7f6e437fe640 10 client.28171 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T09:00:50.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.582+0000 7f6e63fff640 1 --2- 172.21.15.172:0/1119525412 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55a8d827d480 0x55a8d827f8a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8cb0b19a95f7c204 server_cookie=1511611c7bb7ff00 in_seq=0 out_seq=0 2024-09-10T09:00:50.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.584+0000 7f6e437fe640 1 -- 172.21.15.172:0/1119525412 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8d81cfd70 con 0x55a8d827d480 2024-09-10T09:00:50.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.584+0000 7f6e437fe640 10 client.28171 handle_command_reply: tid=0 2024-09-10T09:00:50.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.584+0000 7f6e6cab0740 1 client.28171 shutdown 2024-09-10T09:00:50.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.586+0000 7f6e6cab0740 1 -- 172.21.15.172:0/1119525412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d826a2e0 msgr2=0x55a8d826a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:50.796 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.586+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/1119525412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d826a2e0 0x55a8d826a760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55a8d8263070 tx=0x7f6e580bb950 comp rx=0 tx=0).stop 2024-09-10T09:00:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.586+0000 7f6e6cab0740 1 -- 172.21.15.172:0/1119525412 shutdown_connections 2024-09-10T09:00:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.586+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/1119525412 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55a8d827d480 0x55a8d827f8a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.586+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/1119525412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d826a2e0 0x55a8d826a760 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.586+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/1119525412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 0x55a8d8269da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.586+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/1119525412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d82625d0 0x55a8d826caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.586+0000 7f6e6cab0740 1 -- 172.21.15.172:0/1119525412 >> 172.21.15.172:0/1119525412 conn(0x55a8d822b310 msgr2=0x55a8d8264990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.586+0000 7f6e437fe640 0 client.28171 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T09:00:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.586+0000 7f6e6cab0740 1 -- 172.21.15.172:0/1119525412 shutdown_connections 2024-09-10T09:00:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.586+0000 7f6e6cab0740 1 -- 172.21.15.172:0/1119525412 wait complete. 2024-09-10T09:00:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.586+0000 7f6e6cab0740 20 client.28171 trim_cache size 0 max 16384 2024-09-10T09:00:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.614+0000 7f6e6cab0740 1 Processor -- start 2024-09-10T09:00:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.616+0000 7f6e6cab0740 1 -- start start 2024-09-10T09:00:50.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.616+0000 7f6e6cab0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 0x55a8d825e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.616+0000 7f6e6cab0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d826a2e0 0x55a8d8285130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.616+0000 7f6e63fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 0x55a8d825e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.616+0000 7f6e63fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 0x55a8d825e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:57784/0 (socket says 172.21.15.172:57784) 2024-09-10T09:00:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.617+0000 7f6e637fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d826a2e0 0x55a8d8285130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.617+0000 7f6e637fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d826a2e0 0x55a8d8285130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:53510/0 (socket says 172.21.15.172:53510) 2024-09-10T09:00:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.617+0000 7f6e63fff640 1 -- 172.21.15.172:0/3998100676 learned_addr learned my addr 172.21.15.172:0/3998100676 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.617+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/3998100676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 0x55a8d8285670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.617+0000 7f6e6cab0740 1 -- 172.21.15.172:0/3998100676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5000fb10 con 0x55a8d826d370 2024-09-10T09:00:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.617+0000 7f6e6cab0740 1 -- 172.21.15.172:0/3998100676 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e500116b0 con 0x55a8d826a2e0 2024-09-10T09:00:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.617+0000 7f6e6cab0740 1 -- 172.21.15.172:0/3998100676 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6e500101e0 con 0x55a8d82625d0 2024-09-10T09:00:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.617+0000 7f6e68c74640 1 --2- 172.21.15.172:0/3998100676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 0x55a8d8285670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.621+0000 7f6e68c74640 1 -- 172.21.15.172:0/3998100676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 msgr2=0x55a8d825e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:50.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.621+0000 7f6e68c74640 1 --2- 172.21.15.172:0/3998100676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 0x55a8d825e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.621+0000 7f6e68c74640 1 -- 172.21.15.172:0/3998100676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d826a2e0 msgr2=0x55a8d8285130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.621+0000 7f6e68c74640 1 --2- 172.21.15.172:0/3998100676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d826a2e0 0x55a8d8285130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.621+0000 7f6e68c74640 1 -- 172.21.15.172:0/3998100676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8d825f1a0 con 0x55a8d826d370 2024-09-10T09:00:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.626+0000 7f6e68c74640 1 --2- 172.21.15.172:0/3998100676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 0x55a8d8285670 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6e58014a20 tx=0x7f6e580ba260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa90bae15d8bd7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.629+0000 7f6e49ffb640 1 -- 172.21.15.172:0/3998100676 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e580bbd50 con 0x55a8d826d370 2024-09-10T09:00:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.630+0000 7f6e49ffb640 1 -- 172.21.15.172:0/3998100676 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e580c49c0 con 0x55a8d826d370 2024-09-10T09:00:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.630+0000 7f6e49ffb640 1 -- 172.21.15.172:0/3998100676 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e580147d0 con 0x55a8d826d370 2024-09-10T09:00:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.631+0000 7f6e6cab0740 1 -- 172.21.15.172:0/3998100676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 msgr2=0x55a8d8285670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.631+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/3998100676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 0x55a8d8285670 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6e58014a20 tx=0x7f6e580ba260 comp rx=0 tx=0).stop 2024-09-10T09:00:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.631+0000 7f6e6cab0740 1 -- 172.21.15.172:0/3998100676 shutdown_connections 2024-09-10T09:00:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.631+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/3998100676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 0x55a8d8285670 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.631+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/3998100676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d826a2e0 0x55a8d8285130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.799 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.631+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/3998100676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 0x55a8d825e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.631+0000 7f6e6cab0740 1 -- 172.21.15.172:0/3998100676 >> 172.21.15.172:0/3998100676 conn(0x55a8d822b310 msgr2=0x55a8d8265310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.632+0000 7f6e6cab0740 1 -- 172.21.15.172:0/3998100676 shutdown_connections 2024-09-10T09:00:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.632+0000 7f6e6cab0740 1 -- 172.21.15.172:0/3998100676 wait complete. 2024-09-10T09:00:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.632+0000 7f6e6cab0740 1 -- start start 2024-09-10T09:00:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.633+0000 7f6e6cab0740 1 Processor -- start 2024-09-10T09:00:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.635+0000 7f6e6cab0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 0x55a8d8270c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.636+0000 7f6e6cab0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d826a2e0 0x55a8d82711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.636+0000 7f6e63fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 0x55a8d8270c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.636+0000 7f6e63fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 0x55a8d8270c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:57792/0 (socket says 172.21.15.172:57792) 2024-09-10T09:00:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.636+0000 7f6e63fff640 1 -- 172.21.15.172:0/2556244669 learned_addr learned my addr 172.21.15.172:0/2556244669 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.636+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/2556244669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 0x55a8d82e0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.636+0000 7f6e637fe640 1 --2- 172.21.15.172:0/2556244669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d826a2e0 0x55a8d82711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:50.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.636+0000 7f6e6cab0740 1 -- 172.21.15.172:0/2556244669 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6e50010fe0 con 0x55a8d826d370 2024-09-10T09:00:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.636+0000 7f6e6cab0740 1 -- 172.21.15.172:0/2556244669 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e50010910 con 0x55a8d826a2e0 2024-09-10T09:00:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.636+0000 7f6e6cab0740 1 -- 172.21.15.172:0/2556244669 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5000daf0 con 0x55a8d82625d0 2024-09-10T09:00:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.637+0000 7f6e63fff640 1 -- 172.21.15.172:0/2556244669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d826a2e0 msgr2=0x55a8d82711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.637+0000 7f6e63fff640 1 --2- 172.21.15.172:0/2556244669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d826a2e0 0x55a8d82711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.637+0000 7f6e63fff640 1 -- 172.21.15.172:0/2556244669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 msgr2=0x55a8d82e0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.637+0000 7f6e63fff640 1 --2- 172.21.15.172:0/2556244669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 0x55a8d82e0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.637+0000 7f6e63fff640 1 -- 172.21.15.172:0/2556244669 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8d82e0720 con 0x55a8d82625d0 2024-09-10T09:00:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.637+0000 7f6e6cab0740 10 client.0 _ll_register_callbacks cb 0x7f6e58034f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.637+0000 7f6e63fff640 1 --2- 172.21.15.172:0/2556244669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 0x55a8d8270c90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6e54011cd0 tx=0x7f6e54005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f66810fe7aedd43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.637+0000 7f6e6cab0740 1 -- 172.21.15.172:0/2556244669 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e580c49c0 con 0x55a8d82625d0 2024-09-10T09:00:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.638+0000 7f6e437fe640 10 client.28177 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:00:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.639+0000 7f6e437fe640 1 -- 172.21.15.172:0/2556244669 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e54036070 con 0x55a8d82625d0 2024-09-10T09:00:50.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.639+0000 7f6e437fe640 1 -- 172.21.15.172:0/2556244669 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6e540105d0 con 0x55a8d82625d0 2024-09-10T09:00:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.639+0000 7f6e437fe640 1 -- 172.21.15.172:0/2556244669 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6e54004020 con 0x55a8d82625d0 2024-09-10T09:00:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.639+0000 7f6e6cab0740 10 client.28177 fetch_fsmap learned FSMap version 42 2024-09-10T09:00:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.639+0000 7f6e6cab0740 1 -- 172.21.15.172:0/2556244669 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({fsmap=42}) -- 0x55a8d82e78a0 con 0x55a8d82625d0 2024-09-10T09:00:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.643+0000 7f6e437fe640 1 -- 172.21.15.172:0/2556244669 <== mon.1 v2:172.21.15.191:3300/0 4 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7f6e54032070 con 0x55a8d82625d0 2024-09-10T09:00:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.643+0000 7f6e6cab0740 10 client.28177 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:00:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.643+0000 7f6e6cab0740 10 client.28177 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T09:00:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.643+0000 7f6e6cab0740 20 client.28177 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.643+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/2556244669 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55a8d82e8030 0x55a8d82ea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.643+0000 7f6e6cab0740 4 client.28177 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:00:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.643+0000 7f6e6cab0740 1 -- 172.21.15.172:0/2556244669 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6e580c49c0 con 0x55a8d82e8030 2024-09-10T09:00:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.644+0000 7f6e637fe640 1 --2- 172.21.15.172:0/2556244669 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55a8d82e8030 0x55a8d82ea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.645+0000 7f6e637fe640 1 --2- 172.21.15.172:0/2556244669 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55a8d82e8030 0x55a8d82ea4f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6fae1f32624d96e6 server_cookie=24a111b5829fa7d6 in_seq=0 out_seq=0 2024-09-10T09:00:50.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.646+0000 7f6e437fe640 10 client.28177 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T09:00:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.646+0000 7f6e437fe640 1 -- 172.21.15.172:0/2556244669 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6e580c49c0 con 0x55a8d82e8030 2024-09-10T09:00:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.646+0000 7f6e437fe640 10 client.28177 handle_command_reply: tid=0 2024-09-10T09:00:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.646+0000 7f6e6cab0740 1 client.28177 shutdown 2024-09-10T09:00:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.647+0000 7f6e6cab0740 1 -- 172.21.15.172:0/2556244669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 msgr2=0x55a8d8270c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.647+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/2556244669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 0x55a8d8270c90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6e54011cd0 tx=0x7f6e54005db0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.647+0000 7f6e6cab0740 1 -- 172.21.15.172:0/2556244669 shutdown_connections 2024-09-10T09:00:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.647+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/2556244669 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55a8d82e8030 0x55a8d82ea4f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.647+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/2556244669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8d826d370 0x55a8d82e0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.647+0000 7f6e437fe640 0 client.28177 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T09:00:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.647+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/2556244669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a8d826a2e0 0x55a8d82711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.647+0000 7f6e6cab0740 1 --2- 172.21.15.172:0/2556244669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8d82625d0 0x55a8d8270c90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.647+0000 7f6e6cab0740 1 -- 172.21.15.172:0/2556244669 >> 172.21.15.172:0/2556244669 conn(0x55a8d822b310 msgr2=0x55a8d826be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:50.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.648+0000 7f6e6cab0740 1 -- 172.21.15.172:0/2556244669 shutdown_connections 2024-09-10T09:00:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.648+0000 7f6e6cab0740 1 -- 172.21.15.172:0/2556244669 wait complete. 2024-09-10T09:00:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.648+0000 7f6e6cab0740 20 client.28177 trim_cache size 0 max 16384 2024-09-10T09:00:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.651+0000 7f6e6a6d7640 1 -- 172.21.15.172:0/3629406960 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e4c0a2d50 msgr2=0x7f6e4c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.651+0000 7f6e6a6d7640 1 --2- 172.21.15.172:0/3629406960 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e4c0a2d50 0x7f6e4c0a5210 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009ab0 tx=0x7f6e540094b0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.651+0000 7f6e6a6d7640 1 -- 172.21.15.172:0/3629406960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e641098a0 msgr2=0x7f6e640800b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.651+0000 7f6e6a6d7640 1 --2- 172.21.15.172:0/3629406960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e641098a0 0x7f6e640800b0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009830 tx=0x7f6e58035730 comp rx=0 tx=0).stop 2024-09-10T09:00:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.651+0000 7f6e6a6d7640 1 -- 172.21.15.172:0/3629406960 shutdown_connections 2024-09-10T09:00:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.651+0000 7f6e6a6d7640 1 --2- 172.21.15.172:0/3629406960 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f6e4c0a2d50 0x7f6e4c0a5210 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.651+0000 7f6e6a6d7640 1 --2- 172.21.15.172:0/3629406960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e641098a0 0x7f6e640800b0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.651+0000 7f6e6a6d7640 1 --2- 172.21.15.172:0/3629406960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e64074c90 0x7f6e6407fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.651+0000 7f6e6a6d7640 1 --2- 172.21.15.172:0/3629406960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e640742d0 0x7f6e64084ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.651+0000 7f6e6a6d7640 1 -- 172.21.15.172:0/3629406960 >> 172.21.15.172:0/3629406960 conn(0x7f6e6406e140 msgr2=0x7f6e640764a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.652+0000 7f6e6a6d7640 1 -- 172.21.15.172:0/3629406960 shutdown_connections 2024-09-10T09:00:50.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:00:49.652+0000 7f6e6a6d7640 1 -- 172.21.15.172:0/3629406960 wait complete. 2024-09-10T09:00:51.497 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:51 smithi172 ceph-mon[30854]: pgmap v789: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 MiB/s wr, 499 op/s 2024-09-10T09:00:51.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:51 smithi191 ceph-mon[35314]: pgmap v789: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 MiB/s wr, 499 op/s 2024-09-10T09:00:51.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:51 smithi174 ceph-mon[34068]: pgmap v789: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 MiB/s wr, 499 op/s 2024-09-10T09:00:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.850+0000 7f94ac5c0640 1 -- 172.21.15.172:0/404559160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94a4103860 msgr2=0x7f94a410a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.850+0000 7f94ac5c0640 1 --2- 172.21.15.172:0/404559160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94a4103860 0x7f94a410a0f0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f949c009ca0 tx=0x7f949c02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.850+0000 7f94ac5c0640 1 -- 172.21.15.172:0/404559160 shutdown_connections 2024-09-10T09:00:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.850+0000 7f94ac5c0640 1 --2- 172.21.15.172:0/404559160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94a4103860 0x7f94a410a0f0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.850+0000 7f94ac5c0640 1 --2- 172.21.15.172:0/404559160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94a4102ea0 0x7f94a4103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.850+0000 7f94ac5c0640 1 --2- 172.21.15.172:0/404559160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94a40ffe30 0x7f94a4100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.850+0000 7f94ac5c0640 1 -- 172.21.15.172:0/404559160 >> 172.21.15.172:0/404559160 conn(0x7f94a40fb930 msgr2=0x7f94a40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.850+0000 7f94ac5c0640 1 -- 172.21.15.172:0/404559160 shutdown_connections 2024-09-10T09:00:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.850+0000 7f94ac5c0640 1 -- 172.21.15.172:0/404559160 wait complete. 2024-09-10T09:00:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.851+0000 7f94ac5c0640 1 Processor -- start 2024-09-10T09:00:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.851+0000 7f94ac5c0640 1 -- start start 2024-09-10T09:00:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.851+0000 7f94ac5c0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94a40ffe30 0x7f94a4102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.851+0000 7f94ac5c0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94a4102ea0 0x7f94a4100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.851+0000 7f94ac5c0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94a4103860 0x7f94a4101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.851+0000 7f94ac5c0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f94a410d680 con 0x7f94a4102ea0 2024-09-10T09:00:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.851+0000 7f94ac5c0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94a410d500 con 0x7f94a4103860 2024-09-10T09:00:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.851+0000 7f94ac5c0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f94a410d800 con 0x7f94a40ffe30 2024-09-10T09:00:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.851+0000 7f94a9b5b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94a4102ea0 0x7f94a4100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.851+0000 7f94a9b5b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94a4102ea0 0x7f94a4100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34488/0 (socket says 172.21.15.172:34488) 2024-09-10T09:00:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.851+0000 7f94a9b5b640 1 -- 172.21.15.172:0/1943730554 learned_addr learned my addr 172.21.15.172:0/1943730554 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.851+0000 7f94aa35c640 1 --2- 172.21.15.172:0/1943730554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94a40ffe30 0x7f94a4102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.852+0000 7f94aab5d640 1 --2- 172.21.15.172:0/1943730554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94a4103860 0x7f94a4101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.852+0000 7f94a9b5b640 1 -- 172.21.15.172:0/1943730554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94a40ffe30 msgr2=0x7f94a4102810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.852+0000 7f94a9b5b640 1 --2- 172.21.15.172:0/1943730554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94a40ffe30 0x7f94a4102810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.852+0000 7f94a9b5b640 1 -- 172.21.15.172:0/1943730554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94a4103860 msgr2=0x7f94a4101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.852+0000 7f94a9b5b640 1 --2- 172.21.15.172:0/1943730554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94a4103860 0x7f94a4101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.852+0000 7f94a9b5b640 1 -- 172.21.15.172:0/1943730554 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a4101c20 con 0x7f94a4102ea0 2024-09-10T09:00:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.852+0000 7f94aab5d640 1 --2- 172.21.15.172:0/1943730554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94a4103860 0x7f94a4101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:00:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.852+0000 7f94a9b5b640 1 --2- 172.21.15.172:0/1943730554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94a4102ea0 0x7f94a4100e50 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f949400c920 tx=0x7f949400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.852+0000 7f949b7fe640 1 -- 172.21.15.172:0/1943730554 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494007c70 con 0x7f94a4102ea0 2024-09-10T09:00:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.852+0000 7f949b7fe640 1 -- 172.21.15.172:0/1943730554 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9494007e10 con 0x7f94a4102ea0 2024-09-10T09:00:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.852+0000 7f949b7fe640 1 -- 172.21.15.172:0/1943730554 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494018820 con 0x7f94a4102ea0 2024-09-10T09:00:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.852+0000 7f94ac5c0640 1 -- 172.21.15.172:0/1943730554 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a41a1d10 con 0x7f94a4102ea0 2024-09-10T09:00:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.852+0000 7f94ac5c0640 1 -- 172.21.15.172:0/1943730554 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a41a21f0 con 0x7f94a4102ea0 2024-09-10T09:00:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.854+0000 7f949b7fe640 1 -- 172.21.15.172:0/1943730554 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f94940189c0 con 0x7f94a4102ea0 2024-09-10T09:00:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.856+0000 7f94af42e740 1 Processor -- start 2024-09-10T09:00:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.857+0000 7f94af42e740 1 -- start start 2024-09-10T09:00:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.857+0000 7f94af42e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cb9b0 0x562fe53cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.857+0000 7f94af42e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53cc2f0 0x562fe53d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.857+0000 7f94af42e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53d1380 0x562fe53d3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.857+0000 7f94af42e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562fe531e2f0 con 0x562fe53d1380 2024-09-10T09:00:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.857+0000 7f94af42e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562fe52c6270 con 0x562fe53cc2f0 2024-09-10T09:00:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.857+0000 7f94af42e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562fe531a2b0 con 0x562fe53cb9b0 2024-09-10T09:00:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.857+0000 7f94aab5d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cb9b0 0x562fe53cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.857+0000 7f94aa35c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53cc2f0 0x562fe53d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.858+0000 7f94aab5d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cb9b0 0x562fe53cbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:57834/0 (socket says 172.21.15.172:57834) 2024-09-10T09:00:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.858+0000 7f94aab5d640 1 -- 172.21.15.172:0/4109918687 learned_addr learned my addr 172.21.15.172:0/4109918687 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.860+0000 7f949b7fe640 1 --2- 172.21.15.172:0/1943730554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94900a2cb0 0x7f94900a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.860+0000 7f94aab5d640 1 -- 172.21.15.172:0/4109918687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53cc2f0 msgr2=0x562fe53d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.860+0000 7f949b7fe640 1 -- 172.21.15.172:0/1943730554 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f94940cf4b0 con 0x7f94a4102ea0 2024-09-10T09:00:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.860+0000 7f94aab5d640 1 --2- 172.21.15.172:0/4109918687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53cc2f0 0x562fe53d0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.860+0000 7f94aab5d640 1 -- 172.21.15.172:0/4109918687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53d1380 msgr2=0x562fe53d3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.860+0000 7f94aab5d640 1 --2- 172.21.15.172:0/4109918687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53d1380 0x562fe53d3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.860+0000 7f94aab5d640 1 -- 172.21.15.172:0/4109918687 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe52c5340 con 0x562fe53cb9b0 2024-09-10T09:00:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.861+0000 7f94aab5d640 1 --2- 172.21.15.172:0/4109918687 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cb9b0 0x562fe53cbdb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f949c005fb0 tx=0x7f949c009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=660cf03e1eae9b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.861+0000 7f94aab5d640 1 --2- 172.21.15.172:0/1943730554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94900a2cb0 0x7f94900a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.862+0000 7f9498bfe640 1 -- 172.21.15.172:0/4109918687 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c04d070 con 0x562fe53cb9b0 2024-09-10T09:00:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.862+0000 7f9498bfe640 1 -- 172.21.15.172:0/4109918687 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f949c036040 con 0x562fe53cb9b0 2024-09-10T09:00:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.862+0000 7f9498bfe640 1 -- 172.21.15.172:0/4109918687 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c0305c0 con 0x562fe53cb9b0 2024-09-10T09:00:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.863+0000 7f94aab5d640 1 --2- 172.21.15.172:0/1943730554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94900a2cb0 0x7f94900a5170 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f949c0049d0 tx=0x7f949c0308c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.866+0000 7f94af42e740 1 -- 172.21.15.172:0/4109918687 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cb9b0 msgr2=0x562fe53cbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.866+0000 7f94af42e740 1 --2- 172.21.15.172:0/4109918687 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cb9b0 0x562fe53cbdb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f949c005fb0 tx=0x7f949c009a20 comp rx=0 tx=0).stop 2024-09-10T09:00:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.867+0000 7f94af42e740 1 -- 172.21.15.172:0/4109918687 shutdown_connections 2024-09-10T09:00:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.867+0000 7f94af42e740 1 --2- 172.21.15.172:0/4109918687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53d1380 0x562fe53d3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.867+0000 7f94af42e740 1 --2- 172.21.15.172:0/4109918687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53cc2f0 0x562fe53d0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.867+0000 7f94af42e740 1 --2- 172.21.15.172:0/4109918687 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cb9b0 0x562fe53cbdb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.867+0000 7f94af42e740 1 -- 172.21.15.172:0/4109918687 >> 172.21.15.172:0/4109918687 conn(0x562fe53b6010 msgr2=0x562fe53b6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:52.872 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.871+0000 7f94af42e740 1 -- 172.21.15.172:0/4109918687 shutdown_connections 2024-09-10T09:00:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.872+0000 7f94af42e740 1 -- 172.21.15.172:0/4109918687 wait complete. 2024-09-10T09:00:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.872+0000 7f94af42e740 1 -- start start 2024-09-10T09:00:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.872+0000 7f94af42e740 1 Processor -- start 2024-09-10T09:00:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.874+0000 7f94af42e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cb9b0 0x562fe53d5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.874+0000 7f94af42e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53cc2f0 0x562fe53d6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.874+0000 7f94af42e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53d1380 0x562fe53dd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.874+0000 7f94af42e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562fe53d4350 con 0x562fe53cc2f0 2024-09-10T09:00:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.874+0000 7f94af42e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562fe52a7290 con 0x562fe53d1380 2024-09-10T09:00:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.874+0000 7f94af42e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562fe53d44d0 con 0x562fe53cb9b0 2024-09-10T09:00:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.874+0000 7f94af42e740 10 client.0 _ll_register_callbacks cb 0x562fe52d8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.875+0000 7f94a9b5b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53cc2f0 0x562fe53d6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.875+0000 7f94aa35c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cb9b0 0x562fe53d5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.875+0000 7f94a9b5b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53cc2f0 0x562fe53d6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34500/0 (socket says 172.21.15.172:34500) 2024-09-10T09:00:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.875+0000 7f94a9b5b640 1 -- 172.21.15.172:0/852090739 learned_addr learned my addr 172.21.15.172:0/852090739 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.875+0000 7f94aab5d640 1 --2- 172.21.15.172:0/852090739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53d1380 0x562fe53dd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.875+0000 7f94a9b5b640 1 -- 172.21.15.172:0/852090739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cb9b0 msgr2=0x562fe53d5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.875+0000 7f94a9b5b640 1 --2- 172.21.15.172:0/852090739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cb9b0 0x562fe53d5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.875+0000 7f94a9b5b640 1 -- 172.21.15.172:0/852090739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53d1380 msgr2=0x562fe53dd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.875+0000 7f94a9b5b640 1 --2- 172.21.15.172:0/852090739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53d1380 0x562fe53dd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.875+0000 7f94a9b5b640 1 -- 172.21.15.172:0/852090739 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe5031210 con 0x562fe53cc2f0 2024-09-10T09:00:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.876+0000 7f94a9b5b640 1 --2- 172.21.15.172:0/852090739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53cc2f0 0x562fe53d6240 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f949409b6b0 tx=0x7f9494018d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba964d3ad9105e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.876+0000 7f948dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:00:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.876+0000 7f948dffb640 1 -- 172.21.15.172:0/852090739 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949409b9b0 con 0x562fe53cc2f0 2024-09-10T09:00:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.876+0000 7f94af42e740 1 -- 172.21.15.172:0/852090739 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f949c036040 con 0x562fe53cc2f0 2024-09-10T09:00:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.876+0000 7f948dffb640 1 -- 172.21.15.172:0/852090739 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f94940a4190 con 0x562fe53cc2f0 2024-09-10T09:00:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.877+0000 7f948dffb640 1 -- 172.21.15.172:0/852090739 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f94940b9050 con 0x562fe53cc2f0 2024-09-10T09:00:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.879+0000 7f94af42e740 10 client.19488 fetch_fsmap learned FSMap version 42 2024-09-10T09:00:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.879+0000 7f94af42e740 1 -- 172.21.15.172:0/852090739 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562fe50fd620 con 0x562fe53cc2f0 2024-09-10T09:00:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.884+0000 7f948dffb640 1 -- 172.21.15.172:0/852090739 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f949c036040 con 0x562fe53cc2f0 2024-09-10T09:00:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.884+0000 7f948dffb640 1 -- 172.21.15.172:0/852090739 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7f949409cce0 con 0x562fe53cc2f0 2024-09-10T09:00:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.887+0000 7f94af42e740 10 client.19488 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:00:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.887+0000 7f94af42e740 10 client.19488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:00:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.887+0000 7f94af42e740 20 client.19488 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.887+0000 7f94af42e740 1 --2- 172.21.15.172:0/852090739 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562fe53e18f0 0x562fe53e3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.887+0000 7f94af42e740 4 client.19488 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:00:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.887+0000 7f94af42e740 1 -- 172.21.15.172:0/852090739 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fe532ba20 con 0x562fe53e18f0 2024-09-10T09:00:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.895+0000 7f94aa35c640 1 --2- 172.21.15.172:0/852090739 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562fe53e18f0 0x562fe53e3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.896+0000 7f94aa35c640 1 --2- 172.21.15.172:0/852090739 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562fe53e18f0 0x562fe53e3ce0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f95dbab28ba52105 server_cookie=919da65bc3b58b81 in_seq=0 out_seq=0 2024-09-10T09:00:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.897+0000 7f948dffb640 10 client.19488 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.899+0000 7f948dffb640 1 -- 172.21.15.172:0/852090739 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562fe532ba20 con 0x562fe53e18f0 2024-09-10T09:00:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.899+0000 7f948dffb640 10 client.19488 handle_command_reply: tid=0 2024-09-10T09:00:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.899+0000 7f94af42e740 1 client.19488 shutdown 2024-09-10T09:00:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.913+0000 7f94af42e740 1 -- 172.21.15.172:0/852090739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53cc2f0 msgr2=0x562fe53d6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.913+0000 7f94af42e740 1 --2- 172.21.15.172:0/852090739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53cc2f0 0x562fe53d6240 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f949409b6b0 tx=0x7f9494018d70 comp rx=0 tx=0).stop 2024-09-10T09:00:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.913+0000 7f94af42e740 1 -- 172.21.15.172:0/852090739 shutdown_connections 2024-09-10T09:00:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.913+0000 7f94af42e740 1 --2- 172.21.15.172:0/852090739 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562fe53e18f0 0x562fe53e3ce0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.913+0000 7f94af42e740 1 --2- 172.21.15.172:0/852090739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53d1380 0x562fe53dd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.913+0000 7f94af42e740 1 --2- 172.21.15.172:0/852090739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53cc2f0 0x562fe53d6240 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.913+0000 7f94af42e740 1 --2- 172.21.15.172:0/852090739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cb9b0 0x562fe53d5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.913+0000 7f94af42e740 1 -- 172.21.15.172:0/852090739 >> 172.21.15.172:0/852090739 conn(0x562fe53b6010 msgr2=0x562fe53b7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.913+0000 7f948dffb640 0 client.19488 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.913+0000 7f94af42e740 1 -- 172.21.15.172:0/852090739 shutdown_connections 2024-09-10T09:00:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.913+0000 7f94af42e740 1 -- 172.21.15.172:0/852090739 wait complete. 2024-09-10T09:00:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.913+0000 7f94af42e740 20 client.19488 trim_cache size 0 max 16384 2024-09-10T09:00:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.944+0000 7f94af42e740 1 Processor -- start 2024-09-10T09:00:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.944+0000 7f94af42e740 1 -- start start 2024-09-10T09:00:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.947+0000 7f94af42e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53cb9b0 0x562fe53d5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.947+0000 7f94af42e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53cc2f0 0x562fe53dd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.947+0000 7f94aa35c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53cb9b0 0x562fe53d5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.947+0000 7f94aa35c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53cb9b0 0x562fe53d5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34506/0 (socket says 172.21.15.172:34506) 2024-09-10T09:00:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.947+0000 7f94aa35c640 1 -- 172.21.15.172:0/101831811 learned_addr learned my addr 172.21.15.172:0/101831811 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.947+0000 7f94af42e740 1 --2- 172.21.15.172:0/101831811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53d1380 0x562fe53b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.948+0000 7f94af42e740 1 -- 172.21.15.172:0/101831811 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f94a41a3f30 con 0x562fe53cb9b0 2024-09-10T09:00:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.948+0000 7f94af42e740 1 -- 172.21.15.172:0/101831811 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94a41a5ad0 con 0x562fe53cc2f0 2024-09-10T09:00:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.948+0000 7f94af42e740 1 -- 172.21.15.172:0/101831811 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f94a41a4600 con 0x562fe53d1380 2024-09-10T09:00:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.948+0000 7f94aab5d640 1 --2- 172.21.15.172:0/101831811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53d1380 0x562fe53b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:52.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.948+0000 7f94a9b5b640 1 --2- 172.21.15.172:0/101831811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53cc2f0 0x562fe53dd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.951+0000 7f94a9b5b640 1 -- 172.21.15.172:0/101831811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53d1380 msgr2=0x562fe53b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.951+0000 7f94a9b5b640 1 --2- 172.21.15.172:0/101831811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53d1380 0x562fe53b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.951+0000 7f94a9b5b640 1 -- 172.21.15.172:0/101831811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53cb9b0 msgr2=0x562fe53d5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.951+0000 7f94a9b5b640 1 --2- 172.21.15.172:0/101831811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53cb9b0 0x562fe53d5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.951+0000 7f94a9b5b640 1 -- 172.21.15.172:0/101831811 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe53ccb40 con 0x562fe53cc2f0 2024-09-10T09:00:52.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.953+0000 7f94a9b5b640 1 --2- 172.21.15.172:0/101831811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53cc2f0 0x562fe53dd240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f94940ccca0 tx=0x7f9494000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c456646ef67ad3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.955+0000 7f948ffff640 1 -- 172.21.15.172:0/101831811 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949409b9b0 con 0x562fe53cc2f0 2024-09-10T09:00:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.955+0000 7f948ffff640 1 -- 172.21.15.172:0/101831811 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f949407f070 con 0x562fe53cc2f0 2024-09-10T09:00:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.955+0000 7f948ffff640 1 -- 172.21.15.172:0/101831811 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94940a4620 con 0x562fe53cc2f0 2024-09-10T09:00:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.956+0000 7f94af42e740 1 -- 172.21.15.172:0/101831811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53cc2f0 msgr2=0x562fe53dd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:52.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.957+0000 7f94af42e740 1 --2- 172.21.15.172:0/101831811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53cc2f0 0x562fe53dd240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f94940ccca0 tx=0x7f9494000bc0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.963+0000 7f94af42e740 1 -- 172.21.15.172:0/101831811 shutdown_connections 2024-09-10T09:00:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.963+0000 7f94af42e740 1 --2- 172.21.15.172:0/101831811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53d1380 0x562fe53b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.963+0000 7f94af42e740 1 --2- 172.21.15.172:0/101831811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53cc2f0 0x562fe53dd240 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.963+0000 7f94af42e740 1 --2- 172.21.15.172:0/101831811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53cb9b0 0x562fe53d5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.963+0000 7f94af42e740 1 -- 172.21.15.172:0/101831811 >> 172.21.15.172:0/101831811 conn(0x562fe53b6010 msgr2=0x562fe53b49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.971+0000 7f94af42e740 1 -- 172.21.15.172:0/101831811 shutdown_connections 2024-09-10T09:00:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.975+0000 7f94af42e740 1 -- 172.21.15.172:0/101831811 wait complete. 2024-09-10T09:00:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.977+0000 7f94af42e740 1 -- start start 2024-09-10T09:00:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.977+0000 7f94af42e740 1 Processor -- start 2024-09-10T09:00:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.978+0000 7f94af42e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53cb9b0 0x562fe53db0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.978+0000 7f94af42e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cc2f0 0x562fe53db610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.978+0000 7f94af42e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53d1380 0x562fe53dc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.978+0000 7f94a9b5b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cc2f0 0x562fe53db610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.978+0000 7f94a9b5b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cc2f0 0x562fe53db610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:57850/0 (socket says 172.21.15.172:57850) 2024-09-10T09:00:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.978+0000 7f94a9b5b640 1 -- 172.21.15.172:0/1298698062 learned_addr learned my addr 172.21.15.172:0/1298698062 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:00:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.978+0000 7f94af42e740 1 -- 172.21.15.172:0/1298698062 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f94a41a5400 con 0x562fe53d1380 2024-09-10T09:00:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.979+0000 7f94af42e740 1 -- 172.21.15.172:0/1298698062 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94a41a4d30 con 0x562fe53cb9b0 2024-09-10T09:00:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.979+0000 7f94af42e740 1 -- 172.21.15.172:0/1298698062 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f94a4073e40 con 0x562fe53cc2f0 2024-09-10T09:00:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.979+0000 7f94af42e740 10 client.0 _ll_register_callbacks cb 0x7f949400fe00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:00:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.979+0000 7f94aab5d640 1 --2- 172.21.15.172:0/1298698062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53d1380 0x562fe53dc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.980+0000 7f94aa35c640 1 --2- 172.21.15.172:0/1298698062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53cb9b0 0x562fe53db0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.983+0000 7f94aab5d640 1 -- 172.21.15.172:0/1298698062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cc2f0 msgr2=0x562fe53db610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.983+0000 7f94aab5d640 1 --2- 172.21.15.172:0/1298698062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cc2f0 0x562fe53db610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.983+0000 7f94aab5d640 1 -- 172.21.15.172:0/1298698062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53cb9b0 msgr2=0x562fe53db0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:00:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.983+0000 7f94aab5d640 1 --2- 172.21.15.172:0/1298698062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53cb9b0 0x562fe53db0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:52.983+0000 7f94aab5d640 1 -- 172.21.15.172:0/1298698062 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe53dcc30 con 0x562fe53d1380 2024-09-10T09:00:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.006+0000 7f94aab5d640 1 --2- 172.21.15.172:0/1298698062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53d1380 0x562fe53dc620 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f949c031970 tx=0x7f949c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e82c418dc4013c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:00:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.007+0000 7f948dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:00:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.015+0000 7f948dffb640 1 -- 172.21.15.172:0/1298698062 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c04d040 con 0x562fe53d1380 2024-09-10T09:00:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.015+0000 7f948dffb640 1 -- 172.21.15.172:0/1298698062 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f949c004b40 con 0x562fe53d1380 2024-09-10T09:00:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.016+0000 7f94af42e740 1 -- 172.21.15.172:0/1298698062 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f949407f070 con 0x562fe53d1380 2024-09-10T09:00:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.021+0000 7f948dffb640 1 -- 172.21.15.172:0/1298698062 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f949c004d40 con 0x562fe53d1380 2024-09-10T09:00:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.023+0000 7f94af42e740 10 client.19494 fetch_fsmap learned FSMap version 42 2024-09-10T09:00:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.023+0000 7f94af42e740 1 -- 172.21.15.172:0/1298698062 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562fe544cd90 con 0x562fe53d1380 2024-09-10T09:00:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.028+0000 7f948dffb640 1 -- 172.21.15.172:0/1298698062 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f949407f070 con 0x562fe53d1380 2024-09-10T09:00:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.028+0000 7f948dffb640 1 -- 172.21.15.172:0/1298698062 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 42) ==== 8378+0+0 (secure 0 0 0) 0x7f949c043050 con 0x562fe53d1380 2024-09-10T09:00:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.029+0000 7f94af42e740 10 client.19494 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:00:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.029+0000 7f94af42e740 10 client.19494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:00:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.029+0000 7f94af42e740 20 client.19494 populate_metadata read hostname 'smithi172' 2024-09-10T09:00:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.029+0000 7f94af42e740 1 --2- 172.21.15.172:0/1298698062 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562fe544d2c0 0x562fe544f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:00:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.029+0000 7f94af42e740 4 client.19494 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:00:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.029+0000 7f94af42e740 1 -- 172.21.15.172:0/1298698062 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562fe5325d70 con 0x562fe544d2c0 2024-09-10T09:00:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.037+0000 7f94aa35c640 1 --2- 172.21.15.172:0/1298698062 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562fe544d2c0 0x562fe544f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:00:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.040+0000 7f94aa35c640 1 --2- 172.21.15.172:0/1298698062 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562fe544d2c0 0x562fe544f6b0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca71fd76d9aa898c server_cookie=c6b5e1bc0319d0a9 in_seq=0 out_seq=0 2024-09-10T09:00:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.042+0000 7f948dffb640 10 client.19494 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:53.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.064+0000 7f948dffb640 1 -- 172.21.15.172:0/1298698062 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562fe5325d70 con 0x562fe544d2c0 2024-09-10T09:00:53.065 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.064+0000 7f948dffb640 10 client.19494 handle_command_reply: tid=0 2024-09-10T09:00:53.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.064+0000 7f94af42e740 1 client.19494 shutdown 2024-09-10T09:00:53.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.065+0000 7f94af42e740 1 -- 172.21.15.172:0/1298698062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53d1380 msgr2=0x562fe53dc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:53.066 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.065+0000 7f94af42e740 1 --2- 172.21.15.172:0/1298698062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53d1380 0x562fe53dc620 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f949c031970 tx=0x7f949c041000 comp rx=0 tx=0).stop 2024-09-10T09:00:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.066+0000 7f94af42e740 1 -- 172.21.15.172:0/1298698062 shutdown_connections 2024-09-10T09:00:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.066+0000 7f94af42e740 1 --2- 172.21.15.172:0/1298698062 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562fe544d2c0 0x562fe544f6b0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.066+0000 7f94af42e740 1 --2- 172.21.15.172:0/1298698062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562fe53d1380 0x562fe53dc620 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.066+0000 7f94af42e740 1 --2- 172.21.15.172:0/1298698062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562fe53cc2f0 0x562fe53db610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.066+0000 7f94af42e740 1 --2- 172.21.15.172:0/1298698062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562fe53cb9b0 0x562fe53db0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:53.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.066+0000 7f94af42e740 1 -- 172.21.15.172:0/1298698062 >> 172.21.15.172:0/1298698062 conn(0x562fe53b6010 msgr2=0x562fe53b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:53.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.066+0000 7f948dffb640 0 client.19494 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:00:53.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.067+0000 7f94af42e740 1 -- 172.21.15.172:0/1298698062 shutdown_connections 2024-09-10T09:00:53.069 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.067+0000 7f94af42e740 1 -- 172.21.15.172:0/1298698062 wait complete. 2024-09-10T09:00:53.069 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.067+0000 7f94af42e740 20 client.19494 trim_cache size 0 max 16384 2024-09-10T09:00:53.069 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:00:53.069 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:00:53.069 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:00:53.069 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:00:53.071 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.069+0000 7f94ac5c0640 1 -- 172.21.15.172:0/1943730554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94900a2cb0 msgr2=0x7f94900a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:53.071 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.069+0000 7f94ac5c0640 1 --2- 172.21.15.172:0/1943730554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94900a2cb0 0x7f94900a5170 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f949c0049d0 tx=0x7f949c0308c0 comp rx=0 tx=0).stop 2024-09-10T09:00:53.071 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.069+0000 7f94ac5c0640 1 -- 172.21.15.172:0/1943730554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94a4102ea0 msgr2=0x7f94a4100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:00:53.071 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.069+0000 7f94ac5c0640 1 --2- 172.21.15.172:0/1943730554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94a4102ea0 0x7f94a4100e50 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f949400c920 tx=0x7f949400cde0 comp rx=0 tx=0).stop 2024-09-10T09:00:53.071 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.070+0000 7f94ac5c0640 1 -- 172.21.15.172:0/1943730554 shutdown_connections 2024-09-10T09:00:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.070+0000 7f94ac5c0640 1 --2- 172.21.15.172:0/1943730554 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94900a2cb0 0x7f94900a5170 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.070+0000 7f94ac5c0640 1 --2- 172.21.15.172:0/1943730554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94a4103860 0x7f94a4101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.070+0000 7f94ac5c0640 1 --2- 172.21.15.172:0/1943730554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94a4102ea0 0x7f94a4100e50 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.070+0000 7f94ac5c0640 1 --2- 172.21.15.172:0/1943730554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94a40ffe30 0x7f94a4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:00:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.070+0000 7f94ac5c0640 1 -- 172.21.15.172:0/1943730554 >> 172.21.15.172:0/1943730554 conn(0x7f94a40fb930 msgr2=0x7f94a4106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:00:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.074+0000 7f94ac5c0640 1 -- 172.21.15.172:0/1943730554 shutdown_connections 2024-09-10T09:00:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:00:53.074+0000 7f94ac5c0640 1 -- 172.21.15.172:0/1943730554 wait complete. 2024-09-10T09:00:53.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:53 smithi191 ceph-mon[35314]: pgmap v790: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.6 MiB/s wr, 673 op/s 2024-09-10T09:00:53.836 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:53 smithi172 ceph-mon[30854]: pgmap v790: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.6 MiB/s wr, 673 op/s 2024-09-10T09:00:53.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:53 smithi174 ceph-mon[34068]: pgmap v790: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.6 MiB/s wr, 673 op/s 2024-09-10T09:00:55.009 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:00:55.049 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:00:55.511 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:55 smithi172 ceph-mon[30854]: pgmap v791: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.2 MiB/s wr, 1.59k op/s 2024-09-10T09:00:55.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:55 smithi191 ceph-mon[35314]: pgmap v791: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.2 MiB/s wr, 1.59k op/s 2024-09-10T09:00:55.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:55 smithi174 ceph-mon[34068]: pgmap v791: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.2 MiB/s wr, 1.59k op/s 2024-09-10T09:00:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:57 smithi172 ceph-mon[30854]: pgmap v792: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.8 MiB/s wr, 2.24k op/s 2024-09-10T09:00:57.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:57 smithi191 ceph-mon[35314]: pgmap v792: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.8 MiB/s wr, 2.24k op/s 2024-09-10T09:00:57.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:57 smithi174 ceph-mon[34068]: pgmap v792: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.8 MiB/s wr, 2.24k op/s 2024-09-10T09:00:59.585 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:00:59 smithi172 ceph-mon[30854]: pgmap v793: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 11 MiB/s wr, 2.87k op/s 2024-09-10T09:00:59.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:00:59 smithi191 ceph-mon[35314]: pgmap v793: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 11 MiB/s wr, 2.87k op/s 2024-09-10T09:00:59.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:00:59 smithi174 ceph-mon[34068]: pgmap v793: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 11 MiB/s wr, 2.87k op/s 2024-09-10T09:01:01.587 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:01 smithi172 ceph-mon[30854]: pgmap v794: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 3.13k op/s 2024-09-10T09:01:01.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:01 smithi174 ceph-mon[34068]: pgmap v794: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 3.13k op/s 2024-09-10T09:01:01.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:01 smithi191 ceph-mon[35314]: pgmap v794: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 3.13k op/s 2024-09-10T09:01:02.606 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:02 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/782609118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:01:02.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:02 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/782609118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:01:02.835 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T09:01:02.835 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":42,"btime":"2024-09-10T09:00:18:484858+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T09:00:17.670862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T09:01:02.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.454+0000 7f4420189640 1 -- 172.21.15.172:0/801189375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441811b140 msgr2=0x7f441811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:02.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.454+0000 7f4420189640 1 --2- 172.21.15.172:0/801189375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441811b140 0x7f441811d530 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f4410009ca0 tx=0x7f441002a8f0 comp rx=0 tx=0).stop 2024-09-10T09:01:02.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.454+0000 7f4420189640 1 -- 172.21.15.172:0/801189375 shutdown_connections 2024-09-10T09:01:02.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.454+0000 7f4420189640 1 --2- 172.21.15.172:0/801189375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f441811b140 0x7f441811d530 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:02.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.454+0000 7f4420189640 1 --2- 172.21.15.172:0/801189375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4418109830 0x7f4418109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:02.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.454+0000 7f4420189640 1 --2- 172.21.15.172:0/801189375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4418109450 0x7f4418110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:02.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.454+0000 7f4420189640 1 -- 172.21.15.172:0/801189375 >> 172.21.15.172:0/801189375 conn(0x7f441806e140 msgr2=0x7f441806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:02.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.458+0000 7f4420189640 1 -- 172.21.15.172:0/801189375 shutdown_connections 2024-09-10T09:01:02.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.459+0000 7f4420189640 1 -- 172.21.15.172:0/801189375 wait complete. 2024-09-10T09:01:02.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.459+0000 7f4420189640 1 Processor -- start 2024-09-10T09:01:02.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.459+0000 7f4420189640 1 -- start start 2024-09-10T09:01:02.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.460+0000 7f4420189640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4418109450 0x7f44181a7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:02.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.460+0000 7f4420189640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4418109830 0x7f44181a76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:02.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.460+0000 7f4420189640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f441811b140 0x7f44181aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:02.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.460+0000 7f4420189640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4418120c90 con 0x7f4418109450 2024-09-10T09:01:02.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.460+0000 7f4420189640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4418120b10 con 0x7f441811b140 2024-09-10T09:01:02.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.460+0000 7f4420189640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4418120e10 con 0x7f4418109830 2024-09-10T09:01:02.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.460+0000 7f441e726640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f441811b140 0x7f44181aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:02.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.460+0000 7f441e726640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f441811b140 0x7f44181aba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:42638/0 (socket says 172.21.15.172:42638) 2024-09-10T09:01:02.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.460+0000 7f441e726640 1 -- 172.21.15.172:0/782609118 learned_addr learned my addr 172.21.15.172:0/782609118 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:02.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.460+0000 7f441d724640 1 --2- 172.21.15.172:0/782609118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4418109830 0x7f44181a76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:02.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.461+0000 7f441e726640 1 -- 172.21.15.172:0/782609118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4418109830 msgr2=0x7f44181a76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:02.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.461+0000 7f441e726640 1 --2- 172.21.15.172:0/782609118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4418109830 0x7f44181a76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:02.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.461+0000 7f441e726640 1 -- 172.21.15.172:0/782609118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4418109450 msgr2=0x7f44181a7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:02.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.461+0000 7f441e726640 1 --2- 172.21.15.172:0/782609118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4418109450 0x7f44181a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:02.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.461+0000 7f441e726640 1 -- 172.21.15.172:0/782609118 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44181ac140 con 0x7f441811b140 2024-09-10T09:01:02.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.461+0000 7f441e726640 1 --2- 172.21.15.172:0/782609118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f441811b140 0x7f44181aba60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4410005c40 tx=0x7f44100317f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:02.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.465+0000 7f4416ffd640 1 -- 172.21.15.172:0/782609118 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410003b70 con 0x7f441811b140 2024-09-10T09:01:02.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.465+0000 7f4420189640 1 -- 172.21.15.172:0/782609118 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44181ac3d0 con 0x7f441811b140 2024-09-10T09:01:02.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.465+0000 7f4420189640 1 -- 172.21.15.172:0/782609118 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f441807a450 con 0x7f441811b140 2024-09-10T09:01:02.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.466+0000 7f4416ffd640 1 -- 172.21.15.172:0/782609118 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f441002d070 con 0x7f441811b140 2024-09-10T09:01:02.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.467+0000 7f4416ffd640 1 -- 172.21.15.172:0/782609118 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441002f8a0 con 0x7f441811b140 2024-09-10T09:01:02.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.467+0000 7f4420189640 1 -- 172.21.15.172:0/782609118 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4418073b00 con 0x7f441811b140 2024-09-10T09:01:02.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.472+0000 7f4416ffd640 1 -- 172.21.15.172:0/782609118 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f441002fb00 con 0x7f441811b140 2024-09-10T09:01:02.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.473+0000 7f4416ffd640 1 --2- 172.21.15.172:0/782609118 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44000a2d90 0x7f44000a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:02.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.473+0000 7f4416ffd640 1 -- 172.21.15.172:0/782609118 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f44100f0740 con 0x7f441811b140 2024-09-10T09:01:02.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.473+0000 7f441df25640 1 --2- 172.21.15.172:0/782609118 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44000a2d90 0x7f44000a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:02.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.475+0000 7f441df25640 1 --2- 172.21.15.172:0/782609118 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44000a2d90 0x7f44000a5250 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4408005fd0 tx=0x7f4408005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:02.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.476+0000 7f4416ffd640 1 -- 172.21.15.172:0/782609118 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44100bd2e0 con 0x7f441811b140 2024-09-10T09:01:02.839 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.908+0000 7f4420189640 1 -- 172.21.15.172:0/782609118 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f441810b4b0 con 0x7f441811b140 2024-09-10T09:01:02.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.910+0000 7f4416ffd640 1 -- 172.21.15.172:0/782609118 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+12004 (secure 0 0 0) 0x7f44100bcbe0 con 0x7f441811b140 2024-09-10T09:01:02.840 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 42 2024-09-10T09:01:02.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.918+0000 7f4420189640 1 -- 172.21.15.172:0/782609118 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44000a2d90 msgr2=0x7f44000a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:02.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.918+0000 7f4420189640 1 --2- 172.21.15.172:0/782609118 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44000a2d90 0x7f44000a5250 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4408005fd0 tx=0x7f4408005850 comp rx=0 tx=0).stop 2024-09-10T09:01:02.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.918+0000 7f4420189640 1 -- 172.21.15.172:0/782609118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f441811b140 msgr2=0x7f44181aba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:02.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.918+0000 7f4420189640 1 --2- 172.21.15.172:0/782609118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f441811b140 0x7f44181aba60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4410005c40 tx=0x7f44100317f0 comp rx=0 tx=0).stop 2024-09-10T09:01:02.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.919+0000 7f4420189640 1 -- 172.21.15.172:0/782609118 shutdown_connections 2024-09-10T09:01:02.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.919+0000 7f4420189640 1 --2- 172.21.15.172:0/782609118 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44000a2d90 0x7f44000a5250 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:02.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.920+0000 7f4420189640 1 --2- 172.21.15.172:0/782609118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f441811b140 0x7f44181aba60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:02.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.920+0000 7f4420189640 1 --2- 172.21.15.172:0/782609118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4418109830 0x7f44181a76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:02.840 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.920+0000 7f4420189640 1 --2- 172.21.15.172:0/782609118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4418109450 0x7f44181a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:02.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.920+0000 7f4420189640 1 -- 172.21.15.172:0/782609118 >> 172.21.15.172:0/782609118 conn(0x7f441806e140 msgr2=0x7f441811bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:02.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.920+0000 7f4420189640 1 -- 172.21.15.172:0/782609118 shutdown_connections 2024-09-10T09:01:02.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:01.920+0000 7f4420189640 1 -- 172.21.15.172:0/782609118 wait complete. 2024-09-10T09:01:02.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:02 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/782609118' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:01:03.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:03 smithi191 ceph-mon[35314]: pgmap v795: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 3.27k op/s 2024-09-10T09:01:03.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:03 smithi172 ceph-mon[30854]: pgmap v795: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 3.27k op/s 2024-09-10T09:01:03.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:03 smithi174 ceph-mon[34068]: pgmap v795: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 3.27k op/s 2024-09-10T09:01:04.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:04 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:01:04.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:04 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:01:04.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:04 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:01:05.836 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:05 smithi172 ceph-mon[30854]: pgmap v796: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 3.98k op/s 2024-09-10T09:01:05.893 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:05 smithi174 ceph-mon[34068]: pgmap v796: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 3.98k op/s 2024-09-10T09:01:06.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:05 smithi191 ceph-mon[35314]: pgmap v796: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 3.98k op/s 2024-09-10T09:01:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.266+0000 7fdc54caf640 1 -- 172.21.15.172:0/37144481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc50072cb0 msgr2=0x7fdc500730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.266+0000 7fdc54caf640 1 --2- 172.21.15.172:0/37144481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc50072cb0 0x7fdc500730b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009930 tx=0x7fdc4002a420 comp rx=0 tx=0).stop 2024-09-10T09:01:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.267+0000 7fdc54caf640 1 -- 172.21.15.172:0/37144481 shutdown_connections 2024-09-10T09:01:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.267+0000 7fdc54caf640 1 --2- 172.21.15.172:0/37144481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc5010dbb0 0x7fdc50110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.267+0000 7fdc54caf640 1 --2- 172.21.15.172:0/37144481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc50073680 0x7fdc5010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.267+0000 7fdc54caf640 1 --2- 172.21.15.172:0/37144481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc50072cb0 0x7fdc500730b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.267+0000 7fdc54caf640 1 -- 172.21.15.172:0/37144481 >> 172.21.15.172:0/37144481 conn(0x7fdc5006eb50 msgr2=0x7fdc50070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.273+0000 7fdc54caf640 1 -- 172.21.15.172:0/37144481 shutdown_connections 2024-09-10T09:01:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.277+0000 7fdc54caf640 1 -- 172.21.15.172:0/37144481 wait complete. 2024-09-10T09:01:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.277+0000 7fdc54caf640 1 Processor -- start 2024-09-10T09:01:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.278+0000 7fdc54caf640 1 -- start start 2024-09-10T09:01:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.278+0000 7fdc54caf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc50072cb0 0x7fdc50115980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.278+0000 7fdc54caf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc50073680 0x7fdc50115ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.278+0000 7fdc54caf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc5010dbb0 0x7fdc501a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.278+0000 7fdc54caf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5011c330 con 0x7fdc50073680 2024-09-10T09:01:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.278+0000 7fdc54caf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5011c1b0 con 0x7fdc5010dbb0 2024-09-10T09:01:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.278+0000 7fdc54caf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5011c4b0 con 0x7fdc50072cb0 2024-09-10T09:01:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.279+0000 7fdc4f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc50072cb0 0x7fdc50115980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.280+0000 7fdc4ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc5010dbb0 0x7fdc501a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.280+0000 7fdc4f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc50072cb0 0x7fdc50115980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:60054/0 (socket says 172.21.15.172:60054) 2024-09-10T09:01:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.280+0000 7fdc4f7fe640 1 -- 172.21.15.172:0/4198970021 learned_addr learned my addr 172.21.15.172:0/4198970021 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.281+0000 7fdc4effd640 1 --2- 172.21.15.172:0/4198970021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc50073680 0x7fdc50115ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.282+0000 7fdc4effd640 1 -- 172.21.15.172:0/4198970021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc50072cb0 msgr2=0x7fdc50115980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.282+0000 7fdc4effd640 1 --2- 172.21.15.172:0/4198970021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc50072cb0 0x7fdc50115980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.282+0000 7fdc4effd640 1 -- 172.21.15.172:0/4198970021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc5010dbb0 msgr2=0x7fdc501a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.282+0000 7fdc4effd640 1 --2- 172.21.15.172:0/4198970021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc5010dbb0 0x7fdc501a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.282+0000 7fdc4effd640 1 -- 172.21.15.172:0/4198970021 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc501a5f70 con 0x7fdc50073680 2024-09-10T09:01:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.283+0000 7fdc4effd640 1 --2- 172.21.15.172:0/4198970021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc50073680 0x7fdc50115ec0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fdc500743a0 tx=0x7fdc3800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.283+0000 7fdc4cff9640 1 -- 172.21.15.172:0/4198970021 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38017070 con 0x7fdc50073680 2024-09-10T09:01:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.283+0000 7fdc54caf640 1 -- 172.21.15.172:0/4198970021 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc501a6260 con 0x7fdc50073680 2024-09-10T09:01:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.283+0000 7fdc54caf640 1 -- 172.21.15.172:0/4198970021 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc501a6800 con 0x7fdc50073680 2024-09-10T09:01:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.283+0000 7fdc4cff9640 1 -- 172.21.15.172:0/4198970021 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc38005020 con 0x7fdc50073680 2024-09-10T09:01:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.284+0000 7fdc4cff9640 1 -- 172.21.15.172:0/4198970021 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38004c60 con 0x7fdc50073680 2024-09-10T09:01:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.288+0000 7fdc4cff9640 1 -- 172.21.15.172:0/4198970021 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fdc380051d0 con 0x7fdc50073680 2024-09-10T09:01:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.291+0000 7fdc58cad740 1 Processor -- start 2024-09-10T09:01:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.291+0000 7fdc58cad740 1 -- start start 2024-09-10T09:01:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.291+0000 7fdc58cad740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f42b70 0x556291f42f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.291+0000 7fdc58cad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f434b0 0x556291f47f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.291+0000 7fdc58cad740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f4aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.291+0000 7fdc58cad740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556291ea62f0 con 0x556291f48540 2024-09-10T09:01:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.291+0000 7fdc58cad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556291e4e270 con 0x556291f434b0 2024-09-10T09:01:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.291+0000 7fdc58cad740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556291ea22b0 con 0x556291f42b70 2024-09-10T09:01:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.292+0000 7fdc4ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f42b70 0x556291f42f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.292+0000 7fdc4ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f42b70 0x556291f42f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:60056/0 (socket says 172.21.15.172:60056) 2024-09-10T09:01:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.292+0000 7fdc4ffff640 1 -- 172.21.15.172:0/1022721260 learned_addr learned my addr 172.21.15.172:0/1022721260 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.292+0000 7fdc4effd640 1 --2- 172.21.15.172:0/1022721260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f4aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.292+0000 7fdc4ffff640 1 -- 172.21.15.172:0/1022721260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f434b0 msgr2=0x556291f47f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:01:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.292+0000 7fdc4ffff640 1 --2- 172.21.15.172:0/1022721260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f434b0 0x556291f47f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.292+0000 7fdc4ffff640 1 -- 172.21.15.172:0/1022721260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 msgr2=0x556291f4aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.292+0000 7fdc4ffff640 1 --2- 172.21.15.172:0/1022721260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f4aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.292+0000 7fdc4ffff640 1 -- 172.21.15.172:0/1022721260 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556291e4d340 con 0x556291f42b70 2024-09-10T09:01:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.292+0000 7fdc4ffff640 1 --2- 172.21.15.172:0/1022721260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f42b70 0x556291f42f70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009f20 tx=0x7fdc4800c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2873a37c5e1662a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.295+0000 7fdc45ffb640 1 -- 172.21.15.172:0/1022721260 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4801a070 con 0x556291f42b70 2024-09-10T09:01:06.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.297+0000 7fdc45ffb640 1 -- 172.21.15.172:0/1022721260 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc480158c0 con 0x556291f42b70 2024-09-10T09:01:06.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.297+0000 7fdc45ffb640 1 -- 172.21.15.172:0/1022721260 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4800a950 con 0x556291f42b70 2024-09-10T09:01:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.298+0000 7fdc4cff9640 1 --2- 172.21.15.172:0/4198970021 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdc3c0a2c00 0x7fdc3c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.298+0000 7fdc4cff9640 1 -- 172.21.15.172:0/4198970021 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fdc380cec20 con 0x7fdc50073680 2024-09-10T09:01:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.298+0000 7fdc58cad740 1 -- 172.21.15.172:0/1022721260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f42b70 msgr2=0x556291f42f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.299+0000 7fdc58cad740 1 --2- 172.21.15.172:0/1022721260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f42b70 0x556291f42f70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009f20 tx=0x7fdc4800c8b0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.300+0000 7fdc58cad740 1 -- 172.21.15.172:0/1022721260 shutdown_connections 2024-09-10T09:01:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.300+0000 7fdc58cad740 1 --2- 172.21.15.172:0/1022721260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f4aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.300+0000 7fdc58cad740 1 --2- 172.21.15.172:0/1022721260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f434b0 0x556291f47f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.300+0000 7fdc58cad740 1 --2- 172.21.15.172:0/1022721260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f42b70 0x556291f42f70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.300+0000 7fdc58cad740 1 -- 172.21.15.172:0/1022721260 >> 172.21.15.172:0/1022721260 conn(0x556291f3e1e0 msgr2=0x556291f3e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.322+0000 7fdc58cad740 1 -- 172.21.15.172:0/1022721260 shutdown_connections 2024-09-10T09:01:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.323+0000 7fdc4f7fe640 1 --2- 172.21.15.172:0/4198970021 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdc3c0a2c00 0x7fdc3c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.324+0000 7fdc58cad740 1 -- 172.21.15.172:0/1022721260 wait complete. 2024-09-10T09:01:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.324+0000 7fdc58cad740 1 -- start start 2024-09-10T09:01:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.325+0000 7fdc58cad740 1 Processor -- start 2024-09-10T09:01:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.325+0000 7fdc58cad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 0x556291f4ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.325+0000 7fdc58cad740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f434b0 0x556291f4d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.325+0000 7fdc58cad740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f543d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.325+0000 7fdc58cad740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556291f4b510 con 0x556291f48540 2024-09-10T09:01:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.325+0000 7fdc58cad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556291e2f290 con 0x556291f42b70 2024-09-10T09:01:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.325+0000 7fdc58cad740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556291f4b690 con 0x556291f434b0 2024-09-10T09:01:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.325+0000 7fdc58cad740 10 client.0 _ll_register_callbacks cb 0x556291e60140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:01:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.326+0000 7fdc4ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.326+0000 7fdc4ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52220/0 (socket says 172.21.15.172:52220) 2024-09-10T09:01:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.326+0000 7fdc4ffff640 1 -- 172.21.15.172:0/3579425114 learned_addr learned my addr 172.21.15.172:0/3579425114 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.326+0000 7fdc4effd640 1 --2- 172.21.15.172:0/3579425114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f434b0 0x556291f4d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.326+0000 7fdc4ffff640 1 -- 172.21.15.172:0/3579425114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f434b0 msgr2=0x556291f4d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.326+0000 7fdc4ffff640 1 --2- 172.21.15.172:0/3579425114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f434b0 0x556291f4d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.326+0000 7fdc4ffff640 1 -- 172.21.15.172:0/3579425114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 msgr2=0x556291f4ce50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:01:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.326+0000 7fdc4ffff640 1 --2- 172.21.15.172:0/3579425114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 0x556291f4ce50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.326+0000 7fdc4ffff640 1 -- 172.21.15.172:0/3579425114 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556291bb9210 con 0x556291f48540 2024-09-10T09:01:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.326+0000 7fdc4ffff640 1 --2- 172.21.15.172:0/3579425114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f543d0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fdc48018040 tx=0x7fdc48015c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d59a1fa59cc61ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.326+0000 7fdc2effd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:01:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.327+0000 7fdc2effd640 1 -- 172.21.15.172:0/3579425114 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4801a040 con 0x556291f48540 2024-09-10T09:01:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.327+0000 7fdc2effd640 1 -- 172.21.15.172:0/3579425114 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fdc48015d70 con 0x556291f48540 2024-09-10T09:01:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.327+0000 7fdc58cad740 1 -- 172.21.15.172:0/3579425114 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc480158c0 con 0x556291f48540 2024-09-10T09:01:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.327+0000 7fdc4f7fe640 1 --2- 172.21.15.172:0/4198970021 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdc3c0a2c00 0x7fdc3c0a50c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdc40000c00 tx=0x7fdc40005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.327+0000 7fdc2effd640 1 -- 172.21.15.172:0/3579425114 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fdc48009b90 con 0x556291f48540 2024-09-10T09:01:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.327+0000 7fdc58cad740 10 client.19506 fetch_fsmap learned FSMap version 43 2024-09-10T09:01:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.327+0000 7fdc58cad740 1 -- 172.21.15.172:0/3579425114 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556291c85620 con 0x556291f48540 2024-09-10T09:01:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.327+0000 7fdc2effd640 1 -- 172.21.15.172:0/3579425114 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc480158c0 con 0x556291f48540 2024-09-10T09:01:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.328+0000 7fdc2effd640 1 -- 172.21.15.172:0/3579425114 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fdc480206b0 con 0x556291f48540 2024-09-10T09:01:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.328+0000 7fdc58cad740 10 client.19506 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T09:01:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.328+0000 7fdc58cad740 10 client.19506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:01:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.328+0000 7fdc58cad740 20 client.19506 populate_metadata read hostname 'smithi172' 2024-09-10T09:01:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.328+0000 7fdc58cad740 1 --2- 172.21.15.172:0/3579425114 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556291f58790 0x556291f5ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.328+0000 7fdc58cad740 4 client.19506 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:01:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.328+0000 7fdc58cad740 1 -- 172.21.15.172:0/3579425114 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556291eb3a20 con 0x556291f58790 2024-09-10T09:01:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.328+0000 7fdc4f7fe640 1 --2- 172.21.15.172:0/3579425114 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556291f58790 0x556291f5ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.329+0000 7fdc2effd640 10 client.19506 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:01:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.329+0000 7fdc4f7fe640 1 --2- 172.21.15.172:0/3579425114 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556291f58790 0x556291f5ab80 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d24ea9680b709846 server_cookie=faf696413a2ec6ff in_seq=0 out_seq=0 2024-09-10T09:01:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.330+0000 7fdc2effd640 1 -- 172.21.15.172:0/3579425114 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556291eb3a20 con 0x556291f58790 2024-09-10T09:01:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.330+0000 7fdc2effd640 10 client.19506 handle_command_reply: tid=0 2024-09-10T09:01:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.330+0000 7fdc58cad740 1 client.19506 shutdown 2024-09-10T09:01:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.330+0000 7fdc58cad740 1 -- 172.21.15.172:0/3579425114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 msgr2=0x556291f543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.330+0000 7fdc58cad740 1 --2- 172.21.15.172:0/3579425114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f543d0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fdc48018040 tx=0x7fdc48015c80 comp rx=0 tx=0).stop 2024-09-10T09:01:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.330+0000 7fdc58cad740 1 -- 172.21.15.172:0/3579425114 shutdown_connections 2024-09-10T09:01:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.330+0000 7fdc58cad740 1 --2- 172.21.15.172:0/3579425114 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556291f58790 0x556291f5ab80 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.330+0000 7fdc58cad740 1 --2- 172.21.15.172:0/3579425114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f543d0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.330+0000 7fdc58cad740 1 --2- 172.21.15.172:0/3579425114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f434b0 0x556291f4d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.330+0000 7fdc58cad740 1 --2- 172.21.15.172:0/3579425114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 0x556291f4ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.330+0000 7fdc58cad740 1 -- 172.21.15.172:0/3579425114 >> 172.21.15.172:0/3579425114 conn(0x556291f3e1e0 msgr2=0x556291f3f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.330+0000 7fdc2effd640 0 client.19506 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:01:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.330+0000 7fdc58cad740 1 -- 172.21.15.172:0/3579425114 shutdown_connections 2024-09-10T09:01:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.330+0000 7fdc58cad740 1 -- 172.21.15.172:0/3579425114 wait complete. 2024-09-10T09:01:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.330+0000 7fdc58cad740 20 client.19506 trim_cache size 0 max 16384 2024-09-10T09:01:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.345+0000 7fdc58cad740 1 Processor -- start 2024-09-10T09:01:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.345+0000 7fdc58cad740 1 -- start start 2024-09-10T09:01:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.345+0000 7fdc58cad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 0x556291f3d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.345+0000 7fdc58cad740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f434b0 0x556291f54390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.345+0000 7fdc58cad740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f3d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.345+0000 7fdc58cad740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3000fba0 con 0x556291f48540 2024-09-10T09:01:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.345+0000 7fdc4f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 0x556291f3d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.345+0000 7fdc4f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 0x556291f3d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:33516/0 (socket says 172.21.15.172:33516) 2024-09-10T09:01:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.345+0000 7fdc4f7fe640 1 -- 172.21.15.172:0/531246616 learned_addr learned my addr 172.21.15.172:0/531246616 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.345+0000 7fdc4effd640 1 --2- 172.21.15.172:0/531246616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f434b0 0x556291f54390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.345+0000 7fdc4ffff640 1 --2- 172.21.15.172:0/531246616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f3d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.345+0000 7fdc58cad740 1 -- 172.21.15.172:0/531246616 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc30011710 con 0x556291f42b70 2024-09-10T09:01:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.345+0000 7fdc58cad740 1 -- 172.21.15.172:0/531246616 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdc30010270 con 0x556291f434b0 2024-09-10T09:01:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.346+0000 7fdc4f7fe640 1 -- 172.21.15.172:0/531246616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f434b0 msgr2=0x556291f54390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.346+0000 7fdc4f7fe640 1 --2- 172.21.15.172:0/531246616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f434b0 0x556291f54390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.346+0000 7fdc4f7fe640 1 -- 172.21.15.172:0/531246616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 msgr2=0x556291f3d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.346+0000 7fdc4f7fe640 1 --2- 172.21.15.172:0/531246616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f3d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.346+0000 7fdc4f7fe640 1 -- 172.21.15.172:0/531246616 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556291f3e000 con 0x556291f42b70 2024-09-10T09:01:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.346+0000 7fdc4effd640 1 --2- 172.21.15.172:0/531246616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f434b0 0x556291f54390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:01:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.346+0000 7fdc4ffff640 1 --2- 172.21.15.172:0/531246616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f3d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:01:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.346+0000 7fdc4f7fe640 1 --2- 172.21.15.172:0/531246616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 0x556291f3d1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdc40005310 tx=0x7fdc4003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c55a75c0567d9e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.346+0000 7fdc457fa640 1 -- 172.21.15.172:0/531246616 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40035020 con 0x556291f42b70 2024-09-10T09:01:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.346+0000 7fdc457fa640 1 -- 172.21.15.172:0/531246616 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc40004440 con 0x556291f42b70 2024-09-10T09:01:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.346+0000 7fdc457fa640 1 -- 172.21.15.172:0/531246616 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc400035f0 con 0x556291f42b70 2024-09-10T09:01:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.347+0000 7fdc58cad740 1 -- 172.21.15.172:0/531246616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 msgr2=0x556291f3d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.347+0000 7fdc58cad740 1 --2- 172.21.15.172:0/531246616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 0x556291f3d1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdc40005310 tx=0x7fdc4003f000 comp rx=0 tx=0).stop 2024-09-10T09:01:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.347+0000 7fdc58cad740 1 -- 172.21.15.172:0/531246616 shutdown_connections 2024-09-10T09:01:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.347+0000 7fdc58cad740 1 --2- 172.21.15.172:0/531246616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f3d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.347+0000 7fdc58cad740 1 --2- 172.21.15.172:0/531246616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f434b0 0x556291f54390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.347+0000 7fdc58cad740 1 --2- 172.21.15.172:0/531246616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 0x556291f3d1a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.347+0000 7fdc58cad740 1 -- 172.21.15.172:0/531246616 >> 172.21.15.172:0/531246616 conn(0x556291f3e1e0 msgr2=0x556291f3ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.347+0000 7fdc58cad740 1 -- 172.21.15.172:0/531246616 shutdown_connections 2024-09-10T09:01:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.347+0000 7fdc58cad740 1 -- 172.21.15.172:0/531246616 wait complete. 2024-09-10T09:01:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.347+0000 7fdc58cad740 1 -- start start 2024-09-10T09:01:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.348+0000 7fdc58cad740 1 Processor -- start 2024-09-10T09:01:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.348+0000 7fdc58cad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 0x556291f518d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.348+0000 7fdc58cad740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f434b0 0x556291f51e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.348+0000 7fdc58cad740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f52dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.348+0000 7fdc4f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 0x556291f518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.348+0000 7fdc4f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 0x556291f518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:33518/0 (socket says 172.21.15.172:33518) 2024-09-10T09:01:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.348+0000 7fdc4f7fe640 1 -- 172.21.15.172:0/2427609758 learned_addr learned my addr 172.21.15.172:0/2427609758 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.348+0000 7fdc4effd640 1 --2- 172.21.15.172:0/2427609758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f434b0 0x556291f51e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.348+0000 7fdc58cad740 1 -- 172.21.15.172:0/2427609758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdc30011040 con 0x556291f48540 2024-09-10T09:01:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.348+0000 7fdc58cad740 1 -- 172.21.15.172:0/2427609758 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc30010970 con 0x556291f42b70 2024-09-10T09:01:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.348+0000 7fdc58cad740 1 -- 172.21.15.172:0/2427609758 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3000db80 con 0x556291f434b0 2024-09-10T09:01:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.348+0000 7fdc4ffff640 1 --2- 172.21.15.172:0/2427609758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f52dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.348+0000 7fdc58cad740 10 client.0 _ll_register_callbacks cb 0x7fdc48018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:01:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.348+0000 7fdc4ffff640 1 -- 172.21.15.172:0/2427609758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f434b0 msgr2=0x556291f51e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.348+0000 7fdc4ffff640 1 --2- 172.21.15.172:0/2427609758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f434b0 0x556291f51e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.348+0000 7fdc4ffff640 1 -- 172.21.15.172:0/2427609758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 msgr2=0x556291f518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.349+0000 7fdc4ffff640 1 --2- 172.21.15.172:0/2427609758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 0x556291f518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.349+0000 7fdc4ffff640 1 -- 172.21.15.172:0/2427609758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556291f533e0 con 0x556291f48540 2024-09-10T09:01:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.349+0000 7fdc4effd640 1 --2- 172.21.15.172:0/2427609758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f434b0 0x556291f51e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:01:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.349+0000 7fdc4f7fe640 1 --2- 172.21.15.172:0/2427609758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 0x556291f518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:01:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.349+0000 7fdc4ffff640 1 --2- 172.21.15.172:0/2427609758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f52dd0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fdc4801f3d0 tx=0x7fdc48015950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2738b36e5a35d353 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.349+0000 7fdc2effd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:01:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.349+0000 7fdc2effd640 1 -- 172.21.15.172:0/2427609758 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4801a050 con 0x556291f48540 2024-09-10T09:01:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.349+0000 7fdc2effd640 1 -- 172.21.15.172:0/2427609758 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fdc48020860 con 0x556291f48540 2024-09-10T09:01:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.349+0000 7fdc58cad740 1 -- 172.21.15.172:0/2427609758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc40004440 con 0x556291f48540 2024-09-10T09:01:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.349+0000 7fdc2effd640 1 -- 172.21.15.172:0/2427609758 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fdc4803b050 con 0x556291f48540 2024-09-10T09:01:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.349+0000 7fdc58cad740 10 client.19512 fetch_fsmap learned FSMap version 43 2024-09-10T09:01:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.349+0000 7fdc58cad740 1 -- 172.21.15.172:0/2427609758 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556291f546b0 con 0x556291f48540 2024-09-10T09:01:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.349+0000 7fdc2effd640 1 -- 172.21.15.172:0/2427609758 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc40004440 con 0x556291f48540 2024-09-10T09:01:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.349+0000 7fdc2effd640 1 -- 172.21.15.172:0/2427609758 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fdc48032560 con 0x556291f48540 2024-09-10T09:01:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.349+0000 7fdc58cad740 10 client.19512 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T09:01:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.349+0000 7fdc58cad740 10 client.19512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:01:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.350+0000 7fdc58cad740 20 client.19512 populate_metadata read hostname 'smithi172' 2024-09-10T09:01:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.350+0000 7fdc58cad740 1 --2- 172.21.15.172:0/2427609758 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556291fc3ac0 0x556291fc5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.350+0000 7fdc58cad740 4 client.19512 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:01:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.350+0000 7fdc58cad740 1 -- 172.21.15.172:0/2427609758 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556291eadd70 con 0x556291fc3ac0 2024-09-10T09:01:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.350+0000 7fdc4f7fe640 1 --2- 172.21.15.172:0/2427609758 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556291fc3ac0 0x556291fc5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.350+0000 7fdc2effd640 10 client.19512 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:01:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:06.350+0000 7fdc4f7fe640 1 --2- 172.21.15.172:0/2427609758 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556291fc3ac0 0x556291fc5ee0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bdc7bdfe120d393 server_cookie=725f2651c01ec744 in_seq=0 out_seq=0 2024-09-10T09:01:07.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:07 smithi191 ceph-mon[35314]: pgmap v797: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 3.48k op/s 2024-09-10T09:01:07.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:07 smithi172 ceph-mon[30854]: pgmap v797: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 3.48k op/s 2024-09-10T09:01:07.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:07 smithi174 ceph-mon[34068]: pgmap v797: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 MiB/s wr, 3.48k op/s 2024-09-10T09:01:09.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:09 smithi191 ceph-mon[35314]: pgmap v798: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 3.53k op/s 2024-09-10T09:01:09.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:09 smithi172 ceph-mon[30854]: pgmap v798: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 3.53k op/s 2024-09-10T09:01:09.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:09 smithi174 ceph-mon[34068]: pgmap v798: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 3.53k op/s 2024-09-10T09:01:11.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:11 smithi191 ceph-mon[35314]: pgmap v799: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 3.17k op/s 2024-09-10T09:01:11.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:11 smithi172 ceph-mon[30854]: pgmap v799: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 3.17k op/s 2024-09-10T09:01:11.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:11 smithi174 ceph-mon[34068]: pgmap v799: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 3.17k op/s 2024-09-10T09:01:13.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:13 smithi191 ceph-mon[35314]: pgmap v800: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-10T09:01:13.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:13 smithi172 ceph-mon[30854]: pgmap v800: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-10T09:01:13.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:13 smithi174 ceph-mon[34068]: pgmap v800: 129 pgs: 129 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.31k op/s 2024-09-10T09:01:14.352 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 141.5 --await-for 45.0' 2024-09-10T09:01:14.353 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 141.5 --await-for 45.0 2024-09-10T09:01:14.465 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:01:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.619+0000 7fdc2effd640 1 -- 172.21.15.172:0/2427609758 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556291eadd70 con 0x556291fc3ac0 2024-09-10T09:01:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.619+0000 7fdc2effd640 10 client.19512 handle_command_reply: tid=0 2024-09-10T09:01:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.620+0000 7fdc58cad740 1 client.19512 shutdown 2024-09-10T09:01:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.620+0000 7fdc58cad740 1 -- 172.21.15.172:0/2427609758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 msgr2=0x556291f52dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.620+0000 7fdc58cad740 1 --2- 172.21.15.172:0/2427609758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f52dd0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fdc4801f3d0 tx=0x7fdc48015950 comp rx=0 tx=0).stop 2024-09-10T09:01:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.620+0000 7fdc58cad740 1 -- 172.21.15.172:0/2427609758 shutdown_connections 2024-09-10T09:01:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.620+0000 7fdc58cad740 1 --2- 172.21.15.172:0/2427609758 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556291fc3ac0 0x556291fc5ee0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.620+0000 7fdc58cad740 1 --2- 172.21.15.172:0/2427609758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556291f48540 0x556291f52dd0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.620+0000 7fdc58cad740 1 --2- 172.21.15.172:0/2427609758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556291f434b0 0x556291f51e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.620+0000 7fdc58cad740 1 --2- 172.21.15.172:0/2427609758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556291f42b70 0x556291f518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.620+0000 7fdc58cad740 1 -- 172.21.15.172:0/2427609758 >> 172.21.15.172:0/2427609758 conn(0x556291f3e1e0 msgr2=0x556291f3d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.620+0000 7fdc2effd640 0 client.19512 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:01:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.623+0000 7fdc58cad740 1 -- 172.21.15.172:0/2427609758 shutdown_connections 2024-09-10T09:01:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.623+0000 7fdc58cad740 1 -- 172.21.15.172:0/2427609758 wait complete. 2024-09-10T09:01:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.623+0000 7fdc58cad740 20 client.19512 trim_cache size 0 max 16384 2024-09-10T09:01:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:01:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:01:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:01:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:01:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.626+0000 7fdc54caf640 1 -- 172.21.15.172:0/4198970021 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdc3c0a2c00 msgr2=0x7fdc3c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.626+0000 7fdc54caf640 1 --2- 172.21.15.172:0/4198970021 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdc3c0a2c00 0x7fdc3c0a50c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdc40000c00 tx=0x7fdc40005a20 comp rx=0 tx=0).stop 2024-09-10T09:01:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.626+0000 7fdc54caf640 1 -- 172.21.15.172:0/4198970021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc50073680 msgr2=0x7fdc50115ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.626+0000 7fdc54caf640 1 --2- 172.21.15.172:0/4198970021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc50073680 0x7fdc50115ec0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fdc500743a0 tx=0x7fdc3800dbb0 comp rx=0 tx=0).stop 2024-09-10T09:01:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.626+0000 7fdc54caf640 1 -- 172.21.15.172:0/4198970021 shutdown_connections 2024-09-10T09:01:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.626+0000 7fdc54caf640 1 --2- 172.21.15.172:0/4198970021 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdc3c0a2c00 0x7fdc3c0a50c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.626+0000 7fdc54caf640 1 --2- 172.21.15.172:0/4198970021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc5010dbb0 0x7fdc501a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.626+0000 7fdc54caf640 1 --2- 172.21.15.172:0/4198970021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc50073680 0x7fdc50115ec0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.626+0000 7fdc54caf640 1 --2- 172.21.15.172:0/4198970021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc50072cb0 0x7fdc50115980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.628+0000 7fdc54caf640 1 -- 172.21.15.172:0/4198970021 >> 172.21.15.172:0/4198970021 conn(0x7fdc5006eb50 msgr2=0x7fdc500708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.629+0000 7fdc54caf640 1 -- 172.21.15.172:0/4198970021 shutdown_connections 2024-09-10T09:01:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:15.629+0000 7fdc54caf640 1 -- 172.21.15.172:0/4198970021 wait complete. 2024-09-10T09:01:15.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:15 smithi191 ceph-mon[35314]: pgmap v801: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.72k op/s 2024-09-10T09:01:15.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:15 smithi172 ceph-mon[30854]: pgmap v801: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.72k op/s 2024-09-10T09:01:15.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:15 smithi174 ceph-mon[34068]: pgmap v801: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 3.72k op/s 2024-09-10T09:01:17.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:17 smithi191 ceph-mon[35314]: pgmap v802: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.25k op/s 2024-09-10T09:01:17.835 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:17 smithi172 ceph-mon[30854]: pgmap v802: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.25k op/s 2024-09-10T09:01:17.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:17 smithi174 ceph-mon[34068]: pgmap v802: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.25k op/s 2024-09-10T09:01:19.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:19 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:01:19.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:19 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:01:19.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:19 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:01:20.664 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:01:20.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.245+0000 7f97de418640 1 -- 172.21.15.172:0/980328722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97d8102ef0 msgr2=0x7f97d810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:20.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.245+0000 7f97de418640 1 --2- 172.21.15.172:0/980328722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97d8102ef0 0x7f97d810da40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009ca0 tx=0x7f97d002a950 comp rx=0 tx=0).stop 2024-09-10T09:01:20.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.245+0000 7f97de418640 1 -- 172.21.15.172:0/980328722 shutdown_connections 2024-09-10T09:01:20.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.245+0000 7f97de418640 1 --2- 172.21.15.172:0/980328722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97d8102ef0 0x7f97d810da40 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.245+0000 7f97de418640 1 --2- 172.21.15.172:0/980328722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97d8102570 0x7f97d81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.245+0000 7f97de418640 1 --2- 172.21.15.172:0/980328722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97d8100e20 0x7f97d8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.245+0000 7f97de418640 1 -- 172.21.15.172:0/980328722 >> 172.21.15.172:0/980328722 conn(0x7f97d80fc7d0 msgr2=0x7f97d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:20.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.245+0000 7f97de418640 1 -- 172.21.15.172:0/980328722 shutdown_connections 2024-09-10T09:01:20.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.245+0000 7f97de418640 1 -- 172.21.15.172:0/980328722 wait complete. 2024-09-10T09:01:20.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.245+0000 7f97de418640 1 Processor -- start 2024-09-10T09:01:20.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.245+0000 7f97de418640 1 -- start start 2024-09-10T09:01:20.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97de418640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97d8100e20 0x7f97d819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97de418640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97d8102570 0x7f97d819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97de418640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97d8102ef0 0x7f97d819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97de418640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f97d8110ed0 con 0x7f97d8102570 2024-09-10T09:01:20.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97de418640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97d8110d50 con 0x7f97d8100e20 2024-09-10T09:01:20.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97de418640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f97d8111050 con 0x7f97d8102ef0 2024-09-10T09:01:20.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97d8102570 0x7f97d819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97d8102570 0x7f97d819ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41214/0 (socket says 172.21.15.172:41214) 2024-09-10T09:01:20.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97d77fe640 1 -- 172.21.15.172:0/618049954 learned_addr learned my addr 172.21.15.172:0/618049954 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:20.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97d7fff640 1 --2- 172.21.15.172:0/618049954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97d8100e20 0x7f97d819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97dc9b5640 1 --2- 172.21.15.172:0/618049954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97d8102ef0 0x7f97d819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97d77fe640 1 -- 172.21.15.172:0/618049954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97d8102ef0 msgr2=0x7f97d819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:20.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97d77fe640 1 --2- 172.21.15.172:0/618049954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97d8102ef0 0x7f97d819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97d77fe640 1 -- 172.21.15.172:0/618049954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97d8100e20 msgr2=0x7f97d819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:20.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97d77fe640 1 --2- 172.21.15.172:0/618049954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97d8100e20 0x7f97d819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97d77fe640 1 -- 172.21.15.172:0/618049954 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d819f6f0 con 0x7f97d8102570 2024-09-10T09:01:20.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97d77fe640 1 --2- 172.21.15.172:0/618049954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97d8102570 0x7f97d819ac80 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00b4a0 tx=0x7f97cc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:20.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.246+0000 7f97d57fa640 1 -- 172.21.15.172:0/618049954 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc015020 con 0x7f97d8102570 2024-09-10T09:01:20.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.247+0000 7f97d57fa640 1 -- 172.21.15.172:0/618049954 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97cc011ce0 con 0x7f97d8102570 2024-09-10T09:01:20.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.247+0000 7f97de418640 1 -- 172.21.15.172:0/618049954 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d819f9e0 con 0x7f97d8102570 2024-09-10T09:01:20.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.247+0000 7f97d57fa640 1 -- 172.21.15.172:0/618049954 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc010dc0 con 0x7f97d8102570 2024-09-10T09:01:20.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.247+0000 7f97de418640 1 -- 172.21.15.172:0/618049954 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d81a72c0 con 0x7f97d8102570 2024-09-10T09:01:20.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.249+0000 7f97d57fa640 1 -- 172.21.15.172:0/618049954 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f97cc018020 con 0x7f97d8102570 2024-09-10T09:01:20.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.249+0000 7f97e07f1740 1 Processor -- start 2024-09-10T09:01:20.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.249+0000 7f97e07f1740 1 -- start start 2024-09-10T09:01:20.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.249+0000 7f97e07f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e13c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.249+0000 7f97e07f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e13c5d0 0x55f00e13ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.249+0000 7f97e07f1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e147370 0x55f00e149760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.249+0000 7f97e07f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f00e0a22f0 con 0x55f00e13db40 2024-09-10T09:01:20.668 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.249+0000 7f97e07f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f00e049cf0 con 0x55f00e13c5d0 2024-09-10T09:01:20.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.249+0000 7f97e07f1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f00e09e2b0 con 0x55f00e147370 2024-09-10T09:01:20.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.249+0000 7f97dc9b5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e13c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.249+0000 7f97dc9b5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e13c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41228/0 (socket says 172.21.15.172:41228) 2024-09-10T09:01:20.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.249+0000 7f97dc9b5640 1 -- 172.21.15.172:0/1982143240 learned_addr learned my addr 172.21.15.172:0/1982143240 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:20.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.250+0000 7f97d7fff640 1 --2- 172.21.15.172:0/1982143240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e13c5d0 0x55f00e13ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.250+0000 7f97d77fe640 1 --2- 172.21.15.172:0/1982143240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e147370 0x55f00e149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.250+0000 7f97dc9b5640 1 -- 172.21.15.172:0/1982143240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e147370 msgr2=0x55f00e149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:20.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.250+0000 7f97dc9b5640 1 --2- 172.21.15.172:0/1982143240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e147370 0x55f00e149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.669 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.250+0000 7f97dc9b5640 1 -- 172.21.15.172:0/1982143240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e13c5d0 msgr2=0x55f00e13ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:20.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.250+0000 7f97dc9b5640 1 --2- 172.21.15.172:0/1982143240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e13c5d0 0x55f00e13ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.250+0000 7f97dc9b5640 1 -- 172.21.15.172:0/1982143240 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f00e02e3a0 con 0x55f00e13db40 2024-09-10T09:01:20.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.250+0000 7f97d57fa640 1 --2- 172.21.15.172:0/618049954 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97c00a2b80 0x7f97c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.250+0000 7f97d7fff640 1 --2- 172.21.15.172:0/618049954 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97c00a2b80 0x7f97c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.250+0000 7f97dc9b5640 1 --2- 172.21.15.172:0/1982143240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e13c090 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009a60 tx=0x7f97d002fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654b80f5ead2d942 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:20.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.250+0000 7f97d57fa640 1 -- 172.21.15.172:0/618049954 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f97cc0cecc0 con 0x7f97d8102570 2024-09-10T09:01:20.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.250+0000 7f97be7fc640 1 -- 172.21.15.172:0/1982143240 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0034040 con 0x55f00e13db40 2024-09-10T09:01:20.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.250+0000 7f97be7fc640 1 -- 172.21.15.172:0/1982143240 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d002fb40 con 0x55f00e13db40 2024-09-10T09:01:20.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.251+0000 7f97e07f1740 1 -- 172.21.15.172:0/1982143240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 msgr2=0x55f00e13c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:20.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.251+0000 7f97e07f1740 1 --2- 172.21.15.172:0/1982143240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e13c090 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009a60 tx=0x7f97d002fec0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.251+0000 7f97d7fff640 1 --2- 172.21.15.172:0/618049954 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97c00a2b80 0x7f97c00a5040 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55f00e139ac0 tx=0x7f97c8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:20.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.251+0000 7f97e07f1740 1 -- 172.21.15.172:0/1982143240 shutdown_connections 2024-09-10T09:01:20.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.251+0000 7f97e07f1740 1 --2- 172.21.15.172:0/1982143240 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e147370 0x55f00e149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.251+0000 7f97e07f1740 1 --2- 172.21.15.172:0/1982143240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e13c5d0 0x55f00e13ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.251+0000 7f97e07f1740 1 --2- 172.21.15.172:0/1982143240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e13c090 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.251+0000 7f97e07f1740 1 -- 172.21.15.172:0/1982143240 >> 172.21.15.172:0/1982143240 conn(0x55f00e105310 msgr2=0x55f00e138c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:20.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.251+0000 7f97e07f1740 1 -- 172.21.15.172:0/1982143240 shutdown_connections 2024-09-10T09:01:20.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.251+0000 7f97e07f1740 1 -- 172.21.15.172:0/1982143240 wait complete. 2024-09-10T09:01:20.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.251+0000 7f97e07f1740 1 -- start start 2024-09-10T09:01:20.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97e07f1740 1 Processor -- start 2024-09-10T09:01:20.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97e07f1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e13c5d0 0x55f00e146c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97e07f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97e07f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e147370 0x55f00e144420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97e07f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f00e14a0b0 con 0x55f00e13db40 2024-09-10T09:01:20.671 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97e07f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f00e05d060 con 0x55f00e147370 2024-09-10T09:01:20.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97e07f1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f00e14a230 con 0x55f00e13c5d0 2024-09-10T09:01:20.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97e07f1740 10 client.0 _ll_register_callbacks cb 0x55f00ddb08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:01:20.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e143ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41232/0 (socket says 172.21.15.172:41232) 2024-09-10T09:01:20.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97d77fe640 1 -- 172.21.15.172:0/1722999019 learned_addr learned my addr 172.21.15.172:0/1722999019 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:20.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97d7fff640 1 --2- 172.21.15.172:0/1722999019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e13c5d0 0x55f00e146c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97dc9b5640 1 --2- 172.21.15.172:0/1722999019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e147370 0x55f00e144420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97d77fe640 1 -- 172.21.15.172:0/1722999019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e13c5d0 msgr2=0x55f00e146c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:20.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97d77fe640 1 --2- 172.21.15.172:0/1722999019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e13c5d0 0x55f00e146c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97d77fe640 1 -- 172.21.15.172:0/1722999019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e147370 msgr2=0x55f00e144420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:20.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97d77fe640 1 --2- 172.21.15.172:0/1722999019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e147370 0x55f00e144420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.672 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97d77fe640 1 -- 172.21.15.172:0/1722999019 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f00de133c0 con 0x55f00e13db40 2024-09-10T09:01:20.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.252+0000 7f97dc9b5640 1 --2- 172.21.15.172:0/1722999019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e147370 0x55f00e144420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:01:20.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.253+0000 7f97d77fe640 1 --2- 172.21.15.172:0/1722999019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e143ee0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f97cc09aec0 tx=0x7f97cc09adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cf20e201b37e53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:20.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.253+0000 7f97b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:01:20.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.253+0000 7f97b77fe640 1 -- 172.21.15.172:0/1722999019 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0d0070 con 0x55f00e13db40 2024-09-10T09:01:20.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.253+0000 7f97b77fe640 1 -- 172.21.15.172:0/1722999019 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f97cc097830 con 0x55f00e13db40 2024-09-10T09:01:20.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.253+0000 7f97e07f1740 1 -- 172.21.15.172:0/1722999019 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d002fb40 con 0x55f00e13db40 2024-09-10T09:01:20.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.253+0000 7f97b77fe640 1 -- 172.21.15.172:0/1722999019 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f97cc09b8c0 con 0x55f00e13db40 2024-09-10T09:01:20.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.253+0000 7f97e07f1740 10 client.19524 fetch_fsmap learned FSMap version 44 2024-09-10T09:01:20.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.253+0000 7f97e07f1740 1 -- 172.21.15.172:0/1722999019 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f00e155a90 con 0x55f00e13db40 2024-09-10T09:01:20.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.253+0000 7f97b77fe640 1 -- 172.21.15.172:0/1722999019 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d002fb40 con 0x55f00e13db40 2024-09-10T09:01:20.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.253+0000 7f97b77fe640 1 -- 172.21.15.172:0/1722999019 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f97cc0bb0e0 con 0x55f00e13db40 2024-09-10T09:01:20.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.253+0000 7f97e07f1740 10 client.19524 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:01:20.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.253+0000 7f97e07f1740 10 client.19524 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:01:20.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.253+0000 7f97e07f1740 20 client.19524 populate_metadata read hostname 'smithi172' 2024-09-10T09:01:20.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.253+0000 7f97e07f1740 1 --2- 172.21.15.172:0/1722999019 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55f00e156ef0 0x55f00e1592e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.253+0000 7f97e07f1740 4 client.19524 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:01:20.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.253+0000 7f97e07f1740 1 -- 172.21.15.172:0/1722999019 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f00e0af860 con 0x55f00e156ef0 2024-09-10T09:01:20.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.254+0000 7f97d7fff640 1 --2- 172.21.15.172:0/1722999019 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55f00e156ef0 0x55f00e1592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.254+0000 7f97b77fe640 10 client.19524 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T09:01:20.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.254+0000 7f97d7fff640 1 --2- 172.21.15.172:0/1722999019 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55f00e156ef0 0x55f00e1592e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89eb24c331cac261 server_cookie=dfddea46c50b83a5 in_seq=0 out_seq=0 2024-09-10T09:01:20.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.255+0000 7f97b77fe640 1 -- 172.21.15.172:0/1722999019 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f00e0af860 con 0x55f00e156ef0 2024-09-10T09:01:20.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.255+0000 7f97b77fe640 10 client.19524 handle_command_reply: tid=0 2024-09-10T09:01:20.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.255+0000 7f97e07f1740 1 client.19524 shutdown 2024-09-10T09:01:20.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.255+0000 7f97e07f1740 1 -- 172.21.15.172:0/1722999019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 msgr2=0x55f00e143ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:20.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.255+0000 7f97e07f1740 1 --2- 172.21.15.172:0/1722999019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e143ee0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f97cc09aec0 tx=0x7f97cc09adc0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.255+0000 7f97e07f1740 1 -- 172.21.15.172:0/1722999019 shutdown_connections 2024-09-10T09:01:20.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.255+0000 7f97e07f1740 1 --2- 172.21.15.172:0/1722999019 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55f00e156ef0 0x55f00e1592e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.255+0000 7f97b77fe640 0 client.19524 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T09:01:20.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.255+0000 7f97e07f1740 1 --2- 172.21.15.172:0/1722999019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e147370 0x55f00e144420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.255+0000 7f97e07f1740 1 --2- 172.21.15.172:0/1722999019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e143ee0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.255+0000 7f97e07f1740 1 --2- 172.21.15.172:0/1722999019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e13c5d0 0x55f00e146c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.255+0000 7f97e07f1740 1 -- 172.21.15.172:0/1722999019 >> 172.21.15.172:0/1722999019 conn(0x55f00e105310 msgr2=0x55f00e13e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:20.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.255+0000 7f97e07f1740 1 -- 172.21.15.172:0/1722999019 shutdown_connections 2024-09-10T09:01:20.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.255+0000 7f97e07f1740 1 -- 172.21.15.172:0/1722999019 wait complete. 2024-09-10T09:01:20.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.255+0000 7f97e07f1740 20 client.19524 trim_cache size 0 max 16384 2024-09-10T09:01:20.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.268+0000 7f97e07f1740 1 Processor -- start 2024-09-10T09:01:20.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.268+0000 7f97e07f1740 1 -- start start 2024-09-10T09:01:20.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.268+0000 7f97e07f1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e13c5d0 0x55f00e1533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.268+0000 7f97e07f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e1427b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.268+0000 7f97e07f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e147370 0x55f00e142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.268+0000 7f97e07f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81b0e70 con 0x55f00e13db40 2024-09-10T09:01:20.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.268+0000 7f97e07f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81b29b0 con 0x55f00e147370 2024-09-10T09:01:20.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.268+0000 7f97e07f1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81b1540 con 0x55f00e13c5d0 2024-09-10T09:01:20.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.268+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e1427b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.268+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e1427b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41234/0 (socket says 172.21.15.172:41234) 2024-09-10T09:01:20.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.268+0000 7f97d77fe640 1 -- 172.21.15.172:0/3492678599 learned_addr learned my addr 172.21.15.172:0/3492678599 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:20.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.268+0000 7f97dc9b5640 1 --2- 172.21.15.172:0/3492678599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e147370 0x55f00e142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.269+0000 7f97d7fff640 1 --2- 172.21.15.172:0/3492678599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e13c5d0 0x55f00e1533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.269+0000 7f97d77fe640 1 -- 172.21.15.172:0/3492678599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e13c5d0 msgr2=0x55f00e1533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:20.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.269+0000 7f97d77fe640 1 --2- 172.21.15.172:0/3492678599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e13c5d0 0x55f00e1533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.269+0000 7f97d77fe640 1 -- 172.21.15.172:0/3492678599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e147370 msgr2=0x55f00e142cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:20.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.269+0000 7f97d77fe640 1 --2- 172.21.15.172:0/3492678599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e147370 0x55f00e142cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.269+0000 7f97d77fe640 1 -- 172.21.15.172:0/3492678599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f00e13fae0 con 0x55f00e13db40 2024-09-10T09:01:20.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.269+0000 7f97d7fff640 1 --2- 172.21.15.172:0/3492678599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e13c5d0 0x55f00e1533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:01:20.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.269+0000 7f97d77fe640 1 --2- 172.21.15.172:0/3492678599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e1427b0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0cc7d0 tx=0x7f97cc0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3affe6700110447 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:20.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.269+0000 7f97bdffb640 1 -- 172.21.15.172:0/3492678599 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc09f120 con 0x55f00e13db40 2024-09-10T09:01:20.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.269+0000 7f97bdffb640 1 -- 172.21.15.172:0/3492678599 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97cc09b1c0 con 0x55f00e13db40 2024-09-10T09:01:20.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.269+0000 7f97bdffb640 1 -- 172.21.15.172:0/3492678599 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc09a610 con 0x55f00e13db40 2024-09-10T09:01:20.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.270+0000 7f97e07f1740 1 -- 172.21.15.172:0/3492678599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 msgr2=0x55f00e1427b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:20.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.270+0000 7f97e07f1740 1 --2- 172.21.15.172:0/3492678599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e1427b0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0cc7d0 tx=0x7f97cc0047d0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.270+0000 7f97e07f1740 1 -- 172.21.15.172:0/3492678599 shutdown_connections 2024-09-10T09:01:20.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.270+0000 7f97e07f1740 1 --2- 172.21.15.172:0/3492678599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e147370 0x55f00e142cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.270+0000 7f97e07f1740 1 --2- 172.21.15.172:0/3492678599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e1427b0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.270+0000 7f97e07f1740 1 --2- 172.21.15.172:0/3492678599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e13c5d0 0x55f00e1533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.270+0000 7f97e07f1740 1 -- 172.21.15.172:0/3492678599 >> 172.21.15.172:0/3492678599 conn(0x55f00e105310 msgr2=0x55f00e143ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:20.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.270+0000 7f97e07f1740 1 -- 172.21.15.172:0/3492678599 shutdown_connections 2024-09-10T09:01:20.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.270+0000 7f97e07f1740 1 -- 172.21.15.172:0/3492678599 wait complete. 2024-09-10T09:01:20.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.270+0000 7f97e07f1740 1 -- start start 2024-09-10T09:01:20.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.270+0000 7f97e07f1740 1 Processor -- start 2024-09-10T09:01:20.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97e07f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e13c5d0 0x55f00e14fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97e07f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e14b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97e07f1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e147370 0x55f00e14b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97e07f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81b22e0 con 0x55f00e13db40 2024-09-10T09:01:20.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97e07f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81b1c10 con 0x55f00e13c5d0 2024-09-10T09:01:20.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97e07f1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f97d81aee50 con 0x55f00e147370 2024-09-10T09:01:20.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97e07f1740 10 client.0 _ll_register_callbacks cb 0x7f97cc010380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:01:20.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e14b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97d7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e13c5d0 0x55f00e14fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97d77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e14b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41236/0 (socket says 172.21.15.172:41236) 2024-09-10T09:01:20.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97d77fe640 1 -- 172.21.15.172:0/1261561447 learned_addr learned my addr 172.21.15.172:0/1261561447 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:20.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97dc9b5640 1 --2- 172.21.15.172:0/1261561447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e147370 0x55f00e14b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97d77fe640 1 -- 172.21.15.172:0/1261561447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e147370 msgr2=0x55f00e14b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:20.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97d77fe640 1 --2- 172.21.15.172:0/1261561447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e147370 0x55f00e14b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97d77fe640 1 -- 172.21.15.172:0/1261561447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e13c5d0 msgr2=0x55f00e14fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:20.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97d77fe640 1 --2- 172.21.15.172:0/1261561447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e13c5d0 0x55f00e14fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97d77fe640 1 -- 172.21.15.172:0/1261561447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f00e146fa0 con 0x55f00e13db40 2024-09-10T09:01:20.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97d7fff640 1 --2- 172.21.15.172:0/1261561447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e13c5d0 0x55f00e14fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:01:20.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97d77fe640 1 --2- 172.21.15.172:0/1261561447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e14b0a0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f97cc09ac90 tx=0x7f97cc0b4280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d69103de3af1dcd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:20.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:01:20.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97b77fe640 1 -- 172.21.15.172:0/1261561447 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc09f570 con 0x55f00e13db40 2024-09-10T09:01:20.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97b77fe640 1 -- 172.21.15.172:0/1261561447 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f97cc011830 con 0x55f00e13db40 2024-09-10T09:01:20.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97e07f1740 1 -- 172.21.15.172:0/1261561447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97cc09b1c0 con 0x55f00e13db40 2024-09-10T09:01:20.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97b77fe640 1 -- 172.21.15.172:0/1261561447 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f97cc011a30 con 0x55f00e13db40 2024-09-10T09:01:20.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97e07f1740 10 client.19530 fetch_fsmap learned FSMap version 44 2024-09-10T09:01:20.680 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.271+0000 7f97e07f1740 1 -- 172.21.15.172:0/1261561447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f00e1c1a30 con 0x55f00e13db40 2024-09-10T09:01:20.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.272+0000 7f97b77fe640 1 -- 172.21.15.172:0/1261561447 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97cc09b1c0 con 0x55f00e13db40 2024-09-10T09:01:20.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.272+0000 7f97b77fe640 1 -- 172.21.15.172:0/1261561447 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f97cc09a080 con 0x55f00e13db40 2024-09-10T09:01:20.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.272+0000 7f97e07f1740 10 client.19530 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:01:20.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.272+0000 7f97e07f1740 10 client.19530 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:01:20.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.272+0000 7f97e07f1740 20 client.19530 populate_metadata read hostname 'smithi172' 2024-09-10T09:01:20.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.272+0000 7f97e07f1740 1 --2- 172.21.15.172:0/1261561447 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55f00e156ef0 0x55f00e1c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:20.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.272+0000 7f97e07f1740 4 client.19530 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:01:20.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.272+0000 7f97e07f1740 1 -- 172.21.15.172:0/1261561447 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f00e0a9d70 con 0x55f00e156ef0 2024-09-10T09:01:20.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.272+0000 7f97d7fff640 1 --2- 172.21.15.172:0/1261561447 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55f00e156ef0 0x55f00e1c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:20.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.273+0000 7f97b77fe640 10 client.19530 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T09:01:20.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.273+0000 7f97d7fff640 1 --2- 172.21.15.172:0/1261561447 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55f00e156ef0 0x55f00e1c3f10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eddf96da2cd6ef26 server_cookie=d0fe26f0f767fb87 in_seq=0 out_seq=0 2024-09-10T09:01:20.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.273+0000 7f97b77fe640 1 -- 172.21.15.172:0/1261561447 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f00e0a9d70 con 0x55f00e156ef0 2024-09-10T09:01:20.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.273+0000 7f97b77fe640 10 client.19530 handle_command_reply: tid=0 2024-09-10T09:01:20.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.273+0000 7f97e07f1740 1 client.19530 shutdown 2024-09-10T09:01:20.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.273+0000 7f97e07f1740 1 -- 172.21.15.172:0/1261561447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 msgr2=0x55f00e14b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:20.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.273+0000 7f97e07f1740 1 --2- 172.21.15.172:0/1261561447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e14b0a0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f97cc09ac90 tx=0x7f97cc0b4280 comp rx=0 tx=0).stop 2024-09-10T09:01:20.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.273+0000 7f97e07f1740 1 -- 172.21.15.172:0/1261561447 shutdown_connections 2024-09-10T09:01:20.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.273+0000 7f97e07f1740 1 --2- 172.21.15.172:0/1261561447 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55f00e156ef0 0x55f00e1c3f10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.273+0000 7f97e07f1740 1 --2- 172.21.15.172:0/1261561447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f00e147370 0x55f00e14b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.273+0000 7f97e07f1740 1 --2- 172.21.15.172:0/1261561447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f00e13db40 0x55f00e14b0a0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.273+0000 7f97b77fe640 0 client.19530 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T09:01:20.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.273+0000 7f97e07f1740 1 --2- 172.21.15.172:0/1261561447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f00e13c5d0 0x55f00e14fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.682 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.273+0000 7f97e07f1740 1 -- 172.21.15.172:0/1261561447 >> 172.21.15.172:0/1261561447 conn(0x55f00e105310 msgr2=0x55f00e1374e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:20.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.273+0000 7f97e07f1740 1 -- 172.21.15.172:0/1261561447 shutdown_connections 2024-09-10T09:01:20.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.273+0000 7f97e07f1740 1 -- 172.21.15.172:0/1261561447 wait complete. 2024-09-10T09:01:20.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.273+0000 7f97e07f1740 20 client.19530 trim_cache size 0 max 16384 2024-09-10T09:01:20.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.274+0000 7f97de418640 1 -- 172.21.15.172:0/618049954 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97c00a2b80 msgr2=0x7f97c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:20.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.274+0000 7f97de418640 1 --2- 172.21.15.172:0/618049954 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97c00a2b80 0x7f97c00a5040 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55f00e139ac0 tx=0x7f97c8009110 comp rx=0 tx=0).stop 2024-09-10T09:01:20.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.274+0000 7f97de418640 1 -- 172.21.15.172:0/618049954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97d8102570 msgr2=0x7f97d819ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:20.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.274+0000 7f97de418640 1 --2- 172.21.15.172:0/618049954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97d8102570 0x7f97d819ac80 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00b4a0 tx=0x7f97cc00b8a0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.274+0000 7f97de418640 1 -- 172.21.15.172:0/618049954 shutdown_connections 2024-09-10T09:01:20.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.274+0000 7f97de418640 1 --2- 172.21.15.172:0/618049954 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f97c00a2b80 0x7f97c00a5040 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.274+0000 7f97de418640 1 --2- 172.21.15.172:0/618049954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97d8102ef0 0x7f97d819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.274+0000 7f97de418640 1 --2- 172.21.15.172:0/618049954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97d8102570 0x7f97d819ac80 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.683 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.274+0000 7f97de418640 1 --2- 172.21.15.172:0/618049954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97d8100e20 0x7f97d819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:20.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.274+0000 7f97de418640 1 -- 172.21.15.172:0/618049954 >> 172.21.15.172:0/618049954 conn(0x7f97d80fc7d0 msgr2=0x7f97d806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:20.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.274+0000 7f97de418640 1 -- 172.21.15.172:0/618049954 shutdown_connections 2024-09-10T09:01:20.684 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:19.274+0000 7f97de418640 1 -- 172.21.15.172:0/618049954 wait complete. 2024-09-10T09:01:20.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:20 smithi174 ceph-mon[34068]: pgmap v803: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 3.37k op/s 2024-09-10T09:01:21.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:20 smithi191 ceph-mon[35314]: pgmap v803: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 3.37k op/s 2024-09-10T09:01:21.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:20 smithi172 ceph-mon[30854]: pgmap v803: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 3.37k op/s 2024-09-10T09:01:22.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:22 smithi174 ceph-mon[34068]: pgmap v804: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T09:01:23.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:22 smithi191 ceph-mon[35314]: pgmap v804: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T09:01:23.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:22 smithi172 ceph-mon[30854]: pgmap v804: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T09:01:24.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:24 smithi174 ceph-mon[34068]: pgmap v805: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 3.02k op/s 2024-09-10T09:01:25.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:24 smithi191 ceph-mon[35314]: pgmap v805: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 3.02k op/s 2024-09-10T09:01:25.077 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:24 smithi172 ceph-mon[30854]: pgmap v805: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 3.02k op/s 2024-09-10T09:01:26.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:26 smithi172 ceph-mon[30854]: pgmap v806: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-10T09:01:26.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:26 smithi174 ceph-mon[34068]: pgmap v806: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-10T09:01:27.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:26 smithi191 ceph-mon[35314]: pgmap v806: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 3.28k op/s 2024-09-10T09:01:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.790+0000 7f3ce905e640 1 -- 172.21.15.172:0/3256973277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ce4104ef0 msgr2=0x7f3ce41072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.790+0000 7f3ce905e640 1 --2- 172.21.15.172:0/3256973277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ce4104ef0 0x7f3ce41072e0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009900 tx=0x7f3cd802a410 comp rx=0 tx=0).stop 2024-09-10T09:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.790+0000 7f3ce905e640 1 -- 172.21.15.172:0/3256973277 shutdown_connections 2024-09-10T09:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.790+0000 7f3ce905e640 1 --2- 172.21.15.172:0/3256973277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ce4107820 0x7f3ce4109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.790+0000 7f3ce905e640 1 --2- 172.21.15.172:0/3256973277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ce4104ef0 0x7f3ce41072e0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.790+0000 7f3ce905e640 1 --2- 172.21.15.172:0/3256973277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ce41025c0 0x7f3ce41049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.790+0000 7f3ce905e640 1 -- 172.21.15.172:0/3256973277 >> 172.21.15.172:0/3256973277 conn(0x7f3ce40fc1e0 msgr2=0x7f3ce40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.790+0000 7f3ce905e640 1 -- 172.21.15.172:0/3256973277 shutdown_connections 2024-09-10T09:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.790+0000 7f3ce905e640 1 -- 172.21.15.172:0/3256973277 wait complete. 2024-09-10T09:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.791+0000 7f3ce905e640 1 Processor -- start 2024-09-10T09:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.791+0000 7f3ce905e640 1 -- start start 2024-09-10T09:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.791+0000 7f3ce905e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ce41025c0 0x7f3ce4197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.791+0000 7f3ce905e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ce4104ef0 0x7f3ce41982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.791+0000 7f3ce905e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ce4107820 0x7f3ce419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.791+0000 7f3ce905e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce410d680 con 0x7f3ce41025c0 2024-09-10T09:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.791+0000 7f3ce905e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce410d500 con 0x7f3ce4107820 2024-09-10T09:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.791+0000 7f3ce905e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce410d800 con 0x7f3ce4104ef0 2024-09-10T09:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.791+0000 7f3ce2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ce41025c0 0x7f3ce4197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.791+0000 7f3ce2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ce41025c0 0x7f3ce4197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45500/0 (socket says 172.21.15.172:45500) 2024-09-10T09:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.791+0000 7f3ce2d9d640 1 -- 172.21.15.172:0/3896751407 learned_addr learned my addr 172.21.15.172:0/3896751407 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.791+0000 7f3ce359e640 1 --2- 172.21.15.172:0/3896751407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ce4107820 0x7f3ce419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.791+0000 7f3ce259c640 1 --2- 172.21.15.172:0/3896751407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ce4104ef0 0x7f3ce41982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.792+0000 7f3ce2d9d640 1 -- 172.21.15.172:0/3896751407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ce4104ef0 msgr2=0x7f3ce41982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.792+0000 7f3ce2d9d640 1 --2- 172.21.15.172:0/3896751407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ce4104ef0 0x7f3ce41982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.792+0000 7f3ce2d9d640 1 -- 172.21.15.172:0/3896751407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ce4107820 msgr2=0x7f3ce419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.792+0000 7f3ce2d9d640 1 --2- 172.21.15.172:0/3896751407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ce4107820 0x7f3ce419f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.792+0000 7f3ce2d9d640 1 -- 172.21.15.172:0/3896751407 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce419f980 con 0x7f3ce41025c0 2024-09-10T09:01:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.792+0000 7f3ce2d9d640 1 --2- 172.21.15.172:0/3896751407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ce41025c0 0x7f3ce4197d60 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400e9f0 tx=0x7f3cd400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.792+0000 7f3ccbfff640 1 -- 172.21.15.172:0/3896751407 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd400ccc0 con 0x7f3ce41025c0 2024-09-10T09:01:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.792+0000 7f3ce905e640 1 -- 172.21.15.172:0/3896751407 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce419fc70 con 0x7f3ce41025c0 2024-09-10T09:01:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.792+0000 7f3ccbfff640 1 -- 172.21.15.172:0/3896751407 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cd40108e0 con 0x7f3ce41025c0 2024-09-10T09:01:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.792+0000 7f3ccbfff640 1 -- 172.21.15.172:0/3896751407 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd40187c0 con 0x7f3ce41025c0 2024-09-10T09:01:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.792+0000 7f3ce905e640 1 -- 172.21.15.172:0/3896751407 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce41a0190 con 0x7f3ce41025c0 2024-09-10T09:01:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.794+0000 7f3ccbfff640 1 -- 172.21.15.172:0/3896751407 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3cd4010a90 con 0x7f3ce41025c0 2024-09-10T09:01:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3cebdf8740 1 Processor -- start 2024-09-10T09:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3cebdf8740 1 -- start start 2024-09-10T09:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3cebdf8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec19a0 0x55f188ec1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3cebdf8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec22e0 0x55f188ec6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3cebdf8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec7370 0x55f188ec9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3cebdf8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f188e252f0 con 0x55f188ec7370 2024-09-10T09:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3cebdf8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f188dcd270 con 0x55f188ec19a0 2024-09-10T09:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3cebdf8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f188e212b0 con 0x55f188ec22e0 2024-09-10T09:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3ce259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec7370 0x55f188ec9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3ce359e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec19a0 0x55f188ec1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3ce2d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec22e0 0x55f188ec6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3ce259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec7370 0x55f188ec9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45514/0 (socket says 172.21.15.172:45514) 2024-09-10T09:01:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3ce259c640 1 -- 172.21.15.172:0/318601712 learned_addr learned my addr 172.21.15.172:0/318601712 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3ccbfff640 1 --2- 172.21.15.172:0/3896751407 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3ccc0a2c30 0x7f3ccc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3ce2d9d640 1 -- 172.21.15.172:0/318601712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec19a0 msgr2=0x55f188ec1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3ce2d9d640 1 --2- 172.21.15.172:0/318601712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec19a0 0x55f188ec1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3ce2d9d640 1 -- 172.21.15.172:0/318601712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec7370 msgr2=0x55f188ec9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3ce2d9d640 1 --2- 172.21.15.172:0/318601712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec7370 0x55f188ec9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3ce2d9d640 1 -- 172.21.15.172:0/318601712 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f188dcc340 con 0x55f188ec22e0 2024-09-10T09:01:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3ce259c640 1 --2- 172.21.15.172:0/318601712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec7370 0x55f188ec9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:01:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3ce359e640 1 --2- 172.21.15.172:0/3896751407 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3ccc0a2c30 0x7f3ccc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.795+0000 7f3ccbfff640 1 -- 172.21.15.172:0/3896751407 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3cd40cf8b0 con 0x7f3ce41025c0 2024-09-10T09:01:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.796+0000 7f3ce2d9d640 1 --2- 172.21.15.172:0/318601712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec22e0 0x55f188ec6da0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3cd409ba50 tx=0x7f3cd409b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f24254dff8c2154 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.796+0000 7f3cc97fa640 1 -- 172.21.15.172:0/318601712 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4020770 con 0x55f188ec22e0 2024-09-10T09:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.796+0000 7f3ce359e640 1 --2- 172.21.15.172:0/3896751407 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3ccc0a2c30 0x7f3ccc0a50f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3ce406a980 tx=0x7f3cdc004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.796+0000 7f3cc97fa640 1 -- 172.21.15.172:0/318601712 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cd4010e00 con 0x55f188ec22e0 2024-09-10T09:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.796+0000 7f3cc97fa640 1 -- 172.21.15.172:0/318601712 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4025170 con 0x55f188ec22e0 2024-09-10T09:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.796+0000 7f3cebdf8740 1 -- 172.21.15.172:0/318601712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec22e0 msgr2=0x55f188ec6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.796+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/318601712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec22e0 0x55f188ec6da0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3cd409ba50 tx=0x7f3cd409b7e0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3cebdf8740 1 -- 172.21.15.172:0/318601712 shutdown_connections 2024-09-10T09:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/318601712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec7370 0x55f188ec9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/318601712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec22e0 0x55f188ec6da0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/318601712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec19a0 0x55f188ec1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3cebdf8740 1 -- 172.21.15.172:0/318601712 >> 172.21.15.172:0/318601712 conn(0x55f188ebd010 msgr2=0x55f188ebd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3cebdf8740 1 -- 172.21.15.172:0/318601712 shutdown_connections 2024-09-10T09:01:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3cebdf8740 1 -- 172.21.15.172:0/318601712 wait complete. 2024-09-10T09:01:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3cebdf8740 1 -- start start 2024-09-10T09:01:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3cebdf8740 1 Processor -- start 2024-09-10T09:01:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3cebdf8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec19a0 0x55f188ecbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3cebdf8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec22e0 0x55f188ecc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3cebdf8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec7370 0x55f188ed3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3cebdf8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f188eca340 con 0x55f188ec22e0 2024-09-10T09:01:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3cebdf8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f188dae290 con 0x55f188ec7370 2024-09-10T09:01:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3cebdf8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f188eca4c0 con 0x55f188ec19a0 2024-09-10T09:01:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3cebdf8740 10 client.0 _ll_register_callbacks cb 0x55f188ddf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:01:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3ce259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec22e0 0x55f188ecc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3ce259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec22e0 0x55f188ecc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45528/0 (socket says 172.21.15.172:45528) 2024-09-10T09:01:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3ce259c640 1 -- 172.21.15.172:0/960357447 learned_addr learned my addr 172.21.15.172:0/960357447 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3ce2d9d640 1 --2- 172.21.15.172:0/960357447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec19a0 0x55f188ecbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.797+0000 7f3ce359e640 1 --2- 172.21.15.172:0/960357447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec7370 0x55f188ed3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3ce259c640 1 -- 172.21.15.172:0/960357447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec19a0 msgr2=0x55f188ecbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3ce259c640 1 --2- 172.21.15.172:0/960357447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec19a0 0x55f188ecbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3ce259c640 1 -- 172.21.15.172:0/960357447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec7370 msgr2=0x55f188ed3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3ce259c640 1 --2- 172.21.15.172:0/960357447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec7370 0x55f188ed3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3ce259c640 1 -- 172.21.15.172:0/960357447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f188b38210 con 0x55f188ec22e0 2024-09-10T09:01:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3ce259c640 1 --2- 172.21.15.172:0/960357447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec22e0 0x55f188ecc0d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f3cd802a930 tx=0x7f3cd80318f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ac22b6ad897e16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3cc6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:01:28.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3cc6ff5640 1 -- 172.21.15.172:0/960357447 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8035040 con 0x55f188ec22e0 2024-09-10T09:01:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3cc6ff5640 1 -- 172.21.15.172:0/960357447 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3cd8004380 con 0x55f188ec22e0 2024-09-10T09:01:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3cebdf8740 1 -- 172.21.15.172:0/960357447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f188e32a20 con 0x55f188ec22e0 2024-09-10T09:01:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3cc6ff5640 1 -- 172.21.15.172:0/960357447 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3cd804e050 con 0x55f188ec22e0 2024-09-10T09:01:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3cebdf8740 10 client.19542 fetch_fsmap learned FSMap version 44 2024-09-10T09:01:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3cebdf8740 1 -- 172.21.15.172:0/960357447 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f188c04620 con 0x55f188ec22e0 2024-09-10T09:01:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3cc6ff5640 1 -- 172.21.15.172:0/960357447 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55f188e32a20 con 0x55f188ec22e0 2024-09-10T09:01:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3cc6ff5640 1 -- 172.21.15.172:0/960357447 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f3cd8004bb0 con 0x55f188ec22e0 2024-09-10T09:01:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3cebdf8740 10 client.19542 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:01:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3cebdf8740 10 client.19542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:01:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3cebdf8740 20 client.19542 populate_metadata read hostname 'smithi172' 2024-09-10T09:01:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/960357447 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f188ed77b0 0x55f188ed9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3cebdf8740 4 client.19542 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:01:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.798+0000 7f3cebdf8740 1 -- 172.21.15.172:0/960357447 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f188e2cd70 con 0x55f188ed77b0 2024-09-10T09:01:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.799+0000 7f3ce2d9d640 1 --2- 172.21.15.172:0/960357447 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f188ed77b0 0x55f188ed9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.799+0000 7f3cc6ff5640 10 client.19542 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:01:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.799+0000 7f3ce2d9d640 1 --2- 172.21.15.172:0/960357447 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f188ed77b0 0x55f188ed9ba0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c5fc89d975a6e5 server_cookie=8d3b3ed6a44a49ea in_seq=0 out_seq=0 2024-09-10T09:01:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.800+0000 7f3cc6ff5640 1 -- 172.21.15.172:0/960357447 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f188e2cd70 con 0x55f188ed77b0 2024-09-10T09:01:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.800+0000 7f3cc6ff5640 10 client.19542 handle_command_reply: tid=0 2024-09-10T09:01:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.800+0000 7f3cebdf8740 1 client.19542 shutdown 2024-09-10T09:01:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.800+0000 7f3cebdf8740 1 -- 172.21.15.172:0/960357447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec22e0 msgr2=0x55f188ecc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.800+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/960357447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec22e0 0x55f188ecc0d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f3cd802a930 tx=0x7f3cd80318f0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.800+0000 7f3cebdf8740 1 -- 172.21.15.172:0/960357447 shutdown_connections 2024-09-10T09:01:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.800+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/960357447 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f188ed77b0 0x55f188ed9ba0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.800+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/960357447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec7370 0x55f188ed3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.800+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/960357447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec22e0 0x55f188ecc0d0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.800+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/960357447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec19a0 0x55f188ecbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.800+0000 7f3cc6ff5640 0 client.19542 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:01:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.800+0000 7f3cebdf8740 1 -- 172.21.15.172:0/960357447 >> 172.21.15.172:0/960357447 conn(0x55f188ebd010 msgr2=0x55f188ebe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.800+0000 7f3cebdf8740 1 -- 172.21.15.172:0/960357447 shutdown_connections 2024-09-10T09:01:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.800+0000 7f3cebdf8740 1 -- 172.21.15.172:0/960357447 wait complete. 2024-09-10T09:01:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.800+0000 7f3cebdf8740 20 client.19542 trim_cache size 0 max 16384 2024-09-10T09:01:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3cebdf8740 1 Processor -- start 2024-09-10T09:01:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3cebdf8740 1 -- start start 2024-09-10T09:01:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3cebdf8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec19a0 0x55f188ecb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3cebdf8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec22e0 0x55f188ed30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3cebdf8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec7370 0x55f188ebc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3cebdf8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce41a8380 con 0x55f188ec7370 2024-09-10T09:01:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3cebdf8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce41a9ec0 con 0x55f188ec19a0 2024-09-10T09:01:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3cebdf8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce41a8a50 con 0x55f188ec22e0 2024-09-10T09:01:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3ce359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec7370 0x55f188ebc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3ce359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec7370 0x55f188ebc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45540/0 (socket says 172.21.15.172:45540) 2024-09-10T09:01:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3ce359e640 1 -- 172.21.15.172:0/1399351683 learned_addr learned my addr 172.21.15.172:0/1399351683 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3ce2d9d640 1 --2- 172.21.15.172:0/1399351683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec19a0 0x55f188ecb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3ce259c640 1 --2- 172.21.15.172:0/1399351683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec22e0 0x55f188ed30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3ce359e640 1 -- 172.21.15.172:0/1399351683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec22e0 msgr2=0x55f188ed30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3ce359e640 1 --2- 172.21.15.172:0/1399351683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec22e0 0x55f188ed30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3ce359e640 1 -- 172.21.15.172:0/1399351683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec19a0 msgr2=0x55f188ecb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3ce359e640 1 --2- 172.21.15.172:0/1399351683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec19a0 0x55f188ecb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3ce359e640 1 -- 172.21.15.172:0/1399351683 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f188ec27c0 con 0x55f188ec7370 2024-09-10T09:01:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3ce259c640 1 --2- 172.21.15.172:0/1399351683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec22e0 0x55f188ed30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:01:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.814+0000 7f3ce359e640 1 --2- 172.21.15.172:0/1399351683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec7370 0x55f188ebc0b0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00f9b0 tx=0x7f3cdc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1552ed679ad5cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.815+0000 7f3cc8ff9640 1 -- 172.21.15.172:0/1399351683 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc01c9d0 con 0x55f188ec7370 2024-09-10T09:01:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.815+0000 7f3cc8ff9640 1 -- 172.21.15.172:0/1399351683 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cdc02d070 con 0x55f188ec7370 2024-09-10T09:01:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.815+0000 7f3cc8ff9640 1 -- 172.21.15.172:0/1399351683 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc01b3b0 con 0x55f188ec7370 2024-09-10T09:01:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.815+0000 7f3cebdf8740 1 -- 172.21.15.172:0/1399351683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec7370 msgr2=0x55f188ebc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.815+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/1399351683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec7370 0x55f188ebc0b0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00f9b0 tx=0x7f3cdc01a000 comp rx=0 tx=0).stop 2024-09-10T09:01:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.815+0000 7f3cebdf8740 1 -- 172.21.15.172:0/1399351683 shutdown_connections 2024-09-10T09:01:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.815+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/1399351683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec7370 0x55f188ebc0b0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.815+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/1399351683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec22e0 0x55f188ed30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.815+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/1399351683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec19a0 0x55f188ecb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.815+0000 7f3cebdf8740 1 -- 172.21.15.172:0/1399351683 >> 172.21.15.172:0/1399351683 conn(0x55f188ebd010 msgr2=0x55f188ebb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.815+0000 7f3cebdf8740 1 -- 172.21.15.172:0/1399351683 shutdown_connections 2024-09-10T09:01:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.815+0000 7f3cebdf8740 1 -- 172.21.15.172:0/1399351683 wait complete. 2024-09-10T09:01:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.815+0000 7f3cebdf8740 1 -- start start 2024-09-10T09:01:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.815+0000 7f3cebdf8740 1 Processor -- start 2024-09-10T09:01:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3cebdf8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec19a0 0x55f188ed0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3cebdf8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec22e0 0x55f188ed0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3cebdf8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec7370 0x55f188ed1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3cebdf8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce41a97f0 con 0x55f188ec22e0 2024-09-10T09:01:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3cebdf8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce41a9120 con 0x55f188ec19a0 2024-09-10T09:01:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3cebdf8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce41a6360 con 0x55f188ec7370 2024-09-10T09:01:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3cebdf8740 10 client.0 _ll_register_callbacks cb 0x7f3cd4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:01:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3ce259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec22e0 0x55f188ed0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3ce259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec22e0 0x55f188ed0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45550/0 (socket says 172.21.15.172:45550) 2024-09-10T09:01:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3ce259c640 1 -- 172.21.15.172:0/1591980693 learned_addr learned my addr 172.21.15.172:0/1591980693 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3ce2d9d640 1 --2- 172.21.15.172:0/1591980693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec19a0 0x55f188ed0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3ce359e640 1 --2- 172.21.15.172:0/1591980693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec7370 0x55f188ed1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3ce259c640 1 -- 172.21.15.172:0/1591980693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec7370 msgr2=0x55f188ed1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3ce259c640 1 --2- 172.21.15.172:0/1591980693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec7370 0x55f188ed1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3ce259c640 1 -- 172.21.15.172:0/1591980693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec19a0 msgr2=0x55f188ed0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3ce259c640 1 --2- 172.21.15.172:0/1591980693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec19a0 0x55f188ed0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3ce259c640 1 -- 172.21.15.172:0/1591980693 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f188ed1f80 con 0x55f188ec22e0 2024-09-10T09:01:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3ce359e640 1 --2- 172.21.15.172:0/1591980693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec7370 0x55f188ed1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:01:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3ce259c640 1 --2- 172.21.15.172:0/1591980693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec22e0 0x55f188ed0960 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80314e0 tx=0x7f3cd80037e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b5933fced81490 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3cc6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:01:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3cc6ff5640 1 -- 172.21.15.172:0/1591980693 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8035030 con 0x55f188ec22e0 2024-09-10T09:01:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3cc6ff5640 1 -- 172.21.15.172:0/1591980693 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3cd804b040 con 0x55f188ec22e0 2024-09-10T09:01:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3cebdf8740 1 -- 172.21.15.172:0/1591980693 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cdc02d070 con 0x55f188ec22e0 2024-09-10T09:01:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.816+0000 7f3cc6ff5640 1 -- 172.21.15.172:0/1591980693 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3cd805f050 con 0x55f188ec22e0 2024-09-10T09:01:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.817+0000 7f3cebdf8740 10 client.19548 fetch_fsmap learned FSMap version 44 2024-09-10T09:01:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.817+0000 7f3cebdf8740 1 -- 172.21.15.172:0/1591980693 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f188ed3290 con 0x55f188ec22e0 2024-09-10T09:01:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.817+0000 7f3cc6ff5640 1 -- 172.21.15.172:0/1591980693 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cdc02d070 con 0x55f188ec22e0 2024-09-10T09:01:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.817+0000 7f3cc6ff5640 1 -- 172.21.15.172:0/1591980693 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f3cd80052e0 con 0x55f188ec22e0 2024-09-10T09:01:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.817+0000 7f3cebdf8740 10 client.19548 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:01:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.817+0000 7f3cebdf8740 10 client.19548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:01:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.817+0000 7f3cebdf8740 20 client.19548 populate_metadata read hostname 'smithi172' 2024-09-10T09:01:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.817+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/1591980693 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f188f42aa0 0x55f188f44ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.817+0000 7f3cebdf8740 4 client.19548 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:01:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.817+0000 7f3cebdf8740 1 -- 172.21.15.172:0/1591980693 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f188da9900 con 0x55f188f42aa0 2024-09-10T09:01:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.817+0000 7f3ce2d9d640 1 --2- 172.21.15.172:0/1591980693 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f188f42aa0 0x55f188f44ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.817+0000 7f3cc6ff5640 10 client.19548 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:01:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:28.817+0000 7f3ce2d9d640 1 --2- 172.21.15.172:0/1591980693 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f188f42aa0 0x55f188f44ec0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4b93912e5c2b398 server_cookie=9dffd7541726e98a in_seq=0 out_seq=0 2024-09-10T09:01:28.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:28 smithi174 ceph-mon[34068]: pgmap v807: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 MiB/s wr, 2.67k op/s 2024-09-10T09:01:29.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:28 smithi191 ceph-mon[35314]: pgmap v807: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 MiB/s wr, 2.67k op/s 2024-09-10T09:01:29.085 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:28 smithi172 ceph-mon[30854]: pgmap v807: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 MiB/s wr, 2.67k op/s 2024-09-10T09:01:30.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:30 smithi174 ceph-mon[34068]: pgmap v808: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 2.86k op/s 2024-09-10T09:01:31.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:30 smithi191 ceph-mon[35314]: pgmap v808: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 2.86k op/s 2024-09-10T09:01:31.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:30 smithi172 ceph-mon[30854]: pgmap v808: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 2.86k op/s 2024-09-10T09:01:31.607 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:01:31.671 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:01:32.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:32 smithi174 ceph-mon[34068]: pgmap v809: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 2.68k op/s 2024-09-10T09:01:33.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:32 smithi191 ceph-mon[35314]: pgmap v809: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 2.68k op/s 2024-09-10T09:01:33.084 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:32 smithi172 ceph-mon[30854]: pgmap v809: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 2.68k op/s 2024-09-10T09:01:34.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:34 smithi174 ceph-mon[34068]: pgmap v810: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T09:01:35.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:34 smithi191 ceph-mon[35314]: pgmap v810: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T09:01:35.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:34 smithi172 ceph-mon[30854]: pgmap v810: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T09:01:36.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:36 smithi174 ceph-mon[34068]: pgmap v811: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.40k op/s 2024-09-10T09:01:37.039 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:36 smithi172 ceph-mon[30854]: pgmap v811: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.40k op/s 2024-09-10T09:01:37.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:36 smithi191 ceph-mon[35314]: pgmap v811: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.40k op/s 2024-09-10T09:01:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.667+0000 7f3cc6ff5640 1 -- 172.21.15.172:0/1591980693 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f188da9900 con 0x55f188f42aa0 2024-09-10T09:01:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.667+0000 7f3cc6ff5640 10 client.19548 handle_command_reply: tid=0 2024-09-10T09:01:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.667+0000 7f3cebdf8740 1 client.19548 shutdown 2024-09-10T09:01:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.668+0000 7f3cebdf8740 1 -- 172.21.15.172:0/1591980693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec22e0 msgr2=0x55f188ed0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.668+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/1591980693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec22e0 0x55f188ed0960 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80314e0 tx=0x7f3cd80037e0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.668+0000 7f3cebdf8740 1 -- 172.21.15.172:0/1591980693 shutdown_connections 2024-09-10T09:01:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.668+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/1591980693 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f188f42aa0 0x55f188f44ec0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.668+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/1591980693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f188ec7370 0x55f188ed1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.668+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/1591980693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f188ec22e0 0x55f188ed0960 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.668+0000 7f3cebdf8740 1 --2- 172.21.15.172:0/1591980693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f188ec19a0 0x55f188ed0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.668+0000 7f3cebdf8740 1 -- 172.21.15.172:0/1591980693 >> 172.21.15.172:0/1591980693 conn(0x55f188ebd010 msgr2=0x55f188ebc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.668+0000 7f3cc6ff5640 0 client.19548 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:01:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.668+0000 7f3cebdf8740 1 -- 172.21.15.172:0/1591980693 shutdown_connections 2024-09-10T09:01:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.668+0000 7f3cebdf8740 1 -- 172.21.15.172:0/1591980693 wait complete. 2024-09-10T09:01:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.668+0000 7f3cebdf8740 20 client.19548 trim_cache size 0 max 16384 2024-09-10T09:01:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:01:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:01:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:01:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:01:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.669+0000 7f3ce905e640 1 -- 172.21.15.172:0/3896751407 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3ccc0a2c30 msgr2=0x7f3ccc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.669+0000 7f3ce905e640 1 --2- 172.21.15.172:0/3896751407 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3ccc0a2c30 0x7f3ccc0a50f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3ce406a980 tx=0x7f3cdc004a40 comp rx=0 tx=0).stop 2024-09-10T09:01:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.669+0000 7f3ce905e640 1 -- 172.21.15.172:0/3896751407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ce41025c0 msgr2=0x7f3ce4197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.669+0000 7f3ce905e640 1 --2- 172.21.15.172:0/3896751407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ce41025c0 0x7f3ce4197d60 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400e9f0 tx=0x7f3cd400edf0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.669+0000 7f3ce905e640 1 -- 172.21.15.172:0/3896751407 shutdown_connections 2024-09-10T09:01:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.669+0000 7f3ce905e640 1 --2- 172.21.15.172:0/3896751407 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3ccc0a2c30 0x7f3ccc0a50f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.669+0000 7f3ce905e640 1 --2- 172.21.15.172:0/3896751407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ce4107820 0x7f3ce419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.669+0000 7f3ce905e640 1 --2- 172.21.15.172:0/3896751407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ce4104ef0 0x7f3ce41982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.669+0000 7f3ce905e640 1 --2- 172.21.15.172:0/3896751407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ce41025c0 0x7f3ce4197d60 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.669+0000 7f3ce905e640 1 -- 172.21.15.172:0/3896751407 >> 172.21.15.172:0/3896751407 conn(0x7f3ce40fc1e0 msgr2=0x7f3ce4102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.670+0000 7f3ce905e640 1 -- 172.21.15.172:0/3896751407 shutdown_connections 2024-09-10T09:01:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:38.670+0000 7f3ce905e640 1 -- 172.21.15.172:0/3896751407 wait complete. 2024-09-10T09:01:38.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.179+0000 7f4c3462d640 1 -- 172.21.15.172:0/3576969220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c2c1011e0 msgr2=0x7f4c2c105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:38.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.179+0000 7f4c3462d640 1 --2- 172.21.15.172:0/3576969220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c2c1011e0 0x7f4c2c105980 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009900 tx=0x7f4c1c02a410 comp rx=0 tx=0).stop 2024-09-10T09:01:38.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.179+0000 7f4c3462d640 1 -- 172.21.15.172:0/3576969220 shutdown_connections 2024-09-10T09:01:38.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.179+0000 7f4c3462d640 1 --2- 172.21.15.172:0/3576969220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c105ec0 0x7f4c2c106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.179+0000 7f4c3462d640 1 --2- 172.21.15.172:0/3576969220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c2c1011e0 0x7f4c2c105980 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.179+0000 7f4c3462d640 1 --2- 172.21.15.172:0/3576969220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c2c1008c0 0x7f4c2c100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.179+0000 7f4c3462d640 1 -- 172.21.15.172:0/3576969220 >> 172.21.15.172:0/3576969220 conn(0x7f4c2c0fc7d0 msgr2=0x7f4c2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:38.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.179+0000 7f4c3462d640 1 -- 172.21.15.172:0/3576969220 shutdown_connections 2024-09-10T09:01:38.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.179+0000 7f4c3462d640 1 -- 172.21.15.172:0/3576969220 wait complete. 2024-09-10T09:01:38.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.180+0000 7f4c3462d640 1 Processor -- start 2024-09-10T09:01:38.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.180+0000 7f4c3462d640 1 -- start start 2024-09-10T09:01:38.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.180+0000 7f4c3462d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c2c1008c0 0x7f4c2c1a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.180+0000 7f4c3462d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c1011e0 0x7f4c2c1a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.180+0000 7f4c3462d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c2c105ec0 0x7f4c2c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.180+0000 7f4c3462d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c1184c0 con 0x7f4c2c105ec0 2024-09-10T09:01:38.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.180+0000 7f4c3462d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c118340 con 0x7f4c2c1011e0 2024-09-10T09:01:38.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.180+0000 7f4c3462d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c118640 con 0x7f4c2c1008c0 2024-09-10T09:01:38.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.180+0000 7f4c32bca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c2c105ec0 0x7f4c2c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:38.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.180+0000 7f4c32bca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c2c105ec0 0x7f4c2c19e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53166/0 (socket says 172.21.15.172:53166) 2024-09-10T09:01:38.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.180+0000 7f4c32bca640 1 -- 172.21.15.172:0/1440058377 learned_addr learned my addr 172.21.15.172:0/1440058377 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:38.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.180+0000 7f4c31bc8640 1 --2- 172.21.15.172:0/1440058377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c1011e0 0x7f4c2c1a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:38.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.180+0000 7f4c323c9640 1 --2- 172.21.15.172:0/1440058377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c2c1008c0 0x7f4c2c1a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:38.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.181+0000 7f4c32bca640 1 -- 172.21.15.172:0/1440058377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c2c1008c0 msgr2=0x7f4c2c1a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:38.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.181+0000 7f4c32bca640 1 --2- 172.21.15.172:0/1440058377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c2c1008c0 0x7f4c2c1a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.181+0000 7f4c32bca640 1 -- 172.21.15.172:0/1440058377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c1011e0 msgr2=0x7f4c2c1a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:38.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.181+0000 7f4c32bca640 1 --2- 172.21.15.172:0/1440058377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c1011e0 0x7f4c2c1a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.181+0000 7f4c32bca640 1 -- 172.21.15.172:0/1440058377 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2c19f010 con 0x7f4c2c105ec0 2024-09-10T09:01:38.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.181+0000 7f4c31bc8640 1 --2- 172.21.15.172:0/1440058377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c1011e0 0x7f4c2c1a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:01:38.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.181+0000 7f4c32bca640 1 --2- 172.21.15.172:0/1440058377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c2c105ec0 0x7f4c2c19e8e0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400eb10 tx=0x7f4c2400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:38.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.181+0000 7f4c237fe640 1 -- 172.21.15.172:0/1440058377 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24004b20 con 0x7f4c2c105ec0 2024-09-10T09:01:38.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.181+0000 7f4c237fe640 1 -- 172.21.15.172:0/1440058377 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c24004cc0 con 0x7f4c2c105ec0 2024-09-10T09:01:38.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.181+0000 7f4c237fe640 1 -- 172.21.15.172:0/1440058377 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24010690 con 0x7f4c2c105ec0 2024-09-10T09:01:38.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.181+0000 7f4c3462d640 1 -- 172.21.15.172:0/1440058377 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2c19f300 con 0x7f4c2c105ec0 2024-09-10T09:01:38.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.181+0000 7f4c3462d640 1 -- 172.21.15.172:0/1440058377 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c2c1ab5f0 con 0x7f4c2c105ec0 2024-09-10T09:01:38.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.183+0000 7f4c237fe640 1 -- 172.21.15.172:0/1440058377 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4c24005020 con 0x7f4c2c105ec0 2024-09-10T09:01:38.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.184+0000 7f4c36a06740 1 Processor -- start 2024-09-10T09:01:38.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.184+0000 7f4c36a06740 1 -- start start 2024-09-10T09:01:38.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.184+0000 7f4c237fe640 1 --2- 172.21.15.172:0/1440058377 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4c180a2c00 0x7f4c180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.184+0000 7f4c36a06740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bee760 0x563922beeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.184+0000 7f4c36a06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bef080 0x563922bf9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.184+0000 7f4c36a06740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bf9940 0x563922bfbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.184+0000 7f4c36a06740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563922b542a0 con 0x563922bf9940 2024-09-10T09:01:38.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.184+0000 7f4c36a06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563922afbb10 con 0x563922bef080 2024-09-10T09:01:38.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.184+0000 7f4c36a06740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563922b50260 con 0x563922bee760 2024-09-10T09:01:38.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.184+0000 7f4c31bc8640 1 --2- 172.21.15.172:0/1440058377 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4c180a2c00 0x7f4c180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:38.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.184+0000 7f4c323c9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bee760 0x563922beeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:38.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.185+0000 7f4c237fe640 1 -- 172.21.15.172:0/1440058377 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4c240cf830 con 0x7f4c2c105ec0 2024-09-10T09:01:38.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.185+0000 7f4c32bca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bf9940 0x563922bfbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:38.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.185+0000 7f4c323c9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bee760 0x563922beeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:51000/0 (socket says 172.21.15.172:51000) 2024-09-10T09:01:38.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.185+0000 7f4c32bca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bf9940 0x563922bfbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53168/0 (socket says 172.21.15.172:53168) 2024-09-10T09:01:38.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.185+0000 7f4c323c9640 1 -- 172.21.15.172:0/3474502567 learned_addr learned my addr 172.21.15.172:0/3474502567 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:38.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.185+0000 7f4c323c9640 1 -- 172.21.15.172:0/3474502567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bef080 msgr2=0x563922bf9400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:01:38.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.185+0000 7f4c323c9640 1 --2- 172.21.15.172:0/3474502567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bef080 0x563922bf9400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.185+0000 7f4c323c9640 1 -- 172.21.15.172:0/3474502567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bf9940 msgr2=0x563922bfbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:38.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.185+0000 7f4c323c9640 1 --2- 172.21.15.172:0/3474502567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bf9940 0x563922bfbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.185+0000 7f4c323c9640 1 -- 172.21.15.172:0/3474502567 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639228c5b90 con 0x563922bee760 2024-09-10T09:01:38.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.185+0000 7f4c32bca640 1 --2- 172.21.15.172:0/3474502567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bf9940 0x563922bfbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:01:38.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.185+0000 7f4c31bc8640 1 --2- 172.21.15.172:0/1440058377 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4c180a2c00 0x7f4c180a50c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009230 tx=0x7f4c1c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:38.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.185+0000 7f4c323c9640 1 --2- 172.21.15.172:0/3474502567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bee760 0x563922beeb40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800bcb0 tx=0x7f4c280107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b082437dae36d28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:38.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.185+0000 7f4c20ff9640 1 -- 172.21.15.172:0/3474502567 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2801c480 con 0x563922bee760 2024-09-10T09:01:38.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.185+0000 7f4c20ff9640 1 -- 172.21.15.172:0/3474502567 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c2801cad0 con 0x563922bee760 2024-09-10T09:01:38.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.185+0000 7f4c20ff9640 1 -- 172.21.15.172:0/3474502567 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28011a30 con 0x563922bee760 2024-09-10T09:01:38.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.186+0000 7f4c36a06740 1 -- 172.21.15.172:0/3474502567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bee760 msgr2=0x563922beeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:38.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.186+0000 7f4c36a06740 1 --2- 172.21.15.172:0/3474502567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bee760 0x563922beeb40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800bcb0 tx=0x7f4c280107a0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.186+0000 7f4c36a06740 1 -- 172.21.15.172:0/3474502567 shutdown_connections 2024-09-10T09:01:38.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.186+0000 7f4c36a06740 1 --2- 172.21.15.172:0/3474502567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bf9940 0x563922bfbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.186+0000 7f4c36a06740 1 --2- 172.21.15.172:0/3474502567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bef080 0x563922bf9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.186+0000 7f4c36a06740 1 --2- 172.21.15.172:0/3474502567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bee760 0x563922beeb40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.186+0000 7f4c36a06740 1 -- 172.21.15.172:0/3474502567 >> 172.21.15.172:0/3474502567 conn(0x563922beacf0 msgr2=0x563922beb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:38.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.186+0000 7f4c36a06740 1 -- 172.21.15.172:0/3474502567 shutdown_connections 2024-09-10T09:01:38.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.186+0000 7f4c36a06740 1 -- 172.21.15.172:0/3474502567 wait complete. 2024-09-10T09:01:38.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c36a06740 1 -- start start 2024-09-10T09:01:38.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c36a06740 1 Processor -- start 2024-09-10T09:01:38.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c36a06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bee760 0x563922bf6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c36a06740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bef080 0x563922bf6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c36a06740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bf9940 0x563922bf7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c36a06740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563922bfc510 con 0x563922bef080 2024-09-10T09:01:38.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c36a06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563922adc6e0 con 0x563922bee760 2024-09-10T09:01:38.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c36a06740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563922bfc690 con 0x563922bf9940 2024-09-10T09:01:38.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c36a06740 10 client.0 _ll_register_callbacks cb 0x563922b00a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:01:38.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c31bc8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bef080 0x563922bf6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:38.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c323c9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bee760 0x563922bf6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:38.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c31bc8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bef080 0x563922bf6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53180/0 (socket says 172.21.15.172:53180) 2024-09-10T09:01:38.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c31bc8640 1 -- 172.21.15.172:0/1107714740 learned_addr learned my addr 172.21.15.172:0/1107714740 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:38.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c32bca640 1 --2- 172.21.15.172:0/1107714740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bf9940 0x563922bf7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:38.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c323c9640 1 -- 172.21.15.172:0/1107714740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bf9940 msgr2=0x563922bf7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:38.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c323c9640 1 --2- 172.21.15.172:0/1107714740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bf9940 0x563922bf7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c323c9640 1 -- 172.21.15.172:0/1107714740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bef080 msgr2=0x563922bf6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:38.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c323c9640 1 --2- 172.21.15.172:0/1107714740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bef080 0x563922bf6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.187+0000 7f4c323c9640 1 -- 172.21.15.172:0/1107714740 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563922c059b0 con 0x563922bee760 2024-09-10T09:01:38.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.188+0000 7f4c32bca640 1 --2- 172.21.15.172:0/1107714740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bf9940 0x563922bf7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:01:38.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.188+0000 7f4c323c9640 1 --2- 172.21.15.172:0/1107714740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bee760 0x563922bf6780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4c28000c00 tx=0x7f4c280058f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96f1c8485b6364d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:38.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.188+0000 7f4c15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:01:38.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.188+0000 7f4c15ffb640 1 -- 172.21.15.172:0/1107714740 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2802b060 con 0x563922bee760 2024-09-10T09:01:38.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.188+0000 7f4c36a06740 1 -- 172.21.15.172:0/1107714740 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c2801cad0 con 0x563922bee760 2024-09-10T09:01:38.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.188+0000 7f4c15ffb640 1 -- 172.21.15.172:0/1107714740 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4c2801c790 con 0x563922bee760 2024-09-10T09:01:38.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.188+0000 7f4c15ffb640 1 -- 172.21.15.172:0/1107714740 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4c28032050 con 0x563922bee760 2024-09-10T09:01:38.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.188+0000 7f4c36a06740 10 client.28325 fetch_fsmap learned FSMap version 44 2024-09-10T09:01:38.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.188+0000 7f4c36a06740 1 -- 172.21.15.172:0/1107714740 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563922c08370 con 0x563922bee760 2024-09-10T09:01:38.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.188+0000 7f4c15ffb640 1 -- 172.21.15.172:0/1107714740 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c2801cad0 con 0x563922bee760 2024-09-10T09:01:38.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.189+0000 7f4c15ffb640 1 -- 172.21.15.172:0/1107714740 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f4c28011ab0 con 0x563922bee760 2024-09-10T09:01:38.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.189+0000 7f4c36a06740 10 client.28325 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:01:38.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.189+0000 7f4c36a06740 10 client.28325 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:01:38.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.189+0000 7f4c36a06740 20 client.28325 populate_metadata read hostname 'smithi172' 2024-09-10T09:01:38.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.189+0000 7f4c36a06740 1 --2- 172.21.15.172:0/1107714740 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563922c09780 0x563922c0bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.189+0000 7f4c36a06740 4 client.28325 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:01:38.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.189+0000 7f4c36a06740 1 -- 172.21.15.172:0/1107714740 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563922b619d0 con 0x563922c09780 2024-09-10T09:01:38.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.189+0000 7f4c31bc8640 1 --2- 172.21.15.172:0/1107714740 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563922c09780 0x563922c0bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:38.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.190+0000 7f4c15ffb640 10 client.28325 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:01:38.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.190+0000 7f4c31bc8640 1 --2- 172.21.15.172:0/1107714740 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563922c09780 0x563922c0bb70 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=611d96e336c5727a server_cookie=822bbe92032c1d45 in_seq=0 out_seq=0 2024-09-10T09:01:38.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.190+0000 7f4c15ffb640 1 -- 172.21.15.172:0/1107714740 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563922b619d0 con 0x563922c09780 2024-09-10T09:01:38.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.190+0000 7f4c15ffb640 10 client.28325 handle_command_reply: tid=0 2024-09-10T09:01:38.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.190+0000 7f4c36a06740 1 client.28325 shutdown 2024-09-10T09:01:38.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.190+0000 7f4c36a06740 1 -- 172.21.15.172:0/1107714740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bee760 msgr2=0x563922bf6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:38.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.190+0000 7f4c36a06740 1 --2- 172.21.15.172:0/1107714740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bee760 0x563922bf6780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4c28000c00 tx=0x7f4c280058f0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.190+0000 7f4c36a06740 1 -- 172.21.15.172:0/1107714740 shutdown_connections 2024-09-10T09:01:38.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.190+0000 7f4c36a06740 1 --2- 172.21.15.172:0/1107714740 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563922c09780 0x563922c0bb70 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.190+0000 7f4c15ffb640 0 client.28325 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:01:38.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.190+0000 7f4c36a06740 1 --2- 172.21.15.172:0/1107714740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bf9940 0x563922bf7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.190+0000 7f4c36a06740 1 --2- 172.21.15.172:0/1107714740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bef080 0x563922bf6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.190+0000 7f4c36a06740 1 --2- 172.21.15.172:0/1107714740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bee760 0x563922bf6780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.190+0000 7f4c36a06740 1 -- 172.21.15.172:0/1107714740 >> 172.21.15.172:0/1107714740 conn(0x563922beacf0 msgr2=0x563922be9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:38.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.191+0000 7f4c36a06740 1 -- 172.21.15.172:0/1107714740 shutdown_connections 2024-09-10T09:01:38.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.191+0000 7f4c36a06740 1 -- 172.21.15.172:0/1107714740 wait complete. 2024-09-10T09:01:38.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.191+0000 7f4c36a06740 20 client.28325 trim_cache size 0 max 16384 2024-09-10T09:01:38.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c36a06740 1 Processor -- start 2024-09-10T09:01:38.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c36a06740 1 -- start start 2024-09-10T09:01:38.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c36a06740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bee760 0x563922bf11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c36a06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bef080 0x563922c04ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c36a06740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bf9940 0x563922c053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c36a06740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c1b5150 con 0x563922bee760 2024-09-10T09:01:38.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c36a06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c1b6c90 con 0x563922bef080 2024-09-10T09:01:38.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c36a06740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c1b5820 con 0x563922bf9940 2024-09-10T09:01:38.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c323c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bee760 0x563922bf11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:38.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c323c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bee760 0x563922bf11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53196/0 (socket says 172.21.15.172:53196) 2024-09-10T09:01:38.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c323c9640 1 -- 172.21.15.172:0/3139265128 learned_addr learned my addr 172.21.15.172:0/3139265128 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:38.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c31bc8640 1 --2- 172.21.15.172:0/3139265128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bef080 0x563922c04ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:38.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c32bca640 1 --2- 172.21.15.172:0/3139265128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bf9940 0x563922c053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:38.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c323c9640 1 -- 172.21.15.172:0/3139265128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bf9940 msgr2=0x563922c053e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:38.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c323c9640 1 --2- 172.21.15.172:0/3139265128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bf9940 0x563922c053e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c323c9640 1 -- 172.21.15.172:0/3139265128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bef080 msgr2=0x563922c04ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:38.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c323c9640 1 --2- 172.21.15.172:0/3139265128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bef080 0x563922c04ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c323c9640 1 -- 172.21.15.172:0/3139265128 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563922be6260 con 0x563922bee760 2024-09-10T09:01:38.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.210+0000 7f4c323c9640 1 --2- 172.21.15.172:0/3139265128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bee760 0x563922bf11c0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800b7f0 tx=0x7f4c28011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744a2c82923964c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:38.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.211+0000 7f4c17fff640 1 -- 172.21.15.172:0/3139265128 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2803b070 con 0x563922bee760 2024-09-10T09:01:38.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.211+0000 7f4c17fff640 1 -- 172.21.15.172:0/3139265128 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c28005660 con 0x563922bee760 2024-09-10T09:01:38.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.211+0000 7f4c17fff640 1 -- 172.21.15.172:0/3139265128 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c280266d0 con 0x563922bee760 2024-09-10T09:01:38.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.211+0000 7f4c36a06740 1 -- 172.21.15.172:0/3139265128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bee760 msgr2=0x563922bf11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:38.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.211+0000 7f4c36a06740 1 --2- 172.21.15.172:0/3139265128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bee760 0x563922bf11c0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800b7f0 tx=0x7f4c28011f90 comp rx=0 tx=0).stop 2024-09-10T09:01:38.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.211+0000 7f4c36a06740 1 -- 172.21.15.172:0/3139265128 shutdown_connections 2024-09-10T09:01:38.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.211+0000 7f4c36a06740 1 --2- 172.21.15.172:0/3139265128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bf9940 0x563922c053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.211+0000 7f4c36a06740 1 --2- 172.21.15.172:0/3139265128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bef080 0x563922c04ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.211+0000 7f4c36a06740 1 --2- 172.21.15.172:0/3139265128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bee760 0x563922bf11c0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.211+0000 7f4c36a06740 1 -- 172.21.15.172:0/3139265128 >> 172.21.15.172:0/3139265128 conn(0x563922beacf0 msgr2=0x563922c04790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:38.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.211+0000 7f4c36a06740 1 -- 172.21.15.172:0/3139265128 shutdown_connections 2024-09-10T09:01:38.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.211+0000 7f4c36a06740 1 -- 172.21.15.172:0/3139265128 wait complete. 2024-09-10T09:01:38.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c36a06740 1 -- start start 2024-09-10T09:01:38.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c36a06740 1 Processor -- start 2024-09-10T09:01:38.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c36a06740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bee760 0x563922bfd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c36a06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bef080 0x563922bfde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c36a06740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bf9940 0x563922bfe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c36a06740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c1b65c0 con 0x563922bee760 2024-09-10T09:01:38.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c36a06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c1b5ef0 con 0x563922bef080 2024-09-10T09:01:38.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c36a06740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c1b3130 con 0x563922bf9940 2024-09-10T09:01:38.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c36a06740 10 client.0 _ll_register_callbacks cb 0x7f4c28011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:01:38.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c323c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bee760 0x563922bfd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:38.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c323c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bee760 0x563922bfd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53202/0 (socket says 172.21.15.172:53202) 2024-09-10T09:01:38.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c323c9640 1 -- 172.21.15.172:0/2071119770 learned_addr learned my addr 172.21.15.172:0/2071119770 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:38.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c31bc8640 1 --2- 172.21.15.172:0/2071119770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bef080 0x563922bfde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:38.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c32bca640 1 --2- 172.21.15.172:0/2071119770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bf9940 0x563922bfe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:38.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c323c9640 1 -- 172.21.15.172:0/2071119770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bf9940 msgr2=0x563922bfe3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:38.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c323c9640 1 --2- 172.21.15.172:0/2071119770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bf9940 0x563922bfe3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c323c9640 1 -- 172.21.15.172:0/2071119770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bef080 msgr2=0x563922bfde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:38.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c323c9640 1 --2- 172.21.15.172:0/2071119770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bef080 0x563922bfde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:38.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c323c9640 1 -- 172.21.15.172:0/2071119770 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563922c73370 con 0x563922bee760 2024-09-10T09:01:38.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.212+0000 7f4c32bca640 1 --2- 172.21.15.172:0/2071119770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bf9940 0x563922bfe3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:01:38.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.213+0000 7f4c323c9640 1 --2- 172.21.15.172:0/2071119770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bee760 0x563922bfd920 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f4c28013010 tx=0x7f4c28005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4707ddbb8efc277 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:38.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.213+0000 7f4c15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:01:38.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.213+0000 7f4c15ffb640 1 -- 172.21.15.172:0/2071119770 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28044060 con 0x563922bee760 2024-09-10T09:01:38.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.213+0000 7f4c15ffb640 1 -- 172.21.15.172:0/2071119770 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4c2801ca80 con 0x563922bee760 2024-09-10T09:01:38.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.213+0000 7f4c36a06740 1 -- 172.21.15.172:0/2071119770 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c28005660 con 0x563922bee760 2024-09-10T09:01:38.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.213+0000 7f4c15ffb640 1 -- 172.21.15.172:0/2071119770 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4c2803b040 con 0x563922bee760 2024-09-10T09:01:38.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.213+0000 7f4c36a06740 10 client.19566 fetch_fsmap learned FSMap version 44 2024-09-10T09:01:38.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.213+0000 7f4c36a06740 1 -- 172.21.15.172:0/2071119770 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563922c74630 con 0x563922bee760 2024-09-10T09:01:38.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.213+0000 7f4c15ffb640 1 -- 172.21.15.172:0/2071119770 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c28005660 con 0x563922bee760 2024-09-10T09:01:38.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.213+0000 7f4c15ffb640 1 -- 172.21.15.172:0/2071119770 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f4c280118f0 con 0x563922bee760 2024-09-10T09:01:38.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.213+0000 7f4c36a06740 10 client.19566 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:01:38.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.213+0000 7f4c36a06740 10 client.19566 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:01:38.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.213+0000 7f4c36a06740 20 client.19566 populate_metadata read hostname 'smithi172' 2024-09-10T09:01:38.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.213+0000 7f4c36a06740 1 --2- 172.21.15.172:0/2071119770 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563922c09780 0x563922c76d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:38.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.213+0000 7f4c36a06740 4 client.19566 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.5, "await_for": 45.0}] 2024-09-10T09:01:38.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.213+0000 7f4c36a06740 1 -- 172.21.15.172:0/2071119770 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.5, "await_for": 45.0}) -- 0x563922b5bd20 con 0x563922c09780 2024-09-10T09:01:38.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.213+0000 7f4c31bc8640 1 --2- 172.21.15.172:0/2071119770 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563922c09780 0x563922c76d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:38.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.214+0000 7f4c31bc8640 1 --2- 172.21.15.172:0/2071119770 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563922c09780 0x563922c76d10 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85a154e02f137d5d server_cookie=ea5b17f4db9b0e59 in_seq=0 out_seq=0 2024-09-10T09:01:38.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:37.214+0000 7f4c15ffb640 10 client.19566 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:01:38.973 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:01:38.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:38 smithi174 ceph-mon[34068]: pgmap v812: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 2.92k op/s 2024-09-10T09:01:38.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:38 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:01:39.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:38 smithi191 ceph-mon[35314]: pgmap v812: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 2.92k op/s 2024-09-10T09:01:39.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:38 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:01:39.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:38 smithi172 ceph-mon[30854]: pgmap v812: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 2.92k op/s 2024-09-10T09:01:39.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:38 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:01:39.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:39 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:01:39.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:39 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:01:39.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:39 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:01:40.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:39 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:01:40.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:39 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:01:40.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:39 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:01:40.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:39 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:01:40.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:39 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:01:40.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:39 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:01:40.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.913+0000 7f4c15ffb640 1 -- 172.21.15.172:0/2071119770 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563922b5bd20 con 0x563922c09780 2024-09-10T09:01:40.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.913+0000 7f4c15ffb640 10 client.19566 handle_command_reply: tid=0 2024-09-10T09:01:40.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.913+0000 7f4c36a06740 1 client.19566 shutdown 2024-09-10T09:01:40.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.914+0000 7f4c36a06740 1 -- 172.21.15.172:0/2071119770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bee760 msgr2=0x563922bfd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:40.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.914+0000 7f4c36a06740 1 --2- 172.21.15.172:0/2071119770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bee760 0x563922bfd920 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f4c28013010 tx=0x7f4c28005a70 comp rx=0 tx=0).stop 2024-09-10T09:01:40.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.914+0000 7f4c36a06740 1 -- 172.21.15.172:0/2071119770 shutdown_connections 2024-09-10T09:01:40.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.914+0000 7f4c36a06740 1 --2- 172.21.15.172:0/2071119770 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x563922c09780 0x563922c76d10 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:40.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.914+0000 7f4c15ffb640 0 client.19566 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:01:40.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.914+0000 7f4c36a06740 1 --2- 172.21.15.172:0/2071119770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563922bf9940 0x563922bfe3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:40.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.914+0000 7f4c36a06740 1 --2- 172.21.15.172:0/2071119770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563922bef080 0x563922bfde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:40.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.914+0000 7f4c36a06740 1 --2- 172.21.15.172:0/2071119770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563922bee760 0x563922bfd920 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:40.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.914+0000 7f4c36a06740 1 -- 172.21.15.172:0/2071119770 >> 172.21.15.172:0/2071119770 conn(0x563922beacf0 msgr2=0x563922c04fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:40.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.915+0000 7f4c36a06740 1 -- 172.21.15.172:0/2071119770 shutdown_connections 2024-09-10T09:01:40.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.915+0000 7f4c36a06740 1 -- 172.21.15.172:0/2071119770 wait complete. 2024-09-10T09:01:40.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.915+0000 7f4c36a06740 20 client.19566 trim_cache size 0 max 16384 2024-09-10T09:01:40.917 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:01:40.917 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 44, 2024-09-10T09:01:40.917 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T09:01:40.917 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 50, 2024-09-10T09:01:40.917 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T09:01:40.917 INFO:teuthology.orchestra.run.smithi172.stdout: "3e265114": { 2024-09-10T09:01:40.917 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 50, 2024-09-10T09:01:40.917 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T09:01:40.917 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T09:01:40.918 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T09:01:40.918 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T09:01:40.918 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:01:40.918 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T09:01:40.918 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 141.5, 2024-09-10T09:01:40.918 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T09:01:40.918 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T09:01:40.918 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T09:01:40.918 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T09:01:40.918 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T09:01:40.918 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T09:01:40.918 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:01:40.918 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:01:40.919 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:01:40.919 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:01:40.919 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:01:40.919 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:01:40.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.916+0000 7f4c3462d640 1 -- 172.21.15.172:0/1440058377 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4c180a2c00 msgr2=0x7f4c180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:40.919 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.916+0000 7f4c3462d640 1 --2- 172.21.15.172:0/1440058377 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4c180a2c00 0x7f4c180a50c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009230 tx=0x7f4c1c0058d0 comp rx=0 tx=0).stop 2024-09-10T09:01:40.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.916+0000 7f4c3462d640 1 -- 172.21.15.172:0/1440058377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c2c105ec0 msgr2=0x7f4c2c19e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:40.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.916+0000 7f4c3462d640 1 --2- 172.21.15.172:0/1440058377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c2c105ec0 0x7f4c2c19e8e0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400eb10 tx=0x7f4c2400c2d0 comp rx=0 tx=0).stop 2024-09-10T09:01:40.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.916+0000 7f4c3462d640 1 -- 172.21.15.172:0/1440058377 shutdown_connections 2024-09-10T09:01:40.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.916+0000 7f4c3462d640 1 --2- 172.21.15.172:0/1440058377 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4c180a2c00 0x7f4c180a50c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:40.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.916+0000 7f4c3462d640 1 --2- 172.21.15.172:0/1440058377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c2c105ec0 0x7f4c2c19e8e0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:40.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.916+0000 7f4c3462d640 1 --2- 172.21.15.172:0/1440058377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c1011e0 0x7f4c2c1a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:40.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.916+0000 7f4c3462d640 1 --2- 172.21.15.172:0/1440058377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c2c1008c0 0x7f4c2c1a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:40.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.916+0000 7f4c3462d640 1 -- 172.21.15.172:0/1440058377 >> 172.21.15.172:0/1440058377 conn(0x7f4c2c0fc7d0 msgr2=0x7f4c2c10f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:40.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.918+0000 7f4c3462d640 1 -- 172.21.15.172:0/1440058377 shutdown_connections 2024-09-10T09:01:40.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:40.918+0000 7f4c3462d640 1 -- 172.21.15.172:0/1440058377 wait complete. 2024-09-10T09:01:41.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:41 smithi174 ceph-mon[34068]: pgmap v813: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-10T09:01:41.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:41 smithi191 ceph-mon[35314]: pgmap v813: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-10T09:01:41.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:41 smithi172 ceph-mon[30854]: pgmap v813: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 3.30k op/s 2024-09-10T09:01:43.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:43 smithi174 ceph-mon[34068]: pgmap v814: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 2.95k op/s 2024-09-10T09:01:43.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:43 smithi191 ceph-mon[35314]: pgmap v814: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 2.95k op/s 2024-09-10T09:01:43.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:43 smithi172 ceph-mon[30854]: pgmap v814: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 2.95k op/s 2024-09-10T09:01:45.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:45 smithi174 ceph-mon[34068]: pgmap v815: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T09:01:45.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:45 smithi191 ceph-mon[35314]: pgmap v815: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T09:01:45.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:45 smithi172 ceph-mon[30854]: pgmap v815: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T09:01:47.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:47 smithi174 ceph-mon[34068]: pgmap v816: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 3.03k op/s 2024-09-10T09:01:47.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:47 smithi191 ceph-mon[35314]: pgmap v816: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 3.03k op/s 2024-09-10T09:01:47.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:47 smithi172 ceph-mon[30854]: pgmap v816: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 3.03k op/s 2024-09-10T09:01:49.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:49 smithi174 ceph-mon[34068]: pgmap v817: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.9 MiB/s wr, 2.01k op/s 2024-09-10T09:01:49.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:49 smithi191 ceph-mon[35314]: pgmap v817: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.9 MiB/s wr, 2.01k op/s 2024-09-10T09:01:49.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:49 smithi172 ceph-mon[30854]: pgmap v817: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.9 MiB/s wr, 2.01k op/s 2024-09-10T09:01:51.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:51 smithi174 ceph-mon[34068]: pgmap v818: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 1.71k op/s 2024-09-10T09:01:51.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:51 smithi191 ceph-mon[35314]: pgmap v818: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 1.71k op/s 2024-09-10T09:01:51.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:51 smithi172 ceph-mon[30854]: pgmap v818: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.7 MiB/s wr, 1.71k op/s 2024-09-10T09:01:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.776+0000 7fdf9c83c640 1 -- 172.21.15.172:0/2337937376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf98100a40 msgr2=0x7fdf98100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.776+0000 7fdf9c83c640 1 --2- 172.21.15.172:0/2337937376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf98100a40 0x7fdf98100e40 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009900 tx=0x7fdf8802a410 comp rx=0 tx=0).stop 2024-09-10T09:01:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.776+0000 7fdf9c83c640 1 -- 172.21.15.172:0/2337937376 shutdown_connections 2024-09-10T09:01:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.776+0000 7fdf9c83c640 1 --2- 172.21.15.172:0/2337937376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf981025f0 0x7fdf98108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.776+0000 7fdf9c83c640 1 --2- 172.21.15.172:0/2337937376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf98101c30 0x7fdf981020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.776+0000 7fdf9c83c640 1 --2- 172.21.15.172:0/2337937376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf98100a40 0x7fdf98100e40 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.776+0000 7fdf9c83c640 1 -- 172.21.15.172:0/2337937376 >> 172.21.15.172:0/2337937376 conn(0x7fdf980fc1e0 msgr2=0x7fdf980fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.776+0000 7fdf9c83c640 1 -- 172.21.15.172:0/2337937376 shutdown_connections 2024-09-10T09:01:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.776+0000 7fdf9c83c640 1 -- 172.21.15.172:0/2337937376 wait complete. 2024-09-10T09:01:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.777+0000 7fdf9c83c640 1 Processor -- start 2024-09-10T09:01:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.777+0000 7fdf9c83c640 1 -- start start 2024-09-10T09:01:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.777+0000 7fdf9c83c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf98100a40 0x7fdf9806a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.777+0000 7fdf9c83c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf98101c30 0x7fdf9819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.777+0000 7fdf9c83c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf981025f0 0x7fdf9819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.777+0000 7fdf9c83c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9810c570 con 0x7fdf981025f0 2024-09-10T09:01:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.777+0000 7fdf9c83c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9810c3f0 con 0x7fdf98100a40 2024-09-10T09:01:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.777+0000 7fdf9c83c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9810c6f0 con 0x7fdf98101c30 2024-09-10T09:01:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.777+0000 7fdf96d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf981025f0 0x7fdf9819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.777+0000 7fdf96d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf981025f0 0x7fdf9819f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60766/0 (socket says 172.21.15.172:60766) 2024-09-10T09:01:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.777+0000 7fdf96d9d640 1 -- 172.21.15.172:0/4289544004 learned_addr learned my addr 172.21.15.172:0/4289544004 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.777+0000 7fdf95d9b640 1 --2- 172.21.15.172:0/4289544004 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf98101c30 0x7fdf9819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.777+0000 7fdf9659c640 1 --2- 172.21.15.172:0/4289544004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf98100a40 0x7fdf9806a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.778+0000 7fdf96d9d640 1 -- 172.21.15.172:0/4289544004 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf98101c30 msgr2=0x7fdf9819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.778+0000 7fdf96d9d640 1 --2- 172.21.15.172:0/4289544004 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf98101c30 0x7fdf9819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.778+0000 7fdf96d9d640 1 -- 172.21.15.172:0/4289544004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf98100a40 msgr2=0x7fdf9806a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.778+0000 7fdf96d9d640 1 --2- 172.21.15.172:0/4289544004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf98100a40 0x7fdf9806a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.778+0000 7fdf96d9d640 1 -- 172.21.15.172:0/4289544004 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9819f980 con 0x7fdf981025f0 2024-09-10T09:01:51.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.778+0000 7fdf96d9d640 1 --2- 172.21.15.172:0/4289544004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf981025f0 0x7fdf9819f070 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000eb10 tx=0x7fdf9000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.778+0000 7fdf877fe640 1 -- 172.21.15.172:0/4289544004 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90004b20 con 0x7fdf981025f0 2024-09-10T09:01:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.778+0000 7fdf877fe640 1 -- 172.21.15.172:0/4289544004 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf90004cc0 con 0x7fdf981025f0 2024-09-10T09:01:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.778+0000 7fdf9c83c640 1 -- 172.21.15.172:0/4289544004 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9819fc10 con 0x7fdf981025f0 2024-09-10T09:01:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.778+0000 7fdf877fe640 1 -- 172.21.15.172:0/4289544004 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90010690 con 0x7fdf981025f0 2024-09-10T09:01:51.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.778+0000 7fdf9c83c640 1 -- 172.21.15.172:0/4289544004 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf981a0110 con 0x7fdf981025f0 2024-09-10T09:01:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.780+0000 7fdf877fe640 1 -- 172.21.15.172:0/4289544004 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fdf90005020 con 0x7fdf981025f0 2024-09-10T09:01:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.781+0000 7fdf9f5d8740 1 Processor -- start 2024-09-10T09:01:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.781+0000 7fdf9f5d8740 1 -- start start 2024-09-10T09:01:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.781+0000 7fdf9f5d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.781+0000 7fdf9f5d8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82ec2e0 0x5646b82f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.781+0000 7fdf9f5d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82f1370 0x5646b82f3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.781+0000 7fdf9f5d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5646b824f2f0 con 0x5646b82eb9a0 2024-09-10T09:01:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.781+0000 7fdf9f5d8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5646b81f7270 con 0x5646b82ec2e0 2024-09-10T09:01:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.781+0000 7fdf9f5d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5646b824b2b0 con 0x5646b82f1370 2024-09-10T09:01:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.781+0000 7fdf95d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82ebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.781+0000 7fdf877fe640 1 --2- 172.21.15.172:0/4289544004 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdf800a2c30 0x7fdf800a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.781+0000 7fdf95d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82ebda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60780/0 (socket says 172.21.15.172:60780) 2024-09-10T09:01:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.781+0000 7fdf95d9b640 1 -- 172.21.15.172:0/3219260507 learned_addr learned my addr 172.21.15.172:0/3219260507 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.781+0000 7fdf9659c640 1 --2- 172.21.15.172:0/3219260507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82ec2e0 0x5646b82f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.781+0000 7fdf96d9d640 1 --2- 172.21.15.172:0/4289544004 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdf800a2c30 0x7fdf800a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.782+0000 7fdf877fe640 1 -- 172.21.15.172:0/4289544004 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fdf900cf9e0 con 0x7fdf981025f0 2024-09-10T09:01:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.782+0000 7fdf95d9b640 1 --2- 172.21.15.172:0/3219260507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82f1370 0x5646b82f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.782+0000 7fdf95d9b640 1 -- 172.21.15.172:0/3219260507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82f1370 msgr2=0x5646b82f3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.782+0000 7fdf95d9b640 1 --2- 172.21.15.172:0/3219260507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82f1370 0x5646b82f3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.782+0000 7fdf95d9b640 1 -- 172.21.15.172:0/3219260507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82ec2e0 msgr2=0x5646b82f0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.782+0000 7fdf95d9b640 1 --2- 172.21.15.172:0/3219260507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82ec2e0 0x5646b82f0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.782+0000 7fdf95d9b640 1 -- 172.21.15.172:0/3219260507 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646b81f6340 con 0x5646b82eb9a0 2024-09-10T09:01:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.782+0000 7fdf9659c640 1 --2- 172.21.15.172:0/3219260507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82ec2e0 0x5646b82f0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:01:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.782+0000 7fdf95d9b640 1 --2- 172.21.15.172:0/3219260507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82ebda0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00b8f0 tx=0x7fdf8c004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab886f7666b7d5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.782+0000 7fdf96d9d640 1 --2- 172.21.15.172:0/4289544004 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdf800a2c30 0x7fdf800a50f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdf90098340 tx=0x7fdf900913c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.782+0000 7fdf84ff9640 1 -- 172.21.15.172:0/3219260507 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c003d70 con 0x5646b82eb9a0 2024-09-10T09:01:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.782+0000 7fdf84ff9640 1 -- 172.21.15.172:0/3219260507 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf8c0054d0 con 0x5646b82eb9a0 2024-09-10T09:01:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.782+0000 7fdf84ff9640 1 -- 172.21.15.172:0/3219260507 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c010460 con 0x5646b82eb9a0 2024-09-10T09:01:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.783+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3219260507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 msgr2=0x5646b82ebda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.783+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/3219260507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82ebda0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00b8f0 tx=0x7fdf8c004250 comp rx=0 tx=0).stop 2024-09-10T09:01:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.783+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3219260507 shutdown_connections 2024-09-10T09:01:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.783+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/3219260507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82f1370 0x5646b82f3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.783+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/3219260507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82ec2e0 0x5646b82f0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.783+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/3219260507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82ebda0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.783+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3219260507 >> 172.21.15.172:0/3219260507 conn(0x5646b82e7010 msgr2=0x5646b82e7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.783+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3219260507 shutdown_connections 2024-09-10T09:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.783+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3219260507 wait complete. 2024-09-10T09:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.783+0000 7fdf9f5d8740 1 -- start start 2024-09-10T09:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.783+0000 7fdf9f5d8740 1 Processor -- start 2024-09-10T09:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf9f5d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf9f5d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82ec2e0 0x5646b82f6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf9f5d8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82f1370 0x5646b82fd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf9f5d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5646b82f4340 con 0x5646b82eb9a0 2024-09-10T09:01:51.792 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf9f5d8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5646b81d8290 con 0x5646b82f1370 2024-09-10T09:01:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf9f5d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5646b82f44c0 con 0x5646b82ec2e0 2024-09-10T09:01:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf9f5d8740 10 client.0 _ll_register_callbacks cb 0x5646b8209140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:01:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf95d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf95d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82f5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60794/0 (socket says 172.21.15.172:60794) 2024-09-10T09:01:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf95d9b640 1 -- 172.21.15.172:0/2384287485 learned_addr learned my addr 172.21.15.172:0/2384287485 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf95d9b640 1 --2- 172.21.15.172:0/2384287485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82f1370 0x5646b82fd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf9659c640 1 --2- 172.21.15.172:0/2384287485 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82ec2e0 0x5646b82f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf95d9b640 1 -- 172.21.15.172:0/2384287485 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82ec2e0 msgr2=0x5646b82f6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf95d9b640 1 --2- 172.21.15.172:0/2384287485 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82ec2e0 0x5646b82f6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf95d9b640 1 -- 172.21.15.172:0/2384287485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82f1370 msgr2=0x5646b82fd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf95d9b640 1 --2- 172.21.15.172:0/2384287485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82f1370 0x5646b82fd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf95d9b640 1 -- 172.21.15.172:0/2384287485 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646b7f62210 con 0x5646b82eb9a0 2024-09-10T09:01:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf9659c640 1 --2- 172.21.15.172:0/2384287485 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82ec2e0 0x5646b82f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:01:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf95d9b640 1 --2- 172.21.15.172:0/2384287485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82f5b30 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0057a0 tx=0x7fdf8c005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98b18c6efe65dbf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:01:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf7dffb640 1 -- 172.21.15.172:0/2384287485 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c005130 con 0x5646b82eb9a0 2024-09-10T09:01:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf7dffb640 1 -- 172.21.15.172:0/2384287485 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fdf8c0052d0 con 0x5646b82eb9a0 2024-09-10T09:01:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.784+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/2384287485 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf8c0054d0 con 0x5646b82eb9a0 2024-09-10T09:01:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.785+0000 7fdf7dffb640 1 -- 172.21.15.172:0/2384287485 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fdf8c031050 con 0x5646b82eb9a0 2024-09-10T09:01:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.785+0000 7fdf9f5d8740 10 client.19578 fetch_fsmap learned FSMap version 44 2024-09-10T09:01:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.785+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/2384287485 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5646b802e620 con 0x5646b82eb9a0 2024-09-10T09:01:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.785+0000 7fdf7dffb640 1 -- 172.21.15.172:0/2384287485 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf8c0054d0 con 0x5646b82eb9a0 2024-09-10T09:01:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.785+0000 7fdf7dffb640 1 -- 172.21.15.172:0/2384287485 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fdf8c01e440 con 0x5646b82eb9a0 2024-09-10T09:01:51.794 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.785+0000 7fdf9f5d8740 10 client.19578 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:01:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.785+0000 7fdf9f5d8740 10 client.19578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:01:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.785+0000 7fdf9f5d8740 20 client.19578 populate_metadata read hostname 'smithi172' 2024-09-10T09:01:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.785+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/2384287485 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5646b83015e0 0x5646b8303a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.785+0000 7fdf9f5d8740 4 client.19578 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:01:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.785+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/2384287485 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646b825ca20 con 0x5646b83015e0 2024-09-10T09:01:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.785+0000 7fdf9659c640 1 --2- 172.21.15.172:0/2384287485 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5646b83015e0 0x5646b8303a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.786+0000 7fdf7dffb640 10 client.19578 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:01:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.786+0000 7fdf9659c640 1 --2- 172.21.15.172:0/2384287485 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5646b83015e0 0x5646b8303a40 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65938d305de8104f server_cookie=79d74438e024a8b3 in_seq=0 out_seq=0 2024-09-10T09:01:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.786+0000 7fdf7dffb640 1 -- 172.21.15.172:0/2384287485 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646b825ca20 con 0x5646b83015e0 2024-09-10T09:01:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.787+0000 7fdf7dffb640 10 client.19578 handle_command_reply: tid=0 2024-09-10T09:01:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.787+0000 7fdf9f5d8740 1 client.19578 shutdown 2024-09-10T09:01:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.787+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/2384287485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 msgr2=0x5646b82f5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:51.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.787+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/2384287485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82f5b30 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0057a0 tx=0x7fdf8c005f30 comp rx=0 tx=0).stop 2024-09-10T09:01:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.787+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/2384287485 shutdown_connections 2024-09-10T09:01:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.787+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/2384287485 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5646b83015e0 0x5646b8303a40 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.787+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/2384287485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82f1370 0x5646b82fd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.787+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/2384287485 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82ec2e0 0x5646b82f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.787+0000 7fdf7dffb640 0 client.19578 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:01:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.787+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/2384287485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82f5b30 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.787+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/2384287485 >> 172.21.15.172:0/2384287485 conn(0x5646b82e7010 msgr2=0x5646b82e8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.787+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/2384287485 shutdown_connections 2024-09-10T09:01:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.787+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/2384287485 wait complete. 2024-09-10T09:01:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.787+0000 7fdf9f5d8740 20 client.19578 trim_cache size 0 max 16384 2024-09-10T09:01:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.801+0000 7fdf9f5d8740 1 Processor -- start 2024-09-10T09:01:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.801+0000 7fdf9f5d8740 1 -- start start 2024-09-10T09:01:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.801+0000 7fdf9f5d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.801+0000 7fdf9f5d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82ec2e0 0x5646b82fd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.801+0000 7fdf9f5d8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82f1370 0x5646b82e7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.802+0000 7fdf9f5d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981aa590 con 0x5646b82eb9a0 2024-09-10T09:01:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.802+0000 7fdf9f5d8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981ac0d0 con 0x5646b82f1370 2024-09-10T09:01:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.802+0000 7fdf9f5d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981aac60 con 0x5646b82ec2e0 2024-09-10T09:01:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.802+0000 7fdf95d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.802+0000 7fdf95d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82f5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60806/0 (socket says 172.21.15.172:60806) 2024-09-10T09:01:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.802+0000 7fdf95d9b640 1 -- 172.21.15.172:0/3764447002 learned_addr learned my addr 172.21.15.172:0/3764447002 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.802+0000 7fdf9659c640 1 --2- 172.21.15.172:0/3764447002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82ec2e0 0x5646b82fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.802+0000 7fdf95d9b640 1 -- 172.21.15.172:0/3764447002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82ec2e0 msgr2=0x5646b82fd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.802+0000 7fdf95d9b640 1 --2- 172.21.15.172:0/3764447002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82ec2e0 0x5646b82fd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.802+0000 7fdf95d9b640 1 -- 172.21.15.172:0/3764447002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82f1370 msgr2=0x5646b82e7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.802+0000 7fdf95d9b640 1 --2- 172.21.15.172:0/3764447002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82f1370 0x5646b82e7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.802+0000 7fdf95d9b640 1 -- 172.21.15.172:0/3764447002 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646b82ec7c0 con 0x5646b82eb9a0 2024-09-10T09:01:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.802+0000 7fdf9659c640 1 --2- 172.21.15.172:0/3764447002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82ec2e0 0x5646b82fd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:01:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.802+0000 7fdf95d9b640 1 --2- 172.21.15.172:0/3764447002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82f5ff0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c02f040 tx=0x7fdf8c00bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b24fc82261741a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.802+0000 7fdf7ffff640 1 -- 172.21.15.172:0/3764447002 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0039c0 con 0x5646b82eb9a0 2024-09-10T09:01:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.802+0000 7fdf7ffff640 1 -- 172.21.15.172:0/3764447002 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf8c004590 con 0x5646b82eb9a0 2024-09-10T09:01:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.802+0000 7fdf7ffff640 1 -- 172.21.15.172:0/3764447002 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c01e440 con 0x5646b82eb9a0 2024-09-10T09:01:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.803+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3764447002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 msgr2=0x5646b82f5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.803+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/3764447002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82f5ff0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c02f040 tx=0x7fdf8c00bfb0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.803+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3764447002 shutdown_connections 2024-09-10T09:01:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.803+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/3764447002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82f1370 0x5646b82e7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.803+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/3764447002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82ec2e0 0x5646b82fd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.803+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/3764447002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82f5ff0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.803+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3764447002 >> 172.21.15.172:0/3764447002 conn(0x5646b82e7010 msgr2=0x5646b82f58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.803+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3764447002 shutdown_connections 2024-09-10T09:01:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.803+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3764447002 wait complete. 2024-09-10T09:01:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.803+0000 7fdf9f5d8740 1 -- start start 2024-09-10T09:01:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.803+0000 7fdf9f5d8740 1 Processor -- start 2024-09-10T09:01:51.810 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf9f5d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82fa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf9f5d8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82ec2e0 0x5646b82fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf9f5d8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82f1370 0x5646b82fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf9f5d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981aba00 con 0x5646b82eb9a0 2024-09-10T09:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf9f5d8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981ab330 con 0x5646b82ec2e0 2024-09-10T09:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf9f5d8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdf981a8570 con 0x5646b82f1370 2024-09-10T09:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf9f5d8740 10 client.0 _ll_register_callbacks cb 0x7fdf900093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf95d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82fa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf95d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82fa270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60816/0 (socket says 172.21.15.172:60816) 2024-09-10T09:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf95d9b640 1 -- 172.21.15.172:0/3540242973 learned_addr learned my addr 172.21.15.172:0/3540242973 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf9659c640 1 --2- 172.21.15.172:0/3540242973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82ec2e0 0x5646b82fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf95d9b640 1 -- 172.21.15.172:0/3540242973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82f1370 msgr2=0x5646b82fb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf95d9b640 1 --2- 172.21.15.172:0/3540242973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82f1370 0x5646b82fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf95d9b640 1 -- 172.21.15.172:0/3540242973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82ec2e0 msgr2=0x5646b82fa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf95d9b640 1 --2- 172.21.15.172:0/3540242973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82ec2e0 0x5646b82fa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf95d9b640 1 -- 172.21.15.172:0/3540242973 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646b82fbdd0 con 0x5646b82eb9a0 2024-09-10T09:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf95d9b640 1 --2- 172.21.15.172:0/3540242973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82fa270 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c004400 tx=0x7fdf8c003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980a286e25a2a2f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf7dffb640 1 -- 172.21.15.172:0/3540242973 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c01e640 con 0x5646b82eb9a0 2024-09-10T09:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf7dffb640 1 -- 172.21.15.172:0/3540242973 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fdf8c01ea30 con 0x5646b82eb9a0 2024-09-10T09:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3540242973 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf8c004590 con 0x5646b82eb9a0 2024-09-10T09:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.804+0000 7fdf7dffb640 1 -- 172.21.15.172:0/3540242973 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fdf8c01ec30 con 0x5646b82eb9a0 2024-09-10T09:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.805+0000 7fdf9f5d8740 10 client.19584 fetch_fsmap learned FSMap version 44 2024-09-10T09:01:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.805+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3540242973 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5646b82fd050 con 0x5646b82eb9a0 2024-09-10T09:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.805+0000 7fdf7dffb640 1 -- 172.21.15.172:0/3540242973 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf8c004590 con 0x5646b82eb9a0 2024-09-10T09:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.805+0000 7fdf7dffb640 1 -- 172.21.15.172:0/3540242973 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fdf8c010800 con 0x5646b82eb9a0 2024-09-10T09:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.805+0000 7fdf9f5d8740 10 client.19584 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.805+0000 7fdf9f5d8740 10 client.19584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.805+0000 7fdf9f5d8740 20 client.19584 populate_metadata read hostname 'smithi172' 2024-09-10T09:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.805+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/3540242973 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5646b836c650 0x5646b836ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.805+0000 7fdf9f5d8740 4 client.19584 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.805+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3540242973 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646b8256d70 con 0x5646b836c650 2024-09-10T09:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.805+0000 7fdf9659c640 1 --2- 172.21.15.172:0/3540242973 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5646b836c650 0x5646b836ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.806+0000 7fdf7dffb640 10 client.19584 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:01:51.813 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.806+0000 7fdf9659c640 1 --2- 172.21.15.172:0/3540242973 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5646b836c650 0x5646b836ea70 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27e4361d4faa2e4 server_cookie=1be999e71b4851b4 in_seq=0 out_seq=0 2024-09-10T09:01:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.814+0000 7fdf7dffb640 1 -- 172.21.15.172:0/3540242973 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646b8256d70 con 0x5646b836c650 2024-09-10T09:01:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.814+0000 7fdf7dffb640 10 client.19584 handle_command_reply: tid=0 2024-09-10T09:01:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.814+0000 7fdf9f5d8740 1 client.19584 shutdown 2024-09-10T09:01:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.814+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3540242973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 msgr2=0x5646b82fa270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.814+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/3540242973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82fa270 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c004400 tx=0x7fdf8c003b00 comp rx=0 tx=0).stop 2024-09-10T09:01:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.814+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3540242973 shutdown_connections 2024-09-10T09:01:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.814+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/3540242973 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5646b836c650 0x5646b836ea70 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.814+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/3540242973 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646b82f1370 0x5646b82fb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.814+0000 7fdf7dffb640 0 client.19584 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:01:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.814+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/3540242973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b82ec2e0 0x5646b82fa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.814+0000 7fdf9f5d8740 1 --2- 172.21.15.172:0/3540242973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5646b82eb9a0 0x5646b82fa270 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.814+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3540242973 >> 172.21.15.172:0/3540242973 conn(0x5646b82e7010 msgr2=0x5646b82e59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.814+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3540242973 shutdown_connections 2024-09-10T09:01:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.814+0000 7fdf9f5d8740 1 -- 172.21.15.172:0/3540242973 wait complete. 2024-09-10T09:01:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.814+0000 7fdf9f5d8740 20 client.19584 trim_cache size 0 max 16384 2024-09-10T09:01:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:01:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:01:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:01:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:01:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.815+0000 7fdf9c83c640 1 -- 172.21.15.172:0/4289544004 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdf800a2c30 msgr2=0x7fdf800a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.815+0000 7fdf9c83c640 1 --2- 172.21.15.172:0/4289544004 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdf800a2c30 0x7fdf800a50f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdf90098340 tx=0x7fdf900913c0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.815+0000 7fdf9c83c640 1 -- 172.21.15.172:0/4289544004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf981025f0 msgr2=0x7fdf9819f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.815+0000 7fdf9c83c640 1 --2- 172.21.15.172:0/4289544004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf981025f0 0x7fdf9819f070 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fdf9000eb10 tx=0x7fdf9000c2d0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.815+0000 7fdf9c83c640 1 -- 172.21.15.172:0/4289544004 shutdown_connections 2024-09-10T09:01:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.815+0000 7fdf9c83c640 1 --2- 172.21.15.172:0/4289544004 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fdf800a2c30 0x7fdf800a50f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.815+0000 7fdf9c83c640 1 --2- 172.21.15.172:0/4289544004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf981025f0 0x7fdf9819f070 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.815+0000 7fdf9c83c640 1 --2- 172.21.15.172:0/4289544004 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf98101c30 0x7fdf9819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.815+0000 7fdf9c83c640 1 --2- 172.21.15.172:0/4289544004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf98100a40 0x7fdf9806a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.815+0000 7fdf9c83c640 1 -- 172.21.15.172:0/4289544004 >> 172.21.15.172:0/4289544004 conn(0x7fdf980fc1e0 msgr2=0x7fdf98104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.815+0000 7fdf9c83c640 1 -- 172.21.15.172:0/4289544004 shutdown_connections 2024-09-10T09:01:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:01:51.815+0000 7fdf9c83c640 1 -- 172.21.15.172:0/4289544004 wait complete. 2024-09-10T09:01:53.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:53 smithi174 ceph-mon[34068]: pgmap v819: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.1 MiB/s wr, 1.05k op/s 2024-09-10T09:01:53.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:53 smithi191 ceph-mon[35314]: pgmap v819: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.1 MiB/s wr, 1.05k op/s 2024-09-10T09:01:53.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:53 smithi172 ceph-mon[30854]: pgmap v819: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.1 MiB/s wr, 1.05k op/s 2024-09-10T09:01:53.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.425+0000 7fd8381ce640 1 -- 172.21.15.172:0/1968443352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd830106f40 msgr2=0x7fd83010eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:53.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.425+0000 7fd8381ce640 1 --2- 172.21.15.172:0/1968443352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd830106f40 0x7fd83010eac0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fd820009ca0 tx=0x7fd82002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.425+0000 7fd8381ce640 1 -- 172.21.15.172:0/1968443352 shutdown_connections 2024-09-10T09:01:53.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.425+0000 7fd8381ce640 1 --2- 172.21.15.172:0/1968443352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd830106f40 0x7fd83010eac0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.425+0000 7fd8381ce640 1 --2- 172.21.15.172:0/1968443352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8301027c0 0x7fd830106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.425+0000 7fd8381ce640 1 --2- 172.21.15.172:0/1968443352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd830101e10 0x7fd8301021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.425+0000 7fd8381ce640 1 -- 172.21.15.172:0/1968443352 >> 172.21.15.172:0/1968443352 conn(0x7fd8300fd8d0 msgr2=0x7fd8300ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:53.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.425+0000 7fd8381ce640 1 -- 172.21.15.172:0/1968443352 shutdown_connections 2024-09-10T09:01:53.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.425+0000 7fd8381ce640 1 -- 172.21.15.172:0/1968443352 wait complete. 2024-09-10T09:01:53.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.426+0000 7fd8381ce640 1 Processor -- start 2024-09-10T09:01:53.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.426+0000 7fd8381ce640 1 -- start start 2024-09-10T09:01:53.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.426+0000 7fd8381ce640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd830101e10 0x7fd8301983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.426+0000 7fd8381ce640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8301027c0 0x7fd8301988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.426+0000 7fd8381ce640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd830106f40 0x7fd83019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.426+0000 7fd8381ce640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd830111ec0 con 0x7fd830101e10 2024-09-10T09:01:53.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.426+0000 7fd8381ce640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd830111d40 con 0x7fd8301027c0 2024-09-10T09:01:53.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.426+0000 7fd8381ce640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd830112040 con 0x7fd830106f40 2024-09-10T09:01:53.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.426+0000 7fd835f6a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd830101e10 0x7fd8301983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.426+0000 7fd835f6a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd830101e10 0x7fd8301983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60840/0 (socket says 172.21.15.172:60840) 2024-09-10T09:01:53.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.426+0000 7fd835f6a640 1 -- 172.21.15.172:0/4078012743 learned_addr learned my addr 172.21.15.172:0/4078012743 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:53.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.426+0000 7fd83676b640 1 --2- 172.21.15.172:0/4078012743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd830106f40 0x7fd83019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.426+0000 7fd835769640 1 --2- 172.21.15.172:0/4078012743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8301027c0 0x7fd8301988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.427+0000 7fd835f6a640 1 -- 172.21.15.172:0/4078012743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd830106f40 msgr2=0x7fd83019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:53.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.427+0000 7fd835f6a640 1 --2- 172.21.15.172:0/4078012743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd830106f40 0x7fd83019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.427+0000 7fd835f6a640 1 -- 172.21.15.172:0/4078012743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8301027c0 msgr2=0x7fd8301988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:53.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.427+0000 7fd835f6a640 1 --2- 172.21.15.172:0/4078012743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8301027c0 0x7fd8301988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.427+0000 7fd835f6a640 1 -- 172.21.15.172:0/4078012743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83019d350 con 0x7fd830101e10 2024-09-10T09:01:53.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.427+0000 7fd835769640 1 --2- 172.21.15.172:0/4078012743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8301027c0 0x7fd8301988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:01:53.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.427+0000 7fd83676b640 1 --2- 172.21.15.172:0/4078012743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd830106f40 0x7fd83019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:01:53.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.427+0000 7fd835f6a640 1 --2- 172.21.15.172:0/4078012743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd830101e10 0x7fd8301983a0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fd82800c890 tx=0x7fd82800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:53.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.427+0000 7fd82effd640 1 -- 172.21.15.172:0/4078012743 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828004220 con 0x7fd830101e10 2024-09-10T09:01:53.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.427+0000 7fd82effd640 1 -- 172.21.15.172:0/4078012743 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8280043c0 con 0x7fd830101e10 2024-09-10T09:01:53.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.427+0000 7fd8381ce640 1 -- 172.21.15.172:0/4078012743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd83019d640 con 0x7fd830101e10 2024-09-10T09:01:53.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.427+0000 7fd82effd640 1 -- 172.21.15.172:0/4078012743 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828004a00 con 0x7fd830101e10 2024-09-10T09:01:53.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.427+0000 7fd8381ce640 1 -- 172.21.15.172:0/4078012743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8301a4f20 con 0x7fd830101e10 2024-09-10T09:01:53.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.429+0000 7fd82effd640 1 -- 172.21.15.172:0/4078012743 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd828004ba0 con 0x7fd830101e10 2024-09-10T09:01:53.691 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.429+0000 7fd83a5a7740 1 Processor -- start 2024-09-10T09:01:53.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.430+0000 7fd83a5a7740 1 -- start start 2024-09-10T09:01:53.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.430+0000 7fd83a5a7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a7708b40 0x5563a7707090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.430+0000 7fd83a5a7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a77075d0 0x5563a7707a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.430+0000 7fd83a5a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7716780 0x5563a7718b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.430+0000 7fd83a5a7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5563a766d2f0 con 0x5563a77075d0 2024-09-10T09:01:53.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.430+0000 7fd83a5a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5563a7614cf0 con 0x5563a7716780 2024-09-10T09:01:53.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.430+0000 7fd83a5a7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5563a76692b0 con 0x5563a7708b40 2024-09-10T09:01:53.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.430+0000 7fd835f6a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a77075d0 0x5563a7707a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.430+0000 7fd835f6a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a77075d0 0x5563a7707a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60852/0 (socket says 172.21.15.172:60852) 2024-09-10T09:01:53.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.430+0000 7fd835f6a640 1 -- 172.21.15.172:0/439765069 learned_addr learned my addr 172.21.15.172:0/439765069 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:53.692 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.430+0000 7fd835769640 1 --2- 172.21.15.172:0/439765069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7716780 0x5563a7718b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.430+0000 7fd82effd640 1 --2- 172.21.15.172:0/4078012743 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd8180a2c00 0x7fd8180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.430+0000 7fd83676b640 1 --2- 172.21.15.172:0/439765069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a7708b40 0x5563a7707090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.431+0000 7fd835f6a640 1 -- 172.21.15.172:0/439765069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a7708b40 msgr2=0x5563a7707090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:53.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.431+0000 7fd835f6a640 1 --2- 172.21.15.172:0/439765069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a7708b40 0x5563a7707090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.431+0000 7fd835f6a640 1 -- 172.21.15.172:0/439765069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7716780 msgr2=0x5563a7718b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:53.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.431+0000 7fd835f6a640 1 --2- 172.21.15.172:0/439765069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7716780 0x5563a7718b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.431+0000 7fd835f6a640 1 -- 172.21.15.172:0/439765069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563a75f93a0 con 0x5563a77075d0 2024-09-10T09:01:53.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.431+0000 7fd835769640 1 --2- 172.21.15.172:0/439765069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7716780 0x5563a7718b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:01:53.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.431+0000 7fd83676b640 1 --2- 172.21.15.172:0/439765069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a7708b40 0x5563a7707090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:01:53.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.431+0000 7fd83676b640 1 --2- 172.21.15.172:0/4078012743 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd8180a2c00 0x7fd8180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.431+0000 7fd82effd640 1 -- 172.21.15.172:0/4078012743 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd8280cf090 con 0x7fd830101e10 2024-09-10T09:01:53.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.431+0000 7fd835f6a640 1 --2- 172.21.15.172:0/439765069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a77075d0 0x5563a7707a10 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fd82801f660 tx=0x7fd8280053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2934d4eb723abc24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:53.693 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.431+0000 7fd817fff640 1 -- 172.21.15.172:0/439765069 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82809b0d0 con 0x5563a77075d0 2024-09-10T09:01:53.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.431+0000 7fd817fff640 1 -- 172.21.15.172:0/439765069 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd82801fb60 con 0x5563a77075d0 2024-09-10T09:01:53.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.431+0000 7fd817fff640 1 -- 172.21.15.172:0/439765069 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828004e90 con 0x5563a77075d0 2024-09-10T09:01:53.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.432+0000 7fd83a5a7740 1 -- 172.21.15.172:0/439765069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a77075d0 msgr2=0x5563a7707a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:53.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.432+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/439765069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a77075d0 0x5563a7707a10 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fd82801f660 tx=0x7fd8280053f0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.432+0000 7fd83676b640 1 --2- 172.21.15.172:0/4078012743 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd8180a2c00 0x7fd8180a50c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd820006010 tx=0x7fd8200095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:53.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.432+0000 7fd83a5a7740 1 -- 172.21.15.172:0/439765069 shutdown_connections 2024-09-10T09:01:53.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.432+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/439765069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7716780 0x5563a7718b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.432+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/439765069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a77075d0 0x5563a7707a10 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.432+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/439765069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a7708b40 0x5563a7707090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.432+0000 7fd83a5a7740 1 -- 172.21.15.172:0/439765069 >> 172.21.15.172:0/439765069 conn(0x5563a76d0310 msgr2=0x5563a7703c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:53.694 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.432+0000 7fd83a5a7740 1 -- 172.21.15.172:0/439765069 shutdown_connections 2024-09-10T09:01:53.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.432+0000 7fd83a5a7740 1 -- 172.21.15.172:0/439765069 wait complete. 2024-09-10T09:01:53.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.432+0000 7fd83a5a7740 1 -- start start 2024-09-10T09:01:53.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.432+0000 7fd83a5a7740 1 Processor -- start 2024-09-10T09:01:53.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.433+0000 7fd83a5a7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a77075d0 0x5563a7716190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.433+0000 7fd83a5a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7708b40 0x5563a7713440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.433+0000 7fd83a5a7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7716780 0x5563a7713a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.433+0000 7fd83a5a7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5563a7719420 con 0x5563a7716780 2024-09-10T09:01:53.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.433+0000 7fd83a5a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5563a7628060 con 0x5563a7708b40 2024-09-10T09:01:53.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.433+0000 7fd83a5a7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5563a77195a0 con 0x5563a77075d0 2024-09-10T09:01:53.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.433+0000 7fd83a5a7740 10 client.0 _ll_register_callbacks cb 0x5563a737b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:01:53.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.433+0000 7fd83676b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7716780 0x5563a7713a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.695 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.433+0000 7fd83676b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7716780 0x5563a7713a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60854/0 (socket says 172.21.15.172:60854) 2024-09-10T09:01:53.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.433+0000 7fd83676b640 1 -- 172.21.15.172:0/2449538770 learned_addr learned my addr 172.21.15.172:0/2449538770 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:53.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.433+0000 7fd835769640 1 --2- 172.21.15.172:0/2449538770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7708b40 0x5563a7713440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.433+0000 7fd835f6a640 1 --2- 172.21.15.172:0/2449538770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a77075d0 0x5563a7716190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.433+0000 7fd83676b640 1 -- 172.21.15.172:0/2449538770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a77075d0 msgr2=0x5563a7716190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:53.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.433+0000 7fd83676b640 1 --2- 172.21.15.172:0/2449538770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a77075d0 0x5563a7716190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.433+0000 7fd83676b640 1 -- 172.21.15.172:0/2449538770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7708b40 msgr2=0x5563a7713440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:53.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.433+0000 7fd83676b640 1 --2- 172.21.15.172:0/2449538770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7708b40 0x5563a7713440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.433+0000 7fd83676b640 1 -- 172.21.15.172:0/2449538770 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563a73de3c0 con 0x5563a7716780 2024-09-10T09:01:53.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.434+0000 7fd83676b640 1 --2- 172.21.15.172:0/2449538770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7716780 0x5563a7713a50 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fd820009890 tx=0x7fd82002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d17baf02d869185 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:53.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.434+0000 7fd8157fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:01:53.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.434+0000 7fd8157fa640 1 -- 172.21.15.172:0/2449538770 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8200305c0 con 0x5563a7716780 2024-09-10T09:01:53.696 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.434+0000 7fd83a5a7740 1 -- 172.21.15.172:0/2449538770 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5563a767a860 con 0x5563a7716780 2024-09-10T09:01:53.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.434+0000 7fd8157fa640 1 -- 172.21.15.172:0/2449538770 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd820030760 con 0x5563a7716780 2024-09-10T09:01:53.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.434+0000 7fd8157fa640 1 -- 172.21.15.172:0/2449538770 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd820030960 con 0x5563a7716780 2024-09-10T09:01:53.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.434+0000 7fd83a5a7740 10 client.19596 fetch_fsmap learned FSMap version 44 2024-09-10T09:01:53.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.434+0000 7fd83a5a7740 1 -- 172.21.15.172:0/2449538770 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5563a7724e60 con 0x5563a7716780 2024-09-10T09:01:53.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.434+0000 7fd8157fa640 1 -- 172.21.15.172:0/2449538770 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5563a767a860 con 0x5563a7716780 2024-09-10T09:01:53.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.434+0000 7fd8157fa640 1 -- 172.21.15.172:0/2449538770 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fd820043030 con 0x5563a7716780 2024-09-10T09:01:53.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.434+0000 7fd83a5a7740 10 client.19596 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:01:53.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.434+0000 7fd83a5a7740 10 client.19596 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:01:53.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.434+0000 7fd83a5a7740 20 client.19596 populate_metadata read hostname 'smithi172' 2024-09-10T09:01:53.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.434+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/2449538770 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5563a77262c0 0x5563a7728720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.434+0000 7fd83a5a7740 4 client.19596 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:01:53.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.434+0000 7fd83a5a7740 1 -- 172.21.15.172:0/2449538770 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563a7674d70 con 0x5563a77262c0 2024-09-10T09:01:53.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.435+0000 7fd835f6a640 1 --2- 172.21.15.172:0/2449538770 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5563a77262c0 0x5563a7728720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.435+0000 7fd8157fa640 10 client.19596 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T09:01:53.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.435+0000 7fd835f6a640 1 --2- 172.21.15.172:0/2449538770 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5563a77262c0 0x5563a7728720 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9328813b43d93031 server_cookie=5df9dc1185d7f88d in_seq=0 out_seq=0 2024-09-10T09:01:53.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.436+0000 7fd8157fa640 1 -- 172.21.15.172:0/2449538770 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563a7674d70 con 0x5563a77262c0 2024-09-10T09:01:53.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.436+0000 7fd8157fa640 10 client.19596 handle_command_reply: tid=0 2024-09-10T09:01:53.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.436+0000 7fd83a5a7740 1 client.19596 shutdown 2024-09-10T09:01:53.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.436+0000 7fd83a5a7740 1 -- 172.21.15.172:0/2449538770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7716780 msgr2=0x5563a7713a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:53.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.436+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/2449538770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7716780 0x5563a7713a50 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fd820009890 tx=0x7fd82002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.436+0000 7fd83a5a7740 1 -- 172.21.15.172:0/2449538770 shutdown_connections 2024-09-10T09:01:53.698 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.436+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/2449538770 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5563a77262c0 0x5563a7728720 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.436+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/2449538770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7716780 0x5563a7713a50 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.436+0000 7fd8157fa640 0 client.19596 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T09:01:53.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.436+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/2449538770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7708b40 0x5563a7713440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.436+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/2449538770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a77075d0 0x5563a7716190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.436+0000 7fd83a5a7740 1 -- 172.21.15.172:0/2449538770 >> 172.21.15.172:0/2449538770 conn(0x5563a76d0310 msgr2=0x5563a7709990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:53.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.436+0000 7fd83a5a7740 1 -- 172.21.15.172:0/2449538770 shutdown_connections 2024-09-10T09:01:53.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.436+0000 7fd83a5a7740 1 -- 172.21.15.172:0/2449538770 wait complete. 2024-09-10T09:01:53.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.436+0000 7fd83a5a7740 20 client.19596 trim_cache size 0 max 16384 2024-09-10T09:01:53.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.449+0000 7fd83a5a7740 1 Processor -- start 2024-09-10T09:01:53.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.449+0000 7fd83a5a7740 1 -- start start 2024-09-10T09:01:53.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.449+0000 7fd83a5a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a77075d0 0x5563a7703e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.699 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.449+0000 7fd83a5a7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a7708b40 0x5563a77226c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.449+0000 7fd83a5a7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7716780 0x5563a772dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.449+0000 7fd83a5a7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301ac9e0 con 0x5563a7716780 2024-09-10T09:01:53.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.449+0000 7fd83a5a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301ae520 con 0x5563a77075d0 2024-09-10T09:01:53.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.449+0000 7fd83a5a7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301ad0b0 con 0x5563a7708b40 2024-09-10T09:01:53.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.450+0000 7fd83676b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7716780 0x5563a772dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.450+0000 7fd83676b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7716780 0x5563a772dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60868/0 (socket says 172.21.15.172:60868) 2024-09-10T09:01:53.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.450+0000 7fd83676b640 1 -- 172.21.15.172:0/2675417030 learned_addr learned my addr 172.21.15.172:0/2675417030 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:53.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.450+0000 7fd835769640 1 --2- 172.21.15.172:0/2675417030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a7708b40 0x5563a77226c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.700 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.450+0000 7fd835f6a640 1 --2- 172.21.15.172:0/2675417030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a77075d0 0x5563a7703e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.450+0000 7fd83676b640 1 -- 172.21.15.172:0/2675417030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a7708b40 msgr2=0x5563a77226c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:53.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.450+0000 7fd83676b640 1 --2- 172.21.15.172:0/2675417030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a7708b40 0x5563a77226c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.450+0000 7fd83676b640 1 -- 172.21.15.172:0/2675417030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a77075d0 msgr2=0x5563a7703e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:53.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.450+0000 7fd83676b640 1 --2- 172.21.15.172:0/2675417030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a77075d0 0x5563a7703e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.450+0000 7fd83676b640 1 -- 172.21.15.172:0/2675417030 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563a7728da0 con 0x5563a7716780 2024-09-10T09:01:53.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.450+0000 7fd835f6a640 1 --2- 172.21.15.172:0/2675417030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a77075d0 0x5563a7703e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:01:53.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.450+0000 7fd83676b640 1 --2- 172.21.15.172:0/2675417030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7716780 0x5563a772dff0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fd820009bf0 tx=0x7fd820003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4e96eb63f2d666 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:53.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.450+0000 7fd8177fe640 1 -- 172.21.15.172:0/2675417030 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820061070 con 0x5563a7716780 2024-09-10T09:01:53.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.450+0000 7fd8177fe640 1 -- 172.21.15.172:0/2675417030 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd82004b070 con 0x5563a7716780 2024-09-10T09:01:53.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.450+0000 7fd8177fe640 1 -- 172.21.15.172:0/2675417030 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82003c410 con 0x5563a7716780 2024-09-10T09:01:53.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.451+0000 7fd83a5a7740 1 -- 172.21.15.172:0/2675417030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7716780 msgr2=0x5563a772dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:53.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.451+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/2675417030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7716780 0x5563a772dff0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fd820009bf0 tx=0x7fd820003ec0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.451+0000 7fd83a5a7740 1 -- 172.21.15.172:0/2675417030 shutdown_connections 2024-09-10T09:01:53.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.451+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/2675417030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7716780 0x5563a772dff0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.451+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/2675417030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a7708b40 0x5563a77226c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.451+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/2675417030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a77075d0 0x5563a7703e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.451+0000 7fd83a5a7740 1 -- 172.21.15.172:0/2675417030 >> 172.21.15.172:0/2675417030 conn(0x5563a76d0310 msgr2=0x5563a7703700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:53.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.451+0000 7fd83a5a7740 1 -- 172.21.15.172:0/2675417030 shutdown_connections 2024-09-10T09:01:53.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.451+0000 7fd83a5a7740 1 -- 172.21.15.172:0/2675417030 wait complete. 2024-09-10T09:01:53.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.451+0000 7fd83a5a7740 1 -- start start 2024-09-10T09:01:53.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.451+0000 7fd83a5a7740 1 Processor -- start 2024-09-10T09:01:53.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd83a5a7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a77075d0 0x5563a771a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd83a5a7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7708b40 0x5563a771add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd83a5a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7716780 0x5563a771b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd83a5a7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301ade50 con 0x5563a7708b40 2024-09-10T09:01:53.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd83a5a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301ad780 con 0x5563a7716780 2024-09-10T09:01:53.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd83a5a7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd8301aa9c0 con 0x5563a77075d0 2024-09-10T09:01:53.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd83a5a7740 10 client.0 _ll_register_callbacks cb 0x7fd820009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:01:53.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd835769640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7708b40 0x5563a771add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd835769640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7708b40 0x5563a771add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60872/0 (socket says 172.21.15.172:60872) 2024-09-10T09:01:53.703 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd835769640 1 -- 172.21.15.172:0/913962484 learned_addr learned my addr 172.21.15.172:0/913962484 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:01:53.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd835f6a640 1 --2- 172.21.15.172:0/913962484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a77075d0 0x5563a771a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd83676b640 1 --2- 172.21.15.172:0/913962484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7716780 0x5563a771b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd835769640 1 -- 172.21.15.172:0/913962484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a77075d0 msgr2=0x5563a771a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:53.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd835769640 1 --2- 172.21.15.172:0/913962484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a77075d0 0x5563a771a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd835769640 1 -- 172.21.15.172:0/913962484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7716780 msgr2=0x5563a771b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:01:53.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd835769640 1 --2- 172.21.15.172:0/913962484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7716780 0x5563a771b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd835769640 1 -- 172.21.15.172:0/913962484 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563a77902b0 con 0x5563a7708b40 2024-09-10T09:01:53.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd835769640 1 --2- 172.21.15.172:0/913962484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7708b40 0x5563a771add0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x5563a7708070 tx=0x7fd824015a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685272c6d8927421 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:01:53.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd8157fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:01:53.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd8157fa640 1 -- 172.21.15.172:0/913962484 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8240165b0 con 0x5563a7708b40 2024-09-10T09:01:53.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd8157fa640 1 -- 172.21.15.172:0/913962484 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd824016750 con 0x5563a7708b40 2024-09-10T09:01:53.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.452+0000 7fd83a5a7740 1 -- 172.21.15.172:0/913962484 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd82004b070 con 0x5563a7708b40 2024-09-10T09:01:53.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.453+0000 7fd8157fa640 1 -- 172.21.15.172:0/913962484 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd824016950 con 0x5563a7708b40 2024-09-10T09:01:53.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.453+0000 7fd83a5a7740 10 client.19602 fetch_fsmap learned FSMap version 44 2024-09-10T09:01:53.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.453+0000 7fd83a5a7740 1 -- 172.21.15.172:0/913962484 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5563a7791610 con 0x5563a7708b40 2024-09-10T09:01:53.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.453+0000 7fd8157fa640 1 -- 172.21.15.172:0/913962484 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd82004b070 con 0x5563a7708b40 2024-09-10T09:01:53.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.453+0000 7fd8157fa640 1 -- 172.21.15.172:0/913962484 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fd82401fc60 con 0x5563a7708b40 2024-09-10T09:01:53.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.453+0000 7fd83a5a7740 10 client.19602 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:01:53.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.453+0000 7fd83a5a7740 10 client.19602 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:01:53.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.453+0000 7fd83a5a7740 20 client.19602 populate_metadata read hostname 'smithi172' 2024-09-10T09:01:53.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.453+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/913962484 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5563a7791e40 0x5563a7794260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:01:53.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.453+0000 7fd83a5a7740 4 client.19602 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:01:53.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.453+0000 7fd83a5a7740 1 -- 172.21.15.172:0/913962484 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563a75f1170 con 0x5563a7791e40 2024-09-10T09:01:53.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.453+0000 7fd835f6a640 1 --2- 172.21.15.172:0/913962484 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5563a7791e40 0x5563a7794260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:01:53.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.454+0000 7fd8157fa640 10 client.19602 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T09:01:53.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.454+0000 7fd835f6a640 1 --2- 172.21.15.172:0/913962484 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5563a7791e40 0x5563a7794260 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9fcfc46783c6ff8 server_cookie=719b65b2dbf7bb85 in_seq=0 out_seq=0 2024-09-10T09:01:53.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.454+0000 7fd8157fa640 1 -- 172.21.15.172:0/913962484 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563a75f1170 con 0x5563a7791e40 2024-09-10T09:01:53.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.454+0000 7fd8157fa640 10 client.19602 handle_command_reply: tid=0 2024-09-10T09:01:53.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.454+0000 7fd83a5a7740 1 client.19602 shutdown 2024-09-10T09:01:53.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.454+0000 7fd83a5a7740 1 -- 172.21.15.172:0/913962484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7708b40 msgr2=0x5563a771add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:53.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.454+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/913962484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7708b40 0x5563a771add0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x5563a7708070 tx=0x7fd824015a10 comp rx=0 tx=0).stop 2024-09-10T09:01:53.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.454+0000 7fd83a5a7740 1 -- 172.21.15.172:0/913962484 shutdown_connections 2024-09-10T09:01:53.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.454+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/913962484 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5563a7791e40 0x5563a7794260 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.454+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/913962484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a7716780 0x5563a771b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.454+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/913962484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563a7708b40 0x5563a771add0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.454+0000 7fd8157fa640 0 client.19602 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T09:01:53.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.454+0000 7fd83a5a7740 1 --2- 172.21.15.172:0/913962484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563a77075d0 0x5563a771a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.454+0000 7fd83a5a7740 1 -- 172.21.15.172:0/913962484 >> 172.21.15.172:0/913962484 conn(0x5563a76d0310 msgr2=0x5563a7714ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:53.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.454+0000 7fd83a5a7740 1 -- 172.21.15.172:0/913962484 shutdown_connections 2024-09-10T09:01:53.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.454+0000 7fd83a5a7740 1 -- 172.21.15.172:0/913962484 wait complete. 2024-09-10T09:01:53.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.454+0000 7fd83a5a7740 20 client.19602 trim_cache size 0 max 16384 2024-09-10T09:01:53.707 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:01:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.455+0000 7fd8381ce640 1 -- 172.21.15.172:0/4078012743 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd8180a2c00 msgr2=0x7fd8180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.455+0000 7fd8381ce640 1 --2- 172.21.15.172:0/4078012743 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd8180a2c00 0x7fd8180a50c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd820006010 tx=0x7fd8200095c0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.455+0000 7fd8381ce640 1 -- 172.21.15.172:0/4078012743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd830101e10 msgr2=0x7fd8301983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:01:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.455+0000 7fd8381ce640 1 --2- 172.21.15.172:0/4078012743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd830101e10 0x7fd8301983a0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fd82800c890 tx=0x7fd82800cd50 comp rx=0 tx=0).stop 2024-09-10T09:01:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.455+0000 7fd8381ce640 1 -- 172.21.15.172:0/4078012743 shutdown_connections 2024-09-10T09:01:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.455+0000 7fd8381ce640 1 --2- 172.21.15.172:0/4078012743 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd8180a2c00 0x7fd8180a50c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.455+0000 7fd8381ce640 1 --2- 172.21.15.172:0/4078012743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd830106f40 0x7fd83019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.455+0000 7fd8381ce640 1 --2- 172.21.15.172:0/4078012743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8301027c0 0x7fd8301988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.455+0000 7fd8381ce640 1 --2- 172.21.15.172:0/4078012743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd830101e10 0x7fd8301983a0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:01:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.455+0000 7fd8381ce640 1 -- 172.21.15.172:0/4078012743 >> 172.21.15.172:0/4078012743 conn(0x7fd8300fd8d0 msgr2=0x7fd8300febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:01:53.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.455+0000 7fd8381ce640 1 -- 172.21.15.172:0/4078012743 shutdown_connections 2024-09-10T09:01:53.709 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:01:52.455+0000 7fd8381ce640 1 -- 172.21.15.172:0/4078012743 wait complete. 2024-09-10T09:01:55.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:55 smithi174 ceph-mon[34068]: pgmap v820: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.1 MiB/s wr, 796 op/s 2024-09-10T09:01:55.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:55 smithi191 ceph-mon[35314]: pgmap v820: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.1 MiB/s wr, 796 op/s 2024-09-10T09:01:55.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:55 smithi172 ceph-mon[30854]: pgmap v820: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.1 MiB/s wr, 796 op/s 2024-09-10T09:01:57.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:57 smithi174 ceph-mon[34068]: pgmap v821: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.7 MiB/s wr, 441 op/s 2024-09-10T09:01:57.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:57 smithi191 ceph-mon[35314]: pgmap v821: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.7 MiB/s wr, 441 op/s 2024-09-10T09:01:57.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:57 smithi172 ceph-mon[30854]: pgmap v821: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.7 MiB/s wr, 441 op/s 2024-09-10T09:01:59.278 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3e265114', quiesce took 70.5 seconds. Will release after: -49.0 2024-09-10T09:01:59.278 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:01:59.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:01:59 smithi174 ceph-mon[34068]: pgmap v822: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 221 KiB/s wr, 65 op/s 2024-09-10T09:01:59.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:01:59 smithi191 ceph-mon[35314]: pgmap v822: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 221 KiB/s wr, 65 op/s 2024-09-10T09:01:59.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:01:59 smithi172 ceph-mon[30854]: pgmap v822: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 221 KiB/s wr, 65 op/s 2024-09-10T09:02:01.294 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:02:01.428 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:02:01.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:01 smithi174 ceph-mon[34068]: pgmap v823: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 104 KiB/s wr, 37 op/s 2024-09-10T09:02:01.549 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:01 smithi172 ceph-mon[30854]: pgmap v823: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 104 KiB/s wr, 37 op/s 2024-09-10T09:02:01.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:01 smithi191 ceph-mon[35314]: pgmap v823: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 104 KiB/s wr, 37 op/s 2024-09-10T09:02:01.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.949+0000 7feab05ea640 1 -- 172.21.15.172:0/3201017476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feaa8100e20 msgr2=0x7feaa8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:01.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.949+0000 7feab05ea640 1 --2- 172.21.15.172:0/3201017476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feaa8100e20 0x7feaa8101200 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009900 tx=0x7feaa402a480 comp rx=0 tx=0).stop 2024-09-10T09:02:01.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.949+0000 7feab05ea640 1 -- 172.21.15.172:0/3201017476 shutdown_connections 2024-09-10T09:02:01.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.949+0000 7feab05ea640 1 --2- 172.21.15.172:0/3201017476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaa8102ef0 0x7feaa810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:01.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.949+0000 7feab05ea640 1 --2- 172.21.15.172:0/3201017476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feaa8102570 0x7feaa81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:01.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.949+0000 7feab05ea640 1 --2- 172.21.15.172:0/3201017476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feaa8100e20 0x7feaa8101200 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:01.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.949+0000 7feab05ea640 1 -- 172.21.15.172:0/3201017476 >> 172.21.15.172:0/3201017476 conn(0x7feaa80fc7d0 msgr2=0x7feaa80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:01.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.949+0000 7feab05ea640 1 -- 172.21.15.172:0/3201017476 shutdown_connections 2024-09-10T09:02:01.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.949+0000 7feab05ea640 1 -- 172.21.15.172:0/3201017476 wait complete. 2024-09-10T09:02:01.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.949+0000 7feab05ea640 1 Processor -- start 2024-09-10T09:02:01.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feab05ea640 1 -- start start 2024-09-10T09:02:01.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feab05ea640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feaa8100e20 0x7feaa80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:01.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feab05ea640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feaa8102570 0x7feaa80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:01.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feab05ea640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaa8102ef0 0x7feaa80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:01.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feab05ea640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feaa8110e00 con 0x7feaa8100e20 2024-09-10T09:02:01.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feab05ea640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feaa8110c80 con 0x7feaa8102570 2024-09-10T09:02:01.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feab05ea640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7feaa8110f80 con 0x7feaa8102ef0 2024-09-10T09:02:01.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feaae386640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feaa8100e20 0x7feaa80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:01.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feaae386640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feaa8100e20 0x7feaa80fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58600/0 (socket says 172.21.15.172:58600) 2024-09-10T09:02:01.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feaae386640 1 -- 172.21.15.172:0/510345582 learned_addr learned my addr 172.21.15.172:0/510345582 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:01.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feaadb85640 1 --2- 172.21.15.172:0/510345582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feaa8102570 0x7feaa80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:01.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feaaeb87640 1 --2- 172.21.15.172:0/510345582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaa8102ef0 0x7feaa80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:01.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feaae386640 1 -- 172.21.15.172:0/510345582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaa8102ef0 msgr2=0x7feaa80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:01.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feaae386640 1 --2- 172.21.15.172:0/510345582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaa8102ef0 0x7feaa80fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:01.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feaae386640 1 -- 172.21.15.172:0/510345582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feaa8102570 msgr2=0x7feaa80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:01.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feaae386640 1 --2- 172.21.15.172:0/510345582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feaa8102570 0x7feaa80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:01.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feaae386640 1 -- 172.21.15.172:0/510345582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa81a6920 con 0x7feaa8100e20 2024-09-10T09:02:01.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.950+0000 7feaae386640 1 --2- 172.21.15.172:0/510345582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feaa8100e20 0x7feaa80fcc80 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7feaa4006010 tx=0x7feaa40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:01.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.951+0000 7fea9f7fe640 1 -- 172.21.15.172:0/510345582 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa403f030 con 0x7feaa8100e20 2024-09-10T09:02:01.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.951+0000 7feab05ea640 1 -- 172.21.15.172:0/510345582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa81a6bb0 con 0x7feaa8100e20 2024-09-10T09:02:01.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.951+0000 7fea9f7fe640 1 -- 172.21.15.172:0/510345582 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaa4003cb0 con 0x7feaa8100e20 2024-09-10T09:02:01.957 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.951+0000 7fea9f7fe640 1 -- 172.21.15.172:0/510345582 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa4035070 con 0x7feaa8100e20 2024-09-10T09:02:01.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.951+0000 7feab05ea640 1 -- 172.21.15.172:0/510345582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa81a7090 con 0x7feaa8100e20 2024-09-10T09:02:01.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.952+0000 7feab05ea640 1 -- 172.21.15.172:0/510345582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa8101200 con 0x7feaa8100e20 2024-09-10T09:02:01.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.953+0000 7fea9f7fe640 1 -- 172.21.15.172:0/510345582 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7feaa4042030 con 0x7feaa8100e20 2024-09-10T09:02:01.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.954+0000 7fea9f7fe640 1 --2- 172.21.15.172:0/510345582 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea940a2b80 0x7fea940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:01.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.954+0000 7fea9f7fe640 1 -- 172.21.15.172:0/510345582 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7feaa40ef7c0 con 0x7feaa8100e20 2024-09-10T09:02:01.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.954+0000 7feaadb85640 1 --2- 172.21.15.172:0/510345582 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea940a2b80 0x7fea940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:01.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.955+0000 7feaadb85640 1 --2- 172.21.15.172:0/510345582 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea940a2b80 0x7fea940a5040 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7feaa80fe710 tx=0x7fea9800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:01.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:01.959+0000 7fea9f7fe640 1 -- 172.21.15.172:0/510345582 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feaa40bbd70 con 0x7feaa8100e20 2024-09-10T09:02:02.005 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:02:02.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:02.187+0000 7feab05ea640 1 -- 172.21.15.172:0/510345582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feaa810a0a0 con 0x7feaa8100e20 2024-09-10T09:02:02.190 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:02.188+0000 7fea9f7fe640 1 -- 172.21.15.172:0/510345582 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+12001 (secure 0 0 0) 0x7feaa40c0070 con 0x7feaa8100e20 2024-09-10T09:02:02.190 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T09:02:02.190 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":44,"btime":"2024-09-10T09:01:18:531063+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T09:01:17.697372+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T09:02:02.191 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 44 2024-09-10T09:02:02.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:02.190+0000 7feab05ea640 1 -- 172.21.15.172:0/510345582 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea940a2b80 msgr2=0x7fea940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:02.192 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:02.190+0000 7feab05ea640 1 --2- 172.21.15.172:0/510345582 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea940a2b80 0x7fea940a5040 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7feaa80fe710 tx=0x7fea9800b390 comp rx=0 tx=0).stop 2024-09-10T09:02:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:02.190+0000 7feab05ea640 1 -- 172.21.15.172:0/510345582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feaa8100e20 msgr2=0x7feaa80fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:02.190+0000 7feab05ea640 1 --2- 172.21.15.172:0/510345582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feaa8100e20 0x7feaa80fcc80 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7feaa4006010 tx=0x7feaa40042f0 comp rx=0 tx=0).stop 2024-09-10T09:02:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:02.190+0000 7feab05ea640 1 -- 172.21.15.172:0/510345582 shutdown_connections 2024-09-10T09:02:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:02.190+0000 7feab05ea640 1 --2- 172.21.15.172:0/510345582 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fea940a2b80 0x7fea940a5040 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:02.190+0000 7feab05ea640 1 --2- 172.21.15.172:0/510345582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feaa8102ef0 0x7feaa80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:02.190+0000 7feab05ea640 1 --2- 172.21.15.172:0/510345582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feaa8102570 0x7feaa80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:02.190+0000 7feab05ea640 1 --2- 172.21.15.172:0/510345582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feaa8100e20 0x7feaa80fcc80 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:02.190+0000 7feab05ea640 1 -- 172.21.15.172:0/510345582 >> 172.21.15.172:0/510345582 conn(0x7feaa80fc7d0 msgr2=0x7feaa806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:02.190+0000 7feab05ea640 1 -- 172.21.15.172:0/510345582 shutdown_connections 2024-09-10T09:02:02.193 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:02.190+0000 7feab05ea640 1 -- 172.21.15.172:0/510345582 wait complete. 2024-09-10T09:02:02.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:02 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/510345582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:02:02.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:02 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/510345582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:02:02.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:02 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/510345582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:02:03.052 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 3e265114 --release --await' 2024-09-10T09:02:03.052 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 3e265114 --release --await 2024-09-10T09:02:03.309 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:03 smithi172 ceph-mon[30854]: pgmap v824: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 91 KiB/s wr, 34 op/s 2024-09-10T09:02:03.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.306+0000 7f1c1e00d640 1 -- 172.21.15.172:0/3234451692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c18102d30 msgr2=0x7f1c18103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:03.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.306+0000 7f1c1e00d640 1 --2- 172.21.15.172:0/3234451692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c18102d30 0x7f1c18103110 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009930 tx=0x7f1c0802a440 comp rx=0 tx=0).stop 2024-09-10T09:02:03.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.307+0000 7f1c1e00d640 1 -- 172.21.15.172:0/3234451692 shutdown_connections 2024-09-10T09:02:03.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.307+0000 7f1c1e00d640 1 --2- 172.21.15.172:0/3234451692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c181082a0 0x7f1c1810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.307+0000 7f1c1e00d640 1 --2- 172.21.15.172:0/3234451692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c181036e0 0x7f1c18107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.307+0000 7f1c1e00d640 1 --2- 172.21.15.172:0/3234451692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c18102d30 0x7f1c18103110 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.338 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.307+0000 7f1c1e00d640 1 -- 172.21.15.172:0/3234451692 >> 172.21.15.172:0/3234451692 conn(0x7f1c180fc7d0 msgr2=0x7f1c180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:03.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.307+0000 7f1c1e00d640 1 -- 172.21.15.172:0/3234451692 shutdown_connections 2024-09-10T09:02:03.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.307+0000 7f1c1e00d640 1 -- 172.21.15.172:0/3234451692 wait complete. 2024-09-10T09:02:03.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.307+0000 7f1c1e00d640 1 Processor -- start 2024-09-10T09:02:03.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.307+0000 7f1c1e00d640 1 -- start start 2024-09-10T09:02:03.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c1e00d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c18102d30 0x7f1c18198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c1e00d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c181036e0 0x7f1c18198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c1e00d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c181082a0 0x7f1c1819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c1e00d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18111fb0 con 0x7f1c181082a0 2024-09-10T09:02:03.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c1e00d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18111e30 con 0x7f1c18102d30 2024-09-10T09:02:03.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c1e00d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18112130 con 0x7f1c181036e0 2024-09-10T09:02:03.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c17fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c181082a0 0x7f1c1819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:03.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c17fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c181082a0 0x7f1c1819ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58626/0 (socket says 172.21.15.172:58626) 2024-09-10T09:02:03.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c17fff640 1 -- 172.21.15.172:0/3416892189 learned_addr learned my addr 172.21.15.172:0/3416892189 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:03.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c16ffd640 1 --2- 172.21.15.172:0/3416892189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c181036e0 0x7f1c18198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:03.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c177fe640 1 --2- 172.21.15.172:0/3416892189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c18102d30 0x7f1c18198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:03.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c17fff640 1 -- 172.21.15.172:0/3416892189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c181036e0 msgr2=0x7f1c18198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:03.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c17fff640 1 --2- 172.21.15.172:0/3416892189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c181036e0 0x7f1c18198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c17fff640 1 -- 172.21.15.172:0/3416892189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c18102d30 msgr2=0x7f1c18198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:03.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c17fff640 1 --2- 172.21.15.172:0/3416892189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c18102d30 0x7f1c18198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c17fff640 1 -- 172.21.15.172:0/3416892189 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1819d540 con 0x7f1c181082a0 2024-09-10T09:02:03.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c177fe640 1 --2- 172.21.15.172:0/3416892189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c18102d30 0x7f1c18198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:02:03.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c17fff640 1 --2- 172.21.15.172:0/3416892189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c181082a0 0x7f1c1819ce60 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000cad0 tx=0x7f1c1000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:03.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.308+0000 7f1c14ff9640 1 -- 172.21.15.172:0/3416892189 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1000ad60 con 0x7f1c181082a0 2024-09-10T09:02:03.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.309+0000 7f1c1e00d640 1 -- 172.21.15.172:0/3416892189 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1819d830 con 0x7f1c181082a0 2024-09-10T09:02:03.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.309+0000 7f1c14ff9640 1 -- 172.21.15.172:0/3416892189 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c10005020 con 0x7f1c181082a0 2024-09-10T09:02:03.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.309+0000 7f1c14ff9640 1 -- 172.21.15.172:0/3416892189 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1000f6b0 con 0x7f1c181082a0 2024-09-10T09:02:03.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.309+0000 7f1c1e00d640 1 -- 172.21.15.172:0/3416892189 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c181a5110 con 0x7f1c181082a0 2024-09-10T09:02:03.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.311+0000 7f1c14ff9640 1 -- 172.21.15.172:0/3416892189 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1c1000f850 con 0x7f1c181082a0 2024-09-10T09:02:03.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.311+0000 7f1c203e6740 1 Processor -- start 2024-09-10T09:02:03.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.311+0000 7f1c203e6740 1 -- start start 2024-09-10T09:02:03.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.311+0000 7f1c203e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 0x55b57d8ba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.311+0000 7f1c203e6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8baa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.311+0000 7f1c203e6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8c5370 0x55b57d8c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.311+0000 7f1c203e6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b57d8202f0 con 0x55b57d8ba5d0 2024-09-10T09:02:03.341 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.311+0000 7f1c203e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b57d7c7cf0 con 0x55b57d8bbb40 2024-09-10T09:02:03.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.311+0000 7f1c203e6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b57d81c2b0 con 0x55b57d8c5370 2024-09-10T09:02:03.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1c177fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8baa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:03.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1c177fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8baa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58636/0 (socket says 172.21.15.172:58636) 2024-09-10T09:02:03.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1c177fe640 1 -- 172.21.15.172:0/2534334020 learned_addr learned my addr 172.21.15.172:0/2534334020 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:03.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1c16ffd640 1 --2- 172.21.15.172:0/2534334020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 0x55b57d8ba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:03.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1c14ff9640 1 --2- 172.21.15.172:0/3416892189 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1c000a2b80 0x7f1c000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1c16ffd640 1 --2- 172.21.15.172:0/2534334020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8c5370 0x55b57d8c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:03.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1c177fe640 1 -- 172.21.15.172:0/2534334020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8c5370 msgr2=0x55b57d8c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:03.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1c177fe640 1 --2- 172.21.15.172:0/2534334020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8c5370 0x55b57d8c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1c177fe640 1 -- 172.21.15.172:0/2534334020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 msgr2=0x55b57d8ba090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:03.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1c177fe640 1 --2- 172.21.15.172:0/2534334020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 0x55b57d8ba090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1c177fe640 1 -- 172.21.15.172:0/2534334020 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b57d7ac3a0 con 0x55b57d8ba5d0 2024-09-10T09:02:03.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1c17fff640 1 --2- 172.21.15.172:0/3416892189 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1c000a2b80 0x7f1c000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:03.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1c14ff9640 1 -- 172.21.15.172:0/3416892189 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1c100cf3d0 con 0x7f1c181082a0 2024-09-10T09:02:03.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1c177fe640 1 --2- 172.21.15.172:0/2534334020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8baa10 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f1c0802a960 tx=0x7f1c08031e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14dc721d561e6c14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:03.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1bfdffb640 1 -- 172.21.15.172:0/2534334020 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0803f070 con 0x55b57d8ba5d0 2024-09-10T09:02:03.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1bfdffb640 1 -- 172.21.15.172:0/2534334020 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c080316a0 con 0x55b57d8ba5d0 2024-09-10T09:02:03.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1bfdffb640 1 -- 172.21.15.172:0/2534334020 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08035070 con 0x55b57d8ba5d0 2024-09-10T09:02:03.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.312+0000 7f1c17fff640 1 --2- 172.21.15.172:0/3416892189 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1c000a2b80 0x7f1c000a5040 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1c1009b860 tx=0x7f1c10086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:03.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.313+0000 7f1c203e6740 1 -- 172.21.15.172:0/2534334020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 msgr2=0x55b57d8baa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:03.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.313+0000 7f1c203e6740 1 --2- 172.21.15.172:0/2534334020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8baa10 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f1c0802a960 tx=0x7f1c08031e40 comp rx=0 tx=0).stop 2024-09-10T09:02:03.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.313+0000 7f1c203e6740 1 -- 172.21.15.172:0/2534334020 shutdown_connections 2024-09-10T09:02:03.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.313+0000 7f1c203e6740 1 --2- 172.21.15.172:0/2534334020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8c5370 0x55b57d8c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.313+0000 7f1c203e6740 1 --2- 172.21.15.172:0/2534334020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8baa10 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.313+0000 7f1c203e6740 1 --2- 172.21.15.172:0/2534334020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 0x55b57d8ba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.313+0000 7f1c203e6740 1 -- 172.21.15.172:0/2534334020 >> 172.21.15.172:0/2534334020 conn(0x55b57d883310 msgr2=0x55b57d8b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:03.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.313+0000 7f1c203e6740 1 -- 172.21.15.172:0/2534334020 shutdown_connections 2024-09-10T09:02:03.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.313+0000 7f1c203e6740 1 -- 172.21.15.172:0/2534334020 wait complete. 2024-09-10T09:02:03.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.313+0000 7f1c203e6740 1 -- start start 2024-09-10T09:02:03.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c203e6740 1 Processor -- start 2024-09-10T09:02:03.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c203e6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8ba5d0 0x55b57d8c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c203e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 0x55b57d8c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c203e6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8c5370 0x55b57d8c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c203e6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b57d8c8010 con 0x55b57d8c5370 2024-09-10T09:02:03.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c203e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b57d7db060 con 0x55b57d8bbb40 2024-09-10T09:02:03.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c203e6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b57d8c8190 con 0x55b57d8ba5d0 2024-09-10T09:02:03.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c203e6740 10 client.0 _ll_register_callbacks cb 0x55b57d52e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:02:03.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c16ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8c5370 0x55b57d8c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:03.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c16ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8c5370 0x55b57d8c25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58642/0 (socket says 172.21.15.172:58642) 2024-09-10T09:02:03.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c16ffd640 1 -- 172.21.15.172:0/2477034930 learned_addr learned my addr 172.21.15.172:0/2477034930 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:03.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c177fe640 1 --2- 172.21.15.172:0/2477034930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 0x55b57d8c1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:03.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c16ffd640 1 --2- 172.21.15.172:0/2477034930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8ba5d0 0x55b57d8c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:03.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c16ffd640 1 -- 172.21.15.172:0/2477034930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8ba5d0 msgr2=0x55b57d8c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:03.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c16ffd640 1 --2- 172.21.15.172:0/2477034930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8ba5d0 0x55b57d8c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c16ffd640 1 -- 172.21.15.172:0/2477034930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 msgr2=0x55b57d8c1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:03.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c16ffd640 1 --2- 172.21.15.172:0/2477034930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 0x55b57d8c1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.314+0000 7f1c16ffd640 1 -- 172.21.15.172:0/2477034930 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b57d5913c0 con 0x55b57d8c5370 2024-09-10T09:02:03.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.315+0000 7f1c16ffd640 1 --2- 172.21.15.172:0/2477034930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8c5370 0x55b57d8c25e0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c018c40 tx=0x7f1c0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f3bcc2f20fb4e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:03.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.315+0000 7f1bf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:02:03.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.315+0000 7f1bf6ffd640 1 -- 172.21.15.172:0/2477034930 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c013c30 con 0x55b57d8c5370 2024-09-10T09:02:03.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.315+0000 7f1bf6ffd640 1 -- 172.21.15.172:0/2477034930 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1c0c013dd0 con 0x55b57d8c5370 2024-09-10T09:02:03.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.315+0000 7f1c203e6740 1 -- 172.21.15.172:0/2477034930 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c080316a0 con 0x55b57d8c5370 2024-09-10T09:02:03.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.315+0000 7f1bf6ffd640 1 -- 172.21.15.172:0/2477034930 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1c0c0059c0 con 0x55b57d8c5370 2024-09-10T09:02:03.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.315+0000 7f1c203e6740 10 client.19620 fetch_fsmap learned FSMap version 44 2024-09-10T09:02:03.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.315+0000 7f1c203e6740 1 -- 172.21.15.172:0/2477034930 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b57d8d3ae0 con 0x55b57d8c5370 2024-09-10T09:02:03.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.315+0000 7f1bf6ffd640 1 -- 172.21.15.172:0/2477034930 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c080316a0 con 0x55b57d8c5370 2024-09-10T09:02:03.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.315+0000 7f1bf6ffd640 1 -- 172.21.15.172:0/2477034930 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f1c0c022070 con 0x55b57d8c5370 2024-09-10T09:02:03.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.315+0000 7f1c203e6740 10 client.19620 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:02:03.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.315+0000 7f1c203e6740 10 client.19620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:02:03.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.315+0000 7f1c203e6740 20 client.19620 populate_metadata read hostname 'smithi172' 2024-09-10T09:02:03.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.315+0000 7f1c203e6740 1 --2- 172.21.15.172:0/2477034930 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b57d8d4ef0 0x55b57d8d7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.315+0000 7f1c203e6740 4 client.19620 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:02:03.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.315+0000 7f1c203e6740 1 -- 172.21.15.172:0/2477034930 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b57d82d860 con 0x55b57d8d4ef0 2024-09-10T09:02:03.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.316+0000 7f1c177fe640 1 --2- 172.21.15.172:0/2477034930 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b57d8d4ef0 0x55b57d8d7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:03.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.316+0000 7f1bf6ffd640 10 client.19620 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:03.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.316+0000 7f1c177fe640 1 --2- 172.21.15.172:0/2477034930 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b57d8d4ef0 0x55b57d8d7320 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eac0a62270c2b8f server_cookie=b65a96bc11548e95 in_seq=0 out_seq=0 2024-09-10T09:02:03.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.317+0000 7f1bf6ffd640 1 -- 172.21.15.172:0/2477034930 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b57d82d860 con 0x55b57d8d4ef0 2024-09-10T09:02:03.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.317+0000 7f1bf6ffd640 10 client.19620 handle_command_reply: tid=0 2024-09-10T09:02:03.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.317+0000 7f1c203e6740 1 client.19620 shutdown 2024-09-10T09:02:03.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.317+0000 7f1c203e6740 1 -- 172.21.15.172:0/2477034930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8c5370 msgr2=0x55b57d8c25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:03.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.317+0000 7f1c203e6740 1 --2- 172.21.15.172:0/2477034930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8c5370 0x55b57d8c25e0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c018c40 tx=0x7f1c0c0046c0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.317+0000 7f1c203e6740 1 -- 172.21.15.172:0/2477034930 shutdown_connections 2024-09-10T09:02:03.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.317+0000 7f1c203e6740 1 --2- 172.21.15.172:0/2477034930 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b57d8d4ef0 0x55b57d8d7320 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.317+0000 7f1c203e6740 1 --2- 172.21.15.172:0/2477034930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8c5370 0x55b57d8c25e0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.317+0000 7f1c203e6740 1 --2- 172.21.15.172:0/2477034930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 0x55b57d8c1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.317+0000 7f1c203e6740 1 --2- 172.21.15.172:0/2477034930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8ba5d0 0x55b57d8c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.317+0000 7f1c203e6740 1 -- 172.21.15.172:0/2477034930 >> 172.21.15.172:0/2477034930 conn(0x55b57d883310 msgr2=0x55b57d8bc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:03.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.317+0000 7f1bf6ffd640 0 client.19620 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:03.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.317+0000 7f1c203e6740 1 -- 172.21.15.172:0/2477034930 shutdown_connections 2024-09-10T09:02:03.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.317+0000 7f1c203e6740 1 -- 172.21.15.172:0/2477034930 wait complete. 2024-09-10T09:02:03.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.317+0000 7f1c203e6740 20 client.19620 trim_cache size 0 max 16384 2024-09-10T09:02:03.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c203e6740 1 Processor -- start 2024-09-10T09:02:03.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c203e6740 1 -- start start 2024-09-10T09:02:03.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c203e6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8c0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.348 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c203e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 0x55b57d8b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c203e6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8c5370 0x55b57d8bd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c203e6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181aec30 con 0x55b57d8ba5d0 2024-09-10T09:02:03.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c203e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181b0770 con 0x55b57d8bbb40 2024-09-10T09:02:03.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c203e6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181af300 con 0x55b57d8c5370 2024-09-10T09:02:03.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c16ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8c0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:03.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c16ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8c0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58652/0 (socket says 172.21.15.172:58652) 2024-09-10T09:02:03.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c16ffd640 1 -- 172.21.15.172:0/3945721265 learned_addr learned my addr 172.21.15.172:0/3945721265 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:03.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c177fe640 1 --2- 172.21.15.172:0/3945721265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 0x55b57d8b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:03.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c16ffd640 1 -- 172.21.15.172:0/3945721265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8c5370 msgr2=0x55b57d8bd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:03.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c16ffd640 1 --2- 172.21.15.172:0/3945721265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8c5370 0x55b57d8bd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c16ffd640 1 -- 172.21.15.172:0/3945721265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 msgr2=0x55b57d8b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:03.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c16ffd640 1 --2- 172.21.15.172:0/3945721265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 0x55b57d8b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c16ffd640 1 -- 172.21.15.172:0/3945721265 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b57d8bb300 con 0x55b57d8ba5d0 2024-09-10T09:02:03.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.330+0000 7f1c177fe640 1 --2- 172.21.15.172:0/3945721265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 0x55b57d8b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:02:03.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.331+0000 7f1c16ffd640 1 --2- 172.21.15.172:0/3945721265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8c0f70 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c004810 tx=0x7f1c0c0096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec24ec6b27cb6db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:03.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.331+0000 7f1bfd7fa640 1 -- 172.21.15.172:0/3945721265 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c013e50 con 0x55b57d8ba5d0 2024-09-10T09:02:03.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.331+0000 7f1bfd7fa640 1 -- 172.21.15.172:0/3945721265 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c0c006010 con 0x55b57d8ba5d0 2024-09-10T09:02:03.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.331+0000 7f1bfd7fa640 1 -- 172.21.15.172:0/3945721265 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0246d0 con 0x55b57d8ba5d0 2024-09-10T09:02:03.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.331+0000 7f1c203e6740 1 -- 172.21.15.172:0/3945721265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 msgr2=0x55b57d8c0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:03.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.331+0000 7f1c203e6740 1 --2- 172.21.15.172:0/3945721265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8c0f70 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c004810 tx=0x7f1c0c0096e0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.332+0000 7f1c203e6740 1 -- 172.21.15.172:0/3945721265 shutdown_connections 2024-09-10T09:02:03.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.332+0000 7f1c203e6740 1 --2- 172.21.15.172:0/3945721265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8c5370 0x55b57d8bd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.332+0000 7f1c203e6740 1 --2- 172.21.15.172:0/3945721265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 0x55b57d8b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.332+0000 7f1c203e6740 1 --2- 172.21.15.172:0/3945721265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8c0f70 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.332+0000 7f1c203e6740 1 -- 172.21.15.172:0/3945721265 >> 172.21.15.172:0/3945721265 conn(0x55b57d883310 msgr2=0x55b57d8b5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:03.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.332+0000 7f1c203e6740 1 -- 172.21.15.172:0/3945721265 shutdown_connections 2024-09-10T09:02:03.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.332+0000 7f1c203e6740 1 -- 172.21.15.172:0/3945721265 wait complete. 2024-09-10T09:02:03.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.332+0000 7f1c203e6740 1 -- start start 2024-09-10T09:02:03.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.332+0000 7f1c203e6740 1 Processor -- start 2024-09-10T09:02:03.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.332+0000 7f1c203e6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.332+0000 7f1c203e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 0x55b57d8cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.332+0000 7f1c203e6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8c5370 0x55b57d8be840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.332+0000 7f1c203e6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181b00a0 con 0x55b57d8ba5d0 2024-09-10T09:02:03.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.332+0000 7f1c203e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181af9d0 con 0x55b57d8bbb40 2024-09-10T09:02:03.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.332+0000 7f1c203e6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181acc10 con 0x55b57d8c5370 2024-09-10T09:02:03.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.332+0000 7f1c203e6740 10 client.0 _ll_register_callbacks cb 0x7f1c0c009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:02:03.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.333+0000 7f1c16ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:03.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.333+0000 7f1c16ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58660/0 (socket says 172.21.15.172:58660) 2024-09-10T09:02:03.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.333+0000 7f1c16ffd640 1 -- 172.21.15.172:0/999230845 learned_addr learned my addr 172.21.15.172:0/999230845 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:03.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.333+0000 7f1c177fe640 1 --2- 172.21.15.172:0/999230845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 0x55b57d8cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:03.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.333+0000 7f1c16ffd640 1 --2- 172.21.15.172:0/999230845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8c5370 0x55b57d8be840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:03.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.333+0000 7f1c16ffd640 1 -- 172.21.15.172:0/999230845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8c5370 msgr2=0x55b57d8be840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:03.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.333+0000 7f1c16ffd640 1 --2- 172.21.15.172:0/999230845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8c5370 0x55b57d8be840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.333+0000 7f1c16ffd640 1 -- 172.21.15.172:0/999230845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 msgr2=0x55b57d8cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:03.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.333+0000 7f1c16ffd640 1 --2- 172.21.15.172:0/999230845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 0x55b57d8cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.333+0000 7f1c16ffd640 1 -- 172.21.15.172:0/999230845 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b57d8bef00 con 0x55b57d8ba5d0 2024-09-10T09:02:03.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.333+0000 7f1c16ffd640 1 --2- 172.21.15.172:0/999230845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8cd600 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c018870 tx=0x7f1c0c0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312b4620db65de46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:03.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.333+0000 7f1bf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:02:03.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.333+0000 7f1bf6ffd640 1 -- 172.21.15.172:0/999230845 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c040060 con 0x55b57d8ba5d0 2024-09-10T09:02:03.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.333+0000 7f1bf6ffd640 1 -- 172.21.15.172:0/999230845 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1c0c005d60 con 0x55b57d8ba5d0 2024-09-10T09:02:03.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.333+0000 7f1c203e6740 1 -- 172.21.15.172:0/999230845 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c0c006010 con 0x55b57d8ba5d0 2024-09-10T09:02:03.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.334+0000 7f1bf6ffd640 1 -- 172.21.15.172:0/999230845 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1c0c024470 con 0x55b57d8ba5d0 2024-09-10T09:02:03.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.334+0000 7f1c203e6740 10 client.19626 fetch_fsmap learned FSMap version 44 2024-09-10T09:02:03.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.334+0000 7f1c203e6740 1 -- 172.21.15.172:0/999230845 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b57d93f580 con 0x55b57d8ba5d0 2024-09-10T09:02:03.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.334+0000 7f1bf6ffd640 1 -- 172.21.15.172:0/999230845 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c0c006010 con 0x55b57d8ba5d0 2024-09-10T09:02:03.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.334+0000 7f1bf6ffd640 1 -- 172.21.15.172:0/999230845 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f1c0c022050 con 0x55b57d8ba5d0 2024-09-10T09:02:03.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.334+0000 7f1c203e6740 10 client.19626 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:02:03.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.334+0000 7f1c203e6740 10 client.19626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:02:03.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.334+0000 7f1c203e6740 20 client.19626 populate_metadata read hostname 'smithi172' 2024-09-10T09:02:03.353 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.334+0000 7f1c203e6740 1 --2- 172.21.15.172:0/999230845 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b57d93fb40 0x55b57d941f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:03.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.334+0000 7f1c203e6740 4 client.19626 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:02:03.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.334+0000 7f1c203e6740 1 -- 172.21.15.172:0/999230845 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b57d827d70 con 0x55b57d93fb40 2024-09-10T09:02:03.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.334+0000 7f1c177fe640 1 --2- 172.21.15.172:0/999230845 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b57d93fb40 0x55b57d941f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:03.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.335+0000 7f1c177fe640 1 --2- 172.21.15.172:0/999230845 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b57d93fb40 0x55b57d941f60 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a54ed62c20652379 server_cookie=7d1c658fb22fd434 in_seq=0 out_seq=0 2024-09-10T09:02:03.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.335+0000 7f1bf6ffd640 10 client.19626 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:03.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.335+0000 7f1bf6ffd640 1 -- 172.21.15.172:0/999230845 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b57d827d70 con 0x55b57d93fb40 2024-09-10T09:02:03.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.335+0000 7f1bf6ffd640 10 client.19626 handle_command_reply: tid=0 2024-09-10T09:02:03.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.335+0000 7f1c203e6740 1 client.19626 shutdown 2024-09-10T09:02:03.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.335+0000 7f1c203e6740 1 -- 172.21.15.172:0/999230845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 msgr2=0x55b57d8cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:03.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.335+0000 7f1c203e6740 1 --2- 172.21.15.172:0/999230845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8cd600 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c018870 tx=0x7f1c0c0046a0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.335+0000 7f1c203e6740 1 -- 172.21.15.172:0/999230845 shutdown_connections 2024-09-10T09:02:03.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.335+0000 7f1c203e6740 1 --2- 172.21.15.172:0/999230845 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b57d93fb40 0x55b57d941f60 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.335+0000 7f1c203e6740 1 --2- 172.21.15.172:0/999230845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b57d8c5370 0x55b57d8be840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.335+0000 7f1c203e6740 1 --2- 172.21.15.172:0/999230845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b57d8bbb40 0x55b57d8cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.335+0000 7f1c203e6740 1 --2- 172.21.15.172:0/999230845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b57d8ba5d0 0x55b57d8cd600 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.335+0000 7f1c203e6740 1 -- 172.21.15.172:0/999230845 >> 172.21.15.172:0/999230845 conn(0x55b57d883310 msgr2=0x55b57d8c4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:03.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.335+0000 7f1bf6ffd640 0 client.19626 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:03.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.335+0000 7f1c203e6740 1 -- 172.21.15.172:0/999230845 shutdown_connections 2024-09-10T09:02:03.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.335+0000 7f1c203e6740 1 -- 172.21.15.172:0/999230845 wait complete. 2024-09-10T09:02:03.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.335+0000 7f1c203e6740 20 client.19626 trim_cache size 0 max 16384 2024-09-10T09:02:03.355 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:02:03.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.336+0000 7f1c1e00d640 1 -- 172.21.15.172:0/3416892189 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1c000a2b80 msgr2=0x7f1c000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:03.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.336+0000 7f1c1e00d640 1 --2- 172.21.15.172:0/3416892189 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1c000a2b80 0x7f1c000a5040 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1c1009b860 tx=0x7f1c10086040 comp rx=0 tx=0).stop 2024-09-10T09:02:03.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.336+0000 7f1c1e00d640 1 -- 172.21.15.172:0/3416892189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c181082a0 msgr2=0x7f1c1819ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:03.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.336+0000 7f1c1e00d640 1 --2- 172.21.15.172:0/3416892189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c181082a0 0x7f1c1819ce60 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000cad0 tx=0x7f1c1000ced0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.336+0000 7f1c1e00d640 1 -- 172.21.15.172:0/3416892189 shutdown_connections 2024-09-10T09:02:03.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.336+0000 7f1c1e00d640 1 --2- 172.21.15.172:0/3416892189 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1c000a2b80 0x7f1c000a5040 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.336+0000 7f1c1e00d640 1 --2- 172.21.15.172:0/3416892189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c181082a0 0x7f1c1819ce60 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.336+0000 7f1c1e00d640 1 --2- 172.21.15.172:0/3416892189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c181036e0 0x7f1c18198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.336+0000 7f1c1e00d640 1 --2- 172.21.15.172:0/3416892189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c18102d30 0x7f1c18198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:03.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.336+0000 7f1c1e00d640 1 -- 172.21.15.172:0/3416892189 >> 172.21.15.172:0/3416892189 conn(0x7f1c180fc7d0 msgr2=0x7f1c18105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:03.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.336+0000 7f1c1e00d640 1 -- 172.21.15.172:0/3416892189 shutdown_connections 2024-09-10T09:02:03.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:03.336+0000 7f1c1e00d640 1 -- 172.21.15.172:0/3416892189 wait complete. 2024-09-10T09:02:03.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:03 smithi174 ceph-mon[34068]: pgmap v824: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 91 KiB/s wr, 34 op/s 2024-09-10T09:02:03.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:03 smithi191 ceph-mon[35314]: pgmap v824: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 91 KiB/s wr, 34 op/s 2024-09-10T09:02:04.079 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:02:04.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.562+0000 7f5cd08ed640 1 -- 172.21.15.172:0/4067451387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ccc101de0 msgr2=0x7f5ccc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:04.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.562+0000 7f5cd08ed640 1 --2- 172.21.15.172:0/4067451387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ccc101de0 0x7f5ccc1021c0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009900 tx=0x7f5cbc02a410 comp rx=0 tx=0).stop 2024-09-10T09:02:04.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.562+0000 7f5cd08ed640 1 -- 172.21.15.172:0/4067451387 shutdown_connections 2024-09-10T09:02:04.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.562+0000 7f5cd08ed640 1 --2- 172.21.15.172:0/4067451387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ccc106f10 0x7f5ccc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.562+0000 7f5cd08ed640 1 --2- 172.21.15.172:0/4067451387 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ccc102790 0x7f5ccc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.564 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.562+0000 7f5cd08ed640 1 --2- 172.21.15.172:0/4067451387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ccc101de0 0x7f5ccc1021c0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.562+0000 7f5cd08ed640 1 -- 172.21.15.172:0/4067451387 >> 172.21.15.172:0/4067451387 conn(0x7f5ccc0fd8c0 msgr2=0x7f5ccc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:04.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.562+0000 7f5cd08ed640 1 -- 172.21.15.172:0/4067451387 shutdown_connections 2024-09-10T09:02:04.565 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.562+0000 7f5cd08ed640 1 -- 172.21.15.172:0/4067451387 wait complete. 2024-09-10T09:02:04.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cd08ed640 1 Processor -- start 2024-09-10T09:02:04.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cd08ed640 1 -- start start 2024-09-10T09:02:04.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cd08ed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ccc101de0 0x7f5ccc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cd08ed640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ccc102790 0x7f5ccc198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cd08ed640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ccc106f10 0x7f5ccc19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cd08ed640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc111e20 con 0x7f5ccc101de0 2024-09-10T09:02:04.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cd08ed640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc111ca0 con 0x7f5ccc102790 2024-09-10T09:02:04.566 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cd08ed640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc111fa0 con 0x7f5ccc106f10 2024-09-10T09:02:04.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ccc101de0 0x7f5ccc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ccc101de0 0x7f5ccc198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58670/0 (socket says 172.21.15.172:58670) 2024-09-10T09:02:04.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cca59c640 1 -- 172.21.15.172:0/3264075979 learned_addr learned my addr 172.21.15.172:0/3264075979 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:04.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cc9d9b640 1 --2- 172.21.15.172:0/3264075979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ccc102790 0x7f5ccc198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5ccad9d640 1 --2- 172.21.15.172:0/3264075979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ccc106f10 0x7f5ccc19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cca59c640 1 -- 172.21.15.172:0/3264075979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ccc106f10 msgr2=0x7f5ccc19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:04.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cca59c640 1 --2- 172.21.15.172:0/3264075979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ccc106f10 0x7f5ccc19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cca59c640 1 -- 172.21.15.172:0/3264075979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ccc102790 msgr2=0x7f5ccc198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:04.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cca59c640 1 --2- 172.21.15.172:0/3264075979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ccc102790 0x7f5ccc198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cca59c640 1 -- 172.21.15.172:0/3264075979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc19d300 con 0x7f5ccc101de0 2024-09-10T09:02:04.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.563+0000 7f5cca59c640 1 --2- 172.21.15.172:0/3264075979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ccc101de0 0x7f5ccc198350 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc005940 tx=0x7f5cbc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:04.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.564+0000 7f5cbb7fe640 1 -- 172.21.15.172:0/3264075979 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc03f030 con 0x7f5ccc101de0 2024-09-10T09:02:04.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.564+0000 7f5cbb7fe640 1 -- 172.21.15.172:0/3264075979 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cbc003cb0 con 0x7f5ccc101de0 2024-09-10T09:02:04.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.564+0000 7f5cd08ed640 1 -- 172.21.15.172:0/3264075979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ccc19d590 con 0x7f5ccc101de0 2024-09-10T09:02:04.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.564+0000 7f5cbb7fe640 1 -- 172.21.15.172:0/3264075979 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc035070 con 0x7f5ccc101de0 2024-09-10T09:02:04.568 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.564+0000 7f5cd08ed640 1 -- 172.21.15.172:0/3264075979 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ccc1a4ed0 con 0x7f5ccc101de0 2024-09-10T09:02:04.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.566+0000 7f5cbb7fe640 1 -- 172.21.15.172:0/3264075979 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5cbc042030 con 0x7f5ccc101de0 2024-09-10T09:02:04.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.566+0000 7f5cd2cc6740 1 Processor -- start 2024-09-10T09:02:04.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cd2cc6740 1 -- start start 2024-09-10T09:02:04.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cd2cc6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2825f20 0x564ac2826300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cd2cc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2826840 0x564ac2835110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cd2cc6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2835650 0x564ac2837a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cd2cc6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564ac278b620 con 0x564ac2835650 2024-09-10T09:02:04.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cd2cc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564ac2732e50 con 0x564ac2826840 2024-09-10T09:02:04.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cd2cc6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564ac27875e0 con 0x564ac2825f20 2024-09-10T09:02:04.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cc9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2835650 0x564ac2837a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cc9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2835650 0x564ac2837a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58684/0 (socket says 172.21.15.172:58684) 2024-09-10T09:02:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cc9d9b640 1 -- 172.21.15.172:0/367288380 learned_addr learned my addr 172.21.15.172:0/367288380 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cca59c640 1 --2- 172.21.15.172:0/367288380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2826840 0x564ac2835110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cbb7fe640 1 --2- 172.21.15.172:0/3264075979 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5cb40a2c00 0x7f5cb40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5ccad9d640 1 --2- 172.21.15.172:0/367288380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2825f20 0x564ac2826300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5ccad9d640 1 --2- 172.21.15.172:0/3264075979 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5cb40a2c00 0x7f5cb40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cc9d9b640 1 -- 172.21.15.172:0/367288380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2825f20 msgr2=0x564ac2826300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cc9d9b640 1 --2- 172.21.15.172:0/367288380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2825f20 0x564ac2826300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cc9d9b640 1 -- 172.21.15.172:0/367288380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2826840 msgr2=0x564ac2835110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cc9d9b640 1 --2- 172.21.15.172:0/367288380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2826840 0x564ac2835110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cc9d9b640 1 -- 172.21.15.172:0/367288380 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ac2723d40 con 0x564ac2835650 2024-09-10T09:02:04.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.567+0000 7f5cbb7fe640 1 -- 172.21.15.172:0/3264075979 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5cbc0efa80 con 0x7f5ccc101de0 2024-09-10T09:02:04.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.568+0000 7f5cc9d9b640 1 --2- 172.21.15.172:0/367288380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2835650 0x564ac2837a40 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0009ca0 tx=0x7f5cc0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254985bae6afc0f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:04.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.568+0000 7f5cb8ff9640 1 -- 172.21.15.172:0/367288380 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc001e070 con 0x564ac2835650 2024-09-10T09:02:04.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.568+0000 7f5ccad9d640 1 --2- 172.21.15.172:0/3264075979 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5cb40a2c00 0x7f5cb40a50c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4007310 tx=0x7f5cc4007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:04.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.568+0000 7f5cb8ff9640 1 -- 172.21.15.172:0/367288380 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cc0019ae0 con 0x564ac2835650 2024-09-10T09:02:04.572 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.568+0000 7f5cb8ff9640 1 -- 172.21.15.172:0/367288380 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc000e9b0 con 0x564ac2835650 2024-09-10T09:02:04.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/367288380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2835650 msgr2=0x564ac2837a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:04.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/367288380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2835650 0x564ac2837a40 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0009ca0 tx=0x7f5cc0011810 comp rx=0 tx=0).stop 2024-09-10T09:02:04.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/367288380 shutdown_connections 2024-09-10T09:02:04.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/367288380 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2835650 0x564ac2837a40 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/367288380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2826840 0x564ac2835110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/367288380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2825f20 0x564ac2826300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/367288380 >> 172.21.15.172:0/367288380 conn(0x564ac28225f0 msgr2=0x564ac28229d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:04.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/367288380 shutdown_connections 2024-09-10T09:02:04.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/367288380 wait complete. 2024-09-10T09:02:04.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 1 -- start start 2024-09-10T09:02:04.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 1 Processor -- start 2024-09-10T09:02:04.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2825f20 0x564ac2832630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2826840 0x564ac2832b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2835650 0x564ac28330b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564ac2838220 con 0x564ac2826840 2024-09-10T09:02:04.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564ac2713f10 con 0x564ac2835650 2024-09-10T09:02:04.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564ac28383a0 con 0x564ac2825f20 2024-09-10T09:02:04.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.569+0000 7f5cd2cc6740 10 client.0 _ll_register_callbacks cb 0x564ac2737db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:02:04.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5cc9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2826840 0x564ac2832b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5cc9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2826840 0x564ac2832b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58700/0 (socket says 172.21.15.172:58700) 2024-09-10T09:02:04.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5cc9d9b640 1 -- 172.21.15.172:0/4097278324 learned_addr learned my addr 172.21.15.172:0/4097278324 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:04.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5cca59c640 1 --2- 172.21.15.172:0/4097278324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2825f20 0x564ac2832630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5ccad9d640 1 --2- 172.21.15.172:0/4097278324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2835650 0x564ac28330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5cc9d9b640 1 -- 172.21.15.172:0/4097278324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2825f20 msgr2=0x564ac2832630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:04.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5cc9d9b640 1 --2- 172.21.15.172:0/4097278324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2825f20 0x564ac2832630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5cc9d9b640 1 -- 172.21.15.172:0/4097278324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2835650 msgr2=0x564ac28330b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:04.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5cc9d9b640 1 --2- 172.21.15.172:0/4097278324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2835650 0x564ac28330b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5cc9d9b640 1 -- 172.21.15.172:0/4097278324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ac249a580 con 0x564ac2826840 2024-09-10T09:02:04.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5cc9d9b640 1 --2- 172.21.15.172:0/4097278324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2826840 0x564ac2832b70 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0009c70 tx=0x7f5cc000e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bafe5d4e374bc36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:04.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:02:04.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5cb1ffb640 1 -- 172.21.15.172:0/4097278324 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0027030 con 0x564ac2826840 2024-09-10T09:02:04.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5cb1ffb640 1 -- 172.21.15.172:0/4097278324 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5cc0005320 con 0x564ac2826840 2024-09-10T09:02:04.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/4097278324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cc0019ae0 con 0x564ac2826840 2024-09-10T09:02:04.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5cb1ffb640 1 -- 172.21.15.172:0/4097278324 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5cc0031050 con 0x564ac2826840 2024-09-10T09:02:04.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5cd2cc6740 10 client.19638 fetch_fsmap learned FSMap version 44 2024-09-10T09:02:04.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.570+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/4097278324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564ac2564960 con 0x564ac2826840 2024-09-10T09:02:04.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.571+0000 7f5cb1ffb640 1 -- 172.21.15.172:0/4097278324 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cc0019ae0 con 0x564ac2826840 2024-09-10T09:02:04.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.571+0000 7f5cb1ffb640 1 -- 172.21.15.172:0/4097278324 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f5cc000e740 con 0x564ac2826840 2024-09-10T09:02:04.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.571+0000 7f5cd2cc6740 10 client.19638 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:02:04.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.571+0000 7f5cd2cc6740 10 client.19638 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:02:04.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.571+0000 7f5cd2cc6740 20 client.19638 populate_metadata read hostname 'smithi172' 2024-09-10T09:02:04.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.571+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/4097278324 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564ac2845220 0x564ac2847610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.571+0000 7f5cd2cc6740 4 client.19638 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:02:04.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.571+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/4097278324 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ac2798b90 con 0x564ac2845220 2024-09-10T09:02:04.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.571+0000 7f5cca59c640 1 --2- 172.21.15.172:0/4097278324 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564ac2845220 0x564ac2847610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.572+0000 7f5cca59c640 1 --2- 172.21.15.172:0/4097278324 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564ac2845220 0x564ac2847610 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e69459c810d70a77 server_cookie=4b92dae5fe784c9a in_seq=0 out_seq=0 2024-09-10T09:02:04.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.572+0000 7f5cb1ffb640 10 client.19638 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:04.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.572+0000 7f5cb1ffb640 1 -- 172.21.15.172:0/4097278324 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ac2798b90 con 0x564ac2845220 2024-09-10T09:02:04.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.572+0000 7f5cb1ffb640 10 client.19638 handle_command_reply: tid=0 2024-09-10T09:02:04.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.572+0000 7f5cd2cc6740 1 client.19638 shutdown 2024-09-10T09:02:04.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.572+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/4097278324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2826840 msgr2=0x564ac2832b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:04.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.572+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/4097278324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2826840 0x564ac2832b70 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0009c70 tx=0x7f5cc000e3b0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.573+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/4097278324 shutdown_connections 2024-09-10T09:02:04.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.573+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/4097278324 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564ac2845220 0x564ac2847610 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.573+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/4097278324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2835650 0x564ac28330b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.573+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/4097278324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2826840 0x564ac2832b70 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.573+0000 7f5cb1ffb640 0 client.19638 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:04.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.573+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/4097278324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2825f20 0x564ac2832630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.573+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/4097278324 >> 172.21.15.172:0/4097278324 conn(0x564ac28225f0 msgr2=0x564ac2828350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:04.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.573+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/4097278324 shutdown_connections 2024-09-10T09:02:04.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.573+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/4097278324 wait complete. 2024-09-10T09:02:04.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.573+0000 7f5cd2cc6740 20 client.19638 trim_cache size 0 max 16384 2024-09-10T09:02:04.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cd2cc6740 1 Processor -- start 2024-09-10T09:02:04.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cd2cc6740 1 -- start start 2024-09-10T09:02:04.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cd2cc6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2825f20 0x564ac28314e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cd2cc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2826840 0x564ac284cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cd2cc6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2835650 0x564ac284d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cd2cc6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1ac840 con 0x564ac2825f20 2024-09-10T09:02:04.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cd2cc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1ae380 con 0x564ac2826840 2024-09-10T09:02:04.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cd2cc6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1acf10 con 0x564ac2835650 2024-09-10T09:02:04.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2825f20 0x564ac28314e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2825f20 0x564ac28314e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58708/0 (socket says 172.21.15.172:58708) 2024-09-10T09:02:04.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cca59c640 1 -- 172.21.15.172:0/3998463752 learned_addr learned my addr 172.21.15.172:0/3998463752 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:04.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cc9d9b640 1 --2- 172.21.15.172:0/3998463752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2826840 0x564ac284cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5ccad9d640 1 --2- 172.21.15.172:0/3998463752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2835650 0x564ac284d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cca59c640 1 -- 172.21.15.172:0/3998463752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2835650 msgr2=0x564ac284d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:04.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cca59c640 1 --2- 172.21.15.172:0/3998463752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2835650 0x564ac284d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cca59c640 1 -- 172.21.15.172:0/3998463752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2826840 msgr2=0x564ac284cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:04.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cca59c640 1 --2- 172.21.15.172:0/3998463752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2826840 0x564ac284cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cca59c640 1 -- 172.21.15.172:0/3998463752 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ac2820bb0 con 0x564ac2825f20 2024-09-10T09:02:04.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cc9d9b640 1 --2- 172.21.15.172:0/3998463752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2826840 0x564ac284cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:02:04.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cca59c640 1 --2- 172.21.15.172:0/3998463752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2825f20 0x564ac28314e0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0bbd30 tx=0x7f5cbc0ed0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea2e2a6f513ca02 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:04.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cb3fff640 1 -- 172.21.15.172:0/3998463752 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc049040 con 0x564ac2825f20 2024-09-10T09:02:04.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cb3fff640 1 -- 172.21.15.172:0/3998463752 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cbc03b460 con 0x564ac2825f20 2024-09-10T09:02:04.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.590+0000 7f5cb3fff640 1 -- 172.21.15.172:0/3998463752 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc0c0330 con 0x564ac2825f20 2024-09-10T09:02:04.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.591+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/3998463752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2825f20 msgr2=0x564ac28314e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:04.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.591+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/3998463752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2825f20 0x564ac28314e0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0bbd30 tx=0x7f5cbc0ed0f0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.591+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/3998463752 shutdown_connections 2024-09-10T09:02:04.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.591+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/3998463752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2835650 0x564ac284d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.591+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/3998463752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2826840 0x564ac284cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.591+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/3998463752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2825f20 0x564ac28314e0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.591+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/3998463752 >> 172.21.15.172:0/3998463752 conn(0x564ac28225f0 msgr2=0x564ac2830dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:04.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.591+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/3998463752 shutdown_connections 2024-09-10T09:02:04.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.591+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/3998463752 wait complete. 2024-09-10T09:02:04.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.591+0000 7f5cd2cc6740 1 -- start start 2024-09-10T09:02:04.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.591+0000 7f5cd2cc6740 1 Processor -- start 2024-09-10T09:02:04.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cd2cc6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2825f20 0x564ac2839770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cd2cc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2826840 0x564ac2839cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cd2cc6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2835650 0x564ac28aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cd2cc6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1adcb0 con 0x564ac2825f20 2024-09-10T09:02:04.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cd2cc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1ad5e0 con 0x564ac2826840 2024-09-10T09:02:04.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cd2cc6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc1aa800 con 0x564ac2835650 2024-09-10T09:02:04.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cd2cc6740 10 client.0 _ll_register_callbacks cb 0x7f5cc0009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:02:04.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2825f20 0x564ac2839770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cca59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2825f20 0x564ac2839770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58718/0 (socket says 172.21.15.172:58718) 2024-09-10T09:02:04.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cca59c640 1 -- 172.21.15.172:0/2847314592 learned_addr learned my addr 172.21.15.172:0/2847314592 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:04.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5ccad9d640 1 --2- 172.21.15.172:0/2847314592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2835650 0x564ac28aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cc9d9b640 1 --2- 172.21.15.172:0/2847314592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2826840 0x564ac2839cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cca59c640 1 -- 172.21.15.172:0/2847314592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2835650 msgr2=0x564ac28aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:04.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cca59c640 1 --2- 172.21.15.172:0/2847314592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2835650 0x564ac28aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cca59c640 1 -- 172.21.15.172:0/2847314592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2826840 msgr2=0x564ac2839cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:04.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cca59c640 1 --2- 172.21.15.172:0/2847314592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2826840 0x564ac2839cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cca59c640 1 -- 172.21.15.172:0/2847314592 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ac283a5b0 con 0x564ac2825f20 2024-09-10T09:02:04.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cca59c640 1 --2- 172.21.15.172:0/2847314592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2825f20 0x564ac2839770 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc031440 tx=0x7f5cbc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2068aa10ed7dd76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:04.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:02:04.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cb1ffb640 1 -- 172.21.15.172:0/2847314592 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc049070 con 0x564ac2825f20 2024-09-10T09:02:04.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cb1ffb640 1 -- 172.21.15.172:0/2847314592 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f5cbc0b86a0 con 0x564ac2825f20 2024-09-10T09:02:04.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.592+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/2847314592 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564ac27930a0 con 0x564ac2825f20 2024-09-10T09:02:04.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.593+0000 7f5cb1ffb640 1 -- 172.21.15.172:0/2847314592 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f5cbc0bb990 con 0x564ac2825f20 2024-09-10T09:02:04.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.593+0000 7f5cd2cc6740 10 client.19644 fetch_fsmap learned FSMap version 44 2024-09-10T09:02:04.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.593+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/2847314592 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564ac28b0470 con 0x564ac2825f20 2024-09-10T09:02:04.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.593+0000 7f5cb1ffb640 1 -- 172.21.15.172:0/2847314592 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x564ac27930a0 con 0x564ac2825f20 2024-09-10T09:02:04.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.593+0000 7f5cb1ffb640 1 -- 172.21.15.172:0/2847314592 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f5cbc0c4760 con 0x564ac2825f20 2024-09-10T09:02:04.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.593+0000 7f5cd2cc6740 10 client.19644 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:02:04.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.593+0000 7f5cd2cc6740 10 client.19644 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:02:04.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.593+0000 7f5cd2cc6740 20 client.19644 populate_metadata read hostname 'smithi172' 2024-09-10T09:02:04.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.593+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/2847314592 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564ac28b0b90 0x564ac28b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:04.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.593+0000 7f5cd2cc6740 4 client.19644 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3e265114", "release": true, "await": true}] 2024-09-10T09:02:04.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.593+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/2847314592 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3e265114", "release": true, "await": true}) -- 0x564ac270f5a0 con 0x564ac28b0b90 2024-09-10T09:02:04.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.593+0000 7f5cc9d9b640 1 --2- 172.21.15.172:0/2847314592 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564ac28b0b90 0x564ac28b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:04.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.594+0000 7f5cc9d9b640 1 --2- 172.21.15.172:0/2847314592 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564ac28b0b90 0x564ac28b2f80 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eab378442e6f5289 server_cookie=ef8204d1441060df in_seq=0 out_seq=0 2024-09-10T09:02:04.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.594+0000 7f5cb1ffb640 10 client.19644 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:04.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.611+0000 7f5cb1ffb640 1 -- 172.21.15.172:0/2847314592 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564ac270f5a0 con 0x564ac28b0b90 2024-09-10T09:02:04.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.611+0000 7f5cb1ffb640 10 client.19644 handle_command_reply: tid=0 2024-09-10T09:02:04.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.611+0000 7f5cd2cc6740 1 client.19644 shutdown 2024-09-10T09:02:04.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.611+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/2847314592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2825f20 msgr2=0x564ac2839770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:04.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.611+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/2847314592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2825f20 0x564ac2839770 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc031440 tx=0x7f5cbc005d50 comp rx=0 tx=0).stop 2024-09-10T09:02:04.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.611+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/2847314592 shutdown_connections 2024-09-10T09:02:04.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.611+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/2847314592 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x564ac28b0b90 0x564ac28b2f80 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.611+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/2847314592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564ac2835650 0x564ac28aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.611+0000 7f5cb1ffb640 0 client.19644 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:04.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.611+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/2847314592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ac2826840 0x564ac2839cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.611+0000 7f5cd2cc6740 1 --2- 172.21.15.172:0/2847314592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564ac2825f20 0x564ac2839770 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.611+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/2847314592 >> 172.21.15.172:0/2847314592 conn(0x564ac28225f0 msgr2=0x564ac2821750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:04.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.611+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/2847314592 shutdown_connections 2024-09-10T09:02:04.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.611+0000 7f5cd2cc6740 1 -- 172.21.15.172:0/2847314592 wait complete. 2024-09-10T09:02:04.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.612+0000 7f5cd2cc6740 20 client.19644 trim_cache size 0 max 16384 2024-09-10T09:02:04.615 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:02:04.615 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 44, 2024-09-10T09:02:04.615 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T09:02:04.615 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 52, 2024-09-10T09:02:04.615 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T09:02:04.615 INFO:teuthology.orchestra.run.smithi172.stdout: "3e265114": { 2024-09-10T09:02:04.615 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 52, 2024-09-10T09:02:04.615 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T09:02:04.615 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T09:02:04.616 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T09:02:04.616 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T09:02:04.616 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:02:04.616 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T09:02:04.616 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 141.5, 2024-09-10T09:02:04.616 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T09:02:04.616 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T09:02:04.616 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T09:02:04.616 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T09:02:04.616 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T09:02:04.616 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T09:02:04.616 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:02:04.616 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:02:04.617 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:02:04.617 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:02:04.617 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:02:04.617 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:02:04.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.612+0000 7f5cd08ed640 1 -- 172.21.15.172:0/3264075979 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5cb40a2c00 msgr2=0x7f5cb40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:04.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.612+0000 7f5cd08ed640 1 --2- 172.21.15.172:0/3264075979 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5cb40a2c00 0x7f5cb40a50c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4007310 tx=0x7f5cc4007520 comp rx=0 tx=0).stop 2024-09-10T09:02:04.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.612+0000 7f5cd08ed640 1 -- 172.21.15.172:0/3264075979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ccc101de0 msgr2=0x7f5ccc198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:04.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.612+0000 7f5cd08ed640 1 --2- 172.21.15.172:0/3264075979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ccc101de0 0x7f5ccc198350 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc005940 tx=0x7f5cbc0042f0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.612+0000 7f5cd08ed640 1 -- 172.21.15.172:0/3264075979 shutdown_connections 2024-09-10T09:02:04.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.612+0000 7f5cd08ed640 1 --2- 172.21.15.172:0/3264075979 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f5cb40a2c00 0x7f5cb40a50c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.613+0000 7f5cd08ed640 1 --2- 172.21.15.172:0/3264075979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ccc106f10 0x7f5ccc19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.613+0000 7f5cd08ed640 1 --2- 172.21.15.172:0/3264075979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ccc102790 0x7f5ccc198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.613+0000 7f5cd08ed640 1 --2- 172.21.15.172:0/3264075979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ccc101de0 0x7f5ccc198350 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:04.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.613+0000 7f5cd08ed640 1 -- 172.21.15.172:0/3264075979 >> 172.21.15.172:0/3264075979 conn(0x7f5ccc0fd8c0 msgr2=0x7f5ccc0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:04.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.614+0000 7f5cd08ed640 1 -- 172.21.15.172:0/3264075979 shutdown_connections 2024-09-10T09:02:04.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:04.614+0000 7f5cd08ed640 1 -- 172.21.15.172:0/3264075979 wait complete. 2024-09-10T09:02:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.039+0000 7f4246ce0640 1 -- 172.21.15.172:0/3182466431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4240109b30 msgr2=0x7f42401103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.039+0000 7f4246ce0640 1 --2- 172.21.15.172:0/3182466431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4240109b30 0x7f42401103c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f423801acb0 tx=0x7f423803b900 comp rx=0 tx=0).stop 2024-09-10T09:02:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.040+0000 7f4246ce0640 1 -- 172.21.15.172:0/3182466431 shutdown_connections 2024-09-10T09:02:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.040+0000 7f4246ce0640 1 --2- 172.21.15.172:0/3182466431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4240109b30 0x7f42401103c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.040+0000 7f4246ce0640 1 --2- 172.21.15.172:0/3182466431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4240109170 0x7f42401095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.040+0000 7f4246ce0640 1 --2- 172.21.15.172:0/3182466431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4240108410 0x7f4240108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.040+0000 7f4246ce0640 1 -- 172.21.15.172:0/3182466431 >> 172.21.15.172:0/3182466431 conn(0x7f42400771d0 msgr2=0x7f4240079610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.041+0000 7f4246ce0640 1 -- 172.21.15.172:0/3182466431 shutdown_connections 2024-09-10T09:02:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.042+0000 7f4246ce0640 1 -- 172.21.15.172:0/3182466431 wait complete. 2024-09-10T09:02:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.042+0000 7f4246ce0640 1 Processor -- start 2024-09-10T09:02:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.043+0000 7f4246ce0640 1 -- start start 2024-09-10T09:02:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.043+0000 7f4246ce0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4240108410 0x7f4240195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.043+0000 7f4246ce0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4240109170 0x7f4240196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.043+0000 7f4246ce0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4240109b30 0x7f424007ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.043+0000 7f4246ce0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f424006e730 con 0x7f4240108410 2024-09-10T09:02:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.043+0000 7f4246ce0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f424006e5b0 con 0x7f4240109170 2024-09-10T09:02:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.043+0000 7f4246ce0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f424006e8b0 con 0x7f4240109b30 2024-09-10T09:02:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.043+0000 7f42464df640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4240109b30 0x7f424007ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.043+0000 7f42464df640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4240109b30 0x7f424007ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:50866/0 (socket says 172.21.15.172:50866) 2024-09-10T09:02:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.043+0000 7f42464df640 1 -- 172.21.15.172:0/3610768758 learned_addr learned my addr 172.21.15.172:0/3610768758 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.046+0000 7f42454dd640 1 --2- 172.21.15.172:0/3610768758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4240109170 0x7f4240196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.046+0000 7f42464df640 1 -- 172.21.15.172:0/3610768758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4240109170 msgr2=0x7f4240196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.046+0000 7f42464df640 1 --2- 172.21.15.172:0/3610768758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4240109170 0x7f4240196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.046+0000 7f4245cde640 1 --2- 172.21.15.172:0/3610768758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4240108410 0x7f4240195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.046+0000 7f42464df640 1 -- 172.21.15.172:0/3610768758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4240108410 msgr2=0x7f4240195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.046+0000 7f42464df640 1 --2- 172.21.15.172:0/3610768758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4240108410 0x7f4240195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.046+0000 7f42464df640 1 -- 172.21.15.172:0/3610768758 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424007d0c0 con 0x7f4240109b30 2024-09-10T09:02:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.046+0000 7f4245cde640 1 --2- 172.21.15.172:0/3610768758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4240108410 0x7f4240195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:02:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.050+0000 7f42464df640 1 --2- 172.21.15.172:0/3610768758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4240109b30 0x7f424007ca50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f423801ac00 tx=0x7f4238003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.051+0000 7f4236ffd640 1 -- 172.21.15.172:0/3610768758 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238045070 con 0x7f4240109b30 2024-09-10T09:02:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.051+0000 7f4236ffd640 1 -- 172.21.15.172:0/3610768758 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4238003d80 con 0x7f4240109b30 2024-09-10T09:02:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.052+0000 7f4246ce0640 1 -- 172.21.15.172:0/3610768758 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424007d350 con 0x7f4240109b30 2024-09-10T09:02:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.052+0000 7f4246ce0640 1 -- 172.21.15.172:0/3610768758 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f424007d840 con 0x7f4240109b30 2024-09-10T09:02:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.061+0000 7f424acd9740 1 Processor -- start 2024-09-10T09:02:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.062+0000 7f424acd9740 1 -- start start 2024-09-10T09:02:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.062+0000 7f4236ffd640 1 -- 172.21.15.172:0/3610768758 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380047c0 con 0x7f4240109b30 2024-09-10T09:02:05.071 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.062+0000 7f4236ffd640 1 -- 172.21.15.172:0/3610768758 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4238048020 con 0x7f4240109b30 2024-09-10T09:02:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.062+0000 7f424acd9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcad7b70 0x5581fcad7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.062+0000 7f424acd9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad84b0 0x5581fcadcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.062+0000 7f424acd9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcadd540 0x5581fcadfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.062+0000 7f424acd9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5581fca3b2f0 con 0x5581fcad84b0 2024-09-10T09:02:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.062+0000 7f424acd9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5581fc9e3270 con 0x5581fcadd540 2024-09-10T09:02:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.062+0000 7f424acd9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5581fca372b0 con 0x5581fcad7b70 2024-09-10T09:02:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.063+0000 7f42454dd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcad7b70 0x5581fcad7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.063+0000 7f42454dd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcad7b70 0x5581fcad7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:50878/0 (socket says 172.21.15.172:50878) 2024-09-10T09:02:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.063+0000 7f42454dd640 1 -- 172.21.15.172:0/3566265357 learned_addr learned my addr 172.21.15.172:0/3566265357 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:05.073 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.065+0000 7f42454dd640 1 --2- 172.21.15.172:0/3566265357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcadd540 0x5581fcadfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.065+0000 7f4245cde640 1 --2- 172.21.15.172:0/3566265357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad84b0 0x5581fcadcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.065+0000 7f42454dd640 1 -- 172.21.15.172:0/3566265357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcadd540 msgr2=0x5581fcadfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.065+0000 7f42454dd640 1 --2- 172.21.15.172:0/3566265357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcadd540 0x5581fcadfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.065+0000 7f42454dd640 1 -- 172.21.15.172:0/3566265357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad84b0 msgr2=0x5581fcadcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:05.074 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.065+0000 7f42454dd640 1 --2- 172.21.15.172:0/3566265357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad84b0 0x5581fcadcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.065+0000 7f42454dd640 1 -- 172.21.15.172:0/3566265357 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581fc9e2340 con 0x5581fcad7b70 2024-09-10T09:02:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.065+0000 7f4245cde640 1 --2- 172.21.15.172:0/3566265357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad84b0 0x5581fcadcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:02:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.065+0000 7f42454dd640 1 --2- 172.21.15.172:0/3566265357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcad7b70 0x5581fcad7f70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f423c016500 tx=0x7f423c00a490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa1e2ceabdb90fa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.066+0000 7f422bfff640 1 -- 172.21.15.172:0/3566265357 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c018430 con 0x5581fcad7b70 2024-09-10T09:02:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.068+0000 7f422bfff640 1 -- 172.21.15.172:0/3566265357 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f423c018a80 con 0x5581fcad7b70 2024-09-10T09:02:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.068+0000 7f422bfff640 1 -- 172.21.15.172:0/3566265357 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c014990 con 0x5581fcad7b70 2024-09-10T09:02:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.069+0000 7f424acd9740 1 -- 172.21.15.172:0/3566265357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcad7b70 msgr2=0x5581fcad7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.069+0000 7f424acd9740 1 --2- 172.21.15.172:0/3566265357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcad7b70 0x5581fcad7f70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f423c016500 tx=0x7f423c00a490 comp rx=0 tx=0).stop 2024-09-10T09:02:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.069+0000 7f424acd9740 1 -- 172.21.15.172:0/3566265357 shutdown_connections 2024-09-10T09:02:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.069+0000 7f424acd9740 1 --2- 172.21.15.172:0/3566265357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcadd540 0x5581fcadfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.069+0000 7f424acd9740 1 --2- 172.21.15.172:0/3566265357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad84b0 0x5581fcadcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.069+0000 7f424acd9740 1 --2- 172.21.15.172:0/3566265357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcad7b70 0x5581fcad7f70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.069+0000 7f424acd9740 1 -- 172.21.15.172:0/3566265357 >> 172.21.15.172:0/3566265357 conn(0x5581fcad31e0 msgr2=0x5581fcad35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.069+0000 7f4236ffd640 1 --2- 172.21.15.172:0/3610768758 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f422c0a2c00 0x7f422c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.070+0000 7f4236ffd640 1 -- 172.21.15.172:0/3610768758 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4238102310 con 0x7f4240109b30 2024-09-10T09:02:05.078 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:02:05.079 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:02:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.072+0000 7f4245cde640 1 --2- 172.21.15.172:0/3610768758 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f422c0a2c00 0x7f422c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.073+0000 7f424acd9740 1 -- 172.21.15.172:0/3566265357 shutdown_connections 2024-09-10T09:02:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.075+0000 7f424acd9740 1 -- 172.21.15.172:0/3566265357 wait complete. 2024-09-10T09:02:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.076+0000 7f424acd9740 1 -- start start 2024-09-10T09:02:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.076+0000 7f424acd9740 1 Processor -- start 2024-09-10T09:02:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.077+0000 7f424acd9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcad7b70 0x5581fcae1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.077+0000 7f424acd9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad84b0 0x5581fcae2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.077+0000 7f424acd9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcadd540 0x5581fcae9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.077+0000 7f424acd9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5581fcae04b0 con 0x5581fcad84b0 2024-09-10T09:02:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.077+0000 7f424acd9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5581fc9c4290 con 0x5581fcad7b70 2024-09-10T09:02:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.077+0000 7f424acd9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5581fcae0630 con 0x5581fcadd540 2024-09-10T09:02:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.077+0000 7f424acd9740 10 client.0 _ll_register_callbacks cb 0x5581fc9f5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:02:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.078+0000 7f42454dd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad84b0 0x5581fcae2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.079+0000 7f42454dd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad84b0 0x5581fcae2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58760/0 (socket says 172.21.15.172:58760) 2024-09-10T09:02:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.079+0000 7f42454dd640 1 -- 172.21.15.172:0/1712213743 learned_addr learned my addr 172.21.15.172:0/1712213743 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.084+0000 7f42454dd640 1 -- 172.21.15.172:0/1712213743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcadd540 msgr2=0x5581fcae9380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:02:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.084+0000 7f42454dd640 1 --2- 172.21.15.172:0/1712213743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcadd540 0x5581fcae9380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.084+0000 7f42454dd640 1 -- 172.21.15.172:0/1712213743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcad7b70 msgr2=0x5581fcae1e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:02:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.084+0000 7f42454dd640 1 --2- 172.21.15.172:0/1712213743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcad7b70 0x5581fcae1e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.084+0000 7f42454dd640 1 -- 172.21.15.172:0/1712213743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581fc74e210 con 0x5581fcad84b0 2024-09-10T09:02:05.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.086+0000 7f42454dd640 1 --2- 172.21.15.172:0/1712213743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad84b0 0x5581fcae2340 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f423c0189d0 tx=0x7f423c018a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3ee7a1fab8921b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:05.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.086+0000 7f42297fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:02:05.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.087+0000 7f42297fa640 1 -- 172.21.15.172:0/1712213743 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423c026020 con 0x5581fcad84b0 2024-09-10T09:02:05.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.087+0000 7f42297fa640 1 -- 172.21.15.172:0/1712213743 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f423c004450 con 0x5581fcad84b0 2024-09-10T09:02:05.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.087+0000 7f424acd9740 1 -- 172.21.15.172:0/1712213743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f423c018a80 con 0x5581fcad84b0 2024-09-10T09:02:05.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.087+0000 7f42297fa640 1 -- 172.21.15.172:0/1712213743 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f423c005990 con 0x5581fcad84b0 2024-09-10T09:02:05.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.088+0000 7f4245cde640 1 --2- 172.21.15.172:0/3610768758 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f422c0a2c00 0x7f422c0a50c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5581fcad4060 tx=0x7f4230005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:05.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.090+0000 7f424acd9740 10 client.19647 fetch_fsmap learned FSMap version 44 2024-09-10T09:02:05.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.090+0000 7f424acd9740 1 -- 172.21.15.172:0/1712213743 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5581fc81a620 con 0x5581fcad84b0 2024-09-10T09:02:05.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.092+0000 7f42297fa640 1 -- 172.21.15.172:0/1712213743 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f423c018a80 con 0x5581fcad84b0 2024-09-10T09:02:05.098 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.092+0000 7f42297fa640 1 -- 172.21.15.172:0/1712213743 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f423c015040 con 0x5581fcad84b0 2024-09-10T09:02:05.098 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.092+0000 7f424acd9740 10 client.19647 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:02:05.098 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.092+0000 7f424acd9740 10 client.19647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:02:05.098 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.092+0000 7f424acd9740 20 client.19647 populate_metadata read hostname 'smithi172' 2024-09-10T09:02:05.098 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.092+0000 7f424acd9740 1 --2- 172.21.15.172:0/1712213743 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5581fcaed6b0 0x5581fcaefaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.098 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.092+0000 7f424acd9740 4 client.19647 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:02:05.098 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.092+0000 7f424acd9740 1 -- 172.21.15.172:0/1712213743 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581fca48a20 con 0x5581fcaed6b0 2024-09-10T09:02:05.099 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.095+0000 7f4245cde640 1 --2- 172.21.15.172:0/1712213743 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5581fcaed6b0 0x5581fcaefaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:05.099 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.096+0000 7f4245cde640 1 --2- 172.21.15.172:0/1712213743 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5581fcaed6b0 0x5581fcaefaa0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a60ab6077a412c2d server_cookie=2b20c1303a541769 in_seq=0 out_seq=0 2024-09-10T09:02:05.099 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.096+0000 7f42297fa640 10 client.19647 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:05.101 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.100+0000 7f42297fa640 1 -- 172.21.15.172:0/1712213743 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581fca48a20 con 0x5581fcaed6b0 2024-09-10T09:02:05.101 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.100+0000 7f42297fa640 10 client.19647 handle_command_reply: tid=0 2024-09-10T09:02:05.101 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.101+0000 7f424acd9740 1 client.19647 shutdown 2024-09-10T09:02:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.101+0000 7f424acd9740 1 -- 172.21.15.172:0/1712213743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad84b0 msgr2=0x5581fcae2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:05.102 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.101+0000 7f424acd9740 1 --2- 172.21.15.172:0/1712213743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad84b0 0x5581fcae2340 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f423c0189d0 tx=0x7f423c018a00 comp rx=0 tx=0).stop 2024-09-10T09:02:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.102+0000 7f424acd9740 1 -- 172.21.15.172:0/1712213743 shutdown_connections 2024-09-10T09:02:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.102+0000 7f424acd9740 1 --2- 172.21.15.172:0/1712213743 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5581fcaed6b0 0x5581fcaefaa0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.103 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.102+0000 7f424acd9740 1 --2- 172.21.15.172:0/1712213743 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcadd540 0x5581fcae9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.104 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.102+0000 7f424acd9740 1 --2- 172.21.15.172:0/1712213743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad84b0 0x5581fcae2340 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.104 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.102+0000 7f424acd9740 1 --2- 172.21.15.172:0/1712213743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcad7b70 0x5581fcae1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.104 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.102+0000 7f424acd9740 1 -- 172.21.15.172:0/1712213743 >> 172.21.15.172:0/1712213743 conn(0x5581fcad31e0 msgr2=0x5581fcad49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:05.104 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.102+0000 7f42297fa640 0 client.19647 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.104+0000 7f424acd9740 1 -- 172.21.15.172:0/1712213743 shutdown_connections 2024-09-10T09:02:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.104+0000 7f424acd9740 1 -- 172.21.15.172:0/1712213743 wait complete. 2024-09-10T09:02:05.105 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.105+0000 7f424acd9740 20 client.19647 trim_cache size 0 max 16384 2024-09-10T09:02:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.160+0000 7f424acd9740 1 Processor -- start 2024-09-10T09:02:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.160+0000 7f424acd9740 1 -- start start 2024-09-10T09:02:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.160+0000 7f424acd9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad7b70 0x5581fcad1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.160+0000 7f424acd9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcad84b0 0x5581fcae9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.160+0000 7f424acd9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcadd540 0x5581fcad1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.161+0000 7f424acd9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f422400fbf0 con 0x5581fcad7b70 2024-09-10T09:02:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.161+0000 7f424acd9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4224011730 con 0x5581fcad84b0 2024-09-10T09:02:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.161+0000 7f424acd9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f42240102c0 con 0x5581fcadd540 2024-09-10T09:02:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.161+0000 7f42464df640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcadd540 0x5581fcad1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.167+0000 7f42464df640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcadd540 0x5581fcad1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:50894/0 (socket says 172.21.15.172:50894) 2024-09-10T09:02:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.167+0000 7f42464df640 1 -- 172.21.15.172:0/836632845 learned_addr learned my addr 172.21.15.172:0/836632845 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.168+0000 7f42454dd640 1 --2- 172.21.15.172:0/836632845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcad84b0 0x5581fcae9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.170+0000 7f42464df640 1 -- 172.21.15.172:0/836632845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcad84b0 msgr2=0x5581fcae9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.170+0000 7f42464df640 1 --2- 172.21.15.172:0/836632845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcad84b0 0x5581fcae9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.170+0000 7f42464df640 1 -- 172.21.15.172:0/836632845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad7b70 msgr2=0x5581fcad1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.170+0000 7f42464df640 1 --2- 172.21.15.172:0/836632845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad7b70 0x5581fcad1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.170+0000 7f42464df640 1 -- 172.21.15.172:0/836632845 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581fcad9280 con 0x5581fcadd540 2024-09-10T09:02:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.171+0000 7f42464df640 1 --2- 172.21.15.172:0/836632845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcadd540 0x5581fcad1fd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f42380d6f70 tx=0x7f423803b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b987ad6a8ef86bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.172+0000 7f422b7fe640 1 -- 172.21.15.172:0/836632845 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380e7070 con 0x5581fcadd540 2024-09-10T09:02:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.172+0000 7f422b7fe640 1 -- 172.21.15.172:0/836632845 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4238004040 con 0x5581fcadd540 2024-09-10T09:02:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.172+0000 7f422b7fe640 1 -- 172.21.15.172:0/836632845 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380ffbe0 con 0x5581fcadd540 2024-09-10T09:02:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.178+0000 7f424acd9740 1 -- 172.21.15.172:0/836632845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcadd540 msgr2=0x5581fcad1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.178+0000 7f424acd9740 1 --2- 172.21.15.172:0/836632845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcadd540 0x5581fcad1fd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f42380d6f70 tx=0x7f423803b900 comp rx=0 tx=0).stop 2024-09-10T09:02:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.178+0000 7f424acd9740 1 -- 172.21.15.172:0/836632845 shutdown_connections 2024-09-10T09:02:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.178+0000 7f424acd9740 1 --2- 172.21.15.172:0/836632845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcadd540 0x5581fcad1fd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.178+0000 7f424acd9740 1 --2- 172.21.15.172:0/836632845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcad84b0 0x5581fcae9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.178+0000 7f424acd9740 1 --2- 172.21.15.172:0/836632845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad7b70 0x5581fcad1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.179+0000 7f424acd9740 1 -- 172.21.15.172:0/836632845 >> 172.21.15.172:0/836632845 conn(0x5581fcad31e0 msgr2=0x5581fcae1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.179+0000 7f424acd9740 1 -- 172.21.15.172:0/836632845 shutdown_connections 2024-09-10T09:02:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.179+0000 7f424acd9740 1 -- 172.21.15.172:0/836632845 wait complete. 2024-09-10T09:02:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.180+0000 7f424acd9740 1 -- start start 2024-09-10T09:02:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.181+0000 7f424acd9740 1 Processor -- start 2024-09-10T09:02:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.185+0000 7f424acd9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcad7b70 0x5581fcae61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.185+0000 7f424acd9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad84b0 0x5581fcae66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.185+0000 7f424acd9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcadd540 0x5581fcb562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.185+0000 7f424acd9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4224011060 con 0x5581fcad84b0 2024-09-10T09:02:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.185+0000 7f424acd9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4224010990 con 0x5581fcad7b70 2024-09-10T09:02:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.185+0000 7f424acd9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f422400dbd0 con 0x5581fcadd540 2024-09-10T09:02:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.185+0000 7f424acd9740 10 client.0 _ll_register_callbacks cb 0x7f423c00abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:02:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.186+0000 7f4245cde640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcad7b70 0x5581fcae61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.186+0000 7f4245cde640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcad7b70 0x5581fcae61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:35284/0 (socket says 172.21.15.172:35284) 2024-09-10T09:02:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.186+0000 7f4245cde640 1 -- 172.21.15.172:0/3057830669 learned_addr learned my addr 172.21.15.172:0/3057830669 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.186+0000 7f42464df640 1 --2- 172.21.15.172:0/3057830669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcadd540 0x5581fcb562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.186+0000 7f42464df640 1 -- 172.21.15.172:0/3057830669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcad7b70 msgr2=0x5581fcae61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.186+0000 7f42464df640 1 --2- 172.21.15.172:0/3057830669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcad7b70 0x5581fcae61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.186+0000 7f42464df640 1 -- 172.21.15.172:0/3057830669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad84b0 msgr2=0x5581fcae66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.186+0000 7f42464df640 1 --2- 172.21.15.172:0/3057830669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad84b0 0x5581fcae66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.186+0000 7f42464df640 1 -- 172.21.15.172:0/3057830669 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581fcb56800 con 0x5581fcadd540 2024-09-10T09:02:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.187+0000 7f42464df640 1 --2- 172.21.15.172:0/3057830669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcadd540 0x5581fcb562c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f42380d2da0 tx=0x7f4238003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ae634b8f388427f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.187+0000 7f42297fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:02:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.187+0000 7f42297fa640 1 -- 172.21.15.172:0/3057830669 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380ec060 con 0x5581fcadd540 2024-09-10T09:02:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.187+0000 7f42297fa640 1 -- 172.21.15.172:0/3057830669 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f42380493b0 con 0x5581fcadd540 2024-09-10T09:02:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.187+0000 7f424acd9740 1 -- 172.21.15.172:0/3057830669 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5581fca42d70 con 0x5581fcadd540 2024-09-10T09:02:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.191+0000 7f42297fa640 1 -- 172.21.15.172:0/3057830669 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f42380e7040 con 0x5581fcadd540 2024-09-10T09:02:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.192+0000 7f424acd9740 10 client.28318 fetch_fsmap learned FSMap version 44 2024-09-10T09:02:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.192+0000 7f424acd9740 1 -- 172.21.15.172:0/3057830669 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5581fcb57a10 con 0x5581fcadd540 2024-09-10T09:02:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.199+0000 7f42297fa640 1 -- 172.21.15.172:0/3057830669 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5581fca42d70 con 0x5581fcadd540 2024-09-10T09:02:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.199+0000 7f42297fa640 1 -- 172.21.15.172:0/3057830669 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f42380d7010 con 0x5581fcadd540 2024-09-10T09:02:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.200+0000 7f424acd9740 10 client.28318 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:02:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.200+0000 7f424acd9740 10 client.28318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:02:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.200+0000 7f424acd9740 20 client.28318 populate_metadata read hostname 'smithi172' 2024-09-10T09:02:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.200+0000 7f424acd9740 1 --2- 172.21.15.172:0/3057830669 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5581fcb580b0 0x5581fcb5a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.200+0000 7f424acd9740 4 client.28318 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:02:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.200+0000 7f424acd9740 1 -- 172.21.15.172:0/3057830669 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5581fc9bf900 con 0x5581fcb580b0 2024-09-10T09:02:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.204+0000 7f4245cde640 1 --2- 172.21.15.172:0/3057830669 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5581fcb580b0 0x5581fcb5a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.209+0000 7f4245cde640 1 --2- 172.21.15.172:0/3057830669 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5581fcb580b0 0x5581fcb5a4d0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8508d554f9ba7d93 server_cookie=ab2e227a16fd220b in_seq=0 out_seq=0 2024-09-10T09:02:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.209+0000 7f42297fa640 10 client.28318 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:05.335 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:05 smithi172 ceph-mon[30854]: pgmap v825: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-09-10T09:02:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.350+0000 7f42297fa640 1 -- 172.21.15.172:0/3057830669 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5581fc9bf900 con 0x5581fcb580b0 2024-09-10T09:02:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.350+0000 7f42297fa640 10 client.28318 handle_command_reply: tid=0 2024-09-10T09:02:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.350+0000 7f424acd9740 1 client.28318 shutdown 2024-09-10T09:02:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.354+0000 7f424acd9740 1 -- 172.21.15.172:0/3057830669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcadd540 msgr2=0x5581fcb562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.354+0000 7f424acd9740 1 --2- 172.21.15.172:0/3057830669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcadd540 0x5581fcb562c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f42380d2da0 tx=0x7f4238003c10 comp rx=0 tx=0).stop 2024-09-10T09:02:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.354+0000 7f424acd9740 1 -- 172.21.15.172:0/3057830669 shutdown_connections 2024-09-10T09:02:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.354+0000 7f424acd9740 1 --2- 172.21.15.172:0/3057830669 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5581fcb580b0 0x5581fcb5a4d0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.354+0000 7f424acd9740 1 --2- 172.21.15.172:0/3057830669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5581fcadd540 0x5581fcb562c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.354+0000 7f424acd9740 1 --2- 172.21.15.172:0/3057830669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581fcad84b0 0x5581fcae66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.354+0000 7f424acd9740 1 --2- 172.21.15.172:0/3057830669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581fcad7b70 0x5581fcae61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.354+0000 7f42297fa640 0 client.28318 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.354+0000 7f424acd9740 1 -- 172.21.15.172:0/3057830669 >> 172.21.15.172:0/3057830669 conn(0x5581fcad31e0 msgr2=0x5581fcad4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.356+0000 7f424acd9740 1 -- 172.21.15.172:0/3057830669 shutdown_connections 2024-09-10T09:02:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.356+0000 7f424acd9740 1 -- 172.21.15.172:0/3057830669 wait complete. 2024-09-10T09:02:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.360+0000 7f424acd9740 20 client.28318 trim_cache size 0 max 16384 2024-09-10T09:02:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:02:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:02:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:02:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:02:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.369+0000 7f4246ce0640 1 -- 172.21.15.172:0/3610768758 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f422c0a2c00 msgr2=0x7f422c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.369+0000 7f4246ce0640 1 --2- 172.21.15.172:0/3610768758 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f422c0a2c00 0x7f422c0a50c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5581fcad4060 tx=0x7f4230005d20 comp rx=0 tx=0).stop 2024-09-10T09:02:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.369+0000 7f4246ce0640 1 -- 172.21.15.172:0/3610768758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4240109b30 msgr2=0x7f424007ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.369+0000 7f4246ce0640 1 --2- 172.21.15.172:0/3610768758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4240109b30 0x7f424007ca50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f423801ac00 tx=0x7f4238003ba0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.374+0000 7f4246ce0640 1 -- 172.21.15.172:0/3610768758 shutdown_connections 2024-09-10T09:02:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.374+0000 7f4246ce0640 1 --2- 172.21.15.172:0/3610768758 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f422c0a2c00 0x7f422c0a50c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.374+0000 7f4246ce0640 1 --2- 172.21.15.172:0/3610768758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4240109b30 0x7f424007ca50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.374+0000 7f4246ce0640 1 --2- 172.21.15.172:0/3610768758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4240109170 0x7f4240196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.374+0000 7f4246ce0640 1 --2- 172.21.15.172:0/3610768758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4240108410 0x7f4240195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.374+0000 7f4246ce0640 1 -- 172.21.15.172:0/3610768758 >> 172.21.15.172:0/3610768758 conn(0x7f42400771d0 msgr2=0x7f42400712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.379+0000 7f4246ce0640 1 -- 172.21.15.172:0/3610768758 shutdown_connections 2024-09-10T09:02:05.379 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:05.379+0000 7f4246ce0640 1 -- 172.21.15.172:0/3610768758 wait complete. 2024-09-10T09:02:05.485 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3e265114', total seconds elapsed: 76.7 2024-09-10T09:02:05.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:05 smithi174 ceph-mon[34068]: pgmap v825: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-09-10T09:02:05.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:05 smithi191 ceph-mon[35314]: pgmap v825: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 98 KiB/s wr, 35 op/s 2024-09-10T09:02:05.821 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:02:07.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.467+0000 7f5470237640 1 -- 172.21.15.172:0/2389223032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5468100de0 msgr2=0x7f5468104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:07.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.467+0000 7f5470237640 1 --2- 172.21.15.172:0/2389223032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5468100de0 0x7f5468104e30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5458009900 tx=0x7f545802a440 comp rx=0 tx=0).stop 2024-09-10T09:02:07.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.471+0000 7f5470237640 1 -- 172.21.15.172:0/2389223032 shutdown_connections 2024-09-10T09:02:07.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.471+0000 7f5470237640 1 --2- 172.21.15.172:0/2389223032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5468105560 0x7f546810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.471+0000 7f5470237640 1 --2- 172.21.15.172:0/2389223032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5468100de0 0x7f5468104e30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.471+0000 7f5470237640 1 --2- 172.21.15.172:0/2389223032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468100430 0x7f5468100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.471+0000 7f5470237640 1 -- 172.21.15.172:0/2389223032 >> 172.21.15.172:0/2389223032 conn(0x7f5468077230 msgr2=0x7f54680fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:07.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.474+0000 7f5470237640 1 -- 172.21.15.172:0/2389223032 shutdown_connections 2024-09-10T09:02:07.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.475+0000 7f5470237640 1 -- 172.21.15.172:0/2389223032 wait complete. 2024-09-10T09:02:07.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.476+0000 7f5470237640 1 Processor -- start 2024-09-10T09:02:07.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.476+0000 7f5470237640 1 -- start start 2024-09-10T09:02:07.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.477+0000 7f5470237640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5468100430 0x7f54681961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.477+0000 7f5470237640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468100de0 0x7f5468196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.477+0000 7f5470237640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5468105560 0x7f546819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.477+0000 7f546d7d2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468100de0 0x7f5468196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:07.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.477+0000 7f546d7d2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468100de0 0x7f5468196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:39462/0 (socket says 172.21.15.172:39462) 2024-09-10T09:02:07.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.477+0000 7f546d7d2640 1 -- 172.21.15.172:0/2555949969 learned_addr learned my addr 172.21.15.172:0/2555949969 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:07.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.477+0000 7f5470237640 1 -- 172.21.15.172:0/2555949969 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f546810fc80 con 0x7f5468105560 2024-09-10T09:02:07.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.477+0000 7f5470237640 1 -- 172.21.15.172:0/2555949969 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f546810fb00 con 0x7f5468100de0 2024-09-10T09:02:07.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.478+0000 7f5470237640 1 -- 172.21.15.172:0/2555949969 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f546810fe00 con 0x7f5468100430 2024-09-10T09:02:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.479+0000 7f546dfd3640 1 --2- 172.21.15.172:0/2555949969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5468100430 0x7f54681961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.479+0000 7f546e7d4640 1 --2- 172.21.15.172:0/2555949969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5468105560 0x7f546819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.479+0000 7f546d7d2640 1 -- 172.21.15.172:0/2555949969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5468100430 msgr2=0x7f54681961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.479+0000 7f546d7d2640 1 --2- 172.21.15.172:0/2555949969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5468100430 0x7f54681961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.479+0000 7f546d7d2640 1 -- 172.21.15.172:0/2555949969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5468105560 msgr2=0x7f546819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.479+0000 7f546d7d2640 1 --2- 172.21.15.172:0/2555949969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5468105560 0x7f546819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.479+0000 7f546d7d2640 1 -- 172.21.15.172:0/2555949969 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546819b190 con 0x7f5468100de0 2024-09-10T09:02:07.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.480+0000 7f546d7d2640 1 --2- 172.21.15.172:0/2555949969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468100de0 0x7f5468196720 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5458000c00 tx=0x7f5458003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:07.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.480+0000 7f545effd640 1 -- 172.21.15.172:0/2555949969 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458040030 con 0x7f5468100de0 2024-09-10T09:02:07.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.481+0000 7f545effd640 1 -- 172.21.15.172:0/2555949969 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5458004210 con 0x7f5468100de0 2024-09-10T09:02:07.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.481+0000 7f545effd640 1 -- 172.21.15.172:0/2555949969 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458031730 con 0x7f5468100de0 2024-09-10T09:02:07.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.482+0000 7f5470237640 1 -- 172.21.15.172:0/2555949969 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546819b420 con 0x7f5468100de0 2024-09-10T09:02:07.485 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.482+0000 7f5470237640 1 -- 172.21.15.172:0/2555949969 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f546819b7c0 con 0x7f5468100de0 2024-09-10T09:02:07.488 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:07 smithi172 ceph-mon[30854]: pgmap v826: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 230 KiB/s wr, 71 op/s 2024-09-10T09:02:07.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.487+0000 7f545effd640 1 -- 172.21.15.172:0/2555949969 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5458043030 con 0x7f5468100de0 2024-09-10T09:02:07.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:07 smithi174 ceph-mon[34068]: pgmap v826: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 230 KiB/s wr, 71 op/s 2024-09-10T09:02:07.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.489+0000 7f5472610740 1 Processor -- start 2024-09-10T09:02:07.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.489+0000 7f5472610740 1 -- start start 2024-09-10T09:02:07.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.490+0000 7f5472610740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919464a80 0x561919464e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.490+0000 7f5472610740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619194653a0 0x561919478010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.490+0000 7f546e7d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919464a80 0x561919464e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:07.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.491+0000 7f546e7d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919464a80 0x561919464e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:46608/0 (socket says 172.21.15.172:46608) 2024-09-10T09:02:07.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.491+0000 7f546e7d4640 1 -- 172.21.15.172:0/1220743647 learned_addr learned my addr 172.21.15.172:0/1220743647 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:07.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.491+0000 7f5472610740 1 --2- 172.21.15.172:0/1220743647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919478550 0x56191947a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.491+0000 7f5472610740 1 -- 172.21.15.172:0/1220743647 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5619193ca280 con 0x5619194653a0 2024-09-10T09:02:07.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.491+0000 7f5472610740 1 -- 172.21.15.172:0/1220743647 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561919371b10 con 0x561919478550 2024-09-10T09:02:07.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.491+0000 7f5472610740 1 -- 172.21.15.172:0/1220743647 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5619193c6240 con 0x561919464a80 2024-09-10T09:02:07.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.491+0000 7f546d7d2640 1 --2- 172.21.15.172:0/1220743647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919478550 0x56191947a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:07.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.492+0000 7f546e7d4640 1 -- 172.21.15.172:0/1220743647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919478550 msgr2=0x56191947a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:07.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.492+0000 7f546e7d4640 1 --2- 172.21.15.172:0/1220743647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919478550 0x56191947a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.492+0000 7f546e7d4640 1 -- 172.21.15.172:0/1220743647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619194653a0 msgr2=0x561919478010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:07.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.492+0000 7f546dfd3640 1 --2- 172.21.15.172:0/1220743647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619194653a0 0x561919478010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:07.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.493+0000 7f546e7d4640 1 --2- 172.21.15.172:0/1220743647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619194653a0 0x561919478010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.493+0000 7f546e7d4640 1 -- 172.21.15.172:0/1220743647 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56191937e1f0 con 0x561919464a80 2024-09-10T09:02:07.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.493+0000 7f546e7d4640 1 --2- 172.21.15.172:0/1220743647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919464a80 0x561919464e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5460009f40 tx=0x7f546000c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b9e52e38030cb6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:07.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.494+0000 7f546dfd3640 1 --2- 172.21.15.172:0/1220743647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619194653a0 0x561919478010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:02:07.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.494+0000 7f545effd640 1 --2- 172.21.15.172:0/2555949969 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f54540a2c60 0x7f54540a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.494+0000 7f545effd640 1 -- 172.21.15.172:0/2555949969 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f54580efc50 con 0x7f5468100de0 2024-09-10T09:02:07.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.494+0000 7f5453fff640 1 -- 172.21.15.172:0/1220743647 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546001a070 con 0x561919464a80 2024-09-10T09:02:07.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.494+0000 7f5453fff640 1 -- 172.21.15.172:0/1220743647 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f546000aaa0 con 0x561919464a80 2024-09-10T09:02:07.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.495+0000 7f5472610740 1 -- 172.21.15.172:0/1220743647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919464a80 msgr2=0x561919464e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:07.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.495+0000 7f5472610740 1 --2- 172.21.15.172:0/1220743647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919464a80 0x561919464e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5460009f40 tx=0x7f546000c890 comp rx=0 tx=0).stop 2024-09-10T09:02:07.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.497+0000 7f546dfd3640 1 --2- 172.21.15.172:0/2555949969 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f54540a2c60 0x7f54540a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:07.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.499+0000 7f5472610740 1 -- 172.21.15.172:0/1220743647 shutdown_connections 2024-09-10T09:02:07.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.499+0000 7f5472610740 1 --2- 172.21.15.172:0/1220743647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919478550 0x56191947a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.499+0000 7f5472610740 1 --2- 172.21.15.172:0/1220743647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619194653a0 0x561919478010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.499+0000 7f5472610740 1 --2- 172.21.15.172:0/1220743647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919464a80 0x561919464e60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.499+0000 7f5472610740 1 -- 172.21.15.172:0/1220743647 >> 172.21.15.172:0/1220743647 conn(0x5619194610f0 msgr2=0x5619194614d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:07.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.502+0000 7f5472610740 1 -- 172.21.15.172:0/1220743647 shutdown_connections 2024-09-10T09:02:07.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.502+0000 7f5472610740 1 -- 172.21.15.172:0/1220743647 wait complete. 2024-09-10T09:02:07.509 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.503+0000 7f5472610740 1 -- start start 2024-09-10T09:02:07.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.503+0000 7f5472610740 1 Processor -- start 2024-09-10T09:02:07.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.503+0000 7f546dfd3640 1 --2- 172.21.15.172:0/2555949969 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f54540a2c60 0x7f54540a5120 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5464009ce0 tx=0x7f5464005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:07.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.504+0000 7f5472610740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561919464a80 0x561919475450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.504+0000 7f5472610740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5619194653a0 0x561919475990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.504+0000 7f5472610740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919478550 0x561919475ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.504+0000 7f5472610740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56191947b110 con 0x561919464a80 2024-09-10T09:02:07.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.504+0000 7f5472610740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561919384f10 con 0x561919478550 2024-09-10T09:02:07.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.504+0000 7f5472610740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56191947b290 con 0x5619194653a0 2024-09-10T09:02:07.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.504+0000 7f5472610740 10 client.0 _ll_register_callbacks cb 0x56191934da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:02:07.511 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.504+0000 7f546d7d2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5619194653a0 0x561919475990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:07.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.504+0000 7f546d7d2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5619194653a0 0x561919475990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:46610/0 (socket says 172.21.15.172:46610) 2024-09-10T09:02:07.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.504+0000 7f546d7d2640 1 -- 172.21.15.172:0/376546183 learned_addr learned my addr 172.21.15.172:0/376546183 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:07.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.504+0000 7f546e7d4640 1 --2- 172.21.15.172:0/376546183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919478550 0x561919475ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:07.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.505+0000 7f546e7d4640 1 -- 172.21.15.172:0/376546183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5619194653a0 msgr2=0x561919475990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:07.512 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.505+0000 7f546e7d4640 1 --2- 172.21.15.172:0/376546183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5619194653a0 0x561919475990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.505+0000 7f546e7d4640 1 -- 172.21.15.172:0/376546183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561919464a80 msgr2=0x561919475450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:07.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.505+0000 7f546e7d4640 1 --2- 172.21.15.172:0/376546183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561919464a80 0x561919475450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.505+0000 7f546e7d4640 1 -- 172.21.15.172:0/376546183 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56191913b6a0 con 0x561919478550 2024-09-10T09:02:07.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.505+0000 7f546e7d4640 1 --2- 172.21.15.172:0/376546183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919478550 0x561919475ed0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5460009f10 tx=0x7f5460015480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dc67201e7b4bd3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:07.513 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.505+0000 7f54517fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:02:07.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.506+0000 7f54517fa640 1 -- 172.21.15.172:0/376546183 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460024020 con 0x561919478550 2024-09-10T09:02:07.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.506+0000 7f5472610740 1 -- 172.21.15.172:0/376546183 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f546000aaa0 con 0x561919478550 2024-09-10T09:02:07.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.506+0000 7f54517fa640 1 -- 172.21.15.172:0/376546183 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f546000a5f0 con 0x561919478550 2024-09-10T09:02:07.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.506+0000 7f54517fa640 1 -- 172.21.15.172:0/376546183 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f546001a040 con 0x561919478550 2024-09-10T09:02:07.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.507+0000 7f5472610740 10 client.28424 fetch_fsmap learned FSMap version 44 2024-09-10T09:02:07.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.507+0000 7f5472610740 1 -- 172.21.15.172:0/376546183 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561919469ed0 con 0x561919478550 2024-09-10T09:02:07.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.508+0000 7f54517fa640 1 -- 172.21.15.172:0/376546183 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f546000aaa0 con 0x561919478550 2024-09-10T09:02:07.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.508+0000 7f54517fa640 1 -- 172.21.15.172:0/376546183 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f54600043b0 con 0x561919478550 2024-09-10T09:02:07.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.509+0000 7f5472610740 10 client.28424 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:02:07.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.509+0000 7f5472610740 10 client.28424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:02:07.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.509+0000 7f5472610740 20 client.28424 populate_metadata read hostname 'smithi172' 2024-09-10T09:02:07.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.509+0000 7f5472610740 1 --2- 172.21.15.172:0/376546183 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56191946b600 0x56191946da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.509+0000 7f5472610740 4 client.28424 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:02:07.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.509+0000 7f5472610740 1 -- 172.21.15.172:0/376546183 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619193d77f0 con 0x56191946b600 2024-09-10T09:02:07.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.509+0000 7f546dfd3640 1 --2- 172.21.15.172:0/376546183 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56191946b600 0x56191946da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:07.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.510+0000 7f546dfd3640 1 --2- 172.21.15.172:0/376546183 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56191946b600 0x56191946da50 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c72a2428a2b289fd server_cookie=82b221c6eec8a87b in_seq=0 out_seq=0 2024-09-10T09:02:07.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.510+0000 7f54517fa640 10 client.28424 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:07.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.511+0000 7f54517fa640 1 -- 172.21.15.172:0/376546183 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619193d77f0 con 0x56191946b600 2024-09-10T09:02:07.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.511+0000 7f54517fa640 10 client.28424 handle_command_reply: tid=0 2024-09-10T09:02:07.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.511+0000 7f5472610740 1 client.28424 shutdown 2024-09-10T09:02:07.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.514+0000 7f5472610740 1 -- 172.21.15.172:0/376546183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919478550 msgr2=0x561919475ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:07.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.514+0000 7f5472610740 1 --2- 172.21.15.172:0/376546183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919478550 0x561919475ed0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5460009f10 tx=0x7f5460015480 comp rx=0 tx=0).stop 2024-09-10T09:02:07.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.514+0000 7f5472610740 1 -- 172.21.15.172:0/376546183 shutdown_connections 2024-09-10T09:02:07.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.514+0000 7f5472610740 1 --2- 172.21.15.172:0/376546183 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56191946b600 0x56191946da50 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.514+0000 7f5472610740 1 --2- 172.21.15.172:0/376546183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919478550 0x561919475ed0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.514+0000 7f5472610740 1 --2- 172.21.15.172:0/376546183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5619194653a0 0x561919475990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.514+0000 7f5472610740 1 --2- 172.21.15.172:0/376546183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561919464a80 0x561919475450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.514+0000 7f54517fa640 0 client.28424 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:07.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.514+0000 7f5472610740 1 -- 172.21.15.172:0/376546183 >> 172.21.15.172:0/376546183 conn(0x5619194610f0 msgr2=0x561919466d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:07.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.514+0000 7f5472610740 1 -- 172.21.15.172:0/376546183 shutdown_connections 2024-09-10T09:02:07.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.514+0000 7f5472610740 1 -- 172.21.15.172:0/376546183 wait complete. 2024-09-10T09:02:07.523 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.518+0000 7f5472610740 20 client.28424 trim_cache size 0 max 16384 2024-09-10T09:02:07.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:07 smithi191 ceph-mon[35314]: pgmap v826: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 230 KiB/s wr, 71 op/s 2024-09-10T09:02:07.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.565+0000 7f5472610740 1 Processor -- start 2024-09-10T09:02:07.567 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.565+0000 7f5472610740 1 -- start start 2024-09-10T09:02:07.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.566+0000 7f5472610740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919464a80 0x56191949a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.566+0000 7f5472610740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619194653a0 0x56191949ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.566+0000 7f546dfd3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919464a80 0x56191949a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:07.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.566+0000 7f546dfd3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919464a80 0x56191949a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:39480/0 (socket says 172.21.15.172:39480) 2024-09-10T09:02:07.570 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.566+0000 7f546dfd3640 1 -- 172.21.15.172:0/2895990004 learned_addr learned my addr 172.21.15.172:0/2895990004 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:07.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.566+0000 7f5472610740 1 --2- 172.21.15.172:0/2895990004 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919478550 0x5619194726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.566+0000 7f5472610740 1 -- 172.21.15.172:0/2895990004 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f54681ac970 con 0x5619194653a0 2024-09-10T09:02:07.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.566+0000 7f5472610740 1 -- 172.21.15.172:0/2895990004 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54681ae4b0 con 0x561919464a80 2024-09-10T09:02:07.571 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.566+0000 7f5472610740 1 -- 172.21.15.172:0/2895990004 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f54681ad040 con 0x561919478550 2024-09-10T09:02:07.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.568+0000 7f546e7d4640 1 --2- 172.21.15.172:0/2895990004 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919478550 0x5619194726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:07.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.570+0000 7f546dfd3640 1 -- 172.21.15.172:0/2895990004 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919478550 msgr2=0x5619194726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:07.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.570+0000 7f546dfd3640 1 --2- 172.21.15.172:0/2895990004 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919478550 0x5619194726b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.570+0000 7f546dfd3640 1 -- 172.21.15.172:0/2895990004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619194653a0 msgr2=0x56191949ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:07.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.570+0000 7f546dfd3640 1 --2- 172.21.15.172:0/2895990004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619194653a0 0x56191949ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.570+0000 7f546dfd3640 1 -- 172.21.15.172:0/2895990004 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56191947ac80 con 0x561919464a80 2024-09-10T09:02:07.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.574+0000 7f546dfd3640 1 --2- 172.21.15.172:0/2895990004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919464a80 0x56191949a650 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x56191946e290 tx=0x7f5464016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e0bfa01d065ab0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:07.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.576+0000 7f54537fe640 1 -- 172.21.15.172:0/2895990004 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54640044c0 con 0x561919464a80 2024-09-10T09:02:07.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.577+0000 7f54537fe640 1 -- 172.21.15.172:0/2895990004 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f546402d070 con 0x561919464a80 2024-09-10T09:02:07.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.578+0000 7f5472610740 1 -- 172.21.15.172:0/2895990004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919464a80 msgr2=0x56191949a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:07.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.578+0000 7f5472610740 1 --2- 172.21.15.172:0/2895990004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919464a80 0x56191949a650 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x56191946e290 tx=0x7f5464016000 comp rx=0 tx=0).stop 2024-09-10T09:02:07.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.587+0000 7f5472610740 1 -- 172.21.15.172:0/2895990004 shutdown_connections 2024-09-10T09:02:07.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.587+0000 7f5472610740 1 --2- 172.21.15.172:0/2895990004 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561919478550 0x5619194726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.587+0000 7f5472610740 1 --2- 172.21.15.172:0/2895990004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619194653a0 0x56191949ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.587+0000 7f5472610740 1 --2- 172.21.15.172:0/2895990004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919464a80 0x56191949a650 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.587+0000 7f5472610740 1 -- 172.21.15.172:0/2895990004 >> 172.21.15.172:0/2895990004 conn(0x5619194610f0 msgr2=0x5619194773c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:07.590 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.589+0000 7f5472610740 1 -- 172.21.15.172:0/2895990004 shutdown_connections 2024-09-10T09:02:07.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.590+0000 7f5472610740 1 -- 172.21.15.172:0/2895990004 wait complete. 2024-09-10T09:02:07.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.590+0000 7f5472610740 1 -- start start 2024-09-10T09:02:07.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.590+0000 7f5472610740 1 Processor -- start 2024-09-10T09:02:07.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.591+0000 7f5472610740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919464a80 0x56191947c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.591+0000 7f5472610740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5619194653a0 0x56191947cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.591+0000 7f5472610740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561919478550 0x56191947d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.591+0000 7f5472610740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f54681adde0 con 0x561919478550 2024-09-10T09:02:07.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.591+0000 7f5472610740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54681ad710 con 0x561919464a80 2024-09-10T09:02:07.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.591+0000 7f5472610740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f54681aa950 con 0x5619194653a0 2024-09-10T09:02:07.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.591+0000 7f5472610740 10 client.0 _ll_register_callbacks cb 0x7f5460018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:02:07.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.592+0000 7f546d7d2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5619194653a0 0x56191947cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:07.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.592+0000 7f546d7d2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5619194653a0 0x56191947cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:46620/0 (socket says 172.21.15.172:46620) 2024-09-10T09:02:07.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.592+0000 7f546d7d2640 1 -- 172.21.15.172:0/3820802056 learned_addr learned my addr 172.21.15.172:0/3820802056 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:07.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.594+0000 7f546d7d2640 1 -- 172.21.15.172:0/3820802056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919464a80 msgr2=0x56191947c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:07.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.594+0000 7f546d7d2640 1 --2- 172.21.15.172:0/3820802056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919464a80 0x56191947c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.594+0000 7f546d7d2640 1 -- 172.21.15.172:0/3820802056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561919478550 msgr2=0x56191947d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:07.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.594+0000 7f546d7d2640 1 --2- 172.21.15.172:0/3820802056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561919478550 0x56191947d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.594+0000 7f546d7d2640 1 -- 172.21.15.172:0/3820802056 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56191950b910 con 0x5619194653a0 2024-09-10T09:02:07.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.594+0000 7f546d7d2640 1 --2- 172.21.15.172:0/3820802056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5619194653a0 0x56191947cb10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f545802aa30 tx=0x7f54580f36b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bbf470086a17e69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:07.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.595+0000 7f54517fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:02:07.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.595+0000 7f54517fa640 1 -- 172.21.15.172:0/3820802056 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54580bbc90 con 0x5619194653a0 2024-09-10T09:02:07.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.595+0000 7f5472610740 1 -- 172.21.15.172:0/3820802056 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f546402d070 con 0x5619194653a0 2024-09-10T09:02:07.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.595+0000 7f54517fa640 1 -- 172.21.15.172:0/3820802056 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f545803cb10 con 0x5619194653a0 2024-09-10T09:02:07.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.595+0000 7f54517fa640 1 -- 172.21.15.172:0/3820802056 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f54580bbab0 con 0x5619194653a0 2024-09-10T09:02:07.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.596+0000 7f5472610740 10 client.28336 fetch_fsmap learned FSMap version 44 2024-09-10T09:02:07.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.596+0000 7f5472610740 1 -- 172.21.15.172:0/3820802056 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56191950cd00 con 0x5619194653a0 2024-09-10T09:02:07.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.598+0000 7f54517fa640 1 -- 172.21.15.172:0/3820802056 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f546402d070 con 0x5619194653a0 2024-09-10T09:02:07.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.598+0000 7f54517fa640 1 -- 172.21.15.172:0/3820802056 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f545804d060 con 0x5619194653a0 2024-09-10T09:02:07.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.600+0000 7f5472610740 10 client.28336 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:02:07.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.600+0000 7f5472610740 10 client.28336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:02:07.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.600+0000 7f5472610740 20 client.28336 populate_metadata read hostname 'smithi172' 2024-09-10T09:02:07.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.600+0000 7f5472610740 1 --2- 172.21.15.172:0/3820802056 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56191950d390 0x56191950f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:07.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.600+0000 7f5472610740 4 client.28336 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:02:07.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.600+0000 7f5472610740 1 -- 172.21.15.172:0/3820802056 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5619193d1d00 con 0x56191950d390 2024-09-10T09:02:07.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.606+0000 7f546dfd3640 1 --2- 172.21.15.172:0/3820802056 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56191950d390 0x56191950f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:07.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.619+0000 7f54517fa640 10 client.28336 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:07.622 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.619+0000 7f546dfd3640 1 --2- 172.21.15.172:0/3820802056 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56191950d390 0x56191950f7b0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c29e334cb5fe51bb server_cookie=41d980b955bf2a9e in_seq=0 out_seq=0 2024-09-10T09:02:07.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.623+0000 7f54517fa640 1 -- 172.21.15.172:0/3820802056 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5619193d1d00 con 0x56191950d390 2024-09-10T09:02:07.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.623+0000 7f54517fa640 10 client.28336 handle_command_reply: tid=0 2024-09-10T09:02:07.624 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.623+0000 7f5472610740 1 client.28336 shutdown 2024-09-10T09:02:07.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.626+0000 7f5472610740 1 -- 172.21.15.172:0/3820802056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5619194653a0 msgr2=0x56191947cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:07.627 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.626+0000 7f5472610740 1 --2- 172.21.15.172:0/3820802056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5619194653a0 0x56191947cb10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f545802aa30 tx=0x7f54580f36b0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.630+0000 7f5472610740 1 -- 172.21.15.172:0/3820802056 shutdown_connections 2024-09-10T09:02:07.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.630+0000 7f5472610740 1 --2- 172.21.15.172:0/3820802056 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56191950d390 0x56191950f7b0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.630+0000 7f5472610740 1 --2- 172.21.15.172:0/3820802056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561919478550 0x56191947d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.630+0000 7f5472610740 1 --2- 172.21.15.172:0/3820802056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5619194653a0 0x56191947cb10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.630+0000 7f5472610740 1 --2- 172.21.15.172:0/3820802056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561919464a80 0x56191947c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.631 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.630+0000 7f5472610740 1 -- 172.21.15.172:0/3820802056 >> 172.21.15.172:0/3820802056 conn(0x5619194610f0 msgr2=0x5619194833f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:07.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.631+0000 7f54517fa640 0 client.28336 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:07.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.631+0000 7f5472610740 1 -- 172.21.15.172:0/3820802056 shutdown_connections 2024-09-10T09:02:07.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.631+0000 7f5472610740 1 -- 172.21.15.172:0/3820802056 wait complete. 2024-09-10T09:02:07.632 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.631+0000 7f5472610740 20 client.28336 trim_cache size 0 max 16384 2024-09-10T09:02:07.634 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:02:07.635 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T09:02:07.635 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "097ab02a-d7be-4e1d-96ec-78f5d8c3836d", 2024-09-10T09:02:07.635 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T09:02:07.635 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:02:07.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.634+0000 7f5470237640 1 -- 172.21.15.172:0/2555949969 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f54540a2c60 msgr2=0x7f54540a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:07.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.634+0000 7f5470237640 1 --2- 172.21.15.172:0/2555949969 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f54540a2c60 0x7f54540a5120 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5464009ce0 tx=0x7f5464005f90 comp rx=0 tx=0).stop 2024-09-10T09:02:07.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.634+0000 7f5470237640 1 -- 172.21.15.172:0/2555949969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468100de0 msgr2=0x7f5468196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:07.636 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.634+0000 7f5470237640 1 --2- 172.21.15.172:0/2555949969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468100de0 0x7f5468196720 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5458000c00 tx=0x7f5458003e90 comp rx=0 tx=0).stop 2024-09-10T09:02:07.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.637+0000 7f5470237640 1 -- 172.21.15.172:0/2555949969 shutdown_connections 2024-09-10T09:02:07.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.637+0000 7f5470237640 1 --2- 172.21.15.172:0/2555949969 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f54540a2c60 0x7f54540a5120 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.637+0000 7f5470237640 1 --2- 172.21.15.172:0/2555949969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5468105560 0x7f546819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.637+0000 7f5470237640 1 --2- 172.21.15.172:0/2555949969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468100de0 0x7f5468196720 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.637+0000 7f5470237640 1 --2- 172.21.15.172:0/2555949969 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5468100430 0x7f54681961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:07.640 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.637+0000 7f5470237640 1 -- 172.21.15.172:0/2555949969 >> 172.21.15.172:0/2555949969 conn(0x7f5468077230 msgr2=0x7f54681037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:07.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.639+0000 7f5470237640 1 -- 172.21.15.172:0/2555949969 shutdown_connections 2024-09-10T09:02:07.641 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:07.639+0000 7f5470237640 1 -- 172.21.15.172:0/2555949969 wait complete. 2024-09-10T09:02:08.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:08 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T09:02:08.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:08 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T09:02:08.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:08 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T09:02:08.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:08 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T09:02:08.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:08 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T09:02:08.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:08 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T09:02:09.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:09 smithi191 ceph-mon[35314]: pgmap v827: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 605 KiB/s wr, 159 op/s 2024-09-10T09:02:09.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:09 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T09:02:09.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:09 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:02:09.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:09 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T09:02:09.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:09 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T09:02:09.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:09 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T09:02:09.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:09 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:02:09.551 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:09 smithi172 ceph-mon[30854]: pgmap v827: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 605 KiB/s wr, 159 op/s 2024-09-10T09:02:09.551 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:09 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T09:02:09.552 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:09 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:02:09.552 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:09 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T09:02:09.552 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:09 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T09:02:09.552 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:09 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T09:02:09.552 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:09 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:02:09.645 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:02:09.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:09 smithi174 ceph-mon[34068]: pgmap v827: 129 pgs: 129 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 605 KiB/s wr, 159 op/s 2024-09-10T09:02:09.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:09 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T09:02:09.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:09 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:02:09.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:09 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T09:02:09.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:09 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T09:02:09.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:09 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T09:02:09.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:09 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:02:10.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:10 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:02:10.557 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:02:10.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:10 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:02:10.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:10 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:02:11.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.289+0000 7fbe74cfc640 1 -- 172.21.15.172:0/3133489445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe70102570 msgr2=0x7fbe701029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:11.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.289+0000 7fbe74cfc640 1 --2- 172.21.15.172:0/3133489445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe70102570 0x7fbe701029b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009890 tx=0x7fbe5c02a440 comp rx=0 tx=0).stop 2024-09-10T09:02:11.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.294+0000 7fbe74cfc640 1 -- 172.21.15.172:0/3133489445 shutdown_connections 2024-09-10T09:02:11.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.294+0000 7fbe74cfc640 1 --2- 172.21.15.172:0/3133489445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe70102ef0 0x7fbe7010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.294+0000 7fbe74cfc640 1 --2- 172.21.15.172:0/3133489445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe70102570 0x7fbe701029b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.294+0000 7fbe74cfc640 1 --2- 172.21.15.172:0/3133489445 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe70100e20 0x7fbe70101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.294+0000 7fbe74cfc640 1 -- 172.21.15.172:0/3133489445 >> 172.21.15.172:0/3133489445 conn(0x7fbe700fc7d0 msgr2=0x7fbe700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:11.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.301+0000 7fbe74cfc640 1 -- 172.21.15.172:0/3133489445 shutdown_connections 2024-09-10T09:02:11.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.302+0000 7fbe74cfc640 1 -- 172.21.15.172:0/3133489445 wait complete. 2024-09-10T09:02:11.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.311+0000 7fbe74cfc640 1 Processor -- start 2024-09-10T09:02:11.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.311+0000 7fbe74cfc640 1 -- start start 2024-09-10T09:02:11.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.311+0000 7fbe74cfc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe70100e20 0x7fbe701962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.311+0000 7fbe74cfc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe70102570 0x7fbe70196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.311+0000 7fbe74cfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe70102ef0 0x7fbe7019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.311+0000 7fbe74cfc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7010fdb0 con 0x7fbe70102ef0 2024-09-10T09:02:11.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.311+0000 7fbe74cfc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7010fc30 con 0x7fbe70102570 2024-09-10T09:02:11.312 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.311+0000 7fbe74cfc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7010ff30 con 0x7fbe70100e20 2024-09-10T09:02:11.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.313+0000 7fbe6ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe70102ef0 0x7fbe7019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:11.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.313+0000 7fbe6ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe70102ef0 0x7fbe7019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36098/0 (socket says 172.21.15.172:36098) 2024-09-10T09:02:11.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.313+0000 7fbe6ed9d640 1 -- 172.21.15.172:0/847662468 learned_addr learned my addr 172.21.15.172:0/847662468 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:11.314 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.313+0000 7fbe6e59c640 1 --2- 172.21.15.172:0/847662468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe70100e20 0x7fbe701962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:11.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.314+0000 7fbe6dd9b640 1 --2- 172.21.15.172:0/847662468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe70102570 0x7fbe70196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:11.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.314+0000 7fbe6ed9d640 1 -- 172.21.15.172:0/847662468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe70100e20 msgr2=0x7fbe701962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:11.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.314+0000 7fbe6ed9d640 1 --2- 172.21.15.172:0/847662468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe70100e20 0x7fbe701962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.314+0000 7fbe6ed9d640 1 -- 172.21.15.172:0/847662468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe70102570 msgr2=0x7fbe70196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:11.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.314+0000 7fbe6ed9d640 1 --2- 172.21.15.172:0/847662468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe70102570 0x7fbe70196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.318 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.314+0000 7fbe6ed9d640 1 -- 172.21.15.172:0/847662468 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7019b280 con 0x7fbe70102ef0 2024-09-10T09:02:11.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.315+0000 7fbe6ed9d640 1 --2- 172.21.15.172:0/847662468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe70102ef0 0x7fbe7019aba0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009fd0 tx=0x7fbe6800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:11.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.317+0000 7fbe677fe640 1 -- 172.21.15.172:0/847662468 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68004b20 con 0x7fbe70102ef0 2024-09-10T09:02:11.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.318+0000 7fbe74cfc640 1 -- 172.21.15.172:0/847662468 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7019b570 con 0x7fbe70102ef0 2024-09-10T09:02:11.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.318+0000 7fbe74cfc640 1 -- 172.21.15.172:0/847662468 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe701a2e50 con 0x7fbe70102ef0 2024-09-10T09:02:11.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.318+0000 7fbe677fe640 1 -- 172.21.15.172:0/847662468 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe68003e20 con 0x7fbe70102ef0 2024-09-10T09:02:11.319 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.318+0000 7fbe677fe640 1 -- 172.21.15.172:0/847662468 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68004820 con 0x7fbe70102ef0 2024-09-10T09:02:11.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.337+0000 7fbe770d5740 1 Processor -- start 2024-09-10T09:02:11.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.337+0000 7fbe770d5740 1 -- start start 2024-09-10T09:02:11.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.337+0000 7fbe770d5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39ab70 0x55fdff39af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.337+0000 7fbe770d5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff39b490 0x55fdff396460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.337+0000 7fbe770d5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff3ae900 0x55fdff3b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.337+0000 7fbe770d5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fdff2f82e0 con 0x55fdff39b490 2024-09-10T09:02:11.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.337+0000 7fbe770d5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fdff29fce0 con 0x55fdff39ab70 2024-09-10T09:02:11.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.337+0000 7fbe770d5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55fdff2f42a0 con 0x55fdff3ae900 2024-09-10T09:02:11.339 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.338+0000 7fbe6dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39ab70 0x55fdff39af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:11.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.338+0000 7fbe6dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39ab70 0x55fdff39af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:39516/0 (socket says 172.21.15.172:39516) 2024-09-10T09:02:11.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.338+0000 7fbe6dd9b640 1 -- 172.21.15.172:0/2846707929 learned_addr learned my addr 172.21.15.172:0/2846707929 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:11.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.338+0000 7fbe6dd9b640 1 -- 172.21.15.172:0/2846707929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff3ae900 msgr2=0x55fdff3b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:11.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.338+0000 7fbe6dd9b640 1 --2- 172.21.15.172:0/2846707929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff3ae900 0x55fdff3b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.338+0000 7fbe6dd9b640 1 -- 172.21.15.172:0/2846707929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff39b490 msgr2=0x55fdff396460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:02:11.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.338+0000 7fbe6dd9b640 1 --2- 172.21.15.172:0/2846707929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff39b490 0x55fdff396460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.338+0000 7fbe6dd9b640 1 -- 172.21.15.172:0/2846707929 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdff284440 con 0x55fdff39ab70 2024-09-10T09:02:11.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.339+0000 7fbe677fe640 1 -- 172.21.15.172:0/847662468 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbe6801d020 con 0x7fbe70102ef0 2024-09-10T09:02:11.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.340+0000 7fbe677fe640 1 --2- 172.21.15.172:0/847662468 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbe540a2d90 0x7fbe540a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.340+0000 7fbe677fe640 1 -- 172.21.15.172:0/847662468 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fbe680cf2a0 con 0x7fbe70102ef0 2024-09-10T09:02:11.344 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.343+0000 7fbe6dd9b640 1 --2- 172.21.15.172:0/2846707929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39ab70 0x55fdff39af50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009890 tx=0x7fbe5c009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a72968f482d5df3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:11.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.343+0000 7fbe64ff9640 1 -- 172.21.15.172:0/2846707929 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c043020 con 0x55fdff39ab70 2024-09-10T09:02:11.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.343+0000 7fbe64ff9640 1 -- 172.21.15.172:0/2846707929 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe5c030950 con 0x55fdff39ab70 2024-09-10T09:02:11.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.344+0000 7fbe770d5740 1 -- 172.21.15.172:0/2846707929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39ab70 msgr2=0x55fdff39af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:11.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.344+0000 7fbe770d5740 1 --2- 172.21.15.172:0/2846707929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39ab70 0x55fdff39af50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009890 tx=0x7fbe5c009540 comp rx=0 tx=0).stop 2024-09-10T09:02:11.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.347+0000 7fbe770d5740 1 -- 172.21.15.172:0/2846707929 shutdown_connections 2024-09-10T09:02:11.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.347+0000 7fbe770d5740 1 --2- 172.21.15.172:0/2846707929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff3ae900 0x55fdff3b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.347+0000 7fbe770d5740 1 --2- 172.21.15.172:0/2846707929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff39b490 0x55fdff396460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.347+0000 7fbe770d5740 1 --2- 172.21.15.172:0/2846707929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39ab70 0x55fdff39af50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.347+0000 7fbe770d5740 1 -- 172.21.15.172:0/2846707929 >> 172.21.15.172:0/2846707929 conn(0x55fdff397240 msgr2=0x55fdff397620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:11.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.348+0000 7fbe6e59c640 1 --2- 172.21.15.172:0/847662468 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbe540a2d90 0x7fbe540a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:11.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.349+0000 7fbe770d5740 1 -- 172.21.15.172:0/2846707929 shutdown_connections 2024-09-10T09:02:11.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.349+0000 7fbe6e59c640 1 --2- 172.21.15.172:0/847662468 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbe540a2d90 0x7fbe540a5250 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000a680 tx=0x7fbe60008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:11.354 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:11 smithi172 ceph-mon[30854]: pgmap v828: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.6 MiB/s wr, 933 op/s 2024-09-10T09:02:11.354 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:11 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:02:11.354 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.351+0000 7fbe770d5740 1 -- 172.21.15.172:0/2846707929 wait complete. 2024-09-10T09:02:11.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.351+0000 7fbe770d5740 1 -- start start 2024-09-10T09:02:11.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.351+0000 7fbe770d5740 1 Processor -- start 2024-09-10T09:02:11.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.353+0000 7fbe770d5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff39ab70 0x55fdff396240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.353+0000 7fbe770d5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39b490 0x55fdff3934f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.353+0000 7fbe770d5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff3ae900 0x55fdff393a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.353+0000 7fbe770d5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fdff3b1290 con 0x55fdff39ab70 2024-09-10T09:02:11.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.353+0000 7fbe770d5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fdff2b3050 con 0x55fdff39b490 2024-09-10T09:02:11.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.353+0000 7fbe770d5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55fdff3b1410 con 0x55fdff3ae900 2024-09-10T09:02:11.355 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.353+0000 7fbe770d5740 10 client.0 _ll_register_callbacks cb 0x55fdff2f3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:02:11.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.354+0000 7fbe6dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39b490 0x55fdff3934f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:11.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.355+0000 7fbe6ed9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff3ae900 0x55fdff393a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:11.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.355+0000 7fbe6dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39b490 0x55fdff3934f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:39532/0 (socket says 172.21.15.172:39532) 2024-09-10T09:02:11.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.355+0000 7fbe6dd9b640 1 -- 172.21.15.172:0/1332266234 learned_addr learned my addr 172.21.15.172:0/1332266234 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:11.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.360+0000 7fbe6dd9b640 1 -- 172.21.15.172:0/1332266234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff3ae900 msgr2=0x55fdff393a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:11.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.360+0000 7fbe6dd9b640 1 --2- 172.21.15.172:0/1332266234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff3ae900 0x55fdff393a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.360+0000 7fbe6dd9b640 1 -- 172.21.15.172:0/1332266234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff39ab70 msgr2=0x55fdff396240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:11.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.360+0000 7fbe6dd9b640 1 --2- 172.21.15.172:0/1332266234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff39ab70 0x55fdff396240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.360+0000 7fbe6dd9b640 1 -- 172.21.15.172:0/1332266234 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdff0693c0 con 0x55fdff39b490 2024-09-10T09:02:11.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.360+0000 7fbe6dd9b640 1 --2- 172.21.15.172:0/1332266234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39b490 0x55fdff3934f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c000c00 tx=0x7fbe5c004500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=154eb1556d7f616 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:11.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.361+0000 7fbe51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:02:11.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.365+0000 7fbe51ffb640 1 -- 172.21.15.172:0/1332266234 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c043030 con 0x55fdff39b490 2024-09-10T09:02:11.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.365+0000 7fbe51ffb640 1 -- 172.21.15.172:0/1332266234 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fbe5c004ad0 con 0x55fdff39b490 2024-09-10T09:02:11.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.366+0000 7fbe770d5740 1 -- 172.21.15.172:0/1332266234 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe5c030950 con 0x55fdff39b490 2024-09-10T09:02:11.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.366+0000 7fbe51ffb640 1 -- 172.21.15.172:0/1332266234 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbe5c004d10 con 0x55fdff39b490 2024-09-10T09:02:11.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.367+0000 7fbe770d5740 10 client.28439 fetch_fsmap learned FSMap version 44 2024-09-10T09:02:11.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.367+0000 7fbe770d5740 1 -- 172.21.15.172:0/1332266234 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55fdff3bcc90 con 0x55fdff39b490 2024-09-10T09:02:11.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.369+0000 7fbe51ffb640 1 -- 172.21.15.172:0/1332266234 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe5c030950 con 0x55fdff39b490 2024-09-10T09:02:11.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.369+0000 7fbe51ffb640 1 -- 172.21.15.172:0/1332266234 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fbe5c058020 con 0x55fdff39b490 2024-09-10T09:02:11.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.370+0000 7fbe770d5740 10 client.28439 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:02:11.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.370+0000 7fbe770d5740 10 client.28439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:02:11.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.370+0000 7fbe770d5740 20 client.28439 populate_metadata read hostname 'smithi172' 2024-09-10T09:02:11.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.370+0000 7fbe770d5740 1 --2- 172.21.15.172:0/1332266234 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fdff3be0f0 0x55fdff3c0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.370+0000 7fbe770d5740 4 client.28439 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:02:11.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.370+0000 7fbe770d5740 1 -- 172.21.15.172:0/1332266234 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdff305850 con 0x55fdff3be0f0 2024-09-10T09:02:11.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.376+0000 7fbe6e59c640 1 --2- 172.21.15.172:0/1332266234 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fdff3be0f0 0x55fdff3c0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:11.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.384+0000 7fbe51ffb640 10 client.28439 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:11.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.384+0000 7fbe6e59c640 1 --2- 172.21.15.172:0/1332266234 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fdff3be0f0 0x55fdff3c0530 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ecea127243333c4 server_cookie=8b4a60ecd31ba5ee in_seq=0 out_seq=0 2024-09-10T09:02:11.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.388+0000 7fbe51ffb640 1 -- 172.21.15.172:0/1332266234 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fdff305850 con 0x55fdff3be0f0 2024-09-10T09:02:11.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.388+0000 7fbe51ffb640 10 client.28439 handle_command_reply: tid=0 2024-09-10T09:02:11.390 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.388+0000 7fbe770d5740 1 client.28439 shutdown 2024-09-10T09:02:11.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.396+0000 7fbe770d5740 1 -- 172.21.15.172:0/1332266234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39b490 msgr2=0x55fdff3934f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:11.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.396+0000 7fbe770d5740 1 --2- 172.21.15.172:0/1332266234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39b490 0x55fdff3934f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c000c00 tx=0x7fbe5c004500 comp rx=0 tx=0).stop 2024-09-10T09:02:11.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.399+0000 7fbe770d5740 1 -- 172.21.15.172:0/1332266234 shutdown_connections 2024-09-10T09:02:11.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.399+0000 7fbe770d5740 1 --2- 172.21.15.172:0/1332266234 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fdff3be0f0 0x55fdff3c0530 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.399+0000 7fbe770d5740 1 --2- 172.21.15.172:0/1332266234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff3ae900 0x55fdff393a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.399+0000 7fbe770d5740 1 --2- 172.21.15.172:0/1332266234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39b490 0x55fdff3934f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.399+0000 7fbe770d5740 1 --2- 172.21.15.172:0/1332266234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff39ab70 0x55fdff396240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.399+0000 7fbe51ffb640 0 client.28439 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:11.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.399+0000 7fbe770d5740 1 -- 172.21.15.172:0/1332266234 >> 172.21.15.172:0/1332266234 conn(0x55fdff397240 msgr2=0x55fdff39cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:11.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.399+0000 7fbe770d5740 1 -- 172.21.15.172:0/1332266234 shutdown_connections 2024-09-10T09:02:11.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.399+0000 7fbe770d5740 1 -- 172.21.15.172:0/1332266234 wait complete. 2024-09-10T09:02:11.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.403+0000 7fbe770d5740 20 client.28439 trim_cache size 0 max 16384 2024-09-10T09:02:11.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.436+0000 7fbe770d5740 1 Processor -- start 2024-09-10T09:02:11.438 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.437+0000 7fbe770d5740 1 -- start start 2024-09-10T09:02:11.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.437+0000 7fbe770d5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff39ab70 0x55fdff3ba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.437+0000 7fbe770d5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39b490 0x55fdff3c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.437+0000 7fbe770d5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff3ae900 0x55fdff3c6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.437+0000 7fbe770d5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5800fb40 con 0x55fdff3ae900 2024-09-10T09:02:11.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.437+0000 7fbe770d5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe58011680 con 0x55fdff39b490 2024-09-10T09:02:11.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.437+0000 7fbe770d5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbe58010210 con 0x55fdff39ab70 2024-09-10T09:02:11.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.437+0000 7fbe6dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39b490 0x55fdff3c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:11.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.438+0000 7fbe6dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39b490 0x55fdff3c5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:39546/0 (socket says 172.21.15.172:39546) 2024-09-10T09:02:11.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.438+0000 7fbe6dd9b640 1 -- 172.21.15.172:0/409494282 learned_addr learned my addr 172.21.15.172:0/409494282 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:11.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.438+0000 7fbe6ed9d640 1 --2- 172.21.15.172:0/409494282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff3ae900 0x55fdff3c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:11.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.438+0000 7fbe6dd9b640 1 -- 172.21.15.172:0/409494282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff39ab70 msgr2=0x55fdff3ba5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:02:11.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.438+0000 7fbe6dd9b640 1 --2- 172.21.15.172:0/409494282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff39ab70 0x55fdff3ba5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.438+0000 7fbe6dd9b640 1 -- 172.21.15.172:0/409494282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff3ae900 msgr2=0x55fdff3c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:11.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.438+0000 7fbe6dd9b640 1 --2- 172.21.15.172:0/409494282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff3ae900 0x55fdff3c6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.438+0000 7fbe6dd9b640 1 -- 172.21.15.172:0/409494282 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdff39b2c0 con 0x55fdff39b490 2024-09-10T09:02:11.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.446+0000 7fbe6dd9b640 1 --2- 172.21.15.172:0/409494282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39b490 0x55fdff3c5e00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c02aa30 tx=0x7fbe5c02a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77539d76c7c12c0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:11.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.451+0000 7fbe53fff640 1 -- 172.21.15.172:0/409494282 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c043030 con 0x55fdff39b490 2024-09-10T09:02:11.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.451+0000 7fbe53fff640 1 -- 172.21.15.172:0/409494282 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe5c030950 con 0x55fdff39b490 2024-09-10T09:02:11.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.451+0000 7fbe53fff640 1 -- 172.21.15.172:0/409494282 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c03f520 con 0x55fdff39b490 2024-09-10T09:02:11.455 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.452+0000 7fbe770d5740 1 -- 172.21.15.172:0/409494282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39b490 msgr2=0x55fdff3c5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:11.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.452+0000 7fbe770d5740 1 --2- 172.21.15.172:0/409494282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39b490 0x55fdff3c5e00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c02aa30 tx=0x7fbe5c02a440 comp rx=0 tx=0).stop 2024-09-10T09:02:11.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.452+0000 7fbe770d5740 1 -- 172.21.15.172:0/409494282 shutdown_connections 2024-09-10T09:02:11.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.452+0000 7fbe770d5740 1 --2- 172.21.15.172:0/409494282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff3ae900 0x55fdff3c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.452+0000 7fbe770d5740 1 --2- 172.21.15.172:0/409494282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff39b490 0x55fdff3c5e00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.452+0000 7fbe770d5740 1 --2- 172.21.15.172:0/409494282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff39ab70 0x55fdff3ba5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.456 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.452+0000 7fbe770d5740 1 -- 172.21.15.172:0/409494282 >> 172.21.15.172:0/409494282 conn(0x55fdff397240 msgr2=0x55fdff395570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:11.458 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.457+0000 7fbe770d5740 1 -- 172.21.15.172:0/409494282 shutdown_connections 2024-09-10T09:02:11.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.458+0000 7fbe770d5740 1 -- 172.21.15.172:0/409494282 wait complete. 2024-09-10T09:02:11.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.458+0000 7fbe770d5740 1 -- start start 2024-09-10T09:02:11.460 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.458+0000 7fbe770d5740 1 Processor -- start 2024-09-10T09:02:11.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.459+0000 7fbe770d5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff39ab70 0x55fdff3b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.459+0000 7fbe770d5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff39b490 0x55fdff3b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.459+0000 7fbe770d5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff3ae900 0x55fdff427dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.459+0000 7fbe6dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff39b490 0x55fdff3b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:11.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.459+0000 7fbe6dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff39b490 0x55fdff3b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36128/0 (socket says 172.21.15.172:36128) 2024-09-10T09:02:11.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.459+0000 7fbe6e59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff39ab70 0x55fdff3b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:11.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.459+0000 7fbe6e59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff39ab70 0x55fdff3b2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:46694/0 (socket says 172.21.15.172:46694) 2024-09-10T09:02:11.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.459+0000 7fbe6dd9b640 1 -- 172.21.15.172:0/1285046790 learned_addr learned my addr 172.21.15.172:0/1285046790 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:11.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.459+0000 7fbe770d5740 1 -- 172.21.15.172:0/1285046790 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbe58010fb0 con 0x55fdff39b490 2024-09-10T09:02:11.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.459+0000 7fbe770d5740 1 -- 172.21.15.172:0/1285046790 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe580108e0 con 0x55fdff3ae900 2024-09-10T09:02:11.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.459+0000 7fbe770d5740 1 -- 172.21.15.172:0/1285046790 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5800db20 con 0x55fdff39ab70 2024-09-10T09:02:11.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.459+0000 7fbe6ed9d640 1 --2- 172.21.15.172:0/1285046790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff3ae900 0x55fdff427dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:11.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.460+0000 7fbe770d5740 10 client.0 _ll_register_callbacks cb 0x7fbe5c0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:02:11.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.462+0000 7fbe6ed9d640 1 -- 172.21.15.172:0/1285046790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff39ab70 msgr2=0x55fdff3b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:11.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.462+0000 7fbe6ed9d640 1 --2- 172.21.15.172:0/1285046790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff39ab70 0x55fdff3b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.462+0000 7fbe6ed9d640 1 -- 172.21.15.172:0/1285046790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff39b490 msgr2=0x55fdff3b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:11.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.462+0000 7fbe6ed9d640 1 --2- 172.21.15.172:0/1285046790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff39b490 0x55fdff3b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.462+0000 7fbe6ed9d640 1 -- 172.21.15.172:0/1285046790 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdff428310 con 0x55fdff3ae900 2024-09-10T09:02:11.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.466+0000 7fbe6ed9d640 1 --2- 172.21.15.172:0/1285046790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff3ae900 0x55fdff427dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800cd00 tx=0x7fbe6809b340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b2eb5a602763a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:11.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.472+0000 7fbe51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:02:11.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.472+0000 7fbe51ffb640 1 -- 172.21.15.172:0/1285046790 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6801a040 con 0x55fdff3ae900 2024-09-10T09:02:11.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.472+0000 7fbe51ffb640 1 -- 172.21.15.172:0/1285046790 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fbe68097db0 con 0x55fdff3ae900 2024-09-10T09:02:11.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.472+0000 7fbe770d5740 1 -- 172.21.15.172:0/1285046790 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe5c030950 con 0x55fdff3ae900 2024-09-10T09:02:11.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.473+0000 7fbe51ffb640 1 -- 172.21.15.172:0/1285046790 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbe6809be40 con 0x55fdff3ae900 2024-09-10T09:02:11.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.473+0000 7fbe770d5740 10 client.28445 fetch_fsmap learned FSMap version 44 2024-09-10T09:02:11.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.473+0000 7fbe770d5740 1 -- 172.21.15.172:0/1285046790 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55fdff39d950 con 0x55fdff3ae900 2024-09-10T09:02:11.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.476+0000 7fbe51ffb640 1 -- 172.21.15.172:0/1285046790 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe680b4070 con 0x55fdff3ae900 2024-09-10T09:02:11.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.476+0000 7fbe51ffb640 1 -- 172.21.15.172:0/1285046790 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fbe6800b040 con 0x55fdff3ae900 2024-09-10T09:02:11.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.477+0000 7fbe770d5740 10 client.28445 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:02:11.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.477+0000 7fbe770d5740 10 client.28445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:02:11.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.477+0000 7fbe770d5740 20 client.28445 populate_metadata read hostname 'smithi172' 2024-09-10T09:02:11.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.477+0000 7fbe770d5740 1 --2- 172.21.15.172:0/1285046790 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fdff39e020 0x55fdff3a0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:11.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.477+0000 7fbe770d5740 4 client.28445 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:02:11.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.477+0000 7fbe770d5740 1 -- 172.21.15.172:0/1285046790 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fbe5c030950 con 0x55fdff39e020 2024-09-10T09:02:11.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.480+0000 7fbe6e59c640 1 --2- 172.21.15.172:0/1285046790 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fdff39e020 0x55fdff3a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:11.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.486+0000 7fbe6e59c640 1 --2- 172.21.15.172:0/1285046790 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fdff39e020 0x55fdff3a0410 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdbc02760d9f3029 server_cookie=b058eafaf9f51fa2 in_seq=0 out_seq=0 2024-09-10T09:02:11.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.487+0000 7fbe51ffb640 10 client.28445 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:11.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.487+0000 7fbe51ffb640 1 -- 172.21.15.172:0/1285046790 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x7fbe5c030950 con 0x55fdff39e020 2024-09-10T09:02:11.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.487+0000 7fbe51ffb640 10 client.28445 handle_command_reply: tid=0 2024-09-10T09:02:11.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.487+0000 7fbe770d5740 1 client.28445 shutdown 2024-09-10T09:02:11.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.495+0000 7fbe770d5740 1 -- 172.21.15.172:0/1285046790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff3ae900 msgr2=0x55fdff427dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:11.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.495+0000 7fbe770d5740 1 --2- 172.21.15.172:0/1285046790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff3ae900 0x55fdff427dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800cd00 tx=0x7fbe6809b340 comp rx=0 tx=0).stop 2024-09-10T09:02:11.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.495+0000 7fbe770d5740 1 -- 172.21.15.172:0/1285046790 shutdown_connections 2024-09-10T09:02:11.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.495+0000 7fbe770d5740 1 --2- 172.21.15.172:0/1285046790 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55fdff39e020 0x55fdff3a0410 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.495+0000 7fbe770d5740 1 --2- 172.21.15.172:0/1285046790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fdff3ae900 0x55fdff427dd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.495+0000 7fbe51ffb640 0 client.28445 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:11.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.495+0000 7fbe770d5740 1 --2- 172.21.15.172:0/1285046790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdff39b490 0x55fdff3b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.495+0000 7fbe770d5740 1 --2- 172.21.15.172:0/1285046790 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fdff39ab70 0x55fdff3b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.495+0000 7fbe770d5740 1 -- 172.21.15.172:0/1285046790 >> 172.21.15.172:0/1285046790 conn(0x55fdff397240 msgr2=0x55fdff39c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:11.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.495+0000 7fbe770d5740 1 -- 172.21.15.172:0/1285046790 shutdown_connections 2024-09-10T09:02:11.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.495+0000 7fbe770d5740 1 -- 172.21.15.172:0/1285046790 wait complete. 2024-09-10T09:02:11.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.496+0000 7fbe770d5740 20 client.28445 trim_cache size 0 max 16384 2024-09-10T09:02:11.500 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:02:11.500 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "scrub active (0 inodes in the stack)", 2024-09-10T09:02:11.500 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T09:02:11.500 INFO:teuthology.orchestra.run.smithi172.stdout: "097ab02a-d7be-4e1d-96ec-78f5d8c3836d": { 2024-09-10T09:02:11.500 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T09:02:11.500 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "097ab02a-d7be-4e1d-96ec-78f5d8c3836d", 2024-09-10T09:02:11.500 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T09:02:11.500 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:02:11.500 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:02:11.501 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:02:11.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.498+0000 7fbe74cfc640 1 -- 172.21.15.172:0/847662468 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbe540a2d90 msgr2=0x7fbe540a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:11.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.498+0000 7fbe74cfc640 1 --2- 172.21.15.172:0/847662468 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbe540a2d90 0x7fbe540a5250 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000a680 tx=0x7fbe60008040 comp rx=0 tx=0).stop 2024-09-10T09:02:11.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.498+0000 7fbe74cfc640 1 -- 172.21.15.172:0/847662468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe70102ef0 msgr2=0x7fbe7019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:11.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.498+0000 7fbe74cfc640 1 --2- 172.21.15.172:0/847662468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe70102ef0 0x7fbe7019aba0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009fd0 tx=0x7fbe6800c800 comp rx=0 tx=0).stop 2024-09-10T09:02:11.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.498+0000 7fbe74cfc640 1 -- 172.21.15.172:0/847662468 shutdown_connections 2024-09-10T09:02:11.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.498+0000 7fbe74cfc640 1 --2- 172.21.15.172:0/847662468 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fbe540a2d90 0x7fbe540a5250 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.498+0000 7fbe74cfc640 1 --2- 172.21.15.172:0/847662468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe70102ef0 0x7fbe7019aba0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.498+0000 7fbe74cfc640 1 --2- 172.21.15.172:0/847662468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe70102570 0x7fbe70196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.499+0000 7fbe74cfc640 1 --2- 172.21.15.172:0/847662468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe70100e20 0x7fbe701962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:11.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.499+0000 7fbe74cfc640 1 -- 172.21.15.172:0/847662468 >> 172.21.15.172:0/847662468 conn(0x7fbe700fc7d0 msgr2=0x7fbe701048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:11.504 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.500+0000 7fbe74cfc640 1 -- 172.21.15.172:0/847662468 shutdown_connections 2024-09-10T09:02:11.510 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:11.504+0000 7fbe74cfc640 1 -- 172.21.15.172:0/847662468 wait complete. 2024-09-10T09:02:11.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:11 smithi191 ceph-mon[35314]: pgmap v828: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.6 MiB/s wr, 933 op/s 2024-09-10T09:02:11.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:11 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:02:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:11 smithi174 ceph-mon[34068]: pgmap v828: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.6 MiB/s wr, 933 op/s 2024-09-10T09:02:11.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:11 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:02:12.477 INFO:tasks.cephfs.filesystem:scrub status for tag:097ab02a-d7be-4e1d-96ec-78f5d8c3836d - {'path': '/', 'tag': '097ab02a-d7be-4e1d-96ec-78f5d8c3836d', 'options': 'recursive,force'} 2024-09-10T09:02:13.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:13 smithi191 ceph-mon[35314]: pgmap v829: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.2 MiB/s wr, 1.35k op/s 2024-09-10T09:02:13.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:13 smithi172 ceph-mon[30854]: pgmap v829: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.2 MiB/s wr, 1.35k op/s 2024-09-10T09:02:13.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:13 smithi174 ceph-mon[34068]: pgmap v829: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.2 MiB/s wr, 1.35k op/s 2024-09-10T09:02:15.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:15 smithi191 ceph-mon[35314]: pgmap v830: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.7 MiB/s wr, 1.74k op/s 2024-09-10T09:02:15.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:15 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:02:15.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:15 smithi172 ceph-mon[30854]: pgmap v830: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.7 MiB/s wr, 1.74k op/s 2024-09-10T09:02:15.584 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:15 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:02:15.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:15 smithi174 ceph-mon[34068]: pgmap v830: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.7 MiB/s wr, 1.74k op/s 2024-09-10T09:02:15.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:15 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:02:17.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:17 smithi191 ceph-mon[35314]: pgmap v831: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 3.11k op/s 2024-09-10T09:02:17.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:17 smithi172 ceph-mon[30854]: pgmap v831: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 3.11k op/s 2024-09-10T09:02:17.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:17 smithi174 ceph-mon[34068]: pgmap v831: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 3.11k op/s 2024-09-10T09:02:18.613 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.612+0000 7f09a1c4c640 1 -- 172.21.15.172:0/3729519018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f099c10de90 msgr2=0x7f099c110350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:18.613 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.612+0000 7f09a1c4c640 1 --2- 172.21.15.172:0/3729519018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f099c10de90 0x7f099c110350 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f098c009930 tx=0x7f098c02a480 comp rx=0 tx=0).stop 2024-09-10T09:02:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.615+0000 7f09a1c4c640 1 -- 172.21.15.172:0/3729519018 shutdown_connections 2024-09-10T09:02:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.615+0000 7f09a1c4c640 1 --2- 172.21.15.172:0/3729519018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f099c10de90 0x7f099c110350 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.615+0000 7f09a1c4c640 1 --2- 172.21.15.172:0/3729519018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f099c10b4e0 0x7f099c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.615+0000 7f09a1c4c640 1 --2- 172.21.15.172:0/3729519018 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f099c072d80 0x7f099c073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.615+0000 7f09a1c4c640 1 -- 172.21.15.172:0/3729519018 >> 172.21.15.172:0/3729519018 conn(0x7f099c06eb50 msgr2=0x7f099c070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.615+0000 7f09a1c4c640 1 -- 172.21.15.172:0/3729519018 shutdown_connections 2024-09-10T09:02:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.615+0000 7f09a1c4c640 1 -- 172.21.15.172:0/3729519018 wait complete. 2024-09-10T09:02:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.616+0000 7f09a1c4c640 1 Processor -- start 2024-09-10T09:02:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.616+0000 7f09a1c4c640 1 -- start start 2024-09-10T09:02:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.617+0000 7f09a1c4c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f099c072d80 0x7f099c115860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.617+0000 7f09a1c4c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f099c10b4e0 0x7f099c115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.617+0000 7f09a1c4c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f099c10de90 0x7f099c1a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.617+0000 7f09a1c4c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f099c11c360 con 0x7f099c10b4e0 2024-09-10T09:02:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.617+0000 7f09a1c4c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f099c11c1e0 con 0x7f099c10de90 2024-09-10T09:02:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.617+0000 7f09a1c4c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f099c11c4e0 con 0x7f099c072d80 2024-09-10T09:02:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.617+0000 7f099b7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f099c072d80 0x7f099c115860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.617+0000 7f099b7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f099c072d80 0x7f099c115860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:46326/0 (socket says 172.21.15.172:46326) 2024-09-10T09:02:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.617+0000 7f099b7fe640 1 -- 172.21.15.172:0/297874819 learned_addr learned my addr 172.21.15.172:0/297874819 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.617+0000 7f099bfff640 1 --2- 172.21.15.172:0/297874819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f099c10de90 0x7f099c1a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.618+0000 7f099b7fe640 1 -- 172.21.15.172:0/297874819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f099c10de90 msgr2=0x7f099c1a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.618+0000 7f099b7fe640 1 --2- 172.21.15.172:0/297874819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f099c10de90 0x7f099c1a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.618+0000 7f099b7fe640 1 -- 172.21.15.172:0/297874819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f099c10b4e0 msgr2=0x7f099c115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.618+0000 7f099b7fe640 1 --2- 172.21.15.172:0/297874819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f099c10b4e0 0x7f099c115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.618+0000 7f099b7fe640 1 -- 172.21.15.172:0/297874819 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099c116440 con 0x7f099c072d80 2024-09-10T09:02:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.618+0000 7f099b7fe640 1 --2- 172.21.15.172:0/297874819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f099c072d80 0x7f099c115860 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f099400ebc0 tx=0x7f099400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.618+0000 7f0998ff9640 1 -- 172.21.15.172:0/297874819 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994010630 con 0x7f099c072d80 2024-09-10T09:02:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.618+0000 7f09a1c4c640 1 -- 172.21.15.172:0/297874819 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f099c1a60c0 con 0x7f099c072d80 2024-09-10T09:02:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.618+0000 7f09a1c4c640 1 -- 172.21.15.172:0/297874819 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f099c1a6660 con 0x7f099c072d80 2024-09-10T09:02:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.619+0000 7f0998ff9640 1 -- 172.21.15.172:0/297874819 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0994010c80 con 0x7f099c072d80 2024-09-10T09:02:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.619+0000 7f0998ff9640 1 -- 172.21.15.172:0/297874819 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994018b00 con 0x7f099c072d80 2024-09-10T09:02:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.625+0000 7f09a49e7740 1 Processor -- start 2024-09-10T09:02:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.626+0000 7f0998ff9640 1 -- 172.21.15.172:0/297874819 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0994018ca0 con 0x7f099c072d80 2024-09-10T09:02:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.627+0000 7f0998ff9640 1 --2- 172.21.15.172:0/297874819 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f09840a2cc0 0x7f09840a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.627+0000 7f0998ff9640 1 -- 172.21.15.172:0/297874819 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f09940cfaa0 con 0x7f099c072d80 2024-09-10T09:02:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.628+0000 7f099bfff640 1 --2- 172.21.15.172:0/297874819 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f09840a2cc0 0x7f09840a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.629+0000 7f09a49e7740 1 -- start start 2024-09-10T09:02:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.630+0000 7f09a49e7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1280b70 0x55c7a1280f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.630+0000 7f09a49e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 0x55c7a1285f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.630+0000 7f09a49e7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a1286540 0x55c7a1288a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.630+0000 7f09a49e7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c7a11e42f0 con 0x55c7a1280b70 2024-09-10T09:02:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.630+0000 7f09a49e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c7a118c270 con 0x55c7a12814b0 2024-09-10T09:02:18.631 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.630+0000 7f09a49e7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c7a11e02b0 con 0x55c7a1286540 2024-09-10T09:02:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.631+0000 7f099b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1280b70 0x55c7a1280f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.631+0000 7f099b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1280b70 0x55c7a1280f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42414/0 (socket says 172.21.15.172:42414) 2024-09-10T09:02:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.631+0000 7f099b7fe640 1 -- 172.21.15.172:0/915910681 learned_addr learned my addr 172.21.15.172:0/915910681 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.631+0000 7f099affd640 1 --2- 172.21.15.172:0/915910681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 0x55c7a1285f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.631+0000 7f099bfff640 1 --2- 172.21.15.172:0/915910681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a1286540 0x55c7a1288a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.632+0000 7f099bfff640 1 --2- 172.21.15.172:0/297874819 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f09840a2cc0 0x7f09840a5180 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f098c009900 tx=0x7f098c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.632+0000 7f099bfff640 1 -- 172.21.15.172:0/915910681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 msgr2=0x55c7a1285f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.632+0000 7f099bfff640 1 --2- 172.21.15.172:0/915910681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 0x55c7a1285f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.632+0000 7f099bfff640 1 -- 172.21.15.172:0/915910681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1280b70 msgr2=0x55c7a1280f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.632+0000 7f099bfff640 1 --2- 172.21.15.172:0/915910681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1280b70 0x55c7a1280f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.632+0000 7f099bfff640 1 -- 172.21.15.172:0/915910681 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7a118b340 con 0x55c7a1286540 2024-09-10T09:02:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.635+0000 7f099bfff640 1 --2- 172.21.15.172:0/915910681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a1286540 0x55c7a1288a00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f098c009850 tx=0x7f098c048000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b5e1574ae2e878d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:18.638 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.637+0000 7f0981ffb640 1 -- 172.21.15.172:0/915910681 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c031440 con 0x55c7a1286540 2024-09-10T09:02:18.638 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.638+0000 7f0981ffb640 1 -- 172.21.15.172:0/915910681 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f098c04e070 con 0x55c7a1286540 2024-09-10T09:02:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.639+0000 7f09a49e7740 1 -- 172.21.15.172:0/915910681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a1286540 msgr2=0x55c7a1288a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.639+0000 7f09a49e7740 1 --2- 172.21.15.172:0/915910681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a1286540 0x55c7a1288a00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f098c009850 tx=0x7f098c048000 comp rx=0 tx=0).stop 2024-09-10T09:02:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.640+0000 7f09a49e7740 1 -- 172.21.15.172:0/915910681 shutdown_connections 2024-09-10T09:02:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.640+0000 7f09a49e7740 1 --2- 172.21.15.172:0/915910681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a1286540 0x55c7a1288a00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.640+0000 7f09a49e7740 1 --2- 172.21.15.172:0/915910681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 0x55c7a1285f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.640+0000 7f09a49e7740 1 --2- 172.21.15.172:0/915910681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1280b70 0x55c7a1280f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.640+0000 7f09a49e7740 1 -- 172.21.15.172:0/915910681 >> 172.21.15.172:0/915910681 conn(0x55c7a127c1e0 msgr2=0x55c7a127c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:18.644 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.643+0000 7f09a49e7740 1 -- 172.21.15.172:0/915910681 shutdown_connections 2024-09-10T09:02:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.646+0000 7f09a49e7740 1 -- 172.21.15.172:0/915910681 wait complete. 2024-09-10T09:02:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.647+0000 7f09a49e7740 1 -- start start 2024-09-10T09:02:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.647+0000 7f09a49e7740 1 Processor -- start 2024-09-10T09:02:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.648+0000 7f09a49e7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1280b70 0x55c7a128add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.648+0000 7f09a49e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 0x55c7a128b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.649 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.648+0000 7f09a49e7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a1286540 0x55c7a1292350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.649 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.648+0000 7f09a49e7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c7a1289500 con 0x55c7a1280b70 2024-09-10T09:02:18.649 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.648+0000 7f09a49e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c7a116d290 con 0x55c7a12814b0 2024-09-10T09:02:18.649 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.648+0000 7f09a49e7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c7a1289680 con 0x55c7a1286540 2024-09-10T09:02:18.649 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.648+0000 7f09a49e7740 10 client.0 _ll_register_callbacks cb 0x55c7a119e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:02:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.650+0000 7f099affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 0x55c7a128b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.650+0000 7f099affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 0x55c7a128b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:53486/0 (socket says 172.21.15.172:53486) 2024-09-10T09:02:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.650+0000 7f099affd640 1 -- 172.21.15.172:0/898570299 learned_addr learned my addr 172.21.15.172:0/898570299 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.650+0000 7f099bfff640 1 --2- 172.21.15.172:0/898570299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a1286540 0x55c7a1292350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.651+0000 7f099affd640 1 -- 172.21.15.172:0/898570299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a1286540 msgr2=0x55c7a1292350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.651+0000 7f099affd640 1 --2- 172.21.15.172:0/898570299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a1286540 0x55c7a1292350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.651+0000 7f099affd640 1 -- 172.21.15.172:0/898570299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1280b70 msgr2=0x55c7a128add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.651+0000 7f099affd640 1 --2- 172.21.15.172:0/898570299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1280b70 0x55c7a128add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.651+0000 7f099affd640 1 -- 172.21.15.172:0/898570299 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7a0ef7210 con 0x55c7a12814b0 2024-09-10T09:02:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.656+0000 7f099affd640 1 --2- 172.21.15.172:0/898570299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 0x55c7a128b310 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f099000d160 tx=0x7f099000d620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffc88c2d0fb7a629 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.656+0000 7f097affd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:02:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.656+0000 7f097affd640 1 -- 172.21.15.172:0/898570299 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099000de50 con 0x55c7a12814b0 2024-09-10T09:02:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.656+0000 7f097affd640 1 -- 172.21.15.172:0/898570299 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f09900125f0 con 0x55c7a12814b0 2024-09-10T09:02:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.657+0000 7f09a49e7740 1 -- 172.21.15.172:0/898570299 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f098c04e070 con 0x55c7a12814b0 2024-09-10T09:02:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.657+0000 7f097affd640 1 -- 172.21.15.172:0/898570299 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f09900127f0 con 0x55c7a12814b0 2024-09-10T09:02:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.660+0000 7f09a49e7740 10 client.28457 fetch_fsmap learned FSMap version 44 2024-09-10T09:02:18.660 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.660+0000 7f09a49e7740 1 -- 172.21.15.172:0/898570299 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c7a0fc3620 con 0x55c7a12814b0 2024-09-10T09:02:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.664+0000 7f097affd640 1 -- 172.21.15.172:0/898570299 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f098c04e070 con 0x55c7a12814b0 2024-09-10T09:02:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.664+0000 7f097affd640 1 -- 172.21.15.172:0/898570299 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f0990013070 con 0x55c7a12814b0 2024-09-10T09:02:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.664+0000 7f09a49e7740 10 client.28457 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:02:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.664+0000 7f09a49e7740 10 client.28457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:02:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.664+0000 7f09a49e7740 20 client.28457 populate_metadata read hostname 'smithi172' 2024-09-10T09:02:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.664+0000 7f09a49e7740 1 --2- 172.21.15.172:0/898570299 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55c7a1296b10 0x55c7a1298f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.664+0000 7f09a49e7740 4 client.28457 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:02:18.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.664+0000 7f09a49e7740 1 -- 172.21.15.172:0/898570299 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7a11f1a20 con 0x55c7a1296b10 2024-09-10T09:02:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.664+0000 7f099b7fe640 1 --2- 172.21.15.172:0/898570299 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55c7a1296b10 0x55c7a1298f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.665+0000 7f097affd640 10 client.28457 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:18.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.666+0000 7f099b7fe640 1 --2- 172.21.15.172:0/898570299 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55c7a1296b10 0x55c7a1298f80 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80eea6d5645d885d server_cookie=9acb43a4c2616705 in_seq=0 out_seq=0 2024-09-10T09:02:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.668+0000 7f097affd640 1 -- 172.21.15.172:0/898570299 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7a11f1a20 con 0x55c7a1296b10 2024-09-10T09:02:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.668+0000 7f097affd640 10 client.28457 handle_command_reply: tid=0 2024-09-10T09:02:18.669 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.668+0000 7f09a49e7740 1 client.28457 shutdown 2024-09-10T09:02:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.672+0000 7f09a49e7740 1 -- 172.21.15.172:0/898570299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 msgr2=0x55c7a128b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.672+0000 7f09a49e7740 1 --2- 172.21.15.172:0/898570299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 0x55c7a128b310 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f099000d160 tx=0x7f099000d620 comp rx=0 tx=0).stop 2024-09-10T09:02:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.678+0000 7f09a49e7740 1 -- 172.21.15.172:0/898570299 shutdown_connections 2024-09-10T09:02:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.678+0000 7f09a49e7740 1 --2- 172.21.15.172:0/898570299 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55c7a1296b10 0x55c7a1298f80 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.678+0000 7f09a49e7740 1 --2- 172.21.15.172:0/898570299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a1286540 0x55c7a1292350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.678+0000 7f09a49e7740 1 --2- 172.21.15.172:0/898570299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 0x55c7a128b310 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.678+0000 7f09a49e7740 1 --2- 172.21.15.172:0/898570299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1280b70 0x55c7a128add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.678+0000 7f09a49e7740 1 -- 172.21.15.172:0/898570299 >> 172.21.15.172:0/898570299 conn(0x55c7a127c1e0 msgr2=0x55c7a127d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.679+0000 7f097affd640 0 client.28457 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.683+0000 7f09a49e7740 1 -- 172.21.15.172:0/898570299 shutdown_connections 2024-09-10T09:02:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.684+0000 7f09a49e7740 1 -- 172.21.15.172:0/898570299 wait complete. 2024-09-10T09:02:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.690+0000 7f09a49e7740 20 client.28457 trim_cache size 0 max 16384 2024-09-10T09:02:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.768+0000 7f09a49e7740 1 Processor -- start 2024-09-10T09:02:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.769+0000 7f09a49e7740 1 -- start start 2024-09-10T09:02:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.769+0000 7f09a49e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a1280b70 0x55c7a127b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.769+0000 7f09a49e7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a12814b0 0x55c7a127b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.770+0000 7f09a49e7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1286540 0x55c7a1292310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.770+0000 7f09a49e7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f098800fc30 con 0x55c7a1286540 2024-09-10T09:02:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.770+0000 7f09a49e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0988011770 con 0x55c7a1280b70 2024-09-10T09:02:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.770+0000 7f09a49e7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0988010300 con 0x55c7a12814b0 2024-09-10T09:02:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.770+0000 7f099bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1286540 0x55c7a1292310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.770+0000 7f099bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1286540 0x55c7a1292310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42434/0 (socket says 172.21.15.172:42434) 2024-09-10T09:02:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.770+0000 7f099bfff640 1 -- 172.21.15.172:0/1052985214 learned_addr learned my addr 172.21.15.172:0/1052985214 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.771+0000 7f099bfff640 1 -- 172.21.15.172:0/1052985214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a12814b0 msgr2=0x55c7a127b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.771+0000 7f099bfff640 1 --2- 172.21.15.172:0/1052985214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a12814b0 0x55c7a127b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.771+0000 7f099bfff640 1 -- 172.21.15.172:0/1052985214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a1280b70 msgr2=0x55c7a127b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.771+0000 7f099bfff640 1 --2- 172.21.15.172:0/1052985214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a1280b70 0x55c7a127b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.771+0000 7f099bfff640 1 -- 172.21.15.172:0/1052985214 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7a12862e0 con 0x55c7a1286540 2024-09-10T09:02:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.771+0000 7f099bfff640 1 --2- 172.21.15.172:0/1052985214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1286540 0x55c7a1292310 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f098c009930 tx=0x7f098c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70d031d32612de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.775+0000 7f09817fa640 1 -- 172.21.15.172:0/1052985214 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c004840 con 0x55c7a1286540 2024-09-10T09:02:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.776+0000 7f09817fa640 1 -- 172.21.15.172:0/1052985214 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f098c04e040 con 0x55c7a1286540 2024-09-10T09:02:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.776+0000 7f09817fa640 1 -- 172.21.15.172:0/1052985214 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c031a50 con 0x55c7a1286540 2024-09-10T09:02:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.777+0000 7f09a49e7740 1 -- 172.21.15.172:0/1052985214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1286540 msgr2=0x55c7a1292310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.777+0000 7f09a49e7740 1 --2- 172.21.15.172:0/1052985214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1286540 0x55c7a1292310 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f098c009930 tx=0x7f098c009680 comp rx=0 tx=0).stop 2024-09-10T09:02:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.781+0000 7f09a49e7740 1 -- 172.21.15.172:0/1052985214 shutdown_connections 2024-09-10T09:02:18.781 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.781+0000 7f09a49e7740 1 --2- 172.21.15.172:0/1052985214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1286540 0x55c7a1292310 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.781+0000 7f09a49e7740 1 --2- 172.21.15.172:0/1052985214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a12814b0 0x55c7a127b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.781+0000 7f09a49e7740 1 --2- 172.21.15.172:0/1052985214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a1280b70 0x55c7a127b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.782 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.781+0000 7f09a49e7740 1 -- 172.21.15.172:0/1052985214 >> 172.21.15.172:0/1052985214 conn(0x55c7a127c1e0 msgr2=0x55c7a127aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:18.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.787+0000 7f09a49e7740 1 -- 172.21.15.172:0/1052985214 shutdown_connections 2024-09-10T09:02:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.789+0000 7f09a49e7740 1 -- 172.21.15.172:0/1052985214 wait complete. 2024-09-10T09:02:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.789+0000 7f09a49e7740 1 -- start start 2024-09-10T09:02:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.789+0000 7f09a49e7740 1 Processor -- start 2024-09-10T09:02:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.790+0000 7f09a49e7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a1280b70 0x55c7a128f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.790+0000 7f09a49e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 0x55c7a128fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.790+0000 7f09a49e7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1286540 0x55c7a1290eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.790+0000 7f09a49e7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f09880110a0 con 0x55c7a1286540 2024-09-10T09:02:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.790+0000 7f09a49e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09880109d0 con 0x55c7a12814b0 2024-09-10T09:02:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.790+0000 7f09a49e7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f098800dc10 con 0x55c7a1280b70 2024-09-10T09:02:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.790+0000 7f09a49e7740 10 client.0 _ll_register_callbacks cb 0x7f099000b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:02:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.790+0000 7f099affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 0x55c7a128fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.790+0000 7f099affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 0x55c7a128fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:53508/0 (socket says 172.21.15.172:53508) 2024-09-10T09:02:18.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.790+0000 7f099affd640 1 -- 172.21.15.172:0/1196140201 learned_addr learned my addr 172.21.15.172:0/1196140201 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.791+0000 7f099b7fe640 1 --2- 172.21.15.172:0/1196140201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a1280b70 0x55c7a128f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:18.792 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.791+0000 7f099b7fe640 1 -- 172.21.15.172:0/1196140201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 msgr2=0x55c7a128fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.791+0000 7f099b7fe640 1 --2- 172.21.15.172:0/1196140201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 0x55c7a128fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.791+0000 7f099b7fe640 1 -- 172.21.15.172:0/1196140201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1286540 msgr2=0x55c7a1290eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.791+0000 7f099b7fe640 1 --2- 172.21.15.172:0/1196140201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1286540 0x55c7a1290eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.791+0000 7f099b7fe640 1 -- 172.21.15.172:0/1196140201 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7a12914c0 con 0x55c7a1280b70 2024-09-10T09:02:18.793 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.792+0000 7f099b7fe640 1 --2- 172.21.15.172:0/1196140201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a1280b70 0x55c7a128f960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f099409cea0 tx=0x7f0994010bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27d4d231eb15049c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.794+0000 7f097affd640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:02:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.794+0000 7f097affd640 1 -- 172.21.15.172:0/1196140201 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09940bc070 con 0x55c7a1280b70 2024-09-10T09:02:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.794+0000 7f09a49e7740 1 -- 172.21.15.172:0/1196140201 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f098c04e040 con 0x55c7a1280b70 2024-09-10T09:02:18.795 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.794+0000 7f097affd640 1 -- 172.21.15.172:0/1196140201 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f09940cd2f0 con 0x55c7a1280b70 2024-09-10T09:02:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.799+0000 7f097affd640 1 -- 172.21.15.172:0/1196140201 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f09940a4c20 con 0x55c7a1280b70 2024-09-10T09:02:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.799+0000 7f09a49e7740 10 client.28357 fetch_fsmap learned FSMap version 44 2024-09-10T09:02:18.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.799+0000 7f09a49e7740 1 -- 172.21.15.172:0/1196140201 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c7a1301860 con 0x55c7a1280b70 2024-09-10T09:02:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.803+0000 7f097affd640 1 -- 172.21.15.172:0/1196140201 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f098c04e040 con 0x55c7a1280b70 2024-09-10T09:02:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.804+0000 7f097affd640 1 -- 172.21.15.172:0/1196140201 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f099409bdd0 con 0x55c7a1280b70 2024-09-10T09:02:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.807+0000 7f09a49e7740 10 client.28357 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:02:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.807+0000 7f09a49e7740 10 client.28357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:02:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.807+0000 7f09a49e7740 20 client.28357 populate_metadata read hostname 'smithi172' 2024-09-10T09:02:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.807+0000 7f09a49e7740 1 --2- 172.21.15.172:0/1196140201 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55c7a1302100 0x55c7a1304520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.807+0000 7f09a49e7740 4 client.28357 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:02:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.807+0000 7f09a49e7740 1 -- 172.21.15.172:0/1196140201 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7a11ebd70 con 0x55c7a1302100 2024-09-10T09:02:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.808+0000 7f099affd640 1 --2- 172.21.15.172:0/1196140201 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55c7a1302100 0x55c7a1304520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.814+0000 7f099affd640 1 --2- 172.21.15.172:0/1196140201 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55c7a1302100 0x55c7a1304520 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2c548f92163132 server_cookie=ca2ba22fc49e9018 in_seq=0 out_seq=0 2024-09-10T09:02:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:18.816+0000 7f097affd640 10 client.28357 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:19.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:19 smithi191 ceph-mon[35314]: pgmap v832: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-09-10T09:02:19.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:19 smithi172 ceph-mon[30854]: pgmap v832: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-09-10T09:02:19.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:19 smithi174 ceph-mon[34068]: pgmap v832: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 3.29k op/s 2024-09-10T09:02:21.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:21 smithi191 ceph-mon[35314]: pgmap v833: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 3.71k op/s 2024-09-10T09:02:21.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:21 smithi172 ceph-mon[30854]: pgmap v833: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 3.71k op/s 2024-09-10T09:02:21.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:21 smithi174 ceph-mon[34068]: pgmap v833: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 15 MiB/s wr, 3.71k op/s 2024-09-10T09:02:23.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:23 smithi172 ceph-mon[30854]: pgmap v834: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 3.93k op/s 2024-09-10T09:02:23.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:23 smithi174 ceph-mon[34068]: pgmap v834: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 3.93k op/s 2024-09-10T09:02:23.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:23 smithi191 ceph-mon[35314]: pgmap v834: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 15 MiB/s wr, 3.93k op/s 2024-09-10T09:02:25.583 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:25 smithi172 ceph-mon[30854]: pgmap v835: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-09-10T09:02:25.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:25 smithi174 ceph-mon[34068]: pgmap v835: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-09-10T09:02:25.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:25 smithi191 ceph-mon[35314]: pgmap v835: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 MiB/s wr, 3.88k op/s 2024-09-10T09:02:27.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:27 smithi172 ceph-mon[30854]: pgmap v836: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 4.67k op/s 2024-09-10T09:02:27.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:27 smithi174 ceph-mon[34068]: pgmap v836: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 4.67k op/s 2024-09-10T09:02:27.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:27 smithi191 ceph-mon[35314]: pgmap v836: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 MiB/s wr, 4.67k op/s 2024-09-10T09:02:29.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:29 smithi172 ceph-mon[30854]: pgmap v837: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.71k op/s 2024-09-10T09:02:29.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:29 smithi174 ceph-mon[34068]: pgmap v837: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.71k op/s 2024-09-10T09:02:29.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:29 smithi191 ceph-mon[35314]: pgmap v837: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.71k op/s 2024-09-10T09:02:31.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:31 smithi174 ceph-mon[34068]: pgmap v838: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.84k op/s 2024-09-10T09:02:31.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:31 smithi191 ceph-mon[35314]: pgmap v838: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.84k op/s 2024-09-10T09:02:31.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:31 smithi172 ceph-mon[30854]: pgmap v838: 129 pgs: 129 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 3.84k op/s 2024-09-10T09:02:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.293+0000 7f097affd640 1 -- 172.21.15.172:0/1196140201 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c7a11ebd70 con 0x55c7a1302100 2024-09-10T09:02:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.293+0000 7f097affd640 10 client.28357 handle_command_reply: tid=0 2024-09-10T09:02:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.294+0000 7f09a49e7740 1 client.28357 shutdown 2024-09-10T09:02:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.295+0000 7f09a49e7740 1 -- 172.21.15.172:0/1196140201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a1280b70 msgr2=0x55c7a128f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.295+0000 7f09a49e7740 1 --2- 172.21.15.172:0/1196140201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a1280b70 0x55c7a128f960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f099409cea0 tx=0x7f0994010bf0 comp rx=0 tx=0).stop 2024-09-10T09:02:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.295+0000 7f09a49e7740 1 -- 172.21.15.172:0/1196140201 shutdown_connections 2024-09-10T09:02:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.295+0000 7f09a49e7740 1 --2- 172.21.15.172:0/1196140201 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55c7a1302100 0x55c7a1304520 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.295+0000 7f09a49e7740 1 --2- 172.21.15.172:0/1196140201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c7a1286540 0x55c7a1290eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.295+0000 7f09a49e7740 1 --2- 172.21.15.172:0/1196140201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7a12814b0 0x55c7a128fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.295+0000 7f09a49e7740 1 --2- 172.21.15.172:0/1196140201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c7a1280b70 0x55c7a128f960 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.295+0000 7f09a49e7740 1 -- 172.21.15.172:0/1196140201 >> 172.21.15.172:0/1196140201 conn(0x55c7a127c1e0 msgr2=0x55c7a127b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.296+0000 7f097affd640 0 client.28357 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.296+0000 7f09a49e7740 1 -- 172.21.15.172:0/1196140201 shutdown_connections 2024-09-10T09:02:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.296+0000 7f09a49e7740 1 -- 172.21.15.172:0/1196140201 wait complete. 2024-09-10T09:02:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.296+0000 7f09a49e7740 20 client.28357 trim_cache size 0 max 16384 2024-09-10T09:02:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:02:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:02:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:02:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:02:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.300+0000 7f09a1c4c640 1 -- 172.21.15.172:0/297874819 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f09840a2cc0 msgr2=0x7f09840a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.300+0000 7f09a1c4c640 1 --2- 172.21.15.172:0/297874819 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f09840a2cc0 0x7f09840a5180 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f098c009900 tx=0x7f098c005850 comp rx=0 tx=0).stop 2024-09-10T09:02:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.300+0000 7f09a1c4c640 1 -- 172.21.15.172:0/297874819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f099c072d80 msgr2=0x7f099c115860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.300+0000 7f09a1c4c640 1 --2- 172.21.15.172:0/297874819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f099c072d80 0x7f099c115860 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f099400ebc0 tx=0x7f099400c6a0 comp rx=0 tx=0).stop 2024-09-10T09:02:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.302+0000 7f09a1c4c640 1 -- 172.21.15.172:0/297874819 shutdown_connections 2024-09-10T09:02:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.302+0000 7f09a1c4c640 1 --2- 172.21.15.172:0/297874819 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f09840a2cc0 0x7f09840a5180 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.302+0000 7f09a1c4c640 1 --2- 172.21.15.172:0/297874819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f099c10de90 0x7f099c1a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.302+0000 7f09a1c4c640 1 --2- 172.21.15.172:0/297874819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f099c10b4e0 0x7f099c115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.302+0000 7f09a1c4c640 1 --2- 172.21.15.172:0/297874819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f099c072d80 0x7f099c115860 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.302+0000 7f09a1c4c640 1 -- 172.21.15.172:0/297874819 >> 172.21.15.172:0/297874819 conn(0x7f099c06eb50 msgr2=0x7f099c10c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.310+0000 7f09a1c4c640 1 -- 172.21.15.172:0/297874819 shutdown_connections 2024-09-10T09:02:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:32.310+0000 7f09a1c4c640 1 -- 172.21.15.172:0/297874819 wait complete. 2024-09-10T09:02:33.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:33 smithi174 ceph-mon[34068]: pgmap v839: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.41k op/s 2024-09-10T09:02:33.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:33 smithi191 ceph-mon[35314]: pgmap v839: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.41k op/s 2024-09-10T09:02:33.835 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:33 smithi172 ceph-mon[30854]: pgmap v839: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 4.41k op/s 2024-09-10T09:02:35.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:35 smithi174 ceph-mon[34068]: pgmap v840: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-10T09:02:35.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:35 smithi191 ceph-mon[35314]: pgmap v840: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-10T09:02:35.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:35 smithi172 ceph-mon[30854]: pgmap v840: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-10T09:02:37.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:37 smithi174 ceph-mon[34068]: pgmap v841: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.64k op/s 2024-09-10T09:02:37.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:37 smithi191 ceph-mon[35314]: pgmap v841: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.64k op/s 2024-09-10T09:02:37.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:37 smithi172 ceph-mon[30854]: pgmap v841: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 4.64k op/s 2024-09-10T09:02:39.640 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:39 smithi172 ceph-mon[30854]: pgmap v842: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.87k op/s 2024-09-10T09:02:39.641 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:39 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:02:39.744 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:39 smithi174 ceph-mon[34068]: pgmap v842: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.87k op/s 2024-09-10T09:02:39.744 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:39 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:02:39.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:39 smithi191 ceph-mon[35314]: pgmap v842: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 MiB/s wr, 3.87k op/s 2024-09-10T09:02:39.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:39 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:02:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:41 smithi174 ceph-mon[34068]: pgmap v843: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-10T09:02:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:41 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:02:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:41 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:02:41.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:41 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:02:41.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:41 smithi191 ceph-mon[35314]: pgmap v843: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-10T09:02:41.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:41 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:02:41.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:41 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:02:41.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:41 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:02:41.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:41 smithi172 ceph-mon[30854]: pgmap v843: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 3.86k op/s 2024-09-10T09:02:41.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:41 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:02:41.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:41 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:02:41.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:41 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:02:42.479 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:02:43.302 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:02:43.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:43 smithi174 ceph-mon[34068]: pgmap v844: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 4.44k op/s 2024-09-10T09:02:43.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:43 smithi191 ceph-mon[35314]: pgmap v844: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 4.44k op/s 2024-09-10T09:02:43.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:43 smithi172 ceph-mon[30854]: pgmap v844: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 4.44k op/s 2024-09-10T09:02:44.742 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:44 smithi174 ceph-mon[34068]: pgmap v845: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2024-09-10T09:02:44.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:44 smithi191 ceph-mon[35314]: pgmap v845: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2024-09-10T09:02:44.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:44 smithi172 ceph-mon[30854]: pgmap v845: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 3.75k op/s 2024-09-10T09:02:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.517+0000 7fb35e136640 1 -- 172.21.15.172:0/3897297420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb358073680 msgr2=0x7fb35810d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.517+0000 7fb35e136640 1 --2- 172.21.15.172:0/3897297420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb358073680 0x7fb35810d4f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb350009cd0 tx=0x7fb35002a930 comp rx=0 tx=0).stop 2024-09-10T09:02:45.523 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.522+0000 7fb35e136640 1 -- 172.21.15.172:0/3897297420 shutdown_connections 2024-09-10T09:02:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.522+0000 7fb35e136640 1 --2- 172.21.15.172:0/3897297420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb35810dbb0 0x7fb358110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.522+0000 7fb35e136640 1 --2- 172.21.15.172:0/3897297420 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb358073680 0x7fb35810d4f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.522+0000 7fb35e136640 1 --2- 172.21.15.172:0/3897297420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb358072cb0 0x7fb3580730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.522+0000 7fb35e136640 1 -- 172.21.15.172:0/3897297420 >> 172.21.15.172:0/3897297420 conn(0x7fb35806eb50 msgr2=0x7fb358070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.523+0000 7fb35e136640 1 -- 172.21.15.172:0/3897297420 shutdown_connections 2024-09-10T09:02:45.524 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.523+0000 7fb35e136640 1 -- 172.21.15.172:0/3897297420 wait complete. 2024-09-10T09:02:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.524+0000 7fb35e136640 1 Processor -- start 2024-09-10T09:02:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.524+0000 7fb35e136640 1 -- start start 2024-09-10T09:02:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.524+0000 7fb35e136640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb358072cb0 0x7fb3581167f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.525+0000 7fb35e136640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb358073680 0x7fb358116d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.525+0000 7fb35e136640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb35810dbb0 0x7fb358117d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.525+0000 7fb35e136640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb358110df0 con 0x7fb358073680 2024-09-10T09:02:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.525+0000 7fb35e136640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb358110c70 con 0x7fb35810dbb0 2024-09-10T09:02:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.525+0000 7fb35e136640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb358110f70 con 0x7fb358072cb0 2024-09-10T09:02:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.526+0000 7fb35d134640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb358072cb0 0x7fb3581167f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.526+0000 7fb35d134640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb358072cb0 0x7fb3581167f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:46866/0 (socket says 172.21.15.172:46866) 2024-09-10T09:02:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.526+0000 7fb35d134640 1 -- 172.21.15.172:0/944322615 learned_addr learned my addr 172.21.15.172:0/944322615 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.526+0000 7fb35c933640 1 --2- 172.21.15.172:0/944322615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb358073680 0x7fb358116d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.526+0000 7fb35c933640 1 -- 172.21.15.172:0/944322615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb358072cb0 msgr2=0x7fb3581167f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.526+0000 7fb35c933640 1 --2- 172.21.15.172:0/944322615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb358072cb0 0x7fb3581167f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.526+0000 7fb35c933640 1 -- 172.21.15.172:0/944322615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb35810dbb0 msgr2=0x7fb358117d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.526+0000 7fb35c933640 1 --2- 172.21.15.172:0/944322615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb35810dbb0 0x7fb358117d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.526+0000 7fb35c933640 1 -- 172.21.15.172:0/944322615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3581183e0 con 0x7fb358073680 2024-09-10T09:02:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.527+0000 7fb35c933640 1 --2- 172.21.15.172:0/944322615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb358073680 0x7fb358116d30 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fb350009bf0 tx=0x7fb350030bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.532+0000 7fb34e7fc640 1 -- 172.21.15.172:0/944322615 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350046680 con 0x7fb358073680 2024-09-10T09:02:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.532+0000 7fb34e7fc640 1 -- 172.21.15.172:0/944322615 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb35002d070 con 0x7fb358073680 2024-09-10T09:02:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.533+0000 7fb34e7fc640 1 -- 172.21.15.172:0/944322615 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35003bbd0 con 0x7fb358073680 2024-09-10T09:02:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.534+0000 7fb35e136640 1 -- 172.21.15.172:0/944322615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb358118670 con 0x7fb358073680 2024-09-10T09:02:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.534+0000 7fb35e136640 1 -- 172.21.15.172:0/944322615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb358118b90 con 0x7fb358073680 2024-09-10T09:02:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.540+0000 7fb362211740 1 Processor -- start 2024-09-10T09:02:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.542+0000 7fb362211740 1 -- start start 2024-09-10T09:02:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.542+0000 7fb362211740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e0ba0 0x5645594e0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.542+0000 7fb362211740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e14e0 0x5645594e5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.542+0000 7fb362211740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594e6570 0x5645594e8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.542+0000 7fb362211740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5645594442f0 con 0x5645594e0ba0 2024-09-10T09:02:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.542+0000 7fb362211740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5645593ec270 con 0x5645594e14e0 2024-09-10T09:02:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.542+0000 7fb362211740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5645594402b0 con 0x5645594e6570 2024-09-10T09:02:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.543+0000 7fb34e7fc640 1 -- 172.21.15.172:0/944322615 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb35003b400 con 0x7fb358073680 2024-09-10T09:02:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.544+0000 7fb34e7fc640 1 --2- 172.21.15.172:0/944322615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb3440a2d90 0x7fb3440a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.544+0000 7fb34e7fc640 1 -- 172.21.15.172:0/944322615 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb3500f0f40 con 0x7fb358073680 2024-09-10T09:02:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.544+0000 7fb35c933640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594e6570 0x5645594e8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.546+0000 7fb35d935640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e0ba0 0x5645594e0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.546+0000 7fb35d935640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e0ba0 0x5645594e0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39902/0 (socket says 172.21.15.172:39902) 2024-09-10T09:02:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.546+0000 7fb35d935640 1 -- 172.21.15.172:0/3684622817 learned_addr learned my addr 172.21.15.172:0/3684622817 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.546+0000 7fb35d134640 1 --2- 172.21.15.172:0/3684622817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e14e0 0x5645594e5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.549+0000 7fb35d935640 1 -- 172.21.15.172:0/3684622817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594e6570 msgr2=0x5645594e8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.549+0000 7fb35d935640 1 --2- 172.21.15.172:0/3684622817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594e6570 0x5645594e8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.549+0000 7fb35d935640 1 -- 172.21.15.172:0/3684622817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e14e0 msgr2=0x5645594e5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.549+0000 7fb35d935640 1 --2- 172.21.15.172:0/3684622817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e14e0 0x5645594e5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.551 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.549+0000 7fb35d935640 1 -- 172.21.15.172:0/3684622817 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645593eb340 con 0x5645594e0ba0 2024-09-10T09:02:45.552 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.550+0000 7fb35d935640 1 --2- 172.21.15.172:0/3684622817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e0ba0 0x5645594e0fa0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fb348005fd0 tx=0x7fb34800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5689c295829dc4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.551+0000 7fb3437fe640 1 -- 172.21.15.172:0/3684622817 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34800ac90 con 0x5645594e0ba0 2024-09-10T09:02:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.551+0000 7fb3437fe640 1 -- 172.21.15.172:0/3684622817 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb34800ae30 con 0x5645594e0ba0 2024-09-10T09:02:45.553 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.551+0000 7fb3437fe640 1 -- 172.21.15.172:0/3684622817 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3480107a0 con 0x5645594e0ba0 2024-09-10T09:02:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.554+0000 7fb362211740 1 -- 172.21.15.172:0/3684622817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e0ba0 msgr2=0x5645594e0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.554+0000 7fb362211740 1 --2- 172.21.15.172:0/3684622817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e0ba0 0x5645594e0fa0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fb348005fd0 tx=0x7fb34800f730 comp rx=0 tx=0).stop 2024-09-10T09:02:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.555+0000 7fb35d935640 1 --2- 172.21.15.172:0/944322615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb3440a2d90 0x7fb3440a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.559+0000 7fb35d935640 1 --2- 172.21.15.172:0/944322615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb3440a2d90 0x7fb3440a5250 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb348005fd0 tx=0x7fb34800f730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.561+0000 7fb362211740 1 -- 172.21.15.172:0/3684622817 shutdown_connections 2024-09-10T09:02:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.561+0000 7fb362211740 1 --2- 172.21.15.172:0/3684622817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594e6570 0x5645594e8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.561+0000 7fb362211740 1 --2- 172.21.15.172:0/3684622817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e14e0 0x5645594e5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.561+0000 7fb362211740 1 --2- 172.21.15.172:0/3684622817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e0ba0 0x5645594e0fa0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.561+0000 7fb362211740 1 -- 172.21.15.172:0/3684622817 >> 172.21.15.172:0/3684622817 conn(0x5645594dc210 msgr2=0x5645594dc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.561+0000 7fb362211740 1 -- 172.21.15.172:0/3684622817 shutdown_connections 2024-09-10T09:02:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.562+0000 7fb362211740 1 -- 172.21.15.172:0/3684622817 wait complete. 2024-09-10T09:02:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.562+0000 7fb362211740 1 -- start start 2024-09-10T09:02:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.562+0000 7fb362211740 1 Processor -- start 2024-09-10T09:02:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.563+0000 7fb362211740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e0ba0 0x5645594eae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.563+0000 7fb362211740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e14e0 0x5645594eb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.563+0000 7fb362211740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594e6570 0x5645594f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.563+0000 7fb362211740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5645594e9540 con 0x5645594e14e0 2024-09-10T09:02:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.563+0000 7fb362211740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5645593cd290 con 0x5645594e0ba0 2024-09-10T09:02:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.563+0000 7fb362211740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5645594e96c0 con 0x5645594e6570 2024-09-10T09:02:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.563+0000 7fb362211740 10 client.0 _ll_register_callbacks cb 0x5645593fe140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:02:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.563+0000 7fb35c933640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e14e0 0x5645594eb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.563+0000 7fb35c933640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e14e0 0x5645594eb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39916/0 (socket says 172.21.15.172:39916) 2024-09-10T09:02:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.563+0000 7fb35c933640 1 -- 172.21.15.172:0/3771772328 learned_addr learned my addr 172.21.15.172:0/3771772328 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.563+0000 7fb35d935640 1 --2- 172.21.15.172:0/3771772328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594e6570 0x5645594f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.563+0000 7fb35d134640 1 --2- 172.21.15.172:0/3771772328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e0ba0 0x5645594eae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.564+0000 7fb35c933640 1 -- 172.21.15.172:0/3771772328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594e6570 msgr2=0x5645594f23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.564+0000 7fb35c933640 1 --2- 172.21.15.172:0/3771772328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594e6570 0x5645594f23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.564+0000 7fb35c933640 1 -- 172.21.15.172:0/3771772328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e0ba0 msgr2=0x5645594eae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.564+0000 7fb35c933640 1 --2- 172.21.15.172:0/3771772328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e0ba0 0x5645594eae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.564+0000 7fb35c933640 1 -- 172.21.15.172:0/3771772328 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564559157210 con 0x5645594e14e0 2024-09-10T09:02:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.564+0000 7fb35d935640 1 --2- 172.21.15.172:0/3771772328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594e6570 0x5645594f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:02:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.564+0000 7fb35d134640 1 --2- 172.21.15.172:0/3771772328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e0ba0 0x5645594eae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:02:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.564+0000 7fb35c933640 1 --2- 172.21.15.172:0/3771772328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e14e0 0x5645594eb390 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fb3500318c0 tx=0x7fb350031e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d6cf73e443c378 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.564+0000 7fb340ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:02:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.564+0000 7fb340ff9640 1 -- 172.21.15.172:0/3771772328 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3500d5020 con 0x5645594e14e0 2024-09-10T09:02:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.564+0000 7fb340ff9640 1 -- 172.21.15.172:0/3771772328 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb35003bd70 con 0x5645594e14e0 2024-09-10T09:02:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.564+0000 7fb362211740 1 -- 172.21.15.172:0/3771772328 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb34800ae30 con 0x5645594e14e0 2024-09-10T09:02:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.564+0000 7fb340ff9640 1 -- 172.21.15.172:0/3771772328 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb3500bd3e0 con 0x5645594e14e0 2024-09-10T09:02:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.564+0000 7fb362211740 10 client.19677 fetch_fsmap learned FSMap version 44 2024-09-10T09:02:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.564+0000 7fb362211740 1 -- 172.21.15.172:0/3771772328 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564559223620 con 0x5645594e14e0 2024-09-10T09:02:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.565+0000 7fb340ff9640 1 -- 172.21.15.172:0/3771772328 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb34800ae30 con 0x5645594e14e0 2024-09-10T09:02:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.565+0000 7fb340ff9640 1 -- 172.21.15.172:0/3771772328 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fb35004e070 con 0x5645594e14e0 2024-09-10T09:02:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.565+0000 7fb362211740 10 client.19677 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:02:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.565+0000 7fb362211740 10 client.19677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:02:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.565+0000 7fb362211740 20 client.19677 populate_metadata read hostname 'smithi172' 2024-09-10T09:02:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.565+0000 7fb362211740 1 --2- 172.21.15.172:0/3771772328 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5645594f6ae0 0x5645594f8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.565+0000 7fb362211740 4 client.19677 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:02:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.565+0000 7fb362211740 1 -- 172.21.15.172:0/3771772328 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564559451a20 con 0x5645594f6ae0 2024-09-10T09:02:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.565+0000 7fb35d134640 1 --2- 172.21.15.172:0/3771772328 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5645594f6ae0 0x5645594f8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.566+0000 7fb35d134640 1 --2- 172.21.15.172:0/3771772328 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5645594f6ae0 0x5645594f8f30 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8687857c3cba261 server_cookie=ec4219851e2e33ec in_seq=0 out_seq=0 2024-09-10T09:02:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.566+0000 7fb340ff9640 10 client.19677 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.567+0000 7fb340ff9640 1 -- 172.21.15.172:0/3771772328 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564559451a20 con 0x5645594f6ae0 2024-09-10T09:02:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.567+0000 7fb340ff9640 10 client.19677 handle_command_reply: tid=0 2024-09-10T09:02:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.567+0000 7fb362211740 1 client.19677 shutdown 2024-09-10T09:02:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.567+0000 7fb362211740 1 -- 172.21.15.172:0/3771772328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e14e0 msgr2=0x5645594eb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.567+0000 7fb362211740 1 --2- 172.21.15.172:0/3771772328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e14e0 0x5645594eb390 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fb3500318c0 tx=0x7fb350031e70 comp rx=0 tx=0).stop 2024-09-10T09:02:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.567+0000 7fb362211740 1 -- 172.21.15.172:0/3771772328 shutdown_connections 2024-09-10T09:02:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.567+0000 7fb362211740 1 --2- 172.21.15.172:0/3771772328 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5645594f6ae0 0x5645594f8f30 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.567+0000 7fb362211740 1 --2- 172.21.15.172:0/3771772328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594e6570 0x5645594f23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.567+0000 7fb362211740 1 --2- 172.21.15.172:0/3771772328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e14e0 0x5645594eb390 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.567+0000 7fb362211740 1 --2- 172.21.15.172:0/3771772328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e0ba0 0x5645594eae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.567+0000 7fb362211740 1 -- 172.21.15.172:0/3771772328 >> 172.21.15.172:0/3771772328 conn(0x5645594dc210 msgr2=0x5645594dda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.567+0000 7fb340ff9640 0 client.19677 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.567+0000 7fb362211740 1 -- 172.21.15.172:0/3771772328 shutdown_connections 2024-09-10T09:02:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.567+0000 7fb362211740 1 -- 172.21.15.172:0/3771772328 wait complete. 2024-09-10T09:02:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.567+0000 7fb362211740 20 client.19677 trim_cache size 0 max 16384 2024-09-10T09:02:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.581+0000 7fb362211740 1 Processor -- start 2024-09-10T09:02:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.581+0000 7fb362211740 1 -- start start 2024-09-10T09:02:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.581+0000 7fb362211740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594dc210 0x5645594daae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.581+0000 7fb362211740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e0ba0 0x5645594f2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.582+0000 7fb362211740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e14e0 0x5645594db020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.582+0000 7fb362211740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581c1930 con 0x5645594e14e0 2024-09-10T09:02:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.582+0000 7fb362211740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581c3470 con 0x5645594e0ba0 2024-09-10T09:02:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.582+0000 7fb362211740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581c2000 con 0x5645594dc210 2024-09-10T09:02:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.582+0000 7fb35d935640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e14e0 0x5645594db020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.582+0000 7fb35d935640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e14e0 0x5645594db020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39920/0 (socket says 172.21.15.172:39920) 2024-09-10T09:02:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.582+0000 7fb35d935640 1 -- 172.21.15.172:0/118252039 learned_addr learned my addr 172.21.15.172:0/118252039 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.582+0000 7fb35c933640 1 --2- 172.21.15.172:0/118252039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e0ba0 0x5645594f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.582+0000 7fb35d134640 1 --2- 172.21.15.172:0/118252039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594dc210 0x5645594daae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.582+0000 7fb35d935640 1 -- 172.21.15.172:0/118252039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594dc210 msgr2=0x5645594daae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.582+0000 7fb35d935640 1 --2- 172.21.15.172:0/118252039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594dc210 0x5645594daae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.582+0000 7fb35d935640 1 -- 172.21.15.172:0/118252039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e0ba0 msgr2=0x5645594f2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:45.585 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.582+0000 7fb35d935640 1 --2- 172.21.15.172:0/118252039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e0ba0 0x5645594f2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.582+0000 7fb35d935640 1 -- 172.21.15.172:0/118252039 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645594dc030 con 0x5645594e14e0 2024-09-10T09:02:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.582+0000 7fb35d935640 1 --2- 172.21.15.172:0/118252039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e14e0 0x5645594db020 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb3480108f0 tx=0x7fb34801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd51e2d820c32b78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:45.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.694+0000 7fb342ffd640 1 -- 172.21.15.172:0/118252039 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3480103d0 con 0x5645594e14e0 2024-09-10T09:02:45.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.694+0000 7fb342ffd640 1 -- 172.21.15.172:0/118252039 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb348010570 con 0x5645594e14e0 2024-09-10T09:02:45.696 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.694+0000 7fb342ffd640 1 -- 172.21.15.172:0/118252039 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3480047a0 con 0x5645594e14e0 2024-09-10T09:02:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.695+0000 7fb362211740 1 -- 172.21.15.172:0/118252039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e14e0 msgr2=0x5645594db020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.695+0000 7fb362211740 1 --2- 172.21.15.172:0/118252039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e14e0 0x5645594db020 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb3480108f0 tx=0x7fb34801f000 comp rx=0 tx=0).stop 2024-09-10T09:02:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.695+0000 7fb362211740 1 -- 172.21.15.172:0/118252039 shutdown_connections 2024-09-10T09:02:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.695+0000 7fb362211740 1 --2- 172.21.15.172:0/118252039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594e14e0 0x5645594db020 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.695+0000 7fb362211740 1 --2- 172.21.15.172:0/118252039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e0ba0 0x5645594f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.699 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.695+0000 7fb362211740 1 --2- 172.21.15.172:0/118252039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594dc210 0x5645594daae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.695+0000 7fb362211740 1 -- 172.21.15.172:0/118252039 >> 172.21.15.172:0/118252039 conn(0x5645594e6100 msgr2=0x5645594eae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.695+0000 7fb362211740 1 -- 172.21.15.172:0/118252039 shutdown_connections 2024-09-10T09:02:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.695+0000 7fb362211740 1 -- 172.21.15.172:0/118252039 wait complete. 2024-09-10T09:02:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.695+0000 7fb362211740 1 -- start start 2024-09-10T09:02:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb362211740 1 Processor -- start 2024-09-10T09:02:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb362211740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594dc210 0x5645594f2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb362211740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e0ba0 0x5645594eee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb362211740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594e14e0 0x5645594ef490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb362211740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581c2da0 con 0x5645594dc210 2024-09-10T09:02:45.700 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb362211740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581c26d0 con 0x5645594e0ba0 2024-09-10T09:02:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb362211740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581bf910 con 0x5645594e14e0 2024-09-10T09:02:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb362211740 10 client.0 _ll_register_callbacks cb 0x7fb3500b7f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:02:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb35d134640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594dc210 0x5645594f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb35d134640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594dc210 0x5645594f2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:39924/0 (socket says 172.21.15.172:39924) 2024-09-10T09:02:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb35d134640 1 -- 172.21.15.172:0/1455231073 learned_addr learned my addr 172.21.15.172:0/1455231073 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb35d935640 1 --2- 172.21.15.172:0/1455231073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594e14e0 0x5645594ef490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb35c933640 1 --2- 172.21.15.172:0/1455231073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e0ba0 0x5645594eee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb35d134640 1 -- 172.21.15.172:0/1455231073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594e14e0 msgr2=0x5645594ef490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb35d134640 1 --2- 172.21.15.172:0/1455231073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594e14e0 0x5645594ef490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb35d134640 1 -- 172.21.15.172:0/1455231073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e0ba0 msgr2=0x5645594eee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb35d134640 1 --2- 172.21.15.172:0/1455231073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e0ba0 0x5645594eee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.696+0000 7fb35d134640 1 -- 172.21.15.172:0/1455231073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56455955f8d0 con 0x5645594dc210 2024-09-10T09:02:45.701 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.697+0000 7fb35d134640 1 --2- 172.21.15.172:0/1455231073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594dc210 0x5645594f2390 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fb354004810 tx=0x7fb354016b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624bd3ad373e6414 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:45.702 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.697+0000 7fb340ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:02:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.703+0000 7fb340ff9640 1 -- 172.21.15.172:0/1455231073 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35402a060 con 0x5645594dc210 2024-09-10T09:02:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.703+0000 7fb340ff9640 1 -- 172.21.15.172:0/1455231073 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb354014960 con 0x5645594dc210 2024-09-10T09:02:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.703+0000 7fb362211740 1 -- 172.21.15.172:0/1455231073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb348010570 con 0x5645594dc210 2024-09-10T09:02:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.703+0000 7fb340ff9640 1 -- 172.21.15.172:0/1455231073 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb354031050 con 0x5645594dc210 2024-09-10T09:02:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.703+0000 7fb362211740 10 client.19683 fetch_fsmap learned FSMap version 44 2024-09-10T09:02:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.703+0000 7fb362211740 1 -- 172.21.15.172:0/1455231073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564559560cb0 con 0x5645594dc210 2024-09-10T09:02:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.704+0000 7fb340ff9640 1 -- 172.21.15.172:0/1455231073 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb348010570 con 0x5645594dc210 2024-09-10T09:02:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.704+0000 7fb340ff9640 1 -- 172.21.15.172:0/1455231073 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fb354024030 con 0x5645594dc210 2024-09-10T09:02:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.704+0000 7fb362211740 10 client.19683 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:02:45.706 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.704+0000 7fb362211740 10 client.19683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:02:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.704+0000 7fb362211740 20 client.19683 populate_metadata read hostname 'smithi172' 2024-09-10T09:02:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.704+0000 7fb362211740 1 --2- 172.21.15.172:0/1455231073 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5645594e6570 0x564559563680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.704+0000 7fb362211740 4 client.19683 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:02:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.704+0000 7fb362211740 1 -- 172.21.15.172:0/1455231073 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56455944bd70 con 0x5645594e6570 2024-09-10T09:02:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.704+0000 7fb35c933640 1 --2- 172.21.15.172:0/1455231073 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5645594e6570 0x564559563680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.705+0000 7fb340ff9640 10 client.19683 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:45.707 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:02:45.705+0000 7fb35c933640 1 --2- 172.21.15.172:0/1455231073 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5645594e6570 0x564559563680 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73abb7decebc025b server_cookie=11b6cd1929cb680c in_seq=0 out_seq=0 2024-09-10T09:02:45.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:45 smithi172 ceph-mon[30854]: pgmap v846: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-09-10T09:02:45.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:45 smithi174 ceph-mon[34068]: pgmap v846: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-09-10T09:02:46.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:45 smithi191 ceph-mon[35314]: pgmap v846: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 4.36k op/s 2024-09-10T09:02:47.658 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:47 smithi172 ceph-mon[30854]: pgmap v847: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 3.62k op/s 2024-09-10T09:02:47.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:47 smithi174 ceph-mon[34068]: pgmap v847: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 3.62k op/s 2024-09-10T09:02:48.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:47 smithi191 ceph-mon[35314]: pgmap v847: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 3.62k op/s 2024-09-10T09:02:49.666 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:49 smithi172 ceph-mon[30854]: pgmap v848: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-10T09:02:49.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:49 smithi174 ceph-mon[34068]: pgmap v848: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-10T09:02:50.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:49 smithi191 ceph-mon[35314]: pgmap v848: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 3.45k op/s 2024-09-10T09:02:51.114 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:02:51.114 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:02:51.114 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T09:02:51.114 INFO:teuthology.orchestra.run.smithi172.stdout: "097ab02a-d7be-4e1d-96ec-78f5d8c3836d": { 2024-09-10T09:02:51.115 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T09:02:51.115 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "097ab02a-d7be-4e1d-96ec-78f5d8c3836d", 2024-09-10T09:02:51.115 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T09:02:51.115 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:02:51.115 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:02:51.115 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:02:51.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.535+0000 7f125e110640 1 -- 172.21.15.172:0/2295633881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1258101de0 msgr2=0x7f12581021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:51.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.535+0000 7f125e110640 1 --2- 172.21.15.172:0/2295633881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1258101de0 0x7f12581021c0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f1248009900 tx=0x7f124802a410 comp rx=0 tx=0).stop 2024-09-10T09:02:51.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.535+0000 7f125e110640 1 -- 172.21.15.172:0/2295633881 shutdown_connections 2024-09-10T09:02:51.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.535+0000 7f125e110640 1 --2- 172.21.15.172:0/2295633881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1258106f10 0x7f125810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.535+0000 7f125e110640 1 --2- 172.21.15.172:0/2295633881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1258102790 0x7f12581067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.535+0000 7f125e110640 1 --2- 172.21.15.172:0/2295633881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1258101de0 0x7f12581021c0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.535+0000 7f125e110640 1 -- 172.21.15.172:0/2295633881 >> 172.21.15.172:0/2295633881 conn(0x7f12580fd8c0 msgr2=0x7f12580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:51.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.535+0000 7f125e110640 1 -- 172.21.15.172:0/2295633881 shutdown_connections 2024-09-10T09:02:51.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.535+0000 7f125e110640 1 -- 172.21.15.172:0/2295633881 wait complete. 2024-09-10T09:02:51.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.535+0000 7f125e110640 1 Processor -- start 2024-09-10T09:02:51.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.535+0000 7f125e110640 1 -- start start 2024-09-10T09:02:51.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.535+0000 7f125e110640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1258101de0 0x7f1258198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f125e110640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1258102790 0x7f12581988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f125e110640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1258106f10 0x7f125819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f125e110640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1258111e20 con 0x7f1258101de0 2024-09-10T09:02:51.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f125e110640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1258111ca0 con 0x7f1258102790 2024-09-10T09:02:51.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f125e110640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1258111fa0 con 0x7f1258106f10 2024-09-10T09:02:51.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f12577fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1258101de0 0x7f1258198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:51.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f12577fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1258101de0 0x7f1258198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46566/0 (socket says 172.21.15.172:46566) 2024-09-10T09:02:51.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f12577fe640 1 -- 172.21.15.172:0/3636855901 learned_addr learned my addr 172.21.15.172:0/3636855901 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:51.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f1257fff640 1 --2- 172.21.15.172:0/3636855901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1258106f10 0x7f125819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:51.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f1256ffd640 1 --2- 172.21.15.172:0/3636855901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1258102790 0x7f12581988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:51.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f12577fe640 1 -- 172.21.15.172:0/3636855901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1258106f10 msgr2=0x7f125819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:51.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f12577fe640 1 --2- 172.21.15.172:0/3636855901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1258106f10 0x7f125819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f12577fe640 1 -- 172.21.15.172:0/3636855901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1258102790 msgr2=0x7f12581988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:51.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f12577fe640 1 --2- 172.21.15.172:0/3636855901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1258102790 0x7f12581988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f12577fe640 1 -- 172.21.15.172:0/3636855901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125819d330 con 0x7f1258101de0 2024-09-10T09:02:51.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f12577fe640 1 --2- 172.21.15.172:0/3636855901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1258101de0 0x7f1258198380 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f1248005940 tx=0x7f12480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:51.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f1254ff9640 1 -- 172.21.15.172:0/3636855901 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124803f030 con 0x7f1258101de0 2024-09-10T09:02:51.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f1254ff9640 1 -- 172.21.15.172:0/3636855901 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1248003cb0 con 0x7f1258101de0 2024-09-10T09:02:51.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f1254ff9640 1 -- 172.21.15.172:0/3636855901 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1248035070 con 0x7f1258101de0 2024-09-10T09:02:51.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f125e110640 1 -- 172.21.15.172:0/3636855901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125819d5c0 con 0x7f1258101de0 2024-09-10T09:02:51.118 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.536+0000 7f125e110640 1 -- 172.21.15.172:0/3636855901 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12581a4f00 con 0x7f1258101de0 2024-09-10T09:02:51.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.539+0000 7f1254ff9640 1 -- 172.21.15.172:0/3636855901 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1248042030 con 0x7f1258101de0 2024-09-10T09:02:51.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.539+0000 7f12604e9740 1 Processor -- start 2024-09-10T09:02:51.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.539+0000 7f12604e9740 1 -- start start 2024-09-10T09:02:51.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.539+0000 7f1254ff9640 1 --2- 172.21.15.172:0/3636855901 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12400a2c00 0x7f12400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.539+0000 7f12604e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152753340 0x557152753720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.539+0000 7f12604e9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753c60 0x557152762550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.539+0000 7f12604e9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152762a90 0x557152764e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.539+0000 7f12604e9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5571526b92e0 con 0x557152753c60 2024-09-10T09:02:51.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.539+0000 7f12604e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557152660ce0 con 0x557152753340 2024-09-10T09:02:51.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.539+0000 7f12604e9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5571526b52a0 con 0x557152762a90 2024-09-10T09:02:51.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.539+0000 7f1257fff640 1 --2- 172.21.15.172:0/3636855901 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12400a2c00 0x7f12400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:51.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.540+0000 7f1256ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753c60 0x557152762550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:51.119 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.540+0000 7f1254ff9640 1 -- 172.21.15.172:0/3636855901 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f12480efa80 con 0x7f1258101de0 2024-09-10T09:02:51.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.540+0000 7f1256ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753c60 0x557152762550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46568/0 (socket says 172.21.15.172:46568) 2024-09-10T09:02:51.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.540+0000 7f1256ffd640 1 -- 172.21.15.172:0/3687120641 learned_addr learned my addr 172.21.15.172:0/3687120641 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:51.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.540+0000 7f12577fe640 1 --2- 172.21.15.172:0/3687120641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152753340 0x557152753720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:51.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.540+0000 7f1256ffd640 1 -- 172.21.15.172:0/3687120641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152762a90 msgr2=0x557152764e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:51.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.540+0000 7f1256ffd640 1 --2- 172.21.15.172:0/3687120641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152762a90 0x557152764e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.540+0000 7f1256ffd640 1 -- 172.21.15.172:0/3687120641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152753340 msgr2=0x557152753720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:51.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.540+0000 7f1256ffd640 1 --2- 172.21.15.172:0/3687120641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152753340 0x557152753720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.540+0000 7f1256ffd640 1 -- 172.21.15.172:0/3687120641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557152645440 con 0x557152753c60 2024-09-10T09:02:51.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.540+0000 7f1257fff640 1 --2- 172.21.15.172:0/3636855901 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12400a2c00 0x7f12400a50c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1250005fd0 tx=0x7f1250011810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:51.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.540+0000 7f1256ffd640 1 --2- 172.21.15.172:0/3687120641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753c60 0x557152762550 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f124c00b690 tx=0x7f124c00bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e59b79bcf0d601 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:51.120 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.540+0000 7f123dffb640 1 -- 172.21.15.172:0/3687120641 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c003bd0 con 0x557152753c60 2024-09-10T09:02:51.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.540+0000 7f123dffb640 1 -- 172.21.15.172:0/3687120641 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f124c015070 con 0x557152753c60 2024-09-10T09:02:51.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.540+0000 7f123dffb640 1 -- 172.21.15.172:0/3687120641 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c004d40 con 0x557152753c60 2024-09-10T09:02:51.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.541+0000 7f12604e9740 1 -- 172.21.15.172:0/3687120641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753c60 msgr2=0x557152762550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:51.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.541+0000 7f12604e9740 1 --2- 172.21.15.172:0/3687120641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753c60 0x557152762550 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f124c00b690 tx=0x7f124c00bb50 comp rx=0 tx=0).stop 2024-09-10T09:02:51.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.541+0000 7f12604e9740 1 -- 172.21.15.172:0/3687120641 shutdown_connections 2024-09-10T09:02:51.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.541+0000 7f12604e9740 1 --2- 172.21.15.172:0/3687120641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152762a90 0x557152764e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.541+0000 7f12604e9740 1 --2- 172.21.15.172:0/3687120641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753c60 0x557152762550 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.541+0000 7f12604e9740 1 --2- 172.21.15.172:0/3687120641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152753340 0x557152753720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.541+0000 7f12604e9740 1 -- 172.21.15.172:0/3687120641 >> 172.21.15.172:0/3687120641 conn(0x55715274fa30 msgr2=0x55715274fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:51.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.541+0000 7f12604e9740 1 -- 172.21.15.172:0/3687120641 shutdown_connections 2024-09-10T09:02:51.121 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.541+0000 7f12604e9740 1 -- 172.21.15.172:0/3687120641 wait complete. 2024-09-10T09:02:51.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.541+0000 7f12604e9740 1 -- start start 2024-09-10T09:02:51.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.541+0000 7f12604e9740 1 Processor -- start 2024-09-10T09:02:51.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12604e9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753340 0x557152762420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12604e9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152753c60 0x55715275f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12604e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152762a90 0x55715275fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12604e9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557152765660 con 0x557152753340 2024-09-10T09:02:51.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12604e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557152674050 con 0x557152762a90 2024-09-10T09:02:51.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12604e9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5571527657e0 con 0x557152753c60 2024-09-10T09:02:51.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12604e9740 10 client.0 _ll_register_callbacks cb 0x5571526b4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:02:51.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12577fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753340 0x557152762420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:51.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f1256ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152753c60 0x55715275f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:51.122 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12577fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753340 0x557152762420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46572/0 (socket says 172.21.15.172:46572) 2024-09-10T09:02:51.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12577fe640 1 -- 172.21.15.172:0/671421169 learned_addr learned my addr 172.21.15.172:0/671421169 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:51.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f1257fff640 1 --2- 172.21.15.172:0/671421169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152762a90 0x55715275fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:51.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12577fe640 1 -- 172.21.15.172:0/671421169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152753c60 msgr2=0x55715275f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:51.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12577fe640 1 --2- 172.21.15.172:0/671421169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152753c60 0x55715275f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12577fe640 1 -- 172.21.15.172:0/671421169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152762a90 msgr2=0x55715275fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:51.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12577fe640 1 --2- 172.21.15.172:0/671421169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152762a90 0x55715275fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12577fe640 1 -- 172.21.15.172:0/671421169 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55715242a3c0 con 0x557152753340 2024-09-10T09:02:51.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12577fe640 1 --2- 172.21.15.172:0/671421169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753340 0x557152762420 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f12480b8490 tx=0x7f12480bbc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbb3238eed1663a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:51.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f1236ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:02:51.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f1236ffd640 1 -- 172.21.15.172:0/671421169 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1248049040 con 0x557152753340 2024-09-10T09:02:51.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f1236ffd640 1 -- 172.21.15.172:0/671421169 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f12480b86a0 con 0x557152753340 2024-09-10T09:02:51.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12604e9740 1 -- 172.21.15.172:0/671421169 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f124c015070 con 0x557152753340 2024-09-10T09:02:51.123 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f1236ffd640 1 -- 172.21.15.172:0/671421169 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1248005d50 con 0x557152753340 2024-09-10T09:02:51.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12604e9740 10 client.19695 fetch_fsmap learned FSMap version 44 2024-09-10T09:02:51.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.542+0000 7f12604e9740 1 -- 172.21.15.172:0/671421169 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557152771090 con 0x557152753340 2024-09-10T09:02:51.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.543+0000 7f1236ffd640 1 -- 172.21.15.172:0/671421169 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f124c015070 con 0x557152753340 2024-09-10T09:02:51.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.543+0000 7f1236ffd640 1 -- 172.21.15.172:0/671421169 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f124808e960 con 0x557152753340 2024-09-10T09:02:51.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.543+0000 7f12604e9740 10 client.19695 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:02:51.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.543+0000 7f12604e9740 10 client.19695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:02:51.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.543+0000 7f12604e9740 20 client.19695 populate_metadata read hostname 'smithi172' 2024-09-10T09:02:51.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.543+0000 7f12604e9740 1 --2- 172.21.15.172:0/671421169 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557152772660 0x557152774a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.543+0000 7f12604e9740 4 client.19695 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:02:51.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.543+0000 7f12604e9740 1 -- 172.21.15.172:0/671421169 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571526c6850 con 0x557152772660 2024-09-10T09:02:51.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.543+0000 7f1256ffd640 1 --2- 172.21.15.172:0/671421169 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557152772660 0x557152774a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:51.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.543+0000 7f1236ffd640 10 client.19695 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:51.124 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.544+0000 7f1256ffd640 1 --2- 172.21.15.172:0/671421169 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557152772660 0x557152774a50 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d468313ab769678 server_cookie=6079be04af6ee601 in_seq=0 out_seq=0 2024-09-10T09:02:51.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.544+0000 7f1236ffd640 1 -- 172.21.15.172:0/671421169 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571526c6850 con 0x557152772660 2024-09-10T09:02:51.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.544+0000 7f1236ffd640 10 client.19695 handle_command_reply: tid=0 2024-09-10T09:02:51.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.544+0000 7f12604e9740 1 client.19695 shutdown 2024-09-10T09:02:51.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.544+0000 7f12604e9740 1 -- 172.21.15.172:0/671421169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753340 msgr2=0x557152762420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:51.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.544+0000 7f12604e9740 1 --2- 172.21.15.172:0/671421169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753340 0x557152762420 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f12480b8490 tx=0x7f12480bbc30 comp rx=0 tx=0).stop 2024-09-10T09:02:51.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.544+0000 7f12604e9740 1 -- 172.21.15.172:0/671421169 shutdown_connections 2024-09-10T09:02:51.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.544+0000 7f12604e9740 1 --2- 172.21.15.172:0/671421169 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557152772660 0x557152774a50 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.544+0000 7f12604e9740 1 --2- 172.21.15.172:0/671421169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152762a90 0x55715275fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.544+0000 7f12604e9740 1 --2- 172.21.15.172:0/671421169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152753c60 0x55715275f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.544+0000 7f12604e9740 1 --2- 172.21.15.172:0/671421169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753340 0x557152762420 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.544+0000 7f12604e9740 1 -- 172.21.15.172:0/671421169 >> 172.21.15.172:0/671421169 conn(0x55715274fa30 msgr2=0x557152755790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:51.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.544+0000 7f1236ffd640 0 client.19695 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:51.125 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.544+0000 7f12604e9740 1 -- 172.21.15.172:0/671421169 shutdown_connections 2024-09-10T09:02:51.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.544+0000 7f12604e9740 1 -- 172.21.15.172:0/671421169 wait complete. 2024-09-10T09:02:51.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.545+0000 7f12604e9740 20 client.19695 trim_cache size 0 max 16384 2024-09-10T09:02:51.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.562+0000 7f12604e9740 1 Processor -- start 2024-09-10T09:02:51.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.562+0000 7f12604e9740 1 -- start start 2024-09-10T09:02:51.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.562+0000 7f12604e9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753340 0x557152774c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.562+0000 7f12604e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152753c60 0x55715276e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.562+0000 7f12604e9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152762a90 0x55715277a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.562+0000 7f12604e9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f12581ac910 con 0x557152753340 2024-09-10T09:02:51.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.562+0000 7f12604e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12581ae450 con 0x557152753c60 2024-09-10T09:02:51.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.562+0000 7f12604e9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f12581acfe0 con 0x557152762a90 2024-09-10T09:02:51.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.562+0000 7f12577fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753340 0x557152774c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:51.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.562+0000 7f12577fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753340 0x557152774c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46586/0 (socket says 172.21.15.172:46586) 2024-09-10T09:02:51.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.562+0000 7f1256ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152753c60 0x55715276e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:51.126 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.562+0000 7f12577fe640 1 -- 172.21.15.172:0/2094237945 learned_addr learned my addr 172.21.15.172:0/2094237945 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:51.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.562+0000 7f1257fff640 1 --2- 172.21.15.172:0/2094237945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152762a90 0x55715277a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:51.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.563+0000 7f1256ffd640 1 -- 172.21.15.172:0/2094237945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152762a90 msgr2=0x55715277a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:51.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.563+0000 7f1256ffd640 1 --2- 172.21.15.172:0/2094237945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152762a90 0x55715277a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.563+0000 7f1256ffd640 1 -- 172.21.15.172:0/2094237945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753340 msgr2=0x557152774c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:51.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.563+0000 7f1256ffd640 1 --2- 172.21.15.172:0/2094237945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753340 0x557152774c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.563+0000 7f1256ffd640 1 -- 172.21.15.172:0/2094237945 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55715275ffc0 con 0x557152753c60 2024-09-10T09:02:51.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.563+0000 7f12577fe640 1 --2- 172.21.15.172:0/2094237945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753340 0x557152774c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:02:51.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.563+0000 7f1257fff640 1 --2- 172.21.15.172:0/2094237945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152762a90 0x55715277a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:02:51.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.563+0000 7f1256ffd640 1 --2- 172.21.15.172:0/2094237945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152753c60 0x55715276e900 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f124c004960 tx=0x7f124c005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d286bee5cec0711 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:51.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.563+0000 7f123d7fa640 1 -- 172.21.15.172:0/2094237945 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c003dd0 con 0x557152753c60 2024-09-10T09:02:51.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.563+0000 7f123d7fa640 1 -- 172.21.15.172:0/2094237945 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f124c015040 con 0x557152753c60 2024-09-10T09:02:51.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.563+0000 7f123d7fa640 1 -- 172.21.15.172:0/2094237945 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c02f6a0 con 0x557152753c60 2024-09-10T09:02:51.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.564+0000 7f12604e9740 1 -- 172.21.15.172:0/2094237945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152753c60 msgr2=0x55715276e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:51.127 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.564+0000 7f12604e9740 1 --2- 172.21.15.172:0/2094237945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152753c60 0x55715276e900 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f124c004960 tx=0x7f124c005c50 comp rx=0 tx=0).stop 2024-09-10T09:02:51.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.564+0000 7f12604e9740 1 -- 172.21.15.172:0/2094237945 shutdown_connections 2024-09-10T09:02:51.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.564+0000 7f12604e9740 1 --2- 172.21.15.172:0/2094237945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152762a90 0x55715277a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.564+0000 7f12604e9740 1 --2- 172.21.15.172:0/2094237945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152753c60 0x55715276e900 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.564+0000 7f12604e9740 1 --2- 172.21.15.172:0/2094237945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152753340 0x557152774c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.564+0000 7f12604e9740 1 -- 172.21.15.172:0/2094237945 >> 172.21.15.172:0/2094237945 conn(0x55715274fa30 msgr2=0x557152774580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:51.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.564+0000 7f12604e9740 1 -- 172.21.15.172:0/2094237945 shutdown_connections 2024-09-10T09:02:51.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.564+0000 7f12604e9740 1 -- 172.21.15.172:0/2094237945 wait complete. 2024-09-10T09:02:51.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.564+0000 7f12604e9740 1 -- start start 2024-09-10T09:02:51.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.564+0000 7f12604e9740 1 Processor -- start 2024-09-10T09:02:51.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.564+0000 7f12604e9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152753340 0x55715276b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.564+0000 7f12604e9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152753c60 0x5571527665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.564+0000 7f12604e9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152762a90 0x557152766b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.128 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.564+0000 7f12604e9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f12581add80 con 0x557152762a90 2024-09-10T09:02:51.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.564+0000 7f12604e9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12581ad6b0 con 0x557152753c60 2024-09-10T09:02:51.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.564+0000 7f12604e9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f12581aa8f0 con 0x557152753340 2024-09-10T09:02:51.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.565+0000 7f12604e9740 10 client.0 _ll_register_callbacks cb 0x7f1248005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:02:51.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.565+0000 7f1257fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152762a90 0x557152766b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:51.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.565+0000 7f1257fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152762a90 0x557152766b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46590/0 (socket says 172.21.15.172:46590) 2024-09-10T09:02:51.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.565+0000 7f1256ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152753c60 0x5571527665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:51.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.565+0000 7f1257fff640 1 -- 172.21.15.172:0/1350200393 learned_addr learned my addr 172.21.15.172:0/1350200393 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:02:51.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.565+0000 7f12577fe640 1 --2- 172.21.15.172:0/1350200393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152753340 0x55715276b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:51.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.565+0000 7f1257fff640 1 -- 172.21.15.172:0/1350200393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152753340 msgr2=0x55715276b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:51.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.565+0000 7f1257fff640 1 --2- 172.21.15.172:0/1350200393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152753340 0x55715276b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.565+0000 7f1257fff640 1 -- 172.21.15.172:0/1350200393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152753c60 msgr2=0x5571527665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:02:51.129 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.565+0000 7f1257fff640 1 --2- 172.21.15.172:0/1350200393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152753c60 0x5571527665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.565+0000 7f1257fff640 1 -- 172.21.15.172:0/1350200393 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557152767350 con 0x557152762a90 2024-09-10T09:02:51.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.565+0000 7f1257fff640 1 --2- 172.21.15.172:0/1350200393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152762a90 0x557152766b20 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f1250004280 tx=0x7f1250004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c1a134e177c1bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:02:51.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.565+0000 7f1236ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:02:51.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.637+0000 7f1236ffd640 1 -- 172.21.15.172:0/1350200393 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250017030 con 0x557152762a90 2024-09-10T09:02:51.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.637+0000 7f1236ffd640 1 -- 172.21.15.172:0/1350200393 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1250018070 con 0x557152762a90 2024-09-10T09:02:51.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.637+0000 7f12604e9740 1 -- 172.21.15.172:0/1350200393 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f124c015040 con 0x557152762a90 2024-09-10T09:02:51.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.639+0000 7f1236ffd640 1 -- 172.21.15.172:0/1350200393 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1250004b20 con 0x557152762a90 2024-09-10T09:02:51.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.639+0000 7f12604e9740 10 client.19701 fetch_fsmap learned FSMap version 44 2024-09-10T09:02:51.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.639+0000 7f12604e9740 1 -- 172.21.15.172:0/1350200393 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5571527dd360 con 0x557152762a90 2024-09-10T09:02:51.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.641+0000 7f1236ffd640 1 -- 172.21.15.172:0/1350200393 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1250019070 con 0x557152762a90 2024-09-10T09:02:51.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.641+0000 7f1236ffd640 1 -- 172.21.15.172:0/1350200393 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f1250028030 con 0x557152762a90 2024-09-10T09:02:51.130 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.641+0000 7f12604e9740 10 client.19701 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:02:51.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.641+0000 7f12604e9740 10 client.19701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:02:51.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.641+0000 7f12604e9740 20 client.19701 populate_metadata read hostname 'smithi172' 2024-09-10T09:02:51.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.641+0000 7f12604e9740 1 --2- 172.21.15.172:0/1350200393 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5571527dda90 0x5571527dfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:02:51.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.641+0000 7f12604e9740 4 client.19701 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:02:51.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.641+0000 7f12604e9740 1 -- 172.21.15.172:0/1350200393 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f124c015040 con 0x5571527dda90 2024-09-10T09:02:51.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.641+0000 7f12577fe640 1 --2- 172.21.15.172:0/1350200393 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5571527dda90 0x5571527dfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:02:51.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.642+0000 7f12577fe640 1 --2- 172.21.15.172:0/1350200393 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5571527dda90 0x5571527dfeb0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4e004f50b4e2841 server_cookie=b366a66762d5d5a1 in_seq=0 out_seq=0 2024-09-10T09:02:51.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.642+0000 7f1236ffd640 10 client.19701 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:51.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.642+0000 7f1236ffd640 1 -- 172.21.15.172:0/1350200393 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f124c015040 con 0x5571527dda90 2024-09-10T09:02:51.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.642+0000 7f1236ffd640 10 client.19701 handle_command_reply: tid=0 2024-09-10T09:02:51.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.642+0000 7f12604e9740 1 client.19701 shutdown 2024-09-10T09:02:51.131 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.642+0000 7f12604e9740 1 -- 172.21.15.172:0/1350200393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152762a90 msgr2=0x557152766b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:51.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.642+0000 7f12604e9740 1 --2- 172.21.15.172:0/1350200393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152762a90 0x557152766b20 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f1250004280 tx=0x7f1250004ab0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.643+0000 7f12604e9740 1 -- 172.21.15.172:0/1350200393 shutdown_connections 2024-09-10T09:02:51.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.643+0000 7f12604e9740 1 --2- 172.21.15.172:0/1350200393 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5571527dda90 0x5571527dfeb0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.643+0000 7f12604e9740 1 --2- 172.21.15.172:0/1350200393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557152762a90 0x557152766b20 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.643+0000 7f1236ffd640 0 client.19701 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:02:51.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.643+0000 7f12604e9740 1 --2- 172.21.15.172:0/1350200393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557152753c60 0x5571527665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.643+0000 7f12604e9740 1 --2- 172.21.15.172:0/1350200393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557152753340 0x55715276b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.643+0000 7f12604e9740 1 -- 172.21.15.172:0/1350200393 >> 172.21.15.172:0/1350200393 conn(0x55715274fa30 msgr2=0x55715274e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:51.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.643+0000 7f12604e9740 1 -- 172.21.15.172:0/1350200393 shutdown_connections 2024-09-10T09:02:51.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.643+0000 7f12604e9740 1 -- 172.21.15.172:0/1350200393 wait complete. 2024-09-10T09:02:51.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.643+0000 7f12604e9740 20 client.19701 trim_cache size 0 max 16384 2024-09-10T09:02:51.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.644+0000 7f125e110640 1 -- 172.21.15.172:0/3636855901 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12400a2c00 msgr2=0x7f12400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:51.132 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.644+0000 7f125e110640 1 --2- 172.21.15.172:0/3636855901 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12400a2c00 0x7f12400a50c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1250005fd0 tx=0x7f1250011810 comp rx=0 tx=0).stop 2024-09-10T09:02:51.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.644+0000 7f125e110640 1 -- 172.21.15.172:0/3636855901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1258101de0 msgr2=0x7f1258198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:02:51.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.644+0000 7f125e110640 1 --2- 172.21.15.172:0/3636855901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1258101de0 0x7f1258198380 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f1248005940 tx=0x7f12480042f0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.644+0000 7f125e110640 1 -- 172.21.15.172:0/3636855901 shutdown_connections 2024-09-10T09:02:51.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.644+0000 7f125e110640 1 --2- 172.21.15.172:0/3636855901 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12400a2c00 0x7f12400a50c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.644+0000 7f125e110640 1 --2- 172.21.15.172:0/3636855901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1258106f10 0x7f125819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.644+0000 7f125e110640 1 --2- 172.21.15.172:0/3636855901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1258102790 0x7f12581988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.644+0000 7f125e110640 1 --2- 172.21.15.172:0/3636855901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1258101de0 0x7f1258198380 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:02:51.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.644+0000 7f125e110640 1 -- 172.21.15.172:0/3636855901 >> 172.21.15.172:0/3636855901 conn(0x7f12580fd8c0 msgr2=0x7f12580fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:02:51.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.644+0000 7f125e110640 1 -- 172.21.15.172:0/3636855901 shutdown_connections 2024-09-10T09:02:51.133 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:02:49.644+0000 7f125e110640 1 -- 172.21.15.172:0/3636855901 wait complete. 2024-09-10T09:02:51.834 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:51 smithi172 ceph-mon[30854]: pgmap v849: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 3.65k op/s 2024-09-10T09:02:51.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:51 smithi174 ceph-mon[34068]: pgmap v849: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 3.65k op/s 2024-09-10T09:02:52.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:51 smithi191 ceph-mon[35314]: pgmap v849: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 3.65k op/s 2024-09-10T09:02:53.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:53 smithi174 ceph-mon[34068]: pgmap v850: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 2.99k op/s 2024-09-10T09:02:54.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:53 smithi191 ceph-mon[35314]: pgmap v850: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 2.99k op/s 2024-09-10T09:02:54.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:53 smithi172 ceph-mon[30854]: pgmap v850: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 2.99k op/s 2024-09-10T09:02:55.837 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:55 smithi172 ceph-mon[30854]: pgmap v851: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.39k op/s 2024-09-10T09:02:55.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:55 smithi174 ceph-mon[34068]: pgmap v851: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.39k op/s 2024-09-10T09:02:56.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:55 smithi191 ceph-mon[35314]: pgmap v851: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 3.39k op/s 2024-09-10T09:02:57.570 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:57 smithi172 ceph-mon[30854]: pgmap v852: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 2.60k op/s 2024-09-10T09:02:57.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:57 smithi174 ceph-mon[34068]: pgmap v852: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 2.60k op/s 2024-09-10T09:02:58.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:57 smithi191 ceph-mon[35314]: pgmap v852: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 2.60k op/s 2024-09-10T09:02:59.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:02:59 smithi172 ceph-mon[30854]: pgmap v853: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 2.59k op/s 2024-09-10T09:02:59.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:02:59 smithi174 ceph-mon[34068]: pgmap v853: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 2.59k op/s 2024-09-10T09:03:00.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:02:59 smithi191 ceph-mon[35314]: pgmap v853: 129 pgs: 129 active+clean; 13 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 2.59k op/s 2024-09-10T09:03:01.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:01 smithi172 ceph-mon[30854]: pgmap v854: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T09:03:01.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:01 smithi174 ceph-mon[34068]: pgmap v854: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T09:03:02.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:01 smithi191 ceph-mon[35314]: pgmap v854: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T09:03:02.494 INFO:tasks.cephfs.filesystem:scrub status for tag:097ab02a-d7be-4e1d-96ec-78f5d8c3836d - {'path': '/', 'tag': '097ab02a-d7be-4e1d-96ec-78f5d8c3836d', 'options': 'recursive,force'} 2024-09-10T09:03:03.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:03 smithi174 ceph-mon[34068]: pgmap v855: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 2.71k op/s 2024-09-10T09:03:04.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:03 smithi191 ceph-mon[35314]: pgmap v855: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 2.71k op/s 2024-09-10T09:03:04.084 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:03 smithi172 ceph-mon[30854]: pgmap v855: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 MiB/s wr, 2.71k op/s 2024-09-10T09:03:05.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.032+0000 7fb340ff9640 1 -- 172.21.15.172:0/1455231073 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56455944bd70 con 0x5645594e6570 2024-09-10T09:03:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.032+0000 7fb340ff9640 10 client.19683 handle_command_reply: tid=0 2024-09-10T09:03:05.033 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.032+0000 7fb362211740 1 client.19683 shutdown 2024-09-10T09:03:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.035+0000 7fb362211740 1 -- 172.21.15.172:0/1455231073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594dc210 msgr2=0x5645594f2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.035+0000 7fb362211740 1 --2- 172.21.15.172:0/1455231073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594dc210 0x5645594f2390 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fb354004810 tx=0x7fb354016b40 comp rx=0 tx=0).stop 2024-09-10T09:03:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.035+0000 7fb362211740 1 -- 172.21.15.172:0/1455231073 shutdown_connections 2024-09-10T09:03:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.035+0000 7fb362211740 1 --2- 172.21.15.172:0/1455231073 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5645594e6570 0x564559563680 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.035+0000 7fb362211740 1 --2- 172.21.15.172:0/1455231073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5645594e14e0 0x5645594ef490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.035+0000 7fb362211740 1 --2- 172.21.15.172:0/1455231073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645594e0ba0 0x5645594eee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.035+0000 7fb362211740 1 --2- 172.21.15.172:0/1455231073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5645594dc210 0x5645594f2390 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.035+0000 7fb362211740 1 -- 172.21.15.172:0/1455231073 >> 172.21.15.172:0/1455231073 conn(0x5645594e6100 msgr2=0x5645594dd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.035+0000 7fb340ff9640 0 client.19683 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:03:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.035+0000 7fb362211740 1 -- 172.21.15.172:0/1455231073 shutdown_connections 2024-09-10T09:03:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.035+0000 7fb362211740 1 -- 172.21.15.172:0/1455231073 wait complete. 2024-09-10T09:03:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.036+0000 7fb362211740 20 client.19683 trim_cache size 0 max 16384 2024-09-10T09:03:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:03:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:03:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:03:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:03:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.044+0000 7fb35e136640 1 -- 172.21.15.172:0/944322615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb3440a2d90 msgr2=0x7fb3440a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.044+0000 7fb35e136640 1 --2- 172.21.15.172:0/944322615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb3440a2d90 0x7fb3440a5250 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb348005fd0 tx=0x7fb34800f730 comp rx=0 tx=0).stop 2024-09-10T09:03:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.044+0000 7fb35e136640 1 -- 172.21.15.172:0/944322615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb358073680 msgr2=0x7fb358116d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:05.045 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.044+0000 7fb35e136640 1 --2- 172.21.15.172:0/944322615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb358073680 0x7fb358116d30 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fb350009bf0 tx=0x7fb350030bb0 comp rx=0 tx=0).stop 2024-09-10T09:03:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.048+0000 7fb35e136640 1 -- 172.21.15.172:0/944322615 shutdown_connections 2024-09-10T09:03:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.048+0000 7fb35e136640 1 --2- 172.21.15.172:0/944322615 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb3440a2d90 0x7fb3440a5250 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.048+0000 7fb35e136640 1 --2- 172.21.15.172:0/944322615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb35810dbb0 0x7fb358117d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.048+0000 7fb35e136640 1 --2- 172.21.15.172:0/944322615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb358073680 0x7fb358116d30 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.048+0000 7fb35e136640 1 --2- 172.21.15.172:0/944322615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb358072cb0 0x7fb3581167f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.048+0000 7fb35e136640 1 -- 172.21.15.172:0/944322615 >> 172.21.15.172:0/944322615 conn(0x7fb35806eb50 msgr2=0x7fb358070920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.056+0000 7fb35e136640 1 -- 172.21.15.172:0/944322615 shutdown_connections 2024-09-10T09:03:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:05.059+0000 7fb35e136640 1 -- 172.21.15.172:0/944322615 wait complete. 2024-09-10T09:03:06.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:06 smithi172 ceph-mon[30854]: pgmap v856: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 3.37k op/s 2024-09-10T09:03:06.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:06 smithi174 ceph-mon[34068]: pgmap v856: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 3.37k op/s 2024-09-10T09:03:07.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:06 smithi191 ceph-mon[35314]: pgmap v856: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 3.37k op/s 2024-09-10T09:03:08.579 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:08 smithi172 ceph-mon[30854]: pgmap v857: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 12 MiB/s wr, 3.04k op/s 2024-09-10T09:03:08.852 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:08 smithi191 ceph-mon[35314]: pgmap v857: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 12 MiB/s wr, 3.04k op/s 2024-09-10T09:03:08.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:08 smithi174 ceph-mon[34068]: pgmap v857: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 12 MiB/s wr, 3.04k op/s 2024-09-10T09:03:10.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:10 smithi174 ceph-mon[34068]: pgmap v858: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 MiB/s wr, 3.25k op/s 2024-09-10T09:03:11.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:10 smithi191 ceph-mon[35314]: pgmap v858: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 MiB/s wr, 3.25k op/s 2024-09-10T09:03:11.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:10 smithi172 ceph-mon[30854]: pgmap v858: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 13 MiB/s wr, 3.25k op/s 2024-09-10T09:03:12.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:12 smithi174 ceph-mon[34068]: pgmap v859: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 3.72k op/s 2024-09-10T09:03:13.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:12 smithi191 ceph-mon[35314]: pgmap v859: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 3.72k op/s 2024-09-10T09:03:13.084 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:12 smithi172 ceph-mon[30854]: pgmap v859: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 MiB/s wr, 3.72k op/s 2024-09-10T09:03:13.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:13 smithi174 ceph-mon[34068]: pgmap v860: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 13 MiB/s wr, 3.35k op/s 2024-09-10T09:03:14.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:13 smithi191 ceph-mon[35314]: pgmap v860: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 13 MiB/s wr, 3.35k op/s 2024-09-10T09:03:14.085 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:13 smithi172 ceph-mon[30854]: pgmap v860: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 13 MiB/s wr, 3.35k op/s 2024-09-10T09:03:16.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:16 smithi174 ceph-mon[34068]: pgmap v861: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 4.21k op/s 2024-09-10T09:03:17.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:16 smithi191 ceph-mon[35314]: pgmap v861: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 4.21k op/s 2024-09-10T09:03:17.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:16 smithi172 ceph-mon[30854]: pgmap v861: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 17 MiB/s wr, 4.21k op/s 2024-09-10T09:03:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.262+0000 7f11a61cf640 1 -- 172.21.15.172:0/1621109679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11a0109700 msgr2=0x7f11a0109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.262+0000 7f11a61cf640 1 --2- 172.21.15.172:0/1621109679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11a0109700 0x7f11a0109b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1194009900 tx=0x7f119402a420 comp rx=0 tx=0).stop 2024-09-10T09:03:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.267+0000 7f11a61cf640 1 -- 172.21.15.172:0/1621109679 shutdown_connections 2024-09-10T09:03:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.267+0000 7f11a61cf640 1 --2- 172.21.15.172:0/1621109679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11a00736c0 0x7f11a0119130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.267+0000 7f11a61cf640 1 --2- 172.21.15.172:0/1621109679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0072d20 0x7f11a0073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.267+0000 7f11a61cf640 1 --2- 172.21.15.172:0/1621109679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11a0109700 0x7f11a0109b00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.267+0000 7f11a61cf640 1 -- 172.21.15.172:0/1621109679 >> 172.21.15.172:0/1621109679 conn(0x7f11a006ece0 msgr2=0x7f11a0071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.269+0000 7f11a61cf640 1 -- 172.21.15.172:0/1621109679 shutdown_connections 2024-09-10T09:03:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.269+0000 7f11a61cf640 1 -- 172.21.15.172:0/1621109679 wait complete. 2024-09-10T09:03:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.270+0000 7f11a61cf640 1 Processor -- start 2024-09-10T09:03:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.270+0000 7f11a61cf640 1 -- start start 2024-09-10T09:03:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.270+0000 7f11a61cf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0072d20 0x7f11a0074800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.270+0000 7f11a61cf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11a00736c0 0x7f11a0074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.270+0000 7f11a61cf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11a0109700 0x7f11a007bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.270+0000 7f11a61cf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11a011c510 con 0x7f11a0072d20 2024-09-10T09:03:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.270+0000 7f11a61cf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f11a011c390 con 0x7f11a0109700 2024-09-10T09:03:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.270+0000 7f11a61cf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f11a011c690 con 0x7f11a00736c0 2024-09-10T09:03:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.271+0000 7f119f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0072d20 0x7f11a0074800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.271+0000 7f119f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0072d20 0x7f11a0074800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:50108/0 (socket says 172.21.15.172:50108) 2024-09-10T09:03:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.271+0000 7f119f7fe640 1 -- 172.21.15.172:0/1580134330 learned_addr learned my addr 172.21.15.172:0/1580134330 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:03:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.272+0000 7f119ffff640 1 --2- 172.21.15.172:0/1580134330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11a0109700 0x7f11a007bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.272+0000 7f119ffff640 1 -- 172.21.15.172:0/1580134330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11a00736c0 msgr2=0x7f11a0074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.272+0000 7f119ffff640 1 --2- 172.21.15.172:0/1580134330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11a00736c0 0x7f11a0074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.272+0000 7f119ffff640 1 -- 172.21.15.172:0/1580134330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0072d20 msgr2=0x7f11a0074800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.272+0000 7f119ffff640 1 --2- 172.21.15.172:0/1580134330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0072d20 0x7f11a0074800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.272+0000 7f119ffff640 1 -- 172.21.15.172:0/1580134330 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a007c420 con 0x7f11a0109700 2024-09-10T09:03:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.273+0000 7f119ffff640 1 --2- 172.21.15.172:0/1580134330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11a0109700 0x7f11a007bdb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f119800e940 tx=0x7f119800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.274+0000 7f119cff9640 1 -- 172.21.15.172:0/1580134330 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198004d20 con 0x7f11a0109700 2024-09-10T09:03:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.274+0000 7f11a61cf640 1 -- 172.21.15.172:0/1580134330 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a007c710 con 0x7f11a0109700 2024-09-10T09:03:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.275+0000 7f11a61cf640 1 -- 172.21.15.172:0/1580134330 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a007cc60 con 0x7f11a0109700 2024-09-10T09:03:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.275+0000 7f119cff9640 1 -- 172.21.15.172:0/1580134330 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1198005020 con 0x7f11a0109700 2024-09-10T09:03:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.275+0000 7f119cff9640 1 -- 172.21.15.172:0/1580134330 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11980116d0 con 0x7f11a0109700 2024-09-10T09:03:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.287+0000 7f11a9047740 1 Processor -- start 2024-09-10T09:03:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.287+0000 7f11a9047740 1 -- start start 2024-09-10T09:03:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.288+0000 7f11a9047740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389c99a0 0x5609389c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.288+0000 7f11a9047740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389ca2e0 0x5609389ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.288+0000 7f11a9047740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389cf370 0x5609389d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.288+0000 7f11a9047740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56093892d2f0 con 0x5609389cf370 2024-09-10T09:03:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.288+0000 7f11a9047740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5609388d5270 con 0x5609389ca2e0 2024-09-10T09:03:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.288+0000 7f11a9047740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5609389292b0 con 0x5609389c99a0 2024-09-10T09:03:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.288+0000 7f119cff9640 1 -- 172.21.15.172:0/1580134330 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1198011870 con 0x7f11a0109700 2024-09-10T09:03:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.288+0000 7f119effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389c99a0 0x5609389c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.288+0000 7f119effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389c99a0 0x5609389c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:46434/0 (socket says 172.21.15.172:46434) 2024-09-10T09:03:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.288+0000 7f119effd640 1 -- 172.21.15.172:0/3966205863 learned_addr learned my addr 172.21.15.172:0/3966205863 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:03:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.288+0000 7f119effd640 1 -- 172.21.15.172:0/3966205863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389ca2e0 msgr2=0x5609389ceda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:03:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.288+0000 7f119effd640 1 --2- 172.21.15.172:0/3966205863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389ca2e0 0x5609389ceda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.288+0000 7f119effd640 1 -- 172.21.15.172:0/3966205863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389cf370 msgr2=0x5609389d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.288+0000 7f119effd640 1 --2- 172.21.15.172:0/3966205863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389cf370 0x5609389d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.288+0000 7f119effd640 1 -- 172.21.15.172:0/3966205863 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609388d4340 con 0x5609389c99a0 2024-09-10T09:03:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.289+0000 7f119effd640 1 --2- 172.21.15.172:0/3966205863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389c99a0 0x5609389c9da0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1190001790 tx=0x7f1190001c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56a460dda916f9a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.289+0000 7f119cff9640 1 --2- 172.21.15.172:0/1580134330 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11880a2e60 0x7f11880a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.290+0000 7f119cff9640 1 -- 172.21.15.172:0/1580134330 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f11980d03b0 con 0x7f11a0109700 2024-09-10T09:03:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.293+0000 7f1185ffb640 1 -- 172.21.15.172:0/3966205863 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190004be0 con 0x5609389c99a0 2024-09-10T09:03:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.293+0000 7f1185ffb640 1 -- 172.21.15.172:0/3966205863 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1190008e30 con 0x5609389c99a0 2024-09-10T09:03:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.294+0000 7f11a9047740 1 -- 172.21.15.172:0/3966205863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389c99a0 msgr2=0x5609389c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.294+0000 7f11a9047740 1 --2- 172.21.15.172:0/3966205863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389c99a0 0x5609389c9da0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1190001790 tx=0x7f1190001c50 comp rx=0 tx=0).stop 2024-09-10T09:03:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.294+0000 7f11a9047740 1 -- 172.21.15.172:0/3966205863 shutdown_connections 2024-09-10T09:03:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.294+0000 7f11a9047740 1 --2- 172.21.15.172:0/3966205863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389cf370 0x5609389d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.294+0000 7f11a9047740 1 --2- 172.21.15.172:0/3966205863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389ca2e0 0x5609389ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.294+0000 7f11a9047740 1 --2- 172.21.15.172:0/3966205863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389c99a0 0x5609389c9da0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.294+0000 7f11a9047740 1 -- 172.21.15.172:0/3966205863 >> 172.21.15.172:0/3966205863 conn(0x5609389c5010 msgr2=0x5609389c5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.296+0000 7f11a9047740 1 -- 172.21.15.172:0/3966205863 shutdown_connections 2024-09-10T09:03:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.296+0000 7f119f7fe640 1 --2- 172.21.15.172:0/1580134330 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11880a2e60 0x7f11880a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.296+0000 7f11a9047740 1 -- 172.21.15.172:0/3966205863 wait complete. 2024-09-10T09:03:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.297+0000 7f11a9047740 1 -- start start 2024-09-10T09:03:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.297+0000 7f11a9047740 1 Processor -- start 2024-09-10T09:03:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.297+0000 7f119f7fe640 1 --2- 172.21.15.172:0/1580134330 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11880a2e60 0x7f11880a5320 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5609389cad10 tx=0x7f1194005d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.297+0000 7f11a9047740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389c99a0 0x5609389d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.298+0000 7f11a9047740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389cf370 0x5609389d4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.298+0000 7f119f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389c99a0 0x5609389d3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.298+0000 7f119f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389c99a0 0x5609389d3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:36920/0 (socket says 172.21.15.172:36920) 2024-09-10T09:03:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.298+0000 7f119f7fe640 1 -- 172.21.15.172:0/4255289385 learned_addr learned my addr 172.21.15.172:0/4255289385 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:03:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.298+0000 7f119effd640 1 --2- 172.21.15.172:0/4255289385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389cf370 0x5609389d4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.298+0000 7f11a9047740 1 --2- 172.21.15.172:0/4255289385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389d9290 0x5609389db680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.298+0000 7f11a9047740 1 -- 172.21.15.172:0/4255289385 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5609389d2390 con 0x5609389d9290 2024-09-10T09:03:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.298+0000 7f11a9047740 1 -- 172.21.15.172:0/4255289385 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5609388b6290 con 0x5609389c99a0 2024-09-10T09:03:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.298+0000 7f11a9047740 1 -- 172.21.15.172:0/4255289385 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5609389d2510 con 0x5609389cf370 2024-09-10T09:03:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.299+0000 7f119effd640 1 -- 172.21.15.172:0/4255289385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389c99a0 msgr2=0x5609389d3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.299+0000 7f119effd640 1 --2- 172.21.15.172:0/4255289385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389c99a0 0x5609389d3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.299+0000 7f119ffff640 1 --2- 172.21.15.172:0/4255289385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389d9290 0x5609389db680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.299+0000 7f119effd640 1 -- 172.21.15.172:0/4255289385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389d9290 msgr2=0x5609389db680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.299+0000 7f119effd640 1 --2- 172.21.15.172:0/4255289385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389d9290 0x5609389db680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.299+0000 7f119effd640 1 -- 172.21.15.172:0/4255289385 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560938640210 con 0x5609389cf370 2024-09-10T09:03:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.299+0000 7f119ffff640 1 --2- 172.21.15.172:0/4255289385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389d9290 0x5609389db680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:03:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.299+0000 7f11a9047740 10 client.0 _ll_register_callbacks cb 0x5609388e7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:03:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.299+0000 7f11a9047740 1 -- 172.21.15.172:0/4255289385 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1190008e30 con 0x5609389cf370 2024-09-10T09:03:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.304+0000 7f119effd640 1 --2- 172.21.15.172:0/4255289385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389cf370 0x5609389d4340 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1190001760 tx=0x7f1190008bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1b178288dcd2ed9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.304+0000 7f117effd640 10 client.28399 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:03:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.308+0000 7f117effd640 1 -- 172.21.15.172:0/4255289385 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190023060 con 0x5609389cf370 2024-09-10T09:03:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.308+0000 7f117effd640 1 -- 172.21.15.172:0/4255289385 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f119000ed10 con 0x5609389cf370 2024-09-10T09:03:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.308+0000 7f117effd640 1 -- 172.21.15.172:0/4255289385 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1190000f40 con 0x5609389cf370 2024-09-10T09:03:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.308+0000 7f11a9047740 10 client.28399 fetch_fsmap learned FSMap version 44 2024-09-10T09:03:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.308+0000 7f11a9047740 1 -- 172.21.15.172:0/4255289385 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({fsmap=44}) -- 0x56093870c620 con 0x5609389cf370 2024-09-10T09:03:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.311+0000 7f117effd640 1 -- 172.21.15.172:0/4255289385 <== mon.1 v2:172.21.15.191:3300/0 4 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f11900049e0 con 0x5609389cf370 2024-09-10T09:03:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.313+0000 7f11a9047740 10 client.28399 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:03:18.313 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.313+0000 7f11a9047740 10 client.28399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:03:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.313+0000 7f11a9047740 20 client.28399 populate_metadata read hostname 'smithi172' 2024-09-10T09:03:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.313+0000 7f11a9047740 1 --2- 172.21.15.172:0/4255289385 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5609389df740 0x5609389e1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.313+0000 7f11a9047740 4 client.28399 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:03:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.313+0000 7f11a9047740 1 -- 172.21.15.172:0/4255289385 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56093893aa20 con 0x5609389df740 2024-09-10T09:03:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.313+0000 7f119f7fe640 1 --2- 172.21.15.172:0/4255289385 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5609389df740 0x5609389e1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.314+0000 7f119f7fe640 1 --2- 172.21.15.172:0/4255289385 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5609389df740 0x5609389e1c00 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=212d2db9b575c973 server_cookie=905e01b0d60c6fcb in_seq=0 out_seq=0 2024-09-10T09:03:18.315 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.314+0000 7f117effd640 10 client.28399 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:03:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.320+0000 7f117effd640 1 -- 172.21.15.172:0/4255289385 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56093893aa20 con 0x5609389df740 2024-09-10T09:03:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.320+0000 7f117effd640 10 client.28399 handle_command_reply: tid=0 2024-09-10T09:03:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.320+0000 7f11a9047740 1 client.28399 shutdown 2024-09-10T09:03:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.320+0000 7f11a9047740 1 -- 172.21.15.172:0/4255289385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389cf370 msgr2=0x5609389d4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.320+0000 7f11a9047740 1 --2- 172.21.15.172:0/4255289385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389cf370 0x5609389d4340 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1190001760 tx=0x7f1190008bb0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.320+0000 7f11a9047740 1 -- 172.21.15.172:0/4255289385 shutdown_connections 2024-09-10T09:03:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.320+0000 7f11a9047740 1 --2- 172.21.15.172:0/4255289385 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5609389df740 0x5609389e1c00 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.321+0000 7f11a9047740 1 --2- 172.21.15.172:0/4255289385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389d9290 0x5609389db680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.321+0000 7f11a9047740 1 --2- 172.21.15.172:0/4255289385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389cf370 0x5609389d4340 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.321+0000 7f11a9047740 1 --2- 172.21.15.172:0/4255289385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389c99a0 0x5609389d3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.321+0000 7f11a9047740 1 -- 172.21.15.172:0/4255289385 >> 172.21.15.172:0/4255289385 conn(0x5609389c5010 msgr2=0x5609389c41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:18.322 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.321+0000 7f117effd640 0 client.28399 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:03:18.323 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.323+0000 7f11a9047740 1 -- 172.21.15.172:0/4255289385 shutdown_connections 2024-09-10T09:03:18.324 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.323+0000 7f11a9047740 1 -- 172.21.15.172:0/4255289385 wait complete. 2024-09-10T09:03:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.328+0000 7f11a9047740 20 client.28399 trim_cache size 0 max 16384 2024-09-10T09:03:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.357+0000 7f11a9047740 1 Processor -- start 2024-09-10T09:03:18.358 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.357+0000 7f11a9047740 1 -- start start 2024-09-10T09:03:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.358+0000 7f11a9047740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389c99a0 0x5609389d42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.358+0000 7f11a9047740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389cf370 0x5609389c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.358+0000 7f119f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389c99a0 0x5609389d42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.358+0000 7f119f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389c99a0 0x5609389d42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:36928/0 (socket says 172.21.15.172:36928) 2024-09-10T09:03:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.358+0000 7f119f7fe640 1 -- 172.21.15.172:0/3642872377 learned_addr learned my addr 172.21.15.172:0/3642872377 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:03:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.359+0000 7f11a9047740 1 --2- 172.21.15.172:0/3642872377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389d9290 0x5609389c6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.359+0000 7f119effd640 1 --2- 172.21.15.172:0/3642872377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389cf370 0x5609389c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.359+0000 7f11a9047740 1 -- 172.21.15.172:0/3642872377 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f118c00fbd0 con 0x5609389d9290 2024-09-10T09:03:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.359+0000 7f11a9047740 1 -- 172.21.15.172:0/3642872377 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f118c011710 con 0x5609389c99a0 2024-09-10T09:03:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.359+0000 7f11a9047740 1 -- 172.21.15.172:0/3642872377 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f118c0102a0 con 0x5609389cf370 2024-09-10T09:03:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.359+0000 7f119ffff640 1 --2- 172.21.15.172:0/3642872377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389d9290 0x5609389c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.360+0000 7f119f7fe640 1 -- 172.21.15.172:0/3642872377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389cf370 msgr2=0x5609389c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.360+0000 7f119f7fe640 1 --2- 172.21.15.172:0/3642872377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389cf370 0x5609389c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.360+0000 7f119f7fe640 1 -- 172.21.15.172:0/3642872377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389d9290 msgr2=0x5609389c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.360+0000 7f119f7fe640 1 --2- 172.21.15.172:0/3642872377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389d9290 0x5609389c6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.360+0000 7f119f7fe640 1 -- 172.21.15.172:0/3642872377 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609389d1760 con 0x5609389c99a0 2024-09-10T09:03:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.360+0000 7f119f7fe640 1 --2- 172.21.15.172:0/3642872377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389c99a0 0x5609389d42c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f11940096c0 tx=0x7f1194041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23df338695e36079 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.364+0000 7f11857fa640 1 -- 172.21.15.172:0/3642872377 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194035040 con 0x5609389c99a0 2024-09-10T09:03:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.364+0000 7f11857fa640 1 -- 172.21.15.172:0/3642872377 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1194057070 con 0x5609389c99a0 2024-09-10T09:03:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.364+0000 7f11857fa640 1 -- 172.21.15.172:0/3642872377 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11940517f0 con 0x5609389c99a0 2024-09-10T09:03:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.366+0000 7f11a9047740 1 -- 172.21.15.172:0/3642872377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389c99a0 msgr2=0x5609389d42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.366+0000 7f11a9047740 1 --2- 172.21.15.172:0/3642872377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389c99a0 0x5609389d42c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f11940096c0 tx=0x7f1194041000 comp rx=0 tx=0).stop 2024-09-10T09:03:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.366+0000 7f11a9047740 1 -- 172.21.15.172:0/3642872377 shutdown_connections 2024-09-10T09:03:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.366+0000 7f11a9047740 1 --2- 172.21.15.172:0/3642872377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389d9290 0x5609389c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.366+0000 7f11a9047740 1 --2- 172.21.15.172:0/3642872377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389cf370 0x5609389c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.366+0000 7f11a9047740 1 --2- 172.21.15.172:0/3642872377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389c99a0 0x5609389d42c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.366+0000 7f11a9047740 1 -- 172.21.15.172:0/3642872377 >> 172.21.15.172:0/3642872377 conn(0x5609389c5010 msgr2=0x5609389d3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.369+0000 7f11a9047740 1 -- 172.21.15.172:0/3642872377 shutdown_connections 2024-09-10T09:03:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.369+0000 7f11a9047740 1 -- 172.21.15.172:0/3642872377 wait complete. 2024-09-10T09:03:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.370+0000 7f11a9047740 1 -- start start 2024-09-10T09:03:18.371 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.370+0000 7f11a9047740 1 Processor -- start 2024-09-10T09:03:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.371+0000 7f11a9047740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389c99a0 0x5609389f1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.371+0000 7f11a9047740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389cf370 0x5609389f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.372 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.371+0000 7f119f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389c99a0 0x5609389f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.371+0000 7f119f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389c99a0 0x5609389f1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:46466/0 (socket says 172.21.15.172:46466) 2024-09-10T09:03:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.371+0000 7f119f7fe640 1 -- 172.21.15.172:0/1775567412 learned_addr learned my addr 172.21.15.172:0/1775567412 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:03:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.371+0000 7f119effd640 1 --2- 172.21.15.172:0/1775567412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389cf370 0x5609389f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.371+0000 7f11a9047740 1 --2- 172.21.15.172:0/1775567412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389d9290 0x5609389f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.371+0000 7f11a9047740 1 -- 172.21.15.172:0/1775567412 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f118c011040 con 0x5609389d9290 2024-09-10T09:03:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.371+0000 7f11a9047740 1 -- 172.21.15.172:0/1775567412 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f118c010970 con 0x5609389cf370 2024-09-10T09:03:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.371+0000 7f11a9047740 1 -- 172.21.15.172:0/1775567412 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f118c00dbb0 con 0x5609389c99a0 2024-09-10T09:03:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.372+0000 7f11a9047740 10 client.0 _ll_register_callbacks cb 0x7f1190001660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:03:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.372+0000 7f119effd640 1 -- 172.21.15.172:0/1775567412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389c99a0 msgr2=0x5609389f1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.372+0000 7f119effd640 1 --2- 172.21.15.172:0/1775567412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389c99a0 0x5609389f1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.372+0000 7f119effd640 1 -- 172.21.15.172:0/1775567412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389d9290 msgr2=0x5609389f2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.372+0000 7f119effd640 1 --2- 172.21.15.172:0/1775567412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389d9290 0x5609389f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.372+0000 7f119effd640 1 -- 172.21.15.172:0/1775567412 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609389f3340 con 0x5609389cf370 2024-09-10T09:03:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.372+0000 7f119effd640 1 --2- 172.21.15.172:0/1775567412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389cf370 0x5609389f1dc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1190001730 tx=0x7f1190009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd7bf84735e63151 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.374+0000 7f117effd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:03:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.374+0000 7f117effd640 1 -- 172.21.15.172:0/1775567412 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190004030 con 0x5609389cf370 2024-09-10T09:03:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.374+0000 7f11a9047740 1 -- 172.21.15.172:0/1775567412 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1194057070 con 0x5609389cf370 2024-09-10T09:03:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.374+0000 7f117effd640 1 -- 172.21.15.172:0/1775567412 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f11900041d0 con 0x5609389cf370 2024-09-10T09:03:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.380+0000 7f117effd640 1 -- 172.21.15.172:0/1775567412 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f119003a050 con 0x5609389cf370 2024-09-10T09:03:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.381+0000 7f11a9047740 10 client.28508 fetch_fsmap learned FSMap version 44 2024-09-10T09:03:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.381+0000 7f11a9047740 1 -- 172.21.15.172:0/1775567412 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560938a4a8d0 con 0x5609389cf370 2024-09-10T09:03:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.385+0000 7f117effd640 1 -- 172.21.15.172:0/1775567412 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1194057070 con 0x5609389cf370 2024-09-10T09:03:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.387+0000 7f117effd640 1 -- 172.21.15.172:0/1775567412 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f119002f020 con 0x5609389cf370 2024-09-10T09:03:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.387+0000 7f11a9047740 10 client.28508 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:03:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.387+0000 7f11a9047740 10 client.28508 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:03:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.387+0000 7f11a9047740 20 client.28508 populate_metadata read hostname 'smithi172' 2024-09-10T09:03:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.387+0000 7f11a9047740 1 --2- 172.21.15.172:0/1775567412 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560938a4b1a0 0x560938a4d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.387+0000 7f11a9047740 4 client.28508 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:03:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.387+0000 7f11a9047740 1 -- 172.21.15.172:0/1775567412 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560938934d70 con 0x560938a4b1a0 2024-09-10T09:03:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.388+0000 7f119f7fe640 1 --2- 172.21.15.172:0/1775567412 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560938a4b1a0 0x560938a4d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.391+0000 7f119f7fe640 1 --2- 172.21.15.172:0/1775567412 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560938a4b1a0 0x560938a4d5c0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25d48aae3267ca66 server_cookie=6e5186a59b7955be in_seq=0 out_seq=0 2024-09-10T09:03:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:18.395+0000 7f117effd640 10 client.28508 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:03:18.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:18 smithi174 ceph-mon[34068]: pgmap v862: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-10T09:03:19.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:18 smithi191 ceph-mon[35314]: pgmap v862: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-10T09:03:19.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:18 smithi172 ceph-mon[30854]: pgmap v862: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 15 MiB/s wr, 3.73k op/s 2024-09-10T09:03:19.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:19 smithi174 ceph-mon[34068]: pgmap v863: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 3.51k op/s 2024-09-10T09:03:20.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:19 smithi191 ceph-mon[35314]: pgmap v863: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 3.51k op/s 2024-09-10T09:03:20.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:19 smithi172 ceph-mon[30854]: pgmap v863: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 3.51k op/s 2024-09-10T09:03:21.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:21 smithi174 ceph-mon[34068]: pgmap v864: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 4.23k op/s 2024-09-10T09:03:22.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:21 smithi191 ceph-mon[35314]: pgmap v864: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 4.23k op/s 2024-09-10T09:03:22.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:21 smithi172 ceph-mon[30854]: pgmap v864: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 4.23k op/s 2024-09-10T09:03:23.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:23 smithi172 ceph-mon[30854]: pgmap v865: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 3.61k op/s 2024-09-10T09:03:23.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:23 smithi174 ceph-mon[34068]: pgmap v865: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 3.61k op/s 2024-09-10T09:03:24.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:23 smithi191 ceph-mon[35314]: pgmap v865: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 3.61k op/s 2024-09-10T09:03:25.343 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T09:03:25.344 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.85 2024-09-10T09:03:25.345 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:03:25.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:25 smithi172 ceph-mon[30854]: pgmap v866: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 4.34k op/s 2024-09-10T09:03:25.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:25 smithi174 ceph-mon[34068]: pgmap v866: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 4.34k op/s 2024-09-10T09:03:26.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:25 smithi191 ceph-mon[35314]: pgmap v866: 129 pgs: 129 active+clean; 13 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 4.34k op/s 2024-09-10T09:03:26.342 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:03:27.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:27 smithi172 ceph-mon[30854]: pgmap v867: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 3.74k op/s 2024-09-10T09:03:27.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:27 smithi174 ceph-mon[34068]: pgmap v867: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 3.74k op/s 2024-09-10T09:03:28.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:27 smithi191 ceph-mon[35314]: pgmap v867: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 3.74k op/s 2024-09-10T09:03:29.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:29 smithi172 ceph-mon[30854]: pgmap v868: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 13 MiB/s wr, 3.42k op/s 2024-09-10T09:03:29.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:29 smithi174 ceph-mon[34068]: pgmap v868: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 13 MiB/s wr, 3.42k op/s 2024-09-10T09:03:30.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:29 smithi191 ceph-mon[35314]: pgmap v868: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 13 MiB/s wr, 3.42k op/s 2024-09-10T09:03:31.951 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:31 smithi172 ceph-mon[30854]: pgmap v869: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 3.99k op/s 2024-09-10T09:03:31.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:31 smithi174 ceph-mon[34068]: pgmap v869: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 3.99k op/s 2024-09-10T09:03:32.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:31 smithi191 ceph-mon[35314]: pgmap v869: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 3.99k op/s 2024-09-10T09:03:32.516 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:03:33.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:32 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2881948324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:03:33.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:32 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2881948324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:03:33.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:32 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2881948324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:03:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.828+0000 7f117effd640 1 -- 172.21.15.172:0/1775567412 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560938934d70 con 0x560938a4b1a0 2024-09-10T09:03:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.828+0000 7f117effd640 10 client.28508 handle_command_reply: tid=0 2024-09-10T09:03:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.828+0000 7f11a9047740 1 client.28508 shutdown 2024-09-10T09:03:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.830+0000 7f11a9047740 1 -- 172.21.15.172:0/1775567412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389cf370 msgr2=0x5609389f1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.830+0000 7f11a9047740 1 --2- 172.21.15.172:0/1775567412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389cf370 0x5609389f1dc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1190001730 tx=0x7f1190009f90 comp rx=0 tx=0).stop 2024-09-10T09:03:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.830+0000 7f11a9047740 1 -- 172.21.15.172:0/1775567412 shutdown_connections 2024-09-10T09:03:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.830+0000 7f11a9047740 1 --2- 172.21.15.172:0/1775567412 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560938a4b1a0 0x560938a4d5c0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.830+0000 7f11a9047740 1 --2- 172.21.15.172:0/1775567412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609389d9290 0x5609389f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.830+0000 7f11a9047740 1 --2- 172.21.15.172:0/1775567412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609389cf370 0x5609389f1dc0 secure :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1190001730 tx=0x7f1190009f90 comp rx=0 tx=0).stop 2024-09-10T09:03:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.830+0000 7f11a9047740 1 --2- 172.21.15.172:0/1775567412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609389c99a0 0x5609389f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.830+0000 7f11a9047740 1 -- 172.21.15.172:0/1775567412 >> 172.21.15.172:0/1775567412 conn(0x5609389c5010 msgr2=0x5609389c5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.830+0000 7f117effd640 0 client.28508 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:03:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.835+0000 7f11a9047740 1 -- 172.21.15.172:0/1775567412 shutdown_connections 2024-09-10T09:03:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.835+0000 7f11a9047740 1 -- 172.21.15.172:0/1775567412 wait complete. 2024-09-10T09:03:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.835+0000 7f11a9047740 20 client.28508 trim_cache size 0 max 16384 2024-09-10T09:03:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:03:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:03:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:03:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:03:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.837+0000 7f11a61cf640 1 -- 172.21.15.172:0/1580134330 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11880a2e60 msgr2=0x7f11880a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.837+0000 7f11a61cf640 1 --2- 172.21.15.172:0/1580134330 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11880a2e60 0x7f11880a5320 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5609389cad10 tx=0x7f1194005d10 comp rx=0 tx=0).stop 2024-09-10T09:03:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.837+0000 7f11a61cf640 1 -- 172.21.15.172:0/1580134330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11a0109700 msgr2=0x7f11a007bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.837+0000 7f11a61cf640 1 --2- 172.21.15.172:0/1580134330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11a0109700 0x7f11a007bdb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f119800e940 tx=0x7f119800ee00 comp rx=0 tx=0).stop 2024-09-10T09:03:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.837+0000 7f11a61cf640 1 -- 172.21.15.172:0/1580134330 shutdown_connections 2024-09-10T09:03:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.837+0000 7f11a61cf640 1 --2- 172.21.15.172:0/1580134330 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f11880a2e60 0x7f11880a5320 secure :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5609389cad10 tx=0x7f1194005d10 comp rx=0 tx=0).stop 2024-09-10T09:03:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.837+0000 7f11a61cf640 1 --2- 172.21.15.172:0/1580134330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11a0109700 0x7f11a007bdb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.837+0000 7f11a61cf640 1 --2- 172.21.15.172:0/1580134330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11a00736c0 0x7f11a0074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.837+0000 7f11a61cf640 1 --2- 172.21.15.172:0/1580134330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a0072d20 0x7f11a0074800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.837+0000 7f11a61cf640 1 -- 172.21.15.172:0/1580134330 >> 172.21.15.172:0/1580134330 conn(0x7f11a006ece0 msgr2=0x7f11a0071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.840+0000 7f11a61cf640 1 -- 172.21.15.172:0/1580134330 shutdown_connections 2024-09-10T09:03:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:33.840+0000 7f11a61cf640 1 -- 172.21.15.172:0/1580134330 wait complete. 2024-09-10T09:03:33.971 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T09:03:33.971 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":44,"btime":"2024-09-10T09:01:18:531063+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T09:01:17.697372+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T09:03:33.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.410+0000 7f4c99566640 1 -- 172.21.15.172:0/1142254874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c94100e20 msgr2=0x7f4c94101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:33.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.410+0000 7f4c99566640 1 --2- 172.21.15.172:0/1142254874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c94100e20 0x7f4c94101200 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009900 tx=0x7f4c8402a480 comp rx=0 tx=0).stop 2024-09-10T09:03:33.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.410+0000 7f4c99566640 1 -- 172.21.15.172:0/1142254874 shutdown_connections 2024-09-10T09:03:33.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.410+0000 7f4c99566640 1 --2- 172.21.15.172:0/1142254874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c94102ef0 0x7f4c9410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:33.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.410+0000 7f4c99566640 1 --2- 172.21.15.172:0/1142254874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c94102570 0x7f4c941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:33.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.410+0000 7f4c99566640 1 --2- 172.21.15.172:0/1142254874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c94100e20 0x7f4c94101200 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:33.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.410+0000 7f4c99566640 1 -- 172.21.15.172:0/1142254874 >> 172.21.15.172:0/1142254874 conn(0x7f4c940fc7d0 msgr2=0x7f4c940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:33.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.410+0000 7f4c99566640 1 -- 172.21.15.172:0/1142254874 shutdown_connections 2024-09-10T09:03:33.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.410+0000 7f4c99566640 1 -- 172.21.15.172:0/1142254874 wait complete. 2024-09-10T09:03:33.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.411+0000 7f4c99566640 1 Processor -- start 2024-09-10T09:03:33.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.411+0000 7f4c99566640 1 -- start start 2024-09-10T09:03:33.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.411+0000 7f4c99566640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c94100e20 0x7f4c941972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:33.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.411+0000 7f4c99566640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c94102570 0x7f4c941977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:33.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.411+0000 7f4c99566640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c94102ef0 0x7f4c9419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:33.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.411+0000 7f4c99566640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4c94110e30 con 0x7f4c94102570 2024-09-10T09:03:33.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.411+0000 7f4c99566640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c94110cb0 con 0x7f4c94102ef0 2024-09-10T09:03:33.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.411+0000 7f4c99566640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4c94110fb0 con 0x7f4c94100e20 2024-09-10T09:03:33.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.411+0000 7f4c927fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c94102570 0x7f4c941977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:33.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.411+0000 7f4c927fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c94102570 0x7f4c941977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36228/0 (socket says 172.21.15.172:36228) 2024-09-10T09:03:33.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.411+0000 7f4c927fc640 1 -- 172.21.15.172:0/2881948324 learned_addr learned my addr 172.21.15.172:0/2881948324 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:03:33.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.411+0000 7f4c92ffd640 1 --2- 172.21.15.172:0/2881948324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c94100e20 0x7f4c941972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:33.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.411+0000 7f4c937fe640 1 --2- 172.21.15.172:0/2881948324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c94102ef0 0x7f4c9419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:33.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.412+0000 7f4c927fc640 1 -- 172.21.15.172:0/2881948324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c94100e20 msgr2=0x7f4c941972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:33.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.412+0000 7f4c927fc640 1 --2- 172.21.15.172:0/2881948324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c94100e20 0x7f4c941972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:33.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.412+0000 7f4c927fc640 1 -- 172.21.15.172:0/2881948324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c94102ef0 msgr2=0x7f4c9419bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:33.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.412+0000 7f4c927fc640 1 --2- 172.21.15.172:0/2881948324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c94102ef0 0x7f4c9419bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:33.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.412+0000 7f4c927fc640 1 -- 172.21.15.172:0/2881948324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9419c260 con 0x7f4c94102570 2024-09-10T09:03:33.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.412+0000 7f4c937fe640 1 --2- 172.21.15.172:0/2881948324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c94102ef0 0x7f4c9419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:03:33.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.412+0000 7f4c927fc640 1 --2- 172.21.15.172:0/2881948324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c94102570 0x7f4c941977f0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800e9f0 tx=0x7f4c8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:33.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.412+0000 7f4c7bfff640 1 -- 172.21.15.172:0/2881948324 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800cd80 con 0x7f4c94102570 2024-09-10T09:03:33.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.412+0000 7f4c7bfff640 1 -- 172.21.15.172:0/2881948324 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c880108e0 con 0x7f4c94102570 2024-09-10T09:03:33.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.412+0000 7f4c99566640 1 -- 172.21.15.172:0/2881948324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9419c550 con 0x7f4c94102570 2024-09-10T09:03:33.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.412+0000 7f4c7bfff640 1 -- 172.21.15.172:0/2881948324 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88018920 con 0x7f4c94102570 2024-09-10T09:03:33.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.412+0000 7f4c99566640 1 -- 172.21.15.172:0/2881948324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c941a3e30 con 0x7f4c94102570 2024-09-10T09:03:33.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.413+0000 7f4c99566640 1 -- 172.21.15.172:0/2881948324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c941021b0 con 0x7f4c94102570 2024-09-10T09:03:33.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.414+0000 7f4c7bfff640 1 -- 172.21.15.172:0/2881948324 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4c88010a90 con 0x7f4c94102570 2024-09-10T09:03:33.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.415+0000 7f4c7bfff640 1 --2- 172.21.15.172:0/2881948324 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4c7c0a2c00 0x7f4c7c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:33.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.415+0000 7f4c92ffd640 1 --2- 172.21.15.172:0/2881948324 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4c7c0a2c00 0x7f4c7c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:33.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.415+0000 7f4c7bfff640 1 -- 172.21.15.172:0/2881948324 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4c880cfaf0 con 0x7f4c94102570 2024-09-10T09:03:33.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.415+0000 7f4c92ffd640 1 --2- 172.21.15.172:0/2881948324 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4c7c0a2c00 0x7f4c7c0a50c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009230 tx=0x7f4c840058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:33.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.418+0000 7f4c7bfff640 1 -- 172.21.15.172:0/2881948324 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c8809c040 con 0x7f4c94102570 2024-09-10T09:03:33.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.778+0000 7f4c99566640 1 -- 172.21.15.172:0/2881948324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c94101200 con 0x7f4c94102570 2024-09-10T09:03:33.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.779+0000 7f4c7bfff640 1 -- 172.21.15.172:0/2881948324 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+12001 (secure 0 0 0) 0x7f4c94101200 con 0x7f4c94102570 2024-09-10T09:03:33.975 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 44 2024-09-10T09:03:33.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.781+0000 7f4c99566640 1 -- 172.21.15.172:0/2881948324 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4c7c0a2c00 msgr2=0x7f4c7c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:33.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.781+0000 7f4c99566640 1 --2- 172.21.15.172:0/2881948324 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4c7c0a2c00 0x7f4c7c0a50c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009230 tx=0x7f4c840058d0 comp rx=0 tx=0).stop 2024-09-10T09:03:33.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.781+0000 7f4c99566640 1 -- 172.21.15.172:0/2881948324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c94102570 msgr2=0x7f4c941977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:33.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.781+0000 7f4c99566640 1 --2- 172.21.15.172:0/2881948324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c94102570 0x7f4c941977f0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800e9f0 tx=0x7f4c8800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:03:33.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.781+0000 7f4c99566640 1 -- 172.21.15.172:0/2881948324 shutdown_connections 2024-09-10T09:03:33.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.781+0000 7f4c99566640 1 --2- 172.21.15.172:0/2881948324 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f4c7c0a2c00 0x7f4c7c0a50c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:33.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.781+0000 7f4c99566640 1 --2- 172.21.15.172:0/2881948324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c94102ef0 0x7f4c9419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:33.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.781+0000 7f4c99566640 1 --2- 172.21.15.172:0/2881948324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c94102570 0x7f4c941977f0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:33.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.781+0000 7f4c99566640 1 --2- 172.21.15.172:0/2881948324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c94100e20 0x7f4c941972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:33.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.781+0000 7f4c99566640 1 -- 172.21.15.172:0/2881948324 >> 172.21.15.172:0/2881948324 conn(0x7f4c940fc7d0 msgr2=0x7f4c940fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:33.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.781+0000 7f4c99566640 1 -- 172.21.15.172:0/2881948324 shutdown_connections 2024-09-10T09:03:33.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:32.781+0000 7f4c99566640 1 -- 172.21.15.172:0/2881948324 wait complete. 2024-09-10T09:03:35.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:34 smithi174 ceph-mon[34068]: pgmap v870: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 MiB/s wr, 2.97k op/s 2024-09-10T09:03:35.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:34 smithi191 ceph-mon[35314]: pgmap v870: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 MiB/s wr, 2.97k op/s 2024-09-10T09:03:35.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:34 smithi172 ceph-mon[30854]: pgmap v870: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 MiB/s wr, 2.97k op/s 2024-09-10T09:03:37.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:36 smithi174 ceph-mon[34068]: pgmap v871: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 13 MiB/s wr, 3.35k op/s 2024-09-10T09:03:37.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:36 smithi191 ceph-mon[35314]: pgmap v871: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 13 MiB/s wr, 3.35k op/s 2024-09-10T09:03:37.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:36 smithi172 ceph-mon[30854]: pgmap v871: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 13 MiB/s wr, 3.35k op/s 2024-09-10T09:03:39.103 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:38 smithi191 ceph-mon[35314]: pgmap v872: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 MiB/s wr, 2.77k op/s 2024-09-10T09:03:39.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:38 smithi174 ceph-mon[34068]: pgmap v872: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 MiB/s wr, 2.77k op/s 2024-09-10T09:03:39.262 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:38 smithi172 ceph-mon[30854]: pgmap v872: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 MiB/s wr, 2.77k op/s 2024-09-10T09:03:41.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:40 smithi172 ceph-mon[30854]: pgmap v873: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.2 MiB/s wr, 2.33k op/s 2024-09-10T09:03:41.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:40 smithi174 ceph-mon[34068]: pgmap v873: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.2 MiB/s wr, 2.33k op/s 2024-09-10T09:03:41.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:40 smithi191 ceph-mon[35314]: pgmap v873: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.2 MiB/s wr, 2.33k op/s 2024-09-10T09:03:42.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:41 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:03:42.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:41 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:03:42.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:41 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:03:43.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:42 smithi174 ceph-mon[34068]: pgmap v874: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 2.80k op/s 2024-09-10T09:03:43.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:42 smithi191 ceph-mon[35314]: pgmap v874: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 2.80k op/s 2024-09-10T09:03:43.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:42 smithi172 ceph-mon[30854]: pgmap v874: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 2.80k op/s 2024-09-10T09:03:44.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:43 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:44.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:43 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:44.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:43 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:44.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:43 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:44.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:43 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi191 to 176.0M 2024-09-10T09:03:44.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:43 smithi174 ceph-mon[34068]: Unable to set osd_memory_target on smithi191 to 184585318: error parsing value: Value '184585318' is below minimum 939524096 2024-09-10T09:03:44.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:43 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:44.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:43 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:44.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:43 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:44.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:43 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:44.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:43 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi191 to 176.0M 2024-09-10T09:03:44.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:43 smithi191 ceph-mon[35314]: Unable to set osd_memory_target on smithi191 to 184585318: error parsing value: Value '184585318' is below minimum 939524096 2024-09-10T09:03:44.335 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:43 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:44.335 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:44.335 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:44.335 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:44 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:44.335 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:44 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi191 to 176.0M 2024-09-10T09:03:44.335 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:44 smithi172 ceph-mon[30854]: Unable to set osd_memory_target on smithi191 to 184585318: error parsing value: Value '184585318' is below minimum 939524096 2024-09-10T09:03:45.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:44 smithi174 ceph-mon[34068]: pgmap v875: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.9 MiB/s wr, 2.26k op/s 2024-09-10T09:03:45.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:44 smithi191 ceph-mon[35314]: pgmap v875: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.9 MiB/s wr, 2.26k op/s 2024-09-10T09:03:45.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:44 smithi172 ceph-mon[30854]: pgmap v875: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.9 MiB/s wr, 2.26k op/s 2024-09-10T09:03:46.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:45 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi172 to 176.0M 2024-09-10T09:03:46.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:45 smithi174 ceph-mon[34068]: Unable to set osd_memory_target on smithi172 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-10T09:03:46.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:03:46.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:03:46.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:45 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:03:46.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:45 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi172 to 176.0M 2024-09-10T09:03:46.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:45 smithi191 ceph-mon[35314]: Unable to set osd_memory_target on smithi172 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-10T09:03:46.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:03:46.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:03:46.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:45 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:03:46.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:45 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi172 to 176.0M 2024-09-10T09:03:46.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:45 smithi172 ceph-mon[30854]: Unable to set osd_memory_target on smithi172 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-10T09:03:46.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:03:46.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:03:46.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:03:46.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:45 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:03:46.633 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0' 2024-09-10T09:03:46.633 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0 2024-09-10T09:03:46.765 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:03:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.021+0000 7f12a5aba640 1 -- 172.21.15.172:0/272808145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12a0073980 msgr2=0x7f12a007a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.021+0000 7f12a5aba640 1 --2- 172.21.15.172:0/272808145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12a0073980 0x7f12a007a450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f129800afd0 tx=0x7f1298032b20 comp rx=0 tx=0).stop 2024-09-10T09:03:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.024+0000 7f12a5aba640 1 -- 172.21.15.172:0/272808145 shutdown_connections 2024-09-10T09:03:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.024+0000 7f12a5aba640 1 --2- 172.21.15.172:0/272808145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12a0073980 0x7f12a007a450 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.024+0000 7f12a5aba640 1 --2- 172.21.15.172:0/272808145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12a0072fc0 0x7f12a0073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.024+0000 7f12a5aba640 1 --2- 172.21.15.172:0/272808145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12a01096e0 0x7f12a0109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.024+0000 7f12a5aba640 1 -- 172.21.15.172:0/272808145 >> 172.21.15.172:0/272808145 conn(0x7f12a006ece0 msgr2=0x7f12a0071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.030+0000 7f12a5aba640 1 -- 172.21.15.172:0/272808145 shutdown_connections 2024-09-10T09:03:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.031+0000 7f12a5aba640 1 -- 172.21.15.172:0/272808145 wait complete. 2024-09-10T09:03:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.033+0000 7f12a5aba640 1 Processor -- start 2024-09-10T09:03:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.033+0000 7f12a5aba640 1 -- start start 2024-09-10T09:03:47.036 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.034+0000 7f12a5aba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12a0072fc0 0x7f12a0084c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.034+0000 7f12a5aba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12a0073980 0x7f12a00832e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.034+0000 7f12a5aba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12a01096e0 0x7f12a0083820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.034+0000 7f12a5aba640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f12a007dd10 con 0x7f12a0072fc0 2024-09-10T09:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.034+0000 7f12a5aba640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12a007db90 con 0x7f12a01096e0 2024-09-10T09:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.034+0000 7f12a5aba640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f12a007de90 con 0x7f12a0073980 2024-09-10T09:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.034+0000 7f129effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12a0073980 0x7f12a00832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.034+0000 7f129effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12a0073980 0x7f12a00832e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:35880/0 (socket says 172.21.15.172:35880) 2024-09-10T09:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.034+0000 7f129effd640 1 -- 172.21.15.172:0/2625129650 learned_addr learned my addr 172.21.15.172:0/2625129650 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:03:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.034+0000 7f129ffff640 1 --2- 172.21.15.172:0/2625129650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12a01096e0 0x7f12a0083820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.036+0000 7f129effd640 1 -- 172.21.15.172:0/2625129650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12a01096e0 msgr2=0x7f12a0083820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.036+0000 7f129effd640 1 --2- 172.21.15.172:0/2625129650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12a01096e0 0x7f12a0083820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.036+0000 7f129effd640 1 -- 172.21.15.172:0/2625129650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12a0072fc0 msgr2=0x7f12a0084c60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.036+0000 7f129effd640 1 --2- 172.21.15.172:0/2625129650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12a0072fc0 0x7f12a0084c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.036+0000 7f129effd640 1 -- 172.21.15.172:0/2625129650 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a0083fc0 con 0x7f12a0073980 2024-09-10T09:03:47.040 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.037+0000 7f129effd640 1 --2- 172.21.15.172:0/2625129650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12a0073980 0x7f12a00832e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f129000b450 tx=0x7f129000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.041+0000 7f129cff9640 1 -- 172.21.15.172:0/2625129650 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290015020 con 0x7f12a0073980 2024-09-10T09:03:47.058 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.041+0000 7f129cff9640 1 -- 172.21.15.172:0/2625129650 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1290011d40 con 0x7f12a0073980 2024-09-10T09:03:47.060 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.041+0000 7f129cff9640 1 -- 172.21.15.172:0/2625129650 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129000f340 con 0x7f12a0073980 2024-09-10T09:03:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.042+0000 7f12a5aba640 1 -- 172.21.15.172:0/2625129650 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a01b7260 con 0x7f12a0073980 2024-09-10T09:03:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.042+0000 7f12a5aba640 1 -- 172.21.15.172:0/2625129650 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a01b77e0 con 0x7f12a0073980 2024-09-10T09:03:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.047+0000 7f12a8857740 1 Processor -- start 2024-09-10T09:03:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.058+0000 7f12a8857740 1 -- start start 2024-09-10T09:03:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.060+0000 7f129cff9640 1 -- 172.21.15.172:0/2625129650 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1290018020 con 0x7f12a0073980 2024-09-10T09:03:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.060+0000 7f12a8857740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 0x55d7e314bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.060+0000 7f12a8857740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e314c2e0 0x55d7e3150da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.060+0000 7f12a8857740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e3151370 0x55d7e3153830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.060+0000 7f12a8857740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d7e30af2f0 con 0x55d7e3151370 2024-09-10T09:03:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.060+0000 7f12a8857740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d7e3057270 con 0x55d7e314b9a0 2024-09-10T09:03:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.060+0000 7f12a8857740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d7e30ab2b0 con 0x55d7e314c2e0 2024-09-10T09:03:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.060+0000 7f129ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 0x55d7e314bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.061+0000 7f129ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 0x55d7e314bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:54682/0 (socket says 172.21.15.172:54682) 2024-09-10T09:03:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.061+0000 7f129ffff640 1 -- 172.21.15.172:0/734189482 learned_addr learned my addr 172.21.15.172:0/734189482 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:03:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.061+0000 7f129ffff640 1 -- 172.21.15.172:0/734189482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e314c2e0 msgr2=0x55d7e3150da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:03:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.061+0000 7f129ffff640 1 --2- 172.21.15.172:0/734189482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e314c2e0 0x55d7e3150da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.061+0000 7f129ffff640 1 -- 172.21.15.172:0/734189482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e3151370 msgr2=0x55d7e3153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.061+0000 7f129ffff640 1 --2- 172.21.15.172:0/734189482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e3151370 0x55d7e3153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.061+0000 7f129ffff640 1 -- 172.21.15.172:0/734189482 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7e3056340 con 0x55d7e314b9a0 2024-09-10T09:03:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.062+0000 7f129ffff640 1 --2- 172.21.15.172:0/734189482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 0x55d7e314bda0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f129800afd0 tx=0x7f1298005f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dafb816e7ff2668d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.063+0000 7f129cff9640 1 --2- 172.21.15.172:0/2625129650 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12880a2e70 0x7f12880a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.064+0000 7f129cff9640 1 -- 172.21.15.172:0/2625129650 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f12900cec00 con 0x7f12a0073980 2024-09-10T09:03:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.066+0000 7f1285ffb640 1 -- 172.21.15.172:0/734189482 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1298043020 con 0x55d7e314b9a0 2024-09-10T09:03:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.066+0000 7f1285ffb640 1 -- 172.21.15.172:0/734189482 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1298035810 con 0x55d7e314b9a0 2024-09-10T09:03:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.066+0000 7f1285ffb640 1 -- 172.21.15.172:0/734189482 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129803a040 con 0x55d7e314b9a0 2024-09-10T09:03:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.068+0000 7f12a8857740 1 -- 172.21.15.172:0/734189482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 msgr2=0x55d7e314bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.068+0000 7f12a8857740 1 --2- 172.21.15.172:0/734189482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 0x55d7e314bda0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f129800afd0 tx=0x7f1298005f50 comp rx=0 tx=0).stop 2024-09-10T09:03:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.069+0000 7f129f7fe640 1 --2- 172.21.15.172:0/2625129650 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12880a2e70 0x7f12880a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:47.071 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.070+0000 7f12a8857740 1 -- 172.21.15.172:0/734189482 shutdown_connections 2024-09-10T09:03:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.070+0000 7f12a8857740 1 --2- 172.21.15.172:0/734189482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e3151370 0x55d7e3153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.070+0000 7f12a8857740 1 --2- 172.21.15.172:0/734189482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e314c2e0 0x55d7e3150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.070+0000 7f12a8857740 1 --2- 172.21.15.172:0/734189482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 0x55d7e314bda0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.070+0000 7f12a8857740 1 -- 172.21.15.172:0/734189482 >> 172.21.15.172:0/734189482 conn(0x55d7e3147010 msgr2=0x55d7e3147410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.070+0000 7f129f7fe640 1 --2- 172.21.15.172:0/2625129650 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12880a2e70 0x7f12880a5330 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55d7e314cd10 tx=0x7f129400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.070+0000 7f12a8857740 1 -- 172.21.15.172:0/734189482 shutdown_connections 2024-09-10T09:03:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.072+0000 7f12a8857740 1 -- 172.21.15.172:0/734189482 wait complete. 2024-09-10T09:03:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.072+0000 7f12a8857740 1 -- start start 2024-09-10T09:03:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.072+0000 7f12a8857740 1 Processor -- start 2024-09-10T09:03:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.072+0000 7f12a8857740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e314b9a0 0x55d7e3155c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.073+0000 7f12a8857740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e314c2e0 0x55d7e31561c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.073+0000 7f12a8857740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e3151370 0x55d7e315d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.073+0000 7f12a8857740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d7e31542e0 con 0x55d7e314b9a0 2024-09-10T09:03:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.073+0000 7f12a8857740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d7e3038290 con 0x55d7e3151370 2024-09-10T09:03:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.073+0000 7f12a8857740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d7e3154460 con 0x55d7e314c2e0 2024-09-10T09:03:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.073+0000 7f12a8857740 10 client.0 _ll_register_callbacks cb 0x55d7e3069140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:03:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.073+0000 7f129ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e3151370 0x55d7e315d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.073+0000 7f129f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e314b9a0 0x55d7e3155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.073+0000 7f129ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e3151370 0x55d7e315d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:54694/0 (socket says 172.21.15.172:54694) 2024-09-10T09:03:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.073+0000 7f129f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e314b9a0 0x55d7e3155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44180/0 (socket says 172.21.15.172:44180) 2024-09-10T09:03:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.073+0000 7f129ffff640 1 -- 172.21.15.172:0/1261869776 learned_addr learned my addr 172.21.15.172:0/1261869776 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:03:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.074+0000 7f129ffff640 1 -- 172.21.15.172:0/1261869776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e314c2e0 msgr2=0x55d7e31561c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:03:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.074+0000 7f129ffff640 1 --2- 172.21.15.172:0/1261869776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e314c2e0 0x55d7e31561c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.074+0000 7f129ffff640 1 -- 172.21.15.172:0/1261869776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e314b9a0 msgr2=0x55d7e3155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.074+0000 7f129ffff640 1 --2- 172.21.15.172:0/1261869776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e314b9a0 0x55d7e3155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.074+0000 7f129ffff640 1 -- 172.21.15.172:0/1261869776 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7e2dc2210 con 0x55d7e3151370 2024-09-10T09:03:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.074+0000 7f129ffff640 1 --2- 172.21.15.172:0/1261869776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e3151370 0x55d7e315d200 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1298011010 tx=0x7f12980348b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db613b9b2b9f3ef8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.074+0000 7f127effd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:03:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.079+0000 7f127effd640 1 -- 172.21.15.172:0/1261869776 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129804e020 con 0x55d7e3151370 2024-09-10T09:03:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.079+0000 7f12a8857740 1 -- 172.21.15.172:0/1261869776 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1298035810 con 0x55d7e3151370 2024-09-10T09:03:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.079+0000 7f127effd640 1 -- 172.21.15.172:0/1261869776 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1298003b80 con 0x55d7e3151370 2024-09-10T09:03:47.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.079+0000 7f127effd640 1 -- 172.21.15.172:0/1261869776 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1298059050 con 0x55d7e3151370 2024-09-10T09:03:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.085+0000 7f12a8857740 10 client.28526 fetch_fsmap learned FSMap version 44 2024-09-10T09:03:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.085+0000 7f12a8857740 1 -- 172.21.15.172:0/1261869776 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d7e2e8e620 con 0x55d7e3151370 2024-09-10T09:03:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.086+0000 7f127effd640 1 -- 172.21.15.172:0/1261869776 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1298035810 con 0x55d7e3151370 2024-09-10T09:03:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.086+0000 7f127effd640 1 -- 172.21.15.172:0/1261869776 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f12980554c0 con 0x55d7e3151370 2024-09-10T09:03:47.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.093+0000 7f12a8857740 10 client.28526 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:03:47.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.093+0000 7f12a8857740 10 client.28526 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:03:47.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.093+0000 7f12a8857740 20 client.28526 populate_metadata read hostname 'smithi172' 2024-09-10T09:03:47.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.093+0000 7f12a8857740 1 --2- 172.21.15.172:0/1261869776 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d7e3161500 0x55d7e31638f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.093+0000 7f12a8857740 4 client.28526 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:03:47.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.093+0000 7f12a8857740 1 -- 172.21.15.172:0/1261869776 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7e30bca20 con 0x55d7e3161500 2024-09-10T09:03:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.095+0000 7f129f7fe640 1 --2- 172.21.15.172:0/1261869776 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d7e3161500 0x55d7e31638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:47.098 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.098+0000 7f129f7fe640 1 --2- 172.21.15.172:0/1261869776 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d7e3161500 0x55d7e31638f0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb48ccff03f51fb8 server_cookie=df0776e6ca040909 in_seq=0 out_seq=0 2024-09-10T09:03:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.098+0000 7f127effd640 10 client.28526 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:03:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.098+0000 7f127effd640 1 -- 172.21.15.172:0/1261869776 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7e30bca20 con 0x55d7e3161500 2024-09-10T09:03:47.100 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.099+0000 7f127effd640 10 client.28526 handle_command_reply: tid=0 2024-09-10T09:03:47.103 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.102+0000 7f12a8857740 1 client.28526 shutdown 2024-09-10T09:03:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.110+0000 7f12a8857740 1 -- 172.21.15.172:0/1261869776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e3151370 msgr2=0x55d7e315d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.110+0000 7f12a8857740 1 --2- 172.21.15.172:0/1261869776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e3151370 0x55d7e315d200 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1298011010 tx=0x7f12980348b0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.111+0000 7f12a8857740 1 -- 172.21.15.172:0/1261869776 shutdown_connections 2024-09-10T09:03:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.111+0000 7f12a8857740 1 --2- 172.21.15.172:0/1261869776 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d7e3161500 0x55d7e31638f0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.111+0000 7f12a8857740 1 --2- 172.21.15.172:0/1261869776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e3151370 0x55d7e315d200 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.111+0000 7f12a8857740 1 --2- 172.21.15.172:0/1261869776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e314c2e0 0x55d7e31561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.111+0000 7f12a8857740 1 --2- 172.21.15.172:0/1261869776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e314b9a0 0x55d7e3155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.111+0000 7f12a8857740 1 -- 172.21.15.172:0/1261869776 >> 172.21.15.172:0/1261869776 conn(0x55d7e3147010 msgr2=0x55d7e3148820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.115+0000 7f127effd640 0 client.28526 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:03:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.117+0000 7f12a8857740 1 -- 172.21.15.172:0/1261869776 shutdown_connections 2024-09-10T09:03:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.119+0000 7f12a8857740 1 -- 172.21.15.172:0/1261869776 wait complete. 2024-09-10T09:03:47.133 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.121+0000 7f12a8857740 20 client.28526 trim_cache size 0 max 16384 2024-09-10T09:03:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.152+0000 7f12a8857740 1 Processor -- start 2024-09-10T09:03:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.152+0000 7f12a8857740 1 -- start start 2024-09-10T09:03:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.153+0000 7f12a8857740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 0x55d7e3147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.153+0000 7f12a8857740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e314c2e0 0x55d7e315d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.153+0000 7f129f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 0x55d7e3147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.153+0000 7f129f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 0x55d7e3147ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:54704/0 (socket says 172.21.15.172:54704) 2024-09-10T09:03:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.154+0000 7f129effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e314c2e0 0x55d7e315d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.154+0000 7f129effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e314c2e0 0x55d7e315d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:35882/0 (socket says 172.21.15.172:35882) 2024-09-10T09:03:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.154+0000 7f129f7fe640 1 -- 172.21.15.172:0/2472706664 learned_addr learned my addr 172.21.15.172:0/2472706664 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:03:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.154+0000 7f12a8857740 1 --2- 172.21.15.172:0/2472706664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e3151370 0x55d7e3148400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.155 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.154+0000 7f12a8857740 1 -- 172.21.15.172:0/2472706664 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f12a01c1c40 con 0x55d7e3151370 2024-09-10T09:03:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.154+0000 7f129ffff640 1 --2- 172.21.15.172:0/2472706664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e3151370 0x55d7e3148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.154+0000 7f12a8857740 1 -- 172.21.15.172:0/2472706664 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12a01e5790 con 0x55d7e314b9a0 2024-09-10T09:03:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.154+0000 7f12a8857740 1 -- 172.21.15.172:0/2472706664 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f12a01e4320 con 0x55d7e314c2e0 2024-09-10T09:03:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.155+0000 7f129effd640 1 -- 172.21.15.172:0/2472706664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 msgr2=0x55d7e3147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.155+0000 7f129effd640 1 --2- 172.21.15.172:0/2472706664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 0x55d7e3147ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.155+0000 7f129effd640 1 -- 172.21.15.172:0/2472706664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e3151370 msgr2=0x55d7e3148400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.155+0000 7f129effd640 1 --2- 172.21.15.172:0/2472706664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e3151370 0x55d7e3148400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.156 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.155+0000 7f129effd640 1 -- 172.21.15.172:0/2472706664 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7e314d0b0 con 0x55d7e314c2e0 2024-09-10T09:03:47.157 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.156+0000 7f129effd640 1 --2- 172.21.15.172:0/2472706664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e314c2e0 0x55d7e315d1c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55d7e3157160 tx=0x7f129009aa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ce7d3d13c69376d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.162+0000 7f12857fa640 1 -- 172.21.15.172:0/2472706664 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129009b0a0 con 0x55d7e314c2e0 2024-09-10T09:03:47.163 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.163+0000 7f12857fa640 1 -- 172.21.15.172:0/2472706664 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f129009f450 con 0x55d7e314c2e0 2024-09-10T09:03:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.164+0000 7f12a8857740 1 -- 172.21.15.172:0/2472706664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e314c2e0 msgr2=0x55d7e315d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:47.164 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.164+0000 7f12a8857740 1 --2- 172.21.15.172:0/2472706664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e314c2e0 0x55d7e315d1c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55d7e3157160 tx=0x7f129009aa00 comp rx=0 tx=0).stop 2024-09-10T09:03:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.164+0000 7f12a8857740 1 -- 172.21.15.172:0/2472706664 shutdown_connections 2024-09-10T09:03:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.164+0000 7f12a8857740 1 --2- 172.21.15.172:0/2472706664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e3151370 0x55d7e3148400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.164+0000 7f12a8857740 1 --2- 172.21.15.172:0/2472706664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e314c2e0 0x55d7e315d1c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.164+0000 7f12a8857740 1 --2- 172.21.15.172:0/2472706664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 0x55d7e3147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.164+0000 7f12a8857740 1 -- 172.21.15.172:0/2472706664 >> 172.21.15.172:0/2472706664 conn(0x55d7e3147010 msgr2=0x55d7e3155c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.165+0000 7f12a8857740 1 -- 172.21.15.172:0/2472706664 shutdown_connections 2024-09-10T09:03:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.165+0000 7f12a8857740 1 -- 172.21.15.172:0/2472706664 wait complete. 2024-09-10T09:03:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.165+0000 7f12a8857740 1 -- start start 2024-09-10T09:03:47.166 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.165+0000 7f12a8857740 1 Processor -- start 2024-09-10T09:03:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.166+0000 7f12a8857740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 0x55d7e315a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.166+0000 7f12a8857740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e314c2e0 0x55d7e315a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.166+0000 7f12a8857740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e3151370 0x55d7e315b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.166+0000 7f12a8857740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f12a01e50c0 con 0x55d7e314c2e0 2024-09-10T09:03:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.166+0000 7f12a8857740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12a01e49f0 con 0x55d7e314b9a0 2024-09-10T09:03:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.166+0000 7f12a8857740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f12a01bfc20 con 0x55d7e3151370 2024-09-10T09:03:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.166+0000 7f12a8857740 10 client.0 _ll_register_callbacks cb 0x7f1298008990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:03:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.166+0000 7f129f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 0x55d7e315a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.166+0000 7f129f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 0x55d7e315a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:54710/0 (socket says 172.21.15.172:54710) 2024-09-10T09:03:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.166+0000 7f129f7fe640 1 -- 172.21.15.172:0/4224210425 learned_addr learned my addr 172.21.15.172:0/4224210425 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:03:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.166+0000 7f129ffff640 1 --2- 172.21.15.172:0/4224210425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e3151370 0x55d7e315b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.166+0000 7f129effd640 1 --2- 172.21.15.172:0/4224210425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e314c2e0 0x55d7e315a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.167+0000 7f129ffff640 1 -- 172.21.15.172:0/4224210425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 msgr2=0x55d7e315a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.167+0000 7f129ffff640 1 --2- 172.21.15.172:0/4224210425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 0x55d7e315a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.167+0000 7f129ffff640 1 -- 172.21.15.172:0/4224210425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e314c2e0 msgr2=0x55d7e315a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.167+0000 7f129ffff640 1 --2- 172.21.15.172:0/4224210425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e314c2e0 0x55d7e315a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.167+0000 7f129ffff640 1 -- 172.21.15.172:0/4224210425 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7e315bd70 con 0x55d7e3151370 2024-09-10T09:03:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.167+0000 7f129effd640 1 --2- 172.21.15.172:0/4224210425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e314c2e0 0x55d7e315a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:03:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.167+0000 7f129f7fe640 1 --2- 172.21.15.172:0/4224210425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 0x55d7e315a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:03:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.167+0000 7f129ffff640 1 --2- 172.21.15.172:0/4224210425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e3151370 0x55d7e315b760 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1298055a00 tx=0x7f1298055cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddc4b22342071cbb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.169+0000 7f127effd640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:03:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.169+0000 7f127effd640 1 -- 172.21.15.172:0/4224210425 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1298061020 con 0x55d7e3151370 2024-09-10T09:03:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.169+0000 7f127effd640 1 -- 172.21.15.172:0/4224210425 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1298034b20 con 0x55d7e3151370 2024-09-10T09:03:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.171+0000 7f12a8857740 1 -- 172.21.15.172:0/4224210425 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f129009f450 con 0x55d7e3151370 2024-09-10T09:03:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.173+0000 7f127effd640 1 -- 172.21.15.172:0/4224210425 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f129804e040 con 0x55d7e3151370 2024-09-10T09:03:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.176+0000 7f12a8857740 10 client.28423 fetch_fsmap learned FSMap version 44 2024-09-10T09:03:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.176+0000 7f12a8857740 1 -- 172.21.15.172:0/4224210425 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d7e315d080 con 0x55d7e3151370 2024-09-10T09:03:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.177+0000 7f127effd640 1 -- 172.21.15.172:0/4224210425 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f129009f450 con 0x55d7e3151370 2024-09-10T09:03:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.178+0000 7f127effd640 1 -- 172.21.15.172:0/4224210425 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f129804a460 con 0x55d7e3151370 2024-09-10T09:03:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.178+0000 7f12a8857740 10 client.28423 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:03:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.178+0000 7f12a8857740 10 client.28423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:03:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.178+0000 7f12a8857740 20 client.28423 populate_metadata read hostname 'smithi172' 2024-09-10T09:03:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.178+0000 7f12a8857740 1 --2- 172.21.15.172:0/4224210425 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d7e31cc320 0x55d7e31ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.178+0000 7f12a8857740 4 client.28423 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:03:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.178+0000 7f12a8857740 1 -- 172.21.15.172:0/4224210425 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7e30b6d70 con 0x55d7e31cc320 2024-09-10T09:03:47.179 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.178+0000 7f129f7fe640 1 --2- 172.21.15.172:0/4224210425 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d7e31cc320 0x55d7e31ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.184+0000 7f129f7fe640 1 --2- 172.21.15.172:0/4224210425 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d7e31cc320 0x55d7e31ce740 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=559a804f4e5323e3 server_cookie=fc74010ed616d06d in_seq=0 out_seq=0 2024-09-10T09:03:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:03:47.184+0000 7f127effd640 10 client.28423 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:03:47.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:46 smithi174 ceph-mon[34068]: pgmap v876: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-09-10T09:03:47.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:46 smithi174 ceph-mon[34068]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-10T09:03:47.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:46 smithi174 ceph-mon[34068]: Unable to set osd_memory_target on smithi174 to 184564531: error parsing value: Value '184564531' is below minimum 939524096 2024-09-10T09:03:47.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:46 smithi191 ceph-mon[35314]: pgmap v876: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-09-10T09:03:47.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:46 smithi191 ceph-mon[35314]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-10T09:03:47.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:46 smithi191 ceph-mon[35314]: Unable to set osd_memory_target on smithi174 to 184564531: error parsing value: Value '184564531' is below minimum 939524096 2024-09-10T09:03:47.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:46 smithi172 ceph-mon[30854]: pgmap v876: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.6 MiB/s wr, 2.43k op/s 2024-09-10T09:03:47.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:46 smithi172 ceph-mon[30854]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-10T09:03:47.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:46 smithi172 ceph-mon[30854]: Unable to set osd_memory_target on smithi174 to 184564531: error parsing value: Value '184564531' is below minimum 939524096 2024-09-10T09:03:49.195 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:48 smithi172 ceph-mon[30854]: pgmap v877: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.6 MiB/s wr, 2.45k op/s 2024-09-10T09:03:49.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:48 smithi174 ceph-mon[34068]: pgmap v877: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.6 MiB/s wr, 2.45k op/s 2024-09-10T09:03:49.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:48 smithi191 ceph-mon[35314]: pgmap v877: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.6 MiB/s wr, 2.45k op/s 2024-09-10T09:03:51.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:50 smithi174 ceph-mon[34068]: pgmap v878: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.5 MiB/s wr, 2.17k op/s 2024-09-10T09:03:51.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:50 smithi191 ceph-mon[35314]: pgmap v878: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.5 MiB/s wr, 2.17k op/s 2024-09-10T09:03:51.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:50 smithi172 ceph-mon[30854]: pgmap v878: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.5 MiB/s wr, 2.17k op/s 2024-09-10T09:03:53.223 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:52 smithi172 ceph-mon[30854]: pgmap v879: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 10 MiB/s wr, 2.66k op/s 2024-09-10T09:03:53.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:52 smithi174 ceph-mon[34068]: pgmap v879: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 10 MiB/s wr, 2.66k op/s 2024-09-10T09:03:53.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:52 smithi191 ceph-mon[35314]: pgmap v879: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 10 MiB/s wr, 2.66k op/s 2024-09-10T09:03:54.962 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:03:54.962 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:03:54.963 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T09:03:54.963 INFO:teuthology.orchestra.run.smithi172.stdout: "097ab02a-d7be-4e1d-96ec-78f5d8c3836d": { 2024-09-10T09:03:54.963 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T09:03:54.963 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "097ab02a-d7be-4e1d-96ec-78f5d8c3836d", 2024-09-10T09:03:54.963 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T09:03:54.963 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:03:54.963 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:03:54.963 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:03:54.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.221+0000 7f6784364640 1 -- 172.21.15.172:0/3559567933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f677c100e20 msgr2=0x7f677c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:54.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.221+0000 7f6784364640 1 --2- 172.21.15.172:0/3559567933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f677c100e20 0x7f677c101200 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f6778009900 tx=0x7f677802a480 comp rx=0 tx=0).stop 2024-09-10T09:03:54.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.221+0000 7f6784364640 1 -- 172.21.15.172:0/3559567933 shutdown_connections 2024-09-10T09:03:54.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.221+0000 7f6784364640 1 --2- 172.21.15.172:0/3559567933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f677c102ef0 0x7f677c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.221+0000 7f6784364640 1 --2- 172.21.15.172:0/3559567933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f677c102570 0x7f677c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.221+0000 7f6784364640 1 --2- 172.21.15.172:0/3559567933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f677c100e20 0x7f677c101200 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.221+0000 7f6784364640 1 -- 172.21.15.172:0/3559567933 >> 172.21.15.172:0/3559567933 conn(0x7f677c0fc7d0 msgr2=0x7f677c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:54.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.222+0000 7f6784364640 1 -- 172.21.15.172:0/3559567933 shutdown_connections 2024-09-10T09:03:54.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.222+0000 7f6784364640 1 -- 172.21.15.172:0/3559567933 wait complete. 2024-09-10T09:03:54.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.222+0000 7f6784364640 1 Processor -- start 2024-09-10T09:03:54.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.222+0000 7f6784364640 1 -- start start 2024-09-10T09:03:54.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.222+0000 7f6784364640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f677c100e20 0x7f677c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.222+0000 7f6784364640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f677c102570 0x7f677c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.222+0000 7f6784364640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f677c102ef0 0x7f677c1a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.222+0000 7f6784364640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f677c110e30 con 0x7f677c100e20 2024-09-10T09:03:54.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.222+0000 7f6784364640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f677c110cb0 con 0x7f677c102ef0 2024-09-10T09:03:54.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.222+0000 7f6784364640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f677c110fb0 con 0x7f677c102570 2024-09-10T09:03:54.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.222+0000 7f6782100640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f677c100e20 0x7f677c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:54.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.222+0000 7f6782100640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f677c100e20 0x7f677c1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44212/0 (socket says 172.21.15.172:44212) 2024-09-10T09:03:54.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.222+0000 7f6782100640 1 -- 172.21.15.172:0/3340397104 learned_addr learned my addr 172.21.15.172:0/3340397104 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:03:54.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.223+0000 7f6782901640 1 --2- 172.21.15.172:0/3340397104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f677c102ef0 0x7f677c1a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:54.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.223+0000 7f67818ff640 1 --2- 172.21.15.172:0/3340397104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f677c102570 0x7f677c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:54.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.223+0000 7f6782100640 1 -- 172.21.15.172:0/3340397104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f677c102570 msgr2=0x7f677c199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:54.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.223+0000 7f6782100640 1 --2- 172.21.15.172:0/3340397104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f677c102570 0x7f677c199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.223+0000 7f6782100640 1 -- 172.21.15.172:0/3340397104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f677c102ef0 msgr2=0x7f677c1a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:54.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.223+0000 7f6782100640 1 --2- 172.21.15.172:0/3340397104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f677c102ef0 0x7f677c1a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.223+0000 7f6782100640 1 -- 172.21.15.172:0/3340397104 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c1a47a0 con 0x7f677c100e20 2024-09-10T09:03:54.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.223+0000 7f6782100640 1 --2- 172.21.15.172:0/3340397104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f677c100e20 0x7f677c1991e0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f6778006010 tx=0x7f6778004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:54.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.223+0000 7f67737fe640 1 -- 172.21.15.172:0/3340397104 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677803f030 con 0x7f677c100e20 2024-09-10T09:03:54.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.223+0000 7f67737fe640 1 -- 172.21.15.172:0/3340397104 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6778003aa0 con 0x7f677c100e20 2024-09-10T09:03:54.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.223+0000 7f6784364640 1 -- 172.21.15.172:0/3340397104 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677c1a4970 con 0x7f677c100e20 2024-09-10T09:03:54.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.223+0000 7f67737fe640 1 -- 172.21.15.172:0/3340397104 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778035070 con 0x7f677c100e20 2024-09-10T09:03:54.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.223+0000 7f6784364640 1 -- 172.21.15.172:0/3340397104 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677c1971a0 con 0x7f677c100e20 2024-09-10T09:03:54.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.225+0000 7f67737fe640 1 -- 172.21.15.172:0/3340397104 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6778042030 con 0x7f677c100e20 2024-09-10T09:03:54.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.226+0000 7f67737fe640 1 --2- 172.21.15.172:0/3340397104 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f67680a2c00 0x7f67680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.226+0000 7f67737fe640 1 -- 172.21.15.172:0/3340397104 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f67780ef830 con 0x7f677c100e20 2024-09-10T09:03:54.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.226+0000 7f67818ff640 1 --2- 172.21.15.172:0/3340397104 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f67680a2c00 0x7f67680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:54.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.226+0000 7f678673d740 1 Processor -- start 2024-09-10T09:03:54.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.226+0000 7f678673d740 1 -- start start 2024-09-10T09:03:54.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.226+0000 7f678673d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560855474a0 0x556085547880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.226+0000 7f678673d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085547dc0 0x556085552140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.226+0000 7f678673d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556085552680 0x556085554a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.226+0000 7f678673d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5560854ad2e0 con 0x556085547dc0 2024-09-10T09:03:54.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.226+0000 7f678673d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556085454ce0 con 0x5560855474a0 2024-09-10T09:03:54.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.226+0000 7f678673d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5560854a92a0 con 0x556085552680 2024-09-10T09:03:54.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.226+0000 7f67818ff640 1 --2- 172.21.15.172:0/3340397104 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f67680a2c00 0x7f67680a50c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f677c06ab00 tx=0x7f676c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:54.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.226+0000 7f67818ff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085547dc0 0x556085552140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:54.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.227+0000 7f6782100640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560855474a0 0x556085547880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:54.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.227+0000 7f67818ff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085547dc0 0x556085552140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44228/0 (socket says 172.21.15.172:44228) 2024-09-10T09:03:54.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.227+0000 7f67818ff640 1 -- 172.21.15.172:0/2434990132 learned_addr learned my addr 172.21.15.172:0/2434990132 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:03:54.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.227+0000 7f6782901640 1 --2- 172.21.15.172:0/2434990132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556085552680 0x556085554a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:54.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.227+0000 7f67818ff640 1 -- 172.21.15.172:0/2434990132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556085552680 msgr2=0x556085554a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:54.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.227+0000 7f67818ff640 1 --2- 172.21.15.172:0/2434990132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556085552680 0x556085554a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.227+0000 7f67818ff640 1 -- 172.21.15.172:0/2434990132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560855474a0 msgr2=0x556085547880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.227+0000 7f67818ff640 1 --2- 172.21.15.172:0/2434990132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560855474a0 0x556085547880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.227+0000 7f67818ff640 1 -- 172.21.15.172:0/2434990132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556085439440 con 0x556085547dc0 2024-09-10T09:03:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.227+0000 7f6782100640 1 --2- 172.21.15.172:0/2434990132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560855474a0 0x556085547880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:03:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.227+0000 7f67818ff640 1 --2- 172.21.15.172:0/2434990132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085547dc0 0x556085552140 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f676c005bc0 tx=0x7f676c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47467e9f2d2d46ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.227+0000 7f6770dea640 1 -- 172.21.15.172:0/2434990132 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c005020 con 0x556085547dc0 2024-09-10T09:03:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.227+0000 7f6770dea640 1 -- 172.21.15.172:0/2434990132 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f676c0047a0 con 0x556085547dc0 2024-09-10T09:03:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.227+0000 7f6770dea640 1 -- 172.21.15.172:0/2434990132 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c0266e0 con 0x556085547dc0 2024-09-10T09:03:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.228+0000 7f678673d740 1 -- 172.21.15.172:0/2434990132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085547dc0 msgr2=0x556085552140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.228+0000 7f678673d740 1 --2- 172.21.15.172:0/2434990132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085547dc0 0x556085552140 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f676c005bc0 tx=0x7f676c014000 comp rx=0 tx=0).stop 2024-09-10T09:03:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.228+0000 7f678673d740 1 -- 172.21.15.172:0/2434990132 shutdown_connections 2024-09-10T09:03:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.228+0000 7f678673d740 1 --2- 172.21.15.172:0/2434990132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556085552680 0x556085554a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.228+0000 7f678673d740 1 --2- 172.21.15.172:0/2434990132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085547dc0 0x556085552140 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.228+0000 7f678673d740 1 --2- 172.21.15.172:0/2434990132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560855474a0 0x556085547880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.228+0000 7f678673d740 1 -- 172.21.15.172:0/2434990132 >> 172.21.15.172:0/2434990132 conn(0x556085543a30 msgr2=0x556085543e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.228+0000 7f678673d740 1 -- 172.21.15.172:0/2434990132 shutdown_connections 2024-09-10T09:03:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.228+0000 7f678673d740 1 -- 172.21.15.172:0/2434990132 wait complete. 2024-09-10T09:03:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.228+0000 7f678673d740 1 -- start start 2024-09-10T09:03:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.228+0000 7f678673d740 1 Processor -- start 2024-09-10T09:03:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f678673d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560855474a0 0x556085551fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f678673d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556085547dc0 0x55608554f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f678673d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085552680 0x55608554f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f678673d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556085555250 con 0x556085552680 2024-09-10T09:03:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f678673d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556085468050 con 0x556085547dc0 2024-09-10T09:03:54.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f678673d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5560855553d0 con 0x5560855474a0 2024-09-10T09:03:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f678673d740 10 client.0 _ll_register_callbacks cb 0x5560854a8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:03:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f6782901640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085552680 0x55608554f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f6782901640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085552680 0x55608554f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44234/0 (socket says 172.21.15.172:44234) 2024-09-10T09:03:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f6782901640 1 -- 172.21.15.172:0/2023679581 learned_addr learned my addr 172.21.15.172:0/2023679581 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:03:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f6782100640 1 --2- 172.21.15.172:0/2023679581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560855474a0 0x556085551fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f6782901640 1 -- 172.21.15.172:0/2023679581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560855474a0 msgr2=0x556085551fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f6782901640 1 --2- 172.21.15.172:0/2023679581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560855474a0 0x556085551fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f6782901640 1 -- 172.21.15.172:0/2023679581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556085547dc0 msgr2=0x55608554f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f6782901640 1 --2- 172.21.15.172:0/2023679581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556085547dc0 0x55608554f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f6782901640 1 -- 172.21.15.172:0/2023679581 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55608521e3c0 con 0x556085552680 2024-09-10T09:03:54.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f6782100640 1 --2- 172.21.15.172:0/2023679581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560855474a0 0x556085551fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:03:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f6782901640 1 --2- 172.21.15.172:0/2023679581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085552680 0x55608554f790 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f67740126a0 tx=0x7f6774012aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09256d22cbdf522 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f6765ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:03:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f6765ffb640 1 -- 172.21.15.172:0/2023679581 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774004d20 con 0x556085552680 2024-09-10T09:03:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f6765ffb640 1 -- 172.21.15.172:0/2023679581 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f67740107a0 con 0x556085552680 2024-09-10T09:03:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.229+0000 7f678673d740 1 -- 172.21.15.172:0/2023679581 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f676c0047a0 con 0x556085552680 2024-09-10T09:03:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.230+0000 7f6765ffb640 1 -- 172.21.15.172:0/2023679581 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f67740109a0 con 0x556085552680 2024-09-10T09:03:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.230+0000 7f678673d740 10 client.19734 fetch_fsmap learned FSMap version 44 2024-09-10T09:03:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.230+0000 7f678673d740 1 -- 172.21.15.172:0/2023679581 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x556085560c80 con 0x556085552680 2024-09-10T09:03:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.230+0000 7f6765ffb640 1 -- 172.21.15.172:0/2023679581 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f676c0047a0 con 0x556085552680 2024-09-10T09:03:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.230+0000 7f6765ffb640 1 -- 172.21.15.172:0/2023679581 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f677400f040 con 0x556085552680 2024-09-10T09:03:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.230+0000 7f678673d740 10 client.19734 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:03:54.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.230+0000 7f678673d740 10 client.19734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:03:54.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.230+0000 7f678673d740 20 client.19734 populate_metadata read hostname 'smithi172' 2024-09-10T09:03:54.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.230+0000 7f678673d740 1 --2- 172.21.15.172:0/2023679581 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5560855621e0 0x556085564680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.230+0000 7f678673d740 4 client.19734 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:03:54.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.230+0000 7f678673d740 1 -- 172.21.15.172:0/2023679581 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560854ba850 con 0x5560855621e0 2024-09-10T09:03:54.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.230+0000 7f6782100640 1 --2- 172.21.15.172:0/2023679581 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5560855621e0 0x556085564680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:54.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.230+0000 7f6765ffb640 10 client.19734 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:03:54.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.230+0000 7f6782100640 1 --2- 172.21.15.172:0/2023679581 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5560855621e0 0x556085564680 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5eae8e74b1997ce server_cookie=583a36f24af35ab2 in_seq=0 out_seq=0 2024-09-10T09:03:54.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.231+0000 7f6765ffb640 1 -- 172.21.15.172:0/2023679581 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560854ba850 con 0x5560855621e0 2024-09-10T09:03:54.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.231+0000 7f6765ffb640 10 client.19734 handle_command_reply: tid=0 2024-09-10T09:03:54.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.231+0000 7f678673d740 1 client.19734 shutdown 2024-09-10T09:03:54.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.231+0000 7f678673d740 1 -- 172.21.15.172:0/2023679581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085552680 msgr2=0x55608554f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:54.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.231+0000 7f678673d740 1 --2- 172.21.15.172:0/2023679581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085552680 0x55608554f790 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f67740126a0 tx=0x7f6774012aa0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.231+0000 7f678673d740 1 -- 172.21.15.172:0/2023679581 shutdown_connections 2024-09-10T09:03:54.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.231+0000 7f678673d740 1 --2- 172.21.15.172:0/2023679581 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5560855621e0 0x556085564680 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.231+0000 7f678673d740 1 --2- 172.21.15.172:0/2023679581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085552680 0x55608554f790 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.231+0000 7f6765ffb640 0 client.19734 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:03:54.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.231+0000 7f678673d740 1 --2- 172.21.15.172:0/2023679581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556085547dc0 0x55608554f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.231+0000 7f678673d740 1 --2- 172.21.15.172:0/2023679581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560855474a0 0x556085551fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.231+0000 7f678673d740 1 -- 172.21.15.172:0/2023679581 >> 172.21.15.172:0/2023679581 conn(0x556085543a30 msgr2=0x556085542b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:54.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.231+0000 7f678673d740 1 -- 172.21.15.172:0/2023679581 shutdown_connections 2024-09-10T09:03:54.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.231+0000 7f678673d740 1 -- 172.21.15.172:0/2023679581 wait complete. 2024-09-10T09:03:54.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.232+0000 7f678673d740 20 client.19734 trim_cache size 0 max 16384 2024-09-10T09:03:54.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.248+0000 7f678673d740 1 Processor -- start 2024-09-10T09:03:54.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f678673d740 1 -- start start 2024-09-10T09:03:54.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f678673d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085543a30 0x55608555e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f678673d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560855474a0 0x55608554f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f678673d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556085547dc0 0x55608554a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f678673d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1b0ca0 con 0x556085543a30 2024-09-10T09:03:54.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f678673d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1b27e0 con 0x5560855474a0 2024-09-10T09:03:54.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f678673d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1b1370 con 0x556085547dc0 2024-09-10T09:03:54.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f6782100640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085543a30 0x55608555e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:54.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f67818ff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560855474a0 0x55608554f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:54.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f6782100640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085543a30 0x55608555e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44246/0 (socket says 172.21.15.172:44246) 2024-09-10T09:03:54.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f6782100640 1 -- 172.21.15.172:0/1045684977 learned_addr learned my addr 172.21.15.172:0/1045684977 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:03:54.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f67818ff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560855474a0 0x55608554f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:54742/0 (socket says 172.21.15.172:54742) 2024-09-10T09:03:54.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f6782901640 1 --2- 172.21.15.172:0/1045684977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556085547dc0 0x55608554a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:54.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f6782100640 1 -- 172.21.15.172:0/1045684977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556085547dc0 msgr2=0x55608554a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:54.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f6782100640 1 --2- 172.21.15.172:0/1045684977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556085547dc0 0x55608554a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f6782100640 1 -- 172.21.15.172:0/1045684977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560855474a0 msgr2=0x55608554f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:54.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f6782100640 1 --2- 172.21.15.172:0/1045684977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560855474a0 0x55608554f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f6782100640 1 -- 172.21.15.172:0/1045684977 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556085552310 con 0x556085543a30 2024-09-10T09:03:54.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f6782901640 1 --2- 172.21.15.172:0/1045684977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556085547dc0 0x55608554a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:03:54.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f67818ff640 1 --2- 172.21.15.172:0/1045684977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560855474a0 0x55608554f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:03:54.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.249+0000 7f6782100640 1 --2- 172.21.15.172:0/1045684977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085543a30 0x55608555e4b0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f67780bbd70 tx=0x7f67780ecea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8146596f5e1bf73b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:54.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.250+0000 7f6767fff640 1 -- 172.21.15.172:0/1045684977 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67780d8070 con 0x556085543a30 2024-09-10T09:03:54.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.250+0000 7f6767fff640 1 -- 172.21.15.172:0/1045684977 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67780b9e80 con 0x556085543a30 2024-09-10T09:03:54.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.250+0000 7f6767fff640 1 -- 172.21.15.172:0/1045684977 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778049040 con 0x556085543a30 2024-09-10T09:03:54.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.250+0000 7f678673d740 1 -- 172.21.15.172:0/1045684977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085543a30 msgr2=0x55608555e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:54.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.250+0000 7f678673d740 1 --2- 172.21.15.172:0/1045684977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085543a30 0x55608555e4b0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f67780bbd70 tx=0x7f67780ecea0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.250+0000 7f678673d740 1 -- 172.21.15.172:0/1045684977 shutdown_connections 2024-09-10T09:03:54.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.250+0000 7f678673d740 1 --2- 172.21.15.172:0/1045684977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556085547dc0 0x55608554a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.250+0000 7f678673d740 1 --2- 172.21.15.172:0/1045684977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560855474a0 0x55608554f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.250+0000 7f678673d740 1 --2- 172.21.15.172:0/1045684977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085543a30 0x55608555e4b0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.250+0000 7f678673d740 1 -- 172.21.15.172:0/1045684977 >> 172.21.15.172:0/1045684977 conn(0x55608554f8f0 msgr2=0x55608554a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:54.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.250+0000 7f678673d740 1 -- 172.21.15.172:0/1045684977 shutdown_connections 2024-09-10T09:03:54.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.250+0000 7f678673d740 1 -- 172.21.15.172:0/1045684977 wait complete. 2024-09-10T09:03:54.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f678673d740 1 -- start start 2024-09-10T09:03:54.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f678673d740 1 Processor -- start 2024-09-10T09:03:54.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f678673d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556085543a30 0x55608555ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f678673d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560855474a0 0x556085556080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f678673d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085547dc0 0x5560855565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f678673d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1b2110 con 0x556085547dc0 2024-09-10T09:03:54.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f678673d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1b1a40 con 0x556085543a30 2024-09-10T09:03:54.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f678673d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f677c1aec80 con 0x5560855474a0 2024-09-10T09:03:54.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f678673d740 10 client.0 _ll_register_callbacks cb 0x7f6774009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:03:54.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f6782901640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085547dc0 0x5560855565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:54.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f6782901640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085547dc0 0x5560855565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44256/0 (socket says 172.21.15.172:44256) 2024-09-10T09:03:54.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f6782901640 1 -- 172.21.15.172:0/343904505 learned_addr learned my addr 172.21.15.172:0/343904505 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:03:54.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f67818ff640 1 --2- 172.21.15.172:0/343904505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560855474a0 0x556085556080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:54.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f6782100640 1 --2- 172.21.15.172:0/343904505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556085543a30 0x55608555ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:54.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f6782901640 1 -- 172.21.15.172:0/343904505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560855474a0 msgr2=0x556085556080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:54.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f6782901640 1 --2- 172.21.15.172:0/343904505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560855474a0 0x556085556080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f6782901640 1 -- 172.21.15.172:0/343904505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556085543a30 msgr2=0x55608555ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:03:54.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f6782901640 1 --2- 172.21.15.172:0/343904505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556085543a30 0x55608555ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f6782901640 1 -- 172.21.15.172:0/343904505 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556085556df0 con 0x556085547dc0 2024-09-10T09:03:54.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.251+0000 7f6782100640 1 --2- 172.21.15.172:0/343904505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556085543a30 0x55608555ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:03:54.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.252+0000 7f6782901640 1 --2- 172.21.15.172:0/343904505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085547dc0 0x5560855565c0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f6774013fd0 tx=0x7f6774009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafc1c4c2f5e72ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:03:54.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.252+0000 7f6765ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:03:54.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.252+0000 7f6765ffb640 1 -- 172.21.15.172:0/343904505 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677402e020 con 0x556085547dc0 2024-09-10T09:03:54.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.252+0000 7f6765ffb640 1 -- 172.21.15.172:0/343904505 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f6774010d40 con 0x556085547dc0 2024-09-10T09:03:54.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.252+0000 7f678673d740 1 -- 172.21.15.172:0/343904505 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67780b9e80 con 0x556085547dc0 2024-09-10T09:03:54.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.252+0000 7f6765ffb640 1 -- 172.21.15.172:0/343904505 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6774039050 con 0x556085547dc0 2024-09-10T09:03:54.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.252+0000 7f678673d740 10 client.19740 fetch_fsmap learned FSMap version 44 2024-09-10T09:03:54.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.252+0000 7f678673d740 1 -- 172.21.15.172:0/343904505 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5560855ccf20 con 0x556085547dc0 2024-09-10T09:03:54.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.252+0000 7f6765ffb640 1 -- 172.21.15.172:0/343904505 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67780b9e80 con 0x556085547dc0 2024-09-10T09:03:54.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.252+0000 7f6765ffb640 1 -- 172.21.15.172:0/343904505 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f6774005000 con 0x556085547dc0 2024-09-10T09:03:54.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.252+0000 7f678673d740 10 client.19740 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:03:54.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.252+0000 7f678673d740 10 client.19740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:03:54.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.252+0000 7f678673d740 20 client.19740 populate_metadata read hostname 'smithi172' 2024-09-10T09:03:54.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.252+0000 7f678673d740 1 --2- 172.21.15.172:0/343904505 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556085552680 0x5560855cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:03:54.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.252+0000 7f678673d740 4 client.19740 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:03:54.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.252+0000 7f678673d740 1 -- 172.21.15.172:0/343904505 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5560854b4d60 con 0x556085552680 2024-09-10T09:03:54.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.253+0000 7f6782100640 1 --2- 172.21.15.172:0/343904505 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556085552680 0x5560855cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:03:54.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.253+0000 7f6765ffb640 10 client.19740 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:03:54.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.253+0000 7f6782100640 1 --2- 172.21.15.172:0/343904505 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556085552680 0x5560855cf670 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b428445eaec7909 server_cookie=21b69ed5b211d018 in_seq=0 out_seq=0 2024-09-10T09:03:54.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.253+0000 7f6765ffb640 1 -- 172.21.15.172:0/343904505 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5560854b4d60 con 0x556085552680 2024-09-10T09:03:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.253+0000 7f6765ffb640 10 client.19740 handle_command_reply: tid=0 2024-09-10T09:03:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.253+0000 7f678673d740 1 client.19740 shutdown 2024-09-10T09:03:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.253+0000 7f678673d740 1 -- 172.21.15.172:0/343904505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085547dc0 msgr2=0x5560855565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.253+0000 7f678673d740 1 --2- 172.21.15.172:0/343904505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085547dc0 0x5560855565c0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f6774013fd0 tx=0x7f6774009ed0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.253+0000 7f678673d740 1 -- 172.21.15.172:0/343904505 shutdown_connections 2024-09-10T09:03:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.253+0000 7f678673d740 1 --2- 172.21.15.172:0/343904505 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x556085552680 0x5560855cf670 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.253+0000 7f678673d740 1 --2- 172.21.15.172:0/343904505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556085547dc0 0x5560855565c0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.253+0000 7f678673d740 1 --2- 172.21.15.172:0/343904505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5560855474a0 0x556085556080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.253+0000 7f678673d740 1 --2- 172.21.15.172:0/343904505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556085543a30 0x55608555ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.253+0000 7f6765ffb640 0 client.19740 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:03:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.253+0000 7f678673d740 1 -- 172.21.15.172:0/343904505 >> 172.21.15.172:0/343904505 conn(0x55608554f8f0 msgr2=0x55608554a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.253+0000 7f678673d740 1 -- 172.21.15.172:0/343904505 shutdown_connections 2024-09-10T09:03:54.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.253+0000 7f678673d740 1 -- 172.21.15.172:0/343904505 wait complete. 2024-09-10T09:03:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.254+0000 7f678673d740 20 client.19740 trim_cache size 0 max 16384 2024-09-10T09:03:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.254+0000 7f6784364640 1 -- 172.21.15.172:0/3340397104 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f67680a2c00 msgr2=0x7f67680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.254+0000 7f6784364640 1 --2- 172.21.15.172:0/3340397104 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f67680a2c00 0x7f67680a50c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f677c06ab00 tx=0x7f676c005f90 comp rx=0 tx=0).stop 2024-09-10T09:03:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.254+0000 7f6784364640 1 -- 172.21.15.172:0/3340397104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f677c100e20 msgr2=0x7f677c1991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:03:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.254+0000 7f6784364640 1 --2- 172.21.15.172:0/3340397104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f677c100e20 0x7f677c1991e0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f6778006010 tx=0x7f6778004270 comp rx=0 tx=0).stop 2024-09-10T09:03:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.254+0000 7f6784364640 1 -- 172.21.15.172:0/3340397104 shutdown_connections 2024-09-10T09:03:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.254+0000 7f6784364640 1 --2- 172.21.15.172:0/3340397104 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f67680a2c00 0x7f67680a50c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.254+0000 7f6784364640 1 --2- 172.21.15.172:0/3340397104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f677c102ef0 0x7f677c1a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.254+0000 7f6784364640 1 --2- 172.21.15.172:0/3340397104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f677c102570 0x7f677c199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.254+0000 7f6784364640 1 --2- 172.21.15.172:0/3340397104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f677c100e20 0x7f677c1991e0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:03:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.254+0000 7f6784364640 1 -- 172.21.15.172:0/3340397104 >> 172.21.15.172:0/3340397104 conn(0x7f677c0fc7d0 msgr2=0x7f677c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:03:54.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.254+0000 7f6784364640 1 -- 172.21.15.172:0/3340397104 shutdown_connections 2024-09-10T09:03:54.983 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:03:53.255+0000 7f6784364640 1 -- 172.21.15.172:0/3340397104 wait complete. 2024-09-10T09:03:55.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:54 smithi174 ceph-mon[34068]: pgmap v880: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 9.3 MiB/s wr, 2.37k op/s 2024-09-10T09:03:55.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:54 smithi191 ceph-mon[35314]: pgmap v880: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 9.3 MiB/s wr, 2.37k op/s 2024-09-10T09:03:55.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:54 smithi172 ceph-mon[30854]: pgmap v880: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 9.3 MiB/s wr, 2.37k op/s 2024-09-10T09:03:57.241 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:56 smithi172 ceph-mon[30854]: pgmap v881: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-09-10T09:03:57.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:56 smithi174 ceph-mon[34068]: pgmap v881: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-09-10T09:03:57.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:56 smithi191 ceph-mon[35314]: pgmap v881: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.9 MiB/s wr, 2.52k op/s 2024-09-10T09:03:58.397 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:03:59.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:03:58 smithi174 ceph-mon[34068]: pgmap v882: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T09:03:59.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:03:58 smithi191 ceph-mon[35314]: pgmap v882: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T09:03:59.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:03:58 smithi172 ceph-mon[30854]: pgmap v882: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 12 MiB/s wr, 2.96k op/s 2024-09-10T09:04:01.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:00 smithi174 ceph-mon[34068]: pgmap v883: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.7 MiB/s wr, 2.46k op/s 2024-09-10T09:04:01.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:00 smithi191 ceph-mon[35314]: pgmap v883: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.7 MiB/s wr, 2.46k op/s 2024-09-10T09:04:01.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:00 smithi172 ceph-mon[30854]: pgmap v883: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.7 MiB/s wr, 2.46k op/s 2024-09-10T09:04:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.539+0000 7f127effd640 1 -- 172.21.15.172:0/4224210425 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d7e30b6d70 con 0x55d7e31cc320 2024-09-10T09:04:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.539+0000 7f127effd640 10 client.28423 handle_command_reply: tid=0 2024-09-10T09:04:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.539+0000 7f12a8857740 1 client.28423 shutdown 2024-09-10T09:04:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.539+0000 7f12a8857740 1 -- 172.21.15.172:0/4224210425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e3151370 msgr2=0x55d7e315b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.539+0000 7f12a8857740 1 --2- 172.21.15.172:0/4224210425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e3151370 0x55d7e315b760 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1298055a00 tx=0x7f1298055cb0 comp rx=0 tx=0).stop 2024-09-10T09:04:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.539+0000 7f12a8857740 1 -- 172.21.15.172:0/4224210425 shutdown_connections 2024-09-10T09:04:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.539+0000 7f12a8857740 1 --2- 172.21.15.172:0/4224210425 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d7e31cc320 0x55d7e31ce740 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.539+0000 7f12a8857740 1 --2- 172.21.15.172:0/4224210425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7e3151370 0x55d7e315b760 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.539+0000 7f12a8857740 1 --2- 172.21.15.172:0/4224210425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7e314c2e0 0x55d7e315a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.539+0000 7f12a8857740 1 --2- 172.21.15.172:0/4224210425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7e314b9a0 0x55d7e315a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.539+0000 7f12a8857740 1 -- 172.21.15.172:0/4224210425 >> 172.21.15.172:0/4224210425 conn(0x55d7e3147010 msgr2=0x55d7e31459a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.539+0000 7f127effd640 0 client.28423 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.540+0000 7f12a8857740 1 -- 172.21.15.172:0/4224210425 shutdown_connections 2024-09-10T09:04:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.540+0000 7f12a8857740 1 -- 172.21.15.172:0/4224210425 wait complete. 2024-09-10T09:04:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.540+0000 7f12a8857740 20 client.28423 trim_cache size 0 max 16384 2024-09-10T09:04:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:04:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:04:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:04:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:04:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.541+0000 7f12a5aba640 1 -- 172.21.15.172:0/2625129650 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12880a2e70 msgr2=0x7f12880a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.541+0000 7f12a5aba640 1 --2- 172.21.15.172:0/2625129650 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12880a2e70 0x7f12880a5330 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55d7e314cd10 tx=0x7f129400d040 comp rx=0 tx=0).stop 2024-09-10T09:04:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.541+0000 7f12a5aba640 1 -- 172.21.15.172:0/2625129650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12a0073980 msgr2=0x7f12a00832e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.541+0000 7f12a5aba640 1 --2- 172.21.15.172:0/2625129650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12a0073980 0x7f12a00832e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f129000b450 tx=0x7f129000b910 comp rx=0 tx=0).stop 2024-09-10T09:04:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.541+0000 7f12a5aba640 1 -- 172.21.15.172:0/2625129650 shutdown_connections 2024-09-10T09:04:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.541+0000 7f12a5aba640 1 --2- 172.21.15.172:0/2625129650 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12880a2e70 0x7f12880a5330 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.541+0000 7f12a5aba640 1 --2- 172.21.15.172:0/2625129650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12a01096e0 0x7f12a0083820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.541+0000 7f12a5aba640 1 --2- 172.21.15.172:0/2625129650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12a0073980 0x7f12a00832e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.542+0000 7f12a5aba640 1 --2- 172.21.15.172:0/2625129650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12a0072fc0 0x7f12a0084c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.542+0000 7f12a5aba640 1 -- 172.21.15.172:0/2625129650 >> 172.21.15.172:0/2625129650 conn(0x7f12a006ece0 msgr2=0x7f12a00764c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.542+0000 7f12a5aba640 1 -- 172.21.15.172:0/2625129650 shutdown_connections 2024-09-10T09:04:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:02.542+0000 7f12a5aba640 1 -- 172.21.15.172:0/2625129650 wait complete. 2024-09-10T09:04:03.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:02 smithi174 ceph-mon[34068]: pgmap v884: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 2.89k op/s 2024-09-10T09:04:03.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:02 smithi191 ceph-mon[35314]: pgmap v884: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 2.89k op/s 2024-09-10T09:04:03.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:02 smithi172 ceph-mon[30854]: pgmap v884: 129 pgs: 129 active+clean; 13 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 2.89k op/s 2024-09-10T09:04:05.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:04 smithi174 ceph-mon[34068]: pgmap v885: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T09:04:05.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:04 smithi191 ceph-mon[35314]: pgmap v885: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T09:04:05.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:04 smithi172 ceph-mon[30854]: pgmap v885: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 MiB/s wr, 2.69k op/s 2024-09-10T09:04:07.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:06 smithi174 ceph-mon[34068]: pgmap v886: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-10T09:04:07.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:06 smithi191 ceph-mon[35314]: pgmap v886: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-10T09:04:07.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:06 smithi172 ceph-mon[30854]: pgmap v886: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 MiB/s wr, 2.94k op/s 2024-09-10T09:04:07.976 INFO:tasks.cephfs.filesystem:scrub status for tag:097ab02a-d7be-4e1d-96ec-78f5d8c3836d - {'path': '/', 'tag': '097ab02a-d7be-4e1d-96ec-78f5d8c3836d', 'options': 'recursive,force'} 2024-09-10T09:04:09.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:08 smithi174 ceph-mon[34068]: pgmap v887: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 3.24k op/s 2024-09-10T09:04:09.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:08 smithi191 ceph-mon[35314]: pgmap v887: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 3.24k op/s 2024-09-10T09:04:09.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:08 smithi172 ceph-mon[30854]: pgmap v887: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 3.24k op/s 2024-09-10T09:04:11.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:10 smithi174 ceph-mon[34068]: pgmap v888: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 MiB/s wr, 2.50k op/s 2024-09-10T09:04:11.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:10 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:04:11.258 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:10 smithi172 ceph-mon[30854]: pgmap v888: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 MiB/s wr, 2.50k op/s 2024-09-10T09:04:11.259 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:10 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:04:11.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:10 smithi191 ceph-mon[35314]: pgmap v888: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.8 MiB/s wr, 2.50k op/s 2024-09-10T09:04:11.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:10 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:04:13.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:12 smithi174 ceph-mon[34068]: pgmap v889: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 2.93k op/s 2024-09-10T09:04:13.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:12 smithi191 ceph-mon[35314]: pgmap v889: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 2.93k op/s 2024-09-10T09:04:13.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:12 smithi172 ceph-mon[30854]: pgmap v889: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 MiB/s wr, 2.93k op/s 2024-09-10T09:04:15.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:14 smithi174 ceph-mon[34068]: pgmap v890: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 2.67k op/s 2024-09-10T09:04:15.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:14 smithi191 ceph-mon[35314]: pgmap v890: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 2.67k op/s 2024-09-10T09:04:15.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:14 smithi172 ceph-mon[30854]: pgmap v890: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 11 MiB/s wr, 2.67k op/s 2024-09-10T09:04:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.646+0000 7f92a740e640 1 -- 172.21.15.172:0/1977374638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92a0100a40 msgr2=0x7f92a0100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.646+0000 7f92a740e640 1 --2- 172.21.15.172:0/1977374638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92a0100a40 0x7f92a0100e40 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f9290009ca0 tx=0x7f929002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.646+0000 7f92a740e640 1 -- 172.21.15.172:0/1977374638 shutdown_connections 2024-09-10T09:04:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.646+0000 7f92a740e640 1 --2- 172.21.15.172:0/1977374638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92a01025f0 0x7f92a0108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.646+0000 7f92a740e640 1 --2- 172.21.15.172:0/1977374638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92a0101c30 0x7f92a01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.646+0000 7f92a740e640 1 --2- 172.21.15.172:0/1977374638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92a0100a40 0x7f92a0100e40 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.647+0000 7f92a740e640 1 -- 172.21.15.172:0/1977374638 >> 172.21.15.172:0/1977374638 conn(0x7f92a00fc1e0 msgr2=0x7f92a00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.647+0000 7f92a740e640 1 -- 172.21.15.172:0/1977374638 shutdown_connections 2024-09-10T09:04:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.647+0000 7f92a740e640 1 -- 172.21.15.172:0/1977374638 wait complete. 2024-09-10T09:04:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.647+0000 7f92a740e640 1 Processor -- start 2024-09-10T09:04:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.647+0000 7f92a740e640 1 -- start start 2024-09-10T09:04:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.647+0000 7f92a740e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92a0100a40 0x7f92a0197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.647+0000 7f92a740e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92a0101c30 0x7f92a0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.647+0000 7f92a740e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92a01025f0 0x7f92a019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.647+0000 7f92a740e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f92a010d680 con 0x7f92a01025f0 2024-09-10T09:04:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.647+0000 7f92a740e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f92a010d500 con 0x7f92a0100a40 2024-09-10T09:04:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.647+0000 7f92a740e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f92a010d800 con 0x7f92a0101c30 2024-09-10T09:04:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.647+0000 7f92a59ab640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92a01025f0 0x7f92a019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.648+0000 7f92a59ab640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92a01025f0 0x7f92a019f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45362/0 (socket says 172.21.15.172:45362) 2024-09-10T09:04:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.648+0000 7f92a59ab640 1 -- 172.21.15.172:0/2436574996 learned_addr learned my addr 172.21.15.172:0/2436574996 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.648+0000 7f92a49a9640 1 --2- 172.21.15.172:0/2436574996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92a0101c30 0x7f92a0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.648+0000 7f92a51aa640 1 --2- 172.21.15.172:0/2436574996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92a0100a40 0x7f92a0197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.648+0000 7f92a59ab640 1 -- 172.21.15.172:0/2436574996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92a0101c30 msgr2=0x7f92a0198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.648+0000 7f92a59ab640 1 --2- 172.21.15.172:0/2436574996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92a0101c30 0x7f92a0198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.648+0000 7f92a59ab640 1 -- 172.21.15.172:0/2436574996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92a0100a40 msgr2=0x7f92a0197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.648+0000 7f92a59ab640 1 --2- 172.21.15.172:0/2436574996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92a0100a40 0x7f92a0197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.648+0000 7f92a59ab640 1 -- 172.21.15.172:0/2436574996 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a019f900 con 0x7f92a01025f0 2024-09-10T09:04:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.648+0000 7f92a59ab640 1 --2- 172.21.15.172:0/2436574996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92a01025f0 0x7f92a019f260 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f929800c970 tx=0x7f929800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.648+0000 7f929e7fc640 1 -- 172.21.15.172:0/2436574996 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298007bb0 con 0x7f92a01025f0 2024-09-10T09:04:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.648+0000 7f929e7fc640 1 -- 172.21.15.172:0/2436574996 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9298007d50 con 0x7f92a01025f0 2024-09-10T09:04:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.648+0000 7f92a740e640 1 -- 172.21.15.172:0/2436574996 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a019fbf0 con 0x7f92a01025f0 2024-09-10T09:04:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.648+0000 7f929e7fc640 1 -- 172.21.15.172:0/2436574996 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298018700 con 0x7f92a01025f0 2024-09-10T09:04:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.648+0000 7f92a740e640 1 -- 172.21.15.172:0/2436574996 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a01a0140 con 0x7f92a01025f0 2024-09-10T09:04:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.651+0000 7f92aa283740 1 Processor -- start 2024-09-10T09:04:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.651+0000 7f929e7fc640 1 -- 172.21.15.172:0/2436574996 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f929800fa20 con 0x7f92a01025f0 2024-09-10T09:04:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.651+0000 7f92aa283740 1 -- start start 2024-09-10T09:04:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92aa283740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b2bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92aa283740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2c2e0 0x555690b30da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92aa283740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 0x555690b33830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92aa283740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555690a8f2f0 con 0x555690b2b9a0 2024-09-10T09:04:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92aa283740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555690a37270 con 0x555690b31370 2024-09-10T09:04:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92aa283740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555690a8b2b0 con 0x555690b2c2e0 2024-09-10T09:04:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92a49a9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b2bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92a51aa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2c2e0 0x555690b30da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92a49a9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b2bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45364/0 (socket says 172.21.15.172:45364) 2024-09-10T09:04:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92a49a9640 1 -- 172.21.15.172:0/1996515399 learned_addr learned my addr 172.21.15.172:0/1996515399 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92a49a9640 1 --2- 172.21.15.172:0/1996515399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 0x555690b33830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92a49a9640 1 -- 172.21.15.172:0/1996515399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2c2e0 msgr2=0x555690b30da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92a49a9640 1 --2- 172.21.15.172:0/1996515399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2c2e0 0x555690b30da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92a49a9640 1 -- 172.21.15.172:0/1996515399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 msgr2=0x555690b33830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f929e7fc640 1 --2- 172.21.15.172:0/2436574996 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f92880a2cb0 0x7f92880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92a49a9640 1 --2- 172.21.15.172:0/1996515399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 0x555690b33830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92a49a9640 1 -- 172.21.15.172:0/1996515399 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555690a36340 con 0x555690b2b9a0 2024-09-10T09:04:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92a59ab640 1 --2- 172.21.15.172:0/2436574996 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f92880a2cb0 0x7f92880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92a49a9640 1 --2- 172.21.15.172:0/1996515399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b2bda0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f9294013580 tx=0x7f929400a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57017b2ae875fb46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f929e7fc640 1 -- 172.21.15.172:0/2436574996 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f92980cf350 con 0x7f92a01025f0 2024-09-10T09:04:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.652+0000 7f92877fe640 1 -- 172.21.15.172:0/1996515399 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294019030 con 0x555690b2b9a0 2024-09-10T09:04:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.653+0000 7f92877fe640 1 -- 172.21.15.172:0/1996515399 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92940054d0 con 0x555690b2b9a0 2024-09-10T09:04:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.653+0000 7f92877fe640 1 -- 172.21.15.172:0/1996515399 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929400f630 con 0x555690b2b9a0 2024-09-10T09:04:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.653+0000 7f92aa283740 1 -- 172.21.15.172:0/1996515399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 msgr2=0x555690b2bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.653+0000 7f92a59ab640 1 --2- 172.21.15.172:0/2436574996 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f92880a2cb0 0x7f92880a5170 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f929809b550 tx=0x7f92980664d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.653+0000 7f92aa283740 1 --2- 172.21.15.172:0/1996515399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b2bda0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f9294013580 tx=0x7f929400a760 comp rx=0 tx=0).stop 2024-09-10T09:04:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.653+0000 7f92aa283740 1 -- 172.21.15.172:0/1996515399 shutdown_connections 2024-09-10T09:04:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.653+0000 7f92aa283740 1 --2- 172.21.15.172:0/1996515399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 0x555690b33830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.653+0000 7f92aa283740 1 --2- 172.21.15.172:0/1996515399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2c2e0 0x555690b30da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.653+0000 7f92aa283740 1 --2- 172.21.15.172:0/1996515399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b2bda0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.653+0000 7f92aa283740 1 -- 172.21.15.172:0/1996515399 >> 172.21.15.172:0/1996515399 conn(0x555690b27010 msgr2=0x555690b27410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.653+0000 7f92aa283740 1 -- 172.21.15.172:0/1996515399 shutdown_connections 2024-09-10T09:04:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.653+0000 7f92aa283740 1 -- 172.21.15.172:0/1996515399 wait complete. 2024-09-10T09:04:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92aa283740 1 -- start start 2024-09-10T09:04:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92aa283740 1 Processor -- start 2024-09-10T09:04:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92aa283740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b35b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92aa283740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2c2e0 0x555690b360b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92aa283740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 0x555690b3d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92aa283740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555690b34340 con 0x555690b2b9a0 2024-09-10T09:04:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92aa283740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555690a18290 con 0x555690b31370 2024-09-10T09:04:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92aa283740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555690b344c0 con 0x555690b2c2e0 2024-09-10T09:04:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92aa283740 10 client.0 _ll_register_callbacks cb 0x555690a49140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92a49a9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b35b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92a49a9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b35b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45376/0 (socket says 172.21.15.172:45376) 2024-09-10T09:04:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92a49a9640 1 -- 172.21.15.172:0/36682839 learned_addr learned my addr 172.21.15.172:0/36682839 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92a51aa640 1 --2- 172.21.15.172:0/36682839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2c2e0 0x555690b360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92a49a9640 1 --2- 172.21.15.172:0/36682839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 0x555690b3d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92a49a9640 1 -- 172.21.15.172:0/36682839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2c2e0 msgr2=0x555690b360b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92a49a9640 1 --2- 172.21.15.172:0/36682839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2c2e0 0x555690b360b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92a49a9640 1 -- 172.21.15.172:0/36682839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 msgr2=0x555690b3d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92a49a9640 1 --2- 172.21.15.172:0/36682839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 0x555690b3d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.654+0000 7f92a49a9640 1 -- 172.21.15.172:0/36682839 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556907a2210 con 0x555690b2b9a0 2024-09-10T09:04:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.655+0000 7f92a49a9640 1 --2- 172.21.15.172:0/36682839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b35b70 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f929400a640 tx=0x7f92940048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56859ddce269183b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.655+0000 7f9284ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.655+0000 7f9284ff9640 1 -- 172.21.15.172:0/36682839 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294019040 con 0x555690b2b9a0 2024-09-10T09:04:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.655+0000 7f9284ff9640 1 -- 172.21.15.172:0/36682839 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9294004020 con 0x555690b2b9a0 2024-09-10T09:04:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.655+0000 7f92aa283740 1 -- 172.21.15.172:0/36682839 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92940054d0 con 0x555690b2b9a0 2024-09-10T09:04:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.655+0000 7f9284ff9640 1 -- 172.21.15.172:0/36682839 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9294004220 con 0x555690b2b9a0 2024-09-10T09:04:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.655+0000 7f92aa283740 10 client.19770 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.655+0000 7f92aa283740 1 -- 172.21.15.172:0/36682839 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55569086e620 con 0x555690b2b9a0 2024-09-10T09:04:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.655+0000 7f9284ff9640 1 -- 172.21.15.172:0/36682839 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92940054d0 con 0x555690b2b9a0 2024-09-10T09:04:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.655+0000 7f9284ff9640 1 -- 172.21.15.172:0/36682839 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f92940035f0 con 0x555690b2b9a0 2024-09-10T09:04:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.655+0000 7f92aa283740 10 client.19770 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.655+0000 7f92aa283740 10 client.19770 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:04:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.655+0000 7f92aa283740 20 client.19770 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.655+0000 7f92aa283740 1 --2- 172.21.15.172:0/36682839 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555690b415f0 0x555690b439e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.655+0000 7f92aa283740 4 client.19770 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:04:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.655+0000 7f92aa283740 1 -- 172.21.15.172:0/36682839 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555690a9ca20 con 0x555690b415f0 2024-09-10T09:04:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.656+0000 7f92a51aa640 1 --2- 172.21.15.172:0/36682839 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555690b415f0 0x555690b439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.656+0000 7f9284ff9640 10 client.19770 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.656+0000 7f92a51aa640 1 --2- 172.21.15.172:0/36682839 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555690b415f0 0x555690b439e0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88e0c07e137a765e server_cookie=73821e05f8fb7e76 in_seq=0 out_seq=0 2024-09-10T09:04:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.657+0000 7f9284ff9640 1 -- 172.21.15.172:0/36682839 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555690a9ca20 con 0x555690b415f0 2024-09-10T09:04:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.657+0000 7f9284ff9640 10 client.19770 handle_command_reply: tid=0 2024-09-10T09:04:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.657+0000 7f92aa283740 1 client.19770 shutdown 2024-09-10T09:04:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.657+0000 7f92aa283740 1 -- 172.21.15.172:0/36682839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 msgr2=0x555690b35b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.657+0000 7f92aa283740 1 --2- 172.21.15.172:0/36682839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b35b70 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f929400a640 tx=0x7f92940048b0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.657+0000 7f92aa283740 1 -- 172.21.15.172:0/36682839 shutdown_connections 2024-09-10T09:04:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.657+0000 7f92aa283740 1 --2- 172.21.15.172:0/36682839 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555690b415f0 0x555690b439e0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.657+0000 7f92aa283740 1 --2- 172.21.15.172:0/36682839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 0x555690b3d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.657+0000 7f92aa283740 1 --2- 172.21.15.172:0/36682839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2c2e0 0x555690b360b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.657+0000 7f9284ff9640 0 client.19770 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.657+0000 7f92aa283740 1 --2- 172.21.15.172:0/36682839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b35b70 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.657+0000 7f92aa283740 1 -- 172.21.15.172:0/36682839 >> 172.21.15.172:0/36682839 conn(0x555690b27010 msgr2=0x555690b28820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.657+0000 7f92aa283740 1 -- 172.21.15.172:0/36682839 shutdown_connections 2024-09-10T09:04:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.657+0000 7f92aa283740 1 -- 172.21.15.172:0/36682839 wait complete. 2024-09-10T09:04:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.657+0000 7f92aa283740 20 client.19770 trim_cache size 0 max 16384 2024-09-10T09:04:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.672+0000 7f92aa283740 1 Processor -- start 2024-09-10T09:04:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.672+0000 7f92aa283740 1 -- start start 2024-09-10T09:04:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.672+0000 7f92aa283740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b36140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.672+0000 7f92aa283740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2c2e0 0x555690b3d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.672+0000 7f92aa283740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 0x555690b269b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.672+0000 7f92aa283740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01aa4c0 con 0x555690b2b9a0 2024-09-10T09:04:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.672+0000 7f92aa283740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01ac000 con 0x555690b31370 2024-09-10T09:04:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.672+0000 7f92aa283740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01aab90 con 0x555690b2c2e0 2024-09-10T09:04:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.672+0000 7f92a49a9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b36140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.672+0000 7f92a49a9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b36140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45378/0 (socket says 172.21.15.172:45378) 2024-09-10T09:04:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.672+0000 7f92a49a9640 1 -- 172.21.15.172:0/1283898788 learned_addr learned my addr 172.21.15.172:0/1283898788 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.672+0000 7f92a49a9640 1 --2- 172.21.15.172:0/1283898788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 0x555690b269b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.672+0000 7f92a51aa640 1 --2- 172.21.15.172:0/1283898788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2c2e0 0x555690b3d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.673+0000 7f92a49a9640 1 -- 172.21.15.172:0/1283898788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2c2e0 msgr2=0x555690b3d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.673+0000 7f92a49a9640 1 --2- 172.21.15.172:0/1283898788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2c2e0 0x555690b3d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.673+0000 7f92a49a9640 1 -- 172.21.15.172:0/1283898788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 msgr2=0x555690b269b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.673+0000 7f92a49a9640 1 --2- 172.21.15.172:0/1283898788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 0x555690b269b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.673+0000 7f92a49a9640 1 -- 172.21.15.172:0/1283898788 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555690b43d50 con 0x555690b2b9a0 2024-09-10T09:04:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.673+0000 7f92a51aa640 1 --2- 172.21.15.172:0/1283898788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2c2e0 0x555690b3d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:04:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.673+0000 7f92a49a9640 1 --2- 172.21.15.172:0/1283898788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b36140 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f9294000c00 tx=0x7f9294005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aff18e4b77b7313 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.673+0000 7f9286ffd640 1 -- 172.21.15.172:0/1283898788 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929403b070 con 0x555690b2b9a0 2024-09-10T09:04:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.673+0000 7f9286ffd640 1 -- 172.21.15.172:0/1283898788 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f929400a670 con 0x555690b2b9a0 2024-09-10T09:04:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.673+0000 7f9286ffd640 1 -- 172.21.15.172:0/1283898788 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294019040 con 0x555690b2b9a0 2024-09-10T09:04:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 1 -- 172.21.15.172:0/1283898788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 msgr2=0x555690b36140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 1 --2- 172.21.15.172:0/1283898788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b36140 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f9294000c00 tx=0x7f9294005dc0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 1 -- 172.21.15.172:0/1283898788 shutdown_connections 2024-09-10T09:04:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 1 --2- 172.21.15.172:0/1283898788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 0x555690b269b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 1 --2- 172.21.15.172:0/1283898788 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2c2e0 0x555690b3d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 1 --2- 172.21.15.172:0/1283898788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2b9a0 0x555690b36140 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 1 -- 172.21.15.172:0/1283898788 >> 172.21.15.172:0/1283898788 conn(0x555690b27010 msgr2=0x555690b27f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 1 -- 172.21.15.172:0/1283898788 shutdown_connections 2024-09-10T09:04:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 1 -- 172.21.15.172:0/1283898788 wait complete. 2024-09-10T09:04:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 1 -- start start 2024-09-10T09:04:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 1 Processor -- start 2024-09-10T09:04:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2b9a0 0x555690b3a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2c2e0 0x555690b3aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 0x555690b3bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01ab930 con 0x555690b2c2e0 2024-09-10T09:04:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01ab260 con 0x555690b31370 2024-09-10T09:04:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01a84a0 con 0x555690b2b9a0 2024-09-10T09:04:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92aa283740 10 client.0 _ll_register_callbacks cb 0x7f929400abc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92a51aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2c2e0 0x555690b3aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.674+0000 7f92a51aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2c2e0 0x555690b3aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45382/0 (socket says 172.21.15.172:45382) 2024-09-10T09:04:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.675+0000 7f92a51aa640 1 -- 172.21.15.172:0/82619777 learned_addr learned my addr 172.21.15.172:0/82619777 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.675+0000 7f92a49a9640 1 --2- 172.21.15.172:0/82619777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2b9a0 0x555690b3a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.675+0000 7f92a51aa640 1 -- 172.21.15.172:0/82619777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2b9a0 msgr2=0x555690b3a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.675+0000 7f92a51aa640 1 --2- 172.21.15.172:0/82619777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2b9a0 0x555690b3a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.675+0000 7f92a51aa640 1 -- 172.21.15.172:0/82619777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 msgr2=0x555690b3bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.675+0000 7f92a51aa640 1 --2- 172.21.15.172:0/82619777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 0x555690b3bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.675+0000 7f92a51aa640 1 -- 172.21.15.172:0/82619777 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555690b3c0f0 con 0x555690b2c2e0 2024-09-10T09:04:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.675+0000 7f92a49a9640 1 --2- 172.21.15.172:0/82619777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2b9a0 0x555690b3a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:04:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.675+0000 7f92a51aa640 1 --2- 172.21.15.172:0/82619777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2c2e0 0x555690b3aad0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f9290009a60 tx=0x7f92900314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f402447531b3b6e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.675+0000 7f9284ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.675+0000 7f9284ff9640 1 -- 172.21.15.172:0/82619777 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290055060 con 0x555690b2c2e0 2024-09-10T09:04:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.675+0000 7f9284ff9640 1 -- 172.21.15.172:0/82619777 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f92900307c0 con 0x555690b2c2e0 2024-09-10T09:04:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.675+0000 7f92aa283740 1 -- 172.21.15.172:0/82619777 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f929400a670 con 0x555690b2c2e0 2024-09-10T09:04:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.675+0000 7f9284ff9640 1 -- 172.21.15.172:0/82619777 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f929005c050 con 0x555690b2c2e0 2024-09-10T09:04:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.675+0000 7f92aa283740 10 client.19776 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.675+0000 7f92aa283740 1 -- 172.21.15.172:0/82619777 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555690bac2a0 con 0x555690b2c2e0 2024-09-10T09:04:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.675+0000 7f9284ff9640 1 -- 172.21.15.172:0/82619777 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f929400a670 con 0x555690b2c2e0 2024-09-10T09:04:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.676+0000 7f9284ff9640 1 -- 172.21.15.172:0/82619777 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f929004f030 con 0x555690b2c2e0 2024-09-10T09:04:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.676+0000 7f92aa283740 10 client.19776 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.676+0000 7f92aa283740 10 client.19776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:04:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.676+0000 7f92aa283740 20 client.19776 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.676+0000 7f92aa283740 1 --2- 172.21.15.172:0/82619777 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555690bacac0 0x555690baeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.676+0000 7f92aa283740 4 client.19776 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:04:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.676+0000 7f92aa283740 1 -- 172.21.15.172:0/82619777 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555690a96d70 con 0x555690bacac0 2024-09-10T09:04:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.676+0000 7f92a49a9640 1 --2- 172.21.15.172:0/82619777 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555690bacac0 0x555690baeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.676+0000 7f9284ff9640 10 client.19776 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.676+0000 7f92a49a9640 1 --2- 172.21.15.172:0/82619777 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555690bacac0 0x555690baeee0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec676a04412875c8 server_cookie=d4d401b6822c4d0d in_seq=0 out_seq=0 2024-09-10T09:04:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.687+0000 7f9284ff9640 1 -- 172.21.15.172:0/82619777 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555690a96d70 con 0x555690bacac0 2024-09-10T09:04:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.687+0000 7f9284ff9640 10 client.19776 handle_command_reply: tid=0 2024-09-10T09:04:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.687+0000 7f92aa283740 1 client.19776 shutdown 2024-09-10T09:04:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.687+0000 7f92aa283740 1 -- 172.21.15.172:0/82619777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2c2e0 msgr2=0x555690b3aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.687+0000 7f92aa283740 1 --2- 172.21.15.172:0/82619777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2c2e0 0x555690b3aad0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f9290009a60 tx=0x7f92900314e0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.687+0000 7f92aa283740 1 -- 172.21.15.172:0/82619777 shutdown_connections 2024-09-10T09:04:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.687+0000 7f92aa283740 1 --2- 172.21.15.172:0/82619777 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x555690bacac0 0x555690baeee0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.687+0000 7f92aa283740 1 --2- 172.21.15.172:0/82619777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555690b31370 0x555690b3bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.687+0000 7f9284ff9640 0 client.19776 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.687+0000 7f92aa283740 1 --2- 172.21.15.172:0/82619777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555690b2c2e0 0x555690b3aad0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.687+0000 7f92aa283740 1 --2- 172.21.15.172:0/82619777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555690b2b9a0 0x555690b3a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.687+0000 7f92aa283740 1 -- 172.21.15.172:0/82619777 >> 172.21.15.172:0/82619777 conn(0x555690b27010 msgr2=0x555690b27f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.687+0000 7f92aa283740 1 -- 172.21.15.172:0/82619777 shutdown_connections 2024-09-10T09:04:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.687+0000 7f92aa283740 1 -- 172.21.15.172:0/82619777 wait complete. 2024-09-10T09:04:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.687+0000 7f92aa283740 20 client.19776 trim_cache size 0 max 16384 2024-09-10T09:04:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:04:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:04:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:04:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:04:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.688+0000 7f92a740e640 1 -- 172.21.15.172:0/2436574996 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f92880a2cb0 msgr2=0x7f92880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.688+0000 7f92a740e640 1 --2- 172.21.15.172:0/2436574996 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f92880a2cb0 0x7f92880a5170 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f929809b550 tx=0x7f92980664d0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.688+0000 7f92a740e640 1 -- 172.21.15.172:0/2436574996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92a01025f0 msgr2=0x7f92a019f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.688+0000 7f92a740e640 1 --2- 172.21.15.172:0/2436574996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92a01025f0 0x7f92a019f260 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f929800c970 tx=0x7f929800ce30 comp rx=0 tx=0).stop 2024-09-10T09:04:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.688+0000 7f92a740e640 1 -- 172.21.15.172:0/2436574996 shutdown_connections 2024-09-10T09:04:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.688+0000 7f92a740e640 1 --2- 172.21.15.172:0/2436574996 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f92880a2cb0 0x7f92880a5170 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.688+0000 7f92a740e640 1 --2- 172.21.15.172:0/2436574996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92a01025f0 0x7f92a019f260 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.688+0000 7f92a740e640 1 --2- 172.21.15.172:0/2436574996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92a0101c30 0x7f92a0198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.688+0000 7f92a740e640 1 --2- 172.21.15.172:0/2436574996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92a0100a40 0x7f92a0197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.688+0000 7f92a740e640 1 -- 172.21.15.172:0/2436574996 >> 172.21.15.172:0/2436574996 conn(0x7f92a00fc1e0 msgr2=0x7f92a0105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.688+0000 7f92a740e640 1 -- 172.21.15.172:0/2436574996 shutdown_connections 2024-09-10T09:04:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:15.688+0000 7f92a740e640 1 -- 172.21.15.172:0/2436574996 wait complete. 2024-09-10T09:04:15.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.215+0000 7f16315f6640 1 -- 172.21.15.172:0/881182303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f162c101de0 msgr2=0x7f162c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.215+0000 7f16315f6640 1 --2- 172.21.15.172:0/881182303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f162c101de0 0x7f162c1021c0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f161c009900 tx=0x7f161c02a410 comp rx=0 tx=0).stop 2024-09-10T09:04:15.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.215+0000 7f16315f6640 1 -- 172.21.15.172:0/881182303 shutdown_connections 2024-09-10T09:04:15.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.215+0000 7f16315f6640 1 --2- 172.21.15.172:0/881182303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f162c106f10 0x7f162c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.215+0000 7f16315f6640 1 --2- 172.21.15.172:0/881182303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f162c102790 0x7f162c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.215+0000 7f16315f6640 1 --2- 172.21.15.172:0/881182303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f162c101de0 0x7f162c1021c0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.215+0000 7f16315f6640 1 -- 172.21.15.172:0/881182303 >> 172.21.15.172:0/881182303 conn(0x7f162c0fd8c0 msgr2=0x7f162c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:15.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.215+0000 7f16315f6640 1 -- 172.21.15.172:0/881182303 shutdown_connections 2024-09-10T09:04:15.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.215+0000 7f16315f6640 1 -- 172.21.15.172:0/881182303 wait complete. 2024-09-10T09:04:15.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f16315f6640 1 Processor -- start 2024-09-10T09:04:15.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f16315f6640 1 -- start start 2024-09-10T09:04:15.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f16315f6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f162c101de0 0x7f162c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f16315f6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f162c102790 0x7f162c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f16315f6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f162c106f10 0x7f162c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f16315f6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f162c111e20 con 0x7f162c106f10 2024-09-10T09:04:15.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f16315f6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f162c111ca0 con 0x7f162c102790 2024-09-10T09:04:15.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f16315f6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f162c111fa0 con 0x7f162c101de0 2024-09-10T09:04:15.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f162b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f162c106f10 0x7f162c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f162b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f162c106f10 0x7f162c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45308/0 (socket says 172.21.15.172:45308) 2024-09-10T09:04:15.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f162b7fe640 1 -- 172.21.15.172:0/3514993852 learned_addr learned my addr 172.21.15.172:0/3514993852 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:15.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f162affd640 1 --2- 172.21.15.172:0/3514993852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f162c101de0 0x7f162c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f1623fff640 1 --2- 172.21.15.172:0/3514993852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f162c102790 0x7f162c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f162b7fe640 1 -- 172.21.15.172:0/3514993852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f162c101de0 msgr2=0x7f162c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f162b7fe640 1 --2- 172.21.15.172:0/3514993852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f162c101de0 0x7f162c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f162b7fe640 1 -- 172.21.15.172:0/3514993852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f162c102790 msgr2=0x7f162c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f162b7fe640 1 --2- 172.21.15.172:0/3514993852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f162c102790 0x7f162c198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f162b7fe640 1 -- 172.21.15.172:0/3514993852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c19d2e0 con 0x7f162c106f10 2024-09-10T09:04:15.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.216+0000 7f1623fff640 1 --2- 172.21.15.172:0/3514993852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f162c102790 0x7f162c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:04:15.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.217+0000 7f162b7fe640 1 --2- 172.21.15.172:0/3514993852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f162c106f10 0x7f162c19cc00 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f162400eaa0 tx=0x7f162400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:15.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.217+0000 7f1628ff9640 1 -- 172.21.15.172:0/3514993852 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162400cdb0 con 0x7f162c106f10 2024-09-10T09:04:15.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.217+0000 7f1628ff9640 1 -- 172.21.15.172:0/3514993852 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1624005020 con 0x7f162c106f10 2024-09-10T09:04:15.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.217+0000 7f16315f6640 1 -- 172.21.15.172:0/3514993852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c19d5d0 con 0x7f162c106f10 2024-09-10T09:04:15.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.217+0000 7f1628ff9640 1 -- 172.21.15.172:0/3514993852 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16240116d0 con 0x7f162c106f10 2024-09-10T09:04:15.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.217+0000 7f16315f6640 1 -- 172.21.15.172:0/3514993852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c1a4eb0 con 0x7f162c106f10 2024-09-10T09:04:15.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.219+0000 7f1628ff9640 1 -- 172.21.15.172:0/3514993852 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f16240051d0 con 0x7f162c106f10 2024-09-10T09:04:15.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.219+0000 7f16339cf740 1 Processor -- start 2024-09-10T09:04:15.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.219+0000 7f16339cf740 1 -- start start 2024-09-10T09:04:15.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.219+0000 7f16339cf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 0x558fa37e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.219+0000 7f16339cf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 0x558fa37ed4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.219+0000 7f16339cf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37eda30 0x558fa37efe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.219+0000 7f16339cf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558fa3747160 con 0x558fa37e2850 2024-09-10T09:04:15.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.219+0000 7f16339cf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558fa36ee720 con 0x558fa37e3170 2024-09-10T09:04:15.713 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.219+0000 7f16339cf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558fa3743120 con 0x558fa37eda30 2024-09-10T09:04:15.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f1623fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 0x558fa37e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f1628ff9640 1 --2- 172.21.15.172:0/3514993852 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16180a2c00 0x7f16180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f1623fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 0x558fa37e2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45314/0 (socket says 172.21.15.172:45314) 2024-09-10T09:04:15.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f1623fff640 1 -- 172.21.15.172:0/3247578236 learned_addr learned my addr 172.21.15.172:0/3247578236 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:15.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f162affd640 1 --2- 172.21.15.172:0/3247578236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 0x558fa37ed4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f1623fff640 1 --2- 172.21.15.172:0/3247578236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37eda30 0x558fa37efe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f162b7fe640 1 --2- 172.21.15.172:0/3514993852 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16180a2c00 0x7f16180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f1623fff640 1 -- 172.21.15.172:0/3247578236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37eda30 msgr2=0x558fa37efe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:15.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f1623fff640 1 --2- 172.21.15.172:0/3247578236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37eda30 0x558fa37efe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f1623fff640 1 -- 172.21.15.172:0/3247578236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 msgr2=0x558fa37ed4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:15.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f1623fff640 1 --2- 172.21.15.172:0/3247578236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 0x558fa37ed4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f1623fff640 1 -- 172.21.15.172:0/3247578236 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fa34b8ba0 con 0x558fa37e2850 2024-09-10T09:04:15.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f1628ff9640 1 -- 172.21.15.172:0/3514993852 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f16240cf4e0 con 0x7f162c106f10 2024-09-10T09:04:15.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f1623fff640 1 --2- 172.21.15.172:0/3247578236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 0x558fa37e2c30 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f16140135b0 tx=0x7f161400b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6acad4f0a9d4496 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:15.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f162b7fe640 1 --2- 172.21.15.172:0/3514993852 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16180a2c00 0x7f16180a50c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1624097e40 tx=0x7f162406d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:15.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f16217fa640 1 -- 172.21.15.172:0/3247578236 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614004030 con 0x558fa37e2850 2024-09-10T09:04:15.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f16217fa640 1 -- 172.21.15.172:0/3247578236 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16140041d0 con 0x558fa37e2850 2024-09-10T09:04:15.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.220+0000 7f16217fa640 1 -- 172.21.15.172:0/3247578236 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614011760 con 0x558fa37e2850 2024-09-10T09:04:15.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.221+0000 7f16339cf740 1 -- 172.21.15.172:0/3247578236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 msgr2=0x558fa37e2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.221+0000 7f16339cf740 1 --2- 172.21.15.172:0/3247578236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 0x558fa37e2c30 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f16140135b0 tx=0x7f161400b760 comp rx=0 tx=0).stop 2024-09-10T09:04:15.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.221+0000 7f16339cf740 1 -- 172.21.15.172:0/3247578236 shutdown_connections 2024-09-10T09:04:15.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.221+0000 7f16339cf740 1 --2- 172.21.15.172:0/3247578236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37eda30 0x558fa37efe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.221+0000 7f16339cf740 1 --2- 172.21.15.172:0/3247578236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 0x558fa37ed4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.221+0000 7f16339cf740 1 --2- 172.21.15.172:0/3247578236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 0x558fa37e2c30 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.715 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.221+0000 7f16339cf740 1 -- 172.21.15.172:0/3247578236 >> 172.21.15.172:0/3247578236 conn(0x558fa37dede0 msgr2=0x558fa37df1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:15.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.221+0000 7f16339cf740 1 -- 172.21.15.172:0/3247578236 shutdown_connections 2024-09-10T09:04:15.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.221+0000 7f16339cf740 1 -- 172.21.15.172:0/3247578236 wait complete. 2024-09-10T09:04:15.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.221+0000 7f16339cf740 1 -- start start 2024-09-10T09:04:15.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.221+0000 7f16339cf740 1 Processor -- start 2024-09-10T09:04:15.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f16339cf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 0x558fa37ea970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f16339cf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 0x558fa37eaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f16339cf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37eda30 0x558fa37eb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f16339cf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558fa37f0600 con 0x558fa37e2850 2024-09-10T09:04:15.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f16339cf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558fa36cf900 con 0x558fa37e3170 2024-09-10T09:04:15.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f16339cf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558fa37f0780 con 0x558fa37eda30 2024-09-10T09:04:15.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f16339cf740 10 client.0 _ll_register_callbacks cb 0x558fa36cd780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:15.716 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f1623fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 0x558fa37ea970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f1623fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 0x558fa37ea970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45324/0 (socket says 172.21.15.172:45324) 2024-09-10T09:04:15.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f162affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 0x558fa37eaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f1623fff640 1 -- 172.21.15.172:0/475983473 learned_addr learned my addr 172.21.15.172:0/475983473 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:15.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f1623fff640 1 --2- 172.21.15.172:0/475983473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37eda30 0x558fa37eb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f162affd640 1 -- 172.21.15.172:0/475983473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37eda30 msgr2=0x558fa37eb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:15.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f162affd640 1 --2- 172.21.15.172:0/475983473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37eda30 0x558fa37eb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f162affd640 1 -- 172.21.15.172:0/475983473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 msgr2=0x558fa37ea970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:15.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f162affd640 1 --2- 172.21.15.172:0/475983473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 0x558fa37ea970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f162affd640 1 -- 172.21.15.172:0/475983473 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fa370fc50 con 0x558fa37e3170 2024-09-10T09:04:15.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f1623fff640 1 --2- 172.21.15.172:0/475983473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37eda30 0x558fa37eb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:04:15.717 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f162affd640 1 --2- 172.21.15.172:0/475983473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 0x558fa37eaeb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f161c009230 tx=0x7f161c032f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea31d1e330d196fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:15.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.222+0000 7f16127fc640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:04:15.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.223+0000 7f16127fc640 1 -- 172.21.15.172:0/475983473 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c035040 con 0x558fa37e3170 2024-09-10T09:04:15.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.223+0000 7f16339cf740 1 -- 172.21.15.172:0/475983473 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16140041d0 con 0x558fa37e3170 2024-09-10T09:04:15.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.223+0000 7f16127fc640 1 -- 172.21.15.172:0/475983473 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f161c031860 con 0x558fa37e3170 2024-09-10T09:04:15.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.223+0000 7f16127fc640 1 -- 172.21.15.172:0/475983473 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f161c04f050 con 0x558fa37e3170 2024-09-10T09:04:15.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.223+0000 7f16339cf740 10 client.28559 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:15.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.223+0000 7f16339cf740 1 -- 172.21.15.172:0/475983473 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558fa37fc350 con 0x558fa37e3170 2024-09-10T09:04:15.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.223+0000 7f16127fc640 1 -- 172.21.15.172:0/475983473 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16140041d0 con 0x558fa37e3170 2024-09-10T09:04:15.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.223+0000 7f16127fc640 1 -- 172.21.15.172:0/475983473 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f161c004750 con 0x558fa37e3170 2024-09-10T09:04:15.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.223+0000 7f16339cf740 10 client.28559 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:15.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.223+0000 7f16339cf740 10 client.28559 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:04:15.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.223+0000 7f16339cf740 20 client.28559 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:15.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.223+0000 7f16339cf740 1 --2- 172.21.15.172:0/475983473 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558fa37fd760 0x558fa37ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.223+0000 7f16339cf740 4 client.28559 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:04:15.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.223+0000 7f16339cf740 1 -- 172.21.15.172:0/475983473 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fa3754890 con 0x558fa37fd760 2024-09-10T09:04:15.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.223+0000 7f1623fff640 1 --2- 172.21.15.172:0/475983473 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558fa37fd760 0x558fa37ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.224+0000 7f16127fc640 10 client.28559 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:15.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.224+0000 7f1623fff640 1 --2- 172.21.15.172:0/475983473 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558fa37fd760 0x558fa37ffb50 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2fe7f4718b57807 server_cookie=237773f1e0dcbffc in_seq=0 out_seq=0 2024-09-10T09:04:15.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.224+0000 7f16127fc640 1 -- 172.21.15.172:0/475983473 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558fa3754890 con 0x558fa37fd760 2024-09-10T09:04:15.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.224+0000 7f16127fc640 10 client.28559 handle_command_reply: tid=0 2024-09-10T09:04:15.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.224+0000 7f16339cf740 1 client.28559 shutdown 2024-09-10T09:04:15.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.224+0000 7f16339cf740 1 -- 172.21.15.172:0/475983473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 msgr2=0x558fa37eaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.224+0000 7f16339cf740 1 --2- 172.21.15.172:0/475983473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 0x558fa37eaeb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f161c009230 tx=0x7f161c032f90 comp rx=0 tx=0).stop 2024-09-10T09:04:15.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.224+0000 7f16339cf740 1 -- 172.21.15.172:0/475983473 shutdown_connections 2024-09-10T09:04:15.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.224+0000 7f16339cf740 1 --2- 172.21.15.172:0/475983473 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558fa37fd760 0x558fa37ffb50 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.224+0000 7f16339cf740 1 --2- 172.21.15.172:0/475983473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37eda30 0x558fa37eb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.224+0000 7f16127fc640 0 client.28559 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:15.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.224+0000 7f16339cf740 1 --2- 172.21.15.172:0/475983473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 0x558fa37eaeb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.224+0000 7f16339cf740 1 --2- 172.21.15.172:0/475983473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 0x558fa37ea970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.225+0000 7f16339cf740 1 -- 172.21.15.172:0/475983473 >> 172.21.15.172:0/475983473 conn(0x558fa37dede0 msgr2=0x558fa37e4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:15.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.225+0000 7f16339cf740 1 -- 172.21.15.172:0/475983473 shutdown_connections 2024-09-10T09:04:15.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.225+0000 7f16339cf740 1 -- 172.21.15.172:0/475983473 wait complete. 2024-09-10T09:04:15.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.225+0000 7f16339cf740 20 client.28559 trim_cache size 0 max 16384 2024-09-10T09:04:15.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.241+0000 7f16339cf740 1 Processor -- start 2024-09-10T09:04:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.241+0000 7f16339cf740 1 -- start start 2024-09-10T09:04:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f16339cf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 0x558fa37e52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f16339cf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 0x558fa38053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f16339cf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37eda30 0x558fa3805920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f16339cf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f162c1ac840 con 0x558fa37e2850 2024-09-10T09:04:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f16339cf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f162c1ae380 con 0x558fa37e3170 2024-09-10T09:04:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f16339cf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f162c1acf10 con 0x558fa37eda30 2024-09-10T09:04:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f1623fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 0x558fa37e52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f1623fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 0x558fa37e52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45338/0 (socket says 172.21.15.172:45338) 2024-09-10T09:04:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f1623fff640 1 -- 172.21.15.172:0/1974874674 learned_addr learned my addr 172.21.15.172:0/1974874674 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f1623fff640 1 -- 172.21.15.172:0/1974874674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37eda30 msgr2=0x558fa3805920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f1623fff640 1 --2- 172.21.15.172:0/1974874674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37eda30 0x558fa3805920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f1623fff640 1 -- 172.21.15.172:0/1974874674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 msgr2=0x558fa38053e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:15.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f1623fff640 1 --2- 172.21.15.172:0/1974874674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 0x558fa38053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f1623fff640 1 -- 172.21.15.172:0/1974874674 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fa37e2fa0 con 0x558fa37e2850 2024-09-10T09:04:15.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f1623fff640 1 --2- 172.21.15.172:0/1974874674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 0x558fa37e52b0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f161400c010 tx=0x7f1614005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77bf0d0f61158d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:15.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f1620ff9640 1 -- 172.21.15.172:0/1974874674 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161400b2c0 con 0x558fa37e2850 2024-09-10T09:04:15.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f1620ff9640 1 -- 172.21.15.172:0/1974874674 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16140044b0 con 0x558fa37e2850 2024-09-10T09:04:15.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.242+0000 7f1620ff9640 1 -- 172.21.15.172:0/1974874674 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614005300 con 0x558fa37e2850 2024-09-10T09:04:15.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.243+0000 7f16339cf740 1 -- 172.21.15.172:0/1974874674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 msgr2=0x558fa37e52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.243+0000 7f16339cf740 1 --2- 172.21.15.172:0/1974874674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 0x558fa37e52b0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f161400c010 tx=0x7f1614005fb0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.243+0000 7f16339cf740 1 -- 172.21.15.172:0/1974874674 shutdown_connections 2024-09-10T09:04:15.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.243+0000 7f16339cf740 1 --2- 172.21.15.172:0/1974874674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37eda30 0x558fa3805920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.243+0000 7f16339cf740 1 --2- 172.21.15.172:0/1974874674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 0x558fa38053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.243+0000 7f16339cf740 1 --2- 172.21.15.172:0/1974874674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37e2850 0x558fa37e52b0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.243+0000 7f16339cf740 1 -- 172.21.15.172:0/1974874674 >> 172.21.15.172:0/1974874674 conn(0x558fa37dede0 msgr2=0x558fa37ec9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.243+0000 7f16339cf740 1 -- 172.21.15.172:0/1974874674 shutdown_connections 2024-09-10T09:04:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.243+0000 7f16339cf740 1 -- 172.21.15.172:0/1974874674 wait complete. 2024-09-10T09:04:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f16339cf740 1 -- start start 2024-09-10T09:04:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f16339cf740 1 Processor -- start 2024-09-10T09:04:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f16339cf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37e2850 0x558fa37f1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f16339cf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 0x558fa37f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f16339cf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37eda30 0x558fa37f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f16339cf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f162c1adcb0 con 0x558fa37eda30 2024-09-10T09:04:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f16339cf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f162c1ad5e0 con 0x558fa37e3170 2024-09-10T09:04:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f16339cf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f162c1aa820 con 0x558fa37e2850 2024-09-10T09:04:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f16339cf740 10 client.0 _ll_register_callbacks cb 0x7f161c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:15.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f1623fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37eda30 0x558fa37f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f1623fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37eda30 0x558fa37f22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45340/0 (socket says 172.21.15.172:45340) 2024-09-10T09:04:15.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f1623fff640 1 -- 172.21.15.172:0/1640906654 learned_addr learned my addr 172.21.15.172:0/1640906654 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:15.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f1623fff640 1 --2- 172.21.15.172:0/1640906654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37e2850 0x558fa37f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f162affd640 1 --2- 172.21.15.172:0/1640906654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 0x558fa37f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f1623fff640 1 -- 172.21.15.172:0/1640906654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37e2850 msgr2=0x558fa37f1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:15.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f1623fff640 1 --2- 172.21.15.172:0/1640906654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37e2850 0x558fa37f1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f1623fff640 1 -- 172.21.15.172:0/1640906654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 msgr2=0x558fa37f1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:15.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.244+0000 7f1623fff640 1 --2- 172.21.15.172:0/1640906654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 0x558fa37f1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f1623fff640 1 -- 172.21.15.172:0/1640906654 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fa38674b0 con 0x558fa37eda30 2024-09-10T09:04:15.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f162affd640 1 --2- 172.21.15.172:0/1640906654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 0x558fa37f1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:04:15.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f1623fff640 1 --2- 172.21.15.172:0/1640906654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37eda30 0x558fa37f22a0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f1614031040 tx=0x7f161400bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d14788021965f31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:15.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f16127fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:15.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f16127fc640 1 -- 172.21.15.172:0/1640906654 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614013030 con 0x558fa37eda30 2024-09-10T09:04:15.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f16127fc640 1 -- 172.21.15.172:0/1640906654 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f16140131d0 con 0x558fa37eda30 2024-09-10T09:04:15.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f16339cf740 1 -- 172.21.15.172:0/1640906654 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16140044b0 con 0x558fa37eda30 2024-09-10T09:04:15.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f16127fc640 1 -- 172.21.15.172:0/1640906654 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1614005260 con 0x558fa37eda30 2024-09-10T09:04:15.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f16339cf740 10 client.19758 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:15.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f16339cf740 1 -- 172.21.15.172:0/1640906654 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558fa3868770 con 0x558fa37eda30 2024-09-10T09:04:15.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f16127fc640 1 -- 172.21.15.172:0/1640906654 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16140044b0 con 0x558fa37eda30 2024-09-10T09:04:15.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f16127fc640 1 -- 172.21.15.172:0/1640906654 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f1614037040 con 0x558fa37eda30 2024-09-10T09:04:15.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f16339cf740 10 client.19758 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:15.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f16339cf740 10 client.19758 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:04:15.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f16339cf740 20 client.19758 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:15.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f16339cf740 1 --2- 172.21.15.172:0/1640906654 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558fa37fd760 0x558fa386acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:15.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f16339cf740 4 client.19758 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}] 2024-09-10T09:04:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.245+0000 7f16339cf740 1 -- 172.21.15.172:0/1640906654 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}) -- 0x558fa374ebe0 con 0x558fa37fd760 2024-09-10T09:04:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.246+0000 7f162affd640 1 --2- 172.21.15.172:0/1640906654 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558fa37fd760 0x558fa386acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.246+0000 7f162affd640 1 --2- 172.21.15.172:0/1640906654 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558fa37fd760 0x558fa386acf0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab6c10f8d9ef1001 server_cookie=7f4e322d5049947b in_seq=0 out_seq=0 2024-09-10T09:04:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.246+0000 7f16127fc640 10 client.19758 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.394+0000 7f16127fc640 1 -- 172.21.15.172:0/1640906654 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558fa374ebe0 con 0x558fa37fd760 2024-09-10T09:04:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.394+0000 7f16127fc640 10 client.19758 handle_command_reply: tid=0 2024-09-10T09:04:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.394+0000 7f16339cf740 1 client.19758 shutdown 2024-09-10T09:04:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.394+0000 7f16339cf740 1 -- 172.21.15.172:0/1640906654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37eda30 msgr2=0x558fa37f22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.394+0000 7f16339cf740 1 --2- 172.21.15.172:0/1640906654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37eda30 0x558fa37f22a0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f1614031040 tx=0x7f161400bdf0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16339cf740 1 -- 172.21.15.172:0/1640906654 shutdown_connections 2024-09-10T09:04:15.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16339cf740 1 --2- 172.21.15.172:0/1640906654 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x558fa37fd760 0x558fa386acf0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.727 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:04:15.727 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 44, 2024-09-10T09:04:15.727 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T09:04:15.727 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 54, 2024-09-10T09:04:15.727 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T09:04:15.727 INFO:teuthology.orchestra.run.smithi172.stdout: "d3ffbc76": { 2024-09-10T09:04:15.727 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 54, 2024-09-10T09:04:15.727 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T09:04:15.727 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T09:04:15.728 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T09:04:15.728 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T09:04:15.728 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:04:15.728 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T09:04:15.728 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 139.8, 2024-09-10T09:04:15.728 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T09:04:15.728 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T09:04:15.728 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T09:04:15.728 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T09:04:15.728 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T09:04:15.728 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T09:04:15.728 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:04:15.728 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:04:15.729 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:04:15.729 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:04:15.729 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:04:15.729 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:04:15.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16339cf740 1 --2- 172.21.15.172:0/1640906654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558fa37eda30 0x558fa37f22a0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16339cf740 1 --2- 172.21.15.172:0/1640906654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa37e3170 0x558fa37f1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16127fc640 0 client.19758 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:15.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16339cf740 1 --2- 172.21.15.172:0/1640906654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558fa37e2850 0x558fa37f1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16339cf740 1 -- 172.21.15.172:0/1640906654 >> 172.21.15.172:0/1640906654 conn(0x558fa37dede0 msgr2=0x558fa37dd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:15.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16339cf740 1 -- 172.21.15.172:0/1640906654 shutdown_connections 2024-09-10T09:04:15.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16339cf740 1 -- 172.21.15.172:0/1640906654 wait complete. 2024-09-10T09:04:15.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16339cf740 20 client.19758 trim_cache size 0 max 16384 2024-09-10T09:04:15.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16315f6640 1 -- 172.21.15.172:0/3514993852 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16180a2c00 msgr2=0x7f16180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16315f6640 1 --2- 172.21.15.172:0/3514993852 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16180a2c00 0x7f16180a50c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1624097e40 tx=0x7f162406d040 comp rx=0 tx=0).stop 2024-09-10T09:04:15.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16315f6640 1 -- 172.21.15.172:0/3514993852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f162c106f10 msgr2=0x7f162c19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:15.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16315f6640 1 --2- 172.21.15.172:0/3514993852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f162c106f10 0x7f162c19cc00 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f162400eaa0 tx=0x7f162400ef60 comp rx=0 tx=0).stop 2024-09-10T09:04:15.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16315f6640 1 -- 172.21.15.172:0/3514993852 shutdown_connections 2024-09-10T09:04:15.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16315f6640 1 --2- 172.21.15.172:0/3514993852 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f16180a2c00 0x7f16180a50c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16315f6640 1 --2- 172.21.15.172:0/3514993852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f162c106f10 0x7f162c19cc00 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16315f6640 1 --2- 172.21.15.172:0/3514993852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f162c102790 0x7f162c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16315f6640 1 --2- 172.21.15.172:0/3514993852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f162c101de0 0x7f162c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:15.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.395+0000 7f16315f6640 1 -- 172.21.15.172:0/3514993852 >> 172.21.15.172:0/3514993852 conn(0x7f162c0fd8c0 msgr2=0x7f162c0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:15.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.396+0000 7f16315f6640 1 -- 172.21.15.172:0/3514993852 shutdown_connections 2024-09-10T09:04:15.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:14.396+0000 7f16315f6640 1 -- 172.21.15.172:0/3514993852 wait complete. 2024-09-10T09:04:17.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:16 smithi174 ceph-mon[34068]: pgmap v891: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 2.23k op/s 2024-09-10T09:04:17.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:16 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T09:04:17.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:16 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T09:04:17.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:16 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T09:04:17.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:16 smithi191 ceph-mon[35314]: pgmap v891: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 2.23k op/s 2024-09-10T09:04:17.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:16 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T09:04:17.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:16 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T09:04:17.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:16 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T09:04:17.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:16 smithi172 ceph-mon[30854]: pgmap v891: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 2.23k op/s 2024-09-10T09:04:17.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:16 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T09:04:17.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:16 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T09:04:17.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:16 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T09:04:18.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:17 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T09:04:18.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:17 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T09:04:18.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:17 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T09:04:18.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:17 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T09:04:18.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:17 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T09:04:18.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:17 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T09:04:19.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:18 smithi174 ceph-mon[34068]: pgmap v892: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.3 MiB/s wr, 1.87k op/s 2024-09-10T09:04:19.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:18 smithi191 ceph-mon[35314]: pgmap v892: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.3 MiB/s wr, 1.87k op/s 2024-09-10T09:04:19.312 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:18 smithi172 ceph-mon[30854]: pgmap v892: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.3 MiB/s wr, 1.87k op/s 2024-09-10T09:04:21.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:20 smithi174 ceph-mon[34068]: pgmap v893: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.5 MiB/s wr, 1.16k op/s 2024-09-10T09:04:21.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:20 smithi191 ceph-mon[35314]: pgmap v893: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.5 MiB/s wr, 1.16k op/s 2024-09-10T09:04:21.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:20 smithi172 ceph-mon[30854]: pgmap v893: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.5 MiB/s wr, 1.16k op/s 2024-09-10T09:04:23.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:22 smithi174 ceph-mon[34068]: pgmap v894: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.4 MiB/s wr, 1.14k op/s 2024-09-10T09:04:23.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:22 smithi191 ceph-mon[35314]: pgmap v894: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.4 MiB/s wr, 1.14k op/s 2024-09-10T09:04:23.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:22 smithi172 ceph-mon[30854]: pgmap v894: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.4 MiB/s wr, 1.14k op/s 2024-09-10T09:04:25.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:24 smithi174 ceph-mon[34068]: pgmap v895: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.0 MiB/s wr, 535 op/s 2024-09-10T09:04:25.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:24 smithi191 ceph-mon[35314]: pgmap v895: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.0 MiB/s wr, 535 op/s 2024-09-10T09:04:25.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:24 smithi172 ceph-mon[30854]: pgmap v895: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.0 MiB/s wr, 535 op/s 2024-09-10T09:04:25.916 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd3ffbc76', quiesce took 60.6 seconds. Will release after: -40.75 2024-09-10T09:04:25.917 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:04:26.177 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:04:27.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:26 smithi174 ceph-mon[34068]: pgmap v896: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 896 KiB/s wr, 240 op/s 2024-09-10T09:04:27.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:26 smithi191 ceph-mon[35314]: pgmap v896: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 896 KiB/s wr, 240 op/s 2024-09-10T09:04:27.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:26 smithi172 ceph-mon[30854]: pgmap v896: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 896 KiB/s wr, 240 op/s 2024-09-10T09:04:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.785+0000 7f405c027640 1 -- 172.21.15.172:0/3756174552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4054101c50 msgr2=0x7f40541020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.785+0000 7f405c027640 1 --2- 172.21.15.172:0/3756174552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4054101c50 0x7f40541020d0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f4048009900 tx=0x7f404802a410 comp rx=0 tx=0).stop 2024-09-10T09:04:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.785+0000 7f405c027640 1 -- 172.21.15.172:0/3756174552 shutdown_connections 2024-09-10T09:04:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.785+0000 7f405c027640 1 --2- 172.21.15.172:0/3756174552 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4054102610 0x7f4054108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.785+0000 7f405c027640 1 --2- 172.21.15.172:0/3756174552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4054101c50 0x7f40541020d0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.785+0000 7f405c027640 1 --2- 172.21.15.172:0/3756174552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4054100a60 0x7f4054100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.785+0000 7f405c027640 1 -- 172.21.15.172:0/3756174552 >> 172.21.15.172:0/3756174552 conn(0x7f40540fc1e0 msgr2=0x7f40540fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.785+0000 7f405c027640 1 -- 172.21.15.172:0/3756174552 shutdown_connections 2024-09-10T09:04:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.785+0000 7f405c027640 1 -- 172.21.15.172:0/3756174552 wait complete. 2024-09-10T09:04:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f405c027640 1 Processor -- start 2024-09-10T09:04:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f405c027640 1 -- start start 2024-09-10T09:04:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f405c027640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4054100a60 0x7f4054197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f405c027640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4054101c50 0x7f4054198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f405c027640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4054102610 0x7f405419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f405c027640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f405410c590 con 0x7f4054100a60 2024-09-10T09:04:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f405c027640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f405410c410 con 0x7f4054102610 2024-09-10T09:04:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f405c027640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f405410c710 con 0x7f4054101c50 2024-09-10T09:04:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f4059dc3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4054100a60 0x7f4054197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f4059dc3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4054100a60 0x7f4054197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56008/0 (socket says 172.21.15.172:56008) 2024-09-10T09:04:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f4059dc3640 1 -- 172.21.15.172:0/3355063750 learned_addr learned my addr 172.21.15.172:0/3355063750 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f405a5c4640 1 --2- 172.21.15.172:0/3355063750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4054102610 0x7f405419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f40595c2640 1 --2- 172.21.15.172:0/3355063750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4054101c50 0x7f4054198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f4059dc3640 1 -- 172.21.15.172:0/3355063750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4054101c50 msgr2=0x7f4054198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f4059dc3640 1 --2- 172.21.15.172:0/3355063750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4054101c50 0x7f4054198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f4059dc3640 1 -- 172.21.15.172:0/3355063750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4054102610 msgr2=0x7f405419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f4059dc3640 1 --2- 172.21.15.172:0/3355063750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4054102610 0x7f405419f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f4059dc3640 1 -- 172.21.15.172:0/3355063750 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405419f960 con 0x7f4054100a60 2024-09-10T09:04:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.786+0000 7f4059dc3640 1 --2- 172.21.15.172:0/3355063750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4054100a60 0x7f4054197d50 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f404c00d950 tx=0x7f404c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.787+0000 7f4052ffd640 1 -- 172.21.15.172:0/3355063750 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c017020 con 0x7f4054100a60 2024-09-10T09:04:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.787+0000 7f4052ffd640 1 -- 172.21.15.172:0/3355063750 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f404c009c30 con 0x7f4054100a60 2024-09-10T09:04:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.787+0000 7f405c027640 1 -- 172.21.15.172:0/3355063750 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f405419fc50 con 0x7f4054100a60 2024-09-10T09:04:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.787+0000 7f4052ffd640 1 -- 172.21.15.172:0/3355063750 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c00f6b0 con 0x7f4054100a60 2024-09-10T09:04:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.787+0000 7f405c027640 1 -- 172.21.15.172:0/3355063750 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40541a01a0 con 0x7f4054100a60 2024-09-10T09:04:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.789+0000 7f4052ffd640 1 -- 172.21.15.172:0/3355063750 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f404c01a020 con 0x7f4054100a60 2024-09-10T09:04:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.789+0000 7f405edc1740 1 Processor -- start 2024-09-10T09:04:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.789+0000 7f405edc1740 1 -- start start 2024-09-10T09:04:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f405edc1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c6769a0 0x55f96c676da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f405edc1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c67bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f405edc1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c67c370 0x55f96c67e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f405edc1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f96c5da2f0 con 0x55f96c6772e0 2024-09-10T09:04:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f405edc1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f96c582270 con 0x55f96c67c370 2024-09-10T09:04:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f405edc1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f96c5d62b0 con 0x55f96c6769a0 2024-09-10T09:04:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f4059dc3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c67bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f4059dc3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c67bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56012/0 (socket says 172.21.15.172:56012) 2024-09-10T09:04:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f4059dc3640 1 -- 172.21.15.172:0/3633338025 learned_addr learned my addr 172.21.15.172:0/3633338025 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f405a5c4640 1 --2- 172.21.15.172:0/3633338025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c6769a0 0x55f96c676da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f40595c2640 1 --2- 172.21.15.172:0/3633338025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c67c370 0x55f96c67e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f4052ffd640 1 --2- 172.21.15.172:0/3355063750 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f403c0a2c30 0x7f403c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f4059dc3640 1 -- 172.21.15.172:0/3633338025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c6769a0 msgr2=0x55f96c676da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f4059dc3640 1 --2- 172.21.15.172:0/3633338025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c6769a0 0x55f96c676da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f4059dc3640 1 -- 172.21.15.172:0/3633338025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c67c370 msgr2=0x55f96c67e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f4059dc3640 1 --2- 172.21.15.172:0/3633338025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c67c370 0x55f96c67e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f4059dc3640 1 -- 172.21.15.172:0/3633338025 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f96c581340 con 0x55f96c6772e0 2024-09-10T09:04:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f405a5c4640 1 --2- 172.21.15.172:0/3355063750 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f403c0a2c30 0x7f403c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f4052ffd640 1 -- 172.21.15.172:0/3355063750 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f404c0cf490 con 0x7f4054100a60 2024-09-10T09:04:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.790+0000 7f4059dc3640 1 --2- 172.21.15.172:0/3633338025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c67bda0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f404c063ec0 tx=0x7f404c066470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19dda60566072a46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.791+0000 7f403bfff640 1 -- 172.21.15.172:0/3633338025 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c09b470 con 0x55f96c6772e0 2024-09-10T09:04:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.791+0000 7f403bfff640 1 -- 172.21.15.172:0/3633338025 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f404c06ee60 con 0x55f96c6772e0 2024-09-10T09:04:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.791+0000 7f403bfff640 1 -- 172.21.15.172:0/3633338025 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c06e9d0 con 0x55f96c6772e0 2024-09-10T09:04:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.791+0000 7f405edc1740 1 -- 172.21.15.172:0/3633338025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 msgr2=0x55f96c67bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.791+0000 7f405edc1740 1 --2- 172.21.15.172:0/3633338025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c67bda0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f404c063ec0 tx=0x7f404c066470 comp rx=0 tx=0).stop 2024-09-10T09:04:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.791+0000 7f405a5c4640 1 --2- 172.21.15.172:0/3355063750 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f403c0a2c30 0x7f403c0a50f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4044005fd0 tx=0x7f4044004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.791+0000 7f405edc1740 1 -- 172.21.15.172:0/3633338025 shutdown_connections 2024-09-10T09:04:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.791+0000 7f405edc1740 1 --2- 172.21.15.172:0/3633338025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c67c370 0x55f96c67e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.791+0000 7f405edc1740 1 --2- 172.21.15.172:0/3633338025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c67bda0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.791+0000 7f405edc1740 1 --2- 172.21.15.172:0/3633338025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c6769a0 0x55f96c676da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.791+0000 7f405edc1740 1 -- 172.21.15.172:0/3633338025 >> 172.21.15.172:0/3633338025 conn(0x55f96c672010 msgr2=0x55f96c672410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.791+0000 7f405edc1740 1 -- 172.21.15.172:0/3633338025 shutdown_connections 2024-09-10T09:04:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.791+0000 7f405edc1740 1 -- 172.21.15.172:0/3633338025 wait complete. 2024-09-10T09:04:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f405edc1740 1 -- start start 2024-09-10T09:04:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f405edc1740 1 Processor -- start 2024-09-10T09:04:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f405edc1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c6769a0 0x55f96c680b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f405edc1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c681070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f405edc1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c67c370 0x55f96c688040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f405edc1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f96c67f2e0 con 0x55f96c6772e0 2024-09-10T09:04:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f405edc1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f96c563290 con 0x55f96c6769a0 2024-09-10T09:04:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f405edc1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f96c67f460 con 0x55f96c67c370 2024-09-10T09:04:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f40595c2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c681070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f405edc1740 10 client.0 _ll_register_callbacks cb 0x55f96c594140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f40595c2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c681070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56024/0 (socket says 172.21.15.172:56024) 2024-09-10T09:04:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f40595c2640 1 -- 172.21.15.172:0/3676346304 learned_addr learned my addr 172.21.15.172:0/3676346304 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f405a5c4640 1 --2- 172.21.15.172:0/3676346304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c67c370 0x55f96c688040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f4059dc3640 1 --2- 172.21.15.172:0/3676346304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c6769a0 0x55f96c680b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f40595c2640 1 -- 172.21.15.172:0/3676346304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c67c370 msgr2=0x55f96c688040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f40595c2640 1 --2- 172.21.15.172:0/3676346304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c67c370 0x55f96c688040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f40595c2640 1 -- 172.21.15.172:0/3676346304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c6769a0 msgr2=0x55f96c680b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.792+0000 7f40595c2640 1 --2- 172.21.15.172:0/3676346304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c6769a0 0x55f96c680b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.793+0000 7f40595c2640 1 -- 172.21.15.172:0/3676346304 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f96c2ed210 con 0x55f96c6772e0 2024-09-10T09:04:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.793+0000 7f40595c2640 1 --2- 172.21.15.172:0/3676346304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c681070 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f404802a930 tx=0x7f40480317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca30653335c9b9a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.793+0000 7f40397fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.793+0000 7f40397fa640 1 -- 172.21.15.172:0/3676346304 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048035040 con 0x55f96c6772e0 2024-09-10T09:04:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.793+0000 7f40397fa640 1 -- 172.21.15.172:0/3676346304 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4048004240 con 0x55f96c6772e0 2024-09-10T09:04:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.793+0000 7f405edc1740 1 -- 172.21.15.172:0/3676346304 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f96c5e7a20 con 0x55f96c6772e0 2024-09-10T09:04:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.793+0000 7f40397fa640 1 -- 172.21.15.172:0/3676346304 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f404804e050 con 0x55f96c6772e0 2024-09-10T09:04:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.793+0000 7f405edc1740 10 client.19788 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.793+0000 7f405edc1740 1 -- 172.21.15.172:0/3676346304 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f96c3b9620 con 0x55f96c6772e0 2024-09-10T09:04:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.793+0000 7f40397fa640 1 -- 172.21.15.172:0/3676346304 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55f96c5e7a20 con 0x55f96c6772e0 2024-09-10T09:04:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.793+0000 7f40397fa640 1 -- 172.21.15.172:0/3676346304 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f4048004b00 con 0x55f96c6772e0 2024-09-10T09:04:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.793+0000 7f405edc1740 10 client.19788 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.793+0000 7f405edc1740 10 client.19788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:04:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.793+0000 7f405edc1740 20 client.19788 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.793+0000 7f405edc1740 1 --2- 172.21.15.172:0/3676346304 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f96c68c6e0 0x55f96c68eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.793+0000 7f405edc1740 4 client.19788 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:04:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.794+0000 7f405edc1740 1 -- 172.21.15.172:0/3676346304 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f96c5e1d70 con 0x55f96c68c6e0 2024-09-10T09:04:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.794+0000 7f4059dc3640 1 --2- 172.21.15.172:0/3676346304 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f96c68c6e0 0x55f96c68eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.794+0000 7f4059dc3640 1 --2- 172.21.15.172:0/3676346304 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f96c68c6e0 0x55f96c68eaf0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c12a65278807ca8c server_cookie=c6b859b74735f039 in_seq=0 out_seq=0 2024-09-10T09:04:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.794+0000 7f40397fa640 10 client.19788 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.795+0000 7f40397fa640 1 -- 172.21.15.172:0/3676346304 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f96c5e1d70 con 0x55f96c68c6e0 2024-09-10T09:04:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.795+0000 7f40397fa640 10 client.19788 handle_command_reply: tid=0 2024-09-10T09:04:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.795+0000 7f405edc1740 1 client.19788 shutdown 2024-09-10T09:04:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.795+0000 7f405edc1740 1 -- 172.21.15.172:0/3676346304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 msgr2=0x55f96c681070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.795+0000 7f405edc1740 1 --2- 172.21.15.172:0/3676346304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c681070 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f404802a930 tx=0x7f40480317b0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.795+0000 7f405edc1740 1 -- 172.21.15.172:0/3676346304 shutdown_connections 2024-09-10T09:04:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.795+0000 7f405edc1740 1 --2- 172.21.15.172:0/3676346304 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f96c68c6e0 0x55f96c68eaf0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.795+0000 7f405edc1740 1 --2- 172.21.15.172:0/3676346304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c67c370 0x55f96c688040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.795+0000 7f405edc1740 1 --2- 172.21.15.172:0/3676346304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c681070 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.795+0000 7f405edc1740 1 --2- 172.21.15.172:0/3676346304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c6769a0 0x55f96c680b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.795+0000 7f40397fa640 0 client.19788 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.795+0000 7f405edc1740 1 -- 172.21.15.172:0/3676346304 >> 172.21.15.172:0/3676346304 conn(0x55f96c672010 msgr2=0x55f96c673820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.796+0000 7f405edc1740 1 -- 172.21.15.172:0/3676346304 shutdown_connections 2024-09-10T09:04:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.796+0000 7f405edc1740 1 -- 172.21.15.172:0/3676346304 wait complete. 2024-09-10T09:04:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.796+0000 7f405edc1740 20 client.19788 trim_cache size 0 max 16384 2024-09-10T09:04:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.810+0000 7f405edc1740 1 Processor -- start 2024-09-10T09:04:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.810+0000 7f405edc1740 1 -- start start 2024-09-10T09:04:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.810+0000 7f405edc1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c6769a0 0x55f96c68ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.810+0000 7f405edc1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c6772e0 0x55f96c688000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.810+0000 7f405edc1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c67c370 0x55f96c6710b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.810+0000 7f405edc1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f40541aa5f0 con 0x55f96c67c370 2024-09-10T09:04:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.810+0000 7f405edc1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f40541ac130 con 0x55f96c6772e0 2024-09-10T09:04:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.810+0000 7f405edc1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f40541aacc0 con 0x55f96c6769a0 2024-09-10T09:04:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.810+0000 7f405a5c4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c67c370 0x55f96c6710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.810+0000 7f405a5c4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c67c370 0x55f96c6710b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56028/0 (socket says 172.21.15.172:56028) 2024-09-10T09:04:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.810+0000 7f405a5c4640 1 -- 172.21.15.172:0/2577752912 learned_addr learned my addr 172.21.15.172:0/2577752912 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.810+0000 7f40595c2640 1 --2- 172.21.15.172:0/2577752912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c6772e0 0x55f96c688000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.811+0000 7f4059dc3640 1 --2- 172.21.15.172:0/2577752912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c6769a0 0x55f96c68ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.811+0000 7f405a5c4640 1 -- 172.21.15.172:0/2577752912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c6769a0 msgr2=0x55f96c68ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.811+0000 7f405a5c4640 1 --2- 172.21.15.172:0/2577752912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c6769a0 0x55f96c68ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.811+0000 7f405a5c4640 1 -- 172.21.15.172:0/2577752912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c6772e0 msgr2=0x55f96c688000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.811+0000 7f405a5c4640 1 --2- 172.21.15.172:0/2577752912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c6772e0 0x55f96c688000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.811+0000 7f405a5c4640 1 -- 172.21.15.172:0/2577752912 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f96c67e760 con 0x55f96c67c370 2024-09-10T09:04:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.811+0000 7f40595c2640 1 --2- 172.21.15.172:0/2577752912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c6772e0 0x55f96c688000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:04:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.811+0000 7f405a5c4640 1 --2- 172.21.15.172:0/2577752912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c67c370 0x55f96c6710b0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f404401f8d0 tx=0x7f4044014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc66a70abd2a5cbf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.811+0000 7f403b7fe640 1 -- 172.21.15.172:0/2577752912 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404401f250 con 0x55f96c67c370 2024-09-10T09:04:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.811+0000 7f403b7fe640 1 -- 172.21.15.172:0/2577752912 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f404401fe50 con 0x55f96c67c370 2024-09-10T09:04:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.811+0000 7f403b7fe640 1 -- 172.21.15.172:0/2577752912 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404401eda0 con 0x55f96c67c370 2024-09-10T09:04:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 1 -- 172.21.15.172:0/2577752912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c67c370 msgr2=0x55f96c6710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 1 --2- 172.21.15.172:0/2577752912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c67c370 0x55f96c6710b0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f404401f8d0 tx=0x7f4044014000 comp rx=0 tx=0).stop 2024-09-10T09:04:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 1 -- 172.21.15.172:0/2577752912 shutdown_connections 2024-09-10T09:04:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 1 --2- 172.21.15.172:0/2577752912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c67c370 0x55f96c6710b0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 1 --2- 172.21.15.172:0/2577752912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c6772e0 0x55f96c688000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 1 --2- 172.21.15.172:0/2577752912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c6769a0 0x55f96c68ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 1 -- 172.21.15.172:0/2577752912 >> 172.21.15.172:0/2577752912 conn(0x55f96c672010 msgr2=0x55f96c6709a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 1 -- 172.21.15.172:0/2577752912 shutdown_connections 2024-09-10T09:04:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 1 -- 172.21.15.172:0/2577752912 wait complete. 2024-09-10T09:04:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 1 -- start start 2024-09-10T09:04:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 1 Processor -- start 2024-09-10T09:04:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c6769a0 0x55f96c685580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c685ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c67c370 0x55f96c686ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f40541aba60 con 0x55f96c6772e0 2024-09-10T09:04:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f40541ab390 con 0x55f96c6769a0 2024-09-10T09:04:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f40541a85d0 con 0x55f96c67c370 2024-09-10T09:04:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f405edc1740 10 client.0 _ll_register_callbacks cb 0x7f4048009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f40595c2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c685ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f40595c2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c685ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56040/0 (socket says 172.21.15.172:56040) 2024-09-10T09:04:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.812+0000 7f40595c2640 1 -- 172.21.15.172:0/214178186 learned_addr learned my addr 172.21.15.172:0/214178186 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f405a5c4640 1 --2- 172.21.15.172:0/214178186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c67c370 0x55f96c686ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f40595c2640 1 -- 172.21.15.172:0/214178186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c67c370 msgr2=0x55f96c686ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f4059dc3640 1 --2- 172.21.15.172:0/214178186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c6769a0 0x55f96c685580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f40595c2640 1 --2- 172.21.15.172:0/214178186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c67c370 0x55f96c686ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f40595c2640 1 -- 172.21.15.172:0/214178186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c6769a0 msgr2=0x55f96c685580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f40595c2640 1 --2- 172.21.15.172:0/214178186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c6769a0 0x55f96c685580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f40595c2640 1 -- 172.21.15.172:0/214178186 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f96c6870e0 con 0x55f96c6772e0 2024-09-10T09:04:28.819 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f405a5c4640 1 --2- 172.21.15.172:0/214178186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c67c370 0x55f96c686ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:04:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f4059dc3640 1 --2- 172.21.15.172:0/214178186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c6769a0 0x55f96c685580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:04:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f40595c2640 1 --2- 172.21.15.172:0/214178186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c685ac0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f404802a450 tx=0x7f40480052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7962db8aad2501b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f40397fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f40397fa640 1 -- 172.21.15.172:0/214178186 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048035070 con 0x55f96c6772e0 2024-09-10T09:04:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f40397fa640 1 -- 172.21.15.172:0/214178186 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f40480046c0 con 0x55f96c6772e0 2024-09-10T09:04:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f405edc1740 1 -- 172.21.15.172:0/214178186 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f404401fe50 con 0x55f96c6772e0 2024-09-10T09:04:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f40397fa640 1 -- 172.21.15.172:0/214178186 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4048060050 con 0x55f96c6772e0 2024-09-10T09:04:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f405edc1740 10 client.19794 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f405edc1740 1 -- 172.21.15.172:0/214178186 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f96c6883f0 con 0x55f96c6772e0 2024-09-10T09:04:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.813+0000 7f40397fa640 1 -- 172.21.15.172:0/214178186 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f404401fe50 con 0x55f96c6772e0 2024-09-10T09:04:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.814+0000 7f40397fa640 1 -- 172.21.15.172:0/214178186 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f4048045700 con 0x55f96c6772e0 2024-09-10T09:04:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.814+0000 7f405edc1740 10 client.19794 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:28.820 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.814+0000 7f405edc1740 10 client.19794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:04:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.814+0000 7f405edc1740 20 client.19794 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.814+0000 7f405edc1740 1 --2- 172.21.15.172:0/214178186 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f96c6f7b90 0x55f96c6f9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.814+0000 7f405edc1740 4 client.19794 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:04:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.814+0000 7f405edc1740 1 -- 172.21.15.172:0/214178186 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f96c55e900 con 0x55f96c6f7b90 2024-09-10T09:04:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.814+0000 7f4059dc3640 1 --2- 172.21.15.172:0/214178186 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f96c6f7b90 0x55f96c6f9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.814+0000 7f40397fa640 10 client.19794 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.814+0000 7f4059dc3640 1 --2- 172.21.15.172:0/214178186 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f96c6f7b90 0x55f96c6f9fb0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdae240990caaab6 server_cookie=b89e1496c23e0336 in_seq=0 out_seq=0 2024-09-10T09:04:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.821+0000 7f40397fa640 1 -- 172.21.15.172:0/214178186 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f96c55e900 con 0x55f96c6f7b90 2024-09-10T09:04:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.821+0000 7f40397fa640 10 client.19794 handle_command_reply: tid=0 2024-09-10T09:04:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.821+0000 7f405edc1740 1 client.19794 shutdown 2024-09-10T09:04:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.821+0000 7f405edc1740 1 -- 172.21.15.172:0/214178186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 msgr2=0x55f96c685ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.821+0000 7f405edc1740 1 --2- 172.21.15.172:0/214178186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c685ac0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f404802a450 tx=0x7f40480052b0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.821+0000 7f405edc1740 1 -- 172.21.15.172:0/214178186 shutdown_connections 2024-09-10T09:04:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.821+0000 7f405edc1740 1 --2- 172.21.15.172:0/214178186 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f96c6f7b90 0x55f96c6f9fb0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.821+0000 7f405edc1740 1 --2- 172.21.15.172:0/214178186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f96c67c370 0x55f96c686ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.821+0000 7f405edc1740 1 --2- 172.21.15.172:0/214178186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f96c6772e0 0x55f96c685ac0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.821+0000 7f40397fa640 0 client.19794 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.821+0000 7f405edc1740 1 --2- 172.21.15.172:0/214178186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f96c6769a0 0x55f96c685580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.821+0000 7f405edc1740 1 -- 172.21.15.172:0/214178186 >> 172.21.15.172:0/214178186 conn(0x55f96c672010 msgr2=0x55f96c672f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.821+0000 7f405edc1740 1 -- 172.21.15.172:0/214178186 shutdown_connections 2024-09-10T09:04:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.821+0000 7f405edc1740 1 -- 172.21.15.172:0/214178186 wait complete. 2024-09-10T09:04:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.822+0000 7f405edc1740 20 client.19794 trim_cache size 0 max 16384 2024-09-10T09:04:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:04:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:04:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:04:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:04:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.822+0000 7f405c027640 1 -- 172.21.15.172:0/3355063750 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f403c0a2c30 msgr2=0x7f403c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.822+0000 7f405c027640 1 --2- 172.21.15.172:0/3355063750 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f403c0a2c30 0x7f403c0a50f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4044005fd0 tx=0x7f4044004a40 comp rx=0 tx=0).stop 2024-09-10T09:04:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.822+0000 7f405c027640 1 -- 172.21.15.172:0/3355063750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4054100a60 msgr2=0x7f4054197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.822+0000 7f405c027640 1 --2- 172.21.15.172:0/3355063750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4054100a60 0x7f4054197d50 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f404c00d950 tx=0x7f404c00de10 comp rx=0 tx=0).stop 2024-09-10T09:04:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.822+0000 7f405c027640 1 -- 172.21.15.172:0/3355063750 shutdown_connections 2024-09-10T09:04:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.822+0000 7f405c027640 1 --2- 172.21.15.172:0/3355063750 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f403c0a2c30 0x7f403c0a50f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.822+0000 7f405c027640 1 --2- 172.21.15.172:0/3355063750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4054102610 0x7f405419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.825 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.822+0000 7f405c027640 1 --2- 172.21.15.172:0/3355063750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4054101c50 0x7f4054198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.822+0000 7f405c027640 1 --2- 172.21.15.172:0/3355063750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4054100a60 0x7f4054197d50 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.822+0000 7f405c027640 1 -- 172.21.15.172:0/3355063750 >> 172.21.15.172:0/3355063750 conn(0x7f40540fc1e0 msgr2=0x7f405406a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.822+0000 7f405c027640 1 -- 172.21.15.172:0/3355063750 shutdown_connections 2024-09-10T09:04:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:28.822+0000 7f405c027640 1 -- 172.21.15.172:0/3355063750 wait complete. 2024-09-10T09:04:29.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:28 smithi174 ceph-mon[34068]: pgmap v897: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 854 KiB/s wr, 228 op/s 2024-09-10T09:04:29.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:28 smithi191 ceph-mon[35314]: pgmap v897: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 854 KiB/s wr, 228 op/s 2024-09-10T09:04:29.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:28 smithi172 ceph-mon[30854]: pgmap v897: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 854 KiB/s wr, 228 op/s 2024-09-10T09:04:31.174 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:30 smithi172 ceph-mon[30854]: pgmap v898: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-10T09:04:31.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:30 smithi174 ceph-mon[34068]: pgmap v898: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-10T09:04:31.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:30 smithi191 ceph-mon[35314]: pgmap v898: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-10T09:04:32.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:31 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/4221710124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:04:32.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:31 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/4221710124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:04:32.305 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.961+0000 7feb1b848640 1 -- 172.21.15.172:0/1453868386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb14103780 msgr2=0x7feb14107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:32.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.961+0000 7feb1b848640 1 --2- 172.21.15.172:0/1453868386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb14103780 0x7feb14107bc0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7feb08009930 tx=0x7feb0802a420 comp rx=0 tx=0).stop 2024-09-10T09:04:32.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.961+0000 7feb1b848640 1 -- 172.21.15.172:0/1453868386 shutdown_connections 2024-09-10T09:04:32.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.961+0000 7feb1b848640 1 --2- 172.21.15.172:0/1453868386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb14108300 0x7feb1410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:32.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.961+0000 7feb1b848640 1 --2- 172.21.15.172:0/1453868386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb14103780 0x7feb14107bc0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:32.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.961+0000 7feb1b848640 1 --2- 172.21.15.172:0/1453868386 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb14102dd0 0x7feb141031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:32.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.961+0000 7feb1b848640 1 -- 172.21.15.172:0/1453868386 >> 172.21.15.172:0/1453868386 conn(0x7feb140fc7d0 msgr2=0x7feb140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:32.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.961+0000 7feb1b848640 1 -- 172.21.15.172:0/1453868386 shutdown_connections 2024-09-10T09:04:32.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.961+0000 7feb1b848640 1 -- 172.21.15.172:0/1453868386 wait complete. 2024-09-10T09:04:32.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.962+0000 7feb1b848640 1 Processor -- start 2024-09-10T09:04:32.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.962+0000 7feb1b848640 1 -- start start 2024-09-10T09:04:32.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.962+0000 7feb1b848640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb14102dd0 0x7feb14100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:32.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.962+0000 7feb1b848640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb14103780 0x7feb14100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:32.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.962+0000 7feb1b848640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb14108300 0x7feb141014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:32.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.962+0000 7feb1b848640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feb14111f00 con 0x7feb14102dd0 2024-09-10T09:04:32.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.962+0000 7feb1b848640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb14111d80 con 0x7feb14108300 2024-09-10T09:04:32.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.962+0000 7feb1b848640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7feb14112080 con 0x7feb14103780 2024-09-10T09:04:32.307 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T09:04:32.307 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":44,"btime":"2024-09-10T09:01:18:531063+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T09:01:17.697372+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T09:04:32.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.962+0000 7feb195e4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb14102dd0 0x7feb14100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:32.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.962+0000 7feb195e4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb14102dd0 0x7feb14100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56056/0 (socket says 172.21.15.172:56056) 2024-09-10T09:04:32.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.962+0000 7feb195e4640 1 -- 172.21.15.172:0/4221710124 learned_addr learned my addr 172.21.15.172:0/4221710124 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:32.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.963+0000 7feb195e4640 1 -- 172.21.15.172:0/4221710124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb14103780 msgr2=0x7feb14100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:32.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.963+0000 7feb195e4640 1 --2- 172.21.15.172:0/4221710124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb14103780 0x7feb14100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:32.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.963+0000 7feb195e4640 1 -- 172.21.15.172:0/4221710124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb14108300 msgr2=0x7feb141014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:32.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.963+0000 7feb195e4640 1 --2- 172.21.15.172:0/4221710124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb14108300 0x7feb141014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:32.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.963+0000 7feb195e4640 1 -- 172.21.15.172:0/4221710124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb141a69c0 con 0x7feb14102dd0 2024-09-10T09:04:32.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.963+0000 7feb195e4640 1 --2- 172.21.15.172:0/4221710124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb14102dd0 0x7feb14100a30 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7feb1000e9f0 tx=0x7feb1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:32.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.963+0000 7feb067fc640 1 -- 172.21.15.172:0/4221710124 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1000cd80 con 0x7feb14102dd0 2024-09-10T09:04:32.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.963+0000 7feb067fc640 1 -- 172.21.15.172:0/4221710124 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb100108e0 con 0x7feb14102dd0 2024-09-10T09:04:32.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.963+0000 7feb067fc640 1 -- 172.21.15.172:0/4221710124 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10018920 con 0x7feb14102dd0 2024-09-10T09:04:32.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.963+0000 7feb1b848640 1 -- 172.21.15.172:0/4221710124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb141a6cb0 con 0x7feb14102dd0 2024-09-10T09:04:32.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.963+0000 7feb1b848640 1 -- 172.21.15.172:0/4221710124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb141a71a0 con 0x7feb14102dd0 2024-09-10T09:04:32.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.964+0000 7feb1b848640 1 -- 172.21.15.172:0/4221710124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb141031b0 con 0x7feb14102dd0 2024-09-10T09:04:32.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.965+0000 7feb067fc640 1 -- 172.21.15.172:0/4221710124 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7feb10010a90 con 0x7feb14102dd0 2024-09-10T09:04:32.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.966+0000 7feb067fc640 1 --2- 172.21.15.172:0/4221710124 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feb000a2b80 0x7feb000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:32.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.966+0000 7feb18de3640 1 --2- 172.21.15.172:0/4221710124 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feb000a2b80 0x7feb000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:32.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.966+0000 7feb067fc640 1 -- 172.21.15.172:0/4221710124 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7feb100cfd00 con 0x7feb14102dd0 2024-09-10T09:04:32.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.967+0000 7feb18de3640 1 --2- 172.21.15.172:0/4221710124 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feb000a2b80 0x7feb000a5040 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7feb080096c0 tx=0x7feb08031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:32.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:30.969+0000 7feb067fc640 1 -- 172.21.15.172:0/4221710124 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feb1009c900 con 0x7feb14102dd0 2024-09-10T09:04:32.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:31.173+0000 7feb1b848640 1 -- 172.21.15.172:0/4221710124 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb1410a290 con 0x7feb14102dd0 2024-09-10T09:04:32.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:31.174+0000 7feb067fc640 1 -- 172.21.15.172:0/4221710124 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+12001 (secure 0 0 0) 0x7feb1009c200 con 0x7feb14102dd0 2024-09-10T09:04:32.310 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 44 2024-09-10T09:04:32.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:31.176+0000 7feb1b848640 1 -- 172.21.15.172:0/4221710124 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feb000a2b80 msgr2=0x7feb000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:32.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:31.176+0000 7feb1b848640 1 --2- 172.21.15.172:0/4221710124 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feb000a2b80 0x7feb000a5040 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7feb080096c0 tx=0x7feb08031400 comp rx=0 tx=0).stop 2024-09-10T09:04:32.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:31.176+0000 7feb1b848640 1 -- 172.21.15.172:0/4221710124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb14102dd0 msgr2=0x7feb14100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:32.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:31.176+0000 7feb1b848640 1 --2- 172.21.15.172:0/4221710124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb14102dd0 0x7feb14100a30 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7feb1000e9f0 tx=0x7feb1000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:04:32.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:31.176+0000 7feb1b848640 1 -- 172.21.15.172:0/4221710124 shutdown_connections 2024-09-10T09:04:32.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:31.176+0000 7feb1b848640 1 --2- 172.21.15.172:0/4221710124 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7feb000a2b80 0x7feb000a5040 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:32.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:31.176+0000 7feb1b848640 1 --2- 172.21.15.172:0/4221710124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb14108300 0x7feb141014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:32.310 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:31.176+0000 7feb1b848640 1 --2- 172.21.15.172:0/4221710124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb14103780 0x7feb14100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:32.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:31.176+0000 7feb1b848640 1 --2- 172.21.15.172:0/4221710124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb14102dd0 0x7feb14100a30 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:32.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:31.176+0000 7feb1b848640 1 -- 172.21.15.172:0/4221710124 >> 172.21.15.172:0/4221710124 conn(0x7feb140fc7d0 msgr2=0x7feb14105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:32.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:31.176+0000 7feb1b848640 1 -- 172.21.15.172:0/4221710124 shutdown_connections 2024-09-10T09:04:32.311 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:31.176+0000 7feb1b848640 1 -- 172.21.15.172:0/4221710124 wait complete. 2024-09-10T09:04:32.311 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:31 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/4221710124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:04:33.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:32 smithi174 ceph-mon[34068]: pgmap v899: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-09-10T09:04:33.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:32 smithi191 ceph-mon[35314]: pgmap v899: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-09-10T09:04:33.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:32 smithi172 ceph-mon[30854]: pgmap v899: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-09-10T09:04:35.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:35 smithi191 ceph-mon[35314]: pgmap v900: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-09-10T09:04:35.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:35 smithi172 ceph-mon[30854]: pgmap v900: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-09-10T09:04:35.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:35 smithi174 ceph-mon[34068]: pgmap v900: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-09-10T09:04:37.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:37 smithi191 ceph-mon[35314]: pgmap v901: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-10T09:04:37.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:37 smithi172 ceph-mon[30854]: pgmap v901: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-10T09:04:37.976 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:04:37.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:37 smithi174 ceph-mon[34068]: pgmap v901: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-10T09:04:38.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:38 smithi174 ceph-mon[34068]: pgmap v902: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-10T09:04:39.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:38 smithi191 ceph-mon[35314]: pgmap v902: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-10T09:04:39.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:38 smithi172 ceph-mon[30854]: pgmap v902: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-10T09:04:40.589 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id d3ffbc76 --release --await' 2024-09-10T09:04:40.590 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id d3ffbc76 --release --await 2024-09-10T09:04:40.602 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:04:40.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:40 smithi172 ceph-mon[30854]: pgmap v903: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-10T09:04:40.853 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:40 smithi191 ceph-mon[35314]: pgmap v903: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-10T09:04:40.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:40 smithi174 ceph-mon[34068]: pgmap v903: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-10T09:04:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.920+0000 7f50bb6ba640 1 -- 172.21.15.172:0/939045046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b4100a60 msgr2=0x7f50b4100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.920+0000 7f50bb6ba640 1 --2- 172.21.15.172:0/939045046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b4100a60 0x7f50b4100e60 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009900 tx=0x7f50b002a410 comp rx=0 tx=0).stop 2024-09-10T09:04:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.920+0000 7f50bb6ba640 1 -- 172.21.15.172:0/939045046 shutdown_connections 2024-09-10T09:04:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.920+0000 7f50bb6ba640 1 --2- 172.21.15.172:0/939045046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50b4102610 0x7f50b4108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.920+0000 7f50bb6ba640 1 --2- 172.21.15.172:0/939045046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50b4101c50 0x7f50b41020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.920+0000 7f50bb6ba640 1 --2- 172.21.15.172:0/939045046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b4100a60 0x7f50b4100e60 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.920+0000 7f50bb6ba640 1 -- 172.21.15.172:0/939045046 >> 172.21.15.172:0/939045046 conn(0x7f50b40fc1e0 msgr2=0x7f50b40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.920+0000 7f50bb6ba640 1 -- 172.21.15.172:0/939045046 shutdown_connections 2024-09-10T09:04:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.920+0000 7f50bb6ba640 1 -- 172.21.15.172:0/939045046 wait complete. 2024-09-10T09:04:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50bb6ba640 1 Processor -- start 2024-09-10T09:04:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50bb6ba640 1 -- start start 2024-09-10T09:04:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50bb6ba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50b4100a60 0x7f50b406ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50bb6ba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50b4101c50 0x7f50b419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50bb6ba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b4102610 0x7f50b419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50bb6ba640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f50b410c570 con 0x7f50b4102610 2024-09-10T09:04:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50bb6ba640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50b410c3f0 con 0x7f50b4101c50 2024-09-10T09:04:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50bb6ba640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f50b410c6f0 con 0x7f50b4100a60 2024-09-10T09:04:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50b9c57640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b4102610 0x7f50b419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50b9c57640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b4102610 0x7f50b419f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56518/0 (socket says 172.21.15.172:56518) 2024-09-10T09:04:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50b9c57640 1 -- 172.21.15.172:0/4257228656 learned_addr learned my addr 172.21.15.172:0/4257228656 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50b9456640 1 --2- 172.21.15.172:0/4257228656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50b4100a60 0x7f50b406ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50b8c55640 1 --2- 172.21.15.172:0/4257228656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50b4101c50 0x7f50b419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50b9c57640 1 -- 172.21.15.172:0/4257228656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50b4100a60 msgr2=0x7f50b406ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50b9c57640 1 --2- 172.21.15.172:0/4257228656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50b4100a60 0x7f50b406ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50b9c57640 1 -- 172.21.15.172:0/4257228656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50b4101c50 msgr2=0x7f50b419ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50b9c57640 1 --2- 172.21.15.172:0/4257228656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50b4101c50 0x7f50b419ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50b9c57640 1 -- 172.21.15.172:0/4257228656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b419f940 con 0x7f50b4102610 2024-09-10T09:04:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.921+0000 7f50b8c55640 1 --2- 172.21.15.172:0/4257228656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50b4101c50 0x7f50b419ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:04:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.922+0000 7f50b9c57640 1 --2- 172.21.15.172:0/4257228656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b4102610 0x7f50b419f030 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00cce0 tx=0x7f50ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.922+0000 7f50aa7fc640 1 -- 172.21.15.172:0/4257228656 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac00f450 con 0x7f50b4102610 2024-09-10T09:04:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.922+0000 7f50aa7fc640 1 -- 172.21.15.172:0/4257228656 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50ac00faa0 con 0x7f50b4102610 2024-09-10T09:04:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.922+0000 7f50bb6ba640 1 -- 172.21.15.172:0/4257228656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b419fbd0 con 0x7f50b4102610 2024-09-10T09:04:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.922+0000 7f50aa7fc640 1 -- 172.21.15.172:0/4257228656 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac018b70 con 0x7f50b4102610 2024-09-10T09:04:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.922+0000 7f50bb6ba640 1 -- 172.21.15.172:0/4257228656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b41a0120 con 0x7f50b4102610 2024-09-10T09:04:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.924+0000 7f50aa7fc640 1 -- 172.21.15.172:0/4257228656 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f50ac018d10 con 0x7f50b4102610 2024-09-10T09:04:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.924+0000 7f50be456740 1 Processor -- start 2024-09-10T09:04:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.924+0000 7f50be456740 1 -- start start 2024-09-10T09:04:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.924+0000 7f50be456740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6739a0 0x562ead673da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50be456740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead6742e0 0x562ead678da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50be456740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead679370 0x562ead67b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50be456740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562ead5d72f0 con 0x562ead6739a0 2024-09-10T09:04:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50be456740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562ead57f270 con 0x562ead679370 2024-09-10T09:04:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50be456740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562ead5d32b0 con 0x562ead6742e0 2024-09-10T09:04:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50b8c55640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6739a0 0x562ead673da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50b8c55640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6739a0 0x562ead673da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56526/0 (socket says 172.21.15.172:56526) 2024-09-10T09:04:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50b8c55640 1 -- 172.21.15.172:0/3877047671 learned_addr learned my addr 172.21.15.172:0/3877047671 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50aa7fc640 1 --2- 172.21.15.172:0/4257228656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f50a00a2cb0 0x7f50a00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50b9456640 1 --2- 172.21.15.172:0/3877047671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead6742e0 0x562ead678da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50b8c55640 1 --2- 172.21.15.172:0/3877047671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead679370 0x562ead67b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50b8c55640 1 -- 172.21.15.172:0/3877047671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead6742e0 msgr2=0x562ead678da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50b8c55640 1 --2- 172.21.15.172:0/3877047671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead6742e0 0x562ead678da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50b8c55640 1 -- 172.21.15.172:0/3877047671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead679370 msgr2=0x562ead67b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50b8c55640 1 --2- 172.21.15.172:0/3877047671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead679370 0x562ead67b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50b8c55640 1 -- 172.21.15.172:0/3877047671 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ead57e340 con 0x562ead6739a0 2024-09-10T09:04:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50b9c57640 1 --2- 172.21.15.172:0/4257228656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f50a00a2cb0 0x7f50a00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50b8c55640 1 --2- 172.21.15.172:0/3877047671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6739a0 0x562ead673da0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f50a4013660 tx=0x7f50a400b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81aeb6f7e44e6c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f50aa7fc640 1 -- 172.21.15.172:0/4257228656 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f50ac0cf6f0 con 0x7f50b4102610 2024-09-10T09:04:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f509f7fe640 1 -- 172.21.15.172:0/3877047671 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4019030 con 0x562ead6739a0 2024-09-10T09:04:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f509f7fe640 1 -- 172.21.15.172:0/3877047671 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50a40130d0 con 0x562ead6739a0 2024-09-10T09:04:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.925+0000 7f509f7fe640 1 -- 172.21.15.172:0/3877047671 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a40047a0 con 0x562ead6739a0 2024-09-10T09:04:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.926+0000 7f50be456740 1 -- 172.21.15.172:0/3877047671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6739a0 msgr2=0x562ead673da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.926+0000 7f50be456740 1 --2- 172.21.15.172:0/3877047671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6739a0 0x562ead673da0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f50a4013660 tx=0x7f50a400b9c0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.926+0000 7f50b9c57640 1 --2- 172.21.15.172:0/4257228656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f50a00a2cb0 0x7f50a00a5170 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f50ac09b8f0 tx=0x7f50ac06ab70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.926+0000 7f50be456740 1 -- 172.21.15.172:0/3877047671 shutdown_connections 2024-09-10T09:04:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.926+0000 7f50be456740 1 --2- 172.21.15.172:0/3877047671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead679370 0x562ead67b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.926+0000 7f50be456740 1 --2- 172.21.15.172:0/3877047671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead6742e0 0x562ead678da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.926+0000 7f50be456740 1 --2- 172.21.15.172:0/3877047671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6739a0 0x562ead673da0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.926+0000 7f50be456740 1 -- 172.21.15.172:0/3877047671 >> 172.21.15.172:0/3877047671 conn(0x562ead66f010 msgr2=0x562ead66f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.926+0000 7f50be456740 1 -- 172.21.15.172:0/3877047671 shutdown_connections 2024-09-10T09:04:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.926+0000 7f50be456740 1 -- 172.21.15.172:0/3877047671 wait complete. 2024-09-10T09:04:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.926+0000 7f50be456740 1 -- start start 2024-09-10T09:04:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50be456740 1 Processor -- start 2024-09-10T09:04:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50be456740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead6739a0 0x562ead67dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50be456740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6742e0 0x562ead67e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50be456740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead679370 0x562ead685110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50be456740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562ead67c2e0 con 0x562ead6742e0 2024-09-10T09:04:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50be456740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562ead560290 con 0x562ead679370 2024-09-10T09:04:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50be456740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562ead67c460 con 0x562ead6739a0 2024-09-10T09:04:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50be456740 10 client.0 _ll_register_callbacks cb 0x562ead591140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50b9456640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6742e0 0x562ead67e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50b9456640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6742e0 0x562ead67e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56532/0 (socket says 172.21.15.172:56532) 2024-09-10T09:04:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50b9456640 1 -- 172.21.15.172:0/2887597178 learned_addr learned my addr 172.21.15.172:0/2887597178 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50b8c55640 1 --2- 172.21.15.172:0/2887597178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead6739a0 0x562ead67dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50b8c55640 1 --2- 172.21.15.172:0/2887597178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead679370 0x562ead685110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50b9456640 1 -- 172.21.15.172:0/2887597178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead6739a0 msgr2=0x562ead67dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50b9456640 1 --2- 172.21.15.172:0/2887597178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead6739a0 0x562ead67dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50b9456640 1 -- 172.21.15.172:0/2887597178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead679370 msgr2=0x562ead685110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50b9456640 1 --2- 172.21.15.172:0/2887597178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead679370 0x562ead685110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50b9456640 1 -- 172.21.15.172:0/2887597178 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ead2ea210 con 0x562ead6742e0 2024-09-10T09:04:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f50b9456640 1 --2- 172.21.15.172:0/2887597178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6742e0 0x562ead67e140 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009900 tx=0x7f50b00317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d8526900b54124 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.927+0000 7f509cff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.928+0000 7f509cff9640 1 -- 172.21.15.172:0/2887597178 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0035070 con 0x562ead6742e0 2024-09-10T09:04:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.928+0000 7f509cff9640 1 -- 172.21.15.172:0/2887597178 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f50b0004250 con 0x562ead6742e0 2024-09-10T09:04:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.928+0000 7f50be456740 1 -- 172.21.15.172:0/2887597178 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50a40130d0 con 0x562ead6742e0 2024-09-10T09:04:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.928+0000 7f509cff9640 1 -- 172.21.15.172:0/2887597178 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f50b004d050 con 0x562ead6742e0 2024-09-10T09:04:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.928+0000 7f50be456740 10 client.19812 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.928+0000 7f50be456740 1 -- 172.21.15.172:0/2887597178 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562ead3b6620 con 0x562ead6742e0 2024-09-10T09:04:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.928+0000 7f509cff9640 1 -- 172.21.15.172:0/2887597178 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50a40130d0 con 0x562ead6742e0 2024-09-10T09:04:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.928+0000 7f509cff9640 1 -- 172.21.15.172:0/2887597178 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f50b003b4c0 con 0x562ead6742e0 2024-09-10T09:04:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.928+0000 7f50be456740 10 client.19812 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.928+0000 7f50be456740 10 client.19812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:04:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.928+0000 7f50be456740 20 client.19812 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.928+0000 7f50be456740 1 --2- 172.21.15.172:0/2887597178 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562ead689870 0x562ead68bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.928+0000 7f50be456740 4 client.19812 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:04:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.928+0000 7f50be456740 1 -- 172.21.15.172:0/2887597178 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ead5e4a20 con 0x562ead689870 2024-09-10T09:04:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.929+0000 7f50b8c55640 1 --2- 172.21.15.172:0/2887597178 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562ead689870 0x562ead68bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.929+0000 7f50b8c55640 1 --2- 172.21.15.172:0/2887597178 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562ead689870 0x562ead68bcb0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fb00e95a05c100d server_cookie=9a24162285c47720 in_seq=0 out_seq=0 2024-09-10T09:04:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.929+0000 7f509cff9640 10 client.19812 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.930+0000 7f509cff9640 1 -- 172.21.15.172:0/2887597178 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ead5e4a20 con 0x562ead689870 2024-09-10T09:04:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.930+0000 7f509cff9640 10 client.19812 handle_command_reply: tid=0 2024-09-10T09:04:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.930+0000 7f50be456740 1 client.19812 shutdown 2024-09-10T09:04:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.930+0000 7f50be456740 1 -- 172.21.15.172:0/2887597178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6742e0 msgr2=0x562ead67e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.930+0000 7f50be456740 1 --2- 172.21.15.172:0/2887597178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6742e0 0x562ead67e140 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009900 tx=0x7f50b00317b0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.930+0000 7f50be456740 1 -- 172.21.15.172:0/2887597178 shutdown_connections 2024-09-10T09:04:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.930+0000 7f50be456740 1 --2- 172.21.15.172:0/2887597178 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562ead689870 0x562ead68bcb0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.930+0000 7f50be456740 1 --2- 172.21.15.172:0/2887597178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead679370 0x562ead685110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.930+0000 7f50be456740 1 --2- 172.21.15.172:0/2887597178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6742e0 0x562ead67e140 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.930+0000 7f509cff9640 0 client.19812 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.930+0000 7f50be456740 1 --2- 172.21.15.172:0/2887597178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead6739a0 0x562ead67dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.930+0000 7f50be456740 1 -- 172.21.15.172:0/2887597178 >> 172.21.15.172:0/2887597178 conn(0x562ead66f010 msgr2=0x562ead670820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.930+0000 7f50be456740 1 -- 172.21.15.172:0/2887597178 shutdown_connections 2024-09-10T09:04:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.930+0000 7f50be456740 1 -- 172.21.15.172:0/2887597178 wait complete. 2024-09-10T09:04:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.930+0000 7f50be456740 20 client.19812 trim_cache size 0 max 16384 2024-09-10T09:04:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.944+0000 7f50be456740 1 Processor -- start 2024-09-10T09:04:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.944+0000 7f50be456740 1 -- start start 2024-09-10T09:04:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.944+0000 7f50be456740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead6739a0 0x562ead66e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.944+0000 7f50be456740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6742e0 0x562ead6850d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.944+0000 7f50be456740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead679370 0x562ead6905b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.944+0000 7f50be456740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f50b41aa490 con 0x562ead6742e0 2024-09-10T09:04:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.944+0000 7f50be456740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50b41abfd0 con 0x562ead6739a0 2024-09-10T09:04:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.944+0000 7f50be456740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f50b41aab60 con 0x562ead679370 2024-09-10T09:04:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.945+0000 7f50b9456640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6742e0 0x562ead6850d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.945+0000 7f50b9456640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6742e0 0x562ead6850d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56548/0 (socket says 172.21.15.172:56548) 2024-09-10T09:04:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.945+0000 7f50b9456640 1 -- 172.21.15.172:0/3737695652 learned_addr learned my addr 172.21.15.172:0/3737695652 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.945+0000 7f50b8c55640 1 --2- 172.21.15.172:0/3737695652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead679370 0x562ead6905b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.945+0000 7f50b8c55640 1 --2- 172.21.15.172:0/3737695652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead6739a0 0x562ead66e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.945+0000 7f50b9456640 1 -- 172.21.15.172:0/3737695652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead679370 msgr2=0x562ead6905b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.945+0000 7f50b9456640 1 --2- 172.21.15.172:0/3737695652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead679370 0x562ead6905b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.945+0000 7f50b9456640 1 -- 172.21.15.172:0/3737695652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead6739a0 msgr2=0x562ead66e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.945+0000 7f50b9456640 1 --2- 172.21.15.172:0/3737695652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead6739a0 0x562ead66e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.945+0000 7f50b9456640 1 -- 172.21.15.172:0/3737695652 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ead67b760 con 0x562ead6742e0 2024-09-10T09:04:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.945+0000 7f50b9456640 1 --2- 172.21.15.172:0/3737695652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6742e0 0x562ead6850d0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f50b003dfd0 tx=0x7f50b002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a9de5499e340aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.945+0000 7f509effd640 1 -- 172.21.15.172:0/3737695652 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0035070 con 0x562ead6742e0 2024-09-10T09:04:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.945+0000 7f509effd640 1 -- 172.21.15.172:0/3737695652 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50b003bad0 con 0x562ead6742e0 2024-09-10T09:04:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.945+0000 7f509effd640 1 -- 172.21.15.172:0/3737695652 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b00044b0 con 0x562ead6742e0 2024-09-10T09:04:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 1 -- 172.21.15.172:0/3737695652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6742e0 msgr2=0x562ead6850d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 1 --2- 172.21.15.172:0/3737695652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6742e0 0x562ead6850d0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f50b003dfd0 tx=0x7f50b002a410 comp rx=0 tx=0).stop 2024-09-10T09:04:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 1 -- 172.21.15.172:0/3737695652 shutdown_connections 2024-09-10T09:04:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 1 --2- 172.21.15.172:0/3737695652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead679370 0x562ead6905b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 1 --2- 172.21.15.172:0/3737695652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead6742e0 0x562ead6850d0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 1 --2- 172.21.15.172:0/3737695652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead6739a0 0x562ead66e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 1 -- 172.21.15.172:0/3737695652 >> 172.21.15.172:0/3737695652 conn(0x562ead66f010 msgr2=0x562ead66d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 1 -- 172.21.15.172:0/3737695652 shutdown_connections 2024-09-10T09:04:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 1 -- 172.21.15.172:0/3737695652 wait complete. 2024-09-10T09:04:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 1 -- start start 2024-09-10T09:04:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 1 Processor -- start 2024-09-10T09:04:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead6739a0 0x562ead684b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead6742e0 0x562ead6850a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead679370 0x562ead6831c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f50b41ab900 con 0x562ead679370 2024-09-10T09:04:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50b41ab230 con 0x562ead6742e0 2024-09-10T09:04:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f50b41a8470 con 0x562ead6739a0 2024-09-10T09:04:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50be456740 10 client.0 _ll_register_callbacks cb 0x7f50b0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50b8c55640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead679370 0x562ead6831c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50b8c55640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead679370 0x562ead6831c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56554/0 (socket says 172.21.15.172:56554) 2024-09-10T09:04:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50b8c55640 1 -- 172.21.15.172:0/2425837959 learned_addr learned my addr 172.21.15.172:0/2425837959 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.946+0000 7f50b8c55640 1 --2- 172.21.15.172:0/2425837959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead6739a0 0x562ead684b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.947+0000 7f50b9456640 1 --2- 172.21.15.172:0/2425837959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead6742e0 0x562ead6850a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.947+0000 7f50b8c55640 1 -- 172.21.15.172:0/2425837959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead6739a0 msgr2=0x562ead684b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.947+0000 7f50b8c55640 1 --2- 172.21.15.172:0/2425837959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead6739a0 0x562ead684b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.947+0000 7f50b8c55640 1 -- 172.21.15.172:0/2425837959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead6742e0 msgr2=0x562ead6850a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.947+0000 7f50b8c55640 1 --2- 172.21.15.172:0/2425837959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead6742e0 0x562ead6850a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.947+0000 7f50b8c55640 1 -- 172.21.15.172:0/2425837959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ead6839d0 con 0x562ead679370 2024-09-10T09:04:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.947+0000 7f50b8c55640 1 --2- 172.21.15.172:0/2425837959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead679370 0x562ead6831c0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x562ead6829e0 tx=0x7f50b0031c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b1d19585a9d062 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.947+0000 7f509cff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.947+0000 7f509cff9640 1 -- 172.21.15.172:0/2425837959 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4005070 con 0x562ead679370 2024-09-10T09:04:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.947+0000 7f509cff9640 1 -- 172.21.15.172:0/2425837959 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f50a4005210 con 0x562ead679370 2024-09-10T09:04:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.947+0000 7f50be456740 1 -- 172.21.15.172:0/2425837959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50b003bad0 con 0x562ead679370 2024-09-10T09:04:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.947+0000 7f509cff9640 1 -- 172.21.15.172:0/2425837959 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f50a400f730 con 0x562ead679370 2024-09-10T09:04:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.947+0000 7f50be456740 10 client.19818 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.947+0000 7f50be456740 1 -- 172.21.15.172:0/2425837959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562ead6f50a0 con 0x562ead679370 2024-09-10T09:04:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.947+0000 7f509cff9640 1 -- 172.21.15.172:0/2425837959 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50b003bad0 con 0x562ead679370 2024-09-10T09:04:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.947+0000 7f509cff9640 1 -- 172.21.15.172:0/2425837959 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f50a4003930 con 0x562ead679370 2024-09-10T09:04:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.948+0000 7f50be456740 10 client.19818 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.948+0000 7f50be456740 10 client.19818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:04:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.948+0000 7f50be456740 20 client.19818 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.948+0000 7f50be456740 1 --2- 172.21.15.172:0/2425837959 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562ead6f58f0 0x562ead6f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.948+0000 7f50be456740 4 client.19818 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:04:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.948+0000 7f50be456740 1 -- 172.21.15.172:0/2425837959 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562ead5ded70 con 0x562ead6f58f0 2024-09-10T09:04:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.948+0000 7f50b9456640 1 --2- 172.21.15.172:0/2425837959 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562ead6f58f0 0x562ead6f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.948+0000 7f509cff9640 10 client.19818 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.948+0000 7f50b9456640 1 --2- 172.21.15.172:0/2425837959 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562ead6f58f0 0x562ead6f7d10 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5474a6989af98ecf server_cookie=311a8e8f48a159a6 in_seq=0 out_seq=0 2024-09-10T09:04:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.952+0000 7f509cff9640 1 -- 172.21.15.172:0/2425837959 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562ead5ded70 con 0x562ead6f58f0 2024-09-10T09:04:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.952+0000 7f509cff9640 10 client.19818 handle_command_reply: tid=0 2024-09-10T09:04:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.952+0000 7f50be456740 1 client.19818 shutdown 2024-09-10T09:04:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.952+0000 7f50be456740 1 -- 172.21.15.172:0/2425837959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead679370 msgr2=0x562ead6831c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.952+0000 7f50be456740 1 --2- 172.21.15.172:0/2425837959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead679370 0x562ead6831c0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x562ead6829e0 tx=0x7f50b0031c30 comp rx=0 tx=0).stop 2024-09-10T09:04:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.952+0000 7f50be456740 1 -- 172.21.15.172:0/2425837959 shutdown_connections 2024-09-10T09:04:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.952+0000 7f50be456740 1 --2- 172.21.15.172:0/2425837959 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x562ead6f58f0 0x562ead6f7d10 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.952+0000 7f50be456740 1 --2- 172.21.15.172:0/2425837959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ead679370 0x562ead6831c0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.952+0000 7f509cff9640 0 client.19818 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.952+0000 7f50be456740 1 --2- 172.21.15.172:0/2425837959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ead6742e0 0x562ead6850a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.952+0000 7f50be456740 1 --2- 172.21.15.172:0/2425837959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ead6739a0 0x562ead684b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.952+0000 7f50be456740 1 -- 172.21.15.172:0/2425837959 >> 172.21.15.172:0/2425837959 conn(0x562ead66f010 msgr2=0x562ead670740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.952+0000 7f50be456740 1 -- 172.21.15.172:0/2425837959 shutdown_connections 2024-09-10T09:04:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.952+0000 7f50be456740 1 -- 172.21.15.172:0/2425837959 wait complete. 2024-09-10T09:04:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.953+0000 7f50be456740 20 client.19818 trim_cache size 0 max 16384 2024-09-10T09:04:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:04:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:04:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:04:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:04:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.953+0000 7f50bb6ba640 1 -- 172.21.15.172:0/4257228656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f50a00a2cb0 msgr2=0x7f50a00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.953+0000 7f50bb6ba640 1 --2- 172.21.15.172:0/4257228656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f50a00a2cb0 0x7f50a00a5170 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f50ac09b8f0 tx=0x7f50ac06ab70 comp rx=0 tx=0).stop 2024-09-10T09:04:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.953+0000 7f50bb6ba640 1 -- 172.21.15.172:0/4257228656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b4102610 msgr2=0x7f50b419f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.953+0000 7f50bb6ba640 1 --2- 172.21.15.172:0/4257228656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b4102610 0x7f50b419f030 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00cce0 tx=0x7f50ac00a620 comp rx=0 tx=0).stop 2024-09-10T09:04:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.953+0000 7f50bb6ba640 1 -- 172.21.15.172:0/4257228656 shutdown_connections 2024-09-10T09:04:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.953+0000 7f50bb6ba640 1 --2- 172.21.15.172:0/4257228656 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f50a00a2cb0 0x7f50a00a5170 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.953+0000 7f50bb6ba640 1 --2- 172.21.15.172:0/4257228656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50b4102610 0x7f50b419f030 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.953+0000 7f50bb6ba640 1 --2- 172.21.15.172:0/4257228656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50b4101c50 0x7f50b419ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.953+0000 7f50bb6ba640 1 --2- 172.21.15.172:0/4257228656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50b4100a60 0x7f50b406ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.953+0000 7f50bb6ba640 1 -- 172.21.15.172:0/4257228656 >> 172.21.15.172:0/4257228656 conn(0x7f50b40fc1e0 msgr2=0x7f50b4105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.953+0000 7f50bb6ba640 1 -- 172.21.15.172:0/4257228656 shutdown_connections 2024-09-10T09:04:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:41.953+0000 7f50bb6ba640 1 -- 172.21.15.172:0/4257228656 wait complete. 2024-09-10T09:04:42.985 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:42 smithi172 ceph-mon[30854]: pgmap v904: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:04:42.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:42 smithi174 ceph-mon[34068]: pgmap v904: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:04:43.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:42 smithi191 ceph-mon[35314]: pgmap v904: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:04:44.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.129+0000 7fb0babbd640 1 -- 172.21.15.172:0/504680523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b4102d40 msgr2=0x7fb0b4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:44.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.129+0000 7fb0babbd640 1 --2- 172.21.15.172:0/504680523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b4102d40 0x7fb0b4103120 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009900 tx=0x7fb0a802a480 comp rx=0 tx=0).stop 2024-09-10T09:04:44.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.129+0000 7fb0babbd640 1 -- 172.21.15.172:0/504680523 shutdown_connections 2024-09-10T09:04:44.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.129+0000 7fb0babbd640 1 --2- 172.21.15.172:0/504680523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0b410c170 0x7fb0b410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.129+0000 7fb0babbd640 1 --2- 172.21.15.172:0/504680523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b4103660 0x7fb0b410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.129+0000 7fb0babbd640 1 --2- 172.21.15.172:0/504680523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b4102d40 0x7fb0b4103120 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.129+0000 7fb0babbd640 1 -- 172.21.15.172:0/504680523 >> 172.21.15.172:0/504680523 conn(0x7fb0b40fc7a0 msgr2=0x7fb0b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:44.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.129+0000 7fb0babbd640 1 -- 172.21.15.172:0/504680523 shutdown_connections 2024-09-10T09:04:44.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.129+0000 7fb0babbd640 1 -- 172.21.15.172:0/504680523 wait complete. 2024-09-10T09:04:44.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.130+0000 7fb0babbd640 1 Processor -- start 2024-09-10T09:04:44.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.130+0000 7fb0babbd640 1 -- start start 2024-09-10T09:04:44.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.130+0000 7fb0babbd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0b4102d40 0x7fb0b41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.130+0000 7fb0babbd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b4103660 0x7fb0b4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.130+0000 7fb0babbd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b410c170 0x7fb0b4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.130+0000 7fb0babbd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b4111eb0 con 0x7fb0b4103660 2024-09-10T09:04:44.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.130+0000 7fb0babbd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b4111d30 con 0x7fb0b410c170 2024-09-10T09:04:44.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.130+0000 7fb0babbd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b4112030 con 0x7fb0b4102d40 2024-09-10T09:04:44.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.130+0000 7fb0b3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b4103660 0x7fb0b4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:44.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.130+0000 7fb0b3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b4103660 0x7fb0b4100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56562/0 (socket says 172.21.15.172:56562) 2024-09-10T09:04:44.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.130+0000 7fb0b3fff640 1 -- 172.21.15.172:0/3502373292 learned_addr learned my addr 172.21.15.172:0/3502373292 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:44.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.131+0000 7fb0b8959640 1 --2- 172.21.15.172:0/3502373292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0b4102d40 0x7fb0b41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:44.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.131+0000 7fb0b915a640 1 --2- 172.21.15.172:0/3502373292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b410c170 0x7fb0b4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:44.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.131+0000 7fb0b3fff640 1 -- 172.21.15.172:0/3502373292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0b4102d40 msgr2=0x7fb0b41009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:44.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.131+0000 7fb0b3fff640 1 --2- 172.21.15.172:0/3502373292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0b4102d40 0x7fb0b41009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.131+0000 7fb0b3fff640 1 -- 172.21.15.172:0/3502373292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b410c170 msgr2=0x7fb0b4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:44.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.131+0000 7fb0b3fff640 1 --2- 172.21.15.172:0/3502373292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b410c170 0x7fb0b4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.131+0000 7fb0b3fff640 1 -- 172.21.15.172:0/3502373292 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b41a6840 con 0x7fb0b4103660 2024-09-10T09:04:44.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.131+0000 7fb0b915a640 1 --2- 172.21.15.172:0/3502373292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b410c170 0x7fb0b4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:04:44.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.131+0000 7fb0b3fff640 1 --2- 172.21.15.172:0/3502373292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b4103660 0x7fb0b4100ee0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000c940 tx=0x7fb0a000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:44.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.131+0000 7fb0b1ffb640 1 -- 172.21.15.172:0/3502373292 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0004220 con 0x7fb0b4103660 2024-09-10T09:04:44.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.131+0000 7fb0b1ffb640 1 -- 172.21.15.172:0/3502373292 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0a00043c0 con 0x7fb0b4103660 2024-09-10T09:04:44.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.131+0000 7fb0babbd640 1 -- 172.21.15.172:0/3502373292 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b41a6b30 con 0x7fb0b4103660 2024-09-10T09:04:44.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.131+0000 7fb0b1ffb640 1 -- 172.21.15.172:0/3502373292 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0004a00 con 0x7fb0b4103660 2024-09-10T09:04:44.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.131+0000 7fb0babbd640 1 -- 172.21.15.172:0/3502373292 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b41a70c0 con 0x7fb0b4103660 2024-09-10T09:04:44.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.133+0000 7fb0b1ffb640 1 -- 172.21.15.172:0/3502373292 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb0a00079b0 con 0x7fb0b4103660 2024-09-10T09:04:44.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.134+0000 7fb0bcf96740 1 Processor -- start 2024-09-10T09:04:44.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.134+0000 7fb0bcf96740 1 -- start start 2024-09-10T09:04:44.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.134+0000 7fb0b1ffb640 1 --2- 172.21.15.172:0/3502373292 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0a40a2c00 0x7fb0a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.134+0000 7fb0bcf96740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b591a4a0 0x5629b591a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.134+0000 7fb0bcf96740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591adc0 0x5629b5925140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.134+0000 7fb0bcf96740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b5925680 0x5629b5927a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.134+0000 7fb0bcf96740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5629b58802e0 con 0x5629b591adc0 2024-09-10T09:04:44.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.134+0000 7fb0bcf96740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5629b5827ce0 con 0x5629b5925680 2024-09-10T09:04:44.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.134+0000 7fb0bcf96740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5629b587c2a0 con 0x5629b591a4a0 2024-09-10T09:04:44.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.134+0000 7fb0b1ffb640 1 -- 172.21.15.172:0/3502373292 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb0a00cf3b0 con 0x7fb0b4103660 2024-09-10T09:04:44.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.134+0000 7fb0b915a640 1 --2- 172.21.15.172:0/3502373292 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0a40a2c00 0x7fb0a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:44.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.134+0000 7fb0b3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591adc0 0x5629b5925140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:44.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.134+0000 7fb0b3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591adc0 0x5629b5925140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56572/0 (socket says 172.21.15.172:56572) 2024-09-10T09:04:44.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.134+0000 7fb0b3fff640 1 -- 172.21.15.172:0/3852715676 learned_addr learned my addr 172.21.15.172:0/3852715676 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:44.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.135+0000 7fb0b8959640 1 --2- 172.21.15.172:0/3852715676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b591a4a0 0x5629b591a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:44.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.135+0000 7fb0b3fff640 1 -- 172.21.15.172:0/3852715676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b591a4a0 msgr2=0x5629b591a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:44.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.135+0000 7fb0b3fff640 1 --2- 172.21.15.172:0/3852715676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b591a4a0 0x5629b591a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.135+0000 7fb0b3fff640 1 -- 172.21.15.172:0/3852715676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b5925680 msgr2=0x5629b5927a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:44.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.135+0000 7fb0b3fff640 1 --2- 172.21.15.172:0/3852715676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b5925680 0x5629b5927a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.135+0000 7fb0b3fff640 1 -- 172.21.15.172:0/3852715676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629b580c440 con 0x5629b591adc0 2024-09-10T09:04:44.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.135+0000 7fb0b915a640 1 --2- 172.21.15.172:0/3502373292 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0a40a2c00 0x7fb0a40a50c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac007ea0 tx=0x7fb0ac00c690 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:44.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.135+0000 7fb0b3fff640 1 --2- 172.21.15.172:0/3852715676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591adc0 0x5629b5925140 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00a3ff0 tx=0x7fb0a009b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60728164797e1e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:44.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.135+0000 7fb09effd640 1 -- 172.21.15.172:0/3852715676 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a001d070 con 0x5629b591adc0 2024-09-10T09:04:44.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.135+0000 7fb09effd640 1 -- 172.21.15.172:0/3852715676 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0a009fc60 con 0x5629b591adc0 2024-09-10T09:04:44.967 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.135+0000 7fb09effd640 1 -- 172.21.15.172:0/3852715676 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0097f20 con 0x5629b591adc0 2024-09-10T09:04:44.967 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:44 smithi172 ceph-mon[30854]: pgmap v905: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:04:44.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.136+0000 7fb0bcf96740 1 -- 172.21.15.172:0/3852715676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591adc0 msgr2=0x5629b5925140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:44.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.136+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/3852715676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591adc0 0x5629b5925140 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00a3ff0 tx=0x7fb0a009b340 comp rx=0 tx=0).stop 2024-09-10T09:04:44.968 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.136+0000 7fb0bcf96740 1 -- 172.21.15.172:0/3852715676 shutdown_connections 2024-09-10T09:04:44.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.136+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/3852715676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b5925680 0x5629b5927a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.136+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/3852715676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591adc0 0x5629b5925140 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.136+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/3852715676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b591a4a0 0x5629b591a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.136+0000 7fb0bcf96740 1 -- 172.21.15.172:0/3852715676 >> 172.21.15.172:0/3852715676 conn(0x5629b5916a30 msgr2=0x5629b5916e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:44.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.136+0000 7fb0bcf96740 1 -- 172.21.15.172:0/3852715676 shutdown_connections 2024-09-10T09:04:44.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.136+0000 7fb0bcf96740 1 -- 172.21.15.172:0/3852715676 wait complete. 2024-09-10T09:04:44.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.136+0000 7fb0bcf96740 1 -- start start 2024-09-10T09:04:44.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.136+0000 7fb0bcf96740 1 Processor -- start 2024-09-10T09:04:44.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0bcf96740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b591a4a0 0x5629b5924fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0bcf96740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591adc0 0x5629b5922290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0bcf96740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b5925680 0x5629b59227f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0bcf96740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5629b5928250 con 0x5629b591adc0 2024-09-10T09:04:44.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0bcf96740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5629b583b050 con 0x5629b5925680 2024-09-10T09:04:44.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0bcf96740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5629b59283d0 con 0x5629b591a4a0 2024-09-10T09:04:44.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0bcf96740 10 client.0 _ll_register_callbacks cb 0x5629b587b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:44.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0b3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591adc0 0x5629b5922290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:44.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0b3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591adc0 0x5629b5922290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56576/0 (socket says 172.21.15.172:56576) 2024-09-10T09:04:44.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0b3fff640 1 -- 172.21.15.172:0/2386629011 learned_addr learned my addr 172.21.15.172:0/2386629011 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:44.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0b8959640 1 --2- 172.21.15.172:0/2386629011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b591a4a0 0x5629b5924fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:44.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0b3fff640 1 -- 172.21.15.172:0/2386629011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b591a4a0 msgr2=0x5629b5924fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:44.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0b915a640 1 --2- 172.21.15.172:0/2386629011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b5925680 0x5629b59227f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:44.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0b3fff640 1 --2- 172.21.15.172:0/2386629011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b591a4a0 0x5629b5924fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0b3fff640 1 -- 172.21.15.172:0/2386629011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b5925680 msgr2=0x5629b59227f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:44.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0b3fff640 1 --2- 172.21.15.172:0/2386629011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b5925680 0x5629b59227f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0b3fff640 1 -- 172.21.15.172:0/2386629011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629b55f13c0 con 0x5629b591adc0 2024-09-10T09:04:44.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0b8959640 1 --2- 172.21.15.172:0/2386629011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b591a4a0 0x5629b5924fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:04:44.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0b3fff640 1 --2- 172.21.15.172:0/2386629011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591adc0 0x5629b5922290 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00ccec0 tx=0x7fb0a0099950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3835ec4bd9b597 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:44.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb093fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:44.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb093fff640 1 -- 172.21.15.172:0/2386629011 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a001d040 con 0x5629b591adc0 2024-09-10T09:04:44.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb093fff640 1 -- 172.21.15.172:0/2386629011 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb0a009cbe0 con 0x5629b591adc0 2024-09-10T09:04:44.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.137+0000 7fb0bcf96740 1 -- 172.21.15.172:0/2386629011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a009fc60 con 0x5629b591adc0 2024-09-10T09:04:44.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.138+0000 7fb093fff640 1 -- 172.21.15.172:0/2386629011 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb0a009cde0 con 0x5629b591adc0 2024-09-10T09:04:44.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.138+0000 7fb0bcf96740 10 client.19830 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:44.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.138+0000 7fb0bcf96740 1 -- 172.21.15.172:0/2386629011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5629b5933c20 con 0x5629b591adc0 2024-09-10T09:04:44.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.138+0000 7fb093fff640 1 -- 172.21.15.172:0/2386629011 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0a009fc60 con 0x5629b591adc0 2024-09-10T09:04:44.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.138+0000 7fb093fff640 1 -- 172.21.15.172:0/2386629011 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fb0a0086050 con 0x5629b591adc0 2024-09-10T09:04:44.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.138+0000 7fb0bcf96740 10 client.19830 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:44.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.138+0000 7fb0bcf96740 10 client.19830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:04:44.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.138+0000 7fb0bcf96740 20 client.19830 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:44.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.138+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/2386629011 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5629b5934f70 0x5629b5937360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.138+0000 7fb0bcf96740 4 client.19830 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:04:44.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.138+0000 7fb0bcf96740 1 -- 172.21.15.172:0/2386629011 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629b588d850 con 0x5629b5934f70 2024-09-10T09:04:44.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.138+0000 7fb0b8959640 1 --2- 172.21.15.172:0/2386629011 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5629b5934f70 0x5629b5937360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:44.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.139+0000 7fb093fff640 10 client.19830 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:44.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.139+0000 7fb0b8959640 1 --2- 172.21.15.172:0/2386629011 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5629b5934f70 0x5629b5937360 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dc23cd1ff477811 server_cookie=f1375f270fb63fd6 in_seq=0 out_seq=0 2024-09-10T09:04:44.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.139+0000 7fb093fff640 1 -- 172.21.15.172:0/2386629011 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629b588d850 con 0x5629b5934f70 2024-09-10T09:04:44.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.139+0000 7fb093fff640 10 client.19830 handle_command_reply: tid=0 2024-09-10T09:04:44.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.139+0000 7fb0bcf96740 1 client.19830 shutdown 2024-09-10T09:04:44.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.140+0000 7fb0bcf96740 1 -- 172.21.15.172:0/2386629011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591adc0 msgr2=0x5629b5922290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:44.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.140+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/2386629011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591adc0 0x5629b5922290 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00ccec0 tx=0x7fb0a0099950 comp rx=0 tx=0).stop 2024-09-10T09:04:44.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.140+0000 7fb0bcf96740 1 -- 172.21.15.172:0/2386629011 shutdown_connections 2024-09-10T09:04:44.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.140+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/2386629011 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5629b5934f70 0x5629b5937360 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.140+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/2386629011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b5925680 0x5629b59227f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.140+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/2386629011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591adc0 0x5629b5922290 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.140+0000 7fb093fff640 0 client.19830 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:44.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.140+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/2386629011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b591a4a0 0x5629b5924fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.140+0000 7fb0bcf96740 1 -- 172.21.15.172:0/2386629011 >> 172.21.15.172:0/2386629011 conn(0x5629b5916a30 msgr2=0x5629b591c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:44.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.140+0000 7fb0bcf96740 1 -- 172.21.15.172:0/2386629011 shutdown_connections 2024-09-10T09:04:44.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.140+0000 7fb0bcf96740 1 -- 172.21.15.172:0/2386629011 wait complete. 2024-09-10T09:04:44.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.140+0000 7fb0bcf96740 20 client.19830 trim_cache size 0 max 16384 2024-09-10T09:04:44.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.157+0000 7fb0bcf96740 1 Processor -- start 2024-09-10T09:04:44.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.157+0000 7fb0bcf96740 1 -- start start 2024-09-10T09:04:44.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.157+0000 7fb0bcf96740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591a4a0 0x5629b59314b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.157+0000 7fb0bcf96740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b591adc0 0x5629b593c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.157+0000 7fb0bcf96740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b5925680 0x5629b593cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.157+0000 7fb0bcf96740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41b0e70 con 0x5629b591a4a0 2024-09-10T09:04:44.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.157+0000 7fb0bcf96740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41b29b0 con 0x5629b591adc0 2024-09-10T09:04:44.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.157+0000 7fb0bcf96740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41b1540 con 0x5629b5925680 2024-09-10T09:04:44.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.157+0000 7fb0b8959640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591a4a0 0x5629b59314b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:44.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.157+0000 7fb0b8959640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591a4a0 0x5629b59314b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56588/0 (socket says 172.21.15.172:56588) 2024-09-10T09:04:44.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.157+0000 7fb0b8959640 1 -- 172.21.15.172:0/124055880 learned_addr learned my addr 172.21.15.172:0/124055880 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:44.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.158+0000 7fb0b915a640 1 --2- 172.21.15.172:0/124055880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b5925680 0x5629b593cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:44.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.158+0000 7fb0b3fff640 1 --2- 172.21.15.172:0/124055880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b591adc0 0x5629b593c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:44.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.158+0000 7fb0b8959640 1 -- 172.21.15.172:0/124055880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b5925680 msgr2=0x5629b593cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:44.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.158+0000 7fb0b8959640 1 --2- 172.21.15.172:0/124055880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b5925680 0x5629b593cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.158+0000 7fb0b8959640 1 -- 172.21.15.172:0/124055880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b591adc0 msgr2=0x5629b593c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:44.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.158+0000 7fb0b8959640 1 --2- 172.21.15.172:0/124055880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b591adc0 0x5629b593c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.158+0000 7fb0b8959640 1 -- 172.21.15.172:0/124055880 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629b5925310 con 0x5629b591a4a0 2024-09-10T09:04:44.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.158+0000 7fb0b915a640 1 --2- 172.21.15.172:0/124055880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b5925680 0x5629b593cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:04:44.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.158+0000 7fb0b3fff640 1 --2- 172.21.15.172:0/124055880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b591adc0 0x5629b593c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:04:44.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.158+0000 7fb0b8959640 1 --2- 172.21.15.172:0/124055880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591a4a0 0x5629b59314b0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8005ab0 tx=0x7fb0a8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911945d6600d53e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:44.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.158+0000 7fb09e7fc640 1 -- 172.21.15.172:0/124055880 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8035040 con 0x5629b591a4a0 2024-09-10T09:04:44.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.158+0000 7fb09e7fc640 1 -- 172.21.15.172:0/124055880 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0a8045030 con 0x5629b591a4a0 2024-09-10T09:04:44.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.158+0000 7fb09e7fc640 1 -- 172.21.15.172:0/124055880 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8004e20 con 0x5629b591a4a0 2024-09-10T09:04:44.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.159+0000 7fb0bcf96740 1 -- 172.21.15.172:0/124055880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591a4a0 msgr2=0x5629b59314b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:44.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.159+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/124055880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591a4a0 0x5629b59314b0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8005ab0 tx=0x7fb0a8005e80 comp rx=0 tx=0).stop 2024-09-10T09:04:44.975 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.159+0000 7fb0bcf96740 1 -- 172.21.15.172:0/124055880 shutdown_connections 2024-09-10T09:04:44.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.159+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/124055880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b5925680 0x5629b593cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.159+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/124055880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b591adc0 0x5629b593c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.159+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/124055880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591a4a0 0x5629b59314b0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.159+0000 7fb0bcf96740 1 -- 172.21.15.172:0/124055880 >> 172.21.15.172:0/124055880 conn(0x5629b5916a30 msgr2=0x5629b5921380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:44.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.159+0000 7fb0bcf96740 1 -- 172.21.15.172:0/124055880 shutdown_connections 2024-09-10T09:04:44.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.159+0000 7fb0bcf96740 1 -- 172.21.15.172:0/124055880 wait complete. 2024-09-10T09:04:44.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.159+0000 7fb0bcf96740 1 -- start start 2024-09-10T09:04:44.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.159+0000 7fb0bcf96740 1 Processor -- start 2024-09-10T09:04:44.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0bcf96740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591a4a0 0x5629b592b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0bcf96740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b591adc0 0x5629b592b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0bcf96740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b5925680 0x5629b592bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0bcf96740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41b22e0 con 0x5629b591a4a0 2024-09-10T09:04:44.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0bcf96740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41b1c10 con 0x5629b591adc0 2024-09-10T09:04:44.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0bcf96740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41aee50 con 0x5629b5925680 2024-09-10T09:04:44.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0bcf96740 10 client.0 _ll_register_callbacks cb 0x7fb0a00cf040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:44.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0b8959640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591a4a0 0x5629b592b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:44.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0b8959640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591a4a0 0x5629b592b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:56600/0 (socket says 172.21.15.172:56600) 2024-09-10T09:04:44.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0b8959640 1 -- 172.21.15.172:0/3320786025 learned_addr learned my addr 172.21.15.172:0/3320786025 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:44.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0b3fff640 1 --2- 172.21.15.172:0/3320786025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b591adc0 0x5629b592b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:44.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0b915a640 1 --2- 172.21.15.172:0/3320786025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b5925680 0x5629b592bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:44.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0b8959640 1 -- 172.21.15.172:0/3320786025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b5925680 msgr2=0x5629b592bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:44.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0b8959640 1 --2- 172.21.15.172:0/3320786025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b5925680 0x5629b592bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0b8959640 1 -- 172.21.15.172:0/3320786025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b591adc0 msgr2=0x5629b592b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:44.977 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0b8959640 1 --2- 172.21.15.172:0/3320786025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b591adc0 0x5629b592b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0b8959640 1 -- 172.21.15.172:0/3320786025 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629b59a7340 con 0x5629b591a4a0 2024-09-10T09:04:44.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb0b8959640 1 --2- 172.21.15.172:0/3320786025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591a4a0 0x5629b592b4b0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8004650 tx=0x7fb0a8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1b82c573e0a523 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:44.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.160+0000 7fb093fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:44.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.161+0000 7fb093fff640 1 -- 172.21.15.172:0/3320786025 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8035040 con 0x5629b591a4a0 2024-09-10T09:04:44.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.161+0000 7fb093fff640 1 -- 172.21.15.172:0/3320786025 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb0a80037f0 con 0x5629b591a4a0 2024-09-10T09:04:44.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.161+0000 7fb0bcf96740 1 -- 172.21.15.172:0/3320786025 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a8045030 con 0x5629b591a4a0 2024-09-10T09:04:44.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.161+0000 7fb093fff640 1 -- 172.21.15.172:0/3320786025 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb0a80039f0 con 0x5629b591a4a0 2024-09-10T09:04:44.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.161+0000 7fb0bcf96740 10 client.19836 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:44.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.161+0000 7fb0bcf96740 1 -- 172.21.15.172:0/3320786025 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5629b59a8540 con 0x5629b591a4a0 2024-09-10T09:04:44.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.161+0000 7fb093fff640 1 -- 172.21.15.172:0/3320786025 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0a8045030 con 0x5629b591a4a0 2024-09-10T09:04:44.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.161+0000 7fb093fff640 1 -- 172.21.15.172:0/3320786025 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fb0a8053470 con 0x5629b591a4a0 2024-09-10T09:04:44.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.161+0000 7fb0bcf96740 10 client.19836 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:44.978 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.161+0000 7fb0bcf96740 10 client.19836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:04:44.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.161+0000 7fb0bcf96740 20 client.19836 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:44.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.161+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/3320786025 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5629b59a8ba0 0x5629b59aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:44.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.161+0000 7fb0bcf96740 4 client.19836 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:04:44.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.161+0000 7fb0bcf96740 1 -- 172.21.15.172:0/3320786025 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5629b5887d60 con 0x5629b59a8ba0 2024-09-10T09:04:44.979 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.161+0000 7fb0b3fff640 1 --2- 172.21.15.172:0/3320786025 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5629b59a8ba0 0x5629b59aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:44.979 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:04:44.979 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T09:04:44.979 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T09:04:44.979 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:04:44.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.162+0000 7fb093fff640 10 client.19836 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:44.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.162+0000 7fb0b3fff640 1 --2- 172.21.15.172:0/3320786025 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5629b59a8ba0 0x5629b59aafc0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be28e430649997c4 server_cookie=5af4bcfe0775ff51 in_seq=0 out_seq=0 2024-09-10T09:04:44.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.162+0000 7fb093fff640 1 -- 172.21.15.172:0/3320786025 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5629b5887d60 con 0x5629b59a8ba0 2024-09-10T09:04:44.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.162+0000 7fb093fff640 10 client.19836 handle_command_reply: tid=0 2024-09-10T09:04:44.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.162+0000 7fb0bcf96740 1 client.19836 shutdown 2024-09-10T09:04:44.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.162+0000 7fb0bcf96740 1 -- 172.21.15.172:0/3320786025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591a4a0 msgr2=0x5629b592b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:44.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.162+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/3320786025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591a4a0 0x5629b592b4b0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8004650 tx=0x7fb0a8005bd0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.162+0000 7fb0bcf96740 1 -- 172.21.15.172:0/3320786025 shutdown_connections 2024-09-10T09:04:44.980 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.162+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/3320786025 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5629b59a8ba0 0x5629b59aafc0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.162+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/3320786025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629b5925680 0x5629b592bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.162+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/3320786025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5629b591adc0 0x5629b592b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.162+0000 7fb093fff640 0 client.19836 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:44.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.162+0000 7fb0bcf96740 1 --2- 172.21.15.172:0/3320786025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5629b591a4a0 0x5629b592b4b0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.162+0000 7fb0bcf96740 1 -- 172.21.15.172:0/3320786025 >> 172.21.15.172:0/3320786025 conn(0x5629b5916a30 msgr2=0x5629b5923af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:44.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.162+0000 7fb0bcf96740 1 -- 172.21.15.172:0/3320786025 shutdown_connections 2024-09-10T09:04:44.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.162+0000 7fb0bcf96740 1 -- 172.21.15.172:0/3320786025 wait complete. 2024-09-10T09:04:44.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.162+0000 7fb0bcf96740 20 client.19836 trim_cache size 0 max 16384 2024-09-10T09:04:44.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.163+0000 7fb0babbd640 1 -- 172.21.15.172:0/3502373292 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0a40a2c00 msgr2=0x7fb0a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:44.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.163+0000 7fb0babbd640 1 --2- 172.21.15.172:0/3502373292 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0a40a2c00 0x7fb0a40a50c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac007ea0 tx=0x7fb0ac00c690 comp rx=0 tx=0).stop 2024-09-10T09:04:44.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.163+0000 7fb0babbd640 1 -- 172.21.15.172:0/3502373292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b4103660 msgr2=0x7fb0b4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:44.981 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.163+0000 7fb0babbd640 1 --2- 172.21.15.172:0/3502373292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b4103660 0x7fb0b4100ee0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000c940 tx=0x7fb0a000ce00 comp rx=0 tx=0).stop 2024-09-10T09:04:44.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.163+0000 7fb0babbd640 1 -- 172.21.15.172:0/3502373292 shutdown_connections 2024-09-10T09:04:44.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.163+0000 7fb0babbd640 1 --2- 172.21.15.172:0/3502373292 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb0a40a2c00 0x7fb0a40a50c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.163+0000 7fb0babbd640 1 --2- 172.21.15.172:0/3502373292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b410c170 0x7fb0b4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.163+0000 7fb0babbd640 1 --2- 172.21.15.172:0/3502373292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b4103660 0x7fb0b4100ee0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.163+0000 7fb0babbd640 1 --2- 172.21.15.172:0/3502373292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0b4102d40 0x7fb0b41009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:44.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.163+0000 7fb0babbd640 1 -- 172.21.15.172:0/3502373292 >> 172.21.15.172:0/3502373292 conn(0x7fb0b40fc7a0 msgr2=0x7fb0b40fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:44.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.163+0000 7fb0babbd640 1 -- 172.21.15.172:0/3502373292 shutdown_connections 2024-09-10T09:04:44.982 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:44.163+0000 7fb0babbd640 1 -- 172.21.15.172:0/3502373292 wait complete. 2024-09-10T09:04:44.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:44 smithi174 ceph-mon[34068]: pgmap v905: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:04:45.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:44 smithi191 ceph-mon[35314]: pgmap v905: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:04:46.771 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:04:46.894 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:46 smithi172 ceph-mon[30854]: pgmap v906: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-10T09:04:46.894 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:46 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:04:46.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:46 smithi174 ceph-mon[34068]: pgmap v906: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-10T09:04:46.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:46 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:04:47.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:46 smithi191 ceph-mon[35314]: pgmap v906: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-10T09:04:47.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:46 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:04:48.285 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:04:48.286 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:04:48.853 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:48 smithi172 ceph-mon[30854]: pgmap v907: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-10T09:04:48.853 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:04:48.853 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:04:48.853 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:04:48.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.851+0000 7f94cc949640 1 -- 172.21.15.172:0/3396281650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94c8101de0 msgr2=0x7f94c81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:48.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.851+0000 7f94cc949640 1 --2- 172.21.15.172:0/3396281650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94c8101de0 0x7f94c81021c0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009900 tx=0x7f94b802a410 comp rx=0 tx=0).stop 2024-09-10T09:04:48.869 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.851+0000 7f94cc949640 1 -- 172.21.15.172:0/3396281650 shutdown_connections 2024-09-10T09:04:48.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.851+0000 7f94cc949640 1 --2- 172.21.15.172:0/3396281650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94c8106f10 0x7f94c810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.851+0000 7f94cc949640 1 --2- 172.21.15.172:0/3396281650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94c8102790 0x7f94c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.851+0000 7f94cc949640 1 --2- 172.21.15.172:0/3396281650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94c8101de0 0x7f94c81021c0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.851+0000 7f94cc949640 1 -- 172.21.15.172:0/3396281650 >> 172.21.15.172:0/3396281650 conn(0x7f94c80fd8c0 msgr2=0x7f94c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:48.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.851+0000 7f94cc949640 1 -- 172.21.15.172:0/3396281650 shutdown_connections 2024-09-10T09:04:48.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.851+0000 7f94cc949640 1 -- 172.21.15.172:0/3396281650 wait complete. 2024-09-10T09:04:48.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.852+0000 7f94cc949640 1 Processor -- start 2024-09-10T09:04:48.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.852+0000 7f94cc949640 1 -- start start 2024-09-10T09:04:48.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.852+0000 7f94cc949640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94c8101de0 0x7f94c8198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.852+0000 7f94cc949640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94c8102790 0x7f94c81989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.870 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.852+0000 7f94cc949640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94c8106f10 0x7f94c819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.852+0000 7f94cc949640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f94c8111e40 con 0x7f94c8106f10 2024-09-10T09:04:48.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.852+0000 7f94cc949640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94c8111cc0 con 0x7f94c8102790 2024-09-10T09:04:48.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.852+0000 7f94cc949640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f94c8111fc0 con 0x7f94c8101de0 2024-09-10T09:04:48.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.852+0000 7f94c6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94c8106f10 0x7f94c819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:48.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.852+0000 7f94c6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94c8106f10 0x7f94c819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40046/0 (socket says 172.21.15.172:40046) 2024-09-10T09:04:48.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.852+0000 7f94c6d9d640 1 -- 172.21.15.172:0/743169382 learned_addr learned my addr 172.21.15.172:0/743169382 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:48.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.853+0000 7f94c659c640 1 --2- 172.21.15.172:0/743169382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94c8101de0 0x7f94c8198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:48.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.853+0000 7f94c5d9b640 1 --2- 172.21.15.172:0/743169382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94c8102790 0x7f94c81989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:48.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.853+0000 7f94c6d9d640 1 -- 172.21.15.172:0/743169382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94c8101de0 msgr2=0x7f94c8198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:48.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.853+0000 7f94c6d9d640 1 --2- 172.21.15.172:0/743169382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94c8101de0 0x7f94c8198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.853+0000 7f94c6d9d640 1 -- 172.21.15.172:0/743169382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94c8102790 msgr2=0x7f94c81989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:48.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.853+0000 7f94c6d9d640 1 --2- 172.21.15.172:0/743169382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94c8102790 0x7f94c81989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.853+0000 7f94c6d9d640 1 -- 172.21.15.172:0/743169382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c819d430 con 0x7f94c8106f10 2024-09-10T09:04:48.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.853+0000 7f94c6d9d640 1 --2- 172.21.15.172:0/743169382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94c8106f10 0x7f94c819cd50 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f94c000cad0 tx=0x7f94c000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:48.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.853+0000 7f94b77fe640 1 -- 172.21.15.172:0/743169382 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c000ae20 con 0x7f94c8106f10 2024-09-10T09:04:48.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.853+0000 7f94cc949640 1 -- 172.21.15.172:0/743169382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c819d720 con 0x7f94c8106f10 2024-09-10T09:04:48.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.853+0000 7f94b77fe640 1 -- 172.21.15.172:0/743169382 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94c0005020 con 0x7f94c8106f10 2024-09-10T09:04:48.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.853+0000 7f94b77fe640 1 -- 172.21.15.172:0/743169382 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c000f6b0 con 0x7f94c8106f10 2024-09-10T09:04:48.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.853+0000 7f94cc949640 1 -- 172.21.15.172:0/743169382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c81a5000 con 0x7f94c8106f10 2024-09-10T09:04:48.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.855+0000 7f94b77fe640 1 -- 172.21.15.172:0/743169382 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f94c00051d0 con 0x7f94c8106f10 2024-09-10T09:04:48.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.856+0000 7f94ced22740 1 Processor -- start 2024-09-10T09:04:48.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.856+0000 7f94ced22740 1 -- start start 2024-09-10T09:04:48.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.856+0000 7f94ced22740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3839060 0x559ce3839440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.856+0000 7f94ced22740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3839980 0x559ce3843d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.856+0000 7f94ced22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3844240 0x559ce3846630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.856+0000 7f94ced22740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559ce379e620 con 0x559ce3839060 2024-09-10T09:04:48.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.856+0000 7f94ced22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559ce3745e50 con 0x559ce3844240 2024-09-10T09:04:48.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.856+0000 7f94ced22740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559ce379a5e0 con 0x559ce3839980 2024-09-10T09:04:48.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.856+0000 7f94b77fe640 1 --2- 172.21.15.172:0/743169382 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94b00a2b80 0x7f94b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.856+0000 7f94c5d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3839060 0x559ce3839440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:48.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.856+0000 7f94c5d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3839060 0x559ce3839440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40050/0 (socket says 172.21.15.172:40050) 2024-09-10T09:04:48.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.856+0000 7f94c5d9b640 1 -- 172.21.15.172:0/2975526980 learned_addr learned my addr 172.21.15.172:0/2975526980 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:48.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.856+0000 7f94c6d9d640 1 --2- 172.21.15.172:0/743169382 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94b00a2b80 0x7f94b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:48.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.856+0000 7f94b77fe640 1 -- 172.21.15.172:0/743169382 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f94c00cf440 con 0x7f94c8106f10 2024-09-10T09:04:48.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.857+0000 7f94c5d9b640 1 -- 172.21.15.172:0/2975526980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3839980 msgr2=0x559ce3843d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:04:48.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.857+0000 7f94c5d9b640 1 --2- 172.21.15.172:0/2975526980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3839980 0x559ce3843d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.857+0000 7f94c5d9b640 1 -- 172.21.15.172:0/2975526980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3844240 msgr2=0x559ce3846630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:48.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.857+0000 7f94c5d9b640 1 --2- 172.21.15.172:0/2975526980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3844240 0x559ce3846630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.857+0000 7f94c5d9b640 1 -- 172.21.15.172:0/2975526980 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce3736d40 con 0x559ce3839060 2024-09-10T09:04:48.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.857+0000 7f94c5d9b640 1 --2- 172.21.15.172:0/2975526980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3839060 0x559ce3839440 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f94bc006e10 tx=0x7f94bc009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4f1a8c33776815 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:48.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.857+0000 7f94c6d9d640 1 --2- 172.21.15.172:0/743169382 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94b00a2b80 0x7f94b00a5040 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f94c009b8d0 tx=0x7f94c001b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:48.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.857+0000 7f94b4ff9640 1 -- 172.21.15.172:0/2975526980 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc018070 con 0x559ce3839060 2024-09-10T09:04:48.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.857+0000 7f94b4ff9640 1 -- 172.21.15.172:0/2975526980 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94bc010040 con 0x559ce3839060 2024-09-10T09:04:48.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.857+0000 7f94b4ff9640 1 -- 172.21.15.172:0/2975526980 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc01c670 con 0x559ce3839060 2024-09-10T09:04:48.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 1 -- 172.21.15.172:0/2975526980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3839060 msgr2=0x559ce3839440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:48.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 1 --2- 172.21.15.172:0/2975526980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3839060 0x559ce3839440 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f94bc006e10 tx=0x7f94bc009ea0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 1 -- 172.21.15.172:0/2975526980 shutdown_connections 2024-09-10T09:04:48.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 1 --2- 172.21.15.172:0/2975526980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3844240 0x559ce3846630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 1 --2- 172.21.15.172:0/2975526980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3839980 0x559ce3843d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 1 --2- 172.21.15.172:0/2975526980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3839060 0x559ce3839440 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 1 -- 172.21.15.172:0/2975526980 >> 172.21.15.172:0/2975526980 conn(0x559ce38355f0 msgr2=0x559ce38359d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 1 -- 172.21.15.172:0/2975526980 shutdown_connections 2024-09-10T09:04:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 1 -- 172.21.15.172:0/2975526980 wait complete. 2024-09-10T09:04:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 1 -- start start 2024-09-10T09:04:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 1 Processor -- start 2024-09-10T09:04:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3839060 0x559ce3841190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3839980 0x559ce38416d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3844240 0x559ce3841c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559ce3846e10 con 0x559ce3844240 2024-09-10T09:04:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559ce3726f10 con 0x559ce3839980 2024-09-10T09:04:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559ce3846f90 con 0x559ce3839060 2024-09-10T09:04:48.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94ced22740 10 client.0 _ll_register_callbacks cb 0x559ce374adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94c5d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3844240 0x559ce3841c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.858+0000 7f94c5d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3844240 0x559ce3841c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40054/0 (socket says 172.21.15.172:40054) 2024-09-10T09:04:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.859+0000 7f94c5d9b640 1 -- 172.21.15.172:0/287069043 learned_addr learned my addr 172.21.15.172:0/287069043 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.859+0000 7f94c659c640 1 --2- 172.21.15.172:0/287069043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3839980 0x559ce38416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.859+0000 7f94c5d9b640 1 -- 172.21.15.172:0/287069043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3839060 msgr2=0x559ce3841190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.859+0000 7f94c5d9b640 1 --2- 172.21.15.172:0/287069043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3839060 0x559ce3841190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.859+0000 7f94c5d9b640 1 -- 172.21.15.172:0/287069043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3839980 msgr2=0x559ce38416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.859+0000 7f94c5d9b640 1 --2- 172.21.15.172:0/287069043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3839980 0x559ce38416d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.859+0000 7f94c5d9b640 1 -- 172.21.15.172:0/287069043 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ce34ad580 con 0x559ce3844240 2024-09-10T09:04:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.859+0000 7f94c5d9b640 1 --2- 172.21.15.172:0/287069043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3844240 0x559ce3841c10 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009660 tx=0x7f94bc014b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514ff4896b406640 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.859+0000 7f94adffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:48.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.859+0000 7f94adffb640 1 -- 172.21.15.172:0/287069043 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc018050 con 0x559ce3844240 2024-09-10T09:04:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.859+0000 7f94adffb640 1 -- 172.21.15.172:0/287069043 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f94bc029630 con 0x559ce3844240 2024-09-10T09:04:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.859+0000 7f94ced22740 1 -- 172.21.15.172:0/287069043 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94bc010040 con 0x559ce3844240 2024-09-10T09:04:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.859+0000 7f94adffb640 1 -- 172.21.15.172:0/287069043 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f94bc009690 con 0x559ce3844240 2024-09-10T09:04:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.859+0000 7f94ced22740 10 client.19848 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.859+0000 7f94ced22740 1 -- 172.21.15.172:0/287069043 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559ce3577960 con 0x559ce3844240 2024-09-10T09:04:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.859+0000 7f94adffb640 1 -- 172.21.15.172:0/287069043 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94bc010040 con 0x559ce3844240 2024-09-10T09:04:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.860+0000 7f94adffb640 1 -- 172.21.15.172:0/287069043 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f94bc027450 con 0x559ce3844240 2024-09-10T09:04:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.860+0000 7f94ced22740 10 client.19848 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.860+0000 7f94ced22740 10 client.19848 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:04:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.860+0000 7f94ced22740 20 client.19848 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.860+0000 7f94ced22740 1 --2- 172.21.15.172:0/287069043 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ce3853d00 0x559ce38560f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.860+0000 7f94ced22740 4 client.19848 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:04:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.860+0000 7f94ced22740 1 -- 172.21.15.172:0/287069043 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ce37abb90 con 0x559ce3853d00 2024-09-10T09:04:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.860+0000 7f94c659c640 1 --2- 172.21.15.172:0/287069043 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ce3853d00 0x559ce38560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.860+0000 7f94adffb640 10 client.19848 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.860+0000 7f94c659c640 1 --2- 172.21.15.172:0/287069043 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ce3853d00 0x559ce38560f0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4283fadcc3b8a59 server_cookie=a82a4774becbfedc in_seq=0 out_seq=0 2024-09-10T09:04:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.861+0000 7f94adffb640 1 -- 172.21.15.172:0/287069043 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ce37abb90 con 0x559ce3853d00 2024-09-10T09:04:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.861+0000 7f94adffb640 10 client.19848 handle_command_reply: tid=0 2024-09-10T09:04:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.861+0000 7f94ced22740 1 client.19848 shutdown 2024-09-10T09:04:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.861+0000 7f94ced22740 1 -- 172.21.15.172:0/287069043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3844240 msgr2=0x559ce3841c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.861+0000 7f94ced22740 1 --2- 172.21.15.172:0/287069043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3844240 0x559ce3841c10 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009660 tx=0x7f94bc014b50 comp rx=0 tx=0).stop 2024-09-10T09:04:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.861+0000 7f94ced22740 1 -- 172.21.15.172:0/287069043 shutdown_connections 2024-09-10T09:04:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.861+0000 7f94ced22740 1 --2- 172.21.15.172:0/287069043 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ce3853d00 0x559ce38560f0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.861+0000 7f94ced22740 1 --2- 172.21.15.172:0/287069043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3844240 0x559ce3841c10 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.861+0000 7f94ced22740 1 --2- 172.21.15.172:0/287069043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3839980 0x559ce38416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.861+0000 7f94adffb640 0 client.19848 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.861+0000 7f94ced22740 1 --2- 172.21.15.172:0/287069043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3839060 0x559ce3841190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.861+0000 7f94ced22740 1 -- 172.21.15.172:0/287069043 >> 172.21.15.172:0/287069043 conn(0x559ce38355f0 msgr2=0x559ce383b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.861+0000 7f94ced22740 1 -- 172.21.15.172:0/287069043 shutdown_connections 2024-09-10T09:04:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.861+0000 7f94ced22740 1 -- 172.21.15.172:0/287069043 wait complete. 2024-09-10T09:04:48.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.861+0000 7f94ced22740 20 client.19848 trim_cache size 0 max 16384 2024-09-10T09:04:48.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.878+0000 7f94ced22740 1 Processor -- start 2024-09-10T09:04:48.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.878+0000 7f94ced22740 1 -- start start 2024-09-10T09:04:48.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.879+0000 7f94ced22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3839060 0x559ce38404a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.879+0000 7f94ced22740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3839980 0x559ce383bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.879+0000 7f94ced22740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3844240 0x559ce385b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.879+0000 7f94ced22740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81ac980 con 0x559ce3844240 2024-09-10T09:04:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.879+0000 7f94ced22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81ae4c0 con 0x559ce3839060 2024-09-10T09:04:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.879+0000 7f94ced22740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81ad050 con 0x559ce3839980 2024-09-10T09:04:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.879+0000 7f94c5d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3844240 0x559ce385b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.879+0000 7f94c5d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3844240 0x559ce385b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40066/0 (socket says 172.21.15.172:40066) 2024-09-10T09:04:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.879+0000 7f94c5d9b640 1 -- 172.21.15.172:0/128185727 learned_addr learned my addr 172.21.15.172:0/128185727 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.879+0000 7f94c659c640 1 --2- 172.21.15.172:0/128185727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3839980 0x559ce383bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.879+0000 7f94c5d9b640 1 --2- 172.21.15.172:0/128185727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3839060 0x559ce38404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.879+0000 7f94c5d9b640 1 -- 172.21.15.172:0/128185727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3839980 msgr2=0x559ce383bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:48.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.879+0000 7f94c5d9b640 1 --2- 172.21.15.172:0/128185727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3839980 0x559ce383bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.879+0000 7f94c5d9b640 1 -- 172.21.15.172:0/128185727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3839060 msgr2=0x559ce38404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:48.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.879+0000 7f94c5d9b640 1 --2- 172.21.15.172:0/128185727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3839060 0x559ce38404a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.879+0000 7f94c5d9b640 1 -- 172.21.15.172:0/128185727 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce38397b0 con 0x559ce3844240 2024-09-10T09:04:48.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.879+0000 7f94c5d9b640 1 --2- 172.21.15.172:0/128185727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3844240 0x559ce385b7a0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009fd0 tx=0x7f94bc009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ea419b0d499bd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:48.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.880+0000 7f94affff640 1 -- 172.21.15.172:0/128185727 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc018070 con 0x559ce3844240 2024-09-10T09:04:48.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.880+0000 7f94affff640 1 -- 172.21.15.172:0/128185727 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94bc014e50 con 0x559ce3844240 2024-09-10T09:04:48.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.880+0000 7f94affff640 1 -- 172.21.15.172:0/128185727 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc029a50 con 0x559ce3844240 2024-09-10T09:04:48.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.880+0000 7f94ced22740 1 -- 172.21.15.172:0/128185727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3844240 msgr2=0x559ce385b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:48.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.880+0000 7f94ced22740 1 --2- 172.21.15.172:0/128185727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3844240 0x559ce385b7a0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009fd0 tx=0x7f94bc009ea0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.880+0000 7f94ced22740 1 -- 172.21.15.172:0/128185727 shutdown_connections 2024-09-10T09:04:48.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.880+0000 7f94ced22740 1 --2- 172.21.15.172:0/128185727 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3844240 0x559ce385b7a0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.880+0000 7f94ced22740 1 --2- 172.21.15.172:0/128185727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3839980 0x559ce383bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.881+0000 7f94ced22740 1 --2- 172.21.15.172:0/128185727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3839060 0x559ce38404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.881+0000 7f94ced22740 1 -- 172.21.15.172:0/128185727 >> 172.21.15.172:0/128185727 conn(0x559ce38355f0 msgr2=0x559ce383fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:48.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.881+0000 7f94ced22740 1 -- 172.21.15.172:0/128185727 shutdown_connections 2024-09-10T09:04:48.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.881+0000 7f94ced22740 1 -- 172.21.15.172:0/128185727 wait complete. 2024-09-10T09:04:48.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.881+0000 7f94ced22740 1 -- start start 2024-09-10T09:04:48.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.881+0000 7f94ced22740 1 Processor -- start 2024-09-10T09:04:48.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.881+0000 7f94ced22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3839060 0x559ce3848240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.881+0000 7f94ced22740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3839980 0x559ce3848780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.881+0000 7f94ced22740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3844240 0x559ce3848d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.881+0000 7f94ced22740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81addf0 con 0x559ce3839980 2024-09-10T09:04:48.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.881+0000 7f94ced22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81ad720 con 0x559ce3839060 2024-09-10T09:04:48.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.881+0000 7f94ced22740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f94c81aa960 con 0x559ce3844240 2024-09-10T09:04:48.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.881+0000 7f94ced22740 10 client.0 _ll_register_callbacks cb 0x7f94bc014a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:48.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.881+0000 7f94c659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3839980 0x559ce3848780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:48.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.881+0000 7f94c659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3839980 0x559ce3848780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40074/0 (socket says 172.21.15.172:40074) 2024-09-10T09:04:48.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.881+0000 7f94c659c640 1 -- 172.21.15.172:0/3806657346 learned_addr learned my addr 172.21.15.172:0/3806657346 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:48.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.882+0000 7f94c5d9b640 1 --2- 172.21.15.172:0/3806657346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3839060 0x559ce3848240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:48.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.882+0000 7f94c5d9b640 1 --2- 172.21.15.172:0/3806657346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3844240 0x559ce3848d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:48.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.882+0000 7f94c659c640 1 -- 172.21.15.172:0/3806657346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3844240 msgr2=0x559ce3848d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:48.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.882+0000 7f94c659c640 1 --2- 172.21.15.172:0/3806657346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3844240 0x559ce3848d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.882+0000 7f94c659c640 1 -- 172.21.15.172:0/3806657346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3839060 msgr2=0x559ce3848240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:48.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.882+0000 7f94c659c640 1 --2- 172.21.15.172:0/3806657346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3839060 0x559ce3848240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.882+0000 7f94c659c640 1 -- 172.21.15.172:0/3806657346 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ce38bda60 con 0x559ce3839980 2024-09-10T09:04:48.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.882+0000 7f94c5d9b640 1 --2- 172.21.15.172:0/3806657346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3844240 0x559ce3848d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:04:48.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.882+0000 7f94c659c640 1 --2- 172.21.15.172:0/3806657346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3839980 0x559ce3848780 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009900 tx=0x7f94b8009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce9a8d98cf2ccbd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:48.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.882+0000 7f94adffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:48.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.882+0000 7f94adffb640 1 -- 172.21.15.172:0/3806657346 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b804d060 con 0x559ce3839980 2024-09-10T09:04:48.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.882+0000 7f94adffb640 1 -- 172.21.15.172:0/3806657346 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f94b80318b0 con 0x559ce3839980 2024-09-10T09:04:48.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.882+0000 7f94ced22740 1 -- 172.21.15.172:0/3806657346 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94bc014e50 con 0x559ce3839980 2024-09-10T09:04:48.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.882+0000 7f94adffb640 1 -- 172.21.15.172:0/3806657346 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f94b8054050 con 0x559ce3839980 2024-09-10T09:04:48.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.882+0000 7f94ced22740 10 client.19854 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:48.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.882+0000 7f94ced22740 1 -- 172.21.15.172:0/3806657346 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559ce38bee00 con 0x559ce3839980 2024-09-10T09:04:48.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.883+0000 7f94adffb640 1 -- 172.21.15.172:0/3806657346 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94bc014e50 con 0x559ce3839980 2024-09-10T09:04:48.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.883+0000 7f94adffb640 1 -- 172.21.15.172:0/3806657346 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f94b803c560 con 0x559ce3839980 2024-09-10T09:04:48.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.883+0000 7f94ced22740 10 client.19854 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:48.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.883+0000 7f94ced22740 10 client.19854 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:04:48.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.883+0000 7f94ced22740 20 client.19854 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:48.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.883+0000 7f94ced22740 1 --2- 172.21.15.172:0/3806657346 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ce38bf530 0x559ce38c1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:48.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.883+0000 7f94ced22740 4 client.19854 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d3ffbc76", "release": true, "await": true}] 2024-09-10T09:04:48.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.883+0000 7f94ced22740 1 -- 172.21.15.172:0/3806657346 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d3ffbc76", "release": true, "await": true}) -- 0x559ce37a60a0 con 0x559ce38bf530 2024-09-10T09:04:48.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.883+0000 7f94c5d9b640 1 --2- 172.21.15.172:0/3806657346 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ce38bf530 0x559ce38c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:48.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.883+0000 7f94adffb640 10 client.19854 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:48.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.883+0000 7f94c5d9b640 1 --2- 172.21.15.172:0/3806657346 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ce38bf530 0x559ce38c1950 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1757fd1e45f61fea server_cookie=3a10104680f986bc in_seq=0 out_seq=0 2024-09-10T09:04:48.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.900+0000 7f94adffb640 1 -- 172.21.15.172:0/3806657346 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559ce37a60a0 con 0x559ce38bf530 2024-09-10T09:04:48.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.900+0000 7f94adffb640 10 client.19854 handle_command_reply: tid=0 2024-09-10T09:04:48.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.900+0000 7f94ced22740 1 client.19854 shutdown 2024-09-10T09:04:48.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.901+0000 7f94ced22740 1 -- 172.21.15.172:0/3806657346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3839980 msgr2=0x559ce3848780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:48.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.901+0000 7f94ced22740 1 --2- 172.21.15.172:0/3806657346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3839980 0x559ce3848780 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009900 tx=0x7f94b8009680 comp rx=0 tx=0).stop 2024-09-10T09:04:48.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.901+0000 7f94ced22740 1 -- 172.21.15.172:0/3806657346 shutdown_connections 2024-09-10T09:04:48.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.901+0000 7f94ced22740 1 --2- 172.21.15.172:0/3806657346 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559ce38bf530 0x559ce38c1950 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.901+0000 7f94ced22740 1 --2- 172.21.15.172:0/3806657346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559ce3844240 0x559ce3848d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.901+0000 7f94ced22740 1 --2- 172.21.15.172:0/3806657346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ce3839980 0x559ce3848780 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.901+0000 7f94ced22740 1 --2- 172.21.15.172:0/3806657346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce3839060 0x559ce3848240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.901+0000 7f94ced22740 1 -- 172.21.15.172:0/3806657346 >> 172.21.15.172:0/3806657346 conn(0x559ce38355f0 msgr2=0x559ce3833f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:48.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.901+0000 7f94adffb640 0 client.19854 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:48.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.901+0000 7f94ced22740 1 -- 172.21.15.172:0/3806657346 shutdown_connections 2024-09-10T09:04:48.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.901+0000 7f94ced22740 1 -- 172.21.15.172:0/3806657346 wait complete. 2024-09-10T09:04:48.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.901+0000 7f94ced22740 20 client.19854 trim_cache size 0 max 16384 2024-09-10T09:04:48.904 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:04:48.905 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 44, 2024-09-10T09:04:48.905 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T09:04:48.905 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 56, 2024-09-10T09:04:48.905 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T09:04:48.905 INFO:teuthology.orchestra.run.smithi172.stdout: "d3ffbc76": { 2024-09-10T09:04:48.905 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 56, 2024-09-10T09:04:48.905 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T09:04:48.905 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T09:04:48.905 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T09:04:48.905 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T09:04:48.905 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:04:48.905 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T09:04:48.906 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 139.8, 2024-09-10T09:04:48.906 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T09:04:48.906 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T09:04:48.906 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T09:04:48.906 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T09:04:48.906 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T09:04:48.906 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T09:04:48.906 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:04:48.906 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:04:48.906 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:04:48.906 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:04:48.906 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:04:48.906 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:04:48.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.902+0000 7f94cc949640 1 -- 172.21.15.172:0/743169382 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94b00a2b80 msgr2=0x7f94b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:48.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.902+0000 7f94cc949640 1 --2- 172.21.15.172:0/743169382 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94b00a2b80 0x7f94b00a5040 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f94c009b8d0 tx=0x7f94c001b000 comp rx=0 tx=0).stop 2024-09-10T09:04:48.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.902+0000 7f94cc949640 1 -- 172.21.15.172:0/743169382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94c8106f10 msgr2=0x7f94c819cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:48.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.902+0000 7f94cc949640 1 --2- 172.21.15.172:0/743169382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94c8106f10 0x7f94c819cd50 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f94c000cad0 tx=0x7f94c000cf90 comp rx=0 tx=0).stop 2024-09-10T09:04:48.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.902+0000 7f94cc949640 1 -- 172.21.15.172:0/743169382 shutdown_connections 2024-09-10T09:04:48.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.902+0000 7f94cc949640 1 --2- 172.21.15.172:0/743169382 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f94b00a2b80 0x7f94b00a5040 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.902+0000 7f94cc949640 1 --2- 172.21.15.172:0/743169382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94c8106f10 0x7f94c819cd50 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.902+0000 7f94cc949640 1 --2- 172.21.15.172:0/743169382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94c8102790 0x7f94c81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.902+0000 7f94cc949640 1 --2- 172.21.15.172:0/743169382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94c8101de0 0x7f94c8198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:48.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.902+0000 7f94cc949640 1 -- 172.21.15.172:0/743169382 >> 172.21.15.172:0/743169382 conn(0x7f94c80fd8c0 msgr2=0x7f94c80fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:48.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.902+0000 7f94cc949640 1 -- 172.21.15.172:0/743169382 shutdown_connections 2024-09-10T09:04:48.908 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:48.902+0000 7f94cc949640 1 -- 172.21.15.172:0/743169382 wait complete. 2024-09-10T09:04:48.919 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:04:48.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:48 smithi174 ceph-mon[34068]: pgmap v907: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-10T09:04:48.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:04:48.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:04:48.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:04:49.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:48 smithi191 ceph-mon[35314]: pgmap v907: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-10T09:04:49.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:04:49.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:04:49.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:04:49.623 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd3ffbc76', total seconds elapsed: 84.3 2024-09-10T09:04:49.623 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.85 2024-09-10T09:04:49.623 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:04:50.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.159+0000 7f815636a640 1 -- 172.21.15.172:0/737544557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8150105560 msgr2=0x7f815010d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:50.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.159+0000 7f815636a640 1 --2- 172.21.15.172:0/737544557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8150105560 0x7f815010d0e0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f814800adb0 tx=0x7f814802b7a0 comp rx=0 tx=0).stop 2024-09-10T09:04:50.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.159+0000 7f815636a640 1 -- 172.21.15.172:0/737544557 shutdown_connections 2024-09-10T09:04:50.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.159+0000 7f815636a640 1 --2- 172.21.15.172:0/737544557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8150105560 0x7f815010d0e0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:50.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.159+0000 7f815636a640 1 --2- 172.21.15.172:0/737544557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8150100de0 0x7f8150104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:50.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.159+0000 7f815636a640 1 --2- 172.21.15.172:0/737544557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8150100430 0x7f8150100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:50.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.159+0000 7f815636a640 1 -- 172.21.15.172:0/737544557 >> 172.21.15.172:0/737544557 conn(0x7f8150077230 msgr2=0x7f81500fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:50.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.159+0000 7f815636a640 1 -- 172.21.15.172:0/737544557 shutdown_connections 2024-09-10T09:04:50.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.159+0000 7f815636a640 1 -- 172.21.15.172:0/737544557 wait complete. 2024-09-10T09:04:50.162 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.160+0000 7f815636a640 1 Processor -- start 2024-09-10T09:04:50.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.160+0000 7f815636a640 1 -- start start 2024-09-10T09:04:50.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.160+0000 7f815636a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8150100430 0x7f8150198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:50.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.160+0000 7f815636a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8150100de0 0x7f8150198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:50.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.160+0000 7f815636a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8150105560 0x7f815019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:50.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.160+0000 7f815636a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f815010fbf0 con 0x7f8150100de0 2024-09-10T09:04:50.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.160+0000 7f815636a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f815010fa70 con 0x7f8150105560 2024-09-10T09:04:50.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.160+0000 7f815636a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f815010fd70 con 0x7f8150100430 2024-09-10T09:04:50.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.161+0000 7f8147fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8150100de0 0x7f8150198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:50.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.161+0000 7f814ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8150100430 0x7f8150198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:50.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.161+0000 7f8147fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8150100de0 0x7f8150198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40092/0 (socket says 172.21.15.172:40092) 2024-09-10T09:04:50.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.161+0000 7f8147fff640 1 -- 172.21.15.172:0/2295928093 learned_addr learned my addr 172.21.15.172:0/2295928093 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:50.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.161+0000 7f8154907640 1 --2- 172.21.15.172:0/2295928093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8150105560 0x7f815019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:50.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.161+0000 7f814ffff640 1 -- 172.21.15.172:0/2295928093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8150105560 msgr2=0x7f815019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:50.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.161+0000 7f814ffff640 1 --2- 172.21.15.172:0/2295928093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8150105560 0x7f815019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:50.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.161+0000 7f814ffff640 1 -- 172.21.15.172:0/2295928093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8150100de0 msgr2=0x7f8150198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:50.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.161+0000 7f814ffff640 1 --2- 172.21.15.172:0/2295928093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8150100de0 0x7f8150198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:50.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.161+0000 7f814ffff640 1 -- 172.21.15.172:0/2295928093 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815019d3b0 con 0x7f8150100430 2024-09-10T09:04:50.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.161+0000 7f8147fff640 1 --2- 172.21.15.172:0/2295928093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8150100de0 0x7f8150198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:04:50.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.161+0000 7f814ffff640 1 --2- 172.21.15.172:0/2295928093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8150100430 0x7f8150198400 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f814000e9f0 tx=0x7f814000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:50.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.161+0000 7f814dffb640 1 -- 172.21.15.172:0/2295928093 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814000ccc0 con 0x7f8150100430 2024-09-10T09:04:50.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.161+0000 7f815636a640 1 -- 172.21.15.172:0/2295928093 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815019d6a0 con 0x7f8150100430 2024-09-10T09:04:50.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.161+0000 7f814dffb640 1 -- 172.21.15.172:0/2295928093 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8140005020 con 0x7f8150100430 2024-09-10T09:04:50.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.161+0000 7f814dffb640 1 -- 172.21.15.172:0/2295928093 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140004a00 con 0x7f8150100430 2024-09-10T09:04:50.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.162+0000 7f815636a640 1 -- 172.21.15.172:0/2295928093 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81501a4f80 con 0x7f8150100430 2024-09-10T09:04:50.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.163+0000 7f815636a640 1 -- 172.21.15.172:0/2295928093 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8150100810 con 0x7f8150100430 2024-09-10T09:04:50.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.164+0000 7f814dffb640 1 -- 172.21.15.172:0/2295928093 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8140004ba0 con 0x7f8150100430 2024-09-10T09:04:50.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.164+0000 7f814dffb640 1 --2- 172.21.15.172:0/2295928093 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f813c0a2b80 0x7f813c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:50.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.165+0000 7f8147fff640 1 --2- 172.21.15.172:0/2295928093 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f813c0a2b80 0x7f813c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:50.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.165+0000 7f814dffb640 1 -- 172.21.15.172:0/2295928093 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f81400cf350 con 0x7f8150100430 2024-09-10T09:04:50.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.166+0000 7f8147fff640 1 --2- 172.21.15.172:0/2295928093 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f813c0a2b80 0x7f813c0a5040 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8150102ef0 tx=0x7f8138005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:50.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.171+0000 7f814dffb640 1 -- 172.21.15.172:0/2295928093 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f814009bf50 con 0x7f8150100430 2024-09-10T09:04:50.247 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:04:50.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.495+0000 7f815636a640 1 -- 172.21.15.172:0/2295928093 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81501087a0 con 0x7f8150100430 2024-09-10T09:04:50.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.496+0000 7f814dffb640 1 -- 172.21.15.172:0/2295928093 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+12001 (secure 0 0 0) 0x7f814009b850 con 0x7f8150100430 2024-09-10T09:04:50.498 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T09:04:50.499 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":44,"btime":"2024-09-10T09:01:18:531063+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T09:01:17.697372+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T09:04:50.499 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 44 2024-09-10T09:04:50.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.500+0000 7f815636a640 1 -- 172.21.15.172:0/2295928093 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f813c0a2b80 msgr2=0x7f813c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:50.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.500+0000 7f815636a640 1 --2- 172.21.15.172:0/2295928093 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f813c0a2b80 0x7f813c0a5040 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8150102ef0 tx=0x7f8138005f90 comp rx=0 tx=0).stop 2024-09-10T09:04:50.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.500+0000 7f815636a640 1 -- 172.21.15.172:0/2295928093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8150100430 msgr2=0x7f8150198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:50.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.500+0000 7f815636a640 1 --2- 172.21.15.172:0/2295928093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8150100430 0x7f8150198400 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f814000e9f0 tx=0x7f814000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:04:50.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.500+0000 7f815636a640 1 -- 172.21.15.172:0/2295928093 shutdown_connections 2024-09-10T09:04:50.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.500+0000 7f815636a640 1 --2- 172.21.15.172:0/2295928093 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f813c0a2b80 0x7f813c0a5040 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:50.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.500+0000 7f815636a640 1 --2- 172.21.15.172:0/2295928093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8150105560 0x7f815019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:50.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.500+0000 7f815636a640 1 --2- 172.21.15.172:0/2295928093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8150100de0 0x7f8150198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:50.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.500+0000 7f815636a640 1 --2- 172.21.15.172:0/2295928093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8150100430 0x7f8150198400 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:50.502 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.500+0000 7f815636a640 1 -- 172.21.15.172:0/2295928093 >> 172.21.15.172:0/2295928093 conn(0x7f8150077230 msgr2=0x7f81500fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:50.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.500+0000 7f815636a640 1 -- 172.21.15.172:0/2295928093 shutdown_connections 2024-09-10T09:04:50.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:50.500+0000 7f815636a640 1 -- 172.21.15.172:0/2295928093 wait complete. 2024-09-10T09:04:50.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:50 smithi172 ceph-mon[30854]: pgmap v908: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-10T09:04:50.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:50 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2295928093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:04:50.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:50 smithi174 ceph-mon[34068]: pgmap v908: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-10T09:04:50.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:50 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2295928093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:04:51.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:50 smithi191 ceph-mon[35314]: pgmap v908: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-10T09:04:51.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:50 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2295928093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:04:51.331 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:04:51.605 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.603+0000 7f8e1a9f1640 1 -- 172.21.15.172:0/4026794133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e141036a0 msgr2=0x7f8e1410bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:51.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.603+0000 7f8e1a9f1640 1 --2- 172.21.15.172:0/4026794133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e141036a0 0x7f8e1410bb90 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009900 tx=0x7f8dfc02a480 comp rx=0 tx=0).stop 2024-09-10T09:04:51.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.603+0000 7f8e1a9f1640 1 -- 172.21.15.172:0/4026794133 shutdown_connections 2024-09-10T09:04:51.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.603+0000 7f8e1a9f1640 1 --2- 172.21.15.172:0/4026794133 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e1410c160 0x7f8e1410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:51.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.603+0000 7f8e1a9f1640 1 --2- 172.21.15.172:0/4026794133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e141036a0 0x7f8e1410bb90 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:51.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.603+0000 7f8e1a9f1640 1 --2- 172.21.15.172:0/4026794133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e14102d80 0x7f8e14103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:51.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.603+0000 7f8e1a9f1640 1 -- 172.21.15.172:0/4026794133 >> 172.21.15.172:0/4026794133 conn(0x7f8e140fc780 msgr2=0x7f8e140feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:51.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.604+0000 7f8e1a9f1640 1 -- 172.21.15.172:0/4026794133 shutdown_connections 2024-09-10T09:04:51.606 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.604+0000 7f8e1a9f1640 1 -- 172.21.15.172:0/4026794133 wait complete. 2024-09-10T09:04:51.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.605+0000 7f8e1a9f1640 1 Processor -- start 2024-09-10T09:04:51.607 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.605+0000 7f8e1a9f1640 1 -- start start 2024-09-10T09:04:51.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.605+0000 7f8e1a9f1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e14102d80 0x7f8e141009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:51.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.605+0000 7f8e1a9f1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e141036a0 0x7f8e14100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:51.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.605+0000 7f8e1a9f1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e1410c160 0x7f8e14101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:51.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.605+0000 7f8e1a9f1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8e14111e40 con 0x7f8e1410c160 2024-09-10T09:04:51.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.606+0000 7f8e1a9f1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e14111cc0 con 0x7f8e14102d80 2024-09-10T09:04:51.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.606+0000 7f8e13fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e14102d80 0x7f8e141009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:51.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.606+0000 7f8e13fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e14102d80 0x7f8e141009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:55740/0 (socket says 172.21.15.172:55740) 2024-09-10T09:04:51.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.606+0000 7f8e13fff640 1 -- 172.21.15.172:0/4271983265 learned_addr learned my addr 172.21.15.172:0/4271983265 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:51.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.606+0000 7f8e1a9f1640 1 -- 172.21.15.172:0/4271983265 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8e14111fc0 con 0x7f8e141036a0 2024-09-10T09:04:51.608 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.606+0000 7f8e18f8e640 1 --2- 172.21.15.172:0/4271983265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e1410c160 0x7f8e14101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:51.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.606+0000 7f8e18f8e640 1 -- 172.21.15.172:0/4271983265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e141036a0 msgr2=0x7f8e14100f20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:04:51.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.606+0000 7f8e18f8e640 1 --2- 172.21.15.172:0/4271983265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e141036a0 0x7f8e14100f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:51.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.606+0000 7f8e18f8e640 1 -- 172.21.15.172:0/4271983265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e14102d80 msgr2=0x7f8e141009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:51.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.606+0000 7f8e18f8e640 1 --2- 172.21.15.172:0/4271983265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e14102d80 0x7f8e141009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:51.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.606+0000 7f8e18f8e640 1 -- 172.21.15.172:0/4271983265 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e141a6820 con 0x7f8e1410c160 2024-09-10T09:04:51.609 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.607+0000 7f8e18f8e640 1 --2- 172.21.15.172:0/4271983265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e1410c160 0x7f8e14101460 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009fd0 tx=0x7f8e0c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:51.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.607+0000 7f8e11ffb640 1 -- 172.21.15.172:0/4271983265 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c00ce20 con 0x7f8e1410c160 2024-09-10T09:04:51.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.607+0000 7f8e11ffb640 1 -- 172.21.15.172:0/4271983265 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e0c005020 con 0x7f8e1410c160 2024-09-10T09:04:51.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.607+0000 7f8e11ffb640 1 -- 172.21.15.172:0/4271983265 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c010670 con 0x7f8e1410c160 2024-09-10T09:04:51.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.608+0000 7f8e1a9f1640 1 -- 172.21.15.172:0/4271983265 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e141a6b10 con 0x7f8e1410c160 2024-09-10T09:04:51.610 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.608+0000 7f8e1a9f1640 1 -- 172.21.15.172:0/4271983265 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e141a6fa0 con 0x7f8e1410c160 2024-09-10T09:04:51.612 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.610+0000 7f8e1a9f1640 1 -- 172.21.15.172:0/4271983265 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e14103160 con 0x7f8e1410c160 2024-09-10T09:04:51.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.611+0000 7f8e11ffb640 1 -- 172.21.15.172:0/4271983265 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8e0c0051d0 con 0x7f8e1410c160 2024-09-10T09:04:51.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.612+0000 7f8e11ffb640 1 --2- 172.21.15.172:0/4271983265 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8e000a2d50 0x7f8e000a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:51.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.612+0000 7f8e11ffb640 1 -- 172.21.15.172:0/4271983265 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8e0c0cf7e0 con 0x7f8e1410c160 2024-09-10T09:04:51.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.612+0000 7f8e13fff640 1 --2- 172.21.15.172:0/4271983265 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8e000a2d50 0x7f8e000a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:51.614 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.613+0000 7f8e13fff640 1 --2- 172.21.15.172:0/4271983265 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8e000a2d50 0x7f8e000a5210 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009c90 tx=0x7f8e04005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:51.623 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.621+0000 7f8e11ffb640 1 -- 172.21.15.172:0/4271983265 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e0c09c380 con 0x7f8e1410c160 2024-09-10T09:04:51.709 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:04:51.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.918+0000 7f8e1a9f1640 1 -- 172.21.15.172:0/4271983265 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e1402d7c0 con 0x7f8e1410c160 2024-09-10T09:04:51.922 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.920+0000 7f8e11ffb640 1 -- 172.21.15.172:0/4271983265 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+12001 (secure 0 0 0) 0x7f8e0c09bc80 con 0x7f8e1410c160 2024-09-10T09:04:51.923 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T09:04:51.923 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":44,"btime":"2024-09-10T09:01:18:531063+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T09:01:17.697372+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T09:04:51.924 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 44 2024-09-10T09:04:51.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.923+0000 7f8e1a9f1640 1 -- 172.21.15.172:0/4271983265 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8e000a2d50 msgr2=0x7f8e000a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:51.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.923+0000 7f8e1a9f1640 1 --2- 172.21.15.172:0/4271983265 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8e000a2d50 0x7f8e000a5210 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009c90 tx=0x7f8e04005f90 comp rx=0 tx=0).stop 2024-09-10T09:04:51.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.923+0000 7f8e1a9f1640 1 -- 172.21.15.172:0/4271983265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e1410c160 msgr2=0x7f8e14101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:51.925 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.923+0000 7f8e1a9f1640 1 --2- 172.21.15.172:0/4271983265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e1410c160 0x7f8e14101460 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009fd0 tx=0x7f8e0c00ef10 comp rx=0 tx=0).stop 2024-09-10T09:04:51.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.923+0000 7f8e1a9f1640 1 -- 172.21.15.172:0/4271983265 shutdown_connections 2024-09-10T09:04:51.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.923+0000 7f8e1a9f1640 1 --2- 172.21.15.172:0/4271983265 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8e000a2d50 0x7f8e000a5210 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:51.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.923+0000 7f8e1a9f1640 1 --2- 172.21.15.172:0/4271983265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e1410c160 0x7f8e14101460 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:51.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.923+0000 7f8e1a9f1640 1 --2- 172.21.15.172:0/4271983265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e141036a0 0x7f8e14100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:51.926 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.923+0000 7f8e1a9f1640 1 --2- 172.21.15.172:0/4271983265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e14102d80 0x7f8e141009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:51.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.923+0000 7f8e1a9f1640 1 -- 172.21.15.172:0/4271983265 >> 172.21.15.172:0/4271983265 conn(0x7f8e140fc780 msgr2=0x7f8e140fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:51.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.923+0000 7f8e1a9f1640 1 -- 172.21.15.172:0/4271983265 shutdown_connections 2024-09-10T09:04:51.927 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:51.923+0000 7f8e1a9f1640 1 -- 172.21.15.172:0/4271983265 wait complete. 2024-09-10T09:04:52.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.464+0000 7f34d317b640 1 -- 172.21.15.172:0/2959011211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34cc100a80 msgr2=0x7f34cc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:52.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.464+0000 7f34d317b640 1 --2- 172.21.15.172:0/2959011211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34cc100a80 0x7f34cc100ee0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009930 tx=0x7f34b802a420 comp rx=0 tx=0).stop 2024-09-10T09:04:52.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.464+0000 7f34d317b640 1 -- 172.21.15.172:0/2959011211 shutdown_connections 2024-09-10T09:04:52.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.464+0000 7f34d317b640 1 --2- 172.21.15.172:0/2959011211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34cc101420 0x7f34cc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.464+0000 7f34d317b640 1 --2- 172.21.15.172:0/2959011211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34cc100a80 0x7f34cc100ee0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.464+0000 7f34d317b640 1 --2- 172.21.15.172:0/2959011211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34cc106a80 0x7f34cc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.467 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.464+0000 7f34d317b640 1 -- 172.21.15.172:0/2959011211 >> 172.21.15.172:0/2959011211 conn(0x7f34cc0fc7a0 msgr2=0x7f34cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:52.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.465+0000 7f34d317b640 1 -- 172.21.15.172:0/2959011211 shutdown_connections 2024-09-10T09:04:52.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.465+0000 7f34d317b640 1 -- 172.21.15.172:0/2959011211 wait complete. 2024-09-10T09:04:52.468 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.465+0000 7f34d317b640 1 Processor -- start 2024-09-10T09:04:52.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.466+0000 7f34d317b640 1 -- start start 2024-09-10T09:04:52.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.466+0000 7f34d317b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34cc100a80 0x7f34cc10d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.466+0000 7f34d317b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34cc101420 0x7f34cc109820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.466+0000 7f34d0f17640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34cc100a80 0x7f34cc10d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.466+0000 7f34d0f17640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34cc100a80 0x7f34cc10d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:48184/0 (socket says 172.21.15.172:48184) 2024-09-10T09:04:52.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.466+0000 7f34d0f17640 1 -- 172.21.15.172:0/2349281520 learned_addr learned my addr 172.21.15.172:0/2349281520 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:52.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.466+0000 7f34d317b640 1 --2- 172.21.15.172:0/2349281520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34cc106a80 0x7f34cc109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.466+0000 7f34d317b640 1 -- 172.21.15.172:0/2349281520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc110d80 con 0x7f34cc101420 2024-09-10T09:04:52.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.466+0000 7f34d317b640 1 -- 172.21.15.172:0/2349281520 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc110c00 con 0x7f34cc106a80 2024-09-10T09:04:52.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.466+0000 7f34d317b640 1 -- 172.21.15.172:0/2349281520 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc110f00 con 0x7f34cc100a80 2024-09-10T09:04:52.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.466+0000 7f34cbfff640 1 --2- 172.21.15.172:0/2349281520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34cc101420 0x7f34cc109820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.466+0000 7f34d1718640 1 --2- 172.21.15.172:0/2349281520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34cc106a80 0x7f34cc109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.467+0000 7f34cbfff640 1 -- 172.21.15.172:0/2349281520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34cc100a80 msgr2=0x7f34cc10d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:52.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.467+0000 7f34cbfff640 1 --2- 172.21.15.172:0/2349281520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34cc100a80 0x7f34cc10d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.467+0000 7f34cbfff640 1 -- 172.21.15.172:0/2349281520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34cc106a80 msgr2=0x7f34cc109d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:52.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.467+0000 7f34cbfff640 1 --2- 172.21.15.172:0/2349281520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34cc106a80 0x7f34cc109d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.467+0000 7f34cbfff640 1 -- 172.21.15.172:0/2349281520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc10a620 con 0x7f34cc101420 2024-09-10T09:04:52.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.467+0000 7f34cbfff640 1 --2- 172.21.15.172:0/2349281520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34cc101420 0x7f34cc109820 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009850 tx=0x7f34b8003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:52.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.467+0000 7f34c9ffb640 1 -- 172.21.15.172:0/2349281520 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b803f070 con 0x7f34cc101420 2024-09-10T09:04:52.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.468+0000 7f34d317b640 1 -- 172.21.15.172:0/2349281520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34cc1aaee0 con 0x7f34cc101420 2024-09-10T09:04:52.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.468+0000 7f34c9ffb640 1 -- 172.21.15.172:0/2349281520 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34b8031b50 con 0x7f34cc101420 2024-09-10T09:04:52.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.468+0000 7f34c9ffb640 1 -- 172.21.15.172:0/2349281520 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8035030 con 0x7f34cc101420 2024-09-10T09:04:52.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.468+0000 7f34d317b640 1 -- 172.21.15.172:0/2349281520 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34cc1ab390 con 0x7f34cc101420 2024-09-10T09:04:52.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.472+0000 7f34d5554740 1 Processor -- start 2024-09-10T09:04:52.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.472+0000 7f34d5554740 1 -- start start 2024-09-10T09:04:52.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.472+0000 7f34d5554740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9eb40 0x563a69e9d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.472+0000 7f34d5554740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9d5d0 0x563a69e9da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.472+0000 7f34d5554740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69ea8370 0x563a69eaa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.472+0000 7f34d5554740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563a69e032f0 con 0x563a69e9d5d0 2024-09-10T09:04:52.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.472+0000 7f34d5554740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563a69daacf0 con 0x563a69ea8370 2024-09-10T09:04:52.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.472+0000 7f34d5554740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563a69dff2b0 con 0x563a69e9eb40 2024-09-10T09:04:52.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.472+0000 7f34c9ffb640 1 -- 172.21.15.172:0/2349281520 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f34b8042070 con 0x7f34cc101420 2024-09-10T09:04:52.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.473+0000 7f34d0f17640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9d5d0 0x563a69e9da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.473+0000 7f34d0f17640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9d5d0 0x563a69e9da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40122/0 (socket says 172.21.15.172:40122) 2024-09-10T09:04:52.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.473+0000 7f34d0f17640 1 -- 172.21.15.172:0/837399339 learned_addr learned my addr 172.21.15.172:0/837399339 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:52.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.473+0000 7f34cbfff640 1 --2- 172.21.15.172:0/837399339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69ea8370 0x563a69eaa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.473+0000 7f34d1718640 1 --2- 172.21.15.172:0/837399339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9eb40 0x563a69e9d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.474+0000 7f34c9ffb640 1 --2- 172.21.15.172:0/2349281520 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f34bc0a2cc0 0x7f34bc0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.474+0000 7f34d0f17640 1 -- 172.21.15.172:0/837399339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9eb40 msgr2=0x563a69e9d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:52.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.474+0000 7f34d0f17640 1 --2- 172.21.15.172:0/837399339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9eb40 0x563a69e9d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.474+0000 7f34d0f17640 1 -- 172.21.15.172:0/837399339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69ea8370 msgr2=0x563a69eaa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:52.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.474+0000 7f34d0f17640 1 --2- 172.21.15.172:0/837399339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69ea8370 0x563a69eaa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.474+0000 7f34d0f17640 1 -- 172.21.15.172:0/837399339 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a69d8f3a0 con 0x563a69e9d5d0 2024-09-10T09:04:52.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.474+0000 7f34d1718640 1 --2- 172.21.15.172:0/837399339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9eb40 0x563a69e9d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:04:52.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.474+0000 7f34d1718640 1 --2- 172.21.15.172:0/2349281520 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f34bc0a2cc0 0x7f34bc0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.475+0000 7f34c9ffb640 1 -- 172.21.15.172:0/2349281520 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f34b80ef480 con 0x7f34cc101420 2024-09-10T09:04:52.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.475+0000 7f34d0f17640 1 --2- 172.21.15.172:0/837399339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9d5d0 0x563a69e9da10 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f34c0009d90 tx=0x7f34c0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57db3474917371e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:52.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.475+0000 7f34d1718640 1 --2- 172.21.15.172:0/2349281520 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f34bc0a2cc0 0x7f34bc0a5180 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0fed50 tx=0x7f34c4014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:52.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.475+0000 7f34b6ffd640 1 -- 172.21.15.172:0/837399339 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c001e070 con 0x563a69e9d5d0 2024-09-10T09:04:52.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.476+0000 7f34b6ffd640 1 -- 172.21.15.172:0/837399339 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34c0019ae0 con 0x563a69e9d5d0 2024-09-10T09:04:52.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.476+0000 7f34d5554740 1 -- 172.21.15.172:0/837399339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9d5d0 msgr2=0x563a69e9da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:52.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.476+0000 7f34d5554740 1 --2- 172.21.15.172:0/837399339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9d5d0 0x563a69e9da10 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f34c0009d90 tx=0x7f34c0011810 comp rx=0 tx=0).stop 2024-09-10T09:04:52.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.476+0000 7f34d5554740 1 -- 172.21.15.172:0/837399339 shutdown_connections 2024-09-10T09:04:52.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.476+0000 7f34d5554740 1 --2- 172.21.15.172:0/837399339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69ea8370 0x563a69eaa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.476+0000 7f34d5554740 1 --2- 172.21.15.172:0/837399339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9d5d0 0x563a69e9da10 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.476+0000 7f34d5554740 1 --2- 172.21.15.172:0/837399339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9eb40 0x563a69e9d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.476+0000 7f34d5554740 1 -- 172.21.15.172:0/837399339 >> 172.21.15.172:0/837399339 conn(0x563a69e66310 msgr2=0x563a69e99c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:52.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.477+0000 7f34d5554740 1 -- 172.21.15.172:0/837399339 shutdown_connections 2024-09-10T09:04:52.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.477+0000 7f34d5554740 1 -- 172.21.15.172:0/837399339 wait complete. 2024-09-10T09:04:52.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.477+0000 7f34d5554740 1 -- start start 2024-09-10T09:04:52.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.477+0000 7f34d5554740 1 Processor -- start 2024-09-10T09:04:52.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.477+0000 7f34d5554740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9d5d0 0x563a69ea7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.477+0000 7f34d5554740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9eb40 0x563a69ea4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.477+0000 7f34d0f17640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9d5d0 0x563a69ea7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.477+0000 7f34d5554740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69ea8370 0x563a69ea54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34d5554740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563a69ebc0d0 con 0x563a69e9d5d0 2024-09-10T09:04:52.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34d5554740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563a69dbe060 con 0x563a69ea8370 2024-09-10T09:04:52.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34d5554740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563a69ebc250 con 0x563a69e9eb40 2024-09-10T09:04:52.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34d0f17640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9d5d0 0x563a69ea7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40130/0 (socket says 172.21.15.172:40130) 2024-09-10T09:04:52.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34d0f17640 1 -- 172.21.15.172:0/2280124261 learned_addr learned my addr 172.21.15.172:0/2280124261 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:52.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34d5554740 10 client.0 _ll_register_callbacks cb 0x563a69b118c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:52.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34cbfff640 1 --2- 172.21.15.172:0/2280124261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9eb40 0x563a69ea4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34d1718640 1 --2- 172.21.15.172:0/2280124261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69ea8370 0x563a69ea54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34d0f17640 1 -- 172.21.15.172:0/2280124261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9eb40 msgr2=0x563a69ea4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:52.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34d0f17640 1 --2- 172.21.15.172:0/2280124261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9eb40 0x563a69ea4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34d0f17640 1 -- 172.21.15.172:0/2280124261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69ea8370 msgr2=0x563a69ea54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:52.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34d0f17640 1 --2- 172.21.15.172:0/2280124261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69ea8370 0x563a69ea54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34d0f17640 1 -- 172.21.15.172:0/2280124261 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a69b743c0 con 0x563a69e9d5d0 2024-09-10T09:04:52.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34d0f17640 1 --2- 172.21.15.172:0/2280124261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9d5d0 0x563a69ea7cc0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f34c0009990 tx=0x7f34c0009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a724adc4159ed06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:52.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34abfff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:52.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34abfff640 1 -- 172.21.15.172:0/2280124261 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0026020 con 0x563a69e9d5d0 2024-09-10T09:04:52.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34d5554740 1 -- 172.21.15.172:0/2280124261 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34c0019ae0 con 0x563a69e9d5d0 2024-09-10T09:04:52.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.478+0000 7f34abfff640 1 -- 172.21.15.172:0/2280124261 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f34c00050c0 con 0x563a69e9d5d0 2024-09-10T09:04:52.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.479+0000 7f34abfff640 1 -- 172.21.15.172:0/2280124261 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f34c0031050 con 0x563a69e9d5d0 2024-09-10T09:04:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.479+0000 7f34d5554740 10 client.19878 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.479+0000 7f34d5554740 1 -- 172.21.15.172:0/2280124261 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563a69ec7b10 con 0x563a69e9d5d0 2024-09-10T09:04:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.479+0000 7f34abfff640 1 -- 172.21.15.172:0/2280124261 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34c00035f0 con 0x563a69e9d5d0 2024-09-10T09:04:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.479+0000 7f34abfff640 1 -- 172.21.15.172:0/2280124261 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f34c0021070 con 0x563a69e9d5d0 2024-09-10T09:04:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.479+0000 7f34d5554740 10 client.19878 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.479+0000 7f34d5554740 10 client.19878 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T09:04:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.479+0000 7f34d5554740 20 client.19878 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.479+0000 7f34d5554740 1 --2- 172.21.15.172:0/2280124261 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x563a69ec8f10 0x563a69ecb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.479+0000 7f34d5554740 4 client.19878 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:04:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.479+0000 7f34d5554740 1 -- 172.21.15.172:0/2280124261 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f34c0019ae0 con 0x563a69ec8f10 2024-09-10T09:04:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.479+0000 7f34cbfff640 1 --2- 172.21.15.172:0/2280124261 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x563a69ec8f10 0x563a69ecb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.480+0000 7f34abfff640 10 client.19878 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T09:04:52.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.480+0000 7f34cbfff640 1 --2- 172.21.15.172:0/2280124261 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x563a69ec8f10 0x563a69ecb300 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5e2b1df1c8cae525 server_cookie=67e38ced78e5f7f6 in_seq=0 out_seq=0 2024-09-10T09:04:52.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.481+0000 7f34abfff640 1 -- 172.21.15.172:0/2280124261 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f34c0019ae0 con 0x563a69ec8f10 2024-09-10T09:04:52.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.481+0000 7f34abfff640 10 client.19878 handle_command_reply: tid=0 2024-09-10T09:04:52.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.481+0000 7f34d5554740 1 client.19878 shutdown 2024-09-10T09:04:52.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.481+0000 7f34d5554740 1 -- 172.21.15.172:0/2280124261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9d5d0 msgr2=0x563a69ea7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:52.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.481+0000 7f34d5554740 1 --2- 172.21.15.172:0/2280124261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9d5d0 0x563a69ea7cc0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f34c0009990 tx=0x7f34c0009890 comp rx=0 tx=0).stop 2024-09-10T09:04:52.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.481+0000 7f34d5554740 1 -- 172.21.15.172:0/2280124261 shutdown_connections 2024-09-10T09:04:52.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.481+0000 7f34d5554740 1 --2- 172.21.15.172:0/2280124261 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x563a69ec8f10 0x563a69ecb300 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.481+0000 7f34abfff640 0 client.19878 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T09:04:52.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.481+0000 7f34d5554740 1 --2- 172.21.15.172:0/2280124261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69ea8370 0x563a69ea54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.481+0000 7f34d5554740 1 --2- 172.21.15.172:0/2280124261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9eb40 0x563a69ea4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.481+0000 7f34d5554740 1 --2- 172.21.15.172:0/2280124261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9d5d0 0x563a69ea7cc0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.481+0000 7f34d5554740 1 -- 172.21.15.172:0/2280124261 >> 172.21.15.172:0/2280124261 conn(0x563a69e66310 msgr2=0x563a69e9f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:52.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.481+0000 7f34d5554740 1 -- 172.21.15.172:0/2280124261 shutdown_connections 2024-09-10T09:04:52.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.481+0000 7f34d5554740 1 -- 172.21.15.172:0/2280124261 wait complete. 2024-09-10T09:04:52.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.481+0000 7f34d5554740 20 client.19878 trim_cache size 0 max 16384 2024-09-10T09:04:52.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.501+0000 7f34d5554740 1 Processor -- start 2024-09-10T09:04:52.503 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.501+0000 7f34d5554740 1 -- start start 2024-09-10T09:04:52.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.502+0000 7f34d5554740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9d5d0 0x563a69ec53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.502+0000 7f34d5554740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9eb40 0x563a69ed0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.502+0000 7f34d5554740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69ea8370 0x563a69ed0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.505 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.502+0000 7f34d5554740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc1aca00 con 0x563a69e9eb40 2024-09-10T09:04:52.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.502+0000 7f34d5554740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc1ae540 con 0x563a69ea8370 2024-09-10T09:04:52.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.502+0000 7f34d5554740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc1ad0d0 con 0x563a69e9d5d0 2024-09-10T09:04:52.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.502+0000 7f34d0f17640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9d5d0 0x563a69ec53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.502+0000 7f34d0f17640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9d5d0 0x563a69ec53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:48212/0 (socket says 172.21.15.172:48212) 2024-09-10T09:04:52.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.502+0000 7f34d0f17640 1 -- 172.21.15.172:0/2934335393 learned_addr learned my addr 172.21.15.172:0/2934335393 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:52.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.502+0000 7f34cbfff640 1 --2- 172.21.15.172:0/2934335393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9eb40 0x563a69ed0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.503+0000 7f34d1718640 1 --2- 172.21.15.172:0/2934335393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69ea8370 0x563a69ed0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.503+0000 7f34cbfff640 1 -- 172.21.15.172:0/2934335393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9d5d0 msgr2=0x563a69ec53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:52.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.503+0000 7f34cbfff640 1 --2- 172.21.15.172:0/2934335393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9d5d0 0x563a69ec53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.506 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.503+0000 7f34cbfff640 1 -- 172.21.15.172:0/2934335393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69ea8370 msgr2=0x563a69ed0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:52.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.503+0000 7f34cbfff640 1 --2- 172.21.15.172:0/2934335393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69ea8370 0x563a69ed0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.503+0000 7f34cbfff640 1 -- 172.21.15.172:0/2934335393 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a69ea5fd0 con 0x563a69e9eb40 2024-09-10T09:04:52.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.503+0000 7f34cbfff640 1 --2- 172.21.15.172:0/2934335393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9eb40 0x563a69ed0a70 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f34b80b83a0 tx=0x7f34b8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8187a8235c42d386 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:52.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.503+0000 7f34b67fc640 1 -- 172.21.15.172:0/2934335393 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8049040 con 0x563a69e9eb40 2024-09-10T09:04:52.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.503+0000 7f34b67fc640 1 -- 172.21.15.172:0/2934335393 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34b80bb9d0 con 0x563a69e9eb40 2024-09-10T09:04:52.507 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.503+0000 7f34b67fc640 1 -- 172.21.15.172:0/2934335393 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8031d00 con 0x563a69e9eb40 2024-09-10T09:04:52.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.504+0000 7f34d5554740 1 -- 172.21.15.172:0/2934335393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9eb40 msgr2=0x563a69ed0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:52.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.504+0000 7f34d5554740 1 --2- 172.21.15.172:0/2934335393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9eb40 0x563a69ed0a70 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f34b80b83a0 tx=0x7f34b8000bc0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.504+0000 7f34d5554740 1 -- 172.21.15.172:0/2934335393 shutdown_connections 2024-09-10T09:04:52.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.504+0000 7f34d5554740 1 --2- 172.21.15.172:0/2934335393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69ea8370 0x563a69ed0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.504+0000 7f34d5554740 1 --2- 172.21.15.172:0/2934335393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69e9eb40 0x563a69ed0a70 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.504+0000 7f34d5554740 1 --2- 172.21.15.172:0/2934335393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9d5d0 0x563a69ec53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.504+0000 7f34d5554740 1 -- 172.21.15.172:0/2934335393 >> 172.21.15.172:0/2934335393 conn(0x563a69e66310 msgr2=0x563a69ea4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:52.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.504+0000 7f34d5554740 1 -- 172.21.15.172:0/2934335393 shutdown_connections 2024-09-10T09:04:52.514 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.505+0000 7f34d5554740 1 -- 172.21.15.172:0/2934335393 wait complete. 2024-09-10T09:04:52.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.505+0000 7f34d5554740 1 -- start start 2024-09-10T09:04:52.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.505+0000 7f34d5554740 1 Processor -- start 2024-09-10T09:04:52.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.506+0000 7f34d5554740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9d5d0 0x563a69ec1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.506+0000 7f34d5554740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69e9eb40 0x563a69ea1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.506+0000 7f34d0f17640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9d5d0 0x563a69ec1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.506+0000 7f34d0f17640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9d5d0 0x563a69ec1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:48226/0 (socket says 172.21.15.172:48226) 2024-09-10T09:04:52.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.506+0000 7f34d0f17640 1 -- 172.21.15.172:0/1025595106 learned_addr learned my addr 172.21.15.172:0/1025595106 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:52.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.506+0000 7f34cbfff640 1 --2- 172.21.15.172:0/1025595106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69e9eb40 0x563a69ea1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.506+0000 7f34d5554740 1 --2- 172.21.15.172:0/1025595106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69ea8370 0x563a69ea2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.506+0000 7f34d5554740 1 -- 172.21.15.172:0/1025595106 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc1ade70 con 0x563a69ea8370 2024-09-10T09:04:52.515 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.506+0000 7f34d5554740 1 -- 172.21.15.172:0/1025595106 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc1ad7a0 con 0x563a69e9eb40 2024-09-10T09:04:52.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.506+0000 7f34d5554740 1 -- 172.21.15.172:0/1025595106 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc10ad90 con 0x563a69e9d5d0 2024-09-10T09:04:52.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.506+0000 7f34d1718640 1 --2- 172.21.15.172:0/1025595106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69ea8370 0x563a69ea2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.507+0000 7f34d5554740 10 client.0 _ll_register_callbacks cb 0x7f34c001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:52.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.507+0000 7f34cbfff640 1 -- 172.21.15.172:0/1025595106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9d5d0 msgr2=0x563a69ec1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:52.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.507+0000 7f34cbfff640 1 --2- 172.21.15.172:0/1025595106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9d5d0 0x563a69ec1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.507+0000 7f34cbfff640 1 -- 172.21.15.172:0/1025595106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69ea8370 msgr2=0x563a69ea2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:52.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.507+0000 7f34cbfff640 1 --2- 172.21.15.172:0/1025595106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69ea8370 0x563a69ea2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.507+0000 7f34cbfff640 1 -- 172.21.15.172:0/1025595106 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a69ea2bb0 con 0x563a69e9eb40 2024-09-10T09:04:52.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.507+0000 7f34cbfff640 1 --2- 172.21.15.172:0/1025595106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69e9eb40 0x563a69ea1d90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x563a69e9e320 tx=0x7f34b802a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7553c7b3496a54fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:52.516 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.507+0000 7f34abfff640 10 client.28646 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:04:52.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.507+0000 7f34d5554740 1 -- 172.21.15.172:0/1025595106 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34b80bb9d0 con 0x563a69e9eb40 2024-09-10T09:04:52.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.508+0000 7f34abfff640 1 -- 172.21.15.172:0/1025595106 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8049070 con 0x563a69e9eb40 2024-09-10T09:04:52.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.508+0000 7f34abfff640 1 -- 172.21.15.172:0/1025595106 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f34b8031d00 con 0x563a69e9eb40 2024-09-10T09:04:52.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.508+0000 7f34abfff640 1 -- 172.21.15.172:0/1025595106 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f34b80bb330 con 0x563a69e9eb40 2024-09-10T09:04:52.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.508+0000 7f34d5554740 10 client.28646 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:52.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.508+0000 7f34d5554740 1 -- 172.21.15.172:0/1025595106 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=44}) -- 0x563a69f33990 con 0x563a69e9eb40 2024-09-10T09:04:52.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.509+0000 7f34abfff640 1 -- 172.21.15.172:0/1025595106 <== mon.2 v2:172.21.15.174:3300/0 4 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f34b804c060 con 0x563a69e9eb40 2024-09-10T09:04:52.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.509+0000 7f34d5554740 10 client.28646 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:52.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.509+0000 7f34d5554740 10 client.28646 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T09:04:52.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.509+0000 7f34d5554740 20 client.28646 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:52.517 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.509+0000 7f34d5554740 1 --2- 172.21.15.172:0/1025595106 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x563a69f33d70 0x563a69f36190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:52.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.509+0000 7f34d5554740 4 client.28646 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:04:52.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.509+0000 7f34d5554740 1 -- 172.21.15.172:0/1025595106 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f34b80bb9d0 con 0x563a69f33d70 2024-09-10T09:04:52.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.509+0000 7f34d0f17640 1 --2- 172.21.15.172:0/1025595106 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x563a69f33d70 0x563a69f36190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:52.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.510+0000 7f34abfff640 10 client.28646 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T09:04:52.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.510+0000 7f34d0f17640 1 --2- 172.21.15.172:0/1025595106 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x563a69f33d70 0x563a69f36190 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b02af84be97d6bdd server_cookie=dc965ec5b5fa29fe in_seq=0 out_seq=0 2024-09-10T09:04:52.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.510+0000 7f34abfff640 1 -- 172.21.15.172:0/1025595106 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f34b80bb9d0 con 0x563a69f33d70 2024-09-10T09:04:52.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.510+0000 7f34abfff640 10 client.28646 handle_command_reply: tid=0 2024-09-10T09:04:52.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.510+0000 7f34d5554740 1 client.28646 shutdown 2024-09-10T09:04:52.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.511+0000 7f34d5554740 1 -- 172.21.15.172:0/1025595106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69e9eb40 msgr2=0x563a69ea1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:52.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.511+0000 7f34d5554740 1 --2- 172.21.15.172:0/1025595106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69e9eb40 0x563a69ea1d90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x563a69e9e320 tx=0x7f34b802a420 comp rx=0 tx=0).stop 2024-09-10T09:04:52.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.511+0000 7f34d5554740 1 -- 172.21.15.172:0/1025595106 shutdown_connections 2024-09-10T09:04:52.518 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.511+0000 7f34d5554740 1 --2- 172.21.15.172:0/1025595106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563a69ea8370 0x563a69ea2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.511+0000 7f34d5554740 1 --2- 172.21.15.172:0/1025595106 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x563a69f33d70 0x563a69f36190 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.511+0000 7f34abfff640 0 client.28646 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T09:04:52.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.511+0000 7f34d5554740 1 --2- 172.21.15.172:0/1025595106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a69e9eb40 0x563a69ea1d90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.511+0000 7f34d5554740 1 --2- 172.21.15.172:0/1025595106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563a69e9d5d0 0x563a69ec1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.511+0000 7f34d5554740 1 -- 172.21.15.172:0/1025595106 >> 172.21.15.172:0/1025595106 conn(0x563a69e66310 msgr2=0x563a69e9f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:52.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.511+0000 7f34d5554740 1 -- 172.21.15.172:0/1025595106 shutdown_connections 2024-09-10T09:04:52.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.511+0000 7f34d5554740 1 -- 172.21.15.172:0/1025595106 wait complete. 2024-09-10T09:04:52.519 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.511+0000 7f34d5554740 20 client.28646 trim_cache size 0 max 16384 2024-09-10T09:04:52.519 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:04:52.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.512+0000 7f34d317b640 1 -- 172.21.15.172:0/2349281520 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f34bc0a2cc0 msgr2=0x7f34bc0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:52.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.512+0000 7f34d317b640 1 --2- 172.21.15.172:0/2349281520 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f34bc0a2cc0 0x7f34bc0a5180 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0fed50 tx=0x7f34c4014040 comp rx=0 tx=0).stop 2024-09-10T09:04:52.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.512+0000 7f34d317b640 1 -- 172.21.15.172:0/2349281520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34cc101420 msgr2=0x7f34cc109820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:52.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.512+0000 7f34d317b640 1 --2- 172.21.15.172:0/2349281520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34cc101420 0x7f34cc109820 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009850 tx=0x7f34b8003b20 comp rx=0 tx=0).stop 2024-09-10T09:04:52.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.512+0000 7f34d317b640 1 -- 172.21.15.172:0/2349281520 shutdown_connections 2024-09-10T09:04:52.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.512+0000 7f34d317b640 1 --2- 172.21.15.172:0/2349281520 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f34bc0a2cc0 0x7f34bc0a5180 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.512+0000 7f34d317b640 1 --2- 172.21.15.172:0/2349281520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34cc106a80 0x7f34cc109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.512+0000 7f34d317b640 1 --2- 172.21.15.172:0/2349281520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34cc101420 0x7f34cc109820 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.512+0000 7f34d317b640 1 --2- 172.21.15.172:0/2349281520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34cc100a80 0x7f34cc10d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:52.520 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.512+0000 7f34d317b640 1 -- 172.21.15.172:0/2349281520 >> 172.21.15.172:0/2349281520 conn(0x7f34cc0fc7a0 msgr2=0x7f34cc0fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:52.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.512+0000 7f34d317b640 1 -- 172.21.15.172:0/2349281520 shutdown_connections 2024-09-10T09:04:52.521 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:52.512+0000 7f34d317b640 1 -- 172.21.15.172:0/2349281520 wait complete. 2024-09-10T09:04:52.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:52 smithi174 ceph-mon[34068]: pgmap v909: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 71 KiB/s wr, 28 op/s 2024-09-10T09:04:52.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:52 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/4271983265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:04:53.028 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:52 smithi172 ceph-mon[30854]: pgmap v909: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 71 KiB/s wr, 28 op/s 2024-09-10T09:04:53.028 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:52 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/4271983265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:04:53.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:52 smithi191 ceph-mon[35314]: pgmap v909: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 71 KiB/s wr, 28 op/s 2024-09-10T09:04:53.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:52 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/4271983265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:04:53.250 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:04:53.798 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0' 2024-09-10T09:04:53.798 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0 2024-09-10T09:04:53.821 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:04:54.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.355+0000 7fecd5c3b640 1 -- 172.21.15.172:0/3248702072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd0101df0 msgr2=0x7fecd01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:54.356 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.355+0000 7fecd5c3b640 1 --2- 172.21.15.172:0/3248702072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd0101df0 0x7fecd01021d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009860 tx=0x7fecc002a340 comp rx=0 tx=0).stop 2024-09-10T09:04:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.355+0000 7fecd5c3b640 1 -- 172.21.15.172:0/3248702072 shutdown_connections 2024-09-10T09:04:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.355+0000 7fecd5c3b640 1 --2- 172.21.15.172:0/3248702072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecd0106f20 0x7fecd010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.355+0000 7fecd5c3b640 1 --2- 172.21.15.172:0/3248702072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecd01027a0 0x7fecd01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.355+0000 7fecd5c3b640 1 --2- 172.21.15.172:0/3248702072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd0101df0 0x7fecd01021d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.357 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.355+0000 7fecd5c3b640 1 -- 172.21.15.172:0/3248702072 >> 172.21.15.172:0/3248702072 conn(0x7fecd00fd8f0 msgr2=0x7fecd00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:54.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.357+0000 7fecd5c3b640 1 -- 172.21.15.172:0/3248702072 shutdown_connections 2024-09-10T09:04:54.358 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.358+0000 7fecd5c3b640 1 -- 172.21.15.172:0/3248702072 wait complete. 2024-09-10T09:04:54.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.358+0000 7fecd5c3b640 1 Processor -- start 2024-09-10T09:04:54.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.359+0000 7fecd5c3b640 1 -- start start 2024-09-10T09:04:54.359 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.359+0000 7fecd5c3b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecd0101df0 0x7fecd01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.359+0000 7fecd5c3b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecd01027a0 0x7fecd01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.359+0000 7fecd5c3b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd0106f20 0x7fecd019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.359+0000 7fecd5c3b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fecd0111e30 con 0x7fecd01027a0 2024-09-10T09:04:54.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.359+0000 7fecd5c3b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fecd0111cb0 con 0x7fecd0106f20 2024-09-10T09:04:54.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.359+0000 7fecd5c3b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fecd0111fb0 con 0x7fecd0101df0 2024-09-10T09:04:54.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.359+0000 7feccf7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecd0101df0 0x7fecd01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.359+0000 7feccf7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecd0101df0 0x7fecd01983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:48240/0 (socket says 172.21.15.172:48240) 2024-09-10T09:04:54.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.359+0000 7feccf7fe640 1 -- 172.21.15.172:0/682945734 learned_addr learned my addr 172.21.15.172:0/682945734 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.359+0000 7feccffff640 1 --2- 172.21.15.172:0/682945734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd0106f20 0x7fecd019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.360+0000 7fecceffd640 1 --2- 172.21.15.172:0/682945734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecd01027a0 0x7fecd01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.360+0000 7feccffff640 1 -- 172.21.15.172:0/682945734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecd0101df0 msgr2=0x7fecd01983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.360+0000 7feccffff640 1 --2- 172.21.15.172:0/682945734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecd0101df0 0x7fecd01983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.360+0000 7feccffff640 1 -- 172.21.15.172:0/682945734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecd01027a0 msgr2=0x7fecd01988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:54.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.360+0000 7feccffff640 1 --2- 172.21.15.172:0/682945734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecd01027a0 0x7fecd01988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.360+0000 7feccffff640 1 -- 172.21.15.172:0/682945734 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd019d360 con 0x7fecd0106f20 2024-09-10T09:04:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.360+0000 7feccffff640 1 --2- 172.21.15.172:0/682945734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd0106f20 0x7fecd019cc80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009fd0 tx=0x7fecc800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.360+0000 7fecccff9640 1 -- 172.21.15.172:0/682945734 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc800ce20 con 0x7fecd0106f20 2024-09-10T09:04:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.361+0000 7fecccff9640 1 -- 172.21.15.172:0/682945734 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecc8005020 con 0x7fecd0106f20 2024-09-10T09:04:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.361+0000 7fecd5c3b640 1 -- 172.21.15.172:0/682945734 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd019d650 con 0x7fecd0106f20 2024-09-10T09:04:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.361+0000 7fecccff9640 1 -- 172.21.15.172:0/682945734 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc8010690 con 0x7fecd0106f20 2024-09-10T09:04:54.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.361+0000 7fecd5c3b640 1 -- 172.21.15.172:0/682945734 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd0074420 con 0x7fecd0106f20 2024-09-10T09:04:54.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.364+0000 7fecccff9640 1 -- 172.21.15.172:0/682945734 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fecc8010830 con 0x7fecd0106f20 2024-09-10T09:04:54.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.364+0000 7fecd8014740 1 Processor -- start 2024-09-10T09:04:54.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.364+0000 7fecd8014740 1 -- start start 2024-09-10T09:04:54.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.364+0000 7fecd8014740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ee9b40 0x55ca34ee8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.364+0000 7fecd8014740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34ee8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.364+0000 7fecd8014740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ef3370 0x55ca34ef5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.364+0000 7fecd8014740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ca34e4e2f0 con 0x55ca34ee9b40 2024-09-10T09:04:54.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.364+0000 7fecd8014740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ca34df5cf0 con 0x55ca34ef3370 2024-09-10T09:04:54.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.364+0000 7fecd8014740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ca34e4a2b0 con 0x55ca34ee85d0 2024-09-10T09:04:54.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.364+0000 7fecceffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ee9b40 0x55ca34ee8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.364+0000 7feccf7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34ee8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.364+0000 7fecceffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ee9b40 0x55ca34ee8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40174/0 (socket says 172.21.15.172:40174) 2024-09-10T09:04:54.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.365+0000 7fecceffd640 1 -- 172.21.15.172:0/739015268 learned_addr learned my addr 172.21.15.172:0/739015268 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:54.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.365+0000 7fecceffd640 1 --2- 172.21.15.172:0/739015268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ef3370 0x55ca34ef5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.365+0000 7fecceffd640 1 -- 172.21.15.172:0/739015268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 msgr2=0x55ca34ee8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:54.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.365+0000 7fecceffd640 1 --2- 172.21.15.172:0/739015268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34ee8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.365+0000 7fecceffd640 1 -- 172.21.15.172:0/739015268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ee9b40 msgr2=0x55ca34ee8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:54.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.365+0000 7fecceffd640 1 --2- 172.21.15.172:0/739015268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ee9b40 0x55ca34ee8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.365+0000 7fecceffd640 1 -- 172.21.15.172:0/739015268 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca34dda3a0 con 0x55ca34ef3370 2024-09-10T09:04:54.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.365+0000 7fecccff9640 1 --2- 172.21.15.172:0/682945734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fecb80a2c80 0x7fecb80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.365+0000 7fecceffd640 1 --2- 172.21.15.172:0/739015268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ef3370 0x55ca34ef5760 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fecc40117e0 tx=0x7fecc4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91ffa328046502dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:54.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.365+0000 7fecccff9640 1 -- 172.21.15.172:0/682945734 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fecc80cf8f0 con 0x7fecd0106f20 2024-09-10T09:04:54.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.365+0000 7feccf7fe640 1 --2- 172.21.15.172:0/682945734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fecb80a2c80 0x7fecb80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.366+0000 7fecb5ffb640 1 -- 172.21.15.172:0/739015268 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4011e20 con 0x55ca34ef3370 2024-09-10T09:04:54.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.366+0000 7fecb5ffb640 1 -- 172.21.15.172:0/739015268 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecc40054d0 con 0x55ca34ef3370 2024-09-10T09:04:54.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.366+0000 7fecb5ffb640 1 -- 172.21.15.172:0/739015268 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc40047a0 con 0x55ca34ef3370 2024-09-10T09:04:54.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.367+0000 7fecd8014740 1 -- 172.21.15.172:0/739015268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ef3370 msgr2=0x55ca34ef5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:54.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.367+0000 7feccf7fe640 1 --2- 172.21.15.172:0/682945734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fecb80a2c80 0x7fecb80a5140 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55ca34ee5ac0 tx=0x7fecc003c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:54.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.367+0000 7fecd8014740 1 --2- 172.21.15.172:0/739015268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ef3370 0x55ca34ef5760 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fecc40117e0 tx=0x7fecc4009260 comp rx=0 tx=0).stop 2024-09-10T09:04:54.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.367+0000 7fecd8014740 1 -- 172.21.15.172:0/739015268 shutdown_connections 2024-09-10T09:04:54.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.367+0000 7fecd8014740 1 --2- 172.21.15.172:0/739015268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ef3370 0x55ca34ef5760 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.367+0000 7fecd8014740 1 --2- 172.21.15.172:0/739015268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34ee8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.367+0000 7fecd8014740 1 --2- 172.21.15.172:0/739015268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ee9b40 0x55ca34ee8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.367+0000 7fecd8014740 1 -- 172.21.15.172:0/739015268 >> 172.21.15.172:0/739015268 conn(0x55ca34eb1310 msgr2=0x55ca34ee4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:54.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.367+0000 7fecd8014740 1 -- 172.21.15.172:0/739015268 shutdown_connections 2024-09-10T09:04:54.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.367+0000 7fecd8014740 1 -- 172.21.15.172:0/739015268 wait complete. 2024-09-10T09:04:54.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.367+0000 7fecd8014740 1 -- start start 2024-09-10T09:04:54.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.367+0000 7fecd8014740 1 Processor -- start 2024-09-10T09:04:54.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.368+0000 7fecd8014740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34ef2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.368+0000 7fecd8014740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ee9b40 0x55ca34ef00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.368+0000 7fecd8014740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ef3370 0x55ca34ef0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.368+0000 7feccf7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34ef2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.368+0000 7feccf7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34ef2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:48268/0 (socket says 172.21.15.172:48268) 2024-09-10T09:04:54.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.368+0000 7feccf7fe640 1 -- 172.21.15.172:0/2470802132 learned_addr learned my addr 172.21.15.172:0/2470802132 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:54.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.368+0000 7fecceffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ee9b40 0x55ca34ef00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.368+0000 7fecd8014740 1 -- 172.21.15.172:0/2470802132 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ca34ef6070 con 0x55ca34ef3370 2024-09-10T09:04:54.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.368+0000 7fecd8014740 1 -- 172.21.15.172:0/2470802132 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ca34e09060 con 0x55ca34ee9b40 2024-09-10T09:04:54.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.368+0000 7fecd8014740 1 -- 172.21.15.172:0/2470802132 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ca34ef61f0 con 0x55ca34ee85d0 2024-09-10T09:04:54.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.369+0000 7feccffff640 1 --2- 172.21.15.172:0/2470802132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ef3370 0x55ca34ef0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.369+0000 7fecd8014740 10 client.0 _ll_register_callbacks cb 0x55ca34b5c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:54.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.369+0000 7feccf7fe640 1 -- 172.21.15.172:0/2470802132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ee9b40 msgr2=0x55ca34ef00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:54.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.369+0000 7feccf7fe640 1 --2- 172.21.15.172:0/2470802132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ee9b40 0x55ca34ef00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.369+0000 7feccf7fe640 1 -- 172.21.15.172:0/2470802132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ef3370 msgr2=0x55ca34ef0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:54.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.369+0000 7feccf7fe640 1 --2- 172.21.15.172:0/2470802132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ef3370 0x55ca34ef0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.369+0000 7feccf7fe640 1 -- 172.21.15.172:0/2470802132 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca34bbf3c0 con 0x55ca34ee85d0 2024-09-10T09:04:54.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.369+0000 7feccf7fe640 1 --2- 172.21.15.172:0/2470802132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34ef2e20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fecc0004650 tx=0x7fecc002a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dcb60c14f4eb54d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:54.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.369+0000 7fecaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:04:54.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.370+0000 7fecaeffd640 1 -- 172.21.15.172:0/2470802132 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc00035f0 con 0x55ca34ee85d0 2024-09-10T09:04:54.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.370+0000 7fecaeffd640 1 -- 172.21.15.172:0/2470802132 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fecc0003790 con 0x55ca34ee85d0 2024-09-10T09:04:54.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.370+0000 7fecd8014740 1 -- 172.21.15.172:0/2470802132 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecc40054d0 con 0x55ca34ee85d0 2024-09-10T09:04:54.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.370+0000 7fecaeffd640 1 -- 172.21.15.172:0/2470802132 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fecc00039b0 con 0x55ca34ee85d0 2024-09-10T09:04:54.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.370+0000 7fecd8014740 10 client.28552 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:54.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.370+0000 7fecd8014740 1 -- 172.21.15.172:0/2470802132 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ca34f01c30 con 0x55ca34ee85d0 2024-09-10T09:04:54.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.370+0000 7fecaeffd640 1 -- 172.21.15.172:0/2470802132 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecc40054d0 con 0x55ca34ee85d0 2024-09-10T09:04:54.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.370+0000 7fecaeffd640 1 -- 172.21.15.172:0/2470802132 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fecc0032440 con 0x55ca34ee85d0 2024-09-10T09:04:54.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.371+0000 7fecd8014740 10 client.28552 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:54.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.371+0000 7fecd8014740 10 client.28552 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T09:04:54.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.371+0000 7fecd8014740 20 client.28552 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:54.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.371+0000 7fecd8014740 1 --2- 172.21.15.172:0/2470802132 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55ca34f03140 0x55ca34f055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.371+0000 7fecd8014740 4 client.28552 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:04:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.371+0000 7fecd8014740 1 -- 172.21.15.172:0/2470802132 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca34e5b860 con 0x55ca34f03140 2024-09-10T09:04:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.371+0000 7fecceffd640 1 --2- 172.21.15.172:0/2470802132 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55ca34f03140 0x55ca34f055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.371+0000 7fecceffd640 1 --2- 172.21.15.172:0/2470802132 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55ca34f03140 0x55ca34f055d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=314580384eb4afac server_cookie=e103d0f47f0b3d1f in_seq=0 out_seq=0 2024-09-10T09:04:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.371+0000 7fecaeffd640 10 client.28552 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T09:04:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.372+0000 7fecaeffd640 1 -- 172.21.15.172:0/2470802132 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca34e5b860 con 0x55ca34f03140 2024-09-10T09:04:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.372+0000 7fecaeffd640 10 client.28552 handle_command_reply: tid=0 2024-09-10T09:04:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.372+0000 7fecd8014740 1 client.28552 shutdown 2024-09-10T09:04:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.373+0000 7fecd8014740 1 -- 172.21.15.172:0/2470802132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 msgr2=0x55ca34ef2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.373+0000 7fecd8014740 1 --2- 172.21.15.172:0/2470802132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34ef2e20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fecc0004650 tx=0x7fecc002a340 comp rx=0 tx=0).stop 2024-09-10T09:04:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.373+0000 7fecd8014740 1 -- 172.21.15.172:0/2470802132 shutdown_connections 2024-09-10T09:04:54.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.373+0000 7fecd8014740 1 --2- 172.21.15.172:0/2470802132 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55ca34f03140 0x55ca34f055d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.373+0000 7fecaeffd640 0 client.28552 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T09:04:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.373+0000 7fecd8014740 1 --2- 172.21.15.172:0/2470802132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ef3370 0x55ca34ef0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.373+0000 7fecd8014740 1 --2- 172.21.15.172:0/2470802132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ee9b40 0x55ca34ef00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.373+0000 7fecd8014740 1 --2- 172.21.15.172:0/2470802132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34ef2e20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.373+0000 7fecd8014740 1 -- 172.21.15.172:0/2470802132 >> 172.21.15.172:0/2470802132 conn(0x55ca34eb1310 msgr2=0x55ca34eea930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.374+0000 7fecd8014740 1 -- 172.21.15.172:0/2470802132 shutdown_connections 2024-09-10T09:04:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.374+0000 7fecd8014740 1 -- 172.21.15.172:0/2470802132 wait complete. 2024-09-10T09:04:54.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.374+0000 7fecd8014740 20 client.28552 trim_cache size 0 max 16384 2024-09-10T09:04:54.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.396+0000 7fecd8014740 1 Processor -- start 2024-09-10T09:04:54.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.397+0000 7fecd8014740 1 -- start start 2024-09-10T09:04:54.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.397+0000 7fecd8014740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34ee9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.397+0000 7fecd8014740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ee9b40 0x55ca34f0aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.397+0000 7fecd8014740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ef3370 0x55ca34f0b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.397+0000 7fecd8014740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc00fb70 con 0x55ca34ef3370 2024-09-10T09:04:54.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.397+0000 7fecd8014740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc0116b0 con 0x55ca34ee9b40 2024-09-10T09:04:54.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.397+0000 7fecd8014740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc010240 con 0x55ca34ee85d0 2024-09-10T09:04:54.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.397+0000 7fecceffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ee9b40 0x55ca34f0aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.397+0000 7feccf7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34ee9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.397+0000 7fecceffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ee9b40 0x55ca34f0aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:55854/0 (socket says 172.21.15.172:55854) 2024-09-10T09:04:54.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.397+0000 7feccf7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34ee9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:48274/0 (socket says 172.21.15.172:48274) 2024-09-10T09:04:54.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.397+0000 7fecceffd640 1 -- 172.21.15.172:0/3117202457 learned_addr learned my addr 172.21.15.172:0/3117202457 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:54.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.397+0000 7feccffff640 1 --2- 172.21.15.172:0/3117202457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ef3370 0x55ca34f0b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.398+0000 7fecceffd640 1 -- 172.21.15.172:0/3117202457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 msgr2=0x55ca34ee9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:54.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.398+0000 7fecceffd640 1 --2- 172.21.15.172:0/3117202457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34ee9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.398+0000 7fecceffd640 1 -- 172.21.15.172:0/3117202457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ef3370 msgr2=0x55ca34f0b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:54.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.398+0000 7fecceffd640 1 --2- 172.21.15.172:0/3117202457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ef3370 0x55ca34f0b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.398+0000 7fecceffd640 1 -- 172.21.15.172:0/3117202457 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca34ee5dd0 con 0x55ca34ee9b40 2024-09-10T09:04:54.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.398+0000 7feccf7fe640 1 --2- 172.21.15.172:0/3117202457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34ee9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:04:54.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.398+0000 7fecceffd640 1 --2- 172.21.15.172:0/3117202457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ee9b40 0x55ca34f0aea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fecc4014010 tx=0x7fecc401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=839ffb6de9ea08b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:54.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.398+0000 7fecb57fa640 1 -- 172.21.15.172:0/3117202457 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4009e80 con 0x55ca34ee9b40 2024-09-10T09:04:54.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.398+0000 7fecb57fa640 1 -- 172.21.15.172:0/3117202457 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecc400fe80 con 0x55ca34ee9b40 2024-09-10T09:04:54.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.398+0000 7fecb57fa640 1 -- 172.21.15.172:0/3117202457 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4005440 con 0x55ca34ee9b40 2024-09-10T09:04:54.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.399+0000 7fecd8014740 1 -- 172.21.15.172:0/3117202457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ee9b40 msgr2=0x55ca34f0aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:54.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.399+0000 7fecd8014740 1 --2- 172.21.15.172:0/3117202457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ee9b40 0x55ca34f0aea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fecc4014010 tx=0x7fecc401a040 comp rx=0 tx=0).stop 2024-09-10T09:04:54.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.399+0000 7fecd8014740 1 -- 172.21.15.172:0/3117202457 shutdown_connections 2024-09-10T09:04:54.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.399+0000 7fecd8014740 1 --2- 172.21.15.172:0/3117202457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ef3370 0x55ca34f0b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.399+0000 7fecd8014740 1 --2- 172.21.15.172:0/3117202457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ee9b40 0x55ca34f0aea0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.399+0000 7fecd8014740 1 --2- 172.21.15.172:0/3117202457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34ee9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.399+0000 7fecd8014740 1 -- 172.21.15.172:0/3117202457 >> 172.21.15.172:0/3117202457 conn(0x55ca34eb1310 msgr2=0x55ca34edfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:54.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.399+0000 7fecd8014740 1 -- 172.21.15.172:0/3117202457 shutdown_connections 2024-09-10T09:04:54.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.399+0000 7fecd8014740 1 -- 172.21.15.172:0/3117202457 wait complete. 2024-09-10T09:04:54.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.400+0000 7fecd8014740 1 -- start start 2024-09-10T09:04:54.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.400+0000 7fecd8014740 1 Processor -- start 2024-09-10T09:04:54.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.401+0000 7fecd8014740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34efb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.401+0000 7fecd8014740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ee9b40 0x55ca34efbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.401+0000 7fecd8014740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ef3370 0x55ca34ef6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.401+0000 7fecd8014740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc010fe0 con 0x55ca34ee9b40 2024-09-10T09:04:54.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.401+0000 7fecd8014740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc010910 con 0x55ca34ef3370 2024-09-10T09:04:54.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.401+0000 7fecd8014740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc00db50 con 0x55ca34ee85d0 2024-09-10T09:04:54.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.401+0000 7fecd8014740 10 client.0 _ll_register_callbacks cb 0x7fecc4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:54.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.401+0000 7feccf7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34efb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.401+0000 7feccf7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34efb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:48284/0 (socket says 172.21.15.172:48284) 2024-09-10T09:04:54.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.401+0000 7feccf7fe640 1 -- 172.21.15.172:0/719411914 learned_addr learned my addr 172.21.15.172:0/719411914 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:54.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.401+0000 7fecceffd640 1 --2- 172.21.15.172:0/719411914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ee9b40 0x55ca34efbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.401+0000 7feccffff640 1 --2- 172.21.15.172:0/719411914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ef3370 0x55ca34ef6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.401+0000 7feccf7fe640 1 -- 172.21.15.172:0/719411914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ef3370 msgr2=0x55ca34ef6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:54.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.401+0000 7feccf7fe640 1 --2- 172.21.15.172:0/719411914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ef3370 0x55ca34ef6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.401+0000 7feccf7fe640 1 -- 172.21.15.172:0/719411914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ee9b40 msgr2=0x55ca34efbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:54.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.401+0000 7feccf7fe640 1 --2- 172.21.15.172:0/719411914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ee9b40 0x55ca34efbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.401+0000 7feccf7fe640 1 -- 172.21.15.172:0/719411914 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca34ef7270 con 0x55ca34ee85d0 2024-09-10T09:04:54.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.402+0000 7feccf7fe640 1 --2- 172.21.15.172:0/719411914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34efb5d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fecc0035040 tx=0x7fecc0032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53d1a3a181564dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:54.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.402+0000 7fecaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:04:54.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.402+0000 7fecaeffd640 1 -- 172.21.15.172:0/719411914 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0067060 con 0x55ca34ee85d0 2024-09-10T09:04:54.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.402+0000 7fecaeffd640 1 -- 172.21.15.172:0/719411914 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fecc0003b50 con 0x55ca34ee85d0 2024-09-10T09:04:54.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.402+0000 7fecd8014740 1 -- 172.21.15.172:0/719411914 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ca34e55d70 con 0x55ca34ee85d0 2024-09-10T09:04:54.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.403+0000 7fecaeffd640 1 -- 172.21.15.172:0/719411914 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fecc0003800 con 0x55ca34ee85d0 2024-09-10T09:04:54.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.403+0000 7fecd8014740 10 client.28558 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:54.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.403+0000 7fecd8014740 1 -- 172.21.15.172:0/719411914 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ca34f6d9d0 con 0x55ca34ee85d0 2024-09-10T09:04:54.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.403+0000 7fecaeffd640 1 -- 172.21.15.172:0/719411914 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecc0032640 con 0x55ca34ee85d0 2024-09-10T09:04:54.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.403+0000 7fecaeffd640 1 -- 172.21.15.172:0/719411914 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fecc0062450 con 0x55ca34ee85d0 2024-09-10T09:04:54.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.403+0000 7fecd8014740 10 client.28558 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:54.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.403+0000 7fecd8014740 10 client.28558 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T09:04:54.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.403+0000 7fecd8014740 20 client.28558 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:54.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.403+0000 7fecd8014740 1 --2- 172.21.15.172:0/719411914 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55ca34f6e120 0x55ca34f70540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:54.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.403+0000 7fecd8014740 4 client.28558 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:04:54.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.403+0000 7fecd8014740 1 -- 172.21.15.172:0/719411914 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca34e55d70 con 0x55ca34f6e120 2024-09-10T09:04:54.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.403+0000 7fecceffd640 1 --2- 172.21.15.172:0/719411914 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55ca34f6e120 0x55ca34f70540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:54.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.404+0000 7fecaeffd640 10 client.28558 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T09:04:54.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.404+0000 7fecceffd640 1 --2- 172.21.15.172:0/719411914 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55ca34f6e120 0x55ca34f70540 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e1db0c3de098bc15 server_cookie=995fdbf16344912d in_seq=0 out_seq=0 2024-09-10T09:04:54.408 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:04:54.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.404+0000 7fecaeffd640 1 -- 172.21.15.172:0/719411914 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca34e55d70 con 0x55ca34f6e120 2024-09-10T09:04:54.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.404+0000 7fecaeffd640 10 client.28558 handle_command_reply: tid=0 2024-09-10T09:04:54.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.404+0000 7fecd8014740 1 client.28558 shutdown 2024-09-10T09:04:54.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.405+0000 7fecd8014740 1 -- 172.21.15.172:0/719411914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 msgr2=0x55ca34efb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:54.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.405+0000 7fecd8014740 1 --2- 172.21.15.172:0/719411914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34efb5d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fecc0035040 tx=0x7fecc0032ce0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.405+0000 7fecd8014740 1 -- 172.21.15.172:0/719411914 shutdown_connections 2024-09-10T09:04:54.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.405+0000 7fecd8014740 1 --2- 172.21.15.172:0/719411914 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55ca34f6e120 0x55ca34f70540 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.405+0000 7fecd8014740 1 --2- 172.21.15.172:0/719411914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca34ef3370 0x55ca34ef6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.405+0000 7fecaeffd640 0 client.28558 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T09:04:54.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.405+0000 7fecd8014740 1 --2- 172.21.15.172:0/719411914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca34ee9b40 0x55ca34efbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.405+0000 7fecd8014740 1 --2- 172.21.15.172:0/719411914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca34ee85d0 0x55ca34efb5d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.405+0000 7fecd8014740 1 -- 172.21.15.172:0/719411914 >> 172.21.15.172:0/719411914 conn(0x55ca34eb1310 msgr2=0x55ca34eee9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:54.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.405+0000 7fecd8014740 1 -- 172.21.15.172:0/719411914 shutdown_connections 2024-09-10T09:04:54.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.405+0000 7fecd8014740 1 -- 172.21.15.172:0/719411914 wait complete. 2024-09-10T09:04:54.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.405+0000 7fecd8014740 20 client.28558 trim_cache size 0 max 16384 2024-09-10T09:04:54.410 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:04:54.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.406+0000 7fecd5c3b640 1 -- 172.21.15.172:0/682945734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fecb80a2c80 msgr2=0x7fecb80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:54.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.406+0000 7fecd5c3b640 1 --2- 172.21.15.172:0/682945734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fecb80a2c80 0x7fecb80a5140 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55ca34ee5ac0 tx=0x7fecc003c040 comp rx=0 tx=0).stop 2024-09-10T09:04:54.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.406+0000 7fecd5c3b640 1 -- 172.21.15.172:0/682945734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd0106f20 msgr2=0x7fecd019cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:54.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.406+0000 7fecd5c3b640 1 --2- 172.21.15.172:0/682945734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd0106f20 0x7fecd019cc80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009fd0 tx=0x7fecc800ef10 comp rx=0 tx=0).stop 2024-09-10T09:04:54.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.406+0000 7fecd5c3b640 1 -- 172.21.15.172:0/682945734 shutdown_connections 2024-09-10T09:04:54.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.406+0000 7fecd5c3b640 1 --2- 172.21.15.172:0/682945734 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fecb80a2c80 0x7fecb80a5140 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.406+0000 7fecd5c3b640 1 --2- 172.21.15.172:0/682945734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd0106f20 0x7fecd019cc80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.406+0000 7fecd5c3b640 1 --2- 172.21.15.172:0/682945734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecd01027a0 0x7fecd01988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.406+0000 7fecd5c3b640 1 --2- 172.21.15.172:0/682945734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fecd0101df0 0x7fecd01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:54.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.406+0000 7fecd5c3b640 1 -- 172.21.15.172:0/682945734 >> 172.21.15.172:0/682945734 conn(0x7fecd00fd8f0 msgr2=0x7fecd00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:54.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.406+0000 7fecd5c3b640 1 -- 172.21.15.172:0/682945734 shutdown_connections 2024-09-10T09:04:54.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:54.406+0000 7fecd5c3b640 1 -- 172.21.15.172:0/682945734 wait complete. 2024-09-10T09:04:54.870 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:54 smithi172 ceph-mon[30854]: pgmap v910: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 266 KiB/s wr, 75 op/s 2024-09-10T09:04:54.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:54 smithi174 ceph-mon[34068]: pgmap v910: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 266 KiB/s wr, 75 op/s 2024-09-10T09:04:55.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:54 smithi191 ceph-mon[35314]: pgmap v910: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 266 KiB/s wr, 75 op/s 2024-09-10T09:04:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.074+0000 7f6986e1e640 1 -- 172.21.15.172:0/91962205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6980077df0 msgr2=0x7f6980078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.074+0000 7f6986e1e640 1 --2- 172.21.15.172:0/91962205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6980077df0 0x7f6980078250 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f696c009900 tx=0x7f696c02a410 comp rx=0 tx=0).stop 2024-09-10T09:04:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.074+0000 7f6986e1e640 1 -- 172.21.15.172:0/91962205 shutdown_connections 2024-09-10T09:04:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.074+0000 7f6986e1e640 1 --2- 172.21.15.172:0/91962205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6980078790 0x7f6980079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.074+0000 7f6986e1e640 1 --2- 172.21.15.172:0/91962205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6980077df0 0x7f6980078250 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.074+0000 7f6986e1e640 1 --2- 172.21.15.172:0/91962205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6980076bb0 0x7f6980076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.074+0000 7f6986e1e640 1 -- 172.21.15.172:0/91962205 >> 172.21.15.172:0/91962205 conn(0x7f69800ff4b0 msgr2=0x7f69801018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.075+0000 7f6986e1e640 1 -- 172.21.15.172:0/91962205 shutdown_connections 2024-09-10T09:04:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.075+0000 7f6986e1e640 1 -- 172.21.15.172:0/91962205 wait complete. 2024-09-10T09:04:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.075+0000 7f6986e1e640 1 Processor -- start 2024-09-10T09:04:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.076+0000 7f6986e1e640 1 -- start start 2024-09-10T09:04:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.076+0000 7f6986e1e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6980076bb0 0x7f698019e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.076+0000 7f6986e1e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6980077df0 0x7f698019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.076+0000 7f6986e1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6980078790 0x7f69801a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.076+0000 7f6984bba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6980076bb0 0x7f698019e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.076+0000 7f6984bba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6980076bb0 0x7f698019e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:55882/0 (socket says 172.21.15.172:55882) 2024-09-10T09:04:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.076+0000 7f697ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6980077df0 0x7f698019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.076+0000 7f6984bba640 1 -- 172.21.15.172:0/1633728446 learned_addr learned my addr 172.21.15.172:0/1633728446 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.076+0000 7f6986e1e640 1 -- 172.21.15.172:0/1633728446 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6980113ce0 con 0x7f6980078790 2024-09-10T09:04:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.076+0000 7f697ffff640 1 --2- 172.21.15.172:0/1633728446 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6980077df0 0x7f698019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:48294/0 (socket says 172.21.15.172:48294) 2024-09-10T09:04:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.076+0000 7f6986e1e640 1 -- 172.21.15.172:0/1633728446 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6980113b60 con 0x7f6980076bb0 2024-09-10T09:04:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.076+0000 7f6986e1e640 1 -- 172.21.15.172:0/1633728446 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6980113e60 con 0x7f6980077df0 2024-09-10T09:04:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.077+0000 7f69853bb640 1 --2- 172.21.15.172:0/1633728446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6980078790 0x7f69801a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.077+0000 7f6984bba640 1 -- 172.21.15.172:0/1633728446 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6980077df0 msgr2=0x7f698019e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.077+0000 7f6984bba640 1 --2- 172.21.15.172:0/1633728446 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6980077df0 0x7f698019e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.077+0000 7f6984bba640 1 -- 172.21.15.172:0/1633728446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6980078790 msgr2=0x7f69801a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.077+0000 7f6984bba640 1 --2- 172.21.15.172:0/1633728446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6980078790 0x7f69801a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.077+0000 7f6984bba640 1 -- 172.21.15.172:0/1633728446 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69801a5d30 con 0x7f6980076bb0 2024-09-10T09:04:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.077+0000 7f69853bb640 1 --2- 172.21.15.172:0/1633728446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6980078790 0x7f69801a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:04:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.077+0000 7f6984bba640 1 --2- 172.21.15.172:0/1633728446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6980076bb0 0x7f698019e110 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f697400e9f0 tx=0x7f697400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.077+0000 7f697dffb640 1 -- 172.21.15.172:0/1633728446 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697400cd80 con 0x7f6980076bb0 2024-09-10T09:04:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.077+0000 7f697dffb640 1 -- 172.21.15.172:0/1633728446 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69740108e0 con 0x7f6980076bb0 2024-09-10T09:04:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.077+0000 7f697dffb640 1 -- 172.21.15.172:0/1633728446 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974018920 con 0x7f6980076bb0 2024-09-10T09:04:55.080 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.077+0000 7f6986e1e640 1 -- 172.21.15.172:0/1633728446 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69801a5fc0 con 0x7f6980076bb0 2024-09-10T09:04:55.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.078+0000 7f6986e1e640 1 -- 172.21.15.172:0/1633728446 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69801a6430 con 0x7f6980076bb0 2024-09-10T09:04:55.081 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.080+0000 7f697dffb640 1 -- 172.21.15.172:0/1633728446 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6974010a90 con 0x7f6980076bb0 2024-09-10T09:04:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.081+0000 7f69897fd740 1 Processor -- start 2024-09-10T09:04:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.081+0000 7f69897fd740 1 -- start start 2024-09-10T09:04:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.081+0000 7f69897fd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fad9a0 0x557196fadda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.081+0000 7f69897fd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fae2e0 0x557196fb2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.081+0000 7f69897fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fb3370 0x557196fb5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.081+0000 7f69897fd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557196f112f0 con 0x557196fae2e0 2024-09-10T09:04:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.081+0000 7f69897fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557196eb9270 con 0x557196fb3370 2024-09-10T09:04:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.082+0000 7f69853bb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fad9a0 0x557196fadda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.082+0000 7f69853bb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fad9a0 0x557196fadda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:48304/0 (socket says 172.21.15.172:48304) 2024-09-10T09:04:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.082+0000 7f69853bb640 1 -- 172.21.15.172:0/1623625784 learned_addr learned my addr 172.21.15.172:0/1623625784 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.082+0000 7f69897fd740 1 -- 172.21.15.172:0/1623625784 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557196f0d2b0 con 0x557196fad9a0 2024-09-10T09:04:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.082+0000 7f697ffff640 1 --2- 172.21.15.172:0/1623625784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fb3370 0x557196fb5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.082+0000 7f6984bba640 1 --2- 172.21.15.172:0/1623625784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fae2e0 0x557196fb2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.082+0000 7f697ffff640 1 -- 172.21.15.172:0/1623625784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fad9a0 msgr2=0x557196fadda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.082+0000 7f697ffff640 1 --2- 172.21.15.172:0/1623625784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fad9a0 0x557196fadda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.082+0000 7f697ffff640 1 -- 172.21.15.172:0/1623625784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fae2e0 msgr2=0x557196fb2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.082+0000 7f697ffff640 1 --2- 172.21.15.172:0/1623625784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fae2e0 0x557196fb2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.082+0000 7f697ffff640 1 -- 172.21.15.172:0/1623625784 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557196eb8340 con 0x557196fb3370 2024-09-10T09:04:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.082+0000 7f69853bb640 1 --2- 172.21.15.172:0/1623625784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fad9a0 0x557196fadda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:04:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.083+0000 7f697ffff640 1 --2- 172.21.15.172:0/1623625784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fb3370 0x557196fb5830 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f696c009850 tx=0x7f696c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a09f9a549a1e8be0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.083+0000 7f697dffb640 1 --2- 172.21.15.172:0/1633728446 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f69700a2c80 0x7f69700a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.083+0000 7f696affd640 1 -- 172.21.15.172:0/1623625784 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c040070 con 0x557196fb3370 2024-09-10T09:04:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.083+0000 7f696affd640 1 -- 172.21.15.172:0/1623625784 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f696c005500 con 0x557196fb3370 2024-09-10T09:04:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.083+0000 7f696affd640 1 -- 172.21.15.172:0/1623625784 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c035040 con 0x557196fb3370 2024-09-10T09:04:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.083+0000 7f697dffb640 1 -- 172.21.15.172:0/1633728446 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f69740cf780 con 0x7f6980076bb0 2024-09-10T09:04:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.083+0000 7f69853bb640 1 --2- 172.21.15.172:0/1633728446 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f69700a2c80 0x7f69700a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.084+0000 7f69897fd740 1 -- 172.21.15.172:0/1623625784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fb3370 msgr2=0x557196fb5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.084+0000 7f69897fd740 1 --2- 172.21.15.172:0/1623625784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fb3370 0x557196fb5830 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f696c009850 tx=0x7f696c000bc0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.084+0000 7f69853bb640 1 --2- 172.21.15.172:0/1633728446 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f69700a2c80 0x7f69700a5140 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6978005c40 tx=0x7f6978009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.084+0000 7f69897fd740 1 -- 172.21.15.172:0/1623625784 shutdown_connections 2024-09-10T09:04:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.084+0000 7f69897fd740 1 --2- 172.21.15.172:0/1623625784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fb3370 0x557196fb5830 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.084+0000 7f69897fd740 1 --2- 172.21.15.172:0/1623625784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fae2e0 0x557196fb2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.084+0000 7f69897fd740 1 --2- 172.21.15.172:0/1623625784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fad9a0 0x557196fadda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.084+0000 7f69897fd740 1 -- 172.21.15.172:0/1623625784 >> 172.21.15.172:0/1623625784 conn(0x557196fa9010 msgr2=0x557196fa9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.085+0000 7f69897fd740 1 -- 172.21.15.172:0/1623625784 shutdown_connections 2024-09-10T09:04:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.085+0000 7f69897fd740 1 -- 172.21.15.172:0/1623625784 wait complete. 2024-09-10T09:04:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.085+0000 7f69897fd740 1 -- start start 2024-09-10T09:04:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.085+0000 7f69897fd740 1 Processor -- start 2024-09-10T09:04:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.085+0000 7f69897fd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fb7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.085+0000 7f69897fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 0x557196fb8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f69897fd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fb3370 0x557196fbf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f6984bba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fb7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f6984bba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fb7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40238/0 (socket says 172.21.15.172:40238) 2024-09-10T09:04:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f697ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 0x557196fb8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f697ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 0x557196fb8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:55904/0 (socket says 172.21.15.172:55904) 2024-09-10T09:04:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f6984bba640 1 -- 172.21.15.172:0/2811922810 learned_addr learned my addr 172.21.15.172:0/2811922810 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f69897fd740 1 -- 172.21.15.172:0/2811922810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557196fb62e0 con 0x557196fad9a0 2024-09-10T09:04:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f69897fd740 1 -- 172.21.15.172:0/2811922810 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557196e9a290 con 0x557196fae2e0 2024-09-10T09:04:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f69897fd740 1 -- 172.21.15.172:0/2811922810 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557196fb6460 con 0x557196fb3370 2024-09-10T09:04:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f69853bb640 1 --2- 172.21.15.172:0/2811922810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fb3370 0x557196fbf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f6984bba640 1 -- 172.21.15.172:0/2811922810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fb3370 msgr2=0x557196fbf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f6984bba640 1 --2- 172.21.15.172:0/2811922810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fb3370 0x557196fbf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f6984bba640 1 -- 172.21.15.172:0/2811922810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 msgr2=0x557196fb8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f6984bba640 1 --2- 172.21.15.172:0/2811922810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 0x557196fb8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f6984bba640 1 -- 172.21.15.172:0/2811922810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557196c24210 con 0x557196fad9a0 2024-09-10T09:04:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f697ffff640 1 --2- 172.21.15.172:0/2811922810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 0x557196fb8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:04:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f69897fd740 10 client.0 _ll_register_callbacks cb 0x557196ecb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f69897fd740 1 -- 172.21.15.172:0/2811922810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f696c005500 con 0x557196fad9a0 2024-09-10T09:04:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f6984bba640 1 --2- 172.21.15.172:0/2811922810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fb7b40 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f697409bbb0 tx=0x7f697409b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5da41d040c77f7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.086+0000 7f695ffff640 10 client.19914 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:55.094 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.087+0000 7f695ffff640 1 -- 172.21.15.172:0/2811922810 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697409ffd0 con 0x557196fad9a0 2024-09-10T09:04:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.087+0000 7f695ffff640 1 -- 172.21.15.172:0/2811922810 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f69740cfb70 con 0x557196fad9a0 2024-09-10T09:04:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.087+0000 7f695ffff640 1 -- 172.21.15.172:0/2811922810 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f69740cfd90 con 0x557196fad9a0 2024-09-10T09:04:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.087+0000 7f69897fd740 10 client.19914 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.087+0000 7f69897fd740 1 -- 172.21.15.172:0/2811922810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=44}) -- 0x557196cf0620 con 0x557196fad9a0 2024-09-10T09:04:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.087+0000 7f695ffff640 1 -- 172.21.15.172:0/2811922810 <== mon.0 v2:172.21.15.172:3300/0 4 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f6974010430 con 0x557196fad9a0 2024-09-10T09:04:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.087+0000 7f69897fd740 10 client.19914 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.087+0000 7f69897fd740 10 client.19914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:04:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.087+0000 7f69897fd740 20 client.19914 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.087+0000 7f69897fd740 1 --2- 172.21.15.172:0/2811922810 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557196fc3610 0x557196fc5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.087+0000 7f69897fd740 4 client.19914 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:04:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.087+0000 7f69897fd740 1 -- 172.21.15.172:0/2811922810 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557196f1ea20 con 0x557196fc3610 2024-09-10T09:04:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.087+0000 7f697ffff640 1 --2- 172.21.15.172:0/2811922810 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557196fc3610 0x557196fc5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.088+0000 7f697ffff640 1 --2- 172.21.15.172:0/2811922810 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557196fc3610 0x557196fc5ab0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=155e75ed03a6b514 server_cookie=e189b429a90309fa in_seq=0 out_seq=0 2024-09-10T09:04:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.088+0000 7f695ffff640 10 client.19914 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.088+0000 7f695ffff640 1 -- 172.21.15.172:0/2811922810 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557196f1ea20 con 0x557196fc3610 2024-09-10T09:04:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.088+0000 7f695ffff640 10 client.19914 handle_command_reply: tid=0 2024-09-10T09:04:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.088+0000 7f69897fd740 1 client.19914 shutdown 2024-09-10T09:04:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.089+0000 7f69897fd740 1 -- 172.21.15.172:0/2811922810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 msgr2=0x557196fb7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.089+0000 7f69897fd740 1 --2- 172.21.15.172:0/2811922810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fb7b40 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f697409bbb0 tx=0x7f697409b640 comp rx=0 tx=0).stop 2024-09-10T09:04:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.089+0000 7f69897fd740 1 -- 172.21.15.172:0/2811922810 shutdown_connections 2024-09-10T09:04:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.089+0000 7f69897fd740 1 --2- 172.21.15.172:0/2811922810 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557196fc3610 0x557196fc5ab0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.089+0000 7f695ffff640 0 client.19914 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.089+0000 7f69897fd740 1 --2- 172.21.15.172:0/2811922810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fb3370 0x557196fbf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.089+0000 7f69897fd740 1 --2- 172.21.15.172:0/2811922810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 0x557196fb8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.089+0000 7f69897fd740 1 --2- 172.21.15.172:0/2811922810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fb7b40 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.089+0000 7f69897fd740 1 -- 172.21.15.172:0/2811922810 >> 172.21.15.172:0/2811922810 conn(0x557196fa9010 msgr2=0x557196faa820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.089+0000 7f69897fd740 1 -- 172.21.15.172:0/2811922810 shutdown_connections 2024-09-10T09:04:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.089+0000 7f69897fd740 1 -- 172.21.15.172:0/2811922810 wait complete. 2024-09-10T09:04:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.089+0000 7f69897fd740 20 client.19914 trim_cache size 0 max 16384 2024-09-10T09:04:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.104+0000 7f69897fd740 1 Processor -- start 2024-09-10T09:04:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.104+0000 7f69897fd740 1 -- start start 2024-09-10T09:04:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.104+0000 7f69897fd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fb70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.104+0000 7f69897fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 0x557196fbf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.104+0000 7f69897fd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fb3370 0x557196fb8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.104+0000 7f69897fd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b0b70 con 0x557196fad9a0 2024-09-10T09:04:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.104+0000 7f69897fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b26b0 con 0x557196fae2e0 2024-09-10T09:04:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.104+0000 7f69897fd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b1240 con 0x557196fb3370 2024-09-10T09:04:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.104+0000 7f6984bba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fb70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.104+0000 7f6984bba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fb70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40250/0 (socket says 172.21.15.172:40250) 2024-09-10T09:04:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.104+0000 7f6984bba640 1 -- 172.21.15.172:0/2752129112 learned_addr learned my addr 172.21.15.172:0/2752129112 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.105+0000 7f697ffff640 1 --2- 172.21.15.172:0/2752129112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 0x557196fbf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.105+0000 7f6984bba640 1 -- 172.21.15.172:0/2752129112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fb3370 msgr2=0x557196fb8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.105+0000 7f69853bb640 1 --2- 172.21.15.172:0/2752129112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fb3370 0x557196fb8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.105+0000 7f6984bba640 1 --2- 172.21.15.172:0/2752129112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fb3370 0x557196fb8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.105+0000 7f6984bba640 1 -- 172.21.15.172:0/2752129112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 msgr2=0x557196fbf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.105+0000 7f6984bba640 1 --2- 172.21.15.172:0/2752129112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 0x557196fbf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.105+0000 7f6984bba640 1 -- 172.21.15.172:0/2752129112 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557196fa8e30 con 0x557196fad9a0 2024-09-10T09:04:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.105+0000 7f697ffff640 1 --2- 172.21.15.172:0/2752129112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 0x557196fbf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:04:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.105+0000 7f6984bba640 1 --2- 172.21.15.172:0/2752129112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fb70a0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f697400c6a0 tx=0x7f69740d1000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5ede789d449bcf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.105+0000 7f696a7fc640 1 -- 172.21.15.172:0/2752129112 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69740a4890 con 0x557196fad9a0 2024-09-10T09:04:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.105+0000 7f696a7fc640 1 -- 172.21.15.172:0/2752129112 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f696c005500 con 0x557196fad9a0 2024-09-10T09:04:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.105+0000 7f696a7fc640 1 -- 172.21.15.172:0/2752129112 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69740cfa00 con 0x557196fad9a0 2024-09-10T09:04:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.106+0000 7f69897fd740 1 -- 172.21.15.172:0/2752129112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 msgr2=0x557196fb70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:55.109 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.106+0000 7f69897fd740 1 --2- 172.21.15.172:0/2752129112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fb70a0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f697400c6a0 tx=0x7f69740d1000 comp rx=0 tx=0).stop 2024-09-10T09:04:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.106+0000 7f69897fd740 1 -- 172.21.15.172:0/2752129112 shutdown_connections 2024-09-10T09:04:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.106+0000 7f69897fd740 1 --2- 172.21.15.172:0/2752129112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fb3370 0x557196fb8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.106+0000 7f69897fd740 1 --2- 172.21.15.172:0/2752129112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 0x557196fbf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.106+0000 7f69897fd740 1 --2- 172.21.15.172:0/2752129112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fb70a0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.106+0000 7f69897fd740 1 -- 172.21.15.172:0/2752129112 >> 172.21.15.172:0/2752129112 conn(0x557196fa9010 msgr2=0x557196fb78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.106+0000 7f69897fd740 1 -- 172.21.15.172:0/2752129112 shutdown_connections 2024-09-10T09:04:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.106+0000 7f69897fd740 1 -- 172.21.15.172:0/2752129112 wait complete. 2024-09-10T09:04:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.106+0000 7f69897fd740 1 -- start start 2024-09-10T09:04:55.110 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f69897fd740 1 Processor -- start 2024-09-10T09:04:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f69897fd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fbc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f69897fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 0x557196fbc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f69897fd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fb3370 0x557196fbd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f69897fd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b1fe0 con 0x557196fad9a0 2024-09-10T09:04:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f69897fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f69801b1910 con 0x557196fae2e0 2024-09-10T09:04:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f69897fd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f69801aeae0 con 0x557196fb3370 2024-09-10T09:04:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f69897fd740 10 client.0 _ll_register_callbacks cb 0x7f6974009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f6984bba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fbc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f6984bba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fbc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40262/0 (socket says 172.21.15.172:40262) 2024-09-10T09:04:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f6984bba640 1 -- 172.21.15.172:0/2803337043 learned_addr learned my addr 172.21.15.172:0/2803337043 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f697ffff640 1 --2- 172.21.15.172:0/2803337043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 0x557196fbc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f69853bb640 1 --2- 172.21.15.172:0/2803337043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fb3370 0x557196fbd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f6984bba640 1 -- 172.21.15.172:0/2803337043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fb3370 msgr2=0x557196fbd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f6984bba640 1 --2- 172.21.15.172:0/2803337043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fb3370 0x557196fbd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f6984bba640 1 -- 172.21.15.172:0/2803337043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 msgr2=0x557196fbc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f6984bba640 1 --2- 172.21.15.172:0/2803337043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 0x557196fbc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f6984bba640 1 -- 172.21.15.172:0/2803337043 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557196fbdee0 con 0x557196fad9a0 2024-09-10T09:04:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f6984bba640 1 --2- 172.21.15.172:0/2803337043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fbc3d0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f69740985f0 tx=0x7f6974010840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d348b4a0c77602c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.107+0000 7f695ffff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.108+0000 7f695ffff640 1 -- 172.21.15.172:0/2803337043 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697409b760 con 0x557196fad9a0 2024-09-10T09:04:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.108+0000 7f695ffff640 1 -- 172.21.15.172:0/2803337043 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f697409b270 con 0x557196fad9a0 2024-09-10T09:04:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.108+0000 7f69897fd740 1 -- 172.21.15.172:0/2803337043 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f696c005500 con 0x557196fad9a0 2024-09-10T09:04:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.108+0000 7f695ffff640 1 -- 172.21.15.172:0/2803337043 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6974097970 con 0x557196fad9a0 2024-09-10T09:04:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.108+0000 7f69897fd740 10 client.19920 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.108+0000 7f69897fd740 1 -- 172.21.15.172:0/2803337043 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557196fbf1f0 con 0x557196fad9a0 2024-09-10T09:04:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.108+0000 7f695ffff640 1 -- 172.21.15.172:0/2803337043 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f696c005500 con 0x557196fad9a0 2024-09-10T09:04:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.108+0000 7f695ffff640 1 -- 172.21.15.172:0/2803337043 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f697409aae0 con 0x557196fad9a0 2024-09-10T09:04:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.108+0000 7f69897fd740 10 client.19920 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.108+0000 7f69897fd740 10 client.19920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:04:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.108+0000 7f69897fd740 20 client.19920 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.108+0000 7f69897fd740 1 --2- 172.21.15.172:0/2803337043 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557196fc3610 0x557197030730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.108+0000 7f69897fd740 4 client.19920 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:04:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.108+0000 7f69897fd740 1 -- 172.21.15.172:0/2803337043 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557196f18d70 con 0x557196fc3610 2024-09-10T09:04:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.109+0000 7f697ffff640 1 --2- 172.21.15.172:0/2803337043 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557196fc3610 0x557197030730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.109+0000 7f695ffff640 10 client.19920 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.109+0000 7f697ffff640 1 --2- 172.21.15.172:0/2803337043 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557196fc3610 0x557197030730 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ad38f0a0359d40f server_cookie=cacd3a6424376d6b in_seq=0 out_seq=0 2024-09-10T09:04:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.112+0000 7f695ffff640 1 -- 172.21.15.172:0/2803337043 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557196f18d70 con 0x557196fc3610 2024-09-10T09:04:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.112+0000 7f695ffff640 10 client.19920 handle_command_reply: tid=0 2024-09-10T09:04:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.112+0000 7f69897fd740 1 client.19920 shutdown 2024-09-10T09:04:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.113+0000 7f69897fd740 1 -- 172.21.15.172:0/2803337043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 msgr2=0x557196fbc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.113+0000 7f69897fd740 1 --2- 172.21.15.172:0/2803337043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fbc3d0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f69740985f0 tx=0x7f6974010840 comp rx=0 tx=0).stop 2024-09-10T09:04:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.113+0000 7f69897fd740 1 -- 172.21.15.172:0/2803337043 shutdown_connections 2024-09-10T09:04:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.113+0000 7f69897fd740 1 --2- 172.21.15.172:0/2803337043 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x557196fc3610 0x557197030730 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.113+0000 7f69897fd740 1 --2- 172.21.15.172:0/2803337043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557196fb3370 0x557196fbd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.113+0000 7f695ffff640 0 client.19920 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.113+0000 7f69897fd740 1 --2- 172.21.15.172:0/2803337043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557196fae2e0 0x557196fbc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.113+0000 7f69897fd740 1 --2- 172.21.15.172:0/2803337043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557196fad9a0 0x557196fbc3d0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.113+0000 7f69897fd740 1 -- 172.21.15.172:0/2803337043 >> 172.21.15.172:0/2803337043 conn(0x557196fa9010 msgr2=0x557196fa79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.114+0000 7f69897fd740 1 -- 172.21.15.172:0/2803337043 shutdown_connections 2024-09-10T09:04:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.114+0000 7f69897fd740 1 -- 172.21.15.172:0/2803337043 wait complete. 2024-09-10T09:04:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.114+0000 7f69897fd740 20 client.19920 trim_cache size 0 max 16384 2024-09-10T09:04:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:04:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:04:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:04:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:04:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.116+0000 7f6986e1e640 1 -- 172.21.15.172:0/1633728446 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f69700a2c80 msgr2=0x7f69700a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.116+0000 7f6986e1e640 1 --2- 172.21.15.172:0/1633728446 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f69700a2c80 0x7f69700a5140 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6978005c40 tx=0x7f6978009500 comp rx=0 tx=0).stop 2024-09-10T09:04:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.116+0000 7f6986e1e640 1 -- 172.21.15.172:0/1633728446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6980076bb0 msgr2=0x7f698019e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.116+0000 7f6986e1e640 1 --2- 172.21.15.172:0/1633728446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6980076bb0 0x7f698019e110 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f697400e9f0 tx=0x7f697400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.116+0000 7f6986e1e640 1 -- 172.21.15.172:0/1633728446 shutdown_connections 2024-09-10T09:04:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.116+0000 7f6986e1e640 1 --2- 172.21.15.172:0/1633728446 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f69700a2c80 0x7f69700a5140 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.116+0000 7f6986e1e640 1 --2- 172.21.15.172:0/1633728446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6980078790 0x7f69801a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.117+0000 7f6986e1e640 1 --2- 172.21.15.172:0/1633728446 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6980077df0 0x7f698019e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.117+0000 7f6986e1e640 1 --2- 172.21.15.172:0/1633728446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6980076bb0 0x7f698019e110 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.117+0000 7f6986e1e640 1 -- 172.21.15.172:0/1633728446 >> 172.21.15.172:0/1633728446 conn(0x7f69800ff4b0 msgr2=0x7f6980100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.118+0000 7f6986e1e640 1 -- 172.21.15.172:0/1633728446 shutdown_connections 2024-09-10T09:04:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:04:55.118+0000 7f6986e1e640 1 -- 172.21.15.172:0/1633728446 wait complete. 2024-09-10T09:04:56.636 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:04:56.842 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:56 smithi172 ceph-mon[30854]: pgmap v911: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 413 KiB/s wr, 114 op/s 2024-09-10T09:04:56.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.842+0000 7f9bbee9e640 1 -- 172.21.15.172:0/2835026264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bb8102790 msgr2=0x7f9bb81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:56.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.842+0000 7f9bbee9e640 1 --2- 172.21.15.172:0/2835026264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bb8102790 0x7f9bb81067e0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009900 tx=0x7f9ba402a410 comp rx=0 tx=0).stop 2024-09-10T09:04:56.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.842+0000 7f9bbee9e640 1 -- 172.21.15.172:0/2835026264 shutdown_connections 2024-09-10T09:04:56.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.842+0000 7f9bbee9e640 1 --2- 172.21.15.172:0/2835026264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bb8106f10 0x7f9bb810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.842+0000 7f9bbee9e640 1 --2- 172.21.15.172:0/2835026264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bb8102790 0x7f9bb81067e0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.842+0000 7f9bbee9e640 1 --2- 172.21.15.172:0/2835026264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bb8101de0 0x7f9bb81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.843+0000 7f9bbee9e640 1 -- 172.21.15.172:0/2835026264 >> 172.21.15.172:0/2835026264 conn(0x7f9bb80fd880 msgr2=0x7f9bb80ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:56.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.843+0000 7f9bbee9e640 1 -- 172.21.15.172:0/2835026264 shutdown_connections 2024-09-10T09:04:56.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.843+0000 7f9bbee9e640 1 -- 172.21.15.172:0/2835026264 wait complete. 2024-09-10T09:04:56.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.844+0000 7f9bbee9e640 1 Processor -- start 2024-09-10T09:04:56.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.844+0000 7f9bbee9e640 1 -- start start 2024-09-10T09:04:56.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.844+0000 7f9bbee9e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bb8101de0 0x7f9bb819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.844+0000 7f9bbee9e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bb8102790 0x7f9bb819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.844+0000 7f9bbee9e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bb8106f10 0x7f9bb81a3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.844+0000 7f9bbee9e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb8118440 con 0x7f9bb8106f10 2024-09-10T09:04:56.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.844+0000 7f9bbee9e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb81182c0 con 0x7f9bb8101de0 2024-09-10T09:04:56.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.844+0000 7f9bbee9e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb81185c0 con 0x7f9bb8102790 2024-09-10T09:04:56.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.844+0000 7f9bb7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bb8102790 0x7f9bb819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:56.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.844+0000 7f9bb7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bb8102790 0x7f9bb819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:37266/0 (socket says 172.21.15.172:37266) 2024-09-10T09:04:56.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.844+0000 7f9bb7fff640 1 -- 172.21.15.172:0/182639425 learned_addr learned my addr 172.21.15.172:0/182639425 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:56.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.845+0000 7f9bbcc3a640 1 --2- 172.21.15.172:0/182639425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bb8101de0 0x7f9bb819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:56.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.845+0000 7f9bb7fff640 1 -- 172.21.15.172:0/182639425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bb8101de0 msgr2=0x7f9bb819ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:56.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.845+0000 7f9bb7fff640 1 --2- 172.21.15.172:0/182639425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bb8101de0 0x7f9bb819ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.845+0000 7f9bb7fff640 1 -- 172.21.15.172:0/182639425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bb8106f10 msgr2=0x7f9bb81a3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:56.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.845+0000 7f9bb7fff640 1 --2- 172.21.15.172:0/182639425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bb8106f10 0x7f9bb81a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.845+0000 7f9bb7fff640 1 -- 172.21.15.172:0/182639425 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb81a3ab0 con 0x7f9bb8102790 2024-09-10T09:04:56.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.846+0000 7f9bb7fff640 1 --2- 172.21.15.172:0/182639425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bb8102790 0x7f9bb819efc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4005ab0 tx=0x7f9ba4003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:56.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.846+0000 7f9bb5ffb640 1 -- 172.21.15.172:0/182639425 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba403f070 con 0x7f9bb8102790 2024-09-10T09:04:56.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.846+0000 7f9bb5ffb640 1 -- 172.21.15.172:0/182639425 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ba4031b50 con 0x7f9bb8102790 2024-09-10T09:04:56.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.846+0000 7f9bb5ffb640 1 -- 172.21.15.172:0/182639425 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4035030 con 0x7f9bb8102790 2024-09-10T09:04:56.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.847+0000 7f9bbee9e640 1 -- 172.21.15.172:0/182639425 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb81a3d40 con 0x7f9bb8102790 2024-09-10T09:04:56.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.847+0000 7f9bbee9e640 1 -- 172.21.15.172:0/182639425 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb81a4110 con 0x7f9bb8102790 2024-09-10T09:04:56.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.850+0000 7f9bb5ffb640 1 -- 172.21.15.172:0/182639425 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9ba4042070 con 0x7f9bb8102790 2024-09-10T09:04:56.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.850+0000 7f9bc1277740 1 Processor -- start 2024-09-10T09:04:56.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.850+0000 7f9bc1277740 1 -- start start 2024-09-10T09:04:56.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.850+0000 7f9bc1277740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b669850 0x55c93b669c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.850+0000 7f9bc1277740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b66a170 0x55c93b6744f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.850+0000 7f9bc1277740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b674a30 0x55c93b676e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.850+0000 7f9bc1277740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c93b5ce160 con 0x55c93b669850 2024-09-10T09:04:56.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.850+0000 7f9bc1277740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c93b575720 con 0x55c93b66a170 2024-09-10T09:04:56.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.850+0000 7f9bc1277740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c93b5ca120 con 0x55c93b674a30 2024-09-10T09:04:56.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.851+0000 7f9bbd43b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b669850 0x55c93b669c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:56.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.851+0000 7f9bbd43b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b669850 0x55c93b669c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60106/0 (socket says 172.21.15.172:60106) 2024-09-10T09:04:56.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.851+0000 7f9bbd43b640 1 -- 172.21.15.172:0/3444100166 learned_addr learned my addr 172.21.15.172:0/3444100166 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:56.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.851+0000 7f9bb7fff640 1 --2- 172.21.15.172:0/3444100166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b674a30 0x55c93b676e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:56.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.851+0000 7f9bbd43b640 1 -- 172.21.15.172:0/3444100166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b674a30 msgr2=0x55c93b676e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:56.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.851+0000 7f9bbd43b640 1 --2- 172.21.15.172:0/3444100166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b674a30 0x55c93b676e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.851+0000 7f9bbd43b640 1 -- 172.21.15.172:0/3444100166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b66a170 msgr2=0x55c93b6744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:56.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.852+0000 7f9bbcc3a640 1 --2- 172.21.15.172:0/3444100166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b66a170 0x55c93b6744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:56.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.852+0000 7f9bbd43b640 1 --2- 172.21.15.172:0/3444100166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b66a170 0x55c93b6744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.854 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.852+0000 7f9bbd43b640 1 -- 172.21.15.172:0/3444100166 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c93b33fba0 con 0x55c93b669850 2024-09-10T09:04:56.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.852+0000 7f9bb5ffb640 1 --2- 172.21.15.172:0/182639425 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9ba80a2d90 0x7f9ba80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.853+0000 7f9bbcc3a640 1 --2- 172.21.15.172:0/3444100166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b66a170 0x55c93b6744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:04:56.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.853+0000 7f9bb5ffb640 1 -- 172.21.15.172:0/182639425 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9ba40efb60 con 0x7f9bb8102790 2024-09-10T09:04:56.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.853+0000 7f9bbd43b640 1 --2- 172.21.15.172:0/3444100166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b669850 0x55c93b669c30 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000b580 tx=0x7f9bb000ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c8e7a14e9b03a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:56.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.853+0000 7f9ba2ffd640 1 -- 172.21.15.172:0/3444100166 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0003970 con 0x55c93b669850 2024-09-10T09:04:56.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.853+0000 7f9ba2ffd640 1 -- 172.21.15.172:0/3444100166 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bb0004020 con 0x55c93b669850 2024-09-10T09:04:56.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.853+0000 7f9ba2ffd640 1 -- 172.21.15.172:0/3444100166 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0017780 con 0x55c93b669850 2024-09-10T09:04:56.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.854+0000 7f9bbcc3a640 1 --2- 172.21.15.172:0/182639425 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9ba80a2d90 0x7f9ba80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:56.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.855+0000 7f9bc1277740 1 -- 172.21.15.172:0/3444100166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b669850 msgr2=0x55c93b669c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:56.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.855+0000 7f9bc1277740 1 --2- 172.21.15.172:0/3444100166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b669850 0x55c93b669c30 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000b580 tx=0x7f9bb000ba40 comp rx=0 tx=0).stop 2024-09-10T09:04:56.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.855+0000 7f9bbcc3a640 1 --2- 172.21.15.172:0/182639425 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9ba80a2d90 0x7f9ba80a5250 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x55c93b666c40 tx=0x7f9bac005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:56.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.855+0000 7f9bc1277740 1 -- 172.21.15.172:0/3444100166 shutdown_connections 2024-09-10T09:04:56.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.855+0000 7f9bc1277740 1 --2- 172.21.15.172:0/3444100166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b674a30 0x55c93b676e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.855+0000 7f9bc1277740 1 --2- 172.21.15.172:0/3444100166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b66a170 0x55c93b6744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.855+0000 7f9bc1277740 1 --2- 172.21.15.172:0/3444100166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b669850 0x55c93b669c30 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.855+0000 7f9bc1277740 1 -- 172.21.15.172:0/3444100166 >> 172.21.15.172:0/3444100166 conn(0x55c93b665de0 msgr2=0x55c93b6661c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:56.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.855+0000 7f9bc1277740 1 -- 172.21.15.172:0/3444100166 shutdown_connections 2024-09-10T09:04:56.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.855+0000 7f9bc1277740 1 -- 172.21.15.172:0/3444100166 wait complete. 2024-09-10T09:04:56.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.855+0000 7f9bc1277740 1 -- start start 2024-09-10T09:04:56.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.856+0000 7f9bc1277740 1 Processor -- start 2024-09-10T09:04:56.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.856+0000 7f9bc1277740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b669850 0x55c93b671970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.856+0000 7f9bc1277740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b66a170 0x55c93b671eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.856+0000 7f9bc1277740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b674a30 0x55c93b6723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.856+0000 7f9bc1277740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c93b677600 con 0x55c93b674a30 2024-09-10T09:04:56.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.856+0000 7f9bb7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b66a170 0x55c93b671eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:56.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.856+0000 7f9bbd43b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b674a30 0x55c93b6723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:56.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.856+0000 7f9bb7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b66a170 0x55c93b671eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:53276/0 (socket says 172.21.15.172:53276) 2024-09-10T09:04:56.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.856+0000 7f9bb7fff640 1 -- 172.21.15.172:0/1197339925 learned_addr learned my addr 172.21.15.172:0/1197339925 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:56.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.856+0000 7f9bc1277740 1 -- 172.21.15.172:0/1197339925 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c93b556900 con 0x55c93b66a170 2024-09-10T09:04:56.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.856+0000 7f9bbcc3a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b669850 0x55c93b671970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:56.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.856+0000 7f9bc1277740 1 -- 172.21.15.172:0/1197339925 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c93b677780 con 0x55c93b669850 2024-09-10T09:04:56.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.856+0000 7f9bc1277740 10 client.0 _ll_register_callbacks cb 0x55c93b554780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:56.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.857+0000 7f9bbcc3a640 1 -- 172.21.15.172:0/1197339925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b66a170 msgr2=0x55c93b671eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:56.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.857+0000 7f9bbcc3a640 1 --2- 172.21.15.172:0/1197339925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b66a170 0x55c93b671eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.857+0000 7f9bbcc3a640 1 -- 172.21.15.172:0/1197339925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b674a30 msgr2=0x55c93b6723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:56.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.857+0000 7f9bbcc3a640 1 --2- 172.21.15.172:0/1197339925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b674a30 0x55c93b6723f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.857+0000 7f9bbcc3a640 1 -- 172.21.15.172:0/1197339925 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c93b596c50 con 0x55c93b669850 2024-09-10T09:04:56.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.857+0000 7f9bbd43b640 1 --2- 172.21.15.172:0/1197339925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b674a30 0x55c93b6723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:04:56.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.857+0000 7f9bbcc3a640 1 --2- 172.21.15.172:0/1197339925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b669850 0x55c93b671970 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00ba40 tx=0x7f9bac004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49c61161ae273cf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:56.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.857+0000 7f9b97fff640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:04:56.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.857+0000 7f9b97fff640 1 -- 172.21.15.172:0/1197339925 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac023070 con 0x55c93b669850 2024-09-10T09:04:56.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.857+0000 7f9b97fff640 1 -- 172.21.15.172:0/1197339925 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9bac01f860 con 0x55c93b669850 2024-09-10T09:04:56.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.857+0000 7f9bc1277740 1 -- 172.21.15.172:0/1197339925 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bb0004020 con 0x55c93b669850 2024-09-10T09:04:56.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.857+0000 7f9b97fff640 1 -- 172.21.15.172:0/1197339925 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9bac02f050 con 0x55c93b669850 2024-09-10T09:04:56.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.858+0000 7f9bc1277740 10 client.28585 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:56.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.858+0000 7f9bc1277740 1 -- 172.21.15.172:0/1197339925 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c93b6832d0 con 0x55c93b669850 2024-09-10T09:04:56.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.858+0000 7f9b97fff640 1 -- 172.21.15.172:0/1197339925 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bb0004020 con 0x55c93b669850 2024-09-10T09:04:56.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.858+0000 7f9b97fff640 1 -- 172.21.15.172:0/1197339925 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f9bac00f040 con 0x55c93b669850 2024-09-10T09:04:56.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.858+0000 7f9bc1277740 10 client.28585 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:56.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.858+0000 7f9bc1277740 10 client.28585 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:04:56.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.858+0000 7f9bc1277740 20 client.28585 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:56.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.858+0000 7f9bc1277740 1 --2- 172.21.15.172:0/1197339925 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55c93b6846e0 0x55c93b686af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.858+0000 7f9bc1277740 4 client.28585 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:04:56.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.858+0000 7f9bc1277740 1 -- 172.21.15.172:0/1197339925 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c93b5db890 con 0x55c93b6846e0 2024-09-10T09:04:56.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.861+0000 7f9bb7fff640 1 --2- 172.21.15.172:0/1197339925 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55c93b6846e0 0x55c93b686af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:56.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.862+0000 7f9b97fff640 10 client.28585 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:56.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.862+0000 7f9bb7fff640 1 --2- 172.21.15.172:0/1197339925 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55c93b6846e0 0x55c93b686af0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1179f3e4f50f02c6 server_cookie=c580fe0f994eba0f in_seq=0 out_seq=0 2024-09-10T09:04:56.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.863+0000 7f9b97fff640 1 -- 172.21.15.172:0/1197339925 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c93b5db890 con 0x55c93b6846e0 2024-09-10T09:04:56.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.863+0000 7f9b97fff640 10 client.28585 handle_command_reply: tid=0 2024-09-10T09:04:56.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.863+0000 7f9bc1277740 1 client.28585 shutdown 2024-09-10T09:04:56.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.863+0000 7f9bc1277740 1 -- 172.21.15.172:0/1197339925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b669850 msgr2=0x55c93b671970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:56.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.863+0000 7f9bc1277740 1 --2- 172.21.15.172:0/1197339925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b669850 0x55c93b671970 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00ba40 tx=0x7f9bac004730 comp rx=0 tx=0).stop 2024-09-10T09:04:56.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.863+0000 7f9bc1277740 1 -- 172.21.15.172:0/1197339925 shutdown_connections 2024-09-10T09:04:56.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.863+0000 7f9bc1277740 1 --2- 172.21.15.172:0/1197339925 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55c93b6846e0 0x55c93b686af0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.863+0000 7f9bc1277740 1 --2- 172.21.15.172:0/1197339925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b674a30 0x55c93b6723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.863+0000 7f9bc1277740 1 --2- 172.21.15.172:0/1197339925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b66a170 0x55c93b671eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.863+0000 7f9bc1277740 1 --2- 172.21.15.172:0/1197339925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b669850 0x55c93b671970 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.863+0000 7f9bc1277740 1 -- 172.21.15.172:0/1197339925 >> 172.21.15.172:0/1197339925 conn(0x55c93b665de0 msgr2=0x55c93b66bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:56.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.863+0000 7f9b97fff640 0 client.28585 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:56.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.864+0000 7f9bc1277740 1 -- 172.21.15.172:0/1197339925 shutdown_connections 2024-09-10T09:04:56.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.864+0000 7f9bc1277740 1 -- 172.21.15.172:0/1197339925 wait complete. 2024-09-10T09:04:56.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.864+0000 7f9bc1277740 20 client.28585 trim_cache size 0 max 16384 2024-09-10T09:04:56.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.891+0000 7f9bc1277740 1 Processor -- start 2024-09-10T09:04:56.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.891+0000 7f9bc1277740 1 -- start start 2024-09-10T09:04:56.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.891+0000 7f9bc1277740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b669850 0x55c93b66c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.891+0000 7f9bc1277740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b66a170 0x55c93b68c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.891+0000 7f9bc1277740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b674a30 0x55c93b68c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.891+0000 7f9bc1277740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb81b50d0 con 0x55c93b66a170 2024-09-10T09:04:56.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.891+0000 7f9bc1277740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb81b6c10 con 0x55c93b669850 2024-09-10T09:04:56.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.891+0000 7f9bc1277740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb81b57a0 con 0x55c93b674a30 2024-09-10T09:04:56.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.892+0000 7f9bbcc3a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b669850 0x55c93b66c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:56.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.892+0000 7f9bbd43b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b674a30 0x55c93b68c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:56.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.892+0000 7f9bb7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b66a170 0x55c93b68c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:56.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.892+0000 7f9bbd43b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b674a30 0x55c93b68c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:37286/0 (socket says 172.21.15.172:37286) 2024-09-10T09:04:56.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.892+0000 7f9bbd43b640 1 -- 172.21.15.172:0/2637383538 learned_addr learned my addr 172.21.15.172:0/2637383538 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:56.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.892+0000 7f9bbd43b640 1 -- 172.21.15.172:0/2637383538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b669850 msgr2=0x55c93b66c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:56.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.892+0000 7f9bbd43b640 1 --2- 172.21.15.172:0/2637383538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b669850 0x55c93b66c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.892+0000 7f9bbd43b640 1 -- 172.21.15.172:0/2637383538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b66a170 msgr2=0x55c93b68c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:56.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.892+0000 7f9bbd43b640 1 --2- 172.21.15.172:0/2637383538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b66a170 0x55c93b68c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.892+0000 7f9bbd43b640 1 -- 172.21.15.172:0/2637383538 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c93b666e90 con 0x55c93b674a30 2024-09-10T09:04:56.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.892+0000 7f9bb7fff640 1 --2- 172.21.15.172:0/2637383538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b66a170 0x55c93b68c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:04:56.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.892+0000 7f9bbd43b640 1 --2- 172.21.15.172:0/2637383538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b674a30 0x55c93b68c900 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0018000 tx=0x7f9bb0017440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d115e230a32e7a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:56.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.893+0000 7f9ba27fc640 1 -- 172.21.15.172:0/2637383538 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0026070 con 0x55c93b674a30 2024-09-10T09:04:56.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.893+0000 7f9ba27fc640 1 -- 172.21.15.172:0/2637383538 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bb000d460 con 0x55c93b674a30 2024-09-10T09:04:56.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.893+0000 7f9ba27fc640 1 -- 172.21.15.172:0/2637383538 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0003e00 con 0x55c93b674a30 2024-09-10T09:04:56.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.894+0000 7f9bc1277740 1 -- 172.21.15.172:0/2637383538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b674a30 msgr2=0x55c93b68c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:56.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.894+0000 7f9bc1277740 1 --2- 172.21.15.172:0/2637383538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b674a30 0x55c93b68c900 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0018000 tx=0x7f9bb0017440 comp rx=0 tx=0).stop 2024-09-10T09:04:56.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.894+0000 7f9bc1277740 1 -- 172.21.15.172:0/2637383538 shutdown_connections 2024-09-10T09:04:56.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.894+0000 7f9bc1277740 1 --2- 172.21.15.172:0/2637383538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b674a30 0x55c93b68c900 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.894+0000 7f9bc1277740 1 --2- 172.21.15.172:0/2637383538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b66a170 0x55c93b68c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.894+0000 7f9bc1277740 1 --2- 172.21.15.172:0/2637383538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b669850 0x55c93b66c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.894+0000 7f9bc1277740 1 -- 172.21.15.172:0/2637383538 >> 172.21.15.172:0/2637383538 conn(0x55c93b665de0 msgr2=0x55c93b6739f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:56.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.894+0000 7f9bc1277740 1 -- 172.21.15.172:0/2637383538 shutdown_connections 2024-09-10T09:04:56.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.894+0000 7f9bc1277740 1 -- 172.21.15.172:0/2637383538 wait complete. 2024-09-10T09:04:56.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.894+0000 7f9bc1277740 1 -- start start 2024-09-10T09:04:56.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.894+0000 7f9bc1277740 1 Processor -- start 2024-09-10T09:04:56.901 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.894+0000 7f9bc1277740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b669850 0x55c93b6786b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.895+0000 7f9bc1277740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b66a170 0x55c93b678bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.895+0000 7f9bc1277740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b674a30 0x55c93b679130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.895+0000 7f9bc1277740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb81b6540 con 0x55c93b66a170 2024-09-10T09:04:56.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.895+0000 7f9bc1277740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb81b5e70 con 0x55c93b674a30 2024-09-10T09:04:56.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.895+0000 7f9bb7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b66a170 0x55c93b678bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:56.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.895+0000 7f9bc1277740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb81b30b0 con 0x55c93b669850 2024-09-10T09:04:56.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.895+0000 7f9bc1277740 10 client.0 _ll_register_callbacks cb 0x7f9bac016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:56.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.895+0000 7f9bbcc3a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b669850 0x55c93b6786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:56.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.895+0000 7f9bbcc3a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b669850 0x55c93b6786b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:37296/0 (socket says 172.21.15.172:37296) 2024-09-10T09:04:56.902 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.895+0000 7f9bb7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b66a170 0x55c93b678bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60128/0 (socket says 172.21.15.172:60128) 2024-09-10T09:04:56.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.895+0000 7f9bb7fff640 1 -- 172.21.15.172:0/3587818362 learned_addr learned my addr 172.21.15.172:0/3587818362 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:56.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.895+0000 7f9bbd43b640 1 --2- 172.21.15.172:0/3587818362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b674a30 0x55c93b679130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:56.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.895+0000 7f9bbcc3a640 1 -- 172.21.15.172:0/3587818362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b674a30 msgr2=0x55c93b679130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:56.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.895+0000 7f9bbcc3a640 1 --2- 172.21.15.172:0/3587818362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b674a30 0x55c93b679130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.895+0000 7f9bbcc3a640 1 -- 172.21.15.172:0/3587818362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b66a170 msgr2=0x55c93b678bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:56.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.895+0000 7f9bbcc3a640 1 --2- 172.21.15.172:0/3587818362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b66a170 0x55c93b678bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:56.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.895+0000 7f9bbcc3a640 1 -- 172.21.15.172:0/3587818362 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c93b6613f0 con 0x55c93b669850 2024-09-10T09:04:56.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.896+0000 7f9bb7fff640 1 --2- 172.21.15.172:0/3587818362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b66a170 0x55c93b678bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:04:56.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.896+0000 7f9bbcc3a640 1 --2- 172.21.15.172:0/3587818362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b669850 0x55c93b6786b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9bac011e00 tx=0x7f9bac011e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce453fb38fa0a875 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:56.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.896+0000 7f9b97fff640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:04:56.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.896+0000 7f9b97fff640 1 -- 172.21.15.172:0/3587818362 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac023040 con 0x55c93b669850 2024-09-10T09:04:56.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.896+0000 7f9b97fff640 1 -- 172.21.15.172:0/3587818362 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9bac01f860 con 0x55c93b669850 2024-09-10T09:04:56.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.896+0000 7f9bc1277740 1 -- 172.21.15.172:0/3587818362 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bb000d460 con 0x55c93b669850 2024-09-10T09:04:56.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.897+0000 7f9b97fff640 1 -- 172.21.15.172:0/3587818362 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9bac00bcf0 con 0x55c93b669850 2024-09-10T09:04:56.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.897+0000 7f9bc1277740 10 client.28591 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:56.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.897+0000 7f9bc1277740 1 -- 172.21.15.172:0/3587818362 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c93b6ef400 con 0x55c93b669850 2024-09-10T09:04:56.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.897+0000 7f9b97fff640 1 -- 172.21.15.172:0/3587818362 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bac027600 con 0x55c93b669850 2024-09-10T09:04:56.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.897+0000 7f9b97fff640 1 -- 172.21.15.172:0/3587818362 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f9bac011040 con 0x55c93b669850 2024-09-10T09:04:56.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.897+0000 7f9bc1277740 10 client.28591 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:56.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.897+0000 7f9bc1277740 10 client.28591 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:04:56.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.897+0000 7f9bc1277740 20 client.28591 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:56.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.897+0000 7f9bc1277740 1 --2- 172.21.15.172:0/3587818362 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55c93b6efb30 0x55c93b6f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:56.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.897+0000 7f9bc1277740 4 client.28591 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}] 2024-09-10T09:04:56.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.897+0000 7f9bc1277740 1 -- 172.21.15.172:0/3587818362 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}) -- 0x7f9bb000d460 con 0x55c93b6efb30 2024-09-10T09:04:56.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.898+0000 7f9bb7fff640 1 --2- 172.21.15.172:0/3587818362 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55c93b6efb30 0x55c93b6f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:56.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.898+0000 7f9bb7fff640 1 --2- 172.21.15.172:0/3587818362 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55c93b6efb30 0x55c93b6f1f20 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f325d9662733d507 server_cookie=faf0cd08c28ab324 in_seq=0 out_seq=0 2024-09-10T09:04:56.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:56.898+0000 7f9b97fff640 10 client.28591 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:56.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:56 smithi174 ceph-mon[34068]: pgmap v911: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 413 KiB/s wr, 114 op/s 2024-09-10T09:04:57.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.008+0000 7f9b97fff640 1 -- 172.21.15.172:0/3587818362 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f9bb000d460 con 0x55c93b6efb30 2024-09-10T09:04:57.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.009+0000 7f9b97fff640 10 client.28591 handle_command_reply: tid=0 2024-09-10T09:04:57.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.009+0000 7f9bc1277740 1 client.28591 shutdown 2024-09-10T09:04:57.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.009+0000 7f9bc1277740 1 -- 172.21.15.172:0/3587818362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b669850 msgr2=0x55c93b6786b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:57.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.009+0000 7f9bc1277740 1 --2- 172.21.15.172:0/3587818362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b669850 0x55c93b6786b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9bac011e00 tx=0x7f9bac011e80 comp rx=0 tx=0).stop 2024-09-10T09:04:57.010 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.009+0000 7f9bc1277740 1 -- 172.21.15.172:0/3587818362 shutdown_connections 2024-09-10T09:04:57.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.009+0000 7f9bc1277740 1 --2- 172.21.15.172:0/3587818362 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55c93b6efb30 0x55c93b6f1f20 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:57.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.009+0000 7f9bc1277740 1 --2- 172.21.15.172:0/3587818362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c93b674a30 0x55c93b679130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:57.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.009+0000 7f9bc1277740 1 --2- 172.21.15.172:0/3587818362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c93b66a170 0x55c93b678bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:57.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.009+0000 7f9bc1277740 1 --2- 172.21.15.172:0/3587818362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c93b669850 0x55c93b6786b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:57.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.009+0000 7f9b97fff640 0 client.28591 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:04:57.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.009+0000 7f9bc1277740 1 -- 172.21.15.172:0/3587818362 >> 172.21.15.172:0/3587818362 conn(0x55c93b665de0 msgr2=0x55c93b6646c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:57.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.009+0000 7f9bc1277740 1 -- 172.21.15.172:0/3587818362 shutdown_connections 2024-09-10T09:04:57.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.009+0000 7f9bc1277740 1 -- 172.21.15.172:0/3587818362 wait complete. 2024-09-10T09:04:57.011 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.009+0000 7f9bc1277740 20 client.28591 trim_cache size 0 max 16384 2024-09-10T09:04:57.012 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:04:57.012 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 44, 2024-09-10T09:04:57.012 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T09:04:57.012 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 58, 2024-09-10T09:04:57.012 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T09:04:57.013 INFO:teuthology.orchestra.run.smithi172.stdout: "15ff85c3": { 2024-09-10T09:04:57.013 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 58, 2024-09-10T09:04:57.013 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T09:04:57.013 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T09:04:57.013 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T09:04:57.013 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T09:04:57.013 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:04:57.013 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T09:04:57.013 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 139.8, 2024-09-10T09:04:57.013 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T09:04:57.013 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T09:04:57.013 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T09:04:57.014 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T09:04:57.014 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "QUIESCED", 2024-09-10T09:04:57.014 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T09:04:57.014 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:04:57.014 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:04:57.014 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:04:57.014 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:04:57.014 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:04:57.014 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:04:57.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.010+0000 7f9bbee9e640 1 -- 172.21.15.172:0/182639425 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9ba80a2d90 msgr2=0x7f9ba80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:57.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.010+0000 7f9bbee9e640 1 --2- 172.21.15.172:0/182639425 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9ba80a2d90 0x7f9ba80a5250 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x55c93b666c40 tx=0x7f9bac005f50 comp rx=0 tx=0).stop 2024-09-10T09:04:57.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.010+0000 7f9bbee9e640 1 -- 172.21.15.172:0/182639425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bb8102790 msgr2=0x7f9bb819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:57.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.010+0000 7f9bbee9e640 1 --2- 172.21.15.172:0/182639425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bb8102790 0x7f9bb819efc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4005ab0 tx=0x7f9ba4003c20 comp rx=0 tx=0).stop 2024-09-10T09:04:57.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.010+0000 7f9bbee9e640 1 -- 172.21.15.172:0/182639425 shutdown_connections 2024-09-10T09:04:57.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.010+0000 7f9bbee9e640 1 --2- 172.21.15.172:0/182639425 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9ba80a2d90 0x7f9ba80a5250 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:57.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.010+0000 7f9bbee9e640 1 --2- 172.21.15.172:0/182639425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bb8106f10 0x7f9bb81a3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:57.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.010+0000 7f9bbee9e640 1 --2- 172.21.15.172:0/182639425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bb8102790 0x7f9bb819efc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:57.015 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.010+0000 7f9bbee9e640 1 --2- 172.21.15.172:0/182639425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bb8101de0 0x7f9bb819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:57.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.010+0000 7f9bbee9e640 1 -- 172.21.15.172:0/182639425 >> 172.21.15.172:0/182639425 conn(0x7f9bb80fd880 msgr2=0x7f9bb80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:57.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.010+0000 7f9bbee9e640 1 -- 172.21.15.172:0/182639425 shutdown_connections 2024-09-10T09:04:57.016 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:57.010+0000 7f9bbee9e640 1 -- 172.21.15.172:0/182639425 wait complete. 2024-09-10T09:04:57.020 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:04:57.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:56 smithi191 ceph-mon[35314]: pgmap v911: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 413 KiB/s wr, 114 op/s 2024-09-10T09:04:57.786 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '15ff85c3', quiesce took 8.2 seconds. Will release after: 11.650000000000002 2024-09-10T09:04:58.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.217+0000 7fe104f19640 1 -- 172.21.15.172:0/3675512406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100106f10 msgr2=0x7fe10010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:58.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.217+0000 7fe104f19640 1 --2- 172.21.15.172:0/3675512406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100106f10 0x7fe10010ea90 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009ca0 tx=0x7fe0f802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.217+0000 7fe104f19640 1 -- 172.21.15.172:0/3675512406 shutdown_connections 2024-09-10T09:04:58.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.217+0000 7fe104f19640 1 --2- 172.21.15.172:0/3675512406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100106f10 0x7fe10010ea90 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.217+0000 7fe104f19640 1 --2- 172.21.15.172:0/3675512406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe100102790 0x7fe1001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.217+0000 7fe104f19640 1 --2- 172.21.15.172:0/3675512406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101de0 0x7fe1001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.217+0000 7fe104f19640 1 -- 172.21.15.172:0/3675512406 >> 172.21.15.172:0/3675512406 conn(0x7fe1000fd8c0 msgr2=0x7fe1000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:58.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.217+0000 7fe104f19640 1 -- 172.21.15.172:0/3675512406 shutdown_connections 2024-09-10T09:04:58.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.217+0000 7fe104f19640 1 -- 172.21.15.172:0/3675512406 wait complete. 2024-09-10T09:04:58.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.217+0000 7fe104f19640 1 Processor -- start 2024-09-10T09:04:58.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.217+0000 7fe104f19640 1 -- start start 2024-09-10T09:04:58.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe104f19640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101de0 0x7fe100198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe104f19640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100102790 0x7fe100198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe104f19640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe100106f10 0x7fe10019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe104f19640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe100111e40 con 0x7fe100102790 2024-09-10T09:04:58.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe104f19640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe100111cc0 con 0x7fe100101de0 2024-09-10T09:04:58.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe104f19640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe100111fc0 con 0x7fe100106f10 2024-09-10T09:04:58.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe0fdd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100102790 0x7fe100198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe0fdd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100102790 0x7fe100198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60140/0 (socket says 172.21.15.172:60140) 2024-09-10T09:04:58.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe0fdd9b640 1 -- 172.21.15.172:0/1599983745 learned_addr learned my addr 172.21.15.172:0/1599983745 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:58.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe0fe59c640 1 --2- 172.21.15.172:0/1599983745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101de0 0x7fe100198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe0fed9d640 1 --2- 172.21.15.172:0/1599983745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe100106f10 0x7fe10019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe0fdd9b640 1 -- 172.21.15.172:0/1599983745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe100106f10 msgr2=0x7fe10019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:58.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe0fdd9b640 1 --2- 172.21.15.172:0/1599983745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe100106f10 0x7fe10019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe0fdd9b640 1 -- 172.21.15.172:0/1599983745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101de0 msgr2=0x7fe100198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:58.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe0fdd9b640 1 --2- 172.21.15.172:0/1599983745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101de0 0x7fe100198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe0fdd9b640 1 -- 172.21.15.172:0/1599983745 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10019d2d0 con 0x7fe100102790 2024-09-10T09:04:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe0fdd9b640 1 --2- 172.21.15.172:0/1599983745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100102790 0x7fe100198860 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400b4a0 tx=0x7fe0f400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe0ef7fe640 1 -- 172.21.15.172:0/1599983745 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4015020 con 0x7fe100102790 2024-09-10T09:04:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe0ef7fe640 1 -- 172.21.15.172:0/1599983745 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0f4011ce0 con 0x7fe100102790 2024-09-10T09:04:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.218+0000 7fe104f19640 1 -- 172.21.15.172:0/1599983745 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10019d5c0 con 0x7fe100102790 2024-09-10T09:04:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.219+0000 7fe0ef7fe640 1 -- 172.21.15.172:0/1599983745 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4010dc0 con 0x7fe100102790 2024-09-10T09:04:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.219+0000 7fe104f19640 1 -- 172.21.15.172:0/1599983745 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1001a4ea0 con 0x7fe100102790 2024-09-10T09:04:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.221+0000 7fe0ef7fe640 1 -- 172.21.15.172:0/1599983745 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe0f4018020 con 0x7fe100102790 2024-09-10T09:04:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.221+0000 7fe1072f2740 1 Processor -- start 2024-09-10T09:04:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.221+0000 7fe1072f2740 1 -- start start 2024-09-10T09:04:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.221+0000 7fe0ef7fe640 1 --2- 172.21.15.172:0/1599983745 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0e80a2b80 0x7fe0e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.221+0000 7fe1072f2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c94b40 0x559da6c93090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.221+0000 7fe1072f2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c935d0 0x559da6c93a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.221+0000 7fe1072f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c9e370 0x559da6ca0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.221+0000 7fe1072f2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559da6bf92f0 con 0x559da6c94b40 2024-09-10T09:04:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.221+0000 7fe1072f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559da6ba0cf0 con 0x559da6c9e370 2024-09-10T09:04:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.221+0000 7fe1072f2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559da6bf52b0 con 0x559da6c935d0 2024-09-10T09:04:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.221+0000 7fe0fe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c94b40 0x559da6c93090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.221+0000 7fe0fe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c94b40 0x559da6c93090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60144/0 (socket says 172.21.15.172:60144) 2024-09-10T09:04:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.221+0000 7fe0fe59c640 1 -- 172.21.15.172:0/3391363795 learned_addr learned my addr 172.21.15.172:0/3391363795 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.221+0000 7fe0fed9d640 1 --2- 172.21.15.172:0/1599983745 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0e80a2b80 0x7fe0e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.221+0000 7fe0ef7fe640 1 -- 172.21.15.172:0/1599983745 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe0f40cece0 con 0x7fe100102790 2024-09-10T09:04:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.222+0000 7fe0fdd9b640 1 --2- 172.21.15.172:0/3391363795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c935d0 0x559da6c93a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.222+0000 7fe0fed9d640 1 --2- 172.21.15.172:0/3391363795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c9e370 0x559da6ca0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.222+0000 7fe0fe59c640 1 -- 172.21.15.172:0/3391363795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c935d0 msgr2=0x559da6c93a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.222+0000 7fe0fe59c640 1 --2- 172.21.15.172:0/3391363795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c935d0 0x559da6c93a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.222+0000 7fe0fe59c640 1 -- 172.21.15.172:0/3391363795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c9e370 msgr2=0x559da6ca0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.222+0000 7fe0fe59c640 1 --2- 172.21.15.172:0/3391363795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c9e370 0x559da6ca0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.222+0000 7fe0fe59c640 1 -- 172.21.15.172:0/3391363795 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559da6b853a0 con 0x559da6c94b40 2024-09-10T09:04:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.222+0000 7fe0fed9d640 1 --2- 172.21.15.172:0/3391363795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c9e370 0x559da6ca0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:04:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.222+0000 7fe0fe59c640 1 --2- 172.21.15.172:0/3391363795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c94b40 0x559da6c93090 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000ad00 tx=0x7fe0f000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21b37a1a9592130 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.222+0000 7fe0fed9d640 1 --2- 172.21.15.172:0/1599983745 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0e80a2b80 0x7fe0e80a5040 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x559da6c94070 tx=0x7fe0f8009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.222+0000 7fe0ecff9640 1 -- 172.21.15.172:0/3391363795 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f001d070 con 0x559da6c94b40 2024-09-10T09:04:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.222+0000 7fe0ecff9640 1 -- 172.21.15.172:0/3391363795 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0f0018af0 con 0x559da6c94b40 2024-09-10T09:04:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.222+0000 7fe0ecff9640 1 -- 172.21.15.172:0/3391363795 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0010ae0 con 0x559da6c94b40 2024-09-10T09:04:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.223+0000 7fe1072f2740 1 -- 172.21.15.172:0/3391363795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c94b40 msgr2=0x559da6c93090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:58.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.223+0000 7fe1072f2740 1 --2- 172.21.15.172:0/3391363795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c94b40 0x559da6c93090 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000ad00 tx=0x7fe0f000f790 comp rx=0 tx=0).stop 2024-09-10T09:04:58.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.223+0000 7fe1072f2740 1 -- 172.21.15.172:0/3391363795 shutdown_connections 2024-09-10T09:04:58.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.223+0000 7fe1072f2740 1 --2- 172.21.15.172:0/3391363795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c9e370 0x559da6ca0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.223+0000 7fe1072f2740 1 --2- 172.21.15.172:0/3391363795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c935d0 0x559da6c93a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.223+0000 7fe1072f2740 1 --2- 172.21.15.172:0/3391363795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c94b40 0x559da6c93090 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.223+0000 7fe1072f2740 1 -- 172.21.15.172:0/3391363795 >> 172.21.15.172:0/3391363795 conn(0x559da6c5c310 msgr2=0x559da6c8fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:58.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.223+0000 7fe1072f2740 1 -- 172.21.15.172:0/3391363795 shutdown_connections 2024-09-10T09:04:58.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.223+0000 7fe1072f2740 1 -- 172.21.15.172:0/3391363795 wait complete. 2024-09-10T09:04:58.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.223+0000 7fe1072f2740 1 -- start start 2024-09-10T09:04:58.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.223+0000 7fe1072f2740 1 Processor -- start 2024-09-10T09:04:58.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe1072f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c935d0 0x559da6c9dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe1072f2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c94b40 0x559da6c9afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe1072f2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c9e370 0x559da6c9b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.242 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe1072f2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559da6ca5480 con 0x559da6c9e370 2024-09-10T09:04:58.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe1072f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559da6bb4060 con 0x559da6c935d0 2024-09-10T09:04:58.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe1072f2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559da6ca5600 con 0x559da6c94b40 2024-09-10T09:04:58.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe1072f2740 10 client.0 _ll_register_callbacks cb 0x559da69078c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:58.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe0fed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c9e370 0x559da6c9b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe0fed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c9e370 0x559da6c9b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60148/0 (socket says 172.21.15.172:60148) 2024-09-10T09:04:58.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe0fed9d640 1 -- 172.21.15.172:0/4105689090 learned_addr learned my addr 172.21.15.172:0/4105689090 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:58.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe0fdd9b640 1 --2- 172.21.15.172:0/4105689090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c94b40 0x559da6c9afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe0fe59c640 1 --2- 172.21.15.172:0/4105689090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c935d0 0x559da6c9dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe0fed9d640 1 -- 172.21.15.172:0/4105689090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c94b40 msgr2=0x559da6c9afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:58.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe0fed9d640 1 --2- 172.21.15.172:0/4105689090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c94b40 0x559da6c9afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.243 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe0fed9d640 1 -- 172.21.15.172:0/4105689090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c935d0 msgr2=0x559da6c9dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:58.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe0fed9d640 1 --2- 172.21.15.172:0/4105689090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c935d0 0x559da6c9dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe0fed9d640 1 -- 172.21.15.172:0/4105689090 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559da696a3c0 con 0x559da6c9e370 2024-09-10T09:04:58.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe0fe59c640 1 --2- 172.21.15.172:0/4105689090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c935d0 0x559da6c9dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:04:58.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe0fed9d640 1 --2- 172.21.15.172:0/4105689090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c9e370 0x559da6c9b570 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8032000 tx=0x7fe0f80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152af0fe97bc3c6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:58.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe0e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:58.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe0e5ffb640 1 -- 172.21.15.172:0/4105689090 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8052060 con 0x559da6c9e370 2024-09-10T09:04:58.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe0e5ffb640 1 -- 172.21.15.172:0/4105689090 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe0f80305c0 con 0x559da6c9e370 2024-09-10T09:04:58.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.224+0000 7fe1072f2740 1 -- 172.21.15.172:0/4105689090 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0f0018af0 con 0x559da6c9e370 2024-09-10T09:04:58.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.225+0000 7fe0e5ffb640 1 -- 172.21.15.172:0/4105689090 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe0f8059050 con 0x559da6c9e370 2024-09-10T09:04:58.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.225+0000 7fe1072f2740 10 client.19947 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:58.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.225+0000 7fe1072f2740 1 -- 172.21.15.172:0/4105689090 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559da6cb0e40 con 0x559da6c9e370 2024-09-10T09:04:58.244 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.225+0000 7fe0e5ffb640 1 -- 172.21.15.172:0/4105689090 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0f0018af0 con 0x559da6c9e370 2024-09-10T09:04:58.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.225+0000 7fe0e5ffb640 1 -- 172.21.15.172:0/4105689090 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fe0f804d6b0 con 0x559da6c9e370 2024-09-10T09:04:58.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.225+0000 7fe1072f2740 10 client.19947 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:58.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.225+0000 7fe1072f2740 10 client.19947 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:04:58.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.225+0000 7fe1072f2740 20 client.19947 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:58.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.225+0000 7fe1072f2740 1 --2- 172.21.15.172:0/4105689090 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x559da6cb2300 0x559da6cb4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.225+0000 7fe1072f2740 4 client.19947 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:04:58.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.225+0000 7fe1072f2740 1 -- 172.21.15.172:0/4105689090 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559da6c06860 con 0x559da6cb2300 2024-09-10T09:04:58.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.225+0000 7fe0fe59c640 1 --2- 172.21.15.172:0/4105689090 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x559da6cb2300 0x559da6cb4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.226+0000 7fe0e5ffb640 10 client.19947 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T09:04:58.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.226+0000 7fe0fe59c640 1 --2- 172.21.15.172:0/4105689090 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x559da6cb2300 0x559da6cb4770 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f768d93a2b7b8a54 server_cookie=a160d14e311a1d45 in_seq=0 out_seq=0 2024-09-10T09:04:58.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.226+0000 7fe0e5ffb640 1 -- 172.21.15.172:0/4105689090 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559da6c06860 con 0x559da6cb2300 2024-09-10T09:04:58.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.226+0000 7fe0e5ffb640 10 client.19947 handle_command_reply: tid=0 2024-09-10T09:04:58.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.227+0000 7fe1072f2740 1 client.19947 shutdown 2024-09-10T09:04:58.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.227+0000 7fe1072f2740 1 -- 172.21.15.172:0/4105689090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c9e370 msgr2=0x559da6c9b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:58.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.227+0000 7fe1072f2740 1 --2- 172.21.15.172:0/4105689090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c9e370 0x559da6c9b570 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8032000 tx=0x7fe0f80095c0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.227+0000 7fe1072f2740 1 -- 172.21.15.172:0/4105689090 shutdown_connections 2024-09-10T09:04:58.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.227+0000 7fe1072f2740 1 --2- 172.21.15.172:0/4105689090 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x559da6cb2300 0x559da6cb4770 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.227+0000 7fe1072f2740 1 --2- 172.21.15.172:0/4105689090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c9e370 0x559da6c9b570 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.227+0000 7fe1072f2740 1 --2- 172.21.15.172:0/4105689090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c94b40 0x559da6c9afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.227+0000 7fe1072f2740 1 --2- 172.21.15.172:0/4105689090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c935d0 0x559da6c9dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.227+0000 7fe0e5ffb640 0 client.19947 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T09:04:58.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.227+0000 7fe1072f2740 1 -- 172.21.15.172:0/4105689090 >> 172.21.15.172:0/4105689090 conn(0x559da6c5c310 msgr2=0x559da6c95960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:58.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.227+0000 7fe1072f2740 1 -- 172.21.15.172:0/4105689090 shutdown_connections 2024-09-10T09:04:58.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.227+0000 7fe1072f2740 1 -- 172.21.15.172:0/4105689090 wait complete. 2024-09-10T09:04:58.246 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.227+0000 7fe1072f2740 20 client.19947 trim_cache size 0 max 16384 2024-09-10T09:04:58.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.240+0000 7fe1072f2740 1 Processor -- start 2024-09-10T09:04:58.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe1072f2740 1 -- start start 2024-09-10T09:04:58.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe1072f2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c935d0 0x559da6c9cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe1072f2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c94b40 0x559da6c96120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe1072f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c9e370 0x559da6c96660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe1072f2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe1001ac900 con 0x559da6c94b40 2024-09-10T09:04:58.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe1072f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe1001ae440 con 0x559da6c9e370 2024-09-10T09:04:58.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe1072f2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe1001acfd0 con 0x559da6c935d0 2024-09-10T09:04:58.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe0fdd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c94b40 0x559da6c96120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe0fdd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c94b40 0x559da6c96120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60156/0 (socket says 172.21.15.172:60156) 2024-09-10T09:04:58.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe0fdd9b640 1 -- 172.21.15.172:0/1272164210 learned_addr learned my addr 172.21.15.172:0/1272164210 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:58.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe0fe59c640 1 --2- 172.21.15.172:0/1272164210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c935d0 0x559da6c9cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe0fed9d640 1 --2- 172.21.15.172:0/1272164210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c9e370 0x559da6c96660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe0fdd9b640 1 -- 172.21.15.172:0/1272164210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c935d0 msgr2=0x559da6c9cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:58.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe0fdd9b640 1 --2- 172.21.15.172:0/1272164210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c935d0 0x559da6c9cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe0fdd9b640 1 -- 172.21.15.172:0/1272164210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c9e370 msgr2=0x559da6c96660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:58.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe0fdd9b640 1 --2- 172.21.15.172:0/1272164210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c9e370 0x559da6c96660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe0fdd9b640 1 -- 172.21.15.172:0/1272164210 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559da6c90960 con 0x559da6c94b40 2024-09-10T09:04:58.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe0fed9d640 1 --2- 172.21.15.172:0/1272164210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c9e370 0x559da6c96660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:04:58.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.241+0000 7fe0fdd9b640 1 --2- 172.21.15.172:0/1272164210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c94b40 0x559da6c96120 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fe0f409b170 tx=0x7fe0f40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdaa4c11782ac7e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:58.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.242+0000 7fe0e7fff640 1 -- 172.21.15.172:0/1272164210 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f406cd90 con 0x559da6c94b40 2024-09-10T09:04:58.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.242+0000 7fe0e7fff640 1 -- 172.21.15.172:0/1272164210 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0f409b1e0 con 0x559da6c94b40 2024-09-10T09:04:58.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.242+0000 7fe0e7fff640 1 -- 172.21.15.172:0/1272164210 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f406b480 con 0x559da6c94b40 2024-09-10T09:04:58.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.242+0000 7fe1072f2740 1 -- 172.21.15.172:0/1272164210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c94b40 msgr2=0x559da6c96120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:58.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.242+0000 7fe1072f2740 1 --2- 172.21.15.172:0/1272164210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c94b40 0x559da6c96120 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fe0f409b170 tx=0x7fe0f40047d0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.242+0000 7fe1072f2740 1 -- 172.21.15.172:0/1272164210 shutdown_connections 2024-09-10T09:04:58.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.242+0000 7fe1072f2740 1 --2- 172.21.15.172:0/1272164210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c9e370 0x559da6c96660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.253 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.242+0000 7fe1072f2740 1 --2- 172.21.15.172:0/1272164210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c94b40 0x559da6c96120 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.242+0000 7fe1072f2740 1 --2- 172.21.15.172:0/1272164210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c935d0 0x559da6c9cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.242+0000 7fe1072f2740 1 -- 172.21.15.172:0/1272164210 >> 172.21.15.172:0/1272164210 conn(0x559da6c5c310 msgr2=0x559da6c9c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:58.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.242+0000 7fe1072f2740 1 -- 172.21.15.172:0/1272164210 shutdown_connections 2024-09-10T09:04:58.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.242+0000 7fe1072f2740 1 -- 172.21.15.172:0/1272164210 wait complete. 2024-09-10T09:04:58.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe1072f2740 1 -- start start 2024-09-10T09:04:58.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe1072f2740 1 Processor -- start 2024-09-10T09:04:58.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe1072f2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c935d0 0x559da6ca68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe1072f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c94b40 0x559da6ca6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe1072f2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c9e370 0x559da6ca7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe1072f2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe1001add70 con 0x559da6c935d0 2024-09-10T09:04:58.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe1072f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe1001ad6a0 con 0x559da6c94b40 2024-09-10T09:04:58.254 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe1072f2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe1001aa8e0 con 0x559da6c9e370 2024-09-10T09:04:58.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe1072f2740 10 client.0 _ll_register_callbacks cb 0x7fe0f8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:58.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe0fe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c935d0 0x559da6ca68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe0fe59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c935d0 0x559da6ca68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60162/0 (socket says 172.21.15.172:60162) 2024-09-10T09:04:58.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe0fe59c640 1 -- 172.21.15.172:0/1654336823 learned_addr learned my addr 172.21.15.172:0/1654336823 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:58.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe0fdd9b640 1 --2- 172.21.15.172:0/1654336823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c94b40 0x559da6ca6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe0fed9d640 1 --2- 172.21.15.172:0/1654336823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c9e370 0x559da6ca7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe0fe59c640 1 -- 172.21.15.172:0/1654336823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c9e370 msgr2=0x559da6ca7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:58.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe0fe59c640 1 --2- 172.21.15.172:0/1654336823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c9e370 0x559da6ca7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe0fe59c640 1 -- 172.21.15.172:0/1654336823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c94b40 msgr2=0x559da6ca6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:58.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe0fe59c640 1 --2- 172.21.15.172:0/1654336823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c94b40 0x559da6ca6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.243+0000 7fe0fe59c640 1 -- 172.21.15.172:0/1654336823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559da6d1c2e0 con 0x559da6c935d0 2024-09-10T09:04:58.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.244+0000 7fe0fe59c640 1 --2- 172.21.15.172:0/1654336823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c935d0 0x559da6ca68d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0010510 tx=0x7fe0f000a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0241a17c3cd4957 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:58.255 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.244+0000 7fe0e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:58.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.244+0000 7fe0e5ffb640 1 -- 172.21.15.172:0/1654336823 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0031020 con 0x559da6c935d0 2024-09-10T09:04:58.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.244+0000 7fe0e5ffb640 1 -- 172.21.15.172:0/1654336823 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe0f0005450 con 0x559da6c935d0 2024-09-10T09:04:58.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.244+0000 7fe1072f2740 1 -- 172.21.15.172:0/1654336823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0f409b1e0 con 0x559da6c935d0 2024-09-10T09:04:58.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.244+0000 7fe0e5ffb640 1 -- 172.21.15.172:0/1654336823 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe0f003c050 con 0x559da6c935d0 2024-09-10T09:04:58.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.244+0000 7fe1072f2740 10 client.19953 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:58.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.244+0000 7fe1072f2740 1 -- 172.21.15.172:0/1654336823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559da6d1d5e0 con 0x559da6c935d0 2024-09-10T09:04:58.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.244+0000 7fe0e5ffb640 1 -- 172.21.15.172:0/1654336823 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0f409b1e0 con 0x559da6c935d0 2024-09-10T09:04:58.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.244+0000 7fe0e5ffb640 1 -- 172.21.15.172:0/1654336823 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fe0f00108f0 con 0x559da6c935d0 2024-09-10T09:04:58.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.244+0000 7fe1072f2740 10 client.19953 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:58.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.244+0000 7fe1072f2740 10 client.19953 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:04:58.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.244+0000 7fe1072f2740 20 client.19953 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:58.256 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.244+0000 7fe1072f2740 1 --2- 172.21.15.172:0/1654336823 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x559da6d1da20 0x559da6d1fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:58.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.244+0000 7fe1072f2740 4 client.19953 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:04:58.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.244+0000 7fe1072f2740 1 -- 172.21.15.172:0/1654336823 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559da6c00d70 con 0x559da6d1da20 2024-09-10T09:04:58.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.245+0000 7fe0fdd9b640 1 --2- 172.21.15.172:0/1654336823 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x559da6d1da20 0x559da6d1fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:58.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.245+0000 7fe0e5ffb640 10 client.19953 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T09:04:58.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.245+0000 7fe0fdd9b640 1 --2- 172.21.15.172:0/1654336823 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x559da6d1da20 0x559da6d1fe10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35ebde29c8de70c0 server_cookie=3342b5d5f455c9bd in_seq=0 out_seq=0 2024-09-10T09:04:58.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.245+0000 7fe0e5ffb640 1 -- 172.21.15.172:0/1654336823 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559da6c00d70 con 0x559da6d1da20 2024-09-10T09:04:58.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.245+0000 7fe0e5ffb640 10 client.19953 handle_command_reply: tid=0 2024-09-10T09:04:58.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.245+0000 7fe1072f2740 1 client.19953 shutdown 2024-09-10T09:04:58.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.245+0000 7fe1072f2740 1 -- 172.21.15.172:0/1654336823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c935d0 msgr2=0x559da6ca68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:58.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.245+0000 7fe1072f2740 1 --2- 172.21.15.172:0/1654336823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c935d0 0x559da6ca68d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0010510 tx=0x7fe0f000a2d0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.245+0000 7fe1072f2740 1 -- 172.21.15.172:0/1654336823 shutdown_connections 2024-09-10T09:04:58.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.245+0000 7fe1072f2740 1 --2- 172.21.15.172:0/1654336823 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x559da6d1da20 0x559da6d1fe10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.245+0000 7fe1072f2740 1 --2- 172.21.15.172:0/1654336823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559da6c9e370 0x559da6ca7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.245+0000 7fe1072f2740 1 --2- 172.21.15.172:0/1654336823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559da6c94b40 0x559da6ca6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.245+0000 7fe0e5ffb640 0 client.19953 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T09:04:58.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.245+0000 7fe1072f2740 1 --2- 172.21.15.172:0/1654336823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559da6c935d0 0x559da6ca68d0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.245+0000 7fe1072f2740 1 -- 172.21.15.172:0/1654336823 >> 172.21.15.172:0/1654336823 conn(0x559da6c5c310 msgr2=0x559da6c8ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:58.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.245+0000 7fe1072f2740 1 -- 172.21.15.172:0/1654336823 shutdown_connections 2024-09-10T09:04:58.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.246+0000 7fe1072f2740 1 -- 172.21.15.172:0/1654336823 wait complete. 2024-09-10T09:04:58.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.246+0000 7fe1072f2740 20 client.19953 trim_cache size 0 max 16384 2024-09-10T09:04:58.258 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:04:58.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.246+0000 7fe104f19640 1 -- 172.21.15.172:0/1599983745 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0e80a2b80 msgr2=0x7fe0e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:58.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.246+0000 7fe104f19640 1 --2- 172.21.15.172:0/1599983745 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0e80a2b80 0x7fe0e80a5040 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x559da6c94070 tx=0x7fe0f8009800 comp rx=0 tx=0).stop 2024-09-10T09:04:58.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.246+0000 7fe104f19640 1 -- 172.21.15.172:0/1599983745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100102790 msgr2=0x7fe100198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:58.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.246+0000 7fe104f19640 1 --2- 172.21.15.172:0/1599983745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100102790 0x7fe100198860 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400b4a0 tx=0x7fe0f400b8a0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.246+0000 7fe104f19640 1 -- 172.21.15.172:0/1599983745 shutdown_connections 2024-09-10T09:04:58.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.246+0000 7fe104f19640 1 --2- 172.21.15.172:0/1599983745 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0e80a2b80 0x7fe0e80a5040 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.246+0000 7fe104f19640 1 --2- 172.21.15.172:0/1599983745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe100106f10 0x7fe10019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.246+0000 7fe104f19640 1 --2- 172.21.15.172:0/1599983745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100102790 0x7fe100198860 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.246+0000 7fe104f19640 1 --2- 172.21.15.172:0/1599983745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101de0 0x7fe100198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:58.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.246+0000 7fe104f19640 1 -- 172.21.15.172:0/1599983745 >> 172.21.15.172:0/1599983745 conn(0x7fe1000fd8c0 msgr2=0x7fe1000fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:58.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.246+0000 7fe104f19640 1 -- 172.21.15.172:0/1599983745 shutdown_connections 2024-09-10T09:04:58.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:58.246+0000 7fe104f19640 1 -- 172.21.15.172:0/1599983745 wait complete. 2024-09-10T09:04:58.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:04:58 smithi174 ceph-mon[34068]: pgmap v912: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 835 KiB/s wr, 218 op/s 2024-09-10T09:04:59.034 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:04:58 smithi172 ceph-mon[30854]: pgmap v912: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 835 KiB/s wr, 218 op/s 2024-09-10T09:04:59.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:04:58 smithi191 ceph-mon[35314]: pgmap v912: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 835 KiB/s wr, 218 op/s 2024-09-10T09:04:59.065 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:04:59.319 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:04:59.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.835+0000 7f8e55fa5640 1 -- 172.21.15.172:0/4097664949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e50100e20 msgr2=0x7f8e50101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:59.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.835+0000 7f8e55fa5640 1 --2- 172.21.15.172:0/4097664949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e50100e20 0x7f8e50101200 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009900 tx=0x7f8e4002a480 comp rx=0 tx=0).stop 2024-09-10T09:04:59.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.835+0000 7f8e55fa5640 1 -- 172.21.15.172:0/4097664949 shutdown_connections 2024-09-10T09:04:59.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.835+0000 7f8e55fa5640 1 --2- 172.21.15.172:0/4097664949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e50102ef0 0x7f8e5010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.835+0000 7f8e55fa5640 1 --2- 172.21.15.172:0/4097664949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e50102570 0x7f8e501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.855 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.835+0000 7f8e55fa5640 1 --2- 172.21.15.172:0/4097664949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e50100e20 0x7f8e50101200 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.835+0000 7f8e55fa5640 1 -- 172.21.15.172:0/4097664949 >> 172.21.15.172:0/4097664949 conn(0x7f8e500fc7d0 msgr2=0x7f8e500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:59.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.835+0000 7f8e55fa5640 1 -- 172.21.15.172:0/4097664949 shutdown_connections 2024-09-10T09:04:59.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.835+0000 7f8e55fa5640 1 -- 172.21.15.172:0/4097664949 wait complete. 2024-09-10T09:04:59.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.836+0000 7f8e55fa5640 1 Processor -- start 2024-09-10T09:04:59.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.836+0000 7f8e55fa5640 1 -- start start 2024-09-10T09:04:59.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.836+0000 7f8e55fa5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e50100e20 0x7f8e500fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.836+0000 7f8e55fa5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e50102570 0x7f8e500fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.836+0000 7f8e55fa5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e50102ef0 0x7f8e500fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.856 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.836+0000 7f8e55fa5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8e50110ed0 con 0x7f8e50102ef0 2024-09-10T09:04:59.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.836+0000 7f8e55fa5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e50110d50 con 0x7f8e50102570 2024-09-10T09:04:59.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.836+0000 7f8e55fa5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8e50111050 con 0x7f8e50100e20 2024-09-10T09:04:59.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.836+0000 7f8e4ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e50102ef0 0x7f8e500fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:59.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.836+0000 7f8e4ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e50102ef0 0x7f8e500fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60174/0 (socket says 172.21.15.172:60174) 2024-09-10T09:04:59.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.836+0000 7f8e4ffff640 1 -- 172.21.15.172:0/4228772253 learned_addr learned my addr 172.21.15.172:0/4228772253 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:59.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.837+0000 7f8e4f7fe640 1 --2- 172.21.15.172:0/4228772253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e50100e20 0x7f8e500fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:59.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.837+0000 7f8e46ffd640 1 --2- 172.21.15.172:0/4228772253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e50102570 0x7f8e500fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:59.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.837+0000 7f8e4ffff640 1 -- 172.21.15.172:0/4228772253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e50100e20 msgr2=0x7f8e500fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:59.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.837+0000 7f8e4ffff640 1 --2- 172.21.15.172:0/4228772253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e50100e20 0x7f8e500fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.837+0000 7f8e4ffff640 1 -- 172.21.15.172:0/4228772253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e50102570 msgr2=0x7f8e500fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:59.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.837+0000 7f8e4ffff640 1 --2- 172.21.15.172:0/4228772253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e50102570 0x7f8e500fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.837+0000 7f8e4ffff640 1 -- 172.21.15.172:0/4228772253 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e501a6a10 con 0x7f8e50102ef0 2024-09-10T09:04:59.857 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.837+0000 7f8e4f7fe640 1 --2- 172.21.15.172:0/4228772253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e50100e20 0x7f8e500fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:04:59.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.837+0000 7f8e4ffff640 1 --2- 172.21.15.172:0/4228772253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e50102ef0 0x7f8e500fd700 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800b970 tx=0x7f8e4800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:59.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.837+0000 7f8e4d7fa640 1 -- 172.21.15.172:0/4228772253 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48019070 con 0x7f8e50102ef0 2024-09-10T09:04:59.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.837+0000 7f8e4d7fa640 1 -- 172.21.15.172:0/4228772253 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e48014440 con 0x7f8e50102ef0 2024-09-10T09:04:59.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.837+0000 7f8e55fa5640 1 -- 172.21.15.172:0/4228772253 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e501a6d00 con 0x7f8e50102ef0 2024-09-10T09:04:59.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.837+0000 7f8e4d7fa640 1 -- 172.21.15.172:0/4228772253 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e480075d0 con 0x7f8e50102ef0 2024-09-10T09:04:59.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.837+0000 7f8e55fa5640 1 -- 172.21.15.172:0/4228772253 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e501a71f0 con 0x7f8e50102ef0 2024-09-10T09:04:59.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.839+0000 7f8e4d7fa640 1 -- 172.21.15.172:0/4228772253 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8e4801c070 con 0x7f8e50102ef0 2024-09-10T09:04:59.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e5837e740 1 Processor -- start 2024-09-10T09:04:59.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e5837e740 1 -- start start 2024-09-10T09:04:59.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e5837e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521827b40 0x55a521826090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.858 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e5837e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5218265d0 0x55a521826a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e5837e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521831370 0x55a521833760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e5837e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a52178c2f0 con 0x55a521831370 2024-09-10T09:04:59.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e5837e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a521733cf0 con 0x55a521827b40 2024-09-10T09:04:59.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e5837e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a5217882b0 con 0x55a5218265d0 2024-09-10T09:04:59.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e46ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521831370 0x55a521833760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:59.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e46ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521831370 0x55a521833760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60178/0 (socket says 172.21.15.172:60178) 2024-09-10T09:04:59.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e46ffd640 1 -- 172.21.15.172:0/2780293098 learned_addr learned my addr 172.21.15.172:0/2780293098 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:59.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e4d7fa640 1 --2- 172.21.15.172:0/4228772253 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8e3c0a2b80 0x7f8e3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e46ffd640 1 --2- 172.21.15.172:0/2780293098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521827b40 0x55a521826090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:59.859 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e4f7fe640 1 --2- 172.21.15.172:0/2780293098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5218265d0 0x55a521826a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:59.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e4d7fa640 1 -- 172.21.15.172:0/4228772253 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8e480cea70 con 0x7f8e50102ef0 2024-09-10T09:04:59.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e4ffff640 1 --2- 172.21.15.172:0/4228772253 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8e3c0a2b80 0x7f8e3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:59.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e46ffd640 1 -- 172.21.15.172:0/2780293098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5218265d0 msgr2=0x55a521826a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:59.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e46ffd640 1 --2- 172.21.15.172:0/2780293098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5218265d0 0x55a521826a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e46ffd640 1 -- 172.21.15.172:0/2780293098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521827b40 msgr2=0x55a521826090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:59.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e46ffd640 1 --2- 172.21.15.172:0/2780293098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521827b40 0x55a521826090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.840+0000 7f8e46ffd640 1 -- 172.21.15.172:0/2780293098 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5217183a0 con 0x55a521831370 2024-09-10T09:04:59.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.841+0000 7f8e46ffd640 1 --2- 172.21.15.172:0/2780293098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521831370 0x55a521833760 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800cf60 tx=0x7f8e3800c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84104b5f8349fb1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:59.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.841+0000 7f8e45ffb640 1 -- 172.21.15.172:0/2780293098 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3801d070 con 0x55a521831370 2024-09-10T09:04:59.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.841+0000 7f8e4ffff640 1 --2- 172.21.15.172:0/4228772253 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8e3c0a2b80 0x7f8e3c0a5040 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8e480a3650 tx=0x7f8e48004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:59.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.841+0000 7f8e45ffb640 1 -- 172.21.15.172:0/2780293098 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e38018b20 con 0x55a521831370 2024-09-10T09:04:59.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.841+0000 7f8e45ffb640 1 -- 172.21.15.172:0/2780293098 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3800f040 con 0x55a521831370 2024-09-10T09:04:59.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.841+0000 7f8e5837e740 1 -- 172.21.15.172:0/2780293098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521831370 msgr2=0x55a521833760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:59.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.841+0000 7f8e5837e740 1 --2- 172.21.15.172:0/2780293098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521831370 0x55a521833760 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800cf60 tx=0x7f8e3800c170 comp rx=0 tx=0).stop 2024-09-10T09:04:59.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.841+0000 7f8e5837e740 1 -- 172.21.15.172:0/2780293098 shutdown_connections 2024-09-10T09:04:59.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.841+0000 7f8e5837e740 1 --2- 172.21.15.172:0/2780293098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521831370 0x55a521833760 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.841+0000 7f8e5837e740 1 --2- 172.21.15.172:0/2780293098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5218265d0 0x55a521826a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.841+0000 7f8e5837e740 1 --2- 172.21.15.172:0/2780293098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521827b40 0x55a521826090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.841+0000 7f8e5837e740 1 -- 172.21.15.172:0/2780293098 >> 172.21.15.172:0/2780293098 conn(0x55a5217ef310 msgr2=0x55a521822c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:59.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.842+0000 7f8e5837e740 1 -- 172.21.15.172:0/2780293098 shutdown_connections 2024-09-10T09:04:59.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.842+0000 7f8e5837e740 1 -- 172.21.15.172:0/2780293098 wait complete. 2024-09-10T09:04:59.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.842+0000 7f8e5837e740 1 -- start start 2024-09-10T09:04:59.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.842+0000 7f8e5837e740 1 Processor -- start 2024-09-10T09:04:59.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.842+0000 7f8e5837e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5218265d0 0x55a521830dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.842+0000 7f8e5837e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a521827b40 0x55a52182e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.842+0000 7f8e5837e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521831370 0x55a52182e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.842+0000 7f8e5837e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a521834070 con 0x55a5218265d0 2024-09-10T09:04:59.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.842+0000 7f8e5837e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a521747060 con 0x55a521831370 2024-09-10T09:04:59.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.842+0000 7f8e5837e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a5218341f0 con 0x55a521827b40 2024-09-10T09:04:59.862 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.842+0000 7f8e5837e740 10 client.0 _ll_register_callbacks cb 0x55a52149a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:59.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.842+0000 7f8e46ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5218265d0 0x55a521830dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:59.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.842+0000 7f8e46ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5218265d0 0x55a521830dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60182/0 (socket says 172.21.15.172:60182) 2024-09-10T09:04:59.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.842+0000 7f8e46ffd640 1 -- 172.21.15.172:0/3815728959 learned_addr learned my addr 172.21.15.172:0/3815728959 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:59.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.843+0000 7f8e46ffd640 1 --2- 172.21.15.172:0/3815728959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521831370 0x55a52182e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:59.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.843+0000 7f8e46ffd640 1 -- 172.21.15.172:0/3815728959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a521827b40 msgr2=0x55a52182e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:59.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.843+0000 7f8e46ffd640 1 --2- 172.21.15.172:0/3815728959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a521827b40 0x55a52182e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.843+0000 7f8e46ffd640 1 -- 172.21.15.172:0/3815728959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521831370 msgr2=0x55a52182e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:59.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.843+0000 7f8e46ffd640 1 --2- 172.21.15.172:0/3815728959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521831370 0x55a52182e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.843+0000 7f8e46ffd640 1 -- 172.21.15.172:0/3815728959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5214fd3c0 con 0x55a5218265d0 2024-09-10T09:04:59.863 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.843+0000 7f8e46ffd640 1 --2- 172.21.15.172:0/3815728959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5218265d0 0x55a521830dc0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800d010 tx=0x7f8e38016520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2607895659755f2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:59.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.843+0000 7f8e2effd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:59.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.843+0000 7f8e2effd640 1 -- 172.21.15.172:0/3815728959 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3802a060 con 0x55a5218265d0 2024-09-10T09:04:59.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.843+0000 7f8e2effd640 1 -- 172.21.15.172:0/3815728959 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8e38006b20 con 0x55a5218265d0 2024-09-10T09:04:59.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.843+0000 7f8e5837e740 1 -- 172.21.15.172:0/3815728959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e38018b20 con 0x55a5218265d0 2024-09-10T09:04:59.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.843+0000 7f8e2effd640 1 -- 172.21.15.172:0/3815728959 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8e38031050 con 0x55a5218265d0 2024-09-10T09:04:59.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.843+0000 7f8e5837e740 10 client.19965 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:59.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.843+0000 7f8e5837e740 1 -- 172.21.15.172:0/3815728959 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a52183fbb0 con 0x55a5218265d0 2024-09-10T09:04:59.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.843+0000 7f8e2effd640 1 -- 172.21.15.172:0/3815728959 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e38018b20 con 0x55a5218265d0 2024-09-10T09:04:59.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.844+0000 7f8e2effd640 1 -- 172.21.15.172:0/3815728959 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f8e38015050 con 0x55a5218265d0 2024-09-10T09:04:59.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.844+0000 7f8e5837e740 10 client.19965 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:59.864 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.844+0000 7f8e5837e740 10 client.19965 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:04:59.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.844+0000 7f8e5837e740 20 client.19965 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:59.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.844+0000 7f8e5837e740 1 --2- 172.21.15.172:0/3815728959 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a521840f00 0x55a5218432f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.844+0000 7f8e5837e740 4 client.19965 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:04:59.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.844+0000 7f8e5837e740 1 -- 172.21.15.172:0/3815728959 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a521799860 con 0x55a521840f00 2024-09-10T09:04:59.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.844+0000 7f8e4f7fe640 1 --2- 172.21.15.172:0/3815728959 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a521840f00 0x55a5218432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:59.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.844+0000 7f8e2effd640 10 client.19965 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T09:04:59.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.844+0000 7f8e4f7fe640 1 --2- 172.21.15.172:0/3815728959 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a521840f00 0x55a5218432f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61caa2b3f5d68ef0 server_cookie=4fa53ab492ceadf8 in_seq=0 out_seq=0 2024-09-10T09:04:59.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.845+0000 7f8e2effd640 1 -- 172.21.15.172:0/3815728959 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a521799860 con 0x55a521840f00 2024-09-10T09:04:59.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.845+0000 7f8e2effd640 10 client.19965 handle_command_reply: tid=0 2024-09-10T09:04:59.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.845+0000 7f8e5837e740 1 client.19965 shutdown 2024-09-10T09:04:59.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.845+0000 7f8e5837e740 1 -- 172.21.15.172:0/3815728959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5218265d0 msgr2=0x55a521830dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:59.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.845+0000 7f8e5837e740 1 --2- 172.21.15.172:0/3815728959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5218265d0 0x55a521830dc0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f8e3800d010 tx=0x7f8e38016520 comp rx=0 tx=0).stop 2024-09-10T09:04:59.865 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.845+0000 7f8e5837e740 1 -- 172.21.15.172:0/3815728959 shutdown_connections 2024-09-10T09:04:59.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.845+0000 7f8e5837e740 1 --2- 172.21.15.172:0/3815728959 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a521840f00 0x55a5218432f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.845+0000 7f8e5837e740 1 --2- 172.21.15.172:0/3815728959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521831370 0x55a52182e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.845+0000 7f8e5837e740 1 --2- 172.21.15.172:0/3815728959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a521827b40 0x55a52182e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.845+0000 7f8e5837e740 1 --2- 172.21.15.172:0/3815728959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5218265d0 0x55a521830dc0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.845+0000 7f8e5837e740 1 -- 172.21.15.172:0/3815728959 >> 172.21.15.172:0/3815728959 conn(0x55a5217ef310 msgr2=0x55a521828960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:59.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.845+0000 7f8e2effd640 0 client.19965 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T09:04:59.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.845+0000 7f8e5837e740 1 -- 172.21.15.172:0/3815728959 shutdown_connections 2024-09-10T09:04:59.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.845+0000 7f8e5837e740 1 -- 172.21.15.172:0/3815728959 wait complete. 2024-09-10T09:04:59.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.845+0000 7f8e5837e740 20 client.19965 trim_cache size 0 max 16384 2024-09-10T09:04:59.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.858+0000 7f8e5837e740 1 Processor -- start 2024-09-10T09:04:59.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.858+0000 7f8e5837e740 1 -- start start 2024-09-10T09:04:59.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.858+0000 7f8e5837e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5218265d0 0x55a52182d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.858+0000 7f8e5837e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a521827b40 0x55a521822c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.858+0000 7f8e5837e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521831370 0x55a521845f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.858+0000 7f8e5837e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501b0d90 con 0x55a521831370 2024-09-10T09:04:59.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.858+0000 7f8e5837e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501b28d0 con 0x55a5218265d0 2024-09-10T09:04:59.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.858+0000 7f8e5837e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501b1460 con 0x55a521827b40 2024-09-10T09:04:59.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.859+0000 7f8e46ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521831370 0x55a521845f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:59.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.859+0000 7f8e46ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521831370 0x55a521845f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60188/0 (socket says 172.21.15.172:60188) 2024-09-10T09:04:59.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.859+0000 7f8e46ffd640 1 -- 172.21.15.172:0/3439711756 learned_addr learned my addr 172.21.15.172:0/3439711756 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:59.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.859+0000 7f8e4f7fe640 1 --2- 172.21.15.172:0/3439711756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a521827b40 0x55a521822c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:59.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.859+0000 7f8e46ffd640 1 -- 172.21.15.172:0/3439711756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a521827b40 msgr2=0x55a521822c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:59.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.859+0000 7f8e46ffd640 1 --2- 172.21.15.172:0/3439711756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a521827b40 0x55a521822c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.859+0000 7f8e46ffd640 1 -- 172.21.15.172:0/3439711756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5218265d0 msgr2=0x55a52182d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.859+0000 7f8e46ffd640 1 --2- 172.21.15.172:0/3439711756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5218265d0 0x55a52182d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.859+0000 7f8e46ffd640 1 -- 172.21.15.172:0/3439711756 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a52182eb70 con 0x55a521831370 2024-09-10T09:04:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.859+0000 7f8e4f7fe640 1 --2- 172.21.15.172:0/3439711756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a521827b40 0x55a521822c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:04:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.859+0000 7f8e46ffd640 1 --2- 172.21.15.172:0/3439711756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521831370 0x55a521845f30 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f8e380258f0 tx=0x7f8e38016590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5351156e93a03bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.859+0000 7f8e457fa640 1 -- 172.21.15.172:0/3439711756 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3802a070 con 0x55a521831370 2024-09-10T09:04:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.859+0000 7f8e457fa640 1 -- 172.21.15.172:0/3439711756 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e3800a070 con 0x55a521831370 2024-09-10T09:04:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.859+0000 7f8e457fa640 1 -- 172.21.15.172:0/3439711756 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3801d070 con 0x55a521831370 2024-09-10T09:04:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.860+0000 7f8e5837e740 1 -- 172.21.15.172:0/3439711756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521831370 msgr2=0x55a521845f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.860+0000 7f8e5837e740 1 --2- 172.21.15.172:0/3439711756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521831370 0x55a521845f30 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f8e380258f0 tx=0x7f8e38016590 comp rx=0 tx=0).stop 2024-09-10T09:04:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.860+0000 7f8e5837e740 1 -- 172.21.15.172:0/3439711756 shutdown_connections 2024-09-10T09:04:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.860+0000 7f8e5837e740 1 --2- 172.21.15.172:0/3439711756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521831370 0x55a521845f30 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.860+0000 7f8e5837e740 1 --2- 172.21.15.172:0/3439711756 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a521827b40 0x55a521822c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.860+0000 7f8e5837e740 1 --2- 172.21.15.172:0/3439711756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5218265d0 0x55a52182d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.860+0000 7f8e5837e740 1 -- 172.21.15.172:0/3439711756 >> 172.21.15.172:0/3439711756 conn(0x55a5217ef310 msgr2=0x55a521822510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.860+0000 7f8e5837e740 1 -- 172.21.15.172:0/3439711756 shutdown_connections 2024-09-10T09:04:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.860+0000 7f8e5837e740 1 -- 172.21.15.172:0/3439711756 wait complete. 2024-09-10T09:04:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.860+0000 7f8e5837e740 1 -- start start 2024-09-10T09:04:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.860+0000 7f8e5837e740 1 Processor -- start 2024-09-10T09:04:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e5837e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5218265d0 0x55a521842f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e5837e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521827b40 0x55a521843450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e5837e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a521831370 0x55a521843990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e5837e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501b2200 con 0x55a521827b40 2024-09-10T09:04:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e5837e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501b1b30 con 0x55a5218265d0 2024-09-10T09:04:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e5837e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501aed70 con 0x55a521831370 2024-09-10T09:04:59.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e5837e740 10 client.0 _ll_register_callbacks cb 0x7f8e3801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:04:59.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e4f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521827b40 0x55a521843450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:59.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e4f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521827b40 0x55a521843450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60196/0 (socket says 172.21.15.172:60196) 2024-09-10T09:04:59.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e4f7fe640 1 -- 172.21.15.172:0/2923125247 learned_addr learned my addr 172.21.15.172:0/2923125247 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:04:59.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e46ffd640 1 --2- 172.21.15.172:0/2923125247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a521831370 0x55a521843990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:59.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e46ffd640 1 --2- 172.21.15.172:0/2923125247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5218265d0 0x55a521842f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:59.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e4f7fe640 1 -- 172.21.15.172:0/2923125247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a521831370 msgr2=0x55a521843990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:59.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e4f7fe640 1 --2- 172.21.15.172:0/2923125247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a521831370 0x55a521843990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e4f7fe640 1 -- 172.21.15.172:0/2923125247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5218265d0 msgr2=0x55a521842f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:04:59.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e4f7fe640 1 --2- 172.21.15.172:0/2923125247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5218265d0 0x55a521842f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e4f7fe640 1 -- 172.21.15.172:0/2923125247 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5218a9a30 con 0x55a521827b40 2024-09-10T09:04:59.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e46ffd640 1 --2- 172.21.15.172:0/2923125247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a521831370 0x55a521843990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:04:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e4f7fe640 1 --2- 172.21.15.172:0/2923125247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521827b40 0x55a521843450 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f8e4002a9a0 tx=0x7f8e40004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f6f80b6253ff4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:04:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e2effd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:04:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e2effd640 1 -- 172.21.15.172:0/2923125247 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40031e60 con 0x55a521827b40 2024-09-10T09:04:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e2effd640 1 -- 172.21.15.172:0/2923125247 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f8e40031430 con 0x55a521827b40 2024-09-10T09:04:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e5837e740 1 -- 172.21.15.172:0/2923125247 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e3800a070 con 0x55a521827b40 2024-09-10T09:04:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.861+0000 7f8e2effd640 1 -- 172.21.15.172:0/2923125247 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8e40052050 con 0x55a521827b40 2024-09-10T09:04:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.862+0000 7f8e5837e740 10 client.19971 fetch_fsmap learned FSMap version 44 2024-09-10T09:04:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.862+0000 7f8e5837e740 1 -- 172.21.15.172:0/2923125247 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a5218aad60 con 0x55a521827b40 2024-09-10T09:04:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.862+0000 7f8e2effd640 1 -- 172.21.15.172:0/2923125247 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e3800a070 con 0x55a521827b40 2024-09-10T09:04:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.862+0000 7f8e2effd640 1 -- 172.21.15.172:0/2923125247 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f8e4003c440 con 0x55a521827b40 2024-09-10T09:04:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.862+0000 7f8e5837e740 10 client.19971 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:04:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.862+0000 7f8e5837e740 10 client.19971 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:04:59.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.862+0000 7f8e5837e740 20 client.19971 populate_metadata read hostname 'smithi172' 2024-09-10T09:04:59.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.862+0000 7f8e5837e740 1 --2- 172.21.15.172:0/2923125247 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a5218ab510 0x55a5218ad930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:04:59.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.862+0000 7f8e5837e740 4 client.19971 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:04:59.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.862+0000 7f8e5837e740 1 -- 172.21.15.172:0/2923125247 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a521793d70 con 0x55a5218ab510 2024-09-10T09:04:59.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.862+0000 7f8e46ffd640 1 --2- 172.21.15.172:0/2923125247 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a5218ab510 0x55a5218ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:04:59.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.863+0000 7f8e46ffd640 1 --2- 172.21.15.172:0/2923125247 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a5218ab510 0x55a5218ad930 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c79a3b5e9ef9e20c server_cookie=eb25187b74440989 in_seq=0 out_seq=0 2024-09-10T09:04:59.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.863+0000 7f8e2effd640 10 client.19971 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T09:04:59.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.863+0000 7f8e2effd640 1 -- 172.21.15.172:0/2923125247 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a521793d70 con 0x55a5218ab510 2024-09-10T09:04:59.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.863+0000 7f8e2effd640 10 client.19971 handle_command_reply: tid=0 2024-09-10T09:04:59.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.863+0000 7f8e5837e740 1 client.19971 shutdown 2024-09-10T09:04:59.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.863+0000 7f8e5837e740 1 -- 172.21.15.172:0/2923125247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521827b40 msgr2=0x55a521843450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:59.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.863+0000 7f8e5837e740 1 --2- 172.21.15.172:0/2923125247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521827b40 0x55a521843450 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f8e4002a9a0 tx=0x7f8e40004040 comp rx=0 tx=0).stop 2024-09-10T09:04:59.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.863+0000 7f8e5837e740 1 -- 172.21.15.172:0/2923125247 shutdown_connections 2024-09-10T09:04:59.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.863+0000 7f8e5837e740 1 --2- 172.21.15.172:0/2923125247 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55a5218ab510 0x55a5218ad930 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.863+0000 7f8e5837e740 1 --2- 172.21.15.172:0/2923125247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a521831370 0x55a521843990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.863+0000 7f8e2effd640 0 client.19971 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T09:04:59.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.863+0000 7f8e5837e740 1 --2- 172.21.15.172:0/2923125247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a521827b40 0x55a521843450 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.863+0000 7f8e5837e740 1 --2- 172.21.15.172:0/2923125247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5218265d0 0x55a521842f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.863+0000 7f8e5837e740 1 -- 172.21.15.172:0/2923125247 >> 172.21.15.172:0/2923125247 conn(0x55a5217ef310 msgr2=0x55a52183c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:59.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.863+0000 7f8e5837e740 1 -- 172.21.15.172:0/2923125247 shutdown_connections 2024-09-10T09:04:59.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.863+0000 7f8e5837e740 1 -- 172.21.15.172:0/2923125247 wait complete. 2024-09-10T09:04:59.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.863+0000 7f8e5837e740 20 client.19971 trim_cache size 0 max 16384 2024-09-10T09:04:59.880 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:04:59.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.864+0000 7f8e55fa5640 1 -- 172.21.15.172:0/4228772253 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8e3c0a2b80 msgr2=0x7f8e3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:59.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.864+0000 7f8e55fa5640 1 --2- 172.21.15.172:0/4228772253 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8e3c0a2b80 0x7f8e3c0a5040 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8e480a3650 tx=0x7f8e48004b50 comp rx=0 tx=0).stop 2024-09-10T09:04:59.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.864+0000 7f8e55fa5640 1 -- 172.21.15.172:0/4228772253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e50102ef0 msgr2=0x7f8e500fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:04:59.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.864+0000 7f8e55fa5640 1 --2- 172.21.15.172:0/4228772253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e50102ef0 0x7f8e500fd700 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800b970 tx=0x7f8e4800be30 comp rx=0 tx=0).stop 2024-09-10T09:04:59.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.864+0000 7f8e55fa5640 1 -- 172.21.15.172:0/4228772253 shutdown_connections 2024-09-10T09:04:59.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.864+0000 7f8e55fa5640 1 --2- 172.21.15.172:0/4228772253 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f8e3c0a2b80 0x7f8e3c0a5040 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.864+0000 7f8e55fa5640 1 --2- 172.21.15.172:0/4228772253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e50102ef0 0x7f8e500fd700 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.864+0000 7f8e55fa5640 1 --2- 172.21.15.172:0/4228772253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e50102570 0x7f8e500fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.864+0000 7f8e55fa5640 1 --2- 172.21.15.172:0/4228772253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e50100e20 0x7f8e500fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:04:59.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.864+0000 7f8e55fa5640 1 -- 172.21.15.172:0/4228772253 >> 172.21.15.172:0/4228772253 conn(0x7f8e500fc7d0 msgr2=0x7f8e5006a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:04:59.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.864+0000 7f8e55fa5640 1 -- 172.21.15.172:0/4228772253 shutdown_connections 2024-09-10T09:04:59.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:04:59.864+0000 7f8e55fa5640 1 -- 172.21.15.172:0/4228772253 wait complete. 2024-09-10T09:05:00.654 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:05:00.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:00 smithi172 ceph-mon[30854]: pgmap v913: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 872 KiB/s wr, 227 op/s 2024-09-10T09:05:00.905 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:05:00.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:00 smithi174 ceph-mon[34068]: pgmap v913: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 872 KiB/s wr, 227 op/s 2024-09-10T09:05:01.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:00 smithi191 ceph-mon[35314]: pgmap v913: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 872 KiB/s wr, 227 op/s 2024-09-10T09:05:02.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.315+0000 7f41b42eb640 1 -- 172.21.15.172:0/3199128093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41ac106f10 msgr2=0x7f41ac10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:02.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.315+0000 7f41b42eb640 1 --2- 172.21.15.172:0/3199128093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41ac106f10 0x7f41ac10ea90 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009ca0 tx=0x7f41a402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.315+0000 7f41b42eb640 1 -- 172.21.15.172:0/3199128093 shutdown_connections 2024-09-10T09:05:02.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.315+0000 7f41b42eb640 1 --2- 172.21.15.172:0/3199128093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41ac106f10 0x7f41ac10ea90 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.315+0000 7f41b42eb640 1 --2- 172.21.15.172:0/3199128093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41ac102790 0x7f41ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.315+0000 7f41b42eb640 1 --2- 172.21.15.172:0/3199128093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41ac101de0 0x7f41ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.315+0000 7f41b42eb640 1 -- 172.21.15.172:0/3199128093 >> 172.21.15.172:0/3199128093 conn(0x7f41ac0fd8c0 msgr2=0x7f41ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:02.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.315+0000 7f41b42eb640 1 -- 172.21.15.172:0/3199128093 shutdown_connections 2024-09-10T09:05:02.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.315+0000 7f41b42eb640 1 -- 172.21.15.172:0/3199128093 wait complete. 2024-09-10T09:05:02.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.316+0000 7f41b42eb640 1 Processor -- start 2024-09-10T09:05:02.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.316+0000 7f41b42eb640 1 -- start start 2024-09-10T09:05:02.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.316+0000 7f41b42eb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41ac101de0 0x7f41ac198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.576 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.316+0000 7f41b42eb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41ac102790 0x7f41ac198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.316+0000 7f41b42eb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41ac106f10 0x7f41ac19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.316+0000 7f41b42eb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac111e20 con 0x7f41ac106f10 2024-09-10T09:05:02.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.316+0000 7f41b42eb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac111ca0 con 0x7f41ac102790 2024-09-10T09:05:02.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.316+0000 7f41b42eb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac111fa0 con 0x7f41ac101de0 2024-09-10T09:05:02.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.316+0000 7f41b2888640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41ac106f10 0x7f41ac19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.316+0000 7f41b2888640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41ac106f10 0x7f41ac19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60220/0 (socket says 172.21.15.172:60220) 2024-09-10T09:05:02.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.316+0000 7f41b2888640 1 -- 172.21.15.172:0/2930804236 learned_addr learned my addr 172.21.15.172:0/2930804236 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:02.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.316+0000 7f41b1886640 1 --2- 172.21.15.172:0/2930804236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41ac102790 0x7f41ac198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.316+0000 7f41b2087640 1 --2- 172.21.15.172:0/2930804236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41ac101de0 0x7f41ac198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.577 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.317+0000 7f41b2888640 1 -- 172.21.15.172:0/2930804236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41ac101de0 msgr2=0x7f41ac198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:02.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.317+0000 7f41b2888640 1 --2- 172.21.15.172:0/2930804236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41ac101de0 0x7f41ac198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.317+0000 7f41b2888640 1 -- 172.21.15.172:0/2930804236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41ac102790 msgr2=0x7f41ac198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:02.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.317+0000 7f41b2888640 1 --2- 172.21.15.172:0/2930804236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41ac102790 0x7f41ac198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.317+0000 7f41b2888640 1 -- 172.21.15.172:0/2930804236 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ac19d2f0 con 0x7f41ac106f10 2024-09-10T09:05:02.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.317+0000 7f41b2888640 1 --2- 172.21.15.172:0/2930804236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41ac106f10 0x7f41ac19cc10 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009dc0 tx=0x7f41a4032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:02.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.317+0000 7f41a37fe640 1 -- 172.21.15.172:0/2930804236 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4003cf0 con 0x7f41ac106f10 2024-09-10T09:05:02.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.317+0000 7f41b42eb640 1 -- 172.21.15.172:0/2930804236 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ac19d580 con 0x7f41ac106f10 2024-09-10T09:05:02.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.317+0000 7f41a37fe640 1 -- 172.21.15.172:0/2930804236 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41a40044f0 con 0x7f41ac106f10 2024-09-10T09:05:02.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.317+0000 7f41a37fe640 1 -- 172.21.15.172:0/2930804236 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4030bf0 con 0x7f41ac106f10 2024-09-10T09:05:02.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.317+0000 7f41b42eb640 1 -- 172.21.15.172:0/2930804236 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ac1a4ec0 con 0x7f41ac106f10 2024-09-10T09:05:02.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.320+0000 7f41a37fe640 1 -- 172.21.15.172:0/2930804236 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f41a4004040 con 0x7f41ac106f10 2024-09-10T09:05:02.578 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.320+0000 7f41b66c4740 1 Processor -- start 2024-09-10T09:05:02.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.320+0000 7f41b66c4740 1 -- start start 2024-09-10T09:05:02.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.320+0000 7f41b66c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda0ab40 0x55dcbda09090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.320+0000 7f41b66c4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda095d0 0x55dcbda09a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.320+0000 7f41a37fe640 1 --2- 172.21.15.172:0/2930804236 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f41980a2b80 0x7f41980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.320+0000 7f41b66c4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 0x55dcbda16760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.320+0000 7f41b66c4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55dcbd96f2f0 con 0x55dcbda14370 2024-09-10T09:05:02.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.320+0000 7f41b66c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dcbd916cf0 con 0x55dcbda0ab40 2024-09-10T09:05:02.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.320+0000 7f41b66c4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55dcbd96b2b0 con 0x55dcbda095d0 2024-09-10T09:05:02.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.320+0000 7f41b1886640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda0ab40 0x55dcbda09090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.579 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.320+0000 7f41a37fe640 1 -- 172.21.15.172:0/2930804236 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f41a40f0960 con 0x7f41ac106f10 2024-09-10T09:05:02.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.320+0000 7f41b1886640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda0ab40 0x55dcbda09090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:53416/0 (socket says 172.21.15.172:53416) 2024-09-10T09:05:02.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.320+0000 7f41b1886640 1 -- 172.21.15.172:0/751803746 learned_addr learned my addr 172.21.15.172:0/751803746 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:02.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.321+0000 7f41b2087640 1 --2- 172.21.15.172:0/751803746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda095d0 0x55dcbda09a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.321+0000 7f41b1886640 1 --2- 172.21.15.172:0/2930804236 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f41980a2b80 0x7f41980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.321+0000 7f41b2888640 1 --2- 172.21.15.172:0/751803746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 0x55dcbda16760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.321+0000 7f41b2087640 1 -- 172.21.15.172:0/751803746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda0ab40 msgr2=0x55dcbda09090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:02.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.321+0000 7f41b2087640 1 --2- 172.21.15.172:0/751803746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda0ab40 0x55dcbda09090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.321+0000 7f41b2087640 1 -- 172.21.15.172:0/751803746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 msgr2=0x55dcbda16760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:02.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.321+0000 7f41b2087640 1 --2- 172.21.15.172:0/751803746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 0x55dcbda16760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.321+0000 7f41b2087640 1 -- 172.21.15.172:0/751803746 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcbd8fb3a0 con 0x55dcbda095d0 2024-09-10T09:05:02.580 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.321+0000 7f41b1886640 1 --2- 172.21.15.172:0/2930804236 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f41980a2b80 0x7f41980a5040 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f419c012940 tx=0x7f419c00b810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:02.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.321+0000 7f41b2087640 1 --2- 172.21.15.172:0/751803746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda095d0 0x55dcbda09a10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f41a8004630 tx=0x7f41a800bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dacc500ccc093bae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:02.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.321+0000 7f41a0dea640 1 -- 172.21.15.172:0/751803746 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8003bd0 con 0x55dcbda095d0 2024-09-10T09:05:02.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.321+0000 7f41a0dea640 1 -- 172.21.15.172:0/751803746 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41a8015070 con 0x55dcbda095d0 2024-09-10T09:05:02.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.321+0000 7f41a0dea640 1 -- 172.21.15.172:0/751803746 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8004c70 con 0x55dcbda095d0 2024-09-10T09:05:02.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.322+0000 7f41b66c4740 1 -- 172.21.15.172:0/751803746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda095d0 msgr2=0x55dcbda09a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:02.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.322+0000 7f41b66c4740 1 --2- 172.21.15.172:0/751803746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda095d0 0x55dcbda09a10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f41a8004630 tx=0x7f41a800bb20 comp rx=0 tx=0).stop 2024-09-10T09:05:02.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.322+0000 7f41b66c4740 1 -- 172.21.15.172:0/751803746 shutdown_connections 2024-09-10T09:05:02.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.322+0000 7f41b66c4740 1 --2- 172.21.15.172:0/751803746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 0x55dcbda16760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.581 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.322+0000 7f41b66c4740 1 --2- 172.21.15.172:0/751803746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda095d0 0x55dcbda09a10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.322+0000 7f41b66c4740 1 --2- 172.21.15.172:0/751803746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda0ab40 0x55dcbda09090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.322+0000 7f41b66c4740 1 -- 172.21.15.172:0/751803746 >> 172.21.15.172:0/751803746 conn(0x55dcbd9d2310 msgr2=0x55dcbda05c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:02.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.322+0000 7f41b66c4740 1 -- 172.21.15.172:0/751803746 shutdown_connections 2024-09-10T09:05:02.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.322+0000 7f41b66c4740 1 -- 172.21.15.172:0/751803746 wait complete. 2024-09-10T09:05:02.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.322+0000 7f41b66c4740 1 -- start start 2024-09-10T09:05:02.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b66c4740 1 Processor -- start 2024-09-10T09:05:02.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b66c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda095d0 0x55dcbda13d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b66c4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda0ab40 0x55dcbda11010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b66c4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 0x55dcbda11600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b66c4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55dcbda17070 con 0x55dcbda14370 2024-09-10T09:05:02.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b66c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dcbd92a060 con 0x55dcbda095d0 2024-09-10T09:05:02.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b66c4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55dcbda171f0 con 0x55dcbda0ab40 2024-09-10T09:05:02.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b66c4740 10 client.0 _ll_register_callbacks cb 0x55dcbd67d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:02.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b2888640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 0x55dcbda11600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b2888640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 0x55dcbda11600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60242/0 (socket says 172.21.15.172:60242) 2024-09-10T09:05:02.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b2888640 1 -- 172.21.15.172:0/3128769774 learned_addr learned my addr 172.21.15.172:0/3128769774 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:02.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b2087640 1 --2- 172.21.15.172:0/3128769774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda095d0 0x55dcbda13d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b1886640 1 --2- 172.21.15.172:0/3128769774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda0ab40 0x55dcbda11010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b2888640 1 -- 172.21.15.172:0/3128769774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda0ab40 msgr2=0x55dcbda11010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:02.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b2888640 1 --2- 172.21.15.172:0/3128769774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda0ab40 0x55dcbda11010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b2888640 1 -- 172.21.15.172:0/3128769774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda095d0 msgr2=0x55dcbda13d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:02.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b2888640 1 --2- 172.21.15.172:0/3128769774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda095d0 0x55dcbda13d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b2888640 1 -- 172.21.15.172:0/3128769774 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcbd6e03c0 con 0x55dcbda14370 2024-09-10T09:05:02.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.323+0000 7f41b2087640 1 --2- 172.21.15.172:0/3128769774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda095d0 0x55dcbda13d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:05:02.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.324+0000 7f41b2888640 1 --2- 172.21.15.172:0/3128769774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 0x55dcbda11600 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f41a40ee150 tx=0x7f41a40bca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af8631786836143 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:02.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.324+0000 7f4195ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:05:02.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.324+0000 7f4195ffb640 1 -- 172.21.15.172:0/3128769774 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a40bd560 con 0x55dcbda14370 2024-09-10T09:05:02.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.324+0000 7f4195ffb640 1 -- 172.21.15.172:0/3128769774 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f41a40046a0 con 0x55dcbda14370 2024-09-10T09:05:02.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.324+0000 7f41b66c4740 1 -- 172.21.15.172:0/3128769774 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a8015070 con 0x55dcbda14370 2024-09-10T09:05:02.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.324+0000 7f4195ffb640 1 -- 172.21.15.172:0/3128769774 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f41a40bce60 con 0x55dcbda14370 2024-09-10T09:05:02.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.324+0000 7f41b66c4740 10 client.19983 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:02.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.324+0000 7f41b66c4740 1 -- 172.21.15.172:0/3128769774 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55dcbda17070 con 0x55dcbda14370 2024-09-10T09:05:02.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.324+0000 7f4195ffb640 1 -- 172.21.15.172:0/3128769774 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41a8015070 con 0x55dcbda14370 2024-09-10T09:05:02.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.324+0000 7f4195ffb640 1 -- 172.21.15.172:0/3128769774 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f41a40bd150 con 0x55dcbda14370 2024-09-10T09:05:02.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.324+0000 7f41b66c4740 10 client.19983 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:02.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.324+0000 7f41b66c4740 10 client.19983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:02.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.324+0000 7f41b66c4740 20 client.19983 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:02.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.324+0000 7f41b66c4740 1 --2- 172.21.15.172:0/3128769774 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55dcbda23e90 0x55dcbda26300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.324+0000 7f41b66c4740 4 client.19983 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:05:02.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.324+0000 7f41b66c4740 1 -- 172.21.15.172:0/3128769774 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcbd97c860 con 0x55dcbda23e90 2024-09-10T09:05:02.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.325+0000 7f41b2087640 1 --2- 172.21.15.172:0/3128769774 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55dcbda23e90 0x55dcbda26300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.325+0000 7f4195ffb640 10 client.19983 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:02.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.325+0000 7f41b2087640 1 --2- 172.21.15.172:0/3128769774 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55dcbda23e90 0x55dcbda26300 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df2d2ad99fff666e server_cookie=f05563b82ab9c3d8 in_seq=0 out_seq=0 2024-09-10T09:05:02.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.326+0000 7f4195ffb640 1 -- 172.21.15.172:0/3128769774 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dcbd97c860 con 0x55dcbda23e90 2024-09-10T09:05:02.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.326+0000 7f4195ffb640 10 client.19983 handle_command_reply: tid=0 2024-09-10T09:05:02.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.326+0000 7f41b66c4740 1 client.19983 shutdown 2024-09-10T09:05:02.585 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.326+0000 7f41b66c4740 1 -- 172.21.15.172:0/3128769774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 msgr2=0x55dcbda11600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:02.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.326+0000 7f41b66c4740 1 --2- 172.21.15.172:0/3128769774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 0x55dcbda11600 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f41a40ee150 tx=0x7f41a40bca60 comp rx=0 tx=0).stop 2024-09-10T09:05:02.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.326+0000 7f41b66c4740 1 -- 172.21.15.172:0/3128769774 shutdown_connections 2024-09-10T09:05:02.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.326+0000 7f41b66c4740 1 --2- 172.21.15.172:0/3128769774 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55dcbda23e90 0x55dcbda26300 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.326+0000 7f41b66c4740 1 --2- 172.21.15.172:0/3128769774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 0x55dcbda11600 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.326+0000 7f41b66c4740 1 --2- 172.21.15.172:0/3128769774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda0ab40 0x55dcbda11010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.326+0000 7f4195ffb640 0 client.19983 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:02.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.326+0000 7f41b66c4740 1 --2- 172.21.15.172:0/3128769774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda095d0 0x55dcbda13d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.326+0000 7f41b66c4740 1 -- 172.21.15.172:0/3128769774 >> 172.21.15.172:0/3128769774 conn(0x55dcbd9d2310 msgr2=0x55dcbda0b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:02.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.326+0000 7f41b66c4740 1 -- 172.21.15.172:0/3128769774 shutdown_connections 2024-09-10T09:05:02.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.326+0000 7f41b66c4740 1 -- 172.21.15.172:0/3128769774 wait complete. 2024-09-10T09:05:02.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.326+0000 7f41b66c4740 20 client.19983 trim_cache size 0 max 16384 2024-09-10T09:05:02.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.339+0000 7f41b66c4740 1 Processor -- start 2024-09-10T09:05:02.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.339+0000 7f41b66c4740 1 -- start start 2024-09-10T09:05:02.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.339+0000 7f41b66c4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda095d0 0x55dcbda20310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f41b66c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda0ab40 0x55dcbda108d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f41b66c4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 0x55dcbda05700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f41b66c4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1aea80 con 0x55dcbda14370 2024-09-10T09:05:02.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f41b66c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1b05c0 con 0x55dcbda0ab40 2024-09-10T09:05:02.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f41b66c4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1af150 con 0x55dcbda095d0 2024-09-10T09:05:02.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f41b2888640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 0x55dcbda05700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f41b2888640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 0x55dcbda05700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60254/0 (socket says 172.21.15.172:60254) 2024-09-10T09:05:02.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f41b2888640 1 -- 172.21.15.172:0/1816111358 learned_addr learned my addr 172.21.15.172:0/1816111358 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:02.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f41b1886640 1 --2- 172.21.15.172:0/1816111358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda0ab40 0x55dcbda108d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f41b2087640 1 --2- 172.21.15.172:0/1816111358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda095d0 0x55dcbda20310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f41b2888640 1 -- 172.21.15.172:0/1816111358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda095d0 msgr2=0x55dcbda20310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:02.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f41b2888640 1 --2- 172.21.15.172:0/1816111358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda095d0 0x55dcbda20310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f41b2888640 1 -- 172.21.15.172:0/1816111358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda0ab40 msgr2=0x55dcbda108d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:02.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f41b2888640 1 --2- 172.21.15.172:0/1816111358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda0ab40 0x55dcbda108d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.592 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f41b2888640 1 -- 172.21.15.172:0/1816111358 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcbda0f560 con 0x55dcbda14370 2024-09-10T09:05:02.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f41b2888640 1 --2- 172.21.15.172:0/1816111358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 0x55dcbda05700 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f41a40ee470 tx=0x7f41a40bcad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ce891abbdd8270 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:02.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f4197fff640 1 -- 172.21.15.172:0/1816111358 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a40b9100 con 0x55dcbda14370 2024-09-10T09:05:02.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f4197fff640 1 -- 172.21.15.172:0/1816111358 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41a4036040 con 0x55dcbda14370 2024-09-10T09:05:02.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.340+0000 7f4197fff640 1 -- 172.21.15.172:0/1816111358 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a40c1210 con 0x55dcbda14370 2024-09-10T09:05:02.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.341+0000 7f41b66c4740 1 -- 172.21.15.172:0/1816111358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 msgr2=0x55dcbda05700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:02.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.341+0000 7f41b66c4740 1 --2- 172.21.15.172:0/1816111358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 0x55dcbda05700 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f41a40ee470 tx=0x7f41a40bcad0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.341+0000 7f41b66c4740 1 -- 172.21.15.172:0/1816111358 shutdown_connections 2024-09-10T09:05:02.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.341+0000 7f41b66c4740 1 --2- 172.21.15.172:0/1816111358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda14370 0x55dcbda05700 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.341+0000 7f41b66c4740 1 --2- 172.21.15.172:0/1816111358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda0ab40 0x55dcbda108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.341+0000 7f41b66c4740 1 --2- 172.21.15.172:0/1816111358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda095d0 0x55dcbda20310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.341+0000 7f41b66c4740 1 -- 172.21.15.172:0/1816111358 >> 172.21.15.172:0/1816111358 conn(0x55dcbd9d2310 msgr2=0x55dcbda0f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:02.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.341+0000 7f41b66c4740 1 -- 172.21.15.172:0/1816111358 shutdown_connections 2024-09-10T09:05:02.593 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.341+0000 7f41b66c4740 1 -- 172.21.15.172:0/1816111358 wait complete. 2024-09-10T09:05:02.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.341+0000 7f41b66c4740 1 -- start start 2024-09-10T09:05:02.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.341+0000 7f41b66c4740 1 Processor -- start 2024-09-10T09:05:02.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b66c4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda095d0 0x55dcbda17a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b66c4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda0ab40 0x55dcbda17f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b66c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda14370 0x55dcbda8cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b66c4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1afef0 con 0x55dcbda0ab40 2024-09-10T09:05:02.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b66c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1af820 con 0x55dcbda14370 2024-09-10T09:05:02.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b66c4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac1aca60 con 0x55dcbda095d0 2024-09-10T09:05:02.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b66c4740 10 client.0 _ll_register_callbacks cb 0x7f41a40b8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:02.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b1886640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda0ab40 0x55dcbda17f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b1886640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda0ab40 0x55dcbda17f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60258/0 (socket says 172.21.15.172:60258) 2024-09-10T09:05:02.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b1886640 1 -- 172.21.15.172:0/3252175183 learned_addr learned my addr 172.21.15.172:0/3252175183 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:02.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b2888640 1 --2- 172.21.15.172:0/3252175183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda14370 0x55dcbda8cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b2087640 1 --2- 172.21.15.172:0/3252175183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda095d0 0x55dcbda17a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b1886640 1 -- 172.21.15.172:0/3252175183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda095d0 msgr2=0x55dcbda17a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:02.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b1886640 1 --2- 172.21.15.172:0/3252175183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda095d0 0x55dcbda17a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b1886640 1 -- 172.21.15.172:0/3252175183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda14370 msgr2=0x55dcbda8cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:02.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b1886640 1 --2- 172.21.15.172:0/3252175183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda14370 0x55dcbda8cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b1886640 1 -- 172.21.15.172:0/3252175183 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcbda8d110 con 0x55dcbda0ab40 2024-09-10T09:05:02.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b1886640 1 --2- 172.21.15.172:0/3252175183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda0ab40 0x55dcbda17f40 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f419c012720 tx=0x7f419c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2056f1c3da9a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:02.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f4195ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:05:02.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f4195ffb640 1 -- 172.21.15.172:0/3252175183 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c00ba60 con 0x55dcbda0ab40 2024-09-10T09:05:02.595 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f4195ffb640 1 -- 172.21.15.172:0/3252175183 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f419c005020 con 0x55dcbda0ab40 2024-09-10T09:05:02.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.342+0000 7f41b66c4740 1 -- 172.21.15.172:0/3252175183 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a4036040 con 0x55dcbda0ab40 2024-09-10T09:05:02.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.343+0000 7f4195ffb640 1 -- 172.21.15.172:0/3252175183 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f419c036050 con 0x55dcbda0ab40 2024-09-10T09:05:02.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.343+0000 7f41b66c4740 10 client.19989 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:02.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.343+0000 7f41b66c4740 1 -- 172.21.15.172:0/3252175183 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55dcbda8e410 con 0x55dcbda0ab40 2024-09-10T09:05:02.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.343+0000 7f4195ffb640 1 -- 172.21.15.172:0/3252175183 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41a4036040 con 0x55dcbda0ab40 2024-09-10T09:05:02.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.343+0000 7f4195ffb640 1 -- 172.21.15.172:0/3252175183 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f419c01f070 con 0x55dcbda0ab40 2024-09-10T09:05:02.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.343+0000 7f41b66c4740 10 client.19989 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:02.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.343+0000 7f41b66c4740 10 client.19989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:02.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.343+0000 7f41b66c4740 20 client.19989 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:02.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.343+0000 7f41b66c4740 1 --2- 172.21.15.172:0/3252175183 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55dcbda8ec10 0x55dcbda91030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:02.596 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.343+0000 7f41b66c4740 4 client.19989 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:05:02.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.343+0000 7f41b66c4740 1 -- 172.21.15.172:0/3252175183 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dcbd976d70 con 0x55dcbda8ec10 2024-09-10T09:05:02.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.343+0000 7f41b2087640 1 --2- 172.21.15.172:0/3252175183 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55dcbda8ec10 0x55dcbda91030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:02.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.344+0000 7f4195ffb640 10 client.19989 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:02.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.344+0000 7f41b2087640 1 --2- 172.21.15.172:0/3252175183 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55dcbda8ec10 0x55dcbda91030 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6beacf565a36eb3 server_cookie=e97447087bf9a38f in_seq=0 out_seq=0 2024-09-10T09:05:02.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.344+0000 7f4195ffb640 1 -- 172.21.15.172:0/3252175183 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dcbd976d70 con 0x55dcbda8ec10 2024-09-10T09:05:02.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.344+0000 7f4195ffb640 10 client.19989 handle_command_reply: tid=0 2024-09-10T09:05:02.597 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.344+0000 7f41b66c4740 1 client.19989 shutdown 2024-09-10T09:05:02.597 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:05:02.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.344+0000 7f41b66c4740 1 -- 172.21.15.172:0/3252175183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda0ab40 msgr2=0x55dcbda17f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:02.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.344+0000 7f41b66c4740 1 --2- 172.21.15.172:0/3252175183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda0ab40 0x55dcbda17f40 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f419c012720 tx=0x7f419c01c000 comp rx=0 tx=0).stop 2024-09-10T09:05:02.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.344+0000 7f41b66c4740 1 -- 172.21.15.172:0/3252175183 shutdown_connections 2024-09-10T09:05:02.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.344+0000 7f41b66c4740 1 --2- 172.21.15.172:0/3252175183 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55dcbda8ec10 0x55dcbda91030 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.344+0000 7f41b66c4740 1 --2- 172.21.15.172:0/3252175183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcbda14370 0x55dcbda8cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.344+0000 7f41b66c4740 1 --2- 172.21.15.172:0/3252175183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcbda0ab40 0x55dcbda17f40 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.344+0000 7f41b66c4740 1 --2- 172.21.15.172:0/3252175183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dcbda095d0 0x55dcbda17a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.344+0000 7f41b66c4740 1 -- 172.21.15.172:0/3252175183 >> 172.21.15.172:0/3252175183 conn(0x55dcbd9d2310 msgr2=0x55dcbda0b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:02.598 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.344+0000 7f4195ffb640 0 client.19989 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:02.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.344+0000 7f41b66c4740 1 -- 172.21.15.172:0/3252175183 shutdown_connections 2024-09-10T09:05:02.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.344+0000 7f41b66c4740 1 -- 172.21.15.172:0/3252175183 wait complete. 2024-09-10T09:05:02.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.344+0000 7f41b66c4740 20 client.19989 trim_cache size 0 max 16384 2024-09-10T09:05:02.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.345+0000 7f41b42eb640 1 -- 172.21.15.172:0/2930804236 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f41980a2b80 msgr2=0x7f41980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:02.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.345+0000 7f41b42eb640 1 --2- 172.21.15.172:0/2930804236 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f41980a2b80 0x7f41980a5040 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f419c012940 tx=0x7f419c00b810 comp rx=0 tx=0).stop 2024-09-10T09:05:02.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.345+0000 7f41b42eb640 1 -- 172.21.15.172:0/2930804236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41ac106f10 msgr2=0x7f41ac19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:02.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.345+0000 7f41b42eb640 1 --2- 172.21.15.172:0/2930804236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41ac106f10 0x7f41ac19cc10 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009dc0 tx=0x7f41a4032ff0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.345+0000 7f41b42eb640 1 -- 172.21.15.172:0/2930804236 shutdown_connections 2024-09-10T09:05:02.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.345+0000 7f41b42eb640 1 --2- 172.21.15.172:0/2930804236 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f41980a2b80 0x7f41980a5040 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.345+0000 7f41b42eb640 1 --2- 172.21.15.172:0/2930804236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41ac106f10 0x7f41ac19cc10 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.345+0000 7f41b42eb640 1 --2- 172.21.15.172:0/2930804236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41ac102790 0x7f41ac198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.345+0000 7f41b42eb640 1 --2- 172.21.15.172:0/2930804236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41ac101de0 0x7f41ac198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:02.599 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.345+0000 7f41b42eb640 1 -- 172.21.15.172:0/2930804236 >> 172.21.15.172:0/2930804236 conn(0x7f41ac0fd8c0 msgr2=0x7f41ac0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:02.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.345+0000 7f41b42eb640 1 -- 172.21.15.172:0/2930804236 shutdown_connections 2024-09-10T09:05:02.600 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:02.345+0000 7f41b42eb640 1 -- 172.21.15.172:0/2930804236 wait complete. 2024-09-10T09:05:02.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:02 smithi172 ceph-mon[30854]: pgmap v914: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 935 KiB/s wr, 245 op/s 2024-09-10T09:05:02.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:02 smithi174 ceph-mon[34068]: pgmap v914: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 935 KiB/s wr, 245 op/s 2024-09-10T09:05:03.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:02 smithi191 ceph-mon[35314]: pgmap v914: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 935 KiB/s wr, 245 op/s 2024-09-10T09:05:04.364 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:05:04.364 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:05:04.633 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:05:04.755 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:04 smithi172 ceph-mon[30854]: pgmap v915: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 866 KiB/s wr, 227 op/s 2024-09-10T09:05:04.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:04 smithi174 ceph-mon[34068]: pgmap v915: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 866 KiB/s wr, 227 op/s 2024-09-10T09:05:05.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:04 smithi191 ceph-mon[35314]: pgmap v915: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 866 KiB/s wr, 227 op/s 2024-09-10T09:05:05.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.145+0000 7f445b8ed640 1 -- 172.21.15.172:0/1024378582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4454102dc0 msgr2=0x7f44541031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:05.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.145+0000 7f445b8ed640 1 --2- 172.21.15.172:0/1024378582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4454102dc0 0x7f44541031a0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f4450009900 tx=0x7f445002a480 comp rx=0 tx=0).stop 2024-09-10T09:05:05.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.145+0000 7f445b8ed640 1 -- 172.21.15.172:0/1024378582 shutdown_connections 2024-09-10T09:05:05.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.145+0000 7f445b8ed640 1 --2- 172.21.15.172:0/1024378582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f445410c1a0 0x7f445410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.145+0000 7f445b8ed640 1 --2- 172.21.15.172:0/1024378582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44541036e0 0x7f445410bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.163 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.145+0000 7f445b8ed640 1 --2- 172.21.15.172:0/1024378582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4454102dc0 0x7f44541031a0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.145+0000 7f445b8ed640 1 -- 172.21.15.172:0/1024378582 >> 172.21.15.172:0/1024378582 conn(0x7f44540fc7a0 msgr2=0x7f44540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:05.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.145+0000 7f445b8ed640 1 -- 172.21.15.172:0/1024378582 shutdown_connections 2024-09-10T09:05:05.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.145+0000 7f445b8ed640 1 -- 172.21.15.172:0/1024378582 wait complete. 2024-09-10T09:05:05.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.145+0000 7f445b8ed640 1 Processor -- start 2024-09-10T09:05:05.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f445b8ed640 1 -- start start 2024-09-10T09:05:05.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f445b8ed640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4454102dc0 0x7f4454100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f445b8ed640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44541036e0 0x7f4454100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f445b8ed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f445410c1a0 0x7f44541014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f445b8ed640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4454111e20 con 0x7f445410c1a0 2024-09-10T09:05:05.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f445b8ed640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4454111ca0 con 0x7f44541036e0 2024-09-10T09:05:05.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f445b8ed640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4454111fa0 con 0x7f4454102dc0 2024-09-10T09:05:05.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f4459e8a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f445410c1a0 0x7f44541014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:05.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f4459e8a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f445410c1a0 0x7f44541014a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60278/0 (socket says 172.21.15.172:60278) 2024-09-10T09:05:05.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f4459e8a640 1 -- 172.21.15.172:0/3749927243 learned_addr learned my addr 172.21.15.172:0/3749927243 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:05.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f4458e88640 1 --2- 172.21.15.172:0/3749927243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44541036e0 0x7f4454100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:05.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f4459689640 1 --2- 172.21.15.172:0/3749927243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4454102dc0 0x7f4454100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:05.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f4459e8a640 1 -- 172.21.15.172:0/3749927243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4454102dc0 msgr2=0x7f4454100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:05.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f4459e8a640 1 --2- 172.21.15.172:0/3749927243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4454102dc0 0x7f4454100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f4459e8a640 1 -- 172.21.15.172:0/3749927243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44541036e0 msgr2=0x7f4454100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:05.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f4459e8a640 1 --2- 172.21.15.172:0/3749927243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44541036e0 0x7f4454100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f4459e8a640 1 -- 172.21.15.172:0/3749927243 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44541a6880 con 0x7f445410c1a0 2024-09-10T09:05:05.165 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f4459e8a640 1 --2- 172.21.15.172:0/3749927243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f445410c1a0 0x7f44541014a0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f444c00cad0 tx=0x7f444c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:05.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.146+0000 7f444a7fc640 1 -- 172.21.15.172:0/3749927243 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c00ae20 con 0x7f445410c1a0 2024-09-10T09:05:05.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.147+0000 7f444a7fc640 1 -- 172.21.15.172:0/3749927243 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f444c005020 con 0x7f445410c1a0 2024-09-10T09:05:05.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.147+0000 7f445b8ed640 1 -- 172.21.15.172:0/3749927243 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44541a6b70 con 0x7f445410c1a0 2024-09-10T09:05:05.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.147+0000 7f444a7fc640 1 -- 172.21.15.172:0/3749927243 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c00f6b0 con 0x7f445410c1a0 2024-09-10T09:05:05.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.147+0000 7f445b8ed640 1 -- 172.21.15.172:0/3749927243 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44541a7060 con 0x7f445410c1a0 2024-09-10T09:05:05.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.149+0000 7f444a7fc640 1 -- 172.21.15.172:0/3749927243 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f444c0051d0 con 0x7f445410c1a0 2024-09-10T09:05:05.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.149+0000 7f445dcc6740 1 Processor -- start 2024-09-10T09:05:05.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.149+0000 7f445dcc6740 1 -- start start 2024-09-10T09:05:05.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.149+0000 7f445dcc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d842b60 0x56356d842f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.149+0000 7f445dcc6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d843480 0x56356d84d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.149+0000 7f445dcc6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d84dd40 0x56356d850130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.149+0000 7f445dcc6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56356d7a8280 con 0x56356d843480 2024-09-10T09:05:05.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.149+0000 7f445dcc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56356d74fb10 con 0x56356d842b60 2024-09-10T09:05:05.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.149+0000 7f445dcc6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56356d7a4240 con 0x56356d84dd40 2024-09-10T09:05:05.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f4459689640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d843480 0x56356d84d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:05.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f4459689640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d843480 0x56356d84d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60294/0 (socket says 172.21.15.172:60294) 2024-09-10T09:05:05.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f4459689640 1 -- 172.21.15.172:0/2904572491 learned_addr learned my addr 172.21.15.172:0/2904572491 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:05.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f4458e88640 1 --2- 172.21.15.172:0/2904572491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d842b60 0x56356d842f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:05.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f444a7fc640 1 --2- 172.21.15.172:0/3749927243 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44400a2b80 0x7f44400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f4459689640 1 -- 172.21.15.172:0/2904572491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d84dd40 msgr2=0x56356d850130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:05.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f4458e88640 1 --2- 172.21.15.172:0/2904572491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d84dd40 0x56356d850130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:05.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f4459689640 1 --2- 172.21.15.172:0/2904572491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d84dd40 0x56356d850130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f4459689640 1 -- 172.21.15.172:0/2904572491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d842b60 msgr2=0x56356d842f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:05.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f4459689640 1 --2- 172.21.15.172:0/2904572491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d842b60 0x56356d842f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f4459689640 1 -- 172.21.15.172:0/2904572491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56356d75c1f0 con 0x56356d843480 2024-09-10T09:05:05.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f4459e8a640 1 --2- 172.21.15.172:0/3749927243 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44400a2b80 0x7f44400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:05.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f4458e88640 1 --2- 172.21.15.172:0/2904572491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d84dd40 0x56356d850130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:05:05.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f444a7fc640 1 -- 172.21.15.172:0/3749927243 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f444c0cf2e0 con 0x7f445410c1a0 2024-09-10T09:05:05.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f4459689640 1 --2- 172.21.15.172:0/2904572491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d843480 0x56356d84d800 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f4450009900 tx=0x7f4450031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76549e5a817fd748 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:05.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f443f7fe640 1 -- 172.21.15.172:0/2904572491 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4450040070 con 0x56356d843480 2024-09-10T09:05:05.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f4459e8a640 1 --2- 172.21.15.172:0/3749927243 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44400a2b80 0x7f44400a5040 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f444c097cf0 tx=0x7f444c070040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:05.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f443f7fe640 1 -- 172.21.15.172:0/2904572491 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44500316a0 con 0x56356d843480 2024-09-10T09:05:05.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.150+0000 7f443f7fe640 1 -- 172.21.15.172:0/2904572491 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4450035040 con 0x56356d843480 2024-09-10T09:05:05.168 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.151+0000 7f445dcc6740 1 -- 172.21.15.172:0/2904572491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d843480 msgr2=0x56356d84d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:05.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.151+0000 7f445dcc6740 1 --2- 172.21.15.172:0/2904572491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d843480 0x56356d84d800 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f4450009900 tx=0x7f4450031dc0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.151+0000 7f445dcc6740 1 -- 172.21.15.172:0/2904572491 shutdown_connections 2024-09-10T09:05:05.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.151+0000 7f445dcc6740 1 --2- 172.21.15.172:0/2904572491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d84dd40 0x56356d850130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.151+0000 7f445dcc6740 1 --2- 172.21.15.172:0/2904572491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d843480 0x56356d84d800 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.151+0000 7f445dcc6740 1 --2- 172.21.15.172:0/2904572491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d842b60 0x56356d842f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.151+0000 7f445dcc6740 1 -- 172.21.15.172:0/2904572491 >> 172.21.15.172:0/2904572491 conn(0x56356d83f0f0 msgr2=0x56356d83f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:05.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.151+0000 7f445dcc6740 1 -- 172.21.15.172:0/2904572491 shutdown_connections 2024-09-10T09:05:05.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.151+0000 7f445dcc6740 1 -- 172.21.15.172:0/2904572491 wait complete. 2024-09-10T09:05:05.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.151+0000 7f445dcc6740 1 -- start start 2024-09-10T09:05:05.169 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f445dcc6740 1 Processor -- start 2024-09-10T09:05:05.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f445dcc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d842b60 0x56356d84ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f445dcc6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d843480 0x56356d84b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f445dcc6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d84dd40 0x56356d84b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f445dcc6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56356d850910 con 0x56356d843480 2024-09-10T09:05:05.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f445dcc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56356d762f10 con 0x56356d842b60 2024-09-10T09:05:05.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f445dcc6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56356d850a90 con 0x56356d84dd40 2024-09-10T09:05:05.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f445dcc6740 10 client.0 _ll_register_callbacks cb 0x56356d72ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:05.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f4459689640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d843480 0x56356d84b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:05.170 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f4459689640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d843480 0x56356d84b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60310/0 (socket says 172.21.15.172:60310) 2024-09-10T09:05:05.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f4459689640 1 -- 172.21.15.172:0/2984303490 learned_addr learned my addr 172.21.15.172:0/2984303490 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:05.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f4458e88640 1 --2- 172.21.15.172:0/2984303490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d842b60 0x56356d84ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:05.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f4458e88640 1 --2- 172.21.15.172:0/2984303490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d84dd40 0x56356d84b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:05.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f4459689640 1 -- 172.21.15.172:0/2984303490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d84dd40 msgr2=0x56356d84b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:05.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f4459689640 1 --2- 172.21.15.172:0/2984303490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d84dd40 0x56356d84b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f4459689640 1 -- 172.21.15.172:0/2984303490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d842b60 msgr2=0x56356d84ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:05.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f4459689640 1 --2- 172.21.15.172:0/2984303490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d842b60 0x56356d84ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f4459689640 1 -- 172.21.15.172:0/2984303490 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56356d5196a0 con 0x56356d843480 2024-09-10T09:05:05.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f4458e88640 1 --2- 172.21.15.172:0/2984303490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d84dd40 0x56356d84b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:05:05.171 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f4459689640 1 --2- 172.21.15.172:0/2984303490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d843480 0x56356d84b140 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f445002a9a0 tx=0x7f445002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0dc38131079074 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:05.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f443cff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:05:05.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f443cff9640 1 -- 172.21.15.172:0/2984303490 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4450035070 con 0x56356d843480 2024-09-10T09:05:05.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f443cff9640 1 -- 172.21.15.172:0/2984303490 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f445003c650 con 0x56356d843480 2024-09-10T09:05:05.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.152+0000 7f445dcc6740 1 -- 172.21.15.172:0/2984303490 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44500316a0 con 0x56356d843480 2024-09-10T09:05:05.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.153+0000 7f443cff9640 1 -- 172.21.15.172:0/2984303490 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4450055050 con 0x56356d843480 2024-09-10T09:05:05.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.153+0000 7f445dcc6740 10 client.20001 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:05.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.153+0000 7f445dcc6740 1 -- 172.21.15.172:0/2984303490 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56356d85c490 con 0x56356d843480 2024-09-10T09:05:05.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.153+0000 7f443cff9640 1 -- 172.21.15.172:0/2984303490 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44500316a0 con 0x56356d843480 2024-09-10T09:05:05.172 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.153+0000 7f443cff9640 1 -- 172.21.15.172:0/2984303490 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f4450003f90 con 0x56356d843480 2024-09-10T09:05:05.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.153+0000 7f445dcc6740 10 client.20001 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:05.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.153+0000 7f445dcc6740 10 client.20001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:05.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.153+0000 7f445dcc6740 20 client.20001 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:05.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.153+0000 7f445dcc6740 1 --2- 172.21.15.172:0/2984303490 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56356d85d7e0 0x56356d85fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.153+0000 7f445dcc6740 4 client.20001 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:05:05.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.153+0000 7f445dcc6740 1 -- 172.21.15.172:0/2984303490 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56356d7b57f0 con 0x56356d85d7e0 2024-09-10T09:05:05.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.153+0000 7f4458e88640 1 --2- 172.21.15.172:0/2984303490 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56356d85d7e0 0x56356d85fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:05.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.154+0000 7f443cff9640 10 client.20001 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:05.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.154+0000 7f4458e88640 1 --2- 172.21.15.172:0/2984303490 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56356d85d7e0 0x56356d85fbd0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4605d18237e56400 server_cookie=3eb5d7a5adeb0b9c in_seq=0 out_seq=0 2024-09-10T09:05:05.173 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.154+0000 7f443cff9640 1 -- 172.21.15.172:0/2984303490 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56356d7b57f0 con 0x56356d85d7e0 2024-09-10T09:05:05.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.154+0000 7f443cff9640 10 client.20001 handle_command_reply: tid=0 2024-09-10T09:05:05.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.154+0000 7f445dcc6740 1 client.20001 shutdown 2024-09-10T09:05:05.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.154+0000 7f445dcc6740 1 -- 172.21.15.172:0/2984303490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d843480 msgr2=0x56356d84b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:05.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.154+0000 7f445dcc6740 1 --2- 172.21.15.172:0/2984303490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d843480 0x56356d84b140 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f445002a9a0 tx=0x7f445002a480 comp rx=0 tx=0).stop 2024-09-10T09:05:05.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.155+0000 7f445dcc6740 1 -- 172.21.15.172:0/2984303490 shutdown_connections 2024-09-10T09:05:05.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.155+0000 7f445dcc6740 1 --2- 172.21.15.172:0/2984303490 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56356d85d7e0 0x56356d85fbd0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.155+0000 7f445dcc6740 1 --2- 172.21.15.172:0/2984303490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d84dd40 0x56356d84b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.155+0000 7f445dcc6740 1 --2- 172.21.15.172:0/2984303490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d843480 0x56356d84b140 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.155+0000 7f443cff9640 0 client.20001 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:05.174 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.155+0000 7f445dcc6740 1 --2- 172.21.15.172:0/2984303490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d842b60 0x56356d84ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.155+0000 7f445dcc6740 1 -- 172.21.15.172:0/2984303490 >> 172.21.15.172:0/2984303490 conn(0x56356d83f0f0 msgr2=0x56356d83e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:05.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.155+0000 7f445dcc6740 1 -- 172.21.15.172:0/2984303490 shutdown_connections 2024-09-10T09:05:05.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.155+0000 7f445dcc6740 1 -- 172.21.15.172:0/2984303490 wait complete. 2024-09-10T09:05:05.175 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.155+0000 7f445dcc6740 20 client.20001 trim_cache size 0 max 16384 2024-09-10T09:05:05.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.172+0000 7f445dcc6740 1 Processor -- start 2024-09-10T09:05:05.180 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.172+0000 7f445dcc6740 1 -- start start 2024-09-10T09:05:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.172+0000 7f445dcc6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d842b60 0x56356d844cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.172+0000 7f445dcc6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d843480 0x56356d862bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.172+0000 7f445dcc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d84dd40 0x56356d863130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.172+0000 7f445dcc6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f44541b0d40 con 0x56356d842b60 2024-09-10T09:05:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.172+0000 7f445dcc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44541b2880 con 0x56356d84dd40 2024-09-10T09:05:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.172+0000 7f445dcc6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f44541b1410 con 0x56356d843480 2024-09-10T09:05:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.172+0000 7f4458e88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d842b60 0x56356d844cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.172+0000 7f4458e88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d842b60 0x56356d844cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60316/0 (socket says 172.21.15.172:60316) 2024-09-10T09:05:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.172+0000 7f4458e88640 1 -- 172.21.15.172:0/791333688 learned_addr learned my addr 172.21.15.172:0/791333688 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.172+0000 7f4459689640 1 --2- 172.21.15.172:0/791333688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d843480 0x56356d862bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.172+0000 7f4458e88640 1 -- 172.21.15.172:0/791333688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d843480 msgr2=0x56356d862bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:05.181 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.172+0000 7f4458e88640 1 --2- 172.21.15.172:0/791333688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d843480 0x56356d862bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.172+0000 7f4458e88640 1 -- 172.21.15.172:0/791333688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d84dd40 msgr2=0x56356d863130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.172+0000 7f4458e88640 1 --2- 172.21.15.172:0/791333688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d84dd40 0x56356d863130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.172+0000 7f4458e88640 1 -- 172.21.15.172:0/791333688 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56356d84d1e0 con 0x56356d842b60 2024-09-10T09:05:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.173+0000 7f4458e88640 1 --2- 172.21.15.172:0/791333688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d842b60 0x56356d844cc0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f44541024b0 tx=0x7f44440128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11631ce93e722b7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.173+0000 7f443effd640 1 -- 172.21.15.172:0/791333688 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444005020 con 0x56356d842b60 2024-09-10T09:05:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.173+0000 7f443effd640 1 -- 172.21.15.172:0/791333688 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44440047a0 con 0x56356d842b60 2024-09-10T09:05:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.173+0000 7f443effd640 1 -- 172.21.15.172:0/791333688 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444401f670 con 0x56356d842b60 2024-09-10T09:05:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.173+0000 7f445dcc6740 1 -- 172.21.15.172:0/791333688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d842b60 msgr2=0x56356d844cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.173+0000 7f445dcc6740 1 --2- 172.21.15.172:0/791333688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d842b60 0x56356d844cc0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f44541024b0 tx=0x7f44440128e0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.173+0000 7f445dcc6740 1 -- 172.21.15.172:0/791333688 shutdown_connections 2024-09-10T09:05:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.173+0000 7f445dcc6740 1 --2- 172.21.15.172:0/791333688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d84dd40 0x56356d863130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.182 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.173+0000 7f445dcc6740 1 --2- 172.21.15.172:0/791333688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d843480 0x56356d862bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.173+0000 7f445dcc6740 1 --2- 172.21.15.172:0/791333688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d842b60 0x56356d844cc0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.173+0000 7f445dcc6740 1 -- 172.21.15.172:0/791333688 >> 172.21.15.172:0/791333688 conn(0x56356d83f0f0 msgr2=0x56356d8445b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.173+0000 7f445dcc6740 1 -- 172.21.15.172:0/791333688 shutdown_connections 2024-09-10T09:05:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.173+0000 7f445dcc6740 1 -- 172.21.15.172:0/791333688 wait complete. 2024-09-10T09:05:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f445dcc6740 1 -- start start 2024-09-10T09:05:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f445dcc6740 1 Processor -- start 2024-09-10T09:05:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f445dcc6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d842b60 0x56356d860d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f445dcc6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d843480 0x56356d861260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f445dcc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d84dd40 0x56356d8617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f445dcc6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f44541b21b0 con 0x56356d842b60 2024-09-10T09:05:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f445dcc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44541b1ae0 con 0x56356d84dd40 2024-09-10T09:05:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f445dcc6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f44541aed20 con 0x56356d843480 2024-09-10T09:05:05.183 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f445dcc6740 10 client.0 _ll_register_callbacks cb 0x7f444400a640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f4458e88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d842b60 0x56356d860d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f4458e88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d842b60 0x56356d860d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60326/0 (socket says 172.21.15.172:60326) 2024-09-10T09:05:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f4458e88640 1 -- 172.21.15.172:0/3761085617 learned_addr learned my addr 172.21.15.172:0/3761085617 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f4459689640 1 --2- 172.21.15.172:0/3761085617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d843480 0x56356d861260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f4458e88640 1 -- 172.21.15.172:0/3761085617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d843480 msgr2=0x56356d861260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f4458e88640 1 --2- 172.21.15.172:0/3761085617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d843480 0x56356d861260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f4458e88640 1 -- 172.21.15.172:0/3761085617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d84dd40 msgr2=0x56356d8617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f4458e88640 1 --2- 172.21.15.172:0/3761085617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d84dd40 0x56356d8617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.174+0000 7f4458e88640 1 -- 172.21.15.172:0/3761085617 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56356d8c7410 con 0x56356d842b60 2024-09-10T09:05:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.175+0000 7f4458e88640 1 --2- 172.21.15.172:0/3761085617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d842b60 0x56356d860d20 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f44440058c0 tx=0x7f444400a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c558777a50b9522e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.175+0000 7f443cff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:05:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.175+0000 7f443cff9640 1 -- 172.21.15.172:0/3761085617 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44440052c0 con 0x56356d842b60 2024-09-10T09:05:05.184 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.175+0000 7f443cff9640 1 -- 172.21.15.172:0/3761085617 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4444005460 con 0x56356d842b60 2024-09-10T09:05:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.175+0000 7f445dcc6740 1 -- 172.21.15.172:0/3761085617 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44440047a0 con 0x56356d842b60 2024-09-10T09:05:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.175+0000 7f443cff9640 1 -- 172.21.15.172:0/3761085617 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f444401f410 con 0x56356d842b60 2024-09-10T09:05:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.175+0000 7f445dcc6740 10 client.20007 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.175+0000 7f445dcc6740 1 -- 172.21.15.172:0/3761085617 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56356d8c8750 con 0x56356d842b60 2024-09-10T09:05:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.175+0000 7f443cff9640 1 -- 172.21.15.172:0/3761085617 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44440047a0 con 0x56356d842b60 2024-09-10T09:05:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.175+0000 7f443cff9640 1 -- 172.21.15.172:0/3761085617 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f4444026780 con 0x56356d842b60 2024-09-10T09:05:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.175+0000 7f445dcc6740 10 client.20007 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.175+0000 7f445dcc6740 10 client.20007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.175+0000 7f445dcc6740 20 client.20007 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.175+0000 7f445dcc6740 1 --2- 172.21.15.172:0/3761085617 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56356d8c8d10 0x56356d8cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.175+0000 7f445dcc6740 4 client.20007 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:05:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.175+0000 7f445dcc6740 1 -- 172.21.15.172:0/3761085617 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56356d7afd00 con 0x56356d8c8d10 2024-09-10T09:05:05.185 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.176+0000 7f4459689640 1 --2- 172.21.15.172:0/3761085617 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56356d8c8d10 0x56356d8cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.176+0000 7f443cff9640 10 client.20007 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.176+0000 7f4459689640 1 --2- 172.21.15.172:0/3761085617 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56356d8c8d10 0x56356d8cb130 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bb8dc467c291426 server_cookie=44d6af85c1e19c0 in_seq=0 out_seq=0 2024-09-10T09:05:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.177+0000 7f443cff9640 1 -- 172.21.15.172:0/3761085617 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56356d7afd00 con 0x56356d8c8d10 2024-09-10T09:05:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.177+0000 7f443cff9640 10 client.20007 handle_command_reply: tid=0 2024-09-10T09:05:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.177+0000 7f445dcc6740 1 client.20007 shutdown 2024-09-10T09:05:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.177+0000 7f445dcc6740 1 -- 172.21.15.172:0/3761085617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d842b60 msgr2=0x56356d860d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.177+0000 7f445dcc6740 1 --2- 172.21.15.172:0/3761085617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d842b60 0x56356d860d20 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f44440058c0 tx=0x7f444400a2d0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.177+0000 7f445dcc6740 1 -- 172.21.15.172:0/3761085617 shutdown_connections 2024-09-10T09:05:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.177+0000 7f445dcc6740 1 --2- 172.21.15.172:0/3761085617 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x56356d8c8d10 0x56356d8cb130 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.177+0000 7f445dcc6740 1 --2- 172.21.15.172:0/3761085617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56356d84dd40 0x56356d8617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.177+0000 7f445dcc6740 1 --2- 172.21.15.172:0/3761085617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56356d843480 0x56356d861260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.177+0000 7f445dcc6740 1 --2- 172.21.15.172:0/3761085617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56356d842b60 0x56356d860d20 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.186 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.177+0000 7f445dcc6740 1 -- 172.21.15.172:0/3761085617 >> 172.21.15.172:0/3761085617 conn(0x56356d83f0f0 msgr2=0x56356d859530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.177+0000 7f443cff9640 0 client.20007 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.178+0000 7f445dcc6740 1 -- 172.21.15.172:0/3761085617 shutdown_connections 2024-09-10T09:05:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.178+0000 7f445dcc6740 1 -- 172.21.15.172:0/3761085617 wait complete. 2024-09-10T09:05:05.187 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.178+0000 7f445dcc6740 20 client.20007 trim_cache size 0 max 16384 2024-09-10T09:05:05.187 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:05:05.187 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T09:05:05.187 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "d75a8fbf-83db-49ca-b4a0-4e2a17f37d70", 2024-09-10T09:05:05.187 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T09:05:05.187 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:05:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.178+0000 7f445b8ed640 1 -- 172.21.15.172:0/3749927243 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44400a2b80 msgr2=0x7f44400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.178+0000 7f445b8ed640 1 --2- 172.21.15.172:0/3749927243 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44400a2b80 0x7f44400a5040 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f444c097cf0 tx=0x7f444c070040 comp rx=0 tx=0).stop 2024-09-10T09:05:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.178+0000 7f445b8ed640 1 -- 172.21.15.172:0/3749927243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f445410c1a0 msgr2=0x7f44541014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.178+0000 7f445b8ed640 1 --2- 172.21.15.172:0/3749927243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f445410c1a0 0x7f44541014a0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f444c00cad0 tx=0x7f444c00cf90 comp rx=0 tx=0).stop 2024-09-10T09:05:05.188 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.178+0000 7f445b8ed640 1 -- 172.21.15.172:0/3749927243 shutdown_connections 2024-09-10T09:05:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.178+0000 7f445b8ed640 1 --2- 172.21.15.172:0/3749927243 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f44400a2b80 0x7f44400a5040 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.178+0000 7f445b8ed640 1 --2- 172.21.15.172:0/3749927243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f445410c1a0 0x7f44541014a0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.178+0000 7f445b8ed640 1 --2- 172.21.15.172:0/3749927243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44541036e0 0x7f4454100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.178+0000 7f445b8ed640 1 --2- 172.21.15.172:0/3749927243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4454102dc0 0x7f4454100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.178+0000 7f445b8ed640 1 -- 172.21.15.172:0/3749927243 >> 172.21.15.172:0/3749927243 conn(0x7f44540fc7a0 msgr2=0x7f44540fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.178+0000 7f445b8ed640 1 -- 172.21.15.172:0/3749927243 shutdown_connections 2024-09-10T09:05:05.189 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:05.178+0000 7f445b8ed640 1 -- 172.21.15.172:0/3749927243 wait complete. 2024-09-10T09:05:05.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:05 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T09:05:05.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:05 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:05.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:05 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T09:05:05.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:05 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T09:05:05.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:05 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T09:05:05.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:05 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:05.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:05 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:05.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:05 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:05.882 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:05:05.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:05 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T09:05:05.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:05 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:05.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:05 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T09:05:05.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:05 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T09:05:05.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:05 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T09:05:05.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:05 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:05.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:05 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:05.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:05 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:05 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T09:05:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:05 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:05 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T09:05:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:05 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T09:05:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:05 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T09:05:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:05 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:05 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:06.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:05 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:06.132 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:05:06.642 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:06 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T09:05:06.642 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:06 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T09:05:06.642 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:06 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:06.642 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:06 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:06.642 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:06 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T09:05:06.642 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:06 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:06.643 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:06 smithi172 ceph-mon[30854]: pgmap v916: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 672 KiB/s wr, 181 op/s 2024-09-10T09:05:06.648 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.611+0000 7f9d5c12e640 1 -- 172.21.15.172:0/3686638793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d54101420 msgr2=0x7f9d5410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:06.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.611+0000 7f9d5c12e640 1 --2- 172.21.15.172:0/3686638793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d54101420 0x7f9d5410d950 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009cd0 tx=0x7f9d4c02a8e0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.612+0000 7f9d5c12e640 1 -- 172.21.15.172:0/3686638793 shutdown_connections 2024-09-10T09:05:06.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.612+0000 7f9d5c12e640 1 --2- 172.21.15.172:0/3686638793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d54101420 0x7f9d5410d950 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.612+0000 7f9d5c12e640 1 --2- 172.21.15.172:0/3686638793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d54100a80 0x7f9d54100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.612+0000 7f9d5c12e640 1 --2- 172.21.15.172:0/3686638793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d54106a80 0x7f9d54106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.612+0000 7f9d5c12e640 1 -- 172.21.15.172:0/3686638793 >> 172.21.15.172:0/3686638793 conn(0x7f9d540fc7a0 msgr2=0x7f9d540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:06.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.612+0000 7f9d5c12e640 1 -- 172.21.15.172:0/3686638793 shutdown_connections 2024-09-10T09:05:06.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.612+0000 7f9d5c12e640 1 -- 172.21.15.172:0/3686638793 wait complete. 2024-09-10T09:05:06.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.612+0000 7f9d5c12e640 1 Processor -- start 2024-09-10T09:05:06.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.612+0000 7f9d5c12e640 1 -- start start 2024-09-10T09:05:06.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.612+0000 7f9d5c12e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d54100a80 0x7f9d54198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.649 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.612+0000 7f9d5c12e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d54101420 0x7f9d541989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.612+0000 7f9d5c12e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d54106a80 0x7f9d5419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.612+0000 7f9d5c12e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9d54110dd0 con 0x7f9d54106a80 2024-09-10T09:05:06.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.612+0000 7f9d5c12e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d54110c50 con 0x7f9d54101420 2024-09-10T09:05:06.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.612+0000 7f9d5c12e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9d54110f50 con 0x7f9d54100a80 2024-09-10T09:05:06.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.613+0000 7f9d5a6cb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d54106a80 0x7f9d5419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:06.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.613+0000 7f9d5a6cb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d54106a80 0x7f9d5419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45318/0 (socket says 172.21.15.172:45318) 2024-09-10T09:05:06.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.613+0000 7f9d5a6cb640 1 -- 172.21.15.172:0/4051315005 learned_addr learned my addr 172.21.15.172:0/4051315005 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:06.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.613+0000 7f9d59eca640 1 --2- 172.21.15.172:0/4051315005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d54100a80 0x7f9d54198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:06.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.613+0000 7f9d596c9640 1 --2- 172.21.15.172:0/4051315005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d54101420 0x7f9d541989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:06.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.613+0000 7f9d5a6cb640 1 -- 172.21.15.172:0/4051315005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d54100a80 msgr2=0x7f9d54198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:06.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.613+0000 7f9d5a6cb640 1 --2- 172.21.15.172:0/4051315005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d54100a80 0x7f9d54198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.650 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.613+0000 7f9d5a6cb640 1 -- 172.21.15.172:0/4051315005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d54101420 msgr2=0x7f9d541989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:06.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.613+0000 7f9d5a6cb640 1 --2- 172.21.15.172:0/4051315005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d54101420 0x7f9d541989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.613+0000 7f9d5a6cb640 1 -- 172.21.15.172:0/4051315005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5419d3d0 con 0x7f9d54106a80 2024-09-10T09:05:06.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.613+0000 7f9d5a6cb640 1 --2- 172.21.15.172:0/4051315005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d54106a80 0x7f9d5419ccf0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c005cc0 tx=0x7f9d4c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:06.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.613+0000 7f9d46ffd640 1 -- 172.21.15.172:0/4051315005 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c004240 con 0x7f9d54106a80 2024-09-10T09:05:06.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.613+0000 7f9d46ffd640 1 -- 172.21.15.172:0/4051315005 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d4c0305a0 con 0x7f9d54106a80 2024-09-10T09:05:06.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.613+0000 7f9d5c12e640 1 -- 172.21.15.172:0/4051315005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d5419d660 con 0x7f9d54106a80 2024-09-10T09:05:06.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.613+0000 7f9d46ffd640 1 -- 172.21.15.172:0/4051315005 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0466c0 con 0x7f9d54106a80 2024-09-10T09:05:06.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.613+0000 7f9d5c12e640 1 -- 172.21.15.172:0/4051315005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d541a4fa0 con 0x7f9d54106a80 2024-09-10T09:05:06.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.616+0000 7f9d46ffd640 1 -- 172.21.15.172:0/4051315005 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9d4c030ac0 con 0x7f9d54106a80 2024-09-10T09:05:06.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.616+0000 7f9d5e507740 1 Processor -- start 2024-09-10T09:05:06.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.616+0000 7f9d5e507740 1 -- start start 2024-09-10T09:05:06.651 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d5e507740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29ea4a0 0x5637e29ea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d5e507740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29eadc0 0x5637e29f5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d5e507740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29f5680 0x5637e29f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d5e507740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5637e29502e0 con 0x5637e29ea4a0 2024-09-10T09:05:06.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d5e507740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5637e28f7ce0 con 0x5637e29eadc0 2024-09-10T09:05:06.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d5e507740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5637e294c2a0 con 0x5637e29f5680 2024-09-10T09:05:06.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d46ffd640 1 --2- 172.21.15.172:0/4051315005 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9d400a2c00 0x7f9d400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d596c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29ea4a0 0x5637e29ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:06.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d596c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29ea4a0 0x5637e29ea880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45324/0 (socket says 172.21.15.172:45324) 2024-09-10T09:05:06.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d596c9640 1 -- 172.21.15.172:0/4046503284 learned_addr learned my addr 172.21.15.172:0/4046503284 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:06.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d59eca640 1 --2- 172.21.15.172:0/4046503284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29eadc0 0x5637e29f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:06.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d5a6cb640 1 --2- 172.21.15.172:0/4051315005 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9d400a2c00 0x7f9d400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:06.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d596c9640 1 --2- 172.21.15.172:0/4046503284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29f5680 0x5637e29f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:06.652 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d46ffd640 1 -- 172.21.15.172:0/4051315005 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9d4c02e030 con 0x7f9d54106a80 2024-09-10T09:05:06.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d596c9640 1 -- 172.21.15.172:0/4046503284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29f5680 msgr2=0x5637e29f7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:06.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d596c9640 1 --2- 172.21.15.172:0/4046503284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29f5680 0x5637e29f7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d596c9640 1 -- 172.21.15.172:0/4046503284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29eadc0 msgr2=0x5637e29f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:06.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d596c9640 1 --2- 172.21.15.172:0/4046503284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29eadc0 0x5637e29f5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d596c9640 1 -- 172.21.15.172:0/4046503284 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637e28dc440 con 0x5637e29ea4a0 2024-09-10T09:05:06.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d596c9640 1 --2- 172.21.15.172:0/4046503284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29ea4a0 0x5637e29ea880 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f9d48006aa0 tx=0x7f9d480095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb551786b1a5af5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:06.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.617+0000 7f9d5a6cb640 1 --2- 172.21.15.172:0/4051315005 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9d400a2c00 0x7f9d400a50c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0b91b0 tx=0x7f9d4c08fed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:06.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.618+0000 7f9d3ffff640 1 -- 172.21.15.172:0/4046503284 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48017610 con 0x5637e29ea4a0 2024-09-10T09:05:06.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.618+0000 7f9d3ffff640 1 -- 172.21.15.172:0/4046503284 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d4801c070 con 0x5637e29ea4a0 2024-09-10T09:05:06.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.618+0000 7f9d3ffff640 1 -- 172.21.15.172:0/4046503284 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d480158a0 con 0x5637e29ea4a0 2024-09-10T09:05:06.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.618+0000 7f9d5e507740 1 -- 172.21.15.172:0/4046503284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29ea4a0 msgr2=0x5637e29ea880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:06.653 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.618+0000 7f9d5e507740 1 --2- 172.21.15.172:0/4046503284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29ea4a0 0x5637e29ea880 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f9d48006aa0 tx=0x7f9d480095d0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.618+0000 7f9d5e507740 1 -- 172.21.15.172:0/4046503284 shutdown_connections 2024-09-10T09:05:06.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.618+0000 7f9d5e507740 1 --2- 172.21.15.172:0/4046503284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29f5680 0x5637e29f7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.618+0000 7f9d5e507740 1 --2- 172.21.15.172:0/4046503284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29eadc0 0x5637e29f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.618+0000 7f9d5e507740 1 --2- 172.21.15.172:0/4046503284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29ea4a0 0x5637e29ea880 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.618+0000 7f9d5e507740 1 -- 172.21.15.172:0/4046503284 >> 172.21.15.172:0/4046503284 conn(0x5637e29e6a30 msgr2=0x5637e29e6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:06.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.618+0000 7f9d5e507740 1 -- 172.21.15.172:0/4046503284 shutdown_connections 2024-09-10T09:05:06.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.618+0000 7f9d5e507740 1 -- 172.21.15.172:0/4046503284 wait complete. 2024-09-10T09:05:06.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d5e507740 1 -- start start 2024-09-10T09:05:06.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d5e507740 1 Processor -- start 2024-09-10T09:05:06.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d5e507740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29ea4a0 0x5637e29f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d5e507740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29eadc0 0x5637e29f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d5e507740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29f5680 0x5637e29f2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.654 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d5e507740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5637e29f8250 con 0x5637e29f5680 2024-09-10T09:05:06.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d5e507740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5637e290b050 con 0x5637e29eadc0 2024-09-10T09:05:06.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d5e507740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5637e29f83d0 con 0x5637e29ea4a0 2024-09-10T09:05:06.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d5e507740 10 client.0 _ll_register_callbacks cb 0x5637e294b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:06.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d596c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29f5680 0x5637e29f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:06.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d596c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29f5680 0x5637e29f2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45332/0 (socket says 172.21.15.172:45332) 2024-09-10T09:05:06.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d596c9640 1 -- 172.21.15.172:0/864885220 learned_addr learned my addr 172.21.15.172:0/864885220 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:06.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d59eca640 1 --2- 172.21.15.172:0/864885220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29eadc0 0x5637e29f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:06.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d596c9640 1 --2- 172.21.15.172:0/864885220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29ea4a0 0x5637e29f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:06.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d596c9640 1 -- 172.21.15.172:0/864885220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29ea4a0 msgr2=0x5637e29f4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:06.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d596c9640 1 --2- 172.21.15.172:0/864885220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29ea4a0 0x5637e29f4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d596c9640 1 -- 172.21.15.172:0/864885220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29eadc0 msgr2=0x5637e29f21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:06.655 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.619+0000 7f9d596c9640 1 --2- 172.21.15.172:0/864885220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29eadc0 0x5637e29f21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.620+0000 7f9d596c9640 1 -- 172.21.15.172:0/864885220 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637e26c13c0 con 0x5637e29f5680 2024-09-10T09:05:06.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.620+0000 7f9d596c9640 1 --2- 172.21.15.172:0/864885220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29f5680 0x5637e29f2720 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f9d48006d30 tx=0x7f9d48015480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acd67eb9b8e8dd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:06.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.620+0000 7f9d3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:05:06.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.620+0000 7f9d3d7fa640 1 -- 172.21.15.172:0/864885220 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48017960 con 0x5637e29f5680 2024-09-10T09:05:06.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.620+0000 7f9d3d7fa640 1 -- 172.21.15.172:0/864885220 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9d48017b00 con 0x5637e29f5680 2024-09-10T09:05:06.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.620+0000 7f9d5e507740 1 -- 172.21.15.172:0/864885220 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d4801c070 con 0x5637e29f5680 2024-09-10T09:05:06.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.620+0000 7f9d3d7fa640 1 -- 172.21.15.172:0/864885220 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9d4802d650 con 0x5637e29f5680 2024-09-10T09:05:06.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.620+0000 7f9d5e507740 10 client.20019 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:06.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.620+0000 7f9d5e507740 1 -- 172.21.15.172:0/864885220 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5637e2a03b70 con 0x5637e29f5680 2024-09-10T09:05:06.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.620+0000 7f9d3d7fa640 1 -- 172.21.15.172:0/864885220 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d4801c070 con 0x5637e29f5680 2024-09-10T09:05:06.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.620+0000 7f9d3d7fa640 1 -- 172.21.15.172:0/864885220 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f9d48020420 con 0x5637e29f5680 2024-09-10T09:05:06.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.620+0000 7f9d5e507740 10 client.20019 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:06.656 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.620+0000 7f9d5e507740 10 client.20019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:06.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.620+0000 7f9d5e507740 20 client.20019 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:06.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.620+0000 7f9d5e507740 1 --2- 172.21.15.172:0/864885220 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5637e2a04ec0 0x5637e2a072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.620+0000 7f9d5e507740 4 client.20019 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:05:06.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.620+0000 7f9d5e507740 1 -- 172.21.15.172:0/864885220 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637e295d850 con 0x5637e2a04ec0 2024-09-10T09:05:06.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.621+0000 7f9d59eca640 1 --2- 172.21.15.172:0/864885220 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5637e2a04ec0 0x5637e2a072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:06.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.621+0000 7f9d3d7fa640 10 client.20019 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:06.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.621+0000 7f9d59eca640 1 --2- 172.21.15.172:0/864885220 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5637e2a04ec0 0x5637e2a072b0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6f4fdfa60cbb4d5 server_cookie=14625f5c3b3c3ccd in_seq=0 out_seq=0 2024-09-10T09:05:06.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.622+0000 7f9d3d7fa640 1 -- 172.21.15.172:0/864885220 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5637e295d850 con 0x5637e2a04ec0 2024-09-10T09:05:06.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.622+0000 7f9d3d7fa640 10 client.20019 handle_command_reply: tid=0 2024-09-10T09:05:06.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.622+0000 7f9d5e507740 1 client.20019 shutdown 2024-09-10T09:05:06.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.622+0000 7f9d5e507740 1 -- 172.21.15.172:0/864885220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29f5680 msgr2=0x5637e29f2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:06.657 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.622+0000 7f9d5e507740 1 --2- 172.21.15.172:0/864885220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29f5680 0x5637e29f2720 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f9d48006d30 tx=0x7f9d48015480 comp rx=0 tx=0).stop 2024-09-10T09:05:06.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.622+0000 7f9d5e507740 1 -- 172.21.15.172:0/864885220 shutdown_connections 2024-09-10T09:05:06.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.622+0000 7f9d5e507740 1 --2- 172.21.15.172:0/864885220 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5637e2a04ec0 0x5637e2a072b0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.622+0000 7f9d5e507740 1 --2- 172.21.15.172:0/864885220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29f5680 0x5637e29f2720 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.622+0000 7f9d5e507740 1 --2- 172.21.15.172:0/864885220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29eadc0 0x5637e29f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.622+0000 7f9d3d7fa640 0 client.20019 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:06.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.622+0000 7f9d5e507740 1 --2- 172.21.15.172:0/864885220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29ea4a0 0x5637e29f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.622+0000 7f9d5e507740 1 -- 172.21.15.172:0/864885220 >> 172.21.15.172:0/864885220 conn(0x5637e29e6a30 msgr2=0x5637e29ec760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:06.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.622+0000 7f9d5e507740 1 -- 172.21.15.172:0/864885220 shutdown_connections 2024-09-10T09:05:06.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.622+0000 7f9d5e507740 1 -- 172.21.15.172:0/864885220 wait complete. 2024-09-10T09:05:06.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.622+0000 7f9d5e507740 20 client.20019 trim_cache size 0 max 16384 2024-09-10T09:05:06.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d5e507740 1 Processor -- start 2024-09-10T09:05:06.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d5e507740 1 -- start start 2024-09-10T09:05:06.658 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d5e507740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29e6a30 0x5637e2a0a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d5e507740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29ea4a0 0x5637e2a0a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d5e507740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29eadc0 0x5637e29ef440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d5e507740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541aea80 con 0x5637e29e6a30 2024-09-10T09:05:06.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d5e507740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541b05c0 con 0x5637e29eadc0 2024-09-10T09:05:06.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d5e507740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541af150 con 0x5637e29ea4a0 2024-09-10T09:05:06.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d596c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29e6a30 0x5637e2a0a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:06.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d596c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29e6a30 0x5637e2a0a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45342/0 (socket says 172.21.15.172:45342) 2024-09-10T09:05:06.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d596c9640 1 -- 172.21.15.172:0/2261313660 learned_addr learned my addr 172.21.15.172:0/2261313660 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:06.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d59eca640 1 --2- 172.21.15.172:0/2261313660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29ea4a0 0x5637e2a0a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:06.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d596c9640 1 --2- 172.21.15.172:0/2261313660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29eadc0 0x5637e29ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:06.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d596c9640 1 -- 172.21.15.172:0/2261313660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29ea4a0 msgr2=0x5637e2a0a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:06.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d596c9640 1 --2- 172.21.15.172:0/2261313660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29ea4a0 0x5637e2a0a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.659 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d596c9640 1 -- 172.21.15.172:0/2261313660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29eadc0 msgr2=0x5637e29ef440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:06.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d596c9640 1 --2- 172.21.15.172:0/2261313660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29eadc0 0x5637e29ef440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d596c9640 1 -- 172.21.15.172:0/2261313660 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637e29e6860 con 0x5637e29e6a30 2024-09-10T09:05:06.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d59eca640 1 --2- 172.21.15.172:0/2261313660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29ea4a0 0x5637e2a0a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:05:06.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.640+0000 7f9d596c9640 1 --2- 172.21.15.172:0/2261313660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29e6a30 0x5637e2a0a2d0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f9d48006aa0 tx=0x7f9d48006a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36718003d1848ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:06.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.641+0000 7f9d3f7fe640 1 -- 172.21.15.172:0/2261313660 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4802d430 con 0x5637e29e6a30 2024-09-10T09:05:06.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.641+0000 7f9d3f7fe640 1 -- 172.21.15.172:0/2261313660 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d4801c040 con 0x5637e29e6a30 2024-09-10T09:05:06.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.641+0000 7f9d3f7fe640 1 -- 172.21.15.172:0/2261313660 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48020d60 con 0x5637e29e6a30 2024-09-10T09:05:06.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.641+0000 7f9d5e507740 1 -- 172.21.15.172:0/2261313660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29e6a30 msgr2=0x5637e2a0a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:06.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.641+0000 7f9d5e507740 1 --2- 172.21.15.172:0/2261313660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29e6a30 0x5637e2a0a2d0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f9d48006aa0 tx=0x7f9d48006a30 comp rx=0 tx=0).stop 2024-09-10T09:05:06.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.641+0000 7f9d5e507740 1 -- 172.21.15.172:0/2261313660 shutdown_connections 2024-09-10T09:05:06.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.641+0000 7f9d5e507740 1 --2- 172.21.15.172:0/2261313660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29eadc0 0x5637e29ef440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.641+0000 7f9d5e507740 1 --2- 172.21.15.172:0/2261313660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29ea4a0 0x5637e2a0a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.641+0000 7f9d5e507740 1 --2- 172.21.15.172:0/2261313660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29e6a30 0x5637e2a0a2d0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.641+0000 7f9d5e507740 1 -- 172.21.15.172:0/2261313660 >> 172.21.15.172:0/2261313660 conn(0x5637e29f2880 msgr2=0x5637e29f12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:06.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.641+0000 7f9d5e507740 1 -- 172.21.15.172:0/2261313660 shutdown_connections 2024-09-10T09:05:06.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.642+0000 7f9d5e507740 1 -- 172.21.15.172:0/2261313660 wait complete. 2024-09-10T09:05:06.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.642+0000 7f9d5e507740 1 -- start start 2024-09-10T09:05:06.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.642+0000 7f9d5e507740 1 Processor -- start 2024-09-10T09:05:06.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.642+0000 7f9d5e507740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29e6a30 0x5637e2a0abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.642+0000 7f9d5e507740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29ea4a0 0x5637e2a07f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.642+0000 7f9d5e507740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29eadc0 0x5637e2a08690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.661 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.642+0000 7f9d5e507740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541afef0 con 0x5637e29eadc0 2024-09-10T09:05:06.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.642+0000 7f9d5e507740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541af820 con 0x5637e29ea4a0 2024-09-10T09:05:06.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.642+0000 7f9d5e507740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541aca60 con 0x5637e29e6a30 2024-09-10T09:05:06.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.642+0000 7f9d5e507740 10 client.0 _ll_register_callbacks cb 0x7f9d500093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:06.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.642+0000 7f9d596c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29eadc0 0x5637e2a08690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:06.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.642+0000 7f9d596c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29eadc0 0x5637e2a08690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45344/0 (socket says 172.21.15.172:45344) 2024-09-10T09:05:06.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.642+0000 7f9d596c9640 1 -- 172.21.15.172:0/819294769 learned_addr learned my addr 172.21.15.172:0/819294769 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:06.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d596c9640 1 -- 172.21.15.172:0/819294769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29e6a30 msgr2=0x5637e2a0abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:06.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d596c9640 1 --2- 172.21.15.172:0/819294769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29e6a30 0x5637e2a0abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d596c9640 1 -- 172.21.15.172:0/819294769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29ea4a0 msgr2=0x5637e2a07f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:06.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d596c9640 1 --2- 172.21.15.172:0/819294769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29ea4a0 0x5637e2a07f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d596c9640 1 -- 172.21.15.172:0/819294769 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637e29e76c0 con 0x5637e29eadc0 2024-09-10T09:05:06.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d596c9640 1 --2- 172.21.15.172:0/819294769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29eadc0 0x5637e2a08690 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009330 tx=0x7f9d480046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e609f9f39a8ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:06.662 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:05:06.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d3d7fa640 1 -- 172.21.15.172:0/819294769 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48017b70 con 0x5637e29eadc0 2024-09-10T09:05:06.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d3d7fa640 1 -- 172.21.15.172:0/819294769 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9d48017d10 con 0x5637e29eadc0 2024-09-10T09:05:06.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d5e507740 1 -- 172.21.15.172:0/819294769 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d4801c040 con 0x5637e29eadc0 2024-09-10T09:05:06.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d3d7fa640 1 -- 172.21.15.172:0/819294769 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9d48020960 con 0x5637e29eadc0 2024-09-10T09:05:06.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d5e507740 10 client.20025 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:06.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d5e507740 1 -- 172.21.15.172:0/819294769 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5637e2a6f7d0 con 0x5637e29eadc0 2024-09-10T09:05:06.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d3d7fa640 1 -- 172.21.15.172:0/819294769 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d4801c040 con 0x5637e29eadc0 2024-09-10T09:05:06.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d3d7fa640 1 -- 172.21.15.172:0/819294769 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f9d48034070 con 0x5637e29eadc0 2024-09-10T09:05:06.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d5e507740 10 client.20025 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:06.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d5e507740 10 client.20025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:06.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d5e507740 20 client.20025 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:06.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d5e507740 1 --2- 172.21.15.172:0/819294769 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5637e29f5680 0x5637e2a71ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:06.663 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d5e507740 4 client.20025 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:05:06.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.643+0000 7f9d5e507740 1 -- 172.21.15.172:0/819294769 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5637e2957d60 con 0x5637e29f5680 2024-09-10T09:05:06.664 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.644+0000 7f9d59eca640 1 --2- 172.21.15.172:0/819294769 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5637e29f5680 0x5637e2a71ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:06.664 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:05:06.664 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:05:06.664 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T09:05:06.664 INFO:teuthology.orchestra.run.smithi172.stdout: "d75a8fbf-83db-49ca-b4a0-4e2a17f37d70": { 2024-09-10T09:05:06.664 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T09:05:06.664 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "d75a8fbf-83db-49ca-b4a0-4e2a17f37d70", 2024-09-10T09:05:06.665 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T09:05:06.665 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:05:06.665 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:05:06.665 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:05:06.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.644+0000 7f9d3d7fa640 10 client.20025 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:06.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.644+0000 7f9d59eca640 1 --2- 172.21.15.172:0/819294769 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5637e29f5680 0x5637e2a71ef0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7f0de8e8d869e45 server_cookie=3a575984900f6cf1 in_seq=0 out_seq=0 2024-09-10T09:05:06.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.644+0000 7f9d3d7fa640 1 -- 172.21.15.172:0/819294769 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5637e2957d60 con 0x5637e29f5680 2024-09-10T09:05:06.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.644+0000 7f9d3d7fa640 10 client.20025 handle_command_reply: tid=0 2024-09-10T09:05:06.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.644+0000 7f9d5e507740 1 client.20025 shutdown 2024-09-10T09:05:06.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.644+0000 7f9d5e507740 1 -- 172.21.15.172:0/819294769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29eadc0 msgr2=0x5637e2a08690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:06.665 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.644+0000 7f9d5e507740 1 --2- 172.21.15.172:0/819294769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29eadc0 0x5637e2a08690 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009330 tx=0x7f9d480046b0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.644+0000 7f9d5e507740 1 -- 172.21.15.172:0/819294769 shutdown_connections 2024-09-10T09:05:06.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.644+0000 7f9d5e507740 1 --2- 172.21.15.172:0/819294769 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5637e29f5680 0x5637e2a71ef0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.644+0000 7f9d5e507740 1 --2- 172.21.15.172:0/819294769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5637e29eadc0 0x5637e2a08690 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.644+0000 7f9d5e507740 1 --2- 172.21.15.172:0/819294769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637e29ea4a0 0x5637e2a07f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.644+0000 7f9d3d7fa640 0 client.20025 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:06.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.644+0000 7f9d5e507740 1 --2- 172.21.15.172:0/819294769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5637e29e6a30 0x5637e2a0abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.644+0000 7f9d5e507740 1 -- 172.21.15.172:0/819294769 >> 172.21.15.172:0/819294769 conn(0x5637e29f2880 msgr2=0x5637e29f3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:06.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.645+0000 7f9d5e507740 1 -- 172.21.15.172:0/819294769 shutdown_connections 2024-09-10T09:05:06.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.645+0000 7f9d5e507740 1 -- 172.21.15.172:0/819294769 wait complete. 2024-09-10T09:05:06.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.645+0000 7f9d5e507740 20 client.20025 trim_cache size 0 max 16384 2024-09-10T09:05:06.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.645+0000 7f9d5c12e640 1 -- 172.21.15.172:0/4051315005 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9d400a2c00 msgr2=0x7f9d400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:06.666 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.645+0000 7f9d5c12e640 1 --2- 172.21.15.172:0/4051315005 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9d400a2c00 0x7f9d400a50c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c0b91b0 tx=0x7f9d4c08fed0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.645+0000 7f9d5c12e640 1 -- 172.21.15.172:0/4051315005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d54106a80 msgr2=0x7f9d5419ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:06.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.645+0000 7f9d5c12e640 1 --2- 172.21.15.172:0/4051315005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d54106a80 0x7f9d5419ccf0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c005cc0 tx=0x7f9d4c003ba0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.645+0000 7f9d5c12e640 1 -- 172.21.15.172:0/4051315005 shutdown_connections 2024-09-10T09:05:06.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.645+0000 7f9d5c12e640 1 --2- 172.21.15.172:0/4051315005 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9d400a2c00 0x7f9d400a50c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.645+0000 7f9d5c12e640 1 --2- 172.21.15.172:0/4051315005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d54106a80 0x7f9d5419ccf0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.645+0000 7f9d5c12e640 1 --2- 172.21.15.172:0/4051315005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d54101420 0x7f9d541989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.645+0000 7f9d5c12e640 1 --2- 172.21.15.172:0/4051315005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d54100a80 0x7f9d54198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:06.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.645+0000 7f9d5c12e640 1 -- 172.21.15.172:0/4051315005 >> 172.21.15.172:0/4051315005 conn(0x7f9d540fc7a0 msgr2=0x7f9d5406ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:06.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.645+0000 7f9d5c12e640 1 -- 172.21.15.172:0/4051315005 shutdown_connections 2024-09-10T09:05:06.667 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:06.645+0000 7f9d5c12e640 1 -- 172.21.15.172:0/4051315005 wait complete. 2024-09-10T09:05:06.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:06 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T09:05:06.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:06 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T09:05:06.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:06 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:06.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:06 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:06.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:06 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T09:05:06.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:06 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:06.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:06 smithi174 ceph-mon[34068]: pgmap v916: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 672 KiB/s wr, 181 op/s 2024-09-10T09:05:07.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:06 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T09:05:07.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:06 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T09:05:07.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:06 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:07.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:06 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:07.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:06 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T09:05:07.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:06 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:07.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:06 smithi191 ceph-mon[35314]: pgmap v916: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 672 KiB/s wr, 181 op/s 2024-09-10T09:05:07.325 INFO:tasks.cephfs.filesystem:scrub status for tag:d75a8fbf-83db-49ca-b4a0-4e2a17f37d70 - {'path': '/', 'tag': 'd75a8fbf-83db-49ca-b4a0-4e2a17f37d70', 'options': 'recursive,force'} 2024-09-10T09:05:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.214+0000 7fa441f8d640 1 -- 172.21.15.172:0/3846461577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa43c101b60 msgr2=0x7fa43c101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.214+0000 7fa441f8d640 1 --2- 172.21.15.172:0/3846461577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa43c101b60 0x7fa43c101f60 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009900 tx=0x7fa42c02a410 comp rx=0 tx=0).stop 2024-09-10T09:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.214+0000 7fa441f8d640 1 -- 172.21.15.172:0/3846461577 shutdown_connections 2024-09-10T09:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.214+0000 7fa441f8d640 1 --2- 172.21.15.172:0/3846461577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa43c103710 0x7fa43c109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.214+0000 7fa441f8d640 1 --2- 172.21.15.172:0/3846461577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa43c102d50 0x7fa43c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.214+0000 7fa441f8d640 1 --2- 172.21.15.172:0/3846461577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa43c101b60 0x7fa43c101f60 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.214+0000 7fa441f8d640 1 -- 172.21.15.172:0/3846461577 >> 172.21.15.172:0/3846461577 conn(0x7fa43c0fd2e0 msgr2=0x7fa43c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.214+0000 7fa441f8d640 1 -- 172.21.15.172:0/3846461577 shutdown_connections 2024-09-10T09:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.214+0000 7fa441f8d640 1 -- 172.21.15.172:0/3846461577 wait complete. 2024-09-10T09:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.214+0000 7fa441f8d640 1 Processor -- start 2024-09-10T09:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.214+0000 7fa441f8d640 1 -- start start 2024-09-10T09:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.214+0000 7fa441f8d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa43c101b60 0x7fa43c197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.214+0000 7fa441f8d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa43c102d50 0x7fa43c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.214+0000 7fa441f8d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa43c103710 0x7fa43c19f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.214+0000 7fa441f8d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c10d6f0 con 0x7fa43c103710 2024-09-10T09:05:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.214+0000 7fa441f8d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c10d570 con 0x7fa43c102d50 2024-09-10T09:05:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.214+0000 7fa441f8d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c10d870 con 0x7fa43c101b60 2024-09-10T09:05:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.215+0000 7fa43bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa43c103710 0x7fa43c19f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.215+0000 7fa43bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa43c103710 0x7fa43c19f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45366/0 (socket says 172.21.15.172:45366) 2024-09-10T09:05:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.215+0000 7fa43bfff640 1 -- 172.21.15.172:0/2432318892 learned_addr learned my addr 172.21.15.172:0/2432318892 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.215+0000 7fa43b7fe640 1 --2- 172.21.15.172:0/2432318892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa43c101b60 0x7fa43c197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.215+0000 7fa43affd640 1 --2- 172.21.15.172:0/2432318892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa43c102d50 0x7fa43c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.215+0000 7fa43bfff640 1 -- 172.21.15.172:0/2432318892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa43c101b60 msgr2=0x7fa43c197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.215+0000 7fa43bfff640 1 --2- 172.21.15.172:0/2432318892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa43c101b60 0x7fa43c197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.215+0000 7fa43bfff640 1 -- 172.21.15.172:0/2432318892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa43c102d50 msgr2=0x7fa43c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.215+0000 7fa43bfff640 1 --2- 172.21.15.172:0/2432318892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa43c102d50 0x7fa43c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.215+0000 7fa43bfff640 1 -- 172.21.15.172:0/2432318892 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43c19f970 con 0x7fa43c103710 2024-09-10T09:05:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.215+0000 7fa43bfff640 1 --2- 172.21.15.172:0/2432318892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa43c103710 0x7fa43c19f300 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fa43400b970 tx=0x7fa43400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.215+0000 7fa438ff9640 1 -- 172.21.15.172:0/2432318892 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434019070 con 0x7fa43c103710 2024-09-10T09:05:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.215+0000 7fa438ff9640 1 -- 172.21.15.172:0/2432318892 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa434014440 con 0x7fa43c103710 2024-09-10T09:05:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.215+0000 7fa441f8d640 1 -- 172.21.15.172:0/2432318892 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43c19fc60 con 0x7fa43c103710 2024-09-10T09:05:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.215+0000 7fa438ff9640 1 -- 172.21.15.172:0/2432318892 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4340075d0 con 0x7fa43c103710 2024-09-10T09:05:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.215+0000 7fa441f8d640 1 -- 172.21.15.172:0/2432318892 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43c1a0160 con 0x7fa43c103710 2024-09-10T09:05:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.217+0000 7fa438ff9640 1 -- 172.21.15.172:0/2432318892 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa43401c070 con 0x7fa43c103710 2024-09-10T09:05:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa444e04740 1 Processor -- start 2024-09-10T09:05:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa444e04740 1 -- start start 2024-09-10T09:05:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa444e04740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fd59a0 0x5569c1fd5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa444e04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fd62e0 0x5569c1fdada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa444e04740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fdb370 0x5569c1fdd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa444e04740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5569c1f392f0 con 0x5569c1fd62e0 2024-09-10T09:05:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa444e04740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5569c1ee1270 con 0x5569c1fd59a0 2024-09-10T09:05:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa444e04740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5569c1f352b0 con 0x5569c1fdb370 2024-09-10T09:05:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa43b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fd62e0 0x5569c1fdada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa43b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fd62e0 0x5569c1fdada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45380/0 (socket says 172.21.15.172:45380) 2024-09-10T09:05:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa43b7fe640 1 -- 172.21.15.172:0/3014335617 learned_addr learned my addr 172.21.15.172:0/3014335617 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa43affd640 1 --2- 172.21.15.172:0/3014335617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fd59a0 0x5569c1fd5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa43affd640 1 --2- 172.21.15.172:0/3014335617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fdb370 0x5569c1fdd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa438ff9640 1 --2- 172.21.15.172:0/2432318892 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4280a2c30 0x7fa4280a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa43b7fe640 1 -- 172.21.15.172:0/3014335617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fdb370 msgr2=0x5569c1fdd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa43b7fe640 1 --2- 172.21.15.172:0/3014335617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fdb370 0x5569c1fdd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa43b7fe640 1 -- 172.21.15.172:0/3014335617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fd59a0 msgr2=0x5569c1fd5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa43b7fe640 1 --2- 172.21.15.172:0/3014335617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fd59a0 0x5569c1fd5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa43b7fe640 1 -- 172.21.15.172:0/3014335617 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569c1ee0340 con 0x5569c1fd62e0 2024-09-10T09:05:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa43affd640 1 --2- 172.21.15.172:0/3014335617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fd59a0 0x5569c1fd5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:05:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa43bfff640 1 --2- 172.21.15.172:0/2432318892 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4280a2c30 0x7fa4280a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.218+0000 7fa438ff9640 1 -- 172.21.15.172:0/2432318892 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa4340cea20 con 0x7fa43c103710 2024-09-10T09:05:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.219+0000 7fa43b7fe640 1 --2- 172.21.15.172:0/3014335617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fd62e0 0x5569c1fdada0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009900 tx=0x7fa42c032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675c7ed535a55ad6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.219+0000 7fa431ffb640 1 -- 172.21.15.172:0/3014335617 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c03f070 con 0x5569c1fd62e0 2024-09-10T09:05:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.219+0000 7fa431ffb640 1 -- 172.21.15.172:0/3014335617 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa42c031780 con 0x5569c1fd62e0 2024-09-10T09:05:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.219+0000 7fa431ffb640 1 -- 172.21.15.172:0/3014335617 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c035070 con 0x5569c1fd62e0 2024-09-10T09:05:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.219+0000 7fa444e04740 1 -- 172.21.15.172:0/3014335617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fd62e0 msgr2=0x5569c1fdada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.219+0000 7fa43bfff640 1 --2- 172.21.15.172:0/2432318892 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4280a2c30 0x7fa4280a50f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa43409ac20 tx=0x7fa434090400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.219+0000 7fa444e04740 1 --2- 172.21.15.172:0/3014335617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fd62e0 0x5569c1fdada0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009900 tx=0x7fa42c032cc0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.219+0000 7fa444e04740 1 -- 172.21.15.172:0/3014335617 shutdown_connections 2024-09-10T09:05:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.219+0000 7fa444e04740 1 --2- 172.21.15.172:0/3014335617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fdb370 0x5569c1fdd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.219+0000 7fa444e04740 1 --2- 172.21.15.172:0/3014335617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fd62e0 0x5569c1fdada0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.219+0000 7fa444e04740 1 --2- 172.21.15.172:0/3014335617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fd59a0 0x5569c1fd5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.219+0000 7fa444e04740 1 -- 172.21.15.172:0/3014335617 >> 172.21.15.172:0/3014335617 conn(0x5569c1fd1010 msgr2=0x5569c1fd1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.219+0000 7fa444e04740 1 -- 172.21.15.172:0/3014335617 shutdown_connections 2024-09-10T09:05:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.220+0000 7fa444e04740 1 -- 172.21.15.172:0/3014335617 wait complete. 2024-09-10T09:05:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.220+0000 7fa444e04740 1 -- start start 2024-09-10T09:05:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.220+0000 7fa444e04740 1 Processor -- start 2024-09-10T09:05:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.220+0000 7fa444e04740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fd59a0 0x5569c1fdfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.220+0000 7fa444e04740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fd62e0 0x5569c1fe0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.220+0000 7fa444e04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fdb370 0x5569c1fe70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.220+0000 7fa444e04740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5569c1fde2e0 con 0x5569c1fdb370 2024-09-10T09:05:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.220+0000 7fa444e04740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5569c1ec2290 con 0x5569c1fd59a0 2024-09-10T09:05:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.220+0000 7fa444e04740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5569c1fde460 con 0x5569c1fd62e0 2024-09-10T09:05:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.220+0000 7fa444e04740 10 client.0 _ll_register_callbacks cb 0x5569c1ef3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.220+0000 7fa43affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fdb370 0x5569c1fe70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.220+0000 7fa43affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fdb370 0x5569c1fe70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45384/0 (socket says 172.21.15.172:45384) 2024-09-10T09:05:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.220+0000 7fa43affd640 1 -- 172.21.15.172:0/2238551344 learned_addr learned my addr 172.21.15.172:0/2238551344 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.220+0000 7fa43affd640 1 --2- 172.21.15.172:0/2238551344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fd59a0 0x5569c1fdfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.220+0000 7fa43b7fe640 1 --2- 172.21.15.172:0/2238551344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fd62e0 0x5569c1fe0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa43affd640 1 -- 172.21.15.172:0/2238551344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fd62e0 msgr2=0x5569c1fe0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa43affd640 1 --2- 172.21.15.172:0/2238551344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fd62e0 0x5569c1fe0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa43affd640 1 -- 172.21.15.172:0/2238551344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fd59a0 msgr2=0x5569c1fdfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa43affd640 1 --2- 172.21.15.172:0/2238551344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fd59a0 0x5569c1fdfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa43affd640 1 -- 172.21.15.172:0/2238551344 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569c1c4c210 con 0x5569c1fdb370 2024-09-10T09:05:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa43b7fe640 1 --2- 172.21.15.172:0/2238551344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fd62e0 0x5569c1fe0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:05:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa43affd640 1 --2- 172.21.15.172:0/2238551344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fdb370 0x5569c1fe70b0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fa42400c780 tx=0x7fa424016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3def030135a165c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa41affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:05:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa41affd640 1 -- 172.21.15.172:0/2238551344 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424004220 con 0x5569c1fdb370 2024-09-10T09:05:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa41affd640 1 -- 172.21.15.172:0/2238551344 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa4240043c0 con 0x5569c1fdb370 2024-09-10T09:05:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa444e04740 1 -- 172.21.15.172:0/2238551344 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa42c031780 con 0x5569c1fdb370 2024-09-10T09:05:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa41affd640 1 -- 172.21.15.172:0/2238551344 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa424026050 con 0x5569c1fdb370 2024-09-10T09:05:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa444e04740 10 client.20037 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa444e04740 1 -- 172.21.15.172:0/2238551344 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5569c1d18620 con 0x5569c1fdb370 2024-09-10T09:05:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa41affd640 1 -- 172.21.15.172:0/2238551344 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa42c031780 con 0x5569c1fdb370 2024-09-10T09:05:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa41affd640 1 -- 172.21.15.172:0/2238551344 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fa42400f6b0 con 0x5569c1fdb370 2024-09-10T09:05:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa444e04740 10 client.20037 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa444e04740 10 client.20037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa444e04740 20 client.20037 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa444e04740 1 --2- 172.21.15.172:0/2238551344 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5569c1feb8a0 0x5569c1fedd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa444e04740 4 client.20037 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:05:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.221+0000 7fa444e04740 1 -- 172.21.15.172:0/2238551344 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569c1f46a20 con 0x5569c1feb8a0 2024-09-10T09:05:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.222+0000 7fa43b7fe640 1 --2- 172.21.15.172:0/2238551344 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5569c1feb8a0 0x5569c1fedd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.222+0000 7fa43b7fe640 1 --2- 172.21.15.172:0/2238551344 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5569c1feb8a0 0x5569c1fedd10 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cc67f9c44af0c71 server_cookie=deb44ef12487db3d in_seq=0 out_seq=0 2024-09-10T09:05:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.222+0000 7fa41affd640 10 client.20037 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.223+0000 7fa41affd640 1 -- 172.21.15.172:0/2238551344 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5569c1f46a20 con 0x5569c1feb8a0 2024-09-10T09:05:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.223+0000 7fa41affd640 10 client.20037 handle_command_reply: tid=0 2024-09-10T09:05:08.231 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.223+0000 7fa444e04740 1 client.20037 shutdown 2024-09-10T09:05:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.223+0000 7fa444e04740 1 -- 172.21.15.172:0/2238551344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fdb370 msgr2=0x5569c1fe70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.223+0000 7fa444e04740 1 --2- 172.21.15.172:0/2238551344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fdb370 0x5569c1fe70b0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fa42400c780 tx=0x7fa424016040 comp rx=0 tx=0).stop 2024-09-10T09:05:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.223+0000 7fa444e04740 1 -- 172.21.15.172:0/2238551344 shutdown_connections 2024-09-10T09:05:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.223+0000 7fa444e04740 1 --2- 172.21.15.172:0/2238551344 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5569c1feb8a0 0x5569c1fedd10 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.223+0000 7fa444e04740 1 --2- 172.21.15.172:0/2238551344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fdb370 0x5569c1fe70b0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.223+0000 7fa444e04740 1 --2- 172.21.15.172:0/2238551344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fd62e0 0x5569c1fe0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.223+0000 7fa444e04740 1 --2- 172.21.15.172:0/2238551344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fd59a0 0x5569c1fdfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.223+0000 7fa41affd640 0 client.20037 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.223+0000 7fa444e04740 1 -- 172.21.15.172:0/2238551344 >> 172.21.15.172:0/2238551344 conn(0x5569c1fd1010 msgr2=0x5569c1fd2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.223+0000 7fa444e04740 1 -- 172.21.15.172:0/2238551344 shutdown_connections 2024-09-10T09:05:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.223+0000 7fa444e04740 1 -- 172.21.15.172:0/2238551344 wait complete. 2024-09-10T09:05:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.223+0000 7fa444e04740 20 client.20037 trim_cache size 0 max 16384 2024-09-10T09:05:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.237+0000 7fa444e04740 1 Processor -- start 2024-09-10T09:05:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.237+0000 7fa444e04740 1 -- start start 2024-09-10T09:05:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.237+0000 7fa444e04740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fd59a0 0x5569c1fd00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.237+0000 7fa444e04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fd62e0 0x5569c1fe7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.237+0000 7fa444e04740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fdb370 0x5569c1ff2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.237+0000 7fa444e04740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1aa4e0 con 0x5569c1fd62e0 2024-09-10T09:05:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.237+0000 7fa444e04740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1ac020 con 0x5569c1fdb370 2024-09-10T09:05:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.237+0000 7fa444e04740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1aabb0 con 0x5569c1fd59a0 2024-09-10T09:05:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.238+0000 7fa43b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fd62e0 0x5569c1fe7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.238+0000 7fa43b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fd62e0 0x5569c1fe7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45388/0 (socket says 172.21.15.172:45388) 2024-09-10T09:05:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.238+0000 7fa43b7fe640 1 -- 172.21.15.172:0/1610348125 learned_addr learned my addr 172.21.15.172:0/1610348125 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.238+0000 7fa43affd640 1 --2- 172.21.15.172:0/1610348125 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fd59a0 0x5569c1fd00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.238+0000 7fa43affd640 1 --2- 172.21.15.172:0/1610348125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fdb370 0x5569c1ff2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.238+0000 7fa43b7fe640 1 -- 172.21.15.172:0/1610348125 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fd59a0 msgr2=0x5569c1fd00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.238+0000 7fa43b7fe640 1 --2- 172.21.15.172:0/1610348125 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fd59a0 0x5569c1fd00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.238+0000 7fa43b7fe640 1 -- 172.21.15.172:0/1610348125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fdb370 msgr2=0x5569c1ff2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.238+0000 7fa43b7fe640 1 --2- 172.21.15.172:0/1610348125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fdb370 0x5569c1ff2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.238+0000 7fa43b7fe640 1 -- 172.21.15.172:0/1610348125 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569c1fd67c0 con 0x5569c1fd62e0 2024-09-10T09:05:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.238+0000 7fa43b7fe640 1 --2- 172.21.15.172:0/1610348125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fd62e0 0x5569c1fe7070 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fa42c02a930 tx=0x7fa42c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da43c7dbc6e0c9f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.238+0000 7fa4317fa640 1 -- 172.21.15.172:0/1610348125 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c050070 con 0x5569c1fd62e0 2024-09-10T09:05:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.238+0000 7fa4317fa640 1 -- 172.21.15.172:0/1610348125 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa42c0035f0 con 0x5569c1fd62e0 2024-09-10T09:05:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.238+0000 7fa4317fa640 1 -- 172.21.15.172:0/1610348125 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c03f050 con 0x5569c1fd62e0 2024-09-10T09:05:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 1 -- 172.21.15.172:0/1610348125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fd62e0 msgr2=0x5569c1fe7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 1 --2- 172.21.15.172:0/1610348125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fd62e0 0x5569c1fe7070 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fa42c02a930 tx=0x7fa42c000bc0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 1 -- 172.21.15.172:0/1610348125 shutdown_connections 2024-09-10T09:05:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 1 --2- 172.21.15.172:0/1610348125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fdb370 0x5569c1ff2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 1 --2- 172.21.15.172:0/1610348125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fd62e0 0x5569c1fe7070 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 1 --2- 172.21.15.172:0/1610348125 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fd59a0 0x5569c1fd00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 1 -- 172.21.15.172:0/1610348125 >> 172.21.15.172:0/1610348125 conn(0x5569c1fd1010 msgr2=0x5569c1fcf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 1 -- 172.21.15.172:0/1610348125 shutdown_connections 2024-09-10T09:05:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 1 -- 172.21.15.172:0/1610348125 wait complete. 2024-09-10T09:05:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 1 -- start start 2024-09-10T09:05:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 1 Processor -- start 2024-09-10T09:05:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fd59a0 0x5569c1fe46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fd62e0 0x5569c1fe4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fdb370 0x5569c1fe5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1ab950 con 0x5569c1fdb370 2024-09-10T09:05:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1ab280 con 0x5569c1fd59a0 2024-09-10T09:05:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c1a84c0 con 0x5569c1fd62e0 2024-09-10T09:05:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa444e04740 10 client.0 _ll_register_callbacks cb 0x7fa424012990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa43affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fdb370 0x5569c1fe5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa43affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fdb370 0x5569c1fe5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45404/0 (socket says 172.21.15.172:45404) 2024-09-10T09:05:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa43affd640 1 -- 172.21.15.172:0/3503647781 learned_addr learned my addr 172.21.15.172:0/3503647781 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa43affd640 1 --2- 172.21.15.172:0/3503647781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fd59a0 0x5569c1fe46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.239+0000 7fa43b7fe640 1 --2- 172.21.15.172:0/3503647781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fd62e0 0x5569c1fe4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa43affd640 1 -- 172.21.15.172:0/3503647781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fd62e0 msgr2=0x5569c1fe4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa43affd640 1 --2- 172.21.15.172:0/3503647781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fd62e0 0x5569c1fe4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa43affd640 1 -- 172.21.15.172:0/3503647781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fd59a0 msgr2=0x5569c1fe46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa43affd640 1 --2- 172.21.15.172:0/3503647781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fd59a0 0x5569c1fe46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa43affd640 1 -- 172.21.15.172:0/3503647781 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569c1fe6160 con 0x5569c1fdb370 2024-09-10T09:05:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa43affd640 1 --2- 172.21.15.172:0/3503647781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fdb370 0x5569c1fe5b50 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fa424005fd0 tx=0x7fa424005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338cffca0da70691 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa41affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:05:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa41affd640 1 -- 172.21.15.172:0/3503647781 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42400f750 con 0x5569c1fdb370 2024-09-10T09:05:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa41affd640 1 -- 172.21.15.172:0/3503647781 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa4240035f0 con 0x5569c1fdb370 2024-09-10T09:05:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa444e04740 1 -- 172.21.15.172:0/3503647781 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa42c0035f0 con 0x5569c1fdb370 2024-09-10T09:05:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa41affd640 1 -- 172.21.15.172:0/3503647781 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa42403b050 con 0x5569c1fdb370 2024-09-10T09:05:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa444e04740 10 client.20043 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa444e04740 1 -- 172.21.15.172:0/3503647781 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5569c205ee00 con 0x5569c1fdb370 2024-09-10T09:05:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa41affd640 1 -- 172.21.15.172:0/3503647781 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa42c0035f0 con 0x5569c1fdb370 2024-09-10T09:05:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa41affd640 1 -- 172.21.15.172:0/3503647781 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fa424004290 con 0x5569c1fdb370 2024-09-10T09:05:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa444e04740 10 client.20043 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa444e04740 10 client.20043 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa444e04740 20 client.20043 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.240+0000 7fa444e04740 1 --2- 172.21.15.172:0/3503647781 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5569c205f5e0 0x5569c20619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.241+0000 7fa444e04740 4 client.20043 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:05:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.241+0000 7fa444e04740 1 -- 172.21.15.172:0/3503647781 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5569c1f40d70 con 0x5569c205f5e0 2024-09-10T09:05:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.241+0000 7fa43b7fe640 1 --2- 172.21.15.172:0/3503647781 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5569c205f5e0 0x5569c20619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.241+0000 7fa41affd640 10 client.20043 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.241+0000 7fa43b7fe640 1 --2- 172.21.15.172:0/3503647781 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5569c205f5e0 0x5569c20619d0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=383a45be12891fe5 server_cookie=3660daafc7e801f4 in_seq=0 out_seq=0 2024-09-10T09:05:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.248+0000 7fa41affd640 1 -- 172.21.15.172:0/3503647781 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5569c1f40d70 con 0x5569c205f5e0 2024-09-10T09:05:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.248+0000 7fa41affd640 10 client.20043 handle_command_reply: tid=0 2024-09-10T09:05:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.248+0000 7fa444e04740 1 client.20043 shutdown 2024-09-10T09:05:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.248+0000 7fa444e04740 1 -- 172.21.15.172:0/3503647781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fdb370 msgr2=0x5569c1fe5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.248+0000 7fa444e04740 1 --2- 172.21.15.172:0/3503647781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fdb370 0x5569c1fe5b50 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fa424005fd0 tx=0x7fa424005850 comp rx=0 tx=0).stop 2024-09-10T09:05:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.248+0000 7fa444e04740 1 -- 172.21.15.172:0/3503647781 shutdown_connections 2024-09-10T09:05:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.248+0000 7fa444e04740 1 --2- 172.21.15.172:0/3503647781 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5569c205f5e0 0x5569c20619d0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.248+0000 7fa444e04740 1 --2- 172.21.15.172:0/3503647781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569c1fdb370 0x5569c1fe5b50 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.248+0000 7fa41affd640 0 client.20043 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.248+0000 7fa444e04740 1 --2- 172.21.15.172:0/3503647781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569c1fd62e0 0x5569c1fe4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.248+0000 7fa444e04740 1 --2- 172.21.15.172:0/3503647781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569c1fd59a0 0x5569c1fe46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.248+0000 7fa444e04740 1 -- 172.21.15.172:0/3503647781 >> 172.21.15.172:0/3503647781 conn(0x5569c1fd1010 msgr2=0x5569c1fd01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.248+0000 7fa444e04740 1 -- 172.21.15.172:0/3503647781 shutdown_connections 2024-09-10T09:05:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.248+0000 7fa444e04740 1 -- 172.21.15.172:0/3503647781 wait complete. 2024-09-10T09:05:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.248+0000 7fa444e04740 20 client.20043 trim_cache size 0 max 16384 2024-09-10T09:05:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:05:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:05:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:05:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:05:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.249+0000 7fa441f8d640 1 -- 172.21.15.172:0/2432318892 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4280a2c30 msgr2=0x7fa4280a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.249+0000 7fa441f8d640 1 --2- 172.21.15.172:0/2432318892 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4280a2c30 0x7fa4280a50f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa43409ac20 tx=0x7fa434090400 comp rx=0 tx=0).stop 2024-09-10T09:05:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.249+0000 7fa441f8d640 1 -- 172.21.15.172:0/2432318892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa43c103710 msgr2=0x7fa43c19f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.249+0000 7fa441f8d640 1 --2- 172.21.15.172:0/2432318892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa43c103710 0x7fa43c19f300 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fa43400b970 tx=0x7fa43400be30 comp rx=0 tx=0).stop 2024-09-10T09:05:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.249+0000 7fa441f8d640 1 -- 172.21.15.172:0/2432318892 shutdown_connections 2024-09-10T09:05:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.249+0000 7fa441f8d640 1 --2- 172.21.15.172:0/2432318892 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa4280a2c30 0x7fa4280a50f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.249+0000 7fa441f8d640 1 --2- 172.21.15.172:0/2432318892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa43c103710 0x7fa43c19f300 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.249+0000 7fa441f8d640 1 --2- 172.21.15.172:0/2432318892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa43c102d50 0x7fa43c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.249+0000 7fa441f8d640 1 --2- 172.21.15.172:0/2432318892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa43c101b60 0x7fa43c197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.249+0000 7fa441f8d640 1 -- 172.21.15.172:0/2432318892 >> 172.21.15.172:0/2432318892 conn(0x7fa43c0fd2e0 msgr2=0x7fa43c106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.249+0000 7fa441f8d640 1 -- 172.21.15.172:0/2432318892 shutdown_connections 2024-09-10T09:05:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:08.249+0000 7fa441f8d640 1 -- 172.21.15.172:0/2432318892 wait complete. 2024-09-10T09:05:08.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:08 smithi174 ceph-mon[34068]: pgmap v917: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 526 KiB/s wr, 159 op/s 2024-09-10T09:05:09.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:08 smithi191 ceph-mon[35314]: pgmap v917: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 526 KiB/s wr, 159 op/s 2024-09-10T09:05:09.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:08 smithi172 ceph-mon[30854]: pgmap v917: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 526 KiB/s wr, 159 op/s 2024-09-10T09:05:09.438 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:05:09.693 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:05:10.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.219+0000 7fe1073bb640 1 -- 172.21.15.172:0/3253430418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100106a90 msgr2=0x7fe100106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:10.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.219+0000 7fe1073bb640 1 --2- 172.21.15.172:0/3253430418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100106a90 0x7fe100106e70 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009930 tx=0x7fe0fc02a420 comp rx=0 tx=0).stop 2024-09-10T09:05:10.257 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.219+0000 7fe1073bb640 1 -- 172.21.15.172:0/3253430418 shutdown_connections 2024-09-10T09:05:10.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.219+0000 7fe1073bb640 1 --2- 172.21.15.172:0/3253430418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101430 0x7fe10010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:10.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.219+0000 7fe1073bb640 1 --2- 172.21.15.172:0/3253430418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe100100a90 0x7fe100100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:10.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.219+0000 7fe1073bb640 1 --2- 172.21.15.172:0/3253430418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100106a90 0x7fe100106e70 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:10.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.219+0000 7fe1073bb640 1 -- 172.21.15.172:0/3253430418 >> 172.21.15.172:0/3253430418 conn(0x7fe1000fc7d0 msgr2=0x7fe1000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:10.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.219+0000 7fe1073bb640 1 -- 172.21.15.172:0/3253430418 shutdown_connections 2024-09-10T09:05:10.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.219+0000 7fe1073bb640 1 -- 172.21.15.172:0/3253430418 wait complete. 2024-09-10T09:05:10.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe1073bb640 1 Processor -- start 2024-09-10T09:05:10.258 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe1073bb640 1 -- start start 2024-09-10T09:05:10.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe1073bb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe100100a90 0x7fe1000fcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:10.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe1073bb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101430 0x7fe1000fd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:10.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe1073bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100106a90 0x7fe1000fda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:10.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe1073bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe100110de0 con 0x7fe100106a90 2024-09-10T09:05:10.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe1073bb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe100110c60 con 0x7fe100101430 2024-09-10T09:05:10.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe1073bb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe100110f60 con 0x7fe100100a90 2024-09-10T09:05:10.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe105958640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100106a90 0x7fe1000fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:10.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe105958640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100106a90 0x7fe1000fda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45428/0 (socket says 172.21.15.172:45428) 2024-09-10T09:05:10.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe105958640 1 -- 172.21.15.172:0/450091832 learned_addr learned my addr 172.21.15.172:0/450091832 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:10.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe104956640 1 --2- 172.21.15.172:0/450091832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101430 0x7fe1000fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:10.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe105157640 1 --2- 172.21.15.172:0/450091832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe100100a90 0x7fe1000fcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:10.259 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe105958640 1 -- 172.21.15.172:0/450091832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe100100a90 msgr2=0x7fe1000fcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:10.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe105958640 1 --2- 172.21.15.172:0/450091832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe100100a90 0x7fe1000fcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:10.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe105958640 1 -- 172.21.15.172:0/450091832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101430 msgr2=0x7fe1000fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:10.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe105958640 1 --2- 172.21.15.172:0/450091832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101430 0x7fe1000fd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:10.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe105958640 1 -- 172.21.15.172:0/450091832 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1001a6980 con 0x7fe100106a90 2024-09-10T09:05:10.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.220+0000 7fe105157640 1 --2- 172.21.15.172:0/450091832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe100100a90 0x7fe1000fcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:05:10.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.221+0000 7fe105958640 1 --2- 172.21.15.172:0/450091832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100106a90 0x7fe1000fda50 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800b970 tx=0x7fe0f800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:10.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.221+0000 7fe0f67fc640 1 -- 172.21.15.172:0/450091832 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8019070 con 0x7fe100106a90 2024-09-10T09:05:10.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.221+0000 7fe1073bb640 1 -- 172.21.15.172:0/450091832 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1001a6c70 con 0x7fe100106a90 2024-09-10T09:05:10.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.221+0000 7fe0f67fc640 1 -- 172.21.15.172:0/450091832 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0f8014440 con 0x7fe100106a90 2024-09-10T09:05:10.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.221+0000 7fe0f67fc640 1 -- 172.21.15.172:0/450091832 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80075d0 con 0x7fe100106a90 2024-09-10T09:05:10.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.221+0000 7fe1073bb640 1 -- 172.21.15.172:0/450091832 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1001a7160 con 0x7fe100106a90 2024-09-10T09:05:10.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.222+0000 7fe1073bb640 1 -- 172.21.15.172:0/450091832 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe100100f70 con 0x7fe100106a90 2024-09-10T09:05:10.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.223+0000 7fe0f67fc640 1 -- 172.21.15.172:0/450091832 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe0f801c070 con 0x7fe100106a90 2024-09-10T09:05:10.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.223+0000 7fe0f67fc640 1 --2- 172.21.15.172:0/450091832 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0ec0a2b80 0x7fe0ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:10.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.224+0000 7fe105157640 1 --2- 172.21.15.172:0/450091832 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0ec0a2b80 0x7fe0ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:10.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.224+0000 7fe0f67fc640 1 -- 172.21.15.172:0/450091832 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe0f80ce960 con 0x7fe100106a90 2024-09-10T09:05:10.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.224+0000 7fe105157640 1 --2- 172.21.15.172:0/450091832 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0ec0a2b80 0x7fe0ec0a5040 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc000c00 tx=0x7fe0fc031cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:10.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.228+0000 7fe0f67fc640 1 -- 172.21.15.172:0/450091832 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe0f809b560 con 0x7fe100106a90 2024-09-10T09:05:10.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.430+0000 7fe1073bb640 1 -- 172.21.15.172:0/450091832 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe10002d7c0 con 0x7fe100106a90 2024-09-10T09:05:10.431 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.431+0000 7fe0f67fc640 1 -- 172.21.15.172:0/450091832 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+12001 (secure 0 0 0) 0x7fe0f809ae60 con 0x7fe100106a90 2024-09-10T09:05:10.432 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T09:05:10.432 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":44,"btime":"2024-09-10T09:01:18:531063+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T09:01:17.697372+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T09:05:10.432 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 44 2024-09-10T09:05:10.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.433+0000 7fe1073bb640 1 -- 172.21.15.172:0/450091832 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0ec0a2b80 msgr2=0x7fe0ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:10.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.433+0000 7fe1073bb640 1 --2- 172.21.15.172:0/450091832 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0ec0a2b80 0x7fe0ec0a5040 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc000c00 tx=0x7fe0fc031cf0 comp rx=0 tx=0).stop 2024-09-10T09:05:10.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.433+0000 7fe1073bb640 1 -- 172.21.15.172:0/450091832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100106a90 msgr2=0x7fe1000fda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:10.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.433+0000 7fe1073bb640 1 --2- 172.21.15.172:0/450091832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100106a90 0x7fe1000fda50 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800b970 tx=0x7fe0f800bd70 comp rx=0 tx=0).stop 2024-09-10T09:05:10.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.433+0000 7fe1073bb640 1 -- 172.21.15.172:0/450091832 shutdown_connections 2024-09-10T09:05:10.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.433+0000 7fe1073bb640 1 --2- 172.21.15.172:0/450091832 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0ec0a2b80 0x7fe0ec0a5040 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:10.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.433+0000 7fe1073bb640 1 --2- 172.21.15.172:0/450091832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe100106a90 0x7fe1000fda50 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:10.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.433+0000 7fe1073bb640 1 --2- 172.21.15.172:0/450091832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101430 0x7fe1000fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:10.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.433+0000 7fe1073bb640 1 --2- 172.21.15.172:0/450091832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe100100a90 0x7fe1000fcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:10.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.433+0000 7fe1073bb640 1 -- 172.21.15.172:0/450091832 >> 172.21.15.172:0/450091832 conn(0x7fe1000fc7d0 msgr2=0x7fe10006ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:10.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.433+0000 7fe1073bb640 1 -- 172.21.15.172:0/450091832 shutdown_connections 2024-09-10T09:05:10.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:10.433+0000 7fe1073bb640 1 -- 172.21.15.172:0/450091832 wait complete. 2024-09-10T09:05:10.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:10 smithi172 ceph-mon[30854]: pgmap v918: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 105 KiB/s wr, 57 op/s 2024-09-10T09:05:10.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:10 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/450091832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:05:10.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:10 smithi174 ceph-mon[34068]: pgmap v918: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 105 KiB/s wr, 57 op/s 2024-09-10T09:05:10.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:10 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/450091832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:05:11.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:10 smithi191 ceph-mon[35314]: pgmap v918: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 105 KiB/s wr, 57 op/s 2024-09-10T09:05:11.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:10 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/450091832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:05:11.115 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 15ff85c3 --release --await' 2024-09-10T09:05:11.115 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 15ff85c3 --release --await 2024-09-10T09:05:11.369 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:05:11.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.876+0000 7f58f0cc0640 1 -- 172.21.15.172:0/1498182250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58ec10c190 msgr2=0x7f58ec10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:11.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.876+0000 7f58f0cc0640 1 --2- 172.21.15.172:0/1498182250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58ec10c190 0x7f58ec10e610 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009ca0 tx=0x7f58e402a950 comp rx=0 tx=0).stop 2024-09-10T09:05:11.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.876+0000 7f58f0cc0640 1 -- 172.21.15.172:0/1498182250 shutdown_connections 2024-09-10T09:05:11.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.876+0000 7f58f0cc0640 1 --2- 172.21.15.172:0/1498182250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58ec10c190 0x7f58ec10e610 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.876+0000 7f58f0cc0640 1 --2- 172.21.15.172:0/1498182250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58ec1036d0 0x7f58ec10bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.876+0000 7f58f0cc0640 1 --2- 172.21.15.172:0/1498182250 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58ec102db0 0x7f58ec103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.876+0000 7f58f0cc0640 1 -- 172.21.15.172:0/1498182250 >> 172.21.15.172:0/1498182250 conn(0x7f58ec0fc7d0 msgr2=0x7f58ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:11.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.876+0000 7f58f0cc0640 1 -- 172.21.15.172:0/1498182250 shutdown_connections 2024-09-10T09:05:11.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.876+0000 7f58f0cc0640 1 -- 172.21.15.172:0/1498182250 wait complete. 2024-09-10T09:05:11.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.877+0000 7f58f0cc0640 1 Processor -- start 2024-09-10T09:05:11.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.877+0000 7f58f0cc0640 1 -- start start 2024-09-10T09:05:11.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.877+0000 7f58f0cc0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58ec102db0 0x7f58ec198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.877+0000 7f58f0cc0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58ec1036d0 0x7f58ec1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.877+0000 7f58f0cc0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58ec10c190 0x7f58ec19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.877+0000 7f58f0cc0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec111e60 con 0x7f58ec1036d0 2024-09-10T09:05:11.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.877+0000 7f58f0cc0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec111ce0 con 0x7f58ec10c190 2024-09-10T09:05:11.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.877+0000 7f58f0cc0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec111fe0 con 0x7f58ec102db0 2024-09-10T09:05:11.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.877+0000 7f58e1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58ec1036d0 0x7f58ec1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.877+0000 7f58e1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58ec1036d0 0x7f58ec1988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45458/0 (socket says 172.21.15.172:45458) 2024-09-10T09:05:11.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.877+0000 7f58e1d9b640 1 -- 172.21.15.172:0/921039781 learned_addr learned my addr 172.21.15.172:0/921039781 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:11.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.877+0000 7f58ea59c640 1 --2- 172.21.15.172:0/921039781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58ec102db0 0x7f58ec198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.877+0000 7f58ead9d640 1 --2- 172.21.15.172:0/921039781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58ec10c190 0x7f58ec19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.878+0000 7f58e1d9b640 1 -- 172.21.15.172:0/921039781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58ec102db0 msgr2=0x7f58ec198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:11.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.878+0000 7f58e1d9b640 1 --2- 172.21.15.172:0/921039781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58ec102db0 0x7f58ec198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.878+0000 7f58e1d9b640 1 -- 172.21.15.172:0/921039781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58ec10c190 msgr2=0x7f58ec19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:11.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.878+0000 7f58e1d9b640 1 --2- 172.21.15.172:0/921039781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58ec10c190 0x7f58ec19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.878+0000 7f58e1d9b640 1 -- 172.21.15.172:0/921039781 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ec19d330 con 0x7f58ec1036d0 2024-09-10T09:05:11.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.878+0000 7f58e1d9b640 1 --2- 172.21.15.172:0/921039781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58ec1036d0 0x7f58ec1988c0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f58d400c890 tx=0x7f58d400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:11.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.878+0000 7f58e3fff640 1 -- 172.21.15.172:0/921039781 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4004220 con 0x7f58ec1036d0 2024-09-10T09:05:11.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.878+0000 7f58e3fff640 1 -- 172.21.15.172:0/921039781 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58d40043c0 con 0x7f58ec1036d0 2024-09-10T09:05:11.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.878+0000 7f58f0cc0640 1 -- 172.21.15.172:0/921039781 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ec19d620 con 0x7f58ec1036d0 2024-09-10T09:05:11.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.878+0000 7f58e3fff640 1 -- 172.21.15.172:0/921039781 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4004a00 con 0x7f58ec1036d0 2024-09-10T09:05:11.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.878+0000 7f58f0cc0640 1 -- 172.21.15.172:0/921039781 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ec1a4f00 con 0x7f58ec1036d0 2024-09-10T09:05:11.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.880+0000 7f58e3fff640 1 -- 172.21.15.172:0/921039781 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f58d40079b0 con 0x7f58ec1036d0 2024-09-10T09:05:11.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.880+0000 7f58f3099740 1 Processor -- start 2024-09-10T09:05:11.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.880+0000 7f58f3099740 1 -- start start 2024-09-10T09:05:11.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58f3099740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c060 0x55e31585c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58f3099740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c980 0x55e315866d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58f3099740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e315867240 0x55e315869630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58f3099740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e3157c1620 con 0x55e31585c980 2024-09-10T09:05:11.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58f3099740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e315768e50 con 0x55e31585c060 2024-09-10T09:05:11.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58f3099740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e3157bd5e0 con 0x55e315867240 2024-09-10T09:05:11.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58e3fff640 1 --2- 172.21.15.172:0/921039781 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f58d80a2b80 0x7f58d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c980 0x55e315866d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c980 0x55e315866d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45462/0 (socket says 172.21.15.172:45462) 2024-09-10T09:05:11.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58ea59c640 1 -- 172.21.15.172:0/350862355 learned_addr learned my addr 172.21.15.172:0/350862355 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:11.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58e1d9b640 1 --2- 172.21.15.172:0/350862355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e315867240 0x55e315869630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58ead9d640 1 --2- 172.21.15.172:0/350862355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c060 0x55e31585c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58ead9d640 1 --2- 172.21.15.172:0/921039781 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f58d80a2b80 0x7f58d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58e3fff640 1 -- 172.21.15.172:0/921039781 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f58d40cefe0 con 0x7f58ec1036d0 2024-09-10T09:05:11.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58ea59c640 1 -- 172.21.15.172:0/350862355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e315867240 msgr2=0x55e315869630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:11.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58ea59c640 1 --2- 172.21.15.172:0/350862355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e315867240 0x55e315869630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58ea59c640 1 -- 172.21.15.172:0/350862355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c060 msgr2=0x55e31585c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:11.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58ea59c640 1 --2- 172.21.15.172:0/350862355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c060 0x55e31585c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58ea59c640 1 -- 172.21.15.172:0/350862355 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e315759d40 con 0x55e31585c980 2024-09-10T09:05:11.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58e1d9b640 1 --2- 172.21.15.172:0/350862355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e315867240 0x55e315869630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:05:11.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58ea59c640 1 --2- 172.21.15.172:0/350862355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c980 0x55e315866d00 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009a70 tx=0x7f58dc009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391ecd4f45f7c55f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:11.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.881+0000 7f58ead9d640 1 --2- 172.21.15.172:0/921039781 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f58d80a2b80 0x7f58d80a5040 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f58e4005fe0 tx=0x7f58e4005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:11.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.882+0000 7f58e0d99640 1 -- 172.21.15.172:0/350862355 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc017070 con 0x55e31585c980 2024-09-10T09:05:11.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.882+0000 7f58e0d99640 1 -- 172.21.15.172:0/350862355 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58dc013470 con 0x55e31585c980 2024-09-10T09:05:11.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.882+0000 7f58e0d99640 1 -- 172.21.15.172:0/350862355 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc012610 con 0x55e31585c980 2024-09-10T09:05:11.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.882+0000 7f58f3099740 1 -- 172.21.15.172:0/350862355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c980 msgr2=0x55e315866d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:11.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.882+0000 7f58f3099740 1 --2- 172.21.15.172:0/350862355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c980 0x55e315866d00 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009a70 tx=0x7f58dc009f30 comp rx=0 tx=0).stop 2024-09-10T09:05:11.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.882+0000 7f58f3099740 1 -- 172.21.15.172:0/350862355 shutdown_connections 2024-09-10T09:05:11.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.882+0000 7f58f3099740 1 --2- 172.21.15.172:0/350862355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e315867240 0x55e315869630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.882+0000 7f58f3099740 1 --2- 172.21.15.172:0/350862355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c980 0x55e315866d00 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.882+0000 7f58f3099740 1 --2- 172.21.15.172:0/350862355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c060 0x55e31585c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.882+0000 7f58f3099740 1 -- 172.21.15.172:0/350862355 >> 172.21.15.172:0/350862355 conn(0x55e3158585f0 msgr2=0x55e3158589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:11.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.883+0000 7f58f3099740 1 -- 172.21.15.172:0/350862355 shutdown_connections 2024-09-10T09:05:11.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.883+0000 7f58f3099740 1 -- 172.21.15.172:0/350862355 wait complete. 2024-09-10T09:05:11.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.883+0000 7f58f3099740 1 -- start start 2024-09-10T09:05:11.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.883+0000 7f58f3099740 1 Processor -- start 2024-09-10T09:05:11.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.883+0000 7f58f3099740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c060 0x55e315864090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.883+0000 7f58f3099740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e31585c980 0x55e3158645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.883+0000 7f58f3099740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e315867240 0x55e315864b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.883+0000 7f58f3099740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e315869e10 con 0x55e315867240 2024-09-10T09:05:11.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.883+0000 7f58f3099740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e315749f10 con 0x55e31585c060 2024-09-10T09:05:11.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.883+0000 7f58f3099740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e315869f90 con 0x55e31585c980 2024-09-10T09:05:11.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.883+0000 7f58f3099740 10 client.0 _ll_register_callbacks cb 0x55e31576ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:11.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.883+0000 7f58ead9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e315867240 0x55e315864b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.883+0000 7f58ead9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e315867240 0x55e315864b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45478/0 (socket says 172.21.15.172:45478) 2024-09-10T09:05:11.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.883+0000 7f58ead9d640 1 -- 172.21.15.172:0/2758813159 learned_addr learned my addr 172.21.15.172:0/2758813159 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:11.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.884+0000 7f58ea59c640 1 --2- 172.21.15.172:0/2758813159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c060 0x55e315864090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.884+0000 7f58e1d9b640 1 --2- 172.21.15.172:0/2758813159 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e31585c980 0x55e3158645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.884+0000 7f58ead9d640 1 -- 172.21.15.172:0/2758813159 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e31585c980 msgr2=0x55e3158645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:11.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.884+0000 7f58ead9d640 1 --2- 172.21.15.172:0/2758813159 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e31585c980 0x55e3158645d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.884+0000 7f58ead9d640 1 -- 172.21.15.172:0/2758813159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c060 msgr2=0x55e315864090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:11.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.884+0000 7f58ead9d640 1 --2- 172.21.15.172:0/2758813159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c060 0x55e315864090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.884+0000 7f58ead9d640 1 -- 172.21.15.172:0/2758813159 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3154d0580 con 0x55e315867240 2024-09-10T09:05:11.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.884+0000 7f58ead9d640 1 --2- 172.21.15.172:0/2758813159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e315867240 0x55e315864b10 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f58e4030040 tx=0x7f58e40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef725058362c092f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:11.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.884+0000 7f58d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:05:11.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.884+0000 7f58d1ffb640 1 -- 172.21.15.172:0/2758813159 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e404f030 con 0x55e315867240 2024-09-10T09:05:11.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.884+0000 7f58d1ffb640 1 -- 172.21.15.172:0/2758813159 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f58e4004510 con 0x55e315867240 2024-09-10T09:05:11.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.884+0000 7f58f3099740 1 -- 172.21.15.172:0/2758813159 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58dc013470 con 0x55e315867240 2024-09-10T09:05:11.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.884+0000 7f58d1ffb640 1 -- 172.21.15.172:0/2758813159 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f58e4034040 con 0x55e315867240 2024-09-10T09:05:11.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.884+0000 7f58f3099740 10 client.20061 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:11.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.884+0000 7f58f3099740 1 -- 172.21.15.172:0/2758813159 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e31559a960 con 0x55e315867240 2024-09-10T09:05:11.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.884+0000 7f58d1ffb640 1 -- 172.21.15.172:0/2758813159 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58dc013470 con 0x55e315867240 2024-09-10T09:05:11.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.885+0000 7f58d1ffb640 1 -- 172.21.15.172:0/2758813159 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f58e404bc70 con 0x55e315867240 2024-09-10T09:05:11.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.885+0000 7f58f3099740 10 client.20061 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:11.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.885+0000 7f58f3099740 10 client.20061 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:05:11.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.885+0000 7f58f3099740 20 client.20061 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:11.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.885+0000 7f58f3099740 1 --2- 172.21.15.172:0/2758813159 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e315876c60 0x55e3158790d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.885+0000 7f58f3099740 4 client.20061 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:05:11.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.885+0000 7f58f3099740 1 -- 172.21.15.172:0/2758813159 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3157ceb90 con 0x55e315876c60 2024-09-10T09:05:11.895 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.885+0000 7f58ea59c640 1 --2- 172.21.15.172:0/2758813159 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e315876c60 0x55e3158790d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.885+0000 7f58d1ffb640 10 client.20061 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:11.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.885+0000 7f58ea59c640 1 --2- 172.21.15.172:0/2758813159 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e315876c60 0x55e3158790d0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=454bbfdecb159f7e server_cookie=8a5f7f46ddc14b3 in_seq=0 out_seq=0 2024-09-10T09:05:11.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.886+0000 7f58d1ffb640 1 -- 172.21.15.172:0/2758813159 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3157ceb90 con 0x55e315876c60 2024-09-10T09:05:11.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.886+0000 7f58d1ffb640 10 client.20061 handle_command_reply: tid=0 2024-09-10T09:05:11.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.886+0000 7f58f3099740 1 client.20061 shutdown 2024-09-10T09:05:11.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.886+0000 7f58f3099740 1 -- 172.21.15.172:0/2758813159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e315867240 msgr2=0x55e315864b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:11.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.886+0000 7f58f3099740 1 --2- 172.21.15.172:0/2758813159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e315867240 0x55e315864b10 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f58e4030040 tx=0x7f58e40095c0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.886+0000 7f58f3099740 1 -- 172.21.15.172:0/2758813159 shutdown_connections 2024-09-10T09:05:11.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.886+0000 7f58f3099740 1 --2- 172.21.15.172:0/2758813159 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e315876c60 0x55e3158790d0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.886+0000 7f58f3099740 1 --2- 172.21.15.172:0/2758813159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e315867240 0x55e315864b10 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.886+0000 7f58f3099740 1 --2- 172.21.15.172:0/2758813159 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e31585c980 0x55e3158645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.886+0000 7f58f3099740 1 --2- 172.21.15.172:0/2758813159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c060 0x55e315864090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.886+0000 7f58d1ffb640 0 client.20061 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:11.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.886+0000 7f58f3099740 1 -- 172.21.15.172:0/2758813159 >> 172.21.15.172:0/2758813159 conn(0x55e3158585f0 msgr2=0x55e31585e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:11.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.886+0000 7f58f3099740 1 -- 172.21.15.172:0/2758813159 shutdown_connections 2024-09-10T09:05:11.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.886+0000 7f58f3099740 1 -- 172.21.15.172:0/2758813159 wait complete. 2024-09-10T09:05:11.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.887+0000 7f58f3099740 20 client.20061 trim_cache size 0 max 16384 2024-09-10T09:05:11.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.903+0000 7f58f3099740 1 Processor -- start 2024-09-10T09:05:11.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.903+0000 7f58f3099740 1 -- start start 2024-09-10T09:05:11.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.903+0000 7f58f3099740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c060 0x55e31585e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.903+0000 7f58f3099740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c980 0x55e31587e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.903+0000 7f58f3099740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e315867240 0x55e31587ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.903+0000 7f58f3099740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec1aeb10 con 0x55e31585c980 2024-09-10T09:05:11.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.903+0000 7f58f3099740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec1b0650 con 0x55e31585c060 2024-09-10T09:05:11.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.903+0000 7f58f3099740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec1af1e0 con 0x55e315867240 2024-09-10T09:05:11.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.904+0000 7f58e1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c980 0x55e31587e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.904+0000 7f58e1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c980 0x55e31587e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45490/0 (socket says 172.21.15.172:45490) 2024-09-10T09:05:11.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.904+0000 7f58e1d9b640 1 -- 172.21.15.172:0/2847647522 learned_addr learned my addr 172.21.15.172:0/2847647522 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:11.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.904+0000 7f58ead9d640 1 --2- 172.21.15.172:0/2847647522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e315867240 0x55e31587ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.904+0000 7f58ea59c640 1 --2- 172.21.15.172:0/2847647522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c060 0x55e31585e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.904+0000 7f58e1d9b640 1 -- 172.21.15.172:0/2847647522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e315867240 msgr2=0x55e31587ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:11.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.904+0000 7f58e1d9b640 1 --2- 172.21.15.172:0/2847647522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e315867240 0x55e31587ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.904+0000 7f58e1d9b640 1 -- 172.21.15.172:0/2847647522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c060 msgr2=0x55e31585e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:11.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.904+0000 7f58e1d9b640 1 --2- 172.21.15.172:0/2847647522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c060 0x55e31585e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.904+0000 7f58e1d9b640 1 -- 172.21.15.172:0/2847647522 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e315858420 con 0x55e31585c980 2024-09-10T09:05:11.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.904+0000 7f58e1d9b640 1 --2- 172.21.15.172:0/2847647522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c980 0x55e31587e9f0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f58d4005fd0 tx=0x7f58d406e580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6186d84831a90918 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:11.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.904+0000 7f58d3fff640 1 -- 172.21.15.172:0/2847647522 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d401d040 con 0x55e31585c980 2024-09-10T09:05:11.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.904+0000 7f58d3fff640 1 -- 172.21.15.172:0/2847647522 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58d409b4c0 con 0x55e31585c980 2024-09-10T09:05:11.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.904+0000 7f58d3fff640 1 -- 172.21.15.172:0/2847647522 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4097b30 con 0x55e31585c980 2024-09-10T09:05:11.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.905+0000 7f58f3099740 1 -- 172.21.15.172:0/2847647522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c980 msgr2=0x55e31587e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:11.910 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.905+0000 7f58f3099740 1 --2- 172.21.15.172:0/2847647522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c980 0x55e31587e9f0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f58d4005fd0 tx=0x7f58d406e580 comp rx=0 tx=0).stop 2024-09-10T09:05:11.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.905+0000 7f58f3099740 1 -- 172.21.15.172:0/2847647522 shutdown_connections 2024-09-10T09:05:11.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.905+0000 7f58f3099740 1 --2- 172.21.15.172:0/2847647522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e315867240 0x55e31587ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.905+0000 7f58f3099740 1 --2- 172.21.15.172:0/2847647522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c980 0x55e31587e9f0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.905+0000 7f58f3099740 1 --2- 172.21.15.172:0/2847647522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c060 0x55e31585e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.905+0000 7f58f3099740 1 -- 172.21.15.172:0/2847647522 >> 172.21.15.172:0/2847647522 conn(0x55e3158585f0 msgr2=0x55e31585dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:11.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.905+0000 7f58f3099740 1 -- 172.21.15.172:0/2847647522 shutdown_connections 2024-09-10T09:05:11.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.905+0000 7f58f3099740 1 -- 172.21.15.172:0/2847647522 wait complete. 2024-09-10T09:05:11.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.905+0000 7f58f3099740 1 -- start start 2024-09-10T09:05:11.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.905+0000 7f58f3099740 1 Processor -- start 2024-09-10T09:05:11.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58f3099740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c060 0x55e31586f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58f3099740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c980 0x55e31586f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58f3099740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e315867240 0x55e31586c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.911 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58f3099740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec1aff80 con 0x55e31585c060 2024-09-10T09:05:11.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58f3099740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec1af8b0 con 0x55e31585c980 2024-09-10T09:05:11.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58f3099740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec1acaf0 con 0x55e315867240 2024-09-10T09:05:11.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58f3099740 10 client.0 _ll_register_callbacks cb 0x7f58e4003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:11.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c060 0x55e31586f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c060 0x55e31586f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45500/0 (socket says 172.21.15.172:45500) 2024-09-10T09:05:11.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58ea59c640 1 -- 172.21.15.172:0/2766990642 learned_addr learned my addr 172.21.15.172:0/2766990642 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:11.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58e1d9b640 1 --2- 172.21.15.172:0/2766990642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c980 0x55e31586f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58ead9d640 1 --2- 172.21.15.172:0/2766990642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e315867240 0x55e31586c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58ea59c640 1 -- 172.21.15.172:0/2766990642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e315867240 msgr2=0x55e31586c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:11.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58ea59c640 1 --2- 172.21.15.172:0/2766990642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e315867240 0x55e31586c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58ea59c640 1 -- 172.21.15.172:0/2766990642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c980 msgr2=0x55e31586f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:11.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58ea59c640 1 --2- 172.21.15.172:0/2766990642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c980 0x55e31586f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58ea59c640 1 -- 172.21.15.172:0/2766990642 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e315859280 con 0x55e31585c060 2024-09-10T09:05:11.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58ead9d640 1 --2- 172.21.15.172:0/2766990642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e315867240 0x55e31586c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:05:11.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58ea59c640 1 --2- 172.21.15.172:0/2766990642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c060 0x55e31586f470 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f58dc013e20 tx=0x7f58dc013ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41025c58fff18602 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:11.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:05:11.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58d1ffb640 1 -- 172.21.15.172:0/2766990642 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc017040 con 0x55e31585c060 2024-09-10T09:05:11.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58d1ffb640 1 -- 172.21.15.172:0/2766990642 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f58dc00ed40 con 0x55e31585c060 2024-09-10T09:05:11.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.906+0000 7f58f3099740 1 -- 172.21.15.172:0/2766990642 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58d409b4c0 con 0x55e31585c060 2024-09-10T09:05:11.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.907+0000 7f58d1ffb640 1 -- 172.21.15.172:0/2766990642 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f58dc0059c0 con 0x55e31585c060 2024-09-10T09:05:11.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.907+0000 7f58f3099740 10 client.20067 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:11.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.907+0000 7f58f3099740 1 -- 172.21.15.172:0/2766990642 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e3158e2390 con 0x55e31585c060 2024-09-10T09:05:11.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.907+0000 7f58d1ffb640 1 -- 172.21.15.172:0/2766990642 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58d409b4c0 con 0x55e31585c060 2024-09-10T09:05:11.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.907+0000 7f58d1ffb640 1 -- 172.21.15.172:0/2766990642 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f58dc006040 con 0x55e31585c060 2024-09-10T09:05:11.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.907+0000 7f58f3099740 10 client.20067 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:11.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.907+0000 7f58f3099740 10 client.20067 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:05:11.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.907+0000 7f58f3099740 20 client.20067 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:11.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.907+0000 7f58f3099740 1 --2- 172.21.15.172:0/2766990642 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e3158e27e0 0x55e3158e4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:11.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.907+0000 7f58f3099740 4 client.20067 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "15ff85c3", "release": true, "await": true}] 2024-09-10T09:05:11.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.907+0000 7f58f3099740 1 -- 172.21.15.172:0/2766990642 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "set_id": "15ff85c3", "release": true, "await": true}) -- 0x55e3157c90a0 con 0x55e3158e27e0 2024-09-10T09:05:11.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.907+0000 7f58e1d9b640 1 --2- 172.21.15.172:0/2766990642 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e3158e27e0 0x55e3158e4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:11.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.908+0000 7f58d1ffb640 10 client.20067 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:11.914 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.908+0000 7f58e1d9b640 1 --2- 172.21.15.172:0/2766990642 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e3158e27e0 0x55e3158e4bd0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52a22385db0d3218 server_cookie=7df3ad971a467dbc in_seq=0 out_seq=0 2024-09-10T09:05:11.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.928+0000 7f58d1ffb640 1 -- 172.21.15.172:0/2766990642 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e3157c90a0 con 0x55e3158e27e0 2024-09-10T09:05:11.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.928+0000 7f58d1ffb640 10 client.20067 handle_command_reply: tid=0 2024-09-10T09:05:11.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.928+0000 7f58f3099740 1 client.20067 shutdown 2024-09-10T09:05:11.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.928+0000 7f58f3099740 1 -- 172.21.15.172:0/2766990642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c060 msgr2=0x55e31586f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:11.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.928+0000 7f58f3099740 1 --2- 172.21.15.172:0/2766990642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c060 0x55e31586f470 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f58dc013e20 tx=0x7f58dc013ef0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.929+0000 7f58f3099740 1 -- 172.21.15.172:0/2766990642 shutdown_connections 2024-09-10T09:05:11.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.929+0000 7f58f3099740 1 --2- 172.21.15.172:0/2766990642 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e3158e27e0 0x55e3158e4bd0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.930 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.929+0000 7f58f3099740 1 --2- 172.21.15.172:0/2766990642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e315867240 0x55e31586c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.929+0000 7f58f3099740 1 --2- 172.21.15.172:0/2766990642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31585c980 0x55e31586f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.929+0000 7f58f3099740 1 --2- 172.21.15.172:0/2766990642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e31585c060 0x55e31586f470 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.929+0000 7f58f3099740 1 -- 172.21.15.172:0/2766990642 >> 172.21.15.172:0/2766990642 conn(0x55e3158585f0 msgr2=0x55e31585e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:11.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.929+0000 7f58d1ffb640 0 client.20067 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:11.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.929+0000 7f58f3099740 1 -- 172.21.15.172:0/2766990642 shutdown_connections 2024-09-10T09:05:11.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.929+0000 7f58f3099740 1 -- 172.21.15.172:0/2766990642 wait complete. 2024-09-10T09:05:11.931 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.929+0000 7f58f3099740 20 client.20067 trim_cache size 0 max 16384 2024-09-10T09:05:11.931 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:05:11.932 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 44, 2024-09-10T09:05:11.932 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T09:05:11.932 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 60, 2024-09-10T09:05:11.932 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": { 2024-09-10T09:05:11.933 INFO:teuthology.orchestra.run.smithi172.stdout: "15ff85c3": { 2024-09-10T09:05:11.933 INFO:teuthology.orchestra.run.smithi172.stdout: "version": 60, 2024-09-10T09:05:11.933 INFO:teuthology.orchestra.run.smithi172.stdout: "age_ref": 0.0, 2024-09-10T09:05:11.933 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T09:05:11.933 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T09:05:11.933 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T09:05:11.933 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:05:11.933 INFO:teuthology.orchestra.run.smithi172.stdout: "timeout": 90.0, 2024-09-10T09:05:11.934 INFO:teuthology.orchestra.run.smithi172.stdout: "expiration": 139.8, 2024-09-10T09:05:11.934 INFO:teuthology.orchestra.run.smithi172.stdout: "members": { 2024-09-10T09:05:11.934 INFO:teuthology.orchestra.run.smithi172.stdout: "file:/": { 2024-09-10T09:05:11.934 INFO:teuthology.orchestra.run.smithi172.stdout: "excluded": false, 2024-09-10T09:05:11.934 INFO:teuthology.orchestra.run.smithi172.stdout: "state": { 2024-09-10T09:05:11.934 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RELEASED", 2024-09-10T09:05:11.934 INFO:teuthology.orchestra.run.smithi172.stdout: "age": 0.0 2024-09-10T09:05:11.934 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:05:11.934 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:05:11.934 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:05:11.934 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:05:11.934 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:05:11.935 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:05:11.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.930+0000 7f58f0cc0640 1 -- 172.21.15.172:0/921039781 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f58d80a2b80 msgr2=0x7f58d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:11.935 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.930+0000 7f58f0cc0640 1 --2- 172.21.15.172:0/921039781 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f58d80a2b80 0x7f58d80a5040 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f58e4005fe0 tx=0x7f58e4005c50 comp rx=0 tx=0).stop 2024-09-10T09:05:11.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.930+0000 7f58f0cc0640 1 -- 172.21.15.172:0/921039781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58ec1036d0 msgr2=0x7f58ec1988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:11.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.930+0000 7f58f0cc0640 1 --2- 172.21.15.172:0/921039781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58ec1036d0 0x7f58ec1988c0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f58d400c890 tx=0x7f58d400cc90 comp rx=0 tx=0).stop 2024-09-10T09:05:11.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.930+0000 7f58f0cc0640 1 -- 172.21.15.172:0/921039781 shutdown_connections 2024-09-10T09:05:11.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.930+0000 7f58f0cc0640 1 --2- 172.21.15.172:0/921039781 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f58d80a2b80 0x7f58d80a5040 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.930+0000 7f58f0cc0640 1 --2- 172.21.15.172:0/921039781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58ec10c190 0x7f58ec19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.930+0000 7f58f0cc0640 1 --2- 172.21.15.172:0/921039781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58ec1036d0 0x7f58ec1988c0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.930+0000 7f58f0cc0640 1 --2- 172.21.15.172:0/921039781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58ec102db0 0x7f58ec198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:11.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.930+0000 7f58f0cc0640 1 -- 172.21.15.172:0/921039781 >> 172.21.15.172:0/921039781 conn(0x7f58ec0fc7d0 msgr2=0x7f58ec0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:11.936 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.931+0000 7f58f0cc0640 1 -- 172.21.15.172:0/921039781 shutdown_connections 2024-09-10T09:05:11.937 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:11.931+0000 7f58f0cc0640 1 -- 172.21.15.172:0/921039781 wait complete. 2024-09-10T09:05:12.659 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '15ff85c3', total seconds elapsed: 23.0 2024-09-10T09:05:12.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:12 smithi174 ceph-mon[34068]: pgmap v919: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 68 KiB/s wr, 55 op/s 2024-09-10T09:05:13.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:12 smithi191 ceph-mon[35314]: pgmap v919: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 68 KiB/s wr, 55 op/s 2024-09-10T09:05:13.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:12 smithi172 ceph-mon[30854]: pgmap v919: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 68 KiB/s wr, 55 op/s 2024-09-10T09:05:14.788 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:14 smithi172 ceph-mon[30854]: pgmap v920: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 KiB/s wr, 40 op/s 2024-09-10T09:05:14.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:14 smithi174 ceph-mon[34068]: pgmap v920: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 KiB/s wr, 40 op/s 2024-09-10T09:05:15.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:14 smithi191 ceph-mon[35314]: pgmap v920: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 14 KiB/s wr, 40 op/s 2024-09-10T09:05:16.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:16 smithi174 ceph-mon[34068]: pgmap v921: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 84 KiB/s wr, 59 op/s 2024-09-10T09:05:16.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:16 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T09:05:16.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:16 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T09:05:17.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:16 smithi191 ceph-mon[35314]: pgmap v921: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 84 KiB/s wr, 59 op/s 2024-09-10T09:05:17.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:16 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T09:05:17.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:16 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T09:05:17.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:16 smithi172 ceph-mon[30854]: pgmap v921: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 84 KiB/s wr, 59 op/s 2024-09-10T09:05:17.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:16 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T09:05:17.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:16 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T09:05:17.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:17 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T09:05:17.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:17 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T09:05:17.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:17 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T09:05:18.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:17 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T09:05:18.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:17 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T09:05:18.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:17 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T09:05:18.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:17 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T09:05:18.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:17 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T09:05:18.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:17 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T09:05:18.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:18 smithi174 ceph-mon[34068]: pgmap v922: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 514 KiB/s wr, 165 op/s 2024-09-10T09:05:19.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:18 smithi191 ceph-mon[35314]: pgmap v922: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 514 KiB/s wr, 165 op/s 2024-09-10T09:05:19.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:18 smithi172 ceph-mon[30854]: pgmap v922: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 514 KiB/s wr, 165 op/s 2024-09-10T09:05:20.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:20 smithi174 ceph-mon[34068]: pgmap v923: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 567 KiB/s wr, 162 op/s 2024-09-10T09:05:21.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:20 smithi191 ceph-mon[35314]: pgmap v923: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 567 KiB/s wr, 162 op/s 2024-09-10T09:05:21.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:20 smithi172 ceph-mon[30854]: pgmap v923: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 567 KiB/s wr, 162 op/s 2024-09-10T09:05:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.347+0000 7fee67ed7640 1 -- 172.21.15.172:0/2805769517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee60102670 msgr2=0x7fee60104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.347+0000 7fee67ed7640 1 --2- 172.21.15.172:0/2805769517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee60102670 0x7fee60104a60 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009900 tx=0x7fee5c02a410 comp rx=0 tx=0).stop 2024-09-10T09:05:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.347+0000 7fee67ed7640 1 -- 172.21.15.172:0/2805769517 shutdown_connections 2024-09-10T09:05:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.347+0000 7fee67ed7640 1 --2- 172.21.15.172:0/2805769517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee601078d0 0x7fee60109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.347+0000 7fee67ed7640 1 --2- 172.21.15.172:0/2805769517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee60104fa0 0x7fee60107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.347+0000 7fee67ed7640 1 --2- 172.21.15.172:0/2805769517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee60102670 0x7fee60104a60 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.347+0000 7fee67ed7640 1 -- 172.21.15.172:0/2805769517 >> 172.21.15.172:0/2805769517 conn(0x7fee600fc1b0 msgr2=0x7fee600fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.347+0000 7fee67ed7640 1 -- 172.21.15.172:0/2805769517 shutdown_connections 2024-09-10T09:05:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.347+0000 7fee67ed7640 1 -- 172.21.15.172:0/2805769517 wait complete. 2024-09-10T09:05:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.348+0000 7fee67ed7640 1 Processor -- start 2024-09-10T09:05:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.348+0000 7fee67ed7640 1 -- start start 2024-09-10T09:05:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.348+0000 7fee67ed7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee60102670 0x7fee60197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.348+0000 7fee67ed7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee60104fa0 0x7fee60198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.348+0000 7fee67ed7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee601078d0 0x7fee6019f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.348+0000 7fee67ed7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fee6010d6c0 con 0x7fee60102670 2024-09-10T09:05:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.348+0000 7fee67ed7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee6010d540 con 0x7fee601078d0 2024-09-10T09:05:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.348+0000 7fee67ed7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fee6010d840 con 0x7fee60104fa0 2024-09-10T09:05:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.348+0000 7fee65c73640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee60102670 0x7fee60197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.348+0000 7fee65c73640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee60102670 0x7fee60197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40330/0 (socket says 172.21.15.172:40330) 2024-09-10T09:05:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.348+0000 7fee65c73640 1 -- 172.21.15.172:0/875958295 learned_addr learned my addr 172.21.15.172:0/875958295 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.348+0000 7fee65472640 1 --2- 172.21.15.172:0/875958295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee60104fa0 0x7fee60198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.349+0000 7fee66474640 1 --2- 172.21.15.172:0/875958295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee601078d0 0x7fee6019f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.349+0000 7fee65472640 1 -- 172.21.15.172:0/875958295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee601078d0 msgr2=0x7fee6019f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.349+0000 7fee65472640 1 --2- 172.21.15.172:0/875958295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee601078d0 0x7fee6019f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.349+0000 7fee65472640 1 -- 172.21.15.172:0/875958295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee60102670 msgr2=0x7fee60197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.349+0000 7fee65472640 1 --2- 172.21.15.172:0/875958295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee60102670 0x7fee60197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.349+0000 7fee65472640 1 -- 172.21.15.172:0/875958295 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6019f970 con 0x7fee60104fa0 2024-09-10T09:05:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.349+0000 7fee65c73640 1 --2- 172.21.15.172:0/875958295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee60102670 0x7fee60197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.349+0000 7fee65472640 1 --2- 172.21.15.172:0/875958295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee60104fa0 0x7fee60198290 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fee5000e9c0 tx=0x7fee5000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.349+0000 7fee56ffd640 1 -- 172.21.15.172:0/875958295 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5000ccc0 con 0x7fee60104fa0 2024-09-10T09:05:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.349+0000 7fee56ffd640 1 -- 172.21.15.172:0/875958295 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee500108e0 con 0x7fee60104fa0 2024-09-10T09:05:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.349+0000 7fee67ed7640 1 -- 172.21.15.172:0/875958295 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6019fc60 con 0x7fee60104fa0 2024-09-10T09:05:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.349+0000 7fee56ffd640 1 -- 172.21.15.172:0/875958295 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500187e0 con 0x7fee60104fa0 2024-09-10T09:05:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.349+0000 7fee67ed7640 1 -- 172.21.15.172:0/875958295 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee601a01b0 con 0x7fee60104fa0 2024-09-10T09:05:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.352+0000 7fee56ffd640 1 -- 172.21.15.172:0/875958295 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fee50010a90 con 0x7fee60104fa0 2024-09-10T09:05:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee6ac72740 1 Processor -- start 2024-09-10T09:05:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee6ac72740 1 -- start start 2024-09-10T09:05:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee6ac72740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784bdb0 0x55a03784c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee6ac72740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03784c6f0 0x55a0378511b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee6ac72740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a037851780 0x55a037853c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee6ac72740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a0377ab2f0 con 0x55a03784bdb0 2024-09-10T09:05:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee6ac72740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a037753270 con 0x55a03784c6f0 2024-09-10T09:05:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee6ac72740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a0377a72b0 con 0x55a037851780 2024-09-10T09:05:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee66474640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784bdb0 0x55a03784c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee65472640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a037851780 0x55a037853c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee66474640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784bdb0 0x55a03784c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40346/0 (socket says 172.21.15.172:40346) 2024-09-10T09:05:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee66474640 1 -- 172.21.15.172:0/2111716874 learned_addr learned my addr 172.21.15.172:0/2111716874 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee65c73640 1 --2- 172.21.15.172:0/2111716874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03784c6f0 0x55a0378511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee65472640 1 -- 172.21.15.172:0/2111716874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03784c6f0 msgr2=0x55a0378511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee65472640 1 --2- 172.21.15.172:0/2111716874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03784c6f0 0x55a0378511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee65472640 1 -- 172.21.15.172:0/2111716874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784bdb0 msgr2=0x55a03784c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee65472640 1 --2- 172.21.15.172:0/2111716874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784bdb0 0x55a03784c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.353+0000 7fee65472640 1 -- 172.21.15.172:0/2111716874 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a037752340 con 0x55a037851780 2024-09-10T09:05:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.354+0000 7fee65472640 1 --2- 172.21.15.172:0/2111716874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a037851780 0x55a037853c40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fee500ccdf0 tx=0x7fee500d7670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80c6b9db6a8f19f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.354+0000 7fee4bfff640 1 -- 172.21.15.172:0/2111716874 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500de070 con 0x55a037851780 2024-09-10T09:05:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.354+0000 7fee4bfff640 1 -- 172.21.15.172:0/2111716874 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee500cfcc0 con 0x55a037851780 2024-09-10T09:05:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.354+0000 7fee4bfff640 1 -- 172.21.15.172:0/2111716874 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500da680 con 0x55a037851780 2024-09-10T09:05:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.354+0000 7fee56ffd640 1 --2- 172.21.15.172:0/875958295 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee4c0a2cb0 0x7fee4c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.354+0000 7fee66474640 1 --2- 172.21.15.172:0/875958295 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee4c0a2cb0 0x7fee4c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.354+0000 7fee56ffd640 1 -- 172.21.15.172:0/875958295 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fee500cf710 con 0x7fee60104fa0 2024-09-10T09:05:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.355+0000 7fee6ac72740 1 -- 172.21.15.172:0/2111716874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a037851780 msgr2=0x55a037853c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.355+0000 7fee6ac72740 1 --2- 172.21.15.172:0/2111716874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a037851780 0x55a037853c40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fee500ccdf0 tx=0x7fee500d7670 comp rx=0 tx=0).stop 2024-09-10T09:05:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.355+0000 7fee6ac72740 1 -- 172.21.15.172:0/2111716874 shutdown_connections 2024-09-10T09:05:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.355+0000 7fee6ac72740 1 --2- 172.21.15.172:0/2111716874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a037851780 0x55a037853c40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.355+0000 7fee6ac72740 1 --2- 172.21.15.172:0/2111716874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03784c6f0 0x55a0378511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.355+0000 7fee6ac72740 1 --2- 172.21.15.172:0/2111716874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784bdb0 0x55a03784c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.355+0000 7fee6ac72740 1 -- 172.21.15.172:0/2111716874 >> 172.21.15.172:0/2111716874 conn(0x55a037847420 msgr2=0x55a037847820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.356+0000 7fee66474640 1 --2- 172.21.15.172:0/875958295 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee4c0a2cb0 0x7fee4c0a5170 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fee58011670 tx=0x7fee58009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.356+0000 7fee6ac72740 1 -- 172.21.15.172:0/2111716874 shutdown_connections 2024-09-10T09:05:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.356+0000 7fee6ac72740 1 -- 172.21.15.172:0/2111716874 wait complete. 2024-09-10T09:05:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.356+0000 7fee6ac72740 1 -- start start 2024-09-10T09:05:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.356+0000 7fee6ac72740 1 Processor -- start 2024-09-10T09:05:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.356+0000 7fee6ac72740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a03784bdb0 0x55a037856000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.356+0000 7fee6ac72740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784c6f0 0x55a037856540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.356+0000 7fee6ac72740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a037851780 0x55a03785d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.356+0000 7fee6ac72740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a037854750 con 0x55a03784c6f0 2024-09-10T09:05:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.356+0000 7fee6ac72740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a037734290 con 0x55a037851780 2024-09-10T09:05:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.356+0000 7fee6ac72740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a0378548d0 con 0x55a03784bdb0 2024-09-10T09:05:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.356+0000 7fee6ac72740 10 client.0 _ll_register_callbacks cb 0x55a037765140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.357+0000 7fee65c73640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a03784bdb0 0x55a037856000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.357+0000 7fee65472640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784c6f0 0x55a037856540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.357+0000 7fee66474640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a037851780 0x55a03785d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.357+0000 7fee65472640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784c6f0 0x55a037856540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40358/0 (socket says 172.21.15.172:40358) 2024-09-10T09:05:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.357+0000 7fee65472640 1 -- 172.21.15.172:0/874895935 learned_addr learned my addr 172.21.15.172:0/874895935 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.357+0000 7fee66474640 1 -- 172.21.15.172:0/874895935 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a03784bdb0 msgr2=0x55a037856000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.357+0000 7fee66474640 1 --2- 172.21.15.172:0/874895935 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a03784bdb0 0x55a037856000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.357+0000 7fee66474640 1 -- 172.21.15.172:0/874895935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784c6f0 msgr2=0x55a037856540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.357+0000 7fee66474640 1 --2- 172.21.15.172:0/874895935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784c6f0 0x55a037856540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.357+0000 7fee66474640 1 -- 172.21.15.172:0/874895935 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0374be210 con 0x55a037851780 2024-09-10T09:05:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.357+0000 7fee66474640 1 --2- 172.21.15.172:0/874895935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a037851780 0x55a03785d510 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fee58003f20 tx=0x7fee58004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcbff773cb21b8ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.357+0000 7fee497fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:05:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.357+0000 7fee497fa640 1 -- 172.21.15.172:0/874895935 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58011020 con 0x55a037851780 2024-09-10T09:05:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.357+0000 7fee6ac72740 1 -- 172.21.15.172:0/874895935 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee500cfcc0 con 0x55a037851780 2024-09-10T09:05:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.357+0000 7fee497fa640 1 -- 172.21.15.172:0/874895935 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fee580111c0 con 0x55a037851780 2024-09-10T09:05:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.358+0000 7fee497fa640 1 -- 172.21.15.172:0/874895935 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fee5800f950 con 0x55a037851780 2024-09-10T09:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.358+0000 7fee6ac72740 10 client.28796 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.358+0000 7fee6ac72740 1 -- 172.21.15.172:0/874895935 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a03758a620 con 0x55a037851780 2024-09-10T09:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.358+0000 7fee497fa640 1 -- 172.21.15.172:0/874895935 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee500cfcc0 con 0x55a037851780 2024-09-10T09:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.358+0000 7fee497fa640 1 -- 172.21.15.172:0/874895935 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fee58026070 con 0x55a037851780 2024-09-10T09:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.359+0000 7fee6ac72740 10 client.28796 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.359+0000 7fee6ac72740 10 client.28796 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.359+0000 7fee6ac72740 20 client.28796 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.359+0000 7fee6ac72740 1 --2- 172.21.15.172:0/874895935 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a037845a90 0x55a037845eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.359+0000 7fee6ac72740 4 client.28796 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:05:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.359+0000 7fee6ac72740 1 -- 172.21.15.172:0/874895935 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0377b8a20 con 0x55a037845a90 2024-09-10T09:05:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.359+0000 7fee65c73640 1 --2- 172.21.15.172:0/874895935 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a037845a90 0x55a037845eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.360+0000 7fee65c73640 1 --2- 172.21.15.172:0/874895935 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a037845a90 0x55a037845eb0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7135b41334e62f3 server_cookie=74731e3211bc1025 in_seq=0 out_seq=0 2024-09-10T09:05:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.360+0000 7fee497fa640 10 client.28796 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.360+0000 7fee497fa640 1 -- 172.21.15.172:0/874895935 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0377b8a20 con 0x55a037845a90 2024-09-10T09:05:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.360+0000 7fee497fa640 10 client.28796 handle_command_reply: tid=0 2024-09-10T09:05:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.360+0000 7fee6ac72740 1 client.28796 shutdown 2024-09-10T09:05:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.361+0000 7fee6ac72740 1 -- 172.21.15.172:0/874895935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a037851780 msgr2=0x55a03785d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.361+0000 7fee6ac72740 1 --2- 172.21.15.172:0/874895935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a037851780 0x55a03785d510 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fee58003f20 tx=0x7fee58004ab0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.361+0000 7fee6ac72740 1 -- 172.21.15.172:0/874895935 shutdown_connections 2024-09-10T09:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.361+0000 7fee6ac72740 1 --2- 172.21.15.172:0/874895935 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a037845a90 0x55a037845eb0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.361+0000 7fee497fa640 0 client.28796 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.361+0000 7fee6ac72740 1 --2- 172.21.15.172:0/874895935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a037851780 0x55a03785d510 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.361+0000 7fee6ac72740 1 --2- 172.21.15.172:0/874895935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784c6f0 0x55a037856540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.361+0000 7fee6ac72740 1 --2- 172.21.15.172:0/874895935 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a03784bdb0 0x55a037856000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.361+0000 7fee6ac72740 1 -- 172.21.15.172:0/874895935 >> 172.21.15.172:0/874895935 conn(0x55a037847420 msgr2=0x55a0378421c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.361+0000 7fee6ac72740 1 -- 172.21.15.172:0/874895935 shutdown_connections 2024-09-10T09:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.361+0000 7fee6ac72740 1 -- 172.21.15.172:0/874895935 wait complete. 2024-09-10T09:05:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.361+0000 7fee6ac72740 20 client.28796 trim_cache size 0 max 16384 2024-09-10T09:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.377+0000 7fee6ac72740 1 Processor -- start 2024-09-10T09:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.377+0000 7fee6ac72740 1 -- start start 2024-09-10T09:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.377+0000 7fee6ac72740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a037845a90 0x55a0378420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.377+0000 7fee6ac72740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a03784bdb0 0x55a03785d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.377+0000 7fee6ac72740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03784c6f0 0x55a037847800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.377+0000 7fee6ac72740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fee601a8350 con 0x55a037845a90 2024-09-10T09:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.377+0000 7fee6ac72740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee601a9e90 con 0x55a03784c6f0 2024-09-10T09:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.377+0000 7fee6ac72740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fee601a8a20 con 0x55a03784bdb0 2024-09-10T09:05:21.379 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.377+0000 7fee65c73640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a037845a90 0x55a0378420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.377+0000 7fee65472640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a03784bdb0 0x55a03785d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.377+0000 7fee66474640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03784c6f0 0x55a037847800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.377+0000 7fee65c73640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a037845a90 0x55a0378420b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40372/0 (socket says 172.21.15.172:40372) 2024-09-10T09:05:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.377+0000 7fee65c73640 1 -- 172.21.15.172:0/637258535 learned_addr learned my addr 172.21.15.172:0/637258535 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.378+0000 7fee65c73640 1 -- 172.21.15.172:0/637258535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a03784bdb0 msgr2=0x55a03785d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.378+0000 7fee65c73640 1 --2- 172.21.15.172:0/637258535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a03784bdb0 0x55a03785d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.378+0000 7fee65c73640 1 -- 172.21.15.172:0/637258535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03784c6f0 msgr2=0x55a037847800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.378+0000 7fee65c73640 1 --2- 172.21.15.172:0/637258535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03784c6f0 0x55a037847800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.378+0000 7fee65c73640 1 -- 172.21.15.172:0/637258535 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a037847240 con 0x55a037845a90 2024-09-10T09:05:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.378+0000 7fee65c73640 1 --2- 172.21.15.172:0/637258535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a037845a90 0x55a0378420b0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009230 tx=0x7fee5c032a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfe171fb14a7cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:21.380 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.378+0000 7fee4b7fe640 1 -- 172.21.15.172:0/637258535 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c04d020 con 0x55a037845a90 2024-09-10T09:05:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.378+0000 7fee4b7fe640 1 -- 172.21.15.172:0/637258535 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee5c004240 con 0x55a037845a90 2024-09-10T09:05:21.381 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.378+0000 7fee4b7fe640 1 -- 172.21.15.172:0/637258535 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c035040 con 0x55a037845a90 2024-09-10T09:05:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.379+0000 7fee6ac72740 1 -- 172.21.15.172:0/637258535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a037845a90 msgr2=0x55a0378420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.379+0000 7fee6ac72740 1 --2- 172.21.15.172:0/637258535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a037845a90 0x55a0378420b0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009230 tx=0x7fee5c032a90 comp rx=0 tx=0).stop 2024-09-10T09:05:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.379+0000 7fee6ac72740 1 -- 172.21.15.172:0/637258535 shutdown_connections 2024-09-10T09:05:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.379+0000 7fee6ac72740 1 --2- 172.21.15.172:0/637258535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a03784c6f0 0x55a037847800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.379+0000 7fee6ac72740 1 --2- 172.21.15.172:0/637258535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a03784bdb0 0x55a03785d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.379+0000 7fee6ac72740 1 --2- 172.21.15.172:0/637258535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a037845a90 0x55a0378420b0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.379+0000 7fee6ac72740 1 -- 172.21.15.172:0/637258535 >> 172.21.15.172:0/637258535 conn(0x55a037847420 msgr2=0x55a0378419a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.379+0000 7fee6ac72740 1 -- 172.21.15.172:0/637258535 shutdown_connections 2024-09-10T09:05:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.379+0000 7fee6ac72740 1 -- 172.21.15.172:0/637258535 wait complete. 2024-09-10T09:05:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee6ac72740 1 -- start start 2024-09-10T09:05:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee6ac72740 1 Processor -- start 2024-09-10T09:05:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee6ac72740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a037845a90 0x55a03785d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee6ac72740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784bdb0 0x55a037859f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee6ac72740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a03784c6f0 0x55a03785a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee6ac72740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fee601a97c0 con 0x55a03784bdb0 2024-09-10T09:05:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee6ac72740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee601a90f0 con 0x55a037845a90 2024-09-10T09:05:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee6ac72740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fee601a6330 con 0x55a03784c6f0 2024-09-10T09:05:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee6ac72740 10 client.0 _ll_register_callbacks cb 0x7fee58009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee65472640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784bdb0 0x55a037859f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee66474640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a03784c6f0 0x55a03785a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee65c73640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a037845a90 0x55a03785d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee65472640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784bdb0 0x55a037859f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:40384/0 (socket says 172.21.15.172:40384) 2024-09-10T09:05:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee65472640 1 -- 172.21.15.172:0/3993808397 learned_addr learned my addr 172.21.15.172:0/3993808397 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee66474640 1 -- 172.21.15.172:0/3993808397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a037845a90 msgr2=0x55a03785d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee66474640 1 --2- 172.21.15.172:0/3993808397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a037845a90 0x55a03785d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee66474640 1 -- 172.21.15.172:0/3993808397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784bdb0 msgr2=0x55a037859f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee66474640 1 --2- 172.21.15.172:0/3993808397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784bdb0 0x55a037859f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.380+0000 7fee66474640 1 -- 172.21.15.172:0/3993808397 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0378cf540 con 0x55a03784c6f0 2024-09-10T09:05:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.381+0000 7fee65472640 1 --2- 172.21.15.172:0/3993808397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784bdb0 0x55a037859f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.381+0000 7fee66474640 1 --2- 172.21.15.172:0/3993808397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a03784c6f0 0x55a03785a5c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fee58010ee0 tx=0x7fee58011190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=362fac120ed5ae15 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.381+0000 7fee497fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:05:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.381+0000 7fee497fa640 1 -- 172.21.15.172:0/3993808397 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5803a060 con 0x55a03784c6f0 2024-09-10T09:05:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.381+0000 7fee497fa640 1 -- 172.21.15.172:0/3993808397 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fee58010840 con 0x55a03784c6f0 2024-09-10T09:05:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.381+0000 7fee6ac72740 1 -- 172.21.15.172:0/3993808397 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee5c004240 con 0x55a03784c6f0 2024-09-10T09:05:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.381+0000 7fee497fa640 1 -- 172.21.15.172:0/3993808397 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fee58041050 con 0x55a03784c6f0 2024-09-10T09:05:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.381+0000 7fee6ac72740 10 client.28699 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.381+0000 7fee6ac72740 1 -- 172.21.15.172:0/3993808397 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a0378d08c0 con 0x55a03784c6f0 2024-09-10T09:05:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.381+0000 7fee497fa640 1 -- 172.21.15.172:0/3993808397 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee5c004240 con 0x55a03784c6f0 2024-09-10T09:05:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.381+0000 7fee497fa640 1 -- 172.21.15.172:0/3993808397 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fee58026050 con 0x55a03784c6f0 2024-09-10T09:05:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.381+0000 7fee6ac72740 10 client.28699 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.381+0000 7fee6ac72740 10 client.28699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.381+0000 7fee6ac72740 20 client.28699 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.382+0000 7fee6ac72740 1 --2- 172.21.15.172:0/3993808397 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a037851780 0x55a0378d32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.382+0000 7fee6ac72740 4 client.28699 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:05:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.382+0000 7fee6ac72740 1 -- 172.21.15.172:0/3993808397 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0377b2d70 con 0x55a037851780 2024-09-10T09:05:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.382+0000 7fee65c73640 1 --2- 172.21.15.172:0/3993808397 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a037851780 0x55a0378d32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.382+0000 7fee497fa640 10 client.28699 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.382+0000 7fee65c73640 1 --2- 172.21.15.172:0/3993808397 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a037851780 0x55a0378d32e0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da359bda8259192 server_cookie=2d64d0fcb6165059 in_seq=0 out_seq=0 2024-09-10T09:05:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.389+0000 7fee497fa640 1 -- 172.21.15.172:0/3993808397 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0377b2d70 con 0x55a037851780 2024-09-10T09:05:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.389+0000 7fee497fa640 10 client.28699 handle_command_reply: tid=0 2024-09-10T09:05:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.389+0000 7fee6ac72740 1 client.28699 shutdown 2024-09-10T09:05:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.389+0000 7fee6ac72740 1 -- 172.21.15.172:0/3993808397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a03784c6f0 msgr2=0x55a03785a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.389+0000 7fee6ac72740 1 --2- 172.21.15.172:0/3993808397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a03784c6f0 0x55a03785a5c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fee58010ee0 tx=0x7fee58011190 comp rx=0 tx=0).stop 2024-09-10T09:05:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.389+0000 7fee6ac72740 1 -- 172.21.15.172:0/3993808397 shutdown_connections 2024-09-10T09:05:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.389+0000 7fee6ac72740 1 --2- 172.21.15.172:0/3993808397 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55a037851780 0x55a0378d32e0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.389+0000 7fee6ac72740 1 --2- 172.21.15.172:0/3993808397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a03784c6f0 0x55a03785a5c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.389+0000 7fee6ac72740 1 --2- 172.21.15.172:0/3993808397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a03784bdb0 0x55a037859f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.389+0000 7fee6ac72740 1 --2- 172.21.15.172:0/3993808397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a037845a90 0x55a03785d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.389+0000 7fee497fa640 0 client.28699 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.389+0000 7fee6ac72740 1 -- 172.21.15.172:0/3993808397 >> 172.21.15.172:0/3993808397 conn(0x55a037847420 msgr2=0x55a037855490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.389+0000 7fee6ac72740 1 -- 172.21.15.172:0/3993808397 shutdown_connections 2024-09-10T09:05:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.389+0000 7fee6ac72740 1 -- 172.21.15.172:0/3993808397 wait complete. 2024-09-10T09:05:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.389+0000 7fee6ac72740 20 client.28699 trim_cache size 0 max 16384 2024-09-10T09:05:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:05:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:05:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:05:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:05:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.390+0000 7fee67ed7640 1 -- 172.21.15.172:0/875958295 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee4c0a2cb0 msgr2=0x7fee4c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.390+0000 7fee67ed7640 1 --2- 172.21.15.172:0/875958295 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee4c0a2cb0 0x7fee4c0a5170 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fee58011670 tx=0x7fee58009500 comp rx=0 tx=0).stop 2024-09-10T09:05:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.390+0000 7fee67ed7640 1 -- 172.21.15.172:0/875958295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee60104fa0 msgr2=0x7fee60198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.390+0000 7fee67ed7640 1 --2- 172.21.15.172:0/875958295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee60104fa0 0x7fee60198290 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fee5000e9c0 tx=0x7fee5000ee80 comp rx=0 tx=0).stop 2024-09-10T09:05:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.390+0000 7fee67ed7640 1 -- 172.21.15.172:0/875958295 shutdown_connections 2024-09-10T09:05:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.390+0000 7fee67ed7640 1 --2- 172.21.15.172:0/875958295 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fee4c0a2cb0 0x7fee4c0a5170 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.390+0000 7fee67ed7640 1 --2- 172.21.15.172:0/875958295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee601078d0 0x7fee6019f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.390+0000 7fee67ed7640 1 --2- 172.21.15.172:0/875958295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fee60104fa0 0x7fee60198290 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.390+0000 7fee67ed7640 1 --2- 172.21.15.172:0/875958295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee60102670 0x7fee60197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.390+0000 7fee67ed7640 1 -- 172.21.15.172:0/875958295 >> 172.21.15.172:0/875958295 conn(0x7fee600fc1b0 msgr2=0x7fee601030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.390+0000 7fee67ed7640 1 -- 172.21.15.172:0/875958295 shutdown_connections 2024-09-10T09:05:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:21.390+0000 7fee67ed7640 1 -- 172.21.15.172:0/875958295 wait complete. 2024-09-10T09:05:22.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:22 smithi174 ceph-mon[34068]: pgmap v924: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 782 KiB/s wr, 213 op/s 2024-09-10T09:05:23.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:22 smithi191 ceph-mon[35314]: pgmap v924: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 782 KiB/s wr, 213 op/s 2024-09-10T09:05:23.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:22 smithi172 ceph-mon[30854]: pgmap v924: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 782 KiB/s wr, 213 op/s 2024-09-10T09:05:24.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:24 smithi174 ceph-mon[34068]: pgmap v925: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.0 MiB/s wr, 269 op/s 2024-09-10T09:05:25.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:24 smithi191 ceph-mon[35314]: pgmap v925: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.0 MiB/s wr, 269 op/s 2024-09-10T09:05:25.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:24 smithi172 ceph-mon[30854]: pgmap v925: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.0 MiB/s wr, 269 op/s 2024-09-10T09:05:26.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:26 smithi174 ceph-mon[34068]: pgmap v926: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 MiB/s wr, 296 op/s 2024-09-10T09:05:27.052 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:26 smithi191 ceph-mon[35314]: pgmap v926: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 MiB/s wr, 296 op/s 2024-09-10T09:05:27.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:26 smithi172 ceph-mon[30854]: pgmap v926: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 MiB/s wr, 296 op/s 2024-09-10T09:05:28.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:28 smithi174 ceph-mon[34068]: pgmap v927: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.3 MiB/s wr, 338 op/s 2024-09-10T09:05:29.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:28 smithi191 ceph-mon[35314]: pgmap v927: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.3 MiB/s wr, 338 op/s 2024-09-10T09:05:29.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:28 smithi172 ceph-mon[30854]: pgmap v927: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.3 MiB/s wr, 338 op/s 2024-09-10T09:05:30.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:30 smithi174 ceph-mon[34068]: pgmap v928: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 913 KiB/s wr, 239 op/s 2024-09-10T09:05:31.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:30 smithi191 ceph-mon[35314]: pgmap v928: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 913 KiB/s wr, 239 op/s 2024-09-10T09:05:31.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:30 smithi172 ceph-mon[30854]: pgmap v928: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 913 KiB/s wr, 239 op/s 2024-09-10T09:05:32.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:32 smithi174 ceph-mon[34068]: pgmap v929: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 991 KiB/s wr, 260 op/s 2024-09-10T09:05:33.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:32 smithi191 ceph-mon[35314]: pgmap v929: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 991 KiB/s wr, 260 op/s 2024-09-10T09:05:33.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:32 smithi172 ceph-mon[30854]: pgmap v929: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 991 KiB/s wr, 260 op/s 2024-09-10T09:05:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.490+0000 7f65025e4640 1 -- 172.21.15.172:0/1872820052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64fc1001d0 msgr2=0x7f64fc1005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.490+0000 7f65025e4640 1 --2- 172.21.15.172:0/1872820052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64fc1001d0 0x7f64fc1005d0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009900 tx=0x7f64ec02a410 comp rx=0 tx=0).stop 2024-09-10T09:05:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.490+0000 7f65025e4640 1 -- 172.21.15.172:0/1872820052 shutdown_connections 2024-09-10T09:05:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.490+0000 7f65025e4640 1 --2- 172.21.15.172:0/1872820052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64fc101d80 0x7f64fc108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.490+0000 7f65025e4640 1 --2- 172.21.15.172:0/1872820052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64fc1013c0 0x7f64fc101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.490+0000 7f65025e4640 1 --2- 172.21.15.172:0/1872820052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64fc1001d0 0x7f64fc1005d0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.490+0000 7f65025e4640 1 -- 172.21.15.172:0/1872820052 >> 172.21.15.172:0/1872820052 conn(0x7f64fc0fb930 msgr2=0x7f64fc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.491+0000 7f65025e4640 1 -- 172.21.15.172:0/1872820052 shutdown_connections 2024-09-10T09:05:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.491+0000 7f65025e4640 1 -- 172.21.15.172:0/1872820052 wait complete. 2024-09-10T09:05:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.491+0000 7f65025e4640 1 Processor -- start 2024-09-10T09:05:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.491+0000 7f65025e4640 1 -- start start 2024-09-10T09:05:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.491+0000 7f65025e4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64fc1001d0 0x7f64fc197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.491+0000 7f65025e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64fc1013c0 0x7f64fc198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.491+0000 7f65025e4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64fc101d80 0x7f64fc19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.491+0000 7f65025e4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc10b4b0 con 0x7f64fc101d80 2024-09-10T09:05:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.491+0000 7f65025e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc10b330 con 0x7f64fc1013c0 2024-09-10T09:05:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.491+0000 7f65025e4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc10b630 con 0x7f64fc1001d0 2024-09-10T09:05:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.492+0000 7f64fbfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64fc1001d0 0x7f64fc197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.492+0000 7f6500b81640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64fc101d80 0x7f64fc19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.492+0000 7f64f3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64fc1013c0 0x7f64fc198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.492+0000 7f6500b81640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64fc101d80 0x7f64fc19f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48072/0 (socket says 172.21.15.172:48072) 2024-09-10T09:05:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.492+0000 7f64fbfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64fc1001d0 0x7f64fc197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:48588/0 (socket says 172.21.15.172:48588) 2024-09-10T09:05:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.492+0000 7f6500b81640 1 -- 172.21.15.172:0/1155266568 learned_addr learned my addr 172.21.15.172:0/1155266568 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.492+0000 7f6500b81640 1 -- 172.21.15.172:0/1155266568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64fc1001d0 msgr2=0x7f64fc197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.492+0000 7f6500b81640 1 --2- 172.21.15.172:0/1155266568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64fc1001d0 0x7f64fc197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.492+0000 7f6500b81640 1 -- 172.21.15.172:0/1155266568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64fc1013c0 msgr2=0x7f64fc198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.492+0000 7f6500b81640 1 --2- 172.21.15.172:0/1155266568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64fc1013c0 0x7f64fc198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.492+0000 7f6500b81640 1 -- 172.21.15.172:0/1155266568 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc19f940 con 0x7f64fc101d80 2024-09-10T09:05:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.492+0000 7f6500b81640 1 --2- 172.21.15.172:0/1155266568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64fc101d80 0x7f64fc19f2a0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f64f400ec60 tx=0x7f64f400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.492+0000 7f64f9ffb640 1 -- 172.21.15.172:0/1155266568 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f4004d20 con 0x7f64fc101d80 2024-09-10T09:05:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.492+0000 7f65025e4640 1 -- 172.21.15.172:0/1155266568 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64fc19fc30 con 0x7f64fc101d80 2024-09-10T09:05:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.492+0000 7f64f9ffb640 1 -- 172.21.15.172:0/1155266568 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64f400ba20 con 0x7f64fc101d80 2024-09-10T09:05:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.492+0000 7f64f9ffb640 1 -- 172.21.15.172:0/1155266568 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f40056c0 con 0x7f64fc101d80 2024-09-10T09:05:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.492+0000 7f65025e4640 1 -- 172.21.15.172:0/1155266568 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64fc1a0150 con 0x7f64fc101d80 2024-09-10T09:05:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.495+0000 7f64f9ffb640 1 -- 172.21.15.172:0/1155266568 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f64f400bbd0 con 0x7f64fc101d80 2024-09-10T09:05:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.495+0000 7f64f9ffb640 1 --2- 172.21.15.172:0/1155266568 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f64e80a2c30 0x7f64e80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.496+0000 7f64f9ffb640 1 -- 172.21.15.172:0/1155266568 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f64f40cfa00 con 0x7f64fc101d80 2024-09-10T09:05:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.496+0000 7f64fbfff640 1 --2- 172.21.15.172:0/1155266568 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f64e80a2c30 0x7f64e80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.496+0000 7f650545a740 1 Processor -- start 2024-09-10T09:05:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.496+0000 7f650545a740 1 -- start start 2024-09-10T09:05:34.499 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.496+0000 7f650545a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813929a0 0x559a81392da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.496+0000 7f650545a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813932e0 0x559a81397da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.496+0000 7f650545a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 0x559a8139a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.496+0000 7f650545a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559a812f62f0 con 0x559a81398370 2024-09-10T09:05:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.496+0000 7f650545a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559a8129e270 con 0x559a813929a0 2024-09-10T09:05:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.496+0000 7f650545a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559a812f22b0 con 0x559a813932e0 2024-09-10T09:05:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.496+0000 7f64fbfff640 1 --2- 172.21.15.172:0/1155266568 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f64e80a2c30 0x7f64e80a50f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f64ec004770 tx=0x7f64ec000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.496+0000 7f64fbfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813929a0 0x559a81392da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.496+0000 7f6500b81640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 0x559a8139a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.496+0000 7f6500b81640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 0x559a8139a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48082/0 (socket says 172.21.15.172:48082) 2024-09-10T09:05:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.496+0000 7f6500b81640 1 -- 172.21.15.172:0/2407116950 learned_addr learned my addr 172.21.15.172:0/2407116950 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.496+0000 7f64f3fff640 1 --2- 172.21.15.172:0/2407116950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813932e0 0x559a81397da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.500 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.497+0000 7f64f3fff640 1 -- 172.21.15.172:0/2407116950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813929a0 msgr2=0x559a81392da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.497+0000 7f64f3fff640 1 --2- 172.21.15.172:0/2407116950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813929a0 0x559a81392da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.497+0000 7f64f3fff640 1 -- 172.21.15.172:0/2407116950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 msgr2=0x559a8139a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.497+0000 7f64f3fff640 1 --2- 172.21.15.172:0/2407116950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 0x559a8139a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.497+0000 7f64f3fff640 1 -- 172.21.15.172:0/2407116950 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a8129d340 con 0x559a813932e0 2024-09-10T09:05:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.497+0000 7f6500b81640 1 --2- 172.21.15.172:0/2407116950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 0x559a8139a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.497+0000 7f64f3fff640 1 --2- 172.21.15.172:0/2407116950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813932e0 0x559a81397da0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009c20 tx=0x7f64e4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=621d4aec59d0191a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.497+0000 7f64f27fc640 1 -- 172.21.15.172:0/2407116950 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4004050 con 0x559a813932e0 2024-09-10T09:05:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.497+0000 7f64f27fc640 1 -- 172.21.15.172:0/2407116950 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64e40054d0 con 0x559a813932e0 2024-09-10T09:05:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.497+0000 7f64f27fc640 1 -- 172.21.15.172:0/2407116950 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4004c90 con 0x559a813932e0 2024-09-10T09:05:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.498+0000 7f650545a740 1 -- 172.21.15.172:0/2407116950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813932e0 msgr2=0x559a81397da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.498+0000 7f650545a740 1 --2- 172.21.15.172:0/2407116950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813932e0 0x559a81397da0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009c20 tx=0x7f64e4011810 comp rx=0 tx=0).stop 2024-09-10T09:05:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.498+0000 7f650545a740 1 -- 172.21.15.172:0/2407116950 shutdown_connections 2024-09-10T09:05:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.498+0000 7f650545a740 1 --2- 172.21.15.172:0/2407116950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 0x559a8139a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.498+0000 7f650545a740 1 --2- 172.21.15.172:0/2407116950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813932e0 0x559a81397da0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.498+0000 7f650545a740 1 --2- 172.21.15.172:0/2407116950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813929a0 0x559a81392da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.498+0000 7f650545a740 1 -- 172.21.15.172:0/2407116950 >> 172.21.15.172:0/2407116950 conn(0x559a8138e010 msgr2=0x559a8138e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.498+0000 7f650545a740 1 -- 172.21.15.172:0/2407116950 shutdown_connections 2024-09-10T09:05:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.499+0000 7f650545a740 1 -- 172.21.15.172:0/2407116950 wait complete. 2024-09-10T09:05:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.499+0000 7f650545a740 1 -- start start 2024-09-10T09:05:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.499+0000 7f650545a740 1 Processor -- start 2024-09-10T09:05:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f650545a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813929a0 0x559a813a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f650545a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813932e0 0x559a813a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f650545a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 0x559a813a8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f650545a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559a8139f750 con 0x559a81398370 2024-09-10T09:05:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f650545a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559a8127f290 con 0x559a813932e0 2024-09-10T09:05:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f650545a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559a8139f8d0 con 0x559a813929a0 2024-09-10T09:05:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f64f3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813932e0 0x559a813a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f64f3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813932e0 0x559a813a14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:53348/0 (socket says 172.21.15.172:53348) 2024-09-10T09:05:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f64f3fff640 1 -- 172.21.15.172:0/2120427885 learned_addr learned my addr 172.21.15.172:0/2120427885 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f650545a740 10 client.0 _ll_register_callbacks cb 0x559a812b0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f64fbfff640 1 --2- 172.21.15.172:0/2120427885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813929a0 0x559a813a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f6500b81640 1 --2- 172.21.15.172:0/2120427885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 0x559a813a8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f64f3fff640 1 -- 172.21.15.172:0/2120427885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813929a0 msgr2=0x559a813a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f64f3fff640 1 --2- 172.21.15.172:0/2120427885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813929a0 0x559a813a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f64f3fff640 1 -- 172.21.15.172:0/2120427885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 msgr2=0x559a813a8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f64f3fff640 1 --2- 172.21.15.172:0/2120427885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 0x559a813a8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f64f3fff640 1 -- 172.21.15.172:0/2120427885 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a81009210 con 0x559a813932e0 2024-09-10T09:05:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.500+0000 7f6500b81640 1 --2- 172.21.15.172:0/2120427885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 0x559a813a8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:05:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.501+0000 7f64f3fff640 1 --2- 172.21.15.172:0/2120427885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813932e0 0x559a813a14e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f64e4000c00 tx=0x7f64e4005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5c44cc49e0a19a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.501+0000 7f64db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:05:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.501+0000 7f64db7fe640 1 -- 172.21.15.172:0/2120427885 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4029070 con 0x559a813932e0 2024-09-10T09:05:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.501+0000 7f64db7fe640 1 -- 172.21.15.172:0/2120427885 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f64e4003a00 con 0x559a813932e0 2024-09-10T09:05:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.501+0000 7f650545a740 1 -- 172.21.15.172:0/2120427885 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64e40054d0 con 0x559a813932e0 2024-09-10T09:05:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.501+0000 7f64db7fe640 1 -- 172.21.15.172:0/2120427885 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f64e4035050 con 0x559a813932e0 2024-09-10T09:05:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.501+0000 7f650545a740 10 client.28814 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.501+0000 7f650545a740 1 -- 172.21.15.172:0/2120427885 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559a810d5620 con 0x559a813932e0 2024-09-10T09:05:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.501+0000 7f64db7fe640 1 -- 172.21.15.172:0/2120427885 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64e40054d0 con 0x559a813932e0 2024-09-10T09:05:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.501+0000 7f64db7fe640 1 -- 172.21.15.172:0/2120427885 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f64e4023430 con 0x559a813932e0 2024-09-10T09:05:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.502+0000 7f650545a740 10 client.28814 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.502+0000 7f650545a740 10 client.28814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.502+0000 7f650545a740 20 client.28814 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.502+0000 7f650545a740 1 --2- 172.21.15.172:0/2120427885 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559a813ac980 0x559a813aed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.502+0000 7f650545a740 4 client.28814 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:05:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.502+0000 7f650545a740 1 -- 172.21.15.172:0/2120427885 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a81303a20 con 0x559a813ac980 2024-09-10T09:05:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.502+0000 7f64fbfff640 1 --2- 172.21.15.172:0/2120427885 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559a813ac980 0x559a813aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.502+0000 7f64db7fe640 10 client.28814 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.502+0000 7f64fbfff640 1 --2- 172.21.15.172:0/2120427885 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559a813ac980 0x559a813aed70 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4cc38b8a4128ba8 server_cookie=8e62b90aad3c3a21 in_seq=0 out_seq=0 2024-09-10T09:05:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.503+0000 7f64db7fe640 1 -- 172.21.15.172:0/2120427885 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a81303a20 con 0x559a813ac980 2024-09-10T09:05:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.503+0000 7f64db7fe640 10 client.28814 handle_command_reply: tid=0 2024-09-10T09:05:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.503+0000 7f650545a740 1 client.28814 shutdown 2024-09-10T09:05:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.503+0000 7f650545a740 1 -- 172.21.15.172:0/2120427885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813932e0 msgr2=0x559a813a14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.503+0000 7f650545a740 1 --2- 172.21.15.172:0/2120427885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813932e0 0x559a813a14e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f64e4000c00 tx=0x7f64e4005700 comp rx=0 tx=0).stop 2024-09-10T09:05:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.503+0000 7f650545a740 1 -- 172.21.15.172:0/2120427885 shutdown_connections 2024-09-10T09:05:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.503+0000 7f650545a740 1 --2- 172.21.15.172:0/2120427885 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559a813ac980 0x559a813aed70 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.503+0000 7f650545a740 1 --2- 172.21.15.172:0/2120427885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 0x559a813a8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.503+0000 7f650545a740 1 --2- 172.21.15.172:0/2120427885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813932e0 0x559a813a14e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.503+0000 7f650545a740 1 --2- 172.21.15.172:0/2120427885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813929a0 0x559a813a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.503+0000 7f650545a740 1 -- 172.21.15.172:0/2120427885 >> 172.21.15.172:0/2120427885 conn(0x559a8138e010 msgr2=0x559a8138f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.504+0000 7f64db7fe640 0 client.28814 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.504+0000 7f650545a740 1 -- 172.21.15.172:0/2120427885 shutdown_connections 2024-09-10T09:05:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.504+0000 7f650545a740 1 -- 172.21.15.172:0/2120427885 wait complete. 2024-09-10T09:05:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.504+0000 7f650545a740 20 client.28814 trim_cache size 0 max 16384 2024-09-10T09:05:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.524+0000 7f650545a740 1 Processor -- start 2024-09-10T09:05:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f650545a740 1 -- start start 2024-09-10T09:05:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f650545a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813929a0 0x559a8138f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f650545a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813932e0 0x559a813a84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f650545a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 0x559a8138f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f650545a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1a84c0 con 0x559a81398370 2024-09-10T09:05:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f650545a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1aa000 con 0x559a813932e0 2024-09-10T09:05:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f650545a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1a8b90 con 0x559a813929a0 2024-09-10T09:05:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f64fbfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813929a0 0x559a8138f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f6500b81640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 0x559a8138f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f64fbfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813929a0 0x559a8138f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:48610/0 (socket says 172.21.15.172:48610) 2024-09-10T09:05:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f64fbfff640 1 -- 172.21.15.172:0/3818337234 learned_addr learned my addr 172.21.15.172:0/3818337234 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f64f3fff640 1 --2- 172.21.15.172:0/3818337234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813932e0 0x559a813a84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f64fbfff640 1 -- 172.21.15.172:0/3818337234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813932e0 msgr2=0x559a813a84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f64fbfff640 1 --2- 172.21.15.172:0/3818337234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813932e0 0x559a813a84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f64fbfff640 1 -- 172.21.15.172:0/3818337234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 msgr2=0x559a8138f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f64fbfff640 1 --2- 172.21.15.172:0/3818337234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 0x559a8138f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f64fbfff640 1 -- 172.21.15.172:0/3818337234 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a8139f630 con 0x559a813929a0 2024-09-10T09:05:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.525+0000 7f64fbfff640 1 --2- 172.21.15.172:0/3818337234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813929a0 0x559a8138f400 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f64fc101220 tx=0x7f64ec03e070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1e797d7cfa8e4a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.526+0000 7f64f1ffb640 1 -- 172.21.15.172:0/3818337234 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec035050 con 0x559a813929a0 2024-09-10T09:05:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.526+0000 7f64f1ffb640 1 -- 172.21.15.172:0/3818337234 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64ec003aa0 con 0x559a813929a0 2024-09-10T09:05:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.526+0000 7f64f1ffb640 1 -- 172.21.15.172:0/3818337234 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0538b0 con 0x559a813929a0 2024-09-10T09:05:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.526+0000 7f650545a740 1 -- 172.21.15.172:0/3818337234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813929a0 msgr2=0x559a8138f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.526+0000 7f650545a740 1 --2- 172.21.15.172:0/3818337234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813929a0 0x559a8138f400 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f64fc101220 tx=0x7f64ec03e070 comp rx=0 tx=0).stop 2024-09-10T09:05:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f650545a740 1 -- 172.21.15.172:0/3818337234 shutdown_connections 2024-09-10T09:05:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f650545a740 1 --2- 172.21.15.172:0/3818337234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a81398370 0x559a8138f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f650545a740 1 --2- 172.21.15.172:0/3818337234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813932e0 0x559a813a84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f650545a740 1 --2- 172.21.15.172:0/3818337234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a813929a0 0x559a8138f400 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f650545a740 1 -- 172.21.15.172:0/3818337234 >> 172.21.15.172:0/3818337234 conn(0x559a8138e010 msgr2=0x559a8138d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f650545a740 1 -- 172.21.15.172:0/3818337234 shutdown_connections 2024-09-10T09:05:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f650545a740 1 -- 172.21.15.172:0/3818337234 wait complete. 2024-09-10T09:05:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f650545a740 1 -- start start 2024-09-10T09:05:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f650545a740 1 Processor -- start 2024-09-10T09:05:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f650545a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813929a0 0x559a813a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f650545a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a813932e0 0x559a813a6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f650545a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a81398370 0x559a813a7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f650545a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1a9930 con 0x559a813932e0 2024-09-10T09:05:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f650545a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1a9260 con 0x559a813929a0 2024-09-10T09:05:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f650545a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1a64a0 con 0x559a81398370 2024-09-10T09:05:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f650545a740 10 client.0 _ll_register_callbacks cb 0x7f64f40050c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f64f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a813932e0 0x559a813a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f64f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a813932e0 0x559a813a6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:48100/0 (socket says 172.21.15.172:48100) 2024-09-10T09:05:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f64f3fff640 1 -- 172.21.15.172:0/3986109395 learned_addr learned my addr 172.21.15.172:0/3986109395 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f64fbfff640 1 --2- 172.21.15.172:0/3986109395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813929a0 0x559a813a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.527+0000 7f6500b81640 1 --2- 172.21.15.172:0/3986109395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a81398370 0x559a813a7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.528+0000 7f64fbfff640 1 -- 172.21.15.172:0/3986109395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a81398370 msgr2=0x559a813a7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.528+0000 7f64fbfff640 1 --2- 172.21.15.172:0/3986109395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a81398370 0x559a813a7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.528+0000 7f64fbfff640 1 -- 172.21.15.172:0/3986109395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a813932e0 msgr2=0x559a813a6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.528+0000 7f64fbfff640 1 --2- 172.21.15.172:0/3986109395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a813932e0 0x559a813a6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.528+0000 7f64fbfff640 1 -- 172.21.15.172:0/3986109395 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a813a7a70 con 0x559a813929a0 2024-09-10T09:05:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.528+0000 7f6500b81640 1 --2- 172.21.15.172:0/3986109395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a81398370 0x559a813a7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.528+0000 7f64f3fff640 1 --2- 172.21.15.172:0/3986109395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a813932e0 0x559a813a6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.528+0000 7f64fbfff640 1 --2- 172.21.15.172:0/3986109395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813929a0 0x559a813a5f10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f64ec02a930 tx=0x7f64ec02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20e598d9ff96620f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.528+0000 7f64db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:05:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.528+0000 7f64db7fe640 1 -- 172.21.15.172:0/3986109395 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec035040 con 0x559a813929a0 2024-09-10T09:05:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.528+0000 7f64db7fe640 1 -- 172.21.15.172:0/3986109395 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f64ec003c50 con 0x559a813929a0 2024-09-10T09:05:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.528+0000 7f650545a740 1 -- 172.21.15.172:0/3986109395 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64ec003aa0 con 0x559a813929a0 2024-09-10T09:05:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.528+0000 7f64db7fe640 1 -- 172.21.15.172:0/3986109395 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f64ec06b050 con 0x559a813929a0 2024-09-10T09:05:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.528+0000 7f650545a740 10 client.28820 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.528+0000 7f650545a740 1 -- 172.21.15.172:0/3986109395 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559a814178f0 con 0x559a813929a0 2024-09-10T09:05:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.529+0000 7f64db7fe640 1 -- 172.21.15.172:0/3986109395 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64ec003aa0 con 0x559a813929a0 2024-09-10T09:05:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.529+0000 7f64db7fe640 1 -- 172.21.15.172:0/3986109395 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f64ec053b10 con 0x559a813929a0 2024-09-10T09:05:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.529+0000 7f650545a740 10 client.28820 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.529+0000 7f650545a740 10 client.28820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.529+0000 7f650545a740 20 client.28820 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.529+0000 7f650545a740 1 --2- 172.21.15.172:0/3986109395 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559a814182f0 0x559a8141a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.529+0000 7f650545a740 4 client.28820 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:05:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.529+0000 7f650545a740 1 -- 172.21.15.172:0/3986109395 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a812fdd70 con 0x559a814182f0 2024-09-10T09:05:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.529+0000 7f64f3fff640 1 --2- 172.21.15.172:0/3986109395 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559a814182f0 0x559a8141a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.529+0000 7f64db7fe640 10 client.28820 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.530+0000 7f64f3fff640 1 --2- 172.21.15.172:0/3986109395 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559a814182f0 0x559a8141a710 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2715368327077641 server_cookie=7a9ca11017b3c649 in_seq=0 out_seq=0 2024-09-10T09:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.536+0000 7f64db7fe640 1 -- 172.21.15.172:0/3986109395 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a812fdd70 con 0x559a814182f0 2024-09-10T09:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.536+0000 7f64db7fe640 10 client.28820 handle_command_reply: tid=0 2024-09-10T09:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.536+0000 7f650545a740 1 client.28820 shutdown 2024-09-10T09:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.536+0000 7f650545a740 1 -- 172.21.15.172:0/3986109395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813929a0 msgr2=0x559a813a5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.536+0000 7f650545a740 1 --2- 172.21.15.172:0/3986109395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813929a0 0x559a813a5f10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f64ec02a930 tx=0x7f64ec02a410 comp rx=0 tx=0).stop 2024-09-10T09:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.536+0000 7f650545a740 1 -- 172.21.15.172:0/3986109395 shutdown_connections 2024-09-10T09:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.536+0000 7f650545a740 1 --2- 172.21.15.172:0/3986109395 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x559a814182f0 0x559a8141a710 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.536+0000 7f650545a740 1 --2- 172.21.15.172:0/3986109395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a81398370 0x559a813a7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.536+0000 7f650545a740 1 --2- 172.21.15.172:0/3986109395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559a813932e0 0x559a813a6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.536+0000 7f650545a740 1 --2- 172.21.15.172:0/3986109395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a813929a0 0x559a813a5f10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.536+0000 7f650545a740 1 -- 172.21.15.172:0/3986109395 >> 172.21.15.172:0/3986109395 conn(0x559a8138e010 msgr2=0x559a8138e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.536+0000 7f64db7fe640 0 client.28820 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.536+0000 7f650545a740 1 -- 172.21.15.172:0/3986109395 shutdown_connections 2024-09-10T09:05:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.536+0000 7f650545a740 1 -- 172.21.15.172:0/3986109395 wait complete. 2024-09-10T09:05:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.536+0000 7f650545a740 20 client.28820 trim_cache size 0 max 16384 2024-09-10T09:05:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:05:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:05:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:05:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:05:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.537+0000 7f65025e4640 1 -- 172.21.15.172:0/1155266568 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f64e80a2c30 msgr2=0x7f64e80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.537+0000 7f65025e4640 1 --2- 172.21.15.172:0/1155266568 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f64e80a2c30 0x7f64e80a50f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f64ec004770 tx=0x7f64ec000bc0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.537+0000 7f65025e4640 1 -- 172.21.15.172:0/1155266568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64fc101d80 msgr2=0x7f64fc19f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.537+0000 7f65025e4640 1 --2- 172.21.15.172:0/1155266568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64fc101d80 0x7f64fc19f2a0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f64f400ec60 tx=0x7f64f400c7a0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.537+0000 7f65025e4640 1 -- 172.21.15.172:0/1155266568 shutdown_connections 2024-09-10T09:05:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.537+0000 7f65025e4640 1 --2- 172.21.15.172:0/1155266568 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f64e80a2c30 0x7f64e80a50f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.537+0000 7f65025e4640 1 --2- 172.21.15.172:0/1155266568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64fc101d80 0x7f64fc19f2a0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.537+0000 7f65025e4640 1 --2- 172.21.15.172:0/1155266568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64fc1013c0 0x7f64fc198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.537+0000 7f65025e4640 1 --2- 172.21.15.172:0/1155266568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64fc1001d0 0x7f64fc197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.537+0000 7f65025e4640 1 -- 172.21.15.172:0/1155266568 >> 172.21.15.172:0/1155266568 conn(0x7f64fc0fb930 msgr2=0x7f64fc106900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.537+0000 7f65025e4640 1 -- 172.21.15.172:0/1155266568 shutdown_connections 2024-09-10T09:05:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:34.537+0000 7f65025e4640 1 -- 172.21.15.172:0/1155266568 wait complete. 2024-09-10T09:05:34.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:34 smithi174 ceph-mon[34068]: pgmap v930: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 966 KiB/s wr, 252 op/s 2024-09-10T09:05:35.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:34 smithi191 ceph-mon[35314]: pgmap v930: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 966 KiB/s wr, 252 op/s 2024-09-10T09:05:35.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:34 smithi172 ceph-mon[30854]: pgmap v930: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 966 KiB/s wr, 252 op/s 2024-09-10T09:05:36.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:36 smithi174 ceph-mon[34068]: pgmap v931: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 823 KiB/s wr, 218 op/s 2024-09-10T09:05:37.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:36 smithi191 ceph-mon[35314]: pgmap v931: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 823 KiB/s wr, 218 op/s 2024-09-10T09:05:37.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:36 smithi172 ceph-mon[30854]: pgmap v931: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 823 KiB/s wr, 218 op/s 2024-09-10T09:05:37.326 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:05:37.897 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:05:38.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.703+0000 7fafb3307640 1 -- 172.21.15.172:0/2445533108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafac1027a0 msgr2=0x7fafac1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:38.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.703+0000 7fafb3307640 1 --2- 172.21.15.172:0/2445533108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafac1027a0 0x7fafac1067f0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009900 tx=0x7faf9c02a410 comp rx=0 tx=0).stop 2024-09-10T09:05:38.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.703+0000 7fafb3307640 1 -- 172.21.15.172:0/2445533108 shutdown_connections 2024-09-10T09:05:38.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.703+0000 7fafb3307640 1 --2- 172.21.15.172:0/2445533108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafac106f20 0x7fafac10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.703+0000 7fafb3307640 1 --2- 172.21.15.172:0/2445533108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafac1027a0 0x7fafac1067f0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.703+0000 7fafb3307640 1 --2- 172.21.15.172:0/2445533108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafac101df0 0x7fafac1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.703+0000 7fafb3307640 1 -- 172.21.15.172:0/2445533108 >> 172.21.15.172:0/2445533108 conn(0x7fafac0fd8f0 msgr2=0x7fafac0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:38.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.703+0000 7fafb3307640 1 -- 172.21.15.172:0/2445533108 shutdown_connections 2024-09-10T09:05:38.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.703+0000 7fafb3307640 1 -- 172.21.15.172:0/2445533108 wait complete. 2024-09-10T09:05:38.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.704+0000 7fafb3307640 1 Processor -- start 2024-09-10T09:05:38.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.704+0000 7fafb3307640 1 -- start start 2024-09-10T09:05:38.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.704+0000 7fafb3307640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafac101df0 0x7fafac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.704+0000 7fafb3307640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafac1027a0 0x7fafac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.704+0000 7fafb3307640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafac106f20 0x7fafac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.704+0000 7fafb3307640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fafac111e30 con 0x7fafac1027a0 2024-09-10T09:05:38.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.704+0000 7fafb3307640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fafac111cb0 con 0x7fafac106f20 2024-09-10T09:05:38.721 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.704+0000 7fafb3307640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fafac111fb0 con 0x7fafac101df0 2024-09-10T09:05:38.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.705+0000 7fafb08a2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafac1027a0 0x7fafac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:38.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.705+0000 7fafb10a3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafac101df0 0x7fafac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:38.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.705+0000 7fafb08a2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafac1027a0 0x7fafac074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34828/0 (socket says 172.21.15.172:34828) 2024-09-10T09:05:38.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.705+0000 7fafb08a2640 1 -- 172.21.15.172:0/3741227953 learned_addr learned my addr 172.21.15.172:0/3741227953 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:38.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.705+0000 7fafb18a4640 1 --2- 172.21.15.172:0/3741227953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafac106f20 0x7fafac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:38.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.705+0000 7fafb18a4640 1 -- 172.21.15.172:0/3741227953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafac101df0 msgr2=0x7fafac0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:38.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.705+0000 7fafb18a4640 1 --2- 172.21.15.172:0/3741227953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafac101df0 0x7fafac0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.705+0000 7fafb18a4640 1 -- 172.21.15.172:0/3741227953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafac1027a0 msgr2=0x7fafac074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:38.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.705+0000 7fafb18a4640 1 --2- 172.21.15.172:0/3741227953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafac1027a0 0x7fafac074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.705+0000 7fafb18a4640 1 -- 172.21.15.172:0/3741227953 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafac1a6960 con 0x7fafac106f20 2024-09-10T09:05:38.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.705+0000 7fafb10a3640 1 --2- 172.21.15.172:0/3741227953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafac101df0 0x7fafac0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:38.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.705+0000 7fafb18a4640 1 --2- 172.21.15.172:0/3741227953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafac106f20 0x7fafac074f70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009fd0 tx=0x7fafa400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:38.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.705+0000 7fafa27fc640 1 -- 172.21.15.172:0/3741227953 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa400ce20 con 0x7fafac106f20 2024-09-10T09:05:38.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.705+0000 7fafb3307640 1 -- 172.21.15.172:0/3741227953 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafac1a6c50 con 0x7fafac106f20 2024-09-10T09:05:38.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.705+0000 7fafa27fc640 1 -- 172.21.15.172:0/3741227953 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafa4005020 con 0x7fafac106f20 2024-09-10T09:05:38.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.705+0000 7fafa27fc640 1 -- 172.21.15.172:0/3741227953 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4010690 con 0x7fafac106f20 2024-09-10T09:05:38.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.705+0000 7fafb3307640 1 -- 172.21.15.172:0/3741227953 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafac1a7110 con 0x7fafac106f20 2024-09-10T09:05:38.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.708+0000 7fafa27fc640 1 -- 172.21.15.172:0/3741227953 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fafa40051d0 con 0x7fafac106f20 2024-09-10T09:05:38.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.708+0000 7fafb56e0740 1 Processor -- start 2024-09-10T09:05:38.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.708+0000 7fafb56e0740 1 -- start start 2024-09-10T09:05:38.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafa27fc640 1 --2- 172.21.15.172:0/3741227953 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7faf980a2b80 0x7faf980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafb56e0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed799114a0 0x55ed79911880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafb56e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed79911dc0 0x55ed7991c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafa27fc640 1 -- 172.21.15.172:0/3741227953 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fafa40cf9f0 con 0x7fafac106f20 2024-09-10T09:05:38.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafb56e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed7991c680 0x55ed7991ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafb56e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ed798772e0 con 0x55ed79911dc0 2024-09-10T09:05:38.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafb56e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ed7981ece0 con 0x55ed7991c680 2024-09-10T09:05:38.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafb56e0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ed798732a0 con 0x55ed799114a0 2024-09-10T09:05:38.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafb08a2640 1 --2- 172.21.15.172:0/3741227953 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7faf980a2b80 0x7faf980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:38.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafb18a4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed7991c680 0x55ed7991ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:38.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafb18a4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed7991c680 0x55ed7991ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:38286/0 (socket says 172.21.15.172:38286) 2024-09-10T09:05:38.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafb18a4640 1 -- 172.21.15.172:0/1018072801 learned_addr learned my addr 172.21.15.172:0/1018072801 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:38.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafb08a2640 1 --2- 172.21.15.172:0/1018072801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed79911dc0 0x55ed7991c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:38.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafb18a4640 1 -- 172.21.15.172:0/1018072801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed799114a0 msgr2=0x55ed79911880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:38.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafb18a4640 1 --2- 172.21.15.172:0/1018072801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed799114a0 0x55ed79911880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafb18a4640 1 -- 172.21.15.172:0/1018072801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed79911dc0 msgr2=0x55ed7991c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:38.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafb18a4640 1 --2- 172.21.15.172:0/1018072801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed79911dc0 0x55ed7991c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.709+0000 7fafb18a4640 1 -- 172.21.15.172:0/1018072801 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed79803440 con 0x55ed7991c680 2024-09-10T09:05:38.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.710+0000 7fafb08a2640 1 --2- 172.21.15.172:0/3741227953 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7faf980a2b80 0x7faf980a5040 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7faf9c031440 tx=0x7faf9c005b70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:38.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.710+0000 7fafb18a4640 1 --2- 172.21.15.172:0/1018072801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed7991c680 0x55ed7991ea70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fafa4098350 tx=0x7fafa409bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c7793e46b3f0c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:38.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.710+0000 7faf977fe640 1 -- 172.21.15.172:0/1018072801 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa40b4020 con 0x55ed7991c680 2024-09-10T09:05:38.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.710+0000 7faf977fe640 1 -- 172.21.15.172:0/1018072801 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafa409a1f0 con 0x55ed7991c680 2024-09-10T09:05:38.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.710+0000 7faf977fe640 1 -- 172.21.15.172:0/1018072801 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa40a4930 con 0x55ed7991c680 2024-09-10T09:05:38.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.710+0000 7fafb56e0740 1 -- 172.21.15.172:0/1018072801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed7991c680 msgr2=0x55ed7991ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:38.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.710+0000 7fafb56e0740 1 --2- 172.21.15.172:0/1018072801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed7991c680 0x55ed7991ea70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fafa4098350 tx=0x7fafa409bef0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.711+0000 7fafb56e0740 1 -- 172.21.15.172:0/1018072801 shutdown_connections 2024-09-10T09:05:38.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.711+0000 7fafb56e0740 1 --2- 172.21.15.172:0/1018072801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed7991c680 0x55ed7991ea70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.711+0000 7fafb56e0740 1 --2- 172.21.15.172:0/1018072801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed79911dc0 0x55ed7991c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.711+0000 7fafb56e0740 1 --2- 172.21.15.172:0/1018072801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed799114a0 0x55ed79911880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.711+0000 7fafb56e0740 1 -- 172.21.15.172:0/1018072801 >> 172.21.15.172:0/1018072801 conn(0x55ed7990da30 msgr2=0x55ed7990de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:38.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.711+0000 7fafb56e0740 1 -- 172.21.15.172:0/1018072801 shutdown_connections 2024-09-10T09:05:38.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.711+0000 7fafb56e0740 1 -- 172.21.15.172:0/1018072801 wait complete. 2024-09-10T09:05:38.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.711+0000 7fafb56e0740 1 -- start start 2024-09-10T09:05:38.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.711+0000 7fafb56e0740 1 Processor -- start 2024-09-10T09:05:38.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.711+0000 7fafb56e0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed799114a0 0x55ed7991c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.711+0000 7fafb56e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed79911dc0 0x55ed79919310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.711+0000 7fafb56e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed7991c680 0x55ed799198e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.711+0000 7fafb56e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ed7991f250 con 0x55ed79911dc0 2024-09-10T09:05:38.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.711+0000 7fafb56e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ed79832050 con 0x55ed7991c680 2024-09-10T09:05:38.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.711+0000 7fafb56e0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ed7991f3d0 con 0x55ed799114a0 2024-09-10T09:05:38.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.712+0000 7fafb56e0740 10 client.0 _ll_register_callbacks cb 0x55ed79872160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:38.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.712+0000 7fafb10a3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed799114a0 0x55ed7991c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:38.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.712+0000 7fafb18a4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed7991c680 0x55ed799198e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:38.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.712+0000 7fafb10a3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed799114a0 0x55ed7991c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33702/0 (socket says 172.21.15.172:33702) 2024-09-10T09:05:38.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.712+0000 7fafb10a3640 1 -- 172.21.15.172:0/1182810328 learned_addr learned my addr 172.21.15.172:0/1182810328 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:38.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.712+0000 7fafb18a4640 1 --2- 172.21.15.172:0/1182810328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed7991c680 0x55ed799198e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:38296/0 (socket says 172.21.15.172:38296) 2024-09-10T09:05:38.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.712+0000 7fafb08a2640 1 --2- 172.21.15.172:0/1182810328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed79911dc0 0x55ed79919310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:38.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.712+0000 7fafb18a4640 1 -- 172.21.15.172:0/1182810328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed799114a0 msgr2=0x55ed7991c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:38.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.712+0000 7fafb18a4640 1 --2- 172.21.15.172:0/1182810328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed799114a0 0x55ed7991c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.712+0000 7fafb18a4640 1 -- 172.21.15.172:0/1182810328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed79911dc0 msgr2=0x55ed79919310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:38.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.712+0000 7fafb18a4640 1 --2- 172.21.15.172:0/1182810328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed79911dc0 0x55ed79919310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.712+0000 7fafb18a4640 1 -- 172.21.15.172:0/1182810328 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed795e83c0 con 0x55ed7991c680 2024-09-10T09:05:38.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.712+0000 7fafb18a4640 1 --2- 172.21.15.172:0/1182810328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed7991c680 0x55ed799198e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55ed7990edb0 tx=0x7fafa409a3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af63b804ba25f24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:38.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.712+0000 7faf94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:05:38.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.712+0000 7faf94ff9640 1 -- 172.21.15.172:0/1182810328 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa40b4070 con 0x55ed7991c680 2024-09-10T09:05:38.727 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.712+0000 7faf94ff9640 1 -- 172.21.15.172:0/1182810328 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fafa40bad30 con 0x55ed7991c680 2024-09-10T09:05:38.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.712+0000 7fafb56e0740 1 -- 172.21.15.172:0/1182810328 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafa409a1f0 con 0x55ed7991c680 2024-09-10T09:05:38.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.713+0000 7faf94ff9640 1 -- 172.21.15.172:0/1182810328 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fafa409b340 con 0x55ed7991c680 2024-09-10T09:05:38.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.713+0000 7fafb56e0740 10 client.28832 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:38.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.713+0000 7fafb56e0740 1 -- 172.21.15.172:0/1182810328 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ed7992ac90 con 0x55ed7991c680 2024-09-10T09:05:38.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.713+0000 7faf94ff9640 1 -- 172.21.15.172:0/1182810328 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafa409a1f0 con 0x55ed7991c680 2024-09-10T09:05:38.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.713+0000 7faf94ff9640 1 -- 172.21.15.172:0/1182810328 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fafa40a1430 con 0x55ed7991c680 2024-09-10T09:05:38.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.713+0000 7fafb56e0740 10 client.28832 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:38.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.713+0000 7fafb56e0740 10 client.28832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:38.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.713+0000 7fafb56e0740 20 client.28832 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:38.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.713+0000 7fafb56e0740 1 --2- 172.21.15.172:0/1182810328 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ed7992bfe0 0x55ed7992e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.713+0000 7fafb56e0740 4 client.28832 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:05:38.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.713+0000 7fafb56e0740 1 -- 172.21.15.172:0/1182810328 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed79884850 con 0x55ed7992bfe0 2024-09-10T09:05:38.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.713+0000 7fafb10a3640 1 --2- 172.21.15.172:0/1182810328 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ed7992bfe0 0x55ed7992e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:38.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.714+0000 7fafb10a3640 1 --2- 172.21.15.172:0/1182810328 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ed7992bfe0 0x55ed7992e3d0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8441535028865854 server_cookie=4c37b7680af690d7 in_seq=0 out_seq=0 2024-09-10T09:05:38.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.714+0000 7faf94ff9640 10 client.28832 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:38.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.715+0000 7faf94ff9640 1 -- 172.21.15.172:0/1182810328 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed79884850 con 0x55ed7992bfe0 2024-09-10T09:05:38.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.715+0000 7faf94ff9640 10 client.28832 handle_command_reply: tid=0 2024-09-10T09:05:38.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.715+0000 7fafb56e0740 1 client.28832 shutdown 2024-09-10T09:05:38.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.715+0000 7fafb56e0740 1 -- 172.21.15.172:0/1182810328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed7991c680 msgr2=0x55ed799198e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:38.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.715+0000 7fafb56e0740 1 --2- 172.21.15.172:0/1182810328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed7991c680 0x55ed799198e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55ed7990edb0 tx=0x7fafa409a3a0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.715+0000 7fafb56e0740 1 -- 172.21.15.172:0/1182810328 shutdown_connections 2024-09-10T09:05:38.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.715+0000 7fafb56e0740 1 --2- 172.21.15.172:0/1182810328 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ed7992bfe0 0x55ed7992e3d0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.715+0000 7fafb56e0740 1 --2- 172.21.15.172:0/1182810328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed7991c680 0x55ed799198e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.715+0000 7fafb56e0740 1 --2- 172.21.15.172:0/1182810328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed79911dc0 0x55ed79919310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.715+0000 7fafb56e0740 1 --2- 172.21.15.172:0/1182810328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed799114a0 0x55ed7991c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.715+0000 7fafb56e0740 1 -- 172.21.15.172:0/1182810328 >> 172.21.15.172:0/1182810328 conn(0x55ed7990da30 msgr2=0x55ed79913760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:38.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.715+0000 7faf94ff9640 0 client.28832 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:38.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.715+0000 7fafb56e0740 1 -- 172.21.15.172:0/1182810328 shutdown_connections 2024-09-10T09:05:38.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.715+0000 7fafb56e0740 1 -- 172.21.15.172:0/1182810328 wait complete. 2024-09-10T09:05:38.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.715+0000 7fafb56e0740 20 client.28832 trim_cache size 0 max 16384 2024-09-10T09:05:38.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.736+0000 7fafb56e0740 1 Processor -- start 2024-09-10T09:05:38.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.736+0000 7fafb56e0740 1 -- start start 2024-09-10T09:05:38.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.736+0000 7fafb56e0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed799114a0 0x55ed7990de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.736+0000 7fafb56e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed79911dc0 0x55ed79916570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.736+0000 7fafb56e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed7991c680 0x55ed79916ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.736+0000 7fafb56e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fafac1b0d90 con 0x55ed7991c680 2024-09-10T09:05:38.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.736+0000 7fafb56e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fafac1b28d0 con 0x55ed79911dc0 2024-09-10T09:05:38.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.736+0000 7fafb56e0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fafac1b1460 con 0x55ed799114a0 2024-09-10T09:05:38.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.736+0000 7fafb08a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed79911dc0 0x55ed79916570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:38.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.736+0000 7fafb10a3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed799114a0 0x55ed7990de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:38.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.736+0000 7fafb08a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed79911dc0 0x55ed79916570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:38304/0 (socket says 172.21.15.172:38304) 2024-09-10T09:05:38.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.736+0000 7fafb08a2640 1 -- 172.21.15.172:0/3162904738 learned_addr learned my addr 172.21.15.172:0/3162904738 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:38.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.736+0000 7fafb18a4640 1 --2- 172.21.15.172:0/3162904738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed7991c680 0x55ed79916ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:38.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.736+0000 7fafb08a2640 1 -- 172.21.15.172:0/3162904738 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed799114a0 msgr2=0x55ed7990de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:38.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.736+0000 7fafb08a2640 1 --2- 172.21.15.172:0/3162904738 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed799114a0 0x55ed7990de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.737+0000 7fafb08a2640 1 -- 172.21.15.172:0/3162904738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed7991c680 msgr2=0x55ed79916ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:38.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.737+0000 7fafb08a2640 1 --2- 172.21.15.172:0/3162904738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed7991c680 0x55ed79916ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.737+0000 7fafb08a2640 1 -- 172.21.15.172:0/3162904738 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed799112d0 con 0x55ed79911dc0 2024-09-10T09:05:38.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.737+0000 7fafb18a4640 1 --2- 172.21.15.172:0/3162904738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed7991c680 0x55ed79916ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:05:38.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.737+0000 7fafb10a3640 1 --2- 172.21.15.172:0/3162904738 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed799114a0 0x55ed7990de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:38.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.737+0000 7fafb08a2640 1 --2- 172.21.15.172:0/3162904738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed79911dc0 0x55ed79916570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faf9c000c00 tx=0x7faf9c02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce9303f8f18bfc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:38.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.737+0000 7faf96ffd640 1 -- 172.21.15.172:0/3162904738 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c035040 con 0x55ed79911dc0 2024-09-10T09:05:38.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.737+0000 7faf96ffd640 1 -- 172.21.15.172:0/3162904738 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf9c003aa0 con 0x55ed79911dc0 2024-09-10T09:05:38.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.737+0000 7faf96ffd640 1 -- 172.21.15.172:0/3162904738 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c04c990 con 0x55ed79911dc0 2024-09-10T09:05:38.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.738+0000 7fafb56e0740 1 -- 172.21.15.172:0/3162904738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed79911dc0 msgr2=0x55ed79916570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:38.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.738+0000 7fafb56e0740 1 --2- 172.21.15.172:0/3162904738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed79911dc0 0x55ed79916570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faf9c000c00 tx=0x7faf9c02a410 comp rx=0 tx=0).stop 2024-09-10T09:05:38.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.738+0000 7fafb56e0740 1 -- 172.21.15.172:0/3162904738 shutdown_connections 2024-09-10T09:05:38.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.738+0000 7fafb56e0740 1 --2- 172.21.15.172:0/3162904738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed7991c680 0x55ed79916ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.738+0000 7fafb56e0740 1 --2- 172.21.15.172:0/3162904738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed79911dc0 0x55ed79916570 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.738+0000 7fafb56e0740 1 --2- 172.21.15.172:0/3162904738 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed799114a0 0x55ed7990de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.738+0000 7fafb56e0740 1 -- 172.21.15.172:0/3162904738 >> 172.21.15.172:0/3162904738 conn(0x55ed7990da30 msgr2=0x55ed799148b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:38.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.738+0000 7fafb56e0740 1 -- 172.21.15.172:0/3162904738 shutdown_connections 2024-09-10T09:05:38.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.738+0000 7fafb56e0740 1 -- 172.21.15.172:0/3162904738 wait complete. 2024-09-10T09:05:38.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.739+0000 7fafb56e0740 1 -- start start 2024-09-10T09:05:38.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.739+0000 7fafb56e0740 1 Processor -- start 2024-09-10T09:05:38.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.739+0000 7fafb56e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed799114a0 0x55ed79931450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.739+0000 7fafb56e0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed79911dc0 0x55ed7992e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.739+0000 7fafb56e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed7991c680 0x55ed7992ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.739+0000 7fafb56e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fafac1b2200 con 0x55ed7991c680 2024-09-10T09:05:38.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.739+0000 7fafb56e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fafac1b1b30 con 0x55ed799114a0 2024-09-10T09:05:38.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.739+0000 7fafb10a3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed799114a0 0x55ed79931450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:38.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.739+0000 7fafb56e0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fafac1aed70 con 0x55ed79911dc0 2024-09-10T09:05:38.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.739+0000 7fafb10a3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed799114a0 0x55ed79931450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:38314/0 (socket says 172.21.15.172:38314) 2024-09-10T09:05:38.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.739+0000 7fafb10a3640 1 -- 172.21.15.172:0/147991775 learned_addr learned my addr 172.21.15.172:0/147991775 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:38.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.740+0000 7fafb08a2640 1 --2- 172.21.15.172:0/147991775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed79911dc0 0x55ed7992e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:38.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.740+0000 7fafb56e0740 10 client.0 _ll_register_callbacks cb 0x7fafa8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:38.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.740+0000 7fafb18a4640 1 --2- 172.21.15.172:0/147991775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed7991c680 0x55ed7992ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:38.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.740+0000 7fafb08a2640 1 -- 172.21.15.172:0/147991775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed799114a0 msgr2=0x55ed79931450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:38.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.740+0000 7fafb08a2640 1 --2- 172.21.15.172:0/147991775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed799114a0 0x55ed79931450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.740+0000 7fafb08a2640 1 -- 172.21.15.172:0/147991775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed7991c680 msgr2=0x55ed7992ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:38.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.740+0000 7fafb08a2640 1 --2- 172.21.15.172:0/147991775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed7991c680 0x55ed7992ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.740+0000 7fafb08a2640 1 -- 172.21.15.172:0/147991775 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed7992f220 con 0x55ed79911dc0 2024-09-10T09:05:38.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.740+0000 7fafb08a2640 1 --2- 172.21.15.172:0/147991775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed79911dc0 0x55ed7992e7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faf9c006010 tx=0x7faf9c0045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d97e58d285fd15a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:38.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.740+0000 7faf94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:05:38.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.740+0000 7faf94ff9640 1 -- 172.21.15.172:0/147991775 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c05c020 con 0x55ed79911dc0 2024-09-10T09:05:38.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.740+0000 7faf94ff9640 1 -- 172.21.15.172:0/147991775 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7faf9c0037f0 con 0x55ed79911dc0 2024-09-10T09:05:38.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.740+0000 7fafb56e0740 1 -- 172.21.15.172:0/147991775 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf9c003aa0 con 0x55ed79911dc0 2024-09-10T09:05:38.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.741+0000 7faf94ff9640 1 -- 172.21.15.172:0/147991775 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7faf9c035070 con 0x55ed79911dc0 2024-09-10T09:05:38.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.741+0000 7fafb56e0740 10 client.28732 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:38.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.741+0000 7fafb56e0740 1 -- 172.21.15.172:0/147991775 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ed7999ee10 con 0x55ed79911dc0 2024-09-10T09:05:38.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.741+0000 7faf94ff9640 1 -- 172.21.15.172:0/147991775 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf9c003aa0 con 0x55ed79911dc0 2024-09-10T09:05:38.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.741+0000 7faf94ff9640 1 -- 172.21.15.172:0/147991775 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7faf9c054590 con 0x55ed79911dc0 2024-09-10T09:05:38.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.741+0000 7fafb56e0740 10 client.28732 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:38.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.741+0000 7fafb56e0740 10 client.28732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:38.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.741+0000 7fafb56e0740 20 client.28732 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:38.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.741+0000 7fafb56e0740 1 --2- 172.21.15.172:0/147991775 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ed7999f470 0x55ed799a1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:38.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.741+0000 7fafb56e0740 4 client.28732 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:05:38.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.741+0000 7fafb56e0740 1 -- 172.21.15.172:0/147991775 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ed7987ed60 con 0x55ed7999f470 2024-09-10T09:05:38.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.742+0000 7fafb10a3640 1 --2- 172.21.15.172:0/147991775 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ed7999f470 0x55ed799a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:38.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.742+0000 7fafb10a3640 1 --2- 172.21.15.172:0/147991775 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ed7999f470 0x55ed799a1860 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1d13446f43c351c server_cookie=a0d2d8f961fcd521 in_seq=0 out_seq=0 2024-09-10T09:05:38.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.742+0000 7faf94ff9640 10 client.28732 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:38.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.742+0000 7faf94ff9640 1 -- 172.21.15.172:0/147991775 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ed7987ed60 con 0x55ed7999f470 2024-09-10T09:05:38.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.742+0000 7faf94ff9640 10 client.28732 handle_command_reply: tid=0 2024-09-10T09:05:38.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.742+0000 7fafb56e0740 1 client.28732 shutdown 2024-09-10T09:05:38.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.742+0000 7fafb56e0740 1 -- 172.21.15.172:0/147991775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed79911dc0 msgr2=0x55ed7992e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:38.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.742+0000 7fafb56e0740 1 --2- 172.21.15.172:0/147991775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed79911dc0 0x55ed7992e7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faf9c006010 tx=0x7faf9c0045b0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.742+0000 7fafb56e0740 1 -- 172.21.15.172:0/147991775 shutdown_connections 2024-09-10T09:05:38.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.742+0000 7fafb56e0740 1 --2- 172.21.15.172:0/147991775 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55ed7999f470 0x55ed799a1860 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.743+0000 7fafb56e0740 1 --2- 172.21.15.172:0/147991775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed7991c680 0x55ed7992ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.743+0000 7fafb56e0740 1 --2- 172.21.15.172:0/147991775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ed79911dc0 0x55ed7992e7a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.743+0000 7fafb56e0740 1 --2- 172.21.15.172:0/147991775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed799114a0 0x55ed79931450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.743+0000 7fafb56e0740 1 -- 172.21.15.172:0/147991775 >> 172.21.15.172:0/147991775 conn(0x55ed7990da30 msgr2=0x55ed79914090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:38.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.743+0000 7faf94ff9640 0 client.28732 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:38.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.743+0000 7fafb56e0740 1 -- 172.21.15.172:0/147991775 shutdown_connections 2024-09-10T09:05:38.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.743+0000 7fafb56e0740 1 -- 172.21.15.172:0/147991775 wait complete. 2024-09-10T09:05:38.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.743+0000 7fafb56e0740 20 client.28732 trim_cache size 0 max 16384 2024-09-10T09:05:38.749 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:05:38.749 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T09:05:38.749 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T09:05:38.749 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:05:38.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.744+0000 7fafb3307640 1 -- 172.21.15.172:0/3741227953 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7faf980a2b80 msgr2=0x7faf980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:38.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.744+0000 7fafb3307640 1 --2- 172.21.15.172:0/3741227953 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7faf980a2b80 0x7faf980a5040 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7faf9c031440 tx=0x7faf9c005b70 comp rx=0 tx=0).stop 2024-09-10T09:05:38.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.744+0000 7fafb3307640 1 -- 172.21.15.172:0/3741227953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafac106f20 msgr2=0x7fafac074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:38.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.744+0000 7fafb3307640 1 --2- 172.21.15.172:0/3741227953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafac106f20 0x7fafac074f70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009fd0 tx=0x7fafa400ef10 comp rx=0 tx=0).stop 2024-09-10T09:05:38.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.744+0000 7fafb3307640 1 -- 172.21.15.172:0/3741227953 shutdown_connections 2024-09-10T09:05:38.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.744+0000 7fafb3307640 1 --2- 172.21.15.172:0/3741227953 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7faf980a2b80 0x7faf980a5040 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.744+0000 7fafb3307640 1 --2- 172.21.15.172:0/3741227953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafac106f20 0x7fafac074f70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.744+0000 7fafb3307640 1 --2- 172.21.15.172:0/3741227953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafac1027a0 0x7fafac074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.744+0000 7fafb3307640 1 --2- 172.21.15.172:0/3741227953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafac101df0 0x7fafac0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:38.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.744+0000 7fafb3307640 1 -- 172.21.15.172:0/3741227953 >> 172.21.15.172:0/3741227953 conn(0x7fafac0fd8f0 msgr2=0x7fafac0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:38.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.744+0000 7fafb3307640 1 -- 172.21.15.172:0/3741227953 shutdown_connections 2024-09-10T09:05:38.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:38.744+0000 7fafb3307640 1 -- 172.21.15.172:0/3741227953 wait complete. 2024-09-10T09:05:38.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:38 smithi174 ceph-mon[34068]: pgmap v932: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 956 KiB/s wr, 250 op/s 2024-09-10T09:05:39.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:38 smithi191 ceph-mon[35314]: pgmap v932: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 956 KiB/s wr, 250 op/s 2024-09-10T09:05:39.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:38 smithi172 ceph-mon[30854]: pgmap v932: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 956 KiB/s wr, 250 op/s 2024-09-10T09:05:39.699 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:05:39.700 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:05:40.024 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:05:40.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.609+0000 7fb99704b640 1 -- 172.21.15.172:0/1890481236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb990102780 msgr2=0x7fb9901067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:40.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.609+0000 7fb99704b640 1 --2- 172.21.15.172:0/1890481236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb990102780 0x7fb9901067d0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009900 tx=0x7fb97c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:05:40.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.609+0000 7fb99704b640 1 -- 172.21.15.172:0/1890481236 shutdown_connections 2024-09-10T09:05:40.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.609+0000 7fb99704b640 1 --2- 172.21.15.172:0/1890481236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb990106f00 0x7fb99010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:40.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.609+0000 7fb99704b640 1 --2- 172.21.15.172:0/1890481236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb990102780 0x7fb9901067d0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:40.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.609+0000 7fb99704b640 1 --2- 172.21.15.172:0/1890481236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb990101dd0 0x7fb9901021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:40.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.609+0000 7fb99704b640 1 -- 172.21.15.172:0/1890481236 >> 172.21.15.172:0/1890481236 conn(0x7fb9900fd890 msgr2=0x7fb9900ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:40.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.609+0000 7fb99704b640 1 -- 172.21.15.172:0/1890481236 shutdown_connections 2024-09-10T09:05:40.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.609+0000 7fb99704b640 1 -- 172.21.15.172:0/1890481236 wait complete. 2024-09-10T09:05:40.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.610+0000 7fb99704b640 1 Processor -- start 2024-09-10T09:05:40.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.610+0000 7fb99704b640 1 -- start start 2024-09-10T09:05:40.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.611+0000 7fb99704b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb990101dd0 0x7fb990108b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:40.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.611+0000 7fb99704b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb990102780 0x7fb990109040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:40.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.611+0000 7fb99704b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb990106f00 0x7fb990109580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:40.617 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.611+0000 7fb99704b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb990111e10 con 0x7fb990106f00 2024-09-10T09:05:40.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.611+0000 7fb99704b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb990111c90 con 0x7fb990102780 2024-09-10T09:05:40.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.611+0000 7fb99704b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb990111f90 con 0x7fb990101dd0 2024-09-10T09:05:40.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.611+0000 7fb9955e8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb990106f00 0x7fb990109580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:40.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.611+0000 7fb994de7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb990101dd0 0x7fb990108b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:40.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.611+0000 7fb994de7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb990101dd0 0x7fb990108b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33738/0 (socket says 172.21.15.172:33738) 2024-09-10T09:05:40.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.611+0000 7fb994de7640 1 -- 172.21.15.172:0/1337503291 learned_addr learned my addr 172.21.15.172:0/1337503291 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:40.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.611+0000 7fb98ffff640 1 --2- 172.21.15.172:0/1337503291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb990102780 0x7fb990109040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:40.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.611+0000 7fb994de7640 1 -- 172.21.15.172:0/1337503291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb990102780 msgr2=0x7fb990109040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:40.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.611+0000 7fb994de7640 1 --2- 172.21.15.172:0/1337503291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb990102780 0x7fb990109040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:40.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.611+0000 7fb994de7640 1 -- 172.21.15.172:0/1337503291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb990106f00 msgr2=0x7fb990109580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:40.618 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.611+0000 7fb994de7640 1 --2- 172.21.15.172:0/1337503291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb990106f00 0x7fb990109580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:40.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.611+0000 7fb994de7640 1 -- 172.21.15.172:0/1337503291 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9901aac40 con 0x7fb990101dd0 2024-09-10T09:05:40.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.611+0000 7fb994de7640 1 --2- 172.21.15.172:0/1337503291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb990101dd0 0x7fb990108b00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb98400d950 tx=0x7fb98400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:40.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.612+0000 7fb98dffb640 1 -- 172.21.15.172:0/1337503291 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984017070 con 0x7fb990101dd0 2024-09-10T09:05:40.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.612+0000 7fb99704b640 1 -- 172.21.15.172:0/1337503291 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9901aaf30 con 0x7fb990101dd0 2024-09-10T09:05:40.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.612+0000 7fb98dffb640 1 -- 172.21.15.172:0/1337503291 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb984009c30 con 0x7fb990101dd0 2024-09-10T09:05:40.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.612+0000 7fb98dffb640 1 -- 172.21.15.172:0/1337503291 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98400f620 con 0x7fb990101dd0 2024-09-10T09:05:40.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.612+0000 7fb99704b640 1 -- 172.21.15.172:0/1337503291 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9901ab3e0 con 0x7fb990101dd0 2024-09-10T09:05:40.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.613+0000 7fb99704b640 1 -- 172.21.15.172:0/1337503291 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9901021b0 con 0x7fb990101dd0 2024-09-10T09:05:40.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.614+0000 7fb98dffb640 1 -- 172.21.15.172:0/1337503291 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb9840109a0 con 0x7fb990101dd0 2024-09-10T09:05:40.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.615+0000 7fb98dffb640 1 --2- 172.21.15.172:0/1337503291 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb9800a2b80 0x7fb9800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:40.619 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.615+0000 7fb98dffb640 1 -- 172.21.15.172:0/1337503291 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fb9840cef30 con 0x7fb990101dd0 2024-09-10T09:05:40.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.615+0000 7fb98ffff640 1 --2- 172.21.15.172:0/1337503291 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb9800a2b80 0x7fb9800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:40.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.615+0000 7fb98ffff640 1 --2- 172.21.15.172:0/1337503291 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb9800a2b80 0x7fb9800a5040 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb97c031440 tx=0x7fb97c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:40.620 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.619+0000 7fb98dffb640 1 -- 172.21.15.172:0/1337503291 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb984009de0 con 0x7fb990101dd0 2024-09-10T09:05:40.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.887+0000 7fb99704b640 1 -- 172.21.15.172:0/1337503291 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9901ab6a0 con 0x7fb990101dd0 2024-09-10T09:05:40.889 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:40 smithi172 ceph-mon[30854]: pgmap v933: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 742 KiB/s wr, 196 op/s 2024-09-10T09:05:40.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.888+0000 7fb98dffb640 1 -- 172.21.15.172:0/1337503291 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+12001 (secure 0 0 0) 0x7fb98409b4e0 con 0x7fb990101dd0 2024-09-10T09:05:40.889 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T09:05:40.890 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":44,"btime":"2024-09-10T09:01:18:531063+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T09:01:17.697372+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24457,24416,24458,24406]},"id":1}]} 2024-09-10T09:05:40.890 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 44 2024-09-10T09:05:40.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.891+0000 7fb99704b640 1 -- 172.21.15.172:0/1337503291 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb9800a2b80 msgr2=0x7fb9800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:40.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.891+0000 7fb99704b640 1 --2- 172.21.15.172:0/1337503291 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb9800a2b80 0x7fb9800a5040 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb97c031440 tx=0x7fb97c005b50 comp rx=0 tx=0).stop 2024-09-10T09:05:40.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.891+0000 7fb99704b640 1 -- 172.21.15.172:0/1337503291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb990101dd0 msgr2=0x7fb990108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:40.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.891+0000 7fb99704b640 1 --2- 172.21.15.172:0/1337503291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb990101dd0 0x7fb990108b00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb98400d950 tx=0x7fb98400de10 comp rx=0 tx=0).stop 2024-09-10T09:05:40.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.892+0000 7fb99704b640 1 -- 172.21.15.172:0/1337503291 shutdown_connections 2024-09-10T09:05:40.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.892+0000 7fb99704b640 1 --2- 172.21.15.172:0/1337503291 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fb9800a2b80 0x7fb9800a5040 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:40.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.892+0000 7fb99704b640 1 --2- 172.21.15.172:0/1337503291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb990106f00 0x7fb990109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:40.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.892+0000 7fb99704b640 1 --2- 172.21.15.172:0/1337503291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb990102780 0x7fb990109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:40.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.892+0000 7fb99704b640 1 --2- 172.21.15.172:0/1337503291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb990101dd0 0x7fb990108b00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:40.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.892+0000 7fb99704b640 1 -- 172.21.15.172:0/1337503291 >> 172.21.15.172:0/1337503291 conn(0x7fb9900fd890 msgr2=0x7fb9900fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:40.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.892+0000 7fb99704b640 1 -- 172.21.15.172:0/1337503291 shutdown_connections 2024-09-10T09:05:40.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:40.892+0000 7fb99704b640 1 -- 172.21.15.172:0/1337503291 wait complete. 2024-09-10T09:05:40.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:40 smithi174 ceph-mon[34068]: pgmap v933: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 742 KiB/s wr, 196 op/s 2024-09-10T09:05:41.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:40 smithi191 ceph-mon[35314]: pgmap v933: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 742 KiB/s wr, 196 op/s 2024-09-10T09:05:41.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:41 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1337503291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:05:41.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:41 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1337503291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:05:42.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:41 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1337503291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:05:42.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:42 smithi174 ceph-mon[34068]: pgmap v934: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 822 KiB/s wr, 218 op/s 2024-09-10T09:05:43.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:42 smithi191 ceph-mon[35314]: pgmap v934: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 822 KiB/s wr, 218 op/s 2024-09-10T09:05:43.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:42 smithi172 ceph-mon[30854]: pgmap v934: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 822 KiB/s wr, 218 op/s 2024-09-10T09:05:43.221 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:05:43.524 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:05:44.037 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.034+0000 7fcd56e39640 1 -- 172.21.15.172:0/1618265322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd50106a90 msgr2=0x7fcd50106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:44.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.034+0000 7fcd56e39640 1 --2- 172.21.15.172:0/1618265322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd50106a90 0x7fcd50106e70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009860 tx=0x7fcd4402a340 comp rx=0 tx=0).stop 2024-09-10T09:05:44.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.034+0000 7fcd56e39640 1 -- 172.21.15.172:0/1618265322 shutdown_connections 2024-09-10T09:05:44.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.034+0000 7fcd56e39640 1 --2- 172.21.15.172:0/1618265322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd50101430 0x7fcd5010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.034+0000 7fcd56e39640 1 --2- 172.21.15.172:0/1618265322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd50100a90 0x7fcd50100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.034+0000 7fcd56e39640 1 --2- 172.21.15.172:0/1618265322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd50106a90 0x7fcd50106e70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.034+0000 7fcd56e39640 1 -- 172.21.15.172:0/1618265322 >> 172.21.15.172:0/1618265322 conn(0x7fcd500fc7d0 msgr2=0x7fcd500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:44.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.034+0000 7fcd56e39640 1 -- 172.21.15.172:0/1618265322 shutdown_connections 2024-09-10T09:05:44.038 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.034+0000 7fcd56e39640 1 -- 172.21.15.172:0/1618265322 wait complete. 2024-09-10T09:05:44.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.035+0000 7fcd56e39640 1 Processor -- start 2024-09-10T09:05:44.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.035+0000 7fcd56e39640 1 -- start start 2024-09-10T09:05:44.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.035+0000 7fcd56e39640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd50100a90 0x7fcd5010cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.035+0000 7fcd56e39640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd50101430 0x7fcd5010d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.035+0000 7fcd56e39640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd50106a90 0x7fcd5019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.035+0000 7fcd56e39640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5006b290 con 0x7fcd50106a90 2024-09-10T09:05:44.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.035+0000 7fcd56e39640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5006b110 con 0x7fcd50101430 2024-09-10T09:05:44.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.035+0000 7fcd56e39640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5006b410 con 0x7fcd50100a90 2024-09-10T09:05:44.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.035+0000 7fcd553d6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd50106a90 0x7fcd5019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.035+0000 7fcd54bd5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd50100a90 0x7fcd5010cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.035+0000 7fcd553d6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd50106a90 0x7fcd5019ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34892/0 (socket says 172.21.15.172:34892) 2024-09-10T09:05:44.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.035+0000 7fcd54bd5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd50100a90 0x7fcd5010cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33756/0 (socket says 172.21.15.172:33756) 2024-09-10T09:05:44.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.035+0000 7fcd553d6640 1 -- 172.21.15.172:0/486214549 learned_addr learned my addr 172.21.15.172:0/486214549 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:44.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.035+0000 7fcd4ffff640 1 --2- 172.21.15.172:0/486214549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd50101430 0x7fcd5010d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.036+0000 7fcd54bd5640 1 -- 172.21.15.172:0/486214549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd50101430 msgr2=0x7fcd5010d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:44.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.036+0000 7fcd54bd5640 1 --2- 172.21.15.172:0/486214549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd50101430 0x7fcd5010d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.036+0000 7fcd54bd5640 1 -- 172.21.15.172:0/486214549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd50106a90 msgr2=0x7fcd5019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:44.041 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.036+0000 7fcd54bd5640 1 --2- 172.21.15.172:0/486214549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd50106a90 0x7fcd5019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.036+0000 7fcd54bd5640 1 -- 172.21.15.172:0/486214549 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5019f4c0 con 0x7fcd50100a90 2024-09-10T09:05:44.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.036+0000 7fcd54bd5640 1 --2- 172.21.15.172:0/486214549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd50100a90 0x7fcd5010cf20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcd44005940 tx=0x7fcd44030bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:44.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.036+0000 7fcd4dffb640 1 -- 172.21.15.172:0/486214549 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44037050 con 0x7fcd50100a90 2024-09-10T09:05:44.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.036+0000 7fcd56e39640 1 -- 172.21.15.172:0/486214549 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5019f750 con 0x7fcd50100a90 2024-09-10T09:05:44.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.037+0000 7fcd56e39640 1 -- 172.21.15.172:0/486214549 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd501a7090 con 0x7fcd50100a90 2024-09-10T09:05:44.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.037+0000 7fcd4dffb640 1 -- 172.21.15.172:0/486214549 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd44038040 con 0x7fcd50100a90 2024-09-10T09:05:44.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.037+0000 7fcd4dffb640 1 -- 172.21.15.172:0/486214549 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44003830 con 0x7fcd50100a90 2024-09-10T09:05:44.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.039+0000 7fcd4dffb640 1 -- 172.21.15.172:0/486214549 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fcd44045030 con 0x7fcd50100a90 2024-09-10T09:05:44.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.040+0000 7fcd59212740 1 Processor -- start 2024-09-10T09:05:44.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.040+0000 7fcd59212740 1 -- start start 2024-09-10T09:05:44.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.040+0000 7fcd4dffb640 1 --2- 172.21.15.172:0/486214549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcd380a2c00 0x7fcd380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.040+0000 7fcd59212740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b67968ab40 0x55b679689090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.040+0000 7fcd59212740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6796895d0 0x55b679689a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.046 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.040+0000 7fcd59212740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b679694370 0x55b679696760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.040+0000 7fcd59212740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b6795ef2f0 con 0x55b679694370 2024-09-10T09:05:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.040+0000 7fcd59212740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b679596cf0 con 0x55b67968ab40 2024-09-10T09:05:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.040+0000 7fcd59212740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b6795eb2b0 con 0x55b6796895d0 2024-09-10T09:05:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.040+0000 7fcd553d6640 1 --2- 172.21.15.172:0/486214549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcd380a2c00 0x7fcd380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.040+0000 7fcd4ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6796895d0 0x55b679689a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.040+0000 7fcd553d6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b679694370 0x55b679696760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.040+0000 7fcd4dffb640 1 -- 172.21.15.172:0/486214549 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcd440eee40 con 0x7fcd50100a90 2024-09-10T09:05:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.040+0000 7fcd4ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6796895d0 0x55b679689a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33770/0 (socket says 172.21.15.172:33770) 2024-09-10T09:05:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.040+0000 7fcd4ffff640 1 -- 172.21.15.172:0/1963993630 learned_addr learned my addr 172.21.15.172:0/1963993630 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.041+0000 7fcd54bd5640 1 --2- 172.21.15.172:0/1963993630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b67968ab40 0x55b679689090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.041+0000 7fcd4ffff640 1 -- 172.21.15.172:0/1963993630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b67968ab40 msgr2=0x55b679689090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.041+0000 7fcd4ffff640 1 --2- 172.21.15.172:0/1963993630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b67968ab40 0x55b679689090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.047 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.041+0000 7fcd4ffff640 1 -- 172.21.15.172:0/1963993630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b679694370 msgr2=0x55b679696760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:44.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.041+0000 7fcd4ffff640 1 --2- 172.21.15.172:0/1963993630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b679694370 0x55b679696760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.041+0000 7fcd4ffff640 1 -- 172.21.15.172:0/1963993630 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b67957b3a0 con 0x55b6796895d0 2024-09-10T09:05:44.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.041+0000 7fcd553d6640 1 --2- 172.21.15.172:0/486214549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcd380a2c00 0x7fcd380a50c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fcd48005c40 tx=0x7fcd48009ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:44.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.041+0000 7fcd4ffff640 1 --2- 172.21.15.172:0/1963993630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6796895d0 0x55b679689a10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009c60 tx=0x7fcd40011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43f1650d2a7e290c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:44.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.041+0000 7fcd36ffd640 1 -- 172.21.15.172:0/1963993630 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40004220 con 0x55b6796895d0 2024-09-10T09:05:44.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.041+0000 7fcd36ffd640 1 -- 172.21.15.172:0/1963993630 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd40009e10 con 0x55b6796895d0 2024-09-10T09:05:44.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.041+0000 7fcd36ffd640 1 -- 172.21.15.172:0/1963993630 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40019740 con 0x55b6796895d0 2024-09-10T09:05:44.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.042+0000 7fcd59212740 1 -- 172.21.15.172:0/1963993630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6796895d0 msgr2=0x55b679689a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:44.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.042+0000 7fcd59212740 1 --2- 172.21.15.172:0/1963993630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6796895d0 0x55b679689a10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009c60 tx=0x7fcd40011810 comp rx=0 tx=0).stop 2024-09-10T09:05:44.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.042+0000 7fcd59212740 1 -- 172.21.15.172:0/1963993630 shutdown_connections 2024-09-10T09:05:44.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.042+0000 7fcd59212740 1 --2- 172.21.15.172:0/1963993630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b679694370 0x55b679696760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.042+0000 7fcd59212740 1 --2- 172.21.15.172:0/1963993630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6796895d0 0x55b679689a10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.042+0000 7fcd59212740 1 --2- 172.21.15.172:0/1963993630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b67968ab40 0x55b679689090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.042+0000 7fcd59212740 1 -- 172.21.15.172:0/1963993630 >> 172.21.15.172:0/1963993630 conn(0x55b679652310 msgr2=0x55b679685c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:44.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.042+0000 7fcd59212740 1 -- 172.21.15.172:0/1963993630 shutdown_connections 2024-09-10T09:05:44.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.042+0000 7fcd59212740 1 -- 172.21.15.172:0/1963993630 wait complete. 2024-09-10T09:05:44.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.042+0000 7fcd59212740 1 -- start start 2024-09-10T09:05:44.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.042+0000 7fcd59212740 1 Processor -- start 2024-09-10T09:05:44.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd59212740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6796895d0 0x55b679693ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd59212740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b67968ab40 0x55b679690f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd59212740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b679694370 0x55b679691490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd59212740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b679697070 con 0x55b679694370 2024-09-10T09:05:44.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd59212740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b6795aa060 con 0x55b67968ab40 2024-09-10T09:05:44.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd59212740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b6796971f0 con 0x55b6796895d0 2024-09-10T09:05:44.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd59212740 10 client.0 _ll_register_callbacks cb 0x55b6792fd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:44.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd54bd5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6796895d0 0x55b679693ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd4ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b67968ab40 0x55b679690f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd54bd5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6796895d0 0x55b679693ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33780/0 (socket says 172.21.15.172:33780) 2024-09-10T09:05:44.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd54bd5640 1 -- 172.21.15.172:0/4207920027 learned_addr learned my addr 172.21.15.172:0/4207920027 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:44.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd553d6640 1 --2- 172.21.15.172:0/4207920027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b679694370 0x55b679691490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd54bd5640 1 -- 172.21.15.172:0/4207920027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b67968ab40 msgr2=0x55b679690f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:44.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd54bd5640 1 --2- 172.21.15.172:0/4207920027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b67968ab40 0x55b679690f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd54bd5640 1 -- 172.21.15.172:0/4207920027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b679694370 msgr2=0x55b679691490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:44.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd54bd5640 1 --2- 172.21.15.172:0/4207920027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b679694370 0x55b679691490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd54bd5640 1 -- 172.21.15.172:0/4207920027 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6793603c0 con 0x55b6796895d0 2024-09-10T09:05:44.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd553d6640 1 --2- 172.21.15.172:0/4207920027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b679694370 0x55b679691490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:05:44.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd54bd5640 1 --2- 172.21.15.172:0/4207920027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6796895d0 0x55b679693ca0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcd44005fd0 tx=0x7fcd440303c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b40d385c4fac661e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:44.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:05:44.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.043+0000 7fcd2ffff640 1 -- 172.21.15.172:0/4207920027 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44049560 con 0x55b6796895d0 2024-09-10T09:05:44.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.044+0000 7fcd2ffff640 1 -- 172.21.15.172:0/4207920027 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcd440bf400 con 0x55b6796895d0 2024-09-10T09:05:44.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.044+0000 7fcd59212740 1 -- 172.21.15.172:0/4207920027 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd40009e10 con 0x55b6796895d0 2024-09-10T09:05:44.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.044+0000 7fcd2ffff640 1 -- 172.21.15.172:0/4207920027 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcd44030e70 con 0x55b6796895d0 2024-09-10T09:05:44.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.044+0000 7fcd59212740 10 client.28750 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:44.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.044+0000 7fcd59212740 1 -- 172.21.15.172:0/4207920027 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b679697070 con 0x55b6796895d0 2024-09-10T09:05:44.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.044+0000 7fcd2ffff640 1 -- 172.21.15.172:0/4207920027 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd40009e10 con 0x55b6796895d0 2024-09-10T09:05:44.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.044+0000 7fcd2ffff640 1 -- 172.21.15.172:0/4207920027 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fcd44003af0 con 0x55b6796895d0 2024-09-10T09:05:44.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.044+0000 7fcd59212740 10 client.28750 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:44.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.044+0000 7fcd59212740 10 client.28750 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T09:05:44.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.044+0000 7fcd59212740 20 client.28750 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:44.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.044+0000 7fcd59212740 1 --2- 172.21.15.172:0/4207920027 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55b6796a3f50 0x55b6796a6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.044+0000 7fcd59212740 4 client.28750 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:05:44.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.044+0000 7fcd59212740 1 -- 172.21.15.172:0/4207920027 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6795fc860 con 0x55b6796a3f50 2024-09-10T09:05:44.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.044+0000 7fcd4ffff640 1 --2- 172.21.15.172:0/4207920027 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55b6796a3f50 0x55b6796a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.045+0000 7fcd2ffff640 10 client.28750 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T09:05:44.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.045+0000 7fcd4ffff640 1 --2- 172.21.15.172:0/4207920027 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55b6796a3f50 0x55b6796a6360 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=35e02b43c8f10838 server_cookie=cf10e0797dc3398 in_seq=0 out_seq=0 2024-09-10T09:05:44.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.046+0000 7fcd2ffff640 1 -- 172.21.15.172:0/4207920027 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6795fc860 con 0x55b6796a3f50 2024-09-10T09:05:44.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.046+0000 7fcd2ffff640 10 client.28750 handle_command_reply: tid=0 2024-09-10T09:05:44.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.046+0000 7fcd59212740 1 client.28750 shutdown 2024-09-10T09:05:44.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.046+0000 7fcd59212740 1 -- 172.21.15.172:0/4207920027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6796895d0 msgr2=0x55b679693ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:44.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.046+0000 7fcd59212740 1 --2- 172.21.15.172:0/4207920027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6796895d0 0x55b679693ca0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcd44005fd0 tx=0x7fcd440303c0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.046+0000 7fcd59212740 1 -- 172.21.15.172:0/4207920027 shutdown_connections 2024-09-10T09:05:44.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.046+0000 7fcd59212740 1 --2- 172.21.15.172:0/4207920027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b679694370 0x55b679691490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.046+0000 7fcd59212740 1 --2- 172.21.15.172:0/4207920027 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55b6796a3f50 0x55b6796a6360 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.046+0000 7fcd59212740 1 --2- 172.21.15.172:0/4207920027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b67968ab40 0x55b679690f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.046+0000 7fcd59212740 1 --2- 172.21.15.172:0/4207920027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6796895d0 0x55b679693ca0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.046+0000 7fcd2ffff640 0 client.28750 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T09:05:44.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.046+0000 7fcd59212740 1 -- 172.21.15.172:0/4207920027 >> 172.21.15.172:0/4207920027 conn(0x55b679652310 msgr2=0x55b67968b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:44.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.047+0000 7fcd59212740 1 -- 172.21.15.172:0/4207920027 shutdown_connections 2024-09-10T09:05:44.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.047+0000 7fcd59212740 1 -- 172.21.15.172:0/4207920027 wait complete. 2024-09-10T09:05:44.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.047+0000 7fcd59212740 20 client.28750 trim_cache size 0 max 16384 2024-09-10T09:05:44.066 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.064+0000 7fcd59212740 1 Processor -- start 2024-09-10T09:05:44.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.064+0000 7fcd59212740 1 -- start start 2024-09-10T09:05:44.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.064+0000 7fcd59212740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6796895d0 0x55b6796a0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.064+0000 7fcd59212740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67968ab40 0x55b679695760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.064+0000 7fcd59212740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b679694370 0x55b679695ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.064+0000 7fcd59212740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501b0b20 con 0x55b67968ab40 2024-09-10T09:05:44.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.064+0000 7fcd59212740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501b2660 con 0x55b6796895d0 2024-09-10T09:05:44.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.064+0000 7fcd59212740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501b11f0 con 0x55b679694370 2024-09-10T09:05:44.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.065+0000 7fcd54bd5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6796895d0 0x55b6796a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.065+0000 7fcd54bd5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6796895d0 0x55b6796a0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:38396/0 (socket says 172.21.15.172:38396) 2024-09-10T09:05:44.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.065+0000 7fcd54bd5640 1 -- 172.21.15.172:0/3184784095 learned_addr learned my addr 172.21.15.172:0/3184784095 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:44.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.065+0000 7fcd4ffff640 1 --2- 172.21.15.172:0/3184784095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67968ab40 0x55b679695760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.065+0000 7fcd553d6640 1 --2- 172.21.15.172:0/3184784095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b679694370 0x55b679695ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.065+0000 7fcd54bd5640 1 -- 172.21.15.172:0/3184784095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b679694370 msgr2=0x55b679695ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:44.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.065+0000 7fcd54bd5640 1 --2- 172.21.15.172:0/3184784095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b679694370 0x55b679695ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.065+0000 7fcd54bd5640 1 -- 172.21.15.172:0/3184784095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67968ab40 msgr2=0x55b679695760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:44.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.065+0000 7fcd54bd5640 1 --2- 172.21.15.172:0/3184784095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67968ab40 0x55b679695760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.065+0000 7fcd54bd5640 1 -- 172.21.15.172:0/3184784095 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b67968f4a0 con 0x55b6796895d0 2024-09-10T09:05:44.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.065+0000 7fcd54bd5640 1 --2- 172.21.15.172:0/3184784095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6796895d0 0x55b6796a0340 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcd440b9680 tx=0x7fcd4402a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6edde8c61a445533 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:44.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.066+0000 7fcd367fc640 1 -- 172.21.15.172:0/3184784095 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd440baf10 con 0x55b6796895d0 2024-09-10T09:05:44.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.066+0000 7fcd367fc640 1 -- 172.21.15.172:0/3184784095 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd44030690 con 0x55b6796895d0 2024-09-10T09:05:44.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.066+0000 7fcd367fc640 1 -- 172.21.15.172:0/3184784095 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd440bb630 con 0x55b6796895d0 2024-09-10T09:05:44.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.066+0000 7fcd59212740 1 -- 172.21.15.172:0/3184784095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6796895d0 msgr2=0x55b6796a0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:44.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.066+0000 7fcd59212740 1 --2- 172.21.15.172:0/3184784095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6796895d0 0x55b6796a0340 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcd440b9680 tx=0x7fcd4402a340 comp rx=0 tx=0).stop 2024-09-10T09:05:44.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.067+0000 7fcd59212740 1 -- 172.21.15.172:0/3184784095 shutdown_connections 2024-09-10T09:05:44.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.067+0000 7fcd59212740 1 --2- 172.21.15.172:0/3184784095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b679694370 0x55b679695ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.067+0000 7fcd59212740 1 --2- 172.21.15.172:0/3184784095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b67968ab40 0x55b679695760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.067+0000 7fcd59212740 1 --2- 172.21.15.172:0/3184784095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6796895d0 0x55b6796a0340 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.067+0000 7fcd59212740 1 -- 172.21.15.172:0/3184784095 >> 172.21.15.172:0/3184784095 conn(0x55b679652310 msgr2=0x55b679690f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:44.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.067+0000 7fcd59212740 1 -- 172.21.15.172:0/3184784095 shutdown_connections 2024-09-10T09:05:44.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.067+0000 7fcd59212740 1 -- 172.21.15.172:0/3184784095 wait complete. 2024-09-10T09:05:44.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.067+0000 7fcd59212740 1 -- start start 2024-09-10T09:05:44.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.067+0000 7fcd59212740 1 Processor -- start 2024-09-10T09:05:44.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.068+0000 7fcd59212740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6796895d0 0x55b67969c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.068+0000 7fcd59212740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b67968ab40 0x55b67968e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.068+0000 7fcd59212740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b679694370 0x55b67968e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.068+0000 7fcd54bd5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6796895d0 0x55b67969c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.068+0000 7fcd59212740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501b1f90 con 0x55b6796895d0 2024-09-10T09:05:44.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.068+0000 7fcd59212740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501b18c0 con 0x55b679694370 2024-09-10T09:05:44.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.068+0000 7fcd59212740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcd501aeb00 con 0x55b67968ab40 2024-09-10T09:05:44.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.068+0000 7fcd54bd5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6796895d0 0x55b67969c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34938/0 (socket says 172.21.15.172:34938) 2024-09-10T09:05:44.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.068+0000 7fcd54bd5640 1 -- 172.21.15.172:0/2966072830 learned_addr learned my addr 172.21.15.172:0/2966072830 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:44.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.068+0000 7fcd59212740 10 client.0 _ll_register_callbacks cb 0x7fcd440097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:44.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.068+0000 7fcd4ffff640 1 --2- 172.21.15.172:0/2966072830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b67968ab40 0x55b67968e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.068+0000 7fcd553d6640 1 --2- 172.21.15.172:0/2966072830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b679694370 0x55b67968e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.069+0000 7fcd54bd5640 1 -- 172.21.15.172:0/2966072830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b67968ab40 msgr2=0x55b67968e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:44.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.069+0000 7fcd54bd5640 1 --2- 172.21.15.172:0/2966072830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b67968ab40 0x55b67968e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.069+0000 7fcd54bd5640 1 -- 172.21.15.172:0/2966072830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b679694370 msgr2=0x55b67968e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:44.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.069+0000 7fcd54bd5640 1 --2- 172.21.15.172:0/2966072830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b679694370 0x55b67968e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.069+0000 7fcd54bd5640 1 -- 172.21.15.172:0/2966072830 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b67968ed30 con 0x55b6796895d0 2024-09-10T09:05:44.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.069+0000 7fcd54bd5640 1 --2- 172.21.15.172:0/2966072830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6796895d0 0x55b67969c910 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fcd440bb0d0 tx=0x7fcd440b7670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef9fd7900fbf73e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:44.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.069+0000 7fcd2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:05:44.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.069+0000 7fcd2ffff640 1 -- 172.21.15.172:0/2966072830 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd440bb830 con 0x55b6796895d0 2024-09-10T09:05:44.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.069+0000 7fcd2ffff640 1 -- 172.21.15.172:0/2966072830 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcd440c3b20 con 0x55b6796895d0 2024-09-10T09:05:44.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.070+0000 7fcd59212740 1 -- 172.21.15.172:0/2966072830 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b6795f6d70 con 0x55b6796895d0 2024-09-10T09:05:44.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.070+0000 7fcd2ffff640 1 -- 172.21.15.172:0/2966072830 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcd440c3fb0 con 0x55b6796895d0 2024-09-10T09:05:44.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.070+0000 7fcd59212740 10 client.20145 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:44.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.070+0000 7fcd59212740 1 -- 172.21.15.172:0/2966072830 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b67968ed30 con 0x55b6796895d0 2024-09-10T09:05:44.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.070+0000 7fcd2ffff640 1 -- 172.21.15.172:0/2966072830 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55b6795f6d70 con 0x55b6796895d0 2024-09-10T09:05:44.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.070+0000 7fcd2ffff640 1 -- 172.21.15.172:0/2966072830 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fcd440b7120 con 0x55b6796895d0 2024-09-10T09:05:44.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.070+0000 7fcd59212740 10 client.20145 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:44.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.070+0000 7fcd59212740 10 client.20145 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T09:05:44.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.070+0000 7fcd59212740 20 client.20145 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:44.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.070+0000 7fcd59212740 1 --2- 172.21.15.172:0/2966072830 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55b679717560 0x55b679719950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:44.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.070+0000 7fcd59212740 4 client.20145 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:05:44.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.070+0000 7fcd59212740 1 -- 172.21.15.172:0/2966072830 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b679573170 con 0x55b679717560 2024-09-10T09:05:44.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.071+0000 7fcd4ffff640 1 --2- 172.21.15.172:0/2966072830 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55b679717560 0x55b679719950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:44.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.071+0000 7fcd2ffff640 10 client.20145 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T09:05:44.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.071+0000 7fcd4ffff640 1 --2- 172.21.15.172:0/2966072830 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55b679717560 0x55b679719950 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a6b4f6934a7f37ae server_cookie=3f41a1685e91e5e4 in_seq=0 out_seq=0 2024-09-10T09:05:44.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.071+0000 7fcd2ffff640 1 -- 172.21.15.172:0/2966072830 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b679573170 con 0x55b679717560 2024-09-10T09:05:44.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.071+0000 7fcd2ffff640 10 client.20145 handle_command_reply: tid=0 2024-09-10T09:05:44.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.071+0000 7fcd59212740 1 client.20145 shutdown 2024-09-10T09:05:44.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.071+0000 7fcd59212740 1 -- 172.21.15.172:0/2966072830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6796895d0 msgr2=0x55b67969c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:44.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.071+0000 7fcd59212740 1 --2- 172.21.15.172:0/2966072830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6796895d0 0x55b67969c910 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fcd440bb0d0 tx=0x7fcd440b7670 comp rx=0 tx=0).stop 2024-09-10T09:05:44.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.071+0000 7fcd59212740 1 -- 172.21.15.172:0/2966072830 shutdown_connections 2024-09-10T09:05:44.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.071+0000 7fcd59212740 1 --2- 172.21.15.172:0/2966072830 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x55b679717560 0x55b679719950 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.071+0000 7fcd2ffff640 0 client.20145 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T09:05:44.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.071+0000 7fcd59212740 1 --2- 172.21.15.172:0/2966072830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b679694370 0x55b67968e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.071+0000 7fcd59212740 1 --2- 172.21.15.172:0/2966072830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b67968ab40 0x55b67968e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.071+0000 7fcd59212740 1 --2- 172.21.15.172:0/2966072830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6796895d0 0x55b67969c910 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.071+0000 7fcd59212740 1 -- 172.21.15.172:0/2966072830 >> 172.21.15.172:0/2966072830 conn(0x55b679652310 msgr2=0x55b67968f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:44.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.072+0000 7fcd59212740 1 -- 172.21.15.172:0/2966072830 shutdown_connections 2024-09-10T09:05:44.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.072+0000 7fcd59212740 1 -- 172.21.15.172:0/2966072830 wait complete. 2024-09-10T09:05:44.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.072+0000 7fcd59212740 20 client.20145 trim_cache size 0 max 16384 2024-09-10T09:05:44.078 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:05:44.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.072+0000 7fcd56e39640 1 -- 172.21.15.172:0/486214549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcd380a2c00 msgr2=0x7fcd380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:44.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.072+0000 7fcd56e39640 1 --2- 172.21.15.172:0/486214549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcd380a2c00 0x7fcd380a50c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fcd48005c40 tx=0x7fcd48009ed0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.072+0000 7fcd56e39640 1 -- 172.21.15.172:0/486214549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd50100a90 msgr2=0x7fcd5010cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:44.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.072+0000 7fcd56e39640 1 --2- 172.21.15.172:0/486214549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd50100a90 0x7fcd5010cf20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcd44005940 tx=0x7fcd44030bc0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.072+0000 7fcd56e39640 1 -- 172.21.15.172:0/486214549 shutdown_connections 2024-09-10T09:05:44.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.072+0000 7fcd56e39640 1 --2- 172.21.15.172:0/486214549 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcd380a2c00 0x7fcd380a50c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.073+0000 7fcd56e39640 1 --2- 172.21.15.172:0/486214549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd50106a90 0x7fcd5019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.073+0000 7fcd56e39640 1 --2- 172.21.15.172:0/486214549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd50101430 0x7fcd5010d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.073+0000 7fcd56e39640 1 --2- 172.21.15.172:0/486214549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd50100a90 0x7fcd5010cf20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:44.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.073+0000 7fcd56e39640 1 -- 172.21.15.172:0/486214549 >> 172.21.15.172:0/486214549 conn(0x7fcd500fc7d0 msgr2=0x7fcd501108b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:44.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.073+0000 7fcd56e39640 1 -- 172.21.15.172:0/486214549 shutdown_connections 2024-09-10T09:05:44.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:44.073+0000 7fcd56e39640 1 -- 172.21.15.172:0/486214549 wait complete. 2024-09-10T09:05:44.821 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:05:44.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:44 smithi174 ceph-mon[34068]: pgmap v935: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 865 KiB/s wr, 227 op/s 2024-09-10T09:05:45.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:44 smithi191 ceph-mon[35314]: pgmap v935: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 865 KiB/s wr, 227 op/s 2024-09-10T09:05:45.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:44 smithi172 ceph-mon[30854]: pgmap v935: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 865 KiB/s wr, 227 op/s 2024-09-10T09:05:45.123 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:05:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.646+0000 7fe2df59e640 1 -- 172.21.15.172:0/3809522882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2e010a160 msgr2=0x7fe2e0110b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.646+0000 7fe2df59e640 1 --2- 172.21.15.172:0/3809522882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2e010a160 0x7fe2e0110b00 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009cd0 tx=0x7fe2d802a970 comp rx=0 tx=0).stop 2024-09-10T09:05:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.646+0000 7fe2df59e640 1 -- 172.21.15.172:0/3809522882 shutdown_connections 2024-09-10T09:05:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.646+0000 7fe2df59e640 1 --2- 172.21.15.172:0/3809522882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2e010a160 0x7fe2e0110b00 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.646+0000 7fe2df59e640 1 --2- 172.21.15.172:0/3809522882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2e01005f0 0x7fe2e0109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.646+0000 7fe2df59e640 1 --2- 172.21.15.172:0/3809522882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2e00ffcd0 0x7fe2e01000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.646+0000 7fe2df59e640 1 -- 172.21.15.172:0/3809522882 >> 172.21.15.172:0/3809522882 conn(0x7fe2e00fb920 msgr2=0x7fe2e00fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.646+0000 7fe2df59e640 1 -- 172.21.15.172:0/3809522882 shutdown_connections 2024-09-10T09:05:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.646+0000 7fe2df59e640 1 -- 172.21.15.172:0/3809522882 wait complete. 2024-09-10T09:05:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.646+0000 7fe2df59e640 1 Processor -- start 2024-09-10T09:05:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2df59e640 1 -- start start 2024-09-10T09:05:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2df59e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2e00ffcd0 0x7fe2e019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2df59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2e01005f0 0x7fe2e019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2df59e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2e010a160 0x7fe2e019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2df59e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e0113e60 con 0x7fe2e01005f0 2024-09-10T09:05:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2df59e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e0113ce0 con 0x7fe2e00ffcd0 2024-09-10T09:05:45.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2df59e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e0113fe0 con 0x7fe2e010a160 2024-09-10T09:05:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2e01005f0 0x7fe2e019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2de59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2e00ffcd0 0x7fe2e019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2d7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2e01005f0 0x7fe2e019a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34954/0 (socket says 172.21.15.172:34954) 2024-09-10T09:05:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2d7fff640 1 -- 172.21.15.172:0/1279251027 learned_addr learned my addr 172.21.15.172:0/1279251027 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2ded9d640 1 --2- 172.21.15.172:0/1279251027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2e010a160 0x7fe2e019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2de59c640 1 -- 172.21.15.172:0/1279251027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2e010a160 msgr2=0x7fe2e019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2de59c640 1 --2- 172.21.15.172:0/1279251027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2e010a160 0x7fe2e019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2de59c640 1 -- 172.21.15.172:0/1279251027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2e01005f0 msgr2=0x7fe2e019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2de59c640 1 --2- 172.21.15.172:0/1279251027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2e01005f0 0x7fe2e019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2de59c640 1 -- 172.21.15.172:0/1279251027 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e019f3b0 con 0x7fe2e00ffcd0 2024-09-10T09:05:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2d7fff640 1 --2- 172.21.15.172:0/1279251027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2e01005f0 0x7fe2e019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:45.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.647+0000 7fe2ded9d640 1 --2- 172.21.15.172:0/1279251027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2e010a160 0x7fe2e019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.648+0000 7fe2de59c640 1 --2- 172.21.15.172:0/1279251027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2e00ffcd0 0x7fe2e019a400 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe2d000e9f0 tx=0x7fe2d000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.648+0000 7fe2d77fe640 1 -- 172.21.15.172:0/1279251027 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d000ccc0 con 0x7fe2e00ffcd0 2024-09-10T09:05:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.648+0000 7fe2d77fe640 1 -- 172.21.15.172:0/1279251027 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2d0005020 con 0x7fe2e00ffcd0 2024-09-10T09:05:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.648+0000 7fe2df59e640 1 -- 172.21.15.172:0/1279251027 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e019f6a0 con 0x7fe2e00ffcd0 2024-09-10T09:05:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.648+0000 7fe2d77fe640 1 -- 172.21.15.172:0/1279251027 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0004a00 con 0x7fe2e00ffcd0 2024-09-10T09:05:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.648+0000 7fe2df59e640 1 -- 172.21.15.172:0/1279251027 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e01a6f80 con 0x7fe2e00ffcd0 2024-09-10T09:05:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.651+0000 7fe2d77fe640 1 -- 172.21.15.172:0/1279251027 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe2d0004ba0 con 0x7fe2e00ffcd0 2024-09-10T09:05:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.651+0000 7fe2e73b4740 1 Processor -- start 2024-09-10T09:05:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.652+0000 7fe2e73b4740 1 -- start start 2024-09-10T09:05:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.652+0000 7fe2d77fe640 1 --2- 172.21.15.172:0/1279251027 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe2cc0a2b80 0x7fe2cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.652+0000 7fe2e73b4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e2d20 0x55d1903e1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.652+0000 7fe2e73b4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e17b0 0x55d1903e1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.652+0000 7fe2e73b4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903ec550 0x55d1903ee940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.652+0000 7fe2e73b4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d1903472f0 con 0x55d1903ec550 2024-09-10T09:05:45.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.652+0000 7fe2e73b4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d1902eecf0 con 0x55d1903e2d20 2024-09-10T09:05:45.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.652+0000 7fe2e73b4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d1903432b0 con 0x55d1903e17b0 2024-09-10T09:05:45.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.652+0000 7fe2ded9d640 1 --2- 172.21.15.172:0/1279251027 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe2cc0a2b80 0x7fe2cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.652+0000 7fe2d77fe640 1 -- 172.21.15.172:0/1279251027 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe2d00cf350 con 0x7fe2e00ffcd0 2024-09-10T09:05:45.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.652+0000 7fe2d7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e17b0 0x55d1903e1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.652+0000 7fe2d7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e17b0 0x55d1903e1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33812/0 (socket says 172.21.15.172:33812) 2024-09-10T09:05:45.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.652+0000 7fe2d7fff640 1 -- 172.21.15.172:0/191142898 learned_addr learned my addr 172.21.15.172:0/191142898 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:45.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.652+0000 7fe2ded9d640 1 --2- 172.21.15.172:0/191142898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903ec550 0x55d1903ee940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.653+0000 7fe2de59c640 1 --2- 172.21.15.172:0/191142898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e2d20 0x55d1903e1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.653+0000 7fe2d7fff640 1 -- 172.21.15.172:0/191142898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e2d20 msgr2=0x55d1903e1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:45.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.653+0000 7fe2d7fff640 1 --2- 172.21.15.172:0/191142898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e2d20 0x55d1903e1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.653+0000 7fe2d7fff640 1 -- 172.21.15.172:0/191142898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903ec550 msgr2=0x55d1903ee940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:45.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.653+0000 7fe2d7fff640 1 --2- 172.21.15.172:0/191142898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903ec550 0x55d1903ee940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.653+0000 7fe2d7fff640 1 -- 172.21.15.172:0/191142898 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1902d33a0 con 0x55d1903e17b0 2024-09-10T09:05:45.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.653+0000 7fe2de59c640 1 --2- 172.21.15.172:0/191142898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e2d20 0x55d1903e1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:05:45.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.653+0000 7fe2ded9d640 1 --2- 172.21.15.172:0/1279251027 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe2cc0a2b80 0x7fe2cc0a5040 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009a60 tx=0x7fe2d80099d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:45.736 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:05:45.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.653+0000 7fe2d7fff640 1 --2- 172.21.15.172:0/191142898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e17b0 0x55d1903e1bf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8011510 tx=0x7fe2c80119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dddd2b384f0fdbe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:45.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.653+0000 7fe2d4ff9640 1 -- 172.21.15.172:0/191142898 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c801e070 con 0x55d1903e17b0 2024-09-10T09:05:45.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.653+0000 7fe2d4ff9640 1 -- 172.21.15.172:0/191142898 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2c800fa70 con 0x55d1903e17b0 2024-09-10T09:05:45.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.653+0000 7fe2d4ff9640 1 -- 172.21.15.172:0/191142898 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8019a90 con 0x55d1903e17b0 2024-09-10T09:05:45.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.654+0000 7fe2e73b4740 1 -- 172.21.15.172:0/191142898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e17b0 msgr2=0x55d1903e1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:45.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.654+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/191142898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e17b0 0x55d1903e1bf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8011510 tx=0x7fe2c80119d0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.654+0000 7fe2e73b4740 1 -- 172.21.15.172:0/191142898 shutdown_connections 2024-09-10T09:05:45.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.654+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/191142898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903ec550 0x55d1903ee940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.654+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/191142898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e17b0 0x55d1903e1bf0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.654+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/191142898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e2d20 0x55d1903e1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.654+0000 7fe2e73b4740 1 -- 172.21.15.172:0/191142898 >> 172.21.15.172:0/191142898 conn(0x55d1903aa310 msgr2=0x55d1903dde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:45.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.654+0000 7fe2e73b4740 1 -- 172.21.15.172:0/191142898 shutdown_connections 2024-09-10T09:05:45.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.654+0000 7fe2e73b4740 1 -- 172.21.15.172:0/191142898 wait complete. 2024-09-10T09:05:45.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.654+0000 7fe2e73b4740 1 -- start start 2024-09-10T09:05:45.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.654+0000 7fe2e73b4740 1 Processor -- start 2024-09-10T09:05:45.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2e73b4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903e17b0 0x55d1903ebe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2e73b4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e2d20 0x55d1903e9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2e73b4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903ec550 0x55d1903e9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2e73b4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d1903ef250 con 0x55d1903e17b0 2024-09-10T09:05:45.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2e73b4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d190302060 con 0x55d1903ec550 2024-09-10T09:05:45.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2e73b4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d1903ef3d0 con 0x55d1903e2d20 2024-09-10T09:05:45.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2d7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e2d20 0x55d1903e9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2d7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e2d20 0x55d1903e9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33828/0 (socket says 172.21.15.172:33828) 2024-09-10T09:05:45.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2d7fff640 1 -- 172.21.15.172:0/3114082927 learned_addr learned my addr 172.21.15.172:0/3114082927 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:45.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2ded9d640 1 --2- 172.21.15.172:0/3114082927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903ec550 0x55d1903e9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2e73b4740 10 client.0 _ll_register_callbacks cb 0x55d1900558c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:45.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2de59c640 1 --2- 172.21.15.172:0/3114082927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903e17b0 0x55d1903ebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.739 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2ded9d640 1 -- 172.21.15.172:0/3114082927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e2d20 msgr2=0x55d1903e9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:45.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2ded9d640 1 --2- 172.21.15.172:0/3114082927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e2d20 0x55d1903e9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2ded9d640 1 -- 172.21.15.172:0/3114082927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903e17b0 msgr2=0x55d1903ebe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:45.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2ded9d640 1 --2- 172.21.15.172:0/3114082927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903e17b0 0x55d1903ebe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2ded9d640 1 -- 172.21.15.172:0/3114082927 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1900b83c0 con 0x55d1903ec550 2024-09-10T09:05:45.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2d7fff640 1 --2- 172.21.15.172:0/3114082927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e2d20 0x55d1903e9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:45.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.655+0000 7fe2de59c640 1 --2- 172.21.15.172:0/3114082927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903e17b0 0x55d1903ebe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:45.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.656+0000 7fe2ded9d640 1 --2- 172.21.15.172:0/3114082927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903ec550 0x55d1903e9680 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8005e30 tx=0x7fe2d8049000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51cfbc2f735a0ebf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:45.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.656+0000 7fe2c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:05:45.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.656+0000 7fe2c5ffb640 1 -- 172.21.15.172:0/3114082927 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8043040 con 0x55d1903ec550 2024-09-10T09:05:45.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.656+0000 7fe2c5ffb640 1 -- 172.21.15.172:0/3114082927 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe2d8030b00 con 0x55d1903ec550 2024-09-10T09:05:45.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.656+0000 7fe2e73b4740 1 -- 172.21.15.172:0/3114082927 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c800fa70 con 0x55d1903ec550 2024-09-10T09:05:45.740 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.656+0000 7fe2c5ffb640 1 -- 172.21.15.172:0/3114082927 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe2d8034040 con 0x55d1903ec550 2024-09-10T09:05:45.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.656+0000 7fe2e73b4740 10 client.28868 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:45.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.656+0000 7fe2e73b4740 1 -- 172.21.15.172:0/3114082927 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d1903fac10 con 0x55d1903ec550 2024-09-10T09:05:45.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.656+0000 7fe2c5ffb640 1 -- 172.21.15.172:0/3114082927 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2c800fa70 con 0x55d1903ec550 2024-09-10T09:05:45.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.657+0000 7fe2c5ffb640 1 -- 172.21.15.172:0/3114082927 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fe2d80056e0 con 0x55d1903ec550 2024-09-10T09:05:45.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.657+0000 7fe2e73b4740 10 client.28868 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:45.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.657+0000 7fe2e73b4740 10 client.28868 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T09:05:45.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.657+0000 7fe2e73b4740 20 client.28868 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:45.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.657+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/3114082927 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55d1903fc000 0x55d1903fe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.657+0000 7fe2e73b4740 4 client.28868 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:05:45.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.657+0000 7fe2e73b4740 1 -- 172.21.15.172:0/3114082927 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d190354860 con 0x55d1903fc000 2024-09-10T09:05:45.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.657+0000 7fe2de59c640 1 --2- 172.21.15.172:0/3114082927 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55d1903fc000 0x55d1903fe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.657+0000 7fe2c5ffb640 10 client.28868 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T09:05:45.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.657+0000 7fe2de59c640 1 --2- 172.21.15.172:0/3114082927 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55d1903fc000 0x55d1903fe4a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c9d148b4eeb702d3 server_cookie=ba1109c980f1bccb in_seq=0 out_seq=0 2024-09-10T09:05:45.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.658+0000 7fe2c5ffb640 1 -- 172.21.15.172:0/3114082927 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d190354860 con 0x55d1903fc000 2024-09-10T09:05:45.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.658+0000 7fe2c5ffb640 10 client.28868 handle_command_reply: tid=0 2024-09-10T09:05:45.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.658+0000 7fe2e73b4740 1 client.28868 shutdown 2024-09-10T09:05:45.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.658+0000 7fe2e73b4740 1 -- 172.21.15.172:0/3114082927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903ec550 msgr2=0x55d1903e9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:45.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.658+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/3114082927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903ec550 0x55d1903e9680 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8005e30 tx=0x7fe2d8049000 comp rx=0 tx=0).stop 2024-09-10T09:05:45.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.658+0000 7fe2e73b4740 1 -- 172.21.15.172:0/3114082927 shutdown_connections 2024-09-10T09:05:45.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.658+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/3114082927 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55d1903fc000 0x55d1903fe4a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.658+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/3114082927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903ec550 0x55d1903e9680 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.658+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/3114082927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e2d20 0x55d1903e9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.658+0000 7fe2c5ffb640 0 client.28868 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T09:05:45.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.658+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/3114082927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903e17b0 0x55d1903ebe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.658+0000 7fe2e73b4740 1 -- 172.21.15.172:0/3114082927 >> 172.21.15.172:0/3114082927 conn(0x55d1903aa310 msgr2=0x55d1903e3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:45.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.658+0000 7fe2e73b4740 1 -- 172.21.15.172:0/3114082927 shutdown_connections 2024-09-10T09:05:45.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.658+0000 7fe2e73b4740 1 -- 172.21.15.172:0/3114082927 wait complete. 2024-09-10T09:05:45.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.659+0000 7fe2e73b4740 20 client.28868 trim_cache size 0 max 16384 2024-09-10T09:05:45.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2e73b4740 1 Processor -- start 2024-09-10T09:05:45.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2e73b4740 1 -- start start 2024-09-10T09:05:45.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2e73b4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e17b0 0x55d1903f8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2e73b4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e2d20 0x55d1903ed940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2e73b4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903ec550 0x55d1903ede80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2e73b4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e01b0c30 con 0x55d1903ec550 2024-09-10T09:05:45.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2e73b4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e01b2770 con 0x55d1903e2d20 2024-09-10T09:05:45.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2e73b4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e01b1300 con 0x55d1903e17b0 2024-09-10T09:05:45.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2ded9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903ec550 0x55d1903ede80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2de59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e17b0 0x55d1903f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2d7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e2d20 0x55d1903ed940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2de59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e17b0 0x55d1903f8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33844/0 (socket says 172.21.15.172:33844) 2024-09-10T09:05:45.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2ded9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903ec550 0x55d1903ede80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:34988/0 (socket says 172.21.15.172:34988) 2024-09-10T09:05:45.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2de59c640 1 -- 172.21.15.172:0/2905641073 learned_addr learned my addr 172.21.15.172:0/2905641073 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:45.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2de59c640 1 -- 172.21.15.172:0/2905641073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e2d20 msgr2=0x55d1903ed940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:45.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2de59c640 1 --2- 172.21.15.172:0/2905641073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e2d20 0x55d1903ed940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2de59c640 1 -- 172.21.15.172:0/2905641073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903ec550 msgr2=0x55d1903ede80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:45.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2de59c640 1 --2- 172.21.15.172:0/2905641073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903ec550 0x55d1903ede80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.674+0000 7fe2de59c640 1 -- 172.21.15.172:0/2905641073 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1903deb40 con 0x55d1903e17b0 2024-09-10T09:05:45.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.675+0000 7fe2de59c640 1 --2- 172.21.15.172:0/2905641073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e17b0 0x55d1903f8550 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0004770 tx=0x7fe2d00054d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bfbcbf9702b0792 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:45.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.675+0000 7fe2c7fff640 1 -- 172.21.15.172:0/2905641073 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d001c040 con 0x55d1903e17b0 2024-09-10T09:05:45.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.675+0000 7fe2c7fff640 1 -- 172.21.15.172:0/2905641073 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2d009fc00 con 0x55d1903e17b0 2024-09-10T09:05:45.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.675+0000 7fe2c7fff640 1 -- 172.21.15.172:0/2905641073 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d00cf080 con 0x55d1903e17b0 2024-09-10T09:05:45.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.676+0000 7fe2e73b4740 1 -- 172.21.15.172:0/2905641073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e17b0 msgr2=0x55d1903f8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:45.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.676+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/2905641073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e17b0 0x55d1903f8550 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0004770 tx=0x7fe2d00054d0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.676+0000 7fe2e73b4740 1 -- 172.21.15.172:0/2905641073 shutdown_connections 2024-09-10T09:05:45.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.676+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/2905641073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903ec550 0x55d1903ede80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.676+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/2905641073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e2d20 0x55d1903ed940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.676+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/2905641073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e17b0 0x55d1903f8550 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.676+0000 7fe2e73b4740 1 -- 172.21.15.172:0/2905641073 >> 172.21.15.172:0/2905641073 conn(0x55d1903aa310 msgr2=0x55d1903e7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:45.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.676+0000 7fe2e73b4740 1 -- 172.21.15.172:0/2905641073 shutdown_connections 2024-09-10T09:05:45.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.676+0000 7fe2e73b4740 1 -- 172.21.15.172:0/2905641073 wait complete. 2024-09-10T09:05:45.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.676+0000 7fe2e73b4740 1 -- start start 2024-09-10T09:05:45.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.676+0000 7fe2e73b4740 1 Processor -- start 2024-09-10T09:05:45.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2e73b4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e17b0 0x55d1903f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2e73b4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e2d20 0x55d190403820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2e73b4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903ec550 0x55d190403f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2e73b4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e01b20a0 con 0x55d1903ec550 2024-09-10T09:05:45.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2e73b4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e01b19d0 con 0x55d1903e17b0 2024-09-10T09:05:45.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2e73b4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e01aec10 con 0x55d1903e2d20 2024-09-10T09:05:45.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2de59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e17b0 0x55d1903f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2ded9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903ec550 0x55d190403f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2d7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e2d20 0x55d190403820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.746 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2de59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e17b0 0x55d1903f4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:38452/0 (socket says 172.21.15.172:38452) 2024-09-10T09:05:45.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2de59c640 1 -- 172.21.15.172:0/2367481504 learned_addr learned my addr 172.21.15.172:0/2367481504 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:45.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2ded9d640 1 -- 172.21.15.172:0/2367481504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e2d20 msgr2=0x55d190403820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:45.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2ded9d640 1 --2- 172.21.15.172:0/2367481504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e2d20 0x55d190403820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2ded9d640 1 -- 172.21.15.172:0/2367481504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e17b0 msgr2=0x55d1903f4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:45.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2ded9d640 1 --2- 172.21.15.172:0/2367481504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e17b0 0x55d1903f4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2ded9d640 1 -- 172.21.15.172:0/2367481504 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1903f2a40 con 0x55d1903ec550 2024-09-10T09:05:45.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2e73b4740 10 client.0 _ll_register_callbacks cb 0x7fe2d8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:45.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2de59c640 1 --2- 172.21.15.172:0/2367481504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e17b0 0x55d1903f4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:45.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2e73b4740 1 -- 172.21.15.172:0/2367481504 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2d009fc00 con 0x55d1903ec550 2024-09-10T09:05:45.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2d7fff640 1 --2- 172.21.15.172:0/2367481504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e2d20 0x55d190403820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:45.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.677+0000 7fe2ded9d640 1 --2- 172.21.15.172:0/2367481504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903ec550 0x55d190403f60 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8005ad0 tx=0x7fe2d8030c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87ad87450494ba9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:45.747 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.678+0000 7fe2c5ffb640 10 client.20163 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:05:45.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.678+0000 7fe2c5ffb640 1 -- 172.21.15.172:0/2367481504 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8053590 con 0x55d1903ec550 2024-09-10T09:05:45.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.678+0000 7fe2c5ffb640 1 -- 172.21.15.172:0/2367481504 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe2d8053730 con 0x55d1903ec550 2024-09-10T09:05:45.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.678+0000 7fe2c5ffb640 1 -- 172.21.15.172:0/2367481504 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe2d8053970 con 0x55d1903ec550 2024-09-10T09:05:45.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.678+0000 7fe2e73b4740 10 client.20163 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:45.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.678+0000 7fe2e73b4740 1 -- 172.21.15.172:0/2367481504 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=44}) -- 0x55d190466e10 con 0x55d1903ec550 2024-09-10T09:05:45.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.678+0000 7fe2c5ffb640 1 -- 172.21.15.172:0/2367481504 <== mon.0 v2:172.21.15.172:3300/0 4 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fe2d8053b10 con 0x55d1903ec550 2024-09-10T09:05:45.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.678+0000 7fe2e73b4740 10 client.20163 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:45.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.678+0000 7fe2e73b4740 10 client.20163 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T09:05:45.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.678+0000 7fe2e73b4740 20 client.20163 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:45.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.678+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/2367481504 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55d190467350 0x55d190469760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:45.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.678+0000 7fe2e73b4740 4 client.20163 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:05:45.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.678+0000 7fe2e73b4740 1 -- 172.21.15.172:0/2367481504 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d19034ed70 con 0x55d190467350 2024-09-10T09:05:45.748 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.679+0000 7fe2de59c640 1 --2- 172.21.15.172:0/2367481504 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55d190467350 0x55d190469760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:45.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.679+0000 7fe2c5ffb640 10 client.20163 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T09:05:45.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.679+0000 7fe2de59c640 1 --2- 172.21.15.172:0/2367481504 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55d190467350 0x55d190469760 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d16b7fb90b71fdc7 server_cookie=c985a824a5b0a740 in_seq=0 out_seq=0 2024-09-10T09:05:45.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.679+0000 7fe2c5ffb640 1 -- 172.21.15.172:0/2367481504 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d19034ed70 con 0x55d190467350 2024-09-10T09:05:45.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.679+0000 7fe2c5ffb640 10 client.20163 handle_command_reply: tid=0 2024-09-10T09:05:45.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.679+0000 7fe2e73b4740 1 client.20163 shutdown 2024-09-10T09:05:45.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.679+0000 7fe2e73b4740 1 -- 172.21.15.172:0/2367481504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903ec550 msgr2=0x55d190403f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:45.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.679+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/2367481504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903ec550 0x55d190403f60 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8005ad0 tx=0x7fe2d8030c30 comp rx=0 tx=0).stop 2024-09-10T09:05:45.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.679+0000 7fe2e73b4740 1 -- 172.21.15.172:0/2367481504 shutdown_connections 2024-09-10T09:05:45.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.679+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/2367481504 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55d190467350 0x55d190469760 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.679+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/2367481504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1903ec550 0x55d190403f60 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.679+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/2367481504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d1903e2d20 0x55d190403820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.679+0000 7fe2e73b4740 1 --2- 172.21.15.172:0/2367481504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d1903e17b0 0x55d1903f4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.749 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.680+0000 7fe2e73b4740 1 -- 172.21.15.172:0/2367481504 >> 172.21.15.172:0/2367481504 conn(0x55d1903aa310 msgr2=0x55d1903ea9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:45.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.680+0000 7fe2c5ffb640 0 client.20163 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T09:05:45.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.680+0000 7fe2e73b4740 1 -- 172.21.15.172:0/2367481504 shutdown_connections 2024-09-10T09:05:45.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.680+0000 7fe2e73b4740 1 -- 172.21.15.172:0/2367481504 wait complete. 2024-09-10T09:05:45.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.680+0000 7fe2e73b4740 20 client.20163 trim_cache size 0 max 16384 2024-09-10T09:05:45.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.680+0000 7fe2df59e640 1 -- 172.21.15.172:0/1279251027 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe2cc0a2b80 msgr2=0x7fe2cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:45.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.680+0000 7fe2df59e640 1 --2- 172.21.15.172:0/1279251027 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe2cc0a2b80 0x7fe2cc0a5040 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009a60 tx=0x7fe2d80099d0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.680+0000 7fe2df59e640 1 -- 172.21.15.172:0/1279251027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2e00ffcd0 msgr2=0x7fe2e019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:45.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.680+0000 7fe2df59e640 1 --2- 172.21.15.172:0/1279251027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2e00ffcd0 0x7fe2e019a400 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe2d000e9f0 tx=0x7fe2d000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.681+0000 7fe2df59e640 1 -- 172.21.15.172:0/1279251027 shutdown_connections 2024-09-10T09:05:45.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.681+0000 7fe2df59e640 1 --2- 172.21.15.172:0/1279251027 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe2cc0a2b80 0x7fe2cc0a5040 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.681+0000 7fe2df59e640 1 --2- 172.21.15.172:0/1279251027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2e010a160 0x7fe2e019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.681+0000 7fe2df59e640 1 --2- 172.21.15.172:0/1279251027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe2e01005f0 0x7fe2e019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.681+0000 7fe2df59e640 1 --2- 172.21.15.172:0/1279251027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2e00ffcd0 0x7fe2e019a400 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:45.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.681+0000 7fe2df59e640 1 -- 172.21.15.172:0/1279251027 >> 172.21.15.172:0/1279251027 conn(0x7fe2e00fb920 msgr2=0x7fe2e0107cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:45.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.681+0000 7fe2df59e640 1 -- 172.21.15.172:0/1279251027 shutdown_connections 2024-09-10T09:05:45.751 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:45.681+0000 7fe2df59e640 1 -- 172.21.15.172:0/1279251027 wait complete. 2024-09-10T09:05:46.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:46 smithi174 ceph-mon[34068]: pgmap v936: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 765 KiB/s wr, 203 op/s 2024-09-10T09:05:47.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:46 smithi191 ceph-mon[35314]: pgmap v936: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 765 KiB/s wr, 203 op/s 2024-09-10T09:05:47.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:46 smithi172 ceph-mon[30854]: pgmap v936: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 765 KiB/s wr, 203 op/s 2024-09-10T09:05:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.639+0000 7f2d95d6d640 1 -- 172.21.15.172:0/213449434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d90100cb0 msgr2=0x7f2d90101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.639+0000 7f2d95d6d640 1 --2- 172.21.15.172:0/213449434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d90100cb0 0x7f2d90101130 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009900 tx=0x7f2d8402a410 comp rx=0 tx=0).stop 2024-09-10T09:05:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.639+0000 7f2d95d6d640 1 -- 172.21.15.172:0/213449434 shutdown_connections 2024-09-10T09:05:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.639+0000 7f2d95d6d640 1 --2- 172.21.15.172:0/213449434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d90101670 0x7f2d90108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.639+0000 7f2d95d6d640 1 --2- 172.21.15.172:0/213449434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d90100cb0 0x7f2d90101130 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.639+0000 7f2d95d6d640 1 --2- 172.21.15.172:0/213449434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d90102060 0x7f2d90102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.639+0000 7f2d95d6d640 1 -- 172.21.15.172:0/213449434 >> 172.21.15.172:0/213449434 conn(0x7f2d900fb930 msgr2=0x7f2d900fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.639+0000 7f2d95d6d640 1 -- 172.21.15.172:0/213449434 shutdown_connections 2024-09-10T09:05:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.639+0000 7f2d95d6d640 1 -- 172.21.15.172:0/213449434 wait complete. 2024-09-10T09:05:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.640+0000 7f2d95d6d640 1 Processor -- start 2024-09-10T09:05:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.640+0000 7f2d95d6d640 1 -- start start 2024-09-10T09:05:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.640+0000 7f2d95d6d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d90100cb0 0x7f2d90100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.640+0000 7f2d95d6d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d90101670 0x7f2d9019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.640+0000 7f2d95d6d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d90102060 0x7f2d9019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.640+0000 7f2d95d6d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9010c590 con 0x7f2d90101670 2024-09-10T09:05:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.640+0000 7f2d95d6d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9010c410 con 0x7f2d90102060 2024-09-10T09:05:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.640+0000 7f2d95d6d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9010c710 con 0x7f2d90100cb0 2024-09-10T09:05:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.640+0000 7f2d8effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d90101670 0x7f2d9019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.640+0000 7f2d8f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d90100cb0 0x7f2d90100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.640+0000 7f2d8f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d90100cb0 0x7f2d90100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33208/0 (socket says 172.21.15.172:33208) 2024-09-10T09:05:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.640+0000 7f2d8f7fe640 1 -- 172.21.15.172:0/3375609950 learned_addr learned my addr 172.21.15.172:0/3375609950 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.640+0000 7f2d8ffff640 1 --2- 172.21.15.172:0/3375609950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d90102060 0x7f2d9019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.640+0000 7f2d8effd640 1 -- 172.21.15.172:0/3375609950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d90100cb0 msgr2=0x7f2d90100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.640+0000 7f2d8effd640 1 --2- 172.21.15.172:0/3375609950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d90100cb0 0x7f2d90100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.640+0000 7f2d8effd640 1 -- 172.21.15.172:0/3375609950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d90102060 msgr2=0x7f2d9019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.640+0000 7f2d8effd640 1 --2- 172.21.15.172:0/3375609950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d90102060 0x7f2d9019f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.641+0000 7f2d8effd640 1 -- 172.21.15.172:0/3375609950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9019fa00 con 0x7f2d90101670 2024-09-10T09:05:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.641+0000 7f2d8ffff640 1 --2- 172.21.15.172:0/3375609950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d90102060 0x7f2d9019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.641+0000 7f2d8effd640 1 --2- 172.21.15.172:0/3375609950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d90101670 0x7f2d9019cad0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009230 tx=0x7f2d84004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.641+0000 7f2d8cff9640 1 -- 172.21.15.172:0/3375609950 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8403f030 con 0x7f2d90101670 2024-09-10T09:05:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.641+0000 7f2d8cff9640 1 -- 172.21.15.172:0/3375609950 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d84003aa0 con 0x7f2d90101670 2024-09-10T09:05:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.641+0000 7f2d95d6d640 1 -- 172.21.15.172:0/3375609950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d901968a0 con 0x7f2d90101670 2024-09-10T09:05:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.641+0000 7f2d8cff9640 1 -- 172.21.15.172:0/3375609950 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84035070 con 0x7f2d90101670 2024-09-10T09:05:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.641+0000 7f2d95d6d640 1 -- 172.21.15.172:0/3375609950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d90196d10 con 0x7f2d90101670 2024-09-10T09:05:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.644+0000 7f2d8cff9640 1 -- 172.21.15.172:0/3375609950 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2d84042030 con 0x7f2d90101670 2024-09-10T09:05:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.644+0000 7f2d98be5740 1 Processor -- start 2024-09-10T09:05:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.645+0000 7f2d98be5740 1 -- start start 2024-09-10T09:05:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.645+0000 7f2d98be5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc599a0 0x55d59bc59da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.645+0000 7f2d8cff9640 1 --2- 172.21.15.172:0/3375609950 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2d780a2c30 0x7f2d780a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.645+0000 7f2d98be5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5a2e0 0x55d59bc5eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.645+0000 7f2d8ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc599a0 0x55d59bc59da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.645+0000 7f2d8ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc599a0 0x55d59bc59da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33222/0 (socket says 172.21.15.172:33222) 2024-09-10T09:05:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.645+0000 7f2d8ffff640 1 -- 172.21.15.172:0/2517739001 learned_addr learned my addr 172.21.15.172:0/2517739001 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.645+0000 7f2d98be5740 1 --2- 172.21.15.172:0/2517739001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc5f370 0x55d59bc61830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.645+0000 7f2d98be5740 1 -- 172.21.15.172:0/2517739001 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d59bbbd2f0 con 0x55d59bc5f370 2024-09-10T09:05:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.645+0000 7f2d98be5740 1 -- 172.21.15.172:0/2517739001 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d59bb65270 con 0x55d59bc5a2e0 2024-09-10T09:05:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.645+0000 7f2d8cff9640 1 -- 172.21.15.172:0/3375609950 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2d840ef980 con 0x7f2d90101670 2024-09-10T09:05:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.645+0000 7f2d8f7fe640 1 --2- 172.21.15.172:0/3375609950 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2d780a2c30 0x7f2d780a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.645+0000 7f2d98be5740 1 -- 172.21.15.172:0/2517739001 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d59bbb92b0 con 0x55d59bc599a0 2024-09-10T09:05:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.645+0000 7f2d8effd640 1 --2- 172.21.15.172:0/2517739001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5a2e0 0x55d59bc5eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.645+0000 7f2d8ffff640 1 --2- 172.21.15.172:0/2517739001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc5f370 0x55d59bc61830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.646+0000 7f2d8effd640 1 -- 172.21.15.172:0/2517739001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc599a0 msgr2=0x55d59bc59da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.646+0000 7f2d8effd640 1 --2- 172.21.15.172:0/2517739001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc599a0 0x55d59bc59da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.646+0000 7f2d8effd640 1 -- 172.21.15.172:0/2517739001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc5f370 msgr2=0x55d59bc61830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.646+0000 7f2d8effd640 1 --2- 172.21.15.172:0/2517739001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc5f370 0x55d59bc61830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.646+0000 7f2d8effd640 1 -- 172.21.15.172:0/2517739001 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d59bb64340 con 0x55d59bc5a2e0 2024-09-10T09:05:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.646+0000 7f2d8effd640 1 --2- 172.21.15.172:0/2517739001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5a2e0 0x55d59bc5eda0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2d840ed490 tx=0x7f2d84009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b93d25b1d5ffee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.646+0000 7f2d8f7fe640 1 --2- 172.21.15.172:0/3375609950 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2d780a2c30 0x7f2d780a50f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2d800058c0 tx=0x7f2d80005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.646+0000 7f2d75ffb640 1 -- 172.21.15.172:0/2517739001 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d84049070 con 0x55d59bc5a2e0 2024-09-10T09:05:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.646+0000 7f2d75ffb640 1 -- 172.21.15.172:0/2517739001 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d84039d00 con 0x55d59bc5a2e0 2024-09-10T09:05:47.649 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.646+0000 7f2d75ffb640 1 -- 172.21.15.172:0/2517739001 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d840c4af0 con 0x55d59bc5a2e0 2024-09-10T09:05:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.647+0000 7f2d98be5740 1 -- 172.21.15.172:0/2517739001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5a2e0 msgr2=0x55d59bc5eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.647+0000 7f2d98be5740 1 --2- 172.21.15.172:0/2517739001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5a2e0 0x55d59bc5eda0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2d840ed490 tx=0x7f2d84009260 comp rx=0 tx=0).stop 2024-09-10T09:05:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.647+0000 7f2d98be5740 1 -- 172.21.15.172:0/2517739001 shutdown_connections 2024-09-10T09:05:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.647+0000 7f2d98be5740 1 --2- 172.21.15.172:0/2517739001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc5f370 0x55d59bc61830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.647+0000 7f2d98be5740 1 --2- 172.21.15.172:0/2517739001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5a2e0 0x55d59bc5eda0 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2d840ed490 tx=0x7f2d84009260 comp rx=0 tx=0).stop 2024-09-10T09:05:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.647+0000 7f2d98be5740 1 --2- 172.21.15.172:0/2517739001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc599a0 0x55d59bc59da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.647+0000 7f2d98be5740 1 -- 172.21.15.172:0/2517739001 >> 172.21.15.172:0/2517739001 conn(0x55d59bc55010 msgr2=0x55d59bc55410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.648+0000 7f2d98be5740 1 -- 172.21.15.172:0/2517739001 shutdown_connections 2024-09-10T09:05:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.648+0000 7f2d98be5740 1 -- 172.21.15.172:0/2517739001 wait complete. 2024-09-10T09:05:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.648+0000 7f2d98be5740 1 -- start start 2024-09-10T09:05:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.648+0000 7f2d98be5740 1 Processor -- start 2024-09-10T09:05:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.648+0000 7f2d98be5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc599a0 0x55d59bc6c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.648+0000 7f2d98be5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 0x55d59bc6c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.649+0000 7f2d98be5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc71760 0x55d59bc73b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.653 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.649+0000 7f2d8effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 0x55d59bc6c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.649+0000 7f2d8effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 0x55d59bc6c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:38344/0 (socket says 172.21.15.172:38344) 2024-09-10T09:05:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.649+0000 7f2d8effd640 1 -- 172.21.15.172:0/2070004435 learned_addr learned my addr 172.21.15.172:0/2070004435 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.649+0000 7f2d8f7fe640 1 --2- 172.21.15.172:0/2070004435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc599a0 0x55d59bc6c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.649+0000 7f2d98be5740 1 -- 172.21.15.172:0/2070004435 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d59bb65270 con 0x55d59bc71760 2024-09-10T09:05:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.649+0000 7f2d98be5740 1 -- 172.21.15.172:0/2070004435 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d59bb46290 con 0x55d59bc5f370 2024-09-10T09:05:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.649+0000 7f2d98be5740 1 -- 172.21.15.172:0/2070004435 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d59bc62100 con 0x55d59bc599a0 2024-09-10T09:05:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.649+0000 7f2d8ffff640 1 --2- 172.21.15.172:0/2070004435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc71760 0x55d59bc73b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.649+0000 7f2d98be5740 10 client.0 _ll_register_callbacks cb 0x55d59bb77140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.649+0000 7f2d8f7fe640 1 -- 172.21.15.172:0/2070004435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 msgr2=0x55d59bc6c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.649+0000 7f2d8f7fe640 1 --2- 172.21.15.172:0/2070004435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 0x55d59bc6c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.649+0000 7f2d8f7fe640 1 -- 172.21.15.172:0/2070004435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc71760 msgr2=0x55d59bc73b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.649+0000 7f2d8f7fe640 1 --2- 172.21.15.172:0/2070004435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc71760 0x55d59bc73b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.649+0000 7f2d8f7fe640 1 -- 172.21.15.172:0/2070004435 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d59b8d0210 con 0x55d59bc599a0 2024-09-10T09:05:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.650+0000 7f2d8f7fe640 1 --2- 172.21.15.172:0/2070004435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc599a0 0x55d59bc6c260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55d59bc55e90 tx=0x7f2d80004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6222770d80e3ff4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.650+0000 7f2d6effd640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:05:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.650+0000 7f2d6effd640 1 -- 172.21.15.172:0/2070004435 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8002b060 con 0x55d59bc599a0 2024-09-10T09:05:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.650+0000 7f2d6effd640 1 -- 172.21.15.172:0/2070004435 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2d8000e880 con 0x55d59bc599a0 2024-09-10T09:05:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.650+0000 7f2d98be5740 1 -- 172.21.15.172:0/2070004435 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d84039d00 con 0x55d59bc599a0 2024-09-10T09:05:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.650+0000 7f2d6effd640 1 -- 172.21.15.172:0/2070004435 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2d80032050 con 0x55d59bc599a0 2024-09-10T09:05:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.650+0000 7f2d98be5740 10 client.28786 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.650+0000 7f2d98be5740 1 -- 172.21.15.172:0/2070004435 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d59b99c620 con 0x55d59bc599a0 2024-09-10T09:05:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.650+0000 7f2d6effd640 1 -- 172.21.15.172:0/2070004435 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d80026420 con 0x55d59bc599a0 2024-09-10T09:05:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.651+0000 7f2d6effd640 1 -- 172.21.15.172:0/2070004435 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f2d8001a390 con 0x55d59bc599a0 2024-09-10T09:05:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.651+0000 7f2d98be5740 10 client.28786 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.651+0000 7f2d98be5740 10 client.28786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.651+0000 7f2d98be5740 20 client.28786 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.651+0000 7f2d98be5740 1 --2- 172.21.15.172:0/2070004435 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d59bc78270 0x55d59bc7a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.652+0000 7f2d98be5740 4 client.28786 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:05:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.652+0000 7f2d98be5740 1 -- 172.21.15.172:0/2070004435 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2d84039d00 con 0x55d59bc78270 2024-09-10T09:05:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.652+0000 7f2d8effd640 1 --2- 172.21.15.172:0/2070004435 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d59bc78270 0x55d59bc7a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.652+0000 7f2d8effd640 1 --2- 172.21.15.172:0/2070004435 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d59bc78270 0x55d59bc7a6e0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0d07fed4dbf4c4e server_cookie=364a860754214fa8 in_seq=0 out_seq=0 2024-09-10T09:05:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.652+0000 7f2d6effd640 10 client.28786 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.653+0000 7f2d6effd640 1 -- 172.21.15.172:0/2070004435 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f2d84039d00 con 0x55d59bc78270 2024-09-10T09:05:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.653+0000 7f2d6effd640 10 client.28786 handle_command_reply: tid=0 2024-09-10T09:05:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.653+0000 7f2d98be5740 1 client.28786 shutdown 2024-09-10T09:05:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.653+0000 7f2d98be5740 1 -- 172.21.15.172:0/2070004435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc599a0 msgr2=0x55d59bc6c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.653+0000 7f2d98be5740 1 --2- 172.21.15.172:0/2070004435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc599a0 0x55d59bc6c260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55d59bc55e90 tx=0x7f2d80004730 comp rx=0 tx=0).stop 2024-09-10T09:05:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.653+0000 7f2d98be5740 1 -- 172.21.15.172:0/2070004435 shutdown_connections 2024-09-10T09:05:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.653+0000 7f2d98be5740 1 --2- 172.21.15.172:0/2070004435 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d59bc78270 0x55d59bc7a6e0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.654+0000 7f2d98be5740 1 --2- 172.21.15.172:0/2070004435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc71760 0x55d59bc73b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.654+0000 7f2d6effd640 0 client.28786 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.654+0000 7f2d98be5740 1 --2- 172.21.15.172:0/2070004435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 0x55d59bc6c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.654+0000 7f2d98be5740 1 --2- 172.21.15.172:0/2070004435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc599a0 0x55d59bc6c260 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.654+0000 7f2d98be5740 1 -- 172.21.15.172:0/2070004435 >> 172.21.15.172:0/2070004435 conn(0x55d59bc55010 msgr2=0x55d59bc56820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.654+0000 7f2d98be5740 1 -- 172.21.15.172:0/2070004435 shutdown_connections 2024-09-10T09:05:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.654+0000 7f2d98be5740 1 -- 172.21.15.172:0/2070004435 wait complete. 2024-09-10T09:05:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.654+0000 7f2d98be5740 20 client.28786 trim_cache size 0 max 16384 2024-09-10T09:05:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.674+0000 7f2d98be5740 1 Processor -- start 2024-09-10T09:05:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d98be5740 1 -- start start 2024-09-10T09:05:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d98be5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc599a0 0x55d59bc540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d98be5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 0x55d59bc7efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d98be5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc71760 0x55d59bc7f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d8f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc599a0 0x55d59bc540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d8f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc599a0 0x55d59bc540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57548/0 (socket says 172.21.15.172:57548) 2024-09-10T09:05:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d8f7fe640 1 -- 172.21.15.172:0/3311660470 learned_addr learned my addr 172.21.15.172:0/3311660470 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d8ffff640 1 --2- 172.21.15.172:0/3311660470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc71760 0x55d59bc7f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d98be5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901a8440 con 0x55d59bc599a0 2024-09-10T09:05:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d98be5740 1 -- 172.21.15.172:0/3311660470 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901a9f80 con 0x55d59bc5f370 2024-09-10T09:05:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d98be5740 1 -- 172.21.15.172:0/3311660470 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901a8b10 con 0x55d59bc71760 2024-09-10T09:05:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d8effd640 1 --2- 172.21.15.172:0/3311660470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 0x55d59bc7efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d8f7fe640 1 -- 172.21.15.172:0/3311660470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc71760 msgr2=0x55d59bc7f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d8f7fe640 1 --2- 172.21.15.172:0/3311660470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc71760 0x55d59bc7f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d8f7fe640 1 -- 172.21.15.172:0/3311660470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 msgr2=0x55d59bc7efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d8f7fe640 1 --2- 172.21.15.172:0/3311660470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 0x55d59bc7efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d8f7fe640 1 -- 172.21.15.172:0/3311660470 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d59bc54e30 con 0x55d59bc599a0 2024-09-10T09:05:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.675+0000 7f2d8effd640 1 --2- 172.21.15.172:0/3311660470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 0x55d59bc7efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:05:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.676+0000 7f2d8f7fe640 1 --2- 172.21.15.172:0/3311660470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc599a0 0x55d59bc540b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f2d80005c20 tx=0x7f2d80005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b850732bf23398f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.676+0000 7f2d757fa640 1 -- 172.21.15.172:0/3311660470 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d80009d50 con 0x55d59bc599a0 2024-09-10T09:05:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.676+0000 7f2d757fa640 1 -- 172.21.15.172:0/3311660470 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d8001a390 con 0x55d59bc599a0 2024-09-10T09:05:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.676+0000 7f2d757fa640 1 -- 172.21.15.172:0/3311660470 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8000e9d0 con 0x55d59bc599a0 2024-09-10T09:05:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.677+0000 7f2d98be5740 1 -- 172.21.15.172:0/3311660470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc599a0 msgr2=0x55d59bc540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.677+0000 7f2d98be5740 1 --2- 172.21.15.172:0/3311660470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc599a0 0x55d59bc540b0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f2d80005c20 tx=0x7f2d80005c50 comp rx=0 tx=0).stop 2024-09-10T09:05:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.677+0000 7f2d98be5740 1 -- 172.21.15.172:0/3311660470 shutdown_connections 2024-09-10T09:05:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.677+0000 7f2d98be5740 1 --2- 172.21.15.172:0/3311660470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc71760 0x55d59bc7f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.677+0000 7f2d98be5740 1 --2- 172.21.15.172:0/3311660470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 0x55d59bc7efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.677+0000 7f2d98be5740 1 --2- 172.21.15.172:0/3311660470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc599a0 0x55d59bc540b0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.677+0000 7f2d98be5740 1 -- 172.21.15.172:0/3311660470 >> 172.21.15.172:0/3311660470 conn(0x55d59bc55010 msgr2=0x55d59bc539a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.677+0000 7f2d98be5740 1 -- 172.21.15.172:0/3311660470 shutdown_connections 2024-09-10T09:05:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.677+0000 7f2d98be5740 1 -- 172.21.15.172:0/3311660470 wait complete. 2024-09-10T09:05:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.677+0000 7f2d98be5740 1 -- start start 2024-09-10T09:05:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.677+0000 7f2d98be5740 1 Processor -- start 2024-09-10T09:05:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.677+0000 7f2d98be5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc599a0 0x55d59bc8a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.677+0000 7f2d98be5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 0x55d59bc8a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d98be5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc71760 0x55d59bc8b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d8f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc599a0 0x55d59bc8a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d8effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 0x55d59bc8a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d8f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc599a0 0x55d59bc8a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57554/0 (socket says 172.21.15.172:57554) 2024-09-10T09:05:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d8effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 0x55d59bc8a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:38348/0 (socket says 172.21.15.172:38348) 2024-09-10T09:05:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d8f7fe640 1 -- 172.21.15.172:0/2888837988 learned_addr learned my addr 172.21.15.172:0/2888837988 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d98be5740 1 -- 172.21.15.172:0/2888837988 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901a98b0 con 0x55d59bc599a0 2024-09-10T09:05:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d98be5740 1 -- 172.21.15.172:0/2888837988 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901a91e0 con 0x55d59bc5f370 2024-09-10T09:05:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d98be5740 1 -- 172.21.15.172:0/2888837988 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901a6420 con 0x55d59bc71760 2024-09-10T09:05:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d8ffff640 1 --2- 172.21.15.172:0/2888837988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc71760 0x55d59bc8b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d98be5740 10 client.0 _ll_register_callbacks cb 0x7f2d80009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d8f7fe640 1 -- 172.21.15.172:0/2888837988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc71760 msgr2=0x55d59bc8b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d8f7fe640 1 --2- 172.21.15.172:0/2888837988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc71760 0x55d59bc8b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d8f7fe640 1 -- 172.21.15.172:0/2888837988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 msgr2=0x55d59bc8a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d8f7fe640 1 --2- 172.21.15.172:0/2888837988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 0x55d59bc8a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d8f7fe640 1 -- 172.21.15.172:0/2888837988 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d59bc8bfe0 con 0x55d59bc599a0 2024-09-10T09:05:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d8f7fe640 1 --2- 172.21.15.172:0/2888837988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc599a0 0x55d59bc8a480 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f2d80005bf0 tx=0x7f2d8001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6c55802711d774 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d6effd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:05:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.678+0000 7f2d6effd640 1 -- 172.21.15.172:0/2888837988 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d80043020 con 0x55d59bc599a0 2024-09-10T09:05:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.679+0000 7f2d6effd640 1 -- 172.21.15.172:0/2888837988 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f2d80004250 con 0x55d59bc599a0 2024-09-10T09:05:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.679+0000 7f2d98be5740 1 -- 172.21.15.172:0/2888837988 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d8001a390 con 0x55d59bc599a0 2024-09-10T09:05:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.679+0000 7f2d6effd640 1 -- 172.21.15.172:0/2888837988 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2d8004b050 con 0x55d59bc599a0 2024-09-10T09:05:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.679+0000 7f2d98be5740 10 client.20181 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.679+0000 7f2d98be5740 1 -- 172.21.15.172:0/2888837988 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d59bce3560 con 0x55d59bc599a0 2024-09-10T09:05:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.679+0000 7f2d6effd640 1 -- 172.21.15.172:0/2888837988 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d80009e90 con 0x55d59bc599a0 2024-09-10T09:05:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.679+0000 7f2d6effd640 1 -- 172.21.15.172:0/2888837988 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f2d80003b20 con 0x55d59bc599a0 2024-09-10T09:05:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.679+0000 7f2d98be5740 10 client.20181 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.679+0000 7f2d98be5740 10 client.20181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.679+0000 7f2d98be5740 20 client.20181 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.679+0000 7f2d98be5740 1 --2- 172.21.15.172:0/2888837988 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d59bce3ee0 0x55d59bce6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.679+0000 7f2d98be5740 4 client.20181 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:05:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.679+0000 7f2d98be5740 1 -- 172.21.15.172:0/2888837988 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f2d8001a390 con 0x55d59bce3ee0 2024-09-10T09:05:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.680+0000 7f2d8effd640 1 --2- 172.21.15.172:0/2888837988 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d59bce3ee0 0x55d59bce6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.681+0000 7f2d8effd640 1 --2- 172.21.15.172:0/2888837988 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d59bce3ee0 0x55d59bce6300 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6363c705c9b10648 server_cookie=40529f94df8c6700 in_seq=0 out_seq=0 2024-09-10T09:05:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.681+0000 7f2d6effd640 10 client.20181 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.685+0000 7f2d6effd640 1 -- 172.21.15.172:0/2888837988 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f2d8001a390 con 0x55d59bce3ee0 2024-09-10T09:05:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.685+0000 7f2d6effd640 10 client.20181 handle_command_reply: tid=0 2024-09-10T09:05:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.685+0000 7f2d98be5740 1 client.20181 shutdown 2024-09-10T09:05:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.685+0000 7f2d98be5740 1 -- 172.21.15.172:0/2888837988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc599a0 msgr2=0x55d59bc8a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.685+0000 7f2d98be5740 1 --2- 172.21.15.172:0/2888837988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc599a0 0x55d59bc8a480 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f2d80005bf0 tx=0x7f2d8001ad10 comp rx=0 tx=0).stop 2024-09-10T09:05:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.685+0000 7f2d98be5740 1 -- 172.21.15.172:0/2888837988 shutdown_connections 2024-09-10T09:05:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.685+0000 7f2d98be5740 1 --2- 172.21.15.172:0/2888837988 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d59bce3ee0 0x55d59bce6300 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.685+0000 7f2d98be5740 1 --2- 172.21.15.172:0/2888837988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d59bc71760 0x55d59bc8b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.685+0000 7f2d6effd640 0 client.20181 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.685+0000 7f2d98be5740 1 --2- 172.21.15.172:0/2888837988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d59bc5f370 0x55d59bc8a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.685+0000 7f2d98be5740 1 --2- 172.21.15.172:0/2888837988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d59bc599a0 0x55d59bc8a480 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.685+0000 7f2d98be5740 1 -- 172.21.15.172:0/2888837988 >> 172.21.15.172:0/2888837988 conn(0x55d59bc55010 msgr2=0x55d59bc56680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.685+0000 7f2d98be5740 1 -- 172.21.15.172:0/2888837988 shutdown_connections 2024-09-10T09:05:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.685+0000 7f2d98be5740 1 -- 172.21.15.172:0/2888837988 wait complete. 2024-09-10T09:05:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.685+0000 7f2d98be5740 20 client.20181 trim_cache size 0 max 16384 2024-09-10T09:05:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:05:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:05:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:05:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:05:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.686+0000 7f2d95d6d640 1 -- 172.21.15.172:0/3375609950 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2d780a2c30 msgr2=0x7f2d780a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.686+0000 7f2d95d6d640 1 --2- 172.21.15.172:0/3375609950 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2d780a2c30 0x7f2d780a50f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2d800058c0 tx=0x7f2d80005f90 comp rx=0 tx=0).stop 2024-09-10T09:05:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.686+0000 7f2d95d6d640 1 -- 172.21.15.172:0/3375609950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d90101670 msgr2=0x7f2d9019cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.686+0000 7f2d95d6d640 1 --2- 172.21.15.172:0/3375609950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d90101670 0x7f2d9019cad0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009230 tx=0x7f2d84004270 comp rx=0 tx=0).stop 2024-09-10T09:05:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.686+0000 7f2d95d6d640 1 -- 172.21.15.172:0/3375609950 shutdown_connections 2024-09-10T09:05:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.686+0000 7f2d95d6d640 1 --2- 172.21.15.172:0/3375609950 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f2d780a2c30 0x7f2d780a50f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.686+0000 7f2d95d6d640 1 --2- 172.21.15.172:0/3375609950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d90102060 0x7f2d9019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.686+0000 7f2d95d6d640 1 --2- 172.21.15.172:0/3375609950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d90101670 0x7f2d9019cad0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.686+0000 7f2d95d6d640 1 --2- 172.21.15.172:0/3375609950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d90100cb0 0x7f2d90100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.686+0000 7f2d95d6d640 1 -- 172.21.15.172:0/3375609950 >> 172.21.15.172:0/3375609950 conn(0x7f2d900fb930 msgr2=0x7f2d90105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.686+0000 7f2d95d6d640 1 -- 172.21.15.172:0/3375609950 shutdown_connections 2024-09-10T09:05:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:05:47.686+0000 7f2d95d6d640 1 -- 172.21.15.172:0/3375609950 wait complete. 2024-09-10T09:05:47.990 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:05:48.292 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:05:48.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.792+0000 7f47adf46640 1 -- 172.21.15.172:0/1018129689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47a8100a90 msgr2=0x7f47a8100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:48.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.792+0000 7f47adf46640 1 --2- 172.21.15.172:0/1018129689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47a8100a90 0x7f47a8100ef0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f479c009930 tx=0x7f479c02a440 comp rx=0 tx=0).stop 2024-09-10T09:05:48.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.792+0000 7f47adf46640 1 -- 172.21.15.172:0/1018129689 shutdown_connections 2024-09-10T09:05:48.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.792+0000 7f47adf46640 1 --2- 172.21.15.172:0/1018129689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a8101430 0x7f47a810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.792+0000 7f47adf46640 1 --2- 172.21.15.172:0/1018129689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47a8100a90 0x7f47a8100ef0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.792+0000 7f47adf46640 1 --2- 172.21.15.172:0/1018129689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47a8106a90 0x7f47a8106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.792+0000 7f47adf46640 1 -- 172.21.15.172:0/1018129689 >> 172.21.15.172:0/1018129689 conn(0x7f47a80fc7d0 msgr2=0x7f47a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:48.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.792+0000 7f47adf46640 1 -- 172.21.15.172:0/1018129689 shutdown_connections 2024-09-10T09:05:48.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.792+0000 7f47adf46640 1 -- 172.21.15.172:0/1018129689 wait complete. 2024-09-10T09:05:48.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.793+0000 7f47adf46640 1 Processor -- start 2024-09-10T09:05:48.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.793+0000 7f47adf46640 1 -- start start 2024-09-10T09:05:48.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.793+0000 7f47adf46640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47a8100a90 0x7f47a8198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.793+0000 7f47adf46640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47a8101430 0x7f47a81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.793+0000 7f47adf46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a8106a90 0x7f47a819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.793+0000 7f47adf46640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8111e90 con 0x7f47a8101430 2024-09-10T09:05:48.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.793+0000 7f47adf46640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8111d10 con 0x7f47a8106a90 2024-09-10T09:05:48.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.793+0000 7f47adf46640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8112010 con 0x7f47a8100a90 2024-09-10T09:05:48.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.793+0000 7f47a6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47a8101430 0x7f47a81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:48.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.793+0000 7f47a77fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47a8100a90 0x7f47a8198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:48.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.793+0000 7f47a6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47a8101430 0x7f47a81989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57568/0 (socket says 172.21.15.172:57568) 2024-09-10T09:05:48.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.793+0000 7f47a77fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47a8100a90 0x7f47a8198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33270/0 (socket says 172.21.15.172:33270) 2024-09-10T09:05:48.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.793+0000 7f47a6ffd640 1 -- 172.21.15.172:0/390763287 learned_addr learned my addr 172.21.15.172:0/390763287 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:48.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.793+0000 7f47a7fff640 1 --2- 172.21.15.172:0/390763287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a8106a90 0x7f47a819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:48.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.794+0000 7f47a7fff640 1 -- 172.21.15.172:0/390763287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47a8100a90 msgr2=0x7f47a8198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:48.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.794+0000 7f47a7fff640 1 --2- 172.21.15.172:0/390763287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47a8100a90 0x7f47a8198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.794+0000 7f47a7fff640 1 -- 172.21.15.172:0/390763287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47a8101430 msgr2=0x7f47a81989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:48.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.794+0000 7f47a7fff640 1 --2- 172.21.15.172:0/390763287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47a8101430 0x7f47a81989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.794+0000 7f47a7fff640 1 -- 172.21.15.172:0/390763287 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a819d420 con 0x7f47a8106a90 2024-09-10T09:05:48.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.794+0000 7f47a77fe640 1 --2- 172.21.15.172:0/390763287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47a8100a90 0x7f47a8198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:48.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.794+0000 7f47a7fff640 1 --2- 172.21.15.172:0/390763287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a8106a90 0x7f47a819cd40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009fd0 tx=0x7f47a000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:48.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.794+0000 7f47a4ff9640 1 -- 172.21.15.172:0/390763287 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a000ce20 con 0x7f47a8106a90 2024-09-10T09:05:48.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.794+0000 7f47adf46640 1 -- 172.21.15.172:0/390763287 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a819d710 con 0x7f47a8106a90 2024-09-10T09:05:48.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.794+0000 7f47a4ff9640 1 -- 172.21.15.172:0/390763287 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47a0005020 con 0x7f47a8106a90 2024-09-10T09:05:48.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.794+0000 7f47a4ff9640 1 -- 172.21.15.172:0/390763287 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a0010690 con 0x7f47a8106a90 2024-09-10T09:05:48.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.794+0000 7f47adf46640 1 -- 172.21.15.172:0/390763287 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a81a4ff0 con 0x7f47a8106a90 2024-09-10T09:05:48.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.797+0000 7f47b031f740 1 Processor -- start 2024-09-10T09:05:48.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.797+0000 7f47a4ff9640 1 -- 172.21.15.172:0/390763287 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f47a00051d0 con 0x7f47a8106a90 2024-09-10T09:05:48.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.797+0000 7f47b031f740 1 -- start start 2024-09-10T09:05:48.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47b031f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f5b40 0x55be4e9f4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47b031f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f45d0 0x55be4e9f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47b031f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4ea03780 0x55be4ea05b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47b031f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55be4e95a2f0 con 0x55be4e9f45d0 2024-09-10T09:05:48.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47b031f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55be4e901cf0 con 0x55be4e9f5b40 2024-09-10T09:05:48.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47b031f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55be4e9562b0 con 0x55be4ea03780 2024-09-10T09:05:48.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47a77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f45d0 0x55be4e9f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:48.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47a77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f45d0 0x55be4e9f4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57578/0 (socket says 172.21.15.172:57578) 2024-09-10T09:05:48.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47a77fe640 1 -- 172.21.15.172:0/2394055502 learned_addr learned my addr 172.21.15.172:0/2394055502 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:48.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47a6ffd640 1 --2- 172.21.15.172:0/2394055502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f5b40 0x55be4e9f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:48.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47a4ff9640 1 --2- 172.21.15.172:0/390763287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f47900a2b80 0x7f47900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47a77fe640 1 -- 172.21.15.172:0/2394055502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4ea03780 msgr2=0x55be4ea05b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:48.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47a77fe640 1 --2- 172.21.15.172:0/2394055502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4ea03780 0x55be4ea05b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47a77fe640 1 -- 172.21.15.172:0/2394055502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f5b40 msgr2=0x55be4e9f4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:48.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47a77fe640 1 --2- 172.21.15.172:0/2394055502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f5b40 0x55be4e9f4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47a77fe640 1 -- 172.21.15.172:0/2394055502 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be4e8e63a0 con 0x55be4e9f45d0 2024-09-10T09:05:48.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47a7fff640 1 --2- 172.21.15.172:0/390763287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f47900a2b80 0x7f47900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:48.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.798+0000 7f47a4ff9640 1 -- 172.21.15.172:0/390763287 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f47a00cfa60 con 0x7f47a8106a90 2024-09-10T09:05:48.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.799+0000 7f47a77fe640 1 --2- 172.21.15.172:0/2394055502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f45d0 0x55be4e9f4a10 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f4798009df0 tx=0x7f479800e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b74dd48dd0501fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:48.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.799+0000 7f478dffb640 1 -- 172.21.15.172:0/2394055502 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798017070 con 0x55be4e9f45d0 2024-09-10T09:05:48.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.799+0000 7f478dffb640 1 -- 172.21.15.172:0/2394055502 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4798013920 con 0x55be4e9f45d0 2024-09-10T09:05:48.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.799+0000 7f478dffb640 1 -- 172.21.15.172:0/2394055502 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47980127f0 con 0x55be4e9f45d0 2024-09-10T09:05:48.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.800+0000 7f47b031f740 1 -- 172.21.15.172:0/2394055502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f45d0 msgr2=0x55be4e9f4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:48.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.800+0000 7f47b031f740 1 --2- 172.21.15.172:0/2394055502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f45d0 0x55be4e9f4a10 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f4798009df0 tx=0x7f479800e5a0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.800+0000 7f47a7fff640 1 --2- 172.21.15.172:0/390763287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f47900a2b80 0x7f47900a5040 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f47a00a46a0 tx=0x7f47a0013b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:48.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.800+0000 7f47b031f740 1 -- 172.21.15.172:0/2394055502 shutdown_connections 2024-09-10T09:05:48.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.800+0000 7f47b031f740 1 --2- 172.21.15.172:0/2394055502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4ea03780 0x55be4ea05b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.800+0000 7f47b031f740 1 --2- 172.21.15.172:0/2394055502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f45d0 0x55be4e9f4a10 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.800+0000 7f47b031f740 1 --2- 172.21.15.172:0/2394055502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f5b40 0x55be4e9f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.800+0000 7f47b031f740 1 -- 172.21.15.172:0/2394055502 >> 172.21.15.172:0/2394055502 conn(0x55be4e9bd310 msgr2=0x55be4e9f0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:48.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.800+0000 7f47b031f740 1 -- 172.21.15.172:0/2394055502 shutdown_connections 2024-09-10T09:05:48.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.800+0000 7f47b031f740 1 -- 172.21.15.172:0/2394055502 wait complete. 2024-09-10T09:05:48.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.800+0000 7f47b031f740 1 -- start start 2024-09-10T09:05:48.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.800+0000 7f47b031f740 1 Processor -- start 2024-09-10T09:05:48.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47b031f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f45d0 0x55be4ea030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47b031f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4e9f5b40 0x55be4ea003a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47b031f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4ea03780 0x55be4ea00900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47b031f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55be4ea06480 con 0x55be4e9f45d0 2024-09-10T09:05:48.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47b031f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55be4e915060 con 0x55be4ea03780 2024-09-10T09:05:48.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47b031f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55be4ea06600 con 0x55be4e9f5b40 2024-09-10T09:05:48.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47a77fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4e9f5b40 0x55be4ea003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:48.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47a6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f45d0 0x55be4ea030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:48.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47a77fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4e9f5b40 0x55be4ea003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33294/0 (socket says 172.21.15.172:33294) 2024-09-10T09:05:48.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47a77fe640 1 -- 172.21.15.172:0/875894345 learned_addr learned my addr 172.21.15.172:0/875894345 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:48.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47b031f740 10 client.0 _ll_register_callbacks cb 0x55be4e6688c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:48.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47a77fe640 1 -- 172.21.15.172:0/875894345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4ea03780 msgr2=0x55be4ea00900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:48.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47a6ffd640 1 --2- 172.21.15.172:0/875894345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4ea03780 0x55be4ea00900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:48.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47a77fe640 1 --2- 172.21.15.172:0/875894345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4ea03780 0x55be4ea00900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47a77fe640 1 -- 172.21.15.172:0/875894345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f45d0 msgr2=0x55be4ea030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:48.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47a77fe640 1 --2- 172.21.15.172:0/875894345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f45d0 0x55be4ea030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47a77fe640 1 -- 172.21.15.172:0/875894345 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be4e6cb3c0 con 0x55be4e9f5b40 2024-09-10T09:05:48.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47a6ffd640 1 --2- 172.21.15.172:0/875894345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4ea03780 0x55be4ea00900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:05:48.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f47a77fe640 1 --2- 172.21.15.172:0/875894345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4e9f5b40 0x55be4ea003a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4798012410 tx=0x7f4798014d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=666d708f1c3d67a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:48.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.801+0000 7f4786ffd640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:05:48.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.802+0000 7f4786ffd640 1 -- 172.21.15.172:0/875894345 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798029060 con 0x55be4e9f5b40 2024-09-10T09:05:48.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.802+0000 7f4786ffd640 1 -- 172.21.15.172:0/875894345 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4798004450 con 0x55be4e9f5b40 2024-09-10T09:05:48.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.802+0000 7f47b031f740 1 -- 172.21.15.172:0/875894345 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4798013920 con 0x55be4e9f5b40 2024-09-10T09:05:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.802+0000 7f4786ffd640 1 -- 172.21.15.172:0/875894345 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4798017050 con 0x55be4e9f5b40 2024-09-10T09:05:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.802+0000 7f47b031f740 10 client.28798 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.802+0000 7f47b031f740 1 -- 172.21.15.172:0/875894345 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55be4ea11eb0 con 0x55be4e9f5b40 2024-09-10T09:05:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.803+0000 7f4786ffd640 1 -- 172.21.15.172:0/875894345 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4798013920 con 0x55be4e9f5b40 2024-09-10T09:05:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.803+0000 7f4786ffd640 1 -- 172.21.15.172:0/875894345 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f4798012650 con 0x55be4e9f5b40 2024-09-10T09:05:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.803+0000 7f47b031f740 10 client.28798 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.803+0000 7f47b031f740 10 client.28798 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:05:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.803+0000 7f47b031f740 20 client.28798 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.803+0000 7f47b031f740 1 --2- 172.21.15.172:0/875894345 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55be4ea13200 0x55be4ea155f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.803+0000 7f47b031f740 4 client.28798 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:05:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.803+0000 7f47b031f740 1 -- 172.21.15.172:0/875894345 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be4e967860 con 0x55be4ea13200 2024-09-10T09:05:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.803+0000 7f47a6ffd640 1 --2- 172.21.15.172:0/875894345 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55be4ea13200 0x55be4ea155f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:48.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.804+0000 7f4786ffd640 10 client.28798 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T09:05:48.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.804+0000 7f47a6ffd640 1 --2- 172.21.15.172:0/875894345 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55be4ea13200 0x55be4ea155f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b699966560825ad server_cookie=49252e36ab6690e1 in_seq=0 out_seq=0 2024-09-10T09:05:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.804+0000 7f4786ffd640 1 -- 172.21.15.172:0/875894345 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be4e967860 con 0x55be4ea13200 2024-09-10T09:05:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.804+0000 7f4786ffd640 10 client.28798 handle_command_reply: tid=0 2024-09-10T09:05:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.804+0000 7f47b031f740 1 client.28798 shutdown 2024-09-10T09:05:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.805+0000 7f47b031f740 1 -- 172.21.15.172:0/875894345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4e9f5b40 msgr2=0x55be4ea003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.805+0000 7f47b031f740 1 --2- 172.21.15.172:0/875894345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4e9f5b40 0x55be4ea003a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4798012410 tx=0x7f4798014d90 comp rx=0 tx=0).stop 2024-09-10T09:05:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.805+0000 7f47b031f740 1 -- 172.21.15.172:0/875894345 shutdown_connections 2024-09-10T09:05:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.805+0000 7f47b031f740 1 --2- 172.21.15.172:0/875894345 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55be4ea13200 0x55be4ea155f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.805+0000 7f47b031f740 1 --2- 172.21.15.172:0/875894345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4ea03780 0x55be4ea00900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.805+0000 7f47b031f740 1 --2- 172.21.15.172:0/875894345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4e9f5b40 0x55be4ea003a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.805+0000 7f47b031f740 1 --2- 172.21.15.172:0/875894345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f45d0 0x55be4ea030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.805+0000 7f47b031f740 1 -- 172.21.15.172:0/875894345 >> 172.21.15.172:0/875894345 conn(0x55be4e9bd310 msgr2=0x55be4e9f6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.805+0000 7f4786ffd640 0 client.28798 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T09:05:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.805+0000 7f47b031f740 1 -- 172.21.15.172:0/875894345 shutdown_connections 2024-09-10T09:05:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.805+0000 7f47b031f740 1 -- 172.21.15.172:0/875894345 wait complete. 2024-09-10T09:05:48.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.805+0000 7f47b031f740 20 client.28798 trim_cache size 0 max 16384 2024-09-10T09:05:48.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.820+0000 7f47b031f740 1 Processor -- start 2024-09-10T09:05:48.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.820+0000 7f47b031f740 1 -- start start 2024-09-10T09:05:48.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.821+0000 7f47b031f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f45d0 0x55be4ea0f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.821+0000 7f47b031f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4e9f5b40 0x55be4ea02310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.821+0000 7f47b031f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4ea03780 0x55be4ea1ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.821+0000 7f47b031f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81ac990 con 0x55be4ea03780 2024-09-10T09:05:48.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.821+0000 7f47b031f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81ae4d0 con 0x55be4e9f45d0 2024-09-10T09:05:48.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.821+0000 7f47b031f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81ad060 con 0x55be4e9f5b40 2024-09-10T09:05:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.821+0000 7f47a6ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f45d0 0x55be4ea0f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.821+0000 7f47a77fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4e9f5b40 0x55be4ea02310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.821+0000 7f47a6ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f45d0 0x55be4ea0f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:38412/0 (socket says 172.21.15.172:38412) 2024-09-10T09:05:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.821+0000 7f47a77fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4e9f5b40 0x55be4ea02310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33302/0 (socket says 172.21.15.172:33302) 2024-09-10T09:05:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.821+0000 7f47a6ffd640 1 -- 172.21.15.172:0/3067578949 learned_addr learned my addr 172.21.15.172:0/3067578949 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.821+0000 7f47a6ffd640 1 --2- 172.21.15.172:0/3067578949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4ea03780 0x55be4ea1ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.821+0000 7f47a6ffd640 1 -- 172.21.15.172:0/3067578949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4e9f5b40 msgr2=0x55be4ea02310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.821+0000 7f47a6ffd640 1 --2- 172.21.15.172:0/3067578949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4e9f5b40 0x55be4ea02310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.821+0000 7f47a6ffd640 1 -- 172.21.15.172:0/3067578949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4ea03780 msgr2=0x55be4ea1ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:48.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.821+0000 7f47a6ffd640 1 --2- 172.21.15.172:0/3067578949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4ea03780 0x55be4ea1ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.821+0000 7f47a6ffd640 1 -- 172.21.15.172:0/3067578949 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be4e9ef310 con 0x55be4e9f45d0 2024-09-10T09:05:48.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.822+0000 7f47a6ffd640 1 --2- 172.21.15.172:0/3067578949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f45d0 0x55be4ea0f780 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f479c02a480 tx=0x7f479c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a85d0fff4a18d699 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:48.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.822+0000 7f478d7fa640 1 -- 172.21.15.172:0/3067578949 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c035040 con 0x55be4e9f45d0 2024-09-10T09:05:48.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.822+0000 7f478d7fa640 1 -- 172.21.15.172:0/3067578949 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f479c004c50 con 0x55be4e9f45d0 2024-09-10T09:05:48.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.822+0000 7f478d7fa640 1 -- 172.21.15.172:0/3067578949 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c005310 con 0x55be4e9f45d0 2024-09-10T09:05:48.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.823+0000 7f47b031f740 1 -- 172.21.15.172:0/3067578949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f45d0 msgr2=0x55be4ea0f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:48.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.823+0000 7f47b031f740 1 --2- 172.21.15.172:0/3067578949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f45d0 0x55be4ea0f780 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f479c02a480 tx=0x7f479c009260 comp rx=0 tx=0).stop 2024-09-10T09:05:48.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.823+0000 7f47b031f740 1 -- 172.21.15.172:0/3067578949 shutdown_connections 2024-09-10T09:05:48.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.823+0000 7f47b031f740 1 --2- 172.21.15.172:0/3067578949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4ea03780 0x55be4ea1ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.823+0000 7f47b031f740 1 --2- 172.21.15.172:0/3067578949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4e9f5b40 0x55be4ea02310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.823+0000 7f47b031f740 1 --2- 172.21.15.172:0/3067578949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f45d0 0x55be4ea0f780 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.823+0000 7f47b031f740 1 -- 172.21.15.172:0/3067578949 >> 172.21.15.172:0/3067578949 conn(0x55be4e9bd310 msgr2=0x55be4ea01c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:48.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.823+0000 7f47b031f740 1 -- 172.21.15.172:0/3067578949 shutdown_connections 2024-09-10T09:05:48.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.823+0000 7f47b031f740 1 -- 172.21.15.172:0/3067578949 wait complete. 2024-09-10T09:05:48.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.823+0000 7f47b031f740 1 -- start start 2024-09-10T09:05:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.824+0000 7f47b031f740 1 Processor -- start 2024-09-10T09:05:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.824+0000 7f47b031f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f45d0 0x55be4e9fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.824+0000 7f47b031f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f5b40 0x55be4e9fd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.824+0000 7f47b031f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4ea03780 0x55be4e9fde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.824+0000 7f47b031f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81ade00 con 0x55be4e9f5b40 2024-09-10T09:05:48.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.824+0000 7f47b031f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81ad730 con 0x55be4e9f45d0 2024-09-10T09:05:48.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.824+0000 7f47b031f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81aa970 con 0x55be4ea03780 2024-09-10T09:05:48.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.824+0000 7f47b031f740 10 client.0 _ll_register_callbacks cb 0x7f4798009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:48.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.824+0000 7f47a6ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f45d0 0x55be4e9fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:48.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.824+0000 7f47a6ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f45d0 0x55be4e9fd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:38422/0 (socket says 172.21.15.172:38422) 2024-09-10T09:05:48.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.824+0000 7f47a6ffd640 1 -- 172.21.15.172:0/1731275252 learned_addr learned my addr 172.21.15.172:0/1731275252 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:48.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.824+0000 7f47a6ffd640 1 --2- 172.21.15.172:0/1731275252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4ea03780 0x55be4e9fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:48.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.824+0000 7f47a77fe640 1 --2- 172.21.15.172:0/1731275252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f5b40 0x55be4e9fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:48.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.825+0000 7f47a77fe640 1 -- 172.21.15.172:0/1731275252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4ea03780 msgr2=0x55be4e9fde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:48.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.825+0000 7f47a77fe640 1 --2- 172.21.15.172:0/1731275252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4ea03780 0x55be4e9fde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.825+0000 7f47a77fe640 1 -- 172.21.15.172:0/1731275252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f45d0 msgr2=0x55be4e9fd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:48.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.825+0000 7f47a77fe640 1 --2- 172.21.15.172:0/1731275252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f45d0 0x55be4e9fd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.825+0000 7f47a77fe640 1 -- 172.21.15.172:0/1731275252 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be4e9f1960 con 0x55be4e9f5b40 2024-09-10T09:05:48.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.825+0000 7f47a77fe640 1 --2- 172.21.15.172:0/1731275252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f5b40 0x55be4e9fd900 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f4798012c10 tx=0x7f4798009170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91a34616a63d3be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:48.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.825+0000 7f4786ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:05:48.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.825+0000 7f4786ffd640 1 -- 172.21.15.172:0/1731275252 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798040060 con 0x55be4e9f5b40 2024-09-10T09:05:48.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.825+0000 7f4786ffd640 1 -- 172.21.15.172:0/1731275252 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f4798004460 con 0x55be4e9f5b40 2024-09-10T09:05:48.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.825+0000 7f47b031f740 1 -- 172.21.15.172:0/1731275252 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f479c004c50 con 0x55be4e9f5b40 2024-09-10T09:05:48.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.826+0000 7f4786ffd640 1 -- 172.21.15.172:0/1731275252 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4798029070 con 0x55be4e9f5b40 2024-09-10T09:05:48.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.826+0000 7f47b031f740 10 client.20199 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:48.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.826+0000 7f47b031f740 1 -- 172.21.15.172:0/1731275252 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55be4e9f7310 con 0x55be4e9f5b40 2024-09-10T09:05:48.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.826+0000 7f4786ffd640 1 -- 172.21.15.172:0/1731275252 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f479803ba30 con 0x55be4e9f5b40 2024-09-10T09:05:48.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.826+0000 7f4786ffd640 1 -- 172.21.15.172:0/1731275252 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f4798005bc0 con 0x55be4e9f5b40 2024-09-10T09:05:48.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.826+0000 7f47b031f740 10 client.20199 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:48.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.826+0000 7f47b031f740 10 client.20199 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:05:48.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.826+0000 7f47b031f740 20 client.20199 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:48.831 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.826+0000 7f47b031f740 1 --2- 172.21.15.172:0/1731275252 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55be4e9f7a50 0x55be4e9f9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.826+0000 7f47b031f740 4 client.20199 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:05:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.826+0000 7f47b031f740 1 -- 172.21.15.172:0/1731275252 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f479c004c50 con 0x55be4e9f7a50 2024-09-10T09:05:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.826+0000 7f47a6ffd640 1 --2- 172.21.15.172:0/1731275252 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55be4e9f7a50 0x55be4e9f9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.827+0000 7f4786ffd640 10 client.20199 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T09:05:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.827+0000 7f47a6ffd640 1 --2- 172.21.15.172:0/1731275252 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55be4e9f7a50 0x55be4e9f9e70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac96a29978c24b18 server_cookie=bab485d2d8ac74b2 in_seq=0 out_seq=0 2024-09-10T09:05:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.827+0000 7f4786ffd640 1 -- 172.21.15.172:0/1731275252 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f479c004c50 con 0x55be4e9f7a50 2024-09-10T09:05:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.827+0000 7f4786ffd640 10 client.20199 handle_command_reply: tid=0 2024-09-10T09:05:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.827+0000 7f47b031f740 1 client.20199 shutdown 2024-09-10T09:05:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.827+0000 7f47b031f740 1 -- 172.21.15.172:0/1731275252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f5b40 msgr2=0x55be4e9fd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.827+0000 7f47b031f740 1 --2- 172.21.15.172:0/1731275252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f5b40 0x55be4e9fd900 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f4798012c10 tx=0x7f4798009170 comp rx=0 tx=0).stop 2024-09-10T09:05:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.827+0000 7f47b031f740 1 -- 172.21.15.172:0/1731275252 shutdown_connections 2024-09-10T09:05:48.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.827+0000 7f47b031f740 1 --2- 172.21.15.172:0/1731275252 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x55be4e9f7a50 0x55be4e9f9e70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.827+0000 7f47b031f740 1 --2- 172.21.15.172:0/1731275252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be4ea03780 0x55be4e9fde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.827+0000 7f4786ffd640 0 client.20199 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T09:05:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.827+0000 7f47b031f740 1 --2- 172.21.15.172:0/1731275252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be4e9f5b40 0x55be4e9fd900 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.827+0000 7f47b031f740 1 --2- 172.21.15.172:0/1731275252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be4e9f45d0 0x55be4e9fd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.827+0000 7f47b031f740 1 -- 172.21.15.172:0/1731275252 >> 172.21.15.172:0/1731275252 conn(0x55be4e9bd310 msgr2=0x55be4ea02420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.828+0000 7f47b031f740 1 -- 172.21.15.172:0/1731275252 shutdown_connections 2024-09-10T09:05:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.828+0000 7f47b031f740 1 -- 172.21.15.172:0/1731275252 wait complete. 2024-09-10T09:05:48.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.828+0000 7f47b031f740 20 client.20199 trim_cache size 0 max 16384 2024-09-10T09:05:48.833 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:05:48.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.828+0000 7f47adf46640 1 -- 172.21.15.172:0/390763287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f47900a2b80 msgr2=0x7f47900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:48.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.828+0000 7f47adf46640 1 --2- 172.21.15.172:0/390763287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f47900a2b80 0x7f47900a5040 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f47a00a46a0 tx=0x7f47a0013b60 comp rx=0 tx=0).stop 2024-09-10T09:05:48.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.829+0000 7f47adf46640 1 -- 172.21.15.172:0/390763287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a8106a90 msgr2=0x7f47a819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:48.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.829+0000 7f47adf46640 1 --2- 172.21.15.172:0/390763287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a8106a90 0x7f47a819cd40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009fd0 tx=0x7f47a000ef10 comp rx=0 tx=0).stop 2024-09-10T09:05:48.834 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.829+0000 7f47adf46640 1 -- 172.21.15.172:0/390763287 shutdown_connections 2024-09-10T09:05:48.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.829+0000 7f47adf46640 1 --2- 172.21.15.172:0/390763287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f47900a2b80 0x7f47900a5040 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.829+0000 7f47adf46640 1 --2- 172.21.15.172:0/390763287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a8106a90 0x7f47a819cd40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.829+0000 7f47adf46640 1 --2- 172.21.15.172:0/390763287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47a8101430 0x7f47a81989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.829+0000 7f47adf46640 1 --2- 172.21.15.172:0/390763287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47a8100a90 0x7f47a8198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:48.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.829+0000 7f47adf46640 1 -- 172.21.15.172:0/390763287 >> 172.21.15.172:0/390763287 conn(0x7f47a80fc7d0 msgr2=0x7f47a80fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:48.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.829+0000 7f47adf46640 1 -- 172.21.15.172:0/390763287 shutdown_connections 2024-09-10T09:05:48.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:48.829+0000 7f47adf46640 1 -- 172.21.15.172:0/390763287 wait complete. 2024-09-10T09:05:48.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:48 smithi174 ceph-mon[34068]: pgmap v937: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 864 KiB/s wr, 227 op/s 2024-09-10T09:05:48.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:48 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:05:49.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:48 smithi191 ceph-mon[35314]: pgmap v937: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 864 KiB/s wr, 227 op/s 2024-09-10T09:05:49.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:48 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:05:49.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:48 smithi172 ceph-mon[30854]: pgmap v937: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 864 KiB/s wr, 227 op/s 2024-09-10T09:05:49.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:48 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:05:49.595 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:05:49.917 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:05:50.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.453+0000 7fe1bc25f640 1 -- 172.21.15.172:0/1721202423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1b4100e20 msgr2=0x7fe1b4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:50.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.453+0000 7fe1bc25f640 1 --2- 172.21.15.172:0/1721202423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1b4100e20 0x7fe1b4101200 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0009900 tx=0x7fe1b002a480 comp rx=0 tx=0).stop 2024-09-10T09:05:50.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.453+0000 7fe1bc25f640 1 -- 172.21.15.172:0/1721202423 shutdown_connections 2024-09-10T09:05:50.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.453+0000 7fe1bc25f640 1 --2- 172.21.15.172:0/1721202423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1b4102ef0 0x7fe1b410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.453+0000 7fe1bc25f640 1 --2- 172.21.15.172:0/1721202423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1b4102570 0x7fe1b41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.453+0000 7fe1bc25f640 1 --2- 172.21.15.172:0/1721202423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1b4100e20 0x7fe1b4101200 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.453+0000 7fe1bc25f640 1 -- 172.21.15.172:0/1721202423 >> 172.21.15.172:0/1721202423 conn(0x7fe1b40fc7d0 msgr2=0x7fe1b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:50.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.453+0000 7fe1bc25f640 1 -- 172.21.15.172:0/1721202423 shutdown_connections 2024-09-10T09:05:50.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.453+0000 7fe1bc25f640 1 -- 172.21.15.172:0/1721202423 wait complete. 2024-09-10T09:05:50.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.454+0000 7fe1bc25f640 1 Processor -- start 2024-09-10T09:05:50.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.454+0000 7fe1bc25f640 1 -- start start 2024-09-10T09:05:50.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.454+0000 7fe1bc25f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1b4100e20 0x7fe1b4199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.454+0000 7fe1bc25f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1b4102570 0x7fe1b4199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.455+0000 7fe1bc25f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1b4102ef0 0x7fe1b41a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.455+0000 7fe1bc25f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b4110e30 con 0x7fe1b4102570 2024-09-10T09:05:50.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.455+0000 7fe1bc25f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b4110cb0 con 0x7fe1b4102ef0 2024-09-10T09:05:50.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.455+0000 7fe1bc25f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b4110fb0 con 0x7fe1b4100e20 2024-09-10T09:05:50.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.455+0000 7fe1b97fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1b4102570 0x7fe1b4199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:50.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.455+0000 7fe1b9ffb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1b4100e20 0x7fe1b4199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:50.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.455+0000 7fe1b9ffb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1b4100e20 0x7fe1b4199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33336/0 (socket says 172.21.15.172:33336) 2024-09-10T09:05:50.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.455+0000 7fe1b97fa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1b4102570 0x7fe1b4199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57608/0 (socket says 172.21.15.172:57608) 2024-09-10T09:05:50.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.455+0000 7fe1b97fa640 1 -- 172.21.15.172:0/3946502971 learned_addr learned my addr 172.21.15.172:0/3946502971 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:50.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.455+0000 7fe1ba7fc640 1 --2- 172.21.15.172:0/3946502971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1b4102ef0 0x7fe1b41a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:50.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.455+0000 7fe1b9ffb640 1 -- 172.21.15.172:0/3946502971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1b4102ef0 msgr2=0x7fe1b41a42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:50.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.455+0000 7fe1b9ffb640 1 --2- 172.21.15.172:0/3946502971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1b4102ef0 0x7fe1b41a42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.455+0000 7fe1b9ffb640 1 -- 172.21.15.172:0/3946502971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1b4102570 msgr2=0x7fe1b4199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:50.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.455+0000 7fe1b9ffb640 1 --2- 172.21.15.172:0/3946502971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1b4102570 0x7fe1b4199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.455+0000 7fe1b9ffb640 1 -- 172.21.15.172:0/3946502971 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b41a47e0 con 0x7fe1b4100e20 2024-09-10T09:05:50.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.455+0000 7fe1b97fa640 1 --2- 172.21.15.172:0/3946502971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1b4102570 0x7fe1b4199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:50.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.455+0000 7fe1b9ffb640 1 --2- 172.21.15.172:0/3946502971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1b4100e20 0x7fe1b4199220 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0005940 tx=0x7fe1b0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:50.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.456+0000 7fe1aaffd640 1 -- 172.21.15.172:0/3946502971 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b003f030 con 0x7fe1b4100e20 2024-09-10T09:05:50.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.456+0000 7fe1aaffd640 1 -- 172.21.15.172:0/3946502971 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1b0004440 con 0x7fe1b4100e20 2024-09-10T09:05:50.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.456+0000 7fe1aaffd640 1 -- 172.21.15.172:0/3946502971 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0035070 con 0x7fe1b4100e20 2024-09-10T09:05:50.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.456+0000 7fe1bc25f640 1 -- 172.21.15.172:0/3946502971 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b41a49b0 con 0x7fe1b4100e20 2024-09-10T09:05:50.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.456+0000 7fe1bc25f640 1 -- 172.21.15.172:0/3946502971 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b41a4e10 con 0x7fe1b4100e20 2024-09-10T09:05:50.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.459+0000 7fe1aaffd640 1 -- 172.21.15.172:0/3946502971 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe1b0042030 con 0x7fe1b4100e20 2024-09-10T09:05:50.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.460+0000 7fe1be638740 1 Processor -- start 2024-09-10T09:05:50.474 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.460+0000 7fe1be638740 1 -- start start 2024-09-10T09:05:50.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.460+0000 7fe1be638740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560325595b40 0x560325594090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.460+0000 7fe1be638740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5603255945d0 0x560325594a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.460+0000 7fe1aaffd640 1 --2- 172.21.15.172:0/3946502971 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe1a00a2cd0 0x7fe1a00a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.460+0000 7fe1be638740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032559f370 0x5603255a1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.460+0000 7fe1be638740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5603254fa2f0 con 0x560325595b40 2024-09-10T09:05:50.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.460+0000 7fe1be638740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5603254a1cf0 con 0x56032559f370 2024-09-10T09:05:50.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.460+0000 7fe1be638740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5603254f62b0 con 0x5603255945d0 2024-09-10T09:05:50.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.461+0000 7fe1aaffd640 1 -- 172.21.15.172:0/3946502971 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe1b00f0330 con 0x7fe1b4100e20 2024-09-10T09:05:50.475 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.461+0000 7fe1b97fa640 1 --2- 172.21.15.172:0/3946502971 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe1a00a2cd0 0x7fe1a00a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:50.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.461+0000 7fe1b9ffb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5603255945d0 0x560325594a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:50.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.461+0000 7fe1b9ffb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5603255945d0 0x560325594a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33340/0 (socket says 172.21.15.172:33340) 2024-09-10T09:05:50.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.461+0000 7fe1b9ffb640 1 -- 172.21.15.172:0/466404865 learned_addr learned my addr 172.21.15.172:0/466404865 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:50.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.461+0000 7fe1ba7fc640 1 --2- 172.21.15.172:0/466404865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560325595b40 0x560325594090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:50.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.461+0000 7fe1b9ffb640 1 -- 172.21.15.172:0/466404865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032559f370 msgr2=0x5603255a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:50.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.461+0000 7fe1b9ffb640 1 --2- 172.21.15.172:0/466404865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032559f370 0x5603255a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.461+0000 7fe1b9ffb640 1 -- 172.21.15.172:0/466404865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560325595b40 msgr2=0x560325594090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:50.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.461+0000 7fe1b9ffb640 1 --2- 172.21.15.172:0/466404865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560325595b40 0x560325594090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.461+0000 7fe1b97fa640 1 --2- 172.21.15.172:0/3946502971 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe1a00a2cd0 0x7fe1a00a5190 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe1b406ab00 tx=0x7fe1a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:50.476 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.461+0000 7fe1b9ffb640 1 -- 172.21.15.172:0/466404865 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603254863a0 con 0x5603255945d0 2024-09-10T09:05:50.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.461+0000 7fe1ba7fc640 1 --2- 172.21.15.172:0/466404865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560325595b40 0x560325594090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:05:50.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.462+0000 7fe1b9ffb640 1 --2- 172.21.15.172:0/466404865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5603255945d0 0x560325594a10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0000c00 tx=0x7fe1b00bc810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aad7423a891fd0d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:50.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.462+0000 7fe19ffff640 1 -- 172.21.15.172:0/466404865 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0049040 con 0x5603255945d0 2024-09-10T09:05:50.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.462+0000 7fe19ffff640 1 -- 172.21.15.172:0/466404865 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1b00bc980 con 0x5603255945d0 2024-09-10T09:05:50.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.462+0000 7fe19ffff640 1 -- 172.21.15.172:0/466404865 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b00bac80 con 0x5603255945d0 2024-09-10T09:05:50.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.463+0000 7fe1be638740 1 -- 172.21.15.172:0/466404865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5603255945d0 msgr2=0x560325594a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:50.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.463+0000 7fe1be638740 1 --2- 172.21.15.172:0/466404865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5603255945d0 0x560325594a10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0000c00 tx=0x7fe1b00bc810 comp rx=0 tx=0).stop 2024-09-10T09:05:50.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.463+0000 7fe1be638740 1 -- 172.21.15.172:0/466404865 shutdown_connections 2024-09-10T09:05:50.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.463+0000 7fe1be638740 1 --2- 172.21.15.172:0/466404865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032559f370 0x5603255a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.477 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.463+0000 7fe1be638740 1 --2- 172.21.15.172:0/466404865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5603255945d0 0x560325594a10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.463+0000 7fe1be638740 1 --2- 172.21.15.172:0/466404865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560325595b40 0x560325594090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.463+0000 7fe1be638740 1 -- 172.21.15.172:0/466404865 >> 172.21.15.172:0/466404865 conn(0x56032555d310 msgr2=0x560325590c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:50.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.463+0000 7fe1be638740 1 -- 172.21.15.172:0/466404865 shutdown_connections 2024-09-10T09:05:50.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.463+0000 7fe1be638740 1 -- 172.21.15.172:0/466404865 wait complete. 2024-09-10T09:05:50.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.464+0000 7fe1be638740 1 -- start start 2024-09-10T09:05:50.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.464+0000 7fe1be638740 1 Processor -- start 2024-09-10T09:05:50.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.464+0000 7fe1be638740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603255945d0 0x56032559ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.464+0000 7fe1be638740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 0x56032559c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.464+0000 7fe1be638740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032559f370 0x56032559c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.478 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.464+0000 7fe1be638740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5603255a1e30 con 0x5603255945d0 2024-09-10T09:05:50.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.464+0000 7fe1be638740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5603254b5060 con 0x56032559f370 2024-09-10T09:05:50.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.464+0000 7fe1be638740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5603255a1fb0 con 0x560325595b40 2024-09-10T09:05:50.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.464+0000 7fe1be638740 10 client.0 _ll_register_callbacks cb 0x5603252088c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:50.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.465+0000 7fe1b97fa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 0x56032559c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:50.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.465+0000 7fe1b97fa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 0x56032559c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33348/0 (socket says 172.21.15.172:33348) 2024-09-10T09:05:50.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.465+0000 7fe1b97fa640 1 -- 172.21.15.172:0/3727178547 learned_addr learned my addr 172.21.15.172:0/3727178547 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:50.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.465+0000 7fe1b9ffb640 1 --2- 172.21.15.172:0/3727178547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603255945d0 0x56032559ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:50.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.465+0000 7fe1b97fa640 1 -- 172.21.15.172:0/3727178547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032559f370 msgr2=0x56032559c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:50.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.465+0000 7fe1b97fa640 1 --2- 172.21.15.172:0/3727178547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032559f370 0x56032559c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.465+0000 7fe1b97fa640 1 -- 172.21.15.172:0/3727178547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603255945d0 msgr2=0x56032559ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:50.479 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.465+0000 7fe1b97fa640 1 --2- 172.21.15.172:0/3727178547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603255945d0 0x56032559ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.465+0000 7fe1b97fa640 1 -- 172.21.15.172:0/3727178547 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56032526b3c0 con 0x560325595b40 2024-09-10T09:05:50.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.465+0000 7fe1b97fa640 1 --2- 172.21.15.172:0/3727178547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 0x56032559c010 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe1a400fbd0 tx=0x7fe1a4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d0ec53c753565a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:50.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.465+0000 7fe19d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:05:50.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.465+0000 7fe19d7fa640 1 -- 172.21.15.172:0/3727178547 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a401db20 con 0x560325595b40 2024-09-10T09:05:50.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.465+0000 7fe1be638740 1 -- 172.21.15.172:0/3727178547 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1b00bc980 con 0x560325595b40 2024-09-10T09:05:50.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.465+0000 7fe19d7fa640 1 -- 172.21.15.172:0/3727178547 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe1a401dcc0 con 0x560325595b40 2024-09-10T09:05:50.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.466+0000 7fe19d7fa640 1 -- 172.21.15.172:0/3727178547 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe1a4032050 con 0x560325595b40 2024-09-10T09:05:50.480 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.466+0000 7fe1be638740 10 client.28816 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:50.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.466+0000 7fe1be638740 1 -- 172.21.15.172:0/3727178547 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5603255ad860 con 0x560325595b40 2024-09-10T09:05:50.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.466+0000 7fe19d7fa640 1 -- 172.21.15.172:0/3727178547 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1b00bc980 con 0x560325595b40 2024-09-10T09:05:50.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.466+0000 7fe19d7fa640 1 -- 172.21.15.172:0/3727178547 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fe1a4027480 con 0x560325595b40 2024-09-10T09:05:50.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.466+0000 7fe1be638740 10 client.28816 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:50.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.466+0000 7fe1be638740 10 client.28816 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:05:50.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.466+0000 7fe1be638740 20 client.28816 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:50.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.466+0000 7fe1be638740 1 --2- 172.21.15.172:0/3727178547 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5603255aed10 0x5603255b1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.466+0000 7fe1be638740 4 client.28816 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:05:50.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.466+0000 7fe1be638740 1 -- 172.21.15.172:0/3727178547 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560325507860 con 0x5603255aed10 2024-09-10T09:05:50.481 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.467+0000 7fe1b9ffb640 1 --2- 172.21.15.172:0/3727178547 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5603255aed10 0x5603255b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:50.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.467+0000 7fe1b9ffb640 1 --2- 172.21.15.172:0/3727178547 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5603255aed10 0x5603255b1170 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47cabf7600960a59 server_cookie=d4ccb045c1623101 in_seq=0 out_seq=0 2024-09-10T09:05:50.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.467+0000 7fe19d7fa640 10 client.28816 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T09:05:50.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.468+0000 7fe19d7fa640 1 -- 172.21.15.172:0/3727178547 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560325507860 con 0x5603255aed10 2024-09-10T09:05:50.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.468+0000 7fe19d7fa640 10 client.28816 handle_command_reply: tid=0 2024-09-10T09:05:50.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.468+0000 7fe1be638740 1 client.28816 shutdown 2024-09-10T09:05:50.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.468+0000 7fe1be638740 1 -- 172.21.15.172:0/3727178547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 msgr2=0x56032559c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:50.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.468+0000 7fe1be638740 1 --2- 172.21.15.172:0/3727178547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 0x56032559c010 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe1a400fbd0 tx=0x7fe1a4004730 comp rx=0 tx=0).stop 2024-09-10T09:05:50.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.468+0000 7fe1be638740 1 -- 172.21.15.172:0/3727178547 shutdown_connections 2024-09-10T09:05:50.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.468+0000 7fe1be638740 1 --2- 172.21.15.172:0/3727178547 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x5603255aed10 0x5603255b1170 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.468+0000 7fe1be638740 1 --2- 172.21.15.172:0/3727178547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032559f370 0x56032559c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.482 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.468+0000 7fe1be638740 1 --2- 172.21.15.172:0/3727178547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 0x56032559c010 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.468+0000 7fe19d7fa640 0 client.28816 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T09:05:50.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.468+0000 7fe1be638740 1 --2- 172.21.15.172:0/3727178547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603255945d0 0x56032559ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.468+0000 7fe1be638740 1 -- 172.21.15.172:0/3727178547 >> 172.21.15.172:0/3727178547 conn(0x56032555d310 msgr2=0x5603255969c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:50.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.468+0000 7fe1be638740 1 -- 172.21.15.172:0/3727178547 shutdown_connections 2024-09-10T09:05:50.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.469+0000 7fe1be638740 1 -- 172.21.15.172:0/3727178547 wait complete. 2024-09-10T09:05:50.483 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.469+0000 7fe1be638740 20 client.28816 trim_cache size 0 max 16384 2024-09-10T09:05:50.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1be638740 1 Processor -- start 2024-09-10T09:05:50.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1be638740 1 -- start start 2024-09-10T09:05:50.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1be638740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603255945d0 0x5603255ab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1be638740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 0x56032559b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1be638740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032559f370 0x5603255b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1be638740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe19800faa0 con 0x5603255945d0 2024-09-10T09:05:50.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1be638740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe1980115e0 con 0x56032559f370 2024-09-10T09:05:50.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1be638740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe198010170 con 0x560325595b40 2024-09-10T09:05:50.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1b9ffb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603255945d0 0x5603255ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:50.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1b9ffb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603255945d0 0x5603255ab090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57640/0 (socket says 172.21.15.172:57640) 2024-09-10T09:05:50.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1b9ffb640 1 -- 172.21.15.172:0/1181697893 learned_addr learned my addr 172.21.15.172:0/1181697893 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:50.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1ba7fc640 1 --2- 172.21.15.172:0/1181697893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032559f370 0x5603255b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:50.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1b97fa640 1 --2- 172.21.15.172:0/1181697893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 0x56032559b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:50.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1b9ffb640 1 -- 172.21.15.172:0/1181697893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 msgr2=0x56032559b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:50.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1b9ffb640 1 --2- 172.21.15.172:0/1181697893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 0x56032559b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1b9ffb640 1 -- 172.21.15.172:0/1181697893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032559f370 msgr2=0x5603255b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:50.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1b9ffb640 1 --2- 172.21.15.172:0/1181697893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032559f370 0x5603255b6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.482+0000 7fe1b9ffb640 1 -- 172.21.15.172:0/1181697893 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560325597910 con 0x5603255945d0 2024-09-10T09:05:50.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.483+0000 7fe1b9ffb640 1 --2- 172.21.15.172:0/1181697893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603255945d0 0x5603255ab090 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00bc910 tx=0x7fe1b00edf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f9f7d58c15992a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:50.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.483+0000 7fe19f7fe640 1 -- 172.21.15.172:0/1181697893 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0049040 con 0x5603255945d0 2024-09-10T09:05:50.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.483+0000 7fe19f7fe640 1 -- 172.21.15.172:0/1181697893 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1b00bc980 con 0x5603255945d0 2024-09-10T09:05:50.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.483+0000 7fe19f7fe640 1 -- 172.21.15.172:0/1181697893 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b00bac80 con 0x5603255945d0 2024-09-10T09:05:50.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.483+0000 7fe1be638740 1 -- 172.21.15.172:0/1181697893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603255945d0 msgr2=0x5603255ab090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:50.491 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.483+0000 7fe1be638740 1 --2- 172.21.15.172:0/1181697893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603255945d0 0x5603255ab090 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00bc910 tx=0x7fe1b00edf90 comp rx=0 tx=0).stop 2024-09-10T09:05:50.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.484+0000 7fe1be638740 1 -- 172.21.15.172:0/1181697893 shutdown_connections 2024-09-10T09:05:50.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.484+0000 7fe1be638740 1 --2- 172.21.15.172:0/1181697893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56032559f370 0x5603255b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.484+0000 7fe1be638740 1 --2- 172.21.15.172:0/1181697893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 0x56032559b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.484+0000 7fe1be638740 1 --2- 172.21.15.172:0/1181697893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603255945d0 0x5603255ab090 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.484+0000 7fe1be638740 1 -- 172.21.15.172:0/1181697893 >> 172.21.15.172:0/1181697893 conn(0x56032555d310 msgr2=0x56032559b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:50.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.484+0000 7fe1be638740 1 -- 172.21.15.172:0/1181697893 shutdown_connections 2024-09-10T09:05:50.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.484+0000 7fe1be638740 1 -- 172.21.15.172:0/1181697893 wait complete. 2024-09-10T09:05:50.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.484+0000 7fe1be638740 1 -- start start 2024-09-10T09:05:50.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.484+0000 7fe1be638740 1 Processor -- start 2024-09-10T09:05:50.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.484+0000 7fe1be638740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603255945d0 0x5603255a3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.484+0000 7fe1be638740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 0x5603255a3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.492 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.484+0000 7fe1be638740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56032559f370 0x5603255a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.484+0000 7fe1be638740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe198010f10 con 0x56032559f370 2024-09-10T09:05:50.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.484+0000 7fe1be638740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe198010840 con 0x5603255945d0 2024-09-10T09:05:50.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.484+0000 7fe1be638740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe19800da80 con 0x560325595b40 2024-09-10T09:05:50.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.484+0000 7fe1be638740 10 client.0 _ll_register_callbacks cb 0x7fe1a4009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:50.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe1b97fa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 0x5603255a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:50.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe1b9ffb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603255945d0 0x5603255a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:50.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe1b9ffb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603255945d0 0x5603255a3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:38488/0 (socket says 172.21.15.172:38488) 2024-09-10T09:05:50.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe1b97fa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 0x5603255a3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33374/0 (socket says 172.21.15.172:33374) 2024-09-10T09:05:50.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe1b9ffb640 1 -- 172.21.15.172:0/3578236600 learned_addr learned my addr 172.21.15.172:0/3578236600 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:50.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe1ba7fc640 1 --2- 172.21.15.172:0/3578236600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56032559f370 0x5603255a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:50.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe1b97fa640 1 -- 172.21.15.172:0/3578236600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603255945d0 msgr2=0x5603255a3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:50.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe1b97fa640 1 --2- 172.21.15.172:0/3578236600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603255945d0 0x5603255a3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe1b97fa640 1 -- 172.21.15.172:0/3578236600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56032559f370 msgr2=0x5603255a3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:50.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe1b97fa640 1 --2- 172.21.15.172:0/3578236600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56032559f370 0x5603255a3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe1b97fa640 1 -- 172.21.15.172:0/3578236600 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560325618ca0 con 0x560325595b40 2024-09-10T09:05:50.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe1b9ffb640 1 --2- 172.21.15.172:0/3578236600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603255945d0 0x5603255a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:50.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe1b97fa640 1 --2- 172.21.15.172:0/3578236600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 0x5603255a3750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x56032559d5c0 tx=0x7fe1a4027dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f59e7f419af31b83 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:50.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe19d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:05:50.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe19d7fa640 1 -- 172.21.15.172:0/3578236600 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4027600 con 0x560325595b40 2024-09-10T09:05:50.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe1be638740 1 -- 172.21.15.172:0/3578236600 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1b00bc980 con 0x560325595b40 2024-09-10T09:05:50.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe19d7fa640 1 -- 172.21.15.172:0/3578236600 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fe1a401d7f0 con 0x560325595b40 2024-09-10T09:05:50.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe19d7fa640 1 -- 172.21.15.172:0/3578236600 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe1a4046050 con 0x560325595b40 2024-09-10T09:05:50.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.485+0000 7fe1be638740 10 client.28822 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:50.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.486+0000 7fe1be638740 1 -- 172.21.15.172:0/3578236600 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560325619fa0 con 0x560325595b40 2024-09-10T09:05:50.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.486+0000 7fe19d7fa640 1 -- 172.21.15.172:0/3578236600 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1b00bc980 con 0x560325595b40 2024-09-10T09:05:50.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.486+0000 7fe19d7fa640 1 -- 172.21.15.172:0/3578236600 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fe1a401d9f0 con 0x560325595b40 2024-09-10T09:05:50.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.486+0000 7fe1be638740 10 client.28822 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:50.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.486+0000 7fe1be638740 10 client.28822 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:05:50.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.486+0000 7fe1be638740 20 client.28822 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:50.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.486+0000 7fe1be638740 1 --2- 172.21.15.172:0/3578236600 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x56032561a6b0 0x56032561cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:50.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.486+0000 7fe1be638740 4 client.28822 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:05:50.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.486+0000 7fe1be638740 1 -- 172.21.15.172:0/3578236600 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560325501d70 con 0x56032561a6b0 2024-09-10T09:05:50.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.487+0000 7fe1b9ffb640 1 --2- 172.21.15.172:0/3578236600 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x56032561a6b0 0x56032561cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:50.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.487+0000 7fe19d7fa640 10 client.28822 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T09:05:50.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.487+0000 7fe1b9ffb640 1 --2- 172.21.15.172:0/3578236600 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x56032561a6b0 0x56032561cad0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e9bebd142bed085 server_cookie=ae5bb3c6d3838c90 in_seq=0 out_seq=0 2024-09-10T09:05:50.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.487+0000 7fe19d7fa640 1 -- 172.21.15.172:0/3578236600 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560325501d70 con 0x56032561a6b0 2024-09-10T09:05:50.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.487+0000 7fe19d7fa640 10 client.28822 handle_command_reply: tid=0 2024-09-10T09:05:50.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.487+0000 7fe1be638740 1 client.28822 shutdown 2024-09-10T09:05:50.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.487+0000 7fe1be638740 1 -- 172.21.15.172:0/3578236600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 msgr2=0x5603255a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:50.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.487+0000 7fe1be638740 1 --2- 172.21.15.172:0/3578236600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 0x5603255a3750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x56032559d5c0 tx=0x7fe1a4027dc0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.487+0000 7fe1be638740 1 -- 172.21.15.172:0/3578236600 shutdown_connections 2024-09-10T09:05:50.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.487+0000 7fe1be638740 1 --2- 172.21.15.172:0/3578236600 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x56032561a6b0 0x56032561cad0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.487+0000 7fe1be638740 1 --2- 172.21.15.172:0/3578236600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56032559f370 0x5603255a3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.487+0000 7fe19d7fa640 0 client.28822 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T09:05:50.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.487+0000 7fe1be638740 1 --2- 172.21.15.172:0/3578236600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560325595b40 0x5603255a3750 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.487+0000 7fe1be638740 1 --2- 172.21.15.172:0/3578236600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603255945d0 0x5603255a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.487+0000 7fe1be638740 1 -- 172.21.15.172:0/3578236600 >> 172.21.15.172:0/3578236600 conn(0x56032555d310 msgr2=0x56032559d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:50.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.487+0000 7fe1be638740 1 -- 172.21.15.172:0/3578236600 shutdown_connections 2024-09-10T09:05:50.497 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.487+0000 7fe1be638740 1 -- 172.21.15.172:0/3578236600 wait complete. 2024-09-10T09:05:50.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.488+0000 7fe1be638740 20 client.28822 trim_cache size 0 max 16384 2024-09-10T09:05:50.498 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:05:50.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.488+0000 7fe1bc25f640 1 -- 172.21.15.172:0/3946502971 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe1a00a2cd0 msgr2=0x7fe1a00a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:50.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.488+0000 7fe1bc25f640 1 --2- 172.21.15.172:0/3946502971 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe1a00a2cd0 0x7fe1a00a5190 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe1b406ab00 tx=0x7fe1a4005f90 comp rx=0 tx=0).stop 2024-09-10T09:05:50.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.488+0000 7fe1bc25f640 1 -- 172.21.15.172:0/3946502971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1b4100e20 msgr2=0x7fe1b4199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:50.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.488+0000 7fe1bc25f640 1 --2- 172.21.15.172:0/3946502971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1b4100e20 0x7fe1b4199220 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0005940 tx=0x7fe1b0004040 comp rx=0 tx=0).stop 2024-09-10T09:05:50.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.488+0000 7fe1bc25f640 1 -- 172.21.15.172:0/3946502971 shutdown_connections 2024-09-10T09:05:50.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.488+0000 7fe1bc25f640 1 --2- 172.21.15.172:0/3946502971 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe1a00a2cd0 0x7fe1a00a5190 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.488+0000 7fe1bc25f640 1 --2- 172.21.15.172:0/3946502971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1b4102ef0 0x7fe1b41a42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.488+0000 7fe1bc25f640 1 --2- 172.21.15.172:0/3946502971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1b4102570 0x7fe1b4199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.488+0000 7fe1bc25f640 1 --2- 172.21.15.172:0/3946502971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1b4100e20 0x7fe1b4199220 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:50.499 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.488+0000 7fe1bc25f640 1 -- 172.21.15.172:0/3946502971 >> 172.21.15.172:0/3946502971 conn(0x7fe1b40fc7d0 msgr2=0x7fe1b40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:50.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.488+0000 7fe1bc25f640 1 -- 172.21.15.172:0/3946502971 shutdown_connections 2024-09-10T09:05:50.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:50.488+0000 7fe1bc25f640 1 -- 172.21.15.172:0/3946502971 wait complete. 2024-09-10T09:05:50.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:50 smithi174 ceph-mon[34068]: pgmap v938: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 646 KiB/s wr, 171 op/s 2024-09-10T09:05:50.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:50 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:05:50.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:50 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:05:50.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:50 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:05:51.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:50 smithi191 ceph-mon[35314]: pgmap v938: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 646 KiB/s wr, 171 op/s 2024-09-10T09:05:51.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:50 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:05:51.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:50 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:05:51.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:50 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:05:51.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:50 smithi172 ceph-mon[30854]: pgmap v938: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 646 KiB/s wr, 171 op/s 2024-09-10T09:05:51.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:50 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:05:51.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:50 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:05:51.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:50 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:05:52.549 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:05:52.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:52 smithi172 ceph-mon[30854]: pgmap v939: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 729 KiB/s wr, 193 op/s 2024-09-10T09:05:52.861 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:05:52.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:52 smithi174 ceph-mon[34068]: pgmap v939: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 729 KiB/s wr, 193 op/s 2024-09-10T09:05:53.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:52 smithi191 ceph-mon[35314]: pgmap v939: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 729 KiB/s wr, 193 op/s 2024-09-10T09:05:53.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.382+0000 7fa3b22d8640 1 -- 172.21.15.172:0/225758303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3ac1027a0 msgr2=0x7fa3ac1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:53.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.382+0000 7fa3b22d8640 1 --2- 172.21.15.172:0/225758303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3ac1027a0 0x7fa3ac1067f0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009900 tx=0x7fa3a002a410 comp rx=0 tx=0).stop 2024-09-10T09:05:53.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.382+0000 7fa3b22d8640 1 -- 172.21.15.172:0/225758303 shutdown_connections 2024-09-10T09:05:53.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.382+0000 7fa3b22d8640 1 --2- 172.21.15.172:0/225758303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3ac106f20 0x7fa3ac10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.382+0000 7fa3b22d8640 1 --2- 172.21.15.172:0/225758303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3ac1027a0 0x7fa3ac1067f0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.382+0000 7fa3b22d8640 1 --2- 172.21.15.172:0/225758303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3ac101df0 0x7fa3ac1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.382+0000 7fa3b22d8640 1 -- 172.21.15.172:0/225758303 >> 172.21.15.172:0/225758303 conn(0x7fa3ac0fd8f0 msgr2=0x7fa3ac0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.382+0000 7fa3b22d8640 1 -- 172.21.15.172:0/225758303 shutdown_connections 2024-09-10T09:05:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.383+0000 7fa3b22d8640 1 -- 172.21.15.172:0/225758303 wait complete. 2024-09-10T09:05:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.383+0000 7fa3b22d8640 1 Processor -- start 2024-09-10T09:05:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.383+0000 7fa3b22d8640 1 -- start start 2024-09-10T09:05:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.384+0000 7fa3b22d8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3ac101df0 0x7fa3ac198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.384+0000 7fa3b22d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3ac1027a0 0x7fa3ac198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.384+0000 7fa3b22d8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3ac106f20 0x7fa3ac19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.384+0000 7fa3b22d8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac111e30 con 0x7fa3ac1027a0 2024-09-10T09:05:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.384+0000 7fa3b22d8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac111cb0 con 0x7fa3ac106f20 2024-09-10T09:05:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.384+0000 7fa3b22d8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac111fb0 con 0x7fa3ac101df0 2024-09-10T09:05:53.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.384+0000 7fa3ab7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3ac1027a0 0x7fa3ac198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.384+0000 7fa3abfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3ac101df0 0x7fa3ac198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.384+0000 7fa3ab7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3ac1027a0 0x7fa3ac198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:57680/0 (socket says 172.21.15.172:57680) 2024-09-10T09:05:53.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.384+0000 7fa3ab7fe640 1 -- 172.21.15.172:0/974205424 learned_addr learned my addr 172.21.15.172:0/974205424 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:53.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.384+0000 7fa3b0875640 1 --2- 172.21.15.172:0/974205424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3ac106f20 0x7fa3ac19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.384+0000 7fa3b0875640 1 -- 172.21.15.172:0/974205424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3ac101df0 msgr2=0x7fa3ac198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:53.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.384+0000 7fa3b0875640 1 --2- 172.21.15.172:0/974205424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3ac101df0 0x7fa3ac198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.384+0000 7fa3b0875640 1 -- 172.21.15.172:0/974205424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3ac1027a0 msgr2=0x7fa3ac198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:53.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.384+0000 7fa3b0875640 1 --2- 172.21.15.172:0/974205424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3ac1027a0 0x7fa3ac198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.384+0000 7fa3b0875640 1 -- 172.21.15.172:0/974205424 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac19d300 con 0x7fa3ac106f20 2024-09-10T09:05:53.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.384+0000 7fa3b0875640 1 --2- 172.21.15.172:0/974205424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3ac106f20 0x7fa3ac19cc20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009fd0 tx=0x7fa3a400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:53.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.385+0000 7fa3a97fa640 1 -- 172.21.15.172:0/974205424 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a400ce20 con 0x7fa3ac106f20 2024-09-10T09:05:53.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.385+0000 7fa3b22d8640 1 -- 172.21.15.172:0/974205424 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ac19d5f0 con 0x7fa3ac106f20 2024-09-10T09:05:53.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.385+0000 7fa3a97fa640 1 -- 172.21.15.172:0/974205424 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3a4005020 con 0x7fa3ac106f20 2024-09-10T09:05:53.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.385+0000 7fa3a97fa640 1 -- 172.21.15.172:0/974205424 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4010690 con 0x7fa3ac106f20 2024-09-10T09:05:53.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.385+0000 7fa3b22d8640 1 -- 172.21.15.172:0/974205424 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ac1a4ed0 con 0x7fa3ac106f20 2024-09-10T09:05:53.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.387+0000 7fa3a97fa640 1 -- 172.21.15.172:0/974205424 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa3a40051d0 con 0x7fa3ac106f20 2024-09-10T09:05:53.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.387+0000 7fa3b46b1740 1 Processor -- start 2024-09-10T09:05:53.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.387+0000 7fa3b46b1740 1 -- start start 2024-09-10T09:05:53.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3b46b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 0x55b07edbc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3b46b1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edbc5d0 0x55b07edbca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3b46b1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edc7370 0x55b07edc9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3b46b1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b07ed222f0 con 0x55b07edbc5d0 2024-09-10T09:05:53.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3b46b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b07ecc9cf0 con 0x55b07edbdb40 2024-09-10T09:05:53.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3b46b1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b07ed1e2b0 con 0x55b07edc7370 2024-09-10T09:05:53.408 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3a97fa640 1 --2- 172.21.15.172:0/974205424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3940a2b80 0x7fa3940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3ab7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 0x55b07edbc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3abfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edbc5d0 0x55b07edbca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3ab7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 0x55b07edbc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:38504/0 (socket says 172.21.15.172:38504) 2024-09-10T09:05:53.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3ab7fe640 1 -- 172.21.15.172:0/592436151 learned_addr learned my addr 172.21.15.172:0/592436151 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:53.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3ab7fe640 1 --2- 172.21.15.172:0/592436151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edc7370 0x55b07edc9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3b0875640 1 --2- 172.21.15.172:0/974205424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3940a2b80 0x7fa3940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3abfff640 1 -- 172.21.15.172:0/592436151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edc7370 msgr2=0x55b07edc9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:53.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3abfff640 1 --2- 172.21.15.172:0/592436151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edc7370 0x55b07edc9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3abfff640 1 -- 172.21.15.172:0/592436151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 msgr2=0x55b07edbc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:53.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3abfff640 1 --2- 172.21.15.172:0/592436151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 0x55b07edbc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3abfff640 1 -- 172.21.15.172:0/592436151 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b07ecae3a0 con 0x55b07edbc5d0 2024-09-10T09:05:53.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.388+0000 7fa3abfff640 1 --2- 172.21.15.172:0/592436151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edbc5d0 0x55b07edbca10 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009c20 tx=0x7fa39c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96599c8d51060f2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.389+0000 7fa3927fc640 1 -- 172.21.15.172:0/592436151 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c01e070 con 0x55b07edbc5d0 2024-09-10T09:05:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.389+0000 7fa3927fc640 1 -- 172.21.15.172:0/592436151 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa39c009dd0 con 0x55b07edbc5d0 2024-09-10T09:05:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.389+0000 7fa3927fc640 1 -- 172.21.15.172:0/592436151 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c018730 con 0x55b07edbc5d0 2024-09-10T09:05:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.389+0000 7fa3a97fa640 1 -- 172.21.15.172:0/974205424 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa3a40cf9e0 con 0x7fa3ac106f20 2024-09-10T09:05:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.389+0000 7fa3b46b1740 1 -- 172.21.15.172:0/592436151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edbc5d0 msgr2=0x55b07edbca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.389+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/592436151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edbc5d0 0x55b07edbca10 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009c20 tx=0x7fa39c011810 comp rx=0 tx=0).stop 2024-09-10T09:05:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.389+0000 7fa3b0875640 1 --2- 172.21.15.172:0/974205424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3940a2b80 0x7fa3940a5040 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4098340 tx=0x7fa3a40d9670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.390+0000 7fa3b46b1740 1 -- 172.21.15.172:0/592436151 shutdown_connections 2024-09-10T09:05:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.390+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/592436151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edc7370 0x55b07edc9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.390+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/592436151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edbc5d0 0x55b07edbca10 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.390+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/592436151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 0x55b07edbc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.390+0000 7fa3b46b1740 1 -- 172.21.15.172:0/592436151 >> 172.21.15.172:0/592436151 conn(0x55b07ed85310 msgr2=0x55b07edb8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.390+0000 7fa3b46b1740 1 -- 172.21.15.172:0/592436151 shutdown_connections 2024-09-10T09:05:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.390+0000 7fa3b46b1740 1 -- 172.21.15.172:0/592436151 wait complete. 2024-09-10T09:05:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.390+0000 7fa3b46b1740 1 -- start start 2024-09-10T09:05:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.390+0000 7fa3b46b1740 1 Processor -- start 2024-09-10T09:05:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.390+0000 7fa3b46b1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edbc5d0 0x55b07edc6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.390+0000 7fa3b46b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 0x55b07edc4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.390+0000 7fa3b46b1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edc7370 0x55b07edc4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.390+0000 7fa3b46b1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b07edca070 con 0x55b07edc7370 2024-09-10T09:05:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.390+0000 7fa3b46b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b07ecdd060 con 0x55b07edbdb40 2024-09-10T09:05:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.390+0000 7fa3b46b1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b07edca1f0 con 0x55b07edbc5d0 2024-09-10T09:05:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.390+0000 7fa3b46b1740 10 client.0 _ll_register_callbacks cb 0x55b07ea308c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:53.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa3ab7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edbc5d0 0x55b07edc6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa3abfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 0x55b07edc4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa3ab7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edbc5d0 0x55b07edc6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33410/0 (socket says 172.21.15.172:33410) 2024-09-10T09:05:53.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa3ab7fe640 1 -- 172.21.15.172:0/113016508 learned_addr learned my addr 172.21.15.172:0/113016508 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:53.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa3ab7fe640 1 --2- 172.21.15.172:0/113016508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edc7370 0x55b07edc4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa3ab7fe640 1 -- 172.21.15.172:0/113016508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 msgr2=0x55b07edc4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:53.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa3ab7fe640 1 --2- 172.21.15.172:0/113016508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 0x55b07edc4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa3ab7fe640 1 -- 172.21.15.172:0/113016508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edc7370 msgr2=0x55b07edc4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:53.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa3ab7fe640 1 --2- 172.21.15.172:0/113016508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edc7370 0x55b07edc4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa3ab7fe640 1 -- 172.21.15.172:0/113016508 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b07ea933c0 con 0x55b07edbc5d0 2024-09-10T09:05:53.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa3ab7fe640 1 --2- 172.21.15.172:0/113016508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edbc5d0 0x55b07edc6d80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00057c0 tx=0x7fa3a002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=907329602f787576 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:53.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa38b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:05:53.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa38b7fe640 1 -- 172.21.15.172:0/113016508 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0035040 con 0x55b07edbc5d0 2024-09-10T09:05:53.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa3b46b1740 1 -- 172.21.15.172:0/113016508 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa39c009dd0 con 0x55b07edbc5d0 2024-09-10T09:05:53.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa38b7fe640 1 -- 172.21.15.172:0/113016508 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa3a0005020 con 0x55b07edbc5d0 2024-09-10T09:05:53.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa38b7fe640 1 -- 172.21.15.172:0/113016508 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa3a0054050 con 0x55b07edbc5d0 2024-09-10T09:05:53.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa3b46b1740 10 client.28834 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:53.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.391+0000 7fa3b46b1740 1 -- 172.21.15.172:0/113016508 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b07edd5bc0 con 0x55b07edbc5d0 2024-09-10T09:05:53.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.392+0000 7fa38b7fe640 1 -- 172.21.15.172:0/113016508 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa39c009dd0 con 0x55b07edbc5d0 2024-09-10T09:05:53.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.392+0000 7fa38b7fe640 1 -- 172.21.15.172:0/113016508 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fa3a004e500 con 0x55b07edbc5d0 2024-09-10T09:05:53.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.392+0000 7fa3b46b1740 10 client.28834 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:53.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.392+0000 7fa3b46b1740 10 client.28834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:53.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.392+0000 7fa3b46b1740 20 client.28834 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:53.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.392+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/113016508 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b07edd6fd0 0x55b07edd93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.392+0000 7fa3b46b1740 4 client.28834 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:05:53.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.392+0000 7fa3b46b1740 1 -- 172.21.15.172:0/113016508 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b07ed2f860 con 0x55b07edd6fd0 2024-09-10T09:05:53.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.392+0000 7fa3abfff640 1 --2- 172.21.15.172:0/113016508 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b07edd6fd0 0x55b07edd93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.393+0000 7fa3abfff640 1 --2- 172.21.15.172:0/113016508 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b07edd6fd0 0x55b07edd93c0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d30ae1d192c2f18 server_cookie=be72293a781baaf3 in_seq=0 out_seq=0 2024-09-10T09:05:53.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.393+0000 7fa38b7fe640 10 client.28834 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:53.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.393+0000 7fa38b7fe640 1 -- 172.21.15.172:0/113016508 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b07ed2f860 con 0x55b07edd6fd0 2024-09-10T09:05:53.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.393+0000 7fa38b7fe640 10 client.28834 handle_command_reply: tid=0 2024-09-10T09:05:53.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.393+0000 7fa3b46b1740 1 client.28834 shutdown 2024-09-10T09:05:53.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.394+0000 7fa3b46b1740 1 -- 172.21.15.172:0/113016508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edbc5d0 msgr2=0x55b07edc6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:53.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.394+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/113016508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edbc5d0 0x55b07edc6d80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00057c0 tx=0x7fa3a002a410 comp rx=0 tx=0).stop 2024-09-10T09:05:53.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.394+0000 7fa3b46b1740 1 -- 172.21.15.172:0/113016508 shutdown_connections 2024-09-10T09:05:53.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.394+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/113016508 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b07edd6fd0 0x55b07edd93c0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.394+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/113016508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edc7370 0x55b07edc4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.394+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/113016508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 0x55b07edc4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.394+0000 7fa38b7fe640 0 client.28834 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:53.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.394+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/113016508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edbc5d0 0x55b07edc6d80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.394+0000 7fa3b46b1740 1 -- 172.21.15.172:0/113016508 >> 172.21.15.172:0/113016508 conn(0x55b07ed85310 msgr2=0x55b07edbe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:53.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.394+0000 7fa3b46b1740 1 -- 172.21.15.172:0/113016508 shutdown_connections 2024-09-10T09:05:53.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.394+0000 7fa3b46b1740 1 -- 172.21.15.172:0/113016508 wait complete. 2024-09-10T09:05:53.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.394+0000 7fa3b46b1740 20 client.28834 trim_cache size 0 max 16384 2024-09-10T09:05:53.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.409+0000 7fa3b46b1740 1 Processor -- start 2024-09-10T09:05:53.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.409+0000 7fa3b46b1740 1 -- start start 2024-09-10T09:05:53.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.410+0000 7fa3b46b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 0x55b07edb8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.410+0000 7fa3b46b1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edc7370 0x55b07edd3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.410+0000 7fa3b46b1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edc38f0 0x55b07eddec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.410+0000 7fa3b46b1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac1ac9a0 con 0x55b07edc38f0 2024-09-10T09:05:53.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.410+0000 7fa3b46b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac1ae4e0 con 0x55b07edbdb40 2024-09-10T09:05:53.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.410+0000 7fa3b46b1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac1ad070 con 0x55b07edc7370 2024-09-10T09:05:53.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.410+0000 7fa3ab7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 0x55b07edb8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.421 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.410+0000 7fa3ab7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 0x55b07edb8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:38522/0 (socket says 172.21.15.172:38522) 2024-09-10T09:05:53.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.410+0000 7fa3ab7fe640 1 -- 172.21.15.172:0/874166757 learned_addr learned my addr 172.21.15.172:0/874166757 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:53.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.410+0000 7fa3ab7fe640 1 --2- 172.21.15.172:0/874166757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edc38f0 0x55b07eddec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.410+0000 7fa3abfff640 1 --2- 172.21.15.172:0/874166757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edc7370 0x55b07edd3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.411+0000 7fa3ab7fe640 1 -- 172.21.15.172:0/874166757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edc7370 msgr2=0x55b07edd3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:53.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.411+0000 7fa3ab7fe640 1 --2- 172.21.15.172:0/874166757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edc7370 0x55b07edd3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.411+0000 7fa3ab7fe640 1 -- 172.21.15.172:0/874166757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edc38f0 msgr2=0x55b07eddec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:53.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.411+0000 7fa3ab7fe640 1 --2- 172.21.15.172:0/874166757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edc38f0 0x55b07eddec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.411+0000 7fa3ab7fe640 1 -- 172.21.15.172:0/874166757 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b07edc9ad0 con 0x55b07edbdb40 2024-09-10T09:05:53.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.411+0000 7fa3abfff640 1 --2- 172.21.15.172:0/874166757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edc7370 0x55b07edd3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:53.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.411+0000 7fa3ab7fe640 1 --2- 172.21.15.172:0/874166757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 0x55b07edb8e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0031be0 tx=0x7fa3a0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36663cea8aba5662 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:53.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.411+0000 7fa391ffb640 1 -- 172.21.15.172:0/874166757 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0035040 con 0x55b07edbdb40 2024-09-10T09:05:53.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.411+0000 7fa391ffb640 1 -- 172.21.15.172:0/874166757 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3a0042030 con 0x55b07edbdb40 2024-09-10T09:05:53.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.411+0000 7fa391ffb640 1 -- 172.21.15.172:0/874166757 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a004e310 con 0x55b07edbdb40 2024-09-10T09:05:53.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.412+0000 7fa3b46b1740 1 -- 172.21.15.172:0/874166757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 msgr2=0x55b07edb8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:53.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.412+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/874166757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 0x55b07edb8e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0031be0 tx=0x7fa3a0000bc0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.412+0000 7fa3b46b1740 1 -- 172.21.15.172:0/874166757 shutdown_connections 2024-09-10T09:05:53.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.412+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/874166757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edc38f0 0x55b07eddec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.412+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/874166757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edc7370 0x55b07edd3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.412+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/874166757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edbdb40 0x55b07edb8e10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.412+0000 7fa3b46b1740 1 -- 172.21.15.172:0/874166757 >> 172.21.15.172:0/874166757 conn(0x55b07ed85310 msgr2=0x55b07edb8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:53.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.412+0000 7fa3b46b1740 1 -- 172.21.15.172:0/874166757 shutdown_connections 2024-09-10T09:05:53.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.412+0000 7fa3b46b1740 1 -- 172.21.15.172:0/874166757 wait complete. 2024-09-10T09:05:53.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.412+0000 7fa3b46b1740 1 -- start start 2024-09-10T09:05:53.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.412+0000 7fa3b46b1740 1 Processor -- start 2024-09-10T09:05:53.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3b46b1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edbdb40 0x55b07edcfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3b46b1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edc38f0 0x55b07edc0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3b46b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edc7370 0x55b07edc1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3b46b1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac1ade10 con 0x55b07edc38f0 2024-09-10T09:05:53.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3b46b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac1ad740 con 0x55b07edc7370 2024-09-10T09:05:53.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3b46b1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac1aa980 con 0x55b07edbdb40 2024-09-10T09:05:53.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3b46b1740 10 client.0 _ll_register_callbacks cb 0x7fa3a0031dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:53.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3ab7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edbdb40 0x55b07edcfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3ab7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edbdb40 0x55b07edcfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:33428/0 (socket says 172.21.15.172:33428) 2024-09-10T09:05:53.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3ab7fe640 1 -- 172.21.15.172:0/2833748560 learned_addr learned my addr 172.21.15.172:0/2833748560 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:53.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3abfff640 1 --2- 172.21.15.172:0/2833748560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edc38f0 0x55b07edc0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3ab7fe640 1 --2- 172.21.15.172:0/2833748560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edc7370 0x55b07edc1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3ab7fe640 1 -- 172.21.15.172:0/2833748560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edc7370 msgr2=0x55b07edc1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:53.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3ab7fe640 1 --2- 172.21.15.172:0/2833748560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edc7370 0x55b07edc1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3ab7fe640 1 -- 172.21.15.172:0/2833748560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edc38f0 msgr2=0x55b07edc0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:53.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3ab7fe640 1 --2- 172.21.15.172:0/2833748560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edc38f0 0x55b07edc0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3ab7fe640 1 -- 172.21.15.172:0/2833748560 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b07edc1a40 con 0x55b07edbdb40 2024-09-10T09:05:53.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3abfff640 1 --2- 172.21.15.172:0/2833748560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edc38f0 0x55b07edc0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:05:53.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa3ab7fe640 1 --2- 172.21.15.172:0/2833748560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edbdb40 0x55b07edcfbc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009a20 tx=0x7fa3a0032e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2e6a14fbb3b631 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:53.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.413+0000 7fa38b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:05:53.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.414+0000 7fa38b7fe640 1 -- 172.21.15.172:0/2833748560 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0035040 con 0x55b07edbdb40 2024-09-10T09:05:53.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.414+0000 7fa38b7fe640 1 -- 172.21.15.172:0/2833748560 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa3a004eaf0 con 0x55b07edbdb40 2024-09-10T09:05:53.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.414+0000 7fa3b46b1740 1 -- 172.21.15.172:0/2833748560 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3a0042030 con 0x55b07edbdb40 2024-09-10T09:05:53.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.414+0000 7fa38b7fe640 1 -- 172.21.15.172:0/2833748560 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa3a004ecf0 con 0x55b07edbdb40 2024-09-10T09:05:53.425 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.414+0000 7fa3b46b1740 10 client.28840 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:53.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.414+0000 7fa3b46b1740 1 -- 172.21.15.172:0/2833748560 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b07ee41b90 con 0x55b07edbdb40 2024-09-10T09:05:53.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.414+0000 7fa38b7fe640 1 -- 172.21.15.172:0/2833748560 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3a0042030 con 0x55b07edbdb40 2024-09-10T09:05:53.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.414+0000 7fa38b7fe640 1 -- 172.21.15.172:0/2833748560 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7fa3a0061020 con 0x55b07edbdb40 2024-09-10T09:05:53.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.414+0000 7fa3b46b1740 10 client.28840 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:53.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.414+0000 7fa3b46b1740 10 client.28840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:53.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.414+0000 7fa3b46b1740 20 client.28840 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:53.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.414+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/2833748560 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b07ee421d0 0x55b07ee445f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:53.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.414+0000 7fa3b46b1740 4 client.28840 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:05:53.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.414+0000 7fa3b46b1740 1 -- 172.21.15.172:0/2833748560 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b07ed29d70 con 0x55b07ee421d0 2024-09-10T09:05:53.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.415+0000 7fa3abfff640 1 --2- 172.21.15.172:0/2833748560 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b07ee421d0 0x55b07ee445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:53.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.415+0000 7fa3abfff640 1 --2- 172.21.15.172:0/2833748560 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b07ee421d0 0x55b07ee445f0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa6aedc19a96d315 server_cookie=9e0f5b488c292bc7 in_seq=0 out_seq=0 2024-09-10T09:05:53.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.415+0000 7fa38b7fe640 10 client.28840 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:53.426 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.415+0000 7fa38b7fe640 1 -- 172.21.15.172:0/2833748560 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b07ed29d70 con 0x55b07ee421d0 2024-09-10T09:05:53.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.415+0000 7fa38b7fe640 10 client.28840 handle_command_reply: tid=0 2024-09-10T09:05:53.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.415+0000 7fa3b46b1740 1 client.28840 shutdown 2024-09-10T09:05:53.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.415+0000 7fa3b46b1740 1 -- 172.21.15.172:0/2833748560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edbdb40 msgr2=0x55b07edcfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:53.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.415+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/2833748560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edbdb40 0x55b07edcfbc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009a20 tx=0x7fa3a0032e60 comp rx=0 tx=0).stop 2024-09-10T09:05:53.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.415+0000 7fa3b46b1740 1 -- 172.21.15.172:0/2833748560 shutdown_connections 2024-09-10T09:05:53.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.415+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/2833748560 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55b07ee421d0 0x55b07ee445f0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.415+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/2833748560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b07edc7370 0x55b07edc1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.415+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/2833748560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b07edc38f0 0x55b07edc0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.427 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.415+0000 7fa3b46b1740 1 --2- 172.21.15.172:0/2833748560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b07edbdb40 0x55b07edcfbc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.416+0000 7fa3b46b1740 1 -- 172.21.15.172:0/2833748560 >> 172.21.15.172:0/2833748560 conn(0x55b07ed85310 msgr2=0x55b07edbe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:53.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.416+0000 7fa38b7fe640 0 client.28840 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:53.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.416+0000 7fa3b46b1740 1 -- 172.21.15.172:0/2833748560 shutdown_connections 2024-09-10T09:05:53.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.416+0000 7fa3b46b1740 1 -- 172.21.15.172:0/2833748560 wait complete. 2024-09-10T09:05:53.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.416+0000 7fa3b46b1740 20 client.28840 trim_cache size 0 max 16384 2024-09-10T09:05:53.428 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:05:53.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.416+0000 7fa3b22d8640 1 -- 172.21.15.172:0/974205424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3940a2b80 msgr2=0x7fa3940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:53.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.416+0000 7fa3b22d8640 1 --2- 172.21.15.172:0/974205424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3940a2b80 0x7fa3940a5040 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4098340 tx=0x7fa3a40d9670 comp rx=0 tx=0).stop 2024-09-10T09:05:53.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.416+0000 7fa3b22d8640 1 -- 172.21.15.172:0/974205424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3ac106f20 msgr2=0x7fa3ac19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:53.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.416+0000 7fa3b22d8640 1 --2- 172.21.15.172:0/974205424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3ac106f20 0x7fa3ac19cc20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009fd0 tx=0x7fa3a400ef10 comp rx=0 tx=0).stop 2024-09-10T09:05:53.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.416+0000 7fa3b22d8640 1 -- 172.21.15.172:0/974205424 shutdown_connections 2024-09-10T09:05:53.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.416+0000 7fa3b22d8640 1 --2- 172.21.15.172:0/974205424 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa3940a2b80 0x7fa3940a5040 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.416+0000 7fa3b22d8640 1 --2- 172.21.15.172:0/974205424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3ac106f20 0x7fa3ac19cc20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.416+0000 7fa3b22d8640 1 --2- 172.21.15.172:0/974205424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3ac1027a0 0x7fa3ac198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.416+0000 7fa3b22d8640 1 --2- 172.21.15.172:0/974205424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3ac101df0 0x7fa3ac198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:53.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.416+0000 7fa3b22d8640 1 -- 172.21.15.172:0/974205424 >> 172.21.15.172:0/974205424 conn(0x7fa3ac0fd8f0 msgr2=0x7fa3ac0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:53.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.416+0000 7fa3b22d8640 1 -- 172.21.15.172:0/974205424 shutdown_connections 2024-09-10T09:05:53.430 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:53.416+0000 7fa3b22d8640 1 -- 172.21.15.172:0/974205424 wait complete. 2024-09-10T09:05:54.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:54 smithi174 ceph-mon[34068]: pgmap v940: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 777 KiB/s wr, 203 op/s 2024-09-10T09:05:55.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:54 smithi191 ceph-mon[35314]: pgmap v940: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 777 KiB/s wr, 203 op/s 2024-09-10T09:05:55.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:54 smithi172 ceph-mon[30854]: pgmap v940: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 777 KiB/s wr, 203 op/s 2024-09-10T09:05:55.223 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:05:55.224 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:05:55.526 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:05:56.590 INFO:tasks.workunit.client.0.smithi172.stdout:Syncing()...1 sec 2024-09-10T09:05:56.590 INFO:tasks.workunit.client.0.smithi172.stdout:FFSB benchmark finished at: Tue Sep 10 09:05:56 2024 2024-09-10T09:05:56.590 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T09:05:56.590 INFO:tasks.workunit.client.0.smithi172.stdout:Results: 2024-09-10T09:05:56.590 INFO:tasks.workunit.client.0.smithi172.stdout:Benchmark took 419.76 sec 2024-09-10T09:05:56.590 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T09:05:56.591 INFO:tasks.workunit.client.0.smithi172.stdout:Total Results 2024-09-10T09:05:56.591 INFO:tasks.workunit.client.0.smithi172.stdout:=============== 2024-09-10T09:05:56.591 INFO:tasks.workunit.client.0.smithi172.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-09-10T09:05:56.591 INFO:tasks.workunit.client.0.smithi172.stdout: ======= ============ ========= ======= =========== ========== 2024-09-10T09:05:56.591 INFO:tasks.workunit.client.0.smithi172.stdout: write : 904960 2155.91 100.000% 100.000% 8.42MB/sec 2024-09-10T09:05:56.591 INFO:tasks.workunit.client.0.smithi172.stdout:- 2024-09-10T09:05:56.591 INFO:tasks.workunit.client.0.smithi172.stdout:2155.91 Transactions per Second 2024-09-10T09:05:56.591 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T09:05:56.591 INFO:tasks.workunit.client.0.smithi172.stdout:Throughput Results 2024-09-10T09:05:56.591 INFO:tasks.workunit.client.0.smithi172.stdout:=================== 2024-09-10T09:05:56.591 INFO:tasks.workunit.client.0.smithi172.stdout:Write Throughput: 8.42MB/sec 2024-09-10T09:05:56.595 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T09:05:56.595 INFO:tasks.workunit.client.0.smithi172.stdout:System Call Latency statistics in millisecs 2024-09-10T09:05:56.595 INFO:tasks.workunit.client.0.smithi172.stdout:===== 2024-09-10T09:05:56.595 INFO:tasks.workunit.client.0.smithi172.stdout: Min Avg Max Total Calls 2024-09-10T09:05:56.595 INFO:tasks.workunit.client.0.smithi172.stdout: ======== ======== ======== ============ 2024-09-10T09:05:56.595 INFO:tasks.workunit.client.0.smithi172.stdout:[ open] 0.489000 119.834535 15363.318359 707 2024-09-10T09:05:56.595 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[0] 5000.000000 - 10000.000000 : 1 2024-09-10T09:05:56.595 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[1] 2000.000000 - 5000.000000 : 4 2024-09-10T09:05:56.596 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-10T09:05:56.596 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[3] 500.000000 - 1000.000000 : 1 2024-09-10T09:05:56.596 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[4] 200.000000 - 500.000000 : 7 2024-09-10T09:05:56.596 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[5] 100.000000 - 200.000000 : 36 2024-09-10T09:05:56.596 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[6] 50.000000 - 100.000000 : 146 2024-09-10T09:05:56.596 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[7] 20.000000 - 50.000000 : 285 2024-09-10T09:05:56.596 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[8] 10.000000 - 20.000000 : 113 2024-09-10T09:05:56.596 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[9] 5.000000 - 10.000000 : 53 2024-09-10T09:05:56.597 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[10] 2.000000 - 5.000000 : 22 2024-09-10T09:05:56.597 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[11] 1.000000 - 2.000000 : 23 2024-09-10T09:05:56.597 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[12] 0.500000 - 1.000000 : 12 2024-09-10T09:05:56.597 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[13] 0.200000 - 0.500000 : 1 2024-09-10T09:05:56.597 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-09-10T09:05:56.597 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-09-10T09:05:56.597 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-10T09:05:56.597 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-10T09:05:56.597 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-10T09:05:56.597 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T09:05:56.598 INFO:tasks.workunit.client.0.smithi172.stdout:[ write] 0.088000 11.852533 73369.390625 904960 2024-09-10T09:05:56.598 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[0] 5000.000000 - 10000.000000 : 8 2024-09-10T09:05:56.598 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[1] 2000.000000 - 5000.000000 : 6 2024-09-10T09:05:56.598 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[2] 1000.000000 - 2000.000000 : 19 2024-09-10T09:05:56.598 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[3] 500.000000 - 1000.000000 : 13 2024-09-10T09:05:56.598 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[4] 200.000000 - 500.000000 : 1536 2024-09-10T09:05:56.598 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[5] 100.000000 - 200.000000 : 8086 2024-09-10T09:05:56.598 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[6] 50.000000 - 100.000000 : 18378 2024-09-10T09:05:56.598 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[7] 20.000000 - 50.000000 : 72973 2024-09-10T09:05:56.599 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[8] 10.000000 - 20.000000 : 88852 2024-09-10T09:05:56.599 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[9] 5.000000 - 10.000000 : 90451 2024-09-10T09:05:56.599 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[10] 2.000000 - 5.000000 : 79208 2024-09-10T09:05:56.599 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[11] 1.000000 - 2.000000 : 30469 2024-09-10T09:05:56.599 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[12] 0.500000 - 1.000000 : 36882 2024-09-10T09:05:56.599 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[13] 0.200000 - 0.500000 : 216459 2024-09-10T09:05:56.599 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[14] 0.100000 - 0.200000 : 261410 2024-09-10T09:05:56.599 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[15] 0.050000 - 0.100000 : 98 2024-09-10T09:05:56.600 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-10T09:05:56.600 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-10T09:05:56.600 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-10T09:05:56.600 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T09:05:56.600 INFO:tasks.workunit.client.0.smithi172.stdout:[ lseek] 0.000000 0.000908 9.725000 904960 2024-09-10T09:05:56.600 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-10T09:05:56.600 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-10T09:05:56.600 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-10T09:05:56.600 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-10T09:05:56.600 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-10T09:05:56.600 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-10T09:05:56.600 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-09-10T09:05:56.601 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-09-10T09:05:56.601 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-09-10T09:05:56.601 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[9] 5.000000 - 10.000000 : 3 2024-09-10T09:05:56.601 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[10] 2.000000 - 5.000000 : 2 2024-09-10T09:05:56.601 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[11] 1.000000 - 2.000000 : 1 2024-09-10T09:05:56.601 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[12] 0.500000 - 1.000000 : 3 2024-09-10T09:05:56.601 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[13] 0.200000 - 0.500000 : 5 2024-09-10T09:05:56.601 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[14] 0.100000 - 0.200000 : 8 2024-09-10T09:05:56.601 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[15] 0.050000 - 0.100000 : 17 2024-09-10T09:05:56.601 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[16] 0.020000 - 0.050000 : 162 2024-09-10T09:05:56.601 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[17] 0.010000 - 0.020000 : 679 2024-09-10T09:05:56.601 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[18] 0.000000 - 0.010000 : 904080 2024-09-10T09:05:56.601 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T09:05:56.602 INFO:tasks.workunit.client.0.smithi172.stdout:[ close] 0.024000 4.038184 317.325989 707 2024-09-10T09:05:56.602 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-10T09:05:56.602 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-10T09:05:56.602 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-10T09:05:56.602 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-10T09:05:56.602 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[4] 200.000000 - 500.000000 : 2 2024-09-10T09:05:56.602 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[5] 100.000000 - 200.000000 : 1 2024-09-10T09:05:56.602 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[6] 50.000000 - 100.000000 : 6 2024-09-10T09:05:56.602 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[7] 20.000000 - 50.000000 : 29 2024-09-10T09:05:56.602 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[8] 10.000000 - 20.000000 : 25 2024-09-10T09:05:56.602 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[9] 5.000000 - 10.000000 : 37 2024-09-10T09:05:56.602 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[10] 2.000000 - 5.000000 : 32 2024-09-10T09:05:56.602 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[11] 1.000000 - 2.000000 : 19 2024-09-10T09:05:56.603 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[12] 0.500000 - 1.000000 : 14 2024-09-10T09:05:56.603 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[13] 0.200000 - 0.500000 : 35 2024-09-10T09:05:56.603 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[14] 0.100000 - 0.200000 : 147 2024-09-10T09:05:56.603 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[15] 0.050000 - 0.100000 : 72 2024-09-10T09:05:56.603 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[16] 0.020000 - 0.050000 : 288 2024-09-10T09:05:56.603 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-10T09:05:56.603 INFO:tasks.workunit.client.0.smithi172.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-10T09:05:56.603 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T09:05:56.603 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T09:05:56.603 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T09:05:56.603 INFO:tasks.workunit.client.0.smithi172.stdout: 2024-09-10T09:05:56.603 INFO:tasks.workunit.client.0.smithi172.stdout:0.3% User Time 2024-09-10T09:05:56.603 INFO:tasks.workunit.client.0.smithi172.stdout:22.7% System Time 2024-09-10T09:05:56.603 INFO:tasks.workunit.client.0.smithi172.stdout:23.0% CPU Utilization 2024-09-10T09:05:56.688 INFO:tasks.workunit.client.0.smithi172.stderr:+ cd .. 2024-09-10T09:05:56.695 INFO:tasks.workunit.client.0.smithi172.stderr:+ rm -r tmp ffsb 2024-09-10T09:05:56.965 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:56 smithi172 ceph-mon[30854]: pgmap v941: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 687 KiB/s wr, 183 op/s 2024-09-10T09:05:56.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:56 smithi174 ceph-mon[34068]: pgmap v941: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 687 KiB/s wr, 183 op/s 2024-09-10T09:05:57.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:56 smithi191 ceph-mon[35314]: pgmap v941: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 687 KiB/s wr, 183 op/s 2024-09-10T09:05:57.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.054+0000 7f9b57501640 1 -- 172.21.15.172:0/880118987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b50101360 msgr2=0x7f9b5010df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:57.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.054+0000 7f9b57501640 1 --2- 172.21.15.172:0/880118987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b50101360 0x7f9b5010df90 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009900 tx=0x7f9b4002a410 comp rx=0 tx=0).stop 2024-09-10T09:05:57.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.054+0000 7f9b57501640 1 -- 172.21.15.172:0/880118987 shutdown_connections 2024-09-10T09:05:57.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.054+0000 7f9b57501640 1 --2- 172.21.15.172:0/880118987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b5010e4d0 0x7f9b501108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.054+0000 7f9b57501640 1 --2- 172.21.15.172:0/880118987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b50101360 0x7f9b5010df90 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.054+0000 7f9b57501640 1 --2- 172.21.15.172:0/880118987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b501009b0 0x7f9b50100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.054+0000 7f9b57501640 1 -- 172.21.15.172:0/880118987 >> 172.21.15.172:0/880118987 conn(0x7f9b500fc7d0 msgr2=0x7f9b500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:57.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.054+0000 7f9b57501640 1 -- 172.21.15.172:0/880118987 shutdown_connections 2024-09-10T09:05:57.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.054+0000 7f9b57501640 1 -- 172.21.15.172:0/880118987 wait complete. 2024-09-10T09:05:57.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.055+0000 7f9b57501640 1 Processor -- start 2024-09-10T09:05:57.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.055+0000 7f9b57501640 1 -- start start 2024-09-10T09:05:57.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.055+0000 7f9b57501640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b501009b0 0x7f9b5019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.055+0000 7f9b57501640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b50101360 0x7f9b5019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.055+0000 7f9b57501640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b5010e4d0 0x7f9b5019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.055+0000 7f9b57501640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9b501140d0 con 0x7f9b50101360 2024-09-10T09:05:57.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.055+0000 7f9b57501640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9b50113f50 con 0x7f9b501009b0 2024-09-10T09:05:57.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.055+0000 7f9b57501640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9b50114250 con 0x7f9b5010e4d0 2024-09-10T09:05:57.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.055+0000 7f9b54a9c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b50101360 0x7f9b5019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.055+0000 7f9b5529d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b501009b0 0x7f9b5019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.055+0000 7f9b54a9c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b50101360 0x7f9b5019ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45466/0 (socket says 172.21.15.172:45466) 2024-09-10T09:05:57.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.055+0000 7f9b54a9c640 1 -- 172.21.15.172:0/1974035849 learned_addr learned my addr 172.21.15.172:0/1974035849 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:57.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.056+0000 7f9b55a9e640 1 --2- 172.21.15.172:0/1974035849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b5010e4d0 0x7f9b5019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.056+0000 7f9b5529d640 1 -- 172.21.15.172:0/1974035849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b5010e4d0 msgr2=0x7f9b5019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:57.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.056+0000 7f9b5529d640 1 --2- 172.21.15.172:0/1974035849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b5010e4d0 0x7f9b5019efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.056+0000 7f9b5529d640 1 -- 172.21.15.172:0/1974035849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b50101360 msgr2=0x7f9b5019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:57.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.056+0000 7f9b5529d640 1 --2- 172.21.15.172:0/1974035849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b50101360 0x7f9b5019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.056+0000 7f9b5529d640 1 -- 172.21.15.172:0/1974035849 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5019f6a0 con 0x7f9b501009b0 2024-09-10T09:05:57.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.056+0000 7f9b5529d640 1 --2- 172.21.15.172:0/1974035849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b501009b0 0x7f9b5019a6f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00e9f0 tx=0x7f9b4c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:57.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.056+0000 7f9b467fc640 1 -- 172.21.15.172:0/1974035849 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c00ccc0 con 0x7f9b501009b0 2024-09-10T09:05:57.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.056+0000 7f9b57501640 1 -- 172.21.15.172:0/1974035849 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5019f990 con 0x7f9b501009b0 2024-09-10T09:05:57.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.056+0000 7f9b467fc640 1 -- 172.21.15.172:0/1974035849 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b4c005020 con 0x7f9b501009b0 2024-09-10T09:05:57.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.056+0000 7f9b467fc640 1 -- 172.21.15.172:0/1974035849 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c01c070 con 0x7f9b501009b0 2024-09-10T09:05:57.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.056+0000 7f9b57501640 1 -- 172.21.15.172:0/1974035849 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b501a7270 con 0x7f9b501009b0 2024-09-10T09:05:57.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.059+0000 7f9b467fc640 1 -- 172.21.15.172:0/1974035849 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9b4c0049a0 con 0x7f9b501009b0 2024-09-10T09:05:57.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.059+0000 7f9b598da740 1 Processor -- start 2024-09-10T09:05:57.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.059+0000 7f9b598da740 1 -- start start 2024-09-10T09:05:57.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.059+0000 7f9b598da740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44286b60 0x55dc44286f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.059+0000 7f9b598da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44287480 0x55dc44291800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.059+0000 7f9b598da740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44291d40 0x55dc44294130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.059+0000 7f9b598da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55dc441ec280 con 0x55dc44287480 2024-09-10T09:05:57.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.059+0000 7f9b598da740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc44193b10 con 0x55dc44291d40 2024-09-10T09:05:57.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.059+0000 7f9b598da740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55dc441e8240 con 0x55dc44286b60 2024-09-10T09:05:57.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.059+0000 7f9b5529d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44287480 0x55dc44291800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.059+0000 7f9b5529d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44287480 0x55dc44291800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45482/0 (socket says 172.21.15.172:45482) 2024-09-10T09:05:57.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.059+0000 7f9b5529d640 1 -- 172.21.15.172:0/2712714572 learned_addr learned my addr 172.21.15.172:0/2712714572 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:57.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.059+0000 7f9b54a9c640 1 --2- 172.21.15.172:0/2712714572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44291d40 0x55dc44294130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.059+0000 7f9b55a9e640 1 --2- 172.21.15.172:0/2712714572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44286b60 0x55dc44286f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.079 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.059+0000 7f9b467fc640 1 --2- 172.21.15.172:0/1974035849 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9b3c0a2b80 0x7f9b3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.060+0000 7f9b5529d640 1 -- 172.21.15.172:0/2712714572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44286b60 msgr2=0x55dc44286f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:57.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.060+0000 7f9b5529d640 1 --2- 172.21.15.172:0/2712714572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44286b60 0x55dc44286f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.060+0000 7f9b5529d640 1 -- 172.21.15.172:0/2712714572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44291d40 msgr2=0x55dc44294130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:57.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.060+0000 7f9b5529d640 1 --2- 172.21.15.172:0/2712714572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44291d40 0x55dc44294130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.060+0000 7f9b5529d640 1 -- 172.21.15.172:0/2712714572 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc441a01f0 con 0x55dc44287480 2024-09-10T09:05:57.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.060+0000 7f9b54a9c640 1 --2- 172.21.15.172:0/2712714572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44291d40 0x55dc44294130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:57.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.060+0000 7f9b55a9e640 1 --2- 172.21.15.172:0/1974035849 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9b3c0a2b80 0x7f9b3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.060+0000 7f9b467fc640 1 -- 172.21.15.172:0/1974035849 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9b4c0cf390 con 0x7f9b501009b0 2024-09-10T09:05:57.080 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.060+0000 7f9b5529d640 1 --2- 172.21.15.172:0/2712714572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44287480 0x55dc44291800 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c017c60 tx=0x7f9b4c090d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57611b33f326da7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:57.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.060+0000 7f9b3b7fe640 1 -- 172.21.15.172:0/2712714572 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c09b3b0 con 0x55dc44287480 2024-09-10T09:05:57.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.060+0000 7f9b55a9e640 1 --2- 172.21.15.172:0/1974035849 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9b3c0a2b80 0x7f9b3c0a5040 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9b48005c40 tx=0x7f9b48004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:57.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.060+0000 7f9b3b7fe640 1 -- 172.21.15.172:0/2712714572 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b4c0175c0 con 0x55dc44287480 2024-09-10T09:05:57.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.060+0000 7f9b3b7fe640 1 -- 172.21.15.172:0/2712714572 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c0173d0 con 0x55dc44287480 2024-09-10T09:05:57.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.061+0000 7f9b598da740 1 -- 172.21.15.172:0/2712714572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44287480 msgr2=0x55dc44291800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:57.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.061+0000 7f9b598da740 1 --2- 172.21.15.172:0/2712714572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44287480 0x55dc44291800 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c017c60 tx=0x7f9b4c090d50 comp rx=0 tx=0).stop 2024-09-10T09:05:57.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.061+0000 7f9b598da740 1 -- 172.21.15.172:0/2712714572 shutdown_connections 2024-09-10T09:05:57.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.061+0000 7f9b598da740 1 --2- 172.21.15.172:0/2712714572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44291d40 0x55dc44294130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.061+0000 7f9b598da740 1 --2- 172.21.15.172:0/2712714572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44287480 0x55dc44291800 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.061+0000 7f9b598da740 1 --2- 172.21.15.172:0/2712714572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44286b60 0x55dc44286f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.061+0000 7f9b598da740 1 -- 172.21.15.172:0/2712714572 >> 172.21.15.172:0/2712714572 conn(0x55dc442830f0 msgr2=0x55dc442834d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:57.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.061+0000 7f9b598da740 1 -- 172.21.15.172:0/2712714572 shutdown_connections 2024-09-10T09:05:57.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.061+0000 7f9b598da740 1 -- 172.21.15.172:0/2712714572 wait complete. 2024-09-10T09:05:57.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.061+0000 7f9b598da740 1 -- start start 2024-09-10T09:05:57.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.061+0000 7f9b598da740 1 Processor -- start 2024-09-10T09:05:57.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b598da740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44286b60 0x55dc4428ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b598da740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44287480 0x55dc4428f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b598da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44291d40 0x55dc4428f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b598da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55dc442948b0 con 0x55dc44291d40 2024-09-10T09:05:57.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b598da740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc441a6f10 con 0x55dc44287480 2024-09-10T09:05:57.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b598da740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55dc44294a30 con 0x55dc44286b60 2024-09-10T09:05:57.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b598da740 10 client.0 _ll_register_callbacks cb 0x55dc4416fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:57.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b54a9c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44287480 0x55dc4428f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b55a9e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44291d40 0x55dc4428f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b54a9c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44287480 0x55dc4428f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:35258/0 (socket says 172.21.15.172:35258) 2024-09-10T09:05:57.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b54a9c640 1 -- 172.21.15.172:0/235227858 learned_addr learned my addr 172.21.15.172:0/235227858 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:57.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b5529d640 1 --2- 172.21.15.172:0/235227858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44286b60 0x55dc4428ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b54a9c640 1 -- 172.21.15.172:0/235227858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44286b60 msgr2=0x55dc4428ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:57.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b54a9c640 1 --2- 172.21.15.172:0/235227858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44286b60 0x55dc4428ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b54a9c640 1 -- 172.21.15.172:0/235227858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44291d40 msgr2=0x55dc4428f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:57.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b54a9c640 1 --2- 172.21.15.172:0/235227858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44291d40 0x55dc4428f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b54a9c640 1 -- 172.21.15.172:0/235227858 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc43f5d6a0 con 0x55dc44287480 2024-09-10T09:05:57.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b55a9e640 1 --2- 172.21.15.172:0/235227858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44291d40 0x55dc4428f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:57.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b54a9c640 1 --2- 172.21.15.172:0/235227858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44287480 0x55dc4428f160 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9b40000c00 tx=0x7f9b40031ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62f44b9d1d50e021 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:57.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.062+0000 7f9b38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:05:57.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.063+0000 7f9b38ff9640 1 -- 172.21.15.172:0/235227858 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40035040 con 0x55dc44287480 2024-09-10T09:05:57.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.063+0000 7f9b38ff9640 1 -- 172.21.15.172:0/235227858 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9b400317a0 con 0x55dc44287480 2024-09-10T09:05:57.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.063+0000 7f9b598da740 1 -- 172.21.15.172:0/235227858 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b4c0175c0 con 0x55dc44287480 2024-09-10T09:05:57.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.063+0000 7f9b38ff9640 1 -- 172.21.15.172:0/235227858 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9b4004f050 con 0x55dc44287480 2024-09-10T09:05:57.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.063+0000 7f9b598da740 10 client.28949 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:57.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.063+0000 7f9b598da740 1 -- 172.21.15.172:0/235227858 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55dc442a04b0 con 0x55dc44287480 2024-09-10T09:05:57.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.063+0000 7f9b38ff9640 1 -- 172.21.15.172:0/235227858 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b4c0175c0 con 0x55dc44287480 2024-09-10T09:05:57.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.063+0000 7f9b38ff9640 1 -- 172.21.15.172:0/235227858 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f9b40003ce0 con 0x55dc44287480 2024-09-10T09:05:57.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.063+0000 7f9b598da740 10 client.28949 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:57.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.063+0000 7f9b598da740 10 client.28949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:57.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.063+0000 7f9b598da740 20 client.28949 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:57.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.063+0000 7f9b598da740 1 --2- 172.21.15.172:0/235227858 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55dc442a19c0 0x55dc442a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.063+0000 7f9b598da740 4 client.28949 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:05:57.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.063+0000 7f9b598da740 1 -- 172.21.15.172:0/235227858 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc441f97f0 con 0x55dc442a19c0 2024-09-10T09:05:57.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.063+0000 7f9b5529d640 1 --2- 172.21.15.172:0/235227858 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55dc442a19c0 0x55dc442a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.064+0000 7f9b38ff9640 10 client.28949 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:57.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.064+0000 7f9b5529d640 1 --2- 172.21.15.172:0/235227858 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55dc442a19c0 0x55dc442a3de0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21d85d2f803ae43f server_cookie=5dea1952f0a231ec in_seq=0 out_seq=0 2024-09-10T09:05:57.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.064+0000 7f9b38ff9640 1 -- 172.21.15.172:0/235227858 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc441f97f0 con 0x55dc442a19c0 2024-09-10T09:05:57.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.064+0000 7f9b38ff9640 10 client.28949 handle_command_reply: tid=0 2024-09-10T09:05:57.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.064+0000 7f9b598da740 1 client.28949 shutdown 2024-09-10T09:05:57.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.065+0000 7f9b598da740 1 -- 172.21.15.172:0/235227858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44287480 msgr2=0x55dc4428f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:57.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.065+0000 7f9b598da740 1 --2- 172.21.15.172:0/235227858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44287480 0x55dc4428f160 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9b40000c00 tx=0x7f9b40031ef0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.065+0000 7f9b598da740 1 -- 172.21.15.172:0/235227858 shutdown_connections 2024-09-10T09:05:57.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.065+0000 7f9b598da740 1 --2- 172.21.15.172:0/235227858 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55dc442a19c0 0x55dc442a3de0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.065+0000 7f9b598da740 1 --2- 172.21.15.172:0/235227858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44291d40 0x55dc4428f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.065+0000 7f9b38ff9640 0 client.28949 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:57.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.065+0000 7f9b598da740 1 --2- 172.21.15.172:0/235227858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44287480 0x55dc4428f160 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.065+0000 7f9b598da740 1 --2- 172.21.15.172:0/235227858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44286b60 0x55dc4428ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.065+0000 7f9b598da740 1 -- 172.21.15.172:0/235227858 >> 172.21.15.172:0/235227858 conn(0x55dc442830f0 msgr2=0x55dc44288e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:57.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.065+0000 7f9b598da740 1 -- 172.21.15.172:0/235227858 shutdown_connections 2024-09-10T09:05:57.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.065+0000 7f9b598da740 1 -- 172.21.15.172:0/235227858 wait complete. 2024-09-10T09:05:57.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.065+0000 7f9b598da740 20 client.28949 trim_cache size 0 max 16384 2024-09-10T09:05:57.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b598da740 1 Processor -- start 2024-09-10T09:05:57.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b598da740 1 -- start start 2024-09-10T09:05:57.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b598da740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44286b60 0x55dc44290e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b598da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44287480 0x55dc4428f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b598da740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44291d40 0x55dc442a96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b598da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9b501b0d20 con 0x55dc44287480 2024-09-10T09:05:57.091 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b598da740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9b501b2860 con 0x55dc44286b60 2024-09-10T09:05:57.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b598da740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9b501b13f0 con 0x55dc44291d40 2024-09-10T09:05:57.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b54a9c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44287480 0x55dc4428f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b54a9c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44287480 0x55dc4428f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45502/0 (socket says 172.21.15.172:45502) 2024-09-10T09:05:57.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b54a9c640 1 -- 172.21.15.172:0/1980252397 learned_addr learned my addr 172.21.15.172:0/1980252397 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:57.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b5529d640 1 --2- 172.21.15.172:0/1980252397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44286b60 0x55dc44290e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b55a9e640 1 --2- 172.21.15.172:0/1980252397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44291d40 0x55dc442a96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b54a9c640 1 -- 172.21.15.172:0/1980252397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44291d40 msgr2=0x55dc442a96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:57.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b54a9c640 1 --2- 172.21.15.172:0/1980252397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44291d40 0x55dc442a96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b54a9c640 1 -- 172.21.15.172:0/1980252397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44286b60 msgr2=0x55dc44290e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:57.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b54a9c640 1 --2- 172.21.15.172:0/1980252397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44286b60 0x55dc44290e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b54a9c640 1 -- 172.21.15.172:0/1980252397 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc44282f20 con 0x55dc44287480 2024-09-10T09:05:57.092 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b55a9e640 1 --2- 172.21.15.172:0/1980252397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44291d40 0x55dc442a96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:05:57.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.083+0000 7f9b54a9c640 1 --2- 172.21.15.172:0/1980252397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44287480 0x55dc4428f4f0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f9b40046500 tx=0x7f9b40005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d5279856e290cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:57.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.084+0000 7f9b3affd640 1 -- 172.21.15.172:0/1980252397 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40035070 con 0x55dc44287480 2024-09-10T09:05:57.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.084+0000 7f9b3affd640 1 -- 172.21.15.172:0/1980252397 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b40046d70 con 0x55dc44287480 2024-09-10T09:05:57.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.084+0000 7f9b3affd640 1 -- 172.21.15.172:0/1980252397 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b400035f0 con 0x55dc44287480 2024-09-10T09:05:57.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.084+0000 7f9b598da740 1 -- 172.21.15.172:0/1980252397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44287480 msgr2=0x55dc4428f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:57.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.084+0000 7f9b598da740 1 --2- 172.21.15.172:0/1980252397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44287480 0x55dc4428f4f0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f9b40046500 tx=0x7f9b40005780 comp rx=0 tx=0).stop 2024-09-10T09:05:57.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.085+0000 7f9b598da740 1 -- 172.21.15.172:0/1980252397 shutdown_connections 2024-09-10T09:05:57.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.085+0000 7f9b598da740 1 --2- 172.21.15.172:0/1980252397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44291d40 0x55dc442a96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.085+0000 7f9b598da740 1 --2- 172.21.15.172:0/1980252397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44287480 0x55dc4428f4f0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.085+0000 7f9b598da740 1 --2- 172.21.15.172:0/1980252397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44286b60 0x55dc44290e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.085+0000 7f9b598da740 1 -- 172.21.15.172:0/1980252397 >> 172.21.15.172:0/1980252397 conn(0x55dc442830f0 msgr2=0x55dc4428dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:57.093 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.085+0000 7f9b598da740 1 -- 172.21.15.172:0/1980252397 shutdown_connections 2024-09-10T09:05:57.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.085+0000 7f9b598da740 1 -- 172.21.15.172:0/1980252397 wait complete. 2024-09-10T09:05:57.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.085+0000 7f9b598da740 1 -- start start 2024-09-10T09:05:57.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.085+0000 7f9b598da740 1 Processor -- start 2024-09-10T09:05:57.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.085+0000 7f9b598da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44286b60 0x55dc4429a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.085+0000 7f9b598da740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44287480 0x55dc44295820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.085+0000 7f9b598da740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44291d40 0x55dc44295e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.085+0000 7f9b598da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9b501b2190 con 0x55dc44286b60 2024-09-10T09:05:57.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b598da740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9b501b1ac0 con 0x55dc44291d40 2024-09-10T09:05:57.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b5529d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44286b60 0x55dc4429a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b5529d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44286b60 0x55dc4429a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45512/0 (socket says 172.21.15.172:45512) 2024-09-10T09:05:57.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b5529d640 1 -- 172.21.15.172:0/2043247393 learned_addr learned my addr 172.21.15.172:0/2043247393 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:57.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b55a9e640 1 --2- 172.21.15.172:0/2043247393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44291d40 0x55dc44295e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b598da740 1 -- 172.21.15.172:0/2043247393 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9b501aed00 con 0x55dc44287480 2024-09-10T09:05:57.094 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b598da740 10 client.0 _ll_register_callbacks cb 0x7f9b40009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:57.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b54a9c640 1 --2- 172.21.15.172:0/2043247393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44287480 0x55dc44295820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b55a9e640 1 -- 172.21.15.172:0/2043247393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44287480 msgr2=0x55dc44295820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:57.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b55a9e640 1 --2- 172.21.15.172:0/2043247393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44287480 0x55dc44295820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b55a9e640 1 -- 172.21.15.172:0/2043247393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44286b60 msgr2=0x55dc4429a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:57.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b55a9e640 1 --2- 172.21.15.172:0/2043247393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44286b60 0x55dc4429a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b55a9e640 1 -- 172.21.15.172:0/2043247393 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc44296640 con 0x55dc44291d40 2024-09-10T09:05:57.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b54a9c640 1 --2- 172.21.15.172:0/2043247393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44287480 0x55dc44295820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:05:57.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b5529d640 1 --2- 172.21.15.172:0/2043247393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44286b60 0x55dc4429a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:05:57.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b55a9e640 1 --2- 172.21.15.172:0/2043247393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44291d40 0x55dc44295e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009c10 tx=0x7f9b48009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a118e1a74cac61b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:57.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:05:57.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b38ff9640 1 -- 172.21.15.172:0/2043247393 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48028440 con 0x55dc44291d40 2024-09-10T09:05:57.095 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b38ff9640 1 -- 172.21.15.172:0/2043247393 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f9b4801d420 con 0x55dc44291d40 2024-09-10T09:05:57.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.086+0000 7f9b598da740 1 -- 172.21.15.172:0/2043247393 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b40046d70 con 0x55dc44291d40 2024-09-10T09:05:57.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.087+0000 7f9b38ff9640 1 -- 172.21.15.172:0/2043247393 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9b480285e0 con 0x55dc44291d40 2024-09-10T09:05:57.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.087+0000 7f9b598da740 10 client.28955 fetch_fsmap learned FSMap version 44 2024-09-10T09:05:57.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.087+0000 7f9b598da740 1 -- 172.21.15.172:0/2043247393 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55dc4430c7d0 con 0x55dc44291d40 2024-09-10T09:05:57.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.087+0000 7f9b38ff9640 1 -- 172.21.15.172:0/2043247393 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b40046d70 con 0x55dc44291d40 2024-09-10T09:05:57.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.087+0000 7f9b38ff9640 1 -- 172.21.15.172:0/2043247393 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 44) ==== 8370+0+0 (secure 0 0 0) 0x7f9b48026070 con 0x55dc44291d40 2024-09-10T09:05:57.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.087+0000 7f9b598da740 10 client.28955 fetch_fsmap finished waiting for FSMap version 44 2024-09-10T09:05:57.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.087+0000 7f9b598da740 10 client.28955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:57.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.087+0000 7f9b598da740 20 client.28955 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:57.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.087+0000 7f9b598da740 1 --2- 172.21.15.172:0/2043247393 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55dc4430cfd0 0x55dc4430f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:57.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.087+0000 7f9b598da740 4 client.28955 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:05:57.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.087+0000 7f9b598da740 1 -- 172.21.15.172:0/2043247393 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dc441f3d00 con 0x55dc4430cfd0 2024-09-10T09:05:57.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.087+0000 7f9b5529d640 1 --2- 172.21.15.172:0/2043247393 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55dc4430cfd0 0x55dc4430f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:57.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.087+0000 7f9b38ff9640 10 client.28955 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:57.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.087+0000 7f9b5529d640 1 --2- 172.21.15.172:0/2043247393 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55dc4430cfd0 0x55dc4430f410 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4f03f3ebdd24c00 server_cookie=d8a692d45d4c9da5 in_seq=0 out_seq=0 2024-09-10T09:05:57.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.088+0000 7f9b38ff9640 1 -- 172.21.15.172:0/2043247393 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dc441f3d00 con 0x55dc4430cfd0 2024-09-10T09:05:57.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.088+0000 7f9b38ff9640 10 client.28955 handle_command_reply: tid=0 2024-09-10T09:05:57.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.088+0000 7f9b598da740 1 client.28955 shutdown 2024-09-10T09:05:57.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.089+0000 7f9b598da740 1 -- 172.21.15.172:0/2043247393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44291d40 msgr2=0x55dc44295e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:57.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.089+0000 7f9b598da740 1 --2- 172.21.15.172:0/2043247393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44291d40 0x55dc44295e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009c10 tx=0x7f9b48009500 comp rx=0 tx=0).stop 2024-09-10T09:05:57.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.089+0000 7f9b598da740 1 -- 172.21.15.172:0/2043247393 shutdown_connections 2024-09-10T09:05:57.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.089+0000 7f9b598da740 1 --2- 172.21.15.172:0/2043247393 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55dc4430cfd0 0x55dc4430f410 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.089+0000 7f9b598da740 1 --2- 172.21.15.172:0/2043247393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc44291d40 0x55dc44295e10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.089+0000 7f9b598da740 1 --2- 172.21.15.172:0/2043247393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc44287480 0x55dc44295820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.089+0000 7f9b38ff9640 0 client.28955 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:57.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.089+0000 7f9b598da740 1 --2- 172.21.15.172:0/2043247393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc44286b60 0x55dc4429a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.089+0000 7f9b598da740 1 -- 172.21.15.172:0/2043247393 >> 172.21.15.172:0/2043247393 conn(0x55dc442830f0 msgr2=0x55dc44281a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:57.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.089+0000 7f9b598da740 1 -- 172.21.15.172:0/2043247393 shutdown_connections 2024-09-10T09:05:57.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.089+0000 7f9b598da740 1 -- 172.21.15.172:0/2043247393 wait complete. 2024-09-10T09:05:57.098 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.089+0000 7f9b598da740 20 client.28955 trim_cache size 0 max 16384 2024-09-10T09:05:57.098 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:05:57.098 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0, 2024-09-10T09:05:57.098 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_tag": "84872624-e628-45a1-bb5e-2ecf00a946e9", 2024-09-10T09:05:57.099 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": "asynchronous" 2024-09-10T09:05:57.099 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:05:57.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.089+0000 7f9b57501640 1 -- 172.21.15.172:0/1974035849 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9b3c0a2b80 msgr2=0x7f9b3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:57.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.089+0000 7f9b57501640 1 --2- 172.21.15.172:0/1974035849 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9b3c0a2b80 0x7f9b3c0a5040 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9b48005c40 tx=0x7f9b48004a40 comp rx=0 tx=0).stop 2024-09-10T09:05:57.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.089+0000 7f9b57501640 1 -- 172.21.15.172:0/1974035849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b501009b0 msgr2=0x7f9b5019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:57.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.089+0000 7f9b57501640 1 --2- 172.21.15.172:0/1974035849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b501009b0 0x7f9b5019a6f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00e9f0 tx=0x7f9b4c00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.090+0000 7f9b57501640 1 -- 172.21.15.172:0/1974035849 shutdown_connections 2024-09-10T09:05:57.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.090+0000 7f9b57501640 1 --2- 172.21.15.172:0/1974035849 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f9b3c0a2b80 0x7f9b3c0a5040 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.090+0000 7f9b57501640 1 --2- 172.21.15.172:0/1974035849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b5010e4d0 0x7f9b5019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.090+0000 7f9b57501640 1 --2- 172.21.15.172:0/1974035849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b50101360 0x7f9b5019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.090+0000 7f9b57501640 1 --2- 172.21.15.172:0/1974035849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b501009b0 0x7f9b5019a6f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:57.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.090+0000 7f9b57501640 1 -- 172.21.15.172:0/1974035849 >> 172.21.15.172:0/1974035849 conn(0x7f9b500fc7d0 msgr2=0x7f9b500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:57.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.090+0000 7f9b57501640 1 -- 172.21.15.172:0/1974035849 shutdown_connections 2024-09-10T09:05:57.100 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:57.090+0000 7f9b57501640 1 -- 172.21.15.172:0/1974035849 wait complete. 2024-09-10T09:05:57.837 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:05:57.860 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:57 smithi172 ceph-mon[30854]: scrub queued for path: / 2024-09-10T09:05:57.860 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:57 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:57.860 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:57 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T09:05:57.860 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:57 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T09:05:57.861 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:57 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T09:05:57.861 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:57 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T09:05:57.861 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:57 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:57.861 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:57 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:57.861 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:57 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:57.861 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:57 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:57.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:57 smithi174 ceph-mon[34068]: scrub queued for path: / 2024-09-10T09:05:57.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:57 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:57.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:57 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T09:05:57.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:57 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T09:05:57.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:57 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T09:05:57.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:57 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T09:05:57.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:57 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:57.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:57 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:57.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:57 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:57.993 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:57 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:58.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:57 smithi191 ceph-mon[35314]: scrub queued for path: / 2024-09-10T09:05:58.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:57 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:58.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:57 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T09:05:58.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:57 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T09:05:58.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:57 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T09:05:58.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:57 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T09:05:58.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:57 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:58.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:57 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:58.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:57 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:58.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:57 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:58.106 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:05:58.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.706+0000 7fd4dd2a5640 1 -- 172.21.15.172:0/1125372753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4d810e3f0 msgr2=0x7fd4d8115fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:58.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.706+0000 7fd4dd2a5640 1 --2- 172.21.15.172:0/1125372753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4d810e3f0 0x7fd4d8115fb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00098c0 tx=0x7fd4c002c560 comp rx=0 tx=0).stop 2024-09-10T09:05:58.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.707+0000 7fd4dd2a5640 1 -- 172.21.15.172:0/1125372753 shutdown_connections 2024-09-10T09:05:58.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.707+0000 7fd4dd2a5640 1 --2- 172.21.15.172:0/1125372753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4d810e3f0 0x7fd4d8115fb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.707+0000 7fd4dd2a5640 1 --2- 172.21.15.172:0/1125372753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4d806f100 0x7fd4d810dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.707+0000 7fd4dd2a5640 1 --2- 172.21.15.172:0/1125372753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4d806e750 0x7fd4d806eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.707+0000 7fd4dd2a5640 1 -- 172.21.15.172:0/1125372753 >> 172.21.15.172:0/1125372753 conn(0x7fd4d8073180 msgr2=0x7fd4d8073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:58.718 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.707+0000 7fd4dd2a5640 1 -- 172.21.15.172:0/1125372753 shutdown_connections 2024-09-10T09:05:58.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.707+0000 7fd4dd2a5640 1 -- 172.21.15.172:0/1125372753 wait complete. 2024-09-10T09:05:58.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.708+0000 7fd4dd2a5640 1 Processor -- start 2024-09-10T09:05:58.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.708+0000 7fd4dd2a5640 1 -- start start 2024-09-10T09:05:58.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.709+0000 7fd4dd2a5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4d806e750 0x7fd4d8079f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.709+0000 7fd4dd2a5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4d806f100 0x7fd4d807a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.709+0000 7fd4dd2a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4d810e3f0 0x7fd4d80750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.709+0000 7fd4d7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4d806e750 0x7fd4d8079f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:58.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.709+0000 7fd4d7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4d806e750 0x7fd4d8079f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:40252/0 (socket says 172.21.15.172:40252) 2024-09-10T09:05:58.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.709+0000 7fd4d7fff640 1 -- 172.21.15.172:0/1026785058 learned_addr learned my addr 172.21.15.172:0/1026785058 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:58.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.709+0000 7fd4d77fe640 1 --2- 172.21.15.172:0/1026785058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4d806f100 0x7fd4d807a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:58.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.709+0000 7fd4dd2a5640 1 -- 172.21.15.172:0/1026785058 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d8118280 con 0x7fd4d810e3f0 2024-09-10T09:05:58.719 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.709+0000 7fd4dd2a5640 1 -- 172.21.15.172:0/1026785058 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d8118100 con 0x7fd4d806f100 2024-09-10T09:05:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.709+0000 7fd4dd2a5640 1 -- 172.21.15.172:0/1026785058 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d8118400 con 0x7fd4d806e750 2024-09-10T09:05:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.709+0000 7fd4dcaa4640 1 --2- 172.21.15.172:0/1026785058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4d810e3f0 0x7fd4d80750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.710+0000 7fd4d7fff640 1 -- 172.21.15.172:0/1026785058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4d806f100 msgr2=0x7fd4d807a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.710+0000 7fd4d7fff640 1 --2- 172.21.15.172:0/1026785058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4d806f100 0x7fd4d807a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.710+0000 7fd4d7fff640 1 -- 172.21.15.172:0/1026785058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4d810e3f0 msgr2=0x7fd4d80750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.710+0000 7fd4d7fff640 1 --2- 172.21.15.172:0/1026785058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4d810e3f0 0x7fd4d80750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.710+0000 7fd4d7fff640 1 -- 172.21.15.172:0/1026785058 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d8075960 con 0x7fd4d806e750 2024-09-10T09:05:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.711+0000 7fd4d7fff640 1 --2- 172.21.15.172:0/1026785058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4d806e750 0x7fd4d8079f80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000efb0 tx=0x7fd4d000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.711+0000 7fd4d57fa640 1 -- 172.21.15.172:0/1026785058 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d001b070 con 0x7fd4d806e750 2024-09-10T09:05:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.711+0000 7fd4d57fa640 1 -- 172.21.15.172:0/1026785058 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4d000ba20 con 0x7fd4d806e750 2024-09-10T09:05:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.711+0000 7fd4d57fa640 1 -- 172.21.15.172:0/1026785058 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0016ba0 con 0x7fd4d806e750 2024-09-10T09:05:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.711+0000 7fd4dd2a5640 1 -- 172.21.15.172:0/1026785058 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d8075c50 con 0x7fd4d806e750 2024-09-10T09:05:58.720 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.711+0000 7fd4dd2a5640 1 -- 172.21.15.172:0/1026785058 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d8081e80 con 0x7fd4d806e750 2024-09-10T09:05:58.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.717+0000 7fd4e08e0740 1 Processor -- start 2024-09-10T09:05:58.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.718+0000 7fd4e08e0740 1 -- start start 2024-09-10T09:05:58.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.718+0000 7fd4e08e0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d78b0 0x560b591d7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.718+0000 7fd4e08e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 0x560b591d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.718+0000 7fd4e08e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d2870 0x560b591d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.718+0000 7fd4e08e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560b5912e2e0 con 0x560b591d2870 2024-09-10T09:05:58.722 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.718+0000 7fd4e08e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560b590d5ce0 con 0x560b591d81d0 2024-09-10T09:05:58.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.718+0000 7fd4e08e0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560b5912a2a0 con 0x560b591d78b0 2024-09-10T09:05:58.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.718+0000 7fd4d7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 0x560b591d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:58.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.718+0000 7fd4d7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 0x560b591d2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:35288/0 (socket says 172.21.15.172:35288) 2024-09-10T09:05:58.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.718+0000 7fd4d7fff640 1 -- 172.21.15.172:0/1548724213 learned_addr learned my addr 172.21.15.172:0/1548724213 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:58.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.719+0000 7fd4d77fe640 1 --2- 172.21.15.172:0/1548724213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d2870 0x560b591d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:58.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.719+0000 7fd4d7fff640 1 -- 172.21.15.172:0/1548724213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d78b0 msgr2=0x560b591d7c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:05:58.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.719+0000 7fd4d7fff640 1 --2- 172.21.15.172:0/1548724213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d78b0 0x560b591d7c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.719+0000 7fd4d7fff640 1 -- 172.21.15.172:0/1548724213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d2870 msgr2=0x560b591d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:58.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.719+0000 7fd4d7fff640 1 --2- 172.21.15.172:0/1548724213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d2870 0x560b591d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.723 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.719+0000 7fd4d7fff640 1 -- 172.21.15.172:0/1548724213 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b590ba440 con 0x560b591d81d0 2024-09-10T09:05:58.724 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.720+0000 7fd4d7fff640 1 --2- 172.21.15.172:0/1548724213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 0x560b591d2200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009b90 tx=0x7fd4d000b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=663e925212f7f2d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:58.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.723+0000 7fd4d57fa640 1 -- 172.21.15.172:0/1026785058 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd4d004eaf0 con 0x7fd4d806e750 2024-09-10T09:05:58.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.724+0000 7fd4ce7fc640 1 -- 172.21.15.172:0/1548724213 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d00d2f20 con 0x560b591d81d0 2024-09-10T09:05:58.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.724+0000 7fd4d57fa640 1 --2- 172.21.15.172:0/1026785058 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd4c40a2d50 0x7fd4c40a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.724+0000 7fd4ce7fc640 1 -- 172.21.15.172:0/1548724213 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4d0053070 con 0x560b591d81d0 2024-09-10T09:05:58.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.724+0000 7fd4ce7fc640 1 -- 172.21.15.172:0/1548724213 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d00d4810 con 0x560b591d81d0 2024-09-10T09:05:58.725 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.724+0000 7fd4d57fa640 1 -- 172.21.15.172:0/1026785058 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd4d00d29b0 con 0x7fd4d806e750 2024-09-10T09:05:58.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.725+0000 7fd4dcaa4640 1 --2- 172.21.15.172:0/1026785058 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd4c40a2d50 0x7fd4c40a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:58.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.725+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1548724213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 msgr2=0x560b591d2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:58.726 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.725+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/1548724213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 0x560b591d2200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009b90 tx=0x7fd4d000b940 comp rx=0 tx=0).stop 2024-09-10T09:05:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.729+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1548724213 shutdown_connections 2024-09-10T09:05:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.729+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/1548724213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d2870 0x560b591d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.729+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/1548724213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 0x560b591d2200 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.729+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/1548724213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d78b0 0x560b591d7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.729+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1548724213 >> 172.21.15.172:0/1548724213 conn(0x560b591c46b0 msgr2=0x560b591c4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:58.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.730+0000 7fd4dcaa4640 1 --2- 172.21.15.172:0/1026785058 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd4c40a2d50 0x7fd4c40a5210 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd4d807abb0 tx=0x7fd4c000e420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:58.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.730+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1548724213 shutdown_connections 2024-09-10T09:05:58.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.730+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1548724213 wait complete. 2024-09-10T09:05:58.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.730+0000 7fd4e08e0740 1 -- start start 2024-09-10T09:05:58.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.731+0000 7fd4e08e0740 1 Processor -- start 2024-09-10T09:05:58.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.731+0000 7fd4e08e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d2870 0x560b591de1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.731+0000 7fd4e08e0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d78b0 0x560b591db490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.731+0000 7fd4e08e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 0x560b591dbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.731+0000 7fd4e08e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560b591de9b0 con 0x560b591d2870 2024-09-10T09:05:58.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.731+0000 7fd4e08e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560b590e9050 con 0x560b591d81d0 2024-09-10T09:05:58.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.731+0000 7fd4e08e0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560b591deb30 con 0x560b591d78b0 2024-09-10T09:05:58.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.731+0000 7fd4e08e0740 10 client.0 _ll_register_callbacks cb 0x560b59129160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:58.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.732+0000 7fd4dcaa4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 0x560b591dbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:58.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.732+0000 7fd4d77fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d78b0 0x560b591db490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:58.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.732+0000 7fd4dcaa4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 0x560b591dbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:35298/0 (socket says 172.21.15.172:35298) 2024-09-10T09:05:58.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.732+0000 7fd4dcaa4640 1 -- 172.21.15.172:0/2575925850 learned_addr learned my addr 172.21.15.172:0/2575925850 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:58.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.732+0000 7fd4dcaa4640 1 -- 172.21.15.172:0/2575925850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d78b0 msgr2=0x560b591db490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:58.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.732+0000 7fd4dcaa4640 1 --2- 172.21.15.172:0/2575925850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d78b0 0x560b591db490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.732+0000 7fd4dcaa4640 1 -- 172.21.15.172:0/2575925850 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d2870 msgr2=0x560b591de1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:58.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.732+0000 7fd4dcaa4640 1 --2- 172.21.15.172:0/2575925850 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d2870 0x560b591de1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.732+0000 7fd4dcaa4640 1 -- 172.21.15.172:0/2575925850 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b58e9f3c0 con 0x560b591d81d0 2024-09-10T09:05:58.735 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.733+0000 7fd4dcaa4640 1 --2- 172.21.15.172:0/2575925850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 0x560b591dbbd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00098c0 tx=0x7fd4c002c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f23c274ecd8c879 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:58.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.733+0000 7fd4b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:05:58.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.733+0000 7fd4b77fe640 1 -- 172.21.15.172:0/2575925850 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c004d070 con 0x560b591d81d0 2024-09-10T09:05:58.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.733+0000 7fd4b77fe640 1 -- 172.21.15.172:0/2575925850 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd4c0049900 con 0x560b591d81d0 2024-09-10T09:05:58.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.733+0000 7fd4e08e0740 1 -- 172.21.15.172:0/2575925850 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4d0053070 con 0x560b591d81d0 2024-09-10T09:05:58.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.734+0000 7fd4b77fe640 1 -- 172.21.15.172:0/2575925850 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd4c0057050 con 0x560b591d81d0 2024-09-10T09:05:58.736 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.734+0000 7fd4e08e0740 10 client.28967 fetch_fsmap learned FSMap version 45 2024-09-10T09:05:58.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.734+0000 7fd4e08e0740 1 -- 172.21.15.172:0/2575925850 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x560b591ea4e0 con 0x560b591d81d0 2024-09-10T09:05:58.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.734+0000 7fd4b77fe640 1 -- 172.21.15.172:0/2575925850 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4d0053070 con 0x560b591d81d0 2024-09-10T09:05:58.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.734+0000 7fd4b77fe640 1 -- 172.21.15.172:0/2575925850 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7fd4c0052b90 con 0x560b591d81d0 2024-09-10T09:05:58.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.734+0000 7fd4e08e0740 10 client.28967 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T09:05:58.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.734+0000 7fd4e08e0740 10 client.28967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:58.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.734+0000 7fd4e08e0740 20 client.28967 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:58.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.734+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/2575925850 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560b591eb9a0 0x560b591ede20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.734+0000 7fd4e08e0740 4 client.28967 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:05:58.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.734+0000 7fd4e08e0740 1 -- 172.21.15.172:0/2575925850 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b5913b850 con 0x560b591eb9a0 2024-09-10T09:05:58.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.735+0000 7fd4d7fff640 1 --2- 172.21.15.172:0/2575925850 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560b591eb9a0 0x560b591ede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:58.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.741+0000 7fd4d7fff640 1 --2- 172.21.15.172:0/2575925850 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560b591eb9a0 0x560b591ede20 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d03f4e4c9770d25 server_cookie=fd0aed2f3c8bb3ed in_seq=0 out_seq=0 2024-09-10T09:05:58.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.741+0000 7fd4b77fe640 10 client.28967 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:58.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.742+0000 7fd4b77fe640 1 -- 172.21.15.172:0/2575925850 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b5913b850 con 0x560b591eb9a0 2024-09-10T09:05:58.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.742+0000 7fd4b77fe640 10 client.28967 handle_command_reply: tid=0 2024-09-10T09:05:58.743 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.742+0000 7fd4e08e0740 1 client.28967 shutdown 2024-09-10T09:05:58.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.743+0000 7fd4e08e0740 1 -- 172.21.15.172:0/2575925850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 msgr2=0x560b591dbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:58.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.743+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/2575925850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 0x560b591dbbd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00098c0 tx=0x7fd4c002c560 comp rx=0 tx=0).stop 2024-09-10T09:05:58.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.743+0000 7fd4e08e0740 1 -- 172.21.15.172:0/2575925850 shutdown_connections 2024-09-10T09:05:58.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.743+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/2575925850 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560b591eb9a0 0x560b591ede20 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.744 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.743+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/2575925850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 0x560b591dbbd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.743+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/2575925850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d78b0 0x560b591db490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.743+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/2575925850 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d2870 0x560b591de1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.743+0000 7fd4e08e0740 1 -- 172.21.15.172:0/2575925850 >> 172.21.15.172:0/2575925850 conn(0x560b591c46b0 msgr2=0x560b591c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:58.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.743+0000 7fd4b77fe640 0 client.28967 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:58.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.743+0000 7fd4e08e0740 1 -- 172.21.15.172:0/2575925850 shutdown_connections 2024-09-10T09:05:58.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.743+0000 7fd4e08e0740 1 -- 172.21.15.172:0/2575925850 wait complete. 2024-09-10T09:05:58.745 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.744+0000 7fd4e08e0740 20 client.28967 trim_cache size 0 max 16384 2024-09-10T09:05:58.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.777+0000 7fd4e08e0740 1 Processor -- start 2024-09-10T09:05:58.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.778+0000 7fd4e08e0740 1 -- start start 2024-09-10T09:05:58.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.779+0000 7fd4e08e0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d2870 0x560b591d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.779+0000 7fd4e08e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d78b0 0x560b591e7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.779+0000 7fd4e08e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d81d0 0x560b591c3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.779+0000 7fd4e08e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d81a4110 con 0x560b591d81d0 2024-09-10T09:05:58.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.779+0000 7fd4e08e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d81a5c50 con 0x560b591d78b0 2024-09-10T09:05:58.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.779+0000 7fd4e08e0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d81a47e0 con 0x560b591d2870 2024-09-10T09:05:58.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.779+0000 7fd4dcaa4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d81d0 0x560b591c3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:58.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.779+0000 7fd4d7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d2870 0x560b591d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:58.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.779+0000 7fd4d7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d2870 0x560b591d2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:40268/0 (socket says 172.21.15.172:40268) 2024-09-10T09:05:58.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.779+0000 7fd4d7fff640 1 -- 172.21.15.172:0/1474226870 learned_addr learned my addr 172.21.15.172:0/1474226870 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:58.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.780+0000 7fd4dcaa4640 1 -- 172.21.15.172:0/1474226870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d2870 msgr2=0x560b591d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:58.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.780+0000 7fd4dcaa4640 1 --2- 172.21.15.172:0/1474226870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d2870 0x560b591d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.780+0000 7fd4dcaa4640 1 -- 172.21.15.172:0/1474226870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d78b0 msgr2=0x560b591e7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:58.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.780+0000 7fd4dcaa4640 1 --2- 172.21.15.172:0/1474226870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d78b0 0x560b591e7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.780+0000 7fd4dcaa4640 1 -- 172.21.15.172:0/1474226870 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b591e8190 con 0x560b591d81d0 2024-09-10T09:05:58.782 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.781+0000 7fd4dcaa4640 1 --2- 172.21.15.172:0/1474226870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d81d0 0x560b591c3500 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fd4c002ca80 tx=0x7fd4c003e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89ec0f33aa37d84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:58.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.782+0000 7fd4cdffb640 1 -- 172.21.15.172:0/1474226870 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c004d070 con 0x560b591d81d0 2024-09-10T09:05:58.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.782+0000 7fd4cdffb640 1 -- 172.21.15.172:0/1474226870 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4c0049700 con 0x560b591d81d0 2024-09-10T09:05:58.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.782+0000 7fd4cdffb640 1 -- 172.21.15.172:0/1474226870 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0042070 con 0x560b591d81d0 2024-09-10T09:05:58.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.783+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1474226870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d81d0 msgr2=0x560b591c3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:58.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.783+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/1474226870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d81d0 0x560b591c3500 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fd4c002ca80 tx=0x7fd4c003e770 comp rx=0 tx=0).stop 2024-09-10T09:05:58.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.785+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1474226870 shutdown_connections 2024-09-10T09:05:58.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.785+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/1474226870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d81d0 0x560b591c3500 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.785+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/1474226870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d78b0 0x560b591e7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.785+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/1474226870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d2870 0x560b591d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.785+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1474226870 >> 172.21.15.172:0/1474226870 conn(0x560b591c46b0 msgr2=0x560b591da580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:58.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.786+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1474226870 shutdown_connections 2024-09-10T09:05:58.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.786+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1474226870 wait complete. 2024-09-10T09:05:58.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.786+0000 7fd4e08e0740 1 -- start start 2024-09-10T09:05:58.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.787+0000 7fd4e08e0740 1 Processor -- start 2024-09-10T09:05:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.787+0000 7fd4e08e0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d2870 0x560b591e41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.787+0000 7fd4e08e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d78b0 0x560b591e46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.787+0000 7fd4e08e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 0x560b591f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.787+0000 7fd4e08e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d81a5580 con 0x560b591d78b0 2024-09-10T09:05:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.787+0000 7fd4e08e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d81a4eb0 con 0x560b591d81d0 2024-09-10T09:05:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.787+0000 7fd4e08e0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d81a20c0 con 0x560b591d2870 2024-09-10T09:05:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.787+0000 7fd4e08e0740 10 client.0 _ll_register_callbacks cb 0x7fd4d80688b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:05:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.788+0000 7fd4d7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d2870 0x560b591e41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.788+0000 7fd4d7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d2870 0x560b591e41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:40272/0 (socket says 172.21.15.172:40272) 2024-09-10T09:05:58.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.788+0000 7fd4d7fff640 1 -- 172.21.15.172:0/1633903337 learned_addr learned my addr 172.21.15.172:0/1633903337 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:05:58.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.788+0000 7fd4d7fff640 1 -- 172.21.15.172:0/1633903337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 msgr2=0x560b591f2ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:05:58.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.788+0000 7fd4d7fff640 1 --2- 172.21.15.172:0/1633903337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 0x560b591f2ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.788+0000 7fd4d7fff640 1 -- 172.21.15.172:0/1633903337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d78b0 msgr2=0x560b591e46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:05:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.788+0000 7fd4d7fff640 1 --2- 172.21.15.172:0/1633903337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d78b0 0x560b591e46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.788+0000 7fd4d7fff640 1 -- 172.21.15.172:0/1633903337 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b591f3710 con 0x560b591d2870 2024-09-10T09:05:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.789+0000 7fd4d7fff640 1 --2- 172.21.15.172:0/1633903337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d2870 0x560b591e41b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00d4000 tx=0x7fd4d00d3ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83020ed16c2f8d56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:05:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.789+0000 7fd4b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:05:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.789+0000 7fd4b77fe640 1 -- 172.21.15.172:0/1633903337 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d009e970 con 0x560b591d2870 2024-09-10T09:05:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.789+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1633903337 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4c0049700 con 0x560b591d2870 2024-09-10T09:05:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.789+0000 7fd4b77fe640 1 -- 172.21.15.172:0/1633903337 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fd4d00d0180 con 0x560b591d2870 2024-09-10T09:05:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.790+0000 7fd4b77fe640 1 -- 172.21.15.172:0/1633903337 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd4d009f710 con 0x560b591d2870 2024-09-10T09:05:58.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.790+0000 7fd4e08e0740 10 client.28864 fetch_fsmap learned FSMap version 45 2024-09-10T09:05:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.790+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1633903337 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x560b59256400 con 0x560b591d2870 2024-09-10T09:05:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.790+0000 7fd4b77fe640 1 -- 172.21.15.172:0/1633903337 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4c0049700 con 0x560b591d2870 2024-09-10T09:05:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.790+0000 7fd4b77fe640 1 -- 172.21.15.172:0/1633903337 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 45) ==== 8374+0+0 (secure 0 0 0) 0x7fd4d000b040 con 0x560b591d2870 2024-09-10T09:05:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.790+0000 7fd4e08e0740 10 client.28864 fetch_fsmap finished waiting for FSMap version 45 2024-09-10T09:05:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.790+0000 7fd4e08e0740 10 client.28864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:05:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.790+0000 7fd4e08e0740 20 client.28864 populate_metadata read hostname 'smithi172' 2024-09-10T09:05:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.790+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/1633903337 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560b59256b50 0x560b59258fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:05:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.790+0000 7fd4e08e0740 4 client.28864 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:05:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.790+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1633903337 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560b59135d60 con 0x560b59256b50 2024-09-10T09:05:58.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.791+0000 7fd4d77fe640 1 --2- 172.21.15.172:0/1633903337 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560b59256b50 0x560b59258fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:05:58.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.792+0000 7fd4d77fe640 1 --2- 172.21.15.172:0/1633903337 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560b59256b50 0x560b59258fb0 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef7ed173200b26f server_cookie=c5b8fb22cffe9ac6 in_seq=0 out_seq=0 2024-09-10T09:05:58.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.792+0000 7fd4b77fe640 10 client.28864 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:58.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.792+0000 7fd4b77fe640 1 -- 172.21.15.172:0/1633903337 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560b59135d60 con 0x560b59256b50 2024-09-10T09:05:58.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.792+0000 7fd4b77fe640 10 client.28864 handle_command_reply: tid=0 2024-09-10T09:05:58.794 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.793+0000 7fd4e08e0740 1 client.28864 shutdown 2024-09-10T09:05:58.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.797+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1633903337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d2870 msgr2=0x560b591e41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:58.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.797+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/1633903337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d2870 0x560b591e41b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00d4000 tx=0x7fd4d00d3ef0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.802 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:05:58.802 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:05:58.802 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": { 2024-09-10T09:05:58.802 INFO:teuthology.orchestra.run.smithi172.stdout: "84872624-e628-45a1-bb5e-2ecf00a946e9": { 2024-09-10T09:05:58.803 INFO:teuthology.orchestra.run.smithi172.stdout: "path": "/", 2024-09-10T09:05:58.803 INFO:teuthology.orchestra.run.smithi172.stdout: "tag": "84872624-e628-45a1-bb5e-2ecf00a946e9", 2024-09-10T09:05:58.803 INFO:teuthology.orchestra.run.smithi172.stdout: "options": "recursive,force" 2024-09-10T09:05:58.803 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:05:58.803 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:05:58.803 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:05:58.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.799+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1633903337 shutdown_connections 2024-09-10T09:05:58.804 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.799+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/1633903337 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x560b59256b50 0x560b59258fb0 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.799+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/1633903337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560b591d81d0 0x560b591f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.799+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/1633903337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b591d78b0 0x560b591e46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.799+0000 7fd4e08e0740 1 --2- 172.21.15.172:0/1633903337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560b591d2870 0x560b591e41b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.799+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1633903337 >> 172.21.15.172:0/1633903337 conn(0x560b591c46b0 msgr2=0x560b591dccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:58.805 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.800+0000 7fd4b77fe640 0 client.28864 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:05:58.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.800+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1633903337 shutdown_connections 2024-09-10T09:05:58.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.800+0000 7fd4e08e0740 1 -- 172.21.15.172:0/1633903337 wait complete. 2024-09-10T09:05:58.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.800+0000 7fd4e08e0740 20 client.28864 trim_cache size 0 max 16384 2024-09-10T09:05:58.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.802+0000 7fd4dd2a5640 1 -- 172.21.15.172:0/1026785058 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd4c40a2d50 msgr2=0x7fd4c40a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:58.806 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.802+0000 7fd4dd2a5640 1 --2- 172.21.15.172:0/1026785058 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd4c40a2d50 0x7fd4c40a5210 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd4d807abb0 tx=0x7fd4c000e420 comp rx=0 tx=0).stop 2024-09-10T09:05:58.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.802+0000 7fd4dd2a5640 1 -- 172.21.15.172:0/1026785058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4d806e750 msgr2=0x7fd4d8079f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:05:58.807 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.802+0000 7fd4dd2a5640 1 --2- 172.21.15.172:0/1026785058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4d806e750 0x7fd4d8079f80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000efb0 tx=0x7fd4d000c800 comp rx=0 tx=0).stop 2024-09-10T09:05:58.807 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:58 smithi172 ceph-mon[30854]: scrub summary: active paths [/] 2024-09-10T09:05:58.808 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:58 smithi172 ceph-mon[30854]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:58.808 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:58 smithi172 ceph-mon[30854]: pgmap v942: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 759 KiB/s wr, 202 op/s 2024-09-10T09:05:58.808 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.803+0000 7fd4dd2a5640 1 -- 172.21.15.172:0/1026785058 shutdown_connections 2024-09-10T09:05:58.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.803+0000 7fd4dd2a5640 1 --2- 172.21.15.172:0/1026785058 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fd4c40a2d50 0x7fd4c40a5210 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.803+0000 7fd4dd2a5640 1 --2- 172.21.15.172:0/1026785058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4d810e3f0 0x7fd4d80750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.803+0000 7fd4dd2a5640 1 --2- 172.21.15.172:0/1026785058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4d806f100 0x7fd4d807a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.803+0000 7fd4dd2a5640 1 --2- 172.21.15.172:0/1026785058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4d806e750 0x7fd4d8079f80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:05:58.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.803+0000 7fd4dd2a5640 1 -- 172.21.15.172:0/1026785058 >> 172.21.15.172:0/1026785058 conn(0x7fd4d8073180 msgr2=0x7fd4d806c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:05:58.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.805+0000 7fd4dd2a5640 1 -- 172.21.15.172:0/1026785058 shutdown_connections 2024-09-10T09:05:58.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:05:58.805+0000 7fd4dd2a5640 1 -- 172.21.15.172:0/1026785058 wait complete. 2024-09-10T09:05:58.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:58 smithi174 ceph-mon[34068]: scrub summary: active paths [/] 2024-09-10T09:05:58.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:58 smithi174 ceph-mon[34068]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:58.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:58 smithi174 ceph-mon[34068]: pgmap v942: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 759 KiB/s wr, 202 op/s 2024-09-10T09:05:59.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:58 smithi191 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-10T09:05:59.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:58 smithi191 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-10T09:05:59.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:58 smithi191 ceph-mon[35314]: pgmap v942: 129 pgs: 129 active+clean; 13 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 759 KiB/s wr, 202 op/s 2024-09-10T09:05:59.488 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T09:05:59.488 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T09:05:59.573 INFO:tasks.cephfs.filesystem:scrub status for tag:84872624-e628-45a1-bb5e-2ecf00a946e9 - {'path': '/', 'tag': '84872624-e628-45a1-bb5e-2ecf00a946e9', 'options': 'recursive,force'} 2024-09-10T09:05:59.591 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-09-10T09:05:59.592 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T09:05:59.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:05:59 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:05:59.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:05:59 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:06:00.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:05:59 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:06:00.214 DEBUG:teuthology.parallel:result is None 2024-09-10T09:06:00.214 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T09:06:00.297 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T09:06:00.298 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-10T09:06:00.307 INFO:tasks.quiescer:joining Quiescers 2024-09-10T09:06:00.308 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-10T09:06:00.308 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:06:00.772 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:06:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.835+0000 7fae7500e640 1 -- 172.21.15.172:0/2987795425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae700737d0 msgr2=0x7fae70112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.835+0000 7fae7500e640 1 --2- 172.21.15.172:0/2987795425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae700737d0 0x7fae70112540 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fae64009860 tx=0x7fae6402a360 comp rx=0 tx=0).stop 2024-09-10T09:06:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.835+0000 7fae7500e640 1 -- 172.21.15.172:0/2987795425 shutdown_connections 2024-09-10T09:06:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.835+0000 7fae7500e640 1 --2- 172.21.15.172:0/2987795425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae700737d0 0x7fae70112540 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.835+0000 7fae7500e640 1 --2- 172.21.15.172:0/2987795425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae70072d20 0x7fae70073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.835+0000 7fae7500e640 1 --2- 172.21.15.172:0/2987795425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae70109930 0x7fae70109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.835+0000 7fae7500e640 1 -- 172.21.15.172:0/2987795425 >> 172.21.15.172:0/2987795425 conn(0x7fae7006ece0 msgr2=0x7fae70071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.835+0000 7fae7500e640 1 -- 172.21.15.172:0/2987795425 shutdown_connections 2024-09-10T09:06:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.835+0000 7fae7500e640 1 -- 172.21.15.172:0/2987795425 wait complete. 2024-09-10T09:06:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.835+0000 7fae7500e640 1 Processor -- start 2024-09-10T09:06:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.836+0000 7fae7500e640 1 -- start start 2024-09-10T09:06:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.836+0000 7fae7500e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae70072d20 0x7fae7010be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.836+0000 7fae7500e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae700737d0 0x7fae7010a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.836+0000 7fae7500e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae70109930 0x7fae7010aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.836+0000 7fae7500e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fae70115e80 con 0x7fae70109930 2024-09-10T09:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.836+0000 7fae7500e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fae70115d00 con 0x7fae700737d0 2024-09-10T09:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.836+0000 7fae7500e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fae70116000 con 0x7fae70072d20 2024-09-10T09:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.836+0000 7fae6e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae700737d0 0x7fae7010a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.836+0000 7fae6e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae700737d0 0x7fae7010a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:35332/0 (socket says 172.21.15.172:35332) 2024-09-10T09:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.836+0000 7fae6e59c640 1 -- 172.21.15.172:0/1010893536 learned_addr learned my addr 172.21.15.172:0/1010893536 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.836+0000 7fae6ed9d640 1 --2- 172.21.15.172:0/1010893536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae70072d20 0x7fae7010be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:00.839 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.836+0000 7fae6f59e640 1 --2- 172.21.15.172:0/1010893536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae70109930 0x7fae7010aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.836+0000 7fae6e59c640 1 -- 172.21.15.172:0/1010893536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae70072d20 msgr2=0x7fae7010be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.836+0000 7fae6e59c640 1 --2- 172.21.15.172:0/1010893536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae70072d20 0x7fae7010be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.837+0000 7fae6e59c640 1 -- 172.21.15.172:0/1010893536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae70109930 msgr2=0x7fae7010aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.837+0000 7fae6e59c640 1 --2- 172.21.15.172:0/1010893536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae70109930 0x7fae7010aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.837+0000 7fae6e59c640 1 -- 172.21.15.172:0/1010893536 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae701a43d0 con 0x7fae700737d0 2024-09-10T09:06:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.837+0000 7fae6f59e640 1 --2- 172.21.15.172:0/1010893536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae70109930 0x7fae7010aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.837+0000 7fae6e59c640 1 --2- 172.21.15.172:0/1010893536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae700737d0 0x7fae7010a4c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fae6800eab0 tx=0x7fae6800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.837+0000 7fae57fff640 1 -- 172.21.15.172:0/1010893536 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6800cd80 con 0x7fae700737d0 2024-09-10T09:06:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.837+0000 7fae57fff640 1 -- 172.21.15.172:0/1010893536 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae68011920 con 0x7fae700737d0 2024-09-10T09:06:00.840 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.837+0000 7fae57fff640 1 -- 172.21.15.172:0/1010893536 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68010940 con 0x7fae700737d0 2024-09-10T09:06:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.837+0000 7fae7500e640 1 -- 172.21.15.172:0/1010893536 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae701a46c0 con 0x7fae700737d0 2024-09-10T09:06:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.837+0000 7fae7500e640 1 -- 172.21.15.172:0/1010893536 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae701a4bc0 con 0x7fae700737d0 2024-09-10T09:06:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.840+0000 7fae57fff640 1 -- 172.21.15.172:0/1010893536 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fae68011ad0 con 0x7fae700737d0 2024-09-10T09:06:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.840+0000 7fae57fff640 1 --2- 172.21.15.172:0/1010893536 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fae580a2c30 0x7fae580a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.840+0000 7fae57fff640 1 -- 172.21.15.172:0/1010893536 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fae680cf9f0 con 0x7fae700737d0 2024-09-10T09:06:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.841+0000 7fae6ed9d640 1 --2- 172.21.15.172:0/1010893536 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fae580a2c30 0x7fae580a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.841+0000 7fae77e83740 1 Processor -- start 2024-09-10T09:06:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.841+0000 7fae6ed9d640 1 --2- 172.21.15.172:0/1010893536 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fae580a2c30 0x7fae580a50f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fae60005fd0 tx=0x7fae60009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.846+0000 7fae77e83740 1 -- start start 2024-09-10T09:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.846+0000 7fae77e83740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3a59a0 0x55d88d3a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.846+0000 7fae77e83740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a62e0 0x55d88d3aada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.846+0000 7fae77e83740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3ab370 0x55d88d3ad830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.846+0000 7fae77e83740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d88d3092f0 con 0x55d88d3ab370 2024-09-10T09:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.846+0000 7fae77e83740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d88d2b1270 con 0x55d88d3a62e0 2024-09-10T09:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.846+0000 7fae77e83740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d88d3052b0 con 0x55d88d3a59a0 2024-09-10T09:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.847+0000 7fae6f59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3ab370 0x55d88d3ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.847+0000 7fae6f59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3ab370 0x55d88d3ad830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45616/0 (socket says 172.21.15.172:45616) 2024-09-10T09:06:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.847+0000 7fae6f59e640 1 -- 172.21.15.172:0/2758367158 learned_addr learned my addr 172.21.15.172:0/2758367158 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.847+0000 7fae6e59c640 1 --2- 172.21.15.172:0/2758367158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a62e0 0x55d88d3aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.847+0000 7fae6f59e640 1 -- 172.21.15.172:0/2758367158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3a59a0 msgr2=0x55d88d3a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.847+0000 7fae6f59e640 1 --2- 172.21.15.172:0/2758367158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3a59a0 0x55d88d3a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.847+0000 7fae6f59e640 1 -- 172.21.15.172:0/2758367158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a62e0 msgr2=0x55d88d3aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.847+0000 7fae6f59e640 1 --2- 172.21.15.172:0/2758367158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a62e0 0x55d88d3aada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.847+0000 7fae6f59e640 1 -- 172.21.15.172:0/2758367158 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d88d2b0340 con 0x55d88d3ab370 2024-09-10T09:06:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.848+0000 7fae6e59c640 1 --2- 172.21.15.172:0/2758367158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a62e0 0x55d88d3aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:06:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.848+0000 7fae6f59e640 1 --2- 172.21.15.172:0/2758367158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3ab370 0x55d88d3ad830 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fae64035040 tx=0x7fae64009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9bbaa60b872b557 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.848+0000 7fae557fa640 1 -- 172.21.15.172:0/2758367158 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64037040 con 0x55d88d3ab370 2024-09-10T09:06:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.848+0000 7fae557fa640 1 -- 172.21.15.172:0/2758367158 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae64004030 con 0x55d88d3ab370 2024-09-10T09:06:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.849+0000 7fae77e83740 1 -- 172.21.15.172:0/2758367158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3ab370 msgr2=0x55d88d3ad830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.849+0000 7fae77e83740 1 --2- 172.21.15.172:0/2758367158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3ab370 0x55d88d3ad830 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fae64035040 tx=0x7fae64009580 comp rx=0 tx=0).stop 2024-09-10T09:06:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.850+0000 7fae77e83740 1 -- 172.21.15.172:0/2758367158 shutdown_connections 2024-09-10T09:06:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.850+0000 7fae77e83740 1 --2- 172.21.15.172:0/2758367158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3ab370 0x55d88d3ad830 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.850+0000 7fae77e83740 1 --2- 172.21.15.172:0/2758367158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a62e0 0x55d88d3aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.850+0000 7fae77e83740 1 --2- 172.21.15.172:0/2758367158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3a59a0 0x55d88d3a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.850+0000 7fae77e83740 1 -- 172.21.15.172:0/2758367158 >> 172.21.15.172:0/2758367158 conn(0x55d88d3a1010 msgr2=0x55d88d3a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.852+0000 7fae77e83740 1 -- 172.21.15.172:0/2758367158 shutdown_connections 2024-09-10T09:06:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.854+0000 7fae77e83740 1 -- 172.21.15.172:0/2758367158 wait complete. 2024-09-10T09:06:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.855+0000 7fae77e83740 1 -- start start 2024-09-10T09:06:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.855+0000 7fae77e83740 1 Processor -- start 2024-09-10T09:06:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.859+0000 7fae77e83740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3a59a0 0x55d88d3afc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.859+0000 7fae77e83740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a62e0 0x55d88d3b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.859+0000 7fae77e83740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 0x55d88d3b7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.859+0000 7fae77e83740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d88d3ae330 con 0x55d88d3a59a0 2024-09-10T09:06:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.859+0000 7fae77e83740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d88d292290 con 0x55d88d3a62e0 2024-09-10T09:06:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.859+0000 7fae77e83740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d88d3ae4b0 con 0x55d88d3ab370 2024-09-10T09:06:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.859+0000 7fae77e83740 10 client.0 _ll_register_callbacks cb 0x55d88d2c3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:06:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.862+0000 7fae6e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a62e0 0x55d88d3b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.863+0000 7fae6f59e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 0x55d88d3b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.863+0000 7fae6f59e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 0x55d88d3b7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:40316/0 (socket says 172.21.15.172:40316) 2024-09-10T09:06:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.863+0000 7fae6f59e640 1 -- 172.21.15.172:0/279960650 learned_addr learned my addr 172.21.15.172:0/279960650 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.866+0000 7fae6f59e640 1 -- 172.21.15.172:0/279960650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a62e0 msgr2=0x55d88d3b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.866+0000 7fae6f59e640 1 --2- 172.21.15.172:0/279960650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a62e0 0x55d88d3b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.866+0000 7fae6f59e640 1 -- 172.21.15.172:0/279960650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3a59a0 msgr2=0x55d88d3afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.866+0000 7fae6f59e640 1 --2- 172.21.15.172:0/279960650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3a59a0 0x55d88d3afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.866+0000 7fae6f59e640 1 -- 172.21.15.172:0/279960650 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88d01c210 con 0x55d88d3ab370 2024-09-10T09:06:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.866+0000 7fae6e59c640 1 --2- 172.21.15.172:0/279960650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a62e0 0x55d88d3b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:06:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.872+0000 7fae6f59e640 1 --2- 172.21.15.172:0/279960650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 0x55d88d3b7190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fae64009780 tx=0x7fae64030d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54cdd5fdd8023302 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.872+0000 7fae52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:06:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.872+0000 7fae52ff5640 1 -- 172.21.15.172:0/279960650 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64037030 con 0x55d88d3ab370 2024-09-10T09:06:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.872+0000 7fae77e83740 1 -- 172.21.15.172:0/279960650 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae64004030 con 0x55d88d3ab370 2024-09-10T09:06:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.872+0000 7fae52ff5640 1 -- 172.21.15.172:0/279960650 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fae640035f0 con 0x55d88d3ab370 2024-09-10T09:06:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.873+0000 7fae52ff5640 1 -- 172.21.15.172:0/279960650 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fae64059050 con 0x55d88d3ab370 2024-09-10T09:06:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.874+0000 7fae77e83740 10 client.28873 fetch_fsmap learned FSMap version 46 2024-09-10T09:06:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.874+0000 7fae77e83740 1 -- 172.21.15.172:0/279960650 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d88d0e8620 con 0x55d88d3ab370 2024-09-10T09:06:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.875+0000 7fae52ff5640 1 -- 172.21.15.172:0/279960650 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae64004030 con 0x55d88d3ab370 2024-09-10T09:06:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.875+0000 7fae52ff5640 1 -- 172.21.15.172:0/279960650 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7fae64004a40 con 0x55d88d3ab370 2024-09-10T09:06:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.875+0000 7fae77e83740 10 client.28873 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T09:06:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.875+0000 7fae77e83740 10 client.28873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:06:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.875+0000 7fae77e83740 20 client.28873 populate_metadata read hostname 'smithi172' 2024-09-10T09:06:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.876+0000 7fae77e83740 1 --2- 172.21.15.172:0/279960650 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d88d3bb890 0x55d88d3bdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.876+0000 7fae77e83740 4 client.28873 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:06:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.876+0000 7fae77e83740 1 -- 172.21.15.172:0/279960650 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d88d316a20 con 0x55d88d3bb890 2024-09-10T09:06:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.878+0000 7fae6ed9d640 1 --2- 172.21.15.172:0/279960650 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d88d3bb890 0x55d88d3bdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.888+0000 7fae6ed9d640 1 --2- 172.21.15.172:0/279960650 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d88d3bb890 0x55d88d3bdd30 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b31a063e96ba8e78 server_cookie=512e644eb4be840f in_seq=0 out_seq=0 2024-09-10T09:06:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.888+0000 7fae52ff5640 10 client.28873 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:06:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.893+0000 7fae52ff5640 1 -- 172.21.15.172:0/279960650 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d88d316a20 con 0x55d88d3bb890 2024-09-10T09:06:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.893+0000 7fae52ff5640 10 client.28873 handle_command_reply: tid=0 2024-09-10T09:06:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.894+0000 7fae77e83740 1 client.28873 shutdown 2024-09-10T09:06:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.896+0000 7fae77e83740 1 -- 172.21.15.172:0/279960650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 msgr2=0x55d88d3b7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.896+0000 7fae77e83740 1 --2- 172.21.15.172:0/279960650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 0x55d88d3b7190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fae64009780 tx=0x7fae64030d90 comp rx=0 tx=0).stop 2024-09-10T09:06:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.896+0000 7fae77e83740 1 -- 172.21.15.172:0/279960650 shutdown_connections 2024-09-10T09:06:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.896+0000 7fae77e83740 1 --2- 172.21.15.172:0/279960650 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d88d3bb890 0x55d88d3bdd30 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.896+0000 7fae77e83740 1 --2- 172.21.15.172:0/279960650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 0x55d88d3b7190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.896+0000 7fae77e83740 1 --2- 172.21.15.172:0/279960650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a62e0 0x55d88d3b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.896+0000 7fae77e83740 1 --2- 172.21.15.172:0/279960650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3a59a0 0x55d88d3afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.896+0000 7fae77e83740 1 -- 172.21.15.172:0/279960650 >> 172.21.15.172:0/279960650 conn(0x55d88d3a1010 msgr2=0x55d88d3a2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.897+0000 7fae52ff5640 0 client.28873 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:06:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.898+0000 7fae77e83740 1 -- 172.21.15.172:0/279960650 shutdown_connections 2024-09-10T09:06:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.906+0000 7fae77e83740 1 -- 172.21.15.172:0/279960650 wait complete. 2024-09-10T09:06:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.906+0000 7fae77e83740 20 client.28873 trim_cache size 0 max 16384 2024-09-10T09:06:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.938+0000 7fae77e83740 1 Processor -- start 2024-09-10T09:06:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.938+0000 7fae77e83740 1 -- start start 2024-09-10T09:06:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.938+0000 7fae77e83740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3a59a0 0x55d88d3b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.938+0000 7fae77e83740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a62e0 0x55d88d3b7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.938+0000 7fae77e83740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 0x55d88d3a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.939+0000 7fae77e83740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fae701c1bf0 con 0x55d88d3a59a0 2024-09-10T09:06:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.939+0000 7fae77e83740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fae701c3790 con 0x55d88d3a62e0 2024-09-10T09:06:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.939+0000 7fae77e83740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fae701c22c0 con 0x55d88d3ab370 2024-09-10T09:06:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.939+0000 7fae6f59e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 0x55d88d3a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.939+0000 7fae6ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3a59a0 0x55d88d3b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.939+0000 7fae6ed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3a59a0 0x55d88d3b01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45644/0 (socket says 172.21.15.172:45644) 2024-09-10T09:06:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.939+0000 7fae6ed9d640 1 -- 172.21.15.172:0/1361440834 learned_addr learned my addr 172.21.15.172:0/1361440834 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.939+0000 7fae6ed9d640 1 -- 172.21.15.172:0/1361440834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 msgr2=0x55d88d3a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.939+0000 7fae6ed9d640 1 --2- 172.21.15.172:0/1361440834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 0x55d88d3a00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.939+0000 7fae6ed9d640 1 -- 172.21.15.172:0/1361440834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a62e0 msgr2=0x55d88d3b7150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:06:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.939+0000 7fae6ed9d640 1 --2- 172.21.15.172:0/1361440834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a62e0 0x55d88d3b7150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.939+0000 7fae6ed9d640 1 -- 172.21.15.172:0/1361440834 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d88d3af720 con 0x55d88d3a59a0 2024-09-10T09:06:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.940+0000 7fae6ed9d640 1 --2- 172.21.15.172:0/1361440834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3a59a0 0x55d88d3b01e0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fae600045f0 tx=0x7fae60013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d033ab10a2862 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.943+0000 7fae54ff9640 1 -- 172.21.15.172:0/1361440834 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60005020 con 0x55d88d3a59a0 2024-09-10T09:06:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.943+0000 7fae54ff9640 1 -- 172.21.15.172:0/1361440834 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae600047a0 con 0x55d88d3a59a0 2024-09-10T09:06:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.944+0000 7fae54ff9640 1 -- 172.21.15.172:0/1361440834 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60026530 con 0x55d88d3a59a0 2024-09-10T09:06:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.946+0000 7fae77e83740 1 -- 172.21.15.172:0/1361440834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3a59a0 msgr2=0x55d88d3b01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.946+0000 7fae77e83740 1 --2- 172.21.15.172:0/1361440834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3a59a0 0x55d88d3b01e0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fae600045f0 tx=0x7fae60013040 comp rx=0 tx=0).stop 2024-09-10T09:06:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.949+0000 7fae77e83740 1 -- 172.21.15.172:0/1361440834 shutdown_connections 2024-09-10T09:06:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.949+0000 7fae77e83740 1 --2- 172.21.15.172:0/1361440834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 0x55d88d3a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.949+0000 7fae77e83740 1 --2- 172.21.15.172:0/1361440834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a62e0 0x55d88d3b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.949+0000 7fae77e83740 1 --2- 172.21.15.172:0/1361440834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3a59a0 0x55d88d3b01e0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.949+0000 7fae77e83740 1 -- 172.21.15.172:0/1361440834 >> 172.21.15.172:0/1361440834 conn(0x55d88d3a1010 msgr2=0x55d88d39f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.953+0000 7fae77e83740 1 -- 172.21.15.172:0/1361440834 shutdown_connections 2024-09-10T09:06:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.953+0000 7fae77e83740 1 -- 172.21.15.172:0/1361440834 wait complete. 2024-09-10T09:06:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.954+0000 7fae77e83740 1 -- start start 2024-09-10T09:06:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.954+0000 7fae77e83740 1 Processor -- start 2024-09-10T09:06:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.957+0000 7fae77e83740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a59a0 0x55d88d3b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.957+0000 7fae77e83740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3a62e0 0x55d88d3b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.957+0000 7fae77e83740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 0x55d88d4245c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.957+0000 7fae77e83740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fae701c30c0 con 0x55d88d3a62e0 2024-09-10T09:06:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.957+0000 7fae77e83740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fae701c2990 con 0x55d88d3a59a0 2024-09-10T09:06:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.957+0000 7fae77e83740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fae701bfca0 con 0x55d88d3ab370 2024-09-10T09:06:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.957+0000 7fae77e83740 10 client.0 _ll_register_callbacks cb 0x7fae60005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:06:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.958+0000 7fae6ed9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a59a0 0x55d88d3b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.958+0000 7fae6f59e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 0x55d88d4245c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.958+0000 7fae6f59e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 0x55d88d4245c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:40320/0 (socket says 172.21.15.172:40320) 2024-09-10T09:06:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.958+0000 7fae6f59e640 1 -- 172.21.15.172:0/3629172025 learned_addr learned my addr 172.21.15.172:0/3629172025 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.959+0000 7fae6e59c640 1 --2- 172.21.15.172:0/3629172025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3a62e0 0x55d88d3b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.960+0000 7fae6f59e640 1 -- 172.21.15.172:0/3629172025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a59a0 msgr2=0x55d88d3b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.960+0000 7fae6f59e640 1 --2- 172.21.15.172:0/3629172025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a59a0 0x55d88d3b3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.960+0000 7fae6f59e640 1 -- 172.21.15.172:0/3629172025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3a62e0 msgr2=0x55d88d3b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.960+0000 7fae6f59e640 1 --2- 172.21.15.172:0/3629172025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3a62e0 0x55d88d3b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.960+0000 7fae6f59e640 1 -- 172.21.15.172:0/3629172025 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88d3b73b0 con 0x55d88d3ab370 2024-09-10T09:06:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.961+0000 7fae6f59e640 1 --2- 172.21.15.172:0/3629172025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 0x55d88d4245c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fae64000c00 tx=0x7fae64030fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f9cc1665a48c3cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.961+0000 7fae52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:06:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.964+0000 7fae52ff5640 1 -- 172.21.15.172:0/3629172025 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64037050 con 0x55d88d3ab370 2024-09-10T09:06:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.964+0000 7fae52ff5640 1 -- 172.21.15.172:0/3629172025 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fae640046c0 con 0x55d88d3ab370 2024-09-10T09:06:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.964+0000 7fae77e83740 1 -- 172.21.15.172:0/3629172025 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae600047a0 con 0x55d88d3ab370 2024-09-10T09:06:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.966+0000 7fae52ff5640 1 -- 172.21.15.172:0/3629172025 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fae64063050 con 0x55d88d3ab370 2024-09-10T09:06:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.966+0000 7fae77e83740 10 client.28876 fetch_fsmap learned FSMap version 46 2024-09-10T09:06:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.966+0000 7fae77e83740 1 -- 172.21.15.172:0/3629172025 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d88d425bb0 con 0x55d88d3ab370 2024-09-10T09:06:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.968+0000 7fae52ff5640 1 -- 172.21.15.172:0/3629172025 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae600047a0 con 0x55d88d3ab370 2024-09-10T09:06:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.968+0000 7fae52ff5640 1 -- 172.21.15.172:0/3629172025 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7fae6405c070 con 0x55d88d3ab370 2024-09-10T09:06:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.968+0000 7fae77e83740 10 client.28876 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T09:06:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.968+0000 7fae77e83740 10 client.28876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T09:06:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.968+0000 7fae77e83740 20 client.28876 populate_metadata read hostname 'smithi172' 2024-09-10T09:06:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.968+0000 7fae77e83740 1 --2- 172.21.15.172:0/3629172025 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d88d426590 0x55d88d4289b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.968+0000 7fae77e83740 4 client.28876 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T09:06:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.968+0000 7fae77e83740 1 -- 172.21.15.172:0/3629172025 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d88d310d70 con 0x55d88d426590 2024-09-10T09:06:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.969+0000 7fae6ed9d640 1 --2- 172.21.15.172:0/3629172025 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d88d426590 0x55d88d4289b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.976+0000 7fae6ed9d640 1 --2- 172.21.15.172:0/3629172025 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d88d426590 0x55d88d4289b0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9411c7d9e66ab671 server_cookie=9e9be767a07d8a4d in_seq=0 out_seq=0 2024-09-10T09:06:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:00.977+0000 7fae52ff5640 10 client.28876 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:06:00.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:00 smithi174 ceph-mon[34068]: pgmap v943: 129 pgs: 129 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 570 KiB/s wr, 159 op/s 2024-09-10T09:06:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.012+0000 7fae52ff5640 1 -- 172.21.15.172:0/3629172025 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d88d310d70 con 0x55d88d426590 2024-09-10T09:06:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.012+0000 7fae52ff5640 10 client.28876 handle_command_reply: tid=0 2024-09-10T09:06:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.012+0000 7fae77e83740 1 client.28876 shutdown 2024-09-10T09:06:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.018+0000 7fae77e83740 1 -- 172.21.15.172:0/3629172025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 msgr2=0x55d88d4245c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.018+0000 7fae77e83740 1 --2- 172.21.15.172:0/3629172025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 0x55d88d4245c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fae64000c00 tx=0x7fae64030fb0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.019+0000 7fae77e83740 1 -- 172.21.15.172:0/3629172025 shutdown_connections 2024-09-10T09:06:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.019+0000 7fae77e83740 1 --2- 172.21.15.172:0/3629172025 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55d88d426590 0x55d88d4289b0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.019+0000 7fae77e83740 1 --2- 172.21.15.172:0/3629172025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d88d3ab370 0x55d88d4245c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.019+0000 7fae77e83740 1 --2- 172.21.15.172:0/3629172025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d88d3a62e0 0x55d88d3b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.019+0000 7fae77e83740 1 --2- 172.21.15.172:0/3629172025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d88d3a59a0 0x55d88d3b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.019+0000 7fae77e83740 1 -- 172.21.15.172:0/3629172025 >> 172.21.15.172:0/3629172025 conn(0x55d88d3a1010 msgr2=0x55d88d3a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.019+0000 7fae52ff5640 0 client.28876 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:06:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.022+0000 7fae77e83740 1 -- 172.21.15.172:0/3629172025 shutdown_connections 2024-09-10T09:06:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:{ 2024-09-10T09:06:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "message": "", 2024-09-10T09:06:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout: "return_code": 0 2024-09-10T09:06:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stdout:} 2024-09-10T09:06:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.024+0000 7fae77e83740 1 -- 172.21.15.172:0/3629172025 wait complete. 2024-09-10T09:06:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.024+0000 7fae77e83740 20 client.28876 trim_cache size 0 max 16384 2024-09-10T09:06:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.028+0000 7fae7500e640 1 -- 172.21.15.172:0/1010893536 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fae580a2c30 msgr2=0x7fae580a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.028+0000 7fae7500e640 1 --2- 172.21.15.172:0/1010893536 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fae580a2c30 0x7fae580a50f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fae60005fd0 tx=0x7fae60009260 comp rx=0 tx=0).stop 2024-09-10T09:06:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.028+0000 7fae7500e640 1 -- 172.21.15.172:0/1010893536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae700737d0 msgr2=0x7fae7010a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:01.029 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.028+0000 7fae7500e640 1 --2- 172.21.15.172:0/1010893536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae700737d0 0x7fae7010a4c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fae6800eab0 tx=0x7fae6800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.030+0000 7fae7500e640 1 -- 172.21.15.172:0/1010893536 shutdown_connections 2024-09-10T09:06:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.030+0000 7fae7500e640 1 --2- 172.21.15.172:0/1010893536 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fae580a2c30 0x7fae580a50f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.030+0000 7fae7500e640 1 --2- 172.21.15.172:0/1010893536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae70109930 0x7fae7010aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.030+0000 7fae7500e640 1 --2- 172.21.15.172:0/1010893536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae700737d0 0x7fae7010a4c0 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fae6800eab0 tx=0x7fae6800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.030+0000 7fae7500e640 1 --2- 172.21.15.172:0/1010893536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae70072d20 0x7fae7010be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.032 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.030+0000 7fae7500e640 1 -- 172.21.15.172:0/1010893536 >> 172.21.15.172:0/1010893536 conn(0x7fae7006ece0 msgr2=0x7fae7010e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:01.034 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.032+0000 7fae7500e640 1 -- 172.21.15.172:0/1010893536 shutdown_connections 2024-09-10T09:06:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi172.front.sepia.ceph.com.smithi172.stderr:2024-09-10T09:06:01.034+0000 7fae7500e640 1 -- 172.21.15.172:0/1010893536 wait complete. 2024-09-10T09:06:01.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:00 smithi191 ceph-mon[35314]: pgmap v943: 129 pgs: 129 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 570 KiB/s wr, 159 op/s 2024-09-10T09:06:01.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:00 smithi172 ceph-mon[30854]: pgmap v943: 129 pgs: 129 active+clean; 12 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 570 KiB/s wr, 159 op/s 2024-09-10T09:06:01.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.415+0000 7f5813afe640 1 -- 172.21.15.172:0/3817435236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f580c1011e0 msgr2=0x7f580c105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:01.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.415+0000 7f5813afe640 1 --2- 172.21.15.172:0/3817435236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f580c1011e0 0x7f580c105980 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009930 tx=0x7f57fc02a480 comp rx=0 tx=0).stop 2024-09-10T09:06:01.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.418+0000 7f5813afe640 1 -- 172.21.15.172:0/3817435236 shutdown_connections 2024-09-10T09:06:01.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.418+0000 7f5813afe640 1 --2- 172.21.15.172:0/3817435236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f580c105ec0 0x7f580c106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.418+0000 7f5813afe640 1 --2- 172.21.15.172:0/3817435236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f580c1011e0 0x7f580c105980 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.418+0000 7f5813afe640 1 --2- 172.21.15.172:0/3817435236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f580c1008c0 0x7f580c100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.418+0000 7f5813afe640 1 -- 172.21.15.172:0/3817435236 >> 172.21.15.172:0/3817435236 conn(0x7f580c0fc7d0 msgr2=0x7f580c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:01.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.419+0000 7f5813afe640 1 -- 172.21.15.172:0/3817435236 shutdown_connections 2024-09-10T09:06:01.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.419+0000 7f5813afe640 1 -- 172.21.15.172:0/3817435236 wait complete. 2024-09-10T09:06:01.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.420+0000 7f5813afe640 1 Processor -- start 2024-09-10T09:06:01.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.420+0000 7f5813afe640 1 -- start start 2024-09-10T09:06:01.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.420+0000 7f5813afe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f580c1008c0 0x7f580c19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:01.422 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.420+0000 7f5813afe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f580c1011e0 0x7f580c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:01.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.420+0000 7f5813afe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f580c105ec0 0x7f580c1a3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:01.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.420+0000 7f5813afe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f580c118590 con 0x7f580c1011e0 2024-09-10T09:06:01.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.421+0000 7f5813afe640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f580c118410 con 0x7f580c1008c0 2024-09-10T09:06:01.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.421+0000 7f5813afe640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f580c118710 con 0x7f580c105ec0 2024-09-10T09:06:01.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.421+0000 7f581189a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f580c1008c0 0x7f580c19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:01.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.421+0000 7f581189a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f580c1008c0 0x7f580c19ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:35390/0 (socket says 172.21.15.172:35390) 2024-09-10T09:06:01.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.421+0000 7f581189a640 1 -- 172.21.15.172:0/2029891880 learned_addr learned my addr 172.21.15.172:0/2029891880 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:01.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.421+0000 7f581209b640 1 --2- 172.21.15.172:0/2029891880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f580c105ec0 0x7f580c1a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:01.423 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.422+0000 7f581189a640 1 -- 172.21.15.172:0/2029891880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f580c105ec0 msgr2=0x7f580c1a3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:01.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.422+0000 7f581189a640 1 --2- 172.21.15.172:0/2029891880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f580c105ec0 0x7f580c1a3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.422+0000 7f581189a640 1 -- 172.21.15.172:0/2029891880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f580c1011e0 msgr2=0x7f580c19f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:01.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.422+0000 7f581189a640 1 --2- 172.21.15.172:0/2029891880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f580c1011e0 0x7f580c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.422+0000 7f581189a640 1 -- 172.21.15.172:0/2029891880 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580c1a3c40 con 0x7f580c1008c0 2024-09-10T09:06:01.424 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.422+0000 7f581189a640 1 --2- 172.21.15.172:0/2029891880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f580c1008c0 0x7f580c19ec40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f580800b4a0 tx=0x7f580800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:01.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.426+0000 7f5802ffd640 1 -- 172.21.15.172:0/2029891880 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808014070 con 0x7f580c1008c0 2024-09-10T09:06:01.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.426+0000 7f5813afe640 1 -- 172.21.15.172:0/2029891880 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580c1a3f30 con 0x7f580c1008c0 2024-09-10T09:06:01.428 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.426+0000 7f5813afe640 1 -- 172.21.15.172:0/2029891880 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580c06bc10 con 0x7f580c1008c0 2024-09-10T09:06:01.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.426+0000 7f5802ffd640 1 -- 172.21.15.172:0/2029891880 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5808010ac0 con 0x7f580c1008c0 2024-09-10T09:06:01.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.426+0000 7f5802ffd640 1 -- 172.21.15.172:0/2029891880 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5808017c30 con 0x7f580c1008c0 2024-09-10T09:06:01.429 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.428+0000 7f5813afe640 1 -- 172.21.15.172:0/2029891880 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f0005290 con 0x7f580c1008c0 2024-09-10T09:06:01.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.432+0000 7f5802ffd640 1 -- 172.21.15.172:0/2029891880 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5808017070 con 0x7f580c1008c0 2024-09-10T09:06:01.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.432+0000 7f5802ffd640 1 --2- 172.21.15.172:0/2029891880 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57f80a2d90 0x7f57f80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:01.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.433+0000 7f5802ffd640 1 -- 172.21.15.172:0/2029891880 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f58080ced30 con 0x7f580c1008c0 2024-09-10T09:06:01.434 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.433+0000 7f5811099640 1 --2- 172.21.15.172:0/2029891880 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57f80a2d90 0x7f57f80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:01.435 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.434+0000 7f5811099640 1 --2- 172.21.15.172:0/2029891880 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57f80a2d90 0x7f57f80a5250 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009230 tx=0x7f57fc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:01.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.439+0000 7f5802ffd640 1 -- 172.21.15.172:0/2029891880 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f580809b950 con 0x7f580c1008c0 2024-09-10T09:06:01.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.847+0000 7f5813afe640 1 -- 172.21.15.172:0/2029891880 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57f0005580 con 0x7f580c1008c0 2024-09-10T09:06:01.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.849+0000 7f5802ffd640 1 -- 172.21.15.172:0/2029891880 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+12007 (secure 0 0 0) 0x7f580809b250 con 0x7f580c1008c0 2024-09-10T09:06:01.850 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T09:06:01.851 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":47,"btime":"2024-09-10T09:06:01:751841+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T09:06:01.321574+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T09:06:01.851 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 47 2024-09-10T09:06:01.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.859+0000 7f5813afe640 1 -- 172.21.15.172:0/2029891880 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57f80a2d90 msgr2=0x7f57f80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:01.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.859+0000 7f5813afe640 1 --2- 172.21.15.172:0/2029891880 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57f80a2d90 0x7f57f80a5250 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009230 tx=0x7f57fc000bc0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.859+0000 7f5813afe640 1 -- 172.21.15.172:0/2029891880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f580c1008c0 msgr2=0x7f580c19ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:01.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.859+0000 7f5813afe640 1 --2- 172.21.15.172:0/2029891880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f580c1008c0 0x7f580c19ec40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f580800b4a0 tx=0x7f580800b960 comp rx=0 tx=0).stop 2024-09-10T09:06:01.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.859+0000 7f5813afe640 1 -- 172.21.15.172:0/2029891880 shutdown_connections 2024-09-10T09:06:01.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.859+0000 7f5813afe640 1 --2- 172.21.15.172:0/2029891880 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f57f80a2d90 0x7f57f80a5250 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.859+0000 7f5813afe640 1 --2- 172.21.15.172:0/2029891880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f580c105ec0 0x7f580c1a3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.859+0000 7f5813afe640 1 --2- 172.21.15.172:0/2029891880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f580c1011e0 0x7f580c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.859+0000 7f5813afe640 1 --2- 172.21.15.172:0/2029891880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f580c1008c0 0x7f580c19ec40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:01.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.859+0000 7f5813afe640 1 -- 172.21.15.172:0/2029891880 >> 172.21.15.172:0/2029891880 conn(0x7f580c0fc7d0 msgr2=0x7f580c10f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:01.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.859+0000 7f5813afe640 1 -- 172.21.15.172:0/2029891880 shutdown_connections 2024-09-10T09:06:01.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:01.859+0000 7f5813afe640 1 -- 172.21.15.172:0/2029891880 wait complete. 2024-09-10T09:06:02.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:01 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:06:02.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:01 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:06:02.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:01 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:06:02.705 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --cancel --all' 2024-09-10T09:06:02.706 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --cancel --all 2024-09-10T09:06:03.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:02 smithi191 ceph-mon[35314]: pgmap v944: 129 pgs: 129 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 626 KiB/s wr, 202 op/s 2024-09-10T09:06:03.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:02 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:06:03.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:02 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2029891880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:06:03.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:02 smithi172 ceph-mon[30854]: pgmap v944: 129 pgs: 129 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 626 KiB/s wr, 202 op/s 2024-09-10T09:06:03.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:02 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:06:03.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:02 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2029891880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:06:03.201 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:06:03.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:02 smithi174 ceph-mon[34068]: pgmap v944: 129 pgs: 129 active+clean; 11 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 626 KiB/s wr, 202 op/s 2024-09-10T09:06:03.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:02 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:06:03.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:02 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2029891880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:06:03.860 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.857+0000 7ffa5b59e640 1 -- 172.21.15.172:0/3782170981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa5c10e090 msgr2=0x7ffa5c10e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:03.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.857+0000 7ffa5b59e640 1 --2- 172.21.15.172:0/3782170981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa5c10e090 0x7ffa5c10e470 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00b880 tx=0x7ffa4c005f90 comp rx=0 tx=0).stop 2024-09-10T09:06:03.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.858+0000 7ffa5b59e640 1 -- 172.21.15.172:0/3782170981 shutdown_connections 2024-09-10T09:06:03.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.858+0000 7ffa5b59e640 1 --2- 172.21.15.172:0/3782170981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa5c06ee70 0x7ffa5c07a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.858+0000 7ffa5b59e640 1 --2- 172.21.15.172:0/3782170981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa5c06e4f0 0x7ffa5c06e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.858+0000 7ffa5b59e640 1 --2- 172.21.15.172:0/3782170981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa5c10e090 0x7ffa5c10e470 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.861 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.858+0000 7ffa5b59e640 1 -- 172.21.15.172:0/3782170981 >> 172.21.15.172:0/3782170981 conn(0x7ffa5c073180 msgr2=0x7ffa5c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:03.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.864+0000 7ffa5b59e640 1 -- 172.21.15.172:0/3782170981 shutdown_connections 2024-09-10T09:06:03.866 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.864+0000 7ffa5b59e640 1 -- 172.21.15.172:0/3782170981 wait complete. 2024-09-10T09:06:03.867 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.865+0000 7ffa5b59e640 1 Processor -- start 2024-09-10T09:06:03.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.870+0000 7ffa5b59e640 1 -- start start 2024-09-10T09:06:03.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.871+0000 7ffa5b59e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa5c06e4f0 0x7ffa5c084db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.871+0000 7ffa5b59e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa5c06ee70 0x7ffa5c07ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.871+0000 7ffa5b59e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa5c10e090 0x7ffa5c07f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.871+0000 7ffa5b59e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c07d980 con 0x7ffa5c06ee70 2024-09-10T09:06:03.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.871+0000 7ffa5b59e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c07d800 con 0x7ffa5c10e090 2024-09-10T09:06:03.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.871+0000 7ffa5b59e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c07db00 con 0x7ffa5c06e4f0 2024-09-10T09:06:03.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.871+0000 7ffa5a59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa5c06e4f0 0x7ffa5c084db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:03.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.871+0000 7ffa5a59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa5c06e4f0 0x7ffa5c084db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:40378/0 (socket says 172.21.15.172:40378) 2024-09-10T09:06:03.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.871+0000 7ffa5a59c640 1 -- 172.21.15.172:0/3447838124 learned_addr learned my addr 172.21.15.172:0/3447838124 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:03.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.872+0000 7ffa5ad9d640 1 --2- 172.21.15.172:0/3447838124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa5c10e090 0x7ffa5c07f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:03.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.874+0000 7ffa5a59c640 1 -- 172.21.15.172:0/3447838124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa5c10e090 msgr2=0x7ffa5c07f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:03.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.874+0000 7ffa5a59c640 1 --2- 172.21.15.172:0/3447838124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa5c10e090 0x7ffa5c07f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.874+0000 7ffa5a59c640 1 -- 172.21.15.172:0/3447838124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa5c06ee70 msgr2=0x7ffa5c07ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:03.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.874+0000 7ffa5a59c640 1 --2- 172.21.15.172:0/3447838124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa5c06ee70 0x7ffa5c07ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.874+0000 7ffa5a59c640 1 -- 172.21.15.172:0/3447838124 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5c1ccb90 con 0x7ffa5c06e4f0 2024-09-10T09:06:03.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.876+0000 7ffa5ad9d640 1 --2- 172.21.15.172:0/3447838124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa5c10e090 0x7ffa5c07f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:06:03.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.877+0000 7ffa5a59c640 1 --2- 172.21.15.172:0/3447838124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa5c06e4f0 0x7ffa5c084db0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0057c0 tx=0x7ffa4c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:03.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.881+0000 7ffa4b7fe640 1 -- 172.21.15.172:0/3447838124 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c03f020 con 0x7ffa5c06e4f0 2024-09-10T09:06:03.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.881+0000 7ffa5b59e640 1 -- 172.21.15.172:0/3447838124 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5c1ccd60 con 0x7ffa5c06e4f0 2024-09-10T09:06:03.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.881+0000 7ffa5b59e640 1 -- 172.21.15.172:0/3447838124 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5c1cd210 con 0x7ffa5c06e4f0 2024-09-10T09:06:03.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.881+0000 7ffa4b7fe640 1 -- 172.21.15.172:0/3447838124 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa4c0054e0 con 0x7ffa5c06e4f0 2024-09-10T09:06:03.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.881+0000 7ffa4b7fe640 1 -- 172.21.15.172:0/3447838124 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c003850 con 0x7ffa5c06e4f0 2024-09-10T09:06:03.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.888+0000 7ffa631cd740 1 Processor -- start 2024-09-10T09:06:03.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.890+0000 7ffa631cd740 1 -- start start 2024-09-10T09:06:03.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.890+0000 7ffa631cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a7abb0 0x55f489a7af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.890+0000 7ffa631cd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7b4d0 0x55f489a85850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.890+0000 7ffa631cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a85d90 0x55f489a88180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.890+0000 7ffa631cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f4899df200 con 0x55f489a85d90 2024-09-10T09:06:03.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.890+0000 7ffa631cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f489985dc0 con 0x55f489a7abb0 2024-09-10T09:06:03.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.890+0000 7ffa631cd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f4899db1c0 con 0x55f489a7b4d0 2024-09-10T09:06:03.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.890+0000 7ffa4b7fe640 1 -- 172.21.15.172:0/3447838124 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ffa4c042020 con 0x7ffa5c06e4f0 2024-09-10T09:06:03.892 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.890+0000 7ffa5a59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7b4d0 0x55f489a85850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:03.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.890+0000 7ffa5a59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7b4d0 0x55f489a85850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.172:40392/0 (socket says 172.21.15.172:40392) 2024-09-10T09:06:03.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.890+0000 7ffa5a59c640 1 -- 172.21.15.172:0/1031274383 learned_addr learned my addr 172.21.15.172:0/1031274383 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:03.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.891+0000 7ffa5ad9d640 1 --2- 172.21.15.172:0/1031274383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a7abb0 0x55f489a7af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:03.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.891+0000 7ffa59d9b640 1 --2- 172.21.15.172:0/1031274383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a85d90 0x55f489a88180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:03.893 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.892+0000 7ffa5a59c640 1 -- 172.21.15.172:0/1031274383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a7abb0 msgr2=0x55f489a7af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:03.894 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.892+0000 7ffa5a59c640 1 --2- 172.21.15.172:0/1031274383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a7abb0 0x55f489a7af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.892+0000 7ffa5a59c640 1 -- 172.21.15.172:0/1031274383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a85d90 msgr2=0x55f489a88180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:03.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.892+0000 7ffa5a59c640 1 --2- 172.21.15.172:0/1031274383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a85d90 0x55f489a88180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.896 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.892+0000 7ffa5a59c640 1 -- 172.21.15.172:0/1031274383 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f489750620 con 0x55f489a7b4d0 2024-09-10T09:06:03.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.893+0000 7ffa4b7fe640 1 --2- 172.21.15.172:0/3447838124 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa440a2d90 0x7ffa440a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.893+0000 7ffa5ad9d640 1 --2- 172.21.15.172:0/3447838124 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa440a2d90 0x7ffa440a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:03.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.896+0000 7ffa5a59c640 1 --2- 172.21.15.172:0/1031274383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7b4d0 0x55f489a85850 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0f55d0 tx=0x7ffa4c0f56c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=678fe9c9943d6338 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:03.897 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.896+0000 7ffa4b7fe640 1 -- 172.21.15.172:0/3447838124 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ffa4c0ef6d0 con 0x7ffa5c06e4f0 2024-09-10T09:06:03.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.896+0000 7ffa48ff9640 1 -- 172.21.15.172:0/1031274383 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c0b7e10 con 0x55f489a7b4d0 2024-09-10T09:06:03.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.897+0000 7ffa48ff9640 1 -- 172.21.15.172:0/1031274383 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa4c00a840 con 0x55f489a7b4d0 2024-09-10T09:06:03.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.897+0000 7ffa48ff9640 1 -- 172.21.15.172:0/1031274383 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c0bb4d0 con 0x55f489a7b4d0 2024-09-10T09:06:03.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.898+0000 7ffa631cd740 1 -- 172.21.15.172:0/1031274383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7b4d0 msgr2=0x55f489a85850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:03.898 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.898+0000 7ffa631cd740 1 --2- 172.21.15.172:0/1031274383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7b4d0 0x55f489a85850 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0f55d0 tx=0x7ffa4c0f56c0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.898+0000 7ffa631cd740 1 -- 172.21.15.172:0/1031274383 shutdown_connections 2024-09-10T09:06:03.899 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.898+0000 7ffa631cd740 1 --2- 172.21.15.172:0/1031274383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a85d90 0x55f489a88180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.898+0000 7ffa631cd740 1 --2- 172.21.15.172:0/1031274383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7b4d0 0x55f489a85850 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.898+0000 7ffa631cd740 1 --2- 172.21.15.172:0/1031274383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a7abb0 0x55f489a7af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.898+0000 7ffa631cd740 1 -- 172.21.15.172:0/1031274383 >> 172.21.15.172:0/1031274383 conn(0x55f489a77140 msgr2=0x55f489a77520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:03.900 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.899+0000 7ffa5ad9d640 1 --2- 172.21.15.172:0/3447838124 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa440a2d90 0x7ffa440a5250 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ffa54007390 tx=0x7ffa5400c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:03.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.900+0000 7ffa631cd740 1 -- 172.21.15.172:0/1031274383 shutdown_connections 2024-09-10T09:06:03.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.900+0000 7ffa631cd740 1 -- 172.21.15.172:0/1031274383 wait complete. 2024-09-10T09:06:03.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.900+0000 7ffa631cd740 1 -- start start 2024-09-10T09:06:03.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.900+0000 7ffa631cd740 1 Processor -- start 2024-09-10T09:06:03.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.901+0000 7ffa631cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a7abb0 0x55f489a85600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.901+0000 7ffa631cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a7b4d0 0x55f489a828b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.903 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.901+0000 7ffa631cd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a85d90 0x55f489a82df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.901+0000 7ffa631cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f489a88720 con 0x55f489a7abb0 2024-09-10T09:06:03.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.901+0000 7ffa631cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f489979c50 con 0x55f489a7b4d0 2024-09-10T09:06:03.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.901+0000 7ffa631cd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f489a888a0 con 0x55f489a85d90 2024-09-10T09:06:03.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.901+0000 7ffa631cd740 10 client.0 _ll_register_callbacks cb 0x55f48998bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:06:03.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.901+0000 7ffa59d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a7b4d0 0x55f489a828b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:03.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.901+0000 7ffa5a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a7abb0 0x55f489a85600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:03.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.901+0000 7ffa59d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a7b4d0 0x55f489a828b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:35416/0 (socket says 172.21.15.172:35416) 2024-09-10T09:06:03.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.901+0000 7ffa59d9b640 1 -- 172.21.15.172:0/4090026855 learned_addr learned my addr 172.21.15.172:0/4090026855 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:03.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.902+0000 7ffa59d9b640 1 -- 172.21.15.172:0/4090026855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a85d90 msgr2=0x55f489a82df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:06:03.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.902+0000 7ffa59d9b640 1 --2- 172.21.15.172:0/4090026855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a85d90 0x55f489a82df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.902+0000 7ffa59d9b640 1 -- 172.21.15.172:0/4090026855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a7abb0 msgr2=0x55f489a85600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:03.904 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.902+0000 7ffa59d9b640 1 --2- 172.21.15.172:0/4090026855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a7abb0 0x55f489a85600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.902+0000 7ffa59d9b640 1 -- 172.21.15.172:0/4090026855 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f489a83620 con 0x55f489a7b4d0 2024-09-10T09:06:03.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.902+0000 7ffa59d9b640 1 --2- 172.21.15.172:0/4090026855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a7b4d0 0x55f489a828b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000d4c0 tx=0x7ffa500039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5b35ab22d450b34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:03.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.902+0000 7ffa41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T09:06:03.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.903+0000 7ffa41ffb640 1 -- 172.21.15.172:0/4090026855 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5001a030 con 0x55f489a7b4d0 2024-09-10T09:06:03.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.903+0000 7ffa41ffb640 1 -- 172.21.15.172:0/4090026855 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ffa50009040 con 0x55f489a7b4d0 2024-09-10T09:06:03.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.903+0000 7ffa631cd740 1 -- 172.21.15.172:0/4090026855 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa4c00a840 con 0x55f489a7b4d0 2024-09-10T09:06:03.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.903+0000 7ffa41ffb640 1 -- 172.21.15.172:0/4090026855 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ffa50005020 con 0x55f489a7b4d0 2024-09-10T09:06:03.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.904+0000 7ffa631cd740 10 client.28991 fetch_fsmap learned FSMap version 49 2024-09-10T09:06:03.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.904+0000 7ffa631cd740 1 -- 172.21.15.172:0/4090026855 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f489a94300 con 0x55f489a7b4d0 2024-09-10T09:06:03.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.905+0000 7ffa41ffb640 1 -- 172.21.15.172:0/4090026855 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa4c00a840 con 0x55f489a7b4d0 2024-09-10T09:06:03.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.905+0000 7ffa41ffb640 1 -- 172.21.15.172:0/4090026855 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 49) ==== 8398+0+0 (secure 0 0 0) 0x7ffa500166c0 con 0x55f489a7b4d0 2024-09-10T09:06:03.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.905+0000 7ffa631cd740 10 client.28991 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T09:06:03.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.905+0000 7ffa631cd740 10 client.28991 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:06:03.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.905+0000 7ffa631cd740 20 client.28991 populate_metadata read hostname 'smithi172' 2024-09-10T09:06:03.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.905+0000 7ffa631cd740 1 --2- 172.21.15.172:0/4090026855 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f489a95c60 0x55f489a98080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.906+0000 7ffa631cd740 4 client.28991 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:06:03.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.906+0000 7ffa631cd740 1 -- 172.21.15.172:0/4090026855 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4899ec770 con 0x55f489a95c60 2024-09-10T09:06:03.907 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.906+0000 7ffa5a59c640 1 --2- 172.21.15.172:0/4090026855 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f489a95c60 0x55f489a98080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:03.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.911+0000 7ffa5a59c640 1 --2- 172.21.15.172:0/4090026855 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f489a95c60 0x55f489a98080 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96fb6641ca01bb7e server_cookie=a7546abf1109510a in_seq=0 out_seq=0 2024-09-10T09:06:03.912 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.911+0000 7ffa41ffb640 10 client.28991 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:06:03.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.912+0000 7ffa41ffb640 1 -- 172.21.15.172:0/4090026855 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4899ec770 con 0x55f489a95c60 2024-09-10T09:06:03.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.912+0000 7ffa41ffb640 10 client.28991 handle_command_reply: tid=0 2024-09-10T09:06:03.913 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.912+0000 7ffa631cd740 1 client.28991 shutdown 2024-09-10T09:06:03.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.914+0000 7ffa631cd740 1 -- 172.21.15.172:0/4090026855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a7b4d0 msgr2=0x55f489a828b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:03.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.914+0000 7ffa631cd740 1 --2- 172.21.15.172:0/4090026855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a7b4d0 0x55f489a828b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000d4c0 tx=0x7ffa500039c0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.914+0000 7ffa631cd740 1 -- 172.21.15.172:0/4090026855 shutdown_connections 2024-09-10T09:06:03.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.914+0000 7ffa631cd740 1 --2- 172.21.15.172:0/4090026855 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f489a95c60 0x55f489a98080 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.914+0000 7ffa631cd740 1 --2- 172.21.15.172:0/4090026855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a85d90 0x55f489a82df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.914+0000 7ffa631cd740 1 --2- 172.21.15.172:0/4090026855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a7b4d0 0x55f489a828b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.914+0000 7ffa631cd740 1 --2- 172.21.15.172:0/4090026855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a7abb0 0x55f489a85600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.915 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.914+0000 7ffa631cd740 1 -- 172.21.15.172:0/4090026855 >> 172.21.15.172:0/4090026855 conn(0x55f489a77140 msgr2=0x55f489a76420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:03.916 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.914+0000 7ffa41ffb640 0 client.28991 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:06:03.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.916+0000 7ffa631cd740 1 -- 172.21.15.172:0/4090026855 shutdown_connections 2024-09-10T09:06:03.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.916+0000 7ffa631cd740 1 -- 172.21.15.172:0/4090026855 wait complete. 2024-09-10T09:06:03.917 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.917+0000 7ffa631cd740 20 client.28991 trim_cache size 0 max 16384 2024-09-10T09:06:03.950 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.949+0000 7ffa631cd740 1 Processor -- start 2024-09-10T09:06:03.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.949+0000 7ffa631cd740 1 -- start start 2024-09-10T09:06:03.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.949+0000 7ffa631cd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7abb0 0x55f489a91a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.951 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.949+0000 7ffa631cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a7b4d0 0x55f489a9d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.949+0000 7ffa631cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a85d90 0x55f489a9ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.949+0000 7ffa631cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c00fbd0 con 0x55f489a7b4d0 2024-09-10T09:06:03.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.949+0000 7ffa631cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c0117d0 con 0x55f489a85d90 2024-09-10T09:06:03.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.949+0000 7ffa631cd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c0102a0 con 0x55f489a7abb0 2024-09-10T09:06:03.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.950+0000 7ffa5a59c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7abb0 0x55f489a91a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:03.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.950+0000 7ffa5ad9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a85d90 0x55f489a9ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:03.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.950+0000 7ffa59d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a7b4d0 0x55f489a9d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:03.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.950+0000 7ffa5ad9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a85d90 0x55f489a9ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.172:35424/0 (socket says 172.21.15.172:35424) 2024-09-10T09:06:03.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.950+0000 7ffa5ad9d640 1 -- 172.21.15.172:0/3154499492 learned_addr learned my addr 172.21.15.172:0/3154499492 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:03.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.950+0000 7ffa5a59c640 1 -- 172.21.15.172:0/3154499492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a85d90 msgr2=0x55f489a9ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:03.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.950+0000 7ffa5a59c640 1 --2- 172.21.15.172:0/3154499492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a85d90 0x55f489a9ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.952 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.950+0000 7ffa5a59c640 1 -- 172.21.15.172:0/3154499492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a7b4d0 msgr2=0x55f489a9d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:03.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.950+0000 7ffa5a59c640 1 --2- 172.21.15.172:0/3154499492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a7b4d0 0x55f489a9d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.950+0000 7ffa5a59c640 1 -- 172.21.15.172:0/3154499492 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f489a7b300 con 0x55f489a7abb0 2024-09-10T09:06:03.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.950+0000 7ffa5a59c640 1 --2- 172.21.15.172:0/3154499492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7abb0 0x55f489a91a20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c005c40 tx=0x7ffa4c00d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daf8008133f5db08 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:03.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.950+0000 7ffa43fff640 1 -- 172.21.15.172:0/3154499492 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c0bb6b0 con 0x55f489a7abb0 2024-09-10T09:06:03.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.951+0000 7ffa43fff640 1 -- 172.21.15.172:0/3154499492 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa4c00a840 con 0x55f489a7abb0 2024-09-10T09:06:03.953 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.951+0000 7ffa43fff640 1 -- 172.21.15.172:0/3154499492 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c0bb940 con 0x55f489a7abb0 2024-09-10T09:06:03.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.951+0000 7ffa631cd740 1 -- 172.21.15.172:0/3154499492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7abb0 msgr2=0x55f489a91a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:03.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.951+0000 7ffa631cd740 1 --2- 172.21.15.172:0/3154499492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7abb0 0x55f489a91a20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c005c40 tx=0x7ffa4c00d040 comp rx=0 tx=0).stop 2024-09-10T09:06:03.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.952+0000 7ffa631cd740 1 -- 172.21.15.172:0/3154499492 shutdown_connections 2024-09-10T09:06:03.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.952+0000 7ffa631cd740 1 --2- 172.21.15.172:0/3154499492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a85d90 0x55f489a9ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.952+0000 7ffa631cd740 1 --2- 172.21.15.172:0/3154499492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a7b4d0 0x55f489a9d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.952+0000 7ffa631cd740 1 --2- 172.21.15.172:0/3154499492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7abb0 0x55f489a91a20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.952+0000 7ffa631cd740 1 -- 172.21.15.172:0/3154499492 >> 172.21.15.172:0/3154499492 conn(0x55f489a77140 msgr2=0x55f489a81180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:03.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.952+0000 7ffa631cd740 1 -- 172.21.15.172:0/3154499492 shutdown_connections 2024-09-10T09:06:03.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.952+0000 7ffa631cd740 1 -- 172.21.15.172:0/3154499492 wait complete. 2024-09-10T09:06:03.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.952+0000 7ffa631cd740 1 -- start start 2024-09-10T09:06:03.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.952+0000 7ffa631cd740 1 Processor -- start 2024-09-10T09:06:03.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.953+0000 7ffa631cd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7abb0 0x55f489a861d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.953+0000 7ffa631cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a7b4d0 0x55f489a86710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.953+0000 7ffa631cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a85d90 0x55f489a86c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.953+0000 7ffa631cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c0110d0 con 0x55f489a7b4d0 2024-09-10T09:06:03.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.953+0000 7ffa59d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a7b4d0 0x55f489a86710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:03.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.953+0000 7ffa631cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c0109a0 con 0x55f489a85d90 2024-09-10T09:06:03.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.953+0000 7ffa631cd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c00db50 con 0x55f489a7abb0 2024-09-10T09:06:03.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.953+0000 7ffa59d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a7b4d0 0x55f489a86710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:45722/0 (socket says 172.21.15.172:45722) 2024-09-10T09:06:03.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.953+0000 7ffa59d9b640 1 -- 172.21.15.172:0/3709759514 learned_addr learned my addr 172.21.15.172:0/3709759514 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.953+0000 7ffa5a59c640 1 --2- 172.21.15.172:0/3709759514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7abb0 0x55f489a861d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.953+0000 7ffa631cd740 10 client.0 _ll_register_callbacks cb 0x7ffa50004270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:06:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.953+0000 7ffa5ad9d640 1 --2- 172.21.15.172:0/3709759514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a85d90 0x55f489a86c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.954+0000 7ffa5a59c640 1 -- 172.21.15.172:0/3709759514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a85d90 msgr2=0x55f489a86c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.954+0000 7ffa5a59c640 1 --2- 172.21.15.172:0/3709759514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a85d90 0x55f489a86c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.954+0000 7ffa5a59c640 1 -- 172.21.15.172:0/3709759514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a7b4d0 msgr2=0x55f489a86710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.954+0000 7ffa5a59c640 1 --2- 172.21.15.172:0/3709759514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a7b4d0 0x55f489a86710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.954+0000 7ffa5a59c640 1 -- 172.21.15.172:0/3709759514 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f489a8ccc0 con 0x55f489a7abb0 2024-09-10T09:06:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.954+0000 7ffa5a59c640 1 --2- 172.21.15.172:0/3709759514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7abb0 0x55f489a861d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0bb640 tx=0x7ffa4c005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b4bec14ddc1acc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.954+0000 7ffa41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-10T09:06:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.954+0000 7ffa41ffb640 1 -- 172.21.15.172:0/3709759514 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c0bb940 con 0x55f489a7abb0 2024-09-10T09:06:03.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.954+0000 7ffa41ffb640 1 -- 172.21.15.172:0/3709759514 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ffa4c0ef410 con 0x55f489a7abb0 2024-09-10T09:06:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.954+0000 7ffa631cd740 1 -- 172.21.15.172:0/3709759514 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa4c00a840 con 0x55f489a7abb0 2024-09-10T09:06:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.954+0000 7ffa41ffb640 1 -- 172.21.15.172:0/3709759514 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ffa4c0bb6b0 con 0x55f489a7abb0 2024-09-10T09:06:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.954+0000 7ffa631cd740 10 client.28897 fetch_fsmap learned FSMap version 49 2024-09-10T09:06:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.954+0000 7ffa631cd740 1 -- 172.21.15.172:0/3709759514 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f489a8df40 con 0x55f489a7abb0 2024-09-10T09:06:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.955+0000 7ffa41ffb640 1 -- 172.21.15.172:0/3709759514 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa4c00a840 con 0x55f489a7abb0 2024-09-10T09:06:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.955+0000 7ffa41ffb640 1 -- 172.21.15.172:0/3709759514 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 49) ==== 8398+0+0 (secure 0 0 0) 0x7ffa4c0b81e0 con 0x55f489a7abb0 2024-09-10T09:06:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.955+0000 7ffa631cd740 10 client.28897 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T09:06:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.955+0000 7ffa631cd740 10 client.28897 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-10T09:06:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.955+0000 7ffa631cd740 20 client.28897 populate_metadata read hostname 'smithi172' 2024-09-10T09:06:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.955+0000 7ffa631cd740 1 --2- 172.21.15.172:0/3709759514 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f489a8e2e0 0x55f489b02970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.955+0000 7ffa631cd740 4 client.28897 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-10T09:06:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.955+0000 7ffa631cd740 1 -- 172.21.15.172:0/3709759514 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55f4899e6c80 con 0x55f489a8e2e0 2024-09-10T09:06:03.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.956+0000 7ffa59d9b640 1 --2- 172.21.15.172:0/3709759514 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f489a8e2e0 0x55f489b02970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.956+0000 7ffa41ffb640 10 client.28897 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:06:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.956+0000 7ffa59d9b640 1 --2- 172.21.15.172:0/3709759514 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f489a8e2e0 0x55f489b02970 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f7ab328235be03 server_cookie=fe1deffc46796b49 in_seq=0 out_seq=0 2024-09-10T09:06:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.956+0000 7ffa41ffb640 1 -- 172.21.15.172:0/3709759514 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55f4899e6c80 con 0x55f489a8e2e0 2024-09-10T09:06:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.956+0000 7ffa41ffb640 10 client.28897 handle_command_reply: tid=0 2024-09-10T09:06:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.956+0000 7ffa631cd740 1 client.28897 shutdown 2024-09-10T09:06:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.957+0000 7ffa631cd740 1 -- 172.21.15.172:0/3709759514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7abb0 msgr2=0x55f489a861d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.957+0000 7ffa631cd740 1 --2- 172.21.15.172:0/3709759514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7abb0 0x55f489a861d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0bb640 tx=0x7ffa4c005bd0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.957+0000 7ffa631cd740 1 -- 172.21.15.172:0/3709759514 shutdown_connections 2024-09-10T09:06:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.957+0000 7ffa631cd740 1 --2- 172.21.15.172:0/3709759514 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55f489a8e2e0 0x55f489b02970 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.957+0000 7ffa631cd740 1 --2- 172.21.15.172:0/3709759514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f489a85d90 0x55f489a86c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.957+0000 7ffa631cd740 1 --2- 172.21.15.172:0/3709759514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f489a7b4d0 0x55f489a86710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.957+0000 7ffa631cd740 1 --2- 172.21.15.172:0/3709759514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f489a7abb0 0x55f489a861d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.962 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.957+0000 7ffa631cd740 1 -- 172.21.15.172:0/3709759514 >> 172.21.15.172:0/3709759514 conn(0x55f489a77140 msgr2=0x55f489a7d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:03.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.957+0000 7ffa41ffb640 0 client.28897 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:06:03.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.957+0000 7ffa631cd740 1 -- 172.21.15.172:0/3709759514 shutdown_connections 2024-09-10T09:06:03.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.957+0000 7ffa631cd740 1 -- 172.21.15.172:0/3709759514 wait complete. 2024-09-10T09:06:03.963 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:03 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:06:03.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.957+0000 7ffa631cd740 20 client.28897 trim_cache size 0 max 16384 2024-09-10T09:06:03.964 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:06:03.964 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 49, 2024-09-10T09:06:03.964 INFO:teuthology.orchestra.run.smithi172.stdout: "leader": 24488, 2024-09-10T09:06:03.964 INFO:teuthology.orchestra.run.smithi172.stdout: "set_version": 60, 2024-09-10T09:06:03.964 INFO:teuthology.orchestra.run.smithi172.stdout: "sets": {} 2024-09-10T09:06:03.964 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:06:03.965 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.958+0000 7ffa5b59e640 1 -- 172.21.15.172:0/3447838124 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa440a2d90 msgr2=0x7ffa440a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:03.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.958+0000 7ffa5b59e640 1 --2- 172.21.15.172:0/3447838124 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa440a2d90 0x7ffa440a5250 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ffa54007390 tx=0x7ffa5400c040 comp rx=0 tx=0).stop 2024-09-10T09:06:03.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.958+0000 7ffa5b59e640 1 -- 172.21.15.172:0/3447838124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa5c06e4f0 msgr2=0x7ffa5c084db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:03.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.958+0000 7ffa5b59e640 1 --2- 172.21.15.172:0/3447838124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa5c06e4f0 0x7ffa5c084db0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0057c0 tx=0x7ffa4c004270 comp rx=0 tx=0).stop 2024-09-10T09:06:03.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.958+0000 7ffa5b59e640 1 -- 172.21.15.172:0/3447838124 shutdown_connections 2024-09-10T09:06:03.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.958+0000 7ffa5b59e640 1 --2- 172.21.15.172:0/3447838124 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa440a2d90 0x7ffa440a5250 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.958+0000 7ffa5b59e640 1 --2- 172.21.15.172:0/3447838124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa5c10e090 0x7ffa5c07f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.958+0000 7ffa5b59e640 1 --2- 172.21.15.172:0/3447838124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa5c06ee70 0x7ffa5c07ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.958+0000 7ffa5b59e640 1 --2- 172.21.15.172:0/3447838124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa5c06e4f0 0x7ffa5c084db0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:03.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.958+0000 7ffa5b59e640 1 -- 172.21.15.172:0/3447838124 >> 172.21.15.172:0/3447838124 conn(0x7ffa5c073180 msgr2=0x7ffa5c06d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:03.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.958+0000 7ffa5b59e640 1 -- 172.21.15.172:0/3447838124 shutdown_connections 2024-09-10T09:06:03.966 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:03.958+0000 7ffa5b59e640 1 -- 172.21.15.172:0/3447838124 wait complete. 2024-09-10T09:06:04.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:03 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:06:04.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:03 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:06:04.679 INFO:tasks.quiescer:done joining Quiescers 2024-09-10T09:06:04.679 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-10T09:06:04.689 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi172.front.sepia.ceph.com 2024-09-10T09:06:04.700 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-10T09:06:04.750 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-10T09:06:05.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:04 smithi191 ceph-mon[35314]: pgmap v945: 129 pgs: 129 active+clean; 8.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 554 KiB/s wr, 245 op/s 2024-09-10T09:06:05.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:04 smithi191 ceph-mon[35314]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:06:05.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:04 smithi172 ceph-mon[30854]: pgmap v945: 129 pgs: 129 active+clean; 8.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 554 KiB/s wr, 245 op/s 2024-09-10T09:06:05.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:04 smithi172 ceph-mon[30854]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:06:05.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:04 smithi174 ceph-mon[34068]: pgmap v945: 129 pgs: 129 active+clean; 8.7 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 554 KiB/s wr, 245 op/s 2024-09-10T09:06:05.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:04 smithi174 ceph-mon[34068]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:06:07.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:06 smithi191 ceph-mon[35314]: pgmap v946: 129 pgs: 129 active+clean; 6.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 398 KiB/s wr, 266 op/s 2024-09-10T09:06:07.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:06 smithi172 ceph-mon[30854]: pgmap v946: 129 pgs: 129 active+clean; 6.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 398 KiB/s wr, 266 op/s 2024-09-10T09:06:07.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:06 smithi174 ceph-mon[34068]: pgmap v946: 129 pgs: 129 active+clean; 6.2 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 398 KiB/s wr, 266 op/s 2024-09-10T09:06:09.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:08 smithi191 ceph-mon[35314]: pgmap v947: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 321 KiB/s wr, 363 op/s 2024-09-10T09:06:09.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:08 smithi172 ceph-mon[30854]: pgmap v947: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 321 KiB/s wr, 363 op/s 2024-09-10T09:06:09.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:08 smithi174 ceph-mon[34068]: pgmap v947: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 321 KiB/s wr, 363 op/s 2024-09-10T09:06:11.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:10 smithi172 ceph-mon[30854]: pgmap v948: 129 pgs: 129 active+clean; 932 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 157 KiB/s wr, 336 op/s 2024-09-10T09:06:11.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:10 smithi174 ceph-mon[34068]: pgmap v948: 129 pgs: 129 active+clean; 932 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 157 KiB/s wr, 336 op/s 2024-09-10T09:06:11.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:10 smithi191 ceph-mon[35314]: pgmap v948: 129 pgs: 129 active+clean; 932 MiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 157 KiB/s wr, 336 op/s 2024-09-10T09:06:12.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:11 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T09:06:12.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:11 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T09:06:12.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:11 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T09:06:12.245 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:11 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T09:06:12.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:11 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T09:06:12.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:11 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T09:06:13.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:12 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T09:06:13.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:12 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T09:06:13.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:12 smithi172 ceph-mon[30854]: scrub summary: idle 2024-09-10T09:06:13.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:12 smithi172 ceph-mon[30854]: pgmap v949: 129 pgs: 129 active+clean; 55 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 136 KiB/s wr, 356 op/s 2024-09-10T09:06:13.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:12 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T09:06:13.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:12 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T09:06:13.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:12 smithi174 ceph-mon[34068]: scrub summary: idle 2024-09-10T09:06:13.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:12 smithi174 ceph-mon[34068]: pgmap v949: 129 pgs: 129 active+clean; 55 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 136 KiB/s wr, 356 op/s 2024-09-10T09:06:13.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:12 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T09:06:13.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:12 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T09:06:13.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:12 smithi191 ceph-mon[35314]: scrub summary: idle 2024-09-10T09:06:13.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:12 smithi191 ceph-mon[35314]: pgmap v949: 129 pgs: 129 active+clean; 55 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 136 KiB/s wr, 356 op/s 2024-09-10T09:06:15.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:14 smithi172 ceph-mon[30854]: pgmap v950: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 56 KiB/s wr, 310 op/s 2024-09-10T09:06:15.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:14 smithi174 ceph-mon[34068]: pgmap v950: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 56 KiB/s wr, 310 op/s 2024-09-10T09:06:15.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:14 smithi191 ceph-mon[35314]: pgmap v950: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 56 KiB/s wr, 310 op/s 2024-09-10T09:06:17.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:16 smithi172 ceph-mon[30854]: pgmap v951: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 239 op/s 2024-09-10T09:06:17.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:16 smithi174 ceph-mon[34068]: pgmap v951: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 239 op/s 2024-09-10T09:06:17.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:16 smithi191 ceph-mon[35314]: pgmap v951: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 239 op/s 2024-09-10T09:06:19.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:18 smithi172 ceph-mon[30854]: pgmap v952: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 178 op/s 2024-09-10T09:06:19.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:18 smithi174 ceph-mon[34068]: pgmap v952: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 178 op/s 2024-09-10T09:06:19.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:18 smithi191 ceph-mon[35314]: pgmap v952: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 178 op/s 2024-09-10T09:06:21.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:20 smithi172 ceph-mon[30854]: pgmap v953: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.2 KiB/s wr, 59 op/s 2024-09-10T09:06:21.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:20 smithi174 ceph-mon[34068]: pgmap v953: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.2 KiB/s wr, 59 op/s 2024-09-10T09:06:21.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:20 smithi191 ceph-mon[35314]: pgmap v953: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.2 KiB/s wr, 59 op/s 2024-09-10T09:06:23.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:22 smithi172 ceph-mon[30854]: pgmap v954: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 938 B/s wr, 42 op/s 2024-09-10T09:06:23.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:22 smithi174 ceph-mon[34068]: pgmap v954: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 938 B/s wr, 42 op/s 2024-09-10T09:06:23.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:22 smithi191 ceph-mon[35314]: pgmap v954: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 938 B/s wr, 42 op/s 2024-09-10T09:06:25.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:24 smithi172 ceph-mon[30854]: pgmap v955: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-10T09:06:25.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:24 smithi174 ceph-mon[34068]: pgmap v955: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-10T09:06:25.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:24 smithi191 ceph-mon[35314]: pgmap v955: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 13 op/s 2024-09-10T09:06:27.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:26 smithi172 ceph-mon[30854]: pgmap v956: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-10T09:06:27.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:26 smithi174 ceph-mon[34068]: pgmap v956: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-10T09:06:27.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:26 smithi191 ceph-mon[35314]: pgmap v956: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-10T09:06:29.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:28 smithi174 ceph-mon[34068]: pgmap v957: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T09:06:29.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:28 smithi191 ceph-mon[35314]: pgmap v957: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T09:06:29.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:28 smithi172 ceph-mon[30854]: pgmap v957: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T09:06:29.575 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:06:29.840 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:06:30.392 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.369+0000 7ffa1eb2e640 1 -- 172.21.15.172:0/3264071472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa18102ef0 msgr2=0x7ffa1810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.369+0000 7ffa1eb2e640 1 --2- 172.21.15.172:0/3264071472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa18102ef0 0x7ffa1810da40 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009ca0 tx=0x7ffa1002a950 comp rx=0 tx=0).stop 2024-09-10T09:06:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.369+0000 7ffa1eb2e640 1 -- 172.21.15.172:0/3264071472 shutdown_connections 2024-09-10T09:06:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.369+0000 7ffa1eb2e640 1 --2- 172.21.15.172:0/3264071472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa18102ef0 0x7ffa1810da40 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.369+0000 7ffa1eb2e640 1 --2- 172.21.15.172:0/3264071472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa18102570 0x7ffa181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.369+0000 7ffa1eb2e640 1 --2- 172.21.15.172:0/3264071472 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa18100e20 0x7ffa18101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.369+0000 7ffa1eb2e640 1 -- 172.21.15.172:0/3264071472 >> 172.21.15.172:0/3264071472 conn(0x7ffa180fc7d0 msgr2=0x7ffa180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.369+0000 7ffa1eb2e640 1 -- 172.21.15.172:0/3264071472 shutdown_connections 2024-09-10T09:06:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.369+0000 7ffa1eb2e640 1 -- 172.21.15.172:0/3264071472 wait complete. 2024-09-10T09:06:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.370+0000 7ffa1eb2e640 1 Processor -- start 2024-09-10T09:06:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.370+0000 7ffa1eb2e640 1 -- start start 2024-09-10T09:06:30.393 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.370+0000 7ffa1eb2e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa18100e20 0x7ffa18197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.370+0000 7ffa1eb2e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa18102570 0x7ffa18197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.370+0000 7ffa1eb2e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa18102ef0 0x7ffa1819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa1eb2e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffa18110e30 con 0x7ffa18102570 2024-09-10T09:06:30.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa1eb2e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa18110cb0 con 0x7ffa18102ef0 2024-09-10T09:06:30.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa1eb2e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffa18110fb0 con 0x7ffa18100e20 2024-09-10T09:06:30.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa17fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa18102570 0x7ffa18197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa17fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa18102570 0x7ffa18197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53402/0 (socket says 172.21.15.172:53402) 2024-09-10T09:06:30.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa17fff640 1 -- 172.21.15.172:0/4249739414 learned_addr learned my addr 172.21.15.172:0/4249739414 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:30.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa1c8ca640 1 --2- 172.21.15.172:0/4249739414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa18100e20 0x7ffa18197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa1d0cb640 1 --2- 172.21.15.172:0/4249739414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa18102ef0 0x7ffa1819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa17fff640 1 -- 172.21.15.172:0/4249739414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa18100e20 msgr2=0x7ffa18197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:30.394 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa17fff640 1 --2- 172.21.15.172:0/4249739414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa18100e20 0x7ffa18197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa17fff640 1 -- 172.21.15.172:0/4249739414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa18102ef0 msgr2=0x7ffa1819bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:30.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa17fff640 1 --2- 172.21.15.172:0/4249739414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa18102ef0 0x7ffa1819bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa17fff640 1 -- 172.21.15.172:0/4249739414 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1819c2e0 con 0x7ffa18102570 2024-09-10T09:06:30.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa1d0cb640 1 --2- 172.21.15.172:0/4249739414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa18102ef0 0x7ffa1819bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:06:30.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa1c8ca640 1 --2- 172.21.15.172:0/4249739414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa18100e20 0x7ffa18197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:30.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa17fff640 1 --2- 172.21.15.172:0/4249739414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa18102570 0x7ffa18197870 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400e9f0 tx=0x7ffa0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:30.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa15ffb640 1 -- 172.21.15.172:0/4249739414 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400cd80 con 0x7ffa18102570 2024-09-10T09:06:30.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa15ffb640 1 -- 172.21.15.172:0/4249739414 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa040108e0 con 0x7ffa18102570 2024-09-10T09:06:30.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa1eb2e640 1 -- 172.21.15.172:0/4249739414 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1819c5d0 con 0x7ffa18102570 2024-09-10T09:06:30.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.371+0000 7ffa15ffb640 1 -- 172.21.15.172:0/4249739414 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04018920 con 0x7ffa18102570 2024-09-10T09:06:30.395 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.372+0000 7ffa1eb2e640 1 -- 172.21.15.172:0/4249739414 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa181a3eb0 con 0x7ffa18102570 2024-09-10T09:06:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.374+0000 7ffa15ffb640 1 -- 172.21.15.172:0/4249739414 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ffa04010a90 con 0x7ffa18102570 2024-09-10T09:06:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.374+0000 7ffa20f07740 1 Processor -- start 2024-09-10T09:06:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.374+0000 7ffa20f07740 1 -- start start 2024-09-10T09:06:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.374+0000 7ffa20f07740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf43a0 0x55e9e8cf4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.374+0000 7ffa20f07740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8d01350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.374+0000 7ffa20f07740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8d01890 0x55e9e8d03c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.374+0000 7ffa20f07740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e9e8c5a2e0 con 0x55e9e8cf4cc0 2024-09-10T09:06:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.374+0000 7ffa20f07740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e9e8c01ce0 con 0x55e9e8cf43a0 2024-09-10T09:06:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.374+0000 7ffa20f07740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e9e8c562a0 con 0x55e9e8d01890 2024-09-10T09:06:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.374+0000 7ffa1c8ca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8d01350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.374+0000 7ffa1c8ca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8d01350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53408/0 (socket says 172.21.15.172:53408) 2024-09-10T09:06:30.396 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.375+0000 7ffa1c8ca640 1 -- 172.21.15.172:0/588793075 learned_addr learned my addr 172.21.15.172:0/588793075 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.375+0000 7ffa1d0cb640 1 --2- 172.21.15.172:0/588793075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf43a0 0x55e9e8cf4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.375+0000 7ffa17fff640 1 --2- 172.21.15.172:0/588793075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8d01890 0x55e9e8d03c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.375+0000 7ffa1c8ca640 1 -- 172.21.15.172:0/588793075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8d01890 msgr2=0x55e9e8d03c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.375+0000 7ffa1c8ca640 1 --2- 172.21.15.172:0/588793075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8d01890 0x55e9e8d03c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.375+0000 7ffa1c8ca640 1 -- 172.21.15.172:0/588793075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf43a0 msgr2=0x55e9e8cf4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.375+0000 7ffa1c8ca640 1 --2- 172.21.15.172:0/588793075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf43a0 0x55e9e8cf4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.375+0000 7ffa1c8ca640 1 -- 172.21.15.172:0/588793075 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9e8be6440 con 0x55e9e8cf4cc0 2024-09-10T09:06:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.375+0000 7ffa1d0cb640 1 --2- 172.21.15.172:0/588793075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf43a0 0x55e9e8cf4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.375+0000 7ffa15ffb640 1 --2- 172.21.15.172:0/4249739414 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa080a2b80 0x7ffa080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.375+0000 7ffa1d0cb640 1 --2- 172.21.15.172:0/4249739414 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa080a2b80 0x7ffa080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.375+0000 7ffa1c8ca640 1 --2- 172.21.15.172:0/588793075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8d01350 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00bcc0 tx=0x7ffa0c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f168927c4b33d740 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:30.397 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.375+0000 7ffa15ffb640 1 -- 172.21.15.172:0/4249739414 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ffa040cf7f0 con 0x7ffa18102570 2024-09-10T09:06:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.375+0000 7ffa02ffd640 1 -- 172.21.15.172:0/588793075 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c005020 con 0x55e9e8cf4cc0 2024-09-10T09:06:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.375+0000 7ffa02ffd640 1 -- 172.21.15.172:0/588793075 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa0c0047a0 con 0x55e9e8cf4cc0 2024-09-10T09:06:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.375+0000 7ffa02ffd640 1 -- 172.21.15.172:0/588793075 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c0116a0 con 0x55e9e8cf4cc0 2024-09-10T09:06:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.376+0000 7ffa1d0cb640 1 --2- 172.21.15.172:0/4249739414 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa080a2b80 0x7ffa080a5040 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ffa10006010 tx=0x7ffa10005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.376+0000 7ffa20f07740 1 -- 172.21.15.172:0/588793075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf4cc0 msgr2=0x55e9e8d01350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.376+0000 7ffa20f07740 1 --2- 172.21.15.172:0/588793075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8d01350 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00bcc0 tx=0x7ffa0c00f730 comp rx=0 tx=0).stop 2024-09-10T09:06:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.376+0000 7ffa20f07740 1 -- 172.21.15.172:0/588793075 shutdown_connections 2024-09-10T09:06:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.376+0000 7ffa20f07740 1 --2- 172.21.15.172:0/588793075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8d01890 0x55e9e8d03c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.376+0000 7ffa20f07740 1 --2- 172.21.15.172:0/588793075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8d01350 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.398 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.376+0000 7ffa20f07740 1 --2- 172.21.15.172:0/588793075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf43a0 0x55e9e8cf4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.376+0000 7ffa20f07740 1 -- 172.21.15.172:0/588793075 >> 172.21.15.172:0/588793075 conn(0x55e9e8cf0a30 msgr2=0x55e9e8cf0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.376+0000 7ffa20f07740 1 -- 172.21.15.172:0/588793075 shutdown_connections 2024-09-10T09:06:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.376+0000 7ffa20f07740 1 -- 172.21.15.172:0/588793075 wait complete. 2024-09-10T09:06:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa20f07740 1 -- start start 2024-09-10T09:06:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa20f07740 1 Processor -- start 2024-09-10T09:06:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa20f07740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf43a0 0x55e9e8d011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa20f07740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8cfe480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa20f07740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8d01890 0x55e9e8cfe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa20f07740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e9e8d044c0 con 0x55e9e8cf43a0 2024-09-10T09:06:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa20f07740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e9e8c15050 con 0x55e9e8cf4cc0 2024-09-10T09:06:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa20f07740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e9e8d04640 con 0x55e9e8d01890 2024-09-10T09:06:30.399 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa20f07740 10 client.0 _ll_register_callbacks cb 0x55e9e8c55160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:06:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa1c8ca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf43a0 0x55e9e8d011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa1c8ca640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf43a0 0x55e9e8d011d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53418/0 (socket says 172.21.15.172:53418) 2024-09-10T09:06:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa1c8ca640 1 -- 172.21.15.172:0/2976356603 learned_addr learned my addr 172.21.15.172:0/2976356603 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa17fff640 1 --2- 172.21.15.172:0/2976356603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8cfe480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa1d0cb640 1 --2- 172.21.15.172:0/2976356603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8d01890 0x55e9e8cfe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa1c8ca640 1 -- 172.21.15.172:0/2976356603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8d01890 msgr2=0x55e9e8cfe9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa1c8ca640 1 --2- 172.21.15.172:0/2976356603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8d01890 0x55e9e8cfe9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa1c8ca640 1 -- 172.21.15.172:0/2976356603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf4cc0 msgr2=0x55e9e8cfe480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa1c8ca640 1 --2- 172.21.15.172:0/2976356603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8cfe480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa1c8ca640 1 -- 172.21.15.172:0/2976356603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9e89cb3c0 con 0x55e9e8cf43a0 2024-09-10T09:06:30.400 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.377+0000 7ffa1d0cb640 1 --2- 172.21.15.172:0/2976356603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8d01890 0x55e9e8cfe9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.378+0000 7ffa1c8ca640 1 --2- 172.21.15.172:0/2976356603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf43a0 0x55e9e8d011d0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c000c00 tx=0x7ffa0c004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25cf71cc1565b98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.378+0000 7ff9f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:06:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.378+0000 7ff9f7fff640 1 -- 172.21.15.172:0/2976356603 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c02a060 con 0x55e9e8cf43a0 2024-09-10T09:06:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.378+0000 7ff9f7fff640 1 -- 172.21.15.172:0/2976356603 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ffa0c004c80 con 0x55e9e8cf43a0 2024-09-10T09:06:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.378+0000 7ffa20f07740 1 -- 172.21.15.172:0/2976356603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa0c0047a0 con 0x55e9e8cf43a0 2024-09-10T09:06:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.378+0000 7ff9f7fff640 1 -- 172.21.15.172:0/2976356603 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ffa0c031050 con 0x55e9e8cf43a0 2024-09-10T09:06:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.378+0000 7ffa20f07740 10 client.20304 fetch_fsmap learned FSMap version 49 2024-09-10T09:06:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.378+0000 7ffa20f07740 1 -- 172.21.15.172:0/2976356603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55e9e8d0fee0 con 0x55e9e8cf43a0 2024-09-10T09:06:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.378+0000 7ff9f7fff640 1 -- 172.21.15.172:0/2976356603 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa0c0047a0 con 0x55e9e8cf43a0 2024-09-10T09:06:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.378+0000 7ff9f7fff640 1 -- 172.21.15.172:0/2976356603 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 49) ==== 8398+0+0 (secure 0 0 0) 0x7ffa0c0035f0 con 0x55e9e8cf43a0 2024-09-10T09:06:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.378+0000 7ffa20f07740 10 client.20304 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T09:06:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.378+0000 7ffa20f07740 10 client.20304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:06:30.401 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.378+0000 7ffa20f07740 20 client.20304 populate_metadata read hostname 'smithi172' 2024-09-10T09:06:30.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.378+0000 7ffa20f07740 1 --2- 172.21.15.172:0/2976356603 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e9e8d112e0 0x55e9e8d136d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.378+0000 7ffa20f07740 4 client.20304 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:06:30.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.378+0000 7ffa20f07740 1 -- 172.21.15.172:0/2976356603 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9e8c67850 con 0x55e9e8d112e0 2024-09-10T09:06:30.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.379+0000 7ffa17fff640 1 --2- 172.21.15.172:0/2976356603 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e9e8d112e0 0x55e9e8d136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.379+0000 7ff9f7fff640 10 client.20304 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:06:30.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.379+0000 7ffa17fff640 1 --2- 172.21.15.172:0/2976356603 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e9e8d112e0 0x55e9e8d136d0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4a08deaff0ba11c server_cookie=9612e3a48bf56088 in_seq=0 out_seq=0 2024-09-10T09:06:30.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.380+0000 7ff9f7fff640 1 -- 172.21.15.172:0/2976356603 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e9e8c67850 con 0x55e9e8d112e0 2024-09-10T09:06:30.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.380+0000 7ff9f7fff640 10 client.20304 handle_command_reply: tid=0 2024-09-10T09:06:30.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.380+0000 7ffa20f07740 1 client.20304 shutdown 2024-09-10T09:06:30.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.380+0000 7ffa20f07740 1 -- 172.21.15.172:0/2976356603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf43a0 msgr2=0x55e9e8d011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:30.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.380+0000 7ffa20f07740 1 --2- 172.21.15.172:0/2976356603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf43a0 0x55e9e8d011d0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c000c00 tx=0x7ffa0c004950 comp rx=0 tx=0).stop 2024-09-10T09:06:30.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.380+0000 7ffa20f07740 1 -- 172.21.15.172:0/2976356603 shutdown_connections 2024-09-10T09:06:30.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.380+0000 7ffa20f07740 1 --2- 172.21.15.172:0/2976356603 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e9e8d112e0 0x55e9e8d136d0 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.380+0000 7ffa20f07740 1 --2- 172.21.15.172:0/2976356603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8d01890 0x55e9e8cfe9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.380+0000 7ffa20f07740 1 --2- 172.21.15.172:0/2976356603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8cfe480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.380+0000 7ffa20f07740 1 --2- 172.21.15.172:0/2976356603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf43a0 0x55e9e8d011d0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.380+0000 7ffa20f07740 1 -- 172.21.15.172:0/2976356603 >> 172.21.15.172:0/2976356603 conn(0x55e9e8cf0a30 msgr2=0x55e9e8cf6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:30.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.380+0000 7ff9f7fff640 0 client.20304 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:06:30.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.380+0000 7ffa20f07740 1 -- 172.21.15.172:0/2976356603 shutdown_connections 2024-09-10T09:06:30.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.380+0000 7ffa20f07740 1 -- 172.21.15.172:0/2976356603 wait complete. 2024-09-10T09:06:30.403 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.380+0000 7ffa20f07740 20 client.20304 trim_cache size 0 max 16384 2024-09-10T09:06:30.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa20f07740 1 Processor -- start 2024-09-10T09:06:30.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa20f07740 1 -- start start 2024-09-10T09:06:30.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa20f07740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8cf0a30 0x55e9e8d0d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa20f07740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf43a0 0x55e9e8d18cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa20f07740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8d19210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa20f07740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffa181ac9a0 con 0x55e9e8cf4cc0 2024-09-10T09:06:30.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa20f07740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa181ae5a0 con 0x55e9e8cf43a0 2024-09-10T09:06:30.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa20f07740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffa181ad070 con 0x55e9e8cf0a30 2024-09-10T09:06:30.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa1d0cb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8d19210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa1d0cb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8d19210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53432/0 (socket says 172.21.15.172:53432) 2024-09-10T09:06:30.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa1d0cb640 1 -- 172.21.15.172:0/3003588934 learned_addr learned my addr 172.21.15.172:0/3003588934 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:30.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa17fff640 1 --2- 172.21.15.172:0/3003588934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf43a0 0x55e9e8d18cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa1c8ca640 1 --2- 172.21.15.172:0/3003588934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8cf0a30 0x55e9e8d0d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa1d0cb640 1 -- 172.21.15.172:0/3003588934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8cf0a30 msgr2=0x55e9e8d0d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:30.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa1d0cb640 1 --2- 172.21.15.172:0/3003588934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8cf0a30 0x55e9e8d0d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa1d0cb640 1 -- 172.21.15.172:0/3003588934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf43a0 msgr2=0x55e9e8d18cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:30.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa1d0cb640 1 --2- 172.21.15.172:0/3003588934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf43a0 0x55e9e8d18cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.398+0000 7ffa1d0cb640 1 -- 172.21.15.172:0/3003588934 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9e8cf90f0 con 0x55e9e8cf4cc0 2024-09-10T09:06:30.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.399+0000 7ffa1d0cb640 1 --2- 172.21.15.172:0/3003588934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8d19210 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7ffa10035000 tx=0x7ffa1003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9785876ea0e91868 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:30.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.399+0000 7ffa027fc640 1 -- 172.21.15.172:0/3003588934 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10034040 con 0x55e9e8cf4cc0 2024-09-10T09:06:30.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.399+0000 7ffa027fc640 1 -- 172.21.15.172:0/3003588934 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa10003e20 con 0x55e9e8cf4cc0 2024-09-10T09:06:30.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.399+0000 7ffa027fc640 1 -- 172.21.15.172:0/3003588934 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10052460 con 0x55e9e8cf4cc0 2024-09-10T09:06:30.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.400+0000 7ffa20f07740 1 -- 172.21.15.172:0/3003588934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf4cc0 msgr2=0x55e9e8d19210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:30.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.400+0000 7ffa20f07740 1 --2- 172.21.15.172:0/3003588934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8d19210 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7ffa10035000 tx=0x7ffa1003f000 comp rx=0 tx=0).stop 2024-09-10T09:06:30.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.400+0000 7ffa20f07740 1 -- 172.21.15.172:0/3003588934 shutdown_connections 2024-09-10T09:06:30.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.400+0000 7ffa20f07740 1 --2- 172.21.15.172:0/3003588934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8d19210 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.400+0000 7ffa20f07740 1 --2- 172.21.15.172:0/3003588934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf43a0 0x55e9e8d18cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.400+0000 7ffa20f07740 1 --2- 172.21.15.172:0/3003588934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8cf0a30 0x55e9e8d0d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.400+0000 7ffa20f07740 1 -- 172.21.15.172:0/3003588934 >> 172.21.15.172:0/3003588934 conn(0x55e9e8cfeb20 msgr2=0x55e9e8cfcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.400+0000 7ffa20f07740 1 -- 172.21.15.172:0/3003588934 shutdown_connections 2024-09-10T09:06:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.400+0000 7ffa20f07740 1 -- 172.21.15.172:0/3003588934 wait complete. 2024-09-10T09:06:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.400+0000 7ffa20f07740 1 -- start start 2024-09-10T09:06:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.400+0000 7ffa20f07740 1 Processor -- start 2024-09-10T09:06:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.400+0000 7ffa20f07740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8cf0a30 0x55e9e8d777e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.400+0000 7ffa20f07740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf43a0 0x55e9e8d098f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.400+0000 7ffa20f07740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8d09e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa20f07740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffa181adea0 con 0x55e9e8cf43a0 2024-09-10T09:06:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa20f07740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa181ad770 con 0x55e9e8cf4cc0 2024-09-10T09:06:30.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa20f07740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffa181aa920 con 0x55e9e8cf0a30 2024-09-10T09:06:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa20f07740 10 client.0 _ll_register_callbacks cb 0x7ffa0c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:06:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa17fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf43a0 0x55e9e8d098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa17fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf43a0 0x55e9e8d098f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53448/0 (socket says 172.21.15.172:53448) 2024-09-10T09:06:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa17fff640 1 -- 172.21.15.172:0/2039444659 learned_addr learned my addr 172.21.15.172:0/2039444659 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa1d0cb640 1 --2- 172.21.15.172:0/2039444659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8d09e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa1c8ca640 1 --2- 172.21.15.172:0/2039444659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8cf0a30 0x55e9e8d777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa17fff640 1 -- 172.21.15.172:0/2039444659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8cf0a30 msgr2=0x55e9e8d777e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa17fff640 1 --2- 172.21.15.172:0/2039444659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8cf0a30 0x55e9e8d777e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa17fff640 1 -- 172.21.15.172:0/2039444659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf4cc0 msgr2=0x55e9e8d09e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa17fff640 1 --2- 172.21.15.172:0/2039444659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8d09e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa17fff640 1 -- 172.21.15.172:0/2039444659 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9e8d0a520 con 0x55e9e8cf43a0 2024-09-10T09:06:30.413 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa1c8ca640 1 --2- 172.21.15.172:0/2039444659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8cf0a30 0x55e9e8d777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:06:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa1d0cb640 1 --2- 172.21.15.172:0/2039444659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8d09e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa17fff640 1 --2- 172.21.15.172:0/2039444659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf43a0 0x55e9e8d098f0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7ffa040980f0 tx=0x7ffa04000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a2f2d2db5e38dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ff9f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:06:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ff9f7fff640 1 -- 172.21.15.172:0/2039444659 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa040a0040 con 0x55e9e8cf43a0 2024-09-10T09:06:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ff9f7fff640 1 -- 172.21.15.172:0/2039444659 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7ffa04098300 con 0x55e9e8cf43a0 2024-09-10T09:06:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa20f07740 1 -- 172.21.15.172:0/2039444659 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa10003e20 con 0x55e9e8cf43a0 2024-09-10T09:06:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ff9f7fff640 1 -- 172.21.15.172:0/2039444659 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ffa040a4900 con 0x55e9e8cf43a0 2024-09-10T09:06:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa20f07740 10 client.20310 fetch_fsmap learned FSMap version 49 2024-09-10T09:06:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.401+0000 7ffa20f07740 1 -- 172.21.15.172:0/2039444659 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55e9e8d829b0 con 0x55e9e8cf43a0 2024-09-10T09:06:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.402+0000 7ff9f7fff640 1 -- 172.21.15.172:0/2039444659 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa10003e20 con 0x55e9e8cf43a0 2024-09-10T09:06:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.402+0000 7ff9f7fff640 1 -- 172.21.15.172:0/2039444659 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 49) ==== 8398+0+0 (secure 0 0 0) 0x7ffa04063080 con 0x55e9e8cf43a0 2024-09-10T09:06:30.414 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.402+0000 7ffa20f07740 10 client.20310 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T09:06:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.402+0000 7ffa20f07740 10 client.20310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:06:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.402+0000 7ffa20f07740 20 client.20310 populate_metadata read hostname 'smithi172' 2024-09-10T09:06:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.402+0000 7ffa20f07740 1 --2- 172.21.15.172:0/2039444659 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e9e8d01890 0x55e9e8d850d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.402+0000 7ffa20f07740 4 client.20310 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:06:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.402+0000 7ffa20f07740 1 -- 172.21.15.172:0/2039444659 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e9e8c61d60 con 0x55e9e8d01890 2024-09-10T09:06:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.402+0000 7ffa1c8ca640 1 --2- 172.21.15.172:0/2039444659 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e9e8d01890 0x55e9e8d850d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.403+0000 7ff9f7fff640 10 client.20310 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:06:30.415 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.403+0000 7ffa1c8ca640 1 --2- 172.21.15.172:0/2039444659 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e9e8d01890 0x55e9e8d850d0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42a71e1d48b8d659 server_cookie=d5755dc942eea431 in_seq=0 out_seq=0 2024-09-10T09:06:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.403+0000 7ff9f7fff640 1 -- 172.21.15.172:0/2039444659 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e9e8c61d60 con 0x55e9e8d01890 2024-09-10T09:06:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.403+0000 7ff9f7fff640 10 client.20310 handle_command_reply: tid=0 2024-09-10T09:06:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.403+0000 7ffa20f07740 1 client.20310 shutdown 2024-09-10T09:06:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.403+0000 7ffa20f07740 1 -- 172.21.15.172:0/2039444659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf43a0 msgr2=0x55e9e8d098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.403+0000 7ffa20f07740 1 --2- 172.21.15.172:0/2039444659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf43a0 0x55e9e8d098f0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7ffa040980f0 tx=0x7ffa04000bc0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.403+0000 7ffa20f07740 1 -- 172.21.15.172:0/2039444659 shutdown_connections 2024-09-10T09:06:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.403+0000 7ffa20f07740 1 --2- 172.21.15.172:0/2039444659 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x55e9e8d01890 0x55e9e8d850d0 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.403+0000 7ffa20f07740 1 --2- 172.21.15.172:0/2039444659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9e8cf4cc0 0x55e9e8d09e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.403+0000 7ff9f7fff640 0 client.20310 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:06:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.403+0000 7ffa20f07740 1 --2- 172.21.15.172:0/2039444659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9e8cf43a0 0x55e9e8d098f0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.403+0000 7ffa20f07740 1 --2- 172.21.15.172:0/2039444659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e9e8cf0a30 0x55e9e8d777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.416 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.403+0000 7ffa20f07740 1 -- 172.21.15.172:0/2039444659 >> 172.21.15.172:0/2039444659 conn(0x55e9e8cfeb20 msgr2=0x55e9e8cf6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:30.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.403+0000 7ffa20f07740 1 -- 172.21.15.172:0/2039444659 shutdown_connections 2024-09-10T09:06:30.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.403+0000 7ffa20f07740 1 -- 172.21.15.172:0/2039444659 wait complete. 2024-09-10T09:06:30.417 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.403+0000 7ffa20f07740 20 client.20310 trim_cache size 0 max 16384 2024-09-10T09:06:30.417 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:06:30.417 INFO:teuthology.orchestra.run.smithi172.stdout: "status": "no active scrubs running", 2024-09-10T09:06:30.417 INFO:teuthology.orchestra.run.smithi172.stdout: "scrubs": {} 2024-09-10T09:06:30.417 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:06:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.404+0000 7ffa1eb2e640 1 -- 172.21.15.172:0/4249739414 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa080a2b80 msgr2=0x7ffa080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.404+0000 7ffa1eb2e640 1 --2- 172.21.15.172:0/4249739414 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa080a2b80 0x7ffa080a5040 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ffa10006010 tx=0x7ffa10005d80 comp rx=0 tx=0).stop 2024-09-10T09:06:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.404+0000 7ffa1eb2e640 1 -- 172.21.15.172:0/4249739414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa18102570 msgr2=0x7ffa18197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.404+0000 7ffa1eb2e640 1 --2- 172.21.15.172:0/4249739414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa18102570 0x7ffa18197870 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400e9f0 tx=0x7ffa0400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.418 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.404+0000 7ffa1eb2e640 1 -- 172.21.15.172:0/4249739414 shutdown_connections 2024-09-10T09:06:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.404+0000 7ffa1eb2e640 1 --2- 172.21.15.172:0/4249739414 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffa080a2b80 0x7ffa080a5040 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.404+0000 7ffa1eb2e640 1 --2- 172.21.15.172:0/4249739414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa18102ef0 0x7ffa1819bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.404+0000 7ffa1eb2e640 1 --2- 172.21.15.172:0/4249739414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa18102570 0x7ffa18197870 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.404+0000 7ffa1eb2e640 1 --2- 172.21.15.172:0/4249739414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa18100e20 0x7ffa18197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.404+0000 7ffa1eb2e640 1 -- 172.21.15.172:0/4249739414 >> 172.21.15.172:0/4249739414 conn(0x7ffa180fc7d0 msgr2=0x7ffa180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.404+0000 7ffa1eb2e640 1 -- 172.21.15.172:0/4249739414 shutdown_connections 2024-09-10T09:06:30.419 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:30.404+0000 7ffa1eb2e640 1 -- 172.21.15.172:0/4249739414 wait complete. 2024-09-10T09:06:31.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:30 smithi174 ceph-mon[34068]: pgmap v958: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:06:31.289 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:30 smithi172 ceph-mon[30854]: pgmap v958: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:06:31.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:30 smithi191 ceph-mon[35314]: pgmap v958: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:06:31.318 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:06:31.318 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:06:31.577 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:06:32.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.083+0000 7f3199049640 1 -- 172.21.15.172:0/2643017037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3194100a80 msgr2=0x7f3194100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:32.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.083+0000 7f3199049640 1 --2- 172.21.15.172:0/2643017037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3194100a80 0x7f3194100ee0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f3188009890 tx=0x7f318802a380 comp rx=0 tx=0).stop 2024-09-10T09:06:32.113 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.083+0000 7f3199049640 1 -- 172.21.15.172:0/2643017037 shutdown_connections 2024-09-10T09:06:32.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.083+0000 7f3199049640 1 --2- 172.21.15.172:0/2643017037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3194101420 0x7f319410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:32.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.083+0000 7f3199049640 1 --2- 172.21.15.172:0/2643017037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3194100a80 0x7f3194100ee0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:32.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.083+0000 7f3199049640 1 --2- 172.21.15.172:0/2643017037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3194106a80 0x7f3194106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:32.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.083+0000 7f3199049640 1 -- 172.21.15.172:0/2643017037 >> 172.21.15.172:0/2643017037 conn(0x7f31940fc7a0 msgr2=0x7f31940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:32.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.083+0000 7f3199049640 1 -- 172.21.15.172:0/2643017037 shutdown_connections 2024-09-10T09:06:32.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.083+0000 7f3199049640 1 -- 172.21.15.172:0/2643017037 wait complete. 2024-09-10T09:06:32.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.084+0000 7f3199049640 1 Processor -- start 2024-09-10T09:06:32.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.084+0000 7f3199049640 1 -- start start 2024-09-10T09:06:32.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f3199049640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3194100a80 0x7f3194199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:32.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f3199049640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3194101420 0x7f31941996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:32.114 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f3199049640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3194106a80 0x7f31941a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:32.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f3199049640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3194110d30 con 0x7f3194106a80 2024-09-10T09:06:32.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f3199049640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3194110bb0 con 0x7f3194100a80 2024-09-10T09:06:32.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f3199049640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3194110eb0 con 0x7f3194101420 2024-09-10T09:06:32.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f319359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3194106a80 0x7f31941a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:32.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f319359e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3194106a80 0x7f31941a4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53456/0 (socket says 172.21.15.172:53456) 2024-09-10T09:06:32.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f319359e640 1 -- 172.21.15.172:0/1020628591 learned_addr learned my addr 172.21.15.172:0/1020628591 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:32.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f3192d9d640 1 --2- 172.21.15.172:0/1020628591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3194100a80 0x7f3194199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:32.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f319259c640 1 --2- 172.21.15.172:0/1020628591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3194101420 0x7f31941996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:32.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f319359e640 1 -- 172.21.15.172:0/1020628591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3194101420 msgr2=0x7f31941996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:32.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f319359e640 1 --2- 172.21.15.172:0/1020628591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3194101420 0x7f31941996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:32.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f319359e640 1 -- 172.21.15.172:0/1020628591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3194100a80 msgr2=0x7f3194199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:32.115 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f319359e640 1 --2- 172.21.15.172:0/1020628591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3194100a80 0x7f3194199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:32.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f319359e640 1 -- 172.21.15.172:0/1020628591 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31941a4740 con 0x7f3194106a80 2024-09-10T09:06:32.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f3192d9d640 1 --2- 172.21.15.172:0/1020628591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3194100a80 0x7f3194199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:32.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f319359e640 1 --2- 172.21.15.172:0/1020628591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3194106a80 0x7f31941a4200 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f318c00eb10 tx=0x7f318c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:32.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f317bfff640 1 -- 172.21.15.172:0/1020628591 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c004b20 con 0x7f3194106a80 2024-09-10T09:06:32.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f3199049640 1 -- 172.21.15.172:0/1020628591 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31941a4910 con 0x7f3194106a80 2024-09-10T09:06:32.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f317bfff640 1 -- 172.21.15.172:0/1020628591 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f318c004cc0 con 0x7f3194106a80 2024-09-10T09:06:32.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f317bfff640 1 -- 172.21.15.172:0/1020628591 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c010690 con 0x7f3194106a80 2024-09-10T09:06:32.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.085+0000 7f3199049640 1 -- 172.21.15.172:0/1020628591 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31941a4e50 con 0x7f3194106a80 2024-09-10T09:06:32.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.086+0000 7f3199049640 1 -- 172.21.15.172:0/1020628591 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3194100ee0 con 0x7f3194106a80 2024-09-10T09:06:32.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.088+0000 7f317bfff640 1 -- 172.21.15.172:0/1020628591 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f318c005000 con 0x7f3194106a80 2024-09-10T09:06:32.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.088+0000 7f317bfff640 1 --2- 172.21.15.172:0/1020628591 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f317c0a2b80 0x7f317c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:32.116 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.088+0000 7f3192d9d640 1 --2- 172.21.15.172:0/1020628591 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f317c0a2b80 0x7f317c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:32.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.088+0000 7f317bfff640 1 -- 172.21.15.172:0/1020628591 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f318c0cff10 con 0x7f3194106a80 2024-09-10T09:06:32.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.089+0000 7f3192d9d640 1 --2- 172.21.15.172:0/1020628591 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f317c0a2b80 0x7f317c0a5040 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3184005fd0 tx=0x7f3184009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:32.117 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.091+0000 7f317bfff640 1 -- 172.21.15.172:0/1020628591 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f318c09cb10 con 0x7f3194106a80 2024-09-10T09:06:32.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.294+0000 7f3199049640 1 -- 172.21.15.172:0/1020628591 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f319406adc0 con 0x7f3194106a80 2024-09-10T09:06:32.295 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.295+0000 7f317bfff640 1 -- 172.21.15.172:0/1020628591 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+12010 (secure 0 0 0) 0x7f319406adc0 con 0x7f3194106a80 2024-09-10T09:06:32.296 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T09:06:32.296 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":49,"btime":"2024-09-10T09:06:03:774593+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T09:06:03.109907+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T09:06:32.296 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 49 2024-09-10T09:06:32.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.297+0000 7f3199049640 1 -- 172.21.15.172:0/1020628591 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f317c0a2b80 msgr2=0x7f317c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:32.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.297+0000 7f3199049640 1 --2- 172.21.15.172:0/1020628591 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f317c0a2b80 0x7f317c0a5040 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3184005fd0 tx=0x7f3184009260 comp rx=0 tx=0).stop 2024-09-10T09:06:32.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.297+0000 7f3199049640 1 -- 172.21.15.172:0/1020628591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3194106a80 msgr2=0x7f31941a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:32.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.297+0000 7f3199049640 1 --2- 172.21.15.172:0/1020628591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3194106a80 0x7f31941a4200 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f318c00eb10 tx=0x7f318c00c2d0 comp rx=0 tx=0).stop 2024-09-10T09:06:32.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.297+0000 7f3199049640 1 -- 172.21.15.172:0/1020628591 shutdown_connections 2024-09-10T09:06:32.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.297+0000 7f3199049640 1 --2- 172.21.15.172:0/1020628591 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f317c0a2b80 0x7f317c0a5040 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:32.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.297+0000 7f3199049640 1 --2- 172.21.15.172:0/1020628591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3194106a80 0x7f31941a4200 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:32.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.297+0000 7f3199049640 1 --2- 172.21.15.172:0/1020628591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3194101420 0x7f31941996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:32.298 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.297+0000 7f3199049640 1 --2- 172.21.15.172:0/1020628591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3194100a80 0x7f3194199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:32.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.297+0000 7f3199049640 1 -- 172.21.15.172:0/1020628591 >> 172.21.15.172:0/1020628591 conn(0x7f31940fc7a0 msgr2=0x7f31940fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:32.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.297+0000 7f3199049640 1 -- 172.21.15.172:0/1020628591 shutdown_connections 2024-09-10T09:06:32.299 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:32.297+0000 7f3199049640 1 -- 172.21.15.172:0/1020628591 wait complete. 2024-09-10T09:06:33.044 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:06:33.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:32 smithi174 ceph-mon[34068]: pgmap v959: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T09:06:33.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:32 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1020628591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:06:33.293 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:06:33.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:32 smithi191 ceph-mon[35314]: pgmap v959: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T09:06:33.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:32 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1020628591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:06:33.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:32 smithi172 ceph-mon[30854]: pgmap v959: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-10T09:06:33.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:32 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1020628591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:06:33.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.818+0000 7f1473a4e640 1 -- 172.21.15.172:0/958295202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f146c108290 msgr2=0x7f146c10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:33.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.818+0000 7f1473a4e640 1 --2- 172.21.15.172:0/958295202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f146c108290 0x7f146c10ec30 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f1464009cd0 tx=0x7f146402a8e0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.818+0000 7f1473a4e640 1 -- 172.21.15.172:0/958295202 shutdown_connections 2024-09-10T09:06:33.871 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.818+0000 7f1473a4e640 1 --2- 172.21.15.172:0/958295202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f146c108290 0x7f146c10ec30 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.818+0000 7f1473a4e640 1 --2- 172.21.15.172:0/958295202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f146c103740 0x7f146c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.818+0000 7f1473a4e640 1 --2- 172.21.15.172:0/958295202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f146c102d90 0x7f146c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.818+0000 7f1473a4e640 1 -- 172.21.15.172:0/958295202 >> 172.21.15.172:0/958295202 conn(0x7f146c0fc7d0 msgr2=0x7f146c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.818+0000 7f1473a4e640 1 -- 172.21.15.172:0/958295202 shutdown_connections 2024-09-10T09:06:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.819+0000 7f1473a4e640 1 -- 172.21.15.172:0/958295202 wait complete. 2024-09-10T09:06:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.819+0000 7f1473a4e640 1 Processor -- start 2024-09-10T09:06:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.819+0000 7f1473a4e640 1 -- start start 2024-09-10T09:06:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1473a4e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f146c102d90 0x7f146c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1473a4e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f146c103740 0x7f146c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1473a4e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f146c108290 0x7f146c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1473a4e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f146c111f80 con 0x7f146c103740 2024-09-10T09:06:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1473a4e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f146c111e00 con 0x7f146c108290 2024-09-10T09:06:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1473a4e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f146c112100 con 0x7f146c102d90 2024-09-10T09:06:33.872 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1470fe9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f146c103740 0x7f146c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1470fe9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f146c103740 0x7f146c198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53482/0 (socket says 172.21.15.172:53482) 2024-09-10T09:06:33.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1470fe9640 1 -- 172.21.15.172:0/615661970 learned_addr learned my addr 172.21.15.172:0/615661970 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:33.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f14717ea640 1 --2- 172.21.15.172:0/615661970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f146c102d90 0x7f146c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1471feb640 1 --2- 172.21.15.172:0/615661970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f146c108290 0x7f146c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1470fe9640 1 -- 172.21.15.172:0/615661970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f146c102d90 msgr2=0x7f146c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:33.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1470fe9640 1 --2- 172.21.15.172:0/615661970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f146c102d90 0x7f146c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1470fe9640 1 -- 172.21.15.172:0/615661970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f146c108290 msgr2=0x7f146c19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:33.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1470fe9640 1 --2- 172.21.15.172:0/615661970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f146c108290 0x7f146c19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1470fe9640 1 -- 172.21.15.172:0/615661970 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146c19d4a0 con 0x7f146c103740 2024-09-10T09:06:33.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f14717ea640 1 --2- 172.21.15.172:0/615661970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f146c102d90 0x7f146c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:06:33.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1471feb640 1 --2- 172.21.15.172:0/615661970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f146c108290 0x7f146c19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:06:33.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1470fe9640 1 --2- 172.21.15.172:0/615661970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f146c103740 0x7f146c198a30 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f145c00b450 tx=0x7f145c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:33.873 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f14627fc640 1 -- 172.21.15.172:0/615661970 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c015020 con 0x7f146c103740 2024-09-10T09:06:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f14627fc640 1 -- 172.21.15.172:0/615661970 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f145c011d40 con 0x7f146c103740 2024-09-10T09:06:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1473a4e640 1 -- 172.21.15.172:0/615661970 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146c19d790 con 0x7f146c103740 2024-09-10T09:06:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f14627fc640 1 -- 172.21.15.172:0/615661970 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c00f340 con 0x7f146c103740 2024-09-10T09:06:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.820+0000 7f1473a4e640 1 -- 172.21.15.172:0/615661970 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f146c1a5070 con 0x7f146c103740 2024-09-10T09:06:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.823+0000 7f14627fc640 1 -- 172.21.15.172:0/615661970 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f145c018020 con 0x7f146c103740 2024-09-10T09:06:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.823+0000 7f1475e27740 1 Processor -- start 2024-09-10T09:06:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.823+0000 7f1475e27740 1 -- start start 2024-09-10T09:06:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.823+0000 7f14627fc640 1 --2- 172.21.15.172:0/615661970 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f14580a2c00 0x7f14580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.823+0000 7f14627fc640 1 -- 172.21.15.172:0/615661970 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f145c0cef10 con 0x7f146c103740 2024-09-10T09:06:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.823+0000 7f1475e27740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75eb40 0x5575db75d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1471feb640 1 --2- 172.21.15.172:0/615661970 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f14580a2c00 0x7f14580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1475e27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db75d5d0 0x5575db75da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1475e27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db76c780 0x5575db76eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.874 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1475e27740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5575db6c32f0 con 0x5575db75d5d0 2024-09-10T09:06:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1475e27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5575db66acf0 con 0x5575db76c780 2024-09-10T09:06:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1475e27740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5575db6bf2b0 con 0x5575db75eb40 2024-09-10T09:06:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1470fe9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db75d5d0 0x5575db75da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1470fe9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db75d5d0 0x5575db75da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53498/0 (socket says 172.21.15.172:53498) 2024-09-10T09:06:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1470fe9640 1 -- 172.21.15.172:0/3917404279 learned_addr learned my addr 172.21.15.172:0/3917404279 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f14717ea640 1 --2- 172.21.15.172:0/3917404279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75eb40 0x5575db75d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1471feb640 1 --2- 172.21.15.172:0/3917404279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db76c780 0x5575db76eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1470fe9640 1 -- 172.21.15.172:0/3917404279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75eb40 msgr2=0x5575db75d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1470fe9640 1 --2- 172.21.15.172:0/3917404279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75eb40 0x5575db75d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1470fe9640 1 -- 172.21.15.172:0/3917404279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db76c780 msgr2=0x5575db76eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1470fe9640 1 --2- 172.21.15.172:0/3917404279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db76c780 0x5575db76eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1470fe9640 1 -- 172.21.15.172:0/3917404279 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575db64f3a0 con 0x5575db75d5d0 2024-09-10T09:06:33.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1471feb640 1 --2- 172.21.15.172:0/3917404279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db76c780 0x5575db76eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:06:33.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1471feb640 1 --2- 172.21.15.172:0/615661970 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f14580a2c00 0x7f14580a50c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1464009ca0 tx=0x7f1464031570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:33.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f1470fe9640 1 --2- 172.21.15.172:0/3917404279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db75d5d0 0x5575db75da10 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f145c097810 tx=0x7f145c09bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57b5fa7979f2443 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:33.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f14577fe640 1 -- 172.21.15.172:0/3917404279 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c0b4070 con 0x5575db75d5d0 2024-09-10T09:06:33.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f14577fe640 1 -- 172.21.15.172:0/3917404279 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f145c0a4020 con 0x5575db75d5d0 2024-09-10T09:06:33.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.824+0000 7f14577fe640 1 -- 172.21.15.172:0/3917404279 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c017b70 con 0x5575db75d5d0 2024-09-10T09:06:33.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.825+0000 7f1475e27740 1 -- 172.21.15.172:0/3917404279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db75d5d0 msgr2=0x5575db75da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:33.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.825+0000 7f1475e27740 1 --2- 172.21.15.172:0/3917404279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db75d5d0 0x5575db75da10 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f145c097810 tx=0x7f145c09bb70 comp rx=0 tx=0).stop 2024-09-10T09:06:33.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.825+0000 7f1475e27740 1 -- 172.21.15.172:0/3917404279 shutdown_connections 2024-09-10T09:06:33.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.825+0000 7f1475e27740 1 --2- 172.21.15.172:0/3917404279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db76c780 0x5575db76eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.825+0000 7f1475e27740 1 --2- 172.21.15.172:0/3917404279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db75d5d0 0x5575db75da10 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.825+0000 7f1475e27740 1 --2- 172.21.15.172:0/3917404279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75eb40 0x5575db75d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.825+0000 7f1475e27740 1 -- 172.21.15.172:0/3917404279 >> 172.21.15.172:0/3917404279 conn(0x5575db726310 msgr2=0x5575db759c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:33.876 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.825+0000 7f1475e27740 1 -- 172.21.15.172:0/3917404279 shutdown_connections 2024-09-10T09:06:33.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.825+0000 7f1475e27740 1 -- 172.21.15.172:0/3917404279 wait complete. 2024-09-10T09:06:33.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.825+0000 7f1475e27740 1 -- start start 2024-09-10T09:06:33.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.826+0000 7f1475e27740 1 Processor -- start 2024-09-10T09:06:33.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.826+0000 7f1475e27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db75d5d0 0x5575db76c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.826+0000 7f1475e27740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75eb40 0x5575db7694e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.826+0000 7f1475e27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db76c780 0x5575db769c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.877 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:06:33.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.826+0000 7f1475e27740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5575db76f480 con 0x5575db76c780 2024-09-10T09:06:33.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.826+0000 7f1475e27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5575db67e060 con 0x5575db75d5d0 2024-09-10T09:06:33.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.826+0000 7f1475e27740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5575db76f600 con 0x5575db75eb40 2024-09-10T09:06:33.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.826+0000 7f1475e27740 10 client.0 _ll_register_callbacks cb 0x5575db3d18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:06:33.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.826+0000 7f1471feb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db76c780 0x5575db769c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.826+0000 7f1471feb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db76c780 0x5575db769c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53500/0 (socket says 172.21.15.172:53500) 2024-09-10T09:06:33.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.826+0000 7f1471feb640 1 -- 172.21.15.172:0/2389633011 learned_addr learned my addr 172.21.15.172:0/2389633011 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:33.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.826+0000 7f1470fe9640 1 --2- 172.21.15.172:0/2389633011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75eb40 0x5575db7694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.826+0000 7f14717ea640 1 --2- 172.21.15.172:0/2389633011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db75d5d0 0x5575db76c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.826+0000 7f1471feb640 1 -- 172.21.15.172:0/2389633011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75eb40 msgr2=0x5575db7694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:33.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.826+0000 7f1471feb640 1 --2- 172.21.15.172:0/2389633011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75eb40 0x5575db7694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.826+0000 7f1471feb640 1 -- 172.21.15.172:0/2389633011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db75d5d0 msgr2=0x5575db76c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:33.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.826+0000 7f1471feb640 1 --2- 172.21.15.172:0/2389633011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db75d5d0 0x5575db76c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f1471feb640 1 -- 172.21.15.172:0/2389633011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575db4343c0 con 0x5575db76c780 2024-09-10T09:06:33.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f14717ea640 1 --2- 172.21.15.172:0/2389633011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db75d5d0 0x5575db76c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:06:33.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f1471feb640 1 --2- 172.21.15.172:0/2389633011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db76c780 0x5575db769c70 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f1464003ba0 tx=0x7f146402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861e7b7f5e0404d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:33.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f1454ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:06:33.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f1454ff9640 1 -- 172.21.15.172:0/2389633011 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146404c030 con 0x5575db76c780 2024-09-10T09:06:33.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f1454ff9640 1 -- 172.21.15.172:0/2389633011 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f14640045c0 con 0x5575db76c780 2024-09-10T09:06:33.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f1475e27740 1 -- 172.21.15.172:0/2389633011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f145c0a4020 con 0x5575db76c780 2024-09-10T09:06:33.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f1454ff9640 1 -- 172.21.15.172:0/2389633011 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f1464059050 con 0x5575db76c780 2024-09-10T09:06:33.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f1475e27740 10 client.20328 fetch_fsmap learned FSMap version 49 2024-09-10T09:06:33.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f1475e27740 1 -- 172.21.15.172:0/2389633011 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5575db77af60 con 0x5575db76c780 2024-09-10T09:06:33.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f1454ff9640 1 -- 172.21.15.172:0/2389633011 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f145c0a4020 con 0x5575db76c780 2024-09-10T09:06:33.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f1454ff9640 1 -- 172.21.15.172:0/2389633011 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 49) ==== 8398+0+0 (secure 0 0 0) 0x7f1464030070 con 0x5575db76c780 2024-09-10T09:06:33.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f1475e27740 10 client.20328 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T09:06:33.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f1475e27740 10 client.20328 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T09:06:33.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f1475e27740 20 client.20328 populate_metadata read hostname 'smithi172' 2024-09-10T09:06:33.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f1475e27740 1 --2- 172.21.15.172:0/2389633011 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5575db77c470 0x5575db77e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f1475e27740 4 client.20328 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:06:33.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.827+0000 7f1475e27740 1 -- 172.21.15.172:0/2389633011 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575db6d0860 con 0x5575db77c470 2024-09-10T09:06:33.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.828+0000 7f14717ea640 1 --2- 172.21.15.172:0/2389633011 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5575db77c470 0x5575db77e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.828+0000 7f1454ff9640 10 client.20328 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T09:06:33.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.828+0000 7f14717ea640 1 --2- 172.21.15.172:0/2389633011 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5575db77c470 0x5575db77e8c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6a1da94a5a35c421 server_cookie=37b85544eeebb166 in_seq=0 out_seq=0 2024-09-10T09:06:33.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.829+0000 7f1454ff9640 1 -- 172.21.15.172:0/2389633011 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5575db6d0860 con 0x5575db77c470 2024-09-10T09:06:33.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.829+0000 7f1454ff9640 10 client.20328 handle_command_reply: tid=0 2024-09-10T09:06:33.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.829+0000 7f1475e27740 1 client.20328 shutdown 2024-09-10T09:06:33.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.829+0000 7f1475e27740 1 -- 172.21.15.172:0/2389633011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db76c780 msgr2=0x5575db769c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:33.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.829+0000 7f1475e27740 1 --2- 172.21.15.172:0/2389633011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db76c780 0x5575db769c70 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f1464003ba0 tx=0x7f146402a8e0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.829+0000 7f1475e27740 1 -- 172.21.15.172:0/2389633011 shutdown_connections 2024-09-10T09:06:33.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.829+0000 7f1475e27740 1 --2- 172.21.15.172:0/2389633011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db76c780 0x5575db769c70 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.829+0000 7f1475e27740 1 --2- 172.21.15.172:0/2389633011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75eb40 0x5575db7694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.829+0000 7f1475e27740 1 --2- 172.21.15.172:0/2389633011 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5575db77c470 0x5575db77e8c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.829+0000 7f1475e27740 1 --2- 172.21.15.172:0/2389633011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db75d5d0 0x5575db76c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.829+0000 7f1454ff9640 0 client.20328 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T09:06:33.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.829+0000 7f1475e27740 1 -- 172.21.15.172:0/2389633011 >> 172.21.15.172:0/2389633011 conn(0x5575db726310 msgr2=0x5575db75f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:33.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.829+0000 7f1475e27740 1 -- 172.21.15.172:0/2389633011 shutdown_connections 2024-09-10T09:06:33.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.829+0000 7f1475e27740 1 -- 172.21.15.172:0/2389633011 wait complete. 2024-09-10T09:06:33.881 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.829+0000 7f1475e27740 20 client.20328 trim_cache size 0 max 16384 2024-09-10T09:06:33.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.842+0000 7f1475e27740 1 Processor -- start 2024-09-10T09:06:33.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.842+0000 7f1475e27740 1 -- start start 2024-09-10T09:06:33.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.843+0000 7f1475e27740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75d5d0 0x5575db759e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.843+0000 7f1475e27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db75eb40 0x5575db784190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.843+0000 7f1475e27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db76c780 0x5575db7846d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.843+0000 7f1475e27740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1ac9a0 con 0x5575db75eb40 2024-09-10T09:06:33.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.843+0000 7f1475e27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1ae5a0 con 0x5575db76c780 2024-09-10T09:06:33.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.843+0000 7f1475e27740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1ad070 con 0x5575db75d5d0 2024-09-10T09:06:33.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.843+0000 7f1470fe9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db75eb40 0x5575db784190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.843+0000 7f1470fe9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db75eb40 0x5575db784190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53508/0 (socket says 172.21.15.172:53508) 2024-09-10T09:06:33.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.843+0000 7f1470fe9640 1 -- 172.21.15.172:0/1216870736 learned_addr learned my addr 172.21.15.172:0/1216870736 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:33.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.843+0000 7f1471feb640 1 --2- 172.21.15.172:0/1216870736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db76c780 0x5575db7846d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.843+0000 7f14717ea640 1 --2- 172.21.15.172:0/1216870736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75d5d0 0x5575db759e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.843+0000 7f1470fe9640 1 -- 172.21.15.172:0/1216870736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75d5d0 msgr2=0x5575db759e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:33.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.843+0000 7f1470fe9640 1 --2- 172.21.15.172:0/1216870736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75d5d0 0x5575db759e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.843+0000 7f1470fe9640 1 -- 172.21.15.172:0/1216870736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db76c780 msgr2=0x5575db7846d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:33.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.843+0000 7f1470fe9640 1 --2- 172.21.15.172:0/1216870736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db76c780 0x5575db7846d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.843+0000 7f1470fe9640 1 -- 172.21.15.172:0/1216870736 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575db767a30 con 0x5575db75eb40 2024-09-10T09:06:33.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.843+0000 7f1470fe9640 1 --2- 172.21.15.172:0/1216870736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db75eb40 0x5575db784190 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f145c009fd0 tx=0x7f145c09aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4decb5e227808fdb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:33.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.845+0000 7f1456ffd640 1 -- 172.21.15.172:0/1216870736 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c0b4040 con 0x5575db75eb40 2024-09-10T09:06:33.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.845+0000 7f1456ffd640 1 -- 172.21.15.172:0/1216870736 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f145c099450 con 0x5575db75eb40 2024-09-10T09:06:33.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.845+0000 7f1456ffd640 1 -- 172.21.15.172:0/1216870736 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c017b70 con 0x5575db75eb40 2024-09-10T09:06:33.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.846+0000 7f1475e27740 1 -- 172.21.15.172:0/1216870736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db75eb40 msgr2=0x5575db784190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:33.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.846+0000 7f1475e27740 1 --2- 172.21.15.172:0/1216870736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db75eb40 0x5575db784190 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f145c009fd0 tx=0x7f145c09aef0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.846+0000 7f1475e27740 1 -- 172.21.15.172:0/1216870736 shutdown_connections 2024-09-10T09:06:33.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.846+0000 7f1475e27740 1 --2- 172.21.15.172:0/1216870736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db76c780 0x5575db7846d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.846+0000 7f1475e27740 1 --2- 172.21.15.172:0/1216870736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db75eb40 0x5575db784190 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.846+0000 7f1475e27740 1 --2- 172.21.15.172:0/1216870736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75d5d0 0x5575db759e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.846+0000 7f1475e27740 1 -- 172.21.15.172:0/1216870736 >> 172.21.15.172:0/1216870736 conn(0x5575db726310 msgr2=0x5575db759700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:33.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.846+0000 7f1475e27740 1 -- 172.21.15.172:0/1216870736 shutdown_connections 2024-09-10T09:06:33.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.846+0000 7f1475e27740 1 -- 172.21.15.172:0/1216870736 wait complete. 2024-09-10T09:06:33.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.846+0000 7f1475e27740 1 -- start start 2024-09-10T09:06:33.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.846+0000 7f1475e27740 1 Processor -- start 2024-09-10T09:06:33.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.847+0000 7f1475e27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db75d5d0 0x5575db770860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.847+0000 7f1475e27740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75eb40 0x5575db770da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.847+0000 7f1475e27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db76c780 0x5575db7712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.847+0000 7f1475e27740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1adea0 con 0x5575db76c780 2024-09-10T09:06:33.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.847+0000 7f1475e27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1ad770 con 0x5575db75d5d0 2024-09-10T09:06:33.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.847+0000 7f1475e27740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1aa920 con 0x5575db75eb40 2024-09-10T09:06:33.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.847+0000 7f1475e27740 10 client.0 _ll_register_callbacks cb 0x7f1464041050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:06:33.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.847+0000 7f1471feb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db76c780 0x5575db7712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.847+0000 7f1471feb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db76c780 0x5575db7712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53510/0 (socket says 172.21.15.172:53510) 2024-09-10T09:06:33.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.847+0000 7f1471feb640 1 -- 172.21.15.172:0/3746369663 learned_addr learned my addr 172.21.15.172:0/3746369663 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:33.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.847+0000 7f14717ea640 1 --2- 172.21.15.172:0/3746369663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db75d5d0 0x5575db770860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.847+0000 7f1470fe9640 1 --2- 172.21.15.172:0/3746369663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75eb40 0x5575db770da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.847+0000 7f1471feb640 1 -- 172.21.15.172:0/3746369663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75eb40 msgr2=0x5575db770da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:33.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.847+0000 7f1471feb640 1 --2- 172.21.15.172:0/3746369663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75eb40 0x5575db770da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.847+0000 7f1471feb640 1 -- 172.21.15.172:0/3746369663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db75d5d0 msgr2=0x5575db770860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:33.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.847+0000 7f1471feb640 1 --2- 172.21.15.172:0/3746369663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db75d5d0 0x5575db770860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.847+0000 7f1471feb640 1 -- 172.21.15.172:0/3746369663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575db7e63b0 con 0x5575db76c780 2024-09-10T09:06:33.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.848+0000 7f1471feb640 1 --2- 172.21.15.172:0/3746369663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db76c780 0x5575db7712e0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f1464048580 tx=0x7f14640486a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469529e2bb8e51f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:33.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.848+0000 7f1454ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:06:33.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.848+0000 7f1454ff9640 1 -- 172.21.15.172:0/3746369663 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146404c070 con 0x5575db76c780 2024-09-10T09:06:33.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.848+0000 7f1454ff9640 1 -- 172.21.15.172:0/3746369663 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f1464004620 con 0x5575db76c780 2024-09-10T09:06:33.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.848+0000 7f1475e27740 1 -- 172.21.15.172:0/3746369663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f145c099450 con 0x5575db76c780 2024-09-10T09:06:33.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.848+0000 7f1454ff9640 1 -- 172.21.15.172:0/3746369663 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f1464066050 con 0x5575db76c780 2024-09-10T09:06:33.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.848+0000 7f1475e27740 10 client.20334 fetch_fsmap learned FSMap version 49 2024-09-10T09:06:33.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.848+0000 7f1475e27740 1 -- 172.21.15.172:0/3746369663 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5575db7e76b0 con 0x5575db76c780 2024-09-10T09:06:33.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.848+0000 7f1454ff9640 1 -- 172.21.15.172:0/3746369663 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f145c099450 con 0x5575db76c780 2024-09-10T09:06:33.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.848+0000 7f1454ff9640 1 -- 172.21.15.172:0/3746369663 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 49) ==== 8398+0+0 (secure 0 0 0) 0x7f1464030040 con 0x5575db76c780 2024-09-10T09:06:33.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.848+0000 7f1475e27740 10 client.20334 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T09:06:33.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.848+0000 7f1475e27740 10 client.20334 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-09-10T09:06:33.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.848+0000 7f1475e27740 20 client.20334 populate_metadata read hostname 'smithi172' 2024-09-10T09:06:33.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.848+0000 7f1475e27740 1 --2- 172.21.15.172:0/3746369663 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5575db7e7e30 0x5575db7ea250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:33.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.848+0000 7f1475e27740 4 client.20334 mds_command: new command op to 24406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:06:33.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.848+0000 7f1475e27740 1 -- 172.21.15.172:0/3746369663 --> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575db6cad70 con 0x5575db7e7e30 2024-09-10T09:06:33.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.849+0000 7f14717ea640 1 --2- 172.21.15.172:0/3746369663 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5575db7e7e30 0x5575db7ea250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:33.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.849+0000 7f14717ea640 1 --2- 172.21.15.172:0/3746369663 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5575db7e7e30 0x5575db7ea250 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e46201ee2b551532 server_cookie=f3af850b41d523ec in_seq=0 out_seq=0 2024-09-10T09:06:33.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.849+0000 7f1454ff9640 10 client.20334 ms_handle_connect on v2:172.21.15.191:6832/1662718523 2024-09-10T09:06:33.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.849+0000 7f1454ff9640 1 -- 172.21.15.172:0/3746369663 <== mds.4 v2:172.21.15.191:6832/1662718523 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575db6cad70 con 0x5575db7e7e30 2024-09-10T09:06:33.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.849+0000 7f1454ff9640 10 client.20334 handle_command_reply: tid=0 2024-09-10T09:06:33.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.849+0000 7f1475e27740 1 client.20334 shutdown 2024-09-10T09:06:33.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.849+0000 7f1475e27740 1 -- 172.21.15.172:0/3746369663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db76c780 msgr2=0x5575db7712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:33.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.849+0000 7f1475e27740 1 --2- 172.21.15.172:0/3746369663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db76c780 0x5575db7712e0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f1464048580 tx=0x7f14640486a0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.849+0000 7f1475e27740 1 -- 172.21.15.172:0/3746369663 shutdown_connections 2024-09-10T09:06:33.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.849+0000 7f1475e27740 1 --2- 172.21.15.172:0/3746369663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575db76c780 0x5575db7712e0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.849+0000 7f1475e27740 1 --2- 172.21.15.172:0/3746369663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5575db75eb40 0x5575db770da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.849+0000 7f1475e27740 1 --2- 172.21.15.172:0/3746369663 >> [v2:172.21.15.191:6832/1662718523,v1:172.21.15.191:6833/1662718523] conn(0x5575db7e7e30 0x5575db7ea250 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.849+0000 7f1454ff9640 0 client.20334 ms_handle_reset on v2:172.21.15.191:6832/1662718523 2024-09-10T09:06:33.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.849+0000 7f1475e27740 1 --2- 172.21.15.172:0/3746369663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575db75d5d0 0x5575db770860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.849+0000 7f1475e27740 1 -- 172.21.15.172:0/3746369663 >> 172.21.15.172:0/3746369663 conn(0x5575db726310 msgr2=0x5575db76ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:33.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.850+0000 7f1475e27740 1 -- 172.21.15.172:0/3746369663 shutdown_connections 2024-09-10T09:06:33.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.850+0000 7f1475e27740 1 -- 172.21.15.172:0/3746369663 wait complete. 2024-09-10T09:06:33.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.850+0000 7f1475e27740 20 client.20334 trim_cache size 0 max 16384 2024-09-10T09:06:33.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.850+0000 7f1473a4e640 1 -- 172.21.15.172:0/615661970 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f14580a2c00 msgr2=0x7f14580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:33.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.850+0000 7f1473a4e640 1 --2- 172.21.15.172:0/615661970 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f14580a2c00 0x7f14580a50c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1464009ca0 tx=0x7f1464031570 comp rx=0 tx=0).stop 2024-09-10T09:06:33.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.850+0000 7f1473a4e640 1 -- 172.21.15.172:0/615661970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f146c103740 msgr2=0x7f146c198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:33.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.850+0000 7f1473a4e640 1 --2- 172.21.15.172:0/615661970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f146c103740 0x7f146c198a30 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f145c00b450 tx=0x7f145c00b910 comp rx=0 tx=0).stop 2024-09-10T09:06:33.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.850+0000 7f1473a4e640 1 -- 172.21.15.172:0/615661970 shutdown_connections 2024-09-10T09:06:33.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.850+0000 7f1473a4e640 1 --2- 172.21.15.172:0/615661970 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f14580a2c00 0x7f14580a50c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.850+0000 7f1473a4e640 1 --2- 172.21.15.172:0/615661970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f146c108290 0x7f146c19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.850+0000 7f1473a4e640 1 --2- 172.21.15.172:0/615661970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f146c103740 0x7f146c198a30 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.850+0000 7f1473a4e640 1 --2- 172.21.15.172:0/615661970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f146c102d90 0x7f146c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:33.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.850+0000 7f1473a4e640 1 -- 172.21.15.172:0/615661970 >> 172.21.15.172:0/615661970 conn(0x7f146c0fc7d0 msgr2=0x7f146c105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:33.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.850+0000 7f1473a4e640 1 -- 172.21.15.172:0/615661970 shutdown_connections 2024-09-10T09:06:33.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:33.850+0000 7f1473a4e640 1 -- 172.21.15.172:0/615661970 wait complete. 2024-09-10T09:06:34.590 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:06:34.842 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:06:35.120 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:34 smithi172 ceph-mon[30854]: pgmap v960: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:06:35.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:34 smithi174 ceph-mon[34068]: pgmap v960: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:06:35.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:34 smithi191 ceph-mon[35314]: pgmap v960: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:06:35.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.346+0000 7fec78828640 1 -- 172.21.15.172:0/1120218245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec74102790 msgr2=0x7fec741067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:35.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.346+0000 7fec78828640 1 --2- 172.21.15.172:0/1120218245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec74102790 0x7fec741067e0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fec68009900 tx=0x7fec6802a410 comp rx=0 tx=0).stop 2024-09-10T09:06:35.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.346+0000 7fec78828640 1 -- 172.21.15.172:0/1120218245 shutdown_connections 2024-09-10T09:06:35.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.346+0000 7fec78828640 1 --2- 172.21.15.172:0/1120218245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec74106f10 0x7fec7410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.346+0000 7fec78828640 1 --2- 172.21.15.172:0/1120218245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec74102790 0x7fec741067e0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.346+0000 7fec78828640 1 --2- 172.21.15.172:0/1120218245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec74101de0 0x7fec741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.346+0000 7fec78828640 1 -- 172.21.15.172:0/1120218245 >> 172.21.15.172:0/1120218245 conn(0x7fec740fd8c0 msgr2=0x7fec740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:35.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.346+0000 7fec78828640 1 -- 172.21.15.172:0/1120218245 shutdown_connections 2024-09-10T09:06:35.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.346+0000 7fec78828640 1 -- 172.21.15.172:0/1120218245 wait complete. 2024-09-10T09:06:35.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.347+0000 7fec78828640 1 Processor -- start 2024-09-10T09:06:35.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.347+0000 7fec78828640 1 -- start start 2024-09-10T09:06:35.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.347+0000 7fec78828640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec74101de0 0x7fec741983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.347+0000 7fec78828640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec74102790 0x7fec74198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.347+0000 7fec78828640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec74106f10 0x7fec7419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.347+0000 7fec78828640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fec74111e60 con 0x7fec74101de0 2024-09-10T09:06:35.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.347+0000 7fec78828640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fec74111ce0 con 0x7fec74102790 2024-09-10T09:06:35.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.347+0000 7fec78828640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fec74111fe0 con 0x7fec74106f10 2024-09-10T09:06:35.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.347+0000 7fec7259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec74101de0 0x7fec741983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.347+0000 7fec7259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec74101de0 0x7fec741983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53516/0 (socket says 172.21.15.172:53516) 2024-09-10T09:06:35.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.347+0000 7fec7259c640 1 -- 172.21.15.172:0/772420182 learned_addr learned my addr 172.21.15.172:0/772420182 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:35.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.348+0000 7fec72d9d640 1 --2- 172.21.15.172:0/772420182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec74106f10 0x7fec7419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.348+0000 7fec71d9b640 1 --2- 172.21.15.172:0/772420182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec74102790 0x7fec74198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.348+0000 7fec7259c640 1 -- 172.21.15.172:0/772420182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec74106f10 msgr2=0x7fec7419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:35.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.348+0000 7fec7259c640 1 --2- 172.21.15.172:0/772420182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec74106f10 0x7fec7419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.348+0000 7fec7259c640 1 -- 172.21.15.172:0/772420182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec74102790 msgr2=0x7fec74198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:35.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.348+0000 7fec7259c640 1 --2- 172.21.15.172:0/772420182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec74102790 0x7fec74198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.348+0000 7fec7259c640 1 -- 172.21.15.172:0/772420182 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7419d390 con 0x7fec74101de0 2024-09-10T09:06:35.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.348+0000 7fec71d9b640 1 --2- 172.21.15.172:0/772420182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec74102790 0x7fec74198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:06:35.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.348+0000 7fec72d9d640 1 --2- 172.21.15.172:0/772420182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec74106f10 0x7fec7419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:35.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.348+0000 7fec7259c640 1 --2- 172.21.15.172:0/772420182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec74101de0 0x7fec741983e0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fec6400b4a0 tx=0x7fec6400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:35.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.348+0000 7fec637fe640 1 -- 172.21.15.172:0/772420182 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64014070 con 0x7fec74101de0 2024-09-10T09:06:35.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.348+0000 7fec637fe640 1 -- 172.21.15.172:0/772420182 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec64010ac0 con 0x7fec74101de0 2024-09-10T09:06:35.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.348+0000 7fec78828640 1 -- 172.21.15.172:0/772420182 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7419d680 con 0x7fec74101de0 2024-09-10T09:06:35.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.348+0000 7fec637fe640 1 -- 172.21.15.172:0/772420182 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64017c30 con 0x7fec74101de0 2024-09-10T09:06:35.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.348+0000 7fec78828640 1 -- 172.21.15.172:0/772420182 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec741a4f60 con 0x7fec74101de0 2024-09-10T09:06:35.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.351+0000 7fec637fe640 1 -- 172.21.15.172:0/772420182 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fec64017070 con 0x7fec74101de0 2024-09-10T09:06:35.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.351+0000 7fec7ac01740 1 Processor -- start 2024-09-10T09:06:35.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.351+0000 7fec7ac01740 1 -- start start 2024-09-10T09:06:35.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.351+0000 7fec7ac01740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ad4b40 0x55d5e9ad3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.351+0000 7fec7ac01740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9ad3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.351+0000 7fec7ac01740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ade370 0x55d5e9ae0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.351+0000 7fec7ac01740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d5e9a392f0 con 0x55d5e9ad35d0 2024-09-10T09:06:35.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.351+0000 7fec7ac01740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d5e99e0cf0 con 0x55d5e9ad4b40 2024-09-10T09:06:35.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.351+0000 7fec7ac01740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d5e9a352b0 con 0x55d5e9ade370 2024-09-10T09:06:35.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.351+0000 7fec7259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9ad3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.351+0000 7fec637fe640 1 --2- 172.21.15.172:0/772420182 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fec5c0a2b80 0x7fec5c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.351+0000 7fec7259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9ad3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53520/0 (socket says 172.21.15.172:53520) 2024-09-10T09:06:35.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.352+0000 7fec7259c640 1 -- 172.21.15.172:0/2520152182 learned_addr learned my addr 172.21.15.172:0/2520152182 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:35.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.352+0000 7fec72d9d640 1 --2- 172.21.15.172:0/2520152182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ad4b40 0x55d5e9ad3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.352+0000 7fec72d9d640 1 --2- 172.21.15.172:0/772420182 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fec5c0a2b80 0x7fec5c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.352+0000 7fec71d9b640 1 --2- 172.21.15.172:0/2520152182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ade370 0x55d5e9ae0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.352+0000 7fec637fe640 1 -- 172.21.15.172:0/772420182 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fec640ceb20 con 0x7fec74101de0 2024-09-10T09:06:35.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.352+0000 7fec7259c640 1 -- 172.21.15.172:0/2520152182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ade370 msgr2=0x55d5e9ae0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:35.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.352+0000 7fec7259c640 1 --2- 172.21.15.172:0/2520152182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ade370 0x55d5e9ae0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.352+0000 7fec7259c640 1 -- 172.21.15.172:0/2520152182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ad4b40 msgr2=0x55d5e9ad3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:35.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.352+0000 7fec7259c640 1 --2- 172.21.15.172:0/2520152182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ad4b40 0x55d5e9ad3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.352+0000 7fec7259c640 1 -- 172.21.15.172:0/2520152182 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e99c53a0 con 0x55d5e9ad35d0 2024-09-10T09:06:35.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.352+0000 7fec72d9d640 1 --2- 172.21.15.172:0/772420182 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fec5c0a2b80 0x7fec5c0a5040 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009fd0 tx=0x7fec6c009a70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:35.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.352+0000 7fec7259c640 1 --2- 172.21.15.172:0/2520152182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9ad3a10 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fec640cc2c0 tx=0x7fec640047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e587017e6ecde7a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:35.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.352+0000 7fec60ff9640 1 -- 172.21.15.172:0/2520152182 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6401e070 con 0x55d5e9ad35d0 2024-09-10T09:06:35.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.352+0000 7fec60ff9640 1 -- 172.21.15.172:0/2520152182 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec640bb0e0 con 0x55d5e9ad35d0 2024-09-10T09:06:35.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.352+0000 7fec60ff9640 1 -- 172.21.15.172:0/2520152182 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64099320 con 0x55d5e9ad35d0 2024-09-10T09:06:35.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.353+0000 7fec7ac01740 1 -- 172.21.15.172:0/2520152182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 msgr2=0x55d5e9ad3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:35.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.353+0000 7fec7ac01740 1 --2- 172.21.15.172:0/2520152182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9ad3a10 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fec640cc2c0 tx=0x7fec640047d0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.353+0000 7fec7ac01740 1 -- 172.21.15.172:0/2520152182 shutdown_connections 2024-09-10T09:06:35.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.353+0000 7fec7ac01740 1 --2- 172.21.15.172:0/2520152182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ade370 0x55d5e9ae0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.353+0000 7fec7ac01740 1 --2- 172.21.15.172:0/2520152182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9ad3a10 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.353+0000 7fec7ac01740 1 --2- 172.21.15.172:0/2520152182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ad4b40 0x55d5e9ad3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.353+0000 7fec7ac01740 1 -- 172.21.15.172:0/2520152182 >> 172.21.15.172:0/2520152182 conn(0x55d5e9a9c310 msgr2=0x55d5e9acfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:35.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.353+0000 7fec7ac01740 1 -- 172.21.15.172:0/2520152182 shutdown_connections 2024-09-10T09:06:35.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.353+0000 7fec7ac01740 1 -- 172.21.15.172:0/2520152182 wait complete. 2024-09-10T09:06:35.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.353+0000 7fec7ac01740 1 -- start start 2024-09-10T09:06:35.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec7ac01740 1 Processor -- start 2024-09-10T09:06:35.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec7ac01740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ad35d0 0x55d5e9addda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec7ac01740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad4b40 0x55d5e9adb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec7ac01740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ade370 0x55d5e9adb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec7ac01740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d5e9ae1070 con 0x55d5e9ad4b40 2024-09-10T09:06:35.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec7ac01740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d5e99f4060 con 0x55d5e9ad35d0 2024-09-10T09:06:35.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec7ac01740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d5e9ae11f0 con 0x55d5e9ade370 2024-09-10T09:06:35.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec7ac01740 10 client.0 _ll_register_callbacks cb 0x55d5e97478c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:06:35.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec71d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad4b40 0x55d5e9adb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec71d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad4b40 0x55d5e9adb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53524/0 (socket says 172.21.15.172:53524) 2024-09-10T09:06:35.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec71d9b640 1 -- 172.21.15.172:0/2515538225 learned_addr learned my addr 172.21.15.172:0/2515538225 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:35.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec72d9d640 1 --2- 172.21.15.172:0/2515538225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ade370 0x55d5e9adb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec7259c640 1 --2- 172.21.15.172:0/2515538225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ad35d0 0x55d5e9addda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec71d9b640 1 -- 172.21.15.172:0/2515538225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ade370 msgr2=0x55d5e9adb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:35.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec71d9b640 1 --2- 172.21.15.172:0/2515538225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ade370 0x55d5e9adb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec71d9b640 1 -- 172.21.15.172:0/2515538225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ad35d0 msgr2=0x55d5e9addda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:35.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec71d9b640 1 --2- 172.21.15.172:0/2515538225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ad35d0 0x55d5e9addda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec71d9b640 1 -- 172.21.15.172:0/2515538225 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e97aa3c0 con 0x55d5e9ad4b40 2024-09-10T09:06:35.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec7259c640 1 --2- 172.21.15.172:0/2515538225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ad35d0 0x55d5e9addda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:35.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec71d9b640 1 --2- 172.21.15.172:0/2515538225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad4b40 0x55d5e9adb050 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fec68009230 tx=0x7fec680316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435f8b255dd3f6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:35.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.354+0000 7fec59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:06:35.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.355+0000 7fec59ffb640 1 -- 172.21.15.172:0/2515538225 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68040070 con 0x55d5e9ad4b40 2024-09-10T09:06:35.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.355+0000 7fec59ffb640 1 -- 172.21.15.172:0/2515538225 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fec68004450 con 0x55d5e9ad4b40 2024-09-10T09:06:35.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.355+0000 7fec7ac01740 1 -- 172.21.15.172:0/2515538225 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec640bb0e0 con 0x55d5e9ad4b40 2024-09-10T09:06:35.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.355+0000 7fec59ffb640 1 -- 172.21.15.172:0/2515538225 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fec6804d050 con 0x55d5e9ad4b40 2024-09-10T09:06:35.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.355+0000 7fec7ac01740 10 client.20346 fetch_fsmap learned FSMap version 49 2024-09-10T09:06:35.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.355+0000 7fec7ac01740 1 -- 172.21.15.172:0/2515538225 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55d5e9aecb50 con 0x55d5e9ad4b40 2024-09-10T09:06:35.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.355+0000 7fec59ffb640 1 -- 172.21.15.172:0/2515538225 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec640bb0e0 con 0x55d5e9ad4b40 2024-09-10T09:06:35.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.355+0000 7fec59ffb640 1 -- 172.21.15.172:0/2515538225 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 49) ==== 8398+0+0 (secure 0 0 0) 0x7fec68004920 con 0x55d5e9ad4b40 2024-09-10T09:06:35.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.355+0000 7fec7ac01740 10 client.20346 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T09:06:35.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.355+0000 7fec7ac01740 10 client.20346 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T09:06:35.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.355+0000 7fec7ac01740 20 client.20346 populate_metadata read hostname 'smithi172' 2024-09-10T09:06:35.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.355+0000 7fec7ac01740 1 --2- 172.21.15.172:0/2515538225 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55d5e9aee010 0x55d5e9af04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.355+0000 7fec7ac01740 4 client.20346 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:06:35.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.355+0000 7fec7ac01740 1 -- 172.21.15.172:0/2515538225 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5e9a46860 con 0x55d5e9aee010 2024-09-10T09:06:35.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.356+0000 7fec7259c640 1 --2- 172.21.15.172:0/2515538225 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55d5e9aee010 0x55d5e9af04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.356+0000 7fec7259c640 1 --2- 172.21.15.172:0/2515538225 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55d5e9aee010 0x55d5e9af04b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=473f96fca2a62a2b server_cookie=3d375884044be43f in_seq=0 out_seq=0 2024-09-10T09:06:35.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.356+0000 7fec59ffb640 10 client.20346 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T09:06:35.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.357+0000 7fec59ffb640 1 -- 172.21.15.172:0/2515538225 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5e9a46860 con 0x55d5e9aee010 2024-09-10T09:06:35.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.357+0000 7fec59ffb640 10 client.20346 handle_command_reply: tid=0 2024-09-10T09:06:35.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.357+0000 7fec7ac01740 1 client.20346 shutdown 2024-09-10T09:06:35.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.357+0000 7fec7ac01740 1 -- 172.21.15.172:0/2515538225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad4b40 msgr2=0x55d5e9adb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:35.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.357+0000 7fec7ac01740 1 --2- 172.21.15.172:0/2515538225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad4b40 0x55d5e9adb050 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fec68009230 tx=0x7fec680316d0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.357+0000 7fec7ac01740 1 -- 172.21.15.172:0/2515538225 shutdown_connections 2024-09-10T09:06:35.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.357+0000 7fec7ac01740 1 --2- 172.21.15.172:0/2515538225 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55d5e9aee010 0x55d5e9af04b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.357+0000 7fec7ac01740 1 --2- 172.21.15.172:0/2515538225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ade370 0x55d5e9adb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.357+0000 7fec7ac01740 1 --2- 172.21.15.172:0/2515538225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad4b40 0x55d5e9adb050 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.357+0000 7fec7ac01740 1 --2- 172.21.15.172:0/2515538225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ad35d0 0x55d5e9addda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.357+0000 7fec59ffb640 0 client.20346 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T09:06:35.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.357+0000 7fec7ac01740 1 -- 172.21.15.172:0/2515538225 >> 172.21.15.172:0/2515538225 conn(0x55d5e9a9c310 msgr2=0x55d5e9ad5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:35.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.357+0000 7fec7ac01740 1 -- 172.21.15.172:0/2515538225 shutdown_connections 2024-09-10T09:06:35.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.357+0000 7fec7ac01740 1 -- 172.21.15.172:0/2515538225 wait complete. 2024-09-10T09:06:35.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.357+0000 7fec7ac01740 20 client.20346 trim_cache size 0 max 16384 2024-09-10T09:06:35.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.370+0000 7fec7ac01740 1 Processor -- start 2024-09-10T09:06:35.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.370+0000 7fec7ac01740 1 -- start start 2024-09-10T09:06:35.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.370+0000 7fec7ac01740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9acfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.370+0000 7fec7ac01740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ad4b40 0x55d5e9af5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.370+0000 7fec7ac01740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ade370 0x55d5e9af6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.370+0000 7fec7ac01740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fec741aece0 con 0x55d5e9ad35d0 2024-09-10T09:06:35.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.370+0000 7fec7ac01740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fec741b08e0 con 0x55d5e9ad4b40 2024-09-10T09:06:35.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.370+0000 7fec7ac01740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fec741af3b0 con 0x55d5e9ade370 2024-09-10T09:06:35.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.371+0000 7fec7259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9acfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.371+0000 7fec7259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9acfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53536/0 (socket says 172.21.15.172:53536) 2024-09-10T09:06:35.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.371+0000 7fec7259c640 1 -- 172.21.15.172:0/3566899819 learned_addr learned my addr 172.21.15.172:0/3566899819 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:35.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.371+0000 7fec71d9b640 1 --2- 172.21.15.172:0/3566899819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ad4b40 0x55d5e9af5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.371+0000 7fec7259c640 1 -- 172.21.15.172:0/3566899819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ade370 msgr2=0x55d5e9af6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:35.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.371+0000 7fec72d9d640 1 --2- 172.21.15.172:0/3566899819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ade370 0x55d5e9af6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.371+0000 7fec7259c640 1 --2- 172.21.15.172:0/3566899819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ade370 0x55d5e9af6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.371+0000 7fec7259c640 1 -- 172.21.15.172:0/3566899819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ad4b40 msgr2=0x55d5e9af5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:35.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.371+0000 7fec7259c640 1 --2- 172.21.15.172:0/3566899819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ad4b40 0x55d5e9af5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.371+0000 7fec7259c640 1 -- 172.21.15.172:0/3566899819 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e9aea7f0 con 0x55d5e9ad35d0 2024-09-10T09:06:35.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.371+0000 7fec71d9b640 1 --2- 172.21.15.172:0/3566899819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ad4b40 0x55d5e9af5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:06:35.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.371+0000 7fec7259c640 1 --2- 172.21.15.172:0/3566899819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9acfe10 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fec64096d00 tx=0x7fec6400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd3699ca228a327 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:35.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.371+0000 7fec5bfff640 1 -- 172.21.15.172:0/3566899819 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6401e040 con 0x55d5e9ad35d0 2024-09-10T09:06:35.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.371+0000 7fec5bfff640 1 -- 172.21.15.172:0/3566899819 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec6409f3d0 con 0x55d5e9ad35d0 2024-09-10T09:06:35.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.371+0000 7fec5bfff640 1 -- 172.21.15.172:0/3566899819 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64010c70 con 0x55d5e9ad35d0 2024-09-10T09:06:35.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.372+0000 7fec7ac01740 1 -- 172.21.15.172:0/3566899819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 msgr2=0x55d5e9acfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:35.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.372+0000 7fec7ac01740 1 --2- 172.21.15.172:0/3566899819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9acfe10 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fec64096d00 tx=0x7fec6400bf90 comp rx=0 tx=0).stop 2024-09-10T09:06:35.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.372+0000 7fec7ac01740 1 -- 172.21.15.172:0/3566899819 shutdown_connections 2024-09-10T09:06:35.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.372+0000 7fec7ac01740 1 --2- 172.21.15.172:0/3566899819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ade370 0x55d5e9af6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.372+0000 7fec7ac01740 1 --2- 172.21.15.172:0/3566899819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ad4b40 0x55d5e9af5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.372+0000 7fec7ac01740 1 --2- 172.21.15.172:0/3566899819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9acfe10 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.372+0000 7fec7ac01740 1 -- 172.21.15.172:0/3566899819 >> 172.21.15.172:0/3566899819 conn(0x55d5e9a9c310 msgr2=0x55d5e9acf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:35.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.372+0000 7fec7ac01740 1 -- 172.21.15.172:0/3566899819 shutdown_connections 2024-09-10T09:06:35.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.372+0000 7fec7ac01740 1 -- 172.21.15.172:0/3566899819 wait complete. 2024-09-10T09:06:35.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec7ac01740 1 -- start start 2024-09-10T09:06:35.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec7ac01740 1 Processor -- start 2024-09-10T09:06:35.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec7ac01740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9ae22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec7ac01740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ad4b40 0x55d5e9ae2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec7ac01740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ade370 0x55d5e9ae2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec7ac01740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fec741b01e0 con 0x55d5e9ad35d0 2024-09-10T09:06:35.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec7ac01740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fec741afab0 con 0x55d5e9ade370 2024-09-10T09:06:35.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec7ac01740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fec741acc60 con 0x55d5e9ad4b40 2024-09-10T09:06:35.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec7ac01740 10 client.0 _ll_register_callbacks cb 0x7fec68005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:06:35.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec7259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9ae22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec7259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9ae22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53550/0 (socket says 172.21.15.172:53550) 2024-09-10T09:06:35.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec7259c640 1 -- 172.21.15.172:0/1509869692 learned_addr learned my addr 172.21.15.172:0/1509869692 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:35.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec72d9d640 1 --2- 172.21.15.172:0/1509869692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ade370 0x55d5e9ae2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec71d9b640 1 --2- 172.21.15.172:0/1509869692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ad4b40 0x55d5e9ae2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec7259c640 1 -- 172.21.15.172:0/1509869692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ad4b40 msgr2=0x55d5e9ae2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:35.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec7259c640 1 --2- 172.21.15.172:0/1509869692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ad4b40 0x55d5e9ae2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec7259c640 1 -- 172.21.15.172:0/1509869692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ade370 msgr2=0x55d5e9ae2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:35.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec7259c640 1 --2- 172.21.15.172:0/1509869692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ade370 0x55d5e9ae2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.373+0000 7fec7259c640 1 -- 172.21.15.172:0/1509869692 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e9b5c2a0 con 0x55d5e9ad35d0 2024-09-10T09:06:35.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.374+0000 7fec71d9b640 1 --2- 172.21.15.172:0/1509869692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ad4b40 0x55d5e9ae2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:06:35.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.374+0000 7fec7259c640 1 --2- 172.21.15.172:0/1509869692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9ae22e0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fec6400be60 tx=0x7fec640993f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c8b985f8f7f95d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:35.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.374+0000 7fec59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:06:35.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.374+0000 7fec59ffb640 1 -- 172.21.15.172:0/1509869692 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6401e070 con 0x55d5e9ad35d0 2024-09-10T09:06:35.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.374+0000 7fec59ffb640 1 -- 172.21.15.172:0/1509869692 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fec64097690 con 0x55d5e9ad35d0 2024-09-10T09:06:35.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.374+0000 7fec7ac01740 1 -- 172.21.15.172:0/1509869692 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec6409f3d0 con 0x55d5e9ad35d0 2024-09-10T09:06:35.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.374+0000 7fec59ffb640 1 -- 172.21.15.172:0/1509869692 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fec640bb0b0 con 0x55d5e9ad35d0 2024-09-10T09:06:35.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.374+0000 7fec7ac01740 10 client.20352 fetch_fsmap learned FSMap version 49 2024-09-10T09:06:35.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.374+0000 7fec7ac01740 1 -- 172.21.15.172:0/1509869692 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55d5e9b5d4f0 con 0x55d5e9ad35d0 2024-09-10T09:06:35.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.374+0000 7fec59ffb640 1 -- 172.21.15.172:0/1509869692 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec6409f3d0 con 0x55d5e9ad35d0 2024-09-10T09:06:35.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.374+0000 7fec59ffb640 1 -- 172.21.15.172:0/1509869692 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 49) ==== 8398+0+0 (secure 0 0 0) 0x7fec64095b90 con 0x55d5e9ad35d0 2024-09-10T09:06:35.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.374+0000 7fec7ac01740 10 client.20352 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T09:06:35.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.374+0000 7fec7ac01740 10 client.20352 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-09-10T09:06:35.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.374+0000 7fec7ac01740 20 client.20352 populate_metadata read hostname 'smithi172' 2024-09-10T09:06:35.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.374+0000 7fec7ac01740 1 --2- 172.21.15.172:0/1509869692 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55d5e9b5d730 0x55d5e9b5fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:35.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.374+0000 7fec7ac01740 4 client.20352 mds_command: new command op to 24416 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:06:35.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.374+0000 7fec7ac01740 1 -- 172.21.15.172:0/1509869692 --> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d5e9a40d70 con 0x55d5e9b5d730 2024-09-10T09:06:35.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.375+0000 7fec71d9b640 1 --2- 172.21.15.172:0/1509869692 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55d5e9b5d730 0x55d5e9b5fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:35.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.375+0000 7fec71d9b640 1 --2- 172.21.15.172:0/1509869692 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55d5e9b5d730 0x55d5e9b5fb20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3e98b2f4bbb7dd54 server_cookie=28b7beeacbcfdc59 in_seq=0 out_seq=0 2024-09-10T09:06:35.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.375+0000 7fec59ffb640 10 client.20352 ms_handle_connect on v2:172.21.15.174:6832/3418766024 2024-09-10T09:06:35.385 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.375+0000 7fec59ffb640 1 -- 172.21.15.172:0/1509869692 <== mds.3 v2:172.21.15.174:6832/3418766024 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d5e9a40d70 con 0x55d5e9b5d730 2024-09-10T09:06:35.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.375+0000 7fec59ffb640 10 client.20352 handle_command_reply: tid=0 2024-09-10T09:06:35.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.375+0000 7fec7ac01740 1 client.20352 shutdown 2024-09-10T09:06:35.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.375+0000 7fec7ac01740 1 -- 172.21.15.172:0/1509869692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 msgr2=0x55d5e9ae22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:35.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.375+0000 7fec7ac01740 1 --2- 172.21.15.172:0/1509869692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9ae22e0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fec6400be60 tx=0x7fec640993f0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.375+0000 7fec7ac01740 1 -- 172.21.15.172:0/1509869692 shutdown_connections 2024-09-10T09:06:35.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.375+0000 7fec7ac01740 1 --2- 172.21.15.172:0/1509869692 >> [v2:172.21.15.174:6832/3418766024,v1:172.21.15.174:6833/3418766024] conn(0x55d5e9b5d730 0x55d5e9b5fb20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.375+0000 7fec59ffb640 0 client.20352 ms_handle_reset on v2:172.21.15.174:6832/3418766024 2024-09-10T09:06:35.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.375+0000 7fec7ac01740 1 --2- 172.21.15.172:0/1509869692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5e9ade370 0x55d5e9ae2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.375+0000 7fec7ac01740 1 --2- 172.21.15.172:0/1509869692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5e9ad4b40 0x55d5e9ae2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.376+0000 7fec7ac01740 1 --2- 172.21.15.172:0/1509869692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d5e9ad35d0 0x55d5e9ae22e0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.376+0000 7fec7ac01740 1 -- 172.21.15.172:0/1509869692 >> 172.21.15.172:0/1509869692 conn(0x55d5e9a9c310 msgr2=0x55d5e9adc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:35.386 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.376+0000 7fec7ac01740 1 -- 172.21.15.172:0/1509869692 shutdown_connections 2024-09-10T09:06:35.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.376+0000 7fec7ac01740 1 -- 172.21.15.172:0/1509869692 wait complete. 2024-09-10T09:06:35.387 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.376+0000 7fec7ac01740 20 client.20352 trim_cache size 0 max 16384 2024-09-10T09:06:35.387 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:06:35.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.376+0000 7fec78828640 1 -- 172.21.15.172:0/772420182 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fec5c0a2b80 msgr2=0x7fec5c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:35.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.376+0000 7fec78828640 1 --2- 172.21.15.172:0/772420182 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fec5c0a2b80 0x7fec5c0a5040 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009fd0 tx=0x7fec6c009a70 comp rx=0 tx=0).stop 2024-09-10T09:06:35.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.376+0000 7fec78828640 1 -- 172.21.15.172:0/772420182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec74101de0 msgr2=0x7fec741983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:35.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.376+0000 7fec78828640 1 --2- 172.21.15.172:0/772420182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec74101de0 0x7fec741983e0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fec6400b4a0 tx=0x7fec6400b960 comp rx=0 tx=0).stop 2024-09-10T09:06:35.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.376+0000 7fec78828640 1 -- 172.21.15.172:0/772420182 shutdown_connections 2024-09-10T09:06:35.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.376+0000 7fec78828640 1 --2- 172.21.15.172:0/772420182 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fec5c0a2b80 0x7fec5c0a5040 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.376+0000 7fec78828640 1 --2- 172.21.15.172:0/772420182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec74106f10 0x7fec7419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.376+0000 7fec78828640 1 --2- 172.21.15.172:0/772420182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec74102790 0x7fec74198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.376+0000 7fec78828640 1 --2- 172.21.15.172:0/772420182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec74101de0 0x7fec741983e0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:35.388 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.376+0000 7fec78828640 1 -- 172.21.15.172:0/772420182 >> 172.21.15.172:0/772420182 conn(0x7fec740fd8c0 msgr2=0x7fec740fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:35.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.376+0000 7fec78828640 1 -- 172.21.15.172:0/772420182 shutdown_connections 2024-09-10T09:06:35.389 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:35.376+0000 7fec78828640 1 -- 172.21.15.172:0/772420182 wait complete. 2024-09-10T09:06:36.072 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:06:36.319 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:06:36.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.780+0000 7f5162e7e640 1 -- 172.21.15.172:0/3276447429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f515c106a80 msgr2=0x7f515c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:36.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.780+0000 7f5162e7e640 1 --2- 172.21.15.172:0/3276447429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f515c106a80 0x7f515c106e60 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f5154009930 tx=0x7f515402a420 comp rx=0 tx=0).stop 2024-09-10T09:06:36.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.780+0000 7f5162e7e640 1 -- 172.21.15.172:0/3276447429 shutdown_connections 2024-09-10T09:06:36.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.780+0000 7f5162e7e640 1 --2- 172.21.15.172:0/3276447429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f515c101420 0x7f515c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.809 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.780+0000 7f5162e7e640 1 --2- 172.21.15.172:0/3276447429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f515c100a80 0x7f515c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.780+0000 7f5162e7e640 1 --2- 172.21.15.172:0/3276447429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f515c106a80 0x7f515c106e60 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.780+0000 7f5162e7e640 1 -- 172.21.15.172:0/3276447429 >> 172.21.15.172:0/3276447429 conn(0x7f515c0fc7a0 msgr2=0x7f515c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.780+0000 7f5162e7e640 1 -- 172.21.15.172:0/3276447429 shutdown_connections 2024-09-10T09:06:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.780+0000 7f5162e7e640 1 -- 172.21.15.172:0/3276447429 wait complete. 2024-09-10T09:06:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.780+0000 7f5162e7e640 1 Processor -- start 2024-09-10T09:06:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.780+0000 7f5162e7e640 1 -- start start 2024-09-10T09:06:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f5162e7e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f515c100a80 0x7f515c199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f5162e7e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f515c101420 0x7f515c1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f5162e7e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f515c106a80 0x7f515c1a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f5162e7e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f515c110d30 con 0x7f515c100a80 2024-09-10T09:06:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f5162e7e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f515c110bb0 con 0x7f515c106a80 2024-09-10T09:06:36.810 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f5162e7e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f515c110eb0 con 0x7f515c101420 2024-09-10T09:06:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f5160c1a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f515c100a80 0x7f515c199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f5160c1a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f515c100a80 0x7f515c199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43690/0 (socket says 172.21.15.172:43690) 2024-09-10T09:06:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f5160c1a640 1 -- 172.21.15.172:0/2003547947 learned_addr learned my addr 172.21.15.172:0/2003547947 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f515bfff640 1 --2- 172.21.15.172:0/2003547947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f515c101420 0x7f515c1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f516141b640 1 --2- 172.21.15.172:0/2003547947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f515c106a80 0x7f515c1a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f5160c1a640 1 -- 172.21.15.172:0/2003547947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f515c101420 msgr2=0x7f515c1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f5160c1a640 1 --2- 172.21.15.172:0/2003547947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f515c101420 0x7f515c1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f5160c1a640 1 -- 172.21.15.172:0/2003547947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f515c106a80 msgr2=0x7f515c1a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f5160c1a640 1 --2- 172.21.15.172:0/2003547947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f515c106a80 0x7f515c1a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f5160c1a640 1 -- 172.21.15.172:0/2003547947 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c1a4750 con 0x7f515c100a80 2024-09-10T09:06:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f516141b640 1 --2- 172.21.15.172:0/2003547947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f515c106a80 0x7f515c1a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:36.811 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f5160c1a640 1 --2- 172.21.15.172:0/2003547947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f515c100a80 0x7f515c199190 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f5154004770 tx=0x7f51540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f5159ffb640 1 -- 172.21.15.172:0/2003547947 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515403f070 con 0x7f515c100a80 2024-09-10T09:06:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.781+0000 7f5159ffb640 1 -- 172.21.15.172:0/2003547947 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f515403b460 con 0x7f515c100a80 2024-09-10T09:06:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.782+0000 7f5162e7e640 1 -- 172.21.15.172:0/2003547947 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515c1a4920 con 0x7f515c100a80 2024-09-10T09:06:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.782+0000 7f5159ffb640 1 -- 172.21.15.172:0/2003547947 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154035070 con 0x7f515c100a80 2024-09-10T09:06:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.782+0000 7f5162e7e640 1 -- 172.21.15.172:0/2003547947 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515c1a4dd0 con 0x7f515c100a80 2024-09-10T09:06:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.784+0000 7f5159ffb640 1 -- 172.21.15.172:0/2003547947 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f515403b610 con 0x7f515c100a80 2024-09-10T09:06:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.784+0000 7f5165257740 1 Processor -- start 2024-09-10T09:06:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.784+0000 7f5165257740 1 -- start start 2024-09-10T09:06:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f5165257740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b4197ab40 0x562b41979090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f5165257740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b419795d0 0x562b41979a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f5165257740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b41988780 0x562b4198ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f5165257740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562b418df2f0 con 0x562b419795d0 2024-09-10T09:06:36.812 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f5165257740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562b41886cf0 con 0x562b4197ab40 2024-09-10T09:06:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f5165257740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562b418db2b0 con 0x562b41988780 2024-09-10T09:06:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f5159ffb640 1 --2- 172.21.15.172:0/2003547947 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51480a2b80 0x7f51480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f5160c1a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b419795d0 0x562b41979a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f516141b640 1 --2- 172.21.15.172:0/2003547947 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51480a2b80 0x7f51480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f5160c1a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b419795d0 0x562b41979a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43694/0 (socket says 172.21.15.172:43694) 2024-09-10T09:06:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f5160c1a640 1 -- 172.21.15.172:0/2558924324 learned_addr learned my addr 172.21.15.172:0/2558924324 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f515bfff640 1 --2- 172.21.15.172:0/2558924324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b41988780 0x562b4198ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f516141b640 1 --2- 172.21.15.172:0/2558924324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b4197ab40 0x562b41979090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f5159ffb640 1 -- 172.21.15.172:0/2003547947 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f51540ef730 con 0x7f515c100a80 2024-09-10T09:06:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f5160c1a640 1 -- 172.21.15.172:0/2558924324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b41988780 msgr2=0x562b4198ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f5160c1a640 1 --2- 172.21.15.172:0/2558924324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b41988780 0x562b4198ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.813 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f5160c1a640 1 -- 172.21.15.172:0/2558924324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b4197ab40 msgr2=0x562b41979090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f5160c1a640 1 --2- 172.21.15.172:0/2558924324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b4197ab40 0x562b41979090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f5160c1a640 1 -- 172.21.15.172:0/2558924324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b4186b3a0 con 0x562b419795d0 2024-09-10T09:06:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.785+0000 7f516141b640 1 --2- 172.21.15.172:0/2003547947 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51480a2b80 0x7f51480a5040 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5144005d80 tx=0x7f514400c7a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.786+0000 7f5160c1a640 1 --2- 172.21.15.172:0/2558924324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b419795d0 0x562b41979a10 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f51540c4370 tx=0x7f51540bb590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5463ba22ed151447 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.786+0000 7f5152ffd640 1 -- 172.21.15.172:0/2558924324 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154048030 con 0x562b419795d0 2024-09-10T09:06:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.786+0000 7f5152ffd640 1 -- 172.21.15.172:0/2558924324 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f51540bbc30 con 0x562b419795d0 2024-09-10T09:06:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.786+0000 7f5152ffd640 1 -- 172.21.15.172:0/2558924324 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51540ef440 con 0x562b419795d0 2024-09-10T09:06:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.786+0000 7f5165257740 1 -- 172.21.15.172:0/2558924324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b419795d0 msgr2=0x562b41979a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.786+0000 7f5165257740 1 --2- 172.21.15.172:0/2558924324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b419795d0 0x562b41979a10 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f51540c4370 tx=0x7f51540bb590 comp rx=0 tx=0).stop 2024-09-10T09:06:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.786+0000 7f5165257740 1 -- 172.21.15.172:0/2558924324 shutdown_connections 2024-09-10T09:06:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.786+0000 7f5165257740 1 --2- 172.21.15.172:0/2558924324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b41988780 0x562b4198ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.814 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.786+0000 7f5165257740 1 --2- 172.21.15.172:0/2558924324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b419795d0 0x562b41979a10 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.786+0000 7f5165257740 1 --2- 172.21.15.172:0/2558924324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b4197ab40 0x562b41979090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.786+0000 7f5165257740 1 -- 172.21.15.172:0/2558924324 >> 172.21.15.172:0/2558924324 conn(0x562b41942310 msgr2=0x562b41975c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.787+0000 7f5165257740 1 -- 172.21.15.172:0/2558924324 shutdown_connections 2024-09-10T09:06:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.787+0000 7f5165257740 1 -- 172.21.15.172:0/2558924324 wait complete. 2024-09-10T09:06:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.787+0000 7f5165257740 1 -- start start 2024-09-10T09:06:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.787+0000 7f5165257740 1 Processor -- start 2024-09-10T09:06:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.787+0000 7f5165257740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b419795d0 0x562b41988130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.787+0000 7f5165257740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b4197ab40 0x562b419853e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.787+0000 7f5165257740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b41988780 0x562b41985980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.787+0000 7f5165257740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562b418df2f0 con 0x562b41988780 2024-09-10T09:06:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.787+0000 7f5165257740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562b4189a060 con 0x562b4197ab40 2024-09-10T09:06:36.815 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.787+0000 7f5165257740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562b4198b240 con 0x562b419795d0 2024-09-10T09:06:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.787+0000 7f5165257740 10 client.0 _ll_register_callbacks cb 0x562b415ed8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:06:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.787+0000 7f516141b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b41988780 0x562b41985980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.787+0000 7f516141b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b41988780 0x562b41985980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43700/0 (socket says 172.21.15.172:43700) 2024-09-10T09:06:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f516141b640 1 -- 172.21.15.172:0/1664505277 learned_addr learned my addr 172.21.15.172:0/1664505277 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f515bfff640 1 --2- 172.21.15.172:0/1664505277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b4197ab40 0x562b419853e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f516141b640 1 -- 172.21.15.172:0/1664505277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b419795d0 msgr2=0x562b41988130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f5160c1a640 1 --2- 172.21.15.172:0/1664505277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b419795d0 0x562b41988130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f516141b640 1 --2- 172.21.15.172:0/1664505277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b419795d0 0x562b41988130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f516141b640 1 -- 172.21.15.172:0/1664505277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b4197ab40 msgr2=0x562b419853e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f516141b640 1 --2- 172.21.15.172:0/1664505277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b4197ab40 0x562b419853e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f516141b640 1 -- 172.21.15.172:0/1664505277 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b416503c0 con 0x562b41988780 2024-09-10T09:06:36.816 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f5160c1a640 1 --2- 172.21.15.172:0/1664505277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b419795d0 0x562b41988130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:06:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f515bfff640 1 --2- 172.21.15.172:0/1664505277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b4197ab40 0x562b419853e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f516141b640 1 --2- 172.21.15.172:0/1664505277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b41988780 0x562b41985980 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f5144011ad0 tx=0x7f5144019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145a498295b75dc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f513bfff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:06:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f513bfff640 1 -- 172.21.15.172:0/1664505277 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144027440 con 0x562b41988780 2024-09-10T09:06:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f5165257740 1 -- 172.21.15.172:0/1664505277 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562b418ec860 con 0x562b41988780 2024-09-10T09:06:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f513bfff640 1 -- 172.21.15.172:0/1664505277 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f51440263e0 con 0x562b41988780 2024-09-10T09:06:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f513bfff640 1 -- 172.21.15.172:0/1664505277 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f51440275e0 con 0x562b41988780 2024-09-10T09:06:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f5165257740 10 client.20364 fetch_fsmap learned FSMap version 49 2024-09-10T09:06:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f5165257740 1 -- 172.21.15.172:0/1664505277 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562b41996a40 con 0x562b41988780 2024-09-10T09:06:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.788+0000 7f513bfff640 1 -- 172.21.15.172:0/1664505277 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x562b418ec860 con 0x562b41988780 2024-09-10T09:06:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.789+0000 7f513bfff640 1 -- 172.21.15.172:0/1664505277 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 49) ==== 8398+0+0 (secure 0 0 0) 0x7f5144025070 con 0x562b41988780 2024-09-10T09:06:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.789+0000 7f5165257740 10 client.20364 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T09:06:36.817 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.789+0000 7f5165257740 10 client.20364 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:06:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.789+0000 7f5165257740 20 client.20364 populate_metadata read hostname 'smithi172' 2024-09-10T09:06:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.789+0000 7f5165257740 1 --2- 172.21.15.172:0/1664505277 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x562b41998020 0x562b4199a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.789+0000 7f5165257740 4 client.20364 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:06:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.789+0000 7f5165257740 1 -- 172.21.15.172:0/1664505277 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b418e6d70 con 0x562b41998020 2024-09-10T09:06:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.789+0000 7f5160c1a640 1 --2- 172.21.15.172:0/1664505277 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x562b41998020 0x562b4199a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.789+0000 7f5160c1a640 1 --2- 172.21.15.172:0/1664505277 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x562b41998020 0x562b4199a480 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39d08980eca580e4 server_cookie=66b635d46e38fe87 in_seq=0 out_seq=0 2024-09-10T09:06:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.789+0000 7f513bfff640 10 client.20364 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T09:06:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.790+0000 7f513bfff640 1 -- 172.21.15.172:0/1664505277 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b418e6d70 con 0x562b41998020 2024-09-10T09:06:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.790+0000 7f513bfff640 10 client.20364 handle_command_reply: tid=0 2024-09-10T09:06:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.790+0000 7f5165257740 1 client.20364 shutdown 2024-09-10T09:06:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.790+0000 7f5165257740 1 -- 172.21.15.172:0/1664505277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b41988780 msgr2=0x562b41985980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.790+0000 7f5165257740 1 --2- 172.21.15.172:0/1664505277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b41988780 0x562b41985980 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f5144011ad0 tx=0x7f5144019000 comp rx=0 tx=0).stop 2024-09-10T09:06:36.818 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.790+0000 7f5165257740 1 -- 172.21.15.172:0/1664505277 shutdown_connections 2024-09-10T09:06:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.790+0000 7f5165257740 1 --2- 172.21.15.172:0/1664505277 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x562b41998020 0x562b4199a480 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.790+0000 7f5165257740 1 --2- 172.21.15.172:0/1664505277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b41988780 0x562b41985980 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.790+0000 7f5165257740 1 --2- 172.21.15.172:0/1664505277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b4197ab40 0x562b419853e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.790+0000 7f5165257740 1 --2- 172.21.15.172:0/1664505277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b419795d0 0x562b41988130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.790+0000 7f513bfff640 0 client.20364 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T09:06:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.790+0000 7f5165257740 1 -- 172.21.15.172:0/1664505277 >> 172.21.15.172:0/1664505277 conn(0x562b41942310 msgr2=0x562b4197b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.790+0000 7f5165257740 1 -- 172.21.15.172:0/1664505277 shutdown_connections 2024-09-10T09:06:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.790+0000 7f5165257740 1 -- 172.21.15.172:0/1664505277 wait complete. 2024-09-10T09:06:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.791+0000 7f5165257740 20 client.20364 trim_cache size 0 max 16384 2024-09-10T09:06:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.803+0000 7f5165257740 1 Processor -- start 2024-09-10T09:06:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.803+0000 7f5165257740 1 -- start start 2024-09-10T09:06:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.803+0000 7f5165257740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b419795d0 0x562b41975e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.803+0000 7f5165257740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b4197ab40 0x562b41994250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.803+0000 7f5165257740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b41988780 0x562b4199fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.803+0000 7f5165257740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1ac9f0 con 0x562b41988780 2024-09-10T09:06:36.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.803+0000 7f5165257740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1ae5f0 con 0x562b419795d0 2024-09-10T09:06:36.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.803+0000 7f5165257740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1ad0c0 con 0x562b4197ab40 2024-09-10T09:06:36.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.803+0000 7f516141b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b41988780 0x562b4199fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:36.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.803+0000 7f516141b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b41988780 0x562b4199fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43712/0 (socket says 172.21.15.172:43712) 2024-09-10T09:06:36.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.804+0000 7f516141b640 1 -- 172.21.15.172:0/1169183557 learned_addr learned my addr 172.21.15.172:0/1169183557 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:36.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.804+0000 7f5160c1a640 1 --2- 172.21.15.172:0/1169183557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b419795d0 0x562b41975e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:36.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.804+0000 7f516141b640 1 -- 172.21.15.172:0/1169183557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b4197ab40 msgr2=0x562b41994250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:36.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.804+0000 7f516141b640 1 --2- 172.21.15.172:0/1169183557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b4197ab40 0x562b41994250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.804+0000 7f516141b640 1 -- 172.21.15.172:0/1169183557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b419795d0 msgr2=0x562b41975e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:36.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.804+0000 7f516141b640 1 --2- 172.21.15.172:0/1169183557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b419795d0 0x562b41975e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.804+0000 7f516141b640 1 -- 172.21.15.172:0/1169183557 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b41983930 con 0x562b41988780 2024-09-10T09:06:36.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.804+0000 7f5160c1a640 1 --2- 172.21.15.172:0/1169183557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b419795d0 0x562b41975e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:06:36.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.804+0000 7f516141b640 1 --2- 172.21.15.172:0/1169183557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b41988780 0x562b4199fd50 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f514400caa0 tx=0x7f514400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa9d3709c4a4eab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:36.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.804+0000 7f51527fc640 1 -- 172.21.15.172:0/1169183557 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144026e30 con 0x562b41988780 2024-09-10T09:06:36.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.804+0000 7f51527fc640 1 -- 172.21.15.172:0/1169183557 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f514401e7d0 con 0x562b41988780 2024-09-10T09:06:36.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.804+0000 7f51527fc640 1 -- 172.21.15.172:0/1169183557 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144027780 con 0x562b41988780 2024-09-10T09:06:36.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 1 -- 172.21.15.172:0/1169183557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b41988780 msgr2=0x562b4199fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:36.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 1 --2- 172.21.15.172:0/1169183557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b41988780 0x562b4199fd50 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f514400caa0 tx=0x7f514400c850 comp rx=0 tx=0).stop 2024-09-10T09:06:36.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 1 -- 172.21.15.172:0/1169183557 shutdown_connections 2024-09-10T09:06:36.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 1 --2- 172.21.15.172:0/1169183557 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b41988780 0x562b4199fd50 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 1 --2- 172.21.15.172:0/1169183557 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b4197ab40 0x562b41994250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 1 --2- 172.21.15.172:0/1169183557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b419795d0 0x562b41975e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 1 -- 172.21.15.172:0/1169183557 >> 172.21.15.172:0/1169183557 conn(0x562b41942310 msgr2=0x562b41975700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:36.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 1 -- 172.21.15.172:0/1169183557 shutdown_connections 2024-09-10T09:06:36.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 1 -- 172.21.15.172:0/1169183557 wait complete. 2024-09-10T09:06:36.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 1 -- start start 2024-09-10T09:06:36.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 1 Processor -- start 2024-09-10T09:06:36.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b419795d0 0x562b41990690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b4197ab40 0x562b41990bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b41988780 0x562b41981de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1adef0 con 0x562b419795d0 2024-09-10T09:06:36.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1ad7c0 con 0x562b41988780 2024-09-10T09:06:36.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f515c1aa970 con 0x562b4197ab40 2024-09-10T09:06:36.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.805+0000 7f5165257740 10 client.0 _ll_register_callbacks cb 0x7f514400c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:06:36.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f5160c1a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b419795d0 0x562b41990690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:36.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f5160c1a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b419795d0 0x562b41990690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43728/0 (socket says 172.21.15.172:43728) 2024-09-10T09:06:36.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f5160c1a640 1 -- 172.21.15.172:0/2201823651 learned_addr learned my addr 172.21.15.172:0/2201823651 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:36.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f515bfff640 1 --2- 172.21.15.172:0/2201823651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b4197ab40 0x562b41990bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:36.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f516141b640 1 --2- 172.21.15.172:0/2201823651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b41988780 0x562b41981de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:36.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f5160c1a640 1 -- 172.21.15.172:0/2201823651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b4197ab40 msgr2=0x562b41990bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:36.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f5160c1a640 1 --2- 172.21.15.172:0/2201823651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b4197ab40 0x562b41990bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f5160c1a640 1 -- 172.21.15.172:0/2201823651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b41988780 msgr2=0x562b41981de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:36.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f5160c1a640 1 --2- 172.21.15.172:0/2201823651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b41988780 0x562b41981de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f5160c1a640 1 -- 172.21.15.172:0/2201823651 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b419825e0 con 0x562b419795d0 2024-09-10T09:06:36.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f5160c1a640 1 --2- 172.21.15.172:0/2201823651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b419795d0 0x562b41990690 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f51540b7910 tx=0x7f51540ef930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb8d834ee1574b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:36.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f513bfff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:06:36.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f513bfff640 1 -- 172.21.15.172:0/2201823651 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154048040 con 0x562b419795d0 2024-09-10T09:06:36.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f513bfff640 1 -- 172.21.15.172:0/2201823651 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f51540b9f30 con 0x562b419795d0 2024-09-10T09:06:36.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f5165257740 1 -- 172.21.15.172:0/2201823651 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f514401e7d0 con 0x562b419795d0 2024-09-10T09:06:36.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f513bfff640 1 -- 172.21.15.172:0/2201823651 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f51540bc330 con 0x562b419795d0 2024-09-10T09:06:36.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f5165257740 10 client.20370 fetch_fsmap learned FSMap version 49 2024-09-10T09:06:36.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.806+0000 7f5165257740 1 -- 172.21.15.172:0/2201823651 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562b41a027f0 con 0x562b419795d0 2024-09-10T09:06:36.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.807+0000 7f513bfff640 1 -- 172.21.15.172:0/2201823651 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f514401e7d0 con 0x562b419795d0 2024-09-10T09:06:36.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.807+0000 7f513bfff640 1 -- 172.21.15.172:0/2201823651 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 49) ==== 8398+0+0 (secure 0 0 0) 0x7f515404b060 con 0x562b419795d0 2024-09-10T09:06:36.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.807+0000 7f5165257740 10 client.20370 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T09:06:36.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.807+0000 7f5165257740 10 client.20370 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-10T09:06:36.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.807+0000 7f5165257740 20 client.20370 populate_metadata read hostname 'smithi172' 2024-09-10T09:06:36.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.807+0000 7f5165257740 1 --2- 172.21.15.172:0/2201823651 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x562b41a02d10 0x562b41a05130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:36.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.807+0000 7f5165257740 4 client.20370 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:06:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.807+0000 7f5165257740 1 -- 172.21.15.172:0/2201823651 --> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b41863170 con 0x562b41a02d10 2024-09-10T09:06:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.807+0000 7f515bfff640 1 --2- 172.21.15.172:0/2201823651 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x562b41a02d10 0x562b41a05130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.807+0000 7f515bfff640 1 --2- 172.21.15.172:0/2201823651 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x562b41a02d10 0x562b41a05130 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b09c76af9df8df94 server_cookie=89e3b9c563e61c93 in_seq=0 out_seq=0 2024-09-10T09:06:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.807+0000 7f513bfff640 10 client.20370 ms_handle_connect on v2:172.21.15.191:6838/1702936476 2024-09-10T09:06:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.808+0000 7f513bfff640 1 -- 172.21.15.172:0/2201823651 <== mds.2 v2:172.21.15.191:6838/1702936476 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b41863170 con 0x562b41a02d10 2024-09-10T09:06:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.808+0000 7f513bfff640 10 client.20370 handle_command_reply: tid=0 2024-09-10T09:06:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.808+0000 7f5165257740 1 client.20370 shutdown 2024-09-10T09:06:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.808+0000 7f5165257740 1 -- 172.21.15.172:0/2201823651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b419795d0 msgr2=0x562b41990690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.808+0000 7f5165257740 1 --2- 172.21.15.172:0/2201823651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b419795d0 0x562b41990690 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f51540b7910 tx=0x7f51540ef930 comp rx=0 tx=0).stop 2024-09-10T09:06:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.808+0000 7f5165257740 1 -- 172.21.15.172:0/2201823651 shutdown_connections 2024-09-10T09:06:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.808+0000 7f5165257740 1 --2- 172.21.15.172:0/2201823651 >> [v2:172.21.15.191:6838/1702936476,v1:172.21.15.191:6839/1702936476] conn(0x562b41a02d10 0x562b41a05130 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.808+0000 7f5165257740 1 --2- 172.21.15.172:0/2201823651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b41988780 0x562b41981de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.808+0000 7f5165257740 1 --2- 172.21.15.172:0/2201823651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b4197ab40 0x562b41990bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.808+0000 7f513bfff640 0 client.20370 ms_handle_reset on v2:172.21.15.191:6838/1702936476 2024-09-10T09:06:36.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.808+0000 7f5165257740 1 --2- 172.21.15.172:0/2201823651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562b419795d0 0x562b41990690 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.808+0000 7f5165257740 1 -- 172.21.15.172:0/2201823651 >> 172.21.15.172:0/2201823651 conn(0x562b41942310 msgr2=0x562b41986c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:36.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.808+0000 7f5165257740 1 -- 172.21.15.172:0/2201823651 shutdown_connections 2024-09-10T09:06:36.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.808+0000 7f5165257740 1 -- 172.21.15.172:0/2201823651 wait complete. 2024-09-10T09:06:36.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.808+0000 7f5165257740 20 client.20370 trim_cache size 0 max 16384 2024-09-10T09:06:36.826 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:06:36.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.809+0000 7f5162e7e640 1 -- 172.21.15.172:0/2003547947 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51480a2b80 msgr2=0x7f51480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:36.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.809+0000 7f5162e7e640 1 --2- 172.21.15.172:0/2003547947 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51480a2b80 0x7f51480a5040 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5144005d80 tx=0x7f514400c7a0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.809+0000 7f5162e7e640 1 -- 172.21.15.172:0/2003547947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f515c100a80 msgr2=0x7f515c199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:36.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.809+0000 7f5162e7e640 1 --2- 172.21.15.172:0/2003547947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f515c100a80 0x7f515c199190 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f5154004770 tx=0x7f51540042f0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.809+0000 7f5162e7e640 1 -- 172.21.15.172:0/2003547947 shutdown_connections 2024-09-10T09:06:36.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.809+0000 7f5162e7e640 1 --2- 172.21.15.172:0/2003547947 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f51480a2b80 0x7f51480a5040 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.809+0000 7f5162e7e640 1 --2- 172.21.15.172:0/2003547947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f515c106a80 0x7f515c1a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.809+0000 7f5162e7e640 1 --2- 172.21.15.172:0/2003547947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f515c101420 0x7f515c1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.809+0000 7f5162e7e640 1 --2- 172.21.15.172:0/2003547947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f515c100a80 0x7f515c199190 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:36.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.809+0000 7f5162e7e640 1 -- 172.21.15.172:0/2003547947 >> 172.21.15.172:0/2003547947 conn(0x7f515c0fc7a0 msgr2=0x7f515c0fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:36.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.809+0000 7f5162e7e640 1 -- 172.21.15.172:0/2003547947 shutdown_connections 2024-09-10T09:06:36.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:36.809+0000 7f5162e7e640 1 -- 172.21.15.172:0/2003547947 wait complete. 2024-09-10T09:06:37.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:36 smithi172 ceph-mon[30854]: pgmap v961: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-10T09:06:37.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:36 smithi174 ceph-mon[34068]: pgmap v961: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-10T09:06:37.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:36 smithi191 ceph-mon[35314]: pgmap v961: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-10T09:06:37.601 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:06:37.849 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:06:38.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.331+0000 7f86c47e6640 1 -- 172.21.15.172:0/471376246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86bc101de0 msgr2=0x7f86bc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:38.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.331+0000 7f86c47e6640 1 --2- 172.21.15.172:0/471376246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86bc101de0 0x7f86bc1021c0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009900 tx=0x7f86b802a410 comp rx=0 tx=0).stop 2024-09-10T09:06:38.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.331+0000 7f86c47e6640 1 -- 172.21.15.172:0/471376246 shutdown_connections 2024-09-10T09:06:38.360 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.331+0000 7f86c47e6640 1 --2- 172.21.15.172:0/471376246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc106f10 0x7f86bc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.331+0000 7f86c47e6640 1 --2- 172.21.15.172:0/471376246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86bc102790 0x7f86bc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.331+0000 7f86c47e6640 1 --2- 172.21.15.172:0/471376246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86bc101de0 0x7f86bc1021c0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.331+0000 7f86c47e6640 1 -- 172.21.15.172:0/471376246 >> 172.21.15.172:0/471376246 conn(0x7f86bc0fd8c0 msgr2=0x7f86bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:38.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.331+0000 7f86c47e6640 1 -- 172.21.15.172:0/471376246 shutdown_connections 2024-09-10T09:06:38.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.331+0000 7f86c47e6640 1 -- 172.21.15.172:0/471376246 wait complete. 2024-09-10T09:06:38.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c47e6640 1 Processor -- start 2024-09-10T09:06:38.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c47e6640 1 -- start start 2024-09-10T09:06:38.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c47e6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86bc101de0 0x7f86bc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c47e6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc102790 0x7f86bc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c47e6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86bc106f10 0x7f86bc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c47e6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc111e40 con 0x7f86bc101de0 2024-09-10T09:06:38.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c47e6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc111cc0 con 0x7f86bc106f10 2024-09-10T09:06:38.361 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c47e6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc111fc0 con 0x7f86bc102790 2024-09-10T09:06:38.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c2582640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86bc101de0 0x7f86bc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:38.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c2582640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86bc101de0 0x7f86bc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43742/0 (socket says 172.21.15.172:43742) 2024-09-10T09:06:38.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c2582640 1 -- 172.21.15.172:0/2028680540 learned_addr learned my addr 172.21.15.172:0/2028680540 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:38.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c1d81640 1 --2- 172.21.15.172:0/2028680540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc102790 0x7f86bc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:38.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c2d83640 1 --2- 172.21.15.172:0/2028680540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86bc106f10 0x7f86bc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:38.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c2582640 1 -- 172.21.15.172:0/2028680540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc102790 msgr2=0x7f86bc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:38.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c2582640 1 --2- 172.21.15.172:0/2028680540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc102790 0x7f86bc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c2582640 1 -- 172.21.15.172:0/2028680540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86bc106f10 msgr2=0x7f86bc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:38.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c2582640 1 --2- 172.21.15.172:0/2028680540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86bc106f10 0x7f86bc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c2582640 1 -- 172.21.15.172:0/2028680540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc1a68a0 con 0x7f86bc101de0 2024-09-10T09:06:38.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c1d81640 1 --2- 172.21.15.172:0/2028680540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc102790 0x7f86bc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:38.362 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.332+0000 7f86c2d83640 1 --2- 172.21.15.172:0/2028680540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86bc106f10 0x7f86bc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:06:38.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.333+0000 7f86c2582640 1 --2- 172.21.15.172:0/2028680540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86bc101de0 0x7f86bc0744f0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f86b8005940 tx=0x7f86b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:38.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.333+0000 7f86b37fe640 1 -- 172.21.15.172:0/2028680540 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b803f030 con 0x7f86bc101de0 2024-09-10T09:06:38.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.333+0000 7f86b37fe640 1 -- 172.21.15.172:0/2028680540 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86b8003aa0 con 0x7f86bc101de0 2024-09-10T09:06:38.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.333+0000 7f86c47e6640 1 -- 172.21.15.172:0/2028680540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86bc1a6b30 con 0x7f86bc101de0 2024-09-10T09:06:38.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.333+0000 7f86b37fe640 1 -- 172.21.15.172:0/2028680540 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b8035070 con 0x7f86bc101de0 2024-09-10T09:06:38.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.333+0000 7f86c47e6640 1 -- 172.21.15.172:0/2028680540 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86bc1a7010 con 0x7f86bc101de0 2024-09-10T09:06:38.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.335+0000 7f86b37fe640 1 -- 172.21.15.172:0/2028680540 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f86b8042030 con 0x7f86bc101de0 2024-09-10T09:06:38.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.335+0000 7f86c6bbf740 1 Processor -- start 2024-09-10T09:06:38.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.336+0000 7f86c6bbf740 1 -- start start 2024-09-10T09:06:38.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.336+0000 7f86c6bbf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae65b40 0x55f5dae64090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.336+0000 7f86c6bbf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae645d0 0x55f5dae64a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.336+0000 7f86c6bbf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae6f370 0x55f5dae71760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.363 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.336+0000 7f86c6bbf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f5dadca2f0 con 0x55f5dae6f370 2024-09-10T09:06:38.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.336+0000 7f86c6bbf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f5dad71cf0 con 0x55f5dae65b40 2024-09-10T09:06:38.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.336+0000 7f86c6bbf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f5dadc62b0 con 0x55f5dae645d0 2024-09-10T09:06:38.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.336+0000 7f86b37fe640 1 --2- 172.21.15.172:0/2028680540 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86a80a2b80 0x7f86a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.336+0000 7f86c1d81640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae6f370 0x55f5dae71760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:38.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.336+0000 7f86c1d81640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae6f370 0x55f5dae71760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43744/0 (socket says 172.21.15.172:43744) 2024-09-10T09:06:38.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.336+0000 7f86c1d81640 1 -- 172.21.15.172:0/1071860595 learned_addr learned my addr 172.21.15.172:0/1071860595 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:38.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.336+0000 7f86c2d83640 1 --2- 172.21.15.172:0/1071860595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae65b40 0x55f5dae64090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:38.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.336+0000 7f86c2d83640 1 --2- 172.21.15.172:0/2028680540 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86a80a2b80 0x7f86a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:38.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.336+0000 7f86b37fe640 1 -- 172.21.15.172:0/2028680540 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f86b80efc60 con 0x7f86bc101de0 2024-09-10T09:06:38.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.336+0000 7f86c2582640 1 --2- 172.21.15.172:0/1071860595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae645d0 0x55f5dae64a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:38.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.337+0000 7f86c1d81640 1 -- 172.21.15.172:0/1071860595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae645d0 msgr2=0x55f5dae64a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:38.364 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.337+0000 7f86c1d81640 1 --2- 172.21.15.172:0/1071860595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae645d0 0x55f5dae64a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.337+0000 7f86c1d81640 1 -- 172.21.15.172:0/1071860595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae65b40 msgr2=0x55f5dae64090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:38.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.337+0000 7f86c1d81640 1 --2- 172.21.15.172:0/1071860595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae65b40 0x55f5dae64090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.337+0000 7f86c1d81640 1 -- 172.21.15.172:0/1071860595 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5dad563a0 con 0x55f5dae6f370 2024-09-10T09:06:38.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.337+0000 7f86c2d83640 1 --2- 172.21.15.172:0/1071860595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae65b40 0x55f5dae64090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:06:38.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.337+0000 7f86c1d81640 1 --2- 172.21.15.172:0/1071860595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae6f370 0x55f5dae71760 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00bd30 tx=0x7f86ac00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8dd8bfeaed55ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:38.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.337+0000 7f86c2d83640 1 --2- 172.21.15.172:0/2028680540 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86a80a2b80 0x7f86a80a5040 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009fd0 tx=0x7f86b4008220 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:38.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.337+0000 7f86b0ff9640 1 -- 172.21.15.172:0/1071860595 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac015070 con 0x55f5dae6f370 2024-09-10T09:06:38.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.337+0000 7f86b0ff9640 1 -- 172.21.15.172:0/1071860595 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86ac00be20 con 0x55f5dae6f370 2024-09-10T09:06:38.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.337+0000 7f86b0ff9640 1 -- 172.21.15.172:0/1071860595 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0119d0 con 0x55f5dae6f370 2024-09-10T09:06:38.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.338+0000 7f86c6bbf740 1 -- 172.21.15.172:0/1071860595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae6f370 msgr2=0x55f5dae71760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:38.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.338+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/1071860595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae6f370 0x55f5dae71760 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00bd30 tx=0x7f86ac00f730 comp rx=0 tx=0).stop 2024-09-10T09:06:38.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.338+0000 7f86c6bbf740 1 -- 172.21.15.172:0/1071860595 shutdown_connections 2024-09-10T09:06:38.365 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.338+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/1071860595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae6f370 0x55f5dae71760 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.338+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/1071860595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae645d0 0x55f5dae64a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.338+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/1071860595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae65b40 0x55f5dae64090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.338+0000 7f86c6bbf740 1 -- 172.21.15.172:0/1071860595 >> 172.21.15.172:0/1071860595 conn(0x55f5dae2d310 msgr2=0x55f5dae60c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:38.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.338+0000 7f86c6bbf740 1 -- 172.21.15.172:0/1071860595 shutdown_connections 2024-09-10T09:06:38.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.338+0000 7f86c6bbf740 1 -- 172.21.15.172:0/1071860595 wait complete. 2024-09-10T09:06:38.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.338+0000 7f86c6bbf740 1 -- start start 2024-09-10T09:06:38.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.338+0000 7f86c6bbf740 1 Processor -- start 2024-09-10T09:06:38.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.338+0000 7f86c6bbf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae645d0 0x55f5dae6ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.338+0000 7f86c6bbf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae65b40 0x55f5dae6c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.338+0000 7f86c6bbf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae6f370 0x55f5dae6c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.338+0000 7f86c6bbf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f5dae72070 con 0x55f5dae645d0 2024-09-10T09:06:38.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.338+0000 7f86c6bbf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f5dad85060 con 0x55f5dae6f370 2024-09-10T09:06:38.366 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.338+0000 7f86c6bbf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f5dae721f0 con 0x55f5dae65b40 2024-09-10T09:06:38.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86c6bbf740 10 client.0 _ll_register_callbacks cb 0x55f5daad88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:06:38.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86c2582640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae645d0 0x55f5dae6ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:38.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86c2582640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae645d0 0x55f5dae6ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43760/0 (socket says 172.21.15.172:43760) 2024-09-10T09:06:38.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86c2582640 1 -- 172.21.15.172:0/2075154370 learned_addr learned my addr 172.21.15.172:0/2075154370 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:38.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86c1d81640 1 --2- 172.21.15.172:0/2075154370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae65b40 0x55f5dae6c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:38.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86c2582640 1 -- 172.21.15.172:0/2075154370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae65b40 msgr2=0x55f5dae6c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:38.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86c2582640 1 --2- 172.21.15.172:0/2075154370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae65b40 0x55f5dae6c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86c2582640 1 -- 172.21.15.172:0/2075154370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae6f370 msgr2=0x55f5dae6c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:38.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86c2582640 1 --2- 172.21.15.172:0/2075154370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae6f370 0x55f5dae6c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86c2582640 1 -- 172.21.15.172:0/2075154370 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5dab3b3c0 con 0x55f5dae645d0 2024-09-10T09:06:38.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86c1d81640 1 --2- 172.21.15.172:0/2075154370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae65b40 0x55f5dae6c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:38.367 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86c2582640 1 --2- 172.21.15.172:0/2075154370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae645d0 0x55f5dae6ed60 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f86b80ed3a0 tx=0x7f86b80bc8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3299dd2b5ca9b3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:38.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:06:38.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86a5ffb640 1 -- 172.21.15.172:0/2075154370 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b8049040 con 0x55f5dae645d0 2024-09-10T09:06:38.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86a5ffb640 1 -- 172.21.15.172:0/2075154370 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f86b8003c50 con 0x55f5dae645d0 2024-09-10T09:06:38.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86c6bbf740 1 -- 172.21.15.172:0/2075154370 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86ac00be20 con 0x55f5dae645d0 2024-09-10T09:06:38.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86a5ffb640 1 -- 172.21.15.172:0/2075154370 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f86b80c4d70 con 0x55f5dae645d0 2024-09-10T09:06:38.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86c6bbf740 10 client.20382 fetch_fsmap learned FSMap version 49 2024-09-10T09:06:38.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86c6bbf740 1 -- 172.21.15.172:0/2075154370 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f5dae7db10 con 0x55f5dae645d0 2024-09-10T09:06:38.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.339+0000 7f86a5ffb640 1 -- 172.21.15.172:0/2075154370 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86ac00be20 con 0x55f5dae645d0 2024-09-10T09:06:38.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.340+0000 7f86a5ffb640 1 -- 172.21.15.172:0/2075154370 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 49) ==== 8398+0+0 (secure 0 0 0) 0x7f86b80b8770 con 0x55f5dae645d0 2024-09-10T09:06:38.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.340+0000 7f86c6bbf740 10 client.20382 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T09:06:38.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.340+0000 7f86c6bbf740 10 client.20382 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:06:38.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.340+0000 7f86c6bbf740 20 client.20382 populate_metadata read hostname 'smithi172' 2024-09-10T09:06:38.368 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.340+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/2075154370 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55f5dae7ef20 0x55f5dae813e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.340+0000 7f86c6bbf740 4 client.20382 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:06:38.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.340+0000 7f86c6bbf740 1 -- 172.21.15.172:0/2075154370 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5dadd7860 con 0x55f5dae7ef20 2024-09-10T09:06:38.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.340+0000 7f86c1d81640 1 --2- 172.21.15.172:0/2075154370 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55f5dae7ef20 0x55f5dae813e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:38.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.340+0000 7f86a5ffb640 10 client.20382 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T09:06:38.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.340+0000 7f86c1d81640 1 --2- 172.21.15.172:0/2075154370 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55f5dae7ef20 0x55f5dae813e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef6e7f2b41078b29 server_cookie=4ad6daab140209cb in_seq=0 out_seq=0 2024-09-10T09:06:38.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.341+0000 7f86a5ffb640 1 -- 172.21.15.172:0/2075154370 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5dadd7860 con 0x55f5dae7ef20 2024-09-10T09:06:38.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.341+0000 7f86a5ffb640 10 client.20382 handle_command_reply: tid=0 2024-09-10T09:06:38.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.341+0000 7f86c6bbf740 1 client.20382 shutdown 2024-09-10T09:06:38.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.341+0000 7f86c6bbf740 1 -- 172.21.15.172:0/2075154370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae645d0 msgr2=0x55f5dae6ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:38.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.341+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/2075154370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae645d0 0x55f5dae6ed60 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f86b80ed3a0 tx=0x7f86b80bc8c0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.341+0000 7f86c6bbf740 1 -- 172.21.15.172:0/2075154370 shutdown_connections 2024-09-10T09:06:38.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.341+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/2075154370 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55f5dae7ef20 0x55f5dae813e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.369 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.341+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/2075154370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae6f370 0x55f5dae6c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.341+0000 7f86a5ffb640 0 client.20382 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T09:06:38.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.341+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/2075154370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae65b40 0x55f5dae6c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.341+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/2075154370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae645d0 0x55f5dae6ed60 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.341+0000 7f86c6bbf740 1 -- 172.21.15.172:0/2075154370 >> 172.21.15.172:0/2075154370 conn(0x55f5dae2d310 msgr2=0x55f5dae66960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:38.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.341+0000 7f86c6bbf740 1 -- 172.21.15.172:0/2075154370 shutdown_connections 2024-09-10T09:06:38.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.341+0000 7f86c6bbf740 1 -- 172.21.15.172:0/2075154370 wait complete. 2024-09-10T09:06:38.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.341+0000 7f86c6bbf740 20 client.20382 trim_cache size 0 max 16384 2024-09-10T09:06:38.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.354+0000 7f86c6bbf740 1 Processor -- start 2024-09-10T09:06:38.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.354+0000 7f86c6bbf740 1 -- start start 2024-09-10T09:06:38.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.354+0000 7f86c6bbf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae645d0 0x55f5dae7b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.354+0000 7f86c6bbf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae65b40 0x55f5dae60700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.354+0000 7f86c6bbf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae6f370 0x55f5dae70760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.370 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.354+0000 7f86c6bbf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1b0de0 con 0x55f5dae6f370 2024-09-10T09:06:38.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.354+0000 7f86c6bbf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1b29e0 con 0x55f5dae65b40 2024-09-10T09:06:38.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.354+0000 7f86c6bbf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1b14b0 con 0x55f5dae645d0 2024-09-10T09:06:38.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.354+0000 7f86c2d83640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae6f370 0x55f5dae70760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:38.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.354+0000 7f86c2d83640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae6f370 0x55f5dae70760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43772/0 (socket says 172.21.15.172:43772) 2024-09-10T09:06:38.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.354+0000 7f86c2d83640 1 -- 172.21.15.172:0/174945248 learned_addr learned my addr 172.21.15.172:0/174945248 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:38.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.354+0000 7f86c1d81640 1 --2- 172.21.15.172:0/174945248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae65b40 0x55f5dae60700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:38.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.354+0000 7f86c2582640 1 --2- 172.21.15.172:0/174945248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae645d0 0x55f5dae7b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:38.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.355+0000 7f86c2d83640 1 -- 172.21.15.172:0/174945248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae645d0 msgr2=0x55f5dae7b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:38.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.355+0000 7f86c2d83640 1 --2- 172.21.15.172:0/174945248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae645d0 0x55f5dae7b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.355+0000 7f86c2d83640 1 -- 172.21.15.172:0/174945248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae65b40 msgr2=0x55f5dae60700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:38.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.355+0000 7f86c2d83640 1 --2- 172.21.15.172:0/174945248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae65b40 0x55f5dae60700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.355+0000 7f86c2d83640 1 -- 172.21.15.172:0/174945248 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5dae60ed0 con 0x55f5dae6f370 2024-09-10T09:06:38.371 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.355+0000 7f86c1d81640 1 --2- 172.21.15.172:0/174945248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae65b40 0x55f5dae60700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:38.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.355+0000 7f86c2d83640 1 --2- 172.21.15.172:0/174945248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae6f370 0x55f5dae70760 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f86b400ed20 tx=0x7f86b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99165af43103d0c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:38.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.355+0000 7f86a7fff640 1 -- 172.21.15.172:0/174945248 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4020070 con 0x55f5dae6f370 2024-09-10T09:06:38.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.355+0000 7f86a7fff640 1 -- 172.21.15.172:0/174945248 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86b400bda0 con 0x55f5dae6f370 2024-09-10T09:06:38.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.355+0000 7f86a7fff640 1 -- 172.21.15.172:0/174945248 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4024c70 con 0x55f5dae6f370 2024-09-10T09:06:38.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.355+0000 7f86c6bbf740 1 -- 172.21.15.172:0/174945248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae6f370 msgr2=0x55f5dae70760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:38.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.355+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/174945248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae6f370 0x55f5dae70760 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f86b400ed20 tx=0x7f86b4004a40 comp rx=0 tx=0).stop 2024-09-10T09:06:38.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c6bbf740 1 -- 172.21.15.172:0/174945248 shutdown_connections 2024-09-10T09:06:38.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/174945248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae6f370 0x55f5dae70760 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/174945248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae65b40 0x55f5dae60700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/174945248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae645d0 0x55f5dae7b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c6bbf740 1 -- 172.21.15.172:0/174945248 >> 172.21.15.172:0/174945248 conn(0x55f5dae2d310 msgr2=0x55f5dae6a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:38.372 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c6bbf740 1 -- 172.21.15.172:0/174945248 shutdown_connections 2024-09-10T09:06:38.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c6bbf740 1 -- 172.21.15.172:0/174945248 wait complete. 2024-09-10T09:06:38.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c6bbf740 1 -- start start 2024-09-10T09:06:38.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c6bbf740 1 Processor -- start 2024-09-10T09:06:38.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c6bbf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae645d0 0x55f5dae86920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c6bbf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae65b40 0x55f5dae86e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c6bbf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae6f370 0x55f5dae873a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c6bbf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1b22e0 con 0x55f5dae65b40 2024-09-10T09:06:38.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c6bbf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1b1bb0 con 0x55f5dae6f370 2024-09-10T09:06:38.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c6bbf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1aed60 con 0x55f5dae645d0 2024-09-10T09:06:38.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c6bbf740 10 client.0 _ll_register_callbacks cb 0x7f86b803b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:06:38.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c1d81640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae65b40 0x55f5dae86e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:38.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c1d81640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae65b40 0x55f5dae86e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43782/0 (socket says 172.21.15.172:43782) 2024-09-10T09:06:38.373 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c1d81640 1 -- 172.21.15.172:0/3658245670 learned_addr learned my addr 172.21.15.172:0/3658245670 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:38.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c2d83640 1 --2- 172.21.15.172:0/3658245670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae6f370 0x55f5dae873a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:38.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.356+0000 7f86c2582640 1 --2- 172.21.15.172:0/3658245670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae645d0 0x55f5dae86920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:38.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86c1d81640 1 -- 172.21.15.172:0/3658245670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae645d0 msgr2=0x55f5dae86920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:38.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86c1d81640 1 --2- 172.21.15.172:0/3658245670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae645d0 0x55f5dae86920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86c1d81640 1 -- 172.21.15.172:0/3658245670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae6f370 msgr2=0x55f5dae873a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:38.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86c1d81640 1 --2- 172.21.15.172:0/3658245670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae6f370 0x55f5dae873a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86c1d81640 1 -- 172.21.15.172:0/3658245670 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5dae75ac0 con 0x55f5dae65b40 2024-09-10T09:06:38.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86c2d83640 1 --2- 172.21.15.172:0/3658245670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae6f370 0x55f5dae873a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:38.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86c1d81640 1 --2- 172.21.15.172:0/3658245670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae65b40 0x55f5dae86e60 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00bd30 tx=0x7f86ac00b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6432a9206331f245 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:38.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:06:38.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86a5ffb640 1 -- 172.21.15.172:0/3658245670 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac02c060 con 0x55f5dae65b40 2024-09-10T09:06:38.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86a5ffb640 1 -- 172.21.15.172:0/3658245670 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f86ac004440 con 0x55f5dae65b40 2024-09-10T09:06:38.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86c6bbf740 1 -- 172.21.15.172:0/3658245670 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86b400bda0 con 0x55f5dae65b40 2024-09-10T09:06:38.374 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86a5ffb640 1 -- 172.21.15.172:0/3658245670 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f86ac033050 con 0x55f5dae65b40 2024-09-10T09:06:38.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86c6bbf740 10 client.20388 fetch_fsmap learned FSMap version 49 2024-09-10T09:06:38.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86c6bbf740 1 -- 172.21.15.172:0/3658245670 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f5daeea060 con 0x55f5dae65b40 2024-09-10T09:06:38.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86a5ffb640 1 -- 172.21.15.172:0/3658245670 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86b400bda0 con 0x55f5dae65b40 2024-09-10T09:06:38.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86a5ffb640 1 -- 172.21.15.172:0/3658245670 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 49) ==== 8398+0+0 (secure 0 0 0) 0x7f86ac004b00 con 0x55f5dae65b40 2024-09-10T09:06:38.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86c6bbf740 10 client.20388 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T09:06:38.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86c6bbf740 10 client.20388 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-10T09:06:38.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86c6bbf740 20 client.20388 populate_metadata read hostname 'smithi172' 2024-09-10T09:06:38.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/3658245670 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55f5dae7ef20 0x55f5daeec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:38.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86c6bbf740 4 client.20388 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:06:38.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.357+0000 7f86c6bbf740 1 -- 172.21.15.172:0/3658245670 --> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f5dadd1d70 con 0x55f5dae7ef20 2024-09-10T09:06:38.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.358+0000 7f86c2582640 1 --2- 172.21.15.172:0/3658245670 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55f5dae7ef20 0x55f5daeec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:38.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.358+0000 7f86c2582640 1 --2- 172.21.15.172:0/3658245670 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55f5dae7ef20 0x55f5daeec5f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18894bfd59930a99 server_cookie=354320e1bf3be143 in_seq=0 out_seq=0 2024-09-10T09:06:38.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.358+0000 7f86a5ffb640 10 client.20388 ms_handle_connect on v2:172.21.15.174:6836/2399337052 2024-09-10T09:06:38.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.358+0000 7f86a5ffb640 1 -- 172.21.15.172:0/3658245670 <== mds.1 v2:172.21.15.174:6836/2399337052 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f5dadd1d70 con 0x55f5dae7ef20 2024-09-10T09:06:38.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.358+0000 7f86a5ffb640 10 client.20388 handle_command_reply: tid=0 2024-09-10T09:06:38.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.358+0000 7f86c6bbf740 1 client.20388 shutdown 2024-09-10T09:06:38.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.358+0000 7f86c6bbf740 1 -- 172.21.15.172:0/3658245670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae65b40 msgr2=0x55f5dae86e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:38.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.358+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/3658245670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae65b40 0x55f5dae86e60 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00bd30 tx=0x7f86ac00b690 comp rx=0 tx=0).stop 2024-09-10T09:06:38.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.358+0000 7f86c6bbf740 1 -- 172.21.15.172:0/3658245670 shutdown_connections 2024-09-10T09:06:38.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.358+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/3658245670 >> [v2:172.21.15.174:6836/2399337052,v1:172.21.15.174:6837/2399337052] conn(0x55f5dae7ef20 0x55f5daeec5f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.358+0000 7f86a5ffb640 0 client.20388 ms_handle_reset on v2:172.21.15.174:6836/2399337052 2024-09-10T09:06:38.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.358+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/3658245670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f5dae6f370 0x55f5dae873a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.358+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/3658245670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f5dae65b40 0x55f5dae86e60 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.358+0000 7f86c6bbf740 1 --2- 172.21.15.172:0/3658245670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f5dae645d0 0x55f5dae86920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.358+0000 7f86c6bbf740 1 -- 172.21.15.172:0/3658245670 >> 172.21.15.172:0/3658245670 conn(0x55f5dae2d310 msgr2=0x55f5dae60560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:38.376 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.359+0000 7f86c6bbf740 1 -- 172.21.15.172:0/3658245670 shutdown_connections 2024-09-10T09:06:38.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.359+0000 7f86c6bbf740 1 -- 172.21.15.172:0/3658245670 wait complete. 2024-09-10T09:06:38.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.359+0000 7f86c6bbf740 20 client.20388 trim_cache size 0 max 16384 2024-09-10T09:06:38.377 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:06:38.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.359+0000 7f86c47e6640 1 -- 172.21.15.172:0/2028680540 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86a80a2b80 msgr2=0x7f86a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:38.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.359+0000 7f86c47e6640 1 --2- 172.21.15.172:0/2028680540 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86a80a2b80 0x7f86a80a5040 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009fd0 tx=0x7f86b4008220 comp rx=0 tx=0).stop 2024-09-10T09:06:38.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.359+0000 7f86c47e6640 1 -- 172.21.15.172:0/2028680540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86bc101de0 msgr2=0x7f86bc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:38.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.359+0000 7f86c47e6640 1 --2- 172.21.15.172:0/2028680540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86bc101de0 0x7f86bc0744f0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f86b8005940 tx=0x7f86b8004270 comp rx=0 tx=0).stop 2024-09-10T09:06:38.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.359+0000 7f86c47e6640 1 -- 172.21.15.172:0/2028680540 shutdown_connections 2024-09-10T09:06:38.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.359+0000 7f86c47e6640 1 --2- 172.21.15.172:0/2028680540 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f86a80a2b80 0x7f86a80a5040 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.359+0000 7f86c47e6640 1 --2- 172.21.15.172:0/2028680540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86bc106f10 0x7f86bc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.359+0000 7f86c47e6640 1 --2- 172.21.15.172:0/2028680540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc102790 0x7f86bc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.359+0000 7f86c47e6640 1 --2- 172.21.15.172:0/2028680540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86bc101de0 0x7f86bc0744f0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:38.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.359+0000 7f86c47e6640 1 -- 172.21.15.172:0/2028680540 >> 172.21.15.172:0/2028680540 conn(0x7f86bc0fd8c0 msgr2=0x7f86bc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:38.378 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.359+0000 7f86c47e6640 1 -- 172.21.15.172:0/2028680540 shutdown_connections 2024-09-10T09:06:38.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:38.359+0000 7f86c47e6640 1 -- 172.21.15.172:0/2028680540 wait complete. 2024-09-10T09:06:39.048 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:38 smithi172 ceph-mon[30854]: pgmap v962: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-10T09:06:39.076 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:06:39.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:38 smithi174 ceph-mon[34068]: pgmap v962: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-10T09:06:39.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:38 smithi191 ceph-mon[35314]: pgmap v962: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-10T09:06:39.321 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:06:39.819 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.809+0000 7f376971c640 1 -- 172.21.15.172:0/3675463599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3764100a80 msgr2=0x7f3764100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.809+0000 7f376971c640 1 --2- 172.21.15.172:0/3675463599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3764100a80 0x7f3764100ee0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f374c009930 tx=0x7f374c02a420 comp rx=0 tx=0).stop 2024-09-10T09:06:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.809+0000 7f376971c640 1 -- 172.21.15.172:0/3675463599 shutdown_connections 2024-09-10T09:06:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.809+0000 7f376971c640 1 --2- 172.21.15.172:0/3675463599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3764101420 0x7f376410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.809+0000 7f376971c640 1 --2- 172.21.15.172:0/3675463599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3764100a80 0x7f3764100ee0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.809+0000 7f376971c640 1 --2- 172.21.15.172:0/3675463599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3764106a80 0x7f3764106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.809+0000 7f376971c640 1 -- 172.21.15.172:0/3675463599 >> 172.21.15.172:0/3675463599 conn(0x7f37640fc7a0 msgr2=0x7f37640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.809+0000 7f376971c640 1 -- 172.21.15.172:0/3675463599 shutdown_connections 2024-09-10T09:06:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.809+0000 7f376971c640 1 -- 172.21.15.172:0/3675463599 wait complete. 2024-09-10T09:06:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f376971c640 1 Processor -- start 2024-09-10T09:06:39.820 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f376971c640 1 -- start start 2024-09-10T09:06:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f376971c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3764100a80 0x7f3764198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f376971c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3764101420 0x7f3764198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f376971c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3764106a80 0x7f376419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f376971c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3764110d50 con 0x7f3764100a80 2024-09-10T09:06:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f376971c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3764110bd0 con 0x7f3764101420 2024-09-10T09:06:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f376971c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3764110ed0 con 0x7f3764106a80 2024-09-10T09:06:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f3762ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3764100a80 0x7f3764198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f3762ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3764100a80 0x7f3764198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43804/0 (socket says 172.21.15.172:43804) 2024-09-10T09:06:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f3762ffd640 1 -- 172.21.15.172:0/1298006923 learned_addr learned my addr 172.21.15.172:0/1298006923 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f375bfff640 1 --2- 172.21.15.172:0/1298006923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3764101420 0x7f3764198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f37637fe640 1 --2- 172.21.15.172:0/1298006923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3764106a80 0x7f376419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f3762ffd640 1 -- 172.21.15.172:0/1298006923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3764106a80 msgr2=0x7f376419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:39.821 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f3762ffd640 1 --2- 172.21.15.172:0/1298006923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3764106a80 0x7f376419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f3762ffd640 1 -- 172.21.15.172:0/1298006923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3764101420 msgr2=0x7f3764198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:39.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f3762ffd640 1 --2- 172.21.15.172:0/1298006923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3764101420 0x7f3764198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f3762ffd640 1 -- 172.21.15.172:0/1298006923 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376419d3f0 con 0x7f3764100a80 2024-09-10T09:06:39.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.810+0000 7f375bfff640 1 --2- 172.21.15.172:0/1298006923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3764101420 0x7f3764198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:39.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.811+0000 7f3762ffd640 1 --2- 172.21.15.172:0/1298006923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3764100a80 0x7f3764198440 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f375400b4a0 tx=0x7f375400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:39.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.811+0000 7f3760ff9640 1 -- 172.21.15.172:0/1298006923 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754014070 con 0x7f3764100a80 2024-09-10T09:06:39.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.811+0000 7f376971c640 1 -- 172.21.15.172:0/1298006923 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376419d6e0 con 0x7f3764100a80 2024-09-10T09:06:39.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.811+0000 7f3760ff9640 1 -- 172.21.15.172:0/1298006923 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3754010ac0 con 0x7f3764100a80 2024-09-10T09:06:39.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.811+0000 7f3760ff9640 1 -- 172.21.15.172:0/1298006923 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754017c30 con 0x7f3764100a80 2024-09-10T09:06:39.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.811+0000 7f376971c640 1 -- 172.21.15.172:0/1298006923 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37641a4fc0 con 0x7f3764100a80 2024-09-10T09:06:39.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.813+0000 7f3760ff9640 1 -- 172.21.15.172:0/1298006923 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3754017070 con 0x7f3764100a80 2024-09-10T09:06:39.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.813+0000 7f376baf5740 1 Processor -- start 2024-09-10T09:06:39.822 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.813+0000 7f376baf5740 1 -- start start 2024-09-10T09:06:39.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f376baf5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0ecb40 0x5609ea0eb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f376baf5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0eb5d0 0x5609ea0eba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f376baf5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0f6370 0x5609ea0f8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f376baf5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5609ea0512f0 con 0x5609ea0f6370 2024-09-10T09:06:39.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f376baf5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5609e9ff8cf0 con 0x5609ea0eb5d0 2024-09-10T09:06:39.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f376baf5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5609ea04d2b0 con 0x5609ea0ecb40 2024-09-10T09:06:39.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f375bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0f6370 0x5609ea0f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:39.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f375bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0f6370 0x5609ea0f8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43816/0 (socket says 172.21.15.172:43816) 2024-09-10T09:06:39.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f375bfff640 1 -- 172.21.15.172:0/3541560638 learned_addr learned my addr 172.21.15.172:0/3541560638 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:39.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f3762ffd640 1 --2- 172.21.15.172:0/3541560638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0eb5d0 0x5609ea0eba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:39.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f3760ff9640 1 --2- 172.21.15.172:0/1298006923 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f37500a2b80 0x7f37500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.823 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f37637fe640 1 --2- 172.21.15.172:0/3541560638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0ecb40 0x5609ea0eb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:39.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f375bfff640 1 -- 172.21.15.172:0/3541560638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0ecb40 msgr2=0x5609ea0eb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:39.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f37637fe640 1 --2- 172.21.15.172:0/1298006923 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f37500a2b80 0x7f37500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:39.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f375bfff640 1 --2- 172.21.15.172:0/3541560638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0ecb40 0x5609ea0eb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f375bfff640 1 -- 172.21.15.172:0/3541560638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0eb5d0 msgr2=0x5609ea0eba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:39.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f375bfff640 1 --2- 172.21.15.172:0/3541560638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0eb5d0 0x5609ea0eba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f375bfff640 1 -- 172.21.15.172:0/3541560638 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609e9fdd3a0 con 0x5609ea0f6370 2024-09-10T09:06:39.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.814+0000 7f3760ff9640 1 -- 172.21.15.172:0/1298006923 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f37540ceb30 con 0x7f3764100a80 2024-09-10T09:06:39.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.815+0000 7f375bfff640 1 --2- 172.21.15.172:0/3541560638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0f6370 0x5609ea0f8760 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f374c031440 tx=0x7f374c032e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c572a662be6cefb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:39.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.815+0000 7f37597fa640 1 -- 172.21.15.172:0/3541560638 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c03f070 con 0x5609ea0f6370 2024-09-10T09:06:39.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.815+0000 7f37637fe640 1 --2- 172.21.15.172:0/1298006923 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f37500a2b80 0x7f37500a5040 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f375c009fd0 tx=0x7f375c005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:39.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.815+0000 7f37597fa640 1 -- 172.21.15.172:0/3541560638 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f374c031760 con 0x5609ea0f6370 2024-09-10T09:06:39.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.815+0000 7f37597fa640 1 -- 172.21.15.172:0/3541560638 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c035070 con 0x5609ea0f6370 2024-09-10T09:06:39.824 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.816+0000 7f376baf5740 1 -- 172.21.15.172:0/3541560638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0f6370 msgr2=0x5609ea0f8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:39.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.816+0000 7f376baf5740 1 --2- 172.21.15.172:0/3541560638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0f6370 0x5609ea0f8760 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f374c031440 tx=0x7f374c032e70 comp rx=0 tx=0).stop 2024-09-10T09:06:39.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.816+0000 7f376baf5740 1 -- 172.21.15.172:0/3541560638 shutdown_connections 2024-09-10T09:06:39.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.816+0000 7f376baf5740 1 --2- 172.21.15.172:0/3541560638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0f6370 0x5609ea0f8760 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.816+0000 7f376baf5740 1 --2- 172.21.15.172:0/3541560638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0eb5d0 0x5609ea0eba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.816+0000 7f376baf5740 1 --2- 172.21.15.172:0/3541560638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0ecb40 0x5609ea0eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.816+0000 7f376baf5740 1 -- 172.21.15.172:0/3541560638 >> 172.21.15.172:0/3541560638 conn(0x5609ea0b4310 msgr2=0x5609ea0e7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:39.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.816+0000 7f376baf5740 1 -- 172.21.15.172:0/3541560638 shutdown_connections 2024-09-10T09:06:39.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.816+0000 7f376baf5740 1 -- 172.21.15.172:0/3541560638 wait complete. 2024-09-10T09:06:39.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.816+0000 7f376baf5740 1 -- start start 2024-09-10T09:06:39.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.816+0000 7f376baf5740 1 Processor -- start 2024-09-10T09:06:39.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.816+0000 7f376baf5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0eb5d0 0x5609ea0f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.816+0000 7f376baf5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0ecb40 0x5609ea0f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.825 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.816+0000 7f376baf5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0f6370 0x5609ea0f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.816+0000 7f376baf5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5609ea0f9010 con 0x5609ea0ecb40 2024-09-10T09:06:39.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.816+0000 7f376baf5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5609ea00c060 con 0x5609ea0eb5d0 2024-09-10T09:06:39.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.816+0000 7f376baf5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5609ea0f9190 con 0x5609ea0f6370 2024-09-10T09:06:39.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.817+0000 7f376baf5740 10 client.0 _ll_register_callbacks cb 0x5609e9d5f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:06:39.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.817+0000 7f375bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0ecb40 0x5609ea0f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:39.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.817+0000 7f375bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0ecb40 0x5609ea0f2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43818/0 (socket says 172.21.15.172:43818) 2024-09-10T09:06:39.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.817+0000 7f375bfff640 1 -- 172.21.15.172:0/3564823250 learned_addr learned my addr 172.21.15.172:0/3564823250 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:39.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.817+0000 7f3762ffd640 1 --2- 172.21.15.172:0/3564823250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0eb5d0 0x5609ea0f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:39.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.817+0000 7f375bfff640 1 -- 172.21.15.172:0/3564823250 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0f6370 msgr2=0x5609ea0f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:39.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.817+0000 7f375bfff640 1 --2- 172.21.15.172:0/3564823250 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0f6370 0x5609ea0f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.817+0000 7f375bfff640 1 -- 172.21.15.172:0/3564823250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0eb5d0 msgr2=0x5609ea0f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:39.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.817+0000 7f375bfff640 1 --2- 172.21.15.172:0/3564823250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0eb5d0 0x5609ea0f5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.817+0000 7f375bfff640 1 -- 172.21.15.172:0/3564823250 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609e9dc23c0 con 0x5609ea0ecb40 2024-09-10T09:06:39.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.817+0000 7f3762ffd640 1 --2- 172.21.15.172:0/3564823250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0eb5d0 0x5609ea0f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:39.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.817+0000 7f375bfff640 1 --2- 172.21.15.172:0/3564823250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0ecb40 0x5609ea0f2fe0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f374c005940 tx=0x7f374c03c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9ef9905daa6d8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:39.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.817+0000 7f374a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:06:39.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.817+0000 7f374a7fc640 1 -- 172.21.15.172:0/3564823250 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c03f070 con 0x5609ea0ecb40 2024-09-10T09:06:39.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.817+0000 7f374a7fc640 1 -- 172.21.15.172:0/3564823250 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f374c031d00 con 0x5609ea0ecb40 2024-09-10T09:06:39.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.817+0000 7f376baf5740 1 -- 172.21.15.172:0/3564823250 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f374c031760 con 0x5609ea0ecb40 2024-09-10T09:06:39.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.818+0000 7f374a7fc640 1 -- 172.21.15.172:0/3564823250 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f374c054050 con 0x5609ea0ecb40 2024-09-10T09:06:39.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.818+0000 7f376baf5740 10 client.20400 fetch_fsmap learned FSMap version 49 2024-09-10T09:06:39.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.818+0000 7f376baf5740 1 -- 172.21.15.172:0/3564823250 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5609ea104a70 con 0x5609ea0ecb40 2024-09-10T09:06:39.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.818+0000 7f374a7fc640 1 -- 172.21.15.172:0/3564823250 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f374c031760 con 0x5609ea0ecb40 2024-09-10T09:06:39.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.818+0000 7f374a7fc640 1 -- 172.21.15.172:0/3564823250 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 49) ==== 8398+0+0 (secure 0 0 0) 0x7f374c0035f0 con 0x5609ea0ecb40 2024-09-10T09:06:39.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.818+0000 7f376baf5740 10 client.20400 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T09:06:39.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.818+0000 7f376baf5740 10 client.20400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:06:39.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.818+0000 7f376baf5740 20 client.20400 populate_metadata read hostname 'smithi172' 2024-09-10T09:06:39.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.818+0000 7f376baf5740 1 --2- 172.21.15.172:0/3564823250 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5609ea105dc0 0x5609ea1081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.818+0000 7f376baf5740 4 client.20400 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:06:39.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.818+0000 7f376baf5740 1 -- 172.21.15.172:0/3564823250 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609ea05e860 con 0x5609ea105dc0 2024-09-10T09:06:39.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.818+0000 7f3762ffd640 1 --2- 172.21.15.172:0/3564823250 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5609ea105dc0 0x5609ea1081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:39.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.819+0000 7f374a7fc640 10 client.20400 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:06:39.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.819+0000 7f3762ffd640 1 --2- 172.21.15.172:0/3564823250 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5609ea105dc0 0x5609ea1081b0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d6d9c5298b01605 server_cookie=be80327aaae6dc8 in_seq=0 out_seq=0 2024-09-10T09:06:39.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.819+0000 7f374a7fc640 1 -- 172.21.15.172:0/3564823250 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609ea05e860 con 0x5609ea105dc0 2024-09-10T09:06:39.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.819+0000 7f374a7fc640 10 client.20400 handle_command_reply: tid=0 2024-09-10T09:06:39.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.819+0000 7f376baf5740 1 client.20400 shutdown 2024-09-10T09:06:39.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.819+0000 7f376baf5740 1 -- 172.21.15.172:0/3564823250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0ecb40 msgr2=0x5609ea0f2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:39.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.819+0000 7f376baf5740 1 --2- 172.21.15.172:0/3564823250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0ecb40 0x5609ea0f2fe0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f374c005940 tx=0x7f374c03c6f0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.819+0000 7f376baf5740 1 -- 172.21.15.172:0/3564823250 shutdown_connections 2024-09-10T09:06:39.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.819+0000 7f376baf5740 1 --2- 172.21.15.172:0/3564823250 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5609ea105dc0 0x5609ea1081b0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.819+0000 7f374a7fc640 0 client.20400 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:06:39.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.819+0000 7f376baf5740 1 --2- 172.21.15.172:0/3564823250 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0f6370 0x5609ea0f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.819+0000 7f376baf5740 1 --2- 172.21.15.172:0/3564823250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0ecb40 0x5609ea0f2fe0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.819+0000 7f376baf5740 1 --2- 172.21.15.172:0/3564823250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0eb5d0 0x5609ea0f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.819+0000 7f376baf5740 1 -- 172.21.15.172:0/3564823250 >> 172.21.15.172:0/3564823250 conn(0x5609ea0b4310 msgr2=0x5609ea0ed960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:39.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.820+0000 7f376baf5740 1 -- 172.21.15.172:0/3564823250 shutdown_connections 2024-09-10T09:06:39.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.820+0000 7f376baf5740 1 -- 172.21.15.172:0/3564823250 wait complete. 2024-09-10T09:06:39.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.820+0000 7f376baf5740 20 client.20400 trim_cache size 0 max 16384 2024-09-10T09:06:39.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.832+0000 7f376baf5740 1 Processor -- start 2024-09-10T09:06:39.833 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.832+0000 7f376baf5740 1 -- start start 2024-09-10T09:06:39.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f376baf5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0eb5d0 0x5609ea1022b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f376baf5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0ecb40 0x5609ea0f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f376baf5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0f6370 0x5609ea0f7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f376baf5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f37641aebe0 con 0x5609ea0eb5d0 2024-09-10T09:06:39.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f376baf5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37641b07e0 con 0x5609ea0ecb40 2024-09-10T09:06:39.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f376baf5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f37641af2b0 con 0x5609ea0f6370 2024-09-10T09:06:39.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f3762ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0eb5d0 0x5609ea1022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:39.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f3762ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0eb5d0 0x5609ea1022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43820/0 (socket says 172.21.15.172:43820) 2024-09-10T09:06:39.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f3762ffd640 1 -- 172.21.15.172:0/204780248 learned_addr learned my addr 172.21.15.172:0/204780248 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:39.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f375bfff640 1 --2- 172.21.15.172:0/204780248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0ecb40 0x5609ea0f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:39.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f3762ffd640 1 -- 172.21.15.172:0/204780248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0f6370 msgr2=0x5609ea0f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:39.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f37637fe640 1 --2- 172.21.15.172:0/204780248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0f6370 0x5609ea0f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:39.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f3762ffd640 1 --2- 172.21.15.172:0/204780248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0f6370 0x5609ea0f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f3762ffd640 1 -- 172.21.15.172:0/204780248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0ecb40 msgr2=0x5609ea0f28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:39.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f3762ffd640 1 --2- 172.21.15.172:0/204780248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0ecb40 0x5609ea0f28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f3762ffd640 1 -- 172.21.15.172:0/204780248 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609ea0ee120 con 0x5609ea0eb5d0 2024-09-10T09:06:39.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f37637fe640 1 --2- 172.21.15.172:0/204780248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0f6370 0x5609ea0f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:06:39.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f375bfff640 1 --2- 172.21.15.172:0/204780248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0ecb40 0x5609ea0f28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:39.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f3762ffd640 1 --2- 172.21.15.172:0/204780248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0eb5d0 0x5609ea1022b0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f3754004810 tx=0x7f37540bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb47fb89d88a591 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:39.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f3758ff9640 1 -- 172.21.15.172:0/204780248 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375401e040 con 0x5609ea0eb5d0 2024-09-10T09:06:39.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.833+0000 7f3758ff9640 1 -- 172.21.15.172:0/204780248 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f375409b030 con 0x5609ea0eb5d0 2024-09-10T09:06:39.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.834+0000 7f3758ff9640 1 -- 172.21.15.172:0/204780248 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754017e40 con 0x5609ea0eb5d0 2024-09-10T09:06:39.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.834+0000 7f376baf5740 1 -- 172.21.15.172:0/204780248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0eb5d0 msgr2=0x5609ea1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:39.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.834+0000 7f376baf5740 1 --2- 172.21.15.172:0/204780248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0eb5d0 0x5609ea1022b0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f3754004810 tx=0x7f37540bb040 comp rx=0 tx=0).stop 2024-09-10T09:06:39.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.834+0000 7f376baf5740 1 -- 172.21.15.172:0/204780248 shutdown_connections 2024-09-10T09:06:39.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.834+0000 7f376baf5740 1 --2- 172.21.15.172:0/204780248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0f6370 0x5609ea0f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.834+0000 7f376baf5740 1 --2- 172.21.15.172:0/204780248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0ecb40 0x5609ea0f28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.834+0000 7f376baf5740 1 --2- 172.21.15.172:0/204780248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0eb5d0 0x5609ea1022b0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.834+0000 7f376baf5740 1 -- 172.21.15.172:0/204780248 >> 172.21.15.172:0/204780248 conn(0x5609ea0b4310 msgr2=0x5609ea0f18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:39.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.834+0000 7f376baf5740 1 -- 172.21.15.172:0/204780248 shutdown_connections 2024-09-10T09:06:39.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.834+0000 7f376baf5740 1 -- 172.21.15.172:0/204780248 wait complete. 2024-09-10T09:06:39.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f376baf5740 1 -- start start 2024-09-10T09:06:39.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f376baf5740 1 Processor -- start 2024-09-10T09:06:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f376baf5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0eb5d0 0x5609ea0fe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f376baf5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0ecb40 0x5609ea0fe970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f376baf5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0f6370 0x5609ea0f93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f376baf5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f37641b00e0 con 0x5609ea0ecb40 2024-09-10T09:06:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f376baf5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37641af9b0 con 0x5609ea0eb5d0 2024-09-10T09:06:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f376baf5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f37641acb60 con 0x5609ea0f6370 2024-09-10T09:06:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f376baf5740 10 client.0 _ll_register_callbacks cb 0x7f3754097100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:06:39.841 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f375bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0ecb40 0x5609ea0fe970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:39.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f375bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0ecb40 0x5609ea0fe970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43822/0 (socket says 172.21.15.172:43822) 2024-09-10T09:06:39.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f375bfff640 1 -- 172.21.15.172:0/1483362098 learned_addr learned my addr 172.21.15.172:0/1483362098 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:39.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f3762ffd640 1 --2- 172.21.15.172:0/1483362098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0eb5d0 0x5609ea0fe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:39.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f37637fe640 1 --2- 172.21.15.172:0/1483362098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0f6370 0x5609ea0f93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:39.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f375bfff640 1 -- 172.21.15.172:0/1483362098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0f6370 msgr2=0x5609ea0f93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:39.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f375bfff640 1 --2- 172.21.15.172:0/1483362098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0f6370 0x5609ea0f93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f375bfff640 1 -- 172.21.15.172:0/1483362098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0eb5d0 msgr2=0x5609ea0fe430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:06:39.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f375bfff640 1 --2- 172.21.15.172:0/1483362098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0eb5d0 0x5609ea0fe430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f375bfff640 1 -- 172.21.15.172:0/1483362098 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609ea0f9920 con 0x5609ea0ecb40 2024-09-10T09:06:39.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f37637fe640 1 --2- 172.21.15.172:0/1483362098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0f6370 0x5609ea0f93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:39.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f375bfff640 1 --2- 172.21.15.172:0/1483362098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0ecb40 0x5609ea0fe970 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f374c003bb0 tx=0x7f374c032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c597c590c7fb6870 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:39.842 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.835+0000 7f374a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-10T09:06:39.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.836+0000 7f374a7fc640 1 -- 172.21.15.172:0/1483362098 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c05c070 con 0x5609ea0ecb40 2024-09-10T09:06:39.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.836+0000 7f374a7fc640 1 -- 172.21.15.172:0/1483362098 <== mon.0 v2:172.21.15.172:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f374c031760 con 0x5609ea0ecb40 2024-09-10T09:06:39.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.836+0000 7f376baf5740 1 -- 172.21.15.172:0/1483362098 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f375409b030 con 0x5609ea0ecb40 2024-09-10T09:06:39.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.836+0000 7f374a7fc640 1 -- 172.21.15.172:0/1483362098 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f374c068050 con 0x5609ea0ecb40 2024-09-10T09:06:39.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.836+0000 7f376baf5740 10 client.20406 fetch_fsmap learned FSMap version 49 2024-09-10T09:06:39.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.836+0000 7f376baf5740 1 -- 172.21.15.172:0/1483362098 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5609ea170d80 con 0x5609ea0ecb40 2024-09-10T09:06:39.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.836+0000 7f374a7fc640 1 -- 172.21.15.172:0/1483362098 <== mon.0 v2:172.21.15.172:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f375409b030 con 0x5609ea0ecb40 2024-09-10T09:06:39.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.836+0000 7f374a7fc640 1 -- 172.21.15.172:0/1483362098 <== mon.0 v2:172.21.15.172:3300/0 5 ==== fsmap(e 49) ==== 8398+0+0 (secure 0 0 0) 0x7f374c03cad0 con 0x5609ea0ecb40 2024-09-10T09:06:39.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.836+0000 7f376baf5740 10 client.20406 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T09:06:39.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.836+0000 7f376baf5740 10 client.20406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:06:39.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.836+0000 7f376baf5740 20 client.20406 populate_metadata read hostname 'smithi172' 2024-09-10T09:06:39.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.836+0000 7f376baf5740 1 --2- 172.21.15.172:0/1483362098 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5609ea105dc0 0x5609ea173480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:39.843 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.836+0000 7f376baf5740 4 client.20406 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:06:39.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.836+0000 7f376baf5740 1 -- 172.21.15.172:0/1483362098 --> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609ea058d70 con 0x5609ea105dc0 2024-09-10T09:06:39.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.837+0000 7f3762ffd640 1 --2- 172.21.15.172:0/1483362098 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5609ea105dc0 0x5609ea173480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:39.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.837+0000 7f3762ffd640 1 --2- 172.21.15.172:0/1483362098 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5609ea105dc0 0x5609ea173480 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af55980b7b8e96dd server_cookie=c460a5083866263c in_seq=0 out_seq=0 2024-09-10T09:06:39.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.837+0000 7f374a7fc640 10 client.20406 ms_handle_connect on v2:172.21.15.174:6838/3029849701 2024-09-10T09:06:39.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.837+0000 7f374a7fc640 1 -- 172.21.15.172:0/1483362098 <== mds.0 v2:172.21.15.174:6838/3029849701 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609ea058d70 con 0x5609ea105dc0 2024-09-10T09:06:39.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.837+0000 7f374a7fc640 10 client.20406 handle_command_reply: tid=0 2024-09-10T09:06:39.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.837+0000 7f376baf5740 1 client.20406 shutdown 2024-09-10T09:06:39.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.837+0000 7f376baf5740 1 -- 172.21.15.172:0/1483362098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0ecb40 msgr2=0x5609ea0fe970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:39.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.837+0000 7f376baf5740 1 --2- 172.21.15.172:0/1483362098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0ecb40 0x5609ea0fe970 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f374c003bb0 tx=0x7f374c032fb0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.837+0000 7f376baf5740 1 -- 172.21.15.172:0/1483362098 shutdown_connections 2024-09-10T09:06:39.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.837+0000 7f376baf5740 1 --2- 172.21.15.172:0/1483362098 >> [v2:172.21.15.174:6838/3029849701,v1:172.21.15.174:6839/3029849701] conn(0x5609ea105dc0 0x5609ea173480 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.837+0000 7f374a7fc640 0 client.20406 ms_handle_reset on v2:172.21.15.174:6838/3029849701 2024-09-10T09:06:39.844 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.837+0000 7f376baf5740 1 --2- 172.21.15.172:0/1483362098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609ea0f6370 0x5609ea0f93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.837+0000 7f376baf5740 1 --2- 172.21.15.172:0/1483362098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5609ea0ecb40 0x5609ea0fe970 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.837+0000 7f376baf5740 1 --2- 172.21.15.172:0/1483362098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5609ea0eb5d0 0x5609ea0fe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.837+0000 7f376baf5740 1 -- 172.21.15.172:0/1483362098 >> 172.21.15.172:0/1483362098 conn(0x5609ea0b4310 msgr2=0x5609ea0f5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:39.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.837+0000 7f376baf5740 1 -- 172.21.15.172:0/1483362098 shutdown_connections 2024-09-10T09:06:39.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.838+0000 7f376baf5740 1 -- 172.21.15.172:0/1483362098 wait complete. 2024-09-10T09:06:39.845 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.838+0000 7f376baf5740 20 client.20406 trim_cache size 0 max 16384 2024-09-10T09:06:39.845 INFO:teuthology.orchestra.run.smithi172.stdout:[] 2024-09-10T09:06:39.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.838+0000 7f376971c640 1 -- 172.21.15.172:0/1298006923 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f37500a2b80 msgr2=0x7f37500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:39.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.838+0000 7f376971c640 1 --2- 172.21.15.172:0/1298006923 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f37500a2b80 0x7f37500a5040 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f375c009fd0 tx=0x7f375c005bd0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.838+0000 7f376971c640 1 -- 172.21.15.172:0/1298006923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3764100a80 msgr2=0x7f3764198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:39.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.838+0000 7f376971c640 1 --2- 172.21.15.172:0/1298006923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3764100a80 0x7f3764198440 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f375400b4a0 tx=0x7f375400b960 comp rx=0 tx=0).stop 2024-09-10T09:06:39.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.838+0000 7f376971c640 1 -- 172.21.15.172:0/1298006923 shutdown_connections 2024-09-10T09:06:39.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.838+0000 7f376971c640 1 --2- 172.21.15.172:0/1298006923 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f37500a2b80 0x7f37500a5040 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.846 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.838+0000 7f376971c640 1 --2- 172.21.15.172:0/1298006923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3764106a80 0x7f376419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.838+0000 7f376971c640 1 --2- 172.21.15.172:0/1298006923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3764101420 0x7f3764198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.838+0000 7f376971c640 1 --2- 172.21.15.172:0/1298006923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3764100a80 0x7f3764198440 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:39.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.838+0000 7f376971c640 1 -- 172.21.15.172:0/1298006923 >> 172.21.15.172:0/1298006923 conn(0x7f37640fc7a0 msgr2=0x7f376406aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:39.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.838+0000 7f376971c640 1 -- 172.21.15.172:0/1298006923 shutdown_connections 2024-09-10T09:06:39.847 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:39.838+0000 7f376971c640 1 -- 172.21.15.172:0/1298006923 wait complete. 2024-09-10T09:06:40.669 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-10T09:06:40.670 INFO:tasks.fwd_scrub:done joining 2024-09-10T09:06:40.671 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-10T09:06:40.682 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:06:40.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.783+0000 7fccf8386640 1 -- 172.21.15.172:0/3506838310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccf0104f20 msgr2=0x7fccf0107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:40.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.783+0000 7fccf8386640 1 --2- 172.21.15.172:0/3506838310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccf0104f20 0x7fccf0107310 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009900 tx=0x7fcce002a410 comp rx=0 tx=0).stop 2024-09-10T09:06:40.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.783+0000 7fccf8386640 1 -- 172.21.15.172:0/3506838310 shutdown_connections 2024-09-10T09:06:40.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.783+0000 7fccf8386640 1 --2- 172.21.15.172:0/3506838310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf0107850 0x7fccf0109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:40.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.783+0000 7fccf8386640 1 --2- 172.21.15.172:0/3506838310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccf0104f20 0x7fccf0107310 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:40.786 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.783+0000 7fccf8386640 1 --2- 172.21.15.172:0/3506838310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccf01025f0 0x7fccf01049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:40.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.783+0000 7fccf8386640 1 -- 172.21.15.172:0/3506838310 >> 172.21.15.172:0/3506838310 conn(0x7fccf00fc170 msgr2=0x7fccf00fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:40.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.783+0000 7fccf8386640 1 -- 172.21.15.172:0/3506838310 shutdown_connections 2024-09-10T09:06:40.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.783+0000 7fccf8386640 1 -- 172.21.15.172:0/3506838310 wait complete. 2024-09-10T09:06:40.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.784+0000 7fccf8386640 1 Processor -- start 2024-09-10T09:06:40.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.784+0000 7fccf8386640 1 -- start start 2024-09-10T09:06:40.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.784+0000 7fccf8386640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf01025f0 0x7fccf0197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:40.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.784+0000 7fccf8386640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccf0104f20 0x7fccf0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:40.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.784+0000 7fccf8386640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccf0107850 0x7fccf019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:40.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.784+0000 7fccf8386640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fccf010d670 con 0x7fccf0104f20 2024-09-10T09:06:40.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.784+0000 7fccf8386640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fccf010d4f0 con 0x7fccf0107850 2024-09-10T09:06:40.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.784+0000 7fccf8386640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fccf010d7f0 con 0x7fccf01025f0 2024-09-10T09:06:40.787 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.784+0000 7fccf5921640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccf0104f20 0x7fccf0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:40.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.784+0000 7fccf5921640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccf0104f20 0x7fccf0198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43832/0 (socket says 172.21.15.172:43832) 2024-09-10T09:06:40.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.784+0000 7fccf5921640 1 -- 172.21.15.172:0/3869287235 learned_addr learned my addr 172.21.15.172:0/3869287235 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:40.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.784+0000 7fccf6122640 1 --2- 172.21.15.172:0/3869287235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf01025f0 0x7fccf0197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:40.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.784+0000 7fccf6923640 1 --2- 172.21.15.172:0/3869287235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccf0107850 0x7fccf019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:40.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.785+0000 7fccf5921640 1 -- 172.21.15.172:0/3869287235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf01025f0 msgr2=0x7fccf0197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:40.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.785+0000 7fccf5921640 1 --2- 172.21.15.172:0/3869287235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf01025f0 0x7fccf0197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:40.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.785+0000 7fccf5921640 1 -- 172.21.15.172:0/3869287235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccf0107850 msgr2=0x7fccf019f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:40.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.785+0000 7fccf5921640 1 --2- 172.21.15.172:0/3869287235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccf0107850 0x7fccf019f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:40.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.785+0000 7fccf5921640 1 -- 172.21.15.172:0/3869287235 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf019fa10 con 0x7fccf0104f20 2024-09-10T09:06:40.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.785+0000 7fccf6923640 1 --2- 172.21.15.172:0/3869287235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccf0107850 0x7fccf019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:40.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.785+0000 7fccf5921640 1 --2- 172.21.15.172:0/3869287235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccf0104f20 0x7fccf0198330 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fcce00314e0 tx=0x7fcce0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:40.788 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.785+0000 7fcce77fe640 1 -- 172.21.15.172:0/3869287235 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce003f030 con 0x7fccf0104f20 2024-09-10T09:06:40.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.785+0000 7fccf8386640 1 -- 172.21.15.172:0/3869287235 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf019fca0 con 0x7fccf0104f20 2024-09-10T09:06:40.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.785+0000 7fcce77fe640 1 -- 172.21.15.172:0/3869287235 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcce0003aa0 con 0x7fccf0104f20 2024-09-10T09:06:40.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.785+0000 7fcce77fe640 1 -- 172.21.15.172:0/3869287235 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0035070 con 0x7fccf0104f20 2024-09-10T09:06:40.789 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.785+0000 7fccf8386640 1 -- 172.21.15.172:0/3869287235 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf01a0190 con 0x7fccf0104f20 2024-09-10T09:06:40.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.786+0000 7fccf8386640 1 -- 172.21.15.172:0/3869287235 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf00fdc60 con 0x7fccf0104f20 2024-09-10T09:06:40.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.787+0000 7fcce77fe640 1 -- 172.21.15.172:0/3869287235 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fcce0042030 con 0x7fccf0104f20 2024-09-10T09:06:40.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.788+0000 7fcce77fe640 1 --2- 172.21.15.172:0/3869287235 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccdc0a2b90 0x7fccdc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:40.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.788+0000 7fccf6122640 1 --2- 172.21.15.172:0/3869287235 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccdc0a2b90 0x7fccdc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:40.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.788+0000 7fcce77fe640 1 -- 172.21.15.172:0/3869287235 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcce00efec0 con 0x7fccf0104f20 2024-09-10T09:06:40.790 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.788+0000 7fccf6122640 1 --2- 172.21.15.172:0/3869287235 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccdc0a2b90 0x7fccdc0a5050 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fccec009bd0 tx=0x7fccec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:40.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.791+0000 7fcce77fe640 1 -- 172.21.15.172:0/3869287235 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcce00bcac0 con 0x7fccf0104f20 2024-09-10T09:06:40.960 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.959+0000 7fccf8386640 1 -- 172.21.15.172:0/3869287235 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fccf002d9c0 con 0x7fccf0104f20 2024-09-10T09:06:40.961 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.960+0000 7fcce77fe640 1 -- 172.21.15.172:0/3869287235 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24778 (secure 0 0 0) 0x7fcce00bc3c0 con 0x7fccf0104f20 2024-09-10T09:06:40.962 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T09:06:40.962 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":92,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","created":"2024-09-10T08:36:33.083529+0000","modified":"2024-09-10T08:46:18.799844+0000","last_up_change":"2024-09-10T08:43:19.615640+0000","last_in_change":"2024-09-10T08:42:56.656115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:39:31.328306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:44:07.466637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:44:09.520569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:44:15.926057+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6803","nonce":2678426952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6805","nonce":2678426952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6809","nonce":2678426952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6807","nonce":2678426952}]},"public_addr":"172.21.15.172:6803/2678426952","cluster_addr":"172.21.15.172:6805/2678426952","heartbeat_back_addr":"172.21.15.172:6809/2678426952","heartbeat_front_addr":"172.21.15.172:6807/2678426952","state":["exists","up"]},{"osd":1,"uuid":"55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6801","nonce":3279661439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6803","nonce":3279661439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6807","nonce":3279661439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6805","nonce":3279661439}]},"public_addr":"172.21.15.174:6801/3279661439","cluster_addr":"172.21.15.174:6803/3279661439","heartbeat_back_addr":"172.21.15.174:6807/3279661439","heartbeat_front_addr":"172.21.15.174:6805/3279661439","state":["exists","up"]},{"osd":2,"uuid":"f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6801","nonce":1963796290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6803","nonce":1963796290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6807","nonce":1963796290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6805","nonce":1963796290}]},"public_addr":"172.21.15.191:6801/1963796290","cluster_addr":"172.21.15.191:6803/1963796290","heartbeat_back_addr":"172.21.15.191:6807/1963796290","heartbeat_front_addr":"172.21.15.191:6805/1963796290","state":["exists","up"]},{"osd":3,"uuid":"15ecca3f-bc6b-4a06-b283-b515384beb19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6811","nonce":869546480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6813","nonce":869546480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6817","nonce":869546480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6815","nonce":869546480}]},"public_addr":"172.21.15.172:6811/869546480","cluster_addr":"172.21.15.172:6813/869546480","heartbeat_back_addr":"172.21.15.172:6817/869546480","heartbeat_front_addr":"172.21.15.172:6815/869546480","state":["exists","up"]},{"osd":4,"uuid":"8c2c1d40-72ad-4a15-ab72-54871c7e2427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6809","nonce":961007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6811","nonce":961007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6815","nonce":961007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6813","nonce":961007949}]},"public_addr":"172.21.15.174:6809/961007949","cluster_addr":"172.21.15.174:6811/961007949","heartbeat_back_addr":"172.21.15.174:6815/961007949","heartbeat_front_addr":"172.21.15.174:6813/961007949","state":["exists","up"]},{"osd":5,"uuid":"4de45a1d-ef0e-4bfe-ba15-5eeeb8806155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6809","nonce":4216938371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6811","nonce":4216938371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6815","nonce":4216938371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6813","nonce":4216938371}]},"public_addr":"172.21.15.191:6809/4216938371","cluster_addr":"172.21.15.191:6811/4216938371","heartbeat_back_addr":"172.21.15.191:6815/4216938371","heartbeat_front_addr":"172.21.15.191:6813/4216938371","state":["exists","up"]},{"osd":6,"uuid":"60a9938a-f696-4db2-932c-402997543b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6819","nonce":3210762009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6821","nonce":3210762009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6825","nonce":3210762009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6823","nonce":3210762009}]},"public_addr":"172.21.15.172:6819/3210762009","cluster_addr":"172.21.15.172:6821/3210762009","heartbeat_back_addr":"172.21.15.172:6825/3210762009","heartbeat_front_addr":"172.21.15.172:6823/3210762009","state":["exists","up"]},{"osd":7,"uuid":"7f0202f3-971e-472f-9c4d-ae3c24d39f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6817","nonce":1461923581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6819","nonce":1461923581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6823","nonce":1461923581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6821","nonce":1461923581}]},"public_addr":"172.21.15.174:6817/1461923581","cluster_addr":"172.21.15.174:6819/1461923581","heartbeat_back_addr":"172.21.15.174:6823/1461923581","heartbeat_front_addr":"172.21.15.174:6821/1461923581","state":["exists","up"]},{"osd":8,"uuid":"b4e7327f-ca9f-4833-9cd3-b8529b3d9761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6817","nonce":3675210430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6819","nonce":3675210430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6823","nonce":3675210430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6821","nonce":3675210430}]},"public_addr":"172.21.15.191:6817/3675210430","cluster_addr":"172.21.15.191:6819/3675210430","heartbeat_back_addr":"172.21.15.191:6823/3675210430","heartbeat_front_addr":"172.21.15.191:6821/3675210430","state":["exists","up"]},{"osd":9,"uuid":"9f6153f2-f9db-4c00-a659-c932628016f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6827","nonce":1181268619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6829","nonce":1181268619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6833","nonce":1181268619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6831","nonce":1181268619}]},"public_addr":"172.21.15.172:6827/1181268619","cluster_addr":"172.21.15.172:6829/1181268619","heartbeat_back_addr":"172.21.15.172:6833/1181268619","heartbeat_front_addr":"172.21.15.172:6831/1181268619","state":["exists","up"]},{"osd":10,"uuid":"040a013a-4c3e-442f-bea6-12cbe517716d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6825","nonce":1238812525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6827","nonce":1238812525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6831","nonce":1238812525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6829","nonce":1238812525}]},"public_addr":"172.21.15.174:6825/1238812525","cluster_addr":"172.21.15.174:6827/1238812525","heartbeat_back_addr":"172.21.15.174:6831/1238812525","heartbeat_front_addr":"172.21.15.174:6829/1238812525","state":["exists","up"]},{"osd":11,"uuid":"ae6535bd-1510-4bf3-bf4b-1a2a256add43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6825","nonce":2162719997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6827","nonce":2162719997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6831","nonce":2162719997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6829","nonce":2162719997}]},"public_addr":"172.21.15.191:6825/2162719997","cluster_addr":"172.21.15.191:6827/2162719997","heartbeat_back_addr":"172.21.15.191:6831/2162719997","heartbeat_front_addr":"172.21.15.191:6829/2162719997","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:38:38.812596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:04.765652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:29.103966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:56.085849+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:21.641784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:47.750376+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:11.640697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:36.638840+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:02.167446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:28.655804+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:52.749676+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:43:17.783908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:0/977245917":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/3919952878":"2024-09-11T08:36:54.887256+0000","172.21.15.172:6800/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/1525385361":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/2043423408":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/839344299":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6800/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/3251817286":"2024-09-11T08:36:54.887256+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:06:40.963 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.962+0000 7fccf8386640 1 -- 172.21.15.172:0/3869287235 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccdc0a2b90 msgr2=0x7fccdc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:40.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.963+0000 7fccf8386640 1 --2- 172.21.15.172:0/3869287235 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccdc0a2b90 0x7fccdc0a5050 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fccec009bd0 tx=0x7fccec005f90 comp rx=0 tx=0).stop 2024-09-10T09:06:40.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.963+0000 7fccf8386640 1 -- 172.21.15.172:0/3869287235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccf0104f20 msgr2=0x7fccf0198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:40.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.963+0000 7fccf8386640 1 --2- 172.21.15.172:0/3869287235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccf0104f20 0x7fccf0198330 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fcce00314e0 tx=0x7fcce0004270 comp rx=0 tx=0).stop 2024-09-10T09:06:40.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.963+0000 7fccf8386640 1 -- 172.21.15.172:0/3869287235 shutdown_connections 2024-09-10T09:06:40.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.963+0000 7fccf8386640 1 --2- 172.21.15.172:0/3869287235 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fccdc0a2b90 0x7fccdc0a5050 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:40.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.963+0000 7fccf8386640 1 --2- 172.21.15.172:0/3869287235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccf0107850 0x7fccf019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:40.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.963+0000 7fccf8386640 1 --2- 172.21.15.172:0/3869287235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fccf0104f20 0x7fccf0198330 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:40.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.963+0000 7fccf8386640 1 --2- 172.21.15.172:0/3869287235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf01025f0 0x7fccf0197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:40.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.963+0000 7fccf8386640 1 -- 172.21.15.172:0/3869287235 >> 172.21.15.172:0/3869287235 conn(0x7fccf00fc170 msgr2=0x7fccf0102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:40.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.963+0000 7fccf8386640 1 -- 172.21.15.172:0/3869287235 shutdown_connections 2024-09-10T09:06:40.964 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:40.963+0000 7fccf8386640 1 -- 172.21.15.172:0/3869287235 wait complete. 2024-09-10T09:06:40.973 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:40 smithi172 ceph-mon[30854]: pgmap v963: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T09:06:40.975 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T08:39:31.328306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T08:44:07.466637+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T08:44:09.520569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T08:44:15.926057+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T09:06:40.976 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T09:06:41.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.082+0000 7f08f127b640 1 -- 172.21.15.172:0/1976804298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08ec100a10 msgr2=0x7f08ec100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.082+0000 7f08f127b640 1 --2- 172.21.15.172:0/1976804298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08ec100a10 0x7f08ec100e10 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009900 tx=0x7f08dc02a410 comp rx=0 tx=0).stop 2024-09-10T09:06:41.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.082+0000 7f08f127b640 1 -- 172.21.15.172:0/1976804298 shutdown_connections 2024-09-10T09:06:41.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.082+0000 7f08f127b640 1 --2- 172.21.15.172:0/1976804298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08ec1025c0 0x7f08ec108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.082+0000 7f08f127b640 1 --2- 172.21.15.172:0/1976804298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec101c00 0x7f08ec102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.082+0000 7f08f127b640 1 --2- 172.21.15.172:0/1976804298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08ec100a10 0x7f08ec100e10 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.082+0000 7f08f127b640 1 -- 172.21.15.172:0/1976804298 >> 172.21.15.172:0/1976804298 conn(0x7f08ec0fc1b0 msgr2=0x7f08ec0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:41.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.082+0000 7f08f127b640 1 -- 172.21.15.172:0/1976804298 shutdown_connections 2024-09-10T09:06:41.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.082+0000 7f08f127b640 1 -- 172.21.15.172:0/1976804298 wait complete. 2024-09-10T09:06:41.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.082+0000 7f08f127b640 1 Processor -- start 2024-09-10T09:06:41.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.082+0000 7f08f127b640 1 -- start start 2024-09-10T09:06:41.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08f127b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec100a10 0x7f08ec196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:41.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08f127b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08ec101c00 0x7f08ec197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:41.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08f127b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08ec1025c0 0x7f08ec19e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:41.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08f127b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec10c540 con 0x7f08ec1025c0 2024-09-10T09:06:41.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08f127b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec10c3c0 con 0x7f08ec101c00 2024-09-10T09:06:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08f127b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec10c6c0 con 0x7f08ec100a10 2024-09-10T09:06:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08eb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08ec1025c0 0x7f08ec19e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08eb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08ec1025c0 0x7f08ec19e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43846/0 (socket says 172.21.15.172:43846) 2024-09-10T09:06:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08eb7fe640 1 -- 172.21.15.172:0/2089897897 learned_addr learned my addr 172.21.15.172:0/2089897897 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08eaffd640 1 --2- 172.21.15.172:0/2089897897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec100a10 0x7f08ec196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08ea7fc640 1 --2- 172.21.15.172:0/2089897897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08ec101c00 0x7f08ec197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08eb7fe640 1 -- 172.21.15.172:0/2089897897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec100a10 msgr2=0x7f08ec196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08eb7fe640 1 --2- 172.21.15.172:0/2089897897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec100a10 0x7f08ec196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08eb7fe640 1 -- 172.21.15.172:0/2089897897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08ec101c00 msgr2=0x7f08ec197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08eb7fe640 1 --2- 172.21.15.172:0/2089897897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08ec101c00 0x7f08ec197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08eb7fe640 1 -- 172.21.15.172:0/2089897897 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ec19e940 con 0x7f08ec1025c0 2024-09-10T09:06:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08eb7fe640 1 --2- 172.21.15.172:0/2089897897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08ec1025c0 0x7f08ec19e2a0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f08e400cad0 tx=0x7f08e400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:41.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08d3fff640 1 -- 172.21.15.172:0/2089897897 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e400ae20 con 0x7f08ec1025c0 2024-09-10T09:06:41.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08d3fff640 1 -- 172.21.15.172:0/2089897897 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08e4005020 con 0x7f08ec1025c0 2024-09-10T09:06:41.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08f127b640 1 -- 172.21.15.172:0/2089897897 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08ec19ec30 con 0x7f08ec1025c0 2024-09-10T09:06:41.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.083+0000 7f08d3fff640 1 -- 172.21.15.172:0/2089897897 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e400f6b0 con 0x7f08ec1025c0 2024-09-10T09:06:41.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.084+0000 7f08f127b640 1 -- 172.21.15.172:0/2089897897 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08ec19f130 con 0x7f08ec1025c0 2024-09-10T09:06:41.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.084+0000 7f08f127b640 1 -- 172.21.15.172:0/2089897897 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08ec107f30 con 0x7f08ec1025c0 2024-09-10T09:06:41.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.086+0000 7f08d3fff640 1 -- 172.21.15.172:0/2089897897 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f08e40051d0 con 0x7f08ec1025c0 2024-09-10T09:06:41.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.086+0000 7f08d3fff640 1 --2- 172.21.15.172:0/2089897897 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08d40a2b90 0x7f08d40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:41.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.086+0000 7f08eaffd640 1 --2- 172.21.15.172:0/2089897897 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08d40a2b90 0x7f08d40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:41.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.086+0000 7f08d3fff640 1 -- 172.21.15.172:0/2089897897 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f08e40cf8e0 con 0x7f08ec1025c0 2024-09-10T09:06:41.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.087+0000 7f08eaffd640 1 --2- 172.21.15.172:0/2089897897 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08d40a2b90 0x7f08d40a5050 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f08ec198200 tx=0x7f08dc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:41.090 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.089+0000 7f08d3fff640 1 -- 172.21.15.172:0/2089897897 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f08e4014030 con 0x7f08ec1025c0 2024-09-10T09:06:41.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:40 smithi174 ceph-mon[34068]: pgmap v963: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T09:06:41.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.259+0000 7f08f127b640 1 -- 172.21.15.172:0/2089897897 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f08ec02da10 con 0x7f08ec1025c0 2024-09-10T09:06:41.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.259+0000 7f08d3fff640 1 -- 172.21.15.172:0/2089897897 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f08e409c480 con 0x7f08ec1025c0 2024-09-10T09:06:41.260 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 1 2024-09-10T09:06:41.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.261+0000 7f08f127b640 1 -- 172.21.15.172:0/2089897897 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08d40a2b90 msgr2=0x7f08d40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.261+0000 7f08f127b640 1 --2- 172.21.15.172:0/2089897897 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08d40a2b90 0x7f08d40a5050 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f08ec198200 tx=0x7f08dc000bc0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.261+0000 7f08f127b640 1 -- 172.21.15.172:0/2089897897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08ec1025c0 msgr2=0x7f08ec19e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.261+0000 7f08f127b640 1 --2- 172.21.15.172:0/2089897897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08ec1025c0 0x7f08ec19e2a0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f08e400cad0 tx=0x7f08e400cf90 comp rx=0 tx=0).stop 2024-09-10T09:06:41.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.261+0000 7f08f127b640 1 -- 172.21.15.172:0/2089897897 shutdown_connections 2024-09-10T09:06:41.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.261+0000 7f08f127b640 1 --2- 172.21.15.172:0/2089897897 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f08d40a2b90 0x7f08d40a5050 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.261+0000 7f08f127b640 1 --2- 172.21.15.172:0/2089897897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08ec1025c0 0x7f08ec19e2a0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.261+0000 7f08f127b640 1 --2- 172.21.15.172:0/2089897897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08ec101c00 0x7f08ec197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.261+0000 7f08f127b640 1 --2- 172.21.15.172:0/2089897897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec100a10 0x7f08ec196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.261+0000 7f08f127b640 1 -- 172.21.15.172:0/2089897897 >> 172.21.15.172:0/2089897897 conn(0x7f08ec0fc1b0 msgr2=0x7f08ec105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:41.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.262+0000 7f08f127b640 1 -- 172.21.15.172:0/2089897897 shutdown_connections 2024-09-10T09:06:41.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.262+0000 7f08f127b640 1 -- 172.21.15.172:0/2089897897 wait complete. 2024-09-10T09:06:41.273 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T09:06:41.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:40 smithi191 ceph-mon[35314]: pgmap v963: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T09:06:41.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.375+0000 7f127f0b7640 1 -- 172.21.15.172:0/1920610431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1278104ec0 msgr2=0x7f12781072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.375+0000 7f127f0b7640 1 --2- 172.21.15.172:0/1920610431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1278104ec0 0x7f12781072b0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f1268009900 tx=0x7f126802a410 comp rx=0 tx=0).stop 2024-09-10T09:06:41.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.375+0000 7f127f0b7640 1 -- 172.21.15.172:0/1920610431 shutdown_connections 2024-09-10T09:06:41.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.375+0000 7f127f0b7640 1 --2- 172.21.15.172:0/1920610431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12781077f0 0x7f1278109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.379 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.375+0000 7f127f0b7640 1 --2- 172.21.15.172:0/1920610431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1278104ec0 0x7f12781072b0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.375+0000 7f127f0b7640 1 --2- 172.21.15.172:0/1920610431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1278102590 0x7f1278104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.375+0000 7f127f0b7640 1 -- 172.21.15.172:0/1920610431 >> 172.21.15.172:0/1920610431 conn(0x7f12780fc1b0 msgr2=0x7f12780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:41.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.375+0000 7f127f0b7640 1 -- 172.21.15.172:0/1920610431 shutdown_connections 2024-09-10T09:06:41.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.375+0000 7f127f0b7640 1 -- 172.21.15.172:0/1920610431 wait complete. 2024-09-10T09:06:41.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.376+0000 7f127f0b7640 1 Processor -- start 2024-09-10T09:06:41.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.376+0000 7f127f0b7640 1 -- start start 2024-09-10T09:06:41.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.376+0000 7f127f0b7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1278102590 0x7f1278197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:41.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.376+0000 7f127f0b7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1278104ec0 0x7f1278198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:41.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.376+0000 7f127f0b7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12781077f0 0x7f127819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:41.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.376+0000 7f127f0b7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f127810d670 con 0x7f1278102590 2024-09-10T09:06:41.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.376+0000 7f127f0b7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f127810d4f0 con 0x7f12781077f0 2024-09-10T09:06:41.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.376+0000 7f127f0b7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f127810d7f0 con 0x7f1278104ec0 2024-09-10T09:06:41.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.376+0000 7f127ce53640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1278102590 0x7f1278197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:41.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.376+0000 7f127ce53640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1278102590 0x7f1278197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43864/0 (socket says 172.21.15.172:43864) 2024-09-10T09:06:41.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.376+0000 7f127ce53640 1 -- 172.21.15.172:0/2657853287 learned_addr learned my addr 172.21.15.172:0/2657853287 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:41.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.376+0000 7f126ffff640 1 --2- 172.21.15.172:0/2657853287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1278104ec0 0x7f1278198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:41.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.376+0000 7f127d654640 1 --2- 172.21.15.172:0/2657853287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12781077f0 0x7f127819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:41.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.376+0000 7f127ce53640 1 -- 172.21.15.172:0/2657853287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1278104ec0 msgr2=0x7f1278198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.376+0000 7f127ce53640 1 --2- 172.21.15.172:0/2657853287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1278104ec0 0x7f1278198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.376+0000 7f127ce53640 1 -- 172.21.15.172:0/2657853287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12781077f0 msgr2=0x7f127819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.377+0000 7f127ce53640 1 --2- 172.21.15.172:0/2657853287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12781077f0 0x7f127819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.377+0000 7f127ce53640 1 -- 172.21.15.172:0/2657853287 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127819f9f0 con 0x7f1278102590 2024-09-10T09:06:41.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.377+0000 7f126ffff640 1 --2- 172.21.15.172:0/2657853287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1278104ec0 0x7f1278198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:06:41.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.377+0000 7f127d654640 1 --2- 172.21.15.172:0/2657853287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12781077f0 0x7f127819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:06:41.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.377+0000 7f127ce53640 1 --2- 172.21.15.172:0/2657853287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1278102590 0x7f1278197dd0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f127400d770 tx=0x7f127400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:41.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.377+0000 7f126dffb640 1 -- 172.21.15.172:0/2657853287 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274017070 con 0x7f1278102590 2024-09-10T09:06:41.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.377+0000 7f127f0b7640 1 -- 172.21.15.172:0/2657853287 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f127819fce0 con 0x7f1278102590 2024-09-10T09:06:41.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.377+0000 7f126dffb640 1 -- 172.21.15.172:0/2657853287 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12740054d0 con 0x7f1278102590 2024-09-10T09:06:41.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.377+0000 7f126dffb640 1 -- 172.21.15.172:0/2657853287 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274004cf0 con 0x7f1278102590 2024-09-10T09:06:41.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.377+0000 7f127f0b7640 1 -- 172.21.15.172:0/2657853287 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12781a0180 con 0x7f1278102590 2024-09-10T09:06:41.382 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.378+0000 7f127f0b7640 1 -- 172.21.15.172:0/2657853287 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12780fdca0 con 0x7f1278102590 2024-09-10T09:06:41.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.379+0000 7f126dffb640 1 -- 172.21.15.172:0/2657853287 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1274004e90 con 0x7f1278102590 2024-09-10T09:06:41.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.380+0000 7f126dffb640 1 --2- 172.21.15.172:0/2657853287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12600a2b90 0x7f12600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:41.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.380+0000 7f126dffb640 1 -- 172.21.15.172:0/2657853287 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f12740cef20 con 0x7f1278102590 2024-09-10T09:06:41.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.380+0000 7f126ffff640 1 --2- 172.21.15.172:0/2657853287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12600a2b90 0x7f12600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:41.383 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.380+0000 7f126ffff640 1 --2- 172.21.15.172:0/2657853287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12600a2b90 0x7f12600a5050 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1268006010 tx=0x7f1268000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:41.384 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.384+0000 7f126dffb640 1 -- 172.21.15.172:0/2657853287 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f127409bac0 con 0x7f1278102590 2024-09-10T09:06:41.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.550+0000 7f127f0b7640 1 -- 172.21.15.172:0/2657853287 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f127802d7c0 con 0x7f1278102590 2024-09-10T09:06:41.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.550+0000 7f126dffb640 1 -- 172.21.15.172:0/2657853287 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f127802d7c0 con 0x7f1278102590 2024-09-10T09:06:41.551 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-09-10T09:06:41.553 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.552+0000 7f127f0b7640 1 -- 172.21.15.172:0/2657853287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12600a2b90 msgr2=0x7f12600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.552+0000 7f127f0b7640 1 --2- 172.21.15.172:0/2657853287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12600a2b90 0x7f12600a5050 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1268006010 tx=0x7f1268000bc0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.552+0000 7f127f0b7640 1 -- 172.21.15.172:0/2657853287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1278102590 msgr2=0x7f1278197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.552+0000 7f127f0b7640 1 --2- 172.21.15.172:0/2657853287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1278102590 0x7f1278197dd0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f127400d770 tx=0x7f127400dc30 comp rx=0 tx=0).stop 2024-09-10T09:06:41.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.552+0000 7f127f0b7640 1 -- 172.21.15.172:0/2657853287 shutdown_connections 2024-09-10T09:06:41.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.552+0000 7f127f0b7640 1 --2- 172.21.15.172:0/2657853287 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f12600a2b90 0x7f12600a5050 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.552+0000 7f127f0b7640 1 --2- 172.21.15.172:0/2657853287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12781077f0 0x7f127819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.552+0000 7f127f0b7640 1 --2- 172.21.15.172:0/2657853287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1278104ec0 0x7f1278198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.552+0000 7f127f0b7640 1 --2- 172.21.15.172:0/2657853287 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1278102590 0x7f1278197dd0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.552+0000 7f127f0b7640 1 -- 172.21.15.172:0/2657853287 >> 172.21.15.172:0/2657853287 conn(0x7f12780fc1b0 msgr2=0x7f1278102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:41.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.552+0000 7f127f0b7640 1 -- 172.21.15.172:0/2657853287 shutdown_connections 2024-09-10T09:06:41.554 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.552+0000 7f127f0b7640 1 -- 172.21.15.172:0/2657853287 wait complete. 2024-09-10T09:06:41.565 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T09:06:41.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.672+0000 7fef4b711640 1 -- 172.21.15.172:0/349808987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef44103100 msgr2=0x7fef44103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.672+0000 7fef4b711640 1 --2- 172.21.15.172:0/349808987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef44103100 0x7fef44103500 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fef40009930 tx=0x7fef4002a420 comp rx=0 tx=0).stop 2024-09-10T09:06:41.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.672+0000 7fef4b711640 1 -- 172.21.15.172:0/349808987 shutdown_connections 2024-09-10T09:06:41.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.672+0000 7fef4b711640 1 --2- 172.21.15.172:0/349808987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef44100530 0x7fef4410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.672+0000 7fef4b711640 1 --2- 172.21.15.172:0/349808987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef440ffb70 0x7fef440ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.672+0000 7fef4b711640 1 --2- 172.21.15.172:0/349808987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef44103100 0x7fef44103500 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.675 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.672+0000 7fef4b711640 1 -- 172.21.15.172:0/349808987 >> 172.21.15.172:0/349808987 conn(0x7fef440fb900 msgr2=0x7fef440fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:41.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.672+0000 7fef4b711640 1 -- 172.21.15.172:0/349808987 shutdown_connections 2024-09-10T09:06:41.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.672+0000 7fef4b711640 1 -- 172.21.15.172:0/349808987 wait complete. 2024-09-10T09:06:41.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef4b711640 1 Processor -- start 2024-09-10T09:06:41.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef4b711640 1 -- start start 2024-09-10T09:06:41.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef4b711640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef440ffb70 0x7fef44197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:41.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef4b711640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef44100530 0x7fef441982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:41.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef4b711640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef44103100 0x7fef4419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:41.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef4b711640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fef4410d650 con 0x7fef440ffb70 2024-09-10T09:06:41.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef4b711640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef4410d4d0 con 0x7fef44100530 2024-09-10T09:06:41.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef4b711640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fef4410d7d0 con 0x7fef44103100 2024-09-10T09:06:41.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef494ad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef440ffb70 0x7fef44197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:41.676 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef494ad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef440ffb70 0x7fef44197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43884/0 (socket says 172.21.15.172:43884) 2024-09-10T09:06:41.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef494ad640 1 -- 172.21.15.172:0/36925221 learned_addr learned my addr 172.21.15.172:0/36925221 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:41.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef49cae640 1 --2- 172.21.15.172:0/36925221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef44103100 0x7fef4419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:41.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef48cac640 1 --2- 172.21.15.172:0/36925221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef44100530 0x7fef441982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:41.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef494ad640 1 -- 172.21.15.172:0/36925221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef44103100 msgr2=0x7fef4419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef494ad640 1 --2- 172.21.15.172:0/36925221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef44103100 0x7fef4419f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef494ad640 1 -- 172.21.15.172:0/36925221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef44100530 msgr2=0x7fef441982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef494ad640 1 --2- 172.21.15.172:0/36925221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef44100530 0x7fef441982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef494ad640 1 -- 172.21.15.172:0/36925221 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4419f980 con 0x7fef440ffb70 2024-09-10T09:06:41.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef49cae640 1 --2- 172.21.15.172:0/36925221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef44103100 0x7fef4419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:41.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef494ad640 1 --2- 172.21.15.172:0/36925221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef440ffb70 0x7fef44197d60 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fef400314e0 tx=0x7fef40004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:41.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef3a7fc640 1 -- 172.21.15.172:0/36925221 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4003f030 con 0x7fef440ffb70 2024-09-10T09:06:41.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef3a7fc640 1 -- 172.21.15.172:0/36925221 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef40003cb0 con 0x7fef440ffb70 2024-09-10T09:06:41.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.673+0000 7fef4b711640 1 -- 172.21.15.172:0/36925221 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4419fc10 con 0x7fef440ffb70 2024-09-10T09:06:41.677 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.674+0000 7fef3a7fc640 1 -- 172.21.15.172:0/36925221 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40035070 con 0x7fef440ffb70 2024-09-10T09:06:41.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.674+0000 7fef4b711640 1 -- 172.21.15.172:0/36925221 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef441a0100 con 0x7fef440ffb70 2024-09-10T09:06:41.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.674+0000 7fef4b711640 1 -- 172.21.15.172:0/36925221 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef44109130 con 0x7fef440ffb70 2024-09-10T09:06:41.678 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.676+0000 7fef3a7fc640 1 -- 172.21.15.172:0/36925221 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fef40042030 con 0x7fef440ffb70 2024-09-10T09:06:41.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.677+0000 7fef3a7fc640 1 --2- 172.21.15.172:0/36925221 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fef300a2b90 0x7fef300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:41.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.677+0000 7fef48cac640 1 --2- 172.21.15.172:0/36925221 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fef300a2b90 0x7fef300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:41.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.677+0000 7fef3a7fc640 1 -- 172.21.15.172:0/36925221 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fef400efda0 con 0x7fef440ffb70 2024-09-10T09:06:41.679 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.677+0000 7fef48cac640 1 --2- 172.21.15.172:0/36925221 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fef300a2b90 0x7fef300a5050 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fef44199270 tx=0x7fef3400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:41.681 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.680+0000 7fef3a7fc640 1 -- 172.21.15.172:0/36925221 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fef400bc9a0 con 0x7fef440ffb70 2024-09-10T09:06:41.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.848+0000 7fef4b711640 1 -- 172.21.15.172:0/36925221 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fef4402d990 con 0x7fef440ffb70 2024-09-10T09:06:41.850 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.849+0000 7fef3a7fc640 1 -- 172.21.15.172:0/36925221 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7fef400bc2a0 con 0x7fef440ffb70 2024-09-10T09:06:41.850 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 64 2024-09-10T09:06:41.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.851+0000 7fef4b711640 1 -- 172.21.15.172:0/36925221 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fef300a2b90 msgr2=0x7fef300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.851+0000 7fef4b711640 1 --2- 172.21.15.172:0/36925221 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fef300a2b90 0x7fef300a5050 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fef44199270 tx=0x7fef3400b390 comp rx=0 tx=0).stop 2024-09-10T09:06:41.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.851+0000 7fef4b711640 1 -- 172.21.15.172:0/36925221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef440ffb70 msgr2=0x7fef44197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.851+0000 7fef4b711640 1 --2- 172.21.15.172:0/36925221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef440ffb70 0x7fef44197d60 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fef400314e0 tx=0x7fef40004340 comp rx=0 tx=0).stop 2024-09-10T09:06:41.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.851+0000 7fef4b711640 1 -- 172.21.15.172:0/36925221 shutdown_connections 2024-09-10T09:06:41.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.851+0000 7fef4b711640 1 --2- 172.21.15.172:0/36925221 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fef300a2b90 0x7fef300a5050 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.851+0000 7fef4b711640 1 --2- 172.21.15.172:0/36925221 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef44103100 0x7fef4419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.851+0000 7fef4b711640 1 --2- 172.21.15.172:0/36925221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef44100530 0x7fef441982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.851+0000 7fef4b711640 1 --2- 172.21.15.172:0/36925221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef440ffb70 0x7fef44197d60 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.851+0000 7fef4b711640 1 -- 172.21.15.172:0/36925221 >> 172.21.15.172:0/36925221 conn(0x7fef440fb900 msgr2=0x7fef44106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:41.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.851+0000 7fef4b711640 1 -- 172.21.15.172:0/36925221 shutdown_connections 2024-09-10T09:06:41.853 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.851+0000 7fef4b711640 1 -- 172.21.15.172:0/36925221 wait complete. 2024-09-10T09:06:41.863 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T09:06:41.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.966+0000 7f4895482640 1 -- 172.21.15.172:0/4283990921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48901001a0 msgr2=0x7f48901005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.966+0000 7f4895482640 1 --2- 172.21.15.172:0/4283990921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48901001a0 0x7f48901005a0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f4880009900 tx=0x7f488002a410 comp rx=0 tx=0).stop 2024-09-10T09:06:41.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.966+0000 7f4895482640 1 -- 172.21.15.172:0/4283990921 shutdown_connections 2024-09-10T09:06:41.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.966+0000 7f4895482640 1 --2- 172.21.15.172:0/4283990921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4890101d50 0x7f48901085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.966+0000 7f4895482640 1 --2- 172.21.15.172:0/4283990921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4890101390 0x7f4890101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.970 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.966+0000 7f4895482640 1 --2- 172.21.15.172:0/4283990921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48901001a0 0x7f48901005a0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.966+0000 7f4895482640 1 -- 172.21.15.172:0/4283990921 >> 172.21.15.172:0/4283990921 conn(0x7f48900fb900 msgr2=0x7f48900fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.966+0000 7f4895482640 1 -- 172.21.15.172:0/4283990921 shutdown_connections 2024-09-10T09:06:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.966+0000 7f4895482640 1 -- 172.21.15.172:0/4283990921 wait complete. 2024-09-10T09:06:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f4895482640 1 Processor -- start 2024-09-10T09:06:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f4895482640 1 -- start start 2024-09-10T09:06:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f4895482640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48901001a0 0x7f4890197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f4895482640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4890101390 0x7f4890198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:41.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f4895482640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4890101d50 0x7f489019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f4895482640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f489010b480 con 0x7f4890101d50 2024-09-10T09:06:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f4895482640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f489010b300 con 0x7f4890101390 2024-09-10T09:06:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f4895482640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f489010b600 con 0x7f48901001a0 2024-09-10T09:06:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f488f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4890101d50 0x7f489019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f488f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4890101d50 0x7f489019f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43902/0 (socket says 172.21.15.172:43902) 2024-09-10T09:06:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f488f7fe640 1 -- 172.21.15.172:0/1736717924 learned_addr learned my addr 172.21.15.172:0/1736717924 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f48867fc640 1 --2- 172.21.15.172:0/1736717924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4890101390 0x7f4890198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f488effd640 1 --2- 172.21.15.172:0/1736717924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48901001a0 0x7f4890197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f488f7fe640 1 -- 172.21.15.172:0/1736717924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48901001a0 msgr2=0x7f4890197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f488f7fe640 1 --2- 172.21.15.172:0/1736717924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48901001a0 0x7f4890197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f488f7fe640 1 -- 172.21.15.172:0/1736717924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4890101390 msgr2=0x7f4890198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:41.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f488f7fe640 1 --2- 172.21.15.172:0/1736717924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4890101390 0x7f4890198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.967+0000 7f488f7fe640 1 -- 172.21.15.172:0/1736717924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489019fa10 con 0x7f4890101d50 2024-09-10T09:06:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.968+0000 7f48867fc640 1 --2- 172.21.15.172:0/1736717924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4890101390 0x7f4890198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:06:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.968+0000 7f488f7fe640 1 --2- 172.21.15.172:0/1736717924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4890101d50 0x7f489019f3a0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f488800cad0 tx=0x7f488800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.968+0000 7f488cff9640 1 -- 172.21.15.172:0/1736717924 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488800ae20 con 0x7f4890101d50 2024-09-10T09:06:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.968+0000 7f488cff9640 1 -- 172.21.15.172:0/1736717924 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4888005020 con 0x7f4890101d50 2024-09-10T09:06:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.968+0000 7f4895482640 1 -- 172.21.15.172:0/1736717924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489019fd00 con 0x7f4890101d50 2024-09-10T09:06:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.968+0000 7f488cff9640 1 -- 172.21.15.172:0/1736717924 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488800f6b0 con 0x7f4890101d50 2024-09-10T09:06:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.968+0000 7f4895482640 1 -- 172.21.15.172:0/1736717924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48901a01a0 con 0x7f4890101d50 2024-09-10T09:06:41.973 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.969+0000 7f4895482640 1 -- 172.21.15.172:0/1736717924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4890107620 con 0x7f4890101d50 2024-09-10T09:06:41.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.970+0000 7f488cff9640 1 -- 172.21.15.172:0/1736717924 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f48880051d0 con 0x7f4890101d50 2024-09-10T09:06:41.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.971+0000 7f488cff9640 1 --2- 172.21.15.172:0/1736717924 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f487c0a2b90 0x7f487c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:41.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.971+0000 7f488effd640 1 --2- 172.21.15.172:0/1736717924 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f487c0a2b90 0x7f487c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:41.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.971+0000 7f488cff9640 1 -- 172.21.15.172:0/1736717924 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f48880cf530 con 0x7f4890101d50 2024-09-10T09:06:41.974 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.972+0000 7f488effd640 1 --2- 172.21.15.172:0/1736717924 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f487c0a2b90 0x7f487c0a5050 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4880000c00 tx=0x7f48800314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:41.976 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:41.975+0000 7f488cff9640 1 -- 172.21.15.172:0/1736717924 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4888014030 con 0x7f4890101d50 2024-09-10T09:06:42.143 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.142+0000 7f4895482640 1 -- 172.21.15.172:0/1736717924 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4890062e90 con 0x7f4890101d50 2024-09-10T09:06:42.144 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.143+0000 7f488cff9640 1 -- 172.21.15.172:0/1736717924 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f488809c130 con 0x7f4890101d50 2024-09-10T09:06:42.144 INFO:teuthology.orchestra.run.smithi172.stdout:pg_num: 32 2024-09-10T09:06:42.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.145+0000 7f4895482640 1 -- 172.21.15.172:0/1736717924 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f487c0a2b90 msgr2=0x7f487c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:42.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.145+0000 7f4895482640 1 --2- 172.21.15.172:0/1736717924 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f487c0a2b90 0x7f487c0a5050 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4880000c00 tx=0x7f48800314a0 comp rx=0 tx=0).stop 2024-09-10T09:06:42.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.145+0000 7f4895482640 1 -- 172.21.15.172:0/1736717924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4890101d50 msgr2=0x7f489019f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:42.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.145+0000 7f4895482640 1 --2- 172.21.15.172:0/1736717924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4890101d50 0x7f489019f3a0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f488800cad0 tx=0x7f488800cf90 comp rx=0 tx=0).stop 2024-09-10T09:06:42.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.145+0000 7f4895482640 1 -- 172.21.15.172:0/1736717924 shutdown_connections 2024-09-10T09:06:42.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.145+0000 7f4895482640 1 --2- 172.21.15.172:0/1736717924 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f487c0a2b90 0x7f487c0a5050 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:42.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.145+0000 7f4895482640 1 --2- 172.21.15.172:0/1736717924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4890101d50 0x7f489019f3a0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:42.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.145+0000 7f4895482640 1 --2- 172.21.15.172:0/1736717924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4890101390 0x7f4890198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:42.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.145+0000 7f4895482640 1 --2- 172.21.15.172:0/1736717924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48901001a0 0x7f4890197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:42.146 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.145+0000 7f4895482640 1 -- 172.21.15.172:0/1736717924 >> 172.21.15.172:0/1736717924 conn(0x7f48900fb900 msgr2=0x7f48901047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:42.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.145+0000 7f4895482640 1 -- 172.21.15.172:0/1736717924 shutdown_connections 2024-09-10T09:06:42.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.145+0000 7f4895482640 1 -- 172.21.15.172:0/1736717924 wait complete. 2024-09-10T09:06:42.155 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:41 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3869287235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:06:42.156 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:41 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2089897897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T09:06:42.156 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:41 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2657853287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T09:06:42.156 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:41 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/36925221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T09:06:42.157 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-10T09:06:42.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:41 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3869287235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:06:42.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:41 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2089897897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T09:06:42.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:41 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2657853287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T09:06:42.243 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:41 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/36925221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T09:06:42.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.260+0000 7f41d459f640 1 -- 172.21.15.172:0/3888358986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41cc107870 msgr2=0x7f41cc109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:42.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.260+0000 7f41d459f640 1 --2- 172.21.15.172:0/3888358986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41cc107870 0x7f41cc109c60 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009ca0 tx=0x7f41c402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:06:42.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.260+0000 7f41d459f640 1 -- 172.21.15.172:0/3888358986 shutdown_connections 2024-09-10T09:06:42.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.260+0000 7f41d459f640 1 --2- 172.21.15.172:0/3888358986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41cc107870 0x7f41cc109c60 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:42.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.260+0000 7f41d459f640 1 --2- 172.21.15.172:0/3888358986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41cc104f40 0x7f41cc107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:42.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.260+0000 7f41d459f640 1 --2- 172.21.15.172:0/3888358986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41cc102610 0x7f41cc104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:42.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.260+0000 7f41d459f640 1 -- 172.21.15.172:0/3888358986 >> 172.21.15.172:0/3888358986 conn(0x7f41cc0fc170 msgr2=0x7f41cc0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:42.263 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.260+0000 7f41d459f640 1 -- 172.21.15.172:0/3888358986 shutdown_connections 2024-09-10T09:06:42.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.260+0000 7f41d459f640 1 -- 172.21.15.172:0/3888358986 wait complete. 2024-09-10T09:06:42.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.260+0000 7f41d459f640 1 Processor -- start 2024-09-10T09:06:42.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.260+0000 7f41d459f640 1 -- start start 2024-09-10T09:06:42.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.260+0000 7f41d459f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41cc102610 0x7f41cc101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:42.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.260+0000 7f41d459f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41cc104f40 0x7f41cc1005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:42.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.260+0000 7f41d459f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41cc107870 0x7f41cc100b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:42.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.260+0000 7f41d459f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc10d680 con 0x7f41cc107870 2024-09-10T09:06:42.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.260+0000 7f41d459f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc10d500 con 0x7f41cc104f40 2024-09-10T09:06:42.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.260+0000 7f41d459f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc10d800 con 0x7f41cc102610 2024-09-10T09:06:42.264 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.261+0000 7f41d2b3c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41cc107870 0x7f41cc100b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:42.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.261+0000 7f41d2b3c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41cc107870 0x7f41cc100b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:43934/0 (socket says 172.21.15.172:43934) 2024-09-10T09:06:42.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.261+0000 7f41d2b3c640 1 -- 172.21.15.172:0/3532174082 learned_addr learned my addr 172.21.15.172:0/3532174082 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:06:42.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.261+0000 7f41d233b640 1 --2- 172.21.15.172:0/3532174082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41cc102610 0x7f41cc101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:42.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.261+0000 7f41d1b3a640 1 --2- 172.21.15.172:0/3532174082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41cc104f40 0x7f41cc1005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:42.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.261+0000 7f41d2b3c640 1 -- 172.21.15.172:0/3532174082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41cc102610 msgr2=0x7f41cc101f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:42.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.261+0000 7f41d2b3c640 1 --2- 172.21.15.172:0/3532174082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41cc102610 0x7f41cc101f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:42.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.261+0000 7f41d2b3c640 1 -- 172.21.15.172:0/3532174082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41cc104f40 msgr2=0x7f41cc1005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:42.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.261+0000 7f41d2b3c640 1 --2- 172.21.15.172:0/3532174082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41cc104f40 0x7f41cc1005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:42.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.261+0000 7f41d2b3c640 1 -- 172.21.15.172:0/3532174082 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41cc101390 con 0x7f41cc107870 2024-09-10T09:06:42.265 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.261+0000 7f41d2b3c640 1 --2- 172.21.15.172:0/3532174082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41cc107870 0x7f41cc100b00 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f41c4031970 tx=0x7f41c4009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:42.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.261+0000 7f41bf7fe640 1 -- 172.21.15.172:0/3532174082 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4003c30 con 0x7f41cc107870 2024-09-10T09:06:42.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.261+0000 7f41d459f640 1 -- 172.21.15.172:0/3532174082 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41cc1a1ef0 con 0x7f41cc107870 2024-09-10T09:06:42.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.261+0000 7f41bf7fe640 1 -- 172.21.15.172:0/3532174082 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41c4003dd0 con 0x7f41cc107870 2024-09-10T09:06:42.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.261+0000 7f41bf7fe640 1 -- 172.21.15.172:0/3532174082 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4030b80 con 0x7f41cc107870 2024-09-10T09:06:42.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.261+0000 7f41d459f640 1 -- 172.21.15.172:0/3532174082 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41cc1a2350 con 0x7f41cc107870 2024-09-10T09:06:42.266 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.262+0000 7f41d459f640 1 -- 172.21.15.172:0/3532174082 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41cc0fdc60 con 0x7f41cc107870 2024-09-10T09:06:42.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.263+0000 7f41bf7fe640 1 -- 172.21.15.172:0/3532174082 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f41c4004040 con 0x7f41cc107870 2024-09-10T09:06:42.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.264+0000 7f41bf7fe640 1 --2- 172.21.15.172:0/3532174082 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f41b80a2b90 0x7f41b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:06:42.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.265+0000 7f41d233b640 1 --2- 172.21.15.172:0/3532174082 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f41b80a2b90 0x7f41b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:06:42.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.265+0000 7f41bf7fe640 1 -- 172.21.15.172:0/3532174082 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f41c40f08e0 con 0x7f41cc107870 2024-09-10T09:06:42.267 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.265+0000 7f41d233b640 1 --2- 172.21.15.172:0/3532174082 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f41b80a2b90 0x7f41b80a5050 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009b60 tx=0x7f41c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:06:42.269 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.268+0000 7f41bf7fe640 1 -- 172.21.15.172:0/3532174082 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f41c40bd510 con 0x7f41cc107870 2024-09-10T09:06:42.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:41 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3869287235' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:06:42.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:41 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2089897897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T09:06:42.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:41 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2657853287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T09:06:42.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:41 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/36925221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T09:06:42.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.488+0000 7f41d459f640 1 -- 172.21.15.172:0/3532174082 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f41cc062e90 con 0x7f41cc107870 2024-09-10T09:06:42.501 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.501+0000 7f41bf7fe640 1 -- 172.21.15.172:0/3532174082 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740375 (secure 0 0 0) 0x7f41c40bce10 con 0x7f41cc107870 2024-09-10T09:06:42.514 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T09:06:42.515 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:06:42.515 INFO:teuthology.orchestra.run.smithi172.stdout: "epoch": 18, 2024-09-10T09:06:42.515 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.515 INFO:teuthology.orchestra.run.smithi172.stdout: "active_gid": 14150, 2024-09-10T09:06:42.515 INFO:teuthology.orchestra.run.smithi172.stdout: "active_name": "x", 2024-09-10T09:06:42.515 INFO:teuthology.orchestra.run.smithi172.stdout: "active_addrs": { 2024-09-10T09:06:42.515 INFO:teuthology.orchestra.run.smithi172.stdout: "addrvec": [ 2024-09-10T09:06:42.515 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.515 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v2", 2024-09-10T09:06:42.515 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "172.21.15.172:6800", 2024-09-10T09:06:42.515 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 107199760 2024-09-10T09:06:42.516 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.516 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.516 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v1", 2024-09-10T09:06:42.516 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "172.21.15.172:6801", 2024-09-10T09:06:42.516 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 107199760 2024-09-10T09:06:42.516 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.516 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-09-10T09:06:42.516 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.516 INFO:teuthology.orchestra.run.smithi172.stdout: "active_addr": "172.21.15.172:6801/107199760", 2024-09-10T09:06:42.516 INFO:teuthology.orchestra.run.smithi172.stdout: "active_change": "2024-09-10T08:37:17.019379+0000", 2024-09-10T09:06:42.516 INFO:teuthology.orchestra.run.smithi172.stdout: "active_mgr_features": 4540701547738038271, 2024-09-10T09:06:42.516 INFO:teuthology.orchestra.run.smithi172.stdout: "available": true, 2024-09-10T09:06:42.517 INFO:teuthology.orchestra.run.smithi172.stdout: "standbys": [ 2024-09-10T09:06:42.517 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.517 INFO:teuthology.orchestra.run.smithi172.stdout: "gid": 24106, 2024-09-10T09:06:42.517 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "z", 2024-09-10T09:06:42.517 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr_features": 4540701547738038271, 2024-09-10T09:06:42.517 INFO:teuthology.orchestra.run.smithi172.stdout: "available_modules": [ 2024-09-10T09:06:42.517 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.517 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "alerts", 2024-09-10T09:06:42.517 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.517 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.517 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.517 INFO:teuthology.orchestra.run.smithi172.stdout: "interval": { 2024-09-10T09:06:42.518 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "interval", 2024-09-10T09:06:42.518 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.518 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.518 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.518 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "60", 2024-09-10T09:06:42.518 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.518 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.518 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.518 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T09:06:42.518 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.518 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.518 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.519 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.519 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.519 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.519 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.519 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.519 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.519 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.519 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.519 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.519 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.519 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.519 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.519 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.520 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.520 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.520 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.520 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.520 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.520 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.520 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.520 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.520 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.520 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.520 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.520 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.520 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.521 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.521 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.521 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.521 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.521 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.521 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.521 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.521 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.521 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.521 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.521 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.521 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.522 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.522 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.522 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.522 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.522 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.522 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.522 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.522 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.522 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.522 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.522 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.522 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.522 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.523 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.523 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.523 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.523 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.523 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.523 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.523 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.523 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.523 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.523 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.523 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.523 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.523 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.524 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.524 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.524 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.524 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.524 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.524 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.524 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.524 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_destination": { 2024-09-10T09:06:42.524 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_destination", 2024-09-10T09:06:42.524 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.524 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.524 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.524 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.525 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.525 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.525 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.525 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Email address to send alerts to", 2024-09-10T09:06:42.525 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.525 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.525 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.525 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.525 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_from_name": { 2024-09-10T09:06:42.525 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_from_name", 2024-09-10T09:06:42.525 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.525 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.525 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.526 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "Ceph", 2024-09-10T09:06:42.526 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.526 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.526 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.526 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Email From: name", 2024-09-10T09:06:42.526 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.526 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.526 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.526 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.526 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_host": { 2024-09-10T09:06:42.526 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_host", 2024-09-10T09:06:42.526 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.526 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.527 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.527 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.527 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.527 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.527 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.527 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "SMTP server", 2024-09-10T09:06:42.527 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.527 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.527 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.527 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.527 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_password": { 2024-09-10T09:06:42.527 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_password", 2024-09-10T09:06:42.527 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.528 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.528 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.528 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.528 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.528 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.528 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.528 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Password to authenticate with", 2024-09-10T09:06:42.528 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.528 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.528 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.528 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.528 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_port": { 2024-09-10T09:06:42.528 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_port", 2024-09-10T09:06:42.529 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.529 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.529 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.529 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "465", 2024-09-10T09:06:42.529 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.529 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.529 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.529 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "SMTP port", 2024-09-10T09:06:42.529 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.529 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.529 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.529 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.529 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_sender": { 2024-09-10T09:06:42.530 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_sender", 2024-09-10T09:06:42.530 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.530 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.530 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.530 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.530 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.530 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.530 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.530 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "SMTP envelope sender", 2024-09-10T09:06:42.530 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.530 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.530 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.530 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.531 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_ssl": { 2024-09-10T09:06:42.531 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_ssl", 2024-09-10T09:06:42.531 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.531 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.531 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.531 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.531 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.531 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.531 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.531 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T09:06:42.531 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.531 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.531 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.532 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.532 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_user": { 2024-09-10T09:06:42.532 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_user", 2024-09-10T09:06:42.532 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.532 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.532 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.532 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.532 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.532 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.532 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.532 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "User to authenticate as", 2024-09-10T09:06:42.532 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.532 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.533 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.533 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.533 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.533 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.533 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.533 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "balancer", 2024-09-10T09:06:42.533 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.533 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.533 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.533 INFO:teuthology.orchestra.run.smithi172.stdout: "active": { 2024-09-10T09:06:42.533 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "active", 2024-09-10T09:06:42.533 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.534 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.534 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.534 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.534 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.534 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.534 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.534 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T09:06:42.534 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.534 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.534 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.534 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.534 INFO:teuthology.orchestra.run.smithi172.stdout: "begin_time": { 2024-09-10T09:06:42.534 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "begin_time", 2024-09-10T09:06:42.535 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.535 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.535 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.535 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0000", 2024-09-10T09:06:42.535 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.535 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.535 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.535 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T09:06:42.535 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T09:06:42.535 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.535 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.535 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.535 INFO:teuthology.orchestra.run.smithi172.stdout: "begin_weekday": { 2024-09-10T09:06:42.536 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "begin_weekday", 2024-09-10T09:06:42.536 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "uint", 2024-09-10T09:06:42.536 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.536 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.536 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0", 2024-09-10T09:06:42.536 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "0", 2024-09-10T09:06:42.536 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "6", 2024-09-10T09:06:42.536 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.536 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T09:06:42.536 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T09:06:42.536 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.536 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.537 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.537 INFO:teuthology.orchestra.run.smithi172.stdout: "crush_compat_max_iterations": { 2024-09-10T09:06:42.537 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "crush_compat_max_iterations", 2024-09-10T09:06:42.537 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "uint", 2024-09-10T09:06:42.537 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.537 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.537 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "25", 2024-09-10T09:06:42.537 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "1", 2024-09-10T09:06:42.537 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "250", 2024-09-10T09:06:42.537 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.537 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T09:06:42.537 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.537 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.538 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.538 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.538 INFO:teuthology.orchestra.run.smithi172.stdout: "crush_compat_metrics": { 2024-09-10T09:06:42.538 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "crush_compat_metrics", 2024-09-10T09:06:42.538 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.538 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.538 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.538 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T09:06:42.538 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.538 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.538 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.538 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T09:06:42.538 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T09:06:42.538 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.539 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.539 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.539 INFO:teuthology.orchestra.run.smithi172.stdout: "crush_compat_step": { 2024-09-10T09:06:42.539 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "crush_compat_step", 2024-09-10T09:06:42.539 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:42.539 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.539 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.539 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0.5", 2024-09-10T09:06:42.539 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "0.001", 2024-09-10T09:06:42.539 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "0.999", 2024-09-10T09:06:42.539 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.539 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "aggressiveness of optimization", 2024-09-10T09:06:42.539 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T09:06:42.540 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.540 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.540 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.540 INFO:teuthology.orchestra.run.smithi172.stdout: "end_time": { 2024-09-10T09:06:42.540 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "end_time", 2024-09-10T09:06:42.540 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.540 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.540 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.540 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "2359", 2024-09-10T09:06:42.540 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.540 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.540 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.540 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T09:06:42.541 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T09:06:42.541 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.541 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.541 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.541 INFO:teuthology.orchestra.run.smithi172.stdout: "end_weekday": { 2024-09-10T09:06:42.541 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "end_weekday", 2024-09-10T09:06:42.541 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "uint", 2024-09-10T09:06:42.541 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.541 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.541 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0", 2024-09-10T09:06:42.541 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "0", 2024-09-10T09:06:42.541 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "6", 2024-09-10T09:06:42.542 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.542 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T09:06:42.542 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T09:06:42.542 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.542 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.542 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.542 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.542 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.542 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.542 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.542 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.542 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.542 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.542 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.543 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.543 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.543 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.543 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.543 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.543 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.543 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.543 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.543 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.543 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.543 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.543 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.544 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.544 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.544 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.544 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.544 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.544 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.544 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.544 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.544 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.544 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.544 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.545 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.545 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.545 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.545 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.545 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.545 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.545 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.545 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.545 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.545 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.545 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.545 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.545 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.546 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.546 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.546 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.546 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.546 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.546 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.546 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.546 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.546 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.546 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.546 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.546 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.546 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.547 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.547 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.547 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.547 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.547 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.547 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.547 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.547 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.547 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.547 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.547 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.547 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.547 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.548 INFO:teuthology.orchestra.run.smithi172.stdout: "min_score": { 2024-09-10T09:06:42.548 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "min_score", 2024-09-10T09:06:42.548 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:42.548 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.548 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.548 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0", 2024-09-10T09:06:42.548 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.548 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.548 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.548 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T09:06:42.548 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.548 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.548 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.549 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.549 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": { 2024-09-10T09:06:42.549 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mode", 2024-09-10T09:06:42.549 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.549 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.549 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.549 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "upmap", 2024-09-10T09:06:42.549 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.549 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.549 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.549 INFO:teuthology.orchestra.run.smithi172.stdout: "crush-compat", 2024-09-10T09:06:42.549 INFO:teuthology.orchestra.run.smithi172.stdout: "none", 2024-09-10T09:06:42.549 INFO:teuthology.orchestra.run.smithi172.stdout: "read", 2024-09-10T09:06:42.550 INFO:teuthology.orchestra.run.smithi172.stdout: "upmap", 2024-09-10T09:06:42.550 INFO:teuthology.orchestra.run.smithi172.stdout: "upmap-read" 2024-09-10T09:06:42.550 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.550 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Balancer mode", 2024-09-10T09:06:42.550 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.550 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.550 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.550 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.550 INFO:teuthology.orchestra.run.smithi172.stdout: "pool_ids": { 2024-09-10T09:06:42.550 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "pool_ids", 2024-09-10T09:06:42.550 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.550 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.551 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.551 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.551 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.551 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.551 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.551 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T09:06:42.551 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.551 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.551 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.551 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.551 INFO:teuthology.orchestra.run.smithi172.stdout: "sleep_interval": { 2024-09-10T09:06:42.551 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "sleep_interval", 2024-09-10T09:06:42.551 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.551 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.552 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.552 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "60", 2024-09-10T09:06:42.552 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.552 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.552 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.552 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T09:06:42.552 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.552 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.552 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.552 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.552 INFO:teuthology.orchestra.run.smithi172.stdout: "upmap_max_deviation": { 2024-09-10T09:06:42.552 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "upmap_max_deviation", 2024-09-10T09:06:42.552 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.553 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.553 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.553 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "5", 2024-09-10T09:06:42.553 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "1", 2024-09-10T09:06:42.553 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.553 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.553 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T09:06:42.553 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T09:06:42.553 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.553 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.553 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.553 INFO:teuthology.orchestra.run.smithi172.stdout: "upmap_max_optimizations": { 2024-09-10T09:06:42.553 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "upmap_max_optimizations", 2024-09-10T09:06:42.554 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "uint", 2024-09-10T09:06:42.554 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.554 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.554 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:42.554 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.554 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.554 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.554 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T09:06:42.554 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.554 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.554 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.554 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.554 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.555 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.555 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.555 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "cephadm", 2024-09-10T09:06:42.555 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.555 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.555 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.555 INFO:teuthology.orchestra.run.smithi172.stdout: "agent_down_multiplier": { 2024-09-10T09:06:42.555 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "agent_down_multiplier", 2024-09-10T09:06:42.555 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:42.555 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.555 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.555 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "3.0", 2024-09-10T09:06:42.555 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.556 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.556 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.556 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T09:06:42.556 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.556 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.556 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.556 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.556 INFO:teuthology.orchestra.run.smithi172.stdout: "agent_refresh_rate": { 2024-09-10T09:06:42.556 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "agent_refresh_rate", 2024-09-10T09:06:42.556 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.556 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.556 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.556 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "20", 2024-09-10T09:06:42.557 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.557 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.557 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.557 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T09:06:42.557 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.557 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.557 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.557 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.557 INFO:teuthology.orchestra.run.smithi172.stdout: "agent_starting_port": { 2024-09-10T09:06:42.557 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "agent_starting_port", 2024-09-10T09:06:42.557 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.557 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.557 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.557 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "4721", 2024-09-10T09:06:42.558 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.558 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.558 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.558 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T09:06:42.558 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.558 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.558 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.558 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.558 INFO:teuthology.orchestra.run.smithi172.stdout: "allow_ptrace": { 2024-09-10T09:06:42.558 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "allow_ptrace", 2024-09-10T09:06:42.558 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.559 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.559 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.559 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.559 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.559 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.559 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.559 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T09:06:42.559 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T09:06:42.559 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.559 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.559 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.559 INFO:teuthology.orchestra.run.smithi172.stdout: "autotune_interval": { 2024-09-10T09:06:42.559 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "autotune_interval", 2024-09-10T09:06:42.560 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.560 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.560 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.560 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "600", 2024-09-10T09:06:42.560 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.560 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.560 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.560 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T09:06:42.560 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.560 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.560 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.560 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.561 INFO:teuthology.orchestra.run.smithi172.stdout: "autotune_memory_target_ratio": { 2024-09-10T09:06:42.561 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T09:06:42.561 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:42.561 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.561 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.561 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0.7", 2024-09-10T09:06:42.561 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.561 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.561 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.561 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T09:06:42.561 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.561 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.561 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.562 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.562 INFO:teuthology.orchestra.run.smithi172.stdout: "cephadm_log_destination": { 2024-09-10T09:06:42.562 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "cephadm_log_destination", 2024-09-10T09:06:42.562 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.562 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.562 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.562 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.562 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.562 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.562 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.562 INFO:teuthology.orchestra.run.smithi172.stdout: "file", 2024-09-10T09:06:42.562 INFO:teuthology.orchestra.run.smithi172.stdout: "file,syslog", 2024-09-10T09:06:42.562 INFO:teuthology.orchestra.run.smithi172.stdout: "syslog" 2024-09-10T09:06:42.563 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.563 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T09:06:42.563 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.563 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.563 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.563 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.563 INFO:teuthology.orchestra.run.smithi172.stdout: "cgroups_split": { 2024-09-10T09:06:42.563 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "cgroups_split", 2024-09-10T09:06:42.563 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.563 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.563 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.563 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.563 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.563 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.564 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.564 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T09:06:42.564 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.564 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.564 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.564 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.564 INFO:teuthology.orchestra.run.smithi172.stdout: "config_checks_enabled": { 2024-09-10T09:06:42.564 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "config_checks_enabled", 2024-09-10T09:06:42.564 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.564 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.564 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.564 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.565 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.565 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.565 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.565 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T09:06:42.565 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.565 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.565 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.565 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.565 INFO:teuthology.orchestra.run.smithi172.stdout: "config_dashboard": { 2024-09-10T09:06:42.565 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "config_dashboard", 2024-09-10T09:06:42.565 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.565 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.565 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.566 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.566 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.566 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.566 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.566 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T09:06:42.566 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.566 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.566 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.566 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.566 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_alertmanager": { 2024-09-10T09:06:42.566 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_alertmanager", 2024-09-10T09:06:42.566 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.566 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.567 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.567 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T09:06:42.567 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.567 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.567 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.567 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Prometheus container image", 2024-09-10T09:06:42.567 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.567 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.567 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.567 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.567 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_base": { 2024-09-10T09:06:42.567 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_base", 2024-09-10T09:06:42.567 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.567 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.568 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.568 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T09:06:42.568 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.568 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.568 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.568 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Container image name, without the tag", 2024-09-10T09:06:42.568 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.568 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.568 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.568 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.568 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_elasticsearch": { 2024-09-10T09:06:42.568 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_elasticsearch", 2024-09-10T09:06:42.568 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.569 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.569 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.569 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T09:06:42.569 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.569 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.569 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.569 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "elasticsearch container image", 2024-09-10T09:06:42.569 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.569 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.569 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.569 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.569 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_grafana": { 2024-09-10T09:06:42.570 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_grafana", 2024-09-10T09:06:42.570 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.570 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.570 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.570 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T09:06:42.570 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.570 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.570 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.570 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Prometheus container image", 2024-09-10T09:06:42.570 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.570 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.570 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.570 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.570 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_haproxy": { 2024-09-10T09:06:42.571 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_haproxy", 2024-09-10T09:06:42.571 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.571 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.571 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.571 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T09:06:42.571 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.571 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.571 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.571 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "HAproxy container image", 2024-09-10T09:06:42.571 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.572 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.572 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.572 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.572 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_jaeger_agent": { 2024-09-10T09:06:42.572 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_jaeger_agent", 2024-09-10T09:06:42.572 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.572 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.572 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.572 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T09:06:42.572 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.572 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.572 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.572 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Jaeger agent container image", 2024-09-10T09:06:42.573 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.573 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.573 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.573 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.573 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_jaeger_collector": { 2024-09-10T09:06:42.573 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_jaeger_collector", 2024-09-10T09:06:42.573 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.573 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.573 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.573 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T09:06:42.573 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.574 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.574 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.574 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Jaeger collector container image", 2024-09-10T09:06:42.574 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.574 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.574 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.574 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.574 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_jaeger_query": { 2024-09-10T09:06:42.574 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_jaeger_query", 2024-09-10T09:06:42.575 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.575 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.575 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.575 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T09:06:42.575 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.575 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.575 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.575 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Jaeger query container image", 2024-09-10T09:06:42.575 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.575 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.575 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.575 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.575 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_keepalived": { 2024-09-10T09:06:42.575 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_keepalived", 2024-09-10T09:06:42.576 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.576 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.576 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.576 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T09:06:42.576 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.576 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.576 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.576 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Keepalived container image", 2024-09-10T09:06:42.576 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.576 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.576 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.576 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.577 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_loki": { 2024-09-10T09:06:42.577 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_loki", 2024-09-10T09:06:42.577 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.577 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.577 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.577 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T09:06:42.577 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.577 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.577 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.577 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Loki container image", 2024-09-10T09:06:42.577 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.577 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.577 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.578 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.578 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_node_exporter": { 2024-09-10T09:06:42.578 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_node_exporter", 2024-09-10T09:06:42.578 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.578 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.578 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.578 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T09:06:42.578 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.578 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.578 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.578 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Prometheus container image", 2024-09-10T09:06:42.578 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.578 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.579 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.579 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.579 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_nvmeof": { 2024-09-10T09:06:42.579 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_nvmeof", 2024-09-10T09:06:42.579 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.579 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.579 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.579 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T09:06:42.579 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.579 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.579 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.579 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Nvme-of container image", 2024-09-10T09:06:42.579 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.580 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.580 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.580 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.580 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_prometheus": { 2024-09-10T09:06:42.580 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_prometheus", 2024-09-10T09:06:42.580 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.580 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.580 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.580 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T09:06:42.580 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.580 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.580 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.580 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Prometheus container image", 2024-09-10T09:06:42.581 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.581 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.581 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.581 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.581 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_promtail": { 2024-09-10T09:06:42.581 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_promtail", 2024-09-10T09:06:42.581 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.581 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.581 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.581 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T09:06:42.581 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.581 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.581 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.582 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Promtail container image", 2024-09-10T09:06:42.582 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.582 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.582 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.582 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.582 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_samba": { 2024-09-10T09:06:42.582 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_samba", 2024-09-10T09:06:42.582 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.582 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.582 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.582 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T09:06:42.582 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.582 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.583 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.583 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Samba/SMB container image", 2024-09-10T09:06:42.583 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.583 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.583 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.583 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.583 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_snmp_gateway": { 2024-09-10T09:06:42.583 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_snmp_gateway", 2024-09-10T09:06:42.583 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.583 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.583 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.583 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T09:06:42.583 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.584 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.584 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.584 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "SNMP Gateway container image", 2024-09-10T09:06:42.584 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.584 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.584 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.584 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.584 INFO:teuthology.orchestra.run.smithi172.stdout: "container_init": { 2024-09-10T09:06:42.584 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_init", 2024-09-10T09:06:42.584 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.584 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.584 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.584 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.585 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.585 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.585 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.585 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T09:06:42.585 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.585 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.585 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.585 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.585 INFO:teuthology.orchestra.run.smithi172.stdout: "daemon_cache_timeout": { 2024-09-10T09:06:42.585 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "daemon_cache_timeout", 2024-09-10T09:06:42.585 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.585 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.586 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.586 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "600", 2024-09-10T09:06:42.586 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.586 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.586 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.586 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T09:06:42.586 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.586 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.586 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.586 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.586 INFO:teuthology.orchestra.run.smithi172.stdout: "default_cephadm_command_timeout": { 2024-09-10T09:06:42.586 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T09:06:42.586 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.587 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.587 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.587 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "900", 2024-09-10T09:06:42.587 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.587 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.587 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.587 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T09:06:42.587 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.587 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.587 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.587 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.587 INFO:teuthology.orchestra.run.smithi172.stdout: "default_registry": { 2024-09-10T09:06:42.587 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "default_registry", 2024-09-10T09:06:42.587 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.588 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.588 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.588 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "docker.io", 2024-09-10T09:06:42.588 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.588 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.588 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.588 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T09:06:42.588 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.588 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.588 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.588 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.588 INFO:teuthology.orchestra.run.smithi172.stdout: "device_cache_timeout": { 2024-09-10T09:06:42.588 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "device_cache_timeout", 2024-09-10T09:06:42.589 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.589 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.589 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.589 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "1800", 2024-09-10T09:06:42.589 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.589 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.589 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.589 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "seconds to cache device inventory", 2024-09-10T09:06:42.589 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.589 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.589 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.589 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.589 INFO:teuthology.orchestra.run.smithi172.stdout: "device_enhanced_scan": { 2024-09-10T09:06:42.590 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "device_enhanced_scan", 2024-09-10T09:06:42.590 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.590 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.590 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.590 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.590 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.590 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.590 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.590 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T09:06:42.590 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.590 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.590 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.590 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.591 INFO:teuthology.orchestra.run.smithi172.stdout: "facts_cache_timeout": { 2024-09-10T09:06:42.591 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "facts_cache_timeout", 2024-09-10T09:06:42.591 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.591 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.591 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.591 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "60", 2024-09-10T09:06:42.591 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.591 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.591 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.591 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "seconds to cache host facts data", 2024-09-10T09:06:42.591 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.591 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.591 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.592 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.592 INFO:teuthology.orchestra.run.smithi172.stdout: "grafana_dashboards_path": { 2024-09-10T09:06:42.592 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "grafana_dashboards_path", 2024-09-10T09:06:42.592 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.592 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.592 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.592 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T09:06:42.592 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.592 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.592 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.592 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T09:06:42.592 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.592 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.593 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.593 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.593 INFO:teuthology.orchestra.run.smithi172.stdout: "host_check_interval": { 2024-09-10T09:06:42.593 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "host_check_interval", 2024-09-10T09:06:42.593 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.593 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.593 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.593 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "600", 2024-09-10T09:06:42.593 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.593 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.593 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.593 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how frequently to perform a host check", 2024-09-10T09:06:42.593 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.593 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.594 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.594 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.594 INFO:teuthology.orchestra.run.smithi172.stdout: "hw_monitoring": { 2024-09-10T09:06:42.594 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "hw_monitoring", 2024-09-10T09:06:42.594 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.594 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.594 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.594 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.594 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.594 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.594 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.594 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T09:06:42.594 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.595 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.595 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.595 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.595 INFO:teuthology.orchestra.run.smithi172.stdout: "inventory_list_all": { 2024-09-10T09:06:42.595 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "inventory_list_all", 2024-09-10T09:06:42.595 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.595 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.595 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.595 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.595 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.595 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.595 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.595 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T09:06:42.595 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.596 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.596 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.596 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.596 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.596 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.596 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.596 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.596 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.596 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.596 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.596 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.596 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.596 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.597 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.597 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.597 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.597 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.597 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.597 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.597 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.597 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.597 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.597 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.597 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.597 INFO:teuthology.orchestra.run.smithi172.stdout: "log_refresh_metadata": { 2024-09-10T09:06:42.597 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_refresh_metadata", 2024-09-10T09:06:42.597 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.598 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.598 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.598 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.598 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.598 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.598 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.598 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T09:06:42.598 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.598 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.598 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.598 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.599 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.599 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.599 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.599 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.599 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.599 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.599 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.599 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.599 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.599 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T09:06:42.599 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.599 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.599 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.599 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.600 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.600 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.600 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.600 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.600 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.600 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.600 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.600 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.600 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.600 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.600 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.600 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.600 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.601 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.601 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.601 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.601 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.601 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.601 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.601 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.601 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.601 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.601 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.601 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.601 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.601 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.602 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.602 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.602 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.602 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.602 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.602 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.602 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.602 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.602 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.602 INFO:teuthology.orchestra.run.smithi172.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T09:06:42.602 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T09:06:42.602 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.602 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.603 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.603 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.603 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.603 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.603 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.603 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T09:06:42.603 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.603 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.603 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.603 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.603 INFO:teuthology.orchestra.run.smithi172.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T09:06:42.603 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T09:06:42.604 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.604 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.604 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.604 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "*", 2024-09-10T09:06:42.604 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.604 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.604 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.604 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T09:06:42.604 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.604 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.604 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.604 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.604 INFO:teuthology.orchestra.run.smithi172.stdout: "max_count_per_host": { 2024-09-10T09:06:42.605 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "max_count_per_host", 2024-09-10T09:06:42.605 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.605 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.605 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.605 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:42.605 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.605 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.605 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.605 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "max number of daemons per service per host", 2024-09-10T09:06:42.605 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.605 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.605 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.605 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.606 INFO:teuthology.orchestra.run.smithi172.stdout: "max_osd_draining_count": { 2024-09-10T09:06:42.606 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "max_osd_draining_count", 2024-09-10T09:06:42.606 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.606 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.606 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.606 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:42.606 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.606 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.606 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.606 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T09:06:42.606 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.606 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.606 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.606 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.607 INFO:teuthology.orchestra.run.smithi172.stdout: "migration_current": { 2024-09-10T09:06:42.607 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "migration_current", 2024-09-10T09:06:42.607 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.607 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.607 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.607 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.607 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.607 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.607 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.607 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "internal - do not modify", 2024-09-10T09:06:42.607 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.607 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.607 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.608 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.608 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": { 2024-09-10T09:06:42.608 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mode", 2024-09-10T09:06:42.608 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.608 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.608 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.608 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "root", 2024-09-10T09:06:42.608 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.608 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.608 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.608 INFO:teuthology.orchestra.run.smithi172.stdout: "cephadm-package", 2024-09-10T09:06:42.608 INFO:teuthology.orchestra.run.smithi172.stdout: "root" 2024-09-10T09:06:42.608 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.609 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T09:06:42.609 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.609 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.609 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.609 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.609 INFO:teuthology.orchestra.run.smithi172.stdout: "oob_default_addr": { 2024-09-10T09:06:42.609 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "oob_default_addr", 2024-09-10T09:06:42.609 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.609 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.609 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.609 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "169.254.1.1", 2024-09-10T09:06:42.609 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.609 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.609 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.610 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T09:06:42.610 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.610 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.610 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.610 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.610 INFO:teuthology.orchestra.run.smithi172.stdout: "prometheus_alerts_path": { 2024-09-10T09:06:42.610 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "prometheus_alerts_path", 2024-09-10T09:06:42.610 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.610 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.610 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.610 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T09:06:42.610 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.610 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.610 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.611 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T09:06:42.611 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.611 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.611 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.611 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.611 INFO:teuthology.orchestra.run.smithi172.stdout: "registry_insecure": { 2024-09-10T09:06:42.611 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "registry_insecure", 2024-09-10T09:06:42.611 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.611 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.611 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.611 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.612 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.612 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.612 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.612 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T09:06:42.612 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.612 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.612 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.612 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.612 INFO:teuthology.orchestra.run.smithi172.stdout: "registry_password": { 2024-09-10T09:06:42.612 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "registry_password", 2024-09-10T09:06:42.612 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.612 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.612 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.613 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.613 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.613 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.613 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.613 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T09:06:42.613 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.613 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.613 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.613 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.613 INFO:teuthology.orchestra.run.smithi172.stdout: "registry_url": { 2024-09-10T09:06:42.613 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "registry_url", 2024-09-10T09:06:42.613 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.614 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.614 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.614 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.614 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.614 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.614 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.614 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T09:06:42.614 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.614 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.614 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.614 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.614 INFO:teuthology.orchestra.run.smithi172.stdout: "registry_username": { 2024-09-10T09:06:42.614 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "registry_username", 2024-09-10T09:06:42.615 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.615 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.615 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.615 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.615 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.615 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.615 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.615 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T09:06:42.615 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.615 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.615 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.615 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.615 INFO:teuthology.orchestra.run.smithi172.stdout: "secure_monitoring_stack": { 2024-09-10T09:06:42.616 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "secure_monitoring_stack", 2024-09-10T09:06:42.616 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.616 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.616 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.616 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.616 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.616 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.616 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.616 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T09:06:42.616 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.616 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.616 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.616 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.617 INFO:teuthology.orchestra.run.smithi172.stdout: "service_discovery_port": { 2024-09-10T09:06:42.617 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "service_discovery_port", 2024-09-10T09:06:42.617 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.617 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.617 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.617 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "8765", 2024-09-10T09:06:42.617 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.617 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.617 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.617 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "cephadm service discovery port", 2024-09-10T09:06:42.617 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.617 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.618 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.618 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.618 INFO:teuthology.orchestra.run.smithi172.stdout: "ssh_config_file": { 2024-09-10T09:06:42.618 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ssh_config_file", 2024-09-10T09:06:42.618 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.618 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.618 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.618 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.618 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.618 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.618 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.618 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T09:06:42.618 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.619 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.619 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.619 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.619 INFO:teuthology.orchestra.run.smithi172.stdout: "use_agent": { 2024-09-10T09:06:42.619 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "use_agent", 2024-09-10T09:06:42.619 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.619 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.619 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.619 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.619 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.619 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.619 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.619 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T09:06:42.619 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.620 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.620 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.620 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.620 INFO:teuthology.orchestra.run.smithi172.stdout: "use_repo_digest": { 2024-09-10T09:06:42.620 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "use_repo_digest", 2024-09-10T09:06:42.620 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.620 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.620 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.620 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.620 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.620 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.620 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.620 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T09:06:42.621 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.621 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.621 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.621 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.621 INFO:teuthology.orchestra.run.smithi172.stdout: "warn_on_failed_host_check": { 2024-09-10T09:06:42.621 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "warn_on_failed_host_check", 2024-09-10T09:06:42.621 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.621 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.621 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.621 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.621 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.621 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.621 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.622 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T09:06:42.622 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.622 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.622 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.622 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.622 INFO:teuthology.orchestra.run.smithi172.stdout: "warn_on_stray_daemons": { 2024-09-10T09:06:42.622 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "warn_on_stray_daemons", 2024-09-10T09:06:42.622 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.622 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.622 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.622 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.622 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.622 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.623 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.623 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T09:06:42.623 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.623 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.623 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.623 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.623 INFO:teuthology.orchestra.run.smithi172.stdout: "warn_on_stray_hosts": { 2024-09-10T09:06:42.623 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "warn_on_stray_hosts", 2024-09-10T09:06:42.623 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.623 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.623 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.623 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.623 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.623 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.624 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.624 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T09:06:42.624 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.624 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.624 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.624 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.624 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.624 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.624 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.624 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "crash", 2024-09-10T09:06:42.624 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.624 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.625 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.625 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.625 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.625 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.625 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.625 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.625 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.625 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.625 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.625 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.625 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.625 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.625 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.626 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.626 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.626 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.626 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.626 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.626 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.626 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.626 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.626 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.626 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.626 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.626 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.626 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.627 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.627 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.627 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.627 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.627 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.627 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.627 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.627 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.627 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.627 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.627 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.627 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.627 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.628 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.628 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.628 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.628 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.628 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.628 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.628 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.628 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.628 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.628 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.628 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.628 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.629 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.629 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.629 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.629 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.629 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.629 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.629 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.629 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.629 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.629 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.629 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.629 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.629 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.630 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.630 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.630 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.630 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.630 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.630 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.630 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.630 INFO:teuthology.orchestra.run.smithi172.stdout: "retain_interval": { 2024-09-10T09:06:42.630 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "retain_interval", 2024-09-10T09:06:42.630 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.630 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.630 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.630 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "31536000", 2024-09-10T09:06:42.630 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.631 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.631 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.631 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T09:06:42.631 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.631 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.631 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.631 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.631 INFO:teuthology.orchestra.run.smithi172.stdout: "warn_recent_interval": { 2024-09-10T09:06:42.631 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "warn_recent_interval", 2024-09-10T09:06:42.631 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.631 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.631 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.631 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "1209600", 2024-09-10T09:06:42.632 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.632 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.632 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.632 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T09:06:42.632 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.632 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.632 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.632 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.632 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.632 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.632 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.632 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "dashboard", 2024-09-10T09:06:42.632 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.633 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.633 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.633 INFO:teuthology.orchestra.run.smithi172.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T09:06:42.633 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T09:06:42.633 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.633 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.633 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.633 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:42.633 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.633 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.633 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.633 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.633 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.634 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.634 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.634 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.634 INFO:teuthology.orchestra.run.smithi172.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T09:06:42.634 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T09:06:42.634 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.634 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.634 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.634 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.634 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.634 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.634 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.634 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.635 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.635 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.635 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.635 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.635 INFO:teuthology.orchestra.run.smithi172.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T09:06:42.635 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T09:06:42.635 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.635 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.635 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.635 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.635 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.635 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.635 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.635 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.636 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.636 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.636 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.636 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.636 INFO:teuthology.orchestra.run.smithi172.stdout: "AUDIT_API_ENABLED": { 2024-09-10T09:06:42.636 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T09:06:42.636 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.636 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.636 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.636 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.636 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.636 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.637 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.637 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.637 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.637 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.637 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.637 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.637 INFO:teuthology.orchestra.run.smithi172.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T09:06:42.637 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T09:06:42.637 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.637 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.637 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.637 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.637 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.638 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.638 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.638 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.638 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.638 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.638 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.638 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.638 INFO:teuthology.orchestra.run.smithi172.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T09:06:42.638 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T09:06:42.638 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.638 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.638 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.638 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.638 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.639 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.639 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.639 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.639 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.639 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.639 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.639 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.639 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T09:06:42.639 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T09:06:42.639 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.639 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.639 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.640 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.640 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.640 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.640 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.640 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.640 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.640 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.640 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.640 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.640 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T09:06:42.640 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T09:06:42.640 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.641 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.641 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.641 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.641 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.641 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.641 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.641 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.641 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.641 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.641 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.641 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.641 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T09:06:42.642 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T09:06:42.642 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.642 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.642 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.642 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.642 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.642 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.642 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.642 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.642 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.642 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.642 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.642 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.643 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T09:06:42.643 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T09:06:42.643 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.643 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.643 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.643 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.643 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.643 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.643 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.643 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.643 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.643 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.643 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.644 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.644 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T09:06:42.644 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T09:06:42.644 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.644 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.644 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.644 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.644 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.644 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.644 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.644 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.644 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.645 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.645 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.645 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.645 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T09:06:42.645 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T09:06:42.645 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.645 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.645 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.645 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.645 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.645 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.645 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.645 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.646 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.646 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.646 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.646 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.646 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T09:06:42.646 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T09:06:42.646 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.646 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.646 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.646 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.646 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.646 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.646 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.647 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.647 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.647 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.647 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.647 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.647 INFO:teuthology.orchestra.run.smithi172.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T09:06:42.647 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T09:06:42.647 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.647 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.647 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.647 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.647 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.647 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.648 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.648 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.648 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.648 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.648 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.648 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.648 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T09:06:42.648 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T09:06:42.648 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.648 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.648 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.648 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "admin", 2024-09-10T09:06:42.648 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.649 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.649 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.649 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.649 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.649 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.649 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.649 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.649 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T09:06:42.649 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T09:06:42.649 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.649 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.649 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.649 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.650 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.650 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.650 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.650 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.650 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.650 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.650 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.650 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.650 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_API_URL": { 2024-09-10T09:06:42.650 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_API_URL", 2024-09-10T09:06:42.650 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.650 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.650 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.651 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.651 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.651 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.651 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.651 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.651 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.651 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.651 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.651 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.651 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T09:06:42.651 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T09:06:42.651 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.651 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.652 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.652 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "admin", 2024-09-10T09:06:42.652 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.652 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.652 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.652 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.652 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.652 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.652 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.652 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.652 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T09:06:42.652 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T09:06:42.652 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.653 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.653 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.653 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.653 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.653 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.653 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.653 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.653 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.653 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.653 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.653 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.653 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T09:06:42.653 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T09:06:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T09:06:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T09:06:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T09:06:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T09:06:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T09:06:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T09:06:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T09:06:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T09:06:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T09:06:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T09:06:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T09:06:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T09:06:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T09:06:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T09:06:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T09:06:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T09:06:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T09:06:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T09:06:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T09:06:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T09:06:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T09:06:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T09:06:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T09:06:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T09:06:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T09:06:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T09:06:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.668 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.668 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.668 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T09:06:42.668 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.668 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.668 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.668 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.668 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.668 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.668 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.668 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.668 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T09:06:42.668 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T09:06:42.669 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.669 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.669 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.669 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:42.669 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.669 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.669 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.669 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.669 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.669 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.669 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.669 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.669 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T09:06:42.670 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T09:06:42.670 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.670 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.670 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.670 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "8", 2024-09-10T09:06:42.670 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.670 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.670 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.670 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.670 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.670 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.670 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.670 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.671 INFO:teuthology.orchestra.run.smithi172.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T09:06:42.671 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T09:06:42.671 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.671 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.671 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.671 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "45", 2024-09-10T09:06:42.671 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.671 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.671 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.671 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.671 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.671 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.671 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.672 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.672 INFO:teuthology.orchestra.run.smithi172.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T09:06:42.672 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T09:06:42.672 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.672 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.672 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.672 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.672 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.672 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.672 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.672 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.672 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.673 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.673 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.673 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.673 INFO:teuthology.orchestra.run.smithi172.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T09:06:42.673 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T09:06:42.673 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.673 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.673 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.673 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "admin", 2024-09-10T09:06:42.673 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.673 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.673 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.674 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.674 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.674 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.674 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.674 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.674 INFO:teuthology.orchestra.run.smithi172.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T09:06:42.674 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T09:06:42.674 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.674 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.674 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.674 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.674 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.674 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.675 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.675 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.675 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.675 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.675 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.675 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.675 INFO:teuthology.orchestra.run.smithi172.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T09:06:42.675 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T09:06:42.675 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.675 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.675 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.675 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.676 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.676 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.676 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.676 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.676 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.676 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.676 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.676 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.676 INFO:teuthology.orchestra.run.smithi172.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T09:06:42.676 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T09:06:42.676 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.676 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.676 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.677 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0", 2024-09-10T09:06:42.677 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.677 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.677 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.677 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.677 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.677 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.677 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.677 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.677 INFO:teuthology.orchestra.run.smithi172.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T09:06:42.677 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T09:06:42.677 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.677 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.678 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.678 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:42.678 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.678 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.678 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.678 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.678 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.678 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.678 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.678 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.678 INFO:teuthology.orchestra.run.smithi172.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T09:06:42.678 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T09:06:42.678 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.679 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.679 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.679 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "5", 2024-09-10T09:06:42.679 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.679 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.679 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.679 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.679 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.679 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.679 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.679 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.679 INFO:teuthology.orchestra.run.smithi172.stdout: "cross_origin_url": { 2024-09-10T09:06:42.679 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "cross_origin_url", 2024-09-10T09:06:42.680 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.680 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.680 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.680 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.680 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.680 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.680 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.680 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.680 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.680 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.680 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.680 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.681 INFO:teuthology.orchestra.run.smithi172.stdout: "crt_file": { 2024-09-10T09:06:42.681 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "crt_file", 2024-09-10T09:06:42.681 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.681 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.681 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.681 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.681 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.681 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.681 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.681 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.681 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.681 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.681 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.682 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.682 INFO:teuthology.orchestra.run.smithi172.stdout: "debug": { 2024-09-10T09:06:42.682 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "debug", 2024-09-10T09:06:42.682 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.682 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.682 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.682 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.682 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.682 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.682 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.682 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Enable/disable debug options", 2024-09-10T09:06:42.682 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.683 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.683 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.683 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.683 INFO:teuthology.orchestra.run.smithi172.stdout: "jwt_token_ttl": { 2024-09-10T09:06:42.683 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "jwt_token_ttl", 2024-09-10T09:06:42.683 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.683 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.683 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.683 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "28800", 2024-09-10T09:06:42.683 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.683 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.683 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.683 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.684 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.684 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.684 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.684 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.684 INFO:teuthology.orchestra.run.smithi172.stdout: "key_file": { 2024-09-10T09:06:42.684 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "key_file", 2024-09-10T09:06:42.684 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.684 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.684 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.684 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.684 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.684 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.684 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.685 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.685 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.685 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.685 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.685 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.685 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.685 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.685 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.685 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.685 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.685 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.685 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.685 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.686 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.686 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.686 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.686 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.686 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.686 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.686 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.686 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.686 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.686 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.686 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.686 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.686 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.687 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.687 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.687 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.687 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.687 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.687 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.687 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.687 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.687 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.687 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.687 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.687 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.687 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.687 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.688 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.688 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.688 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.688 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.688 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.688 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.688 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.688 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.688 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.688 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.688 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.688 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.689 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.689 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.689 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.689 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.689 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.689 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.689 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.689 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.689 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.689 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.689 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.689 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.690 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.690 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.690 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.690 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.690 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.690 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.690 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.690 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.690 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.690 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.690 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.690 INFO:teuthology.orchestra.run.smithi172.stdout: "motd": { 2024-09-10T09:06:42.690 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "motd", 2024-09-10T09:06:42.690 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.691 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.691 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.691 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.691 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.691 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.691 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.691 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "The message of the day", 2024-09-10T09:06:42.691 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.691 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.691 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.691 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.691 INFO:teuthology.orchestra.run.smithi172.stdout: "redirect_resolve_ip_addr": { 2024-09-10T09:06:42.691 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T09:06:42.692 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.692 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.692 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.692 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.692 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.692 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.692 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.692 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.692 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.692 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.692 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.692 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.692 INFO:teuthology.orchestra.run.smithi172.stdout: "server_addr": { 2024-09-10T09:06:42.693 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_addr", 2024-09-10T09:06:42.693 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.693 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.693 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.693 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "::", 2024-09-10T09:06:42.693 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.693 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.693 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.693 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.693 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.693 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.693 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.693 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.694 INFO:teuthology.orchestra.run.smithi172.stdout: "server_port": { 2024-09-10T09:06:42.694 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_port", 2024-09-10T09:06:42.694 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.694 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.694 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.694 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "8080", 2024-09-10T09:06:42.694 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.694 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.694 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.694 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.694 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.694 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.694 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.695 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.695 INFO:teuthology.orchestra.run.smithi172.stdout: "ssl": { 2024-09-10T09:06:42.695 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ssl", 2024-09-10T09:06:42.695 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.695 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.695 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.695 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.695 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.695 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.695 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.695 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.695 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.695 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.696 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.696 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.696 INFO:teuthology.orchestra.run.smithi172.stdout: "ssl_server_port": { 2024-09-10T09:06:42.696 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ssl_server_port", 2024-09-10T09:06:42.696 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.696 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.696 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.696 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "8443", 2024-09-10T09:06:42.696 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.696 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.696 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.696 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.696 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.697 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.697 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.697 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.697 INFO:teuthology.orchestra.run.smithi172.stdout: "standby_behaviour": { 2024-09-10T09:06:42.697 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "standby_behaviour", 2024-09-10T09:06:42.697 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.697 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.697 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.697 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "redirect", 2024-09-10T09:06:42.697 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.697 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.697 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.697 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.698 INFO:teuthology.orchestra.run.smithi172.stdout: "redirect" 2024-09-10T09:06:42.698 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.698 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.698 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.698 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.698 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.698 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.698 INFO:teuthology.orchestra.run.smithi172.stdout: "standby_error_status_code": { 2024-09-10T09:06:42.698 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "standby_error_status_code", 2024-09-10T09:06:42.698 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.698 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.698 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.698 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "500", 2024-09-10T09:06:42.699 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "400", 2024-09-10T09:06:42.699 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "599", 2024-09-10T09:06:42.699 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.699 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.699 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.699 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.699 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.699 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.699 INFO:teuthology.orchestra.run.smithi172.stdout: "url_prefix": { 2024-09-10T09:06:42.699 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "url_prefix", 2024-09-10T09:06:42.699 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.699 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.699 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.700 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.700 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.700 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.700 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.700 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.700 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.700 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.700 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.700 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.700 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.700 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.700 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.700 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "devicehealth", 2024-09-10T09:06:42.701 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.701 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.701 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.701 INFO:teuthology.orchestra.run.smithi172.stdout: "enable_monitoring": { 2024-09-10T09:06:42.701 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "enable_monitoring", 2024-09-10T09:06:42.701 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.701 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.701 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.701 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.701 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.701 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.701 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.701 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "monitor device health metrics", 2024-09-10T09:06:42.702 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.702 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.702 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.702 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.702 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.702 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.702 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.702 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.702 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.702 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.702 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.702 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.702 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.703 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.703 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.703 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.703 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.703 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.703 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.703 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.703 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.703 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.703 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.703 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.703 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.703 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.704 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.704 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.704 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.704 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.704 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.704 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.704 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.704 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.704 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.704 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.704 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.704 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.704 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.704 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.705 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.705 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.705 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.705 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.705 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.705 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.705 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.705 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.705 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.705 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.705 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.705 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.706 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.706 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.706 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.706 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.706 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.706 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.706 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.706 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.706 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.706 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.706 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.706 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.706 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.706 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.707 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.707 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.707 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.707 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.707 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.707 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.707 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.707 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.707 INFO:teuthology.orchestra.run.smithi172.stdout: "mark_out_threshold": { 2024-09-10T09:06:42.707 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mark_out_threshold", 2024-09-10T09:06:42.707 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.707 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.708 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.708 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "2419200", 2024-09-10T09:06:42.708 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.708 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.708 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.708 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T09:06:42.708 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.708 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.708 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.708 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.708 INFO:teuthology.orchestra.run.smithi172.stdout: "pool_name": { 2024-09-10T09:06:42.708 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "pool_name", 2024-09-10T09:06:42.708 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.709 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.709 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.709 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "device_health_metrics", 2024-09-10T09:06:42.709 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.709 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.709 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.709 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T09:06:42.709 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.709 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.709 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.709 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.709 INFO:teuthology.orchestra.run.smithi172.stdout: "retention_period": { 2024-09-10T09:06:42.709 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "retention_period", 2024-09-10T09:06:42.710 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.710 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.710 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.710 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "15552000", 2024-09-10T09:06:42.710 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.710 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.710 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.710 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how long to retain device health metrics", 2024-09-10T09:06:42.710 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.710 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.710 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.710 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.710 INFO:teuthology.orchestra.run.smithi172.stdout: "scrape_frequency": { 2024-09-10T09:06:42.711 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "scrape_frequency", 2024-09-10T09:06:42.711 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.711 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.711 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.711 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "86400", 2024-09-10T09:06:42.711 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.711 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.711 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.711 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T09:06:42.711 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.711 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.711 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.712 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.712 INFO:teuthology.orchestra.run.smithi172.stdout: "self_heal": { 2024-09-10T09:06:42.712 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "self_heal", 2024-09-10T09:06:42.712 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.712 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.712 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.712 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.712 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.712 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.712 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.712 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T09:06:42.712 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.713 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.713 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.713 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.713 INFO:teuthology.orchestra.run.smithi172.stdout: "sleep_interval": { 2024-09-10T09:06:42.715 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "sleep_interval", 2024-09-10T09:06:42.715 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.715 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.716 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.716 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "600", 2024-09-10T09:06:42.716 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.716 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.716 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.716 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T09:06:42.716 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.716 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.716 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.716 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.716 INFO:teuthology.orchestra.run.smithi172.stdout: "warn_threshold": { 2024-09-10T09:06:42.716 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "warn_threshold", 2024-09-10T09:06:42.717 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.717 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.717 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.717 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "7257600", 2024-09-10T09:06:42.717 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.717 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.717 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.717 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T09:06:42.717 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.717 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.717 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.717 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.717 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.718 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.718 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.718 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "diskprediction_local", 2024-09-10T09:06:42.718 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.718 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.718 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.718 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.718 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.718 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.718 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.718 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.718 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.718 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.719 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.719 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.719 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.719 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.719 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.719 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.719 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.719 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.719 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.719 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.719 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.719 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.719 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.720 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.720 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.720 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.720 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.720 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.720 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.720 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.720 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.720 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.720 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.720 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.720 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.721 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.721 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.721 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.721 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.721 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.721 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.721 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.721 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.721 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.721 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.721 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.721 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.721 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.722 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.722 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.722 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.722 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.722 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.722 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.722 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.722 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.722 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.722 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.722 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.722 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.722 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.723 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.723 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.723 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.723 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.723 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.723 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.723 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.723 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.723 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.723 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.723 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.723 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.724 INFO:teuthology.orchestra.run.smithi172.stdout: "predict_interval": { 2024-09-10T09:06:42.724 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "predict_interval", 2024-09-10T09:06:42.724 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.724 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.724 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.724 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "86400", 2024-09-10T09:06:42.724 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.724 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.724 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.724 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.724 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.724 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.724 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.725 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.725 INFO:teuthology.orchestra.run.smithi172.stdout: "predictor_model": { 2024-09-10T09:06:42.725 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "predictor_model", 2024-09-10T09:06:42.725 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.725 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.725 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.725 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "prophetstor", 2024-09-10T09:06:42.725 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.725 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.725 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.725 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.725 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.725 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.726 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.726 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.726 INFO:teuthology.orchestra.run.smithi172.stdout: "sleep_interval": { 2024-09-10T09:06:42.726 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "sleep_interval", 2024-09-10T09:06:42.726 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.726 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.726 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.726 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "600", 2024-09-10T09:06:42.726 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.726 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.726 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.726 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.726 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.727 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.727 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.727 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.727 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.727 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.727 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.727 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "influx", 2024-09-10T09:06:42.727 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": false, 2024-09-10T09:06:42.727 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "influxdb python module not found", 2024-09-10T09:06:42.727 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.727 INFO:teuthology.orchestra.run.smithi172.stdout: "batch_size": { 2024-09-10T09:06:42.727 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "batch_size", 2024-09-10T09:06:42.727 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.727 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.728 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.728 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "5000", 2024-09-10T09:06:42.728 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.728 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.728 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.728 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T09:06:42.728 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.728 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.728 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.728 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.728 INFO:teuthology.orchestra.run.smithi172.stdout: "database": { 2024-09-10T09:06:42.728 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "database", 2024-09-10T09:06:42.729 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.729 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.729 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.729 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "ceph", 2024-09-10T09:06:42.729 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.729 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.729 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.729 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T09:06:42.729 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.729 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.729 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.729 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.729 INFO:teuthology.orchestra.run.smithi172.stdout: "hostname": { 2024-09-10T09:06:42.730 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "hostname", 2024-09-10T09:06:42.730 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.730 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.730 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.730 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.730 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.730 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.730 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.730 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "InfluxDB server hostname", 2024-09-10T09:06:42.730 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.730 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.730 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.730 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.731 INFO:teuthology.orchestra.run.smithi172.stdout: "interval": { 2024-09-10T09:06:42.731 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "interval", 2024-09-10T09:06:42.731 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.731 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.731 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.731 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "30", 2024-09-10T09:06:42.731 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "5", 2024-09-10T09:06:42.731 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.731 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.731 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T09:06:42.731 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.731 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.731 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.732 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.732 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.732 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.732 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.732 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.732 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.732 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.732 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.732 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.732 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.732 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.732 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.733 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.733 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.733 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.733 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.733 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.733 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.733 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.733 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.733 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.733 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.733 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.733 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.733 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.734 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.734 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.734 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.734 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.734 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.734 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.734 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.734 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.734 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.734 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.734 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.734 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.734 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.734 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.735 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.735 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.735 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.735 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.735 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.735 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.735 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.735 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.735 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.735 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.735 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.735 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.736 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.736 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.736 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.736 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.736 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.736 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.736 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.736 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.736 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.736 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.736 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.736 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.736 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.737 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.737 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.737 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.737 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.737 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.737 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.737 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.737 INFO:teuthology.orchestra.run.smithi172.stdout: "password": { 2024-09-10T09:06:42.737 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "password", 2024-09-10T09:06:42.737 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.737 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.737 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.737 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.738 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.738 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.738 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.738 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "password of InfluxDB server user", 2024-09-10T09:06:42.738 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.738 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.738 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.738 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.738 INFO:teuthology.orchestra.run.smithi172.stdout: "port": { 2024-09-10T09:06:42.738 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "port", 2024-09-10T09:06:42.738 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.738 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.739 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.739 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "8086", 2024-09-10T09:06:42.739 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.739 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.739 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.739 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "InfluxDB server port", 2024-09-10T09:06:42.739 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.739 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.739 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.739 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.739 INFO:teuthology.orchestra.run.smithi172.stdout: "ssl": { 2024-09-10T09:06:42.739 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ssl", 2024-09-10T09:06:42.739 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.740 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.740 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.740 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "false", 2024-09-10T09:06:42.740 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.740 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.740 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.740 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T09:06:42.740 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.740 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.740 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.740 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.740 INFO:teuthology.orchestra.run.smithi172.stdout: "threads": { 2024-09-10T09:06:42.741 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "threads", 2024-09-10T09:06:42.741 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.741 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.741 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.741 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "5", 2024-09-10T09:06:42.741 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "1", 2024-09-10T09:06:42.741 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "32", 2024-09-10T09:06:42.741 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.741 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T09:06:42.741 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.741 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.741 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.741 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.741 INFO:teuthology.orchestra.run.smithi172.stdout: "username": { 2024-09-10T09:06:42.742 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "username", 2024-09-10T09:06:42.742 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.742 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.742 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.742 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.742 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.742 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.742 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.742 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "username of InfluxDB server user", 2024-09-10T09:06:42.742 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.742 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.742 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.742 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.743 INFO:teuthology.orchestra.run.smithi172.stdout: "verify_ssl": { 2024-09-10T09:06:42.743 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "verify_ssl", 2024-09-10T09:06:42.743 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.743 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.743 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.743 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "true", 2024-09-10T09:06:42.743 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.743 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.743 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.743 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T09:06:42.743 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.743 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.744 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.744 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.744 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.744 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.744 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.744 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "insights", 2024-09-10T09:06:42.744 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.744 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.744 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.744 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.744 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.744 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.744 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.745 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.745 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.745 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.745 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.745 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.745 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.745 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.745 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.745 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.745 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.745 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.745 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.745 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.746 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.746 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.746 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.746 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.746 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.746 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.746 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.746 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.746 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.746 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.746 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.746 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.746 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.747 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.747 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.747 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.747 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.747 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.747 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.747 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.747 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.747 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.747 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.747 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.747 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.747 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.748 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.748 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.748 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.748 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.748 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.748 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.748 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.748 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.748 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.748 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.748 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.748 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.748 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.749 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.749 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.749 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.749 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.749 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.749 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.749 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.749 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.749 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.749 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.749 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.749 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.749 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.750 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.750 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.750 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.750 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.750 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "iostat", 2024-09-10T09:06:42.750 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.750 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.750 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.750 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.750 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.750 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.750 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.750 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.751 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.751 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.751 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.751 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.751 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.751 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.751 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.751 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.751 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.751 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.751 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.751 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.751 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.751 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.752 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.752 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.752 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.752 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.752 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.752 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.752 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.752 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.752 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.752 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.752 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.752 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.753 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.753 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.753 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.753 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.753 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.753 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.753 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.753 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.753 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.753 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.753 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.753 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.753 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.754 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.754 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.754 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.754 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.754 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.754 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.754 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.754 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.754 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.754 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.754 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.754 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.754 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.755 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.755 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.755 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.755 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.755 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.755 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.755 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.755 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.755 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.755 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.755 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.755 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.755 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.755 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.756 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.756 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.756 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "k8sevents", 2024-09-10T09:06:42.756 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.756 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.756 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.756 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_event_retention_days": { 2024-09-10T09:06:42.756 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ceph_event_retention_days", 2024-09-10T09:06:42.756 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.756 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.756 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.756 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "7", 2024-09-10T09:06:42.756 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.757 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.757 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.757 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T09:06:42.757 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.757 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.757 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.757 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.757 INFO:teuthology.orchestra.run.smithi172.stdout: "config_check_secs": { 2024-09-10T09:06:42.757 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "config_check_secs", 2024-09-10T09:06:42.757 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.757 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.757 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.757 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:42.757 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "10", 2024-09-10T09:06:42.758 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.758 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.758 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T09:06:42.758 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.758 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.758 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.758 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.758 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.758 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.758 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.758 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.758 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.759 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.759 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.759 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.759 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.759 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.759 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.759 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.759 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.759 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.759 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.759 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.759 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.759 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.759 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.760 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.760 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.760 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.760 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.760 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.760 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.760 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.760 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.760 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.760 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.760 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.760 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.760 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.761 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.761 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.761 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.761 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.761 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.761 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.761 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.761 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.761 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.761 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.761 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.761 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.761 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.762 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.762 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.762 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.762 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.762 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.762 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.762 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.762 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.762 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.762 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.762 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.762 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.763 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.763 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.763 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.763 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.763 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.763 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.763 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.763 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.763 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.763 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.763 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.763 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.763 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.764 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.764 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.764 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.764 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "localpool", 2024-09-10T09:06:42.764 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.764 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.764 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.764 INFO:teuthology.orchestra.run.smithi172.stdout: "failure_domain": { 2024-09-10T09:06:42.764 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "failure_domain", 2024-09-10T09:06:42.764 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.764 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.764 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.764 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "host", 2024-09-10T09:06:42.765 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.765 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.765 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.765 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "failure domain for any created local pool", 2024-09-10T09:06:42.765 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T09:06:42.765 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.765 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.765 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.765 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.765 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.765 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.765 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.765 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.766 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.766 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.766 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.766 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.766 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.766 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.766 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.766 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.766 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.766 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.766 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.766 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.766 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.767 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.767 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.767 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.767 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.767 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.767 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.767 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.767 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.767 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.767 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.767 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.767 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.767 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.768 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.768 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.768 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.768 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.768 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.768 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.768 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.768 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.768 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.768 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.768 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.768 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.768 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.769 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.769 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.769 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.769 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.769 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.769 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.769 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.769 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.769 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.769 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.769 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.769 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.769 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.770 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.770 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.770 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.770 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.770 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.770 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.770 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.770 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.770 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.770 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.770 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.770 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.770 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.771 INFO:teuthology.orchestra.run.smithi172.stdout: "min_size": { 2024-09-10T09:06:42.771 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "min_size", 2024-09-10T09:06:42.771 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.771 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.771 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.771 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.771 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.771 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.771 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.771 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "default min_size for any created local pool", 2024-09-10T09:06:42.771 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T09:06:42.771 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.772 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.772 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.772 INFO:teuthology.orchestra.run.smithi172.stdout: "num_rep": { 2024-09-10T09:06:42.772 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "num_rep", 2024-09-10T09:06:42.772 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.772 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.772 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.772 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "3", 2024-09-10T09:06:42.772 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.772 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.772 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.772 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "default replica count for any created local pool", 2024-09-10T09:06:42.773 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.773 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.773 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.773 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.773 INFO:teuthology.orchestra.run.smithi172.stdout: "pg_num": { 2024-09-10T09:06:42.773 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "pg_num", 2024-09-10T09:06:42.773 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.773 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.773 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.773 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "128", 2024-09-10T09:06:42.773 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.773 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.773 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.774 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T09:06:42.774 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.774 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.774 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.774 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.774 INFO:teuthology.orchestra.run.smithi172.stdout: "prefix": { 2024-09-10T09:06:42.774 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "prefix", 2024-09-10T09:06:42.774 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.774 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.774 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.774 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.774 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.775 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.775 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.775 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "name prefix for any created local pool", 2024-09-10T09:06:42.775 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.775 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.775 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.775 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.775 INFO:teuthology.orchestra.run.smithi172.stdout: "subtree": { 2024-09-10T09:06:42.775 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "subtree", 2024-09-10T09:06:42.775 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.775 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.775 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.776 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "rack", 2024-09-10T09:06:42.776 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.776 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.776 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.776 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T09:06:42.776 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T09:06:42.776 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.776 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.776 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.776 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.776 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.776 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.776 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mds_autoscaler", 2024-09-10T09:06:42.777 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.777 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.777 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.777 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.777 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.777 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.777 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.777 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.777 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.777 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.777 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.777 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.777 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.778 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.778 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.778 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.778 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.778 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.778 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.778 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.778 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.778 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.778 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.778 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.778 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.778 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.779 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.779 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.779 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.779 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.779 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.779 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.779 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.779 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.779 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.779 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.779 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.779 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.779 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.779 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.780 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.780 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.780 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.780 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.780 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.780 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.780 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.780 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.780 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.780 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.780 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.780 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.780 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.781 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.781 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.781 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.781 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.781 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.781 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.781 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.781 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.781 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.781 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.781 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.781 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.781 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.782 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.782 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.782 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.782 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.782 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.782 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.782 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.782 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.782 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.782 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.782 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mirroring", 2024-09-10T09:06:42.782 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.782 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.783 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.783 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.783 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.783 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.783 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.783 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.783 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.783 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.783 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.783 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.783 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.783 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.783 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.783 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.784 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.784 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.784 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.784 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.784 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.784 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.784 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.784 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.784 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.784 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.784 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.784 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.784 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.785 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.785 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.785 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.785 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.785 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.785 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.785 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.785 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.785 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.785 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.785 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.785 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.785 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.786 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.786 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.786 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.786 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.786 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.786 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.786 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.786 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.786 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.786 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.786 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.786 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.786 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.787 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.787 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.787 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.787 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.787 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.787 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.787 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.787 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.787 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.787 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.787 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.787 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.787 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.788 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.788 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.788 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.788 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.788 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.788 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.788 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.788 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.788 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "nfs", 2024-09-10T09:06:42.788 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.788 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.788 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.788 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.789 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.789 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.789 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.789 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.789 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.789 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.789 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.789 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.789 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.789 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.789 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.789 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.789 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.790 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.790 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.790 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.790 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.790 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.790 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.790 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.790 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.790 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.790 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.790 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.790 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.790 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.791 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.791 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.791 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.791 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.791 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.791 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.791 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.791 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.791 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.791 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.791 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.791 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.791 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.792 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.792 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.792 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.792 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.792 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.792 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.792 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.792 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.792 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.792 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.792 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.792 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.792 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.793 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.793 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.793 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.793 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.793 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.793 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.793 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.793 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.793 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.793 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.793 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.793 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.793 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.794 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.794 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.794 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.794 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.794 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.794 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.794 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.794 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "orchestrator", 2024-09-10T09:06:42.794 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.794 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.794 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.794 INFO:teuthology.orchestra.run.smithi172.stdout: "fail_fs": { 2024-09-10T09:06:42.794 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "fail_fs", 2024-09-10T09:06:42.794 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.795 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.795 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.795 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.795 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.795 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.795 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.795 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T09:06:42.795 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.795 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.795 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.795 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.795 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.795 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.796 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.796 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.796 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.796 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.796 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.796 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.796 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.796 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.796 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.796 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.796 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.796 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.796 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.797 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.797 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.797 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.797 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.797 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.797 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.797 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.797 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.797 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.797 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.797 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.797 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.797 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.797 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.798 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.798 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.798 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.798 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.798 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.798 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.798 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.798 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.798 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.798 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.798 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.798 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.798 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.799 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.799 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.799 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.799 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.799 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.799 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.799 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.799 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.799 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.799 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.799 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.799 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.799 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.799 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.800 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.800 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.800 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.800 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.800 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.800 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.800 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.800 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.800 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.800 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.800 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.800 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.800 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.801 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.801 INFO:teuthology.orchestra.run.smithi172.stdout: "orchestrator": { 2024-09-10T09:06:42.801 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "orchestrator", 2024-09-10T09:06:42.801 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.801 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.801 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.801 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.801 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.801 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.801 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.801 INFO:teuthology.orchestra.run.smithi172.stdout: "cephadm", 2024-09-10T09:06:42.801 INFO:teuthology.orchestra.run.smithi172.stdout: "rook", 2024-09-10T09:06:42.801 INFO:teuthology.orchestra.run.smithi172.stdout: "test_orchestrator" 2024-09-10T09:06:42.802 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.802 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Orchestrator backend", 2024-09-10T09:06:42.802 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.802 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.802 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.802 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.802 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.802 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.802 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.802 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "osd_perf_query", 2024-09-10T09:06:42.802 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.802 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.803 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.803 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.803 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.803 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.803 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.803 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.803 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.803 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.803 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.803 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.803 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.803 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.803 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.804 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.804 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.804 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.804 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.804 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.804 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.804 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.804 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.804 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.804 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.804 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.804 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.804 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.805 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.805 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.805 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.805 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.805 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.805 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.805 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.805 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.805 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.805 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.805 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.805 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.805 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.806 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.806 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.806 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.806 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.806 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.806 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.806 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.806 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.806 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.806 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.806 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.806 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.806 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.806 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.807 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.807 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.807 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.807 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.807 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.807 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.807 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.807 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.807 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.807 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.807 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.807 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.807 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.808 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.808 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.808 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.808 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.808 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.808 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.808 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.808 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.808 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "osd_support", 2024-09-10T09:06:42.808 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.808 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.808 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.808 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.809 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.809 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.809 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.809 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.809 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.809 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.809 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.809 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.809 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.809 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.809 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.809 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.809 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.810 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.810 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.810 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.810 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.810 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.810 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.810 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.810 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.810 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.810 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.810 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.810 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.810 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.810 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.811 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.811 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.811 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.811 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.811 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.811 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.811 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.811 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.811 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.811 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.811 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.811 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.811 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.812 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.812 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.812 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.812 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.812 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.812 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.812 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.812 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.812 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.812 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.812 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.812 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.812 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.813 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.813 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.813 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.813 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.813 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.813 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.813 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.813 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.813 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.813 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.813 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.813 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.813 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.814 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.814 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.814 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.814 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.814 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.814 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.814 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "pg_autoscaler", 2024-09-10T09:06:42.814 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.814 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.814 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.814 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.814 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.814 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.814 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.815 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.815 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.815 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.815 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.815 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.815 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.815 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.815 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.815 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.815 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.815 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.815 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.815 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.815 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.816 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.816 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.816 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.816 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.816 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.816 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.816 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.816 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.816 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.816 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.816 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.816 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.816 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.817 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.817 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.817 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.817 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.817 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.817 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.817 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.817 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.817 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.817 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.817 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.817 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.818 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.818 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.818 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.818 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.818 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.818 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.818 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.818 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.818 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.818 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.818 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.818 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.818 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.819 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.819 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.819 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.819 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.819 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.819 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.819 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.819 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.819 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.819 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.819 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.819 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.819 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.820 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.820 INFO:teuthology.orchestra.run.smithi172.stdout: "sleep_interval": { 2024-09-10T09:06:42.820 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "sleep_interval", 2024-09-10T09:06:42.820 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.820 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.820 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.820 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "60", 2024-09-10T09:06:42.820 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.820 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.820 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.820 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.820 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.820 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.821 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.821 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.821 INFO:teuthology.orchestra.run.smithi172.stdout: "threshold": { 2024-09-10T09:06:42.821 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "threshold", 2024-09-10T09:06:42.821 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:42.821 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.821 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.821 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "3.0", 2024-09-10T09:06:42.821 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "1.0", 2024-09-10T09:06:42.821 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.821 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.821 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "scaling threshold", 2024-09-10T09:06:42.822 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T09:06:42.822 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.822 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.822 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.822 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.822 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.822 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.822 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "progress", 2024-09-10T09:06:42.822 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.822 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.822 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.822 INFO:teuthology.orchestra.run.smithi172.stdout: "allow_pg_recovery_event": { 2024-09-10T09:06:42.822 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "allow_pg_recovery_event", 2024-09-10T09:06:42.823 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.823 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.823 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.823 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.823 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.823 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.823 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.823 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T09:06:42.823 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.823 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.823 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.823 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.823 INFO:teuthology.orchestra.run.smithi172.stdout: "enabled": { 2024-09-10T09:06:42.824 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "enabled", 2024-09-10T09:06:42.824 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.824 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.824 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.824 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.824 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.824 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.824 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.824 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.824 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.824 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.824 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.825 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.825 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.825 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.825 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.825 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.825 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.825 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.825 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.825 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.825 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.825 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.825 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.825 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.826 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.826 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.826 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.826 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.826 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.826 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.826 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.826 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.826 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.826 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.826 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.826 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.826 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.827 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.827 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.827 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.827 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.827 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.827 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.827 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.827 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.827 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.827 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.827 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.827 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.827 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.827 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.828 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.828 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.828 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.828 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.828 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.828 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.828 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.828 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.828 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.828 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.828 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.828 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.829 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.829 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.829 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.829 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.829 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.829 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.829 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.829 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.829 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.829 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.829 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.829 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.829 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.830 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.830 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.830 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.830 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.830 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.830 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.830 INFO:teuthology.orchestra.run.smithi172.stdout: "max_completed_events": { 2024-09-10T09:06:42.830 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "max_completed_events", 2024-09-10T09:06:42.830 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.830 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.830 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.830 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "50", 2024-09-10T09:06:42.830 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.830 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.831 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.831 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "number of past completed events to remember", 2024-09-10T09:06:42.831 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.831 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.831 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.831 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.831 INFO:teuthology.orchestra.run.smithi172.stdout: "sleep_interval": { 2024-09-10T09:06:42.831 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "sleep_interval", 2024-09-10T09:06:42.831 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.831 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.831 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.831 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "5", 2024-09-10T09:06:42.831 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.832 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.832 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.832 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how long the module is going to sleep", 2024-09-10T09:06:42.832 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.832 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.832 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.832 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.832 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.832 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.832 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.832 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "prometheus", 2024-09-10T09:06:42.832 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.833 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.833 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.833 INFO:teuthology.orchestra.run.smithi172.stdout: "cache": { 2024-09-10T09:06:42.833 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "cache", 2024-09-10T09:06:42.833 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.833 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.833 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.833 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.833 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.833 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.833 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.833 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.833 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.834 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.834 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.834 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.834 INFO:teuthology.orchestra.run.smithi172.stdout: "exclude_perf_counters": { 2024-09-10T09:06:42.834 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "exclude_perf_counters", 2024-09-10T09:06:42.834 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.834 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.834 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.834 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.834 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.834 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.834 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.834 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T09:06:42.835 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T09:06:42.835 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.835 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.835 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.835 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.835 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.835 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.835 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.835 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.835 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.835 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.835 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.835 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.836 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.836 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.836 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.836 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.836 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.836 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.836 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.836 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.836 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.836 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.836 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.836 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.836 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.837 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.837 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.837 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.837 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.837 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.837 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.837 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.837 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.837 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.837 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.837 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.837 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.837 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.838 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.838 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.838 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.838 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.838 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.838 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.838 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.838 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.838 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.838 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.838 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.838 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.838 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.839 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.839 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.839 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.839 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.839 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.839 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.839 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.839 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.839 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.839 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.839 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.839 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.839 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.839 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.840 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.840 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.840 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.840 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.840 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.840 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.840 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.840 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.840 INFO:teuthology.orchestra.run.smithi172.stdout: "rbd_stats_pools": { 2024-09-10T09:06:42.840 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rbd_stats_pools", 2024-09-10T09:06:42.840 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.840 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.840 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.841 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.841 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.841 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.841 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.841 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.841 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.841 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.841 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.841 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.841 INFO:teuthology.orchestra.run.smithi172.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T09:06:42.841 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T09:06:42.841 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.841 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.842 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.842 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "300", 2024-09-10T09:06:42.842 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.842 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.842 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.842 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.842 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.842 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.842 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.842 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.842 INFO:teuthology.orchestra.run.smithi172.stdout: "scrape_interval": { 2024-09-10T09:06:42.842 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "scrape_interval", 2024-09-10T09:06:42.842 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:42.843 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.843 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.843 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "15.0", 2024-09-10T09:06:42.843 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.843 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.843 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.843 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.843 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.843 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.843 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.843 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.843 INFO:teuthology.orchestra.run.smithi172.stdout: "server_addr": { 2024-09-10T09:06:42.843 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_addr", 2024-09-10T09:06:42.844 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.844 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.844 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.844 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "::", 2024-09-10T09:06:42.844 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.844 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.844 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.844 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T09:06:42.844 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.844 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.844 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.844 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.844 INFO:teuthology.orchestra.run.smithi172.stdout: "server_port": { 2024-09-10T09:06:42.844 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_port", 2024-09-10T09:06:42.845 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.845 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.845 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.845 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "9283", 2024-09-10T09:06:42.845 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.845 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.845 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.845 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T09:06:42.845 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.845 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.845 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.845 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.845 INFO:teuthology.orchestra.run.smithi172.stdout: "stale_cache_strategy": { 2024-09-10T09:06:42.846 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "stale_cache_strategy", 2024-09-10T09:06:42.846 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.846 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.846 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.846 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "log", 2024-09-10T09:06:42.846 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.846 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.846 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.846 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.846 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.846 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.846 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.846 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.847 INFO:teuthology.orchestra.run.smithi172.stdout: "standby_behaviour": { 2024-09-10T09:06:42.847 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "standby_behaviour", 2024-09-10T09:06:42.847 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.847 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.847 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.847 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "default", 2024-09-10T09:06:42.847 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.847 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.847 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.847 INFO:teuthology.orchestra.run.smithi172.stdout: "default", 2024-09-10T09:06:42.847 INFO:teuthology.orchestra.run.smithi172.stdout: "error" 2024-09-10T09:06:42.847 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.847 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.848 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.848 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.848 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.848 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.848 INFO:teuthology.orchestra.run.smithi172.stdout: "standby_error_status_code": { 2024-09-10T09:06:42.848 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "standby_error_status_code", 2024-09-10T09:06:42.848 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.848 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.848 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.848 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "500", 2024-09-10T09:06:42.848 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "400", 2024-09-10T09:06:42.848 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "599", 2024-09-10T09:06:42.849 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.849 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.849 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.849 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.849 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.849 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.849 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.849 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.849 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.849 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rbd_support", 2024-09-10T09:06:42.849 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.849 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.849 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.849 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.850 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.850 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.850 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.850 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.850 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.850 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.850 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.850 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.850 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.850 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.850 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.850 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.850 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.851 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.851 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.851 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.851 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.851 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.851 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.851 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.851 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.851 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.851 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.851 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.851 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.852 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.852 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.852 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.852 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.852 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.852 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.852 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.852 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.852 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.852 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.852 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.852 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.852 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.853 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.853 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.853 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.853 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.853 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.853 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.853 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.853 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.853 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.853 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.853 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.853 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.853 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.853 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.854 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.854 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.854 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.854 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.854 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.854 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.854 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.854 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.854 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.854 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.854 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.854 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.854 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.855 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.855 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.855 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.855 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.855 INFO:teuthology.orchestra.run.smithi172.stdout: "max_concurrent_snap_create": { 2024-09-10T09:06:42.855 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "max_concurrent_snap_create", 2024-09-10T09:06:42.855 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.855 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.855 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.855 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:42.855 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.855 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.855 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.856 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.856 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.856 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.856 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.856 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.856 INFO:teuthology.orchestra.run.smithi172.stdout: "mirror_snapshot_schedule": { 2024-09-10T09:06:42.856 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T09:06:42.856 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.856 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.856 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.856 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.856 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.856 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.857 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.857 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.857 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.857 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.857 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.857 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.857 INFO:teuthology.orchestra.run.smithi172.stdout: "trash_purge_schedule": { 2024-09-10T09:06:42.857 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "trash_purge_schedule", 2024-09-10T09:06:42.857 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.857 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.857 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.857 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.857 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.857 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.858 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.858 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.858 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.858 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.858 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.858 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.858 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.858 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.858 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.858 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "restful", 2024-09-10T09:06:42.858 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.858 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.858 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.859 INFO:teuthology.orchestra.run.smithi172.stdout: "enable_auth": { 2024-09-10T09:06:42.859 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "enable_auth", 2024-09-10T09:06:42.859 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.859 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.859 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.859 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.859 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.859 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.859 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.859 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.859 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.859 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.859 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.859 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.860 INFO:teuthology.orchestra.run.smithi172.stdout: "key_file": { 2024-09-10T09:06:42.860 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "key_file", 2024-09-10T09:06:42.860 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.860 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.860 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.860 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.860 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.860 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.860 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.860 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.860 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.861 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.861 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.861 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.861 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.861 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.861 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.861 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.861 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.861 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.861 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.861 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.861 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.862 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.862 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.862 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.862 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.862 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.862 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.862 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.862 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.862 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.862 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.862 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.862 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.862 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.863 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.863 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.863 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.863 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.863 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.863 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.863 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.863 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.863 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.863 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.863 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.863 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.863 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.864 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.864 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.864 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.864 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.864 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.864 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.864 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.864 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.864 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.864 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.864 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.864 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.864 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.864 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.865 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.865 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.865 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.865 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.865 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.865 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.865 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.865 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.865 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.865 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.865 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.865 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.865 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.866 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.866 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.866 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.866 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.866 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.866 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.866 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.866 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.866 INFO:teuthology.orchestra.run.smithi172.stdout: "server_addr": { 2024-09-10T09:06:42.866 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_addr", 2024-09-10T09:06:42.866 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.866 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.866 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.867 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.867 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.867 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.867 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.867 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.867 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.867 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.867 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.867 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.867 INFO:teuthology.orchestra.run.smithi172.stdout: "server_port": { 2024-09-10T09:06:42.867 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_port", 2024-09-10T09:06:42.867 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.867 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.867 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.868 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.868 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.868 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.868 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.868 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.868 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.868 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.868 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.868 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.868 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.868 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.868 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.868 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rgw", 2024-09-10T09:06:42.869 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.869 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.869 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.869 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.869 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.869 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.869 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.869 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.869 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.869 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.869 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.869 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.869 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.869 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.870 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.870 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.870 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.870 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.870 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.870 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.870 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.870 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.870 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.870 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.870 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.870 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.870 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.871 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.871 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.871 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.871 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.871 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.871 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.871 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.871 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.871 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.871 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.871 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.871 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.871 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.871 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.872 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.872 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.872 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.872 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.872 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.872 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.872 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.872 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.872 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.872 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.872 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.872 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.872 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.873 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.873 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.873 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.873 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.873 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.873 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.873 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.873 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.873 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.873 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.873 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.873 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.873 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.874 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.874 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.874 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.874 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.874 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.874 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.874 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.874 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.874 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.874 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rook", 2024-09-10T09:06:42.874 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.874 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.874 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.875 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.875 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.875 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.875 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.875 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.875 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.875 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.875 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.875 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.875 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.875 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.875 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.875 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.876 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.876 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.876 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.876 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.876 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.876 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.876 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.876 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.876 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.876 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.876 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.877 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.877 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.877 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.877 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.877 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.877 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.877 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.877 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.877 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.877 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.877 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.877 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.877 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.878 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.878 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.878 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.878 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.878 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.878 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.878 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.878 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.878 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.878 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.878 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.878 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.879 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.879 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.879 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.879 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.879 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.879 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.879 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.879 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.879 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.879 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.879 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.879 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.879 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.879 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.880 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.880 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.880 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.880 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.880 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.880 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.880 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.880 INFO:teuthology.orchestra.run.smithi172.stdout: "storage_class": { 2024-09-10T09:06:42.880 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "storage_class", 2024-09-10T09:06:42.880 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.880 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.880 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.880 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "local", 2024-09-10T09:06:42.881 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.881 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.881 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.881 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T09:06:42.881 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.881 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.881 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.881 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.881 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.881 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.881 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.881 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "selftest", 2024-09-10T09:06:42.881 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.882 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.882 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.882 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.882 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.882 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.882 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.882 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.882 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.882 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.882 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.882 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.882 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.883 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.883 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.883 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.883 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.883 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.883 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.883 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.883 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.883 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.883 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.883 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.883 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.884 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.884 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.884 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.884 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.884 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.884 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.884 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.884 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.884 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.884 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.884 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.884 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.884 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.885 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.885 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.885 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.885 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.885 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.885 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.885 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.885 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.885 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.885 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.885 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.885 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.886 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.886 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.886 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.886 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.886 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.886 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.886 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.886 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.886 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.886 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.886 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.886 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.886 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.886 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.887 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.887 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.887 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.887 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.887 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.887 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.887 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.887 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.887 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.887 INFO:teuthology.orchestra.run.smithi172.stdout: "roption1": { 2024-09-10T09:06:42.887 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "roption1", 2024-09-10T09:06:42.887 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.887 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.887 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.888 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.888 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.888 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.888 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.888 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.888 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.888 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.888 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.888 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.888 INFO:teuthology.orchestra.run.smithi172.stdout: "roption2": { 2024-09-10T09:06:42.888 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "roption2", 2024-09-10T09:06:42.888 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.888 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.888 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.889 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "xyz", 2024-09-10T09:06:42.889 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.889 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.889 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.889 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.889 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.889 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.889 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.889 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.889 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption1": { 2024-09-10T09:06:42.889 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption1", 2024-09-10T09:06:42.889 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.889 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.889 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.890 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.890 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.890 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.890 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.890 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.890 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.890 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.890 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.890 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.890 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption2": { 2024-09-10T09:06:42.890 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption2", 2024-09-10T09:06:42.890 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.890 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.891 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.891 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.891 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.891 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.891 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.891 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.891 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.891 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.891 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.891 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.891 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption3": { 2024-09-10T09:06:42.891 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption3", 2024-09-10T09:06:42.892 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:42.892 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.892 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.892 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.892 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.892 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.892 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.892 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.892 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.892 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.892 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.892 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.893 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption4": { 2024-09-10T09:06:42.893 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption4", 2024-09-10T09:06:42.893 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.893 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.893 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.893 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.893 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.893 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.893 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.893 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.893 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.893 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.894 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.894 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.894 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption5": { 2024-09-10T09:06:42.894 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption5", 2024-09-10T09:06:42.894 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.894 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.894 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.894 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.894 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.894 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.894 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.894 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.894 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.895 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.895 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.895 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.895 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption6": { 2024-09-10T09:06:42.895 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption6", 2024-09-10T09:06:42.895 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.895 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.895 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.895 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.895 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.895 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.895 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.895 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.896 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.896 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.896 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.896 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.896 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption7": { 2024-09-10T09:06:42.896 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption7", 2024-09-10T09:06:42.896 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.896 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.896 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.896 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.896 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "1", 2024-09-10T09:06:42.896 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "42", 2024-09-10T09:06:42.897 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.897 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.897 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.897 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.897 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.897 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.897 INFO:teuthology.orchestra.run.smithi172.stdout: "testkey": { 2024-09-10T09:06:42.897 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "testkey", 2024-09-10T09:06:42.897 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.897 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.897 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.897 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.898 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.898 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.898 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.898 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.898 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.898 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.898 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.898 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.898 INFO:teuthology.orchestra.run.smithi172.stdout: "testlkey": { 2024-09-10T09:06:42.898 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "testlkey", 2024-09-10T09:06:42.898 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.898 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.898 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.899 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.899 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.899 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.899 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.899 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.899 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.899 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.899 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.899 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.899 INFO:teuthology.orchestra.run.smithi172.stdout: "testnewline": { 2024-09-10T09:06:42.899 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "testnewline", 2024-09-10T09:06:42.899 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.899 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.900 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.900 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.900 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.900 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.900 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.900 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.900 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.900 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.900 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.900 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.900 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.900 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.900 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.901 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "snap_schedule", 2024-09-10T09:06:42.901 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.901 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.901 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.901 INFO:teuthology.orchestra.run.smithi172.stdout: "allow_m_granularity": { 2024-09-10T09:06:42.901 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "allow_m_granularity", 2024-09-10T09:06:42.901 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.901 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.901 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.901 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.901 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.901 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.902 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.902 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T09:06:42.902 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.902 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.902 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.902 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.902 INFO:teuthology.orchestra.run.smithi172.stdout: "dump_on_update": { 2024-09-10T09:06:42.902 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "dump_on_update", 2024-09-10T09:06:42.902 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.902 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.902 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.902 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.903 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.903 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.903 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.903 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "dump database to debug log on update", 2024-09-10T09:06:42.903 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.903 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.903 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.903 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.903 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.903 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.903 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.903 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.903 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.904 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.904 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.904 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.904 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.904 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.904 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.904 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.904 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.904 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.904 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.904 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.904 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.905 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.905 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.905 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.905 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.905 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.905 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.905 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.905 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.905 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.905 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.905 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.905 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.905 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.906 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.906 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.906 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.906 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.906 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.906 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.906 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.906 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.906 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.906 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.906 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.906 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.907 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.907 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.907 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.907 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.907 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.907 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.907 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.907 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.907 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.907 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.907 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.907 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.908 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.908 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.908 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.908 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.908 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.908 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.908 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.908 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.908 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.908 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.908 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.908 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.909 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.909 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.909 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.909 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.909 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.909 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.909 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.909 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "stats", 2024-09-10T09:06:42.909 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.909 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.909 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.909 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.909 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.910 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.910 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.910 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.910 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.910 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.910 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.910 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.910 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.910 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.910 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.910 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.910 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.910 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.911 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.911 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.911 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.911 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.911 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.911 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.911 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.911 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.911 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.911 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.911 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.911 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.911 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.912 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.912 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.912 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.912 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.912 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.912 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.912 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.912 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.912 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.912 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.912 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.912 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.912 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.913 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.913 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.913 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.913 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.913 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.913 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.913 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.913 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.913 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.913 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.913 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.913 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.913 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.914 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.914 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.914 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.914 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.914 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.914 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.914 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.914 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.914 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.914 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.914 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.914 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.914 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.915 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.915 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.915 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.915 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.915 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.915 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.915 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "status", 2024-09-10T09:06:42.915 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.915 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.915 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.915 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.915 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.915 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.916 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.916 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.916 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.916 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.916 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.916 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.916 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.916 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.916 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.916 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.916 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.916 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.917 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.917 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.917 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.917 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.917 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.917 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.917 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.917 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.917 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.917 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.917 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.917 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.917 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.917 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.918 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.918 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.918 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.918 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.918 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.918 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.918 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.919 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.920 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.920 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.920 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.920 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.920 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.920 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.920 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.920 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.920 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.920 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.920 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.920 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.921 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.921 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.921 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.921 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.921 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.921 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.921 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.921 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.921 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.921 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.921 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.921 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.921 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.922 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.922 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.922 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.922 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.922 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.922 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.922 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.922 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.922 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.922 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.922 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.922 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "telegraf", 2024-09-10T09:06:42.922 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: "address": { 2024-09-10T09:06:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "address", 2024-09-10T09:06:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T09:06:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.923 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: "interval": { 2024-09-10T09:06:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "interval", 2024-09-10T09:06:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "15", 2024-09-10T09:06:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.924 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.925 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.925 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.925 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.925 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.925 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.925 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.925 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.925 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.925 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.925 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.925 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.925 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.925 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.926 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.926 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.926 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.926 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.926 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.926 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.926 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.926 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.926 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.926 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.926 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.926 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.926 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.927 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.927 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.927 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.927 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.927 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.927 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.927 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.927 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.927 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.927 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.927 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.927 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.927 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.927 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.928 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.928 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.928 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.928 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.928 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.928 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.928 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.928 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.928 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.928 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.928 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.928 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.929 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.929 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.929 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.929 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.929 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.929 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.929 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.929 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.929 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.929 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.929 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.929 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.929 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.929 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.930 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.930 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.930 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.930 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.930 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.930 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.930 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.930 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.930 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.930 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.930 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.930 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "telemetry", 2024-09-10T09:06:42.931 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.931 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.931 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.931 INFO:teuthology.orchestra.run.smithi172.stdout: "channel_basic": { 2024-09-10T09:06:42.931 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "channel_basic", 2024-09-10T09:06:42.931 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.931 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.931 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.931 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.931 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.931 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.931 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.931 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T09:06:42.932 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.932 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.932 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.932 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.932 INFO:teuthology.orchestra.run.smithi172.stdout: "channel_crash": { 2024-09-10T09:06:42.932 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "channel_crash", 2024-09-10T09:06:42.932 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.932 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.932 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.932 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.932 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.932 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.932 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.933 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T09:06:42.933 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.933 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.933 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.933 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.933 INFO:teuthology.orchestra.run.smithi172.stdout: "channel_device": { 2024-09-10T09:06:42.933 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "channel_device", 2024-09-10T09:06:42.933 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.933 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.933 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.933 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.933 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.934 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.934 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.934 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T09:06:42.934 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.934 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.934 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.934 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.934 INFO:teuthology.orchestra.run.smithi172.stdout: "channel_ident": { 2024-09-10T09:06:42.934 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "channel_ident", 2024-09-10T09:06:42.934 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.934 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.934 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.934 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.935 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.935 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.935 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.935 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T09:06:42.935 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.935 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.935 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.935 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.935 INFO:teuthology.orchestra.run.smithi172.stdout: "channel_perf": { 2024-09-10T09:06:42.935 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "channel_perf", 2024-09-10T09:06:42.935 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.935 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.935 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.935 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.936 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.936 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.936 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.936 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T09:06:42.936 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.936 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.936 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.936 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.936 INFO:teuthology.orchestra.run.smithi172.stdout: "contact": { 2024-09-10T09:06:42.936 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "contact", 2024-09-10T09:06:42.936 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.936 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.937 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.937 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.937 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.937 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.937 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.937 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.937 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.937 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.937 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.937 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.937 INFO:teuthology.orchestra.run.smithi172.stdout: "description": { 2024-09-10T09:06:42.937 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "description", 2024-09-10T09:06:42.937 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.938 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.938 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.938 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.938 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.938 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.938 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.938 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.938 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.938 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.938 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.938 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.938 INFO:teuthology.orchestra.run.smithi172.stdout: "device_url": { 2024-09-10T09:06:42.939 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "device_url", 2024-09-10T09:06:42.939 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.939 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.939 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.939 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T09:06:42.939 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.939 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.939 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.939 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.939 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.939 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.939 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.939 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.939 INFO:teuthology.orchestra.run.smithi172.stdout: "enabled": { 2024-09-10T09:06:42.940 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "enabled", 2024-09-10T09:06:42.940 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.940 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.940 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.940 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.940 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.940 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.940 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.940 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.940 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.940 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.940 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.940 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.941 INFO:teuthology.orchestra.run.smithi172.stdout: "interval": { 2024-09-10T09:06:42.941 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "interval", 2024-09-10T09:06:42.941 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.941 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.941 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.941 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "24", 2024-09-10T09:06:42.941 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "8", 2024-09-10T09:06:42.941 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.941 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.941 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.941 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.941 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.941 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.942 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.942 INFO:teuthology.orchestra.run.smithi172.stdout: "last_opt_revision": { 2024-09-10T09:06:42.942 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "last_opt_revision", 2024-09-10T09:06:42.942 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.942 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.942 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.942 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "1", 2024-09-10T09:06:42.942 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.942 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.942 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.942 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.942 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.942 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.943 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.943 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.943 INFO:teuthology.orchestra.run.smithi172.stdout: "leaderboard": { 2024-09-10T09:06:42.943 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "leaderboard", 2024-09-10T09:06:42.943 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.943 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.943 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.943 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.943 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.943 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.943 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.943 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.943 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.943 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.944 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.944 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.944 INFO:teuthology.orchestra.run.smithi172.stdout: "leaderboard_description": { 2024-09-10T09:06:42.944 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "leaderboard_description", 2024-09-10T09:06:42.944 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.944 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.944 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.944 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.944 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.944 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.944 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.944 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.944 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.945 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.945 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.945 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.945 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.945 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.945 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.945 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.945 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.945 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.945 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.945 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.945 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.946 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.946 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.946 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.946 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.946 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.946 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.946 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.946 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.946 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.946 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.946 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.946 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.946 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.947 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.947 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.947 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.947 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.947 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.947 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.947 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.947 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.947 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.947 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.947 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.947 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.947 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.948 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.948 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.948 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.948 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.948 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.948 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.948 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.948 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.948 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.948 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.948 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.948 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.948 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.949 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.949 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.949 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.949 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.949 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.949 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.949 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.949 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.949 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.949 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.949 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.949 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.949 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.950 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.950 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.950 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.950 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.950 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.950 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.950 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.950 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.950 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.950 INFO:teuthology.orchestra.run.smithi172.stdout: "organization": { 2024-09-10T09:06:42.950 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "organization", 2024-09-10T09:06:42.950 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.950 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.950 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.951 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.951 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.951 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.951 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.951 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.951 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.951 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.951 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.951 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.951 INFO:teuthology.orchestra.run.smithi172.stdout: "proxy": { 2024-09-10T09:06:42.951 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "proxy", 2024-09-10T09:06:42.951 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.951 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.952 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.952 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.952 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.952 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.952 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.952 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.952 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.952 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.952 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.952 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.952 INFO:teuthology.orchestra.run.smithi172.stdout: "url": { 2024-09-10T09:06:42.952 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "url", 2024-09-10T09:06:42.953 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.953 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.953 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.953 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T09:06:42.953 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.953 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.953 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.953 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.953 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.953 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.953 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.953 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.954 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.954 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.954 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.954 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "test_orchestrator", 2024-09-10T09:06:42.954 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.954 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.954 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.954 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.954 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.954 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.954 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.954 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.954 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.955 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.955 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.955 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.955 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.955 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.955 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.955 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.955 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.955 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.955 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.955 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.955 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.955 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.956 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.956 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.956 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.956 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.956 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.956 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.956 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.956 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.956 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.956 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.956 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.957 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.957 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.957 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.957 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.957 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.957 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.957 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.957 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.957 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.957 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.957 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.957 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.957 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.958 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.958 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.958 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.958 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.958 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.958 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.958 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.958 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.958 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.958 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.958 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.958 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.958 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.959 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.959 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.959 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.959 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.959 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.959 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.959 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.959 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.959 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.959 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.959 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.959 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.959 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.960 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.960 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.960 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.960 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.960 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "volumes", 2024-09-10T09:06:42.960 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.960 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.960 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.960 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.960 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.960 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.960 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.961 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.961 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.961 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.961 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.961 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.961 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.961 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.961 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.961 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.961 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.961 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.961 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.961 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.962 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.962 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.962 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.962 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.962 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.962 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.962 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.962 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.962 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.962 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.962 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.962 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.962 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.963 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.963 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.963 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.963 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.963 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.963 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.963 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.963 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.963 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.963 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.963 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.963 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.963 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.964 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.964 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.964 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.964 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.964 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.964 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.964 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.964 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.964 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.964 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.964 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.964 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.964 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.965 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.965 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.965 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.965 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.965 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.965 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.965 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.965 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.965 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.965 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.965 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.965 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.965 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.966 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.966 INFO:teuthology.orchestra.run.smithi172.stdout: "max_concurrent_clones": { 2024-09-10T09:06:42.966 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "max_concurrent_clones", 2024-09-10T09:06:42.966 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.966 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.966 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.966 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "4", 2024-09-10T09:06:42.966 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.966 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.966 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.966 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T09:06:42.966 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.967 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.967 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.967 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.967 INFO:teuthology.orchestra.run.smithi172.stdout: "periodic_async_work": { 2024-09-10T09:06:42.967 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "periodic_async_work", 2024-09-10T09:06:42.967 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.967 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.967 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.967 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.967 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.967 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.967 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.968 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Periodically check for async work", 2024-09-10T09:06:42.968 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.968 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.968 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.968 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.968 INFO:teuthology.orchestra.run.smithi172.stdout: "snapshot_clone_delay": { 2024-09-10T09:06:42.968 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "snapshot_clone_delay", 2024-09-10T09:06:42.968 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.968 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.968 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.968 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0", 2024-09-10T09:06:42.968 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.969 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.969 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.969 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T09:06:42.969 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.969 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.969 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.969 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.969 INFO:teuthology.orchestra.run.smithi172.stdout: "snapshot_clone_no_wait": { 2024-09-10T09:06:42.969 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T09:06:42.969 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.969 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.969 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.969 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.970 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.970 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.970 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.970 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T09:06:42.970 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.970 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.970 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.970 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.970 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.970 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.970 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.970 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "zabbix", 2024-09-10T09:06:42.970 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.971 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.971 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.971 INFO:teuthology.orchestra.run.smithi172.stdout: "discovery_interval": { 2024-09-10T09:06:42.971 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "discovery_interval", 2024-09-10T09:06:42.971 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "uint", 2024-09-10T09:06:42.971 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.971 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.971 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "100", 2024-09-10T09:06:42.971 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.971 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.971 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.971 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.971 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.972 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.972 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.972 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.972 INFO:teuthology.orchestra.run.smithi172.stdout: "identifier": { 2024-09-10T09:06:42.972 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "identifier", 2024-09-10T09:06:42.972 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.972 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.972 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.972 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.972 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.972 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.972 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.972 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.973 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.973 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.973 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.973 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.973 INFO:teuthology.orchestra.run.smithi172.stdout: "interval": { 2024-09-10T09:06:42.973 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "interval", 2024-09-10T09:06:42.973 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.973 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.973 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.973 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "60", 2024-09-10T09:06:42.973 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.973 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.973 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.974 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.974 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.974 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.974 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.974 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.974 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.974 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.974 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.974 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.974 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.974 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.974 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.974 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.975 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.975 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.975 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.975 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.975 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.975 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.975 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.975 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.975 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.975 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.975 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.975 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.975 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.976 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.976 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.976 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.976 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.976 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.976 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.976 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.976 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.976 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.976 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.976 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.976 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.976 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.977 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.977 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.977 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.977 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.977 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.977 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.977 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.977 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.977 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.977 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.977 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.977 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.977 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.978 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.978 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.978 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.978 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.978 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.978 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.978 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.978 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.978 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.978 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.978 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.978 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.978 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.978 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.979 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.979 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.979 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.979 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.979 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.979 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.979 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.979 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.979 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.979 INFO:teuthology.orchestra.run.smithi172.stdout: "zabbix_host": { 2024-09-10T09:06:42.979 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "zabbix_host", 2024-09-10T09:06:42.979 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.980 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.980 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.980 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.980 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.980 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.980 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.980 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.980 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.980 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.980 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.980 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.980 INFO:teuthology.orchestra.run.smithi172.stdout: "zabbix_port": { 2024-09-10T09:06:42.980 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "zabbix_port", 2024-09-10T09:06:42.980 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.981 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.981 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.981 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10051", 2024-09-10T09:06:42.981 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.981 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.981 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.981 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.981 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.981 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.981 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.981 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.981 INFO:teuthology.orchestra.run.smithi172.stdout: "zabbix_sender": { 2024-09-10T09:06:42.981 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "zabbix_sender", 2024-09-10T09:06:42.982 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.982 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.982 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:42.982 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T09:06:42.982 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.982 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.982 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.982 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.982 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.982 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.982 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.982 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.983 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.983 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.983 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-09-10T09:06:42.983 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.983 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.983 INFO:teuthology.orchestra.run.smithi172.stdout: "gid": 24110, 2024-09-10T09:06:42.983 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "y", 2024-09-10T09:06:42.983 INFO:teuthology.orchestra.run.smithi172.stdout: "mgr_features": 4540701547738038271, 2024-09-10T09:06:42.983 INFO:teuthology.orchestra.run.smithi172.stdout: "available_modules": [ 2024-09-10T09:06:42.983 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.983 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "alerts", 2024-09-10T09:06:42.983 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.983 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.984 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.984 INFO:teuthology.orchestra.run.smithi172.stdout: "interval": { 2024-09-10T09:06:42.984 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "interval", 2024-09-10T09:06:42.984 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:42.984 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.984 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.984 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "60", 2024-09-10T09:06:42.984 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.984 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.984 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.984 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T09:06:42.984 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.984 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.985 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.985 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.985 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:42.985 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:42.985 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.985 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.985 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.985 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.985 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.985 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.985 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.985 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.985 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.986 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.986 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.986 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.986 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.986 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.986 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.986 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.986 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.986 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.986 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.986 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:42.986 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:42.986 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.986 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.987 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.987 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.987 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.987 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.987 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.987 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.987 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.987 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.987 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.987 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.987 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:42.987 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:42.987 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.988 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.988 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.988 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:42.988 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.988 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.988 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:42.988 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:42.988 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:42.988 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:42.988 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:42.988 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:42.988 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:42.988 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:42.988 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.989 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.989 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.989 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.989 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.989 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:42.989 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:42.989 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.989 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.989 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.989 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:42.989 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.989 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.990 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.990 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:42.990 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.990 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.990 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.990 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.990 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_destination": { 2024-09-10T09:06:42.990 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_destination", 2024-09-10T09:06:42.990 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.990 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.990 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.990 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.990 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.991 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.991 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.991 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Email address to send alerts to", 2024-09-10T09:06:42.991 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.991 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.991 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.991 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.991 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_from_name": { 2024-09-10T09:06:42.991 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_from_name", 2024-09-10T09:06:42.991 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.991 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.991 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.991 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "Ceph", 2024-09-10T09:06:42.992 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.992 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.992 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.992 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Email From: name", 2024-09-10T09:06:42.992 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.992 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.992 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.992 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.992 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_host": { 2024-09-10T09:06:42.992 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_host", 2024-09-10T09:06:42.992 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.992 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.993 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.993 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.993 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.993 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.993 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.993 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "SMTP server", 2024-09-10T09:06:42.993 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.993 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.993 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.993 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.993 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_password": { 2024-09-10T09:06:42.993 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_password", 2024-09-10T09:06:42.994 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.994 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.994 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.994 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.994 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.994 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.994 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.994 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Password to authenticate with", 2024-09-10T09:06:42.994 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.994 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.994 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.994 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.994 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_port": { 2024-09-10T09:06:42.994 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_port", 2024-09-10T09:06:42.995 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:42.995 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.995 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.995 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "465", 2024-09-10T09:06:42.995 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.995 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.995 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.995 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "SMTP port", 2024-09-10T09:06:42.995 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.995 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.995 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.995 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.995 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_sender": { 2024-09-10T09:06:42.996 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_sender", 2024-09-10T09:06:42.996 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.996 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.996 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.996 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.996 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.996 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.996 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.996 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "SMTP envelope sender", 2024-09-10T09:06:42.996 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.996 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.996 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.996 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.997 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_ssl": { 2024-09-10T09:06:42.997 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_ssl", 2024-09-10T09:06:42.997 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.997 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.997 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.997 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:42.997 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.997 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.997 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.997 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T09:06:42.997 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.997 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.997 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.998 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.998 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_user": { 2024-09-10T09:06:42.998 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_user", 2024-09-10T09:06:42.998 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:42.998 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.998 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:42.998 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:42.998 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:42.998 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:42.998 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:42.998 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "User to authenticate as", 2024-09-10T09:06:42.998 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:42.998 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:42.999 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:42.999 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.999 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:42.999 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:42.999 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:42.999 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "balancer", 2024-09-10T09:06:42.999 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:42.999 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:42.999 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:42.999 INFO:teuthology.orchestra.run.smithi172.stdout: "active": { 2024-09-10T09:06:42.999 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "active", 2024-09-10T09:06:42.999 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:42.999 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:42.999 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.000 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.000 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.000 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.000 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.000 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T09:06:43.000 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.000 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.000 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.000 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.000 INFO:teuthology.orchestra.run.smithi172.stdout: "begin_time": { 2024-09-10T09:06:43.000 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "begin_time", 2024-09-10T09:06:43.000 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.000 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.001 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.001 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0000", 2024-09-10T09:06:43.001 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.001 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.001 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.001 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T09:06:43.001 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T09:06:43.001 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.001 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.001 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.001 INFO:teuthology.orchestra.run.smithi172.stdout: "begin_weekday": { 2024-09-10T09:06:43.001 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "begin_weekday", 2024-09-10T09:06:43.002 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "uint", 2024-09-10T09:06:43.002 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.002 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.002 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0", 2024-09-10T09:06:43.002 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "0", 2024-09-10T09:06:43.002 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "6", 2024-09-10T09:06:43.002 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.002 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T09:06:43.002 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T09:06:43.002 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.002 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.002 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.003 INFO:teuthology.orchestra.run.smithi172.stdout: "crush_compat_max_iterations": { 2024-09-10T09:06:43.003 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "crush_compat_max_iterations", 2024-09-10T09:06:43.003 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "uint", 2024-09-10T09:06:43.003 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.003 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.003 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "25", 2024-09-10T09:06:43.003 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "1", 2024-09-10T09:06:43.003 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "250", 2024-09-10T09:06:43.003 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.003 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T09:06:43.003 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.003 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.004 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.004 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.004 INFO:teuthology.orchestra.run.smithi172.stdout: "crush_compat_metrics": { 2024-09-10T09:06:43.004 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "crush_compat_metrics", 2024-09-10T09:06:43.004 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.004 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.004 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.004 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T09:06:43.004 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.004 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.004 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.004 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T09:06:43.004 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T09:06:43.005 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.005 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.005 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.005 INFO:teuthology.orchestra.run.smithi172.stdout: "crush_compat_step": { 2024-09-10T09:06:43.005 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "crush_compat_step", 2024-09-10T09:06:43.005 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:43.005 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.005 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.005 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0.5", 2024-09-10T09:06:43.005 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "0.001", 2024-09-10T09:06:43.005 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "0.999", 2024-09-10T09:06:43.005 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.005 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "aggressiveness of optimization", 2024-09-10T09:06:43.006 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T09:06:43.006 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.006 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.006 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.006 INFO:teuthology.orchestra.run.smithi172.stdout: "end_time": { 2024-09-10T09:06:43.006 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "end_time", 2024-09-10T09:06:43.006 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.006 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.006 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.006 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "2359", 2024-09-10T09:06:43.006 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.006 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.006 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.007 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T09:06:43.007 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T09:06:43.007 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.007 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.007 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.007 INFO:teuthology.orchestra.run.smithi172.stdout: "end_weekday": { 2024-09-10T09:06:43.007 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "end_weekday", 2024-09-10T09:06:43.007 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "uint", 2024-09-10T09:06:43.007 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.007 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.007 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0", 2024-09-10T09:06:43.007 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "0", 2024-09-10T09:06:43.007 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "6", 2024-09-10T09:06:43.007 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.008 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T09:06:43.008 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T09:06:43.008 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.008 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.008 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.008 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.008 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.008 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.008 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.008 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.008 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.008 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.008 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.009 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.009 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.009 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.009 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.009 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.009 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.009 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.009 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.009 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.009 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.009 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.009 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.009 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.010 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.010 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.010 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.010 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.010 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.010 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.010 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.010 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.010 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.010 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.010 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.010 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.010 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.011 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.011 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.011 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.011 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.011 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.011 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.011 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.011 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.011 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.011 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.011 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.011 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.011 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.012 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.012 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.012 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.012 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.012 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.012 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.012 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.012 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.012 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.012 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.012 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.012 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.012 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.013 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.013 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.013 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.013 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.013 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.013 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.013 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.013 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.013 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.013 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.013 INFO:teuthology.orchestra.run.smithi172.stdout: "min_score": { 2024-09-10T09:06:43.013 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "min_score", 2024-09-10T09:06:43.013 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:43.014 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.014 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.014 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0", 2024-09-10T09:06:43.014 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.014 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.014 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.014 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T09:06:43.014 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.014 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.014 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.014 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.014 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": { 2024-09-10T09:06:43.014 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mode", 2024-09-10T09:06:43.015 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.015 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.015 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.015 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "upmap", 2024-09-10T09:06:43.015 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.015 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.015 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.015 INFO:teuthology.orchestra.run.smithi172.stdout: "crush-compat", 2024-09-10T09:06:43.015 INFO:teuthology.orchestra.run.smithi172.stdout: "none", 2024-09-10T09:06:43.015 INFO:teuthology.orchestra.run.smithi172.stdout: "read", 2024-09-10T09:06:43.015 INFO:teuthology.orchestra.run.smithi172.stdout: "upmap", 2024-09-10T09:06:43.015 INFO:teuthology.orchestra.run.smithi172.stdout: "upmap-read" 2024-09-10T09:06:43.016 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.016 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Balancer mode", 2024-09-10T09:06:43.016 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.016 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.016 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.016 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.016 INFO:teuthology.orchestra.run.smithi172.stdout: "pool_ids": { 2024-09-10T09:06:43.016 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "pool_ids", 2024-09-10T09:06:43.016 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.016 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.016 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.016 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.017 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.017 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.017 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.017 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T09:06:43.017 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.017 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.017 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.017 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.017 INFO:teuthology.orchestra.run.smithi172.stdout: "sleep_interval": { 2024-09-10T09:06:43.017 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "sleep_interval", 2024-09-10T09:06:43.017 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.017 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.018 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.018 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "60", 2024-09-10T09:06:43.018 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.018 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.018 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.018 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T09:06:43.018 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.018 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.018 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.018 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.018 INFO:teuthology.orchestra.run.smithi172.stdout: "upmap_max_deviation": { 2024-09-10T09:06:43.018 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "upmap_max_deviation", 2024-09-10T09:06:43.019 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.019 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.019 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.019 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "5", 2024-09-10T09:06:43.019 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "1", 2024-09-10T09:06:43.019 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.019 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.019 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T09:06:43.019 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T09:06:43.019 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.019 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.019 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.019 INFO:teuthology.orchestra.run.smithi172.stdout: "upmap_max_optimizations": { 2024-09-10T09:06:43.020 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "upmap_max_optimizations", 2024-09-10T09:06:43.020 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "uint", 2024-09-10T09:06:43.020 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.020 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.020 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:43.020 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.020 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.020 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.020 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T09:06:43.020 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.020 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.020 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.021 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.021 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.021 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.021 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.021 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "cephadm", 2024-09-10T09:06:43.021 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.021 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.021 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.021 INFO:teuthology.orchestra.run.smithi172.stdout: "agent_down_multiplier": { 2024-09-10T09:06:43.021 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "agent_down_multiplier", 2024-09-10T09:06:43.021 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:43.021 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.022 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.022 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "3.0", 2024-09-10T09:06:43.022 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.022 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.022 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.022 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T09:06:43.022 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.022 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.022 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.022 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.022 INFO:teuthology.orchestra.run.smithi172.stdout: "agent_refresh_rate": { 2024-09-10T09:06:43.022 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "agent_refresh_rate", 2024-09-10T09:06:43.022 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.023 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.023 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.023 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "20", 2024-09-10T09:06:43.023 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.023 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.023 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.023 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T09:06:43.023 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.023 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.023 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.023 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.023 INFO:teuthology.orchestra.run.smithi172.stdout: "agent_starting_port": { 2024-09-10T09:06:43.023 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "agent_starting_port", 2024-09-10T09:06:43.023 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.024 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.024 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.024 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "4721", 2024-09-10T09:06:43.024 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.024 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.024 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.024 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T09:06:43.024 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.024 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.024 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.024 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.024 INFO:teuthology.orchestra.run.smithi172.stdout: "allow_ptrace": { 2024-09-10T09:06:43.024 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "allow_ptrace", 2024-09-10T09:06:43.025 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.025 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.025 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.025 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.025 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.025 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.025 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.025 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T09:06:43.025 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T09:06:43.025 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.025 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.025 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.026 INFO:teuthology.orchestra.run.smithi172.stdout: "autotune_interval": { 2024-09-10T09:06:43.026 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "autotune_interval", 2024-09-10T09:06:43.026 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.026 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.026 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.026 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "600", 2024-09-10T09:06:43.026 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.026 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.026 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.026 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T09:06:43.026 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.026 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.026 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.027 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.027 INFO:teuthology.orchestra.run.smithi172.stdout: "autotune_memory_target_ratio": { 2024-09-10T09:06:43.027 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T09:06:43.027 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:43.027 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.027 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.027 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0.7", 2024-09-10T09:06:43.027 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.027 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.027 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.027 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T09:06:43.027 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.027 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.028 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.028 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.028 INFO:teuthology.orchestra.run.smithi172.stdout: "cephadm_log_destination": { 2024-09-10T09:06:43.028 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "cephadm_log_destination", 2024-09-10T09:06:43.028 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.028 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.028 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.028 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.028 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.028 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.028 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.028 INFO:teuthology.orchestra.run.smithi172.stdout: "file", 2024-09-10T09:06:43.028 INFO:teuthology.orchestra.run.smithi172.stdout: "file,syslog", 2024-09-10T09:06:43.028 INFO:teuthology.orchestra.run.smithi172.stdout: "syslog" 2024-09-10T09:06:43.029 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.029 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T09:06:43.029 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.029 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.029 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.029 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.029 INFO:teuthology.orchestra.run.smithi172.stdout: "cgroups_split": { 2024-09-10T09:06:43.029 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "cgroups_split", 2024-09-10T09:06:43.029 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.029 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.029 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.029 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.029 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.030 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.030 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.030 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T09:06:43.030 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.030 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.030 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.030 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.030 INFO:teuthology.orchestra.run.smithi172.stdout: "config_checks_enabled": { 2024-09-10T09:06:43.030 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "config_checks_enabled", 2024-09-10T09:06:43.030 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.030 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.030 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.030 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.031 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.031 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.031 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.031 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T09:06:43.031 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.031 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.031 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.031 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.031 INFO:teuthology.orchestra.run.smithi172.stdout: "config_dashboard": { 2024-09-10T09:06:43.031 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "config_dashboard", 2024-09-10T09:06:43.031 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.031 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.032 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.032 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.032 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.032 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.032 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.032 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T09:06:43.032 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.032 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.032 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.032 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.032 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_alertmanager": { 2024-09-10T09:06:43.032 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_alertmanager", 2024-09-10T09:06:43.032 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.033 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.033 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.033 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T09:06:43.033 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.033 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.033 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.033 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Prometheus container image", 2024-09-10T09:06:43.033 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.033 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.033 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.033 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.033 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_base": { 2024-09-10T09:06:43.033 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_base", 2024-09-10T09:06:43.033 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.034 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.034 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.034 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T09:06:43.034 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.034 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.034 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.034 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Container image name, without the tag", 2024-09-10T09:06:43.034 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.034 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.034 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.034 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.034 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_elasticsearch": { 2024-09-10T09:06:43.034 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_elasticsearch", 2024-09-10T09:06:43.034 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.035 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.035 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.035 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T09:06:43.035 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.035 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.035 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.035 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "elasticsearch container image", 2024-09-10T09:06:43.035 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.035 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.035 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.035 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.035 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_grafana": { 2024-09-10T09:06:43.035 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_grafana", 2024-09-10T09:06:43.035 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.035 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.036 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.036 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T09:06:43.036 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.036 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.036 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.036 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Prometheus container image", 2024-09-10T09:06:43.036 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.036 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.036 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.036 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.036 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_haproxy": { 2024-09-10T09:06:43.036 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_haproxy", 2024-09-10T09:06:43.036 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.036 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.037 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.037 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T09:06:43.037 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.037 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.037 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.037 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "HAproxy container image", 2024-09-10T09:06:43.037 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.037 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.037 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.037 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.037 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_jaeger_agent": { 2024-09-10T09:06:43.037 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_jaeger_agent", 2024-09-10T09:06:43.037 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.037 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.038 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.038 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T09:06:43.038 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.038 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.038 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.038 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Jaeger agent container image", 2024-09-10T09:06:43.038 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.038 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.038 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.038 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.038 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_jaeger_collector": { 2024-09-10T09:06:43.038 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_jaeger_collector", 2024-09-10T09:06:43.038 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.038 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.039 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.039 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T09:06:43.039 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.039 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.039 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.039 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Jaeger collector container image", 2024-09-10T09:06:43.039 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.039 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.039 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.039 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.039 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_jaeger_query": { 2024-09-10T09:06:43.039 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_jaeger_query", 2024-09-10T09:06:43.039 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.039 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.040 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.040 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T09:06:43.040 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.040 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.040 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.040 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Jaeger query container image", 2024-09-10T09:06:43.040 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.040 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.040 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.040 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.040 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_keepalived": { 2024-09-10T09:06:43.040 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_keepalived", 2024-09-10T09:06:43.040 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.040 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.041 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.041 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T09:06:43.041 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.041 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.041 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.041 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Keepalived container image", 2024-09-10T09:06:43.041 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.041 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.041 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.041 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.041 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_loki": { 2024-09-10T09:06:43.041 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_loki", 2024-09-10T09:06:43.041 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.042 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.042 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.042 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T09:06:43.042 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.042 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.042 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.042 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Loki container image", 2024-09-10T09:06:43.042 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.042 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.042 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.042 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.042 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_node_exporter": { 2024-09-10T09:06:43.042 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_node_exporter", 2024-09-10T09:06:43.042 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.043 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.043 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.043 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T09:06:43.043 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.043 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.043 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.043 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Prometheus container image", 2024-09-10T09:06:43.043 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.043 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.043 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.043 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.043 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_nvmeof": { 2024-09-10T09:06:43.043 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_nvmeof", 2024-09-10T09:06:43.043 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.044 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.044 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.044 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T09:06:43.044 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.044 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.044 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.044 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Nvme-of container image", 2024-09-10T09:06:43.044 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.044 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.044 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.044 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.044 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_prometheus": { 2024-09-10T09:06:43.044 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_prometheus", 2024-09-10T09:06:43.044 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.045 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.045 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.045 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T09:06:43.045 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.045 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.045 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.045 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Prometheus container image", 2024-09-10T09:06:43.045 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.045 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.045 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.045 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.045 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_promtail": { 2024-09-10T09:06:43.045 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_promtail", 2024-09-10T09:06:43.046 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.046 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.046 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.046 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T09:06:43.046 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.046 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.046 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.046 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Promtail container image", 2024-09-10T09:06:43.046 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.046 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.046 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.046 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.046 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_samba": { 2024-09-10T09:06:43.046 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_samba", 2024-09-10T09:06:43.047 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.047 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.047 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.047 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T09:06:43.047 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.047 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.047 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.047 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Samba/SMB container image", 2024-09-10T09:06:43.047 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.047 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.047 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.047 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.047 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_snmp_gateway": { 2024-09-10T09:06:43.047 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_snmp_gateway", 2024-09-10T09:06:43.047 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.048 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.048 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.048 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T09:06:43.048 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.048 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.048 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.048 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "SNMP Gateway container image", 2024-09-10T09:06:43.048 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.048 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.048 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.048 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.048 INFO:teuthology.orchestra.run.smithi172.stdout: "container_init": { 2024-09-10T09:06:43.048 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_init", 2024-09-10T09:06:43.049 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.049 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.049 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.049 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.049 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.049 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.049 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.049 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T09:06:43.049 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.049 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.049 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.049 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.049 INFO:teuthology.orchestra.run.smithi172.stdout: "daemon_cache_timeout": { 2024-09-10T09:06:43.049 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "daemon_cache_timeout", 2024-09-10T09:06:43.049 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.050 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.050 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.050 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "600", 2024-09-10T09:06:43.050 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.050 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.050 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.050 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T09:06:43.050 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.050 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.050 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.050 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.050 INFO:teuthology.orchestra.run.smithi172.stdout: "default_cephadm_command_timeout": { 2024-09-10T09:06:43.051 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T09:06:43.051 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.051 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.051 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.051 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "900", 2024-09-10T09:06:43.051 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.051 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.051 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.051 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T09:06:43.051 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.051 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.051 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.051 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.052 INFO:teuthology.orchestra.run.smithi172.stdout: "default_registry": { 2024-09-10T09:06:43.052 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "default_registry", 2024-09-10T09:06:43.052 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.052 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.052 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.052 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "docker.io", 2024-09-10T09:06:43.052 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.052 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.052 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.052 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T09:06:43.052 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.052 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.052 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.053 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.053 INFO:teuthology.orchestra.run.smithi172.stdout: "device_cache_timeout": { 2024-09-10T09:06:43.053 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "device_cache_timeout", 2024-09-10T09:06:43.053 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.053 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.053 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.053 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "1800", 2024-09-10T09:06:43.053 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.053 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.053 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.053 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "seconds to cache device inventory", 2024-09-10T09:06:43.054 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.054 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.054 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.054 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.054 INFO:teuthology.orchestra.run.smithi172.stdout: "device_enhanced_scan": { 2024-09-10T09:06:43.054 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "device_enhanced_scan", 2024-09-10T09:06:43.054 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.054 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.054 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.054 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.054 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.054 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.054 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.054 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T09:06:43.055 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.055 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.055 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.055 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.055 INFO:teuthology.orchestra.run.smithi172.stdout: "facts_cache_timeout": { 2024-09-10T09:06:43.055 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "facts_cache_timeout", 2024-09-10T09:06:43.055 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.055 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.055 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.055 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "60", 2024-09-10T09:06:43.055 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.055 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.055 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.056 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "seconds to cache host facts data", 2024-09-10T09:06:43.056 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.056 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.056 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.056 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.056 INFO:teuthology.orchestra.run.smithi172.stdout: "grafana_dashboards_path": { 2024-09-10T09:06:43.056 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "grafana_dashboards_path", 2024-09-10T09:06:43.056 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.056 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.056 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.056 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T09:06:43.056 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.056 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.057 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.057 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T09:06:43.057 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.057 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.057 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.057 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.057 INFO:teuthology.orchestra.run.smithi172.stdout: "host_check_interval": { 2024-09-10T09:06:43.057 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "host_check_interval", 2024-09-10T09:06:43.057 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.057 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.057 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.057 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "600", 2024-09-10T09:06:43.057 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.058 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.058 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.058 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how frequently to perform a host check", 2024-09-10T09:06:43.058 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.058 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.058 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.058 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.058 INFO:teuthology.orchestra.run.smithi172.stdout: "hw_monitoring": { 2024-09-10T09:06:43.058 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "hw_monitoring", 2024-09-10T09:06:43.058 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.058 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.058 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.058 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.058 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.059 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.059 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.059 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T09:06:43.059 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.059 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.059 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.059 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.059 INFO:teuthology.orchestra.run.smithi172.stdout: "inventory_list_all": { 2024-09-10T09:06:43.059 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "inventory_list_all", 2024-09-10T09:06:43.059 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.059 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.059 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.059 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.059 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.060 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.060 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.060 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T09:06:43.060 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.060 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.060 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.060 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.060 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.060 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.060 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.060 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.060 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.060 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.061 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.061 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.061 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.061 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.061 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.061 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.061 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.061 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.061 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.061 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.061 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.061 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.062 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.062 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.062 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.062 INFO:teuthology.orchestra.run.smithi172.stdout: "log_refresh_metadata": { 2024-09-10T09:06:43.062 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_refresh_metadata", 2024-09-10T09:06:43.062 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.062 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.062 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.062 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.062 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.062 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.062 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.062 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T09:06:43.062 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.063 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.063 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.063 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.063 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.063 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.063 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.063 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.063 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.063 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.063 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.063 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.063 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.063 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T09:06:43.064 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.064 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.064 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.064 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.064 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.064 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.064 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.064 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.064 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.064 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.064 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.064 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.064 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.065 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.065 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.065 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.065 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.065 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.065 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.065 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.065 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.065 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.065 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.065 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.065 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.065 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.066 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.066 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.066 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.066 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.066 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.066 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.066 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.066 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.066 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.066 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.066 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.066 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.067 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.067 INFO:teuthology.orchestra.run.smithi172.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T09:06:43.067 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T09:06:43.067 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.067 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.067 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.067 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.067 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.067 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.067 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.067 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T09:06:43.067 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.067 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.068 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.068 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.068 INFO:teuthology.orchestra.run.smithi172.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T09:06:43.068 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T09:06:43.068 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.068 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.068 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.068 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "*", 2024-09-10T09:06:43.068 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.068 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.068 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.068 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T09:06:43.069 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.069 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.069 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.069 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.069 INFO:teuthology.orchestra.run.smithi172.stdout: "max_count_per_host": { 2024-09-10T09:06:43.069 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "max_count_per_host", 2024-09-10T09:06:43.069 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.069 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.069 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.069 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:43.069 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.069 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.069 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.070 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "max number of daemons per service per host", 2024-09-10T09:06:43.070 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.070 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.070 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.070 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.070 INFO:teuthology.orchestra.run.smithi172.stdout: "max_osd_draining_count": { 2024-09-10T09:06:43.070 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "max_osd_draining_count", 2024-09-10T09:06:43.070 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.070 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.070 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.070 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:43.070 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.070 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.071 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.071 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T09:06:43.071 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.071 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.071 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.071 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.071 INFO:teuthology.orchestra.run.smithi172.stdout: "migration_current": { 2024-09-10T09:06:43.071 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "migration_current", 2024-09-10T09:06:43.071 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.071 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.071 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.071 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.072 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.072 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.072 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.072 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "internal - do not modify", 2024-09-10T09:06:43.072 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.072 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.072 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.072 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.072 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": { 2024-09-10T09:06:43.072 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mode", 2024-09-10T09:06:43.072 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.072 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.072 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.073 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "root", 2024-09-10T09:06:43.073 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.073 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.073 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.073 INFO:teuthology.orchestra.run.smithi172.stdout: "cephadm-package", 2024-09-10T09:06:43.073 INFO:teuthology.orchestra.run.smithi172.stdout: "root" 2024-09-10T09:06:43.073 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.073 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T09:06:43.073 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.073 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.073 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.073 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.074 INFO:teuthology.orchestra.run.smithi172.stdout: "oob_default_addr": { 2024-09-10T09:06:43.074 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "oob_default_addr", 2024-09-10T09:06:43.074 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.074 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.074 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.074 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "169.254.1.1", 2024-09-10T09:06:43.074 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.074 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.074 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.074 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T09:06:43.074 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.074 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.074 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.075 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.075 INFO:teuthology.orchestra.run.smithi172.stdout: "prometheus_alerts_path": { 2024-09-10T09:06:43.075 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "prometheus_alerts_path", 2024-09-10T09:06:43.075 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.075 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.075 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.075 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T09:06:43.075 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.075 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.075 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.075 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T09:06:43.075 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.075 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.076 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.076 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.076 INFO:teuthology.orchestra.run.smithi172.stdout: "registry_insecure": { 2024-09-10T09:06:43.076 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "registry_insecure", 2024-09-10T09:06:43.076 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.076 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.076 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.076 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.076 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.076 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.076 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.076 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T09:06:43.076 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.077 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.077 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.077 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.077 INFO:teuthology.orchestra.run.smithi172.stdout: "registry_password": { 2024-09-10T09:06:43.077 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "registry_password", 2024-09-10T09:06:43.077 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.077 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.077 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.077 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.077 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.077 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.077 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.078 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T09:06:43.078 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.078 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.078 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.078 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.078 INFO:teuthology.orchestra.run.smithi172.stdout: "registry_url": { 2024-09-10T09:06:43.078 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "registry_url", 2024-09-10T09:06:43.078 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.078 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.078 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.078 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.078 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.079 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.079 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.079 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T09:06:43.079 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.079 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.079 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.079 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.079 INFO:teuthology.orchestra.run.smithi172.stdout: "registry_username": { 2024-09-10T09:06:43.079 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "registry_username", 2024-09-10T09:06:43.079 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.079 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.079 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.080 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.080 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.080 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.080 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.080 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T09:06:43.080 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.080 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.080 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.080 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.080 INFO:teuthology.orchestra.run.smithi172.stdout: "secure_monitoring_stack": { 2024-09-10T09:06:43.080 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "secure_monitoring_stack", 2024-09-10T09:06:43.080 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.080 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.081 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.081 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.081 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.081 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.081 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.081 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T09:06:43.081 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.081 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.081 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.081 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.081 INFO:teuthology.orchestra.run.smithi172.stdout: "service_discovery_port": { 2024-09-10T09:06:43.081 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "service_discovery_port", 2024-09-10T09:06:43.082 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.082 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.082 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.082 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "8765", 2024-09-10T09:06:43.082 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.082 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.082 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.082 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "cephadm service discovery port", 2024-09-10T09:06:43.082 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.082 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.082 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.082 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.082 INFO:teuthology.orchestra.run.smithi172.stdout: "ssh_config_file": { 2024-09-10T09:06:43.083 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ssh_config_file", 2024-09-10T09:06:43.083 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.083 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.083 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.083 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.083 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.083 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.083 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.083 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T09:06:43.083 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.083 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.083 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.083 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.084 INFO:teuthology.orchestra.run.smithi172.stdout: "use_agent": { 2024-09-10T09:06:43.084 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "use_agent", 2024-09-10T09:06:43.084 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.084 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.084 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.084 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.084 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.084 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.084 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.084 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T09:06:43.084 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.084 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.085 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.085 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.085 INFO:teuthology.orchestra.run.smithi172.stdout: "use_repo_digest": { 2024-09-10T09:06:43.085 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "use_repo_digest", 2024-09-10T09:06:43.085 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.085 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.085 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.085 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.085 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.085 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.085 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.085 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T09:06:43.085 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.086 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.086 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.086 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.086 INFO:teuthology.orchestra.run.smithi172.stdout: "warn_on_failed_host_check": { 2024-09-10T09:06:43.086 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "warn_on_failed_host_check", 2024-09-10T09:06:43.086 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.086 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.086 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.086 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.086 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.086 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.086 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.086 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T09:06:43.087 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.087 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.087 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.087 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.087 INFO:teuthology.orchestra.run.smithi172.stdout: "warn_on_stray_daemons": { 2024-09-10T09:06:43.087 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "warn_on_stray_daemons", 2024-09-10T09:06:43.087 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.087 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.087 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.087 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.087 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.087 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.087 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.088 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T09:06:43.088 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.088 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.088 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.088 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.088 INFO:teuthology.orchestra.run.smithi172.stdout: "warn_on_stray_hosts": { 2024-09-10T09:06:43.088 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "warn_on_stray_hosts", 2024-09-10T09:06:43.088 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.088 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.088 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.088 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.088 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.089 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.089 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.089 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T09:06:43.089 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.089 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.089 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.089 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.089 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.089 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.089 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.089 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "crash", 2024-09-10T09:06:43.089 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.089 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.090 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.090 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.090 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.090 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.090 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.090 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.090 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.090 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.090 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.090 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.090 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.090 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.090 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.091 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.091 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.091 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.091 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.091 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.091 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.091 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.091 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.091 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.091 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.091 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.091 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.092 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.092 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.092 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.092 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.092 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.092 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.092 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.092 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.092 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.092 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.092 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.092 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.092 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.093 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.093 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.093 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.093 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.093 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.093 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.093 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.093 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.093 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.093 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.093 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.093 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.094 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.094 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.094 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.094 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.094 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.094 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.094 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.094 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.094 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.094 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.094 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.094 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.094 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.095 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.095 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.095 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.095 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.095 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.095 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.095 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.095 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.095 INFO:teuthology.orchestra.run.smithi172.stdout: "retain_interval": { 2024-09-10T09:06:43.095 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "retain_interval", 2024-09-10T09:06:43.095 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.095 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.095 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.096 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "31536000", 2024-09-10T09:06:43.096 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.096 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.096 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.096 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T09:06:43.096 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.096 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.096 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.096 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.096 INFO:teuthology.orchestra.run.smithi172.stdout: "warn_recent_interval": { 2024-09-10T09:06:43.096 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "warn_recent_interval", 2024-09-10T09:06:43.096 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.096 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.097 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.097 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "1209600", 2024-09-10T09:06:43.097 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.097 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.097 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.097 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T09:06:43.097 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.097 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.097 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.097 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.097 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.097 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.097 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.098 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "dashboard", 2024-09-10T09:06:43.098 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.098 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.098 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.098 INFO:teuthology.orchestra.run.smithi172.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T09:06:43.098 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T09:06:43.098 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.098 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.098 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.098 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:43.098 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.098 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.098 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.098 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.099 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.099 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.099 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.099 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.099 INFO:teuthology.orchestra.run.smithi172.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T09:06:43.099 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T09:06:43.099 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.099 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.099 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.099 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.099 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.099 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.100 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.100 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.100 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.100 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.100 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.100 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.100 INFO:teuthology.orchestra.run.smithi172.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T09:06:43.100 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T09:06:43.100 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.100 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.100 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.100 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.101 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.101 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.101 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.101 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.101 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.101 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.101 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.101 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.101 INFO:teuthology.orchestra.run.smithi172.stdout: "AUDIT_API_ENABLED": { 2024-09-10T09:06:43.101 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T09:06:43.101 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.101 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.101 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.102 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.102 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.102 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.102 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.102 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.102 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.102 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.102 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.102 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.102 INFO:teuthology.orchestra.run.smithi172.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T09:06:43.102 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T09:06:43.102 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.103 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.103 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.103 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.103 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.103 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.103 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.103 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.103 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.103 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.103 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.103 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.103 INFO:teuthology.orchestra.run.smithi172.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T09:06:43.104 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T09:06:43.104 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.104 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.104 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.104 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.104 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.104 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.104 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.104 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.104 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.104 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.104 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.104 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.105 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T09:06:43.105 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T09:06:43.105 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.105 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.105 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.105 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.105 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.105 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.105 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.105 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.105 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.105 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.106 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.106 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.106 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T09:06:43.106 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T09:06:43.106 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.106 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.106 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.106 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.106 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.106 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.106 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.106 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.106 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.106 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.107 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.107 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.107 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T09:06:43.107 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T09:06:43.107 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.107 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.107 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.107 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.107 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.107 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.107 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.107 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.107 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.108 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.108 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.108 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.108 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T09:06:43.108 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T09:06:43.108 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.108 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.108 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.108 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.108 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.108 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.108 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.108 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.109 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.109 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.109 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.109 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.109 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T09:06:43.109 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T09:06:43.109 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.109 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.109 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.109 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.109 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.109 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.110 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.110 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.110 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.110 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.110 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.110 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.110 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T09:06:43.110 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T09:06:43.110 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.110 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.110 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.110 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.110 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.111 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.111 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.111 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.111 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.111 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.111 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.111 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.111 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T09:06:43.111 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T09:06:43.111 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.111 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.111 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.111 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.112 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.112 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.112 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.112 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.112 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.112 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.112 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.112 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.112 INFO:teuthology.orchestra.run.smithi172.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T09:06:43.112 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T09:06:43.112 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.112 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.113 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.113 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.113 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.113 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.113 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.113 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.113 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.113 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.113 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.113 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.113 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T09:06:43.113 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T09:06:43.113 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.114 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.114 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.114 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "admin", 2024-09-10T09:06:43.114 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.114 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.114 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.114 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.114 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.114 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.114 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.114 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.114 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T09:06:43.114 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T09:06:43.114 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.115 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.115 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.115 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.115 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.115 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.115 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.115 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.115 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.115 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.115 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.115 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.115 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_API_URL": { 2024-09-10T09:06:43.115 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_API_URL", 2024-09-10T09:06:43.116 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.116 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.116 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.116 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.116 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.116 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.116 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.116 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.116 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.116 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.116 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.116 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.116 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T09:06:43.117 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T09:06:43.117 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.117 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.117 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.117 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "admin", 2024-09-10T09:06:43.117 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.117 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.117 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.117 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.117 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.117 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.117 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.117 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.118 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T09:06:43.118 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T09:06:43.118 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.118 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.118 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.118 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.118 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.118 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.118 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.118 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.118 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.118 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.118 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.119 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.119 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T09:06:43.119 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T09:06:43.119 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.119 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.119 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.119 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.119 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.119 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.119 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.119 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.119 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.120 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.120 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.120 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.120 INFO:teuthology.orchestra.run.smithi172.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T09:06:43.120 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T09:06:43.120 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.120 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.120 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.120 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.120 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.120 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.120 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.121 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.121 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.121 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.121 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.121 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.121 INFO:teuthology.orchestra.run.smithi172.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T09:06:43.121 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T09:06:43.121 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.121 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.121 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.121 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.121 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.121 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.122 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.122 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.122 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.122 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.122 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.122 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.122 INFO:teuthology.orchestra.run.smithi172.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T09:06:43.122 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T09:06:43.122 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.122 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.122 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.122 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.122 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.123 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.123 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.123 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.123 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.123 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.123 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.123 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.123 INFO:teuthology.orchestra.run.smithi172.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T09:06:43.123 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T09:06:43.123 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.123 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.123 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.124 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.124 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.124 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.124 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.124 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.124 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.124 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.124 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.124 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.124 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T09:06:43.124 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T09:06:43.124 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.124 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.124 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.125 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.125 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.125 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.125 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.125 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.125 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.125 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.125 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.125 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.125 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T09:06:43.125 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T09:06:43.125 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.126 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.126 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.126 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.126 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.126 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.126 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.126 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.127 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.128 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.128 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.128 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.128 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T09:06:43.128 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T09:06:43.128 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.128 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.128 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.128 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.128 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.129 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.129 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.129 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.129 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.129 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.129 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.129 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.129 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T09:06:43.129 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T09:06:43.129 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.129 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.129 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.130 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.130 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.130 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.130 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.130 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.130 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.130 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.130 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.130 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.130 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T09:06:43.130 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T09:06:43.130 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.130 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.131 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.131 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.131 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.131 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.131 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.131 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.131 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.131 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.131 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.131 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.131 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T09:06:43.131 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T09:06:43.131 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.132 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.132 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.132 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.132 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.132 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.132 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.132 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.132 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.132 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.132 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.132 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.132 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T09:06:43.132 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T09:06:43.133 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.133 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.133 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.133 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.133 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.133 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.133 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.133 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.133 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.133 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.133 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.133 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.133 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T09:06:43.134 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T09:06:43.134 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.134 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.134 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.134 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.134 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.134 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.134 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.134 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.134 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.134 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.134 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.134 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.134 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T09:06:43.134 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T09:06:43.135 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.135 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.135 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.135 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T09:06:43.135 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.135 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.135 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.135 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.135 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.135 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.135 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.136 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.136 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T09:06:43.136 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T09:06:43.136 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.136 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.136 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.136 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:43.136 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.136 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.136 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.136 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.136 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.136 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.137 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.137 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.137 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T09:06:43.137 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T09:06:43.137 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.137 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.137 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.137 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "8", 2024-09-10T09:06:43.137 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.137 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.137 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.137 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.137 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.138 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.138 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.138 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.138 INFO:teuthology.orchestra.run.smithi172.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T09:06:43.138 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T09:06:43.138 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.138 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.138 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.138 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "45", 2024-09-10T09:06:43.138 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.138 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.138 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.138 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.138 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.139 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.139 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.139 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.139 INFO:teuthology.orchestra.run.smithi172.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T09:06:43.139 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T09:06:43.139 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.139 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.139 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.139 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.139 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.139 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.139 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.139 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.139 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.140 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.140 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.140 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.140 INFO:teuthology.orchestra.run.smithi172.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T09:06:43.140 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T09:06:43.140 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.140 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.140 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.140 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "admin", 2024-09-10T09:06:43.140 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.140 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.140 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.141 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.141 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.141 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.141 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.141 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.141 INFO:teuthology.orchestra.run.smithi172.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T09:06:43.141 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T09:06:43.141 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.141 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.141 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.141 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.141 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.141 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.141 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.142 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.142 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.142 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.142 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.142 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.142 INFO:teuthology.orchestra.run.smithi172.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T09:06:43.142 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T09:06:43.142 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.142 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.142 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.142 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.142 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.142 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.143 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.143 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.143 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.143 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.143 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.143 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.143 INFO:teuthology.orchestra.run.smithi172.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T09:06:43.143 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T09:06:43.143 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.143 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.143 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.143 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0", 2024-09-10T09:06:43.143 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.143 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.144 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.144 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.144 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.144 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.144 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.144 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.144 INFO:teuthology.orchestra.run.smithi172.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T09:06:43.144 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T09:06:43.144 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.144 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.144 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.144 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:43.144 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.145 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.145 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.145 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.145 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.145 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.145 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.145 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.145 INFO:teuthology.orchestra.run.smithi172.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T09:06:43.145 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T09:06:43.145 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.145 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.145 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.145 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "5", 2024-09-10T09:06:43.146 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.146 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.146 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.146 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.146 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.146 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.146 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.146 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.146 INFO:teuthology.orchestra.run.smithi172.stdout: "cross_origin_url": { 2024-09-10T09:06:43.146 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "cross_origin_url", 2024-09-10T09:06:43.146 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.146 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.146 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.146 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.147 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.147 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.147 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.147 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.147 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.147 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.147 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.147 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.147 INFO:teuthology.orchestra.run.smithi172.stdout: "crt_file": { 2024-09-10T09:06:43.147 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "crt_file", 2024-09-10T09:06:43.147 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.147 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.148 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.148 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.148 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.148 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.148 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.148 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.148 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.148 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.148 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.148 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.148 INFO:teuthology.orchestra.run.smithi172.stdout: "debug": { 2024-09-10T09:06:43.148 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "debug", 2024-09-10T09:06:43.148 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.149 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.149 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.149 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.149 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.149 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.149 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.149 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Enable/disable debug options", 2024-09-10T09:06:43.149 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.149 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.149 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.149 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.149 INFO:teuthology.orchestra.run.smithi172.stdout: "jwt_token_ttl": { 2024-09-10T09:06:43.150 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "jwt_token_ttl", 2024-09-10T09:06:43.150 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.150 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.150 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.150 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "28800", 2024-09-10T09:06:43.150 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.150 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.150 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.150 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.150 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.150 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.150 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.150 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.151 INFO:teuthology.orchestra.run.smithi172.stdout: "key_file": { 2024-09-10T09:06:43.151 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "key_file", 2024-09-10T09:06:43.151 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.151 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.151 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.151 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.151 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.151 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.151 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.151 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.151 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.151 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.151 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.152 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.152 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.152 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.152 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.152 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.152 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.152 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.152 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.152 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.152 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.153 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.153 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.153 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.153 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.153 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.153 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.153 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.153 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.153 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.153 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.153 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.153 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.153 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.154 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.154 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.154 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.154 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.154 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.154 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.154 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.154 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.154 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.154 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.154 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.154 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.155 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.155 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.155 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.155 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.155 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.155 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.155 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.155 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.155 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.155 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.155 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.155 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.155 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.156 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.156 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.156 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.156 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.156 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.156 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.156 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.156 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.156 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.156 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.156 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.156 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.157 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.157 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.157 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.157 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.157 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.157 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.157 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.157 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.157 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.157 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.157 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.157 INFO:teuthology.orchestra.run.smithi172.stdout: "motd": { 2024-09-10T09:06:43.157 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "motd", 2024-09-10T09:06:43.158 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.158 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.158 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.158 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.158 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.158 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.158 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.158 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "The message of the day", 2024-09-10T09:06:43.158 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.158 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.158 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.158 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.158 INFO:teuthology.orchestra.run.smithi172.stdout: "redirect_resolve_ip_addr": { 2024-09-10T09:06:43.159 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T09:06:43.159 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.159 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.159 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.159 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.159 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.159 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.159 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.159 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.159 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.159 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.159 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.159 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.159 INFO:teuthology.orchestra.run.smithi172.stdout: "server_addr": { 2024-09-10T09:06:43.160 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_addr", 2024-09-10T09:06:43.160 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.160 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.160 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.160 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "::", 2024-09-10T09:06:43.160 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.160 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.160 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.160 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.160 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.160 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.160 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.160 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.161 INFO:teuthology.orchestra.run.smithi172.stdout: "server_port": { 2024-09-10T09:06:43.161 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_port", 2024-09-10T09:06:43.161 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.161 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.161 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.161 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "8080", 2024-09-10T09:06:43.161 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.161 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.161 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.161 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.161 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.161 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.162 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.162 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.162 INFO:teuthology.orchestra.run.smithi172.stdout: "ssl": { 2024-09-10T09:06:43.162 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ssl", 2024-09-10T09:06:43.162 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.162 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.162 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.162 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.162 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.162 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.162 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.162 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.162 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.163 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.163 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.163 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.163 INFO:teuthology.orchestra.run.smithi172.stdout: "ssl_server_port": { 2024-09-10T09:06:43.163 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ssl_server_port", 2024-09-10T09:06:43.163 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.163 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.163 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.163 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "8443", 2024-09-10T09:06:43.163 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.163 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.163 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.164 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.164 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.164 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.164 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.164 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.164 INFO:teuthology.orchestra.run.smithi172.stdout: "standby_behaviour": { 2024-09-10T09:06:43.164 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "standby_behaviour", 2024-09-10T09:06:43.164 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.164 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.164 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.164 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "redirect", 2024-09-10T09:06:43.164 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.164 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.165 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.165 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.165 INFO:teuthology.orchestra.run.smithi172.stdout: "redirect" 2024-09-10T09:06:43.165 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.165 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.165 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.165 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.165 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.165 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.165 INFO:teuthology.orchestra.run.smithi172.stdout: "standby_error_status_code": { 2024-09-10T09:06:43.165 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "standby_error_status_code", 2024-09-10T09:06:43.166 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.166 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.166 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.166 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "500", 2024-09-10T09:06:43.166 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "400", 2024-09-10T09:06:43.166 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "599", 2024-09-10T09:06:43.166 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.166 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.166 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.166 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.166 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.166 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.166 INFO:teuthology.orchestra.run.smithi172.stdout: "url_prefix": { 2024-09-10T09:06:43.167 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "url_prefix", 2024-09-10T09:06:43.167 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.167 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.167 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.167 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.167 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.167 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.167 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.167 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.167 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.167 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.167 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.167 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.168 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.168 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.168 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.168 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "devicehealth", 2024-09-10T09:06:43.168 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.168 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.168 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.168 INFO:teuthology.orchestra.run.smithi172.stdout: "enable_monitoring": { 2024-09-10T09:06:43.168 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "enable_monitoring", 2024-09-10T09:06:43.168 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.168 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.168 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.169 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.169 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.169 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.169 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.169 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "monitor device health metrics", 2024-09-10T09:06:43.169 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.169 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.169 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.169 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.169 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.169 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.169 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.169 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.169 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.170 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.170 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.170 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.170 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.170 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.170 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.170 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.170 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.170 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.170 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.170 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.170 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.171 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.171 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.171 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.171 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.171 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.171 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.171 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.171 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.171 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.171 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.171 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.171 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.172 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.172 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.172 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.172 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.172 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.172 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.172 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.172 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.172 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.172 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.172 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.172 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.172 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.173 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.173 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.173 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.173 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.173 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.173 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.173 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.173 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.173 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.173 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.173 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.173 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.173 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.174 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.174 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.174 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.174 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.174 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.174 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.174 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.174 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.174 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.174 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.174 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.174 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.174 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.175 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.175 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.175 INFO:teuthology.orchestra.run.smithi172.stdout: "mark_out_threshold": { 2024-09-10T09:06:43.175 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mark_out_threshold", 2024-09-10T09:06:43.175 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.175 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.175 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.175 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "2419200", 2024-09-10T09:06:43.175 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.175 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.175 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.175 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T09:06:43.175 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.175 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.176 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.176 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.176 INFO:teuthology.orchestra.run.smithi172.stdout: "pool_name": { 2024-09-10T09:06:43.176 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "pool_name", 2024-09-10T09:06:43.176 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.176 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.176 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.176 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "device_health_metrics", 2024-09-10T09:06:43.176 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.176 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.176 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.176 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T09:06:43.177 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.177 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.177 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.177 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.177 INFO:teuthology.orchestra.run.smithi172.stdout: "retention_period": { 2024-09-10T09:06:43.177 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "retention_period", 2024-09-10T09:06:43.177 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.177 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.177 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.177 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "15552000", 2024-09-10T09:06:43.177 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.177 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.177 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.178 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how long to retain device health metrics", 2024-09-10T09:06:43.178 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.178 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.178 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.178 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.178 INFO:teuthology.orchestra.run.smithi172.stdout: "scrape_frequency": { 2024-09-10T09:06:43.178 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "scrape_frequency", 2024-09-10T09:06:43.178 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.178 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.178 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.178 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "86400", 2024-09-10T09:06:43.178 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.178 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.178 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.179 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T09:06:43.179 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.179 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.179 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.179 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.179 INFO:teuthology.orchestra.run.smithi172.stdout: "self_heal": { 2024-09-10T09:06:43.179 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "self_heal", 2024-09-10T09:06:43.179 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.179 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.179 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.179 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.179 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.180 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.180 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.180 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T09:06:43.180 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.180 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.180 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.180 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.180 INFO:teuthology.orchestra.run.smithi172.stdout: "sleep_interval": { 2024-09-10T09:06:43.180 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "sleep_interval", 2024-09-10T09:06:43.180 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.180 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.180 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.181 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "600", 2024-09-10T09:06:43.181 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.181 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.181 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.181 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T09:06:43.181 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.181 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.181 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.181 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.181 INFO:teuthology.orchestra.run.smithi172.stdout: "warn_threshold": { 2024-09-10T09:06:43.181 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "warn_threshold", 2024-09-10T09:06:43.182 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.182 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.182 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.182 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "7257600", 2024-09-10T09:06:43.182 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.182 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.182 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.182 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T09:06:43.182 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.182 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.182 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.182 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.182 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.183 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.183 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.183 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "diskprediction_local", 2024-09-10T09:06:43.183 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.183 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.183 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.183 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.183 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.183 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.183 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.183 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.183 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.183 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.184 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.184 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.184 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.184 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.184 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.184 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.184 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.184 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.184 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.184 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.184 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.184 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.185 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.185 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.185 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.185 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.185 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.185 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.185 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.185 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.185 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.185 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.185 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.185 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.186 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.186 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.186 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.186 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.186 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.186 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.186 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.186 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.186 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.186 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.186 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.186 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.187 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.187 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.187 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.187 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.187 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.187 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.187 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.187 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.187 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.187 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.187 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.187 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.188 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.188 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.188 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.188 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.188 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.188 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.188 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.188 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.188 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.188 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.188 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.188 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.189 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.189 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.189 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.189 INFO:teuthology.orchestra.run.smithi172.stdout: "predict_interval": { 2024-09-10T09:06:43.189 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "predict_interval", 2024-09-10T09:06:43.189 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.189 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.189 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.189 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "86400", 2024-09-10T09:06:43.189 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.189 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.189 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.190 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.190 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.190 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.190 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.190 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.190 INFO:teuthology.orchestra.run.smithi172.stdout: "predictor_model": { 2024-09-10T09:06:43.190 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "predictor_model", 2024-09-10T09:06:43.190 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.190 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.190 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.190 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "prophetstor", 2024-09-10T09:06:43.190 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.191 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.191 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.191 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.191 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.191 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.191 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.191 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.191 INFO:teuthology.orchestra.run.smithi172.stdout: "sleep_interval": { 2024-09-10T09:06:43.191 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "sleep_interval", 2024-09-10T09:06:43.191 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.191 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.191 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.191 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "600", 2024-09-10T09:06:43.192 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.192 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.192 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.192 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.192 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.192 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.192 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.192 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.192 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.192 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.192 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.192 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "influx", 2024-09-10T09:06:43.193 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": false, 2024-09-10T09:06:43.193 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "influxdb python module not found", 2024-09-10T09:06:43.193 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.193 INFO:teuthology.orchestra.run.smithi172.stdout: "batch_size": { 2024-09-10T09:06:43.193 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "batch_size", 2024-09-10T09:06:43.193 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.193 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.193 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.193 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "5000", 2024-09-10T09:06:43.193 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.193 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.193 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.194 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T09:06:43.194 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.194 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.194 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.194 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.194 INFO:teuthology.orchestra.run.smithi172.stdout: "database": { 2024-09-10T09:06:43.194 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "database", 2024-09-10T09:06:43.194 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.194 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.194 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.194 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "ceph", 2024-09-10T09:06:43.194 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.194 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.194 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.195 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T09:06:43.195 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.195 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.195 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.195 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.195 INFO:teuthology.orchestra.run.smithi172.stdout: "hostname": { 2024-09-10T09:06:43.195 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "hostname", 2024-09-10T09:06:43.195 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.195 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.195 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.195 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.195 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.195 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.196 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.196 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "InfluxDB server hostname", 2024-09-10T09:06:43.196 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.196 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.196 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.196 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.196 INFO:teuthology.orchestra.run.smithi172.stdout: "interval": { 2024-09-10T09:06:43.196 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "interval", 2024-09-10T09:06:43.196 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.196 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.196 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.196 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "30", 2024-09-10T09:06:43.197 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "5", 2024-09-10T09:06:43.197 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.197 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.197 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T09:06:43.197 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.197 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.197 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.197 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.197 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.197 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.197 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.197 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.198 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.198 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.198 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.198 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.198 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.198 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.198 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.198 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.198 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.198 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.198 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.198 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.198 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.199 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.199 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.199 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.199 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.199 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.199 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.199 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.199 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.199 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.199 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.199 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.199 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.200 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.200 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.200 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.200 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.200 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.200 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.200 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.200 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.200 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.200 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.200 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.200 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.200 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.201 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.201 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.201 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.201 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.201 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.201 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.201 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.201 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.201 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.201 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.201 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.201 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.202 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.202 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.202 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.202 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.202 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.202 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.202 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.202 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.202 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.202 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.202 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.202 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.202 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.203 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.203 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.203 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.203 INFO:teuthology.orchestra.run.smithi172.stdout: "password": { 2024-09-10T09:06:43.203 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "password", 2024-09-10T09:06:43.203 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.203 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.203 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.203 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.203 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.203 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.203 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.204 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "password of InfluxDB server user", 2024-09-10T09:06:43.204 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.204 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.204 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.204 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.204 INFO:teuthology.orchestra.run.smithi172.stdout: "port": { 2024-09-10T09:06:43.204 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "port", 2024-09-10T09:06:43.204 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.204 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.204 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.204 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "8086", 2024-09-10T09:06:43.204 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.204 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.205 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.205 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "InfluxDB server port", 2024-09-10T09:06:43.205 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.205 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.205 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.205 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.205 INFO:teuthology.orchestra.run.smithi172.stdout: "ssl": { 2024-09-10T09:06:43.205 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ssl", 2024-09-10T09:06:43.205 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.205 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.205 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.205 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "false", 2024-09-10T09:06:43.206 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.206 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.206 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.206 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T09:06:43.206 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.206 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.206 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.206 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.206 INFO:teuthology.orchestra.run.smithi172.stdout: "threads": { 2024-09-10T09:06:43.206 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "threads", 2024-09-10T09:06:43.206 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.206 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.206 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.207 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "5", 2024-09-10T09:06:43.207 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "1", 2024-09-10T09:06:43.207 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "32", 2024-09-10T09:06:43.207 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.207 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T09:06:43.207 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.207 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.207 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.207 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.207 INFO:teuthology.orchestra.run.smithi172.stdout: "username": { 2024-09-10T09:06:43.207 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "username", 2024-09-10T09:06:43.207 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.208 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.208 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.208 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.208 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.208 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.208 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.208 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "username of InfluxDB server user", 2024-09-10T09:06:43.208 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.208 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.208 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.208 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.208 INFO:teuthology.orchestra.run.smithi172.stdout: "verify_ssl": { 2024-09-10T09:06:43.208 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "verify_ssl", 2024-09-10T09:06:43.208 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.209 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.209 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.209 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "true", 2024-09-10T09:06:43.209 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.209 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.209 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.209 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T09:06:43.209 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.209 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.209 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.209 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.209 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.209 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.210 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.210 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "insights", 2024-09-10T09:06:43.210 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.210 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.210 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.210 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.210 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.210 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.210 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.210 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.210 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.210 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.210 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.210 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.210 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.211 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.211 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.211 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.211 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.211 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.211 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.211 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.211 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.211 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.211 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.211 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.211 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.211 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.212 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.212 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.212 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.212 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.212 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.212 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.212 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.212 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.212 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.212 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.212 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.212 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.212 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.212 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.213 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.213 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.213 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.213 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.213 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.213 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.213 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.213 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.213 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.213 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.213 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.213 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.213 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.214 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.214 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.214 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.214 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.214 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.214 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.214 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.214 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.214 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.214 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.214 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.214 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.214 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.214 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.215 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.215 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.215 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.215 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.215 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.215 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.215 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.215 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.215 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.215 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "iostat", 2024-09-10T09:06:43.215 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.215 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.215 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.215 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.216 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.216 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.216 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.216 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.216 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.216 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.216 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.216 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.216 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.216 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.216 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.216 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.216 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.217 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.217 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.217 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.217 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.217 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.217 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.217 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.217 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.217 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.217 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.217 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.217 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.218 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.218 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.218 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.218 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.218 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.218 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.218 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.218 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.218 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.218 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.218 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.218 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.218 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.219 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.219 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.219 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.219 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.219 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.219 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.219 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.219 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.219 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.219 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.219 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.219 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.219 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.220 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.220 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.220 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.220 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.220 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.220 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.220 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.220 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.220 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.220 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.220 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.220 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.220 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.221 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.221 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.221 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.221 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.221 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.221 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.221 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.221 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.221 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "k8sevents", 2024-09-10T09:06:43.221 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.221 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.221 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.221 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_event_retention_days": { 2024-09-10T09:06:43.222 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ceph_event_retention_days", 2024-09-10T09:06:43.222 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.222 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.222 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.222 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "7", 2024-09-10T09:06:43.222 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.222 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.222 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.222 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T09:06:43.222 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.222 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.222 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.222 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.223 INFO:teuthology.orchestra.run.smithi172.stdout: "config_check_secs": { 2024-09-10T09:06:43.223 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "config_check_secs", 2024-09-10T09:06:43.223 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.223 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.223 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.223 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:43.223 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "10", 2024-09-10T09:06:43.223 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.223 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.223 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T09:06:43.223 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.223 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.224 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.224 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.224 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.224 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.224 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.224 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.224 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.224 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.224 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.224 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.224 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.224 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.225 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.225 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.225 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.225 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.225 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.225 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.225 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.225 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.225 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.225 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.225 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.225 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.226 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.226 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.226 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.226 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.226 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.226 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.226 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.226 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.226 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.226 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.226 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.226 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.226 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.227 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.227 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.227 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.227 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.227 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.227 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.227 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.227 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.227 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.227 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.227 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.227 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.227 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.227 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.228 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.228 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.228 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.228 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.228 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.228 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.228 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.228 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.228 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.228 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.228 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.228 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.228 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.228 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.229 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.229 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.229 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.229 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.229 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.229 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.229 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.229 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.229 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.229 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.229 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "localpool", 2024-09-10T09:06:43.229 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.230 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.230 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.230 INFO:teuthology.orchestra.run.smithi172.stdout: "failure_domain": { 2024-09-10T09:06:43.230 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "failure_domain", 2024-09-10T09:06:43.230 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.230 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.230 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.230 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "host", 2024-09-10T09:06:43.230 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.230 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.230 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.230 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "failure domain for any created local pool", 2024-09-10T09:06:43.231 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T09:06:43.231 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.231 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.231 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.231 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.231 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.231 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.231 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.231 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.231 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.231 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.231 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.231 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.231 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.231 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.232 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.232 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.232 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.232 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.232 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.232 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.232 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.232 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.232 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.232 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.232 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.232 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.232 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.233 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.233 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.233 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.233 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.233 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.233 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.233 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.233 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.233 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.233 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.233 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.233 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.233 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.233 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.234 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.234 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.234 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.234 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.234 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.234 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.234 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.234 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.234 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.234 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.234 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.234 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.234 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.235 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.235 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.235 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.235 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.235 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.235 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.235 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.235 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.235 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.235 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.235 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.235 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.235 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.235 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.235 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.236 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.236 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.236 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.236 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.236 INFO:teuthology.orchestra.run.smithi172.stdout: "min_size": { 2024-09-10T09:06:43.236 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "min_size", 2024-09-10T09:06:43.236 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.236 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.236 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.236 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.236 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.236 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.237 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.237 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "default min_size for any created local pool", 2024-09-10T09:06:43.237 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T09:06:43.237 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.237 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.237 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.237 INFO:teuthology.orchestra.run.smithi172.stdout: "num_rep": { 2024-09-10T09:06:43.237 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "num_rep", 2024-09-10T09:06:43.237 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.237 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.237 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.237 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "3", 2024-09-10T09:06:43.237 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.238 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.238 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.238 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "default replica count for any created local pool", 2024-09-10T09:06:43.238 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.238 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.238 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.238 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.238 INFO:teuthology.orchestra.run.smithi172.stdout: "pg_num": { 2024-09-10T09:06:43.238 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "pg_num", 2024-09-10T09:06:43.238 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.238 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.238 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.239 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "128", 2024-09-10T09:06:43.239 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.239 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.239 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.239 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T09:06:43.239 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.239 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.239 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.239 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.239 INFO:teuthology.orchestra.run.smithi172.stdout: "prefix": { 2024-09-10T09:06:43.239 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "prefix", 2024-09-10T09:06:43.239 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.239 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.239 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.240 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.240 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.240 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.240 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.240 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "name prefix for any created local pool", 2024-09-10T09:06:43.240 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.240 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.240 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.240 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.240 INFO:teuthology.orchestra.run.smithi172.stdout: "subtree": { 2024-09-10T09:06:43.240 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "subtree", 2024-09-10T09:06:43.240 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.240 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.241 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.241 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "rack", 2024-09-10T09:06:43.241 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.241 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.241 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.241 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T09:06:43.241 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T09:06:43.241 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.241 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.241 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.241 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.241 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.242 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.242 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mds_autoscaler", 2024-09-10T09:06:43.242 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.242 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.242 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.242 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.242 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.242 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.242 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.242 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.242 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.242 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.242 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.242 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.243 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.243 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.243 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.243 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.243 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.243 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.243 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.243 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.243 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.243 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.243 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.243 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.244 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.244 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.244 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.244 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.244 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.244 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.244 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.244 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.244 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.244 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.244 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.244 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.244 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.245 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.245 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.245 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.245 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.245 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.245 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.245 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.245 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.245 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.245 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.245 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.245 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.245 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.246 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.246 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.246 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.246 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.246 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.246 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.246 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.246 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.246 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.246 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.246 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.246 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.247 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.247 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.247 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.247 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.247 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.247 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.247 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.247 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.247 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.247 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.247 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.247 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.247 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.248 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.248 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mirroring", 2024-09-10T09:06:43.248 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.248 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.248 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.248 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.248 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.248 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.248 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.248 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.248 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.248 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.249 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.249 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.249 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.249 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.249 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.249 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.249 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.249 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.249 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.249 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.249 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.249 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.249 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.250 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.250 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.250 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.250 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.250 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.250 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.250 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.250 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.250 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.250 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.250 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.250 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.251 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.251 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.251 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.251 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.251 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.251 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.251 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.251 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.251 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.251 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.251 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.251 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.251 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.251 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.252 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.252 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.252 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.252 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.252 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.252 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.252 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.252 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.252 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.252 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.252 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.252 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.252 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.252 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.253 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.253 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.253 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.253 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.253 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.253 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.253 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.253 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.253 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.253 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.253 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.253 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.253 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.254 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "nfs", 2024-09-10T09:06:43.254 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.254 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.254 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.254 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.254 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.254 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.254 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.254 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.254 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.254 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.254 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.255 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.255 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.255 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.255 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.255 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.255 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.255 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.255 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.255 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.255 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.255 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.255 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.255 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.256 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.256 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.256 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.256 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.256 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.256 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.256 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.256 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.256 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.256 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.256 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.256 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.256 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.257 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.257 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.257 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.257 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.257 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.257 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.257 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.257 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.257 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.257 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.257 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.257 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.257 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.258 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.258 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.258 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.258 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.258 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.258 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.258 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.258 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.258 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.258 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.258 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.258 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.259 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.259 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.259 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.259 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.259 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.259 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.259 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.259 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.259 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.259 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.259 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.259 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.259 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.260 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.260 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "orchestrator", 2024-09-10T09:06:43.260 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.260 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.260 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.260 INFO:teuthology.orchestra.run.smithi172.stdout: "fail_fs": { 2024-09-10T09:06:43.260 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "fail_fs", 2024-09-10T09:06:43.260 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.260 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.260 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.260 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.260 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.260 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.261 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.261 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T09:06:43.261 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.261 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.261 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.261 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.261 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.261 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.261 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.261 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.261 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.261 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.261 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.262 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.262 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.262 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.262 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.262 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.262 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.262 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.262 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.262 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.262 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.262 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.262 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.263 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.263 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.263 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.263 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.263 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.263 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.263 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.263 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.263 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.263 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.263 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.263 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.263 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.264 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.264 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.264 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.264 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.264 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.264 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.264 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.264 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.264 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.264 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.264 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.264 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.264 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.265 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.265 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.265 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.265 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.265 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.265 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.265 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.265 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.265 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.265 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.265 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.265 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.266 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.266 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.266 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.266 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.266 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.266 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.266 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.266 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.266 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.266 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.266 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.266 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.266 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.267 INFO:teuthology.orchestra.run.smithi172.stdout: "orchestrator": { 2024-09-10T09:06:43.267 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "orchestrator", 2024-09-10T09:06:43.267 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.267 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.267 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.267 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.267 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.267 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.267 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.267 INFO:teuthology.orchestra.run.smithi172.stdout: "cephadm", 2024-09-10T09:06:43.267 INFO:teuthology.orchestra.run.smithi172.stdout: "rook", 2024-09-10T09:06:43.267 INFO:teuthology.orchestra.run.smithi172.stdout: "test_orchestrator" 2024-09-10T09:06:43.267 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.268 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Orchestrator backend", 2024-09-10T09:06:43.268 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.268 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.268 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.268 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.268 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.268 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.268 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.268 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "osd_perf_query", 2024-09-10T09:06:43.268 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.268 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.268 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.268 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.268 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.269 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.269 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.269 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.269 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.269 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.269 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.269 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.269 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.269 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.269 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.269 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.269 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.270 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.270 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.270 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.270 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.270 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.270 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.270 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.270 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.270 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.270 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.270 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.270 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.270 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.271 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.271 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.271 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.271 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.271 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.271 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.271 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.271 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.271 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.271 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.271 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.271 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.272 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.272 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.272 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.272 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.272 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.272 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.272 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.272 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.272 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.272 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.272 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.272 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.273 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.273 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.273 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.273 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.273 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.273 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.273 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.273 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.273 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.273 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.273 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.273 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.274 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.274 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.274 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.274 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.274 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.274 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.274 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.274 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.274 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.274 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.274 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "osd_support", 2024-09-10T09:06:43.274 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.275 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.275 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.275 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.275 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.275 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.275 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.275 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.275 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.275 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.275 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.275 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.275 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.275 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.276 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.276 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.276 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.276 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.276 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.276 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.276 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.276 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.276 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.276 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.276 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.276 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.277 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.277 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.277 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.277 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.277 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.277 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.277 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.277 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.277 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.277 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.277 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.277 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.277 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.278 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.278 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.278 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.278 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.278 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.278 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.278 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.278 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.278 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.278 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.278 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.278 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.279 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.279 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.279 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.279 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.279 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.279 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.279 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.279 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.279 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.279 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.279 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.279 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.279 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.280 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.280 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.280 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.280 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.280 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.280 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.280 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.280 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.280 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.280 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.280 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.280 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.281 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "pg_autoscaler", 2024-09-10T09:06:43.281 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.281 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.281 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.281 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.281 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.281 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.281 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.281 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.281 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.281 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.281 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.281 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.282 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.282 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.282 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.282 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.282 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.282 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.282 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.282 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.282 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.282 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.282 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.282 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.282 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.283 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.283 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.283 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.283 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.283 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.283 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.283 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.283 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.283 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.283 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.283 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.283 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.283 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.284 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.284 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.284 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.284 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.284 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.284 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.284 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.284 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.284 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.284 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.284 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.284 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.284 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.285 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.285 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.285 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.285 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.285 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.285 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.285 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.285 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.285 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.285 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.285 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.285 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.285 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.285 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.286 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.286 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.286 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.286 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.286 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.286 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.286 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.286 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.286 INFO:teuthology.orchestra.run.smithi172.stdout: "sleep_interval": { 2024-09-10T09:06:43.286 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "sleep_interval", 2024-09-10T09:06:43.286 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.286 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.286 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.287 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "60", 2024-09-10T09:06:43.287 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.287 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.287 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.287 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.287 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.287 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.287 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.287 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.287 INFO:teuthology.orchestra.run.smithi172.stdout: "threshold": { 2024-09-10T09:06:43.287 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "threshold", 2024-09-10T09:06:43.287 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:43.287 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.288 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.288 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "3.0", 2024-09-10T09:06:43.288 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "1.0", 2024-09-10T09:06:43.288 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.288 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.288 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "scaling threshold", 2024-09-10T09:06:43.288 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T09:06:43.288 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.288 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.288 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.288 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.288 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.289 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.289 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "progress", 2024-09-10T09:06:43.289 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.289 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.289 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.289 INFO:teuthology.orchestra.run.smithi172.stdout: "allow_pg_recovery_event": { 2024-09-10T09:06:43.289 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "allow_pg_recovery_event", 2024-09-10T09:06:43.289 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.289 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.289 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.289 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.289 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.290 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.290 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.290 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T09:06:43.290 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.290 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.290 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.290 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.290 INFO:teuthology.orchestra.run.smithi172.stdout: "enabled": { 2024-09-10T09:06:43.290 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "enabled", 2024-09-10T09:06:43.290 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.290 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.290 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.290 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.291 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.291 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.291 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.291 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.291 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.291 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.291 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.291 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.291 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.291 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.291 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.291 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.292 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.292 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.292 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.292 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.292 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.292 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.292 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.292 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.292 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.292 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.292 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.292 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.293 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.293 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.293 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.293 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.293 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.293 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.293 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.293 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.293 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.293 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.293 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.293 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.294 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.294 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.294 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.294 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.294 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.294 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.294 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.294 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.294 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.294 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.294 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.294 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.294 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.295 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.295 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.295 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.295 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.295 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.295 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.295 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.295 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.295 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.295 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.295 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.295 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.296 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.296 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.296 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.296 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.296 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.296 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.296 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.296 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.296 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.296 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.296 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.296 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.297 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.297 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.297 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.297 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.297 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.297 INFO:teuthology.orchestra.run.smithi172.stdout: "max_completed_events": { 2024-09-10T09:06:43.297 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "max_completed_events", 2024-09-10T09:06:43.297 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.297 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.297 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.297 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "50", 2024-09-10T09:06:43.297 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.298 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.298 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.298 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "number of past completed events to remember", 2024-09-10T09:06:43.298 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.298 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.298 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.298 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.298 INFO:teuthology.orchestra.run.smithi172.stdout: "sleep_interval": { 2024-09-10T09:06:43.298 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "sleep_interval", 2024-09-10T09:06:43.298 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.298 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.298 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.298 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "5", 2024-09-10T09:06:43.298 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.299 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.299 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.299 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how long the module is going to sleep", 2024-09-10T09:06:43.299 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.299 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.299 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.299 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.299 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.299 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.299 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.299 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "prometheus", 2024-09-10T09:06:43.299 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.300 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.300 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.300 INFO:teuthology.orchestra.run.smithi172.stdout: "cache": { 2024-09-10T09:06:43.300 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "cache", 2024-09-10T09:06:43.300 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.300 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.300 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.300 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.300 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.300 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.300 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.300 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.301 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.301 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.301 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.301 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.301 INFO:teuthology.orchestra.run.smithi172.stdout: "exclude_perf_counters": { 2024-09-10T09:06:43.301 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "exclude_perf_counters", 2024-09-10T09:06:43.301 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.301 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.301 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.301 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.301 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.301 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.301 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.302 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T09:06:43.302 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T09:06:43.302 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.302 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.302 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.302 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.302 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.302 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.302 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.302 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.302 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.302 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.303 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.303 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.303 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.303 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.303 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.303 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.303 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.303 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.303 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.303 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.303 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.303 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.304 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.304 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.304 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.304 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.304 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.304 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.304 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.304 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.304 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.304 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.304 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.304 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.305 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.305 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.305 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.305 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.305 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.305 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.305 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.305 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.305 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.305 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.305 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.305 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.306 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.306 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.306 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.306 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.306 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.306 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.306 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.306 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.306 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.306 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.306 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.306 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.306 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.307 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.307 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.307 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.307 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.307 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.307 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.307 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.307 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.307 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.307 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.307 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.307 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.307 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.308 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.308 INFO:teuthology.orchestra.run.smithi172.stdout: "rbd_stats_pools": { 2024-09-10T09:06:43.308 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rbd_stats_pools", 2024-09-10T09:06:43.308 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.308 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.308 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.308 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.308 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.308 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.308 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.308 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.308 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.309 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.309 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.309 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.309 INFO:teuthology.orchestra.run.smithi172.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T09:06:43.309 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T09:06:43.309 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.309 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.309 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.309 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "300", 2024-09-10T09:06:43.309 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.309 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.309 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.309 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.310 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.310 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.310 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.310 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.310 INFO:teuthology.orchestra.run.smithi172.stdout: "scrape_interval": { 2024-09-10T09:06:43.310 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "scrape_interval", 2024-09-10T09:06:43.310 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:43.310 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.310 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.310 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "15.0", 2024-09-10T09:06:43.310 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.310 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.311 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.311 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.311 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.311 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.311 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.311 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.311 INFO:teuthology.orchestra.run.smithi172.stdout: "server_addr": { 2024-09-10T09:06:43.311 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_addr", 2024-09-10T09:06:43.311 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.311 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.311 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.311 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "::", 2024-09-10T09:06:43.311 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.312 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.312 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.312 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T09:06:43.312 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.312 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.312 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.312 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.312 INFO:teuthology.orchestra.run.smithi172.stdout: "server_port": { 2024-09-10T09:06:43.312 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_port", 2024-09-10T09:06:43.312 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.312 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.312 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.312 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "9283", 2024-09-10T09:06:43.313 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.313 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.313 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.313 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T09:06:43.313 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.313 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.313 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.313 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.313 INFO:teuthology.orchestra.run.smithi172.stdout: "stale_cache_strategy": { 2024-09-10T09:06:43.313 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "stale_cache_strategy", 2024-09-10T09:06:43.313 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.313 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.313 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.313 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "log", 2024-09-10T09:06:43.314 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.314 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.314 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.314 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.314 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.314 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.314 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.314 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.314 INFO:teuthology.orchestra.run.smithi172.stdout: "standby_behaviour": { 2024-09-10T09:06:43.314 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "standby_behaviour", 2024-09-10T09:06:43.314 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.314 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.315 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.315 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "default", 2024-09-10T09:06:43.315 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.315 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.315 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.315 INFO:teuthology.orchestra.run.smithi172.stdout: "default", 2024-09-10T09:06:43.315 INFO:teuthology.orchestra.run.smithi172.stdout: "error" 2024-09-10T09:06:43.315 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.315 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.315 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.315 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.315 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.315 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.316 INFO:teuthology.orchestra.run.smithi172.stdout: "standby_error_status_code": { 2024-09-10T09:06:43.316 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "standby_error_status_code", 2024-09-10T09:06:43.316 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.316 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.316 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.316 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "500", 2024-09-10T09:06:43.316 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "400", 2024-09-10T09:06:43.316 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "599", 2024-09-10T09:06:43.316 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.316 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.316 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.316 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.316 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.316 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.317 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.317 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.317 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.317 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rbd_support", 2024-09-10T09:06:43.317 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.317 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.317 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.317 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.317 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.317 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.317 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.317 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.318 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.318 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.318 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.318 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.318 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.318 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.318 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.318 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.318 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.318 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.318 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.318 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.319 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.319 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.319 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.319 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.319 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.319 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.319 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.319 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.319 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.319 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.319 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.319 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.319 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.320 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.320 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.320 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.320 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.320 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.320 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.320 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.320 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.320 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.320 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.320 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.321 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.321 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.321 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.321 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.321 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.321 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.321 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.321 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.321 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.321 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.321 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.321 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.321 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.322 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.322 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.322 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.322 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.322 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.322 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.322 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.322 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.322 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.322 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.322 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.322 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.323 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.323 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.323 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.323 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.323 INFO:teuthology.orchestra.run.smithi172.stdout: "max_concurrent_snap_create": { 2024-09-10T09:06:43.323 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "max_concurrent_snap_create", 2024-09-10T09:06:43.323 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.323 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.323 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.323 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:43.323 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.323 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.323 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.323 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.324 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.324 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.324 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.324 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.324 INFO:teuthology.orchestra.run.smithi172.stdout: "mirror_snapshot_schedule": { 2024-09-10T09:06:43.324 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T09:06:43.324 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.324 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.324 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.324 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.324 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.324 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.325 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.325 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.325 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.325 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.325 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.325 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.325 INFO:teuthology.orchestra.run.smithi172.stdout: "trash_purge_schedule": { 2024-09-10T09:06:43.325 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "trash_purge_schedule", 2024-09-10T09:06:43.325 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.325 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.325 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.325 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.325 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.326 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.326 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.326 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.326 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.326 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.326 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.326 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.326 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.326 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.326 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.326 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "restful", 2024-09-10T09:06:43.326 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.326 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.326 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.327 INFO:teuthology.orchestra.run.smithi172.stdout: "enable_auth": { 2024-09-10T09:06:43.327 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "enable_auth", 2024-09-10T09:06:43.327 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.327 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.327 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.327 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.327 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.327 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.327 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.327 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.327 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.327 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.327 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.327 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.328 INFO:teuthology.orchestra.run.smithi172.stdout: "key_file": { 2024-09-10T09:06:43.328 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "key_file", 2024-09-10T09:06:43.328 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.328 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.328 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.328 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.328 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.328 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.328 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.328 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.328 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.328 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.328 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.329 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.329 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.329 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.329 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.329 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.329 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.329 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.329 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.329 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.329 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.329 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.329 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.330 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.330 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.330 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.330 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.330 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.330 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.330 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.330 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.330 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.330 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.330 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.330 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.330 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.331 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.331 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.331 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.331 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.331 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.331 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.331 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.331 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.331 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.331 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.331 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.331 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.331 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.331 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.332 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.332 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.332 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.332 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.332 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.332 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.332 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.332 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.332 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.332 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.332 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.332 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.332 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.333 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.333 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.333 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.333 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.333 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.333 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.333 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.333 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.333 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.333 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.333 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.333 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.333 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.334 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.334 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.334 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.334 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.334 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.334 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.334 INFO:teuthology.orchestra.run.smithi172.stdout: "server_addr": { 2024-09-10T09:06:43.334 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_addr", 2024-09-10T09:06:43.334 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.334 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.334 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.334 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.334 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.334 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.335 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.335 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.335 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.335 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.335 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.335 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.335 INFO:teuthology.orchestra.run.smithi172.stdout: "server_port": { 2024-09-10T09:06:43.335 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_port", 2024-09-10T09:06:43.335 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.335 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.335 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.335 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.336 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.336 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.336 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.336 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.336 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.336 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.336 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.336 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.336 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.336 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.336 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.336 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rgw", 2024-09-10T09:06:43.336 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.337 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.337 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.337 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.337 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.337 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.337 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.337 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.337 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.337 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.337 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.337 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.337 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.338 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.338 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.338 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.338 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.338 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.338 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.338 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.338 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.338 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.338 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.338 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.338 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.338 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.339 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.339 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.339 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.339 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.339 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.339 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.339 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.339 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.339 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.339 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.339 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.339 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.340 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.340 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.340 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.340 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.340 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.340 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.340 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.340 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.340 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.340 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.340 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.340 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.340 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.341 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.341 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.341 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.341 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.341 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.341 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.341 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.341 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.341 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.341 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.341 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.341 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.342 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.342 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.342 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.342 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.342 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.342 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.342 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.342 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.342 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.342 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.342 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.342 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.342 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.343 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rook", 2024-09-10T09:06:43.343 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.343 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.343 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.343 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.343 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.343 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.343 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.343 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.343 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.343 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.343 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.343 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.344 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.344 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.344 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.344 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.344 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.344 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.344 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.344 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.344 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.344 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.344 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.344 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.345 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.345 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.345 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.345 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.345 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.345 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.345 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.345 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.345 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.345 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.345 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.345 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.346 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.346 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.346 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.346 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.346 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.346 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.346 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.346 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.346 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.346 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.346 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.346 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.347 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.347 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.347 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.347 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.347 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.347 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.347 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.347 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.347 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.347 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.347 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.347 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.347 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.348 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.348 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.348 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.348 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.348 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.348 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.348 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.348 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.348 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.348 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.348 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.348 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.349 INFO:teuthology.orchestra.run.smithi172.stdout: "storage_class": { 2024-09-10T09:06:43.349 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "storage_class", 2024-09-10T09:06:43.349 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.349 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.349 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.349 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "local", 2024-09-10T09:06:43.349 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.349 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.349 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.349 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T09:06:43.349 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.349 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.350 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.350 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.350 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.350 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.350 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.350 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "selftest", 2024-09-10T09:06:43.350 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.350 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.350 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.350 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.350 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.350 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.350 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.351 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.351 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.351 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.351 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.351 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.351 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.351 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.351 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.351 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.351 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.351 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.351 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.351 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.352 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.352 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.352 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.352 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.352 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.352 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.352 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.352 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.352 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.352 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.352 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.352 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.352 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.353 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.353 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.353 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.353 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.353 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.353 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.353 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.353 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.353 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.353 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.353 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.353 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.353 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.354 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.354 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.354 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.354 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.354 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.354 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.354 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.354 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.354 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.354 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.354 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.354 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.354 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.354 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.355 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.355 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.355 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.355 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.355 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.355 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.355 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.355 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.355 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.355 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.355 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.355 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.355 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.356 INFO:teuthology.orchestra.run.smithi172.stdout: "roption1": { 2024-09-10T09:06:43.356 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "roption1", 2024-09-10T09:06:43.356 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.356 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.356 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.356 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.356 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.356 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.356 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.356 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.356 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.356 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.356 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.357 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.357 INFO:teuthology.orchestra.run.smithi172.stdout: "roption2": { 2024-09-10T09:06:43.357 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "roption2", 2024-09-10T09:06:43.357 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.357 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.357 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.357 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "xyz", 2024-09-10T09:06:43.357 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.357 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.357 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.357 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.357 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.357 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.358 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.358 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.358 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption1": { 2024-09-10T09:06:43.358 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption1", 2024-09-10T09:06:43.358 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.358 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.358 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.358 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.358 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.358 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.358 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.358 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.359 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.359 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.359 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.359 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.359 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption2": { 2024-09-10T09:06:43.359 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption2", 2024-09-10T09:06:43.359 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.359 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.359 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.359 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.359 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.359 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.359 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.359 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.359 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.360 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.360 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.360 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.360 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption3": { 2024-09-10T09:06:43.360 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption3", 2024-09-10T09:06:43.360 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:43.360 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.360 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.360 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.360 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.360 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.360 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.361 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.361 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.361 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.361 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.361 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.361 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption4": { 2024-09-10T09:06:43.361 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption4", 2024-09-10T09:06:43.361 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.361 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.361 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.361 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.361 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.361 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.362 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.362 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.362 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.362 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.362 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.362 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.362 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption5": { 2024-09-10T09:06:43.362 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption5", 2024-09-10T09:06:43.362 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.362 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.362 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.363 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.363 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.363 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.363 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.363 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.363 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.363 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.363 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.363 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.363 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption6": { 2024-09-10T09:06:43.363 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption6", 2024-09-10T09:06:43.363 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.364 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.364 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.364 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.364 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.364 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.364 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.364 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.364 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.364 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.364 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.364 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.364 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption7": { 2024-09-10T09:06:43.364 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption7", 2024-09-10T09:06:43.364 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.365 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.365 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.365 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.365 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "1", 2024-09-10T09:06:43.365 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "42", 2024-09-10T09:06:43.365 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.365 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.365 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.365 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.365 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.365 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.365 INFO:teuthology.orchestra.run.smithi172.stdout: "testkey": { 2024-09-10T09:06:43.365 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "testkey", 2024-09-10T09:06:43.366 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.366 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.366 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.366 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.366 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.366 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.366 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.366 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.366 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.366 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.366 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.366 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.366 INFO:teuthology.orchestra.run.smithi172.stdout: "testlkey": { 2024-09-10T09:06:43.367 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "testlkey", 2024-09-10T09:06:43.367 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.367 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.367 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.367 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.367 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.367 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.367 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.367 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.367 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.367 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.367 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.367 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.367 INFO:teuthology.orchestra.run.smithi172.stdout: "testnewline": { 2024-09-10T09:06:43.368 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "testnewline", 2024-09-10T09:06:43.368 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.368 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.368 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.368 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.368 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.368 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.368 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.368 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.368 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.368 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.368 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.368 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.369 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.369 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.369 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.369 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "snap_schedule", 2024-09-10T09:06:43.369 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.369 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.369 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.369 INFO:teuthology.orchestra.run.smithi172.stdout: "allow_m_granularity": { 2024-09-10T09:06:43.369 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "allow_m_granularity", 2024-09-10T09:06:43.369 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.369 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.369 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.369 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.370 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.370 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.370 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.370 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T09:06:43.370 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.370 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.370 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.370 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.370 INFO:teuthology.orchestra.run.smithi172.stdout: "dump_on_update": { 2024-09-10T09:06:43.370 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "dump_on_update", 2024-09-10T09:06:43.370 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.370 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.370 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.370 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.371 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.371 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.371 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.371 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "dump database to debug log on update", 2024-09-10T09:06:43.371 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.371 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.371 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.371 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.371 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.371 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.371 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.371 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.371 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.371 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.372 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.372 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.372 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.372 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.372 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.372 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.372 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.372 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.372 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.372 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.372 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.372 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.372 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.373 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.373 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.373 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.373 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.373 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.373 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.373 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.373 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.373 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.373 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.373 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.373 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.373 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.374 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.374 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.374 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.374 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.374 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.374 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.374 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.374 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.374 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.374 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.374 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.374 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.374 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.375 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.375 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.375 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.375 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.375 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.375 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.375 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.375 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.375 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.375 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.375 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.375 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.376 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.376 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.376 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.376 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.376 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.376 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.376 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.376 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.376 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.376 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.376 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.376 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.377 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.377 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.377 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.377 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.377 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "stats", 2024-09-10T09:06:43.377 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.377 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.377 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.377 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.377 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.377 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.377 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.378 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.378 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.378 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.378 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.378 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.378 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.378 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.378 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.378 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.378 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.378 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.378 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.378 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.379 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.379 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.379 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.379 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.379 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.379 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.379 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.379 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.379 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.379 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.379 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.379 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.379 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.380 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.380 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.380 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.380 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.380 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.380 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.380 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.380 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.380 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.380 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.380 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.380 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.381 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.381 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.381 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.381 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.381 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.381 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.381 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.381 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.381 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.381 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.381 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.381 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.381 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.382 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.382 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.382 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.382 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.382 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.382 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.382 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.382 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.382 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.382 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.382 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.382 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.382 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.383 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.383 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.383 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.383 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.383 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.383 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "status", 2024-09-10T09:06:43.383 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.383 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.383 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.383 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.383 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.383 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.383 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.384 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.384 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.384 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.384 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.384 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.384 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.384 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.384 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.384 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.384 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.384 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.384 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.385 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.385 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.385 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.385 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.385 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.385 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.385 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.385 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.385 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.385 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.385 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.385 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.385 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.386 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.386 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.386 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.386 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.386 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.386 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.386 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.386 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.386 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.386 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.386 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.386 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.386 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.387 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.387 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.387 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.387 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.387 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.387 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.387 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.387 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.387 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.387 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.387 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.387 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.387 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.388 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.388 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.388 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.388 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.388 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.388 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.388 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.388 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.388 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.388 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.388 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.388 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.388 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.389 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.389 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.389 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.389 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.389 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.389 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "telegraf", 2024-09-10T09:06:43.389 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.389 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.389 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.389 INFO:teuthology.orchestra.run.smithi172.stdout: "address": { 2024-09-10T09:06:43.389 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "address", 2024-09-10T09:06:43.389 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.389 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.390 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.390 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T09:06:43.390 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.390 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.390 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.390 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.390 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.390 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.390 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.390 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.392 INFO:teuthology.orchestra.run.smithi172.stdout: "interval": { 2024-09-10T09:06:43.392 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "interval", 2024-09-10T09:06:43.392 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.392 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.392 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.392 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "15", 2024-09-10T09:06:43.392 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.392 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.392 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.392 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.392 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.393 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.393 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.393 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.393 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.393 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.393 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.393 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.393 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.393 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.393 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.393 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.393 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.393 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.394 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.394 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.394 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.394 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.394 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.394 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.394 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.394 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.394 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.394 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.394 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.394 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.394 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.394 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.395 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.395 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.395 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.395 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.395 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.395 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.395 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.395 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.395 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.395 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.395 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.395 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.395 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.396 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.396 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.396 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.396 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.396 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.396 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.396 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.396 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.396 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.396 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.396 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.396 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.397 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.397 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.397 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.397 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.397 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.397 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.397 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.397 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.397 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.397 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.397 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.397 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.398 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.398 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.398 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.398 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.398 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.398 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.398 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.398 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.398 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.398 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.398 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.398 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.399 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "telemetry", 2024-09-10T09:06:43.399 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.399 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.399 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.399 INFO:teuthology.orchestra.run.smithi172.stdout: "channel_basic": { 2024-09-10T09:06:43.399 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "channel_basic", 2024-09-10T09:06:43.399 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.399 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.399 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.399 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.399 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.399 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.400 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.400 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T09:06:43.400 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.400 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.400 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.400 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.400 INFO:teuthology.orchestra.run.smithi172.stdout: "channel_crash": { 2024-09-10T09:06:43.400 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "channel_crash", 2024-09-10T09:06:43.400 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.400 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.400 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.400 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.401 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.401 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.401 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.401 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T09:06:43.401 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.401 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.401 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.401 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.401 INFO:teuthology.orchestra.run.smithi172.stdout: "channel_device": { 2024-09-10T09:06:43.401 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "channel_device", 2024-09-10T09:06:43.401 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.401 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.401 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.401 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.402 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.402 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.402 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.402 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T09:06:43.402 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.402 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.402 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.402 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.402 INFO:teuthology.orchestra.run.smithi172.stdout: "channel_ident": { 2024-09-10T09:06:43.402 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "channel_ident", 2024-09-10T09:06:43.402 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.402 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.402 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.402 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.403 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.403 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.403 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.403 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T09:06:43.403 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.403 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.403 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.403 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.403 INFO:teuthology.orchestra.run.smithi172.stdout: "channel_perf": { 2024-09-10T09:06:43.403 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "channel_perf", 2024-09-10T09:06:43.403 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.403 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.404 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.404 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.404 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.404 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.404 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.404 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T09:06:43.404 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.404 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.404 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.404 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.404 INFO:teuthology.orchestra.run.smithi172.stdout: "contact": { 2024-09-10T09:06:43.404 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "contact", 2024-09-10T09:06:43.404 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.405 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.405 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.405 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.405 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.405 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.405 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.405 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.405 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.405 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.405 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.405 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.405 INFO:teuthology.orchestra.run.smithi172.stdout: "description": { 2024-09-10T09:06:43.406 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "description", 2024-09-10T09:06:43.406 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.406 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.406 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.406 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.406 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.406 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.406 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.406 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.406 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.406 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.406 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.406 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.407 INFO:teuthology.orchestra.run.smithi172.stdout: "device_url": { 2024-09-10T09:06:43.407 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "device_url", 2024-09-10T09:06:43.407 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.407 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.407 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.407 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T09:06:43.407 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.407 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.407 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.407 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.407 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.407 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.407 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.407 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.408 INFO:teuthology.orchestra.run.smithi172.stdout: "enabled": { 2024-09-10T09:06:43.408 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "enabled", 2024-09-10T09:06:43.408 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.408 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.408 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.408 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.408 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.408 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.408 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.408 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.408 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.408 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.408 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.409 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.409 INFO:teuthology.orchestra.run.smithi172.stdout: "interval": { 2024-09-10T09:06:43.409 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "interval", 2024-09-10T09:06:43.409 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.409 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.409 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.409 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "24", 2024-09-10T09:06:43.409 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "8", 2024-09-10T09:06:43.409 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.409 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.409 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.409 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.409 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.410 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.410 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.410 INFO:teuthology.orchestra.run.smithi172.stdout: "last_opt_revision": { 2024-09-10T09:06:43.410 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "last_opt_revision", 2024-09-10T09:06:43.410 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.410 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.410 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.410 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "1", 2024-09-10T09:06:43.410 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.410 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.410 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.410 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.410 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.411 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.411 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.411 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.411 INFO:teuthology.orchestra.run.smithi172.stdout: "leaderboard": { 2024-09-10T09:06:43.411 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "leaderboard", 2024-09-10T09:06:43.411 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.411 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.411 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.411 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.411 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.411 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.411 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.411 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.411 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.412 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.412 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.412 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.412 INFO:teuthology.orchestra.run.smithi172.stdout: "leaderboard_description": { 2024-09-10T09:06:43.412 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "leaderboard_description", 2024-09-10T09:06:43.412 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.412 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.412 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.412 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.412 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.412 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.412 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.412 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.413 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.413 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.413 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.413 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.413 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.413 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.413 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.413 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.413 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.413 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.413 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.413 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.413 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.414 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.414 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.414 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.414 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.414 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.414 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.414 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.414 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.414 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.414 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.414 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.414 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.414 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.415 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.415 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.415 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.415 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.415 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.415 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.415 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.415 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.415 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.415 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.415 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.415 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.415 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.415 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.416 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.416 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.416 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.416 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.416 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.416 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.416 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.416 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.416 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.416 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.416 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.416 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.416 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.417 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.417 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.417 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.417 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.417 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.417 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.417 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.417 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.417 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.417 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.417 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.417 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.418 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.418 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.418 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.418 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.418 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.418 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.418 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.418 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.418 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.418 INFO:teuthology.orchestra.run.smithi172.stdout: "organization": { 2024-09-10T09:06:43.418 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "organization", 2024-09-10T09:06:43.418 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.418 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.419 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.419 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.419 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.419 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.419 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.419 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.419 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.419 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.419 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.419 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.419 INFO:teuthology.orchestra.run.smithi172.stdout: "proxy": { 2024-09-10T09:06:43.419 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "proxy", 2024-09-10T09:06:43.420 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.420 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.420 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.420 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.420 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.420 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.420 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.420 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.420 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.420 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.420 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.420 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.420 INFO:teuthology.orchestra.run.smithi172.stdout: "url": { 2024-09-10T09:06:43.421 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "url", 2024-09-10T09:06:43.421 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.421 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.421 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.421 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T09:06:43.421 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.421 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.421 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.421 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.421 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.421 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.421 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.422 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.422 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.422 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.422 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.422 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "test_orchestrator", 2024-09-10T09:06:43.422 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.422 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.422 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.422 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.422 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.422 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.422 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.423 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.423 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.423 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.423 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.423 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.423 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.423 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.423 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.423 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.423 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.423 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.423 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.423 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.424 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.424 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.424 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.424 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.424 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.424 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.424 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.424 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.424 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.424 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.424 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.425 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.425 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.425 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.425 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.425 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.425 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.425 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.425 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.425 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.425 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.425 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.425 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.425 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.426 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.426 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.426 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.426 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.426 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.426 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.426 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.426 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.426 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.426 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.426 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.426 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.426 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.426 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.427 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.427 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.427 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.427 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.427 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.427 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.427 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.427 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.427 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.427 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.427 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.427 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.427 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.428 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.428 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.428 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.428 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.428 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.428 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "volumes", 2024-09-10T09:06:43.428 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.428 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.428 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.428 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.428 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.428 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.429 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.429 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.429 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.429 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.429 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.429 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.429 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.429 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.429 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.429 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.429 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.429 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.429 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.430 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.430 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.430 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.430 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.430 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.430 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.430 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.430 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.430 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.430 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.430 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.430 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.430 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.431 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.431 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.431 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.431 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.431 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.431 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.431 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.431 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.431 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.431 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.431 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.431 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.431 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.431 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.432 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.432 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.432 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.432 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.432 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.432 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.432 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.432 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.432 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.432 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.432 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.432 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.432 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.433 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.433 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.433 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.433 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.433 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.433 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.433 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.433 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.433 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.433 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.433 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.433 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.434 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.434 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.434 INFO:teuthology.orchestra.run.smithi172.stdout: "max_concurrent_clones": { 2024-09-10T09:06:43.434 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "max_concurrent_clones", 2024-09-10T09:06:43.434 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.434 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.434 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.434 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "4", 2024-09-10T09:06:43.434 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.434 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.434 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.434 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T09:06:43.434 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.434 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.435 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.435 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.435 INFO:teuthology.orchestra.run.smithi172.stdout: "periodic_async_work": { 2024-09-10T09:06:43.435 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "periodic_async_work", 2024-09-10T09:06:43.435 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.435 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.435 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.435 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.435 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.435 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.435 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.435 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Periodically check for async work", 2024-09-10T09:06:43.436 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.436 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.436 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.436 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.436 INFO:teuthology.orchestra.run.smithi172.stdout: "snapshot_clone_delay": { 2024-09-10T09:06:43.436 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "snapshot_clone_delay", 2024-09-10T09:06:43.436 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.436 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.436 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.436 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0", 2024-09-10T09:06:43.436 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.436 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.436 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.437 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T09:06:43.437 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.437 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.437 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.437 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.437 INFO:teuthology.orchestra.run.smithi172.stdout: "snapshot_clone_no_wait": { 2024-09-10T09:06:43.437 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T09:06:43.437 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.437 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.437 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.437 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.437 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.438 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.438 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.438 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T09:06:43.438 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.438 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.438 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.438 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.438 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.438 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.438 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.438 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "zabbix", 2024-09-10T09:06:43.438 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.438 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.439 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.439 INFO:teuthology.orchestra.run.smithi172.stdout: "discovery_interval": { 2024-09-10T09:06:43.439 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "discovery_interval", 2024-09-10T09:06:43.439 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "uint", 2024-09-10T09:06:43.439 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.439 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.439 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "100", 2024-09-10T09:06:43.439 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.439 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.439 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.439 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.439 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.439 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.440 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.440 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.440 INFO:teuthology.orchestra.run.smithi172.stdout: "identifier": { 2024-09-10T09:06:43.440 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "identifier", 2024-09-10T09:06:43.440 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.440 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.440 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.440 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.440 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.440 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.440 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.440 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.440 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.441 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.441 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.441 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.441 INFO:teuthology.orchestra.run.smithi172.stdout: "interval": { 2024-09-10T09:06:43.441 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "interval", 2024-09-10T09:06:43.441 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.441 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.441 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.441 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "60", 2024-09-10T09:06:43.441 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.441 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.441 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.442 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.442 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.442 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.442 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.442 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.442 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.442 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.442 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.442 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.442 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.442 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.442 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.442 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.443 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.443 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.443 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.443 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.443 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.443 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.443 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.443 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.443 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.443 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.443 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.443 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.443 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.443 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.444 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.444 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.444 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.444 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.444 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.444 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.444 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.444 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.444 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.444 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.444 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.444 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.444 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.445 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.445 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.445 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.445 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.445 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.445 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.445 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.445 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.445 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.445 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.445 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.445 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.446 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.446 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.446 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.446 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.446 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.446 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.446 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.446 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.446 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.446 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.446 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.446 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.446 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.447 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.447 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.447 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.447 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.447 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.447 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.447 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.447 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.447 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.447 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.447 INFO:teuthology.orchestra.run.smithi172.stdout: "zabbix_host": { 2024-09-10T09:06:43.447 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "zabbix_host", 2024-09-10T09:06:43.448 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.448 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.448 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.448 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.448 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.448 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.448 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.448 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.448 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.448 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.448 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.448 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.448 INFO:teuthology.orchestra.run.smithi172.stdout: "zabbix_port": { 2024-09-10T09:06:43.449 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "zabbix_port", 2024-09-10T09:06:43.449 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.449 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.449 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.449 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10051", 2024-09-10T09:06:43.449 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.449 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.449 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.449 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.449 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.449 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.449 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.449 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.450 INFO:teuthology.orchestra.run.smithi172.stdout: "zabbix_sender": { 2024-09-10T09:06:43.450 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "zabbix_sender", 2024-09-10T09:06:43.450 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.450 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.450 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.450 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T09:06:43.450 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.450 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.450 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.450 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.450 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.450 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.450 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.451 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.451 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.451 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.451 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-09-10T09:06:43.451 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.451 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.451 INFO:teuthology.orchestra.run.smithi172.stdout: "modules": [ 2024-09-10T09:06:43.451 INFO:teuthology.orchestra.run.smithi172.stdout: "cephadm", 2024-09-10T09:06:43.451 INFO:teuthology.orchestra.run.smithi172.stdout: "dashboard", 2024-09-10T09:06:43.451 INFO:teuthology.orchestra.run.smithi172.stdout: "iostat", 2024-09-10T09:06:43.451 INFO:teuthology.orchestra.run.smithi172.stdout: "nfs", 2024-09-10T09:06:43.451 INFO:teuthology.orchestra.run.smithi172.stdout: "restful" 2024-09-10T09:06:43.451 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.452 INFO:teuthology.orchestra.run.smithi172.stdout: "available_modules": [ 2024-09-10T09:06:43.452 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.452 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "alerts", 2024-09-10T09:06:43.452 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.452 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.452 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.452 INFO:teuthology.orchestra.run.smithi172.stdout: "interval": { 2024-09-10T09:06:43.452 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "interval", 2024-09-10T09:06:43.452 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.452 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.452 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.453 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "60", 2024-09-10T09:06:43.453 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.453 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.453 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.453 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T09:06:43.453 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.453 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.453 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.453 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.453 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.453 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.453 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.453 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.454 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.454 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.454 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.454 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.454 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.454 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.454 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.454 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.454 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.454 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.454 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.454 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.454 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.455 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.455 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.455 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.455 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.455 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.455 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.455 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.455 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.455 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.455 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.455 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.455 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.455 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.456 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.456 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.456 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.456 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.456 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.456 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.456 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.456 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.456 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.456 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.456 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.456 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.457 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.457 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.457 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.457 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.457 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.457 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.457 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.457 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.457 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.457 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.457 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.457 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.457 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.458 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.458 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.458 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.458 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.458 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.458 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.458 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.458 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.458 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.458 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.458 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.458 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.458 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.459 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.459 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.459 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_destination": { 2024-09-10T09:06:43.459 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_destination", 2024-09-10T09:06:43.459 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.459 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.459 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.459 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.459 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.459 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.459 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.459 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Email address to send alerts to", 2024-09-10T09:06:43.459 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.459 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.460 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.460 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.460 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_from_name": { 2024-09-10T09:06:43.460 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_from_name", 2024-09-10T09:06:43.460 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.460 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.460 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.460 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "Ceph", 2024-09-10T09:06:43.460 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.460 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.460 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.460 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Email From: name", 2024-09-10T09:06:43.461 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.461 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.461 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.461 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.461 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_host": { 2024-09-10T09:06:43.461 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_host", 2024-09-10T09:06:43.461 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.461 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.461 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.461 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.461 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.461 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.461 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.462 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "SMTP server", 2024-09-10T09:06:43.462 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.462 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.462 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.462 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.462 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_password": { 2024-09-10T09:06:43.462 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_password", 2024-09-10T09:06:43.462 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.462 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.462 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.462 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.462 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.462 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.463 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.463 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Password to authenticate with", 2024-09-10T09:06:43.463 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.463 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.463 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.463 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.463 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_port": { 2024-09-10T09:06:43.463 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_port", 2024-09-10T09:06:43.463 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.463 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.463 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.463 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "465", 2024-09-10T09:06:43.464 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.464 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.464 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.464 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "SMTP port", 2024-09-10T09:06:43.464 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.464 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.464 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.464 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.464 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_sender": { 2024-09-10T09:06:43.464 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_sender", 2024-09-10T09:06:43.464 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.464 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.464 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.465 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.465 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.465 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.465 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.465 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "SMTP envelope sender", 2024-09-10T09:06:43.465 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.465 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.465 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.465 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.465 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_ssl": { 2024-09-10T09:06:43.465 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_ssl", 2024-09-10T09:06:43.465 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.465 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.466 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.466 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.466 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.466 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.466 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.466 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T09:06:43.466 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.466 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.466 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.466 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.466 INFO:teuthology.orchestra.run.smithi172.stdout: "smtp_user": { 2024-09-10T09:06:43.466 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "smtp_user", 2024-09-10T09:06:43.466 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.467 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.467 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.467 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.467 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.467 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.467 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.467 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "User to authenticate as", 2024-09-10T09:06:43.467 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.467 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.467 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.467 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.467 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.467 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.468 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.468 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "balancer", 2024-09-10T09:06:43.468 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.468 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.468 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.468 INFO:teuthology.orchestra.run.smithi172.stdout: "active": { 2024-09-10T09:06:43.468 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "active", 2024-09-10T09:06:43.468 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.468 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.468 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.468 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.468 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.468 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.469 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.469 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T09:06:43.469 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.469 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.469 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.469 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.469 INFO:teuthology.orchestra.run.smithi172.stdout: "begin_time": { 2024-09-10T09:06:43.469 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "begin_time", 2024-09-10T09:06:43.469 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.469 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.469 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.469 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0000", 2024-09-10T09:06:43.469 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.470 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.470 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.470 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T09:06:43.470 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T09:06:43.470 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.470 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.470 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.470 INFO:teuthology.orchestra.run.smithi172.stdout: "begin_weekday": { 2024-09-10T09:06:43.470 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "begin_weekday", 2024-09-10T09:06:43.470 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "uint", 2024-09-10T09:06:43.470 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.470 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.470 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0", 2024-09-10T09:06:43.471 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "0", 2024-09-10T09:06:43.471 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "6", 2024-09-10T09:06:43.471 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.471 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T09:06:43.471 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T09:06:43.471 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.471 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.471 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.471 INFO:teuthology.orchestra.run.smithi172.stdout: "crush_compat_max_iterations": { 2024-09-10T09:06:43.471 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "crush_compat_max_iterations", 2024-09-10T09:06:43.471 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "uint", 2024-09-10T09:06:43.471 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.472 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.472 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "25", 2024-09-10T09:06:43.472 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "1", 2024-09-10T09:06:43.472 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "250", 2024-09-10T09:06:43.472 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.472 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T09:06:43.472 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.472 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.472 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.472 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.472 INFO:teuthology.orchestra.run.smithi172.stdout: "crush_compat_metrics": { 2024-09-10T09:06:43.472 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "crush_compat_metrics", 2024-09-10T09:06:43.473 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.473 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.473 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.473 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T09:06:43.473 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.473 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.473 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.473 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T09:06:43.473 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T09:06:43.473 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.473 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.473 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.474 INFO:teuthology.orchestra.run.smithi172.stdout: "crush_compat_step": { 2024-09-10T09:06:43.474 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "crush_compat_step", 2024-09-10T09:06:43.474 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:43.474 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.474 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.474 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0.5", 2024-09-10T09:06:43.474 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "0.001", 2024-09-10T09:06:43.474 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "0.999", 2024-09-10T09:06:43.474 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.474 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "aggressiveness of optimization", 2024-09-10T09:06:43.474 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T09:06:43.474 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.475 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.475 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.475 INFO:teuthology.orchestra.run.smithi172.stdout: "end_time": { 2024-09-10T09:06:43.475 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "end_time", 2024-09-10T09:06:43.475 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.475 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.475 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.475 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "2359", 2024-09-10T09:06:43.475 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.475 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.475 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.475 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T09:06:43.475 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T09:06:43.476 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.476 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.476 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.476 INFO:teuthology.orchestra.run.smithi172.stdout: "end_weekday": { 2024-09-10T09:06:43.476 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "end_weekday", 2024-09-10T09:06:43.476 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "uint", 2024-09-10T09:06:43.476 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.476 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.476 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0", 2024-09-10T09:06:43.476 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "0", 2024-09-10T09:06:43.476 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "6", 2024-09-10T09:06:43.476 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T09:06:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T09:06:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.477 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.478 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.478 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.478 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.478 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.478 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.478 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.478 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.478 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.478 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.478 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.478 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.478 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.478 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.479 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.479 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.479 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.479 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.479 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.479 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.479 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.479 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.479 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.479 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.479 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.479 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.479 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.480 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.480 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.480 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.480 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.480 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.480 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.480 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.480 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.480 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.480 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.480 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.480 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.480 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.480 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.481 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.481 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.481 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.481 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.481 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.481 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.481 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.481 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.481 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.481 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.481 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.481 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.482 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.482 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.482 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.482 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.482 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.482 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.482 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.482 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.482 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.482 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.482 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.482 INFO:teuthology.orchestra.run.smithi172.stdout: "min_score": { 2024-09-10T09:06:43.482 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "min_score", 2024-09-10T09:06:43.483 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:43.483 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.483 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.483 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0", 2024-09-10T09:06:43.483 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.483 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.483 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.483 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T09:06:43.483 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.483 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.483 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.483 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.483 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": { 2024-09-10T09:06:43.484 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mode", 2024-09-10T09:06:43.484 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.484 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.484 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.484 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "upmap", 2024-09-10T09:06:43.484 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.484 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.484 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.484 INFO:teuthology.orchestra.run.smithi172.stdout: "crush-compat", 2024-09-10T09:06:43.484 INFO:teuthology.orchestra.run.smithi172.stdout: "none", 2024-09-10T09:06:43.484 INFO:teuthology.orchestra.run.smithi172.stdout: "read", 2024-09-10T09:06:43.484 INFO:teuthology.orchestra.run.smithi172.stdout: "upmap", 2024-09-10T09:06:43.485 INFO:teuthology.orchestra.run.smithi172.stdout: "upmap-read" 2024-09-10T09:06:43.485 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.485 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Balancer mode", 2024-09-10T09:06:43.485 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.485 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.485 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.485 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.485 INFO:teuthology.orchestra.run.smithi172.stdout: "pool_ids": { 2024-09-10T09:06:43.485 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "pool_ids", 2024-09-10T09:06:43.485 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.485 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.485 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.485 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.486 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.486 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.486 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.486 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T09:06:43.486 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.486 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.486 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.486 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.486 INFO:teuthology.orchestra.run.smithi172.stdout: "sleep_interval": { 2024-09-10T09:06:43.486 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "sleep_interval", 2024-09-10T09:06:43.486 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.486 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.486 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.487 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "60", 2024-09-10T09:06:43.487 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.487 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.487 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.487 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T09:06:43.487 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.487 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.487 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.487 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.487 INFO:teuthology.orchestra.run.smithi172.stdout: "upmap_max_deviation": { 2024-09-10T09:06:43.487 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "upmap_max_deviation", 2024-09-10T09:06:43.487 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.487 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.488 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.488 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "5", 2024-09-10T09:06:43.488 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "1", 2024-09-10T09:06:43.488 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.488 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.488 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T09:06:43.488 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T09:06:43.488 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.488 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.488 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.488 INFO:teuthology.orchestra.run.smithi172.stdout: "upmap_max_optimizations": { 2024-09-10T09:06:43.488 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "upmap_max_optimizations", 2024-09-10T09:06:43.489 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "uint", 2024-09-10T09:06:43.489 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.489 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.489 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:43.489 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.489 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.489 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.489 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T09:06:43.489 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.489 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.489 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.489 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.490 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.490 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.490 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.490 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "cephadm", 2024-09-10T09:06:43.490 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.490 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.490 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.490 INFO:teuthology.orchestra.run.smithi172.stdout: "agent_down_multiplier": { 2024-09-10T09:06:43.490 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "agent_down_multiplier", 2024-09-10T09:06:43.490 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:43.490 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.490 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.490 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "3.0", 2024-09-10T09:06:43.491 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.491 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.491 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.491 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T09:06:43.491 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.491 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.491 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.491 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.491 INFO:teuthology.orchestra.run.smithi172.stdout: "agent_refresh_rate": { 2024-09-10T09:06:43.491 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "agent_refresh_rate", 2024-09-10T09:06:43.491 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.491 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.491 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.492 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "20", 2024-09-10T09:06:43.492 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.492 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.492 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.492 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T09:06:43.492 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.492 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.492 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.492 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.492 INFO:teuthology.orchestra.run.smithi172.stdout: "agent_starting_port": { 2024-09-10T09:06:43.492 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "agent_starting_port", 2024-09-10T09:06:43.492 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.492 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.493 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.493 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "4721", 2024-09-10T09:06:43.493 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.493 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.493 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.493 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T09:06:43.493 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.493 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.493 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.493 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.493 INFO:teuthology.orchestra.run.smithi172.stdout: "allow_ptrace": { 2024-09-10T09:06:43.493 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "allow_ptrace", 2024-09-10T09:06:43.493 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.494 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.494 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.494 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.494 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.494 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.494 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.494 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T09:06:43.494 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T09:06:43.494 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.494 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.494 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.494 INFO:teuthology.orchestra.run.smithi172.stdout: "autotune_interval": { 2024-09-10T09:06:43.495 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "autotune_interval", 2024-09-10T09:06:43.495 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.495 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.495 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.495 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "600", 2024-09-10T09:06:43.495 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.495 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.495 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.495 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T09:06:43.495 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.495 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.495 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.495 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.496 INFO:teuthology.orchestra.run.smithi172.stdout: "autotune_memory_target_ratio": { 2024-09-10T09:06:43.496 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T09:06:43.496 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:43.496 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.496 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.496 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0.7", 2024-09-10T09:06:43.496 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.496 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.496 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.496 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T09:06:43.496 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.496 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.497 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.497 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.497 INFO:teuthology.orchestra.run.smithi172.stdout: "cephadm_log_destination": { 2024-09-10T09:06:43.497 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "cephadm_log_destination", 2024-09-10T09:06:43.497 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.497 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.497 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.497 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.497 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.497 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.497 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.497 INFO:teuthology.orchestra.run.smithi172.stdout: "file", 2024-09-10T09:06:43.497 INFO:teuthology.orchestra.run.smithi172.stdout: "file,syslog", 2024-09-10T09:06:43.498 INFO:teuthology.orchestra.run.smithi172.stdout: "syslog" 2024-09-10T09:06:43.498 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.498 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T09:06:43.498 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.498 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.498 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.498 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.498 INFO:teuthology.orchestra.run.smithi172.stdout: "cgroups_split": { 2024-09-10T09:06:43.498 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "cgroups_split", 2024-09-10T09:06:43.498 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.498 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.499 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.499 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.499 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.499 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.499 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.499 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T09:06:43.499 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.499 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.499 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.499 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.499 INFO:teuthology.orchestra.run.smithi172.stdout: "config_checks_enabled": { 2024-09-10T09:06:43.499 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "config_checks_enabled", 2024-09-10T09:06:43.499 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.499 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.500 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.500 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.500 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.500 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.500 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.500 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T09:06:43.500 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.500 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.500 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.500 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.500 INFO:teuthology.orchestra.run.smithi172.stdout: "config_dashboard": { 2024-09-10T09:06:43.501 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "config_dashboard", 2024-09-10T09:06:43.501 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.501 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.501 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.501 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.501 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.501 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.501 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.501 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T09:06:43.501 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.501 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.501 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.502 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.502 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_alertmanager": { 2024-09-10T09:06:43.502 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_alertmanager", 2024-09-10T09:06:43.502 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.502 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.502 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.502 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T09:06:43.502 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.502 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.502 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.502 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Prometheus container image", 2024-09-10T09:06:43.502 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.502 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.503 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.503 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.503 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_base": { 2024-09-10T09:06:43.503 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_base", 2024-09-10T09:06:43.503 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.503 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.503 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.503 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T09:06:43.503 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.503 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.503 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.503 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Container image name, without the tag", 2024-09-10T09:06:43.503 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.504 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.504 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.504 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.504 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_elasticsearch": { 2024-09-10T09:06:43.504 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_elasticsearch", 2024-09-10T09:06:43.504 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.504 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.504 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.504 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T09:06:43.504 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.504 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.504 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.504 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "elasticsearch container image", 2024-09-10T09:06:43.505 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.505 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.505 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.505 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.505 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_grafana": { 2024-09-10T09:06:43.505 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_grafana", 2024-09-10T09:06:43.505 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.505 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.505 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.505 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T09:06:43.505 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.505 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.505 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.506 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Prometheus container image", 2024-09-10T09:06:43.506 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.506 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.506 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.506 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.506 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_haproxy": { 2024-09-10T09:06:43.506 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_haproxy", 2024-09-10T09:06:43.506 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.506 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.506 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.506 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T09:06:43.506 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.506 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.507 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.507 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "HAproxy container image", 2024-09-10T09:06:43.507 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.507 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.507 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.507 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.507 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_jaeger_agent": { 2024-09-10T09:06:43.507 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_jaeger_agent", 2024-09-10T09:06:43.507 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.507 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.507 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.507 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T09:06:43.507 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.508 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.508 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.508 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Jaeger agent container image", 2024-09-10T09:06:43.508 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.508 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.508 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.508 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.508 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_jaeger_collector": { 2024-09-10T09:06:43.508 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_jaeger_collector", 2024-09-10T09:06:43.508 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.508 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.508 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.508 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T09:06:43.509 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.509 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.509 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.509 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Jaeger collector container image", 2024-09-10T09:06:43.509 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.509 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.509 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.509 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.509 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_jaeger_query": { 2024-09-10T09:06:43.509 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_jaeger_query", 2024-09-10T09:06:43.509 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.509 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.509 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.510 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T09:06:43.510 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.510 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.510 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.510 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Jaeger query container image", 2024-09-10T09:06:43.510 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.510 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.510 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.510 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.510 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_keepalived": { 2024-09-10T09:06:43.510 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_keepalived", 2024-09-10T09:06:43.510 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.510 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.511 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.511 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T09:06:43.511 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.511 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.511 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.511 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Keepalived container image", 2024-09-10T09:06:43.511 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.511 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.511 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.511 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.511 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_loki": { 2024-09-10T09:06:43.511 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_loki", 2024-09-10T09:06:43.511 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.511 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.512 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.512 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T09:06:43.512 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.512 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.512 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.512 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Loki container image", 2024-09-10T09:06:43.512 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.512 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.512 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.512 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.512 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_node_exporter": { 2024-09-10T09:06:43.512 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_node_exporter", 2024-09-10T09:06:43.512 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.513 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.513 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.513 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T09:06:43.513 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.513 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.513 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.513 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Prometheus container image", 2024-09-10T09:06:43.513 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.513 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.513 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.513 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.513 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_nvmeof": { 2024-09-10T09:06:43.513 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_nvmeof", 2024-09-10T09:06:43.514 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.514 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.514 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.514 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T09:06:43.514 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.514 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.514 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.514 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Nvme-of container image", 2024-09-10T09:06:43.514 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.514 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.514 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.514 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.514 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_prometheus": { 2024-09-10T09:06:43.515 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_prometheus", 2024-09-10T09:06:43.515 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.515 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.515 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.515 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T09:06:43.515 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.515 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.515 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.515 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Prometheus container image", 2024-09-10T09:06:43.515 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.515 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.515 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.515 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.516 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_promtail": { 2024-09-10T09:06:43.516 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_promtail", 2024-09-10T09:06:43.516 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.516 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.516 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.516 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T09:06:43.516 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.516 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.516 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.516 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Promtail container image", 2024-09-10T09:06:43.516 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.516 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.516 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.517 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.517 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_samba": { 2024-09-10T09:06:43.517 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_samba", 2024-09-10T09:06:43.517 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.517 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.517 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.517 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T09:06:43.517 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.517 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.517 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.517 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Samba/SMB container image", 2024-09-10T09:06:43.517 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.518 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.518 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.518 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.518 INFO:teuthology.orchestra.run.smithi172.stdout: "container_image_snmp_gateway": { 2024-09-10T09:06:43.518 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_image_snmp_gateway", 2024-09-10T09:06:43.518 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.518 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.518 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.518 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T09:06:43.518 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.518 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.518 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.518 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "SNMP Gateway container image", 2024-09-10T09:06:43.518 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.519 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.519 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.519 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.519 INFO:teuthology.orchestra.run.smithi172.stdout: "container_init": { 2024-09-10T09:06:43.519 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "container_init", 2024-09-10T09:06:43.519 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.519 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.519 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.519 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.519 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.519 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.519 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.520 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T09:06:43.520 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.520 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.520 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.520 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.520 INFO:teuthology.orchestra.run.smithi172.stdout: "daemon_cache_timeout": { 2024-09-10T09:06:43.520 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "daemon_cache_timeout", 2024-09-10T09:06:43.520 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.520 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.520 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.520 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "600", 2024-09-10T09:06:43.520 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.521 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.521 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.521 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T09:06:43.521 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.521 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.521 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.521 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.521 INFO:teuthology.orchestra.run.smithi172.stdout: "default_cephadm_command_timeout": { 2024-09-10T09:06:43.521 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T09:06:43.521 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.521 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.521 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.522 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "900", 2024-09-10T09:06:43.522 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.522 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.522 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.522 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T09:06:43.522 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.522 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.522 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.522 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.522 INFO:teuthology.orchestra.run.smithi172.stdout: "default_registry": { 2024-09-10T09:06:43.522 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "default_registry", 2024-09-10T09:06:43.522 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.522 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.523 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.523 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "docker.io", 2024-09-10T09:06:43.523 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.523 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.523 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.523 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T09:06:43.523 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.523 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.523 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.523 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.523 INFO:teuthology.orchestra.run.smithi172.stdout: "device_cache_timeout": { 2024-09-10T09:06:43.523 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "device_cache_timeout", 2024-09-10T09:06:43.524 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.524 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.524 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.524 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "1800", 2024-09-10T09:06:43.524 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.524 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.524 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.524 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "seconds to cache device inventory", 2024-09-10T09:06:43.524 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.524 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.524 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.524 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.525 INFO:teuthology.orchestra.run.smithi172.stdout: "device_enhanced_scan": { 2024-09-10T09:06:43.525 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "device_enhanced_scan", 2024-09-10T09:06:43.525 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.525 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.525 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.525 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.525 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.525 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.525 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.525 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T09:06:43.525 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.525 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.525 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.526 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.526 INFO:teuthology.orchestra.run.smithi172.stdout: "facts_cache_timeout": { 2024-09-10T09:06:43.526 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "facts_cache_timeout", 2024-09-10T09:06:43.526 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.526 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.526 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.526 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "60", 2024-09-10T09:06:43.526 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.526 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.526 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.526 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "seconds to cache host facts data", 2024-09-10T09:06:43.526 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.526 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.527 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.527 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.527 INFO:teuthology.orchestra.run.smithi172.stdout: "grafana_dashboards_path": { 2024-09-10T09:06:43.527 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "grafana_dashboards_path", 2024-09-10T09:06:43.527 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.527 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.527 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.527 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T09:06:43.527 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.527 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.527 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.527 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T09:06:43.527 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.528 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.528 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.528 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.528 INFO:teuthology.orchestra.run.smithi172.stdout: "host_check_interval": { 2024-09-10T09:06:43.528 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "host_check_interval", 2024-09-10T09:06:43.528 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.528 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.528 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.528 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "600", 2024-09-10T09:06:43.528 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.528 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.528 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.529 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how frequently to perform a host check", 2024-09-10T09:06:43.529 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.529 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.529 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.529 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.529 INFO:teuthology.orchestra.run.smithi172.stdout: "hw_monitoring": { 2024-09-10T09:06:43.529 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "hw_monitoring", 2024-09-10T09:06:43.529 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.529 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.529 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.529 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.529 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.529 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.530 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.530 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T09:06:43.530 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.530 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.530 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.530 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.530 INFO:teuthology.orchestra.run.smithi172.stdout: "inventory_list_all": { 2024-09-10T09:06:43.530 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "inventory_list_all", 2024-09-10T09:06:43.530 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.530 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.530 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.530 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.530 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.531 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.531 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.531 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T09:06:43.531 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.531 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.531 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.531 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.531 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.531 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.531 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.531 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.531 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.531 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.532 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.532 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.532 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.532 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.532 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.532 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.532 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.532 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.532 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.532 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.532 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.532 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.532 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.533 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.533 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.533 INFO:teuthology.orchestra.run.smithi172.stdout: "log_refresh_metadata": { 2024-09-10T09:06:43.533 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_refresh_metadata", 2024-09-10T09:06:43.533 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.533 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.533 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.533 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.533 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.533 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.533 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.533 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T09:06:43.533 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.534 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.534 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.534 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.534 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.534 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.534 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.534 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.534 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.534 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.534 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.534 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.534 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.534 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T09:06:43.535 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.535 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.535 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.535 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.535 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.535 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.535 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.535 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.535 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.535 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.535 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.535 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.536 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.536 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.536 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.536 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.536 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.536 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.536 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.536 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.536 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.536 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.536 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.536 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.537 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.537 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.537 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.537 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.537 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.537 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.537 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.537 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.537 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.537 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.537 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.537 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.537 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.538 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.538 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.538 INFO:teuthology.orchestra.run.smithi172.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T09:06:43.538 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T09:06:43.538 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.538 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.538 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.538 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.538 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.538 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.538 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.538 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T09:06:43.539 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.539 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.539 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.539 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.539 INFO:teuthology.orchestra.run.smithi172.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T09:06:43.539 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T09:06:43.539 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.539 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.539 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.539 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "*", 2024-09-10T09:06:43.539 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.539 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.539 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.540 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T09:06:43.540 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.540 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.540 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.540 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.540 INFO:teuthology.orchestra.run.smithi172.stdout: "max_count_per_host": { 2024-09-10T09:06:43.540 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "max_count_per_host", 2024-09-10T09:06:43.540 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.540 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.540 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.540 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "max number of daemons per service per host", 2024-09-10T09:06:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: "max_osd_draining_count": { 2024-09-10T09:06:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "max_osd_draining_count", 2024-09-10T09:06:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.541 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T09:06:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: "migration_current": { 2024-09-10T09:06:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "migration_current", 2024-09-10T09:06:43.542 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "internal - do not modify", 2024-09-10T09:06:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: "mode": { 2024-09-10T09:06:43.543 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mode", 2024-09-10T09:06:43.544 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.544 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.544 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.544 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "root", 2024-09-10T09:06:43.544 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.544 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.544 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.544 INFO:teuthology.orchestra.run.smithi172.stdout: "cephadm-package", 2024-09-10T09:06:43.544 INFO:teuthology.orchestra.run.smithi172.stdout: "root" 2024-09-10T09:06:43.544 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.544 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T09:06:43.544 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.545 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.545 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.545 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.545 INFO:teuthology.orchestra.run.smithi172.stdout: "oob_default_addr": { 2024-09-10T09:06:43.545 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "oob_default_addr", 2024-09-10T09:06:43.545 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.545 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.545 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.545 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "169.254.1.1", 2024-09-10T09:06:43.545 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.545 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.545 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.546 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T09:06:43.546 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.546 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.546 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.546 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.546 INFO:teuthology.orchestra.run.smithi172.stdout: "prometheus_alerts_path": { 2024-09-10T09:06:43.546 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "prometheus_alerts_path", 2024-09-10T09:06:43.546 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.546 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.546 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.546 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T09:06:43.546 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.546 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.547 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.547 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T09:06:43.547 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.547 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.547 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.547 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.547 INFO:teuthology.orchestra.run.smithi172.stdout: "registry_insecure": { 2024-09-10T09:06:43.547 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "registry_insecure", 2024-09-10T09:06:43.547 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.547 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.547 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.547 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.547 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.548 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.548 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.548 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T09:06:43.548 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.548 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.548 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.548 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.548 INFO:teuthology.orchestra.run.smithi172.stdout: "registry_password": { 2024-09-10T09:06:43.548 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "registry_password", 2024-09-10T09:06:43.548 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.548 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.548 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.548 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.549 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.549 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.549 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.549 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T09:06:43.549 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.549 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.549 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.549 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.549 INFO:teuthology.orchestra.run.smithi172.stdout: "registry_url": { 2024-09-10T09:06:43.549 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "registry_url", 2024-09-10T09:06:43.549 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.549 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.549 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.550 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.550 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.550 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.550 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.550 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T09:06:43.550 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.550 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.550 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.550 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.550 INFO:teuthology.orchestra.run.smithi172.stdout: "registry_username": { 2024-09-10T09:06:43.550 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "registry_username", 2024-09-10T09:06:43.550 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.550 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.551 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.551 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.551 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.551 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.551 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.551 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T09:06:43.551 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.551 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.551 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.551 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.551 INFO:teuthology.orchestra.run.smithi172.stdout: "secure_monitoring_stack": { 2024-09-10T09:06:43.551 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "secure_monitoring_stack", 2024-09-10T09:06:43.551 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.552 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.552 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.552 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.552 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.552 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.552 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.552 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T09:06:43.552 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.552 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.552 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.552 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.553 INFO:teuthology.orchestra.run.smithi172.stdout: "service_discovery_port": { 2024-09-10T09:06:43.553 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "service_discovery_port", 2024-09-10T09:06:43.553 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.553 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.553 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.553 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "8765", 2024-09-10T09:06:43.553 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.553 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.553 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.553 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "cephadm service discovery port", 2024-09-10T09:06:43.553 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.553 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.554 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.554 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.554 INFO:teuthology.orchestra.run.smithi172.stdout: "ssh_config_file": { 2024-09-10T09:06:43.554 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ssh_config_file", 2024-09-10T09:06:43.554 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.554 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.554 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.554 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.554 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.554 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.554 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.554 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T09:06:43.554 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.555 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.555 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.555 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.555 INFO:teuthology.orchestra.run.smithi172.stdout: "use_agent": { 2024-09-10T09:06:43.555 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "use_agent", 2024-09-10T09:06:43.555 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.555 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.555 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.555 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.555 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.555 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.555 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.556 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T09:06:43.556 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.556 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.556 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.556 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.556 INFO:teuthology.orchestra.run.smithi172.stdout: "use_repo_digest": { 2024-09-10T09:06:43.556 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "use_repo_digest", 2024-09-10T09:06:43.556 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.556 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.556 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.556 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.556 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.556 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.556 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.557 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T09:06:43.557 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.557 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.557 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.557 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.557 INFO:teuthology.orchestra.run.smithi172.stdout: "warn_on_failed_host_check": { 2024-09-10T09:06:43.557 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "warn_on_failed_host_check", 2024-09-10T09:06:43.557 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.557 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.557 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.557 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.557 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.558 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.558 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.558 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T09:06:43.558 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.558 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.558 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.558 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.558 INFO:teuthology.orchestra.run.smithi172.stdout: "warn_on_stray_daemons": { 2024-09-10T09:06:43.558 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "warn_on_stray_daemons", 2024-09-10T09:06:43.558 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.558 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.558 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.558 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.559 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.559 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.559 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.559 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T09:06:43.559 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.559 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.559 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.559 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.559 INFO:teuthology.orchestra.run.smithi172.stdout: "warn_on_stray_hosts": { 2024-09-10T09:06:43.559 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "warn_on_stray_hosts", 2024-09-10T09:06:43.559 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.559 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.559 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.559 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.560 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.560 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.560 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.560 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T09:06:43.560 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.560 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.560 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.560 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.560 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.560 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.560 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.560 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "crash", 2024-09-10T09:06:43.560 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.560 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.561 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.561 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.561 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.561 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.561 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.561 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.561 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.561 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.561 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.561 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.561 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.561 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.562 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.562 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.562 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.562 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.562 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.562 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.562 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.562 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.562 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.562 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.562 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.562 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.562 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.562 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.562 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.563 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.563 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.563 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.563 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.563 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.563 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.563 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.563 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.563 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.563 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.563 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.563 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.564 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.564 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.564 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.564 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.564 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.564 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.564 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.564 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.564 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.564 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.564 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.564 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.564 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.564 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.565 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.565 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.565 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.565 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.565 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.565 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.565 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.565 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.565 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.565 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.565 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.565 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.565 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.565 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.566 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.566 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.566 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.566 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.566 INFO:teuthology.orchestra.run.smithi172.stdout: "retain_interval": { 2024-09-10T09:06:43.566 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "retain_interval", 2024-09-10T09:06:43.566 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.566 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.566 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.566 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "31536000", 2024-09-10T09:06:43.566 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.566 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.566 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.567 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T09:06:43.567 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.567 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.567 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.567 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.567 INFO:teuthology.orchestra.run.smithi172.stdout: "warn_recent_interval": { 2024-09-10T09:06:43.567 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "warn_recent_interval", 2024-09-10T09:06:43.567 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.567 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.567 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.567 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "1209600", 2024-09-10T09:06:43.567 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.567 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.568 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.568 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T09:06:43.568 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.568 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.568 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.568 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.568 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.568 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.568 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.568 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "dashboard", 2024-09-10T09:06:43.568 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.568 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.568 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.568 INFO:teuthology.orchestra.run.smithi172.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T09:06:43.569 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T09:06:43.569 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.569 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.569 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.569 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:43.569 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.569 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.569 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.569 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.569 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.569 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.569 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.569 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.570 INFO:teuthology.orchestra.run.smithi172.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T09:06:43.570 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T09:06:43.570 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.570 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.570 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.570 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.570 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.570 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.570 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.570 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.570 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.570 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.570 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.571 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.571 INFO:teuthology.orchestra.run.smithi172.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T09:06:43.571 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T09:06:43.571 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.571 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.571 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.571 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.571 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.571 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.571 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.571 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.571 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.572 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.572 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.572 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.572 INFO:teuthology.orchestra.run.smithi172.stdout: "AUDIT_API_ENABLED": { 2024-09-10T09:06:43.572 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T09:06:43.572 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.572 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.572 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.572 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.572 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.572 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.572 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.573 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.573 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.573 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.573 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.573 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.573 INFO:teuthology.orchestra.run.smithi172.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T09:06:43.573 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T09:06:43.573 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.573 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.573 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.573 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.573 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.573 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.574 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.574 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.574 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.574 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.574 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.574 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.574 INFO:teuthology.orchestra.run.smithi172.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T09:06:43.574 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T09:06:43.574 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.574 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.574 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.575 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.575 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.575 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.575 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.575 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.575 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.575 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.575 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.575 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.575 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T09:06:43.575 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T09:06:43.575 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.575 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.576 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.576 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.576 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.576 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.576 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.576 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.576 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.576 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.576 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.576 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.576 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T09:06:43.576 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T09:06:43.577 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.577 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.577 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.577 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.577 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.577 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.577 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.577 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.577 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.577 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.577 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.577 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.577 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T09:06:43.577 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T09:06:43.578 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.578 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.578 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.578 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.578 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.578 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.578 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.578 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.578 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.578 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.578 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.578 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.578 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T09:06:43.578 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T09:06:43.579 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.579 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.579 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.579 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.579 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.579 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.579 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.579 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.579 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.579 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.579 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.579 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.579 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T09:06:43.580 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T09:06:43.580 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.580 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.580 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.580 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.580 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.580 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.580 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.580 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.580 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.580 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.580 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.581 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.581 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T09:06:43.581 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T09:06:43.581 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.581 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.581 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.581 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.581 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.581 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.581 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.581 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.581 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.581 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.582 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.582 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.582 INFO:teuthology.orchestra.run.smithi172.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T09:06:43.582 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T09:06:43.582 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.582 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.582 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.582 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.582 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.582 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.582 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.582 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.583 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.583 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.583 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.583 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.583 INFO:teuthology.orchestra.run.smithi172.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T09:06:43.583 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T09:06:43.583 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.583 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.583 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.583 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.583 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.583 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.583 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.584 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.584 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.584 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.584 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.584 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.584 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T09:06:43.584 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T09:06:43.584 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.584 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.584 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.584 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "admin", 2024-09-10T09:06:43.584 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.585 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.585 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.585 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.585 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.585 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.585 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.585 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.585 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T09:06:43.585 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T09:06:43.585 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.585 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.585 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.585 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.586 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.586 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.586 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.586 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.586 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.586 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.586 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.586 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.586 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_API_URL": { 2024-09-10T09:06:43.586 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_API_URL", 2024-09-10T09:06:43.586 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.586 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.586 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.586 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.587 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.587 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.587 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.587 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.587 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.587 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.587 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.587 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.587 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T09:06:43.587 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T09:06:43.587 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.587 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.588 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.588 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "admin", 2024-09-10T09:06:43.588 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.588 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.588 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.588 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.588 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.588 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.588 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.588 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.588 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T09:06:43.588 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T09:06:43.589 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.589 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.589 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.589 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.589 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.589 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.589 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.589 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.589 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.589 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.589 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.589 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.590 INFO:teuthology.orchestra.run.smithi172.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T09:06:43.590 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T09:06:43.590 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.590 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.590 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.590 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.590 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.590 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.590 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.590 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.590 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.590 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.590 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.591 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.591 INFO:teuthology.orchestra.run.smithi172.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T09:06:43.591 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T09:06:43.591 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.591 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.591 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.591 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.591 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.591 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.591 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.591 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.591 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.591 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.592 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.592 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.592 INFO:teuthology.orchestra.run.smithi172.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T09:06:43.592 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T09:06:43.592 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.592 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.592 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.592 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.592 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.592 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.592 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.592 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.592 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.593 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.593 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.593 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.593 INFO:teuthology.orchestra.run.smithi172.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T09:06:43.593 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T09:06:43.593 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.593 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.593 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.593 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.593 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.593 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.593 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.594 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.594 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.594 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.594 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.594 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.594 INFO:teuthology.orchestra.run.smithi172.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T09:06:43.594 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T09:06:43.594 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.594 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.594 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.594 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.594 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.594 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.595 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.595 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.595 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.595 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.595 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.595 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.595 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T09:06:43.595 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T09:06:43.595 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.595 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.595 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.595 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.596 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.596 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.596 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.596 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.596 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.596 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.596 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.596 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.596 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T09:06:43.596 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T09:06:43.596 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.596 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.596 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.597 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.597 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.597 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.597 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.597 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.597 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.597 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.597 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.597 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.597 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T09:06:43.597 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T09:06:43.597 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.598 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.598 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.598 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.598 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.598 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.598 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.598 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.598 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.598 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.598 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.598 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.598 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T09:06:43.598 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T09:06:43.598 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.599 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.599 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.599 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.599 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.599 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.599 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.599 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.599 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.599 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.599 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.599 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.599 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T09:06:43.599 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T09:06:43.600 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.600 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.600 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.600 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.600 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.600 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.600 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.600 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.600 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.600 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.600 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.600 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.600 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T09:06:43.601 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T09:06:43.601 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.601 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.601 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.601 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.601 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.601 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.601 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.601 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.601 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.601 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.601 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.602 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.602 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T09:06:43.602 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T09:06:43.602 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.602 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.602 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.602 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.602 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.602 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.602 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.602 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.602 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.602 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.603 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.603 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.603 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T09:06:43.603 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T09:06:43.603 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.603 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.603 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.603 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.603 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.603 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.603 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.603 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.603 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.603 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.604 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.604 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.604 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T09:06:43.604 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T09:06:43.604 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.604 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.604 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.604 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T09:06:43.604 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.604 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.604 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.605 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.605 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.605 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.605 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.605 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.605 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T09:06:43.605 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T09:06:43.605 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.605 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.605 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.605 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:43.605 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.606 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.606 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.606 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.606 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.606 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.606 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.606 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.606 INFO:teuthology.orchestra.run.smithi172.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T09:06:43.606 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T09:06:43.606 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.606 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.606 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.606 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "8", 2024-09-10T09:06:43.607 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.607 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.607 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.607 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.607 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.607 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.607 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.607 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.607 INFO:teuthology.orchestra.run.smithi172.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T09:06:43.607 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T09:06:43.607 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.607 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.607 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.608 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "45", 2024-09-10T09:06:43.608 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.608 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.608 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.608 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.608 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.608 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.608 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.608 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.608 INFO:teuthology.orchestra.run.smithi172.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T09:06:43.608 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T09:06:43.608 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.609 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.609 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.609 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.609 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.609 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.609 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.609 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.609 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.609 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.609 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.609 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.609 INFO:teuthology.orchestra.run.smithi172.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T09:06:43.610 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T09:06:43.610 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.610 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.610 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.610 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "admin", 2024-09-10T09:06:43.610 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.610 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.610 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.610 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.610 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.610 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.610 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.611 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.611 INFO:teuthology.orchestra.run.smithi172.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T09:06:43.611 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T09:06:43.611 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.611 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.611 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.611 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.611 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.611 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.611 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.611 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.611 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.611 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.611 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.612 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.612 INFO:teuthology.orchestra.run.smithi172.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T09:06:43.612 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T09:06:43.612 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.612 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.612 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.612 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.612 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.612 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.612 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.612 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.612 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.612 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.613 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.613 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.613 INFO:teuthology.orchestra.run.smithi172.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T09:06:43.613 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T09:06:43.613 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.613 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.613 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.613 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0", 2024-09-10T09:06:43.613 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.613 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.613 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.613 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.613 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.614 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.614 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.614 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.614 INFO:teuthology.orchestra.run.smithi172.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T09:06:43.614 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T09:06:43.614 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.614 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.614 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.614 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:43.614 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.614 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.614 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.614 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.615 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.615 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.615 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.615 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.615 INFO:teuthology.orchestra.run.smithi172.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T09:06:43.615 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T09:06:43.615 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.615 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.615 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.615 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "5", 2024-09-10T09:06:43.615 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.615 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.615 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.616 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.616 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.616 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.616 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.616 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.616 INFO:teuthology.orchestra.run.smithi172.stdout: "cross_origin_url": { 2024-09-10T09:06:43.616 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "cross_origin_url", 2024-09-10T09:06:43.616 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.616 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.616 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.616 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.616 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.617 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.617 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.617 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.617 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.617 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.617 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.617 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.617 INFO:teuthology.orchestra.run.smithi172.stdout: "crt_file": { 2024-09-10T09:06:43.617 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "crt_file", 2024-09-10T09:06:43.617 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.617 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.617 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.618 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.618 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.618 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.618 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.618 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.618 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.618 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.618 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.618 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.618 INFO:teuthology.orchestra.run.smithi172.stdout: "debug": { 2024-09-10T09:06:43.618 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "debug", 2024-09-10T09:06:43.618 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.618 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.619 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.619 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.619 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.619 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.619 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.619 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Enable/disable debug options", 2024-09-10T09:06:43.619 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.619 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.619 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.619 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.619 INFO:teuthology.orchestra.run.smithi172.stdout: "jwt_token_ttl": { 2024-09-10T09:06:43.619 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "jwt_token_ttl", 2024-09-10T09:06:43.619 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.620 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.620 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.620 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "28800", 2024-09-10T09:06:43.620 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.620 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.620 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.620 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.620 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.620 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.620 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.620 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.620 INFO:teuthology.orchestra.run.smithi172.stdout: "key_file": { 2024-09-10T09:06:43.620 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "key_file", 2024-09-10T09:06:43.621 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.621 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.621 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.621 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.621 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.621 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.621 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.621 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.621 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.621 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.621 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.621 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.622 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.622 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.622 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.622 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.622 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.622 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.622 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.622 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.622 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.622 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.622 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.622 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.623 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.623 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.623 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.623 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.623 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.623 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.623 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.623 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.623 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.623 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.623 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.623 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.623 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.624 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.624 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.624 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.624 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.624 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.624 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.624 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.624 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.624 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.624 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.624 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.624 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.625 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.625 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.625 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.625 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.625 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.625 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.625 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.625 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.625 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.625 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.625 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.625 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.625 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.626 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.626 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.626 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.626 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.626 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.626 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.626 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.626 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.626 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.626 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.626 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.626 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.626 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.626 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.627 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.627 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.627 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.627 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.627 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.627 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.627 INFO:teuthology.orchestra.run.smithi172.stdout: "motd": { 2024-09-10T09:06:43.627 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "motd", 2024-09-10T09:06:43.627 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.627 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.627 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.627 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.628 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.628 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.628 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.628 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "The message of the day", 2024-09-10T09:06:43.628 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.628 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.628 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.628 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.628 INFO:teuthology.orchestra.run.smithi172.stdout: "redirect_resolve_ip_addr": { 2024-09-10T09:06:43.628 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T09:06:43.628 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.628 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.628 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.629 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.629 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.629 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.629 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.629 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.629 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.629 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.629 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.629 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.629 INFO:teuthology.orchestra.run.smithi172.stdout: "server_addr": { 2024-09-10T09:06:43.629 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_addr", 2024-09-10T09:06:43.629 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.629 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.629 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.630 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "::", 2024-09-10T09:06:43.630 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.630 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.630 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.630 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.630 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.630 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.630 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.630 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.630 INFO:teuthology.orchestra.run.smithi172.stdout: "server_port": { 2024-09-10T09:06:43.630 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_port", 2024-09-10T09:06:43.630 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.631 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.631 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.631 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "8080", 2024-09-10T09:06:43.631 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.631 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.631 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.631 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.631 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.631 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.631 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.631 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.631 INFO:teuthology.orchestra.run.smithi172.stdout: "ssl": { 2024-09-10T09:06:43.631 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ssl", 2024-09-10T09:06:43.632 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.632 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.632 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.632 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.632 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.632 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.632 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.632 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.632 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.632 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.632 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.632 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.632 INFO:teuthology.orchestra.run.smithi172.stdout: "ssl_server_port": { 2024-09-10T09:06:43.633 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ssl_server_port", 2024-09-10T09:06:43.633 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.633 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.633 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.633 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "8443", 2024-09-10T09:06:43.633 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.633 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.633 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.633 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.633 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.633 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.633 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.634 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.634 INFO:teuthology.orchestra.run.smithi172.stdout: "standby_behaviour": { 2024-09-10T09:06:43.634 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "standby_behaviour", 2024-09-10T09:06:43.634 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.634 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.634 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.634 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "redirect", 2024-09-10T09:06:43.634 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.634 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.634 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.634 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.634 INFO:teuthology.orchestra.run.smithi172.stdout: "redirect" 2024-09-10T09:06:43.634 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.635 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.635 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.635 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.635 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.635 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.635 INFO:teuthology.orchestra.run.smithi172.stdout: "standby_error_status_code": { 2024-09-10T09:06:43.635 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "standby_error_status_code", 2024-09-10T09:06:43.635 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.635 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.635 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.635 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "500", 2024-09-10T09:06:43.635 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "400", 2024-09-10T09:06:43.635 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "599", 2024-09-10T09:06:43.635 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.636 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.636 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.636 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.636 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.636 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.636 INFO:teuthology.orchestra.run.smithi172.stdout: "url_prefix": { 2024-09-10T09:06:43.636 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "url_prefix", 2024-09-10T09:06:43.636 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.636 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.636 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.636 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.636 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.636 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.637 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.637 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.637 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.637 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.637 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.637 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.637 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.637 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.637 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.637 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "devicehealth", 2024-09-10T09:06:43.637 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.637 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.637 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.638 INFO:teuthology.orchestra.run.smithi172.stdout: "enable_monitoring": { 2024-09-10T09:06:43.638 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "enable_monitoring", 2024-09-10T09:06:43.638 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.638 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.638 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.638 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.638 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.638 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.638 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.638 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "monitor device health metrics", 2024-09-10T09:06:43.638 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.638 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.638 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.639 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.639 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.639 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.639 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.639 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.639 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.639 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.639 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.639 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.639 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.639 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.639 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.639 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.640 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.640 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.640 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.640 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.640 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.640 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.640 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.640 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.640 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.640 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.640 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.640 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.640 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.641 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.641 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.641 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.641 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.641 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.641 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.641 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.641 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.641 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.641 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.641 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.641 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.641 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.642 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.642 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.642 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.642 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.642 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.642 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.642 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.642 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.642 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.642 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.642 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.642 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.642 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.643 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.643 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.643 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.643 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.643 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.643 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.643 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.643 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.643 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.643 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.643 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.643 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.643 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.644 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.644 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.644 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.644 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.644 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.644 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.644 INFO:teuthology.orchestra.run.smithi172.stdout: "mark_out_threshold": { 2024-09-10T09:06:43.644 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mark_out_threshold", 2024-09-10T09:06:43.644 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.644 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.644 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.644 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "2419200", 2024-09-10T09:06:43.645 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.645 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.645 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.645 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T09:06:43.645 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.645 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.645 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.645 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.645 INFO:teuthology.orchestra.run.smithi172.stdout: "pool_name": { 2024-09-10T09:06:43.645 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "pool_name", 2024-09-10T09:06:43.645 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.645 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.645 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.646 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "device_health_metrics", 2024-09-10T09:06:43.646 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.646 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.646 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.646 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T09:06:43.646 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.646 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.646 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.646 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.646 INFO:teuthology.orchestra.run.smithi172.stdout: "retention_period": { 2024-09-10T09:06:43.646 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "retention_period", 2024-09-10T09:06:43.646 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.647 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.647 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.647 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "15552000", 2024-09-10T09:06:43.647 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.647 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.647 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.647 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how long to retain device health metrics", 2024-09-10T09:06:43.647 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.647 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.647 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.647 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.647 INFO:teuthology.orchestra.run.smithi172.stdout: "scrape_frequency": { 2024-09-10T09:06:43.647 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "scrape_frequency", 2024-09-10T09:06:43.648 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.648 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.648 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.648 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "86400", 2024-09-10T09:06:43.648 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.648 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.648 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.648 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T09:06:43.648 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.648 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.648 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.648 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.649 INFO:teuthology.orchestra.run.smithi172.stdout: "self_heal": { 2024-09-10T09:06:43.649 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "self_heal", 2024-09-10T09:06:43.649 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.649 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.649 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.649 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.649 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.649 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.649 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.649 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T09:06:43.649 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.649 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.650 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.650 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.650 INFO:teuthology.orchestra.run.smithi172.stdout: "sleep_interval": { 2024-09-10T09:06:43.650 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "sleep_interval", 2024-09-10T09:06:43.650 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.650 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.650 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.650 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "600", 2024-09-10T09:06:43.650 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.650 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.650 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.650 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T09:06:43.650 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.651 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.651 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.651 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.651 INFO:teuthology.orchestra.run.smithi172.stdout: "warn_threshold": { 2024-09-10T09:06:43.651 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "warn_threshold", 2024-09-10T09:06:43.651 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.651 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.651 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.651 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "7257600", 2024-09-10T09:06:43.651 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.651 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.651 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.651 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T09:06:43.652 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.652 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.652 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.652 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.652 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.652 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.652 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.652 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "diskprediction_local", 2024-09-10T09:06:43.652 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.652 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.652 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.652 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.653 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.653 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.653 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.653 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.653 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.653 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.653 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.653 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.653 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.653 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.653 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.653 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.653 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.654 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.654 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.654 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.654 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.654 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.654 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.654 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.654 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.654 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.654 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.654 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.654 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.654 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.655 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.655 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.655 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.655 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.655 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.655 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.655 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.655 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.655 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.655 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.655 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.655 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.655 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.655 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.656 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.656 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.656 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.656 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.656 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.656 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.656 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.656 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.656 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.656 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.656 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.656 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.657 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.657 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.657 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.657 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.657 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.657 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.657 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.657 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.657 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.657 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.657 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.657 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.657 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.657 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.658 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.658 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.658 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.658 INFO:teuthology.orchestra.run.smithi172.stdout: "predict_interval": { 2024-09-10T09:06:43.658 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "predict_interval", 2024-09-10T09:06:43.658 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.658 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.658 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.658 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "86400", 2024-09-10T09:06:43.658 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.658 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.659 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.659 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.659 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.659 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.659 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.659 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.659 INFO:teuthology.orchestra.run.smithi172.stdout: "predictor_model": { 2024-09-10T09:06:43.659 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "predictor_model", 2024-09-10T09:06:43.659 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.659 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.659 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.659 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "prophetstor", 2024-09-10T09:06:43.659 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.660 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.660 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.660 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.660 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.660 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.660 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.660 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.660 INFO:teuthology.orchestra.run.smithi172.stdout: "sleep_interval": { 2024-09-10T09:06:43.660 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "sleep_interval", 2024-09-10T09:06:43.660 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.660 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.660 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.661 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "600", 2024-09-10T09:06:43.661 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.661 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.661 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.661 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.661 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.661 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.661 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.661 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.661 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.661 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.661 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.661 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "influx", 2024-09-10T09:06:43.662 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": false, 2024-09-10T09:06:43.662 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "influxdb python module not found", 2024-09-10T09:06:43.662 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.662 INFO:teuthology.orchestra.run.smithi172.stdout: "batch_size": { 2024-09-10T09:06:43.662 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "batch_size", 2024-09-10T09:06:43.662 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.662 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.662 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.662 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "5000", 2024-09-10T09:06:43.662 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.662 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.662 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.663 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T09:06:43.663 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.663 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.663 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.663 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.663 INFO:teuthology.orchestra.run.smithi172.stdout: "database": { 2024-09-10T09:06:43.663 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "database", 2024-09-10T09:06:43.663 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.663 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.663 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.663 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "ceph", 2024-09-10T09:06:43.663 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.663 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.664 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.664 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T09:06:43.664 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.664 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.664 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.664 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.664 INFO:teuthology.orchestra.run.smithi172.stdout: "hostname": { 2024-09-10T09:06:43.664 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "hostname", 2024-09-10T09:06:43.664 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.664 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.664 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.664 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.665 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.665 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.665 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.665 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "InfluxDB server hostname", 2024-09-10T09:06:43.665 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.665 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.665 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.665 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.665 INFO:teuthology.orchestra.run.smithi172.stdout: "interval": { 2024-09-10T09:06:43.665 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "interval", 2024-09-10T09:06:43.665 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.665 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.666 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.666 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "30", 2024-09-10T09:06:43.666 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "5", 2024-09-10T09:06:43.666 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.666 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.666 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T09:06:43.666 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.666 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.666 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.666 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.666 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.666 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.666 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.667 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.667 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.667 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.667 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.667 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.667 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.667 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.667 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.667 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.667 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.667 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.667 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.667 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.668 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.668 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.668 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.668 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.668 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.668 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.668 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.668 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.668 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.668 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.668 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.668 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.668 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.669 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.669 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.669 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.669 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.669 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.669 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.669 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.669 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.669 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.669 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.669 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.669 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.669 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.670 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.670 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.670 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.670 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.670 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.670 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.670 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.670 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.670 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.670 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.670 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.670 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.671 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.671 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.671 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.671 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.671 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.671 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.671 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.671 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.671 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.671 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.671 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.671 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.671 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.672 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.672 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.672 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.672 INFO:teuthology.orchestra.run.smithi172.stdout: "password": { 2024-09-10T09:06:43.672 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "password", 2024-09-10T09:06:43.672 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.672 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.672 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.672 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.672 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.672 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.672 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.672 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "password of InfluxDB server user", 2024-09-10T09:06:43.673 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.673 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.673 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.673 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.673 INFO:teuthology.orchestra.run.smithi172.stdout: "port": { 2024-09-10T09:06:43.673 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "port", 2024-09-10T09:06:43.673 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.673 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.673 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.673 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "8086", 2024-09-10T09:06:43.673 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.673 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.674 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.674 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "InfluxDB server port", 2024-09-10T09:06:43.674 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.674 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.674 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.674 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.674 INFO:teuthology.orchestra.run.smithi172.stdout: "ssl": { 2024-09-10T09:06:43.674 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ssl", 2024-09-10T09:06:43.674 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.674 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.674 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.674 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "false", 2024-09-10T09:06:43.674 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.675 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.675 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.675 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T09:06:43.675 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.675 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.675 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.675 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.675 INFO:teuthology.orchestra.run.smithi172.stdout: "threads": { 2024-09-10T09:06:43.675 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "threads", 2024-09-10T09:06:43.675 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.675 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.675 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.675 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "5", 2024-09-10T09:06:43.676 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "1", 2024-09-10T09:06:43.676 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "32", 2024-09-10T09:06:43.676 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.676 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T09:06:43.676 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.676 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.676 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.676 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.676 INFO:teuthology.orchestra.run.smithi172.stdout: "username": { 2024-09-10T09:06:43.676 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "username", 2024-09-10T09:06:43.676 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.676 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.676 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.677 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.677 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.677 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.677 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.677 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "username of InfluxDB server user", 2024-09-10T09:06:43.677 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.677 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.677 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.677 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.677 INFO:teuthology.orchestra.run.smithi172.stdout: "verify_ssl": { 2024-09-10T09:06:43.677 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "verify_ssl", 2024-09-10T09:06:43.677 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.678 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.678 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.678 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "true", 2024-09-10T09:06:43.678 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.678 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.678 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.678 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T09:06:43.678 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.678 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.678 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.678 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.678 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.678 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.679 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.679 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "insights", 2024-09-10T09:06:43.679 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.679 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.679 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.679 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.679 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.679 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.679 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.679 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.679 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.679 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.679 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.679 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.679 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.680 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.680 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.680 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.680 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.680 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.680 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.680 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.680 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.680 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.680 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.680 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.680 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.681 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.681 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.681 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.681 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.681 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.681 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.681 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.681 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.681 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.681 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.681 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.681 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.681 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.682 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.682 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.682 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.682 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.682 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.682 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.682 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.682 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.682 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.682 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.682 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.682 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.682 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.682 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.683 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.683 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.683 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.683 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.683 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.683 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.683 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.683 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.683 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.683 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.683 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.683 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.684 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.684 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.684 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.684 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.684 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.684 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.684 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.684 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.684 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.684 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.684 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.684 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.684 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "iostat", 2024-09-10T09:06:43.684 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.685 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.685 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.685 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.685 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.685 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.685 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.685 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.685 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.685 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.685 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.685 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.685 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.685 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.686 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.686 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.686 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.686 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.686 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.686 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.686 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.686 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.686 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.686 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.686 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.686 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.686 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.686 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.687 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.687 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.687 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.687 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.687 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.687 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.687 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.687 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.687 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.687 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.687 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.687 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.687 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.687 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.687 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.688 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.688 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.688 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.688 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.688 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.688 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.688 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.688 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.688 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.688 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.688 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.688 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.688 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.689 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.689 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.689 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.689 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.689 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.689 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.689 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.689 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.689 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.689 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.689 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.689 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.689 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.690 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.690 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.690 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.690 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.690 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.690 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.690 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.690 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "k8sevents", 2024-09-10T09:06:43.690 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.690 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.690 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.690 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_event_retention_days": { 2024-09-10T09:06:43.690 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "ceph_event_retention_days", 2024-09-10T09:06:43.690 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.690 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.691 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.691 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "7", 2024-09-10T09:06:43.691 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.691 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.691 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.691 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T09:06:43.691 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.691 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.691 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.691 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.691 INFO:teuthology.orchestra.run.smithi172.stdout: "config_check_secs": { 2024-09-10T09:06:43.691 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "config_check_secs", 2024-09-10T09:06:43.691 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.692 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.692 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.692 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:43.692 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "10", 2024-09-10T09:06:43.692 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.692 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.692 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T09:06:43.692 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.692 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.692 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.692 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.692 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.693 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.693 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.693 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.693 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.693 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.693 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.693 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.693 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.693 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.693 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.693 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.693 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.693 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.693 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.694 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.694 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.694 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.694 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.694 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.694 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.694 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.694 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.694 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.694 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.694 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.694 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.695 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.695 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.695 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.695 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.695 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.695 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.695 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.695 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.695 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.695 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.695 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.695 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.696 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.696 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.696 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.696 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.696 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.696 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.696 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.696 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.696 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.696 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.696 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.697 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.697 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.697 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.697 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.697 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.697 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.697 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.697 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.697 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.697 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.697 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.697 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.698 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.698 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.698 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.698 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.698 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.698 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.698 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.698 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.698 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.698 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.698 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.698 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "localpool", 2024-09-10T09:06:43.699 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.699 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.699 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.699 INFO:teuthology.orchestra.run.smithi172.stdout: "failure_domain": { 2024-09-10T09:06:43.699 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "failure_domain", 2024-09-10T09:06:43.699 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.699 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.699 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.699 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "host", 2024-09-10T09:06:43.699 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.699 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.699 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.700 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "failure domain for any created local pool", 2024-09-10T09:06:43.700 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T09:06:43.700 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.700 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.700 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.700 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.700 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.700 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.700 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.700 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.700 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.700 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.700 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.701 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.701 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.701 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.701 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.701 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.701 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.701 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.701 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.701 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.701 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.701 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.701 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.701 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.702 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.702 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.702 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.702 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.702 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.702 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.702 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.702 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.702 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.702 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.702 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.702 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.702 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.703 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.703 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.703 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.703 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.703 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.703 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.703 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.703 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.703 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.703 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.703 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.703 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.703 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.703 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.704 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.704 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.704 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.704 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.704 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.704 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.704 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.704 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.704 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.704 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.704 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.704 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.704 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.704 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.705 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.705 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.705 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.705 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.705 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.705 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.705 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.705 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.705 INFO:teuthology.orchestra.run.smithi172.stdout: "min_size": { 2024-09-10T09:06:43.705 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "min_size", 2024-09-10T09:06:43.705 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.705 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.705 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.706 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.706 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.706 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.706 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.706 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "default min_size for any created local pool", 2024-09-10T09:06:43.706 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T09:06:43.706 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.706 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.706 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.706 INFO:teuthology.orchestra.run.smithi172.stdout: "num_rep": { 2024-09-10T09:06:43.706 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "num_rep", 2024-09-10T09:06:43.706 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.706 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.706 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.707 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "3", 2024-09-10T09:06:43.707 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.707 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.707 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.707 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "default replica count for any created local pool", 2024-09-10T09:06:43.707 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.707 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.707 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.707 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.707 INFO:teuthology.orchestra.run.smithi172.stdout: "pg_num": { 2024-09-10T09:06:43.707 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "pg_num", 2024-09-10T09:06:43.707 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.707 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.707 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.707 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "128", 2024-09-10T09:06:43.708 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.708 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.708 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.708 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T09:06:43.708 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.708 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.708 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.708 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.708 INFO:teuthology.orchestra.run.smithi172.stdout: "prefix": { 2024-09-10T09:06:43.708 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "prefix", 2024-09-10T09:06:43.708 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.708 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.709 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.709 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.709 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.709 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.709 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.709 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "name prefix for any created local pool", 2024-09-10T09:06:43.709 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.709 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.709 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.709 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.709 INFO:teuthology.orchestra.run.smithi172.stdout: "subtree": { 2024-09-10T09:06:43.709 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "subtree", 2024-09-10T09:06:43.709 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.710 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.710 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.710 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "rack", 2024-09-10T09:06:43.710 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.710 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.710 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.710 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T09:06:43.710 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T09:06:43.710 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.710 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.710 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.710 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.710 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.711 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.711 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mds_autoscaler", 2024-09-10T09:06:43.711 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.711 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.711 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.711 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.711 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.711 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.711 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.711 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.711 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.711 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.711 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.712 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.712 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.712 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.712 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.712 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.712 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.712 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.712 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.712 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.712 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.712 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.712 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.713 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.713 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.713 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.713 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.713 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.713 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.713 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.713 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.713 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.713 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.713 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.713 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.714 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.714 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.714 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.714 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.714 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.714 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.714 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.714 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.714 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.714 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.714 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.714 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.714 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.715 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.715 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.715 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.715 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.715 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.715 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.715 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.715 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.715 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.715 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.715 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.715 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.715 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.716 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.716 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.716 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.716 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.716 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.716 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.716 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.716 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.716 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.716 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.716 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.716 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.717 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.717 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.717 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.717 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mirroring", 2024-09-10T09:06:43.717 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.717 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.717 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.717 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.717 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.717 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.717 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.717 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.717 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.718 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.718 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.718 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.718 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.718 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.718 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.718 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.718 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.718 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.718 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.718 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.718 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.718 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.719 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.719 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.719 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.719 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.719 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.719 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.719 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.719 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.719 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.719 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.719 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.719 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.719 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.719 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.720 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.720 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.720 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.720 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.720 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.720 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.720 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.720 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.720 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.720 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.720 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.720 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.721 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.721 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.721 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.721 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.721 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.721 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.721 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.721 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.721 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.721 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.721 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.721 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.722 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.722 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.722 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.722 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.722 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.722 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.722 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.722 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.722 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.722 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.722 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.722 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.722 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.723 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.723 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.723 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.723 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "nfs", 2024-09-10T09:06:43.723 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.723 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.723 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.723 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.723 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.723 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.723 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.723 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.723 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.724 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.724 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.724 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.724 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.724 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.724 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.724 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.724 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.724 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.724 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.724 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.724 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.725 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.725 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.725 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.725 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.725 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.725 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.725 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.725 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.725 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.725 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.725 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.725 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.725 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.725 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.726 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.726 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.726 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.726 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.726 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.726 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.726 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.726 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.726 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.726 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.726 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.726 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.726 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.726 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.727 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.727 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.727 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.727 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.727 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.727 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.727 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.727 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.727 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.727 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.727 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.727 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.727 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.727 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.727 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.728 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.728 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.728 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.728 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.728 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.728 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.728 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.728 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.728 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.728 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.728 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.728 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.728 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "orchestrator", 2024-09-10T09:06:43.729 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.729 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.729 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.729 INFO:teuthology.orchestra.run.smithi172.stdout: "fail_fs": { 2024-09-10T09:06:43.729 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "fail_fs", 2024-09-10T09:06:43.729 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.729 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.729 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.729 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.729 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.729 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.729 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.729 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T09:06:43.729 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.730 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.730 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.730 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.730 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.730 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.730 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.730 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.730 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.730 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.730 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.730 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.730 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.730 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.730 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.731 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.731 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.731 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.731 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.731 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.731 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.731 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.731 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.731 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.731 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.731 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.731 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.731 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.731 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.732 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.732 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.732 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.732 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.732 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.732 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.732 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.732 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.732 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.732 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.732 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.732 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.732 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.732 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.733 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.733 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.733 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.733 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.733 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.733 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.733 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.733 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.733 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.733 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.733 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.733 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.733 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.734 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.734 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.734 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.734 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.734 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.734 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.734 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.734 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.734 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.734 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.734 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.734 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.734 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.735 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.735 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.735 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.735 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.735 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.735 INFO:teuthology.orchestra.run.smithi172.stdout: "orchestrator": { 2024-09-10T09:06:43.735 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "orchestrator", 2024-09-10T09:06:43.735 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.735 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.735 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.735 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.735 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.735 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.736 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.736 INFO:teuthology.orchestra.run.smithi172.stdout: "cephadm", 2024-09-10T09:06:43.736 INFO:teuthology.orchestra.run.smithi172.stdout: "rook", 2024-09-10T09:06:43.736 INFO:teuthology.orchestra.run.smithi172.stdout: "test_orchestrator" 2024-09-10T09:06:43.736 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.736 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Orchestrator backend", 2024-09-10T09:06:43.736 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.736 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.736 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.736 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.736 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.736 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.736 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.737 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "osd_perf_query", 2024-09-10T09:06:43.737 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.737 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.737 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.737 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.737 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.737 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.737 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.737 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.737 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.737 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.737 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.737 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.738 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.738 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.738 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.738 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.738 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.738 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.738 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.738 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.738 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.738 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.738 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.738 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.738 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.739 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.739 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.739 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.739 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.739 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.739 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.739 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.739 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.739 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.739 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.739 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.739 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.739 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.739 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.739 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.740 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.740 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.740 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.740 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.740 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.740 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.740 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.740 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.740 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.740 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.740 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.740 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.740 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.740 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.740 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.741 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.741 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.741 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.741 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.741 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.741 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.741 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.741 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.741 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.741 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.741 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.741 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.741 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.742 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.742 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.742 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.742 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.742 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.742 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.742 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.742 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.742 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "osd_support", 2024-09-10T09:06:43.742 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.742 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.742 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.742 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.742 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.742 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.743 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.743 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.743 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.743 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.743 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.743 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.743 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.743 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.743 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.743 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.743 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.743 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.743 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.743 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.744 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.744 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.744 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.744 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.744 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.744 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.744 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.744 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.744 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.744 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.744 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.744 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.744 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.744 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.745 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.745 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.745 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.745 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.745 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.745 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.745 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.745 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.745 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.745 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.745 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.745 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.745 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.745 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.746 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.746 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.746 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.746 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.746 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.746 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.746 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.746 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.746 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.746 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.746 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.746 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.746 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.746 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.746 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.747 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.747 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.747 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.747 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.747 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.747 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.747 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.747 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.747 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.747 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.747 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.747 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.747 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.747 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "pg_autoscaler", 2024-09-10T09:06:43.748 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.748 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.748 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.748 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.748 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.748 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.748 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.748 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.748 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.748 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.748 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.748 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.748 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.749 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.749 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.749 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.749 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.749 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.749 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.749 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.749 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.749 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.749 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.749 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.749 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.749 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.750 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.750 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.750 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.750 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.750 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.750 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.750 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.750 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.750 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.750 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.750 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.750 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.750 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.750 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.751 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.751 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.751 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.751 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.751 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.751 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.751 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.751 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.751 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.751 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.751 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.751 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.751 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.751 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.752 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.752 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.752 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.752 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.752 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.752 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.752 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.752 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.752 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.752 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.752 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.752 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.752 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.753 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.753 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.753 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.753 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.753 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.753 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.753 INFO:teuthology.orchestra.run.smithi172.stdout: "sleep_interval": { 2024-09-10T09:06:43.753 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "sleep_interval", 2024-09-10T09:06:43.753 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.753 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.753 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.753 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "60", 2024-09-10T09:06:43.754 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.754 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.754 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.754 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.754 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.754 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.754 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.754 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.754 INFO:teuthology.orchestra.run.smithi172.stdout: "threshold": { 2024-09-10T09:06:43.754 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "threshold", 2024-09-10T09:06:43.754 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:43.754 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.754 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.754 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "3.0", 2024-09-10T09:06:43.755 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "1.0", 2024-09-10T09:06:43.755 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.755 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.755 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "scaling threshold", 2024-09-10T09:06:43.755 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T09:06:43.755 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.755 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.755 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.755 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.755 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.755 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.755 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "progress", 2024-09-10T09:06:43.755 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.755 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.755 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.756 INFO:teuthology.orchestra.run.smithi172.stdout: "allow_pg_recovery_event": { 2024-09-10T09:06:43.756 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "allow_pg_recovery_event", 2024-09-10T09:06:43.756 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.756 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.756 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.756 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.756 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.756 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.756 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.756 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T09:06:43.756 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.756 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.756 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "enabled": { 2024-09-10T09:06:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "enabled", 2024-09-10T09:06:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.757 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.758 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.759 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.759 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.759 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.759 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.759 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.759 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.759 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.759 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.759 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.759 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.759 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.759 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.759 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.759 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.760 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.760 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.760 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.760 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.760 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.760 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.760 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.760 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.760 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.760 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.760 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.760 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.760 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.760 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.761 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.761 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.761 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.761 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.761 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.761 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.761 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.761 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.761 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.761 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.761 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.761 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.761 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.761 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.762 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.762 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.762 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.762 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.762 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.762 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.762 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.762 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.762 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.762 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.762 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.762 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.762 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.762 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.762 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.763 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.763 INFO:teuthology.orchestra.run.smithi172.stdout: "max_completed_events": { 2024-09-10T09:06:43.763 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "max_completed_events", 2024-09-10T09:06:43.763 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.763 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.763 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.763 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "50", 2024-09-10T09:06:43.763 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.763 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.763 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.763 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "number of past completed events to remember", 2024-09-10T09:06:43.763 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.763 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.763 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.764 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.764 INFO:teuthology.orchestra.run.smithi172.stdout: "sleep_interval": { 2024-09-10T09:06:43.764 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "sleep_interval", 2024-09-10T09:06:43.764 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.764 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.764 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.764 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "5", 2024-09-10T09:06:43.764 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.764 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.764 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.764 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "how long the module is going to sleep", 2024-09-10T09:06:43.764 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.764 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.764 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.765 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.765 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.765 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.765 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.765 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "prometheus", 2024-09-10T09:06:43.765 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.765 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.765 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.765 INFO:teuthology.orchestra.run.smithi172.stdout: "cache": { 2024-09-10T09:06:43.765 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "cache", 2024-09-10T09:06:43.765 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.765 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.765 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.766 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.766 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.766 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.766 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.766 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.766 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.766 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.766 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.766 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.766 INFO:teuthology.orchestra.run.smithi172.stdout: "exclude_perf_counters": { 2024-09-10T09:06:43.766 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "exclude_perf_counters", 2024-09-10T09:06:43.766 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.766 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.767 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.767 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.767 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.767 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.767 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.767 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T09:06:43.767 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T09:06:43.767 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.767 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.767 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.767 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.767 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.767 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.767 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.768 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.768 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.768 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.768 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.768 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.768 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.768 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.768 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.768 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.768 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.768 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.768 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.768 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.769 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.769 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.769 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.769 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.769 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.769 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.769 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.769 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.769 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.769 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.769 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.769 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.769 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.769 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.770 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.770 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.770 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.770 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.770 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.770 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.770 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.770 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.770 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.770 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.770 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.770 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.770 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.770 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.771 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.771 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.771 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.771 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.771 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.771 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.771 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.771 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.771 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.771 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.771 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.771 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.771 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.771 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.771 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.772 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.772 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.772 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.772 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.772 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.772 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.772 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.772 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.772 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.772 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.772 INFO:teuthology.orchestra.run.smithi172.stdout: "rbd_stats_pools": { 2024-09-10T09:06:43.772 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rbd_stats_pools", 2024-09-10T09:06:43.772 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.772 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.773 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.773 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.773 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.773 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.773 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.773 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.773 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.773 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.773 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.773 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.773 INFO:teuthology.orchestra.run.smithi172.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T09:06:43.773 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T09:06:43.773 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.774 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.774 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.774 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "300", 2024-09-10T09:06:43.774 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.774 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.774 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.774 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.774 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.774 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.774 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.774 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.774 INFO:teuthology.orchestra.run.smithi172.stdout: "scrape_interval": { 2024-09-10T09:06:43.774 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "scrape_interval", 2024-09-10T09:06:43.774 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:43.774 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.775 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.775 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "15.0", 2024-09-10T09:06:43.775 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.775 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.775 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.775 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.775 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.775 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.775 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.775 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.775 INFO:teuthology.orchestra.run.smithi172.stdout: "server_addr": { 2024-09-10T09:06:43.775 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_addr", 2024-09-10T09:06:43.775 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.776 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.776 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.776 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "::", 2024-09-10T09:06:43.776 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.776 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.776 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.776 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T09:06:43.776 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.776 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.776 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.776 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.776 INFO:teuthology.orchestra.run.smithi172.stdout: "server_port": { 2024-09-10T09:06:43.776 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_port", 2024-09-10T09:06:43.776 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.777 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.777 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.777 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "9283", 2024-09-10T09:06:43.777 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.777 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.777 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.777 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T09:06:43.777 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.777 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.777 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.777 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.777 INFO:teuthology.orchestra.run.smithi172.stdout: "stale_cache_strategy": { 2024-09-10T09:06:43.778 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "stale_cache_strategy", 2024-09-10T09:06:43.778 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.778 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.778 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.778 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "log", 2024-09-10T09:06:43.778 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.778 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.778 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.778 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.778 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.778 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.778 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.778 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.778 INFO:teuthology.orchestra.run.smithi172.stdout: "standby_behaviour": { 2024-09-10T09:06:43.779 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "standby_behaviour", 2024-09-10T09:06:43.779 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.779 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.779 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.779 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "default", 2024-09-10T09:06:43.779 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.779 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.779 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.779 INFO:teuthology.orchestra.run.smithi172.stdout: "default", 2024-09-10T09:06:43.779 INFO:teuthology.orchestra.run.smithi172.stdout: "error" 2024-09-10T09:06:43.779 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.779 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.779 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.779 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.779 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.780 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.780 INFO:teuthology.orchestra.run.smithi172.stdout: "standby_error_status_code": { 2024-09-10T09:06:43.780 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "standby_error_status_code", 2024-09-10T09:06:43.780 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.780 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.780 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.780 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "500", 2024-09-10T09:06:43.780 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "400", 2024-09-10T09:06:43.780 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "599", 2024-09-10T09:06:43.780 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.780 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.780 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.780 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.781 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.781 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.781 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.781 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.781 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.781 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rbd_support", 2024-09-10T09:06:43.781 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.781 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.781 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.781 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.781 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.781 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.781 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.781 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.781 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.782 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.782 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.782 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.782 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.782 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.782 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.782 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.782 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.782 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.782 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.782 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.782 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.782 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.783 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.783 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.783 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.783 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.783 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.783 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.783 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.783 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.783 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.783 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.783 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.783 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.783 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.783 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.784 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.784 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.784 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.784 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.784 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.784 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.784 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.784 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.784 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.784 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.784 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.784 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.784 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.784 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.785 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.785 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.785 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.785 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.785 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.785 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.785 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.785 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.785 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.785 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.785 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.785 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.785 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.786 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.786 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.786 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.786 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.786 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.786 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.786 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.786 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.786 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.786 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.786 INFO:teuthology.orchestra.run.smithi172.stdout: "max_concurrent_snap_create": { 2024-09-10T09:06:43.786 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "max_concurrent_snap_create", 2024-09-10T09:06:43.786 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.786 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.786 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.787 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10", 2024-09-10T09:06:43.787 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.787 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.787 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.787 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.787 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.787 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.787 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.787 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.787 INFO:teuthology.orchestra.run.smithi172.stdout: "mirror_snapshot_schedule": { 2024-09-10T09:06:43.787 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T09:06:43.787 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.787 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.787 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.787 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.788 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.788 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.788 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.788 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.788 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.788 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.788 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.788 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.788 INFO:teuthology.orchestra.run.smithi172.stdout: "trash_purge_schedule": { 2024-09-10T09:06:43.788 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "trash_purge_schedule", 2024-09-10T09:06:43.788 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.788 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.789 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.789 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.789 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.789 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.789 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.789 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.789 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.789 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.789 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.789 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.789 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.789 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.789 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.790 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "restful", 2024-09-10T09:06:43.790 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.790 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.790 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.790 INFO:teuthology.orchestra.run.smithi172.stdout: "enable_auth": { 2024-09-10T09:06:43.790 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "enable_auth", 2024-09-10T09:06:43.790 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.790 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.790 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.790 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.790 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.790 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.790 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.790 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.791 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.791 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.791 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.791 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.791 INFO:teuthology.orchestra.run.smithi172.stdout: "key_file": { 2024-09-10T09:06:43.791 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "key_file", 2024-09-10T09:06:43.791 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.791 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.791 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.791 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.791 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.791 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.791 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.791 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.792 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.792 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.792 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.792 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.792 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.792 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.792 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.792 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.792 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.792 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.792 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.792 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.792 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.792 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.793 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.793 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.793 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.793 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.793 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.793 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.793 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.793 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.793 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.793 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.793 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.793 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.794 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.794 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.794 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.794 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.794 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.794 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.794 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.794 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.794 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.794 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.794 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.794 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.794 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.794 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.795 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.795 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.795 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.795 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.795 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.795 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.795 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.795 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.795 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.795 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.795 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.795 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.795 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.796 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.796 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.796 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.796 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.796 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.796 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.796 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.796 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.796 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.796 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.796 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.796 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.796 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.797 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.797 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.797 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.797 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.797 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.797 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.797 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.797 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.797 INFO:teuthology.orchestra.run.smithi172.stdout: "server_addr": { 2024-09-10T09:06:43.797 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_addr", 2024-09-10T09:06:43.797 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.797 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.797 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.798 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.798 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.798 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.798 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.798 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.798 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.798 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.798 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.798 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.798 INFO:teuthology.orchestra.run.smithi172.stdout: "server_port": { 2024-09-10T09:06:43.798 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "server_port", 2024-09-10T09:06:43.798 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.798 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.799 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.799 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.799 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.799 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.799 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.799 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.799 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.799 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.799 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.799 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.799 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.799 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.799 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rgw", 2024-09-10T09:06:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.800 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.801 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.802 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.802 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.802 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.802 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.802 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.802 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.802 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.802 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.802 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.802 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.802 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.803 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.803 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.803 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.803 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.803 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.803 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.803 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.803 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.803 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.803 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.803 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.803 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.803 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.803 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.804 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.804 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.804 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.804 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.804 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.804 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.804 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.804 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.804 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.804 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.804 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.804 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.804 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.804 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.805 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.805 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.805 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.805 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.805 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.805 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.805 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.805 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.805 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.805 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.805 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.805 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.806 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rook", 2024-09-10T09:06:43.806 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.806 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.806 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.806 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.806 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.806 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.806 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.806 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.806 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.806 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.806 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.806 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.806 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.807 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.807 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.807 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.807 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.807 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.807 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.807 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.807 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.807 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.807 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.807 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.807 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.808 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.808 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.808 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.808 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.808 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.808 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.808 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.808 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.808 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.808 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.808 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.808 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.808 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.809 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.809 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.809 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.809 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.809 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.809 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.809 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.809 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.809 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.809 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.809 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.809 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.809 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.810 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.810 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.810 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.810 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.810 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.810 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.810 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.810 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.810 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.810 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.810 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.810 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.810 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.811 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.811 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.811 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.811 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.811 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.811 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.811 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.811 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.811 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.811 INFO:teuthology.orchestra.run.smithi172.stdout: "storage_class": { 2024-09-10T09:06:43.811 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "storage_class", 2024-09-10T09:06:43.811 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.811 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.811 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.812 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "local", 2024-09-10T09:06:43.812 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.812 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.812 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.812 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T09:06:43.812 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.812 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.812 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.812 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.812 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.812 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.812 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.812 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "selftest", 2024-09-10T09:06:43.812 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.813 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.813 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.813 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.813 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.813 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.813 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.813 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.813 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.813 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.813 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.813 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.813 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.813 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.813 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.814 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.814 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.814 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.814 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.814 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.814 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.814 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.814 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.814 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.814 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.814 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.814 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.814 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.814 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.814 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.815 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.815 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.815 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.815 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.815 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.815 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.815 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.815 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.815 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.815 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.815 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.815 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.815 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.815 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.816 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.816 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.816 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.816 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.816 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.816 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.816 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.816 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.816 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.816 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.816 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.816 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.816 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.816 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.817 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.817 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.817 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.817 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.817 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.817 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.817 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.817 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.817 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.817 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.817 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.817 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.818 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.818 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.818 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.818 INFO:teuthology.orchestra.run.smithi172.stdout: "roption1": { 2024-09-10T09:06:43.818 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "roption1", 2024-09-10T09:06:43.818 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.818 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.818 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.818 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.818 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.818 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.818 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.819 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.819 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.819 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.819 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.819 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.819 INFO:teuthology.orchestra.run.smithi172.stdout: "roption2": { 2024-09-10T09:06:43.819 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "roption2", 2024-09-10T09:06:43.819 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.819 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.819 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.819 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "xyz", 2024-09-10T09:06:43.819 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.819 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.820 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.820 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.820 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.820 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.820 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.820 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.820 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption1": { 2024-09-10T09:06:43.820 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption1", 2024-09-10T09:06:43.820 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.820 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.820 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.820 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.820 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.821 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.821 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.821 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.821 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.821 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.821 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.821 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.821 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption2": { 2024-09-10T09:06:43.821 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption2", 2024-09-10T09:06:43.821 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.821 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.821 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.822 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.822 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.822 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.822 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.822 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.822 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.822 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.822 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.822 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.822 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption3": { 2024-09-10T09:06:43.822 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption3", 2024-09-10T09:06:43.822 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "float", 2024-09-10T09:06:43.823 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.823 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.823 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.823 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.823 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.823 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.823 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.823 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.823 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.823 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.823 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.823 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption4": { 2024-09-10T09:06:43.824 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption4", 2024-09-10T09:06:43.824 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.824 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.824 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.824 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.824 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.824 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.824 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.824 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.824 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.824 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.824 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.825 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.825 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption5": { 2024-09-10T09:06:43.825 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption5", 2024-09-10T09:06:43.825 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.825 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.825 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.825 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.825 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.825 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.825 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.825 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.825 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.826 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.826 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.826 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.826 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption6": { 2024-09-10T09:06:43.826 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption6", 2024-09-10T09:06:43.826 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.826 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.826 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.826 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.826 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.826 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.826 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.827 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.827 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.827 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.827 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.827 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.827 INFO:teuthology.orchestra.run.smithi172.stdout: "rwoption7": { 2024-09-10T09:06:43.827 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rwoption7", 2024-09-10T09:06:43.827 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.827 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.827 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.827 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.827 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "1", 2024-09-10T09:06:43.827 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "42", 2024-09-10T09:06:43.828 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.828 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.828 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.828 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.828 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.828 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.828 INFO:teuthology.orchestra.run.smithi172.stdout: "testkey": { 2024-09-10T09:06:43.828 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "testkey", 2024-09-10T09:06:43.828 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.828 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.828 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.828 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.828 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.828 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.828 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.829 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.829 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.829 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.829 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.829 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.829 INFO:teuthology.orchestra.run.smithi172.stdout: "testlkey": { 2024-09-10T09:06:43.829 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "testlkey", 2024-09-10T09:06:43.829 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.829 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.829 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.829 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.829 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.830 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.830 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.830 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.830 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.830 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.830 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.830 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.830 INFO:teuthology.orchestra.run.smithi172.stdout: "testnewline": { 2024-09-10T09:06:43.830 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "testnewline", 2024-09-10T09:06:43.830 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.830 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.830 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.830 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.831 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.831 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.831 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.831 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.831 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.831 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.831 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.831 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.831 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.831 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.831 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.831 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "snap_schedule", 2024-09-10T09:06:43.831 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.831 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.831 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.832 INFO:teuthology.orchestra.run.smithi172.stdout: "allow_m_granularity": { 2024-09-10T09:06:43.832 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "allow_m_granularity", 2024-09-10T09:06:43.832 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.832 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.832 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.832 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.832 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.832 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.832 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.832 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T09:06:43.832 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.832 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.832 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.832 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.833 INFO:teuthology.orchestra.run.smithi172.stdout: "dump_on_update": { 2024-09-10T09:06:43.833 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "dump_on_update", 2024-09-10T09:06:43.833 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.833 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.833 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.833 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.833 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.833 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.833 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.833 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "dump database to debug log on update", 2024-09-10T09:06:43.833 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.833 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.833 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.833 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.834 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.834 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.834 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.834 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.834 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.834 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.834 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.834 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.834 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.834 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.834 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.834 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.834 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.834 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.835 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.835 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.835 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.835 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.835 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.835 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.835 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.835 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.835 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.835 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.835 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.835 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.835 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.835 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.835 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.836 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.836 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.836 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.836 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.836 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.836 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.836 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.836 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.836 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.836 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.836 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.836 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.836 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.837 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.838 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.838 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.838 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.838 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.838 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.838 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.838 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.838 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.838 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.838 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.838 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.838 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.838 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.838 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.839 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.839 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.839 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.839 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "stats", 2024-09-10T09:06:43.839 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.839 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.839 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.839 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.839 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.839 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.839 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.839 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.840 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.840 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.840 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.840 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.840 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.840 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.840 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.840 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.840 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.840 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.840 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.840 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.840 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.840 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.841 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.841 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.841 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.841 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.841 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.841 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.841 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.841 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.841 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.841 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.841 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.841 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.841 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.841 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.841 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.842 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.842 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.842 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.842 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.842 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.842 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.842 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.842 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.842 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.842 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.842 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.842 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.842 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.842 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.843 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.843 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.843 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.843 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.843 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.843 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.843 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.843 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.843 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.843 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.843 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.843 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.844 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.844 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.844 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.844 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.844 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.844 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.844 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.844 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.844 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.844 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.844 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.844 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.844 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.845 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "status", 2024-09-10T09:06:43.845 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.845 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.845 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.845 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.845 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.845 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.845 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.845 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.845 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.845 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.845 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.845 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.846 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.846 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.846 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.846 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.846 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.846 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.846 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.846 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.846 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.846 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.846 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.846 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.846 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.847 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.847 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.847 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.847 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.847 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.847 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.847 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.847 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.847 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.847 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.847 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.847 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.847 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.848 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.848 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.848 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.848 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.848 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.848 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.848 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.848 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.848 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.848 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.848 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.848 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.848 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.849 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.849 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.849 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.849 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.849 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.849 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.849 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.849 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.849 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.849 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.849 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.849 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.849 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.850 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.850 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.850 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.850 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.850 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.850 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.850 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.850 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.850 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.850 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.850 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.850 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.850 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "telegraf", 2024-09-10T09:06:43.851 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.851 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.851 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.851 INFO:teuthology.orchestra.run.smithi172.stdout: "address": { 2024-09-10T09:06:43.851 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "address", 2024-09-10T09:06:43.851 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.851 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.851 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.851 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T09:06:43.851 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.851 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.851 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.851 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.851 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.852 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.852 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.852 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.852 INFO:teuthology.orchestra.run.smithi172.stdout: "interval": { 2024-09-10T09:06:43.852 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "interval", 2024-09-10T09:06:43.852 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.852 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.852 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.852 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "15", 2024-09-10T09:06:43.852 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.852 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.852 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.853 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.853 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.853 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.853 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.853 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.853 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.853 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.853 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.853 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.853 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.853 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.853 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.853 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.854 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.854 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.854 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.854 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.854 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.854 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.854 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.854 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.854 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.854 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.854 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.854 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.854 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.854 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.855 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.855 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.855 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.855 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.855 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.855 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.855 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.855 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.855 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.855 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.855 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.855 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.855 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.856 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.856 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.856 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.856 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.856 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.856 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.856 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.856 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.856 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.856 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.856 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.856 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.856 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.857 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.857 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.857 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.857 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.857 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.857 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.857 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.857 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.857 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.857 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.857 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.857 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.857 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.858 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.858 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.858 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.858 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.858 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.858 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.858 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.858 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.858 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.858 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.858 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.858 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.858 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "telemetry", 2024-09-10T09:06:43.859 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.859 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.859 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.859 INFO:teuthology.orchestra.run.smithi172.stdout: "channel_basic": { 2024-09-10T09:06:43.859 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "channel_basic", 2024-09-10T09:06:43.859 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.859 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.859 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.859 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.859 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.859 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.859 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.859 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T09:06:43.860 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.860 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.860 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.860 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.860 INFO:teuthology.orchestra.run.smithi172.stdout: "channel_crash": { 2024-09-10T09:06:43.860 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "channel_crash", 2024-09-10T09:06:43.860 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.860 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.860 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.860 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.860 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.860 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.860 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.861 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T09:06:43.861 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.861 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.861 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.861 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.861 INFO:teuthology.orchestra.run.smithi172.stdout: "channel_device": { 2024-09-10T09:06:43.861 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "channel_device", 2024-09-10T09:06:43.861 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.861 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.861 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.861 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.861 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.861 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.862 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.862 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T09:06:43.862 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.862 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.862 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.862 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.862 INFO:teuthology.orchestra.run.smithi172.stdout: "channel_ident": { 2024-09-10T09:06:43.862 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "channel_ident", 2024-09-10T09:06:43.862 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.862 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.862 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.862 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.862 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.862 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.863 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.863 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T09:06:43.863 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.863 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.863 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.863 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.863 INFO:teuthology.orchestra.run.smithi172.stdout: "channel_perf": { 2024-09-10T09:06:43.863 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "channel_perf", 2024-09-10T09:06:43.863 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.863 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.863 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.863 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.863 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.864 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.864 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.864 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T09:06:43.864 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.864 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.864 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.864 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.864 INFO:teuthology.orchestra.run.smithi172.stdout: "contact": { 2024-09-10T09:06:43.864 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "contact", 2024-09-10T09:06:43.864 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.864 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.864 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.865 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.865 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.865 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.865 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.865 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.865 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.865 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.865 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.865 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.865 INFO:teuthology.orchestra.run.smithi172.stdout: "description": { 2024-09-10T09:06:43.865 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "description", 2024-09-10T09:06:43.865 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.865 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.866 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.866 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.866 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.866 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.866 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.866 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.866 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.866 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.866 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.866 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.866 INFO:teuthology.orchestra.run.smithi172.stdout: "device_url": { 2024-09-10T09:06:43.866 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "device_url", 2024-09-10T09:06:43.866 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.866 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.867 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.867 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T09:06:43.867 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.867 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.867 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.867 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.867 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.867 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.867 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.867 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.867 INFO:teuthology.orchestra.run.smithi172.stdout: "enabled": { 2024-09-10T09:06:43.867 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "enabled", 2024-09-10T09:06:43.867 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.867 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.867 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.868 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.868 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.868 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.868 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.868 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.868 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.868 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.868 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.868 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.868 INFO:teuthology.orchestra.run.smithi172.stdout: "interval": { 2024-09-10T09:06:43.868 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "interval", 2024-09-10T09:06:43.868 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.868 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.868 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.869 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "24", 2024-09-10T09:06:43.869 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "8", 2024-09-10T09:06:43.869 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.869 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.869 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.869 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.869 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.869 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.869 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.869 INFO:teuthology.orchestra.run.smithi172.stdout: "last_opt_revision": { 2024-09-10T09:06:43.869 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "last_opt_revision", 2024-09-10T09:06:43.869 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.869 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.869 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.870 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "1", 2024-09-10T09:06:43.870 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.870 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.870 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.870 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.870 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.870 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.870 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.870 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.870 INFO:teuthology.orchestra.run.smithi172.stdout: "leaderboard": { 2024-09-10T09:06:43.870 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "leaderboard", 2024-09-10T09:06:43.870 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.870 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.871 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.871 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.871 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.871 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.871 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.871 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.871 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.871 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.871 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.871 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.871 INFO:teuthology.orchestra.run.smithi172.stdout: "leaderboard_description": { 2024-09-10T09:06:43.871 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "leaderboard_description", 2024-09-10T09:06:43.871 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.871 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.871 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.872 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.872 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.872 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.872 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.872 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.872 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.872 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.872 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.872 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.872 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.872 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.872 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.872 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.873 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.873 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.873 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.873 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.873 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.873 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.873 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.873 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.873 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.873 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.873 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.873 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.873 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.873 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.874 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.874 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.874 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.874 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.874 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.874 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.874 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.874 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.874 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.874 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.874 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.874 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.874 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.875 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.875 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.875 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.875 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.875 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.875 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.875 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.875 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.875 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.875 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.875 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.875 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.875 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.876 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.876 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.876 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.876 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.876 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.876 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.876 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.876 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.876 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.876 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.876 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.876 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.877 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.877 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.877 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.877 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.877 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.877 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.877 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.877 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.877 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.877 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.877 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.877 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.878 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.878 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.878 INFO:teuthology.orchestra.run.smithi172.stdout: "organization": { 2024-09-10T09:06:43.878 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "organization", 2024-09-10T09:06:43.878 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.878 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.878 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.878 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.878 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.878 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.878 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.878 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.878 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.879 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.879 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.879 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.879 INFO:teuthology.orchestra.run.smithi172.stdout: "proxy": { 2024-09-10T09:06:43.879 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "proxy", 2024-09-10T09:06:43.879 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.879 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.879 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.879 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.879 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.879 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.879 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.879 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.880 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.880 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.880 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.880 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.880 INFO:teuthology.orchestra.run.smithi172.stdout: "url": { 2024-09-10T09:06:43.880 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "url", 2024-09-10T09:06:43.880 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.880 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.880 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.880 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T09:06:43.880 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.880 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.880 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.881 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.881 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.881 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.881 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.881 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.881 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.881 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.881 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.881 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "test_orchestrator", 2024-09-10T09:06:43.881 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.881 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.881 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.881 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.882 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.882 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.882 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.882 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.882 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.882 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.882 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.882 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.882 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.882 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.882 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.882 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.882 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.883 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.883 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.883 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.883 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.883 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.883 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.883 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.883 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.883 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.883 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.883 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.883 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.883 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.884 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.884 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.884 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.884 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.884 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.884 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.884 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.884 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.884 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.884 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.884 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.884 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.884 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.885 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.885 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.885 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.885 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.885 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.885 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.885 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.885 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.885 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.885 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.885 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.885 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.885 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.886 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.886 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.886 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.886 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.886 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.886 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.886 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.886 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.886 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.886 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.886 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.886 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.886 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.886 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.887 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.887 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.887 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.887 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.887 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.887 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.887 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "volumes", 2024-09-10T09:06:43.887 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.887 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.887 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.887 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.887 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.887 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.887 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.888 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.888 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.888 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.888 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.888 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.888 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.888 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.888 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.888 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.888 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.888 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.888 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.888 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.888 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.889 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.889 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.889 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.889 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.889 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.889 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.889 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.889 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.889 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.889 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.889 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.889 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.889 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.889 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.890 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.890 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.890 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.890 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.890 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.890 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.890 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.890 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.890 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.890 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.890 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.890 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.890 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.891 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.891 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.891 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.891 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.891 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.891 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.891 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.891 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.891 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.891 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.891 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.891 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.891 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.892 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.892 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.892 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.892 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.892 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.892 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.892 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.892 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.892 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.892 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.892 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.892 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.893 INFO:teuthology.orchestra.run.smithi172.stdout: "max_concurrent_clones": { 2024-09-10T09:06:43.893 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "max_concurrent_clones", 2024-09-10T09:06:43.893 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.893 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.893 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.893 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "4", 2024-09-10T09:06:43.893 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.893 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.893 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.893 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T09:06:43.893 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.893 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.893 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.894 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.894 INFO:teuthology.orchestra.run.smithi172.stdout: "periodic_async_work": { 2024-09-10T09:06:43.894 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "periodic_async_work", 2024-09-10T09:06:43.894 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.894 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.894 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.894 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.894 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.894 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.894 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.894 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Periodically check for async work", 2024-09-10T09:06:43.894 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.894 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.895 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.895 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.895 INFO:teuthology.orchestra.run.smithi172.stdout: "snapshot_clone_delay": { 2024-09-10T09:06:43.895 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "snapshot_clone_delay", 2024-09-10T09:06:43.895 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.895 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.895 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.895 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "0", 2024-09-10T09:06:43.895 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.895 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.895 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.895 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T09:06:43.895 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.896 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.896 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.896 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.896 INFO:teuthology.orchestra.run.smithi172.stdout: "snapshot_clone_no_wait": { 2024-09-10T09:06:43.896 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T09:06:43.896 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.896 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.896 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.896 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "True", 2024-09-10T09:06:43.896 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.896 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.896 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.896 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T09:06:43.897 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.897 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.897 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.897 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.897 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.897 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.897 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.897 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "zabbix", 2024-09-10T09:06:43.897 INFO:teuthology.orchestra.run.smithi172.stdout: "can_run": true, 2024-09-10T09:06:43.897 INFO:teuthology.orchestra.run.smithi172.stdout: "error_string": "", 2024-09-10T09:06:43.897 INFO:teuthology.orchestra.run.smithi172.stdout: "module_options": { 2024-09-10T09:06:43.897 INFO:teuthology.orchestra.run.smithi172.stdout: "discovery_interval": { 2024-09-10T09:06:43.898 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "discovery_interval", 2024-09-10T09:06:43.898 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "uint", 2024-09-10T09:06:43.898 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.898 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.898 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "100", 2024-09-10T09:06:43.898 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.898 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.898 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.898 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.898 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.898 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.898 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.898 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.899 INFO:teuthology.orchestra.run.smithi172.stdout: "identifier": { 2024-09-10T09:06:43.899 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "identifier", 2024-09-10T09:06:43.899 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.899 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.899 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.899 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.899 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.899 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.899 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.899 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.899 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.899 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.899 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.900 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.900 INFO:teuthology.orchestra.run.smithi172.stdout: "interval": { 2024-09-10T09:06:43.900 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "interval", 2024-09-10T09:06:43.900 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "secs", 2024-09-10T09:06:43.900 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.900 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.900 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "60", 2024-09-10T09:06:43.900 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.900 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.900 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.900 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.900 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.901 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.901 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.901 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.901 INFO:teuthology.orchestra.run.smithi172.stdout: "log_level": { 2024-09-10T09:06:43.901 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_level", 2024-09-10T09:06:43.901 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.901 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.901 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.901 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.901 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.901 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.901 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.901 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.902 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.902 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.902 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.902 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.902 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.902 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.902 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.902 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.902 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.902 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.902 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.902 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster": { 2024-09-10T09:06:43.902 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster", 2024-09-10T09:06:43.903 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.903 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.903 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.903 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.903 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.903 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.903 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.903 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.903 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.903 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.903 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.903 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.904 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_cluster_level": { 2024-09-10T09:06:43.904 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_cluster_level", 2024-09-10T09:06:43.904 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.904 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.904 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.904 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "info", 2024-09-10T09:06:43.904 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.904 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.904 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [ 2024-09-10T09:06:43.904 INFO:teuthology.orchestra.run.smithi172.stdout: "", 2024-09-10T09:06:43.904 INFO:teuthology.orchestra.run.smithi172.stdout: "critical", 2024-09-10T09:06:43.904 INFO:teuthology.orchestra.run.smithi172.stdout: "debug", 2024-09-10T09:06:43.904 INFO:teuthology.orchestra.run.smithi172.stdout: "error", 2024-09-10T09:06:43.905 INFO:teuthology.orchestra.run.smithi172.stdout: "info", 2024-09-10T09:06:43.905 INFO:teuthology.orchestra.run.smithi172.stdout: "warning" 2024-09-10T09:06:43.905 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.905 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.905 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.905 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.905 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.905 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.905 INFO:teuthology.orchestra.run.smithi172.stdout: "log_to_file": { 2024-09-10T09:06:43.905 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "log_to_file", 2024-09-10T09:06:43.905 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "bool", 2024-09-10T09:06:43.905 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.905 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 1, 2024-09-10T09:06:43.905 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "False", 2024-09-10T09:06:43.906 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.906 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.906 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.906 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.906 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.906 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.906 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.906 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.906 INFO:teuthology.orchestra.run.smithi172.stdout: "zabbix_host": { 2024-09-10T09:06:43.906 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "zabbix_host", 2024-09-10T09:06:43.906 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.906 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.906 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.906 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "", 2024-09-10T09:06:43.906 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.907 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.907 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.907 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.907 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.907 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.907 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.907 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.907 INFO:teuthology.orchestra.run.smithi172.stdout: "zabbix_port": { 2024-09-10T09:06:43.907 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "zabbix_port", 2024-09-10T09:06:43.907 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "int", 2024-09-10T09:06:43.907 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.907 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.907 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "10051", 2024-09-10T09:06:43.908 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.908 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.908 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.908 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.908 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.908 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.908 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.908 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.908 INFO:teuthology.orchestra.run.smithi172.stdout: "zabbix_sender": { 2024-09-10T09:06:43.908 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "zabbix_sender", 2024-09-10T09:06:43.908 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "str", 2024-09-10T09:06:43.908 INFO:teuthology.orchestra.run.smithi172.stdout: "level": "advanced", 2024-09-10T09:06:43.908 INFO:teuthology.orchestra.run.smithi172.stdout: "flags": 0, 2024-09-10T09:06:43.908 INFO:teuthology.orchestra.run.smithi172.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T09:06:43.909 INFO:teuthology.orchestra.run.smithi172.stdout: "min": "", 2024-09-10T09:06:43.909 INFO:teuthology.orchestra.run.smithi172.stdout: "max": "", 2024-09-10T09:06:43.909 INFO:teuthology.orchestra.run.smithi172.stdout: "enum_allowed": [], 2024-09-10T09:06:43.909 INFO:teuthology.orchestra.run.smithi172.stdout: "desc": "", 2024-09-10T09:06:43.909 INFO:teuthology.orchestra.run.smithi172.stdout: "long_desc": "", 2024-09-10T09:06:43.909 INFO:teuthology.orchestra.run.smithi172.stdout: "tags": [], 2024-09-10T09:06:43.909 INFO:teuthology.orchestra.run.smithi172.stdout: "see_also": [] 2024-09-10T09:06:43.909 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.909 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.909 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.909 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.909 INFO:teuthology.orchestra.run.smithi172.stdout: "services": { 2024-09-10T09:06:43.909 INFO:teuthology.orchestra.run.smithi172.stdout: "dashboard": "https://172.21.15.172:8443/" 2024-09-10T09:06:43.910 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.910 INFO:teuthology.orchestra.run.smithi172.stdout: "always_on_modules": { 2024-09-10T09:06:43.910 INFO:teuthology.orchestra.run.smithi172.stdout: "octopus": [ 2024-09-10T09:06:43.910 INFO:teuthology.orchestra.run.smithi172.stdout: "balancer", 2024-09-10T09:06:43.910 INFO:teuthology.orchestra.run.smithi172.stdout: "crash", 2024-09-10T09:06:43.910 INFO:teuthology.orchestra.run.smithi172.stdout: "devicehealth", 2024-09-10T09:06:43.910 INFO:teuthology.orchestra.run.smithi172.stdout: "orchestrator", 2024-09-10T09:06:43.910 INFO:teuthology.orchestra.run.smithi172.stdout: "pg_autoscaler", 2024-09-10T09:06:43.910 INFO:teuthology.orchestra.run.smithi172.stdout: "progress", 2024-09-10T09:06:43.910 INFO:teuthology.orchestra.run.smithi172.stdout: "rbd_support", 2024-09-10T09:06:43.910 INFO:teuthology.orchestra.run.smithi172.stdout: "status", 2024-09-10T09:06:43.910 INFO:teuthology.orchestra.run.smithi172.stdout: "telemetry", 2024-09-10T09:06:43.911 INFO:teuthology.orchestra.run.smithi172.stdout: "volumes" 2024-09-10T09:06:43.911 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.911 INFO:teuthology.orchestra.run.smithi172.stdout: "pacific": [ 2024-09-10T09:06:43.911 INFO:teuthology.orchestra.run.smithi172.stdout: "balancer", 2024-09-10T09:06:43.911 INFO:teuthology.orchestra.run.smithi172.stdout: "crash", 2024-09-10T09:06:43.911 INFO:teuthology.orchestra.run.smithi172.stdout: "devicehealth", 2024-09-10T09:06:43.911 INFO:teuthology.orchestra.run.smithi172.stdout: "orchestrator", 2024-09-10T09:06:43.911 INFO:teuthology.orchestra.run.smithi172.stdout: "pg_autoscaler", 2024-09-10T09:06:43.911 INFO:teuthology.orchestra.run.smithi172.stdout: "progress", 2024-09-10T09:06:43.911 INFO:teuthology.orchestra.run.smithi172.stdout: "rbd_support", 2024-09-10T09:06:43.911 INFO:teuthology.orchestra.run.smithi172.stdout: "status", 2024-09-10T09:06:43.911 INFO:teuthology.orchestra.run.smithi172.stdout: "telemetry", 2024-09-10T09:06:43.911 INFO:teuthology.orchestra.run.smithi172.stdout: "volumes" 2024-09-10T09:06:43.911 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.912 INFO:teuthology.orchestra.run.smithi172.stdout: "quincy": [ 2024-09-10T09:06:43.912 INFO:teuthology.orchestra.run.smithi172.stdout: "balancer", 2024-09-10T09:06:43.912 INFO:teuthology.orchestra.run.smithi172.stdout: "crash", 2024-09-10T09:06:43.912 INFO:teuthology.orchestra.run.smithi172.stdout: "devicehealth", 2024-09-10T09:06:43.912 INFO:teuthology.orchestra.run.smithi172.stdout: "orchestrator", 2024-09-10T09:06:43.912 INFO:teuthology.orchestra.run.smithi172.stdout: "pg_autoscaler", 2024-09-10T09:06:43.912 INFO:teuthology.orchestra.run.smithi172.stdout: "progress", 2024-09-10T09:06:43.912 INFO:teuthology.orchestra.run.smithi172.stdout: "rbd_support", 2024-09-10T09:06:43.912 INFO:teuthology.orchestra.run.smithi172.stdout: "status", 2024-09-10T09:06:43.912 INFO:teuthology.orchestra.run.smithi172.stdout: "telemetry", 2024-09-10T09:06:43.912 INFO:teuthology.orchestra.run.smithi172.stdout: "volumes" 2024-09-10T09:06:43.912 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.912 INFO:teuthology.orchestra.run.smithi172.stdout: "reef": [ 2024-09-10T09:06:43.912 INFO:teuthology.orchestra.run.smithi172.stdout: "balancer", 2024-09-10T09:06:43.913 INFO:teuthology.orchestra.run.smithi172.stdout: "crash", 2024-09-10T09:06:43.913 INFO:teuthology.orchestra.run.smithi172.stdout: "devicehealth", 2024-09-10T09:06:43.913 INFO:teuthology.orchestra.run.smithi172.stdout: "orchestrator", 2024-09-10T09:06:43.913 INFO:teuthology.orchestra.run.smithi172.stdout: "pg_autoscaler", 2024-09-10T09:06:43.913 INFO:teuthology.orchestra.run.smithi172.stdout: "progress", 2024-09-10T09:06:43.913 INFO:teuthology.orchestra.run.smithi172.stdout: "rbd_support", 2024-09-10T09:06:43.913 INFO:teuthology.orchestra.run.smithi172.stdout: "status", 2024-09-10T09:06:43.913 INFO:teuthology.orchestra.run.smithi172.stdout: "telemetry", 2024-09-10T09:06:43.913 INFO:teuthology.orchestra.run.smithi172.stdout: "volumes" 2024-09-10T09:06:43.913 INFO:teuthology.orchestra.run.smithi172.stdout: ], 2024-09-10T09:06:43.913 INFO:teuthology.orchestra.run.smithi172.stdout: "squid": [ 2024-09-10T09:06:43.913 INFO:teuthology.orchestra.run.smithi172.stdout: "balancer", 2024-09-10T09:06:43.913 INFO:teuthology.orchestra.run.smithi172.stdout: "crash", 2024-09-10T09:06:43.914 INFO:teuthology.orchestra.run.smithi172.stdout: "devicehealth", 2024-09-10T09:06:43.914 INFO:teuthology.orchestra.run.smithi172.stdout: "orchestrator", 2024-09-10T09:06:43.914 INFO:teuthology.orchestra.run.smithi172.stdout: "pg_autoscaler", 2024-09-10T09:06:43.914 INFO:teuthology.orchestra.run.smithi172.stdout: "progress", 2024-09-10T09:06:43.914 INFO:teuthology.orchestra.run.smithi172.stdout: "rbd_support", 2024-09-10T09:06:43.914 INFO:teuthology.orchestra.run.smithi172.stdout: "status", 2024-09-10T09:06:43.914 INFO:teuthology.orchestra.run.smithi172.stdout: "telemetry", 2024-09-10T09:06:43.914 INFO:teuthology.orchestra.run.smithi172.stdout: "volumes" 2024-09-10T09:06:43.914 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-09-10T09:06:43.914 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.914 INFO:teuthology.orchestra.run.smithi172.stdout: "last_failure_osd_epoch": 3, 2024-09-10T09:06:43.914 INFO:teuthology.orchestra.run.smithi172.stdout: "active_clients": [ 2024-09-10T09:06:43.914 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.914 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "devicehealth", 2024-09-10T09:06:43.914 INFO:teuthology.orchestra.run.smithi172.stdout: "addrvec": [ 2024-09-10T09:06:43.915 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.915 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v2", 2024-09-10T09:06:43.915 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "172.21.15.172:0", 2024-09-10T09:06:43.915 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 3583026383 2024-09-10T09:06:43.915 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.915 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-09-10T09:06:43.915 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.915 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.915 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "libcephsqlite", 2024-09-10T09:06:43.915 INFO:teuthology.orchestra.run.smithi172.stdout: "addrvec": [ 2024-09-10T09:06:43.915 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.915 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v2", 2024-09-10T09:06:43.915 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "172.21.15.172:0", 2024-09-10T09:06:43.916 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 535101180 2024-09-10T09:06:43.916 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.916 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-09-10T09:06:43.916 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.916 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.916 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "rbd_support", 2024-09-10T09:06:43.916 INFO:teuthology.orchestra.run.smithi172.stdout: "addrvec": [ 2024-09-10T09:06:43.916 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.916 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v2", 2024-09-10T09:06:43.916 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "172.21.15.172:0", 2024-09-10T09:06:43.916 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 1155523672 2024-09-10T09:06:43.916 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.916 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-09-10T09:06:43.917 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:43.917 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.917 INFO:teuthology.orchestra.run.smithi172.stdout: "name": "volumes", 2024-09-10T09:06:43.917 INFO:teuthology.orchestra.run.smithi172.stdout: "addrvec": [ 2024-09-10T09:06:43.917 INFO:teuthology.orchestra.run.smithi172.stdout: { 2024-09-10T09:06:43.917 INFO:teuthology.orchestra.run.smithi172.stdout: "type": "v2", 2024-09-10T09:06:43.917 INFO:teuthology.orchestra.run.smithi172.stdout: "addr": "172.21.15.172:0", 2024-09-10T09:06:43.917 INFO:teuthology.orchestra.run.smithi172.stdout: "nonce": 3363064156 2024-09-10T09:06:43.917 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.917 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-09-10T09:06:43.917 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:43.917 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-09-10T09:06:43.917 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:06:43.919 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:42 smithi191 ceph-mon[35314]: pgmap v964: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:06:43.919 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:42 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1736717924' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T09:06:43.919 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:42 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3532174082' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T09:06:43.919 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:42 smithi174 ceph-mon[34068]: pgmap v964: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:06:43.919 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:42 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1736717924' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T09:06:43.919 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:42 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3532174082' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T09:06:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.506+0000 7f41d459f640 1 -- 172.21.15.172:0/3532174082 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f41b80a2b90 msgr2=0x7f41b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.506+0000 7f41d459f640 1 --2- 172.21.15.172:0/3532174082 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f41b80a2b90 0x7f41b80a5050 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009b60 tx=0x7f41c0005f90 comp rx=0 tx=0).stop 2024-09-10T09:06:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.506+0000 7f41d459f640 1 -- 172.21.15.172:0/3532174082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41cc107870 msgr2=0x7f41cc100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:06:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.506+0000 7f41d459f640 1 --2- 172.21.15.172:0/3532174082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41cc107870 0x7f41cc100b00 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f41c4031970 tx=0x7f41c4009800 comp rx=0 tx=0).stop 2024-09-10T09:06:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.506+0000 7f41d459f640 1 -- 172.21.15.172:0/3532174082 shutdown_connections 2024-09-10T09:06:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.506+0000 7f41d459f640 1 --2- 172.21.15.172:0/3532174082 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f41b80a2b90 0x7f41b80a5050 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:43.920 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.506+0000 7f41d459f640 1 --2- 172.21.15.172:0/3532174082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41cc107870 0x7f41cc100b00 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.506+0000 7f41d459f640 1 --2- 172.21.15.172:0/3532174082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41cc104f40 0x7f41cc1005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.506+0000 7f41d459f640 1 --2- 172.21.15.172:0/3532174082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41cc102610 0x7f41cc101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:06:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.506+0000 7f41d459f640 1 -- 172.21.15.172:0/3532174082 >> 172.21.15.172:0/3532174082 conn(0x7f41cc0fc170 msgr2=0x7f41cc102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:06:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.507+0000 7f41d459f640 1 -- 172.21.15.172:0/3532174082 shutdown_connections 2024-09-10T09:06:43.921 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:06:42.507+0000 7f41d459f640 1 -- 172.21.15.172:0/3532174082 wait complete. 2024-09-10T09:06:43.927 DEBUG:tasks.check_counter:Getting stats from a 2024-09-10T09:06:43.927 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-10T09:06:43.928 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:42 smithi172 ceph-mon[30854]: pgmap v964: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:06:43.928 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:42 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1736717924' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T09:06:43.928 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:42 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3532174082' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T09:06:44.179 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:06:44.756 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:06:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T09:06:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 1700, 2024-09-10T09:06:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 1676, 2024-09-10T09:06:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 1118384, 2024-09-10T09:06:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 640624, 2024-09-10T09:06:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 5, 2024-09-10T09:06:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 1, 2024-09-10T09:06:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.181215416, 2024-09-10T09:06:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.071624468, 2024-09-10T09:06:44.757 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.059357530, 2024-09-10T09:06:44.758 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.043216762, 2024-09-10T09:06:44.758 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:44.758 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1675, 2024-09-10T09:06:44.758 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.120074967, 2024-09-10T09:06:44.758 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000071686 2024-09-10T09:06:44.758 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.758 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:44.758 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.758 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.758 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.758 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.758 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 680240, 2024-09-10T09:06:44.759 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 225952 2024-09-10T09:06:44.759 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.760 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T09:06:44.760 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 1226, 2024-09-10T09:06:44.760 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 1502, 2024-09-10T09:06:44.761 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 302162, 2024-09-10T09:06:44.761 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 949474, 2024-09-10T09:06:44.761 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 6, 2024-09-10T09:06:44.761 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 4, 2024-09-10T09:06:44.761 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.121408719, 2024-09-10T09:06:44.761 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.044547431, 2024-09-10T09:06:44.761 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.024712299, 2024-09-10T09:06:44.761 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.041985253, 2024-09-10T09:06:44.761 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:44.761 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1500, 2024-09-10T09:06:44.761 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.060745668, 2024-09-10T09:06:44.761 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000040497 2024-09-10T09:06:44.762 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.762 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:44.762 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.762 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.762 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.762 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.762 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T09:06:44.762 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 559856 2024-09-10T09:06:44.762 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.762 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T09:06:44.762 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 1537, 2024-09-10T09:06:44.762 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 1537, 2024-09-10T09:06:44.762 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 1710395, 2024-09-10T09:06:44.763 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 487943, 2024-09-10T09:06:44.763 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 5, 2024-09-10T09:06:44.763 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 3, 2024-09-10T09:06:44.763 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.128897125, 2024-09-10T09:06:44.763 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.036363453, 2024-09-10T09:06:44.763 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.036691509, 2024-09-10T09:06:44.763 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.045701804, 2024-09-10T09:06:44.763 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:44.763 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1536, 2024-09-10T09:06:44.763 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.071673182, 2024-09-10T09:06:44.763 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000046662 2024-09-10T09:06:44.763 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.764 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:44.764 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.764 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.764 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.764 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.764 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-10T09:06:44.764 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T09:06:44.764 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.764 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-09-10T09:06:44.764 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-09-10T09:06:44.764 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-09-10T09:06:44.764 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.765 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-09-10T09:06:44.765 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-10T09:06:44.765 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-10T09:06:44.765 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 6669, 2024-09-10T09:06:44.765 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.176212187, 2024-09-10T09:06:44.765 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000026422 2024-09-10T09:06:44.765 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:44.765 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.765 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-09-10T09:06:44.765 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-10T09:06:44.765 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-10T09:06:44.765 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.765 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.766 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.766 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:44.766 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.766 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-09-10T09:06:44.766 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 0, 2024-09-10T09:06:44.766 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 0, 2024-09-10T09:06:44.766 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-09-10T09:06:44.766 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.766 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.766 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.766 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.766 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-09-10T09:06:44.767 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 0, 2024-09-10T09:06:44.767 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 0, 2024-09-10T09:06:44.767 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-09-10T09:06:44.767 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 0, 2024-09-10T09:06:44.767 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 0, 2024-09-10T09:06:44.767 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 0, 2024-09-10T09:06:44.767 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 124, 2024-09-10T09:06:44.767 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 0, 2024-09-10T09:06:44.767 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 124, 2024-09-10T09:06:44.767 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 0, 2024-09-10T09:06:44.767 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 123, 2024-09-10T09:06:44.767 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 0, 2024-09-10T09:06:44.767 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 0, 2024-09-10T09:06:44.767 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 0, 2024-09-10T09:06:44.768 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 3, 2024-09-10T09:06:44.768 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 0, 2024-09-10T09:06:44.768 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 0, 2024-09-10T09:06:44.768 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 0, 2024-09-10T09:06:44.768 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 0, 2024-09-10T09:06:44.768 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-09-10T09:06:44.768 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-09-10T09:06:44.768 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 0, 2024-09-10T09:06:44.768 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 0, 2024-09-10T09:06:44.768 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-09-10T09:06:44.768 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 0, 2024-09-10T09:06:44.768 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 0, 2024-09-10T09:06:44.768 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 0, 2024-09-10T09:06:44.768 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 0, 2024-09-10T09:06:44.769 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-09-10T09:06:44.769 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 0, 2024-09-10T09:06:44.769 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-09-10T09:06:44.769 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 0, 2024-09-10T09:06:44.769 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 0, 2024-09-10T09:06:44.769 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 0, 2024-09-10T09:06:44.769 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T09:06:44.769 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-09-10T09:06:44.769 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T09:06:44.769 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-09-10T09:06:44.769 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 0, 2024-09-10T09:06:44.769 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T09:06:44.769 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T09:06:44.770 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 0, 2024-09-10T09:06:44.770 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 0, 2024-09-10T09:06:44.770 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T09:06:44.770 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 0, 2024-09-10T09:06:44.770 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T09:06:44.770 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T09:06:44.770 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T09:06:44.770 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 0, 2024-09-10T09:06:44.770 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 0, 2024-09-10T09:06:44.770 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 0, 2024-09-10T09:06:44.770 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 0 2024-09-10T09:06:44.770 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.771 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-09-10T09:06:44.771 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 0, 2024-09-10T09:06:44.771 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 0, 2024-09-10T09:06:44.771 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 0, 2024-09-10T09:06:44.771 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 0, 2024-09-10T09:06:44.771 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 0, 2024-09-10T09:06:44.771 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 0, 2024-09-10T09:06:44.771 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-09-10T09:06:44.771 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-09-10T09:06:44.771 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 0, 2024-09-10T09:06:44.771 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 0, 2024-09-10T09:06:44.771 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-09-10T09:06:44.771 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-09-10T09:06:44.771 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-09-10T09:06:44.771 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-09-10T09:06:44.772 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-09-10T09:06:44.772 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-09-10T09:06:44.772 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-09-10T09:06:44.772 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 0, 2024-09-10T09:06:44.772 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 0, 2024-09-10T09:06:44.772 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T09:06:44.772 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 0, 2024-09-10T09:06:44.772 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-09-10T09:06:44.772 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 0, 2024-09-10T09:06:44.772 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-09-10T09:06:44.772 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-09-10T09:06:44.772 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.773 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-09-10T09:06:44.773 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-09-10T09:06:44.773 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 0, 2024-09-10T09:06:44.773 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-09-10T09:06:44.773 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-09-10T09:06:44.773 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 832, 2024-09-10T09:06:44.773 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-09-10T09:06:44.773 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-09-10T09:06:44.773 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 0, 2024-09-10T09:06:44.773 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-09-10T09:06:44.773 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-09-10T09:06:44.773 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 2, 2024-09-10T09:06:44.773 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 2, 2024-09-10T09:06:44.774 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-09-10T09:06:44.774 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-09-10T09:06:44.774 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 4194304, 2024-09-10T09:06:44.774 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 5503716, 2024-09-10T09:06:44.774 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 5503142, 2024-09-10T09:06:44.774 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-09-10T09:06:44.774 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.774 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.774 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.774 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.774 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 832 2024-09-10T09:06:44.774 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.774 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-09-10T09:06:44.774 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 9, 2024-09-10T09:06:44.775 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 129, 2024-09-10T09:06:44.775 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 120, 2024-09-10T09:06:44.775 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 7, 2024-09-10T09:06:44.775 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 19, 2024-09-10T09:06:44.775 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 12, 2024-09-10T09:06:44.775 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 124, 2024-09-10T09:06:44.775 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 244, 2024-09-10T09:06:44.775 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 120, 2024-09-10T09:06:44.775 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 0, 2024-09-10T09:06:44.775 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 0, 2024-09-10T09:06:44.775 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 0, 2024-09-10T09:06:44.775 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 51040, 2024-09-10T09:06:44.775 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 215328 2024-09-10T09:06:44.775 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.775 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-09-10T09:06:44.776 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 0, 2024-09-10T09:06:44.776 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 0, 2024-09-10T09:06:44.776 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 0, 2024-09-10T09:06:44.776 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 0, 2024-09-10T09:06:44.776 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 0, 2024-09-10T09:06:44.776 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-09-10T09:06:44.776 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.776 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.776 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.776 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.776 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-09-10T09:06:44.776 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.776 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.776 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.777 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.777 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-09-10T09:06:44.777 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.777 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.777 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.777 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.777 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-09-10T09:06:44.777 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.777 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.777 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.777 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.777 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-09-10T09:06:44.777 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.778 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.778 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.778 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.778 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-09-10T09:06:44.778 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.778 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.778 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.778 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.778 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-09-10T09:06:44.778 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.778 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.778 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.778 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.779 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-09-10T09:06:44.779 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.779 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.779 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.779 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.779 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-09-10T09:06:44.779 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.779 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.779 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.779 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.779 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-09-10T09:06:44.779 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.779 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.779 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.780 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.780 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-09-10T09:06:44.780 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.780 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.780 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.780 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.780 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-09-10T09:06:44.780 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.781 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.781 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.781 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.781 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-09-10T09:06:44.781 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.781 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.781 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.781 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.781 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-09-10T09:06:44.781 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.781 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.782 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.782 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.782 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-09-10T09:06:44.782 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.782 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.782 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.782 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.782 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-09-10T09:06:44.782 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.782 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.782 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.782 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.783 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-09-10T09:06:44.783 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.783 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.783 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.783 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.783 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-09-10T09:06:44.783 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.783 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.783 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.783 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.783 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-09-10T09:06:44.783 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.783 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.784 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.784 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.784 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-09-10T09:06:44.784 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.784 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.784 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.784 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.784 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-09-10T09:06:44.784 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.784 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.784 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.784 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.784 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-09-10T09:06:44.785 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.785 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.785 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.785 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.785 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-09-10T09:06:44.785 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.785 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.785 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.785 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.785 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-09-10T09:06:44.785 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.785 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.785 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.786 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.786 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-09-10T09:06:44.786 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.786 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.786 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.786 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.786 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-09-10T09:06:44.786 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.786 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.786 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.786 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.786 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-09-10T09:06:44.787 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.787 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.787 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.787 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.787 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-09-10T09:06:44.787 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.787 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.787 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.787 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.787 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-09-10T09:06:44.787 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.787 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.788 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.788 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.788 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-09-10T09:06:44.788 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-09-10T09:06:44.788 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-09-10T09:06:44.788 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.788 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.788 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.788 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:44.788 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.788 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-09-10T09:06:44.788 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 1, 2024-09-10T09:06:44.788 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 2, 2024-09-10T09:06:44.789 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 1, 2024-09-10T09:06:44.789 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 1, 2024-09-10T09:06:44.789 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-09-10T09:06:44.789 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 0, 2024-09-10T09:06:44.789 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 0, 2024-09-10T09:06:44.789 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 2290, 2024-09-10T09:06:44.789 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-09-10T09:06:44.789 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.789 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-09-10T09:06:44.789 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-09-10T09:06:44.790 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-09-10T09:06:44.790 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T09:06:44.790 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-09-10T09:06:44.790 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T09:06:44.790 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-09-10T09:06:44.790 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T09:06:44.790 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T09:06:44.790 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T09:06:44.790 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T09:06:44.790 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T09:06:44.790 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-09-10T09:06:44.790 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T09:06:44.791 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T09:06:44.791 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-09-10T09:06:44.791 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-09-10T09:06:44.791 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-09-10T09:06:44.791 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-09-10T09:06:44.791 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T09:06:44.791 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T09:06:44.791 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T09:06:44.791 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T09:06:44.791 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T09:06:44.791 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-09-10T09:06:44.791 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-09-10T09:06:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-09-10T09:06:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T09:06:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T09:06:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-09-10T09:06:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-09-10T09:06:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-09-10T09:06:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-09-10T09:06:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T09:06:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-09-10T09:06:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T09:06:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-09-10T09:06:44.792 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 190167, 2024-09-10T09:06:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 52, 2024-09-10T09:06:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-09-10T09:06:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-09-10T09:06:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-09-10T09:06:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-09-10T09:06:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-09-10T09:06:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-09-10T09:06:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-09-10T09:06:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-09-10T09:06:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 8672, 2024-09-10T09:06:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 163, 2024-09-10T09:06:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T09:06:44.793 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-09-10T09:06:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-09-10T09:06:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-09-10T09:06:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 116789, 2024-09-10T09:06:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 598, 2024-09-10T09:06:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-09-10T09:06:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-09-10T09:06:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-09-10T09:06:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-09-10T09:06:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-09-10T09:06:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-09-10T09:06:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-09-10T09:06:44.794 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 4062, 2024-09-10T09:06:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 21, 2024-09-10T09:06:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 3, 2024-09-10T09:06:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 4062, 2024-09-10T09:06:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-09-10T09:06:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 4062, 2024-09-10T09:06:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 5.739861226, 2024-09-10T09:06:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.001413062 2024-09-10T09:06:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-09-10T09:06:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-09-10T09:06:44.795 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 4062, 2024-09-10T09:06:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 4063 2024-09-10T09:06:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 4062, 2024-09-10T09:06:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 4062, 2024-09-10T09:06:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 0, 2024-09-10T09:06:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-09-10T09:06:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-09-10T09:06:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 2526, 2024-09-10T09:06:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 0, 2024-09-10T09:06:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 1534, 2024-09-10T09:06:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 0, 2024-09-10T09:06:44.796 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 0, 2024-09-10T09:06:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-09-10T09:06:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-09-10T09:06:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-09-10T09:06:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 0, 2024-09-10T09:06:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 0, 2024-09-10T09:06:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-09-10T09:06:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-09-10T09:06:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-09-10T09:06:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 1, 2024-09-10T09:06:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 0, 2024-09-10T09:06:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-09-10T09:06:44.797 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-09-10T09:06:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-09-10T09:06:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-09-10T09:06:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-09-10T09:06:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-09-10T09:06:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T09:06:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-09-10T09:06:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-09-10T09:06:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-09-10T09:06:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-09-10T09:06:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-09-10T09:06:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-09-10T09:06:44.798 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-09-10T09:06:44.799 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-09-10T09:06:44.799 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-09-10T09:06:44.799 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-09-10T09:06:44.799 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-09-10T09:06:44.799 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-09-10T09:06:44.799 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-09-10T09:06:44.799 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-09-10T09:06:44.799 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-09-10T09:06:44.799 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-09-10T09:06:44.799 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-09-10T09:06:44.799 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-09-10T09:06:44.799 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-09-10T09:06:44.800 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 92, 2024-09-10T09:06:44.800 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-09-10T09:06:44.800 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 4, 2024-09-10T09:06:44.800 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 5, 2024-09-10T09:06:44.800 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 7, 2024-09-10T09:06:44.800 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 2, 2024-09-10T09:06:44.800 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-09-10T09:06:44.800 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 0, 2024-09-10T09:06:44.800 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 2, 2024-09-10T09:06:44.800 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 0 2024-09-10T09:06:44.800 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.800 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-09-10T09:06:44.800 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 0, 2024-09-10T09:06:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-09-10T09:06:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-09-10T09:06:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-09-10T09:06:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-09-10T09:06:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 0 2024-09-10T09:06:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-09-10T09:06:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-09-10T09:06:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T09:06:44.801 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-09-10T09:06:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 0, 2024-09-10T09:06:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 0, 2024-09-10T09:06:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 0, 2024-09-10T09:06:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-09-10T09:06:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T09:06:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-10T09:06:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-10T09:06:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-10T09:06:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 4459, 2024-09-10T09:06:44.802 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 2706404, 2024-09-10T09:06:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 4459, 2024-09-10T09:06:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-10T09:06:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-10T09:06:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 4459, 2024-09-10T09:06:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 2706404, 2024-09-10T09:06:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-10T09:06:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.803 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-09-10T09:06:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-10T09:06:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-10T09:06:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-10T09:06:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-10T09:06:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-10T09:06:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 4062, 2024-09-10T09:06:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 1309417, 2024-09-10T09:06:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 4062, 2024-09-10T09:06:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 1309417, 2024-09-10T09:06:44.804 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-10T09:06:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-09-10T09:06:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-10T09:06:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-09-10T09:06:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-10T09:06:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-10T09:06:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-10T09:06:44.805 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 4062, 2024-09-10T09:06:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 4062, 2024-09-10T09:06:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 4062, 2024-09-10T09:06:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 4062, 2024-09-10T09:06:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-10T09:06:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-09-10T09:06:44.806 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-10T09:06:44.807 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-10T09:06:44.807 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-10T09:06:44.807 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-10T09:06:44.807 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-10T09:06:44.807 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:44.807 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:44.807 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-10T09:06:44.807 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-10T09:06:44.807 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-09-10T09:06:44.807 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-09-10T09:06:44.807 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-10T09:06:44.807 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.807 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.808 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.808 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:44.808 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:44.808 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x55c569c4ab40": { 2024-09-10T09:06:44.808 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-10T09:06:44.808 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-10T09:06:44.808 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-10T09:06:44.808 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-10T09:06:44.808 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-10T09:06:44.808 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:44.808 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:44.808 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-10T09:06:44.808 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-10T09:06:44.808 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-09-10T09:06:44.809 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-09-10T09:06:44.809 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-10T09:06:44.809 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:44.809 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:44.809 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:44.809 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:44.809 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:44.809 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:06:45.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:45 smithi191 ceph-mon[35314]: pgmap v965: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:06:45.330 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:45 smithi172 ceph-mon[30854]: pgmap v965: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:06:45.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:45 smithi174 ceph-mon[34068]: pgmap v965: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:06:45.792 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-10T09:06:45.792 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-10T09:06:45.792 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-09-10T09:06:45.792 DEBUG:tasks.check_counter:Getting stats from d 2024-09-10T09:06:45.793 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-10T09:06:46.043 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:06:46.594 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:06:46.594 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T09:06:46.594 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 0, 2024-09-10T09:06:46.595 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 2, 2024-09-10T09:06:46.595 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 0, 2024-09-10T09:06:46.595 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 0, 2024-09-10T09:06:46.595 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 4, 2024-09-10T09:06:46.595 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 0, 2024-09-10T09:06:46.595 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.001110832, 2024-09-10T09:06:46.595 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T09:06:46.595 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.000217302, 2024-09-10T09:06:46.595 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T09:06:46.595 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:46.595 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:46.595 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:46.596 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:46.596 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:46.596 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:46.596 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:46.596 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:46.596 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:46.596 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:46.596 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T09:06:46.596 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T09:06:46.596 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:46.596 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T09:06:46.597 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 2, 2024-09-10T09:06:46.597 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 282, 2024-09-10T09:06:46.597 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 320, 2024-09-10T09:06:46.597 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 206848, 2024-09-10T09:06:46.597 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 4, 2024-09-10T09:06:46.597 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 2, 2024-09-10T09:06:46.597 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.020964993, 2024-09-10T09:06:46.597 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.016129462, 2024-09-10T09:06:46.597 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.002023803, 2024-09-10T09:06:46.597 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T09:06:46.597 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:46.597 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 280, 2024-09-10T09:06:46.598 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.020710443, 2024-09-10T09:06:46.598 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000073965 2024-09-10T09:06:46.598 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:46.598 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:46.598 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:46.598 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:46.598 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:46.598 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:46.598 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T09:06:46.598 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 206848 2024-09-10T09:06:46.598 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:46.600 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T09:06:46.600 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 375, 2024-09-10T09:06:46.600 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 370, 2024-09-10T09:06:46.600 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 524576, 2024-09-10T09:06:46.600 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 228064, 2024-09-10T09:06:46.600 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 2, 2024-09-10T09:06:46.600 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 1, 2024-09-10T09:06:46.600 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.048940882, 2024-09-10T09:06:46.600 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.023560237, 2024-09-10T09:06:46.601 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.029003025, 2024-09-10T09:06:46.601 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T09:06:46.601 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:46.601 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 370, 2024-09-10T09:06:46.601 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.024861111, 2024-09-10T09:06:46.601 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000067192 2024-09-10T09:06:46.601 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:46.601 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:46.601 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:46.601 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:46.601 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:46.601 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:46.602 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 524576, 2024-09-10T09:06:46.602 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 228064 2024-09-10T09:06:46.602 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:46.602 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-09-10T09:06:46.602 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 0, 2024-09-10T09:06:46.602 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-09-10T09:06:46.602 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:46.602 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-09-10T09:06:46.602 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-09-10T09:06:46.602 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-09-10T09:06:46.602 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T09:06:46.603 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-09-10T09:06:46.603 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T09:06:46.603 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-09-10T09:06:46.603 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T09:06:46.603 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T09:06:46.603 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T09:06:46.603 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T09:06:46.603 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T09:06:46.603 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-09-10T09:06:46.603 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T09:06:46.603 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T09:06:46.603 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-09-10T09:06:46.603 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-09-10T09:06:46.604 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-09-10T09:06:46.604 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-09-10T09:06:46.604 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T09:06:46.604 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T09:06:46.604 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T09:06:46.604 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T09:06:46.604 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T09:06:46.604 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-09-10T09:06:46.604 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-09-10T09:06:46.604 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-09-10T09:06:46.604 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T09:06:46.604 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T09:06:46.604 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-09-10T09:06:46.604 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-09-10T09:06:46.605 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-09-10T09:06:46.605 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-09-10T09:06:46.605 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T09:06:46.605 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-09-10T09:06:46.605 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T09:06:46.605 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-09-10T09:06:46.605 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 148704, 2024-09-10T09:06:46.605 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 29, 2024-09-10T09:06:46.605 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-09-10T09:06:46.605 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-09-10T09:06:46.605 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-09-10T09:06:46.605 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-09-10T09:06:46.605 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-09-10T09:06:46.606 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-09-10T09:06:46.606 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-09-10T09:06:46.606 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-09-10T09:06:46.606 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 0, 2024-09-10T09:06:46.606 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 0, 2024-09-10T09:06:46.606 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T09:06:46.606 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-09-10T09:06:46.606 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-09-10T09:06:46.606 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-09-10T09:06:46.606 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 928, 2024-09-10T09:06:46.606 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 2, 2024-09-10T09:06:46.606 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-09-10T09:06:46.606 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-09-10T09:06:46.606 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-09-10T09:06:46.606 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-09-10T09:06:46.607 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:46.607 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T09:06:46.607 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-10T09:06:46.607 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-10T09:06:46.607 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-10T09:06:46.607 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 373, 2024-09-10T09:06:46.607 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 469182, 2024-09-10T09:06:46.607 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:46.607 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 373, 2024-09-10T09:06:46.607 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-10T09:06:46.607 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-10T09:06:46.607 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 373, 2024-09-10T09:06:46.607 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 469182, 2024-09-10T09:06:46.607 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-10T09:06:46.607 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:46.608 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:46.608 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:46.608 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:46.608 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:46.608 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:06:47.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:47 smithi191 ceph-mon[35314]: pgmap v966: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:06:47.315 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:47 smithi172 ceph-mon[30854]: pgmap v966: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:06:47.344 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-10T09:06:47.344 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-10T09:06:47.344 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-10T09:06:47.344 DEBUG:tasks.check_counter:Getting stats from g 2024-09-10T09:06:47.344 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-10T09:06:47.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:47 smithi174 ceph-mon[34068]: pgmap v966: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:06:47.681 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:06:48.298 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:06:48.298 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T09:06:48.298 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 1137, 2024-09-10T09:06:48.298 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 1144, 2024-09-10T09:06:48.298 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 383577, 2024-09-10T09:06:48.298 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 361282, 2024-09-10T09:06:48.298 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 7, 2024-09-10T09:06:48.298 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 2, 2024-09-10T09:06:48.298 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.129834933, 2024-09-10T09:06:48.298 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.055687566, 2024-09-10T09:06:48.299 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.031032590, 2024-09-10T09:06:48.299 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.032419947, 2024-09-10T09:06:48.299 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:48.299 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1143, 2024-09-10T09:06:48.299 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.066978893, 2024-09-10T09:06:48.299 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000058599 2024-09-10T09:06:48.299 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.299 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:48.299 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.299 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.299 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.299 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.299 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-10T09:06:48.299 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T09:06:48.300 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.300 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T09:06:48.300 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 2650, 2024-09-10T09:06:48.300 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 2626, 2024-09-10T09:06:48.300 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 1880096, 2024-09-10T09:06:48.300 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 943425, 2024-09-10T09:06:48.300 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 4, 2024-09-10T09:06:48.300 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 3, 2024-09-10T09:06:48.300 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.190716306, 2024-09-10T09:06:48.300 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.064468670, 2024-09-10T09:06:48.300 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.067712241, 2024-09-10T09:06:48.300 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.052981654, 2024-09-10T09:06:48.301 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:48.301 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2625, 2024-09-10T09:06:48.301 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.107782740, 2024-09-10T09:06:48.301 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000041060 2024-09-10T09:06:48.301 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.301 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:48.301 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.301 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.301 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.301 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.301 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 680784, 2024-09-10T09:06:48.302 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 227232 2024-09-10T09:06:48.302 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.302 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T09:06:48.302 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 5, 2024-09-10T09:06:48.302 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 282, 2024-09-10T09:06:48.302 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 1668, 2024-09-10T09:06:48.302 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 650853, 2024-09-10T09:06:48.302 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 8, 2024-09-10T09:06:48.302 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 4, 2024-09-10T09:06:48.302 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.021801409, 2024-09-10T09:06:48.302 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.016348179, 2024-09-10T09:06:48.302 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.002898533, 2024-09-10T09:06:48.302 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000060141, 2024-09-10T09:06:48.303 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:48.303 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 280, 2024-09-10T09:06:48.303 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.014886426, 2024-09-10T09:06:48.303 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000053165 2024-09-10T09:06:48.303 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.303 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:48.303 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.303 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.303 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.303 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.303 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T09:06:48.303 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 649888 2024-09-10T09:06:48.303 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.304 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-09-10T09:06:48.304 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-09-10T09:06:48.304 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-09-10T09:06:48.304 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.304 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-09-10T09:06:48.304 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-10T09:06:48.304 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-10T09:06:48.304 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 5689, 2024-09-10T09:06:48.304 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.130885715, 2024-09-10T09:06:48.304 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000023006 2024-09-10T09:06:48.304 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:48.304 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.304 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-09-10T09:06:48.304 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-10T09:06:48.305 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-10T09:06:48.305 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.305 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.305 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.305 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:48.305 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.305 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-09-10T09:06:48.305 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 0, 2024-09-10T09:06:48.305 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 0, 2024-09-10T09:06:48.305 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-09-10T09:06:48.305 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.305 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.305 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.306 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.306 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-09-10T09:06:48.306 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 0, 2024-09-10T09:06:48.306 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 0, 2024-09-10T09:06:48.306 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-09-10T09:06:48.306 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 0, 2024-09-10T09:06:48.306 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 0, 2024-09-10T09:06:48.306 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 0, 2024-09-10T09:06:48.306 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 32, 2024-09-10T09:06:48.306 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 0, 2024-09-10T09:06:48.306 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 32, 2024-09-10T09:06:48.306 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 0, 2024-09-10T09:06:48.306 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 32, 2024-09-10T09:06:48.306 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 0, 2024-09-10T09:06:48.307 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 0, 2024-09-10T09:06:48.307 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 0, 2024-09-10T09:06:48.307 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 1, 2024-09-10T09:06:48.307 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 0, 2024-09-10T09:06:48.307 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 0, 2024-09-10T09:06:48.307 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 0, 2024-09-10T09:06:48.307 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 0, 2024-09-10T09:06:48.307 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-09-10T09:06:48.307 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-09-10T09:06:48.307 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 0, 2024-09-10T09:06:48.307 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 0, 2024-09-10T09:06:48.307 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-09-10T09:06:48.308 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 0, 2024-09-10T09:06:48.308 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 0, 2024-09-10T09:06:48.308 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 0, 2024-09-10T09:06:48.308 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 0, 2024-09-10T09:06:48.308 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-09-10T09:06:48.308 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 0, 2024-09-10T09:06:48.308 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-09-10T09:06:48.308 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 0, 2024-09-10T09:06:48.308 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 0, 2024-09-10T09:06:48.308 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 0, 2024-09-10T09:06:48.308 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T09:06:48.308 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-09-10T09:06:48.308 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T09:06:48.309 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-09-10T09:06:48.309 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 0, 2024-09-10T09:06:48.309 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T09:06:48.309 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T09:06:48.309 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 0, 2024-09-10T09:06:48.309 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 0, 2024-09-10T09:06:48.309 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T09:06:48.309 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 0, 2024-09-10T09:06:48.309 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T09:06:48.309 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T09:06:48.309 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T09:06:48.309 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 0, 2024-09-10T09:06:48.309 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 0, 2024-09-10T09:06:48.309 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 0, 2024-09-10T09:06:48.310 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 0 2024-09-10T09:06:48.310 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.310 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-09-10T09:06:48.310 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 0, 2024-09-10T09:06:48.310 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 0, 2024-09-10T09:06:48.310 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 0, 2024-09-10T09:06:48.310 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 0, 2024-09-10T09:06:48.310 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 0, 2024-09-10T09:06:48.310 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 0, 2024-09-10T09:06:48.310 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-09-10T09:06:48.310 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-09-10T09:06:48.310 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 0, 2024-09-10T09:06:48.310 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 0, 2024-09-10T09:06:48.310 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-09-10T09:06:48.310 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-09-10T09:06:48.311 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-09-10T09:06:48.311 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-09-10T09:06:48.311 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-09-10T09:06:48.311 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-09-10T09:06:48.311 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-09-10T09:06:48.311 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 0, 2024-09-10T09:06:48.311 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 0, 2024-09-10T09:06:48.311 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T09:06:48.311 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 0, 2024-09-10T09:06:48.311 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-09-10T09:06:48.311 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 0, 2024-09-10T09:06:48.311 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-09-10T09:06:48.311 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-09-10T09:06:48.311 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.311 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-09-10T09:06:48.312 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-09-10T09:06:48.312 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 0, 2024-09-10T09:06:48.312 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-09-10T09:06:48.312 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-09-10T09:06:48.312 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 535, 2024-09-10T09:06:48.312 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-09-10T09:06:48.312 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-09-10T09:06:48.312 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 0, 2024-09-10T09:06:48.312 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-09-10T09:06:48.312 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-09-10T09:06:48.312 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 2, 2024-09-10T09:06:48.312 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 2, 2024-09-10T09:06:48.312 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-09-10T09:06:48.312 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-09-10T09:06:48.313 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 4194304, 2024-09-10T09:06:48.313 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 4821448, 2024-09-10T09:06:48.313 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 4819797, 2024-09-10T09:06:48.313 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-09-10T09:06:48.313 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.313 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.313 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.313 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.313 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 535 2024-09-10T09:06:48.313 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.313 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-09-10T09:06:48.313 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 4, 2024-09-10T09:06:48.313 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 45, 2024-09-10T09:06:48.313 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 41, 2024-09-10T09:06:48.314 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 2, 2024-09-10T09:06:48.314 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 14, 2024-09-10T09:06:48.314 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 12, 2024-09-10T09:06:48.314 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 32, 2024-09-10T09:06:48.314 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 73, 2024-09-10T09:06:48.314 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 41, 2024-09-10T09:06:48.314 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 0, 2024-09-10T09:06:48.314 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 0, 2024-09-10T09:06:48.314 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 0, 2024-09-10T09:06:48.314 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 49380, 2024-09-10T09:06:48.314 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 215328 2024-09-10T09:06:48.314 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.314 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-09-10T09:06:48.314 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 0, 2024-09-10T09:06:48.314 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 0, 2024-09-10T09:06:48.315 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 0, 2024-09-10T09:06:48.315 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 0, 2024-09-10T09:06:48.315 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 0, 2024-09-10T09:06:48.315 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-09-10T09:06:48.315 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.315 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.315 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.315 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.315 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-09-10T09:06:48.315 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.315 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.315 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.316 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.316 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-09-10T09:06:48.316 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.316 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.316 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.316 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.316 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-09-10T09:06:48.316 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.316 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.316 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.316 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.316 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-09-10T09:06:48.316 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.317 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.317 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.317 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.317 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-09-10T09:06:48.317 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.317 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.317 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.317 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.317 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-09-10T09:06:48.317 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.317 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.317 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.317 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.318 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-09-10T09:06:48.318 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.318 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.318 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.318 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.318 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-09-10T09:06:48.318 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.318 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.318 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.318 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.318 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-09-10T09:06:48.318 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.318 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.319 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.319 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.319 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-09-10T09:06:48.319 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.319 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.319 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.319 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.319 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-09-10T09:06:48.319 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.320 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.320 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.320 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.320 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-09-10T09:06:48.320 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.320 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.320 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.320 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.320 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-09-10T09:06:48.320 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.320 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.320 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.321 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.321 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-09-10T09:06:48.321 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.321 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.321 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.321 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.321 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-09-10T09:06:48.321 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.321 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.321 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.321 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.321 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-09-10T09:06:48.321 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.322 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.322 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.322 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.322 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-09-10T09:06:48.322 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.322 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.322 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.322 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.322 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-09-10T09:06:48.322 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.322 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.322 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.323 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.323 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-09-10T09:06:48.323 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.323 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.323 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.323 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.323 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-09-10T09:06:48.323 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.323 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.323 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.323 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.323 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-09-10T09:06:48.323 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.324 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.324 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.324 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.324 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-09-10T09:06:48.324 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.324 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.324 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.324 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.324 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-09-10T09:06:48.324 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.324 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.324 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.324 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.325 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-09-10T09:06:48.325 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.325 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.325 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.325 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.325 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-09-10T09:06:48.325 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.325 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.325 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.325 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.325 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-09-10T09:06:48.325 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.326 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.326 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.326 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.326 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-09-10T09:06:48.326 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.326 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.326 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.326 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.326 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-09-10T09:06:48.326 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.326 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.326 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.326 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.327 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-09-10T09:06:48.327 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-09-10T09:06:48.327 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-09-10T09:06:48.327 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.327 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.327 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.327 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:48.327 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.327 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-09-10T09:06:48.327 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 1, 2024-09-10T09:06:48.327 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 1, 2024-09-10T09:06:48.327 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 0, 2024-09-10T09:06:48.328 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 1, 2024-09-10T09:06:48.328 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-09-10T09:06:48.328 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 0, 2024-09-10T09:06:48.328 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 0, 2024-09-10T09:06:48.328 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 2292, 2024-09-10T09:06:48.328 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-09-10T09:06:48.328 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.328 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-09-10T09:06:48.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-09-10T09:06:48.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-09-10T09:06:48.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T09:06:48.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-09-10T09:06:48.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T09:06:48.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-09-10T09:06:48.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T09:06:48.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T09:06:48.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T09:06:48.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T09:06:48.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T09:06:48.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-09-10T09:06:48.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T09:06:48.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T09:06:48.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-09-10T09:06:48.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-09-10T09:06:48.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-09-10T09:06:48.330 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-09-10T09:06:48.330 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T09:06:48.330 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T09:06:48.330 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T09:06:48.330 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T09:06:48.330 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T09:06:48.330 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-09-10T09:06:48.330 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-09-10T09:06:48.330 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-09-10T09:06:48.330 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T09:06:48.330 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T09:06:48.330 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-09-10T09:06:48.330 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-09-10T09:06:48.331 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-09-10T09:06:48.331 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-09-10T09:06:48.331 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T09:06:48.331 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-09-10T09:06:48.331 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T09:06:48.331 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-09-10T09:06:48.331 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 198304, 2024-09-10T09:06:48.331 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 58, 2024-09-10T09:06:48.331 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-09-10T09:06:48.331 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-09-10T09:06:48.331 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-09-10T09:06:48.331 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-09-10T09:06:48.332 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-09-10T09:06:48.332 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-09-10T09:06:48.332 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-09-10T09:06:48.332 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-09-10T09:06:48.332 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 8672, 2024-09-10T09:06:48.332 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 163, 2024-09-10T09:06:48.332 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T09:06:48.332 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-09-10T09:06:48.332 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-09-10T09:06:48.332 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-09-10T09:06:48.332 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 34576, 2024-09-10T09:06:48.332 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 156, 2024-09-10T09:06:48.332 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-09-10T09:06:48.333 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-09-10T09:06:48.333 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-09-10T09:06:48.333 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-09-10T09:06:48.333 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.333 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-09-10T09:06:48.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-09-10T09:06:48.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-09-10T09:06:48.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 3388, 2024-09-10T09:06:48.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 21, 2024-09-10T09:06:48.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 3, 2024-09-10T09:06:48.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 3388, 2024-09-10T09:06:48.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-09-10T09:06:48.334 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3388, 2024-09-10T09:06:48.334 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 182.042634790, 2024-09-10T09:06:48.334 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.053731592 2024-09-10T09:06:48.334 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.334 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-09-10T09:06:48.334 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-09-10T09:06:48.334 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3388, 2024-09-10T09:06:48.334 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 3389 2024-09-10T09:06:48.334 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.334 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 3388, 2024-09-10T09:06:48.334 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 3388, 2024-09-10T09:06:48.334 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 0, 2024-09-10T09:06:48.334 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-09-10T09:06:48.334 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-09-10T09:06:48.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 2165, 2024-09-10T09:06:48.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 0, 2024-09-10T09:06:48.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 1221, 2024-09-10T09:06:48.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 0, 2024-09-10T09:06:48.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 0, 2024-09-10T09:06:48.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-09-10T09:06:48.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-09-10T09:06:48.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-09-10T09:06:48.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 0, 2024-09-10T09:06:48.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 0, 2024-09-10T09:06:48.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-09-10T09:06:48.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-09-10T09:06:48.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-09-10T09:06:48.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 1, 2024-09-10T09:06:48.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 0, 2024-09-10T09:06:48.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-09-10T09:06:48.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-09-10T09:06:48.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-09-10T09:06:48.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-09-10T09:06:48.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-09-10T09:06:48.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-09-10T09:06:48.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T09:06:48.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-09-10T09:06:48.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-09-10T09:06:48.337 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-09-10T09:06:48.337 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-09-10T09:06:48.337 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-09-10T09:06:48.337 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-09-10T09:06:48.337 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-09-10T09:06:48.337 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-09-10T09:06:48.337 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-09-10T09:06:48.337 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-09-10T09:06:48.337 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-09-10T09:06:48.337 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-09-10T09:06:48.337 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-09-10T09:06:48.337 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-09-10T09:06:48.337 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-09-10T09:06:48.338 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-09-10T09:06:48.338 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-09-10T09:06:48.338 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-09-10T09:06:48.338 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-09-10T09:06:48.338 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 92, 2024-09-10T09:06:48.338 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-09-10T09:06:48.338 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 4, 2024-09-10T09:06:48.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 6, 2024-09-10T09:06:48.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 9, 2024-09-10T09:06:48.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 3, 2024-09-10T09:06:48.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-09-10T09:06:48.338 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 0, 2024-09-10T09:06:48.338 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 2, 2024-09-10T09:06:48.338 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 0 2024-09-10T09:06:48.339 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.339 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-09-10T09:06:48.339 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 0, 2024-09-10T09:06:48.339 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-09-10T09:06:48.339 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-09-10T09:06:48.339 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-09-10T09:06:48.339 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.339 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-09-10T09:06:48.339 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 0 2024-09-10T09:06:48.339 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.339 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-09-10T09:06:48.339 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-09-10T09:06:48.340 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T09:06:48.340 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-09-10T09:06:48.340 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 0, 2024-09-10T09:06:48.340 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 0, 2024-09-10T09:06:48.340 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 0, 2024-09-10T09:06:48.340 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-09-10T09:06:48.340 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.340 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T09:06:48.340 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-10T09:06:48.340 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-10T09:06:48.340 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-10T09:06:48.340 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 3788, 2024-09-10T09:06:48.341 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 1901047, 2024-09-10T09:06:48.341 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:48.341 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 3788, 2024-09-10T09:06:48.341 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-10T09:06:48.341 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-10T09:06:48.341 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 3788, 2024-09-10T09:06:48.341 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 1901047, 2024-09-10T09:06:48.341 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-10T09:06:48.341 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.341 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.341 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.341 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:48.341 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.342 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-09-10T09:06:48.342 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-10T09:06:48.342 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-10T09:06:48.342 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-10T09:06:48.342 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-10T09:06:48.342 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-10T09:06:48.342 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:48.342 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:48.342 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 3388, 2024-09-10T09:06:48.342 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 627149, 2024-09-10T09:06:48.342 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 3388, 2024-09-10T09:06:48.342 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 627149, 2024-09-10T09:06:48.342 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-10T09:06:48.343 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.343 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.343 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.343 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:48.343 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.343 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-09-10T09:06:48.343 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-10T09:06:48.343 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-09-10T09:06:48.343 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-10T09:06:48.343 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-10T09:06:48.343 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-10T09:06:48.343 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:48.343 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:48.344 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 3388, 2024-09-10T09:06:48.344 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 3388, 2024-09-10T09:06:48.344 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 3388, 2024-09-10T09:06:48.344 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 3388, 2024-09-10T09:06:48.344 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-10T09:06:48.344 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.344 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.344 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.344 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:48.344 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.344 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-09-10T09:06:48.344 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-10T09:06:48.344 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-10T09:06:48.344 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-10T09:06:48.345 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-10T09:06:48.345 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-10T09:06:48.345 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:48.345 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:48.345 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-10T09:06:48.345 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-10T09:06:48.345 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-09-10T09:06:48.345 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-09-10T09:06:48.345 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-10T09:06:48.345 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.345 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.345 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.345 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:48.345 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:48.346 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x55d1b24aab40": { 2024-09-10T09:06:48.346 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-10T09:06:48.346 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-10T09:06:48.346 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-10T09:06:48.346 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-10T09:06:48.346 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-10T09:06:48.346 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:48.346 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:48.346 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-10T09:06:48.346 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-10T09:06:48.346 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-09-10T09:06:48.346 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-09-10T09:06:48.346 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-10T09:06:48.346 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:48.346 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:48.347 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:48.347 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:48.347 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:48.347 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:06:49.095 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-10T09:06:49.095 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-10T09:06:49.096 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-09-10T09:06:49.096 DEBUG:tasks.check_counter:Getting stats from j 2024-09-10T09:06:49.096 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-10T09:06:49.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:49 smithi191 ceph-mon[35314]: pgmap v967: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:06:49.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:49 smithi172 ceph-mon[30854]: pgmap v967: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:06:49.344 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:06:49.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:49 smithi174 ceph-mon[34068]: pgmap v967: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:06:49.841 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-10T09:06:49.841 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T09:06:49.841 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 373, 2024-09-10T09:06:49.841 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 370, 2024-09-10T09:06:49.842 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 522176, 2024-09-10T09:06:49.842 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 228896, 2024-09-10T09:06:49.842 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 4, 2024-09-10T09:06:49.842 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 0, 2024-09-10T09:06:49.842 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.050323486, 2024-09-10T09:06:49.842 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.024385988, 2024-09-10T09:06:49.842 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.028895861, 2024-09-10T09:06:49.842 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T09:06:49.843 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:49.843 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 369, 2024-09-10T09:06:49.843 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.016097802, 2024-09-10T09:06:49.843 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000043625 2024-09-10T09:06:49.843 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:49.843 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:49.843 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:49.843 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:49.843 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:49.843 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:49.843 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 522176, 2024-09-10T09:06:49.843 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 228896 2024-09-10T09:06:49.844 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:49.844 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T09:06:49.844 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 2, 2024-09-10T09:06:49.844 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 283, 2024-09-10T09:06:49.844 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 320, 2024-09-10T09:06:49.844 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 207024, 2024-09-10T09:06:49.844 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 4, 2024-09-10T09:06:49.844 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 2, 2024-09-10T09:06:49.844 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.021014342, 2024-09-10T09:06:49.844 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.016596622, 2024-09-10T09:06:49.844 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.001787617, 2024-09-10T09:06:49.844 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T09:06:49.845 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:49.845 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 281, 2024-09-10T09:06:49.845 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.017780474, 2024-09-10T09:06:49.845 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000063275 2024-09-10T09:06:49.845 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:49.845 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:49.845 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:49.845 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:49.845 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:49.845 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:49.845 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T09:06:49.845 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 207024 2024-09-10T09:06:49.845 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:49.846 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T09:06:49.846 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 4, 2024-09-10T09:06:49.846 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 4, 2024-09-10T09:06:49.846 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 2944, 2024-09-10T09:06:49.846 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 448, 2024-09-10T09:06:49.846 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 2, 2024-09-10T09:06:49.846 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 1, 2024-09-10T09:06:49.846 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.002318991, 2024-09-10T09:06:49.846 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.000030851, 2024-09-10T09:06:49.846 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.002282851, 2024-09-10T09:06:49.847 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T09:06:49.847 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:49.847 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3, 2024-09-10T09:06:49.847 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.004171612, 2024-09-10T09:06:49.847 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.001390537 2024-09-10T09:06:49.847 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:49.847 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:49.847 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:49.847 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:49.847 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:49.847 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:49.847 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-10T09:06:49.847 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T09:06:49.848 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:49.848 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-09-10T09:06:49.848 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 0, 2024-09-10T09:06:49.848 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-09-10T09:06:49.848 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:49.848 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-09-10T09:06:49.848 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-09-10T09:06:49.848 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-09-10T09:06:49.848 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T09:06:49.848 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-09-10T09:06:49.848 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T09:06:49.848 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-09-10T09:06:49.848 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T09:06:49.849 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T09:06:49.849 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T09:06:49.849 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T09:06:49.849 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T09:06:49.849 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-09-10T09:06:49.849 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T09:06:49.849 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T09:06:49.849 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-09-10T09:06:49.849 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-09-10T09:06:49.849 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-09-10T09:06:49.849 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-09-10T09:06:49.849 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T09:06:49.849 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T09:06:49.850 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T09:06:49.850 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T09:06:49.850 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T09:06:49.850 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-09-10T09:06:49.850 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-09-10T09:06:49.850 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-09-10T09:06:49.850 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T09:06:49.850 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T09:06:49.850 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-09-10T09:06:49.850 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-09-10T09:06:49.850 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-09-10T09:06:49.850 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-09-10T09:06:49.850 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T09:06:49.851 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-09-10T09:06:49.851 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T09:06:49.851 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-09-10T09:06:49.851 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 148744, 2024-09-10T09:06:49.851 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 30, 2024-09-10T09:06:49.851 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-09-10T09:06:49.851 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-09-10T09:06:49.851 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-09-10T09:06:49.851 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-09-10T09:06:49.851 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-09-10T09:06:49.851 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-09-10T09:06:49.851 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-09-10T09:06:49.852 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-09-10T09:06:49.852 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 0, 2024-09-10T09:06:49.852 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 0, 2024-09-10T09:06:49.852 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T09:06:49.852 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-09-10T09:06:49.852 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-09-10T09:06:49.852 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-09-10T09:06:49.852 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 928, 2024-09-10T09:06:49.852 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 2, 2024-09-10T09:06:49.852 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-09-10T09:06:49.852 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-09-10T09:06:49.852 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-09-10T09:06:49.852 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-09-10T09:06:49.852 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-10T09:06:49.853 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T09:06:49.853 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-10T09:06:49.853 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-10T09:06:49.853 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-10T09:06:49.853 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 375, 2024-09-10T09:06:49.853 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 469442, 2024-09-10T09:06:49.853 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:49.853 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 375, 2024-09-10T09:06:49.853 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-10T09:06:49.853 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-10T09:06:49.853 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 375, 2024-09-10T09:06:49.853 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 469442, 2024-09-10T09:06:49.855 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-10T09:06:49.855 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-10T09:06:49.855 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-10T09:06:49.855 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-10T09:06:49.855 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:49.855 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-10T09:06:49.855 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-10T09:06:50.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:50 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:06:50.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:50 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:06:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:50 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:06:50.528 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-10T09:06:50.528 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-10T09:06:50.528 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-10T09:06:50.528 DEBUG:tasks.check_counter:Getting stats from b 2024-09-10T09:06:50.528 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-10T09:06:51.216 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:06:51.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:51 smithi174 ceph-mon[34068]: pgmap v968: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T09:06:51.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:51 smithi191 ceph-mon[35314]: pgmap v968: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T09:06:51.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:51 smithi172 ceph-mon[30854]: pgmap v968: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T09:06:51.839 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-10T09:06:51.840 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T09:06:51.840 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 4694, 2024-09-10T09:06:51.840 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 4271, 2024-09-10T09:06:51.840 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 1684811, 2024-09-10T09:06:51.840 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 10415886, 2024-09-10T09:06:51.840 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 18, 2024-09-10T09:06:51.840 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-09-10T09:06:51.840 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.395010152, 2024-09-10T09:06:51.840 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.153939211, 2024-09-10T09:06:51.840 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.182477247, 2024-09-10T09:06:51.840 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.056056505, 2024-09-10T09:06:51.840 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:51.841 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4275, 2024-09-10T09:06:51.841 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.536912965, 2024-09-10T09:06:51.841 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000125593 2024-09-10T09:06:51.841 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.841 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:51.841 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4421, 2024-09-10T09:06:51.841 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000905241, 2024-09-10T09:06:51.841 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000204 2024-09-10T09:06:51.841 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.841 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T09:06:51.841 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T09:06:51.841 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.841 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T09:06:51.842 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 4650, 2024-09-10T09:06:51.842 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 3876, 2024-09-10T09:06:51.842 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 2189588, 2024-09-10T09:06:51.842 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 2035474, 2024-09-10T09:06:51.842 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 11, 2024-09-10T09:06:51.842 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 6, 2024-09-10T09:06:51.842 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.359220523, 2024-09-10T09:06:51.842 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.152583368, 2024-09-10T09:06:51.842 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.201518636, 2024-09-10T09:06:51.842 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.041559082, 2024-09-10T09:06:51.842 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:51.842 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3876, 2024-09-10T09:06:51.842 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2.020059356, 2024-09-10T09:06:51.843 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000521171 2024-09-10T09:06:51.843 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.843 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:51.843 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4098, 2024-09-10T09:06:51.843 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000801669, 2024-09-10T09:06:51.843 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000195 2024-09-10T09:06:51.843 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.843 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 738064, 2024-09-10T09:06:51.843 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 256624 2024-09-10T09:06:51.843 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.843 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T09:06:51.843 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 10509, 2024-09-10T09:06:51.843 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 11678, 2024-09-10T09:06:51.843 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 3758699, 2024-09-10T09:06:51.844 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 12015449, 2024-09-10T09:06:51.844 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 387, 2024-09-10T09:06:51.844 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 9, 2024-09-10T09:06:51.844 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.877000641, 2024-09-10T09:06:51.844 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.281772302, 2024-09-10T09:06:51.844 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.518835637, 2024-09-10T09:06:51.844 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.120638139, 2024-09-10T09:06:51.844 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:51.844 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 11670, 2024-09-10T09:06:51.844 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1.352626904, 2024-09-10T09:06:51.844 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000115906 2024-09-10T09:06:51.844 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.844 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:51.845 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 17833, 2024-09-10T09:06:51.845 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.002752402, 2024-09-10T09:06:51.845 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000154 2024-09-10T09:06:51.845 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.845 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 53680, 2024-09-10T09:06:51.845 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 613024 2024-09-10T09:06:51.845 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.845 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-10T09:06:51.845 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-10T09:06:51.845 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-10T09:06:51.845 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.845 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-10T09:06:51.845 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-10T09:06:51.846 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-10T09:06:51.846 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3030, 2024-09-10T09:06:51.846 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1.845541195, 2024-09-10T09:06:51.846 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000609089 2024-09-10T09:06:51.846 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:51.846 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.846 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-10T09:06:51.846 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-10T09:06:51.846 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-10T09:06:51.846 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1633, 2024-09-10T09:06:51.846 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.082805609, 2024-09-10T09:06:51.846 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000050707 2024-09-10T09:06:51.846 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:51.846 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.847 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-10T09:06:51.847 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 2882, 2024-09-10T09:06:51.847 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 2812, 2024-09-10T09:06:51.847 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-10T09:06:51.847 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2812, 2024-09-10T09:06:51.847 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 96.006025855, 2024-09-10T09:06:51.847 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.034141545 2024-09-10T09:06:51.847 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.847 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-10T09:06:51.847 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 70, 2024-09-10T09:06:51.847 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 9, 2024-09-10T09:06:51.847 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 4, 2024-09-10T09:06:51.847 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 381, 2024-09-10T09:06:51.848 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-09-10T09:06:51.848 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-10T09:06:51.848 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 20, 2024-09-10T09:06:51.848 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 3, 2024-09-10T09:06:51.848 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 7, 2024-09-10T09:06:51.848 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 10, 2024-09-10T09:06:51.848 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 14, 2024-09-10T09:06:51.848 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 415, 2024-09-10T09:06:51.848 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 1, 2024-09-10T09:06:51.848 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 1, 2024-09-10T09:06:51.848 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 5, 2024-09-10T09:06:51.848 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 4402, 2024-09-10T09:06:51.849 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 4108, 2024-09-10T09:06:51.849 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 70, 2024-09-10T09:06:51.849 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 5, 2024-09-10T09:06:51.849 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-10T09:06:51.849 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-10T09:06:51.849 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 55, 2024-09-10T09:06:51.849 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-09-10T09:06:51.849 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-10T09:06:51.849 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 7, 2024-09-10T09:06:51.849 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 343, 2024-09-10T09:06:51.849 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 1, 2024-09-10T09:06:51.849 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 28, 2024-09-10T09:06:51.849 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-10T09:06:51.850 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 621, 2024-09-10T09:06:51.850 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-10T09:06:51.850 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 4, 2024-09-10T09:06:51.850 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 234, 2024-09-10T09:06:51.850 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-09-10T09:06:51.850 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 621, 2024-09-10T09:06:51.850 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-10T09:06:51.850 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T09:06:51.850 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-10T09:06:51.850 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 208, 2024-09-10T09:06:51.850 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 19, 2024-09-10T09:06:51.850 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 189, 2024-09-10T09:06:51.850 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 413, 2024-09-10T09:06:51.851 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 3, 2024-09-10T09:06:51.851 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 1152, 2024-09-10T09:06:51.851 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 1435, 2024-09-10T09:06:51.851 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 6, 2024-09-10T09:06:51.851 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T09:06:51.851 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T09:06:51.851 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 2989, 2024-09-10T09:06:51.851 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 978, 2024-09-10T09:06:51.851 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 18, 2024-09-10T09:06:51.851 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 2005 2024-09-10T09:06:51.851 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.851 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-10T09:06:51.851 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-10T09:06:51.852 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-10T09:06:51.852 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-10T09:06:51.852 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 8, 2024-09-10T09:06:51.852 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 20, 2024-09-10T09:06:51.852 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-10T09:06:51.852 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-10T09:06:51.852 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-10T09:06:51.852 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 558, 2024-09-10T09:06:51.852 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 497, 2024-09-10T09:06:51.852 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 61, 2024-09-10T09:06:51.852 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-10T09:06:51.852 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-10T09:06:51.853 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-10T09:06:51.853 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-10T09:06:51.853 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-10T09:06:51.853 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-10T09:06:51.853 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 15, 2024-09-10T09:06:51.853 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 378, 2024-09-10T09:06:51.853 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 19, 2024-09-10T09:06:51.853 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 7, 2024-09-10T09:06:51.853 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-10T09:06:51.853 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-09-10T09:06:51.853 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-10T09:06:51.853 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-10T09:06:51.853 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.854 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-10T09:06:51.854 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-10T09:06:51.854 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 4065, 2024-09-10T09:06:51.854 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 4060, 2024-09-10T09:06:51.854 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 4060, 2024-09-10T09:06:51.854 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 5, 2024-09-10T09:06:51.854 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-10T09:06:51.854 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 64, 2024-09-10T09:06:51.854 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 87, 2024-09-10T09:06:51.854 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 86, 2024-09-10T09:06:51.854 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 86, 2024-09-10T09:06:51.854 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 1, 2024-09-10T09:06:51.854 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 1, 2024-09-10T09:06:51.854 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-10T09:06:51.854 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 1, 2024-09-10T09:06:51.855 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 12752163, 2024-09-10T09:06:51.855 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 12767963, 2024-09-10T09:06:51.855 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-09-10T09:06:51.855 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-10T09:06:51.855 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 875, 2024-09-10T09:06:51.855 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 6.050523330, 2024-09-10T09:06:51.855 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.006914883 2024-09-10T09:06:51.855 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.855 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-09-10T09:06:51.855 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.855 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-10T09:06:51.855 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 25, 2024-09-10T09:06:51.855 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 838, 2024-09-10T09:06:51.856 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 813, 2024-09-10T09:06:51.856 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 22, 2024-09-10T09:06:51.856 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 130, 2024-09-10T09:06:51.856 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 108, 2024-09-10T09:06:51.856 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 20, 2024-09-10T09:06:51.856 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 1098, 2024-09-10T09:06:51.856 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 1078, 2024-09-10T09:06:51.856 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 1, 2024-09-10T09:06:51.856 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 795, 2024-09-10T09:06:51.856 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 794, 2024-09-10T09:06:51.856 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 64748, 2024-09-10T09:06:51.856 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 231712 2024-09-10T09:06:51.856 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.857 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-10T09:06:51.857 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 3467, 2024-09-10T09:06:51.857 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 6, 2024-09-10T09:06:51.857 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 2882, 2024-09-10T09:06:51.857 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 31, 2024-09-10T09:06:51.857 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 49, 2024-09-10T09:06:51.857 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-10T09:06:51.857 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 643, 2024-09-10T09:06:51.857 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.629534343, 2024-09-10T09:06:51.857 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000979058 2024-09-10T09:06:51.857 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.857 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-10T09:06:51.857 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 737, 2024-09-10T09:06:51.858 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.178001412, 2024-09-10T09:06:51.858 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000241521 2024-09-10T09:06:51.858 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.858 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-10T09:06:51.858 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.858 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.858 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.858 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.858 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-10T09:06:51.858 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 61, 2024-09-10T09:06:51.858 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.141581071, 2024-09-10T09:06:51.858 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002321001 2024-09-10T09:06:51.858 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.859 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-10T09:06:51.859 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 241, 2024-09-10T09:06:51.859 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 78.097262761, 2024-09-10T09:06:51.859 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.324055032 2024-09-10T09:06:51.859 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.859 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-10T09:06:51.859 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.859 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.859 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.859 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.859 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-10T09:06:51.859 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.859 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.859 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.860 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.860 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-10T09:06:51.860 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.860 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.860 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.860 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.860 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-10T09:06:51.860 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.860 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.860 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.860 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.860 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-10T09:06:51.860 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.860 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.861 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.861 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.861 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-10T09:06:51.861 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.861 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.861 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.861 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.861 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-10T09:06:51.861 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 105, 2024-09-10T09:06:51.861 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 13.045413859, 2024-09-10T09:06:51.861 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.124242036 2024-09-10T09:06:51.861 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.862 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-10T09:06:51.862 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.862 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.862 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.862 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.862 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-10T09:06:51.862 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.862 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.862 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.862 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.862 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-10T09:06:51.862 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 197, 2024-09-10T09:06:51.863 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.126812496, 2024-09-10T09:06:51.863 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000643718 2024-09-10T09:06:51.863 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.863 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-10T09:06:51.863 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 160, 2024-09-10T09:06:51.863 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.523038179, 2024-09-10T09:06:51.863 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003268988 2024-09-10T09:06:51.863 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.863 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-10T09:06:51.863 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 113, 2024-09-10T09:06:51.863 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2.438174842, 2024-09-10T09:06:51.863 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.021576768 2024-09-10T09:06:51.863 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.864 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-10T09:06:51.864 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.864 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.864 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.864 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.864 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-10T09:06:51.864 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.864 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.864 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.864 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.864 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-10T09:06:51.864 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 16, 2024-09-10T09:06:51.864 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.102783424, 2024-09-10T09:06:51.865 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.006423964 2024-09-10T09:06:51.865 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.865 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-10T09:06:51.865 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.865 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.865 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.865 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.865 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-10T09:06:51.865 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.865 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.865 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.866 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.866 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-10T09:06:51.866 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5, 2024-09-10T09:06:51.866 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.004803057, 2024-09-10T09:06:51.866 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000960611 2024-09-10T09:06:51.866 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.866 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-10T09:06:51.866 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.866 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.866 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.866 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.866 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-10T09:06:51.866 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4, 2024-09-10T09:06:51.867 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.001596872, 2024-09-10T09:06:51.867 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000399218 2024-09-10T09:06:51.867 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.867 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-10T09:06:51.867 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.867 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.867 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.867 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.867 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-10T09:06:51.867 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 8, 2024-09-10T09:06:51.867 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.020062190, 2024-09-10T09:06:51.867 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002507773 2024-09-10T09:06:51.867 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.867 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-10T09:06:51.868 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 13, 2024-09-10T09:06:51.868 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.009269153, 2024-09-10T09:06:51.868 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000713011 2024-09-10T09:06:51.868 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.868 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-10T09:06:51.868 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 509, 2024-09-10T09:06:51.868 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.687692196, 2024-09-10T09:06:51.868 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001351065 2024-09-10T09:06:51.868 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.868 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-10T09:06:51.868 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-10T09:06:51.868 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-10T09:06:51.868 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.869 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.869 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.869 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:51.869 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.869 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-10T09:06:51.869 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-09-10T09:06:51.869 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 3, 2024-09-10T09:06:51.869 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 2, 2024-09-10T09:06:51.869 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2024-09-10T09:06:51.869 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-10T09:06:51.869 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 71, 2024-09-10T09:06:51.869 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 71, 2024-09-10T09:06:51.870 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 1217, 2024-09-10T09:06:51.870 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-10T09:06:51.870 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.870 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-10T09:06:51.870 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 120, 2024-09-10T09:06:51.870 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 120, 2024-09-10T09:06:51.870 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T09:06:51.870 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-10T09:06:51.870 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T09:06:51.870 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-10T09:06:51.870 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T09:06:51.870 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T09:06:51.870 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T09:06:51.870 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T09:06:51.871 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T09:06:51.871 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-10T09:06:51.871 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T09:06:51.871 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T09:06:51.871 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-10T09:06:51.871 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-10T09:06:51.871 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-10T09:06:51.871 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-10T09:06:51.871 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T09:06:51.871 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T09:06:51.871 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T09:06:51.871 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T09:06:51.871 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T09:06:51.871 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-10T09:06:51.872 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-10T09:06:51.872 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-10T09:06:51.872 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T09:06:51.872 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T09:06:51.872 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-10T09:06:51.872 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-10T09:06:51.872 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-10T09:06:51.872 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-10T09:06:51.872 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T09:06:51.872 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-10T09:06:51.872 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T09:06:51.872 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-10T09:06:51.872 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 302647, 2024-09-10T09:06:51.872 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 104, 2024-09-10T09:06:51.872 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-10T09:06:51.873 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-10T09:06:51.873 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-10T09:06:51.873 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-10T09:06:51.873 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-10T09:06:51.873 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-10T09:06:51.873 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-10T09:06:51.873 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-10T09:06:51.873 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8672, 2024-09-10T09:06:51.873 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 163, 2024-09-10T09:06:51.873 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T09:06:51.873 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-10T09:06:51.873 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-10T09:06:51.873 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-10T09:06:51.874 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 101954, 2024-09-10T09:06:51.874 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 620, 2024-09-10T09:06:51.874 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-10T09:06:51.874 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-10T09:06:51.874 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-10T09:06:51.874 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-10T09:06:51.874 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.874 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-10T09:06:51.874 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-10T09:06:51.874 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-10T09:06:51.874 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 5331, 2024-09-10T09:06:51.874 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 9587501, 2024-09-10T09:06:51.874 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 33, 2024-09-10T09:06:51.874 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 5331, 2024-09-10T09:06:51.875 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-10T09:06:51.875 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5331, 2024-09-10T09:06:51.875 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 415.825917947, 2024-09-10T09:06:51.875 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.078001485 2024-09-10T09:06:51.875 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.875 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-10T09:06:51.875 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-10T09:06:51.875 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5331, 2024-09-10T09:06:51.875 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 7335 2024-09-10T09:06:51.875 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.875 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 5331, 2024-09-10T09:06:51.875 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 705, 2024-09-10T09:06:51.875 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 4626, 2024-09-10T09:06:51.875 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-10T09:06:51.876 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-10T09:06:51.876 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 280, 2024-09-10T09:06:51.876 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 727, 2024-09-10T09:06:51.876 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 52, 2024-09-10T09:06:51.876 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 908, 2024-09-10T09:06:51.876 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 268, 2024-09-10T09:06:51.876 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-10T09:06:51.876 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-10T09:06:51.876 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-10T09:06:51.876 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-09-10T09:06:51.876 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 2153, 2024-09-10T09:06:51.876 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-10T09:06:51.876 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-10T09:06:51.876 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-10T09:06:51.877 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 653, 2024-09-10T09:06:51.877 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 1155, 2024-09-10T09:06:51.877 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-10T09:06:51.877 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-10T09:06:51.877 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-10T09:06:51.877 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-10T09:06:51.877 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-10T09:06:51.877 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-10T09:06:51.877 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T09:06:51.877 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-10T09:06:51.877 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-10T09:06:51.877 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 6, 2024-09-10T09:06:51.877 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-10T09:06:51.877 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-10T09:06:51.878 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-10T09:06:51.878 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-10T09:06:51.878 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-10T09:06:51.878 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-10T09:06:51.878 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-10T09:06:51.878 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-10T09:06:51.878 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-10T09:06:51.878 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-10T09:06:51.878 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-10T09:06:51.878 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-10T09:06:51.878 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-10T09:06:51.878 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-10T09:06:51.878 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-10T09:06:51.878 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-10T09:06:51.879 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 92, 2024-09-10T09:06:51.879 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-10T09:06:51.879 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 10, 2024-09-10T09:06:51.879 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 12, 2024-09-10T09:06:51.879 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2024-09-10T09:06:51.879 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-09-10T09:06:51.879 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-10T09:06:51.879 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 978, 2024-09-10T09:06:51.879 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 26, 2024-09-10T09:06:51.879 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 129 2024-09-10T09:06:51.879 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.879 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-10T09:06:51.879 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-09-10T09:06:51.879 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-10T09:06:51.879 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-10T09:06:51.880 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-10T09:06:51.880 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.880 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-10T09:06:51.880 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 382 2024-09-10T09:06:51.880 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.880 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-10T09:06:51.880 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-10T09:06:51.880 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 26, 2024-09-10T09:06:51.880 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-10T09:06:51.880 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 7, 2024-09-10T09:06:51.880 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 1637, 2024-09-10T09:06:51.880 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 497, 2024-09-10T09:06:51.881 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-10T09:06:51.881 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.881 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T09:06:51.881 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:51.881 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-10T09:06:51.881 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:51.881 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 19849, 2024-09-10T09:06:51.881 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 5813583, 2024-09-10T09:06:51.881 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:51.881 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 19849, 2024-09-10T09:06:51.881 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T09:06:51.881 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T09:06:51.881 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 19849, 2024-09-10T09:06:51.882 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 5813583, 2024-09-10T09:06:51.882 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:51.882 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.882 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.882 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.882 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:51.882 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.882 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-10T09:06:51.882 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:51.882 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-10T09:06:51.882 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:51.882 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T09:06:51.882 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T09:06:51.882 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:51.882 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:51.883 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 5331, 2024-09-10T09:06:51.883 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 9641282, 2024-09-10T09:06:51.883 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 5331, 2024-09-10T09:06:51.883 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 9641282, 2024-09-10T09:06:51.883 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:51.883 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.883 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.883 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.883 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:51.883 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.883 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-10T09:06:51.883 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:51.883 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-10T09:06:51.883 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:51.884 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T09:06:51.884 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T09:06:51.884 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:51.884 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:51.884 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 5331, 2024-09-10T09:06:51.884 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 5331, 2024-09-10T09:06:51.884 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 5331, 2024-09-10T09:06:51.884 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 5331, 2024-09-10T09:06:51.884 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:51.884 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.884 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.884 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.884 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:51.884 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.885 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-10T09:06:51.885 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:51.885 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-10T09:06:51.885 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:51.885 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 497, 2024-09-10T09:06:51.885 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 53941, 2024-09-10T09:06:51.885 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:51.885 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 497, 2024-09-10T09:06:51.885 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T09:06:51.885 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T09:06:51.885 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 33, 2024-09-10T09:06:51.885 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 53941, 2024-09-10T09:06:51.885 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:51.885 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.886 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.886 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.886 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:51.886 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:51.886 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55bdaf698960": { 2024-09-10T09:06:51.886 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:51.886 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-10T09:06:51.886 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:51.886 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 4065, 2024-09-10T09:06:51.886 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 8573659, 2024-09-10T09:06:51.886 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:51.886 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 4065, 2024-09-10T09:06:51.886 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T09:06:51.886 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T09:06:51.887 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 875, 2024-09-10T09:06:51.887 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 8573659, 2024-09-10T09:06:51.887 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:51.887 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:51.887 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:51.887 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:51.887 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:51.887 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:51.887 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-10T09:06:52.744 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:52 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:52.745 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:52 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:52.745 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:52 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:52.745 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:52 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:52.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:52 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:52.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:52 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:52.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:52 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:52.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:52 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:52.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:52 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:52.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:52 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:52.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:52 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:52.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:52 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:52.870 INFO:tasks.check_counter:Daemon mds.b mds.exported=7 2024-09-10T09:06:52.870 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-09-10T09:06:52.870 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-09-10T09:06:52.870 DEBUG:tasks.check_counter:Getting stats from e 2024-09-10T09:06:52.871 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-10T09:06:53.146 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:06:53.558 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:53 smithi174 ceph-mon[34068]: pgmap v969: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:06:53.558 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:53 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:53.558 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:53 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:53.663 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-10T09:06:53.664 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T09:06:53.664 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 3985, 2024-09-10T09:06:53.664 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 3987, 2024-09-10T09:06:53.664 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 2508195, 2024-09-10T09:06:53.664 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 1267237, 2024-09-10T09:06:53.664 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 6, 2024-09-10T09:06:53.664 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 2, 2024-09-10T09:06:53.664 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.301354673, 2024-09-10T09:06:53.664 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.090273496, 2024-09-10T09:06:53.664 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.076082203, 2024-09-10T09:06:53.664 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.113623337, 2024-09-10T09:06:53.665 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:53.665 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3985, 2024-09-10T09:06:53.665 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.116985068, 2024-09-10T09:06:53.665 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000029356 2024-09-10T09:06:53.665 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.665 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:53.665 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.665 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.665 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.665 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.665 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T09:06:53.665 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T09:06:53.665 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.666 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T09:06:53.666 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 413, 2024-09-10T09:06:53.666 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 382, 2024-09-10T09:06:53.666 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 710796, 2024-09-10T09:06:53.666 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 230938, 2024-09-10T09:06:53.666 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 3, 2024-09-10T09:06:53.666 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 2, 2024-09-10T09:06:53.666 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.053996565, 2024-09-10T09:06:53.666 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.024850248, 2024-09-10T09:06:53.666 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.036200389, 2024-09-10T09:06:53.666 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000076210, 2024-09-10T09:06:53.666 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:53.667 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 382, 2024-09-10T09:06:53.667 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.028762140, 2024-09-10T09:06:53.667 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000075293 2024-09-10T09:06:53.667 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.667 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:53.667 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.667 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.667 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.667 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.667 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 710240, 2024-09-10T09:06:53.667 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 230240 2024-09-10T09:06:53.667 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.667 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T09:06:53.668 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 3, 2024-09-10T09:06:53.668 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 284, 2024-09-10T09:06:53.668 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 588, 2024-09-10T09:06:53.668 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 652526, 2024-09-10T09:06:53.668 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 5, 2024-09-10T09:06:53.668 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-09-10T09:06:53.668 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.014955675, 2024-09-10T09:06:53.668 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.010724118, 2024-09-10T09:06:53.668 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.001757087, 2024-09-10T09:06:53.668 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000030523, 2024-09-10T09:06:53.668 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:53.668 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 282, 2024-09-10T09:06:53.668 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.016287713, 2024-09-10T09:06:53.669 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000057757 2024-09-10T09:06:53.669 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.669 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:53.669 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.669 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.669 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.669 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.669 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T09:06:53.669 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 652208 2024-09-10T09:06:53.669 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.669 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-10T09:06:53.669 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-10T09:06:53.670 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-10T09:06:53.670 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.670 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-10T09:06:53.670 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-10T09:06:53.670 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-10T09:06:53.670 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 6693, 2024-09-10T09:06:53.670 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.154559654, 2024-09-10T09:06:53.670 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000023092 2024-09-10T09:06:53.670 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:53.670 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.670 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-10T09:06:53.670 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-10T09:06:53.670 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-10T09:06:53.671 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.671 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.671 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.671 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:53.671 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.671 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-10T09:06:53.671 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 0, 2024-09-10T09:06:53.671 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 0, 2024-09-10T09:06:53.671 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-10T09:06:53.671 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.671 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.671 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.671 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.671 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-10T09:06:53.672 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2024-09-10T09:06:53.672 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-10T09:06:53.672 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-10T09:06:53.672 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 0, 2024-09-10T09:06:53.672 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-09-10T09:06:53.672 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-10T09:06:53.672 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 207, 2024-09-10T09:06:53.672 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 0, 2024-09-10T09:06:53.672 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 207, 2024-09-10T09:06:53.672 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-09-10T09:06:53.672 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 207, 2024-09-10T09:06:53.672 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 0, 2024-09-10T09:06:53.673 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-09-10T09:06:53.673 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-09-10T09:06:53.673 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 1, 2024-09-10T09:06:53.673 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 0, 2024-09-10T09:06:53.673 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 0, 2024-09-10T09:06:53.673 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-09-10T09:06:53.673 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 0, 2024-09-10T09:06:53.673 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-10T09:06:53.673 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-10T09:06:53.673 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-09-10T09:06:53.673 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-09-10T09:06:53.673 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-10T09:06:53.673 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-09-10T09:06:53.674 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-09-10T09:06:53.674 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-09-10T09:06:53.674 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-09-10T09:06:53.674 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-10T09:06:53.674 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-09-10T09:06:53.674 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-10T09:06:53.674 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 0, 2024-09-10T09:06:53.674 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 0, 2024-09-10T09:06:53.674 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-09-10T09:06:53.674 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T09:06:53.674 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-10T09:06:53.675 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T09:06:53.675 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-10T09:06:53.675 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-09-10T09:06:53.675 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T09:06:53.675 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T09:06:53.675 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-09-10T09:06:53.675 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-09-10T09:06:53.675 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T09:06:53.675 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-09-10T09:06:53.675 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T09:06:53.675 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T09:06:53.675 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T09:06:53.675 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-09-10T09:06:53.676 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-09-10T09:06:53.676 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-09-10T09:06:53.676 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-09-10T09:06:53.676 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.676 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-10T09:06:53.676 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-10T09:06:53.676 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-10T09:06:53.676 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-10T09:06:53.676 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 0, 2024-09-10T09:06:53.676 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-09-10T09:06:53.676 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-10T09:06:53.676 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-10T09:06:53.676 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-10T09:06:53.677 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-09-10T09:06:53.677 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-09-10T09:06:53.677 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-10T09:06:53.677 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-10T09:06:53.677 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-10T09:06:53.677 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-10T09:06:53.677 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-10T09:06:53.677 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-10T09:06:53.677 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-10T09:06:53.677 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 0, 2024-09-10T09:06:53.677 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 0, 2024-09-10T09:06:53.677 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T09:06:53.677 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-09-10T09:06:53.677 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-10T09:06:53.678 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-09-10T09:06:53.678 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-10T09:06:53.678 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-10T09:06:53.678 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.678 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-10T09:06:53.678 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-10T09:06:53.678 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 0, 2024-09-10T09:06:53.678 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-10T09:06:53.678 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-10T09:06:53.678 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 864, 2024-09-10T09:06:53.678 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-10T09:06:53.678 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-10T09:06:53.678 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 0, 2024-09-10T09:06:53.678 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-10T09:06:53.678 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-10T09:06:53.679 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-09-10T09:06:53.679 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-09-10T09:06:53.679 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-10T09:06:53.679 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-10T09:06:53.679 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-09-10T09:06:53.679 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 5581964, 2024-09-10T09:06:53.679 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 5580313, 2024-09-10T09:06:53.679 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-10T09:06:53.679 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.679 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.679 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.679 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.679 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 864 2024-09-10T09:06:53.679 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.680 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-10T09:06:53.680 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 4, 2024-09-10T09:06:53.680 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 226, 2024-09-10T09:06:53.680 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 222, 2024-09-10T09:06:53.680 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 2, 2024-09-10T09:06:53.680 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 93, 2024-09-10T09:06:53.680 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 91, 2024-09-10T09:06:53.680 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 207, 2024-09-10T09:06:53.680 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 489, 2024-09-10T09:06:53.680 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 282, 2024-09-10T09:06:53.680 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-09-10T09:06:53.680 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 0, 2024-09-10T09:06:53.680 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 0, 2024-09-10T09:06:53.680 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 51304, 2024-09-10T09:06:53.681 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 215328 2024-09-10T09:06:53.681 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.681 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-10T09:06:53.681 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 0, 2024-09-10T09:06:53.681 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-09-10T09:06:53.681 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 0, 2024-09-10T09:06:53.681 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 0, 2024-09-10T09:06:53.681 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-09-10T09:06:53.681 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-10T09:06:53.681 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.681 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.681 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.681 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.681 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-10T09:06:53.682 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.682 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.682 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.682 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.682 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-10T09:06:53.682 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.682 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.682 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.682 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.682 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-10T09:06:53.682 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.682 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.682 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.683 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.683 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-10T09:06:53.683 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.683 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.683 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.683 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.683 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-10T09:06:53.683 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.683 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.683 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.683 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.683 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-10T09:06:53.683 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.684 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.684 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.684 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.684 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-10T09:06:53.684 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.684 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.684 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.684 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.684 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-10T09:06:53.684 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.684 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.684 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.684 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.684 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-10T09:06:53.685 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.685 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.685 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.685 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.685 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-10T09:06:53.685 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.685 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.685 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.685 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.685 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-10T09:06:53.685 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.686 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.686 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.686 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.686 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-10T09:06:53.686 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.686 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.686 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.686 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.686 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-10T09:06:53.686 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.686 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.686 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.687 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.687 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-10T09:06:53.687 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.687 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.687 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.687 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.687 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-10T09:06:53.687 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.687 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.687 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.687 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.688 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-10T09:06:53.688 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.688 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.688 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.688 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.688 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-10T09:06:53.688 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.688 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.688 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.688 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.688 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-10T09:06:53.688 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.688 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.688 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.689 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.689 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-10T09:06:53.689 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.689 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.689 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.689 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.689 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-10T09:06:53.689 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.689 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.689 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.689 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.689 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-10T09:06:53.689 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.689 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.690 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.690 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.690 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-10T09:06:53.690 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.690 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.690 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.690 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.690 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-10T09:06:53.690 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.690 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.690 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.690 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.690 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-10T09:06:53.691 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.691 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.691 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.691 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.691 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-10T09:06:53.691 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.691 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.691 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.691 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.691 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-10T09:06:53.691 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.691 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.691 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.692 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.692 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-10T09:06:53.692 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.692 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.692 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.692 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.692 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-10T09:06:53.692 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.692 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.692 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.692 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.692 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-10T09:06:53.692 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-10T09:06:53.692 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-10T09:06:53.693 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.693 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.693 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.693 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:53.693 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.693 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-10T09:06:53.693 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-09-10T09:06:53.693 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 1, 2024-09-10T09:06:53.693 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 0, 2024-09-10T09:06:53.693 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2024-09-10T09:06:53.693 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-10T09:06:53.693 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 0, 2024-09-10T09:06:53.693 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 0, 2024-09-10T09:06:53.693 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 2338, 2024-09-10T09:06:53.694 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-10T09:06:53.694 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.694 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-10T09:06:53.694 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-10T09:06:53.694 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-10T09:06:53.694 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T09:06:53.694 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-10T09:06:53.694 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T09:06:53.694 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-10T09:06:53.694 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T09:06:53.694 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T09:06:53.694 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T09:06:53.694 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T09:06:53.694 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T09:06:53.695 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-10T09:06:53.695 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T09:06:53.695 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T09:06:53.695 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-10T09:06:53.695 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-10T09:06:53.695 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-10T09:06:53.695 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-10T09:06:53.695 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T09:06:53.695 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T09:06:53.695 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T09:06:53.695 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T09:06:53.695 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T09:06:53.695 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-10T09:06:53.695 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-10T09:06:53.696 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-10T09:06:53.696 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T09:06:53.696 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T09:06:53.696 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-10T09:06:53.696 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-10T09:06:53.696 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-10T09:06:53.696 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-10T09:06:53.696 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T09:06:53.696 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-10T09:06:53.696 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T09:06:53.696 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-10T09:06:53.696 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 182027, 2024-09-10T09:06:53.697 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 47, 2024-09-10T09:06:53.697 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-10T09:06:53.697 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-10T09:06:53.697 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-10T09:06:53.697 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-10T09:06:53.697 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-10T09:06:53.697 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-10T09:06:53.697 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-10T09:06:53.697 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-10T09:06:53.697 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8672, 2024-09-10T09:06:53.697 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 163, 2024-09-10T09:06:53.697 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T09:06:53.697 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-10T09:06:53.698 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-10T09:06:53.698 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-10T09:06:53.698 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 157776, 2024-09-10T09:06:53.698 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 856, 2024-09-10T09:06:53.698 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-10T09:06:53.698 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-10T09:06:53.698 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-10T09:06:53.698 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-10T09:06:53.698 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.698 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-10T09:06:53.698 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-10T09:06:53.698 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-10T09:06:53.698 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 3988, 2024-09-10T09:06:53.698 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 21, 2024-09-10T09:06:53.698 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 3, 2024-09-10T09:06:53.699 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 3988, 2024-09-10T09:06:53.699 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-10T09:06:53.699 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3988, 2024-09-10T09:06:53.699 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 4.320935794, 2024-09-10T09:06:53.699 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001083484 2024-09-10T09:06:53.699 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.699 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-10T09:06:53.699 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-10T09:06:53.699 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3988, 2024-09-10T09:06:53.699 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 3989 2024-09-10T09:06:53.699 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.699 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 3988, 2024-09-10T09:06:53.699 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 3988, 2024-09-10T09:06:53.700 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 0, 2024-09-10T09:06:53.700 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-10T09:06:53.700 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-10T09:06:53.700 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 2533, 2024-09-10T09:06:53.700 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 0, 2024-09-10T09:06:53.700 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 1453, 2024-09-10T09:06:53.700 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 0, 2024-09-10T09:06:53.700 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 0, 2024-09-10T09:06:53.700 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-10T09:06:53.700 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-10T09:06:53.700 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-10T09:06:53.700 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-09-10T09:06:53.700 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 0, 2024-09-10T09:06:53.701 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-10T09:06:53.701 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-10T09:06:53.701 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-10T09:06:53.701 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 1, 2024-09-10T09:06:53.701 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 0, 2024-09-10T09:06:53.701 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-10T09:06:53.701 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-10T09:06:53.701 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-10T09:06:53.701 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-10T09:06:53.701 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-10T09:06:53.701 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-10T09:06:53.702 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T09:06:53.702 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-10T09:06:53.702 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-10T09:06:53.702 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-10T09:06:53.702 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-10T09:06:53.702 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-10T09:06:53.702 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-10T09:06:53.702 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-10T09:06:53.702 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-10T09:06:53.702 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-10T09:06:53.702 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-10T09:06:53.702 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-10T09:06:53.702 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-10T09:06:53.702 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-10T09:06:53.702 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-10T09:06:53.703 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-10T09:06:53.703 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-10T09:06:53.703 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-10T09:06:53.703 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-10T09:06:53.703 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-10T09:06:53.703 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 92, 2024-09-10T09:06:53.703 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-10T09:06:53.703 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 4, 2024-09-10T09:06:53.703 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 4, 2024-09-10T09:06:53.703 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 4, 2024-09-10T09:06:53.703 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-09-10T09:06:53.703 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-10T09:06:53.703 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 0, 2024-09-10T09:06:53.703 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 2, 2024-09-10T09:06:53.704 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 0 2024-09-10T09:06:53.704 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.704 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-10T09:06:53.704 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 0, 2024-09-10T09:06:53.704 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-10T09:06:53.704 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-10T09:06:53.704 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-10T09:06:53.704 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.704 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-10T09:06:53.704 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-09-10T09:06:53.704 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.704 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-10T09:06:53.704 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-10T09:06:53.704 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T09:06:53.705 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-10T09:06:53.705 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-09-10T09:06:53.705 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-09-10T09:06:53.705 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-09-10T09:06:53.705 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-10T09:06:53.705 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.705 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T09:06:53.705 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:53.705 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-10T09:06:53.705 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:53.705 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 4397, 2024-09-10T09:06:53.705 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 2800040, 2024-09-10T09:06:53.705 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:53.705 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 4397, 2024-09-10T09:06:53.706 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T09:06:53.706 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T09:06:53.706 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 4397, 2024-09-10T09:06:53.706 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 2800040, 2024-09-10T09:06:53.706 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:53.706 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.706 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.706 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.706 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:53.706 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.706 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-10T09:06:53.706 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:53.706 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-10T09:06:53.706 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:53.707 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T09:06:53.707 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T09:06:53.707 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:53.707 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:53.707 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 3988, 2024-09-10T09:06:53.707 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 1387665, 2024-09-10T09:06:53.707 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 3988, 2024-09-10T09:06:53.707 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1387665, 2024-09-10T09:06:53.707 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:53.707 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.707 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.707 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.707 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:53.707 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.708 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-10T09:06:53.708 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:53.708 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-10T09:06:53.708 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:53.708 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T09:06:53.708 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T09:06:53.708 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:53.708 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:53.708 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 3988, 2024-09-10T09:06:53.708 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 3988, 2024-09-10T09:06:53.708 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 3988, 2024-09-10T09:06:53.708 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 3988, 2024-09-10T09:06:53.708 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:53.709 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.709 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.709 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.709 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:53.709 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.709 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-10T09:06:53.709 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:53.709 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-10T09:06:53.709 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:53.709 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T09:06:53.709 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T09:06:53.709 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:53.709 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:53.709 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T09:06:53.710 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T09:06:53.710 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-10T09:06:53.710 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-10T09:06:53.710 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:53.710 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.710 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.710 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.710 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:53.710 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:53.710 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55d07c256b40": { 2024-09-10T09:06:53.710 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:53.710 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-10T09:06:53.711 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:53.711 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T09:06:53.711 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T09:06:53.711 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:53.711 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:53.711 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T09:06:53.711 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T09:06:53.711 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-10T09:06:53.711 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-10T09:06:53.711 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:53.711 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:53.711 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:53.711 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:53.711 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:53.711 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:53.712 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-10T09:06:53.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:53 smithi191 ceph-mon[35314]: pgmap v969: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:06:53.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:53 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:53.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:53 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:53.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:53 smithi172 ceph-mon[30854]: pgmap v969: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:06:53.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:53 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:53.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:53 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:54.691 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:54 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:06:54.691 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:54 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:06:54.691 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:54 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:54.725 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-10T09:06:54.725 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-10T09:06:54.725 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-10T09:06:54.726 DEBUG:tasks.check_counter:Getting stats from h 2024-09-10T09:06:54.726 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-10T09:06:54.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:54 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:06:54.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:54 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:06:54.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:54 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:54.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:54 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:06:54.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:54 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:06:54.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:54 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' 2024-09-10T09:06:54.990 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:06:55.646 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-10T09:06:55.646 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T09:06:55.646 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 4228, 2024-09-10T09:06:55.647 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 4794, 2024-09-10T09:06:55.647 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 2342029, 2024-09-10T09:06:55.647 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 3526420, 2024-09-10T09:06:55.647 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 15, 2024-09-10T09:06:55.647 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-09-10T09:06:55.647 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.306491120, 2024-09-10T09:06:55.647 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.110789305, 2024-09-10T09:06:55.647 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.181005243, 2024-09-10T09:06:55.647 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.029705408, 2024-09-10T09:06:55.647 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:55.647 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4794, 2024-09-10T09:06:55.648 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.252192434, 2024-09-10T09:06:55.648 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000052605 2024-09-10T09:06:55.648 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.648 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:55.648 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5013, 2024-09-10T09:06:55.648 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000966367, 2024-09-10T09:06:55.648 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000192 2024-09-10T09:06:55.648 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.648 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 725792, 2024-09-10T09:06:55.648 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 253920 2024-09-10T09:06:55.648 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.648 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T09:06:55.649 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 3384, 2024-09-10T09:06:55.649 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 3369, 2024-09-10T09:06:55.649 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 1608758, 2024-09-10T09:06:55.649 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 1924240, 2024-09-10T09:06:55.649 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 10, 2024-09-10T09:06:55.649 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 6, 2024-09-10T09:06:55.649 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.285770642, 2024-09-10T09:06:55.649 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.130210792, 2024-09-10T09:06:55.649 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.144960061, 2024-09-10T09:06:55.649 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.022811767, 2024-09-10T09:06:55.649 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:55.650 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3369, 2024-09-10T09:06:55.650 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.186009361, 2024-09-10T09:06:55.650 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000055212 2024-09-10T09:06:55.650 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.650 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:55.650 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4674, 2024-09-10T09:06:55.650 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000910662, 2024-09-10T09:06:55.650 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000194 2024-09-10T09:06:55.650 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.650 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T09:06:55.650 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 655440 2024-09-10T09:06:55.650 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.651 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T09:06:55.651 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 2210, 2024-09-10T09:06:55.651 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 2214, 2024-09-10T09:06:55.651 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 1087300, 2024-09-10T09:06:55.651 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 1263269, 2024-09-10T09:06:55.651 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 50, 2024-09-10T09:06:55.651 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 6, 2024-09-10T09:06:55.651 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.153175489, 2024-09-10T09:06:55.651 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.041629697, 2024-09-10T09:06:55.651 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.050495582, 2024-09-10T09:06:55.651 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.046097881, 2024-09-10T09:06:55.651 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:55.652 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2210, 2024-09-10T09:06:55.652 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.123732687, 2024-09-10T09:06:55.652 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000055987 2024-09-10T09:06:55.652 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.652 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:55.652 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 40, 2024-09-10T09:06:55.652 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000003378, 2024-09-10T09:06:55.652 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000084 2024-09-10T09:06:55.652 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.652 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T09:06:55.652 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T09:06:55.652 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.653 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-10T09:06:55.653 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-10T09:06:55.653 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-10T09:06:55.653 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.653 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-10T09:06:55.653 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-10T09:06:55.653 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-10T09:06:55.653 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2535, 2024-09-10T09:06:55.653 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.800370983, 2024-09-10T09:06:55.653 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000315728 2024-09-10T09:06:55.653 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:55.654 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.654 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-10T09:06:55.654 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-10T09:06:55.654 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-10T09:06:55.654 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 978, 2024-09-10T09:06:55.654 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.068443458, 2024-09-10T09:06:55.654 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000069983 2024-09-10T09:06:55.654 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:55.654 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.654 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-10T09:06:55.654 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 1022, 2024-09-10T09:06:55.654 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 884, 2024-09-10T09:06:55.654 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-10T09:06:55.655 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 884, 2024-09-10T09:06:55.655 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 8.186846109, 2024-09-10T09:06:55.655 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.009261138 2024-09-10T09:06:55.655 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.655 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-10T09:06:55.655 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 138, 2024-09-10T09:06:55.655 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-10T09:06:55.655 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-10T09:06:55.655 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 11, 2024-09-10T09:06:55.655 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-09-10T09:06:55.655 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-10T09:06:55.655 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 18, 2024-09-10T09:06:55.656 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 3, 2024-09-10T09:06:55.656 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 15, 2024-09-10T09:06:55.656 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-09-10T09:06:55.656 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 13, 2024-09-10T09:06:55.656 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 28, 2024-09-10T09:06:55.656 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 2, 2024-09-10T09:06:55.656 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 2, 2024-09-10T09:06:55.656 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 7, 2024-09-10T09:06:55.656 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 1650, 2024-09-10T09:06:55.656 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 1384, 2024-09-10T09:06:55.656 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 138, 2024-09-10T09:06:55.656 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 17, 2024-09-10T09:06:55.657 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-10T09:06:55.657 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-10T09:06:55.657 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-09-10T09:06:55.657 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-09-10T09:06:55.657 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-10T09:06:55.657 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 5, 2024-09-10T09:06:55.657 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-09-10T09:06:55.657 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 5, 2024-09-10T09:06:55.657 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 261, 2024-09-10T09:06:55.657 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-10T09:06:55.657 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 2862, 2024-09-10T09:06:55.657 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-10T09:06:55.658 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 325, 2024-09-10T09:06:55.658 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 13424230450, 2024-09-10T09:06:55.658 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-09-10T09:06:55.658 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 2862, 2024-09-10T09:06:55.658 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-10T09:06:55.658 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T09:06:55.658 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-10T09:06:55.658 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 1095, 2024-09-10T09:06:55.658 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T09:06:55.658 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 1095, 2024-09-10T09:06:55.658 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 1767, 2024-09-10T09:06:55.658 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-09-10T09:06:55.658 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 573, 2024-09-10T09:06:55.659 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 388, 2024-09-10T09:06:55.659 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T09:06:55.659 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T09:06:55.659 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T09:06:55.659 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 783, 2024-09-10T09:06:55.659 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 173, 2024-09-10T09:06:55.659 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 4, 2024-09-10T09:06:55.659 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 415 2024-09-10T09:06:55.659 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.659 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-10T09:06:55.659 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-10T09:06:55.659 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-10T09:06:55.660 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-10T09:06:55.660 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 18, 2024-09-10T09:06:55.660 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 3, 2024-09-10T09:06:55.660 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-10T09:06:55.660 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-10T09:06:55.660 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-10T09:06:55.660 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 206, 2024-09-10T09:06:55.660 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 206, 2024-09-10T09:06:55.660 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-10T09:06:55.660 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-10T09:06:55.660 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-10T09:06:55.660 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-10T09:06:55.660 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-10T09:06:55.661 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-10T09:06:55.661 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-10T09:06:55.661 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 15, 2024-09-10T09:06:55.661 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 2720, 2024-09-10T09:06:55.661 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T09:06:55.661 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 5, 2024-09-10T09:06:55.661 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-10T09:06:55.661 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-09-10T09:06:55.661 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-10T09:06:55.661 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-10T09:06:55.661 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.661 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-10T09:06:55.661 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-10T09:06:55.662 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 864, 2024-09-10T09:06:55.662 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-10T09:06:55.662 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-10T09:06:55.662 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 864, 2024-09-10T09:06:55.662 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-10T09:06:55.662 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-10T09:06:55.662 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 2, 2024-09-10T09:06:55.662 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-10T09:06:55.662 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-10T09:06:55.662 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-09-10T09:06:55.662 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-09-10T09:06:55.662 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-10T09:06:55.662 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-10T09:06:55.663 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-09-10T09:06:55.663 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 5581964, 2024-09-10T09:06:55.663 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-09-10T09:06:55.663 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-10T09:06:55.663 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 309, 2024-09-10T09:06:55.663 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2.181693018, 2024-09-10T09:06:55.663 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.007060495 2024-09-10T09:06:55.663 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.663 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-09-10T09:06:55.663 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.664 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-10T09:06:55.664 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 25, 2024-09-10T09:06:55.664 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 245, 2024-09-10T09:06:55.664 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 220, 2024-09-10T09:06:55.664 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 24, 2024-09-10T09:06:55.664 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 113, 2024-09-10T09:06:55.664 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 89, 2024-09-10T09:06:55.664 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 18, 2024-09-10T09:06:55.664 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 516, 2024-09-10T09:06:55.664 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 498, 2024-09-10T09:06:55.664 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 2, 2024-09-10T09:06:55.665 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 218, 2024-09-10T09:06:55.665 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 216, 2024-09-10T09:06:55.665 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 59376, 2024-09-10T09:06:55.665 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 223520 2024-09-10T09:06:55.665 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.665 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-10T09:06:55.665 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 1423, 2024-09-10T09:06:55.665 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 5, 2024-09-10T09:06:55.665 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 1022, 2024-09-10T09:06:55.665 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 15, 2024-09-10T09:06:55.665 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 86, 2024-09-10T09:06:55.665 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-10T09:06:55.665 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3, 2024-09-10T09:06:55.666 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.008269987, 2024-09-10T09:06:55.666 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002756662 2024-09-10T09:06:55.666 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.666 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-10T09:06:55.666 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 348, 2024-09-10T09:06:55.666 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.085726249, 2024-09-10T09:06:55.666 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000246339 2024-09-10T09:06:55.666 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.666 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-10T09:06:55.666 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.666 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.666 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.666 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.667 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-10T09:06:55.667 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.667 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.667 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.667 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.667 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-10T09:06:55.667 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 155, 2024-09-10T09:06:55.667 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 7.073261118, 2024-09-10T09:06:55.667 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.045633942 2024-09-10T09:06:55.667 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.667 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-10T09:06:55.667 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.667 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.667 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.668 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.668 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-10T09:06:55.668 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.668 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.668 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.668 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.668 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-10T09:06:55.668 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.668 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.668 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.668 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.668 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-10T09:06:55.668 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.669 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.669 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.669 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.669 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-10T09:06:55.669 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.669 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.669 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.669 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.669 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-10T09:06:55.669 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.669 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.670 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.670 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.670 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-10T09:06:55.670 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.670 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.670 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.671 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.671 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-10T09:06:55.671 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.671 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.671 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.671 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.671 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-10T09:06:55.671 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.671 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.671 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.671 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.671 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-10T09:06:55.671 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 96, 2024-09-10T09:06:55.672 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.071400426, 2024-09-10T09:06:55.672 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000743754 2024-09-10T09:06:55.672 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.672 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-10T09:06:55.672 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 76, 2024-09-10T09:06:55.672 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.062765472, 2024-09-10T09:06:55.672 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000825861 2024-09-10T09:06:55.672 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.672 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-10T09:06:55.672 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.672 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.672 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.673 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.673 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-10T09:06:55.673 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.673 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.673 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.673 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.673 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-10T09:06:55.673 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.673 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.673 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.673 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.673 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-10T09:06:55.673 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 73, 2024-09-10T09:06:55.674 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.700153052, 2024-09-10T09:06:55.674 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.009591137 2024-09-10T09:06:55.674 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.674 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-10T09:06:55.674 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.674 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.674 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.674 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.674 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-10T09:06:55.674 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.674 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.674 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.675 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.675 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-10T09:06:55.675 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.675 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.675 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.675 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.675 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-10T09:06:55.675 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.675 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.675 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.675 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.675 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-10T09:06:55.675 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.676 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.676 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.676 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.676 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-10T09:06:55.676 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.676 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.676 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.676 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.676 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-10T09:06:55.676 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.676 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.676 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.676 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.677 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-10T09:06:55.677 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.677 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.677 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.677 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.677 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-10T09:06:55.677 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 133, 2024-09-10T09:06:55.677 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.185269805, 2024-09-10T09:06:55.677 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001393006 2024-09-10T09:06:55.677 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.677 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-10T09:06:55.677 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-10T09:06:55.678 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-10T09:06:55.678 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.678 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.678 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.678 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:55.678 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.678 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-10T09:06:55.678 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-09-10T09:06:55.678 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 1, 2024-09-10T09:06:55.678 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 0, 2024-09-10T09:06:55.678 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2024-09-10T09:06:55.678 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-10T09:06:55.678 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 686, 2024-09-10T09:06:55.679 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 686, 2024-09-10T09:06:55.679 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 1220, 2024-09-10T09:06:55.679 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-10T09:06:55.679 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.679 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-10T09:06:55.679 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-10T09:06:55.679 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-10T09:06:55.679 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T09:06:55.679 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-10T09:06:55.679 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T09:06:55.679 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-10T09:06:55.679 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T09:06:55.679 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T09:06:55.680 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T09:06:55.680 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T09:06:55.680 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T09:06:55.680 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-10T09:06:55.680 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T09:06:55.680 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T09:06:55.680 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-10T09:06:55.680 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-10T09:06:55.680 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-10T09:06:55.680 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-10T09:06:55.680 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T09:06:55.680 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T09:06:55.681 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T09:06:55.681 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T09:06:55.681 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T09:06:55.681 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-10T09:06:55.681 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-10T09:06:55.681 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-10T09:06:55.681 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T09:06:55.681 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T09:06:55.681 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-10T09:06:55.681 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-10T09:06:55.681 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-10T09:06:55.681 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-10T09:06:55.681 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T09:06:55.682 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-10T09:06:55.682 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T09:06:55.682 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-10T09:06:55.682 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 303943, 2024-09-10T09:06:55.682 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 104, 2024-09-10T09:06:55.682 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-10T09:06:55.682 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-10T09:06:55.682 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-10T09:06:55.682 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-10T09:06:55.682 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-10T09:06:55.682 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-10T09:06:55.682 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-10T09:06:55.682 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-10T09:06:55.682 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8672, 2024-09-10T09:06:55.683 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 163, 2024-09-10T09:06:55.683 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T09:06:55.683 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-10T09:06:55.683 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-10T09:06:55.683 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-10T09:06:55.683 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 100573, 2024-09-10T09:06:55.683 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 387, 2024-09-10T09:06:55.683 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-10T09:06:55.683 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-10T09:06:55.683 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-10T09:06:55.683 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-10T09:06:55.683 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.683 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-10T09:06:55.684 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-10T09:06:55.684 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-10T09:06:55.684 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 4278, 2024-09-10T09:06:55.684 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 1463684, 2024-09-10T09:06:55.684 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 28, 2024-09-10T09:06:55.684 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 4278, 2024-09-10T09:06:55.684 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-10T09:06:55.684 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4278, 2024-09-10T09:06:55.684 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 675.280278793, 2024-09-10T09:06:55.684 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.157849527 2024-09-10T09:06:55.684 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.684 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-10T09:06:55.685 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-10T09:06:55.685 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4278, 2024-09-10T09:06:55.685 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 4399 2024-09-10T09:06:55.685 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.685 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 4278, 2024-09-10T09:06:55.685 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 2864, 2024-09-10T09:06:55.685 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 1414, 2024-09-10T09:06:55.685 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-10T09:06:55.685 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-10T09:06:55.685 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2024-09-10T09:06:55.685 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2024-09-10T09:06:55.685 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 2, 2024-09-10T09:06:55.685 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 311, 2024-09-10T09:06:55.686 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 38, 2024-09-10T09:06:55.686 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-10T09:06:55.686 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-10T09:06:55.686 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-10T09:06:55.686 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-09-10T09:06:55.686 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 951, 2024-09-10T09:06:55.686 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-10T09:06:55.686 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-10T09:06:55.686 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-10T09:06:55.686 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 2862, 2024-09-10T09:06:55.686 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2024-09-10T09:06:55.686 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-10T09:06:55.686 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-10T09:06:55.686 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-10T09:06:55.687 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-10T09:06:55.687 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-10T09:06:55.687 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-10T09:06:55.687 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T09:06:55.687 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-10T09:06:55.687 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-10T09:06:55.687 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-10T09:06:55.687 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-10T09:06:55.687 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-10T09:06:55.687 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-10T09:06:55.687 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-10T09:06:55.687 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-10T09:06:55.687 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-10T09:06:55.688 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-10T09:06:55.688 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-10T09:06:55.688 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-10T09:06:55.688 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-10T09:06:55.688 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-10T09:06:55.688 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-10T09:06:55.688 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-10T09:06:55.688 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-10T09:06:55.688 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-10T09:06:55.688 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-10T09:06:55.688 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 92, 2024-09-10T09:06:55.688 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-10T09:06:55.688 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 4, 2024-09-10T09:06:55.689 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 12, 2024-09-10T09:06:55.689 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2024-09-10T09:06:55.689 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-09-10T09:06:55.689 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-10T09:06:55.689 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 166, 2024-09-10T09:06:55.689 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 0, 2024-09-10T09:06:55.689 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 39 2024-09-10T09:06:55.689 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.689 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-10T09:06:55.689 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-09-10T09:06:55.689 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-10T09:06:55.689 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-10T09:06:55.689 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-10T09:06:55.690 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.690 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-10T09:06:55.690 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 2710 2024-09-10T09:06:55.690 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.690 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-10T09:06:55.690 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-10T09:06:55.690 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 26, 2024-09-10T09:06:55.690 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-10T09:06:55.690 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 7, 2024-09-10T09:06:55.690 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 939, 2024-09-10T09:06:55.690 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 206, 2024-09-10T09:06:55.690 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-10T09:06:55.690 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.691 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T09:06:55.691 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:55.691 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-10T09:06:55.691 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:55.691 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 9818, 2024-09-10T09:06:55.691 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 4128064, 2024-09-10T09:06:55.691 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:55.691 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 9818, 2024-09-10T09:06:55.691 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T09:06:55.691 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T09:06:55.691 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 9818, 2024-09-10T09:06:55.691 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 4128064, 2024-09-10T09:06:55.691 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:55.691 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.692 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.692 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.692 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:55.692 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.692 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-10T09:06:55.692 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:55.692 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-10T09:06:55.692 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:55.692 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T09:06:55.692 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T09:06:55.692 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:55.692 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:55.693 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 4278, 2024-09-10T09:06:55.693 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 1485554, 2024-09-10T09:06:55.693 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 4278, 2024-09-10T09:06:55.693 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1485554, 2024-09-10T09:06:55.693 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:55.693 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.693 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.693 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.693 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:55.693 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.693 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-10T09:06:55.693 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:55.693 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-10T09:06:55.694 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:55.694 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T09:06:55.694 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T09:06:55.694 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:55.694 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:55.694 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 4278, 2024-09-10T09:06:55.694 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 4278, 2024-09-10T09:06:55.694 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 4278, 2024-09-10T09:06:55.694 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 4278, 2024-09-10T09:06:55.694 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:55.694 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.694 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.694 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.695 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:55.695 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.695 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-10T09:06:55.695 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:55.695 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-10T09:06:55.695 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:55.695 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 206, 2024-09-10T09:06:55.695 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 21870, 2024-09-10T09:06:55.695 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:55.695 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 206, 2024-09-10T09:06:55.695 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T09:06:55.695 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T09:06:55.695 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 2, 2024-09-10T09:06:55.695 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 21870, 2024-09-10T09:06:55.696 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:55.696 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.696 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.696 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.696 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:55.696 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:55.696 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55f0ef0e8960": { 2024-09-10T09:06:55.696 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:55.696 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-10T09:06:55.696 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:55.696 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 864, 2024-09-10T09:06:55.696 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 1387660, 2024-09-10T09:06:55.696 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:55.697 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 864, 2024-09-10T09:06:55.697 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T09:06:55.697 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T09:06:55.697 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 309, 2024-09-10T09:06:55.697 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1387660, 2024-09-10T09:06:55.697 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:55.697 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:55.697 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:55.697 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:55.697 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:55.697 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:55.697 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-10T09:06:55.698 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:55 smithi174 ceph-mon[34068]: pgmap v970: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:06:55.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:55 smithi172 ceph-mon[30854]: pgmap v970: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:06:56.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:55 smithi191 ceph-mon[35314]: pgmap v970: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:06:56.564 INFO:tasks.check_counter:Daemon mds.h mds.exported=5 2024-09-10T09:06:56.564 INFO:tasks.check_counter:Daemon mds.h mds.imported=5 2024-09-10T09:06:56.564 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-09-10T09:06:56.564 DEBUG:tasks.check_counter:Getting stats from k 2024-09-10T09:06:56.565 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-10T09:06:56.817 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:06:56.938 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:56 smithi174 ceph-mon[34068]: pgmap v971: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:06:57.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:56 smithi191 ceph-mon[35314]: pgmap v971: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:06:57.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:56 smithi172 ceph-mon[30854]: pgmap v971: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:06:57.363 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-10T09:06:57.364 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T09:06:57.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 2385, 2024-09-10T09:06:57.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 3008, 2024-09-10T09:06:57.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 916310, 2024-09-10T09:06:57.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 2627331, 2024-09-10T09:06:57.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 15, 2024-09-10T09:06:57.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 6, 2024-09-10T09:06:57.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.198129642, 2024-09-10T09:06:57.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.063517600, 2024-09-10T09:06:57.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.101498442, 2024-09-10T09:06:57.364 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.030411761, 2024-09-10T09:06:57.365 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:57.365 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3005, 2024-09-10T09:06:57.365 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.188072099, 2024-09-10T09:06:57.365 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000062586 2024-09-10T09:06:57.365 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.365 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:57.365 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2799, 2024-09-10T09:06:57.365 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000550820, 2024-09-10T09:06:57.365 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000196 2024-09-10T09:06:57.365 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.365 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 4528, 2024-09-10T09:06:57.365 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 1888 2024-09-10T09:06:57.365 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.366 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T09:06:57.366 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 2283, 2024-09-10T09:06:57.366 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 2807, 2024-09-10T09:06:57.366 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 1034571, 2024-09-10T09:06:57.366 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 1797523, 2024-09-10T09:06:57.366 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 31, 2024-09-10T09:06:57.366 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 7, 2024-09-10T09:06:57.366 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.189106487, 2024-09-10T09:06:57.366 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.069092590, 2024-09-10T09:06:57.366 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.081752371, 2024-09-10T09:06:57.366 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.033296166, 2024-09-10T09:06:57.366 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:57.367 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2805, 2024-09-10T09:06:57.367 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.139773271, 2024-09-10T09:06:57.367 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000049830 2024-09-10T09:06:57.367 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.367 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:57.367 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2015, 2024-09-10T09:06:57.367 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000423295, 2024-09-10T09:06:57.367 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000210 2024-09-10T09:06:57.367 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.367 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T09:06:57.367 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 655872 2024-09-10T09:06:57.367 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.367 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T09:06:57.368 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 3352, 2024-09-10T09:06:57.368 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 4523, 2024-09-10T09:06:57.368 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 1737625, 2024-09-10T09:06:57.368 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 1988598, 2024-09-10T09:06:57.368 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 30, 2024-09-10T09:06:57.368 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 6, 2024-09-10T09:06:57.368 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.343998927, 2024-09-10T09:06:57.368 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.163545249, 2024-09-10T09:06:57.368 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.177482001, 2024-09-10T09:06:57.368 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.016978825, 2024-09-10T09:06:57.368 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:57.368 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4524, 2024-09-10T09:06:57.369 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.206148455, 2024-09-10T09:06:57.369 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000045567 2024-09-10T09:06:57.369 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.369 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:57.369 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5956, 2024-09-10T09:06:57.369 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.001256316, 2024-09-10T09:06:57.369 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000210 2024-09-10T09:06:57.369 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.369 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 713744, 2024-09-10T09:06:57.369 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 249952 2024-09-10T09:06:57.370 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.370 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-10T09:06:57.370 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-10T09:06:57.370 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-10T09:06:57.370 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.370 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-10T09:06:57.370 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-10T09:06:57.370 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-10T09:06:57.370 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1921, 2024-09-10T09:06:57.370 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.563607205, 2024-09-10T09:06:57.370 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000293392 2024-09-10T09:06:57.370 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:57.371 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.371 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-10T09:06:57.371 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-10T09:06:57.371 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-10T09:06:57.371 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 812, 2024-09-10T09:06:57.371 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.037955524, 2024-09-10T09:06:57.371 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000046743 2024-09-10T09:06:57.371 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:57.371 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.371 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-10T09:06:57.371 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 538, 2024-09-10T09:06:57.371 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 473, 2024-09-10T09:06:57.371 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-10T09:06:57.372 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 473, 2024-09-10T09:06:57.372 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 5.176235441, 2024-09-10T09:06:57.372 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.010943415 2024-09-10T09:06:57.372 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.372 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-10T09:06:57.372 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 65, 2024-09-10T09:06:57.372 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-10T09:06:57.372 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-10T09:06:57.372 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 11, 2024-09-10T09:06:57.372 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-09-10T09:06:57.372 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-10T09:06:57.373 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 16, 2024-09-10T09:06:57.373 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 0, 2024-09-10T09:06:57.373 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 2, 2024-09-10T09:06:57.373 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 14, 2024-09-10T09:06:57.373 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 15, 2024-09-10T09:06:57.373 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 5, 2024-09-10T09:06:57.373 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 1, 2024-09-10T09:06:57.373 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 1, 2024-09-10T09:06:57.373 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 4, 2024-09-10T09:06:57.373 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 859, 2024-09-10T09:06:57.373 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 740, 2024-09-10T09:06:57.373 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 56, 2024-09-10T09:06:57.373 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 4, 2024-09-10T09:06:57.374 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-10T09:06:57.374 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-10T09:06:57.374 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-09-10T09:06:57.374 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-09-10T09:06:57.374 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-10T09:06:57.374 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 2, 2024-09-10T09:06:57.374 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 2, 2024-09-10T09:06:57.374 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 8, 2024-09-10T09:06:57.374 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 118, 2024-09-10T09:06:57.374 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-10T09:06:57.374 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 1449, 2024-09-10T09:06:57.375 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-10T09:06:57.375 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 325, 2024-09-10T09:06:57.375 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 13424230450, 2024-09-10T09:06:57.375 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-09-10T09:06:57.375 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 1449, 2024-09-10T09:06:57.375 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-10T09:06:57.375 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T09:06:57.375 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-10T09:06:57.375 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 126, 2024-09-10T09:06:57.375 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T09:06:57.375 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 126, 2024-09-10T09:06:57.375 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 1323, 2024-09-10T09:06:57.376 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 21, 2024-09-10T09:06:57.376 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 269, 2024-09-10T09:06:57.376 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 247, 2024-09-10T09:06:57.376 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T09:06:57.376 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T09:06:57.376 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T09:06:57.376 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 460, 2024-09-10T09:06:57.376 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 169, 2024-09-10T09:06:57.376 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 6, 2024-09-10T09:06:57.376 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 240 2024-09-10T09:06:57.376 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.377 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-10T09:06:57.377 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-10T09:06:57.377 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-10T09:06:57.377 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-10T09:06:57.377 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 15, 2024-09-10T09:06:57.377 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 10, 2024-09-10T09:06:57.377 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-10T09:06:57.377 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-10T09:06:57.377 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-10T09:06:57.377 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 117, 2024-09-10T09:06:57.377 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 117, 2024-09-10T09:06:57.377 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-10T09:06:57.378 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-10T09:06:57.378 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-10T09:06:57.378 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-10T09:06:57.378 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-10T09:06:57.378 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-10T09:06:57.378 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-10T09:06:57.378 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 15, 2024-09-10T09:06:57.378 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 1422, 2024-09-10T09:06:57.378 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T09:06:57.378 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 2, 2024-09-10T09:06:57.378 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-10T09:06:57.378 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-09-10T09:06:57.378 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-10T09:06:57.379 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-10T09:06:57.379 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.379 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-10T09:06:57.379 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-10T09:06:57.379 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 832, 2024-09-10T09:06:57.379 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-10T09:06:57.379 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-10T09:06:57.379 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 832, 2024-09-10T09:06:57.379 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-10T09:06:57.379 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-10T09:06:57.379 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 2, 2024-09-10T09:06:57.379 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-10T09:06:57.379 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-10T09:06:57.379 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-09-10T09:06:57.380 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-09-10T09:06:57.380 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-10T09:06:57.380 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-10T09:06:57.380 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-09-10T09:06:57.380 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 5503716, 2024-09-10T09:06:57.380 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-09-10T09:06:57.380 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-10T09:06:57.380 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 391, 2024-09-10T09:06:57.380 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 19.880103940, 2024-09-10T09:06:57.380 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.050844255 2024-09-10T09:06:57.380 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.380 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-09-10T09:06:57.380 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.380 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-10T09:06:57.381 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 19, 2024-09-10T09:06:57.381 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 138, 2024-09-10T09:06:57.381 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 119, 2024-09-10T09:06:57.381 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 18, 2024-09-10T09:06:57.381 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 29, 2024-09-10T09:06:57.381 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 11, 2024-09-10T09:06:57.381 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 16, 2024-09-10T09:06:57.381 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 253, 2024-09-10T09:06:57.381 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 237, 2024-09-10T09:06:57.381 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 1, 2024-09-10T09:06:57.381 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 120, 2024-09-10T09:06:57.381 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 119, 2024-09-10T09:06:57.381 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 57180, 2024-09-10T09:06:57.382 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 223520 2024-09-10T09:06:57.382 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.382 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-10T09:06:57.382 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 647, 2024-09-10T09:06:57.382 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 43, 2024-09-10T09:06:57.382 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 538, 2024-09-10T09:06:57.382 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 19, 2024-09-10T09:06:57.382 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 136, 2024-09-10T09:06:57.382 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-10T09:06:57.382 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-09-10T09:06:57.382 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.008497921, 2024-09-10T09:06:57.382 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004248960 2024-09-10T09:06:57.382 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.382 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-10T09:06:57.383 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 52, 2024-09-10T09:06:57.383 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.018077378, 2024-09-10T09:06:57.383 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000347641 2024-09-10T09:06:57.383 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.383 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-10T09:06:57.383 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.383 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.383 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.383 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.383 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-10T09:06:57.383 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.383 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.383 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.383 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.384 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-10T09:06:57.384 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 189, 2024-09-10T09:06:57.384 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 4.738686904, 2024-09-10T09:06:57.384 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.025072417 2024-09-10T09:06:57.384 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.384 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-10T09:06:57.384 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.384 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.384 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.384 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.384 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-10T09:06:57.384 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.384 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.385 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.385 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.385 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-10T09:06:57.385 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.385 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.385 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.385 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.385 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-10T09:06:57.385 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.385 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.385 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.385 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.386 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-10T09:06:57.386 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.386 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.386 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.386 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.386 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-10T09:06:57.386 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.386 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.386 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.386 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.386 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-10T09:06:57.386 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.386 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.386 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.387 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.387 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-10T09:06:57.387 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.387 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.387 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.387 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.387 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-10T09:06:57.387 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.387 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.387 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.387 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.387 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-10T09:06:57.387 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 107, 2024-09-10T09:06:57.388 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.077720641, 2024-09-10T09:06:57.388 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000726361 2024-09-10T09:06:57.388 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.388 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-10T09:06:57.388 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 6, 2024-09-10T09:06:57.388 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.003552829, 2024-09-10T09:06:57.388 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000592138 2024-09-10T09:06:57.388 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.388 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-10T09:06:57.388 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.388 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.388 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.388 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.389 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-10T09:06:57.389 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.389 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.389 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.389 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.389 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-10T09:06:57.389 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.389 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.389 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.389 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.389 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-10T09:06:57.389 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 8, 2024-09-10T09:06:57.390 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.164965663, 2024-09-10T09:06:57.390 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.020620707 2024-09-10T09:06:57.390 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.390 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-10T09:06:57.390 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.390 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.390 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.390 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.390 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-10T09:06:57.390 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.390 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.390 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.390 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.390 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-10T09:06:57.391 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.391 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.391 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.391 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.391 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-10T09:06:57.391 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.391 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.391 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.391 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.391 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-10T09:06:57.391 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.391 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.391 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.391 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.392 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-10T09:06:57.392 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.392 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.392 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.392 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.392 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-10T09:06:57.392 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.392 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.392 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.392 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.392 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-10T09:06:57.392 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.392 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.393 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.393 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.393 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-10T09:06:57.393 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 109, 2024-09-10T09:06:57.393 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.164734105, 2024-09-10T09:06:57.393 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001511322 2024-09-10T09:06:57.393 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.393 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-10T09:06:57.393 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-10T09:06:57.393 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-10T09:06:57.393 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.394 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.394 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.394 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:57.394 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.394 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-10T09:06:57.394 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-09-10T09:06:57.394 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 2, 2024-09-10T09:06:57.394 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 1, 2024-09-10T09:06:57.394 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2024-09-10T09:06:57.394 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-10T09:06:57.394 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 294, 2024-09-10T09:06:57.394 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 294, 2024-09-10T09:06:57.394 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 1218, 2024-09-10T09:06:57.395 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-10T09:06:57.395 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.395 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-10T09:06:57.395 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 120, 2024-09-10T09:06:57.395 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 120, 2024-09-10T09:06:57.395 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T09:06:57.395 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-10T09:06:57.395 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T09:06:57.395 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-10T09:06:57.395 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T09:06:57.395 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T09:06:57.395 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T09:06:57.395 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T09:06:57.395 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T09:06:57.395 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-10T09:06:57.396 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T09:06:57.396 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T09:06:57.396 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-10T09:06:57.396 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-10T09:06:57.396 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-10T09:06:57.396 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-10T09:06:57.396 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T09:06:57.396 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T09:06:57.396 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T09:06:57.396 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T09:06:57.396 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T09:06:57.396 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-10T09:06:57.396 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-10T09:06:57.397 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-10T09:06:57.397 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T09:06:57.397 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T09:06:57.397 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-10T09:06:57.397 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-10T09:06:57.397 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-10T09:06:57.397 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-10T09:06:57.397 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T09:06:57.397 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-10T09:06:57.397 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T09:06:57.397 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-10T09:06:57.397 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 281511, 2024-09-10T09:06:57.397 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 101, 2024-09-10T09:06:57.398 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-10T09:06:57.398 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-10T09:06:57.398 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-10T09:06:57.398 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-10T09:06:57.398 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-10T09:06:57.398 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-10T09:06:57.398 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-10T09:06:57.398 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-10T09:06:57.398 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8672, 2024-09-10T09:06:57.398 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 163, 2024-09-10T09:06:57.398 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T09:06:57.398 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-10T09:06:57.398 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-10T09:06:57.398 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-10T09:06:57.399 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 79421, 2024-09-10T09:06:57.399 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 405, 2024-09-10T09:06:57.399 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-10T09:06:57.399 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-10T09:06:57.399 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-10T09:06:57.399 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-10T09:06:57.399 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.399 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-10T09:06:57.399 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-10T09:06:57.399 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-10T09:06:57.399 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 2844, 2024-09-10T09:06:57.399 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 1357074, 2024-09-10T09:06:57.399 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 33, 2024-09-10T09:06:57.400 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 2844, 2024-09-10T09:06:57.400 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-10T09:06:57.400 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2844, 2024-09-10T09:06:57.400 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 98.572589239, 2024-09-10T09:06:57.400 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.034659841 2024-09-10T09:06:57.400 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.400 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-10T09:06:57.400 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-10T09:06:57.400 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2844, 2024-09-10T09:06:57.400 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2970 2024-09-10T09:06:57.400 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.400 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 2844, 2024-09-10T09:06:57.400 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 1454, 2024-09-10T09:06:57.400 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 1390, 2024-09-10T09:06:57.401 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-10T09:06:57.401 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-10T09:06:57.401 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2024-09-10T09:06:57.401 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2024-09-10T09:06:57.401 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 5, 2024-09-10T09:06:57.401 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 396, 2024-09-10T09:06:57.401 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 63, 2024-09-10T09:06:57.401 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-10T09:06:57.401 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-10T09:06:57.401 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-10T09:06:57.401 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-09-10T09:06:57.402 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 814, 2024-09-10T09:06:57.402 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-10T09:06:57.402 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-10T09:06:57.402 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-10T09:06:57.402 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 1449, 2024-09-10T09:06:57.402 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2024-09-10T09:06:57.402 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-10T09:06:57.402 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-10T09:06:57.402 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-10T09:06:57.402 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-10T09:06:57.402 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-10T09:06:57.402 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-10T09:06:57.402 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T09:06:57.402 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-10T09:06:57.403 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-10T09:06:57.403 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-10T09:06:57.403 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-10T09:06:57.403 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-10T09:06:57.403 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-10T09:06:57.403 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-10T09:06:57.403 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-10T09:06:57.403 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-10T09:06:57.403 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-10T09:06:57.403 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-10T09:06:57.403 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-10T09:06:57.403 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-10T09:06:57.403 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-10T09:06:57.403 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-10T09:06:57.404 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-10T09:06:57.404 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-10T09:06:57.404 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-10T09:06:57.404 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-10T09:06:57.404 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 92, 2024-09-10T09:06:57.404 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-10T09:06:57.404 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 4, 2024-09-10T09:06:57.404 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 12, 2024-09-10T09:06:57.404 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2024-09-10T09:06:57.404 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-09-10T09:06:57.404 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-10T09:06:57.404 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 171, 2024-09-10T09:06:57.404 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 0, 2024-09-10T09:06:57.404 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 42 2024-09-10T09:06:57.405 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.405 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-10T09:06:57.405 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-09-10T09:06:57.405 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-10T09:06:57.405 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-10T09:06:57.405 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-10T09:06:57.405 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.405 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-10T09:06:57.405 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 1344 2024-09-10T09:06:57.405 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.405 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-10T09:06:57.405 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-10T09:06:57.405 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 34, 2024-09-10T09:06:57.406 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-10T09:06:57.406 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 7, 2024-09-10T09:06:57.406 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 802, 2024-09-10T09:06:57.406 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 117, 2024-09-10T09:06:57.406 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-10T09:06:57.406 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.406 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T09:06:57.406 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:57.406 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-10T09:06:57.406 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:57.406 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 8016, 2024-09-10T09:06:57.406 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 2940185, 2024-09-10T09:06:57.406 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:57.406 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 8016, 2024-09-10T09:06:57.407 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T09:06:57.407 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T09:06:57.407 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 8016, 2024-09-10T09:06:57.407 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 2940185, 2024-09-10T09:06:57.407 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:57.407 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.407 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.407 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.407 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:57.407 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.407 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-10T09:06:57.407 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:57.407 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-10T09:06:57.408 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:57.408 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T09:06:57.408 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T09:06:57.408 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:57.408 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:57.408 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 2844, 2024-09-10T09:06:57.408 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 1369763, 2024-09-10T09:06:57.408 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 2844, 2024-09-10T09:06:57.408 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1369763, 2024-09-10T09:06:57.408 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:57.408 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.408 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.408 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.409 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:57.409 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.409 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-10T09:06:57.409 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:57.409 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-10T09:06:57.409 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:57.409 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T09:06:57.409 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T09:06:57.409 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:57.409 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:57.409 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 2844, 2024-09-10T09:06:57.409 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 2844, 2024-09-10T09:06:57.410 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 2844, 2024-09-10T09:06:57.410 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 2844, 2024-09-10T09:06:57.410 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:57.410 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.410 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.410 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.410 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:57.410 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.410 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-10T09:06:57.410 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:57.410 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-10T09:06:57.410 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:57.410 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 117, 2024-09-10T09:06:57.411 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 12689, 2024-09-10T09:06:57.411 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:57.411 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 117, 2024-09-10T09:06:57.411 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T09:06:57.411 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T09:06:57.411 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 5, 2024-09-10T09:06:57.411 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 12689, 2024-09-10T09:06:57.411 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:57.411 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.411 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.411 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.411 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:57.411 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T09:06:57.412 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x56050f31c960": { 2024-09-10T09:06:57.412 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T09:06:57.412 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-10T09:06:57.412 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T09:06:57.412 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 832, 2024-09-10T09:06:57.412 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 1309412, 2024-09-10T09:06:57.412 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:57.412 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 832, 2024-09-10T09:06:57.412 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T09:06:57.412 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T09:06:57.412 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 391, 2024-09-10T09:06:57.412 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1309412, 2024-09-10T09:06:57.412 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T09:06:57.413 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T09:06:57.413 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T09:06:57.413 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T09:06:57.413 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:57.413 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T09:06:57.413 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-10T09:06:58.164 INFO:tasks.check_counter:Daemon mds.k mds.exported=2 2024-09-10T09:06:58.164 INFO:tasks.check_counter:Daemon mds.k mds.imported=8 2024-09-10T09:06:58.164 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-09-10T09:06:58.164 DEBUG:tasks.check_counter:Getting stats from c 2024-09-10T09:06:58.165 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-10T09:06:58.481 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:06:58.916 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:06:58 smithi191 ceph-mon[35314]: pgmap v972: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:06:58.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:06:58 smithi174 ceph-mon[34068]: pgmap v972: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:06:59.067 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-10T09:06:59.067 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T09:06:59.067 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 1587, 2024-09-10T09:06:59.067 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 1587, 2024-09-10T09:06:59.067 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 4710037, 2024-09-10T09:06:59.067 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 503852, 2024-09-10T09:06:59.067 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 6, 2024-09-10T09:06:59.068 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 2, 2024-09-10T09:06:59.068 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.173594521, 2024-09-10T09:06:59.068 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.051272882, 2024-09-10T09:06:59.068 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.049262639, 2024-09-10T09:06:59.068 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.055576065, 2024-09-10T09:06:59.068 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:59.068 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1586, 2024-09-10T09:06:59.068 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.088205401, 2024-09-10T09:06:59.068 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000055615 2024-09-10T09:06:59.068 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.068 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:59.068 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.069 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.069 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.069 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.069 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-10T09:06:59.069 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T09:06:59.069 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.069 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T09:06:59.069 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 1280, 2024-09-10T09:06:59.069 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 1561, 2024-09-10T09:06:59.069 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 503125, 2024-09-10T09:06:59.069 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 972372, 2024-09-10T09:06:59.069 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 6, 2024-09-10T09:06:59.069 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 4, 2024-09-10T09:06:59.070 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.124811436, 2024-09-10T09:06:59.070 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.050502361, 2024-09-10T09:06:59.070 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.025190075, 2024-09-10T09:06:59.070 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.037247958, 2024-09-10T09:06:59.070 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:59.070 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1559, 2024-09-10T09:06:59.070 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.065336397, 2024-09-10T09:06:59.070 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000041909 2024-09-10T09:06:59.070 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.070 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:59.070 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.070 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.070 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.071 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.071 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T09:06:59.071 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 565584 2024-09-10T09:06:59.071 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.071 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T09:06:59.071 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 1827, 2024-09-10T09:06:59.071 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 1798, 2024-09-10T09:06:59.071 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 5265897, 2024-09-10T09:06:59.071 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 681195, 2024-09-10T09:06:59.071 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 3, 2024-09-10T09:06:59.071 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 2, 2024-09-10T09:06:59.071 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.172019220, 2024-09-10T09:06:59.071 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.070694866, 2024-09-10T09:06:59.072 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.063845138, 2024-09-10T09:06:59.072 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.032831340, 2024-09-10T09:06:59.072 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:06:59.072 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1797, 2024-09-10T09:06:59.072 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.073248332, 2024-09-10T09:06:59.072 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000040761 2024-09-10T09:06:59.072 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.072 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:06:59.072 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.072 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.073 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.073 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.073 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 707456, 2024-09-10T09:06:59.073 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 229952 2024-09-10T09:06:59.073 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.073 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-09-10T09:06:59.073 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-09-10T09:06:59.073 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-09-10T09:06:59.073 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.073 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-09-10T09:06:59.073 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-09-10T09:06:59.073 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-09-10T09:06:59.073 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 7118, 2024-09-10T09:06:59.074 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.188485748, 2024-09-10T09:06:59.074 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000026480 2024-09-10T09:06:59.074 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:06:59.074 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.074 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-09-10T09:06:59.074 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-09-10T09:06:59.074 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-09-10T09:06:59.074 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.074 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.074 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.074 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:06:59.074 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.074 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-09-10T09:06:59.075 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 0, 2024-09-10T09:06:59.075 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 0, 2024-09-10T09:06:59.075 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-09-10T09:06:59.075 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.075 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.075 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.075 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.075 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-09-10T09:06:59.075 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 0, 2024-09-10T09:06:59.075 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-09-10T09:06:59.075 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-09-10T09:06:59.075 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 0, 2024-09-10T09:06:59.076 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 0, 2024-09-10T09:06:59.076 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-09-10T09:06:59.076 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 574, 2024-09-10T09:06:59.076 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 394, 2024-09-10T09:06:59.076 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 180, 2024-09-10T09:06:59.076 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 0, 2024-09-10T09:06:59.076 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 11, 2024-09-10T09:06:59.076 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 0, 2024-09-10T09:06:59.076 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 0, 2024-09-10T09:06:59.076 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 0, 2024-09-10T09:06:59.077 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 3, 2024-09-10T09:06:59.077 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 0, 2024-09-10T09:06:59.077 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 0, 2024-09-10T09:06:59.077 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 0, 2024-09-10T09:06:59.077 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 0, 2024-09-10T09:06:59.077 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-09-10T09:06:59.077 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-09-10T09:06:59.077 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 0, 2024-09-10T09:06:59.077 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 0, 2024-09-10T09:06:59.077 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 0, 2024-09-10T09:06:59.077 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 0, 2024-09-10T09:06:59.077 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 0, 2024-09-10T09:06:59.077 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 0, 2024-09-10T09:06:59.078 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 0, 2024-09-10T09:06:59.078 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-09-10T09:06:59.078 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 0, 2024-09-10T09:06:59.078 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 0, 2024-09-10T09:06:59.078 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 4, 2024-09-10T09:06:59.078 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 234, 2024-09-10T09:06:59.078 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 0, 2024-09-10T09:06:59.078 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T09:06:59.078 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-09-10T09:06:59.078 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T09:06:59.078 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-09-10T09:06:59.079 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 0, 2024-09-10T09:06:59.079 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T09:06:59.079 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T09:06:59.079 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 0, 2024-09-10T09:06:59.079 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 0, 2024-09-10T09:06:59.079 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T09:06:59.079 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 0, 2024-09-10T09:06:59.079 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T09:06:59.079 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T09:06:59.079 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T09:06:59.079 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 0, 2024-09-10T09:06:59.079 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 0, 2024-09-10T09:06:59.079 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 0, 2024-09-10T09:06:59.079 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 0 2024-09-10T09:06:59.080 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.080 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-09-10T09:06:59.080 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-09-10T09:06:59.080 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 0, 2024-09-10T09:06:59.080 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 0, 2024-09-10T09:06:59.080 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 0, 2024-09-10T09:06:59.080 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 0, 2024-09-10T09:06:59.080 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 0, 2024-09-10T09:06:59.080 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-09-10T09:06:59.080 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-09-10T09:06:59.080 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 0, 2024-09-10T09:06:59.080 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 0, 2024-09-10T09:06:59.080 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 0, 2024-09-10T09:06:59.081 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-09-10T09:06:59.081 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-09-10T09:06:59.081 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-09-10T09:06:59.081 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-09-10T09:06:59.081 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-09-10T09:06:59.081 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-09-10T09:06:59.081 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_quiesce_path": 0, 2024-09-10T09:06:59.081 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_quiesce_inode": 0, 2024-09-10T09:06:59.081 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T09:06:59.081 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 0, 2024-09-10T09:06:59.081 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-09-10T09:06:59.081 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 0, 2024-09-10T09:06:59.082 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-09-10T09:06:59.082 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-09-10T09:06:59.082 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.082 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-09-10T09:06:59.082 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-09-10T09:06:59.082 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 0, 2024-09-10T09:06:59.082 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 0, 2024-09-10T09:06:59.082 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 0, 2024-09-10T09:06:59.082 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 4065, 2024-09-10T09:06:59.082 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-09-10T09:06:59.082 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 0, 2024-09-10T09:06:59.082 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 0, 2024-09-10T09:06:59.082 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 0, 2024-09-10T09:06:59.082 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 0, 2024-09-10T09:06:59.083 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 2, 2024-09-10T09:06:59.083 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 83, 2024-09-10T09:06:59.083 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-09-10T09:06:59.083 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 0, 2024-09-10T09:06:59.083 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 12752163, 2024-09-10T09:06:59.083 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 12767963, 2024-09-10T09:06:59.083 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 12764541, 2024-09-10T09:06:59.083 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-09-10T09:06:59.083 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.083 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.083 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.083 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.083 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 4065 2024-09-10T09:06:59.084 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.084 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-09-10T09:06:59.084 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 18, 2024-09-10T09:06:59.084 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 806, 2024-09-10T09:06:59.084 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 788, 2024-09-10T09:06:59.084 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 14, 2024-09-10T09:06:59.084 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 121, 2024-09-10T09:06:59.084 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 107, 2024-09-10T09:06:59.084 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 574, 2024-09-10T09:06:59.084 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 1077, 2024-09-10T09:06:59.084 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 503, 2024-09-10T09:06:59.084 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 0, 2024-09-10T09:06:59.084 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 0, 2024-09-10T09:06:59.085 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 0, 2024-09-10T09:06:59.085 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 49680, 2024-09-10T09:06:59.085 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 215328 2024-09-10T09:06:59.085 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.085 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-09-10T09:06:59.085 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 0, 2024-09-10T09:06:59.085 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 0, 2024-09-10T09:06:59.085 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 0, 2024-09-10T09:06:59.085 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 0, 2024-09-10T09:06:59.085 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 0, 2024-09-10T09:06:59.085 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-09-10T09:06:59.086 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.086 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.086 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.086 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.086 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-09-10T09:06:59.086 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.086 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.086 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.086 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.086 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-09-10T09:06:59.086 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.086 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.086 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.087 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.087 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-09-10T09:06:59.087 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.087 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.087 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.087 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.087 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-09-10T09:06:59.087 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.087 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.087 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.087 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.087 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-09-10T09:06:59.087 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.088 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.088 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.088 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.088 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-09-10T09:06:59.088 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.088 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.088 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.088 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.088 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-09-10T09:06:59.088 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.088 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.088 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.088 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.089 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-09-10T09:06:59.089 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.089 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.089 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.089 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.089 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-09-10T09:06:59.089 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.089 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.089 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.090 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.090 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-09-10T09:06:59.090 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.090 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.090 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.090 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.090 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-09-10T09:06:59.090 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.090 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.090 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.090 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.090 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-09-10T09:06:59.090 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.091 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.091 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.091 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.091 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-09-10T09:06:59.091 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.091 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.091 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.091 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.092 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-09-10T09:06:59.092 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.092 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.092 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.092 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.092 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-09-10T09:06:59.092 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.092 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.093 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.093 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.093 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-09-10T09:06:59.093 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.093 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.093 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.093 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.093 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-09-10T09:06:59.093 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.093 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.093 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.093 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.093 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-09-10T09:06:59.094 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.094 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.094 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.094 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.094 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-09-10T09:06:59.094 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.094 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.094 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.094 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.094 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-09-10T09:06:59.094 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.094 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.094 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.095 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.095 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-09-10T09:06:59.095 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.095 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.095 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.095 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.095 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-09-10T09:06:59.095 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.095 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.095 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.095 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.095 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-09-10T09:06:59.095 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.096 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.096 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.096 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.096 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-09-10T09:06:59.096 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.096 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.096 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.096 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.096 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-09-10T09:06:59.096 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.096 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.096 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.096 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.097 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-09-10T09:06:59.097 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.097 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.097 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.097 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.097 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-09-10T09:06:59.097 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.097 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.097 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.098 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.098 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-09-10T09:06:59.098 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.098 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.098 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.099 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.099 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-09-10T09:06:59.099 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-09-10T09:06:59.099 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-09-10T09:06:59.099 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.099 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.099 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.099 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:06:59.099 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.099 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-09-10T09:06:59.099 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 1, 2024-09-10T09:06:59.100 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 3, 2024-09-10T09:06:59.100 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 2, 2024-09-10T09:06:59.100 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 1, 2024-09-10T09:06:59.100 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-09-10T09:06:59.100 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 0, 2024-09-10T09:06:59.100 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 0, 2024-09-10T09:06:59.100 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 2345, 2024-09-10T09:06:59.100 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-09-10T09:06:59.100 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.100 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-09-10T09:06:59.100 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-09-10T09:06:59.100 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-09-10T09:06:59.101 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T09:06:59.101 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-09-10T09:06:59.101 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T09:06:59.101 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-09-10T09:06:59.101 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T09:06:59.101 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T09:06:59.101 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T09:06:59.101 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T09:06:59.101 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T09:06:59.101 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-09-10T09:06:59.101 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T09:06:59.101 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T09:06:59.102 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-09-10T09:06:59.102 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-09-10T09:06:59.102 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-09-10T09:06:59.102 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-09-10T09:06:59.102 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T09:06:59.102 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T09:06:59.102 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T09:06:59.102 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T09:06:59.102 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T09:06:59.102 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-09-10T09:06:59.102 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-09-10T09:06:59.102 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-09-10T09:06:59.102 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T09:06:59.103 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T09:06:59.103 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-09-10T09:06:59.103 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-09-10T09:06:59.103 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-09-10T09:06:59.103 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-09-10T09:06:59.103 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T09:06:59.103 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-09-10T09:06:59.103 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T09:06:59.103 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-09-10T09:06:59.103 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 190167, 2024-09-10T09:06:59.103 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 52, 2024-09-10T09:06:59.103 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 0, 2024-09-10T09:06:59.103 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 0, 2024-09-10T09:06:59.104 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-09-10T09:06:59.104 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-09-10T09:06:59.104 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-09-10T09:06:59.104 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-09-10T09:06:59.104 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-09-10T09:06:59.104 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-09-10T09:06:59.104 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 8672, 2024-09-10T09:06:59.104 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 163, 2024-09-10T09:06:59.104 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T09:06:59.104 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-09-10T09:06:59.104 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-09-10T09:06:59.104 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-09-10T09:06:59.104 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 462493, 2024-09-10T09:06:59.104 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 2504, 2024-09-10T09:06:59.105 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-09-10T09:06:59.105 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-09-10T09:06:59.105 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-09-10T09:06:59.105 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-09-10T09:06:59.105 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.105 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-09-10T09:06:59.105 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-09-10T09:06:59.105 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-09-10T09:06:59.105 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 4281, 2024-09-10T09:06:59.105 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 26, 2024-09-10T09:06:59.105 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 4, 2024-09-10T09:06:59.105 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 4281, 2024-09-10T09:06:59.106 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-09-10T09:06:59.106 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 4281, 2024-09-10T09:06:59.106 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 3.145940694, 2024-09-10T09:06:59.106 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000734861 2024-09-10T09:06:59.106 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.106 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-09-10T09:06:59.106 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-09-10T09:06:59.106 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 4281, 2024-09-10T09:06:59.106 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 4282 2024-09-10T09:06:59.106 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.106 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 4281, 2024-09-10T09:06:59.106 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 4281, 2024-09-10T09:06:59.106 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 0, 2024-09-10T09:06:59.107 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-09-10T09:06:59.107 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-09-10T09:06:59.107 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 2646, 2024-09-10T09:06:59.107 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 0, 2024-09-10T09:06:59.107 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 1632, 2024-09-10T09:06:59.107 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 0, 2024-09-10T09:06:59.107 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 0, 2024-09-10T09:06:59.107 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-09-10T09:06:59.107 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-09-10T09:06:59.107 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-09-10T09:06:59.107 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 0, 2024-09-10T09:06:59.107 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 0, 2024-09-10T09:06:59.107 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-09-10T09:06:59.107 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-09-10T09:06:59.108 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-09-10T09:06:59.108 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 2, 2024-09-10T09:06:59.108 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 0, 2024-09-10T09:06:59.108 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-09-10T09:06:59.108 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-09-10T09:06:59.108 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-09-10T09:06:59.108 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-09-10T09:06:59.108 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-09-10T09:06:59.108 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-09-10T09:06:59.108 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T09:06:59.108 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-09-10T09:06:59.108 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-09-10T09:06:59.108 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 0, 2024-09-10T09:06:59.109 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-09-10T09:06:59.109 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-09-10T09:06:59.109 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-09-10T09:06:59.109 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-09-10T09:06:59.109 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-09-10T09:06:59.109 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-09-10T09:06:59.109 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-09-10T09:06:59.109 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-09-10T09:06:59.109 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-09-10T09:06:59.109 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-09-10T09:06:59.109 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-09-10T09:06:59.109 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-09-10T09:06:59.109 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-09-10T09:06:59.110 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-09-10T09:06:59.110 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-09-10T09:06:59.110 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-09-10T09:06:59.110 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 92, 2024-09-10T09:06:59.110 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-09-10T09:06:59.110 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 4, 2024-09-10T09:06:59.110 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 5, 2024-09-10T09:06:59.110 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 6, 2024-09-10T09:06:59.110 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 1, 2024-09-10T09:06:59.110 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-09-10T09:06:59.110 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 0, 2024-09-10T09:06:59.110 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 2, 2024-09-10T09:06:59.110 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 0 2024-09-10T09:06:59.110 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.111 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-09-10T09:06:59.111 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 0, 2024-09-10T09:06:59.111 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-09-10T09:06:59.111 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-09-10T09:06:59.111 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-09-10T09:06:59.111 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.111 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-09-10T09:06:59.111 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 0 2024-09-10T09:06:59.111 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.111 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-09-10T09:06:59.111 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-09-10T09:06:59.111 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T09:06:59.111 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-09-10T09:06:59.111 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 0, 2024-09-10T09:06:59.112 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 0, 2024-09-10T09:06:59.112 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 0, 2024-09-10T09:06:59.112 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-09-10T09:06:59.112 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.112 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T09:06:59.112 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:06:59.112 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-09-10T09:06:59.112 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:06:59.112 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 4690, 2024-09-10T09:06:59.112 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 10033147, 2024-09-10T09:06:59.112 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:59.112 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 4690, 2024-09-10T09:06:59.112 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-10T09:06:59.113 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-10T09:06:59.113 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 4690, 2024-09-10T09:06:59.113 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 10033147, 2024-09-10T09:06:59.113 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:06:59.113 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.113 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.113 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.113 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:06:59.113 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.113 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-09-10T09:06:59.113 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:06:59.113 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-09-10T09:06:59.113 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:06:59.114 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-10T09:06:59.114 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-10T09:06:59.114 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:59.114 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:59.114 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 4281, 2024-09-10T09:06:59.114 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 8573669, 2024-09-10T09:06:59.114 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 4281, 2024-09-10T09:06:59.114 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 8573669, 2024-09-10T09:06:59.114 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:06:59.114 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.114 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.114 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.114 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:06:59.115 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.115 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-09-10T09:06:59.115 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:06:59.115 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-09-10T09:06:59.115 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:06:59.115 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-10T09:06:59.115 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-10T09:06:59.115 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:59.115 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:59.115 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 4281, 2024-09-10T09:06:59.115 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 4281, 2024-09-10T09:06:59.115 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 4281, 2024-09-10T09:06:59.115 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 4281, 2024-09-10T09:06:59.115 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:06:59.116 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.116 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.116 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.116 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:06:59.116 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.116 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-09-10T09:06:59.116 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:06:59.116 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-09-10T09:06:59.116 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:06:59.116 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-10T09:06:59.116 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-10T09:06:59.116 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:59.116 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:59.117 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-10T09:06:59.117 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-10T09:06:59.117 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-09-10T09:06:59.117 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-09-10T09:06:59.117 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:06:59.117 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.117 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.117 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.117 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:06:59.117 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:06:59.117 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x55f547ea0b40": { 2024-09-10T09:06:59.117 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:06:59.118 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-09-10T09:06:59.118 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:06:59.118 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-10T09:06:59.118 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-10T09:06:59.118 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:06:59.118 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-10T09:06:59.118 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-10T09:06:59.118 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-10T09:06:59.118 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-09-10T09:06:59.118 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-09-10T09:06:59.118 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:06:59.119 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:06:59.119 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:06:59.119 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:06:59.119 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:06:59.119 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:06:59.119 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-10T09:06:59.120 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:06:58 smithi172 ceph-mon[30854]: pgmap v972: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:07:00.028 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-10T09:07:00.029 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-10T09:07:00.029 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-09-10T09:07:00.029 DEBUG:tasks.check_counter:Getting stats from f 2024-09-10T09:07:00.029 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-10T09:07:00.282 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:07:00.693 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:00 smithi191 ceph-mon[35314]: pgmap v973: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T09:07:00.811 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-10T09:07:00.811 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T09:07:00.811 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 1820, 2024-09-10T09:07:00.811 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 1820, 2024-09-10T09:07:00.811 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 3549531, 2024-09-10T09:07:00.811 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 578000, 2024-09-10T09:07:00.811 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 7, 2024-09-10T09:07:00.812 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 2, 2024-09-10T09:07:00.812 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.156637212, 2024-09-10T09:07:00.812 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.043899095, 2024-09-10T09:07:00.812 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.049156222, 2024-09-10T09:07:00.812 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.049249880, 2024-09-10T09:07:00.812 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:07:00.812 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1819, 2024-09-10T09:07:00.812 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.075216305, 2024-09-10T09:07:00.812 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000041350 2024-09-10T09:07:00.812 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.812 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:07:00.812 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.812 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.813 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.813 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.813 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-10T09:07:00.813 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T09:07:00.813 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.813 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T09:07:00.813 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 1574, 2024-09-10T09:07:00.813 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 1545, 2024-09-10T09:07:00.813 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 994452, 2024-09-10T09:07:00.813 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 601547, 2024-09-10T09:07:00.813 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 4, 2024-09-10T09:07:00.813 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 3, 2024-09-10T09:07:00.813 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.157642909, 2024-09-10T09:07:00.814 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.052589317, 2024-09-10T09:07:00.814 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.059100454, 2024-09-10T09:07:00.814 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.044033658, 2024-09-10T09:07:00.814 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:07:00.814 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1544, 2024-09-10T09:07:00.814 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.058935371, 2024-09-10T09:07:00.814 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000038170 2024-09-10T09:07:00.814 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.814 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:07:00.814 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.814 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.814 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.814 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.815 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 707840, 2024-09-10T09:07:00.815 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 231072 2024-09-10T09:07:00.815 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.815 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T09:07:00.815 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 1265, 2024-09-10T09:07:00.815 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 1547, 2024-09-10T09:07:00.815 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 424864, 2024-09-10T09:07:00.815 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 969328, 2024-09-10T09:07:00.815 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 4, 2024-09-10T09:07:00.815 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 3, 2024-09-10T09:07:00.815 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.157829184, 2024-09-10T09:07:00.815 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.055300289, 2024-09-10T09:07:00.816 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.039168449, 2024-09-10T09:07:00.816 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.046808644, 2024-09-10T09:07:00.816 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:07:00.816 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1545, 2024-09-10T09:07:00.816 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.088057552, 2024-09-10T09:07:00.816 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000056995 2024-09-10T09:07:00.816 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.816 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:07:00.816 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.816 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.816 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.817 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.817 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T09:07:00.817 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 567376 2024-09-10T09:07:00.817 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.817 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-09-10T09:07:00.817 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-09-10T09:07:00.817 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-09-10T09:07:00.817 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.817 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-09-10T09:07:00.817 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-09-10T09:07:00.817 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-09-10T09:07:00.817 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 7211, 2024-09-10T09:07:00.817 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.191487991, 2024-09-10T09:07:00.818 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000026554 2024-09-10T09:07:00.818 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:00.818 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.818 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-09-10T09:07:00.818 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-09-10T09:07:00.818 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-09-10T09:07:00.818 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.818 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.818 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.818 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:00.818 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.818 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-09-10T09:07:00.819 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 0, 2024-09-10T09:07:00.819 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 0, 2024-09-10T09:07:00.819 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-09-10T09:07:00.819 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.819 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.819 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.819 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.819 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-09-10T09:07:00.819 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 0, 2024-09-10T09:07:00.819 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-09-10T09:07:00.819 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-09-10T09:07:00.819 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 0, 2024-09-10T09:07:00.819 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 0, 2024-09-10T09:07:00.819 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-09-10T09:07:00.820 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 23, 2024-09-10T09:07:00.820 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 0, 2024-09-10T09:07:00.820 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 23, 2024-09-10T09:07:00.820 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 0, 2024-09-10T09:07:00.820 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 23, 2024-09-10T09:07:00.820 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 0, 2024-09-10T09:07:00.820 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 0, 2024-09-10T09:07:00.820 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 0, 2024-09-10T09:07:00.820 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 1, 2024-09-10T09:07:00.820 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 0, 2024-09-10T09:07:00.820 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 0, 2024-09-10T09:07:00.820 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 0, 2024-09-10T09:07:00.821 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 0, 2024-09-10T09:07:00.821 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-09-10T09:07:00.821 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-09-10T09:07:00.821 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 0, 2024-09-10T09:07:00.821 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 0, 2024-09-10T09:07:00.821 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 0, 2024-09-10T09:07:00.821 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 0, 2024-09-10T09:07:00.821 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 0, 2024-09-10T09:07:00.821 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 0, 2024-09-10T09:07:00.821 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 0, 2024-09-10T09:07:00.821 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-09-10T09:07:00.822 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 0, 2024-09-10T09:07:00.822 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 0, 2024-09-10T09:07:00.822 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 0, 2024-09-10T09:07:00.822 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 0, 2024-09-10T09:07:00.822 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 0, 2024-09-10T09:07:00.822 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T09:07:00.822 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-09-10T09:07:00.822 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T09:07:00.822 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-09-10T09:07:00.822 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 0, 2024-09-10T09:07:00.822 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T09:07:00.822 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T09:07:00.823 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 0, 2024-09-10T09:07:00.823 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 0, 2024-09-10T09:07:00.823 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T09:07:00.823 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 0, 2024-09-10T09:07:00.823 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T09:07:00.823 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T09:07:00.823 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T09:07:00.823 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 0, 2024-09-10T09:07:00.823 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 0, 2024-09-10T09:07:00.823 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 0, 2024-09-10T09:07:00.823 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 0 2024-09-10T09:07:00.823 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.823 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-09-10T09:07:00.824 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-09-10T09:07:00.824 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 0, 2024-09-10T09:07:00.824 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 0, 2024-09-10T09:07:00.824 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 0, 2024-09-10T09:07:00.824 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 0, 2024-09-10T09:07:00.824 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 0, 2024-09-10T09:07:00.824 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-09-10T09:07:00.824 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-09-10T09:07:00.824 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 0, 2024-09-10T09:07:00.824 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 0, 2024-09-10T09:07:00.824 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 0, 2024-09-10T09:07:00.824 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-09-10T09:07:00.824 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-09-10T09:07:00.825 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-09-10T09:07:00.825 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-09-10T09:07:00.825 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-09-10T09:07:00.825 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-09-10T09:07:00.825 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_quiesce_path": 0, 2024-09-10T09:07:00.825 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_quiesce_inode": 0, 2024-09-10T09:07:00.825 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T09:07:00.825 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 0, 2024-09-10T09:07:00.825 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-09-10T09:07:00.825 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 0, 2024-09-10T09:07:00.825 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-09-10T09:07:00.825 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-09-10T09:07:00.825 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.826 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-09-10T09:07:00.826 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-09-10T09:07:00.826 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 0, 2024-09-10T09:07:00.826 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 0, 2024-09-10T09:07:00.826 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 0, 2024-09-10T09:07:00.826 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 1615, 2024-09-10T09:07:00.826 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-09-10T09:07:00.826 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 0, 2024-09-10T09:07:00.826 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 0, 2024-09-10T09:07:00.826 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 0, 2024-09-10T09:07:00.826 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 0, 2024-09-10T09:07:00.826 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 3, 2024-09-10T09:07:00.826 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 2, 2024-09-10T09:07:00.827 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-09-10T09:07:00.827 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 0, 2024-09-10T09:07:00.827 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 4194304, 2024-09-10T09:07:00.827 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 7273244, 2024-09-10T09:07:00.827 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 7271021, 2024-09-10T09:07:00.827 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-09-10T09:07:00.827 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.827 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.827 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.827 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.827 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 1615 2024-09-10T09:07:00.827 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.827 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-09-10T09:07:00.827 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 4, 2024-09-10T09:07:00.828 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 138, 2024-09-10T09:07:00.828 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 134, 2024-09-10T09:07:00.828 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 2, 2024-09-10T09:07:00.828 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 24, 2024-09-10T09:07:00.828 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 22, 2024-09-10T09:07:00.828 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 23, 2024-09-10T09:07:00.828 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 162, 2024-09-10T09:07:00.828 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 139, 2024-09-10T09:07:00.828 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 0, 2024-09-10T09:07:00.828 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 0, 2024-09-10T09:07:00.829 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 0, 2024-09-10T09:07:00.829 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 49952, 2024-09-10T09:07:00.829 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 215328 2024-09-10T09:07:00.829 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.829 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-09-10T09:07:00.829 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 0, 2024-09-10T09:07:00.829 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 0, 2024-09-10T09:07:00.829 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 0, 2024-09-10T09:07:00.829 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 0, 2024-09-10T09:07:00.829 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 0, 2024-09-10T09:07:00.829 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-09-10T09:07:00.829 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.829 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.830 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.830 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.830 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-09-10T09:07:00.830 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.830 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.830 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.830 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.830 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-09-10T09:07:00.830 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.830 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.830 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.830 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.830 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-09-10T09:07:00.831 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.831 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.831 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.831 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.831 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-09-10T09:07:00.831 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.831 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.831 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.831 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.831 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-09-10T09:07:00.831 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.831 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.831 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.832 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.832 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-09-10T09:07:00.832 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.832 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.832 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.832 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.832 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-09-10T09:07:00.832 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.832 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.832 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.832 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.832 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-09-10T09:07:00.833 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.833 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.833 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.833 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.833 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-09-10T09:07:00.833 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.833 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.833 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.833 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.833 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-09-10T09:07:00.833 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.833 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.833 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.834 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.834 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-09-10T09:07:00.834 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.834 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.834 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.834 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.834 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-09-10T09:07:00.834 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.834 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.834 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.834 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.834 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-09-10T09:07:00.834 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.835 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.835 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.835 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.835 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-09-10T09:07:00.835 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.835 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.835 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.835 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.835 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-09-10T09:07:00.835 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.835 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.835 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.836 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.836 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-09-10T09:07:00.836 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.836 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.836 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.836 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.836 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-09-10T09:07:00.836 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.836 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.836 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.836 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.837 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-09-10T09:07:00.837 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.837 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.837 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.837 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.837 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-09-10T09:07:00.837 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.837 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.837 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.837 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.837 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-09-10T09:07:00.838 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.838 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.838 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.838 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.838 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-09-10T09:07:00.838 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.838 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.838 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.838 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.838 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-09-10T09:07:00.838 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.839 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.839 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.839 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.839 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-09-10T09:07:00.839 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.839 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.839 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.839 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.839 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-09-10T09:07:00.839 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.839 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.839 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.840 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.840 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-09-10T09:07:00.840 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.840 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.840 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.840 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.840 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-09-10T09:07:00.840 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.840 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.840 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.840 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.840 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-09-10T09:07:00.841 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.841 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.841 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.841 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.841 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-09-10T09:07:00.841 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.841 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.841 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.841 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.841 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-09-10T09:07:00.841 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-09-10T09:07:00.842 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-09-10T09:07:00.842 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.842 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.842 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.842 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:00.842 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.842 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-09-10T09:07:00.842 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 1, 2024-09-10T09:07:00.842 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 1, 2024-09-10T09:07:00.842 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 0, 2024-09-10T09:07:00.842 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 1, 2024-09-10T09:07:00.842 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-09-10T09:07:00.843 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 0, 2024-09-10T09:07:00.843 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 0, 2024-09-10T09:07:00.843 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 2348, 2024-09-10T09:07:00.843 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-09-10T09:07:00.843 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.843 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-09-10T09:07:00.843 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-09-10T09:07:00.843 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-09-10T09:07:00.843 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T09:07:00.843 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-09-10T09:07:00.843 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T09:07:00.843 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-09-10T09:07:00.844 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T09:07:00.844 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T09:07:00.844 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T09:07:00.844 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T09:07:00.844 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T09:07:00.844 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-09-10T09:07:00.844 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T09:07:00.844 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T09:07:00.844 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-09-10T09:07:00.844 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-09-10T09:07:00.844 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-09-10T09:07:00.845 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-09-10T09:07:00.845 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T09:07:00.845 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T09:07:00.845 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T09:07:00.845 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T09:07:00.845 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T09:07:00.845 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-09-10T09:07:00.845 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-09-10T09:07:00.845 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-09-10T09:07:00.845 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T09:07:00.845 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T09:07:00.845 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-09-10T09:07:00.846 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-09-10T09:07:00.846 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-09-10T09:07:00.846 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-09-10T09:07:00.846 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T09:07:00.846 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-09-10T09:07:00.846 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T09:07:00.846 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-09-10T09:07:00.846 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 190224, 2024-09-10T09:07:00.846 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 54, 2024-09-10T09:07:00.846 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 0, 2024-09-10T09:07:00.846 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 0, 2024-09-10T09:07:00.846 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-09-10T09:07:00.846 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-09-10T09:07:00.847 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-09-10T09:07:00.847 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-09-10T09:07:00.847 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-09-10T09:07:00.847 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-09-10T09:07:00.847 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 8672, 2024-09-10T09:07:00.847 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 163, 2024-09-10T09:07:00.847 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T09:07:00.847 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-09-10T09:07:00.847 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-09-10T09:07:00.847 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-09-10T09:07:00.847 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 28240, 2024-09-10T09:07:00.847 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 120, 2024-09-10T09:07:00.847 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-09-10T09:07:00.848 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-09-10T09:07:00.848 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-09-10T09:07:00.848 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-09-10T09:07:00.848 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.848 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-09-10T09:07:00.848 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-09-10T09:07:00.848 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-09-10T09:07:00.848 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 4245, 2024-09-10T09:07:00.848 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 21, 2024-09-10T09:07:00.848 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 3, 2024-09-10T09:07:00.848 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 4245, 2024-09-10T09:07:00.848 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-09-10T09:07:00.849 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 4245, 2024-09-10T09:07:00.849 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 64.478799447, 2024-09-10T09:07:00.849 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.015189352 2024-09-10T09:07:00.849 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.849 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-09-10T09:07:00.849 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-09-10T09:07:00.849 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 4245, 2024-09-10T09:07:00.849 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 4246 2024-09-10T09:07:00.849 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.849 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 4245, 2024-09-10T09:07:00.849 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 4245, 2024-09-10T09:07:00.850 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 0, 2024-09-10T09:07:00.850 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-09-10T09:07:00.850 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-09-10T09:07:00.850 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 2427, 2024-09-10T09:07:00.850 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 0, 2024-09-10T09:07:00.850 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 1816, 2024-09-10T09:07:00.850 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 0, 2024-09-10T09:07:00.850 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 0, 2024-09-10T09:07:00.850 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-09-10T09:07:00.850 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-09-10T09:07:00.850 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-09-10T09:07:00.850 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 0, 2024-09-10T09:07:00.851 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 0, 2024-09-10T09:07:00.851 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-09-10T09:07:00.851 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-09-10T09:07:00.851 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-09-10T09:07:00.851 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 1, 2024-09-10T09:07:00.851 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 0, 2024-09-10T09:07:00.851 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-09-10T09:07:00.851 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-09-10T09:07:00.851 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-09-10T09:07:00.851 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-09-10T09:07:00.851 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-09-10T09:07:00.851 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-09-10T09:07:00.851 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T09:07:00.852 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-09-10T09:07:00.852 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-09-10T09:07:00.852 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 0, 2024-09-10T09:07:00.852 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-09-10T09:07:00.852 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-09-10T09:07:00.852 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-09-10T09:07:00.852 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-09-10T09:07:00.852 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-09-10T09:07:00.852 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-09-10T09:07:00.852 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-09-10T09:07:00.852 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-09-10T09:07:00.852 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-09-10T09:07:00.853 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-09-10T09:07:00.853 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-09-10T09:07:00.853 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-09-10T09:07:00.853 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-09-10T09:07:00.853 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-09-10T09:07:00.853 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-09-10T09:07:00.853 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-09-10T09:07:00.853 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 92, 2024-09-10T09:07:00.853 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-09-10T09:07:00.853 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 4, 2024-09-10T09:07:00.853 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 5, 2024-09-10T09:07:00.853 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 6, 2024-09-10T09:07:00.853 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 1, 2024-09-10T09:07:00.854 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-09-10T09:07:00.854 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 0, 2024-09-10T09:07:00.854 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 2, 2024-09-10T09:07:00.854 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 0 2024-09-10T09:07:00.854 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.854 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-09-10T09:07:00.854 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 0, 2024-09-10T09:07:00.854 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-09-10T09:07:00.854 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-09-10T09:07:00.854 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-09-10T09:07:00.854 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.854 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-09-10T09:07:00.855 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 0 2024-09-10T09:07:00.855 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.855 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-09-10T09:07:00.855 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-09-10T09:07:00.855 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T09:07:00.855 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-09-10T09:07:00.855 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 0, 2024-09-10T09:07:00.855 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 0, 2024-09-10T09:07:00.855 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 0, 2024-09-10T09:07:00.855 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-09-10T09:07:00.855 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.855 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T09:07:00.855 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:07:00.856 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-09-10T09:07:00.856 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:07:00.856 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 4655, 2024-09-10T09:07:00.856 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 4526027, 2024-09-10T09:07:00.856 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:07:00.856 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 4655, 2024-09-10T09:07:00.856 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-10T09:07:00.856 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-10T09:07:00.856 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 4655, 2024-09-10T09:07:00.856 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 4526027, 2024-09-10T09:07:00.857 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:07:00.857 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.857 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.857 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.857 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:00.857 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.857 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-09-10T09:07:00.857 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:07:00.857 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-09-10T09:07:00.857 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:07:00.858 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-10T09:07:00.858 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-10T09:07:00.858 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:07:00.858 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-10T09:07:00.858 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 4245, 2024-09-10T09:07:00.858 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 3078945, 2024-09-10T09:07:00.858 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 4245, 2024-09-10T09:07:00.858 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 3078945, 2024-09-10T09:07:00.858 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:07:00.858 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.858 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.858 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.859 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:00.859 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.859 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-09-10T09:07:00.859 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:07:00.859 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-09-10T09:07:00.859 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:07:00.859 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-10T09:07:00.859 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-10T09:07:00.860 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:07:00.860 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-10T09:07:00.860 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 4245, 2024-09-10T09:07:00.860 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 4245, 2024-09-10T09:07:00.860 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 4245, 2024-09-10T09:07:00.860 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 4245, 2024-09-10T09:07:00.860 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:07:00.860 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.860 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.860 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.860 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:00.861 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.861 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-09-10T09:07:00.861 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:07:00.861 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-09-10T09:07:00.861 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:07:00.861 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-10T09:07:00.861 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-10T09:07:00.861 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:07:00.861 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-10T09:07:00.861 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-10T09:07:00.861 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-10T09:07:00.861 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-09-10T09:07:00.862 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-09-10T09:07:00.862 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:07:00.862 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.862 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.862 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.862 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:00.862 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:00.862 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x55839bfdeb40": { 2024-09-10T09:07:00.862 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:07:00.862 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-09-10T09:07:00.863 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:07:00.863 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-10T09:07:00.863 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-10T09:07:00.863 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:07:00.863 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-10T09:07:00.863 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-10T09:07:00.863 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-10T09:07:00.863 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-09-10T09:07:00.863 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-09-10T09:07:00.863 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:07:00.863 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:00.863 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:00.863 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:00.863 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:00.864 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:00.864 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-10T09:07:00.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:00 smithi174 ceph-mon[34068]: pgmap v973: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T09:07:01.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:00 smithi172 ceph-mon[30854]: pgmap v973: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T09:07:01.755 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-10T09:07:01.755 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-10T09:07:01.755 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-10T09:07:01.755 DEBUG:tasks.check_counter:Getting stats from i 2024-09-10T09:07:01.756 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-10T09:07:02.009 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:07:02.593 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-10T09:07:02.593 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T09:07:02.593 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 3463, 2024-09-10T09:07:02.594 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 3598, 2024-09-10T09:07:02.594 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 992033, 2024-09-10T09:07:02.594 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 4625863, 2024-09-10T09:07:02.594 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 21, 2024-09-10T09:07:02.594 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 5, 2024-09-10T09:07:02.594 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.307012509, 2024-09-10T09:07:02.594 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.100774360, 2024-09-10T09:07:02.594 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.127736101, 2024-09-10T09:07:02.594 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.066531979, 2024-09-10T09:07:02.594 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:07:02.594 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 3596, 2024-09-10T09:07:02.594 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.164658947, 2024-09-10T09:07:02.594 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000045789 2024-09-10T09:07:02.595 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.595 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:07:02.595 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2897, 2024-09-10T09:07:02.595 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000627937, 2024-09-10T09:07:02.595 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000216 2024-09-10T09:07:02.595 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.595 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T09:07:02.595 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T09:07:02.595 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.595 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T09:07:02.596 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 2323, 2024-09-10T09:07:02.596 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 2543, 2024-09-10T09:07:02.596 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 870138, 2024-09-10T09:07:02.596 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 1708155, 2024-09-10T09:07:02.596 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 10, 2024-09-10T09:07:02.596 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 7, 2024-09-10T09:07:02.596 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.175971577, 2024-09-10T09:07:02.596 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.068598553, 2024-09-10T09:07:02.596 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.090549026, 2024-09-10T09:07:02.596 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.022941141, 2024-09-10T09:07:02.596 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:07:02.596 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2541, 2024-09-10T09:07:02.597 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.129422028, 2024-09-10T09:07:02.597 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000050933 2024-09-10T09:07:02.597 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.597 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:07:02.597 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 3019, 2024-09-10T09:07:02.597 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000634864, 2024-09-10T09:07:02.597 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000210 2024-09-10T09:07:02.597 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.597 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T09:07:02.597 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 568848 2024-09-10T09:07:02.597 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.597 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T09:07:02.598 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 3222, 2024-09-10T09:07:02.598 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 4722, 2024-09-10T09:07:02.598 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 1722431, 2024-09-10T09:07:02.598 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 2459138, 2024-09-10T09:07:02.598 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 43, 2024-09-10T09:07:02.598 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 5, 2024-09-10T09:07:02.598 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.340637206, 2024-09-10T09:07:02.598 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.135211709, 2024-09-10T09:07:02.598 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.233437102, 2024-09-10T09:07:02.598 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.005596684, 2024-09-10T09:07:02.598 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:07:02.599 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 4722, 2024-09-10T09:07:02.599 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.153705410, 2024-09-10T09:07:02.599 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000032550 2024-09-10T09:07:02.599 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.599 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:07:02.599 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 6224, 2024-09-10T09:07:02.599 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.001249756, 2024-09-10T09:07:02.599 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000200 2024-09-10T09:07:02.599 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.599 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 722128, 2024-09-10T09:07:02.599 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 249248 2024-09-10T09:07:02.599 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.600 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-09-10T09:07:02.600 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-09-10T09:07:02.600 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-09-10T09:07:02.600 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.600 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-09-10T09:07:02.600 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-09-10T09:07:02.600 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-09-10T09:07:02.600 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 3377, 2024-09-10T09:07:02.600 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.620926326, 2024-09-10T09:07:02.600 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000183869 2024-09-10T09:07:02.600 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:02.600 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.601 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-09-10T09:07:02.601 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-09-10T09:07:02.601 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-09-10T09:07:02.601 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 550, 2024-09-10T09:07:02.601 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.018816106, 2024-09-10T09:07:02.601 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000034211 2024-09-10T09:07:02.601 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:02.601 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.601 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-09-10T09:07:02.601 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 751, 2024-09-10T09:07:02.601 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 594, 2024-09-10T09:07:02.601 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-09-10T09:07:02.601 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 594, 2024-09-10T09:07:02.602 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 17.563084761, 2024-09-10T09:07:02.602 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.029567482 2024-09-10T09:07:02.602 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.602 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-09-10T09:07:02.602 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 157, 2024-09-10T09:07:02.602 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-09-10T09:07:02.602 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-09-10T09:07:02.602 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 19, 2024-09-10T09:07:02.602 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 1, 2024-09-10T09:07:02.602 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-09-10T09:07:02.602 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 14, 2024-09-10T09:07:02.602 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 0, 2024-09-10T09:07:02.602 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 2, 2024-09-10T09:07:02.603 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 12, 2024-09-10T09:07:02.603 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 12, 2024-09-10T09:07:02.603 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 113, 2024-09-10T09:07:02.603 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 1, 2024-09-10T09:07:02.603 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 1, 2024-09-10T09:07:02.603 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 3, 2024-09-10T09:07:02.603 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 1173, 2024-09-10T09:07:02.603 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 657, 2024-09-10T09:07:02.603 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 144, 2024-09-10T09:07:02.603 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 4, 2024-09-10T09:07:02.603 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-09-10T09:07:02.604 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-09-10T09:07:02.604 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 0, 2024-09-10T09:07:02.604 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 0, 2024-09-10T09:07:02.604 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 0, 2024-09-10T09:07:02.604 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 9, 2024-09-10T09:07:02.604 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 102, 2024-09-10T09:07:02.604 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 9, 2024-09-10T09:07:02.604 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 14, 2024-09-10T09:07:02.604 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-09-10T09:07:02.604 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 746, 2024-09-10T09:07:02.604 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 0, 2024-09-10T09:07:02.604 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 325, 2024-09-10T09:07:02.604 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 13424230450, 2024-09-10T09:07:02.605 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 0, 2024-09-10T09:07:02.605 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 746, 2024-09-10T09:07:02.605 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-09-10T09:07:02.605 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T09:07:02.605 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-09-10T09:07:02.605 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 30, 2024-09-10T09:07:02.605 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T09:07:02.605 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 30, 2024-09-10T09:07:02.605 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 716, 2024-09-10T09:07:02.605 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 8, 2024-09-10T09:07:02.605 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 282, 2024-09-10T09:07:02.605 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 1191, 2024-09-10T09:07:02.605 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T09:07:02.605 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T09:07:02.606 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T09:07:02.606 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 1466, 2024-09-10T09:07:02.606 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 1169, 2024-09-10T09:07:02.606 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 3, 2024-09-10T09:07:02.606 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 162 2024-09-10T09:07:02.606 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.606 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-09-10T09:07:02.606 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-09-10T09:07:02.606 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 0, 2024-09-10T09:07:02.606 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 0, 2024-09-10T09:07:02.606 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 4, 2024-09-10T09:07:02.606 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 23, 2024-09-10T09:07:02.606 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 0, 2024-09-10T09:07:02.607 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-09-10T09:07:02.607 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-09-10T09:07:02.607 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 22, 2024-09-10T09:07:02.607 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 22, 2024-09-10T09:07:02.607 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 0, 2024-09-10T09:07:02.607 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-09-10T09:07:02.607 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-09-10T09:07:02.607 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-09-10T09:07:02.607 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-09-10T09:07:02.607 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-09-10T09:07:02.607 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-09-10T09:07:02.607 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_quiesce_path": 15, 2024-09-10T09:07:02.607 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_quiesce_inode": 802, 2024-09-10T09:07:02.608 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T09:07:02.608 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 9, 2024-09-10T09:07:02.608 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-09-10T09:07:02.608 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 1, 2024-09-10T09:07:02.608 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-09-10T09:07:02.608 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-09-10T09:07:02.608 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.608 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-09-10T09:07:02.608 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-09-10T09:07:02.608 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 1615, 2024-09-10T09:07:02.608 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 0, 2024-09-10T09:07:02.608 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 0, 2024-09-10T09:07:02.608 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 1615, 2024-09-10T09:07:02.608 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-09-10T09:07:02.609 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 0, 2024-09-10T09:07:02.609 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 3, 2024-09-10T09:07:02.609 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 0, 2024-09-10T09:07:02.609 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 0, 2024-09-10T09:07:02.609 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 3, 2024-09-10T09:07:02.609 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 2, 2024-09-10T09:07:02.609 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-09-10T09:07:02.609 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 0, 2024-09-10T09:07:02.609 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 4194304, 2024-09-10T09:07:02.609 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 7273244, 2024-09-10T09:07:02.609 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 0, 2024-09-10T09:07:02.609 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-09-10T09:07:02.609 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1234, 2024-09-10T09:07:02.609 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 5.223103958, 2024-09-10T09:07:02.610 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.004232661 2024-09-10T09:07:02.610 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.610 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 0 2024-09-10T09:07:02.610 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.610 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-09-10T09:07:02.610 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 18, 2024-09-10T09:07:02.610 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 147, 2024-09-10T09:07:02.610 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 129, 2024-09-10T09:07:02.610 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 16, 2024-09-10T09:07:02.610 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 34, 2024-09-10T09:07:02.610 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 18, 2024-09-10T09:07:02.610 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 14, 2024-09-10T09:07:02.610 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 175, 2024-09-10T09:07:02.610 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 161, 2024-09-10T09:07:02.610 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 1, 2024-09-10T09:07:02.611 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 127, 2024-09-10T09:07:02.611 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 126, 2024-09-10T09:07:02.611 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 59088, 2024-09-10T09:07:02.611 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 223520 2024-09-10T09:07:02.611 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.611 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-09-10T09:07:02.611 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 817, 2024-09-10T09:07:02.611 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 143, 2024-09-10T09:07:02.611 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 751, 2024-09-10T09:07:02.611 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 23, 2024-09-10T09:07:02.611 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 299, 2024-09-10T09:07:02.611 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-09-10T09:07:02.611 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 114, 2024-09-10T09:07:02.611 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.117112797, 2024-09-10T09:07:02.611 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.001027305 2024-09-10T09:07:02.612 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.612 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-09-10T09:07:02.612 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 8, 2024-09-10T09:07:02.612 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 14.531063029, 2024-09-10T09:07:02.612 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 1.816382878 2024-09-10T09:07:02.612 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.612 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-09-10T09:07:02.612 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.612 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.612 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.612 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.612 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-09-10T09:07:02.612 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.612 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.613 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.613 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.613 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-09-10T09:07:02.613 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 348, 2024-09-10T09:07:02.613 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 2.795342916, 2024-09-10T09:07:02.613 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.008032594 2024-09-10T09:07:02.613 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.613 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-09-10T09:07:02.613 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.613 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.613 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.613 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.613 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-09-10T09:07:02.614 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.614 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.614 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.614 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.614 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-09-10T09:07:02.614 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.614 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.614 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.614 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.614 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-09-10T09:07:02.614 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.614 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.614 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.614 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.615 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-09-10T09:07:02.615 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.615 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.615 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.615 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.615 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-09-10T09:07:02.615 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.615 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.616 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.616 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.616 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-09-10T09:07:02.616 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.616 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.616 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.616 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.616 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-09-10T09:07:02.616 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.616 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.617 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.617 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.617 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-09-10T09:07:02.617 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.617 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.617 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.617 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.617 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-09-10T09:07:02.617 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 99, 2024-09-10T09:07:02.617 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.080454865, 2024-09-10T09:07:02.617 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000812675 2024-09-10T09:07:02.617 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.617 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-09-10T09:07:02.618 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 3, 2024-09-10T09:07:02.618 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.001658292, 2024-09-10T09:07:02.618 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000552764 2024-09-10T09:07:02.618 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.618 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-09-10T09:07:02.618 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.618 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.618 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.618 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.618 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-09-10T09:07:02.618 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.619 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.619 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.619 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.619 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-09-10T09:07:02.619 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.619 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.619 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.619 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.619 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-09-10T09:07:02.619 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2, 2024-09-10T09:07:02.619 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.012468725, 2024-09-10T09:07:02.619 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.006234362 2024-09-10T09:07:02.619 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.619 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-09-10T09:07:02.620 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.620 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.620 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.620 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.620 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-09-10T09:07:02.620 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.620 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.620 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.620 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.620 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-09-10T09:07:02.620 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.620 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.620 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.621 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.621 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-09-10T09:07:02.621 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.621 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.621 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.621 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.621 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-09-10T09:07:02.621 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.621 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.621 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.621 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.621 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-09-10T09:07:02.621 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.622 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.622 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.622 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.622 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-09-10T09:07:02.622 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.622 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.622 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.622 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.622 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-09-10T09:07:02.622 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.622 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.622 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.622 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.622 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-09-10T09:07:02.622 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 20, 2024-09-10T09:07:02.623 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.024984137, 2024-09-10T09:07:02.623 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.001249206 2024-09-10T09:07:02.623 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.623 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-09-10T09:07:02.623 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-09-10T09:07:02.623 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-09-10T09:07:02.623 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.623 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.623 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.623 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:02.623 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.623 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-09-10T09:07:02.623 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 1, 2024-09-10T09:07:02.623 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 1, 2024-09-10T09:07:02.624 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 0, 2024-09-10T09:07:02.624 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 1, 2024-09-10T09:07:02.624 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-09-10T09:07:02.624 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 58, 2024-09-10T09:07:02.624 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 58, 2024-09-10T09:07:02.624 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 1223, 2024-09-10T09:07:02.624 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-09-10T09:07:02.624 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.624 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-09-10T09:07:02.624 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-09-10T09:07:02.624 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-09-10T09:07:02.624 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T09:07:02.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-09-10T09:07:02.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T09:07:02.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-09-10T09:07:02.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T09:07:02.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T09:07:02.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T09:07:02.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T09:07:02.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T09:07:02.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-09-10T09:07:02.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T09:07:02.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T09:07:02.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-09-10T09:07:02.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-09-10T09:07:02.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-09-10T09:07:02.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-09-10T09:07:02.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T09:07:02.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T09:07:02.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T09:07:02.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T09:07:02.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T09:07:02.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-09-10T09:07:02.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-09-10T09:07:02.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-09-10T09:07:02.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T09:07:02.627 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T09:07:02.627 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-09-10T09:07:02.627 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-09-10T09:07:02.627 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-09-10T09:07:02.627 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-09-10T09:07:02.627 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T09:07:02.627 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-09-10T09:07:02.627 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T09:07:02.627 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-09-10T09:07:02.627 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 287079, 2024-09-10T09:07:02.627 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 97, 2024-09-10T09:07:02.627 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 0, 2024-09-10T09:07:02.627 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 0, 2024-09-10T09:07:02.627 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-09-10T09:07:02.628 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-09-10T09:07:02.628 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-09-10T09:07:02.628 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-09-10T09:07:02.628 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-09-10T09:07:02.628 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-09-10T09:07:02.628 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 8672, 2024-09-10T09:07:02.628 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 163, 2024-09-10T09:07:02.628 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T09:07:02.628 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-09-10T09:07:02.628 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-09-10T09:07:02.628 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-09-10T09:07:02.628 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 72464, 2024-09-10T09:07:02.629 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 302, 2024-09-10T09:07:02.629 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-09-10T09:07:02.629 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-09-10T09:07:02.629 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-09-10T09:07:02.629 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-09-10T09:07:02.629 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.629 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-09-10T09:07:02.629 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-09-10T09:07:02.629 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-09-10T09:07:02.629 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 2939, 2024-09-10T09:07:02.629 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 3189381, 2024-09-10T09:07:02.629 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 30, 2024-09-10T09:07:02.629 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 2939, 2024-09-10T09:07:02.630 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-09-10T09:07:02.630 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2939, 2024-09-10T09:07:02.630 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 53.402021742, 2024-09-10T09:07:02.630 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.018170133 2024-09-10T09:07:02.630 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.630 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-09-10T09:07:02.630 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-09-10T09:07:02.630 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2939, 2024-09-10T09:07:02.630 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 3302 2024-09-10T09:07:02.630 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.630 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 2939, 2024-09-10T09:07:02.630 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 748, 2024-09-10T09:07:02.630 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 2191, 2024-09-10T09:07:02.630 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-09-10T09:07:02.631 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-09-10T09:07:02.631 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 0, 2024-09-10T09:07:02.631 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 10, 2024-09-10T09:07:02.631 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 2, 2024-09-10T09:07:02.631 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 1236, 2024-09-10T09:07:02.631 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 65, 2024-09-10T09:07:02.631 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-09-10T09:07:02.631 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-09-10T09:07:02.631 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-09-10T09:07:02.631 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 1, 2024-09-10T09:07:02.631 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 542, 2024-09-10T09:07:02.631 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-09-10T09:07:02.631 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-09-10T09:07:02.632 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-09-10T09:07:02.632 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 746, 2024-09-10T09:07:02.632 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 20, 2024-09-10T09:07:02.632 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-09-10T09:07:02.632 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-09-10T09:07:02.632 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-09-10T09:07:02.632 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-09-10T09:07:02.632 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-09-10T09:07:02.632 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-09-10T09:07:02.632 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T09:07:02.632 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-09-10T09:07:02.632 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-09-10T09:07:02.632 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 0, 2024-09-10T09:07:02.633 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-09-10T09:07:02.633 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-09-10T09:07:02.633 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-09-10T09:07:02.633 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-09-10T09:07:02.633 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-09-10T09:07:02.633 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-09-10T09:07:02.633 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-09-10T09:07:02.633 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-09-10T09:07:02.633 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-09-10T09:07:02.633 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-09-10T09:07:02.633 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-09-10T09:07:02.633 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-09-10T09:07:02.633 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-09-10T09:07:02.634 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-09-10T09:07:02.634 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-09-10T09:07:02.634 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-09-10T09:07:02.634 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 92, 2024-09-10T09:07:02.634 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-09-10T09:07:02.634 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 4, 2024-09-10T09:07:02.634 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 12, 2024-09-10T09:07:02.634 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 12, 2024-09-10T09:07:02.635 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 0, 2024-09-10T09:07:02.635 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-09-10T09:07:02.635 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 526, 2024-09-10T09:07:02.635 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 0, 2024-09-10T09:07:02.635 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 154 2024-09-10T09:07:02.635 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.635 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-09-10T09:07:02.635 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 1, 2024-09-10T09:07:02.635 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-09-10T09:07:02.635 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-09-10T09:07:02.635 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-09-10T09:07:02.636 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.636 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-09-10T09:07:02.636 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 739 2024-09-10T09:07:02.636 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.636 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-09-10T09:07:02.636 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-09-10T09:07:02.636 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 27, 2024-09-10T09:07:02.637 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-09-10T09:07:02.637 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 2, 2024-09-10T09:07:02.637 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 530, 2024-09-10T09:07:02.637 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 22, 2024-09-10T09:07:02.637 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-09-10T09:07:02.637 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.637 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T09:07:02.637 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:07:02.637 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-09-10T09:07:02.637 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:07:02.637 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 9004, 2024-09-10T09:07:02.638 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 2747880, 2024-09-10T09:07:02.638 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:07:02.638 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 9004, 2024-09-10T09:07:02.638 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-10T09:07:02.638 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-10T09:07:02.638 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 9004, 2024-09-10T09:07:02.638 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 2747880, 2024-09-10T09:07:02.638 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:07:02.638 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.638 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.638 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.638 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:02.638 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.639 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-09-10T09:07:02.639 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:07:02.639 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-09-10T09:07:02.639 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:07:02.639 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-10T09:07:02.639 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-10T09:07:02.639 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:07:02.639 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-10T09:07:02.639 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 2939, 2024-09-10T09:07:02.639 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 3191771, 2024-09-10T09:07:02.639 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2939, 2024-09-10T09:07:02.639 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 3191771, 2024-09-10T09:07:02.639 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:07:02.640 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.640 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.640 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.640 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:02.640 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.640 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-09-10T09:07:02.640 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:07:02.640 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-09-10T09:07:02.640 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:07:02.640 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-10T09:07:02.640 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-10T09:07:02.640 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:07:02.641 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-10T09:07:02.641 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 2939, 2024-09-10T09:07:02.641 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 2939, 2024-09-10T09:07:02.641 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2939, 2024-09-10T09:07:02.641 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 2939, 2024-09-10T09:07:02.641 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:07:02.641 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.641 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.641 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.641 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:02.641 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.641 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-09-10T09:07:02.642 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:07:02.642 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-09-10T09:07:02.642 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:07:02.642 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 22, 2024-09-10T09:07:02.642 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 2390, 2024-09-10T09:07:02.642 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:07:02.642 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 22, 2024-09-10T09:07:02.642 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-10T09:07:02.642 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-10T09:07:02.642 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2, 2024-09-10T09:07:02.642 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 2390, 2024-09-10T09:07:02.642 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:07:02.643 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.643 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.643 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.643 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:02.643 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:02.643 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x5573a13a2960": { 2024-09-10T09:07:02.643 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:07:02.643 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-09-10T09:07:02.643 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:07:02.643 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 1615, 2024-09-10T09:07:02.643 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 3078940, 2024-09-10T09:07:02.643 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:07:02.643 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 1615, 2024-09-10T09:07:02.643 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-10T09:07:02.644 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-10T09:07:02.644 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 1234, 2024-09-10T09:07:02.644 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 3078940, 2024-09-10T09:07:02.644 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:07:02.644 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:02.644 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:02.644 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:02.644 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:02.644 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:02.644 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-10T09:07:02.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:02 smithi191 ceph-mon[35314]: pgmap v974: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:07:02.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:02 smithi174 ceph-mon[34068]: pgmap v974: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:07:03.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:02 smithi172 ceph-mon[30854]: pgmap v974: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:07:03.363 INFO:tasks.check_counter:Daemon mds.i mds.exported=9 2024-09-10T09:07:03.363 INFO:tasks.check_counter:Daemon mds.i mds.imported=9 2024-09-10T09:07:03.363 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=1 2024-09-10T09:07:03.363 DEBUG:tasks.check_counter:Getting stats from l 2024-09-10T09:07:03.363 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-10T09:07:03.613 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:07:04.183 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-10T09:07:04.183 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T09:07:04.183 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 1756, 2024-09-10T09:07:04.183 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 1904, 2024-09-10T09:07:04.184 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 740362, 2024-09-10T09:07:04.184 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 1476878, 2024-09-10T09:07:04.184 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 12, 2024-09-10T09:07:04.184 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 4, 2024-09-10T09:07:04.184 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.182504237, 2024-09-10T09:07:04.184 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.064213296, 2024-09-10T09:07:04.184 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.085992152, 2024-09-10T09:07:04.184 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.029227960, 2024-09-10T09:07:04.184 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:07:04.184 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1902, 2024-09-10T09:07:04.184 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.147680538, 2024-09-10T09:07:04.184 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000077644 2024-09-10T09:07:04.185 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.185 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:07:04.185 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2548, 2024-09-10T09:07:04.185 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000531523, 2024-09-10T09:07:04.185 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000208 2024-09-10T09:07:04.185 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.185 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T09:07:04.185 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T09:07:04.185 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.185 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T09:07:04.185 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 1554, 2024-09-10T09:07:04.185 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 1851, 2024-09-10T09:07:04.186 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 564825, 2024-09-10T09:07:04.186 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 1549673, 2024-09-10T09:07:04.186 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 24, 2024-09-10T09:07:04.186 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 5, 2024-09-10T09:07:04.186 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.147866834, 2024-09-10T09:07:04.186 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.049462976, 2024-09-10T09:07:04.186 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.086477209, 2024-09-10T09:07:04.186 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.018475709, 2024-09-10T09:07:04.186 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:07:04.186 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1850, 2024-09-10T09:07:04.186 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.149758227, 2024-09-10T09:07:04.186 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000080950 2024-09-10T09:07:04.186 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.187 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:07:04.187 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2046, 2024-09-10T09:07:04.187 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000506821, 2024-09-10T09:07:04.187 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000247 2024-09-10T09:07:04.187 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.187 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-10T09:07:04.187 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 657744 2024-09-10T09:07:04.187 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.187 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T09:07:04.187 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 1093, 2024-09-10T09:07:04.187 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 1085, 2024-09-10T09:07:04.187 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 963967, 2024-09-10T09:07:04.187 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 943817, 2024-09-10T09:07:04.188 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 40, 2024-09-10T09:07:04.188 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 5, 2024-09-10T09:07:04.188 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.122819230, 2024-09-10T09:07:04.188 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.045330152, 2024-09-10T09:07:04.188 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.066030580, 2024-09-10T09:07:04.188 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.016402178, 2024-09-10T09:07:04.188 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T09:07:04.188 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1084, 2024-09-10T09:07:04.188 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.084265182, 2024-09-10T09:07:04.188 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000077735 2024-09-10T09:07:04.188 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.188 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-10T09:07:04.188 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 302, 2024-09-10T09:07:04.189 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000064424, 2024-09-10T09:07:04.189 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000213 2024-09-10T09:07:04.189 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.189 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 701216, 2024-09-10T09:07:04.189 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 242928 2024-09-10T09:07:04.189 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.189 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-09-10T09:07:04.189 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-09-10T09:07:04.189 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-09-10T09:07:04.189 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.189 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-09-10T09:07:04.189 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-09-10T09:07:04.190 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-09-10T09:07:04.190 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 759, 2024-09-10T09:07:04.190 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.151843022, 2024-09-10T09:07:04.190 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000200056 2024-09-10T09:07:04.190 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:04.190 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.190 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-09-10T09:07:04.190 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-09-10T09:07:04.190 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-09-10T09:07:04.190 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 827, 2024-09-10T09:07:04.190 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.027730911, 2024-09-10T09:07:04.190 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000033531 2024-09-10T09:07:04.190 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:04.191 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.191 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-09-10T09:07:04.191 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 285, 2024-09-10T09:07:04.191 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 285, 2024-09-10T09:07:04.191 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-09-10T09:07:04.191 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 285, 2024-09-10T09:07:04.191 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.237372827, 2024-09-10T09:07:04.191 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000832887 2024-09-10T09:07:04.191 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.191 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-09-10T09:07:04.191 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 0, 2024-09-10T09:07:04.191 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-09-10T09:07:04.191 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-09-10T09:07:04.192 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 11, 2024-09-10T09:07:04.192 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 0, 2024-09-10T09:07:04.192 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-09-10T09:07:04.192 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 14, 2024-09-10T09:07:04.192 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 1, 2024-09-10T09:07:04.192 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 2, 2024-09-10T09:07:04.192 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 11, 2024-09-10T09:07:04.192 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 11, 2024-09-10T09:07:04.192 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 12, 2024-09-10T09:07:04.192 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 0, 2024-09-10T09:07:04.192 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 0, 2024-09-10T09:07:04.193 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 4, 2024-09-10T09:07:04.193 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 486, 2024-09-10T09:07:04.193 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 424, 2024-09-10T09:07:04.193 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 0, 2024-09-10T09:07:04.193 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 10, 2024-09-10T09:07:04.193 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-09-10T09:07:04.193 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-09-10T09:07:04.193 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 0, 2024-09-10T09:07:04.194 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 0, 2024-09-10T09:07:04.194 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 0, 2024-09-10T09:07:04.194 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 2, 2024-09-10T09:07:04.194 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 2, 2024-09-10T09:07:04.194 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 2, 2024-09-10T09:07:04.194 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 28, 2024-09-10T09:07:04.195 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-09-10T09:07:04.195 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 153, 2024-09-10T09:07:04.195 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 0, 2024-09-10T09:07:04.195 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 325, 2024-09-10T09:07:04.195 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 13424230450, 2024-09-10T09:07:04.195 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 0, 2024-09-10T09:07:04.195 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 153, 2024-09-10T09:07:04.196 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-09-10T09:07:04.196 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T09:07:04.196 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-09-10T09:07:04.196 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 0, 2024-09-10T09:07:04.196 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T09:07:04.196 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T09:07:04.196 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 153, 2024-09-10T09:07:04.196 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 6, 2024-09-10T09:07:04.196 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 156, 2024-09-10T09:07:04.197 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 168, 2024-09-10T09:07:04.197 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T09:07:04.197 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T09:07:04.197 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T09:07:04.197 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 407, 2024-09-10T09:07:04.197 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 204, 2024-09-10T09:07:04.197 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 2, 2024-09-10T09:07:04.197 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 64 2024-09-10T09:07:04.197 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.197 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-09-10T09:07:04.197 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-09-10T09:07:04.197 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 0, 2024-09-10T09:07:04.197 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 0, 2024-09-10T09:07:04.198 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 11, 2024-09-10T09:07:04.198 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 0, 2024-09-10T09:07:04.198 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 0, 2024-09-10T09:07:04.198 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-09-10T09:07:04.198 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-09-10T09:07:04.198 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 31, 2024-09-10T09:07:04.198 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 31, 2024-09-10T09:07:04.198 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 0, 2024-09-10T09:07:04.198 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-09-10T09:07:04.198 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-09-10T09:07:04.198 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-09-10T09:07:04.198 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-09-10T09:07:04.198 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-09-10T09:07:04.199 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-09-10T09:07:04.199 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_quiesce_path": 15, 2024-09-10T09:07:04.199 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_quiesce_inode": 209, 2024-09-10T09:07:04.199 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T09:07:04.199 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 2, 2024-09-10T09:07:04.199 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-09-10T09:07:04.199 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 0, 2024-09-10T09:07:04.199 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-09-10T09:07:04.199 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-09-10T09:07:04.200 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.200 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-09-10T09:07:04.200 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-09-10T09:07:04.200 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 535, 2024-09-10T09:07:04.200 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 0, 2024-09-10T09:07:04.200 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 0, 2024-09-10T09:07:04.200 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 535, 2024-09-10T09:07:04.200 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-09-10T09:07:04.200 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 0, 2024-09-10T09:07:04.201 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 2, 2024-09-10T09:07:04.201 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 0, 2024-09-10T09:07:04.201 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 0, 2024-09-10T09:07:04.201 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 2, 2024-09-10T09:07:04.201 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 2, 2024-09-10T09:07:04.201 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-09-10T09:07:04.201 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 0, 2024-09-10T09:07:04.201 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 4194304, 2024-09-10T09:07:04.201 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 4821448, 2024-09-10T09:07:04.201 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 0, 2024-09-10T09:07:04.201 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-09-10T09:07:04.201 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 258, 2024-09-10T09:07:04.201 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 12.693433635, 2024-09-10T09:07:04.202 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.049199355 2024-09-10T09:07:04.202 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.202 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 0 2024-09-10T09:07:04.202 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.202 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-09-10T09:07:04.202 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 19, 2024-09-10T09:07:04.202 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 56, 2024-09-10T09:07:04.202 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 37, 2024-09-10T09:07:04.202 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 18, 2024-09-10T09:07:04.203 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 26, 2024-09-10T09:07:04.203 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 8, 2024-09-10T09:07:04.203 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 14, 2024-09-10T09:07:04.203 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 86, 2024-09-10T09:07:04.203 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 72, 2024-09-10T09:07:04.203 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 0, 2024-09-10T09:07:04.203 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 33, 2024-09-10T09:07:04.203 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 33, 2024-09-10T09:07:04.203 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 53652, 2024-09-10T09:07:04.203 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 223520 2024-09-10T09:07:04.203 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.203 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-09-10T09:07:04.204 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 335, 2024-09-10T09:07:04.204 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 2, 2024-09-10T09:07:04.204 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 285, 2024-09-10T09:07:04.204 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 15, 2024-09-10T09:07:04.204 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 40, 2024-09-10T09:07:04.204 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-09-10T09:07:04.204 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 3, 2024-09-10T09:07:04.204 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.009322511, 2024-09-10T09:07:04.204 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.003107503 2024-09-10T09:07:04.204 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.204 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-09-10T09:07:04.204 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.204 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.205 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.205 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.205 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-09-10T09:07:04.205 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.205 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.205 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.205 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.205 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-09-10T09:07:04.205 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.205 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.205 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.205 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.206 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-09-10T09:07:04.206 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 122, 2024-09-10T09:07:04.206 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.085418110, 2024-09-10T09:07:04.206 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000700148 2024-09-10T09:07:04.206 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.206 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-09-10T09:07:04.206 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.206 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.206 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.206 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.206 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-09-10T09:07:04.206 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.206 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.206 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.207 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.207 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-09-10T09:07:04.207 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.207 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.207 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.207 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.207 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-09-10T09:07:04.207 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.207 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.207 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.207 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.207 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-09-10T09:07:04.207 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.208 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.208 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.208 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.208 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-09-10T09:07:04.208 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.208 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.208 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.209 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.209 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-09-10T09:07:04.209 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.209 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.209 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.209 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.210 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-09-10T09:07:04.210 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.210 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.210 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.210 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.210 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-09-10T09:07:04.210 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.210 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.210 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.210 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.210 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-09-10T09:07:04.210 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 125, 2024-09-10T09:07:04.210 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.098874028, 2024-09-10T09:07:04.211 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000790992 2024-09-10T09:07:04.211 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.211 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-09-10T09:07:04.211 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 4, 2024-09-10T09:07:04.211 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.001850935, 2024-09-10T09:07:04.211 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000462733 2024-09-10T09:07:04.211 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.211 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-09-10T09:07:04.211 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.211 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.211 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.211 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.212 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-09-10T09:07:04.212 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.212 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.212 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.212 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.212 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-09-10T09:07:04.212 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.212 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.212 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.213 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.213 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-09-10T09:07:04.213 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.213 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.213 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.213 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.213 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-09-10T09:07:04.213 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.213 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.213 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.213 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.213 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-09-10T09:07:04.214 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.214 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.214 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.214 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.214 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-09-10T09:07:04.214 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.214 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.214 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.214 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.214 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-09-10T09:07:04.214 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.214 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.215 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.215 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.215 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-09-10T09:07:04.215 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.215 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.215 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.215 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.215 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-09-10T09:07:04.215 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.215 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.215 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.215 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.215 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-09-10T09:07:04.216 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.216 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.216 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.216 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.216 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-09-10T09:07:04.216 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.216 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.216 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.216 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.216 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-09-10T09:07:04.216 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 31, 2024-09-10T09:07:04.216 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.041907243, 2024-09-10T09:07:04.216 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.001351846 2024-09-10T09:07:04.217 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.217 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-09-10T09:07:04.217 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-09-10T09:07:04.217 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-09-10T09:07:04.217 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.217 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.218 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.218 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:04.218 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.218 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-09-10T09:07:04.218 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 1, 2024-09-10T09:07:04.218 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 1, 2024-09-10T09:07:04.218 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 0, 2024-09-10T09:07:04.218 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 1, 2024-09-10T09:07:04.218 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-09-10T09:07:04.218 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 79, 2024-09-10T09:07:04.219 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 79, 2024-09-10T09:07:04.219 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 1226, 2024-09-10T09:07:04.219 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-09-10T09:07:04.219 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.219 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-09-10T09:07:04.219 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-09-10T09:07:04.219 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-09-10T09:07:04.219 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T09:07:04.219 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-09-10T09:07:04.219 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T09:07:04.219 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-09-10T09:07:04.219 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T09:07:04.219 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T09:07:04.219 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T09:07:04.220 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T09:07:04.220 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T09:07:04.220 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-09-10T09:07:04.220 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T09:07:04.220 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T09:07:04.220 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-09-10T09:07:04.220 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-09-10T09:07:04.220 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-09-10T09:07:04.220 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-09-10T09:07:04.220 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T09:07:04.220 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T09:07:04.220 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T09:07:04.220 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T09:07:04.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T09:07:04.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-09-10T09:07:04.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-09-10T09:07:04.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-09-10T09:07:04.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T09:07:04.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T09:07:04.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-09-10T09:07:04.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-09-10T09:07:04.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-09-10T09:07:04.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-09-10T09:07:04.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T09:07:04.221 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-09-10T09:07:04.222 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T09:07:04.222 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-09-10T09:07:04.222 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 304775, 2024-09-10T09:07:04.222 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 108, 2024-09-10T09:07:04.222 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 0, 2024-09-10T09:07:04.222 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 0, 2024-09-10T09:07:04.222 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-09-10T09:07:04.222 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-09-10T09:07:04.222 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-09-10T09:07:04.222 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-09-10T09:07:04.222 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-09-10T09:07:04.222 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-09-10T09:07:04.222 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 8672, 2024-09-10T09:07:04.222 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 163, 2024-09-10T09:07:04.223 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T09:07:04.223 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-09-10T09:07:04.223 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-09-10T09:07:04.223 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-09-10T09:07:04.223 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 76352, 2024-09-10T09:07:04.223 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 293, 2024-09-10T09:07:04.223 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-09-10T09:07:04.223 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-09-10T09:07:04.223 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-09-10T09:07:04.223 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-09-10T09:07:04.223 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.223 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-09-10T09:07:04.223 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-09-10T09:07:04.224 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-09-10T09:07:04.224 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 1391, 2024-09-10T09:07:04.224 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 657380, 2024-09-10T09:07:04.224 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 33, 2024-09-10T09:07:04.224 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 1391, 2024-09-10T09:07:04.224 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-09-10T09:07:04.224 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1391, 2024-09-10T09:07:04.224 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 215.123018939, 2024-09-10T09:07:04.224 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.154653500 2024-09-10T09:07:04.224 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.224 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-09-10T09:07:04.224 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-09-10T09:07:04.225 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1391, 2024-09-10T09:07:04.225 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 1527 2024-09-10T09:07:04.225 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.225 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 1391, 2024-09-10T09:07:04.225 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 155, 2024-09-10T09:07:04.225 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 1236, 2024-09-10T09:07:04.225 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-09-10T09:07:04.225 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-09-10T09:07:04.225 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 0, 2024-09-10T09:07:04.225 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 10, 2024-09-10T09:07:04.225 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 2, 2024-09-10T09:07:04.225 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 260, 2024-09-10T09:07:04.225 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 35, 2024-09-10T09:07:04.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-09-10T09:07:04.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-09-10T09:07:04.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-09-10T09:07:04.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 0, 2024-09-10T09:07:04.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 818, 2024-09-10T09:07:04.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-09-10T09:07:04.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-09-10T09:07:04.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-09-10T09:07:04.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 153, 2024-09-10T09:07:04.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 20, 2024-09-10T09:07:04.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-09-10T09:07:04.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-09-10T09:07:04.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-09-10T09:07:04.226 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-09-10T09:07:04.227 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-09-10T09:07:04.227 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-09-10T09:07:04.227 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T09:07:04.227 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-09-10T09:07:04.227 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-09-10T09:07:04.227 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 0, 2024-09-10T09:07:04.227 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-09-10T09:07:04.227 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-09-10T09:07:04.227 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-09-10T09:07:04.227 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-09-10T09:07:04.227 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-09-10T09:07:04.227 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-09-10T09:07:04.227 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-09-10T09:07:04.227 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-09-10T09:07:04.228 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-09-10T09:07:04.228 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-09-10T09:07:04.228 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-09-10T09:07:04.228 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-09-10T09:07:04.228 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-09-10T09:07:04.228 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-09-10T09:07:04.228 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-09-10T09:07:04.228 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-09-10T09:07:04.228 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 92, 2024-09-10T09:07:04.228 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-09-10T09:07:04.228 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 4, 2024-09-10T09:07:04.228 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 12, 2024-09-10T09:07:04.228 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 13, 2024-09-10T09:07:04.229 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 1, 2024-09-10T09:07:04.229 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-09-10T09:07:04.229 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 185, 2024-09-10T09:07:04.229 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 0, 2024-09-10T09:07:04.229 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 44 2024-09-10T09:07:04.229 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.229 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-09-10T09:07:04.229 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 1, 2024-09-10T09:07:04.229 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-09-10T09:07:04.229 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-09-10T09:07:04.229 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-09-10T09:07:04.229 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.230 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-09-10T09:07:04.230 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 224 2024-09-10T09:07:04.230 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.230 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-09-10T09:07:04.230 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-09-10T09:07:04.230 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 26, 2024-09-10T09:07:04.230 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-09-10T09:07:04.230 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 1, 2024-09-10T09:07:04.230 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 806, 2024-09-10T09:07:04.230 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 31, 2024-09-10T09:07:04.230 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-09-10T09:07:04.230 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.230 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T09:07:04.231 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:07:04.231 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-09-10T09:07:04.231 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:07:04.231 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 4399, 2024-09-10T09:07:04.231 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 1847414, 2024-09-10T09:07:04.231 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:07:04.231 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 4399, 2024-09-10T09:07:04.231 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-10T09:07:04.231 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-10T09:07:04.231 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 4399, 2024-09-10T09:07:04.231 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 1847414, 2024-09-10T09:07:04.231 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:07:04.231 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.231 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.232 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.232 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:04.232 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.232 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-09-10T09:07:04.232 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:07:04.232 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-09-10T09:07:04.232 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:07:04.232 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-10T09:07:04.232 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-10T09:07:04.232 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:07:04.232 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-10T09:07:04.232 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 1391, 2024-09-10T09:07:04.232 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 660759, 2024-09-10T09:07:04.233 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 1391, 2024-09-10T09:07:04.233 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 660759, 2024-09-10T09:07:04.233 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:07:04.233 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.233 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.233 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.233 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:04.233 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.233 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-09-10T09:07:04.234 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:07:04.234 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-09-10T09:07:04.234 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:07:04.234 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-10T09:07:04.234 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-10T09:07:04.234 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:07:04.234 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-10T09:07:04.234 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 1391, 2024-09-10T09:07:04.234 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 1391, 2024-09-10T09:07:04.234 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 1391, 2024-09-10T09:07:04.234 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 1391, 2024-09-10T09:07:04.234 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:07:04.234 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.234 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.235 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.235 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:04.235 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.235 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-09-10T09:07:04.235 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:07:04.235 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-09-10T09:07:04.235 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:07:04.235 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 31, 2024-09-10T09:07:04.235 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 3379, 2024-09-10T09:07:04.235 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:07:04.235 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 31, 2024-09-10T09:07:04.235 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-10T09:07:04.235 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-10T09:07:04.236 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2, 2024-09-10T09:07:04.236 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 3379, 2024-09-10T09:07:04.236 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:07:04.236 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.236 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.236 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.236 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:04.236 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-10T09:07:04.236 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x55a334e7c960": { 2024-09-10T09:07:04.236 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-10T09:07:04.236 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-09-10T09:07:04.236 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-10T09:07:04.236 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 535, 2024-09-10T09:07:04.237 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 627144, 2024-09-10T09:07:04.237 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-10T09:07:04.237 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 535, 2024-09-10T09:07:04.237 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-10T09:07:04.237 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-10T09:07:04.237 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 258, 2024-09-10T09:07:04.237 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 627144, 2024-09-10T09:07:04.237 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-10T09:07:04.237 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-10T09:07:04.237 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-10T09:07:04.237 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-10T09:07:04.237 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:04.238 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-10T09:07:04.238 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-10T09:07:04.801 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:04 smithi191 ceph-mon[35314]: pgmap v975: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:07:04.988 INFO:tasks.check_counter:Daemon mds.l mds.exported=2 2024-09-10T09:07:04.988 INFO:tasks.check_counter:Daemon mds.l mds.imported=2 2024-09-10T09:07:04.988 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-09-10T09:07:04.988 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-10T09:07:05.004 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-10T09:07:05.005 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T09:07:05.005 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:07:05.006 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:04 smithi174 ceph-mon[34068]: pgmap v975: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:07:05.034 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T09:07:05.034 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:07:05.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:04 smithi172 ceph-mon[30854]: pgmap v975: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:07:05.095 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T09:07:05.391 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:07:05.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.861+0000 7f656a549640 1 -- 172.21.15.172:0/87145025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6564106f20 msgr2=0x7f656410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:05.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.861+0000 7f656a549640 1 --2- 172.21.15.172:0/87145025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6564106f20 0x7f656410eaa0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f655c009ca0 tx=0x7f655c02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:07:05.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.861+0000 7f656a549640 1 -- 172.21.15.172:0/87145025 shutdown_connections 2024-09-10T09:07:05.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.861+0000 7f656a549640 1 --2- 172.21.15.172:0/87145025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6564106f20 0x7f656410eaa0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:05.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.861+0000 7f656a549640 1 --2- 172.21.15.172:0/87145025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65641027a0 0x7f65641067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:05.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.861+0000 7f656a549640 1 --2- 172.21.15.172:0/87145025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6564101df0 0x7f65641021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:05.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.861+0000 7f656a549640 1 -- 172.21.15.172:0/87145025 >> 172.21.15.172:0/87145025 conn(0x7f65640fd8f0 msgr2=0x7f65640ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:05.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.861+0000 7f656a549640 1 -- 172.21.15.172:0/87145025 shutdown_connections 2024-09-10T09:07:05.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.861+0000 7f656a549640 1 -- 172.21.15.172:0/87145025 wait complete. 2024-09-10T09:07:05.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f656a549640 1 Processor -- start 2024-09-10T09:07:05.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f656a549640 1 -- start start 2024-09-10T09:07:05.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f656a549640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6564101df0 0x7f65640744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:05.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f656a549640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65641027a0 0x7f6564074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:05.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f656a549640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6564106f20 0x7f6564074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:05.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f656a549640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6564111e30 con 0x7f6564106f20 2024-09-10T09:07:05.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f656a549640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6564111cb0 con 0x7f65641027a0 2024-09-10T09:07:05.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f656a549640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6564111fb0 con 0x7f6564101df0 2024-09-10T09:07:05.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f6568ae6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6564106f20 0x7f6564074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:05.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f655bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65641027a0 0x7f6564074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:05.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f6568ae6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6564106f20 0x7f6564074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:41778/0 (socket says 172.21.15.172:41778) 2024-09-10T09:07:05.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f6568ae6640 1 -- 172.21.15.172:0/4160135193 learned_addr learned my addr 172.21.15.172:0/4160135193 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:07:05.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f6563fff640 1 --2- 172.21.15.172:0/4160135193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6564101df0 0x7f65640744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:05.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f6568ae6640 1 -- 172.21.15.172:0/4160135193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6564101df0 msgr2=0x7f65640744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:05.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f6568ae6640 1 --2- 172.21.15.172:0/4160135193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6564101df0 0x7f65640744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:05.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f6568ae6640 1 -- 172.21.15.172:0/4160135193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65641027a0 msgr2=0x7f6564074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:05.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f6568ae6640 1 --2- 172.21.15.172:0/4160135193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65641027a0 0x7f6564074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:05.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.862+0000 7f6568ae6640 1 -- 172.21.15.172:0/4160135193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65641a6810 con 0x7f6564106f20 2024-09-10T09:07:05.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.863+0000 7f6568ae6640 1 --2- 172.21.15.172:0/4160135193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6564106f20 0x7f6564074f70 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f655c009dc0 tx=0x7f655c032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:05.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.863+0000 7f6561ffb640 1 -- 172.21.15.172:0/4160135193 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c003cf0 con 0x7f6564106f20 2024-09-10T09:07:05.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.863+0000 7f656a549640 1 -- 172.21.15.172:0/4160135193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65641a6aa0 con 0x7f6564106f20 2024-09-10T09:07:05.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.863+0000 7f6561ffb640 1 -- 172.21.15.172:0/4160135193 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f655c0044f0 con 0x7f6564106f20 2024-09-10T09:07:05.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.863+0000 7f6561ffb640 1 -- 172.21.15.172:0/4160135193 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c030bf0 con 0x7f6564106f20 2024-09-10T09:07:05.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.863+0000 7f656a549640 1 -- 172.21.15.172:0/4160135193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65641a7030 con 0x7f6564106f20 2024-09-10T09:07:05.879 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.864+0000 7f656a549640 1 -- 172.21.15.172:0/4160135193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65641021d0 con 0x7f6564106f20 2024-09-10T09:07:05.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.865+0000 7f6561ffb640 1 -- 172.21.15.172:0/4160135193 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f655c004040 con 0x7f6564106f20 2024-09-10T09:07:05.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.866+0000 7f6561ffb640 1 --2- 172.21.15.172:0/4160135193 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f65500a2b80 0x7f65500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:05.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.866+0000 7f6563fff640 1 --2- 172.21.15.172:0/4160135193 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f65500a2b80 0x7f65500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:05.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.866+0000 7f6561ffb640 1 -- 172.21.15.172:0/4160135193 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f655c0f0b00 con 0x7f6564106f20 2024-09-10T09:07:05.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.867+0000 7f6563fff640 1 --2- 172.21.15.172:0/4160135193 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f65500a2b80 0x7f65500a5040 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6554005fd0 tx=0x7f655400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:05.880 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:05.870+0000 7f6561ffb640 1 -- 172.21.15.172:0/4160135193 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f655c0bd6a0 con 0x7f6564106f20 2024-09-10T09:07:06.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:06.056+0000 7f656a549640 1 -- 172.21.15.172:0/4160135193 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f656410a160 con 0x7f6564106f20 2024-09-10T09:07:06.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:06.056+0000 7f6561ffb640 1 -- 172.21.15.172:0/4160135193 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+600 (secure 0 0 0) 0x7f655c0bcfa0 con 0x7f6564106f20 2024-09-10T09:07:06.058 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/977245917 2024-09-11T08:37:17.019272+0000 2024-09-10T09:07:06.058 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6801/2333617420 2024-09-11T08:37:17.019272+0000 2024-09-10T09:07:06.058 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/3919952878 2024-09-11T08:36:54.887256+0000 2024-09-10T09:07:06.058 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6800/2333617420 2024-09-11T08:37:17.019272+0000 2024-09-10T09:07:06.058 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/1525385361 2024-09-11T08:36:54.887256+0000 2024-09-10T09:07:06.058 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/2043423408 2024-09-11T08:37:17.019272+0000 2024-09-10T09:07:06.058 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6801/2639090029 2024-09-11T08:36:54.887256+0000 2024-09-10T09:07:06.058 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/839344299 2024-09-11T08:37:17.019272+0000 2024-09-10T09:07:06.058 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6800/2639090029 2024-09-11T08:36:54.887256+0000 2024-09-10T09:07:06.058 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/3251817286 2024-09-11T08:36:54.887256+0000 2024-09-10T09:07:06.059 INFO:teuthology.orchestra.run.smithi172.stderr:listed 10 entries 2024-09-10T09:07:06.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:06.060+0000 7f656a549640 1 -- 172.21.15.172:0/4160135193 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f65500a2b80 msgr2=0x7f65500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:06.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:06.060+0000 7f656a549640 1 --2- 172.21.15.172:0/4160135193 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f65500a2b80 0x7f65500a5040 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6554005fd0 tx=0x7f655400b390 comp rx=0 tx=0).stop 2024-09-10T09:07:06.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:06.060+0000 7f656a549640 1 -- 172.21.15.172:0/4160135193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6564106f20 msgr2=0x7f6564074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:06.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:06.060+0000 7f656a549640 1 --2- 172.21.15.172:0/4160135193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6564106f20 0x7f6564074f70 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f655c009dc0 tx=0x7f655c032ff0 comp rx=0 tx=0).stop 2024-09-10T09:07:06.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:06.060+0000 7f656a549640 1 -- 172.21.15.172:0/4160135193 shutdown_connections 2024-09-10T09:07:06.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:06.060+0000 7f656a549640 1 --2- 172.21.15.172:0/4160135193 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f65500a2b80 0x7f65500a5040 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:06.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:06.060+0000 7f656a549640 1 --2- 172.21.15.172:0/4160135193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6564106f20 0x7f6564074f70 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:06.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:06.060+0000 7f656a549640 1 --2- 172.21.15.172:0/4160135193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65641027a0 0x7f6564074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:06.061 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:06.060+0000 7f656a549640 1 --2- 172.21.15.172:0/4160135193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6564101df0 0x7f65640744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:06.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:06.060+0000 7f656a549640 1 -- 172.21.15.172:0/4160135193 >> 172.21.15.172:0/4160135193 conn(0x7f65640fd8f0 msgr2=0x7f65640fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:06.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:06.060+0000 7f656a549640 1 -- 172.21.15.172:0/4160135193 shutdown_connections 2024-09-10T09:07:06.062 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:06.060+0000 7f656a549640 1 -- 172.21.15.172:0/4160135193 wait complete. 2024-09-10T09:07:06.809 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:06 smithi172 ceph-mon[30854]: pgmap v976: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:07:06.809 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:06 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/4160135193' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:07:06.810 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-10T09:07:06.810 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:07:06.828 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T09:07:06.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:06 smithi174 ceph-mon[34068]: pgmap v976: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:07:06.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:06 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/4160135193' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:07:07.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:06 smithi191 ceph-mon[35314]: pgmap v976: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:07:07.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:06 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/4160135193' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:07:07.123 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:07:07.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.660+0000 7f56f3be9640 1 -- 172.21.15.172:0/3632768491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56ec102790 msgr2=0x7f56ec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:07.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.660+0000 7f56f3be9640 1 --2- 172.21.15.172:0/3632768491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56ec102790 0x7f56ec1067e0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009900 tx=0x7f56e802a410 comp rx=0 tx=0).stop 2024-09-10T09:07:07.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.660+0000 7f56f3be9640 1 -- 172.21.15.172:0/3632768491 shutdown_connections 2024-09-10T09:07:07.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.660+0000 7f56f3be9640 1 --2- 172.21.15.172:0/3632768491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56ec106f10 0x7f56ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:07.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.660+0000 7f56f3be9640 1 --2- 172.21.15.172:0/3632768491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56ec102790 0x7f56ec1067e0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:07.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.660+0000 7f56f3be9640 1 --2- 172.21.15.172:0/3632768491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56ec101de0 0x7f56ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:07.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.660+0000 7f56f3be9640 1 -- 172.21.15.172:0/3632768491 >> 172.21.15.172:0/3632768491 conn(0x7f56ec0fd8c0 msgr2=0x7f56ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:07.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.661+0000 7f56f3be9640 1 -- 172.21.15.172:0/3632768491 shutdown_connections 2024-09-10T09:07:07.687 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.661+0000 7f56f3be9640 1 -- 172.21.15.172:0/3632768491 wait complete. 2024-09-10T09:07:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.661+0000 7f56f3be9640 1 Processor -- start 2024-09-10T09:07:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.661+0000 7f56f3be9640 1 -- start start 2024-09-10T09:07:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.661+0000 7f56f3be9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56ec101de0 0x7f56ec0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.661+0000 7f56f3be9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56ec102790 0x7f56ec074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.661+0000 7f56f3be9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56ec106f10 0x7f56ec074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.661+0000 7f56f3be9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec111de0 con 0x7f56ec101de0 2024-09-10T09:07:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.661+0000 7f56f3be9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec111c60 con 0x7f56ec102790 2024-09-10T09:07:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.661+0000 7f56f3be9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec111f60 con 0x7f56ec106f10 2024-09-10T09:07:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.661+0000 7f56f1985640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56ec101de0 0x7f56ec0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.661+0000 7f56f1985640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56ec101de0 0x7f56ec0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60688/0 (socket says 172.21.15.172:60688) 2024-09-10T09:07:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.662+0000 7f56f1985640 1 -- 172.21.15.172:0/3044527009 learned_addr learned my addr 172.21.15.172:0/3044527009 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:07:07.688 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.662+0000 7f56f1184640 1 --2- 172.21.15.172:0/3044527009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56ec102790 0x7f56ec074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:07.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.662+0000 7f56f2186640 1 --2- 172.21.15.172:0/3044527009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56ec106f10 0x7f56ec074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:07.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.662+0000 7f56f1985640 1 -- 172.21.15.172:0/3044527009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56ec106f10 msgr2=0x7f56ec074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:07.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.662+0000 7f56f1985640 1 --2- 172.21.15.172:0/3044527009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56ec106f10 0x7f56ec074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:07.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.662+0000 7f56f1985640 1 -- 172.21.15.172:0/3044527009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56ec102790 msgr2=0x7f56ec074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:07.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.662+0000 7f56f1985640 1 --2- 172.21.15.172:0/3044527009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56ec102790 0x7f56ec074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:07.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.662+0000 7f56f1985640 1 -- 172.21.15.172:0/3044527009 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ec1a6880 con 0x7f56ec101de0 2024-09-10T09:07:07.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.662+0000 7f56f1184640 1 --2- 172.21.15.172:0/3044527009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56ec102790 0x7f56ec074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:07:07.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.662+0000 7f56f2186640 1 --2- 172.21.15.172:0/3044527009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56ec106f10 0x7f56ec074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:07:07.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.662+0000 7f56f1985640 1 --2- 172.21.15.172:0/3044527009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56ec101de0 0x7f56ec0744f0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f56e000d950 tx=0x7f56e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:07.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.662+0000 7f56deffd640 1 -- 172.21.15.172:0/3044527009 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0017070 con 0x7f56ec101de0 2024-09-10T09:07:07.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.662+0000 7f56f3be9640 1 -- 172.21.15.172:0/3044527009 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ec1a6b70 con 0x7f56ec101de0 2024-09-10T09:07:07.689 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.662+0000 7f56deffd640 1 -- 172.21.15.172:0/3044527009 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56e0009c30 con 0x7f56ec101de0 2024-09-10T09:07:07.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.662+0000 7f56deffd640 1 -- 172.21.15.172:0/3044527009 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e000f620 con 0x7f56ec101de0 2024-09-10T09:07:07.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.662+0000 7f56f3be9640 1 -- 172.21.15.172:0/3044527009 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ec1a70d0 con 0x7f56ec101de0 2024-09-10T09:07:07.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.663+0000 7f56f3be9640 1 -- 172.21.15.172:0/3044527009 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56ec1021c0 con 0x7f56ec101de0 2024-09-10T09:07:07.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.664+0000 7f56deffd640 1 -- 172.21.15.172:0/3044527009 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f56e00109a0 con 0x7f56ec101de0 2024-09-10T09:07:07.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.665+0000 7f56deffd640 1 --2- 172.21.15.172:0/3044527009 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56d80a2b80 0x7f56d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:07.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.665+0000 7f56f1184640 1 --2- 172.21.15.172:0/3044527009 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56d80a2b80 0x7f56d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:07.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.665+0000 7f56deffd640 1 -- 172.21.15.172:0/3044527009 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f56e00cf080 con 0x7f56ec101de0 2024-09-10T09:07:07.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.666+0000 7f56f1184640 1 --2- 172.21.15.172:0/3044527009 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56d80a2b80 0x7f56d80a5040 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f56e8031440 tx=0x7f56e8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:07.690 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.670+0000 7f56deffd640 1 -- 172.21.15.172:0/3044527009 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56e0009de0 con 0x7f56ec101de0 2024-09-10T09:07:07.848 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.847+0000 7f56f3be9640 1 -- 172.21.15.172:0/3044527009 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f56ec02d7c0 con 0x7f56ec101de0 2024-09-10T09:07:07.849 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.848+0000 7f56deffd640 1 -- 172.21.15.172:0/3044527009 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+600 (secure 0 0 0) 0x7f56e009bc80 con 0x7f56ec101de0 2024-09-10T09:07:07.849 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/977245917 2024-09-11T08:37:17.019272+0000 2024-09-10T09:07:07.849 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6801/2333617420 2024-09-11T08:37:17.019272+0000 2024-09-10T09:07:07.849 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/3919952878 2024-09-11T08:36:54.887256+0000 2024-09-10T09:07:07.849 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6800/2333617420 2024-09-11T08:37:17.019272+0000 2024-09-10T09:07:07.849 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/1525385361 2024-09-11T08:36:54.887256+0000 2024-09-10T09:07:07.849 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/2043423408 2024-09-11T08:37:17.019272+0000 2024-09-10T09:07:07.850 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6801/2639090029 2024-09-11T08:36:54.887256+0000 2024-09-10T09:07:07.850 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/839344299 2024-09-11T08:37:17.019272+0000 2024-09-10T09:07:07.850 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:6800/2639090029 2024-09-11T08:36:54.887256+0000 2024-09-10T09:07:07.850 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172:0/3251817286 2024-09-11T08:36:54.887256+0000 2024-09-10T09:07:07.850 INFO:teuthology.orchestra.run.smithi172.stderr:listed 10 entries 2024-09-10T09:07:07.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.850+0000 7f56f3be9640 1 -- 172.21.15.172:0/3044527009 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56d80a2b80 msgr2=0x7f56d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:07.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.850+0000 7f56f3be9640 1 --2- 172.21.15.172:0/3044527009 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56d80a2b80 0x7f56d80a5040 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f56e8031440 tx=0x7f56e8005a20 comp rx=0 tx=0).stop 2024-09-10T09:07:07.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.850+0000 7f56f3be9640 1 -- 172.21.15.172:0/3044527009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56ec101de0 msgr2=0x7f56ec0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:07.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.850+0000 7f56f3be9640 1 --2- 172.21.15.172:0/3044527009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56ec101de0 0x7f56ec0744f0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f56e000d950 tx=0x7f56e000de10 comp rx=0 tx=0).stop 2024-09-10T09:07:07.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.850+0000 7f56f3be9640 1 -- 172.21.15.172:0/3044527009 shutdown_connections 2024-09-10T09:07:07.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.850+0000 7f56f3be9640 1 --2- 172.21.15.172:0/3044527009 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f56d80a2b80 0x7f56d80a5040 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:07.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.850+0000 7f56f3be9640 1 --2- 172.21.15.172:0/3044527009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56ec106f10 0x7f56ec074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:07.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.850+0000 7f56f3be9640 1 --2- 172.21.15.172:0/3044527009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56ec102790 0x7f56ec074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:07.851 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.850+0000 7f56f3be9640 1 --2- 172.21.15.172:0/3044527009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56ec101de0 0x7f56ec0744f0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:07.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.850+0000 7f56f3be9640 1 -- 172.21.15.172:0/3044527009 >> 172.21.15.172:0/3044527009 conn(0x7f56ec0fd8c0 msgr2=0x7f56ec0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:07.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.850+0000 7f56f3be9640 1 -- 172.21.15.172:0/3044527009 shutdown_connections 2024-09-10T09:07:07.852 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:07.850+0000 7f56f3be9640 1 -- 172.21.15.172:0/3044527009 wait complete. 2024-09-10T09:07:08.554 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi172.front.sepia.ceph.com... 2024-09-10T09:07:08.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:07:08.554 DEBUG:teuthology.orchestra.run.smithi172:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T09:07:08.597 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T09:07:08.603 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi172.stderr:ceph-fuse[82703]: fuse finished with error 0 and tester_r 0 2024-09-10T09:07:08.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:08 smithi172 ceph-mon[30854]: pgmap v977: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:07:08.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:08 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3044527009' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:07:08.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:08 smithi174 ceph-mon[34068]: pgmap v977: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:07:08.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:08 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3044527009' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:07:09.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:08 smithi191 ceph-mon[35314]: pgmap v977: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T09:07:09.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:08 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3044527009' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:07:10.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:10 smithi174 ceph-mon[34068]: pgmap v978: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T09:07:11.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:10 smithi191 ceph-mon[35314]: pgmap v978: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T09:07:11.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:10 smithi172 ceph-mon[30854]: pgmap v978: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T09:07:12.800 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:12 smithi191 ceph-mon[35314]: pgmap v979: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T09:07:12.885 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:12 smithi172 ceph-mon[30854]: pgmap v979: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T09:07:12.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:12 smithi174 ceph-mon[34068]: pgmap v979: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T09:07:14.698 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-09-10T09:07:14.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:07:14.699 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T09:07:14.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:07:14.719 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-09-10T09:07:14.777 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T09:07:14.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:07:14.777 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T09:07:14.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:07:14.847 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2024-09-10T09:07:14.910 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:14 smithi172 ceph-mon[30854]: pgmap v980: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T09:07:14.949 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T09:07:14.962 INFO:tasks.cephadm:Teardown begin 2024-09-10T09:07:14.962 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T09:07:14.962 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:07:14.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:14 smithi174 ceph-mon[34068]: pgmap v980: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T09:07:15.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:14 smithi191 ceph-mon[35314]: pgmap v980: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T09:07:15.247 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:07:15.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.743+0000 7fa70a6dc640 1 -- 172.21.15.172:0/4277001381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa704101420 msgr2=0x7fa70410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:15.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.743+0000 7fa70a6dc640 1 --2- 172.21.15.172:0/4277001381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa704101420 0x7fa70410d950 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc009cd0 tx=0x7fa6fc02a8e0 comp rx=0 tx=0).stop 2024-09-10T09:07:15.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.743+0000 7fa70a6dc640 1 -- 172.21.15.172:0/4277001381 shutdown_connections 2024-09-10T09:07:15.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.743+0000 7fa70a6dc640 1 --2- 172.21.15.172:0/4277001381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa704101420 0x7fa70410d950 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:15.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.743+0000 7fa70a6dc640 1 --2- 172.21.15.172:0/4277001381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa704100a80 0x7fa704100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:15.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.743+0000 7fa70a6dc640 1 --2- 172.21.15.172:0/4277001381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa704106a80 0x7fa704106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:15.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.743+0000 7fa70a6dc640 1 -- 172.21.15.172:0/4277001381 >> 172.21.15.172:0/4277001381 conn(0x7fa7040fc7a0 msgr2=0x7fa7040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:15.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.743+0000 7fa70a6dc640 1 -- 172.21.15.172:0/4277001381 shutdown_connections 2024-09-10T09:07:15.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.743+0000 7fa70a6dc640 1 -- 172.21.15.172:0/4277001381 wait complete. 2024-09-10T09:07:15.756 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.744+0000 7fa70a6dc640 1 Processor -- start 2024-09-10T09:07:15.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.744+0000 7fa70a6dc640 1 -- start start 2024-09-10T09:07:15.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.744+0000 7fa70a6dc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa704100a80 0x7fa7041982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:15.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.744+0000 7fa70a6dc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa704101420 0x7fa704198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:15.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.744+0000 7fa70a6dc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa704106a80 0x7fa70419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:15.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.744+0000 7fa70a6dc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa704110d30 con 0x7fa704101420 2024-09-10T09:07:15.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.744+0000 7fa70a6dc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa704110bb0 con 0x7fa704106a80 2024-09-10T09:07:15.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.744+0000 7fa70a6dc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa704110eb0 con 0x7fa704100a80 2024-09-10T09:07:15.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.744+0000 7fa7037fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa704101420 0x7fa704198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:15.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.744+0000 7fa7037fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa704101420 0x7fa704198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:60704/0 (socket says 172.21.15.172:60704) 2024-09-10T09:07:15.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.744+0000 7fa7037fe640 1 -- 172.21.15.172:0/2592975953 learned_addr learned my addr 172.21.15.172:0/2592975953 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:07:15.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.744+0000 7fa703fff640 1 --2- 172.21.15.172:0/2592975953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa704100a80 0x7fa7041982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:15.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.745+0000 7fa708c79640 1 --2- 172.21.15.172:0/2592975953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa704106a80 0x7fa70419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:15.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.745+0000 7fa7037fe640 1 -- 172.21.15.172:0/2592975953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa704100a80 msgr2=0x7fa7041982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:15.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.745+0000 7fa7037fe640 1 --2- 172.21.15.172:0/2592975953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa704100a80 0x7fa7041982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:15.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.745+0000 7fa7037fe640 1 -- 172.21.15.172:0/2592975953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa704106a80 msgr2=0x7fa70419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:15.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.745+0000 7fa7037fe640 1 --2- 172.21.15.172:0/2592975953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa704106a80 0x7fa70419cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:15.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.745+0000 7fa7037fe640 1 -- 172.21.15.172:0/2592975953 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70419d270 con 0x7fa704101420 2024-09-10T09:07:15.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.745+0000 7fa703fff640 1 --2- 172.21.15.172:0/2592975953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa704100a80 0x7fa7041982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:07:15.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.745+0000 7fa7037fe640 1 --2- 172.21.15.172:0/2592975953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa704101420 0x7fa704198800 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00d770 tx=0x7fa6ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:15.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.745+0000 7fa7017fa640 1 -- 172.21.15.172:0/2592975953 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec017070 con 0x7fa704101420 2024-09-10T09:07:15.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.745+0000 7fa7017fa640 1 -- 172.21.15.172:0/2592975953 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6ec0054d0 con 0x7fa704101420 2024-09-10T09:07:15.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.745+0000 7fa7017fa640 1 -- 172.21.15.172:0/2592975953 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec004cf0 con 0x7fa704101420 2024-09-10T09:07:15.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.745+0000 7fa70a6dc640 1 -- 172.21.15.172:0/2592975953 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa70419d560 con 0x7fa704101420 2024-09-10T09:07:15.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.745+0000 7fa70a6dc640 1 -- 172.21.15.172:0/2592975953 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7041a4e40 con 0x7fa704101420 2024-09-10T09:07:15.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.746+0000 7fa70a6dc640 1 -- 172.21.15.172:0/2592975953 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa704100ee0 con 0x7fa704101420 2024-09-10T09:07:15.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.747+0000 7fa7017fa640 1 -- 172.21.15.172:0/2592975953 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa6ec004e90 con 0x7fa704101420 2024-09-10T09:07:15.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.748+0000 7fa7017fa640 1 --2- 172.21.15.172:0/2592975953 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa6f00a2c00 0x7fa6f00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:15.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.748+0000 7fa703fff640 1 --2- 172.21.15.172:0/2592975953 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa6f00a2c00 0x7fa6f00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:15.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.748+0000 7fa7017fa640 1 -- 172.21.15.172:0/2592975953 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fa6ec0cee40 con 0x7fa704101420 2024-09-10T09:07:15.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.749+0000 7fa703fff640 1 --2- 172.21.15.172:0/2592975953 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa6f00a2c00 0x7fa6f00a50c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400baa0 tx=0x7fa6f4005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:15.759 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.752+0000 7fa7017fa640 1 -- 172.21.15.172:0/2592975953 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6ec09ba40 con 0x7fa704101420 2024-09-10T09:07:15.955 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.954+0000 7fa70a6dc640 1 -- 172.21.15.172:0/2592975953 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa704109a90 con 0x7fa704101420 2024-09-10T09:07:15.956 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.955+0000 7fa7017fa640 1 -- 172.21.15.172:0/2592975953 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+12010 (secure 0 0 0) 0x7fa6ec09b340 con 0x7fa704101420 2024-09-10T09:07:15.956 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T09:07:15.956 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":49,"btime":"2024-09-10T09:06:03:774593+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T09:06:03.109907+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T09:07:15.957 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 49 2024-09-10T09:07:15.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.957+0000 7fa70a6dc640 1 -- 172.21.15.172:0/2592975953 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa6f00a2c00 msgr2=0x7fa6f00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:15.958 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.957+0000 7fa70a6dc640 1 --2- 172.21.15.172:0/2592975953 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa6f00a2c00 0x7fa6f00a50c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400baa0 tx=0x7fa6f4005d20 comp rx=0 tx=0).stop 2024-09-10T09:07:15.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.957+0000 7fa70a6dc640 1 -- 172.21.15.172:0/2592975953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa704101420 msgr2=0x7fa704198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:15.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.957+0000 7fa70a6dc640 1 --2- 172.21.15.172:0/2592975953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa704101420 0x7fa704198800 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00d770 tx=0x7fa6ec00dc30 comp rx=0 tx=0).stop 2024-09-10T09:07:15.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.958+0000 7fa70a6dc640 1 -- 172.21.15.172:0/2592975953 shutdown_connections 2024-09-10T09:07:15.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.958+0000 7fa70a6dc640 1 --2- 172.21.15.172:0/2592975953 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fa6f00a2c00 0x7fa6f00a50c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:15.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.958+0000 7fa70a6dc640 1 --2- 172.21.15.172:0/2592975953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa704106a80 0x7fa70419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:15.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.958+0000 7fa70a6dc640 1 --2- 172.21.15.172:0/2592975953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa704101420 0x7fa704198800 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:15.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.958+0000 7fa70a6dc640 1 --2- 172.21.15.172:0/2592975953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa704100a80 0x7fa7041982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:15.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.958+0000 7fa70a6dc640 1 -- 172.21.15.172:0/2592975953 >> 172.21.15.172:0/2592975953 conn(0x7fa7040fc7a0 msgr2=0x7fa70406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:15.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.958+0000 7fa70a6dc640 1 -- 172.21.15.172:0/2592975953 shutdown_connections 2024-09-10T09:07:15.959 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:15.958+0000 7fa70a6dc640 1 -- 172.21.15.172:0/2592975953 wait complete. 2024-09-10T09:07:16.743 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:07:16.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:16 smithi174 ceph-mon[34068]: pgmap v981: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-10T09:07:16.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:16 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2592975953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:07:16.994 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:07:17.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:16 smithi191 ceph-mon[35314]: pgmap v981: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-10T09:07:17.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:16 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2592975953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:07:17.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:16 smithi172 ceph-mon[30854]: pgmap v981: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-10T09:07:17.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:16 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2592975953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:07:17.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.528+0000 7f3cf203b640 1 -- 172.21.15.172:0/1924946631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cec102790 msgr2=0x7f3cec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:17.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.528+0000 7f3cf203b640 1 --2- 172.21.15.172:0/1924946631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cec102790 0x7f3cec1067e0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009900 tx=0x7f3cd402a410 comp rx=0 tx=0).stop 2024-09-10T09:07:17.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.528+0000 7f3cf203b640 1 -- 172.21.15.172:0/1924946631 shutdown_connections 2024-09-10T09:07:17.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.528+0000 7f3cf203b640 1 --2- 172.21.15.172:0/1924946631 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cec106f10 0x7f3cec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:17.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.528+0000 7f3cf203b640 1 --2- 172.21.15.172:0/1924946631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cec102790 0x7f3cec1067e0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:17.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.528+0000 7f3cf203b640 1 --2- 172.21.15.172:0/1924946631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cec101de0 0x7f3cec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:17.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.528+0000 7f3cf203b640 1 -- 172.21.15.172:0/1924946631 >> 172.21.15.172:0/1924946631 conn(0x7f3cec0fd8c0 msgr2=0x7f3cec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.528+0000 7f3cf203b640 1 -- 172.21.15.172:0/1924946631 shutdown_connections 2024-09-10T09:07:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.528+0000 7f3cf203b640 1 -- 172.21.15.172:0/1924946631 wait complete. 2024-09-10T09:07:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.529+0000 7f3cf203b640 1 Processor -- start 2024-09-10T09:07:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.529+0000 7f3cf203b640 1 -- start start 2024-09-10T09:07:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.529+0000 7f3cf203b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cec101de0 0x7f3cec198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.529+0000 7f3cf203b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cec102790 0x7f3cec198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.529+0000 7f3cf203b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cec106f10 0x7f3cec19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.529+0000 7f3cf203b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec111e40 con 0x7f3cec101de0 2024-09-10T09:07:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.529+0000 7f3cf203b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec111cc0 con 0x7f3cec106f10 2024-09-10T09:07:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.529+0000 7f3cf203b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec111fc0 con 0x7f3cec102790 2024-09-10T09:07:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.529+0000 7f3ceb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cec101de0 0x7f3cec198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.529+0000 7f3ceb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cec101de0 0x7f3cec198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47210/0 (socket says 172.21.15.172:47210) 2024-09-10T09:07:17.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.529+0000 7f3ceb7fe640 1 -- 172.21.15.172:0/3846930918 learned_addr learned my addr 172.21.15.172:0/3846930918 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:07:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.530+0000 7f3ce2ffd640 1 --2- 172.21.15.172:0/3846930918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cec102790 0x7f3cec198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.530+0000 7f3cebfff640 1 --2- 172.21.15.172:0/3846930918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cec106f10 0x7f3cec19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.530+0000 7f3ceb7fe640 1 -- 172.21.15.172:0/3846930918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cec102790 msgr2=0x7f3cec198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.530+0000 7f3ceb7fe640 1 --2- 172.21.15.172:0/3846930918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cec102790 0x7f3cec198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.530+0000 7f3ceb7fe640 1 -- 172.21.15.172:0/3846930918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cec106f10 msgr2=0x7f3cec19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.530+0000 7f3ceb7fe640 1 --2- 172.21.15.172:0/3846930918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cec106f10 0x7f3cec19cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.530+0000 7f3ceb7fe640 1 -- 172.21.15.172:0/3846930918 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cec19d2d0 con 0x7f3cec101de0 2024-09-10T09:07:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.530+0000 7f3ce2ffd640 1 --2- 172.21.15.172:0/3846930918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cec102790 0x7f3cec198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:07:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.530+0000 7f3ceb7fe640 1 --2- 172.21.15.172:0/3846930918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cec101de0 0x7f3cec198320 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c920 tx=0x7f3cdc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.530+0000 7f3ce97fa640 1 -- 172.21.15.172:0/3846930918 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc007c70 con 0x7f3cec101de0 2024-09-10T09:07:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.530+0000 7f3cf203b640 1 -- 172.21.15.172:0/3846930918 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cec19d5c0 con 0x7f3cec101de0 2024-09-10T09:07:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.530+0000 7f3ce97fa640 1 -- 172.21.15.172:0/3846930918 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3cdc007e10 con 0x7f3cec101de0 2024-09-10T09:07:17.551 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.530+0000 7f3ce97fa640 1 -- 172.21.15.172:0/3846930918 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc018820 con 0x7f3cec101de0 2024-09-10T09:07:17.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.530+0000 7f3cf203b640 1 -- 172.21.15.172:0/3846930918 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cec1a4ea0 con 0x7f3cec101de0 2024-09-10T09:07:17.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.531+0000 7f3cf203b640 1 -- 172.21.15.172:0/3846930918 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cec1021c0 con 0x7f3cec101de0 2024-09-10T09:07:17.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.532+0000 7f3ce97fa640 1 -- 172.21.15.172:0/3846930918 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3cdc0189c0 con 0x7f3cec101de0 2024-09-10T09:07:17.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.533+0000 7f3ce97fa640 1 --2- 172.21.15.172:0/3846930918 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cd80a2b80 0x7f3cd80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:17.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.533+0000 7f3ce2ffd640 1 --2- 172.21.15.172:0/3846930918 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cd80a2b80 0x7f3cd80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:17.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.533+0000 7f3ce97fa640 1 -- 172.21.15.172:0/3846930918 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7f3cdc0cf5c0 con 0x7f3cec101de0 2024-09-10T09:07:17.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.534+0000 7f3ce2ffd640 1 --2- 172.21.15.172:0/3846930918 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cd80a2b80 0x7f3cd80a5040 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4031440 tx=0x7f3cd40095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:17.552 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.537+0000 7f3ce97fa640 1 -- 172.21.15.172:0/3846930918 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3cdc014030 con 0x7f3cec101de0 2024-09-10T09:07:17.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.737+0000 7f3cf203b640 1 -- 172.21.15.172:0/3846930918 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3cec10a150 con 0x7f3cec101de0 2024-09-10T09:07:17.738 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.738+0000 7f3ce97fa640 1 -- 172.21.15.172:0/3846930918 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+12010 (secure 0 0 0) 0x7f3cdc09c1c0 con 0x7f3cec101de0 2024-09-10T09:07:17.739 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T09:07:17.739 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":49,"btime":"2024-09-10T09:06:03:774593+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2272815205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2272815205},{"type":"v1","addr":"172.21.15.172:6835","nonce":2272815205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/2122290739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":2122290739},{"type":"v1","addr":"172.21.15.172:6837","nonce":2122290739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T08:44:11.310930+0000","modified":"2024-09-10T09:06:03.109907+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24458,"mds_2":24457,"mds_3":24416,"mds_4":24406},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6839/1621056220","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1621056220},{"type":"v1","addr":"172.21.15.172:6839","nonce":1621056220}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14622":{"gid":14622,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6841/176040463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6840","nonce":176040463},{"type":"v1","addr":"172.21.15.172:6841","nonce":176040463}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":22,"addr":"172.21.15.191:6833/1662718523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1662718523},{"type":"v1","addr":"172.21.15.191:6833","nonce":1662718523}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":22,"addr":"172.21.15.174:6833/3418766024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":3418766024},{"type":"v1","addr":"172.21.15.174:6833","nonce":3418766024}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/1774553543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":1774553543},{"type":"v1","addr":"172.21.15.191:6835","nonce":1774553543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/3197756201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":3197756201},{"type":"v1","addr":"172.21.15.174:6835","nonce":3197756201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/1802631256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":1802631256},{"type":"v1","addr":"172.21.15.191:6837","nonce":1802631256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":16,"addr":"172.21.15.191:6839/1702936476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":1702936476},{"type":"v1","addr":"172.21.15.191:6839","nonce":1702936476}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.174:6837/2399337052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":2399337052},{"type":"v1","addr":"172.21.15.174:6837","nonce":2399337052}]},"join_fscid":-1,"export_targets":[2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.174:6839/3029849701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3029849701},{"type":"v1","addr":"172.21.15.174:6839","nonce":3029849701}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24406,24458,24416,24457,24488]},"id":1}]} 2024-09-10T09:07:17.739 INFO:teuthology.orchestra.run.smithi172.stderr:dumped fsmap epoch 49 2024-09-10T09:07:17.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.740+0000 7f3cf203b640 1 -- 172.21.15.172:0/3846930918 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cd80a2b80 msgr2=0x7f3cd80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:17.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.740+0000 7f3cf203b640 1 --2- 172.21.15.172:0/3846930918 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cd80a2b80 0x7f3cd80a5040 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4031440 tx=0x7f3cd40095f0 comp rx=0 tx=0).stop 2024-09-10T09:07:17.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.740+0000 7f3cf203b640 1 -- 172.21.15.172:0/3846930918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cec101de0 msgr2=0x7f3cec198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:17.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.740+0000 7f3cf203b640 1 --2- 172.21.15.172:0/3846930918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cec101de0 0x7f3cec198320 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c920 tx=0x7f3cdc00cde0 comp rx=0 tx=0).stop 2024-09-10T09:07:17.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.740+0000 7f3cf203b640 1 -- 172.21.15.172:0/3846930918 shutdown_connections 2024-09-10T09:07:17.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.740+0000 7f3cf203b640 1 --2- 172.21.15.172:0/3846930918 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f3cd80a2b80 0x7f3cd80a5040 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:17.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.740+0000 7f3cf203b640 1 --2- 172.21.15.172:0/3846930918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cec106f10 0x7f3cec19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:17.741 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.740+0000 7f3cf203b640 1 --2- 172.21.15.172:0/3846930918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cec102790 0x7f3cec198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:17.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.740+0000 7f3cf203b640 1 --2- 172.21.15.172:0/3846930918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cec101de0 0x7f3cec198320 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:17.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.740+0000 7f3cf203b640 1 -- 172.21.15.172:0/3846930918 >> 172.21.15.172:0/3846930918 conn(0x7f3cec0fd8c0 msgr2=0x7f3cec0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:17.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.740+0000 7f3cf203b640 1 -- 172.21.15.172:0/3846930918 shutdown_connections 2024-09-10T09:07:17.742 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:17.740+0000 7f3cf203b640 1 -- 172.21.15.172:0/3846930918 wait complete. 2024-09-10T09:07:18.442 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:07:18.693 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:07:18.814 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:18 smithi172 ceph-mon[30854]: pgmap v982: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T09:07:18.814 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:18 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/3846930918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:07:18.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:18 smithi174 ceph-mon[34068]: pgmap v982: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T09:07:18.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:18 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/3846930918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:07:19.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:18 smithi191 ceph-mon[35314]: pgmap v982: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T09:07:19.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:18 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/3846930918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:07:19.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.196+0000 7fcdcf258640 1 -- 172.21.15.172:0/1444068845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcdc8101de0 msgr2=0x7fcdc81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:19.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.196+0000 7fcdcf258640 1 --2- 172.21.15.172:0/1444068845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcdc8101de0 0x7fcdc81021c0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009900 tx=0x7fcdb802a410 comp rx=0 tx=0).stop 2024-09-10T09:07:19.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.196+0000 7fcdcf258640 1 -- 172.21.15.172:0/1444068845 shutdown_connections 2024-09-10T09:07:19.225 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.196+0000 7fcdcf258640 1 --2- 172.21.15.172:0/1444068845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc8106f10 0x7fcdc810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.196+0000 7fcdcf258640 1 --2- 172.21.15.172:0/1444068845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdc8102790 0x7fcdc81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.196+0000 7fcdcf258640 1 --2- 172.21.15.172:0/1444068845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcdc8101de0 0x7fcdc81021c0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.196+0000 7fcdcf258640 1 -- 172.21.15.172:0/1444068845 >> 172.21.15.172:0/1444068845 conn(0x7fcdc80fd8c0 msgr2=0x7fcdc80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.196+0000 7fcdcf258640 1 -- 172.21.15.172:0/1444068845 shutdown_connections 2024-09-10T09:07:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.196+0000 7fcdcf258640 1 -- 172.21.15.172:0/1444068845 wait complete. 2024-09-10T09:07:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.196+0000 7fcdcf258640 1 Processor -- start 2024-09-10T09:07:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.196+0000 7fcdcf258640 1 -- start start 2024-09-10T09:07:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdcf258640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdc8101de0 0x7fcdc819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdcf258640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc8102790 0x7fcdc819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdcf258640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcdc8106f10 0x7fcdc819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:19.226 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdcf258640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8111e20 con 0x7fcdc8106f10 2024-09-10T09:07:19.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdcf258640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8111ca0 con 0x7fcdc8102790 2024-09-10T09:07:19.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdcf258640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8111fa0 con 0x7fcdc8101de0 2024-09-10T09:07:19.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdcd7f5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcdc8106f10 0x7fcdc819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:19.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdcd7f5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcdc8106f10 0x7fcdc819ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47236/0 (socket says 172.21.15.172:47236) 2024-09-10T09:07:19.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdcd7f5640 1 -- 172.21.15.172:0/975518799 learned_addr learned my addr 172.21.15.172:0/975518799 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:07:19.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdc7fff640 1 --2- 172.21.15.172:0/975518799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc8102790 0x7fcdc819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:19.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdccff4640 1 --2- 172.21.15.172:0/975518799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdc8101de0 0x7fcdc819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:19.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdcd7f5640 1 -- 172.21.15.172:0/975518799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdc8101de0 msgr2=0x7fcdc819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:19.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdcd7f5640 1 --2- 172.21.15.172:0/975518799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdc8101de0 0x7fcdc819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:19.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdcd7f5640 1 -- 172.21.15.172:0/975518799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc8102790 msgr2=0x7fcdc819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:19.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdcd7f5640 1 --2- 172.21.15.172:0/975518799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc8102790 0x7fcdc819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:19.227 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdcd7f5640 1 -- 172.21.15.172:0/975518799 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc819f550 con 0x7fcdc8106f10 2024-09-10T09:07:19.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdccff4640 1 --2- 172.21.15.172:0/975518799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdc8101de0 0x7fcdc819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:07:19.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdcd7f5640 1 --2- 172.21.15.172:0/975518799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcdc8106f10 0x7fcdc819ee70 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009fd0 tx=0x7fcdc000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:19.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdc5ffb640 1 -- 172.21.15.172:0/975518799 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0017070 con 0x7fcdc8106f10 2024-09-10T09:07:19.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdcf258640 1 -- 172.21.15.172:0/975518799 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc819f840 con 0x7fcdc8106f10 2024-09-10T09:07:19.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdc5ffb640 1 -- 172.21.15.172:0/975518799 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdc0003e20 con 0x7fcdc8106f10 2024-09-10T09:07:19.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.197+0000 7fcdc5ffb640 1 -- 172.21.15.172:0/975518799 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0010460 con 0x7fcdc8106f10 2024-09-10T09:07:19.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.198+0000 7fcdcf258640 1 -- 172.21.15.172:0/975518799 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc81a7120 con 0x7fcdc8106f10 2024-09-10T09:07:19.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.199+0000 7fcdcf258640 1 -- 172.21.15.172:0/975518799 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdc81021c0 con 0x7fcdc8106f10 2024-09-10T09:07:19.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.199+0000 7fcdc5ffb640 1 -- 172.21.15.172:0/975518799 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fcdc0003970 con 0x7fcdc8106f10 2024-09-10T09:07:19.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.200+0000 7fcdc5ffb640 1 --2- 172.21.15.172:0/975518799 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcdb00a2b80 0x7fcdb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:19.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.200+0000 7fcdc5ffb640 1 -- 172.21.15.172:0/975518799 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7fcdc00cf250 con 0x7fcdc8106f10 2024-09-10T09:07:19.228 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.200+0000 7fcdccff4640 1 --2- 172.21.15.172:0/975518799 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcdb00a2b80 0x7fcdb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:19.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.201+0000 7fcdccff4640 1 --2- 172.21.15.172:0/975518799 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcdb00a2b80 0x7fcdb00a5040 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8000c00 tx=0x7fcdb8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:19.229 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.204+0000 7fcdc5ffb640 1 -- 172.21.15.172:0/975518799 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcdc009be50 con 0x7fcdc8106f10 2024-09-10T09:07:19.375 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.374+0000 7fcdcf258640 1 -- 172.21.15.172:0/975518799 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcdc81a7350 con 0x7fcdc8106f10 2024-09-10T09:07:19.377 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.377+0000 7fcdc5ffb640 1 -- 172.21.15.172:0/975518799 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24778 (secure 0 0 0) 0x7fcdc009b750 con 0x7fcdc8106f10 2024-09-10T09:07:19.378 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-10T09:07:19.378 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":92,"fsid":"a7dba522-6f4f-11ef-bcea-c7b262605968","created":"2024-09-10T08:36:33.083529+0000","modified":"2024-09-10T08:46:18.799844+0000","last_up_change":"2024-09-10T08:43:19.615640+0000","last_in_change":"2024-09-10T08:42:56.656115+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T08:39:31.328306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T08:44:07.466637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T08:44:09.520569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T08:44:15.926057+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c783e1f2-8d5c-4fa6-bf0c-a2ed26bbeb12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6803","nonce":2678426952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6805","nonce":2678426952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6809","nonce":2678426952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":2678426952},{"type":"v1","addr":"172.21.15.172:6807","nonce":2678426952}]},"public_addr":"172.21.15.172:6803/2678426952","cluster_addr":"172.21.15.172:6805/2678426952","heartbeat_back_addr":"172.21.15.172:6809/2678426952","heartbeat_front_addr":"172.21.15.172:6807/2678426952","state":["exists","up"]},{"osd":1,"uuid":"55f6ae92-3a0e-48a1-8f70-f9a5c85f0d71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6801","nonce":3279661439}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6803","nonce":3279661439}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6807","nonce":3279661439}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3279661439},{"type":"v1","addr":"172.21.15.174:6805","nonce":3279661439}]},"public_addr":"172.21.15.174:6801/3279661439","cluster_addr":"172.21.15.174:6803/3279661439","heartbeat_back_addr":"172.21.15.174:6807/3279661439","heartbeat_front_addr":"172.21.15.174:6805/3279661439","state":["exists","up"]},{"osd":2,"uuid":"f596de25-bc55-4bd7-b9c7-02e1d8bf7bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6801","nonce":1963796290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6803","nonce":1963796290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6807","nonce":1963796290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1963796290},{"type":"v1","addr":"172.21.15.191:6805","nonce":1963796290}]},"public_addr":"172.21.15.191:6801/1963796290","cluster_addr":"172.21.15.191:6803/1963796290","heartbeat_back_addr":"172.21.15.191:6807/1963796290","heartbeat_front_addr":"172.21.15.191:6805/1963796290","state":["exists","up"]},{"osd":3,"uuid":"15ecca3f-bc6b-4a06-b283-b515384beb19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6811","nonce":869546480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6813","nonce":869546480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6817","nonce":869546480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":869546480},{"type":"v1","addr":"172.21.15.172:6815","nonce":869546480}]},"public_addr":"172.21.15.172:6811/869546480","cluster_addr":"172.21.15.172:6813/869546480","heartbeat_back_addr":"172.21.15.172:6817/869546480","heartbeat_front_addr":"172.21.15.172:6815/869546480","state":["exists","up"]},{"osd":4,"uuid":"8c2c1d40-72ad-4a15-ab72-54871c7e2427","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6809","nonce":961007949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6811","nonce":961007949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6815","nonce":961007949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":961007949},{"type":"v1","addr":"172.21.15.174:6813","nonce":961007949}]},"public_addr":"172.21.15.174:6809/961007949","cluster_addr":"172.21.15.174:6811/961007949","heartbeat_back_addr":"172.21.15.174:6815/961007949","heartbeat_front_addr":"172.21.15.174:6813/961007949","state":["exists","up"]},{"osd":5,"uuid":"4de45a1d-ef0e-4bfe-ba15-5eeeb8806155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6809","nonce":4216938371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6811","nonce":4216938371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6815","nonce":4216938371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":4216938371},{"type":"v1","addr":"172.21.15.191:6813","nonce":4216938371}]},"public_addr":"172.21.15.191:6809/4216938371","cluster_addr":"172.21.15.191:6811/4216938371","heartbeat_back_addr":"172.21.15.191:6815/4216938371","heartbeat_front_addr":"172.21.15.191:6813/4216938371","state":["exists","up"]},{"osd":6,"uuid":"60a9938a-f696-4db2-932c-402997543b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6819","nonce":3210762009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6821","nonce":3210762009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6825","nonce":3210762009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":3210762009},{"type":"v1","addr":"172.21.15.172:6823","nonce":3210762009}]},"public_addr":"172.21.15.172:6819/3210762009","cluster_addr":"172.21.15.172:6821/3210762009","heartbeat_back_addr":"172.21.15.172:6825/3210762009","heartbeat_front_addr":"172.21.15.172:6823/3210762009","state":["exists","up"]},{"osd":7,"uuid":"7f0202f3-971e-472f-9c4d-ae3c24d39f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6817","nonce":1461923581}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6819","nonce":1461923581}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6823","nonce":1461923581}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":1461923581},{"type":"v1","addr":"172.21.15.174:6821","nonce":1461923581}]},"public_addr":"172.21.15.174:6817/1461923581","cluster_addr":"172.21.15.174:6819/1461923581","heartbeat_back_addr":"172.21.15.174:6823/1461923581","heartbeat_front_addr":"172.21.15.174:6821/1461923581","state":["exists","up"]},{"osd":8,"uuid":"b4e7327f-ca9f-4833-9cd3-b8529b3d9761","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6817","nonce":3675210430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6819","nonce":3675210430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6823","nonce":3675210430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":3675210430},{"type":"v1","addr":"172.21.15.191:6821","nonce":3675210430}]},"public_addr":"172.21.15.191:6817/3675210430","cluster_addr":"172.21.15.191:6819/3675210430","heartbeat_back_addr":"172.21.15.191:6823/3675210430","heartbeat_front_addr":"172.21.15.191:6821/3675210430","state":["exists","up"]},{"osd":9,"uuid":"9f6153f2-f9db-4c00-a659-c932628016f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6827","nonce":1181268619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6829","nonce":1181268619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6833","nonce":1181268619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":1181268619},{"type":"v1","addr":"172.21.15.172:6831","nonce":1181268619}]},"public_addr":"172.21.15.172:6827/1181268619","cluster_addr":"172.21.15.172:6829/1181268619","heartbeat_back_addr":"172.21.15.172:6833/1181268619","heartbeat_front_addr":"172.21.15.172:6831/1181268619","state":["exists","up"]},{"osd":10,"uuid":"040a013a-4c3e-442f-bea6-12cbe517716d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6825","nonce":1238812525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6827","nonce":1238812525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6831","nonce":1238812525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1238812525},{"type":"v1","addr":"172.21.15.174:6829","nonce":1238812525}]},"public_addr":"172.21.15.174:6825/1238812525","cluster_addr":"172.21.15.174:6827/1238812525","heartbeat_back_addr":"172.21.15.174:6831/1238812525","heartbeat_front_addr":"172.21.15.174:6829/1238812525","state":["exists","up"]},{"osd":11,"uuid":"ae6535bd-1510-4bf3-bf4b-1a2a256add43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6825","nonce":2162719997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6827","nonce":2162719997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6831","nonce":2162719997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2162719997},{"type":"v1","addr":"172.21.15.191:6829","nonce":2162719997}]},"public_addr":"172.21.15.191:6825/2162719997","cluster_addr":"172.21.15.191:6827/2162719997","heartbeat_back_addr":"172.21.15.191:6831/2162719997","heartbeat_front_addr":"172.21.15.191:6829/2162719997","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:38:38.812596+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:04.765652+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:29.103966+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:39:56.085849+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:21.641784+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:40:47.750376+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:11.640697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:41:36.638840+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:02.167446+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:28.655804+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:42:52.749676+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T08:43:17.783908+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:0/977245917":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/3919952878":"2024-09-11T08:36:54.887256+0000","172.21.15.172:6800/2333617420":"2024-09-11T08:37:17.019272+0000","172.21.15.172:0/1525385361":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/2043423408":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6801/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/839344299":"2024-09-11T08:37:17.019272+0000","172.21.15.172:6800/2639090029":"2024-09-11T08:36:54.887256+0000","172.21.15.172:0/3251817286":"2024-09-11T08:36:54.887256+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:07:19.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.379+0000 7fcdcf258640 1 -- 172.21.15.172:0/975518799 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcdb00a2b80 msgr2=0x7fcdb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:19.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.379+0000 7fcdcf258640 1 --2- 172.21.15.172:0/975518799 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcdb00a2b80 0x7fcdb00a5040 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8000c00 tx=0x7fcdb8031400 comp rx=0 tx=0).stop 2024-09-10T09:07:19.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.379+0000 7fcdcf258640 1 -- 172.21.15.172:0/975518799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcdc8106f10 msgr2=0x7fcdc819ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:19.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.379+0000 7fcdcf258640 1 --2- 172.21.15.172:0/975518799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcdc8106f10 0x7fcdc819ee70 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009fd0 tx=0x7fcdc000b260 comp rx=0 tx=0).stop 2024-09-10T09:07:19.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.379+0000 7fcdcf258640 1 -- 172.21.15.172:0/975518799 shutdown_connections 2024-09-10T09:07:19.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.379+0000 7fcdcf258640 1 --2- 172.21.15.172:0/975518799 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fcdb00a2b80 0x7fcdb00a5040 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:19.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.379+0000 7fcdcf258640 1 --2- 172.21.15.172:0/975518799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcdc8106f10 0x7fcdc819ee70 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:19.380 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.379+0000 7fcdcf258640 1 --2- 172.21.15.172:0/975518799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdc8102790 0x7fcdc819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:19.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.379+0000 7fcdcf258640 1 --2- 172.21.15.172:0/975518799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdc8101de0 0x7fcdc819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:19.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.379+0000 7fcdcf258640 1 -- 172.21.15.172:0/975518799 >> 172.21.15.172:0/975518799 conn(0x7fcdc80fd8c0 msgr2=0x7fcdc80fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:19.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.379+0000 7fcdcf258640 1 -- 172.21.15.172:0/975518799 shutdown_connections 2024-09-10T09:07:19.381 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:19.379+0000 7fcdcf258640 1 -- 172.21.15.172:0/975518799 wait complete. 2024-09-10T09:07:19.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:19 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/975518799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:07:20.047 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:19 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/975518799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:07:20.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:19 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/975518799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:07:20.075 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-10T09:07:20.322 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:07:20.742 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:20 smithi172 ceph-mon[30854]: pgmap v983: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T09:07:20.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.822+0000 7efdfd004640 1 -- 172.21.15.172:0/239144581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf8106a80 msgr2=0x7efdf8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:20.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.822+0000 7efdfd004640 1 --2- 172.21.15.172:0/239144581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf8106a80 0x7efdf8106e60 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7efde8009930 tx=0x7efde802a420 comp rx=0 tx=0).stop 2024-09-10T09:07:20.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.822+0000 7efdfd004640 1 -- 172.21.15.172:0/239144581 shutdown_connections 2024-09-10T09:07:20.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.822+0000 7efdfd004640 1 --2- 172.21.15.172:0/239144581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efdf8101420 0x7efdf810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:20.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.822+0000 7efdfd004640 1 --2- 172.21.15.172:0/239144581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf8100a80 0x7efdf8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:20.826 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.822+0000 7efdfd004640 1 --2- 172.21.15.172:0/239144581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf8106a80 0x7efdf8106e60 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:20.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.822+0000 7efdfd004640 1 -- 172.21.15.172:0/239144581 >> 172.21.15.172:0/239144581 conn(0x7efdf80fc7a0 msgr2=0x7efdf80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:20.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.822+0000 7efdfd004640 1 -- 172.21.15.172:0/239144581 shutdown_connections 2024-09-10T09:07:20.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.822+0000 7efdfd004640 1 -- 172.21.15.172:0/239144581 wait complete. 2024-09-10T09:07:20.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdfd004640 1 Processor -- start 2024-09-10T09:07:20.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdfd004640 1 -- start start 2024-09-10T09:07:20.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdfd004640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf8100a80 0x7efdf8197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:20.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdfd004640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efdf8101420 0x7efdf8197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:20.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdfd004640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf8106a80 0x7efdf819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:20.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdfd004640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efdf8110d30 con 0x7efdf8100a80 2024-09-10T09:07:20.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdfd004640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efdf8110bb0 con 0x7efdf8106a80 2024-09-10T09:07:20.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdfd004640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efdf8110eb0 con 0x7efdf8101420 2024-09-10T09:07:20.827 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdf6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf8100a80 0x7efdf8197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:20.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdf6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf8100a80 0x7efdf8197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47252/0 (socket says 172.21.15.172:47252) 2024-09-10T09:07:20.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdf6d9d640 1 -- 172.21.15.172:0/761995998 learned_addr learned my addr 172.21.15.172:0/761995998 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:07:20.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdf659c640 1 --2- 172.21.15.172:0/761995998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efdf8101420 0x7efdf8197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:20.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdf6d9d640 1 -- 172.21.15.172:0/761995998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efdf8101420 msgr2=0x7efdf8197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:20.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdf6d9d640 1 --2- 172.21.15.172:0/761995998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efdf8101420 0x7efdf8197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:20.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdf6d9d640 1 -- 172.21.15.172:0/761995998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf8106a80 msgr2=0x7efdf819bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:20.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdf759e640 1 --2- 172.21.15.172:0/761995998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf8106a80 0x7efdf819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:20.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdf6d9d640 1 --2- 172.21.15.172:0/761995998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf8106a80 0x7efdf819bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:20.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdf6d9d640 1 -- 172.21.15.172:0/761995998 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf819c1c0 con 0x7efdf8100a80 2024-09-10T09:07:20.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.823+0000 7efdf659c640 1 --2- 172.21.15.172:0/761995998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efdf8101420 0x7efdf8197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:07:20.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.824+0000 7efdf6d9d640 1 --2- 172.21.15.172:0/761995998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf8100a80 0x7efdf8197210 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7efde8004770 tx=0x7efde8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:20.828 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.824+0000 7efddffff640 1 -- 172.21.15.172:0/761995998 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde8035050 con 0x7efdf8100a80 2024-09-10T09:07:20.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.824+0000 7efdfd004640 1 -- 172.21.15.172:0/761995998 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf819c450 con 0x7efdf8100a80 2024-09-10T09:07:20.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.824+0000 7efddffff640 1 -- 172.21.15.172:0/761995998 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efde8003aa0 con 0x7efdf8100a80 2024-09-10T09:07:20.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.824+0000 7efdfd004640 1 -- 172.21.15.172:0/761995998 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf806a790 con 0x7efdf8100a80 2024-09-10T09:07:20.829 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.824+0000 7efddffff640 1 -- 172.21.15.172:0/761995998 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde8038ac0 con 0x7efdf8100a80 2024-09-10T09:07:20.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.826+0000 7efdfd004640 1 -- 172.21.15.172:0/761995998 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf81021c0 con 0x7efdf8100a80 2024-09-10T09:07:20.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.826+0000 7efddffff640 1 -- 172.21.15.172:0/761995998 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efde8003c50 con 0x7efdf8100a80 2024-09-10T09:07:20.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.827+0000 7efddffff640 1 --2- 172.21.15.172:0/761995998 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efde00a2b80 0x7efde00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:20.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.827+0000 7efdf659c640 1 --2- 172.21.15.172:0/761995998 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efde00a2b80 0x7efde00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:20.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.828+0000 7efddffff640 1 -- 172.21.15.172:0/761995998 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9100+0+0 (secure 0 0 0) 0x7efde80f1280 con 0x7efdf8100a80 2024-09-10T09:07:20.830 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.828+0000 7efdf659c640 1 --2- 172.21.15.172:0/761995998 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efde00a2b80 0x7efde00a5040 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efdec0058c0 tx=0x7efdec005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:20.832 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:20.832+0000 7efddffff640 1 -- 172.21.15.172:0/761995998 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efde80bde20 con 0x7efdf8100a80 2024-09-10T09:07:20.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:20 smithi174 ceph-mon[34068]: pgmap v983: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T09:07:21.032 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:21.031+0000 7efdfd004640 1 -- 172.21.15.172:0/761995998 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efdf806b270 con 0x7efdf8100a80 2024-09-10T09:07:21.034 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:21.033+0000 7efddffff640 1 -- 172.21.15.172:0/761995998 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) ==== 161+0+0 (secure 0 0 0) 0x7efde80bd720 con 0x7efdf8100a80 2024-09-10T09:07:21.034 INFO:teuthology.orchestra.run.smithi172.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T09:07:21.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:21.037+0000 7efdfd004640 1 -- 172.21.15.172:0/761995998 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efde00a2b80 msgr2=0x7efde00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:21.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:21.037+0000 7efdfd004640 1 --2- 172.21.15.172:0/761995998 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efde00a2b80 0x7efde00a5040 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efdec0058c0 tx=0x7efdec005850 comp rx=0 tx=0).stop 2024-09-10T09:07:21.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:21.037+0000 7efdfd004640 1 -- 172.21.15.172:0/761995998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf8100a80 msgr2=0x7efdf8197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:21.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:21.037+0000 7efdfd004640 1 --2- 172.21.15.172:0/761995998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf8100a80 0x7efdf8197210 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7efde8004770 tx=0x7efde8004270 comp rx=0 tx=0).stop 2024-09-10T09:07:21.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:21.038+0000 7efdfd004640 1 -- 172.21.15.172:0/761995998 shutdown_connections 2024-09-10T09:07:21.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:21.038+0000 7efdfd004640 1 --2- 172.21.15.172:0/761995998 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7efde00a2b80 0x7efde00a5040 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:21.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:21.038+0000 7efdfd004640 1 --2- 172.21.15.172:0/761995998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf8106a80 0x7efdf819bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:21.039 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:21.038+0000 7efdfd004640 1 --2- 172.21.15.172:0/761995998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efdf8101420 0x7efdf8197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:21.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:21.038+0000 7efdfd004640 1 --2- 172.21.15.172:0/761995998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdf8100a80 0x7efdf8197210 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:21.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:21.038+0000 7efdfd004640 1 -- 172.21.15.172:0/761995998 >> 172.21.15.172:0/761995998 conn(0x7efdf80fc7a0 msgr2=0x7efdf80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:21.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:21.038+0000 7efdfd004640 1 -- 172.21.15.172:0/761995998 shutdown_connections 2024-09-10T09:07:21.040 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:21.038+0000 7efdfd004640 1 -- 172.21.15.172:0/761995998 wait complete. 2024-09-10T09:07:21.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:20 smithi191 ceph-mon[35314]: pgmap v983: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T09:07:21.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:21 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a[30850]: 2024-09-10T09:07:21.032+0000 7fa12614c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T09:07:21.333 INFO:journalctl@ceph.mds.g.smithi172.stdout:Sep 10 09:07:21 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-g[69048]: ignoring --setuser ceph since I am not root 2024-09-10T09:07:21.333 INFO:journalctl@ceph.mds.g.smithi172.stdout:Sep 10 09:07:21 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-g[69048]: ignoring --setgroup ceph since I am not root 2024-09-10T09:07:21.333 INFO:journalctl@ceph.mds.g.smithi172.stdout:Sep 10 09:07:21 smithi172 ceph-mds[69056]: -- 172.21.15.172:0/3989274495 <== mon.0 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5594b2861860 con 0x5594b283f400 2024-09-10T09:07:21.333 INFO:journalctl@ceph.mds.g.smithi172.stdout:Sep 10 09:07:21 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-g[69048]: starting mds.g at 2024-09-10T09:07:21.333 INFO:journalctl@ceph.mds.a.smithi172.stdout:Sep 10 09:07:21 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-a[69385]: ignoring --setuser ceph since I am not root 2024-09-10T09:07:21.334 INFO:journalctl@ceph.mds.a.smithi172.stdout:Sep 10 09:07:21 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-a[69385]: ignoring --setgroup ceph since I am not root 2024-09-10T09:07:21.334 INFO:journalctl@ceph.mds.a.smithi172.stdout:Sep 10 09:07:21 smithi172 ceph-mds[69401]: -- 172.21.15.172:0/2405469336 <== mon.0 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56509f351860 con 0x56509f32e800 2024-09-10T09:07:21.334 INFO:journalctl@ceph.mds.a.smithi172.stdout:Sep 10 09:07:21 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-a[69385]: starting mds.a at 2024-09-10T09:07:21.492 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-b[63897]: ignoring --setuser ceph since I am not root 2024-09-10T09:07:21.492 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-b[63897]: ignoring --setgroup ceph since I am not root 2024-09-10T09:07:21.492 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-mds[63901]: -- 172.21.15.174:0/2532333571 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d74a0f4680 con 0x55d74a0c8800 2024-09-10T09:07:21.493 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-mds[63901]: -- 172.21.15.174:0/2532333571 <== mon.2 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d74a0ebc20 con 0x55d74a0c8800 2024-09-10T09:07:21.493 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-b[63897]: starting mds.b at 2024-09-10T09:07:21.493 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-k[62978]: ignoring --setuser ceph since I am not root 2024-09-10T09:07:21.493 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-k[62978]: ignoring --setgroup ceph since I am not root 2024-09-10T09:07:21.493 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-mds[62982]: -- 172.21.15.174:0/3671792603 <== mon.0 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563915f35860 con 0x563915f13000 2024-09-10T09:07:21.493 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-k[62978]: starting mds.k at 2024-09-10T09:07:21.494 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-e[63283]: ignoring --setuser ceph since I am not root 2024-09-10T09:07:21.494 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-e[63283]: ignoring --setgroup ceph since I am not root 2024-09-10T09:07:21.494 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-mds[63287]: -- 172.21.15.174:0/3100190310 <== mon.2 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555daaa29860 con 0x555daaa06800 2024-09-10T09:07:21.494 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-e[63283]: starting mds.e at 2024-09-10T09:07:21.494 INFO:journalctl@ceph.mds.h.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-h[63584]: ignoring --setuser ceph since I am not root 2024-09-10T09:07:21.494 INFO:journalctl@ceph.mds.h.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-h[63584]: ignoring --setgroup ceph since I am not root 2024-09-10T09:07:21.494 INFO:journalctl@ceph.mds.h.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-h[63584]: starting mds.h at 2024-09-10T09:07:21.550 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-c[64990]: ignoring --setuser ceph since I am not root 2024-09-10T09:07:21.551 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-c[64990]: ignoring --setgroup ceph since I am not root 2024-09-10T09:07:21.551 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-mds[64994]: -- 172.21.15.191:0/3202023794 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5653417b1860 con 0x56534178f000 2024-09-10T09:07:21.551 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-c[64990]: starting mds.c at 2024-09-10T09:07:21.551 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-l[64389]: ignoring --setuser ceph since I am not root 2024-09-10T09:07:21.551 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-l[64389]: ignoring --setgroup ceph since I am not root 2024-09-10T09:07:21.551 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-mds[64393]: -- 172.21.15.191:0/3987438363 <== mon.0 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55845f49b860 con 0x55845f478800 2024-09-10T09:07:21.551 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-l[64389]: starting mds.l at 2024-09-10T09:07:21.552 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-f[64693]: ignoring --setuser ceph since I am not root 2024-09-10T09:07:21.552 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-f[64693]: ignoring --setgroup ceph since I am not root 2024-09-10T09:07:21.552 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-mds[64697]: -- 172.21.15.191:0/2707813195 <== mon.0 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560905659c20 con 0x560905636800 2024-09-10T09:07:21.552 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-f[64693]: starting mds.f at 2024-09-10T09:07:21.552 INFO:journalctl@ceph.mds.i.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-i[65292]: ignoring --setuser ceph since I am not root 2024-09-10T09:07:21.552 INFO:journalctl@ceph.mds.i.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-i[65292]: ignoring --setgroup ceph since I am not root 2024-09-10T09:07:21.552 INFO:journalctl@ceph.mds.i.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-mds[65296]: -- 172.21.15.191:0/275369119 <== mon.0 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5638a4459860 con 0x5638a4437000 2024-09-10T09:07:21.552 INFO:journalctl@ceph.mds.i.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-i[65292]: starting mds.i at 2024-09-10T09:07:21.725 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T09:07:21.975 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:07:21.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/761995998' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T09:07:21.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-mon[34068]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T09:07:21.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-mon[34068]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T09:07:21.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/761995998' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T09:07:21.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-mon[34068]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T09:07:21.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:21 smithi174 ceph-mon[34068]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T09:07:22.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/761995998' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T09:07:22.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-mon[35314]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T09:07:22.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-mon[35314]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T09:07:22.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/761995998' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T09:07:22.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-mon[35314]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T09:07:22.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:21 smithi191 ceph-mon[35314]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T09:07:22.082 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:21 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/761995998' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T09:07:22.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:21 smithi172 ceph-mon[30854]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T09:07:22.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:21 smithi172 ceph-mon[30854]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T09:07:22.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:21 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/761995998' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T09:07:22.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:21 smithi172 ceph-mon[30854]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T09:07:22.083 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:21 smithi172 ceph-mon[30854]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T09:07:22.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.451+0000 7f4993bd9640 1 -- 172.21.15.172:0/752429341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f498c10c180 msgr2=0x7f498c10e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:22.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.452+0000 7f4993bd9640 1 --2- 172.21.15.172:0/752429341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f498c10c180 0x7f498c10e620 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f4984009900 tx=0x7f498402a480 comp rx=0 tx=0).stop 2024-09-10T09:07:22.469 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.452+0000 7f4993bd9640 1 -- 172.21.15.172:0/752429341 shutdown_connections 2024-09-10T09:07:22.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.452+0000 7f4993bd9640 1 --2- 172.21.15.172:0/752429341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f498c10c180 0x7f498c10e620 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:22.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.452+0000 7f4993bd9640 1 --2- 172.21.15.172:0/752429341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f498c103670 0x7f498c10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:22.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.452+0000 7f4993bd9640 1 --2- 172.21.15.172:0/752429341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f498c102d50 0x7f498c103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:22.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.452+0000 7f4993bd9640 1 -- 172.21.15.172:0/752429341 >> 172.21.15.172:0/752429341 conn(0x7f498c0fc7d0 msgr2=0x7f498c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:22.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.452+0000 7f4993bd9640 1 -- 172.21.15.172:0/752429341 shutdown_connections 2024-09-10T09:07:22.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.452+0000 7f4993bd9640 1 -- 172.21.15.172:0/752429341 wait complete. 2024-09-10T09:07:22.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.452+0000 7f4993bd9640 1 Processor -- start 2024-09-10T09:07:22.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.452+0000 7f4993bd9640 1 -- start start 2024-09-10T09:07:22.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.452+0000 7f4993bd9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f498c102d50 0x7f498c1009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:22.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.452+0000 7f4993bd9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f498c103670 0x7f498c100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:22.470 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.452+0000 7f4993bd9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f498c10c180 0x7f498c101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:22.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.452+0000 7f4993bd9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f498c111e30 con 0x7f498c102d50 2024-09-10T09:07:22.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.452+0000 7f4993bd9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f498c111cb0 con 0x7f498c10c180 2024-09-10T09:07:22.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.452+0000 7f4993bd9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f498c111fb0 con 0x7f498c103670 2024-09-10T09:07:22.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.453+0000 7f4991975640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f498c102d50 0x7f498c1009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:22.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.453+0000 7f4991975640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f498c102d50 0x7f498c1009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47286/0 (socket says 172.21.15.172:47286) 2024-09-10T09:07:22.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.453+0000 7f4991975640 1 -- 172.21.15.172:0/2482116597 learned_addr learned my addr 172.21.15.172:0/2482116597 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:07:22.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.453+0000 7f4992176640 1 --2- 172.21.15.172:0/2482116597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f498c10c180 0x7f498c101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:22.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.453+0000 7f4991975640 1 -- 172.21.15.172:0/2482116597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f498c103670 msgr2=0x7f498c100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:22.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.453+0000 7f4991174640 1 --2- 172.21.15.172:0/2482116597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f498c103670 0x7f498c100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:22.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.453+0000 7f4991975640 1 --2- 172.21.15.172:0/2482116597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f498c103670 0x7f498c100ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:22.471 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.453+0000 7f4991975640 1 -- 172.21.15.172:0/2482116597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f498c10c180 msgr2=0x7f498c101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:22.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.453+0000 7f4991975640 1 --2- 172.21.15.172:0/2482116597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f498c10c180 0x7f498c101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:22.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.453+0000 7f4991975640 1 -- 172.21.15.172:0/2482116597 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498c1a66b0 con 0x7f498c102d50 2024-09-10T09:07:22.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.453+0000 7f4992176640 1 --2- 172.21.15.172:0/2482116597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f498c10c180 0x7f498c101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:07:22.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.453+0000 7f4991975640 1 --2- 172.21.15.172:0/2482116597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f498c102d50 0x7f498c1009b0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f497c00cce0 tx=0x7f497c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:22.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.453+0000 7f498affd640 1 -- 172.21.15.172:0/2482116597 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c00f650 con 0x7f498c102d50 2024-09-10T09:07:22.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.453+0000 7f498affd640 1 -- 172.21.15.172:0/2482116597 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f497c00fca0 con 0x7f498c102d50 2024-09-10T09:07:22.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.453+0000 7f498affd640 1 -- 172.21.15.172:0/2482116597 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c018b90 con 0x7f498c102d50 2024-09-10T09:07:22.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.453+0000 7f4993bd9640 1 -- 172.21.15.172:0/2482116597 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498c1a69a0 con 0x7f498c102d50 2024-09-10T09:07:22.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.453+0000 7f4993bd9640 1 -- 172.21.15.172:0/2482116597 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498c1a6ee0 con 0x7f498c102d50 2024-09-10T09:07:22.472 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.454+0000 7f4993bd9640 1 -- 172.21.15.172:0/2482116597 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f498c103130 con 0x7f498c102d50 2024-09-10T09:07:22.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.455+0000 7f498affd640 1 -- 172.21.15.172:0/2482116597 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f497c00f7f0 con 0x7f498c102d50 2024-09-10T09:07:22.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.456+0000 7f498affd640 1 --2- 172.21.15.172:0/2482116597 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49740a2b80 0x7f49740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:22.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.456+0000 7f498affd640 1 -- 172.21.15.172:0/2482116597 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9530+0+0 (secure 0 0 0) 0x7f497c0cfd60 con 0x7f498c102d50 2024-09-10T09:07:22.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.456+0000 7f4991174640 1 --2- 172.21.15.172:0/2482116597 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49740a2b80 0x7f49740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:22.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.457+0000 7f4991174640 1 --2- 172.21.15.172:0/2482116597 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49740a2b80 0x7f49740a5040 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f498c102440 tx=0x7f4980005cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:22.473 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.459+0000 7f498affd640 1 -- 172.21.15.172:0/2482116597 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f497c014030 con 0x7f498c102d50 2024-09-10T09:07:22.660 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:22.660+0000 7f4993bd9640 1 -- 172.21.15.172:0/2482116597 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f498c1a7240 con 0x7f498c102d50 2024-09-10T09:07:23.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:23.048+0000 7f498affd640 1 -- 172.21.15.172:0/2482116597 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) ==== 104+0+0 (secure 0 0 0) 0x7f497c09c840 con 0x7f498c102d50 2024-09-10T09:07:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:23.051+0000 7f4993bd9640 1 -- 172.21.15.172:0/2482116597 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49740a2b80 msgr2=0x7f49740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:23.051+0000 7f4993bd9640 1 --2- 172.21.15.172:0/2482116597 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49740a2b80 0x7f49740a5040 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f498c102440 tx=0x7f4980005cf0 comp rx=0 tx=0).stop 2024-09-10T09:07:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:23.051+0000 7f4993bd9640 1 -- 172.21.15.172:0/2482116597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f498c102d50 msgr2=0x7f498c1009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:23.051+0000 7f4993bd9640 1 --2- 172.21.15.172:0/2482116597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f498c102d50 0x7f498c1009b0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f497c00cce0 tx=0x7f497c00a620 comp rx=0 tx=0).stop 2024-09-10T09:07:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:23.051+0000 7f4993bd9640 1 -- 172.21.15.172:0/2482116597 shutdown_connections 2024-09-10T09:07:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:23.051+0000 7f4993bd9640 1 --2- 172.21.15.172:0/2482116597 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f49740a2b80 0x7f49740a5040 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:23.051+0000 7f4993bd9640 1 --2- 172.21.15.172:0/2482116597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f498c10c180 0x7f498c101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:23.051+0000 7f4993bd9640 1 --2- 172.21.15.172:0/2482116597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f498c103670 0x7f498c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:23.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:23.051+0000 7f4993bd9640 1 --2- 172.21.15.172:0/2482116597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f498c102d50 0x7f498c1009b0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:23.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:23.051+0000 7f4993bd9640 1 -- 172.21.15.172:0/2482116597 >> 172.21.15.172:0/2482116597 conn(0x7f498c0fc7d0 msgr2=0x7f498c0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:23.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:23.051+0000 7f4993bd9640 1 -- 172.21.15.172:0/2482116597 shutdown_connections 2024-09-10T09:07:23.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:23.051+0000 7f4993bd9640 1 -- 172.21.15.172:0/2482116597 wait complete. 2024-09-10T09:07:23.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: pgmap v985: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 818 B/s wr, 11 op/s 2024-09-10T09:07:23.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.191:6833/2592985306,v1:172.21.15.191:6835/2592985306] up:boot 2024-09-10T09:07:23.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.191:6832/2830077451,v1:172.21.15.191:6834/2830077451] up:boot 2024-09-10T09:07:23.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.191:6836/2901090457,v1:172.21.15.191:6837/2901090457] up:boot 2024-09-10T09:07:23.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.174:6832/564334275,v1:172.21.15.174:6834/564334275] up:boot 2024-09-10T09:07:23.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.174:6833/1707844835,v1:172.21.15.174:6835/1707844835] up:boot 2024-09-10T09:07:23.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.174:6836/3943326999,v1:172.21.15.174:6837/3943326999] up:boot 2024-09-10T09:07:23.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.191:6838/4214669195,v1:172.21.15.191:6839/4214669195] up:boot 2024-09-10T09:07:23.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.174:6838/2139981333,v1:172.21.15.174:6839/2139981333] up:boot 2024-09-10T09:07:23.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.172:6838/207193914,v1:172.21.15.172:6840/207193914] up:boot 2024-09-10T09:07:23.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: mds.? [v2:172.21.15.172:6839/744949407,v1:172.21.15.172:6841/744949407] up:boot 2024-09-10T09:07:23.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T09:07:23.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T09:07:23.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T09:07:23.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T09:07:23.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T09:07:23.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T09:07:23.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T09:07:23.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T09:07:23.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T09:07:23.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T09:07:23.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T09:07:23.302 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:23 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2482116597' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:07:23.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: pgmap v985: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 818 B/s wr, 11 op/s 2024-09-10T09:07:23.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.191:6833/2592985306,v1:172.21.15.191:6835/2592985306] up:boot 2024-09-10T09:07:23.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.191:6832/2830077451,v1:172.21.15.191:6834/2830077451] up:boot 2024-09-10T09:07:23.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.191:6836/2901090457,v1:172.21.15.191:6837/2901090457] up:boot 2024-09-10T09:07:23.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.174:6832/564334275,v1:172.21.15.174:6834/564334275] up:boot 2024-09-10T09:07:23.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.174:6833/1707844835,v1:172.21.15.174:6835/1707844835] up:boot 2024-09-10T09:07:23.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.174:6836/3943326999,v1:172.21.15.174:6837/3943326999] up:boot 2024-09-10T09:07:23.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.191:6838/4214669195,v1:172.21.15.191:6839/4214669195] up:boot 2024-09-10T09:07:23.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.174:6838/2139981333,v1:172.21.15.174:6839/2139981333] up:boot 2024-09-10T09:07:23.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.172:6838/207193914,v1:172.21.15.172:6840/207193914] up:boot 2024-09-10T09:07:23.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: mds.? [v2:172.21.15.172:6839/744949407,v1:172.21.15.172:6841/744949407] up:boot 2024-09-10T09:07:23.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T09:07:23.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T09:07:23.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T09:07:23.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T09:07:23.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T09:07:23.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T09:07:23.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T09:07:23.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T09:07:23.334 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T09:07:23.335 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T09:07:23.335 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T09:07:23.335 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:23 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2482116597' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:07:23.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: pgmap v985: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 818 B/s wr, 11 op/s 2024-09-10T09:07:23.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.191:6833/2592985306,v1:172.21.15.191:6835/2592985306] up:boot 2024-09-10T09:07:23.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.191:6832/2830077451,v1:172.21.15.191:6834/2830077451] up:boot 2024-09-10T09:07:23.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.191:6836/2901090457,v1:172.21.15.191:6837/2901090457] up:boot 2024-09-10T09:07:23.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.174:6832/564334275,v1:172.21.15.174:6834/564334275] up:boot 2024-09-10T09:07:23.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.174:6833/1707844835,v1:172.21.15.174:6835/1707844835] up:boot 2024-09-10T09:07:23.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.174:6836/3943326999,v1:172.21.15.174:6837/3943326999] up:boot 2024-09-10T09:07:23.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.191:6838/4214669195,v1:172.21.15.191:6839/4214669195] up:boot 2024-09-10T09:07:23.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.174:6838/2139981333,v1:172.21.15.174:6839/2139981333] up:boot 2024-09-10T09:07:23.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.172:6838/207193914,v1:172.21.15.172:6840/207193914] up:boot 2024-09-10T09:07:23.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: mds.? [v2:172.21.15.172:6839/744949407,v1:172.21.15.172:6841/744949407] up:boot 2024-09-10T09:07:23.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T09:07:23.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T09:07:23.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T09:07:23.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T09:07:23.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T09:07:23.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T09:07:23.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T09:07:23.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T09:07:23.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T09:07:23.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T09:07:23.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: from='mgr.14150 172.21.15.172:0/1803944482' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T09:07:23.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:23 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2482116597' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:07:23.790 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T09:07:24.038 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:07:24.300 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:24 smithi191 ceph-mon[35314]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T09:07:24.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:24 smithi191 ceph-mon[35314]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T09:07:24.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:24 smithi191 ceph-mon[35314]: Cluster is now healthy 2024-09-10T09:07:24.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:24 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/2482116597' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T09:07:24.301 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:24 smithi191 ceph-mon[35314]: fsmap 12 up:standby 2024-09-10T09:07:24.310 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:24 smithi172 ceph-mon[30854]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T09:07:24.311 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:24 smithi172 ceph-mon[30854]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T09:07:24.311 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:24 smithi172 ceph-mon[30854]: Cluster is now healthy 2024-09-10T09:07:24.311 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:24 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/2482116597' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T09:07:24.311 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:24 smithi172 ceph-mon[30854]: fsmap 12 up:standby 2024-09-10T09:07:24.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:24 smithi174 ceph-mon[34068]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T09:07:24.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:24 smithi174 ceph-mon[34068]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T09:07:24.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:24 smithi174 ceph-mon[34068]: Cluster is now healthy 2024-09-10T09:07:24.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:24 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/2482116597' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T09:07:24.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:24 smithi174 ceph-mon[34068]: fsmap 12 up:standby 2024-09-10T09:07:24.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.535+0000 7fe09d1a2640 1 -- 172.21.15.172:0/3366273810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe098102790 msgr2=0x7fe0981067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:24.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.535+0000 7fe09d1a2640 1 --2- 172.21.15.172:0/3366273810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe098102790 0x7fe0981067e0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fe08c009900 tx=0x7fe08c02a410 comp rx=0 tx=0).stop 2024-09-10T09:07:24.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.535+0000 7fe09d1a2640 1 -- 172.21.15.172:0/3366273810 shutdown_connections 2024-09-10T09:07:24.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.535+0000 7fe09d1a2640 1 --2- 172.21.15.172:0/3366273810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe098106f10 0x7fe09810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:24.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.535+0000 7fe09d1a2640 1 --2- 172.21.15.172:0/3366273810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe098102790 0x7fe0981067e0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:24.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.535+0000 7fe09d1a2640 1 --2- 172.21.15.172:0/3366273810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe098101de0 0x7fe0981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:24.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.535+0000 7fe09d1a2640 1 -- 172.21.15.172:0/3366273810 >> 172.21.15.172:0/3366273810 conn(0x7fe0980fd8c0 msgr2=0x7fe0980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:24.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.535+0000 7fe09d1a2640 1 -- 172.21.15.172:0/3366273810 shutdown_connections 2024-09-10T09:07:24.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.536+0000 7fe09d1a2640 1 -- 172.21.15.172:0/3366273810 wait complete. 2024-09-10T09:07:24.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.536+0000 7fe09d1a2640 1 Processor -- start 2024-09-10T09:07:24.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.536+0000 7fe09d1a2640 1 -- start start 2024-09-10T09:07:24.545 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe09d1a2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe098101de0 0x7fe0980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:24.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe09d1a2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe098102790 0x7fe098074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:24.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe09d1a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe098106f10 0x7fe098074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:24.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe09d1a2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe098111ec0 con 0x7fe098101de0 2024-09-10T09:07:24.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe09d1a2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe098111d40 con 0x7fe098106f10 2024-09-10T09:07:24.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe09d1a2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe098112040 con 0x7fe098102790 2024-09-10T09:07:24.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe096d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe098101de0 0x7fe0980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:24.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe096d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe098101de0 0x7fe0980744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:47298/0 (socket says 172.21.15.172:47298) 2024-09-10T09:07:24.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe096d9d640 1 -- 172.21.15.172:0/1577301398 learned_addr learned my addr 172.21.15.172:0/1577301398 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:07:24.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe09659c640 1 --2- 172.21.15.172:0/1577301398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe098102790 0x7fe098074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:24.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe09759e640 1 --2- 172.21.15.172:0/1577301398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe098106f10 0x7fe098074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:24.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe096d9d640 1 -- 172.21.15.172:0/1577301398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe098102790 msgr2=0x7fe098074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:24.546 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe096d9d640 1 --2- 172.21.15.172:0/1577301398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe098102790 0x7fe098074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:24.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe096d9d640 1 -- 172.21.15.172:0/1577301398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe098106f10 msgr2=0x7fe098074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:24.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe096d9d640 1 --2- 172.21.15.172:0/1577301398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe098106f10 0x7fe098074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:24.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe096d9d640 1 -- 172.21.15.172:0/1577301398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0981a69e0 con 0x7fe098101de0 2024-09-10T09:07:24.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe09759e640 1 --2- 172.21.15.172:0/1577301398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe098106f10 0x7fe098074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:07:24.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe096d9d640 1 --2- 172.21.15.172:0/1577301398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe098101de0 0x7fe0980744f0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fe08800b4a0 tx=0x7fe08800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:24.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe07ffff640 1 -- 172.21.15.172:0/1577301398 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088014070 con 0x7fe098101de0 2024-09-10T09:07:24.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe09d1a2640 1 -- 172.21.15.172:0/1577301398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0981a6cd0 con 0x7fe098101de0 2024-09-10T09:07:24.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.537+0000 7fe07ffff640 1 -- 172.21.15.172:0/1577301398 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe088010ac0 con 0x7fe098101de0 2024-09-10T09:07:24.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.538+0000 7fe07ffff640 1 -- 172.21.15.172:0/1577301398 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088017c30 con 0x7fe098101de0 2024-09-10T09:07:24.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.538+0000 7fe09d1a2640 1 -- 172.21.15.172:0/1577301398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0981a71d0 con 0x7fe098101de0 2024-09-10T09:07:24.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.538+0000 7fe09d1a2640 1 -- 172.21.15.172:0/1577301398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe098102240 con 0x7fe098101de0 2024-09-10T09:07:24.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.540+0000 7fe07ffff640 1 -- 172.21.15.172:0/1577301398 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe088017070 con 0x7fe098101de0 2024-09-10T09:07:24.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.540+0000 7fe07ffff640 1 --2- 172.21.15.172:0/1577301398 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0800a2b80 0x7fe0800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:24.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.540+0000 7fe09659c640 1 --2- 172.21.15.172:0/1577301398 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0800a2b80 0x7fe0800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:24.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.540+0000 7fe07ffff640 1 -- 172.21.15.172:0/1577301398 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9530+0+0 (secure 0 0 0) 0x7fe0880cecc0 con 0x7fe098101de0 2024-09-10T09:07:24.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.541+0000 7fe09659c640 1 --2- 172.21.15.172:0/1577301398 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0800a2b80 0x7fe0800a5040 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe08c031440 tx=0x7fe08c0095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:24.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.545+0000 7fe07ffff640 1 -- 172.21.15.172:0/1577301398 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe08809b710 con 0x7fe098101de0 2024-09-10T09:07:24.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:24.711+0000 7fe09d1a2640 1 -- 172.21.15.172:0/1577301398 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe09810a0f0 con 0x7fe098101de0 2024-09-10T09:07:25.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:25.051+0000 7fe07ffff640 1 -- 172.21.15.172:0/1577301398 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) ==== 181+0+0 (secure 0 0 0) 0x7fe08809b010 con 0x7fe098101de0 2024-09-10T09:07:25.052 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_metadata' removed 2024-09-10T09:07:25.056 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:25.055+0000 7fe09d1a2640 1 -- 172.21.15.172:0/1577301398 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0800a2b80 msgr2=0x7fe0800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:25.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:25.055+0000 7fe09d1a2640 1 --2- 172.21.15.172:0/1577301398 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0800a2b80 0x7fe0800a5040 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe08c031440 tx=0x7fe08c0095f0 comp rx=0 tx=0).stop 2024-09-10T09:07:25.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:25.055+0000 7fe09d1a2640 1 -- 172.21.15.172:0/1577301398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe098101de0 msgr2=0x7fe0980744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:25.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:25.055+0000 7fe09d1a2640 1 --2- 172.21.15.172:0/1577301398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe098101de0 0x7fe0980744f0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fe08800b4a0 tx=0x7fe08800b960 comp rx=0 tx=0).stop 2024-09-10T09:07:25.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:25.055+0000 7fe09d1a2640 1 -- 172.21.15.172:0/1577301398 shutdown_connections 2024-09-10T09:07:25.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:25.055+0000 7fe09d1a2640 1 --2- 172.21.15.172:0/1577301398 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7fe0800a2b80 0x7fe0800a5040 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:25.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:25.055+0000 7fe09d1a2640 1 --2- 172.21.15.172:0/1577301398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe098106f10 0x7fe098074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:25.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:25.055+0000 7fe09d1a2640 1 --2- 172.21.15.172:0/1577301398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe098102790 0x7fe098074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:25.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:25.055+0000 7fe09d1a2640 1 --2- 172.21.15.172:0/1577301398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe098101de0 0x7fe0980744f0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:25.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:25.055+0000 7fe09d1a2640 1 -- 172.21.15.172:0/1577301398 >> 172.21.15.172:0/1577301398 conn(0x7fe0980fd8c0 msgr2=0x7fe0980fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:25.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:25.055+0000 7fe09d1a2640 1 -- 172.21.15.172:0/1577301398 shutdown_connections 2024-09-10T09:07:25.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:25.056+0000 7fe09d1a2640 1 -- 172.21.15.172:0/1577301398 wait complete. 2024-09-10T09:07:25.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:25 smithi172 ceph-mon[30854]: pgmap v986: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-09-10T09:07:25.333 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:25 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1577301398' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T09:07:25.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:25 smithi174 ceph-mon[34068]: pgmap v986: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-09-10T09:07:25.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:25 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1577301398' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T09:07:25.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:25 smithi191 ceph-mon[35314]: pgmap v986: 129 pgs: 129 active+clean; 6.9 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-09-10T09:07:25.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:25 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1577301398' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T09:07:25.780 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T09:07:26.027 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:07:26.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:26 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/1577301398' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T09:07:26.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:26 smithi172 ceph-mon[30854]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T09:07:26.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:26 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/1577301398' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T09:07:26.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:26 smithi174 ceph-mon[34068]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T09:07:26.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.519+0000 7f1da6eeb640 1 -- 172.21.15.172:0/1736882431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da0100a80 msgr2=0x7f1da0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:26.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.519+0000 7f1da6eeb640 1 --2- 172.21.15.172:0/1736882431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da0100a80 0x7f1da0100ee0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009930 tx=0x7f1d8c02a420 comp rx=0 tx=0).stop 2024-09-10T09:07:26.538 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.519+0000 7f1da6eeb640 1 -- 172.21.15.172:0/1736882431 shutdown_connections 2024-09-10T09:07:26.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.519+0000 7f1da6eeb640 1 --2- 172.21.15.172:0/1736882431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da0101420 0x7f1da010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:26.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.519+0000 7f1da6eeb640 1 --2- 172.21.15.172:0/1736882431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da0100a80 0x7f1da0100ee0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:26.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.519+0000 7f1da6eeb640 1 --2- 172.21.15.172:0/1736882431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1da0106a80 0x7f1da0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:26.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.519+0000 7f1da6eeb640 1 -- 172.21.15.172:0/1736882431 >> 172.21.15.172:0/1736882431 conn(0x7f1da00fc7a0 msgr2=0x7f1da00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:26.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.519+0000 7f1da6eeb640 1 -- 172.21.15.172:0/1736882431 shutdown_connections 2024-09-10T09:07:26.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.519+0000 7f1da6eeb640 1 -- 172.21.15.172:0/1736882431 wait complete. 2024-09-10T09:07:26.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1da6eeb640 1 Processor -- start 2024-09-10T09:07:26.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1da6eeb640 1 -- start start 2024-09-10T09:07:26.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1da6eeb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da0100a80 0x7f1da0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:26.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1da6eeb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da0101420 0x7f1da01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:26.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1da6eeb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1da0106a80 0x7f1da019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:26.539 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1da6eeb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0110d50 con 0x7f1da0100a80 2024-09-10T09:07:26.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1da6eeb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0110bd0 con 0x7f1da0101420 2024-09-10T09:07:26.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1da6eeb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0110ed0 con 0x7f1da0106a80 2024-09-10T09:07:26.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1da4c87640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da0100a80 0x7f1da0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:26.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1da4c87640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da0100a80 0x7f1da0198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46162/0 (socket says 172.21.15.172:46162) 2024-09-10T09:07:26.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1da4c87640 1 -- 172.21.15.172:0/766685247 learned_addr learned my addr 172.21.15.172:0/766685247 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:07:26.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1d9ffff640 1 --2- 172.21.15.172:0/766685247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da0101420 0x7f1da01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:26.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1da4c87640 1 -- 172.21.15.172:0/766685247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1da0106a80 msgr2=0x7f1da019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:26.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1da5488640 1 --2- 172.21.15.172:0/766685247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1da0106a80 0x7f1da019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:26.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1da4c87640 1 --2- 172.21.15.172:0/766685247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1da0106a80 0x7f1da019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:26.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1da4c87640 1 -- 172.21.15.172:0/766685247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da0101420 msgr2=0x7f1da01988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:26.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1da4c87640 1 --2- 172.21.15.172:0/766685247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da0101420 0x7f1da01988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:26.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1da4c87640 1 -- 172.21.15.172:0/766685247 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da019d340 con 0x7f1da0100a80 2024-09-10T09:07:26.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.520+0000 7f1d9ffff640 1 --2- 172.21.15.172:0/766685247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da0101420 0x7f1da01988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:07:26.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.521+0000 7f1da4c87640 1 --2- 172.21.15.172:0/766685247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da0100a80 0x7f1da0198390 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400b4a0 tx=0x7f1d9400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:26.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.521+0000 7f1d9dffb640 1 -- 172.21.15.172:0/766685247 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94014070 con 0x7f1da0100a80 2024-09-10T09:07:26.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.521+0000 7f1d9dffb640 1 -- 172.21.15.172:0/766685247 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d94010ac0 con 0x7f1da0100a80 2024-09-10T09:07:26.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.521+0000 7f1da6eeb640 1 -- 172.21.15.172:0/766685247 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da019d630 con 0x7f1da0100a80 2024-09-10T09:07:26.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.521+0000 7f1d9dffb640 1 -- 172.21.15.172:0/766685247 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94017c30 con 0x7f1da0100a80 2024-09-10T09:07:26.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.521+0000 7f1da6eeb640 1 -- 172.21.15.172:0/766685247 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da01a4f10 con 0x7f1da0100a80 2024-09-10T09:07:26.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.522+0000 7f1da6eeb640 1 -- 172.21.15.172:0/766685247 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da0100ee0 con 0x7f1da0100a80 2024-09-10T09:07:26.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.523+0000 7f1d9dffb640 1 -- 172.21.15.172:0/766685247 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1d94017070 con 0x7f1da0100a80 2024-09-10T09:07:26.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.524+0000 7f1d9dffb640 1 --2- 172.21.15.172:0/766685247 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1d900a2b80 0x7f1d900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:26.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.524+0000 7f1d9dffb640 1 -- 172.21.15.172:0/766685247 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9026+0+0 (secure 0 0 0) 0x7f1d940ceae0 con 0x7f1da0100a80 2024-09-10T09:07:26.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.524+0000 7f1d9ffff640 1 --2- 172.21.15.172:0/766685247 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1d900a2b80 0x7f1d900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:26.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.524+0000 7f1d9ffff640 1 --2- 172.21.15.172:0/766685247 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1d900a2b80 0x7f1d900a5040 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c000c00 tx=0x7f1d8c031cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:26.542 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.528+0000 7f1d9dffb640 1 -- 172.21.15.172:0/766685247 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d9409b720 con 0x7f1da0100a80 2024-09-10T09:07:26.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:26 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/1577301398' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T09:07:26.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:26 smithi191 ceph-mon[35314]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T09:07:26.697 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:26.695+0000 7f1da6eeb640 1 -- 172.21.15.172:0/766685247 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1da002d7c0 con 0x7f1da0100a80 2024-09-10T09:07:27.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:27.068+0000 7f1d9dffb640 1 -- 172.21.15.172:0/766685247 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) ==== 169+0+0 (secure 0 0 0) 0x7f1d9409b020 con 0x7f1da0100a80 2024-09-10T09:07:27.070 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data' removed 2024-09-10T09:07:27.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:27.071+0000 7f1da6eeb640 1 -- 172.21.15.172:0/766685247 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1d900a2b80 msgr2=0x7f1d900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:27.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:27.071+0000 7f1da6eeb640 1 --2- 172.21.15.172:0/766685247 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1d900a2b80 0x7f1d900a5040 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c000c00 tx=0x7f1d8c031cf0 comp rx=0 tx=0).stop 2024-09-10T09:07:27.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:27.071+0000 7f1da6eeb640 1 -- 172.21.15.172:0/766685247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da0100a80 msgr2=0x7f1da0198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:27.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:27.071+0000 7f1da6eeb640 1 --2- 172.21.15.172:0/766685247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da0100a80 0x7f1da0198390 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400b4a0 tx=0x7f1d9400b960 comp rx=0 tx=0).stop 2024-09-10T09:07:27.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:27.071+0000 7f1da6eeb640 1 -- 172.21.15.172:0/766685247 shutdown_connections 2024-09-10T09:07:27.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:27.071+0000 7f1da6eeb640 1 --2- 172.21.15.172:0/766685247 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7f1d900a2b80 0x7f1d900a5040 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:27.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:27.071+0000 7f1da6eeb640 1 --2- 172.21.15.172:0/766685247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1da0106a80 0x7f1da019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:27.073 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:27.071+0000 7f1da6eeb640 1 --2- 172.21.15.172:0/766685247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da0101420 0x7f1da01988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:27.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:27.071+0000 7f1da6eeb640 1 --2- 172.21.15.172:0/766685247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da0100a80 0x7f1da0198390 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:27.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:27.071+0000 7f1da6eeb640 1 -- 172.21.15.172:0/766685247 >> 172.21.15.172:0/766685247 conn(0x7f1da00fc7a0 msgr2=0x7f1da006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:27.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:27.071+0000 7f1da6eeb640 1 -- 172.21.15.172:0/766685247 shutdown_connections 2024-09-10T09:07:27.074 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:27.071+0000 7f1da6eeb640 1 -- 172.21.15.172:0/766685247 wait complete. 2024-09-10T09:07:27.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:27 smithi172 ceph-mon[30854]: pgmap v988: 97 pgs: 97 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:07:27.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:27 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/766685247' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T09:07:27.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:27 smithi174 ceph-mon[34068]: pgmap v988: 97 pgs: 97 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:07:27.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:27 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/766685247' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T09:07:27.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:27 smithi191 ceph-mon[35314]: pgmap v988: 97 pgs: 97 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:07:27.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:27 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/766685247' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T09:07:27.798 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-10T09:07:28.047 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:07:28.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:28 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/766685247' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T09:07:28.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:28 smithi172 ceph-mon[30854]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T09:07:28.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:28 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/766685247' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T09:07:28.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:28 smithi174 ceph-mon[34068]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T09:07:28.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.527+0000 7ffb16b50640 1 -- 172.21.15.172:0/619437288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb10100a80 msgr2=0x7ffb10100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:28.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.527+0000 7ffb16b50640 1 --2- 172.21.15.172:0/619437288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb10100a80 0x7ffb10100ee0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009930 tx=0x7ffb0002a420 comp rx=0 tx=0).stop 2024-09-10T09:07:28.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.527+0000 7ffb16b50640 1 -- 172.21.15.172:0/619437288 shutdown_connections 2024-09-10T09:07:28.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.527+0000 7ffb16b50640 1 --2- 172.21.15.172:0/619437288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10101420 0x7ffb1010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:28.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.527+0000 7ffb16b50640 1 --2- 172.21.15.172:0/619437288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb10100a80 0x7ffb10100ee0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:28.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.527+0000 7ffb16b50640 1 --2- 172.21.15.172:0/619437288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb10106a80 0x7ffb10106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:28.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.527+0000 7ffb16b50640 1 -- 172.21.15.172:0/619437288 >> 172.21.15.172:0/619437288 conn(0x7ffb100fc7a0 msgr2=0x7ffb100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:28.547 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.527+0000 7ffb16b50640 1 -- 172.21.15.172:0/619437288 shutdown_connections 2024-09-10T09:07:28.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.527+0000 7ffb16b50640 1 -- 172.21.15.172:0/619437288 wait complete. 2024-09-10T09:07:28.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb16b50640 1 Processor -- start 2024-09-10T09:07:28.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb16b50640 1 -- start start 2024-09-10T09:07:28.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb16b50640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb10100a80 0x7ffb10198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:28.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb16b50640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10101420 0x7ffb10198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:28.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb16b50640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb10106a80 0x7ffb1019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:28.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb16b50640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffb10111e10 con 0x7ffb10100a80 2024-09-10T09:07:28.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb16b50640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb10111c90 con 0x7ffb10106a80 2024-09-10T09:07:28.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb16b50640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffb10111f90 con 0x7ffb10101420 2024-09-10T09:07:28.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb148ec640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb10100a80 0x7ffb10198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:28.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb148ec640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb10100a80 0x7ffb10198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:46176/0 (socket says 172.21.15.172:46176) 2024-09-10T09:07:28.548 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb148ec640 1 -- 172.21.15.172:0/97121367 learned_addr learned my addr 172.21.15.172:0/97121367 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-10T09:07:28.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb07fff640 1 --2- 172.21.15.172:0/97121367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10101420 0x7ffb10198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:28.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb150ed640 1 --2- 172.21.15.172:0/97121367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb10106a80 0x7ffb1019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:28.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb148ec640 1 -- 172.21.15.172:0/97121367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10101420 msgr2=0x7ffb10198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:28.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb148ec640 1 --2- 172.21.15.172:0/97121367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10101420 0x7ffb10198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:28.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb148ec640 1 -- 172.21.15.172:0/97121367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb10106a80 msgr2=0x7ffb1019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:28.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb148ec640 1 --2- 172.21.15.172:0/97121367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb10106a80 0x7ffb1019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:28.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb148ec640 1 -- 172.21.15.172:0/97121367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1019d2c0 con 0x7ffb10100a80 2024-09-10T09:07:28.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb07fff640 1 --2- 172.21.15.172:0/97121367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10101420 0x7ffb10198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:07:28.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.528+0000 7ffb148ec640 1 --2- 172.21.15.172:0/97121367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb10100a80 0x7ffb10198310 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c920 tx=0x7ffb0c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:28.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.529+0000 7ffb05ffb640 1 -- 172.21.15.172:0/97121367 <== mon.0 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c007c70 con 0x7ffb10100a80 2024-09-10T09:07:28.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.529+0000 7ffb05ffb640 1 -- 172.21.15.172:0/97121367 <== mon.0 v2:172.21.15.172:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb0c007e10 con 0x7ffb10100a80 2024-09-10T09:07:28.549 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.529+0000 7ffb16b50640 1 -- 172.21.15.172:0/97121367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1019d5b0 con 0x7ffb10100a80 2024-09-10T09:07:28.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.529+0000 7ffb05ffb640 1 -- 172.21.15.172:0/97121367 <== mon.0 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c018820 con 0x7ffb10100a80 2024-09-10T09:07:28.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.529+0000 7ffb16b50640 1 -- 172.21.15.172:0/97121367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb101a4e90 con 0x7ffb10100a80 2024-09-10T09:07:28.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.530+0000 7ffb16b50640 1 -- 172.21.15.172:0/97121367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb10100ee0 con 0x7ffb10100a80 2024-09-10T09:07:28.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.531+0000 7ffb05ffb640 1 -- 172.21.15.172:0/97121367 <== mon.0 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ffb0c0189c0 con 0x7ffb10100a80 2024-09-10T09:07:28.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.531+0000 7ffb05ffb640 1 --2- 172.21.15.172:0/97121367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffaf80a2b80 0x7ffaf80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:07:28.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.531+0000 7ffb07fff640 1 --2- 172.21.15.172:0/97121367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffaf80a2b80 0x7ffaf80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:07:28.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.532+0000 7ffb05ffb640 1 -- 172.21.15.172:0/97121367 <== mon.0 v2:172.21.15.172:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8496+0+0 (secure 0 0 0) 0x7ffb0c0cf300 con 0x7ffb10100a80 2024-09-10T09:07:28.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.532+0000 7ffb07fff640 1 --2- 172.21.15.172:0/97121367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffaf80a2b80 0x7ffaf80a5040 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ffb00000c00 tx=0x7ffb00031cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:07:28.550 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.535+0000 7ffb05ffb640 1 -- 172.21.15.172:0/97121367 <== mon.0 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb0c014030 con 0x7ffb10100a80 2024-09-10T09:07:28.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:28 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/766685247' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T09:07:28.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:28 smithi191 ceph-mon[35314]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T09:07:28.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:28.702+0000 7ffb16b50640 1 -- 172.21.15.172:0/97121367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7ffb1002d7c0 con 0x7ffb10100a80 2024-09-10T09:07:29.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:29.075+0000 7ffb05ffb640 1 -- 172.21.15.172:0/97121367 <== mon.0 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v96) ==== 178+0+0 (secure 0 0 0) 0x7ffb0c09c160 con 0x7ffb10100a80 2024-09-10T09:07:29.077 INFO:teuthology.orchestra.run.smithi172.stderr:pool 'cephfs_data_ec' removed 2024-09-10T09:07:29.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:29.078+0000 7ffb16b50640 1 -- 172.21.15.172:0/97121367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffaf80a2b80 msgr2=0x7ffaf80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:29.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:29.078+0000 7ffb16b50640 1 --2- 172.21.15.172:0/97121367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffaf80a2b80 0x7ffaf80a5040 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ffb00000c00 tx=0x7ffb00031cf0 comp rx=0 tx=0).stop 2024-09-10T09:07:29.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:29.079+0000 7ffb16b50640 1 -- 172.21.15.172:0/97121367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb10100a80 msgr2=0x7ffb10198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:07:29.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:29.079+0000 7ffb16b50640 1 --2- 172.21.15.172:0/97121367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb10100a80 0x7ffb10198310 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c920 tx=0x7ffb0c00cde0 comp rx=0 tx=0).stop 2024-09-10T09:07:29.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:29.079+0000 7ffb16b50640 1 -- 172.21.15.172:0/97121367 shutdown_connections 2024-09-10T09:07:29.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:29.079+0000 7ffb16b50640 1 --2- 172.21.15.172:0/97121367 >> [v2:172.21.15.172:6800/107199760,v1:172.21.15.172:6801/107199760] conn(0x7ffaf80a2b80 0x7ffaf80a5040 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:29.081 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:29.079+0000 7ffb16b50640 1 --2- 172.21.15.172:0/97121367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb10106a80 0x7ffb1019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:29.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:29.079+0000 7ffb16b50640 1 --2- 172.21.15.172:0/97121367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb10101420 0x7ffb10198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:29.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:29.079+0000 7ffb16b50640 1 --2- 172.21.15.172:0/97121367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb10100a80 0x7ffb10198310 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:07:29.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:29.079+0000 7ffb16b50640 1 -- 172.21.15.172:0/97121367 >> 172.21.15.172:0/97121367 conn(0x7ffb100fc7a0 msgr2=0x7ffb100fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:07:29.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:29.079+0000 7ffb16b50640 1 -- 172.21.15.172:0/97121367 shutdown_connections 2024-09-10T09:07:29.082 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-10T09:07:29.079+0000 7ffb16b50640 1 -- 172.21.15.172:0/97121367 wait complete. 2024-09-10T09:07:29.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:29 smithi172 ceph-mon[30854]: pgmap v990: 33 pgs: 33 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:07:29.332 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:29 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/97121367' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T09:07:29.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:29 smithi174 ceph-mon[34068]: pgmap v990: 33 pgs: 33 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:07:29.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:29 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/97121367' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T09:07:29.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:29 smithi191 ceph-mon[35314]: pgmap v990: 33 pgs: 33 active+clean; 577 KiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:07:29.551 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:29 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/97121367' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T09:07:29.822 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T09:07:29.858 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T09:07:30.117 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T09:07:30.161 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T09:07:30.161 DEBUG:teuthology.orchestra.run.smithi172:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T09:07:30.178 DEBUG:teuthology.orchestra.run.smithi174:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T09:07:30.210 DEBUG:teuthology.orchestra.run.smithi191:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T09:07:30.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:30 smithi174 ceph-mon[34068]: from='client.? 172.21.15.172:0/97121367' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T09:07:30.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:30 smithi174 ceph-mon[34068]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T09:07:30.248 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T09:07:30.249 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T09:07:30.249 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mon.a 2024-09-10T09:07:30.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:30 smithi191 ceph-mon[35314]: from='client.? 172.21.15.172:0/97121367' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T09:07:30.550 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:30 smithi191 ceph-mon[35314]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T09:07:30.565 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:30 smithi172 ceph-mon[30854]: from='client.? 172.21.15.172:0/97121367' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T09:07:30.565 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:30 smithi172 ceph-mon[30854]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T09:07:30.565 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:30 smithi172 systemd[1]: Stopping Ceph mon.a for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:07:30.832 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:30 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a[30850]: 2024-09-10T09:07:30.562+0000 7fa12b957640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:07:30.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:30 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a[30850]: 2024-09-10T09:07:30.562+0000 7fa12b957640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T09:07:30.833 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:30 smithi172 podman[138519]: 2024-09-10 09:07:30.808501609 +0000 UTC m=+0.367078927 container died 4ca591a9b8a003ee6f1e3edc34e471234001816eb1d1bd40e33699195fa18d1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T09:07:31.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:31 smithi172 podman[138519]: 2024-09-10 09:07:31.264663201 +0000 UTC m=+0.823240526 container cleanup 4ca591a9b8a003ee6f1e3edc34e471234001816eb1d1bd40e33699195fa18d1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T09:07:31.582 INFO:journalctl@ceph.mon.a.smithi172.stdout:Sep 10 09:07:31 smithi172 bash[138519]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-a 2024-09-10T09:07:31.843 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T09:07:31.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:07:31.898 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T09:07:31.898 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-10T09:07:31.898 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mon.b 2024-09-10T09:07:32.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:31 smithi174 systemd[1]: Stopping Ceph mon.b for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:07:32.714 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:32 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-b[34064]: 2024-09-10T09:07:32.336+0000 7fe3e5e75640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:07:32.714 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:32 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-b[34064]: 2024-09-10T09:07:32.336+0000 7fe3e5e75640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T09:07:32.715 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:32 smithi174 podman[69864]: 2024-09-10 09:07:32.388143785 +0000 UTC m=+0.179993980 container died 1732bb4b34821853f713868f7395977a6c69b292829fa44eb1ef692812610e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T09:07:32.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:32 smithi174 podman[69864]: 2024-09-10 09:07:32.812000947 +0000 UTC m=+0.603851143 container cleanup 1732bb4b34821853f713868f7395977a6c69b292829fa44eb1ef692812610e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-b, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:07:32.992 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:32 smithi174 bash[69864]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-b 2024-09-10T09:07:33.347 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:33 smithi174 podman[69876]: 2024-09-10 09:07:33.096972409 +0000 UTC m=+0.703000071 container remove 1732bb4b34821853f713868f7395977a6c69b292829fa44eb1ef692812610e1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-b, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T09:07:33.347 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:33 smithi174 systemd[1]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-10T09:07:33.347 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:33 smithi174 systemd[1]: Stopped Ceph mon.b for a7dba522-6f4f-11ef-bcea-c7b262605968. 2024-09-10T09:07:33.348 INFO:journalctl@ceph.mon.b.smithi174.stdout:Sep 10 09:07:33 smithi174 systemd[1]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mon.b.service: Consumed 16.945s CPU time. 2024-09-10T09:07:33.349 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T09:07:33.398 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:07:33.399 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-10T09:07:33.399 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-10T09:07:33.399 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mon.c 2024-09-10T09:07:33.748 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:33 smithi191 systemd[1]: Stopping Ceph mon.c for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:07:34.050 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:33 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-c[35310]: 2024-09-10T09:07:33.747+0000 7f4d5d9cb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:07:34.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:33 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-c[35310]: 2024-09-10T09:07:33.747+0000 7f4d5d9cb640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T09:07:34.051 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:33 smithi191 podman[72081]: 2024-09-10 09:07:33.778265818 +0000 UTC m=+0.159612096 container died 319a9d1e051643949f5c2563a131fbc546bc20a949d6b804deed429993f065a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T09:07:34.492 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:34 smithi191 podman[72081]: 2024-09-10 09:07:34.189709791 +0000 UTC m=+0.571056055 container cleanup 319a9d1e051643949f5c2563a131fbc546bc20a949d6b804deed429993f065a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T09:07:34.492 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:34 smithi191 bash[72081]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-c 2024-09-10T09:07:34.750 INFO:journalctl@ceph.mon.c.smithi191.stdout:Sep 10 09:07:34 smithi191 podman[72094]: 2024-09-10 09:07:34.492071253 +0000 UTC m=+0.713591113 container remove 319a9d1e051643949f5c2563a131fbc546bc20a949d6b804deed429993f065a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T09:07:34.766 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T09:07:34.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:07:34.817 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-10T09:07:34.817 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T09:07:34.817 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mgr.x 2024-09-10T09:07:35.129 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 09:07:34 smithi172 systemd[1]: Stopping Ceph mgr.x for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:07:35.582 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 09:07:35 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T09:07:35.126+0000 7f04c6095640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:07:35.583 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 09:07:35 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x[31079]: 2024-09-10T09:07:35.126+0000 7f04c6095640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-10T09:07:35.583 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 09:07:35 smithi172 podman[138727]: 2024-09-10 09:07:35.158906121 +0000 UTC m=+0.147316566 container died 381553d8692dd7997969ccd5686d607cbc12fd57bdc01b65d77572f57e130d63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T09:07:36.082 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 09:07:35 smithi172 podman[138727]: 2024-09-10 09:07:35.670551155 +0000 UTC m=+0.658961600 container cleanup 381553d8692dd7997969ccd5686d607cbc12fd57bdc01b65d77572f57e130d63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T09:07:36.083 INFO:journalctl@ceph.mgr.x.smithi172.stdout:Sep 10 09:07:35 smithi172 bash[138727]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-x 2024-09-10T09:07:36.487 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T09:07:36.522 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:07:36.523 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T09:07:36.523 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T09:07:36.523 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mgr.y 2024-09-10T09:07:36.863 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 09:07:36 smithi174 systemd[1]: Stopping Ceph mgr.y for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:07:37.168 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 09:07:36 smithi174 podman[69989]: 2024-09-10 09:07:36.862759793 +0000 UTC m=+0.141757385 container died 6ac7282b613dfc4170521b231a804e35c66bff364b7c0182b896f6552fe3b273 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T09:07:37.492 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 09:07:37 smithi174 podman[69989]: 2024-09-10 09:07:37.267590858 +0000 UTC m=+0.546588450 container cleanup 6ac7282b613dfc4170521b231a804e35c66bff364b7c0182b896f6552fe3b273 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True) 2024-09-10T09:07:37.492 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Sep 10 09:07:37 smithi174 bash[69989]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-y 2024-09-10T09:07:37.694 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T09:07:37.728 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:07:37.728 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T09:07:37.729 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-10T09:07:37.729 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mgr.z 2024-09-10T09:07:38.050 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 09:07:37 smithi191 systemd[1]: Stopping Ceph mgr.z for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:07:38.381 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 09:07:38 smithi191 podman[72201]: 2024-09-10 09:07:38.057938062 +0000 UTC m=+0.136406003 container died 26426940635a3dc2c870118c44331d80a4b4271727428f1ee61bc75123c987d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2) 2024-09-10T09:07:38.731 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 09:07:38 smithi191 podman[72201]: 2024-09-10 09:07:38.487502777 +0000 UTC m=+0.565970706 container cleanup 26426940635a3dc2c870118c44331d80a4b4271727428f1ee61bc75123c987d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T09:07:38.731 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Sep 10 09:07:38 smithi191 bash[72201]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mgr-z 2024-09-10T09:07:38.955 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-10T09:07:38.991 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:07:38.991 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-10T09:07:38.991 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T09:07:38.991 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.0 2024-09-10T09:07:39.285 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 09:07:39 smithi172 systemd[1]: Stopping Ceph osd.0 for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:07:39.582 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 09:07:39 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-0[42060]: 2024-09-10T09:07:39.283+0000 7fc94ded5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:07:39.582 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 09:07:39 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-0[42060]: 2024-09-10T09:07:39.283+0000 7fc94ded5640 -1 osd.0 96 *** Got signal Terminated *** 2024-09-10T09:07:39.582 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 09:07:39 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-0[42060]: 2024-09-10T09:07:39.283+0000 7fc94ded5640 -1 osd.0 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T09:07:44.988 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 09:07:44 smithi172 podman[138850]: 2024-09-10 09:07:44.737082101 +0000 UTC m=+5.556478994 container died 62944d5e6d010924f5997d26e89bf9c1216bbcc89386c04bad4d0a8da9011dfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T09:07:45.333 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 09:07:45 smithi172 podman[138850]: 2024-09-10 09:07:45.136075449 +0000 UTC m=+5.955472327 container cleanup 62944d5e6d010924f5997d26e89bf9c1216bbcc89386c04bad4d0a8da9011dfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T09:07:45.333 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 09:07:45 smithi172 bash[138850]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-0 2024-09-10T09:07:45.742 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 09:07:45 smithi172 podman[138869]: 2024-09-10 09:07:45.404763591 +0000 UTC m=+0.662889193 container remove 62944d5e6d010924f5997d26e89bf9c1216bbcc89386c04bad4d0a8da9011dfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-10T09:07:46.059 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 09:07:45 smithi172 podman[138958]: 2024-09-10 09:07:45.644239524 +0000 UTC m=+0.016028822 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:07:46.059 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 09:07:45 smithi172 podman[138958]: 2024-09-10 09:07:45.782016804 +0000 UTC m=+0.153806098 container create 76bb2132ff8beaa3e455e8a957bd34d9d82fac1c1ef14d65eff6eff0159a293b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T09:07:46.059 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 09:07:46 smithi172 podman[138958]: 2024-09-10 09:07:46.059012781 +0000 UTC m=+0.430802080 container init 76bb2132ff8beaa3e455e8a957bd34d9d82fac1c1ef14d65eff6eff0159a293b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:07:46.332 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 09:07:46 smithi172 podman[138958]: 2024-09-10 09:07:46.062456561 +0000 UTC m=+0.434245855 container start 76bb2132ff8beaa3e455e8a957bd34d9d82fac1c1ef14d65eff6eff0159a293b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T09:07:46.332 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 09:07:46 smithi172 podman[138958]: 2024-09-10 09:07:46.167326626 +0000 UTC m=+0.539115937 container attach 76bb2132ff8beaa3e455e8a957bd34d9d82fac1c1ef14d65eff6eff0159a293b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T09:07:46.690 INFO:journalctl@ceph.osd.0.smithi172.stdout:Sep 10 09:07:46 smithi172 podman[138958]: 2024-09-10 09:07:46.419490753 +0000 UTC m=+0.791280058 container died 76bb2132ff8beaa3e455e8a957bd34d9d82fac1c1ef14d65eff6eff0159a293b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T09:07:47.086 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T09:07:47.126 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:07:47.126 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T09:07:47.126 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T09:07:47.126 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.1 2024-09-10T09:07:47.242 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 09:07:47 smithi174 systemd[1]: Stopping Ceph osd.1 for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:07:47.992 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 09:07:47 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-1[38530]: 2024-09-10T09:07:47.505+0000 7f6de0f31640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:07:47.992 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 09:07:47 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-1[38530]: 2024-09-10T09:07:47.505+0000 7f6de0f31640 -1 osd.1 96 *** Got signal Terminated *** 2024-09-10T09:07:47.993 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 09:07:47 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-1[38530]: 2024-09-10T09:07:47.505+0000 7f6de0f31640 -1 osd.1 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T09:07:53.306 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 09:07:52 smithi174 podman[70129]: 2024-09-10 09:07:52.99367288 +0000 UTC m=+5.682291914 container died d9dc945be20dde16f76c5d6e428a5dd10cee35727428777a521dfff1ab222704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True) 2024-09-10T09:07:53.673 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 09:07:53 smithi174 podman[70129]: 2024-09-10 09:07:53.405042199 +0000 UTC m=+6.093661243 container cleanup d9dc945be20dde16f76c5d6e428a5dd10cee35727428777a521dfff1ab222704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T09:07:53.674 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 09:07:53 smithi174 bash[70129]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-1 2024-09-10T09:07:53.983 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 09:07:53 smithi174 podman[70144]: 2024-09-10 09:07:53.673487009 +0000 UTC m=+0.678421675 container remove d9dc945be20dde16f76c5d6e428a5dd10cee35727428777a521dfff1ab222704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-10T09:07:54.242 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 09:07:53 smithi174 podman[70231]: 2024-09-10 09:07:53.885369263 +0000 UTC m=+0.015800907 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:07:54.242 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 09:07:54 smithi174 podman[70231]: 2024-09-10 09:07:54.032816268 +0000 UTC m=+0.163247913 container create 780c6ed850553c68e9ed8bfd69a4880c5cadb72422caf309c5e4a731315a6a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T09:07:54.583 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 09:07:54 smithi174 podman[70231]: 2024-09-10 09:07:54.327434435 +0000 UTC m=+0.457866081 container init 780c6ed850553c68e9ed8bfd69a4880c5cadb72422caf309c5e4a731315a6a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True) 2024-09-10T09:07:54.584 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 09:07:54 smithi174 podman[70231]: 2024-09-10 09:07:54.330784832 +0000 UTC m=+0.461216486 container start 780c6ed850553c68e9ed8bfd69a4880c5cadb72422caf309c5e4a731315a6a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T09:07:54.584 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 09:07:54 smithi174 podman[70231]: 2024-09-10 09:07:54.427459244 +0000 UTC m=+0.557890898 container attach 780c6ed850553c68e9ed8bfd69a4880c5cadb72422caf309c5e4a731315a6a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T09:07:54.939 INFO:journalctl@ceph.osd.1.smithi174.stdout:Sep 10 09:07:54 smithi174 podman[70231]: 2024-09-10 09:07:54.583193953 +0000 UTC m=+0.713625607 container died 780c6ed850553c68e9ed8bfd69a4880c5cadb72422caf309c5e4a731315a6a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-10T09:07:55.266 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T09:07:55.306 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:07:55.306 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T09:07:55.307 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T09:07:55.307 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.2 2024-09-10T09:07:55.617 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 09:07:55 smithi191 systemd[1]: Stopping Ceph osd.2 for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:07:56.051 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 09:07:55 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-2[40272]: 2024-09-10T09:07:55.616+0000 7f32980dc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:07:56.051 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 09:07:55 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-2[40272]: 2024-09-10T09:07:55.616+0000 7f32980dc640 -1 osd.2 96 *** Got signal Terminated *** 2024-09-10T09:07:56.051 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 09:07:55 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-2[40272]: 2024-09-10T09:07:55.616+0000 7f32980dc640 -1 osd.2 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T09:08:01.402 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 09:08:01 smithi191 podman[72339]: 2024-09-10 09:08:01.124190094 +0000 UTC m=+5.621318241 container died 09fb1ee4b8c28ba0dc4eab7184106f116f70f857dca737c3e8ae9e5898afacf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-10T09:08:01.800 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 09:08:01 smithi191 podman[72339]: 2024-09-10 09:08:01.499909528 +0000 UTC m=+5.997037674 container cleanup 09fb1ee4b8c28ba0dc4eab7184106f116f70f857dca737c3e8ae9e5898afacf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-10T09:08:01.801 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 09:08:01 smithi191 bash[72339]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-2 2024-09-10T09:08:02.167 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 09:08:01 smithi191 podman[72354]: 2024-09-10 09:08:01.809898962 +0000 UTC m=+0.674982325 container remove 09fb1ee4b8c28ba0dc4eab7184106f116f70f857dca737c3e8ae9e5898afacf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-09-10T09:08:02.481 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 09:08:02 smithi191 podman[72445]: 2024-09-10 09:08:02.070058667 +0000 UTC m=+0.015759063 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:08:02.481 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 09:08:02 smithi191 podman[72445]: 2024-09-10 09:08:02.168962346 +0000 UTC m=+0.114662768 container create 213f37107e5a9015c4af67d51167497caf4c4c74cb14d0f40b749b563d1d2e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T09:08:02.801 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 09:08:02 smithi191 podman[72445]: 2024-09-10 09:08:02.480339734 +0000 UTC m=+0.426040114 container init 213f37107e5a9015c4af67d51167497caf4c4c74cb14d0f40b749b563d1d2e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T09:08:02.801 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 09:08:02 smithi191 podman[72445]: 2024-09-10 09:08:02.484267274 +0000 UTC m=+0.429967657 container start 213f37107e5a9015c4af67d51167497caf4c4c74cb14d0f40b749b563d1d2e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T09:08:02.801 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 09:08:02 smithi191 podman[72445]: 2024-09-10 09:08:02.572070023 +0000 UTC m=+0.517770422 container attach 213f37107e5a9015c4af67d51167497caf4c4c74cb14d0f40b749b563d1d2e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=) 2024-09-10T09:08:02.801 INFO:journalctl@ceph.osd.2.smithi191.stdout:Sep 10 09:08:02 smithi191 podman[72445]: 2024-09-10 09:08:02.721040079 +0000 UTC m=+0.666740468 container died 213f37107e5a9015c4af67d51167497caf4c4c74cb14d0f40b749b563d1d2e78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T09:08:03.511 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T09:08:03.553 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:08:03.553 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T09:08:03.553 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T09:08:03.553 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.3 2024-09-10T09:08:03.874 INFO:journalctl@ceph.osd.3.smithi172.stdout:Sep 10 09:08:03 smithi172 systemd[1]: Stopping Ceph osd.3 for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:08:04.332 INFO:journalctl@ceph.osd.3.smithi172.stdout:Sep 10 09:08:03 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-3[48485]: 2024-09-10T09:08:03.873+0000 7faeb6563640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:08:04.332 INFO:journalctl@ceph.osd.3.smithi172.stdout:Sep 10 09:08:03 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-3[48485]: 2024-09-10T09:08:03.873+0000 7faeb6563640 -1 osd.3 96 *** Got signal Terminated *** 2024-09-10T09:08:04.332 INFO:journalctl@ceph.osd.3.smithi172.stdout:Sep 10 09:08:03 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-3[48485]: 2024-09-10T09:08:03.873+0000 7faeb6563640 -1 osd.3 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T09:08:09.816 INFO:journalctl@ceph.osd.3.smithi172.stdout:Sep 10 09:08:09 smithi172 podman[139074]: 2024-09-10 09:08:09.520393553 +0000 UTC m=+5.764198588 container died e7784c24161317b1fd1ea7a422bd9cf67e5ea6b58ef8e5608b0be17c008b8090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T09:08:10.082 INFO:journalctl@ceph.osd.3.smithi172.stdout:Sep 10 09:08:10 smithi172 podman[139074]: 2024-09-10 09:08:10.007393522 +0000 UTC m=+6.251198557 container cleanup e7784c24161317b1fd1ea7a422bd9cf67e5ea6b58ef8e5608b0be17c008b8090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, ceph=True) 2024-09-10T09:08:10.082 INFO:journalctl@ceph.osd.3.smithi172.stdout:Sep 10 09:08:10 smithi172 bash[139074]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-3 2024-09-10T09:08:10.660 INFO:journalctl@ceph.osd.3.smithi172.stdout:Sep 10 09:08:10 smithi172 podman[139089]: 2024-09-10 09:08:10.349092923 +0000 UTC m=+0.828775473 container remove e7784c24161317b1fd1ea7a422bd9cf67e5ea6b58ef8e5608b0be17c008b8090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-3, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T09:08:10.963 INFO:journalctl@ceph.osd.3.smithi172.stdout:Sep 10 09:08:10 smithi172 podman[139175]: 2024-09-10 09:08:10.562197762 +0000 UTC m=+0.016168540 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:08:10.963 INFO:journalctl@ceph.osd.3.smithi172.stdout:Sep 10 09:08:10 smithi172 podman[139175]: 2024-09-10 09:08:10.685089897 +0000 UTC m=+0.139060667 container create d41103e5a8151d2dd6ca4ef4efd6fd0332c7e1287d3715c34a4c9bd2b1b3ada2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T09:08:11.333 INFO:journalctl@ceph.osd.3.smithi172.stdout:Sep 10 09:08:10 smithi172 podman[139175]: 2024-09-10 09:08:10.962797056 +0000 UTC m=+0.416767830 container init d41103e5a8151d2dd6ca4ef4efd6fd0332c7e1287d3715c34a4c9bd2b1b3ada2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:08:11.333 INFO:journalctl@ceph.osd.3.smithi172.stdout:Sep 10 09:08:10 smithi172 podman[139175]: 2024-09-10 09:08:10.966676913 +0000 UTC m=+0.420647685 container start d41103e5a8151d2dd6ca4ef4efd6fd0332c7e1287d3715c34a4c9bd2b1b3ada2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T09:08:11.333 INFO:journalctl@ceph.osd.3.smithi172.stdout:Sep 10 09:08:11 smithi172 podman[139175]: 2024-09-10 09:08:11.063063487 +0000 UTC m=+0.517034268 container attach d41103e5a8151d2dd6ca4ef4efd6fd0332c7e1287d3715c34a4c9bd2b1b3ada2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T09:08:11.333 INFO:journalctl@ceph.osd.3.smithi172.stdout:Sep 10 09:08:11 smithi172 podman[139175]: 2024-09-10 09:08:11.150387382 +0000 UTC m=+0.604358155 container died d41103e5a8151d2dd6ca4ef4efd6fd0332c7e1287d3715c34a4c9bd2b1b3ada2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True) 2024-09-10T09:08:11.780 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T09:08:11.819 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:08:11.820 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T09:08:11.820 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T09:08:11.820 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.4 2024-09-10T09:08:12.143 INFO:journalctl@ceph.osd.4.smithi174.stdout:Sep 10 09:08:11 smithi174 systemd[1]: Stopping Ceph osd.4 for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:08:12.492 INFO:journalctl@ceph.osd.4.smithi174.stdout:Sep 10 09:08:12 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-4[44962]: 2024-09-10T09:08:12.142+0000 7fd5bd4e0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:08:12.492 INFO:journalctl@ceph.osd.4.smithi174.stdout:Sep 10 09:08:12 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-4[44962]: 2024-09-10T09:08:12.142+0000 7fd5bd4e0640 -1 osd.4 96 *** Got signal Terminated *** 2024-09-10T09:08:12.492 INFO:journalctl@ceph.osd.4.smithi174.stdout:Sep 10 09:08:12 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-4[44962]: 2024-09-10T09:08:12.142+0000 7fd5bd4e0640 -1 osd.4 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T09:08:17.897 INFO:journalctl@ceph.osd.4.smithi174.stdout:Sep 10 09:08:17 smithi174 podman[70332]: 2024-09-10 09:08:17.619544849 +0000 UTC m=+5.597873213 container died aa043b0b04084cf9b9853f65100dafe75d8654f0baf6f3d4253cfda7284f0bc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T09:08:18.242 INFO:journalctl@ceph.osd.4.smithi174.stdout:Sep 10 09:08:18 smithi174 podman[70332]: 2024-09-10 09:08:18.016620086 +0000 UTC m=+5.994948440 container cleanup aa043b0b04084cf9b9853f65100dafe75d8654f0baf6f3d4253cfda7284f0bc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-4, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:08:18.242 INFO:journalctl@ceph.osd.4.smithi174.stdout:Sep 10 09:08:18 smithi174 bash[70332]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-4 2024-09-10T09:08:18.633 INFO:journalctl@ceph.osd.4.smithi174.stdout:Sep 10 09:08:18 smithi174 podman[70349]: 2024-09-10 09:08:18.301564693 +0000 UTC m=+0.680469280 container remove aa043b0b04084cf9b9853f65100dafe75d8654f0baf6f3d4253cfda7284f0bc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:08:18.897 INFO:journalctl@ceph.osd.4.smithi174.stdout:Sep 10 09:08:18 smithi174 podman[70438]: 2024-09-10 09:08:18.535943601 +0000 UTC m=+0.016124732 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:08:18.897 INFO:journalctl@ceph.osd.4.smithi174.stdout:Sep 10 09:08:18 smithi174 podman[70438]: 2024-09-10 09:08:18.661172532 +0000 UTC m=+0.141353662 container create 94c16a1e21c8d9ac32ba491e938c41ea7ad3ddf4657293bf4a5a59e1c0bd0204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T09:08:18.897 INFO:journalctl@ceph.osd.4.smithi174.stdout:Sep 10 09:08:18 smithi174 podman[70438]: 2024-09-10 09:08:18.896943355 +0000 UTC m=+0.377124487 container init 94c16a1e21c8d9ac32ba491e938c41ea7ad3ddf4657293bf4a5a59e1c0bd0204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-10T09:08:19.242 INFO:journalctl@ceph.osd.4.smithi174.stdout:Sep 10 09:08:18 smithi174 podman[70438]: 2024-09-10 09:08:18.900321232 +0000 UTC m=+0.380502362 container start 94c16a1e21c8d9ac32ba491e938c41ea7ad3ddf4657293bf4a5a59e1c0bd0204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:08:19.243 INFO:journalctl@ceph.osd.4.smithi174.stdout:Sep 10 09:08:18 smithi174 podman[70438]: 2024-09-10 09:08:18.980120434 +0000 UTC m=+0.460301581 container attach 94c16a1e21c8d9ac32ba491e938c41ea7ad3ddf4657293bf4a5a59e1c0bd0204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T09:08:19.243 INFO:journalctl@ceph.osd.4.smithi174.stdout:Sep 10 09:08:19 smithi174 podman[70438]: 2024-09-10 09:08:19.093387249 +0000 UTC m=+0.573568384 container died 94c16a1e21c8d9ac32ba491e938c41ea7ad3ddf4657293bf4a5a59e1c0bd0204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T09:08:19.755 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T09:08:19.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:08:19.799 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T09:08:19.799 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T09:08:19.799 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.5 2024-09-10T09:08:20.171 INFO:journalctl@ceph.osd.5.smithi191.stdout:Sep 10 09:08:19 smithi191 systemd[1]: Stopping Ceph osd.5 for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:08:20.485 INFO:journalctl@ceph.osd.5.smithi191.stdout:Sep 10 09:08:20 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-5[46711]: 2024-09-10T09:08:20.169+0000 7fbd8a230640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:08:20.485 INFO:journalctl@ceph.osd.5.smithi191.stdout:Sep 10 09:08:20 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-5[46711]: 2024-09-10T09:08:20.169+0000 7fbd8a230640 -1 osd.5 96 *** Got signal Terminated *** 2024-09-10T09:08:20.485 INFO:journalctl@ceph.osd.5.smithi191.stdout:Sep 10 09:08:20 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-5[46711]: 2024-09-10T09:08:20.169+0000 7fbd8a230640 -1 osd.5 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T09:08:26.170 INFO:journalctl@ceph.osd.5.smithi191.stdout:Sep 10 09:08:25 smithi191 podman[72547]: 2024-09-10 09:08:25.888420364 +0000 UTC m=+5.898042346 container died ae1af52f1859eed050935781b30aff5a1027d73b01e8f160f518745af2071a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T09:08:26.550 INFO:journalctl@ceph.osd.5.smithi191.stdout:Sep 10 09:08:26 smithi191 podman[72547]: 2024-09-10 09:08:26.277954991 +0000 UTC m=+6.287576973 container cleanup ae1af52f1859eed050935781b30aff5a1027d73b01e8f160f518745af2071a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:08:26.551 INFO:journalctl@ceph.osd.5.smithi191.stdout:Sep 10 09:08:26 smithi191 bash[72547]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-5 2024-09-10T09:08:26.881 INFO:journalctl@ceph.osd.5.smithi191.stdout:Sep 10 09:08:26 smithi191 podman[72575]: 2024-09-10 09:08:26.571040294 +0000 UTC m=+0.678012213 container remove ae1af52f1859eed050935781b30aff5a1027d73b01e8f160f518745af2071a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T09:08:27.133 INFO:journalctl@ceph.osd.5.smithi191.stdout:Sep 10 09:08:26 smithi191 podman[72662]: 2024-09-10 09:08:26.783734915 +0000 UTC m=+0.015883701 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:08:27.133 INFO:journalctl@ceph.osd.5.smithi191.stdout:Sep 10 09:08:26 smithi191 podman[72662]: 2024-09-10 09:08:26.88846231 +0000 UTC m=+0.120611138 container create b5ebd73aa311ec6fc27c4353efd1881717920c3d2f5578484afb6e405a444a61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-10T09:08:27.133 INFO:journalctl@ceph.osd.5.smithi191.stdout:Sep 10 09:08:27 smithi191 podman[72662]: 2024-09-10 09:08:27.132515931 +0000 UTC m=+0.364664717 container init b5ebd73aa311ec6fc27c4353efd1881717920c3d2f5578484afb6e405a444a61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T09:08:27.551 INFO:journalctl@ceph.osd.5.smithi191.stdout:Sep 10 09:08:27 smithi191 podman[72662]: 2024-09-10 09:08:27.135752203 +0000 UTC m=+0.367900994 container start b5ebd73aa311ec6fc27c4353efd1881717920c3d2f5578484afb6e405a444a61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=) 2024-09-10T09:08:27.551 INFO:journalctl@ceph.osd.5.smithi191.stdout:Sep 10 09:08:27 smithi191 podman[72662]: 2024-09-10 09:08:27.232744505 +0000 UTC m=+0.464893303 container attach b5ebd73aa311ec6fc27c4353efd1881717920c3d2f5578484afb6e405a444a61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-10T09:08:27.551 INFO:journalctl@ceph.osd.5.smithi191.stdout:Sep 10 09:08:27 smithi191 podman[72662]: 2024-09-10 09:08:27.337280724 +0000 UTC m=+0.569429517 container died b5ebd73aa311ec6fc27c4353efd1881717920c3d2f5578484afb6e405a444a61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T09:08:27.937 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T09:08:27.978 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:08:27.979 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T09:08:27.979 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T09:08:27.979 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.6 2024-09-10T09:08:28.300 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 10 09:08:28 smithi172 systemd[1]: Stopping Ceph osd.6 for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:08:28.583 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 10 09:08:28 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-6[55363]: 2024-09-10T09:08:28.310+0000 7f5a83946640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:08:28.583 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 10 09:08:28 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-6[55363]: 2024-09-10T09:08:28.310+0000 7f5a83946640 -1 osd.6 96 *** Got signal Terminated *** 2024-09-10T09:08:28.583 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 10 09:08:28 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-6[55363]: 2024-09-10T09:08:28.310+0000 7f5a83946640 -1 osd.6 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T09:08:34.137 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 10 09:08:33 smithi172 podman[139277]: 2024-09-10 09:08:33.845489841 +0000 UTC m=+5.674515022 container died e5723f647f88d24a47210f11a99ceb0ab3c2187afb43c147cf92fb69cfd47017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-6, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T09:08:34.582 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 10 09:08:34 smithi172 podman[139277]: 2024-09-10 09:08:34.318985695 +0000 UTC m=+6.148010878 container cleanup e5723f647f88d24a47210f11a99ceb0ab3c2187afb43c147cf92fb69cfd47017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-6, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T09:08:34.582 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 10 09:08:34 smithi172 bash[139277]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-6 2024-09-10T09:08:34.925 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 10 09:08:34 smithi172 podman[139303]: 2024-09-10 09:08:34.587079316 +0000 UTC m=+0.731923460 container remove e5723f647f88d24a47210f11a99ceb0ab3c2187afb43c147cf92fb69cfd47017 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:08:35.177 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 10 09:08:34 smithi172 podman[139391]: 2024-09-10 09:08:34.827722131 +0000 UTC m=+0.015696311 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:08:35.178 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 10 09:08:34 smithi172 podman[139391]: 2024-09-10 09:08:34.929701823 +0000 UTC m=+0.117676003 container create 7c1aa9bfb34cc10d3f3f2c96821bf8b65a2acde321eea82fecb898e3fb635877 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T09:08:35.178 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 10 09:08:35 smithi172 podman[139391]: 2024-09-10 09:08:35.17399214 +0000 UTC m=+0.361966322 container init 7c1aa9bfb34cc10d3f3f2c96821bf8b65a2acde321eea82fecb898e3fb635877 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:08:35.178 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 10 09:08:35 smithi172 podman[139391]: 2024-09-10 09:08:35.177435151 +0000 UTC m=+0.365409340 container start 7c1aa9bfb34cc10d3f3f2c96821bf8b65a2acde321eea82fecb898e3fb635877 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T09:08:35.582 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 10 09:08:35 smithi172 podman[139391]: 2024-09-10 09:08:35.290841102 +0000 UTC m=+0.478815299 container attach 7c1aa9bfb34cc10d3f3f2c96821bf8b65a2acde321eea82fecb898e3fb635877 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-10T09:08:35.583 INFO:journalctl@ceph.osd.6.smithi172.stdout:Sep 10 09:08:35 smithi172 podman[139391]: 2024-09-10 09:08:35.367336113 +0000 UTC m=+0.555310297 container died 7c1aa9bfb34cc10d3f3f2c96821bf8b65a2acde321eea82fecb898e3fb635877 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T09:08:36.057 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T09:08:36.101 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:08:36.101 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T09:08:36.101 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T09:08:36.101 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.7 2024-09-10T09:08:36.428 INFO:journalctl@ceph.osd.7.smithi174.stdout:Sep 10 09:08:36 smithi174 systemd[1]: Stopping Ceph osd.7 for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:08:36.742 INFO:journalctl@ceph.osd.7.smithi174.stdout:Sep 10 09:08:36 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-7[51746]: 2024-09-10T09:08:36.426+0000 7fc0f481f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:08:36.742 INFO:journalctl@ceph.osd.7.smithi174.stdout:Sep 10 09:08:36 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-7[51746]: 2024-09-10T09:08:36.426+0000 7fc0f481f640 -1 osd.7 96 *** Got signal Terminated *** 2024-09-10T09:08:36.742 INFO:journalctl@ceph.osd.7.smithi174.stdout:Sep 10 09:08:36 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-7[51746]: 2024-09-10T09:08:36.426+0000 7fc0f481f640 -1 osd.7 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T09:08:42.436 INFO:journalctl@ceph.osd.7.smithi174.stdout:Sep 10 09:08:42 smithi174 podman[70540]: 2024-09-10 09:08:42.097801247 +0000 UTC m=+5.794699617 container died 6dc02288f4876886adde374f93fad27b24f100ec21ca81c64684ac3fbfc18c0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:08:42.742 INFO:journalctl@ceph.osd.7.smithi174.stdout:Sep 10 09:08:42 smithi174 podman[70540]: 2024-09-10 09:08:42.543478978 +0000 UTC m=+6.240377347 container cleanup 6dc02288f4876886adde374f93fad27b24f100ec21ca81c64684ac3fbfc18c0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T09:08:42.742 INFO:journalctl@ceph.osd.7.smithi174.stdout:Sep 10 09:08:42 smithi174 bash[70540]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-7 2024-09-10T09:08:43.172 INFO:journalctl@ceph.osd.7.smithi174.stdout:Sep 10 09:08:42 smithi174 podman[70557]: 2024-09-10 09:08:42.86116345 +0000 UTC m=+0.761210538 container remove 6dc02288f4876886adde374f93fad27b24f100ec21ca81c64684ac3fbfc18c0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T09:08:43.472 INFO:journalctl@ceph.osd.7.smithi174.stdout:Sep 10 09:08:43 smithi174 podman[70645]: 2024-09-10 09:08:43.074168991 +0000 UTC m=+0.015624034 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:08:43.472 INFO:journalctl@ceph.osd.7.smithi174.stdout:Sep 10 09:08:43 smithi174 podman[70645]: 2024-09-10 09:08:43.195430431 +0000 UTC m=+0.136885465 container create 5bf21c6b33de1acaf4fbb584fcd60768cc9107c73a4e76b4e14f137f0f630a10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T09:08:43.472 INFO:journalctl@ceph.osd.7.smithi174.stdout:Sep 10 09:08:43 smithi174 podman[70645]: 2024-09-10 09:08:43.471852932 +0000 UTC m=+0.413307970 container init 5bf21c6b33de1acaf4fbb584fcd60768cc9107c73a4e76b4e14f137f0f630a10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T09:08:43.742 INFO:journalctl@ceph.osd.7.smithi174.stdout:Sep 10 09:08:43 smithi174 podman[70645]: 2024-09-10 09:08:43.475391989 +0000 UTC m=+0.416847025 container start 5bf21c6b33de1acaf4fbb584fcd60768cc9107c73a4e76b4e14f137f0f630a10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T09:08:43.742 INFO:journalctl@ceph.osd.7.smithi174.stdout:Sep 10 09:08:43 smithi174 podman[70645]: 2024-09-10 09:08:43.556222496 +0000 UTC m=+0.497677530 container attach 5bf21c6b33de1acaf4fbb584fcd60768cc9107c73a4e76b4e14f137f0f630a10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T09:08:43.742 INFO:journalctl@ceph.osd.7.smithi174.stdout:Sep 10 09:08:43 smithi174 podman[70645]: 2024-09-10 09:08:43.660414568 +0000 UTC m=+0.601869609 container died 5bf21c6b33de1acaf4fbb584fcd60768cc9107c73a4e76b4e14f137f0f630a10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T09:08:44.286 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T09:08:44.324 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:08:44.325 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T09:08:44.325 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-10T09:08:44.325 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.8 2024-09-10T09:08:44.739 INFO:journalctl@ceph.osd.8.smithi191.stdout:Sep 10 09:08:44 smithi191 systemd[1]: Stopping Ceph osd.8 for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:08:45.050 INFO:journalctl@ceph.osd.8.smithi191.stdout:Sep 10 09:08:44 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-8[53344]: 2024-09-10T09:08:44.737+0000 7f1d6a296640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:08:45.050 INFO:journalctl@ceph.osd.8.smithi191.stdout:Sep 10 09:08:44 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-8[53344]: 2024-09-10T09:08:44.737+0000 7f1d6a296640 -1 osd.8 96 *** Got signal Terminated *** 2024-09-10T09:08:45.050 INFO:journalctl@ceph.osd.8.smithi191.stdout:Sep 10 09:08:44 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-8[53344]: 2024-09-10T09:08:44.737+0000 7f1d6a296640 -1 osd.8 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T09:08:45.332 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:45 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:45.022+0000 7f550df04640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.624731+0000 front 2024-09-10T09:07:42.624880+0000 (oldest deadline 2024-09-10T09:08:44.924879+0000) 2024-09-10T09:08:46.242 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:45 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:45.822+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:08:46.332 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:46 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:46.040+0000 7f550df04640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.624731+0000 front 2024-09-10T09:07:42.624880+0000 (oldest deadline 2024-09-10T09:08:44.924879+0000) 2024-09-10T09:08:47.143 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:46 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:46.808+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:08:47.332 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:47 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:47.061+0000 7f550df04640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.624731+0000 front 2024-09-10T09:07:42.624880+0000 (oldest deadline 2024-09-10T09:08:44.924879+0000) 2024-09-10T09:08:48.051 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:08:47 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:08:47.752+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:08:48.242 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:47 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:47.773+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:08:48.582 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:48 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:48.096+0000 7f550df04640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.624731+0000 front 2024-09-10T09:07:42.624880+0000 (oldest deadline 2024-09-10T09:08:44.924879+0000) 2024-09-10T09:08:49.050 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:08:48 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:08:48.708+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:08:49.242 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:48 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:48.751+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:08:49.332 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:49 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:49.077+0000 7f550df04640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.624731+0000 front 2024-09-10T09:07:42.624880+0000 (oldest deadline 2024-09-10T09:08:44.924879+0000) 2024-09-10T09:08:49.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:49 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:49.726+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:08:50.050 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:08:49 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:08:49.705+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:08:50.530 INFO:journalctl@ceph.osd.8.smithi191.stdout:Sep 10 09:08:50 smithi191 podman[72766]: 2024-09-10 09:08:50.240761432 +0000 UTC m=+5.719296898 container died 00e7d3dc6041a9ee9f09431bd0314111f72c8554087c97dae8b5b80e87919d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True) 2024-09-10T09:08:50.582 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:50 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:50.084+0000 7f550df04640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.624731+0000 front 2024-09-10T09:07:42.624880+0000 (oldest deadline 2024-09-10T09:08:44.924879+0000) 2024-09-10T09:08:50.800 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:08:50 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:08:50.744+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:08:50.801 INFO:journalctl@ceph.osd.8.smithi191.stdout:Sep 10 09:08:50 smithi191 podman[72766]: 2024-09-10 09:08:50.604545671 +0000 UTC m=+6.083081137 container cleanup 00e7d3dc6041a9ee9f09431bd0314111f72c8554087c97dae8b5b80e87919d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T09:08:50.801 INFO:journalctl@ceph.osd.8.smithi191.stdout:Sep 10 09:08:50 smithi191 bash[72766]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-8 2024-09-10T09:08:51.242 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:50 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:50.741+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:08:51.272 INFO:journalctl@ceph.osd.8.smithi191.stdout:Sep 10 09:08:50 smithi191 podman[72782]: 2024-09-10 09:08:50.922695711 +0000 UTC m=+0.683651898 container remove 00e7d3dc6041a9ee9f09431bd0314111f72c8554087c97dae8b5b80e87919d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T09:08:51.550 INFO:journalctl@ceph.osd.8.smithi191.stdout:Sep 10 09:08:51 smithi191 podman[72874]: 2024-09-10 09:08:51.175268921 +0000 UTC m=+0.015802130 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:08:51.550 INFO:journalctl@ceph.osd.8.smithi191.stdout:Sep 10 09:08:51 smithi191 podman[72874]: 2024-09-10 09:08:51.298540345 +0000 UTC m=+0.139073545 container create 68b7bffc4f4898f477ef677c91bb5e22fc744e11bed13793fe06070cf5e20dc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-09-10T09:08:51.582 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:51 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:51.117+0000 7f550df04640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.624731+0000 front 2024-09-10T09:07:42.624880+0000 (oldest deadline 2024-09-10T09:08:44.924879+0000) 2024-09-10T09:08:52.050 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:08:51 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:08:51.765+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:08:52.051 INFO:journalctl@ceph.osd.8.smithi191.stdout:Sep 10 09:08:51 smithi191 podman[72874]: 2024-09-10 09:08:51.579448585 +0000 UTC m=+0.419981803 container init 68b7bffc4f4898f477ef677c91bb5e22fc744e11bed13793fe06070cf5e20dc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux ) 2024-09-10T09:08:52.051 INFO:journalctl@ceph.osd.8.smithi191.stdout:Sep 10 09:08:51 smithi191 podman[72874]: 2024-09-10 09:08:51.583099725 +0000 UTC m=+0.423632926 container start 68b7bffc4f4898f477ef677c91bb5e22fc744e11bed13793fe06070cf5e20dc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:08:52.051 INFO:journalctl@ceph.osd.8.smithi191.stdout:Sep 10 09:08:51 smithi191 podman[72874]: 2024-09-10 09:08:51.684420959 +0000 UTC m=+0.524954181 container attach 68b7bffc4f4898f477ef677c91bb5e22fc744e11bed13793fe06070cf5e20dc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-8-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T09:08:52.051 INFO:journalctl@ceph.osd.8.smithi191.stdout:Sep 10 09:08:51 smithi191 podman[72874]: 2024-09-10 09:08:51.785174721 +0000 UTC m=+0.625707936 container died 68b7bffc4f4898f477ef677c91bb5e22fc744e11bed13793fe06070cf5e20dc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:08:52.242 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:51 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:51.772+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:08:52.499 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.8.service' 2024-09-10T09:08:52.539 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:08:52.539 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-10T09:08:52.540 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-10T09:08:52.540 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.9 2024-09-10T09:08:52.566 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:52 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:52.110+0000 7f550df04640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.624731+0000 front 2024-09-10T09:07:42.624880+0000 (oldest deadline 2024-09-10T09:08:44.924879+0000) 2024-09-10T09:08:52.833 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:52 smithi172 systemd[1]: Stopping Ceph osd.9 for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:08:53.050 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:08:52 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:08:52.779+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:08:53.097 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:52 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:52.837+0000 7f55158f6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:08:53.098 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:52 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:52.837+0000 7f55158f6640 -1 osd.9 96 *** Got signal Terminated *** 2024-09-10T09:08:53.098 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:52 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:52.837+0000 7f55158f6640 -1 osd.9 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T09:08:53.098 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:53 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:53.096+0000 7f550df04640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.624731+0000 front 2024-09-10T09:07:42.624880+0000 (oldest deadline 2024-09-10T09:08:44.924879+0000) 2024-09-10T09:08:53.242 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:52 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:52.743+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:08:53.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:53 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:53.735+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:08:54.051 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:08:53 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:08:53.809+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:08:54.582 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:54 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:54.132+0000 7f550df04640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.624731+0000 front 2024-09-10T09:07:42.624880+0000 (oldest deadline 2024-09-10T09:08:44.924879+0000) 2024-09-10T09:08:54.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:54 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:54.708+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:08:55.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:08:54 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:08:54.806+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:08:55.582 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:55 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:55.164+0000 7f550df04640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.624731+0000 front 2024-09-10T09:07:42.624880+0000 (oldest deadline 2024-09-10T09:08:44.924879+0000) 2024-09-10T09:08:55.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:55 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:55.661+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:08:55.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:55 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:55.661+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2024-09-10T09:07:50.787588+0000 front 2024-09-10T09:07:50.787570+0000 (oldest deadline 2024-09-10T09:08:55.487692+0000) 2024-09-10T09:08:56.050 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:08:55 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:08:55.760+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:08:56.583 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:56.187+0000 7f550df04640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.624731+0000 front 2024-09-10T09:07:42.624880+0000 (oldest deadline 2024-09-10T09:08:44.924879+0000) 2024-09-10T09:08:56.583 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:56 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:56.187+0000 7f550df04640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2024-09-10T09:07:51.326141+0000 front 2024-09-10T09:07:51.326300+0000 (oldest deadline 2024-09-10T09:08:56.026127+0000) 2024-09-10T09:08:56.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:56 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:56.613+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:08:56.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:56 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:56.613+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2024-09-10T09:07:50.787588+0000 front 2024-09-10T09:07:50.787570+0000 (oldest deadline 2024-09-10T09:08:55.487692+0000) 2024-09-10T09:08:57.050 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:08:56 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:08:56.760+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:08:57.582 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:57 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:57.237+0000 7f550df04640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.624731+0000 front 2024-09-10T09:07:42.624880+0000 (oldest deadline 2024-09-10T09:08:44.924879+0000) 2024-09-10T09:08:57.582 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:57 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9[62349]: 2024-09-10T09:08:57.237+0000 7f550df04640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2024-09-10T09:07:51.326141+0000 front 2024-09-10T09:07:51.326300+0000 (oldest deadline 2024-09-10T09:08:56.026127+0000) 2024-09-10T09:08:57.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:57 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:57.660+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:08:57.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:57 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:57.660+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2024-09-10T09:07:50.787588+0000 front 2024-09-10T09:07:50.787570+0000 (oldest deadline 2024-09-10T09:08:55.487692+0000) 2024-09-10T09:08:58.050 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:08:57 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:08:57.776+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:08:58.730 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:58 smithi172 podman[139492]: 2024-09-10 09:08:58.452471799 +0000 UTC m=+5.720243946 container died abb43fcacbacf0c97e29fc4c89a288df57b703f0d08e1cb01a3bcb2c0f5917f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0) 2024-09-10T09:08:58.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:58 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:58.639+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:08:58.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:58 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:58.639+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2024-09-10T09:07:50.787588+0000 front 2024-09-10T09:07:50.787570+0000 (oldest deadline 2024-09-10T09:08:55.487692+0000) 2024-09-10T09:08:59.050 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:08:58 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:08:58.783+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:08:59.082 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:58 smithi172 podman[139492]: 2024-09-10 09:08:58.904171503 +0000 UTC m=+6.171943649 container cleanup abb43fcacbacf0c97e29fc4c89a288df57b703f0d08e1cb01a3bcb2c0f5917f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:08:59.082 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:58 smithi172 bash[139492]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9 2024-09-10T09:08:59.582 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:59 smithi172 podman[139509]: 2024-09-10 09:08:59.247723722 +0000 UTC m=+0.791969008 container remove abb43fcacbacf0c97e29fc4c89a288df57b703f0d08e1cb01a3bcb2c0f5917f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-10T09:08:59.860 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:59 smithi172 podman[139606]: 2024-09-10 09:08:59.50785234 +0000 UTC m=+0.016191021 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:08:59.860 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:59 smithi172 podman[139606]: 2024-09-10 09:08:59.606883197 +0000 UTC m=+0.115221873 container create f026bae1af7be556ba770b952aaae92473d9897d01acc7ef2ca7df5ec322e46a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:08:59.860 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:59 smithi172 podman[139606]: 2024-09-10 09:08:59.859406943 +0000 UTC m=+0.367745618 container init f026bae1af7be556ba770b952aaae92473d9897d01acc7ef2ca7df5ec322e46a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T09:08:59.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:59 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:59.609+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:08:59.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:08:59 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:08:59.609+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2024-09-10T09:07:50.787588+0000 front 2024-09-10T09:07:50.787570+0000 (oldest deadline 2024-09-10T09:08:55.487692+0000) 2024-09-10T09:09:00.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:08:59 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:08:59.826+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:09:00.332 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:59 smithi172 podman[139606]: 2024-09-10 09:08:59.8630701 +0000 UTC m=+0.371408772 container start f026bae1af7be556ba770b952aaae92473d9897d01acc7ef2ca7df5ec322e46a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T09:09:00.333 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:08:59 smithi172 podman[139606]: 2024-09-10 09:08:59.951048059 +0000 UTC m=+0.459386733 container attach f026bae1af7be556ba770b952aaae92473d9897d01acc7ef2ca7df5ec322e46a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-10T09:09:00.333 INFO:journalctl@ceph.osd.9.smithi172.stdout:Sep 10 09:09:00 smithi172 podman[139606]: 2024-09-10 09:09:00.059860689 +0000 UTC m=+0.568199366 container died f026bae1af7be556ba770b952aaae92473d9897d01acc7ef2ca7df5ec322e46a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-9-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T09:09:00.726 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.9.service' 2024-09-10T09:09:00.807 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:09:00.807 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-10T09:09:00.807 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-10T09:09:00.807 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.10 2024-09-10T09:09:00.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:00 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:00.646+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:09:00.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:00 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:00.646+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2024-09-10T09:07:50.787588+0000 front 2024-09-10T09:07:50.787570+0000 (oldest deadline 2024-09-10T09:08:55.487692+0000) 2024-09-10T09:09:00.993 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:00 smithi174 systemd[1]: Stopping Ceph osd.10 for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:09:01.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:00 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:00.855+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:09:01.492 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:01 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:01.103+0000 7f983a639640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:09:01.493 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:01 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:01.103+0000 7f983a639640 -1 osd.10 96 *** Got signal Terminated *** 2024-09-10T09:09:01.493 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:01 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:01.103+0000 7f983a639640 -1 osd.10 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T09:09:01.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:01 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:01.643+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:09:01.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:01 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:01.643+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2024-09-10T09:07:50.787588+0000 front 2024-09-10T09:07:50.787570+0000 (oldest deadline 2024-09-10T09:08:55.487692+0000) 2024-09-10T09:09:02.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:01 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:01.879+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:09:02.301 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:01 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:01.879+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-09-10T09:07:55.679164+0000 front 2024-09-10T09:07:55.679135+0000 (oldest deadline 2024-09-10T09:09:01.579232+0000) 2024-09-10T09:09:02.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:02 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:02.661+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:09:02.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:02 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:02.661+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2024-09-10T09:07:50.787588+0000 front 2024-09-10T09:07:50.787570+0000 (oldest deadline 2024-09-10T09:08:55.487692+0000) 2024-09-10T09:09:02.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:02 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:02.661+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-09-10T09:07:59.588819+0000 front 2024-09-10T09:07:59.588853+0000 (oldest deadline 2024-09-10T09:09:02.488887+0000) 2024-09-10T09:09:03.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:02 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:02.868+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:09:03.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:02 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:02.868+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-09-10T09:07:55.679164+0000 front 2024-09-10T09:07:55.679135+0000 (oldest deadline 2024-09-10T09:09:01.579232+0000) 2024-09-10T09:09:03.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:03 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:03.680+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:09:03.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:03 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:03.680+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2024-09-10T09:07:50.787588+0000 front 2024-09-10T09:07:50.787570+0000 (oldest deadline 2024-09-10T09:08:55.487692+0000) 2024-09-10T09:09:03.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:03 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:03.680+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-09-10T09:07:59.588819+0000 front 2024-09-10T09:07:59.588853+0000 (oldest deadline 2024-09-10T09:09:02.488887+0000) 2024-09-10T09:09:04.050 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:03 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:03.820+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:09:04.050 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:03 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:03.820+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-09-10T09:07:55.679164+0000 front 2024-09-10T09:07:55.679135+0000 (oldest deadline 2024-09-10T09:09:01.579232+0000) 2024-09-10T09:09:04.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:04 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:04.685+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:09:04.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:04 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:04.685+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2024-09-10T09:07:50.787588+0000 front 2024-09-10T09:07:50.787570+0000 (oldest deadline 2024-09-10T09:08:55.487692+0000) 2024-09-10T09:09:04.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:04 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:04.685+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-09-10T09:07:59.588819+0000 front 2024-09-10T09:07:59.588853+0000 (oldest deadline 2024-09-10T09:09:02.488887+0000) 2024-09-10T09:09:05.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:04 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:04.865+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:09:05.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:04 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:04.865+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-09-10T09:07:55.679164+0000 front 2024-09-10T09:07:55.679135+0000 (oldest deadline 2024-09-10T09:09:01.579232+0000) 2024-09-10T09:09:05.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:05 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:05.681+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:39.586634+0000 front 2024-09-10T09:07:39.586469+0000 (oldest deadline 2024-09-10T09:08:44.886671+0000) 2024-09-10T09:09:05.992 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:05 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:05.681+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2024-09-10T09:07:50.787588+0000 front 2024-09-10T09:07:50.787570+0000 (oldest deadline 2024-09-10T09:08:55.487692+0000) 2024-09-10T09:09:05.993 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:05 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10[58703]: 2024-09-10T09:09:05.681+0000 7f9836c4f640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-09-10T09:07:59.588819+0000 front 2024-09-10T09:07:59.588853+0000 (oldest deadline 2024-09-10T09:09:02.488887+0000) 2024-09-10T09:09:06.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:05 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:05.829+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:09:06.301 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:05 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:05.829+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-09-10T09:07:55.679164+0000 front 2024-09-10T09:07:55.679135+0000 (oldest deadline 2024-09-10T09:09:01.579232+0000) 2024-09-10T09:09:06.862 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:06 smithi174 podman[70759]: 2024-09-10 09:09:06.588304483 +0000 UTC m=+5.589170152 container died 3b7183de11ddeb5e8fcacb31c726e978fff433c2946fe96e44d31afdcfef0a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T09:09:07.242 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:06 smithi174 podman[70759]: 2024-09-10 09:09:06.969066338 +0000 UTC m=+5.969932001 container cleanup 3b7183de11ddeb5e8fcacb31c726e978fff433c2946fe96e44d31afdcfef0a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:09:07.242 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:06 smithi174 bash[70759]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10 2024-09-10T09:09:07.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:06 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:06.817+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:09:07.301 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:06 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:06.817+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-09-10T09:07:55.679164+0000 front 2024-09-10T09:07:55.679135+0000 (oldest deadline 2024-09-10T09:09:01.579232+0000) 2024-09-10T09:09:07.571 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:07 smithi174 podman[70774]: 2024-09-10 09:09:07.262330595 +0000 UTC m=+0.671978825 container remove 3b7183de11ddeb5e8fcacb31c726e978fff433c2946fe96e44d31afdcfef0a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:09:07.882 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:07 smithi174 podman[70865]: 2024-09-10 09:09:07.473801932 +0000 UTC m=+0.015721885 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:09:07.882 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:07 smithi174 podman[70865]: 2024-09-10 09:09:07.579636737 +0000 UTC m=+0.121556684 container create 8e8e383e8e54162638305a9a1145b908717b191aa56174942cd9cfec82ee2feb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T09:09:07.882 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:07 smithi174 podman[70865]: 2024-09-10 09:09:07.806994318 +0000 UTC m=+0.348914266 container init 8e8e383e8e54162638305a9a1145b908717b191aa56174942cd9cfec82ee2feb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:09:07.883 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:07 smithi174 podman[70865]: 2024-09-10 09:09:07.810458921 +0000 UTC m=+0.352378873 container start 8e8e383e8e54162638305a9a1145b908717b191aa56174942cd9cfec82ee2feb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T09:09:07.883 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:07 smithi174 podman[70865]: 2024-09-10 09:09:07.881917597 +0000 UTC m=+0.423837556 container attach 8e8e383e8e54162638305a9a1145b908717b191aa56174942cd9cfec82ee2feb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T09:09:08.242 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:07 smithi174 podman[70865]: 2024-09-10 09:09:07.995167424 +0000 UTC m=+0.537087380 container died 8e8e383e8e54162638305a9a1145b908717b191aa56174942cd9cfec82ee2feb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T09:09:08.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:07 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:07.827+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:09:08.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:07 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:07.827+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-09-10T09:07:55.679164+0000 front 2024-09-10T09:07:55.679135+0000 (oldest deadline 2024-09-10T09:09:01.579232+0000) 2024-09-10T09:09:08.541 INFO:journalctl@ceph.osd.10.smithi174.stdout:Sep 10 09:09:08 smithi174 podman[70888]: 2024-09-10 09:09:08.518306613 +0000 UTC m=+0.516694485 container remove 8e8e383e8e54162638305a9a1145b908717b191aa56174942cd9cfec82ee2feb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T09:09:08.547 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.10.service' 2024-09-10T09:09:08.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:09:08.587 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-10T09:09:08.587 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-10T09:09:08.587 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.11 2024-09-10T09:09:08.870 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:08 smithi191 systemd[1]: Stopping Ceph osd.11 for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:09:09.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:08 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:08.869+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:09:09.301 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:08 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:08.869+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-09-10T09:07:55.679164+0000 front 2024-09-10T09:07:55.679135+0000 (oldest deadline 2024-09-10T09:09:01.579232+0000) 2024-09-10T09:09:09.301 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:08 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:08.906+0000 7f2493def640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:09:09.301 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:08 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:08.906+0000 7f2493def640 -1 osd.11 96 *** Got signal Terminated *** 2024-09-10T09:09:09.301 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:08 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:08.906+0000 7f2493def640 -1 osd.11 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T09:09:10.301 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:09 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:09.882+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:09:10.301 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:09 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:09.882+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-09-10T09:07:55.679164+0000 front 2024-09-10T09:07:55.679135+0000 (oldest deadline 2024-09-10T09:09:01.579232+0000) 2024-09-10T09:09:11.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:10 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:10.906+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:09:11.301 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:10 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:10.906+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-09-10T09:07:55.679164+0000 front 2024-09-10T09:07:55.679135+0000 (oldest deadline 2024-09-10T09:09:01.579232+0000) 2024-09-10T09:09:12.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:11 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:11.874+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:09:12.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:11 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:11.874+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-09-10T09:07:55.679164+0000 front 2024-09-10T09:07:55.679135+0000 (oldest deadline 2024-09-10T09:09:01.579232+0000) 2024-09-10T09:09:13.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:12 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:12.877+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.172:6806 osd.0 since back 2024-09-10T09:07:42.976189+0000 front 2024-09-10T09:07:42.976271+0000 (oldest deadline 2024-09-10T09:08:47.076324+0000) 2024-09-10T09:09:13.301 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:12 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11[59879]: 2024-09-10T09:09:12.877+0000 7f248c3fd640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-09-10T09:07:55.679164+0000 front 2024-09-10T09:07:55.679135+0000 (oldest deadline 2024-09-10T09:09:01.579232+0000) 2024-09-10T09:09:14.624 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:14 smithi191 podman[72989]: 2024-09-10 09:09:14.354732587 +0000 UTC m=+5.572790113 container died f3d58ec1b6df781561132d4eeb77492afead7c3d76ca8486cee0c13104e37348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T09:09:14.999 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:14 smithi191 podman[72989]: 2024-09-10 09:09:14.722524019 +0000 UTC m=+5.940581553 container cleanup f3d58ec1b6df781561132d4eeb77492afead7c3d76ca8486cee0c13104e37348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T09:09:15.000 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:14 smithi191 bash[72989]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11 2024-09-10T09:09:15.300 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:14 smithi191 podman[73005]: 2024-09-10 09:09:14.998839936 +0000 UTC m=+0.638641157 container remove f3d58ec1b6df781561132d4eeb77492afead7c3d76ca8486cee0c13104e37348 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T09:09:15.594 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:15 smithi191 podman[73094]: 2024-09-10 09:09:15.238132876 +0000 UTC m=+0.015923504 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:09:15.594 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:15 smithi191 podman[73094]: 2024-09-10 09:09:15.341099666 +0000 UTC m=+0.118890293 container create 2242d600528155aa70bc9ab4e1e058ec83f78adbf5fa13428124c52ff1b80c71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:09:16.051 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:15 smithi191 podman[73094]: 2024-09-10 09:09:15.593742971 +0000 UTC m=+0.371533604 container init 2242d600528155aa70bc9ab4e1e058ec83f78adbf5fa13428124c52ff1b80c71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T09:09:16.051 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:15 smithi191 podman[73094]: 2024-09-10 09:09:15.597092956 +0000 UTC m=+0.374883583 container start 2242d600528155aa70bc9ab4e1e058ec83f78adbf5fa13428124c52ff1b80c71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T09:09:16.051 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:15 smithi191 podman[73094]: 2024-09-10 09:09:15.677104584 +0000 UTC m=+0.454895211 container attach 2242d600528155aa70bc9ab4e1e058ec83f78adbf5fa13428124c52ff1b80c71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T09:09:16.051 INFO:journalctl@ceph.osd.11.smithi191.stdout:Sep 10 09:09:15 smithi191 podman[73094]: 2024-09-10 09:09:15.781880896 +0000 UTC m=+0.559671527 container died 2242d600528155aa70bc9ab4e1e058ec83f78adbf5fa13428124c52ff1b80c71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T09:09:16.357 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@osd.11.service' 2024-09-10T09:09:16.397 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:09:16.397 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-10T09:09:16.398 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-10T09:09:16.398 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.a 2024-09-10T09:09:16.690 INFO:journalctl@ceph.mds.a.smithi172.stdout:Sep 10 09:09:16 smithi172 systemd[1]: Stopping Ceph mds.a for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:09:17.082 INFO:journalctl@ceph.mds.a.smithi172.stdout:Sep 10 09:09:16 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-a[69385]: 2024-09-10T09:09:16.689+0000 7f61f3b61640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:09:17.083 INFO:journalctl@ceph.mds.a.smithi172.stdout:Sep 10 09:09:16 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-a[69385]: 2024-09-10T09:09:16.689+0000 7f61f3b61640 -1 mds.a *** got signal Terminated *** 2024-09-10T09:09:20.989 INFO:journalctl@ceph.mds.a.smithi172.stdout:Sep 10 09:09:20 smithi172 podman[139708]: 2024-09-10 09:09:20.706581202 +0000 UTC m=+4.117646306 container died d653c3296f850005cca0c701f4d87ad1f6d2c3e829a863335e9a90c28782df69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-a, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T09:09:21.332 INFO:journalctl@ceph.mds.a.smithi172.stdout:Sep 10 09:09:21 smithi172 podman[139708]: 2024-09-10 09:09:21.180654749 +0000 UTC m=+4.591719849 container cleanup d653c3296f850005cca0c701f4d87ad1f6d2c3e829a863335e9a90c28782df69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T09:09:21.332 INFO:journalctl@ceph.mds.a.smithi172.stdout:Sep 10 09:09:21 smithi172 bash[139708]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-a 2024-09-10T09:09:21.563 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.a.service' 2024-09-10T09:09:21.599 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:09:21.599 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-10T09:09:21.599 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-10T09:09:21.599 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.d 2024-09-10T09:09:22.083 INFO:journalctl@ceph.mds.d.smithi172.stdout:Sep 10 09:09:21 smithi172 systemd[1]: Stopping Ceph mds.d for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:09:22.083 INFO:journalctl@ceph.mds.d.smithi172.stdout:Sep 10 09:09:21 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-d[68171]: 2024-09-10T09:09:21.918+0000 7fe94191d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:09:22.083 INFO:journalctl@ceph.mds.d.smithi172.stdout:Sep 10 09:09:21 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-d[68171]: 2024-09-10T09:09:21.918+0000 7fe94191d640 -1 mds.d *** got signal Terminated *** 2024-09-10T09:09:23.662 INFO:journalctl@ceph.mds.d.smithi172.stdout:Sep 10 09:09:23 smithi172 podman[139827]: 2024-09-10 09:09:23.353092826 +0000 UTC m=+1.537114727 container died 50cb24e942a546d1f0022ad9aae7418514c52c0f983d04166b736bf6eafb7024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T09:09:24.003 INFO:journalctl@ceph.mds.d.smithi172.stdout:Sep 10 09:09:23 smithi172 podman[139827]: 2024-09-10 09:09:23.760208833 +0000 UTC m=+1.944230738 container cleanup 50cb24e942a546d1f0022ad9aae7418514c52c0f983d04166b736bf6eafb7024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T09:09:24.004 INFO:journalctl@ceph.mds.d.smithi172.stdout:Sep 10 09:09:23 smithi172 bash[139827]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-d 2024-09-10T09:09:24.127 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.d.service' 2024-09-10T09:09:24.160 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:09:24.160 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-10T09:09:24.160 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-10T09:09:24.161 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.g 2024-09-10T09:09:24.332 INFO:journalctl@ceph.mds.g.smithi172.stdout:Sep 10 09:09:24 smithi172 systemd[1]: Stopping Ceph mds.g for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:09:24.832 INFO:journalctl@ceph.mds.g.smithi172.stdout:Sep 10 09:09:24 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-g[69048]: 2024-09-10T09:09:24.497+0000 7f098ef6b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:09:24.833 INFO:journalctl@ceph.mds.g.smithi172.stdout:Sep 10 09:09:24 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-g[69048]: 2024-09-10T09:09:24.497+0000 7f098ef6b640 -1 mds.g *** got signal Terminated *** 2024-09-10T09:09:28.765 INFO:journalctl@ceph.mds.g.smithi172.stdout:Sep 10 09:09:28 smithi172 podman[139949]: 2024-09-10 09:09:28.504875465 +0000 UTC m=+4.131042981 container died c1224c865ee01c925fd4192e10b44db0458feb9949327a5af765b6b337e7d817 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-10T09:09:29.082 INFO:journalctl@ceph.mds.g.smithi172.stdout:Sep 10 09:09:28 smithi172 podman[139949]: 2024-09-10 09:09:28.855264712 +0000 UTC m=+4.481432227 container cleanup c1224c865ee01c925fd4192e10b44db0458feb9949327a5af765b6b337e7d817 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-g, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T09:09:29.083 INFO:journalctl@ceph.mds.g.smithi172.stdout:Sep 10 09:09:28 smithi172 bash[139949]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-g 2024-09-10T09:09:29.540 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.g.service' 2024-09-10T09:09:29.573 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:09:29.573 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-10T09:09:29.573 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-10T09:09:29.573 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.j 2024-09-10T09:09:29.918 INFO:journalctl@ceph.mds.j.smithi172.stdout:Sep 10 09:09:29 smithi172 systemd[1]: Stopping Ceph mds.j for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:09:30.332 INFO:journalctl@ceph.mds.j.smithi172.stdout:Sep 10 09:09:29 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-j[67774]: 2024-09-10T09:09:29.916+0000 7f796d814640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:09:30.333 INFO:journalctl@ceph.mds.j.smithi172.stdout:Sep 10 09:09:29 smithi172 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-j[67774]: 2024-09-10T09:09:29.916+0000 7f796d814640 -1 mds.j *** got signal Terminated *** 2024-09-10T09:09:31.997 INFO:journalctl@ceph.mds.j.smithi172.stdout:Sep 10 09:09:31 smithi172 podman[140071]: 2024-09-10 09:09:31.681312874 +0000 UTC m=+1.882580948 container died 1278041f7ff316f6ef968d6f4507a932bb7dbfa946b20c378181f827ae99fef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-j, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T09:09:32.332 INFO:journalctl@ceph.mds.j.smithi172.stdout:Sep 10 09:09:32 smithi172 podman[140071]: 2024-09-10 09:09:32.185402116 +0000 UTC m=+2.386670193 container cleanup 1278041f7ff316f6ef968d6f4507a932bb7dbfa946b20c378181f827ae99fef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-10T09:09:32.333 INFO:journalctl@ceph.mds.j.smithi172.stdout:Sep 10 09:09:32 smithi172 bash[140071]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-j 2024-09-10T09:09:32.553 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.j.service' 2024-09-10T09:09:32.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:09:32.588 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-10T09:09:32.589 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-10T09:09:32.589 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.b 2024-09-10T09:09:32.894 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 09:09:32 smithi174 systemd[1]: Stopping Ceph mds.b for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:09:33.242 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 09:09:32 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-b[63897]: 2024-09-10T09:09:32.893+0000 7f9eaf56a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:09:33.242 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 09:09:32 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-b[63897]: 2024-09-10T09:09:32.893+0000 7f9eaf56a640 -1 mds.b *** got signal Terminated *** 2024-09-10T09:09:37.242 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 09:09:36 smithi174 podman[70966]: 2024-09-10 09:09:36.900067029 +0000 UTC m=+4.115950040 container died 455496ffd6ee0a4e7c4e04eb9f0238088dae30678cebe2098b501f4e29b52c46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T09:09:37.661 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 09:09:37 smithi174 podman[70966]: 2024-09-10 09:09:37.392979981 +0000 UTC m=+4.608862991 container cleanup 455496ffd6ee0a4e7c4e04eb9f0238088dae30678cebe2098b501f4e29b52c46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-09-10T09:09:37.661 INFO:journalctl@ceph.mds.b.smithi174.stdout:Sep 10 09:09:37 smithi174 bash[70966]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-b 2024-09-10T09:09:37.784 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.b.service' 2024-09-10T09:09:37.821 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:09:37.821 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-10T09:09:37.822 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-10T09:09:37.822 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.e 2024-09-10T09:09:37.992 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 09:09:37 smithi174 systemd[1]: Stopping Ceph mds.e for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:09:38.492 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 09:09:38 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-e[63283]: 2024-09-10T09:09:38.146+0000 7fee9348c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:09:38.492 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 09:09:38 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-e[63283]: 2024-09-10T09:09:38.146+0000 7fee9348c640 -1 mds.e *** got signal Terminated *** 2024-09-10T09:09:41.398 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 09:09:41 smithi174 podman[71099]: 2024-09-10 09:09:41.119039841 +0000 UTC m=+3.077369424 container died 0214c70d606fc6cbad0550b07b280a718b9a92f118ccecd09451a12406d51246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-e, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T09:09:41.689 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 09:09:41 smithi174 podman[71099]: 2024-09-10 09:09:41.479690052 +0000 UTC m=+3.438019637 container cleanup 0214c70d606fc6cbad0550b07b280a718b9a92f118ccecd09451a12406d51246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-e, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T09:09:41.689 INFO:journalctl@ceph.mds.e.smithi174.stdout:Sep 10 09:09:41 smithi174 bash[71099]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-e 2024-09-10T09:09:41.911 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.e.service' 2024-09-10T09:09:41.944 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:09:41.945 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-10T09:09:41.945 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-10T09:09:41.945 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.h 2024-09-10T09:09:42.242 INFO:journalctl@ceph.mds.h.smithi174.stdout:Sep 10 09:09:42 smithi174 systemd[1]: Stopping Ceph mds.h for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:09:42.742 INFO:journalctl@ceph.mds.h.smithi174.stdout:Sep 10 09:09:42 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-h[63584]: 2024-09-10T09:09:42.400+0000 7fd750927640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:09:42.742 INFO:journalctl@ceph.mds.h.smithi174.stdout:Sep 10 09:09:42 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-h[63584]: 2024-09-10T09:09:42.400+0000 7fd750927640 -1 mds.h *** got signal Terminated *** 2024-09-10T09:09:45.443 INFO:journalctl@ceph.mds.h.smithi174.stdout:Sep 10 09:09:45 smithi174 podman[71220]: 2024-09-10 09:09:45.113082112 +0000 UTC m=+2.953461351 container died 809cc2da9db78ac8eed0b8a867389a3a37aa2dab470ac3dca575a0e55e6ec5f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T09:09:45.742 INFO:journalctl@ceph.mds.h.smithi174.stdout:Sep 10 09:09:45 smithi174 podman[71220]: 2024-09-10 09:09:45.541806773 +0000 UTC m=+3.382186020 container cleanup 809cc2da9db78ac8eed0b8a867389a3a37aa2dab470ac3dca575a0e55e6ec5f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-h, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T09:09:45.742 INFO:journalctl@ceph.mds.h.smithi174.stdout:Sep 10 09:09:45 smithi174 bash[71220]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-h 2024-09-10T09:09:45.968 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.h.service' 2024-09-10T09:09:46.000 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:09:46.001 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-10T09:09:46.001 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-10T09:09:46.001 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.k 2024-09-10T09:09:46.329 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 09:09:46 smithi174 systemd[1]: Stopping Ceph mds.k for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:09:46.742 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 09:09:46 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-k[62978]: 2024-09-10T09:09:46.327+0000 7fc8f673c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:09:46.742 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 09:09:46 smithi174 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-k[62978]: 2024-09-10T09:09:46.327+0000 7fc8f673c640 -1 mds.k *** got signal Terminated *** 2024-09-10T09:09:49.413 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 09:09:49 smithi174 podman[71337]: 2024-09-10 09:09:49.126128796 +0000 UTC m=+2.906340883 container died 7e751f71648b913eb23b4c5ec898a63c3d0c7e7dd3f589c31d01944a655f93ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-k, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:09:49.742 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 09:09:49 smithi174 podman[71337]: 2024-09-10 09:09:49.512006792 +0000 UTC m=+3.292218835 container cleanup 7e751f71648b913eb23b4c5ec898a63c3d0c7e7dd3f589c31d01944a655f93ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True) 2024-09-10T09:09:49.742 INFO:journalctl@ceph.mds.k.smithi174.stdout:Sep 10 09:09:49 smithi174 bash[71337]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-k 2024-09-10T09:09:49.877 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.k.service' 2024-09-10T09:09:49.908 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:09:49.908 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-10T09:09:49.908 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-10T09:09:49.909 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.c 2024-09-10T09:09:50.229 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 09:09:49 smithi191 systemd[1]: Stopping Ceph mds.c for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:09:50.550 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 09:09:50 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-c[64990]: 2024-09-10T09:09:50.228+0000 7f1605d3b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:09:50.550 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 09:09:50 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-c[64990]: 2024-09-10T09:09:50.228+0000 7f1605d3b640 -1 mds.c *** got signal Terminated *** 2024-09-10T09:09:53.405 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 09:09:53 smithi191 podman[73196]: 2024-09-10 09:09:53.100159893 +0000 UTC m=+2.993335276 container died b8308ec924a8a0584993412fc3c47e807510c8717b37cc2a2c3f02c546439540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:09:53.721 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 09:09:53 smithi191 podman[73196]: 2024-09-10 09:09:53.503253149 +0000 UTC m=+3.396428539 container cleanup b8308ec924a8a0584993412fc3c47e807510c8717b37cc2a2c3f02c546439540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-09-10T09:09:53.721 INFO:journalctl@ceph.mds.c.smithi191.stdout:Sep 10 09:09:53 smithi191 bash[73196]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-c 2024-09-10T09:09:53.863 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.c.service' 2024-09-10T09:09:53.899 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:09:53.899 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-10T09:09:53.899 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-10T09:09:53.899 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.f 2024-09-10T09:09:54.300 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 09:09:53 smithi191 systemd[1]: Stopping Ceph mds.f for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:09:54.300 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 09:09:54 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-f[64693]: 2024-09-10T09:09:54.223+0000 7f07085a3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:09:54.300 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 09:09:54 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-f[64693]: 2024-09-10T09:09:54.223+0000 7f07085a3640 -1 mds.f *** got signal Terminated *** 2024-09-10T09:09:57.391 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 09:09:57 smithi191 podman[73330]: 2024-09-10 09:09:57.101197271 +0000 UTC m=+2.981673187 container died 465ebd26bf991023b18ef3f5537d5365ab637e831fb4c92113cdeff23df94bff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-f, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T09:09:57.741 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 09:09:57 smithi191 podman[73330]: 2024-09-10 09:09:57.490225158 +0000 UTC m=+3.370701062 container cleanup 465ebd26bf991023b18ef3f5537d5365ab637e831fb4c92113cdeff23df94bff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:09:57.742 INFO:journalctl@ceph.mds.f.smithi191.stdout:Sep 10 09:09:57 smithi191 bash[73330]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-f 2024-09-10T09:09:57.877 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.f.service' 2024-09-10T09:09:57.911 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:09:57.911 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-10T09:09:57.911 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-10T09:09:57.911 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.i 2024-09-10T09:09:58.050 INFO:journalctl@ceph.mds.i.smithi191.stdout:Sep 10 09:09:57 smithi191 systemd[1]: Stopping Ceph mds.i for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:09:58.550 INFO:journalctl@ceph.mds.i.smithi191.stdout:Sep 10 09:09:58 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-i[65292]: 2024-09-10T09:09:58.225+0000 7f133d823640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:09:58.551 INFO:journalctl@ceph.mds.i.smithi191.stdout:Sep 10 09:09:58 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-i[65292]: 2024-09-10T09:09:58.225+0000 7f133d823640 -1 mds.i *** got signal Terminated *** 2024-09-10T09:10:01.300 INFO:journalctl@ceph.mds.i.smithi191.stdout:Sep 10 09:10:01 smithi191 podman[73452]: 2024-09-10 09:10:01.110793427 +0000 UTC m=+2.986025959 container died 68bd573d32e4e5d1a0f15c4da426faacf9fc3292a89dbdfb495e3cdc20892ba6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-i, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True) 2024-09-10T09:10:01.745 INFO:journalctl@ceph.mds.i.smithi191.stdout:Sep 10 09:10:01 smithi191 podman[73452]: 2024-09-10 09:10:01.460442594 +0000 UTC m=+3.335675125 container cleanup 68bd573d32e4e5d1a0f15c4da426faacf9fc3292a89dbdfb495e3cdc20892ba6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T09:10:01.746 INFO:journalctl@ceph.mds.i.smithi191.stdout:Sep 10 09:10:01 smithi191 bash[73452]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-i 2024-09-10T09:10:01.884 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.i.service' 2024-09-10T09:10:01.918 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:10:01.918 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-10T09:10:01.918 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-10T09:10:01.918 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.l 2024-09-10T09:10:02.300 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 09:10:01 smithi191 systemd[1]: Stopping Ceph mds.l for a7dba522-6f4f-11ef-bcea-c7b262605968... 2024-09-10T09:10:02.300 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 09:10:02 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-l[64389]: 2024-09-10T09:10:02.238+0000 7fd6243b3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:10:02.301 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 09:10:02 smithi191 ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-l[64389]: 2024-09-10T09:10:02.238+0000 7fd6243b3640 -1 mds.l *** got signal Terminated *** 2024-09-10T09:10:05.533 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 09:10:05 smithi191 podman[73574]: 2024-09-10 09:10:05.119183999 +0000 UTC m=+2.985109266 container died b309f6acf4fc96b10f84c9a7d7a3c98a2f0d7d2d38ecb510321164a17384d4a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T09:10:05.800 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 09:10:05 smithi191 podman[73574]: 2024-09-10 09:10:05.614338541 +0000 UTC m=+3.480263807 container cleanup b309f6acf4fc96b10f84c9a7d7a3c98a2f0d7d2d38ecb510321164a17384d4a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-l, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T09:10:05.801 INFO:journalctl@ceph.mds.l.smithi191.stdout:Sep 10 09:10:05 smithi191 bash[73574]: ceph-a7dba522-6f4f-11ef-bcea-c7b262605968-mds-l 2024-09-10T09:10:06.014 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7dba522-6f4f-11ef-bcea-c7b262605968@mds.l.service' 2024-09-10T09:10:06.045 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T09:10:06.046 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-10T09:10:06.046 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T09:10:06.252 INFO:teuthology.orchestra.run.smithi172.stdout:Deleting cluster with fsid: a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T09:10:08.760 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T09:10:08.965 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T09:10:11.510 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T09:10:11.711 INFO:teuthology.orchestra.run.smithi191.stdout:Deleting cluster with fsid: a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T09:10:14.105 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T09:10:14.140 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T09:10:14.174 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T09:10:14.201 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T09:10:14.202 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898925/remote/smithi172/crash 2024-09-10T09:10:14.203 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/crash -- . 2024-09-10T09:10:14.282 INFO:teuthology.orchestra.run.smithi172.stderr:tar: /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T09:10:14.282 INFO:teuthology.orchestra.run.smithi172.stderr:tar: Error is not recoverable: exiting now 2024-09-10T09:10:14.284 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898925/remote/smithi174/crash 2024-09-10T09:10:14.285 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/crash -- . 2024-09-10T09:10:14.336 INFO:teuthology.orchestra.run.smithi174.stderr:tar: /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T09:10:14.336 INFO:teuthology.orchestra.run.smithi174.stderr:tar: Error is not recoverable: exiting now 2024-09-10T09:10:14.338 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898925/remote/smithi191/crash 2024-09-10T09:10:14.339 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/crash -- . 2024-09-10T09:10:14.376 INFO:teuthology.orchestra.run.smithi191.stderr:tar: /var/lib/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T09:10:14.376 INFO:teuthology.orchestra.run.smithi191.stderr:tar: Error is not recoverable: exiting now 2024-09-10T09:10:14.377 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T09:10:14.378 DEBUG:teuthology.orchestra.run.smithi172:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T09:10:14.482 INFO:tasks.cephadm:Compressing logs... 2024-09-10T09:10:14.482 DEBUG:teuthology.orchestra.run.smithi172:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T09:10:14.485 DEBUG:teuthology.orchestra.run.smithi174:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T09:10:14.487 DEBUG:teuthology.orchestra.run.smithi191:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T09:10:14.516 INFO:teuthology.orchestra.run.smithi172.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T09:10:14.517 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-10T09:10:14.518 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T09:10:14.518 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-10T09:10:14.518 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T09:10:14.518 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T09:10:14.529 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-10T09:10:14.529 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T09:10:14.529 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T09:10:14.530 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-10T09:10:14.530 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T09:10:14.530 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.log 2024-09-10T09:10:14.530 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-10T09:10:14.530 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T09:10:14.530 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-10T09:10:14.530 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T09:10:14.530 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-10T09:10:14.530 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T09:10:14.530 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 94.7% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-10T09:10:14.531 INFO:teuthology.orchestra.run.smithi172.stderr: 94.7% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-10T09:10:14.532 INFO:teuthology.orchestra.run.smithi172.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-10T09:10:14.532 INFO:teuthology.orchestra.run.smithi191.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T09:10:14.533 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T09:10:14.533 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T09:10:14.533 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-10T09:10:14.533 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T09:10:14.533 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.audit.log: /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T09:10:14.533 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-10T09:10:14.533 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T09:10:14.533 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T09:10:14.533 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.log 2024-09-10T09:10:14.533 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-10T09:10:14.533 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-10T09:10:14.534 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-10T09:10:14.534 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-10T09:10:14.534 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-10T09:10:14.534 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T09:10:14.536 INFO:teuthology.orchestra.run.smithi174.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T09:10:14.536 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-10T09:10:14.536 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T09:10:14.536 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T09:10:14.536 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T09:10:14.537 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-10T09:10:14.537 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.10.log: /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.1.log: /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.7.log: /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-10T09:10:14.537 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T09:10:14.537 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T09:10:14.537 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T09:10:14.537 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T09:10:14.537 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.log 2024-09-10T09:10:14.537 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-10T09:10:14.537 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T09:10:14.537 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-10T09:10:14.538 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T09:10:14.617 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 83.9% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T09:10:14.634 INFO:teuthology.orchestra.run.smithi174.stderr: 90.5% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T09:10:14.637 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.c.log: /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 93.4% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-10T09:10:14.649 INFO:teuthology.orchestra.run.smithi172.stderr: 90.3% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T09:10:14.659 INFO:teuthology.orchestra.run.smithi174.stderr: 93.3% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T09:10:14.702 INFO:teuthology.orchestra.run.smithi174.stderr: 87.4% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T09:10:14.707 INFO:teuthology.orchestra.run.smithi191.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T09:10:14.709 INFO:teuthology.orchestra.run.smithi191.stderr: 83.8% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T09:10:14.718 INFO:teuthology.orchestra.run.smithi191.stderr: 87.4% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T09:10:14.722 INFO:teuthology.orchestra.run.smithi172.stderr: 90.9% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T09:10:14.755 INFO:teuthology.orchestra.run.smithi172.stderr: 94.2% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-10T09:10:14.776 INFO:teuthology.orchestra.run.smithi174.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T09:10:14.799 INFO:teuthology.orchestra.run.smithi191.stderr: 91.0% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T09:10:14.814 INFO:teuthology.orchestra.run.smithi191.stderr: 90.5% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T09:10:14.931 INFO:teuthology.orchestra.run.smithi172.stderr: 90.2% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T09:10:15.045 INFO:teuthology.orchestra.run.smithi174.stderr: 91.0% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T09:10:15.053 INFO:teuthology.orchestra.run.smithi172.stderr: 83.3% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T09:10:15.053 INFO:teuthology.orchestra.run.smithi172.stderr: 87.4% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T09:10:15.094 INFO:teuthology.orchestra.run.smithi172.stderr: 94.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T09:10:15.633 INFO:teuthology.orchestra.run.smithi172.stderr: 93.9% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-10T09:10:15.679 INFO:teuthology.orchestra.run.smithi191.stderr: 93.6% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-10T09:10:16.749 INFO:teuthology.orchestra.run.smithi174.stderr: 91.7% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T09:10:16.904 INFO:teuthology.orchestra.run.smithi191.stderr: 93.9% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-10T09:10:17.057 INFO:teuthology.orchestra.run.smithi191.stderr: 94.5% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-10T09:10:17.381 INFO:teuthology.orchestra.run.smithi191.stderr: 91.6% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T09:10:18.135 INFO:teuthology.orchestra.run.smithi174.stderr: 94.4% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-10T09:10:20.139 INFO:teuthology.orchestra.run.smithi174.stderr: 95.0% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-10T09:10:22.842 INFO:teuthology.orchestra.run.smithi191.stderr: 93.7% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-10T09:10:22.943 INFO:teuthology.orchestra.run.smithi174.stderr: 94.6% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-10T09:10:23.450 INFO:teuthology.orchestra.run.smithi174.stderr: 93.5% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-10T09:10:29.188 INFO:teuthology.orchestra.run.smithi172.stderr: 90.4% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T09:11:21.871 INFO:teuthology.orchestra.run.smithi191.stderr: 91.5% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-10T09:11:49.343 INFO:teuthology.orchestra.run.smithi174.stderr: 91.9% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-10T09:11:55.373 INFO:teuthology.orchestra.run.smithi172.stderr: 91.1% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T09:13:18.656 INFO:teuthology.orchestra.run.smithi174.stderr: 91.8% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T09:14:06.372 INFO:teuthology.orchestra.run.smithi172.stderr: 91.7% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T09:14:18.807 INFO:teuthology.orchestra.run.smithi191.stderr: 91.7% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-10T09:14:33.270 INFO:teuthology.orchestra.run.smithi174.stderr: 91.7% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T09:15:02.444 INFO:teuthology.orchestra.run.smithi174.stderr: 91.3% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T09:15:02.446 INFO:teuthology.orchestra.run.smithi174.stderr: 2024-09-10T09:15:02.446 INFO:teuthology.orchestra.run.smithi174.stderr:real 4m47.943s 2024-09-10T09:15:02.446 INFO:teuthology.orchestra.run.smithi174.stderr:user 5m16.605s 2024-09-10T09:15:02.446 INFO:teuthology.orchestra.run.smithi174.stderr:sys 0m11.600s 2024-09-10T09:15:35.676 INFO:teuthology.orchestra.run.smithi191.stderr: 91.4% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T09:16:18.647 INFO:teuthology.orchestra.run.smithi191.stderr: 91.6% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T09:16:18.649 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-09-10T09:16:18.649 INFO:teuthology.orchestra.run.smithi191.stderr:real 6m4.146s 2024-09-10T09:16:18.649 INFO:teuthology.orchestra.run.smithi191.stderr:user 5m29.127s 2024-09-10T09:16:18.649 INFO:teuthology.orchestra.run.smithi191.stderr:sys 0m12.135s 2024-09-10T09:20:15.306 INFO:teuthology.orchestra.run.smithi172.stderr: 91.6% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T09:22:49.001 INFO:teuthology.orchestra.run.smithi172.stderr: 91.5% -- replaced with /var/log/ceph/a7dba522-6f4f-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-10T09:23:45.626 INFO:teuthology.orchestra.run.smithi172.stderr: 96.6% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-10T09:23:45.628 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-09-10T09:23:45.629 INFO:teuthology.orchestra.run.smithi172.stderr:real 13m31.129s 2024-09-10T09:23:45.629 INFO:teuthology.orchestra.run.smithi172.stderr:user 8m21.642s 2024-09-10T09:23:45.629 INFO:teuthology.orchestra.run.smithi172.stderr:sys 0m22.731s 2024-09-10T09:23:45.630 INFO:tasks.cephadm:Archiving logs... 2024-09-10T09:23:45.631 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898925/remote/smithi172/log 2024-09-10T09:23:45.632 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T09:25:02.911 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898925/remote/smithi174/log 2024-09-10T09:25:03.465 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T09:25:59.213 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898925/remote/smithi191/log 2024-09-10T09:25:59.214 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T09:26:56.084 INFO:tasks.cephadm:Removing cluster... 2024-09-10T09:26:56.084 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 --force 2024-09-10T09:26:56.288 INFO:teuthology.orchestra.run.smithi172.stdout:Deleting cluster with fsid: a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T09:26:56.985 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 --force 2024-09-10T09:26:57.200 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T09:26:57.846 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a7dba522-6f4f-11ef-bcea-c7b262605968 --force 2024-09-10T09:26:58.051 INFO:teuthology.orchestra.run.smithi191.stdout:Deleting cluster with fsid: a7dba522-6f4f-11ef-bcea-c7b262605968 2024-09-10T09:26:58.717 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T09:26:58.718 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T09:26:58.737 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T09:26:58.754 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T09:26:58.771 INFO:tasks.cephadm:Teardown complete 2024-09-10T09:26:58.771 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T09:26:58.784 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T09:26:58.784 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T09:26:58.787 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T09:26:58.797 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T09:26:58.919 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T09:26:58.930 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T09:26:58.930 DEBUG:teuthology.orchestra.run.smithi172:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T09:26:58.933 DEBUG:teuthology.orchestra.run.smithi174:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T09:26:58.936 DEBUG:teuthology.orchestra.run.smithi191:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T09:26:58.947 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-09-10T09:26:58.948 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-09-10T09:26:58.950 INFO:teuthology.orchestra.run.smithi191.stderr:bash: line 1: ntpq: command not found 2024-09-10T09:26:58.953 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T09:26:58.953 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-09-10T09:26:58.954 INFO:teuthology.orchestra.run.smithi172.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 +335us[ +350us] +/- 45ms 2024-09-10T09:26:58.954 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 36 +1354us[+1369us] +/- 69ms 2024-09-10T09:26:58.954 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 -1874us[-1859us] +/- 68ms 2024-09-10T09:26:58.954 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:26:58.965 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T09:26:58.965 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-09-10T09:26:58.965 INFO:teuthology.orchestra.run.smithi174.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 +350us[ +305us] +/- 45ms 2024-09-10T09:26:58.965 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 +1386us[+1340us] +/- 69ms 2024-09-10T09:26:58.965 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -1859us[-1904us] +/- 68ms 2024-09-10T09:26:58.965 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:26:58.966 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T09:26:58.966 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-09-10T09:26:58.967 INFO:teuthology.orchestra.run.smithi191.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 +347us[ +331us] +/- 45ms 2024-09-10T09:26:58.967 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 +1372us[+1357us] +/- 69ms 2024-09-10T09:26:58.967 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 -1872us[-1887us] +/- 68ms 2024-09-10T09:26:58.967 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:26:58.967 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T09:26:58.978 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T09:26:58.978 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T09:26:58.989 DEBUG:teuthology.orchestra.run.smithi172:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T09:26:59.137 DEBUG:teuthology.orchestra.run.smithi174:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T09:26:59.272 DEBUG:teuthology.orchestra.run.smithi191:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T09:26:59.420 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T09:26:59.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:26:59.453 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T09:26:59.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:26:59.481 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T09:26:59.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:26:59.509 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T09:26:59.522 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T09:26:59.533 INFO:teuthology.task.internal:Duration was 3442.491376 seconds 2024-09-10T09:26:59.533 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T09:26:59.543 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T09:26:59.544 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T09:26:59.546 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T09:26:59.549 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T09:26:59.603 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T09:26:59.614 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T09:26:59.641 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T09:27:00.105 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T09:27:00.106 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi172.front.sepia.ceph.com 2024-09-10T09:27:00.106 DEBUG:teuthology.orchestra.run.smithi172:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T09:27:00.132 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi174.front.sepia.ceph.com 2024-09-10T09:27:00.132 DEBUG:teuthology.orchestra.run.smithi174:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T09:27:00.161 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi191.front.sepia.ceph.com 2024-09-10T09:27:00.161 DEBUG:teuthology.orchestra.run.smithi191:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T09:27:00.189 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T09:27:00.189 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T09:27:00.192 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T09:27:00.204 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T09:27:00.259 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T09:27:00.259 DEBUG:teuthology.orchestra.run.smithi172:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T09:27:00.262 DEBUG:teuthology.orchestra.run.smithi174:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T09:27:00.271 DEBUG:teuthology.orchestra.run.smithi191:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T09:27:01.878 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T09:27:01.890 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T09:27:01.891 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T09:27:01.925 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T09:27:01.956 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T09:27:01.987 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T09:27:01.999 DEBUG:teuthology.orchestra.run.smithi172:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T09:27:02.002 DEBUG:teuthology.orchestra.run.smithi174:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T09:27:02.004 DEBUG:teuthology.orchestra.run.smithi191:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T09:27:02.029 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = core 2024-09-10T09:27:02.048 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = core 2024-09-10T09:27:02.081 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = core 2024-09-10T09:27:02.107 DEBUG:teuthology.orchestra.run.smithi172:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T09:27:02.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:27:02.122 DEBUG:teuthology.orchestra.run.smithi174:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T09:27:02.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:27:02.138 DEBUG:teuthology.orchestra.run.smithi191:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T09:27:02.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:27:02.155 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T09:27:02.167 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T09:27:02.168 DEBUG:teuthology.misc:Transferring archived files from smithi172:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898925/remote/smithi172 2024-09-10T09:27:02.169 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T09:27:02.253 DEBUG:teuthology.misc:Transferring archived files from smithi174:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898925/remote/smithi174 2024-09-10T09:27:02.253 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T09:27:02.293 DEBUG:teuthology.misc:Transferring archived files from smithi191:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898925/remote/smithi191 2024-09-10T09:27:02.293 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T09:27:02.335 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T09:27:02.335 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T09:27:02.338 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T09:27:02.340 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T09:27:02.387 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T09:27:02.401 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T09:27:02.402 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T09:27:02.413 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T09:27:02.413 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T09:27:02.415 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T09:27:02.418 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T09:27:02.428 INFO:teuthology.orchestra.run.smithi172.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 09:27 /home/ubuntu/cephtest 2024-09-10T09:27:02.432 INFO:teuthology.orchestra.run.smithi174.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 09:27 /home/ubuntu/cephtest 2024-09-10T09:27:02.444 INFO:teuthology.orchestra.run.smithi191.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 09:27 /home/ubuntu/cephtest 2024-09-10T09:27:02.445 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T09:27:02.458 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T09:27:02.483 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} duration: 3442.4913761615753 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-10T09:27:02.483 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T09:27:02.570 INFO:teuthology.run:pass